2024-04-25T08:24:38.429 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T08:24:38.430 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992 --owner scheduled_neesingh@teuthology --name neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --verbose --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} -- /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992/orig.config.yaml 2024-04-25T08:24:38.475 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T08:24:38.578 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7672992' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: 5787d42d10024ba410e822a8aba474331cb9af02 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5869 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 5787d42d10024ba410e822a8aba474331cb9af02 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIYx+4NFbsXvRNQVeU3mlgj+I6GCNPAF8eodiBAugkhbrJfWcXhH8A57EDUE4AfANJckXLAVWUwHuFcoH7TBTDk= smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbgaLzu3yYtGUBP1iXTo1WsORaHNe1iKjeGse+xaRn5nBw3LS6RLZsuqA8SP4gt2Upkh5nfbMGbWrg4Y/lEdX0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_08:15:18 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T08:24:38.578 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa; will attempt to use it 2024-04-25T08:24:38.579 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks 2024-04-25T08:24:38.579 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T08:24:38.581 INFO:teuthology.task.internal:Checking packages... 2024-04-25T08:24:38.607 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T08:24:38.608 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T08:24:38.608 INFO:teuthology.packaging:ref: None 2024-04-25T08:24:38.608 INFO:teuthology.packaging:tag: None 2024-04-25T08:24:38.608 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T08:24:38.608 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:24:38.608 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T08:24:38.748 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628.g97ae6cc9 2024-04-25T08:24:38.751 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T08:24:38.758 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T08:24:38.758 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T08:24:38.764 INFO:teuthology.task.internal:Saving configuration 2024-04-25T08:24:38.790 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T08:24:38.796 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T08:24:38.822 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:19:36.104352', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIYx+4NFbsXvRNQVeU3mlgj+I6GCNPAF8eodiBAugkhbrJfWcXhH8A57EDUE4AfANJckXLAVWUwHuFcoH7TBTDk='} 2024-04-25T08:24:38.845 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:19:36.106337', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbgaLzu3yYtGUBP1iXTo1WsORaHNe1iKjeGse+xaRn5nBw3LS6RLZsuqA8SP4gt2Upkh5nfbMGbWrg4Y/lEdX0='} 2024-04-25T08:24:38.845 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T08:24:38.851 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T08:24:38.851 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T08:24:38.851 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T08:24:38.921 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb884f7d160>, signals=[15]) 2024-04-25T08:24:38.921 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T08:24:38.930 INFO:teuthology.task.internal:Opening connections... 2024-04-25T08:24:38.930 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:24:38.932 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:24:39.011 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:24:39.012 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:24:39.107 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T08:24:39.114 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-04-25T08:24:39.133 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-04-25T08:24:39.134 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-04-25T08:24:39.194 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-04-25T08:24:39.194 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9" 2024-04-25T08:24:39.194 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-04-25T08:24:39.194 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-04-25T08:24:39.194 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9" 2024-04-25T08:24:39.195 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9" 2024-04-25T08:24:39.195 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T08:24:39.195 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-04-25T08:24:39.195 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon" 2024-04-25T08:24:39.195 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T08:24:39.195 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-04-25T08:24:39.195 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T08:24:39.195 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T08:24:39.196 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T08:24:39.196 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-04-25T08:24:39.216 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-04-25T08:24:39.234 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-04-25T08:24:39.234 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-04-25T08:24:39.292 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-04-25T08:24:39.292 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-04-25T08:24:39.292 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-04-25T08:24:39.292 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-04-25T08:24:39.292 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-04-25T08:24:39.292 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-04-25T08:24:39.292 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T08:24:39.292 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-04-25T08:24:39.292 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-04-25T08:24:39.293 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T08:24:39.293 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-04-25T08:24:39.293 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T08:24:39.293 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T08:24:39.293 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T08:24:39.293 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-04-25T08:24:39.316 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T08:24:39.326 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T08:24:39.360 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T08:24:39.360 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:24:39.363 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:24:39.379 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T08:24:39.386 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T08:24:39.386 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:24:39.420 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:24:39.520 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T08:24:39.582 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T08:24:39.595 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T08:24:39.596 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T08:24:39.596 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:24:39.596 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:24:39.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:24:39.614 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-04-25T08:24:39.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:24:39.615 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-04-25T08:24:39.671 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-437.el9.x86_64 2024-04-25T08:24:39.672 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.14.0-437.el9.x86_64 2024-04-25T08:24:39.672 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum install -y kernel 2024-04-25T08:24:39.673 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-437.el9.x86_64 2024-04-25T08:24:39.673 INFO:teuthology.task.kernel:Running kernel on smithi038: 5.14.0-437.el9.x86_64 2024-04-25T08:24:39.673 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum install -y kernel 2024-04-25T08:24:42.322 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 8.5 MB/s | 8.0 MB 00:00 2024-04-25T08:24:43.147 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.0 MB 00:01 2024-04-25T08:24:45.356 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 19 MB 00:01 2024-04-25T08:24:47.315 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 6.2 MB/s | 19 MB 00:03 2024-04-25T08:24:49.998 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.2 MB 00:00 2024-04-25T08:24:51.415 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 16 kB 00:00 2024-04-25T08:24:51.485 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-25T08:24:52.949 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 16 kB 00:00 2024-04-25T08:24:54.109 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-25T08:24:59.335 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-04-25T08:25:01.291 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T08:25:01.291 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T08:25:01.373 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-04-25T08:25:01.374 INFO:teuthology.orchestra.run.smithi038.stdout:Nothing to do. 2024-04-25T08:25:01.374 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-04-25T08:25:01.509 DEBUG:teuthology.orchestra.run.smithi038:> echo no | sudo yum reinstall kernel || true 2024-04-25T08:25:02.026 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:24:59 AM UTC. 2024-04-25T08:25:02.150 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-04-25T08:25:02.163 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-04-25T08:25:02.163 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-04-25T08:25:02.163 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-04-25T08:25:02.163 INFO:teuthology.orchestra.run.smithi038.stdout:Reinstalling: 2024-04-25T08:25:02.163 INFO:teuthology.orchestra.run.smithi038.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T08:25:02.164 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:25:02.164 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-04-25T08:25:02.164 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-04-25T08:25:02.164 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:25:02.164 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 255 k 2024-04-25T08:25:02.164 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 0 2024-04-25T08:25:02.164 INFO:teuthology.orchestra.run.smithi038.stderr:Operation aborted. 2024-04-25T08:25:02.223 INFO:teuthology.orchestra.run.smithi038.stdout:Is this ok [y/N]: 2024-04-25T08:25:02.224 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum reinstall -y kernel || true 2024-04-25T08:25:02.743 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:24:59 AM UTC. 2024-04-25T08:25:02.864 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-04-25T08:25:02.864 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-04-25T08:25:02.865 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-04-25T08:25:02.865 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-04-25T08:25:02.865 INFO:teuthology.orchestra.run.smithi038.stdout:Reinstalling: 2024-04-25T08:25:02.865 INFO:teuthology.orchestra.run.smithi038.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T08:25:02.865 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:25:02.865 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-04-25T08:25:02.865 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-04-25T08:25:02.865 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:25:02.865 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 255 k 2024-04-25T08:25:02.865 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 0 2024-04-25T08:25:02.865 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-04-25T08:25:03.478 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-437.el9.x86_64.rpm 809 kB/s | 255 kB 00:00 2024-04-25T08:25:03.479 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-04-25T08:25:03.479 INFO:teuthology.orchestra.run.smithi038.stdout:Total 415 kB/s | 255 kB 00:00 2024-04-25T08:25:03.499 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-04-25T08:25:03.675 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-04-25T08:25:03.675 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-04-25T08:25:03.680 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-04-25T08:25:03.680 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-04-25T08:25:03.804 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-04-25T08:25:03.896 INFO:teuthology.orchestra.run.smithi038.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T08:25:03.897 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:25:04.540 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:25:04.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T08:25:05.259 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:25:05.259 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:25:05.259 INFO:teuthology.orchestra.run.smithi038.stdout:Reinstalled: 2024-04-25T08:25:05.259 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T08:25:05.259 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:25:05.259 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-04-25T08:25:05.444 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T08:25:05.492 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T08:25:05.493 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T08:25:05.493 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T08:25:05.493 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T08:25:05.493 DEBUG:teuthology.orchestra.run.smithi038:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T08:25:05.522 INFO:teuthology.orchestra.run.smithi038.stdout:ttyS1 2024-04-25T08:25:05.545 DEBUG:teuthology.parallel:result is None 2024-04-25T08:25:10.750 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-25T08:25:15.929 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 9.7 kB/s | 1.7 kB 00:00 2024-04-25T08:25:17.802 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T08:25:17.803 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T08:25:17.882 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-25T08:25:17.883 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-04-25T08:25:17.886 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-25T08:25:17.996 DEBUG:teuthology.orchestra.run.smithi107:> echo no | sudo yum reinstall kernel || true 2024-04-25T08:25:18.497 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:25:15 AM UTC. 2024-04-25T08:25:18.620 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-25T08:25:18.627 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-25T08:25:18.627 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-04-25T08:25:18.627 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-25T08:25:18.627 INFO:teuthology.orchestra.run.smithi107.stdout:Reinstalling: 2024-04-25T08:25:18.627 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T08:25:18.627 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:25:18.627 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-25T08:25:18.627 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-25T08:25:18.627 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:25:18.627 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 255 k 2024-04-25T08:25:18.627 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 0 2024-04-25T08:25:18.627 INFO:teuthology.orchestra.run.smithi107.stderr:Operation aborted. 2024-04-25T08:25:18.690 INFO:teuthology.orchestra.run.smithi107.stdout:Is this ok [y/N]: 2024-04-25T08:25:18.690 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum reinstall -y kernel || true 2024-04-25T08:25:19.182 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 08:25:15 AM UTC. 2024-04-25T08:25:19.297 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-25T08:25:19.298 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-25T08:25:19.298 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-04-25T08:25:19.298 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-25T08:25:19.298 INFO:teuthology.orchestra.run.smithi107.stdout:Reinstalling: 2024-04-25T08:25:19.298 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T08:25:19.298 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:25:19.298 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-25T08:25:19.298 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-04-25T08:25:19.298 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:25:19.298 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 255 k 2024-04-25T08:25:19.298 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 0 2024-04-25T08:25:19.298 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-25T08:25:19.698 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.0 MB/s | 255 kB 00:00 2024-04-25T08:25:19.698 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-25T08:25:19.698 INFO:teuthology.orchestra.run.smithi107.stdout:Total 637 kB/s | 255 kB 00:00 2024-04-25T08:25:19.718 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-25T08:25:19.917 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-25T08:25:19.917 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-25T08:25:19.922 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-25T08:25:19.922 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-25T08:25:20.061 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-25T08:25:20.132 INFO:teuthology.orchestra.run.smithi107.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T08:25:20.132 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:25:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:25:20.750 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T08:25:21.455 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:25:21.455 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:25:21.456 INFO:teuthology.orchestra.run.smithi107.stdout:Reinstalled: 2024-04-25T08:25:21.456 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T08:25:21.456 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:25:21.456 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-25T08:25:21.635 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T08:25:21.681 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T08:25:21.682 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T08:25:21.682 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T08:25:21.682 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T08:25:21.682 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T08:25:21.712 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-04-25T08:25:21.732 DEBUG:teuthology.parallel:result is None 2024-04-25T08:25:21.732 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T08:25:21.742 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T08:25:21.743 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:25:21.745 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:25:21.787 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T08:25:21.796 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T08:25:21.857 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T08:25:21.857 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:25:21.860 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:25:21.887 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T08:25:21.898 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T08:25:21.898 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:25:21.923 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:25:21.957 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:25:21.962 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:25:21.966 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:25:21.975 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:25:21.976 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T08:25:21.984 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T08:25:21.985 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:25:22.018 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:25:22.043 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T08:25:22.087 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T08:25:22.087 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:25:22.090 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:25:22.104 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:25:22.187 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:25:22.236 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:25:22.320 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:25:22.362 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:25:22.363 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:25:22.429 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:25:22.468 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:25:22.523 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:25:22.603 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:25:22.643 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:25:22.643 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:25:22.707 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-04-25T08:25:22.710 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-04-25T08:25:22.754 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T08:25:22.792 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T08:25:23.187 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T08:25:23.196 INFO:teuthology.task.internal:Starting timer... 2024-04-25T08:25:23.196 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T08:25:23.218 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T08:25:23.233 DEBUG:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2024-04-25T08:25:23.286 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: 2024-04-25T08:25:23.288 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-04-25T08:25:23.345 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-04-25T08:25:23.347 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T08:25:23.347 DEBUG:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2024-04-25T08:25:23.369 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2024-04-25T08:25:23.369 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-04-25T08:25:23.391 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-04-25T08:25:23.391 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi038.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi107.front.sepia.ceph.com': 'permissive'} 2024-04-25T08:25:23.391 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T08:25:23.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:25:23.437 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T08:25:23.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:25:23.460 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T08:25:23.461 DEBUG:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2024-04-25T08:25:23.513 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-04-25T08:25:23.537 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T08:25:23.547 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T08:25:23.552 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T08:25:23.689 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T08:25:23.704 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T08:25:23.705 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi038.front.sepia.ceph.com,smithi107.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T08:28:53.163 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi038.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com')] 2024-04-25T08:28:53.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-04-25T08:28:53.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:28:53.245 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-04-25T08:28:53.334 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-04-25T08:28:53.334 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-25T08:28:53.335 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:28:53.408 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-04-25T08:28:53.486 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-04-25T08:28:53.486 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T08:28:53.499 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T08:28:53.499 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T08:28:53.500 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:28:53.502 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T08:28:53.502 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:28:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T08:28:53.543 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T08:28:53.555 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T08:28:53.569 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T08:28:53.576 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2024-04-25T08:28:53.587 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2024-04-25T08:28:53.592 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-04-25T08:28:53.601 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T08:28:53.602 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-04-25T08:28:53.615 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T08:28:53.616 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T08:28:53.629 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T08:28:53.678 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-04-25T08:28:53.681 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T08:28:53.682 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-04-25T08:28:53.682 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:28:53.682 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:28:53.682 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:28:53.682 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:28:53.692 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-04-25T08:28:53.695 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T08:28:53.695 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-04-25T08:28:53.695 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:28:53.695 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:28:53.695 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:28:53.695 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:28:53.696 INFO:teuthology.run_tasks:Running task install... 2024-04-25T08:28:53.704 DEBUG:teuthology.task.install:project ceph 2024-04-25T08:28:53.705 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T08:28:53.705 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T08:28:53.705 INFO:teuthology.task.install:Using flavor: default 2024-04-25T08:28:53.711 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T08:28:53.711 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T08:28:53.711 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T08:28:53.712 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:28:53.714 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T08:28:53.715 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:28:53.864 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T08:28:53.864 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T08:28:53.873 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T08:28:53.873 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T08:28:53.985 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T08:28:53.985 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:28:53.985 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T08:28:53.988 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T08:28:53.988 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:28:53.988 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T08:28:54.011 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T08:28:54.011 DEBUG:teuthology.orchestra.run.smithi038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T08:28:54.013 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T08:28:54.014 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T08:28:54.076 DEBUG:teuthology.orchestra.run.smithi038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T08:28:54.078 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T08:28:54.150 DEBUG:teuthology.orchestra.run.smithi038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T08:28:54.151 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T08:28:54.213 INFO:teuthology.orchestra.run.smithi038.stdout:check_obsoletes = 1 2024-04-25T08:28:54.213 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-04-25T08:28:54.214 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum clean all 2024-04-25T08:28:54.215 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-04-25T08:28:54.439 INFO:teuthology.orchestra.run.smithi107.stdout:48 files removed 2024-04-25T08:28:54.449 INFO:teuthology.orchestra.run.smithi038.stdout:48 files removed 2024-04-25T08:28:54.466 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T08:28:54.476 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T08:28:55.044 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 254 kB/s | 82 kB 00:00 2024-04-25T08:28:55.079 INFO:teuthology.orchestra.run.smithi038.stdout:ceph packages for x86_64 249 kB/s | 82 kB 00:00 2024-04-25T08:28:55.378 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 35 kB/s | 10 kB 00:00 2024-04-25T08:28:55.454 INFO:teuthology.orchestra.run.smithi038.stdout:ceph noarch packages 30 kB/s | 10 kB 00:00 2024-04-25T08:28:55.704 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-04-25T08:28:55.796 INFO:teuthology.orchestra.run.smithi038.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-04-25T08:28:56.329 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-25T08:28:57.005 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.0 MB 00:01 2024-04-25T08:28:59.713 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-04-25T08:29:00.456 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 8.1 MB/s | 19 MB 00:02 2024-04-25T08:29:04.297 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 6.2 MB 00:01 2024-04-25T08:29:04.849 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-04-25T08:29:05.873 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-25T08:29:06.283 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 16 kB 00:00 2024-04-25T08:29:07.183 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-04-25T08:29:07.262 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-25T08:29:12.174 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-25T08:29:12.619 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-25T08:29:14.308 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T08:29:14.309 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T08:29:14.319 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T08:29:14.320 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T08:29:14.321 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T08:29:14.322 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T08:29:14.324 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T08:29:14.326 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T08:29:14.326 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T08:29:14.388 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-04-25T08:29:14.397 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-04-25T08:29:14.397 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-04-25T08:29:14.397 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-04-25T08:29:14.397 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-04-25T08:29:14.397 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T08:29:14.398 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T08:29:14.399 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T08:29:14.400 INFO:teuthology.orchestra.run.smithi107.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T08:29:14.401 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T08:29:14.402 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T08:29:14.403 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T08:29:14.403 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T08:29:14.403 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T08:29:14.403 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T08:29:14.403 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T08:29:14.403 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T08:29:14.403 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T08:29:14.403 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T08:29:14.403 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T08:29:14.403 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T08:29:14.403 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T08:29:14.403 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T08:29:14.404 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T08:29:14.405 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T08:29:14.406 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T08:29:14.407 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T08:29:14.408 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T08:29:14.409 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T08:29:14.409 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T08:29:14.409 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-04-25T08:29:14.409 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T08:29:14.409 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:29:14.409 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-04-25T08:29:14.409 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-04-25T08:29:14.409 INFO:teuthology.orchestra.run.smithi107.stdout:Install 148 Packages 2024-04-25T08:29:14.409 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 4 Packages 2024-04-25T08:29:14.409 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:29:14.410 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 215 M 2024-04-25T08:29:14.410 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-04-25T08:29:14.854 INFO:teuthology.orchestra.run.smithi038.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T08:29:14.855 INFO:teuthology.orchestra.run.smithi038.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T08:29:14.866 INFO:teuthology.orchestra.run.smithi038.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T08:29:14.867 INFO:teuthology.orchestra.run.smithi038.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T08:29:14.868 INFO:teuthology.orchestra.run.smithi038.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T08:29:14.869 INFO:teuthology.orchestra.run.smithi038.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T08:29:14.871 INFO:teuthology.orchestra.run.smithi038.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T08:29:14.873 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T08:29:14.874 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T08:29:14.938 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-04-25T08:29:14.948 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-04-25T08:29:14.948 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-04-25T08:29:14.948 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-04-25T08:29:14.948 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-04-25T08:29:14.948 INFO:teuthology.orchestra.run.smithi038.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T08:29:14.948 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T08:29:14.948 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T08:29:14.949 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T08:29:14.950 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T08:29:14.951 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T08:29:14.952 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T08:29:14.953 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T08:29:14.954 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T08:29:14.955 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T08:29:14.956 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T08:29:14.956 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T08:29:14.956 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T08:29:14.956 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T08:29:14.956 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T08:29:14.956 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T08:29:14.956 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T08:29:14.956 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T08:29:14.956 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T08:29:14.956 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T08:29:14.956 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T08:29:14.956 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T08:29:14.957 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T08:29:14.958 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout:Installing weak dependencies: 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout:======================================================================================= 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout:Install 148 Packages 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 4 Packages 2024-04-25T08:29:14.959 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:29:14.960 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 215 M 2024-04-25T08:29:14.960 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-04-25T08:29:15.241 INFO:teuthology.orchestra.run.smithi107.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-25T08:29:15.483 INFO:teuthology.orchestra.run.smithi107.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 3.6 MB/s | 887 kB 00:00 2024-04-25T08:29:15.542 INFO:teuthology.orchestra.run.smithi107.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 2.4 MB/s | 145 kB 00:00 2024-04-25T08:29:15.678 INFO:teuthology.orchestra.run.smithi038.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-04-25T08:29:15.875 INFO:teuthology.orchestra.run.smithi107.stdout:(4/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 6.7 MB/s | 5.3 MB 00:00 2024-04-25T08:29:15.917 INFO:teuthology.orchestra.run.smithi107.stdout:(5/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 5.7 MB/s | 2.1 MB 00:00 2024-04-25T08:29:16.067 INFO:teuthology.orchestra.run.smithi107.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 7.8 MB/s | 1.5 MB 00:00 2024-04-25T08:29:16.312 INFO:teuthology.orchestra.run.smithi038.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.4 MB/s | 887 kB 00:00 2024-04-25T08:29:16.487 INFO:teuthology.orchestra.run.smithi038.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 831 kB/s | 145 kB 00:00 2024-04-25T08:29:16.726 INFO:teuthology.orchestra.run.smithi107.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 5.8 MB/s | 4.7 MB 00:00 2024-04-25T08:29:17.409 INFO:teuthology.orchestra.run.smithi107.stdout:(8/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 8.5 MB/s | 20 MB 00:02 2024-04-25T08:29:17.476 INFO:teuthology.orchestra.run.smithi107.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 379 kB/s | 25 kB 00:00 2024-04-25T08:29:17.771 INFO:teuthology.orchestra.run.smithi038.stdout:(4/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 2.3 MB/s | 5.3 MB 00:02 2024-04-25T08:29:18.046 INFO:teuthology.orchestra.run.smithi038.stdout:(5/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 1.4 MB/s | 2.1 MB 00:01 2024-04-25T08:29:18.110 INFO:teuthology.orchestra.run.smithi107.stdout:(10/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 8.2 MB/s | 17 MB 00:02 2024-04-25T08:29:18.146 INFO:teuthology.orchestra.run.smithi038.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 4.0 MB/s | 1.5 MB 00:00 2024-04-25T08:29:18.193 INFO:teuthology.orchestra.run.smithi107.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 2.6 MB/s | 220 kB 00:00 2024-04-25T08:29:18.260 INFO:teuthology.orchestra.run.smithi107.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 407 kB/s | 27 kB 00:00 2024-04-25T08:29:18.327 INFO:teuthology.orchestra.run.smithi107.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 468 kB/s | 31 kB 00:00 2024-04-25T08:29:18.494 INFO:teuthology.orchestra.run.smithi107.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 4.4 MB/s | 744 kB 00:00 2024-04-25T08:29:18.577 INFO:teuthology.orchestra.run.smithi107.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 1.9 MB/s | 163 kB 00:00 2024-04-25T08:29:18.661 INFO:teuthology.orchestra.run.smithi107.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.5 MB/s | 127 kB 00:00 2024-04-25T08:29:18.781 INFO:teuthology.orchestra.run.smithi107.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 4.1 MB/s | 501 kB 00:00 2024-04-25T08:29:18.920 INFO:teuthology.orchestra.run.smithi107.stdout:(18/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 4.5 MB/s | 9.8 MB 00:02 2024-04-25T08:29:19.012 INFO:teuthology.orchestra.run.smithi107.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 493 kB/s | 45 kB 00:00 2024-04-25T08:29:19.104 INFO:teuthology.orchestra.run.smithi107.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.4 MB/s | 131 kB 00:00 2024-04-25T08:29:19.196 INFO:teuthology.orchestra.run.smithi107.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 1.7 MB/s | 162 kB 00:00 2024-04-25T08:29:19.296 INFO:teuthology.orchestra.run.smithi107.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 3.1 MB/s | 321 kB 00:00 2024-04-25T08:29:19.397 INFO:teuthology.orchestra.run.smithi107.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 2.9 MB/s | 298 kB 00:00 2024-04-25T08:29:19.526 INFO:teuthology.orchestra.run.smithi107.stdout:(24/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 6.9 MB/s | 5.1 MB 00:00 2024-04-25T08:29:19.555 INFO:teuthology.orchestra.run.smithi107.stdout:(25/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 626 kB/s | 99 kB 00:00 2024-04-25T08:29:19.597 INFO:teuthology.orchestra.run.smithi107.stdout:(26/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-25T08:29:19.681 INFO:teuthology.orchestra.run.smithi107.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 2.0 MB/s | 170 kB 00:00 2024-04-25T08:29:19.765 INFO:teuthology.orchestra.run.smithi107.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 377 kB/s | 31 kB 00:00 2024-04-25T08:29:19.848 INFO:teuthology.orchestra.run.smithi107.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.6 MB/s | 138 kB 00:00 2024-04-25T08:29:20.124 INFO:teuthology.orchestra.run.smithi107.stdout:(30/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 6.5 MB/s | 1.8 MB 00:00 2024-04-25T08:29:20.182 INFO:teuthology.orchestra.run.smithi107.stdout:(31/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 4.9 MB/s | 3.1 MB 00:00 2024-04-25T08:29:20.282 INFO:teuthology.orchestra.run.smithi107.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 2.4 MB/s | 248 kB 00:00 2024-04-25T08:29:20.374 INFO:teuthology.orchestra.run.smithi107.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 539 kB/s | 49 kB 00:00 2024-04-25T08:29:20.458 INFO:teuthology.orchestra.run.smithi107.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 184 kB/s | 15 kB 00:00 2024-04-25T08:29:20.534 INFO:teuthology.orchestra.run.smithi107.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 3.4 MB/s | 264 kB 00:00 2024-04-25T08:29:20.605 INFO:teuthology.orchestra.run.smithi038.stdout:(7/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 3.9 MB/s | 20 MB 00:05 2024-04-25T08:29:20.676 INFO:teuthology.orchestra.run.smithi107.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 5.2 MB/s | 748 kB 00:00 2024-04-25T08:29:20.759 INFO:teuthology.orchestra.run.smithi107.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 219 kB/s | 18 kB 00:00 2024-04-25T08:29:20.851 INFO:teuthology.orchestra.run.smithi107.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 10 MB/s | 7.4 MB 00:00 2024-04-25T08:29:20.968 INFO:teuthology.orchestra.run.smithi107.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 385 kB/s | 80 kB 00:00 2024-04-25T08:29:21.068 INFO:teuthology.orchestra.run.smithi107.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-25T08:29:21.185 INFO:teuthology.orchestra.run.smithi107.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-25T08:29:21.246 INFO:teuthology.orchestra.run.smithi038.stdout:(8/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 1.5 MB/s | 4.7 MB 00:03 2024-04-25T08:29:21.305 INFO:teuthology.orchestra.run.smithi038.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 432 kB/s | 25 kB 00:00 2024-04-25T08:29:21.369 INFO:teuthology.orchestra.run.smithi107.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-04-25T08:29:21.444 INFO:teuthology.orchestra.run.smithi107.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-25T08:29:21.478 INFO:teuthology.orchestra.run.smithi107.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 4.2 MB/s | 1.2 MB 00:00 2024-04-25T08:29:21.537 INFO:teuthology.orchestra.run.smithi107.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-04-25T08:29:21.555 INFO:teuthology.orchestra.run.smithi038.stdout:(10/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 4.9 MB/s | 17 MB 00:03 2024-04-25T08:29:21.562 INFO:teuthology.orchestra.run.smithi107.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-25T08:29:21.614 INFO:teuthology.orchestra.run.smithi038.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 3.7 MB/s | 220 kB 00:00 2024-04-25T08:29:21.629 INFO:teuthology.orchestra.run.smithi107.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-25T08:29:21.664 INFO:teuthology.orchestra.run.smithi038.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 543 kB/s | 27 kB 00:00 2024-04-25T08:29:21.723 INFO:teuthology.orchestra.run.smithi038.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 534 kB/s | 31 kB 00:00 2024-04-25T08:29:21.787 INFO:teuthology.orchestra.run.smithi107.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-04-25T08:29:21.846 INFO:teuthology.orchestra.run.smithi107.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 4.2 MB/s | 943 kB 00:00 2024-04-25T08:29:21.856 INFO:teuthology.orchestra.run.smithi038.stdout:(14/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 7.8 MB/s | 9.8 MB 00:01 2024-04-25T08:29:21.872 INFO:teuthology.orchestra.run.smithi107.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-04-25T08:29:21.896 INFO:teuthology.orchestra.run.smithi107.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-04-25T08:29:21.922 INFO:teuthology.orchestra.run.smithi107.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T08:29:21.923 INFO:teuthology.orchestra.run.smithi038.stdout:(15/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 3.6 MB/s | 744 kB 00:00 2024-04-25T08:29:21.947 INFO:teuthology.orchestra.run.smithi107.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-04-25T08:29:21.948 INFO:teuthology.orchestra.run.smithi038.stdout:(16/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 1.7 MB/s | 163 kB 00:00 2024-04-25T08:29:21.982 INFO:teuthology.orchestra.run.smithi038.stdout:(17/152): librados-devel-19.0.0-1628.g97ae6cc9. 2.1 MB/s | 127 kB 00:00 2024-04-25T08:29:21.989 INFO:teuthology.orchestra.run.smithi107.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-04-25T08:29:22.026 INFO:teuthology.orchestra.run.smithi038.stdout:(18/152): libradosstriper1-19.0.0-1628.g97ae6cc 6.3 MB/s | 501 kB 00:00 2024-04-25T08:29:22.031 INFO:teuthology.orchestra.run.smithi107.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-04-25T08:29:22.077 INFO:teuthology.orchestra.run.smithi038.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 907 kB/s | 45 kB 00:00 2024-04-25T08:29:22.098 INFO:teuthology.orchestra.run.smithi107.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-04-25T08:29:22.123 INFO:teuthology.orchestra.run.smithi107.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 160 kB 00:00 2024-04-25T08:29:22.127 INFO:teuthology.orchestra.run.smithi038.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 2.6 MB/s | 131 kB 00:00 2024-04-25T08:29:22.149 INFO:teuthology.orchestra.run.smithi107.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-04-25T08:29:22.182 INFO:teuthology.orchestra.run.smithi107.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T08:29:22.194 INFO:teuthology.orchestra.run.smithi038.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 2.4 MB/s | 162 kB 00:00 2024-04-25T08:29:22.216 INFO:teuthology.orchestra.run.smithi107.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-25T08:29:22.241 INFO:teuthology.orchestra.run.smithi107.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 434 kB/s | 25 kB 00:00 2024-04-25T08:29:22.261 INFO:teuthology.orchestra.run.smithi038.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 4.7 MB/s | 321 kB 00:00 2024-04-25T08:29:22.266 INFO:teuthology.orchestra.run.smithi107.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-25T08:29:22.292 INFO:teuthology.orchestra.run.smithi107.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T08:29:22.317 INFO:teuthology.orchestra.run.smithi107.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T08:29:22.320 INFO:teuthology.orchestra.run.smithi038.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 5.0 MB/s | 298 kB 00:00 2024-04-25T08:29:22.334 INFO:teuthology.orchestra.run.smithi107.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 840 kB/s | 35 kB 00:00 2024-04-25T08:29:22.387 INFO:teuthology.orchestra.run.smithi038.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.5 MB/s | 99 kB 00:00 2024-04-25T08:29:22.446 INFO:teuthology.orchestra.run.smithi038.stdout:(25/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-25T08:29:22.486 INFO:teuthology.orchestra.run.smithi107.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-04-25T08:29:22.663 INFO:teuthology.orchestra.run.smithi038.stdout:(26/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 7.5 MB/s | 5.1 MB 00:00 2024-04-25T08:29:22.721 INFO:teuthology.orchestra.run.smithi038.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 2.9 MB/s | 170 kB 00:00 2024-04-25T08:29:22.780 INFO:teuthology.orchestra.run.smithi038.stdout:(28/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 9.2 MB/s | 3.1 MB 00:00 2024-04-25T08:29:22.805 INFO:teuthology.orchestra.run.smithi038.stdout:(29/152): ceph-grafana-dashboards-19.0.0-1628.g 376 kB/s | 31 kB 00:00 2024-04-25T08:29:22.830 INFO:teuthology.orchestra.run.smithi038.stdout:(30/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 2.7 MB/s | 138 kB 00:00 2024-04-25T08:29:22.918 INFO:teuthology.orchestra.run.smithi107.stdout:(67/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 8.7 MB/s | 47 MB 00:05 2024-04-25T08:29:22.960 INFO:teuthology.orchestra.run.smithi107.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 3.4 MB/s | 1.6 MB 00:00 2024-04-25T08:29:23.014 INFO:teuthology.orchestra.run.smithi038.stdout:(31/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 8.5 MB/s | 1.8 MB 00:00 2024-04-25T08:29:23.027 INFO:teuthology.orchestra.run.smithi107.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-04-25T08:29:23.052 INFO:teuthology.orchestra.run.smithi107.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-25T08:29:23.073 INFO:teuthology.orchestra.run.smithi038.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 4.2 MB/s | 248 kB 00:00 2024-04-25T08:29:23.119 INFO:teuthology.orchestra.run.smithi107.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T08:29:23.123 INFO:teuthology.orchestra.run.smithi038.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 991 kB/s | 49 kB 00:00 2024-04-25T08:29:23.181 INFO:teuthology.orchestra.run.smithi038.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 263 kB/s | 15 kB 00:00 2024-04-25T08:29:23.186 INFO:teuthology.orchestra.run.smithi107.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-25T08:29:23.228 INFO:teuthology.orchestra.run.smithi107.stdout:(73/152): postgresql-contrib-13.14-1.el9.x86_64 2.9 MB/s | 911 kB 00:00 2024-04-25T08:29:23.257 INFO:teuthology.orchestra.run.smithi038.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 3.4 MB/s | 264 kB 00:00 2024-04-25T08:29:23.303 INFO:teuthology.orchestra.run.smithi107.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-04-25T08:29:23.328 INFO:teuthology.orchestra.run.smithi107.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-04-25T08:29:23.353 INFO:teuthology.orchestra.run.smithi107.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-04-25T08:29:23.365 INFO:teuthology.orchestra.run.smithi038.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 6.7 MB/s | 748 kB 00:00 2024-04-25T08:29:23.379 INFO:teuthology.orchestra.run.smithi107.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-04-25T08:29:23.404 INFO:teuthology.orchestra.run.smithi107.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-04-25T08:29:23.416 INFO:teuthology.orchestra.run.smithi038.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 365 kB/s | 18 kB 00:00 2024-04-25T08:29:23.429 INFO:teuthology.orchestra.run.smithi107.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-25T08:29:23.454 INFO:teuthology.orchestra.run.smithi107.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-04-25T08:29:23.505 INFO:teuthology.orchestra.run.smithi107.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-25T08:29:23.525 INFO:teuthology.orchestra.run.smithi038.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-04-25T08:29:23.572 INFO:teuthology.orchestra.run.smithi107.stdout:(82/152): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-25T08:29:23.597 INFO:teuthology.orchestra.run.smithi107.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-25T08:29:23.622 INFO:teuthology.orchestra.run.smithi107.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T08:29:23.648 INFO:teuthology.orchestra.run.smithi107.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-04-25T08:29:23.673 INFO:teuthology.orchestra.run.smithi107.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T08:29:23.706 INFO:teuthology.orchestra.run.smithi107.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-25T08:29:23.732 INFO:teuthology.orchestra.run.smithi107.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-04-25T08:29:23.750 INFO:teuthology.orchestra.run.smithi038.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 241 kB/s | 80 kB 00:00 2024-04-25T08:29:23.757 INFO:teuthology.orchestra.run.smithi107.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T08:29:23.782 INFO:teuthology.orchestra.run.smithi107.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T08:29:23.857 INFO:teuthology.orchestra.run.smithi107.stdout:(91/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-25T08:29:23.883 INFO:teuthology.orchestra.run.smithi107.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-04-25T08:29:23.900 INFO:teuthology.orchestra.run.smithi038.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-25T08:29:23.908 INFO:teuthology.orchestra.run.smithi107.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-04-25T08:29:23.933 INFO:teuthology.orchestra.run.smithi107.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T08:29:23.959 INFO:teuthology.orchestra.run.smithi107.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-04-25T08:29:23.984 INFO:teuthology.orchestra.run.smithi107.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-25T08:29:24.009 INFO:teuthology.orchestra.run.smithi107.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-25T08:29:24.017 INFO:teuthology.orchestra.run.smithi038.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-25T08:29:24.043 INFO:teuthology.orchestra.run.smithi107.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 988 kB/s | 58 kB 00:00 2024-04-25T08:29:24.051 INFO:teuthology.orchestra.run.smithi038.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-04-25T08:29:24.068 INFO:teuthology.orchestra.run.smithi107.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T08:29:24.093 INFO:teuthology.orchestra.run.smithi107.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T08:29:24.109 INFO:teuthology.orchestra.run.smithi038.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T08:29:24.119 INFO:teuthology.orchestra.run.smithi107.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-04-25T08:29:24.218 INFO:teuthology.orchestra.run.smithi038.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-25T08:29:24.219 INFO:teuthology.orchestra.run.smithi107.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 177 kB/s | 22 kB 00:00 2024-04-25T08:29:24.318 INFO:teuthology.orchestra.run.smithi038.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-25T08:29:24.419 INFO:teuthology.orchestra.run.smithi038.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-25T08:29:24.428 INFO:teuthology.orchestra.run.smithi107.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-25T08:29:24.452 INFO:teuthology.orchestra.run.smithi038.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 2.8 MB/s | 1.2 MB 00:00 2024-04-25T08:29:24.461 INFO:teuthology.orchestra.run.smithi107.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-04-25T08:29:24.537 INFO:teuthology.orchestra.run.smithi107.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-04-25T08:29:24.620 INFO:teuthology.orchestra.run.smithi038.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-04-25T08:29:24.653 INFO:teuthology.orchestra.run.smithi107.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-04-25T08:29:24.670 INFO:teuthology.orchestra.run.smithi038.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-25T08:29:24.696 INFO:teuthology.orchestra.run.smithi107.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 625 kB/s | 26 kB 00:00 2024-04-25T08:29:24.712 INFO:teuthology.orchestra.run.smithi038.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-04-25T08:29:24.729 INFO:teuthology.orchestra.run.smithi038.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 366 kB/s | 21 kB 00:00 2024-04-25T08:29:24.738 INFO:teuthology.orchestra.run.smithi107.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-25T08:29:24.763 INFO:teuthology.orchestra.run.smithi107.stdout:(109/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-25T08:29:24.771 INFO:teuthology.orchestra.run.smithi038.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-04-25T08:29:24.796 INFO:teuthology.orchestra.run.smithi038.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-04-25T08:29:24.821 INFO:teuthology.orchestra.run.smithi107.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-25T08:29:24.846 INFO:teuthology.orchestra.run.smithi038.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-04-25T08:29:24.847 INFO:teuthology.orchestra.run.smithi107.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 619 kB/s | 67 kB 00:00 2024-04-25T08:29:24.888 INFO:teuthology.orchestra.run.smithi038.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-04-25T08:29:24.939 INFO:teuthology.orchestra.run.smithi038.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-04-25T08:29:24.964 INFO:teuthology.orchestra.run.smithi038.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-25T08:29:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-04-25T08:29:24.989 INFO:teuthology.orchestra.run.smithi038.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-04-25T08:29:25.014 INFO:teuthology.orchestra.run.smithi107.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-04-25T08:29:25.023 INFO:teuthology.orchestra.run.smithi038.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T08:29:25.056 INFO:teuthology.orchestra.run.smithi107.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 773 kB/s | 32 kB 00:00 2024-04-25T08:29:25.057 INFO:teuthology.orchestra.run.smithi038.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-25T08:29:25.082 INFO:teuthology.orchestra.run.smithi038.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-25T08:29:25.107 INFO:teuthology.orchestra.run.smithi038.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-25T08:29:25.133 INFO:teuthology.orchestra.run.smithi038.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T08:29:25.140 INFO:teuthology.orchestra.run.smithi107.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 180 kB/s | 15 kB 00:00 2024-04-25T08:29:25.158 INFO:teuthology.orchestra.run.smithi038.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T08:29:25.183 INFO:teuthology.orchestra.run.smithi038.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 703 kB/s | 35 kB 00:00 2024-04-25T08:29:25.190 INFO:teuthology.orchestra.run.smithi107.stdout:(116/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-25T08:29:25.215 INFO:teuthology.orchestra.run.smithi107.stdout:(117/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.1 MB/s | 838 kB 00:00 2024-04-25T08:29:25.257 INFO:teuthology.orchestra.run.smithi107.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-04-25T08:29:25.298 INFO:teuthology.orchestra.run.smithi107.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 852 kB/s | 35 kB 00:00 2024-04-25T08:29:25.325 INFO:teuthology.orchestra.run.smithi038.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-04-25T08:29:25.366 INFO:teuthology.orchestra.run.smithi107.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-04-25T08:29:25.391 INFO:teuthology.orchestra.run.smithi107.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-25T08:29:25.434 INFO:teuthology.orchestra.run.smithi038.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T08:29:25.450 INFO:teuthology.orchestra.run.smithi107.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-25T08:29:25.476 INFO:teuthology.orchestra.run.smithi038.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T08:29:25.491 INFO:teuthology.orchestra.run.smithi107.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-04-25T08:29:25.518 INFO:teuthology.orchestra.run.smithi038.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T08:29:25.543 INFO:teuthology.orchestra.run.smithi038.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.9 MB/s | 133 kB 00:00 2024-04-25T08:29:25.558 INFO:teuthology.orchestra.run.smithi107.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-04-25T08:29:25.601 INFO:teuthology.orchestra.run.smithi107.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 558 kB/s | 23 kB 00:00 2024-04-25T08:29:25.618 INFO:teuthology.orchestra.run.smithi038.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-04-25T08:29:25.642 INFO:teuthology.orchestra.run.smithi107.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 468 kB/s | 19 kB 00:00 2024-04-25T08:29:25.669 INFO:teuthology.orchestra.run.smithi038.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-25T08:29:25.676 INFO:teuthology.orchestra.run.smithi107.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 594 kB/s | 20 kB 00:00 2024-04-25T08:29:25.769 INFO:teuthology.orchestra.run.smithi038.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-04-25T08:29:25.820 INFO:teuthology.orchestra.run.smithi038.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-25T08:29:25.851 INFO:teuthology.orchestra.run.smithi107.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-04-25T08:29:25.870 INFO:teuthology.orchestra.run.smithi038.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-25T08:29:25.885 INFO:teuthology.orchestra.run.smithi107.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-04-25T08:29:25.927 INFO:teuthology.orchestra.run.smithi107.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-04-25T08:29:25.953 INFO:teuthology.orchestra.run.smithi038.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 6.0 MB 00:00 2024-04-25T08:29:25.970 INFO:teuthology.orchestra.run.smithi038.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 255 kB/s | 26 kB 00:00 2024-04-25T08:29:25.987 INFO:teuthology.orchestra.run.smithi038.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 571 kB/s | 19 kB 00:00 2024-04-25T08:29:25.988 INFO:teuthology.orchestra.run.smithi107.stdout:(131/152): python3-natsort-7.1.1-5.el9.noarch.r 956 kB/s | 58 kB 00:00 2024-04-25T08:29:26.013 INFO:teuthology.orchestra.run.smithi038.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T08:29:26.030 INFO:teuthology.orchestra.run.smithi038.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 827 kB/s | 35 kB 00:00 2024-04-25T08:29:26.046 INFO:teuthology.orchestra.run.smithi107.stdout:(132/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-04-25T08:29:26.080 INFO:teuthology.orchestra.run.smithi038.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-25T08:29:26.088 INFO:teuthology.orchestra.run.smithi107.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 394 kB/s | 16 kB 00:00 2024-04-25T08:29:26.138 INFO:teuthology.orchestra.run.smithi038.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-25T08:29:26.163 INFO:teuthology.orchestra.run.smithi107.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T08:29:26.222 INFO:teuthology.orchestra.run.smithi107.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 484 kB/s | 28 kB 00:00 2024-04-25T08:29:26.357 INFO:teuthology.orchestra.run.smithi107.stdout:(136/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.3 MB/s | 2.0 MB 00:00 2024-04-25T08:29:26.357 INFO:teuthology.orchestra.run.smithi107.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-04-25T08:29:26.357 INFO:teuthology.orchestra.run.smithi107.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-04-25T08:29:26.357 INFO:teuthology.orchestra.run.smithi107.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-25T08:29:26.357 INFO:teuthology.orchestra.run.smithi038.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-04-25T08:29:26.358 INFO:teuthology.orchestra.run.smithi038.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 659 kB/s | 77 kB 00:00 2024-04-25T08:29:26.358 INFO:teuthology.orchestra.run.smithi038.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-04-25T08:29:26.358 INFO:teuthology.orchestra.run.smithi038.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T08:29:26.358 INFO:teuthology.orchestra.run.smithi038.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T08:29:26.365 INFO:teuthology.orchestra.run.smithi038.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T08:29:26.382 INFO:teuthology.orchestra.run.smithi107.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-04-25T08:29:26.391 INFO:teuthology.orchestra.run.smithi038.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 872 kB/s | 51 kB 00:00 2024-04-25T08:29:26.408 INFO:teuthology.orchestra.run.smithi107.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T08:29:26.416 INFO:teuthology.orchestra.run.smithi038.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T08:29:26.449 INFO:teuthology.orchestra.run.smithi107.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-04-25T08:29:26.466 INFO:teuthology.orchestra.run.smithi038.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-04-25T08:29:26.474 INFO:teuthology.orchestra.run.smithi107.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-25T08:29:26.525 INFO:teuthology.orchestra.run.smithi107.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 401 kB/s | 20 kB 00:00 2024-04-25T08:29:26.525 INFO:teuthology.orchestra.run.smithi038.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 760 kB/s | 44 kB 00:00 2024-04-25T08:29:26.550 INFO:teuthology.orchestra.run.smithi107.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-04-25T08:29:26.584 INFO:teuthology.orchestra.run.smithi038.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-25T08:29:26.617 INFO:teuthology.orchestra.run.smithi107.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-25T08:29:26.634 INFO:teuthology.orchestra.run.smithi038.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-04-25T08:29:26.684 INFO:teuthology.orchestra.run.smithi038.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 953 kB/s | 47 kB 00:00 2024-04-25T08:29:26.760 INFO:teuthology.orchestra.run.smithi038.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T08:29:26.776 INFO:teuthology.orchestra.run.smithi107.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 7.0 MB/s | 1.6 MB 00:00 2024-04-25T08:29:26.834 INFO:teuthology.orchestra.run.smithi107.stdout:(148/152): libarrow-9.0.0-10.el9.x86_64.rpm 1.9 MB/s | 4.4 MB 00:02 2024-04-25T08:29:26.927 INFO:teuthology.orchestra.run.smithi038.stdout:(97/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-25T08:29:26.952 INFO:teuthology.orchestra.run.smithi038.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 301 kB/s | 58 kB 00:00 2024-04-25T08:29:26.977 INFO:teuthology.orchestra.run.smithi038.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T08:29:27.003 INFO:teuthology.orchestra.run.smithi038.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T08:29:27.028 INFO:teuthology.orchestra.run.smithi038.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-04-25T08:29:27.034 INFO:teuthology.orchestra.run.smithi107.stdout:(149/152): librados2-19.0.0-1628.g97ae6cc9.el9. 8.1 MB/s | 3.4 MB 00:00 2024-04-25T08:29:27.053 INFO:teuthology.orchestra.run.smithi038.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 443 kB/s | 22 kB 00:00 2024-04-25T08:29:27.095 INFO:teuthology.orchestra.run.smithi038.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-04-25T08:29:27.126 INFO:teuthology.orchestra.run.smithi107.stdout:(150/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 9.0 MB/s | 3.1 MB 00:00 2024-04-25T08:29:27.226 INFO:teuthology.orchestra.run.smithi107.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 1.3 MB 00:00 2024-04-25T08:29:27.554 INFO:teuthology.orchestra.run.smithi038.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 674 kB/s | 309 kB 00:00 2024-04-25T08:29:27.579 INFO:teuthology.orchestra.run.smithi038.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-25T08:29:27.679 INFO:teuthology.orchestra.run.smithi038.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-04-25T08:29:27.735 INFO:teuthology.orchestra.run.smithi107.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.4 MB/s | 2.4 MB 00:00 2024-04-25T08:29:27.738 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-04-25T08:29:27.738 INFO:teuthology.orchestra.run.smithi107.stdout:Total 16 MB/s | 215 MB 00:13 2024-04-25T08:29:27.796 INFO:teuthology.orchestra.run.smithi038.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-04-25T08:29:27.905 INFO:teuthology.orchestra.run.smithi038.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2024-04-25T08:29:28.014 INFO:teuthology.orchestra.run.smithi038.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-04-25T08:29:28.139 INFO:teuthology.orchestra.run.smithi038.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-25T08:29:28.323 INFO:teuthology.orchestra.run.smithi038.stdout:(111/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.9 MB/s | 4.4 MB 00:00 2024-04-25T08:29:28.465 INFO:teuthology.orchestra.run.smithi038.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-04-25T08:29:28.490 INFO:teuthology.orchestra.run.smithi038.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-25T08:29:28.574 INFO:teuthology.orchestra.run.smithi038.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-04-25T08:29:28.599 INFO:teuthology.orchestra.run.smithi038.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-25T08:29:28.682 INFO:teuthology.orchestra.run.smithi038.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-25T08:29:28.708 INFO:teuthology.orchestra.run.smithi038.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-25T08:29:28.816 INFO:teuthology.orchestra.run.smithi038.stdout:(118/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-25T08:29:28.850 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-04-25T08:29:28.875 INFO:teuthology.orchestra.run.smithi038.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-04-25T08:29:28.908 INFO:teuthology.orchestra.run.smithi038.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 384 kB/s | 35 kB 00:00 2024-04-25T08:29:28.991 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-04-25T08:29:28.991 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-04-25T08:29:29.334 INFO:teuthology.orchestra.run.smithi038.stdout:(121/152): ceph-test-19.0.0-1628.g97ae6cc9.el9. 5.9 MB/s | 47 MB 00:08 2024-04-25T08:29:29.359 INFO:teuthology.orchestra.run.smithi038.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 425 kB/s | 206 kB 00:00 2024-04-25T08:29:29.410 INFO:teuthology.orchestra.run.smithi038.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.1 MB/s | 2.0 MB 00:00 2024-04-25T08:29:29.468 INFO:teuthology.orchestra.run.smithi038.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-04-25T08:29:29.519 INFO:teuthology.orchestra.run.smithi038.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-25T08:29:29.577 INFO:teuthology.orchestra.run.smithi038.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-04-25T08:29:29.619 INFO:teuthology.orchestra.run.smithi038.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-25T08:29:29.686 INFO:teuthology.orchestra.run.smithi038.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-25T08:29:29.761 INFO:teuthology.orchestra.run.smithi038.stdout:(129/152): python3-grpcio-tools-1.46.7-10.el9.x 338 kB/s | 144 kB 00:00 2024-04-25T08:29:29.795 INFO:teuthology.orchestra.run.smithi038.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-04-25T08:29:29.828 INFO:teuthology.orchestra.run.smithi038.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-04-25T08:29:29.854 INFO:teuthology.orchestra.run.smithi038.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 853 kB/s | 79 kB 00:00 2024-04-25T08:29:29.904 INFO:teuthology.orchestra.run.smithi038.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 531 kB/s | 58 kB 00:00 2024-04-25T08:29:29.946 INFO:teuthology.orchestra.run.smithi038.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-25T08:29:29.971 INFO:teuthology.orchestra.run.smithi038.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 140 kB/s | 16 kB 00:00 2024-04-25T08:29:30.039 INFO:teuthology.orchestra.run.smithi038.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 670 kB/s | 90 kB 00:00 2024-04-25T08:29:30.064 INFO:teuthology.orchestra.run.smithi038.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 241 kB/s | 28 kB 00:00 2024-04-25T08:29:30.089 INFO:teuthology.orchestra.run.smithi038.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 262 kB/s | 31 kB 00:00 2024-04-25T08:29:30.172 INFO:teuthology.orchestra.run.smithi038.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 544 kB/s | 59 kB 00:00 2024-04-25T08:29:30.198 INFO:teuthology.orchestra.run.smithi038.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-04-25T08:29:30.240 INFO:teuthology.orchestra.run.smithi038.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 938 kB/s | 188 kB 00:00 2024-04-25T08:29:30.282 INFO:teuthology.orchestra.run.smithi038.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 468 kB/s | 51 kB 00:00 2024-04-25T08:29:30.348 INFO:teuthology.orchestra.run.smithi038.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 825 kB/s | 90 kB 00:00 2024-04-25T08:29:30.374 INFO:teuthology.orchestra.run.smithi038.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-04-25T08:29:30.399 INFO:teuthology.orchestra.run.smithi038.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-25T08:29:30.458 INFO:teuthology.orchestra.run.smithi038.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 183 kB/s | 20 kB 00:00 2024-04-25T08:29:30.483 INFO:teuthology.orchestra.run.smithi038.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T08:29:30.675 INFO:teuthology.orchestra.run.smithi038.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.7 MB/s | 1.6 MB 00:00 2024-04-25T08:29:30.717 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-04-25T08:29:30.718 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-04-25T08:29:30.867 INFO:teuthology.orchestra.run.smithi038.stdout:(149/152): librados2-19.0.0-1628.g97ae6cc9.el9. 8.2 MB/s | 3.4 MB 00:00 2024-04-25T08:29:30.934 INFO:teuthology.orchestra.run.smithi038.stdout:(150/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 7.0 MB/s | 3.1 MB 00:00 2024-04-25T08:29:31.209 INFO:teuthology.orchestra.run.smithi038.stdout:(151/152): openssl-3.2.1-1.el9.x86_64.rpm 2.5 MB/s | 1.3 MB 00:00 2024-04-25T08:29:31.635 INFO:teuthology.orchestra.run.smithi038.stdout:(152/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 3.1 MB/s | 2.4 MB 00:00 2024-04-25T08:29:31.637 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-04-25T08:29:31.638 INFO:teuthology.orchestra.run.smithi038.stdout:Total 13 MB/s | 215 MB 00:16 2024-04-25T08:29:32.770 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-04-25T08:29:32.930 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-04-25T08:29:32.930 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-04-25T08:29:34.760 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-04-25T08:29:34.760 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-04-25T08:29:36.620 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-04-25T08:29:36.713 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T08:29:36.769 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T08:29:36.830 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T08:29:36.864 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T08:29:36.890 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T08:29:36.932 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T08:29:37.261 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T08:29:37.393 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:29:37.504 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:29:37.542 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:29:37.616 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:29:37.646 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T08:29:37.796 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T08:29:37.843 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T08:29:37.871 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:29:37.974 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:29:37.997 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T08:29:38.059 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T08:29:38.081 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T08:29:38.101 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T08:29:38.298 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T08:29:38.330 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T08:29:38.439 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T08:29:38.532 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T08:29:38.581 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T08:29:38.632 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T08:29:38.706 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T08:29:38.748 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T08:29:38.792 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T08:29:38.824 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T08:29:38.852 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T08:29:38.922 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T08:29:38.974 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T08:29:39.005 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T08:29:39.049 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T08:29:39.125 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T08:29:39.160 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T08:29:39.203 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T08:29:39.231 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T08:29:39.300 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T08:29:39.346 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T08:29:39.419 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T08:29:39.563 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T08:29:39.615 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T08:29:40.138 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T08:29:40.218 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T08:29:40.248 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T08:29:40.290 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T08:29:40.328 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T08:29:40.370 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T08:29:40.548 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T08:29:40.606 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T08:29:40.662 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T08:29:40.734 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T08:29:40.773 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T08:29:40.810 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T08:29:40.857 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T08:29:40.887 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-04-25T08:29:40.892 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T08:29:40.944 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T08:29:40.987 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T08:29:41.005 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T08:29:41.123 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T08:29:41.167 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T08:29:41.197 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T08:29:41.212 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T08:29:41.238 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T08:29:41.332 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T08:29:41.363 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T08:29:41.395 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T08:29:41.403 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T08:29:41.429 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T08:29:41.440 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T08:29:41.453 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T08:29:41.465 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T08:29:41.510 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T08:29:41.528 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T08:29:41.655 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T08:29:41.686 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T08:29:41.714 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T08:29:41.749 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T08:29:41.787 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T08:29:41.819 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T08:29:41.849 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T08:29:41.852 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T08:29:41.883 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T08:29:41.917 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T08:29:41.958 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:29:41.958 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T08:29:41.993 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T08:29:42.079 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T08:29:42.082 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:29:42.109 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:29:42.123 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T08:29:42.155 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T08:29:42.201 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T08:29:42.225 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:29:42.255 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T08:29:42.300 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T08:29:42.347 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T08:29:42.412 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T08:29:42.461 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T08:29:42.488 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:29:42.594 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:29:42.623 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T08:29:42.669 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T08:29:42.691 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T08:29:42.714 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T08:29:42.867 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T08:29:42.899 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T08:29:43.011 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T08:29:43.065 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T08:29:43.111 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T08:29:43.114 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T08:29:43.145 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T08:29:43.148 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T08:29:43.185 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T08:29:43.245 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T08:29:43.294 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T08:29:43.382 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T08:29:43.422 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T08:29:43.426 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T08:29:43.467 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T08:29:43.511 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T08:29:43.551 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T08:29:43.579 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T08:29:43.600 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T08:29:43.628 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T08:29:43.652 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T08:29:43.698 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T08:29:43.712 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T08:29:43.740 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T08:29:43.785 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T08:29:43.863 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T08:29:43.895 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T08:29:43.939 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T08:29:43.967 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T08:29:44.039 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T08:29:44.082 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T08:29:44.167 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T08:29:44.305 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T08:29:44.338 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T08:29:44.361 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T08:29:44.451 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T08:29:44.935 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T08:29:45.025 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T08:29:45.059 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T08:29:45.102 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T08:29:45.141 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T08:29:45.183 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T08:29:45.350 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T08:29:45.402 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T08:29:45.458 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T08:29:45.533 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T08:29:45.569 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T08:29:45.606 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T08:29:45.654 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T08:29:45.671 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T08:29:45.689 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T08:29:45.733 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T08:29:45.757 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T08:29:45.775 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T08:29:45.788 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T08:29:45.818 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T08:29:45.920 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T08:29:45.964 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T08:29:46.002 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T08:29:46.044 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T08:29:46.142 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T08:29:46.185 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T08:29:46.186 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T08:29:46.225 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T08:29:46.251 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T08:29:46.275 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T08:29:46.331 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T08:29:46.353 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T08:29:46.484 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T08:29:46.517 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T08:29:46.530 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T08:29:46.545 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T08:29:46.561 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T08:29:46.579 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T08:29:46.596 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T08:29:46.619 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T08:29:46.658 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T08:29:46.688 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T08:29:46.723 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T08:29:46.756 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T08:29:46.925 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T08:29:46.958 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T08:29:47.050 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T08:29:47.053 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T08:29:47.084 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T08:29:47.098 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T08:29:47.128 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T08:29:47.155 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T08:29:47.176 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T08:29:47.185 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T08:29:47.287 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T08:29:47.347 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T08:29:48.123 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T08:29:48.168 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T08:29:48.202 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T08:29:48.309 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T08:29:48.362 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T08:29:48.452 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T08:29:48.488 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T08:29:48.674 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T08:29:48.702 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T08:29:48.783 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T08:29:49.389 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:29:49.423 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:29:49.478 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T08:29:49.597 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T08:29:49.978 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:29:50.018 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T08:29:50.117 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T08:29:50.117 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T08:29:50.117 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T08:29:50.117 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:29:50.145 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T08:29:50.895 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T08:29:50.975 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T08:29:51.003 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T08:29:51.036 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T08:29:51.364 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T08:29:51.440 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T08:29:51.532 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T08:29:51.562 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T08:29:51.599 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T08:29:52.077 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T08:29:52.102 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T08:29:52.173 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T08:29:52.204 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T08:29:54.509 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:29:54.542 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:29:55.109 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:29:55.154 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T08:29:55.245 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T08:29:55.246 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T08:29:55.246 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T08:29:55.246 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:29:55.272 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T08:30:01.426 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T08:30:01.427 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-04-25T08:30:01.427 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-04-25T08:30:01.427 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-04-25T08:30:01.427 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-04-25T08:30:01.427 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-04-25T08:30:01.427 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-04-25T08:30:01.427 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-04-25T08:30:01.427 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:01.606 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T08:30:01.621 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T08:30:01.621 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:01.621 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T08:30:01.621 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:03.109 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T08:30:03.135 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T08:30:03.135 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:03.135 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T08:30:03.135 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T08:30:03.135 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T08:30:03.135 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:03.343 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T08:30:03.366 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T08:30:03.366 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:03.366 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T08:30:03.366 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T08:30:03.367 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T08:30:03.367 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:03.810 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T08:30:03.833 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T08:30:03.833 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:03.834 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T08:30:03.834 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T08:30:03.834 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T08:30:03.834 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:03.932 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T08:30:03.980 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T08:30:04.036 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T08:30:04.221 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T08:30:04.278 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T08:30:04.355 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T08:30:04.453 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T08:30:04.599 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T08:30:04.636 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T08:30:04.668 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T08:30:04.696 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T08:30:05.072 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T08:30:05.111 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T08:30:05.142 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T08:30:05.202 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T08:30:05.226 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T08:30:05.454 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T08:30:05.485 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T08:30:06.450 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T08:30:06.477 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T08:30:06.578 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T08:30:06.732 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T08:30:06.754 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T08:30:06.777 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T08:30:06.777 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:06.777 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T08:30:06.777 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T08:30:06.777 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T08:30:06.777 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:06.821 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T08:30:06.833 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T08:30:06.929 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T08:30:06.977 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T08:30:07.500 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T08:30:07.500 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2024-04-25T08:30:07.500 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2024-04-25T08:30:07.501 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2024-04-25T08:30:07.501 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2024-04-25T08:30:07.501 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2024-04-25T08:30:07.501 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /root 2024-04-25T08:30:07.501 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2024-04-25T08:30:07.501 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:07.671 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T08:30:07.687 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T08:30:07.687 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:07.688 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T08:30:07.688 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:07.836 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T08:30:07.865 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T08:30:07.888 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T08:30:07.888 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:07.888 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T08:30:07.888 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:30:07.889 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:30:07.889 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:07.933 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T08:30:07.955 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T08:30:07.955 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:07.955 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T08:30:07.955 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:08.000 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T08:30:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T08:30:08.023 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:08.023 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T08:30:08.023 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:30:08.023 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:30:08.023 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:08.326 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T08:30:08.348 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T08:30:08.349 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:08.349 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T08:30:08.349 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T08:30:08.349 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T08:30:08.349 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:09.216 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T08:30:09.244 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T08:30:09.244 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:09.244 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T08:30:09.244 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T08:30:09.244 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T08:30:09.244 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:09.458 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T08:30:09.482 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T08:30:09.482 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:09.482 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T08:30:09.483 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T08:30:09.483 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T08:30:09.483 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:09.985 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T08:30:10.009 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T08:30:10.009 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:10.009 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T08:30:10.009 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T08:30:10.009 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T08:30:10.009 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:10.116 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T08:30:10.174 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T08:30:10.238 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T08:30:10.444 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T08:30:10.503 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T08:30:10.585 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T08:30:10.674 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T08:30:10.849 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T08:30:10.887 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T08:30:10.919 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T08:30:10.946 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T08:30:11.394 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T08:30:11.420 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T08:30:11.452 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T08:30:11.517 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T08:30:11.543 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T08:30:11.786 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T08:30:11.819 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T08:30:12.770 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T08:30:12.811 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T08:30:12.830 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T08:30:12.853 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T08:30:12.869 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T08:30:12.884 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T08:30:12.912 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T08:30:12.977 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T08:30:13.094 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T08:30:13.140 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T08:30:13.141 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T08:30:13.171 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T08:30:13.171 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T08:30:13.196 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T08:30:13.196 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:13.196 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T08:30:13.196 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T08:30:13.196 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T08:30:13.196 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:13.247 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T08:30:13.259 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T08:30:13.360 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T08:30:13.451 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T08:30:13.802 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:30:13.844 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:30:13.863 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:30:14.349 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T08:30:14.386 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T08:30:14.408 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T08:30:14.409 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:14.409 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T08:30:14.409 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:30:14.409 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:30:14.409 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:14.459 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T08:30:14.483 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T08:30:14.483 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:14.483 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T08:30:14.483 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:14.527 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T08:30:14.550 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T08:30:14.550 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:14.550 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T08:30:14.550 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:30:14.551 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:30:14.551 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:14.693 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T08:30:14.782 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T08:30:14.823 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T08:30:14.869 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T08:30:14.893 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T08:30:14.893 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:30:14.893 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T08:30:14.893 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T08:30:14.893 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T08:30:14.893 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:14.904 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T08:30:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T08:30:14.936 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T08:30:15.029 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T08:30:15.061 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T08:30:15.061 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T08:30:15.147 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T08:30:15.218 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:30:19.556 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T08:30:19.598 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T08:30:19.639 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T08:30:19.670 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T08:30:19.698 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T08:30:19.883 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T08:30:19.928 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T08:30:19.957 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T08:30:20.383 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:30:20.383 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T08:30:20.383 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T08:30:20.383 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T08:30:20.383 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T08:30:20.383 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T08:30:20.383 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T08:30:20.384 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T08:30:20.385 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T08:30:20.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T08:30:20.387 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T08:30:20.388 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T08:30:20.389 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T08:30:20.390 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T08:30:20.391 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T08:30:20.392 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T08:30:20.393 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T08:30:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T08:30:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T08:30:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T08:30:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T08:30:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T08:30:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T08:30:20.394 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T08:30:20.725 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:30:20.764 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:30:20.785 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:30:21.681 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T08:30:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T08:30:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T08:30:21.887 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T08:30:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T08:30:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T08:30:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T08:30:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T08:30:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T08:30:22.136 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T08:30:22.209 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:30:22.487 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:30:22.487 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.488 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:22.489 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.490 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.490 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.490 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:22.490 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:22.490 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.490 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.490 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:22.490 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T08:30:22.491 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T08:30:22.491 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T08:30:22.491 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T08:30:22.491 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T08:30:22.491 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T08:30:22.491 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T08:30:22.492 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T08:30:22.492 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T08:30:22.492 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T08:30:22.492 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T08:30:22.492 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.492 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.492 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.493 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T08:30:22.493 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T08:30:22.493 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T08:30:22.493 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T08:30:22.493 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T08:30:22.493 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.493 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.494 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T08:30:22.494 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.494 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T08:30:22.494 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T08:30:22.494 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T08:30:22.494 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T08:30:22.494 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T08:30:22.494 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T08:30:22.495 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T08:30:22.496 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T08:30:22.496 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.496 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.496 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.496 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T08:30:22.496 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T08:30:22.496 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T08:30:22.497 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T08:30:22.497 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T08:30:22.497 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T08:30:22.497 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T08:30:22.497 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T08:30:22.497 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T08:30:22.497 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T08:30:22.497 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T08:30:22.497 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T08:30:22.497 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T08:30:22.497 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T08:30:22.497 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T08:30:22.498 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T08:30:22.499 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T08:30:22.500 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T08:30:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T08:30:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T08:30:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T08:30:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T08:30:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T08:30:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T08:30:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T08:30:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T08:30:22.501 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T08:30:22.502 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:22.502 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-04-25T08:30:22.898 DEBUG:teuthology.parallel:result is None 2024-04-25T08:30:27.482 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:30:27.482 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T08:30:27.482 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T08:30:27.482 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T08:30:27.482 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T08:30:27.482 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T08:30:27.482 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T08:30:27.482 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T08:30:27.483 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T08:30:27.484 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T08:30:27.485 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T08:30:27.486 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T08:30:27.487 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T08:30:27.488 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T08:30:27.489 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T08:30:27.490 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T08:30:27.491 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T08:30:27.492 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T08:30:27.493 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T08:30:27.493 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T08:30:27.493 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T08:30:27.493 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T08:30:27.493 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T08:30:27.493 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T08:30:27.493 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T08:30:27.493 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T08:30:27.493 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T08:30:27.493 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T08:30:29.708 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:30:29.708 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:29.708 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2024-04-25T08:30:29.708 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.708 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.708 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T08:30:29.708 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T08:30:29.708 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-04-25T08:30:29.708 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T08:30:29.708 INFO:teuthology.orchestra.run.smithi038.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:29.709 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.710 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T08:30:29.711 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T08:30:29.712 INFO:teuthology.orchestra.run.smithi038.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T08:30:29.713 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T08:30:29.714 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T08:30:29.715 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T08:30:29.716 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T08:30:29.717 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T08:30:29.718 INFO:teuthology.orchestra.run.smithi038.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T08:30:29.719 INFO:teuthology.orchestra.run.smithi038.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T08:30:29.719 INFO:teuthology.orchestra.run.smithi038.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T08:30:29.719 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T08:30:29.719 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T08:30:29.719 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T08:30:29.719 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:29.719 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-04-25T08:30:30.103 DEBUG:teuthology.parallel:result is None 2024-04-25T08:30:30.104 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:30:30.259 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T08:30:30.280 INFO:teuthology.orchestra.run.smithi038.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T08:30:30.281 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T08:30:30.281 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T08:30:30.283 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:30:30.470 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T08:30:30.496 INFO:teuthology.orchestra.run.smithi107.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T08:30:30.496 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T08:30:30.496 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T08:30:30.498 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T08:30:30.498 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:30:30.498 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T08:30:30.524 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:30:30.524 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T08:30:30.566 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T08:30:30.566 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:30:30.566 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T08:30:30.590 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T08:30:30.655 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:30:30.655 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T08:30:30.681 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T08:30:30.749 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T08:30:30.749 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:30:30.749 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T08:30:30.774 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T08:30:30.840 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:30:30.840 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T08:30:30.867 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T08:30:30.934 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T08:30:30.935 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:30:30.935 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T08:30:30.959 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T08:30:31.023 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:30:31.023 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T08:30:31.049 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T08:30:31.115 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T08:30:31.198 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T08:30:31.198 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /var/log/ceph 2024-04-25T08:30:31.201 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /var/log/ceph 2024-04-25T08:30:31.225 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T08:30:31.226 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T08:30:31.268 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T08:30:31.293 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T08:30:31.293 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T08:30:31.332 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T08:30:31.363 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T08:30:31.363 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T08:30:31.363 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T08:30:31.363 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:30:31.364 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T08:30:31.413 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T08:30:31.523 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T08:30:31.547 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:30:31.547 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T08:30:31.564 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T08:30:31.673 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T08:30:31.701 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T08:30:31.701 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T08:30:31.702 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7672992', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': '5787d42d10024ba410e822a8aba474331cb9af02'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5869, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '5787d42d10024ba410e822a8aba474331cb9af02', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi038.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIYx+4NFbsXvRNQVeU3mlgj+I6GCNPAF8eodiBAugkhbrJfWcXhH8A57EDUE4AfANJckXLAVWUwHuFcoH7TBTDk=', 'smithi107.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbgaLzu3yYtGUBP1iXTo1WsORaHNe1iKjeGse+xaRn5nBw3LS6RLZsuqA8SP4gt2Upkh5nfbMGbWrg4Y/lEdX0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_08:15:18', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T08:30:31.702 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T08:30:31.705 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T08:30:31.752 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T08:30:31.760 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T08:30:31.815 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:30:31.815 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T08:30:31.838 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T08:30:31.838 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-04-25T08:30:31.893 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T08:30:31.893 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:30:31.894 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T08:30:31.894 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:30:31.894 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:30:31.894 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-04-25 08:30:22.605717502 +0000 2024-04-25T08:30:31.894 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-04-25 08:28:33.756721378 +0000 2024-04-25T08:30:31.894 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-04-25 08:28:33.756721378 +0000 2024-04-25T08:30:31.894 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-04-25 08:28:33.756721378 +0000 2024-04-25T08:30:31.894 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T08:30:31.958 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:30:31.958 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:30:31.958 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000298217 s, 1.7 MB/s 2024-04-25T08:30:31.959 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T08:30:32.016 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-04-25T08:30:32.071 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T08:30:32.071 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:30:32.071 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T08:30:32.071 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:30:32.071 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:30:32.071 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-04-25 08:30:22.606717483 +0000 2024-04-25T08:30:32.071 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-04-25 08:28:34.099714924 +0000 2024-04-25T08:30:32.071 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-04-25 08:28:34.099714924 +0000 2024-04-25T08:30:32.071 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-04-25 08:28:34.099714924 +0000 2024-04-25T08:30:32.072 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T08:30:32.134 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:30:32.134 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:30:32.134 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000260534 s, 2.0 MB/s 2024-04-25T08:30:32.135 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T08:30:32.191 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-04-25T08:30:32.246 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T08:30:32.246 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:30:32.246 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T08:30:32.246 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:30:32.246 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:30:32.246 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-04-25 08:30:22.606717483 +0000 2024-04-25T08:30:32.246 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-04-25 08:28:34.448708357 +0000 2024-04-25T08:30:32.246 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-04-25 08:28:34.448708357 +0000 2024-04-25T08:30:32.246 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-04-25 08:28:34.448708357 +0000 2024-04-25T08:30:32.246 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T08:30:32.308 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:30:32.308 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:30:32.308 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000263043 s, 1.9 MB/s 2024-04-25T08:30:32.309 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T08:30:32.365 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-04-25T08:30:32.421 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T08:30:32.421 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:30:32.421 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T08:30:32.421 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:30:32.421 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:30:32.421 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-04-25 08:30:22.607717465 +0000 2024-04-25T08:30:32.421 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-04-25 08:28:34.793701866 +0000 2024-04-25T08:30:32.421 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-04-25 08:28:34.793701866 +0000 2024-04-25T08:30:32.421 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-04-25 08:28:34.793701866 +0000 2024-04-25T08:30:32.422 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T08:30:32.483 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:30:32.483 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:30:32.483 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000275043 s, 1.9 MB/s 2024-04-25T08:30:32.484 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T08:30:32.540 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:30:32.540 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:30:32.540 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T08:30:32.558 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T08:30:32.558 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-04-25T08:30:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T08:30:32.614 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:30:32.614 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T08:30:32.614 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:30:32.614 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:30:32.614 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-25 08:30:15.561292358 +0000 2024-04-25T08:30:32.615 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-25 08:28:31.642713690 +0000 2024-04-25T08:30:32.615 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-25 08:28:31.642713690 +0000 2024-04-25T08:30:32.615 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-25 08:28:31.642713690 +0000 2024-04-25T08:30:32.615 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T08:30:32.678 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-25T08:30:32.678 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-25T08:30:32.678 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000340541 s, 1.5 MB/s 2024-04-25T08:30:32.680 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T08:30:32.737 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-04-25T08:30:32.792 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T08:30:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:30:32.793 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T08:30:32.793 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:30:32.793 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:30:32.793 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-25 08:30:15.561292358 +0000 2024-04-25T08:30:32.793 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-25 08:28:31.989707642 +0000 2024-04-25T08:30:32.793 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-25 08:28:31.989707642 +0000 2024-04-25T08:30:32.793 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-25 08:28:31.989707642 +0000 2024-04-25T08:30:32.793 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T08:30:32.856 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-25T08:30:32.856 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-25T08:30:32.856 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000252658 s, 2.0 MB/s 2024-04-25T08:30:32.857 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T08:30:32.913 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-04-25T08:30:32.967 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T08:30:32.968 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:30:32.968 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T08:30:32.968 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:30:32.968 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:30:32.968 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-25 08:30:15.562292342 +0000 2024-04-25T08:30:32.968 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-25 08:28:32.329701717 +0000 2024-04-25T08:30:32.968 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-25 08:28:32.329701717 +0000 2024-04-25T08:30:32.968 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-25 08:28:32.329701717 +0000 2024-04-25T08:30:32.968 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T08:30:33.029 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-25T08:30:33.029 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-25T08:30:33.030 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000202999 s, 2.5 MB/s 2024-04-25T08:30:33.031 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T08:30:33.086 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-04-25T08:30:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T08:30:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:30:33.141 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T08:30:33.141 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:30:33.141 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:30:33.141 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-04-25 08:30:15.562292342 +0000 2024-04-25T08:30:33.141 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-04-25 08:28:32.666695844 +0000 2024-04-25T08:30:33.141 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-04-25 08:28:32.666695844 +0000 2024-04-25T08:30:33.141 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-04-25 08:28:32.666695844 +0000 2024-04-25T08:30:33.141 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T08:30:33.202 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-04-25T08:30:33.202 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-04-25T08:30:33.202 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000298707 s, 1.7 MB/s 2024-04-25T08:30:33.203 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T08:30:33.259 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:30:33.259 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi038.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi107.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T08:30:33.259 INFO:tasks.ceph:Generating config... 2024-04-25T08:30:33.260 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T08:30:33.260 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T08:30:33.260 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T08:30:33.260 INFO:tasks.ceph:[client] pid file = 2024-04-25T08:30:33.260 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T08:30:33.261 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T08:30:33.261 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T08:30:33.262 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T08:30:33.262 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T08:30:33.262 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T08:30:33.262 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T08:30:33.262 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T08:30:33.262 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T08:30:33.262 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T08:30:33.262 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T08:30:33.262 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T08:30:33.262 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T08:30:33.262 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T08:30:33.262 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T08:30:33.262 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T08:30:33.262 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T08:30:33.262 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T08:30:33.262 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T08:30:33.263 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T08:30:33.263 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T08:30:33.263 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T08:30:33.321 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T08:30:33.324 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T08:30:33.368 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:30:33.393 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.38'), ('mon.b', '172.21.15.107'), ('mon.c', '[v2:172.21.15.107:3301,v1:172.21.15.107:6790]')] 2024-04-25T08:30:33.393 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.38,172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T08:30:33.394 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:30:33.394 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T08:30:33.450 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.38 --add b 172.21.15.107 --addv c '[v2:172.21.15.107:3301,v1:172.21.15.107:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool: generated fsid b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:setting min_mon_release = quincy 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 0 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:fsid b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:last_changed 2024-04-25T08:30:33.542920+0000 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-04-25T08:30:33.542920+0000 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:min_mon_release 17 (quincy) 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:election_strategy: 1 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:1: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.b 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:2: [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] mon.c 2024-04-25T08:30:33.539 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T08:30:33.541 DEBUG:teuthology.orchestra.run.smithi038:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T08:30:33.555 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762... 2024-04-25T08:30:33.556 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:30:33.598 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:30:33.637 INFO:teuthology.orchestra.run.smithi038.stdout:[global] 2024-04-25T08:30:33.637 INFO:teuthology.orchestra.run.smithi038.stdout: chdir = "" 2024-04-25T08:30:33.637 INFO:teuthology.orchestra.run.smithi038.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:30:33.637 INFO:teuthology.orchestra.run.smithi038.stdout: auth supported = cephx 2024-04-25T08:30:33.637 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.637 INFO:teuthology.orchestra.run.smithi038.stdout: filestore xattr use omap = true 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: mon clock drift allowed = 1.000 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: osd crush chooseleaf type = 0 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: auth debug = true 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: ms die on old message = true 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: ms die on bug = true 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: mon pg warn max object skew = 0 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:30:33.638 INFO:teuthology.orchestra.run.smithi038.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: osd pool default size = 2 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd allow primary affinity = true 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd allow pg remap = true 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on no sortbitwise = false 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: mon warn on too few osds = false 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: mon_allow_pool_size_one = true 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:30:33.639 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: osd default data pool replay window = 5 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: mon allow pool delete = true 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: mon cluster log file level = debug 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: debug asserts on shutdown = true 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: mon health detail to clog = false 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: mon host = "172.21.15.38,172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790]" 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: fsid = b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout:[osd] 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: osd journal size = 100 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: osd scrub load threshold = 5.0 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: osd scrub max interval = 600 2024-04-25T08:30:33.640 INFO:teuthology.orchestra.run.smithi038.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: osd recover clone overlap = true 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: osd recovery max chunk = 1048576 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug shutdown = true 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug op order = true 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug verify stray on activate = true 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug trim objects = true 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: osd open classes on start = true 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug pg log writeout = true 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: osd map max advance = 10 2024-04-25T08:30:33.641 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: journal zero on create = true 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: filestore apply finisher threads = 3 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: bdev debug aio = true 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: osd debug misdirected ops = true 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: bluestore block size = 96636764160 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: bluestore fsck on mount = True 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: debug bluefs = 20 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: debug bluestore = 20 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: debug osd = 20 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: debug rocksdb = 10 2024-04-25T08:30:33.642 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd full ratio = 0.9 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: osd objectstore = bluestore 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: osd op complaint time = 180 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr] 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: debug mgr = 20 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: debug mon = 20 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: debug auth = 20 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: mgr/telemetry/nag = false 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.643 INFO:teuthology.orchestra.run.smithi038.stdout:[mon] 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: debug mon = 20 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: debug paxos = 20 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: debug auth = 20 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: mon data avail warn = 5 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: mon mgr mkfs grace = 240 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd reporter subtree level = osd 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: mon osd prime pg temp = true 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:30:33.644 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: # 1m isn't quite enough 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: mon op complaint time = 120 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout:[client] 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: rgw cache enabled = true 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: rgw enable ops log = true 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: rgw enable usage log = true 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: client mount timeout = 600 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: debug client = 20 2024-04-25T08:30:33.645 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout: pid file = "" 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout: rados mon op timeout = 900 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout: rados osd op timeout = 900 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.a] 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.b] 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout:[mon.c] 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a] 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.c] 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.b] 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.d] 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout:[mds] 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout: debug mds = 20 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout: debug mds balancer = 20 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout: debug ms = 1 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout: mds cache max size = 1000 2024-04-25T08:30:33.646 INFO:teuthology.orchestra.run.smithi038.stdout: mds debug frag = True 2024-04-25T08:30:33.647 INFO:teuthology.orchestra.run.smithi038.stdout: mds debug scatterstat = True 2024-04-25T08:30:33.647 INFO:teuthology.orchestra.run.smithi038.stdout: mds log max segments = 1 2024-04-25T08:30:33.647 INFO:teuthology.orchestra.run.smithi038.stdout: mds op complaint time = 180 2024-04-25T08:30:33.647 INFO:teuthology.orchestra.run.smithi038.stdout: mds verify scatter = True 2024-04-25T08:30:33.647 INFO:teuthology.orchestra.run.smithi038.stdout: osd op complaint time = 180 2024-04-25T08:30:33.647 INFO:teuthology.orchestra.run.smithi038.stdout: rados mon op timeout = 900 2024-04-25T08:30:33.647 INFO:teuthology.orchestra.run.smithi038.stdout: rados osd op timeout = 900 2024-04-25T08:30:33.647 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-04-25T08:30:33.647 INFO:teuthology.orchestra.run.smithi107.stdout: chdir = "" 2024-04-25T08:30:33.647 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:30:33.647 INFO:teuthology.orchestra.run.smithi107.stdout: auth supported = cephx 2024-04-25T08:30:33.647 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: filestore xattr use omap = true 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: mon clock drift allowed = 1.000 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: osd crush chooseleaf type = 0 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: auth debug = true 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on old message = true 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: ms die on bug = true 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: mon pg warn max object skew = 0 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:30:33.648 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default size = 2 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow primary affinity = true 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd allow pg remap = true 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on no sortbitwise = false 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: mon warn on too few osds = false 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: mon_allow_pool_size_one = true 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.649 INFO:teuthology.orchestra.run.smithi107.stdout: osd default data pool replay window = 5 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: mon allow pool delete = true 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: mon cluster log file level = debug 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: debug asserts on shutdown = true 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: mon health detail to clog = false 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: mon host = "172.21.15.38,172.21.15.107,[v2:172.21.15.107:3301,v1:172.21.15.107:6790]" 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout:[osd] 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: osd journal size = 100 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub load threshold = 5.0 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: osd scrub max interval = 600 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:30:33.650 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: osd recover clone overlap = true 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: osd recovery max chunk = 1048576 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug shutdown = true 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug op order = true 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug verify stray on activate = true 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug trim objects = true 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: osd open classes on start = true 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug pg log writeout = true 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: osd map max advance = 10 2024-04-25T08:30:33.651 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: journal zero on create = true 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: filestore apply finisher threads = 3 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: bdev debug aio = true 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: osd debug misdirected ops = true 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore block size = 96636764160 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: bluestore fsck on mount = True 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluefs = 20 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: debug bluestore = 20 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: debug osd = 20 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: debug rocksdb = 10 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd full ratio = 0.9 2024-04-25T08:30:33.652 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: osd objectstore = bluestore 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr] 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: debug mgr = 20 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: mgr/telemetry/nag = false 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout:[mon] 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-04-25T08:30:33.653 INFO:teuthology.orchestra.run.smithi107.stdout: debug mon = 20 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: debug paxos = 20 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: debug auth = 20 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: mon data avail warn = 5 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: mon mgr mkfs grace = 240 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd reporter subtree level = osd 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: mon osd prime pg temp = true 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:30:33.654 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: # 1m isn't quite enough 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: mon op complaint time = 120 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout:[client] 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: rgw cache enabled = true 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable ops log = true 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: rgw enable usage log = true 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: client mount timeout = 600 2024-04-25T08:30:33.655 INFO:teuthology.orchestra.run.smithi107.stdout: debug client = 20 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout: pid file = "" 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 900 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 900 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.a] 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.b] 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout:[mon.c] 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a] 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c] 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b] 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d] 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout:[mds] 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout: debug mds = 20 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout: debug mds balancer = 20 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout: debug ms = 1 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout: mds cache max size = 1000 2024-04-25T08:30:33.656 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug frag = True 2024-04-25T08:30:33.657 INFO:teuthology.orchestra.run.smithi107.stdout: mds debug scatterstat = True 2024-04-25T08:30:33.657 INFO:teuthology.orchestra.run.smithi107.stdout: mds log max segments = 1 2024-04-25T08:30:33.657 INFO:teuthology.orchestra.run.smithi107.stdout: mds op complaint time = 180 2024-04-25T08:30:33.657 INFO:teuthology.orchestra.run.smithi107.stdout: mds verify scatter = True 2024-04-25T08:30:33.657 INFO:teuthology.orchestra.run.smithi107.stdout: osd op complaint time = 180 2024-04-25T08:30:33.657 INFO:teuthology.orchestra.run.smithi107.stdout: rados mon op timeout = 900 2024-04-25T08:30:33.657 INFO:teuthology.orchestra.run.smithi107.stdout: rados osd op timeout = 900 2024-04-25T08:30:33.658 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T08:30:33.658 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T08:30:33.751 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T08:30:33.751 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:30:33.751 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T08:30:33.767 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:30:33.767 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T08:30:33.822 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:30:33.822 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:30:33.822 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T08:30:33.822 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:30:33.895 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:30:33.895 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:30:33.951 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:30:33.951 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:30:33.951 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T08:30:33.951 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:30:33.983 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:30:33.984 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:30:34.038 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T08:30:34.039 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T08:30:34.039 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T08:30:34.088 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T08:30:34.090 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T08:30:34.144 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T08:30:34.147 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T08:30:34.147 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T08:30:34.197 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T08:30:34.200 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T08:30:34.224 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T08:30:34.313 INFO:teuthology.orchestra.run.smithi038.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T08:30:34.316 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T08:30:34.340 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T08:30:34.387 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T08:30:34.390 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T08:30:34.413 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T08:30:34.500 INFO:teuthology.orchestra.run.smithi107.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T08:30:34.503 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T08:30:34.525 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T08:30:34.526 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T08:30:34.566 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T08:30:34.578 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T08:30:34.659 INFO:teuthology.orchestra.run.smithi038.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T08:30:34.671 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T08:30:34.709 INFO:teuthology.orchestra.run.smithi107.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T08:30:34.721 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T08:30:34.801 INFO:teuthology.orchestra.run.smithi107.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T08:30:34.813 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T08:30:34.813 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi038.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi107.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T08:30:34.813 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T08:30:34.837 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:30:34.837 INFO:tasks.ceph:role: osd.0 2024-04-25T08:30:34.838 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:30:34.838 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T08:30:34.930 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:30:34.930 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:30:34.930 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:30:34.930 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:30:34.930 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:30:34.930 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:30:34.930 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:30:34.930 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:30:34.930 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:30:34.930 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:30:36.274 INFO:teuthology.orchestra.run.smithi038.stdout:Discarding blocks...Done. 2024-04-25T08:30:36.276 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2024-04-25T08:30:36.276 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T08:30:36.313 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T08:30:36.387 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T08:30:36.455 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:30:36.455 INFO:tasks.ceph:role: osd.1 2024-04-25T08:30:36.455 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:30:36.455 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T08:30:36.550 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:30:36.550 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:30:36.550 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:30:36.550 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:30:36.550 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:30:36.550 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:30:36.550 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:30:36.550 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:30:36.550 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:30:36.550 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:30:37.903 INFO:teuthology.orchestra.run.smithi038.stdout:Discarding blocks...Done. 2024-04-25T08:30:37.928 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2024-04-25T08:30:37.929 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T08:30:37.964 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T08:30:38.036 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T08:30:38.104 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:30:38.104 INFO:tasks.ceph:role: osd.2 2024-04-25T08:30:38.104 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:30:38.104 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T08:30:38.199 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:30:38.199 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:30:38.199 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:30:38.199 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:30:38.199 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:30:38.199 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:30:38.199 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:30:38.199 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:30:38.199 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:30:38.199 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:30:39.519 INFO:teuthology.orchestra.run.smithi038.stdout:Discarding blocks...Done. 2024-04-25T08:30:39.521 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2024-04-25T08:30:39.521 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T08:30:39.557 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T08:30:39.626 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T08:30:39.690 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:30:39.690 INFO:tasks.ceph:role: osd.3 2024-04-25T08:30:39.690 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:30:39.690 DEBUG:teuthology.orchestra.run.smithi038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T08:30:39.784 INFO:teuthology.orchestra.run.smithi038.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:30:39.785 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:30:39.785 INFO:teuthology.orchestra.run.smithi038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:30:39.785 INFO:teuthology.orchestra.run.smithi038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:30:39.785 INFO:teuthology.orchestra.run.smithi038.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:30:39.785 INFO:teuthology.orchestra.run.smithi038.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:30:39.785 INFO:teuthology.orchestra.run.smithi038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:30:39.785 INFO:teuthology.orchestra.run.smithi038.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:30:39.785 INFO:teuthology.orchestra.run.smithi038.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:30:39.785 INFO:teuthology.orchestra.run.smithi038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:30:41.116 INFO:teuthology.orchestra.run.smithi038.stdout:Discarding blocks...Done. 2024-04-25T08:30:41.119 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi038.front.sepia.ceph.com -o noatime 2024-04-25T08:30:41.120 DEBUG:teuthology.orchestra.run.smithi038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T08:30:41.156 DEBUG:teuthology.orchestra.run.smithi038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T08:30:41.229 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:30:41.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:41.327+0000 7fa60c661640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T08:30:41.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:41.328+0000 7fa60c661640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T08:30:41.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:41.328+0000 7fa60c661640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:30:41.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:41.328+0000 7fa60c661640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:30:41.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:41.328+0000 7fa60c661640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:30:41.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:41.329+0000 7fa60c661640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T08:30:44.214 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T08:30:44.243 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:30:44.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:44.335+0000 7f0aca357640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T08:30:44.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:44.335+0000 7f0aca357640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T08:30:44.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:44.335+0000 7f0aca357640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:30:44.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:44.335+0000 7f0aca357640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:30:44.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:44.335+0000 7f0aca357640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:30:44.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:44.336+0000 7f0aca357640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T08:30:47.199 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T08:30:47.229 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:30:47.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:47.322+0000 7fb396ef0640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T08:30:47.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:47.322+0000 7fb396ef0640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T08:30:47.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:47.322+0000 7fb396ef0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:30:47.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:47.322+0000 7fb396ef0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:30:47.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:47.322+0000 7fb396ef0640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:30:47.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:47.323+0000 7fb396ef0640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T08:30:50.187 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T08:30:50.216 DEBUG:teuthology.orchestra.run.smithi038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:30:50.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:50.309+0000 7f8e7b51f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T08:30:50.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:50.309+0000 7f8e7b51f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T08:30:50.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:50.309+0000 7f8e7b51f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:30:50.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:50.309+0000 7f8e7b51f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:30:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:50.309+0000 7f8e7b51f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:30:50.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:30:50.310+0000 7f8e7b51f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T08:30:53.188 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T08:30:53.217 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T08:30:53.245 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:30:53.245 INFO:tasks.ceph:role: osd.4 2024-04-25T08:30:53.245 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:30:53.245 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T08:30:53.320 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:30:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:30:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:30:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:30:53.321 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:30:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:30:53.321 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:30:53.321 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:30:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:30:53.321 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:30:53.821 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-04-25T08:30:53.823 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-04-25T08:30:53.823 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T08:30:53.947 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T08:30:53.976 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T08:30:54.040 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:30:54.041 INFO:tasks.ceph:role: osd.5 2024-04-25T08:30:54.041 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:30:54.041 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T08:30:54.111 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:30:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:30:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:30:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:30:54.112 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:30:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:30:54.112 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:30:54.112 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:30:54.112 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:30:54.112 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:30:54.570 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-04-25T08:30:54.572 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-04-25T08:30:54.572 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T08:30:54.667 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T08:30:54.694 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T08:30:54.758 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:30:54.758 INFO:tasks.ceph:role: osd.6 2024-04-25T08:30:54.759 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:30:54.759 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T08:30:54.830 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:30:54.830 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:30:54.830 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:30:54.830 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:30:54.830 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:30:54.830 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:30:54.830 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:30:54.830 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:30:54.830 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:30:54.830 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:30:55.312 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-04-25T08:30:55.314 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-04-25T08:30:55.314 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T08:30:55.441 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T08:30:55.470 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T08:30:55.534 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:30:55.534 INFO:tasks.ceph:role: osd.7 2024-04-25T08:30:55.534 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:30:55.535 DEBUG:teuthology.orchestra.run.smithi107:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T08:30:55.605 INFO:teuthology.orchestra.run.smithi107.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:30:55.605 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:30:55.605 INFO:teuthology.orchestra.run.smithi107.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:30:55.606 INFO:teuthology.orchestra.run.smithi107.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:30:55.606 INFO:teuthology.orchestra.run.smithi107.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:30:55.606 INFO:teuthology.orchestra.run.smithi107.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:30:55.606 INFO:teuthology.orchestra.run.smithi107.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:30:55.606 INFO:teuthology.orchestra.run.smithi107.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:30:55.606 INFO:teuthology.orchestra.run.smithi107.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:30:55.606 INFO:teuthology.orchestra.run.smithi107.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:30:56.100 INFO:teuthology.orchestra.run.smithi107.stdout:Discarding blocks...Done. 2024-04-25T08:30:56.102 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi107.front.sepia.ceph.com -o noatime 2024-04-25T08:30:56.102 DEBUG:teuthology.orchestra.run.smithi107:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T08:30:56.251 DEBUG:teuthology.orchestra.run.smithi107:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T08:30:56.280 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:30:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:30:56.372+0000 7fb52621c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T08:30:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:30:56.372+0000 7fb52621c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T08:30:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:30:56.372+0000 7fb52621c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:30:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:30:56.373+0000 7fb52621c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:30:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:30:56.373+0000 7fb52621c640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:30:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:30:56.373+0000 7fb52621c640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T08:30:59.221 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T08:30:59.248 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:30:59.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:30:59.339+0000 7f7b9b430640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T08:30:59.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:30:59.339+0000 7f7b9b430640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T08:30:59.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:30:59.339+0000 7f7b9b430640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:30:59.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:30:59.339+0000 7f7b9b430640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:30:59.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:30:59.339+0000 7f7b9b430640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:30:59.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:30:59.340+0000 7f7b9b430640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T08:31:01.704 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:31:01.706 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:31:02.182 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T08:31:02.210 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:31:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:02.302+0000 7fbbb4412640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T08:31:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:02.302+0000 7fbbb4412640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T08:31:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:02.302+0000 7fbbb4412640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:31:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:02.302+0000 7fbbb4412640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:31:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:02.302+0000 7fbbb4412640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:31:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:02.302+0000 7fbbb4412640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T08:31:05.157 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T08:31:05.183 DEBUG:teuthology.orchestra.run.smithi107:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:31:05.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:05.272+0000 7fbd40fa7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T08:31:05.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:05.273+0000 7fbd40fa7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T08:31:05.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:05.273+0000 7fbd40fa7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:31:05.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:05.273+0000 7fbd40fa7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:31:05.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:05.273+0000 7fbd40fa7640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:31:05.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:05.273+0000 7fbd40fa7640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T08:31:08.128 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T08:31:08.155 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T08:31:08.155 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:08.156 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T08:31:08.185 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:08.186 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T08:31:08.254 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:08.254 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T08:31:08.322 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:08.323 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T08:31:08.389 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:08.389 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T08:31:08.455 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:08.455 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T08:31:08.521 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:08.521 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T08:31:08.587 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:08.648 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T08:31:08.676 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:08.676 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T08:31:08.744 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:08.744 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T08:31:08.810 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:08.810 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T08:31:08.875 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:08.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T08:31:08.940 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:08.940 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T08:31:09.004 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:09.004 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T08:31:09.067 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:09.068 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T08:31:09.086 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:09.086 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T08:31:09.145 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:09.145 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T08:31:09.160 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:09.160 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T08:31:09.214 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T08:31:09.214 DEBUG:teuthology.orchestra.run.smithi038:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T08:31:09.216 DEBUG:teuthology.orchestra.run.smithi107:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T08:31:09.256 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr.y] 2024-04-25T08:31:09.256 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmcr96BRAAQwUPxILJukYERAiEXeur+g== 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a] 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmwq8CDBAA+cjd+QXtFgwZ3kxdABRWlA== 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.c] 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmGLjqEhAAyPRgyy5iMgtccapydv4uOQ== 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.0] 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCxFCpmv+iTExAAEJihmsvBbp9xXxPnj49D6g== 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.1] 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQC0FCpmdnQKFBAADLAsYZEUhoJx292jIYZroQ== 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.2] 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQC3FCpm4f5BExAAMVSPkqsrnuLcO88AXKrGHQ== 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.3] 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQC6FCpmHNN7EhAABPhmnxGo70ZOd8Oex62Mdw== 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout:[mgr.x] 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmcXHhCBAAvI5zmWidVTvOmEcVRWasEA== 2024-04-25T08:31:09.257 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.b] 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmPRFbFxAAojn2r/62vvf1qTy9QVuyIw== 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.d] 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmt0AUHhAAoQyQ7wXBMGE+7Hi84tUfAg== 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.4] 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQDAFCpmiThGFhAAG5Pyvn/JzL9NUoxHJkPMvg== 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.5] 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQDDFCpmb/FHFBAA2cMhMqQQc4Fk6j8FpjhnaQ== 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.6] 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQDGFCpmAZsSEhAAnGIqqcvuHMVSc8Zbk309kA== 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout:[osd.7] 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQDJFCpmqdVUEBAAb9abblyUztrjwxxe4jsAwQ== 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw== 2024-04-25T08:31:09.258 INFO:teuthology.orchestra.run.smithi038.stdout:[client.2] 2024-04-25T08:31:09.259 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmH22HJxAAWyrooyJn5hRs/bb0Rn1HHw== 2024-04-25T08:31:09.259 INFO:teuthology.orchestra.run.smithi038.stdout:[client.1] 2024-04-25T08:31:09.259 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmL7eKKhAASeQEdHDBInfozdLVzP4DQw== 2024-04-25T08:31:09.259 INFO:teuthology.orchestra.run.smithi038.stdout:[client.3] 2024-04-25T08:31:09.259 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpm6Zr/LxAAxh9Y2gvvqj1jIaVVrQAvgw== 2024-04-25T08:31:09.275 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr.y] 2024-04-25T08:31:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCqFCpmcr96BRAAQwUPxILJukYERAiEXeur+g== 2024-04-25T08:31:09.275 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a] 2024-04-25T08:31:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCqFCpmwq8CDBAA+cjd+QXtFgwZ3kxdABRWlA== 2024-04-25T08:31:09.275 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c] 2024-04-25T08:31:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCqFCpmGLjqEhAAyPRgyy5iMgtccapydv4uOQ== 2024-04-25T08:31:09.275 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.0] 2024-04-25T08:31:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCxFCpmv+iTExAAEJihmsvBbp9xXxPnj49D6g== 2024-04-25T08:31:09.275 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.1] 2024-04-25T08:31:09.275 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQC0FCpmdnQKFBAADLAsYZEUhoJx292jIYZroQ== 2024-04-25T08:31:09.275 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.2] 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQC3FCpm4f5BExAAMVSPkqsrnuLcO88AXKrGHQ== 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.3] 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQC6FCpmHNN7EhAABPhmnxGo70ZOd8Oex62Mdw== 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout:[mgr.x] 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCqFCpmcXHhCBAAvI5zmWidVTvOmEcVRWasEA== 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b] 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCqFCpmPRFbFxAAojn2r/62vvf1qTy9QVuyIw== 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d] 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCqFCpmt0AUHhAAoQyQ7wXBMGE+7Hi84tUfAg== 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.4] 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDAFCpmiThGFhAAG5Pyvn/JzL9NUoxHJkPMvg== 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.5] 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDDFCpmb/FHFBAA2cMhMqQQc4Fk6j8FpjhnaQ== 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.6] 2024-04-25T08:31:09.276 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDGFCpmAZsSEhAAnGIqqcvuHMVSc8Zbk309kA== 2024-04-25T08:31:09.277 INFO:teuthology.orchestra.run.smithi107.stdout:[osd.7] 2024-04-25T08:31:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQDJFCpmqdVUEBAAb9abblyUztrjwxxe4jsAwQ== 2024-04-25T08:31:09.277 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-04-25T08:31:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw== 2024-04-25T08:31:09.277 INFO:teuthology.orchestra.run.smithi107.stdout:[client.2] 2024-04-25T08:31:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCqFCpmH22HJxAAWyrooyJn5hRs/bb0Rn1HHw== 2024-04-25T08:31:09.277 INFO:teuthology.orchestra.run.smithi107.stdout:[client.1] 2024-04-25T08:31:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCqFCpmL7eKKhAASeQEdHDBInfozdLVzP4DQw== 2024-04-25T08:31:09.277 INFO:teuthology.orchestra.run.smithi107.stdout:[client.3] 2024-04-25T08:31:09.277 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQCqFCpm6Zr/LxAAxh9Y2gvvqj1jIaVVrQAvgw== 2024-04-25T08:31:09.278 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:31:09.302 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:31:09.367 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:31:09.369 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:31:09.422 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:31:09.424 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:31:09.476 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.478 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.527 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.529 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.577 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.579 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.627 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.629 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.677 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:31:09.679 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:31:09.727 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:31:09.729 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:31:09.781 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:31:09.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:31:09.834 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.836 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.884 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.886 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.934 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.936 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.985 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:09.987 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:31:10.035 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:31:10.037 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:31:10.089 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:31:10.091 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:31:10.141 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:31:10.143 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:31:10.195 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:31:10.197 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:31:10.248 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T08:31:10.248 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T08:31:10.272 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:31:10.377 DEBUG:teuthology.orchestra.run.smithi038:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T08:31:10.402 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T08:31:10.425 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:31:10.518 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T08:31:10.541 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T08:31:10.604 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:31:10.697 DEBUG:teuthology.orchestra.run.smithi107:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T08:31:10.720 DEBUG:teuthology.orchestra.run.smithi038:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:31:10.722 DEBUG:teuthology.orchestra.run.smithi107:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:31:10.774 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T08:31:10.774 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T08:31:10.774 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T08:31:10.778 INFO:tasks.ceph.mon.a:Started 2024-04-25T08:31:10.778 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T08:31:10.778 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T08:31:10.816 INFO:tasks.ceph.mon.b:Started 2024-04-25T08:31:10.816 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T08:31:10.816 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T08:31:10.818 INFO:tasks.ceph.mon.c:Started 2024-04-25T08:31:10.818 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T08:31:10.818 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T08:31:10.818 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T08:31:10.820 INFO:tasks.ceph.mgr.y:Started 2024-04-25T08:31:10.820 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T08:31:10.821 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T08:31:10.822 INFO:tasks.ceph.mgr.x:Started 2024-04-25T08:31:10.823 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T08:31:10.823 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T08:31:10.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.927+0000 7f2003b7e640 1 Processor -- start 2024-04-25T08:31:10.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.927+0000 7f2003b7e640 1 -- start start 2024-04-25T08:31:10.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f2003b7e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1ffc15baf0 0x7f1ffc0cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:10.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f2003b7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc0cfb70 0x7f1ffc071f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:10.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f2003b7e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc0b3250 con 0x7f1ffc072490 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f2003b7e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ffc0731d0 con 0x7f1ffc15baf0 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f2003b7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc072c10 con 0x7f1ffc0cfb70 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f200211b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1ffc072490 0x7f1ffc0cbdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51170/0 (socket says 172.21.15.38:51170) 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f200211b640 1 -- 172.21.15.38:0/2693106225 learned_addr learned my addr 172.21.15.38:0/2693106225 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f2001119640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc0cfb70 0x7f1ffc071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f200191a640 1 --2- 172.21.15.38:0/2693106225 >> v2:172.21.15.107:3300/0 conn(0x7f1ffc15baf0 0x7f1ffc0cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f2001119640 1 -- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc0cfb70 msgr2=0x7f1ffc071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f2001119640 1 -- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc0cfb70 msgr2=0x7f1ffc071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f2001119640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc0cfb70 0x7f1ffc071f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f2001119640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc0cfb70 0x7f1ffc071f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f200191a640 1 -- 172.21.15.38:0/2693106225 >> v2:172.21.15.107:3300/0 conn(0x7f1ffc15baf0 msgr2=0x7f1ffc0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f200191a640 1 -- 172.21.15.38:0/2693106225 >> v2:172.21.15.107:3300/0 conn(0x7f1ffc15baf0 msgr2=0x7f1ffc0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:31:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f200191a640 1 --2- 172.21.15.38:0/2693106225 >> v2:172.21.15.107:3300/0 conn(0x7f1ffc15baf0 0x7f1ffc0cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:31:10.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:10.928+0000 7f200191a640 1 --2- 172.21.15.38:0/2693106225 >> v2:172.21.15.107:3300/0 conn(0x7f1ffc15baf0 0x7f1ffc0cf630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T08:31:11.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.098+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345875960 0 0) 0x7f1ff4001490 con 0x7f1ffc072490 2024-04-25T08:31:11.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.098+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fe8003520 con 0x7f1ffc072490 2024-04-25T08:31:11.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3165169393 0 0) 0x7f1fe8003520 con 0x7f1ffc072490 2024-04-25T08:31:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc0cfb70 msgr2=0x7f1ffc071f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:31:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc0cfb70 0x7f1ffc071f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 >> v2:172.21.15.107:3300/0 conn(0x7f1ffc15baf0 msgr2=0x7f1ffc0cf630 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:31:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 --2- 172.21.15.38:0/2693106225 >> v2:172.21.15.107:3300/0 conn(0x7f1ffc15baf0 0x7f1ffc0cf630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc072ef0 con 0x7f1ffc072490 2024-04-25T08:31:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f1ff4001bc0 con 0x7f1ffc072490 2024-04-25T08:31:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 >> v1:172.21.15.38:6789/0 conn(0x7f1ffc072490 legacy=0x7f1ffc0cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe80039b0 0x7f1fe8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fe8004800 0x7f1fe8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc0cfb70 0x7f1fe8005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc0731d0 con 0x7f1fe8004800 2024-04-25T08:31:11.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc072c10 con 0x7f1fe80039b0 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe800bba0 con 0x7f1ffc0cfb70 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2001119640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fe8004800 0x7f1fe8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f200191a640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe80039b0 0x7f1fe8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f200211b640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc0cfb70 0x7f1fe8005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2001119640 1 -- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc0cfb70 msgr2=0x7f1fe8005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2001119640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc0cfb70 0x7f1fe8005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2001119640 1 -- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe80039b0 msgr2=0x7f1fe8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2001119640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe80039b0 0x7f1fe8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.099+0000 7f2001119640 1 -- 172.21.15.38:0/2693106225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe8003520 con 0x7f1fe8004800 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.100+0000 7f2001119640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fe8004800 0x7f1fe8004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00b9e0 tx=0x7f1fe8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1852c6bd63c8c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.100+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec030070 con 0x7f1fe8004800 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.101+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fec02ada0 con 0x7f1fe8004800 2024-04-25T08:31:11.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.101+0000 7f2000918640 1 -- 172.21.15.38:0/2693106225 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fec00b5e0 con 0x7f1fe8004800 2024-04-25T08:31:11.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.101+0000 7f2003b7e640 1 -- 172.21.15.38:0/2693106225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fe8004800 msgr2=0x7f1fe8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:11.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.101+0000 7f2003b7e640 1 --2- 172.21.15.38:0/2693106225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1fe8004800 0x7f1fe8004c80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1fec00b9e0 tx=0x7f1fe8005830 comp rx=0 tx=0).stop 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.101+0000 7f200211b640 1 -- 172.21.15.38:0/2693106225 reap_dead start 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.101+0000 7f2003b7e640 1 -- 172.21.15.38:0/2693106225 shutdown_connections 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.101+0000 7f2003b7e640 1 -- 172.21.15.38:0/2693106225 >> 172.21.15.38:0/2693106225 conn(0x7f1ffc06adc0 msgr2=0x7f1ffc06b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.101+0000 7f2003b7e640 1 -- 172.21.15.38:0/2693106225 shutdown_connections 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.101+0000 7f2003b7e640 1 -- 172.21.15.38:0/2693106225 wait complete. 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.103+0000 7f2003b7e640 1 Processor -- start 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.103+0000 7f2003b7e640 1 -- start start 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f2003b7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe8004800 0x7f1ffc0c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f2003b7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ffc0c8c20 0x7f1ffc1e6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f2003b7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc1e67b0 0x7f1ffc0caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f2003b7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc0adb20 con 0x7f1ffc0c8c20 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f2003b7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc072920 con 0x7f1fe8004800 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f2003b7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc0adca0 con 0x7f1ffc1e67b0 2024-04-25T08:31:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f200191a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe8004800 0x7f1ffc0c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f2001119640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ffc0c8c20 0x7f1ffc1e6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f200191a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe8004800 0x7f1ffc0c5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:42376/0 (socket says 172.21.15.38:42376) 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f200191a640 1 -- 172.21.15.38:0/2045401095 learned_addr learned my addr 172.21.15.38:0/2045401095 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f200211b640 1 --2- 172.21.15.38:0/2045401095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc1e67b0 0x7f1ffc0caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f200191a640 1 -- 172.21.15.38:0/2045401095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc1e67b0 msgr2=0x7f1ffc0caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f200191a640 1 --2- 172.21.15.38:0/2045401095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc1e67b0 0x7f1ffc0caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f200191a640 1 -- 172.21.15.38:0/2045401095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ffc0c8c20 msgr2=0x7f1ffc1e6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f200191a640 1 --2- 172.21.15.38:0/2045401095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ffc0c8c20 0x7f1ffc1e6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f200191a640 1 -- 172.21.15.38:0/2045401095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc0ca180 con 0x7f1fe8004800 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.104+0000 7f2001119640 1 --2- 172.21.15.38:0/2045401095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ffc0c8c20 0x7f1ffc1e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.105+0000 7f200191a640 1 --2- 172.21.15.38:0/2045401095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe8004800 0x7f1ffc0c5100 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80108c0 tx=0x7f1ff8010d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.105+0000 7f1ff27fc640 1 -- 172.21.15.38:0/2045401095 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff801a020 con 0x7f1fe8004800 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.105+0000 7f1ff27fc640 1 -- 172.21.15.38:0/2045401095 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff800e9c0 con 0x7f1fe8004800 2024-04-25T08:31:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.105+0000 7f2003b7e640 1 -- 172.21.15.38:0/2045401095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ffc0ca460 con 0x7f1fe8004800 2024-04-25T08:31:11.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.105+0000 7f1ff27fc640 1 -- 172.21.15.38:0/2045401095 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800ed90 con 0x7f1fe8004800 2024-04-25T08:31:11.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.105+0000 7f2003b7e640 1 -- 172.21.15.38:0/2045401095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc0caa20 con 0x7f1fe8004800 2024-04-25T08:31:11.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.105+0000 7f1ff27fc640 1 -- 172.21.15.38:0/2045401095 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1ff801d020 con 0x7f1fe8004800 2024-04-25T08:31:11.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.105+0000 7f1ff27fc640 1 -- 172.21.15.38:0/2045401095 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1ff80149a0 con 0x7f1fe8004800 2024-04-25T08:31:11.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.106+0000 7f2003b7e640 1 -- 172.21.15.38:0/2045401095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ffc0b42e0 con 0x7f1fe8004800 2024-04-25T08:31:11.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.108+0000 7f1ff27fc640 1 -- 172.21.15.38:0/2045401095 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1ffc0b42e0 con 0x7f1fe8004800 2024-04-25T08:31:11.196 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:11.199+0000 7f1f8a212040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T08:31:11.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.221+0000 7f2003b7e640 1 -- 172.21.15.38:0/2045401095 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f1ffc068e80 con 0x7f1fe8004800 2024-04-25T08:31:11.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.229+0000 7f1ff27fc640 1 -- 172.21.15.38:0/2045401095 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1ff8024020 con 0x7f1fe8004800 2024-04-25T08:31:11.226 INFO:teuthology.orchestra.run.smithi038.stderr:adjusted tunables profile to default 2024-04-25T08:31:11.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.230+0000 7f2003b7e640 1 -- 172.21.15.38:0/2045401095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe8004800 msgr2=0x7f1ffc0c5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:11.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.230+0000 7f2003b7e640 1 --2- 172.21.15.38:0/2045401095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe8004800 0x7f1ffc0c5100 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80108c0 tx=0x7f1ff8010d80 comp rx=0 tx=0).stop 2024-04-25T08:31:11.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.230+0000 7f2003b7e640 1 -- 172.21.15.38:0/2045401095 shutdown_connections 2024-04-25T08:31:11.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.230+0000 7f2003b7e640 1 --2- 172.21.15.38:0/2045401095 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ffc1e67b0 0x7f1ffc0caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.230+0000 7f2003b7e640 1 --2- 172.21.15.38:0/2045401095 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ffc0c8c20 0x7f1ffc1e6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.230+0000 7f2003b7e640 1 --2- 172.21.15.38:0/2045401095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1fe8004800 0x7f1ffc0c5100 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.230+0000 7f2003b7e640 1 -- 172.21.15.38:0/2045401095 >> 172.21.15.38:0/2045401095 conn(0x7f1ffc06adc0 msgr2=0x7f1ffc0cc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:11.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.230+0000 7f2003b7e640 1 -- 172.21.15.38:0/2045401095 shutdown_connections 2024-04-25T08:31:11.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:11.230+0000 7f2003b7e640 1 -- 172.21.15.38:0/2045401095 wait complete. 2024-04-25T08:31:11.238 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T08:31:11.238 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T08:31:11.238 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:11.238 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T08:31:11.244 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:11.248+0000 7ff975c12040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T08:31:11.263 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:11.263 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T08:31:11.327 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:11.327 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T08:31:11.350 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:11.353+0000 7f1f8a212040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:31:11.391 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:11.391 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T08:31:11.403 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:11.406+0000 7f1f8a212040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T08:31:11.408 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:11.412+0000 7ff975c12040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:31:11.432 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:11.432 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T08:31:11.456 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:11.456 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T08:31:11.465 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:11.468+0000 7ff975c12040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T08:31:11.519 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:11.519 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T08:31:11.582 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:31:11.582 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T08:31:11.646 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:11.649+0000 7f1f8a212040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T08:31:11.647 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8 0 2024-04-25T08:31:11.693 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:11.696+0000 7f1f8a212040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T08:31:11.713 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:11.717+0000 7ff975c12040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T08:31:11.761 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:11.764+0000 7ff975c12040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T08:31:11.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.787+0000 7f0307b8b640 1 Processor -- start 2024-04-25T08:31:11.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.788+0000 7f0307b8b640 1 -- start start 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.788+0000 7f0307b8b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000d00e0 0x7f03000cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.788+0000 7f0307b8b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f03000cd3b0 0x7f03000cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.788+0000 7f0307b8b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03000cdd80 con 0x7f03000b1d60 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.788+0000 7f0307b8b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f030006abe0 con 0x7f03000cd3b0 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.788+0000 7f0307b8b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030006b480 con 0x7f03000d00e0 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.788+0000 7f0305126640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000d00e0 0x7f03000cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.788+0000 7f0306128640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f03000cd3b0 0x7f03000cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.788+0000 7f0305126640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000d00e0 0x7f03000cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:51208/0 (socket says 172.21.15.107:51208) 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.788+0000 7f0306128640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f03000cd3b0 0x7f03000cd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35240/0 (socket says 172.21.15.107:35240) 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.788+0000 7f0305126640 1 -- 172.21.15.107:0/3460786958 learned_addr learned my addr 172.21.15.107:0/3460786958 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.789+0000 7f0305126640 1 -- 172.21.15.107:0/3460786958 >> v2:172.21.15.107:3300/0 conn(0x7f03000cd3b0 msgr2=0x7f03000cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.789+0000 7f0305126640 1 --2- 172.21.15.107:0/3460786958 >> v2:172.21.15.107:3300/0 conn(0x7f03000cd3b0 0x7f03000cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.789+0000 7f0305126640 1 -- 172.21.15.107:0/3460786958 >> v1:172.21.15.38:6789/0 conn(0x7f03000b1d60 legacy=0x7f03000cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:11.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.789+0000 7f0305126640 1 -- 172.21.15.107:0/3460786958 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030015ef40 con 0x7f03000d00e0 2024-04-25T08:31:11.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.789+0000 7f0305126640 1 --2- 172.21.15.107:0/3460786958 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000d00e0 0x7f03000cce70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009870 tx=0x7f02fc02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aae3aebde86782 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:11.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.790+0000 7f0304925640 1 -- 172.21.15.107:0/3460786958 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc037070 con 0x7f03000d00e0 2024-04-25T08:31:11.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.790+0000 7f0304925640 1 -- 172.21.15.107:0/3460786958 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc032440 con 0x7f03000d00e0 2024-04-25T08:31:11.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.790+0000 7f0304925640 1 -- 172.21.15.107:0/3460786958 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc032810 con 0x7f03000d00e0 2024-04-25T08:31:11.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.790+0000 7f0307b8b640 1 -- 172.21.15.107:0/3460786958 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000d00e0 msgr2=0x7f03000cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:11.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.790+0000 7f0307b8b640 1 --2- 172.21.15.107:0/3460786958 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000d00e0 0x7f03000cce70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f02fc009870 tx=0x7f02fc02a270 comp rx=0 tx=0).stop 2024-04-25T08:31:11.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.790+0000 7f0307b8b640 1 -- 172.21.15.107:0/3460786958 shutdown_connections 2024-04-25T08:31:11.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.790+0000 7f0307b8b640 1 --2- 172.21.15.107:0/3460786958 >> v2:172.21.15.107:3300/0 conn(0x7f03000cd3b0 0x7f03000cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.790+0000 7f0307b8b640 1 --2- 172.21.15.107:0/3460786958 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000d00e0 0x7f03000cce70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.790+0000 7f0307b8b640 1 -- 172.21.15.107:0/3460786958 >> 172.21.15.107:0/3460786958 conn(0x7f03000d06b0 msgr2=0x7f03000d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:11.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.790+0000 7f0307b8b640 1 -- 172.21.15.107:0/3460786958 shutdown_connections 2024-04-25T08:31:11.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.790+0000 7f0307b8b640 1 -- 172.21.15.107:0/3460786958 wait complete. 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.791+0000 7f0307b8b640 1 Processor -- start 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.791+0000 7f0307b8b640 1 -- start start 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0307b8b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000b1d60 0x7f03001575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0307b8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03000cd3b0 0x7f03000c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0307b8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03000d00e0 0x7f030015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0307b8b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030006b910 con 0x7f03000d00e0 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0307b8b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030006b110 con 0x7f03000cd3b0 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0307b8b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030006ba90 con 0x7f03000b1d60 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0305927640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000b1d60 0x7f03001575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0305126640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03000cd3b0 0x7f03000c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0305927640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000b1d60 0x7f03001575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:51216/0 (socket says 172.21.15.107:51216) 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0305126640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03000cd3b0 0x7f03000c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35254/0 (socket says 172.21.15.107:35254) 2024-04-25T08:31:11.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0305927640 1 -- 172.21.15.107:0/913835125 learned_addr learned my addr 172.21.15.107:0/913835125 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0306128640 1 --2- 172.21.15.107:0/913835125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03000d00e0 0x7f030015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0305927640 1 -- 172.21.15.107:0/913835125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03000cd3b0 msgr2=0x7f03000c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0305927640 1 --2- 172.21.15.107:0/913835125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03000cd3b0 0x7f03000c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0305927640 1 -- 172.21.15.107:0/913835125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03000d00e0 msgr2=0x7f030015a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0305927640 1 --2- 172.21.15.107:0/913835125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03000d00e0 0x7f030015a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.792+0000 7f0305927640 1 -- 172.21.15.107:0/913835125 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03001f4120 con 0x7f03000b1d60 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.793+0000 7f0305927640 1 --2- 172.21.15.107:0/913835125 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000b1d60 0x7f03001575c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f02f400c3c0 tx=0x7f02f400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.793+0000 7f02f27fc640 1 -- 172.21.15.107:0/913835125 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f40052d0 con 0x7f03000b1d60 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.793+0000 7f02f27fc640 1 -- 172.21.15.107:0/913835125 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f4005490 con 0x7f03000b1d60 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.793+0000 7f0307b8b640 1 -- 172.21.15.107:0/913835125 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03000c9310 con 0x7f03000b1d60 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.793+0000 7f02f27fc640 1 -- 172.21.15.107:0/913835125 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4007260 con 0x7f03000b1d60 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.793+0000 7f0307b8b640 1 -- 172.21.15.107:0/913835125 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03001f46c0 con 0x7f03000b1d60 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.793+0000 7f02f27fc640 1 -- 172.21.15.107:0/913835125 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f02f4007400 con 0x7f03000b1d60 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.793+0000 7f02f27fc640 1 -- 172.21.15.107:0/913835125 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f02f4003ad0 con 0x7f03000b1d60 2024-04-25T08:31:11.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.794+0000 7f0307b8b640 1 -- 172.21.15.107:0/913835125 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0300068e80 con 0x7f03000b1d60 2024-04-25T08:31:11.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.796+0000 7f02f27fc640 1 -- 172.21.15.107:0/913835125 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f02f40035f0 con 0x7f03000b1d60 2024-04-25T08:31:11.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.868+0000 7f0307b8b640 1 -- 172.21.15.107:0/913835125 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8", "id": 0} v 0) v1 -- 0x7f030006c9e0 con 0x7f03000b1d60 2024-04-25T08:31:11.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.875+0000 7f02f27fc640 1 -- 172.21.15.107:0/913835125 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f02f4003d80 con 0x7f03000b1d60 2024-04-25T08:31:11.872 INFO:teuthology.orchestra.run.smithi107.stdout:0 2024-04-25T08:31:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.876+0000 7f0307b8b640 1 -- 172.21.15.107:0/913835125 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000b1d60 msgr2=0x7f03001575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.876+0000 7f0307b8b640 1 --2- 172.21.15.107:0/913835125 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000b1d60 0x7f03001575c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f02f400c3c0 tx=0x7f02f400c880 comp rx=0 tx=0).stop 2024-04-25T08:31:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.876+0000 7f0307b8b640 1 -- 172.21.15.107:0/913835125 shutdown_connections 2024-04-25T08:31:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.876+0000 7f0307b8b640 1 --2- 172.21.15.107:0/913835125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03000d00e0 0x7f030015a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.876+0000 7f0307b8b640 1 --2- 172.21.15.107:0/913835125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03000cd3b0 0x7f03000c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.876+0000 7f0307b8b640 1 --2- 172.21.15.107:0/913835125 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03000b1d60 0x7f03001575c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:11.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.876+0000 7f0307b8b640 1 -- 172.21.15.107:0/913835125 >> 172.21.15.107:0/913835125 conn(0x7f03000d06b0 msgr2=0x7f030015f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:11.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.876+0000 7f0307b8b640 1 -- 172.21.15.107:0/913835125 shutdown_connections 2024-04-25T08:31:11.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:11.876+0000 7f0307b8b640 1 -- 172.21.15.107:0/913835125 wait complete. 2024-04-25T08:31:11.883 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 45c7df16-5d20-4dfe-af4f-9178e69b2658 1 2024-04-25T08:31:12.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.022+0000 7f8d5b6cb640 1 Processor -- start 2024-04-25T08:31:12.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.023+0000 7f8d5b6cb640 1 -- start start 2024-04-25T08:31:12.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.024+0000 7f8d5b6cb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d540ccfa0 0x7f8d540cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.024+0000 7f8d5b6cb640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8d540ae050 0x7f8d540ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.024+0000 7f8d5b6cb640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d540cd8c0 con 0x7f8d540cdc90 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.024+0000 7f8d5b6cb640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d540cc4b0 con 0x7f8d540ae050 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.024+0000 7f8d5b6cb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540ad570 con 0x7f8d540ccfa0 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.024+0000 7f8d59467640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d540ccfa0 0x7f8d540cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.024+0000 7f8d59c68640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8d540ae050 0x7f8d540ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.024+0000 7f8d58c66640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f8d540cdc90 0x7f8d540cbc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:59446/0 (socket says 172.21.15.107:59446) 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.024+0000 7f8d58c66640 1 -- 172.21.15.107:0/122059959 learned_addr learned my addr 172.21.15.107:0/122059959 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.025+0000 7f8d59467640 1 -- 172.21.15.107:0/122059959 >> v2:172.21.15.107:3300/0 conn(0x7f8d540ae050 msgr2=0x7f8d540ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.025+0000 7f8d59467640 1 --2- 172.21.15.107:0/122059959 >> v2:172.21.15.107:3300/0 conn(0x7f8d540ae050 0x7f8d540ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.025+0000 7f8d59467640 1 -- 172.21.15.107:0/122059959 >> v1:172.21.15.107:6789/0 conn(0x7f8d540cdc90 legacy=0x7f8d540cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.025+0000 7f8d59467640 1 -- 172.21.15.107:0/122059959 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540ad850 con 0x7f8d540ccfa0 2024-04-25T08:31:12.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.025+0000 7f8d59c68640 1 --2- 172.21.15.107:0/122059959 >> v2:172.21.15.107:3300/0 conn(0x7f8d540ae050 0x7f8d540ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.025+0000 7f8d59467640 1 --2- 172.21.15.107:0/122059959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d540ccfa0 0x7f8d540cd380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8d440097a0 tx=0x7f8d4402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d883110da7b9a83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.025+0000 7f8d4bfff640 1 -- 172.21.15.107:0/122059959 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44037070 con 0x7f8d540ccfa0 2024-04-25T08:31:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.025+0000 7f8d4bfff640 1 -- 172.21.15.107:0/122059959 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d44032440 con 0x7f8d540ccfa0 2024-04-25T08:31:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.025+0000 7f8d4bfff640 1 -- 172.21.15.107:0/122059959 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44032810 con 0x7f8d540ccfa0 2024-04-25T08:31:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.026+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/122059959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d540ccfa0 msgr2=0x7f8d540cd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.026+0000 7f8d5b6cb640 1 --2- 172.21.15.107:0/122059959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d540ccfa0 0x7f8d540cd380 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8d440097a0 tx=0x7f8d4402a210 comp rx=0 tx=0).stop 2024-04-25T08:31:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.026+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/122059959 shutdown_connections 2024-04-25T08:31:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.026+0000 7f8d5b6cb640 1 --2- 172.21.15.107:0/122059959 >> v2:172.21.15.107:3300/0 conn(0x7f8d540ae050 0x7f8d540ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.026+0000 7f8d5b6cb640 1 --2- 172.21.15.107:0/122059959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d540ccfa0 0x7f8d540cd380 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.026+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/122059959 >> 172.21.15.107:0/122059959 conn(0x7f8d540d06b0 msgr2=0x7f8d540d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:12.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.026+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/122059959 shutdown_connections 2024-04-25T08:31:12.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.026+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/122059959 wait complete. 2024-04-25T08:31:12.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.028+0000 7f8d5b6cb640 1 Processor -- start 2024-04-25T08:31:12.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.028+0000 7f8d5b6cb640 1 -- start start 2024-04-25T08:31:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.029+0000 7f8d5b6cb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d540ae050 0x7f8d54150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.029+0000 7f8d5b6cb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d540ccfa0 0x7f8d540c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.029+0000 7f8d5b6cb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d540cdc90 0x7f8d540c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.029+0000 7f8d5b6cb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540aea20 con 0x7f8d540ccfa0 2024-04-25T08:31:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.029+0000 7f8d5b6cb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5415ad80 con 0x7f8d540ae050 2024-04-25T08:31:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.029+0000 7f8d5b6cb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d540aeba0 con 0x7f8d540cdc90 2024-04-25T08:31:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d59467640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d540ae050 0x7f8d54150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d59467640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d540ae050 0x7f8d54150ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35268/0 (socket says 172.21.15.107:35268) 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d59467640 1 -- 172.21.15.107:0/3917951540 learned_addr learned my addr 172.21.15.107:0/3917951540 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d59c68640 1 --2- 172.21.15.107:0/3917951540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d540cdc90 0x7f8d540c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d59467640 1 -- 172.21.15.107:0/3917951540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d540cdc90 msgr2=0x7f8d540c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d59467640 1 --2- 172.21.15.107:0/3917951540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d540cdc90 0x7f8d540c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d58c66640 1 --2- 172.21.15.107:0/3917951540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d540ccfa0 0x7f8d540c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d59467640 1 -- 172.21.15.107:0/3917951540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d540ccfa0 msgr2=0x7f8d540c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d59467640 1 --2- 172.21.15.107:0/3917951540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d540ccfa0 0x7f8d540c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d59467640 1 -- 172.21.15.107:0/3917951540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d540c9160 con 0x7f8d540ae050 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d58c66640 1 --2- 172.21.15.107:0/3917951540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d540ccfa0 0x7f8d540c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d59467640 1 --2- 172.21.15.107:0/3917951540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d540ae050 0x7f8d54150ae0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8d4402a800 tx=0x7f8d440046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d49ffb640 1 -- 172.21.15.107:0/3917951540 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44037040 con 0x7f8d540ae050 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d49ffb640 1 -- 172.21.15.107:0/3917951540 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d44030440 con 0x7f8d540ae050 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/3917951540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d540c9440 con 0x7f8d540ae050 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d49ffb640 1 -- 172.21.15.107:0/3917951540 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44030810 con 0x7f8d540ae050 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.030+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/3917951540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d540c9a00 con 0x7f8d540ae050 2024-04-25T08:31:12.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.031+0000 7f8d49ffb640 1 -- 172.21.15.107:0/3917951540 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8d44044020 con 0x7f8d540ae050 2024-04-25T08:31:12.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.031+0000 7f8d49ffb640 1 -- 172.21.15.107:0/3917951540 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f8d4403bb60 con 0x7f8d540ae050 2024-04-25T08:31:12.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.031+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/3917951540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d540675d0 con 0x7f8d540ae050 2024-04-25T08:31:12.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.033+0000 7f8d49ffb640 1 -- 172.21.15.107:0/3917951540 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8d4403bdb0 con 0x7f8d540ae050 2024-04-25T08:31:12.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.107+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/3917951540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "45c7df16-5d20-4dfe-af4f-9178e69b2658", "id": 1} v 0) v1 -- 0x7f8d54068e80 con 0x7f8d540ae050 2024-04-25T08:31:12.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.110+0000 7f8d49ffb640 1 -- 172.21.15.107:0/3917951540 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "45c7df16-5d20-4dfe-af4f-9178e69b2658", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8d4403b430 con 0x7f8d540ae050 2024-04-25T08:31:12.107 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-04-25T08:31:12.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.111+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/3917951540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d540ae050 msgr2=0x7f8d54150ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.111+0000 7f8d5b6cb640 1 --2- 172.21.15.107:0/3917951540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d540ae050 0x7f8d54150ae0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8d4402a800 tx=0x7f8d440046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.111+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/3917951540 shutdown_connections 2024-04-25T08:31:12.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.111+0000 7f8d5b6cb640 1 --2- 172.21.15.107:0/3917951540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d540cdc90 0x7f8d540c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.111+0000 7f8d5b6cb640 1 --2- 172.21.15.107:0/3917951540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d540ccfa0 0x7f8d540c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.111+0000 7f8d5b6cb640 1 --2- 172.21.15.107:0/3917951540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d540ae050 0x7f8d54150ae0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.111+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/3917951540 >> 172.21.15.107:0/3917951540 conn(0x7f8d540d06b0 msgr2=0x7f8d5406e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:12.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.111+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/3917951540 shutdown_connections 2024-04-25T08:31:12.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.111+0000 7f8d5b6cb640 1 -- 172.21.15.107:0/3917951540 wait complete. 2024-04-25T08:31:12.118 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 1a522f66-7604-437a-bca6-3fdec3ad936b 2 2024-04-25T08:31:12.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.218+0000 7fcd9c142640 1 Processor -- start 2024-04-25T08:31:12.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.218+0000 7fcd9c142640 1 -- start start 2024-04-25T08:31:12.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd9c142640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcd94157c20 0x7fcd940ce3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd9c142640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcd9415a510 0x7fcd9415a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd9c142640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9415b000 con 0x7fcd9406bbf0 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd9c142640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd940cd950 con 0x7fcd94157c20 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd9c142640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd940cdf10 con 0x7fcd9415a510 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd996dd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcd9415a510 0x7fcd9415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd996dd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcd9415a510 0x7fcd9415a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35278/0 (socket says 172.21.15.107:35278) 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd996dd640 1 -- 172.21.15.107:0/3821597451 learned_addr learned my addr 172.21.15.107:0/3821597451 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd99ede640 1 --2- 172.21.15.107:0/3821597451 >> v2:172.21.15.38:3300/0 conn(0x7fcd94157c20 0x7fcd940ce3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd996dd640 1 -- 172.21.15.107:0/3821597451 >> v2:172.21.15.38:3300/0 conn(0x7fcd94157c20 msgr2=0x7fcd940ce3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd996dd640 1 --2- 172.21.15.107:0/3821597451 >> v2:172.21.15.38:3300/0 conn(0x7fcd94157c20 0x7fcd940ce3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd996dd640 1 -- 172.21.15.107:0/3821597451 >> v1:172.21.15.38:6789/0 conn(0x7fcd9406bbf0 legacy=0x7fcd940ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.219+0000 7fcd996dd640 1 -- 172.21.15.107:0/3821597451 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9415b2e0 con 0x7fcd9415a510 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.220+0000 7fcd996dd640 1 --2- 172.21.15.107:0/3821597451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd9415a510 0x7fcd9415a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcd940727e0 tx=0x7fcd8402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22be4dbaecc98e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.220+0000 7fcd98edc640 1 -- 172.21.15.107:0/3821597451 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84037070 con 0x7fcd9415a510 2024-04-25T08:31:12.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.220+0000 7fcd98edc640 1 -- 172.21.15.107:0/3821597451 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd84032440 con 0x7fcd9415a510 2024-04-25T08:31:12.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.220+0000 7fcd98edc640 1 -- 172.21.15.107:0/3821597451 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84032810 con 0x7fcd9415a510 2024-04-25T08:31:12.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.221+0000 7fcd9c142640 1 -- 172.21.15.107:0/3821597451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd9415a510 msgr2=0x7fcd9415a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.221+0000 7fcd9c142640 1 --2- 172.21.15.107:0/3821597451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd9415a510 0x7fcd9415a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcd940727e0 tx=0x7fcd8402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.221+0000 7fcd9c142640 1 -- 172.21.15.107:0/3821597451 shutdown_connections 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.221+0000 7fcd9c142640 1 --2- 172.21.15.107:0/3821597451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd9415a510 0x7fcd9415a990 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.221+0000 7fcd9c142640 1 --2- 172.21.15.107:0/3821597451 >> v2:172.21.15.38:3300/0 conn(0x7fcd94157c20 0x7fcd940ce3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.221+0000 7fcd9c142640 1 -- 172.21.15.107:0/3821597451 >> 172.21.15.107:0/3821597451 conn(0x7fcd94071760 msgr2=0x7fcd94071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.221+0000 7fcd9c142640 1 -- 172.21.15.107:0/3821597451 shutdown_connections 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.221+0000 7fcd9c142640 1 -- 172.21.15.107:0/3821597451 wait complete. 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.222+0000 7fcd9c142640 1 Processor -- start 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.222+0000 7fcd9c142640 1 -- start start 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd9c142640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd9406bbf0 0x7fcd94149d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd9c142640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd94157c20 0x7fcd9414c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd9c142640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd9415a510 0x7fcd940ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd9c142640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940cf6f0 con 0x7fcd9406bbf0 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd9c142640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940cc370 con 0x7fcd94157c20 2024-04-25T08:31:12.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd9c142640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd940cf870 con 0x7fcd9415a510 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd996dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd94157c20 0x7fcd9414c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd996dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd94157c20 0x7fcd9414c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35280/0 (socket says 172.21.15.107:35280) 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd996dd640 1 -- 172.21.15.107:0/2518756849 learned_addr learned my addr 172.21.15.107:0/2518756849 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd9a6df640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd9415a510 0x7fcd940ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd99ede640 1 --2- 172.21.15.107:0/2518756849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd9406bbf0 0x7fcd94149d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd996dd640 1 -- 172.21.15.107:0/2518756849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd9415a510 msgr2=0x7fcd940ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd996dd640 1 --2- 172.21.15.107:0/2518756849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd9415a510 0x7fcd940ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd996dd640 1 -- 172.21.15.107:0/2518756849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd9406bbf0 msgr2=0x7fcd94149d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd996dd640 1 --2- 172.21.15.107:0/2518756849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd9406bbf0 0x7fcd94149d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd996dd640 1 -- 172.21.15.107:0/2518756849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd940ac990 con 0x7fcd94157c20 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.223+0000 7fcd996dd640 1 --2- 172.21.15.107:0/2518756849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd94157c20 0x7fcd9414c5a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcd8402a1e0 tx=0x7fcd840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.224+0000 7fcd8a7fc640 1 -- 172.21.15.107:0/2518756849 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84041020 con 0x7fcd94157c20 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.224+0000 7fcd8a7fc640 1 -- 172.21.15.107:0/2518756849 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd84030990 con 0x7fcd94157c20 2024-04-25T08:31:12.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.224+0000 7fcd9c142640 1 -- 172.21.15.107:0/2518756849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd940acc90 con 0x7fcd94157c20 2024-04-25T08:31:12.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.224+0000 7fcd8a7fc640 1 -- 172.21.15.107:0/2518756849 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84037040 con 0x7fcd94157c20 2024-04-25T08:31:12.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.224+0000 7fcd9c142640 1 -- 172.21.15.107:0/2518756849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd940ad4d0 con 0x7fcd94157c20 2024-04-25T08:31:12.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.224+0000 7fcd8a7fc640 1 -- 172.21.15.107:0/2518756849 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fcd84044020 con 0x7fcd94157c20 2024-04-25T08:31:12.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.224+0000 7fcd8a7fc640 1 -- 172.21.15.107:0/2518756849 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fcd84046820 con 0x7fcd94157c20 2024-04-25T08:31:12.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.225+0000 7fcd9c142640 1 -- 172.21.15.107:0/2518756849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9415d870 con 0x7fcd94157c20 2024-04-25T08:31:12.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.227+0000 7fcd8a7fc640 1 -- 172.21.15.107:0/2518756849 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fcd9415d870 con 0x7fcd94157c20 2024-04-25T08:31:12.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.299+0000 7fcd9c142640 1 -- 172.21.15.107:0/2518756849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1a522f66-7604-437a-bca6-3fdec3ad936b", "id": 2} v 0) v1 -- 0x7fcd94067bc0 con 0x7fcd94157c20 2024-04-25T08:31:12.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.302+0000 7fcd8a7fc640 1 -- 172.21.15.107:0/2518756849 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1a522f66-7604-437a-bca6-3fdec3ad936b", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcd94067bc0 con 0x7fcd94157c20 2024-04-25T08:31:12.299 INFO:teuthology.orchestra.run.smithi107.stdout:2 2024-04-25T08:31:12.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.303+0000 7fcd9c142640 1 -- 172.21.15.107:0/2518756849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd94157c20 msgr2=0x7fcd9414c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.303+0000 7fcd9c142640 1 --2- 172.21.15.107:0/2518756849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd94157c20 0x7fcd9414c5a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcd8402a1e0 tx=0x7fcd840046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.303+0000 7fcd9c142640 1 -- 172.21.15.107:0/2518756849 shutdown_connections 2024-04-25T08:31:12.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.303+0000 7fcd9c142640 1 --2- 172.21.15.107:0/2518756849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd9415a510 0x7fcd940ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.303+0000 7fcd9c142640 1 --2- 172.21.15.107:0/2518756849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd94157c20 0x7fcd9414c5a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.303+0000 7fcd9c142640 1 --2- 172.21.15.107:0/2518756849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd9406bbf0 0x7fcd94149d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.303+0000 7fcd9c142640 1 -- 172.21.15.107:0/2518756849 >> 172.21.15.107:0/2518756849 conn(0x7fcd94071760 msgr2=0x7fcd940cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:12.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.303+0000 7fcd9c142640 1 -- 172.21.15.107:0/2518756849 shutdown_connections 2024-04-25T08:31:12.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.303+0000 7fcd9c142640 1 -- 172.21.15.107:0/2518756849 wait complete. 2024-04-25T08:31:12.310 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 184041f7-fe13-41f4-b876-93aef9649d6d 3 2024-04-25T08:31:12.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.411+0000 7f8a372f5640 1 Processor -- start 2024-04-25T08:31:12.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.411+0000 7f8a372f5640 1 -- start start 2024-04-25T08:31:12.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a372f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8a3006abe0 0x7f8a3006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a372f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a301635e0 0x7f8a301659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a372f5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3006b500 con 0x7f8a300cc2c0 2024-04-25T08:31:12.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a372f5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3015c1b0 con 0x7f8a3006abe0 2024-04-25T08:31:12.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a372f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a300ccf30 con 0x7f8a301635e0 2024-04-25T08:31:12.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a35091640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8a3006abe0 0x7f8a3006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a35892640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a301635e0 0x7f8a301659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a34890640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f8a300cc2c0 0x7f8a300cc740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:59452/0 (socket says 172.21.15.107:59452) 2024-04-25T08:31:12.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a34890640 1 -- 172.21.15.107:0/498923412 learned_addr learned my addr 172.21.15.107:0/498923412 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-04-25T08:31:12.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a35892640 1 -- 172.21.15.107:0/498923412 >> v2:172.21.15.107:3300/0 conn(0x7f8a3006abe0 msgr2=0x7f8a3006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:12.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a35892640 1 --2- 172.21.15.107:0/498923412 >> v2:172.21.15.107:3300/0 conn(0x7f8a3006abe0 0x7f8a3006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a35892640 1 -- 172.21.15.107:0/498923412 >> v1:172.21.15.107:6789/0 conn(0x7f8a300cc2c0 legacy=0x7f8a300cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a27fff640 1 -- 172.21.15.107:0/498923412 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122553854 0 0) 0x7f8a3006b500 con 0x7f8a300cc2c0 2024-04-25T08:31:12.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a35892640 1 -- 172.21.15.107:0/498923412 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a300cd210 con 0x7f8a301635e0 2024-04-25T08:31:12.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.412+0000 7f8a35091640 1 --2- 172.21.15.107:0/498923412 >> v2:172.21.15.107:3300/0 conn(0x7f8a3006abe0 0x7f8a3006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.413+0000 7f8a35892640 1 --2- 172.21.15.107:0/498923412 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a301635e0 0x7f8a301659d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a28009d90 tx=0x7f8a2802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab480f0f628c05dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.413+0000 7f8a27fff640 1 -- 172.21.15.107:0/498923412 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28031600 con 0x7f8a301635e0 2024-04-25T08:31:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.413+0000 7f8a27fff640 1 -- 172.21.15.107:0/498923412 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a280317a0 con 0x7f8a301635e0 2024-04-25T08:31:12.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.413+0000 7f8a27fff640 1 -- 172.21.15.107:0/498923412 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28031b70 con 0x7f8a301635e0 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.414+0000 7f8a372f5640 1 -- 172.21.15.107:0/498923412 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a301635e0 msgr2=0x7f8a301659d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.414+0000 7f8a372f5640 1 --2- 172.21.15.107:0/498923412 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a301635e0 0x7f8a301659d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8a28009d90 tx=0x7f8a2802a5d0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.414+0000 7f8a372f5640 1 -- 172.21.15.107:0/498923412 shutdown_connections 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.414+0000 7f8a372f5640 1 --2- 172.21.15.107:0/498923412 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a301635e0 0x7f8a301659d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.414+0000 7f8a372f5640 1 --2- 172.21.15.107:0/498923412 >> v2:172.21.15.107:3300/0 conn(0x7f8a3006abe0 0x7f8a3006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.414+0000 7f8a372f5640 1 -- 172.21.15.107:0/498923412 >> 172.21.15.107:0/498923412 conn(0x7f8a300d06b0 msgr2=0x7f8a300d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.414+0000 7f8a372f5640 1 -- 172.21.15.107:0/498923412 shutdown_connections 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.414+0000 7f8a372f5640 1 -- 172.21.15.107:0/498923412 wait complete. 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.415+0000 7f8a372f5640 1 Processor -- start 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.415+0000 7f8a372f5640 1 -- start start 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a372f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a3006abe0 0x7f8a300cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a372f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a300cc2c0 0x7f8a30161d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a372f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a301635e0 0x7f8a30155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a372f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a30159790 con 0x7f8a301635e0 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a372f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a300cd7d0 con 0x7f8a300cc2c0 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a372f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a30159910 con 0x7f8a3006abe0 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a35091640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a3006abe0 0x7f8a300cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a35091640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a3006abe0 0x7f8a300cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:51268/0 (socket says 172.21.15.107:51268) 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a35091640 1 -- 172.21.15.107:0/1448386259 learned_addr learned my addr 172.21.15.107:0/1448386259 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a34890640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a300cc2c0 0x7f8a30161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a35892640 1 --2- 172.21.15.107:0/1448386259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a301635e0 0x7f8a30155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a35091640 1 -- 172.21.15.107:0/1448386259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a300cc2c0 msgr2=0x7f8a30161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a35091640 1 --2- 172.21.15.107:0/1448386259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a300cc2c0 0x7f8a30161d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a35091640 1 -- 172.21.15.107:0/1448386259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a301635e0 msgr2=0x7f8a30155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a35091640 1 --2- 172.21.15.107:0/1448386259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a301635e0 0x7f8a30155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a35091640 1 -- 172.21.15.107:0/1448386259 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a300cff60 con 0x7f8a3006abe0 2024-04-25T08:31:12.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a35892640 1 --2- 172.21.15.107:0/1448386259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a301635e0 0x7f8a30155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:12.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a35091640 1 --2- 172.21.15.107:0/1448386259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a3006abe0 0x7f8a300cf8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00ebe0 tx=0x7f8a2c00c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:12.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a25ffb640 1 -- 172.21.15.107:0/1448386259 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c017070 con 0x7f8a3006abe0 2024-04-25T08:31:12.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a25ffb640 1 -- 172.21.15.107:0/1448386259 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a2c00cc30 con 0x7f8a3006abe0 2024-04-25T08:31:12.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a25ffb640 1 -- 172.21.15.107:0/1448386259 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c013470 con 0x7f8a3006abe0 2024-04-25T08:31:12.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.416+0000 7f8a372f5640 1 -- 172.21.15.107:0/1448386259 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a300c78b0 con 0x7f8a3006abe0 2024-04-25T08:31:12.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.417+0000 7f8a372f5640 1 -- 172.21.15.107:0/1448386259 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a300c7e30 con 0x7f8a3006abe0 2024-04-25T08:31:12.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.417+0000 7f8a25ffb640 1 -- 172.21.15.107:0/1448386259 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8a2c01a070 con 0x7f8a3006abe0 2024-04-25T08:31:12.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.417+0000 7f8a25ffb640 1 -- 172.21.15.107:0/1448386259 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f8a2c006020 con 0x7f8a3006abe0 2024-04-25T08:31:12.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.417+0000 7f8a372f5640 1 -- 172.21.15.107:0/1448386259 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a30068e80 con 0x7f8a3006abe0 2024-04-25T08:31:12.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.419+0000 7f8a25ffb640 1 -- 172.21.15.107:0/1448386259 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8a2c013a20 con 0x7f8a3006abe0 2024-04-25T08:31:12.420 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:12.421+0000 7f1f8a212040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T08:31:12.475 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:12.478+0000 7ff975c12040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T08:31:12.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.492+0000 7f8a372f5640 1 -- 172.21.15.107:0/1448386259 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "184041f7-fe13-41f4-b876-93aef9649d6d", "id": 3} v 0) v1 -- 0x7f8a30159170 con 0x7f8a3006abe0 2024-04-25T08:31:12.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.496+0000 7f8a25ffb640 1 -- 172.21.15.107:0/1448386259 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "184041f7-fe13-41f4-b876-93aef9649d6d", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8a2c013c60 con 0x7f8a3006abe0 2024-04-25T08:31:12.493 INFO:teuthology.orchestra.run.smithi107.stdout:3 2024-04-25T08:31:12.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.497+0000 7f8a372f5640 1 -- 172.21.15.107:0/1448386259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a3006abe0 msgr2=0x7f8a300cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.497+0000 7f8a372f5640 1 --2- 172.21.15.107:0/1448386259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a3006abe0 0x7f8a300cf8d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c00ebe0 tx=0x7f8a2c00c520 comp rx=0 tx=0).stop 2024-04-25T08:31:12.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.497+0000 7f8a372f5640 1 -- 172.21.15.107:0/1448386259 shutdown_connections 2024-04-25T08:31:12.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.497+0000 7f8a372f5640 1 --2- 172.21.15.107:0/1448386259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a301635e0 0x7f8a30155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.497+0000 7f8a372f5640 1 --2- 172.21.15.107:0/1448386259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a300cc2c0 0x7f8a30161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.497+0000 7f8a372f5640 1 --2- 172.21.15.107:0/1448386259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a3006abe0 0x7f8a300cf8d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.497+0000 7f8a372f5640 1 -- 172.21.15.107:0/1448386259 >> 172.21.15.107:0/1448386259 conn(0x7f8a300d06b0 msgr2=0x7f8a30160870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:12.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.497+0000 7f8a372f5640 1 -- 172.21.15.107:0/1448386259 shutdown_connections 2024-04-25T08:31:12.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.497+0000 7f8a372f5640 1 -- 172.21.15.107:0/1448386259 wait complete. 2024-04-25T08:31:12.504 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436 4 2024-04-25T08:31:12.510 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:12.513+0000 7f1f8a212040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T08:31:12.555 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:12.558+0000 7f1f8a212040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T08:31:12.566 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:12.570+0000 7ff975c12040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T08:31:12.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.605+0000 7f12d629a640 1 Processor -- start 2024-04-25T08:31:12.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.605+0000 7f12d629a640 1 -- start start 2024-04-25T08:31:12.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.605+0000 7f12d629a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d015dc80 0x7f12d00cec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.605+0000 7f12d629a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d0072750 con 0x7f12d006f710 2024-04-25T08:31:12.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.605+0000 7f12d629a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d006e670 con 0x7f12d006df90 2024-04-25T08:31:12.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.605+0000 7f12d629a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d006e410 con 0x7f12d015dc80 2024-04-25T08:31:12.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.605+0000 7f12cffff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d015dc80 0x7f12d00cec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.606+0000 7f12cf7fe640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f12d006df90 0x7f12d00ca3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.107:59456/0 (socket says 172.21.15.107:59456) 2024-04-25T08:31:12.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.606+0000 7f12cf7fe640 1 -- 172.21.15.107:0/2158356950 learned_addr learned my addr 172.21.15.107:0/2158356950 (peer_addr_for_me v1:172.21.15.107:0/0) 2024-04-25T08:31:12.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.606+0000 7f12ceffd640 1 -- 172.21.15.107:0/2158356950 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 719091465 0 0) 0x7f12d006e670 con 0x7f12d006df90 2024-04-25T08:31:12.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.606+0000 7f12ceffd640 1 -- 172.21.15.107:0/2158356950 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b8003520 con 0x7f12d006df90 2024-04-25T08:31:12.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.606+0000 7f12cffff640 1 -- 172.21.15.107:0/2158356950 >> v1:172.21.15.107:6789/0 conn(0x7f12d006df90 legacy=0x7f12d00ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.606+0000 7f12cffff640 1 -- 172.21.15.107:0/2158356950 >> v1:172.21.15.38:6789/0 conn(0x7f12d006f710 legacy=0x7f12d006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.606+0000 7f12cffff640 1 -- 172.21.15.107:0/2158356950 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d006e950 con 0x7f12d015dc80 2024-04-25T08:31:12.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.607+0000 7f12cffff640 1 --2- 172.21.15.107:0/2158356950 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d015dc80 0x7f12d00cec50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f12c40097d0 tx=0x7f12c402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=324b789ca889a9a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:12.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.607+0000 7f12ceffd640 1 -- 172.21.15.107:0/2158356950 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4037070 con 0x7f12d015dc80 2024-04-25T08:31:12.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.607+0000 7f12ceffd640 1 -- 172.21.15.107:0/2158356950 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c4032440 con 0x7f12d015dc80 2024-04-25T08:31:12.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.607+0000 7f12ceffd640 1 -- 172.21.15.107:0/2158356950 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4032810 con 0x7f12d015dc80 2024-04-25T08:31:12.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.607+0000 7f12d629a640 1 -- 172.21.15.107:0/2158356950 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d015dc80 msgr2=0x7f12d00cec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.607+0000 7f12d629a640 1 --2- 172.21.15.107:0/2158356950 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d015dc80 0x7f12d00cec50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f12c40097d0 tx=0x7f12c402a210 comp rx=0 tx=0).stop 2024-04-25T08:31:12.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.607+0000 7f12d629a640 1 -- 172.21.15.107:0/2158356950 shutdown_connections 2024-04-25T08:31:12.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.607+0000 7f12d629a640 1 --2- 172.21.15.107:0/2158356950 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d015dc80 0x7f12d00cec50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.607+0000 7f12d629a640 1 -- 172.21.15.107:0/2158356950 >> 172.21.15.107:0/2158356950 conn(0x7f12d0071760 msgr2=0x7f12d0071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:12.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.607+0000 7f12d629a640 1 -- 172.21.15.107:0/2158356950 shutdown_connections 2024-04-25T08:31:12.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.607+0000 7f12d629a640 1 -- 172.21.15.107:0/2158356950 wait complete. 2024-04-25T08:31:12.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.608+0000 7f12d629a640 1 Processor -- start 2024-04-25T08:31:12.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.609+0000 7f12d629a640 1 -- start start 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.609+0000 7f12d629a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12d006df90 0x7f12d00c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.609+0000 7f12d629a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d006f710 0x7f12d00ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.609+0000 7f12d629a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d015dc80 0x7f12d00c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.609+0000 7f12d629a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d006f1f0 con 0x7f12d015dc80 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.609+0000 7f12d629a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d006ef10 con 0x7f12d006df90 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.609+0000 7f12d629a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00caae0 con 0x7f12d006f710 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.609+0000 7f12cf7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d006f710 0x7f12d00ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12cffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12d006df90 0x7f12d00c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12cf7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d006f710 0x7f12d00ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:51282/0 (socket says 172.21.15.107:51282) 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12cf7fe640 1 -- 172.21.15.107:0/703090378 learned_addr learned my addr 172.21.15.107:0/703090378 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12cffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12d006df90 0x7f12d00c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35304/0 (socket says 172.21.15.107:35304) 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12d4837640 1 --2- 172.21.15.107:0/703090378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d015dc80 0x7f12d00c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12cffff640 1 -- 172.21.15.107:0/703090378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d006f710 msgr2=0x7f12d00ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12cffff640 1 --2- 172.21.15.107:0/703090378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d006f710 0x7f12d00ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12cffff640 1 -- 172.21.15.107:0/703090378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d015dc80 msgr2=0x7f12d00c8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12cffff640 1 --2- 172.21.15.107:0/703090378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d015dc80 0x7f12d00c8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12cffff640 1 -- 172.21.15.107:0/703090378 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d0151d10 con 0x7f12d006df90 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12cf7fe640 1 --2- 172.21.15.107:0/703090378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d006f710 0x7f12d00ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12cffff640 1 --2- 172.21.15.107:0/703090378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12d006df90 0x7f12d00c7860 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f12c4031000 tx=0x7f12c40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12ccff9640 1 -- 172.21.15.107:0/703090378 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4041020 con 0x7f12d006df90 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12ccff9640 1 -- 172.21.15.107:0/703090378 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c4030740 con 0x7f12d006df90 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12ccff9640 1 -- 172.21.15.107:0/703090378 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4037040 con 0x7f12d006df90 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12d629a640 1 -- 172.21.15.107:0/703090378 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d00c8120 con 0x7f12d006df90 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12d629a640 1 -- 172.21.15.107:0/703090378 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d00ac990 con 0x7f12d006df90 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12ccff9640 1 -- 172.21.15.107:0/703090378 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f12c4044020 con 0x7f12d006df90 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.610+0000 7f12ccff9640 1 -- 172.21.15.107:0/703090378 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f12c403db50 con 0x7f12d006df90 2024-04-25T08:31:12.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.611+0000 7f12d629a640 1 -- 172.21.15.107:0/703090378 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d0068e80 con 0x7f12d006df90 2024-04-25T08:31:12.611 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:12.614+0000 7ff975c12040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T08:31:12.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.615+0000 7f12ccff9640 1 -- 172.21.15.107:0/703090378 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f12c404b030 con 0x7f12d006df90 2024-04-25T08:31:12.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.687+0000 7f12d629a640 1 -- 172.21.15.107:0/703090378 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436", "id": 4} v 0) v1 -- 0x7f12d00b5d30 con 0x7f12d006df90 2024-04-25T08:31:12.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.694+0000 7f12ccff9640 1 -- 172.21.15.107:0/703090378 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f12d0151d10 con 0x7f12d006df90 2024-04-25T08:31:12.691 INFO:teuthology.orchestra.run.smithi107.stdout:4 2024-04-25T08:31:12.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.695+0000 7f12d629a640 1 -- 172.21.15.107:0/703090378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12d006df90 msgr2=0x7f12d00c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.695+0000 7f12d629a640 1 --2- 172.21.15.107:0/703090378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12d006df90 0x7f12d00c7860 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f12c4031000 tx=0x7f12c40046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.695+0000 7f12d629a640 1 -- 172.21.15.107:0/703090378 shutdown_connections 2024-04-25T08:31:12.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.695+0000 7f12d629a640 1 --2- 172.21.15.107:0/703090378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12d015dc80 0x7f12d00c8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.695+0000 7f12d629a640 1 --2- 172.21.15.107:0/703090378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f12d006f710 0x7f12d00ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.695+0000 7f12d629a640 1 --2- 172.21.15.107:0/703090378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12d006df90 0x7f12d00c7860 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.695+0000 7f12d629a640 1 -- 172.21.15.107:0/703090378 >> 172.21.15.107:0/703090378 conn(0x7f12d0071760 msgr2=0x7f12d00af440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:12.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.695+0000 7f12d629a640 1 -- 172.21.15.107:0/703090378 shutdown_connections 2024-04-25T08:31:12.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.695+0000 7f12d629a640 1 -- 172.21.15.107:0/703090378 wait complete. 2024-04-25T08:31:12.703 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 9d2fe8a8-ca71-408d-96d9-aaebe0863faf 5 2024-04-25T08:31:12.732 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:12.735+0000 7f1f8a212040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T08:31:12.782 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:12.786+0000 7f1f8a212040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T08:31:12.783 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:12.786+0000 7ff975c12040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T08:31:12.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.832+0000 7f7afd94c640 1 Processor -- start 2024-04-25T08:31:12.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.832+0000 7f7afd94c640 1 -- start start 2024-04-25T08:31:12.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.832+0000 7f7afd94c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7af806abe0 0x7f7af806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.832+0000 7f7afd94c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7af80cc2c0 0x7f7af80cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.832+0000 7f7afd94c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 0x7f7af815f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.832+0000 7f7afd94c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af806b500 con 0x7f7af80cc2c0 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.832+0000 7f7afd94c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af80ccc80 con 0x7f7af806abe0 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.832+0000 7f7afd94c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af815c1b0 con 0x7f7af815f490 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af6ffd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7af806abe0 0x7f7af806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af77fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 0x7f7af815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af77fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 0x7f7af815f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:51292/0 (socket says 172.21.15.107:51292) 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af6ffd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7af806abe0 0x7f7af806afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35314/0 (socket says 172.21.15.107:35314) 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af77fe640 1 -- 172.21.15.107:0/3553351470 learned_addr learned my addr 172.21.15.107:0/3553351470 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af67fc640 1 --2- 172.21.15.107:0/3553351470 >> v2:172.21.15.38:3300/0 conn(0x7f7af80cc2c0 0x7f7af80cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af77fe640 1 -- 172.21.15.107:0/3553351470 >> v2:172.21.15.107:3300/0 conn(0x7f7af806abe0 msgr2=0x7f7af806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:12.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af77fe640 1 --2- 172.21.15.107:0/3553351470 >> v2:172.21.15.107:3300/0 conn(0x7f7af806abe0 0x7f7af806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af77fe640 1 -- 172.21.15.107:0/3553351470 >> v2:172.21.15.38:3300/0 conn(0x7f7af80cc2c0 msgr2=0x7f7af80cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:12.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af77fe640 1 --2- 172.21.15.107:0/3553351470 >> v2:172.21.15.38:3300/0 conn(0x7f7af80cc2c0 0x7f7af80cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af77fe640 1 -- 172.21.15.107:0/3553351470 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af80ccf30 con 0x7f7af815f490 2024-04-25T08:31:12.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af6ffd640 1 --2- 172.21.15.107:0/3553351470 >> v2:172.21.15.107:3300/0 conn(0x7f7af806abe0 0x7f7af806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:12.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.833+0000 7f7af67fc640 1 --2- 172.21.15.107:0/3553351470 >> v2:172.21.15.38:3300/0 conn(0x7f7af80cc2c0 0x7f7af80cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:12.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7af77fe640 1 --2- 172.21.15.107:0/3553351470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 0x7f7af815f920 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7af80730b0 tx=0x7f7af002a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d63032b29409a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:12.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7af5ffb640 1 -- 172.21.15.107:0/3553351470 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0036070 con 0x7f7af815f490 2024-04-25T08:31:12.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7af5ffb640 1 -- 172.21.15.107:0/3553351470 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af0031600 con 0x7f7af815f490 2024-04-25T08:31:12.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7af5ffb640 1 -- 172.21.15.107:0/3553351470 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af00319d0 con 0x7f7af815f490 2024-04-25T08:31:12.833 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:12.836+0000 7ff975c12040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T08:31:12.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7afd94c640 1 -- 172.21.15.107:0/3553351470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 msgr2=0x7f7af815f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7afd94c640 1 --2- 172.21.15.107:0/3553351470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 0x7f7af815f920 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7af80730b0 tx=0x7f7af002a610 comp rx=0 tx=0).stop 2024-04-25T08:31:12.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7afd94c640 1 -- 172.21.15.107:0/3553351470 shutdown_connections 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7afd94c640 1 --2- 172.21.15.107:0/3553351470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 0x7f7af815f920 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7afd94c640 1 --2- 172.21.15.107:0/3553351470 >> v2:172.21.15.38:3300/0 conn(0x7f7af80cc2c0 0x7f7af80cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7afd94c640 1 --2- 172.21.15.107:0/3553351470 >> v2:172.21.15.107:3300/0 conn(0x7f7af806abe0 0x7f7af806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7afd94c640 1 -- 172.21.15.107:0/3553351470 >> 172.21.15.107:0/3553351470 conn(0x7f7af80d06b0 msgr2=0x7f7af80d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7afd94c640 1 -- 172.21.15.107:0/3553351470 shutdown_connections 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.834+0000 7f7afd94c640 1 -- 172.21.15.107:0/3553351470 wait complete. 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.836+0000 7f7afd94c640 1 Processor -- start 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.836+0000 7f7afd94c640 1 -- start start 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.836+0000 7f7afd94c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af806abe0 0x7f7af80c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.836+0000 7f7afd94c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af80cc2c0 0x7f7af81f0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.836+0000 7f7afd94c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 0x7f7af81f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.836+0000 7f7afd94c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80cd7d0 con 0x7f7af80cc2c0 2024-04-25T08:31:12.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.836+0000 7f7afd94c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80cd4f0 con 0x7f7af806abe0 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.836+0000 7f7afd94c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af8159790 con 0x7f7af815f490 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7af6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af806abe0 0x7f7af80c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7af77fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 0x7f7af81f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7af6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af806abe0 0x7f7af80c66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35318/0 (socket says 172.21.15.107:35318) 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7af77fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 0x7f7af81f08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:51300/0 (socket says 172.21.15.107:51300) 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7af6ffd640 1 -- 172.21.15.107:0/2254613891 learned_addr learned my addr 172.21.15.107:0/2254613891 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7af67fc640 1 --2- 172.21.15.107:0/2254613891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af80cc2c0 0x7f7af81f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7af77fe640 1 -- 172.21.15.107:0/2254613891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af806abe0 msgr2=0x7f7af80c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7af77fe640 1 --2- 172.21.15.107:0/2254613891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af806abe0 0x7f7af80c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7af77fe640 1 -- 172.21.15.107:0/2254613891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af80cc2c0 msgr2=0x7f7af81f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7af77fe640 1 --2- 172.21.15.107:0/2254613891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af80cc2c0 0x7f7af81f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7af77fe640 1 -- 172.21.15.107:0/2254613891 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af80c6c00 con 0x7f7af815f490 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7af77fe640 1 --2- 172.21.15.107:0/2254613891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 0x7f7af81f08c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7af0030000 tx=0x7f7af0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7adffff640 1 -- 172.21.15.107:0/2254613891 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0036050 con 0x7f7af815f490 2024-04-25T08:31:12.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7adffff640 1 -- 172.21.15.107:0/2254613891 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af0042070 con 0x7f7af815f490 2024-04-25T08:31:12.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7afd94c640 1 -- 172.21.15.107:0/2254613891 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af80ca1d0 con 0x7f7af815f490 2024-04-25T08:31:12.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7adffff640 1 -- 172.21.15.107:0/2254613891 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0009510 con 0x7f7af815f490 2024-04-25T08:31:12.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7afd94c640 1 -- 172.21.15.107:0/2254613891 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af80ca760 con 0x7f7af815f490 2024-04-25T08:31:12.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7adffff640 1 -- 172.21.15.107:0/2254613891 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7af0007bd0 con 0x7f7af815f490 2024-04-25T08:31:12.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.837+0000 7f7adffff640 1 -- 172.21.15.107:0/2254613891 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f7af0047aa0 con 0x7f7af815f490 2024-04-25T08:31:12.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.838+0000 7f7afd94c640 1 -- 172.21.15.107:0/2254613891 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af8068e80 con 0x7f7af815f490 2024-04-25T08:31:12.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.840+0000 7f7adffff640 1 -- 172.21.15.107:0/2254613891 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7af0047460 con 0x7f7af815f490 2024-04-25T08:31:12.839 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:12.842+0000 7f1f8a212040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T08:31:12.887 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:12.891+0000 7ff975c12040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T08:31:12.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.912+0000 7f7afd94c640 1 -- 172.21.15.107:0/2254613891 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9d2fe8a8-ca71-408d-96d9-aaebe0863faf", "id": 5} v 0) v1 -- 0x7f7af815aea0 con 0x7f7af815f490 2024-04-25T08:31:12.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.919+0000 7f7adffff640 1 -- 172.21.15.107:0/2254613891 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9d2fe8a8-ca71-408d-96d9-aaebe0863faf", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7af0047640 con 0x7f7af815f490 2024-04-25T08:31:12.916 INFO:teuthology.orchestra.run.smithi107.stdout:5 2024-04-25T08:31:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.920+0000 7f7afd94c640 1 -- 172.21.15.107:0/2254613891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 msgr2=0x7f7af81f08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:12.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.920+0000 7f7afd94c640 1 --2- 172.21.15.107:0/2254613891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 0x7f7af81f08c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7af0030000 tx=0x7f7af0004a40 comp rx=0 tx=0).stop 2024-04-25T08:31:12.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.920+0000 7f7afd94c640 1 -- 172.21.15.107:0/2254613891 shutdown_connections 2024-04-25T08:31:12.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.920+0000 7f7afd94c640 1 --2- 172.21.15.107:0/2254613891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7af815f490 0x7f7af81f08c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.920+0000 7f7afd94c640 1 --2- 172.21.15.107:0/2254613891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7af80cc2c0 0x7f7af81f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.920+0000 7f7afd94c640 1 --2- 172.21.15.107:0/2254613891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7af806abe0 0x7f7af80c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:12.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.920+0000 7f7afd94c640 1 -- 172.21.15.107:0/2254613891 >> 172.21.15.107:0/2254613891 conn(0x7f7af80d06b0 msgr2=0x7f7af815d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:12.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.920+0000 7f7afd94c640 1 -- 172.21.15.107:0/2254613891 shutdown_connections 2024-04-25T08:31:12.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:12.920+0000 7f7afd94c640 1 -- 172.21.15.107:0/2254613891 wait complete. 2024-04-25T08:31:12.927 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new 9da74044-e310-4263-a10a-2210377872e9 6 2024-04-25T08:31:12.956 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:12.960+0000 7f1f8a212040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:31:13.004 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:13.007+0000 7ff975c12040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:31:13.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.027+0000 7ff9a0a07640 1 Processor -- start 2024-04-25T08:31:13.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.027+0000 7ff9a0a07640 1 -- start start 2024-04-25T08:31:13.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff9a0a07640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff99c15b990 0x7ff99c0cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff9a0a07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c0d1d20 0x7ff99c0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff9a0a07640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff99c15fd00 con 0x7ff99c15f5f0 2024-04-25T08:31:13.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff9a0a07640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99c160580 con 0x7ff99c15b990 2024-04-25T08:31:13.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff9a0a07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c15fa30 con 0x7ff99c0d1d20 2024-04-25T08:31:13.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff99ad9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c0d1d20 0x7ff99c0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff99ad9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c0d1d20 0x7ff99c0b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:51312/0 (socket says 172.21.15.107:51312) 2024-04-25T08:31:13.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff99ad9d640 1 -- 172.21.15.107:0/1618703083 learned_addr learned my addr 172.21.15.107:0/1618703083 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:13.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff99a59c640 1 --2- 172.21.15.107:0/1618703083 >> v2:172.21.15.38:3300/0 conn(0x7ff99c15b990 0x7ff99c0cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff99ad9d640 1 -- 172.21.15.107:0/1618703083 >> v2:172.21.15.38:3300/0 conn(0x7ff99c15b990 msgr2=0x7ff99c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:13.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff99ad9d640 1 --2- 172.21.15.107:0/1618703083 >> v2:172.21.15.38:3300/0 conn(0x7ff99c15b990 0x7ff99c0cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff99ad9d640 1 -- 172.21.15.107:0/1618703083 >> v1:172.21.15.38:6789/0 conn(0x7ff99c15f5f0 legacy=0x7ff99c0d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.028+0000 7ff99ad9d640 1 -- 172.21.15.107:0/1618703083 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c15ffc0 con 0x7ff99c0d1d20 2024-04-25T08:31:13.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.029+0000 7ff99ad9d640 1 --2- 172.21.15.107:0/1618703083 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c0d1d20 0x7ff99c0b1fe0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff994009e50 tx=0x7ff99402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3740eefbb310c4d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:13.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.029+0000 7ff999d9b640 1 -- 172.21.15.107:0/1618703083 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994036070 con 0x7ff99c0d1d20 2024-04-25T08:31:13.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.029+0000 7ff999d9b640 1 -- 172.21.15.107:0/1618703083 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff994031600 con 0x7ff99c0d1d20 2024-04-25T08:31:13.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.029+0000 7ff999d9b640 1 -- 172.21.15.107:0/1618703083 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9940319d0 con 0x7ff99c0d1d20 2024-04-25T08:31:13.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.030+0000 7ff9a0a07640 1 -- 172.21.15.107:0/1618703083 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c0d1d20 msgr2=0x7ff99c0b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.030+0000 7ff9a0a07640 1 --2- 172.21.15.107:0/1618703083 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c0d1d20 0x7ff99c0b1fe0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff994009e50 tx=0x7ff99402a680 comp rx=0 tx=0).stop 2024-04-25T08:31:13.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.030+0000 7ff9a0a07640 1 -- 172.21.15.107:0/1618703083 shutdown_connections 2024-04-25T08:31:13.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.030+0000 7ff9a0a07640 1 --2- 172.21.15.107:0/1618703083 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c0d1d20 0x7ff99c0b1fe0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.030+0000 7ff9a0a07640 1 --2- 172.21.15.107:0/1618703083 >> v2:172.21.15.38:3300/0 conn(0x7ff99c15b990 0x7ff99c0cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.030+0000 7ff9a0a07640 1 -- 172.21.15.107:0/1618703083 >> 172.21.15.107:0/1618703083 conn(0x7ff99c08b000 msgr2=0x7ff99c08b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:13.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.030+0000 7ff9a0a07640 1 -- 172.21.15.107:0/1618703083 shutdown_connections 2024-04-25T08:31:13.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.030+0000 7ff9a0a07640 1 -- 172.21.15.107:0/1618703083 wait complete. 2024-04-25T08:31:13.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.031+0000 7ff9a0a07640 1 Processor -- start 2024-04-25T08:31:13.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.031+0000 7ff9a0a07640 1 -- start start 2024-04-25T08:31:13.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff9a0a07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff99c0d1d20 0x7ff99c15e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff9a0a07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff99c15b990 0x7ff99c0c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff9a0a07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c15f5f0 0x7ff99c144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff9a0a07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c15bd70 con 0x7ff99c15b990 2024-04-25T08:31:13.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff9a0a07640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c0b1ac0 con 0x7ff99c0d1d20 2024-04-25T08:31:13.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff9a0a07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c15bef0 con 0x7ff99c15f5f0 2024-04-25T08:31:13.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff99a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff99c0d1d20 0x7ff99c15e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff99ad9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c15f5f0 0x7ff99c144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff99a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff99c0d1d20 0x7ff99c15e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35330/0 (socket says 172.21.15.107:35330) 2024-04-25T08:31:13.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff99a59c640 1 -- 172.21.15.107:0/503002487 learned_addr learned my addr 172.21.15.107:0/503002487 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:13.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff993fff640 1 --2- 172.21.15.107:0/503002487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff99c15b990 0x7ff99c0c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff99ad9d640 1 -- 172.21.15.107:0/503002487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff99c0d1d20 msgr2=0x7ff99c15e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff99ad9d640 1 --2- 172.21.15.107:0/503002487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff99c0d1d20 0x7ff99c15e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff99ad9d640 1 -- 172.21.15.107:0/503002487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff99c15b990 msgr2=0x7ff99c0c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff99ad9d640 1 --2- 172.21.15.107:0/503002487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff99c15b990 0x7ff99c0c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff99ad9d640 1 -- 172.21.15.107:0/503002487 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c1584d0 con 0x7ff99c15f5f0 2024-04-25T08:31:13.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.032+0000 7ff99ad9d640 1 --2- 172.21.15.107:0/503002487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c15f5f0 0x7ff99c144b10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff994030000 tx=0x7ff994004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:13.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.033+0000 7ff992ffd640 1 -- 172.21.15.107:0/503002487 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99403f020 con 0x7ff99c15f5f0 2024-04-25T08:31:13.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.033+0000 7ff992ffd640 1 -- 172.21.15.107:0/503002487 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff994037040 con 0x7ff99c15f5f0 2024-04-25T08:31:13.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.033+0000 7ff992ffd640 1 -- 172.21.15.107:0/503002487 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994036070 con 0x7ff99c15f5f0 2024-04-25T08:31:13.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.033+0000 7ff9a0a07640 1 -- 172.21.15.107:0/503002487 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99c157430 con 0x7ff99c15f5f0 2024-04-25T08:31:13.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.033+0000 7ff9a0a07640 1 -- 172.21.15.107:0/503002487 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99c157cd0 con 0x7ff99c15f5f0 2024-04-25T08:31:13.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.033+0000 7ff992ffd640 1 -- 172.21.15.107:0/503002487 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff994042020 con 0x7ff99c15f5f0 2024-04-25T08:31:13.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.033+0000 7ff992ffd640 1 -- 172.21.15.107:0/503002487 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7ff994044c20 con 0x7ff99c15f5f0 2024-04-25T08:31:13.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.034+0000 7ff9a0a07640 1 -- 172.21.15.107:0/503002487 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99c068e80 con 0x7ff99c15f5f0 2024-04-25T08:31:13.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.035+0000 7ff992ffd640 1 -- 172.21.15.107:0/503002487 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff994049030 con 0x7ff99c15f5f0 2024-04-25T08:31:13.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.108+0000 7ff9a0a07640 1 -- 172.21.15.107:0/503002487 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9da74044-e310-4263-a10a-2210377872e9", "id": 6} v 0) v1 -- 0x7ff99c1577c0 con 0x7ff99c15f5f0 2024-04-25T08:31:13.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.111+0000 7ff992ffd640 1 -- 172.21.15.107:0/503002487 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9da74044-e310-4263-a10a-2210377872e9", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff994044630 con 0x7ff99c15f5f0 2024-04-25T08:31:13.108 INFO:teuthology.orchestra.run.smithi107.stdout:6 2024-04-25T08:31:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.112+0000 7ff9a0a07640 1 -- 172.21.15.107:0/503002487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c15f5f0 msgr2=0x7ff99c144b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.112+0000 7ff9a0a07640 1 --2- 172.21.15.107:0/503002487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c15f5f0 0x7ff99c144b10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff994030000 tx=0x7ff994004a40 comp rx=0 tx=0).stop 2024-04-25T08:31:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.112+0000 7ff9a0a07640 1 -- 172.21.15.107:0/503002487 shutdown_connections 2024-04-25T08:31:13.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.112+0000 7ff9a0a07640 1 --2- 172.21.15.107:0/503002487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff99c15f5f0 0x7ff99c144b10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.112+0000 7ff9a0a07640 1 --2- 172.21.15.107:0/503002487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff99c15b990 0x7ff99c0c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.112+0000 7ff9a0a07640 1 --2- 172.21.15.107:0/503002487 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff99c0d1d20 0x7ff99c15e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.112+0000 7ff9a0a07640 1 -- 172.21.15.107:0/503002487 >> 172.21.15.107:0/503002487 conn(0x7ff99c08b000 msgr2=0x7ff99c15ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:13.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.112+0000 7ff9a0a07640 1 -- 172.21.15.107:0/503002487 shutdown_connections 2024-04-25T08:31:13.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.112+0000 7ff9a0a07640 1 -- 172.21.15.107:0/503002487 wait complete. 2024-04-25T08:31:13.120 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --cluster ceph osd new f0e8cad8-190a-4288-87a4-c8d4b4ead817 7 2024-04-25T08:31:13.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.260+0000 7f381b29c640 1 Processor -- start 2024-04-25T08:31:13.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.260+0000 7f381b29c640 1 -- start start 2024-04-25T08:31:13.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f381b29c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38140b1d60 0x7f38140cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f381b29c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3814161e20 0x7f381415d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f381b29c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381415ef40 con 0x7f38140cff10 2024-04-25T08:31:13.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f381b29c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381406abe0 con 0x7f3814161e20 2024-04-25T08:31:13.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f381b29c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381406b480 con 0x7f38140b1d60 2024-04-25T08:31:13.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f3819839640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3814161e20 0x7f381415d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f3819038640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38140b1d60 0x7f38140cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f3819839640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3814161e20 0x7f381415d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35340/0 (socket says 172.21.15.107:35340) 2024-04-25T08:31:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f3819839640 1 -- 172.21.15.107:0/1431656690 learned_addr learned my addr 172.21.15.107:0/1431656690 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f3819839640 1 -- 172.21.15.107:0/1431656690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38140b1d60 msgr2=0x7f38140cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f3819839640 1 --2- 172.21.15.107:0/1431656690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38140b1d60 0x7f38140cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f3819839640 1 -- 172.21.15.107:0/1431656690 >> v1:172.21.15.107:6789/0 conn(0x7f38140cff10 legacy=0x7f3814161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f3819839640 1 -- 172.21.15.107:0/1431656690 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381406ae30 con 0x7f3814161e20 2024-04-25T08:31:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f3807fff640 1 -- 172.21.15.107:0/1431656690 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211824145 0 0) 0x7f381415ef40 con 0x7f38140cff10 2024-04-25T08:31:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.261+0000 7f3819038640 1 --2- 172.21.15.107:0/1431656690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38140b1d60 0x7f38140cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.262+0000 7f3819839640 1 --2- 172.21.15.107:0/1431656690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3814161e20 0x7f381415d330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f380c009e50 tx=0x7f380c02a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=900db6712854eef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.262+0000 7f3807fff640 1 -- 172.21.15.107:0/1431656690 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c036070 con 0x7f3814161e20 2024-04-25T08:31:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.262+0000 7f3807fff640 1 -- 172.21.15.107:0/1431656690 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380c031600 con 0x7f3814161e20 2024-04-25T08:31:13.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.262+0000 7f3807fff640 1 -- 172.21.15.107:0/1431656690 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380c0319d0 con 0x7f3814161e20 2024-04-25T08:31:13.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.263+0000 7f381b29c640 1 -- 172.21.15.107:0/1431656690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3814161e20 msgr2=0x7f381415d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.263+0000 7f381b29c640 1 --2- 172.21.15.107:0/1431656690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3814161e20 0x7f381415d330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f380c009e50 tx=0x7f380c02a730 comp rx=0 tx=0).stop 2024-04-25T08:31:13.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.263+0000 7f381b29c640 1 -- 172.21.15.107:0/1431656690 shutdown_connections 2024-04-25T08:31:13.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.263+0000 7f381b29c640 1 --2- 172.21.15.107:0/1431656690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3814161e20 0x7f381415d330 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.263+0000 7f381b29c640 1 --2- 172.21.15.107:0/1431656690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38140b1d60 0x7f38140cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.263+0000 7f381b29c640 1 -- 172.21.15.107:0/1431656690 >> 172.21.15.107:0/1431656690 conn(0x7f38140d06b0 msgr2=0x7f38140d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.263+0000 7f381b29c640 1 -- 172.21.15.107:0/1431656690 shutdown_connections 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.263+0000 7f381b29c640 1 -- 172.21.15.107:0/1431656690 wait complete. 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.264+0000 7f381b29c640 1 Processor -- start 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.264+0000 7f381b29c640 1 -- start start 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f381b29c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38140b1d60 0x7f38140b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f381b29c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38140cff10 0x7f38141f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f381b29c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3814161e20 0x7f38140c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f381b29c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381406b910 con 0x7f38140cff10 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f381b29c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38140d0370 con 0x7f38140b1d60 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f381b29c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381406ba90 con 0x7f3814161e20 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3819038640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38140b1d60 0x7f38140b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3819038640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38140b1d60 0x7f38140b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35356/0 (socket says 172.21.15.107:35356) 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3819839640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3814161e20 0x7f38140c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3819038640 1 -- 172.21.15.107:0/19143022 learned_addr learned my addr 172.21.15.107:0/19143022 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-04-25T08:31:13.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3818837640 1 --2- 172.21.15.107:0/19143022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38140cff10 0x7f38141f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3819038640 1 -- 172.21.15.107:0/19143022 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3814161e20 msgr2=0x7f38140c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3819038640 1 --2- 172.21.15.107:0/19143022 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3814161e20 0x7f38140c94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3819038640 1 -- 172.21.15.107:0/19143022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38140cff10 msgr2=0x7f38141f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3819038640 1 --2- 172.21.15.107:0/19143022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38140cff10 0x7f38141f4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3819038640 1 -- 172.21.15.107:0/19143022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140c7900 con 0x7f38140b1d60 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3819038640 1 --2- 172.21.15.107:0/19143022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38140b1d60 0x7f38140b4f60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f380800e960 tx=0x7f380800ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3805ffb640 1 -- 172.21.15.107:0/19143022 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808017070 con 0x7f38140b1d60 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3805ffb640 1 -- 172.21.15.107:0/19143022 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380800ca00 con 0x7f38140b1d60 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f381b29c640 1 -- 172.21.15.107:0/19143022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38140c7bc0 con 0x7f38140b1d60 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.265+0000 7f3805ffb640 1 -- 172.21.15.107:0/19143022 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380800cdd0 con 0x7f38140b1d60 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.266+0000 7f381b29c640 1 -- 172.21.15.107:0/19143022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38140c8180 con 0x7f38140b1d60 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.266+0000 7f3805ffb640 1 -- 172.21.15.107:0/19143022 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f380801a070 con 0x7f38140b1d60 2024-04-25T08:31:13.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.266+0000 7f3805ffb640 1 -- 172.21.15.107:0/19143022 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f3808013ad0 con 0x7f38140b1d60 2024-04-25T08:31:13.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.267+0000 7f381b29c640 1 -- 172.21.15.107:0/19143022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3814068e80 con 0x7f38140b1d60 2024-04-25T08:31:13.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.269+0000 7f3805ffb640 1 -- 172.21.15.107:0/19143022 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3808013db0 con 0x7f38140b1d60 2024-04-25T08:31:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.340+0000 7f381b29c640 1 -- 172.21.15.107:0/19143022 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f0e8cad8-190a-4288-87a4-c8d4b4ead817", "id": 7} v 0) v1 -- 0x7f381406c9e0 con 0x7f38140b1d60 2024-04-25T08:31:13.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.344+0000 7f3805ffb640 1 -- 172.21.15.107:0/19143022 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f0e8cad8-190a-4288-87a4-c8d4b4ead817", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f38080064d0 con 0x7f38140b1d60 2024-04-25T08:31:13.341 INFO:teuthology.orchestra.run.smithi107.stdout:7 2024-04-25T08:31:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.345+0000 7f381b29c640 1 -- 172.21.15.107:0/19143022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38140b1d60 msgr2=0x7f38140b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.345+0000 7f381b29c640 1 --2- 172.21.15.107:0/19143022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38140b1d60 0x7f38140b4f60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f380800e960 tx=0x7f380800ee20 comp rx=0 tx=0).stop 2024-04-25T08:31:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.345+0000 7f381b29c640 1 -- 172.21.15.107:0/19143022 shutdown_connections 2024-04-25T08:31:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.345+0000 7f381b29c640 1 --2- 172.21.15.107:0/19143022 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3814161e20 0x7f38140c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.345+0000 7f381b29c640 1 --2- 172.21.15.107:0/19143022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38140cff10 0x7f38141f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.345+0000 7f381b29c640 1 --2- 172.21.15.107:0/19143022 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38140b1d60 0x7f38140b4f60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.345+0000 7f381b29c640 1 -- 172.21.15.107:0/19143022 >> 172.21.15.107:0/19143022 conn(0x7f38140d06b0 msgr2=0x7f381406f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:13.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.345+0000 7f381b29c640 1 -- 172.21.15.107:0/19143022 shutdown_connections 2024-04-25T08:31:13.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:31:13.345+0000 7f381b29c640 1 -- 172.21.15.107:0/19143022 wait complete. 2024-04-25T08:31:13.352 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T08:31:13.352 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T08:31:13.354 INFO:tasks.ceph.osd.0:Started 2024-04-25T08:31:13.354 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T08:31:13.355 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T08:31:13.357 INFO:tasks.ceph.osd.1:Started 2024-04-25T08:31:13.357 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T08:31:13.357 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T08:31:13.357 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:13.360+0000 7f1f8a212040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T08:31:13.359 INFO:tasks.ceph.osd.2:Started 2024-04-25T08:31:13.359 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T08:31:13.359 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T08:31:13.361 INFO:tasks.ceph.osd.3:Started 2024-04-25T08:31:13.361 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T08:31:13.362 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T08:31:13.364 INFO:tasks.ceph.osd.4:Started 2024-04-25T08:31:13.364 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T08:31:13.364 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T08:31:13.366 INFO:tasks.ceph.osd.5:Started 2024-04-25T08:31:13.366 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T08:31:13.366 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T08:31:13.368 INFO:tasks.ceph.osd.6:Started 2024-04-25T08:31:13.368 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T08:31:13.368 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T08:31:13.370 INFO:tasks.ceph.osd.7:Started 2024-04-25T08:31:13.371 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:13.438 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:13.442+0000 7ff975c12040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T08:31:13.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.509+0000 7fb394c88640 1 Processor -- start 2024-04-25T08:31:13.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.509+0000 7fb394c88640 1 -- start start 2024-04-25T08:31:13.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.510+0000 7fb394c88640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb390159d30 0x7fb39006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.510+0000 7fb394c88640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3900b4e30 con 0x7fb39006e8d0 2024-04-25T08:31:13.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.510+0000 7fb394c88640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3900b3900 con 0x7fb3900b3470 2024-04-25T08:31:13.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.510+0000 7fb394c88640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3900acdb0 con 0x7fb390159d30 2024-04-25T08:31:13.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.510+0000 7fb387fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb39006e8d0 0x7fb3900b2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51196/0 (socket says 172.21.15.38:51196) 2024-04-25T08:31:13.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.510+0000 7fb387fff640 1 -- 172.21.15.38:0/1356637 learned_addr learned my addr 172.21.15.38:0/1356637 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:13.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.510+0000 7fb38e59c640 1 --2- 172.21.15.38:0/1356637 >> v2:172.21.15.107:3300/0 conn(0x7fb390159d30 0x7fb39006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.510+0000 7fb38dd9b640 1 -- 172.21.15.38:0/1356637 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129483136 0 0) 0x7fb3900b4e30 con 0x7fb39006e8d0 2024-04-25T08:31:13.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.510+0000 7fb38dd9b640 1 -- 172.21.15.38:0/1356637 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb37c003520 con 0x7fb39006e8d0 2024-04-25T08:31:13.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.511+0000 7fb38dd9b640 1 -- 172.21.15.38:0/1356637 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491789166 0 0) 0x7fb3900b3900 con 0x7fb3900b3470 2024-04-25T08:31:13.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.511+0000 7fb38dd9b640 1 -- 172.21.15.38:0/1356637 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3900b4e30 con 0x7fb3900b3470 2024-04-25T08:31:13.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.511+0000 7fb38e59c640 1 -- 172.21.15.38:0/1356637 >> v1:172.21.15.107:6789/0 conn(0x7fb3900b3470 legacy=0x7fb39006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.511+0000 7fb38dd9b640 1 -- 172.21.15.38:0/1356637 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 440062625 0 0) 0x7fb37c003520 con 0x7fb39006e8d0 2024-04-25T08:31:13.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.511+0000 7fb38e59c640 1 -- 172.21.15.38:0/1356637 >> v1:172.21.15.38:6789/0 conn(0x7fb39006e8d0 legacy=0x7fb3900b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.511+0000 7fb38e59c640 1 -- 172.21.15.38:0/1356637 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3900b3be0 con 0x7fb390159d30 2024-04-25T08:31:13.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.511+0000 7fb38e59c640 1 --2- 172.21.15.38:0/1356637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb390159d30 0x7fb39006e390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3800097d0 tx=0x7fb38002a160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e430ab5dddc61dba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:13.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.512+0000 7fb38dd9b640 1 -- 172.21.15.38:0/1356637 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb380037070 con 0x7fb390159d30 2024-04-25T08:31:13.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.512+0000 7fb38dd9b640 1 -- 172.21.15.38:0/1356637 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38002abf0 con 0x7fb390159d30 2024-04-25T08:31:13.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.512+0000 7fb38dd9b640 1 -- 172.21.15.38:0/1356637 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378001690 con 0x7fb390159d30 2024-04-25T08:31:13.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.512+0000 7fb394c88640 1 -- 172.21.15.38:0/1356637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb390159d30 msgr2=0x7fb39006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.512+0000 7fb394c88640 1 --2- 172.21.15.38:0/1356637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb390159d30 0x7fb39006e390 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb3800097d0 tx=0x7fb38002a160 comp rx=0 tx=0).stop 2024-04-25T08:31:13.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.512+0000 7fb394c88640 1 -- 172.21.15.38:0/1356637 shutdown_connections 2024-04-25T08:31:13.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.512+0000 7fb394c88640 1 --2- 172.21.15.38:0/1356637 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb390159d30 0x7fb39006e390 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.512+0000 7fb394c88640 1 -- 172.21.15.38:0/1356637 >> 172.21.15.38:0/1356637 conn(0x7fb390071920 msgr2=0x7fb390071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:13.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.512+0000 7fb394c88640 1 -- 172.21.15.38:0/1356637 shutdown_connections 2024-04-25T08:31:13.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.512+0000 7fb394c88640 1 -- 172.21.15.38:0/1356637 wait complete. 2024-04-25T08:31:13.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.514+0000 7fb394c88640 1 Processor -- start 2024-04-25T08:31:13.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.514+0000 7fb394c88640 1 -- start start 2024-04-25T08:31:13.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.514+0000 7fb394c88640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb39006e8d0 0x7fb3901515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.514+0000 7fb394c88640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb390159d30 0x7fb39014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.514+0000 7fb394c88640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb390140ff0 0x7fb3901f2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:13.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.514+0000 7fb394c88640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39015a290 con 0x7fb390159d30 2024-04-25T08:31:13.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.514+0000 7fb394c88640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb39015a110 con 0x7fb39006e8d0 2024-04-25T08:31:13.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.514+0000 7fb394c88640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39006c300 con 0x7fb390140ff0 2024-04-25T08:31:13.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb387fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb390159d30 0x7fb39014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb387fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb390159d30 0x7fb39014fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56926/0 (socket says 172.21.15.38:56926) 2024-04-25T08:31:13.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb387fff640 1 -- 172.21.15.38:0/1248618898 learned_addr learned my addr 172.21.15.38:0/1248618898 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:13.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb38e59c640 1 --2- 172.21.15.38:0/1248618898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb39006e8d0 0x7fb3901515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb38ed9d640 1 --2- 172.21.15.38:0/1248618898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb390140ff0 0x7fb3901f2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb387fff640 1 -- 172.21.15.38:0/1248618898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb390140ff0 msgr2=0x7fb3901f2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb387fff640 1 --2- 172.21.15.38:0/1248618898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb390140ff0 0x7fb3901f2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb387fff640 1 -- 172.21.15.38:0/1248618898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb39006e8d0 msgr2=0x7fb3901515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb387fff640 1 --2- 172.21.15.38:0/1248618898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb39006e8d0 0x7fb3901515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb387fff640 1 -- 172.21.15.38:0/1248618898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb390151c40 con 0x7fb390159d30 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb38e59c640 1 --2- 172.21.15.38:0/1248618898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb39006e8d0 0x7fb3901515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb387fff640 1 --2- 172.21.15.38:0/1248618898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb390159d30 0x7fb39014fd00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c700 tx=0x7fb37800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb386ffd640 1 -- 172.21.15.38:0/1248618898 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378017070 con 0x7fb390159d30 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb386ffd640 1 -- 172.21.15.38:0/1248618898 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb378005ad0 con 0x7fb390159d30 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb394c88640 1 -- 172.21.15.38:0/1248618898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb390151f20 con 0x7fb390159d30 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb386ffd640 1 -- 172.21.15.38:0/1248618898 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378007070 con 0x7fb390159d30 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.515+0000 7fb394c88640 1 -- 172.21.15.38:0/1248618898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3901524e0 con 0x7fb390159d30 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.516+0000 7fb386ffd640 1 -- 172.21.15.38:0/1248618898 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb37801a070 con 0x7fb390159d30 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.516+0000 7fb386ffd640 1 -- 172.21.15.38:0/1248618898 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb37801cb10 con 0x7fb390159d30 2024-04-25T08:31:13.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.516+0000 7fb394c88640 1 -- 172.21.15.38:0/1248618898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb390072790 con 0x7fb390159d30 2024-04-25T08:31:13.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.519+0000 7fb386ffd640 1 -- 172.21.15.38:0/1248618898 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb37800a640 con 0x7fb390159d30 2024-04-25T08:31:13.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.581+0000 7fb394c88640 1 -- 172.21.15.38:0/1248618898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb390157d90 con 0x7fb390159d30 2024-04-25T08:31:13.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.581+0000 7fb386ffd640 1 -- 172.21.15.38:0/1248618898 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fb37800aa50 con 0x7fb390159d30 2024-04-25T08:31:13.578 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:13.578 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":10,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:13.342921+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:13.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.582+0000 7fb394c88640 1 -- 172.21.15.38:0/1248618898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb390159d30 msgr2=0x7fb39014fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:13.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.582+0000 7fb394c88640 1 --2- 172.21.15.38:0/1248618898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb390159d30 0x7fb39014fd00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c700 tx=0x7fb37800cbc0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.582+0000 7fb394c88640 1 -- 172.21.15.38:0/1248618898 shutdown_connections 2024-04-25T08:31:13.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.582+0000 7fb394c88640 1 --2- 172.21.15.38:0/1248618898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb390140ff0 0x7fb3901f2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.582+0000 7fb394c88640 1 --2- 172.21.15.38:0/1248618898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb390159d30 0x7fb39014fd00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.582+0000 7fb394c88640 1 --2- 172.21.15.38:0/1248618898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb39006e8d0 0x7fb3901515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:13.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.582+0000 7fb394c88640 1 -- 172.21.15.38:0/1248618898 >> 172.21.15.38:0/1248618898 conn(0x7fb390071920 msgr2=0x7fb3901568b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:13.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.582+0000 7fb394c88640 1 -- 172.21.15.38:0/1248618898 shutdown_connections 2024-04-25T08:31:13.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:13.582+0000 7fb394c88640 1 -- 172.21.15.38:0/1248618898 wait complete. 2024-04-25T08:31:13.590 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T08:31:13.590 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T08:31:13.641 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:13.644+0000 7f1f8a212040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T08:31:13.693 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:13.696+0000 7f1f8a212040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T08:31:13.716 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:13.720+0000 7ff975c12040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T08:31:13.772 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:13.774+0000 7ff975c12040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T08:31:13.808 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:13.811+0000 7f1f8a212040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T08:31:13.892 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:13.895+0000 7ff975c12040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T08:31:13.963 INFO:tasks.ceph.osd.0.smithi038.stderr:2024-04-25T08:31:13.966+0000 7f1e05614640 -1 Falling back to public interface 2024-04-25T08:31:13.964 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-04-25T08:31:13.966+0000 7fb3775a3640 -1 Falling back to public interface 2024-04-25T08:31:13.964 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-04-25T08:31:13.966+0000 7fb376552640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-25T08:31:13.964 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-04-25T08:31:13.966+0000 7fb376552640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2024-04-25T08:31:13.964 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-04-25T08:31:13.967+0000 7fb376552640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6807/0: (98) Address already in use 2024-04-25T08:31:13.970 INFO:tasks.ceph.osd.4.smithi107.stderr:2024-04-25T08:31:13.973+0000 7ff61d630640 -1 Falling back to public interface 2024-04-25T08:31:13.971 INFO:tasks.ceph.osd.3.smithi038.stderr:2024-04-25T08:31:13.975+0000 7fdd95d40640 -1 Falling back to public interface 2024-04-25T08:31:13.976 INFO:tasks.ceph.osd.6.smithi107.stderr:2024-04-25T08:31:13.979+0000 7f97f42ad640 -1 Falling back to public interface 2024-04-25T08:31:13.989 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-04-25T08:31:13.992+0000 7fc53c7b1640 -1 Falling back to public interface 2024-04-25T08:31:13.992 INFO:tasks.ceph.osd.2.smithi038.stderr:2024-04-25T08:31:13.996+0000 7f4f1a00f640 -1 Falling back to public interface 2024-04-25T08:31:13.993 INFO:tasks.ceph.osd.7.smithi107.stderr:2024-04-25T08:31:13.996+0000 7fe2e25cb640 -1 Falling back to public interface 2024-04-25T08:31:14.091 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:14.094+0000 7f1f8a212040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T08:31:14.139 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:14.143+0000 7f1f8a212040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T08:31:14.193 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:14.196+0000 7ff975c12040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T08:31:14.243 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:14.246+0000 7ff975c12040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T08:31:14.296 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:14.299+0000 7f1f8a212040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T08:31:14.394 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:14.407 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:14.411+0000 7ff975c12040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T08:31:14.431 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:14.434+0000 7f1f8a212040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T08:31:14.480 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:14.483+0000 7f1f8a212040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T08:31:14.487 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.490+0000 7f689e47c640 1 Processor -- start 2024-04-25T08:31:14.487 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.490+0000 7f689e47c640 1 -- start start 2024-04-25T08:31:14.488 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f689e47c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f689806b2d0 0x7f68980b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:14.488 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f689e47c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f68980b19b0 0x7f689815d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:14.488 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f689e47c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68980afcb0 con 0x7f68980cc3f0 2024-04-25T08:31:14.488 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f689e47c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68980aff50 con 0x7f68980b19b0 2024-04-25T08:31:14.488 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f689e47c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68980b0230 con 0x7f689806b2d0 2024-04-25T08:31:14.489 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f689ca19640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f68980b19b0 0x7f689815d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:14.489 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f689ca19640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f68980b19b0 0x7f689815d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56934/0 (socket says 172.21.15.38:56934) 2024-04-25T08:31:14.489 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f689ca19640 1 -- 172.21.15.38:0/3852645757 learned_addr learned my addr 172.21.15.38:0/3852645757 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:14.489 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f68977fe640 1 --2- 172.21.15.38:0/3852645757 >> v2:172.21.15.107:3300/0 conn(0x7f689806b2d0 0x7f68980b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:14.489 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f689ca19640 1 -- 172.21.15.38:0/3852645757 >> v2:172.21.15.107:3300/0 conn(0x7f689806b2d0 msgr2=0x7f68980b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:14.489 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f689ca19640 1 --2- 172.21.15.38:0/3852645757 >> v2:172.21.15.107:3300/0 conn(0x7f689806b2d0 0x7f68980b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:14.489 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f689ca19640 1 -- 172.21.15.38:0/3852645757 >> v1:172.21.15.107:6789/0 conn(0x7f68980cc3f0 legacy=0x7f689806abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:14.489 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.491+0000 7f689ca19640 1 -- 172.21.15.38:0/3852645757 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68980b0510 con 0x7f68980b19b0 2024-04-25T08:31:14.490 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.492+0000 7f689ca19640 1 --2- 172.21.15.38:0/3852645757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68980b19b0 0x7f689815d1b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f68980ce0b0 tx=0x7f689002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dbb145353939a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:14.490 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.493+0000 7f6896ffd640 1 -- 172.21.15.38:0/3852645757 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890036070 con 0x7f68980b19b0 2024-04-25T08:31:14.490 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.493+0000 7f6896ffd640 1 -- 172.21.15.38:0/3852645757 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6890031600 con 0x7f68980b19b0 2024-04-25T08:31:14.490 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.493+0000 7f6896ffd640 1 -- 172.21.15.38:0/3852645757 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68900319d0 con 0x7f68980b19b0 2024-04-25T08:31:14.490 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.493+0000 7f689e47c640 1 -- 172.21.15.38:0/3852645757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68980b19b0 msgr2=0x7f689815d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:14.491 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.493+0000 7f689e47c640 1 --2- 172.21.15.38:0/3852645757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68980b19b0 0x7f689815d1b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f68980ce0b0 tx=0x7f689002a730 comp rx=0 tx=0).stop 2024-04-25T08:31:14.491 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.493+0000 7f689e47c640 1 -- 172.21.15.38:0/3852645757 shutdown_connections 2024-04-25T08:31:14.491 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.493+0000 7f689e47c640 1 --2- 172.21.15.38:0/3852645757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68980b19b0 0x7f689815d1b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:14.491 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.493+0000 7f689e47c640 1 --2- 172.21.15.38:0/3852645757 >> v2:172.21.15.107:3300/0 conn(0x7f689806b2d0 0x7f68980b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:14.491 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.493+0000 7f689e47c640 1 -- 172.21.15.38:0/3852645757 >> 172.21.15.38:0/3852645757 conn(0x7f6898072620 msgr2=0x7f6898099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:14.491 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.493+0000 7f689e47c640 1 -- 172.21.15.38:0/3852645757 shutdown_connections 2024-04-25T08:31:14.491 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.493+0000 7f689e47c640 1 -- 172.21.15.38:0/3852645757 wait complete. 2024-04-25T08:31:14.491 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.494+0000 7f689e47c640 1 Processor -- start 2024-04-25T08:31:14.491 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.495+0000 7f689e47c640 1 -- start start 2024-04-25T08:31:14.494 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.495+0000 7f689e47c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f689806b2d0 0x7f6898150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:14.494 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.495+0000 7f689e47c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68980b19b0 0x7f68980c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:14.494 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.495+0000 7f689e47c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68980cc3f0 0x7f689806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:14.494 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.495+0000 7f689e47c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980cc7d0 con 0x7f689806b2d0 2024-04-25T08:31:14.494 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.495+0000 7f689e47c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980b1e40 con 0x7f68980b19b0 2024-04-25T08:31:14.494 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.495+0000 7f689e47c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68980cc950 con 0x7f68980cc3f0 2024-04-25T08:31:14.494 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f6897fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f689806b2d0 0x7f6898150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:14.494 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f6897fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f689806b2d0 0x7f6898150af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56948/0 (socket says 172.21.15.38:56948) 2024-04-25T08:31:14.494 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f6897fff640 1 -- 172.21.15.38:0/3074396726 learned_addr learned my addr 172.21.15.38:0/3074396726 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:14.494 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f68977fe640 1 --2- 172.21.15.38:0/3074396726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68980b19b0 0x7f68980c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f689ca19640 1 --2- 172.21.15.38:0/3074396726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68980cc3f0 0x7f689806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f6897fff640 1 -- 172.21.15.38:0/3074396726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68980cc3f0 msgr2=0x7f689806e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f6897fff640 1 --2- 172.21.15.38:0/3074396726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68980cc3f0 0x7f689806e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f6897fff640 1 -- 172.21.15.38:0/3074396726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68980b19b0 msgr2=0x7f68980c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f6897fff640 1 --2- 172.21.15.38:0/3074396726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68980b19b0 0x7f68980c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f6897fff640 1 -- 172.21.15.38:0/3074396726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68980b3290 con 0x7f689806b2d0 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f6897fff640 1 --2- 172.21.15.38:0/3074396726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f689806b2d0 0x7f6898150af0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c3f0 tx=0x7f688c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f6894ff9640 1 -- 172.21.15.38:0/3074396726 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c0052d0 con 0x7f689806b2d0 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f6894ff9640 1 -- 172.21.15.38:0/3074396726 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688c005490 con 0x7f689806b2d0 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f689e47c640 1 -- 172.21.15.38:0/3074396726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68980b3500 con 0x7f689806b2d0 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f6894ff9640 1 -- 172.21.15.38:0/3074396726 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c007260 con 0x7f689806b2d0 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.496+0000 7f689e47c640 1 -- 172.21.15.38:0/3074396726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6898151030 con 0x7f689806b2d0 2024-04-25T08:31:14.495 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.497+0000 7f6894ff9640 1 -- 172.21.15.38:0/3074396726 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f688c007400 con 0x7f689806b2d0 2024-04-25T08:31:14.496 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.497+0000 7f6894ff9640 1 -- 172.21.15.38:0/3074396726 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f688c011c60 con 0x7f689806b2d0 2024-04-25T08:31:14.496 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.497+0000 7f689e47c640 1 -- 172.21.15.38:0/3074396726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6898074d30 con 0x7f689806b2d0 2024-04-25T08:31:14.496 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.500+0000 7f6894ff9640 1 -- 172.21.15.38:0/3074396726 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f688c0035f0 con 0x7f689806b2d0 2024-04-25T08:31:14.552 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:14.555+0000 7f1f8a212040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T08:31:14.555 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:14.559+0000 7ff975c12040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T08:31:14.559 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.563+0000 7f689e47c640 1 -- 172.21.15.38:0/3074396726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68980cfe20 con 0x7f689806b2d0 2024-04-25T08:31:14.560 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.563+0000 7f6894ff9640 1 -- 172.21.15.38:0/3074396726 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f688c003a00 con 0x7f689806b2d0 2024-04-25T08:31:14.560 INFO:teuthology.misc.health.smithi038.stdout: 2024-04-25T08:31:14.560 INFO:teuthology.misc.health.smithi038.stdout:{"epoch":10,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:13.342921+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:14.561 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.564+0000 7f689e47c640 1 -- 172.21.15.38:0/3074396726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f689806b2d0 msgr2=0x7f6898150af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:14.561 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.564+0000 7f689e47c640 1 --2- 172.21.15.38:0/3074396726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f689806b2d0 0x7f6898150af0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f688c00c3f0 tx=0x7f688c00c8b0 comp rx=0 tx=0).stop 2024-04-25T08:31:14.562 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.564+0000 7f689e47c640 1 -- 172.21.15.38:0/3074396726 shutdown_connections 2024-04-25T08:31:14.562 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.564+0000 7f689e47c640 1 --2- 172.21.15.38:0/3074396726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68980cc3f0 0x7f689806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:14.562 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.564+0000 7f689e47c640 1 --2- 172.21.15.38:0/3074396726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68980b19b0 0x7f68980c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:14.562 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.564+0000 7f689e47c640 1 --2- 172.21.15.38:0/3074396726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f689806b2d0 0x7f6898150af0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:14.562 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.564+0000 7f689e47c640 1 -- 172.21.15.38:0/3074396726 >> 172.21.15.38:0/3074396726 conn(0x7f6898072620 msgr2=0x7f6898099a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:14.562 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.564+0000 7f689e47c640 1 -- 172.21.15.38:0/3074396726 shutdown_connections 2024-04-25T08:31:14.562 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:14.564+0000 7f689e47c640 1 -- 172.21.15.38:0/3074396726 wait complete. 2024-04-25T08:31:14.570 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T08:31:14.597 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:14.601+0000 7f1f8a212040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T08:31:14.608 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:14.612+0000 7ff975c12040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T08:31:14.682 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:14.685+0000 7ff975c12040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T08:31:14.729 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:14.732+0000 7ff975c12040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T08:31:15.068 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:15.071+0000 7f1f8a212040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T08:31:15.185 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:15.188+0000 7f1f8a212040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T08:31:15.231 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:15.234+0000 7ff975c12040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T08:31:15.352 INFO:tasks.ceph.mgr.y.smithi038.stderr:2024-04-25T08:31:15.355+0000 7ff975c12040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T08:31:16.202 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:16.205+0000 7f1f2aacf640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T08:31:16.203 INFO:tasks.ceph.mgr.x.smithi107.stderr:2024-04-25T08:31:16.206+0000 7f1f2aacf640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T08:31:17.412 INFO:tasks.ceph.osd.0.smithi038.stderr:2024-04-25T08:31:17.416+0000 7f1e05614640 -1 osd.0 0 log_to_monitors true 2024-04-25T08:31:17.450 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-04-25T08:31:17.454+0000 7fb3775a3640 -1 osd.1 0 log_to_monitors true 2024-04-25T08:31:17.454 INFO:tasks.ceph.osd.6.smithi107.stderr:2024-04-25T08:31:17.458+0000 7f97f42ad640 -1 osd.6 0 log_to_monitors true 2024-04-25T08:31:17.485 INFO:tasks.ceph.osd.4.smithi107.stderr:2024-04-25T08:31:17.489+0000 7ff61d630640 -1 osd.4 0 log_to_monitors true 2024-04-25T08:31:17.487 INFO:tasks.ceph.osd.3.smithi038.stderr:2024-04-25T08:31:17.490+0000 7fdd95d40640 -1 osd.3 0 log_to_monitors true 2024-04-25T08:31:17.504 INFO:tasks.ceph.osd.2.smithi038.stderr:2024-04-25T08:31:17.508+0000 7f4f1a00f640 -1 osd.2 0 log_to_monitors true 2024-04-25T08:31:17.508 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-04-25T08:31:17.512+0000 7fc53c7b1640 -1 osd.5 0 log_to_monitors true 2024-04-25T08:31:17.511 INFO:tasks.ceph.osd.7.smithi107.stderr:2024-04-25T08:31:17.514+0000 7fe2e25cb640 -1 osd.7 0 log_to_monitors true 2024-04-25T08:31:21.051 INFO:tasks.ceph.osd.7.smithi107.stderr:2024-04-25T08:31:21.054+0000 7fe2de561640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T08:31:21.071 INFO:tasks.ceph.osd.7.smithi107.stderr:2024-04-25T08:31:21.074+0000 7fe2d6381640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:31:21.086 INFO:tasks.ceph.osd.2.smithi038.stderr:2024-04-25T08:31:21.089+0000 7f4f15924640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T08:31:21.099 INFO:tasks.ceph.osd.2.smithi038.stderr:2024-04-25T08:31:21.103+0000 7f4f0d744640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:31:21.188 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-04-25T08:31:21.191+0000 7fc538747640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T08:31:21.200 INFO:tasks.ceph.osd.5.smithi107.stderr:2024-04-25T08:31:21.204+0000 7fc530567640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:31:21.257 INFO:tasks.ceph.osd.3.smithi038.stderr:2024-04-25T08:31:21.260+0000 7fdd91cd6640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T08:31:21.269 INFO:tasks.ceph.osd.3.smithi038.stderr:2024-04-25T08:31:21.273+0000 7fdd89af6640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:31:21.318 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-04-25T08:31:21.321+0000 7fb373539640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T08:31:21.325 INFO:tasks.ceph.osd.4.smithi107.stderr:2024-04-25T08:31:21.328+0000 7ff618b78640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T08:31:21.333 INFO:tasks.ceph.osd.6.smithi107.stderr:2024-04-25T08:31:21.337+0000 7f97f0243640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T08:31:21.337 INFO:tasks.ceph.osd.1.smithi038.stderr:2024-04-25T08:31:21.341+0000 7fb36b359640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:31:21.342 INFO:tasks.ceph.osd.4.smithi107.stderr:2024-04-25T08:31:21.346+0000 7ff610998640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:31:21.356 INFO:tasks.ceph.osd.6.smithi107.stderr:2024-04-25T08:31:21.360+0000 7f97e8063640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:31:21.357 INFO:tasks.ceph.osd.0.smithi038.stderr:2024-04-25T08:31:21.361+0000 7f1e00ea5640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T08:31:21.371 INFO:tasks.ceph.osd.0.smithi038.stderr:2024-04-25T08:31:21.375+0000 7f1df8cc5640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:31:21.377 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:21.468 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.472+0000 7f395421c640 1 Processor -- start 2024-04-25T08:31:21.468 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.472+0000 7f395421c640 1 -- start start 2024-04-25T08:31:21.472 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f395421c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c0b9360 0x7f394c0ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:21.472 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f395421c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f394c0cea20 0x7f394c0b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:21.472 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f395421c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394c0b97c0 con 0x7f394c15dab0 2024-04-25T08:31:21.472 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f395421c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394c0d23c0 con 0x7f394c0cea20 2024-04-25T08:31:21.472 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f395421c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0d26a0 con 0x7f394c0b9360 2024-04-25T08:31:21.472 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3951fb8640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f394c15dab0 0x7f394c0d1be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33724/0 (socket says 172.21.15.38:33724) 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3951fb8640 1 -- 172.21.15.38:0/1830332292 learned_addr learned my addr 172.21.15.38:0/1830332292 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f39517b7640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c0b9360 0x7f394c0ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093621066 0 0) 0x7f394c0b97c0 con 0x7f394c15dab0 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f39527b9640 1 --2- 172.21.15.38:0/1830332292 >> v2:172.21.15.107:3300/0 conn(0x7f394c0cea20 0x7f394c0b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3938003520 con 0x7f394c15dab0 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1072498924 0 0) 0x7f3938003520 con 0x7f394c15dab0 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c0b9360 msgr2=0x7f394c0ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c0b9360 0x7f394c0ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 >> v2:172.21.15.107:3300/0 conn(0x7f394c0cea20 msgr2=0x7f394c0b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 --2- 172.21.15.38:0/1830332292 >> v2:172.21.15.107:3300/0 conn(0x7f394c0cea20 0x7f394c0b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c0d2980 con 0x7f394c15dab0 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f3948001670 con 0x7f394c15dab0 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 >> v1:172.21.15.38:6789/0 conn(0x7f394c15dab0 legacy=0x7f394c0d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39380039b0 0x7f3938003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:21.473 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c0b9360 0x7f3938004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3938004d40 0x7f39380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0d23c0 con 0x7f39380039b0 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0d26a0 con 0x7f3938004d40 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.473+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393800b9c0 con 0x7f394c0b9360 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.474+0000 7f3951fb8640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39380039b0 0x7f3938003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.474+0000 7f39517b7640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c0b9360 0x7f3938004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.474+0000 7f39527b9640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3938004d40 0x7f39380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.474+0000 7f3951fb8640 1 -- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c0b9360 msgr2=0x7f3938004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.474+0000 7f3951fb8640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c0b9360 0x7f3938004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.474+0000 7f3951fb8640 1 -- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3938004d40 msgr2=0x7f39380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.474+0000 7f3951fb8640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3938004d40 0x7f39380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.474+0000 7f3951fb8640 1 -- 172.21.15.38:0/1830332292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3938003520 con 0x7f39380039b0 2024-04-25T08:31:21.474 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.474+0000 7f39517b7640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c0b9360 0x7f3938004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:21.475 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.474+0000 7f39527b9640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3938004d40 0x7f39380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:21.475 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.474+0000 7f3951fb8640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39380039b0 0x7f3938003dd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3948007fd0 tx=0x7f3938005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e2c38c8b9f17bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:21.475 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.475+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948007d50 con 0x7f39380039b0 2024-04-25T08:31:21.475 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.475+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3948005ad0 con 0x7f39380039b0 2024-04-25T08:31:21.475 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.475+0000 7f3950fb6640 1 -- 172.21.15.38:0/1830332292 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3948007070 con 0x7f39380039b0 2024-04-25T08:31:21.475 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.475+0000 7f395421c640 1 -- 172.21.15.38:0/1830332292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39380039b0 msgr2=0x7f3938003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:21.475 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.475+0000 7f395421c640 1 --2- 172.21.15.38:0/1830332292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39380039b0 0x7f3938003dd0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3948007fd0 tx=0x7f3938005830 comp rx=0 tx=0).stop 2024-04-25T08:31:21.475 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.475+0000 7f39527b9640 1 -- 172.21.15.38:0/1830332292 reap_dead start 2024-04-25T08:31:21.475 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.475+0000 7f395421c640 1 -- 172.21.15.38:0/1830332292 shutdown_connections 2024-04-25T08:31:21.475 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.475+0000 7f395421c640 1 -- 172.21.15.38:0/1830332292 >> 172.21.15.38:0/1830332292 conn(0x7f394c06be70 msgr2=0x7f394c06c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:21.475 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.475+0000 7f395421c640 1 -- 172.21.15.38:0/1830332292 shutdown_connections 2024-04-25T08:31:21.475 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.475+0000 7f395421c640 1 -- 172.21.15.38:0/1830332292 wait complete. 2024-04-25T08:31:21.478 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.476+0000 7f395421c640 1 Processor -- start 2024-04-25T08:31:21.478 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.476+0000 7f395421c640 1 -- start start 2024-04-25T08:31:21.478 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f395421c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39380039b0 0x7f394c14efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:21.478 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f395421c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f394c15de90 0x7f394c15e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:21.478 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f395421c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c15a860 0x7f394c15ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:21.478 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f395421c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0d3460 con 0x7f394c15de90 2024-04-25T08:31:21.478 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f395421c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0d2f40 con 0x7f39380039b0 2024-04-25T08:31:21.478 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f395421c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394c0ce4e0 con 0x7f394c15a860 2024-04-25T08:31:21.478 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f39517b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f394c15de90 0x7f394c15e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f39517b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f394c15de90 0x7f394c15e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33400/0 (socket says 172.21.15.38:33400) 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f39517b7640 1 -- 172.21.15.38:0/2000158510 learned_addr learned my addr 172.21.15.38:0/2000158510 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f39527b9640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c15a860 0x7f394c15ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f3951fb8640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39380039b0 0x7f394c14efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f39517b7640 1 -- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c15a860 msgr2=0x7f394c15ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f39517b7640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c15a860 0x7f394c15ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f39517b7640 1 -- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39380039b0 msgr2=0x7f394c14efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.477+0000 7f39517b7640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39380039b0 0x7f394c14efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.478+0000 7f39517b7640 1 -- 172.21.15.38:0/2000158510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c14f500 con 0x7f394c15de90 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.478+0000 7f39517b7640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f394c15de90 0x7f394c15e310 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f393c011630 tx=0x7f393c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.478+0000 7f39427fc640 1 -- 172.21.15.38:0/2000158510 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c00f920 con 0x7f394c15de90 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.478+0000 7f39427fc640 1 -- 172.21.15.38:0/2000158510 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c00fac0 con 0x7f394c15de90 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.478+0000 7f395421c640 1 -- 172.21.15.38:0/2000158510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f394c15b5d0 con 0x7f394c15de90 2024-04-25T08:31:21.479 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.478+0000 7f39427fc640 1 -- 172.21.15.38:0/2000158510 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c00fe90 con 0x7f394c15de90 2024-04-25T08:31:21.480 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.478+0000 7f395421c640 1 -- 172.21.15.38:0/2000158510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c14c8e0 con 0x7f394c15de90 2024-04-25T08:31:21.480 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.479+0000 7f395421c640 1 -- 172.21.15.38:0/2000158510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c074d30 con 0x7f394c15de90 2024-04-25T08:31:21.480 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.479+0000 7f39427fc640 1 -- 172.21.15.38:0/2000158510 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f393c005990 con 0x7f394c15de90 2024-04-25T08:31:21.480 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.480+0000 7f39427fc640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f393006b610 0x7f393006dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:21.480 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.480+0000 7f39427fc640 1 -- 172.21.15.38:0/2000158510 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f393c089bd0 con 0x7f394c15de90 2024-04-25T08:31:21.480 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.480+0000 7f3951fb8640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f393006b610 0x7f393006dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:21.480 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.480+0000 7f3951fb8640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f393006b610 0x7f393006dad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3948033000 tx=0x7f3948009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:21.482 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.484+0000 7f39427fc640 1 -- 172.21.15.38:0/2000158510 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f393c019030 con 0x7f394c15de90 2024-04-25T08:31:21.662 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.666+0000 7f395421c640 1 -- 172.21.15.38:0/2000158510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f394c160450 con 0x7f394c15de90 2024-04-25T08:31:21.663 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.666+0000 7f39427fc640 1 -- 172.21.15.38:0/2000158510 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f393c059950 con 0x7f394c15de90 2024-04-25T08:31:21.663 INFO:teuthology.misc.health.smithi038.stdout: 2024-04-25T08:31:21.663 INFO:teuthology.misc.health.smithi038.stdout:{"epoch":12,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:19.209660+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:21.665 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.668+0000 7f395421c640 1 -- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f393006b610 msgr2=0x7f393006dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:21.665 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.668+0000 7f395421c640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f393006b610 0x7f393006dad0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3948033000 tx=0x7f3948009030 comp rx=0 tx=0).stop 2024-04-25T08:31:21.665 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.668+0000 7f395421c640 1 -- 172.21.15.38:0/2000158510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f394c15de90 msgr2=0x7f394c15e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:21.665 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.668+0000 7f395421c640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f394c15de90 0x7f394c15e310 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f393c011630 tx=0x7f393c011af0 comp rx=0 tx=0).stop 2024-04-25T08:31:21.665 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.668+0000 7f395421c640 1 -- 172.21.15.38:0/2000158510 shutdown_connections 2024-04-25T08:31:21.666 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.668+0000 7f395421c640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f393006b610 0x7f393006dad0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:21.666 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.668+0000 7f395421c640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f394c15a860 0x7f394c15ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:21.666 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.668+0000 7f395421c640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f394c15de90 0x7f394c15e310 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:21.666 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.668+0000 7f395421c640 1 --2- 172.21.15.38:0/2000158510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39380039b0 0x7f394c14efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:21.666 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.668+0000 7f395421c640 1 -- 172.21.15.38:0/2000158510 >> 172.21.15.38:0/2000158510 conn(0x7f394c06be70 msgr2=0x7f394c0b8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:21.666 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.668+0000 7f395421c640 1 -- 172.21.15.38:0/2000158510 shutdown_connections 2024-04-25T08:31:21.666 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:21.668+0000 7f395421c640 1 -- 172.21.15.38:0/2000158510 wait complete. 2024-04-25T08:31:21.674 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T08:31:28.481 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:28.585 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.589+0000 7f995bc5d640 1 Processor -- start 2024-04-25T08:31:28.586 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.589+0000 7f995bc5d640 1 -- start start 2024-04-25T08:31:28.587 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.590+0000 7f995bc5d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f995415dc10 0x7f99540d3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:28.587 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.590+0000 7f995bc5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99540cec50 0x7f99540d18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:28.587 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.590+0000 7f995bc5d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99540d1fd0 con 0x7f99540b9190 2024-04-25T08:31:28.587 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.590+0000 7f995bc5d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99540d2be0 con 0x7f995415dc10 2024-04-25T08:31:28.587 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.590+0000 7f995bc5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99540d22b0 con 0x7f99540cec50 2024-04-25T08:31:28.587 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.590+0000 7f99599f9640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f995415dc10 0x7f99540d3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:28.587 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.590+0000 7f995a1fa640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99540cec50 0x7f99540d18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:28.587 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.590+0000 7f99591f8640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f99540b9190 0x7f99540ce540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:54282/0 (socket says 172.21.15.38:54282) 2024-04-25T08:31:28.587 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.590+0000 7f99599f9640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f995415dc10 0x7f99540d3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:46204/0 (socket says 172.21.15.38:46204) 2024-04-25T08:31:28.587 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.590+0000 7f99591f8640 1 -- 172.21.15.38:0/3121880746 learned_addr learned my addr 172.21.15.38:0/3121880746 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:28.589 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.591+0000 7f99599f9640 1 -- 172.21.15.38:0/3121880746 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99540cec50 msgr2=0x7f99540d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:28.589 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.591+0000 7f99599f9640 1 --2- 172.21.15.38:0/3121880746 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99540cec50 0x7f99540d18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.589 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.591+0000 7f99599f9640 1 -- 172.21.15.38:0/3121880746 >> v1:172.21.15.107:6789/0 conn(0x7f99540b9190 legacy=0x7f99540ce540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:28.589 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.591+0000 7f99599f9640 1 -- 172.21.15.38:0/3121880746 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99540d2590 con 0x7f995415dc10 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.591+0000 7f995a1fa640 1 --2- 172.21.15.38:0/3121880746 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99540cec50 0x7f99540d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.591+0000 7f99589f7640 1 -- 172.21.15.38:0/3121880746 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442848172 0 0) 0x7f99540d1fd0 con 0x7f99540b9190 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.592+0000 7f99599f9640 1 --2- 172.21.15.38:0/3121880746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f995415dc10 0x7f99540d3070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99500097a0 tx=0x7f995002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aec29bfc1a2c5c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.592+0000 7f99589f7640 1 -- 172.21.15.38:0/3121880746 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950037070 con 0x7f995415dc10 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.592+0000 7f99589f7640 1 -- 172.21.15.38:0/3121880746 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9950032440 con 0x7f995415dc10 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.592+0000 7f99589f7640 1 -- 172.21.15.38:0/3121880746 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950032810 con 0x7f995415dc10 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.592+0000 7f995bc5d640 1 -- 172.21.15.38:0/3121880746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f995415dc10 msgr2=0x7f99540d3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.592+0000 7f995bc5d640 1 --2- 172.21.15.38:0/3121880746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f995415dc10 0x7f99540d3070 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99500097a0 tx=0x7f995002a210 comp rx=0 tx=0).stop 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.592+0000 7f995bc5d640 1 -- 172.21.15.38:0/3121880746 shutdown_connections 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.592+0000 7f995bc5d640 1 --2- 172.21.15.38:0/3121880746 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99540cec50 0x7f99540d18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.592+0000 7f995bc5d640 1 --2- 172.21.15.38:0/3121880746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f995415dc10 0x7f99540d3070 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.592+0000 7f995bc5d640 1 -- 172.21.15.38:0/3121880746 >> 172.21.15.38:0/3121880746 conn(0x7f9954072710 msgr2=0x7f9954072b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:28.590 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.592+0000 7f995bc5d640 1 -- 172.21.15.38:0/3121880746 shutdown_connections 2024-04-25T08:31:28.591 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.593+0000 7f995bc5d640 1 -- 172.21.15.38:0/3121880746 wait complete. 2024-04-25T08:31:28.593 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.594+0000 7f995bc5d640 1 Processor -- start 2024-04-25T08:31:28.593 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.594+0000 7f995bc5d640 1 -- start start 2024-04-25T08:31:28.593 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.594+0000 7f995bc5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99540b9190 0x7f99541577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:28.593 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.594+0000 7f995bc5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99540cec50 0x7f9954156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:28.593 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.594+0000 7f995bc5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f995415dc10 0x7f99540b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:28.593 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.594+0000 7f995bc5d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540b6b20 con 0x7f995415dc10 2024-04-25T08:31:28.593 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.594+0000 7f995bc5d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540b69a0 con 0x7f99540b9190 2024-04-25T08:31:28.593 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.594+0000 7f995bc5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99540b6ca0 con 0x7f99540cec50 2024-04-25T08:31:28.593 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f995a1fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f995415dc10 0x7f99540b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:28.593 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f995a1fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f995415dc10 0x7f99540b7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33408/0 (socket says 172.21.15.38:33408) 2024-04-25T08:31:28.593 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f995a1fa640 1 -- 172.21.15.38:0/1436633762 learned_addr learned my addr 172.21.15.38:0/1436633762 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:28.593 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f99591f8640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99540cec50 0x7f9954156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f99599f9640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99540b9190 0x7f99541577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f995a1fa640 1 -- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99540cec50 msgr2=0x7f9954156e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f995a1fa640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99540cec50 0x7f9954156e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f995a1fa640 1 -- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99540b9190 msgr2=0x7f99541577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f995a1fa640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99540b9190 0x7f99541577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f995a1fa640 1 -- 172.21.15.38:0/1436633762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99541573a0 con 0x7f995415dc10 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f995a1fa640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f995415dc10 0x7f99540b7610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f994c00efb0 tx=0x7f994c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f994a7fc640 1 -- 172.21.15.38:0/1436633762 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c018020 con 0x7f995415dc10 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f994a7fc640 1 -- 172.21.15.38:0/1436633762 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c014480 con 0x7f995415dc10 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f995bc5d640 1 -- 172.21.15.38:0/1436633762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99541f6ba0 con 0x7f995415dc10 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f994a7fc640 1 -- 172.21.15.38:0/1436633762 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c014850 con 0x7f995415dc10 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.595+0000 7f995bc5d640 1 -- 172.21.15.38:0/1436633762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99541f7160 con 0x7f995415dc10 2024-04-25T08:31:28.594 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.596+0000 7f995bc5d640 1 -- 172.21.15.38:0/1436633762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9954074d30 con 0x7f995415dc10 2024-04-25T08:31:28.595 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.597+0000 7f994a7fc640 1 -- 172.21.15.38:0/1436633762 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f994c005d10 con 0x7f995415dc10 2024-04-25T08:31:28.595 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.597+0000 7f994a7fc640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f993806b500 0x7f993806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:28.595 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.597+0000 7f994a7fc640 1 -- 172.21.15.38:0/1436633762 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f994c08ab90 con 0x7f995415dc10 2024-04-25T08:31:28.595 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.598+0000 7f99599f9640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f993806b500 0x7f993806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:28.595 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.598+0000 7f99599f9640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f993806b500 0x7f993806d9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9950034000 tx=0x7f995002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:28.599 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.602+0000 7f994a7fc640 1 -- 172.21.15.38:0/1436633762 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f994c022030 con 0x7f995415dc10 2024-04-25T08:31:28.778 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.782+0000 7f995bc5d640 1 -- 172.21.15.38:0/1436633762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99540d0a90 con 0x7f995415dc10 2024-04-25T08:31:28.779 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.782+0000 7f994a7fc640 1 -- 172.21.15.38:0/1436633762 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12872 (secure 0 0 0) 0x7f99541573a0 con 0x7f995415dc10 2024-04-25T08:31:28.779 INFO:teuthology.misc.health.smithi038.stdout: 2024-04-25T08:31:28.779 INFO:teuthology.misc.health.smithi038.stdout:{"epoch":16,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:25.295929+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:28.781 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.784+0000 7f995bc5d640 1 -- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f993806b500 msgr2=0x7f993806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:28.781 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.784+0000 7f995bc5d640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f993806b500 0x7f993806d9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9950034000 tx=0x7f995002e000 comp rx=0 tx=0).stop 2024-04-25T08:31:28.781 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.784+0000 7f995bc5d640 1 -- 172.21.15.38:0/1436633762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f995415dc10 msgr2=0x7f99540b7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:28.781 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.784+0000 7f995bc5d640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f995415dc10 0x7f99540b7610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f994c00efb0 tx=0x7f994c00c490 comp rx=0 tx=0).stop 2024-04-25T08:31:28.781 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.784+0000 7f995bc5d640 1 -- 172.21.15.38:0/1436633762 shutdown_connections 2024-04-25T08:31:28.781 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.784+0000 7f995bc5d640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f995415dc10 0x7f99540b7610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.782 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.784+0000 7f995bc5d640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f993806b500 0x7f993806d9c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.782 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.784+0000 7f995bc5d640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99540cec50 0x7f9954156e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.782 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.784+0000 7f995bc5d640 1 --2- 172.21.15.38:0/1436633762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99540b9190 0x7f99541577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.782 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.784+0000 7f995bc5d640 1 -- 172.21.15.38:0/1436633762 >> 172.21.15.38:0/1436633762 conn(0x7f9954072710 msgr2=0x7f995415cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:28.782 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.785+0000 7f995bc5d640 1 -- 172.21.15.38:0/1436633762 shutdown_connections 2024-04-25T08:31:28.782 INFO:teuthology.misc.health.smithi038.stderr:2024-04-25T08:31:28.785+0000 7f995bc5d640 1 -- 172.21.15.38:0/1436633762 wait complete. 2024-04-25T08:31:28.791 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T08:31:28.791 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T08:31:28.791 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:31:28.791 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:31:28.794 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:31:28.794 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:31:28.794 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:31:28.796 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:31:28.796 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:31:28.796 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:31:28.799 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:31:28.799 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:31:28.799 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:31:28.801 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:31:28.801 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T08:31:28.801 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:28.869 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:31:28.869 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:31:28.881 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-25T08:31:28.881 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:31:28.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.910+0000 7f7c1ce1b640 1 Processor -- start 2024-04-25T08:31:28.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.910+0000 7f7c1ce1b640 1 -- start start 2024-04-25T08:31:28.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c1ce1b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c1815da90 0x7f7c180d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:28.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c1ce1b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7c180ceca0 0x7f7c180d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:28.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c1ce1b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c180d1b60 con 0x7f7c180d3de0 2024-04-25T08:31:28.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c1ce1b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c180d2bc0 con 0x7f7c180ceca0 2024-04-25T08:31:28.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c1ce1b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c180d1e00 con 0x7f7c1815da90 2024-04-25T08:31:28.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c15d9b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7c180ceca0 0x7f7c180d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:28.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c1659c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c1815da90 0x7f7c180d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:28.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c16d9d640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f7c180d3de0 0x7f7c180d0320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:51376/0 (socket says 172.21.15.38:51376) 2024-04-25T08:31:28.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c16d9d640 1 -- 172.21.15.38:0/2260538195 learned_addr learned my addr 172.21.15.38:0/2260538195 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:28.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c1659c640 1 -- 172.21.15.38:0/2260538195 >> v2:172.21.15.107:3300/0 conn(0x7f7c180ceca0 msgr2=0x7f7c180d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:28.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c1659c640 1 --2- 172.21.15.38:0/2260538195 >> v2:172.21.15.107:3300/0 conn(0x7f7c180ceca0 0x7f7c180d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c1559a640 1 -- 172.21.15.38:0/2260538195 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117587576 0 0) 0x7f7c180d1b60 con 0x7f7c180d3de0 2024-04-25T08:31:28.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c1659c640 1 -- 172.21.15.38:0/2260538195 >> v1:172.21.15.107:6789/0 conn(0x7f7c180d3de0 legacy=0x7f7c180d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:28.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.911+0000 7f7c1659c640 1 -- 172.21.15.38:0/2260538195 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c180d20e0 con 0x7f7c1815da90 2024-04-25T08:31:28.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.912+0000 7f7c1659c640 1 --2- 172.21.15.38:0/2260538195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c1815da90 0x7f7c180d2f70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0097a0 tx=0x7f7c0c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3096ce6b376c6ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:28.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.912+0000 7f7c1559a640 1 -- 172.21.15.38:0/2260538195 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c037070 con 0x7f7c1815da90 2024-04-25T08:31:28.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.912+0000 7f7c1559a640 1 -- 172.21.15.38:0/2260538195 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0c032440 con 0x7f7c1815da90 2024-04-25T08:31:28.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.912+0000 7f7c1559a640 1 -- 172.21.15.38:0/2260538195 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c032810 con 0x7f7c1815da90 2024-04-25T08:31:28.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.913+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/2260538195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c1815da90 msgr2=0x7f7c180d2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:28.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.913+0000 7f7c1ce1b640 1 --2- 172.21.15.38:0/2260538195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c1815da90 0x7f7c180d2f70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0097a0 tx=0x7f7c0c02a180 comp rx=0 tx=0).stop 2024-04-25T08:31:28.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.913+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/2260538195 shutdown_connections 2024-04-25T08:31:28.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.913+0000 7f7c1ce1b640 1 --2- 172.21.15.38:0/2260538195 >> v2:172.21.15.107:3300/0 conn(0x7f7c180ceca0 0x7f7c180d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.913+0000 7f7c1ce1b640 1 --2- 172.21.15.38:0/2260538195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c1815da90 0x7f7c180d2f70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.913+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/2260538195 >> 172.21.15.38:0/2260538195 conn(0x7f7c1806bed0 msgr2=0x7f7c1806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:28.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.913+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/2260538195 shutdown_connections 2024-04-25T08:31:28.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.913+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/2260538195 wait complete. 2024-04-25T08:31:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.914+0000 7f7c1ce1b640 1 Processor -- start 2024-04-25T08:31:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.914+0000 7f7c1ce1b640 1 -- start start 2024-04-25T08:31:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1ce1b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c180d3de0 0x7f7c1814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1ce1b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c1815da90 0x7f7c18160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1ce1b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c18160840 0x7f7c1814e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1ce1b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1815dff0 con 0x7f7c180d3de0 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1ce1b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1815de70 con 0x7f7c18160840 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1ce1b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c180d34b0 con 0x7f7c1815da90 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c180d3de0 0x7f7c1814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c180d3de0 0x7f7c1814dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39268/0 (socket says 172.21.15.38:39268) 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1659c640 1 -- 172.21.15.38:0/4080944539 learned_addr learned my addr 172.21.15.38:0/4080944539 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c15d9b640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c1815da90 0x7f7c18160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c16d9d640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c18160840 0x7f7c1814e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1659c640 1 -- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c1815da90 msgr2=0x7f7c18160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1659c640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c1815da90 0x7f7c18160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1659c640 1 -- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c18160840 msgr2=0x7f7c1814e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1659c640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c18160840 0x7f7c1814e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c1659c640 1 -- 172.21.15.38:0/4080944539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1814e290 con 0x7f7c180d3de0 2024-04-25T08:31:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.915+0000 7f7c16d9d640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c18160840 0x7f7c1814e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:28.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.916+0000 7f7c1659c640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c180d3de0 0x7f7c1814dd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0097a0 tx=0x7f7c0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:28.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.916+0000 7f7c06ffd640 1 -- 172.21.15.38:0/4080944539 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c037050 con 0x7f7c180d3de0 2024-04-25T08:31:28.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.916+0000 7f7c06ffd640 1 -- 172.21.15.38:0/4080944539 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0c038040 con 0x7f7c180d3de0 2024-04-25T08:31:28.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.916+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/4080944539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c1815f260 con 0x7f7c180d3de0 2024-04-25T08:31:28.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.916+0000 7f7c06ffd640 1 -- 172.21.15.38:0/4080944539 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c004490 con 0x7f7c180d3de0 2024-04-25T08:31:28.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.916+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/4080944539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c1815f820 con 0x7f7c180d3de0 2024-04-25T08:31:28.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.917+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/4080944539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c18068e80 con 0x7f7c180d3de0 2024-04-25T08:31:28.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.917+0000 7f7c06ffd640 1 -- 172.21.15.38:0/4080944539 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c0c03c6d0 con 0x7f7c180d3de0 2024-04-25T08:31:28.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.918+0000 7f7c06ffd640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7bf806b500 0x7f7bf806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:28.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.918+0000 7f7c06ffd640 1 -- 172.21.15.38:0/4080944539 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7c0c0aa9b0 con 0x7f7c180d3de0 2024-04-25T08:31:28.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.918+0000 7f7c15d9b640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7bf806b500 0x7f7bf806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:28.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.918+0000 7f7c15d9b640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7bf806b500 0x7f7bf806d9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7c180b8070 tx=0x7f7c08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:28.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:28.922+0000 7f7c06ffd640 1 -- 172.21.15.38:0/4080944539 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c1814e290 con 0x7f7c180d3de0 2024-04-25T08:31:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.103+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/4080944539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c180c71f0 con 0x7f7c180d3de0 2024-04-25T08:31:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.104+0000 7f7c06ffd640 1 -- 172.21.15.38:0/4080944539 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12872 (secure 0 0 0) 0x7f7c0c079af0 con 0x7f7c180d3de0 2024-04-25T08:31:29.101 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:29.101 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":16,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:25.295929+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.106+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7bf806b500 msgr2=0x7f7bf806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.106+0000 7f7c1ce1b640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7bf806b500 0x7f7bf806d9c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7c180b8070 tx=0x7f7c08008040 comp rx=0 tx=0).stop 2024-04-25T08:31:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.106+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/4080944539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c180d3de0 msgr2=0x7f7c1814dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.106+0000 7f7c1ce1b640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c180d3de0 0x7f7c1814dd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0097a0 tx=0x7f7c0c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.106+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/4080944539 shutdown_connections 2024-04-25T08:31:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.106+0000 7f7c1ce1b640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c18160840 0x7f7c1814e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.106+0000 7f7c1ce1b640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7bf806b500 0x7f7bf806d9c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.106+0000 7f7c1ce1b640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c1815da90 0x7f7c18160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.106+0000 7f7c1ce1b640 1 --2- 172.21.15.38:0/4080944539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c180d3de0 0x7f7c1814dd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.106+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/4080944539 >> 172.21.15.38:0/4080944539 conn(0x7f7c1806bed0 msgr2=0x7f7c1806c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.106+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/4080944539 shutdown_connections 2024-04-25T08:31:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.106+0000 7f7c1ce1b640 1 -- 172.21.15.38:0/4080944539 wait complete. 2024-04-25T08:31:29.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:31:29.114 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:31:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.221+0000 7fc9568fc640 1 Processor -- start 2024-04-25T08:31:29.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.221+0000 7fc9568fc640 1 -- start start 2024-04-25T08:31:29.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc9568fc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc9500d0320 0x7fc9500d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc9568fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9500b6980 0x7fc9500b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc9568fc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9500d2060 con 0x7fc9500d0700 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc9568fc640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9500d2330 con 0x7fc9500d0320 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc9568fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500d2610 con 0x7fc9500b6980 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc94f7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc9500d0700 0x7fc9500d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32954/0 (socket says 172.21.15.38:32954) 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc94f7fe640 1 -- 172.21.15.38:0/1851239391 learned_addr learned my addr 172.21.15.38:0/1851239391 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc94ffff640 1 --2- 172.21.15.38:0/1851239391 >> v2:172.21.15.107:3300/0 conn(0x7fc9500d0320 0x7fc9500d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc954e99640 1 --2- 172.21.15.38:0/1851239391 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9500b6980 0x7fc9500b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc94effd640 1 -- 172.21.15.38:0/1851239391 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 426018288 0 0) 0x7fc9500d2060 con 0x7fc9500d0700 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc94effd640 1 -- 172.21.15.38:0/1851239391 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc938003520 con 0x7fc9500d0700 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc954e99640 1 -- 172.21.15.38:0/1851239391 >> v2:172.21.15.107:3300/0 conn(0x7fc9500d0320 msgr2=0x7fc9500d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc954e99640 1 --2- 172.21.15.38:0/1851239391 >> v2:172.21.15.107:3300/0 conn(0x7fc9500d0320 0x7fc9500d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc954e99640 1 -- 172.21.15.38:0/1851239391 >> v1:172.21.15.38:6789/0 conn(0x7fc9500d0700 legacy=0x7fc9500d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.222+0000 7fc954e99640 1 -- 172.21.15.38:0/1851239391 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9500d28f0 con 0x7fc9500b6980 2024-04-25T08:31:29.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.223+0000 7fc954e99640 1 --2- 172.21.15.38:0/1851239391 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9500b6980 0x7fc9500b6e10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc948009d90 tx=0x7fc94802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bc8b7c3fe0f2b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:29.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.223+0000 7fc94effd640 1 -- 172.21.15.38:0/1851239391 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948007740 con 0x7fc9500b6980 2024-04-25T08:31:29.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.223+0000 7fc94effd640 1 -- 172.21.15.38:0/1851239391 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9480078e0 con 0x7fc9500b6980 2024-04-25T08:31:29.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.223+0000 7fc94effd640 1 -- 172.21.15.38:0/1851239391 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948007cb0 con 0x7fc9500b6980 2024-04-25T08:31:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.224+0000 7fc9568fc640 1 -- 172.21.15.38:0/1851239391 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9500b6980 msgr2=0x7fc9500b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.224+0000 7fc9568fc640 1 --2- 172.21.15.38:0/1851239391 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9500b6980 0x7fc9500b6e10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc948009d90 tx=0x7fc94802a6f0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.224+0000 7fc9568fc640 1 -- 172.21.15.38:0/1851239391 shutdown_connections 2024-04-25T08:31:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.224+0000 7fc9568fc640 1 --2- 172.21.15.38:0/1851239391 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9500b6980 0x7fc9500b6e10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.224+0000 7fc9568fc640 1 --2- 172.21.15.38:0/1851239391 >> v2:172.21.15.107:3300/0 conn(0x7fc9500d0320 0x7fc9500d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.224+0000 7fc9568fc640 1 -- 172.21.15.38:0/1851239391 >> 172.21.15.38:0/1851239391 conn(0x7fc9500668f0 msgr2=0x7fc950066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.224+0000 7fc9568fc640 1 -- 172.21.15.38:0/1851239391 shutdown_connections 2024-04-25T08:31:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.224+0000 7fc9568fc640 1 -- 172.21.15.38:0/1851239391 wait complete. 2024-04-25T08:31:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.225+0000 7fc9568fc640 1 Processor -- start 2024-04-25T08:31:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.225+0000 7fc9568fc640 1 -- start start 2024-04-25T08:31:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc9568fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9500b6980 0x7fc950153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc9568fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9500d0320 0x7fc9501554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc9568fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9500d0700 0x7fc950158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc9568fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500d0b40 con 0x7fc9500d0320 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc9568fc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500d2eb0 con 0x7fc9500b6980 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc9568fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9500b3310 con 0x7fc9500d0700 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9500d0320 0x7fc9501554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9500d0320 0x7fc9501554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39276/0 (socket says 172.21.15.38:39276) 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94f7fe640 1 -- 172.21.15.38:0/4155494984 learned_addr learned my addr 172.21.15.38:0/4155494984 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc954e99640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9500d0700 0x7fc950158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94ffff640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9500b6980 0x7fc950153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94f7fe640 1 -- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9500d0700 msgr2=0x7fc950158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94f7fe640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9500d0700 0x7fc950158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94f7fe640 1 -- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9500b6980 msgr2=0x7fc950153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94f7fe640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9500b6980 0x7fc950153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94f7fe640 1 -- 172.21.15.38:0/4155494984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9500b35f0 con 0x7fc9500d0320 2024-04-25T08:31:29.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94f7fe640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9500d0320 0x7fc9501554c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c390 tx=0x7fc94400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:29.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94cff9640 1 -- 172.21.15.38:0/4155494984 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944017070 con 0x7fc9500d0320 2024-04-25T08:31:29.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94cff9640 1 -- 172.21.15.38:0/4155494984 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc944005ce0 con 0x7fc9500d0320 2024-04-25T08:31:29.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc9568fc640 1 -- 172.21.15.38:0/4155494984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9500b38d0 con 0x7fc9500d0320 2024-04-25T08:31:29.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc94cff9640 1 -- 172.21.15.38:0/4155494984 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9440072f0 con 0x7fc9500d0320 2024-04-25T08:31:29.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.226+0000 7fc9568fc640 1 -- 172.21.15.38:0/4155494984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9500b3e90 con 0x7fc9500d0320 2024-04-25T08:31:29.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.227+0000 7fc9568fc640 1 -- 172.21.15.38:0/4155494984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc950068e80 con 0x7fc9500d0320 2024-04-25T08:31:29.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.228+0000 7fc94cff9640 1 -- 172.21.15.38:0/4155494984 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc94401a070 con 0x7fc9500d0320 2024-04-25T08:31:29.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.228+0000 7fc94cff9640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc93c06b4d0 0x7fc93c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.228+0000 7fc94cff9640 1 -- 172.21.15.38:0/4155494984 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc944089210 con 0x7fc9500d0320 2024-04-25T08:31:29.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.228+0000 7fc94ffff640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc93c06b4d0 0x7fc93c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.229+0000 7fc94ffff640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc93c06b4d0 0x7fc93c06d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc940009b40 tx=0x7fc940009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:29.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.232+0000 7fc94cff9640 1 -- 172.21.15.38:0/4155494984 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc944057d90 con 0x7fc9500d0320 2024-04-25T08:31:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.412+0000 7fc9568fc640 1 -- 172.21.15.38:0/4155494984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9500b3050 con 0x7fc9500d0320 2024-04-25T08:31:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.413+0000 7fc94cff9640 1 -- 172.21.15.38:0/4155494984 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc94405be30 con 0x7fc9500d0320 2024-04-25T08:31:29.410 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:31:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.415+0000 7fc9568fc640 1 -- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc93c06b4d0 msgr2=0x7fc93c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.415+0000 7fc9568fc640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc93c06b4d0 0x7fc93c06d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc940009b40 tx=0x7fc940009190 comp rx=0 tx=0).stop 2024-04-25T08:31:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.415+0000 7fc9568fc640 1 -- 172.21.15.38:0/4155494984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9500d0320 msgr2=0x7fc9501554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.415+0000 7fc9568fc640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9500d0320 0x7fc9501554c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c390 tx=0x7fc94400c850 comp rx=0 tx=0).stop 2024-04-25T08:31:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.415+0000 7fc9568fc640 1 -- 172.21.15.38:0/4155494984 shutdown_connections 2024-04-25T08:31:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.415+0000 7fc9568fc640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc93c06b4d0 0x7fc93c06d990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.415+0000 7fc9568fc640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9500d0700 0x7fc950158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.415+0000 7fc9568fc640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9500d0320 0x7fc9501554c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.415+0000 7fc9568fc640 1 --2- 172.21.15.38:0/4155494984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9500b6980 0x7fc950153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.415+0000 7fc9568fc640 1 -- 172.21.15.38:0/4155494984 >> 172.21.15.38:0/4155494984 conn(0x7fc9500668f0 msgr2=0x7fc95015f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.415+0000 7fc9568fc640 1 -- 172.21.15.38:0/4155494984 shutdown_connections 2024-04-25T08:31:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.415+0000 7fc9568fc640 1 -- 172.21.15.38:0/4155494984 wait complete. 2024-04-25T08:31:29.422 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:31:29.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.526+0000 7f22b6970640 1 Processor -- start 2024-04-25T08:31:29.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.526+0000 7f22b6970640 1 -- start start 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22b6970640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f22b015dc10 0x7f22b00ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22b6970640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22b00ced60 0x7f22b00d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22b6970640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f22b00d3900 0x7f22b00d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22b6970640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b015dff0 con 0x7f22b015dc10 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22b6970640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b00b6990 con 0x7f22b00d3900 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22b6970640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00d4710 con 0x7f22b00ced60 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22affff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f22b015dc10 0x7f22b00ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22affff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f22b015dc10 0x7f22b00ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39286/0 (socket says 172.21.15.38:39286) 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22affff640 1 -- 172.21.15.38:0/2559645843 learned_addr learned my addr 172.21.15.38:0/2559645843 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22af7fe640 1 --2- 172.21.15.38:0/2559645843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22b00ced60 0x7f22b00d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22b4f0d640 1 --2- 172.21.15.38:0/2559645843 >> v2:172.21.15.107:3300/0 conn(0x7f22b00d3900 0x7f22b00d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22affff640 1 -- 172.21.15.38:0/2559645843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22b00ced60 msgr2=0x7f22b00d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:29.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22affff640 1 --2- 172.21.15.38:0/2559645843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22b00ced60 0x7f22b00d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22affff640 1 -- 172.21.15.38:0/2559645843 >> v2:172.21.15.107:3300/0 conn(0x7f22b00d3900 msgr2=0x7f22b00d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22affff640 1 --2- 172.21.15.38:0/2559645843 >> v2:172.21.15.107:3300/0 conn(0x7f22b00d3900 0x7f22b00d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22affff640 1 -- 172.21.15.38:0/2559645843 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b00d3e70 con 0x7f22b015dc10 2024-04-25T08:31:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.527+0000 7f22b4f0d640 1 --2- 172.21.15.38:0/2559645843 >> v2:172.21.15.107:3300/0 conn(0x7f22b00d3900 0x7f22b00d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.528+0000 7f22affff640 1 --2- 172.21.15.38:0/2559645843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b015dc10 0x7f22b00ce820 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f22b00b9380 tx=0x7f22a402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0095136e74404c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.528+0000 7f22aeffd640 1 -- 172.21.15.38:0/2559645843 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4037070 con 0x7f22b015dc10 2024-04-25T08:31:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.528+0000 7f22aeffd640 1 -- 172.21.15.38:0/2559645843 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a4032440 con 0x7f22b015dc10 2024-04-25T08:31:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.528+0000 7f22aeffd640 1 -- 172.21.15.38:0/2559645843 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4032810 con 0x7f22b015dc10 2024-04-25T08:31:29.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.529+0000 7f22b6970640 1 -- 172.21.15.38:0/2559645843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b015dc10 msgr2=0x7f22b00ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.529+0000 7f22b6970640 1 --2- 172.21.15.38:0/2559645843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b015dc10 0x7f22b00ce820 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f22b00b9380 tx=0x7f22a402a140 comp rx=0 tx=0).stop 2024-04-25T08:31:29.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.529+0000 7f22b6970640 1 -- 172.21.15.38:0/2559645843 shutdown_connections 2024-04-25T08:31:29.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.529+0000 7f22b6970640 1 --2- 172.21.15.38:0/2559645843 >> v2:172.21.15.107:3300/0 conn(0x7f22b00d3900 0x7f22b00d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.529+0000 7f22b6970640 1 --2- 172.21.15.38:0/2559645843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22b00ced60 0x7f22b00d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.529+0000 7f22b6970640 1 --2- 172.21.15.38:0/2559645843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b015dc10 0x7f22b00ce820 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.529+0000 7f22b6970640 1 -- 172.21.15.38:0/2559645843 >> 172.21.15.38:0/2559645843 conn(0x7f22b0066d10 msgr2=0x7f22b0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:29.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.529+0000 7f22b6970640 1 -- 172.21.15.38:0/2559645843 shutdown_connections 2024-04-25T08:31:29.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.529+0000 7f22b6970640 1 -- 172.21.15.38:0/2559645843 wait complete. 2024-04-25T08:31:29.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.530+0000 7f22b6970640 1 Processor -- start 2024-04-25T08:31:29.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.530+0000 7f22b6970640 1 -- start start 2024-04-25T08:31:29.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22b6970640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22b00ced60 0x7f22b01f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22b6970640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b00d3900 0x7f22b0159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22b6970640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b015dc10 0x7f22b01524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22b6970640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00b6bc0 con 0x7f22b00d3900 2024-04-25T08:31:29.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22b6970640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00d4430 con 0x7f22b015dc10 2024-04-25T08:31:29.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22b6970640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00b2370 con 0x7f22b00ced60 2024-04-25T08:31:29.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b00d3900 0x7f22b0159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b00d3900 0x7f22b0159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39298/0 (socket says 172.21.15.38:39298) 2024-04-25T08:31:29.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22af7fe640 1 -- 172.21.15.38:0/1716445815 learned_addr learned my addr 172.21.15.38:0/1716445815 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:29.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22affff640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22b00ced60 0x7f22b01f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22b4f0d640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b015dc10 0x7f22b01524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22af7fe640 1 -- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22b00ced60 msgr2=0x7f22b01f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22af7fe640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22b00ced60 0x7f22b01f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22af7fe640 1 -- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b015dc10 msgr2=0x7f22b01524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22af7fe640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b015dc10 0x7f22b01524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22af7fe640 1 -- 172.21.15.38:0/1716445815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b01f3e40 con 0x7f22b00d3900 2024-04-25T08:31:29.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22b4f0d640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b015dc10 0x7f22b01524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:29.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22af7fe640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b00d3900 0x7f22b0159b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f22a000e890 tx=0x7f22a000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:29.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22acff9640 1 -- 172.21.15.38:0/1716445815 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0017070 con 0x7f22b00d3900 2024-04-25T08:31:29.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22acff9640 1 -- 172.21.15.38:0/1716445815 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a000c940 con 0x7f22b00d3900 2024-04-25T08:31:29.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.531+0000 7f22b6970640 1 -- 172.21.15.38:0/1716445815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b01f4120 con 0x7f22b00d3900 2024-04-25T08:31:29.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.532+0000 7f22acff9640 1 -- 172.21.15.38:0/1716445815 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a000cd10 con 0x7f22b00d3900 2024-04-25T08:31:29.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.532+0000 7f22b6970640 1 -- 172.21.15.38:0/1716445815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b00cbbc0 con 0x7f22b00d3900 2024-04-25T08:31:29.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.533+0000 7f22b6970640 1 -- 172.21.15.38:0/1716445815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b0068e80 con 0x7f22b00d3900 2024-04-25T08:31:29.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.533+0000 7f22acff9640 1 -- 172.21.15.38:0/1716445815 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22a001a070 con 0x7f22b00d3900 2024-04-25T08:31:29.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.533+0000 7f22acff9640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f229c06b4d0 0x7f229c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:29.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.533+0000 7f22acff9640 1 -- 172.21.15.38:0/1716445815 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f22a008a0c0 con 0x7f22b00d3900 2024-04-25T08:31:29.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.534+0000 7f22affff640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f229c06b4d0 0x7f229c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:29.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.534+0000 7f22affff640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f229c06b4d0 0x7f229c06d990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f22a4034000 tx=0x7f22a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:29.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.538+0000 7f22acff9640 1 -- 172.21.15.38:0/1716445815 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22a0058cd0 con 0x7f22b00d3900 2024-04-25T08:31:29.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.757+0000 7f22b6970640 1 -- 172.21.15.38:0/1716445815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22b00d1b50 con 0x7f22b00d3900 2024-04-25T08:31:29.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.758+0000 7f22acff9640 1 -- 172.21.15.38:0/1716445815 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1321 (secure 0 0 0) 0x7f22a005cd70 con 0x7f22b00d3900 2024-04-25T08:31:29.755 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:29.755 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/678452936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":678452936},{"type":"v1","addr":"172.21.15.38:6834","nonce":678452936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T08:31:29.755 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 2 2024-04-25T08:31:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.760+0000 7f22b6970640 1 -- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f229c06b4d0 msgr2=0x7f229c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.760+0000 7f22b6970640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f229c06b4d0 0x7f229c06d990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f22a4034000 tx=0x7f22a402e000 comp rx=0 tx=0).stop 2024-04-25T08:31:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.760+0000 7f22b6970640 1 -- 172.21.15.38:0/1716445815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b00d3900 msgr2=0x7f22b0159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.760+0000 7f22b6970640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b00d3900 0x7f22b0159b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f22a000e890 tx=0x7f22a000ed50 comp rx=0 tx=0).stop 2024-04-25T08:31:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.760+0000 7f22b6970640 1 -- 172.21.15.38:0/1716445815 shutdown_connections 2024-04-25T08:31:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.760+0000 7f22b6970640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22b015dc10 0x7f22b01524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.760+0000 7f22b6970640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22b00d3900 0x7f22b0159b50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.760+0000 7f22b6970640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f229c06b4d0 0x7f229c06d990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.760+0000 7f22b6970640 1 --2- 172.21.15.38:0/1716445815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22b00ced60 0x7f22b01f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.760+0000 7f22b6970640 1 -- 172.21.15.38:0/1716445815 >> 172.21.15.38:0/1716445815 conn(0x7f22b0066d10 msgr2=0x7f22b00b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.760+0000 7f22b6970640 1 -- 172.21.15.38:0/1716445815 shutdown_connections 2024-04-25T08:31:29.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:29.760+0000 7f22b6970640 1 -- 172.21.15.38:0/1716445815 wait complete. 2024-04-25T08:31:31.768 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:31:31.787 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:31:31.814 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:31:31.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.885+0000 7fbc8b0ac640 1 Processor -- start 2024-04-25T08:31:31.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.885+0000 7fbc8b0ac640 1 -- start start 2024-04-25T08:31:31.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc8b0ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc8415dc10 0x7fbc840ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:31.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc8b0ac640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbc840ced60 0x7fbc840d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:31.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc8b0ac640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc840d3e70 con 0x7fbc840d3900 2024-04-25T08:31:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc8b0ac640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc8415dff0 con 0x7fbc840ced60 2024-04-25T08:31:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc8b0ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840b6990 con 0x7fbc8415dc10 2024-04-25T08:31:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc89649640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbc840d3900 0x7fbc840b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32966/0 (socket says 172.21.15.38:32966) 2024-04-25T08:31:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc89649640 1 -- 172.21.15.38:0/3944278079 learned_addr learned my addr 172.21.15.38:0/3944278079 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc83fff640 1 --2- 172.21.15.38:0/3944278079 >> v2:172.21.15.107:3300/0 conn(0x7fbc840ced60 0x7fbc840d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc88e48640 1 --2- 172.21.15.38:0/3944278079 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc8415dc10 0x7fbc840ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2270192674 0 0) 0x7fbc840d3e70 con 0x7fbc840d3900 2024-04-25T08:31:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc74003520 con 0x7fbc840d3900 2024-04-25T08:31:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1729053592 0 0) 0x7fbc74003520 con 0x7fbc840d3900 2024-04-25T08:31:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc8415dc10 msgr2=0x7fbc840ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc837fe640 1 --2- 172.21.15.38:0/3944278079 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc8415dc10 0x7fbc840ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 >> v2:172.21.15.107:3300/0 conn(0x7fbc840ced60 msgr2=0x7fbc840d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc837fe640 1 --2- 172.21.15.38:0/3944278079 >> v2:172.21.15.107:3300/0 conn(0x7fbc840ced60 0x7fbc840d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.886+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc840d4150 con 0x7fbc840d3900 2024-04-25T08:31:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fbc7c0019f0 con 0x7fbc840d3900 2024-04-25T08:31:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 >> v1:172.21.15.38:6789/0 conn(0x7fbc840d3900 legacy=0x7fbc840b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc837fe640 1 --2- 172.21.15.38:0/3944278079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc740039b0 0x7fbc74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc837fe640 1 --2- 172.21.15.38:0/3944278079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc74004800 0x7fbc74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc837fe640 1 --2- 172.21.15.38:0/3944278079 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc8415dc10 0x7fbc740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8415dff0 con 0x7fbc740039b0 2024-04-25T08:31:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7400b9c0 con 0x7fbc74004800 2024-04-25T08:31:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7400bb40 con 0x7fbc8415dc10 2024-04-25T08:31:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc88e48640 1 --2- 172.21.15.38:0/3944278079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc740039b0 0x7fbc74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:31.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc83fff640 1 --2- 172.21.15.38:0/3944278079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc74004800 0x7fbc74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc89649640 1 --2- 172.21.15.38:0/3944278079 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc8415dc10 0x7fbc740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc88e48640 1 -- 172.21.15.38:0/3944278079 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc8415dc10 msgr2=0x7fbc740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc88e48640 1 --2- 172.21.15.38:0/3944278079 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc8415dc10 0x7fbc740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc88e48640 1 -- 172.21.15.38:0/3944278079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc74004800 msgr2=0x7fbc74004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc88e48640 1 --2- 172.21.15.38:0/3944278079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc74004800 0x7fbc74004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.887+0000 7fbc88e48640 1 -- 172.21.15.38:0/3944278079 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc74003520 con 0x7fbc740039b0 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.888+0000 7fbc88e48640 1 --2- 172.21.15.38:0/3944278079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc740039b0 0x7fbc74003dd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009af0 tx=0x7fbc74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca64cd461655f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.888+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78036070 con 0x7fbc740039b0 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.888+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc78031400 con 0x7fbc740039b0 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.888+0000 7fbc837fe640 1 -- 172.21.15.38:0/3944278079 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78009690 con 0x7fbc740039b0 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.888+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/3944278079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc740039b0 msgr2=0x7fbc74003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.888+0000 7fbc8b0ac640 1 --2- 172.21.15.38:0/3944278079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc740039b0 0x7fbc74003dd0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc78009af0 tx=0x7fbc74005830 comp rx=0 tx=0).stop 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.888+0000 7fbc89649640 1 -- 172.21.15.38:0/3944278079 reap_dead start 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.888+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/3944278079 shutdown_connections 2024-04-25T08:31:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.888+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/3944278079 >> 172.21.15.38:0/3944278079 conn(0x7fbc84066d10 msgr2=0x7fbc84067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:31.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.888+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/3944278079 shutdown_connections 2024-04-25T08:31:31.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.888+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/3944278079 wait complete. 2024-04-25T08:31:31.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.890+0000 7fbc8b0ac640 1 Processor -- start 2024-04-25T08:31:31.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.890+0000 7fbc8b0ac640 1 -- start start 2024-04-25T08:31:31.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc8b0ac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc740039b0 0x7fbc84150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:31.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc8b0ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc840caea0 0x7fbc84157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:31.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc8b0ac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc84158240 0x7fbc841548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:31.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc8b0ac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840b6bc0 con 0x7fbc740039b0 2024-04-25T08:31:31.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc8b0ac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840d4710 con 0x7fbc84158240 2024-04-25T08:31:31.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc8b0ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc840ce820 con 0x7fbc840caea0 2024-04-25T08:31:31.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc88e48640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc740039b0 0x7fbc84150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:31.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc88e48640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc740039b0 0x7fbc84150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39304/0 (socket says 172.21.15.38:39304) 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc88e48640 1 -- 172.21.15.38:0/4023886435 learned_addr learned my addr 172.21.15.38:0/4023886435 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc83fff640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc840caea0 0x7fbc84157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc88e48640 1 -- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc840caea0 msgr2=0x7fbc84157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc88e48640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc840caea0 0x7fbc84157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc89649640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc84158240 0x7fbc841548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc88e48640 1 -- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc84158240 msgr2=0x7fbc841548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc88e48640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc84158240 0x7fbc841548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc88e48640 1 -- 172.21.15.38:0/4023886435 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc84156c60 con 0x7fbc740039b0 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc83fff640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc840caea0 0x7fbc84157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc89649640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc84158240 0x7fbc841548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc88e48640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc740039b0 0x7fbc84150560 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc78034040 tx=0x7fbc78004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc817fa640 1 -- 172.21.15.38:0/4023886435 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78041020 con 0x7fbc740039b0 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc817fa640 1 -- 172.21.15.38:0/4023886435 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7803c470 con 0x7fbc740039b0 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/4023886435 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc84156f40 con 0x7fbc740039b0 2024-04-25T08:31:31.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc817fa640 1 -- 172.21.15.38:0/4023886435 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78036040 con 0x7fbc740039b0 2024-04-25T08:31:31.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.891+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/4023886435 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc84157500 con 0x7fbc740039b0 2024-04-25T08:31:31.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.893+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/4023886435 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc84068e80 con 0x7fbc740039b0 2024-04-25T08:31:31.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.893+0000 7fbc817fa640 1 -- 172.21.15.38:0/4023886435 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc78044020 con 0x7fbc740039b0 2024-04-25T08:31:31.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.893+0000 7fbc817fa640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbc6806b4d0 0x7fbc6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:31.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.893+0000 7fbc817fa640 1 -- 172.21.15.38:0/4023886435 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbc780aa610 con 0x7fbc740039b0 2024-04-25T08:31:31.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.894+0000 7fbc83fff640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbc6806b4d0 0x7fbc6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:31.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.894+0000 7fbc83fff640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbc6806b4d0 0x7fbc6806d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbc84153580 tx=0x7fbc70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:31.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:31.898+0000 7fbc817fa640 1 -- 172.21.15.38:0/4023886435 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc7804b030 con 0x7fbc740039b0 2024-04-25T08:31:32.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.114+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/4023886435 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc840d2710 con 0x7fbc740039b0 2024-04-25T08:31:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.114+0000 7fbc817fa640 1 -- 172.21.15.38:0/4023886435 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7fbc780797e0 con 0x7fbc740039b0 2024-04-25T08:31:32.111 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:32.111 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4242612806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4242612806},{"type":"v1","addr":"172.21.15.38:6835","nonce":4242612806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2154167637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":2154167637},{"type":"v1","addr":"172.21.15.107:6837","nonce":2154167637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/678452936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":678452936},{"type":"v1","addr":"172.21.15.38:6834","nonce":678452936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4294,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/1104921666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1104921666},{"type":"v1","addr":"172.21.15.107:6836","nonce":1104921666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T08:31:32.111 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 3 2024-04-25T08:31:32.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.116+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbc6806b4d0 msgr2=0x7fbc6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.116+0000 7fbc8b0ac640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbc6806b4d0 0x7fbc6806d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbc84153580 tx=0x7fbc70008040 comp rx=0 tx=0).stop 2024-04-25T08:31:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.116+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/4023886435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc740039b0 msgr2=0x7fbc84150560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.116+0000 7fbc8b0ac640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc740039b0 0x7fbc84150560 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbc78034040 tx=0x7fbc78004730 comp rx=0 tx=0).stop 2024-04-25T08:31:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.116+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/4023886435 shutdown_connections 2024-04-25T08:31:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.116+0000 7fbc8b0ac640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc84158240 0x7fbc841548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.117+0000 7fbc8b0ac640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbc6806b4d0 0x7fbc6806d990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.117+0000 7fbc8b0ac640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc840caea0 0x7fbc84157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.117+0000 7fbc8b0ac640 1 --2- 172.21.15.38:0/4023886435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc740039b0 0x7fbc84150560 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.117+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/4023886435 >> 172.21.15.38:0/4023886435 conn(0x7fbc84066d10 msgr2=0x7fbc840b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.117+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/4023886435 shutdown_connections 2024-04-25T08:31:32.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.117+0000 7fbc8b0ac640 1 -- 172.21.15.38:0/4023886435 wait complete. 2024-04-25T08:31:32.124 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.268+0000 7fbb18053640 1 Processor -- start 2024-04-25T08:31:32.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.268+0000 7fbb18053640 1 -- start start 2024-04-25T08:31:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb18053640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbb1015dc10 0x7fbb100ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb18053640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb100ced60 0x7fbb100d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb18053640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbb100d3900 0x7fbb100d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb18053640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1015dff0 con 0x7fbb1015dc10 2024-04-25T08:31:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb18053640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb100b6990 con 0x7fbb100d3900 2024-04-25T08:31:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb18053640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100d4710 con 0x7fbb100ced60 2024-04-25T08:31:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb15def640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbb1015dc10 0x7fbb100ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb15def640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbb1015dc10 0x7fbb100ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39320/0 (socket says 172.21.15.38:39320) 2024-04-25T08:31:32.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb15def640 1 -- 172.21.15.38:0/3576508784 learned_addr learned my addr 172.21.15.38:0/3576508784 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb165f0640 1 --2- 172.21.15.38:0/3576508784 >> v2:172.21.15.107:3300/0 conn(0x7fbb100d3900 0x7fbb100d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb155ee640 1 --2- 172.21.15.38:0/3576508784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb100ced60 0x7fbb100d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb15def640 1 -- 172.21.15.38:0/3576508784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb100ced60 msgr2=0x7fbb100d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb15def640 1 --2- 172.21.15.38:0/3576508784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb100ced60 0x7fbb100d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb15def640 1 -- 172.21.15.38:0/3576508784 >> v2:172.21.15.107:3300/0 conn(0x7fbb100d3900 msgr2=0x7fbb100d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb15def640 1 --2- 172.21.15.38:0/3576508784 >> v2:172.21.15.107:3300/0 conn(0x7fbb100d3900 0x7fbb100d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb15def640 1 -- 172.21.15.38:0/3576508784 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100d3e70 con 0x7fbb1015dc10 2024-04-25T08:31:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.269+0000 7fbb165f0640 1 --2- 172.21.15.38:0/3576508784 >> v2:172.21.15.107:3300/0 conn(0x7fbb100d3900 0x7fbb100d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.270+0000 7fbb15def640 1 --2- 172.21.15.38:0/3576508784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb1015dc10 0x7fbb100ce820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb100b9380 tx=0x7fbb0c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11008ed3333f9515 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.270+0000 7fbb14ded640 1 -- 172.21.15.38:0/3576508784 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c037070 con 0x7fbb1015dc10 2024-04-25T08:31:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.270+0000 7fbb14ded640 1 -- 172.21.15.38:0/3576508784 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c032440 con 0x7fbb1015dc10 2024-04-25T08:31:32.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.270+0000 7fbb14ded640 1 -- 172.21.15.38:0/3576508784 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c032810 con 0x7fbb1015dc10 2024-04-25T08:31:32.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.271+0000 7fbb18053640 1 -- 172.21.15.38:0/3576508784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb1015dc10 msgr2=0x7fbb100ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.271+0000 7fbb18053640 1 --2- 172.21.15.38:0/3576508784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb1015dc10 0x7fbb100ce820 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb100b9380 tx=0x7fbb0c02a140 comp rx=0 tx=0).stop 2024-04-25T08:31:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.271+0000 7fbb18053640 1 -- 172.21.15.38:0/3576508784 shutdown_connections 2024-04-25T08:31:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.271+0000 7fbb18053640 1 --2- 172.21.15.38:0/3576508784 >> v2:172.21.15.107:3300/0 conn(0x7fbb100d3900 0x7fbb100d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.271+0000 7fbb18053640 1 --2- 172.21.15.38:0/3576508784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb100ced60 0x7fbb100d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.271+0000 7fbb18053640 1 --2- 172.21.15.38:0/3576508784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb1015dc10 0x7fbb100ce820 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.271+0000 7fbb18053640 1 -- 172.21.15.38:0/3576508784 >> 172.21.15.38:0/3576508784 conn(0x7fbb10066d10 msgr2=0x7fbb10067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.271+0000 7fbb18053640 1 -- 172.21.15.38:0/3576508784 shutdown_connections 2024-04-25T08:31:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.271+0000 7fbb18053640 1 -- 172.21.15.38:0/3576508784 wait complete. 2024-04-25T08:31:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.272+0000 7fbb18053640 1 Processor -- start 2024-04-25T08:31:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.272+0000 7fbb18053640 1 -- start start 2024-04-25T08:31:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb18053640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb100ced60 0x7fbb101f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb18053640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb100d3900 0x7fbb10159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb18053640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb1015dc10 0x7fbb101524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb18053640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100b6bc0 con 0x7fbb100ced60 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb18053640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100d4430 con 0x7fbb100d3900 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb18053640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100b2370 con 0x7fbb1015dc10 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb15def640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb100ced60 0x7fbb101f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb15def640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb100ced60 0x7fbb101f3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39322/0 (socket says 172.21.15.38:39322) 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb15def640 1 -- 172.21.15.38:0/787057420 learned_addr learned my addr 172.21.15.38:0/787057420 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb155ee640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb100d3900 0x7fbb10159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb165f0640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb1015dc10 0x7fbb101524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb15def640 1 -- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb1015dc10 msgr2=0x7fbb101524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb15def640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb1015dc10 0x7fbb101524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb15def640 1 -- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb100d3900 msgr2=0x7fbb10159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb15def640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb100d3900 0x7fbb10159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.273+0000 7fbb15def640 1 -- 172.21.15.38:0/787057420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb101f3e40 con 0x7fbb100ced60 2024-04-25T08:31:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.274+0000 7fbb15def640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb100ced60 0x7fbb101f3740 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c000c00 tx=0x7fbb0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.274+0000 7fbb027fc640 1 -- 172.21.15.38:0/787057420 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c041070 con 0x7fbb100ced60 2024-04-25T08:31:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.274+0000 7fbb027fc640 1 -- 172.21.15.38:0/787057420 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c03c870 con 0x7fbb100ced60 2024-04-25T08:31:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.274+0000 7fbb18053640 1 -- 172.21.15.38:0/787057420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb101f4120 con 0x7fbb100ced60 2024-04-25T08:31:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.274+0000 7fbb027fc640 1 -- 172.21.15.38:0/787057420 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c037040 con 0x7fbb100ced60 2024-04-25T08:31:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.274+0000 7fbb18053640 1 -- 172.21.15.38:0/787057420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb100cbbc0 con 0x7fbb100ced60 2024-04-25T08:31:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.275+0000 7fbb18053640 1 -- 172.21.15.38:0/787057420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb101f4640 con 0x7fbb100ced60 2024-04-25T08:31:32.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.275+0000 7fbb027fc640 1 -- 172.21.15.38:0/787057420 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb0c044070 con 0x7fbb100ced60 2024-04-25T08:31:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.275+0000 7fbb027fc640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbaf406b4d0 0x7fbaf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.276+0000 7fbb027fc640 1 -- 172.21.15.38:0/787057420 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbb0c0aabb0 con 0x7fbb100ced60 2024-04-25T08:31:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.276+0000 7fbb155ee640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbaf406b4d0 0x7fbaf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.276+0000 7fbb155ee640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbaf406b4d0 0x7fbaf406d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb100cd6e0 tx=0x7fbb04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:32.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.280+0000 7fbb027fc640 1 -- 172.21.15.38:0/787057420 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb101f4640 con 0x7fbb100ced60 2024-04-25T08:31:32.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.459+0000 7fbb18053640 1 -- 172.21.15.38:0/787057420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb10068e80 con 0x7fbb100ced60 2024-04-25T08:31:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.460+0000 7fbb027fc640 1 -- 172.21.15.38:0/787057420 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12872 (secure 0 0 0) 0x7fbb0c0a2e20 con 0x7fbb100ced60 2024-04-25T08:31:32.457 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:32.457 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":16,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:25.295929+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:32.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.462+0000 7fbb18053640 1 -- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbaf406b4d0 msgr2=0x7fbaf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.462+0000 7fbb18053640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbaf406b4d0 0x7fbaf406d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb100cd6e0 tx=0x7fbb04008040 comp rx=0 tx=0).stop 2024-04-25T08:31:32.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.462+0000 7fbb18053640 1 -- 172.21.15.38:0/787057420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb100ced60 msgr2=0x7fbb101f3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.462+0000 7fbb18053640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb100ced60 0x7fbb101f3740 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c000c00 tx=0x7fbb0c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.462+0000 7fbb18053640 1 -- 172.21.15.38:0/787057420 shutdown_connections 2024-04-25T08:31:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.462+0000 7fbb18053640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbaf406b4d0 0x7fbaf406d990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.462+0000 7fbb18053640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb1015dc10 0x7fbb101524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.462+0000 7fbb18053640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb100d3900 0x7fbb10159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.462+0000 7fbb18053640 1 --2- 172.21.15.38:0/787057420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb100ced60 0x7fbb101f3740 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.462+0000 7fbb18053640 1 -- 172.21.15.38:0/787057420 >> 172.21.15.38:0/787057420 conn(0x7fbb10066d10 msgr2=0x7fbb100b4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.462+0000 7fbb18053640 1 -- 172.21.15.38:0/787057420 shutdown_connections 2024-04-25T08:31:32.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.462+0000 7fbb18053640 1 -- 172.21.15.38:0/787057420 wait complete. 2024-04-25T08:31:32.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:31:32.470 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:31:32.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.577+0000 7fee9f415640 1 Processor -- start 2024-04-25T08:31:32.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.577+0000 7fee9f415640 1 -- start start 2024-04-25T08:31:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee9f415640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee980d0320 0x7fee980d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee9f415640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee980d2060 con 0x7fee980b6980 2024-04-25T08:31:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee9f415640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee980d2330 con 0x7fee980d0700 2024-04-25T08:31:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee9f415640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee980d2610 con 0x7fee980d0320 2024-04-25T08:31:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee9d9b2640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fee980b6980 0x7fee980b6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32980/0 (socket says 172.21.15.38:32980) 2024-04-25T08:31:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee9d9b2640 1 -- 172.21.15.38:0/1241700290 learned_addr learned my addr 172.21.15.38:0/1241700290 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee9d1b1640 1 --2- 172.21.15.38:0/1241700290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee980d0320 0x7fee980d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602140402 0 0) 0x7fee980d2060 con 0x7fee980b6980 2024-04-25T08:31:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee84003520 con 0x7fee980b6980 2024-04-25T08:31:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4205463094 0 0) 0x7fee84003520 con 0x7fee980b6980 2024-04-25T08:31:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee980d0320 msgr2=0x7fee980d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:32.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 --2- 172.21.15.38:0/1241700290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee980d0320 0x7fee980d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 >> v1:172.21.15.107:6789/0 conn(0x7fee980d0700 legacy=0x7fee980d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee980d28f0 con 0x7fee980b6980 2024-04-25T08:31:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fee900019f0 con 0x7fee980b6980 2024-04-25T08:31:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 >> v1:172.21.15.38:6789/0 conn(0x7fee980b6980 legacy=0x7fee980b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 --2- 172.21.15.38:0/1241700290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee840039e0 0x7fee84003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 --2- 172.21.15.38:0/1241700290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee980d0320 0x7fee84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 --2- 172.21.15.38:0/1241700290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee84004d40 0x7fee840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee980d2610 con 0x7fee84004d40 2024-04-25T08:31:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8400bb00 con 0x7fee840039e0 2024-04-25T08:31:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8400bd00 con 0x7fee980d0320 2024-04-25T08:31:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.578+0000 7fee9d9b2640 1 --2- 172.21.15.38:0/1241700290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee84004d40 0x7fee840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.579+0000 7fee9d1b1640 1 --2- 172.21.15.38:0/1241700290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee840039e0 0x7fee84003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.579+0000 7fee9c9b0640 1 --2- 172.21.15.38:0/1241700290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee980d0320 0x7fee84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.579+0000 7fee9d9b2640 1 -- 172.21.15.38:0/1241700290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee980d0320 msgr2=0x7fee84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.579+0000 7fee9d9b2640 1 --2- 172.21.15.38:0/1241700290 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee980d0320 0x7fee84004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.579+0000 7fee9d9b2640 1 -- 172.21.15.38:0/1241700290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee840039e0 msgr2=0x7fee84003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.579+0000 7fee9d9b2640 1 --2- 172.21.15.38:0/1241700290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee840039e0 0x7fee84003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.579+0000 7fee9d9b2640 1 -- 172.21.15.38:0/1241700290 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee84003520 con 0x7fee84004d40 2024-04-25T08:31:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.579+0000 7fee9d9b2640 1 --2- 172.21.15.38:0/1241700290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee84004d40 0x7fee840051c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fee90002230 tx=0x7fee84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aadd116283cf2a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.580+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9000a4a0 con 0x7fee84004d40 2024-04-25T08:31:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.580+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9002f070 con 0x7fee84004d40 2024-04-25T08:31:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.580+0000 7fee8ffff640 1 -- 172.21.15.38:0/1241700290 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9002bcb0 con 0x7fee84004d40 2024-04-25T08:31:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.580+0000 7fee9f415640 1 -- 172.21.15.38:0/1241700290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee84004d40 msgr2=0x7fee840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.580+0000 7fee9f415640 1 --2- 172.21.15.38:0/1241700290 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee84004d40 0x7fee840051c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fee90002230 tx=0x7fee84005830 comp rx=0 tx=0).stop 2024-04-25T08:31:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.580+0000 7fee9d9b2640 1 -- 172.21.15.38:0/1241700290 reap_dead start 2024-04-25T08:31:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.580+0000 7fee9f415640 1 -- 172.21.15.38:0/1241700290 shutdown_connections 2024-04-25T08:31:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.580+0000 7fee9f415640 1 -- 172.21.15.38:0/1241700290 >> 172.21.15.38:0/1241700290 conn(0x7fee980668f0 msgr2=0x7fee98066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.580+0000 7fee9f415640 1 -- 172.21.15.38:0/1241700290 shutdown_connections 2024-04-25T08:31:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.580+0000 7fee9f415640 1 -- 172.21.15.38:0/1241700290 wait complete. 2024-04-25T08:31:32.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.581+0000 7fee9f415640 1 Processor -- start 2024-04-25T08:31:32.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.581+0000 7fee9f415640 1 -- start start 2024-04-25T08:31:32.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.582+0000 7fee9f415640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee84004d40 0x7fee980ac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.582+0000 7fee9f415640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee980b5810 0x7fee980b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.582+0000 7fee9f415640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee981f35a0 0x7fee981f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.582+0000 7fee9f415640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee980d0b40 con 0x7fee981f35a0 2024-04-25T08:31:32.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.582+0000 7fee9f415640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee980d2eb0 con 0x7fee980b5810 2024-04-25T08:31:32.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.582+0000 7fee9f415640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee980aad50 con 0x7fee84004d40 2024-04-25T08:31:32.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.582+0000 7fee9d9b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee981f35a0 0x7fee981f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.582+0000 7fee9d9b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee981f35a0 0x7fee981f3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39342/0 (socket says 172.21.15.38:39342) 2024-04-25T08:31:32.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.582+0000 7fee9d9b2640 1 -- 172.21.15.38:0/1145367253 learned_addr learned my addr 172.21.15.38:0/1145367253 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:32.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.582+0000 7fee9d1b1640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee84004d40 0x7fee980ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.582+0000 7fee9c9b0640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee980b5810 0x7fee980b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.583+0000 7fee9d9b2640 1 -- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee84004d40 msgr2=0x7fee980ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.583+0000 7fee9d9b2640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee84004d40 0x7fee980ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.583+0000 7fee9d9b2640 1 -- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee980b5810 msgr2=0x7fee980b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.583+0000 7fee9d9b2640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee980b5810 0x7fee980b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.583+0000 7fee9d9b2640 1 -- 172.21.15.38:0/1145367253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee980aa420 con 0x7fee981f35a0 2024-04-25T08:31:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.583+0000 7fee9d9b2640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee981f35a0 0x7fee981f3a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fee9000a940 tx=0x7fee90001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.583+0000 7fee8dffb640 1 -- 172.21.15.38:0/1145367253 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9000a480 con 0x7fee981f35a0 2024-04-25T08:31:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.583+0000 7fee8dffb640 1 -- 172.21.15.38:0/1145367253 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee90004b60 con 0x7fee981f35a0 2024-04-25T08:31:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.583+0000 7fee9f415640 1 -- 172.21.15.38:0/1145367253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee980aa700 con 0x7fee981f35a0 2024-04-25T08:31:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.583+0000 7fee8dffb640 1 -- 172.21.15.38:0/1145367253 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90034470 con 0x7fee981f35a0 2024-04-25T08:31:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.583+0000 7fee9f415640 1 -- 172.21.15.38:0/1145367253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee980ab1e0 con 0x7fee981f35a0 2024-04-25T08:31:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.584+0000 7fee9f415640 1 -- 172.21.15.38:0/1145367253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee98068e80 con 0x7fee981f35a0 2024-04-25T08:31:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.584+0000 7fee8dffb640 1 -- 172.21.15.38:0/1145367253 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee90034610 con 0x7fee981f35a0 2024-04-25T08:31:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.585+0000 7fee8dffb640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fee7c06b4d0 0x7fee7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.585+0000 7fee8dffb640 1 -- 172.21.15.38:0/1145367253 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fee900a9ff0 con 0x7fee981f35a0 2024-04-25T08:31:32.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.585+0000 7fee9d1b1640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fee7c06b4d0 0x7fee7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.585+0000 7fee9d1b1640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fee7c06b4d0 0x7fee7c06d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fee980b7b30 tx=0x7fee94009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:32.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.589+0000 7fee8dffb640 1 -- 172.21.15.38:0/1145367253 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fee90078d50 con 0x7fee981f35a0 2024-04-25T08:31:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.769+0000 7fee9f415640 1 -- 172.21.15.38:0/1145367253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fee980cd4d0 con 0x7fee981f35a0 2024-04-25T08:31:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.770+0000 7fee8dffb640 1 -- 172.21.15.38:0/1145367253 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fee9007cdf0 con 0x7fee981f35a0 2024-04-25T08:31:32.766 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:31:32.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.771+0000 7fee9f415640 1 -- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fee7c06b4d0 msgr2=0x7fee7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.771+0000 7fee9f415640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fee7c06b4d0 0x7fee7c06d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fee980b7b30 tx=0x7fee94009190 comp rx=0 tx=0).stop 2024-04-25T08:31:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.771+0000 7fee9f415640 1 -- 172.21.15.38:0/1145367253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee981f35a0 msgr2=0x7fee981f3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.771+0000 7fee9f415640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee981f35a0 0x7fee981f3a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fee9000a940 tx=0x7fee90001dc0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.771+0000 7fee9f415640 1 -- 172.21.15.38:0/1145367253 shutdown_connections 2024-04-25T08:31:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.771+0000 7fee9f415640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee981f35a0 0x7fee981f3a30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.771+0000 7fee9f415640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee980b5810 0x7fee980b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.771+0000 7fee9f415640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fee7c06b4d0 0x7fee7c06d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.771+0000 7fee9f415640 1 --2- 172.21.15.38:0/1145367253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee84004d40 0x7fee980ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.772+0000 7fee9f415640 1 -- 172.21.15.38:0/1145367253 >> 172.21.15.38:0/1145367253 conn(0x7fee980668f0 msgr2=0x7fee9815efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.772+0000 7fee9f415640 1 -- 172.21.15.38:0/1145367253 shutdown_connections 2024-04-25T08:31:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.772+0000 7fee9f415640 1 -- 172.21.15.38:0/1145367253 wait complete. 2024-04-25T08:31:32.779 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:31:32.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.882+0000 7f577f7be640 1 Processor -- start 2024-04-25T08:31:32.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.883+0000 7f577f7be640 1 -- start start 2024-04-25T08:31:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.883+0000 7f577f7be640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f57780d3900 0x7f57780d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.883+0000 7f577f7be640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57780d3e70 con 0x7f577815dc10 2024-04-25T08:31:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.883+0000 7f577f7be640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57780d4150 con 0x7f57780d0d90 2024-04-25T08:31:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.883+0000 7f577f7be640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577815dff0 con 0x7f57780d3900 2024-04-25T08:31:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.883+0000 7f577d55a640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f577815dc10 0x7f57780ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32992/0 (socket says 172.21.15.38:32992) 2024-04-25T08:31:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.883+0000 7f577d55a640 1 -- 172.21.15.38:0/428391578 learned_addr learned my addr 172.21.15.38:0/428391578 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179038309 0 0) 0x7f57780d3e70 con 0x7f577815dc10 2024-04-25T08:31:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5760003520 con 0x7f577815dc10 2024-04-25T08:31:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f577cd59640 1 --2- 172.21.15.38:0/428391578 >> v2:172.21.15.107:3300/0 conn(0x7f57780d3900 0x7f57780d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2326141583 0 0) 0x7f5760003520 con 0x7f577815dc10 2024-04-25T08:31:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 >> v2:172.21.15.107:3300/0 conn(0x7f57780d3900 msgr2=0x7f57780d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 --2- 172.21.15.38:0/428391578 >> v2:172.21.15.107:3300/0 conn(0x7f57780d3900 0x7f57780d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 >> v1:172.21.15.107:6789/0 conn(0x7f57780d0d90 legacy=0x7f57780b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57780d4430 con 0x7f577815dc10 2024-04-25T08:31:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f5770001670 con 0x7f577815dc10 2024-04-25T08:31:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 >> v1:172.21.15.38:6789/0 conn(0x7f577815dc10 legacy=0x7f57780ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 --2- 172.21.15.38:0/428391578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57600039b0 0x7f5760003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 --2- 172.21.15.38:0/428391578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 0x7f5760004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 --2- 172.21.15.38:0/428391578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57600051c0 0x7f576000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577815dff0 con 0x7f5760004800 2024-04-25T08:31:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576000c160 con 0x7f57600039b0 2024-04-25T08:31:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576000c360 con 0x7f57600051c0 2024-04-25T08:31:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f577cd59640 1 --2- 172.21.15.38:0/428391578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 0x7f5760004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f577d55a640 1 --2- 172.21.15.38:0/428391578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57600039b0 0x7f5760003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f577dd5b640 1 --2- 172.21.15.38:0/428391578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57600051c0 0x7f576000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f577cd59640 1 -- 172.21.15.38:0/428391578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57600051c0 msgr2=0x7f576000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f577cd59640 1 --2- 172.21.15.38:0/428391578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57600051c0 0x7f576000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f577cd59640 1 -- 172.21.15.38:0/428391578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57600039b0 msgr2=0x7f5760003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f577cd59640 1 --2- 172.21.15.38:0/428391578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57600039b0 0x7f5760003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f577cd59640 1 -- 172.21.15.38:0/428391578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5760003520 con 0x7f5760004800 2024-04-25T08:31:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.884+0000 7f577dd5b640 1 -- 172.21.15.38:0/428391578 reap_dead start 2024-04-25T08:31:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.885+0000 7f577cd59640 1 --2- 172.21.15.38:0/428391578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 0x7f5760004c80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f576c009ac0 tx=0x7f5760005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d10e3096b11d97d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.885+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c036070 con 0x7f5760004800 2024-04-25T08:31:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.885+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c031400 con 0x7f5760004800 2024-04-25T08:31:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.885+0000 7f5777fff640 1 -- 172.21.15.38:0/428391578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c009690 con 0x7f5760004800 2024-04-25T08:31:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.886+0000 7f577f7be640 1 -- 172.21.15.38:0/428391578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 msgr2=0x7f5760004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.886+0000 7f577f7be640 1 --2- 172.21.15.38:0/428391578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 0x7f5760004c80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f576c009ac0 tx=0x7f5760005830 comp rx=0 tx=0).stop 2024-04-25T08:31:32.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.886+0000 7f577f7be640 1 -- 172.21.15.38:0/428391578 shutdown_connections 2024-04-25T08:31:32.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.886+0000 7f577f7be640 1 --2- 172.21.15.38:0/428391578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 0x7f5760004c80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.886+0000 7f577f7be640 1 -- 172.21.15.38:0/428391578 >> 172.21.15.38:0/428391578 conn(0x7f5778066d10 msgr2=0x7f5778067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:32.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.886+0000 7f577f7be640 1 -- 172.21.15.38:0/428391578 shutdown_connections 2024-04-25T08:31:32.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.886+0000 7f577f7be640 1 -- 172.21.15.38:0/428391578 wait complete. 2024-04-25T08:31:32.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.887+0000 7f577f7be640 1 Processor -- start 2024-04-25T08:31:32.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.887+0000 7f577f7be640 1 -- start start 2024-04-25T08:31:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577f7be640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 0x7f5778154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577f7be640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5778159df0 0x7f57780b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577f7be640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57781474f0 0x7f5778147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577f7be640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57780d11e0 con 0x7f5760004800 2024-04-25T08:31:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577f7be640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57780d4710 con 0x7f5778159df0 2024-04-25T08:31:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577f7be640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57780d0850 con 0x7f57781474f0 2024-04-25T08:31:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577d55a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 0x7f5778154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577d55a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 0x7f5778154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39358/0 (socket says 172.21.15.38:39358) 2024-04-25T08:31:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577d55a640 1 -- 172.21.15.38:0/1068286192 learned_addr learned my addr 172.21.15.38:0/1068286192 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577cd59640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5778159df0 0x7f57780b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577dd5b640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57781474f0 0x7f5778147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577d55a640 1 -- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57781474f0 msgr2=0x7f5778147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577d55a640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57781474f0 0x7f5778147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577d55a640 1 -- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5778159df0 msgr2=0x7f57780b4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577d55a640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5778159df0 0x7f57780b4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577d55a640 1 -- 172.21.15.38:0/1068286192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5778155450 con 0x7f5760004800 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577d55a640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 0x7f5778154e70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5770007370 tx=0x7f5770007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f5775ffb640 1 -- 172.21.15.38:0/1068286192 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577001a070 con 0x7f5760004800 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f5775ffb640 1 -- 172.21.15.38:0/1068286192 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577000fa80 con 0x7f5760004800 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f577f7be640 1 -- 172.21.15.38:0/1068286192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5778155730 con 0x7f5760004800 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.888+0000 7f5775ffb640 1 -- 172.21.15.38:0/1068286192 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577000fe70 con 0x7f5760004800 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.889+0000 7f577f7be640 1 -- 172.21.15.38:0/1068286192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5778155cf0 con 0x7f5760004800 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.890+0000 7f577f7be640 1 -- 172.21.15.38:0/1068286192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5778068e80 con 0x7f5760004800 2024-04-25T08:31:32.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.890+0000 7f5775ffb640 1 -- 172.21.15.38:0/1068286192 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f577001d070 con 0x7f5760004800 2024-04-25T08:31:32.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.890+0000 7f5775ffb640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f576406b4d0 0x7f576406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:32.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.891+0000 7f5775ffb640 1 -- 172.21.15.38:0/1068286192 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5770089870 con 0x7f5760004800 2024-04-25T08:31:32.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.891+0000 7f577cd59640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f576406b4d0 0x7f576406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:32.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.891+0000 7f577cd59640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f576406b4d0 0x7f576406d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f576c033000 tx=0x7f576c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:32.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:32.895+0000 7f5775ffb640 1 -- 172.21.15.38:0/1068286192 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5770058480 con 0x7f5760004800 2024-04-25T08:31:33.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.079+0000 7f577f7be640 1 -- 172.21.15.38:0/1068286192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f577815f4a0 con 0x7f5760004800 2024-04-25T08:31:33.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.079+0000 7f5775ffb640 1 -- 172.21.15.38:0/1068286192 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f577005c520 con 0x7f5760004800 2024-04-25T08:31:33.076 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:33.076 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T08:31:33.076 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:31:33.076 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-04-25T08:31:33.076 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-04-25T08:31:33.076 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:31:33.076 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T08:31:33.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.081+0000 7f577f7be640 1 -- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f576406b4d0 msgr2=0x7f576406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.081+0000 7f577f7be640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f576406b4d0 0x7f576406d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f576c033000 tx=0x7f576c008030 comp rx=0 tx=0).stop 2024-04-25T08:31:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.081+0000 7f577f7be640 1 -- 172.21.15.38:0/1068286192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 msgr2=0x7f5778154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.081+0000 7f577f7be640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 0x7f5778154e70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5770007370 tx=0x7f5770007830 comp rx=0 tx=0).stop 2024-04-25T08:31:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.081+0000 7f577f7be640 1 -- 172.21.15.38:0/1068286192 shutdown_connections 2024-04-25T08:31:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.081+0000 7f577f7be640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f576406b4d0 0x7f576406d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.081+0000 7f577f7be640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f57781474f0 0x7f5778147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.081+0000 7f577f7be640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5778159df0 0x7f57780b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.081+0000 7f577f7be640 1 --2- 172.21.15.38:0/1068286192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5760004800 0x7f5778154e70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.081+0000 7f577f7be640 1 -- 172.21.15.38:0/1068286192 >> 172.21.15.38:0/1068286192 conn(0x7f5778066d10 msgr2=0x7f577815ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.081+0000 7f577f7be640 1 -- 172.21.15.38:0/1068286192 shutdown_connections 2024-04-25T08:31:33.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.081+0000 7f577f7be640 1 -- 172.21.15.38:0/1068286192 wait complete. 2024-04-25T08:31:33.089 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:31:33.089 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:31:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.197+0000 7fa671553640 1 Processor -- start 2024-04-25T08:31:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.197+0000 7fa671553640 1 -- start start 2024-04-25T08:31:33.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.198+0000 7fa671553640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa66c15b9a0 0x7fa66c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:33.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.198+0000 7fa671553640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa66c0b1c60 0x7fa66c0afe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:33.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.198+0000 7fa671553640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa66c15f4b0 con 0x7fa66c0cdbc0 2024-04-25T08:31:33.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.198+0000 7fa671553640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66c15f790 con 0x7fa66c15b9a0 2024-04-25T08:31:33.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.198+0000 7fa671553640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66c15fa70 con 0x7fa66c0b1c60 2024-04-25T08:31:33.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.198+0000 7fa66affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa66c15b9a0 0x7fa66c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.198+0000 7fa66affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa66c15b9a0 0x7fa66c06abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39368/0 (socket says 172.21.15.38:39368) 2024-04-25T08:31:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.198+0000 7fa66affd640 1 -- 172.21.15.38:0/3748166293 learned_addr learned my addr 172.21.15.38:0/3748166293 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.198+0000 7fa66b7fe640 1 --2- 172.21.15.38:0/3748166293 >> v2:172.21.15.107:3300/0 conn(0x7fa66c0b1c60 0x7fa66c0afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.198+0000 7fa66affd640 1 -- 172.21.15.38:0/3748166293 >> v2:172.21.15.107:3300/0 conn(0x7fa66c0b1c60 msgr2=0x7fa66c0afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.199+0000 7fa66affd640 1 --2- 172.21.15.38:0/3748166293 >> v2:172.21.15.107:3300/0 conn(0x7fa66c0b1c60 0x7fa66c0afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.199+0000 7fa66affd640 1 -- 172.21.15.38:0/3748166293 >> v1:172.21.15.107:6789/0 conn(0x7fa66c0cdbc0 legacy=0x7fa66c0b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.199+0000 7fa66affd640 1 -- 172.21.15.38:0/3748166293 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c15fd50 con 0x7fa66c15b9a0 2024-04-25T08:31:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.199+0000 7fa66b7fe640 1 --2- 172.21.15.38:0/3748166293 >> v2:172.21.15.107:3300/0 conn(0x7fa66c0b1c60 0x7fa66c0afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.199+0000 7fa66affd640 1 --2- 172.21.15.38:0/3748166293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa66c15b9a0 0x7fa66c06abe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0cd180 tx=0x7fa66402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e588305a0a1fc598 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.199+0000 7fa669ffb640 1 -- 172.21.15.38:0/3748166293 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa664037070 con 0x7fa66c15b9a0 2024-04-25T08:31:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.199+0000 7fa669ffb640 1 -- 172.21.15.38:0/3748166293 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa664032440 con 0x7fa66c15b9a0 2024-04-25T08:31:33.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.199+0000 7fa669ffb640 1 -- 172.21.15.38:0/3748166293 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa664032810 con 0x7fa66c15b9a0 2024-04-25T08:31:33.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.200+0000 7fa671553640 1 -- 172.21.15.38:0/3748166293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa66c15b9a0 msgr2=0x7fa66c06abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:33.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.200+0000 7fa671553640 1 --2- 172.21.15.38:0/3748166293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa66c15b9a0 0x7fa66c06abe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0cd180 tx=0x7fa66402a230 comp rx=0 tx=0).stop 2024-04-25T08:31:33.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.200+0000 7fa671553640 1 -- 172.21.15.38:0/3748166293 shutdown_connections 2024-04-25T08:31:33.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.200+0000 7fa671553640 1 --2- 172.21.15.38:0/3748166293 >> v2:172.21.15.107:3300/0 conn(0x7fa66c0b1c60 0x7fa66c0afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:33.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.200+0000 7fa671553640 1 --2- 172.21.15.38:0/3748166293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa66c15b9a0 0x7fa66c06abe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:33.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.200+0000 7fa671553640 1 -- 172.21.15.38:0/3748166293 >> 172.21.15.38:0/3748166293 conn(0x7fa66c072610 msgr2=0x7fa66c099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:33.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.200+0000 7fa671553640 1 -- 172.21.15.38:0/3748166293 shutdown_connections 2024-04-25T08:31:33.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.200+0000 7fa671553640 1 -- 172.21.15.38:0/3748166293 wait complete. 2024-04-25T08:31:33.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.201+0000 7fa671553640 1 Processor -- start 2024-04-25T08:31:33.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.201+0000 7fa671553640 1 -- start start 2024-04-25T08:31:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.202+0000 7fa671553640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa66c0b1c60 0x7fa66c0c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.202+0000 7fa671553640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c0cdbc0 0x7fa66c0c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.202+0000 7fa671553640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa66c15b9a0 0x7fa66c1f73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.202+0000 7fa671553640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c06b120 con 0x7fa66c0b1c60 2024-04-25T08:31:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.202+0000 7fa671553640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c06b550 con 0x7fa66c0cdbc0 2024-04-25T08:31:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.202+0000 7fa671553640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66c06b2a0 con 0x7fa66c15b9a0 2024-04-25T08:31:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.202+0000 7fa66affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa66c0b1c60 0x7fa66c0c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.202+0000 7fa66affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa66c0b1c60 0x7fa66c0c6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39384/0 (socket says 172.21.15.38:39384) 2024-04-25T08:31:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.202+0000 7fa66affd640 1 -- 172.21.15.38:0/1899135338 learned_addr learned my addr 172.21.15.38:0/1899135338 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.202+0000 7fa66a7fc640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c0cdbc0 0x7fa66c0c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.203+0000 7fa66affd640 1 -- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa66c15b9a0 msgr2=0x7fa66c1f73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:33.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.203+0000 7fa66affd640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa66c15b9a0 0x7fa66c1f73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.203+0000 7fa66affd640 1 -- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c0cdbc0 msgr2=0x7fa66c0c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.203+0000 7fa66affd640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c0cdbc0 0x7fa66c0c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.203+0000 7fa66affd640 1 -- 172.21.15.38:0/1899135338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66c0cee50 con 0x7fa66c0b1c60 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.203+0000 7fa66affd640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa66c0b1c60 0x7fa66c0c6570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa664000c00 tx=0x7fa6640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.203+0000 7fa6637fe640 1 -- 172.21.15.38:0/1899135338 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa664041020 con 0x7fa66c0b1c60 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.203+0000 7fa6637fe640 1 -- 172.21.15.38:0/1899135338 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa664030970 con 0x7fa66c0b1c60 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.203+0000 7fa671553640 1 -- 172.21.15.38:0/1899135338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66c0cf130 con 0x7fa66c0b1c60 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.203+0000 7fa6637fe640 1 -- 172.21.15.38:0/1899135338 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa664037040 con 0x7fa66c0b1c60 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.203+0000 7fa671553640 1 -- 172.21.15.38:0/1899135338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa66c0cf6f0 con 0x7fa66c0b1c60 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.204+0000 7fa671553640 1 -- 172.21.15.38:0/1899135338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66c0ca830 con 0x7fa66c0b1c60 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.205+0000 7fa6637fe640 1 -- 172.21.15.38:0/1899135338 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa664044020 con 0x7fa66c0b1c60 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.205+0000 7fa6637fe640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa64c06b4d0 0x7fa64c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.205+0000 7fa6637fe640 1 -- 172.21.15.38:0/1899135338 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa6640aa840 con 0x7fa66c0b1c60 2024-04-25T08:31:33.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.205+0000 7fa66a7fc640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa64c06b4d0 0x7fa64c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:33.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.206+0000 7fa66a7fc640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa64c06b4d0 0x7fa64c06d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0b62a0 tx=0x7fa654007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:33.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.209+0000 7fa6637fe640 1 -- 172.21.15.38:0/1899135338 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa66404b030 con 0x7fa66c0b1c60 2024-04-25T08:31:33.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:33.394+0000 7fa671553640 1 -- 172.21.15.38:0/1899135338 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa66c0ad340 con 0x7fa66c0b1c60 2024-04-25T08:31:34.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.288+0000 7fa6637fe640 1 -- 172.21.15.38:0/1899135338 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa664079a10 con 0x7fa66c0b1c60 2024-04-25T08:31:34.285 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-04-25T08:31:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.292+0000 7fa671553640 1 -- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa64c06b4d0 msgr2=0x7fa64c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.292+0000 7fa671553640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa64c06b4d0 0x7fa64c06d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa66c0b62a0 tx=0x7fa654007e30 comp rx=0 tx=0).stop 2024-04-25T08:31:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.292+0000 7fa671553640 1 -- 172.21.15.38:0/1899135338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa66c0b1c60 msgr2=0x7fa66c0c6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.292+0000 7fa671553640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa66c0b1c60 0x7fa66c0c6570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa664000c00 tx=0x7fa6640046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.292+0000 7fa671553640 1 -- 172.21.15.38:0/1899135338 shutdown_connections 2024-04-25T08:31:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.292+0000 7fa671553640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa64c06b4d0 0x7fa64c06d990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.292+0000 7fa671553640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa66c15b9a0 0x7fa66c1f73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.292+0000 7fa671553640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa66c0cdbc0 0x7fa66c0c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.292+0000 7fa671553640 1 --2- 172.21.15.38:0/1899135338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa66c0b1c60 0x7fa66c0c6570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.292+0000 7fa671553640 1 -- 172.21.15.38:0/1899135338 >> 172.21.15.38:0/1899135338 conn(0x7fa66c072610 msgr2=0x7fa66c099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.292+0000 7fa671553640 1 -- 172.21.15.38:0/1899135338 shutdown_connections 2024-04-25T08:31:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.292+0000 7fa671553640 1 -- 172.21.15.38:0/1899135338 wait complete. 2024-04-25T08:31:34.305 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T08:31:34.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.422+0000 7faa9f64a640 1 Processor -- start 2024-04-25T08:31:34.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.422+0000 7faa9f64a640 1 -- start start 2024-04-25T08:31:34.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9f64a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980d0320 0x7faa980d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:34.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9f64a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7faa980d0700 0x7faa980d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:34.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9f64a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa980d2330 con 0x7faa980d1e90 2024-04-25T08:31:34.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9f64a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa980d2610 con 0x7faa980d0700 2024-04-25T08:31:34.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9f64a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa980d28f0 con 0x7faa980d0320 2024-04-25T08:31:34.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9cbe5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7faa980d0700 0x7faa980d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:34.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9d3e6640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980d0320 0x7faa980d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:34.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9d3e6640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980d0320 0x7faa980d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:43348/0 (socket says 172.21.15.38:43348) 2024-04-25T08:31:34.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9cbe5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7faa980d0700 0x7faa980d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39390/0 (socket says 172.21.15.38:39390) 2024-04-25T08:31:34.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9d3e6640 1 -- 172.21.15.38:0/1129926162 learned_addr learned my addr 172.21.15.38:0/1129926162 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:34.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9cbe5640 1 -- 172.21.15.38:0/1129926162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980d0320 msgr2=0x7faa980d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:34.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9cbe5640 1 --2- 172.21.15.38:0/1129926162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980d0320 0x7faa980d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:34.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9cbe5640 1 -- 172.21.15.38:0/1129926162 >> v1:172.21.15.107:6789/0 conn(0x7faa980d1e90 legacy=0x7faa980b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:34.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.423+0000 7faa9cbe5640 1 -- 172.21.15.38:0/1129926162 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa980d2bd0 con 0x7faa980d0700 2024-04-25T08:31:34.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.424+0000 7faa9cbe5640 1 --2- 172.21.15.38:0/1129926162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980d0700 0x7faa980d1950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa880097a0 tx=0x7faa8802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6023be0414cfee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:34.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.424+0000 7faa8ffff640 1 -- 172.21.15.38:0/1129926162 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88037070 con 0x7faa980d0700 2024-04-25T08:31:34.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.424+0000 7faa8ffff640 1 -- 172.21.15.38:0/1129926162 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa88032440 con 0x7faa980d0700 2024-04-25T08:31:34.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.424+0000 7faa8ffff640 1 -- 172.21.15.38:0/1129926162 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88032810 con 0x7faa980d0700 2024-04-25T08:31:34.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.425+0000 7faa9f64a640 1 -- 172.21.15.38:0/1129926162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980d0700 msgr2=0x7faa980d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:34.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.425+0000 7faa9f64a640 1 --2- 172.21.15.38:0/1129926162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980d0700 0x7faa980d1950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faa880097a0 tx=0x7faa8802a210 comp rx=0 tx=0).stop 2024-04-25T08:31:34.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.425+0000 7faa9f64a640 1 -- 172.21.15.38:0/1129926162 shutdown_connections 2024-04-25T08:31:34.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.425+0000 7faa9f64a640 1 --2- 172.21.15.38:0/1129926162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980d0700 0x7faa980d1950 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:34.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.425+0000 7faa9f64a640 1 --2- 172.21.15.38:0/1129926162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980d0320 0x7faa980d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:34.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.425+0000 7faa9f64a640 1 -- 172.21.15.38:0/1129926162 >> 172.21.15.38:0/1129926162 conn(0x7faa980668f0 msgr2=0x7faa98066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:34.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.425+0000 7faa9f64a640 1 -- 172.21.15.38:0/1129926162 shutdown_connections 2024-04-25T08:31:34.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.425+0000 7faa9f64a640 1 -- 172.21.15.38:0/1129926162 wait complete. 2024-04-25T08:31:34.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.426+0000 7faa9f64a640 1 Processor -- start 2024-04-25T08:31:34.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.426+0000 7faa9f64a640 1 -- start start 2024-04-25T08:31:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9f64a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980d0320 0x7faa9814dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9f64a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa980d0700 0x7faa980ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9f64a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980d1e90 0x7faa980ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9f64a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa980d0b40 con 0x7faa980d0320 2024-04-25T08:31:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9f64a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa98161210 con 0x7faa980d0700 2024-04-25T08:31:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9f64a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa981e62d0 con 0x7faa980d1e90 2024-04-25T08:31:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9d3e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980d0320 0x7faa9814dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9d3e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980d0320 0x7faa9814dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39394/0 (socket says 172.21.15.38:39394) 2024-04-25T08:31:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9d3e6640 1 -- 172.21.15.38:0/2445026299 learned_addr learned my addr 172.21.15.38:0/2445026299 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9cbe5640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa980d0700 0x7faa980ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9dbe7640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980d1e90 0x7faa980ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9d3e6640 1 -- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980d1e90 msgr2=0x7faa980ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:34.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9d3e6640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980d1e90 0x7faa980ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:34.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9d3e6640 1 -- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa980d0700 msgr2=0x7faa980ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:34.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9d3e6640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa980d0700 0x7faa980ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:34.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9d3e6640 1 -- 172.21.15.38:0/2445026299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa981e6540 con 0x7faa980d0320 2024-04-25T08:31:34.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.427+0000 7faa9dbe7640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980d1e90 0x7faa980ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:34.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.428+0000 7faa9d3e6640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980d0320 0x7faa9814dc20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa94009fd0 tx=0x7faa9400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:34.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.428+0000 7faa8dffb640 1 -- 172.21.15.38:0/2445026299 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa94017070 con 0x7faa980d0320 2024-04-25T08:31:34.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.428+0000 7faa8dffb640 1 -- 172.21.15.38:0/2445026299 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9400c940 con 0x7faa980d0320 2024-04-25T08:31:34.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.428+0000 7faa8dffb640 1 -- 172.21.15.38:0/2445026299 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9400cd10 con 0x7faa980d0320 2024-04-25T08:31:34.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.428+0000 7faa9f64a640 1 -- 172.21.15.38:0/2445026299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa981e6820 con 0x7faa980d0320 2024-04-25T08:31:34.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.428+0000 7faa9f64a640 1 -- 172.21.15.38:0/2445026299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa981e6de0 con 0x7faa980d0320 2024-04-25T08:31:34.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.429+0000 7faa9f64a640 1 -- 172.21.15.38:0/2445026299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa98068e80 con 0x7faa980d0320 2024-04-25T08:31:34.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.429+0000 7faa8dffb640 1 -- 172.21.15.38:0/2445026299 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa9401a070 con 0x7faa980d0320 2024-04-25T08:31:34.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.430+0000 7faa8dffb640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faa7c06b4d0 0x7faa7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:34.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.430+0000 7faa8dffb640 1 -- 172.21.15.38:0/2445026299 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7faa9408a280 con 0x7faa980d0320 2024-04-25T08:31:34.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.430+0000 7faa9cbe5640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faa7c06b4d0 0x7faa7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:34.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.430+0000 7faa9cbe5640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faa7c06b4d0 0x7faa7c06d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faa88034000 tx=0x7faa8802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:34.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.434+0000 7faa8dffb640 1 -- 172.21.15.38:0/2445026299 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa94058c90 con 0x7faa980d0320 2024-04-25T08:31:34.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:34.618+0000 7faa9f64a640 1 -- 172.21.15.38:0/2445026299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7faa980b3050 con 0x7faa980d0320 2024-04-25T08:31:35.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.294+0000 7faa8dffb640 1 -- 172.21.15.38:0/2445026299 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7faa9405cd30 con 0x7faa980d0320 2024-04-25T08:31:35.290 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-04-25T08:31:35.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.297+0000 7faa9f64a640 1 -- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faa7c06b4d0 msgr2=0x7faa7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:35.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.297+0000 7faa9f64a640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faa7c06b4d0 0x7faa7c06d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faa88034000 tx=0x7faa8802e000 comp rx=0 tx=0).stop 2024-04-25T08:31:35.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.297+0000 7faa9f64a640 1 -- 172.21.15.38:0/2445026299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980d0320 msgr2=0x7faa9814dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:35.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.297+0000 7faa9f64a640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980d0320 0x7faa9814dc20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa94009fd0 tx=0x7faa9400ed40 comp rx=0 tx=0).stop 2024-04-25T08:31:35.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.297+0000 7faa9f64a640 1 -- 172.21.15.38:0/2445026299 shutdown_connections 2024-04-25T08:31:35.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.297+0000 7faa9f64a640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faa7c06b4d0 0x7faa7c06d990 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:35.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.297+0000 7faa9f64a640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980d1e90 0x7faa980ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:35.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.297+0000 7faa9f64a640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa980d0700 0x7faa980ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:35.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.297+0000 7faa9f64a640 1 --2- 172.21.15.38:0/2445026299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980d0320 0x7faa9814dc20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:35.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.297+0000 7faa9f64a640 1 -- 172.21.15.38:0/2445026299 >> 172.21.15.38:0/2445026299 conn(0x7faa980668f0 msgr2=0x7faa9815e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:35.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.298+0000 7faa9f64a640 1 -- 172.21.15.38:0/2445026299 shutdown_connections 2024-04-25T08:31:35.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.298+0000 7faa9f64a640 1 -- 172.21.15.38:0/2445026299 wait complete. 2024-04-25T08:31:35.317 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:31:35.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.428+0000 7fbd5b7f9640 1 Processor -- start 2024-04-25T08:31:35.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.429+0000 7fbd5b7f9640 1 -- start start 2024-04-25T08:31:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.429+0000 7fbd5b7f9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd541645f0 0x7fbd541669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.429+0000 7fbd5b7f9640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5415c1b0 con 0x7fbd5406abe0 2024-04-25T08:31:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.429+0000 7fbd5b7f9640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd540ccf30 con 0x7fbd540cc2c0 2024-04-25T08:31:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.429+0000 7fbd5b7f9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540cd210 con 0x7fbd541645f0 2024-04-25T08:31:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.429+0000 7fbd59595640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbd5406abe0 0x7fbd5406afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32996/0 (socket says 172.21.15.38:32996) 2024-04-25T08:31:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.429+0000 7fbd59595640 1 -- 172.21.15.38:0/2716947806 learned_addr learned my addr 172.21.15.38:0/2716947806 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587250205 0 0) 0x7fbd5415c1b0 con 0x7fbd5406abe0 2024-04-25T08:31:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd3c003520 con 0x7fbd5406abe0 2024-04-25T08:31:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd59d96640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd541645f0 0x7fbd541669e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 414809385 0 0) 0x7fbd3c003520 con 0x7fbd5406abe0 2024-04-25T08:31:35.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd541645f0 msgr2=0x7fbd541669e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd541645f0 0x7fbd541669e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 >> v1:172.21.15.107:6789/0 conn(0x7fbd540cc2c0 legacy=0x7fbd540cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd540cd4f0 con 0x7fbd5406abe0 2024-04-25T08:31:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fbd4c001670 con 0x7fbd5406abe0 2024-04-25T08:31:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 >> v1:172.21.15.38:6789/0 conn(0x7fbd5406abe0 legacy=0x7fbd5406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd3c0039e0 0x7fbd3c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd3c004800 0x7fbd3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd541645f0 0x7fbd3c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540cd210 con 0x7fbd3c004800 2024-04-25T08:31:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c00bb20 con 0x7fbd3c0039e0 2024-04-25T08:31:35.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3c00bd20 con 0x7fbd541645f0 2024-04-25T08:31:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd58d94640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd3c004800 0x7fbd3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd59595640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd3c0039e0 0x7fbd3c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd59d96640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd541645f0 0x7fbd3c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd58d94640 1 -- 172.21.15.38:0/2716947806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd541645f0 msgr2=0x7fbd3c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd58d94640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd541645f0 0x7fbd3c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd58d94640 1 -- 172.21.15.38:0/2716947806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd3c0039e0 msgr2=0x7fbd3c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd58d94640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd3c0039e0 0x7fbd3c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd58d94640 1 -- 172.21.15.38:0/2716947806 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c003520 con 0x7fbd3c004800 2024-04-25T08:31:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.430+0000 7fbd59d96640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd541645f0 0x7fbd3c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.431+0000 7fbd58d94640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd3c004800 0x7fbd3c004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbd48001750 tx=0x7fbd3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f965533270e0b60d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.431+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48001370 con 0x7fbd3c004800 2024-04-25T08:31:35.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.431+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd480052d0 con 0x7fbd3c004800 2024-04-25T08:31:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.431+0000 7fbd53fff640 1 -- 172.21.15.38:0/2716947806 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4802abe0 con 0x7fbd3c004800 2024-04-25T08:31:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.432+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/2716947806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd3c004800 msgr2=0x7fbd3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.432+0000 7fbd5b7f9640 1 --2- 172.21.15.38:0/2716947806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd3c004800 0x7fbd3c004c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbd48001750 tx=0x7fbd3c005830 comp rx=0 tx=0).stop 2024-04-25T08:31:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.432+0000 7fbd59d96640 1 -- 172.21.15.38:0/2716947806 reap_dead start 2024-04-25T08:31:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.432+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/2716947806 shutdown_connections 2024-04-25T08:31:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.432+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/2716947806 >> 172.21.15.38:0/2716947806 conn(0x7fbd540d06b0 msgr2=0x7fbd540d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.432+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/2716947806 shutdown_connections 2024-04-25T08:31:35.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.432+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/2716947806 wait complete. 2024-04-25T08:31:35.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.433+0000 7fbd5b7f9640 1 Processor -- start 2024-04-25T08:31:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.433+0000 7fbd5b7f9640 1 -- start start 2024-04-25T08:31:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd5b7f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd3c004800 0x7fbd540cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd5b7f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd540ce280 0x7fbd54151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd5b7f9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd54156f60 0x7fbd541573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd5b7f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54159910 con 0x7fbd540ce280 2024-04-25T08:31:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd5b7f9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd54159790 con 0x7fbd3c004800 2024-04-25T08:31:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd5b7f9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5415f070 con 0x7fbd54156f60 2024-04-25T08:31:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd58d94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd540ce280 0x7fbd54151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd58d94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd540ce280 0x7fbd54151710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39416/0 (socket says 172.21.15.38:39416) 2024-04-25T08:31:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd58d94640 1 -- 172.21.15.38:0/1726932748 learned_addr learned my addr 172.21.15.38:0/1726932748 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd59d96640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd54156f60 0x7fbd541573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd58d94640 1 -- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd54156f60 msgr2=0x7fbd541573e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd59595640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd3c004800 0x7fbd540cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd58d94640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd54156f60 0x7fbd541573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd58d94640 1 -- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd3c004800 msgr2=0x7fbd540cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd58d94640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd3c004800 0x7fbd540cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd58d94640 1 -- 172.21.15.38:0/1726932748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd54152160 con 0x7fbd540ce280 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.434+0000 7fbd58d94640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd540ce280 0x7fbd54151710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd4802a590 tx=0x7fbd48001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.435+0000 7fbd51ffb640 1 -- 172.21.15.38:0/1726932748 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48037040 con 0x7fbd540ce280 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.435+0000 7fbd51ffb640 1 -- 172.21.15.38:0/1726932748 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd48041070 con 0x7fbd540ce280 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.435+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/1726932748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd540ce820 con 0x7fbd540ce280 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.435+0000 7fbd51ffb640 1 -- 172.21.15.38:0/1726932748 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48007a30 con 0x7fbd540ce280 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.435+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/1726932748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd540cede0 con 0x7fbd540ce280 2024-04-25T08:31:35.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.436+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/1726932748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd54068e80 con 0x7fbd540ce280 2024-04-25T08:31:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.436+0000 7fbd51ffb640 1 -- 172.21.15.38:0/1726932748 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd480052d0 con 0x7fbd540ce280 2024-04-25T08:31:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.437+0000 7fbd51ffb640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd4006b4d0 0x7fbd4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.437+0000 7fbd51ffb640 1 -- 172.21.15.38:0/1726932748 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbd480a9f50 con 0x7fbd540ce280 2024-04-25T08:31:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.437+0000 7fbd59595640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd4006b4d0 0x7fbd4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.437+0000 7fbd59595640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd4006b4d0 0x7fbd4006d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c001050 tx=0x7fbd4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.441+0000 7fbd51ffb640 1 -- 172.21.15.38:0/1726932748 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd54152160 con 0x7fbd540ce280 2024-04-25T08:31:35.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:35.663+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/1726932748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbd541643d0 con 0x7fbd540ce280 2024-04-25T08:31:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.296+0000 7fbd51ffb640 1 -- 172.21.15.38:0/1726932748 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbd48078d00 con 0x7fbd540ce280 2024-04-25T08:31:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T08:31:36.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.299+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd4006b4d0 msgr2=0x7fbd4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:36.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.299+0000 7fbd5b7f9640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd4006b4d0 0x7fbd4006d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c001050 tx=0x7fbd4c009040 comp rx=0 tx=0).stop 2024-04-25T08:31:36.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.299+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/1726932748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd540ce280 msgr2=0x7fbd54151710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:36.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.299+0000 7fbd5b7f9640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd540ce280 0x7fbd54151710 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd4802a590 tx=0x7fbd48001120 comp rx=0 tx=0).stop 2024-04-25T08:31:36.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.299+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/1726932748 shutdown_connections 2024-04-25T08:31:36.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.299+0000 7fbd5b7f9640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd4006b4d0 0x7fbd4006d990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:36.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.299+0000 7fbd5b7f9640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd54156f60 0x7fbd541573e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:36.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.299+0000 7fbd5b7f9640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd540ce280 0x7fbd54151710 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:36.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.299+0000 7fbd5b7f9640 1 --2- 172.21.15.38:0/1726932748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd3c004800 0x7fbd540cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:36.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.299+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/1726932748 >> 172.21.15.38:0/1726932748 conn(0x7fbd540d06b0 msgr2=0x7fbd5415e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:36.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.299+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/1726932748 shutdown_connections 2024-04-25T08:31:36.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.299+0000 7fbd5b7f9640 1 -- 172.21.15.38:0/1726932748 wait complete. 2024-04-25T08:31:36.325 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T08:31:36.326 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T08:31:36.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.440+0000 7f0c419d5640 1 Processor -- start 2024-04-25T08:31:36.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.440+0000 7f0c419d5640 1 -- start start 2024-04-25T08:31:36.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c419d5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0c3c0d0860 0x7f0c3c0d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:36.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c419d5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d19e0 0x7f0c3c0d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:36.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c419d5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3c0d1180 con 0x7f0c3c0d3860 2024-04-25T08:31:36.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c419d5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c3c15e2b0 con 0x7f0c3c0d0860 2024-04-25T08:31:36.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c419d5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0d24a0 con 0x7f0c3c0d19e0 2024-04-25T08:31:36.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c3affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0c3c0d0860 0x7f0c3c0d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:36.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c3affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0c3c0d0860 0x7f0c3c0d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39432/0 (socket says 172.21.15.38:39432) 2024-04-25T08:31:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c3affd640 1 -- 172.21.15.38:0/3363547039 learned_addr learned my addr 172.21.15.38:0/3363547039 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c3a7fc640 1 --2- 172.21.15.38:0/3363547039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d19e0 0x7f0c3c0d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c3affd640 1 -- 172.21.15.38:0/3363547039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d19e0 msgr2=0x7f0c3c0d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c3affd640 1 --2- 172.21.15.38:0/3363547039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d19e0 0x7f0c3c0d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c3affd640 1 -- 172.21.15.38:0/3363547039 >> v1:172.21.15.107:6789/0 conn(0x7f0c3c0d3860 legacy=0x7f0c3c0d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:36.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.441+0000 7f0c3affd640 1 -- 172.21.15.38:0/3363547039 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c0d2780 con 0x7f0c3c0d0860 2024-04-25T08:31:36.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.442+0000 7f0c3affd640 1 --2- 172.21.15.38:0/3363547039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c3c0d0860 0x7f0c3c0d0c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009870 tx=0x7f0c2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173c004deb805a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:36.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.442+0000 7f0c39ffb640 1 -- 172.21.15.38:0/3363547039 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c037070 con 0x7f0c3c0d0860 2024-04-25T08:31:36.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.442+0000 7f0c39ffb640 1 -- 172.21.15.38:0/3363547039 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c032440 con 0x7f0c3c0d0860 2024-04-25T08:31:36.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.442+0000 7f0c39ffb640 1 -- 172.21.15.38:0/3363547039 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c032810 con 0x7f0c3c0d0860 2024-04-25T08:31:36.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.443+0000 7f0c419d5640 1 -- 172.21.15.38:0/3363547039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c3c0d0860 msgr2=0x7f0c3c0d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:36.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.443+0000 7f0c419d5640 1 --2- 172.21.15.38:0/3363547039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c3c0d0860 0x7f0c3c0d0c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c009870 tx=0x7f0c2c02a270 comp rx=0 tx=0).stop 2024-04-25T08:31:36.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.443+0000 7f0c419d5640 1 -- 172.21.15.38:0/3363547039 shutdown_connections 2024-04-25T08:31:36.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.443+0000 7f0c419d5640 1 --2- 172.21.15.38:0/3363547039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d19e0 0x7f0c3c0d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:36.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.443+0000 7f0c419d5640 1 --2- 172.21.15.38:0/3363547039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c3c0d0860 0x7f0c3c0d0c40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:36.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.443+0000 7f0c419d5640 1 -- 172.21.15.38:0/3363547039 >> 172.21.15.38:0/3363547039 conn(0x7f0c3c066d10 msgr2=0x7f0c3c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:36.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.443+0000 7f0c419d5640 1 -- 172.21.15.38:0/3363547039 shutdown_connections 2024-04-25T08:31:36.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.443+0000 7f0c419d5640 1 -- 172.21.15.38:0/3363547039 wait complete. 2024-04-25T08:31:36.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.444+0000 7f0c419d5640 1 Processor -- start 2024-04-25T08:31:36.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.444+0000 7f0c419d5640 1 -- start start 2024-04-25T08:31:36.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c419d5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d0860 0x7f0c3c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:36.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c419d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c3c0d19e0 0x7f0c3c0c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:36.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c419d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c3c0d3860 0x7f0c3c0c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:36.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c419d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0d3540 con 0x7f0c3c0d3860 2024-04-25T08:31:36.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c419d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0d2d40 con 0x7f0c3c0d19e0 2024-04-25T08:31:36.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c419d5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0b9770 con 0x7f0c3c0d0860 2024-04-25T08:31:36.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c3c0d3860 0x7f0c3c0c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:36.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c3b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c3c0d3860 0x7f0c3c0c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39448/0 (socket says 172.21.15.38:39448) 2024-04-25T08:31:36.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c3b7fe640 1 -- 172.21.15.38:0/213296034 learned_addr learned my addr 172.21.15.38:0/213296034 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:36.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c3a7fc640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c3c0d19e0 0x7f0c3c0c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:36.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c3affd640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d0860 0x7f0c3c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:36.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c3b7fe640 1 -- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d0860 msgr2=0x7f0c3c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:36.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c3b7fe640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d0860 0x7f0c3c0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:36.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c3b7fe640 1 -- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c3c0d19e0 msgr2=0x7f0c3c0c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:36.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c3b7fe640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c3c0d19e0 0x7f0c3c0c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:36.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.445+0000 7f0c3b7fe640 1 -- 172.21.15.38:0/213296034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c0cd370 con 0x7f0c3c0d3860 2024-04-25T08:31:36.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.446+0000 7f0c3b7fe640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c3c0d3860 0x7f0c3c0c9e80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400c6b0 tx=0x7f0c3400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:36.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.446+0000 7f0c23fff640 1 -- 172.21.15.38:0/213296034 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34003b70 con 0x7f0c3c0d3860 2024-04-25T08:31:36.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.446+0000 7f0c23fff640 1 -- 172.21.15.38:0/213296034 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c34003d30 con 0x7f0c3c0d3860 2024-04-25T08:31:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.446+0000 7f0c23fff640 1 -- 172.21.15.38:0/213296034 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34005310 con 0x7f0c3c0d3860 2024-04-25T08:31:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.446+0000 7f0c419d5640 1 -- 172.21.15.38:0/213296034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3c0cb890 con 0x7f0c3c0d3860 2024-04-25T08:31:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.446+0000 7f0c419d5640 1 -- 172.21.15.38:0/213296034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3c0cbe50 con 0x7f0c3c0d3860 2024-04-25T08:31:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.447+0000 7f0c419d5640 1 -- 172.21.15.38:0/213296034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3c068e80 con 0x7f0c3c0d3860 2024-04-25T08:31:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.447+0000 7f0c23fff640 1 -- 172.21.15.38:0/213296034 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c340054b0 con 0x7f0c3c0d3860 2024-04-25T08:31:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.448+0000 7f0c23fff640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0c2806b4d0 0x7f0c2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.448+0000 7f0c23fff640 1 -- 172.21.15.38:0/213296034 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c34089f20 con 0x7f0c3c0d3860 2024-04-25T08:31:36.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.448+0000 7f0c3affd640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0c2806b4d0 0x7f0c2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.448+0000 7f0c3affd640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0c2806b4d0 0x7f0c2806d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c034000 tx=0x7f0c2c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:36.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.452+0000 7f0c23fff640 1 -- 172.21.15.38:0/213296034 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c34058750 con 0x7f0c3c0d3860 2024-04-25T08:31:36.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:36.640+0000 7f0c419d5640 1 -- 172.21.15.38:0/213296034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f0c3c0b3640 con 0x7f0c3c0d3860 2024-04-25T08:31:37.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.316+0000 7f0c23fff640 1 -- 172.21.15.38:0/213296034 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f0c3405c7f0 con 0x7f0c3c0d3860 2024-04-25T08:31:37.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.319+0000 7f0c419d5640 1 -- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0c2806b4d0 msgr2=0x7f0c2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:37.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.319+0000 7f0c419d5640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0c2806b4d0 0x7f0c2806d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c034000 tx=0x7f0c2c02e000 comp rx=0 tx=0).stop 2024-04-25T08:31:37.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.319+0000 7f0c419d5640 1 -- 172.21.15.38:0/213296034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c3c0d3860 msgr2=0x7f0c3c0c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:37.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.319+0000 7f0c419d5640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c3c0d3860 0x7f0c3c0c9e80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400c6b0 tx=0x7f0c3400cb70 comp rx=0 tx=0).stop 2024-04-25T08:31:37.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.319+0000 7f0c419d5640 1 -- 172.21.15.38:0/213296034 shutdown_connections 2024-04-25T08:31:37.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.319+0000 7f0c419d5640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c3c0d3860 0x7f0c3c0c9e80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:37.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.319+0000 7f0c419d5640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c3c0d19e0 0x7f0c3c0c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:37.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.319+0000 7f0c419d5640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0c2806b4d0 0x7f0c2806d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:37.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.319+0000 7f0c419d5640 1 --2- 172.21.15.38:0/213296034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c3c0d0860 0x7f0c3c0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:37.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.319+0000 7f0c419d5640 1 -- 172.21.15.38:0/213296034 >> 172.21.15.38:0/213296034 conn(0x7f0c3c066d10 msgr2=0x7f0c3c0d3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:37.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.320+0000 7f0c419d5640 1 -- 172.21.15.38:0/213296034 shutdown_connections 2024-04-25T08:31:37.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.320+0000 7f0c419d5640 1 -- 172.21.15.38:0/213296034 wait complete. 2024-04-25T08:31:37.330 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:31:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.448+0000 7f5a77f5a640 1 Processor -- start 2024-04-25T08:31:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.448+0000 7f5a77f5a640 1 -- start start 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.449+0000 7f5a77f5a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a700d0320 0x7f5a700d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.449+0000 7f5a77f5a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a700d2060 con 0x7f5a700b6980 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.449+0000 7f5a77f5a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a700d2330 con 0x7f5a700d0700 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.449+0000 7f5a77f5a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700d2610 con 0x7f5a700d0320 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.449+0000 7f5a764f7640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5a700b6980 0x7f5a700b6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33002/0 (socket says 172.21.15.38:33002) 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.449+0000 7f5a764f7640 1 -- 172.21.15.38:0/3567254789 learned_addr learned my addr 172.21.15.38:0/3567254789 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.449+0000 7f5a75cf6640 1 --2- 172.21.15.38:0/3567254789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a700d0320 0x7f5a700d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.449+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766579625 0 0) 0x7f5a700d2060 con 0x7f5a700b6980 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.449+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a58003520 con 0x7f5a700b6980 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 263441581 0 0) 0x7f5a58003520 con 0x7f5a700b6980 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a700d0320 msgr2=0x7f5a700d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 --2- 172.21.15.38:0/3567254789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a700d0320 0x7f5a700d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 >> v1:172.21.15.107:6789/0 conn(0x7f5a700d0700 legacy=0x7f5a700d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:37.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a700d28f0 con 0x7f5a700b6980 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f5a600019f0 con 0x7f5a700b6980 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 >> v1:172.21.15.38:6789/0 conn(0x7f5a700b6980 legacy=0x7f5a700b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 --2- 172.21.15.38:0/3567254789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a580039e0 0x7f5a58003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 --2- 172.21.15.38:0/3567254789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a700d0320 0x7f5a58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 --2- 172.21.15.38:0/3567254789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a58004d40 0x7f5a580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700d2610 con 0x7f5a58004d40 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5800b9c0 con 0x7f5a580039e0 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5800bbc0 con 0x7f5a700d0320 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a764f7640 1 --2- 172.21.15.38:0/3567254789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a58004d40 0x7f5a580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a75cf6640 1 --2- 172.21.15.38:0/3567254789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a580039e0 0x7f5a58003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a754f5640 1 --2- 172.21.15.38:0/3567254789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a700d0320 0x7f5a58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a764f7640 1 -- 172.21.15.38:0/3567254789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a700d0320 msgr2=0x7f5a58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a764f7640 1 --2- 172.21.15.38:0/3567254789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a700d0320 0x7f5a58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a764f7640 1 -- 172.21.15.38:0/3567254789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a580039e0 msgr2=0x7f5a58003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a764f7640 1 --2- 172.21.15.38:0/3567254789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a580039e0 0x7f5a58003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.450+0000 7f5a764f7640 1 -- 172.21.15.38:0/3567254789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a58003520 con 0x7f5a58004d40 2024-04-25T08:31:37.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.451+0000 7f5a764f7640 1 --2- 172.21.15.38:0/3567254789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a58004d40 0x7f5a580051c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000a810 tx=0x7f5a58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bdbf1d03a77772 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:37.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.451+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000a430 con 0x7f5a58004d40 2024-04-25T08:31:37.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.451+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6002f070 con 0x7f5a58004d40 2024-04-25T08:31:37.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.451+0000 7f5a74cf4640 1 -- 172.21.15.38:0/3567254789 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6002bc50 con 0x7f5a58004d40 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.452+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3567254789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a58004d40 msgr2=0x7f5a580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.452+0000 7f5a77f5a640 1 --2- 172.21.15.38:0/3567254789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a58004d40 0x7f5a580051c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000a810 tx=0x7f5a58005830 comp rx=0 tx=0).stop 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.452+0000 7f5a764f7640 1 -- 172.21.15.38:0/3567254789 reap_dead start 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.452+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3567254789 shutdown_connections 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.452+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3567254789 >> 172.21.15.38:0/3567254789 conn(0x7f5a700668f0 msgr2=0x7f5a70066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.452+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3567254789 shutdown_connections 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.452+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3567254789 wait complete. 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.453+0000 7f5a77f5a640 1 Processor -- start 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.453+0000 7f5a77f5a640 1 -- start start 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a77f5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70159410 0x7f5a7015b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a77f5a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a701587e0 0x7f5a70156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a77f5a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a70156b50 0x7f5a701f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a77f5a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700d0b40 con 0x7f5a70159410 2024-04-25T08:31:37.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a77f5a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a700d2eb0 con 0x7f5a70156b50 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a77f5a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7015cf70 con 0x7f5a701587e0 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a75cf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70159410 0x7f5a7015b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a75cf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70159410 0x7f5a7015b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39456/0 (socket says 172.21.15.38:39456) 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a75cf6640 1 -- 172.21.15.38:0/3614020910 learned_addr learned my addr 172.21.15.38:0/3614020910 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a764f7640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a70156b50 0x7f5a701f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a754f5640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a701587e0 0x7f5a70156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a75cf6640 1 -- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a701587e0 msgr2=0x7f5a70156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a75cf6640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a701587e0 0x7f5a70156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a75cf6640 1 -- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a70156b50 msgr2=0x7f5a701f6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a75cf6640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a70156b50 0x7f5a701f6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a75cf6640 1 -- 172.21.15.38:0/3614020910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7015bed0 con 0x7f5a70159410 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a764f7640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a70156b50 0x7f5a701f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a754f5640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a701587e0 0x7f5a70156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a75cf6640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70159410 0x7f5a7015b5d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5a68011a00 tx=0x7f5a68011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a6e7fc640 1 -- 172.21.15.38:0/3614020910 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800faf0 con 0x7f5a70159410 2024-04-25T08:31:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a6e7fc640 1 -- 172.21.15.38:0/3614020910 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6800fc90 con 0x7f5a70159410 2024-04-25T08:31:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a6e7fc640 1 -- 172.21.15.38:0/3614020910 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68014440 con 0x7f5a70159410 2024-04-25T08:31:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.454+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3614020910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7015c1b0 con 0x7f5a70159410 2024-04-25T08:31:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.455+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3614020910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7015c770 con 0x7f5a70159410 2024-04-25T08:31:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.456+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3614020910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a70068e80 con 0x7f5a70159410 2024-04-25T08:31:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.456+0000 7f5a6e7fc640 1 -- 172.21.15.38:0/3614020910 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a68012040 con 0x7f5a70159410 2024-04-25T08:31:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.456+0000 7f5a6e7fc640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a5c06b4d0 0x7f5a5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.456+0000 7f5a6e7fc640 1 -- 172.21.15.38:0/3614020910 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6007+0+0 (secure 0 0 0) 0x7f5a6808a910 con 0x7f5a70159410 2024-04-25T08:31:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.456+0000 7f5a754f5640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a5c06b4d0 0x7f5a5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:37.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.457+0000 7f5a754f5640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a5c06b4d0 0x7f5a5c06d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a700b3990 tx=0x7f5a64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:37.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.460+0000 7f5a6e7fc640 1 -- 172.21.15.38:0/3614020910 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a68019030 con 0x7f5a70159410 2024-04-25T08:31:37.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:37.642+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3614020910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5a70156f30 con 0x7f5a70159410 2024-04-25T08:31:39.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.324+0000 7f5a6e7fc640 1 -- 172.21.15.38:0/3614020910 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7f5a680596f0 con 0x7f5a70159410 2024-04-25T08:31:39.587 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data_ec' created 2024-04-25T08:31:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.326+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a5c06b4d0 msgr2=0x7f5a5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.326+0000 7f5a77f5a640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a5c06b4d0 0x7f5a5c06d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5a700b3990 tx=0x7f5a64009040 comp rx=0 tx=0).stop 2024-04-25T08:31:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.326+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3614020910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70159410 msgr2=0x7f5a7015b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.326+0000 7f5a77f5a640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70159410 0x7f5a7015b5d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5a68011a00 tx=0x7f5a68011ec0 comp rx=0 tx=0).stop 2024-04-25T08:31:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.327+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3614020910 shutdown_connections 2024-04-25T08:31:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.327+0000 7f5a77f5a640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a70156b50 0x7f5a701f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.327+0000 7f5a77f5a640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a5c06b4d0 0x7f5a5c06d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.327+0000 7f5a77f5a640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a701587e0 0x7f5a70156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.327+0000 7f5a77f5a640 1 --2- 172.21.15.38:0/3614020910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a70159410 0x7f5a7015b5d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.327+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3614020910 >> 172.21.15.38:0/3614020910 conn(0x7f5a700668f0 msgr2=0x7f5a700b8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.327+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3614020910 shutdown_connections 2024-04-25T08:31:39.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.327+0000 7f5a77f5a640 1 -- 172.21.15.38:0/3614020910 wait complete. 2024-04-25T08:31:39.590 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T08:31:39.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.710+0000 7f9dcbde2640 1 Processor -- start 2024-04-25T08:31:39.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.710+0000 7f9dcbde2640 1 -- start start 2024-04-25T08:31:39.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dcbde2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc40d19e0 0x7f9dc40d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:39.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dcbde2640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9dc40b6a00 0x7f9dc40b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:39.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dcbde2640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc40d24a0 con 0x7f9dc40d0860 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dcbde2640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc415e2b0 con 0x7f9dc40b6a00 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dcbde2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40d2780 con 0x7f9dc40d19e0 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc9b7e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9dc40d0860 0x7f9dc40d0c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51808/0 (socket says 172.21.15.38:51808) 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc9b7e640 1 -- 172.21.15.38:0/2576751034 learned_addr learned my addr 172.21.15.38:0/2576751034 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dca37f640 1 --2- 172.21.15.38:0/2576751034 >> v2:172.21.15.107:3300/0 conn(0x7f9dc40b6a00 0x7f9dc40b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc937d640 1 --2- 172.21.15.38:0/2576751034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc40d19e0 0x7f9dc40d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284405098 0 0) 0x7f9dc40d24a0 con 0x7f9dc40d0860 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9db0003520 con 0x7f9dc40d0860 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4257516220 0 0) 0x7f9db0003520 con 0x7f9dc40d0860 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc40d19e0 msgr2=0x7f9dc40d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc8b7c640 1 --2- 172.21.15.38:0/2576751034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc40d19e0 0x7f9dc40d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 >> v2:172.21.15.107:3300/0 conn(0x7f9dc40b6a00 msgr2=0x7f9dc40b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc8b7c640 1 --2- 172.21.15.38:0/2576751034 >> v2:172.21.15.107:3300/0 conn(0x7f9dc40b6a00 0x7f9dc40b6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc40d2a60 con 0x7f9dc40d0860 2024-04-25T08:31:39.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f9dc0001670 con 0x7f9dc40d0860 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dca37f640 1 --2- 172.21.15.38:0/2576751034 >> v2:172.21.15.107:3300/0 conn(0x7f9dc40b6a00 0x7f9dc40b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 >> v1:172.21.15.38:6789/0 conn(0x7f9dc40d0860 legacy=0x7f9dc40d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.711+0000 7f9dc8b7c640 1 --2- 172.21.15.38:0/2576751034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db00039b0 0x7f9db0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dc8b7c640 1 --2- 172.21.15.38:0/2576751034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9db0004800 0x7f9db0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dc8b7c640 1 --2- 172.21.15.38:0/2576751034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc40d19e0 0x7f9db00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc415e2b0 con 0x7f9db00039b0 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40d2780 con 0x7f9db0004800 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db000b9c0 con 0x7f9dc40d19e0 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dc9b7e640 1 --2- 172.21.15.38:0/2576751034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db00039b0 0x7f9db0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dc937d640 1 --2- 172.21.15.38:0/2576751034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9db0004800 0x7f9db0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dca37f640 1 --2- 172.21.15.38:0/2576751034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc40d19e0 0x7f9db00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dc9b7e640 1 -- 172.21.15.38:0/2576751034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc40d19e0 msgr2=0x7f9db00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dc9b7e640 1 --2- 172.21.15.38:0/2576751034 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc40d19e0 0x7f9db00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:39.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dc9b7e640 1 -- 172.21.15.38:0/2576751034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9db0004800 msgr2=0x7f9db0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dc9b7e640 1 --2- 172.21.15.38:0/2576751034 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9db0004800 0x7f9db0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.712+0000 7f9dc9b7e640 1 -- 172.21.15.38:0/2576751034 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db0003520 con 0x7f9db00039b0 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.713+0000 7f9dc9b7e640 1 --2- 172.21.15.38:0/2576751034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db00039b0 0x7f9db0003dd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000c240 tx=0x7f9db0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f284b14d01197228 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.713+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc002e070 con 0x7f9db00039b0 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.713+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc0005af0 con 0x7f9db00039b0 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.713+0000 7f9dc8b7c640 1 -- 172.21.15.38:0/2576751034 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0007d30 con 0x7f9db00039b0 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.713+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2576751034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db00039b0 msgr2=0x7f9db0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.713+0000 7f9dcbde2640 1 --2- 172.21.15.38:0/2576751034 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9db00039b0 0x7f9db0003dd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000c240 tx=0x7f9db0005830 comp rx=0 tx=0).stop 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.713+0000 7f9dca37f640 1 -- 172.21.15.38:0/2576751034 reap_dead start 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.713+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2576751034 shutdown_connections 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.713+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2576751034 >> 172.21.15.38:0/2576751034 conn(0x7f9dc4066d10 msgr2=0x7f9dc4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.714+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2576751034 shutdown_connections 2024-04-25T08:31:39.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.714+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2576751034 wait complete. 2024-04-25T08:31:39.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.715+0000 7f9dcbde2640 1 Processor -- start 2024-04-25T08:31:39.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.715+0000 7f9dcbde2640 1 -- start start 2024-04-25T08:31:39.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.715+0000 7f9dcbde2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9db00039b0 0x7f9dc40cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:39.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dcbde2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc41e40b0 0x7f9dc41e4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:39.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dcbde2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc4159ce0 0x7f9dc415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:39.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dcbde2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40b9770 con 0x7f9dc4159ce0 2024-04-25T08:31:39.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dcbde2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40d3540 con 0x7f9db00039b0 2024-04-25T08:31:39.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dcbde2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc40b98f0 con 0x7f9dc41e40b0 2024-04-25T08:31:39.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dca37f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc4159ce0 0x7f9dc415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dca37f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc4159ce0 0x7f9dc415a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39972/0 (socket says 172.21.15.38:39972) 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dca37f640 1 -- 172.21.15.38:0/2040746144 learned_addr learned my addr 172.21.15.38:0/2040746144 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dc937d640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc41e40b0 0x7f9dc41e4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dc9b7e640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9db00039b0 0x7f9dc40cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dca37f640 1 -- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc41e40b0 msgr2=0x7f9dc41e4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dca37f640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc41e40b0 0x7f9dc41e4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dca37f640 1 -- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9db00039b0 msgr2=0x7f9dc40cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dca37f640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9db00039b0 0x7f9dc40cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dca37f640 1 -- 172.21.15.38:0/2040746144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc40c7510 con 0x7f9dc4159ce0 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dc9b7e640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9db00039b0 0x7f9dc40cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dca37f640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc4159ce0 0x7f9dc415a160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40b6440 tx=0x7f9dbc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dba7fc640 1 -- 172.21.15.38:0/2040746144 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc003b90 con 0x7f9dc4159ce0 2024-04-25T08:31:39.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dba7fc640 1 -- 172.21.15.38:0/2040746144 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dbc003d30 con 0x7f9dc4159ce0 2024-04-25T08:31:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2040746144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc40c77f0 con 0x7f9dc4159ce0 2024-04-25T08:31:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dba7fc640 1 -- 172.21.15.38:0/2040746144 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc004260 con 0x7f9dc4159ce0 2024-04-25T08:31:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.716+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2040746144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc40c7db0 con 0x7f9dc4159ce0 2024-04-25T08:31:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.717+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2040746144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc40c7fb0 con 0x7f9dc4159ce0 2024-04-25T08:31:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.718+0000 7f9dba7fc640 1 -- 172.21.15.38:0/2040746144 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9dbc004400 con 0x7f9dc4159ce0 2024-04-25T08:31:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.718+0000 7f9dba7fc640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9da806b4d0 0x7f9da806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.719+0000 7f9dba7fc640 1 -- 172.21.15.38:0/2040746144 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f9dbc08b290 con 0x7f9dc4159ce0 2024-04-25T08:31:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.719+0000 7f9dc9b7e640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9da806b4d0 0x7f9da806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:39.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.719+0000 7f9dc9b7e640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9da806b4d0 0x7f9da806d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0033000 tx=0x7f9dc0009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:39.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.723+0000 7f9dba7fc640 1 -- 172.21.15.38:0/2040746144 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9dc40c7fb0 con 0x7f9dc4159ce0 2024-04-25T08:31:39.884 INFO:tasks.ceph.mds.b.smithi107.stderr:2024-04-25T08:31:39.887+0000 7fa78f396640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:31:39.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:39.902+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2040746144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f9dc4068e80 con 0x7f9dc4159ce0 2024-04-25T08:31:40.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.326+0000 7f9dba7fc640 1 -- 172.21.15.38:0/2040746144 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9dbc019030 con 0x7f9dc4159ce0 2024-04-25T08:31:40.322 INFO:teuthology.orchestra.run.smithi038.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T08:31:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.328+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9da806b4d0 msgr2=0x7f9da806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.328+0000 7f9dcbde2640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9da806b4d0 0x7f9da806d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0033000 tx=0x7f9dc0009030 comp rx=0 tx=0).stop 2024-04-25T08:31:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.328+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2040746144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc4159ce0 msgr2=0x7f9dc415a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.328+0000 7f9dcbde2640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc4159ce0 0x7f9dc415a160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40b6440 tx=0x7f9dbc00f580 comp rx=0 tx=0).stop 2024-04-25T08:31:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.328+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2040746144 shutdown_connections 2024-04-25T08:31:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.328+0000 7f9dcbde2640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc4159ce0 0x7f9dc415a160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.328+0000 7f9dcbde2640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9da806b4d0 0x7f9da806d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.328+0000 7f9dcbde2640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc41e40b0 0x7f9dc41e4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.328+0000 7f9dcbde2640 1 --2- 172.21.15.38:0/2040746144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9db00039b0 0x7f9dc40cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.328+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2040746144 >> 172.21.15.38:0/2040746144 conn(0x7f9dc4066d10 msgr2=0x7f9dc40b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.328+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2040746144 shutdown_connections 2024-04-25T08:31:40.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.328+0000 7f9dcbde2640 1 -- 172.21.15.38:0/2040746144 wait complete. 2024-04-25T08:31:40.344 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T08:31:40.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.455+0000 7fca14fc8640 1 Processor -- start 2024-04-25T08:31:40.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.455+0000 7fca14fc8640 1 -- start start 2024-04-25T08:31:40.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.456+0000 7fca14fc8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fca1015d980 0x7fca100d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.456+0000 7fca14fc8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca100d0ef0 0x7fca10167520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.456+0000 7fca14fc8640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca100cec90 con 0x7fca100b2580 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.456+0000 7fca14fc8640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca100d2450 con 0x7fca1015d980 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.456+0000 7fca14fc8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca100d2730 con 0x7fca100d0ef0 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.456+0000 7fca0e59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fca1015d980 0x7fca100d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.456+0000 7fca0e59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fca1015d980 0x7fca100d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39980/0 (socket says 172.21.15.38:39980) 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.456+0000 7fca0e59c640 1 -- 172.21.15.38:0/860279855 learned_addr learned my addr 172.21.15.38:0/860279855 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.457+0000 7fca0ed9d640 1 --2- 172.21.15.38:0/860279855 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca100d0ef0 0x7fca10167520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.457+0000 7fca0e59c640 1 -- 172.21.15.38:0/860279855 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca100d0ef0 msgr2=0x7fca10167520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.457+0000 7fca0e59c640 1 --2- 172.21.15.38:0/860279855 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca100d0ef0 0x7fca10167520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.457+0000 7fca0e59c640 1 -- 172.21.15.38:0/860279855 >> v1:172.21.15.107:6789/0 conn(0x7fca100b2580 legacy=0x7fca100d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.457+0000 7fca0e59c640 1 -- 172.21.15.38:0/860279855 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca100d2a10 con 0x7fca1015d980 2024-04-25T08:31:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.457+0000 7fca0e59c640 1 --2- 172.21.15.38:0/860279855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca1015d980 0x7fca100d02a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fca08009870 tx=0x7fca0802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55240afd0e0c0210 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:40.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.458+0000 7fca0d59a640 1 -- 172.21.15.38:0/860279855 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08037070 con 0x7fca1015d980 2024-04-25T08:31:40.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.458+0000 7fca0d59a640 1 -- 172.21.15.38:0/860279855 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca08032440 con 0x7fca1015d980 2024-04-25T08:31:40.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.458+0000 7fca0d59a640 1 -- 172.21.15.38:0/860279855 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca08032810 con 0x7fca1015d980 2024-04-25T08:31:40.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.458+0000 7fca14fc8640 1 -- 172.21.15.38:0/860279855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca1015d980 msgr2=0x7fca100d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:40.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.458+0000 7fca14fc8640 1 --2- 172.21.15.38:0/860279855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca1015d980 0x7fca100d02a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fca08009870 tx=0x7fca0802a180 comp rx=0 tx=0).stop 2024-04-25T08:31:40.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.458+0000 7fca14fc8640 1 -- 172.21.15.38:0/860279855 shutdown_connections 2024-04-25T08:31:40.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.458+0000 7fca14fc8640 1 --2- 172.21.15.38:0/860279855 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca100d0ef0 0x7fca10167520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:40.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.458+0000 7fca14fc8640 1 --2- 172.21.15.38:0/860279855 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca1015d980 0x7fca100d02a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:40.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.458+0000 7fca14fc8640 1 -- 172.21.15.38:0/860279855 >> 172.21.15.38:0/860279855 conn(0x7fca1006bed0 msgr2=0x7fca1006c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:40.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.458+0000 7fca14fc8640 1 -- 172.21.15.38:0/860279855 shutdown_connections 2024-04-25T08:31:40.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.458+0000 7fca14fc8640 1 -- 172.21.15.38:0/860279855 wait complete. 2024-04-25T08:31:40.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.460+0000 7fca14fc8640 1 Processor -- start 2024-04-25T08:31:40.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.460+0000 7fca14fc8640 1 -- start start 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.460+0000 7fca14fc8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca100b2580 0x7fca100cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.460+0000 7fca14fc8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca100d0ef0 0x7fca100cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.460+0000 7fca14fc8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca1015d980 0x7fca101e7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.460+0000 7fca14fc8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca100cea50 con 0x7fca1015d980 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.460+0000 7fca14fc8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca100ce8d0 con 0x7fca100b2580 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.460+0000 7fca14fc8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca100d3f40 con 0x7fca100d0ef0 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca0ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca1015d980 0x7fca101e7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca0ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca1015d980 0x7fca101e7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39990/0 (socket says 172.21.15.38:39990) 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca0ed9d640 1 -- 172.21.15.38:0/3754564848 learned_addr learned my addr 172.21.15.38:0/3754564848 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca0dd9b640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca100d0ef0 0x7fca100cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca0e59c640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca100b2580 0x7fca100cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca0ed9d640 1 -- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca100d0ef0 msgr2=0x7fca100cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:40.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca0ed9d640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca100d0ef0 0x7fca100cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:40.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca0ed9d640 1 -- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca100b2580 msgr2=0x7fca100cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:40.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca0ed9d640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca100b2580 0x7fca100cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:40.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca0ed9d640 1 -- 172.21.15.38:0/3754564848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca100ce050 con 0x7fca1015d980 2024-04-25T08:31:40.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca0ed9d640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca1015d980 0x7fca101e7190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca00007d90 tx=0x7fca0000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:40.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fc9feffd640 1 -- 172.21.15.38:0/3754564848 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00014420 con 0x7fca1015d980 2024-04-25T08:31:40.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fc9feffd640 1 -- 172.21.15.38:0/3754564848 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca00019070 con 0x7fca1015d980 2024-04-25T08:31:40.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca14fc8640 1 -- 172.21.15.38:0/3754564848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca100cd4d0 con 0x7fca1015d980 2024-04-25T08:31:40.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fc9feffd640 1 -- 172.21.15.38:0/3754564848 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00014820 con 0x7fca1015d980 2024-04-25T08:31:40.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.461+0000 7fca14fc8640 1 -- 172.21.15.38:0/3754564848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca100cda90 con 0x7fca1015d980 2024-04-25T08:31:40.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.462+0000 7fca14fc8640 1 -- 172.21.15.38:0/3754564848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca10068e80 con 0x7fca1015d980 2024-04-25T08:31:40.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.463+0000 7fc9feffd640 1 -- 172.21.15.38:0/3754564848 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca000149c0 con 0x7fca1015d980 2024-04-25T08:31:40.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.463+0000 7fc9feffd640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc9f006b4d0 0x7fc9f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:40.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.463+0000 7fc9feffd640 1 -- 172.21.15.38:0/3754564848 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fca0008acd0 con 0x7fca1015d980 2024-04-25T08:31:40.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.463+0000 7fca0e59c640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc9f006b4d0 0x7fc9f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:40.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.464+0000 7fca0e59c640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc9f006b4d0 0x7fc9f006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fca08034000 tx=0x7fca0802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:40.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.467+0000 7fc9feffd640 1 -- 172.21.15.38:0/3754564848 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca00059140 con 0x7fca1015d980 2024-04-25T08:31:40.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:40.690+0000 7fca14fc8640 1 -- 172.21.15.38:0/3754564848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fca100d0810 con 0x7fca1015d980 2024-04-25T08:31:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.389+0000 7fc9feffd640 1 -- 172.21.15.38:0/3754564848 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7fca0005d1e0 con 0x7fca1015d980 2024-04-25T08:31:41.386 INFO:teuthology.orchestra.run.smithi038.stderr:added data pool 4 to fsmap 2024-04-25T08:31:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.392+0000 7fca14fc8640 1 -- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc9f006b4d0 msgr2=0x7fc9f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.392+0000 7fca14fc8640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc9f006b4d0 0x7fc9f006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fca08034000 tx=0x7fca0802e000 comp rx=0 tx=0).stop 2024-04-25T08:31:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.392+0000 7fca14fc8640 1 -- 172.21.15.38:0/3754564848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca1015d980 msgr2=0x7fca101e7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.392+0000 7fca14fc8640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca1015d980 0x7fca101e7190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca00007d90 tx=0x7fca0000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.392+0000 7fca14fc8640 1 -- 172.21.15.38:0/3754564848 shutdown_connections 2024-04-25T08:31:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.392+0000 7fca14fc8640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca1015d980 0x7fca101e7190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.392+0000 7fca14fc8640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc9f006b4d0 0x7fc9f006d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.392+0000 7fca14fc8640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca100d0ef0 0x7fca100cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.392+0000 7fca14fc8640 1 --2- 172.21.15.38:0/3754564848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca100b2580 0x7fca100cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.392+0000 7fca14fc8640 1 -- 172.21.15.38:0/3754564848 >> 172.21.15.38:0/3754564848 conn(0x7fca1006bed0 msgr2=0x7fca1015e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.392+0000 7fca14fc8640 1 -- 172.21.15.38:0/3754564848 shutdown_connections 2024-04-25T08:31:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.392+0000 7fca14fc8640 1 -- 172.21.15.38:0/3754564848 wait complete. 2024-04-25T08:31:41.401 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:31:41.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.510+0000 7f34b7609640 1 Processor -- start 2024-04-25T08:31:41.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.510+0000 7f34b7609640 1 -- start start 2024-04-25T08:31:41.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.511+0000 7f34b7609640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f34b009ab90 0x7f34b015f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:41.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.511+0000 7f34b7609640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b0160340 con 0x7f34b00d0460 2024-04-25T08:31:41.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.511+0000 7f34b7609640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b00cf400 con 0x7f34b015bb10 2024-04-25T08:31:41.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.511+0000 7f34b7609640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b00cffd0 con 0x7f34b009ab90 2024-04-25T08:31:41.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.511+0000 7f34b4ba4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f34b009ab90 0x7f34b015f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:41.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.511+0000 7f34b5ba6640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f34b00d0460 0x7f34b00c5250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51824/0 (socket says 172.21.15.38:51824) 2024-04-25T08:31:41.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.511+0000 7f34b5ba6640 1 -- 172.21.15.38:0/1810123651 learned_addr learned my addr 172.21.15.38:0/1810123651 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:41.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.512+0000 7f34b4ba4640 1 -- 172.21.15.38:0/1810123651 >> v1:172.21.15.107:6789/0 conn(0x7f34b015bb10 legacy=0x7f34b009a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:41.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.512+0000 7f34b4ba4640 1 -- 172.21.15.38:0/1810123651 >> v1:172.21.15.38:6789/0 conn(0x7f34b00d0460 legacy=0x7f34b00c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:41.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.512+0000 7f34b4ba4640 1 -- 172.21.15.38:0/1810123651 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b00cf6a0 con 0x7f34b009ab90 2024-04-25T08:31:41.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.512+0000 7f34b4ba4640 1 --2- 172.21.15.38:0/1810123651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34b009ab90 0x7f34b015f4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f34a40097a0 tx=0x7f34a402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a48ccfd7a2ef27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:41.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.512+0000 7f34affff640 1 -- 172.21.15.38:0/1810123651 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4037070 con 0x7f34b009ab90 2024-04-25T08:31:41.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.513+0000 7f34affff640 1 -- 172.21.15.38:0/1810123651 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a402ae50 con 0x7f34b009ab90 2024-04-25T08:31:41.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.513+0000 7f34affff640 1 -- 172.21.15.38:0/1810123651 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4032660 con 0x7f34b009ab90 2024-04-25T08:31:41.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.513+0000 7f34b7609640 1 -- 172.21.15.38:0/1810123651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34b009ab90 msgr2=0x7f34b015f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:41.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.513+0000 7f34b7609640 1 --2- 172.21.15.38:0/1810123651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34b009ab90 0x7f34b015f4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f34a40097a0 tx=0x7f34a402a140 comp rx=0 tx=0).stop 2024-04-25T08:31:41.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.513+0000 7f34b7609640 1 -- 172.21.15.38:0/1810123651 shutdown_connections 2024-04-25T08:31:41.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.513+0000 7f34b7609640 1 --2- 172.21.15.38:0/1810123651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34b009ab90 0x7f34b015f4e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.513+0000 7f34b7609640 1 -- 172.21.15.38:0/1810123651 >> 172.21.15.38:0/1810123651 conn(0x7f34b00b31f0 msgr2=0x7f34b00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:41.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.513+0000 7f34b7609640 1 -- 172.21.15.38:0/1810123651 shutdown_connections 2024-04-25T08:31:41.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.513+0000 7f34b7609640 1 -- 172.21.15.38:0/1810123651 wait complete. 2024-04-25T08:31:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.514+0000 7f34b7609640 1 Processor -- start 2024-04-25T08:31:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.514+0000 7f34b7609640 1 -- start start 2024-04-25T08:31:41.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.515+0000 7f34b7609640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34b009ab90 0x7f34b00b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:41.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.515+0000 7f34b7609640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34b00d0460 0x7f34b00b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:41.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.515+0000 7f34b7609640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b015bb10 0x7f34b00b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:41.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.515+0000 7f34b7609640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b015fa20 con 0x7f34b009ab90 2024-04-25T08:31:41.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.515+0000 7f34b7609640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b00cfc60 con 0x7f34b015bb10 2024-04-25T08:31:41.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.515+0000 7f34b7609640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b015fba0 con 0x7f34b00d0460 2024-04-25T08:31:41.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.515+0000 7f34b53a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34b009ab90 0x7f34b00b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:41.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.515+0000 7f34b53a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34b009ab90 0x7f34b00b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39998/0 (socket says 172.21.15.38:39998) 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.515+0000 7f34b53a5640 1 -- 172.21.15.38:0/558107191 learned_addr learned my addr 172.21.15.38:0/558107191 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.515+0000 7f34b53a5640 1 -- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34b00d0460 msgr2=0x7f34b00b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.515+0000 7f34b4ba4640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34b00d0460 0x7f34b00b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.515+0000 7f34b5ba6640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b015bb10 0x7f34b00b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.516+0000 7f34b53a5640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34b00d0460 0x7f34b00b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.516+0000 7f34b53a5640 1 -- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b015bb10 msgr2=0x7f34b00b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.516+0000 7f34b53a5640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b015bb10 0x7f34b00b4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.516+0000 7f34b53a5640 1 -- 172.21.15.38:0/558107191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b00b4e20 con 0x7f34b009ab90 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.516+0000 7f34b53a5640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34b009ab90 0x7f34b00b5ec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f34a800b3f0 tx=0x7f34a800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.516+0000 7f34adffb640 1 -- 172.21.15.38:0/558107191 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8015020 con 0x7f34b009ab90 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.516+0000 7f34adffb640 1 -- 172.21.15.38:0/558107191 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a8011830 con 0x7f34b009ab90 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.516+0000 7f34b7609640 1 -- 172.21.15.38:0/558107191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b00b5100 con 0x7f34b009ab90 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.516+0000 7f34adffb640 1 -- 172.21.15.38:0/558107191 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8011c20 con 0x7f34b009ab90 2024-04-25T08:31:41.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.516+0000 7f34b7609640 1 -- 172.21.15.38:0/558107191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b00b56c0 con 0x7f34b009ab90 2024-04-25T08:31:41.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.517+0000 7f34b7609640 1 -- 172.21.15.38:0/558107191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b0068e80 con 0x7f34b009ab90 2024-04-25T08:31:41.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.518+0000 7f34adffb640 1 -- 172.21.15.38:0/558107191 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34a8018020 con 0x7f34b009ab90 2024-04-25T08:31:41.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.518+0000 7f34adffb640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f349c06b4d0 0x7f349c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:41.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.518+0000 7f34adffb640 1 -- 172.21.15.38:0/558107191 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f34a808a180 con 0x7f34b009ab90 2024-04-25T08:31:41.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.518+0000 7f34b4ba4640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f349c06b4d0 0x7f349c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:41.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.519+0000 7f34b4ba4640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f349c06b4d0 0x7f349c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f34a4034000 tx=0x7f34a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:41.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.522+0000 7f34adffb640 1 -- 172.21.15.38:0/558107191 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34a801f030 con 0x7f34b009ab90 2024-04-25T08:31:41.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.744+0000 7f34b7609640 1 -- 172.21.15.38:0/558107191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34b00cdbf0 con 0x7f34b009ab90 2024-04-25T08:31:41.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.744+0000 7f34adffb640 1 -- 172.21.15.38:0/558107191 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f34a8058cc0 con 0x7f34b009ab90 2024-04-25T08:31:41.741 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:41.741 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4242612806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4242612806},{"type":"v1","addr":"172.21.15.38:6835","nonce":4242612806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2154167637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":2154167637},{"type":"v1","addr":"172.21.15.107:6837","nonce":2154167637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/678452936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":678452936},{"type":"v1","addr":"172.21.15.38:6834","nonce":678452936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:31:35.665008+0000","modified":"2024-04-25T08:31:40.691473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6836/1104921666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1104921666},{"type":"v1","addr":"172.21.15.107:6836","nonce":1104921666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T08:31:41.741 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-04-25T08:31:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.746+0000 7f34b7609640 1 -- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f349c06b4d0 msgr2=0x7f349c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.746+0000 7f34b7609640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f349c06b4d0 0x7f349c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f34a4034000 tx=0x7f34a402e000 comp rx=0 tx=0).stop 2024-04-25T08:31:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.746+0000 7f34b7609640 1 -- 172.21.15.38:0/558107191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34b009ab90 msgr2=0x7f34b00b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.746+0000 7f34b7609640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34b009ab90 0x7f34b00b5ec0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f34a800b3f0 tx=0x7f34a800b8b0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.746+0000 7f34b7609640 1 -- 172.21.15.38:0/558107191 shutdown_connections 2024-04-25T08:31:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.746+0000 7f34b7609640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34b015bb10 0x7f34b00b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.746+0000 7f34b7609640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f349c06b4d0 0x7f349c06d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.746+0000 7f34b7609640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34b00d0460 0x7f34b00b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.746+0000 7f34b7609640 1 --2- 172.21.15.38:0/558107191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34b009ab90 0x7f34b00b5ec0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.747+0000 7f34b7609640 1 -- 172.21.15.38:0/558107191 >> 172.21.15.38:0/558107191 conn(0x7f34b00b31f0 msgr2=0x7f34b009a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.747+0000 7f34b7609640 1 -- 172.21.15.38:0/558107191 shutdown_connections 2024-04-25T08:31:41.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.747+0000 7f34b7609640 1 -- 172.21.15.38:0/558107191 wait complete. 2024-04-25T08:31:41.754 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:41.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.858+0000 7fdb4879d640 1 Processor -- start 2024-04-25T08:31:41.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.858+0000 7fdb4879d640 1 -- start start 2024-04-25T08:31:41.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.859+0000 7fdb4879d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb4015dc10 0x7fdb400ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:41.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.859+0000 7fdb4879d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb400d3e70 con 0x7fdb400d0f60 2024-04-25T08:31:41.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.859+0000 7fdb4879d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb400d4150 con 0x7fdb400ced60 2024-04-25T08:31:41.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.859+0000 7fdb4879d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4015dff0 con 0x7fdb4015dc10 2024-04-25T08:31:41.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.859+0000 7fdb46d3a640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdb400d0f60 0x7fdb400b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51828/0 (socket says 172.21.15.38:51828) 2024-04-25T08:31:41.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.859+0000 7fdb46539640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb4015dc10 0x7fdb400ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:41.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.859+0000 7fdb46d3a640 1 -- 172.21.15.38:0/2336234901 learned_addr learned my addr 172.21.15.38:0/2336234901 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:41.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.859+0000 7fdb46539640 1 -- 172.21.15.38:0/2336234901 >> v1:172.21.15.107:6789/0 conn(0x7fdb400ced60 legacy=0x7fdb400d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:41.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.859+0000 7fdb46539640 1 -- 172.21.15.38:0/2336234901 >> v1:172.21.15.38:6789/0 conn(0x7fdb400d0f60 legacy=0x7fdb400b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:41.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.859+0000 7fdb46539640 1 -- 172.21.15.38:0/2336234901 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb400d4430 con 0x7fdb4015dc10 2024-04-25T08:31:41.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.860+0000 7fdb46539640 1 --2- 172.21.15.38:0/2336234901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb4015dc10 0x7fdb400ce820 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0097a0 tx=0x7fdb3c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede323e47f0a13b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:41.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.860+0000 7fdb45537640 1 -- 172.21.15.38:0/2336234901 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c037070 con 0x7fdb4015dc10 2024-04-25T08:31:41.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.860+0000 7fdb45537640 1 -- 172.21.15.38:0/2336234901 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c02ae50 con 0x7fdb4015dc10 2024-04-25T08:31:41.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.860+0000 7fdb45537640 1 -- 172.21.15.38:0/2336234901 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c032660 con 0x7fdb4015dc10 2024-04-25T08:31:41.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.861+0000 7fdb4879d640 1 -- 172.21.15.38:0/2336234901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb4015dc10 msgr2=0x7fdb400ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:41.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.861+0000 7fdb4879d640 1 --2- 172.21.15.38:0/2336234901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb4015dc10 0x7fdb400ce820 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0097a0 tx=0x7fdb3c02a140 comp rx=0 tx=0).stop 2024-04-25T08:31:41.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.861+0000 7fdb4879d640 1 -- 172.21.15.38:0/2336234901 shutdown_connections 2024-04-25T08:31:41.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.861+0000 7fdb4879d640 1 --2- 172.21.15.38:0/2336234901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb4015dc10 0x7fdb400ce820 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.861+0000 7fdb4879d640 1 -- 172.21.15.38:0/2336234901 >> 172.21.15.38:0/2336234901 conn(0x7fdb40066d10 msgr2=0x7fdb40067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:41.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.861+0000 7fdb4879d640 1 -- 172.21.15.38:0/2336234901 shutdown_connections 2024-04-25T08:31:41.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.861+0000 7fdb4879d640 1 -- 172.21.15.38:0/2336234901 wait complete. 2024-04-25T08:31:41.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.862+0000 7fdb4879d640 1 Processor -- start 2024-04-25T08:31:41.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.862+0000 7fdb4879d640 1 -- start start 2024-04-25T08:31:41.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb4879d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb400ced60 0x7fdb401f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb4879d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb400d0f60 0x7fdb400ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb4879d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb4015dc10 0x7fdb400ac780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb4879d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb400d4710 con 0x7fdb400d0f60 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb4879d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb400b80f0 con 0x7fdb400ced60 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb4879d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb400b8270 con 0x7fdb4015dc10 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb45d38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb400d0f60 0x7fdb400ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb45d38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb400d0f60 0x7fdb400ac240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40014/0 (socket says 172.21.15.38:40014) 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb45d38640 1 -- 172.21.15.38:0/3778082430 learned_addr learned my addr 172.21.15.38:0/3778082430 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb46539640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb400ced60 0x7fdb401f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb46d3a640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb4015dc10 0x7fdb400ac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb45d38640 1 -- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb4015dc10 msgr2=0x7fdb400ac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb45d38640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb4015dc10 0x7fdb400ac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb45d38640 1 -- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb400ced60 msgr2=0x7fdb401f6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:41.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb45d38640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb400ced60 0x7fdb401f6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:41.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb45d38640 1 -- 172.21.15.38:0/3778082430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb400ca3c0 con 0x7fdb400d0f60 2024-04-25T08:31:41.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb46539640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb400ced60 0x7fdb401f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:41.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb46d3a640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb4015dc10 0x7fdb400ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:41.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb45d38640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb400d0f60 0x7fdb400ac240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000c430 tx=0x7fdb3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:41.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb36ffd640 1 -- 172.21.15.38:0/3778082430 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30011790 con 0x7fdb400d0f60 2024-04-25T08:31:41.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb36ffd640 1 -- 172.21.15.38:0/3778082430 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb30017070 con 0x7fdb400d0f60 2024-04-25T08:31:41.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb36ffd640 1 -- 172.21.15.38:0/3778082430 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb30011b70 con 0x7fdb400d0f60 2024-04-25T08:31:41.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.863+0000 7fdb4879d640 1 -- 172.21.15.38:0/3778082430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb400c7fc0 con 0x7fdb400d0f60 2024-04-25T08:31:41.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.864+0000 7fdb4879d640 1 -- 172.21.15.38:0/3778082430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb400c8580 con 0x7fdb400d0f60 2024-04-25T08:31:41.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.864+0000 7fdb4879d640 1 -- 172.21.15.38:0/3778082430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb40068e80 con 0x7fdb400d0f60 2024-04-25T08:31:41.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.865+0000 7fdb36ffd640 1 -- 172.21.15.38:0/3778082430 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb30011d10 con 0x7fdb400d0f60 2024-04-25T08:31:41.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.865+0000 7fdb36ffd640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdb2406b4d0 0x7fdb2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:41.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.865+0000 7fdb36ffd640 1 -- 172.21.15.38:0/3778082430 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdb3008a390 con 0x7fdb400d0f60 2024-04-25T08:31:41.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.866+0000 7fdb46539640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdb2406b4d0 0x7fdb2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:41.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.866+0000 7fdb46539640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdb2406b4d0 0x7fdb2406d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c034000 tx=0x7fdb3c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:41.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:41.869+0000 7fdb36ffd640 1 -- 172.21.15.38:0/3778082430 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb30058f40 con 0x7fdb400d0f60 2024-04-25T08:31:42.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.051+0000 7fdb4879d640 1 -- 172.21.15.38:0/3778082430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb400d3290 con 0x7fdb400d0f60 2024-04-25T08:31:42.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.053+0000 7fdb36ffd640 1 -- 172.21.15.38:0/3778082430 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fdb30058840 con 0x7fdb400d0f60 2024-04-25T08:31:42.050 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:42.050 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:42.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.055+0000 7fdb4879d640 1 -- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdb2406b4d0 msgr2=0x7fdb2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.055+0000 7fdb4879d640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdb2406b4d0 0x7fdb2406d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c034000 tx=0x7fdb3c02e000 comp rx=0 tx=0).stop 2024-04-25T08:31:42.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.055+0000 7fdb4879d640 1 -- 172.21.15.38:0/3778082430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb400d0f60 msgr2=0x7fdb400ac240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.055+0000 7fdb4879d640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb400d0f60 0x7fdb400ac240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000c430 tx=0x7fdb3000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.055+0000 7fdb4879d640 1 -- 172.21.15.38:0/3778082430 shutdown_connections 2024-04-25T08:31:42.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.055+0000 7fdb4879d640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdb2406b4d0 0x7fdb2406d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.055+0000 7fdb4879d640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb4015dc10 0x7fdb400ac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.055+0000 7fdb4879d640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb400d0f60 0x7fdb400ac240 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.055+0000 7fdb4879d640 1 --2- 172.21.15.38:0/3778082430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb400ced60 0x7fdb401f6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.055+0000 7fdb4879d640 1 -- 172.21.15.38:0/3778082430 >> 172.21.15.38:0/3778082430 conn(0x7fdb40066d10 msgr2=0x7fdb400d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:42.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.055+0000 7fdb4879d640 1 -- 172.21.15.38:0/3778082430 shutdown_connections 2024-04-25T08:31:42.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.055+0000 7fdb4879d640 1 -- 172.21.15.38:0/3778082430 wait complete. 2024-04-25T08:31:42.062 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:42.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.167+0000 7f505550a640 1 Processor -- start 2024-04-25T08:31:42.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.167+0000 7f505550a640 1 -- start start 2024-04-25T08:31:42.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f505550a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f505015dc10 0x7f50500ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f505550a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f50500ced60 0x7f50500d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f505550a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50500d3e70 con 0x7f50500d3900 2024-04-25T08:31:42.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f505550a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505015dff0 con 0x7f50500ced60 2024-04-25T08:31:42.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f505550a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50500b6990 con 0x7f505015dc10 2024-04-25T08:31:42.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f5047fff640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f50500ced60 0x7f50500d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f504effd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f505015dc10 0x7f50500ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f504f7fe640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f50500d3900 0x7f50500b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:52534/0 (socket says 172.21.15.38:52534) 2024-04-25T08:31:42.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f504f7fe640 1 -- 172.21.15.38:0/1077058238 learned_addr learned my addr 172.21.15.38:0/1077058238 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:42.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f504e7fc640 1 -- 172.21.15.38:0/1077058238 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698921583 0 0) 0x7f50500d3e70 con 0x7f50500d3900 2024-04-25T08:31:42.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f504e7fc640 1 -- 172.21.15.38:0/1077058238 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f503c003520 con 0x7f50500d3900 2024-04-25T08:31:42.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f504effd640 1 -- 172.21.15.38:0/1077058238 >> v2:172.21.15.107:3300/0 conn(0x7f50500ced60 msgr2=0x7f50500d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:42.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f504effd640 1 --2- 172.21.15.38:0/1077058238 >> v2:172.21.15.107:3300/0 conn(0x7f50500ced60 0x7f50500d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f504effd640 1 -- 172.21.15.38:0/1077058238 >> v1:172.21.15.107:6789/0 conn(0x7f50500d3900 legacy=0x7f50500b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f504effd640 1 -- 172.21.15.38:0/1077058238 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500d4150 con 0x7f505015dc10 2024-04-25T08:31:42.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.168+0000 7f5047fff640 1 --2- 172.21.15.38:0/1077058238 >> v2:172.21.15.107:3300/0 conn(0x7f50500ced60 0x7f50500d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:42.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.169+0000 7f504effd640 1 --2- 172.21.15.38:0/1077058238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f505015dc10 0x7f50500ce820 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f50400097a0 tx=0x7f504002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a6fd73fff7badec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:42.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.169+0000 7f504e7fc640 1 -- 172.21.15.38:0/1077058238 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5040037070 con 0x7f505015dc10 2024-04-25T08:31:42.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.169+0000 7f504e7fc640 1 -- 172.21.15.38:0/1077058238 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504002ae50 con 0x7f505015dc10 2024-04-25T08:31:42.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.169+0000 7f504e7fc640 1 -- 172.21.15.38:0/1077058238 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5040032640 con 0x7f505015dc10 2024-04-25T08:31:42.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.170+0000 7f505550a640 1 -- 172.21.15.38:0/1077058238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f505015dc10 msgr2=0x7f50500ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.170+0000 7f505550a640 1 --2- 172.21.15.38:0/1077058238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f505015dc10 0x7f50500ce820 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f50400097a0 tx=0x7f504002a100 comp rx=0 tx=0).stop 2024-04-25T08:31:42.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.170+0000 7f505550a640 1 -- 172.21.15.38:0/1077058238 shutdown_connections 2024-04-25T08:31:42.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.170+0000 7f505550a640 1 --2- 172.21.15.38:0/1077058238 >> v2:172.21.15.107:3300/0 conn(0x7f50500ced60 0x7f50500d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.170+0000 7f505550a640 1 --2- 172.21.15.38:0/1077058238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f505015dc10 0x7f50500ce820 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.170+0000 7f505550a640 1 -- 172.21.15.38:0/1077058238 >> 172.21.15.38:0/1077058238 conn(0x7f5050066d10 msgr2=0x7f5050067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:42.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.170+0000 7f505550a640 1 -- 172.21.15.38:0/1077058238 shutdown_connections 2024-04-25T08:31:42.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.170+0000 7f505550a640 1 -- 172.21.15.38:0/1077058238 wait complete. 2024-04-25T08:31:42.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.171+0000 7f505550a640 1 Processor -- start 2024-04-25T08:31:42.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.171+0000 7f505550a640 1 -- start start 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f505550a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50500ced60 0x7f50501f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f505550a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50500d3900 0x7f50501f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f505550a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f505015dc10 0x7f50500c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f505550a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50500b6bc0 con 0x7f50500d3900 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f505550a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50500d4710 con 0x7f50500ced60 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f505550a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50500d0d90 con 0x7f505015dc10 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f5047fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50500d3900 0x7f50501f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f5047fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50500d3900 0x7f50501f44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40018/0 (socket says 172.21.15.38:40018) 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f5047fff640 1 -- 172.21.15.38:0/2986616238 learned_addr learned my addr 172.21.15.38:0/2986616238 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f504effd640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50500ced60 0x7f50501f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f504f7fe640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f505015dc10 0x7f50500c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f5047fff640 1 -- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f505015dc10 msgr2=0x7f50500c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f5047fff640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f505015dc10 0x7f50500c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f5047fff640 1 -- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50500ced60 msgr2=0x7f50501f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f5047fff640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50500ced60 0x7f50501f3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f5047fff640 1 -- 172.21.15.38:0/2986616238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500c7610 con 0x7f50500d3900 2024-04-25T08:31:42.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.172+0000 7f5047fff640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50500d3900 0x7f50501f44e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f503800ebe0 tx=0x7f503800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:42.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.173+0000 7f50477fe640 1 -- 172.21.15.38:0/2986616238 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038017070 con 0x7f50500d3900 2024-04-25T08:31:42.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.173+0000 7f50477fe640 1 -- 172.21.15.38:0/2986616238 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503800cc30 con 0x7f50500d3900 2024-04-25T08:31:42.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.173+0000 7f50477fe640 1 -- 172.21.15.38:0/2986616238 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5038013470 con 0x7f50500d3900 2024-04-25T08:31:42.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.173+0000 7f505550a640 1 -- 172.21.15.38:0/2986616238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50500c78f0 con 0x7f50500d3900 2024-04-25T08:31:42.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.173+0000 7f505550a640 1 -- 172.21.15.38:0/2986616238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50500c7eb0 con 0x7f50500d3900 2024-04-25T08:31:42.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.174+0000 7f505550a640 1 -- 172.21.15.38:0/2986616238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5050068e80 con 0x7f50500d3900 2024-04-25T08:31:42.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.174+0000 7f50477fe640 1 -- 172.21.15.38:0/2986616238 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f503801a070 con 0x7f50500d3900 2024-04-25T08:31:42.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.174+0000 7f50477fe640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f503006b500 0x7f503006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.175+0000 7f50477fe640 1 -- 172.21.15.38:0/2986616238 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f503808a780 con 0x7f50500d3900 2024-04-25T08:31:42.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.175+0000 7f504effd640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f503006b500 0x7f503006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.175+0000 7f504effd640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f503006b500 0x7f503006d9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5040034000 tx=0x7f504002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:42.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.179+0000 7f50477fe640 1 -- 172.21.15.38:0/2986616238 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5038058c70 con 0x7f50500d3900 2024-04-25T08:31:42.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.359+0000 7f505550a640 1 -- 172.21.15.38:0/2986616238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50500b42e0 con 0x7f50500d3900 2024-04-25T08:31:42.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.360+0000 7f50477fe640 1 -- 172.21.15.38:0/2986616238 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f503805cd10 con 0x7f50500d3900 2024-04-25T08:31:42.357 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:42.357 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:42.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.362+0000 7f505550a640 1 -- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f503006b500 msgr2=0x7f503006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.362+0000 7f505550a640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f503006b500 0x7f503006d9c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5040034000 tx=0x7f504002e000 comp rx=0 tx=0).stop 2024-04-25T08:31:42.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.362+0000 7f505550a640 1 -- 172.21.15.38:0/2986616238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50500d3900 msgr2=0x7f50501f44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.362+0000 7f505550a640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50500d3900 0x7f50501f44e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f503800ebe0 tx=0x7f503800c520 comp rx=0 tx=0).stop 2024-04-25T08:31:42.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.362+0000 7f505550a640 1 -- 172.21.15.38:0/2986616238 shutdown_connections 2024-04-25T08:31:42.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.362+0000 7f505550a640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f503006b500 0x7f503006d9c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.362+0000 7f505550a640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f505015dc10 0x7f50500c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.362+0000 7f505550a640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50500d3900 0x7f50501f44e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.362+0000 7f505550a640 1 --2- 172.21.15.38:0/2986616238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50500ced60 0x7f50501f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.362+0000 7f505550a640 1 -- 172.21.15.38:0/2986616238 >> 172.21.15.38:0/2986616238 conn(0x7f5050066d10 msgr2=0x7f5050160890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:42.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.362+0000 7f505550a640 1 -- 172.21.15.38:0/2986616238 shutdown_connections 2024-04-25T08:31:42.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.362+0000 7f505550a640 1 -- 172.21.15.38:0/2986616238 wait complete. 2024-04-25T08:31:42.387 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:31:42.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.490+0000 7efd866e1640 1 Processor -- start 2024-04-25T08:31:42.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.491+0000 7efd866e1640 1 -- start start 2024-04-25T08:31:42.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.491+0000 7efd866e1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efd8015dc10 0x7efd800ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.491+0000 7efd866e1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd800ced60 0x7efd800d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.491+0000 7efd866e1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd800d3e70 con 0x7efd800d3900 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.491+0000 7efd866e1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd8015dff0 con 0x7efd8015dc10 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.491+0000 7efd866e1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd800b6990 con 0x7efd800ced60 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.491+0000 7efd7ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efd8015dc10 0x7efd800ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.491+0000 7efd84c7e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7efd800d3900 0x7efd800b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51836/0 (socket says 172.21.15.38:51836) 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.491+0000 7efd84c7e640 1 -- 172.21.15.38:0/1085569073 learned_addr learned my addr 172.21.15.38:0/1085569073 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.492+0000 7efd7f7fe640 1 --2- 172.21.15.38:0/1085569073 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd800ced60 0x7efd800d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.492+0000 7efd7ffff640 1 -- 172.21.15.38:0/1085569073 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd800ced60 msgr2=0x7efd800d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.492+0000 7efd7ffff640 1 --2- 172.21.15.38:0/1085569073 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd800ced60 0x7efd800d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.492+0000 7efd7ffff640 1 -- 172.21.15.38:0/1085569073 >> v1:172.21.15.38:6789/0 conn(0x7efd800d3900 legacy=0x7efd800b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.492+0000 7efd7ffff640 1 -- 172.21.15.38:0/1085569073 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800d4150 con 0x7efd8015dc10 2024-04-25T08:31:42.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.492+0000 7efd7effd640 1 -- 172.21.15.38:0/1085569073 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970194336 0 0) 0x7efd800d3e70 con 0x7efd800d3900 2024-04-25T08:31:42.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.493+0000 7efd7ffff640 1 --2- 172.21.15.38:0/1085569073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd8015dc10 0x7efd800ce820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd700097a0 tx=0x7efd7002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b231c4e6fa27e3bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:42.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.493+0000 7efd7effd640 1 -- 172.21.15.38:0/1085569073 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70037070 con 0x7efd8015dc10 2024-04-25T08:31:42.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.493+0000 7efd7effd640 1 -- 172.21.15.38:0/1085569073 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7002ae50 con 0x7efd8015dc10 2024-04-25T08:31:42.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.493+0000 7efd7effd640 1 -- 172.21.15.38:0/1085569073 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd70032660 con 0x7efd8015dc10 2024-04-25T08:31:42.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.493+0000 7efd866e1640 1 -- 172.21.15.38:0/1085569073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd8015dc10 msgr2=0x7efd800ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.493+0000 7efd866e1640 1 --2- 172.21.15.38:0/1085569073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd8015dc10 0x7efd800ce820 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd700097a0 tx=0x7efd7002a100 comp rx=0 tx=0).stop 2024-04-25T08:31:42.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.493+0000 7efd866e1640 1 -- 172.21.15.38:0/1085569073 shutdown_connections 2024-04-25T08:31:42.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.493+0000 7efd866e1640 1 --2- 172.21.15.38:0/1085569073 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd800ced60 0x7efd800d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.493+0000 7efd866e1640 1 --2- 172.21.15.38:0/1085569073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd8015dc10 0x7efd800ce820 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.493+0000 7efd866e1640 1 -- 172.21.15.38:0/1085569073 >> 172.21.15.38:0/1085569073 conn(0x7efd80066d10 msgr2=0x7efd80067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:42.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.493+0000 7efd866e1640 1 -- 172.21.15.38:0/1085569073 shutdown_connections 2024-04-25T08:31:42.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.493+0000 7efd866e1640 1 -- 172.21.15.38:0/1085569073 wait complete. 2024-04-25T08:31:42.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.494+0000 7efd866e1640 1 Processor -- start 2024-04-25T08:31:42.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.495+0000 7efd866e1640 1 -- start start 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.495+0000 7efd866e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd800ced60 0x7efd80159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.495+0000 7efd866e1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd800d3900 0x7efd800cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.495+0000 7efd866e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd8015dc10 0x7efd800cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.495+0000 7efd866e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800b6bc0 con 0x7efd8015dc10 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.495+0000 7efd866e1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800d4710 con 0x7efd800ced60 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.495+0000 7efd866e1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd800d0d90 con 0x7efd800d3900 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.495+0000 7efd84c7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd8015dc10 0x7efd800cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.495+0000 7efd84c7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd8015dc10 0x7efd800cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40044/0 (socket says 172.21.15.38:40044) 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.495+0000 7efd84c7e640 1 -- 172.21.15.38:0/1268419887 learned_addr learned my addr 172.21.15.38:0/1268419887 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd7ffff640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd800ced60 0x7efd80159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd7f7fe640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd800d3900 0x7efd800cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd84c7e640 1 -- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd800d3900 msgr2=0x7efd800cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd84c7e640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd800d3900 0x7efd800cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd84c7e640 1 -- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd800ced60 msgr2=0x7efd80159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd84c7e640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd800ced60 0x7efd80159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd84c7e640 1 -- 172.21.15.38:0/1268419887 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800c7680 con 0x7efd8015dc10 2024-04-25T08:31:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd7ffff640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd800ced60 0x7efd80159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd84c7e640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd8015dc10 0x7efd800cc100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd7800c7d0 tx=0x7efd7800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd7cff9640 1 -- 172.21.15.38:0/1268419887 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd78018070 con 0x7efd8015dc10 2024-04-25T08:31:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd7cff9640 1 -- 172.21.15.38:0/1268419887 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7800a640 con 0x7efd8015dc10 2024-04-25T08:31:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd7cff9640 1 -- 172.21.15.38:0/1268419887 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7800aa70 con 0x7efd8015dc10 2024-04-25T08:31:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd866e1640 1 -- 172.21.15.38:0/1268419887 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd800c7960 con 0x7efd8015dc10 2024-04-25T08:31:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.496+0000 7efd866e1640 1 -- 172.21.15.38:0/1268419887 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd800c7f20 con 0x7efd8015dc10 2024-04-25T08:31:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.497+0000 7efd866e1640 1 -- 172.21.15.38:0/1268419887 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd800c8120 con 0x7efd8015dc10 2024-04-25T08:31:42.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.497+0000 7efd7cff9640 1 -- 172.21.15.38:0/1268419887 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd7801b070 con 0x7efd8015dc10 2024-04-25T08:31:42.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.498+0000 7efd7cff9640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd6c06b4d0 0x7efd6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.498+0000 7efd7cff9640 1 -- 172.21.15.38:0/1268419887 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efd78089ab0 con 0x7efd8015dc10 2024-04-25T08:31:42.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.498+0000 7efd7ffff640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd6c06b4d0 0x7efd6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.499+0000 7efd7ffff640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd6c06b4d0 0x7efd6c06d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd70034000 tx=0x7efd7002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:42.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.502+0000 7efd7cff9640 1 -- 172.21.15.38:0/1268419887 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd800c8120 con 0x7efd8015dc10 2024-04-25T08:31:42.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.718+0000 7efd866e1640 1 -- 172.21.15.38:0/1268419887 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd80068e80 con 0x7efd8015dc10 2024-04-25T08:31:42.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.718+0000 7efd7cff9640 1 -- 172.21.15.38:0/1268419887 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4937 (secure 0 0 0) 0x7efd78058680 con 0x7efd8015dc10 2024-04-25T08:31:42.715 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:42.715 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4242612806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4242612806},{"type":"v1","addr":"172.21.15.38:6835","nonce":4242612806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2154167637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":2154167637},{"type":"v1","addr":"172.21.15.107:6837","nonce":2154167637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/678452936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":678452936},{"type":"v1","addr":"172.21.15.38:6834","nonce":678452936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:31:35.665008+0000","modified":"2024-04-25T08:31:40.691473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6836/1104921666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1104921666},{"type":"v1","addr":"172.21.15.107:6836","nonce":1104921666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T08:31:42.715 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-04-25T08:31:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.720+0000 7efd866e1640 1 -- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd6c06b4d0 msgr2=0x7efd6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.720+0000 7efd866e1640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd6c06b4d0 0x7efd6c06d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efd70034000 tx=0x7efd7002e000 comp rx=0 tx=0).stop 2024-04-25T08:31:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.720+0000 7efd866e1640 1 -- 172.21.15.38:0/1268419887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd8015dc10 msgr2=0x7efd800cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.720+0000 7efd866e1640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd8015dc10 0x7efd800cc100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd7800c7d0 tx=0x7efd7800cc90 comp rx=0 tx=0).stop 2024-04-25T08:31:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.720+0000 7efd866e1640 1 -- 172.21.15.38:0/1268419887 shutdown_connections 2024-04-25T08:31:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.720+0000 7efd866e1640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd8015dc10 0x7efd800cc100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.720+0000 7efd866e1640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd6c06b4d0 0x7efd6c06d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.720+0000 7efd866e1640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd800d3900 0x7efd800cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.720+0000 7efd866e1640 1 --2- 172.21.15.38:0/1268419887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd800ced60 0x7efd80159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.720+0000 7efd866e1640 1 -- 172.21.15.38:0/1268419887 >> 172.21.15.38:0/1268419887 conn(0x7efd80066d10 msgr2=0x7efd8015eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:42.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.720+0000 7efd866e1640 1 -- 172.21.15.38:0/1268419887 shutdown_connections 2024-04-25T08:31:42.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.720+0000 7efd866e1640 1 -- 172.21.15.38:0/1268419887 wait complete. 2024-04-25T08:31:42.727 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:42.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.831+0000 7f0b7ccac640 1 Processor -- start 2024-04-25T08:31:42.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.831+0000 7f0b7ccac640 1 -- start start 2024-04-25T08:31:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b7ccac640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0b7815dc10 0x7f0b780ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b7ccac640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0b780ced60 0x7f0b780d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b7ccac640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b780d3e70 con 0x7f0b780d3900 2024-04-25T08:31:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b7ccac640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7815dff0 con 0x7f0b7815dc10 2024-04-25T08:31:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b7ccac640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b780b6990 con 0x7f0b780ced60 2024-04-25T08:31:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b7659c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0b7815dc10 0x7f0b780ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b76d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0b780d3900 0x7f0b780b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51842/0 (socket says 172.21.15.38:51842) 2024-04-25T08:31:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b76d9d640 1 -- 172.21.15.38:0/3284612092 learned_addr learned my addr 172.21.15.38:0/3284612092 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b6ffff640 1 --2- 172.21.15.38:0/3284612092 >> v2:172.21.15.107:3300/0 conn(0x7f0b780ced60 0x7f0b780d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b7659c640 1 -- 172.21.15.38:0/3284612092 >> v2:172.21.15.107:3300/0 conn(0x7f0b780ced60 msgr2=0x7f0b780d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b7659c640 1 --2- 172.21.15.38:0/3284612092 >> v2:172.21.15.107:3300/0 conn(0x7f0b780ced60 0x7f0b780d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b7659c640 1 -- 172.21.15.38:0/3284612092 >> v1:172.21.15.38:6789/0 conn(0x7f0b780d3900 legacy=0x7f0b780b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b7659c640 1 -- 172.21.15.38:0/3284612092 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b780d4150 con 0x7f0b7815dc10 2024-04-25T08:31:42.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b6ffff640 1 --2- 172.21.15.38:0/3284612092 >> v2:172.21.15.107:3300/0 conn(0x7f0b780ced60 0x7f0b780d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:42.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.832+0000 7f0b75d9b640 1 -- 172.21.15.38:0/3284612092 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874568990 0 0) 0x7f0b780d3e70 con 0x7f0b780d3900 2024-04-25T08:31:42.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.833+0000 7f0b7659c640 1 --2- 172.21.15.38:0/3284612092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7815dc10 0x7f0b780ce820 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b780b9600 tx=0x7f0b6802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841c9e7ad560d1f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:42.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.833+0000 7f0b75d9b640 1 -- 172.21.15.38:0/3284612092 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68037070 con 0x7f0b7815dc10 2024-04-25T08:31:42.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.833+0000 7f0b75d9b640 1 -- 172.21.15.38:0/3284612092 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68032440 con 0x7f0b7815dc10 2024-04-25T08:31:42.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.833+0000 7f0b75d9b640 1 -- 172.21.15.38:0/3284612092 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6802ae60 con 0x7f0b7815dc10 2024-04-25T08:31:42.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.833+0000 7f0b7ccac640 1 -- 172.21.15.38:0/3284612092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7815dc10 msgr2=0x7f0b780ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.833+0000 7f0b7ccac640 1 --2- 172.21.15.38:0/3284612092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7815dc10 0x7f0b780ce820 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b780b9600 tx=0x7f0b6802a110 comp rx=0 tx=0).stop 2024-04-25T08:31:42.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.833+0000 7f0b7ccac640 1 -- 172.21.15.38:0/3284612092 shutdown_connections 2024-04-25T08:31:42.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.833+0000 7f0b7ccac640 1 --2- 172.21.15.38:0/3284612092 >> v2:172.21.15.107:3300/0 conn(0x7f0b780ced60 0x7f0b780d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.833+0000 7f0b7ccac640 1 --2- 172.21.15.38:0/3284612092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7815dc10 0x7f0b780ce820 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.834+0000 7f0b7ccac640 1 -- 172.21.15.38:0/3284612092 >> 172.21.15.38:0/3284612092 conn(0x7f0b78066d10 msgr2=0x7f0b78067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:42.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.834+0000 7f0b7ccac640 1 -- 172.21.15.38:0/3284612092 shutdown_connections 2024-04-25T08:31:42.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.834+0000 7f0b7ccac640 1 -- 172.21.15.38:0/3284612092 wait complete. 2024-04-25T08:31:42.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.835+0000 7f0b7ccac640 1 Processor -- start 2024-04-25T08:31:42.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.835+0000 7f0b7ccac640 1 -- start start 2024-04-25T08:31:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b7ccac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b780ced60 0x7f0b7815c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b7ccac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b780d3900 0x7f0b780aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b7ccac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7815dc10 0x7f0b780c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b7ccac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780b6bc0 con 0x7f0b780d3900 2024-04-25T08:31:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b7ccac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780d4710 con 0x7f0b780ced60 2024-04-25T08:31:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b7ccac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780d0d90 con 0x7f0b7815dc10 2024-04-25T08:31:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b6ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b780d3900 0x7f0b780aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b6ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b780d3900 0x7f0b780aad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40064/0 (socket says 172.21.15.38:40064) 2024-04-25T08:31:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b6ffff640 1 -- 172.21.15.38:0/1609976184 learned_addr learned my addr 172.21.15.38:0/1609976184 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b7659c640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b780ced60 0x7f0b7815c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b76d9d640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7815dc10 0x7f0b780c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b6ffff640 1 -- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7815dc10 msgr2=0x7f0b780c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b6ffff640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7815dc10 0x7f0b780c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b6ffff640 1 -- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b780ced60 msgr2=0x7f0b7815c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:42.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b6ffff640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b780ced60 0x7f0b7815c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:42.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b6ffff640 1 -- 172.21.15.38:0/1609976184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b780c7430 con 0x7f0b780d3900 2024-04-25T08:31:42.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b76d9d640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7815dc10 0x7f0b780c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:42.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b6ffff640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b780d3900 0x7f0b780aad00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000c7f0 tx=0x7f0b6000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:42.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b6effd640 1 -- 172.21.15.38:0/1609976184 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60015070 con 0x7f0b780d3900 2024-04-25T08:31:42.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b6effd640 1 -- 172.21.15.38:0/1609976184 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b60007a30 con 0x7f0b780d3900 2024-04-25T08:31:42.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b7ccac640 1 -- 172.21.15.38:0/1609976184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b780c7710 con 0x7f0b780d3900 2024-04-25T08:31:42.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b6effd640 1 -- 172.21.15.38:0/1609976184 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60007e00 con 0x7f0b780d3900 2024-04-25T08:31:42.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.836+0000 7f0b7ccac640 1 -- 172.21.15.38:0/1609976184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b780c7cd0 con 0x7f0b780d3900 2024-04-25T08:31:42.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.837+0000 7f0b7ccac640 1 -- 172.21.15.38:0/1609976184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b78068e80 con 0x7f0b780d3900 2024-04-25T08:31:42.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.838+0000 7f0b6effd640 1 -- 172.21.15.38:0/1609976184 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b60018070 con 0x7f0b780d3900 2024-04-25T08:31:42.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.838+0000 7f0b6effd640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b5806b4d0 0x7f0b5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:42.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.838+0000 7f0b6effd640 1 -- 172.21.15.38:0/1609976184 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0b6008a4f0 con 0x7f0b780d3900 2024-04-25T08:31:42.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.838+0000 7f0b7659c640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b5806b4d0 0x7f0b5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:42.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.839+0000 7f0b7659c640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b5806b4d0 0x7f0b5806d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0b68034000 tx=0x7f0b6802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:42.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:42.842+0000 7f0b6effd640 1 -- 172.21.15.38:0/1609976184 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b60059030 con 0x7f0b780d3900 2024-04-25T08:31:43.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.025+0000 7f0b7ccac640 1 -- 172.21.15.38:0/1609976184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b780ca090 con 0x7f0b780d3900 2024-04-25T08:31:43.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.027+0000 7f0b6effd640 1 -- 172.21.15.38:0/1609976184 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f0b60058930 con 0x7f0b780d3900 2024-04-25T08:31:43.023 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:43.024 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:43.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.029+0000 7f0b7ccac640 1 -- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b5806b4d0 msgr2=0x7f0b5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.029+0000 7f0b7ccac640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b5806b4d0 0x7f0b5806d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0b68034000 tx=0x7f0b6802e000 comp rx=0 tx=0).stop 2024-04-25T08:31:43.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.029+0000 7f0b7ccac640 1 -- 172.21.15.38:0/1609976184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b780d3900 msgr2=0x7f0b780aad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.029+0000 7f0b7ccac640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b780d3900 0x7f0b780aad00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000c7f0 tx=0x7f0b6000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.029+0000 7f0b7ccac640 1 -- 172.21.15.38:0/1609976184 shutdown_connections 2024-04-25T08:31:43.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.029+0000 7f0b7ccac640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b5806b4d0 0x7f0b5806d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.029+0000 7f0b7ccac640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7815dc10 0x7f0b780c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.029+0000 7f0b7ccac640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b780d3900 0x7f0b780aad00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.029+0000 7f0b7ccac640 1 --2- 172.21.15.38:0/1609976184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b780ced60 0x7f0b7815c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.029+0000 7f0b7ccac640 1 -- 172.21.15.38:0/1609976184 >> 172.21.15.38:0/1609976184 conn(0x7f0b78066d10 msgr2=0x7f0b780b6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:43.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.029+0000 7f0b7ccac640 1 -- 172.21.15.38:0/1609976184 shutdown_connections 2024-04-25T08:31:43.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.029+0000 7f0b7ccac640 1 -- 172.21.15.38:0/1609976184 wait complete. 2024-04-25T08:31:43.439 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:31:35.665008+0000', 'modified': '2024-04-25T08:31:40.691473+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.107:6836/1104921666', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 1104921666}, {'type': 'v1', 'addr': '172.21.15.107:6836', 'nonce': 1104921666}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-04-25T08:31:43.439 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:31:43.439 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T08:31:43.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.547+0000 7f636ccfc640 1 Processor -- start 2024-04-25T08:31:43.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.547+0000 7f636ccfc640 1 -- start start 2024-04-25T08:31:43.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.548+0000 7f636ccfc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6368071690 0x7f636815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.548+0000 7f636ccfc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63680b2430 con 0x7f636815a600 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.548+0000 7f636ccfc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6368071d70 con 0x7f63680ceba0 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.548+0000 7f636ccfc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63680b26d0 con 0x7f6368071690 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.548+0000 7f636659c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f636815a600 0x7f6368070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51858/0 (socket says 172.21.15.38:51858) 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.548+0000 7f636659c640 1 -- 172.21.15.38:0/2213565909 learned_addr learned my addr 172.21.15.38:0/2213565909 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.548+0000 7f6365d9b640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6368071690 0x7f636815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.548+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903194913 0 0) 0x7f63680b2430 con 0x7f636815a600 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6350003520 con 0x7f636815a600 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856741180 0 0) 0x7f6368071d70 con 0x7f63680ceba0 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63680b2430 con 0x7f63680ceba0 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2425669031 0 0) 0x7f6350003520 con 0x7f636815a600 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6368071690 msgr2=0x7f636815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6368071690 0x7f636815ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 >> v1:172.21.15.107:6789/0 conn(0x7f63680ceba0 legacy=0x7f636806d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63680b29b0 con 0x7f636815a600 2024-04-25T08:31:43.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f6360001670 con 0x7f636815a600 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 >> v1:172.21.15.38:6789/0 conn(0x7f636815a600 legacy=0x7f6368070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6350003b70 0x7f6350003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6350004800 0x7f6350004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63500051c0 0x7f635000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680b26d0 con 0x7f6350004800 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635000c010 con 0x7f63500051c0 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635000c210 con 0x7f6350003b70 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f6365d9b640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6368071690 0x7f636815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636659c640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6350003b70 0x7f6350003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f6365d9b640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6350004800 0x7f6350004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f6366d9d640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63500051c0 0x7f635000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f6365d9b640 1 -- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6350003b70 msgr2=0x7f6350003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f6365d9b640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6350003b70 0x7f6350003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f6365d9b640 1 -- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63500051c0 msgr2=0x7f635000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:43.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f6365d9b640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63500051c0 0x7f635000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f6365d9b640 1 -- 172.21.15.38:0/2213565909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6350003520 con 0x7f6350004800 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.549+0000 7f636659c640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6350003b70 0x7f6350003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.550+0000 7f6365d9b640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6350004800 0x7f6350004c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f63680b21f0 tx=0x7f6350005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3b8b0feb60c8db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.550+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c036070 con 0x7f6350004800 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.550+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c004030 con 0x7f6350004800 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.550+0000 7f636559a640 1 -- 172.21.15.38:0/2213565909 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c0097d0 con 0x7f6350004800 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.551+0000 7f636ccfc640 1 -- 172.21.15.38:0/2213565909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6350004800 msgr2=0x7f6350004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.551+0000 7f636ccfc640 1 --2- 172.21.15.38:0/2213565909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6350004800 0x7f6350004c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f63680b21f0 tx=0x7f6350005830 comp rx=0 tx=0).stop 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.551+0000 7f6366d9d640 1 -- 172.21.15.38:0/2213565909 reap_dead start 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.551+0000 7f636ccfc640 1 -- 172.21.15.38:0/2213565909 shutdown_connections 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.551+0000 7f636ccfc640 1 -- 172.21.15.38:0/2213565909 >> 172.21.15.38:0/2213565909 conn(0x7f636806abe0 msgr2=0x7f636806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.551+0000 7f636ccfc640 1 -- 172.21.15.38:0/2213565909 shutdown_connections 2024-04-25T08:31:43.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.551+0000 7f636ccfc640 1 -- 172.21.15.38:0/2213565909 wait complete. 2024-04-25T08:31:43.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.552+0000 7f636ccfc640 1 Processor -- start 2024-04-25T08:31:43.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.552+0000 7f636ccfc640 1 -- start start 2024-04-25T08:31:43.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f636ccfc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6350004800 0x7f636815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f636ccfc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63681e59c0 0x7f63680cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f636ccfc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63681421b0 0x7f6368142630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f636ccfc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63680b3600 con 0x7f63681e59c0 2024-04-25T08:31:43.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f636ccfc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636815a9e0 con 0x7f63681421b0 2024-04-25T08:31:43.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f636ccfc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636815c710 con 0x7f6350004800 2024-04-25T08:31:43.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f6365d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63681e59c0 0x7f63680cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f6365d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63681e59c0 0x7f63680cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40084/0 (socket says 172.21.15.38:40084) 2024-04-25T08:31:43.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f6365d9b640 1 -- 172.21.15.38:0/625802709 learned_addr learned my addr 172.21.15.38:0/625802709 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f6365d9b640 1 -- 172.21.15.38:0/625802709 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6350004800 msgr2=0x7f636815c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f6365d9b640 1 --2- 172.21.15.38:0/625802709 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6350004800 0x7f636815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f6365d9b640 1 -- 172.21.15.38:0/625802709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63681421b0 msgr2=0x7f6368142630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f6365d9b640 1 --2- 172.21.15.38:0/625802709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63681421b0 0x7f6368142630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f6365d9b640 1 -- 172.21.15.38:0/625802709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636815c9f0 con 0x7f63681e59c0 2024-04-25T08:31:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.553+0000 7f6365d9b640 1 --2- 172.21.15.38:0/625802709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63681e59c0 0x7f63680cf650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f635c009bf0 tx=0x7f635c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.554+0000 7f6356ffd640 1 -- 172.21.15.38:0/625802709 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c040020 con 0x7f63681e59c0 2024-04-25T08:31:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.554+0000 7f6356ffd640 1 -- 172.21.15.38:0/625802709 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c003c40 con 0x7f63681e59c0 2024-04-25T08:31:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.554+0000 7f636ccfc640 1 -- 172.21.15.38:0/625802709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636815ccd0 con 0x7f63681e59c0 2024-04-25T08:31:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.554+0000 7f6356ffd640 1 -- 172.21.15.38:0/625802709 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c036040 con 0x7f63681e59c0 2024-04-25T08:31:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.554+0000 7f636ccfc640 1 -- 172.21.15.38:0/625802709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636815d290 con 0x7f63681e59c0 2024-04-25T08:31:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.555+0000 7f6356ffd640 1 -- 172.21.15.38:0/625802709 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f635c043020 con 0x7f63681e59c0 2024-04-25T08:31:43.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.556+0000 7f6356ffd640 1 --2- 172.21.15.38:0/625802709 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f634806b4d0 0x7f634806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.556+0000 7f6356ffd640 1 -- 172.21.15.38:0/625802709 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f635c0aad40 con 0x7f63681e59c0 2024-04-25T08:31:43.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.556+0000 7f636659c640 1 --2- 172.21.15.38:0/625802709 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f634806b4d0 0x7f634806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.556+0000 7f636659c640 1 --2- 172.21.15.38:0/625802709 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f634806b4d0 0x7f634806d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6360001bb0 tx=0x7f6360001b00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:43.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.556+0000 7f636fa00740 1 Processor -- start 2024-04-25T08:31:43.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.557+0000 7f636fa00740 1 -- start start 2024-04-25T08:31:43.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.557+0000 7f636fa00740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8a1230 0x55922e8a1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.557+0000 7f636fa00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8bead0 0x55922e8bf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.557+0000 7f636fa00740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e876f10 0x55922e8bd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.557+0000 7f636fa00740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e7d9a10 con 0x55922e876f10 2024-04-25T08:31:43.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.557+0000 7f636fa00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e782b30 con 0x55922e8bead0 2024-04-25T08:31:43.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.557+0000 7f636fa00740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55922e7d5c50 con 0x55922e8a1230 2024-04-25T08:31:43.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.557+0000 7f6366d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e876f10 0x55922e8bd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.557+0000 7f6366d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e876f10 0x55922e8bd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40088/0 (socket says 172.21.15.38:40088) 2024-04-25T08:31:43.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.557+0000 7f6366d9d640 1 -- 172.21.15.38:0/2723841172 learned_addr learned my addr 172.21.15.38:0/2723841172 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.558+0000 7f636659c640 1 --2- 172.21.15.38:0/2723841172 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8a1230 0x55922e8a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.558+0000 7f6365d9b640 1 --2- 172.21.15.38:0/2723841172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8bead0 0x55922e8bf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.558+0000 7f6366d9d640 1 -- 172.21.15.38:0/2723841172 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8a1230 msgr2=0x55922e8a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.558+0000 7f6366d9d640 1 --2- 172.21.15.38:0/2723841172 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8a1230 0x55922e8a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.558+0000 7f6366d9d640 1 -- 172.21.15.38:0/2723841172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8bead0 msgr2=0x55922e8bf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.558+0000 7f6366d9d640 1 --2- 172.21.15.38:0/2723841172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8bead0 0x55922e8bf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.558+0000 7f6366d9d640 1 -- 172.21.15.38:0/2723841172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55922e797e10 con 0x55922e876f10 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.558+0000 7f6366d9d640 1 --2- 172.21.15.38:0/2723841172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e876f10 0x55922e8bd190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63680af040 tx=0x7f635800f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43049e073c348e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.558+0000 7f6347fff640 1 -- 172.21.15.38:0/2723841172 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360007be0 con 0x55922e876f10 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.558+0000 7f6347fff640 1 -- 172.21.15.38:0/2723841172 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6358018070 con 0x55922e876f10 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.558+0000 7f6347fff640 1 -- 172.21.15.38:0/2723841172 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358003970 con 0x55922e876f10 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.559+0000 7f636fa00740 1 -- 172.21.15.38:0/2723841172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e876f10 msgr2=0x55922e8bd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.559+0000 7f636fa00740 1 --2- 172.21.15.38:0/2723841172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e876f10 0x55922e8bd190 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f63680af040 tx=0x7f635800f5a0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.559+0000 7f636fa00740 1 -- 172.21.15.38:0/2723841172 shutdown_connections 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.559+0000 7f636fa00740 1 --2- 172.21.15.38:0/2723841172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e876f10 0x55922e8bd190 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.559+0000 7f636fa00740 1 --2- 172.21.15.38:0/2723841172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8bead0 0x55922e8bf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.559+0000 7f636fa00740 1 --2- 172.21.15.38:0/2723841172 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8a1230 0x55922e8a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.559+0000 7f636fa00740 1 -- 172.21.15.38:0/2723841172 >> 172.21.15.38:0/2723841172 conn(0x55922e873840 msgr2=0x55922e8bc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.559+0000 7f636fa00740 1 -- 172.21.15.38:0/2723841172 shutdown_connections 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.559+0000 7f636fa00740 1 -- 172.21.15.38:0/2723841172 wait complete. 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.559+0000 7f636fa00740 1 -- start start 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.559+0000 7f636fa00740 1 Processor -- start 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.560+0000 7f636fa00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8a1230 0x55922e8a5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.560+0000 7f636fa00740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 0x55922e8708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.560+0000 7f636fa00740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e892ea0 0x55922e8b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.560+0000 7f636fa00740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e877cb0 con 0x55922e8bead0 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.560+0000 7f636fa00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e7960c0 con 0x55922e8a1230 2024-04-25T08:31:43.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.560+0000 7f636fa00740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55922e871770 con 0x55922e892ea0 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.560+0000 7f636fa00740 10 client.0 _ll_register_callbacks cb 0x55922e789ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.560+0000 7f6365d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 0x55922e8708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.560+0000 7f6365d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 0x55922e8708a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40104/0 (socket says 172.21.15.38:40104) 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.560+0000 7f6365d9b640 1 -- 172.21.15.38:0/3862641072 learned_addr learned my addr 172.21.15.38:0/3862641072 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f636659c640 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8a1230 0x55922e8a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f6366d9d640 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e892ea0 0x55922e8b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f6365d9b640 1 -- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e892ea0 msgr2=0x55922e8b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f6365d9b640 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e892ea0 0x55922e8b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f6365d9b640 1 -- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8a1230 msgr2=0x55922e8a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f6365d9b640 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8a1230 0x55922e8a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f6365d9b640 1 -- 172.21.15.38:0/3862641072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55922e4ee2d0 con 0x55922e8bead0 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f636659c640 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8a1230 0x55922e8a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f6365d9b640 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 0x55922e8708a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f635c078de0 tx=0x7f635c0790e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b38acae5eec740 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f63457fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:31:43.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f63457fa640 1 -- 172.21.15.38:0/3862641072 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c0a7060 con 0x55922e8bead0 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f63457fa640 1 -- 172.21.15.38:0/3862641072 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f635c078730 con 0x55922e8bead0 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f636fa00740 1 -- 172.21.15.38:0/3862641072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6358018070 con 0x55922e8bead0 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f63457fa640 1 -- 172.21.15.38:0/3862641072 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f635c0817f0 con 0x55922e8bead0 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f636fa00740 10 client.4395 fetch_fsmap learned FSMap version 7 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f636fa00740 1 -- 172.21.15.38:0/3862641072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55922e5b86b0 con 0x55922e8bead0 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f63457fa640 1 -- 172.21.15.38:0/3862641072 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6358018070 con 0x55922e8bead0 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.561+0000 7f63457fa640 1 -- 172.21.15.38:0/3862641072 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f635c0aa040 con 0x55922e8bead0 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.562+0000 7f636fa00740 10 client.4395 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.562+0000 7f636fa00740 10 client.4395 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.562+0000 7f636fa00740 20 client.4395 populate_metadata read hostname 'smithi038' 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.562+0000 7f636fa00740 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x55922e8a2f40 0x55922e8a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.562+0000 7f636fa00740 4 client.4395 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.562+0000 7f636fa00740 1 -- 172.21.15.38:0/3862641072 --> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55922e838920 con 0x55922e8a2f40 2024-04-25T08:31:43.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.562+0000 7f636659c640 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x55922e8a2f40 0x55922e8a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.562+0000 7f636659c640 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x55922e8a2f40 0x55922e8a3320 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c41ab0c05845d2b server_cookie=35eac654395b4c88 in_seq=0 out_seq=0 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.562+0000 7f63457fa640 10 client.4395 ms_handle_connect on v2:172.21.15.107:6834/1104921666 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f63457fa640 1 -- 172.21.15.38:0/3862641072 <== mds.0 v2:172.21.15.107:6834/1104921666 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55922e838920 con 0x55922e8a2f40 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f63457fa640 10 client.4395 handle_command_reply: tid=0 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f636fa00740 1 client.4395 shutdown 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f636fa00740 1 -- 172.21.15.38:0/3862641072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 msgr2=0x55922e8708a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f636fa00740 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 0x55922e8708a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f635c078de0 tx=0x7f635c0790e0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f636fa00740 1 -- 172.21.15.38:0/3862641072 shutdown_connections 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f636fa00740 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x55922e8a2f40 0x55922e8a3320 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f636fa00740 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e892ea0 0x55922e8b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f636fa00740 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 0x55922e8708a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f63457fa640 0 client.4395 ms_handle_reset on v2:172.21.15.107:6834/1104921666 2024-04-25T08:31:43.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f636fa00740 1 --2- 172.21.15.38:0/3862641072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8a1230 0x55922e8a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f636fa00740 1 -- 172.21.15.38:0/3862641072 >> 172.21.15.38:0/3862641072 conn(0x55922e876f10 msgr2=0x55922e8bfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:43.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f636fa00740 1 -- 172.21.15.38:0/3862641072 shutdown_connections 2024-04-25T08:31:43.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f636fa00740 1 -- 172.21.15.38:0/3862641072 wait complete. 2024-04-25T08:31:43.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.563+0000 7f636fa00740 20 client.4395 trim_cache size 0 max 16384 2024-04-25T08:31:43.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.579+0000 7f636fa00740 1 Processor -- start 2024-04-25T08:31:43.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.579+0000 7f636fa00740 1 -- start start 2024-04-25T08:31:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636fa00740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e892ea0 0x55922e8b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636fa00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8a1230 0x55922e8a3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636fa00740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8bead0 0x55922e8a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636fa00740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6368204ef0 con 0x55922e892ea0 2024-04-25T08:31:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636fa00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63682042f0 con 0x55922e8a1230 2024-04-25T08:31:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636fa00740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63682048f0 con 0x55922e8bead0 2024-04-25T08:31:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e892ea0 0x55922e8b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e892ea0 0x55922e8b6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40106/0 (socket says 172.21.15.38:40106) 2024-04-25T08:31:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636659c640 1 -- 172.21.15.38:0/3527549806 learned_addr learned my addr 172.21.15.38:0/3527549806 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636659c640 1 -- 172.21.15.38:0/3527549806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8bead0 msgr2=0x55922e8a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f6366d9d640 1 --2- 172.21.15.38:0/3527549806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8bead0 0x55922e8a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f6365d9b640 1 --2- 172.21.15.38:0/3527549806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8a1230 0x55922e8a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636659c640 1 --2- 172.21.15.38:0/3527549806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8bead0 0x55922e8a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636659c640 1 -- 172.21.15.38:0/3527549806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8a1230 msgr2=0x55922e8a3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636659c640 1 --2- 172.21.15.38:0/3527549806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8a1230 0x55922e8a3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636659c640 1 -- 172.21.15.38:0/3527549806 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55922e8b6960 con 0x55922e892ea0 2024-04-25T08:31:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f6366d9d640 1 --2- 172.21.15.38:0/3527549806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8bead0 0x55922e8a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f6365d9b640 1 --2- 172.21.15.38:0/3527549806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8a1230 0x55922e8a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f636659c640 1 --2- 172.21.15.38:0/3527549806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e892ea0 0x55922e8b6420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6360028d20 tx=0x7f6360009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b284013026c6ac2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f63477fe640 1 -- 172.21.15.38:0/3527549806 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63600240d0 con 0x55922e892ea0 2024-04-25T08:31:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f63477fe640 1 -- 172.21.15.38:0/3527549806 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6360024be0 con 0x55922e892ea0 2024-04-25T08:31:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.580+0000 7f63477fe640 1 -- 172.21.15.38:0/3527549806 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6360024270 con 0x55922e892ea0 2024-04-25T08:31:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.581+0000 7f636fa00740 1 -- 172.21.15.38:0/3527549806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e892ea0 msgr2=0x55922e8b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.581+0000 7f636fa00740 1 --2- 172.21.15.38:0/3527549806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e892ea0 0x55922e8b6420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6360028d20 tx=0x7f6360009030 comp rx=0 tx=0).stop 2024-04-25T08:31:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.581+0000 7f636fa00740 1 -- 172.21.15.38:0/3527549806 shutdown_connections 2024-04-25T08:31:43.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.581+0000 7f636fa00740 1 --2- 172.21.15.38:0/3527549806 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8bead0 0x55922e8a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.581+0000 7f636fa00740 1 --2- 172.21.15.38:0/3527549806 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e8a1230 0x55922e8a3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.581+0000 7f636fa00740 1 --2- 172.21.15.38:0/3527549806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e892ea0 0x55922e8b6420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.581+0000 7f636fa00740 1 -- 172.21.15.38:0/3527549806 >> 172.21.15.38:0/3527549806 conn(0x55922e876f10 msgr2=0x55922e8b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:43.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.581+0000 7f636fa00740 1 -- 172.21.15.38:0/3527549806 shutdown_connections 2024-04-25T08:31:43.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.581+0000 7f636fa00740 1 -- 172.21.15.38:0/3527549806 wait complete. 2024-04-25T08:31:43.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.582+0000 7f636fa00740 1 -- start start 2024-04-25T08:31:43.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.582+0000 7f636fa00740 1 Processor -- start 2024-04-25T08:31:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.582+0000 7f636fa00740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e892ea0 0x55922e8d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f636fa00740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8a1230 0x55922e926ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f636fa00740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 0x55922e9271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f636fa00740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e877cb0 con 0x55922e8bead0 2024-04-25T08:31:43.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f636fa00740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922e7960c0 con 0x55922e892ea0 2024-04-25T08:31:43.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f636fa00740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55922e8b3830 con 0x55922e8a1230 2024-04-25T08:31:43.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f636fa00740 10 client.0 _ll_register_callbacks cb 0x55922e871c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:31:43.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f6366d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 0x55922e9271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f6366d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 0x55922e9271e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40122/0 (socket says 172.21.15.38:40122) 2024-04-25T08:31:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f6366d9d640 1 -- 172.21.15.38:0/3962933105 learned_addr learned my addr 172.21.15.38:0/3962933105 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f6365d9b640 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8a1230 0x55922e926ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f636659c640 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e892ea0 0x55922e8d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f6366d9d640 1 -- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8a1230 msgr2=0x55922e926ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:43.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f6366d9d640 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8a1230 0x55922e926ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f6366d9d640 1 -- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e892ea0 msgr2=0x55922e8d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f6366d9d640 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e892ea0 0x55922e8d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f6366d9d640 1 -- 172.21.15.38:0/3962933105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55922e8bff50 con 0x55922e8bead0 2024-04-25T08:31:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f636659c640 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e892ea0 0x55922e8d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f6365d9b640 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8a1230 0x55922e926ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f6366d9d640 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 0x55922e9271e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6358014000 tx=0x7f63580017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00cc6826d48eaff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f63457fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:31:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f63457fa640 1 -- 172.21.15.38:0/3962933105 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358001c10 con 0x55922e8bead0 2024-04-25T08:31:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f63457fa640 1 -- 172.21.15.38:0/3962933105 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f63580052e0 con 0x55922e8bead0 2024-04-25T08:31:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f636fa00740 1 -- 172.21.15.38:0/3962933105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6360024be0 con 0x55922e8bead0 2024-04-25T08:31:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f63457fa640 1 -- 172.21.15.38:0/3962933105 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f635802e050 con 0x55922e8bead0 2024-04-25T08:31:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f636fa00740 10 client.4401 fetch_fsmap learned FSMap version 7 2024-04-25T08:31:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.583+0000 7f636fa00740 1 -- 172.21.15.38:0/3962933105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55922e927870 con 0x55922e8bead0 2024-04-25T08:31:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f63457fa640 1 -- 172.21.15.38:0/3962933105 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6360024be0 con 0x55922e8bead0 2024-04-25T08:31:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f63457fa640 1 -- 172.21.15.38:0/3962933105 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f6358031020 con 0x55922e8bead0 2024-04-25T08:31:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f636fa00740 10 client.4401 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T08:31:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f636fa00740 10 client.4401 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:31:43.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f636fa00740 20 client.4401 populate_metadata read hostname 'smithi038' 2024-04-25T08:31:43.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f636fa00740 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x55922e8bf820 0x55922e8b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f636fa00740 4 client.4401 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:31:43.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f636fa00740 1 -- 172.21.15.38:0/3962933105 --> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55922e7e6f50 con 0x55922e8bf820 2024-04-25T08:31:43.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f636659c640 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x55922e8bf820 0x55922e8b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f63457fa640 10 client.4401 ms_handle_connect on v2:172.21.15.107:6834/1104921666 2024-04-25T08:31:43.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f636659c640 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x55922e8bf820 0x55922e8b19d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65c31fc2c3a96659 server_cookie=b11f83aad37f534e in_seq=0 out_seq=0 2024-04-25T08:31:43.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f63457fa640 1 -- 172.21.15.38:0/3962933105 <== mds.0 v2:172.21.15.107:6834/1104921666 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x55922e7e6f50 con 0x55922e8bf820 2024-04-25T08:31:43.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f63457fa640 10 client.4401 handle_command_reply: tid=0 2024-04-25T08:31:43.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f636fa00740 1 client.4401 shutdown 2024-04-25T08:31:43.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f636fa00740 1 -- 172.21.15.38:0/3962933105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 msgr2=0x55922e9271e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.584+0000 7f636fa00740 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 0x55922e9271e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6358014000 tx=0x7f63580017e0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636fa00740 1 -- 172.21.15.38:0/3962933105 shutdown_connections 2024-04-25T08:31:43.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636fa00740 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x55922e8bf820 0x55922e8b19d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636fa00740 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55922e8bead0 0x55922e9271e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636fa00740 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55922e8a1230 0x55922e926ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636fa00740 1 --2- 172.21.15.38:0/3962933105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55922e892ea0 0x55922e8d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f63457fa640 0 client.4401 ms_handle_reset on v2:172.21.15.107:6834/1104921666 2024-04-25T08:31:43.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636fa00740 1 -- 172.21.15.38:0/3962933105 >> 172.21.15.38:0/3962933105 conn(0x55922e876f10 msgr2=0x55922e877380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:43.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636fa00740 1 -- 172.21.15.38:0/3962933105 shutdown_connections 2024-04-25T08:31:43.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636fa00740 1 -- 172.21.15.38:0/3962933105 wait complete. 2024-04-25T08:31:43.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636fa00740 20 client.4401 trim_cache size 0 max 16384 2024-04-25T08:31:43.600 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:31:43.600 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:31:43.600 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:31:43.600 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4294, 2024-04-25T08:31:43.601 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:31:43.601 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:31:43.601 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:31:43.601 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 7.286303921, 2024-04-25T08:31:43.601 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 7, 2024-04-25T08:31:43.601 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 24, 2024-04-25T08:31:43.601 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T08:31:43.601 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 14.698290805999999 2024-04-25T08:31:43.601 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:31:43.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636ccfc640 1 -- 172.21.15.38:0/625802709 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f634806b4d0 msgr2=0x7f634806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636ccfc640 1 --2- 172.21.15.38:0/625802709 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f634806b4d0 0x7f634806d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6360001bb0 tx=0x7f6360001b00 comp rx=0 tx=0).stop 2024-04-25T08:31:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636ccfc640 1 -- 172.21.15.38:0/625802709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63681e59c0 msgr2=0x7f63680cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636ccfc640 1 --2- 172.21.15.38:0/625802709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63681e59c0 0x7f63680cf650 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f635c009bf0 tx=0x7f635c004730 comp rx=0 tx=0).stop 2024-04-25T08:31:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636ccfc640 1 -- 172.21.15.38:0/625802709 shutdown_connections 2024-04-25T08:31:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636ccfc640 1 --2- 172.21.15.38:0/625802709 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63681421b0 0x7f6368142630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636ccfc640 1 --2- 172.21.15.38:0/625802709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63681e59c0 0x7f63680cf650 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636ccfc640 1 --2- 172.21.15.38:0/625802709 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f634806b4d0 0x7f634806d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636ccfc640 1 --2- 172.21.15.38:0/625802709 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6350004800 0x7f636815c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636ccfc640 1 -- 172.21.15.38:0/625802709 >> 172.21.15.38:0/625802709 conn(0x7f636806abe0 msgr2=0x7f63681598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636ccfc640 1 -- 172.21.15.38:0/625802709 shutdown_connections 2024-04-25T08:31:43.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.585+0000 7f636ccfc640 1 -- 172.21.15.38:0/625802709 wait complete. 2024-04-25T08:31:43.603 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:31:43.604 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:31:43.604 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:31:43.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.707+0000 7f3bb3b5a640 1 Processor -- start 2024-04-25T08:31:43.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.707+0000 7f3bb3b5a640 1 -- start start 2024-04-25T08:31:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.708+0000 7f3bb3b5a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3bac0d3900 0x7f3bac0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.708+0000 7f3bb3b5a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bac0d0d90 0x7f3bac0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.708+0000 7f3bb3b5a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bac0d3e70 con 0x7f3bac15dc10 2024-04-25T08:31:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.708+0000 7f3bb3b5a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bac15dff0 con 0x7f3bac0d3900 2024-04-25T08:31:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.708+0000 7f3bb3b5a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac0b6990 con 0x7f3bac0d0d90 2024-04-25T08:31:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.708+0000 7f3bb10f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3bac0d3900 0x7f3bac0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.708+0000 7f3bb20f7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bac0d0d90 0x7f3bac0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.708+0000 7f3bb20f7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bac0d0d90 0x7f3bac0b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:33500/0 (socket says 172.21.15.38:33500) 2024-04-25T08:31:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.708+0000 7f3bb10f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3bac0d3900 0x7f3bac0d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:51136/0 (socket says 172.21.15.38:51136) 2024-04-25T08:31:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.708+0000 7f3bb20f7640 1 -- 172.21.15.38:0/2104014173 learned_addr learned my addr 172.21.15.38:0/2104014173 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:43.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.708+0000 7f3bb18f6640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f3bac15dc10 0x7f3bac0ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:52544/0 (socket says 172.21.15.38:52544) 2024-04-25T08:31:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.709+0000 7f3bb10f5640 1 -- 172.21.15.38:0/2104014173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bac0d0d90 msgr2=0x7f3bac0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.709+0000 7f3bb10f5640 1 --2- 172.21.15.38:0/2104014173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bac0d0d90 0x7f3bac0b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.709+0000 7f3bb10f5640 1 -- 172.21.15.38:0/2104014173 >> v1:172.21.15.107:6789/0 conn(0x7f3bac15dc10 legacy=0x7f3bac0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.709+0000 7f3bb10f5640 1 -- 172.21.15.38:0/2104014173 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac0d4150 con 0x7f3bac0d3900 2024-04-25T08:31:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.709+0000 7f3bb20f7640 1 --2- 172.21.15.38:0/2104014173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bac0d0d90 0x7f3bac0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.709+0000 7f3bb10f5640 1 --2- 172.21.15.38:0/2104014173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bac0d3900 0x7f3bac0d0850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00097a0 tx=0x7f3ba002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=216ca7d7c34c7060 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.710+0000 7f3bb08f4640 1 -- 172.21.15.38:0/2104014173 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0037070 con 0x7f3bac0d3900 2024-04-25T08:31:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.710+0000 7f3bb08f4640 1 -- 172.21.15.38:0/2104014173 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba002ae50 con 0x7f3bac0d3900 2024-04-25T08:31:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.710+0000 7f3bb08f4640 1 -- 172.21.15.38:0/2104014173 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0032660 con 0x7f3bac0d3900 2024-04-25T08:31:43.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.710+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/2104014173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bac0d3900 msgr2=0x7f3bac0d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.710+0000 7f3bb3b5a640 1 --2- 172.21.15.38:0/2104014173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bac0d3900 0x7f3bac0d0850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00097a0 tx=0x7f3ba002a100 comp rx=0 tx=0).stop 2024-04-25T08:31:43.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.710+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/2104014173 shutdown_connections 2024-04-25T08:31:43.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.710+0000 7f3bb3b5a640 1 --2- 172.21.15.38:0/2104014173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bac0d0d90 0x7f3bac0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.710+0000 7f3bb3b5a640 1 --2- 172.21.15.38:0/2104014173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bac0d3900 0x7f3bac0d0850 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.710+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/2104014173 >> 172.21.15.38:0/2104014173 conn(0x7f3bac066d10 msgr2=0x7f3bac067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:43.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.710+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/2104014173 shutdown_connections 2024-04-25T08:31:43.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.710+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/2104014173 wait complete. 2024-04-25T08:31:43.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.711+0000 7f3bb3b5a640 1 Processor -- start 2024-04-25T08:31:43.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.711+0000 7f3bb3b5a640 1 -- start start 2024-04-25T08:31:43.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.712+0000 7f3bb3b5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bac0d0d90 0x7f3bac152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.712+0000 7f3bb3b5a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bac0d3900 0x7f3bac0b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.712+0000 7f3bb3b5a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bac15dc10 0x7f3bac0cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.712+0000 7f3bb3b5a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac0b6bc0 con 0x7f3bac0d0d90 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.712+0000 7f3bb3b5a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac0d4710 con 0x7f3bac0d3900 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.712+0000 7f3bb3b5a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac0cef30 con 0x7f3bac15dc10 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.712+0000 7f3bb18f6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bac0d0d90 0x7f3bac152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.712+0000 7f3bb18f6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bac0d0d90 0x7f3bac152840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40132/0 (socket says 172.21.15.38:40132) 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.712+0000 7f3bb18f6640 1 -- 172.21.15.38:0/3824502674 learned_addr learned my addr 172.21.15.38:0/3824502674 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3bb10f5640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bac0d3900 0x7f3bac0b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3bb20f7640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bac15dc10 0x7f3bac0cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3bb18f6640 1 -- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bac15dc10 msgr2=0x7f3bac0cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3bb18f6640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bac15dc10 0x7f3bac0cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3bb18f6640 1 -- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bac0d3900 msgr2=0x7f3bac0b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3bb18f6640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bac0d3900 0x7f3bac0b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3bb18f6640 1 -- 172.21.15.38:0/3824502674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac0caa70 con 0x7f3bac0d0d90 2024-04-25T08:31:43.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3bb10f5640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bac0d3900 0x7f3bac0b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:43.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3bb18f6640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bac0d0d90 0x7f3bac152840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400c4d0 tx=0x7f3ba400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:43.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3baa7fc640 1 -- 172.21.15.38:0/3824502674 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4018070 con 0x7f3bac0d0d90 2024-04-25T08:31:43.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3baa7fc640 1 -- 172.21.15.38:0/3824502674 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba4007480 con 0x7f3bac0d0d90 2024-04-25T08:31:43.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3baa7fc640 1 -- 172.21.15.38:0/3824502674 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40049e0 con 0x7f3bac0d0d90 2024-04-25T08:31:43.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/3824502674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bac152e90 con 0x7f3bac0d0d90 2024-04-25T08:31:43.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.713+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/3824502674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bac0cafc0 con 0x7f3bac0d0d90 2024-04-25T08:31:43.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.714+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/3824502674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bac068e80 con 0x7f3bac0d0d90 2024-04-25T08:31:43.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.714+0000 7f3baa7fc640 1 -- 172.21.15.38:0/3824502674 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ba401b070 con 0x7f3bac0d0d90 2024-04-25T08:31:43.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.715+0000 7f3baa7fc640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b9806b4d0 0x7f3b9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:43.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.715+0000 7f3baa7fc640 1 -- 172.21.15.38:0/3824502674 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3ba40897f0 con 0x7f3bac0d0d90 2024-04-25T08:31:43.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.715+0000 7f3bb10f5640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b9806b4d0 0x7f3b9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:43.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.716+0000 7f3bb10f5640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b9806b4d0 0x7f3b9806d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0034000 tx=0x7f3ba002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:43.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.719+0000 7f3baa7fc640 1 -- 172.21.15.38:0/3824502674 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ba4007670 con 0x7f3bac0d0d90 2024-04-25T08:31:43.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.936+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/3824502674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3bac0b4a10 con 0x7f3bac0d0d90 2024-04-25T08:31:43.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.937+0000 7f3baa7fc640 1 -- 172.21.15.38:0/3824502674 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3ba40583c0 con 0x7f3bac0d0d90 2024-04-25T08:31:43.934 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:31:43.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.939+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b9806b4d0 msgr2=0x7f3b9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.939+0000 7f3bb3b5a640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b9806b4d0 0x7f3b9806d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0034000 tx=0x7f3ba002e000 comp rx=0 tx=0).stop 2024-04-25T08:31:43.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.939+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/3824502674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bac0d0d90 msgr2=0x7f3bac152840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:43.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.939+0000 7f3bb3b5a640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bac0d0d90 0x7f3bac152840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400c4d0 tx=0x7f3ba400c990 comp rx=0 tx=0).stop 2024-04-25T08:31:43.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.939+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/3824502674 shutdown_connections 2024-04-25T08:31:43.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.939+0000 7f3bb3b5a640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b9806b4d0 0x7f3b9806d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.939+0000 7f3bb3b5a640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3bac15dc10 0x7f3bac0cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.939+0000 7f3bb3b5a640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bac0d3900 0x7f3bac0b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.939+0000 7f3bb3b5a640 1 --2- 172.21.15.38:0/3824502674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3bac0d0d90 0x7f3bac152840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:43.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.939+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/3824502674 >> 172.21.15.38:0/3824502674 conn(0x7f3bac066d10 msgr2=0x7f3bac15e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:43.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.939+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/3824502674 shutdown_connections 2024-04-25T08:31:43.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:43.939+0000 7f3bb3b5a640 1 -- 172.21.15.38:0/3824502674 wait complete. 2024-04-25T08:31:43.945 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:31:43.946 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:31:43.946 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:31:43.946 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T08:31:43.946 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:31:43.946 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:31:43.946 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:31:43.946 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:31:43.946 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T08:31:43.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:43.946 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:31:43.962 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:31:43.962 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:31:43.962 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:31:43.962 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:31:43.962 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:31:43.962 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:31:43.962 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:31:43.962 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:31:43.962 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:31:43.962 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:31:43.963 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:31:43.963 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:31:43.963 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:31:43.963 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:31:43.963 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:31:43.963 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:31:43.963 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42734sec preferred_lft 42734sec 2024-04-25T08:31:43.963 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:31:43.963 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:31:43.963 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:31:43.963 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:31:43.963 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:31:43.964 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:31:43.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:43.964 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:31:43.964 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:31:43.964 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:31:43.964 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-04-25T08:31:43.964 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-04-25T08:31:43.964 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:31:43.964 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:31:44.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:44.111 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:31:44.135 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:31:44.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:44.137 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:31:44.202 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:31:44.202 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:31:44.202 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:31:44.202 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:31:44.202 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:31:44.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:44.203 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:31:44.203 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:31:44.203 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:31:44.203 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:31:44.203 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:31:44.203 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:31:44.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:44.449 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:31:44.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:44.464 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-04-25T08:31:44.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:44.519 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:31:44.520 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:31:44.520 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:31:44.520 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T08:31:44.520 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:31:44.597 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T08:31:44.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:44.597 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:31:44.597 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:31:44.598 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T08:31:44.598 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:31:44.598 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T08:31:44.598 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T08:31:44.598 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T08:31:44.598 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:31:44.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:44.729 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:31:44.729 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:31:44.729 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T08:31:44.729 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:31:44.729 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:31:44.766 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T08:31:44.766 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:31:44.766 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:31:44.822 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T08:31:44.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:31:44.822 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:31:44.878 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:31:44.944 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:31:45.000 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:31:45.001 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:31:45.002 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:31:45.003 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:31:45.003 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:31:45.003 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:31:45.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:45.004 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:31:45.059 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:31:45.060 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T08:31:45.102 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:31:45.127 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:31:45.184 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:31:45.184 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:31:45.184 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:31:45.185 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:31:45.186 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:31:45.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:45.187 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:31:45.199 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:2024-04-25T08:31:45.202+0000 7febd73d7480 -1 init, newargv = 0x560c62092150 newargc=15 2024-04-25T08:31:45.199 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:ceph-fuse[41793]: starting ceph client 2024-04-25T08:31:45.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:ceph-fuse[41793]: starting fuse 2024-04-25T08:31:45.215 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:31:45.215 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:31:45.215 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:31:45.215 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:31:45.215 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:31:45.215 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:31:45.216 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:31:45.217 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:31:45.217 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:31:45.217 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:31:45.217 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.admin")) 2024-04-25T08:31:45.217 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:31:45.334 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.admin.41793.asok 2024-04-25T08:31:45.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:31:45.345 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.41793.asok 2024-04-25T08:31:45.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:45.346 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.41793.asok status 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "admin", 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "41793", 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:31:45.483 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4410, 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 4410 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2719309768 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2719309768 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:31:45.484 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.4410 192.168.0.1:0/2719309768", 2024-04-25T08:31:45.485 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/2719309768", 2024-04-25T08:31:45.485 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:31:45.485 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 7, 2024-04-25T08:31:45.485 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 24, 2024-04-25T08:31:45.485 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:31:45.485 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:31:45.485 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:31:45.485 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:31:45.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:45.491 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:31:45.492 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:31:45.507 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:31:45.507 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:31:45.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:45.507 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:31:45.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:45.573 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T08:31:45.645 INFO:teuthology.orchestra.run.smithi038.stdout:# file: . 2024-04-25T08:31:45.645 INFO:teuthology.orchestra.run.smithi038.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T08:31:45.645 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:45.646 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:45.646 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:31:45.701 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:45.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.845+0000 7f4a2102a640 1 Processor -- start 2024-04-25T08:31:45.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.845+0000 7f4a2102a640 1 -- start start 2024-04-25T08:31:45.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a2102a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a1c15dc40 0x7f4a1c0d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:45.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a2102a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4a1c0b9580 0x7f4a1c0d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:45.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a2102a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a1c0b8400 con 0x7f4a1c0d1e40 2024-04-25T08:31:45.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a2102a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a1c0d19b0 con 0x7f4a1c0b9580 2024-04-25T08:31:45.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a2102a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0b86e0 con 0x7f4a1c15dc40 2024-04-25T08:31:45.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a13fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4a1c0b9580 0x7f4a1c0d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:45.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a13fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4a1c0b9580 0x7f4a1c0d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40134/0 (socket says 172.21.15.38:40134) 2024-04-25T08:31:45.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a13fff640 1 -- 172.21.15.38:0/416177824 learned_addr learned my addr 172.21.15.38:0/416177824 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:45.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a1ad9d640 1 --2- 172.21.15.38:0/416177824 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a1c15dc40 0x7f4a1c0d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:45.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a13fff640 1 -- 172.21.15.38:0/416177824 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a1c15dc40 msgr2=0x7f4a1c0d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:45.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a13fff640 1 --2- 172.21.15.38:0/416177824 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a1c15dc40 0x7f4a1c0d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:45.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a13fff640 1 -- 172.21.15.38:0/416177824 >> v1:172.21.15.107:6789/0 conn(0x7f4a1c0d1e40 legacy=0x7f4a1c0d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:45.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.846+0000 7f4a13fff640 1 -- 172.21.15.38:0/416177824 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c0b89c0 con 0x7f4a1c0b9580 2024-04-25T08:31:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.847+0000 7f4a13fff640 1 --2- 172.21.15.38:0/416177824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a1c0b9580 0x7f4a1c0d0e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009870 tx=0x7f4a0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7570925e0d7229e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.847+0000 7f4a1a59c640 1 -- 172.21.15.38:0/416177824 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04037070 con 0x7f4a1c0b9580 2024-04-25T08:31:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.847+0000 7f4a1a59c640 1 -- 172.21.15.38:0/416177824 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a04032440 con 0x7f4a1c0b9580 2024-04-25T08:31:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.847+0000 7f4a1a59c640 1 -- 172.21.15.38:0/416177824 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04032810 con 0x7f4a1c0b9580 2024-04-25T08:31:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.848+0000 7f4a2102a640 1 -- 172.21.15.38:0/416177824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a1c0b9580 msgr2=0x7f4a1c0d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.848+0000 7f4a2102a640 1 --2- 172.21.15.38:0/416177824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a1c0b9580 0x7f4a1c0d0e10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4a04009870 tx=0x7f4a0402a270 comp rx=0 tx=0).stop 2024-04-25T08:31:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.848+0000 7f4a2102a640 1 -- 172.21.15.38:0/416177824 shutdown_connections 2024-04-25T08:31:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.848+0000 7f4a2102a640 1 --2- 172.21.15.38:0/416177824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a1c0b9580 0x7f4a1c0d0e10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.848+0000 7f4a2102a640 1 --2- 172.21.15.38:0/416177824 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a1c15dc40 0x7f4a1c0d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.848+0000 7f4a2102a640 1 -- 172.21.15.38:0/416177824 >> 172.21.15.38:0/416177824 conn(0x7f4a1c066ce0 msgr2=0x7f4a1c0670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.848+0000 7f4a2102a640 1 -- 172.21.15.38:0/416177824 shutdown_connections 2024-04-25T08:31:45.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.848+0000 7f4a2102a640 1 -- 172.21.15.38:0/416177824 wait complete. 2024-04-25T08:31:45.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.849+0000 7f4a2102a640 1 Processor -- start 2024-04-25T08:31:45.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.849+0000 7f4a2102a640 1 -- start start 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a2102a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a1c0b9580 0x7f4a1c0cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a2102a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c0d1e40 0x7f4a1c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a2102a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a1c15dc40 0x7f4a1c160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a2102a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0b7bf0 con 0x7f4a1c0b9580 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a2102a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0b7a70 con 0x7f4a1c0d1e40 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a2102a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1c0b7d70 con 0x7f4a1c15dc40 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a1ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a1c0b9580 0x7f4a1c0cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a1ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a1c0b9580 0x7f4a1c0cbe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40144/0 (socket says 172.21.15.38:40144) 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a1ad9d640 1 -- 172.21.15.38:0/1253396895 learned_addr learned my addr 172.21.15.38:0/1253396895 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a13fff640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c0d1e40 0x7f4a1c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a1b59e640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a1c15dc40 0x7f4a1c160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a1ad9d640 1 -- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a1c15dc40 msgr2=0x7f4a1c160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a1ad9d640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a1c15dc40 0x7f4a1c160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:45.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a1ad9d640 1 -- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c0d1e40 msgr2=0x7f4a1c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:45.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a1ad9d640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c0d1e40 0x7f4a1c15ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:45.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a1ad9d640 1 -- 172.21.15.38:0/1253396895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c0c7350 con 0x7f4a1c0b9580 2024-04-25T08:31:45.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.850+0000 7f4a1b59e640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a1c15dc40 0x7f4a1c160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:45.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.851+0000 7f4a1ad9d640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a1c0b9580 0x7f4a1c0cbe60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00d5f0 tx=0x7f4a0c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:45.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.851+0000 7f4a137fe640 1 -- 172.21.15.38:0/1253396895 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c0125f0 con 0x7f4a1c0b9580 2024-04-25T08:31:45.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.851+0000 7f4a137fe640 1 -- 172.21.15.38:0/1253396895 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0c019070 con 0x7f4a1c0b9580 2024-04-25T08:31:45.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.851+0000 7f4a2102a640 1 -- 172.21.15.38:0/1253396895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1c15f380 con 0x7f4a1c0b9580 2024-04-25T08:31:45.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.851+0000 7f4a137fe640 1 -- 172.21.15.38:0/1253396895 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c0129d0 con 0x7f4a1c0b9580 2024-04-25T08:31:45.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.851+0000 7f4a2102a640 1 -- 172.21.15.38:0/1253396895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1c15f660 con 0x7f4a1c0b9580 2024-04-25T08:31:45.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.852+0000 7f4a2102a640 1 -- 172.21.15.38:0/1253396895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a1c068e80 con 0x7f4a1c0b9580 2024-04-25T08:31:45.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.852+0000 7f4a137fe640 1 -- 172.21.15.38:0/1253396895 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a0c012b70 con 0x7f4a1c0b9580 2024-04-25T08:31:45.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.853+0000 7f4a137fe640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f49fc06b4d0 0x7f49fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:45.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.853+0000 7f4a137fe640 1 -- 172.21.15.38:0/1253396895 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4a0c08a950 con 0x7f4a1c0b9580 2024-04-25T08:31:45.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.853+0000 7f4a13fff640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f49fc06b4d0 0x7f49fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:45.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.854+0000 7f4a13fff640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f49fc06b4d0 0x7f49fc06d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4a04034000 tx=0x7f4a0402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:45.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:45.857+0000 7f4a137fe640 1 -- 172.21.15.38:0/1253396895 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a0c059490 con 0x7f4a1c0b9580 2024-04-25T08:31:46.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.038+0000 7f4a2102a640 1 -- 172.21.15.38:0/1253396895 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a1c0cef00 con 0x7f4a1c0b9580 2024-04-25T08:31:46.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.039+0000 7f4a137fe640 1 -- 172.21.15.38:0/1253396895 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f4a0c058d90 con 0x7f4a1c0b9580 2024-04-25T08:31:46.035 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:46.036 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:46.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.041+0000 7f4a2102a640 1 -- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f49fc06b4d0 msgr2=0x7f49fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.041+0000 7f4a2102a640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f49fc06b4d0 0x7f49fc06d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4a04034000 tx=0x7f4a0402e000 comp rx=0 tx=0).stop 2024-04-25T08:31:46.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.041+0000 7f4a2102a640 1 -- 172.21.15.38:0/1253396895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a1c0b9580 msgr2=0x7f4a1c0cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.041+0000 7f4a2102a640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a1c0b9580 0x7f4a1c0cbe60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00d5f0 tx=0x7f4a0c00dab0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.041+0000 7f4a2102a640 1 -- 172.21.15.38:0/1253396895 shutdown_connections 2024-04-25T08:31:46.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.041+0000 7f4a2102a640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f49fc06b4d0 0x7f49fc06d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.041+0000 7f4a2102a640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a1c15dc40 0x7f4a1c160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.041+0000 7f4a2102a640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a1c0d1e40 0x7f4a1c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.041+0000 7f4a2102a640 1 --2- 172.21.15.38:0/1253396895 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a1c0b9580 0x7f4a1c0cbe60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.041+0000 7f4a2102a640 1 -- 172.21.15.38:0/1253396895 >> 172.21.15.38:0/1253396895 conn(0x7f4a1c066ce0 msgr2=0x7f4a1c0d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:46.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.041+0000 7f4a2102a640 1 -- 172.21.15.38:0/1253396895 shutdown_connections 2024-04-25T08:31:46.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.041+0000 7f4a2102a640 1 -- 172.21.15.38:0/1253396895 wait complete. 2024-04-25T08:31:46.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:31:46.048 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:31:46.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.157+0000 7fc2b7c07640 1 Processor -- start 2024-04-25T08:31:46.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.157+0000 7fc2b7c07640 1 -- start start 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b7c07640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc2b00cf450 0x7fc2b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b7c07640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc2b0071aa0 0x7fc2b015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b7c07640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b015d1a0 con 0x7fc2b006d8d0 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b7c07640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b015d480 con 0x7fc2b00cf450 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b7c07640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b015d760 con 0x7fc2b0071aa0 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b59a3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc2b00cf450 0x7fc2b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b51a2640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc2b006d8d0 0x7fc2b0071390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51870/0 (socket says 172.21.15.38:51870) 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b59a3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc2b00cf450 0x7fc2b006d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40156/0 (socket says 172.21.15.38:40156) 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b51a2640 1 -- 172.21.15.38:0/825565963 learned_addr learned my addr 172.21.15.38:0/825565963 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b61a4640 1 --2- 172.21.15.38:0/825565963 >> v2:172.21.15.107:3300/0 conn(0x7fc2b0071aa0 0x7fc2b015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b59a3640 1 -- 172.21.15.38:0/825565963 >> v2:172.21.15.107:3300/0 conn(0x7fc2b0071aa0 msgr2=0x7fc2b015bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b59a3640 1 --2- 172.21.15.38:0/825565963 >> v2:172.21.15.107:3300/0 conn(0x7fc2b0071aa0 0x7fc2b015bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b59a3640 1 -- 172.21.15.38:0/825565963 >> v1:172.21.15.38:6789/0 conn(0x7fc2b006d8d0 legacy=0x7fc2b0071390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b59a3640 1 -- 172.21.15.38:0/825565963 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b015da40 con 0x7fc2b00cf450 2024-04-25T08:31:46.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.158+0000 7fc2b49a1640 1 -- 172.21.15.38:0/825565963 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506350754 0 0) 0x7fc2b015d1a0 con 0x7fc2b006d8d0 2024-04-25T08:31:46.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.159+0000 7fc2b59a3640 1 --2- 172.21.15.38:0/825565963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2b00cf450 0x7fc2b006d390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc2b006f2b0 tx=0x7fc2ac02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6506e286b9e9fe40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:46.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.159+0000 7fc2b49a1640 1 -- 172.21.15.38:0/825565963 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac037070 con 0x7fc2b00cf450 2024-04-25T08:31:46.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.159+0000 7fc2b49a1640 1 -- 172.21.15.38:0/825565963 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac032440 con 0x7fc2b00cf450 2024-04-25T08:31:46.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.159+0000 7fc2b49a1640 1 -- 172.21.15.38:0/825565963 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac02ae60 con 0x7fc2b00cf450 2024-04-25T08:31:46.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.159+0000 7fc2b7c07640 1 -- 172.21.15.38:0/825565963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2b00cf450 msgr2=0x7fc2b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.159+0000 7fc2b7c07640 1 --2- 172.21.15.38:0/825565963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2b00cf450 0x7fc2b006d390 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc2b006f2b0 tx=0x7fc2ac02a110 comp rx=0 tx=0).stop 2024-04-25T08:31:46.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.160+0000 7fc2b7c07640 1 -- 172.21.15.38:0/825565963 shutdown_connections 2024-04-25T08:31:46.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.160+0000 7fc2b7c07640 1 --2- 172.21.15.38:0/825565963 >> v2:172.21.15.107:3300/0 conn(0x7fc2b0071aa0 0x7fc2b015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.160+0000 7fc2b7c07640 1 --2- 172.21.15.38:0/825565963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2b00cf450 0x7fc2b006d390 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.160+0000 7fc2b7c07640 1 -- 172.21.15.38:0/825565963 >> 172.21.15.38:0/825565963 conn(0x7fc2b00cc2d0 msgr2=0x7fc2b00cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:46.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.160+0000 7fc2b7c07640 1 -- 172.21.15.38:0/825565963 shutdown_connections 2024-04-25T08:31:46.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.160+0000 7fc2b7c07640 1 -- 172.21.15.38:0/825565963 wait complete. 2024-04-25T08:31:46.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.161+0000 7fc2b7c07640 1 Processor -- start 2024-04-25T08:31:46.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.161+0000 7fc2b7c07640 1 -- start start 2024-04-25T08:31:46.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.161+0000 7fc2b7c07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2b006d8d0 0x7fc2b01554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.161+0000 7fc2b7c07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2b0071aa0 0x7fc2b0153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.161+0000 7fc2b7c07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2b00cf450 0x7fc2b0153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.161+0000 7fc2b7c07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00cd420 con 0x7fc2b0071aa0 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.161+0000 7fc2b7c07640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b015ceb0 con 0x7fc2b00cf450 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.161+0000 7fc2b7c07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b00cd5a0 con 0x7fc2b006d8d0 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b51a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2b0071aa0 0x7fc2b0153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b51a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2b0071aa0 0x7fc2b0153200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40172/0 (socket says 172.21.15.38:40172) 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b51a2640 1 -- 172.21.15.38:0/1477127883 learned_addr learned my addr 172.21.15.38:0/1477127883 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b59a3640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2b006d8d0 0x7fc2b01554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b61a4640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2b00cf450 0x7fc2b0153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b51a2640 1 -- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2b006d8d0 msgr2=0x7fc2b01554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b51a2640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2b006d8d0 0x7fc2b01554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b51a2640 1 -- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2b00cf450 msgr2=0x7fc2b0153740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b51a2640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2b00cf450 0x7fc2b0153740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b51a2640 1 -- 172.21.15.38:0/1477127883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b0155ec0 con 0x7fc2b0071aa0 2024-04-25T08:31:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b51a2640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2b0071aa0 0x7fc2b0153200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c4d0 tx=0x7fc2a000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:46.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2a67fc640 1 -- 172.21.15.38:0/1477127883 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0018070 con 0x7fc2b0071aa0 2024-04-25T08:31:46.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2a67fc640 1 -- 172.21.15.38:0/1477127883 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a0007500 con 0x7fc2b0071aa0 2024-04-25T08:31:46.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b7c07640 1 -- 172.21.15.38:0/1477127883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b01561a0 con 0x7fc2b0071aa0 2024-04-25T08:31:46.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2a67fc640 1 -- 172.21.15.38:0/1477127883 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0004a20 con 0x7fc2b0071aa0 2024-04-25T08:31:46.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.162+0000 7fc2b7c07640 1 -- 172.21.15.38:0/1477127883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b0156760 con 0x7fc2b0071aa0 2024-04-25T08:31:46.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.163+0000 7fc2b7c07640 1 -- 172.21.15.38:0/1477127883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b0068e80 con 0x7fc2b0071aa0 2024-04-25T08:31:46.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.164+0000 7fc2a67fc640 1 -- 172.21.15.38:0/1477127883 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2a001b070 con 0x7fc2b0071aa0 2024-04-25T08:31:46.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.164+0000 7fc2a67fc640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc29406b4d0 0x7fc29406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.164+0000 7fc2a67fc640 1 -- 172.21.15.38:0/1477127883 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc2a0089790 con 0x7fc2b0071aa0 2024-04-25T08:31:46.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.164+0000 7fc2b59a3640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc29406b4d0 0x7fc29406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.165+0000 7fc2b59a3640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc29406b4d0 0x7fc29406d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac034000 tx=0x7fc2ac02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:46.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.168+0000 7fc2a67fc640 1 -- 172.21.15.38:0/1477127883 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2a0057c80 con 0x7fc2b0071aa0 2024-04-25T08:31:46.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.350+0000 7fc2b7c07640 1 -- 172.21.15.38:0/1477127883 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2b00b1190 con 0x7fc2b0071aa0 2024-04-25T08:31:46.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.350+0000 7fc2a67fc640 1 -- 172.21.15.38:0/1477127883 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc2a005bd20 con 0x7fc2b0071aa0 2024-04-25T08:31:46.347 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:31:46.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.352+0000 7fc2b7c07640 1 -- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc29406b4d0 msgr2=0x7fc29406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.352+0000 7fc2b7c07640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc29406b4d0 0x7fc29406d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac034000 tx=0x7fc2ac02e000 comp rx=0 tx=0).stop 2024-04-25T08:31:46.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.352+0000 7fc2b7c07640 1 -- 172.21.15.38:0/1477127883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2b0071aa0 msgr2=0x7fc2b0153200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.352+0000 7fc2b7c07640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2b0071aa0 0x7fc2b0153200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000c4d0 tx=0x7fc2a000c990 comp rx=0 tx=0).stop 2024-04-25T08:31:46.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.353+0000 7fc2b7c07640 1 -- 172.21.15.38:0/1477127883 shutdown_connections 2024-04-25T08:31:46.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.353+0000 7fc2b7c07640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2b00cf450 0x7fc2b0153740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.353+0000 7fc2b7c07640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2b0071aa0 0x7fc2b0153200 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.353+0000 7fc2b7c07640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc29406b4d0 0x7fc29406d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.353+0000 7fc2b7c07640 1 --2- 172.21.15.38:0/1477127883 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2b006d8d0 0x7fc2b01554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.353+0000 7fc2b7c07640 1 -- 172.21.15.38:0/1477127883 >> 172.21.15.38:0/1477127883 conn(0x7fc2b00cc2d0 msgr2=0x7fc2b015b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:46.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.353+0000 7fc2b7c07640 1 -- 172.21.15.38:0/1477127883 shutdown_connections 2024-04-25T08:31:46.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.353+0000 7fc2b7c07640 1 -- 172.21.15.38:0/1477127883 wait complete. 2024-04-25T08:31:46.360 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:31:46.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.467+0000 7f2fa035c640 1 Processor -- start 2024-04-25T08:31:46.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.467+0000 7f2fa035c640 1 -- start start 2024-04-25T08:31:46.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.468+0000 7f2fa035c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f980d0320 0x7f2f980d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.468+0000 7f2fa035c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2f980b6980 0x7f2f980b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.468+0000 7f2fa035c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f980d2060 con 0x7f2f980d0700 2024-04-25T08:31:46.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.468+0000 7f2fa035c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f980d2330 con 0x7f2f980b6980 2024-04-25T08:31:46.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.468+0000 7f2fa035c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980d2610 con 0x7f2f980d0320 2024-04-25T08:31:46.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.468+0000 7f2f9e0f8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f980d0320 0x7f2f980d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.468+0000 7f2f9e8f9640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2f980b6980 0x7f2f980b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.468+0000 7f2f9d8f7640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f2f980d0700 0x7f2f980d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:52558/0 (socket says 172.21.15.38:52558) 2024-04-25T08:31:46.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.468+0000 7f2f9d8f7640 1 -- 172.21.15.38:0/1218395161 learned_addr learned my addr 172.21.15.38:0/1218395161 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.469+0000 7f2f9d0f6640 1 -- 172.21.15.38:0/1218395161 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536083543 0 0) 0x7f2f980d2060 con 0x7f2f980d0700 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.469+0000 7f2f9d0f6640 1 -- 172.21.15.38:0/1218395161 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f80003520 con 0x7f2f980d0700 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.469+0000 7f2f9e0f8640 1 -- 172.21.15.38:0/1218395161 >> v2:172.21.15.107:3300/0 conn(0x7f2f980b6980 msgr2=0x7f2f980b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.469+0000 7f2f9e0f8640 1 --2- 172.21.15.38:0/1218395161 >> v2:172.21.15.107:3300/0 conn(0x7f2f980b6980 0x7f2f980b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.469+0000 7f2f9e0f8640 1 -- 172.21.15.38:0/1218395161 >> v1:172.21.15.107:6789/0 conn(0x7f2f980d0700 legacy=0x7f2f980d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.469+0000 7f2f9e0f8640 1 -- 172.21.15.38:0/1218395161 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f980d28f0 con 0x7f2f980d0320 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.469+0000 7f2f9e8f9640 1 --2- 172.21.15.38:0/1218395161 >> v2:172.21.15.107:3300/0 conn(0x7f2f980b6980 0x7f2f980b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.469+0000 7f2f9e0f8640 1 --2- 172.21.15.38:0/1218395161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f980d0320 0x7f2f980d33d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2f900097a0 tx=0x7f2f9002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9378e64f7ef202 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.470+0000 7f2f9d0f6640 1 -- 172.21.15.38:0/1218395161 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90037070 con 0x7f2f980d0320 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.470+0000 7f2f9d0f6640 1 -- 172.21.15.38:0/1218395161 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f90032440 con 0x7f2f980d0320 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.470+0000 7f2f9d0f6640 1 -- 172.21.15.38:0/1218395161 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90032810 con 0x7f2f980d0320 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.470+0000 7f2fa035c640 1 -- 172.21.15.38:0/1218395161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f980d0320 msgr2=0x7f2f980d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.470+0000 7f2fa035c640 1 --2- 172.21.15.38:0/1218395161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f980d0320 0x7f2f980d33d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2f900097a0 tx=0x7f2f9002a210 comp rx=0 tx=0).stop 2024-04-25T08:31:46.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.470+0000 7f2fa035c640 1 -- 172.21.15.38:0/1218395161 shutdown_connections 2024-04-25T08:31:46.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.470+0000 7f2fa035c640 1 --2- 172.21.15.38:0/1218395161 >> v2:172.21.15.107:3300/0 conn(0x7f2f980b6980 0x7f2f980b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.470+0000 7f2fa035c640 1 --2- 172.21.15.38:0/1218395161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f980d0320 0x7f2f980d33d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.470+0000 7f2fa035c640 1 -- 172.21.15.38:0/1218395161 >> 172.21.15.38:0/1218395161 conn(0x7f2f980668f0 msgr2=0x7f2f98066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:46.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.470+0000 7f2fa035c640 1 -- 172.21.15.38:0/1218395161 shutdown_connections 2024-04-25T08:31:46.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.470+0000 7f2fa035c640 1 -- 172.21.15.38:0/1218395161 wait complete. 2024-04-25T08:31:46.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.471+0000 7f2fa035c640 1 Processor -- start 2024-04-25T08:31:46.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.471+0000 7f2fa035c640 1 -- start start 2024-04-25T08:31:46.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.472+0000 7f2fa035c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f980b6980 0x7f2f980b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.472+0000 7f2fa035c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f980d0320 0x7f2f980c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.472+0000 7f2fa035c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f980d0700 0x7f2f980c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.472+0000 7f2fa035c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980d0b40 con 0x7f2f980b6980 2024-04-25T08:31:46.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.472+0000 7f2fa035c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f980d2eb0 con 0x7f2f980d0700 2024-04-25T08:31:46.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.472+0000 7f2fa035c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9814be60 con 0x7f2f980d0320 2024-04-25T08:31:46.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.472+0000 7f2f9e0f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f980b6980 0x7f2f980b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.472+0000 7f2f9e0f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f980b6980 0x7f2f980b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40184/0 (socket says 172.21.15.38:40184) 2024-04-25T08:31:46.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.472+0000 7f2f9e0f8640 1 -- 172.21.15.38:0/4078151191 learned_addr learned my addr 172.21.15.38:0/4078151191 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.472+0000 7f2f9d8f7640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f980d0320 0x7f2f980c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.472+0000 7f2f9e8f9640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f980d0700 0x7f2f980c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2f9e0f8640 1 -- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f980d0320 msgr2=0x7f2f980c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2f9e0f8640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f980d0320 0x7f2f980c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2f9e0f8640 1 -- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f980d0700 msgr2=0x7f2f980c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2f9e0f8640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f980d0700 0x7f2f980c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2f9e0f8640 1 -- 172.21.15.38:0/4078151191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f981e6f00 con 0x7f2f980b6980 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2f9d8f7640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f980d0320 0x7f2f980c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2f9e8f9640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f980d0700 0x7f2f980c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2f9e0f8640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f980b6980 0x7f2f980b4ec0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2f90035040 tx=0x7f2f900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2f96ffd640 1 -- 172.21.15.38:0/4078151191 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90041030 con 0x7f2f980b6980 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2f96ffd640 1 -- 172.21.15.38:0/4078151191 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9003d4e0 con 0x7f2f980b6980 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2fa035c640 1 -- 172.21.15.38:0/4078151191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f981e71e0 con 0x7f2f980b6980 2024-04-25T08:31:46.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2f96ffd640 1 -- 172.21.15.38:0/4078151191 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f90037040 con 0x7f2f980b6980 2024-04-25T08:31:46.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.473+0000 7f2fa035c640 1 -- 172.21.15.38:0/4078151191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f9814baf0 con 0x7f2f980b6980 2024-04-25T08:31:46.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.474+0000 7f2fa035c640 1 -- 172.21.15.38:0/4078151191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f98068e80 con 0x7f2f980b6980 2024-04-25T08:31:46.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.474+0000 7f2f96ffd640 1 -- 172.21.15.38:0/4078151191 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f90044070 con 0x7f2f980b6980 2024-04-25T08:31:46.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.475+0000 7f2f96ffd640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2f8406b4d0 0x7f2f8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.475+0000 7f2f96ffd640 1 -- 172.21.15.38:0/4078151191 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2f900aabc0 con 0x7f2f980b6980 2024-04-25T08:31:46.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.475+0000 7f2f9d8f7640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2f8406b4d0 0x7f2f8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.476+0000 7f2f9d8f7640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2f8406b4d0 0x7f2f8406d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2f981f7c20 tx=0x7f2f8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:46.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.479+0000 7f2f96ffd640 1 -- 172.21.15.38:0/4078151191 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f90044e40 con 0x7f2f980b6980 2024-04-25T08:31:46.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.659+0000 7f2fa035c640 1 -- 172.21.15.38:0/4078151191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f980b3050 con 0x7f2f980b6980 2024-04-25T08:31:46.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.660+0000 7f2f96ffd640 1 -- 172.21.15.38:0/4078151191 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f90079710 con 0x7f2f980b6980 2024-04-25T08:31:46.656 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:31:46.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.661+0000 7f2fa035c640 1 -- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2f8406b4d0 msgr2=0x7f2f8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.661+0000 7f2fa035c640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2f8406b4d0 0x7f2f8406d990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2f981f7c20 tx=0x7f2f8c009040 comp rx=0 tx=0).stop 2024-04-25T08:31:46.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.661+0000 7f2fa035c640 1 -- 172.21.15.38:0/4078151191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f980b6980 msgr2=0x7f2f980b4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.661+0000 7f2fa035c640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f980b6980 0x7f2f980b4ec0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2f90035040 tx=0x7f2f900046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.662+0000 7f2fa035c640 1 -- 172.21.15.38:0/4078151191 shutdown_connections 2024-04-25T08:31:46.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.662+0000 7f2fa035c640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f980d0700 0x7f2f980c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.662+0000 7f2fa035c640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2f8406b4d0 0x7f2f8406d990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.662+0000 7f2fa035c640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f980d0320 0x7f2f980c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.662+0000 7f2fa035c640 1 --2- 172.21.15.38:0/4078151191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f980b6980 0x7f2f980b4ec0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.662+0000 7f2fa035c640 1 -- 172.21.15.38:0/4078151191 >> 172.21.15.38:0/4078151191 conn(0x7f2f980668f0 msgr2=0x7f2f9815f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:46.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.662+0000 7f2fa035c640 1 -- 172.21.15.38:0/4078151191 shutdown_connections 2024-04-25T08:31:46.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.662+0000 7f2fa035c640 1 -- 172.21.15.38:0/4078151191 wait complete. 2024-04-25T08:31:46.669 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:31:46.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.776+0000 7fe01e770640 1 Processor -- start 2024-04-25T08:31:46.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.776+0000 7fe01e770640 1 -- start start 2024-04-25T08:31:46.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.777+0000 7fe01e770640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe0180d0320 0x7fe0180d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.777+0000 7fe01e770640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0180d2060 con 0x7fe0180d0700 2024-04-25T08:31:46.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.777+0000 7fe01e770640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0180d2330 con 0x7fe0180b6980 2024-04-25T08:31:46.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.777+0000 7fe01e770640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0180d2610 con 0x7fe0180d0320 2024-04-25T08:31:46.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.777+0000 7fe0177fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe0180d0700 0x7fe0180d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51876/0 (socket says 172.21.15.38:51876) 2024-04-25T08:31:46.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.777+0000 7fe0177fe640 1 -- 172.21.15.38:0/1401007911 learned_addr learned my addr 172.21.15.38:0/1401007911 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:46.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.777+0000 7fe017fff640 1 --2- 172.21.15.38:0/1401007911 >> v2:172.21.15.107:3300/0 conn(0x7fe0180d0320 0x7fe0180d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.778+0000 7fe016ffd640 1 -- 172.21.15.38:0/1401007911 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659945656 0 0) 0x7fe0180d2060 con 0x7fe0180d0700 2024-04-25T08:31:46.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.778+0000 7fe016ffd640 1 -- 172.21.15.38:0/1401007911 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe000003520 con 0x7fe0180d0700 2024-04-25T08:31:46.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.778+0000 7fe017fff640 1 -- 172.21.15.38:0/1401007911 >> v1:172.21.15.107:6789/0 conn(0x7fe0180b6980 legacy=0x7fe0180b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.778+0000 7fe017fff640 1 -- 172.21.15.38:0/1401007911 >> v1:172.21.15.38:6789/0 conn(0x7fe0180d0700 legacy=0x7fe0180d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.778+0000 7fe016ffd640 1 -- 172.21.15.38:0/1401007911 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1427565899 0 0) 0x7fe000003520 con 0x7fe0180d0700 2024-04-25T08:31:46.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.778+0000 7fe017fff640 1 -- 172.21.15.38:0/1401007911 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0180d28f0 con 0x7fe0180d0320 2024-04-25T08:31:46.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.779+0000 7fe017fff640 1 --2- 172.21.15.38:0/1401007911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0180d0320 0x7fe0180d33d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0097a0 tx=0x7fe00c02a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24b23bcbd01f1443 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:46.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.779+0000 7fe016ffd640 1 -- 172.21.15.38:0/1401007911 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c037070 con 0x7fe0180d0320 2024-04-25T08:31:46.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.779+0000 7fe016ffd640 1 -- 172.21.15.38:0/1401007911 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c032640 con 0x7fe0180d0320 2024-04-25T08:31:46.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.779+0000 7fe016ffd640 1 -- 172.21.15.38:0/1401007911 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c032a10 con 0x7fe0180d0320 2024-04-25T08:31:46.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.779+0000 7fe01e770640 1 -- 172.21.15.38:0/1401007911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0180d0320 msgr2=0x7fe0180d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.779+0000 7fe01e770640 1 --2- 172.21.15.38:0/1401007911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0180d0320 0x7fe0180d33d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0097a0 tx=0x7fe00c02a2c0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.779+0000 7fe01e770640 1 -- 172.21.15.38:0/1401007911 shutdown_connections 2024-04-25T08:31:46.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.779+0000 7fe01e770640 1 --2- 172.21.15.38:0/1401007911 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0180d0320 0x7fe0180d33d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.779+0000 7fe01e770640 1 -- 172.21.15.38:0/1401007911 >> 172.21.15.38:0/1401007911 conn(0x7fe0180668f0 msgr2=0x7fe018066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:46.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.779+0000 7fe01e770640 1 -- 172.21.15.38:0/1401007911 shutdown_connections 2024-04-25T08:31:46.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.779+0000 7fe01e770640 1 -- 172.21.15.38:0/1401007911 wait complete. 2024-04-25T08:31:46.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.780+0000 7fe01e770640 1 Processor -- start 2024-04-25T08:31:46.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.781+0000 7fe01e770640 1 -- start start 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.781+0000 7fe01e770640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0180b6980 0x7fe0180b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.781+0000 7fe01e770640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0180d0320 0x7fe0180b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.781+0000 7fe01e770640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0180d0700 0x7fe0180b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.781+0000 7fe01e770640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180d0b40 con 0x7fe0180d0700 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.781+0000 7fe01e770640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180d2eb0 con 0x7fe0180b6980 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.781+0000 7fe01e770640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0180b5b40 con 0x7fe0180d0320 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.781+0000 7fe01cd0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0180d0700 0x7fe0180b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.781+0000 7fe01cd0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0180d0700 0x7fe0180b8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40190/0 (socket says 172.21.15.38:40190) 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe01cd0d640 1 -- 172.21.15.38:0/4228316196 learned_addr learned my addr 172.21.15.38:0/4228316196 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe0177fe640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0180d0320 0x7fe0180b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe017fff640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0180b6980 0x7fe0180b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe01cd0d640 1 -- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0180d0320 msgr2=0x7fe0180b7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe01cd0d640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0180d0320 0x7fe0180b7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe01cd0d640 1 -- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0180b6980 msgr2=0x7fe0180b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe01cd0d640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0180b6980 0x7fe0180b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe01cd0d640 1 -- 172.21.15.38:0/4228316196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe018149b80 con 0x7fe0180d0700 2024-04-25T08:31:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe01cd0d640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0180d0700 0x7fe0180b8490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe01000c7d0 tx=0x7fe01000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe014ff9640 1 -- 172.21.15.38:0/4228316196 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010003b70 con 0x7fe0180d0700 2024-04-25T08:31:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe014ff9640 1 -- 172.21.15.38:0/4228316196 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010003d30 con 0x7fe0180d0700 2024-04-25T08:31:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe01e770640 1 -- 172.21.15.38:0/4228316196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0180b5dc0 con 0x7fe0180d0700 2024-04-25T08:31:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe014ff9640 1 -- 172.21.15.38:0/4228316196 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01000aa30 con 0x7fe0180d0700 2024-04-25T08:31:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.782+0000 7fe01e770640 1 -- 172.21.15.38:0/4228316196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe018149830 con 0x7fe0180d0700 2024-04-25T08:31:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.783+0000 7fe01e770640 1 -- 172.21.15.38:0/4228316196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe018068e80 con 0x7fe0180d0700 2024-04-25T08:31:46.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.784+0000 7fe014ff9640 1 -- 172.21.15.38:0/4228316196 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe01000abd0 con 0x7fe0180d0700 2024-04-25T08:31:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.784+0000 7fe014ff9640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe00406b4d0 0x7fe00406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.784+0000 7fe014ff9640 1 -- 172.21.15.38:0/4228316196 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe01008a330 con 0x7fe0180d0700 2024-04-25T08:31:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.784+0000 7fe017fff640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe00406b4d0 0x7fe00406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:46.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.785+0000 7fe017fff640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe00406b4d0 0x7fe00406d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe00c034000 tx=0x7fe00c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:46.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.788+0000 7fe014ff9640 1 -- 172.21.15.38:0/4228316196 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe010058790 con 0x7fe0180d0700 2024-04-25T08:31:46.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.966+0000 7fe01e770640 1 -- 172.21.15.38:0/4228316196 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe0180b3050 con 0x7fe0180d0700 2024-04-25T08:31:46.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.967+0000 7fe014ff9640 1 -- 172.21.15.38:0/4228316196 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe01005c830 con 0x7fe0180d0700 2024-04-25T08:31:46.963 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:31:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.968+0000 7fe01e770640 1 -- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe00406b4d0 msgr2=0x7fe00406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.968+0000 7fe01e770640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe00406b4d0 0x7fe00406d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe00c034000 tx=0x7fe00c02e000 comp rx=0 tx=0).stop 2024-04-25T08:31:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.968+0000 7fe01e770640 1 -- 172.21.15.38:0/4228316196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0180d0700 msgr2=0x7fe0180b8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.968+0000 7fe01e770640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0180d0700 0x7fe0180b8490 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe01000c7d0 tx=0x7fe01000cc90 comp rx=0 tx=0).stop 2024-04-25T08:31:46.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.968+0000 7fe01e770640 1 -- 172.21.15.38:0/4228316196 shutdown_connections 2024-04-25T08:31:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.968+0000 7fe01e770640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0180d0700 0x7fe0180b8490 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.968+0000 7fe01e770640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe00406b4d0 0x7fe00406d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.968+0000 7fe01e770640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0180d0320 0x7fe0180b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.968+0000 7fe01e770640 1 --2- 172.21.15.38:0/4228316196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0180b6980 0x7fe0180b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.968+0000 7fe01e770640 1 -- 172.21.15.38:0/4228316196 >> 172.21.15.38:0/4228316196 conn(0x7fe0180668f0 msgr2=0x7fe0180ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.969+0000 7fe01e770640 1 -- 172.21.15.38:0/4228316196 shutdown_connections 2024-04-25T08:31:46.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:46.969+0000 7fe01e770640 1 -- 172.21.15.38:0/4228316196 wait complete. 2024-04-25T08:31:46.976 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:31:47.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.083+0000 7f2ec72d1640 1 Processor -- start 2024-04-25T08:31:47.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.083+0000 7f2ec72d1640 1 -- start start 2024-04-25T08:31:47.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec72d1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2ec00d33d0 0x7f2ec00d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec72d1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ec00d1e90 0x7f2ec00b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec72d1640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec00d2330 con 0x7f2ec00d0320 2024-04-25T08:31:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec72d1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec00d2610 con 0x7f2ec00d33d0 2024-04-25T08:31:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec72d1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00d28f0 con 0x7f2ec00d1e90 2024-04-25T08:31:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec486c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2ec00d33d0 0x7f2ec00d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec586e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ec00d1e90 0x7f2ec00b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec486c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2ec00d33d0 0x7f2ec00d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:51200/0 (socket says 172.21.15.38:51200) 2024-04-25T08:31:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec506d640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f2ec00d0320 0x7f2ec00d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:52582/0 (socket says 172.21.15.38:52582) 2024-04-25T08:31:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec486c640 1 -- 172.21.15.38:0/3180328765 learned_addr learned my addr 172.21.15.38:0/3180328765 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec486c640 1 -- 172.21.15.38:0/3180328765 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ec00d1e90 msgr2=0x7f2ec00b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec486c640 1 --2- 172.21.15.38:0/3180328765 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ec00d1e90 0x7f2ec00b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec486c640 1 -- 172.21.15.38:0/3180328765 >> v1:172.21.15.107:6789/0 conn(0x7f2ec00d0320 legacy=0x7f2ec00d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.084+0000 7f2ec486c640 1 -- 172.21.15.38:0/3180328765 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec00d2bd0 con 0x7f2ec00d33d0 2024-04-25T08:31:47.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.085+0000 7f2ec486c640 1 --2- 172.21.15.38:0/3180328765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec00d33d0 0x7f2ec00d1950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009870 tx=0x7f2eb002a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea5cc66b3862bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:47.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.085+0000 7f2eb7fff640 1 -- 172.21.15.38:0/3180328765 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0037070 con 0x7f2ec00d33d0 2024-04-25T08:31:47.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.085+0000 7f2eb7fff640 1 -- 172.21.15.38:0/3180328765 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb002ae50 con 0x7f2ec00d33d0 2024-04-25T08:31:47.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.085+0000 7f2eb7fff640 1 -- 172.21.15.38:0/3180328765 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0032860 con 0x7f2ec00d33d0 2024-04-25T08:31:47.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.086+0000 7f2ec72d1640 1 -- 172.21.15.38:0/3180328765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec00d33d0 msgr2=0x7f2ec00d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.086+0000 7f2ec72d1640 1 --2- 172.21.15.38:0/3180328765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec00d33d0 0x7f2ec00d1950 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0009870 tx=0x7f2eb002a300 comp rx=0 tx=0).stop 2024-04-25T08:31:47.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.086+0000 7f2ec72d1640 1 -- 172.21.15.38:0/3180328765 shutdown_connections 2024-04-25T08:31:47.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.086+0000 7f2ec72d1640 1 --2- 172.21.15.38:0/3180328765 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ec00d1e90 0x7f2ec00b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.086+0000 7f2ec72d1640 1 --2- 172.21.15.38:0/3180328765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec00d33d0 0x7f2ec00d1950 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.086+0000 7f2ec72d1640 1 -- 172.21.15.38:0/3180328765 >> 172.21.15.38:0/3180328765 conn(0x7f2ec00668f0 msgr2=0x7f2ec0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:47.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.086+0000 7f2ec72d1640 1 -- 172.21.15.38:0/3180328765 shutdown_connections 2024-04-25T08:31:47.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.086+0000 7f2ec72d1640 1 -- 172.21.15.38:0/3180328765 wait complete. 2024-04-25T08:31:47.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.087+0000 7f2ec72d1640 1 Processor -- start 2024-04-25T08:31:47.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.087+0000 7f2ec72d1640 1 -- start start 2024-04-25T08:31:47.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec72d1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec00d0320 0x7f2ec00c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec72d1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec00d1e90 0x7f2ec00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec72d1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ec00d33d0 0x7f2ec00ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec72d1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec0161390 con 0x7f2ec00d0320 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec72d1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec0161210 con 0x7f2ec00d1e90 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec72d1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00b9720 con 0x7f2ec00d33d0 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec506d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec00d0320 0x7f2ec00c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec506d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec00d0320 0x7f2ec00c93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40198/0 (socket says 172.21.15.38:40198) 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec506d640 1 -- 172.21.15.38:0/2355130826 learned_addr learned my addr 172.21.15.38:0/2355130826 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec486c640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec00d1e90 0x7f2ec00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec586e640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ec00d33d0 0x7f2ec00ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec506d640 1 -- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ec00d33d0 msgr2=0x7f2ec00ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec506d640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ec00d33d0 0x7f2ec00ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec506d640 1 -- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec00d1e90 msgr2=0x7f2ec00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec506d640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec00d1e90 0x7f2ec00ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec506d640 1 -- 172.21.15.38:0/2355130826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec00b5b00 con 0x7f2ec00d0320 2024-04-25T08:31:47.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec486c640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec00d1e90 0x7f2ec00ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:47.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.088+0000 7f2ec506d640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec00d0320 0x7f2ec00c93e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c400 tx=0x7f2ebc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:47.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.089+0000 7f2eb5ffb640 1 -- 172.21.15.38:0/2355130826 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0074e0 con 0x7f2ec00d0320 2024-04-25T08:31:47.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.089+0000 7f2ec72d1640 1 -- 172.21.15.38:0/2355130826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec00b5de0 con 0x7f2ec00d0320 2024-04-25T08:31:47.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.089+0000 7f2eb5ffb640 1 -- 172.21.15.38:0/2355130826 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc007680 con 0x7f2ec00d0320 2024-04-25T08:31:47.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.089+0000 7f2eb5ffb640 1 -- 172.21.15.38:0/2355130826 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0049e0 con 0x7f2ec00d0320 2024-04-25T08:31:47.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.089+0000 7f2ec72d1640 1 -- 172.21.15.38:0/2355130826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec01e6220 con 0x7f2ec00d0320 2024-04-25T08:31:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.090+0000 7f2ec72d1640 1 -- 172.21.15.38:0/2355130826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec0068e80 con 0x7f2ec00d0320 2024-04-25T08:31:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.090+0000 7f2eb5ffb640 1 -- 172.21.15.38:0/2355130826 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ebc01a020 con 0x7f2ec00d0320 2024-04-25T08:31:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.091+0000 7f2eb5ffb640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ea406b4d0 0x7f2ea406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.091+0000 7f2eb5ffb640 1 -- 172.21.15.38:0/2355130826 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2ebc089c80 con 0x7f2ec00d0320 2024-04-25T08:31:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.091+0000 7f2ec486c640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ea406b4d0 0x7f2ea406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.091+0000 7f2ec486c640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ea406b4d0 0x7f2ea406d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0034000 tx=0x7f2eb002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:47.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.095+0000 7f2eb5ffb640 1 -- 172.21.15.38:0/2355130826 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ebc0580f0 con 0x7f2ec00d0320 2024-04-25T08:31:47.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.274+0000 7f2ec72d1640 1 -- 172.21.15.38:0/2355130826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2ec00b2ab0 con 0x7f2ec00d0320 2024-04-25T08:31:47.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.274+0000 7f2eb5ffb640 1 -- 172.21.15.38:0/2355130826 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2ebc05c190 con 0x7f2ec00d0320 2024-04-25T08:31:47.271 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:31:47.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.276+0000 7f2ec72d1640 1 -- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ea406b4d0 msgr2=0x7f2ea406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.276+0000 7f2ec72d1640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ea406b4d0 0x7f2ea406d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0034000 tx=0x7f2eb002e000 comp rx=0 tx=0).stop 2024-04-25T08:31:47.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.276+0000 7f2ec72d1640 1 -- 172.21.15.38:0/2355130826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec00d0320 msgr2=0x7f2ec00c93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.276+0000 7f2ec72d1640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec00d0320 0x7f2ec00c93e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c400 tx=0x7f2ebc00c8c0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.276+0000 7f2ec72d1640 1 -- 172.21.15.38:0/2355130826 shutdown_connections 2024-04-25T08:31:47.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.276+0000 7f2ec72d1640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ea406b4d0 0x7f2ea406d990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.276+0000 7f2ec72d1640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ec00d33d0 0x7f2ec00ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.276+0000 7f2ec72d1640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ec00d1e90 0x7f2ec00ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.276+0000 7f2ec72d1640 1 --2- 172.21.15.38:0/2355130826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ec00d0320 0x7f2ec00c93e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.276+0000 7f2ec72d1640 1 -- 172.21.15.38:0/2355130826 >> 172.21.15.38:0/2355130826 conn(0x7f2ec00668f0 msgr2=0x7f2ec00d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:47.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.277+0000 7f2ec72d1640 1 -- 172.21.15.38:0/2355130826 shutdown_connections 2024-04-25T08:31:47.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.277+0000 7f2ec72d1640 1 -- 172.21.15.38:0/2355130826 wait complete. 2024-04-25T08:31:47.284 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:31:47.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.389+0000 7fb9b0010640 1 Processor -- start 2024-04-25T08:31:47.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.389+0000 7fb9b0010640 1 -- start start 2024-04-25T08:31:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.390+0000 7fb9b0010640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb9a80b9180 0x7fb9a80d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.390+0000 7fb9b0010640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a80b9610 con 0x7fb9a8160ef0 2024-04-25T08:31:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.390+0000 7fb9b0010640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a81616a0 con 0x7fb9a80b2ab0 2024-04-25T08:31:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.390+0000 7fb9b0010640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a8161cf0 con 0x7fb9a80b9180 2024-04-25T08:31:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.390+0000 7fb9addac640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb9a8160ef0 0x7fb9a80d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51878/0 (socket says 172.21.15.38:51878) 2024-04-25T08:31:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.390+0000 7fb9addac640 1 -- 172.21.15.38:0/1818991172 learned_addr learned my addr 172.21.15.38:0/1818991172 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.390+0000 7fb9ad5ab640 1 --2- 172.21.15.38:0/1818991172 >> v2:172.21.15.107:3300/0 conn(0x7fb9a80b9180 0x7fb9a80d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.390+0000 7fb9acdaa640 1 -- 172.21.15.38:0/1818991172 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769535171 0 0) 0x7fb9a80b9610 con 0x7fb9a8160ef0 2024-04-25T08:31:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.390+0000 7fb9acdaa640 1 -- 172.21.15.38:0/1818991172 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb990003520 con 0x7fb9a8160ef0 2024-04-25T08:31:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.390+0000 7fb9ad5ab640 1 -- 172.21.15.38:0/1818991172 >> v1:172.21.15.107:6789/0 conn(0x7fb9a80b2ab0 legacy=0x7fb9a80d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.391+0000 7fb9acdaa640 1 -- 172.21.15.38:0/1818991172 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3676494980 0 0) 0x7fb990003520 con 0x7fb9a8160ef0 2024-04-25T08:31:47.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.391+0000 7fb9ad5ab640 1 -- 172.21.15.38:0/1818991172 >> v1:172.21.15.38:6789/0 conn(0x7fb9a8160ef0 legacy=0x7fb9a80d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.391+0000 7fb9ad5ab640 1 -- 172.21.15.38:0/1818991172 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a8161980 con 0x7fb9a80b9180 2024-04-25T08:31:47.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.391+0000 7fb9ad5ab640 1 --2- 172.21.15.38:0/1818991172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9a80b9180 0x7fb9a80d1970 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0097a0 tx=0x7fb99c02a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c82f9b2aba2cf79d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:47.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.391+0000 7fb9acdaa640 1 -- 172.21.15.38:0/1818991172 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c037070 con 0x7fb9a80b9180 2024-04-25T08:31:47.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.391+0000 7fb9acdaa640 1 -- 172.21.15.38:0/1818991172 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c02aad0 con 0x7fb9a80b9180 2024-04-25T08:31:47.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.391+0000 7fb9acdaa640 1 -- 172.21.15.38:0/1818991172 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0001670 con 0x7fb9a80b9180 2024-04-25T08:31:47.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.392+0000 7fb9b0010640 1 -- 172.21.15.38:0/1818991172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9a80b9180 msgr2=0x7fb9a80d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.392+0000 7fb9b0010640 1 --2- 172.21.15.38:0/1818991172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9a80b9180 0x7fb9a80d1970 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0097a0 tx=0x7fb99c02a040 comp rx=0 tx=0).stop 2024-04-25T08:31:47.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.392+0000 7fb9b0010640 1 -- 172.21.15.38:0/1818991172 shutdown_connections 2024-04-25T08:31:47.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.392+0000 7fb9b0010640 1 --2- 172.21.15.38:0/1818991172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9a80b9180 0x7fb9a80d1970 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.392+0000 7fb9b0010640 1 -- 172.21.15.38:0/1818991172 >> 172.21.15.38:0/1818991172 conn(0x7fb9a8066d10 msgr2=0x7fb9a8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:47.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.392+0000 7fb9b0010640 1 -- 172.21.15.38:0/1818991172 shutdown_connections 2024-04-25T08:31:47.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.392+0000 7fb9b0010640 1 -- 172.21.15.38:0/1818991172 wait complete. 2024-04-25T08:31:47.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.393+0000 7fb9b0010640 1 Processor -- start 2024-04-25T08:31:47.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.393+0000 7fb9b0010640 1 -- start start 2024-04-25T08:31:47.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9b0010640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9a80b2ab0 0x7fb9a81f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9b0010640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a8160ef0 0x7fb9a814a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9b0010640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9a814b7c0 0x7fb9a814bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9b0010640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a81612d0 con 0x7fb9a8160ef0 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9b0010640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a80d2810 con 0x7fb9a80b2ab0 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9b0010640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a80d0ec0 con 0x7fb9a814b7c0 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9ad5ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a8160ef0 0x7fb9a814a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9ad5ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a8160ef0 0x7fb9a814a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40202/0 (socket says 172.21.15.38:40202) 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9ad5ab640 1 -- 172.21.15.38:0/691844374 learned_addr learned my addr 172.21.15.38:0/691844374 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9addac640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9a80b2ab0 0x7fb9a81f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9ae5ad640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9a814b7c0 0x7fb9a814bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9ad5ab640 1 -- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9a814b7c0 msgr2=0x7fb9a814bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9ad5ab640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9a814b7c0 0x7fb9a814bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9ad5ab640 1 -- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9a80b2ab0 msgr2=0x7fb9a81f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9ad5ab640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9a80b2ab0 0x7fb9a81f3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9ad5ab640 1 -- 172.21.15.38:0/691844374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a814a840 con 0x7fb9a8160ef0 2024-04-25T08:31:47.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9ae5ad640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9a814b7c0 0x7fb9a814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.394+0000 7fb9addac640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9a80b2ab0 0x7fb9a81f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.395+0000 7fb9ad5ab640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a8160ef0 0x7fb9a814a100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0097a0 tx=0x7fb99c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.395+0000 7fb9a67fc640 1 -- 172.21.15.38:0/691844374 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c037040 con 0x7fb9a8160ef0 2024-04-25T08:31:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.395+0000 7fb9a67fc640 1 -- 172.21.15.38:0/691844374 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c004030 con 0x7fb9a8160ef0 2024-04-25T08:31:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.395+0000 7fb9b0010640 1 -- 172.21.15.38:0/691844374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a814ab20 con 0x7fb9a8160ef0 2024-04-25T08:31:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.395+0000 7fb9a67fc640 1 -- 172.21.15.38:0/691844374 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c004420 con 0x7fb9a8160ef0 2024-04-25T08:31:47.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.395+0000 7fb9b0010640 1 -- 172.21.15.38:0/691844374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a814b0e0 con 0x7fb9a8160ef0 2024-04-25T08:31:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.396+0000 7fb9b0010640 1 -- 172.21.15.38:0/691844374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a804ef90 con 0x7fb9a8160ef0 2024-04-25T08:31:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.396+0000 7fb9a67fc640 1 -- 172.21.15.38:0/691844374 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb99c03a040 con 0x7fb9a8160ef0 2024-04-25T08:31:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.397+0000 7fb9a67fc640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb99406b500 0x7fb99406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.397+0000 7fb9a67fc640 1 -- 172.21.15.38:0/691844374 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb99c0aa890 con 0x7fb9a8160ef0 2024-04-25T08:31:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.397+0000 7fb9addac640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb99406b500 0x7fb99406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.397+0000 7fb9addac640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb99406b500 0x7fb99406d9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00017e0 tx=0x7fb9a00015b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:47.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.401+0000 7fb9a67fc640 1 -- 172.21.15.38:0/691844374 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb99c042020 con 0x7fb9a8160ef0 2024-04-25T08:31:47.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.600+0000 7fb9b0010640 1 -- 172.21.15.38:0/691844374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb9a8160460 con 0x7fb9a8160ef0 2024-04-25T08:31:47.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.600+0000 7fb9a67fc640 1 -- 172.21.15.38:0/691844374 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb99c0793e0 con 0x7fb9a8160ef0 2024-04-25T08:31:47.597 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:31:47.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.603+0000 7fb9b0010640 1 -- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb99406b500 msgr2=0x7fb99406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.603+0000 7fb9b0010640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb99406b500 0x7fb99406d9c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00017e0 tx=0x7fb9a00015b0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.603+0000 7fb9b0010640 1 -- 172.21.15.38:0/691844374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a8160ef0 msgr2=0x7fb9a814a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.603+0000 7fb9b0010640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a8160ef0 0x7fb9a814a100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0097a0 tx=0x7fb99c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.603+0000 7fb9b0010640 1 -- 172.21.15.38:0/691844374 shutdown_connections 2024-04-25T08:31:47.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.603+0000 7fb9b0010640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb99406b500 0x7fb99406d9c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.603+0000 7fb9b0010640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9a814b7c0 0x7fb9a814bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.603+0000 7fb9b0010640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a8160ef0 0x7fb9a814a100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.603+0000 7fb9b0010640 1 --2- 172.21.15.38:0/691844374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9a80b2ab0 0x7fb9a81f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.603+0000 7fb9b0010640 1 -- 172.21.15.38:0/691844374 >> 172.21.15.38:0/691844374 conn(0x7fb9a8066d10 msgr2=0x7fb9a80ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:47.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.603+0000 7fb9b0010640 1 -- 172.21.15.38:0/691844374 shutdown_connections 2024-04-25T08:31:47.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.603+0000 7fb9b0010640 1 -- 172.21.15.38:0/691844374 wait complete. 2024-04-25T08:31:47.610 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:31:47.610 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:31:47.625 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:47.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.770+0000 7f88b5bb7640 1 Processor -- start 2024-04-25T08:31:47.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.770+0000 7f88b5bb7640 1 -- start start 2024-04-25T08:31:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88b5bb7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88b00d3900 0x7f88b00d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88b5bb7640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f88b00d0d90 0x7f88b00b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88b5bb7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b00d3e70 con 0x7f88b015dc10 2024-04-25T08:31:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88b5bb7640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b015dff0 con 0x7f88b00d0d90 2024-04-25T08:31:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88b5bb7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00b6990 con 0x7f88b00d3900 2024-04-25T08:31:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88af7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f88b015dc10 0x7f88b00ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51882/0 (socket says 172.21.15.38:51882) 2024-04-25T08:31:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88af7fe640 1 -- 172.21.15.38:0/1786243713 learned_addr learned my addr 172.21.15.38:0/1786243713 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88affff640 1 --2- 172.21.15.38:0/1786243713 >> v2:172.21.15.107:3300/0 conn(0x7f88b00d0d90 0x7f88b00b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88aeffd640 1 --2- 172.21.15.38:0/1786243713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88b00d3900 0x7f88b00d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88ae7fc640 1 -- 172.21.15.38:0/1786243713 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296293887 0 0) 0x7f88b00d3e70 con 0x7f88b015dc10 2024-04-25T08:31:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88ae7fc640 1 -- 172.21.15.38:0/1786243713 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8898003520 con 0x7f88b015dc10 2024-04-25T08:31:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88affff640 1 -- 172.21.15.38:0/1786243713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88b00d3900 msgr2=0x7f88b00d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88affff640 1 --2- 172.21.15.38:0/1786243713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88b00d3900 0x7f88b00d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88affff640 1 -- 172.21.15.38:0/1786243713 >> v1:172.21.15.38:6789/0 conn(0x7f88b015dc10 legacy=0x7f88b00ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.771+0000 7f88affff640 1 -- 172.21.15.38:0/1786243713 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b00d4150 con 0x7f88b00d0d90 2024-04-25T08:31:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.772+0000 7f88affff640 1 --2- 172.21.15.38:0/1786243713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b00d0d90 0x7f88b00b2370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009d90 tx=0x7f88a802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb8d8bcd857ae63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.772+0000 7f88ae7fc640 1 -- 172.21.15.38:0/1786243713 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8031600 con 0x7f88b00d0d90 2024-04-25T08:31:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.772+0000 7f88ae7fc640 1 -- 172.21.15.38:0/1786243713 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a80317a0 con 0x7f88b00d0d90 2024-04-25T08:31:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.772+0000 7f88ae7fc640 1 -- 172.21.15.38:0/1786243713 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8031b70 con 0x7f88b00d0d90 2024-04-25T08:31:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.773+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1786243713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b00d0d90 msgr2=0x7f88b00b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.773+0000 7f88b5bb7640 1 --2- 172.21.15.38:0/1786243713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b00d0d90 0x7f88b00b2370 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009d90 tx=0x7f88a802a5d0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.773+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1786243713 shutdown_connections 2024-04-25T08:31:47.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.773+0000 7f88b5bb7640 1 --2- 172.21.15.38:0/1786243713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b00d0d90 0x7f88b00b2370 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.773+0000 7f88b5bb7640 1 --2- 172.21.15.38:0/1786243713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88b00d3900 0x7f88b00d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.773+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1786243713 >> 172.21.15.38:0/1786243713 conn(0x7f88b0066d10 msgr2=0x7f88b0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:47.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.773+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1786243713 shutdown_connections 2024-04-25T08:31:47.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.773+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1786243713 wait complete. 2024-04-25T08:31:47.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.774+0000 7f88b5bb7640 1 Processor -- start 2024-04-25T08:31:47.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.774+0000 7f88b5bb7640 1 -- start start 2024-04-25T08:31:47.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.775+0000 7f88b5bb7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b00d0d90 0x7f88b00c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.775+0000 7f88b5bb7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88b00d3900 0x7f88b00c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.775+0000 7f88b5bb7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b015dc10 0x7f88b00cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.775+0000 7f88b5bb7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00b6bc0 con 0x7f88b015dc10 2024-04-25T08:31:47.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.775+0000 7f88b5bb7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00d4710 con 0x7f88b00d0d90 2024-04-25T08:31:47.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.775+0000 7f88b5bb7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00cef30 con 0x7f88b00d3900 2024-04-25T08:31:47.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.775+0000 7f88affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b015dc10 0x7f88b00cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.775+0000 7f88affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b015dc10 0x7f88b00cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40216/0 (socket says 172.21.15.38:40216) 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.775+0000 7f88affff640 1 -- 172.21.15.38:0/1506981521 learned_addr learned my addr 172.21.15.38:0/1506981521 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.775+0000 7f88aeffd640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88b00d3900 0x7f88b00c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.775+0000 7f88af7fe640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b00d0d90 0x7f88b00c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.776+0000 7f88affff640 1 -- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88b00d3900 msgr2=0x7f88b00c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.776+0000 7f88affff640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88b00d3900 0x7f88b00c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.776+0000 7f88affff640 1 -- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b00d0d90 msgr2=0x7f88b00c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.776+0000 7f88affff640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b00d0d90 0x7f88b00c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.776+0000 7f88affff640 1 -- 172.21.15.38:0/1506981521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b00ca1d0 con 0x7f88b015dc10 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.776+0000 7f88affff640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b015dc10 0x7f88b00cb5f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009eb0 tx=0x7f88a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.776+0000 7f8897fff640 1 -- 172.21.15.38:0/1506981521 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8040070 con 0x7f88b015dc10 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.776+0000 7f8897fff640 1 -- 172.21.15.38:0/1506981521 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a803c830 con 0x7f88b015dc10 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.776+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1506981521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b00ca4b0 con 0x7f88b015dc10 2024-04-25T08:31:47.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.776+0000 7f8897fff640 1 -- 172.21.15.38:0/1506981521 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a803cbe0 con 0x7f88b015dc10 2024-04-25T08:31:47.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.776+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1506981521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b0151ef0 con 0x7f88b015dc10 2024-04-25T08:31:47.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.777+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1506981521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b0068e80 con 0x7f88b015dc10 2024-04-25T08:31:47.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.777+0000 7f8897fff640 1 -- 172.21.15.38:0/1506981521 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88a8039040 con 0x7f88b015dc10 2024-04-25T08:31:47.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.778+0000 7f8897fff640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f889c06b4d0 0x7f889c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:47.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.778+0000 7f8897fff640 1 -- 172.21.15.38:0/1506981521 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f88a80abaa0 con 0x7f88b015dc10 2024-04-25T08:31:47.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.778+0000 7f88af7fe640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f889c06b4d0 0x7f889c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:47.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.779+0000 7f88af7fe640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f889c06b4d0 0x7f889c06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f88a0005880 tx=0x7f88a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:47.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.782+0000 7f8897fff640 1 -- 172.21.15.38:0/1506981521 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88a807a0b0 con 0x7f88b015dc10 2024-04-25T08:31:47.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.966+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1506981521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88b00ca680 con 0x7f88b015dc10 2024-04-25T08:31:47.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.967+0000 7f8897fff640 1 -- 172.21.15.38:0/1506981521 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f88a807e150 con 0x7f88b015dc10 2024-04-25T08:31:47.964 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:47.964 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:47.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.969+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f889c06b4d0 msgr2=0x7f889c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.969+0000 7f88b5bb7640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f889c06b4d0 0x7f889c06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f88a0005880 tx=0x7f88a0009040 comp rx=0 tx=0).stop 2024-04-25T08:31:47.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.969+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1506981521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b015dc10 msgr2=0x7f88b00cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:47.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.969+0000 7f88b5bb7640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b015dc10 0x7f88b00cb5f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f88a8009eb0 tx=0x7f88a8004a40 comp rx=0 tx=0).stop 2024-04-25T08:31:47.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.969+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1506981521 shutdown_connections 2024-04-25T08:31:47.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.969+0000 7f88b5bb7640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b015dc10 0x7f88b00cb5f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.969+0000 7f88b5bb7640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f889c06b4d0 0x7f889c06d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.969+0000 7f88b5bb7640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88b00d3900 0x7f88b00c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.969+0000 7f88b5bb7640 1 --2- 172.21.15.38:0/1506981521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88b00d0d90 0x7f88b00c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:47.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.969+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1506981521 >> 172.21.15.38:0/1506981521 conn(0x7f88b0066d10 msgr2=0x7f88b00b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:47.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.969+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1506981521 shutdown_connections 2024-04-25T08:31:47.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:47.969+0000 7f88b5bb7640 1 -- 172.21.15.38:0/1506981521 wait complete. 2024-04-25T08:31:47.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:31:47.977 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:31:48.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.085+0000 7f078babc640 1 Processor -- start 2024-04-25T08:31:48.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.085+0000 7f078babc640 1 -- start start 2024-04-25T08:31:48.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f078babc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840d0320 0x7f07840d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f078babc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07840d2060 con 0x7f07840d0700 2024-04-25T08:31:48.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f078babc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07840d2330 con 0x7f07840b6980 2024-04-25T08:31:48.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f078babc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07840d2610 con 0x7f07840d0320 2024-04-25T08:31:48.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f0789057640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f07840d0700 0x7f07840d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51896/0 (socket says 172.21.15.38:51896) 2024-04-25T08:31:48.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f0789057640 1 -- 172.21.15.38:0/850422453 learned_addr learned my addr 172.21.15.38:0/850422453 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:48.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f0789858640 1 --2- 172.21.15.38:0/850422453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840d0320 0x7f07840d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f0788856640 1 -- 172.21.15.38:0/850422453 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200022688 0 0) 0x7f07840d2060 con 0x7f07840d0700 2024-04-25T08:31:48.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f0788856640 1 -- 172.21.15.38:0/850422453 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0770003520 con 0x7f07840d0700 2024-04-25T08:31:48.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f0788856640 1 -- 172.21.15.38:0/850422453 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573784385 0 0) 0x7f07840d2330 con 0x7f07840b6980 2024-04-25T08:31:48.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f0788856640 1 -- 172.21.15.38:0/850422453 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07840d2060 con 0x7f07840b6980 2024-04-25T08:31:48.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f0788856640 1 -- 172.21.15.38:0/850422453 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2682369262 0 0) 0x7f0770003520 con 0x7f07840d0700 2024-04-25T08:31:48.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f0789858640 1 -- 172.21.15.38:0/850422453 >> v1:172.21.15.107:6789/0 conn(0x7f07840b6980 legacy=0x7f07840b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f0789858640 1 -- 172.21.15.38:0/850422453 >> v1:172.21.15.38:6789/0 conn(0x7f07840d0700 legacy=0x7f07840d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.086+0000 7f0789858640 1 -- 172.21.15.38:0/850422453 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07840d28f0 con 0x7f07840d0320 2024-04-25T08:31:48.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.087+0000 7f0789858640 1 --2- 172.21.15.38:0/850422453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840d0320 0x7f07840d33d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f07800097d0 tx=0x7f078002a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61bafab4ee58822a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:48.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.087+0000 7f0788856640 1 -- 172.21.15.38:0/850422453 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780037070 con 0x7f07840d0320 2024-04-25T08:31:48.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.087+0000 7f0788856640 1 -- 172.21.15.38:0/850422453 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078002acc0 con 0x7f07840d0320 2024-04-25T08:31:48.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.087+0000 7f0788856640 1 -- 172.21.15.38:0/850422453 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774001670 con 0x7f07840d0320 2024-04-25T08:31:48.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.088+0000 7f078babc640 1 -- 172.21.15.38:0/850422453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840d0320 msgr2=0x7f07840d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.088+0000 7f078babc640 1 --2- 172.21.15.38:0/850422453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840d0320 0x7f07840d33d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f07800097d0 tx=0x7f078002a230 comp rx=0 tx=0).stop 2024-04-25T08:31:48.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.088+0000 7f078babc640 1 -- 172.21.15.38:0/850422453 shutdown_connections 2024-04-25T08:31:48.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.088+0000 7f078babc640 1 --2- 172.21.15.38:0/850422453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840d0320 0x7f07840d33d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.088+0000 7f078babc640 1 -- 172.21.15.38:0/850422453 >> 172.21.15.38:0/850422453 conn(0x7f07840668f0 msgr2=0x7f0784066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:48.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.088+0000 7f078babc640 1 -- 172.21.15.38:0/850422453 shutdown_connections 2024-04-25T08:31:48.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.088+0000 7f078babc640 1 -- 172.21.15.38:0/850422453 wait complete. 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.089+0000 7f078babc640 1 Processor -- start 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.089+0000 7f078babc640 1 -- start start 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f078babc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07840b6980 0x7f0784157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f078babc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840d0320 0x7f07841596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f078babc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840d0700 0x7f078415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f078babc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07840d0b40 con 0x7f07840d0320 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f078babc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07840d2eb0 con 0x7f07840b6980 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f078babc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0784159250 con 0x7f07840d0700 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f0789057640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840d0320 0x7f07841596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f0789057640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840d0320 0x7f07841596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40228/0 (socket says 172.21.15.38:40228) 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f0789057640 1 -- 172.21.15.38:0/2243910573 learned_addr learned my addr 172.21.15.38:0/2243910573 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f0789858640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07840b6980 0x7f0784157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f078a059640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840d0700 0x7f078415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f0789057640 1 -- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840d0700 msgr2=0x7f078415a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f0789057640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840d0700 0x7f078415a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f0789057640 1 -- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07840b6980 msgr2=0x7f0784157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f0789057640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07840b6980 0x7f0784157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f0789057640 1 -- 172.21.15.38:0/2243910573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784158640 con 0x7f07840d0320 2024-04-25T08:31:48.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.090+0000 7f0789858640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07840b6980 0x7f0784157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:48.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.091+0000 7f0789057640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840d0320 0x7f07841596e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0774000fd0 tx=0x7f077400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:48.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.091+0000 7f077a7fc640 1 -- 172.21.15.38:0/2243910573 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774017070 con 0x7f07840d0320 2024-04-25T08:31:48.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.091+0000 7f077a7fc640 1 -- 172.21.15.38:0/2243910573 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0774005d00 con 0x7f07840d0320 2024-04-25T08:31:48.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.091+0000 7f077a7fc640 1 -- 172.21.15.38:0/2243910573 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07740072b0 con 0x7f07840d0320 2024-04-25T08:31:48.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.091+0000 7f078babc640 1 -- 172.21.15.38:0/2243910573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0784158920 con 0x7f07840d0320 2024-04-25T08:31:48.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.091+0000 7f078babc640 1 -- 172.21.15.38:0/2243910573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0784158ee0 con 0x7f07840d0320 2024-04-25T08:31:48.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.092+0000 7f078babc640 1 -- 172.21.15.38:0/2243910573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0784068e80 con 0x7f07840d0320 2024-04-25T08:31:48.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.092+0000 7f077a7fc640 1 -- 172.21.15.38:0/2243910573 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f077401a070 con 0x7f07840d0320 2024-04-25T08:31:48.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.093+0000 7f077a7fc640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f076806b4d0 0x7f076806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.093+0000 7f077a7fc640 1 -- 172.21.15.38:0/2243910573 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0774089910 con 0x7f07840d0320 2024-04-25T08:31:48.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.093+0000 7f0789858640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f076806b4d0 0x7f076806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.093+0000 7f0789858640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f076806b4d0 0x7f076806d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0780034000 tx=0x7f078002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:48.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.097+0000 7f077a7fc640 1 -- 172.21.15.38:0/2243910573 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0774057e10 con 0x7f07840d0320 2024-04-25T08:31:48.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.279+0000 7f078babc640 1 -- 172.21.15.38:0/2243910573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f078415d790 con 0x7f07840d0320 2024-04-25T08:31:48.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.279+0000 7f077a7fc640 1 -- 172.21.15.38:0/2243910573 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f077405beb0 con 0x7f07840d0320 2024-04-25T08:31:48.275 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:31:48.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.281+0000 7f078babc640 1 -- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f076806b4d0 msgr2=0x7f076806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.281+0000 7f078babc640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f076806b4d0 0x7f076806d990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0780034000 tx=0x7f078002e000 comp rx=0 tx=0).stop 2024-04-25T08:31:48.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.281+0000 7f078babc640 1 -- 172.21.15.38:0/2243910573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840d0320 msgr2=0x7f07841596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.281+0000 7f078babc640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840d0320 0x7f07841596e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0774000fd0 tx=0x7f077400c9b0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.281+0000 7f078babc640 1 -- 172.21.15.38:0/2243910573 shutdown_connections 2024-04-25T08:31:48.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.281+0000 7f078babc640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f076806b4d0 0x7f076806d990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.281+0000 7f078babc640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840d0700 0x7f078415a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.281+0000 7f078babc640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840d0320 0x7f07841596e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.281+0000 7f078babc640 1 --2- 172.21.15.38:0/2243910573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07840b6980 0x7f0784157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.281+0000 7f078babc640 1 -- 172.21.15.38:0/2243910573 >> 172.21.15.38:0/2243910573 conn(0x7f07840668f0 msgr2=0x7f07840ce610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.281+0000 7f078babc640 1 -- 172.21.15.38:0/2243910573 shutdown_connections 2024-04-25T08:31:48.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.281+0000 7f078babc640 1 -- 172.21.15.38:0/2243910573 wait complete. 2024-04-25T08:31:48.288 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:31:48.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.398+0000 7f5d12e76640 1 Processor -- start 2024-04-25T08:31:48.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.398+0000 7f5d12e76640 1 -- start start 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d12e76640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5d0c15b990 0x7f5d0c0cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d12e76640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5d0c15f5f0 0x7f5d0c0d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d12e76640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0c15fd00 con 0x7f5d0c0d1b50 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d12e76640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c0d1fa0 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d12e76640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c160580 con 0x7f5d0c15b990 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d0bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5d0c15f5f0 0x7f5d0c0d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d0bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5d0c15f5f0 0x7f5d0c0d1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40242/0 (socket says 172.21.15.38:40242) 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d0bfff640 1 -- 172.21.15.38:0/667972072 learned_addr learned my addr 172.21.15.38:0/667972072 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d10c12640 1 --2- 172.21.15.38:0/667972072 >> v2:172.21.15.107:3300/0 conn(0x7f5d0c15b990 0x7f5d0c0cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d0bfff640 1 -- 172.21.15.38:0/667972072 >> v2:172.21.15.107:3300/0 conn(0x7f5d0c15b990 msgr2=0x7f5d0c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d0bfff640 1 --2- 172.21.15.38:0/667972072 >> v2:172.21.15.107:3300/0 conn(0x7f5d0c15b990 0x7f5d0c0cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d0bfff640 1 -- 172.21.15.38:0/667972072 >> v1:172.21.15.107:6789/0 conn(0x7f5d0c0d1b50 legacy=0x7f5d0c0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.399+0000 7f5d0bfff640 1 -- 172.21.15.38:0/667972072 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c15fa30 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.400+0000 7f5d0bfff640 1 --2- 172.21.15.38:0/667972072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d0c15f5f0 0x7f5d0c0d1610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0d15e0 tx=0x7f5cf802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18efe81df920cde7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.400+0000 7f5d0b7fe640 1 -- 172.21.15.38:0/667972072 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8037070 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.400+0000 7f5d0b7fe640 1 -- 172.21.15.38:0/667972072 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf8032440 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.400+0000 7f5d0b7fe640 1 -- 172.21.15.38:0/667972072 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf8032810 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.401+0000 7f5d12e76640 1 -- 172.21.15.38:0/667972072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d0c15f5f0 msgr2=0x7f5d0c0d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.401+0000 7f5d12e76640 1 --2- 172.21.15.38:0/667972072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d0c15f5f0 0x7f5d0c0d1610 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0d15e0 tx=0x7f5cf802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.401+0000 7f5d12e76640 1 -- 172.21.15.38:0/667972072 shutdown_connections 2024-04-25T08:31:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.401+0000 7f5d12e76640 1 --2- 172.21.15.38:0/667972072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d0c15f5f0 0x7f5d0c0d1610 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.401+0000 7f5d12e76640 1 --2- 172.21.15.38:0/667972072 >> v2:172.21.15.107:3300/0 conn(0x7f5d0c15b990 0x7f5d0c0cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.401+0000 7f5d12e76640 1 -- 172.21.15.38:0/667972072 >> 172.21.15.38:0/667972072 conn(0x7f5d0c08b000 msgr2=0x7f5d0c08b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.401+0000 7f5d12e76640 1 -- 172.21.15.38:0/667972072 shutdown_connections 2024-04-25T08:31:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.401+0000 7f5d12e76640 1 -- 172.21.15.38:0/667972072 wait complete. 2024-04-25T08:31:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.402+0000 7f5d12e76640 1 Processor -- start 2024-04-25T08:31:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.402+0000 7f5d12e76640 1 -- start start 2024-04-25T08:31:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d12e76640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d0c0d1b50 0x7f5d0c15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d12e76640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d0c15b990 0x7f5d0c154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d12e76640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d0c15f5f0 0x7f5d0c0c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d12e76640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0b1ac0 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d12e76640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c1602a0 con 0x7f5d0c0d1b50 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d12e76640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c15bd70 con 0x7f5d0c15b990 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d11413640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d0c15f5f0 0x7f5d0c0c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d11413640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d0c15f5f0 0x7f5d0c0c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40258/0 (socket says 172.21.15.38:40258) 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d11413640 1 -- 172.21.15.38:0/3989970395 learned_addr learned my addr 172.21.15.38:0/3989970395 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d10c12640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d0c0d1b50 0x7f5d0c15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d0bfff640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d0c15b990 0x7f5d0c154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d11413640 1 -- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d0c15b990 msgr2=0x7f5d0c154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d11413640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d0c15b990 0x7f5d0c154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d11413640 1 -- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d0c0d1b50 msgr2=0x7f5d0c15eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d11413640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d0c0d1b50 0x7f5d0c15eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d11413640 1 -- 172.21.15.38:0/3989970395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c1555a0 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.403+0000 7f5d11413640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d0c15f5f0 0x7f5d0c0c7080 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400c7a0 tx=0x7f5d0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.404+0000 7f5d097fa640 1 -- 172.21.15.38:0/3989970395 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04003b70 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.404+0000 7f5d097fa640 1 -- 172.21.15.38:0/3989970395 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d04003d30 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.404+0000 7f5d12e76640 1 -- 172.21.15.38:0/3989970395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c155880 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.404+0000 7f5d097fa640 1 -- 172.21.15.38:0/3989970395 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0400aa90 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.404+0000 7f5d12e76640 1 -- 172.21.15.38:0/3989970395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c155e40 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.405+0000 7f5d12e76640 1 -- 172.21.15.38:0/3989970395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c068e80 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.405+0000 7f5d097fa640 1 -- 172.21.15.38:0/3989970395 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d0400ac30 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.406+0000 7f5d097fa640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5cf006b4d0 0x7f5cf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.406+0000 7f5d097fa640 1 -- 172.21.15.38:0/3989970395 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5d0408a350 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.406+0000 7f5d10c12640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5cf006b4d0 0x7f5cf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.406+0000 7f5d10c12640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5cf006b4d0 0x7f5cf006d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009b40 tx=0x7f5d00009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.410+0000 7f5d097fa640 1 -- 172.21.15.38:0/3989970395 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d04058850 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.591+0000 7f5d12e76640 1 -- 172.21.15.38:0/3989970395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d0c0b1220 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.592+0000 7f5d097fa640 1 -- 172.21.15.38:0/3989970395 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d0405c8f0 con 0x7f5d0c15f5f0 2024-04-25T08:31:48.588 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:31:48.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.594+0000 7f5d12e76640 1 -- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5cf006b4d0 msgr2=0x7f5cf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.594+0000 7f5d12e76640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5cf006b4d0 0x7f5cf006d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009b40 tx=0x7f5d00009190 comp rx=0 tx=0).stop 2024-04-25T08:31:48.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.594+0000 7f5d12e76640 1 -- 172.21.15.38:0/3989970395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d0c15f5f0 msgr2=0x7f5d0c0c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.594+0000 7f5d12e76640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d0c15f5f0 0x7f5d0c0c7080 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400c7a0 tx=0x7f5d0400cc60 comp rx=0 tx=0).stop 2024-04-25T08:31:48.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.594+0000 7f5d12e76640 1 -- 172.21.15.38:0/3989970395 shutdown_connections 2024-04-25T08:31:48.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.594+0000 7f5d12e76640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d0c15f5f0 0x7f5d0c0c7080 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.594+0000 7f5d12e76640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5cf006b4d0 0x7f5cf006d990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.594+0000 7f5d12e76640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d0c15b990 0x7f5d0c154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.594+0000 7f5d12e76640 1 --2- 172.21.15.38:0/3989970395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d0c0d1b50 0x7f5d0c15eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.594+0000 7f5d12e76640 1 -- 172.21.15.38:0/3989970395 >> 172.21.15.38:0/3989970395 conn(0x7f5d0c08b000 msgr2=0x7f5d0c0ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:48.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.594+0000 7f5d12e76640 1 -- 172.21.15.38:0/3989970395 shutdown_connections 2024-04-25T08:31:48.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.594+0000 7f5d12e76640 1 -- 172.21.15.38:0/3989970395 wait complete. 2024-04-25T08:31:48.602 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:31:48.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.710+0000 7f51ba8fc640 1 Processor -- start 2024-04-25T08:31:48.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.710+0000 7f51ba8fc640 1 -- start start 2024-04-25T08:31:48.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.711+0000 7f51ba8fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f51b40c92b0 0x7f51b4071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.711+0000 7f51ba8fc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b415e240 con 0x7f51b4161010 2024-04-25T08:31:48.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.711+0000 7f51ba8fc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b415e520 con 0x7f51b415f0b0 2024-04-25T08:31:48.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.711+0000 7f51ba8fc640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b40cd770 con 0x7f51b40c92b0 2024-04-25T08:31:48.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.711+0000 7f51b3fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f51b4161010 0x7f51b40ced60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51904/0 (socket says 172.21.15.38:51904) 2024-04-25T08:31:48.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.711+0000 7f51b8e99640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f51b40c92b0 0x7f51b4071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.711+0000 7f51b3fff640 1 -- 172.21.15.38:0/3237317563 learned_addr learned my addr 172.21.15.38:0/3237317563 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:48.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.711+0000 7f51b8e99640 1 -- 172.21.15.38:0/3237317563 >> v1:172.21.15.107:6789/0 conn(0x7f51b415f0b0 legacy=0x7f51b40c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.711+0000 7f51b8e99640 1 -- 172.21.15.38:0/3237317563 >> v1:172.21.15.38:6789/0 conn(0x7f51b4161010 legacy=0x7f51b40ced60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.711+0000 7f51b8e99640 1 -- 172.21.15.38:0/3237317563 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b415e800 con 0x7f51b40c92b0 2024-04-25T08:31:48.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.711+0000 7f51b2ffd640 1 -- 172.21.15.38:0/3237317563 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078615113 0 0) 0x7f51b415e240 con 0x7f51b4161010 2024-04-25T08:31:48.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.712+0000 7f51b8e99640 1 --2- 172.21.15.38:0/3237317563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51b40c92b0 0x7f51b4071670 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009d90 tx=0x7f51ac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19c0beed139d586 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:48.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.712+0000 7f51b2ffd640 1 -- 172.21.15.38:0/3237317563 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac007740 con 0x7f51b40c92b0 2024-04-25T08:31:48.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.712+0000 7f51b2ffd640 1 -- 172.21.15.38:0/3237317563 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac0078e0 con 0x7f51b40c92b0 2024-04-25T08:31:48.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.712+0000 7f51b2ffd640 1 -- 172.21.15.38:0/3237317563 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac007cb0 con 0x7f51b40c92b0 2024-04-25T08:31:48.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.713+0000 7f51ba8fc640 1 -- 172.21.15.38:0/3237317563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51b40c92b0 msgr2=0x7f51b4071670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.713+0000 7f51ba8fc640 1 --2- 172.21.15.38:0/3237317563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51b40c92b0 0x7f51b4071670 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f51ac009d90 tx=0x7f51ac02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.713+0000 7f51ba8fc640 1 -- 172.21.15.38:0/3237317563 shutdown_connections 2024-04-25T08:31:48.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.713+0000 7f51ba8fc640 1 --2- 172.21.15.38:0/3237317563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51b40c92b0 0x7f51b4071670 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.713+0000 7f51ba8fc640 1 -- 172.21.15.38:0/3237317563 >> 172.21.15.38:0/3237317563 conn(0x7f51b40cc2d0 msgr2=0x7f51b40cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:48.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.713+0000 7f51ba8fc640 1 -- 172.21.15.38:0/3237317563 shutdown_connections 2024-04-25T08:31:48.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.713+0000 7f51ba8fc640 1 -- 172.21.15.38:0/3237317563 wait complete. 2024-04-25T08:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.714+0000 7f51ba8fc640 1 Processor -- start 2024-04-25T08:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.714+0000 7f51ba8fc640 1 -- start start 2024-04-25T08:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51ba8fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51b40c92b0 0x7f51b40b5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51ba8fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51b415f0b0 0x7f51b413eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51ba8fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51b4161010 0x7f51b414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51ba8fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b415edc0 con 0x7f51b415f0b0 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51ba8fc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b415def0 con 0x7f51b4161010 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51ba8fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b40cf470 con 0x7f51b40c92b0 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51b37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51b415f0b0 0x7f51b413eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51b37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51b415f0b0 0x7f51b413eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40274/0 (socket says 172.21.15.38:40274) 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51b37fe640 1 -- 172.21.15.38:0/1989850261 learned_addr learned my addr 172.21.15.38:0/1989850261 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51b8e99640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51b4161010 0x7f51b414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51b3fff640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51b40c92b0 0x7f51b40b5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51b37fe640 1 -- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51b40c92b0 msgr2=0x7f51b40b5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51b37fe640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51b40c92b0 0x7f51b40b5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51b37fe640 1 -- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51b4161010 msgr2=0x7f51b414fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51b37fe640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51b4161010 0x7f51b414fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.715+0000 7f51b37fe640 1 -- 172.21.15.38:0/1989850261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b41400a0 con 0x7f51b415f0b0 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.716+0000 7f51b37fe640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51b415f0b0 0x7f51b413eb20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f51a400c450 tx=0x7f51a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.716+0000 7f51b0ff9640 1 -- 172.21.15.38:0/1989850261 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a40052d0 con 0x7f51b415f0b0 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.716+0000 7f51b0ff9640 1 -- 172.21.15.38:0/1989850261 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a4005470 con 0x7f51b415f0b0 2024-04-25T08:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.716+0000 7f51ba8fc640 1 -- 172.21.15.38:0/1989850261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b413f520 con 0x7f51b415f0b0 2024-04-25T08:31:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.716+0000 7f51b0ff9640 1 -- 172.21.15.38:0/1989850261 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4012870 con 0x7f51b415f0b0 2024-04-25T08:31:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.716+0000 7f51ba8fc640 1 -- 172.21.15.38:0/1989850261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b413fae0 con 0x7f51b415f0b0 2024-04-25T08:31:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.717+0000 7f51ba8fc640 1 -- 172.21.15.38:0/1989850261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b4068e80 con 0x7f51b415f0b0 2024-04-25T08:31:48.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.717+0000 7f51b0ff9640 1 -- 172.21.15.38:0/1989850261 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51a4012a10 con 0x7f51b415f0b0 2024-04-25T08:31:48.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.718+0000 7f51b0ff9640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f519406b4d0 0x7f519406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:48.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.718+0000 7f51b0ff9640 1 -- 172.21.15.38:0/1989850261 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f51a4089f60 con 0x7f51b415f0b0 2024-04-25T08:31:48.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.718+0000 7f51b3fff640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f519406b4d0 0x7f519406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:48.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.718+0000 7f51b3fff640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f519406b4d0 0x7f519406d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f519c000f60 tx=0x7f519c007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:48.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.722+0000 7f51b0ff9640 1 -- 172.21.15.38:0/1989850261 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51a40584e0 con 0x7f51b415f0b0 2024-04-25T08:31:48.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.902+0000 7f51ba8fc640 1 -- 172.21.15.38:0/1989850261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f51b4073b40 con 0x7f51b415f0b0 2024-04-25T08:31:48.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.902+0000 7f51b0ff9640 1 -- 172.21.15.38:0/1989850261 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51a405c580 con 0x7f51b415f0b0 2024-04-25T08:31:48.899 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:31:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.904+0000 7f51ba8fc640 1 -- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f519406b4d0 msgr2=0x7f519406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.904+0000 7f51ba8fc640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f519406b4d0 0x7f519406d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f519c000f60 tx=0x7f519c007e40 comp rx=0 tx=0).stop 2024-04-25T08:31:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.904+0000 7f51ba8fc640 1 -- 172.21.15.38:0/1989850261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51b415f0b0 msgr2=0x7f51b413eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.904+0000 7f51ba8fc640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51b415f0b0 0x7f51b413eb20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f51a400c450 tx=0x7f51a400c910 comp rx=0 tx=0).stop 2024-04-25T08:31:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.904+0000 7f51ba8fc640 1 -- 172.21.15.38:0/1989850261 shutdown_connections 2024-04-25T08:31:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.904+0000 7f51ba8fc640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51b4161010 0x7f51b414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.904+0000 7f51ba8fc640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51b415f0b0 0x7f51b413eb20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.904+0000 7f51ba8fc640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f519406b4d0 0x7f519406d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.904+0000 7f51ba8fc640 1 --2- 172.21.15.38:0/1989850261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51b40c92b0 0x7f51b40b5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.904+0000 7f51ba8fc640 1 -- 172.21.15.38:0/1989850261 >> 172.21.15.38:0/1989850261 conn(0x7f51b40cc2d0 msgr2=0x7f51b406e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.904+0000 7f51ba8fc640 1 -- 172.21.15.38:0/1989850261 shutdown_connections 2024-04-25T08:31:48.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:48.904+0000 7f51ba8fc640 1 -- 172.21.15.38:0/1989850261 wait complete. 2024-04-25T08:31:48.911 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:31:49.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.018+0000 7fd8f5abd640 1 Processor -- start 2024-04-25T08:31:49.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.019+0000 7fd8f5abd640 1 -- start start 2024-04-25T08:31:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.019+0000 7fd8f5abd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd8f00d00e0 0x7fd8f00cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.019+0000 7fd8f5abd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd8f00cd3b0 0x7fd8f00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.019+0000 7fd8f5abd640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f00cdd80 con 0x7fd8f00b1d60 2024-04-25T08:31:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.019+0000 7fd8f5abd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f006abe0 con 0x7fd8f00d00e0 2024-04-25T08:31:49.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.019+0000 7fd8f5abd640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f006b480 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.019+0000 7fd8eeffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd8f00d00e0 0x7fd8f00cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.019+0000 7fd8eeffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd8f00d00e0 0x7fd8f00cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33116/0 (socket says 172.21.15.38:33116) 2024-04-25T08:31:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.019+0000 7fd8eeffd640 1 -- 172.21.15.38:0/585595209 learned_addr learned my addr 172.21.15.38:0/585595209 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.020+0000 7fd8effff640 1 --2- 172.21.15.38:0/585595209 >> v2:172.21.15.107:3300/0 conn(0x7fd8f00cd3b0 0x7fd8f00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.020+0000 7fd8eeffd640 1 -- 172.21.15.38:0/585595209 >> v2:172.21.15.107:3300/0 conn(0x7fd8f00cd3b0 msgr2=0x7fd8f00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.020+0000 7fd8eeffd640 1 --2- 172.21.15.38:0/585595209 >> v2:172.21.15.107:3300/0 conn(0x7fd8f00cd3b0 0x7fd8f00cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.020+0000 7fd8eeffd640 1 -- 172.21.15.38:0/585595209 >> v1:172.21.15.38:6789/0 conn(0x7fd8f00b1d60 legacy=0x7fd8f00cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.020+0000 7fd8eeffd640 1 -- 172.21.15.38:0/585595209 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f015ef40 con 0x7fd8f00d00e0 2024-04-25T08:31:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.020+0000 7fd8eeffd640 1 --2- 172.21.15.38:0/585595209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8f00d00e0 0x7fd8f00cce70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd8f01627d0 tx=0x7fd8e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78bf6724d94f020 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:49.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.021+0000 7fd8ee7fc640 1 -- 172.21.15.38:0/585595209 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4037070 con 0x7fd8f00d00e0 2024-04-25T08:31:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.021+0000 7fd8ee7fc640 1 -- 172.21.15.38:0/585595209 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e4032440 con 0x7fd8f00d00e0 2024-04-25T08:31:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.021+0000 7fd8ee7fc640 1 -- 172.21.15.38:0/585595209 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4032810 con 0x7fd8f00d00e0 2024-04-25T08:31:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.021+0000 7fd8f5abd640 1 -- 172.21.15.38:0/585595209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8f00d00e0 msgr2=0x7fd8f00cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.021+0000 7fd8f5abd640 1 --2- 172.21.15.38:0/585595209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8f00d00e0 0x7fd8f00cce70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd8f01627d0 tx=0x7fd8e402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.021+0000 7fd8f5abd640 1 -- 172.21.15.38:0/585595209 shutdown_connections 2024-04-25T08:31:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.021+0000 7fd8f5abd640 1 --2- 172.21.15.38:0/585595209 >> v2:172.21.15.107:3300/0 conn(0x7fd8f00cd3b0 0x7fd8f00cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.021+0000 7fd8f5abd640 1 --2- 172.21.15.38:0/585595209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8f00d00e0 0x7fd8f00cce70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.021+0000 7fd8f5abd640 1 -- 172.21.15.38:0/585595209 >> 172.21.15.38:0/585595209 conn(0x7fd8f00d06b0 msgr2=0x7fd8f00d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.021+0000 7fd8f5abd640 1 -- 172.21.15.38:0/585595209 shutdown_connections 2024-04-25T08:31:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.021+0000 7fd8f5abd640 1 -- 172.21.15.38:0/585595209 wait complete. 2024-04-25T08:31:49.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.022+0000 7fd8f5abd640 1 Processor -- start 2024-04-25T08:31:49.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.023+0000 7fd8f5abd640 1 -- start start 2024-04-25T08:31:49.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.023+0000 7fd8f5abd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f00b1d60 0x7fd8f0159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.023+0000 7fd8f5abd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8f00cd3b0 0x7fd8f0157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.023+0000 7fd8f5abd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8f00d00e0 0x7fd8f015db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.023+0000 7fd8f5abd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f006b910 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.023+0000 7fd8f5abd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f006b110 con 0x7fd8f00b1d60 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.023+0000 7fd8f5abd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f006ba90 con 0x7fd8f00d00e0 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.023+0000 7fd8eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8f00cd3b0 0x7fd8f0157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.023+0000 7fd8eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8f00cd3b0 0x7fd8f0157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33126/0 (socket says 172.21.15.38:33126) 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.023+0000 7fd8eeffd640 1 -- 172.21.15.38:0/2312615837 learned_addr learned my addr 172.21.15.38:0/2312615837 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8effff640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8f00d00e0 0x7fd8f015db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8ef7fe640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f00b1d60 0x7fd8f0159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8eeffd640 1 -- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8f00d00e0 msgr2=0x7fd8f015db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8eeffd640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8f00d00e0 0x7fd8f015db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8eeffd640 1 -- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f00b1d60 msgr2=0x7fd8f0159800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8eeffd640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f00b1d60 0x7fd8f0159800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8eeffd640 1 -- 172.21.15.38:0/2312615837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f0159d40 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8ef7fe640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f00b1d60 0x7fd8f0159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8eeffd640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8f00cd3b0 0x7fd8f0157ce0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40090b0 tx=0x7fd8e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:49.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8d7fff640 1 -- 172.21.15.38:0/2312615837 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4041070 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8d7fff640 1 -- 172.21.15.38:0/2312615837 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e403c870 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8d7fff640 1 -- 172.21.15.38:0/2312615837 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4037040 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8f5abd640 1 -- 172.21.15.38:0/2312615837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f0158760 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.024+0000 7fd8f5abd640 1 -- 172.21.15.38:0/2312615837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f0158d20 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.025+0000 7fd8f5abd640 1 -- 172.21.15.38:0/2312615837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f0068e80 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.026+0000 7fd8d7fff640 1 -- 172.21.15.38:0/2312615837 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8e4044070 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.026+0000 7fd8d7fff640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd8dc06b4d0 0x7fd8dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.026+0000 7fd8d7fff640 1 -- 172.21.15.38:0/2312615837 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd8e40ab2b0 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.026+0000 7fd8ef7fe640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd8dc06b4d0 0x7fd8dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.027+0000 7fd8ef7fe640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd8dc06b4d0 0x7fd8dc06d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00015e0 tx=0x7fd8e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:49.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.030+0000 7fd8d7fff640 1 -- 172.21.15.38:0/2312615837 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8e4079fa0 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.209+0000 7fd8f5abd640 1 -- 172.21.15.38:0/2312615837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd8f0161440 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.209+0000 7fd8d7fff640 1 -- 172.21.15.38:0/2312615837 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd8e40798a0 con 0x7fd8f00cd3b0 2024-04-25T08:31:49.206 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:31:49.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.211+0000 7fd8f5abd640 1 -- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd8dc06b4d0 msgr2=0x7fd8dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.211+0000 7fd8f5abd640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd8dc06b4d0 0x7fd8dc06d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00015e0 tx=0x7fd8e0009040 comp rx=0 tx=0).stop 2024-04-25T08:31:49.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.211+0000 7fd8f5abd640 1 -- 172.21.15.38:0/2312615837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8f00cd3b0 msgr2=0x7fd8f0157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.211+0000 7fd8f5abd640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8f00cd3b0 0x7fd8f0157ce0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd8e40090b0 tx=0x7fd8e40046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.211+0000 7fd8f5abd640 1 -- 172.21.15.38:0/2312615837 shutdown_connections 2024-04-25T08:31:49.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.211+0000 7fd8f5abd640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd8dc06b4d0 0x7fd8dc06d990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.211+0000 7fd8f5abd640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd8f00d00e0 0x7fd8f015db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.211+0000 7fd8f5abd640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd8f00cd3b0 0x7fd8f0157ce0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.211+0000 7fd8f5abd640 1 --2- 172.21.15.38:0/2312615837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd8f00b1d60 0x7fd8f0159800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.211+0000 7fd8f5abd640 1 -- 172.21.15.38:0/2312615837 >> 172.21.15.38:0/2312615837 conn(0x7fd8f00d06b0 msgr2=0x7fd8f006f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:49.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.211+0000 7fd8f5abd640 1 -- 172.21.15.38:0/2312615837 shutdown_connections 2024-04-25T08:31:49.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.211+0000 7fd8f5abd640 1 -- 172.21.15.38:0/2312615837 wait complete. 2024-04-25T08:31:49.218 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:31:49.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.323+0000 7f2deb96a640 1 Processor -- start 2024-04-25T08:31:49.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.323+0000 7f2deb96a640 1 -- start start 2024-04-25T08:31:49.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2deb96a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2de415db80 0x7f2de41616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2deb96a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2de40d0830 0x7f2de40d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2deb96a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de4162030 con 0x7f2de40b9600 2024-04-25T08:31:49.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2deb96a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de41625f0 con 0x7f2de40d0830 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2deb96a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de4162310 con 0x7f2de415db80 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2de9f07640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2de40d0830 0x7f2de40d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2de9f07640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2de40d0830 0x7f2de40d22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33140/0 (socket says 172.21.15.38:33140) 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2de9f07640 1 -- 172.21.15.38:0/497036824 learned_addr learned my addr 172.21.15.38:0/497036824 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2de9706640 1 --2- 172.21.15.38:0/497036824 >> v2:172.21.15.107:3300/0 conn(0x7f2de415db80 0x7f2de41616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2de9f07640 1 -- 172.21.15.38:0/497036824 >> v2:172.21.15.107:3300/0 conn(0x7f2de415db80 msgr2=0x7f2de41616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2de9f07640 1 --2- 172.21.15.38:0/497036824 >> v2:172.21.15.107:3300/0 conn(0x7f2de415db80 0x7f2de41616e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2de9f07640 1 -- 172.21.15.38:0/497036824 >> v1:172.21.15.107:6789/0 conn(0x7f2de40b9600 legacy=0x7f2de40d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.324+0000 7f2de9f07640 1 -- 172.21.15.38:0/497036824 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de40d1ab0 con 0x7f2de40d0830 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.325+0000 7f2de9f07640 1 --2- 172.21.15.38:0/497036824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de40d0830 0x7f2de40d22b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2de40ce0c0 tx=0x7f2ddc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcdc7c116b589ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.325+0000 7f2ddbfff640 1 -- 172.21.15.38:0/497036824 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc036070 con 0x7f2de40d0830 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.325+0000 7f2ddbfff640 1 -- 172.21.15.38:0/497036824 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc031600 con 0x7f2de40d0830 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.325+0000 7f2ddbfff640 1 -- 172.21.15.38:0/497036824 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc0319d0 con 0x7f2de40d0830 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.326+0000 7f2deb96a640 1 -- 172.21.15.38:0/497036824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de40d0830 msgr2=0x7f2de40d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.326+0000 7f2deb96a640 1 --2- 172.21.15.38:0/497036824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de40d0830 0x7f2de40d22b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2de40ce0c0 tx=0x7f2ddc02a660 comp rx=0 tx=0).stop 2024-04-25T08:31:49.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.326+0000 7f2deb96a640 1 -- 172.21.15.38:0/497036824 shutdown_connections 2024-04-25T08:31:49.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.326+0000 7f2deb96a640 1 --2- 172.21.15.38:0/497036824 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de40d0830 0x7f2de40d22b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.326+0000 7f2deb96a640 1 --2- 172.21.15.38:0/497036824 >> v2:172.21.15.107:3300/0 conn(0x7f2de415db80 0x7f2de41616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.326+0000 7f2deb96a640 1 -- 172.21.15.38:0/497036824 >> 172.21.15.38:0/497036824 conn(0x7f2de40668f0 msgr2=0x7f2de4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:49.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.326+0000 7f2deb96a640 1 -- 172.21.15.38:0/497036824 shutdown_connections 2024-04-25T08:31:49.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.326+0000 7f2deb96a640 1 -- 172.21.15.38:0/497036824 wait complete. 2024-04-25T08:31:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.327+0000 7f2deb96a640 1 Processor -- start 2024-04-25T08:31:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.327+0000 7f2deb96a640 1 -- start start 2024-04-25T08:31:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2deb96a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2de40b9600 0x7f2de4155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2deb96a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2de40d0830 0x7f2de4154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2deb96a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de415db80 0x7f2de40c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2deb96a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4161da0 con 0x7f2de415db80 2024-04-25T08:31:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2deb96a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4161c20 con 0x7f2de40d0830 2024-04-25T08:31:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2deb96a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40d27f0 con 0x7f2de40b9600 2024-04-25T08:31:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2de9f07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de415db80 0x7f2de40c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2de9f07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de415db80 0x7f2de40c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33154/0 (socket says 172.21.15.38:33154) 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2de9f07640 1 -- 172.21.15.38:0/2592519981 learned_addr learned my addr 172.21.15.38:0/2592519981 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2de9706640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2de40b9600 0x7f2de4155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2de8f05640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2de40d0830 0x7f2de4154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2de9f07640 1 -- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2de40b9600 msgr2=0x7f2de4155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2de9f07640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2de40b9600 0x7f2de4155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2de9f07640 1 -- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2de40d0830 msgr2=0x7f2de4154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2de9f07640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2de40d0830 0x7f2de4154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2de9f07640 1 -- 172.21.15.38:0/2592519981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de4156040 con 0x7f2de415db80 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2de8f05640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2de40d0830 0x7f2de4154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2de9f07640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de415db80 0x7f2de40c9390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc02ac50 tx=0x7f2ddc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2dd9ffb640 1 -- 172.21.15.38:0/2592519981 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc036050 con 0x7f2de415db80 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2dd9ffb640 1 -- 172.21.15.38:0/2592519981 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc042070 con 0x7f2de415db80 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2deb96a640 1 -- 172.21.15.38:0/2592519981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de40c9dc0 con 0x7f2de415db80 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.328+0000 7f2dd9ffb640 1 -- 172.21.15.38:0/2592519981 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc009510 con 0x7f2de415db80 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.329+0000 7f2deb96a640 1 -- 172.21.15.38:0/2592519981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de40cab90 con 0x7f2de415db80 2024-04-25T08:31:49.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.330+0000 7f2deb96a640 1 -- 172.21.15.38:0/2592519981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de404ef90 con 0x7f2de415db80 2024-04-25T08:31:49.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.330+0000 7f2dd9ffb640 1 -- 172.21.15.38:0/2592519981 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ddc007bf0 con 0x7f2de415db80 2024-04-25T08:31:49.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.331+0000 7f2dd9ffb640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2dc806b4d0 0x7f2dc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.331+0000 7f2dd9ffb640 1 -- 172.21.15.38:0/2592519981 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2ddc0ab4d0 con 0x7f2de415db80 2024-04-25T08:31:49.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.331+0000 7f2de9706640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2dc806b4d0 0x7f2dc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.331+0000 7f2de9706640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2dc806b4d0 0x7f2dc806d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006fd0 tx=0x7f2de0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:49.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.335+0000 7f2dd9ffb640 1 -- 172.21.15.38:0/2592519981 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ddc045360 con 0x7f2de415db80 2024-04-25T08:31:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.531+0000 7f2deb96a640 1 -- 172.21.15.38:0/2592519981 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2de415d1c0 con 0x7f2de415db80 2024-04-25T08:31:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.531+0000 7f2dd9ffb640 1 -- 172.21.15.38:0/2592519981 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2ddc079f70 con 0x7f2de415db80 2024-04-25T08:31:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:31:49.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.533+0000 7f2deb96a640 1 -- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2dc806b4d0 msgr2=0x7f2dc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.533+0000 7f2deb96a640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2dc806b4d0 0x7f2dc806d990 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2de0006fd0 tx=0x7f2de0008040 comp rx=0 tx=0).stop 2024-04-25T08:31:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.533+0000 7f2deb96a640 1 -- 172.21.15.38:0/2592519981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de415db80 msgr2=0x7f2de40c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.533+0000 7f2deb96a640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de415db80 0x7f2de40c9390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc02ac50 tx=0x7f2ddc004a40 comp rx=0 tx=0).stop 2024-04-25T08:31:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.533+0000 7f2deb96a640 1 -- 172.21.15.38:0/2592519981 shutdown_connections 2024-04-25T08:31:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.533+0000 7f2deb96a640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2de415db80 0x7f2de40c9390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.533+0000 7f2deb96a640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2de40d0830 0x7f2de4154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.533+0000 7f2deb96a640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2dc806b4d0 0x7f2dc806d990 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.533+0000 7f2deb96a640 1 --2- 172.21.15.38:0/2592519981 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2de40b9600 0x7f2de4155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.533+0000 7f2deb96a640 1 -- 172.21.15.38:0/2592519981 >> 172.21.15.38:0/2592519981 conn(0x7f2de40668f0 msgr2=0x7f2de40b4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.533+0000 7f2deb96a640 1 -- 172.21.15.38:0/2592519981 shutdown_connections 2024-04-25T08:31:49.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.533+0000 7f2deb96a640 1 -- 172.21.15.38:0/2592519981 wait complete. 2024-04-25T08:31:49.540 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:31:49.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:49.540 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:31:49.562 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:ceph-fuse[41793]: fuse finished with error 0 and tester_r 0 2024-04-25T08:31:49.565 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:31:49.589 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:31:49.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:31:49.590 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T08:31:49.658 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:49.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.803+0000 7f7996990640 1 Processor -- start 2024-04-25T08:31:49.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.803+0000 7f7996990640 1 -- start start 2024-04-25T08:31:49.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.804+0000 7f7996990640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f79900d0f60 0x7f79900b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.804+0000 7f7996990640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79900d3e70 con 0x7f799015dc10 2024-04-25T08:31:49.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.804+0000 7f7996990640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79900d4150 con 0x7f79900d3900 2024-04-25T08:31:49.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.804+0000 7f7996990640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f799015dff0 con 0x7f79900d0f60 2024-04-25T08:31:49.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.804+0000 7f7994f2d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f79900d0f60 0x7f79900b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.804+0000 7f798ffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f799015dc10 0x7f79900ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49726/0 (socket says 172.21.15.38:49726) 2024-04-25T08:31:49.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.804+0000 7f798ffff640 1 -- 172.21.15.38:0/3255641160 learned_addr learned my addr 172.21.15.38:0/3255641160 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:49.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.804+0000 7f7994f2d640 1 -- 172.21.15.38:0/3255641160 >> v1:172.21.15.107:6789/0 conn(0x7f79900d3900 legacy=0x7f79900d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:49.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.804+0000 7f7994f2d640 1 -- 172.21.15.38:0/3255641160 >> v1:172.21.15.38:6789/0 conn(0x7f799015dc10 legacy=0x7f79900ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.804+0000 7f798effd640 1 -- 172.21.15.38:0/3255641160 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1574917390 0 0) 0x7f79900d3e70 con 0x7f799015dc10 2024-04-25T08:31:49.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.804+0000 7f7994f2d640 1 -- 172.21.15.38:0/3255641160 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79900d4430 con 0x7f79900d0f60 2024-04-25T08:31:49.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.805+0000 7f7994f2d640 1 --2- 172.21.15.38:0/3255641160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79900d0f60 0x7f79900b2370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7988009d90 tx=0x7f798802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd59d523b43d146 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:49.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.805+0000 7f798effd640 1 -- 172.21.15.38:0/3255641160 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988007740 con 0x7f79900d0f60 2024-04-25T08:31:49.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.805+0000 7f798effd640 1 -- 172.21.15.38:0/3255641160 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79880078e0 con 0x7f79900d0f60 2024-04-25T08:31:49.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.805+0000 7f798effd640 1 -- 172.21.15.38:0/3255641160 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988007cb0 con 0x7f79900d0f60 2024-04-25T08:31:49.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.806+0000 7f7996990640 1 -- 172.21.15.38:0/3255641160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79900d0f60 msgr2=0x7f79900b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.806+0000 7f7996990640 1 --2- 172.21.15.38:0/3255641160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79900d0f60 0x7f79900b2370 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7988009d90 tx=0x7f798802a660 comp rx=0 tx=0).stop 2024-04-25T08:31:49.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.806+0000 7f7996990640 1 -- 172.21.15.38:0/3255641160 shutdown_connections 2024-04-25T08:31:49.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.806+0000 7f7996990640 1 --2- 172.21.15.38:0/3255641160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79900d0f60 0x7f79900b2370 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.806+0000 7f7996990640 1 -- 172.21.15.38:0/3255641160 >> 172.21.15.38:0/3255641160 conn(0x7f7990066d10 msgr2=0x7f7990067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:49.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.806+0000 7f7996990640 1 -- 172.21.15.38:0/3255641160 shutdown_connections 2024-04-25T08:31:49.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.806+0000 7f7996990640 1 -- 172.21.15.38:0/3255641160 wait complete. 2024-04-25T08:31:49.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.807+0000 7f7996990640 1 Processor -- start 2024-04-25T08:31:49.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.807+0000 7f7996990640 1 -- start start 2024-04-25T08:31:49.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7996990640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900d0f60 0x7f799015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7996990640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79900d3900 0x7f79900aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7996990640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799015dc10 0x7f79900abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7996990640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79900d4710 con 0x7f799015dc10 2024-04-25T08:31:49.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7996990640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79900cef30 con 0x7f79900d3900 2024-04-25T08:31:49.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7996990640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79900b80f0 con 0x7f79900d0f60 2024-04-25T08:31:49.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7994f2d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799015dc10 0x7f79900abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7994f2d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799015dc10 0x7f79900abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33172/0 (socket says 172.21.15.38:33172) 2024-04-25T08:31:49.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7994f2d640 1 -- 172.21.15.38:0/3357647737 learned_addr learned my addr 172.21.15.38:0/3357647737 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:49.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f798ffff640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900d0f60 0x7f799015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f798f7fe640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79900d3900 0x7f79900aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7994f2d640 1 -- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900d0f60 msgr2=0x7f799015bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7994f2d640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900d0f60 0x7f799015bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7994f2d640 1 -- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79900d3900 msgr2=0x7f79900aa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7994f2d640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79900d3900 0x7f79900aa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7994f2d640 1 -- 172.21.15.38:0/3357647737 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79900aacb0 con 0x7f799015dc10 2024-04-25T08:31:49.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f7994f2d640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799015dc10 0x7f79900abfb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7988009eb0 tx=0x7f7988004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:49.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.808+0000 7f798cff9640 1 -- 172.21.15.38:0/3357647737 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988040070 con 0x7f799015dc10 2024-04-25T08:31:49.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.809+0000 7f798cff9640 1 -- 172.21.15.38:0/3357647737 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7988031c90 con 0x7f799015dc10 2024-04-25T08:31:49.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.809+0000 7f798cff9640 1 -- 172.21.15.38:0/3357647737 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798803c640 con 0x7f799015dc10 2024-04-25T08:31:49.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.809+0000 7f7996990640 1 -- 172.21.15.38:0/3357647737 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79900aaf90 con 0x7f799015dc10 2024-04-25T08:31:49.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.809+0000 7f7996990640 1 -- 172.21.15.38:0/3357647737 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79900b3dc0 con 0x7f799015dc10 2024-04-25T08:31:49.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.810+0000 7f7996990640 1 -- 172.21.15.38:0/3357647737 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7990068e80 con 0x7f799015dc10 2024-04-25T08:31:49.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.810+0000 7f798cff9640 1 -- 172.21.15.38:0/3357647737 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7988039040 con 0x7f799015dc10 2024-04-25T08:31:49.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.811+0000 7f798cff9640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f797006b500 0x7f797006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:49.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.811+0000 7f798cff9640 1 -- 172.21.15.38:0/3357647737 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f79880ab7e0 con 0x7f799015dc10 2024-04-25T08:31:49.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.811+0000 7f798ffff640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f797006b500 0x7f797006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:49.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.811+0000 7f798ffff640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f797006b500 0x7f797006d9c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7978000f60 tx=0x7f7978007d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:49.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.815+0000 7f798cff9640 1 -- 172.21.15.38:0/3357647737 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7988039330 con 0x7f799015dc10 2024-04-25T08:31:49.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.996+0000 7f7996990640 1 -- 172.21.15.38:0/3357647737 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79900d2ff0 con 0x7f799015dc10 2024-04-25T08:31:49.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.998+0000 7f798cff9640 1 -- 172.21.15.38:0/3357647737 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f798807a3d0 con 0x7f799015dc10 2024-04-25T08:31:49.994 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:49.994 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:49.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.999+0000 7f7996990640 1 -- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f797006b500 msgr2=0x7f797006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.999+0000 7f7996990640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f797006b500 0x7f797006d9c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7978000f60 tx=0x7f7978007d00 comp rx=0 tx=0).stop 2024-04-25T08:31:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.999+0000 7f7996990640 1 -- 172.21.15.38:0/3357647737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799015dc10 msgr2=0x7f79900abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:49.999+0000 7f7996990640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799015dc10 0x7f79900abfb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7988009eb0 tx=0x7f7988004a40 comp rx=0 tx=0).stop 2024-04-25T08:31:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.000+0000 7f7996990640 1 -- 172.21.15.38:0/3357647737 shutdown_connections 2024-04-25T08:31:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.000+0000 7f7996990640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f799015dc10 0x7f79900abfb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.000+0000 7f7996990640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79900d3900 0x7f79900aa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.000+0000 7f7996990640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f797006b500 0x7f797006d9c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.000+0000 7f7996990640 1 --2- 172.21.15.38:0/3357647737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79900d0f60 0x7f799015bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.000+0000 7f7996990640 1 -- 172.21.15.38:0/3357647737 >> 172.21.15.38:0/3357647737 conn(0x7f7990066d10 msgr2=0x7f799015ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.000+0000 7f7996990640 1 -- 172.21.15.38:0/3357647737 shutdown_connections 2024-04-25T08:31:49.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.000+0000 7f7996990640 1 -- 172.21.15.38:0/3357647737 wait complete. 2024-04-25T08:31:50.007 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:50.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.112+0000 7fc6d1b9d640 1 Processor -- start 2024-04-25T08:31:50.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.112+0000 7fc6d1b9d640 1 -- start start 2024-04-25T08:31:50.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6d1b9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc6cc0d0fe0 0x7fc6cc0b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6d1b9d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc6cc0d2e50 0x7fc6cc0b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6d1b9d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6cc0b8400 con 0x7fc6cc15dc40 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6d1b9d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6cc0d19b0 con 0x7fc6cc0d0fe0 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6d1b9d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6cc0b86e0 con 0x7fc6cc0d2e50 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6caffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc6cc0d0fe0 0x7fc6cc0b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6cb7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc6cc15dc40 0x7fc6cc0d08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49738/0 (socket says 172.21.15.38:49738) 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6cb7fe640 1 -- 172.21.15.38:0/3351394695 learned_addr learned my addr 172.21.15.38:0/3351394695 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6cbfff640 1 --2- 172.21.15.38:0/3351394695 >> v2:172.21.15.107:3300/0 conn(0x7fc6cc0d2e50 0x7fc6cc0b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6caffd640 1 -- 172.21.15.38:0/3351394695 >> v2:172.21.15.107:3300/0 conn(0x7fc6cc0d2e50 msgr2=0x7fc6cc0b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6caffd640 1 --2- 172.21.15.38:0/3351394695 >> v2:172.21.15.107:3300/0 conn(0x7fc6cc0d2e50 0x7fc6cc0b6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6caffd640 1 -- 172.21.15.38:0/3351394695 >> v1:172.21.15.38:6789/0 conn(0x7fc6cc15dc40 legacy=0x7fc6cc0d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6caffd640 1 -- 172.21.15.38:0/3351394695 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc0b89c0 con 0x7fc6cc0d0fe0 2024-04-25T08:31:50.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.113+0000 7fc6ca7fc640 1 -- 172.21.15.38:0/3351394695 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213988961 0 0) 0x7fc6cc0b8400 con 0x7fc6cc15dc40 2024-04-25T08:31:50.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.114+0000 7fc6caffd640 1 --2- 172.21.15.38:0/3351394695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc0d0fe0 0x7fc6cc0b6a60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0c7250 tx=0x7fc6c002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a18b581900f435 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:50.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.114+0000 7fc6ca7fc640 1 -- 172.21.15.38:0/3351394695 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0037070 con 0x7fc6cc0d0fe0 2024-04-25T08:31:50.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.114+0000 7fc6ca7fc640 1 -- 172.21.15.38:0/3351394695 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c0032440 con 0x7fc6cc0d0fe0 2024-04-25T08:31:50.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.114+0000 7fc6ca7fc640 1 -- 172.21.15.38:0/3351394695 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0032810 con 0x7fc6cc0d0fe0 2024-04-25T08:31:50.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.115+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/3351394695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc0d0fe0 msgr2=0x7fc6cc0b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:50.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.115+0000 7fc6d1b9d640 1 --2- 172.21.15.38:0/3351394695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc0d0fe0 0x7fc6cc0b6a60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc0c7250 tx=0x7fc6c002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.115+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/3351394695 shutdown_connections 2024-04-25T08:31:50.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.115+0000 7fc6d1b9d640 1 --2- 172.21.15.38:0/3351394695 >> v2:172.21.15.107:3300/0 conn(0x7fc6cc0d2e50 0x7fc6cc0b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.115+0000 7fc6d1b9d640 1 --2- 172.21.15.38:0/3351394695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc0d0fe0 0x7fc6cc0b6a60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.115+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/3351394695 >> 172.21.15.38:0/3351394695 conn(0x7fc6cc066ce0 msgr2=0x7fc6cc0670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:50.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.115+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/3351394695 shutdown_connections 2024-04-25T08:31:50.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.115+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/3351394695 wait complete. 2024-04-25T08:31:50.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.116+0000 7fc6d1b9d640 1 Processor -- start 2024-04-25T08:31:50.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.116+0000 7fc6d1b9d640 1 -- start start 2024-04-25T08:31:50.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6d1b9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6cc0d0fe0 0x7fc6cc0cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:50.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6d1b9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc0d2e50 0x7fc6cc15b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:50.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6d1b9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15dc40 0x7fc6cc0b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6d1b9d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0b9700 con 0x7fc6cc15dc40 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6d1b9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0b9580 con 0x7fc6cc0d2e50 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6d1b9d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0b9880 con 0x7fc6cc0d0fe0 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6cbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15dc40 0x7fc6cc0b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6cbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15dc40 0x7fc6cc0b1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33192/0 (socket says 172.21.15.38:33192) 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6cbfff640 1 -- 172.21.15.38:0/1636697999 learned_addr learned my addr 172.21.15.38:0/1636697999 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6caffd640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc0d2e50 0x7fc6cc15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6cb7fe640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6cc0d0fe0 0x7fc6cc0cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6cbfff640 1 -- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6cc0d0fe0 msgr2=0x7fc6cc0cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6cbfff640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6cc0d0fe0 0x7fc6cc0cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6cbfff640 1 -- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc0d2e50 msgr2=0x7fc6cc15b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6cbfff640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc0d2e50 0x7fc6cc15b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6cbfff640 1 -- 172.21.15.38:0/1636697999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc0aa7c0 con 0x7fc6cc15dc40 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6caffd640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc0d2e50 0x7fc6cc15b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.117+0000 7fc6cbfff640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15dc40 0x7fc6cc0b1780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400cda0 tx=0x7fc6c400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:50.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.118+0000 7fc6b3fff640 1 -- 172.21.15.38:0/1636697999 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4003b90 con 0x7fc6cc15dc40 2024-04-25T08:31:50.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.118+0000 7fc6b3fff640 1 -- 172.21.15.38:0/1636697999 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c4003d30 con 0x7fc6cc15dc40 2024-04-25T08:31:50.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.118+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/1636697999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc0aaaa0 con 0x7fc6cc15dc40 2024-04-25T08:31:50.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.118+0000 7fc6b3fff640 1 -- 172.21.15.38:0/1636697999 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4004240 con 0x7fc6cc15dc40 2024-04-25T08:31:50.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.118+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/1636697999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc0ab060 con 0x7fc6cc15dc40 2024-04-25T08:31:50.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.119+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/1636697999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc068e80 con 0x7fc6cc15dc40 2024-04-25T08:31:50.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.119+0000 7fc6b3fff640 1 -- 172.21.15.38:0/1636697999 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6c40043e0 con 0x7fc6cc15dc40 2024-04-25T08:31:50.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.119+0000 7fc6b3fff640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc6b806b4d0 0x7fc6b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:50.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.120+0000 7fc6b3fff640 1 -- 172.21.15.38:0/1636697999 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc6c408b0b0 con 0x7fc6cc15dc40 2024-04-25T08:31:50.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.120+0000 7fc6cb7fe640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc6b806b4d0 0x7fc6b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:50.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.120+0000 7fc6cb7fe640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc6b806b4d0 0x7fc6b806d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc000f60 tx=0x7fc6bc007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:50.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.124+0000 7fc6b3fff640 1 -- 172.21.15.38:0/1636697999 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6c401d340 con 0x7fc6cc15dc40 2024-04-25T08:31:50.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.303+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/1636697999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6cc0b4e20 con 0x7fc6cc15dc40 2024-04-25T08:31:50.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.305+0000 7fc6b3fff640 1 -- 172.21.15.38:0/1636697999 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fc6c4059bd0 con 0x7fc6cc15dc40 2024-04-25T08:31:50.301 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:50.301 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.306+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc6b806b4d0 msgr2=0x7fc6b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:50.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.306+0000 7fc6d1b9d640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc6b806b4d0 0x7fc6b806d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc000f60 tx=0x7fc6bc007c00 comp rx=0 tx=0).stop 2024-04-25T08:31:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.306+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/1636697999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15dc40 msgr2=0x7fc6cc0b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.306+0000 7fc6d1b9d640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15dc40 0x7fc6cc0b1780 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400cda0 tx=0x7fc6c400a660 comp rx=0 tx=0).stop 2024-04-25T08:31:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.307+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/1636697999 shutdown_connections 2024-04-25T08:31:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.307+0000 7fc6d1b9d640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6cc15dc40 0x7fc6cc0b1780 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.307+0000 7fc6d1b9d640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6cc0d2e50 0x7fc6cc15b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.307+0000 7fc6d1b9d640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc6b806b4d0 0x7fc6b806d990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.307+0000 7fc6d1b9d640 1 --2- 172.21.15.38:0/1636697999 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6cc0d0fe0 0x7fc6cc0cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.307+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/1636697999 >> 172.21.15.38:0/1636697999 conn(0x7fc6cc066ce0 msgr2=0x7fc6cc15ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.307+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/1636697999 shutdown_connections 2024-04-25T08:31:50.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.307+0000 7fc6d1b9d640 1 -- 172.21.15.38:0/1636697999 wait complete. 2024-04-25T08:31:50.314 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:31:50.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.421+0000 7fb95b038640 1 Processor -- start 2024-04-25T08:31:50.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.422+0000 7fb95b038640 1 -- start start 2024-04-25T08:31:50.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.422+0000 7fb95b038640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb9540b1d60 0x7fb9540cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:50.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.422+0000 7fb95b038640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb9540cff10 0x7fb954161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:50.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.422+0000 7fb95b038640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95415ef40 con 0x7fb954161c50 2024-04-25T08:31:50.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.422+0000 7fb95b038640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95406abe0 con 0x7fb9540cff10 2024-04-25T08:31:50.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.422+0000 7fb95b038640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95406b480 con 0x7fb9540b1d60 2024-04-25T08:31:50.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.422+0000 7fb953fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb9540cff10 0x7fb954161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:50.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.422+0000 7fb953fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb9540cff10 0x7fb954161710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33208/0 (socket says 172.21.15.38:33208) 2024-04-25T08:31:50.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.422+0000 7fb953fff640 1 -- 172.21.15.38:0/3542258276 learned_addr learned my addr 172.21.15.38:0/3542258276 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:50.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.423+0000 7fb958dd4640 1 --2- 172.21.15.38:0/3542258276 >> v2:172.21.15.107:3300/0 conn(0x7fb9540b1d60 0x7fb9540cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:50.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.423+0000 7fb953fff640 1 -- 172.21.15.38:0/3542258276 >> v2:172.21.15.107:3300/0 conn(0x7fb9540b1d60 msgr2=0x7fb9540cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:50.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.423+0000 7fb953fff640 1 --2- 172.21.15.38:0/3542258276 >> v2:172.21.15.107:3300/0 conn(0x7fb9540b1d60 0x7fb9540cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.423+0000 7fb953fff640 1 -- 172.21.15.38:0/3542258276 >> v1:172.21.15.107:6789/0 conn(0x7fb954161c50 legacy=0x7fb9540cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:50.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.423+0000 7fb953fff640 1 -- 172.21.15.38:0/3542258276 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95406ae30 con 0x7fb9540cff10 2024-04-25T08:31:50.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.423+0000 7fb958dd4640 1 --2- 172.21.15.38:0/3542258276 >> v2:172.21.15.107:3300/0 conn(0x7fb9540b1d60 0x7fb9540cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:50.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.424+0000 7fb953fff640 1 --2- 172.21.15.38:0/3542258276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9540cff10 0x7fb954161710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb95406d780 tx=0x7fb94802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4748076c769ffa94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:50.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.424+0000 7fb9537fe640 1 -- 172.21.15.38:0/3542258276 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948037070 con 0x7fb9540cff10 2024-04-25T08:31:50.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.424+0000 7fb9537fe640 1 -- 172.21.15.38:0/3542258276 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb948032440 con 0x7fb9540cff10 2024-04-25T08:31:50.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.424+0000 7fb9537fe640 1 -- 172.21.15.38:0/3542258276 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948032810 con 0x7fb9540cff10 2024-04-25T08:31:50.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.424+0000 7fb95b038640 1 -- 172.21.15.38:0/3542258276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9540cff10 msgr2=0x7fb954161710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:50.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.424+0000 7fb95b038640 1 --2- 172.21.15.38:0/3542258276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9540cff10 0x7fb954161710 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb95406d780 tx=0x7fb94802a270 comp rx=0 tx=0).stop 2024-04-25T08:31:50.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.424+0000 7fb95b038640 1 -- 172.21.15.38:0/3542258276 shutdown_connections 2024-04-25T08:31:50.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.424+0000 7fb95b038640 1 --2- 172.21.15.38:0/3542258276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9540cff10 0x7fb954161710 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.424+0000 7fb95b038640 1 --2- 172.21.15.38:0/3542258276 >> v2:172.21.15.107:3300/0 conn(0x7fb9540b1d60 0x7fb9540cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.424+0000 7fb95b038640 1 -- 172.21.15.38:0/3542258276 >> 172.21.15.38:0/3542258276 conn(0x7fb9540d06b0 msgr2=0x7fb9540d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:50.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.424+0000 7fb95b038640 1 -- 172.21.15.38:0/3542258276 shutdown_connections 2024-04-25T08:31:50.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.424+0000 7fb95b038640 1 -- 172.21.15.38:0/3542258276 wait complete. 2024-04-25T08:31:50.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.425+0000 7fb95b038640 1 Processor -- start 2024-04-25T08:31:50.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.426+0000 7fb95b038640 1 -- start start 2024-04-25T08:31:50.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.426+0000 7fb95b038640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9540b1d60 0x7fb9540ad360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:50.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.426+0000 7fb95b038640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9540cff10 0x7fb954156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:50.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.426+0000 7fb95b038640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb954161c50 0x7fb954157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.426+0000 7fb95b038640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95406b910 con 0x7fb9540b1d60 2024-04-25T08:31:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.426+0000 7fb95b038640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9540d03a0 con 0x7fb954161c50 2024-04-25T08:31:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.426+0000 7fb95b038640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95406ba90 con 0x7fb9540cff10 2024-04-25T08:31:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.426+0000 7fb958dd4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9540b1d60 0x7fb9540ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb958dd4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9540b1d60 0x7fb9540ad360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33224/0 (socket says 172.21.15.38:33224) 2024-04-25T08:31:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb958dd4640 1 -- 172.21.15.38:0/4059088474 learned_addr learned my addr 172.21.15.38:0/4059088474 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb9595d5640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb954161c50 0x7fb954157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb953fff640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9540cff10 0x7fb954156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb958dd4640 1 -- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9540cff10 msgr2=0x7fb954156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb958dd4640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9540cff10 0x7fb954156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb958dd4640 1 -- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb954161c50 msgr2=0x7fb954157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:50.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb958dd4640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb954161c50 0x7fb954157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb958dd4640 1 -- 172.21.15.38:0/4059088474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9541540c0 con 0x7fb9540b1d60 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb9595d5640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb954161c50 0x7fb954157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb953fff640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9540cff10 0x7fb954156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb958dd4640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9540b1d60 0x7fb9540ad360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb94c007f20 tx=0x7fb94c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb9517fa640 1 -- 172.21.15.38:0/4059088474 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c01a070 con 0x7fb9540b1d60 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb9517fa640 1 -- 172.21.15.38:0/4059088474 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c003b90 con 0x7fb9540b1d60 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb9517fa640 1 -- 172.21.15.38:0/4059088474 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c004030 con 0x7fb9540b1d60 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb95b038640 1 -- 172.21.15.38:0/4059088474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9541f4550 con 0x7fb9540b1d60 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.427+0000 7fb95b038640 1 -- 172.21.15.38:0/4059088474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb954156ab0 con 0x7fb9540b1d60 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.428+0000 7fb95b038640 1 -- 172.21.15.38:0/4059088474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb954068e80 con 0x7fb9540b1d60 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.429+0000 7fb9517fa640 1 -- 172.21.15.38:0/4059088474 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb94c01d070 con 0x7fb9540b1d60 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.429+0000 7fb9517fa640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb94006b4d0 0x7fb94006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.429+0000 7fb9517fa640 1 -- 172.21.15.38:0/4059088474 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb94c08a720 con 0x7fb9540b1d60 2024-04-25T08:31:50.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.429+0000 7fb953fff640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb94006b4d0 0x7fb94006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:50.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.430+0000 7fb953fff640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb94006b4d0 0x7fb94006d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb948034000 tx=0x7fb94802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:50.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.433+0000 7fb9517fa640 1 -- 172.21.15.38:0/4059088474 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb94c010070 con 0x7fb9540b1d60 2024-04-25T08:31:50.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:50.655+0000 7fb95b038640 1 -- 172.21.15.38:0/4059088474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb95406c9e0 con 0x7fb9540b1d60 2024-04-25T08:31:51.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.444+0000 7fb9517fa640 1 -- 172.21.15.38:0/4059088474 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb94c058c10 con 0x7fb9540b1d60 2024-04-25T08:31:51.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.447+0000 7fb95b038640 1 -- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb94006b4d0 msgr2=0x7fb94006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:51.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.447+0000 7fb95b038640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb94006b4d0 0x7fb94006d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb948034000 tx=0x7fb94802e000 comp rx=0 tx=0).stop 2024-04-25T08:31:51.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.447+0000 7fb95b038640 1 -- 172.21.15.38:0/4059088474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9540b1d60 msgr2=0x7fb9540ad360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:51.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.447+0000 7fb95b038640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9540b1d60 0x7fb9540ad360 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb94c007f20 tx=0x7fb94c00d560 comp rx=0 tx=0).stop 2024-04-25T08:31:51.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.447+0000 7fb95b038640 1 -- 172.21.15.38:0/4059088474 shutdown_connections 2024-04-25T08:31:51.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.447+0000 7fb95b038640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb954161c50 0x7fb954157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:51.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.447+0000 7fb95b038640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb94006b4d0 0x7fb94006d990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:51.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.447+0000 7fb95b038640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9540cff10 0x7fb954156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:51.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.447+0000 7fb95b038640 1 --2- 172.21.15.38:0/4059088474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9540b1d60 0x7fb9540ad360 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:51.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.447+0000 7fb95b038640 1 -- 172.21.15.38:0/4059088474 >> 172.21.15.38:0/4059088474 conn(0x7fb9540d06b0 msgr2=0x7fb95415f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:51.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.447+0000 7fb95b038640 1 -- 172.21.15.38:0/4059088474 shutdown_connections 2024-04-25T08:31:51.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.447+0000 7fb95b038640 1 -- 172.21.15.38:0/4059088474 wait complete. 2024-04-25T08:31:51.457 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T08:31:51.457 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:31:51.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.571+0000 7fe4112ee640 1 Processor -- start 2024-04-25T08:31:51.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.571+0000 7fe4112ee640 1 -- start start 2024-04-25T08:31:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe4112ee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe40c15b990 0x7fe40c0cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe4112ee640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe40c15f5f0 0x7fe40c0d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe4112ee640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40c15fd00 con 0x7fe40c0d1b50 2024-04-25T08:31:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe4112ee640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40c160580 con 0x7fe40c15f5f0 2024-04-25T08:31:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe4112ee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c15fa30 con 0x7fe40c15b990 2024-04-25T08:31:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe40a7fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe40c15f5f0 0x7fe40c0d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe40a7fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe40c15f5f0 0x7fe40c0d1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33240/0 (socket says 172.21.15.38:33240) 2024-04-25T08:31:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe40a7fc640 1 -- 172.21.15.38:0/838041124 learned_addr learned my addr 172.21.15.38:0/838041124 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe40affd640 1 --2- 172.21.15.38:0/838041124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe40c15b990 0x7fe40c0cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe40a7fc640 1 -- 172.21.15.38:0/838041124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe40c15b990 msgr2=0x7fe40c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:51.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe40a7fc640 1 --2- 172.21.15.38:0/838041124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe40c15b990 0x7fe40c0cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:51.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe40a7fc640 1 -- 172.21.15.38:0/838041124 >> v1:172.21.15.107:6789/0 conn(0x7fe40c0d1b50 legacy=0x7fe40c0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:51.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe40a7fc640 1 -- 172.21.15.38:0/838041124 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c15ffc0 con 0x7fe40c15f5f0 2024-04-25T08:31:51.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.572+0000 7fe40affd640 1 --2- 172.21.15.38:0/838041124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe40c15b990 0x7fe40c0cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:51.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.573+0000 7fe40a7fc640 1 --2- 172.21.15.38:0/838041124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe40c15f5f0 0x7fe40c0d1610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe400009870 tx=0x7fe40002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3f19a6a85f56f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:51.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.573+0000 7fe409ffb640 1 -- 172.21.15.38:0/838041124 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400037070 con 0x7fe40c15f5f0 2024-04-25T08:31:51.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.573+0000 7fe409ffb640 1 -- 172.21.15.38:0/838041124 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe400032440 con 0x7fe40c15f5f0 2024-04-25T08:31:51.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.573+0000 7fe409ffb640 1 -- 172.21.15.38:0/838041124 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400032810 con 0x7fe40c15f5f0 2024-04-25T08:31:51.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.573+0000 7fe4112ee640 1 -- 172.21.15.38:0/838041124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe40c15f5f0 msgr2=0x7fe40c0d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:51.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.573+0000 7fe4112ee640 1 --2- 172.21.15.38:0/838041124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe40c15f5f0 0x7fe40c0d1610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe400009870 tx=0x7fe40002a180 comp rx=0 tx=0).stop 2024-04-25T08:31:51.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.574+0000 7fe4112ee640 1 -- 172.21.15.38:0/838041124 shutdown_connections 2024-04-25T08:31:51.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.574+0000 7fe4112ee640 1 --2- 172.21.15.38:0/838041124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe40c15f5f0 0x7fe40c0d1610 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:51.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.574+0000 7fe4112ee640 1 --2- 172.21.15.38:0/838041124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe40c15b990 0x7fe40c0cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:51.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.574+0000 7fe4112ee640 1 -- 172.21.15.38:0/838041124 >> 172.21.15.38:0/838041124 conn(0x7fe40c08b000 msgr2=0x7fe40c08b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:51.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.574+0000 7fe4112ee640 1 -- 172.21.15.38:0/838041124 shutdown_connections 2024-04-25T08:31:51.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.574+0000 7fe4112ee640 1 -- 172.21.15.38:0/838041124 wait complete. 2024-04-25T08:31:51.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.575+0000 7fe4112ee640 1 Processor -- start 2024-04-25T08:31:51.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.575+0000 7fe4112ee640 1 -- start start 2024-04-25T08:31:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe4112ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe40c0d1b50 0x7fe40c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe4112ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe40c15b990 0x7fe40c15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe4112ee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe40c15f5f0 0x7fe40c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe4112ee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c15bd70 con 0x7fe40c15b990 2024-04-25T08:31:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe4112ee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0b1ac0 con 0x7fe40c0d1b50 2024-04-25T08:31:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe4112ee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c15bef0 con 0x7fe40c15f5f0 2024-04-25T08:31:51.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe40a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe40c15b990 0x7fe40c15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe40a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe40c15b990 0x7fe40c15ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33246/0 (socket says 172.21.15.38:33246) 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe40a7fc640 1 -- 172.21.15.38:0/2102549409 learned_addr learned my addr 172.21.15.38:0/2102549409 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe40affd640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe40c0d1b50 0x7fe40c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe40b7fe640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe40c15f5f0 0x7fe40c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe40a7fc640 1 -- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe40c15f5f0 msgr2=0x7fe40c15cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe40a7fc640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe40c15f5f0 0x7fe40c15cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe40a7fc640 1 -- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe40c0d1b50 msgr2=0x7fe40c152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe40a7fc640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe40c0d1b50 0x7fe40c152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe40a7fc640 1 -- 172.21.15.38:0/2102549409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c158510 con 0x7fe40c15b990 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe40a7fc640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe40c15b990 0x7fe40c15ca50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe400000c00 tx=0x7fe4000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe3f37fe640 1 -- 172.21.15.38:0/2102549409 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400037040 con 0x7fe40c15b990 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.576+0000 7fe3f37fe640 1 -- 172.21.15.38:0/2102549409 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe400042070 con 0x7fe40c15b990 2024-04-25T08:31:51.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.577+0000 7fe4112ee640 1 -- 172.21.15.38:0/2102549409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe40c1587f0 con 0x7fe40c15b990 2024-04-25T08:31:51.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.577+0000 7fe3f37fe640 1 -- 172.21.15.38:0/2102549409 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400030bd0 con 0x7fe40c15b990 2024-04-25T08:31:51.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.577+0000 7fe4112ee640 1 -- 172.21.15.38:0/2102549409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40c0b68e0 con 0x7fe40c15b990 2024-04-25T08:31:51.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.578+0000 7fe4112ee640 1 -- 172.21.15.38:0/2102549409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40c068e80 con 0x7fe40c15b990 2024-04-25T08:31:51.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.578+0000 7fe3f37fe640 1 -- 172.21.15.38:0/2102549409 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe400047460 con 0x7fe40c15b990 2024-04-25T08:31:51.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.578+0000 7fe3f37fe640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe3f806b4d0 0x7fe3f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:51.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.579+0000 7fe3f37fe640 1 -- 172.21.15.38:0/2102549409 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe4000aafb0 con 0x7fe40c15b990 2024-04-25T08:31:51.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.579+0000 7fe40affd640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe3f806b4d0 0x7fe3f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:51.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.579+0000 7fe40affd640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe3f806b4d0 0x7fe3f806d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00ab60 tx=0x7fe3fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:51.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.583+0000 7fe3f37fe640 1 -- 172.21.15.38:0/2102549409 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe400079530 con 0x7fe40c15b990 2024-04-25T08:31:51.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:51.798+0000 7fe4112ee640 1 -- 172.21.15.38:0/2102549409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe40c0b1220 con 0x7fe40c15b990 2024-04-25T08:31:52.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.447+0000 7fe3f37fe640 1 -- 172.21.15.38:0/2102549409 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe40007d5d0 con 0x7fe40c15b990 2024-04-25T08:31:52.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.449+0000 7fe4112ee640 1 -- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe3f806b4d0 msgr2=0x7fe3f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:52.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.449+0000 7fe4112ee640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe3f806b4d0 0x7fe3f806d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00ab60 tx=0x7fe3fc008040 comp rx=0 tx=0).stop 2024-04-25T08:31:52.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.450+0000 7fe4112ee640 1 -- 172.21.15.38:0/2102549409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe40c15b990 msgr2=0x7fe40c15ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:52.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.450+0000 7fe4112ee640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe40c15b990 0x7fe40c15ca50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe400000c00 tx=0x7fe4000046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.450+0000 7fe4112ee640 1 -- 172.21.15.38:0/2102549409 shutdown_connections 2024-04-25T08:31:52.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.450+0000 7fe4112ee640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe3f806b4d0 0x7fe3f806d990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.450+0000 7fe4112ee640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe40c15f5f0 0x7fe40c15cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.450+0000 7fe4112ee640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe40c15b990 0x7fe40c15ca50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.450+0000 7fe4112ee640 1 --2- 172.21.15.38:0/2102549409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe40c0d1b50 0x7fe40c152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.450+0000 7fe4112ee640 1 -- 172.21.15.38:0/2102549409 >> 172.21.15.38:0/2102549409 conn(0x7fe40c08b000 msgr2=0x7fe40c15abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:52.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.450+0000 7fe4112ee640 1 -- 172.21.15.38:0/2102549409 shutdown_connections 2024-04-25T08:31:52.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.450+0000 7fe4112ee640 1 -- 172.21.15.38:0/2102549409 wait complete. 2024-04-25T08:31:52.459 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:31:52.460 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:31:52.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.570+0000 7f3e6b9e0640 1 Processor -- start 2024-04-25T08:31:52.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.570+0000 7f3e6b9e0640 1 -- start start 2024-04-25T08:31:52.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.571+0000 7f3e6b9e0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3e6406b2f0 0x7f3e640b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.571+0000 7f3e6b9e0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e640cac60 con 0x7f3e640b1610 2024-04-25T08:31:52.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.571+0000 7f3e6b9e0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e640caf40 con 0x7f3e6415daf0 2024-04-25T08:31:52.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.571+0000 7f3e6b9e0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e640cb220 con 0x7f3e6406b2f0 2024-04-25T08:31:52.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.571+0000 7f3e68f7b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f3e640b1610 0x7f3e6406abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49744/0 (socket says 172.21.15.38:49744) 2024-04-25T08:31:52.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.571+0000 7f3e68f7b640 1 -- 172.21.15.38:0/835235629 learned_addr learned my addr 172.21.15.38:0/835235629 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:52.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e69f7d640 1 --2- 172.21.15.38:0/835235629 >> v2:172.21.15.107:3300/0 conn(0x7f3e6406b2f0 0x7f3e640b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071491083 0 0) 0x7f3e640cac60 con 0x7f3e640b1610 2024-04-25T08:31:52.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e50003520 con 0x7f3e640b1610 2024-04-25T08:31:52.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2755592540 0 0) 0x7f3e50003520 con 0x7f3e640b1610 2024-04-25T08:31:52.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 >> v2:172.21.15.107:3300/0 conn(0x7f3e6406b2f0 msgr2=0x7f3e640b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:52.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 --2- 172.21.15.38:0/835235629 >> v2:172.21.15.107:3300/0 conn(0x7f3e6406b2f0 0x7f3e640b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 >> v1:172.21.15.107:6789/0 conn(0x7f3e6415daf0 legacy=0x7f3e640cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e640cb500 con 0x7f3e640b1610 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f3e54001670 con 0x7f3e640b1610 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 >> v1:172.21.15.38:6789/0 conn(0x7f3e640b1610 legacy=0x7f3e6406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 --2- 172.21.15.38:0/835235629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e500039b0 0x7f3e50003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 --2- 172.21.15.38:0/835235629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e50004800 0x7f3e50004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 --2- 172.21.15.38:0/835235629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 0x7f3e5000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e640cb220 con 0x7f3e500051c0 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5000c070 con 0x7f3e500039b0 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5000c270 con 0x7f3e50004800 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e69f7d640 1 --2- 172.21.15.38:0/835235629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 0x7f3e5000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e6977c640 1 --2- 172.21.15.38:0/835235629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e500039b0 0x7f3e50003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.572+0000 7f3e68f7b640 1 --2- 172.21.15.38:0/835235629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e50004800 0x7f3e50004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.573+0000 7f3e69f7d640 1 -- 172.21.15.38:0/835235629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e50004800 msgr2=0x7f3e50004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:52.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.573+0000 7f3e69f7d640 1 --2- 172.21.15.38:0/835235629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e50004800 0x7f3e50004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.573+0000 7f3e69f7d640 1 -- 172.21.15.38:0/835235629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e500039b0 msgr2=0x7f3e50003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:52.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.573+0000 7f3e69f7d640 1 --2- 172.21.15.38:0/835235629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e500039b0 0x7f3e50003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.573+0000 7f3e69f7d640 1 -- 172.21.15.38:0/835235629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50003520 con 0x7f3e500051c0 2024-04-25T08:31:52.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.573+0000 7f3e69f7d640 1 -- 172.21.15.38:0/835235629 reap_dead start 2024-04-25T08:31:52.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.573+0000 7f3e69f7d640 1 --2- 172.21.15.38:0/835235629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 0x7f3e5000b9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009fc0 tx=0x7f3e50005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8fc1daa5c1ddc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:52.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.573+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c035020 con 0x7f3e500051c0 2024-04-25T08:31:52.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.573+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5c003bd0 con 0x7f3e500051c0 2024-04-25T08:31:52.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.573+0000 7f3e5bfff640 1 -- 172.21.15.38:0/835235629 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c009c10 con 0x7f3e500051c0 2024-04-25T08:31:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.574+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/835235629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 msgr2=0x7f3e5000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.574+0000 7f3e6b9e0640 1 --2- 172.21.15.38:0/835235629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 0x7f3e5000b9c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009fc0 tx=0x7f3e50005830 comp rx=0 tx=0).stop 2024-04-25T08:31:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.574+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/835235629 shutdown_connections 2024-04-25T08:31:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.574+0000 7f3e6b9e0640 1 --2- 172.21.15.38:0/835235629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 0x7f3e5000b9c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.574+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/835235629 >> 172.21.15.38:0/835235629 conn(0x7f3e64099a70 msgr2=0x7f3e640d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.574+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/835235629 shutdown_connections 2024-04-25T08:31:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.574+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/835235629 wait complete. 2024-04-25T08:31:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.575+0000 7f3e6b9e0640 1 Processor -- start 2024-04-25T08:31:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.575+0000 7f3e6b9e0640 1 -- start start 2024-04-25T08:31:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.576+0000 7f3e6b9e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 0x7f3e640c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.576+0000 7f3e6b9e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e64153300 0x7f3e640c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.576+0000 7f3e6b9e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e64153bf0 0x7f3e64154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.576+0000 7f3e6b9e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6415ded0 con 0x7f3e500051c0 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.576+0000 7f3e6b9e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6409a040 con 0x7f3e64153300 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.576+0000 7f3e6b9e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e64151590 con 0x7f3e64153bf0 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.576+0000 7f3e6977c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 0x7f3e640c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.576+0000 7f3e6977c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 0x7f3e640c8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33264/0 (socket says 172.21.15.38:33264) 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.576+0000 7f3e6977c640 1 -- 172.21.15.38:0/3728991646 learned_addr learned my addr 172.21.15.38:0/3728991646 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.576+0000 7f3e68f7b640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e64153300 0x7f3e640c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.576+0000 7f3e69f7d640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e64153bf0 0x7f3e64154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.576+0000 7f3e6977c640 1 -- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e64153bf0 msgr2=0x7f3e64154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.577+0000 7f3e6977c640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e64153bf0 0x7f3e64154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.577+0000 7f3e6977c640 1 -- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e64153300 msgr2=0x7f3e640c6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.577+0000 7f3e6977c640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e64153300 0x7f3e640c6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.577+0000 7f3e6977c640 1 -- 172.21.15.38:0/3728991646 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e640c70e0 con 0x7f3e500051c0 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.577+0000 7f3e68f7b640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e64153300 0x7f3e640c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:52.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.577+0000 7f3e6977c640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 0x7f3e640c8570 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000f620 tx=0x7f3e6000fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:52.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.577+0000 7f3e59ffb640 1 -- 172.21.15.38:0/3728991646 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e600052e0 con 0x7f3e500051c0 2024-04-25T08:31:52.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.577+0000 7f3e59ffb640 1 -- 172.21.15.38:0/3728991646 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e60018070 con 0x7f3e500051c0 2024-04-25T08:31:52.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.577+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/3728991646 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e64153780 con 0x7f3e500051c0 2024-04-25T08:31:52.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.577+0000 7f3e59ffb640 1 -- 172.21.15.38:0/3728991646 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e60003810 con 0x7f3e500051c0 2024-04-25T08:31:52.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.577+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/3728991646 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e640c8e30 con 0x7f3e500051c0 2024-04-25T08:31:52.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.578+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/3728991646 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e640cebc0 con 0x7f3e500051c0 2024-04-25T08:31:52.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.579+0000 7f3e59ffb640 1 -- 172.21.15.38:0/3728991646 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e600039b0 con 0x7f3e500051c0 2024-04-25T08:31:52.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.579+0000 7f3e59ffb640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e4806b4d0 0x7f3e4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.579+0000 7f3e59ffb640 1 -- 172.21.15.38:0/3728991646 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3e6008a280 con 0x7f3e500051c0 2024-04-25T08:31:52.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.579+0000 7f3e68f7b640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e4806b4d0 0x7f3e4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.580+0000 7f3e68f7b640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e4806b4d0 0x7f3e4806d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e640ca100 tx=0x7f3e54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:52.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.583+0000 7f3e59ffb640 1 -- 172.21.15.38:0/3728991646 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e640cebc0 con 0x7f3e500051c0 2024-04-25T08:31:52.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.801+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/3728991646 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e640b6590 con 0x7f3e500051c0 2024-04-25T08:31:52.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.802+0000 7f3e59ffb640 1 -- 172.21.15.38:0/3728991646 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f3e60058dc0 con 0x7f3e500051c0 2024-04-25T08:31:52.799 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:52.799 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4242612806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4242612806},{"type":"v1","addr":"172.21.15.38:6835","nonce":4242612806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2154167637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":2154167637},{"type":"v1","addr":"172.21.15.107:6837","nonce":2154167637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/678452936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":678452936},{"type":"v1","addr":"172.21.15.38:6834","nonce":678452936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:31:35.665008+0000","modified":"2024-04-25T08:31:51.799830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6836/1104921666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1104921666},{"type":"v1","addr":"172.21.15.107:6836","nonce":1104921666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:31:52.799 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-04-25T08:31:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.804+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e4806b4d0 msgr2=0x7f3e4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.804+0000 7f3e6b9e0640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e4806b4d0 0x7f3e4806d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3e640ca100 tx=0x7f3e54009040 comp rx=0 tx=0).stop 2024-04-25T08:31:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.804+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/3728991646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 msgr2=0x7f3e640c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.804+0000 7f3e6b9e0640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 0x7f3e640c8570 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000f620 tx=0x7f3e6000fae0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.804+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/3728991646 shutdown_connections 2024-04-25T08:31:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.804+0000 7f3e6b9e0640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e4806b4d0 0x7f3e4806d990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.804+0000 7f3e6b9e0640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e64153bf0 0x7f3e64154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.804+0000 7f3e6b9e0640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e64153300 0x7f3e640c6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.804+0000 7f3e6b9e0640 1 --2- 172.21.15.38:0/3728991646 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e500051c0 0x7f3e640c8570 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.804+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/3728991646 >> 172.21.15.38:0/3728991646 conn(0x7f3e64099a70 msgr2=0x7f3e640cc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.804+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/3728991646 shutdown_connections 2024-04-25T08:31:52.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.804+0000 7f3e6b9e0640 1 -- 172.21.15.38:0/3728991646 wait complete. 2024-04-25T08:31:52.811 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:52.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.915+0000 7f753d789640 1 Processor -- start 2024-04-25T08:31:52.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.915+0000 7f753d789640 1 -- start start 2024-04-25T08:31:52.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.916+0000 7f753d789640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f753815dc10 0x7f75380ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.916+0000 7f753d789640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f75380ced60 0x7f75380d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.916+0000 7f753d789640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f75380d3900 0x7f75380d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.916+0000 7f753d789640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753815dff0 con 0x7f75380d3900 2024-04-25T08:31:52.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.916+0000 7f753d789640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75380b6990 con 0x7f75380ced60 2024-04-25T08:31:52.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.916+0000 7f753d789640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75380d4710 con 0x7f753815dc10 2024-04-25T08:31:52.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.916+0000 7f75377fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f75380d3900 0x7f75380d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.916+0000 7f75377fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f75380d3900 0x7f75380d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33280/0 (socket says 172.21.15.38:33280) 2024-04-25T08:31:52.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.916+0000 7f75377fe640 1 -- 172.21.15.38:0/3996291224 learned_addr learned my addr 172.21.15.38:0/3996291224 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:52.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.917+0000 7f75367fc640 1 --2- 172.21.15.38:0/3996291224 >> v2:172.21.15.107:3300/0 conn(0x7f75380ced60 0x7f75380d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.917+0000 7f7536ffd640 1 --2- 172.21.15.38:0/3996291224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f753815dc10 0x7f75380ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.917+0000 7f75377fe640 1 -- 172.21.15.38:0/3996291224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f753815dc10 msgr2=0x7f75380ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:52.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.917+0000 7f75377fe640 1 --2- 172.21.15.38:0/3996291224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f753815dc10 0x7f75380ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.917+0000 7f75377fe640 1 -- 172.21.15.38:0/3996291224 >> v2:172.21.15.107:3300/0 conn(0x7f75380ced60 msgr2=0x7f75380d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:52.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.917+0000 7f75377fe640 1 --2- 172.21.15.38:0/3996291224 >> v2:172.21.15.107:3300/0 conn(0x7f75380ced60 0x7f75380d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.917+0000 7f75377fe640 1 -- 172.21.15.38:0/3996291224 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75380d3e70 con 0x7f75380d3900 2024-04-25T08:31:52.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.917+0000 7f75377fe640 1 --2- 172.21.15.38:0/3996291224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75380d3900 0x7f75380d0d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f753806cfd0 tx=0x7f753002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620b76dab523d779 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:52.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.918+0000 7f7535ffb640 1 -- 172.21.15.38:0/3996291224 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530037070 con 0x7f75380d3900 2024-04-25T08:31:52.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.918+0000 7f7535ffb640 1 -- 172.21.15.38:0/3996291224 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7530032440 con 0x7f75380d3900 2024-04-25T08:31:52.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.918+0000 7f7535ffb640 1 -- 172.21.15.38:0/3996291224 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530032810 con 0x7f75380d3900 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.918+0000 7f753d789640 1 -- 172.21.15.38:0/3996291224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75380d3900 msgr2=0x7f75380d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.918+0000 7f753d789640 1 --2- 172.21.15.38:0/3996291224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75380d3900 0x7f75380d0d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f753806cfd0 tx=0x7f753002a140 comp rx=0 tx=0).stop 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.918+0000 7f753d789640 1 -- 172.21.15.38:0/3996291224 shutdown_connections 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.918+0000 7f753d789640 1 --2- 172.21.15.38:0/3996291224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75380d3900 0x7f75380d0d90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.918+0000 7f753d789640 1 --2- 172.21.15.38:0/3996291224 >> v2:172.21.15.107:3300/0 conn(0x7f75380ced60 0x7f75380d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.918+0000 7f753d789640 1 --2- 172.21.15.38:0/3996291224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f753815dc10 0x7f75380ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.918+0000 7f753d789640 1 -- 172.21.15.38:0/3996291224 >> 172.21.15.38:0/3996291224 conn(0x7f7538066d10 msgr2=0x7f7538067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.918+0000 7f753d789640 1 -- 172.21.15.38:0/3996291224 shutdown_connections 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.918+0000 7f753d789640 1 -- 172.21.15.38:0/3996291224 wait complete. 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.919+0000 7f753d789640 1 Processor -- start 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.920+0000 7f753d789640 1 -- start start 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.920+0000 7f753d789640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75380ced60 0x7f753814e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.920+0000 7f753d789640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75380d3900 0x7f753815fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.920+0000 7f753d789640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f753815dc10 0x7f753815e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.920+0000 7f753d789640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75380b6bc0 con 0x7f75380d3900 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.920+0000 7f753d789640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75380d4430 con 0x7f75380ced60 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.920+0000 7f753d789640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75380b2370 con 0x7f753815dc10 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.920+0000 7f75367fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75380d3900 0x7f753815fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.920+0000 7f75367fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75380d3900 0x7f753815fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33294/0 (socket says 172.21.15.38:33294) 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.920+0000 7f75367fc640 1 -- 172.21.15.38:0/1958010051 learned_addr learned my addr 172.21.15.38:0/1958010051 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f7536ffd640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75380ced60 0x7f753814e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f75377fe640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f753815dc10 0x7f753815e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f75367fc640 1 -- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f753815dc10 msgr2=0x7f753815e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f75367fc640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f753815dc10 0x7f753815e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f75367fc640 1 -- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75380ced60 msgr2=0x7f753814e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f75367fc640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75380ced60 0x7f753814e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f75367fc640 1 -- 172.21.15.38:0/1958010051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7538150e50 con 0x7f75380d3900 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f75367fc640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75380d3900 0x7f753815fa60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f752000d830 tx=0x7f752000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:52.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f752f7fe640 1 -- 172.21.15.38:0/1958010051 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520018020 con 0x7f75380d3900 2024-04-25T08:31:52.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f753d789640 1 -- 172.21.15.38:0/1958010051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7538151130 con 0x7f75380d3900 2024-04-25T08:31:52.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f752f7fe640 1 -- 172.21.15.38:0/1958010051 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75200096a0 con 0x7f75380d3900 2024-04-25T08:31:52.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f752f7fe640 1 -- 172.21.15.38:0/1958010051 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7520009a70 con 0x7f75380d3900 2024-04-25T08:31:52.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.921+0000 7f753d789640 1 -- 172.21.15.38:0/1958010051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7538142920 con 0x7f75380d3900 2024-04-25T08:31:52.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.922+0000 7f753d789640 1 -- 172.21.15.38:0/1958010051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7538068e80 con 0x7f75380d3900 2024-04-25T08:31:52.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.922+0000 7f752f7fe640 1 -- 172.21.15.38:0/1958010051 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f752001b020 con 0x7f75380d3900 2024-04-25T08:31:52.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.923+0000 7f752f7fe640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f751806b4d0 0x7f751806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:52.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.923+0000 7f752f7fe640 1 -- 172.21.15.38:0/1958010051 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f752008a8e0 con 0x7f75380d3900 2024-04-25T08:31:52.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.923+0000 7f7536ffd640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f751806b4d0 0x7f751806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:52.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.924+0000 7f7536ffd640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f751806b4d0 0x7f751806d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f75280099c0 tx=0x7f7528003990 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:52.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:52.927+0000 7f752f7fe640 1 -- 172.21.15.38:0/1958010051 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7520022030 con 0x7f75380d3900 2024-04-25T08:31:53.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.110+0000 7f753d789640 1 -- 172.21.15.38:0/1958010051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75380d2070 con 0x7f75380d3900 2024-04-25T08:31:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.111+0000 7f752f7fe640 1 -- 172.21.15.38:0/1958010051 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f75200592f0 con 0x7f75380d3900 2024-04-25T08:31:53.108 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:53.108 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.113+0000 7f753d789640 1 -- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f751806b4d0 msgr2=0x7f751806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.113+0000 7f753d789640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f751806b4d0 0x7f751806d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f75280099c0 tx=0x7f7528003990 comp rx=0 tx=0).stop 2024-04-25T08:31:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.113+0000 7f753d789640 1 -- 172.21.15.38:0/1958010051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75380d3900 msgr2=0x7f753815fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.113+0000 7f753d789640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75380d3900 0x7f753815fa60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f752000d830 tx=0x7f752000dcf0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.113+0000 7f753d789640 1 -- 172.21.15.38:0/1958010051 shutdown_connections 2024-04-25T08:31:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.113+0000 7f753d789640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f751806b4d0 0x7f751806d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.113+0000 7f753d789640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f753815dc10 0x7f753815e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.113+0000 7f753d789640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f75380d3900 0x7f753815fa60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.113+0000 7f753d789640 1 --2- 172.21.15.38:0/1958010051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75380ced60 0x7f753814e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.113+0000 7f753d789640 1 -- 172.21.15.38:0/1958010051 >> 172.21.15.38:0/1958010051 conn(0x7f7538066d10 msgr2=0x7f7538073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.113+0000 7f753d789640 1 -- 172.21.15.38:0/1958010051 shutdown_connections 2024-04-25T08:31:53.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.113+0000 7f753d789640 1 -- 172.21.15.38:0/1958010051 wait complete. 2024-04-25T08:31:53.122 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:31:53.122 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:31:53.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.227+0000 7f58eb45a640 1 Processor -- start 2024-04-25T08:31:53.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.227+0000 7f58eb45a640 1 -- start start 2024-04-25T08:31:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58eb45a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58e40ceca0 0x7f58e40d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58eb45a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e40d1b60 con 0x7f58e415da90 2024-04-25T08:31:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58eb45a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e40d3680 con 0x7f58e40d3de0 2024-04-25T08:31:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58eb45a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e40d2bc0 con 0x7f58e40ceca0 2024-04-25T08:31:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58e91f6640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f58e415da90 0x7f58e40d2f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49752/0 (socket says 172.21.15.38:49752) 2024-04-25T08:31:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58e91f6640 1 -- 172.21.15.38:0/3970176786 learned_addr learned my addr 172.21.15.38:0/3970176786 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58e89f5640 1 --2- 172.21.15.38:0/3970176786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58e40ceca0 0x7f58e40d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58dbfff640 1 -- 172.21.15.38:0/3970176786 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568038593 0 0) 0x7f58e40d1b60 con 0x7f58e415da90 2024-04-25T08:31:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58dbfff640 1 -- 172.21.15.38:0/3970176786 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58d0003520 con 0x7f58e415da90 2024-04-25T08:31:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58dbfff640 1 -- 172.21.15.38:0/3970176786 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855445583 0 0) 0x7f58e40d3680 con 0x7f58e40d3de0 2024-04-25T08:31:53.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58dbfff640 1 -- 172.21.15.38:0/3970176786 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58e40d1b60 con 0x7f58e40d3de0 2024-04-25T08:31:53.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58e89f5640 1 -- 172.21.15.38:0/3970176786 >> v1:172.21.15.107:6789/0 conn(0x7f58e40d3de0 legacy=0x7f58e40b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58e89f5640 1 -- 172.21.15.38:0/3970176786 >> v1:172.21.15.38:6789/0 conn(0x7f58e415da90 legacy=0x7f58e40d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.228+0000 7f58e89f5640 1 -- 172.21.15.38:0/3970176786 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e40d1e00 con 0x7f58e40ceca0 2024-04-25T08:31:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.229+0000 7f58e89f5640 1 --2- 172.21.15.38:0/3970176786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58e40ceca0 0x7f58e40d38a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f58d40097d0 tx=0x7f58d402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3c80c1b56095ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.229+0000 7f58dbfff640 1 -- 172.21.15.38:0/3970176786 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4037070 con 0x7f58e40ceca0 2024-04-25T08:31:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.229+0000 7f58dbfff640 1 -- 172.21.15.38:0/3970176786 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d4032440 con 0x7f58e40ceca0 2024-04-25T08:31:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.229+0000 7f58dbfff640 1 -- 172.21.15.38:0/3970176786 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4032810 con 0x7f58e40ceca0 2024-04-25T08:31:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.230+0000 7f58eb45a640 1 -- 172.21.15.38:0/3970176786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58e40ceca0 msgr2=0x7f58e40d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.230+0000 7f58eb45a640 1 --2- 172.21.15.38:0/3970176786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58e40ceca0 0x7f58e40d38a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f58d40097d0 tx=0x7f58d402a210 comp rx=0 tx=0).stop 2024-04-25T08:31:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.230+0000 7f58eb45a640 1 -- 172.21.15.38:0/3970176786 shutdown_connections 2024-04-25T08:31:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.230+0000 7f58eb45a640 1 --2- 172.21.15.38:0/3970176786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58e40ceca0 0x7f58e40d38a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.230+0000 7f58eb45a640 1 -- 172.21.15.38:0/3970176786 >> 172.21.15.38:0/3970176786 conn(0x7f58e406bed0 msgr2=0x7f58e406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:53.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.230+0000 7f58eb45a640 1 -- 172.21.15.38:0/3970176786 shutdown_connections 2024-04-25T08:31:53.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.230+0000 7f58eb45a640 1 -- 172.21.15.38:0/3970176786 wait complete. 2024-04-25T08:31:53.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.231+0000 7f58eb45a640 1 Processor -- start 2024-04-25T08:31:53.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.231+0000 7f58eb45a640 1 -- start start 2024-04-25T08:31:53.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58eb45a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e40d3de0 0x7f58e415b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58eb45a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58e415da90 0x7f58e4156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58eb45a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e40cc460 0x7f58e40cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58eb45a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e415de70 con 0x7f58e40d3de0 2024-04-25T08:31:53.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58eb45a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e40d23c0 con 0x7f58e40cc460 2024-04-25T08:31:53.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58eb45a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e415dff0 con 0x7f58e415da90 2024-04-25T08:31:53.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58e91f6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e40d3de0 0x7f58e415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58e91f6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e40d3de0 0x7f58e415b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33304/0 (socket says 172.21.15.38:33304) 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58e91f6640 1 -- 172.21.15.38:0/4248360413 learned_addr learned my addr 172.21.15.38:0/4248360413 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58e89f5640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58e415da90 0x7f58e4156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58e99f7640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e40cc460 0x7f58e40cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58e91f6640 1 -- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58e415da90 msgr2=0x7f58e4156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58e91f6640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58e415da90 0x7f58e4156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58e91f6640 1 -- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e40cc460 msgr2=0x7f58e40cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58e91f6640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e40cc460 0x7f58e40cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.232+0000 7f58e91f6640 1 -- 172.21.15.38:0/4248360413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e40cd400 con 0x7f58e40d3de0 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.233+0000 7f58e91f6640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e40d3de0 0x7f58e415b850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f58e000c430 tx=0x7f58e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.233+0000 7f58d9ffb640 1 -- 172.21.15.38:0/4248360413 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e0017070 con 0x7f58e40d3de0 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.233+0000 7f58eb45a640 1 -- 172.21.15.38:0/4248360413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e40cd6e0 con 0x7f58e40d3de0 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.233+0000 7f58d9ffb640 1 -- 172.21.15.38:0/4248360413 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e0005da0 con 0x7f58e40d3de0 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.233+0000 7f58d9ffb640 1 -- 172.21.15.38:0/4248360413 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e0007310 con 0x7f58e40d3de0 2024-04-25T08:31:53.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.233+0000 7f58eb45a640 1 -- 172.21.15.38:0/4248360413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e40cdca0 con 0x7f58e40d3de0 2024-04-25T08:31:53.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.234+0000 7f58eb45a640 1 -- 172.21.15.38:0/4248360413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e4068e80 con 0x7f58e40d3de0 2024-04-25T08:31:53.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.234+0000 7f58d9ffb640 1 -- 172.21.15.38:0/4248360413 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58e001a070 con 0x7f58e40d3de0 2024-04-25T08:31:53.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.235+0000 7f58d9ffb640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58c806b500 0x7f58c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.235+0000 7f58d9ffb640 1 -- 172.21.15.38:0/4248360413 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f58e0089950 con 0x7f58e40d3de0 2024-04-25T08:31:53.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.235+0000 7f58e89f5640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58c806b500 0x7f58c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.235+0000 7f58e89f5640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58c806b500 0x7f58c806d9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f58d4034000 tx=0x7f58d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:53.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.239+0000 7f58d9ffb640 1 -- 172.21.15.38:0/4248360413 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58e0058410 con 0x7f58e40d3de0 2024-04-25T08:31:53.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.416+0000 7f58eb45a640 1 -- 172.21.15.38:0/4248360413 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58e4159ad0 con 0x7f58c806b500 2024-04-25T08:31:53.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.425+0000 7f58d9ffb640 1 -- 172.21.15.38:0/4248360413 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f58d40325c0 con 0x7f58c806b500 2024-04-25T08:31:53.425 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:53.427 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:31:53.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.428+0000 7f58eb45a640 1 -- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58c806b500 msgr2=0x7f58c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.428+0000 7f58eb45a640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58c806b500 0x7f58c806d9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f58d4034000 tx=0x7f58d402e000 comp rx=0 tx=0).stop 2024-04-25T08:31:53.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.428+0000 7f58eb45a640 1 -- 172.21.15.38:0/4248360413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e40d3de0 msgr2=0x7f58e415b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.428+0000 7f58eb45a640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e40d3de0 0x7f58e415b850 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f58e000c430 tx=0x7f58e000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.428+0000 7f58eb45a640 1 -- 172.21.15.38:0/4248360413 shutdown_connections 2024-04-25T08:31:53.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.428+0000 7f58eb45a640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e40cc460 0x7f58e40cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.428+0000 7f58eb45a640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58c806b500 0x7f58c806d9c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.428+0000 7f58eb45a640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58e415da90 0x7f58e4156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.428+0000 7f58eb45a640 1 --2- 172.21.15.38:0/4248360413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e40d3de0 0x7f58e415b850 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.428+0000 7f58eb45a640 1 -- 172.21.15.38:0/4248360413 >> 172.21.15.38:0/4248360413 conn(0x7f58e406bed0 msgr2=0x7f58e40b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.428+0000 7f58eb45a640 1 -- 172.21.15.38:0/4248360413 shutdown_connections 2024-04-25T08:31:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.428+0000 7f58eb45a640 1 -- 172.21.15.38:0/4248360413 wait complete. 2024-04-25T08:31:53.435 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T08:31:53.207485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223544,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751176,"statfs":{"total":773094113280,"available":772865204224,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.828416"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366309+0000","last_change":"2024-04-25T08:31:35.317991+0000","last_active":"2024-04-25T08:31:42.366309+0000","last_peered":"2024-04-25T08:31:42.366309+0000","last_clean":"2024-04-25T08:31:42.366309+0000","last_became_active":"2024-04-25T08:31:35.317664+0000","last_became_peered":"2024-04-25T08:31:35.317664+0000","last_unstale":"2024-04-25T08:31:42.366309+0000","last_undegraded":"2024-04-25T08:31:42.366309+0000","last_fullsized":"2024-04-25T08:31:42.366309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:56.580134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998066+0000","last_change":"2024-04-25T08:31:36.331619+0000","last_active":"2024-04-25T08:31:41.998066+0000","last_peered":"2024-04-25T08:31:41.998066+0000","last_clean":"2024-04-25T08:31:41.998066+0000","last_became_active":"2024-04-25T08:31:36.331332+0000","last_became_peered":"2024-04-25T08:31:36.331332+0000","last_unstale":"2024-04-25T08:31:41.998066+0000","last_undegraded":"2024-04-25T08:31:41.998066+0000","last_fullsized":"2024-04-25T08:31:41.998066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:12.212803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997585+0000","last_change":"2024-04-25T08:31:40.599503+0000","last_active":"2024-04-25T08:31:41.997585+0000","last_peered":"2024-04-25T08:31:41.997585+0000","last_clean":"2024-04-25T08:31:41.997585+0000","last_became_active":"2024-04-25T08:31:40.599122+0000","last_became_peered":"2024-04-25T08:31:40.599122+0000","last_unstale":"2024-04-25T08:31:41.997585+0000","last_undegraded":"2024-04-25T08:31:41.997585+0000","last_fullsized":"2024-04-25T08:31:41.997585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.735540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998226+0000","last_change":"2024-04-25T08:31:35.318801+0000","last_active":"2024-04-25T08:31:41.998226+0000","last_peered":"2024-04-25T08:31:41.998226+0000","last_clean":"2024-04-25T08:31:41.998226+0000","last_became_active":"2024-04-25T08:31:35.318505+0000","last_became_peered":"2024-04-25T08:31:35.318505+0000","last_unstale":"2024-04-25T08:31:41.998226+0000","last_undegraded":"2024-04-25T08:31:41.998226+0000","last_fullsized":"2024-04-25T08:31:41.998226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365961+0000","last_change":"2024-04-25T08:31:36.323797+0000","last_active":"2024-04-25T08:31:42.365961+0000","last_peered":"2024-04-25T08:31:42.365961+0000","last_clean":"2024-04-25T08:31:42.365961+0000","last_became_active":"2024-04-25T08:31:36.323045+0000","last_became_peered":"2024-04-25T08:31:36.323045+0000","last_unstale":"2024-04-25T08:31:42.365961+0000","last_undegraded":"2024-04-25T08:31:42.365961+0000","last_fullsized":"2024-04-25T08:31:42.365961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:15.226896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381055+0000","last_change":"2024-04-25T08:31:40.356739+0000","last_active":"2024-04-25T08:31:41.381055+0000","last_peered":"2024-04-25T08:31:41.381055+0000","last_clean":"2024-04-25T08:31:41.381055+0000","last_became_active":"2024-04-25T08:31:40.355756+0000","last_became_peered":"2024-04-25T08:31:40.355756+0000","last_unstale":"2024-04-25T08:31:41.381055+0000","last_undegraded":"2024-04-25T08:31:41.381055+0000","last_fullsized":"2024-04-25T08:31:41.381055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:40.045196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422218+0000","last_change":"2024-04-25T08:31:35.316916+0000","last_active":"2024-04-25T08:31:41.422218+0000","last_peered":"2024-04-25T08:31:41.422218+0000","last_clean":"2024-04-25T08:31:41.422218+0000","last_became_active":"2024-04-25T08:31:35.316358+0000","last_became_peered":"2024-04-25T08:31:35.316358+0000","last_unstale":"2024-04-25T08:31:41.422218+0000","last_undegraded":"2024-04-25T08:31:41.422218+0000","last_fullsized":"2024-04-25T08:31:41.422218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:00.982337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119271+0000","last_change":"2024-04-25T08:31:36.323115+0000","last_active":"2024-04-25T08:31:42.119271+0000","last_peered":"2024-04-25T08:31:42.119271+0000","last_clean":"2024-04-25T08:31:42.119271+0000","last_became_active":"2024-04-25T08:31:36.322848+0000","last_became_peered":"2024-04-25T08:31:36.322848+0000","last_unstale":"2024-04-25T08:31:42.119271+0000","last_undegraded":"2024-04-25T08:31:42.119271+0000","last_fullsized":"2024-04-25T08:31:42.119271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382566+0000","last_change":"2024-04-25T08:31:40.357214+0000","last_active":"2024-04-25T08:31:41.382566+0000","last_peered":"2024-04-25T08:31:41.382566+0000","last_clean":"2024-04-25T08:31:41.382566+0000","last_became_active":"2024-04-25T08:31:40.355877+0000","last_became_peered":"2024-04-25T08:31:40.355877+0000","last_unstale":"2024-04-25T08:31:41.382566+0000","last_undegraded":"2024-04-25T08:31:41.382566+0000","last_fullsized":"2024-04-25T08:31:41.382566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422059+0000","last_change":"2024-04-25T08:31:35.319905+0000","last_active":"2024-04-25T08:31:41.422059+0000","last_peered":"2024-04-25T08:31:41.422059+0000","last_clean":"2024-04-25T08:31:41.422059+0000","last_became_active":"2024-04-25T08:31:35.319127+0000","last_became_peered":"2024-04-25T08:31:35.319127+0000","last_unstale":"2024-04-25T08:31:41.422059+0000","last_undegraded":"2024-04-25T08:31:41.422059+0000","last_fullsized":"2024-04-25T08:31:41.422059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:43.779456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119380+0000","last_change":"2024-04-25T08:31:36.322498+0000","last_active":"2024-04-25T08:31:42.119380+0000","last_peered":"2024-04-25T08:31:42.119380+0000","last_clean":"2024-04-25T08:31:42.119380+0000","last_became_active":"2024-04-25T08:31:36.322250+0000","last_became_peered":"2024-04-25T08:31:36.322250+0000","last_unstale":"2024-04-25T08:31:42.119380+0000","last_undegraded":"2024-04-25T08:31:42.119380+0000","last_fullsized":"2024-04-25T08:31:42.119380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.111565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381899+0000","last_change":"2024-04-25T08:31:40.352372+0000","last_active":"2024-04-25T08:31:41.381899+0000","last_peered":"2024-04-25T08:31:41.381899+0000","last_clean":"2024-04-25T08:31:41.381899+0000","last_became_active":"2024-04-25T08:31:40.351860+0000","last_became_peered":"2024-04-25T08:31:40.351860+0000","last_unstale":"2024-04-25T08:31:41.381899+0000","last_undegraded":"2024-04-25T08:31:41.381899+0000","last_fullsized":"2024-04-25T08:31:41.381899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:50.728874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381228+0000","last_change":"2024-04-25T08:31:36.321222+0000","last_active":"2024-04-25T08:31:41.381228+0000","last_peered":"2024-04-25T08:31:41.381228+0000","last_clean":"2024-04-25T08:31:41.381228+0000","last_became_active":"2024-04-25T08:31:36.320947+0000","last_became_peered":"2024-04-25T08:31:36.320947+0000","last_unstale":"2024-04-25T08:31:41.381228+0000","last_undegraded":"2024-04-25T08:31:41.381228+0000","last_fullsized":"2024-04-25T08:31:41.381228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421943+0000","last_change":"2024-04-25T08:31:35.316969+0000","last_active":"2024-04-25T08:31:41.421943+0000","last_peered":"2024-04-25T08:31:41.421943+0000","last_clean":"2024-04-25T08:31:41.421943+0000","last_became_active":"2024-04-25T08:31:35.316693+0000","last_became_peered":"2024-04-25T08:31:35.316693+0000","last_unstale":"2024-04-25T08:31:41.421943+0000","last_undegraded":"2024-04-25T08:31:41.421943+0000","last_fullsized":"2024-04-25T08:31:41.421943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:20.652651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382507+0000","last_change":"2024-04-25T08:31:40.351507+0000","last_active":"2024-04-25T08:31:41.382507+0000","last_peered":"2024-04-25T08:31:41.382507+0000","last_clean":"2024-04-25T08:31:41.382507+0000","last_became_active":"2024-04-25T08:31:40.351053+0000","last_became_peered":"2024-04-25T08:31:40.351053+0000","last_unstale":"2024-04-25T08:31:41.382507+0000","last_undegraded":"2024-04-25T08:31:41.382507+0000","last_fullsized":"2024-04-25T08:31:41.382507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997831+0000","last_change":"2024-04-25T08:31:36.321459+0000","last_active":"2024-04-25T08:31:41.997831+0000","last_peered":"2024-04-25T08:31:41.997831+0000","last_clean":"2024-04-25T08:31:41.997831+0000","last_became_active":"2024-04-25T08:31:36.321137+0000","last_became_peered":"2024-04-25T08:31:36.321137+0000","last_unstale":"2024-04-25T08:31:41.997831+0000","last_undegraded":"2024-04-25T08:31:41.997831+0000","last_fullsized":"2024-04-25T08:31:41.997831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:14.177622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366532+0000","last_change":"2024-04-25T08:31:35.317075+0000","last_active":"2024-04-25T08:31:42.366532+0000","last_peered":"2024-04-25T08:31:42.366532+0000","last_clean":"2024-04-25T08:31:42.366532+0000","last_became_active":"2024-04-25T08:31:35.316764+0000","last_became_peered":"2024-04-25T08:31:35.316764+0000","last_unstale":"2024-04-25T08:31:42.366532+0000","last_undegraded":"2024-04-25T08:31:42.366532+0000","last_fullsized":"2024-04-25T08:31:42.366532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382109+0000","last_change":"2024-04-25T08:31:40.352414+0000","last_active":"2024-04-25T08:31:41.382109+0000","last_peered":"2024-04-25T08:31:41.382109+0000","last_clean":"2024-04-25T08:31:41.382109+0000","last_became_active":"2024-04-25T08:31:40.351024+0000","last_became_peered":"2024-04-25T08:31:40.351024+0000","last_unstale":"2024-04-25T08:31:41.382109+0000","last_undegraded":"2024-04-25T08:31:41.382109+0000","last_fullsized":"2024-04-25T08:31:41.382109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:16.241561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997717+0000","last_change":"2024-04-25T08:31:36.320330+0000","last_active":"2024-04-25T08:31:41.997717+0000","last_peered":"2024-04-25T08:31:41.997717+0000","last_clean":"2024-04-25T08:31:41.997717+0000","last_became_active":"2024-04-25T08:31:36.313715+0000","last_became_peered":"2024-04-25T08:31:36.313715+0000","last_unstale":"2024-04-25T08:31:41.997717+0000","last_undegraded":"2024-04-25T08:31:41.997717+0000","last_fullsized":"2024-04-25T08:31:41.997717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367128+0000","last_change":"2024-04-25T08:31:35.314788+0000","last_active":"2024-04-25T08:31:42.367128+0000","last_peered":"2024-04-25T08:31:42.367128+0000","last_clean":"2024-04-25T08:31:42.367128+0000","last_became_active":"2024-04-25T08:31:35.314353+0000","last_became_peered":"2024-04-25T08:31:35.314353+0000","last_unstale":"2024-04-25T08:31:42.367128+0000","last_undegraded":"2024-04-25T08:31:42.367128+0000","last_fullsized":"2024-04-25T08:31:42.367128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.726901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367441+0000","last_change":"2024-04-25T08:31:40.594652+0000","last_active":"2024-04-25T08:31:42.367441+0000","last_peered":"2024-04-25T08:31:42.367441+0000","last_clean":"2024-04-25T08:31:42.367441+0000","last_became_active":"2024-04-25T08:31:40.594138+0000","last_became_peered":"2024-04-25T08:31:40.594138+0000","last_unstale":"2024-04-25T08:31:42.367441+0000","last_undegraded":"2024-04-25T08:31:42.367441+0000","last_fullsized":"2024-04-25T08:31:42.367441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:23.148774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381474+0000","last_change":"2024-04-25T08:31:36.324636+0000","last_active":"2024-04-25T08:31:41.381474+0000","last_peered":"2024-04-25T08:31:41.381474+0000","last_clean":"2024-04-25T08:31:41.381474+0000","last_became_active":"2024-04-25T08:31:36.322240+0000","last_became_peered":"2024-04-25T08:31:36.322240+0000","last_unstale":"2024-04-25T08:31:41.381474+0000","last_undegraded":"2024-04-25T08:31:41.381474+0000","last_fullsized":"2024-04-25T08:31:41.381474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:49.809945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421616+0000","last_change":"2024-04-25T08:31:35.318327+0000","last_active":"2024-04-25T08:31:41.421616+0000","last_peered":"2024-04-25T08:31:41.421616+0000","last_clean":"2024-04-25T08:31:41.421616+0000","last_became_active":"2024-04-25T08:31:35.317749+0000","last_became_peered":"2024-04-25T08:31:35.317749+0000","last_unstale":"2024-04-25T08:31:41.421616+0000","last_undegraded":"2024-04-25T08:31:41.421616+0000","last_fullsized":"2024-04-25T08:31:41.421616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:46.434047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382327+0000","last_change":"2024-04-25T08:31:40.357220+0000","last_active":"2024-04-25T08:31:41.382327+0000","last_peered":"2024-04-25T08:31:41.382327+0000","last_clean":"2024-04-25T08:31:41.382327+0000","last_became_active":"2024-04-25T08:31:40.355770+0000","last_became_peered":"2024-04-25T08:31:40.355770+0000","last_unstale":"2024-04-25T08:31:41.382327+0000","last_undegraded":"2024-04-25T08:31:41.382327+0000","last_fullsized":"2024-04-25T08:31:41.382327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:04.998392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366774+0000","last_change":"2024-04-25T08:31:36.312756+0000","last_active":"2024-04-25T08:31:42.366774+0000","last_peered":"2024-04-25T08:31:42.366774+0000","last_clean":"2024-04-25T08:31:42.366774+0000","last_became_active":"2024-04-25T08:31:36.312492+0000","last_became_peered":"2024-04-25T08:31:36.312492+0000","last_unstale":"2024-04-25T08:31:42.366774+0000","last_undegraded":"2024-04-25T08:31:42.366774+0000","last_fullsized":"2024-04-25T08:31:42.366774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382375+0000","last_change":"2024-04-25T08:31:35.314188+0000","last_active":"2024-04-25T08:31:41.382375+0000","last_peered":"2024-04-25T08:31:41.382375+0000","last_clean":"2024-04-25T08:31:41.382375+0000","last_became_active":"2024-04-25T08:31:35.313682+0000","last_became_peered":"2024-04-25T08:31:35.313682+0000","last_unstale":"2024-04-25T08:31:41.382375+0000","last_undegraded":"2024-04-25T08:31:41.382375+0000","last_fullsized":"2024-04-25T08:31:41.382375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.726901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381316+0000","last_change":"2024-04-25T08:31:40.596307+0000","last_active":"2024-04-25T08:31:41.381316+0000","last_peered":"2024-04-25T08:31:41.381316+0000","last_clean":"2024-04-25T08:31:41.381316+0000","last_became_active":"2024-04-25T08:31:40.595853+0000","last_became_peered":"2024-04-25T08:31:40.595853+0000","last_unstale":"2024-04-25T08:31:41.381316+0000","last_undegraded":"2024-04-25T08:31:41.381316+0000","last_fullsized":"2024-04-25T08:31:41.381316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:18.206380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998305+0000","last_change":"2024-04-25T08:31:36.327135+0000","last_active":"2024-04-25T08:31:41.998305+0000","last_peered":"2024-04-25T08:31:41.998305+0000","last_clean":"2024-04-25T08:31:41.998305+0000","last_became_active":"2024-04-25T08:31:36.326839+0000","last_became_peered":"2024-04-25T08:31:36.326839+0000","last_unstale":"2024-04-25T08:31:41.998305+0000","last_undegraded":"2024-04-25T08:31:41.998305+0000","last_fullsized":"2024-04-25T08:31:41.998305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:47.399399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367454+0000","last_change":"2024-04-25T08:31:35.309749+0000","last_active":"2024-04-25T08:31:42.367454+0000","last_peered":"2024-04-25T08:31:42.367454+0000","last_clean":"2024-04-25T08:31:42.367454+0000","last_became_active":"2024-04-25T08:31:35.309352+0000","last_became_peered":"2024-04-25T08:31:35.309352+0000","last_unstale":"2024-04-25T08:31:42.367454+0000","last_undegraded":"2024-04-25T08:31:42.367454+0000","last_fullsized":"2024-04-25T08:31:42.367454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:54.173888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997493+0000","last_change":"2024-04-25T08:31:40.598224+0000","last_active":"2024-04-25T08:31:41.997493+0000","last_peered":"2024-04-25T08:31:41.997493+0000","last_clean":"2024-04-25T08:31:41.997493+0000","last_became_active":"2024-04-25T08:31:40.597800+0000","last_became_peered":"2024-04-25T08:31:40.597800+0000","last_unstale":"2024-04-25T08:31:41.997493+0000","last_undegraded":"2024-04-25T08:31:41.997493+0000","last_fullsized":"2024-04-25T08:31:41.997493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:10.902214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367151+0000","last_change":"2024-04-25T08:31:36.325531+0000","last_active":"2024-04-25T08:31:42.367151+0000","last_peered":"2024-04-25T08:31:42.367151+0000","last_clean":"2024-04-25T08:31:42.367151+0000","last_became_active":"2024-04-25T08:31:36.325240+0000","last_became_peered":"2024-04-25T08:31:36.325240+0000","last_unstale":"2024-04-25T08:31:42.367151+0000","last_undegraded":"2024-04-25T08:31:42.367151+0000","last_fullsized":"2024-04-25T08:31:42.367151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422321+0000","last_change":"2024-04-25T08:31:35.317032+0000","last_active":"2024-04-25T08:31:41.422321+0000","last_peered":"2024-04-25T08:31:41.422321+0000","last_clean":"2024-04-25T08:31:41.422321+0000","last_became_active":"2024-04-25T08:31:35.316659+0000","last_became_peered":"2024-04-25T08:31:35.316659+0000","last_unstale":"2024-04-25T08:31:41.422321+0000","last_undegraded":"2024-04-25T08:31:41.422321+0000","last_fullsized":"2024-04-25T08:31:41.422321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:57.762013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366270+0000","last_change":"2024-04-25T08:31:40.595457+0000","last_active":"2024-04-25T08:31:42.366270+0000","last_peered":"2024-04-25T08:31:42.366270+0000","last_clean":"2024-04-25T08:31:42.366270+0000","last_became_active":"2024-04-25T08:31:40.595077+0000","last_became_peered":"2024-04-25T08:31:40.595077+0000","last_unstale":"2024-04-25T08:31:42.366270+0000","last_undegraded":"2024-04-25T08:31:42.366270+0000","last_fullsized":"2024-04-25T08:31:42.366270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119506+0000","last_change":"2024-04-25T08:31:36.322361+0000","last_active":"2024-04-25T08:31:42.119506+0000","last_peered":"2024-04-25T08:31:42.119506+0000","last_clean":"2024-04-25T08:31:42.119506+0000","last_became_active":"2024-04-25T08:31:36.322107+0000","last_became_peered":"2024-04-25T08:31:36.322107+0000","last_unstale":"2024-04-25T08:31:42.119506+0000","last_undegraded":"2024-04-25T08:31:42.119506+0000","last_fullsized":"2024-04-25T08:31:42.119506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:51.984540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367254+0000","last_change":"2024-04-25T08:31:35.315993+0000","last_active":"2024-04-25T08:31:42.367254+0000","last_peered":"2024-04-25T08:31:42.367254+0000","last_clean":"2024-04-25T08:31:42.367254+0000","last_became_active":"2024-04-25T08:31:35.315664+0000","last_became_peered":"2024-04-25T08:31:35.315664+0000","last_unstale":"2024-04-25T08:31:42.367254+0000","last_undegraded":"2024-04-25T08:31:42.367254+0000","last_fullsized":"2024-04-25T08:31:42.367254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365979+0000","last_change":"2024-04-25T08:31:40.355607+0000","last_active":"2024-04-25T08:31:42.365979+0000","last_peered":"2024-04-25T08:31:42.365979+0000","last_clean":"2024-04-25T08:31:42.365979+0000","last_became_active":"2024-04-25T08:31:40.355024+0000","last_became_peered":"2024-04-25T08:31:40.355024+0000","last_unstale":"2024-04-25T08:31:42.365979+0000","last_undegraded":"2024-04-25T08:31:42.365979+0000","last_fullsized":"2024-04-25T08:31:42.365979+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382608+0000","last_change":"2024-04-25T08:31:36.323675+0000","last_active":"2024-04-25T08:31:41.382608+0000","last_peered":"2024-04-25T08:31:41.382608+0000","last_clean":"2024-04-25T08:31:41.382608+0000","last_became_active":"2024-04-25T08:31:36.323102+0000","last_became_peered":"2024-04-25T08:31:36.323102+0000","last_unstale":"2024-04-25T08:31:41.382608+0000","last_undegraded":"2024-04-25T08:31:41.382608+0000","last_fullsized":"2024-04-25T08:31:41.382608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:49.744475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119335+0000","last_change":"2024-04-25T08:31:35.317234+0000","last_active":"2024-04-25T08:31:42.119335+0000","last_peered":"2024-04-25T08:31:42.119335+0000","last_clean":"2024-04-25T08:31:42.119335+0000","last_became_active":"2024-04-25T08:31:35.316747+0000","last_became_peered":"2024-04-25T08:31:35.316747+0000","last_unstale":"2024-04-25T08:31:42.119335+0000","last_undegraded":"2024-04-25T08:31:42.119335+0000","last_fullsized":"2024-04-25T08:31:42.119335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367030+0000","last_change":"2024-04-25T08:31:40.352712+0000","last_active":"2024-04-25T08:31:42.367030+0000","last_peered":"2024-04-25T08:31:42.367030+0000","last_clean":"2024-04-25T08:31:42.367030+0000","last_became_active":"2024-04-25T08:31:40.352328+0000","last_became_peered":"2024-04-25T08:31:40.352328+0000","last_unstale":"2024-04-25T08:31:42.367030+0000","last_undegraded":"2024-04-25T08:31:42.367030+0000","last_fullsized":"2024-04-25T08:31:42.367030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:56.013298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366618+0000","last_change":"2024-04-25T08:31:36.326854+0000","last_active":"2024-04-25T08:31:42.366618+0000","last_peered":"2024-04-25T08:31:42.366618+0000","last_clean":"2024-04-25T08:31:42.366618+0000","last_became_active":"2024-04-25T08:31:36.326564+0000","last_became_peered":"2024-04-25T08:31:36.326564+0000","last_unstale":"2024-04-25T08:31:42.366618+0000","last_undegraded":"2024-04-25T08:31:42.366618+0000","last_fullsized":"2024-04-25T08:31:42.366618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384554+0000","last_change":"2024-04-25T08:31:35.317497+0000","last_active":"2024-04-25T08:31:41.384554+0000","last_peered":"2024-04-25T08:31:41.384554+0000","last_clean":"2024-04-25T08:31:41.384554+0000","last_became_active":"2024-04-25T08:31:35.317202+0000","last_became_peered":"2024-04-25T08:31:35.317202+0000","last_unstale":"2024-04-25T08:31:41.384554+0000","last_undegraded":"2024-04-25T08:31:41.384554+0000","last_fullsized":"2024-04-25T08:31:41.384554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367456+0000","last_change":"2024-04-25T08:31:40.352169+0000","last_active":"2024-04-25T08:31:42.367456+0000","last_peered":"2024-04-25T08:31:42.367456+0000","last_clean":"2024-04-25T08:31:42.367456+0000","last_became_active":"2024-04-25T08:31:40.351743+0000","last_became_peered":"2024-04-25T08:31:40.351743+0000","last_unstale":"2024-04-25T08:31:42.367456+0000","last_undegraded":"2024-04-25T08:31:42.367456+0000","last_fullsized":"2024-04-25T08:31:42.367456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:09.948828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365773+0000","last_change":"2024-04-25T08:31:36.324307+0000","last_active":"2024-04-25T08:31:42.365773+0000","last_peered":"2024-04-25T08:31:42.365773+0000","last_clean":"2024-04-25T08:31:42.365773+0000","last_became_active":"2024-04-25T08:31:36.324011+0000","last_became_peered":"2024-04-25T08:31:36.324011+0000","last_unstale":"2024-04-25T08:31:42.365773+0000","last_undegraded":"2024-04-25T08:31:42.365773+0000","last_fullsized":"2024-04-25T08:31:42.365773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:01.999912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421648+0000","last_change":"2024-04-25T08:31:35.317490+0000","last_active":"2024-04-25T08:31:41.421648+0000","last_peered":"2024-04-25T08:31:41.421648+0000","last_clean":"2024-04-25T08:31:41.421648+0000","last_became_active":"2024-04-25T08:31:35.316569+0000","last_became_peered":"2024-04-25T08:31:35.316569+0000","last_unstale":"2024-04-25T08:31:41.421648+0000","last_undegraded":"2024-04-25T08:31:41.421648+0000","last_fullsized":"2024-04-25T08:31:41.421648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:09.703716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366542+0000","last_change":"2024-04-25T08:31:40.598075+0000","last_active":"2024-04-25T08:31:42.366542+0000","last_peered":"2024-04-25T08:31:42.366542+0000","last_clean":"2024-04-25T08:31:42.366542+0000","last_became_active":"2024-04-25T08:31:40.597669+0000","last_became_peered":"2024-04-25T08:31:40.597669+0000","last_unstale":"2024-04-25T08:31:42.366542+0000","last_undegraded":"2024-04-25T08:31:42.366542+0000","last_fullsized":"2024-04-25T08:31:42.366542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:09.948828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997776+0000","last_change":"2024-04-25T08:31:36.320808+0000","last_active":"2024-04-25T08:31:41.997776+0000","last_peered":"2024-04-25T08:31:41.997776+0000","last_clean":"2024-04-25T08:31:41.997776+0000","last_became_active":"2024-04-25T08:31:36.320489+0000","last_became_peered":"2024-04-25T08:31:36.320489+0000","last_unstale":"2024-04-25T08:31:41.997776+0000","last_undegraded":"2024-04-25T08:31:41.997776+0000","last_fullsized":"2024-04-25T08:31:41.997776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119140+0000","last_change":"2024-04-25T08:31:35.312515+0000","last_active":"2024-04-25T08:31:42.119140+0000","last_peered":"2024-04-25T08:31:42.119140+0000","last_clean":"2024-04-25T08:31:42.119140+0000","last_became_active":"2024-04-25T08:31:35.312156+0000","last_became_peered":"2024-04-25T08:31:35.312156+0000","last_unstale":"2024-04-25T08:31:42.119140+0000","last_undegraded":"2024-04-25T08:31:42.119140+0000","last_fullsized":"2024-04-25T08:31:42.119140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421744+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:31:41.421744+0000","last_peered":"2024-04-25T08:31:41.421744+0000","last_clean":"2024-04-25T08:31:41.421744+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:31:41.421744+0000","last_undegraded":"2024-04-25T08:31:41.421744+0000","last_fullsized":"2024-04-25T08:31:41.421744+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381120+0000","last_change":"2024-04-25T08:31:40.597789+0000","last_active":"2024-04-25T08:31:41.381120+0000","last_peered":"2024-04-25T08:31:41.381120+0000","last_clean":"2024-04-25T08:31:41.381120+0000","last_became_active":"2024-04-25T08:31:40.597373+0000","last_became_peered":"2024-04-25T08:31:40.597373+0000","last_unstale":"2024-04-25T08:31:41.381120+0000","last_undegraded":"2024-04-25T08:31:41.381120+0000","last_fullsized":"2024-04-25T08:31:41.381120+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422089+0000","last_change":"2024-04-25T08:31:36.323510+0000","last_active":"2024-04-25T08:31:41.422089+0000","last_peered":"2024-04-25T08:31:41.422089+0000","last_clean":"2024-04-25T08:31:41.422089+0000","last_became_active":"2024-04-25T08:31:36.322941+0000","last_became_peered":"2024-04-25T08:31:36.322941+0000","last_unstale":"2024-04-25T08:31:41.422089+0000","last_undegraded":"2024-04-25T08:31:41.422089+0000","last_fullsized":"2024-04-25T08:31:41.422089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:51.984540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384803+0000","last_change":"2024-04-25T08:31:35.318779+0000","last_active":"2024-04-25T08:31:41.384803+0000","last_peered":"2024-04-25T08:31:41.384803+0000","last_clean":"2024-04-25T08:31:41.384803+0000","last_became_active":"2024-04-25T08:31:35.318485+0000","last_became_peered":"2024-04-25T08:31:35.318485+0000","last_unstale":"2024-04-25T08:31:41.384803+0000","last_undegraded":"2024-04-25T08:31:41.384803+0000","last_fullsized":"2024-04-25T08:31:41.384803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381168+0000","last_change":"2024-04-25T08:31:40.594633+0000","last_active":"2024-04-25T08:31:41.381168+0000","last_peered":"2024-04-25T08:31:41.381168+0000","last_clean":"2024-04-25T08:31:41.381168+0000","last_became_active":"2024-04-25T08:31:40.594193+0000","last_became_peered":"2024-04-25T08:31:40.594193+0000","last_unstale":"2024-04-25T08:31:41.381168+0000","last_undegraded":"2024-04-25T08:31:41.381168+0000","last_fullsized":"2024-04-25T08:31:41.381168+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119593+0000","last_change":"2024-04-25T08:31:36.327715+0000","last_active":"2024-04-25T08:31:42.119593+0000","last_peered":"2024-04-25T08:31:42.119593+0000","last_clean":"2024-04-25T08:31:42.119593+0000","last_became_active":"2024-04-25T08:31:36.327408+0000","last_became_peered":"2024-04-25T08:31:36.327408+0000","last_unstale":"2024-04-25T08:31:42.119593+0000","last_undegraded":"2024-04-25T08:31:42.119593+0000","last_fullsized":"2024-04-25T08:31:42.119593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:43.612460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367236+0000","last_change":"2024-04-25T08:31:35.310104+0000","last_active":"2024-04-25T08:31:42.367236+0000","last_peered":"2024-04-25T08:31:42.367236+0000","last_clean":"2024-04-25T08:31:42.367236+0000","last_became_active":"2024-04-25T08:31:35.309469+0000","last_became_peered":"2024-04-25T08:31:35.309469+0000","last_unstale":"2024-04-25T08:31:42.367236+0000","last_undegraded":"2024-04-25T08:31:42.367236+0000","last_fullsized":"2024-04-25T08:31:42.367236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:09.703716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366537+0000","last_change":"2024-04-25T08:31:40.356105+0000","last_active":"2024-04-25T08:31:42.366537+0000","last_peered":"2024-04-25T08:31:42.366537+0000","last_clean":"2024-04-25T08:31:42.366537+0000","last_became_active":"2024-04-25T08:31:40.355734+0000","last_became_peered":"2024-04-25T08:31:40.355734+0000","last_unstale":"2024-04-25T08:31:42.366537+0000","last_undegraded":"2024-04-25T08:31:42.366537+0000","last_fullsized":"2024-04-25T08:31:42.366537+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:23.148774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422428+0000","last_change":"2024-04-25T08:31:36.325987+0000","last_active":"2024-04-25T08:31:41.422428+0000","last_peered":"2024-04-25T08:31:41.422428+0000","last_clean":"2024-04-25T08:31:41.422428+0000","last_became_active":"2024-04-25T08:31:36.324131+0000","last_became_peered":"2024-04-25T08:31:36.324131+0000","last_unstale":"2024-04-25T08:31:41.422428+0000","last_undegraded":"2024-04-25T08:31:41.422428+0000","last_fullsized":"2024-04-25T08:31:41.422428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:32.255703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384855+0000","last_change":"2024-04-25T08:31:35.321968+0000","last_active":"2024-04-25T08:31:41.384855+0000","last_peered":"2024-04-25T08:31:41.384855+0000","last_clean":"2024-04-25T08:31:41.384855+0000","last_became_active":"2024-04-25T08:31:35.321609+0000","last_became_peered":"2024-04-25T08:31:35.321609+0000","last_unstale":"2024-04-25T08:31:41.384855+0000","last_undegraded":"2024-04-25T08:31:41.384855+0000","last_fullsized":"2024-04-25T08:31:41.384855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118903+0000","last_change":"2024-04-25T08:31:40.597471+0000","last_active":"2024-04-25T08:31:42.118903+0000","last_peered":"2024-04-25T08:31:42.118903+0000","last_clean":"2024-04-25T08:31:42.118903+0000","last_became_active":"2024-04-25T08:31:40.597017+0000","last_became_peered":"2024-04-25T08:31:40.597017+0000","last_unstale":"2024-04-25T08:31:42.118903+0000","last_undegraded":"2024-04-25T08:31:42.118903+0000","last_fullsized":"2024-04-25T08:31:42.118903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:10.902214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422271+0000","last_change":"2024-04-25T08:31:36.325470+0000","last_active":"2024-04-25T08:31:41.422271+0000","last_peered":"2024-04-25T08:31:41.422271+0000","last_clean":"2024-04-25T08:31:41.422271+0000","last_became_active":"2024-04-25T08:31:36.325177+0000","last_became_peered":"2024-04-25T08:31:36.325177+0000","last_unstale":"2024-04-25T08:31:41.422271+0000","last_undegraded":"2024-04-25T08:31:41.422271+0000","last_fullsized":"2024-04-25T08:31:41.422271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:19.120016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381794+0000","last_change":"2024-04-25T08:31:35.312440+0000","last_active":"2024-04-25T08:31:41.381794+0000","last_peered":"2024-04-25T08:31:41.381794+0000","last_clean":"2024-04-25T08:31:41.381794+0000","last_became_active":"2024-04-25T08:31:35.312130+0000","last_became_peered":"2024-04-25T08:31:35.312130+0000","last_unstale":"2024-04-25T08:31:41.381794+0000","last_undegraded":"2024-04-25T08:31:41.381794+0000","last_fullsized":"2024-04-25T08:31:41.381794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:32.280995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381222+0000","last_change":"2024-04-25T08:31:40.351471+0000","last_active":"2024-04-25T08:31:41.381222+0000","last_peered":"2024-04-25T08:31:41.381222+0000","last_clean":"2024-04-25T08:31:41.381222+0000","last_became_active":"2024-04-25T08:31:40.351036+0000","last_became_peered":"2024-04-25T08:31:40.351036+0000","last_unstale":"2024-04-25T08:31:41.381222+0000","last_undegraded":"2024-04-25T08:31:41.381222+0000","last_fullsized":"2024-04-25T08:31:41.381222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:30.214541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421850+0000","last_change":"2024-04-25T08:31:36.325960+0000","last_active":"2024-04-25T08:31:41.421850+0000","last_peered":"2024-04-25T08:31:41.421850+0000","last_clean":"2024-04-25T08:31:41.421850+0000","last_became_active":"2024-04-25T08:31:36.325667+0000","last_became_peered":"2024-04-25T08:31:36.325667+0000","last_unstale":"2024-04-25T08:31:41.421850+0000","last_undegraded":"2024-04-25T08:31:41.421850+0000","last_fullsized":"2024-04-25T08:31:41.421850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:47.399399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384402+0000","last_change":"2024-04-25T08:31:35.307681+0000","last_active":"2024-04-25T08:31:41.384402+0000","last_peered":"2024-04-25T08:31:41.384402+0000","last_clean":"2024-04-25T08:31:41.384402+0000","last_became_active":"2024-04-25T08:31:35.307213+0000","last_became_peered":"2024-04-25T08:31:35.307213+0000","last_unstale":"2024-04-25T08:31:41.384402+0000","last_undegraded":"2024-04-25T08:31:41.384402+0000","last_fullsized":"2024-04-25T08:31:41.384402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:34.641593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366369+0000","last_change":"2024-04-25T08:31:40.352841+0000","last_active":"2024-04-25T08:31:42.366369+0000","last_peered":"2024-04-25T08:31:42.366369+0000","last_clean":"2024-04-25T08:31:42.366369+0000","last_became_active":"2024-04-25T08:31:40.352418+0000","last_became_peered":"2024-04-25T08:31:40.352418+0000","last_unstale":"2024-04-25T08:31:42.366369+0000","last_undegraded":"2024-04-25T08:31:42.366369+0000","last_fullsized":"2024-04-25T08:31:42.366369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:56.013298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.380985+0000","last_change":"2024-04-25T08:31:36.312845+0000","last_active":"2024-04-25T08:31:41.380985+0000","last_peered":"2024-04-25T08:31:41.380985+0000","last_clean":"2024-04-25T08:31:41.380985+0000","last_became_active":"2024-04-25T08:31:36.312538+0000","last_became_peered":"2024-04-25T08:31:36.312538+0000","last_unstale":"2024-04-25T08:31:41.380985+0000","last_undegraded":"2024-04-25T08:31:41.380985+0000","last_fullsized":"2024-04-25T08:31:41.380985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381431+0000","last_change":"2024-04-25T08:31:35.305736+0000","last_active":"2024-04-25T08:31:41.381431+0000","last_peered":"2024-04-25T08:31:41.381431+0000","last_clean":"2024-04-25T08:31:41.381431+0000","last_became_active":"2024-04-25T08:31:35.305481+0000","last_became_peered":"2024-04-25T08:31:35.305481+0000","last_unstale":"2024-04-25T08:31:41.381431+0000","last_undegraded":"2024-04-25T08:31:41.381431+0000","last_fullsized":"2024-04-25T08:31:41.381431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998328+0000","last_change":"2024-04-25T08:31:40.599441+0000","last_active":"2024-04-25T08:31:41.998328+0000","last_peered":"2024-04-25T08:31:41.998328+0000","last_clean":"2024-04-25T08:31:41.998328+0000","last_became_active":"2024-04-25T08:31:40.599023+0000","last_became_peered":"2024-04-25T08:31:40.599023+0000","last_unstale":"2024-04-25T08:31:41.998328+0000","last_undegraded":"2024-04-25T08:31:41.998328+0000","last_fullsized":"2024-04-25T08:31:41.998328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:09.443843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421936+0000","last_change":"2024-04-25T08:31:36.323783+0000","last_active":"2024-04-25T08:31:41.421936+0000","last_peered":"2024-04-25T08:31:41.421936+0000","last_clean":"2024-04-25T08:31:41.421936+0000","last_became_active":"2024-04-25T08:31:36.323460+0000","last_became_peered":"2024-04-25T08:31:36.323460+0000","last_unstale":"2024-04-25T08:31:41.421936+0000","last_undegraded":"2024-04-25T08:31:41.421936+0000","last_fullsized":"2024-04-25T08:31:41.421936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119196+0000","last_change":"2024-04-25T08:31:35.318378+0000","last_active":"2024-04-25T08:31:42.119196+0000","last_peered":"2024-04-25T08:31:42.119196+0000","last_clean":"2024-04-25T08:31:42.119196+0000","last_became_active":"2024-04-25T08:31:35.318103+0000","last_became_peered":"2024-04-25T08:31:35.318103+0000","last_unstale":"2024-04-25T08:31:42.119196+0000","last_undegraded":"2024-04-25T08:31:42.119196+0000","last_fullsized":"2024-04-25T08:31:42.119196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998302+0000","last_change":"2024-04-25T08:31:40.595629+0000","last_active":"2024-04-25T08:31:41.998302+0000","last_peered":"2024-04-25T08:31:41.998302+0000","last_clean":"2024-04-25T08:31:41.998302+0000","last_became_active":"2024-04-25T08:31:40.594160+0000","last_became_peered":"2024-04-25T08:31:40.594160+0000","last_unstale":"2024-04-25T08:31:41.998302+0000","last_undegraded":"2024-04-25T08:31:41.998302+0000","last_fullsized":"2024-04-25T08:31:41.998302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:00.245750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366188+0000","last_change":"2024-04-25T08:31:36.329619+0000","last_active":"2024-04-25T08:31:42.366188+0000","last_peered":"2024-04-25T08:31:42.366188+0000","last_clean":"2024-04-25T08:31:42.366188+0000","last_became_active":"2024-04-25T08:31:36.329319+0000","last_became_peered":"2024-04-25T08:31:36.329319+0000","last_unstale":"2024-04-25T08:31:42.366188+0000","last_undegraded":"2024-04-25T08:31:42.366188+0000","last_fullsized":"2024-04-25T08:31:42.366188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:32.370451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366817+0000","last_change":"2024-04-25T08:31:35.317195+0000","last_active":"2024-04-25T08:31:42.366817+0000","last_peered":"2024-04-25T08:31:42.366817+0000","last_clean":"2024-04-25T08:31:42.366817+0000","last_became_active":"2024-04-25T08:31:35.316819+0000","last_became_peered":"2024-04-25T08:31:35.316819+0000","last_unstale":"2024-04-25T08:31:42.366817+0000","last_undegraded":"2024-04-25T08:31:42.366817+0000","last_fullsized":"2024-04-25T08:31:42.366817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:32.280995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119540+0000","last_change":"2024-04-25T08:31:40.594853+0000","last_active":"2024-04-25T08:31:42.119540+0000","last_peered":"2024-04-25T08:31:42.119540+0000","last_clean":"2024-04-25T08:31:42.119540+0000","last_became_active":"2024-04-25T08:31:40.594456+0000","last_became_peered":"2024-04-25T08:31:40.594456+0000","last_unstale":"2024-04-25T08:31:42.119540+0000","last_undegraded":"2024-04-25T08:31:42.119540+0000","last_fullsized":"2024-04-25T08:31:42.119540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.735540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422019+0000","last_change":"2024-04-25T08:31:36.324840+0000","last_active":"2024-04-25T08:31:41.422019+0000","last_peered":"2024-04-25T08:31:41.422019+0000","last_clean":"2024-04-25T08:31:41.422019+0000","last_became_active":"2024-04-25T08:31:36.324574+0000","last_became_peered":"2024-04-25T08:31:36.324574+0000","last_unstale":"2024-04-25T08:31:41.422019+0000","last_undegraded":"2024-04-25T08:31:41.422019+0000","last_fullsized":"2024-04-25T08:31:41.422019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.111565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:45.647660+0000","last_change":"2024-04-25T08:31:35.316767+0000","last_active":"2024-04-25T08:31:45.647660+0000","last_peered":"2024-04-25T08:31:45.647660+0000","last_clean":"2024-04-25T08:31:45.647660+0000","last_became_active":"2024-04-25T08:31:35.316484+0000","last_became_peered":"2024-04-25T08:31:35.316484+0000","last_unstale":"2024-04-25T08:31:45.647660+0000","last_undegraded":"2024-04-25T08:31:45.647660+0000","last_fullsized":"2024-04-25T08:31:45.647660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366316+0000","last_change":"2024-04-25T08:31:40.355742+0000","last_active":"2024-04-25T08:31:42.366316+0000","last_peered":"2024-04-25T08:31:42.366316+0000","last_clean":"2024-04-25T08:31:42.366316+0000","last_became_active":"2024-04-25T08:31:40.355357+0000","last_became_peered":"2024-04-25T08:31:40.355357+0000","last_unstale":"2024-04-25T08:31:42.366316+0000","last_undegraded":"2024-04-25T08:31:42.366316+0000","last_fullsized":"2024-04-25T08:31:42.366316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382374+0000","last_change":"2024-04-25T08:31:36.313482+0000","last_active":"2024-04-25T08:31:41.382374+0000","last_peered":"2024-04-25T08:31:41.382374+0000","last_clean":"2024-04-25T08:31:41.382374+0000","last_became_active":"2024-04-25T08:31:36.312650+0000","last_became_peered":"2024-04-25T08:31:36.312650+0000","last_unstale":"2024-04-25T08:31:41.382374+0000","last_undegraded":"2024-04-25T08:31:41.382374+0000","last_fullsized":"2024-04-25T08:31:41.382374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:10.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367199+0000","last_change":"2024-04-25T08:31:35.320355+0000","last_active":"2024-04-25T08:31:42.367199+0000","last_peered":"2024-04-25T08:31:42.367199+0000","last_clean":"2024-04-25T08:31:42.367199+0000","last_became_active":"2024-04-25T08:31:35.320046+0000","last_became_peered":"2024-04-25T08:31:35.320046+0000","last_unstale":"2024-04-25T08:31:42.367199+0000","last_undegraded":"2024-04-25T08:31:42.367199+0000","last_fullsized":"2024-04-25T08:31:42.367199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:46.434047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382283+0000","last_change":"2024-04-25T08:31:40.356433+0000","last_active":"2024-04-25T08:31:41.382283+0000","last_peered":"2024-04-25T08:31:41.382283+0000","last_clean":"2024-04-25T08:31:41.382283+0000","last_became_active":"2024-04-25T08:31:40.355755+0000","last_became_peered":"2024-04-25T08:31:40.355755+0000","last_unstale":"2024-04-25T08:31:41.382283+0000","last_undegraded":"2024-04-25T08:31:41.382283+0000","last_fullsized":"2024-04-25T08:31:41.382283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:51.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366137+0000","last_change":"2024-04-25T08:31:36.323385+0000","last_active":"2024-04-25T08:31:42.366137+0000","last_peered":"2024-04-25T08:31:42.366137+0000","last_clean":"2024-04-25T08:31:42.366137+0000","last_became_active":"2024-04-25T08:31:36.322674+0000","last_became_peered":"2024-04-25T08:31:36.322674+0000","last_unstale":"2024-04-25T08:31:42.366137+0000","last_undegraded":"2024-04-25T08:31:42.366137+0000","last_fullsized":"2024-04-25T08:31:42.366137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119486+0000","last_change":"2024-04-25T08:31:35.317212+0000","last_active":"2024-04-25T08:31:42.119486+0000","last_peered":"2024-04-25T08:31:42.119486+0000","last_clean":"2024-04-25T08:31:42.119486+0000","last_became_active":"2024-04-25T08:31:35.316922+0000","last_became_peered":"2024-04-25T08:31:35.316922+0000","last_unstale":"2024-04-25T08:31:42.119486+0000","last_undegraded":"2024-04-25T08:31:42.119486+0000","last_fullsized":"2024-04-25T08:31:42.119486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.726901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997655+0000","last_change":"2024-04-25T08:31:40.353781+0000","last_active":"2024-04-25T08:31:41.997655+0000","last_peered":"2024-04-25T08:31:41.997655+0000","last_clean":"2024-04-25T08:31:41.997655+0000","last_became_active":"2024-04-25T08:31:40.352269+0000","last_became_peered":"2024-04-25T08:31:40.352269+0000","last_unstale":"2024-04-25T08:31:41.997655+0000","last_undegraded":"2024-04-25T08:31:41.997655+0000","last_fullsized":"2024-04-25T08:31:41.997655+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:23.063716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119364+0000","last_change":"2024-04-25T08:31:36.324056+0000","last_active":"2024-04-25T08:31:42.119364+0000","last_peered":"2024-04-25T08:31:42.119364+0000","last_clean":"2024-04-25T08:31:42.119364+0000","last_became_active":"2024-04-25T08:31:36.323790+0000","last_became_peered":"2024-04-25T08:31:36.323790+0000","last_unstale":"2024-04-25T08:31:42.119364+0000","last_undegraded":"2024-04-25T08:31:42.119364+0000","last_fullsized":"2024-04-25T08:31:42.119364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:19.120016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382208+0000","last_change":"2024-04-25T08:31:35.318093+0000","last_active":"2024-04-25T08:31:41.382208+0000","last_peered":"2024-04-25T08:31:41.382208+0000","last_clean":"2024-04-25T08:31:41.382208+0000","last_became_active":"2024-04-25T08:31:35.317828+0000","last_became_peered":"2024-04-25T08:31:35.317828+0000","last_unstale":"2024-04-25T08:31:41.382208+0000","last_undegraded":"2024-04-25T08:31:41.382208+0000","last_fullsized":"2024-04-25T08:31:41.382208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:54.173888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997667+0000","last_change":"2024-04-25T08:31:40.599421+0000","last_active":"2024-04-25T08:31:41.997667+0000","last_peered":"2024-04-25T08:31:41.997667+0000","last_clean":"2024-04-25T08:31:41.997667+0000","last_became_active":"2024-04-25T08:31:40.599105+0000","last_became_peered":"2024-04-25T08:31:40.599105+0000","last_unstale":"2024-04-25T08:31:41.997667+0000","last_undegraded":"2024-04-25T08:31:41.997667+0000","last_fullsized":"2024-04-25T08:31:41.997667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:50.738838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366324+0000","last_change":"2024-04-25T08:31:40.597001+0000","last_active":"2024-04-25T08:31:42.366324+0000","last_peered":"2024-04-25T08:31:42.366324+0000","last_clean":"2024-04-25T08:31:42.366324+0000","last_became_active":"2024-04-25T08:31:40.596569+0000","last_became_peered":"2024-04-25T08:31:40.596569+0000","last_unstale":"2024-04-25T08:31:42.366324+0000","last_undegraded":"2024-04-25T08:31:42.366324+0000","last_fullsized":"2024-04-25T08:31:42.366324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367007+0000","last_change":"2024-04-25T08:31:36.332669+0000","last_active":"2024-04-25T08:31:42.367007+0000","last_peered":"2024-04-25T08:31:42.367007+0000","last_clean":"2024-04-25T08:31:42.367007+0000","last_became_active":"2024-04-25T08:31:36.332305+0000","last_became_peered":"2024-04-25T08:31:36.332305+0000","last_unstale":"2024-04-25T08:31:42.367007+0000","last_undegraded":"2024-04-25T08:31:42.367007+0000","last_fullsized":"2024-04-25T08:31:42.367007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:15.226896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118969+0000","last_change":"2024-04-25T08:31:35.317959+0000","last_active":"2024-04-25T08:31:42.118969+0000","last_peered":"2024-04-25T08:31:42.118969+0000","last_clean":"2024-04-25T08:31:42.118969+0000","last_became_active":"2024-04-25T08:31:35.317680+0000","last_became_peered":"2024-04-25T08:31:35.317680+0000","last_unstale":"2024-04-25T08:31:42.118969+0000","last_undegraded":"2024-04-25T08:31:42.118969+0000","last_fullsized":"2024-04-25T08:31:42.118969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:54.173888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382101+0000","last_change":"2024-04-25T08:31:40.352855+0000","last_active":"2024-04-25T08:31:41.382101+0000","last_peered":"2024-04-25T08:31:41.382101+0000","last_clean":"2024-04-25T08:31:41.382101+0000","last_became_active":"2024-04-25T08:31:40.352373+0000","last_became_peered":"2024-04-25T08:31:40.352373+0000","last_unstale":"2024-04-25T08:31:41.382101+0000","last_undegraded":"2024-04-25T08:31:41.382101+0000","last_fullsized":"2024-04-25T08:31:41.382101+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:56.013298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997618+0000","last_change":"2024-04-25T08:31:36.327572+0000","last_active":"2024-04-25T08:31:41.997618+0000","last_peered":"2024-04-25T08:31:41.997618+0000","last_clean":"2024-04-25T08:31:41.997618+0000","last_became_active":"2024-04-25T08:31:36.320635+0000","last_became_peered":"2024-04-25T08:31:36.320635+0000","last_unstale":"2024-04-25T08:31:41.997618+0000","last_undegraded":"2024-04-25T08:31:41.997618+0000","last_fullsized":"2024-04-25T08:31:41.997618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997788+0000","last_change":"2024-04-25T08:31:35.317367+0000","last_active":"2024-04-25T08:31:41.997788+0000","last_peered":"2024-04-25T08:31:41.997788+0000","last_clean":"2024-04-25T08:31:41.997788+0000","last_became_active":"2024-04-25T08:31:35.317013+0000","last_became_peered":"2024-04-25T08:31:35.317013+0000","last_unstale":"2024-04-25T08:31:41.997788+0000","last_undegraded":"2024-04-25T08:31:41.997788+0000","last_fullsized":"2024-04-25T08:31:41.997788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:34.641593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381422+0000","last_change":"2024-04-25T08:31:35.305651+0000","last_active":"2024-04-25T08:31:41.381422+0000","last_peered":"2024-04-25T08:31:41.381422+0000","last_clean":"2024-04-25T08:31:41.381422+0000","last_became_active":"2024-04-25T08:31:35.305381+0000","last_became_peered":"2024-04-25T08:31:35.305381+0000","last_unstale":"2024-04-25T08:31:41.381422+0000","last_undegraded":"2024-04-25T08:31:41.381422+0000","last_fullsized":"2024-04-25T08:31:41.381422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366718+0000","last_change":"2024-04-25T08:31:36.323372+0000","last_active":"2024-04-25T08:31:42.366718+0000","last_peered":"2024-04-25T08:31:42.366718+0000","last_clean":"2024-04-25T08:31:42.366718+0000","last_became_active":"2024-04-25T08:31:36.322950+0000","last_became_peered":"2024-04-25T08:31:36.322950+0000","last_unstale":"2024-04-25T08:31:42.366718+0000","last_undegraded":"2024-04-25T08:31:42.366718+0000","last_fullsized":"2024-04-25T08:31:42.366718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384753+0000","last_change":"2024-04-25T08:31:40.353992+0000","last_active":"2024-04-25T08:31:41.384753+0000","last_peered":"2024-04-25T08:31:41.384753+0000","last_clean":"2024-04-25T08:31:41.384753+0000","last_became_active":"2024-04-25T08:31:40.353601+0000","last_became_peered":"2024-04-25T08:31:40.353601+0000","last_unstale":"2024-04-25T08:31:41.384753+0000","last_undegraded":"2024-04-25T08:31:41.384753+0000","last_fullsized":"2024-04-25T08:31:41.384753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:02.808345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997972+0000","last_change":"2024-04-25T08:31:35.321307+0000","last_active":"2024-04-25T08:31:41.997972+0000","last_peered":"2024-04-25T08:31:41.997972+0000","last_clean":"2024-04-25T08:31:41.997972+0000","last_became_active":"2024-04-25T08:31:35.320720+0000","last_became_peered":"2024-04-25T08:31:35.320720+0000","last_unstale":"2024-04-25T08:31:41.997972+0000","last_undegraded":"2024-04-25T08:31:41.997972+0000","last_fullsized":"2024-04-25T08:31:41.997972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381993+0000","last_change":"2024-04-25T08:31:36.317597+0000","last_active":"2024-04-25T08:31:41.381993+0000","last_peered":"2024-04-25T08:31:41.381993+0000","last_clean":"2024-04-25T08:31:41.381993+0000","last_became_active":"2024-04-25T08:31:36.317333+0000","last_became_peered":"2024-04-25T08:31:36.317333+0000","last_unstale":"2024-04-25T08:31:41.381993+0000","last_undegraded":"2024-04-25T08:31:41.381993+0000","last_fullsized":"2024-04-25T08:31:41.381993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:55.191462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998205+0000","last_change":"2024-04-25T08:31:40.357847+0000","last_active":"2024-04-25T08:31:41.998205+0000","last_peered":"2024-04-25T08:31:41.998205+0000","last_clean":"2024-04-25T08:31:41.998205+0000","last_became_active":"2024-04-25T08:31:40.357445+0000","last_became_peered":"2024-04-25T08:31:40.357445+0000","last_unstale":"2024-04-25T08:31:41.998205+0000","last_undegraded":"2024-04-25T08:31:41.998205+0000","last_fullsized":"2024-04-25T08:31:41.998205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:45.166751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381524+0000","last_change":"2024-04-25T08:31:36.310801+0000","last_active":"2024-04-25T08:31:41.381524+0000","last_peered":"2024-04-25T08:31:41.381524+0000","last_clean":"2024-04-25T08:31:41.381524+0000","last_became_active":"2024-04-25T08:31:36.310509+0000","last_became_peered":"2024-04-25T08:31:36.310509+0000","last_unstale":"2024-04-25T08:31:41.381524+0000","last_undegraded":"2024-04-25T08:31:41.381524+0000","last_fullsized":"2024-04-25T08:31:41.381524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384592+0000","last_change":"2024-04-25T08:31:36.320666+0000","last_active":"2024-04-25T08:31:41.384592+0000","last_peered":"2024-04-25T08:31:41.384592+0000","last_clean":"2024-04-25T08:31:41.384592+0000","last_became_active":"2024-04-25T08:31:36.320402+0000","last_became_peered":"2024-04-25T08:31:36.320402+0000","last_unstale":"2024-04-25T08:31:41.384592+0000","last_undegraded":"2024-04-25T08:31:41.384592+0000","last_fullsized":"2024-04-25T08:31:41.384592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:47.451621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119194+0000","last_change":"2024-04-25T08:31:36.323523+0000","last_active":"2024-04-25T08:31:42.119194+0000","last_peered":"2024-04-25T08:31:42.119194+0000","last_clean":"2024-04-25T08:31:42.119194+0000","last_became_active":"2024-04-25T08:31:36.323233+0000","last_became_peered":"2024-04-25T08:31:36.323233+0000","last_unstale":"2024-04-25T08:31:42.119194+0000","last_undegraded":"2024-04-25T08:31:42.119194+0000","last_fullsized":"2024-04-25T08:31:42.119194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:46.700115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366631+0000","last_change":"2024-04-25T08:31:36.332567+0000","last_active":"2024-04-25T08:31:42.366631+0000","last_peered":"2024-04-25T08:31:42.366631+0000","last_clean":"2024-04-25T08:31:42.366631+0000","last_became_active":"2024-04-25T08:31:36.332300+0000","last_became_peered":"2024-04-25T08:31:36.332300+0000","last_unstale":"2024-04-25T08:31:42.366631+0000","last_undegraded":"2024-04-25T08:31:42.366631+0000","last_fullsized":"2024-04-25T08:31:42.366631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:57.933443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998117+0000","last_change":"2024-04-25T08:31:36.326949+0000","last_active":"2024-04-25T08:31:41.998117+0000","last_peered":"2024-04-25T08:31:41.998117+0000","last_clean":"2024-04-25T08:31:41.998117+0000","last_became_active":"2024-04-25T08:31:36.322467+0000","last_became_peered":"2024-04-25T08:31:36.322467+0000","last_unstale":"2024-04-25T08:31:41.998117+0000","last_undegraded":"2024-04-25T08:31:41.998117+0000","last_fullsized":"2024-04-25T08:31:41.998117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:46.700115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367093+0000","last_change":"2024-04-25T08:31:36.331852+0000","last_active":"2024-04-25T08:31:42.367093+0000","last_peered":"2024-04-25T08:31:42.367093+0000","last_clean":"2024-04-25T08:31:42.367093+0000","last_became_active":"2024-04-25T08:31:36.331588+0000","last_became_peered":"2024-04-25T08:31:36.331588+0000","last_unstale":"2024-04-25T08:31:42.367093+0000","last_undegraded":"2024-04-25T08:31:42.367093+0000","last_fullsized":"2024-04-25T08:31:42.367093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381821+0000","last_change":"2024-04-25T08:31:36.309132+0000","last_active":"2024-04-25T08:31:41.381821+0000","last_peered":"2024-04-25T08:31:41.381821+0000","last_clean":"2024-04-25T08:31:41.381821+0000","last_became_active":"2024-04-25T08:31:36.308790+0000","last_became_peered":"2024-04-25T08:31:36.308790+0000","last_unstale":"2024-04-25T08:31:41.381821+0000","last_undegraded":"2024-04-25T08:31:41.381821+0000","last_fullsized":"2024-04-25T08:31:41.381821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:14.177622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998237+0000","last_change":"2024-04-25T08:31:36.320300+0000","last_active":"2024-04-25T08:31:41.998237+0000","last_peered":"2024-04-25T08:31:41.998237+0000","last_clean":"2024-04-25T08:31:41.998237+0000","last_became_active":"2024-04-25T08:31:36.319984+0000","last_became_peered":"2024-04-25T08:31:36.319984+0000","last_unstale":"2024-04-25T08:31:41.998237+0000","last_undegraded":"2024-04-25T08:31:41.998237+0000","last_fullsized":"2024-04-25T08:31:41.998237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118927+0000","last_change":"2024-04-25T08:31:36.329425+0000","last_active":"2024-04-25T08:31:42.118927+0000","last_peered":"2024-04-25T08:31:42.118927+0000","last_clean":"2024-04-25T08:31:42.118927+0000","last_became_active":"2024-04-25T08:31:36.328879+0000","last_became_peered":"2024-04-25T08:31:36.328879+0000","last_unstale":"2024-04-25T08:31:42.118927+0000","last_undegraded":"2024-04-25T08:31:42.118927+0000","last_fullsized":"2024-04-25T08:31:42.118927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997886+0000","last_change":"2024-04-25T08:31:36.322034+0000","last_active":"2024-04-25T08:31:41.997886+0000","last_peered":"2024-04-25T08:31:41.997886+0000","last_clean":"2024-04-25T08:31:41.997886+0000","last_became_active":"2024-04-25T08:31:36.321723+0000","last_became_peered":"2024-04-25T08:31:36.321723+0000","last_unstale":"2024-04-25T08:31:41.997886+0000","last_undegraded":"2024-04-25T08:31:41.997886+0000","last_fullsized":"2024-04-25T08:31:41.997886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119094+0000","last_change":"2024-04-25T08:31:36.325939+0000","last_active":"2024-04-25T08:31:42.119094+0000","last_peered":"2024-04-25T08:31:42.119094+0000","last_clean":"2024-04-25T08:31:42.119094+0000","last_became_active":"2024-04-25T08:31:36.325469+0000","last_became_peered":"2024-04-25T08:31:36.325469+0000","last_unstale":"2024-04-25T08:31:42.119094+0000","last_undegraded":"2024-04-25T08:31:42.119094+0000","last_fullsized":"2024-04-25T08:31:42.119094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:14.177622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366378+0000","last_change":"2024-04-25T08:31:36.328064+0000","last_active":"2024-04-25T08:31:42.366378+0000","last_peered":"2024-04-25T08:31:42.366378+0000","last_clean":"2024-04-25T08:31:42.366378+0000","last_became_active":"2024-04-25T08:31:36.327769+0000","last_became_peered":"2024-04-25T08:31:36.327769+0000","last_unstale":"2024-04-25T08:31:42.366378+0000","last_undegraded":"2024-04-25T08:31:42.366378+0000","last_fullsized":"2024-04-25T08:31:42.366378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:58.779587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381873+0000","last_change":"2024-04-25T08:31:36.318263+0000","last_active":"2024-04-25T08:31:41.381873+0000","last_peered":"2024-04-25T08:31:41.381873+0000","last_clean":"2024-04-25T08:31:41.381873+0000","last_became_active":"2024-04-25T08:31:36.317987+0000","last_became_peered":"2024-04-25T08:31:36.317987+0000","last_unstale":"2024-04-25T08:31:41.381873+0000","last_undegraded":"2024-04-25T08:31:41.381873+0000","last_fullsized":"2024-04-25T08:31:41.381873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422245+0000","last_change":"2024-04-25T08:31:36.318589+0000","last_active":"2024-04-25T08:31:41.422245+0000","last_peered":"2024-04-25T08:31:41.422245+0000","last_clean":"2024-04-25T08:31:41.422245+0000","last_became_active":"2024-04-25T08:31:36.318315+0000","last_became_peered":"2024-04-25T08:31:36.318315+0000","last_unstale":"2024-04-25T08:31:41.422245+0000","last_undegraded":"2024-04-25T08:31:41.422245+0000","last_fullsized":"2024-04-25T08:31:41.422245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:05.920070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384795+0000","last_change":"2024-04-25T08:31:36.320980+0000","last_active":"2024-04-25T08:31:41.384795+0000","last_peered":"2024-04-25T08:31:41.384795+0000","last_clean":"2024-04-25T08:31:41.384795+0000","last_became_active":"2024-04-25T08:31:36.320663+0000","last_became_peered":"2024-04-25T08:31:36.320663+0000","last_unstale":"2024-04-25T08:31:41.384795+0000","last_undegraded":"2024-04-25T08:31:41.384795+0000","last_fullsized":"2024-04-25T08:31:41.384795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:10.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119636+0000","last_change":"2024-04-25T08:31:36.328219+0000","last_active":"2024-04-25T08:31:42.119636+0000","last_peered":"2024-04-25T08:31:42.119636+0000","last_clean":"2024-04-25T08:31:42.119636+0000","last_became_active":"2024-04-25T08:31:36.327914+0000","last_became_peered":"2024-04-25T08:31:36.327914+0000","last_unstale":"2024-04-25T08:31:42.119636+0000","last_undegraded":"2024-04-25T08:31:42.119636+0000","last_fullsized":"2024-04-25T08:31:42.119636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:47.399399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118812+0000","last_change":"2024-04-25T08:31:36.322834+0000","last_active":"2024-04-25T08:31:42.118812+0000","last_peered":"2024-04-25T08:31:42.118812+0000","last_clean":"2024-04-25T08:31:42.118812+0000","last_became_active":"2024-04-25T08:31:36.322555+0000","last_became_peered":"2024-04-25T08:31:36.322555+0000","last_unstale":"2024-04-25T08:31:42.118812+0000","last_undegraded":"2024-04-25T08:31:42.118812+0000","last_fullsized":"2024-04-25T08:31:42.118812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367013+0000","last_change":"2024-04-25T08:31:36.323473+0000","last_active":"2024-04-25T08:31:42.367013+0000","last_peered":"2024-04-25T08:31:42.367013+0000","last_clean":"2024-04-25T08:31:42.367013+0000","last_became_active":"2024-04-25T08:31:36.323180+0000","last_became_peered":"2024-04-25T08:31:36.323180+0000","last_unstale":"2024-04-25T08:31:42.367013+0000","last_undegraded":"2024-04-25T08:31:42.367013+0000","last_fullsized":"2024-04-25T08:31:42.367013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366156+0000","last_change":"2024-04-25T08:31:36.314459+0000","last_active":"2024-04-25T08:31:42.366156+0000","last_peered":"2024-04-25T08:31:42.366156+0000","last_clean":"2024-04-25T08:31:42.366156+0000","last_became_active":"2024-04-25T08:31:36.313669+0000","last_became_peered":"2024-04-25T08:31:36.313669+0000","last_unstale":"2024-04-25T08:31:42.366156+0000","last_undegraded":"2024-04-25T08:31:42.366156+0000","last_fullsized":"2024-04-25T08:31:42.366156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997797+0000","last_change":"2024-04-25T08:31:36.329217+0000","last_active":"2024-04-25T08:31:41.997797+0000","last_peered":"2024-04-25T08:31:41.997797+0000","last_clean":"2024-04-25T08:31:41.997797+0000","last_became_active":"2024-04-25T08:31:36.328793+0000","last_became_peered":"2024-04-25T08:31:36.328793+0000","last_unstale":"2024-04-25T08:31:41.997797+0000","last_undegraded":"2024-04-25T08:31:41.997797+0000","last_fullsized":"2024-04-25T08:31:41.997797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:26.185783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382287+0000","last_change":"2024-04-25T08:31:36.321711+0000","last_active":"2024-04-25T08:31:41.382287+0000","last_peered":"2024-04-25T08:31:41.382287+0000","last_clean":"2024-04-25T08:31:41.382287+0000","last_became_active":"2024-04-25T08:31:36.321446+0000","last_became_peered":"2024-04-25T08:31:36.321446+0000","last_unstale":"2024-04-25T08:31:41.382287+0000","last_undegraded":"2024-04-25T08:31:41.382287+0000","last_fullsized":"2024-04-25T08:31:41.382287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:47.451621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381253+0000","last_change":"2024-04-25T08:31:36.319922+0000","last_active":"2024-04-25T08:31:41.381253+0000","last_peered":"2024-04-25T08:31:41.381253+0000","last_clean":"2024-04-25T08:31:41.381253+0000","last_became_active":"2024-04-25T08:31:36.319650+0000","last_became_peered":"2024-04-25T08:31:36.319650+0000","last_unstale":"2024-04-25T08:31:41.381253+0000","last_undegraded":"2024-04-25T08:31:41.381253+0000","last_fullsized":"2024-04-25T08:31:41.381253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381734+0000","last_change":"2024-04-25T08:31:36.319962+0000","last_active":"2024-04-25T08:31:41.381734+0000","last_peered":"2024-04-25T08:31:41.381734+0000","last_clean":"2024-04-25T08:31:41.381734+0000","last_became_active":"2024-04-25T08:31:36.319634+0000","last_became_peered":"2024-04-25T08:31:36.319634+0000","last_unstale":"2024-04-25T08:31:41.381734+0000","last_undegraded":"2024-04-25T08:31:41.381734+0000","last_fullsized":"2024-04-25T08:31:41.381734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:26.185783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422427+0000","last_change":"2024-04-25T08:31:36.320766+0000","last_active":"2024-04-25T08:31:41.422427+0000","last_peered":"2024-04-25T08:31:41.422427+0000","last_clean":"2024-04-25T08:31:41.422427+0000","last_became_active":"2024-04-25T08:31:36.320483+0000","last_became_peered":"2024-04-25T08:31:36.320483+0000","last_unstale":"2024-04-25T08:31:41.422427+0000","last_undegraded":"2024-04-25T08:31:41.422427+0000","last_fullsized":"2024-04-25T08:31:41.422427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:45.805418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997517+0000","last_change":"2024-04-25T08:31:36.327853+0000","last_active":"2024-04-25T08:31:41.997517+0000","last_peered":"2024-04-25T08:31:41.997517+0000","last_clean":"2024-04-25T08:31:41.997517+0000","last_became_active":"2024-04-25T08:31:36.319850+0000","last_became_peered":"2024-04-25T08:31:36.319850+0000","last_unstale":"2024-04-25T08:31:41.997517+0000","last_undegraded":"2024-04-25T08:31:41.997517+0000","last_fullsized":"2024-04-25T08:31:41.997517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421704+0000","last_change":"2024-04-25T08:31:36.311917+0000","last_active":"2024-04-25T08:31:41.421704+0000","last_peered":"2024-04-25T08:31:41.421704+0000","last_clean":"2024-04-25T08:31:41.421704+0000","last_became_active":"2024-04-25T08:31:36.311657+0000","last_became_peered":"2024-04-25T08:31:36.311657+0000","last_unstale":"2024-04-25T08:31:41.421704+0000","last_undegraded":"2024-04-25T08:31:41.421704+0000","last_fullsized":"2024-04-25T08:31:41.421704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:12.212803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119023+0000","last_change":"2024-04-25T08:31:36.318421+0000","last_active":"2024-04-25T08:31:42.119023+0000","last_peered":"2024-04-25T08:31:42.119023+0000","last_clean":"2024-04-25T08:31:42.119023+0000","last_became_active":"2024-04-25T08:31:36.318157+0000","last_became_peered":"2024-04-25T08:31:36.318157+0000","last_unstale":"2024-04-25T08:31:42.119023+0000","last_undegraded":"2024-04-25T08:31:42.119023+0000","last_fullsized":"2024-04-25T08:31:42.119023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:26.185783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421798+0000","last_change":"2024-04-25T08:31:36.314252+0000","last_active":"2024-04-25T08:31:41.421798+0000","last_peered":"2024-04-25T08:31:41.421798+0000","last_clean":"2024-04-25T08:31:41.421798+0000","last_became_active":"2024-04-25T08:31:36.313732+0000","last_became_peered":"2024-04-25T08:31:36.313732+0000","last_unstale":"2024-04-25T08:31:41.421798+0000","last_undegraded":"2024-04-25T08:31:41.421798+0000","last_fullsized":"2024-04-25T08:31:41.421798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:43.612460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119289+0000","last_change":"2024-04-25T08:31:36.330053+0000","last_active":"2024-04-25T08:31:42.119289+0000","last_peered":"2024-04-25T08:31:42.119289+0000","last_clean":"2024-04-25T08:31:42.119289+0000","last_became_active":"2024-04-25T08:31:36.329775+0000","last_became_peered":"2024-04-25T08:31:36.329775+0000","last_unstale":"2024-04-25T08:31:42.119289+0000","last_undegraded":"2024-04-25T08:31:42.119289+0000","last_fullsized":"2024-04-25T08:31:42.119289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:12.212803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365851+0000","last_change":"2024-04-25T08:31:36.317826+0000","last_active":"2024-04-25T08:31:42.365851+0000","last_peered":"2024-04-25T08:31:42.365851+0000","last_clean":"2024-04-25T08:31:42.365851+0000","last_became_active":"2024-04-25T08:31:36.317514+0000","last_became_peered":"2024-04-25T08:31:36.317514+0000","last_unstale":"2024-04-25T08:31:42.365851+0000","last_undegraded":"2024-04-25T08:31:42.365851+0000","last_fullsized":"2024-04-25T08:31:42.365851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367390+0000","last_change":"2024-04-25T08:31:36.310887+0000","last_active":"2024-04-25T08:31:42.367390+0000","last_peered":"2024-04-25T08:31:42.367390+0000","last_clean":"2024-04-25T08:31:42.367390+0000","last_became_active":"2024-04-25T08:31:36.310581+0000","last_became_peered":"2024-04-25T08:31:36.310581+0000","last_unstale":"2024-04-25T08:31:42.367390+0000","last_undegraded":"2024-04-25T08:31:42.367390+0000","last_fullsized":"2024-04-25T08:31:42.367390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422313+0000","last_change":"2024-04-25T08:31:36.315442+0000","last_active":"2024-04-25T08:31:41.422313+0000","last_peered":"2024-04-25T08:31:41.422313+0000","last_clean":"2024-04-25T08:31:41.422313+0000","last_became_active":"2024-04-25T08:31:36.315178+0000","last_became_peered":"2024-04-25T08:31:36.315178+0000","last_unstale":"2024-04-25T08:31:41.422313+0000","last_undegraded":"2024-04-25T08:31:41.422313+0000","last_fullsized":"2024-04-25T08:31:41.422313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:52.353609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:31:53.441 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:31:53.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.579+0000 7fc81d370640 1 Processor -- start 2024-04-25T08:31:53.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.579+0000 7fc81d370640 1 -- start start 2024-04-25T08:31:53.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.580+0000 7fc81d370640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc8180cf8f0 0x7fc81806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.580+0000 7fc81d370640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc81806c300 0x7fc81815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.580+0000 7fc81d370640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81815ce50 con 0x7fc8180cf510 2024-04-25T08:31:53.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.580+0000 7fc81d370640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81806e170 con 0x7fc81806c300 2024-04-25T08:31:53.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.580+0000 7fc81d370640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81815d100 con 0x7fc8180cf8f0 2024-04-25T08:31:53.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.580+0000 7fc816ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc8180cf510 0x7fc81806bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49768/0 (socket says 172.21.15.38:49768) 2024-04-25T08:31:53.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.580+0000 7fc816ffd640 1 -- 172.21.15.38:0/256186979 learned_addr learned my addr 172.21.15.38:0/256186979 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:53.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.580+0000 7fc8177fe640 1 --2- 172.21.15.38:0/256186979 >> v2:172.21.15.38:3300/0 conn(0x7fc81806c300 0x7fc81815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.581+0000 7fc8167fc640 1 --2- 172.21.15.38:0/256186979 >> v2:172.21.15.107:3300/0 conn(0x7fc8180cf8f0 0x7fc81806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.581+0000 7fc815ffb640 1 -- 172.21.15.38:0/256186979 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911292294 0 0) 0x7fc81815ce50 con 0x7fc8180cf510 2024-04-25T08:31:53.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.581+0000 7fc815ffb640 1 -- 172.21.15.38:0/256186979 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc804003520 con 0x7fc8180cf510 2024-04-25T08:31:53.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.581+0000 7fc8177fe640 1 -- 172.21.15.38:0/256186979 >> v2:172.21.15.107:3300/0 conn(0x7fc8180cf8f0 msgr2=0x7fc81806fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:53.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.581+0000 7fc8177fe640 1 --2- 172.21.15.38:0/256186979 >> v2:172.21.15.107:3300/0 conn(0x7fc8180cf8f0 0x7fc81806fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.581+0000 7fc8177fe640 1 -- 172.21.15.38:0/256186979 >> v1:172.21.15.38:6789/0 conn(0x7fc8180cf510 legacy=0x7fc81806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.581+0000 7fc8177fe640 1 -- 172.21.15.38:0/256186979 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81806e410 con 0x7fc81806c300 2024-04-25T08:31:53.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.581+0000 7fc8167fc640 1 --2- 172.21.15.38:0/256186979 >> v2:172.21.15.107:3300/0 conn(0x7fc8180cf8f0 0x7fc81806fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:53.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.582+0000 7fc8177fe640 1 --2- 172.21.15.38:0/256186979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc81806c300 0x7fc81815d5e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc81806f800 tx=0x7fc80802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1ff7b7d3033e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:53.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.582+0000 7fc815ffb640 1 -- 172.21.15.38:0/256186979 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808007740 con 0x7fc81806c300 2024-04-25T08:31:53.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.582+0000 7fc815ffb640 1 -- 172.21.15.38:0/256186979 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8080078e0 con 0x7fc81806c300 2024-04-25T08:31:53.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.582+0000 7fc815ffb640 1 -- 172.21.15.38:0/256186979 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808007cb0 con 0x7fc81806c300 2024-04-25T08:31:53.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.582+0000 7fc81d370640 1 -- 172.21.15.38:0/256186979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc81806c300 msgr2=0x7fc81815d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.582+0000 7fc81d370640 1 --2- 172.21.15.38:0/256186979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc81806c300 0x7fc81815d5e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc81806f800 tx=0x7fc80802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.582+0000 7fc81d370640 1 -- 172.21.15.38:0/256186979 shutdown_connections 2024-04-25T08:31:53.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.582+0000 7fc81d370640 1 --2- 172.21.15.38:0/256186979 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc81806c300 0x7fc81815d5e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.582+0000 7fc81d370640 1 --2- 172.21.15.38:0/256186979 >> v2:172.21.15.107:3300/0 conn(0x7fc8180cf8f0 0x7fc81806fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.582+0000 7fc81d370640 1 -- 172.21.15.38:0/256186979 >> 172.21.15.38:0/256186979 conn(0x7fc818071920 msgr2=0x7fc818071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:53.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.582+0000 7fc81d370640 1 -- 172.21.15.38:0/256186979 shutdown_connections 2024-04-25T08:31:53.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.582+0000 7fc81d370640 1 -- 172.21.15.38:0/256186979 wait complete. 2024-04-25T08:31:53.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.584+0000 7fc81d370640 1 Processor -- start 2024-04-25T08:31:53.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.584+0000 7fc81d370640 1 -- start start 2024-04-25T08:31:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.584+0000 7fc81d370640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc81806c300 0x7fc8181590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.584+0000 7fc81d370640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8180cf510 0x7fc8181564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.584+0000 7fc81d370640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8181e5a20 0x7fc8181e3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.584+0000 7fc81d370640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81806d9e0 con 0x7fc8181e5a20 2024-04-25T08:31:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.584+0000 7fc81d370640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81806d860 con 0x7fc81806c300 2024-04-25T08:31:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.584+0000 7fc81d370640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81815c920 con 0x7fc8180cf510 2024-04-25T08:31:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.584+0000 7fc8177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8181e5a20 0x7fc8181e3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.584+0000 7fc8177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8181e5a20 0x7fc8181e3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33318/0 (socket says 172.21.15.38:33318) 2024-04-25T08:31:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.584+0000 7fc8177fe640 1 -- 172.21.15.38:0/19368838 learned_addr learned my addr 172.21.15.38:0/19368838 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc816ffd640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc81806c300 0x7fc8181590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc8167fc640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8180cf510 0x7fc8181564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc8177fe640 1 -- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8180cf510 msgr2=0x7fc8181564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc8177fe640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8180cf510 0x7fc8181564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc8177fe640 1 -- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc81806c300 msgr2=0x7fc8181590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc8177fe640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc81806c300 0x7fc8181590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc8177fe640 1 -- 172.21.15.38:0/19368838 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8181569e0 con 0x7fc8181e5a20 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc816ffd640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc81806c300 0x7fc8181590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc8167fc640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8180cf510 0x7fc8181564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc8177fe640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8181e5a20 0x7fc8181e3cf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc808009e50 tx=0x7fc808004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc80f7fe640 1 -- 172.21.15.38:0/19368838 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808040070 con 0x7fc8181e5a20 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc80f7fe640 1 -- 172.21.15.38:0/19368838 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808031bf0 con 0x7fc8181e5a20 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc81d370640 1 -- 172.21.15.38:0/19368838 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc818156ce0 con 0x7fc8181e5a20 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc80f7fe640 1 -- 172.21.15.38:0/19368838 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808009450 con 0x7fc8181e5a20 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.585+0000 7fc81d370640 1 -- 172.21.15.38:0/19368838 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc818157520 con 0x7fc8181e5a20 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.586+0000 7fc81d370640 1 -- 172.21.15.38:0/19368838 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8180af5f0 con 0x7fc8181e5a20 2024-04-25T08:31:53.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.587+0000 7fc80f7fe640 1 -- 172.21.15.38:0/19368838 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc808039040 con 0x7fc8181e5a20 2024-04-25T08:31:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.587+0000 7fc80f7fe640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc7f806b4d0 0x7fc7f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.587+0000 7fc80f7fe640 1 -- 172.21.15.38:0/19368838 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc8080abd80 con 0x7fc8181e5a20 2024-04-25T08:31:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.587+0000 7fc816ffd640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc7f806b4d0 0x7fc7f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.588+0000 7fc816ffd640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc7f806b4d0 0x7fc7f806d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc810007fd0 tx=0x7fc810009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:53.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.591+0000 7fc80f7fe640 1 -- 172.21.15.38:0/19368838 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc818156ce0 con 0x7fc8181e5a20 2024-04-25T08:31:53.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.768+0000 7fc81d370640 1 -- 172.21.15.38:0/19368838 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc818066bc0 con 0x7fc7f806b4d0 2024-04-25T08:31:53.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.776+0000 7fc80f7fe640 1 -- 172.21.15.38:0/19368838 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7fc818066bc0 con 0x7fc7f806b4d0 2024-04-25T08:31:53.776 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:53.778 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:31:53.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.779+0000 7fc81d370640 1 -- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc7f806b4d0 msgr2=0x7fc7f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.779+0000 7fc81d370640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc7f806b4d0 0x7fc7f806d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc810007fd0 tx=0x7fc810009040 comp rx=0 tx=0).stop 2024-04-25T08:31:53.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.779+0000 7fc81d370640 1 -- 172.21.15.38:0/19368838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8181e5a20 msgr2=0x7fc8181e3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.779+0000 7fc81d370640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8181e5a20 0x7fc8181e3cf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc808009e50 tx=0x7fc808004a40 comp rx=0 tx=0).stop 2024-04-25T08:31:53.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.779+0000 7fc81d370640 1 -- 172.21.15.38:0/19368838 shutdown_connections 2024-04-25T08:31:53.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.779+0000 7fc81d370640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8181e5a20 0x7fc8181e3cf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.779+0000 7fc81d370640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc7f806b4d0 0x7fc7f806d990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.779+0000 7fc81d370640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8180cf510 0x7fc8181564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.779+0000 7fc81d370640 1 --2- 172.21.15.38:0/19368838 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc81806c300 0x7fc8181590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.779+0000 7fc81d370640 1 -- 172.21.15.38:0/19368838 >> 172.21.15.38:0/19368838 conn(0x7fc818071920 msgr2=0x7fc81815aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:53.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.780+0000 7fc81d370640 1 -- 172.21.15.38:0/19368838 shutdown_connections 2024-04-25T08:31:53.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.780+0000 7fc81d370640 1 -- 172.21.15.38:0/19368838 wait complete. 2024-04-25T08:31:53.787 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-04-25T08:31:53.207485+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223544,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751176,"statfs":{"total":773094113280,"available":772865204224,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.828416"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366309+0000","last_change":"2024-04-25T08:31:35.317991+0000","last_active":"2024-04-25T08:31:42.366309+0000","last_peered":"2024-04-25T08:31:42.366309+0000","last_clean":"2024-04-25T08:31:42.366309+0000","last_became_active":"2024-04-25T08:31:35.317664+0000","last_became_peered":"2024-04-25T08:31:35.317664+0000","last_unstale":"2024-04-25T08:31:42.366309+0000","last_undegraded":"2024-04-25T08:31:42.366309+0000","last_fullsized":"2024-04-25T08:31:42.366309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:56.580134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998066+0000","last_change":"2024-04-25T08:31:36.331619+0000","last_active":"2024-04-25T08:31:41.998066+0000","last_peered":"2024-04-25T08:31:41.998066+0000","last_clean":"2024-04-25T08:31:41.998066+0000","last_became_active":"2024-04-25T08:31:36.331332+0000","last_became_peered":"2024-04-25T08:31:36.331332+0000","last_unstale":"2024-04-25T08:31:41.998066+0000","last_undegraded":"2024-04-25T08:31:41.998066+0000","last_fullsized":"2024-04-25T08:31:41.998066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:12.212803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997585+0000","last_change":"2024-04-25T08:31:40.599503+0000","last_active":"2024-04-25T08:31:41.997585+0000","last_peered":"2024-04-25T08:31:41.997585+0000","last_clean":"2024-04-25T08:31:41.997585+0000","last_became_active":"2024-04-25T08:31:40.599122+0000","last_became_peered":"2024-04-25T08:31:40.599122+0000","last_unstale":"2024-04-25T08:31:41.997585+0000","last_undegraded":"2024-04-25T08:31:41.997585+0000","last_fullsized":"2024-04-25T08:31:41.997585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.735540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998226+0000","last_change":"2024-04-25T08:31:35.318801+0000","last_active":"2024-04-25T08:31:41.998226+0000","last_peered":"2024-04-25T08:31:41.998226+0000","last_clean":"2024-04-25T08:31:41.998226+0000","last_became_active":"2024-04-25T08:31:35.318505+0000","last_became_peered":"2024-04-25T08:31:35.318505+0000","last_unstale":"2024-04-25T08:31:41.998226+0000","last_undegraded":"2024-04-25T08:31:41.998226+0000","last_fullsized":"2024-04-25T08:31:41.998226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365961+0000","last_change":"2024-04-25T08:31:36.323797+0000","last_active":"2024-04-25T08:31:42.365961+0000","last_peered":"2024-04-25T08:31:42.365961+0000","last_clean":"2024-04-25T08:31:42.365961+0000","last_became_active":"2024-04-25T08:31:36.323045+0000","last_became_peered":"2024-04-25T08:31:36.323045+0000","last_unstale":"2024-04-25T08:31:42.365961+0000","last_undegraded":"2024-04-25T08:31:42.365961+0000","last_fullsized":"2024-04-25T08:31:42.365961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:15.226896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381055+0000","last_change":"2024-04-25T08:31:40.356739+0000","last_active":"2024-04-25T08:31:41.381055+0000","last_peered":"2024-04-25T08:31:41.381055+0000","last_clean":"2024-04-25T08:31:41.381055+0000","last_became_active":"2024-04-25T08:31:40.355756+0000","last_became_peered":"2024-04-25T08:31:40.355756+0000","last_unstale":"2024-04-25T08:31:41.381055+0000","last_undegraded":"2024-04-25T08:31:41.381055+0000","last_fullsized":"2024-04-25T08:31:41.381055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:40.045196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422218+0000","last_change":"2024-04-25T08:31:35.316916+0000","last_active":"2024-04-25T08:31:41.422218+0000","last_peered":"2024-04-25T08:31:41.422218+0000","last_clean":"2024-04-25T08:31:41.422218+0000","last_became_active":"2024-04-25T08:31:35.316358+0000","last_became_peered":"2024-04-25T08:31:35.316358+0000","last_unstale":"2024-04-25T08:31:41.422218+0000","last_undegraded":"2024-04-25T08:31:41.422218+0000","last_fullsized":"2024-04-25T08:31:41.422218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:00.982337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119271+0000","last_change":"2024-04-25T08:31:36.323115+0000","last_active":"2024-04-25T08:31:42.119271+0000","last_peered":"2024-04-25T08:31:42.119271+0000","last_clean":"2024-04-25T08:31:42.119271+0000","last_became_active":"2024-04-25T08:31:36.322848+0000","last_became_peered":"2024-04-25T08:31:36.322848+0000","last_unstale":"2024-04-25T08:31:42.119271+0000","last_undegraded":"2024-04-25T08:31:42.119271+0000","last_fullsized":"2024-04-25T08:31:42.119271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382566+0000","last_change":"2024-04-25T08:31:40.357214+0000","last_active":"2024-04-25T08:31:41.382566+0000","last_peered":"2024-04-25T08:31:41.382566+0000","last_clean":"2024-04-25T08:31:41.382566+0000","last_became_active":"2024-04-25T08:31:40.355877+0000","last_became_peered":"2024-04-25T08:31:40.355877+0000","last_unstale":"2024-04-25T08:31:41.382566+0000","last_undegraded":"2024-04-25T08:31:41.382566+0000","last_fullsized":"2024-04-25T08:31:41.382566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422059+0000","last_change":"2024-04-25T08:31:35.319905+0000","last_active":"2024-04-25T08:31:41.422059+0000","last_peered":"2024-04-25T08:31:41.422059+0000","last_clean":"2024-04-25T08:31:41.422059+0000","last_became_active":"2024-04-25T08:31:35.319127+0000","last_became_peered":"2024-04-25T08:31:35.319127+0000","last_unstale":"2024-04-25T08:31:41.422059+0000","last_undegraded":"2024-04-25T08:31:41.422059+0000","last_fullsized":"2024-04-25T08:31:41.422059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:43.779456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119380+0000","last_change":"2024-04-25T08:31:36.322498+0000","last_active":"2024-04-25T08:31:42.119380+0000","last_peered":"2024-04-25T08:31:42.119380+0000","last_clean":"2024-04-25T08:31:42.119380+0000","last_became_active":"2024-04-25T08:31:36.322250+0000","last_became_peered":"2024-04-25T08:31:36.322250+0000","last_unstale":"2024-04-25T08:31:42.119380+0000","last_undegraded":"2024-04-25T08:31:42.119380+0000","last_fullsized":"2024-04-25T08:31:42.119380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.111565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381899+0000","last_change":"2024-04-25T08:31:40.352372+0000","last_active":"2024-04-25T08:31:41.381899+0000","last_peered":"2024-04-25T08:31:41.381899+0000","last_clean":"2024-04-25T08:31:41.381899+0000","last_became_active":"2024-04-25T08:31:40.351860+0000","last_became_peered":"2024-04-25T08:31:40.351860+0000","last_unstale":"2024-04-25T08:31:41.381899+0000","last_undegraded":"2024-04-25T08:31:41.381899+0000","last_fullsized":"2024-04-25T08:31:41.381899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:50.728874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381228+0000","last_change":"2024-04-25T08:31:36.321222+0000","last_active":"2024-04-25T08:31:41.381228+0000","last_peered":"2024-04-25T08:31:41.381228+0000","last_clean":"2024-04-25T08:31:41.381228+0000","last_became_active":"2024-04-25T08:31:36.320947+0000","last_became_peered":"2024-04-25T08:31:36.320947+0000","last_unstale":"2024-04-25T08:31:41.381228+0000","last_undegraded":"2024-04-25T08:31:41.381228+0000","last_fullsized":"2024-04-25T08:31:41.381228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421943+0000","last_change":"2024-04-25T08:31:35.316969+0000","last_active":"2024-04-25T08:31:41.421943+0000","last_peered":"2024-04-25T08:31:41.421943+0000","last_clean":"2024-04-25T08:31:41.421943+0000","last_became_active":"2024-04-25T08:31:35.316693+0000","last_became_peered":"2024-04-25T08:31:35.316693+0000","last_unstale":"2024-04-25T08:31:41.421943+0000","last_undegraded":"2024-04-25T08:31:41.421943+0000","last_fullsized":"2024-04-25T08:31:41.421943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:20.652651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382507+0000","last_change":"2024-04-25T08:31:40.351507+0000","last_active":"2024-04-25T08:31:41.382507+0000","last_peered":"2024-04-25T08:31:41.382507+0000","last_clean":"2024-04-25T08:31:41.382507+0000","last_became_active":"2024-04-25T08:31:40.351053+0000","last_became_peered":"2024-04-25T08:31:40.351053+0000","last_unstale":"2024-04-25T08:31:41.382507+0000","last_undegraded":"2024-04-25T08:31:41.382507+0000","last_fullsized":"2024-04-25T08:31:41.382507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997831+0000","last_change":"2024-04-25T08:31:36.321459+0000","last_active":"2024-04-25T08:31:41.997831+0000","last_peered":"2024-04-25T08:31:41.997831+0000","last_clean":"2024-04-25T08:31:41.997831+0000","last_became_active":"2024-04-25T08:31:36.321137+0000","last_became_peered":"2024-04-25T08:31:36.321137+0000","last_unstale":"2024-04-25T08:31:41.997831+0000","last_undegraded":"2024-04-25T08:31:41.997831+0000","last_fullsized":"2024-04-25T08:31:41.997831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:14.177622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366532+0000","last_change":"2024-04-25T08:31:35.317075+0000","last_active":"2024-04-25T08:31:42.366532+0000","last_peered":"2024-04-25T08:31:42.366532+0000","last_clean":"2024-04-25T08:31:42.366532+0000","last_became_active":"2024-04-25T08:31:35.316764+0000","last_became_peered":"2024-04-25T08:31:35.316764+0000","last_unstale":"2024-04-25T08:31:42.366532+0000","last_undegraded":"2024-04-25T08:31:42.366532+0000","last_fullsized":"2024-04-25T08:31:42.366532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382109+0000","last_change":"2024-04-25T08:31:40.352414+0000","last_active":"2024-04-25T08:31:41.382109+0000","last_peered":"2024-04-25T08:31:41.382109+0000","last_clean":"2024-04-25T08:31:41.382109+0000","last_became_active":"2024-04-25T08:31:40.351024+0000","last_became_peered":"2024-04-25T08:31:40.351024+0000","last_unstale":"2024-04-25T08:31:41.382109+0000","last_undegraded":"2024-04-25T08:31:41.382109+0000","last_fullsized":"2024-04-25T08:31:41.382109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:16.241561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997717+0000","last_change":"2024-04-25T08:31:36.320330+0000","last_active":"2024-04-25T08:31:41.997717+0000","last_peered":"2024-04-25T08:31:41.997717+0000","last_clean":"2024-04-25T08:31:41.997717+0000","last_became_active":"2024-04-25T08:31:36.313715+0000","last_became_peered":"2024-04-25T08:31:36.313715+0000","last_unstale":"2024-04-25T08:31:41.997717+0000","last_undegraded":"2024-04-25T08:31:41.997717+0000","last_fullsized":"2024-04-25T08:31:41.997717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367128+0000","last_change":"2024-04-25T08:31:35.314788+0000","last_active":"2024-04-25T08:31:42.367128+0000","last_peered":"2024-04-25T08:31:42.367128+0000","last_clean":"2024-04-25T08:31:42.367128+0000","last_became_active":"2024-04-25T08:31:35.314353+0000","last_became_peered":"2024-04-25T08:31:35.314353+0000","last_unstale":"2024-04-25T08:31:42.367128+0000","last_undegraded":"2024-04-25T08:31:42.367128+0000","last_fullsized":"2024-04-25T08:31:42.367128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.726901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367441+0000","last_change":"2024-04-25T08:31:40.594652+0000","last_active":"2024-04-25T08:31:42.367441+0000","last_peered":"2024-04-25T08:31:42.367441+0000","last_clean":"2024-04-25T08:31:42.367441+0000","last_became_active":"2024-04-25T08:31:40.594138+0000","last_became_peered":"2024-04-25T08:31:40.594138+0000","last_unstale":"2024-04-25T08:31:42.367441+0000","last_undegraded":"2024-04-25T08:31:42.367441+0000","last_fullsized":"2024-04-25T08:31:42.367441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:23.148774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381474+0000","last_change":"2024-04-25T08:31:36.324636+0000","last_active":"2024-04-25T08:31:41.381474+0000","last_peered":"2024-04-25T08:31:41.381474+0000","last_clean":"2024-04-25T08:31:41.381474+0000","last_became_active":"2024-04-25T08:31:36.322240+0000","last_became_peered":"2024-04-25T08:31:36.322240+0000","last_unstale":"2024-04-25T08:31:41.381474+0000","last_undegraded":"2024-04-25T08:31:41.381474+0000","last_fullsized":"2024-04-25T08:31:41.381474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:49.809945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421616+0000","last_change":"2024-04-25T08:31:35.318327+0000","last_active":"2024-04-25T08:31:41.421616+0000","last_peered":"2024-04-25T08:31:41.421616+0000","last_clean":"2024-04-25T08:31:41.421616+0000","last_became_active":"2024-04-25T08:31:35.317749+0000","last_became_peered":"2024-04-25T08:31:35.317749+0000","last_unstale":"2024-04-25T08:31:41.421616+0000","last_undegraded":"2024-04-25T08:31:41.421616+0000","last_fullsized":"2024-04-25T08:31:41.421616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:46.434047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382327+0000","last_change":"2024-04-25T08:31:40.357220+0000","last_active":"2024-04-25T08:31:41.382327+0000","last_peered":"2024-04-25T08:31:41.382327+0000","last_clean":"2024-04-25T08:31:41.382327+0000","last_became_active":"2024-04-25T08:31:40.355770+0000","last_became_peered":"2024-04-25T08:31:40.355770+0000","last_unstale":"2024-04-25T08:31:41.382327+0000","last_undegraded":"2024-04-25T08:31:41.382327+0000","last_fullsized":"2024-04-25T08:31:41.382327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:04.998392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366774+0000","last_change":"2024-04-25T08:31:36.312756+0000","last_active":"2024-04-25T08:31:42.366774+0000","last_peered":"2024-04-25T08:31:42.366774+0000","last_clean":"2024-04-25T08:31:42.366774+0000","last_became_active":"2024-04-25T08:31:36.312492+0000","last_became_peered":"2024-04-25T08:31:36.312492+0000","last_unstale":"2024-04-25T08:31:42.366774+0000","last_undegraded":"2024-04-25T08:31:42.366774+0000","last_fullsized":"2024-04-25T08:31:42.366774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382375+0000","last_change":"2024-04-25T08:31:35.314188+0000","last_active":"2024-04-25T08:31:41.382375+0000","last_peered":"2024-04-25T08:31:41.382375+0000","last_clean":"2024-04-25T08:31:41.382375+0000","last_became_active":"2024-04-25T08:31:35.313682+0000","last_became_peered":"2024-04-25T08:31:35.313682+0000","last_unstale":"2024-04-25T08:31:41.382375+0000","last_undegraded":"2024-04-25T08:31:41.382375+0000","last_fullsized":"2024-04-25T08:31:41.382375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.726901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381316+0000","last_change":"2024-04-25T08:31:40.596307+0000","last_active":"2024-04-25T08:31:41.381316+0000","last_peered":"2024-04-25T08:31:41.381316+0000","last_clean":"2024-04-25T08:31:41.381316+0000","last_became_active":"2024-04-25T08:31:40.595853+0000","last_became_peered":"2024-04-25T08:31:40.595853+0000","last_unstale":"2024-04-25T08:31:41.381316+0000","last_undegraded":"2024-04-25T08:31:41.381316+0000","last_fullsized":"2024-04-25T08:31:41.381316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:18.206380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998305+0000","last_change":"2024-04-25T08:31:36.327135+0000","last_active":"2024-04-25T08:31:41.998305+0000","last_peered":"2024-04-25T08:31:41.998305+0000","last_clean":"2024-04-25T08:31:41.998305+0000","last_became_active":"2024-04-25T08:31:36.326839+0000","last_became_peered":"2024-04-25T08:31:36.326839+0000","last_unstale":"2024-04-25T08:31:41.998305+0000","last_undegraded":"2024-04-25T08:31:41.998305+0000","last_fullsized":"2024-04-25T08:31:41.998305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:47.399399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367454+0000","last_change":"2024-04-25T08:31:35.309749+0000","last_active":"2024-04-25T08:31:42.367454+0000","last_peered":"2024-04-25T08:31:42.367454+0000","last_clean":"2024-04-25T08:31:42.367454+0000","last_became_active":"2024-04-25T08:31:35.309352+0000","last_became_peered":"2024-04-25T08:31:35.309352+0000","last_unstale":"2024-04-25T08:31:42.367454+0000","last_undegraded":"2024-04-25T08:31:42.367454+0000","last_fullsized":"2024-04-25T08:31:42.367454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:54.173888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997493+0000","last_change":"2024-04-25T08:31:40.598224+0000","last_active":"2024-04-25T08:31:41.997493+0000","last_peered":"2024-04-25T08:31:41.997493+0000","last_clean":"2024-04-25T08:31:41.997493+0000","last_became_active":"2024-04-25T08:31:40.597800+0000","last_became_peered":"2024-04-25T08:31:40.597800+0000","last_unstale":"2024-04-25T08:31:41.997493+0000","last_undegraded":"2024-04-25T08:31:41.997493+0000","last_fullsized":"2024-04-25T08:31:41.997493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:10.902214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367151+0000","last_change":"2024-04-25T08:31:36.325531+0000","last_active":"2024-04-25T08:31:42.367151+0000","last_peered":"2024-04-25T08:31:42.367151+0000","last_clean":"2024-04-25T08:31:42.367151+0000","last_became_active":"2024-04-25T08:31:36.325240+0000","last_became_peered":"2024-04-25T08:31:36.325240+0000","last_unstale":"2024-04-25T08:31:42.367151+0000","last_undegraded":"2024-04-25T08:31:42.367151+0000","last_fullsized":"2024-04-25T08:31:42.367151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422321+0000","last_change":"2024-04-25T08:31:35.317032+0000","last_active":"2024-04-25T08:31:41.422321+0000","last_peered":"2024-04-25T08:31:41.422321+0000","last_clean":"2024-04-25T08:31:41.422321+0000","last_became_active":"2024-04-25T08:31:35.316659+0000","last_became_peered":"2024-04-25T08:31:35.316659+0000","last_unstale":"2024-04-25T08:31:41.422321+0000","last_undegraded":"2024-04-25T08:31:41.422321+0000","last_fullsized":"2024-04-25T08:31:41.422321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:57.762013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366270+0000","last_change":"2024-04-25T08:31:40.595457+0000","last_active":"2024-04-25T08:31:42.366270+0000","last_peered":"2024-04-25T08:31:42.366270+0000","last_clean":"2024-04-25T08:31:42.366270+0000","last_became_active":"2024-04-25T08:31:40.595077+0000","last_became_peered":"2024-04-25T08:31:40.595077+0000","last_unstale":"2024-04-25T08:31:42.366270+0000","last_undegraded":"2024-04-25T08:31:42.366270+0000","last_fullsized":"2024-04-25T08:31:42.366270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119506+0000","last_change":"2024-04-25T08:31:36.322361+0000","last_active":"2024-04-25T08:31:42.119506+0000","last_peered":"2024-04-25T08:31:42.119506+0000","last_clean":"2024-04-25T08:31:42.119506+0000","last_became_active":"2024-04-25T08:31:36.322107+0000","last_became_peered":"2024-04-25T08:31:36.322107+0000","last_unstale":"2024-04-25T08:31:42.119506+0000","last_undegraded":"2024-04-25T08:31:42.119506+0000","last_fullsized":"2024-04-25T08:31:42.119506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:51.984540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367254+0000","last_change":"2024-04-25T08:31:35.315993+0000","last_active":"2024-04-25T08:31:42.367254+0000","last_peered":"2024-04-25T08:31:42.367254+0000","last_clean":"2024-04-25T08:31:42.367254+0000","last_became_active":"2024-04-25T08:31:35.315664+0000","last_became_peered":"2024-04-25T08:31:35.315664+0000","last_unstale":"2024-04-25T08:31:42.367254+0000","last_undegraded":"2024-04-25T08:31:42.367254+0000","last_fullsized":"2024-04-25T08:31:42.367254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365979+0000","last_change":"2024-04-25T08:31:40.355607+0000","last_active":"2024-04-25T08:31:42.365979+0000","last_peered":"2024-04-25T08:31:42.365979+0000","last_clean":"2024-04-25T08:31:42.365979+0000","last_became_active":"2024-04-25T08:31:40.355024+0000","last_became_peered":"2024-04-25T08:31:40.355024+0000","last_unstale":"2024-04-25T08:31:42.365979+0000","last_undegraded":"2024-04-25T08:31:42.365979+0000","last_fullsized":"2024-04-25T08:31:42.365979+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382608+0000","last_change":"2024-04-25T08:31:36.323675+0000","last_active":"2024-04-25T08:31:41.382608+0000","last_peered":"2024-04-25T08:31:41.382608+0000","last_clean":"2024-04-25T08:31:41.382608+0000","last_became_active":"2024-04-25T08:31:36.323102+0000","last_became_peered":"2024-04-25T08:31:36.323102+0000","last_unstale":"2024-04-25T08:31:41.382608+0000","last_undegraded":"2024-04-25T08:31:41.382608+0000","last_fullsized":"2024-04-25T08:31:41.382608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:49.744475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119335+0000","last_change":"2024-04-25T08:31:35.317234+0000","last_active":"2024-04-25T08:31:42.119335+0000","last_peered":"2024-04-25T08:31:42.119335+0000","last_clean":"2024-04-25T08:31:42.119335+0000","last_became_active":"2024-04-25T08:31:35.316747+0000","last_became_peered":"2024-04-25T08:31:35.316747+0000","last_unstale":"2024-04-25T08:31:42.119335+0000","last_undegraded":"2024-04-25T08:31:42.119335+0000","last_fullsized":"2024-04-25T08:31:42.119335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367030+0000","last_change":"2024-04-25T08:31:40.352712+0000","last_active":"2024-04-25T08:31:42.367030+0000","last_peered":"2024-04-25T08:31:42.367030+0000","last_clean":"2024-04-25T08:31:42.367030+0000","last_became_active":"2024-04-25T08:31:40.352328+0000","last_became_peered":"2024-04-25T08:31:40.352328+0000","last_unstale":"2024-04-25T08:31:42.367030+0000","last_undegraded":"2024-04-25T08:31:42.367030+0000","last_fullsized":"2024-04-25T08:31:42.367030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:56.013298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366618+0000","last_change":"2024-04-25T08:31:36.326854+0000","last_active":"2024-04-25T08:31:42.366618+0000","last_peered":"2024-04-25T08:31:42.366618+0000","last_clean":"2024-04-25T08:31:42.366618+0000","last_became_active":"2024-04-25T08:31:36.326564+0000","last_became_peered":"2024-04-25T08:31:36.326564+0000","last_unstale":"2024-04-25T08:31:42.366618+0000","last_undegraded":"2024-04-25T08:31:42.366618+0000","last_fullsized":"2024-04-25T08:31:42.366618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384554+0000","last_change":"2024-04-25T08:31:35.317497+0000","last_active":"2024-04-25T08:31:41.384554+0000","last_peered":"2024-04-25T08:31:41.384554+0000","last_clean":"2024-04-25T08:31:41.384554+0000","last_became_active":"2024-04-25T08:31:35.317202+0000","last_became_peered":"2024-04-25T08:31:35.317202+0000","last_unstale":"2024-04-25T08:31:41.384554+0000","last_undegraded":"2024-04-25T08:31:41.384554+0000","last_fullsized":"2024-04-25T08:31:41.384554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367456+0000","last_change":"2024-04-25T08:31:40.352169+0000","last_active":"2024-04-25T08:31:42.367456+0000","last_peered":"2024-04-25T08:31:42.367456+0000","last_clean":"2024-04-25T08:31:42.367456+0000","last_became_active":"2024-04-25T08:31:40.351743+0000","last_became_peered":"2024-04-25T08:31:40.351743+0000","last_unstale":"2024-04-25T08:31:42.367456+0000","last_undegraded":"2024-04-25T08:31:42.367456+0000","last_fullsized":"2024-04-25T08:31:42.367456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:09.948828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365773+0000","last_change":"2024-04-25T08:31:36.324307+0000","last_active":"2024-04-25T08:31:42.365773+0000","last_peered":"2024-04-25T08:31:42.365773+0000","last_clean":"2024-04-25T08:31:42.365773+0000","last_became_active":"2024-04-25T08:31:36.324011+0000","last_became_peered":"2024-04-25T08:31:36.324011+0000","last_unstale":"2024-04-25T08:31:42.365773+0000","last_undegraded":"2024-04-25T08:31:42.365773+0000","last_fullsized":"2024-04-25T08:31:42.365773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:01.999912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421648+0000","last_change":"2024-04-25T08:31:35.317490+0000","last_active":"2024-04-25T08:31:41.421648+0000","last_peered":"2024-04-25T08:31:41.421648+0000","last_clean":"2024-04-25T08:31:41.421648+0000","last_became_active":"2024-04-25T08:31:35.316569+0000","last_became_peered":"2024-04-25T08:31:35.316569+0000","last_unstale":"2024-04-25T08:31:41.421648+0000","last_undegraded":"2024-04-25T08:31:41.421648+0000","last_fullsized":"2024-04-25T08:31:41.421648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:09.703716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366542+0000","last_change":"2024-04-25T08:31:40.598075+0000","last_active":"2024-04-25T08:31:42.366542+0000","last_peered":"2024-04-25T08:31:42.366542+0000","last_clean":"2024-04-25T08:31:42.366542+0000","last_became_active":"2024-04-25T08:31:40.597669+0000","last_became_peered":"2024-04-25T08:31:40.597669+0000","last_unstale":"2024-04-25T08:31:42.366542+0000","last_undegraded":"2024-04-25T08:31:42.366542+0000","last_fullsized":"2024-04-25T08:31:42.366542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:09.948828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997776+0000","last_change":"2024-04-25T08:31:36.320808+0000","last_active":"2024-04-25T08:31:41.997776+0000","last_peered":"2024-04-25T08:31:41.997776+0000","last_clean":"2024-04-25T08:31:41.997776+0000","last_became_active":"2024-04-25T08:31:36.320489+0000","last_became_peered":"2024-04-25T08:31:36.320489+0000","last_unstale":"2024-04-25T08:31:41.997776+0000","last_undegraded":"2024-04-25T08:31:41.997776+0000","last_fullsized":"2024-04-25T08:31:41.997776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119140+0000","last_change":"2024-04-25T08:31:35.312515+0000","last_active":"2024-04-25T08:31:42.119140+0000","last_peered":"2024-04-25T08:31:42.119140+0000","last_clean":"2024-04-25T08:31:42.119140+0000","last_became_active":"2024-04-25T08:31:35.312156+0000","last_became_peered":"2024-04-25T08:31:35.312156+0000","last_unstale":"2024-04-25T08:31:42.119140+0000","last_undegraded":"2024-04-25T08:31:42.119140+0000","last_fullsized":"2024-04-25T08:31:42.119140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421744+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:31:41.421744+0000","last_peered":"2024-04-25T08:31:41.421744+0000","last_clean":"2024-04-25T08:31:41.421744+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:31:41.421744+0000","last_undegraded":"2024-04-25T08:31:41.421744+0000","last_fullsized":"2024-04-25T08:31:41.421744+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381120+0000","last_change":"2024-04-25T08:31:40.597789+0000","last_active":"2024-04-25T08:31:41.381120+0000","last_peered":"2024-04-25T08:31:41.381120+0000","last_clean":"2024-04-25T08:31:41.381120+0000","last_became_active":"2024-04-25T08:31:40.597373+0000","last_became_peered":"2024-04-25T08:31:40.597373+0000","last_unstale":"2024-04-25T08:31:41.381120+0000","last_undegraded":"2024-04-25T08:31:41.381120+0000","last_fullsized":"2024-04-25T08:31:41.381120+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422089+0000","last_change":"2024-04-25T08:31:36.323510+0000","last_active":"2024-04-25T08:31:41.422089+0000","last_peered":"2024-04-25T08:31:41.422089+0000","last_clean":"2024-04-25T08:31:41.422089+0000","last_became_active":"2024-04-25T08:31:36.322941+0000","last_became_peered":"2024-04-25T08:31:36.322941+0000","last_unstale":"2024-04-25T08:31:41.422089+0000","last_undegraded":"2024-04-25T08:31:41.422089+0000","last_fullsized":"2024-04-25T08:31:41.422089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:51.984540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384803+0000","last_change":"2024-04-25T08:31:35.318779+0000","last_active":"2024-04-25T08:31:41.384803+0000","last_peered":"2024-04-25T08:31:41.384803+0000","last_clean":"2024-04-25T08:31:41.384803+0000","last_became_active":"2024-04-25T08:31:35.318485+0000","last_became_peered":"2024-04-25T08:31:35.318485+0000","last_unstale":"2024-04-25T08:31:41.384803+0000","last_undegraded":"2024-04-25T08:31:41.384803+0000","last_fullsized":"2024-04-25T08:31:41.384803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381168+0000","last_change":"2024-04-25T08:31:40.594633+0000","last_active":"2024-04-25T08:31:41.381168+0000","last_peered":"2024-04-25T08:31:41.381168+0000","last_clean":"2024-04-25T08:31:41.381168+0000","last_became_active":"2024-04-25T08:31:40.594193+0000","last_became_peered":"2024-04-25T08:31:40.594193+0000","last_unstale":"2024-04-25T08:31:41.381168+0000","last_undegraded":"2024-04-25T08:31:41.381168+0000","last_fullsized":"2024-04-25T08:31:41.381168+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119593+0000","last_change":"2024-04-25T08:31:36.327715+0000","last_active":"2024-04-25T08:31:42.119593+0000","last_peered":"2024-04-25T08:31:42.119593+0000","last_clean":"2024-04-25T08:31:42.119593+0000","last_became_active":"2024-04-25T08:31:36.327408+0000","last_became_peered":"2024-04-25T08:31:36.327408+0000","last_unstale":"2024-04-25T08:31:42.119593+0000","last_undegraded":"2024-04-25T08:31:42.119593+0000","last_fullsized":"2024-04-25T08:31:42.119593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:43.612460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367236+0000","last_change":"2024-04-25T08:31:35.310104+0000","last_active":"2024-04-25T08:31:42.367236+0000","last_peered":"2024-04-25T08:31:42.367236+0000","last_clean":"2024-04-25T08:31:42.367236+0000","last_became_active":"2024-04-25T08:31:35.309469+0000","last_became_peered":"2024-04-25T08:31:35.309469+0000","last_unstale":"2024-04-25T08:31:42.367236+0000","last_undegraded":"2024-04-25T08:31:42.367236+0000","last_fullsized":"2024-04-25T08:31:42.367236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:09.703716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366537+0000","last_change":"2024-04-25T08:31:40.356105+0000","last_active":"2024-04-25T08:31:42.366537+0000","last_peered":"2024-04-25T08:31:42.366537+0000","last_clean":"2024-04-25T08:31:42.366537+0000","last_became_active":"2024-04-25T08:31:40.355734+0000","last_became_peered":"2024-04-25T08:31:40.355734+0000","last_unstale":"2024-04-25T08:31:42.366537+0000","last_undegraded":"2024-04-25T08:31:42.366537+0000","last_fullsized":"2024-04-25T08:31:42.366537+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:23.148774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422428+0000","last_change":"2024-04-25T08:31:36.325987+0000","last_active":"2024-04-25T08:31:41.422428+0000","last_peered":"2024-04-25T08:31:41.422428+0000","last_clean":"2024-04-25T08:31:41.422428+0000","last_became_active":"2024-04-25T08:31:36.324131+0000","last_became_peered":"2024-04-25T08:31:36.324131+0000","last_unstale":"2024-04-25T08:31:41.422428+0000","last_undegraded":"2024-04-25T08:31:41.422428+0000","last_fullsized":"2024-04-25T08:31:41.422428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:32.255703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384855+0000","last_change":"2024-04-25T08:31:35.321968+0000","last_active":"2024-04-25T08:31:41.384855+0000","last_peered":"2024-04-25T08:31:41.384855+0000","last_clean":"2024-04-25T08:31:41.384855+0000","last_became_active":"2024-04-25T08:31:35.321609+0000","last_became_peered":"2024-04-25T08:31:35.321609+0000","last_unstale":"2024-04-25T08:31:41.384855+0000","last_undegraded":"2024-04-25T08:31:41.384855+0000","last_fullsized":"2024-04-25T08:31:41.384855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118903+0000","last_change":"2024-04-25T08:31:40.597471+0000","last_active":"2024-04-25T08:31:42.118903+0000","last_peered":"2024-04-25T08:31:42.118903+0000","last_clean":"2024-04-25T08:31:42.118903+0000","last_became_active":"2024-04-25T08:31:40.597017+0000","last_became_peered":"2024-04-25T08:31:40.597017+0000","last_unstale":"2024-04-25T08:31:42.118903+0000","last_undegraded":"2024-04-25T08:31:42.118903+0000","last_fullsized":"2024-04-25T08:31:42.118903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:10.902214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422271+0000","last_change":"2024-04-25T08:31:36.325470+0000","last_active":"2024-04-25T08:31:41.422271+0000","last_peered":"2024-04-25T08:31:41.422271+0000","last_clean":"2024-04-25T08:31:41.422271+0000","last_became_active":"2024-04-25T08:31:36.325177+0000","last_became_peered":"2024-04-25T08:31:36.325177+0000","last_unstale":"2024-04-25T08:31:41.422271+0000","last_undegraded":"2024-04-25T08:31:41.422271+0000","last_fullsized":"2024-04-25T08:31:41.422271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:19.120016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381794+0000","last_change":"2024-04-25T08:31:35.312440+0000","last_active":"2024-04-25T08:31:41.381794+0000","last_peered":"2024-04-25T08:31:41.381794+0000","last_clean":"2024-04-25T08:31:41.381794+0000","last_became_active":"2024-04-25T08:31:35.312130+0000","last_became_peered":"2024-04-25T08:31:35.312130+0000","last_unstale":"2024-04-25T08:31:41.381794+0000","last_undegraded":"2024-04-25T08:31:41.381794+0000","last_fullsized":"2024-04-25T08:31:41.381794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:32.280995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381222+0000","last_change":"2024-04-25T08:31:40.351471+0000","last_active":"2024-04-25T08:31:41.381222+0000","last_peered":"2024-04-25T08:31:41.381222+0000","last_clean":"2024-04-25T08:31:41.381222+0000","last_became_active":"2024-04-25T08:31:40.351036+0000","last_became_peered":"2024-04-25T08:31:40.351036+0000","last_unstale":"2024-04-25T08:31:41.381222+0000","last_undegraded":"2024-04-25T08:31:41.381222+0000","last_fullsized":"2024-04-25T08:31:41.381222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:30.214541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421850+0000","last_change":"2024-04-25T08:31:36.325960+0000","last_active":"2024-04-25T08:31:41.421850+0000","last_peered":"2024-04-25T08:31:41.421850+0000","last_clean":"2024-04-25T08:31:41.421850+0000","last_became_active":"2024-04-25T08:31:36.325667+0000","last_became_peered":"2024-04-25T08:31:36.325667+0000","last_unstale":"2024-04-25T08:31:41.421850+0000","last_undegraded":"2024-04-25T08:31:41.421850+0000","last_fullsized":"2024-04-25T08:31:41.421850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:47.399399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384402+0000","last_change":"2024-04-25T08:31:35.307681+0000","last_active":"2024-04-25T08:31:41.384402+0000","last_peered":"2024-04-25T08:31:41.384402+0000","last_clean":"2024-04-25T08:31:41.384402+0000","last_became_active":"2024-04-25T08:31:35.307213+0000","last_became_peered":"2024-04-25T08:31:35.307213+0000","last_unstale":"2024-04-25T08:31:41.384402+0000","last_undegraded":"2024-04-25T08:31:41.384402+0000","last_fullsized":"2024-04-25T08:31:41.384402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:34.641593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366369+0000","last_change":"2024-04-25T08:31:40.352841+0000","last_active":"2024-04-25T08:31:42.366369+0000","last_peered":"2024-04-25T08:31:42.366369+0000","last_clean":"2024-04-25T08:31:42.366369+0000","last_became_active":"2024-04-25T08:31:40.352418+0000","last_became_peered":"2024-04-25T08:31:40.352418+0000","last_unstale":"2024-04-25T08:31:42.366369+0000","last_undegraded":"2024-04-25T08:31:42.366369+0000","last_fullsized":"2024-04-25T08:31:42.366369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:56.013298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.380985+0000","last_change":"2024-04-25T08:31:36.312845+0000","last_active":"2024-04-25T08:31:41.380985+0000","last_peered":"2024-04-25T08:31:41.380985+0000","last_clean":"2024-04-25T08:31:41.380985+0000","last_became_active":"2024-04-25T08:31:36.312538+0000","last_became_peered":"2024-04-25T08:31:36.312538+0000","last_unstale":"2024-04-25T08:31:41.380985+0000","last_undegraded":"2024-04-25T08:31:41.380985+0000","last_fullsized":"2024-04-25T08:31:41.380985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381431+0000","last_change":"2024-04-25T08:31:35.305736+0000","last_active":"2024-04-25T08:31:41.381431+0000","last_peered":"2024-04-25T08:31:41.381431+0000","last_clean":"2024-04-25T08:31:41.381431+0000","last_became_active":"2024-04-25T08:31:35.305481+0000","last_became_peered":"2024-04-25T08:31:35.305481+0000","last_unstale":"2024-04-25T08:31:41.381431+0000","last_undegraded":"2024-04-25T08:31:41.381431+0000","last_fullsized":"2024-04-25T08:31:41.381431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998328+0000","last_change":"2024-04-25T08:31:40.599441+0000","last_active":"2024-04-25T08:31:41.998328+0000","last_peered":"2024-04-25T08:31:41.998328+0000","last_clean":"2024-04-25T08:31:41.998328+0000","last_became_active":"2024-04-25T08:31:40.599023+0000","last_became_peered":"2024-04-25T08:31:40.599023+0000","last_unstale":"2024-04-25T08:31:41.998328+0000","last_undegraded":"2024-04-25T08:31:41.998328+0000","last_fullsized":"2024-04-25T08:31:41.998328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:09.443843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421936+0000","last_change":"2024-04-25T08:31:36.323783+0000","last_active":"2024-04-25T08:31:41.421936+0000","last_peered":"2024-04-25T08:31:41.421936+0000","last_clean":"2024-04-25T08:31:41.421936+0000","last_became_active":"2024-04-25T08:31:36.323460+0000","last_became_peered":"2024-04-25T08:31:36.323460+0000","last_unstale":"2024-04-25T08:31:41.421936+0000","last_undegraded":"2024-04-25T08:31:41.421936+0000","last_fullsized":"2024-04-25T08:31:41.421936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119196+0000","last_change":"2024-04-25T08:31:35.318378+0000","last_active":"2024-04-25T08:31:42.119196+0000","last_peered":"2024-04-25T08:31:42.119196+0000","last_clean":"2024-04-25T08:31:42.119196+0000","last_became_active":"2024-04-25T08:31:35.318103+0000","last_became_peered":"2024-04-25T08:31:35.318103+0000","last_unstale":"2024-04-25T08:31:42.119196+0000","last_undegraded":"2024-04-25T08:31:42.119196+0000","last_fullsized":"2024-04-25T08:31:42.119196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998302+0000","last_change":"2024-04-25T08:31:40.595629+0000","last_active":"2024-04-25T08:31:41.998302+0000","last_peered":"2024-04-25T08:31:41.998302+0000","last_clean":"2024-04-25T08:31:41.998302+0000","last_became_active":"2024-04-25T08:31:40.594160+0000","last_became_peered":"2024-04-25T08:31:40.594160+0000","last_unstale":"2024-04-25T08:31:41.998302+0000","last_undegraded":"2024-04-25T08:31:41.998302+0000","last_fullsized":"2024-04-25T08:31:41.998302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:00.245750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366188+0000","last_change":"2024-04-25T08:31:36.329619+0000","last_active":"2024-04-25T08:31:42.366188+0000","last_peered":"2024-04-25T08:31:42.366188+0000","last_clean":"2024-04-25T08:31:42.366188+0000","last_became_active":"2024-04-25T08:31:36.329319+0000","last_became_peered":"2024-04-25T08:31:36.329319+0000","last_unstale":"2024-04-25T08:31:42.366188+0000","last_undegraded":"2024-04-25T08:31:42.366188+0000","last_fullsized":"2024-04-25T08:31:42.366188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:32.370451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366817+0000","last_change":"2024-04-25T08:31:35.317195+0000","last_active":"2024-04-25T08:31:42.366817+0000","last_peered":"2024-04-25T08:31:42.366817+0000","last_clean":"2024-04-25T08:31:42.366817+0000","last_became_active":"2024-04-25T08:31:35.316819+0000","last_became_peered":"2024-04-25T08:31:35.316819+0000","last_unstale":"2024-04-25T08:31:42.366817+0000","last_undegraded":"2024-04-25T08:31:42.366817+0000","last_fullsized":"2024-04-25T08:31:42.366817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:32.280995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119540+0000","last_change":"2024-04-25T08:31:40.594853+0000","last_active":"2024-04-25T08:31:42.119540+0000","last_peered":"2024-04-25T08:31:42.119540+0000","last_clean":"2024-04-25T08:31:42.119540+0000","last_became_active":"2024-04-25T08:31:40.594456+0000","last_became_peered":"2024-04-25T08:31:40.594456+0000","last_unstale":"2024-04-25T08:31:42.119540+0000","last_undegraded":"2024-04-25T08:31:42.119540+0000","last_fullsized":"2024-04-25T08:31:42.119540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.735540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422019+0000","last_change":"2024-04-25T08:31:36.324840+0000","last_active":"2024-04-25T08:31:41.422019+0000","last_peered":"2024-04-25T08:31:41.422019+0000","last_clean":"2024-04-25T08:31:41.422019+0000","last_became_active":"2024-04-25T08:31:36.324574+0000","last_became_peered":"2024-04-25T08:31:36.324574+0000","last_unstale":"2024-04-25T08:31:41.422019+0000","last_undegraded":"2024-04-25T08:31:41.422019+0000","last_fullsized":"2024-04-25T08:31:41.422019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.111565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:45.647660+0000","last_change":"2024-04-25T08:31:35.316767+0000","last_active":"2024-04-25T08:31:45.647660+0000","last_peered":"2024-04-25T08:31:45.647660+0000","last_clean":"2024-04-25T08:31:45.647660+0000","last_became_active":"2024-04-25T08:31:35.316484+0000","last_became_peered":"2024-04-25T08:31:35.316484+0000","last_unstale":"2024-04-25T08:31:45.647660+0000","last_undegraded":"2024-04-25T08:31:45.647660+0000","last_fullsized":"2024-04-25T08:31:45.647660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366316+0000","last_change":"2024-04-25T08:31:40.355742+0000","last_active":"2024-04-25T08:31:42.366316+0000","last_peered":"2024-04-25T08:31:42.366316+0000","last_clean":"2024-04-25T08:31:42.366316+0000","last_became_active":"2024-04-25T08:31:40.355357+0000","last_became_peered":"2024-04-25T08:31:40.355357+0000","last_unstale":"2024-04-25T08:31:42.366316+0000","last_undegraded":"2024-04-25T08:31:42.366316+0000","last_fullsized":"2024-04-25T08:31:42.366316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382374+0000","last_change":"2024-04-25T08:31:36.313482+0000","last_active":"2024-04-25T08:31:41.382374+0000","last_peered":"2024-04-25T08:31:41.382374+0000","last_clean":"2024-04-25T08:31:41.382374+0000","last_became_active":"2024-04-25T08:31:36.312650+0000","last_became_peered":"2024-04-25T08:31:36.312650+0000","last_unstale":"2024-04-25T08:31:41.382374+0000","last_undegraded":"2024-04-25T08:31:41.382374+0000","last_fullsized":"2024-04-25T08:31:41.382374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:10.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367199+0000","last_change":"2024-04-25T08:31:35.320355+0000","last_active":"2024-04-25T08:31:42.367199+0000","last_peered":"2024-04-25T08:31:42.367199+0000","last_clean":"2024-04-25T08:31:42.367199+0000","last_became_active":"2024-04-25T08:31:35.320046+0000","last_became_peered":"2024-04-25T08:31:35.320046+0000","last_unstale":"2024-04-25T08:31:42.367199+0000","last_undegraded":"2024-04-25T08:31:42.367199+0000","last_fullsized":"2024-04-25T08:31:42.367199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:46.434047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382283+0000","last_change":"2024-04-25T08:31:40.356433+0000","last_active":"2024-04-25T08:31:41.382283+0000","last_peered":"2024-04-25T08:31:41.382283+0000","last_clean":"2024-04-25T08:31:41.382283+0000","last_became_active":"2024-04-25T08:31:40.355755+0000","last_became_peered":"2024-04-25T08:31:40.355755+0000","last_unstale":"2024-04-25T08:31:41.382283+0000","last_undegraded":"2024-04-25T08:31:41.382283+0000","last_fullsized":"2024-04-25T08:31:41.382283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:51.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366137+0000","last_change":"2024-04-25T08:31:36.323385+0000","last_active":"2024-04-25T08:31:42.366137+0000","last_peered":"2024-04-25T08:31:42.366137+0000","last_clean":"2024-04-25T08:31:42.366137+0000","last_became_active":"2024-04-25T08:31:36.322674+0000","last_became_peered":"2024-04-25T08:31:36.322674+0000","last_unstale":"2024-04-25T08:31:42.366137+0000","last_undegraded":"2024-04-25T08:31:42.366137+0000","last_fullsized":"2024-04-25T08:31:42.366137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119486+0000","last_change":"2024-04-25T08:31:35.317212+0000","last_active":"2024-04-25T08:31:42.119486+0000","last_peered":"2024-04-25T08:31:42.119486+0000","last_clean":"2024-04-25T08:31:42.119486+0000","last_became_active":"2024-04-25T08:31:35.316922+0000","last_became_peered":"2024-04-25T08:31:35.316922+0000","last_unstale":"2024-04-25T08:31:42.119486+0000","last_undegraded":"2024-04-25T08:31:42.119486+0000","last_fullsized":"2024-04-25T08:31:42.119486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.726901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997655+0000","last_change":"2024-04-25T08:31:40.353781+0000","last_active":"2024-04-25T08:31:41.997655+0000","last_peered":"2024-04-25T08:31:41.997655+0000","last_clean":"2024-04-25T08:31:41.997655+0000","last_became_active":"2024-04-25T08:31:40.352269+0000","last_became_peered":"2024-04-25T08:31:40.352269+0000","last_unstale":"2024-04-25T08:31:41.997655+0000","last_undegraded":"2024-04-25T08:31:41.997655+0000","last_fullsized":"2024-04-25T08:31:41.997655+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:23.063716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119364+0000","last_change":"2024-04-25T08:31:36.324056+0000","last_active":"2024-04-25T08:31:42.119364+0000","last_peered":"2024-04-25T08:31:42.119364+0000","last_clean":"2024-04-25T08:31:42.119364+0000","last_became_active":"2024-04-25T08:31:36.323790+0000","last_became_peered":"2024-04-25T08:31:36.323790+0000","last_unstale":"2024-04-25T08:31:42.119364+0000","last_undegraded":"2024-04-25T08:31:42.119364+0000","last_fullsized":"2024-04-25T08:31:42.119364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:19.120016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382208+0000","last_change":"2024-04-25T08:31:35.318093+0000","last_active":"2024-04-25T08:31:41.382208+0000","last_peered":"2024-04-25T08:31:41.382208+0000","last_clean":"2024-04-25T08:31:41.382208+0000","last_became_active":"2024-04-25T08:31:35.317828+0000","last_became_peered":"2024-04-25T08:31:35.317828+0000","last_unstale":"2024-04-25T08:31:41.382208+0000","last_undegraded":"2024-04-25T08:31:41.382208+0000","last_fullsized":"2024-04-25T08:31:41.382208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:54.173888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997667+0000","last_change":"2024-04-25T08:31:40.599421+0000","last_active":"2024-04-25T08:31:41.997667+0000","last_peered":"2024-04-25T08:31:41.997667+0000","last_clean":"2024-04-25T08:31:41.997667+0000","last_became_active":"2024-04-25T08:31:40.599105+0000","last_became_peered":"2024-04-25T08:31:40.599105+0000","last_unstale":"2024-04-25T08:31:41.997667+0000","last_undegraded":"2024-04-25T08:31:41.997667+0000","last_fullsized":"2024-04-25T08:31:41.997667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:50.738838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366324+0000","last_change":"2024-04-25T08:31:40.597001+0000","last_active":"2024-04-25T08:31:42.366324+0000","last_peered":"2024-04-25T08:31:42.366324+0000","last_clean":"2024-04-25T08:31:42.366324+0000","last_became_active":"2024-04-25T08:31:40.596569+0000","last_became_peered":"2024-04-25T08:31:40.596569+0000","last_unstale":"2024-04-25T08:31:42.366324+0000","last_undegraded":"2024-04-25T08:31:42.366324+0000","last_fullsized":"2024-04-25T08:31:42.366324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367007+0000","last_change":"2024-04-25T08:31:36.332669+0000","last_active":"2024-04-25T08:31:42.367007+0000","last_peered":"2024-04-25T08:31:42.367007+0000","last_clean":"2024-04-25T08:31:42.367007+0000","last_became_active":"2024-04-25T08:31:36.332305+0000","last_became_peered":"2024-04-25T08:31:36.332305+0000","last_unstale":"2024-04-25T08:31:42.367007+0000","last_undegraded":"2024-04-25T08:31:42.367007+0000","last_fullsized":"2024-04-25T08:31:42.367007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:15.226896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118969+0000","last_change":"2024-04-25T08:31:35.317959+0000","last_active":"2024-04-25T08:31:42.118969+0000","last_peered":"2024-04-25T08:31:42.118969+0000","last_clean":"2024-04-25T08:31:42.118969+0000","last_became_active":"2024-04-25T08:31:35.317680+0000","last_became_peered":"2024-04-25T08:31:35.317680+0000","last_unstale":"2024-04-25T08:31:42.118969+0000","last_undegraded":"2024-04-25T08:31:42.118969+0000","last_fullsized":"2024-04-25T08:31:42.118969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:54.173888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382101+0000","last_change":"2024-04-25T08:31:40.352855+0000","last_active":"2024-04-25T08:31:41.382101+0000","last_peered":"2024-04-25T08:31:41.382101+0000","last_clean":"2024-04-25T08:31:41.382101+0000","last_became_active":"2024-04-25T08:31:40.352373+0000","last_became_peered":"2024-04-25T08:31:40.352373+0000","last_unstale":"2024-04-25T08:31:41.382101+0000","last_undegraded":"2024-04-25T08:31:41.382101+0000","last_fullsized":"2024-04-25T08:31:41.382101+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:56.013298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997618+0000","last_change":"2024-04-25T08:31:36.327572+0000","last_active":"2024-04-25T08:31:41.997618+0000","last_peered":"2024-04-25T08:31:41.997618+0000","last_clean":"2024-04-25T08:31:41.997618+0000","last_became_active":"2024-04-25T08:31:36.320635+0000","last_became_peered":"2024-04-25T08:31:36.320635+0000","last_unstale":"2024-04-25T08:31:41.997618+0000","last_undegraded":"2024-04-25T08:31:41.997618+0000","last_fullsized":"2024-04-25T08:31:41.997618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997788+0000","last_change":"2024-04-25T08:31:35.317367+0000","last_active":"2024-04-25T08:31:41.997788+0000","last_peered":"2024-04-25T08:31:41.997788+0000","last_clean":"2024-04-25T08:31:41.997788+0000","last_became_active":"2024-04-25T08:31:35.317013+0000","last_became_peered":"2024-04-25T08:31:35.317013+0000","last_unstale":"2024-04-25T08:31:41.997788+0000","last_undegraded":"2024-04-25T08:31:41.997788+0000","last_fullsized":"2024-04-25T08:31:41.997788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:34.641593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381422+0000","last_change":"2024-04-25T08:31:35.305651+0000","last_active":"2024-04-25T08:31:41.381422+0000","last_peered":"2024-04-25T08:31:41.381422+0000","last_clean":"2024-04-25T08:31:41.381422+0000","last_became_active":"2024-04-25T08:31:35.305381+0000","last_became_peered":"2024-04-25T08:31:35.305381+0000","last_unstale":"2024-04-25T08:31:41.381422+0000","last_undegraded":"2024-04-25T08:31:41.381422+0000","last_fullsized":"2024-04-25T08:31:41.381422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366718+0000","last_change":"2024-04-25T08:31:36.323372+0000","last_active":"2024-04-25T08:31:42.366718+0000","last_peered":"2024-04-25T08:31:42.366718+0000","last_clean":"2024-04-25T08:31:42.366718+0000","last_became_active":"2024-04-25T08:31:36.322950+0000","last_became_peered":"2024-04-25T08:31:36.322950+0000","last_unstale":"2024-04-25T08:31:42.366718+0000","last_undegraded":"2024-04-25T08:31:42.366718+0000","last_fullsized":"2024-04-25T08:31:42.366718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384753+0000","last_change":"2024-04-25T08:31:40.353992+0000","last_active":"2024-04-25T08:31:41.384753+0000","last_peered":"2024-04-25T08:31:41.384753+0000","last_clean":"2024-04-25T08:31:41.384753+0000","last_became_active":"2024-04-25T08:31:40.353601+0000","last_became_peered":"2024-04-25T08:31:40.353601+0000","last_unstale":"2024-04-25T08:31:41.384753+0000","last_undegraded":"2024-04-25T08:31:41.384753+0000","last_fullsized":"2024-04-25T08:31:41.384753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:02.808345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997972+0000","last_change":"2024-04-25T08:31:35.321307+0000","last_active":"2024-04-25T08:31:41.997972+0000","last_peered":"2024-04-25T08:31:41.997972+0000","last_clean":"2024-04-25T08:31:41.997972+0000","last_became_active":"2024-04-25T08:31:35.320720+0000","last_became_peered":"2024-04-25T08:31:35.320720+0000","last_unstale":"2024-04-25T08:31:41.997972+0000","last_undegraded":"2024-04-25T08:31:41.997972+0000","last_fullsized":"2024-04-25T08:31:41.997972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381993+0000","last_change":"2024-04-25T08:31:36.317597+0000","last_active":"2024-04-25T08:31:41.381993+0000","last_peered":"2024-04-25T08:31:41.381993+0000","last_clean":"2024-04-25T08:31:41.381993+0000","last_became_active":"2024-04-25T08:31:36.317333+0000","last_became_peered":"2024-04-25T08:31:36.317333+0000","last_unstale":"2024-04-25T08:31:41.381993+0000","last_undegraded":"2024-04-25T08:31:41.381993+0000","last_fullsized":"2024-04-25T08:31:41.381993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:55.191462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998205+0000","last_change":"2024-04-25T08:31:40.357847+0000","last_active":"2024-04-25T08:31:41.998205+0000","last_peered":"2024-04-25T08:31:41.998205+0000","last_clean":"2024-04-25T08:31:41.998205+0000","last_became_active":"2024-04-25T08:31:40.357445+0000","last_became_peered":"2024-04-25T08:31:40.357445+0000","last_unstale":"2024-04-25T08:31:41.998205+0000","last_undegraded":"2024-04-25T08:31:41.998205+0000","last_fullsized":"2024-04-25T08:31:41.998205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:45.166751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381524+0000","last_change":"2024-04-25T08:31:36.310801+0000","last_active":"2024-04-25T08:31:41.381524+0000","last_peered":"2024-04-25T08:31:41.381524+0000","last_clean":"2024-04-25T08:31:41.381524+0000","last_became_active":"2024-04-25T08:31:36.310509+0000","last_became_peered":"2024-04-25T08:31:36.310509+0000","last_unstale":"2024-04-25T08:31:41.381524+0000","last_undegraded":"2024-04-25T08:31:41.381524+0000","last_fullsized":"2024-04-25T08:31:41.381524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384592+0000","last_change":"2024-04-25T08:31:36.320666+0000","last_active":"2024-04-25T08:31:41.384592+0000","last_peered":"2024-04-25T08:31:41.384592+0000","last_clean":"2024-04-25T08:31:41.384592+0000","last_became_active":"2024-04-25T08:31:36.320402+0000","last_became_peered":"2024-04-25T08:31:36.320402+0000","last_unstale":"2024-04-25T08:31:41.384592+0000","last_undegraded":"2024-04-25T08:31:41.384592+0000","last_fullsized":"2024-04-25T08:31:41.384592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:47.451621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119194+0000","last_change":"2024-04-25T08:31:36.323523+0000","last_active":"2024-04-25T08:31:42.119194+0000","last_peered":"2024-04-25T08:31:42.119194+0000","last_clean":"2024-04-25T08:31:42.119194+0000","last_became_active":"2024-04-25T08:31:36.323233+0000","last_became_peered":"2024-04-25T08:31:36.323233+0000","last_unstale":"2024-04-25T08:31:42.119194+0000","last_undegraded":"2024-04-25T08:31:42.119194+0000","last_fullsized":"2024-04-25T08:31:42.119194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:46.700115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366631+0000","last_change":"2024-04-25T08:31:36.332567+0000","last_active":"2024-04-25T08:31:42.366631+0000","last_peered":"2024-04-25T08:31:42.366631+0000","last_clean":"2024-04-25T08:31:42.366631+0000","last_became_active":"2024-04-25T08:31:36.332300+0000","last_became_peered":"2024-04-25T08:31:36.332300+0000","last_unstale":"2024-04-25T08:31:42.366631+0000","last_undegraded":"2024-04-25T08:31:42.366631+0000","last_fullsized":"2024-04-25T08:31:42.366631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:57.933443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998117+0000","last_change":"2024-04-25T08:31:36.326949+0000","last_active":"2024-04-25T08:31:41.998117+0000","last_peered":"2024-04-25T08:31:41.998117+0000","last_clean":"2024-04-25T08:31:41.998117+0000","last_became_active":"2024-04-25T08:31:36.322467+0000","last_became_peered":"2024-04-25T08:31:36.322467+0000","last_unstale":"2024-04-25T08:31:41.998117+0000","last_undegraded":"2024-04-25T08:31:41.998117+0000","last_fullsized":"2024-04-25T08:31:41.998117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:46.700115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367093+0000","last_change":"2024-04-25T08:31:36.331852+0000","last_active":"2024-04-25T08:31:42.367093+0000","last_peered":"2024-04-25T08:31:42.367093+0000","last_clean":"2024-04-25T08:31:42.367093+0000","last_became_active":"2024-04-25T08:31:36.331588+0000","last_became_peered":"2024-04-25T08:31:36.331588+0000","last_unstale":"2024-04-25T08:31:42.367093+0000","last_undegraded":"2024-04-25T08:31:42.367093+0000","last_fullsized":"2024-04-25T08:31:42.367093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381821+0000","last_change":"2024-04-25T08:31:36.309132+0000","last_active":"2024-04-25T08:31:41.381821+0000","last_peered":"2024-04-25T08:31:41.381821+0000","last_clean":"2024-04-25T08:31:41.381821+0000","last_became_active":"2024-04-25T08:31:36.308790+0000","last_became_peered":"2024-04-25T08:31:36.308790+0000","last_unstale":"2024-04-25T08:31:41.381821+0000","last_undegraded":"2024-04-25T08:31:41.381821+0000","last_fullsized":"2024-04-25T08:31:41.381821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:14.177622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998237+0000","last_change":"2024-04-25T08:31:36.320300+0000","last_active":"2024-04-25T08:31:41.998237+0000","last_peered":"2024-04-25T08:31:41.998237+0000","last_clean":"2024-04-25T08:31:41.998237+0000","last_became_active":"2024-04-25T08:31:36.319984+0000","last_became_peered":"2024-04-25T08:31:36.319984+0000","last_unstale":"2024-04-25T08:31:41.998237+0000","last_undegraded":"2024-04-25T08:31:41.998237+0000","last_fullsized":"2024-04-25T08:31:41.998237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118927+0000","last_change":"2024-04-25T08:31:36.329425+0000","last_active":"2024-04-25T08:31:42.118927+0000","last_peered":"2024-04-25T08:31:42.118927+0000","last_clean":"2024-04-25T08:31:42.118927+0000","last_became_active":"2024-04-25T08:31:36.328879+0000","last_became_peered":"2024-04-25T08:31:36.328879+0000","last_unstale":"2024-04-25T08:31:42.118927+0000","last_undegraded":"2024-04-25T08:31:42.118927+0000","last_fullsized":"2024-04-25T08:31:42.118927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997886+0000","last_change":"2024-04-25T08:31:36.322034+0000","last_active":"2024-04-25T08:31:41.997886+0000","last_peered":"2024-04-25T08:31:41.997886+0000","last_clean":"2024-04-25T08:31:41.997886+0000","last_became_active":"2024-04-25T08:31:36.321723+0000","last_became_peered":"2024-04-25T08:31:36.321723+0000","last_unstale":"2024-04-25T08:31:41.997886+0000","last_undegraded":"2024-04-25T08:31:41.997886+0000","last_fullsized":"2024-04-25T08:31:41.997886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119094+0000","last_change":"2024-04-25T08:31:36.325939+0000","last_active":"2024-04-25T08:31:42.119094+0000","last_peered":"2024-04-25T08:31:42.119094+0000","last_clean":"2024-04-25T08:31:42.119094+0000","last_became_active":"2024-04-25T08:31:36.325469+0000","last_became_peered":"2024-04-25T08:31:36.325469+0000","last_unstale":"2024-04-25T08:31:42.119094+0000","last_undegraded":"2024-04-25T08:31:42.119094+0000","last_fullsized":"2024-04-25T08:31:42.119094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:14.177622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366378+0000","last_change":"2024-04-25T08:31:36.328064+0000","last_active":"2024-04-25T08:31:42.366378+0000","last_peered":"2024-04-25T08:31:42.366378+0000","last_clean":"2024-04-25T08:31:42.366378+0000","last_became_active":"2024-04-25T08:31:36.327769+0000","last_became_peered":"2024-04-25T08:31:36.327769+0000","last_unstale":"2024-04-25T08:31:42.366378+0000","last_undegraded":"2024-04-25T08:31:42.366378+0000","last_fullsized":"2024-04-25T08:31:42.366378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:58.779587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381873+0000","last_change":"2024-04-25T08:31:36.318263+0000","last_active":"2024-04-25T08:31:41.381873+0000","last_peered":"2024-04-25T08:31:41.381873+0000","last_clean":"2024-04-25T08:31:41.381873+0000","last_became_active":"2024-04-25T08:31:36.317987+0000","last_became_peered":"2024-04-25T08:31:36.317987+0000","last_unstale":"2024-04-25T08:31:41.381873+0000","last_undegraded":"2024-04-25T08:31:41.381873+0000","last_fullsized":"2024-04-25T08:31:41.381873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422245+0000","last_change":"2024-04-25T08:31:36.318589+0000","last_active":"2024-04-25T08:31:41.422245+0000","last_peered":"2024-04-25T08:31:41.422245+0000","last_clean":"2024-04-25T08:31:41.422245+0000","last_became_active":"2024-04-25T08:31:36.318315+0000","last_became_peered":"2024-04-25T08:31:36.318315+0000","last_unstale":"2024-04-25T08:31:41.422245+0000","last_undegraded":"2024-04-25T08:31:41.422245+0000","last_fullsized":"2024-04-25T08:31:41.422245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:05.920070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384795+0000","last_change":"2024-04-25T08:31:36.320980+0000","last_active":"2024-04-25T08:31:41.384795+0000","last_peered":"2024-04-25T08:31:41.384795+0000","last_clean":"2024-04-25T08:31:41.384795+0000","last_became_active":"2024-04-25T08:31:36.320663+0000","last_became_peered":"2024-04-25T08:31:36.320663+0000","last_unstale":"2024-04-25T08:31:41.384795+0000","last_undegraded":"2024-04-25T08:31:41.384795+0000","last_fullsized":"2024-04-25T08:31:41.384795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:10.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119636+0000","last_change":"2024-04-25T08:31:36.328219+0000","last_active":"2024-04-25T08:31:42.119636+0000","last_peered":"2024-04-25T08:31:42.119636+0000","last_clean":"2024-04-25T08:31:42.119636+0000","last_became_active":"2024-04-25T08:31:36.327914+0000","last_became_peered":"2024-04-25T08:31:36.327914+0000","last_unstale":"2024-04-25T08:31:42.119636+0000","last_undegraded":"2024-04-25T08:31:42.119636+0000","last_fullsized":"2024-04-25T08:31:42.119636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:47.399399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118812+0000","last_change":"2024-04-25T08:31:36.322834+0000","last_active":"2024-04-25T08:31:42.118812+0000","last_peered":"2024-04-25T08:31:42.118812+0000","last_clean":"2024-04-25T08:31:42.118812+0000","last_became_active":"2024-04-25T08:31:36.322555+0000","last_became_peered":"2024-04-25T08:31:36.322555+0000","last_unstale":"2024-04-25T08:31:42.118812+0000","last_undegraded":"2024-04-25T08:31:42.118812+0000","last_fullsized":"2024-04-25T08:31:42.118812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367013+0000","last_change":"2024-04-25T08:31:36.323473+0000","last_active":"2024-04-25T08:31:42.367013+0000","last_peered":"2024-04-25T08:31:42.367013+0000","last_clean":"2024-04-25T08:31:42.367013+0000","last_became_active":"2024-04-25T08:31:36.323180+0000","last_became_peered":"2024-04-25T08:31:36.323180+0000","last_unstale":"2024-04-25T08:31:42.367013+0000","last_undegraded":"2024-04-25T08:31:42.367013+0000","last_fullsized":"2024-04-25T08:31:42.367013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366156+0000","last_change":"2024-04-25T08:31:36.314459+0000","last_active":"2024-04-25T08:31:42.366156+0000","last_peered":"2024-04-25T08:31:42.366156+0000","last_clean":"2024-04-25T08:31:42.366156+0000","last_became_active":"2024-04-25T08:31:36.313669+0000","last_became_peered":"2024-04-25T08:31:36.313669+0000","last_unstale":"2024-04-25T08:31:42.366156+0000","last_undegraded":"2024-04-25T08:31:42.366156+0000","last_fullsized":"2024-04-25T08:31:42.366156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997797+0000","last_change":"2024-04-25T08:31:36.329217+0000","last_active":"2024-04-25T08:31:41.997797+0000","last_peered":"2024-04-25T08:31:41.997797+0000","last_clean":"2024-04-25T08:31:41.997797+0000","last_became_active":"2024-04-25T08:31:36.328793+0000","last_became_peered":"2024-04-25T08:31:36.328793+0000","last_unstale":"2024-04-25T08:31:41.997797+0000","last_undegraded":"2024-04-25T08:31:41.997797+0000","last_fullsized":"2024-04-25T08:31:41.997797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:26.185783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382287+0000","last_change":"2024-04-25T08:31:36.321711+0000","last_active":"2024-04-25T08:31:41.382287+0000","last_peered":"2024-04-25T08:31:41.382287+0000","last_clean":"2024-04-25T08:31:41.382287+0000","last_became_active":"2024-04-25T08:31:36.321446+0000","last_became_peered":"2024-04-25T08:31:36.321446+0000","last_unstale":"2024-04-25T08:31:41.382287+0000","last_undegraded":"2024-04-25T08:31:41.382287+0000","last_fullsized":"2024-04-25T08:31:41.382287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:47.451621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381253+0000","last_change":"2024-04-25T08:31:36.319922+0000","last_active":"2024-04-25T08:31:41.381253+0000","last_peered":"2024-04-25T08:31:41.381253+0000","last_clean":"2024-04-25T08:31:41.381253+0000","last_became_active":"2024-04-25T08:31:36.319650+0000","last_became_peered":"2024-04-25T08:31:36.319650+0000","last_unstale":"2024-04-25T08:31:41.381253+0000","last_undegraded":"2024-04-25T08:31:41.381253+0000","last_fullsized":"2024-04-25T08:31:41.381253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381734+0000","last_change":"2024-04-25T08:31:36.319962+0000","last_active":"2024-04-25T08:31:41.381734+0000","last_peered":"2024-04-25T08:31:41.381734+0000","last_clean":"2024-04-25T08:31:41.381734+0000","last_became_active":"2024-04-25T08:31:36.319634+0000","last_became_peered":"2024-04-25T08:31:36.319634+0000","last_unstale":"2024-04-25T08:31:41.381734+0000","last_undegraded":"2024-04-25T08:31:41.381734+0000","last_fullsized":"2024-04-25T08:31:41.381734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:26.185783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422427+0000","last_change":"2024-04-25T08:31:36.320766+0000","last_active":"2024-04-25T08:31:41.422427+0000","last_peered":"2024-04-25T08:31:41.422427+0000","last_clean":"2024-04-25T08:31:41.422427+0000","last_became_active":"2024-04-25T08:31:36.320483+0000","last_became_peered":"2024-04-25T08:31:36.320483+0000","last_unstale":"2024-04-25T08:31:41.422427+0000","last_undegraded":"2024-04-25T08:31:41.422427+0000","last_fullsized":"2024-04-25T08:31:41.422427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:45.805418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997517+0000","last_change":"2024-04-25T08:31:36.327853+0000","last_active":"2024-04-25T08:31:41.997517+0000","last_peered":"2024-04-25T08:31:41.997517+0000","last_clean":"2024-04-25T08:31:41.997517+0000","last_became_active":"2024-04-25T08:31:36.319850+0000","last_became_peered":"2024-04-25T08:31:36.319850+0000","last_unstale":"2024-04-25T08:31:41.997517+0000","last_undegraded":"2024-04-25T08:31:41.997517+0000","last_fullsized":"2024-04-25T08:31:41.997517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421704+0000","last_change":"2024-04-25T08:31:36.311917+0000","last_active":"2024-04-25T08:31:41.421704+0000","last_peered":"2024-04-25T08:31:41.421704+0000","last_clean":"2024-04-25T08:31:41.421704+0000","last_became_active":"2024-04-25T08:31:36.311657+0000","last_became_peered":"2024-04-25T08:31:36.311657+0000","last_unstale":"2024-04-25T08:31:41.421704+0000","last_undegraded":"2024-04-25T08:31:41.421704+0000","last_fullsized":"2024-04-25T08:31:41.421704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:12.212803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119023+0000","last_change":"2024-04-25T08:31:36.318421+0000","last_active":"2024-04-25T08:31:42.119023+0000","last_peered":"2024-04-25T08:31:42.119023+0000","last_clean":"2024-04-25T08:31:42.119023+0000","last_became_active":"2024-04-25T08:31:36.318157+0000","last_became_peered":"2024-04-25T08:31:36.318157+0000","last_unstale":"2024-04-25T08:31:42.119023+0000","last_undegraded":"2024-04-25T08:31:42.119023+0000","last_fullsized":"2024-04-25T08:31:42.119023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:26.185783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421798+0000","last_change":"2024-04-25T08:31:36.314252+0000","last_active":"2024-04-25T08:31:41.421798+0000","last_peered":"2024-04-25T08:31:41.421798+0000","last_clean":"2024-04-25T08:31:41.421798+0000","last_became_active":"2024-04-25T08:31:36.313732+0000","last_became_peered":"2024-04-25T08:31:36.313732+0000","last_unstale":"2024-04-25T08:31:41.421798+0000","last_undegraded":"2024-04-25T08:31:41.421798+0000","last_fullsized":"2024-04-25T08:31:41.421798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:43.612460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119289+0000","last_change":"2024-04-25T08:31:36.330053+0000","last_active":"2024-04-25T08:31:42.119289+0000","last_peered":"2024-04-25T08:31:42.119289+0000","last_clean":"2024-04-25T08:31:42.119289+0000","last_became_active":"2024-04-25T08:31:36.329775+0000","last_became_peered":"2024-04-25T08:31:36.329775+0000","last_unstale":"2024-04-25T08:31:42.119289+0000","last_undegraded":"2024-04-25T08:31:42.119289+0000","last_fullsized":"2024-04-25T08:31:42.119289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:12.212803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365851+0000","last_change":"2024-04-25T08:31:36.317826+0000","last_active":"2024-04-25T08:31:42.365851+0000","last_peered":"2024-04-25T08:31:42.365851+0000","last_clean":"2024-04-25T08:31:42.365851+0000","last_became_active":"2024-04-25T08:31:36.317514+0000","last_became_peered":"2024-04-25T08:31:36.317514+0000","last_unstale":"2024-04-25T08:31:42.365851+0000","last_undegraded":"2024-04-25T08:31:42.365851+0000","last_fullsized":"2024-04-25T08:31:42.365851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367390+0000","last_change":"2024-04-25T08:31:36.310887+0000","last_active":"2024-04-25T08:31:42.367390+0000","last_peered":"2024-04-25T08:31:42.367390+0000","last_clean":"2024-04-25T08:31:42.367390+0000","last_became_active":"2024-04-25T08:31:36.310581+0000","last_became_peered":"2024-04-25T08:31:36.310581+0000","last_unstale":"2024-04-25T08:31:42.367390+0000","last_undegraded":"2024-04-25T08:31:42.367390+0000","last_fullsized":"2024-04-25T08:31:42.367390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422313+0000","last_change":"2024-04-25T08:31:36.315442+0000","last_active":"2024-04-25T08:31:41.422313+0000","last_peered":"2024-04-25T08:31:41.422313+0000","last_clean":"2024-04-25T08:31:41.422313+0000","last_became_active":"2024-04-25T08:31:36.315178+0000","last_became_peered":"2024-04-25T08:31:36.315178+0000","last_unstale":"2024-04-25T08:31:41.422313+0000","last_undegraded":"2024-04-25T08:31:41.422313+0000","last_fullsized":"2024-04-25T08:31:41.422313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:52.353609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:31:53.791 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:31:53.792 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T08:31:53.792 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T08:31:53.792 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T08:31:53.792 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T08:31:53.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.931+0000 7fe34829a640 1 Processor -- start 2024-04-25T08:31:53.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.931+0000 7fe34829a640 1 -- start start 2024-04-25T08:31:53.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.932+0000 7fe34829a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe34015a8f0 0x7fe3400cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.932+0000 7fe34829a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe3400d0950 0x7fe3400c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.932+0000 7fe34829a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3400cbd20 con 0x7fe3400cb6c0 2024-04-25T08:31:53.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.932+0000 7fe34829a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3400cc000 con 0x7fe3400d0950 2024-04-25T08:31:53.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.932+0000 7fe34829a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400cc2e0 con 0x7fe34015a8f0 2024-04-25T08:31:53.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.932+0000 7fe345835640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe3400cb6c0 0x7fe3400d0240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49774/0 (socket says 172.21.15.38:49774) 2024-04-25T08:31:53.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.932+0000 7fe345835640 1 -- 172.21.15.38:0/2312697076 learned_addr learned my addr 172.21.15.38:0/2312697076 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:53.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe346837640 1 --2- 172.21.15.38:0/2312697076 >> v2:172.21.15.107:3300/0 conn(0x7fe3400d0950 0x7fe3400c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe346036640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe34015a8f0 0x7fe3400cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 671444920 0 0) 0x7fe3400cbd20 con 0x7fe3400cb6c0 2024-04-25T08:31:53.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe328003520 con 0x7fe3400cb6c0 2024-04-25T08:31:53.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3149228553 0 0) 0x7fe328003520 con 0x7fe3400cb6c0 2024-04-25T08:31:53.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe34015a8f0 msgr2=0x7fe3400cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:53.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe34015a8f0 0x7fe3400cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 >> v2:172.21.15.107:3300/0 conn(0x7fe3400d0950 msgr2=0x7fe3400c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:53.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 --2- 172.21.15.38:0/2312697076 >> v2:172.21.15.107:3300/0 conn(0x7fe3400d0950 0x7fe3400c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3400cc5c0 con 0x7fe3400cb6c0 2024-04-25T08:31:53.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fe334001670 con 0x7fe3400cb6c0 2024-04-25T08:31:53.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe346837640 1 --2- 172.21.15.38:0/2312697076 >> v2:172.21.15.107:3300/0 conn(0x7fe3400d0950 0x7fe3400c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:53.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 >> v1:172.21.15.38:6789/0 conn(0x7fe3400cb6c0 legacy=0x7fe3400d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3280039b0 0x7fe328003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe328004800 0x7fe328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3280051c0 0x7fe32800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400cc000 con 0x7fe3280051c0 2024-04-25T08:31:53.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400cc2e0 con 0x7fe3280039b0 2024-04-25T08:31:53.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32800bf00 con 0x7fe328004800 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe346036640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe34015a8f0 0x7fe3400cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe346837640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3280051c0 0x7fe32800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe345835640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe328004800 0x7fe328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.933+0000 7fe346036640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3280039b0 0x7fe328003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.934+0000 7fe346837640 1 -- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe328004800 msgr2=0x7fe328004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.934+0000 7fe346837640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe328004800 0x7fe328004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.934+0000 7fe346837640 1 -- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3280039b0 msgr2=0x7fe328003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.934+0000 7fe346837640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3280039b0 0x7fe328003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.934+0000 7fe346837640 1 -- 172.21.15.38:0/2312697076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe328003520 con 0x7fe3280051c0 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.934+0000 7fe346036640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3280039b0 0x7fe328003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.934+0000 7fe346837640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3280051c0 0x7fe32800b9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe33000e380 tx=0x7fe328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c7589e25f9909 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.934+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330009de0 con 0x7fe3280051c0 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.934+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33002c620 con 0x7fe3280051c0 2024-04-25T08:31:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.934+0000 7fe345034640 1 -- 172.21.15.38:0/2312697076 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33002c9d0 con 0x7fe3280051c0 2024-04-25T08:31:53.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.935+0000 7fe34829a640 1 -- 172.21.15.38:0/2312697076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3280051c0 msgr2=0x7fe32800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.935+0000 7fe34829a640 1 --2- 172.21.15.38:0/2312697076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3280051c0 0x7fe32800b9c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe33000e380 tx=0x7fe328005830 comp rx=0 tx=0).stop 2024-04-25T08:31:53.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.935+0000 7fe346837640 1 -- 172.21.15.38:0/2312697076 reap_dead start 2024-04-25T08:31:53.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.935+0000 7fe34829a640 1 -- 172.21.15.38:0/2312697076 shutdown_connections 2024-04-25T08:31:53.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.935+0000 7fe34829a640 1 -- 172.21.15.38:0/2312697076 >> 172.21.15.38:0/2312697076 conn(0x7fe340072610 msgr2=0x7fe340099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:53.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.935+0000 7fe34829a640 1 -- 172.21.15.38:0/2312697076 shutdown_connections 2024-04-25T08:31:53.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.935+0000 7fe34829a640 1 -- 172.21.15.38:0/2312697076 wait complete. 2024-04-25T08:31:53.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.936+0000 7fe34829a640 1 Processor -- start 2024-04-25T08:31:53.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.936+0000 7fe34829a640 1 -- start start 2024-04-25T08:31:53.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe34829a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3280051c0 0x7fe340157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe34829a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe34015d820 0x7fe3401f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe34829a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3400c9910 0x7fe3400c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe34829a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34015acd0 con 0x7fe34015d820 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe34829a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400ce470 con 0x7fe3400c9910 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe34829a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34015ae50 con 0x7fe3280051c0 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe345835640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe34015d820 0x7fe3401f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe345835640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe34015d820 0x7fe3401f1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33346/0 (socket says 172.21.15.38:33346) 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe345835640 1 -- 172.21.15.38:0/4163611538 learned_addr learned my addr 172.21.15.38:0/4163611538 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe345835640 1 -- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3280051c0 msgr2=0x7fe340157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe346837640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3400c9910 0x7fe3400c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe345835640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3280051c0 0x7fe340157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe345835640 1 -- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3400c9910 msgr2=0x7fe3400c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:53.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe345835640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3400c9910 0x7fe3400c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe345835640 1 -- 172.21.15.38:0/4163611538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34009b220 con 0x7fe34015d820 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe346837640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3400c9910 0x7fe3400c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.937+0000 7fe345835640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe34015d820 0x7fe3401f1ab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe334004fa0 tx=0x7fe3340074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.938+0000 7fe33effd640 1 -- 172.21.15.38:0/4163611538 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33401a070 con 0x7fe34015d820 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.938+0000 7fe33effd640 1 -- 172.21.15.38:0/4163611538 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe334007740 con 0x7fe34015d820 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.938+0000 7fe34829a640 1 -- 172.21.15.38:0/4163611538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3401f1ff0 con 0x7fe34015d820 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.938+0000 7fe33effd640 1 -- 172.21.15.38:0/4163611538 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33400fb70 con 0x7fe34015d820 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.938+0000 7fe34829a640 1 -- 172.21.15.38:0/4163611538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3401f2570 con 0x7fe34015d820 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.939+0000 7fe34829a640 1 -- 172.21.15.38:0/4163611538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3400ca2d0 con 0x7fe34015d820 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.939+0000 7fe33effd640 1 -- 172.21.15.38:0/4163611538 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe33401d070 con 0x7fe34015d820 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.939+0000 7fe33effd640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe32c06b4d0 0x7fe32c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.940+0000 7fe33effd640 1 -- 172.21.15.38:0/4163611538 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe334089d90 con 0x7fe34015d820 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.940+0000 7fe346036640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe32c06b4d0 0x7fe32c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.940+0000 7fe346036640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe32c06b4d0 0x7fe32c06d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3401f2be0 tx=0x7fe338008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:53.944+0000 7fe33effd640 1 -- 172.21.15.38:0/4163611538 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe334058960 con 0x7fe34015d820 2024-04-25T08:31:54.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.172+0000 7fe34829a640 1 -- 172.21.15.38:0/4163611538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe3400d0330 con 0x7fe34015d820 2024-04-25T08:31:54.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.175+0000 7fe33effd640 1 -- 172.21.15.38:0/4163611538 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171792 (secure 0 0 0) 0x7fe334058260 con 0x7fe34015d820 2024-04-25T08:31:54.174 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:54.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.178+0000 7fe34829a640 1 -- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe32c06b4d0 msgr2=0x7fe32c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.178+0000 7fe34829a640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe32c06b4d0 0x7fe32c06d990 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe3401f2be0 tx=0x7fe338008040 comp rx=0 tx=0).stop 2024-04-25T08:31:54.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.178+0000 7fe34829a640 1 -- 172.21.15.38:0/4163611538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe34015d820 msgr2=0x7fe3401f1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.178+0000 7fe34829a640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe34015d820 0x7fe3401f1ab0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe334004fa0 tx=0x7fe3340074f0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.178+0000 7fe34829a640 1 -- 172.21.15.38:0/4163611538 shutdown_connections 2024-04-25T08:31:54.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.178+0000 7fe34829a640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3400c9910 0x7fe3400c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.178+0000 7fe34829a640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe34015d820 0x7fe3401f1ab0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.178+0000 7fe34829a640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe32c06b4d0 0x7fe32c06d990 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.178+0000 7fe34829a640 1 --2- 172.21.15.38:0/4163611538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3280051c0 0x7fe340157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.178+0000 7fe34829a640 1 -- 172.21.15.38:0/4163611538 >> 172.21.15.38:0/4163611538 conn(0x7fe340072610 msgr2=0x7fe34006fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:54.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.178+0000 7fe34829a640 1 -- 172.21.15.38:0/4163611538 shutdown_connections 2024-04-25T08:31:54.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.178+0000 7fe34829a640 1 -- 172.21.15.38:0/4163611538 wait complete. 2024-04-25T08:31:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1055593366},{"type":"v1","addr":"172.21.15.107:6833","nonce":1055593366}]},"active_addr":"172.21.15.107:6833/1055593366","active_change":"2024-04-25T08:31:15.191148+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3710997297}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2533894020}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2919514795}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1976958183}]}]} 2024-04-25T08:31:54.188 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T08:31:54.188 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T08:31:54.188 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.293+0000 7fc98406b640 1 Processor -- start 2024-04-25T08:31:54.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.293+0000 7fc98406b640 1 -- start start 2024-04-25T08:31:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.294+0000 7fc98406b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c0cf510 0x7fc97c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.294+0000 7fc98406b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc97c0cf8f0 0x7fc97c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.294+0000 7fc98406b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97c15ce50 con 0x7fc97c15c920 2024-04-25T08:31:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.294+0000 7fc98406b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97c06e170 con 0x7fc97c0cf8f0 2024-04-25T08:31:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.294+0000 7fc98406b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c15d100 con 0x7fc97c0cf510 2024-04-25T08:31:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.294+0000 7fc981606640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc97c0cf8f0 0x7fc97c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.294+0000 7fc982608640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc97c15c920 0x7fc97c15f490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49790/0 (socket says 172.21.15.38:49790) 2024-04-25T08:31:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.294+0000 7fc981606640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc97c0cf8f0 0x7fc97c06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33348/0 (socket says 172.21.15.38:33348) 2024-04-25T08:31:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.294+0000 7fc982608640 1 -- 172.21.15.38:0/2452497894 learned_addr learned my addr 172.21.15.38:0/2452497894 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.295+0000 7fc981e07640 1 --2- 172.21.15.38:0/2452497894 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c0cf510 0x7fc97c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.295+0000 7fc981606640 1 -- 172.21.15.38:0/2452497894 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c0cf510 msgr2=0x7fc97c06fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:54.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.295+0000 7fc981606640 1 --2- 172.21.15.38:0/2452497894 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c0cf510 0x7fc97c06fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.295+0000 7fc981606640 1 -- 172.21.15.38:0/2452497894 >> v1:172.21.15.38:6789/0 conn(0x7fc97c15c920 legacy=0x7fc97c15f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.295+0000 7fc981606640 1 -- 172.21.15.38:0/2452497894 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c06e410 con 0x7fc97c0cf8f0 2024-04-25T08:31:54.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.295+0000 7fc980e05640 1 -- 172.21.15.38:0/2452497894 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685408793 0 0) 0x7fc97c15ce50 con 0x7fc97c15c920 2024-04-25T08:31:54.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.295+0000 7fc981606640 1 --2- 172.21.15.38:0/2452497894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc97c0cf8f0 0x7fc97c06bbf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc9700097a0 tx=0x7fc97002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb107cd202d8dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:54.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.296+0000 7fc980e05640 1 -- 172.21.15.38:0/2452497894 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970037070 con 0x7fc97c0cf8f0 2024-04-25T08:31:54.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.296+0000 7fc980e05640 1 -- 172.21.15.38:0/2452497894 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc97002ae50 con 0x7fc97c0cf8f0 2024-04-25T08:31:54.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.296+0000 7fc980e05640 1 -- 172.21.15.38:0/2452497894 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970032660 con 0x7fc97c0cf8f0 2024-04-25T08:31:54.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.296+0000 7fc98406b640 1 -- 172.21.15.38:0/2452497894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc97c0cf8f0 msgr2=0x7fc97c06bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.296+0000 7fc98406b640 1 --2- 172.21.15.38:0/2452497894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc97c0cf8f0 0x7fc97c06bbf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc9700097a0 tx=0x7fc97002a100 comp rx=0 tx=0).stop 2024-04-25T08:31:54.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.296+0000 7fc98406b640 1 -- 172.21.15.38:0/2452497894 shutdown_connections 2024-04-25T08:31:54.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.296+0000 7fc98406b640 1 --2- 172.21.15.38:0/2452497894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc97c0cf8f0 0x7fc97c06bbf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.296+0000 7fc98406b640 1 --2- 172.21.15.38:0/2452497894 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c0cf510 0x7fc97c06fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.296+0000 7fc98406b640 1 -- 172.21.15.38:0/2452497894 >> 172.21.15.38:0/2452497894 conn(0x7fc97c071920 msgr2=0x7fc97c071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:54.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.296+0000 7fc98406b640 1 -- 172.21.15.38:0/2452497894 shutdown_connections 2024-04-25T08:31:54.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.296+0000 7fc98406b640 1 -- 172.21.15.38:0/2452497894 wait complete. 2024-04-25T08:31:54.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.297+0000 7fc98406b640 1 Processor -- start 2024-04-25T08:31:54.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.298+0000 7fc98406b640 1 -- start start 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.298+0000 7fc98406b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c0cf510 0x7fc97c147cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.298+0000 7fc98406b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c15c920 0x7fc97c144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.298+0000 7fc98406b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc97c1f4b30 0x7fc97c0b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.298+0000 7fc98406b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c06d9e0 con 0x7fc97c1f4b30 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.298+0000 7fc98406b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c06d860 con 0x7fc97c0cf510 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.298+0000 7fc98406b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97c15e5f0 con 0x7fc97c15c920 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.298+0000 7fc982608640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc97c1f4b30 0x7fc97c0b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.298+0000 7fc982608640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc97c1f4b30 0x7fc97c0b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33358/0 (socket says 172.21.15.38:33358) 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.298+0000 7fc982608640 1 -- 172.21.15.38:0/2291047027 learned_addr learned my addr 172.21.15.38:0/2291047027 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc981606640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c15c920 0x7fc97c144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc981e07640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c0cf510 0x7fc97c147cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc982608640 1 -- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c15c920 msgr2=0x7fc97c144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc982608640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c15c920 0x7fc97c144830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc982608640 1 -- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c0cf510 msgr2=0x7fc97c147cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc982608640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c0cf510 0x7fc97c147cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc982608640 1 -- 172.21.15.38:0/2291047027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c144d70 con 0x7fc97c1f4b30 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc981e07640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c0cf510 0x7fc97c147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc981606640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c15c920 0x7fc97c144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc982608640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc97c1f4b30 0x7fc97c0b3b80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00c700 tx=0x7fc96c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc97a7fc640 1 -- 172.21.15.38:0/2291047027 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c018070 con 0x7fc97c1f4b30 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc97a7fc640 1 -- 172.21.15.38:0/2291047027 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c00ce10 con 0x7fc97c1f4b30 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc98406b640 1 -- 172.21.15.38:0/2291047027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97c1f5310 con 0x7fc97c1f4b30 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc97a7fc640 1 -- 172.21.15.38:0/2291047027 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c00a880 con 0x7fc97c1f4b30 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.299+0000 7fc98406b640 1 -- 172.21.15.38:0/2291047027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97c1f58d0 con 0x7fc97c1f4b30 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.300+0000 7fc98406b640 1 -- 172.21.15.38:0/2291047027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97c0af5f0 con 0x7fc97c1f4b30 2024-04-25T08:31:54.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.301+0000 7fc97a7fc640 1 -- 172.21.15.38:0/2291047027 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc96c01b070 con 0x7fc97c1f4b30 2024-04-25T08:31:54.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.301+0000 7fc97a7fc640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc96806b500 0x7fc96806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.301+0000 7fc97a7fc640 1 -- 172.21.15.38:0/2291047027 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc96c089a50 con 0x7fc97c1f4b30 2024-04-25T08:31:54.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.301+0000 7fc981e07640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc96806b500 0x7fc96806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.302+0000 7fc981e07640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc96806b500 0x7fc96806d9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc974006fd0 tx=0x7fc974008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:54.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.305+0000 7fc97a7fc640 1 -- 172.21.15.38:0/2291047027 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc97c144d70 con 0x7fc97c1f4b30 2024-04-25T08:31:54.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.491+0000 7fc98406b640 1 -- 172.21.15.38:0/2291047027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc97c06fe10 con 0x7fc97c1f4b30 2024-04-25T08:31:54.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.492+0000 7fc97a7fc640 1 -- 172.21.15.38:0/2291047027 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fc96c058590 con 0x7fc97c1f4b30 2024-04-25T08:31:54.489 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:54.489 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:54.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.494+0000 7fc98406b640 1 -- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc96806b500 msgr2=0x7fc96806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.494+0000 7fc98406b640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc96806b500 0x7fc96806d9c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc974006fd0 tx=0x7fc974008040 comp rx=0 tx=0).stop 2024-04-25T08:31:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.494+0000 7fc98406b640 1 -- 172.21.15.38:0/2291047027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc97c1f4b30 msgr2=0x7fc97c0b3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.494+0000 7fc98406b640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc97c1f4b30 0x7fc97c0b3b80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00c700 tx=0x7fc96c00cbc0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.495+0000 7fc98406b640 1 -- 172.21.15.38:0/2291047027 shutdown_connections 2024-04-25T08:31:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.495+0000 7fc98406b640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc97c1f4b30 0x7fc97c0b3b80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.495+0000 7fc98406b640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc96806b500 0x7fc96806d9c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.495+0000 7fc98406b640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc97c15c920 0x7fc97c144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.495+0000 7fc98406b640 1 --2- 172.21.15.38:0/2291047027 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc97c0cf510 0x7fc97c147cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.495+0000 7fc98406b640 1 -- 172.21.15.38:0/2291047027 >> 172.21.15.38:0/2291047027 conn(0x7fc97c071920 msgr2=0x7fc97c1557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.495+0000 7fc98406b640 1 -- 172.21.15.38:0/2291047027 shutdown_connections 2024-04-25T08:31:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.495+0000 7fc98406b640 1 -- 172.21.15.38:0/2291047027 wait complete. 2024-04-25T08:31:54.502 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T08:31:54.503 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:54.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.607+0000 7f974bfab640 1 Processor -- start 2024-04-25T08:31:54.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.607+0000 7f974bfab640 1 -- start start 2024-04-25T08:31:54.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f974bfab640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f974415dc10 0x7f97440ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f974bfab640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f97440ced60 0x7f97440d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f974bfab640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97440d3e70 con 0x7f97440d3900 2024-04-25T08:31:54.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f974bfab640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f974415dff0 con 0x7f974415dc10 2024-04-25T08:31:54.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f974bfab640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97440b6990 con 0x7f97440ced60 2024-04-25T08:31:54.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f9749d47640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f974415dc10 0x7f97440ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f974a548640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f97440d3900 0x7f97440b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49792/0 (socket says 172.21.15.38:49792) 2024-04-25T08:31:54.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f974a548640 1 -- 172.21.15.38:0/791834914 learned_addr learned my addr 172.21.15.38:0/791834914 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:54.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f9749546640 1 --2- 172.21.15.38:0/791834914 >> v2:172.21.15.107:3300/0 conn(0x7f97440ced60 0x7f97440d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f9749d47640 1 -- 172.21.15.38:0/791834914 >> v2:172.21.15.107:3300/0 conn(0x7f97440ced60 msgr2=0x7f97440d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:54.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f9749d47640 1 --2- 172.21.15.38:0/791834914 >> v2:172.21.15.107:3300/0 conn(0x7f97440ced60 0x7f97440d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f9749d47640 1 -- 172.21.15.38:0/791834914 >> v1:172.21.15.38:6789/0 conn(0x7f97440d3900 legacy=0x7f97440b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f9748d45640 1 -- 172.21.15.38:0/791834914 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61111452 0 0) 0x7f97440d3e70 con 0x7f97440d3900 2024-04-25T08:31:54.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f9749d47640 1 -- 172.21.15.38:0/791834914 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97440d4150 con 0x7f974415dc10 2024-04-25T08:31:54.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.608+0000 7f9749546640 1 --2- 172.21.15.38:0/791834914 >> v2:172.21.15.107:3300/0 conn(0x7f97440ced60 0x7f97440d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.609+0000 7f9749d47640 1 --2- 172.21.15.38:0/791834914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974415dc10 0x7f97440ce820 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97440b9600 tx=0x7f974002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb563a1886d308e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.609+0000 7f9748d45640 1 -- 172.21.15.38:0/791834914 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740037070 con 0x7f974415dc10 2024-04-25T08:31:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.609+0000 7f9748d45640 1 -- 172.21.15.38:0/791834914 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9740032440 con 0x7f974415dc10 2024-04-25T08:31:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.609+0000 7f9748d45640 1 -- 172.21.15.38:0/791834914 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974002ae60 con 0x7f974415dc10 2024-04-25T08:31:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.610+0000 7f974bfab640 1 -- 172.21.15.38:0/791834914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974415dc10 msgr2=0x7f97440ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.610+0000 7f974bfab640 1 --2- 172.21.15.38:0/791834914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974415dc10 0x7f97440ce820 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f97440b9600 tx=0x7f974002a110 comp rx=0 tx=0).stop 2024-04-25T08:31:54.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.610+0000 7f974bfab640 1 -- 172.21.15.38:0/791834914 shutdown_connections 2024-04-25T08:31:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.610+0000 7f974bfab640 1 --2- 172.21.15.38:0/791834914 >> v2:172.21.15.107:3300/0 conn(0x7f97440ced60 0x7f97440d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.610+0000 7f974bfab640 1 --2- 172.21.15.38:0/791834914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f974415dc10 0x7f97440ce820 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.610+0000 7f974bfab640 1 -- 172.21.15.38:0/791834914 >> 172.21.15.38:0/791834914 conn(0x7f9744066d10 msgr2=0x7f9744067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.610+0000 7f974bfab640 1 -- 172.21.15.38:0/791834914 shutdown_connections 2024-04-25T08:31:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.610+0000 7f974bfab640 1 -- 172.21.15.38:0/791834914 wait complete. 2024-04-25T08:31:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.611+0000 7f974bfab640 1 Processor -- start 2024-04-25T08:31:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.611+0000 7f974bfab640 1 -- start start 2024-04-25T08:31:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f974bfab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97440ced60 0x7f97440c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f974bfab640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97440d3900 0x7f97440b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f974bfab640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974415dc10 0x7f97440cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f974bfab640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97440b6bc0 con 0x7f97440ced60 2024-04-25T08:31:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f974bfab640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97440d4710 con 0x7f974415dc10 2024-04-25T08:31:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f974bfab640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97440d0d90 con 0x7f97440d3900 2024-04-25T08:31:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f9749d47640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97440ced60 0x7f97440c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f9749d47640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97440ced60 0x7f97440c7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33372/0 (socket says 172.21.15.38:33372) 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f9749d47640 1 -- 172.21.15.38:0/1733096156 learned_addr learned my addr 172.21.15.38:0/1733096156 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f974a548640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974415dc10 0x7f97440cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f9749546640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97440d3900 0x7f97440b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f9749d47640 1 -- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97440d3900 msgr2=0x7f97440b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f9749d47640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97440d3900 0x7f97440b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f9749d47640 1 -- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974415dc10 msgr2=0x7f97440cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f9749d47640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974415dc10 0x7f97440cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f9749d47640 1 -- 172.21.15.38:0/1733096156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97440c7a10 con 0x7f97440ced60 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f974a548640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974415dc10 0x7f97440cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f9749546640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97440d3900 0x7f97440b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f9749d47640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97440ced60 0x7f97440c7430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9740009890 tx=0x7f97400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f973a7fc640 1 -- 172.21.15.38:0/1733096156 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740037040 con 0x7f97440ced60 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f973a7fc640 1 -- 172.21.15.38:0/1733096156 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9740042070 con 0x7f97440ced60 2024-04-25T08:31:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f974bfab640 1 -- 172.21.15.38:0/1733096156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97440c7cf0 con 0x7f97440ced60 2024-04-25T08:31:54.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.612+0000 7f973a7fc640 1 -- 172.21.15.38:0/1733096156 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9740030a50 con 0x7f97440ced60 2024-04-25T08:31:54.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.613+0000 7f974bfab640 1 -- 172.21.15.38:0/1733096156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97440c82b0 con 0x7f97440ced60 2024-04-25T08:31:54.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.613+0000 7f974bfab640 1 -- 172.21.15.38:0/1733096156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9744068e80 con 0x7f97440ced60 2024-04-25T08:31:54.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.614+0000 7f973a7fc640 1 -- 172.21.15.38:0/1733096156 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9740030bf0 con 0x7f97440ced60 2024-04-25T08:31:54.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.614+0000 7f973a7fc640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f972806b4d0 0x7f972806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.614+0000 7f973a7fc640 1 -- 172.21.15.38:0/1733096156 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f97400ab050 con 0x7f97440ced60 2024-04-25T08:31:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.615+0000 7f9749546640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f972806b4d0 0x7f972806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.615+0000 7f9749546640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f972806b4d0 0x7f972806d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f97441f68f0 tx=0x7f9734008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:54.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.619+0000 7f973a7fc640 1 -- 172.21.15.38:0/1733096156 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9740045360 con 0x7f97440ced60 2024-04-25T08:31:54.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.799+0000 7f974bfab640 1 -- 172.21.15.38:0/1733096156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97441e92e0 con 0x7f97440ced60 2024-04-25T08:31:54.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.800+0000 7f973a7fc640 1 -- 172.21.15.38:0/1733096156 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f9740079b90 con 0x7f97440ced60 2024-04-25T08:31:54.796 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:54.796 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:54.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.802+0000 7f974bfab640 1 -- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f972806b4d0 msgr2=0x7f972806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.802+0000 7f974bfab640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f972806b4d0 0x7f972806d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f97441f68f0 tx=0x7f9734008040 comp rx=0 tx=0).stop 2024-04-25T08:31:54.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.802+0000 7f974bfab640 1 -- 172.21.15.38:0/1733096156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97440ced60 msgr2=0x7f97440c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.802+0000 7f974bfab640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97440ced60 0x7f97440c7430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9740009890 tx=0x7f97400046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.802+0000 7f974bfab640 1 -- 172.21.15.38:0/1733096156 shutdown_connections 2024-04-25T08:31:54.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.802+0000 7f974bfab640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f974415dc10 0x7f97440cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.802+0000 7f974bfab640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f972806b4d0 0x7f972806d990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.802+0000 7f974bfab640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97440d3900 0x7f97440b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.802+0000 7f974bfab640 1 --2- 172.21.15.38:0/1733096156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97440ced60 0x7f97440c7430 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.802+0000 7f974bfab640 1 -- 172.21.15.38:0/1733096156 >> 172.21.15.38:0/1733096156 conn(0x7f9744066d10 msgr2=0x7f974415e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:54.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.802+0000 7f974bfab640 1 -- 172.21.15.38:0/1733096156 shutdown_connections 2024-04-25T08:31:54.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.802+0000 7f974bfab640 1 -- 172.21.15.38:0/1733096156 wait complete. 2024-04-25T08:31:54.809 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T08:31:54.810 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T08:31:54.810 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T08:31:54.811 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T08:31:54.811 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T08:31:54.811 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T08:31:54.812 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T08:31:54.812 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T08:31:54.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.991+0000 7f66e3fff640 1 Processor -- start 2024-04-25T08:31:54.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.991+0000 7f66e3fff640 1 -- start start 2024-04-25T08:31:54.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.992+0000 7f7f32792640 1 Processor -- start 2024-04-25T08:31:54.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.992+0000 7f7f32792640 1 -- start start 2024-04-25T08:31:54.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.993+0000 7f7f32792640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f24005e90 0x7f7f240062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.993+0000 7f7f32792640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7f24006830 0x7f7f24006cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.993+0000 7f7f32792640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f24048420 con 0x7f7f24048040 2024-04-25T08:31:54.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.993+0000 7f7f32792640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f240501a0 con 0x7f7f24006830 2024-04-25T08:31:54.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.993+0000 7f7f32792640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240547a0 con 0x7f7f24005e90 2024-04-25T08:31:54.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f7f30f8f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f24005e90 0x7f7f240062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f7f30f8f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f24005e90 0x7f7f240062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:57194/0 (socket says 172.21.15.38:57194) 2024-04-25T08:31:54.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f7f30f8f640 1 -- 172.21.15.38:0/3083300720 learned_addr learned my addr 172.21.15.38:0/3083300720 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:54.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.992+0000 7fce88933640 1 Processor -- start 2024-04-25T08:31:54.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.992+0000 7fce88933640 1 -- start start 2024-04-25T08:31:54.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.993+0000 7fce88933640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fce7c0eac60 0x7fce7c052670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fce88933640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c03a2e0 con 0x7fce7c048ac0 2024-04-25T08:31:54.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fce88933640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c03a5c0 con 0x7fce7c03b0a0 2024-04-25T08:31:54.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fce88933640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c052bb0 con 0x7fce7c0eac60 2024-04-25T08:31:54.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fce8359e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fce7c048ac0 0x7fce7c053790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49804/0 (socket says 172.21.15.38:49804) 2024-04-25T08:31:54.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fce8359e640 1 -- 172.21.15.38:0/717713317 learned_addr learned my addr 172.21.15.38:0/717713317 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:54.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fce82d9d640 1 --2- 172.21.15.38:0/717713317 >> v2:172.21.15.107:3300/0 conn(0x7fce7c0eac60 0x7fce7c052670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.992+0000 7f66e3fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f66e415e200 0x7f66e40b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.992+0000 7f66e3fff640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e40ca240 con 0x7f66e40b1240 2024-04-25T08:31:54.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.992+0000 7f66e3fff640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e40ca520 con 0x7f66e4159b50 2024-04-25T08:31:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.992+0000 7f66e3fff640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e40ac3e0 con 0x7f66e415e200 2024-04-25T08:31:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.993+0000 7f66e27fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f66e415e200 0x7f66e40b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.993+0000 7f66e27fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f66e415e200 0x7f66e40b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33384/0 (socket says 172.21.15.38:33384) 2024-04-25T08:31:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.993+0000 7f66e27fc640 1 -- 172.21.15.38:0/1541218812 learned_addr learned my addr 172.21.15.38:0/1541218812 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.993+0000 7f66e27fc640 1 -- 172.21.15.38:0/1541218812 >> v1:172.21.15.107:6789/0 conn(0x7f66e4159b50 legacy=0x7f66e40c98d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:31:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.993+0000 7f66e27fc640 1 -- 172.21.15.38:0/1541218812 >> v1:172.21.15.38:6789/0 conn(0x7f66e40b1240 legacy=0x7f66e40ac870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:31:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.993+0000 7f66e27fc640 1 -- 172.21.15.38:0/1541218812 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e40ac070 con 0x7f66e415e200 2024-04-25T08:31:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.992+0000 7fcfdbfff640 1 Processor -- start 2024-04-25T08:31:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.992+0000 7fcfdbfff640 1 -- start start 2024-04-25T08:31:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fcfdbfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcfd4050190 0x7fcfd4038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fcfdaffd640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fcfd40df210 0x7fcfd404fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:41950/0 (socket says 172.21.15.38:41950) 2024-04-25T08:31:54.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fcfdaffd640 1 -- 172.21.15.38:0/2921780879 learned_addr learned my addr 172.21.15.38:0/2921780879 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:54.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.992+0000 7f79a466f640 1 Processor -- start 2024-04-25T08:31:54.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.992+0000 7f79a466f640 1 -- start start 2024-04-25T08:31:54.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f79a466f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7994050140 0x7f7994038170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f79a466f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7994005f60 con 0x7f79940df030 2024-04-25T08:31:54.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f79a466f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79940e38e0 con 0x7f79940df410 2024-04-25T08:31:54.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f79a466f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7994006200 con 0x7f7994050140 2024-04-25T08:31:54.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fcfdbfff640 1 -- 172.21.15.38:0/2921780879 --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd40544b0 con 0x7fcfd4053c60 2024-04-25T08:31:54.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fcfdbfff640 1 -- 172.21.15.38:0/2921780879 --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd40541d0 con 0x7fcfd40df210 2024-04-25T08:31:54.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fcfdbfff640 1 -- 172.21.15.38:0/2921780879 --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd4005f20 con 0x7fcfd4050190 2024-04-25T08:31:54.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f79a2c0c640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f79940df410 0x7f7994006a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:41956/0 (socket says 172.21.15.38:41956) 2024-04-25T08:31:54.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f79a2c0c640 1 -- 172.21.15.38:0/1577561889 learned_addr learned my addr 172.21.15.38:0/1577561889 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:54.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f79a1c0a640 1 --2- 172.21.15.38:0/1577561889 >> v2:172.21.15.38:3300/0 conn(0x7f7994050140 0x7f7994038170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046387290 0 0) 0x7fcfd40541d0 con 0x7fcfd40df210 2024-04-25T08:31:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfc4003520 con 0x7fcfd40df210 2024-04-25T08:31:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f7f30f8f640 1 -- 172.21.15.38:0/3083300720 >> v2:172.21.15.38:3300/0 conn(0x7f7f24006830 msgr2=0x7f7f24006cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f7f30f8f640 1 --2- 172.21.15.38:0/3083300720 >> v2:172.21.15.38:3300/0 conn(0x7f7f24006830 0x7f7f24006cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f7f30f8f640 1 -- 172.21.15.38:0/3083300720 >> v1:172.21.15.38:6789/0 conn(0x7f7f24048040 legacy=0x7f7f2404fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.994+0000 7f7f30f8f640 1 -- 172.21.15.38:0/3083300720 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f24053700 con 0x7f7f24005e90 2024-04-25T08:31:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fce82d9d640 1 -- 172.21.15.38:0/717713317 >> v1:172.21.15.107:6789/0 conn(0x7fce7c03b0a0 legacy=0x7fce7c0483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fce82d9d640 1 -- 172.21.15.38:0/717713317 >> v1:172.21.15.38:6789/0 conn(0x7fce7c048ac0 legacy=0x7fce7c053790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fce82d9d640 1 -- 172.21.15.38:0/717713317 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c03a8a0 con 0x7fce7c0eac60 2024-04-25T08:31:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1142033021 0 0) 0x7fcfc4003520 con 0x7fcfd40df210 2024-04-25T08:31:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 >> v2:172.21.15.38:3300/0 conn(0x7fcfd4050190 msgr2=0x7fcfd4038070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 --2- 172.21.15.38:0/2921780879 >> v2:172.21.15.38:3300/0 conn(0x7fcfd4050190 0x7fcfd4038070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 >> v1:172.21.15.38:6789/0 conn(0x7fcfd4053c60 legacy=0x7fcfd4006a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd4006200 con 0x7fcfd40df210 2024-04-25T08:31:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fcfcc001670 con 0x7fcfd40df210 2024-04-25T08:31:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 >> v1:172.21.15.107:6789/0 conn(0x7fcfd40df210 legacy=0x7fcfd404fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 --2- 172.21.15.38:0/2921780879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfc40037a0 0x7fcfc4003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 --2- 172.21.15.38:0/2921780879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfc4004200 0x7fcfc400b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 --2- 172.21.15.38:0/2921780879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfc400b730 0x7fcfc400dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4005f20 con 0x7fcfc40037a0 2024-04-25T08:31:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc400e4d0 con 0x7fcfc4004200 2024-04-25T08:31:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc400e6d0 con 0x7fcfc400b730 2024-04-25T08:31:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7f79a1409640 1 -- 172.21.15.38:0/1577561889 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938098019 0 0) 0x7f79940e38e0 con 0x7f79940df410 2024-04-25T08:31:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7f79a1409640 1 -- 172.21.15.38:0/1577561889 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7988003440 con 0x7f79940df410 2024-04-25T08:31:54.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f7f30f8f640 1 --2- 172.21.15.38:0/3083300720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f24005e90 0x7f7f240062f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c075a90 tx=0x7f7f2c0a5020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15ac04d14a6b68d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:54.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f66e27fc640 1 --2- 172.21.15.38:0/1541218812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66e415e200 0x7f66e40b0d00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f66dc00b1a0 tx=0x7f66dc02a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65072b6b5f4bfdcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:54.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7fce82d9d640 1 --2- 172.21.15.38:0/717713317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7c0eac60 0x7fce7c052670 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce740097a0 tx=0x7fce7402a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f39ff0bcc47019ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:54.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f66e1ffb640 1 -- 172.21.15.38:0/1541218812 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc030420 con 0x7f66e415e200 2024-04-25T08:31:54.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f66e1ffb640 1 -- 172.21.15.38:0/1541218812 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66dc035070 con 0x7f66e415e200 2024-04-25T08:31:54.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f66e1ffb640 1 -- 172.21.15.38:0/1541218812 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc0307e0 con 0x7f66e415e200 2024-04-25T08:31:55.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f79a1c0a640 1 -- 172.21.15.38:0/1577561889 >> v1:172.21.15.107:6789/0 conn(0x7f79940df410 legacy=0x7f7994006a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f79a1c0a640 1 -- 172.21.15.38:0/1577561889 >> v1:172.21.15.38:6789/0 conn(0x7f79940df030 legacy=0x7f799404fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f79a1c0a640 1 -- 172.21.15.38:0/1577561889 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79940064e0 con 0x7f7994050140 2024-04-25T08:31:55.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7f117564d640 1 Processor -- start 2024-04-25T08:31:55.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.995+0000 7f117564d640 1 -- start start 2024-04-25T08:31:55.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f117564d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f11700710d0 0x7f1170162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f117564d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f117006ea60 0x7f11701658e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f117564d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11700b27b0 con 0x7f11700714b0 2024-04-25T08:31:55.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f117564d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11700b2a90 con 0x7f11700710d0 2024-04-25T08:31:55.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f117564d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11700b2d70 con 0x7f117006ea60 2024-04-25T08:31:55.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f116e7fc640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f11700714b0 0x7f117006e350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49840/0 (socket says 172.21.15.38:49840) 2024-04-25T08:31:55.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f116e7fc640 1 -- 172.21.15.38:0/4099956207 learned_addr learned my addr 172.21.15.38:0/4099956207 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:55.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7f7f23fff640 1 -- 172.21.15.38:0/3083300720 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0b2070 con 0x7f7f24005e90 2024-04-25T08:31:55.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f7f23fff640 1 -- 172.21.15.38:0/3083300720 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c0a5e30 con 0x7f7f24005e90 2024-04-25T08:31:55.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f7f23fff640 1 -- 172.21.15.38:0/3083300720 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0ad640 con 0x7f7f24005e90 2024-04-25T08:31:55.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7fcfdb7fe640 1 --2- 172.21.15.38:0/2921780879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfc400b730 0x7fcfc400dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.996+0000 7fcfdaffd640 1 --2- 172.21.15.38:0/2921780879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfc40037a0 0x7fcfc4003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7fcfdb7fe640 1 -- 172.21.15.38:0/2921780879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfc4004200 msgr2=0x7fcfc400b010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:31:55.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7fcfdb7fe640 1 --2- 172.21.15.38:0/2921780879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfc4004200 0x7fcfc400b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7fcfdb7fe640 1 -- 172.21.15.38:0/2921780879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfc40037a0 msgr2=0x7fcfc4003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7fcfdb7fe640 1 --2- 172.21.15.38:0/2921780879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfc40037a0 0x7fcfc4003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7fcfdb7fe640 1 -- 172.21.15.38:0/2921780879 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc4003520 con 0x7fcfc400b730 2024-04-25T08:31:55.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f116effd640 1 --2- 172.21.15.38:0/4099956207 >> v2:172.21.15.38:3300/0 conn(0x7f11700710d0 0x7f1170162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f116f7fe640 1 --2- 172.21.15.38:0/4099956207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f117006ea60 0x7f11701658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7fcfdaffd640 1 --2- 172.21.15.38:0/2921780879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfc40037a0 0x7fcfc4003ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:55.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7fce81d9b640 1 -- 172.21.15.38:0/717713317 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74037070 con 0x7fce7c0eac60 2024-04-25T08:31:55.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7fce81d9b640 1 -- 172.21.15.38:0/717713317 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7402adc0 con 0x7fce7c0eac60 2024-04-25T08:31:55.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f116dffb640 1 -- 172.21.15.38:0/4099956207 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422225616 0 0) 0x7f11700b27b0 con 0x7f11700714b0 2024-04-25T08:31:55.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f116dffb640 1 -- 172.21.15.38:0/4099956207 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1158003520 con 0x7f11700714b0 2024-04-25T08:31:55.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f79a1c0a640 1 --2- 172.21.15.38:0/1577561889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7994050140 0x7f7994038170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f798c0097a0 tx=0x7f798c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9697ff344f8a08b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f116effd640 1 -- 172.21.15.38:0/4099956207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f117006ea60 msgr2=0x7f11701658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f116effd640 1 --2- 172.21.15.38:0/4099956207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f117006ea60 0x7f11701658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f7f32792640 1 -- 172.21.15.38:0/3083300720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f24005e90 msgr2=0x7f7f240062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f7f32792640 1 --2- 172.21.15.38:0/3083300720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f24005e90 0x7f7f240062f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c075a90 tx=0x7f7f2c0a5020 comp rx=0 tx=0).stop 2024-04-25T08:31:55.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f116effd640 1 -- 172.21.15.38:0/4099956207 >> v1:172.21.15.38:6789/0 conn(0x7f11700714b0 legacy=0x7f117006e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.997+0000 7f116effd640 1 -- 172.21.15.38:0/4099956207 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11700b3050 con 0x7f11700710d0 2024-04-25T08:31:55.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f7f32792640 1 -- 172.21.15.38:0/3083300720 shutdown_connections 2024-04-25T08:31:55.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f7f32792640 1 --2- 172.21.15.38:0/3083300720 >> v2:172.21.15.38:3300/0 conn(0x7f7f24006830 0x7f7f24006cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f7f32792640 1 --2- 172.21.15.38:0/3083300720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f24005e90 0x7f7f240062f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f7f32792640 1 -- 172.21.15.38:0/3083300720 >> 172.21.15.38:0/3083300720 conn(0x7f7f24009fb0 msgr2=0x7f7f2400a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7fcfdb7fe640 1 -- 172.21.15.38:0/2921780879 reap_dead start 2024-04-25T08:31:55.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7fcfdb7fe640 1 --2- 172.21.15.38:0/2921780879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfc400b730 0x7fcfc400dbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc050190 tx=0x7fcfc4004820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e76260f6ba11ea3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f7f32792640 1 -- 172.21.15.38:0/3083300720 shutdown_connections 2024-04-25T08:31:55.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7fc82f45d640 1 Processor -- start 2024-04-25T08:31:55.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7fce81d9b640 1 -- 172.21.15.38:0/717713317 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74032660 con 0x7fce7c0eac60 2024-04-25T08:31:55.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc0bb070 con 0x7fcfc400b730 2024-04-25T08:31:55.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfdc0b6410 con 0x7fcfc400b730 2024-04-25T08:31:55.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7fcfd9ffb640 1 -- 172.21.15.38:0/2921780879 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc007c80 con 0x7fcfc400b730 2024-04-25T08:31:55.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f79a1409640 1 -- 172.21.15.38:0/1577561889 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c037070 con 0x7f7994050140 2024-04-25T08:31:55.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f79a1409640 1 -- 172.21.15.38:0/1577561889 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c032440 con 0x7f7994050140 2024-04-25T08:31:55.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f79a1409640 1 -- 172.21.15.38:0/1577561889 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c032810 con 0x7f7994050140 2024-04-25T08:31:55.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7fce88933640 1 -- 172.21.15.38:0/717713317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7c0eac60 msgr2=0x7fce7c052670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f66e3fff640 1 -- 172.21.15.38:0/1541218812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66e415e200 msgr2=0x7f66e40b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f66e3fff640 1 --2- 172.21.15.38:0/1541218812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66e415e200 0x7f66e40b0d00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f66dc00b1a0 tx=0x7f66dc02a970 comp rx=0 tx=0).stop 2024-04-25T08:31:55.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7fce88933640 1 --2- 172.21.15.38:0/717713317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7c0eac60 0x7fce7c052670 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fce740097a0 tx=0x7fce7402a0b0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7fc82f45d640 1 -- start start 2024-04-25T08:31:55.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.998+0000 7f7f32792640 1 -- 172.21.15.38:0/3083300720 wait complete. 2024-04-25T08:31:55.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7f66e3fff640 1 -- 172.21.15.38:0/1541218812 shutdown_connections 2024-04-25T08:31:55.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7f66e3fff640 1 --2- 172.21.15.38:0/1541218812 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66e415e200 0x7f66e40b0d00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7f66e3fff640 1 -- 172.21.15.38:0/1541218812 >> 172.21.15.38:0/1541218812 conn(0x7f66e40cbd00 msgr2=0x7f66e40cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7f66e3fff640 1 -- 172.21.15.38:0/1541218812 shutdown_connections 2024-04-25T08:31:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7f66e3fff640 1 -- 172.21.15.38:0/1541218812 wait complete. 2024-04-25T08:31:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7fce88933640 1 -- 172.21.15.38:0/717713317 shutdown_connections 2024-04-25T08:31:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7fce88933640 1 --2- 172.21.15.38:0/717713317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7c0eac60 0x7fce7c052670 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7fce88933640 1 -- 172.21.15.38:0/717713317 >> 172.21.15.38:0/717713317 conn(0x7fce7c01e6b0 msgr2=0x7fce7c0553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7fcfdbfff640 1 -- 172.21.15.38:0/2921780879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfc400b730 msgr2=0x7fcfc400dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7fcfdbfff640 1 --2- 172.21.15.38:0/2921780879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfc400b730 0x7fcfc400dbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc050190 tx=0x7fcfc4004820 comp rx=0 tx=0).stop 2024-04-25T08:31:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7fcfdbfff640 1 -- 172.21.15.38:0/2921780879 shutdown_connections 2024-04-25T08:31:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7fcfdbfff640 1 --2- 172.21.15.38:0/2921780879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfc400b730 0x7fcfc400dbf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7fcfdbfff640 1 -- 172.21.15.38:0/2921780879 >> 172.21.15.38:0/2921780879 conn(0x7fcfd4009fa0 msgr2=0x7fcfd400a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7fcfdbfff640 1 -- 172.21.15.38:0/2921780879 shutdown_connections 2024-04-25T08:31:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7fcfdbfff640 1 -- 172.21.15.38:0/2921780879 wait complete. 2024-04-25T08:31:55.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7f09b5b5c640 1 Processor -- start 2024-04-25T08:31:55.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7f09b5b5c640 1 -- start start 2024-04-25T08:31:55.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7f7f32792640 1 Processor -- start 2024-04-25T08:31:55.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7fce88933640 1 -- 172.21.15.38:0/717713317 shutdown_connections 2024-04-25T08:31:55.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7f79a466f640 1 -- 172.21.15.38:0/1577561889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7994050140 msgr2=0x7f7994038170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7f79a466f640 1 --2- 172.21.15.38:0/1577561889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7994050140 0x7f7994038170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f798c0097a0 tx=0x7f798c02a270 comp rx=0 tx=0).stop 2024-04-25T08:31:55.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:54.999+0000 7f116effd640 1 --2- 172.21.15.38:0/4099956207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11700710d0 0x7f1170162110 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11600097a0 tx=0x7f116002a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94acbff9602e8eab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7f116dffb640 1 -- 172.21.15.38:0/4099956207 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160037070 con 0x7f11700710d0 2024-04-25T08:31:55.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7f116dffb640 1 -- 172.21.15.38:0/4099956207 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116002adb0 con 0x7f11700710d0 2024-04-25T08:31:55.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7f116dffb640 1 -- 172.21.15.38:0/4099956207 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160032660 con 0x7f11700710d0 2024-04-25T08:31:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7fc82d1f9640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc82806f430 0x7fc82806cfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49854/0 (socket says 172.21.15.38:49854) 2024-04-25T08:31:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7fc82d1f9640 1 -- 172.21.15.38:0/1401661792 learned_addr learned my addr 172.21.15.38:0/1401661792 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7fc82f45d640 1 --2- 172.21.15.38:0/1401661792 >> v2:172.21.15.107:3300/0 conn(0x7fc8280cfa80 0x7fc828166720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7fc82f45d640 1 -- 172.21.15.38:0/1401661792 --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82815bda0 con 0x7fc82806f430 2024-04-25T08:31:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7fc82f45d640 1 -- 172.21.15.38:0/1401661792 --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc82815c020 con 0x7fc8280b4310 2024-04-25T08:31:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7fc82f45d640 1 -- 172.21.15.38:0/1401661792 --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82815c300 con 0x7fc8280cfa80 2024-04-25T08:31:55.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7f09b5b5c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f09b015cf20 0x7f09b00cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302617477 0 0) 0x7fc818001110 con 0x7fc82806f430 2024-04-25T08:31:55.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc810003520 con 0x7fc82806f430 2024-04-25T08:31:55.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7f66e3fff640 1 Processor -- start 2024-04-25T08:31:55.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119937269 0 0) 0x7fc82815c020 con 0x7fc8280b4310 2024-04-25T08:31:55.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc818001110 con 0x7fc8280b4310 2024-04-25T08:31:55.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7f09b5b5c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b015d380 con 0x7f09b00cc350 2024-04-25T08:31:55.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7f09b5b5c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b0072240 con 0x7f09b0145080 2024-04-25T08:31:55.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7f117564d640 1 -- 172.21.15.38:0/4099956207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11700710d0 msgr2=0x7f1170162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7f117564d640 1 --2- 172.21.15.38:0/4099956207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11700710d0 0x7f1170162110 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11600097a0 tx=0x7f116002a060 comp rx=0 tx=0).stop 2024-04-25T08:31:55.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7f09b5b5c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b00cc7c0 con 0x7f09b015cf20 2024-04-25T08:31:55.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7f66e3fff640 1 -- start start 2024-04-25T08:31:55.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.001+0000 7f09affff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f09b00cc350 0x7f09b015f630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49868/0 (socket says 172.21.15.38:49868) 2024-04-25T08:31:55.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f09affff640 1 -- 172.21.15.38:0/486623133 learned_addr learned my addr 172.21.15.38:0/486623133 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:55.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f117564d640 1 -- 172.21.15.38:0/4099956207 shutdown_connections 2024-04-25T08:31:55.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f117564d640 1 --2- 172.21.15.38:0/4099956207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f117006ea60 0x7f11701658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f117564d640 1 --2- 172.21.15.38:0/4099956207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11700710d0 0x7f1170162110 secure :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f11600097a0 tx=0x7f116002a060 comp rx=0 tx=0).stop 2024-04-25T08:31:55.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f117564d640 1 -- 172.21.15.38:0/4099956207 >> 172.21.15.38:0/4099956207 conn(0x7f11700b1380 msgr2=0x7f11700b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.000+0000 7f7f32792640 1 -- start start 2024-04-25T08:31:55.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f32792640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f24005e90 0x7f7f240d8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f32792640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f24006830 0x7f7f240d6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f32792640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f24048040 0x7f7f240d6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f32792640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24053fa0 con 0x7f7f24048040 2024-04-25T08:31:55.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f32792640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24053cc0 con 0x7f7f24006830 2024-04-25T08:31:55.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f32792640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24054280 con 0x7f7f24005e90 2024-04-25T08:31:55.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2302264625 0 0) 0x7fc818001110 con 0x7fc8280b4310 2024-04-25T08:31:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f30f8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f24006830 0x7f7f240d6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 >> v2:172.21.15.107:3300/0 conn(0x7fc8280cfa80 msgr2=0x7fc828166720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:31:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 --2- 172.21.15.38:0/1401661792 >> v2:172.21.15.107:3300/0 conn(0x7fc8280cfa80 0x7fc828166720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 >> v1:172.21.15.38:6789/0 conn(0x7fc82806f430 legacy=0x7fc82806cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945976497 0 0) 0x7f09b015d380 con 0x7f09b00cc350 2024-04-25T08:31:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0998003520 con 0x7f09b00cc350 2024-04-25T08:31:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82815c5e0 con 0x7fc8280b4310 2024-04-25T08:31:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fc8240015a0 con 0x7fc8280b4310 2024-04-25T08:31:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 >> v1:172.21.15.107:6789/0 conn(0x7fc8280b4310 legacy=0x7fc8280cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f30f8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f24006830 0x7f7f240d6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:48370/0 (socket says 172.21.15.38:48370) 2024-04-25T08:31:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f30f8f640 1 -- 172.21.15.38:0/4112292369 learned_addr learned my addr 172.21.15.38:0/4112292369 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f31790640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f24005e90 0x7f7f240d8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 --2- 172.21.15.38:0/1401661792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc810003b40 0x7fc810003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 --2- 172.21.15.38:0/1401661792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc810004800 0x7fc810004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 --2- 172.21.15.38:0/1401661792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8100052e0 0x7fc8100079a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82815c300 con 0x7fc810003b40 2024-04-25T08:31:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810008280 con 0x7fc8100052e0 2024-04-25T08:31:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc810008480 con 0x7fc810004800 2024-04-25T08:31:55.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f31f91640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f24048040 0x7f7f240d6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fcfdbfff640 1 Processor -- start 2024-04-25T08:31:55.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7fcfdbfff640 1 -- start start 2024-04-25T08:31:55.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f30f8f640 1 -- 172.21.15.38:0/4112292369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f24005e90 msgr2=0x7f7f240d8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f30f8f640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f24005e90 0x7f7f240d8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f30f8f640 1 -- 172.21.15.38:0/4112292369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f24048040 msgr2=0x7f7f240d6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f30f8f640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f24048040 0x7f7f240d6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.002+0000 7f7f30f8f640 1 -- 172.21.15.38:0/4112292369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f240d6a70 con 0x7f7f24006830 2024-04-25T08:31:55.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1221614683 0 0) 0x7f0998003520 con 0x7f09b00cc350 2024-04-25T08:31:55.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 >> v2:172.21.15.38:3300/0 conn(0x7f09b015cf20 msgr2=0x7f09b00cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f117564d640 1 -- 172.21.15.38:0/4099956207 shutdown_connections 2024-04-25T08:31:55.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f7f30f8f640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f24006830 0x7f7f240d6530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c050820 tx=0x7f7f2c069b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f7f21ffb640 1 -- 172.21.15.38:0/4112292369 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0bd070 con 0x7f7f24006830 2024-04-25T08:31:55.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f7f21ffb640 1 -- 172.21.15.38:0/4112292369 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f2c0b9660 con 0x7f7f24006830 2024-04-25T08:31:55.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f7f21ffb640 1 -- 172.21.15.38:0/4112292369 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2c0bd070 con 0x7f7f24006830 2024-04-25T08:31:55.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f7f32792640 1 -- 172.21.15.38:0/4112292369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f240d76d0 con 0x7f7f24006830 2024-04-25T08:31:55.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f7f32792640 1 -- 172.21.15.38:0/4112292369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f240d7c90 con 0x7f7f24006830 2024-04-25T08:31:55.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7fce88933640 1 -- 172.21.15.38:0/717713317 wait complete. 2024-04-25T08:31:55.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f09aeffd640 1 --2- 172.21.15.38:0/486623133 >> v2:172.21.15.38:3300/0 conn(0x7f09b015cf20 0x7f09b00cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f09ae7fc640 1 --2- 172.21.15.38:0/486623133 >> v2:172.21.15.38:3300/0 conn(0x7f09b015cf20 0x7f09b00cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 >> v1:172.21.15.107:6789/0 conn(0x7f09b0145080 legacy=0x7f09b0071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b015c130 con 0x7f09b00cc350 2024-04-25T08:31:55.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f09a80019f0 con 0x7f09b00cc350 2024-04-25T08:31:55.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f09aeffd640 1 --2- 172.21.15.38:0/486623133 >> v2:172.21.15.38:3300/0 conn(0x7f09b015cf20 0x7f09b00cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:55.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 >> v1:172.21.15.38:6789/0 conn(0x7f09b00cc350 legacy=0x7f09b015f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f09ae7fc640 1 --2- 172.21.15.38:0/486623133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09980039b0 0x7f0998003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f09ae7fc640 1 --2- 172.21.15.38:0/486623133 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0998004800 0x7f0998004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f09af7fe640 1 --2- 172.21.15.38:0/486623133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09980039b0 0x7f0998003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f09ae7fc640 1 --2- 172.21.15.38:0/486623133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09980052e0 0x7f099800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f09aeffd640 1 --2- 172.21.15.38:0/486623133 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0998004800 0x7f0998004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00cc7c0 con 0x7f09980052e0 2024-04-25T08:31:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099800c1b0 con 0x7f09980039b0 2024-04-25T08:31:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099800c3b0 con 0x7f0998004800 2024-04-25T08:31:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e3fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f66e40b1240 0x7f66e4072b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e4159b50 0x7f66e40c1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66e415e200 0x7f66e40b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e3fff640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40cc7f0 con 0x7f66e415e200 2024-04-25T08:31:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e3fff640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40c9fe0 con 0x7f66e4159b50 2024-04-25T08:31:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e3fff640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40cc970 con 0x7f66e40b1240 2024-04-25T08:31:55.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e27fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e4159b50 0x7f66e40c1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e27fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e4159b50 0x7f66e40c1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:48374/0 (socket says 172.21.15.38:48374) 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e27fc640 1 -- 172.21.15.38:0/984091092 learned_addr learned my addr 172.21.15.38:0/984091092 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e37fe640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66e415e200 0x7f66e40b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e27fc640 1 -- 172.21.15.38:0/984091092 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f66e40b1240 msgr2=0x7f66e4072b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e27fc640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f66e40b1240 0x7f66e4072b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e27fc640 1 -- 172.21.15.38:0/984091092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66e415e200 msgr2=0x7f66e40b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e27fc640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66e415e200 0x7f66e40b2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f66e27fc640 1 -- 172.21.15.38:0/984091092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e4148510 con 0x7f66e4159b50 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f66e27fc640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e4159b50 0x7f66e40c1d60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f66dc02ae90 tx=0x7f66dc004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfc400b730 0x7fcfd40d2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfd40d2a90 0x7fcfd40c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfd40c46b0 0x7fcfd41781b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7fcfdbfff640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4037a80 con 0x7fcfd40d2a90 2024-04-25T08:31:55.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7fcfdbfff640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4037900 con 0x7fcfc400b730 2024-04-25T08:31:55.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7fcfdbfff640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd40df5f0 con 0x7fcfd40c46b0 2024-04-25T08:31:55.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7fcfda7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfd40d2a90 0x7fcfd40c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7fcfda7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfd40d2a90 0x7fcfd40c4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33508/0 (socket says 172.21.15.38:33508) 2024-04-25T08:31:55.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7fcfda7fc640 1 -- 172.21.15.38:0/4137020864 learned_addr learned my addr 172.21.15.38:0/4137020864 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f79a466f640 1 -- 172.21.15.38:0/1577561889 shutdown_connections 2024-04-25T08:31:55.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f79a466f640 1 --2- 172.21.15.38:0/1577561889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7994050140 0x7f7994038170 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7f79a466f640 1 -- 172.21.15.38:0/1577561889 >> 172.21.15.38:0/1577561889 conn(0x7f7994009fe0 msgr2=0x7f799400a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7f117564d640 1 -- 172.21.15.38:0/4099956207 wait complete. 2024-04-25T08:31:55.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7fc82c9f8640 1 --2- 172.21.15.38:0/1401661792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc810004800 0x7fc810004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7fc82d1f9640 1 --2- 172.21.15.38:0/1401661792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc810003b40 0x7fc810003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7fc82c9f8640 1 -- 172.21.15.38:0/1401661792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8100052e0 msgr2=0x7fc8100079a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:31:55.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7fc82c9f8640 1 --2- 172.21.15.38:0/1401661792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8100052e0 0x7fc8100079a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7fc82c9f8640 1 -- 172.21.15.38:0/1401661792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc810003b40 msgr2=0x7fc810003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7fc82c9f8640 1 --2- 172.21.15.38:0/1401661792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc810003b40 0x7fc810003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.003+0000 7fc82c9f8640 1 -- 172.21.15.38:0/1401661792 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82815c5e0 con 0x7fc810004800 2024-04-25T08:31:55.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7fc82d1f9640 1 --2- 172.21.15.38:0/1401661792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc810003b40 0x7fc810003f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:55.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.004+0000 7fc82d9fa640 1 -- 172.21.15.38:0/1401661792 reap_dead start 2024-04-25T08:31:55.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7f09affff640 1 --2- 172.21.15.38:0/486623133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09980052e0 0x7f099800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7f09af7fe640 1 -- 172.21.15.38:0/486623133 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0998004800 msgr2=0x7f0998004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7f09af7fe640 1 --2- 172.21.15.38:0/486623133 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0998004800 0x7f0998004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7f09af7fe640 1 -- 172.21.15.38:0/486623133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09980052e0 msgr2=0x7f099800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7fcfda7fc640 1 -- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfd40c46b0 msgr2=0x7fcfd41781b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7fcfda7fc640 1 --2- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcfd40c46b0 0x7fcfd41781b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7fcfda7fc640 1 -- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfc400b730 msgr2=0x7fcfd40d2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7fcfda7fc640 1 --2- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcfc400b730 0x7fcfd40d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7fcfda7fc640 1 -- 172.21.15.38:0/4137020864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd404eb00 con 0x7fcfd40d2a90 2024-04-25T08:31:55.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7fce88933640 1 Processor -- start 2024-04-25T08:31:55.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7f09af7fe640 1 --2- 172.21.15.38:0/486623133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09980052e0 0x7f099800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7f09af7fe640 1 -- 172.21.15.38:0/486623133 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0998003520 con 0x7f09980039b0 2024-04-25T08:31:55.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7fcfda7fc640 1 --2- 172.21.15.38:0/4137020864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfd40d2a90 0x7fcfd40c4170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000b4a0 tx=0x7fcfd000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7fc82c9f8640 1 --2- 172.21.15.38:0/1401661792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc810004800 0x7fc810004c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc824007f90 tx=0x7fc810005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebac776de0fb93ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7f09affff640 1 --2- 172.21.15.38:0/486623133 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09980052e0 0x7f099800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:55.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7f09affff640 1 -- 172.21.15.38:0/486623133 reap_dead start 2024-04-25T08:31:55.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824007b90 con 0x7fc810004800 2024-04-25T08:31:55.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc824031600 con 0x7fc810004800 2024-04-25T08:31:55.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7fc81ffff640 1 -- 172.21.15.38:0/1401661792 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8240319f0 con 0x7fc810004800 2024-04-25T08:31:55.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7f7f32792640 1 -- 172.21.15.38:0/4112292369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f24005390 con 0x7f7f24006830 2024-04-25T08:31:55.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7fcfc37fe640 1 -- 172.21.15.38:0/4137020864 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0014070 con 0x7fcfd40d2a90 2024-04-25T08:31:55.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7fcfc37fe640 1 -- 172.21.15.38:0/4137020864 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd0010610 con 0x7fcfd40d2a90 2024-04-25T08:31:55.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7fcfc37fe640 1 -- 172.21.15.38:0/4137020864 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00109c0 con 0x7fcfd40d2a90 2024-04-25T08:31:55.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7fcfdbfff640 1 -- 172.21.15.38:0/4137020864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd40d49b0 con 0x7fcfd40d2a90 2024-04-25T08:31:55.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7fcfdbfff640 1 -- 172.21.15.38:0/4137020864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd40d4f70 con 0x7fcfd40d2a90 2024-04-25T08:31:55.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7f117564d640 1 Processor -- start 2024-04-25T08:31:55.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7f117564d640 1 -- start start 2024-04-25T08:31:55.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7f117564d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f117006ea60 0x7f1170142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7f117564d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11700714b0 0x7f117006d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7f117564d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f117006dcd0 0x7f1170163be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7f117564d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11700aefb0 con 0x7f117006ea60 2024-04-25T08:31:55.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7f117564d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1170071910 con 0x7f11700714b0 2024-04-25T08:31:55.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7f117564d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11700af130 con 0x7f117006dcd0 2024-04-25T08:31:55.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7f116e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11700714b0 0x7f117006d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7f116f7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f117006dcd0 0x7f1170163be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7f116f7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f117006dcd0 0x7f1170163be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:57254/0 (socket says 172.21.15.38:57254) 2024-04-25T08:31:55.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.006+0000 7f116f7fe640 1 -- 172.21.15.38:0/3240435527 learned_addr learned my addr 172.21.15.38:0/3240435527 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fc82f45d640 1 -- 172.21.15.38:0/1401661792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc810004800 msgr2=0x7fc810004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fc82f45d640 1 --2- 172.21.15.38:0/1401661792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc810004800 0x7fc810004c80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc824007f90 tx=0x7fc810005830 comp rx=0 tx=0).stop 2024-04-25T08:31:55.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.005+0000 7fce88933640 1 -- start start 2024-04-25T08:31:55.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce88933640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce7c03b0a0 0x7fce7c048ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce88933640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7c048ac0 0x7fce7c04eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce88933640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce7c0eac60 0x7fce7c0d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f7f21ffb640 1 -- 172.21.15.38:0/4112292369 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f2c0c0070 con 0x7f7f24006830 2024-04-25T08:31:55.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce88933640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c03a090 con 0x7fce7c0eac60 2024-04-25T08:31:55.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce88933640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0eb2d0 con 0x7fce7c048ac0 2024-04-25T08:31:55.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce88933640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c03ac10 con 0x7fce7c03b0a0 2024-04-25T08:31:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fc82f45d640 1 -- 172.21.15.38:0/1401661792 shutdown_connections 2024-04-25T08:31:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce8359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce7c0eac60 0x7fce7c0d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce8359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce7c0eac60 0x7fce7c0d6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33512/0 (socket says 172.21.15.38:33512) 2024-04-25T08:31:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce8359e640 1 -- 172.21.15.38:0/3868415171 learned_addr learned my addr 172.21.15.38:0/3868415171 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f116f7fe640 1 -- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11700714b0 msgr2=0x7f117006d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f116f7fe640 1 --2- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11700714b0 0x7f117006d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f116f7fe640 1 -- 172.21.15.38:0/3240435527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f117006ea60 msgr2=0x7f1170142cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f116f7fe640 1 --2- 172.21.15.38:0/3240435527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f117006ea60 0x7f1170142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f116f7fe640 1 -- 172.21.15.38:0/3240435527 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11700aeb70 con 0x7f117006dcd0 2024-04-25T08:31:55.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fc82f45d640 1 --2- 172.21.15.38:0/1401661792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc810004800 0x7fc810004c80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fc82f45d640 1 -- 172.21.15.38:0/1401661792 >> 172.21.15.38:0/1401661792 conn(0x7fc8280b1130 msgr2=0x7fc8280b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f09af7fe640 1 --2- 172.21.15.38:0/486623133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09980039b0 0x7f0998003d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09a0001860 tx=0x7f0998005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=339e79d50427eed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f7f21ffb640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f1006b900 0x7f7f1006ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce8359e640 1 -- 172.21.15.38:0/3868415171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce7c03b0a0 msgr2=0x7fce7c048ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T08:31:55.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce8359e640 1 --2- 172.21.15.38:0/3868415171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce7c03b0a0 0x7fce7c048ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce8359e640 1 -- 172.21.15.38:0/3868415171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7c048ac0 msgr2=0x7fce7c04eb90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T08:31:55.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce8359e640 1 --2- 172.21.15.38:0/3868415171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7c048ac0 0x7fce7c04eb90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7fce8359e640 1 -- 172.21.15.38:0/3868415171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c0ce080 con 0x7fce7c0eac60 2024-04-25T08:31:55.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.008+0000 7f7f21ffb640 1 -- 172.21.15.38:0/4112292369 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7f2c0b9460 con 0x7f7f24006830 2024-04-25T08:31:55.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.008+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0037070 con 0x7f09980039b0 2024-04-25T08:31:55.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.008+0000 7f7f21ffb640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] conn(0x7f7f10075bc0 0x7f7f10078060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.008+0000 7f7f21ffb640 1 -- 172.21.15.38:0/4112292369 --> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f10078730 con 0x7f7f10075bc0 2024-04-25T08:31:55.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.008+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a002abf0 con 0x7f09980039b0 2024-04-25T08:31:55.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.008+0000 7f09ae7fc640 1 -- 172.21.15.38:0/486623133 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0001460 con 0x7f09980039b0 2024-04-25T08:31:55.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.008+0000 7f7f31790640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f1006b900 0x7f7f1006ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.008+0000 7f7f21ffb640 1 -- 172.21.15.38:0/4112292369 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f2c0f02d0 con 0x7f7f24006830 2024-04-25T08:31:55.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.008+0000 7f7f31f91640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] conn(0x7f7f10075bc0 0x7f7f10078060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.008+0000 7fcfdbfff640 1 -- 172.21.15.38:0/4137020864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcfd4005380 con 0x7fcfd40d2a90 2024-04-25T08:31:55.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7fcfc37fe640 1 -- 172.21.15.38:0/4137020864 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfd0017070 con 0x7fcfd40d2a90 2024-04-25T08:31:55.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7f7f31790640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f1006b900 0x7f7f1006ddc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7f280017f0 tx=0x7f7f28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7f09b5b5c640 1 -- 172.21.15.38:0/486623133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09980039b0 msgr2=0x7f0998003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7f09b5b5c640 1 --2- 172.21.15.38:0/486623133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09980039b0 0x7f0998003d90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f09a0001860 tx=0x7f0998005830 comp rx=0 tx=0).stop 2024-04-25T08:31:55.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7fc82f45d640 1 -- 172.21.15.38:0/1401661792 shutdown_connections 2024-04-25T08:31:55.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7f116f7fe640 1 --2- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f117006dcd0 0x7f1170163be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f116400e7f0 tx=0x7f116400ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7f7f31f91640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] conn(0x7f7f10075bc0 0x7f7f10078060 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7f09b5b5c640 1 -- 172.21.15.38:0/486623133 shutdown_connections 2024-04-25T08:31:55.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7f09b5b5c640 1 --2- 172.21.15.38:0/486623133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09980039b0 0x7f0998003d90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7f09b5b5c640 1 -- 172.21.15.38:0/486623133 >> 172.21.15.38:0/486623133 conn(0x7f09b006abe0 msgr2=0x7f09b006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7f09b5b5c640 1 -- 172.21.15.38:0/486623133 shutdown_connections 2024-04-25T08:31:55.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7f09b5b5c640 1 -- 172.21.15.38:0/486623133 wait complete. 2024-04-25T08:31:55.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f66cb7fe640 1 -- 172.21.15.38:0/984091092 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc039b50 con 0x7f66e4159b50 2024-04-25T08:31:55.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f66cb7fe640 1 -- 172.21.15.38:0/984091092 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66dc039cf0 con 0x7f66e4159b50 2024-04-25T08:31:55.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f66cb7fe640 1 -- 172.21.15.38:0/984091092 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc003bd0 con 0x7f66e4159b50 2024-04-25T08:31:55.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f66e3fff640 1 -- 172.21.15.38:0/984091092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e41487f0 con 0x7f66e4159b50 2024-04-25T08:31:55.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.007+0000 7f66e3fff640 1 -- 172.21.15.38:0/984091092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e4147c60 con 0x7f66e4159b50 2024-04-25T08:31:55.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7f66c97fa640 1 -- 172.21.15.38:0/984091092 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f66bc001030 con 0x7f66e4159b50 2024-04-25T08:31:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7f66cb7fe640 1 -- 172.21.15.38:0/984091092 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66dc004030 con 0x7f66e4159b50 2024-04-25T08:31:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7fcfc37fe640 1 --2- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcfc806b550 0x7fcfc806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7fcfc37fe640 1 -- 172.21.15.38:0/4137020864 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcfd0089e20 con 0x7fcfd40d2a90 2024-04-25T08:31:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7fcfc37fe640 1 --2- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x7fcfc8074cd0 0x7fcfc8077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7fcfc37fe640 1 -- 172.21.15.38:0/4137020864 --> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcfd0018ec0 con 0x7fcfc8074cd0 2024-04-25T08:31:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7fcfc37fe640 1 -- 172.21.15.38:0/4137020864 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfd008c520 con 0x7fcfd40d2a90 2024-04-25T08:31:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7fcfdaffd640 1 --2- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcfc806b550 0x7fcfc806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7fcfdb7fe640 1 --2- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x7fcfc8074cd0 0x7fcfc8077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7f79a466f640 1 -- 172.21.15.38:0/1577561889 shutdown_connections 2024-04-25T08:31:55.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7f11577fe640 1 -- 172.21.15.38:0/3240435527 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164017070 con 0x7f117006dcd0 2024-04-25T08:31:55.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7f11577fe640 1 -- 172.21.15.38:0/3240435527 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116400ee40 con 0x7f117006dcd0 2024-04-25T08:31:55.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7f11577fe640 1 -- 172.21.15.38:0/3240435527 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116400cb60 con 0x7f117006dcd0 2024-04-25T08:31:55.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.009+0000 7fc82f45d640 1 -- 172.21.15.38:0/1401661792 wait complete. 2024-04-25T08:31:55.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7f7f21ffb640 1 -- 172.21.15.38:0/4112292369 <== osd.0 v2:172.21.15.38:6800/1562259299 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7f10078730 con 0x7f7f10075bc0 2024-04-25T08:31:55.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7f66cb7fe640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f66d006b520 0x7f66d006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7f66cb7fe640 1 -- 172.21.15.38:0/984091092 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f66dc0aaee0 con 0x7f66e4159b50 2024-04-25T08:31:55.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.011+0000 7fcfdaffd640 1 --2- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcfc806b550 0x7fcfc806da10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40db270 tx=0x7fcfcc001430 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.011+0000 7f66cb7fe640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] conn(0x7f66d0074c90 0x7f66d00770d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.011+0000 7f66cb7fe640 1 -- 172.21.15.38:0/984091092 --> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f66dc031e90 con 0x7f66d0074c90 2024-04-25T08:31:55.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.011+0000 7f66cb7fe640 1 -- 172.21.15.38:0/984091092 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66dc0ad500 con 0x7f66e4159b50 2024-04-25T08:31:55.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.011+0000 7f117564d640 1 -- 172.21.15.38:0/3240435527 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117006d2d0 con 0x7f117006dcd0 2024-04-25T08:31:55.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.011+0000 7f117564d640 1 -- 172.21.15.38:0/3240435527 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117013fcc0 con 0x7f117006dcd0 2024-04-25T08:31:55.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.011+0000 7f66e2ffd640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f66d006b520 0x7f66d006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.011+0000 7f66e37fe640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] conn(0x7f66d0074c90 0x7f66d00770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.012+0000 7f09b5b5c640 1 Processor -- start 2024-04-25T08:31:55.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.012+0000 7fcfdb7fe640 1 --2- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x7fcfc8074cd0 0x7fcfc8077110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.012+0000 7fc82f45d640 1 Processor -- start 2024-04-25T08:31:55.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.012+0000 7f117564d640 1 -- 172.21.15.38:0/3240435527 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f117004ef20 con 0x7f117006dcd0 2024-04-25T08:31:55.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.012+0000 7fc82f45d640 1 -- start start 2024-04-25T08:31:55.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.012+0000 7f09b5b5c640 1 -- start start 2024-04-25T08:31:55.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.012+0000 7f66e37fe640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] conn(0x7f66d0074c90 0x7f66d00770d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.010+0000 7f79a466f640 1 -- 172.21.15.38:0/1577561889 wait complete. 2024-04-25T08:31:55.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.012+0000 7f79a466f640 1 Processor -- start 2024-04-25T08:31:55.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.012+0000 7f79a466f640 1 -- start start 2024-04-25T08:31:55.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.013+0000 7f66e2ffd640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f66d006b520 0x7f66d006d9e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f66d40096f0 tx=0x7f66d4006c40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.011+0000 7fce8359e640 1 --2- 172.21.15.38:0/3868415171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce7c0eac60 0x7fce7c0d6a80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fce840742c0 tx=0x7fce840686a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.011+0000 7fce6b7fe640 1 -- 172.21.15.38:0/3868415171 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce840731c0 con 0x7fce7c0eac60 2024-04-25T08:31:55.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.011+0000 7fce6b7fe640 1 -- 172.21.15.38:0/3868415171 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8408f020 con 0x7fce7c0eac60 2024-04-25T08:31:55.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.012+0000 7fce6b7fe640 1 -- 172.21.15.38:0/3868415171 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8406ade0 con 0x7fce7c0eac60 2024-04-25T08:31:55.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.013+0000 7f79a466f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7994050140 0x7f79940d7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.013+0000 7f79a466f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79940df030 0x7f7994049010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.013+0000 7f79a466f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79940df410 0x7f79940e1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.013+0000 7f79a466f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79940507c0 con 0x7f79940df030 2024-04-25T08:31:55.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.013+0000 7f79a466f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7994050640 con 0x7f79940df410 2024-04-25T08:31:55.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.013+0000 7f79a466f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79940546c0 con 0x7f7994050140 2024-04-25T08:31:55.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.013+0000 7f79a2c0c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79940df410 0x7f79940e1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.013+0000 7fc82f45d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc810004800 0x7fc828141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.013+0000 7f79a2c0c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79940df410 0x7f79940e1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:48402/0 (socket says 172.21.15.38:48402) 2024-04-25T08:31:55.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7fc82f45d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc82813f0c0 0x7fc828155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.013+0000 7f79a2c0c640 1 -- 172.21.15.38:0/3693419528 learned_addr learned my addr 172.21.15.38:0/3693419528 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.013+0000 7f11577fe640 1 -- 172.21.15.38:0/3240435527 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f116401a070 con 0x7f117006dcd0 2024-04-25T08:31:55.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f09b5b5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09980039b0 0x7f09b00adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f11577fe640 1 --2- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f115c06b5a0 0x7f115c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f11577fe640 1 -- 172.21.15.38:0/3240435527 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f116408a340 con 0x7f117006dcd0 2024-04-25T08:31:55.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f79a240b640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7994050140 0x7f79940d7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f66cb7fe640 1 -- 172.21.15.38:0/984091092 <== osd.3 v2:172.21.15.38:6816/831384614 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f66dc031e90 con 0x7f66d0074c90 2024-04-25T08:31:55.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f09b5b5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09b01f7830 0x7f09b0142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f09af7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09980039b0 0x7f09b00adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f09af7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09980039b0 0x7f09b00adba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:48422/0 (socket says 172.21.15.38:48422) 2024-04-25T08:31:55.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7fcfc37fe640 1 -- 172.21.15.38:0/4137020864 <== osd.4 v2:172.21.15.107:6800/592699329 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcfd0018ec0 con 0x7fcfc8074cd0 2024-04-25T08:31:55.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f09af7fe640 1 -- 172.21.15.38:0/4025353335 learned_addr learned my addr 172.21.15.38:0/4025353335 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f09b5b5c640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09b0143030 0x7f09b0158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f09aeffd640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09b01f7830 0x7f09b0142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f79a1c0a640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79940df030 0x7f7994049010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b015c6f0 con 0x7f09b0143030 2024-04-25T08:31:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f11577fe640 1 --2- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] conn(0x7f115c074d10 0x7f115c077150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f116effd640 1 --2- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f115c06b5a0 0x7f115c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f11577fe640 1 -- 172.21.15.38:0/3240435527 --> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f116401ae50 con 0x7f115c074d10 2024-04-25T08:31:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7fc82d1f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc810004800 0x7fc828141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7fc82c9f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc82813f0c0 0x7fc828155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7fc82d1f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc810004800 0x7fc828141a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:48410/0 (socket says 172.21.15.38:48410) 2024-04-25T08:31:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f11577fe640 1 -- 172.21.15.38:0/3240435527 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1164091050 con 0x7f117006dcd0 2024-04-25T08:31:55.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7f116e7fc640 1 --2- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] conn(0x7f115c074d10 0x7f115c077150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7fc82c9f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc82813f0c0 0x7fc828155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33526/0 (socket says 172.21.15.38:33526) 2024-04-25T08:31:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7fc82d1f9640 1 -- 172.21.15.38:0/851637094 learned_addr learned my addr 172.21.15.38:0/851637094 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7fce88933640 1 -- 172.21.15.38:0/3868415171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c0d0ab0 con 0x7fce7c0eac60 2024-04-25T08:31:55.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.014+0000 7fce88933640 1 -- 172.21.15.38:0/3868415171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c04f0d0 con 0x7fce7c0eac60 2024-04-25T08:31:55.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7fc82f45d640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc828140040 0x7fc8280ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b015c9d0 con 0x7f09980039b0 2024-04-25T08:31:55.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b015ccb0 con 0x7f09b01f7830 2024-04-25T08:31:55.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82806f810 con 0x7fc82813f0c0 2024-04-25T08:31:55.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8280b2430 con 0x7fc810004800 2024-04-25T08:31:55.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82806f990 con 0x7fc828140040 2024-04-25T08:31:55.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7fc82d9fa640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc828140040 0x7fc8280ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f09affff640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09b0143030 0x7f09b0158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f09af7fe640 1 -- 172.21.15.38:0/4025353335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09b01f7830 msgr2=0x7f09b0142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f09af7fe640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09b01f7830 0x7f09b0142af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f09af7fe640 1 -- 172.21.15.38:0/4025353335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09b0143030 msgr2=0x7f09b0158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f09af7fe640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09b0143030 0x7f09b0158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f09af7fe640 1 -- 172.21.15.38:0/4025353335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b0141430 con 0x7f09980039b0 2024-04-25T08:31:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7f09affff640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09b0143030 0x7f09b0158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7f09af7fe640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09980039b0 0x7f09b00adba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f09a002a6a0 tx=0x7f09a0001230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f116effd640 1 --2- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f115c06b5a0 0x7f115c06da60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1160034000 tx=0x7f116002c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7f116e7fc640 1 --2- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] conn(0x7f115c074d10 0x7f115c077150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7f0997fff640 1 -- 172.21.15.38:0/4025353335 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0037050 con 0x7f09980039b0 2024-04-25T08:31:55.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7fc82c9f8640 1 -- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc828140040 msgr2=0x7fc8280ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7f0997fff640 1 -- 172.21.15.38:0/4025353335 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a0043070 con 0x7f09980039b0 2024-04-25T08:31:55.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7f0997fff640 1 -- 172.21.15.38:0/4025353335 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a0007a70 con 0x7f09980039b0 2024-04-25T08:31:55.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7fc82c9f8640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc828140040 0x7fc8280ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7fc82c9f8640 1 -- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc810004800 msgr2=0x7fc828141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7fc82c9f8640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc810004800 0x7fc828141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7fc82c9f8640 1 -- 172.21.15.38:0/851637094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc828141f70 con 0x7fc82813f0c0 2024-04-25T08:31:55.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7fc82c9f8640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc82813f0c0 0x7fc828155760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc824031000 tx=0x7fc8240018a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b0143620 con 0x7f09980039b0 2024-04-25T08:31:55.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b01f7030 con 0x7f09980039b0 2024-04-25T08:31:55.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.017+0000 7f11577fe640 1 -- 172.21.15.38:0/3240435527 <== osd.6 v2:172.21.15.107:6808/1351784056 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f116401ae50 con 0x7f115c074d10 2024-04-25T08:31:55.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f79a2c0c640 1 -- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7994050140 msgr2=0x7f79940d7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f79a2c0c640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7994050140 0x7f79940d7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f79a2c0c640 1 -- 172.21.15.38:0/3693419528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79940df030 msgr2=0x7f7994049010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f79a2c0c640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79940df030 0x7f7994049010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.015+0000 7f79a2c0c640 1 -- 172.21.15.38:0/3693419528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79940497a0 con 0x7f79940df410 2024-04-25T08:31:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.016+0000 7f79a2c0c640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79940df410 0x7f79940e1810 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f799c074500 tx=0x7f799c0686a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.017+0000 7fce6b7fe640 1 -- 172.21.15.38:0/3868415171 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce84073a70 con 0x7fce7c0eac60 2024-04-25T08:31:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.017+0000 7f7992ffd640 1 -- 172.21.15.38:0/3693419528 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c04f490 con 0x7f79940df410 2024-04-25T08:31:55.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.018+0000 7fce6b7fe640 1 --2- 172.21.15.38:0/3868415171 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce7006b5d0 0x7fce7006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.018+0000 7f7992ffd640 1 -- 172.21.15.38:0/3693419528 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c08f020 con 0x7f79940df410 2024-04-25T08:31:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.018+0000 7f79a466f640 1 -- 172.21.15.38:0/3693419528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7994049a80 con 0x7f79940df410 2024-04-25T08:31:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.018+0000 7f79a466f640 1 -- 172.21.15.38:0/3693419528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799404a040 con 0x7f79940df410 2024-04-25T08:31:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.018+0000 7f79a240b640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7994050140 0x7f79940d7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:31:55.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.018+0000 7fce6b7fe640 1 -- 172.21.15.38:0/3868415171 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fce84104630 con 0x7fce7c0eac60 2024-04-25T08:31:55.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.018+0000 7f79a1c0a640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79940df030 0x7f7994049010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:55.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.019+0000 7fce82d9d640 1 --2- 172.21.15.38:0/3868415171 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce7006b5d0 0x7fce7006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.019+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09b004ef90 con 0x7f09980039b0 2024-04-25T08:31:55.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.019+0000 7f7992ffd640 1 -- 172.21.15.38:0/3693419528 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c06abe0 con 0x7f79940df410 2024-04-25T08:31:55.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.019+0000 7fce88933640 1 --2- 172.21.15.38:0/3868415171 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x7fce7c00bdb0 0x7fce7c051e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.019+0000 7fce8259c640 1 --2- 172.21.15.38:0/3868415171 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x7fce7c00bdb0 0x7fce7c051e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.019+0000 7fce88933640 1 -- 172.21.15.38:0/3868415171 --> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fce7c0057d0 con 0x7fce7c00bdb0 2024-04-25T08:31:55.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.020+0000 7f79a466f640 1 -- 172.21.15.38:0/3693419528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7984001030 con 0x7f79940df410 2024-04-25T08:31:55.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.020+0000 7fc81dffb640 1 -- 172.21.15.38:0/851637094 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82403f070 con 0x7fc82813f0c0 2024-04-25T08:31:55.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.020+0000 7fc81dffb640 1 -- 172.21.15.38:0/851637094 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc824007b70 con 0x7fc82813f0c0 2024-04-25T08:31:55.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.020+0000 7fc81dffb640 1 -- 172.21.15.38:0/851637094 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824005270 con 0x7fc82813f0c0 2024-04-25T08:31:55.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.020+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc828140990 con 0x7fc82813f0c0 2024-04-25T08:31:55.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.020+0000 7fce82d9d640 1 --2- 172.21.15.38:0/3868415171 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce7006b5d0 0x7fce7006da90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce74034000 tx=0x7fce7402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.020+0000 7f0997fff640 1 -- 172.21.15.38:0/4025353335 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09a0032890 con 0x7f09980039b0 2024-04-25T08:31:55.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.020+0000 7f0997fff640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f099c06b4d0 0x7f099c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.021+0000 7f0997fff640 1 -- 172.21.15.38:0/4025353335 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f09a00aae40 con 0x7f09980039b0 2024-04-25T08:31:55.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.020+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc828140f50 con 0x7fc82813f0c0 2024-04-25T08:31:55.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.021+0000 7f0997fff640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] conn(0x7f099c074bd0 0x7f099c077010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.021+0000 7f0997fff640 1 -- 172.21.15.38:0/4025353335 --> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f099c0776e0 con 0x7f099c074bd0 2024-04-25T08:31:55.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.021+0000 7f09affff640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] conn(0x7f099c074bd0 0x7f099c077010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.021+0000 7f0997fff640 1 -- 172.21.15.38:0/4025353335 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09a00755c0 con 0x7f09980039b0 2024-04-25T08:31:55.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.021+0000 7fce8259c640 1 --2- 172.21.15.38:0/3868415171 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x7fce7c00bdb0 0x7fce7c051e30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.022+0000 7f09affff640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] conn(0x7f099c074bd0 0x7f099c077010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.023+0000 7fce6b7fe640 1 -- 172.21.15.38:0/3868415171 <== osd.1 v2:172.21.15.38:6802/3579687234 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fce7c0057d0 con 0x7fce7c00bdb0 2024-04-25T08:31:55.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.021+0000 7f7992ffd640 1 -- 172.21.15.38:0/3693419528 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f799c073a70 con 0x7f79940df410 2024-04-25T08:31:55.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.022+0000 7f7992ffd640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f798006b5d0 0x7f798006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.022+0000 7f7992ffd640 1 -- 172.21.15.38:0/3693419528 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f799c104650 con 0x7f79940df410 2024-04-25T08:31:55.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.022+0000 7f7992ffd640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] conn(0x7f7980074cd0 0x7f7980077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.022+0000 7f79a240b640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f798006b5d0 0x7f798006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.022+0000 7f7992ffd640 1 -- 172.21.15.38:0/3693419528 --> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f799c08cec0 con 0x7f7980074cd0 2024-04-25T08:31:55.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.023+0000 7f79a1c0a640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] conn(0x7f7980074cd0 0x7f7980077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.022+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc82804ef90 con 0x7fc82813f0c0 2024-04-25T08:31:55.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.023+0000 7f0997fff640 1 -- 172.21.15.38:0/4025353335 <== osd.2 v2:172.21.15.38:6824/3010466496 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f099c0776e0 con 0x7f099c074bd0 2024-04-25T08:31:55.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.023+0000 7fc81dffb640 1 -- 172.21.15.38:0/851637094 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc824042070 con 0x7fc82813f0c0 2024-04-25T08:31:55.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.023+0000 7fc81dffb640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc81406b5a0 0x7fc81406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.023+0000 7fc81dffb640 1 -- 172.21.15.38:0/851637094 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc8240aae30 con 0x7fc82813f0c0 2024-04-25T08:31:55.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.024+0000 7fc81dffb640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] conn(0x7fc814074ca0 0x7fc8140770e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.024+0000 7fc81dffb640 1 -- 172.21.15.38:0/851637094 --> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc824007630 con 0x7fc814074ca0 2024-04-25T08:31:55.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.024+0000 7fc81dffb640 1 -- 172.21.15.38:0/851637094 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8240755a0 con 0x7fc82813f0c0 2024-04-25T08:31:55.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.024+0000 7fc82d1f9640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc81406b5a0 0x7fc81406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.025+0000 7f79a1c0a640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] conn(0x7f7980074cd0 0x7f7980077110 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.025+0000 7f79a240b640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f798006b5d0 0x7f798006da90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f799404a530 tx=0x7f7998009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.025+0000 7f7992ffd640 1 -- 172.21.15.38:0/3693419528 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f799c10b050 con 0x7f79940df410 2024-04-25T08:31:55.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.025+0000 7f09aeffd640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f099c06b4d0 0x7f099c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.026+0000 7f7992ffd640 1 -- 172.21.15.38:0/3693419528 <== osd.5 v2:172.21.15.107:6816/4056807981 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f799c08cec0 con 0x7f7980074cd0 2024-04-25T08:31:55.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.026+0000 7fc82d1f9640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc81406b5a0 0x7fc81406da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc818001510 tx=0x7fc8180011b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.026+0000 7fc82d9fa640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] conn(0x7fc814074ca0 0x7fc8140770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.026+0000 7f09aeffd640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f099c06b4d0 0x7f099c06d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f09b00cece0 tx=0x7f09a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.032+0000 7fc82d9fa640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] conn(0x7fc814074ca0 0x7fc8140770e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.033+0000 7fc81dffb640 1 -- 172.21.15.38:0/851637094 <== osd.7 v2:172.21.15.107:6824/3757070988 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc824007630 con 0x7fc814074ca0 2024-04-25T08:31:55.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.045+0000 7fcfdbfff640 1 -- 172.21.15.38:0/4137020864 --> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcfd4052bd0 con 0x7fcfc8074cd0 2024-04-25T08:31:55.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.045+0000 7fcfc37fe640 1 -- 172.21.15.38:0/4137020864 <== osd.4 v2:172.21.15.107:6800/592699329 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcfd4052bd0 con 0x7fcfc8074cd0 2024-04-25T08:31:55.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.046+0000 7fcfc17fa640 1 -- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x7fcfc8074cd0 msgr2=0x7fcfc8077110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.046+0000 7fcfc17fa640 1 --2- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x7fcfc8074cd0 0x7fcfc8077110 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.046+0000 7fcfc17fa640 1 -- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcfc806b550 msgr2=0x7fcfc806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.046+0000 7fcfc17fa640 1 --2- 172.21.15.38:0/4137020864 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcfc806b550 0x7fcfc806da10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40db270 tx=0x7fcfcc001430 comp rx=0 tx=0).stop 2024-04-25T08:31:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.046+0000 7fcfc17fa640 1 -- 172.21.15.38:0/4137020864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfd40d2a90 msgr2=0x7fcfd40c4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.046+0000 7fcfc17fa640 1 --2- 172.21.15.38:0/4137020864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcfd40d2a90 0x7fcfd40c4170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000b4a0 tx=0x7fcfd000b960 comp rx=0 tx=0).stop 2024-04-25T08:31:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.047+0000 7fcfdb7fe640 1 -- 172.21.15.38:0/4137020864 reap_dead start 2024-04-25T08:31:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.047+0000 7fcfc17fa640 1 -- 172.21.15.38:0/4137020864 shutdown_connections 2024-04-25T08:31:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.047+0000 7fcfc17fa640 1 -- 172.21.15.38:0/4137020864 >> 172.21.15.38:0/4137020864 conn(0x7fcfd4009fa0 msgr2=0x7fcfd4006900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.047+0000 7fcfc17fa640 1 -- 172.21.15.38:0/4137020864 shutdown_connections 2024-04-25T08:31:55.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.047+0000 7fcfc17fa640 1 -- 172.21.15.38:0/4137020864 wait complete. 2024-04-25T08:31:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.047+0000 7f7f32792640 1 -- 172.21.15.38:0/4112292369 --> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f240ea9f0 con 0x7f7f10075bc0 2024-04-25T08:31:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.048+0000 7f7f21ffb640 1 -- 172.21.15.38:0/4112292369 <== osd.0 v2:172.21.15.38:6800/1562259299 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7f240ea9f0 con 0x7f7f10075bc0 2024-04-25T08:31:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.049+0000 7f7f32792640 1 -- 172.21.15.38:0/4112292369 >> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] conn(0x7f7f10075bc0 msgr2=0x7f7f10078060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.049+0000 7f7f32792640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] conn(0x7f7f10075bc0 0x7f7f10078060 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.049+0000 7f7f32792640 1 -- 172.21.15.38:0/4112292369 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f1006b900 msgr2=0x7f7f1006ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.049+0000 7f7f32792640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f1006b900 0x7f7f1006ddc0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7f280017f0 tx=0x7f7f28009040 comp rx=0 tx=0).stop 2024-04-25T08:31:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.049+0000 7f7f32792640 1 -- 172.21.15.38:0/4112292369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f24006830 msgr2=0x7f7f240d6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.049+0000 7f7f32792640 1 --2- 172.21.15.38:0/4112292369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f24006830 0x7f7f240d6530 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c050820 tx=0x7f7f2c069b80 comp rx=0 tx=0).stop 2024-04-25T08:31:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.049+0000 7f7f31f91640 1 -- 172.21.15.38:0/4112292369 reap_dead start 2024-04-25T08:31:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.050+0000 7f7f32792640 1 -- 172.21.15.38:0/4112292369 shutdown_connections 2024-04-25T08:31:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.050+0000 7f7f32792640 1 -- 172.21.15.38:0/4112292369 >> 172.21.15.38:0/4112292369 conn(0x7f7f24009fb0 msgr2=0x7f7f24037d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.050+0000 7f66c97fa640 1 -- 172.21.15.38:0/984091092 --> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f66bc002dd0 con 0x7f66d0074c90 2024-04-25T08:31:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.050+0000 7f66cb7fe640 1 -- 172.21.15.38:0/984091092 <== osd.3 v2:172.21.15.38:6816/831384614 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f66bc002dd0 con 0x7f66d0074c90 2024-04-25T08:31:55.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.051+0000 7f7f32792640 1 -- 172.21.15.38:0/4112292369 shutdown_connections 2024-04-25T08:31:55.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.051+0000 7f7f32792640 1 -- 172.21.15.38:0/4112292369 wait complete. 2024-04-25T08:31:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.051+0000 7f66c97fa640 1 -- 172.21.15.38:0/984091092 >> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] conn(0x7f66d0074c90 msgr2=0x7f66d00770d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.051+0000 7f66c97fa640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] conn(0x7f66d0074c90 0x7f66d00770d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.051+0000 7f66c97fa640 1 -- 172.21.15.38:0/984091092 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f66d006b520 msgr2=0x7f66d006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.052+0000 7f66c97fa640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f66d006b520 0x7f66d006d9e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f66d40096f0 tx=0x7f66d4006c40 comp rx=0 tx=0).stop 2024-04-25T08:31:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.052+0000 7f66c97fa640 1 -- 172.21.15.38:0/984091092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e4159b50 msgr2=0x7f66e40c1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.052+0000 7f66c97fa640 1 --2- 172.21.15.38:0/984091092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66e4159b50 0x7f66e40c1d60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f66dc02ae90 tx=0x7f66dc004a40 comp rx=0 tx=0).stop 2024-04-25T08:31:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.052+0000 7f66e37fe640 1 -- 172.21.15.38:0/984091092 reap_dead start 2024-04-25T08:31:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.052+0000 7f66c97fa640 1 -- 172.21.15.38:0/984091092 shutdown_connections 2024-04-25T08:31:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.052+0000 7f66c97fa640 1 -- 172.21.15.38:0/984091092 >> 172.21.15.38:0/984091092 conn(0x7f66e40cbd00 msgr2=0x7f66e414d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.052+0000 7f66c97fa640 1 -- 172.21.15.38:0/984091092 shutdown_connections 2024-04-25T08:31:55.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.052+0000 7f66c97fa640 1 -- 172.21.15.38:0/984091092 wait complete. 2024-04-25T08:31:55.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.053+0000 7f117564d640 1 -- 172.21.15.38:0/3240435527 --> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f11700b0cb0 con 0x7f115c074d10 2024-04-25T08:31:55.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.054+0000 7f11577fe640 1 -- 172.21.15.38:0/3240435527 <== osd.6 v2:172.21.15.107:6808/1351784056 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f11700b0cb0 con 0x7f115c074d10 2024-04-25T08:31:55.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.055+0000 7f11557fa640 1 -- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] conn(0x7f115c074d10 msgr2=0x7f115c077150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.055+0000 7f11557fa640 1 --2- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] conn(0x7f115c074d10 0x7f115c077150 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.055+0000 7f11557fa640 1 -- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f115c06b5a0 msgr2=0x7f115c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.055+0000 7f11557fa640 1 --2- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f115c06b5a0 0x7f115c06da60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1160034000 tx=0x7f116002c000 comp rx=0 tx=0).stop 2024-04-25T08:31:55.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.056+0000 7f11557fa640 1 -- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f117006dcd0 msgr2=0x7f1170163be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.056+0000 7f11557fa640 1 --2- 172.21.15.38:0/3240435527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f117006dcd0 0x7f1170163be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f116400e7f0 tx=0x7f116400ebf0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.056+0000 7f116f7fe640 1 -- 172.21.15.38:0/3240435527 reap_dead start 2024-04-25T08:31:55.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.056+0000 7f11557fa640 1 -- 172.21.15.38:0/3240435527 shutdown_connections 2024-04-25T08:31:55.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.056+0000 7f11557fa640 1 -- 172.21.15.38:0/3240435527 >> 172.21.15.38:0/3240435527 conn(0x7f11700b1380 msgr2=0x7f11700af660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.056+0000 7f11557fa640 1 -- 172.21.15.38:0/3240435527 shutdown_connections 2024-04-25T08:31:55.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.056+0000 7f11557fa640 1 -- 172.21.15.38:0/3240435527 wait complete. 2024-04-25T08:31:55.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.056+0000 7fce88933640 1 -- 172.21.15.38:0/3868415171 --> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fce7c00b570 con 0x7fce7c00bdb0 2024-04-25T08:31:55.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.057+0000 7fce6b7fe640 1 -- 172.21.15.38:0/3868415171 <== osd.1 v2:172.21.15.38:6802/3579687234 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fce7c00b570 con 0x7fce7c00bdb0 2024-04-25T08:31:55.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.057+0000 7fce88933640 1 -- 172.21.15.38:0/3868415171 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x7fce7c00bdb0 msgr2=0x7fce7c051e30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.058+0000 7fce88933640 1 --2- 172.21.15.38:0/3868415171 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x7fce7c00bdb0 0x7fce7c051e30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.058+0000 7fce88933640 1 -- 172.21.15.38:0/3868415171 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce7006b5d0 msgr2=0x7fce7006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.058+0000 7fce88933640 1 --2- 172.21.15.38:0/3868415171 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce7006b5d0 0x7fce7006da90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fce74034000 tx=0x7fce7402e000 comp rx=0 tx=0).stop 2024-04-25T08:31:55.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.058+0000 7fce88933640 1 -- 172.21.15.38:0/3868415171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce7c0eac60 msgr2=0x7fce7c0d6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.058+0000 7fce88933640 1 --2- 172.21.15.38:0/3868415171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce7c0eac60 0x7fce7c0d6a80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fce840742c0 tx=0x7fce840686a0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.058+0000 7fce8359e640 1 -- 172.21.15.38:0/3868415171 reap_dead start 2024-04-25T08:31:55.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.059+0000 7fce88933640 1 -- 172.21.15.38:0/3868415171 shutdown_connections 2024-04-25T08:31:55.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.059+0000 7fce88933640 1 -- 172.21.15.38:0/3868415171 >> 172.21.15.38:0/3868415171 conn(0x7fce7c01e6b0 msgr2=0x7fce7c037780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.059+0000 7f79a466f640 1 -- 172.21.15.38:0/3693419528 --> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7984002dd0 con 0x7f7980074cd0 2024-04-25T08:31:55.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.059+0000 7fce88933640 1 -- 172.21.15.38:0/3868415171 shutdown_connections 2024-04-25T08:31:55.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.059+0000 7fce88933640 1 -- 172.21.15.38:0/3868415171 wait complete. 2024-04-25T08:31:55.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.060+0000 7f7992ffd640 1 -- 172.21.15.38:0/3693419528 <== osd.5 v2:172.21.15.107:6816/4056807981 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7984002dd0 con 0x7f7980074cd0 2024-04-25T08:31:55.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.060+0000 7f79a466f640 1 -- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] conn(0x7f7980074cd0 msgr2=0x7f7980077110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.060+0000 7f79a466f640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] conn(0x7f7980074cd0 0x7f7980077110 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.061+0000 7f79a466f640 1 -- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f798006b5d0 msgr2=0x7f798006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.061+0000 7f79a466f640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f798006b5d0 0x7f798006da90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f799404a530 tx=0x7f7998009040 comp rx=0 tx=0).stop 2024-04-25T08:31:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.061+0000 7f79a466f640 1 -- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79940df410 msgr2=0x7f79940e1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.061+0000 7f79a466f640 1 --2- 172.21.15.38:0/3693419528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79940df410 0x7f79940e1810 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f799c074500 tx=0x7f799c0686a0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.061+0000 7f79a2c0c640 1 -- 172.21.15.38:0/3693419528 reap_dead start 2024-04-25T08:31:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.061+0000 7f79a466f640 1 -- 172.21.15.38:0/3693419528 shutdown_connections 2024-04-25T08:31:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.061+0000 7f79a466f640 1 -- 172.21.15.38:0/3693419528 >> 172.21.15.38:0/3693419528 conn(0x7f7994009fe0 msgr2=0x7f79940365c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.061+0000 7f79a466f640 1 -- 172.21.15.38:0/3693419528 shutdown_connections 2024-04-25T08:31:55.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.061+0000 7f79a466f640 1 -- 172.21.15.38:0/3693419528 wait complete. 2024-04-25T08:31:55.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.062+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 --> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f09b004ef90 con 0x7f099c074bd0 2024-04-25T08:31:55.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.063+0000 7f0997fff640 1 -- 172.21.15.38:0/4025353335 <== osd.2 v2:172.21.15.38:6824/3010466496 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f09b004ef90 con 0x7f099c074bd0 2024-04-25T08:31:55.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.063+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 --> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc82815d660 con 0x7fc814074ca0 2024-04-25T08:31:55.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.063+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 >> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] conn(0x7f099c074bd0 msgr2=0x7f099c077010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.063+0000 7f09b5b5c640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] conn(0x7f099c074bd0 0x7f099c077010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.064+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f099c06b4d0 msgr2=0x7f099c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.064+0000 7f09b5b5c640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f099c06b4d0 0x7f099c06d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f09b00cece0 tx=0x7f09a4008040 comp rx=0 tx=0).stop 2024-04-25T08:31:55.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.064+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09980039b0 msgr2=0x7f09b00adba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.064+0000 7f09b5b5c640 1 --2- 172.21.15.38:0/4025353335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09980039b0 0x7f09b00adba0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f09a002a6a0 tx=0x7f09a0001230 comp rx=0 tx=0).stop 2024-04-25T08:31:55.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.064+0000 7f09affff640 1 -- 172.21.15.38:0/4025353335 reap_dead start 2024-04-25T08:31:55.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.064+0000 7fc81dffb640 1 -- 172.21.15.38:0/851637094 <== osd.7 v2:172.21.15.107:6824/3757070988 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc82815d660 con 0x7fc814074ca0 2024-04-25T08:31:55.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.064+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 shutdown_connections 2024-04-25T08:31:55.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.064+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 >> 172.21.15.38:0/4025353335 conn(0x7f09b006abe0 msgr2=0x7f09b015e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.065+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 shutdown_connections 2024-04-25T08:31:55.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.065+0000 7f09b5b5c640 1 -- 172.21.15.38:0/4025353335 wait complete. 2024-04-25T08:31:55.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.065+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] conn(0x7fc814074ca0 msgr2=0x7fc8140770e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.065+0000 7fc82f45d640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] conn(0x7fc814074ca0 0x7fc8140770e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.094 INFO:teuthology.orchestra.run.smithi038.stdout:55834574857 2024-04-25T08:31:55.094 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T08:31:55.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.066+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc81406b5a0 msgr2=0x7fc81406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.066+0000 7fc82f45d640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc81406b5a0 0x7fc81406da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc818001510 tx=0x7fc8180011b0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.066+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc82813f0c0 msgr2=0x7fc828155760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.066+0000 7fc82f45d640 1 --2- 172.21.15.38:0/851637094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc82813f0c0 0x7fc828155760 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc824031000 tx=0x7fc8240018a0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.066+0000 7fc82d9fa640 1 -- 172.21.15.38:0/851637094 reap_dead start 2024-04-25T08:31:55.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.067+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 shutdown_connections 2024-04-25T08:31:55.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.067+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 >> 172.21.15.38:0/851637094 conn(0x7fc8280b1130 msgr2=0x7fc82815e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.067+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 shutdown_connections 2024-04-25T08:31:55.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.067+0000 7fc82f45d640 1 -- 172.21.15.38:0/851637094 wait complete. 2024-04-25T08:31:55.096 INFO:teuthology.orchestra.run.smithi038.stdout:55834574858 2024-04-25T08:31:55.096 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T08:31:55.097 INFO:teuthology.orchestra.run.smithi038.stdout:55834574859 2024-04-25T08:31:55.097 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T08:31:55.098 INFO:teuthology.orchestra.run.smithi038.stdout:55834574858 2024-04-25T08:31:55.098 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T08:31:55.099 INFO:teuthology.orchestra.run.smithi038.stdout:55834574857 2024-04-25T08:31:55.099 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T08:31:55.100 INFO:teuthology.orchestra.run.smithi038.stdout:55834574857 2024-04-25T08:31:55.100 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T08:31:55.101 INFO:teuthology.orchestra.run.smithi038.stdout:55834574858 2024-04-25T08:31:55.101 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T08:31:55.102 INFO:teuthology.orchestra.run.smithi038.stdout:55834574858 2024-04-25T08:31:55.103 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T08:31:55.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.314+0000 7f6803fff640 1 Processor -- start 2024-04-25T08:31:55.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.314+0000 7f6803fff640 1 -- start start 2024-04-25T08:31:55.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.314+0000 7f4d1f573640 1 Processor -- start 2024-04-25T08:31:55.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.314+0000 7f4d1f573640 1 -- start start 2024-04-25T08:31:55.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.315+0000 7f6803fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67fc033990 0x7f67fc0543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.315+0000 7f6803fff640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67fc053e00 con 0x7f67fc0075b0 2024-04-25T08:31:55.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.315+0000 7f6803fff640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67fc0540e0 con 0x7f67fc007190 2024-04-25T08:31:55.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.315+0000 7f6803fff640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc108730 con 0x7f67fc033990 2024-04-25T08:31:55.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f6802ffd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67fc033990 0x7f67fc0543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f68027fc640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f67fc007190 0x7f67fc037c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:41988/0 (socket says 172.21.15.38:41988) 2024-04-25T08:31:55.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f68027fc640 1 -- 172.21.15.38:0/3024907430 learned_addr learned my addr 172.21.15.38:0/3024907430 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:55.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7fbdf299d640 1 Processor -- start 2024-04-25T08:31:55.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f4d1f573640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d100521b0 0x7f4d100ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f4d1f573640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d100e1430 con 0x7f4d100e1050 2024-04-25T08:31:55.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f4d1f573640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d10055710 con 0x7f4d1003ca10 2024-04-25T08:31:55.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f4d1f573640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10056320 con 0x7f4d100521b0 2024-04-25T08:31:55.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7fbdf299d640 1 -- start start 2024-04-25T08:31:55.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f4fc6cc0640 1 Processor -- start 2024-04-25T08:31:55.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7fb6b270e640 1 Processor -- start 2024-04-25T08:31:55.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f4d1ed72640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d100521b0 0x7f4d100ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f4d1dd70640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f4d1003ca10 0x7f4d10051aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:41996/0 (socket says 172.21.15.38:41996) 2024-04-25T08:31:55.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f4d1dd70640 1 -- 172.21.15.38:0/1436167529 learned_addr learned my addr 172.21.15.38:0/1436167529 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:55.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f4fc6cc0640 1 -- start start 2024-04-25T08:31:55.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f6802ffd640 1 -- 172.21.15.38:0/3024907430 >> v1:172.21.15.107:6789/0 conn(0x7f67fc007190 legacy=0x7f67fc037c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f6802ffd640 1 -- 172.21.15.38:0/3024907430 >> v1:172.21.15.38:6789/0 conn(0x7f67fc0075b0 legacy=0x7f67fc052c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f6802ffd640 1 -- 172.21.15.38:0/3024907430 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc006c70 con 0x7f67fc033990 2024-04-25T08:31:55.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.317+0000 7f27c0f89640 1 Processor -- start 2024-04-25T08:31:55.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.317+0000 7f27c0f89640 1 -- start start 2024-04-25T08:31:55.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.317+0000 7fb6b270e640 1 -- start start 2024-04-25T08:31:55.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f6eb5e76640 1 Processor -- start 2024-04-25T08:31:55.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.316+0000 7f6eb5e76640 1 -- start start 2024-04-25T08:31:55.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6802ffd640 1 --2- 172.21.15.38:0/3024907430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67fc033990 0x7f67fc0543c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009750 tx=0x7f67f402a030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0f9eb592bafe929 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.317+0000 7f4d1ed72640 1 -- 172.21.15.38:0/1436167529 >> v1:172.21.15.107:6789/0 conn(0x7f4d1003ca10 legacy=0x7f4d10051aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.317+0000 7f4d1ed72640 1 -- 172.21.15.38:0/1436167529 >> v1:172.21.15.38:6789/0 conn(0x7f4d100e1050 legacy=0x7f4d10054da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.317+0000 7f4d1ed72640 1 -- 172.21.15.38:0/1436167529 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d100559f0 con 0x7f4d100521b0 2024-04-25T08:31:55.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6eb5e76640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ea8033f80 0x7f6ea80539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6eb5e76640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6ea8038330 0x7f6ea804f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6eb5e76640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea8052630 con 0x7f6ea8053ef0 2024-04-25T08:31:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6eb5e76640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea8053520 con 0x7f6ea8038330 2024-04-25T08:31:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6eb5e76640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea8052910 con 0x7f6ea8033f80 2024-04-25T08:31:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6eaeffd640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f6ea8053ef0 0x7f6ea8037c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:41998/0 (socket says 172.21.15.38:41998) 2024-04-25T08:31:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6eaeffd640 1 -- 172.21.15.38:0/1285729127 learned_addr learned my addr 172.21.15.38:0/1285729127 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6eaffff640 1 --2- 172.21.15.38:0/1285729127 >> v2:172.21.15.38:3300/0 conn(0x7f6ea8038330 0x7f6ea804f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6801ffb640 1 -- 172.21.15.38:0/3024907430 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4037070 con 0x7f67fc033990 2024-04-25T08:31:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6801ffb640 1 -- 172.21.15.38:0/3024907430 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f402ad40 con 0x7f67fc033990 2024-04-25T08:31:55.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6801ffb640 1 -- 172.21.15.38:0/3024907430 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4032660 con 0x7f67fc033990 2024-04-25T08:31:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7fbdebfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbde40df030 0x7fbde404fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49878/0 (socket says 172.21.15.38:49878) 2024-04-25T08:31:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7fbdebfff640 1 -- 172.21.15.38:0/3013149349 learned_addr learned my addr 172.21.15.38:0/3013149349 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f4fc6cc0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fc00b26f0 0x7f4fc015a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f4fc6cc0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc0150d90 con 0x7f4fc0067bc0 2024-04-25T08:31:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f4fc6cc0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc0159f50 con 0x7f4fc00ae860 2024-04-25T08:31:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f4fc6cc0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc015dfd0 con 0x7f4fc00b26f0 2024-04-25T08:31:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f4fc54bd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fc00b26f0 0x7f4fc015a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f4fc54bd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fc00b26f0 0x7f4fc015a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:57324/0 (socket says 172.21.15.38:57324) 2024-04-25T08:31:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f4fc54bd640 1 -- 172.21.15.38:0/2742762695 learned_addr learned my addr 172.21.15.38:0/2742762695 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f4fc54bd640 1 -- 172.21.15.38:0/2742762695 >> v1:172.21.15.107:6789/0 conn(0x7f4fc00ae860 legacy=0x7f4fc006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f4fc54bd640 1 -- 172.21.15.38:0/2742762695 >> v1:172.21.15.38:6789/0 conn(0x7f4fc0067bc0 legacy=0x7f4fc015d430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:31:55.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f4fc54bd640 1 -- 172.21.15.38:0/2742762695 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc015a1c0 con 0x7f4fc00b26f0 2024-04-25T08:31:55.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7fb6b270e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6ac1599e0 0x7fb6ac0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7fb6b270e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb6ac071da0 0x7fb6ac0b1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7fb6b0f0b640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fb6ac06d750 0x7fb6ac0c98d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:42024/0 (socket says 172.21.15.38:42024) 2024-04-25T08:31:55.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7f6eaf7fe640 1 --2- 172.21.15.38:0/1285729127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ea8033f80 0x7f6ea80539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f6eaffff640 1 -- 172.21.15.38:0/1285729127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ea8033f80 msgr2=0x7f6ea80539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f6eaffff640 1 --2- 172.21.15.38:0/1285729127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ea8033f80 0x7f6ea80539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f6eaffff640 1 -- 172.21.15.38:0/1285729127 >> v1:172.21.15.107:6789/0 conn(0x7f6ea8053ef0 legacy=0x7f6ea8037c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f6eaffff640 1 -- 172.21.15.38:0/1285729127 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea8052bf0 con 0x7f6ea8038330 2024-04-25T08:31:55.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f6803fff640 1 -- 172.21.15.38:0/3024907430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67fc033990 msgr2=0x7f67fc0543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f6803fff640 1 --2- 172.21.15.38:0/3024907430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67fc033990 0x7f67fc0543c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009750 tx=0x7f67f402a030 comp rx=0 tx=0).stop 2024-04-25T08:31:55.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7fbdf299d640 1 --2- 172.21.15.38:0/3013149349 >> v2:172.21.15.38:3300/0 conn(0x7fbde4050140 0x7fbde4038170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.318+0000 7fbdf299d640 1 --2- 172.21.15.38:0/3013149349 >> v2:172.21.15.107:3300/0 conn(0x7fbde40df410 0x7fbde4006a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7fbdf299d640 1 -- 172.21.15.38:0/3013149349 --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde4005f60 con 0x7fbde40df030 2024-04-25T08:31:55.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7fbdf299d640 1 -- 172.21.15.38:0/3013149349 --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde40e38e0 con 0x7fbde4050140 2024-04-25T08:31:55.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7fbdf299d640 1 -- 172.21.15.38:0/3013149349 --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde4006200 con 0x7fbde40df410 2024-04-25T08:31:55.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7fbdeaffd640 1 -- 172.21.15.38:0/3013149349 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3974923866 0 0) 0x7fbde4005f60 con 0x7fbde40df030 2024-04-25T08:31:55.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7fbdeaffd640 1 -- 172.21.15.38:0/3013149349 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbddc003520 con 0x7fbde40df030 2024-04-25T08:31:55.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f27c0f89640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27b403a5e0 0x7f27b4034080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f27c0f89640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b4033e30 con 0x7f27b404eab0 2024-04-25T08:31:55.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f27c0f89640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b4006100 con 0x7f27b40e1030 2024-04-25T08:31:55.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f27c0f89640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b4054880 con 0x7f27b403a5e0 2024-04-25T08:31:55.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f27ba59c640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f27b40e1030 0x7f27b4039ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:42022/0 (socket says 172.21.15.38:42022) 2024-04-25T08:31:55.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f27ba59c640 1 -- 172.21.15.38:0/2517701049 learned_addr learned my addr 172.21.15.38:0/2517701049 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:55.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f27bb59e640 1 --2- 172.21.15.38:0/2517701049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27b403a5e0 0x7f27b4034080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.319+0000 7f4d1ed72640 1 --2- 172.21.15.38:0/1436167529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d100521b0 0x7f4d100ebb80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009750 tx=0x7f4d0802a010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd87e18d11b60f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fb6b0f0b640 1 -- 172.21.15.38:0/1002541032 learned_addr learned my addr 172.21.15.38:0/1002541032 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:55.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fb6b270e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ac15d7d0 con 0x7fb6ac06d750 2024-04-25T08:31:55.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fb6b270e640 1 -- 172.21.15.38:0/1002541032 --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ac15e590 con 0x7fb6ac071da0 2024-04-25T08:31:55.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fbdeb7fe640 1 --2- 172.21.15.38:0/3013149349 >> v2:172.21.15.38:3300/0 conn(0x7fbde4050140 0x7fbde4038170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fbdf0f3a640 1 --2- 172.21.15.38:0/3013149349 >> v2:172.21.15.107:3300/0 conn(0x7fbde40df410 0x7fbde4006a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f4d1d56f640 1 -- 172.21.15.38:0/1436167529 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08037070 con 0x7f4d100521b0 2024-04-25T08:31:55.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f8f0545d640 1 Processor -- start 2024-04-25T08:31:55.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f4d1d56f640 1 -- 172.21.15.38:0/1436167529 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0802ad20 con 0x7f4d100521b0 2024-04-25T08:31:55.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f4d1d56f640 1 -- 172.21.15.38:0/1436167529 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08032660 con 0x7f4d100521b0 2024-04-25T08:31:55.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f6803fff640 1 -- 172.21.15.38:0/3024907430 shutdown_connections 2024-04-25T08:31:55.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f6803fff640 1 --2- 172.21.15.38:0/3024907430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67fc033990 0x7f67fc0543c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f6803fff640 1 -- 172.21.15.38:0/3024907430 >> 172.21.15.38:0/3024907430 conn(0x7f67fc009fb0 msgr2=0x7f67fc00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f6eaffff640 1 --2- 172.21.15.38:0/1285729127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ea8038330 0x7f6ea804f0b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6eb002d370 tx=0x7f6eb00a4f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31ea5c915b51fbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f27b959a640 1 -- 172.21.15.38:0/2517701049 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217729762 0 0) 0x7f27b4006100 con 0x7f27b40e1030 2024-04-25T08:31:55.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fb6b270e640 1 -- 172.21.15.38:0/1002541032 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac15e100 con 0x7fb6ac1599e0 2024-04-25T08:31:55.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f27b959a640 1 -- 172.21.15.38:0/2517701049 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27a4003520 con 0x7f27b40e1030 2024-04-25T08:31:55.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f6803fff640 1 -- 172.21.15.38:0/3024907430 shutdown_connections 2024-04-25T08:31:55.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f8f0545d640 1 -- start start 2024-04-25T08:31:55.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f27bb59e640 1 -- 172.21.15.38:0/2517701049 >> v1:172.21.15.107:6789/0 conn(0x7f27b40e1030 legacy=0x7f27b4039ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fbdeaffd640 1 -- 172.21.15.38:0/3013149349 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3226781604 0 0) 0x7fbddc003520 con 0x7fbde40df030 2024-04-25T08:31:55.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f27bb59e640 1 -- 172.21.15.38:0/2517701049 >> v1:172.21.15.38:6789/0 conn(0x7f27b404eab0 legacy=0x7f27b40068c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f27bb59e640 1 -- 172.21.15.38:0/2517701049 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b404f4f0 con 0x7f27b403a5e0 2024-04-25T08:31:55.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fbdeb7fe640 1 -- 172.21.15.38:0/3013149349 >> v2:172.21.15.107:3300/0 conn(0x7fbde40df410 msgr2=0x7fbde4006a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fbdeb7fe640 1 --2- 172.21.15.38:0/3013149349 >> v2:172.21.15.107:3300/0 conn(0x7fbde40df410 0x7fbde4006a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fbdeb7fe640 1 -- 172.21.15.38:0/3013149349 >> v1:172.21.15.38:6789/0 conn(0x7fbde40df030 legacy=0x7fbde404fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fbdeb7fe640 1 -- 172.21.15.38:0/3013149349 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde40064e0 con 0x7fbde4050140 2024-04-25T08:31:55.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f6eae7fc640 1 -- 172.21.15.38:0/1285729127 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00b2070 con 0x7f6ea8038330 2024-04-25T08:31:55.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f6eae7fc640 1 -- 172.21.15.38:0/1285729127 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb00a5d20 con 0x7f6ea8038330 2024-04-25T08:31:55.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f6eae7fc640 1 -- 172.21.15.38:0/1285729127 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00ad640 con 0x7f6ea8038330 2024-04-25T08:31:55.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f4fc54bd640 1 --2- 172.21.15.38:0/2742762695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fc00b26f0 0x7f4fc015a6e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc013180 tx=0x7f4fbc032a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0eea0d0817c1ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fb6b170c640 1 --2- 172.21.15.38:0/1002541032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6ac1599e0 0x7fb6ac0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fb6b1f0d640 1 --2- 172.21.15.38:0/1002541032 >> v2:172.21.15.38:3300/0 conn(0x7fb6ac071da0 0x7fb6ac0b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751403801 0 0) 0x7fb6ac15d7d0 con 0x7fb6ac06d750 2024-04-25T08:31:55.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb69800bd30 con 0x7fb6ac06d750 2024-04-25T08:31:55.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1535399937 0 0) 0x7fb69800bd30 con 0x7fb6ac06d750 2024-04-25T08:31:55.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6ac1599e0 msgr2=0x7fb6ac0b0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 --2- 172.21.15.38:0/1002541032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6ac1599e0 0x7fb6ac0b0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 >> v2:172.21.15.38:3300/0 conn(0x7fb6ac071da0 msgr2=0x7fb6ac0b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 --2- 172.21.15.38:0/1002541032 >> v2:172.21.15.38:3300/0 conn(0x7fb6ac071da0 0x7fb6ac0b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac15dab0 con 0x7fb6ac06d750 2024-04-25T08:31:55.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fb69c001670 con 0x7fb6ac06d750 2024-04-25T08:31:55.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 >> v1:172.21.15.107:6789/0 conn(0x7fb6ac06d750 legacy=0x7fb6ac0c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 --2- 172.21.15.38:0/1002541032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb69800c1c0 0x7fb69800c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f4d1f573640 1 -- 172.21.15.38:0/1436167529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d100521b0 msgr2=0x7f4d100ebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 --2- 172.21.15.38:0/1002541032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb69800d010 0x7fb69800d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f4d1f573640 1 --2- 172.21.15.38:0/1436167529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d100521b0 0x7f4d100ebb80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009750 tx=0x7f4d0802a010 comp rx=0 tx=0).stop 2024-04-25T08:31:55.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 --2- 172.21.15.38:0/1002541032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb69800d9d0 0x7fb698009550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f4fc4cbc640 1 -- 172.21.15.38:0/2742762695 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc03b020 con 0x7f4fc00b26f0 2024-04-25T08:31:55.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f4fc4cbc640 1 -- 172.21.15.38:0/2742762695 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fbc0114e0 con 0x7f4fc00b26f0 2024-04-25T08:31:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f4fc4cbc640 1 -- 172.21.15.38:0/2742762695 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc00fa70 con 0x7f4fc00b26f0 2024-04-25T08:31:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac15e100 con 0x7fb69800d010 2024-04-25T08:31:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb698009a90 con 0x7fb69800d9d0 2024-04-25T08:31:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb698009c90 con 0x7fb69800c1c0 2024-04-25T08:31:55.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f8f0545d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ef80069d0 0x7f8ef8006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f8f0545d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef8053710 con 0x7f8ef8005e80 2024-04-25T08:31:55.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f8f0545d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef80539f0 con 0x7f8ef80eacf0 2024-04-25T08:31:55.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f8f0545d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef8054320 con 0x7f8ef80069d0 2024-04-25T08:31:55.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f27bb59e640 1 --2- 172.21.15.38:0/2517701049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27b403a5e0 0x7f27b4034080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f27bc02d370 tx=0x7f27bc0a4f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eb37053525bb17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7fb6b1f0d640 1 --2- 172.21.15.38:0/1002541032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb69800d9d0 0x7fb698009550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f4d1f573640 1 -- 172.21.15.38:0/1436167529 shutdown_connections 2024-04-25T08:31:55.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f4d1f573640 1 --2- 172.21.15.38:0/1436167529 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d100521b0 0x7f4d100ebb80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.321+0000 7f4d1f573640 1 -- 172.21.15.38:0/1436167529 >> 172.21.15.38:0/1436167529 conn(0x7f4d10056cd0 msgr2=0x7f4d100570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ef80069d0 0x7f8ef8006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7fb6b0f0b640 1 --2- 172.21.15.38:0/1002541032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb69800d010 0x7fb69800d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f8efeffd640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f8ef80eacf0 0x7f8ef804fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:42028/0 (socket says 172.21.15.38:42028) 2024-04-25T08:31:55.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f8efeffd640 1 -- 172.21.15.38:0/3742256655 learned_addr learned my addr 172.21.15.38:0/3742256655 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:55.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f27b959a640 1 -- 172.21.15.38:0/2517701049 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0b2070 con 0x7f27b403a5e0 2024-04-25T08:31:55.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f4fc6cc0640 1 -- 172.21.15.38:0/2742762695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fc00b26f0 msgr2=0x7f4fc015a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f4fc6cc0640 1 --2- 172.21.15.38:0/2742762695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fc00b26f0 0x7f4fc015a6e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc013180 tx=0x7f4fbc032a20 comp rx=0 tx=0).stop 2024-04-25T08:31:55.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f4d1f573640 1 -- 172.21.15.38:0/1436167529 shutdown_connections 2024-04-25T08:31:55.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f27b959a640 1 -- 172.21.15.38:0/2517701049 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc0a5d20 con 0x7f27b403a5e0 2024-04-25T08:31:55.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f27b959a640 1 -- 172.21.15.38:0/2517701049 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc0ad640 con 0x7f27b403a5e0 2024-04-25T08:31:55.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f4fc6cc0640 1 -- 172.21.15.38:0/2742762695 shutdown_connections 2024-04-25T08:31:55.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f4fc6cc0640 1 --2- 172.21.15.38:0/2742762695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fc00b26f0 0x7f4fc015a6e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.320+0000 7f6803fff640 1 -- 172.21.15.38:0/3024907430 wait complete. 2024-04-25T08:31:55.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f6803fff640 1 Processor -- start 2024-04-25T08:31:55.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7fb6b0f0b640 1 -- 172.21.15.38:0/1002541032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb69800c1c0 msgr2=0x7fb69800c5e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:31:55.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7fb6b0f0b640 1 --2- 172.21.15.38:0/1002541032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb69800c1c0 0x7fb69800c5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7fb6b0f0b640 1 -- 172.21.15.38:0/1002541032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb69800d9d0 msgr2=0x7fb698009550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f6eb5e76640 1 -- 172.21.15.38:0/1285729127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ea8038330 msgr2=0x7f6ea804f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f6eb5e76640 1 --2- 172.21.15.38:0/1285729127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ea8038330 0x7f6ea804f0b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6eb002d370 tx=0x7f6eb00a4f10 comp rx=0 tx=0).stop 2024-04-25T08:31:55.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7fb6b0f0b640 1 --2- 172.21.15.38:0/1002541032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb69800d9d0 0x7fb698009550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7fbdeb7fe640 1 --2- 172.21.15.38:0/3013149349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbde4050140 0x7fbde4038170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbde4034500 tx=0x7fbde002c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6f731cca20e64b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7fb6b0f0b640 1 -- 172.21.15.38:0/1002541032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69800bd30 con 0x7fb69800d010 2024-04-25T08:31:55.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f4fc6cc0640 1 -- 172.21.15.38:0/2742762695 >> 172.21.15.38:0/2742762695 conn(0x7f4fc00b1380 msgr2=0x7f4fc00b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.323+0000 7fb6b1f0d640 1 --2- 172.21.15.38:0/1002541032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb69800d9d0 0x7fb698009550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:55.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.323+0000 7f6803fff640 1 -- start start 2024-04-25T08:31:55.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f6eb5e76640 1 -- 172.21.15.38:0/1285729127 shutdown_connections 2024-04-25T08:31:55.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.323+0000 7f27c0f89640 1 -- 172.21.15.38:0/2517701049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27b403a5e0 msgr2=0x7f27b4034080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.323+0000 7f6eb5e76640 1 --2- 172.21.15.38:0/1285729127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ea8038330 0x7f6ea804f0b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.323+0000 7f6eb5e76640 1 --2- 172.21.15.38:0/1285729127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ea8033f80 0x7f6ea80539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.323+0000 7f6eb5e76640 1 -- 172.21.15.38:0/1285729127 >> 172.21.15.38:0/1285729127 conn(0x7f6ea8009fe0 msgr2=0x7f6ea800a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f8eff7fe640 1 -- 172.21.15.38:0/3742256655 >> v1:172.21.15.107:6789/0 conn(0x7f8ef80eacf0 legacy=0x7f8ef804fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f8eff7fe640 1 -- 172.21.15.38:0/3742256655 >> v1:172.21.15.38:6789/0 conn(0x7f8ef8005e80 legacy=0x7f8ef80062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f8eff7fe640 1 -- 172.21.15.38:0/3742256655 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef8053cd0 con 0x7f8ef80069d0 2024-04-25T08:31:55.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.323+0000 7f27c0f89640 1 --2- 172.21.15.38:0/2517701049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27b403a5e0 0x7f27b4034080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f27bc02d370 tx=0x7f27bc0a4f10 comp rx=0 tx=0).stop 2024-04-25T08:31:55.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.323+0000 7f4fc6cc0640 1 -- 172.21.15.38:0/2742762695 shutdown_connections 2024-04-25T08:31:55.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.323+0000 7f4fc6cc0640 1 -- 172.21.15.38:0/2742762695 wait complete. 2024-04-25T08:31:55.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.323+0000 7f8eff7fe640 1 --2- 172.21.15.38:0/3742256655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ef80069d0 0x7f8ef8006e60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00097d0 tx=0x7f8ef002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e0ba14ea2e86e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.323+0000 7fbdeaffd640 1 -- 172.21.15.38:0/3013149349 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde003a070 con 0x7fbde4050140 2024-04-25T08:31:55.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.322+0000 7f4d1f573640 1 -- 172.21.15.38:0/1436167529 wait complete. 2024-04-25T08:31:55.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f6eb5e76640 1 -- 172.21.15.38:0/1285729127 shutdown_connections 2024-04-25T08:31:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7fbdeaffd640 1 -- 172.21.15.38:0/3013149349 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde0032480 con 0x7fbde4050140 2024-04-25T08:31:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7fbdeaffd640 1 -- 172.21.15.38:0/3013149349 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec04f770 con 0x7fbde4050140 2024-04-25T08:31:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f8efdffb640 1 -- 172.21.15.38:0/3742256655 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0037070 con 0x7f8ef80069d0 2024-04-25T08:31:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f8efdffb640 1 -- 172.21.15.38:0/3742256655 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef0032440 con 0x7f8ef80069d0 2024-04-25T08:31:55.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7fb6b0f0b640 1 --2- 172.21.15.38:0/1002541032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb69800d010 0x7fb69800d490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb69c007fd0 tx=0x7fb69800e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a6ba163b6cb090 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f6eb5e76640 1 -- 172.21.15.38:0/1285729127 wait complete. 2024-04-25T08:31:55.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f4d1f573640 1 Processor -- start 2024-04-25T08:31:55.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f8efdffb640 1 -- 172.21.15.38:0/3742256655 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0037070 con 0x7f8ef80069d0 2024-04-25T08:31:55.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f27c0f89640 1 -- 172.21.15.38:0/2517701049 shutdown_connections 2024-04-25T08:31:55.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f27c0f89640 1 --2- 172.21.15.38:0/2517701049 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27b403a5e0 0x7f27b4034080 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f27c0f89640 1 -- 172.21.15.38:0/2517701049 >> 172.21.15.38:0/2517701049 conn(0x7f27b4054c90 msgr2=0x7f27b40550a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f4d1f573640 1 -- start start 2024-04-25T08:31:55.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f6803fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67fc007190 0x7f67fc035e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f6803fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67fc033990 0x7f67fc046ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f6803fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67fc0375a0 0x7f67fc0efe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f6803fff640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc038350 con 0x7f67fc033990 2024-04-25T08:31:55.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f6803fff640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc101250 con 0x7f67fc0375a0 2024-04-25T08:31:55.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f6803fff640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc0384d0 con 0x7f67fc007190 2024-04-25T08:31:55.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f68027fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67fc033990 0x7f67fc046ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f68027fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67fc033990 0x7f67fc046ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33562/0 (socket says 172.21.15.38:33562) 2024-04-25T08:31:55.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f68027fc640 1 -- 172.21.15.38:0/1978305071 learned_addr learned my addr 172.21.15.38:0/1978305071 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f27c0f89640 1 -- 172.21.15.38:0/2517701049 shutdown_connections 2024-04-25T08:31:55.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f27c0f89640 1 -- 172.21.15.38:0/2517701049 wait complete. 2024-04-25T08:31:55.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f4fc6cc0640 1 Processor -- start 2024-04-25T08:31:55.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.324+0000 7f6802ffd640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67fc007190 0x7f67fc035e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f4fc6cc0640 1 -- start start 2024-04-25T08:31:55.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f8f0545d640 1 -- 172.21.15.38:0/3742256655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ef80069d0 msgr2=0x7f8ef8006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f8f0545d640 1 --2- 172.21.15.38:0/3742256655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ef80069d0 0x7f8ef8006e60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8ef00097d0 tx=0x7f8ef002a250 comp rx=0 tx=0).stop 2024-04-25T08:31:55.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7fbdf299d640 1 -- 172.21.15.38:0/3013149349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbde4050140 msgr2=0x7fbde4038170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7fbdf299d640 1 --2- 172.21.15.38:0/3013149349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbde4050140 0x7fbde4038170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbde4034500 tx=0x7fbde002c660 comp rx=0 tx=0).stop 2024-04-25T08:31:55.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c007ca0 con 0x7fb69800d010 2024-04-25T08:31:55.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f8f0545d640 1 -- 172.21.15.38:0/3742256655 shutdown_connections 2024-04-25T08:31:55.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f8f0545d640 1 --2- 172.21.15.38:0/3742256655 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ef80069d0 0x7f8ef8006e60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f8f0545d640 1 -- 172.21.15.38:0/3742256655 >> 172.21.15.38:0/3742256655 conn(0x7f8ef8009fa0 msgr2=0x7f8ef800a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c005ad0 con 0x7fb69800d010 2024-04-25T08:31:55.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f8f0545d640 1 -- 172.21.15.38:0/3742256655 shutdown_connections 2024-04-25T08:31:55.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7fb6a3fff640 1 -- 172.21.15.38:0/1002541032 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c007070 con 0x7fb69800d010 2024-04-25T08:31:55.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f68037fe640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67fc0375a0 0x7f67fc0efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f6802ffd640 1 -- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67fc0375a0 msgr2=0x7f67fc0efe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f6802ffd640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67fc0375a0 0x7f67fc0efe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f6802ffd640 1 -- 172.21.15.38:0/1978305071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67fc033990 msgr2=0x7f67fc046ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f6802ffd640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67fc033990 0x7f67fc046ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f6802ffd640 1 -- 172.21.15.38:0/1978305071 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc047000 con 0x7f67fc007190 2024-04-25T08:31:55.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f68037fe640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67fc0375a0 0x7f67fc0efe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:55.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7fbdf299d640 1 -- 172.21.15.38:0/3013149349 shutdown_connections 2024-04-25T08:31:55.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7fbdf299d640 1 --2- 172.21.15.38:0/3013149349 >> v2:172.21.15.107:3300/0 conn(0x7fbde40df410 0x7fbde4006a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7fbdf299d640 1 --2- 172.21.15.38:0/3013149349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbde4050140 0x7fbde4038170 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7fbdf299d640 1 -- 172.21.15.38:0/3013149349 >> 172.21.15.38:0/3013149349 conn(0x7fbde4009fe0 msgr2=0x7fbde400a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f6eb5e76640 1 Processor -- start 2024-04-25T08:31:55.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7fbdf299d640 1 -- 172.21.15.38:0/3013149349 shutdown_connections 2024-04-25T08:31:55.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7fbdf299d640 1 -- 172.21.15.38:0/3013149349 wait complete. 2024-04-25T08:31:55.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f6802ffd640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67fc007190 0x7f67fc035e90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f67fc008680 tx=0x7f67f40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f6eb5e76640 1 -- start start 2024-04-25T08:31:55.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f4d1f573640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1003ca10 0x7f4d100e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.325+0000 7f4d1f573640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d100521b0 0x7f4d100e2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1dd70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d100521b0 0x7f4d100e2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1dd70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d100521b0 0x7f4d100e2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33574/0 (socket says 172.21.15.38:33574) 2024-04-25T08:31:55.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1e571640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1003ca10 0x7f4d100e1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1e571640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1003ca10 0x7f4d100e1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:57378/0 (socket says 172.21.15.38:57378) 2024-04-25T08:31:55.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1dd70640 1 -- 172.21.15.38:0/1481308458 learned_addr learned my addr 172.21.15.38:0/1481308458 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1f573640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d100e1050 0x7f4d100dbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d10055fb0 con 0x7f4d100521b0 2024-04-25T08:31:55.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100554b0 con 0x7f4d100e1050 2024-04-25T08:31:55.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100567b0 con 0x7f4d1003ca10 2024-04-25T08:31:55.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4fc6cc0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fc0067bc0 0x7f4fc0142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f67ebfff640 1 -- 172.21.15.38:0/1978305071 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4037070 con 0x7f67fc007190 2024-04-25T08:31:55.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4fc6cc0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fc00ae860 0x7f4fc00cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7fb6b270e640 1 -- 172.21.15.38:0/1002541032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb69800d010 msgr2=0x7fb69800d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4fc6cc0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc00b26f0 0x7f4fc00cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7fb6b270e640 1 --2- 172.21.15.38:0/1002541032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb69800d010 0x7fb69800d490 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb69c007fd0 tx=0x7fb69800e040 comp rx=0 tx=0).stop 2024-04-25T08:31:55.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4fc6cc0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc015ada0 con 0x7f4fc00b26f0 2024-04-25T08:31:55.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7fb6b1f0d640 1 -- 172.21.15.38:0/1002541032 reap_dead start 2024-04-25T08:31:55.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7fb6b270e640 1 -- 172.21.15.38:0/1002541032 shutdown_connections 2024-04-25T08:31:55.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f4fc6cc0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc015ac20 con 0x7f4fc00ae860 2024-04-25T08:31:55.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f4fc6cc0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc006b2f0 con 0x7f4fc0067bc0 2024-04-25T08:31:55.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eb5e76640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ea8033f80 0x7f6ea804a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eb5e76640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ea8038330 0x7f6ea804c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eb5e76640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ea8053ef0 0x7f6ea80d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eb5e76640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea80546f0 con 0x7f6ea8033f80 2024-04-25T08:31:55.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eb5e76640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea80531b0 con 0x7f6ea8053ef0 2024-04-25T08:31:55.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eb5e76640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea8054870 con 0x7f6ea8038330 2024-04-25T08:31:55.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f4fc64bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc00b26f0 0x7f4fc00cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f4fc64bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc00b26f0 0x7f4fc00cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33576/0 (socket says 172.21.15.38:33576) 2024-04-25T08:31:55.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f4fc64bf640 1 -- 172.21.15.38:0/2896487314 learned_addr learned my addr 172.21.15.38:0/2896487314 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f27c0f89640 1 Processor -- start 2024-04-25T08:31:55.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f4fc54bd640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fc00ae860 0x7f4fc00cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7fb6b270e640 1 -- 172.21.15.38:0/1002541032 >> 172.21.15.38:0/1002541032 conn(0x7fb6ac0cbd00 msgr2=0x7fb6ac0cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eaffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ea8053ef0 0x7f6ea80d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eaffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ea8053ef0 0x7f6ea80d5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:48494/0 (socket says 172.21.15.38:48494) 2024-04-25T08:31:55.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eaffff640 1 -- 172.21.15.38:0/3091312614 learned_addr learned my addr 172.21.15.38:0/3091312614 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f27c0f89640 1 -- start start 2024-04-25T08:31:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7fb6b270e640 1 -- 172.21.15.38:0/1002541032 shutdown_connections 2024-04-25T08:31:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7fb6b270e640 1 -- 172.21.15.38:0/1002541032 wait complete. 2024-04-25T08:31:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1ed72640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d100e1050 0x7f4d100dbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1dd70640 1 -- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1003ca10 msgr2=0x7f4d100e1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1dd70640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1003ca10 0x7f4d100e1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1dd70640 1 -- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d100e1050 msgr2=0x7f4d100dbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1dd70640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d100e1050 0x7f4d100dbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1dd70640 1 -- 172.21.15.38:0/1481308458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d100e20b0 con 0x7f4d100521b0 2024-04-25T08:31:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f4d1ed72640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d100e1050 0x7f4d100dbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:55.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f4d1e571640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1003ca10 0x7f4d100e1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:55.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f6803fff640 1 -- 172.21.15.38:0/1978305071 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc036500 con 0x7f67fc007190 2024-04-25T08:31:55.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.326+0000 7f6803fff640 1 -- 172.21.15.38:0/1978305071 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc036ac0 con 0x7f67fc007190 2024-04-25T08:31:55.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f67ebfff640 1 -- 172.21.15.38:0/1978305071 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f4038040 con 0x7f67fc007190 2024-04-25T08:31:55.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f67ebfff640 1 -- 172.21.15.38:0/1978305071 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4004450 con 0x7f67fc007190 2024-04-25T08:31:55.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eaffff640 1 -- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ea8038330 msgr2=0x7f6ea804c5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T08:31:55.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eaf7fe640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ea8033f80 0x7f6ea804a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eaffff640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ea8038330 0x7f6ea804c5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.327+0000 7f6eaffff640 1 -- 172.21.15.38:0/3091312614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ea8033f80 msgr2=0x7f6ea804a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f6eaffff640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ea8033f80 0x7f6ea804a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f6eaffff640 1 -- 172.21.15.38:0/3091312614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea804ce20 con 0x7f6ea8053ef0 2024-04-25T08:31:55.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f6eaf7fe640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ea8033f80 0x7f6ea804a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:55.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f6eaffff640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ea8053ef0 0x7f6ea80d5e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0074b90 tx=0x7f6eb0069b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f4d1dd70640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d100521b0 0x7f4d100e2560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4d180646b0 tx=0x7f4d180686a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7fbdf299d640 1 Processor -- start 2024-04-25T08:31:55.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f6e9bfff640 1 -- 172.21.15.38:0/3091312614 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00bc070 con 0x7f6ea8053ef0 2024-04-25T08:31:55.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f6e9bfff640 1 -- 172.21.15.38:0/3091312614 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb00b7660 con 0x7f6ea8053ef0 2024-04-25T08:31:55.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f6803fff640 1 -- 172.21.15.38:0/1978305071 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e4005290 con 0x7f67fc007190 2024-04-25T08:31:55.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f6e9bfff640 1 -- 172.21.15.38:0/3091312614 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00b2040 con 0x7f6ea8053ef0 2024-04-25T08:31:55.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f4fc5cbe640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fc0067bc0 0x7f4fc0142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f6eb5e76640 1 -- 172.21.15.38:0/3091312614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea804d100 con 0x7f6ea8053ef0 2024-04-25T08:31:55.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f27c0f89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b403a5e0 0x7f27b4035e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f6eb5e76640 1 -- 172.21.15.38:0/3091312614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea804d6c0 con 0x7f6ea8053ef0 2024-04-25T08:31:55.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f4fc64bf640 1 -- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fc0067bc0 msgr2=0x7f4fc0142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f4fc64bf640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fc0067bc0 0x7f4fc0142040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f4fc64bf640 1 -- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fc00ae860 msgr2=0x7f4fc00cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f4fc64bf640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fc00ae860 0x7f4fc00cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7f4fc64bf640 1 -- 172.21.15.38:0/2896487314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc00cdc00 con 0x7f4fc00b26f0 2024-04-25T08:31:55.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.328+0000 7fbdf299d640 1 -- start start 2024-04-25T08:31:55.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.329+0000 7f8f0545d640 1 -- 172.21.15.38:0/3742256655 wait complete. 2024-04-25T08:31:55.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.329+0000 7f4d0effd640 1 -- 172.21.15.38:0/1481308458 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1806abe0 con 0x7f4d100521b0 2024-04-25T08:31:55.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.329+0000 7f4d0effd640 1 -- 172.21.15.38:0/1481308458 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1808f020 con 0x7f4d100521b0 2024-04-25T08:31:55.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.329+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d10038720 con 0x7f4d100521b0 2024-04-25T08:31:55.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.329+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d10038ce0 con 0x7f4d100521b0 2024-04-25T08:31:55.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7fb6b270e640 1 Processor -- start 2024-04-25T08:31:55.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.329+0000 7f67ebfff640 1 -- 172.21.15.38:0/1978305071 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67f403e690 con 0x7f67fc007190 2024-04-25T08:31:55.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7f67ebfff640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f67f006b5a0 0x7f67f006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7fb6b270e640 1 -- start start 2024-04-25T08:31:55.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7f4d0effd640 1 -- 172.21.15.38:0/1481308458 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18073a70 con 0x7f4d100521b0 2024-04-25T08:31:55.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7f68027fc640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f67f006b5a0 0x7f67f006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7f67ebfff640 1 -- 172.21.15.38:0/1978305071 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f67f40ab060 con 0x7f67fc007190 2024-04-25T08:31:55.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7fbdf299d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbde4050140 0x7fbde40dc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7f4fc64bf640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc00b26f0 0x7f4fc00cd5d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800c710 tx=0x7f4fb800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7fbdf299d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbde40df030 0x7fbde40dcee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7f68027fc640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f67f006b5a0 0x7f67f006da60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6804075a90 tx=0x7f6804068940 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7f6eb5e76640 1 -- 172.21.15.38:0/3091312614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea80053c0 con 0x7f6ea8053ef0 2024-04-25T08:31:55.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7fbdebfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbde4050140 0x7fbde40dc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7fbdebfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbde4050140 0x7fbde40dc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:57406/0 (socket says 172.21.15.38:57406) 2024-04-25T08:31:55.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.330+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d00005290 con 0x7f4d100521b0 2024-04-25T08:31:55.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27c0f89640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27b404eab0 0x7f27b40cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27c0f89640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27b40e1030 0x7f27b404a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27c0f89640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b40e5a20 con 0x7f27b403a5e0 2024-04-25T08:31:55.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27c0f89640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b40063a0 con 0x7f27b40e1030 2024-04-25T08:31:55.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27c0f89640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b40e1410 con 0x7f27b404eab0 2024-04-25T08:31:55.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27ba59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b403a5e0 0x7f27b4035e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27ba59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b403a5e0 0x7f27b4035e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33588/0 (socket says 172.21.15.38:33588) 2024-04-25T08:31:55.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27ba59c640 1 -- 172.21.15.38:0/3740795466 learned_addr learned my addr 172.21.15.38:0/3740795466 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f4fb67fc640 1 -- 172.21.15.38:0/2896487314 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8016020 con 0x7f4fc00b26f0 2024-04-25T08:31:55.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f8f0545d640 1 Processor -- start 2024-04-25T08:31:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f4fb67fc640 1 -- 172.21.15.38:0/2896487314 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb8006d50 con 0x7f4fc00b26f0 2024-04-25T08:31:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f4fb67fc640 1 -- 172.21.15.38:0/2896487314 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8016020 con 0x7f4fc00b26f0 2024-04-25T08:31:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f8f0545d640 1 -- start start 2024-04-25T08:31:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f4fc6cc0640 1 -- 172.21.15.38:0/2896487314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc00cdee0 con 0x7f4fc00b26f0 2024-04-25T08:31:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7fb6b270e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac1565f0 0x7fb6ac0b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27ba59c640 1 -- 172.21.15.38:0/3740795466 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27b404eab0 msgr2=0x7f27b40cf1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T08:31:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27ba59c640 1 --2- 172.21.15.38:0/3740795466 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27b404eab0 0x7f27b40cf1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27ba59c640 1 -- 172.21.15.38:0/3740795466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27b40e1030 msgr2=0x7f27b404a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T08:31:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27ba59c640 1 --2- 172.21.15.38:0/3740795466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27b40e1030 0x7f27b404a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f27ba59c640 1 -- 172.21.15.38:0/3740795466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b4036360 con 0x7f27b403a5e0 2024-04-25T08:31:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7fb6b270e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6ac1569d0 0x7fb6ac1f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f4d0effd640 1 -- 172.21.15.38:0/1481308458 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d18073c30 con 0x7f4d100521b0 2024-04-25T08:31:55.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7f27ba59c640 1 --2- 172.21.15.38:0/3740795466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b403a5e0 0x7f27b4035e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00c530 tx=0x7f27ac00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7f4d0effd640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4cfc06b5d0 0x7f4cfc06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fbdeb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbde40df030 0x7fbde40dcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fbdeb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbde40df030 0x7fbde40dcee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33596/0 (socket says 172.21.15.38:33596) 2024-04-25T08:31:55.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fbdebfff640 1 -- 172.21.15.38:0/3399927178 learned_addr learned my addr 172.21.15.38:0/3399927178 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.331+0000 7f4fc6cc0640 1 -- 172.21.15.38:0/2896487314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc00ce780 con 0x7f4fc00b26f0 2024-04-25T08:31:55.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fb6b170c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac1565f0 0x7fb6ac0b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fb6b170c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac1565f0 0x7fb6ac0b45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:48498/0 (socket says 172.21.15.38:48498) 2024-04-25T08:31:55.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fbdf299d640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde40df410 0x7fbde40dd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fb6b170c640 1 -- 172.21.15.38:0/2499716686 learned_addr learned my addr 172.21.15.38:0/2499716686 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7f6e9bfff640 1 -- 172.21.15.38:0/3091312614 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6eb00bf070 con 0x7f6ea8053ef0 2024-04-25T08:31:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fbdf299d640 1 -- 172.21.15.38:0/3399927178 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40507c0 con 0x7fbde40df030 2024-04-25T08:31:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fbdf299d640 1 -- 172.21.15.38:0/3399927178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde4050640 con 0x7fbde40df410 2024-04-25T08:31:55.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fbdf299d640 1 -- 172.21.15.38:0/3399927178 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40546c0 con 0x7fbde4050140 2024-04-25T08:31:55.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7f4d0effd640 1 -- 172.21.15.38:0/1481308458 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4d181048e0 con 0x7f4d100521b0 2024-04-25T08:31:55.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fb6b270e640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6ac1f3000 0x7fb6ac1ef4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac159dc0 con 0x7fb6ac1569d0 2024-04-25T08:31:55.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac072960 con 0x7fb6ac1565f0 2024-04-25T08:31:55.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac159f40 con 0x7fb6ac1f3000 2024-04-25T08:31:55.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7f6e9bfff640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e9406b520 0x7f6e9406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7f27aaffd640 1 -- 172.21.15.38:0/3740795466 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac017070 con 0x7f27b403a5e0 2024-04-25T08:31:55.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7f27aaffd640 1 -- 172.21.15.38:0/3740795466 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac005ce0 con 0x7f27b403a5e0 2024-04-25T08:31:55.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7f27aaffd640 1 -- 172.21.15.38:0/3740795466 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0072b0 con 0x7f27b403a5e0 2024-04-25T08:31:55.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7f27c0f89640 1 -- 172.21.15.38:0/3740795466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b40d62d0 con 0x7f27b403a5e0 2024-04-25T08:31:55.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.332+0000 7f8f0545d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ef8005e80 0x7f8ef8044090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f27c0f89640 1 -- 172.21.15.38:0/3740795466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b40d6890 con 0x7f27b403a5e0 2024-04-25T08:31:55.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f6e9bfff640 1 -- 172.21.15.38:0/3091312614 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6eb0125be0 con 0x7f6ea8053ef0 2024-04-25T08:31:55.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f8f0545d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef80069d0 0x7f8ef8046740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f4d1e571640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4cfc06b5d0 0x7f4cfc06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f8f0545d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ef80eacf0 0x7f8ef8046c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f6eaf7fe640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e9406b520 0x7f6e9406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fbdf0f3a640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde40df410 0x7fbde40dd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ef8005e80 0x7f8ef8044090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f8efeffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ef8005e80 0x7f8ef8044090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:48518/0 (socket says 172.21.15.38:48518) 2024-04-25T08:31:55.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fb6b0f0b640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6ac1569d0 0x7fb6ac1f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f8f0545d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80547b0 con 0x7f8ef80069d0 2024-04-25T08:31:55.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f8f0545d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef8050190 con 0x7f8ef8005e80 2024-04-25T08:31:55.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f8f0545d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef8054930 con 0x7f8ef80eacf0 2024-04-25T08:31:55.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f8efeffd640 1 -- 172.21.15.38:0/531243314 learned_addr learned my addr 172.21.15.38:0/531243314 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f8eff7fe640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ef80eacf0 0x7f8ef8046c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f4fa3fff640 1 -- 172.21.15.38:0/2896487314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc00ce0b0 con 0x7f4fc00b26f0 2024-04-25T08:31:55.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f8efe7fc640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef80069d0 0x7f8ef8046740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7f6eaf7fe640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e9406b520 0x7f6e9406d9e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c006fd0 tx=0x7f6e9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fbdebfff640 1 -- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde40df410 msgr2=0x7fbde40dd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fbdebfff640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde40df410 0x7fbde40dd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fbdebfff640 1 -- 172.21.15.38:0/3399927178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbde40df030 msgr2=0x7fbde40dcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fbdebfff640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbde40df030 0x7fbde40dcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fbdebfff640 1 -- 172.21.15.38:0/3399927178 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde40dc5b0 con 0x7fbde4050140 2024-04-25T08:31:55.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fbdebfff640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbde4050140 0x7fbde40dc070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbdec050700 tx=0x7fbdec0686a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.334+0000 7f4d1e571640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4cfc06b5d0 0x7f4cfc06da90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d14007fd0 tx=0x7f4d14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.334+0000 7f8efeffd640 1 -- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ef80eacf0 msgr2=0x7f8ef8046c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.334+0000 7f8efeffd640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ef80eacf0 0x7f8ef8046c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.334+0000 7f8efeffd640 1 -- 172.21.15.38:0/531243314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef80069d0 msgr2=0x7f8ef8046740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fb6b0f0b640 1 -- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6ac1f3000 msgr2=0x7fb6ac1ef4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fb6b0f0b640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6ac1f3000 0x7fb6ac1ef4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fb6b0f0b640 1 -- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac1565f0 msgr2=0x7fb6ac0b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fb6b0f0b640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac1565f0 0x7fb6ac0b45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.333+0000 7fb6b0f0b640 1 -- 172.21.15.38:0/2499716686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac1f38b0 con 0x7fb6ac1569d0 2024-04-25T08:31:55.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.334+0000 7fb6b170c640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac1565f0 0x7fb6ac0b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:55.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.334+0000 7f8efeffd640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef80069d0 0x7f8ef8046740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.334+0000 7f8efeffd640 1 -- 172.21.15.38:0/531243314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef80445d0 con 0x7f8ef8005e80 2024-04-25T08:31:55.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.334+0000 7f8eff7fe640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ef80eacf0 0x7f8ef8046c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:55.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.334+0000 7f8efe7fc640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef80069d0 0x7f8ef8046740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:55.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.334+0000 7f8efeffd640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ef8005e80 0x7f8ef8044090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f000743e0 tx=0x7f8f000686a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.334+0000 7fb6b0f0b640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6ac1569d0 0x7fb6ac1f2ac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb69c02f000 tx=0x7fb6ac15e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.335+0000 7f27c0f89640 1 -- 172.21.15.38:0/3740795466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b4005560 con 0x7f27b403a5e0 2024-04-25T08:31:55.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.335+0000 7fbde8ff9640 1 -- 172.21.15.38:0/3399927178 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec06abe0 con 0x7fbde4050140 2024-04-25T08:31:55.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.335+0000 7fbde8ff9640 1 -- 172.21.15.38:0/3399927178 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdec08f020 con 0x7fbde4050140 2024-04-25T08:31:55.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.335+0000 7fbde8ff9640 1 -- 172.21.15.38:0/3399927178 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec06ad80 con 0x7fbde4050140 2024-04-25T08:31:55.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.335+0000 7fbdf299d640 1 -- 172.21.15.38:0/3399927178 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde4176090 con 0x7fbde4050140 2024-04-25T08:31:55.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.335+0000 7f67ebfff640 1 -- 172.21.15.38:0/1978305071 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67fc047000 con 0x7f67fc007190 2024-04-25T08:31:55.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.335+0000 7fbdf299d640 1 -- 172.21.15.38:0/3399927178 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde404cb10 con 0x7fbde4050140 2024-04-25T08:31:55.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.336+0000 7fbdf299d640 1 -- 172.21.15.38:0/3399927178 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde40053c0 con 0x7fbde4050140 2024-04-25T08:31:55.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.337+0000 7f8ee77fe640 1 -- 172.21.15.38:0/531243314 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f000731c0 con 0x7f8ef8005e80 2024-04-25T08:31:55.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.337+0000 7f6e9bfff640 1 -- 172.21.15.38:0/3091312614 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6eb00c6030 con 0x7f6ea8053ef0 2024-04-25T08:31:55.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.337+0000 7f8ee77fe640 1 -- 172.21.15.38:0/531243314 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0008f020 con 0x7f8ef8005e80 2024-04-25T08:31:55.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.337+0000 7f8ee77fe640 1 -- 172.21.15.38:0/531243314 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0006ade0 con 0x7f8ef8005e80 2024-04-25T08:31:55.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.337+0000 7fbde8ff9640 1 -- 172.21.15.38:0/3399927178 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdec073a70 con 0x7fbde4050140 2024-04-25T08:31:55.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.337+0000 7f8f0545d640 1 -- 172.21.15.38:0/531243314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef80455d0 con 0x7f8ef8005e80 2024-04-25T08:31:55.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.337+0000 7f8f0545d640 1 -- 172.21.15.38:0/531243314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef8045b90 con 0x7f8ef8005e80 2024-04-25T08:31:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.337+0000 7fbde8ff9640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbdd006b520 0x7fbdd006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.338+0000 7fbde8ff9640 1 -- 172.21.15.38:0/3399927178 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbdec104790 con 0x7fbde4050140 2024-04-25T08:31:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.338+0000 7fbdeb7fe640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbdd006b520 0x7fbdd006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.338+0000 7fbdeb7fe640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbdd006b520 0x7fbdd006d9e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbde0037000 tx=0x7fbde0030000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.339+0000 7f4d0effd640 1 -- 172.21.15.38:0/1481308458 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d18099030 con 0x7f4d100521b0 2024-04-25T08:31:55.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.339+0000 7f8f0545d640 1 -- 172.21.15.38:0/531243314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef8005380 con 0x7f8ef8005e80 2024-04-25T08:31:55.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.339+0000 7f8ee77fe640 1 -- 172.21.15.38:0/531243314 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f00073a70 con 0x7f8ef8005e80 2024-04-25T08:31:55.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.340+0000 7f8ee77fe640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8eec06b520 0x7f8eec06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.340+0000 7f8ee77fe640 1 -- 172.21.15.38:0/531243314 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8f001047b0 con 0x7f8ef8005e80 2024-04-25T08:31:55.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.341+0000 7f8efe7fc640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8eec06b520 0x7f8eec06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.340+0000 7f4fb67fc640 1 -- 172.21.15.38:0/2896487314 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fb8003970 con 0x7f4fc00b26f0 2024-04-25T08:31:55.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.341+0000 7f4fb67fc640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4fa406b8d0 0x7f4fa406dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.341+0000 7f4fb67fc640 1 -- 172.21.15.38:0/2896487314 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4fb8006b50 con 0x7f4fc00b26f0 2024-04-25T08:31:55.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.341+0000 7f4fc5cbe640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4fa406b8d0 0x7f4fa406dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.341+0000 7fb6a1ffb640 1 -- 172.21.15.38:0/2499716686 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c001630 con 0x7fb6ac1569d0 2024-04-25T08:31:55.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.341+0000 7f27aaffd640 1 -- 172.21.15.38:0/3740795466 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27ac01a070 con 0x7f27b403a5e0 2024-04-25T08:31:55.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.341+0000 7fb6a1ffb640 1 -- 172.21.15.38:0/2499716686 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c042070 con 0x7fb6ac1569d0 2024-04-25T08:31:55.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.341+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac0b0a60 con 0x7fb6ac1569d0 2024-04-25T08:31:55.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.341+0000 7f27aaffd640 1 --2- 172.21.15.38:0/3740795466 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f279c06b520 0x7f279c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.342+0000 7fb6a1ffb640 1 -- 172.21.15.38:0/2499716686 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69c007270 con 0x7fb6ac1569d0 2024-04-25T08:31:55.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.342+0000 7f4fc5cbe640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4fa406b8d0 0x7f4fa406dd90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0006fd0 tx=0x7f4fb0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.342+0000 7f27aaffd640 1 -- 172.21.15.38:0/3740795466 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f27ac089dd0 con 0x7f27b403a5e0 2024-04-25T08:31:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.342+0000 7f8efe7fc640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8eec06b520 0x7f8eec06d9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4007fd0 tx=0x7f8ef4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.342+0000 7f27b9d9b640 1 --2- 172.21.15.38:0/3740795466 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f279c06b520 0x7f279c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.342+0000 7fbde8ff9640 1 -- 172.21.15.38:0/3399927178 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbdec099030 con 0x7fbde4050140 2024-04-25T08:31:55.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.343+0000 7f27b9d9b640 1 --2- 172.21.15.38:0/3740795466 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f279c06b520 0x7f279c06d9e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f27b0000f60 tx=0x7f27b00079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.342+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac0b1020 con 0x7fb6ac1569d0 2024-04-25T08:31:55.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.344+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ac04ef20 con 0x7fb6ac1569d0 2024-04-25T08:31:55.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.344+0000 7f8ee77fe640 1 -- 172.21.15.38:0/531243314 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f00099030 con 0x7f8ef8005e80 2024-04-25T08:31:55.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.346+0000 7f4fb67fc640 1 -- 172.21.15.38:0/2896487314 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fb8020020 con 0x7f4fc00b26f0 2024-04-25T08:31:55.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.352+0000 7f27aaffd640 1 -- 172.21.15.38:0/3740795466 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27ac021020 con 0x7f27b403a5e0 2024-04-25T08:31:55.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.353+0000 7fb6a1ffb640 1 -- 172.21.15.38:0/2499716686 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb69c007410 con 0x7fb6ac1569d0 2024-04-25T08:31:55.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.357+0000 7fb6a1ffb640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb69006b5d0 0x7fb69006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.357+0000 7fb6a1ffb640 1 -- 172.21.15.38:0/2499716686 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb69c0aaff0 con 0x7fb6ac1569d0 2024-04-25T08:31:55.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.359+0000 7fb6a1ffb640 1 -- 172.21.15.38:0/2499716686 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb69c04c020 con 0x7fb6ac1569d0 2024-04-25T08:31:55.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.362+0000 7fb6b170c640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb69006b5d0 0x7fb69006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.371+0000 7fb6b170c640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb69006b5d0 0x7fb69006da90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac09b9c0 tx=0x7fb6a400d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.758+0000 7f6eb5e76640 1 -- 172.21.15.38:0/3091312614 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6ea8007580 con 0x7f6ea8053ef0 2024-04-25T08:31:55.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.759+0000 7f6e9bfff640 1 -- 172.21.15.38:0/3091312614 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6eb00f4700 con 0x7f6ea8053ef0 2024-04-25T08:31:55.755 INFO:teuthology.orchestra.run.smithi038.stdout:55834574859 2024-04-25T08:31:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.762+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e9406b520 msgr2=0x7f6e9406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.762+0000 7f6e99ffb640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e9406b520 0x7f6e9406d9e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c006fd0 tx=0x7f6e9c008040 comp rx=0 tx=0).stop 2024-04-25T08:31:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.762+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ea8053ef0 msgr2=0x7f6ea80d5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.763+0000 7f6e99ffb640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ea8053ef0 0x7f6ea80d5e00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0074b90 tx=0x7f6eb0069b80 comp rx=0 tx=0).stop 2024-04-25T08:31:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.763+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3091312614 shutdown_connections 2024-04-25T08:31:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.763+0000 7f6e99ffb640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ea8053ef0 0x7f6ea80d5e00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.763+0000 7f6e99ffb640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e9406b520 0x7f6e9406d9e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.763+0000 7f6e99ffb640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ea8038330 0x7f6ea804c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.763+0000 7f6e99ffb640 1 --2- 172.21.15.38:0/3091312614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ea8033f80 0x7f6ea804a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.763+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3091312614 >> 172.21.15.38:0/3091312614 conn(0x7f6ea8009fe0 msgr2=0x7f6ea8035c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.763+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3091312614 shutdown_connections 2024-04-25T08:31:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.763+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3091312614 wait complete. 2024-04-25T08:31:55.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.766+0000 7f6803fff640 1 -- 172.21.15.38:0/1978305071 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f67e4005580 con 0x7f67fc007190 2024-04-25T08:31:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.766+0000 7f67ebfff640 1 -- 172.21.15.38:0/1978305071 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f67f4079d30 con 0x7f67fc007190 2024-04-25T08:31:55.763 INFO:teuthology.orchestra.run.smithi038.stdout:55834574858 2024-04-25T08:31:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.768+0000 7fbdf299d640 1 -- 172.21.15.38:0/3399927178 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fbde4053e00 con 0x7fbde4050140 2024-04-25T08:31:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.769+0000 7fbde8ff9640 1 -- 172.21.15.38:0/3399927178 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbde40dc5b0 con 0x7fbde4050140 2024-04-25T08:31:55.765 INFO:teuthology.orchestra.run.smithi038.stdout:55834574857 2024-04-25T08:31:55.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.770+0000 7f6803fff640 1 -- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f67f006b5a0 msgr2=0x7f67f006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.770+0000 7f6803fff640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f67f006b5a0 0x7f67f006da60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6804075a90 tx=0x7f6804068940 comp rx=0 tx=0).stop 2024-04-25T08:31:55.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.770+0000 7f6803fff640 1 -- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67fc007190 msgr2=0x7f67fc035e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.770+0000 7f6803fff640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67fc007190 0x7f67fc035e90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f67fc008680 tx=0x7f67f40046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.770+0000 7f6803fff640 1 -- 172.21.15.38:0/1978305071 shutdown_connections 2024-04-25T08:31:55.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.770+0000 7f6803fff640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67fc0375a0 0x7f67fc0efe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.770+0000 7f6803fff640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67fc033990 0x7f67fc046ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.770+0000 7f6803fff640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f67f006b5a0 0x7f67f006da60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.771+0000 7f6803fff640 1 --2- 172.21.15.38:0/1978305071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67fc007190 0x7f67fc035e90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.771+0000 7f6803fff640 1 -- 172.21.15.38:0/1978305071 >> 172.21.15.38:0/1978305071 conn(0x7f67fc009fb0 msgr2=0x7f67fc0fa230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.771+0000 7f6803fff640 1 -- 172.21.15.38:0/1978305071 shutdown_connections 2024-04-25T08:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.771+0000 7f6803fff640 1 -- 172.21.15.38:0/1978305071 wait complete. 2024-04-25T08:31:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.772+0000 7f8f0545d640 1 -- 172.21.15.38:0/531243314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8ef8034f90 con 0x7f8ef8005e80 2024-04-25T08:31:55.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.773+0000 7fbdd67fc640 1 -- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbdd006b520 msgr2=0x7fbdd006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.773+0000 7fbdd67fc640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbdd006b520 0x7fbdd006d9e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbde0037000 tx=0x7fbde0030000 comp rx=0 tx=0).stop 2024-04-25T08:31:55.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.773+0000 7fbdd67fc640 1 -- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbde4050140 msgr2=0x7fbde40dc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.773+0000 7fbdd67fc640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbde4050140 0x7fbde40dc070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbdec050700 tx=0x7fbdec0686a0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.773+0000 7fbdd67fc640 1 -- 172.21.15.38:0/3399927178 shutdown_connections 2024-04-25T08:31:55.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.773+0000 7fbdd67fc640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbde40df410 0x7fbde40dd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.773+0000 7fbdd67fc640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbde40df030 0x7fbde40dcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.773+0000 7f8ee77fe640 1 -- 172.21.15.38:0/531243314 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8f000d32d0 con 0x7f8ef8005e80 2024-04-25T08:31:55.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.773+0000 7fbdd67fc640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbdd006b520 0x7fbdd006d9e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.773+0000 7fbdd67fc640 1 --2- 172.21.15.38:0/3399927178 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbde4050140 0x7fbde40dc070 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.773+0000 7fbdd67fc640 1 -- 172.21.15.38:0/3399927178 >> 172.21.15.38:0/3399927178 conn(0x7fbde4009fe0 msgr2=0x7fbde4034e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.771 INFO:teuthology.orchestra.run.smithi038.stdout:55834574858 2024-04-25T08:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.774+0000 7fbdd67fc640 1 -- 172.21.15.38:0/3399927178 shutdown_connections 2024-04-25T08:31:55.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.774+0000 7fbdd67fc640 1 -- 172.21.15.38:0/3399927178 wait complete. 2024-04-25T08:31:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f8ee57fa640 1 -- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8eec06b520 msgr2=0x7f8eec06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f8ee57fa640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8eec06b520 0x7f8eec06d9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4007fd0 tx=0x7f8ef4009040 comp rx=0 tx=0).stop 2024-04-25T08:31:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f8ee57fa640 1 -- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ef8005e80 msgr2=0x7f8ef8044090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f8ee57fa640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ef8005e80 0x7f8ef8044090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8f000743e0 tx=0x7f8f000686a0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f4fa3fff640 1 -- 172.21.15.38:0/2896487314 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4fc004ef20 con 0x7f4fc00b26f0 2024-04-25T08:31:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f8ee57fa640 1 -- 172.21.15.38:0/531243314 shutdown_connections 2024-04-25T08:31:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f8ee57fa640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8eec06b520 0x7f8eec06d9e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f8ee57fa640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ef80eacf0 0x7f8ef8046c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f8ee57fa640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ef80069d0 0x7f8ef8046740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f8ee57fa640 1 --2- 172.21.15.38:0/531243314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ef8005e80 0x7f8ef8044090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f8ee57fa640 1 -- 172.21.15.38:0/531243314 >> 172.21.15.38:0/531243314 conn(0x7f8ef8009fa0 msgr2=0x7f8ef8034880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f8ee57fa640 1 -- 172.21.15.38:0/531243314 shutdown_connections 2024-04-25T08:31:55.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.777+0000 7f8ee57fa640 1 -- 172.21.15.38:0/531243314 wait complete. 2024-04-25T08:31:55.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.778+0000 7f4fb67fc640 1 -- 172.21.15.38:0/2896487314 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4fb8020020 con 0x7f4fc00b26f0 2024-04-25T08:31:55.775 INFO:teuthology.orchestra.run.smithi038.stdout:55834574858 2024-04-25T08:31:55.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.780+0000 7f27c0f89640 1 -- 172.21.15.38:0/3740795466 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f27b404e4c0 con 0x7f27b403a5e0 2024-04-25T08:31:55.777 INFO:teuthology.orchestra.run.smithi038.stdout:55834574857 2024-04-25T08:31:55.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.780+0000 7f27aaffd640 1 -- 172.21.15.38:0/3740795466 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f27ac0588f0 con 0x7f27b403a5e0 2024-04-25T08:31:55.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-25T08:31:55.779 DEBUG:teuthology.parallel:result is None 2024-04-25T08:31:55.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.781+0000 7f4fa3fff640 1 -- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4fa406b8d0 msgr2=0x7f4fa406dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.781+0000 7f4fa3fff640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4fa406b8d0 0x7f4fa406dd90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0006fd0 tx=0x7f4fb0008040 comp rx=0 tx=0).stop 2024-04-25T08:31:55.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.782+0000 7f4fa3fff640 1 -- 172.21.15.38:0/2896487314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc00b26f0 msgr2=0x7f4fc00cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.782+0000 7f4fa3fff640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc00b26f0 0x7f4fc00cd5d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800c710 tx=0x7f4fb800cbd0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.782+0000 7f4fa3fff640 1 -- 172.21.15.38:0/2896487314 shutdown_connections 2024-04-25T08:31:55.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.782+0000 7f4fa3fff640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fc00b26f0 0x7f4fc00cd5d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.782+0000 7f4fa3fff640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fc00ae860 0x7f4fc00cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.782+0000 7f4fa3fff640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4fa406b8d0 0x7f4fa406dd90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.782+0000 7f4fa3fff640 1 --2- 172.21.15.38:0/2896487314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fc0067bc0 0x7f4fc0142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.782+0000 7f4fa3fff640 1 -- 172.21.15.38:0/2896487314 >> 172.21.15.38:0/2896487314 conn(0x7f4fc00b1380 msgr2=0x7f4fc00b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.782+0000 7f4fa3fff640 1 -- 172.21.15.38:0/2896487314 shutdown_connections 2024-04-25T08:31:55.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.782+0000 7f4fa3fff640 1 -- 172.21.15.38:0/2896487314 wait complete. 2024-04-25T08:31:55.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.784+0000 7f27c0f89640 1 -- 172.21.15.38:0/3740795466 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f279c06b520 msgr2=0x7f279c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.784+0000 7f27c0f89640 1 --2- 172.21.15.38:0/3740795466 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f279c06b520 0x7f279c06d9e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f27b0000f60 tx=0x7f27b00079b0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.784+0000 7f27c0f89640 1 -- 172.21.15.38:0/3740795466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b403a5e0 msgr2=0x7f27b4035e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.784+0000 7f27c0f89640 1 --2- 172.21.15.38:0/3740795466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b403a5e0 0x7f27b4035e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00c530 tx=0x7f27ac00c9f0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.784+0000 7f27c0f89640 1 -- 172.21.15.38:0/3740795466 shutdown_connections 2024-04-25T08:31:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.784+0000 7f27c0f89640 1 --2- 172.21.15.38:0/3740795466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f27b40e1030 0x7f27b404a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.784+0000 7f27c0f89640 1 --2- 172.21.15.38:0/3740795466 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f279c06b520 0x7f279c06d9e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.784+0000 7f27c0f89640 1 --2- 172.21.15.38:0/3740795466 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27b404eab0 0x7f27b40cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.784+0000 7f27c0f89640 1 --2- 172.21.15.38:0/3740795466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b403a5e0 0x7f27b4035e20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.784+0000 7f27c0f89640 1 -- 172.21.15.38:0/3740795466 >> 172.21.15.38:0/3740795466 conn(0x7f27b4054c90 msgr2=0x7f27b40378b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.784+0000 7f27c0f89640 1 -- 172.21.15.38:0/3740795466 shutdown_connections 2024-04-25T08:31:55.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.784+0000 7f27c0f89640 1 -- 172.21.15.38:0/3740795466 wait complete. 2024-04-25T08:31:55.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.793+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb6ac15c430 con 0x7fb6ac1569d0 2024-04-25T08:31:55.790 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2024-04-25T08:31:55.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.794+0000 7fb6a1ffb640 1 -- 172.21.15.38:0/2499716686 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb69c079b10 con 0x7fb6ac1569d0 2024-04-25T08:31:55.791 INFO:teuthology.orchestra.run.smithi038.stdout:55834574857 2024-04-25T08:31:55.791 DEBUG:teuthology.parallel:result is None 2024-04-25T08:31:55.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-04-25T08:31:55.792 DEBUG:teuthology.parallel:result is None 2024-04-25T08:31:55.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.797+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb69006b5d0 msgr2=0x7fb69006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.797+0000 7fb6b270e640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb69006b5d0 0x7fb69006da90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac09b9c0 tx=0x7fb6a400d040 comp rx=0 tx=0).stop 2024-04-25T08:31:55.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.797+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6ac1569d0 msgr2=0x7fb6ac1f2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.797+0000 7fb6b270e640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6ac1569d0 0x7fb6ac1f2ac0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb69c02f000 tx=0x7fb6ac15e070 comp rx=0 tx=0).stop 2024-04-25T08:31:55.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.797+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 shutdown_connections 2024-04-25T08:31:55.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.797+0000 7fb6b270e640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb69006b5d0 0x7fb69006da90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.797+0000 7fb6b270e640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6ac1f3000 0x7fb6ac1ef4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.797+0000 7fb6b270e640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6ac1569d0 0x7fb6ac1f2ac0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.797+0000 7fb6b270e640 1 --2- 172.21.15.38:0/2499716686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6ac1565f0 0x7fb6ac0b45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.797+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 >> 172.21.15.38:0/2499716686 conn(0x7fb6ac0cbd00 msgr2=0x7fb6ac15c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.797+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 shutdown_connections 2024-04-25T08:31:55.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.797+0000 7fb6b270e640 1 -- 172.21.15.38:0/2499716686 wait complete. 2024-04-25T08:31:55.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-04-25T08:31:55.795 DEBUG:teuthology.parallel:result is None 2024-04-25T08:31:55.796 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.0 2024-04-25T08:31:55.796 DEBUG:teuthology.parallel:result is None 2024-04-25T08:31:55.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2024-04-25T08:31:55.800 DEBUG:teuthology.parallel:result is None 2024-04-25T08:31:55.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.805+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4d00005580 con 0x7f4d100521b0 2024-04-25T08:31:55.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.805+0000 7f4d0effd640 1 -- 172.21.15.38:0/1481308458 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4d180d3490 con 0x7f4d100521b0 2024-04-25T08:31:55.802 INFO:teuthology.orchestra.run.smithi038.stdout:55834574858 2024-04-25T08:31:55.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.807+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4cfc06b5d0 msgr2=0x7f4cfc06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.807+0000 7f4d1f573640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4cfc06b5d0 0x7f4cfc06da90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4d14007fd0 tx=0x7f4d14009040 comp rx=0 tx=0).stop 2024-04-25T08:31:55.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.807+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d100521b0 msgr2=0x7f4d100e2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.807+0000 7f4d1f573640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d100521b0 0x7f4d100e2560 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4d180646b0 tx=0x7f4d180686a0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.807+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 shutdown_connections 2024-04-25T08:31:55.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.807+0000 7f4d1f573640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d100e1050 0x7f4d100dbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.807+0000 7f4d1f573640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d100521b0 0x7f4d100e2560 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.807+0000 7f4d1f573640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4cfc06b5d0 0x7f4cfc06da90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.807+0000 7f4d1f573640 1 --2- 172.21.15.38:0/1481308458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d1003ca10 0x7f4d100e1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.807+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 >> 172.21.15.38:0/1481308458 conn(0x7f4d10056cd0 msgr2=0x7f4d100506e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.807+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 shutdown_connections 2024-04-25T08:31:55.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.807+0000 7f4d1f573640 1 -- 172.21.15.38:0/1481308458 wait complete. 2024-04-25T08:31:55.806 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2024-04-25T08:31:55.806 DEBUG:teuthology.parallel:result is None 2024-04-25T08:31:55.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-04-25T08:31:55.815 DEBUG:teuthology.parallel:result is None 2024-04-25T08:31:55.815 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T08:31:55.815 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:31:55.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.960+0000 7f6a219f9640 1 Processor -- start 2024-04-25T08:31:55.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.960+0000 7f6a219f9640 1 -- start start 2024-04-25T08:31:55.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a219f9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6a1c15dc10 0x7f6a1c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a219f9640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6a1c0ced60 0x7f6a1c0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a219f9640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a1c0d3e70 con 0x7f6a1c0d3900 2024-04-25T08:31:55.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a219f9640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1c15dff0 con 0x7f6a1c15dc10 2024-04-25T08:31:55.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a219f9640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a1c0b6990 con 0x7f6a1c0ced60 2024-04-25T08:31:55.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a1affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6a1c15dc10 0x7f6a1c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a1b7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6a1c0d3900 0x7f6a1c0b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49902/0 (socket says 172.21.15.38:49902) 2024-04-25T08:31:55.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a1b7fe640 1 -- 172.21.15.38:0/3102912407 learned_addr learned my addr 172.21.15.38:0/3102912407 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:55.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a13fff640 1 --2- 172.21.15.38:0/3102912407 >> v2:172.21.15.107:3300/0 conn(0x7f6a1c0ced60 0x7f6a1c0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a1affd640 1 -- 172.21.15.38:0/3102912407 >> v2:172.21.15.107:3300/0 conn(0x7f6a1c0ced60 msgr2=0x7f6a1c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:55.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a1affd640 1 --2- 172.21.15.38:0/3102912407 >> v2:172.21.15.107:3300/0 conn(0x7f6a1c0ced60 0x7f6a1c0d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a1affd640 1 -- 172.21.15.38:0/3102912407 >> v1:172.21.15.38:6789/0 conn(0x7f6a1c0d3900 legacy=0x7f6a1c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.961+0000 7f6a1affd640 1 -- 172.21.15.38:0/3102912407 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c0d4150 con 0x7f6a1c15dc10 2024-04-25T08:31:55.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.962+0000 7f6a1affd640 1 --2- 172.21.15.38:0/3102912407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a1c15dc10 0x7f6a1c0ce820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0b9600 tx=0x7f6a0c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5992fd761c5e242 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.962+0000 7f6a1a7fc640 1 -- 172.21.15.38:0/3102912407 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c037070 con 0x7f6a1c15dc10 2024-04-25T08:31:55.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.962+0000 7f6a1a7fc640 1 -- 172.21.15.38:0/3102912407 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c032440 con 0x7f6a1c15dc10 2024-04-25T08:31:55.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.962+0000 7f6a1a7fc640 1 -- 172.21.15.38:0/3102912407 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c02ae60 con 0x7f6a1c15dc10 2024-04-25T08:31:55.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.963+0000 7f6a219f9640 1 -- 172.21.15.38:0/3102912407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a1c15dc10 msgr2=0x7f6a1c0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.963+0000 7f6a219f9640 1 --2- 172.21.15.38:0/3102912407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a1c15dc10 0x7f6a1c0ce820 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c0b9600 tx=0x7f6a0c02a110 comp rx=0 tx=0).stop 2024-04-25T08:31:55.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.963+0000 7f6a219f9640 1 -- 172.21.15.38:0/3102912407 shutdown_connections 2024-04-25T08:31:55.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.963+0000 7f6a219f9640 1 --2- 172.21.15.38:0/3102912407 >> v2:172.21.15.107:3300/0 conn(0x7f6a1c0ced60 0x7f6a1c0d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.963+0000 7f6a219f9640 1 --2- 172.21.15.38:0/3102912407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a1c15dc10 0x7f6a1c0ce820 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.963+0000 7f6a219f9640 1 -- 172.21.15.38:0/3102912407 >> 172.21.15.38:0/3102912407 conn(0x7f6a1c066d10 msgr2=0x7f6a1c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:55.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.963+0000 7f6a219f9640 1 -- 172.21.15.38:0/3102912407 shutdown_connections 2024-04-25T08:31:55.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.963+0000 7f6a219f9640 1 -- 172.21.15.38:0/3102912407 wait complete. 2024-04-25T08:31:55.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.964+0000 7f6a219f9640 1 Processor -- start 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.964+0000 7f6a219f9640 1 -- start start 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a219f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0ced60 0x7f6a1c1f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a219f9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a1c0d3900 0x7f6a1c154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a219f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a1c15dc10 0x7f6a1c0b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a219f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c0b6bc0 con 0x7f6a1c15dc10 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a219f9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c0d4710 con 0x7f6a1c0ced60 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a219f9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c0d0d90 con 0x7f6a1c0d3900 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a1b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a1c15dc10 0x7f6a1c0b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a1b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a1c15dc10 0x7f6a1c0b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33632/0 (socket says 172.21.15.38:33632) 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a1b7fe640 1 -- 172.21.15.38:0/3174298345 learned_addr learned my addr 172.21.15.38:0/3174298345 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a1affd640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0ced60 0x7f6a1c1f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a13fff640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a1c0d3900 0x7f6a1c154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a1b7fe640 1 -- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a1c0d3900 msgr2=0x7f6a1c154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a1b7fe640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a1c0d3900 0x7f6a1c154880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a1b7fe640 1 -- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0ced60 msgr2=0x7f6a1c1f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a1b7fe640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0ced60 0x7f6a1c1f71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a1b7fe640 1 -- 172.21.15.38:0/3174298345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1c1f7730 con 0x7f6a1c15dc10 2024-04-25T08:31:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a13fff640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a1c0d3900 0x7f6a1c154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a1b7fe640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a1c15dc10 0x7f6a1c0b7560 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400cbc0 tx=0x7f6a1400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a18ff9640 1 -- 172.21.15.38:0/3174298345 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14017070 con 0x7f6a1c15dc10 2024-04-25T08:31:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a18ff9640 1 -- 172.21.15.38:0/3174298345 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1400ad40 con 0x7f6a1c15dc10 2024-04-25T08:31:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a219f9640 1 -- 172.21.15.38:0/3174298345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1c156670 con 0x7f6a1c15dc10 2024-04-25T08:31:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a18ff9640 1 -- 172.21.15.38:0/3174298345 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a14012600 con 0x7f6a1c15dc10 2024-04-25T08:31:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.965+0000 7f6a219f9640 1 -- 172.21.15.38:0/3174298345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c154080 con 0x7f6a1c15dc10 2024-04-25T08:31:55.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.967+0000 7f6a219f9640 1 -- 172.21.15.38:0/3174298345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a1c068e80 con 0x7f6a1c15dc10 2024-04-25T08:31:55.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.967+0000 7f6a18ff9640 1 -- 172.21.15.38:0/3174298345 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a1401a070 con 0x7f6a1c15dc10 2024-04-25T08:31:55.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.967+0000 7f6a18ff9640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f69fc06b4d0 0x7f69fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:55.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.968+0000 7f6a18ff9640 1 -- 172.21.15.38:0/3174298345 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6a1408a1f0 con 0x7f6a1c15dc10 2024-04-25T08:31:55.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.968+0000 7f6a1affd640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f69fc06b4d0 0x7f69fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:55.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.968+0000 7f6a1affd640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f69fc06b4d0 0x7f69fc06d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c034000 tx=0x7f6a0c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:55.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:55.972+0000 7f6a18ff9640 1 -- 172.21.15.38:0/3174298345 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a1c1f7730 con 0x7f6a1c15dc10 2024-04-25T08:31:56.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.151+0000 7f6a219f9640 1 -- 172.21.15.38:0/3174298345 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6a1c0cda00 con 0x7f69fc06b4d0 2024-04-25T08:31:56.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.159+0000 7f6a18ff9640 1 -- 172.21.15.38:0/3174298345 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f6a1c0cda00 con 0x7f69fc06b4d0 2024-04-25T08:31:56.159 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:56.160 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:31:56.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.162+0000 7f6a219f9640 1 -- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f69fc06b4d0 msgr2=0x7f69fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.162+0000 7f6a219f9640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f69fc06b4d0 0x7f69fc06d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c034000 tx=0x7f6a0c02e000 comp rx=0 tx=0).stop 2024-04-25T08:31:56.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.162+0000 7f6a219f9640 1 -- 172.21.15.38:0/3174298345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a1c15dc10 msgr2=0x7f6a1c0b7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.162+0000 7f6a219f9640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a1c15dc10 0x7f6a1c0b7560 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400cbc0 tx=0x7f6a1400a430 comp rx=0 tx=0).stop 2024-04-25T08:31:56.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.162+0000 7f6a219f9640 1 -- 172.21.15.38:0/3174298345 shutdown_connections 2024-04-25T08:31:56.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.162+0000 7f6a219f9640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a1c15dc10 0x7f6a1c0b7560 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.162+0000 7f6a219f9640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f69fc06b4d0 0x7f69fc06d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.162+0000 7f6a219f9640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a1c0d3900 0x7f6a1c154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.162+0000 7f6a219f9640 1 --2- 172.21.15.38:0/3174298345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a1c0ced60 0x7f6a1c1f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.162+0000 7f6a219f9640 1 -- 172.21.15.38:0/3174298345 >> 172.21.15.38:0/3174298345 conn(0x7f6a1c066d10 msgr2=0x7f6a1c160780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:56.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.162+0000 7f6a219f9640 1 -- 172.21.15.38:0/3174298345 shutdown_connections 2024-04-25T08:31:56.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.162+0000 7f6a219f9640 1 -- 172.21.15.38:0/3174298345 wait complete. 2024-04-25T08:31:56.169 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-25T08:31:55.208179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223544,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751176,"statfs":{"total":773094113280,"available":772865204224,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003428"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366309+0000","last_change":"2024-04-25T08:31:35.317991+0000","last_active":"2024-04-25T08:31:42.366309+0000","last_peered":"2024-04-25T08:31:42.366309+0000","last_clean":"2024-04-25T08:31:42.366309+0000","last_became_active":"2024-04-25T08:31:35.317664+0000","last_became_peered":"2024-04-25T08:31:35.317664+0000","last_unstale":"2024-04-25T08:31:42.366309+0000","last_undegraded":"2024-04-25T08:31:42.366309+0000","last_fullsized":"2024-04-25T08:31:42.366309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:56.580134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998066+0000","last_change":"2024-04-25T08:31:36.331619+0000","last_active":"2024-04-25T08:31:41.998066+0000","last_peered":"2024-04-25T08:31:41.998066+0000","last_clean":"2024-04-25T08:31:41.998066+0000","last_became_active":"2024-04-25T08:31:36.331332+0000","last_became_peered":"2024-04-25T08:31:36.331332+0000","last_unstale":"2024-04-25T08:31:41.998066+0000","last_undegraded":"2024-04-25T08:31:41.998066+0000","last_fullsized":"2024-04-25T08:31:41.998066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:12.212803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997585+0000","last_change":"2024-04-25T08:31:40.599503+0000","last_active":"2024-04-25T08:31:41.997585+0000","last_peered":"2024-04-25T08:31:41.997585+0000","last_clean":"2024-04-25T08:31:41.997585+0000","last_became_active":"2024-04-25T08:31:40.599122+0000","last_became_peered":"2024-04-25T08:31:40.599122+0000","last_unstale":"2024-04-25T08:31:41.997585+0000","last_undegraded":"2024-04-25T08:31:41.997585+0000","last_fullsized":"2024-04-25T08:31:41.997585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.735540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998226+0000","last_change":"2024-04-25T08:31:35.318801+0000","last_active":"2024-04-25T08:31:41.998226+0000","last_peered":"2024-04-25T08:31:41.998226+0000","last_clean":"2024-04-25T08:31:41.998226+0000","last_became_active":"2024-04-25T08:31:35.318505+0000","last_became_peered":"2024-04-25T08:31:35.318505+0000","last_unstale":"2024-04-25T08:31:41.998226+0000","last_undegraded":"2024-04-25T08:31:41.998226+0000","last_fullsized":"2024-04-25T08:31:41.998226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365961+0000","last_change":"2024-04-25T08:31:36.323797+0000","last_active":"2024-04-25T08:31:42.365961+0000","last_peered":"2024-04-25T08:31:42.365961+0000","last_clean":"2024-04-25T08:31:42.365961+0000","last_became_active":"2024-04-25T08:31:36.323045+0000","last_became_peered":"2024-04-25T08:31:36.323045+0000","last_unstale":"2024-04-25T08:31:42.365961+0000","last_undegraded":"2024-04-25T08:31:42.365961+0000","last_fullsized":"2024-04-25T08:31:42.365961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:15.226896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381055+0000","last_change":"2024-04-25T08:31:40.356739+0000","last_active":"2024-04-25T08:31:41.381055+0000","last_peered":"2024-04-25T08:31:41.381055+0000","last_clean":"2024-04-25T08:31:41.381055+0000","last_became_active":"2024-04-25T08:31:40.355756+0000","last_became_peered":"2024-04-25T08:31:40.355756+0000","last_unstale":"2024-04-25T08:31:41.381055+0000","last_undegraded":"2024-04-25T08:31:41.381055+0000","last_fullsized":"2024-04-25T08:31:41.381055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:40.045196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422218+0000","last_change":"2024-04-25T08:31:35.316916+0000","last_active":"2024-04-25T08:31:41.422218+0000","last_peered":"2024-04-25T08:31:41.422218+0000","last_clean":"2024-04-25T08:31:41.422218+0000","last_became_active":"2024-04-25T08:31:35.316358+0000","last_became_peered":"2024-04-25T08:31:35.316358+0000","last_unstale":"2024-04-25T08:31:41.422218+0000","last_undegraded":"2024-04-25T08:31:41.422218+0000","last_fullsized":"2024-04-25T08:31:41.422218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:00.982337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119271+0000","last_change":"2024-04-25T08:31:36.323115+0000","last_active":"2024-04-25T08:31:42.119271+0000","last_peered":"2024-04-25T08:31:42.119271+0000","last_clean":"2024-04-25T08:31:42.119271+0000","last_became_active":"2024-04-25T08:31:36.322848+0000","last_became_peered":"2024-04-25T08:31:36.322848+0000","last_unstale":"2024-04-25T08:31:42.119271+0000","last_undegraded":"2024-04-25T08:31:42.119271+0000","last_fullsized":"2024-04-25T08:31:42.119271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382566+0000","last_change":"2024-04-25T08:31:40.357214+0000","last_active":"2024-04-25T08:31:41.382566+0000","last_peered":"2024-04-25T08:31:41.382566+0000","last_clean":"2024-04-25T08:31:41.382566+0000","last_became_active":"2024-04-25T08:31:40.355877+0000","last_became_peered":"2024-04-25T08:31:40.355877+0000","last_unstale":"2024-04-25T08:31:41.382566+0000","last_undegraded":"2024-04-25T08:31:41.382566+0000","last_fullsized":"2024-04-25T08:31:41.382566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422059+0000","last_change":"2024-04-25T08:31:35.319905+0000","last_active":"2024-04-25T08:31:41.422059+0000","last_peered":"2024-04-25T08:31:41.422059+0000","last_clean":"2024-04-25T08:31:41.422059+0000","last_became_active":"2024-04-25T08:31:35.319127+0000","last_became_peered":"2024-04-25T08:31:35.319127+0000","last_unstale":"2024-04-25T08:31:41.422059+0000","last_undegraded":"2024-04-25T08:31:41.422059+0000","last_fullsized":"2024-04-25T08:31:41.422059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:43.779456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119380+0000","last_change":"2024-04-25T08:31:36.322498+0000","last_active":"2024-04-25T08:31:42.119380+0000","last_peered":"2024-04-25T08:31:42.119380+0000","last_clean":"2024-04-25T08:31:42.119380+0000","last_became_active":"2024-04-25T08:31:36.322250+0000","last_became_peered":"2024-04-25T08:31:36.322250+0000","last_unstale":"2024-04-25T08:31:42.119380+0000","last_undegraded":"2024-04-25T08:31:42.119380+0000","last_fullsized":"2024-04-25T08:31:42.119380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.111565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381899+0000","last_change":"2024-04-25T08:31:40.352372+0000","last_active":"2024-04-25T08:31:41.381899+0000","last_peered":"2024-04-25T08:31:41.381899+0000","last_clean":"2024-04-25T08:31:41.381899+0000","last_became_active":"2024-04-25T08:31:40.351860+0000","last_became_peered":"2024-04-25T08:31:40.351860+0000","last_unstale":"2024-04-25T08:31:41.381899+0000","last_undegraded":"2024-04-25T08:31:41.381899+0000","last_fullsized":"2024-04-25T08:31:41.381899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:50.728874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381228+0000","last_change":"2024-04-25T08:31:36.321222+0000","last_active":"2024-04-25T08:31:41.381228+0000","last_peered":"2024-04-25T08:31:41.381228+0000","last_clean":"2024-04-25T08:31:41.381228+0000","last_became_active":"2024-04-25T08:31:36.320947+0000","last_became_peered":"2024-04-25T08:31:36.320947+0000","last_unstale":"2024-04-25T08:31:41.381228+0000","last_undegraded":"2024-04-25T08:31:41.381228+0000","last_fullsized":"2024-04-25T08:31:41.381228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421943+0000","last_change":"2024-04-25T08:31:35.316969+0000","last_active":"2024-04-25T08:31:41.421943+0000","last_peered":"2024-04-25T08:31:41.421943+0000","last_clean":"2024-04-25T08:31:41.421943+0000","last_became_active":"2024-04-25T08:31:35.316693+0000","last_became_peered":"2024-04-25T08:31:35.316693+0000","last_unstale":"2024-04-25T08:31:41.421943+0000","last_undegraded":"2024-04-25T08:31:41.421943+0000","last_fullsized":"2024-04-25T08:31:41.421943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:20.652651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382507+0000","last_change":"2024-04-25T08:31:40.351507+0000","last_active":"2024-04-25T08:31:41.382507+0000","last_peered":"2024-04-25T08:31:41.382507+0000","last_clean":"2024-04-25T08:31:41.382507+0000","last_became_active":"2024-04-25T08:31:40.351053+0000","last_became_peered":"2024-04-25T08:31:40.351053+0000","last_unstale":"2024-04-25T08:31:41.382507+0000","last_undegraded":"2024-04-25T08:31:41.382507+0000","last_fullsized":"2024-04-25T08:31:41.382507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997831+0000","last_change":"2024-04-25T08:31:36.321459+0000","last_active":"2024-04-25T08:31:41.997831+0000","last_peered":"2024-04-25T08:31:41.997831+0000","last_clean":"2024-04-25T08:31:41.997831+0000","last_became_active":"2024-04-25T08:31:36.321137+0000","last_became_peered":"2024-04-25T08:31:36.321137+0000","last_unstale":"2024-04-25T08:31:41.997831+0000","last_undegraded":"2024-04-25T08:31:41.997831+0000","last_fullsized":"2024-04-25T08:31:41.997831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:14.177622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366532+0000","last_change":"2024-04-25T08:31:35.317075+0000","last_active":"2024-04-25T08:31:42.366532+0000","last_peered":"2024-04-25T08:31:42.366532+0000","last_clean":"2024-04-25T08:31:42.366532+0000","last_became_active":"2024-04-25T08:31:35.316764+0000","last_became_peered":"2024-04-25T08:31:35.316764+0000","last_unstale":"2024-04-25T08:31:42.366532+0000","last_undegraded":"2024-04-25T08:31:42.366532+0000","last_fullsized":"2024-04-25T08:31:42.366532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382109+0000","last_change":"2024-04-25T08:31:40.352414+0000","last_active":"2024-04-25T08:31:41.382109+0000","last_peered":"2024-04-25T08:31:41.382109+0000","last_clean":"2024-04-25T08:31:41.382109+0000","last_became_active":"2024-04-25T08:31:40.351024+0000","last_became_peered":"2024-04-25T08:31:40.351024+0000","last_unstale":"2024-04-25T08:31:41.382109+0000","last_undegraded":"2024-04-25T08:31:41.382109+0000","last_fullsized":"2024-04-25T08:31:41.382109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:16.241561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997717+0000","last_change":"2024-04-25T08:31:36.320330+0000","last_active":"2024-04-25T08:31:41.997717+0000","last_peered":"2024-04-25T08:31:41.997717+0000","last_clean":"2024-04-25T08:31:41.997717+0000","last_became_active":"2024-04-25T08:31:36.313715+0000","last_became_peered":"2024-04-25T08:31:36.313715+0000","last_unstale":"2024-04-25T08:31:41.997717+0000","last_undegraded":"2024-04-25T08:31:41.997717+0000","last_fullsized":"2024-04-25T08:31:41.997717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367128+0000","last_change":"2024-04-25T08:31:35.314788+0000","last_active":"2024-04-25T08:31:42.367128+0000","last_peered":"2024-04-25T08:31:42.367128+0000","last_clean":"2024-04-25T08:31:42.367128+0000","last_became_active":"2024-04-25T08:31:35.314353+0000","last_became_peered":"2024-04-25T08:31:35.314353+0000","last_unstale":"2024-04-25T08:31:42.367128+0000","last_undegraded":"2024-04-25T08:31:42.367128+0000","last_fullsized":"2024-04-25T08:31:42.367128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.726901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367441+0000","last_change":"2024-04-25T08:31:40.594652+0000","last_active":"2024-04-25T08:31:42.367441+0000","last_peered":"2024-04-25T08:31:42.367441+0000","last_clean":"2024-04-25T08:31:42.367441+0000","last_became_active":"2024-04-25T08:31:40.594138+0000","last_became_peered":"2024-04-25T08:31:40.594138+0000","last_unstale":"2024-04-25T08:31:42.367441+0000","last_undegraded":"2024-04-25T08:31:42.367441+0000","last_fullsized":"2024-04-25T08:31:42.367441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:23.148774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381474+0000","last_change":"2024-04-25T08:31:36.324636+0000","last_active":"2024-04-25T08:31:41.381474+0000","last_peered":"2024-04-25T08:31:41.381474+0000","last_clean":"2024-04-25T08:31:41.381474+0000","last_became_active":"2024-04-25T08:31:36.322240+0000","last_became_peered":"2024-04-25T08:31:36.322240+0000","last_unstale":"2024-04-25T08:31:41.381474+0000","last_undegraded":"2024-04-25T08:31:41.381474+0000","last_fullsized":"2024-04-25T08:31:41.381474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:49.809945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421616+0000","last_change":"2024-04-25T08:31:35.318327+0000","last_active":"2024-04-25T08:31:41.421616+0000","last_peered":"2024-04-25T08:31:41.421616+0000","last_clean":"2024-04-25T08:31:41.421616+0000","last_became_active":"2024-04-25T08:31:35.317749+0000","last_became_peered":"2024-04-25T08:31:35.317749+0000","last_unstale":"2024-04-25T08:31:41.421616+0000","last_undegraded":"2024-04-25T08:31:41.421616+0000","last_fullsized":"2024-04-25T08:31:41.421616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:46.434047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382327+0000","last_change":"2024-04-25T08:31:40.357220+0000","last_active":"2024-04-25T08:31:41.382327+0000","last_peered":"2024-04-25T08:31:41.382327+0000","last_clean":"2024-04-25T08:31:41.382327+0000","last_became_active":"2024-04-25T08:31:40.355770+0000","last_became_peered":"2024-04-25T08:31:40.355770+0000","last_unstale":"2024-04-25T08:31:41.382327+0000","last_undegraded":"2024-04-25T08:31:41.382327+0000","last_fullsized":"2024-04-25T08:31:41.382327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:04.998392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366774+0000","last_change":"2024-04-25T08:31:36.312756+0000","last_active":"2024-04-25T08:31:42.366774+0000","last_peered":"2024-04-25T08:31:42.366774+0000","last_clean":"2024-04-25T08:31:42.366774+0000","last_became_active":"2024-04-25T08:31:36.312492+0000","last_became_peered":"2024-04-25T08:31:36.312492+0000","last_unstale":"2024-04-25T08:31:42.366774+0000","last_undegraded":"2024-04-25T08:31:42.366774+0000","last_fullsized":"2024-04-25T08:31:42.366774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382375+0000","last_change":"2024-04-25T08:31:35.314188+0000","last_active":"2024-04-25T08:31:41.382375+0000","last_peered":"2024-04-25T08:31:41.382375+0000","last_clean":"2024-04-25T08:31:41.382375+0000","last_became_active":"2024-04-25T08:31:35.313682+0000","last_became_peered":"2024-04-25T08:31:35.313682+0000","last_unstale":"2024-04-25T08:31:41.382375+0000","last_undegraded":"2024-04-25T08:31:41.382375+0000","last_fullsized":"2024-04-25T08:31:41.382375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.726901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381316+0000","last_change":"2024-04-25T08:31:40.596307+0000","last_active":"2024-04-25T08:31:41.381316+0000","last_peered":"2024-04-25T08:31:41.381316+0000","last_clean":"2024-04-25T08:31:41.381316+0000","last_became_active":"2024-04-25T08:31:40.595853+0000","last_became_peered":"2024-04-25T08:31:40.595853+0000","last_unstale":"2024-04-25T08:31:41.381316+0000","last_undegraded":"2024-04-25T08:31:41.381316+0000","last_fullsized":"2024-04-25T08:31:41.381316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:18.206380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998305+0000","last_change":"2024-04-25T08:31:36.327135+0000","last_active":"2024-04-25T08:31:41.998305+0000","last_peered":"2024-04-25T08:31:41.998305+0000","last_clean":"2024-04-25T08:31:41.998305+0000","last_became_active":"2024-04-25T08:31:36.326839+0000","last_became_peered":"2024-04-25T08:31:36.326839+0000","last_unstale":"2024-04-25T08:31:41.998305+0000","last_undegraded":"2024-04-25T08:31:41.998305+0000","last_fullsized":"2024-04-25T08:31:41.998305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:47.399399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367454+0000","last_change":"2024-04-25T08:31:35.309749+0000","last_active":"2024-04-25T08:31:42.367454+0000","last_peered":"2024-04-25T08:31:42.367454+0000","last_clean":"2024-04-25T08:31:42.367454+0000","last_became_active":"2024-04-25T08:31:35.309352+0000","last_became_peered":"2024-04-25T08:31:35.309352+0000","last_unstale":"2024-04-25T08:31:42.367454+0000","last_undegraded":"2024-04-25T08:31:42.367454+0000","last_fullsized":"2024-04-25T08:31:42.367454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:54.173888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997493+0000","last_change":"2024-04-25T08:31:40.598224+0000","last_active":"2024-04-25T08:31:41.997493+0000","last_peered":"2024-04-25T08:31:41.997493+0000","last_clean":"2024-04-25T08:31:41.997493+0000","last_became_active":"2024-04-25T08:31:40.597800+0000","last_became_peered":"2024-04-25T08:31:40.597800+0000","last_unstale":"2024-04-25T08:31:41.997493+0000","last_undegraded":"2024-04-25T08:31:41.997493+0000","last_fullsized":"2024-04-25T08:31:41.997493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:10.902214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367151+0000","last_change":"2024-04-25T08:31:36.325531+0000","last_active":"2024-04-25T08:31:42.367151+0000","last_peered":"2024-04-25T08:31:42.367151+0000","last_clean":"2024-04-25T08:31:42.367151+0000","last_became_active":"2024-04-25T08:31:36.325240+0000","last_became_peered":"2024-04-25T08:31:36.325240+0000","last_unstale":"2024-04-25T08:31:42.367151+0000","last_undegraded":"2024-04-25T08:31:42.367151+0000","last_fullsized":"2024-04-25T08:31:42.367151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422321+0000","last_change":"2024-04-25T08:31:35.317032+0000","last_active":"2024-04-25T08:31:41.422321+0000","last_peered":"2024-04-25T08:31:41.422321+0000","last_clean":"2024-04-25T08:31:41.422321+0000","last_became_active":"2024-04-25T08:31:35.316659+0000","last_became_peered":"2024-04-25T08:31:35.316659+0000","last_unstale":"2024-04-25T08:31:41.422321+0000","last_undegraded":"2024-04-25T08:31:41.422321+0000","last_fullsized":"2024-04-25T08:31:41.422321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:57.762013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366270+0000","last_change":"2024-04-25T08:31:40.595457+0000","last_active":"2024-04-25T08:31:42.366270+0000","last_peered":"2024-04-25T08:31:42.366270+0000","last_clean":"2024-04-25T08:31:42.366270+0000","last_became_active":"2024-04-25T08:31:40.595077+0000","last_became_peered":"2024-04-25T08:31:40.595077+0000","last_unstale":"2024-04-25T08:31:42.366270+0000","last_undegraded":"2024-04-25T08:31:42.366270+0000","last_fullsized":"2024-04-25T08:31:42.366270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119506+0000","last_change":"2024-04-25T08:31:36.322361+0000","last_active":"2024-04-25T08:31:42.119506+0000","last_peered":"2024-04-25T08:31:42.119506+0000","last_clean":"2024-04-25T08:31:42.119506+0000","last_became_active":"2024-04-25T08:31:36.322107+0000","last_became_peered":"2024-04-25T08:31:36.322107+0000","last_unstale":"2024-04-25T08:31:42.119506+0000","last_undegraded":"2024-04-25T08:31:42.119506+0000","last_fullsized":"2024-04-25T08:31:42.119506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:51.984540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367254+0000","last_change":"2024-04-25T08:31:35.315993+0000","last_active":"2024-04-25T08:31:42.367254+0000","last_peered":"2024-04-25T08:31:42.367254+0000","last_clean":"2024-04-25T08:31:42.367254+0000","last_became_active":"2024-04-25T08:31:35.315664+0000","last_became_peered":"2024-04-25T08:31:35.315664+0000","last_unstale":"2024-04-25T08:31:42.367254+0000","last_undegraded":"2024-04-25T08:31:42.367254+0000","last_fullsized":"2024-04-25T08:31:42.367254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365979+0000","last_change":"2024-04-25T08:31:40.355607+0000","last_active":"2024-04-25T08:31:42.365979+0000","last_peered":"2024-04-25T08:31:42.365979+0000","last_clean":"2024-04-25T08:31:42.365979+0000","last_became_active":"2024-04-25T08:31:40.355024+0000","last_became_peered":"2024-04-25T08:31:40.355024+0000","last_unstale":"2024-04-25T08:31:42.365979+0000","last_undegraded":"2024-04-25T08:31:42.365979+0000","last_fullsized":"2024-04-25T08:31:42.365979+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382608+0000","last_change":"2024-04-25T08:31:36.323675+0000","last_active":"2024-04-25T08:31:41.382608+0000","last_peered":"2024-04-25T08:31:41.382608+0000","last_clean":"2024-04-25T08:31:41.382608+0000","last_became_active":"2024-04-25T08:31:36.323102+0000","last_became_peered":"2024-04-25T08:31:36.323102+0000","last_unstale":"2024-04-25T08:31:41.382608+0000","last_undegraded":"2024-04-25T08:31:41.382608+0000","last_fullsized":"2024-04-25T08:31:41.382608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:49.744475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119335+0000","last_change":"2024-04-25T08:31:35.317234+0000","last_active":"2024-04-25T08:31:42.119335+0000","last_peered":"2024-04-25T08:31:42.119335+0000","last_clean":"2024-04-25T08:31:42.119335+0000","last_became_active":"2024-04-25T08:31:35.316747+0000","last_became_peered":"2024-04-25T08:31:35.316747+0000","last_unstale":"2024-04-25T08:31:42.119335+0000","last_undegraded":"2024-04-25T08:31:42.119335+0000","last_fullsized":"2024-04-25T08:31:42.119335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367030+0000","last_change":"2024-04-25T08:31:40.352712+0000","last_active":"2024-04-25T08:31:42.367030+0000","last_peered":"2024-04-25T08:31:42.367030+0000","last_clean":"2024-04-25T08:31:42.367030+0000","last_became_active":"2024-04-25T08:31:40.352328+0000","last_became_peered":"2024-04-25T08:31:40.352328+0000","last_unstale":"2024-04-25T08:31:42.367030+0000","last_undegraded":"2024-04-25T08:31:42.367030+0000","last_fullsized":"2024-04-25T08:31:42.367030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:56.013298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366618+0000","last_change":"2024-04-25T08:31:36.326854+0000","last_active":"2024-04-25T08:31:42.366618+0000","last_peered":"2024-04-25T08:31:42.366618+0000","last_clean":"2024-04-25T08:31:42.366618+0000","last_became_active":"2024-04-25T08:31:36.326564+0000","last_became_peered":"2024-04-25T08:31:36.326564+0000","last_unstale":"2024-04-25T08:31:42.366618+0000","last_undegraded":"2024-04-25T08:31:42.366618+0000","last_fullsized":"2024-04-25T08:31:42.366618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384554+0000","last_change":"2024-04-25T08:31:35.317497+0000","last_active":"2024-04-25T08:31:41.384554+0000","last_peered":"2024-04-25T08:31:41.384554+0000","last_clean":"2024-04-25T08:31:41.384554+0000","last_became_active":"2024-04-25T08:31:35.317202+0000","last_became_peered":"2024-04-25T08:31:35.317202+0000","last_unstale":"2024-04-25T08:31:41.384554+0000","last_undegraded":"2024-04-25T08:31:41.384554+0000","last_fullsized":"2024-04-25T08:31:41.384554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367456+0000","last_change":"2024-04-25T08:31:40.352169+0000","last_active":"2024-04-25T08:31:42.367456+0000","last_peered":"2024-04-25T08:31:42.367456+0000","last_clean":"2024-04-25T08:31:42.367456+0000","last_became_active":"2024-04-25T08:31:40.351743+0000","last_became_peered":"2024-04-25T08:31:40.351743+0000","last_unstale":"2024-04-25T08:31:42.367456+0000","last_undegraded":"2024-04-25T08:31:42.367456+0000","last_fullsized":"2024-04-25T08:31:42.367456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:09.948828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365773+0000","last_change":"2024-04-25T08:31:36.324307+0000","last_active":"2024-04-25T08:31:42.365773+0000","last_peered":"2024-04-25T08:31:42.365773+0000","last_clean":"2024-04-25T08:31:42.365773+0000","last_became_active":"2024-04-25T08:31:36.324011+0000","last_became_peered":"2024-04-25T08:31:36.324011+0000","last_unstale":"2024-04-25T08:31:42.365773+0000","last_undegraded":"2024-04-25T08:31:42.365773+0000","last_fullsized":"2024-04-25T08:31:42.365773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:01.999912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421648+0000","last_change":"2024-04-25T08:31:35.317490+0000","last_active":"2024-04-25T08:31:41.421648+0000","last_peered":"2024-04-25T08:31:41.421648+0000","last_clean":"2024-04-25T08:31:41.421648+0000","last_became_active":"2024-04-25T08:31:35.316569+0000","last_became_peered":"2024-04-25T08:31:35.316569+0000","last_unstale":"2024-04-25T08:31:41.421648+0000","last_undegraded":"2024-04-25T08:31:41.421648+0000","last_fullsized":"2024-04-25T08:31:41.421648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:09.703716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366542+0000","last_change":"2024-04-25T08:31:40.598075+0000","last_active":"2024-04-25T08:31:42.366542+0000","last_peered":"2024-04-25T08:31:42.366542+0000","last_clean":"2024-04-25T08:31:42.366542+0000","last_became_active":"2024-04-25T08:31:40.597669+0000","last_became_peered":"2024-04-25T08:31:40.597669+0000","last_unstale":"2024-04-25T08:31:42.366542+0000","last_undegraded":"2024-04-25T08:31:42.366542+0000","last_fullsized":"2024-04-25T08:31:42.366542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:09.948828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997776+0000","last_change":"2024-04-25T08:31:36.320808+0000","last_active":"2024-04-25T08:31:41.997776+0000","last_peered":"2024-04-25T08:31:41.997776+0000","last_clean":"2024-04-25T08:31:41.997776+0000","last_became_active":"2024-04-25T08:31:36.320489+0000","last_became_peered":"2024-04-25T08:31:36.320489+0000","last_unstale":"2024-04-25T08:31:41.997776+0000","last_undegraded":"2024-04-25T08:31:41.997776+0000","last_fullsized":"2024-04-25T08:31:41.997776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119140+0000","last_change":"2024-04-25T08:31:35.312515+0000","last_active":"2024-04-25T08:31:42.119140+0000","last_peered":"2024-04-25T08:31:42.119140+0000","last_clean":"2024-04-25T08:31:42.119140+0000","last_became_active":"2024-04-25T08:31:35.312156+0000","last_became_peered":"2024-04-25T08:31:35.312156+0000","last_unstale":"2024-04-25T08:31:42.119140+0000","last_undegraded":"2024-04-25T08:31:42.119140+0000","last_fullsized":"2024-04-25T08:31:42.119140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421744+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:31:41.421744+0000","last_peered":"2024-04-25T08:31:41.421744+0000","last_clean":"2024-04-25T08:31:41.421744+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:31:41.421744+0000","last_undegraded":"2024-04-25T08:31:41.421744+0000","last_fullsized":"2024-04-25T08:31:41.421744+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381120+0000","last_change":"2024-04-25T08:31:40.597789+0000","last_active":"2024-04-25T08:31:41.381120+0000","last_peered":"2024-04-25T08:31:41.381120+0000","last_clean":"2024-04-25T08:31:41.381120+0000","last_became_active":"2024-04-25T08:31:40.597373+0000","last_became_peered":"2024-04-25T08:31:40.597373+0000","last_unstale":"2024-04-25T08:31:41.381120+0000","last_undegraded":"2024-04-25T08:31:41.381120+0000","last_fullsized":"2024-04-25T08:31:41.381120+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422089+0000","last_change":"2024-04-25T08:31:36.323510+0000","last_active":"2024-04-25T08:31:41.422089+0000","last_peered":"2024-04-25T08:31:41.422089+0000","last_clean":"2024-04-25T08:31:41.422089+0000","last_became_active":"2024-04-25T08:31:36.322941+0000","last_became_peered":"2024-04-25T08:31:36.322941+0000","last_unstale":"2024-04-25T08:31:41.422089+0000","last_undegraded":"2024-04-25T08:31:41.422089+0000","last_fullsized":"2024-04-25T08:31:41.422089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:51.984540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384803+0000","last_change":"2024-04-25T08:31:35.318779+0000","last_active":"2024-04-25T08:31:41.384803+0000","last_peered":"2024-04-25T08:31:41.384803+0000","last_clean":"2024-04-25T08:31:41.384803+0000","last_became_active":"2024-04-25T08:31:35.318485+0000","last_became_peered":"2024-04-25T08:31:35.318485+0000","last_unstale":"2024-04-25T08:31:41.384803+0000","last_undegraded":"2024-04-25T08:31:41.384803+0000","last_fullsized":"2024-04-25T08:31:41.384803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381168+0000","last_change":"2024-04-25T08:31:40.594633+0000","last_active":"2024-04-25T08:31:41.381168+0000","last_peered":"2024-04-25T08:31:41.381168+0000","last_clean":"2024-04-25T08:31:41.381168+0000","last_became_active":"2024-04-25T08:31:40.594193+0000","last_became_peered":"2024-04-25T08:31:40.594193+0000","last_unstale":"2024-04-25T08:31:41.381168+0000","last_undegraded":"2024-04-25T08:31:41.381168+0000","last_fullsized":"2024-04-25T08:31:41.381168+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119593+0000","last_change":"2024-04-25T08:31:36.327715+0000","last_active":"2024-04-25T08:31:42.119593+0000","last_peered":"2024-04-25T08:31:42.119593+0000","last_clean":"2024-04-25T08:31:42.119593+0000","last_became_active":"2024-04-25T08:31:36.327408+0000","last_became_peered":"2024-04-25T08:31:36.327408+0000","last_unstale":"2024-04-25T08:31:42.119593+0000","last_undegraded":"2024-04-25T08:31:42.119593+0000","last_fullsized":"2024-04-25T08:31:42.119593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:43.612460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367236+0000","last_change":"2024-04-25T08:31:35.310104+0000","last_active":"2024-04-25T08:31:42.367236+0000","last_peered":"2024-04-25T08:31:42.367236+0000","last_clean":"2024-04-25T08:31:42.367236+0000","last_became_active":"2024-04-25T08:31:35.309469+0000","last_became_peered":"2024-04-25T08:31:35.309469+0000","last_unstale":"2024-04-25T08:31:42.367236+0000","last_undegraded":"2024-04-25T08:31:42.367236+0000","last_fullsized":"2024-04-25T08:31:42.367236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:09.703716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366537+0000","last_change":"2024-04-25T08:31:40.356105+0000","last_active":"2024-04-25T08:31:42.366537+0000","last_peered":"2024-04-25T08:31:42.366537+0000","last_clean":"2024-04-25T08:31:42.366537+0000","last_became_active":"2024-04-25T08:31:40.355734+0000","last_became_peered":"2024-04-25T08:31:40.355734+0000","last_unstale":"2024-04-25T08:31:42.366537+0000","last_undegraded":"2024-04-25T08:31:42.366537+0000","last_fullsized":"2024-04-25T08:31:42.366537+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:23.148774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422428+0000","last_change":"2024-04-25T08:31:36.325987+0000","last_active":"2024-04-25T08:31:41.422428+0000","last_peered":"2024-04-25T08:31:41.422428+0000","last_clean":"2024-04-25T08:31:41.422428+0000","last_became_active":"2024-04-25T08:31:36.324131+0000","last_became_peered":"2024-04-25T08:31:36.324131+0000","last_unstale":"2024-04-25T08:31:41.422428+0000","last_undegraded":"2024-04-25T08:31:41.422428+0000","last_fullsized":"2024-04-25T08:31:41.422428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:32.255703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384855+0000","last_change":"2024-04-25T08:31:35.321968+0000","last_active":"2024-04-25T08:31:41.384855+0000","last_peered":"2024-04-25T08:31:41.384855+0000","last_clean":"2024-04-25T08:31:41.384855+0000","last_became_active":"2024-04-25T08:31:35.321609+0000","last_became_peered":"2024-04-25T08:31:35.321609+0000","last_unstale":"2024-04-25T08:31:41.384855+0000","last_undegraded":"2024-04-25T08:31:41.384855+0000","last_fullsized":"2024-04-25T08:31:41.384855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118903+0000","last_change":"2024-04-25T08:31:40.597471+0000","last_active":"2024-04-25T08:31:42.118903+0000","last_peered":"2024-04-25T08:31:42.118903+0000","last_clean":"2024-04-25T08:31:42.118903+0000","last_became_active":"2024-04-25T08:31:40.597017+0000","last_became_peered":"2024-04-25T08:31:40.597017+0000","last_unstale":"2024-04-25T08:31:42.118903+0000","last_undegraded":"2024-04-25T08:31:42.118903+0000","last_fullsized":"2024-04-25T08:31:42.118903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:10.902214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422271+0000","last_change":"2024-04-25T08:31:36.325470+0000","last_active":"2024-04-25T08:31:41.422271+0000","last_peered":"2024-04-25T08:31:41.422271+0000","last_clean":"2024-04-25T08:31:41.422271+0000","last_became_active":"2024-04-25T08:31:36.325177+0000","last_became_peered":"2024-04-25T08:31:36.325177+0000","last_unstale":"2024-04-25T08:31:41.422271+0000","last_undegraded":"2024-04-25T08:31:41.422271+0000","last_fullsized":"2024-04-25T08:31:41.422271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:19.120016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381794+0000","last_change":"2024-04-25T08:31:35.312440+0000","last_active":"2024-04-25T08:31:41.381794+0000","last_peered":"2024-04-25T08:31:41.381794+0000","last_clean":"2024-04-25T08:31:41.381794+0000","last_became_active":"2024-04-25T08:31:35.312130+0000","last_became_peered":"2024-04-25T08:31:35.312130+0000","last_unstale":"2024-04-25T08:31:41.381794+0000","last_undegraded":"2024-04-25T08:31:41.381794+0000","last_fullsized":"2024-04-25T08:31:41.381794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:32.280995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381222+0000","last_change":"2024-04-25T08:31:40.351471+0000","last_active":"2024-04-25T08:31:41.381222+0000","last_peered":"2024-04-25T08:31:41.381222+0000","last_clean":"2024-04-25T08:31:41.381222+0000","last_became_active":"2024-04-25T08:31:40.351036+0000","last_became_peered":"2024-04-25T08:31:40.351036+0000","last_unstale":"2024-04-25T08:31:41.381222+0000","last_undegraded":"2024-04-25T08:31:41.381222+0000","last_fullsized":"2024-04-25T08:31:41.381222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:30.214541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421850+0000","last_change":"2024-04-25T08:31:36.325960+0000","last_active":"2024-04-25T08:31:41.421850+0000","last_peered":"2024-04-25T08:31:41.421850+0000","last_clean":"2024-04-25T08:31:41.421850+0000","last_became_active":"2024-04-25T08:31:36.325667+0000","last_became_peered":"2024-04-25T08:31:36.325667+0000","last_unstale":"2024-04-25T08:31:41.421850+0000","last_undegraded":"2024-04-25T08:31:41.421850+0000","last_fullsized":"2024-04-25T08:31:41.421850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:47.399399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384402+0000","last_change":"2024-04-25T08:31:35.307681+0000","last_active":"2024-04-25T08:31:41.384402+0000","last_peered":"2024-04-25T08:31:41.384402+0000","last_clean":"2024-04-25T08:31:41.384402+0000","last_became_active":"2024-04-25T08:31:35.307213+0000","last_became_peered":"2024-04-25T08:31:35.307213+0000","last_unstale":"2024-04-25T08:31:41.384402+0000","last_undegraded":"2024-04-25T08:31:41.384402+0000","last_fullsized":"2024-04-25T08:31:41.384402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:34.641593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366369+0000","last_change":"2024-04-25T08:31:40.352841+0000","last_active":"2024-04-25T08:31:42.366369+0000","last_peered":"2024-04-25T08:31:42.366369+0000","last_clean":"2024-04-25T08:31:42.366369+0000","last_became_active":"2024-04-25T08:31:40.352418+0000","last_became_peered":"2024-04-25T08:31:40.352418+0000","last_unstale":"2024-04-25T08:31:42.366369+0000","last_undegraded":"2024-04-25T08:31:42.366369+0000","last_fullsized":"2024-04-25T08:31:42.366369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:56.013298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.380985+0000","last_change":"2024-04-25T08:31:36.312845+0000","last_active":"2024-04-25T08:31:41.380985+0000","last_peered":"2024-04-25T08:31:41.380985+0000","last_clean":"2024-04-25T08:31:41.380985+0000","last_became_active":"2024-04-25T08:31:36.312538+0000","last_became_peered":"2024-04-25T08:31:36.312538+0000","last_unstale":"2024-04-25T08:31:41.380985+0000","last_undegraded":"2024-04-25T08:31:41.380985+0000","last_fullsized":"2024-04-25T08:31:41.380985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381431+0000","last_change":"2024-04-25T08:31:35.305736+0000","last_active":"2024-04-25T08:31:41.381431+0000","last_peered":"2024-04-25T08:31:41.381431+0000","last_clean":"2024-04-25T08:31:41.381431+0000","last_became_active":"2024-04-25T08:31:35.305481+0000","last_became_peered":"2024-04-25T08:31:35.305481+0000","last_unstale":"2024-04-25T08:31:41.381431+0000","last_undegraded":"2024-04-25T08:31:41.381431+0000","last_fullsized":"2024-04-25T08:31:41.381431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998328+0000","last_change":"2024-04-25T08:31:40.599441+0000","last_active":"2024-04-25T08:31:41.998328+0000","last_peered":"2024-04-25T08:31:41.998328+0000","last_clean":"2024-04-25T08:31:41.998328+0000","last_became_active":"2024-04-25T08:31:40.599023+0000","last_became_peered":"2024-04-25T08:31:40.599023+0000","last_unstale":"2024-04-25T08:31:41.998328+0000","last_undegraded":"2024-04-25T08:31:41.998328+0000","last_fullsized":"2024-04-25T08:31:41.998328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:09.443843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421936+0000","last_change":"2024-04-25T08:31:36.323783+0000","last_active":"2024-04-25T08:31:41.421936+0000","last_peered":"2024-04-25T08:31:41.421936+0000","last_clean":"2024-04-25T08:31:41.421936+0000","last_became_active":"2024-04-25T08:31:36.323460+0000","last_became_peered":"2024-04-25T08:31:36.323460+0000","last_unstale":"2024-04-25T08:31:41.421936+0000","last_undegraded":"2024-04-25T08:31:41.421936+0000","last_fullsized":"2024-04-25T08:31:41.421936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119196+0000","last_change":"2024-04-25T08:31:35.318378+0000","last_active":"2024-04-25T08:31:42.119196+0000","last_peered":"2024-04-25T08:31:42.119196+0000","last_clean":"2024-04-25T08:31:42.119196+0000","last_became_active":"2024-04-25T08:31:35.318103+0000","last_became_peered":"2024-04-25T08:31:35.318103+0000","last_unstale":"2024-04-25T08:31:42.119196+0000","last_undegraded":"2024-04-25T08:31:42.119196+0000","last_fullsized":"2024-04-25T08:31:42.119196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998302+0000","last_change":"2024-04-25T08:31:40.595629+0000","last_active":"2024-04-25T08:31:41.998302+0000","last_peered":"2024-04-25T08:31:41.998302+0000","last_clean":"2024-04-25T08:31:41.998302+0000","last_became_active":"2024-04-25T08:31:40.594160+0000","last_became_peered":"2024-04-25T08:31:40.594160+0000","last_unstale":"2024-04-25T08:31:41.998302+0000","last_undegraded":"2024-04-25T08:31:41.998302+0000","last_fullsized":"2024-04-25T08:31:41.998302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:00.245750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366188+0000","last_change":"2024-04-25T08:31:36.329619+0000","last_active":"2024-04-25T08:31:42.366188+0000","last_peered":"2024-04-25T08:31:42.366188+0000","last_clean":"2024-04-25T08:31:42.366188+0000","last_became_active":"2024-04-25T08:31:36.329319+0000","last_became_peered":"2024-04-25T08:31:36.329319+0000","last_unstale":"2024-04-25T08:31:42.366188+0000","last_undegraded":"2024-04-25T08:31:42.366188+0000","last_fullsized":"2024-04-25T08:31:42.366188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:32.370451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366817+0000","last_change":"2024-04-25T08:31:35.317195+0000","last_active":"2024-04-25T08:31:42.366817+0000","last_peered":"2024-04-25T08:31:42.366817+0000","last_clean":"2024-04-25T08:31:42.366817+0000","last_became_active":"2024-04-25T08:31:35.316819+0000","last_became_peered":"2024-04-25T08:31:35.316819+0000","last_unstale":"2024-04-25T08:31:42.366817+0000","last_undegraded":"2024-04-25T08:31:42.366817+0000","last_fullsized":"2024-04-25T08:31:42.366817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:32.280995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119540+0000","last_change":"2024-04-25T08:31:40.594853+0000","last_active":"2024-04-25T08:31:42.119540+0000","last_peered":"2024-04-25T08:31:42.119540+0000","last_clean":"2024-04-25T08:31:42.119540+0000","last_became_active":"2024-04-25T08:31:40.594456+0000","last_became_peered":"2024-04-25T08:31:40.594456+0000","last_unstale":"2024-04-25T08:31:42.119540+0000","last_undegraded":"2024-04-25T08:31:42.119540+0000","last_fullsized":"2024-04-25T08:31:42.119540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.735540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422019+0000","last_change":"2024-04-25T08:31:36.324840+0000","last_active":"2024-04-25T08:31:41.422019+0000","last_peered":"2024-04-25T08:31:41.422019+0000","last_clean":"2024-04-25T08:31:41.422019+0000","last_became_active":"2024-04-25T08:31:36.324574+0000","last_became_peered":"2024-04-25T08:31:36.324574+0000","last_unstale":"2024-04-25T08:31:41.422019+0000","last_undegraded":"2024-04-25T08:31:41.422019+0000","last_fullsized":"2024-04-25T08:31:41.422019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.111565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:45.647660+0000","last_change":"2024-04-25T08:31:35.316767+0000","last_active":"2024-04-25T08:31:45.647660+0000","last_peered":"2024-04-25T08:31:45.647660+0000","last_clean":"2024-04-25T08:31:45.647660+0000","last_became_active":"2024-04-25T08:31:35.316484+0000","last_became_peered":"2024-04-25T08:31:35.316484+0000","last_unstale":"2024-04-25T08:31:45.647660+0000","last_undegraded":"2024-04-25T08:31:45.647660+0000","last_fullsized":"2024-04-25T08:31:45.647660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366316+0000","last_change":"2024-04-25T08:31:40.355742+0000","last_active":"2024-04-25T08:31:42.366316+0000","last_peered":"2024-04-25T08:31:42.366316+0000","last_clean":"2024-04-25T08:31:42.366316+0000","last_became_active":"2024-04-25T08:31:40.355357+0000","last_became_peered":"2024-04-25T08:31:40.355357+0000","last_unstale":"2024-04-25T08:31:42.366316+0000","last_undegraded":"2024-04-25T08:31:42.366316+0000","last_fullsized":"2024-04-25T08:31:42.366316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382374+0000","last_change":"2024-04-25T08:31:36.313482+0000","last_active":"2024-04-25T08:31:41.382374+0000","last_peered":"2024-04-25T08:31:41.382374+0000","last_clean":"2024-04-25T08:31:41.382374+0000","last_became_active":"2024-04-25T08:31:36.312650+0000","last_became_peered":"2024-04-25T08:31:36.312650+0000","last_unstale":"2024-04-25T08:31:41.382374+0000","last_undegraded":"2024-04-25T08:31:41.382374+0000","last_fullsized":"2024-04-25T08:31:41.382374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:10.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367199+0000","last_change":"2024-04-25T08:31:35.320355+0000","last_active":"2024-04-25T08:31:42.367199+0000","last_peered":"2024-04-25T08:31:42.367199+0000","last_clean":"2024-04-25T08:31:42.367199+0000","last_became_active":"2024-04-25T08:31:35.320046+0000","last_became_peered":"2024-04-25T08:31:35.320046+0000","last_unstale":"2024-04-25T08:31:42.367199+0000","last_undegraded":"2024-04-25T08:31:42.367199+0000","last_fullsized":"2024-04-25T08:31:42.367199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:46.434047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382283+0000","last_change":"2024-04-25T08:31:40.356433+0000","last_active":"2024-04-25T08:31:41.382283+0000","last_peered":"2024-04-25T08:31:41.382283+0000","last_clean":"2024-04-25T08:31:41.382283+0000","last_became_active":"2024-04-25T08:31:40.355755+0000","last_became_peered":"2024-04-25T08:31:40.355755+0000","last_unstale":"2024-04-25T08:31:41.382283+0000","last_undegraded":"2024-04-25T08:31:41.382283+0000","last_fullsized":"2024-04-25T08:31:41.382283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:51.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366137+0000","last_change":"2024-04-25T08:31:36.323385+0000","last_active":"2024-04-25T08:31:42.366137+0000","last_peered":"2024-04-25T08:31:42.366137+0000","last_clean":"2024-04-25T08:31:42.366137+0000","last_became_active":"2024-04-25T08:31:36.322674+0000","last_became_peered":"2024-04-25T08:31:36.322674+0000","last_unstale":"2024-04-25T08:31:42.366137+0000","last_undegraded":"2024-04-25T08:31:42.366137+0000","last_fullsized":"2024-04-25T08:31:42.366137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119486+0000","last_change":"2024-04-25T08:31:35.317212+0000","last_active":"2024-04-25T08:31:42.119486+0000","last_peered":"2024-04-25T08:31:42.119486+0000","last_clean":"2024-04-25T08:31:42.119486+0000","last_became_active":"2024-04-25T08:31:35.316922+0000","last_became_peered":"2024-04-25T08:31:35.316922+0000","last_unstale":"2024-04-25T08:31:42.119486+0000","last_undegraded":"2024-04-25T08:31:42.119486+0000","last_fullsized":"2024-04-25T08:31:42.119486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.726901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997655+0000","last_change":"2024-04-25T08:31:40.353781+0000","last_active":"2024-04-25T08:31:41.997655+0000","last_peered":"2024-04-25T08:31:41.997655+0000","last_clean":"2024-04-25T08:31:41.997655+0000","last_became_active":"2024-04-25T08:31:40.352269+0000","last_became_peered":"2024-04-25T08:31:40.352269+0000","last_unstale":"2024-04-25T08:31:41.997655+0000","last_undegraded":"2024-04-25T08:31:41.997655+0000","last_fullsized":"2024-04-25T08:31:41.997655+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:23.063716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119364+0000","last_change":"2024-04-25T08:31:36.324056+0000","last_active":"2024-04-25T08:31:42.119364+0000","last_peered":"2024-04-25T08:31:42.119364+0000","last_clean":"2024-04-25T08:31:42.119364+0000","last_became_active":"2024-04-25T08:31:36.323790+0000","last_became_peered":"2024-04-25T08:31:36.323790+0000","last_unstale":"2024-04-25T08:31:42.119364+0000","last_undegraded":"2024-04-25T08:31:42.119364+0000","last_fullsized":"2024-04-25T08:31:42.119364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:19.120016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382208+0000","last_change":"2024-04-25T08:31:35.318093+0000","last_active":"2024-04-25T08:31:41.382208+0000","last_peered":"2024-04-25T08:31:41.382208+0000","last_clean":"2024-04-25T08:31:41.382208+0000","last_became_active":"2024-04-25T08:31:35.317828+0000","last_became_peered":"2024-04-25T08:31:35.317828+0000","last_unstale":"2024-04-25T08:31:41.382208+0000","last_undegraded":"2024-04-25T08:31:41.382208+0000","last_fullsized":"2024-04-25T08:31:41.382208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:54.173888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997667+0000","last_change":"2024-04-25T08:31:40.599421+0000","last_active":"2024-04-25T08:31:41.997667+0000","last_peered":"2024-04-25T08:31:41.997667+0000","last_clean":"2024-04-25T08:31:41.997667+0000","last_became_active":"2024-04-25T08:31:40.599105+0000","last_became_peered":"2024-04-25T08:31:40.599105+0000","last_unstale":"2024-04-25T08:31:41.997667+0000","last_undegraded":"2024-04-25T08:31:41.997667+0000","last_fullsized":"2024-04-25T08:31:41.997667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:50.738838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366324+0000","last_change":"2024-04-25T08:31:40.597001+0000","last_active":"2024-04-25T08:31:42.366324+0000","last_peered":"2024-04-25T08:31:42.366324+0000","last_clean":"2024-04-25T08:31:42.366324+0000","last_became_active":"2024-04-25T08:31:40.596569+0000","last_became_peered":"2024-04-25T08:31:40.596569+0000","last_unstale":"2024-04-25T08:31:42.366324+0000","last_undegraded":"2024-04-25T08:31:42.366324+0000","last_fullsized":"2024-04-25T08:31:42.366324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367007+0000","last_change":"2024-04-25T08:31:36.332669+0000","last_active":"2024-04-25T08:31:42.367007+0000","last_peered":"2024-04-25T08:31:42.367007+0000","last_clean":"2024-04-25T08:31:42.367007+0000","last_became_active":"2024-04-25T08:31:36.332305+0000","last_became_peered":"2024-04-25T08:31:36.332305+0000","last_unstale":"2024-04-25T08:31:42.367007+0000","last_undegraded":"2024-04-25T08:31:42.367007+0000","last_fullsized":"2024-04-25T08:31:42.367007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:15.226896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118969+0000","last_change":"2024-04-25T08:31:35.317959+0000","last_active":"2024-04-25T08:31:42.118969+0000","last_peered":"2024-04-25T08:31:42.118969+0000","last_clean":"2024-04-25T08:31:42.118969+0000","last_became_active":"2024-04-25T08:31:35.317680+0000","last_became_peered":"2024-04-25T08:31:35.317680+0000","last_unstale":"2024-04-25T08:31:42.118969+0000","last_undegraded":"2024-04-25T08:31:42.118969+0000","last_fullsized":"2024-04-25T08:31:42.118969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:54.173888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382101+0000","last_change":"2024-04-25T08:31:40.352855+0000","last_active":"2024-04-25T08:31:41.382101+0000","last_peered":"2024-04-25T08:31:41.382101+0000","last_clean":"2024-04-25T08:31:41.382101+0000","last_became_active":"2024-04-25T08:31:40.352373+0000","last_became_peered":"2024-04-25T08:31:40.352373+0000","last_unstale":"2024-04-25T08:31:41.382101+0000","last_undegraded":"2024-04-25T08:31:41.382101+0000","last_fullsized":"2024-04-25T08:31:41.382101+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:56.013298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997618+0000","last_change":"2024-04-25T08:31:36.327572+0000","last_active":"2024-04-25T08:31:41.997618+0000","last_peered":"2024-04-25T08:31:41.997618+0000","last_clean":"2024-04-25T08:31:41.997618+0000","last_became_active":"2024-04-25T08:31:36.320635+0000","last_became_peered":"2024-04-25T08:31:36.320635+0000","last_unstale":"2024-04-25T08:31:41.997618+0000","last_undegraded":"2024-04-25T08:31:41.997618+0000","last_fullsized":"2024-04-25T08:31:41.997618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997788+0000","last_change":"2024-04-25T08:31:35.317367+0000","last_active":"2024-04-25T08:31:41.997788+0000","last_peered":"2024-04-25T08:31:41.997788+0000","last_clean":"2024-04-25T08:31:41.997788+0000","last_became_active":"2024-04-25T08:31:35.317013+0000","last_became_peered":"2024-04-25T08:31:35.317013+0000","last_unstale":"2024-04-25T08:31:41.997788+0000","last_undegraded":"2024-04-25T08:31:41.997788+0000","last_fullsized":"2024-04-25T08:31:41.997788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:34.641593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381422+0000","last_change":"2024-04-25T08:31:35.305651+0000","last_active":"2024-04-25T08:31:41.381422+0000","last_peered":"2024-04-25T08:31:41.381422+0000","last_clean":"2024-04-25T08:31:41.381422+0000","last_became_active":"2024-04-25T08:31:35.305381+0000","last_became_peered":"2024-04-25T08:31:35.305381+0000","last_unstale":"2024-04-25T08:31:41.381422+0000","last_undegraded":"2024-04-25T08:31:41.381422+0000","last_fullsized":"2024-04-25T08:31:41.381422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366718+0000","last_change":"2024-04-25T08:31:36.323372+0000","last_active":"2024-04-25T08:31:42.366718+0000","last_peered":"2024-04-25T08:31:42.366718+0000","last_clean":"2024-04-25T08:31:42.366718+0000","last_became_active":"2024-04-25T08:31:36.322950+0000","last_became_peered":"2024-04-25T08:31:36.322950+0000","last_unstale":"2024-04-25T08:31:42.366718+0000","last_undegraded":"2024-04-25T08:31:42.366718+0000","last_fullsized":"2024-04-25T08:31:42.366718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384753+0000","last_change":"2024-04-25T08:31:40.353992+0000","last_active":"2024-04-25T08:31:41.384753+0000","last_peered":"2024-04-25T08:31:41.384753+0000","last_clean":"2024-04-25T08:31:41.384753+0000","last_became_active":"2024-04-25T08:31:40.353601+0000","last_became_peered":"2024-04-25T08:31:40.353601+0000","last_unstale":"2024-04-25T08:31:41.384753+0000","last_undegraded":"2024-04-25T08:31:41.384753+0000","last_fullsized":"2024-04-25T08:31:41.384753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:02.808345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997972+0000","last_change":"2024-04-25T08:31:35.321307+0000","last_active":"2024-04-25T08:31:41.997972+0000","last_peered":"2024-04-25T08:31:41.997972+0000","last_clean":"2024-04-25T08:31:41.997972+0000","last_became_active":"2024-04-25T08:31:35.320720+0000","last_became_peered":"2024-04-25T08:31:35.320720+0000","last_unstale":"2024-04-25T08:31:41.997972+0000","last_undegraded":"2024-04-25T08:31:41.997972+0000","last_fullsized":"2024-04-25T08:31:41.997972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381993+0000","last_change":"2024-04-25T08:31:36.317597+0000","last_active":"2024-04-25T08:31:41.381993+0000","last_peered":"2024-04-25T08:31:41.381993+0000","last_clean":"2024-04-25T08:31:41.381993+0000","last_became_active":"2024-04-25T08:31:36.317333+0000","last_became_peered":"2024-04-25T08:31:36.317333+0000","last_unstale":"2024-04-25T08:31:41.381993+0000","last_undegraded":"2024-04-25T08:31:41.381993+0000","last_fullsized":"2024-04-25T08:31:41.381993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:55.191462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998205+0000","last_change":"2024-04-25T08:31:40.357847+0000","last_active":"2024-04-25T08:31:41.998205+0000","last_peered":"2024-04-25T08:31:41.998205+0000","last_clean":"2024-04-25T08:31:41.998205+0000","last_became_active":"2024-04-25T08:31:40.357445+0000","last_became_peered":"2024-04-25T08:31:40.357445+0000","last_unstale":"2024-04-25T08:31:41.998205+0000","last_undegraded":"2024-04-25T08:31:41.998205+0000","last_fullsized":"2024-04-25T08:31:41.998205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:45.166751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381524+0000","last_change":"2024-04-25T08:31:36.310801+0000","last_active":"2024-04-25T08:31:41.381524+0000","last_peered":"2024-04-25T08:31:41.381524+0000","last_clean":"2024-04-25T08:31:41.381524+0000","last_became_active":"2024-04-25T08:31:36.310509+0000","last_became_peered":"2024-04-25T08:31:36.310509+0000","last_unstale":"2024-04-25T08:31:41.381524+0000","last_undegraded":"2024-04-25T08:31:41.381524+0000","last_fullsized":"2024-04-25T08:31:41.381524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384592+0000","last_change":"2024-04-25T08:31:36.320666+0000","last_active":"2024-04-25T08:31:41.384592+0000","last_peered":"2024-04-25T08:31:41.384592+0000","last_clean":"2024-04-25T08:31:41.384592+0000","last_became_active":"2024-04-25T08:31:36.320402+0000","last_became_peered":"2024-04-25T08:31:36.320402+0000","last_unstale":"2024-04-25T08:31:41.384592+0000","last_undegraded":"2024-04-25T08:31:41.384592+0000","last_fullsized":"2024-04-25T08:31:41.384592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:47.451621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119194+0000","last_change":"2024-04-25T08:31:36.323523+0000","last_active":"2024-04-25T08:31:42.119194+0000","last_peered":"2024-04-25T08:31:42.119194+0000","last_clean":"2024-04-25T08:31:42.119194+0000","last_became_active":"2024-04-25T08:31:36.323233+0000","last_became_peered":"2024-04-25T08:31:36.323233+0000","last_unstale":"2024-04-25T08:31:42.119194+0000","last_undegraded":"2024-04-25T08:31:42.119194+0000","last_fullsized":"2024-04-25T08:31:42.119194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:46.700115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366631+0000","last_change":"2024-04-25T08:31:36.332567+0000","last_active":"2024-04-25T08:31:42.366631+0000","last_peered":"2024-04-25T08:31:42.366631+0000","last_clean":"2024-04-25T08:31:42.366631+0000","last_became_active":"2024-04-25T08:31:36.332300+0000","last_became_peered":"2024-04-25T08:31:36.332300+0000","last_unstale":"2024-04-25T08:31:42.366631+0000","last_undegraded":"2024-04-25T08:31:42.366631+0000","last_fullsized":"2024-04-25T08:31:42.366631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:57.933443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998117+0000","last_change":"2024-04-25T08:31:36.326949+0000","last_active":"2024-04-25T08:31:41.998117+0000","last_peered":"2024-04-25T08:31:41.998117+0000","last_clean":"2024-04-25T08:31:41.998117+0000","last_became_active":"2024-04-25T08:31:36.322467+0000","last_became_peered":"2024-04-25T08:31:36.322467+0000","last_unstale":"2024-04-25T08:31:41.998117+0000","last_undegraded":"2024-04-25T08:31:41.998117+0000","last_fullsized":"2024-04-25T08:31:41.998117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:46.700115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367093+0000","last_change":"2024-04-25T08:31:36.331852+0000","last_active":"2024-04-25T08:31:42.367093+0000","last_peered":"2024-04-25T08:31:42.367093+0000","last_clean":"2024-04-25T08:31:42.367093+0000","last_became_active":"2024-04-25T08:31:36.331588+0000","last_became_peered":"2024-04-25T08:31:36.331588+0000","last_unstale":"2024-04-25T08:31:42.367093+0000","last_undegraded":"2024-04-25T08:31:42.367093+0000","last_fullsized":"2024-04-25T08:31:42.367093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381821+0000","last_change":"2024-04-25T08:31:36.309132+0000","last_active":"2024-04-25T08:31:41.381821+0000","last_peered":"2024-04-25T08:31:41.381821+0000","last_clean":"2024-04-25T08:31:41.381821+0000","last_became_active":"2024-04-25T08:31:36.308790+0000","last_became_peered":"2024-04-25T08:31:36.308790+0000","last_unstale":"2024-04-25T08:31:41.381821+0000","last_undegraded":"2024-04-25T08:31:41.381821+0000","last_fullsized":"2024-04-25T08:31:41.381821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:14.177622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998237+0000","last_change":"2024-04-25T08:31:36.320300+0000","last_active":"2024-04-25T08:31:41.998237+0000","last_peered":"2024-04-25T08:31:41.998237+0000","last_clean":"2024-04-25T08:31:41.998237+0000","last_became_active":"2024-04-25T08:31:36.319984+0000","last_became_peered":"2024-04-25T08:31:36.319984+0000","last_unstale":"2024-04-25T08:31:41.998237+0000","last_undegraded":"2024-04-25T08:31:41.998237+0000","last_fullsized":"2024-04-25T08:31:41.998237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118927+0000","last_change":"2024-04-25T08:31:36.329425+0000","last_active":"2024-04-25T08:31:42.118927+0000","last_peered":"2024-04-25T08:31:42.118927+0000","last_clean":"2024-04-25T08:31:42.118927+0000","last_became_active":"2024-04-25T08:31:36.328879+0000","last_became_peered":"2024-04-25T08:31:36.328879+0000","last_unstale":"2024-04-25T08:31:42.118927+0000","last_undegraded":"2024-04-25T08:31:42.118927+0000","last_fullsized":"2024-04-25T08:31:42.118927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997886+0000","last_change":"2024-04-25T08:31:36.322034+0000","last_active":"2024-04-25T08:31:41.997886+0000","last_peered":"2024-04-25T08:31:41.997886+0000","last_clean":"2024-04-25T08:31:41.997886+0000","last_became_active":"2024-04-25T08:31:36.321723+0000","last_became_peered":"2024-04-25T08:31:36.321723+0000","last_unstale":"2024-04-25T08:31:41.997886+0000","last_undegraded":"2024-04-25T08:31:41.997886+0000","last_fullsized":"2024-04-25T08:31:41.997886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119094+0000","last_change":"2024-04-25T08:31:36.325939+0000","last_active":"2024-04-25T08:31:42.119094+0000","last_peered":"2024-04-25T08:31:42.119094+0000","last_clean":"2024-04-25T08:31:42.119094+0000","last_became_active":"2024-04-25T08:31:36.325469+0000","last_became_peered":"2024-04-25T08:31:36.325469+0000","last_unstale":"2024-04-25T08:31:42.119094+0000","last_undegraded":"2024-04-25T08:31:42.119094+0000","last_fullsized":"2024-04-25T08:31:42.119094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:14.177622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366378+0000","last_change":"2024-04-25T08:31:36.328064+0000","last_active":"2024-04-25T08:31:42.366378+0000","last_peered":"2024-04-25T08:31:42.366378+0000","last_clean":"2024-04-25T08:31:42.366378+0000","last_became_active":"2024-04-25T08:31:36.327769+0000","last_became_peered":"2024-04-25T08:31:36.327769+0000","last_unstale":"2024-04-25T08:31:42.366378+0000","last_undegraded":"2024-04-25T08:31:42.366378+0000","last_fullsized":"2024-04-25T08:31:42.366378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:58.779587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381873+0000","last_change":"2024-04-25T08:31:36.318263+0000","last_active":"2024-04-25T08:31:41.381873+0000","last_peered":"2024-04-25T08:31:41.381873+0000","last_clean":"2024-04-25T08:31:41.381873+0000","last_became_active":"2024-04-25T08:31:36.317987+0000","last_became_peered":"2024-04-25T08:31:36.317987+0000","last_unstale":"2024-04-25T08:31:41.381873+0000","last_undegraded":"2024-04-25T08:31:41.381873+0000","last_fullsized":"2024-04-25T08:31:41.381873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422245+0000","last_change":"2024-04-25T08:31:36.318589+0000","last_active":"2024-04-25T08:31:41.422245+0000","last_peered":"2024-04-25T08:31:41.422245+0000","last_clean":"2024-04-25T08:31:41.422245+0000","last_became_active":"2024-04-25T08:31:36.318315+0000","last_became_peered":"2024-04-25T08:31:36.318315+0000","last_unstale":"2024-04-25T08:31:41.422245+0000","last_undegraded":"2024-04-25T08:31:41.422245+0000","last_fullsized":"2024-04-25T08:31:41.422245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:05.920070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384795+0000","last_change":"2024-04-25T08:31:36.320980+0000","last_active":"2024-04-25T08:31:41.384795+0000","last_peered":"2024-04-25T08:31:41.384795+0000","last_clean":"2024-04-25T08:31:41.384795+0000","last_became_active":"2024-04-25T08:31:36.320663+0000","last_became_peered":"2024-04-25T08:31:36.320663+0000","last_unstale":"2024-04-25T08:31:41.384795+0000","last_undegraded":"2024-04-25T08:31:41.384795+0000","last_fullsized":"2024-04-25T08:31:41.384795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:10.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119636+0000","last_change":"2024-04-25T08:31:36.328219+0000","last_active":"2024-04-25T08:31:42.119636+0000","last_peered":"2024-04-25T08:31:42.119636+0000","last_clean":"2024-04-25T08:31:42.119636+0000","last_became_active":"2024-04-25T08:31:36.327914+0000","last_became_peered":"2024-04-25T08:31:36.327914+0000","last_unstale":"2024-04-25T08:31:42.119636+0000","last_undegraded":"2024-04-25T08:31:42.119636+0000","last_fullsized":"2024-04-25T08:31:42.119636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:47.399399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118812+0000","last_change":"2024-04-25T08:31:36.322834+0000","last_active":"2024-04-25T08:31:42.118812+0000","last_peered":"2024-04-25T08:31:42.118812+0000","last_clean":"2024-04-25T08:31:42.118812+0000","last_became_active":"2024-04-25T08:31:36.322555+0000","last_became_peered":"2024-04-25T08:31:36.322555+0000","last_unstale":"2024-04-25T08:31:42.118812+0000","last_undegraded":"2024-04-25T08:31:42.118812+0000","last_fullsized":"2024-04-25T08:31:42.118812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367013+0000","last_change":"2024-04-25T08:31:36.323473+0000","last_active":"2024-04-25T08:31:42.367013+0000","last_peered":"2024-04-25T08:31:42.367013+0000","last_clean":"2024-04-25T08:31:42.367013+0000","last_became_active":"2024-04-25T08:31:36.323180+0000","last_became_peered":"2024-04-25T08:31:36.323180+0000","last_unstale":"2024-04-25T08:31:42.367013+0000","last_undegraded":"2024-04-25T08:31:42.367013+0000","last_fullsized":"2024-04-25T08:31:42.367013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366156+0000","last_change":"2024-04-25T08:31:36.314459+0000","last_active":"2024-04-25T08:31:42.366156+0000","last_peered":"2024-04-25T08:31:42.366156+0000","last_clean":"2024-04-25T08:31:42.366156+0000","last_became_active":"2024-04-25T08:31:36.313669+0000","last_became_peered":"2024-04-25T08:31:36.313669+0000","last_unstale":"2024-04-25T08:31:42.366156+0000","last_undegraded":"2024-04-25T08:31:42.366156+0000","last_fullsized":"2024-04-25T08:31:42.366156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997797+0000","last_change":"2024-04-25T08:31:36.329217+0000","last_active":"2024-04-25T08:31:41.997797+0000","last_peered":"2024-04-25T08:31:41.997797+0000","last_clean":"2024-04-25T08:31:41.997797+0000","last_became_active":"2024-04-25T08:31:36.328793+0000","last_became_peered":"2024-04-25T08:31:36.328793+0000","last_unstale":"2024-04-25T08:31:41.997797+0000","last_undegraded":"2024-04-25T08:31:41.997797+0000","last_fullsized":"2024-04-25T08:31:41.997797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:26.185783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382287+0000","last_change":"2024-04-25T08:31:36.321711+0000","last_active":"2024-04-25T08:31:41.382287+0000","last_peered":"2024-04-25T08:31:41.382287+0000","last_clean":"2024-04-25T08:31:41.382287+0000","last_became_active":"2024-04-25T08:31:36.321446+0000","last_became_peered":"2024-04-25T08:31:36.321446+0000","last_unstale":"2024-04-25T08:31:41.382287+0000","last_undegraded":"2024-04-25T08:31:41.382287+0000","last_fullsized":"2024-04-25T08:31:41.382287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:47.451621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381253+0000","last_change":"2024-04-25T08:31:36.319922+0000","last_active":"2024-04-25T08:31:41.381253+0000","last_peered":"2024-04-25T08:31:41.381253+0000","last_clean":"2024-04-25T08:31:41.381253+0000","last_became_active":"2024-04-25T08:31:36.319650+0000","last_became_peered":"2024-04-25T08:31:36.319650+0000","last_unstale":"2024-04-25T08:31:41.381253+0000","last_undegraded":"2024-04-25T08:31:41.381253+0000","last_fullsized":"2024-04-25T08:31:41.381253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381734+0000","last_change":"2024-04-25T08:31:36.319962+0000","last_active":"2024-04-25T08:31:41.381734+0000","last_peered":"2024-04-25T08:31:41.381734+0000","last_clean":"2024-04-25T08:31:41.381734+0000","last_became_active":"2024-04-25T08:31:36.319634+0000","last_became_peered":"2024-04-25T08:31:36.319634+0000","last_unstale":"2024-04-25T08:31:41.381734+0000","last_undegraded":"2024-04-25T08:31:41.381734+0000","last_fullsized":"2024-04-25T08:31:41.381734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:26.185783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422427+0000","last_change":"2024-04-25T08:31:36.320766+0000","last_active":"2024-04-25T08:31:41.422427+0000","last_peered":"2024-04-25T08:31:41.422427+0000","last_clean":"2024-04-25T08:31:41.422427+0000","last_became_active":"2024-04-25T08:31:36.320483+0000","last_became_peered":"2024-04-25T08:31:36.320483+0000","last_unstale":"2024-04-25T08:31:41.422427+0000","last_undegraded":"2024-04-25T08:31:41.422427+0000","last_fullsized":"2024-04-25T08:31:41.422427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:45.805418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997517+0000","last_change":"2024-04-25T08:31:36.327853+0000","last_active":"2024-04-25T08:31:41.997517+0000","last_peered":"2024-04-25T08:31:41.997517+0000","last_clean":"2024-04-25T08:31:41.997517+0000","last_became_active":"2024-04-25T08:31:36.319850+0000","last_became_peered":"2024-04-25T08:31:36.319850+0000","last_unstale":"2024-04-25T08:31:41.997517+0000","last_undegraded":"2024-04-25T08:31:41.997517+0000","last_fullsized":"2024-04-25T08:31:41.997517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421704+0000","last_change":"2024-04-25T08:31:36.311917+0000","last_active":"2024-04-25T08:31:41.421704+0000","last_peered":"2024-04-25T08:31:41.421704+0000","last_clean":"2024-04-25T08:31:41.421704+0000","last_became_active":"2024-04-25T08:31:36.311657+0000","last_became_peered":"2024-04-25T08:31:36.311657+0000","last_unstale":"2024-04-25T08:31:41.421704+0000","last_undegraded":"2024-04-25T08:31:41.421704+0000","last_fullsized":"2024-04-25T08:31:41.421704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:12.212803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119023+0000","last_change":"2024-04-25T08:31:36.318421+0000","last_active":"2024-04-25T08:31:42.119023+0000","last_peered":"2024-04-25T08:31:42.119023+0000","last_clean":"2024-04-25T08:31:42.119023+0000","last_became_active":"2024-04-25T08:31:36.318157+0000","last_became_peered":"2024-04-25T08:31:36.318157+0000","last_unstale":"2024-04-25T08:31:42.119023+0000","last_undegraded":"2024-04-25T08:31:42.119023+0000","last_fullsized":"2024-04-25T08:31:42.119023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:26.185783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421798+0000","last_change":"2024-04-25T08:31:36.314252+0000","last_active":"2024-04-25T08:31:41.421798+0000","last_peered":"2024-04-25T08:31:41.421798+0000","last_clean":"2024-04-25T08:31:41.421798+0000","last_became_active":"2024-04-25T08:31:36.313732+0000","last_became_peered":"2024-04-25T08:31:36.313732+0000","last_unstale":"2024-04-25T08:31:41.421798+0000","last_undegraded":"2024-04-25T08:31:41.421798+0000","last_fullsized":"2024-04-25T08:31:41.421798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:43.612460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119289+0000","last_change":"2024-04-25T08:31:36.330053+0000","last_active":"2024-04-25T08:31:42.119289+0000","last_peered":"2024-04-25T08:31:42.119289+0000","last_clean":"2024-04-25T08:31:42.119289+0000","last_became_active":"2024-04-25T08:31:36.329775+0000","last_became_peered":"2024-04-25T08:31:36.329775+0000","last_unstale":"2024-04-25T08:31:42.119289+0000","last_undegraded":"2024-04-25T08:31:42.119289+0000","last_fullsized":"2024-04-25T08:31:42.119289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:12.212803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365851+0000","last_change":"2024-04-25T08:31:36.317826+0000","last_active":"2024-04-25T08:31:42.365851+0000","last_peered":"2024-04-25T08:31:42.365851+0000","last_clean":"2024-04-25T08:31:42.365851+0000","last_became_active":"2024-04-25T08:31:36.317514+0000","last_became_peered":"2024-04-25T08:31:36.317514+0000","last_unstale":"2024-04-25T08:31:42.365851+0000","last_undegraded":"2024-04-25T08:31:42.365851+0000","last_fullsized":"2024-04-25T08:31:42.365851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367390+0000","last_change":"2024-04-25T08:31:36.310887+0000","last_active":"2024-04-25T08:31:42.367390+0000","last_peered":"2024-04-25T08:31:42.367390+0000","last_clean":"2024-04-25T08:31:42.367390+0000","last_became_active":"2024-04-25T08:31:36.310581+0000","last_became_peered":"2024-04-25T08:31:36.310581+0000","last_unstale":"2024-04-25T08:31:42.367390+0000","last_undegraded":"2024-04-25T08:31:42.367390+0000","last_fullsized":"2024-04-25T08:31:42.367390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422313+0000","last_change":"2024-04-25T08:31:36.315442+0000","last_active":"2024-04-25T08:31:41.422313+0000","last_peered":"2024-04-25T08:31:41.422313+0000","last_clean":"2024-04-25T08:31:41.422313+0000","last_became_active":"2024-04-25T08:31:36.315178+0000","last_became_peered":"2024-04-25T08:31:36.315178+0000","last_unstale":"2024-04-25T08:31:41.422313+0000","last_undegraded":"2024-04-25T08:31:41.422313+0000","last_fullsized":"2024-04-25T08:31:41.422313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:52.353609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:31:56.174 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:31:56.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.314+0000 7fda61742640 1 Processor -- start 2024-04-25T08:31:56.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.315+0000 7fda61742640 1 -- start start 2024-04-25T08:31:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.315+0000 7fda61742640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fda5c15dc10 0x7fda5c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.315+0000 7fda61742640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0ced60 0x7fda5c0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.315+0000 7fda61742640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fda5c0d3900 0x7fda5c0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.315+0000 7fda61742640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c15dff0 con 0x7fda5c15dc10 2024-04-25T08:31:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.315+0000 7fda61742640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0b6990 con 0x7fda5c0d3900 2024-04-25T08:31:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.315+0000 7fda61742640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0d4710 con 0x7fda5c0ced60 2024-04-25T08:31:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.315+0000 7fda5affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fda5c15dc10 0x7fda5c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.316+0000 7fda5affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fda5c15dc10 0x7fda5c0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33642/0 (socket says 172.21.15.38:33642) 2024-04-25T08:31:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.316+0000 7fda5affd640 1 -- 172.21.15.38:0/3148085198 learned_addr learned my addr 172.21.15.38:0/3148085198 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.316+0000 7fda5b7fe640 1 --2- 172.21.15.38:0/3148085198 >> v2:172.21.15.107:3300/0 conn(0x7fda5c0d3900 0x7fda5c0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.316+0000 7fda5a7fc640 1 --2- 172.21.15.38:0/3148085198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0ced60 0x7fda5c0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.316+0000 7fda5affd640 1 -- 172.21.15.38:0/3148085198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0ced60 msgr2=0x7fda5c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.316+0000 7fda5affd640 1 --2- 172.21.15.38:0/3148085198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0ced60 0x7fda5c0d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.316+0000 7fda5affd640 1 -- 172.21.15.38:0/3148085198 >> v2:172.21.15.107:3300/0 conn(0x7fda5c0d3900 msgr2=0x7fda5c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.316+0000 7fda5affd640 1 --2- 172.21.15.38:0/3148085198 >> v2:172.21.15.107:3300/0 conn(0x7fda5c0d3900 0x7fda5c0d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.316+0000 7fda5affd640 1 -- 172.21.15.38:0/3148085198 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c0d3e70 con 0x7fda5c15dc10 2024-04-25T08:31:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.316+0000 7fda5a7fc640 1 --2- 172.21.15.38:0/3148085198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0ced60 0x7fda5c0d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:56.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda5affd640 1 --2- 172.21.15.38:0/3148085198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c15dc10 0x7fda5c0ce820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0b9380 tx=0x7fda5402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f88f46812aa3cd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda59ffb640 1 -- 172.21.15.38:0/3148085198 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54037070 con 0x7fda5c15dc10 2024-04-25T08:31:56.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda59ffb640 1 -- 172.21.15.38:0/3148085198 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda54032440 con 0x7fda5c15dc10 2024-04-25T08:31:56.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda59ffb640 1 -- 172.21.15.38:0/3148085198 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54032810 con 0x7fda5c15dc10 2024-04-25T08:31:56.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda61742640 1 -- 172.21.15.38:0/3148085198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c15dc10 msgr2=0x7fda5c0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda61742640 1 --2- 172.21.15.38:0/3148085198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c15dc10 0x7fda5c0ce820 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0b9380 tx=0x7fda5402a140 comp rx=0 tx=0).stop 2024-04-25T08:31:56.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda61742640 1 -- 172.21.15.38:0/3148085198 shutdown_connections 2024-04-25T08:31:56.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda61742640 1 --2- 172.21.15.38:0/3148085198 >> v2:172.21.15.107:3300/0 conn(0x7fda5c0d3900 0x7fda5c0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda61742640 1 --2- 172.21.15.38:0/3148085198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0ced60 0x7fda5c0d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda61742640 1 --2- 172.21.15.38:0/3148085198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c15dc10 0x7fda5c0ce820 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda61742640 1 -- 172.21.15.38:0/3148085198 >> 172.21.15.38:0/3148085198 conn(0x7fda5c066d10 msgr2=0x7fda5c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:56.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda61742640 1 -- 172.21.15.38:0/3148085198 shutdown_connections 2024-04-25T08:31:56.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.317+0000 7fda61742640 1 -- 172.21.15.38:0/3148085198 wait complete. 2024-04-25T08:31:56.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.319+0000 7fda61742640 1 Processor -- start 2024-04-25T08:31:56.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.319+0000 7fda61742640 1 -- start start 2024-04-25T08:31:56.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.319+0000 7fda61742640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0ced60 0x7fda5c158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.319+0000 7fda61742640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c0d3900 0x7fda5c0cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.319+0000 7fda61742640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c15dc10 0x7fda5c1f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.319+0000 7fda61742640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0b6bc0 con 0x7fda5c15dc10 2024-04-25T08:31:56.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.319+0000 7fda61742640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0d4430 con 0x7fda5c0d3900 2024-04-25T08:31:56.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.319+0000 7fda61742640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0b2370 con 0x7fda5c0ced60 2024-04-25T08:31:56.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.319+0000 7fda5b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c15dc10 0x7fda5c1f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.319+0000 7fda5b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c15dc10 0x7fda5c1f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33658/0 (socket says 172.21.15.38:33658) 2024-04-25T08:31:56.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.319+0000 7fda5b7fe640 1 -- 172.21.15.38:0/617106043 learned_addr learned my addr 172.21.15.38:0/617106043 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda5affd640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0ced60 0x7fda5c158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda5a7fc640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c0d3900 0x7fda5c0cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda5b7fe640 1 -- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0ced60 msgr2=0x7fda5c158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda5b7fe640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0ced60 0x7fda5c158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda5b7fe640 1 -- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c0d3900 msgr2=0x7fda5c0cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda5b7fe640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c0d3900 0x7fda5c0cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda5b7fe640 1 -- 172.21.15.38:0/617106043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c1590e0 con 0x7fda5c15dc10 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda5affd640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0ced60 0x7fda5c158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda5a7fc640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c0d3900 0x7fda5c0cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda5b7fe640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c15dc10 0x7fda5c1f6750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00efc0 tx=0x7fda4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda537fe640 1 -- 172.21.15.38:0/617106043 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c018020 con 0x7fda5c15dc10 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda537fe640 1 -- 172.21.15.38:0/617106043 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c014480 con 0x7fda5c15dc10 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda61742640 1 -- 172.21.15.38:0/617106043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c1593b0 con 0x7fda5c15dc10 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda537fe640 1 -- 172.21.15.38:0/617106043 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c014870 con 0x7fda5c15dc10 2024-04-25T08:31:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.320+0000 7fda61742640 1 -- 172.21.15.38:0/617106043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c159970 con 0x7fda5c15dc10 2024-04-25T08:31:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.321+0000 7fda61742640 1 -- 172.21.15.38:0/617106043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c068e80 con 0x7fda5c15dc10 2024-04-25T08:31:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.322+0000 7fda537fe640 1 -- 172.21.15.38:0/617106043 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda4c005d10 con 0x7fda5c15dc10 2024-04-25T08:31:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.322+0000 7fda537fe640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda3c06b4d0 0x7fda3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.322+0000 7fda537fe640 1 -- 172.21.15.38:0/617106043 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fda4c08b030 con 0x7fda5c15dc10 2024-04-25T08:31:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.322+0000 7fda5affd640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda3c06b4d0 0x7fda3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.323+0000 7fda5affd640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda3c06b4d0 0x7fda3c06d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fda54034000 tx=0x7fda5402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.326+0000 7fda537fe640 1 -- 172.21.15.38:0/617106043 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda4c022070 con 0x7fda5c15dc10 2024-04-25T08:31:56.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.505+0000 7fda61742640 1 -- 172.21.15.38:0/617106043 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fda5c0b5f10 con 0x7fda3c06b4d0 2024-04-25T08:31:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.513+0000 7fda537fe640 1 -- 172.21.15.38:0/617106043 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7fda5c0b5f10 con 0x7fda3c06b4d0 2024-04-25T08:31:56.513 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:56.515 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:31:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.517+0000 7fda61742640 1 -- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda3c06b4d0 msgr2=0x7fda3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.517+0000 7fda61742640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda3c06b4d0 0x7fda3c06d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fda54034000 tx=0x7fda5402e000 comp rx=0 tx=0).stop 2024-04-25T08:31:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.517+0000 7fda61742640 1 -- 172.21.15.38:0/617106043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c15dc10 msgr2=0x7fda5c1f6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.517+0000 7fda61742640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c15dc10 0x7fda5c1f6750 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00efc0 tx=0x7fda4c00c490 comp rx=0 tx=0).stop 2024-04-25T08:31:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.517+0000 7fda61742640 1 -- 172.21.15.38:0/617106043 shutdown_connections 2024-04-25T08:31:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.517+0000 7fda61742640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c15dc10 0x7fda5c1f6750 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.517+0000 7fda61742640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c0d3900 0x7fda5c0cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.517+0000 7fda61742640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda3c06b4d0 0x7fda3c06d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.517+0000 7fda61742640 1 --2- 172.21.15.38:0/617106043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0ced60 0x7fda5c158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.517+0000 7fda61742640 1 -- 172.21.15.38:0/617106043 >> 172.21.15.38:0/617106043 conn(0x7fda5c066d10 msgr2=0x7fda5c15cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:56.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.517+0000 7fda61742640 1 -- 172.21.15.38:0/617106043 shutdown_connections 2024-04-25T08:31:56.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.517+0000 7fda61742640 1 -- 172.21.15.38:0/617106043 wait complete. 2024-04-25T08:31:56.524 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-25T08:31:55.208179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223544,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751176,"statfs":{"total":773094113280,"available":772865204224,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003428"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366309+0000","last_change":"2024-04-25T08:31:35.317991+0000","last_active":"2024-04-25T08:31:42.366309+0000","last_peered":"2024-04-25T08:31:42.366309+0000","last_clean":"2024-04-25T08:31:42.366309+0000","last_became_active":"2024-04-25T08:31:35.317664+0000","last_became_peered":"2024-04-25T08:31:35.317664+0000","last_unstale":"2024-04-25T08:31:42.366309+0000","last_undegraded":"2024-04-25T08:31:42.366309+0000","last_fullsized":"2024-04-25T08:31:42.366309+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:56.580134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998066+0000","last_change":"2024-04-25T08:31:36.331619+0000","last_active":"2024-04-25T08:31:41.998066+0000","last_peered":"2024-04-25T08:31:41.998066+0000","last_clean":"2024-04-25T08:31:41.998066+0000","last_became_active":"2024-04-25T08:31:36.331332+0000","last_became_peered":"2024-04-25T08:31:36.331332+0000","last_unstale":"2024-04-25T08:31:41.998066+0000","last_undegraded":"2024-04-25T08:31:41.998066+0000","last_fullsized":"2024-04-25T08:31:41.998066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:12.212803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997585+0000","last_change":"2024-04-25T08:31:40.599503+0000","last_active":"2024-04-25T08:31:41.997585+0000","last_peered":"2024-04-25T08:31:41.997585+0000","last_clean":"2024-04-25T08:31:41.997585+0000","last_became_active":"2024-04-25T08:31:40.599122+0000","last_became_peered":"2024-04-25T08:31:40.599122+0000","last_unstale":"2024-04-25T08:31:41.997585+0000","last_undegraded":"2024-04-25T08:31:41.997585+0000","last_fullsized":"2024-04-25T08:31:41.997585+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.735540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998226+0000","last_change":"2024-04-25T08:31:35.318801+0000","last_active":"2024-04-25T08:31:41.998226+0000","last_peered":"2024-04-25T08:31:41.998226+0000","last_clean":"2024-04-25T08:31:41.998226+0000","last_became_active":"2024-04-25T08:31:35.318505+0000","last_became_peered":"2024-04-25T08:31:35.318505+0000","last_unstale":"2024-04-25T08:31:41.998226+0000","last_undegraded":"2024-04-25T08:31:41.998226+0000","last_fullsized":"2024-04-25T08:31:41.998226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365961+0000","last_change":"2024-04-25T08:31:36.323797+0000","last_active":"2024-04-25T08:31:42.365961+0000","last_peered":"2024-04-25T08:31:42.365961+0000","last_clean":"2024-04-25T08:31:42.365961+0000","last_became_active":"2024-04-25T08:31:36.323045+0000","last_became_peered":"2024-04-25T08:31:36.323045+0000","last_unstale":"2024-04-25T08:31:42.365961+0000","last_undegraded":"2024-04-25T08:31:42.365961+0000","last_fullsized":"2024-04-25T08:31:42.365961+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:15.226896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381055+0000","last_change":"2024-04-25T08:31:40.356739+0000","last_active":"2024-04-25T08:31:41.381055+0000","last_peered":"2024-04-25T08:31:41.381055+0000","last_clean":"2024-04-25T08:31:41.381055+0000","last_became_active":"2024-04-25T08:31:40.355756+0000","last_became_peered":"2024-04-25T08:31:40.355756+0000","last_unstale":"2024-04-25T08:31:41.381055+0000","last_undegraded":"2024-04-25T08:31:41.381055+0000","last_fullsized":"2024-04-25T08:31:41.381055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:40.045196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422218+0000","last_change":"2024-04-25T08:31:35.316916+0000","last_active":"2024-04-25T08:31:41.422218+0000","last_peered":"2024-04-25T08:31:41.422218+0000","last_clean":"2024-04-25T08:31:41.422218+0000","last_became_active":"2024-04-25T08:31:35.316358+0000","last_became_peered":"2024-04-25T08:31:35.316358+0000","last_unstale":"2024-04-25T08:31:41.422218+0000","last_undegraded":"2024-04-25T08:31:41.422218+0000","last_fullsized":"2024-04-25T08:31:41.422218+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:00.982337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119271+0000","last_change":"2024-04-25T08:31:36.323115+0000","last_active":"2024-04-25T08:31:42.119271+0000","last_peered":"2024-04-25T08:31:42.119271+0000","last_clean":"2024-04-25T08:31:42.119271+0000","last_became_active":"2024-04-25T08:31:36.322848+0000","last_became_peered":"2024-04-25T08:31:36.322848+0000","last_unstale":"2024-04-25T08:31:42.119271+0000","last_undegraded":"2024-04-25T08:31:42.119271+0000","last_fullsized":"2024-04-25T08:31:42.119271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382566+0000","last_change":"2024-04-25T08:31:40.357214+0000","last_active":"2024-04-25T08:31:41.382566+0000","last_peered":"2024-04-25T08:31:41.382566+0000","last_clean":"2024-04-25T08:31:41.382566+0000","last_became_active":"2024-04-25T08:31:40.355877+0000","last_became_peered":"2024-04-25T08:31:40.355877+0000","last_unstale":"2024-04-25T08:31:41.382566+0000","last_undegraded":"2024-04-25T08:31:41.382566+0000","last_fullsized":"2024-04-25T08:31:41.382566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422059+0000","last_change":"2024-04-25T08:31:35.319905+0000","last_active":"2024-04-25T08:31:41.422059+0000","last_peered":"2024-04-25T08:31:41.422059+0000","last_clean":"2024-04-25T08:31:41.422059+0000","last_became_active":"2024-04-25T08:31:35.319127+0000","last_became_peered":"2024-04-25T08:31:35.319127+0000","last_unstale":"2024-04-25T08:31:41.422059+0000","last_undegraded":"2024-04-25T08:31:41.422059+0000","last_fullsized":"2024-04-25T08:31:41.422059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:43.779456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119380+0000","last_change":"2024-04-25T08:31:36.322498+0000","last_active":"2024-04-25T08:31:42.119380+0000","last_peered":"2024-04-25T08:31:42.119380+0000","last_clean":"2024-04-25T08:31:42.119380+0000","last_became_active":"2024-04-25T08:31:36.322250+0000","last_became_peered":"2024-04-25T08:31:36.322250+0000","last_unstale":"2024-04-25T08:31:42.119380+0000","last_undegraded":"2024-04-25T08:31:42.119380+0000","last_fullsized":"2024-04-25T08:31:42.119380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.111565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381899+0000","last_change":"2024-04-25T08:31:40.352372+0000","last_active":"2024-04-25T08:31:41.381899+0000","last_peered":"2024-04-25T08:31:41.381899+0000","last_clean":"2024-04-25T08:31:41.381899+0000","last_became_active":"2024-04-25T08:31:40.351860+0000","last_became_peered":"2024-04-25T08:31:40.351860+0000","last_unstale":"2024-04-25T08:31:41.381899+0000","last_undegraded":"2024-04-25T08:31:41.381899+0000","last_fullsized":"2024-04-25T08:31:41.381899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:50.728874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381228+0000","last_change":"2024-04-25T08:31:36.321222+0000","last_active":"2024-04-25T08:31:41.381228+0000","last_peered":"2024-04-25T08:31:41.381228+0000","last_clean":"2024-04-25T08:31:41.381228+0000","last_became_active":"2024-04-25T08:31:36.320947+0000","last_became_peered":"2024-04-25T08:31:36.320947+0000","last_unstale":"2024-04-25T08:31:41.381228+0000","last_undegraded":"2024-04-25T08:31:41.381228+0000","last_fullsized":"2024-04-25T08:31:41.381228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421943+0000","last_change":"2024-04-25T08:31:35.316969+0000","last_active":"2024-04-25T08:31:41.421943+0000","last_peered":"2024-04-25T08:31:41.421943+0000","last_clean":"2024-04-25T08:31:41.421943+0000","last_became_active":"2024-04-25T08:31:35.316693+0000","last_became_peered":"2024-04-25T08:31:35.316693+0000","last_unstale":"2024-04-25T08:31:41.421943+0000","last_undegraded":"2024-04-25T08:31:41.421943+0000","last_fullsized":"2024-04-25T08:31:41.421943+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:20.652651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382507+0000","last_change":"2024-04-25T08:31:40.351507+0000","last_active":"2024-04-25T08:31:41.382507+0000","last_peered":"2024-04-25T08:31:41.382507+0000","last_clean":"2024-04-25T08:31:41.382507+0000","last_became_active":"2024-04-25T08:31:40.351053+0000","last_became_peered":"2024-04-25T08:31:40.351053+0000","last_unstale":"2024-04-25T08:31:41.382507+0000","last_undegraded":"2024-04-25T08:31:41.382507+0000","last_fullsized":"2024-04-25T08:31:41.382507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997831+0000","last_change":"2024-04-25T08:31:36.321459+0000","last_active":"2024-04-25T08:31:41.997831+0000","last_peered":"2024-04-25T08:31:41.997831+0000","last_clean":"2024-04-25T08:31:41.997831+0000","last_became_active":"2024-04-25T08:31:36.321137+0000","last_became_peered":"2024-04-25T08:31:36.321137+0000","last_unstale":"2024-04-25T08:31:41.997831+0000","last_undegraded":"2024-04-25T08:31:41.997831+0000","last_fullsized":"2024-04-25T08:31:41.997831+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:14.177622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366532+0000","last_change":"2024-04-25T08:31:35.317075+0000","last_active":"2024-04-25T08:31:42.366532+0000","last_peered":"2024-04-25T08:31:42.366532+0000","last_clean":"2024-04-25T08:31:42.366532+0000","last_became_active":"2024-04-25T08:31:35.316764+0000","last_became_peered":"2024-04-25T08:31:35.316764+0000","last_unstale":"2024-04-25T08:31:42.366532+0000","last_undegraded":"2024-04-25T08:31:42.366532+0000","last_fullsized":"2024-04-25T08:31:42.366532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382109+0000","last_change":"2024-04-25T08:31:40.352414+0000","last_active":"2024-04-25T08:31:41.382109+0000","last_peered":"2024-04-25T08:31:41.382109+0000","last_clean":"2024-04-25T08:31:41.382109+0000","last_became_active":"2024-04-25T08:31:40.351024+0000","last_became_peered":"2024-04-25T08:31:40.351024+0000","last_unstale":"2024-04-25T08:31:41.382109+0000","last_undegraded":"2024-04-25T08:31:41.382109+0000","last_fullsized":"2024-04-25T08:31:41.382109+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:16.241561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997717+0000","last_change":"2024-04-25T08:31:36.320330+0000","last_active":"2024-04-25T08:31:41.997717+0000","last_peered":"2024-04-25T08:31:41.997717+0000","last_clean":"2024-04-25T08:31:41.997717+0000","last_became_active":"2024-04-25T08:31:36.313715+0000","last_became_peered":"2024-04-25T08:31:36.313715+0000","last_unstale":"2024-04-25T08:31:41.997717+0000","last_undegraded":"2024-04-25T08:31:41.997717+0000","last_fullsized":"2024-04-25T08:31:41.997717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367128+0000","last_change":"2024-04-25T08:31:35.314788+0000","last_active":"2024-04-25T08:31:42.367128+0000","last_peered":"2024-04-25T08:31:42.367128+0000","last_clean":"2024-04-25T08:31:42.367128+0000","last_became_active":"2024-04-25T08:31:35.314353+0000","last_became_peered":"2024-04-25T08:31:35.314353+0000","last_unstale":"2024-04-25T08:31:42.367128+0000","last_undegraded":"2024-04-25T08:31:42.367128+0000","last_fullsized":"2024-04-25T08:31:42.367128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.726901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367441+0000","last_change":"2024-04-25T08:31:40.594652+0000","last_active":"2024-04-25T08:31:42.367441+0000","last_peered":"2024-04-25T08:31:42.367441+0000","last_clean":"2024-04-25T08:31:42.367441+0000","last_became_active":"2024-04-25T08:31:40.594138+0000","last_became_peered":"2024-04-25T08:31:40.594138+0000","last_unstale":"2024-04-25T08:31:42.367441+0000","last_undegraded":"2024-04-25T08:31:42.367441+0000","last_fullsized":"2024-04-25T08:31:42.367441+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:23.148774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381474+0000","last_change":"2024-04-25T08:31:36.324636+0000","last_active":"2024-04-25T08:31:41.381474+0000","last_peered":"2024-04-25T08:31:41.381474+0000","last_clean":"2024-04-25T08:31:41.381474+0000","last_became_active":"2024-04-25T08:31:36.322240+0000","last_became_peered":"2024-04-25T08:31:36.322240+0000","last_unstale":"2024-04-25T08:31:41.381474+0000","last_undegraded":"2024-04-25T08:31:41.381474+0000","last_fullsized":"2024-04-25T08:31:41.381474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:49.809945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421616+0000","last_change":"2024-04-25T08:31:35.318327+0000","last_active":"2024-04-25T08:31:41.421616+0000","last_peered":"2024-04-25T08:31:41.421616+0000","last_clean":"2024-04-25T08:31:41.421616+0000","last_became_active":"2024-04-25T08:31:35.317749+0000","last_became_peered":"2024-04-25T08:31:35.317749+0000","last_unstale":"2024-04-25T08:31:41.421616+0000","last_undegraded":"2024-04-25T08:31:41.421616+0000","last_fullsized":"2024-04-25T08:31:41.421616+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:46.434047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382327+0000","last_change":"2024-04-25T08:31:40.357220+0000","last_active":"2024-04-25T08:31:41.382327+0000","last_peered":"2024-04-25T08:31:41.382327+0000","last_clean":"2024-04-25T08:31:41.382327+0000","last_became_active":"2024-04-25T08:31:40.355770+0000","last_became_peered":"2024-04-25T08:31:40.355770+0000","last_unstale":"2024-04-25T08:31:41.382327+0000","last_undegraded":"2024-04-25T08:31:41.382327+0000","last_fullsized":"2024-04-25T08:31:41.382327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:04.998392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366774+0000","last_change":"2024-04-25T08:31:36.312756+0000","last_active":"2024-04-25T08:31:42.366774+0000","last_peered":"2024-04-25T08:31:42.366774+0000","last_clean":"2024-04-25T08:31:42.366774+0000","last_became_active":"2024-04-25T08:31:36.312492+0000","last_became_peered":"2024-04-25T08:31:36.312492+0000","last_unstale":"2024-04-25T08:31:42.366774+0000","last_undegraded":"2024-04-25T08:31:42.366774+0000","last_fullsized":"2024-04-25T08:31:42.366774+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382375+0000","last_change":"2024-04-25T08:31:35.314188+0000","last_active":"2024-04-25T08:31:41.382375+0000","last_peered":"2024-04-25T08:31:41.382375+0000","last_clean":"2024-04-25T08:31:41.382375+0000","last_became_active":"2024-04-25T08:31:35.313682+0000","last_became_peered":"2024-04-25T08:31:35.313682+0000","last_unstale":"2024-04-25T08:31:41.382375+0000","last_undegraded":"2024-04-25T08:31:41.382375+0000","last_fullsized":"2024-04-25T08:31:41.382375+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.726901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381316+0000","last_change":"2024-04-25T08:31:40.596307+0000","last_active":"2024-04-25T08:31:41.381316+0000","last_peered":"2024-04-25T08:31:41.381316+0000","last_clean":"2024-04-25T08:31:41.381316+0000","last_became_active":"2024-04-25T08:31:40.595853+0000","last_became_peered":"2024-04-25T08:31:40.595853+0000","last_unstale":"2024-04-25T08:31:41.381316+0000","last_undegraded":"2024-04-25T08:31:41.381316+0000","last_fullsized":"2024-04-25T08:31:41.381316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:18.206380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998305+0000","last_change":"2024-04-25T08:31:36.327135+0000","last_active":"2024-04-25T08:31:41.998305+0000","last_peered":"2024-04-25T08:31:41.998305+0000","last_clean":"2024-04-25T08:31:41.998305+0000","last_became_active":"2024-04-25T08:31:36.326839+0000","last_became_peered":"2024-04-25T08:31:36.326839+0000","last_unstale":"2024-04-25T08:31:41.998305+0000","last_undegraded":"2024-04-25T08:31:41.998305+0000","last_fullsized":"2024-04-25T08:31:41.998305+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:47.399399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367454+0000","last_change":"2024-04-25T08:31:35.309749+0000","last_active":"2024-04-25T08:31:42.367454+0000","last_peered":"2024-04-25T08:31:42.367454+0000","last_clean":"2024-04-25T08:31:42.367454+0000","last_became_active":"2024-04-25T08:31:35.309352+0000","last_became_peered":"2024-04-25T08:31:35.309352+0000","last_unstale":"2024-04-25T08:31:42.367454+0000","last_undegraded":"2024-04-25T08:31:42.367454+0000","last_fullsized":"2024-04-25T08:31:42.367454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:54.173888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997493+0000","last_change":"2024-04-25T08:31:40.598224+0000","last_active":"2024-04-25T08:31:41.997493+0000","last_peered":"2024-04-25T08:31:41.997493+0000","last_clean":"2024-04-25T08:31:41.997493+0000","last_became_active":"2024-04-25T08:31:40.597800+0000","last_became_peered":"2024-04-25T08:31:40.597800+0000","last_unstale":"2024-04-25T08:31:41.997493+0000","last_undegraded":"2024-04-25T08:31:41.997493+0000","last_fullsized":"2024-04-25T08:31:41.997493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:10.902214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367151+0000","last_change":"2024-04-25T08:31:36.325531+0000","last_active":"2024-04-25T08:31:42.367151+0000","last_peered":"2024-04-25T08:31:42.367151+0000","last_clean":"2024-04-25T08:31:42.367151+0000","last_became_active":"2024-04-25T08:31:36.325240+0000","last_became_peered":"2024-04-25T08:31:36.325240+0000","last_unstale":"2024-04-25T08:31:42.367151+0000","last_undegraded":"2024-04-25T08:31:42.367151+0000","last_fullsized":"2024-04-25T08:31:42.367151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422321+0000","last_change":"2024-04-25T08:31:35.317032+0000","last_active":"2024-04-25T08:31:41.422321+0000","last_peered":"2024-04-25T08:31:41.422321+0000","last_clean":"2024-04-25T08:31:41.422321+0000","last_became_active":"2024-04-25T08:31:35.316659+0000","last_became_peered":"2024-04-25T08:31:35.316659+0000","last_unstale":"2024-04-25T08:31:41.422321+0000","last_undegraded":"2024-04-25T08:31:41.422321+0000","last_fullsized":"2024-04-25T08:31:41.422321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:57.762013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366270+0000","last_change":"2024-04-25T08:31:40.595457+0000","last_active":"2024-04-25T08:31:42.366270+0000","last_peered":"2024-04-25T08:31:42.366270+0000","last_clean":"2024-04-25T08:31:42.366270+0000","last_became_active":"2024-04-25T08:31:40.595077+0000","last_became_peered":"2024-04-25T08:31:40.595077+0000","last_unstale":"2024-04-25T08:31:42.366270+0000","last_undegraded":"2024-04-25T08:31:42.366270+0000","last_fullsized":"2024-04-25T08:31:42.366270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119506+0000","last_change":"2024-04-25T08:31:36.322361+0000","last_active":"2024-04-25T08:31:42.119506+0000","last_peered":"2024-04-25T08:31:42.119506+0000","last_clean":"2024-04-25T08:31:42.119506+0000","last_became_active":"2024-04-25T08:31:36.322107+0000","last_became_peered":"2024-04-25T08:31:36.322107+0000","last_unstale":"2024-04-25T08:31:42.119506+0000","last_undegraded":"2024-04-25T08:31:42.119506+0000","last_fullsized":"2024-04-25T08:31:42.119506+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:51.984540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367254+0000","last_change":"2024-04-25T08:31:35.315993+0000","last_active":"2024-04-25T08:31:42.367254+0000","last_peered":"2024-04-25T08:31:42.367254+0000","last_clean":"2024-04-25T08:31:42.367254+0000","last_became_active":"2024-04-25T08:31:35.315664+0000","last_became_peered":"2024-04-25T08:31:35.315664+0000","last_unstale":"2024-04-25T08:31:42.367254+0000","last_undegraded":"2024-04-25T08:31:42.367254+0000","last_fullsized":"2024-04-25T08:31:42.367254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365979+0000","last_change":"2024-04-25T08:31:40.355607+0000","last_active":"2024-04-25T08:31:42.365979+0000","last_peered":"2024-04-25T08:31:42.365979+0000","last_clean":"2024-04-25T08:31:42.365979+0000","last_became_active":"2024-04-25T08:31:40.355024+0000","last_became_peered":"2024-04-25T08:31:40.355024+0000","last_unstale":"2024-04-25T08:31:42.365979+0000","last_undegraded":"2024-04-25T08:31:42.365979+0000","last_fullsized":"2024-04-25T08:31:42.365979+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382608+0000","last_change":"2024-04-25T08:31:36.323675+0000","last_active":"2024-04-25T08:31:41.382608+0000","last_peered":"2024-04-25T08:31:41.382608+0000","last_clean":"2024-04-25T08:31:41.382608+0000","last_became_active":"2024-04-25T08:31:36.323102+0000","last_became_peered":"2024-04-25T08:31:36.323102+0000","last_unstale":"2024-04-25T08:31:41.382608+0000","last_undegraded":"2024-04-25T08:31:41.382608+0000","last_fullsized":"2024-04-25T08:31:41.382608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:49.744475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119335+0000","last_change":"2024-04-25T08:31:35.317234+0000","last_active":"2024-04-25T08:31:42.119335+0000","last_peered":"2024-04-25T08:31:42.119335+0000","last_clean":"2024-04-25T08:31:42.119335+0000","last_became_active":"2024-04-25T08:31:35.316747+0000","last_became_peered":"2024-04-25T08:31:35.316747+0000","last_unstale":"2024-04-25T08:31:42.119335+0000","last_undegraded":"2024-04-25T08:31:42.119335+0000","last_fullsized":"2024-04-25T08:31:42.119335+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367030+0000","last_change":"2024-04-25T08:31:40.352712+0000","last_active":"2024-04-25T08:31:42.367030+0000","last_peered":"2024-04-25T08:31:42.367030+0000","last_clean":"2024-04-25T08:31:42.367030+0000","last_became_active":"2024-04-25T08:31:40.352328+0000","last_became_peered":"2024-04-25T08:31:40.352328+0000","last_unstale":"2024-04-25T08:31:42.367030+0000","last_undegraded":"2024-04-25T08:31:42.367030+0000","last_fullsized":"2024-04-25T08:31:42.367030+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:56.013298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366618+0000","last_change":"2024-04-25T08:31:36.326854+0000","last_active":"2024-04-25T08:31:42.366618+0000","last_peered":"2024-04-25T08:31:42.366618+0000","last_clean":"2024-04-25T08:31:42.366618+0000","last_became_active":"2024-04-25T08:31:36.326564+0000","last_became_peered":"2024-04-25T08:31:36.326564+0000","last_unstale":"2024-04-25T08:31:42.366618+0000","last_undegraded":"2024-04-25T08:31:42.366618+0000","last_fullsized":"2024-04-25T08:31:42.366618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384554+0000","last_change":"2024-04-25T08:31:35.317497+0000","last_active":"2024-04-25T08:31:41.384554+0000","last_peered":"2024-04-25T08:31:41.384554+0000","last_clean":"2024-04-25T08:31:41.384554+0000","last_became_active":"2024-04-25T08:31:35.317202+0000","last_became_peered":"2024-04-25T08:31:35.317202+0000","last_unstale":"2024-04-25T08:31:41.384554+0000","last_undegraded":"2024-04-25T08:31:41.384554+0000","last_fullsized":"2024-04-25T08:31:41.384554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367456+0000","last_change":"2024-04-25T08:31:40.352169+0000","last_active":"2024-04-25T08:31:42.367456+0000","last_peered":"2024-04-25T08:31:42.367456+0000","last_clean":"2024-04-25T08:31:42.367456+0000","last_became_active":"2024-04-25T08:31:40.351743+0000","last_became_peered":"2024-04-25T08:31:40.351743+0000","last_unstale":"2024-04-25T08:31:42.367456+0000","last_undegraded":"2024-04-25T08:31:42.367456+0000","last_fullsized":"2024-04-25T08:31:42.367456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:09.948828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365773+0000","last_change":"2024-04-25T08:31:36.324307+0000","last_active":"2024-04-25T08:31:42.365773+0000","last_peered":"2024-04-25T08:31:42.365773+0000","last_clean":"2024-04-25T08:31:42.365773+0000","last_became_active":"2024-04-25T08:31:36.324011+0000","last_became_peered":"2024-04-25T08:31:36.324011+0000","last_unstale":"2024-04-25T08:31:42.365773+0000","last_undegraded":"2024-04-25T08:31:42.365773+0000","last_fullsized":"2024-04-25T08:31:42.365773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:01.999912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421648+0000","last_change":"2024-04-25T08:31:35.317490+0000","last_active":"2024-04-25T08:31:41.421648+0000","last_peered":"2024-04-25T08:31:41.421648+0000","last_clean":"2024-04-25T08:31:41.421648+0000","last_became_active":"2024-04-25T08:31:35.316569+0000","last_became_peered":"2024-04-25T08:31:35.316569+0000","last_unstale":"2024-04-25T08:31:41.421648+0000","last_undegraded":"2024-04-25T08:31:41.421648+0000","last_fullsized":"2024-04-25T08:31:41.421648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:09.703716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366542+0000","last_change":"2024-04-25T08:31:40.598075+0000","last_active":"2024-04-25T08:31:42.366542+0000","last_peered":"2024-04-25T08:31:42.366542+0000","last_clean":"2024-04-25T08:31:42.366542+0000","last_became_active":"2024-04-25T08:31:40.597669+0000","last_became_peered":"2024-04-25T08:31:40.597669+0000","last_unstale":"2024-04-25T08:31:42.366542+0000","last_undegraded":"2024-04-25T08:31:42.366542+0000","last_fullsized":"2024-04-25T08:31:42.366542+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:09.948828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997776+0000","last_change":"2024-04-25T08:31:36.320808+0000","last_active":"2024-04-25T08:31:41.997776+0000","last_peered":"2024-04-25T08:31:41.997776+0000","last_clean":"2024-04-25T08:31:41.997776+0000","last_became_active":"2024-04-25T08:31:36.320489+0000","last_became_peered":"2024-04-25T08:31:36.320489+0000","last_unstale":"2024-04-25T08:31:41.997776+0000","last_undegraded":"2024-04-25T08:31:41.997776+0000","last_fullsized":"2024-04-25T08:31:41.997776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119140+0000","last_change":"2024-04-25T08:31:35.312515+0000","last_active":"2024-04-25T08:31:42.119140+0000","last_peered":"2024-04-25T08:31:42.119140+0000","last_clean":"2024-04-25T08:31:42.119140+0000","last_became_active":"2024-04-25T08:31:35.312156+0000","last_became_peered":"2024-04-25T08:31:35.312156+0000","last_unstale":"2024-04-25T08:31:42.119140+0000","last_undegraded":"2024-04-25T08:31:42.119140+0000","last_fullsized":"2024-04-25T08:31:42.119140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421744+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:31:41.421744+0000","last_peered":"2024-04-25T08:31:41.421744+0000","last_clean":"2024-04-25T08:31:41.421744+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:31:41.421744+0000","last_undegraded":"2024-04-25T08:31:41.421744+0000","last_fullsized":"2024-04-25T08:31:41.421744+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381120+0000","last_change":"2024-04-25T08:31:40.597789+0000","last_active":"2024-04-25T08:31:41.381120+0000","last_peered":"2024-04-25T08:31:41.381120+0000","last_clean":"2024-04-25T08:31:41.381120+0000","last_became_active":"2024-04-25T08:31:40.597373+0000","last_became_peered":"2024-04-25T08:31:40.597373+0000","last_unstale":"2024-04-25T08:31:41.381120+0000","last_undegraded":"2024-04-25T08:31:41.381120+0000","last_fullsized":"2024-04-25T08:31:41.381120+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:25.140324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422089+0000","last_change":"2024-04-25T08:31:36.323510+0000","last_active":"2024-04-25T08:31:41.422089+0000","last_peered":"2024-04-25T08:31:41.422089+0000","last_clean":"2024-04-25T08:31:41.422089+0000","last_became_active":"2024-04-25T08:31:36.322941+0000","last_became_peered":"2024-04-25T08:31:36.322941+0000","last_unstale":"2024-04-25T08:31:41.422089+0000","last_undegraded":"2024-04-25T08:31:41.422089+0000","last_fullsized":"2024-04-25T08:31:41.422089+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:51.984540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384803+0000","last_change":"2024-04-25T08:31:35.318779+0000","last_active":"2024-04-25T08:31:41.384803+0000","last_peered":"2024-04-25T08:31:41.384803+0000","last_clean":"2024-04-25T08:31:41.384803+0000","last_became_active":"2024-04-25T08:31:35.318485+0000","last_became_peered":"2024-04-25T08:31:35.318485+0000","last_unstale":"2024-04-25T08:31:41.384803+0000","last_undegraded":"2024-04-25T08:31:41.384803+0000","last_fullsized":"2024-04-25T08:31:41.384803+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381168+0000","last_change":"2024-04-25T08:31:40.594633+0000","last_active":"2024-04-25T08:31:41.381168+0000","last_peered":"2024-04-25T08:31:41.381168+0000","last_clean":"2024-04-25T08:31:41.381168+0000","last_became_active":"2024-04-25T08:31:40.594193+0000","last_became_peered":"2024-04-25T08:31:40.594193+0000","last_unstale":"2024-04-25T08:31:41.381168+0000","last_undegraded":"2024-04-25T08:31:41.381168+0000","last_fullsized":"2024-04-25T08:31:41.381168+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119593+0000","last_change":"2024-04-25T08:31:36.327715+0000","last_active":"2024-04-25T08:31:42.119593+0000","last_peered":"2024-04-25T08:31:42.119593+0000","last_clean":"2024-04-25T08:31:42.119593+0000","last_became_active":"2024-04-25T08:31:36.327408+0000","last_became_peered":"2024-04-25T08:31:36.327408+0000","last_unstale":"2024-04-25T08:31:42.119593+0000","last_undegraded":"2024-04-25T08:31:42.119593+0000","last_fullsized":"2024-04-25T08:31:42.119593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:43.612460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367236+0000","last_change":"2024-04-25T08:31:35.310104+0000","last_active":"2024-04-25T08:31:42.367236+0000","last_peered":"2024-04-25T08:31:42.367236+0000","last_clean":"2024-04-25T08:31:42.367236+0000","last_became_active":"2024-04-25T08:31:35.309469+0000","last_became_peered":"2024-04-25T08:31:35.309469+0000","last_unstale":"2024-04-25T08:31:42.367236+0000","last_undegraded":"2024-04-25T08:31:42.367236+0000","last_fullsized":"2024-04-25T08:31:42.367236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:09.703716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366537+0000","last_change":"2024-04-25T08:31:40.356105+0000","last_active":"2024-04-25T08:31:42.366537+0000","last_peered":"2024-04-25T08:31:42.366537+0000","last_clean":"2024-04-25T08:31:42.366537+0000","last_became_active":"2024-04-25T08:31:40.355734+0000","last_became_peered":"2024-04-25T08:31:40.355734+0000","last_unstale":"2024-04-25T08:31:42.366537+0000","last_undegraded":"2024-04-25T08:31:42.366537+0000","last_fullsized":"2024-04-25T08:31:42.366537+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:23.148774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422428+0000","last_change":"2024-04-25T08:31:36.325987+0000","last_active":"2024-04-25T08:31:41.422428+0000","last_peered":"2024-04-25T08:31:41.422428+0000","last_clean":"2024-04-25T08:31:41.422428+0000","last_became_active":"2024-04-25T08:31:36.324131+0000","last_became_peered":"2024-04-25T08:31:36.324131+0000","last_unstale":"2024-04-25T08:31:41.422428+0000","last_undegraded":"2024-04-25T08:31:41.422428+0000","last_fullsized":"2024-04-25T08:31:41.422428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:32.255703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384855+0000","last_change":"2024-04-25T08:31:35.321968+0000","last_active":"2024-04-25T08:31:41.384855+0000","last_peered":"2024-04-25T08:31:41.384855+0000","last_clean":"2024-04-25T08:31:41.384855+0000","last_became_active":"2024-04-25T08:31:35.321609+0000","last_became_peered":"2024-04-25T08:31:35.321609+0000","last_unstale":"2024-04-25T08:31:41.384855+0000","last_undegraded":"2024-04-25T08:31:41.384855+0000","last_fullsized":"2024-04-25T08:31:41.384855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118903+0000","last_change":"2024-04-25T08:31:40.597471+0000","last_active":"2024-04-25T08:31:42.118903+0000","last_peered":"2024-04-25T08:31:42.118903+0000","last_clean":"2024-04-25T08:31:42.118903+0000","last_became_active":"2024-04-25T08:31:40.597017+0000","last_became_peered":"2024-04-25T08:31:40.597017+0000","last_unstale":"2024-04-25T08:31:42.118903+0000","last_undegraded":"2024-04-25T08:31:42.118903+0000","last_fullsized":"2024-04-25T08:31:42.118903+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:10.902214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422271+0000","last_change":"2024-04-25T08:31:36.325470+0000","last_active":"2024-04-25T08:31:41.422271+0000","last_peered":"2024-04-25T08:31:41.422271+0000","last_clean":"2024-04-25T08:31:41.422271+0000","last_became_active":"2024-04-25T08:31:36.325177+0000","last_became_peered":"2024-04-25T08:31:36.325177+0000","last_unstale":"2024-04-25T08:31:41.422271+0000","last_undegraded":"2024-04-25T08:31:41.422271+0000","last_fullsized":"2024-04-25T08:31:41.422271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:19.120016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381794+0000","last_change":"2024-04-25T08:31:35.312440+0000","last_active":"2024-04-25T08:31:41.381794+0000","last_peered":"2024-04-25T08:31:41.381794+0000","last_clean":"2024-04-25T08:31:41.381794+0000","last_became_active":"2024-04-25T08:31:35.312130+0000","last_became_peered":"2024-04-25T08:31:35.312130+0000","last_unstale":"2024-04-25T08:31:41.381794+0000","last_undegraded":"2024-04-25T08:31:41.381794+0000","last_fullsized":"2024-04-25T08:31:41.381794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:32.280995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381222+0000","last_change":"2024-04-25T08:31:40.351471+0000","last_active":"2024-04-25T08:31:41.381222+0000","last_peered":"2024-04-25T08:31:41.381222+0000","last_clean":"2024-04-25T08:31:41.381222+0000","last_became_active":"2024-04-25T08:31:40.351036+0000","last_became_peered":"2024-04-25T08:31:40.351036+0000","last_unstale":"2024-04-25T08:31:41.381222+0000","last_undegraded":"2024-04-25T08:31:41.381222+0000","last_fullsized":"2024-04-25T08:31:41.381222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:30.214541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421850+0000","last_change":"2024-04-25T08:31:36.325960+0000","last_active":"2024-04-25T08:31:41.421850+0000","last_peered":"2024-04-25T08:31:41.421850+0000","last_clean":"2024-04-25T08:31:41.421850+0000","last_became_active":"2024-04-25T08:31:36.325667+0000","last_became_peered":"2024-04-25T08:31:36.325667+0000","last_unstale":"2024-04-25T08:31:41.421850+0000","last_undegraded":"2024-04-25T08:31:41.421850+0000","last_fullsized":"2024-04-25T08:31:41.421850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:47.399399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384402+0000","last_change":"2024-04-25T08:31:35.307681+0000","last_active":"2024-04-25T08:31:41.384402+0000","last_peered":"2024-04-25T08:31:41.384402+0000","last_clean":"2024-04-25T08:31:41.384402+0000","last_became_active":"2024-04-25T08:31:35.307213+0000","last_became_peered":"2024-04-25T08:31:35.307213+0000","last_unstale":"2024-04-25T08:31:41.384402+0000","last_undegraded":"2024-04-25T08:31:41.384402+0000","last_fullsized":"2024-04-25T08:31:41.384402+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:34.641593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366369+0000","last_change":"2024-04-25T08:31:40.352841+0000","last_active":"2024-04-25T08:31:42.366369+0000","last_peered":"2024-04-25T08:31:42.366369+0000","last_clean":"2024-04-25T08:31:42.366369+0000","last_became_active":"2024-04-25T08:31:40.352418+0000","last_became_peered":"2024-04-25T08:31:40.352418+0000","last_unstale":"2024-04-25T08:31:42.366369+0000","last_undegraded":"2024-04-25T08:31:42.366369+0000","last_fullsized":"2024-04-25T08:31:42.366369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:56.013298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.380985+0000","last_change":"2024-04-25T08:31:36.312845+0000","last_active":"2024-04-25T08:31:41.380985+0000","last_peered":"2024-04-25T08:31:41.380985+0000","last_clean":"2024-04-25T08:31:41.380985+0000","last_became_active":"2024-04-25T08:31:36.312538+0000","last_became_peered":"2024-04-25T08:31:36.312538+0000","last_unstale":"2024-04-25T08:31:41.380985+0000","last_undegraded":"2024-04-25T08:31:41.380985+0000","last_fullsized":"2024-04-25T08:31:41.380985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381431+0000","last_change":"2024-04-25T08:31:35.305736+0000","last_active":"2024-04-25T08:31:41.381431+0000","last_peered":"2024-04-25T08:31:41.381431+0000","last_clean":"2024-04-25T08:31:41.381431+0000","last_became_active":"2024-04-25T08:31:35.305481+0000","last_became_peered":"2024-04-25T08:31:35.305481+0000","last_unstale":"2024-04-25T08:31:41.381431+0000","last_undegraded":"2024-04-25T08:31:41.381431+0000","last_fullsized":"2024-04-25T08:31:41.381431+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998328+0000","last_change":"2024-04-25T08:31:40.599441+0000","last_active":"2024-04-25T08:31:41.998328+0000","last_peered":"2024-04-25T08:31:41.998328+0000","last_clean":"2024-04-25T08:31:41.998328+0000","last_became_active":"2024-04-25T08:31:40.599023+0000","last_became_peered":"2024-04-25T08:31:40.599023+0000","last_unstale":"2024-04-25T08:31:41.998328+0000","last_undegraded":"2024-04-25T08:31:41.998328+0000","last_fullsized":"2024-04-25T08:31:41.998328+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:09.443843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421936+0000","last_change":"2024-04-25T08:31:36.323783+0000","last_active":"2024-04-25T08:31:41.421936+0000","last_peered":"2024-04-25T08:31:41.421936+0000","last_clean":"2024-04-25T08:31:41.421936+0000","last_became_active":"2024-04-25T08:31:36.323460+0000","last_became_peered":"2024-04-25T08:31:36.323460+0000","last_unstale":"2024-04-25T08:31:41.421936+0000","last_undegraded":"2024-04-25T08:31:41.421936+0000","last_fullsized":"2024-04-25T08:31:41.421936+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119196+0000","last_change":"2024-04-25T08:31:35.318378+0000","last_active":"2024-04-25T08:31:42.119196+0000","last_peered":"2024-04-25T08:31:42.119196+0000","last_clean":"2024-04-25T08:31:42.119196+0000","last_became_active":"2024-04-25T08:31:35.318103+0000","last_became_peered":"2024-04-25T08:31:35.318103+0000","last_unstale":"2024-04-25T08:31:42.119196+0000","last_undegraded":"2024-04-25T08:31:42.119196+0000","last_fullsized":"2024-04-25T08:31:42.119196+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998302+0000","last_change":"2024-04-25T08:31:40.595629+0000","last_active":"2024-04-25T08:31:41.998302+0000","last_peered":"2024-04-25T08:31:41.998302+0000","last_clean":"2024-04-25T08:31:41.998302+0000","last_became_active":"2024-04-25T08:31:40.594160+0000","last_became_peered":"2024-04-25T08:31:40.594160+0000","last_unstale":"2024-04-25T08:31:41.998302+0000","last_undegraded":"2024-04-25T08:31:41.998302+0000","last_fullsized":"2024-04-25T08:31:41.998302+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:00.245750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366188+0000","last_change":"2024-04-25T08:31:36.329619+0000","last_active":"2024-04-25T08:31:42.366188+0000","last_peered":"2024-04-25T08:31:42.366188+0000","last_clean":"2024-04-25T08:31:42.366188+0000","last_became_active":"2024-04-25T08:31:36.329319+0000","last_became_peered":"2024-04-25T08:31:36.329319+0000","last_unstale":"2024-04-25T08:31:42.366188+0000","last_undegraded":"2024-04-25T08:31:42.366188+0000","last_fullsized":"2024-04-25T08:31:42.366188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:45:32.370451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366817+0000","last_change":"2024-04-25T08:31:35.317195+0000","last_active":"2024-04-25T08:31:42.366817+0000","last_peered":"2024-04-25T08:31:42.366817+0000","last_clean":"2024-04-25T08:31:42.366817+0000","last_became_active":"2024-04-25T08:31:35.316819+0000","last_became_peered":"2024-04-25T08:31:35.316819+0000","last_unstale":"2024-04-25T08:31:42.366817+0000","last_undegraded":"2024-04-25T08:31:42.366817+0000","last_fullsized":"2024-04-25T08:31:42.366817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:32.280995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119540+0000","last_change":"2024-04-25T08:31:40.594853+0000","last_active":"2024-04-25T08:31:42.119540+0000","last_peered":"2024-04-25T08:31:42.119540+0000","last_clean":"2024-04-25T08:31:42.119540+0000","last_became_active":"2024-04-25T08:31:40.594456+0000","last_became_peered":"2024-04-25T08:31:40.594456+0000","last_unstale":"2024-04-25T08:31:42.119540+0000","last_undegraded":"2024-04-25T08:31:42.119540+0000","last_fullsized":"2024-04-25T08:31:42.119540+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:09.735540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422019+0000","last_change":"2024-04-25T08:31:36.324840+0000","last_active":"2024-04-25T08:31:41.422019+0000","last_peered":"2024-04-25T08:31:41.422019+0000","last_clean":"2024-04-25T08:31:41.422019+0000","last_became_active":"2024-04-25T08:31:36.324574+0000","last_became_peered":"2024-04-25T08:31:36.324574+0000","last_unstale":"2024-04-25T08:31:41.422019+0000","last_undegraded":"2024-04-25T08:31:41.422019+0000","last_fullsized":"2024-04-25T08:31:41.422019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.111565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'9","reported_seq":31,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:45.647660+0000","last_change":"2024-04-25T08:31:35.316767+0000","last_active":"2024-04-25T08:31:45.647660+0000","last_peered":"2024-04-25T08:31:45.647660+0000","last_clean":"2024-04-25T08:31:45.647660+0000","last_became_active":"2024-04-25T08:31:35.316484+0000","last_became_peered":"2024-04-25T08:31:35.316484+0000","last_unstale":"2024-04-25T08:31:45.647660+0000","last_undegraded":"2024-04-25T08:31:45.647660+0000","last_fullsized":"2024-04-25T08:31:45.647660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:00.608138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366316+0000","last_change":"2024-04-25T08:31:40.355742+0000","last_active":"2024-04-25T08:31:42.366316+0000","last_peered":"2024-04-25T08:31:42.366316+0000","last_clean":"2024-04-25T08:31:42.366316+0000","last_became_active":"2024-04-25T08:31:40.355357+0000","last_became_peered":"2024-04-25T08:31:40.355357+0000","last_unstale":"2024-04-25T08:31:42.366316+0000","last_undegraded":"2024-04-25T08:31:42.366316+0000","last_fullsized":"2024-04-25T08:31:42.366316+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382374+0000","last_change":"2024-04-25T08:31:36.313482+0000","last_active":"2024-04-25T08:31:41.382374+0000","last_peered":"2024-04-25T08:31:41.382374+0000","last_clean":"2024-04-25T08:31:41.382374+0000","last_became_active":"2024-04-25T08:31:36.312650+0000","last_became_peered":"2024-04-25T08:31:36.312650+0000","last_unstale":"2024-04-25T08:31:41.382374+0000","last_undegraded":"2024-04-25T08:31:41.382374+0000","last_fullsized":"2024-04-25T08:31:41.382374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:10.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367199+0000","last_change":"2024-04-25T08:31:35.320355+0000","last_active":"2024-04-25T08:31:42.367199+0000","last_peered":"2024-04-25T08:31:42.367199+0000","last_clean":"2024-04-25T08:31:42.367199+0000","last_became_active":"2024-04-25T08:31:35.320046+0000","last_became_peered":"2024-04-25T08:31:35.320046+0000","last_unstale":"2024-04-25T08:31:42.367199+0000","last_undegraded":"2024-04-25T08:31:42.367199+0000","last_fullsized":"2024-04-25T08:31:42.367199+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:46.434047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382283+0000","last_change":"2024-04-25T08:31:40.356433+0000","last_active":"2024-04-25T08:31:41.382283+0000","last_peered":"2024-04-25T08:31:41.382283+0000","last_clean":"2024-04-25T08:31:41.382283+0000","last_became_active":"2024-04-25T08:31:40.355755+0000","last_became_peered":"2024-04-25T08:31:40.355755+0000","last_unstale":"2024-04-25T08:31:41.382283+0000","last_undegraded":"2024-04-25T08:31:41.382283+0000","last_fullsized":"2024-04-25T08:31:41.382283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:51.428157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366137+0000","last_change":"2024-04-25T08:31:36.323385+0000","last_active":"2024-04-25T08:31:42.366137+0000","last_peered":"2024-04-25T08:31:42.366137+0000","last_clean":"2024-04-25T08:31:42.366137+0000","last_became_active":"2024-04-25T08:31:36.322674+0000","last_became_peered":"2024-04-25T08:31:36.322674+0000","last_unstale":"2024-04-25T08:31:42.366137+0000","last_undegraded":"2024-04-25T08:31:42.366137+0000","last_fullsized":"2024-04-25T08:31:42.366137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119486+0000","last_change":"2024-04-25T08:31:35.317212+0000","last_active":"2024-04-25T08:31:42.119486+0000","last_peered":"2024-04-25T08:31:42.119486+0000","last_clean":"2024-04-25T08:31:42.119486+0000","last_became_active":"2024-04-25T08:31:35.316922+0000","last_became_peered":"2024-04-25T08:31:35.316922+0000","last_unstale":"2024-04-25T08:31:42.119486+0000","last_undegraded":"2024-04-25T08:31:42.119486+0000","last_fullsized":"2024-04-25T08:31:42.119486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.726901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997655+0000","last_change":"2024-04-25T08:31:40.353781+0000","last_active":"2024-04-25T08:31:41.997655+0000","last_peered":"2024-04-25T08:31:41.997655+0000","last_clean":"2024-04-25T08:31:41.997655+0000","last_became_active":"2024-04-25T08:31:40.352269+0000","last_became_peered":"2024-04-25T08:31:40.352269+0000","last_unstale":"2024-04-25T08:31:41.997655+0000","last_undegraded":"2024-04-25T08:31:41.997655+0000","last_fullsized":"2024-04-25T08:31:41.997655+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:23.063716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119364+0000","last_change":"2024-04-25T08:31:36.324056+0000","last_active":"2024-04-25T08:31:42.119364+0000","last_peered":"2024-04-25T08:31:42.119364+0000","last_clean":"2024-04-25T08:31:42.119364+0000","last_became_active":"2024-04-25T08:31:36.323790+0000","last_became_peered":"2024-04-25T08:31:36.323790+0000","last_unstale":"2024-04-25T08:31:42.119364+0000","last_undegraded":"2024-04-25T08:31:42.119364+0000","last_fullsized":"2024-04-25T08:31:42.119364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:19.120016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382208+0000","last_change":"2024-04-25T08:31:35.318093+0000","last_active":"2024-04-25T08:31:41.382208+0000","last_peered":"2024-04-25T08:31:41.382208+0000","last_clean":"2024-04-25T08:31:41.382208+0000","last_became_active":"2024-04-25T08:31:35.317828+0000","last_became_peered":"2024-04-25T08:31:35.317828+0000","last_unstale":"2024-04-25T08:31:41.382208+0000","last_undegraded":"2024-04-25T08:31:41.382208+0000","last_fullsized":"2024-04-25T08:31:41.382208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:54.173888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997667+0000","last_change":"2024-04-25T08:31:40.599421+0000","last_active":"2024-04-25T08:31:41.997667+0000","last_peered":"2024-04-25T08:31:41.997667+0000","last_clean":"2024-04-25T08:31:41.997667+0000","last_became_active":"2024-04-25T08:31:40.599105+0000","last_became_peered":"2024-04-25T08:31:40.599105+0000","last_unstale":"2024-04-25T08:31:41.997667+0000","last_undegraded":"2024-04-25T08:31:41.997667+0000","last_fullsized":"2024-04-25T08:31:41.997667+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:50.738838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366324+0000","last_change":"2024-04-25T08:31:40.597001+0000","last_active":"2024-04-25T08:31:42.366324+0000","last_peered":"2024-04-25T08:31:42.366324+0000","last_clean":"2024-04-25T08:31:42.366324+0000","last_became_active":"2024-04-25T08:31:40.596569+0000","last_became_peered":"2024-04-25T08:31:40.596569+0000","last_unstale":"2024-04-25T08:31:42.366324+0000","last_undegraded":"2024-04-25T08:31:42.366324+0000","last_fullsized":"2024-04-25T08:31:42.366324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:47.641218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367007+0000","last_change":"2024-04-25T08:31:36.332669+0000","last_active":"2024-04-25T08:31:42.367007+0000","last_peered":"2024-04-25T08:31:42.367007+0000","last_clean":"2024-04-25T08:31:42.367007+0000","last_became_active":"2024-04-25T08:31:36.332305+0000","last_became_peered":"2024-04-25T08:31:36.332305+0000","last_unstale":"2024-04-25T08:31:42.367007+0000","last_undegraded":"2024-04-25T08:31:42.367007+0000","last_fullsized":"2024-04-25T08:31:42.367007+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:15.226896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118969+0000","last_change":"2024-04-25T08:31:35.317959+0000","last_active":"2024-04-25T08:31:42.118969+0000","last_peered":"2024-04-25T08:31:42.118969+0000","last_clean":"2024-04-25T08:31:42.118969+0000","last_became_active":"2024-04-25T08:31:35.317680+0000","last_became_peered":"2024-04-25T08:31:35.317680+0000","last_unstale":"2024-04-25T08:31:42.118969+0000","last_undegraded":"2024-04-25T08:31:42.118969+0000","last_fullsized":"2024-04-25T08:31:42.118969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:54.173888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382101+0000","last_change":"2024-04-25T08:31:40.352855+0000","last_active":"2024-04-25T08:31:41.382101+0000","last_peered":"2024-04-25T08:31:41.382101+0000","last_clean":"2024-04-25T08:31:41.382101+0000","last_became_active":"2024-04-25T08:31:40.352373+0000","last_became_peered":"2024-04-25T08:31:40.352373+0000","last_unstale":"2024-04-25T08:31:41.382101+0000","last_undegraded":"2024-04-25T08:31:41.382101+0000","last_fullsized":"2024-04-25T08:31:41.382101+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:56.013298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997618+0000","last_change":"2024-04-25T08:31:36.327572+0000","last_active":"2024-04-25T08:31:41.997618+0000","last_peered":"2024-04-25T08:31:41.997618+0000","last_clean":"2024-04-25T08:31:41.997618+0000","last_became_active":"2024-04-25T08:31:36.320635+0000","last_became_peered":"2024-04-25T08:31:36.320635+0000","last_unstale":"2024-04-25T08:31:41.997618+0000","last_undegraded":"2024-04-25T08:31:41.997618+0000","last_fullsized":"2024-04-25T08:31:41.997618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997788+0000","last_change":"2024-04-25T08:31:35.317367+0000","last_active":"2024-04-25T08:31:41.997788+0000","last_peered":"2024-04-25T08:31:41.997788+0000","last_clean":"2024-04-25T08:31:41.997788+0000","last_became_active":"2024-04-25T08:31:35.317013+0000","last_became_peered":"2024-04-25T08:31:35.317013+0000","last_unstale":"2024-04-25T08:31:41.997788+0000","last_undegraded":"2024-04-25T08:31:41.997788+0000","last_fullsized":"2024-04-25T08:31:41.997788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:34.641593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381422+0000","last_change":"2024-04-25T08:31:35.305651+0000","last_active":"2024-04-25T08:31:41.381422+0000","last_peered":"2024-04-25T08:31:41.381422+0000","last_clean":"2024-04-25T08:31:41.381422+0000","last_became_active":"2024-04-25T08:31:35.305381+0000","last_became_peered":"2024-04-25T08:31:35.305381+0000","last_unstale":"2024-04-25T08:31:41.381422+0000","last_undegraded":"2024-04-25T08:31:41.381422+0000","last_fullsized":"2024-04-25T08:31:41.381422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:48.792371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366718+0000","last_change":"2024-04-25T08:31:36.323372+0000","last_active":"2024-04-25T08:31:42.366718+0000","last_peered":"2024-04-25T08:31:42.366718+0000","last_clean":"2024-04-25T08:31:42.366718+0000","last_became_active":"2024-04-25T08:31:36.322950+0000","last_became_peered":"2024-04-25T08:31:36.322950+0000","last_unstale":"2024-04-25T08:31:42.366718+0000","last_undegraded":"2024-04-25T08:31:42.366718+0000","last_fullsized":"2024-04-25T08:31:42.366718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384753+0000","last_change":"2024-04-25T08:31:40.353992+0000","last_active":"2024-04-25T08:31:41.384753+0000","last_peered":"2024-04-25T08:31:41.384753+0000","last_clean":"2024-04-25T08:31:41.384753+0000","last_became_active":"2024-04-25T08:31:40.353601+0000","last_became_peered":"2024-04-25T08:31:40.353601+0000","last_unstale":"2024-04-25T08:31:41.384753+0000","last_undegraded":"2024-04-25T08:31:41.384753+0000","last_fullsized":"2024-04-25T08:31:41.384753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:02.808345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997972+0000","last_change":"2024-04-25T08:31:35.321307+0000","last_active":"2024-04-25T08:31:41.997972+0000","last_peered":"2024-04-25T08:31:41.997972+0000","last_clean":"2024-04-25T08:31:41.997972+0000","last_became_active":"2024-04-25T08:31:35.320720+0000","last_became_peered":"2024-04-25T08:31:35.320720+0000","last_unstale":"2024-04-25T08:31:41.997972+0000","last_undegraded":"2024-04-25T08:31:41.997972+0000","last_fullsized":"2024-04-25T08:31:41.997972+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:34.272524+0000","last_clean_scrub_stamp":"2024-04-25T08:31:34.272524+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:00.957946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381993+0000","last_change":"2024-04-25T08:31:36.317597+0000","last_active":"2024-04-25T08:31:41.381993+0000","last_peered":"2024-04-25T08:31:41.381993+0000","last_clean":"2024-04-25T08:31:41.381993+0000","last_became_active":"2024-04-25T08:31:36.317333+0000","last_became_peered":"2024-04-25T08:31:36.317333+0000","last_unstale":"2024-04-25T08:31:41.381993+0000","last_undegraded":"2024-04-25T08:31:41.381993+0000","last_fullsized":"2024-04-25T08:31:41.381993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:55.191462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998205+0000","last_change":"2024-04-25T08:31:40.357847+0000","last_active":"2024-04-25T08:31:41.998205+0000","last_peered":"2024-04-25T08:31:41.998205+0000","last_clean":"2024-04-25T08:31:41.998205+0000","last_became_active":"2024-04-25T08:31:40.357445+0000","last_became_peered":"2024-04-25T08:31:40.357445+0000","last_unstale":"2024-04-25T08:31:41.998205+0000","last_undegraded":"2024-04-25T08:31:41.998205+0000","last_fullsized":"2024-04-25T08:31:41.998205+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:39.318857+0000","last_clean_scrub_stamp":"2024-04-25T08:31:39.318857+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:45.166751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381524+0000","last_change":"2024-04-25T08:31:36.310801+0000","last_active":"2024-04-25T08:31:41.381524+0000","last_peered":"2024-04-25T08:31:41.381524+0000","last_clean":"2024-04-25T08:31:41.381524+0000","last_became_active":"2024-04-25T08:31:36.310509+0000","last_became_peered":"2024-04-25T08:31:36.310509+0000","last_unstale":"2024-04-25T08:31:41.381524+0000","last_undegraded":"2024-04-25T08:31:41.381524+0000","last_fullsized":"2024-04-25T08:31:41.381524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384592+0000","last_change":"2024-04-25T08:31:36.320666+0000","last_active":"2024-04-25T08:31:41.384592+0000","last_peered":"2024-04-25T08:31:41.384592+0000","last_clean":"2024-04-25T08:31:41.384592+0000","last_became_active":"2024-04-25T08:31:36.320402+0000","last_became_peered":"2024-04-25T08:31:36.320402+0000","last_unstale":"2024-04-25T08:31:41.384592+0000","last_undegraded":"2024-04-25T08:31:41.384592+0000","last_fullsized":"2024-04-25T08:31:41.384592+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:47.451621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119194+0000","last_change":"2024-04-25T08:31:36.323523+0000","last_active":"2024-04-25T08:31:42.119194+0000","last_peered":"2024-04-25T08:31:42.119194+0000","last_clean":"2024-04-25T08:31:42.119194+0000","last_became_active":"2024-04-25T08:31:36.323233+0000","last_became_peered":"2024-04-25T08:31:36.323233+0000","last_unstale":"2024-04-25T08:31:42.119194+0000","last_undegraded":"2024-04-25T08:31:42.119194+0000","last_fullsized":"2024-04-25T08:31:42.119194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:46.700115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366631+0000","last_change":"2024-04-25T08:31:36.332567+0000","last_active":"2024-04-25T08:31:42.366631+0000","last_peered":"2024-04-25T08:31:42.366631+0000","last_clean":"2024-04-25T08:31:42.366631+0000","last_became_active":"2024-04-25T08:31:36.332300+0000","last_became_peered":"2024-04-25T08:31:36.332300+0000","last_unstale":"2024-04-25T08:31:42.366631+0000","last_undegraded":"2024-04-25T08:31:42.366631+0000","last_fullsized":"2024-04-25T08:31:42.366631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:12:57.933443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998117+0000","last_change":"2024-04-25T08:31:36.326949+0000","last_active":"2024-04-25T08:31:41.998117+0000","last_peered":"2024-04-25T08:31:41.998117+0000","last_clean":"2024-04-25T08:31:41.998117+0000","last_became_active":"2024-04-25T08:31:36.322467+0000","last_became_peered":"2024-04-25T08:31:36.322467+0000","last_unstale":"2024-04-25T08:31:41.998117+0000","last_undegraded":"2024-04-25T08:31:41.998117+0000","last_fullsized":"2024-04-25T08:31:41.998117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:46.700115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367093+0000","last_change":"2024-04-25T08:31:36.331852+0000","last_active":"2024-04-25T08:31:42.367093+0000","last_peered":"2024-04-25T08:31:42.367093+0000","last_clean":"2024-04-25T08:31:42.367093+0000","last_became_active":"2024-04-25T08:31:36.331588+0000","last_became_peered":"2024-04-25T08:31:36.331588+0000","last_unstale":"2024-04-25T08:31:42.367093+0000","last_undegraded":"2024-04-25T08:31:42.367093+0000","last_fullsized":"2024-04-25T08:31:42.367093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381821+0000","last_change":"2024-04-25T08:31:36.309132+0000","last_active":"2024-04-25T08:31:41.381821+0000","last_peered":"2024-04-25T08:31:41.381821+0000","last_clean":"2024-04-25T08:31:41.381821+0000","last_became_active":"2024-04-25T08:31:36.308790+0000","last_became_peered":"2024-04-25T08:31:36.308790+0000","last_unstale":"2024-04-25T08:31:41.381821+0000","last_undegraded":"2024-04-25T08:31:41.381821+0000","last_fullsized":"2024-04-25T08:31:41.381821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:14.177622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.998237+0000","last_change":"2024-04-25T08:31:36.320300+0000","last_active":"2024-04-25T08:31:41.998237+0000","last_peered":"2024-04-25T08:31:41.998237+0000","last_clean":"2024-04-25T08:31:41.998237+0000","last_became_active":"2024-04-25T08:31:36.319984+0000","last_became_peered":"2024-04-25T08:31:36.319984+0000","last_unstale":"2024-04-25T08:31:41.998237+0000","last_undegraded":"2024-04-25T08:31:41.998237+0000","last_fullsized":"2024-04-25T08:31:41.998237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:00.969634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118927+0000","last_change":"2024-04-25T08:31:36.329425+0000","last_active":"2024-04-25T08:31:42.118927+0000","last_peered":"2024-04-25T08:31:42.118927+0000","last_clean":"2024-04-25T08:31:42.118927+0000","last_became_active":"2024-04-25T08:31:36.328879+0000","last_became_peered":"2024-04-25T08:31:36.328879+0000","last_unstale":"2024-04-25T08:31:42.118927+0000","last_undegraded":"2024-04-25T08:31:42.118927+0000","last_fullsized":"2024-04-25T08:31:42.118927+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997886+0000","last_change":"2024-04-25T08:31:36.322034+0000","last_active":"2024-04-25T08:31:41.997886+0000","last_peered":"2024-04-25T08:31:41.997886+0000","last_clean":"2024-04-25T08:31:41.997886+0000","last_became_active":"2024-04-25T08:31:36.321723+0000","last_became_peered":"2024-04-25T08:31:36.321723+0000","last_unstale":"2024-04-25T08:31:41.997886+0000","last_undegraded":"2024-04-25T08:31:41.997886+0000","last_fullsized":"2024-04-25T08:31:41.997886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119094+0000","last_change":"2024-04-25T08:31:36.325939+0000","last_active":"2024-04-25T08:31:42.119094+0000","last_peered":"2024-04-25T08:31:42.119094+0000","last_clean":"2024-04-25T08:31:42.119094+0000","last_became_active":"2024-04-25T08:31:36.325469+0000","last_became_peered":"2024-04-25T08:31:36.325469+0000","last_unstale":"2024-04-25T08:31:42.119094+0000","last_undegraded":"2024-04-25T08:31:42.119094+0000","last_fullsized":"2024-04-25T08:31:42.119094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:57:14.177622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366378+0000","last_change":"2024-04-25T08:31:36.328064+0000","last_active":"2024-04-25T08:31:42.366378+0000","last_peered":"2024-04-25T08:31:42.366378+0000","last_clean":"2024-04-25T08:31:42.366378+0000","last_became_active":"2024-04-25T08:31:36.327769+0000","last_became_peered":"2024-04-25T08:31:36.327769+0000","last_unstale":"2024-04-25T08:31:42.366378+0000","last_undegraded":"2024-04-25T08:31:42.366378+0000","last_fullsized":"2024-04-25T08:31:42.366378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:58.779587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381873+0000","last_change":"2024-04-25T08:31:36.318263+0000","last_active":"2024-04-25T08:31:41.381873+0000","last_peered":"2024-04-25T08:31:41.381873+0000","last_clean":"2024-04-25T08:31:41.381873+0000","last_became_active":"2024-04-25T08:31:36.317987+0000","last_became_peered":"2024-04-25T08:31:36.317987+0000","last_unstale":"2024-04-25T08:31:41.381873+0000","last_undegraded":"2024-04-25T08:31:41.381873+0000","last_fullsized":"2024-04-25T08:31:41.381873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422245+0000","last_change":"2024-04-25T08:31:36.318589+0000","last_active":"2024-04-25T08:31:41.422245+0000","last_peered":"2024-04-25T08:31:41.422245+0000","last_clean":"2024-04-25T08:31:41.422245+0000","last_became_active":"2024-04-25T08:31:36.318315+0000","last_became_peered":"2024-04-25T08:31:36.318315+0000","last_unstale":"2024-04-25T08:31:41.422245+0000","last_undegraded":"2024-04-25T08:31:41.422245+0000","last_fullsized":"2024-04-25T08:31:41.422245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:05.920070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.384795+0000","last_change":"2024-04-25T08:31:36.320980+0000","last_active":"2024-04-25T08:31:41.384795+0000","last_peered":"2024-04-25T08:31:41.384795+0000","last_clean":"2024-04-25T08:31:41.384795+0000","last_became_active":"2024-04-25T08:31:36.320663+0000","last_became_peered":"2024-04-25T08:31:36.320663+0000","last_unstale":"2024-04-25T08:31:41.384795+0000","last_undegraded":"2024-04-25T08:31:41.384795+0000","last_fullsized":"2024-04-25T08:31:41.384795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:10.721290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119636+0000","last_change":"2024-04-25T08:31:36.328219+0000","last_active":"2024-04-25T08:31:42.119636+0000","last_peered":"2024-04-25T08:31:42.119636+0000","last_clean":"2024-04-25T08:31:42.119636+0000","last_became_active":"2024-04-25T08:31:36.327914+0000","last_became_peered":"2024-04-25T08:31:36.327914+0000","last_unstale":"2024-04-25T08:31:42.119636+0000","last_undegraded":"2024-04-25T08:31:42.119636+0000","last_fullsized":"2024-04-25T08:31:42.119636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:47.399399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.118812+0000","last_change":"2024-04-25T08:31:36.322834+0000","last_active":"2024-04-25T08:31:42.118812+0000","last_peered":"2024-04-25T08:31:42.118812+0000","last_clean":"2024-04-25T08:31:42.118812+0000","last_became_active":"2024-04-25T08:31:36.322555+0000","last_became_peered":"2024-04-25T08:31:36.322555+0000","last_unstale":"2024-04-25T08:31:42.118812+0000","last_undegraded":"2024-04-25T08:31:42.118812+0000","last_fullsized":"2024-04-25T08:31:42.118812+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367013+0000","last_change":"2024-04-25T08:31:36.323473+0000","last_active":"2024-04-25T08:31:42.367013+0000","last_peered":"2024-04-25T08:31:42.367013+0000","last_clean":"2024-04-25T08:31:42.367013+0000","last_became_active":"2024-04-25T08:31:36.323180+0000","last_became_peered":"2024-04-25T08:31:36.323180+0000","last_unstale":"2024-04-25T08:31:42.367013+0000","last_undegraded":"2024-04-25T08:31:42.367013+0000","last_fullsized":"2024-04-25T08:31:42.367013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.366156+0000","last_change":"2024-04-25T08:31:36.314459+0000","last_active":"2024-04-25T08:31:42.366156+0000","last_peered":"2024-04-25T08:31:42.366156+0000","last_clean":"2024-04-25T08:31:42.366156+0000","last_became_active":"2024-04-25T08:31:36.313669+0000","last_became_peered":"2024-04-25T08:31:36.313669+0000","last_unstale":"2024-04-25T08:31:42.366156+0000","last_undegraded":"2024-04-25T08:31:42.366156+0000","last_fullsized":"2024-04-25T08:31:42.366156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:00.952254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997797+0000","last_change":"2024-04-25T08:31:36.329217+0000","last_active":"2024-04-25T08:31:41.997797+0000","last_peered":"2024-04-25T08:31:41.997797+0000","last_clean":"2024-04-25T08:31:41.997797+0000","last_became_active":"2024-04-25T08:31:36.328793+0000","last_became_peered":"2024-04-25T08:31:36.328793+0000","last_unstale":"2024-04-25T08:31:41.997797+0000","last_undegraded":"2024-04-25T08:31:41.997797+0000","last_fullsized":"2024-04-25T08:31:41.997797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:26.185783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.382287+0000","last_change":"2024-04-25T08:31:36.321711+0000","last_active":"2024-04-25T08:31:41.382287+0000","last_peered":"2024-04-25T08:31:41.382287+0000","last_clean":"2024-04-25T08:31:41.382287+0000","last_became_active":"2024-04-25T08:31:36.321446+0000","last_became_peered":"2024-04-25T08:31:36.321446+0000","last_unstale":"2024-04-25T08:31:41.382287+0000","last_undegraded":"2024-04-25T08:31:41.382287+0000","last_fullsized":"2024-04-25T08:31:41.382287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:47.451621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381253+0000","last_change":"2024-04-25T08:31:36.319922+0000","last_active":"2024-04-25T08:31:41.381253+0000","last_peered":"2024-04-25T08:31:41.381253+0000","last_clean":"2024-04-25T08:31:41.381253+0000","last_became_active":"2024-04-25T08:31:36.319650+0000","last_became_peered":"2024-04-25T08:31:36.319650+0000","last_unstale":"2024-04-25T08:31:41.381253+0000","last_undegraded":"2024-04-25T08:31:41.381253+0000","last_fullsized":"2024-04-25T08:31:41.381253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:23.257023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.381734+0000","last_change":"2024-04-25T08:31:36.319962+0000","last_active":"2024-04-25T08:31:41.381734+0000","last_peered":"2024-04-25T08:31:41.381734+0000","last_clean":"2024-04-25T08:31:41.381734+0000","last_became_active":"2024-04-25T08:31:36.319634+0000","last_became_peered":"2024-04-25T08:31:36.319634+0000","last_unstale":"2024-04-25T08:31:41.381734+0000","last_undegraded":"2024-04-25T08:31:41.381734+0000","last_fullsized":"2024-04-25T08:31:41.381734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:26.185783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422427+0000","last_change":"2024-04-25T08:31:36.320766+0000","last_active":"2024-04-25T08:31:41.422427+0000","last_peered":"2024-04-25T08:31:41.422427+0000","last_clean":"2024-04-25T08:31:41.422427+0000","last_became_active":"2024-04-25T08:31:36.320483+0000","last_became_peered":"2024-04-25T08:31:36.320483+0000","last_unstale":"2024-04-25T08:31:41.422427+0000","last_undegraded":"2024-04-25T08:31:41.422427+0000","last_fullsized":"2024-04-25T08:31:41.422427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:46:45.805418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.997517+0000","last_change":"2024-04-25T08:31:36.327853+0000","last_active":"2024-04-25T08:31:41.997517+0000","last_peered":"2024-04-25T08:31:41.997517+0000","last_clean":"2024-04-25T08:31:41.997517+0000","last_became_active":"2024-04-25T08:31:36.319850+0000","last_became_peered":"2024-04-25T08:31:36.319850+0000","last_unstale":"2024-04-25T08:31:41.997517+0000","last_undegraded":"2024-04-25T08:31:41.997517+0000","last_fullsized":"2024-04-25T08:31:41.997517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:37.299996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421704+0000","last_change":"2024-04-25T08:31:36.311917+0000","last_active":"2024-04-25T08:31:41.421704+0000","last_peered":"2024-04-25T08:31:41.421704+0000","last_clean":"2024-04-25T08:31:41.421704+0000","last_became_active":"2024-04-25T08:31:36.311657+0000","last_became_peered":"2024-04-25T08:31:36.311657+0000","last_unstale":"2024-04-25T08:31:41.421704+0000","last_undegraded":"2024-04-25T08:31:41.421704+0000","last_fullsized":"2024-04-25T08:31:41.421704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:12.212803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119023+0000","last_change":"2024-04-25T08:31:36.318421+0000","last_active":"2024-04-25T08:31:42.119023+0000","last_peered":"2024-04-25T08:31:42.119023+0000","last_clean":"2024-04-25T08:31:42.119023+0000","last_became_active":"2024-04-25T08:31:36.318157+0000","last_became_peered":"2024-04-25T08:31:36.318157+0000","last_unstale":"2024-04-25T08:31:42.119023+0000","last_undegraded":"2024-04-25T08:31:42.119023+0000","last_fullsized":"2024-04-25T08:31:42.119023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:26.185783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.421798+0000","last_change":"2024-04-25T08:31:36.314252+0000","last_active":"2024-04-25T08:31:41.421798+0000","last_peered":"2024-04-25T08:31:41.421798+0000","last_clean":"2024-04-25T08:31:41.421798+0000","last_became_active":"2024-04-25T08:31:36.313732+0000","last_became_peered":"2024-04-25T08:31:36.313732+0000","last_unstale":"2024-04-25T08:31:41.421798+0000","last_undegraded":"2024-04-25T08:31:41.421798+0000","last_fullsized":"2024-04-25T08:31:41.421798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:43.612460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.119289+0000","last_change":"2024-04-25T08:31:36.330053+0000","last_active":"2024-04-25T08:31:42.119289+0000","last_peered":"2024-04-25T08:31:42.119289+0000","last_clean":"2024-04-25T08:31:42.119289+0000","last_became_active":"2024-04-25T08:31:36.329775+0000","last_became_peered":"2024-04-25T08:31:36.329775+0000","last_unstale":"2024-04-25T08:31:42.119289+0000","last_undegraded":"2024-04-25T08:31:42.119289+0000","last_fullsized":"2024-04-25T08:31:42.119289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:12.212803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.365851+0000","last_change":"2024-04-25T08:31:36.317826+0000","last_active":"2024-04-25T08:31:42.365851+0000","last_peered":"2024-04-25T08:31:42.365851+0000","last_clean":"2024-04-25T08:31:42.365851+0000","last_became_active":"2024-04-25T08:31:36.317514+0000","last_became_peered":"2024-04-25T08:31:36.317514+0000","last_unstale":"2024-04-25T08:31:42.365851+0000","last_undegraded":"2024-04-25T08:31:42.365851+0000","last_fullsized":"2024-04-25T08:31:42.365851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:52.137039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:42.367390+0000","last_change":"2024-04-25T08:31:36.310887+0000","last_active":"2024-04-25T08:31:42.367390+0000","last_peered":"2024-04-25T08:31:42.367390+0000","last_clean":"2024-04-25T08:31:42.367390+0000","last_became_active":"2024-04-25T08:31:36.310581+0000","last_became_peered":"2024-04-25T08:31:36.310581+0000","last_unstale":"2024-04-25T08:31:42.367390+0000","last_undegraded":"2024-04-25T08:31:42.367390+0000","last_fullsized":"2024-04-25T08:31:42.367390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:36.016438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T08:31:41.422313+0000","last_change":"2024-04-25T08:31:36.315442+0000","last_active":"2024-04-25T08:31:41.422313+0000","last_peered":"2024-04-25T08:31:41.422313+0000","last_clean":"2024-04-25T08:31:41.422313+0000","last_became_active":"2024-04-25T08:31:36.315178+0000","last_became_peered":"2024-04-25T08:31:36.315178+0000","last_unstale":"2024-04-25T08:31:41.422313+0000","last_undegraded":"2024-04-25T08:31:41.422313+0000","last_fullsized":"2024-04-25T08:31:41.422313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:35.290098+0000","last_clean_scrub_stamp":"2024-04-25T08:31:35.290098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:52.353609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:31:56.528 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T08:31:56.528 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T08:31:56.529 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T08:31:56.529 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:31:56.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.669+0000 7fc2040dd640 1 Processor -- start 2024-04-25T08:31:56.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.669+0000 7fc2040dd640 1 -- start start 2024-04-25T08:31:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.670+0000 7fc2040dd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc1fc15dc10 0x7fc1fc0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.670+0000 7fc2040dd640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc0d3e70 con 0x7fc1fc0d0f60 2024-04-25T08:31:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.670+0000 7fc2040dd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc0d4150 con 0x7fc1fc0ced60 2024-04-25T08:31:56.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.670+0000 7fc2040dd640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1fc15dff0 con 0x7fc1fc15dc10 2024-04-25T08:31:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.670+0000 7fc20267a640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc1fc0d0f60 0x7fc1fc0b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49910/0 (socket says 172.21.15.38:49910) 2024-04-25T08:31:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.670+0000 7fc20267a640 1 -- 172.21.15.38:0/1176061118 learned_addr learned my addr 172.21.15.38:0/1176061118 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.670+0000 7fc201e79640 1 --2- 172.21.15.38:0/1176061118 >> v2:172.21.15.107:3300/0 conn(0x7fc1fc15dc10 0x7fc1fc0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.670+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143817820 0 0) 0x7fc1fc0d3e70 con 0x7fc1fc0d0f60 2024-04-25T08:31:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e8003520 con 0x7fc1fc0d0f60 2024-04-25T08:31:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1784599536 0 0) 0x7fc1e8003520 con 0x7fc1fc0d0f60 2024-04-25T08:31:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 >> v2:172.21.15.107:3300/0 conn(0x7fc1fc15dc10 msgr2=0x7fc1fc0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 --2- 172.21.15.38:0/1176061118 >> v2:172.21.15.107:3300/0 conn(0x7fc1fc15dc10 0x7fc1fc0ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 >> v1:172.21.15.107:6789/0 conn(0x7fc1fc0ced60 legacy=0x7fc1fc0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc0d4430 con 0x7fc1fc0d0f60 2024-04-25T08:31:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fc1f40019f0 con 0x7fc1fc0d0f60 2024-04-25T08:31:56.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 >> v1:172.21.15.38:6789/0 conn(0x7fc1fc0d0f60 legacy=0x7fc1fc0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 --2- 172.21.15.38:0/1176061118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e80039b0 0x7fc1e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 --2- 172.21.15.38:0/1176061118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 0x7fc1e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 --2- 172.21.15.38:0/1176061118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e80051c0 0x7fc1e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc15dff0 con 0x7fc1e8004800 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e800c160 con 0x7fc1e80039b0 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e800c360 con 0x7fc1e80051c0 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc201678640 1 --2- 172.21.15.38:0/1176061118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 0x7fc1e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc201e79640 1 --2- 172.21.15.38:0/1176061118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e80039b0 0x7fc1e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc20267a640 1 --2- 172.21.15.38:0/1176061118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e80051c0 0x7fc1e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc201678640 1 -- 172.21.15.38:0/1176061118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e80051c0 msgr2=0x7fc1e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc201678640 1 --2- 172.21.15.38:0/1176061118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e80051c0 0x7fc1e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc201678640 1 -- 172.21.15.38:0/1176061118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e80039b0 msgr2=0x7fc1e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:56.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc201678640 1 --2- 172.21.15.38:0/1176061118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e80039b0 0x7fc1e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc201678640 1 -- 172.21.15.38:0/1176061118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e8003520 con 0x7fc1e8004800 2024-04-25T08:31:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc20267a640 1 --2- 172.21.15.38:0/1176061118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e80051c0 0x7fc1e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.671+0000 7fc20267a640 1 -- 172.21.15.38:0/1176061118 reap_dead start 2024-04-25T08:31:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.672+0000 7fc201678640 1 --2- 172.21.15.38:0/1176061118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 0x7fc1e8004c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec001750 tx=0x7fc1e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72472a02f3136b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.672+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec001370 con 0x7fc1e8004800 2024-04-25T08:31:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.672+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ec0052d0 con 0x7fc1e8004800 2024-04-25T08:31:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.672+0000 7fc200e77640 1 -- 172.21.15.38:0/1176061118 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec02abc0 con 0x7fc1e8004800 2024-04-25T08:31:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.673+0000 7fc2040dd640 1 -- 172.21.15.38:0/1176061118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 msgr2=0x7fc1e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.673+0000 7fc2040dd640 1 --2- 172.21.15.38:0/1176061118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 0x7fc1e8004c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec001750 tx=0x7fc1e8005830 comp rx=0 tx=0).stop 2024-04-25T08:31:56.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.673+0000 7fc2040dd640 1 -- 172.21.15.38:0/1176061118 shutdown_connections 2024-04-25T08:31:56.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.673+0000 7fc2040dd640 1 --2- 172.21.15.38:0/1176061118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 0x7fc1e8004c80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.673+0000 7fc2040dd640 1 -- 172.21.15.38:0/1176061118 >> 172.21.15.38:0/1176061118 conn(0x7fc1fc066d10 msgr2=0x7fc1fc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:56.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.673+0000 7fc2040dd640 1 -- 172.21.15.38:0/1176061118 shutdown_connections 2024-04-25T08:31:56.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.673+0000 7fc2040dd640 1 -- 172.21.15.38:0/1176061118 wait complete. 2024-04-25T08:31:56.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.674+0000 7fc2040dd640 1 Processor -- start 2024-04-25T08:31:56.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.674+0000 7fc2040dd640 1 -- start start 2024-04-25T08:31:56.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc2040dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 0x7fc1fc157240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc2040dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1fc0c9b70 0x7fc1fc0c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc2040dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1fc0cba20 0x7fc1fc0cbea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc2040dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0d4710 con 0x7fc1e8004800 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc2040dd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0d3900 con 0x7fc1fc0cba20 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc2040dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0d3a80 con 0x7fc1fc0c9b70 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc201e79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 0x7fc1fc157240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc201e79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 0x7fc1fc157240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33680/0 (socket says 172.21.15.38:33680) 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc201e79640 1 -- 172.21.15.38:0/3122031184 learned_addr learned my addr 172.21.15.38:0/3122031184 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc20267a640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1fc0cba20 0x7fc1fc0cbea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc201678640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1fc0c9b70 0x7fc1fc0c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc201e79640 1 -- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1fc0c9b70 msgr2=0x7fc1fc0c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc201e79640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1fc0c9b70 0x7fc1fc0c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc201e79640 1 -- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1fc0cba20 msgr2=0x7fc1fc0cbea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc201e79640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1fc0cba20 0x7fc1fc0cbea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc201e79640 1 -- 172.21.15.38:0/3122031184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc159f00 con 0x7fc1e8004800 2024-04-25T08:31:56.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc201e79640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 0x7fc1fc157240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8011a70 tx=0x7fc1f8011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.675+0000 7fc1f27fc640 1 -- 172.21.15.38:0/3122031184 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800fb50 con 0x7fc1e8004800 2024-04-25T08:31:56.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.676+0000 7fc1f27fc640 1 -- 172.21.15.38:0/3122031184 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f800fcf0 con 0x7fc1e8004800 2024-04-25T08:31:56.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.676+0000 7fc1f27fc640 1 -- 172.21.15.38:0/3122031184 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f800fea0 con 0x7fc1e8004800 2024-04-25T08:31:56.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.676+0000 7fc2040dd640 1 -- 172.21.15.38:0/3122031184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc15a1e0 con 0x7fc1e8004800 2024-04-25T08:31:56.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.676+0000 7fc2040dd640 1 -- 172.21.15.38:0/3122031184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc1543b0 con 0x7fc1e8004800 2024-04-25T08:31:56.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.676+0000 7fc2040dd640 1 -- 172.21.15.38:0/3122031184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1fc068e80 con 0x7fc1e8004800 2024-04-25T08:31:56.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.677+0000 7fc1f27fc640 1 -- 172.21.15.38:0/3122031184 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1f8012040 con 0x7fc1e8004800 2024-04-25T08:31:56.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.677+0000 7fc1f27fc640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1e006b4d0 0x7fc1e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:56.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.677+0000 7fc1f27fc640 1 -- 172.21.15.38:0/3122031184 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc1f808ab10 con 0x7fc1e8004800 2024-04-25T08:31:56.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.678+0000 7fc201678640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1e006b4d0 0x7fc1e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:56.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.678+0000 7fc201678640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1e006b4d0 0x7fc1e006d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec032000 tx=0x7fc1ec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:56.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.681+0000 7fc1f27fc640 1 -- 172.21.15.38:0/3122031184 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1f8019030 con 0x7fc1e8004800 2024-04-25T08:31:56.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.901+0000 7fc2040dd640 1 -- 172.21.15.38:0/3122031184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc1fc0cda00 con 0x7fc1e8004800 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.901+0000 7fc1f27fc640 1 -- 172.21.15.38:0/3122031184 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc1f8059090 con 0x7fc1e8004800 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:56.898 INFO:teuthology.orchestra.run.smithi038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T08:31:56.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.904+0000 7fc2040dd640 1 -- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1e006b4d0 msgr2=0x7fc1e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.904+0000 7fc2040dd640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1e006b4d0 0x7fc1e006d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec032000 tx=0x7fc1ec02e000 comp rx=0 tx=0).stop 2024-04-25T08:31:56.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.904+0000 7fc2040dd640 1 -- 172.21.15.38:0/3122031184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 msgr2=0x7fc1fc157240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:56.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.904+0000 7fc2040dd640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 0x7fc1fc157240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8011a70 tx=0x7fc1f8011f30 comp rx=0 tx=0).stop 2024-04-25T08:31:56.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.904+0000 7fc2040dd640 1 -- 172.21.15.38:0/3122031184 shutdown_connections 2024-04-25T08:31:56.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.904+0000 7fc2040dd640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1fc0cba20 0x7fc1fc0cbea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.904+0000 7fc2040dd640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1e006b4d0 0x7fc1e006d990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.904+0000 7fc2040dd640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1fc0c9b70 0x7fc1fc0c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.904+0000 7fc2040dd640 1 --2- 172.21.15.38:0/3122031184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e8004800 0x7fc1fc157240 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:56.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.904+0000 7fc2040dd640 1 -- 172.21.15.38:0/3122031184 >> 172.21.15.38:0/3122031184 conn(0x7fc1fc066d10 msgr2=0x7fc1fc15e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:56.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.904+0000 7fc2040dd640 1 -- 172.21.15.38:0/3122031184 shutdown_connections 2024-04-25T08:31:56.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:56.904+0000 7fc2040dd640 1 -- 172.21.15.38:0/3122031184 wait complete. 2024-04-25T08:31:56.911 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T08:31:56.911 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:57.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.016+0000 7f5ee5b53640 1 Processor -- start 2024-04-25T08:31:57.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.016+0000 7f5ee5b53640 1 -- start start 2024-04-25T08:31:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5ee5b53640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5ee006c110 0x7f5ee006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5ee5b53640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ee00ce2e0 0x7f5ee00d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5ee5b53640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee006ca30 con 0x7f5ee015df50 2024-04-25T08:31:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5ee5b53640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee00cf2d0 con 0x7f5ee006c110 2024-04-25T08:31:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5ee5b53640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee00ceff0 con 0x7f5ee00ce2e0 2024-04-25T08:31:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5edf7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5ee006c110 0x7f5ee006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5edeffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5ee015df50 0x7f5ee00ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49918/0 (socket says 172.21.15.38:49918) 2024-04-25T08:31:57.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5edeffd640 1 -- 172.21.15.38:0/3046648472 learned_addr learned my addr 172.21.15.38:0/3046648472 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5edffff640 1 --2- 172.21.15.38:0/3046648472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ee00ce2e0 0x7f5ee00d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5edf7fe640 1 -- 172.21.15.38:0/3046648472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ee00ce2e0 msgr2=0x7f5ee00d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5edf7fe640 1 --2- 172.21.15.38:0/3046648472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ee00ce2e0 0x7f5ee00d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5edf7fe640 1 -- 172.21.15.38:0/3046648472 >> v1:172.21.15.38:6789/0 conn(0x7f5ee015df50 legacy=0x7f5ee00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5edf7fe640 1 -- 172.21.15.38:0/3046648472 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee00d1f70 con 0x7f5ee006c110 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.017+0000 7f5ede7fc640 1 -- 172.21.15.38:0/3046648472 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2182234327 0 0) 0x7f5ee006ca30 con 0x7f5ee015df50 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.018+0000 7f5edf7fe640 1 --2- 172.21.15.38:0/3046648472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ee006c110 0x7f5ee006c4f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00097a0 tx=0x7f5ed002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704d510b06d3c99e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.018+0000 7f5ede7fc640 1 -- 172.21.15.38:0/3046648472 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0037070 con 0x7f5ee006c110 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.018+0000 7f5ede7fc640 1 -- 172.21.15.38:0/3046648472 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed0032440 con 0x7f5ee006c110 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.018+0000 7f5ede7fc640 1 -- 172.21.15.38:0/3046648472 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0032810 con 0x7f5ee006c110 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.019+0000 7f5ee5b53640 1 -- 172.21.15.38:0/3046648472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ee006c110 msgr2=0x7f5ee006c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.019+0000 7f5ee5b53640 1 --2- 172.21.15.38:0/3046648472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ee006c110 0x7f5ee006c4f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00097a0 tx=0x7f5ed002a180 comp rx=0 tx=0).stop 2024-04-25T08:31:57.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.019+0000 7f5ee5b53640 1 -- 172.21.15.38:0/3046648472 shutdown_connections 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.019+0000 7f5ee5b53640 1 --2- 172.21.15.38:0/3046648472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ee00ce2e0 0x7f5ee00d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.019+0000 7f5ee5b53640 1 --2- 172.21.15.38:0/3046648472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ee006c110 0x7f5ee006c4f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.019+0000 7f5ee5b53640 1 -- 172.21.15.38:0/3046648472 >> 172.21.15.38:0/3046648472 conn(0x7f5ee00d3fb0 msgr2=0x7f5ee00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.019+0000 7f5ee5b53640 1 -- 172.21.15.38:0/3046648472 shutdown_connections 2024-04-25T08:31:57.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.019+0000 7f5ee5b53640 1 -- 172.21.15.38:0/3046648472 wait complete. 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.020+0000 7f5ee5b53640 1 Processor -- start 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.020+0000 7f5ee5b53640 1 -- start start 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5ee5b53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ee006c110 0x7f5ee00b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5ee5b53640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ee00ce2e0 0x7f5ee01f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5ee5b53640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee015df50 0x7f5ee014cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5ee5b53640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee00d2f90 con 0x7f5ee006c110 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5ee5b53640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee00d2530 con 0x7f5ee015df50 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5ee5b53640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee00d3110 con 0x7f5ee00ce2e0 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5edf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ee006c110 0x7f5ee00b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5edf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ee006c110 0x7f5ee00b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33700/0 (socket says 172.21.15.38:33700) 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5edf7fe640 1 -- 172.21.15.38:0/2075512540 learned_addr learned my addr 172.21.15.38:0/2075512540 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5edeffd640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ee00ce2e0 0x7f5ee01f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5edffff640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee015df50 0x7f5ee014cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5edf7fe640 1 -- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ee00ce2e0 msgr2=0x7f5ee01f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5edf7fe640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ee00ce2e0 0x7f5ee01f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5edf7fe640 1 -- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee015df50 msgr2=0x7f5ee014cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5edf7fe640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee015df50 0x7f5ee014cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5edf7fe640 1 -- 172.21.15.38:0/2075512540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee01f4060 con 0x7f5ee006c110 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.021+0000 7f5edf7fe640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ee006c110 0x7f5ee00b7760 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00097a0 tx=0x7f5ed00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.022+0000 7f5ed7fff640 1 -- 172.21.15.38:0/2075512540 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0037050 con 0x7f5ee006c110 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.022+0000 7f5ed7fff640 1 -- 172.21.15.38:0/2075512540 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed0038040 con 0x7f5ee006c110 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.022+0000 7f5ee5b53640 1 -- 172.21.15.38:0/2075512540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee01f4340 con 0x7f5ee006c110 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.022+0000 7f5ed7fff640 1 -- 172.21.15.38:0/2075512540 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0004430 con 0x7f5ee006c110 2024-04-25T08:31:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.022+0000 7f5ee5b53640 1 -- 172.21.15.38:0/2075512540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee015e4d0 con 0x7f5ee006c110 2024-04-25T08:31:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.023+0000 7f5ee5b53640 1 -- 172.21.15.38:0/2075512540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ee01f4860 con 0x7f5ee006c110 2024-04-25T08:31:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.023+0000 7f5ed7fff640 1 -- 172.21.15.38:0/2075512540 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ed003c6d0 con 0x7f5ee006c110 2024-04-25T08:31:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.024+0000 7f5ed7fff640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5ec006b4d0 0x7f5ec006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.024+0000 7f5ed7fff640 1 -- 172.21.15.38:0/2075512540 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5ed00aae10 con 0x7f5ee006c110 2024-04-25T08:31:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.024+0000 7f5edeffd640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5ec006b4d0 0x7f5ec006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.024+0000 7f5edeffd640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5ec006b4d0 0x7f5ec006d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5ee015faf0 tx=0x7f5ec8007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.028+0000 7f5ed7fff640 1 -- 172.21.15.38:0/2075512540 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ee01f4860 con 0x7f5ee006c110 2024-04-25T08:31:57.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.211+0000 7f5ee5b53640 1 -- 172.21.15.38:0/2075512540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ee0072e40 con 0x7f5ee006c110 2024-04-25T08:31:57.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.212+0000 7f5ed7fff640 1 -- 172.21.15.38:0/2075512540 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f5ee0072e40 con 0x7f5ee006c110 2024-04-25T08:31:57.209 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:57.209 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.215+0000 7f5ee5b53640 1 -- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5ec006b4d0 msgr2=0x7f5ec006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.215+0000 7f5ee5b53640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5ec006b4d0 0x7f5ec006d990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5ee015faf0 tx=0x7f5ec8007de0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.215+0000 7f5ee5b53640 1 -- 172.21.15.38:0/2075512540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ee006c110 msgr2=0x7f5ee00b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.215+0000 7f5ee5b53640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ee006c110 0x7f5ee00b7760 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5ed00097a0 tx=0x7f5ed00046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.215+0000 7f5ee5b53640 1 -- 172.21.15.38:0/2075512540 shutdown_connections 2024-04-25T08:31:57.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.215+0000 7f5ee5b53640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ee015df50 0x7f5ee014cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.215+0000 7f5ee5b53640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5ec006b4d0 0x7f5ec006d990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.215+0000 7f5ee5b53640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ee00ce2e0 0x7f5ee01f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.215+0000 7f5ee5b53640 1 --2- 172.21.15.38:0/2075512540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ee006c110 0x7f5ee00b7760 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.215+0000 7f5ee5b53640 1 -- 172.21.15.38:0/2075512540 >> 172.21.15.38:0/2075512540 conn(0x7f5ee00d3fb0 msgr2=0x7f5ee015cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.215+0000 7f5ee5b53640 1 -- 172.21.15.38:0/2075512540 shutdown_connections 2024-04-25T08:31:57.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.215+0000 7f5ee5b53640 1 -- 172.21.15.38:0/2075512540 wait complete. 2024-04-25T08:31:57.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:31:57.223 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:31:57.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.331+0000 7f304b8f5640 1 Processor -- start 2024-04-25T08:31:57.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.331+0000 7f304b8f5640 1 -- start start 2024-04-25T08:31:57.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f304b8f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f30440d0320 0x7f30440d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f304b8f5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30440d2060 con 0x7f30440d0700 2024-04-25T08:31:57.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f304b8f5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30440d2330 con 0x7f30440b6980 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f304b8f5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30440d2610 con 0x7f30440d0320 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f3048e90640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f30440d0700 0x7f30440d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49924/0 (socket says 172.21.15.38:49924) 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f3048e90640 1 -- 172.21.15.38:0/2071698737 learned_addr learned my addr 172.21.15.38:0/2071698737 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421988429 0 0) 0x7f30440d2060 con 0x7f30440d0700 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3030003520 con 0x7f30440d0700 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f3049691640 1 --2- 172.21.15.38:0/2071698737 >> v2:172.21.15.107:3300/0 conn(0x7f30440d0320 0x7f30440d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 381765956 0 0) 0x7f3030003520 con 0x7f30440d0700 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 >> v2:172.21.15.107:3300/0 conn(0x7f30440d0320 msgr2=0x7f30440d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f303bfff640 1 --2- 172.21.15.38:0/2071698737 >> v2:172.21.15.107:3300/0 conn(0x7f30440d0320 0x7f30440d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 >> v1:172.21.15.107:6789/0 conn(0x7f30440b6980 legacy=0x7f30440b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30440d28f0 con 0x7f30440d0700 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f3034001670 con 0x7f30440d0700 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 >> v1:172.21.15.38:6789/0 conn(0x7f30440d0700 legacy=0x7f30440d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f303bfff640 1 --2- 172.21.15.38:0/2071698737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30300039b0 0x7f3030003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f303bfff640 1 --2- 172.21.15.38:0/2071698737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3030004800 0x7f3030004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.332+0000 7f303bfff640 1 --2- 172.21.15.38:0/2071698737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30300051c0 0x7f303000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.333+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30440d2610 con 0x7f3030004800 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.333+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303000c070 con 0x7f30300039b0 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.333+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303000c270 con 0x7f30300051c0 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.333+0000 7f3048e90640 1 --2- 172.21.15.38:0/2071698737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3030004800 0x7f3030004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.333+0000 7f3049691640 1 --2- 172.21.15.38:0/2071698737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30300039b0 0x7f3030003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.333+0000 7f3049e92640 1 --2- 172.21.15.38:0/2071698737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30300051c0 0x7f303000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.333+0000 7f3048e90640 1 -- 172.21.15.38:0/2071698737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30300051c0 msgr2=0x7f303000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.333+0000 7f3048e90640 1 --2- 172.21.15.38:0/2071698737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30300051c0 0x7f303000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.333+0000 7f3048e90640 1 -- 172.21.15.38:0/2071698737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30300039b0 msgr2=0x7f3030003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.333+0000 7f3048e90640 1 --2- 172.21.15.38:0/2071698737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30300039b0 0x7f3030003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.333+0000 7f3049e92640 1 -- 172.21.15.38:0/2071698737 reap_dead start 2024-04-25T08:31:57.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.333+0000 7f3048e90640 1 -- 172.21.15.38:0/2071698737 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3030003520 con 0x7f3030004800 2024-04-25T08:31:57.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.334+0000 7f3048e90640 1 --2- 172.21.15.38:0/2071698737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3030004800 0x7f3030004c80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f303400c240 tx=0x7f3030005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7382b8414814862a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.334+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034007d10 con 0x7f3030004800 2024-04-25T08:31:57.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.334+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3034005ad0 con 0x7f3030004800 2024-04-25T08:31:57.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.334+0000 7f303bfff640 1 -- 172.21.15.38:0/2071698737 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034007070 con 0x7f3030004800 2024-04-25T08:31:57.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.334+0000 7f304b8f5640 1 -- 172.21.15.38:0/2071698737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3030004800 msgr2=0x7f3030004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.334+0000 7f304b8f5640 1 --2- 172.21.15.38:0/2071698737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3030004800 0x7f3030004c80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f303400c240 tx=0x7f3030005830 comp rx=0 tx=0).stop 2024-04-25T08:31:57.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.334+0000 7f304b8f5640 1 -- 172.21.15.38:0/2071698737 shutdown_connections 2024-04-25T08:31:57.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.334+0000 7f304b8f5640 1 --2- 172.21.15.38:0/2071698737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3030004800 0x7f3030004c80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.334+0000 7f304b8f5640 1 -- 172.21.15.38:0/2071698737 >> 172.21.15.38:0/2071698737 conn(0x7f30440668f0 msgr2=0x7f3044066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.334+0000 7f304b8f5640 1 -- 172.21.15.38:0/2071698737 shutdown_connections 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.334+0000 7f304b8f5640 1 -- 172.21.15.38:0/2071698737 wait complete. 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.336+0000 7f304b8f5640 1 Processor -- start 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.336+0000 7f304b8f5640 1 -- start start 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.336+0000 7f304b8f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3030004800 0x7f304415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.336+0000 7f304b8f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3044159730 0x7f30441f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.336+0000 7f304b8f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f304415a580 0x7f30441f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.336+0000 7f304b8f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30440d0b40 con 0x7f3044159730 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.336+0000 7f304b8f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30440d2eb0 con 0x7f3030004800 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.336+0000 7f304b8f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304415c750 con 0x7f304415a580 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.336+0000 7f3048e90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3044159730 0x7f30441f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.336+0000 7f3048e90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3044159730 0x7f30441f3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33720/0 (socket says 172.21.15.38:33720) 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.336+0000 7f3048e90640 1 -- 172.21.15.38:0/227365237 learned_addr learned my addr 172.21.15.38:0/227365237 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f3049691640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3030004800 0x7f304415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f3049e92640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f304415a580 0x7f30441f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f3048e90640 1 -- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f304415a580 msgr2=0x7f30441f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f3048e90640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f304415a580 0x7f30441f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f3048e90640 1 -- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3030004800 msgr2=0x7f304415c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f3048e90640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3030004800 0x7f304415c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f3048e90640 1 -- 172.21.15.38:0/227365237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3044156800 con 0x7f3044159730 2024-04-25T08:31:57.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f3049691640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3030004800 0x7f304415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:57.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f3048e90640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3044159730 0x7f30441f3bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f303400c240 tx=0x7f30440b68d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f3039ffb640 1 -- 172.21.15.38:0/227365237 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034041070 con 0x7f3044159730 2024-04-25T08:31:57.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f3039ffb640 1 -- 172.21.15.38:0/227365237 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3034000ed0 con 0x7f3044159730 2024-04-25T08:31:57.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f304b8f5640 1 -- 172.21.15.38:0/227365237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3044156ae0 con 0x7f3044159730 2024-04-25T08:31:57.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f3039ffb640 1 -- 172.21.15.38:0/227365237 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034007d10 con 0x7f3044159730 2024-04-25T08:31:57.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.337+0000 7f304b8f5640 1 -- 172.21.15.38:0/227365237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3044158f30 con 0x7f3044159730 2024-04-25T08:31:57.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.338+0000 7f304b8f5640 1 -- 172.21.15.38:0/227365237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30440cf730 con 0x7f3044159730 2024-04-25T08:31:57.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.339+0000 7f3039ffb640 1 -- 172.21.15.38:0/227365237 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3034044070 con 0x7f3044159730 2024-04-25T08:31:57.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.339+0000 7f3039ffb640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f302806b4d0 0x7f302806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.339+0000 7f3039ffb640 1 -- 172.21.15.38:0/227365237 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f30340aac90 con 0x7f3044159730 2024-04-25T08:31:57.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.339+0000 7f3049691640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f302806b4d0 0x7f302806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.340+0000 7f3049691640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f302806b4d0 0x7f302806d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f30440692d0 tx=0x7f3040008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.343+0000 7f3039ffb640 1 -- 172.21.15.38:0/227365237 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30440cf730 con 0x7f3044159730 2024-04-25T08:31:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.523+0000 7f304b8f5640 1 -- 172.21.15.38:0/227365237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3044072e40 con 0x7f3044159730 2024-04-25T08:31:57.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.523+0000 7f3039ffb640 1 -- 172.21.15.38:0/227365237 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3044072e40 con 0x7f3044159730 2024-04-25T08:31:57.519 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:31:57.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.525+0000 7f304b8f5640 1 -- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f302806b4d0 msgr2=0x7f302806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.525+0000 7f304b8f5640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f302806b4d0 0x7f302806d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f30440692d0 tx=0x7f3040008040 comp rx=0 tx=0).stop 2024-04-25T08:31:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.525+0000 7f304b8f5640 1 -- 172.21.15.38:0/227365237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3044159730 msgr2=0x7f30441f3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.525+0000 7f304b8f5640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3044159730 0x7f30441f3bd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f303400c240 tx=0x7f30440b68d0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.525+0000 7f304b8f5640 1 -- 172.21.15.38:0/227365237 shutdown_connections 2024-04-25T08:31:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.525+0000 7f304b8f5640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f302806b4d0 0x7f302806d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.525+0000 7f304b8f5640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f304415a580 0x7f30441f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.525+0000 7f304b8f5640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3044159730 0x7f30441f3bd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.525+0000 7f304b8f5640 1 --2- 172.21.15.38:0/227365237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3030004800 0x7f304415c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.525+0000 7f304b8f5640 1 -- 172.21.15.38:0/227365237 >> 172.21.15.38:0/227365237 conn(0x7f30440668f0 msgr2=0x7f30440c9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.525+0000 7f304b8f5640 1 -- 172.21.15.38:0/227365237 shutdown_connections 2024-04-25T08:31:57.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.525+0000 7f304b8f5640 1 -- 172.21.15.38:0/227365237 wait complete. 2024-04-25T08:31:57.532 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:31:57.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.639+0000 7f4e5f5b0640 1 Processor -- start 2024-04-25T08:31:57.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.639+0000 7f4e5f5b0640 1 -- start start 2024-04-25T08:31:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.640+0000 7f4e5f5b0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e580d0320 0x7f4e580d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.640+0000 7f4e5f5b0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4e580d0700 0x7f4e580d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.640+0000 7f4e5f5b0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e580d2330 con 0x7f4e580d1e90 2024-04-25T08:31:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.640+0000 7f4e5f5b0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e580d2610 con 0x7f4e580d0700 2024-04-25T08:31:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.640+0000 7f4e5f5b0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580d28f0 con 0x7f4e580d0320 2024-04-25T08:31:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.640+0000 7f4e5db4d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4e580d1e90 0x7f4e580b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49934/0 (socket says 172.21.15.38:49934) 2024-04-25T08:31:57.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.640+0000 7f4e5db4d640 1 -- 172.21.15.38:0/650139063 learned_addr learned my addr 172.21.15.38:0/650139063 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.640+0000 7f4e5d34c640 1 --2- 172.21.15.38:0/650139063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e580d0320 0x7f4e580d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.640+0000 7f4e5cb4b640 1 --2- 172.21.15.38:0/650139063 >> v2:172.21.15.107:3300/0 conn(0x7f4e580d0700 0x7f4e580d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.640+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753140824 0 0) 0x7f4e580d2330 con 0x7f4e580d1e90 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e40003520 con 0x7f4e580d1e90 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3539084248 0 0) 0x7f4e40003520 con 0x7f4e580d1e90 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e580d0320 msgr2=0x7f4e580d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 --2- 172.21.15.38:0/650139063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e580d0320 0x7f4e580d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 >> v2:172.21.15.107:3300/0 conn(0x7f4e580d0700 msgr2=0x7f4e580d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 --2- 172.21.15.38:0/650139063 >> v2:172.21.15.107:3300/0 conn(0x7f4e580d0700 0x7f4e580d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e580d2bd0 con 0x7f4e580d1e90 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f4e480019f0 con 0x7f4e580d1e90 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 >> v1:172.21.15.38:6789/0 conn(0x7f4e580d1e90 legacy=0x7f4e580b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 --2- 172.21.15.38:0/650139063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e580d0320 0x7f4e400039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 --2- 172.21.15.38:0/650139063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e40003f30 0x7f4e40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 --2- 172.21.15.38:0/650139063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e40004d40 0x7f4e400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580d2610 con 0x7f4e40003f30 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580d28f0 con 0x7f4e40004d40 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4000b9f0 con 0x7f4e580d0320 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e5cb4b640 1 --2- 172.21.15.38:0/650139063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e40003f30 0x7f4e40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e5d34c640 1 --2- 172.21.15.38:0/650139063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e580d0320 0x7f4e400039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e5db4d640 1 --2- 172.21.15.38:0/650139063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e40004d40 0x7f4e400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e5cb4b640 1 -- 172.21.15.38:0/650139063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e580d0320 msgr2=0x7f4e400039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e5cb4b640 1 --2- 172.21.15.38:0/650139063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e580d0320 0x7f4e400039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e5cb4b640 1 -- 172.21.15.38:0/650139063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e40004d40 msgr2=0x7f4e400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e5cb4b640 1 --2- 172.21.15.38:0/650139063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e40004d40 0x7f4e400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.641+0000 7f4e5cb4b640 1 -- 172.21.15.38:0/650139063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e40003520 con 0x7f4e40003f30 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.642+0000 7f4e5cb4b640 1 --2- 172.21.15.38:0/650139063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e40003f30 0x7f4e40004800 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009ac0 tx=0x7f4e40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19feafead7df30c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.642+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c036070 con 0x7f4e40003f30 2024-04-25T08:31:57.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.642+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4c031400 con 0x7f4e40003f30 2024-04-25T08:31:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.642+0000 7f4e57fff640 1 -- 172.21.15.38:0/650139063 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c009690 con 0x7f4e40003f30 2024-04-25T08:31:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.643+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/650139063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e40003f30 msgr2=0x7f4e40004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.643+0000 7f4e5f5b0640 1 --2- 172.21.15.38:0/650139063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e40003f30 0x7f4e40004800 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009ac0 tx=0x7f4e40005830 comp rx=0 tx=0).stop 2024-04-25T08:31:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.643+0000 7f4e5db4d640 1 -- 172.21.15.38:0/650139063 reap_dead start 2024-04-25T08:31:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.643+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/650139063 shutdown_connections 2024-04-25T08:31:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.643+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/650139063 >> 172.21.15.38:0/650139063 conn(0x7f4e580668f0 msgr2=0x7f4e58066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.643+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/650139063 shutdown_connections 2024-04-25T08:31:57.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.643+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/650139063 wait complete. 2024-04-25T08:31:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.644+0000 7f4e5f5b0640 1 Processor -- start 2024-04-25T08:31:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.644+0000 7f4e5f5b0640 1 -- start start 2024-04-25T08:31:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5f5b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e40003f30 0x7f4e581f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5f5b0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e581f6830 0x7f4e580ac420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5f5b0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e580ac960 0x7f4e580b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5f5b0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e580d0b40 con 0x7f4e40003f30 2024-04-25T08:31:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5f5b0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e58161210 con 0x7f4e580ac960 2024-04-25T08:31:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5f5b0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e581f4070 con 0x7f4e581f6830 2024-04-25T08:31:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5d34c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e40003f30 0x7f4e581f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5d34c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e40003f30 0x7f4e581f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33736/0 (socket says 172.21.15.38:33736) 2024-04-25T08:31:57.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5d34c640 1 -- 172.21.15.38:0/3417113565 learned_addr learned my addr 172.21.15.38:0/3417113565 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5db4d640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e580ac960 0x7f4e580b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5cb4b640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e581f6830 0x7f4e580ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5d34c640 1 -- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e581f6830 msgr2=0x7f4e580ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5d34c640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e581f6830 0x7f4e580ac420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5d34c640 1 -- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e580ac960 msgr2=0x7f4e580b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5d34c640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e580ac960 0x7f4e580b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5d34c640 1 -- 172.21.15.38:0/3417113565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e580b4a20 con 0x7f4e40003f30 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5d34c640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e40003f30 0x7f4e581f3b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e500119a0 tx=0x7f4e50011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e55ffb640 1 -- 172.21.15.38:0/3417113565 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5000fa30 con 0x7f4e40003f30 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e55ffb640 1 -- 172.21.15.38:0/3417113565 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5000fbd0 con 0x7f4e40003f30 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/3417113565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e580aa4b0 con 0x7f4e40003f30 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.645+0000 7f4e55ffb640 1 -- 172.21.15.38:0/3417113565 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e50014440 con 0x7f4e40003f30 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.646+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/3417113565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e580aaa70 con 0x7f4e40003f30 2024-04-25T08:31:57.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.647+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/3417113565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e580aa2a0 con 0x7f4e40003f30 2024-04-25T08:31:57.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.647+0000 7f4e55ffb640 1 -- 172.21.15.38:0/3417113565 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e50012040 con 0x7f4e40003f30 2024-04-25T08:31:57.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.648+0000 7f4e55ffb640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e4406b500 0x7f4e4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.648+0000 7f4e55ffb640 1 -- 172.21.15.38:0/3417113565 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4e5008a9d0 con 0x7f4e40003f30 2024-04-25T08:31:57.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.648+0000 7f4e5cb4b640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e4406b500 0x7f4e4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.648+0000 7f4e5cb4b640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e4406b500 0x7f4e4406d9c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c033000 tx=0x7f4e4c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.652+0000 7f4e55ffb640 1 -- 172.21.15.38:0/3417113565 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e50019030 con 0x7f4e40003f30 2024-04-25T08:31:57.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.832+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/3417113565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e58068e80 con 0x7f4e40003f30 2024-04-25T08:31:57.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.832+0000 7f4e55ffb640 1 -- 172.21.15.38:0/3417113565 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e500595a0 con 0x7f4e40003f30 2024-04-25T08:31:57.829 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:31:57.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.834+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e4406b500 msgr2=0x7f4e4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.834+0000 7f4e5f5b0640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e4406b500 0x7f4e4406d9c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c033000 tx=0x7f4e4c008030 comp rx=0 tx=0).stop 2024-04-25T08:31:57.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.834+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/3417113565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e40003f30 msgr2=0x7f4e581f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.834+0000 7f4e5f5b0640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e40003f30 0x7f4e581f3b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4e500119a0 tx=0x7f4e50011da0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.834+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/3417113565 shutdown_connections 2024-04-25T08:31:57.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.834+0000 7f4e5f5b0640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e580ac960 0x7f4e580b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.834+0000 7f4e5f5b0640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e4406b500 0x7f4e4406d9c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.834+0000 7f4e5f5b0640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e581f6830 0x7f4e580ac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.834+0000 7f4e5f5b0640 1 --2- 172.21.15.38:0/3417113565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e40003f30 0x7f4e581f3b30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.834+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/3417113565 >> 172.21.15.38:0/3417113565 conn(0x7f4e580668f0 msgr2=0x7f4e5815e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.834+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/3417113565 shutdown_connections 2024-04-25T08:31:57.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.834+0000 7f4e5f5b0640 1 -- 172.21.15.38:0/3417113565 wait complete. 2024-04-25T08:31:57.841 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:31:57.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.948+0000 7f5d974e1640 1 Processor -- start 2024-04-25T08:31:57.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.948+0000 7f5d974e1640 1 -- start start 2024-04-25T08:31:57.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d974e1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5d900d0320 0x7f5d900d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d974e1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d900b6980 0x7f5d900b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d974e1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d900d2060 con 0x7f5d900d0700 2024-04-25T08:31:57.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d974e1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d900d2330 con 0x7f5d900d0320 2024-04-25T08:31:57.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d974e1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d900d2610 con 0x7f5d900b6980 2024-04-25T08:31:57.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d94a7c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5d900d0700 0x7f5d900d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49942/0 (socket says 172.21.15.38:49942) 2024-04-25T08:31:57.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d94a7c640 1 -- 172.21.15.38:0/156068557 learned_addr learned my addr 172.21.15.38:0/156068557 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:57.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d9527d640 1 --2- 172.21.15.38:0/156068557 >> v2:172.21.15.107:3300/0 conn(0x7f5d900d0320 0x7f5d900d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d95a7e640 1 --2- 172.21.15.38:0/156068557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d900b6980 0x7f5d900b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d87fff640 1 -- 172.21.15.38:0/156068557 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158152549 0 0) 0x7f5d900d2060 con 0x7f5d900d0700 2024-04-25T08:31:57.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d87fff640 1 -- 172.21.15.38:0/156068557 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d7c003520 con 0x7f5d900d0700 2024-04-25T08:31:57.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d87fff640 1 -- 172.21.15.38:0/156068557 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1370931963 0 0) 0x7f5d7c003520 con 0x7f5d900d0700 2024-04-25T08:31:57.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d9527d640 1 -- 172.21.15.38:0/156068557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d900b6980 msgr2=0x7f5d900b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:57.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d9527d640 1 --2- 172.21.15.38:0/156068557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d900b6980 0x7f5d900b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d9527d640 1 -- 172.21.15.38:0/156068557 >> v1:172.21.15.38:6789/0 conn(0x7f5d900d0700 legacy=0x7f5d900d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.949+0000 7f5d9527d640 1 -- 172.21.15.38:0/156068557 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d900d28f0 con 0x7f5d900d0320 2024-04-25T08:31:57.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.950+0000 7f5d9527d640 1 --2- 172.21.15.38:0/156068557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d900d0320 0x7f5d900d33d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0097a0 tx=0x7f5d8c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42099d5182497865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.950+0000 7f5d87fff640 1 -- 172.21.15.38:0/156068557 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c037070 con 0x7f5d900d0320 2024-04-25T08:31:57.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.950+0000 7f5d87fff640 1 -- 172.21.15.38:0/156068557 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c02ac30 con 0x7f5d900d0320 2024-04-25T08:31:57.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.950+0000 7f5d87fff640 1 -- 172.21.15.38:0/156068557 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80001670 con 0x7f5d900d0320 2024-04-25T08:31:57.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.951+0000 7f5d974e1640 1 -- 172.21.15.38:0/156068557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d900d0320 msgr2=0x7f5d900d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.951+0000 7f5d974e1640 1 --2- 172.21.15.38:0/156068557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d900d0320 0x7f5d900d33d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0097a0 tx=0x7f5d8c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.951+0000 7f5d974e1640 1 -- 172.21.15.38:0/156068557 shutdown_connections 2024-04-25T08:31:57.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.951+0000 7f5d974e1640 1 --2- 172.21.15.38:0/156068557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d900b6980 0x7f5d900b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.951+0000 7f5d974e1640 1 --2- 172.21.15.38:0/156068557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d900d0320 0x7f5d900d33d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.951+0000 7f5d974e1640 1 -- 172.21.15.38:0/156068557 >> 172.21.15.38:0/156068557 conn(0x7f5d900668f0 msgr2=0x7f5d90066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:57.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.951+0000 7f5d974e1640 1 -- 172.21.15.38:0/156068557 shutdown_connections 2024-04-25T08:31:57.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.951+0000 7f5d974e1640 1 -- 172.21.15.38:0/156068557 wait complete. 2024-04-25T08:31:57.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.952+0000 7f5d974e1640 1 Processor -- start 2024-04-25T08:31:57.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.952+0000 7f5d974e1640 1 -- start start 2024-04-25T08:31:57.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d974e1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d900b6980 0x7f5d90159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d974e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d900d0320 0x7f5d900c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d974e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d900d0700 0x7f5d900c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d974e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d900d0b40 con 0x7f5d900d0320 2024-04-25T08:31:57.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d974e1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d900d2eb0 con 0x7f5d900d0700 2024-04-25T08:31:57.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d974e1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d90156bb0 con 0x7f5d900b6980 2024-04-25T08:31:57.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d94a7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d900d0320 0x7f5d900c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d94a7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d900d0320 0x7f5d900c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33744/0 (socket says 172.21.15.38:33744) 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d94a7c640 1 -- 172.21.15.38:0/1161560146 learned_addr learned my addr 172.21.15.38:0/1161560146 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d95a7e640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d900d0700 0x7f5d900c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d9527d640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d900b6980 0x7f5d90159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d94a7c640 1 -- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d900b6980 msgr2=0x7f5d90159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d94a7c640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d900b6980 0x7f5d90159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d94a7c640 1 -- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d900d0700 msgr2=0x7f5d900c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d94a7c640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d900d0700 0x7f5d900c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.953+0000 7f5d94a7c640 1 -- 172.21.15.38:0/1161560146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d901f3fb0 con 0x7f5d900d0320 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.954+0000 7f5d9527d640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d900b6980 0x7f5d90159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.954+0000 7f5d94a7c640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d900d0320 0x7f5d900c9590 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000c730 tx=0x7f5d8000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.954+0000 7f5d85ffb640 1 -- 172.21.15.38:0/1161560146 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80017070 con 0x7f5d900d0320 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.954+0000 7f5d85ffb640 1 -- 172.21.15.38:0/1161560146 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d80005ad0 con 0x7f5d900d0320 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.954+0000 7f5d974e1640 1 -- 172.21.15.38:0/1161560146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d901f4290 con 0x7f5d900d0320 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.954+0000 7f5d85ffb640 1 -- 172.21.15.38:0/1161560146 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d80007070 con 0x7f5d900d0320 2024-04-25T08:31:57.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.954+0000 7f5d974e1640 1 -- 172.21.15.38:0/1161560146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9015b9b0 con 0x7f5d900d0320 2024-04-25T08:31:57.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.955+0000 7f5d974e1640 1 -- 172.21.15.38:0/1161560146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d90068e80 con 0x7f5d900d0320 2024-04-25T08:31:57.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.956+0000 7f5d85ffb640 1 -- 172.21.15.38:0/1161560146 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d8001a070 con 0x7f5d900d0320 2024-04-25T08:31:57.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.956+0000 7f5d85ffb640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5d7406b4d0 0x7f5d7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:57.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.956+0000 7f5d85ffb640 1 -- 172.21.15.38:0/1161560146 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5d80089910 con 0x7f5d900d0320 2024-04-25T08:31:57.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.956+0000 7f5d9527d640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5d7406b4d0 0x7f5d7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:57.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.957+0000 7f5d9527d640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5d7406b4d0 0x7f5d7406d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c034000 tx=0x7f5d8c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:57.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:57.960+0000 7f5d85ffb640 1 -- 172.21.15.38:0/1161560146 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d80057e10 con 0x7f5d900d0320 2024-04-25T08:31:58.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.138+0000 7f5d974e1640 1 -- 172.21.15.38:0/1161560146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d900b3050 con 0x7f5d900d0320 2024-04-25T08:31:58.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.139+0000 7f5d85ffb640 1 -- 172.21.15.38:0/1161560146 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5d8005beb0 con 0x7f5d900d0320 2024-04-25T08:31:58.135 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:31:58.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.141+0000 7f5d974e1640 1 -- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5d7406b4d0 msgr2=0x7f5d7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.141+0000 7f5d974e1640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5d7406b4d0 0x7f5d7406d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c034000 tx=0x7f5d8c02e000 comp rx=0 tx=0).stop 2024-04-25T08:31:58.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.141+0000 7f5d974e1640 1 -- 172.21.15.38:0/1161560146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d900d0320 msgr2=0x7f5d900c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.141+0000 7f5d974e1640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d900d0320 0x7f5d900c9590 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5d8000c730 tx=0x7f5d8000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.141+0000 7f5d974e1640 1 -- 172.21.15.38:0/1161560146 shutdown_connections 2024-04-25T08:31:58.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.141+0000 7f5d974e1640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d900d0700 0x7f5d900c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.141+0000 7f5d974e1640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d900d0320 0x7f5d900c9590 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.141+0000 7f5d974e1640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5d7406b4d0 0x7f5d7406d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.141+0000 7f5d974e1640 1 --2- 172.21.15.38:0/1161560146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d900b6980 0x7f5d90159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.141+0000 7f5d974e1640 1 -- 172.21.15.38:0/1161560146 >> 172.21.15.38:0/1161560146 conn(0x7f5d900668f0 msgr2=0x7f5d900cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:58.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.141+0000 7f5d974e1640 1 -- 172.21.15.38:0/1161560146 shutdown_connections 2024-04-25T08:31:58.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.141+0000 7f5d974e1640 1 -- 172.21.15.38:0/1161560146 wait complete. 2024-04-25T08:31:58.148 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:31:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.256+0000 7f8c1d5f5640 1 Processor -- start 2024-04-25T08:31:58.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.256+0000 7f8c1d5f5640 1 -- start start 2024-04-25T08:31:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.256+0000 7f8c1d5f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c180d0320 0x7f8c180d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.256+0000 7f8c1d5f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8c180d0700 0x7f8c180d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.256+0000 7f8c1d5f5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8c180d1e90 0x7f8c180b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.256+0000 7f8c1d5f5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c180d2330 con 0x7f8c180d1e90 2024-04-25T08:31:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.256+0000 7f8c1d5f5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c180d2610 con 0x7f8c180d0700 2024-04-25T08:31:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.256+0000 7f8c1d5f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180d28f0 con 0x7f8c180d0320 2024-04-25T08:31:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.257+0000 7f8c177fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8c180d1e90 0x7f8c180b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.257+0000 7f8c177fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8c180d1e90 0x7f8c180b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33750/0 (socket says 172.21.15.38:33750) 2024-04-25T08:31:58.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.257+0000 7f8c177fe640 1 -- 172.21.15.38:0/230246021 learned_addr learned my addr 172.21.15.38:0/230246021 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.257+0000 7f8c167fc640 1 --2- 172.21.15.38:0/230246021 >> v2:172.21.15.107:3300/0 conn(0x7f8c180d0700 0x7f8c180d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.257+0000 7f8c16ffd640 1 --2- 172.21.15.38:0/230246021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c180d0320 0x7f8c180d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.257+0000 7f8c177fe640 1 -- 172.21.15.38:0/230246021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c180d0320 msgr2=0x7f8c180d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.257+0000 7f8c177fe640 1 --2- 172.21.15.38:0/230246021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c180d0320 0x7f8c180d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.257+0000 7f8c177fe640 1 -- 172.21.15.38:0/230246021 >> v2:172.21.15.107:3300/0 conn(0x7f8c180d0700 msgr2=0x7f8c180d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.257+0000 7f8c177fe640 1 --2- 172.21.15.38:0/230246021 >> v2:172.21.15.107:3300/0 conn(0x7f8c180d0700 0x7f8c180d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.257+0000 7f8c177fe640 1 -- 172.21.15.38:0/230246021 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c180d2bd0 con 0x7f8c180d1e90 2024-04-25T08:31:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.258+0000 7f8c177fe640 1 --2- 172.21.15.38:0/230246021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c180d1e90 0x7f8c180b5390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8c180d1490 tx=0x7f8c0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c0f9bf2995983c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.258+0000 7f8c15ffb640 1 -- 172.21.15.38:0/230246021 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08037070 con 0x7f8c180d1e90 2024-04-25T08:31:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.258+0000 7f8c15ffb640 1 -- 172.21.15.38:0/230246021 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c08032440 con 0x7f8c180d1e90 2024-04-25T08:31:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.258+0000 7f8c15ffb640 1 -- 172.21.15.38:0/230246021 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08032810 con 0x7f8c180d1e90 2024-04-25T08:31:58.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.258+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/230246021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c180d1e90 msgr2=0x7f8c180b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.258+0000 7f8c1d5f5640 1 --2- 172.21.15.38:0/230246021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c180d1e90 0x7f8c180b5390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8c180d1490 tx=0x7f8c0802a270 comp rx=0 tx=0).stop 2024-04-25T08:31:58.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.258+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/230246021 shutdown_connections 2024-04-25T08:31:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.258+0000 7f8c1d5f5640 1 --2- 172.21.15.38:0/230246021 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c180d1e90 0x7f8c180b5390 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.258+0000 7f8c1d5f5640 1 --2- 172.21.15.38:0/230246021 >> v2:172.21.15.107:3300/0 conn(0x7f8c180d0700 0x7f8c180d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.258+0000 7f8c1d5f5640 1 --2- 172.21.15.38:0/230246021 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c180d0320 0x7f8c180d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.258+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/230246021 >> 172.21.15.38:0/230246021 conn(0x7f8c180668f0 msgr2=0x7f8c18066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.258+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/230246021 shutdown_connections 2024-04-25T08:31:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.259+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/230246021 wait complete. 2024-04-25T08:31:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.260+0000 7f8c1d5f5640 1 Processor -- start 2024-04-25T08:31:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.260+0000 7f8c1d5f5640 1 -- start start 2024-04-25T08:31:58.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c1d5f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c180d0320 0x7f8c1815a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c1d5f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c180d0700 0x7f8c181f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c1d5f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c180d1e90 0x7f8c180ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c1d5f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180d0b40 con 0x7f8c180d0700 2024-04-25T08:31:58.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c1d5f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18161210 con 0x7f8c180d0320 2024-04-25T08:31:58.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c1d5f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c180cd140 con 0x7f8c180d1e90 2024-04-25T08:31:58.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c180d0700 0x7f8c181f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c180d0700 0x7f8c181f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33760/0 (socket says 172.21.15.38:33760) 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c167fc640 1 -- 172.21.15.38:0/2274099723 learned_addr learned my addr 172.21.15.38:0/2274099723 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c177fe640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c180d1e90 0x7f8c180ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c16ffd640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c180d0320 0x7f8c1815a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c167fc640 1 -- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c180d1e90 msgr2=0x7f8c180ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c167fc640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c180d1e90 0x7f8c180ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c167fc640 1 -- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c180d0320 msgr2=0x7f8c1815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c167fc640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c180d0320 0x7f8c1815a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c167fc640 1 -- 172.21.15.38:0/2274099723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1815ade0 con 0x7f8c180d0700 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c167fc640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c180d0700 0x7f8c181f76a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00e9b0 tx=0x7f8c0c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8bff7fe640 1 -- 172.21.15.38:0/2274099723 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c017070 con 0x7f8c180d0700 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8bff7fe640 1 -- 172.21.15.38:0/2274099723 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c00c9a0 con 0x7f8c180d0700 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/2274099723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1815b0c0 con 0x7f8c180d0700 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.261+0000 7f8bff7fe640 1 -- 172.21.15.38:0/2274099723 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00cd90 con 0x7f8c180d0700 2024-04-25T08:31:58.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.262+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/2274099723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c181f6ea0 con 0x7f8c180d0700 2024-04-25T08:31:58.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.262+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/2274099723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c18068e80 con 0x7f8c180d0700 2024-04-25T08:31:58.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.263+0000 7f8bff7fe640 1 -- 172.21.15.38:0/2274099723 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c0c01a070 con 0x7f8c180d0700 2024-04-25T08:31:58.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.263+0000 7f8bff7fe640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c0406b500 0x7f8c0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.263+0000 7f8bff7fe640 1 -- 172.21.15.38:0/2274099723 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8c0c08a820 con 0x7f8c180d0700 2024-04-25T08:31:58.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.263+0000 7f8c16ffd640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c0406b500 0x7f8c0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.264+0000 7f8c16ffd640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c0406b500 0x7f8c0406d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8c18157a00 tx=0x7f8c1000d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.268+0000 7f8bff7fe640 1 -- 172.21.15.38:0/2274099723 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c0c058d10 con 0x7f8c180d0700 2024-04-25T08:31:58.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.448+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/2274099723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8c180cc950 con 0x7f8c180d0700 2024-04-25T08:31:58.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.449+0000 7f8bff7fe640 1 -- 172.21.15.38:0/2274099723 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8c0c05cdb0 con 0x7f8c180d0700 2024-04-25T08:31:58.446 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:31:58.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.451+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c0406b500 msgr2=0x7f8c0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.451+0000 7f8c1d5f5640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c0406b500 0x7f8c0406d9c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8c18157a00 tx=0x7f8c1000d040 comp rx=0 tx=0).stop 2024-04-25T08:31:58.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.451+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/2274099723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c180d0700 msgr2=0x7f8c181f76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.451+0000 7f8c1d5f5640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c180d0700 0x7f8c181f76a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00e9b0 tx=0x7f8c0c00ee70 comp rx=0 tx=0).stop 2024-04-25T08:31:58.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.451+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/2274099723 shutdown_connections 2024-04-25T08:31:58.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.451+0000 7f8c1d5f5640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c0406b500 0x7f8c0406d9c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.451+0000 7f8c1d5f5640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c180d1e90 0x7f8c180ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.451+0000 7f8c1d5f5640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c180d0700 0x7f8c181f76a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.451+0000 7f8c1d5f5640 1 --2- 172.21.15.38:0/2274099723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c180d0320 0x7f8c1815a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.451+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/2274099723 >> 172.21.15.38:0/2274099723 conn(0x7f8c180668f0 msgr2=0x7f8c1815f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:58.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.451+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/2274099723 shutdown_connections 2024-04-25T08:31:58.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.451+0000 7f8c1d5f5640 1 -- 172.21.15.38:0/2274099723 wait complete. 2024-04-25T08:31:58.458 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:31:58.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.563+0000 7f4afb36a640 1 Processor -- start 2024-04-25T08:31:58.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.564+0000 7f4afb36a640 1 -- start start 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.564+0000 7f4afb36a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4af415df50 0x7f4af40ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.564+0000 7f4afb36a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4af40ce2e0 0x7f4af40d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.564+0000 7f4afb36a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af40ceff0 con 0x7f4af406c110 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.564+0000 7f4afb36a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af40cf2d0 con 0x7f4af40ce2e0 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.564+0000 7f4afb36a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af40d1f70 con 0x7f4af415df50 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.564+0000 7f4af9907640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4af40ce2e0 0x7f4af40d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.564+0000 7f4af9907640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4af40ce2e0 0x7f4af40d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33768/0 (socket says 172.21.15.38:33768) 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.564+0000 7f4af9907640 1 -- 172.21.15.38:0/1242072147 learned_addr learned my addr 172.21.15.38:0/1242072147 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.565+0000 7f4af8905640 1 --2- 172.21.15.38:0/1242072147 >> v2:172.21.15.107:3300/0 conn(0x7f4af415df50 0x7f4af40ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.565+0000 7f4af9907640 1 -- 172.21.15.38:0/1242072147 >> v2:172.21.15.107:3300/0 conn(0x7f4af415df50 msgr2=0x7f4af40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.565+0000 7f4af9907640 1 --2- 172.21.15.38:0/1242072147 >> v2:172.21.15.107:3300/0 conn(0x7f4af415df50 0x7f4af40ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.565+0000 7f4af9907640 1 -- 172.21.15.38:0/1242072147 >> v1:172.21.15.107:6789/0 conn(0x7f4af406c110 legacy=0x7f4af406c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.565+0000 7f4af9907640 1 -- 172.21.15.38:0/1242072147 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af40d2250 con 0x7f4af40ce2e0 2024-04-25T08:31:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.565+0000 7f4af8905640 1 --2- 172.21.15.38:0/1242072147 >> v2:172.21.15.107:3300/0 conn(0x7f4af415df50 0x7f4af40ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.565+0000 7f4af9907640 1 --2- 172.21.15.38:0/1242072147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40ce2e0 0x7f4af40d2a50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4af40d4960 tx=0x7f4ae402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85e8d73edbaadf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.566+0000 7f4af3fff640 1 -- 172.21.15.38:0/1242072147 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4036070 con 0x7f4af40ce2e0 2024-04-25T08:31:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.566+0000 7f4af3fff640 1 -- 172.21.15.38:0/1242072147 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae4031600 con 0x7f4af40ce2e0 2024-04-25T08:31:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.566+0000 7f4af3fff640 1 -- 172.21.15.38:0/1242072147 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40319d0 con 0x7f4af40ce2e0 2024-04-25T08:31:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.566+0000 7f4afb36a640 1 -- 172.21.15.38:0/1242072147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40ce2e0 msgr2=0x7f4af40d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.566+0000 7f4afb36a640 1 --2- 172.21.15.38:0/1242072147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40ce2e0 0x7f4af40d2a50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4af40d4960 tx=0x7f4ae402a730 comp rx=0 tx=0).stop 2024-04-25T08:31:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.566+0000 7f4afb36a640 1 -- 172.21.15.38:0/1242072147 shutdown_connections 2024-04-25T08:31:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.566+0000 7f4afb36a640 1 --2- 172.21.15.38:0/1242072147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af40ce2e0 0x7f4af40d2a50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.566+0000 7f4afb36a640 1 --2- 172.21.15.38:0/1242072147 >> v2:172.21.15.107:3300/0 conn(0x7f4af415df50 0x7f4af40ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.566+0000 7f4afb36a640 1 -- 172.21.15.38:0/1242072147 >> 172.21.15.38:0/1242072147 conn(0x7f4af40d3fb0 msgr2=0x7f4af40d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.566+0000 7f4afb36a640 1 -- 172.21.15.38:0/1242072147 shutdown_connections 2024-04-25T08:31:58.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.566+0000 7f4afb36a640 1 -- 172.21.15.38:0/1242072147 wait complete. 2024-04-25T08:31:58.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.567+0000 7f4afb36a640 1 Processor -- start 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.567+0000 7f4afb36a640 1 -- start start 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4afb36a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4af406c110 0x7f4af4148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4afb36a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4af40ce2e0 0x7f4af4157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4afb36a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af415df50 0x7f4af4158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4afb36a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af40d2f90 con 0x7f4af415df50 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4afb36a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af40cece0 con 0x7f4af40ce2e0 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4afb36a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af40d3110 con 0x7f4af406c110 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4af9907640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af415df50 0x7f4af4158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4af9907640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af415df50 0x7f4af4158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33770/0 (socket says 172.21.15.38:33770) 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4af9907640 1 -- 172.21.15.38:0/525998557 learned_addr learned my addr 172.21.15.38:0/525998557 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4af9907640 1 -- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4af406c110 msgr2=0x7f4af4148de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4af9907640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4af406c110 0x7f4af4148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4af8905640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4af40ce2e0 0x7f4af4157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4af9907640 1 -- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4af40ce2e0 msgr2=0x7f4af4157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.568+0000 7f4af9907640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4af40ce2e0 0x7f4af4157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.569+0000 7f4af9907640 1 -- 172.21.15.38:0/525998557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af40c7790 con 0x7f4af415df50 2024-04-25T08:31:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.569+0000 7f4af9907640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af415df50 0x7f4af4158360 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4034040 tx=0x7f4ae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.569+0000 7f4af1ffb640 1 -- 172.21.15.38:0/525998557 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4036050 con 0x7f4af415df50 2024-04-25T08:31:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.569+0000 7f4af1ffb640 1 -- 172.21.15.38:0/525998557 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae4042070 con 0x7f4af415df50 2024-04-25T08:31:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.569+0000 7f4afb36a640 1 -- 172.21.15.38:0/525998557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af40c7a70 con 0x7f4af415df50 2024-04-25T08:31:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.569+0000 7f4af1ffb640 1 -- 172.21.15.38:0/525998557 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4009510 con 0x7f4af415df50 2024-04-25T08:31:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.569+0000 7f4afb36a640 1 -- 172.21.15.38:0/525998557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af40c8030 con 0x7f4af415df50 2024-04-25T08:31:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.570+0000 7f4afb36a640 1 -- 172.21.15.38:0/525998557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af4068e80 con 0x7f4af415df50 2024-04-25T08:31:58.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.570+0000 7f4af1ffb640 1 -- 172.21.15.38:0/525998557 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ae4007bf0 con 0x7f4af415df50 2024-04-25T08:31:58.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.571+0000 7f4af1ffb640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4ae006b4d0 0x7f4ae006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.571+0000 7f4af1ffb640 1 -- 172.21.15.38:0/525998557 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4ae40ab2e0 con 0x7f4af415df50 2024-04-25T08:31:58.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.571+0000 7f4af9106640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4ae006b4d0 0x7f4ae006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.572+0000 7f4af9106640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4ae006b4d0 0x7f4ae006d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4aec000f60 tx=0x7f4aec0079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.575+0000 7f4af1ffb640 1 -- 172.21.15.38:0/525998557 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ae4079eb0 con 0x7f4af415df50 2024-04-25T08:31:58.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.793+0000 7f4afb36a640 1 -- 172.21.15.38:0/525998557 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4af40b9110 con 0x7f4af415df50 2024-04-25T08:31:58.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.793+0000 7f4af1ffb640 1 -- 172.21.15.38:0/525998557 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f4ae40797b0 con 0x7f4af415df50 2024-04-25T08:31:58.790 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:58.790 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4242612806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4242612806},{"type":"v1","addr":"172.21.15.38:6835","nonce":4242612806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2154167637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":2154167637},{"type":"v1","addr":"172.21.15.107:6837","nonce":2154167637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/678452936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":678452936},{"type":"v1","addr":"172.21.15.38:6834","nonce":678452936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:31:35.665008+0000","modified":"2024-04-25T08:31:51.799830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6836/1104921666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1104921666},{"type":"v1","addr":"172.21.15.107:6836","nonce":1104921666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:31:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-04-25T08:31:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.795+0000 7f4afb36a640 1 -- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4ae006b4d0 msgr2=0x7f4ae006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.795+0000 7f4afb36a640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4ae006b4d0 0x7f4ae006d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4aec000f60 tx=0x7f4aec0079d0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.795+0000 7f4afb36a640 1 -- 172.21.15.38:0/525998557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af415df50 msgr2=0x7f4af4158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.795+0000 7f4afb36a640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af415df50 0x7f4af4158360 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4034040 tx=0x7f4ae4004a40 comp rx=0 tx=0).stop 2024-04-25T08:31:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.795+0000 7f4afb36a640 1 -- 172.21.15.38:0/525998557 shutdown_connections 2024-04-25T08:31:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.795+0000 7f4afb36a640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4af415df50 0x7f4af4158360 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.795+0000 7f4afb36a640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4af40ce2e0 0x7f4af4157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.795+0000 7f4afb36a640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4ae006b4d0 0x7f4ae006d990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.795+0000 7f4afb36a640 1 --2- 172.21.15.38:0/525998557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4af406c110 0x7f4af4148de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.795+0000 7f4afb36a640 1 -- 172.21.15.38:0/525998557 >> 172.21.15.38:0/525998557 conn(0x7f4af40d3fb0 msgr2=0x7f4af40d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.795+0000 7f4afb36a640 1 -- 172.21.15.38:0/525998557 shutdown_connections 2024-04-25T08:31:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.795+0000 7f4afb36a640 1 -- 172.21.15.38:0/525998557 wait complete. 2024-04-25T08:31:58.803 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:31:58.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.907+0000 7fb8a877e640 1 Processor -- start 2024-04-25T08:31:58.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.907+0000 7fb8a877e640 1 -- start start 2024-04-25T08:31:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a877e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8a006cff0 0x7fb8a0099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a877e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb8a0099fb0 0x7fb8a015d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a877e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a00d05c0 con 0x7fb8a006cc10 2024-04-25T08:31:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a877e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a009a3e0 con 0x7fb8a0099fb0 2024-04-25T08:31:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a877e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a00d08a0 con 0x7fb8a006cff0 2024-04-25T08:31:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a6d1b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb8a0099fb0 0x7fb8a015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a6d1b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb8a0099fb0 0x7fb8a015d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56570/0 (socket says 172.21.15.38:56570) 2024-04-25T08:31:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a651a640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb8a006cc10 0x7fb8a00cfa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54824/0 (socket says 172.21.15.38:54824) 2024-04-25T08:31:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a6d1b640 1 -- 172.21.15.38:0/276265195 learned_addr learned my addr 172.21.15.38:0/276265195 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a5d19640 1 --2- 172.21.15.38:0/276265195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8a006cff0 0x7fb8a0099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a6d1b640 1 -- 172.21.15.38:0/276265195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8a006cff0 msgr2=0x7fb8a0099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:58.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a6d1b640 1 --2- 172.21.15.38:0/276265195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8a006cff0 0x7fb8a0099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a6d1b640 1 -- 172.21.15.38:0/276265195 >> v1:172.21.15.38:6789/0 conn(0x7fb8a006cc10 legacy=0x7fb8a00cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a6d1b640 1 -- 172.21.15.38:0/276265195 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a00d0b80 con 0x7fb8a0099fb0 2024-04-25T08:31:58.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.908+0000 7fb8a5518640 1 -- 172.21.15.38:0/276265195 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305780928 0 0) 0x7fb8a00d05c0 con 0x7fb8a006cc10 2024-04-25T08:31:58.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.909+0000 7fb8a6d1b640 1 --2- 172.21.15.38:0/276265195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8a0099fb0 0x7fb8a015d370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb898009e50 tx=0x7fb89802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59447b51d8e7fdde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.909+0000 7fb8a5518640 1 -- 172.21.15.38:0/276265195 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898036070 con 0x7fb8a0099fb0 2024-04-25T08:31:58.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.909+0000 7fb8a5518640 1 -- 172.21.15.38:0/276265195 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb898031600 con 0x7fb8a0099fb0 2024-04-25T08:31:58.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.909+0000 7fb8a5518640 1 -- 172.21.15.38:0/276265195 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8980319d0 con 0x7fb8a0099fb0 2024-04-25T08:31:58.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.910+0000 7fb8a877e640 1 -- 172.21.15.38:0/276265195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8a0099fb0 msgr2=0x7fb8a015d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.910+0000 7fb8a877e640 1 --2- 172.21.15.38:0/276265195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8a0099fb0 0x7fb8a015d370 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb898009e50 tx=0x7fb89802a710 comp rx=0 tx=0).stop 2024-04-25T08:31:58.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.910+0000 7fb8a877e640 1 -- 172.21.15.38:0/276265195 shutdown_connections 2024-04-25T08:31:58.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.910+0000 7fb8a877e640 1 --2- 172.21.15.38:0/276265195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8a0099fb0 0x7fb8a015d370 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.910+0000 7fb8a877e640 1 --2- 172.21.15.38:0/276265195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8a006cff0 0x7fb8a0099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.910+0000 7fb8a877e640 1 -- 172.21.15.38:0/276265195 >> 172.21.15.38:0/276265195 conn(0x7fb8a00b31f0 msgr2=0x7fb8a00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:58.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.910+0000 7fb8a877e640 1 -- 172.21.15.38:0/276265195 shutdown_connections 2024-04-25T08:31:58.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.910+0000 7fb8a877e640 1 -- 172.21.15.38:0/276265195 wait complete. 2024-04-25T08:31:58.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.911+0000 7fb8a877e640 1 Processor -- start 2024-04-25T08:31:58.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.911+0000 7fb8a877e640 1 -- start start 2024-04-25T08:31:58.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a877e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8a006cc10 0x7fb8a00c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a877e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8a006cff0 0x7fb8a00cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a877e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8a0099fb0 0x7fb8a00b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a877e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a006c150 con 0x7fb8a006cff0 2024-04-25T08:31:58.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a877e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a00d1140 con 0x7fb8a0099fb0 2024-04-25T08:31:58.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a877e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a006c2d0 con 0x7fb8a006cc10 2024-04-25T08:31:58.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a5d19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8a006cff0 0x7fb8a00cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a5d19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8a006cff0 0x7fb8a00cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56582/0 (socket says 172.21.15.38:56582) 2024-04-25T08:31:58.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a5d19640 1 -- 172.21.15.38:0/1839060966 learned_addr learned my addr 172.21.15.38:0/1839060966 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:58.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a6d1b640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8a0099fb0 0x7fb8a00b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a651a640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8a006cc10 0x7fb8a00c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a5d19640 1 -- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8a006cc10 msgr2=0x7fb8a00c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a5d19640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8a006cc10 0x7fb8a00c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a5d19640 1 -- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8a0099fb0 msgr2=0x7fb8a00b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a5d19640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8a0099fb0 0x7fb8a00b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a5d19640 1 -- 172.21.15.38:0/1839060966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a00b61e0 con 0x7fb8a006cff0 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a6d1b640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8a0099fb0 0x7fb8a00b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a5d19640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8a006cff0 0x7fb8a00cb470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb89000e9e0 tx=0x7fb89000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb896ffd640 1 -- 172.21.15.38:0/1839060966 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890017070 con 0x7fb8a006cff0 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb896ffd640 1 -- 172.21.15.38:0/1839060966 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89000c9d0 con 0x7fb8a006cff0 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a877e640 1 -- 172.21.15.38:0/1839060966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a00b6480 con 0x7fb8a006cff0 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb896ffd640 1 -- 172.21.15.38:0/1839060966 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89000cdc0 con 0x7fb8a006cff0 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.912+0000 7fb8a877e640 1 -- 172.21.15.38:0/1839060966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a00b6a40 con 0x7fb8a006cff0 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.913+0000 7fb8a877e640 1 -- 172.21.15.38:0/1839060966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a0068e80 con 0x7fb8a006cff0 2024-04-25T08:31:58.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.914+0000 7fb896ffd640 1 -- 172.21.15.38:0/1839060966 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb89001a070 con 0x7fb8a006cff0 2024-04-25T08:31:58.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.914+0000 7fb896ffd640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb88406b4d0 0x7fb88406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:58.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.914+0000 7fb896ffd640 1 -- 172.21.15.38:0/1839060966 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb89008a8f0 con 0x7fb8a006cff0 2024-04-25T08:31:58.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.915+0000 7fb8a651a640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb88406b4d0 0x7fb88406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:58.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.915+0000 7fb8a651a640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb88406b4d0 0x7fb88406d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb89c000f60 tx=0x7fb89c007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:58.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:58.919+0000 7fb896ffd640 1 -- 172.21.15.38:0/1839060966 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8900594c0 con 0x7fb8a006cff0 2024-04-25T08:31:59.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.098+0000 7fb8a877e640 1 -- 172.21.15.38:0/1839060966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8a00b4020 con 0x7fb8a006cff0 2024-04-25T08:31:59.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.099+0000 7fb896ffd640 1 -- 172.21.15.38:0/1839060966 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fb890058dc0 con 0x7fb8a006cff0 2024-04-25T08:31:59.097 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:31:59.097 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:31:59.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.101+0000 7fb8a877e640 1 -- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb88406b4d0 msgr2=0x7fb88406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.101+0000 7fb8a877e640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb88406b4d0 0x7fb88406d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb89c000f60 tx=0x7fb89c007c00 comp rx=0 tx=0).stop 2024-04-25T08:31:59.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.101+0000 7fb8a877e640 1 -- 172.21.15.38:0/1839060966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8a006cff0 msgr2=0x7fb8a00cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.101+0000 7fb8a877e640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8a006cff0 0x7fb8a00cb470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb89000e9e0 tx=0x7fb89000eea0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.101+0000 7fb8a877e640 1 -- 172.21.15.38:0/1839060966 shutdown_connections 2024-04-25T08:31:59.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.101+0000 7fb8a877e640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8a0099fb0 0x7fb8a00b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.101+0000 7fb8a877e640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8a006cff0 0x7fb8a00cb470 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.101+0000 7fb8a877e640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb88406b4d0 0x7fb88406d990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.101+0000 7fb8a877e640 1 --2- 172.21.15.38:0/1839060966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb8a006cc10 0x7fb8a00c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.101+0000 7fb8a877e640 1 -- 172.21.15.38:0/1839060966 >> 172.21.15.38:0/1839060966 conn(0x7fb8a00b31f0 msgr2=0x7fb8a00cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:59.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.101+0000 7fb8a877e640 1 -- 172.21.15.38:0/1839060966 shutdown_connections 2024-04-25T08:31:59.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.101+0000 7fb8a877e640 1 -- 172.21.15.38:0/1839060966 wait complete. 2024-04-25T08:31:59.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:31:59.110 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:31:59.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.216+0000 7f053770a640 1 Processor -- start 2024-04-25T08:31:59.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.216+0000 7f053770a640 1 -- start start 2024-04-25T08:31:59.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f053770a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f05300d33d0 0x7f05300d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f053770a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05300d1e90 0x7f05300b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f053770a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300d2330 con 0x7f05300d0320 2024-04-25T08:31:59.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f053770a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05300d2610 con 0x7f05300d33d0 2024-04-25T08:31:59.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f053770a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05300d28f0 con 0x7f05300d1e90 2024-04-25T08:31:59.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f0534ca5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f05300d33d0 0x7f05300d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f05354a6640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f05300d0320 0x7f05300d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54840/0 (socket says 172.21.15.38:54840) 2024-04-25T08:31:59.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f05354a6640 1 -- 172.21.15.38:0/1423345462 learned_addr learned my addr 172.21.15.38:0/1423345462 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f0535ca7640 1 --2- 172.21.15.38:0/1423345462 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05300d1e90 0x7f05300b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f0534ca5640 1 -- 172.21.15.38:0/1423345462 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05300d1e90 msgr2=0x7f05300b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f0534ca5640 1 --2- 172.21.15.38:0/1423345462 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05300d1e90 0x7f05300b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f0534ca5640 1 -- 172.21.15.38:0/1423345462 >> v1:172.21.15.38:6789/0 conn(0x7f05300d0320 legacy=0x7f05300d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f0527fff640 1 -- 172.21.15.38:0/1423345462 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742377087 0 0) 0x7f05300d2330 con 0x7f05300d0320 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f0534ca5640 1 -- 172.21.15.38:0/1423345462 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300d2bd0 con 0x7f05300d33d0 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.217+0000 7f0535ca7640 1 --2- 172.21.15.38:0/1423345462 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05300d1e90 0x7f05300b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.218+0000 7f0534ca5640 1 --2- 172.21.15.38:0/1423345462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05300d33d0 0x7f05300d1950 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f05200097a0 tx=0x7f052002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90759f45f2b7cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.218+0000 7f0527fff640 1 -- 172.21.15.38:0/1423345462 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520037070 con 0x7f05300d33d0 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.218+0000 7f0527fff640 1 -- 172.21.15.38:0/1423345462 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0520032440 con 0x7f05300d33d0 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.218+0000 7f0527fff640 1 -- 172.21.15.38:0/1423345462 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520032810 con 0x7f05300d33d0 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.218+0000 7f053770a640 1 -- 172.21.15.38:0/1423345462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05300d33d0 msgr2=0x7f05300d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.218+0000 7f053770a640 1 --2- 172.21.15.38:0/1423345462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05300d33d0 0x7f05300d1950 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f05200097a0 tx=0x7f052002a210 comp rx=0 tx=0).stop 2024-04-25T08:31:59.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.219+0000 7f053770a640 1 -- 172.21.15.38:0/1423345462 shutdown_connections 2024-04-25T08:31:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.219+0000 7f053770a640 1 --2- 172.21.15.38:0/1423345462 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05300d1e90 0x7f05300b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.219+0000 7f053770a640 1 --2- 172.21.15.38:0/1423345462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05300d33d0 0x7f05300d1950 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.219+0000 7f053770a640 1 -- 172.21.15.38:0/1423345462 >> 172.21.15.38:0/1423345462 conn(0x7f05300668f0 msgr2=0x7f0530066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.219+0000 7f053770a640 1 -- 172.21.15.38:0/1423345462 shutdown_connections 2024-04-25T08:31:59.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.219+0000 7f053770a640 1 -- 172.21.15.38:0/1423345462 wait complete. 2024-04-25T08:31:59.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.220+0000 7f053770a640 1 Processor -- start 2024-04-25T08:31:59.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.220+0000 7f053770a640 1 -- start start 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.220+0000 7f053770a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05300d0320 0x7f05300b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.220+0000 7f053770a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05300d1e90 0x7f0530159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.220+0000 7f053770a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05300d33d0 0x7f05300cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.220+0000 7f053770a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530161390 con 0x7f05300d0320 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.220+0000 7f053770a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530161210 con 0x7f05300d1e90 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.220+0000 7f053770a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05300b9720 con 0x7f05300d33d0 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f05354a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05300d0320 0x7f05300b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f05354a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05300d0320 0x7f05300b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56596/0 (socket says 172.21.15.38:56596) 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f05354a6640 1 -- 172.21.15.38:0/2357271074 learned_addr learned my addr 172.21.15.38:0/2357271074 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f0534ca5640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05300d1e90 0x7f0530159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f05354a6640 1 -- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05300d33d0 msgr2=0x7f05300cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f0535ca7640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05300d33d0 0x7f05300cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f05354a6640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05300d33d0 0x7f05300cc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f05354a6640 1 -- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05300d1e90 msgr2=0x7f0530159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f05354a6640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05300d1e90 0x7f0530159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f05354a6640 1 -- 172.21.15.38:0/2357271074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0530159ba0 con 0x7f05300d0320 2024-04-25T08:31:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f05354a6640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05300d0320 0x7f05300b1ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f052c00c4d0 tx=0x7f052c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f0525ffb640 1 -- 172.21.15.38:0/2357271074 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c018070 con 0x7f05300d0320 2024-04-25T08:31:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f0525ffb640 1 -- 172.21.15.38:0/2357271074 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c007500 con 0x7f05300d0320 2024-04-25T08:31:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f053770a640 1 -- 172.21.15.38:0/2357271074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0530159e80 con 0x7f05300d0320 2024-04-25T08:31:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.221+0000 7f0525ffb640 1 -- 172.21.15.38:0/2357271074 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c004a20 con 0x7f05300d0320 2024-04-25T08:31:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.222+0000 7f053770a640 1 -- 172.21.15.38:0/2357271074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053015a720 con 0x7f05300d0320 2024-04-25T08:31:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.223+0000 7f053770a640 1 -- 172.21.15.38:0/2357271074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053015a050 con 0x7f05300d0320 2024-04-25T08:31:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.223+0000 7f0525ffb640 1 -- 172.21.15.38:0/2357271074 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f052c01b070 con 0x7f05300d0320 2024-04-25T08:31:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.223+0000 7f0525ffb640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f051406b500 0x7f051406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.223+0000 7f0525ffb640 1 -- 172.21.15.38:0/2357271074 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f052c089820 con 0x7f05300d0320 2024-04-25T08:31:59.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.223+0000 7f0534ca5640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f051406b500 0x7f051406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.224+0000 7f0534ca5640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f051406b500 0x7f051406d9c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0520034000 tx=0x7f052002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.227+0000 7f0525ffb640 1 -- 172.21.15.38:0/2357271074 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f052c058360 con 0x7f05300d0320 2024-04-25T08:31:59.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.409+0000 7f053770a640 1 -- 172.21.15.38:0/2357271074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0530068e80 con 0x7f05300d0320 2024-04-25T08:31:59.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.409+0000 7f0525ffb640 1 -- 172.21.15.38:0/2357271074 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f052c057c60 con 0x7f05300d0320 2024-04-25T08:31:59.407 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:31:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.411+0000 7f053770a640 1 -- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f051406b500 msgr2=0x7f051406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.411+0000 7f053770a640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f051406b500 0x7f051406d9c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0520034000 tx=0x7f052002e000 comp rx=0 tx=0).stop 2024-04-25T08:31:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.411+0000 7f053770a640 1 -- 172.21.15.38:0/2357271074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05300d0320 msgr2=0x7f05300b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.411+0000 7f053770a640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05300d0320 0x7f05300b1ad0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f052c00c4d0 tx=0x7f052c00c990 comp rx=0 tx=0).stop 2024-04-25T08:31:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.411+0000 7f053770a640 1 -- 172.21.15.38:0/2357271074 shutdown_connections 2024-04-25T08:31:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.411+0000 7f053770a640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f051406b500 0x7f051406d9c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.411+0000 7f053770a640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05300d33d0 0x7f05300cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.411+0000 7f053770a640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05300d1e90 0x7f0530159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.411+0000 7f053770a640 1 --2- 172.21.15.38:0/2357271074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05300d0320 0x7f05300b1ad0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.411+0000 7f053770a640 1 -- 172.21.15.38:0/2357271074 >> 172.21.15.38:0/2357271074 conn(0x7f05300668f0 msgr2=0x7f05300b7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.411+0000 7f053770a640 1 -- 172.21.15.38:0/2357271074 shutdown_connections 2024-04-25T08:31:59.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.411+0000 7f053770a640 1 -- 172.21.15.38:0/2357271074 wait complete. 2024-04-25T08:31:59.419 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:31:59.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.525+0000 7f3aea858640 1 Processor -- start 2024-04-25T08:31:59.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.525+0000 7f3aea858640 1 -- start start 2024-04-25T08:31:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.526+0000 7f3aea858640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3ae415de60 0x7f3ae40d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.526+0000 7f3aea858640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae40ce710 con 0x7f3ae40d1e00 2024-04-25T08:31:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.526+0000 7f3aea858640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae40ce9f0 con 0x7f3ae40cf7b0 2024-04-25T08:31:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.526+0000 7f3aea858640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae40ce4e0 con 0x7f3ae415de60 2024-04-25T08:31:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.526+0000 7f3ae37fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3ae415de60 0x7f3ae40d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.526+0000 7f3ae3fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f3ae40d1e00 0x7f3ae40d21e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54854/0 (socket says 172.21.15.38:54854) 2024-04-25T08:31:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.526+0000 7f3ae3fff640 1 -- 172.21.15.38:0/1356608848 learned_addr learned my addr 172.21.15.38:0/1356608848 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:31:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.527+0000 7f3ae37fe640 1 -- 172.21.15.38:0/1356608848 >> v1:172.21.15.107:6789/0 conn(0x7f3ae40cf7b0 legacy=0x7f3ae40b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.527+0000 7f3ae37fe640 1 -- 172.21.15.38:0/1356608848 >> v1:172.21.15.38:6789/0 conn(0x7f3ae40d1e00 legacy=0x7f3ae40d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.527+0000 7f3ae37fe640 1 -- 172.21.15.38:0/1356608848 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae40cecd0 con 0x7f3ae415de60 2024-04-25T08:31:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.527+0000 7f3ae2ffd640 1 -- 172.21.15.38:0/1356608848 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973973391 0 0) 0x7f3ae40ce710 con 0x7f3ae40d1e00 2024-04-25T08:31:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.527+0000 7f3ae37fe640 1 --2- 172.21.15.38:0/1356608848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae415de60 0x7f3ae40d42a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80097d0 tx=0x7f3ad802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d1685842f08ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.527+0000 7f3ae2ffd640 1 -- 172.21.15.38:0/1356608848 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8037070 con 0x7f3ae415de60 2024-04-25T08:31:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.528+0000 7f3ae2ffd640 1 -- 172.21.15.38:0/1356608848 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad8032440 con 0x7f3ae415de60 2024-04-25T08:31:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.528+0000 7f3ae2ffd640 1 -- 172.21.15.38:0/1356608848 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8032810 con 0x7f3ae415de60 2024-04-25T08:31:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.528+0000 7f3aea858640 1 -- 172.21.15.38:0/1356608848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae415de60 msgr2=0x7f3ae40d42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.528+0000 7f3aea858640 1 --2- 172.21.15.38:0/1356608848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae415de60 0x7f3ae40d42a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ad80097d0 tx=0x7f3ad802a210 comp rx=0 tx=0).stop 2024-04-25T08:31:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.528+0000 7f3aea858640 1 -- 172.21.15.38:0/1356608848 shutdown_connections 2024-04-25T08:31:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.528+0000 7f3aea858640 1 --2- 172.21.15.38:0/1356608848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae415de60 0x7f3ae40d42a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.528+0000 7f3aea858640 1 -- 172.21.15.38:0/1356608848 >> 172.21.15.38:0/1356608848 conn(0x7f3ae408b000 msgr2=0x7f3ae40d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.528+0000 7f3aea858640 1 -- 172.21.15.38:0/1356608848 shutdown_connections 2024-04-25T08:31:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.528+0000 7f3aea858640 1 -- 172.21.15.38:0/1356608848 wait complete. 2024-04-25T08:31:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.529+0000 7f3aea858640 1 Processor -- start 2024-04-25T08:31:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.529+0000 7f3aea858640 1 -- start start 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.530+0000 7f3aea858640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae40cf7b0 0x7f3ae40b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.530+0000 7f3aea858640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae40d1e00 0x7f3ae415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.530+0000 7f3aea858640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ae415de60 0x7f3ae40cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.530+0000 7f3aea858640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae40d1600 con 0x7f3ae40d1e00 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.530+0000 7f3aea858640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae40cf320 con 0x7f3ae40cf7b0 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.530+0000 7f3aea858640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae40d1780 con 0x7f3ae415de60 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.530+0000 7f3ae37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae40d1e00 0x7f3ae415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.530+0000 7f3ae37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae40d1e00 0x7f3ae415bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56610/0 (socket says 172.21.15.38:56610) 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.530+0000 7f3ae37fe640 1 -- 172.21.15.38:0/2902523933 learned_addr learned my addr 172.21.15.38:0/2902523933 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae3fff640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae40cf7b0 0x7f3ae40b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae8df5640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ae415de60 0x7f3ae40cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae37fe640 1 -- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ae415de60 msgr2=0x7f3ae40cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae37fe640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ae415de60 0x7f3ae40cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae37fe640 1 -- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae40cf7b0 msgr2=0x7f3ae40b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae37fe640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae40cf7b0 0x7f3ae40b8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae37fe640 1 -- 172.21.15.38:0/2902523933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae40b8490 con 0x7f3ae40d1e00 2024-04-25T08:31:59.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae3fff640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae40cf7b0 0x7f3ae40b8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:59.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae8df5640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ae415de60 0x7f3ae40cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:31:59.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae37fe640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae40d1e00 0x7f3ae415bfb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8035040 tx=0x7f3ad80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae0ff9640 1 -- 172.21.15.38:0/2902523933 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8037040 con 0x7f3ae40d1e00 2024-04-25T08:31:59.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae0ff9640 1 -- 172.21.15.38:0/2902523933 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad8004230 con 0x7f3ae40d1e00 2024-04-25T08:31:59.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3ae0ff9640 1 -- 172.21.15.38:0/2902523933 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad803c670 con 0x7f3ae40d1e00 2024-04-25T08:31:59.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3aea858640 1 -- 172.21.15.38:0/2902523933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae40b8770 con 0x7f3ae40d1e00 2024-04-25T08:31:59.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.531+0000 7f3aea858640 1 -- 172.21.15.38:0/2902523933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae41e7bd0 con 0x7f3ae40d1e00 2024-04-25T08:31:59.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.532+0000 7f3aea858640 1 -- 172.21.15.38:0/2902523933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae4068e80 con 0x7f3ae40d1e00 2024-04-25T08:31:59.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.533+0000 7f3ae0ff9640 1 -- 172.21.15.38:0/2902523933 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ad803a070 con 0x7f3ae40d1e00 2024-04-25T08:31:59.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.533+0000 7f3ae0ff9640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3ad006b4d0 0x7f3ad006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.533+0000 7f3ae0ff9640 1 -- 172.21.15.38:0/2902523933 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3ad80aaef0 con 0x7f3ae40d1e00 2024-04-25T08:31:59.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.533+0000 7f3ae3fff640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3ad006b4d0 0x7f3ad006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.534+0000 7f3ae3fff640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3ad006b4d0 0x7f3ad006d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4000f60 tx=0x7f3ad4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.537+0000 7f3ae0ff9640 1 -- 172.21.15.38:0/2902523933 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ad8041020 con 0x7f3ae40d1e00 2024-04-25T08:31:59.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.716+0000 7f3aea858640 1 -- 172.21.15.38:0/2902523933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ae40d1be0 con 0x7f3ae40d1e00 2024-04-25T08:31:59.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.716+0000 7f3ae0ff9640 1 -- 172.21.15.38:0/2902523933 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ad8079a30 con 0x7f3ae40d1e00 2024-04-25T08:31:59.714 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:31:59.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.718+0000 7f3aea858640 1 -- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3ad006b4d0 msgr2=0x7f3ad006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.718+0000 7f3aea858640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3ad006b4d0 0x7f3ad006d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4000f60 tx=0x7f3ad4009040 comp rx=0 tx=0).stop 2024-04-25T08:31:59.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.718+0000 7f3aea858640 1 -- 172.21.15.38:0/2902523933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae40d1e00 msgr2=0x7f3ae415bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.718+0000 7f3aea858640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae40d1e00 0x7f3ae415bfb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8035040 tx=0x7f3ad80046c0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.718+0000 7f3aea858640 1 -- 172.21.15.38:0/2902523933 shutdown_connections 2024-04-25T08:31:59.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.718+0000 7f3aea858640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3ad006b4d0 0x7f3ad006d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.718+0000 7f3aea858640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ae415de60 0x7f3ae40cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.718+0000 7f3aea858640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ae40d1e00 0x7f3ae415bfb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.718+0000 7f3aea858640 1 --2- 172.21.15.38:0/2902523933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ae40cf7b0 0x7f3ae40b8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.718+0000 7f3aea858640 1 -- 172.21.15.38:0/2902523933 >> 172.21.15.38:0/2902523933 conn(0x7f3ae408b000 msgr2=0x7f3ae4073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:59.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.719+0000 7f3aea858640 1 -- 172.21.15.38:0/2902523933 shutdown_connections 2024-04-25T08:31:59.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.719+0000 7f3aea858640 1 -- 172.21.15.38:0/2902523933 wait complete. 2024-04-25T08:31:59.727 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:31:59.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.833+0000 7fd09a50a640 1 Processor -- start 2024-04-25T08:31:59.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.833+0000 7fd09a50a640 1 -- start start 2024-04-25T08:31:59.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd09a50a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd0940d0320 0x7fd0940d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd09a50a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd0940b6980 0x7fd0940b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd09a50a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0940d2060 con 0x7fd0940d0700 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd09a50a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0940d2330 con 0x7fd0940b6980 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd09a50a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0940d2610 con 0x7fd0940d0320 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd098aa7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd0940b6980 0x7fd0940b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd098aa7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd0940b6980 0x7fd0940b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56612/0 (socket says 172.21.15.38:56612) 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd098aa7640 1 -- 172.21.15.38:0/2132288825 learned_addr learned my addr 172.21.15.38:0/2132288825 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd093fff640 1 --2- 172.21.15.38:0/2132288825 >> v2:172.21.15.107:3300/0 conn(0x7fd0940d0320 0x7fd0940d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd098aa7640 1 -- 172.21.15.38:0/2132288825 >> v2:172.21.15.107:3300/0 conn(0x7fd0940d0320 msgr2=0x7fd0940d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd098aa7640 1 --2- 172.21.15.38:0/2132288825 >> v2:172.21.15.107:3300/0 conn(0x7fd0940d0320 0x7fd0940d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd098aa7640 1 -- 172.21.15.38:0/2132288825 >> v1:172.21.15.107:6789/0 conn(0x7fd0940d0700 legacy=0x7fd0940d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd098aa7640 1 -- 172.21.15.38:0/2132288825 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940d28f0 con 0x7fd0940b6980 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.834+0000 7fd093fff640 1 --2- 172.21.15.38:0/2132288825 >> v2:172.21.15.107:3300/0 conn(0x7fd0940d0320 0x7fd0940d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:59.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.835+0000 7fd098aa7640 1 --2- 172.21.15.38:0/2132288825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0940b6980 0x7fd0940b6dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd094069100 tx=0x7fd08c02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc23faf0ebd33e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.835+0000 7fd092ffd640 1 -- 172.21.15.38:0/2132288825 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c036070 con 0x7fd0940b6980 2024-04-25T08:31:59.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.835+0000 7fd092ffd640 1 -- 172.21.15.38:0/2132288825 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c031600 con 0x7fd0940b6980 2024-04-25T08:31:59.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.835+0000 7fd092ffd640 1 -- 172.21.15.38:0/2132288825 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c0319f0 con 0x7fd0940b6980 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.836+0000 7fd09a50a640 1 -- 172.21.15.38:0/2132288825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0940b6980 msgr2=0x7fd0940b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.836+0000 7fd09a50a640 1 --2- 172.21.15.38:0/2132288825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0940b6980 0x7fd0940b6dd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd094069100 tx=0x7fd08c02a7c0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.836+0000 7fd09a50a640 1 -- 172.21.15.38:0/2132288825 shutdown_connections 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.836+0000 7fd09a50a640 1 --2- 172.21.15.38:0/2132288825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0940b6980 0x7fd0940b6dd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.836+0000 7fd09a50a640 1 --2- 172.21.15.38:0/2132288825 >> v2:172.21.15.107:3300/0 conn(0x7fd0940d0320 0x7fd0940d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.836+0000 7fd09a50a640 1 -- 172.21.15.38:0/2132288825 >> 172.21.15.38:0/2132288825 conn(0x7fd0940668f0 msgr2=0x7fd094066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.836+0000 7fd09a50a640 1 -- 172.21.15.38:0/2132288825 shutdown_connections 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.836+0000 7fd09a50a640 1 -- 172.21.15.38:0/2132288825 wait complete. 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.837+0000 7fd09a50a640 1 Processor -- start 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.837+0000 7fd09a50a640 1 -- start start 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd09a50a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0940b6980 0x7fd0940b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd09a50a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0940d0320 0x7fd094156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd09a50a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0940d0700 0x7fd094156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd09a50a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940d0b40 con 0x7fd0940d0320 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd09a50a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940d2eb0 con 0x7fd0940b6980 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd09a50a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0941574e0 con 0x7fd0940d0700 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd0937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0940d0320 0x7fd094156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd0937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0940d0320 0x7fd094156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56616/0 (socket says 172.21.15.38:56616) 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd0937fe640 1 -- 172.21.15.38:0/104194430 learned_addr learned my addr 172.21.15.38:0/104194430 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd093fff640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0940b6980 0x7fd0940b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd098aa7640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0940d0700 0x7fd094156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd0937fe640 1 -- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0940d0700 msgr2=0x7fd094156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd0937fe640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0940d0700 0x7fd094156fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd0937fe640 1 -- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0940b6980 msgr2=0x7fd0940b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:31:59.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd0937fe640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0940b6980 0x7fd0940b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd0937fe640 1 -- 172.21.15.38:0/104194430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0941f7950 con 0x7fd0940d0320 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd098aa7640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0940d0700 0x7fd094156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.838+0000 7fd0937fe640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0940d0320 0x7fd094156a60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c3f0 tx=0x7fd08400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.839+0000 7fd090ff9640 1 -- 172.21.15.38:0/104194430 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0840052d0 con 0x7fd0940d0320 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.839+0000 7fd090ff9640 1 -- 172.21.15.38:0/104194430 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd084005490 con 0x7fd0940d0320 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.839+0000 7fd09a50a640 1 -- 172.21.15.38:0/104194430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0941f7c30 con 0x7fd0940d0320 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.839+0000 7fd090ff9640 1 -- 172.21.15.38:0/104194430 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084007260 con 0x7fd0940d0320 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.839+0000 7fd09a50a640 1 -- 172.21.15.38:0/104194430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0941f81f0 con 0x7fd0940d0320 2024-04-25T08:31:59.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.840+0000 7fd09a50a640 1 -- 172.21.15.38:0/104194430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd094068e80 con 0x7fd0940d0320 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.840+0000 7fd090ff9640 1 -- 172.21.15.38:0/104194430 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd084007400 con 0x7fd0940d0320 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.840+0000 7fd090ff9640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd08006b4d0 0x7fd08006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.841+0000 7fd090ff9640 1 -- 172.21.15.38:0/104194430 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd084089900 con 0x7fd0940d0320 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.841+0000 7fd093fff640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd08006b4d0 0x7fd08006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:31:59.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.841+0000 7fd093fff640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd08006b4d0 0x7fd08006d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd088006fd0 tx=0x7fd088008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:31:59.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:31:59.845+0000 7fd090ff9640 1 -- 172.21.15.38:0/104194430 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd084057df0 con 0x7fd0940d0320 2024-04-25T08:32:00.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.025+0000 7fd09a50a640 1 -- 172.21.15.38:0/104194430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0940b3050 con 0x7fd0940d0320 2024-04-25T08:32:00.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.026+0000 7fd090ff9640 1 -- 172.21.15.38:0/104194430 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd08405be90 con 0x7fd0940d0320 2024-04-25T08:32:00.023 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:00.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.028+0000 7fd09a50a640 1 -- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd08006b4d0 msgr2=0x7fd08006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.028+0000 7fd09a50a640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd08006b4d0 0x7fd08006d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd088006fd0 tx=0x7fd088008040 comp rx=0 tx=0).stop 2024-04-25T08:32:00.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.028+0000 7fd09a50a640 1 -- 172.21.15.38:0/104194430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0940d0320 msgr2=0x7fd094156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.028+0000 7fd09a50a640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0940d0320 0x7fd094156a60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c3f0 tx=0x7fd08400c8b0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.028+0000 7fd09a50a640 1 -- 172.21.15.38:0/104194430 shutdown_connections 2024-04-25T08:32:00.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.028+0000 7fd09a50a640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd08006b4d0 0x7fd08006d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.028+0000 7fd09a50a640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0940d0700 0x7fd094156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.028+0000 7fd09a50a640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0940d0320 0x7fd094156a60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.028+0000 7fd09a50a640 1 --2- 172.21.15.38:0/104194430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0940b6980 0x7fd0940b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.028+0000 7fd09a50a640 1 -- 172.21.15.38:0/104194430 >> 172.21.15.38:0/104194430 conn(0x7fd0940668f0 msgr2=0x7fd0940ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.028+0000 7fd09a50a640 1 -- 172.21.15.38:0/104194430 shutdown_connections 2024-04-25T08:32:00.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.028+0000 7fd09a50a640 1 -- 172.21.15.38:0/104194430 wait complete. 2024-04-25T08:32:00.036 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:00.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.143+0000 7fef0545d640 1 Processor -- start 2024-04-25T08:32:00.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.143+0000 7fef0545d640 1 -- start start 2024-04-25T08:32:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7fef0545d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fef000d33d0 0x7fef000d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7fef0545d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef000d1e90 0x7fef000b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7fef0545d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef000d2330 con 0x7fef000d0320 2024-04-25T08:32:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7fef0545d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef000d2610 con 0x7fef000d33d0 2024-04-25T08:32:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7fef0545d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef000d28f0 con 0x7fef000d1e90 2024-04-25T08:32:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7feefe7fc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fef000d33d0 0x7fef000d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7feeff7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef000d1e90 0x7fef000b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7feefeffd640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fef000d0320 0x7fef000d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:42862/0 (socket says 172.21.15.38:42862) 2024-04-25T08:32:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7feefeffd640 1 -- 172.21.15.38:0/2779133760 learned_addr learned my addr 172.21.15.38:0/2779133760 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7feefe7fc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fef000d33d0 0x7fef000d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:40500/0 (socket says 172.21.15.38:40500) 2024-04-25T08:32:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7feeff7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef000d1e90 0x7fef000b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:46006/0 (socket says 172.21.15.38:46006) 2024-04-25T08:32:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7feeff7fe640 1 -- 172.21.15.38:0/2779133760 >> v2:172.21.15.107:3300/0 conn(0x7fef000d33d0 msgr2=0x7fef000d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7feeff7fe640 1 --2- 172.21.15.38:0/2779133760 >> v2:172.21.15.107:3300/0 conn(0x7fef000d33d0 0x7fef000d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7feeff7fe640 1 -- 172.21.15.38:0/2779133760 >> v1:172.21.15.107:6789/0 conn(0x7fef000d0320 legacy=0x7fef000d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7feeff7fe640 1 -- 172.21.15.38:0/2779133760 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef000d2bd0 con 0x7fef000d1e90 2024-04-25T08:32:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.144+0000 7feefe7fc640 1 --2- 172.21.15.38:0/2779133760 >> v2:172.21.15.107:3300/0 conn(0x7fef000d33d0 0x7fef000d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:00.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.145+0000 7feeff7fe640 1 --2- 172.21.15.38:0/2779133760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef000d1e90 0x7fef000b6980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feef8009d90 tx=0x7feef802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8829fdd4d91df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.145+0000 7feefdffb640 1 -- 172.21.15.38:0/2779133760 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef8007740 con 0x7fef000d1e90 2024-04-25T08:32:00.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.145+0000 7feefdffb640 1 -- 172.21.15.38:0/2779133760 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef80078e0 con 0x7fef000d1e90 2024-04-25T08:32:00.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.146+0000 7feefdffb640 1 -- 172.21.15.38:0/2779133760 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef8007cb0 con 0x7fef000d1e90 2024-04-25T08:32:00.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.146+0000 7fef0545d640 1 -- 172.21.15.38:0/2779133760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef000d1e90 msgr2=0x7fef000b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.146+0000 7fef0545d640 1 --2- 172.21.15.38:0/2779133760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef000d1e90 0x7fef000b6980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7feef8009d90 tx=0x7feef802a6f0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.146+0000 7fef0545d640 1 -- 172.21.15.38:0/2779133760 shutdown_connections 2024-04-25T08:32:00.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.146+0000 7fef0545d640 1 --2- 172.21.15.38:0/2779133760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef000d1e90 0x7fef000b6980 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.146+0000 7fef0545d640 1 --2- 172.21.15.38:0/2779133760 >> v2:172.21.15.107:3300/0 conn(0x7fef000d33d0 0x7fef000d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.146+0000 7fef0545d640 1 -- 172.21.15.38:0/2779133760 >> 172.21.15.38:0/2779133760 conn(0x7fef000668f0 msgr2=0x7fef00066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.146+0000 7fef0545d640 1 -- 172.21.15.38:0/2779133760 shutdown_connections 2024-04-25T08:32:00.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.146+0000 7fef0545d640 1 -- 172.21.15.38:0/2779133760 wait complete. 2024-04-25T08:32:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.147+0000 7fef0545d640 1 Processor -- start 2024-04-25T08:32:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.147+0000 7fef0545d640 1 -- start start 2024-04-25T08:32:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.148+0000 7fef0545d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef000d0320 0x7fef0014dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.148+0000 7fef0545d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef000d1e90 0x7fef000cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.148+0000 7fef0545d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef000d33d0 0x7fef000cd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.148+0000 7fef0545d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef00161390 con 0x7fef000d1e90 2024-04-25T08:32:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.148+0000 7fef0545d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef00161210 con 0x7fef000d33d0 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.148+0000 7fef0545d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef000b9720 con 0x7fef000d0320 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.148+0000 7feefe7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef000d1e90 0x7fef000cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.148+0000 7feefe7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef000d1e90 0x7fef000cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56626/0 (socket says 172.21.15.38:56626) 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.148+0000 7feefe7fc640 1 -- 172.21.15.38:0/1367267418 learned_addr learned my addr 172.21.15.38:0/1367267418 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7feeff7fe640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef000d33d0 0x7fef000cd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7feefeffd640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef000d0320 0x7fef0014dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7feefe7fc640 1 -- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef000d0320 msgr2=0x7fef0014dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7feefe7fc640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef000d0320 0x7fef0014dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7feefe7fc640 1 -- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef000d33d0 msgr2=0x7fef000cd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7feefe7fc640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef000d33d0 0x7fef000cd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7feefe7fc640 1 -- 172.21.15.38:0/1367267418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef001f75a0 con 0x7fef000d1e90 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7feefe7fc640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef000d1e90 0x7fef000cccc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feef400e960 tx=0x7feef400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7feee77fe640 1 -- 172.21.15.38:0/1367267418 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4017070 con 0x7fef000d1e90 2024-04-25T08:32:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7feee77fe640 1 -- 172.21.15.38:0/1367267418 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef400ca00 con 0x7fef000d1e90 2024-04-25T08:32:00.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7fef0545d640 1 -- 172.21.15.38:0/1367267418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef001f7880 con 0x7fef000d1e90 2024-04-25T08:32:00.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7feee77fe640 1 -- 172.21.15.38:0/1367267418 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef400cdd0 con 0x7fef000d1e90 2024-04-25T08:32:00.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.149+0000 7fef0545d640 1 -- 172.21.15.38:0/1367267418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef001f7e40 con 0x7fef000d1e90 2024-04-25T08:32:00.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.150+0000 7fef0545d640 1 -- 172.21.15.38:0/1367267418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef00068e80 con 0x7fef000d1e90 2024-04-25T08:32:00.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.151+0000 7feee77fe640 1 -- 172.21.15.38:0/1367267418 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feef401a070 con 0x7fef000d1e90 2024-04-25T08:32:00.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.151+0000 7feee77fe640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feeec06b4d0 0x7feeec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.151+0000 7feee77fe640 1 -- 172.21.15.38:0/1367267418 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7feef408a7b0 con 0x7fef000d1e90 2024-04-25T08:32:00.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.151+0000 7feefeffd640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feeec06b4d0 0x7feeec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.152+0000 7feefeffd640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feeec06b4d0 0x7feeec06d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feef0000f60 tx=0x7feef0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.155+0000 7feee77fe640 1 -- 172.21.15.38:0/1367267418 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feef40592f0 con 0x7fef000d1e90 2024-04-25T08:32:00.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.336+0000 7fef0545d640 1 -- 172.21.15.38:0/1367267418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fef000cee40 con 0x7fef000d1e90 2024-04-25T08:32:00.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.337+0000 7feee77fe640 1 -- 172.21.15.38:0/1367267418 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7feef4058bf0 con 0x7fef000d1e90 2024-04-25T08:32:00.334 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:00.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.339+0000 7fef0545d640 1 -- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feeec06b4d0 msgr2=0x7feeec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.339+0000 7fef0545d640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feeec06b4d0 0x7feeec06d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7feef0000f60 tx=0x7feef0009040 comp rx=0 tx=0).stop 2024-04-25T08:32:00.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.339+0000 7fef0545d640 1 -- 172.21.15.38:0/1367267418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef000d1e90 msgr2=0x7fef000cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.339+0000 7fef0545d640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef000d1e90 0x7fef000cccc0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7feef400e960 tx=0x7feef400ee20 comp rx=0 tx=0).stop 2024-04-25T08:32:00.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.339+0000 7fef0545d640 1 -- 172.21.15.38:0/1367267418 shutdown_connections 2024-04-25T08:32:00.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.339+0000 7fef0545d640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef000d33d0 0x7fef000cd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.339+0000 7fef0545d640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef000d1e90 0x7fef000cccc0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.339+0000 7fef0545d640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feeec06b4d0 0x7feeec06d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.339+0000 7fef0545d640 1 --2- 172.21.15.38:0/1367267418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef000d0320 0x7fef0014dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.339+0000 7fef0545d640 1 -- 172.21.15.38:0/1367267418 >> 172.21.15.38:0/1367267418 conn(0x7fef000668f0 msgr2=0x7fef000d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.339+0000 7fef0545d640 1 -- 172.21.15.38:0/1367267418 shutdown_connections 2024-04-25T08:32:00.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.339+0000 7fef0545d640 1 -- 172.21.15.38:0/1367267418 wait complete. 2024-04-25T08:32:00.347 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:00.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.451+0000 7f0dbe953640 1 Processor -- start 2024-04-25T08:32:00.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.451+0000 7f0dbe953640 1 -- start start 2024-04-25T08:32:00.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.452+0000 7f0dbe953640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db806c110 0x7f0db806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.452+0000 7f0dbe953640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0db815df50 0x7f0db80ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.452+0000 7f0dbe953640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db806ca30 con 0x7f0db80ce2e0 2024-04-25T08:32:00.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.452+0000 7f0dbe953640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db80cf2d0 con 0x7f0db815df50 2024-04-25T08:32:00.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.452+0000 7f0dbe953640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80ceff0 con 0x7f0db806c110 2024-04-25T08:32:00.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.452+0000 7f0db77fe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0db815df50 0x7f0db80ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.452+0000 7f0db7fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db806c110 0x7f0db806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.452+0000 7f0db7fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db806c110 0x7f0db806c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:46022/0 (socket says 172.21.15.38:46022) 2024-04-25T08:32:00.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.452+0000 7f0db77fe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0db815df50 0x7f0db80ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:40530/0 (socket says 172.21.15.38:40530) 2024-04-25T08:32:00.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.452+0000 7f0db7fff640 1 -- 172.21.15.38:0/906047905 learned_addr learned my addr 172.21.15.38:0/906047905 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:00.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.452+0000 7f0dbcef0640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f0db80ce2e0 0x7f0db80d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:42874/0 (socket says 172.21.15.38:42874) 2024-04-25T08:32:00.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.453+0000 7f0db7fff640 1 -- 172.21.15.38:0/906047905 >> v2:172.21.15.107:3300/0 conn(0x7f0db815df50 msgr2=0x7f0db80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:00.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.453+0000 7f0db7fff640 1 --2- 172.21.15.38:0/906047905 >> v2:172.21.15.107:3300/0 conn(0x7f0db815df50 0x7f0db80ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.453+0000 7f0db7fff640 1 -- 172.21.15.38:0/906047905 >> v1:172.21.15.107:6789/0 conn(0x7f0db80ce2e0 legacy=0x7f0db80d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.453+0000 7f0db7fff640 1 -- 172.21.15.38:0/906047905 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db80d1f70 con 0x7f0db806c110 2024-04-25T08:32:00.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.453+0000 7f0db7fff640 1 --2- 172.21.15.38:0/906047905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db806c110 0x7f0db806c4f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009870 tx=0x7f0da002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b5c35a0181c96c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.453+0000 7f0db6ffd640 1 -- 172.21.15.38:0/906047905 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da0037070 con 0x7f0db806c110 2024-04-25T08:32:00.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.454+0000 7f0db6ffd640 1 -- 172.21.15.38:0/906047905 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da0032440 con 0x7f0db806c110 2024-04-25T08:32:00.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.454+0000 7f0db6ffd640 1 -- 172.21.15.38:0/906047905 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da0032810 con 0x7f0db806c110 2024-04-25T08:32:00.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.454+0000 7f0dbe953640 1 -- 172.21.15.38:0/906047905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db806c110 msgr2=0x7f0db806c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.454+0000 7f0dbe953640 1 --2- 172.21.15.38:0/906047905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db806c110 0x7f0db806c4f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009870 tx=0x7f0da002a270 comp rx=0 tx=0).stop 2024-04-25T08:32:00.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.454+0000 7f0dbe953640 1 -- 172.21.15.38:0/906047905 shutdown_connections 2024-04-25T08:32:00.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.454+0000 7f0dbe953640 1 --2- 172.21.15.38:0/906047905 >> v2:172.21.15.107:3300/0 conn(0x7f0db815df50 0x7f0db80ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.454+0000 7f0dbe953640 1 --2- 172.21.15.38:0/906047905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db806c110 0x7f0db806c4f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.454+0000 7f0dbe953640 1 -- 172.21.15.38:0/906047905 >> 172.21.15.38:0/906047905 conn(0x7f0db80d3fb0 msgr2=0x7f0db80d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.454+0000 7f0dbe953640 1 -- 172.21.15.38:0/906047905 shutdown_connections 2024-04-25T08:32:00.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.454+0000 7f0dbe953640 1 -- 172.21.15.38:0/906047905 wait complete. 2024-04-25T08:32:00.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.455+0000 7f0dbe953640 1 Processor -- start 2024-04-25T08:32:00.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.455+0000 7f0dbe953640 1 -- start start 2024-04-25T08:32:00.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.456+0000 7f0dbe953640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db806c110 0x7f0db8161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.456+0000 7f0dbe953640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80ce2e0 0x7f0db81557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.456+0000 7f0dbe953640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db815df50 0x7f0db80cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.456+0000 7f0dbe953640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80cece0 con 0x7f0db80ce2e0 2024-04-25T08:32:00.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.456+0000 7f0dbe953640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80d2530 con 0x7f0db806c110 2024-04-25T08:32:00.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.456+0000 7f0dbe953640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80d3160 con 0x7f0db815df50 2024-04-25T08:32:00.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.456+0000 7f0db77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80ce2e0 0x7f0db81557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.456+0000 7f0db77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80ce2e0 0x7f0db81557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56634/0 (socket says 172.21.15.38:56634) 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.456+0000 7f0db77fe640 1 -- 172.21.15.38:0/1391629372 learned_addr learned my addr 172.21.15.38:0/1391629372 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.456+0000 7f0dbcef0640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db815df50 0x7f0db80cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.456+0000 7f0db7fff640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db806c110 0x7f0db8161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.457+0000 7f0db77fe640 1 -- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db815df50 msgr2=0x7f0db80cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.457+0000 7f0db77fe640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db815df50 0x7f0db80cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.457+0000 7f0db77fe640 1 -- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db806c110 msgr2=0x7f0db8161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.457+0000 7f0db77fe640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db806c110 0x7f0db8161230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.457+0000 7f0db77fe640 1 -- 172.21.15.38:0/1391629372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db815ec30 con 0x7f0db80ce2e0 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.457+0000 7f0db77fe640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80ce2e0 0x7f0db81557c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0da800e8a0 tx=0x7f0da800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.457+0000 7f0db4ff9640 1 -- 172.21.15.38:0/1391629372 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8017070 con 0x7f0db80ce2e0 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.457+0000 7f0db4ff9640 1 -- 172.21.15.38:0/1391629372 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da800c940 con 0x7f0db80ce2e0 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.457+0000 7f0dbe953640 1 -- 172.21.15.38:0/1391629372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db8156670 con 0x7f0db80ce2e0 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.457+0000 7f0db4ff9640 1 -- 172.21.15.38:0/1391629372 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da800cd30 con 0x7f0db80ce2e0 2024-04-25T08:32:00.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.457+0000 7f0dbe953640 1 -- 172.21.15.38:0/1391629372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db815eea0 con 0x7f0db80ce2e0 2024-04-25T08:32:00.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.458+0000 7f0dbe953640 1 -- 172.21.15.38:0/1391629372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db8068e80 con 0x7f0db80ce2e0 2024-04-25T08:32:00.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.459+0000 7f0db4ff9640 1 -- 172.21.15.38:0/1391629372 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0da801a070 con 0x7f0db80ce2e0 2024-04-25T08:32:00.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.459+0000 7f0db4ff9640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d9806b4d0 0x7f0d9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.459+0000 7f0db4ff9640 1 -- 172.21.15.38:0/1391629372 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0da808a3d0 con 0x7f0db80ce2e0 2024-04-25T08:32:00.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.459+0000 7f0db7fff640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d9806b4d0 0x7f0d9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.460+0000 7f0db7fff640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d9806b4d0 0x7f0d9806d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0da0034000 tx=0x7f0da002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.463+0000 7f0db4ff9640 1 -- 172.21.15.38:0/1391629372 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0db8156670 con 0x7f0db80ce2e0 2024-04-25T08:32:00.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.682+0000 7f0dbe953640 1 -- 172.21.15.38:0/1391629372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0db80c8480 con 0x7f0db80ce2e0 2024-04-25T08:32:00.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.682+0000 7f0db4ff9640 1 -- 172.21.15.38:0/1391629372 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f0da8058f10 con 0x7f0db80ce2e0 2024-04-25T08:32:00.680 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:00.680 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4242612806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4242612806},{"type":"v1","addr":"172.21.15.38:6835","nonce":4242612806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2154167637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":2154167637},{"type":"v1","addr":"172.21.15.107:6837","nonce":2154167637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/678452936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":678452936},{"type":"v1","addr":"172.21.15.38:6834","nonce":678452936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:31:35.665008+0000","modified":"2024-04-25T08:31:51.799830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6836/1104921666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1104921666},{"type":"v1","addr":"172.21.15.107:6836","nonce":1104921666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:32:00.680 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-04-25T08:32:00.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.685+0000 7f0dbe953640 1 -- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d9806b4d0 msgr2=0x7f0d9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.685+0000 7f0dbe953640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d9806b4d0 0x7f0d9806d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0da0034000 tx=0x7f0da002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.685+0000 7f0dbe953640 1 -- 172.21.15.38:0/1391629372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80ce2e0 msgr2=0x7f0db81557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.685+0000 7f0dbe953640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80ce2e0 0x7f0db81557c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0da800e8a0 tx=0x7f0da800ed60 comp rx=0 tx=0).stop 2024-04-25T08:32:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.685+0000 7f0dbe953640 1 -- 172.21.15.38:0/1391629372 shutdown_connections 2024-04-25T08:32:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.685+0000 7f0dbe953640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d9806b4d0 0x7f0d9806d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.685+0000 7f0dbe953640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db815df50 0x7f0db80cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.685+0000 7f0dbe953640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80ce2e0 0x7f0db81557c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.685+0000 7f0dbe953640 1 --2- 172.21.15.38:0/1391629372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db806c110 0x7f0db8161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.685+0000 7f0dbe953640 1 -- 172.21.15.38:0/1391629372 >> 172.21.15.38:0/1391629372 conn(0x7f0db80d3fb0 msgr2=0x7f0db8161840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.685+0000 7f0dbe953640 1 -- 172.21.15.38:0/1391629372 shutdown_connections 2024-04-25T08:32:00.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.685+0000 7f0dbe953640 1 -- 172.21.15.38:0/1391629372 wait complete. 2024-04-25T08:32:00.693 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:00.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.796+0000 7fab1c57e640 1 Processor -- start 2024-04-25T08:32:00.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.796+0000 7fab1c57e640 1 -- start start 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab1c57e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fab1415dc40 0x7fab140d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab1c57e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fab140b9580 0x7fab140d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab1c57e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab140b8400 con 0x7fab140d1e40 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab1c57e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab140d19b0 con 0x7fab140b9580 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab1c57e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab140b86e0 con 0x7fab1415dc40 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab19b19640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fab140b9580 0x7fab140d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab1ab1b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fab140d1e40 0x7fab140d2250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54862/0 (socket says 172.21.15.38:54862) 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab1ab1b640 1 -- 172.21.15.38:0/3115478682 learned_addr learned my addr 172.21.15.38:0/3115478682 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab1a31a640 1 --2- 172.21.15.38:0/3115478682 >> v2:172.21.15.107:3300/0 conn(0x7fab1415dc40 0x7fab140d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab19b19640 1 -- 172.21.15.38:0/3115478682 >> v2:172.21.15.107:3300/0 conn(0x7fab1415dc40 msgr2=0x7fab140d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab19b19640 1 --2- 172.21.15.38:0/3115478682 >> v2:172.21.15.107:3300/0 conn(0x7fab1415dc40 0x7fab140d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab19b19640 1 -- 172.21.15.38:0/3115478682 >> v1:172.21.15.38:6789/0 conn(0x7fab140d1e40 legacy=0x7fab140d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab19318640 1 -- 172.21.15.38:0/3115478682 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214309869 0 0) 0x7fab140b8400 con 0x7fab140d1e40 2024-04-25T08:32:00.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.797+0000 7fab19b19640 1 -- 172.21.15.38:0/3115478682 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab140b89c0 con 0x7fab140b9580 2024-04-25T08:32:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.798+0000 7fab19b19640 1 --2- 172.21.15.38:0/3115478682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab140b9580 0x7fab140d0e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fab140d1b30 tx=0x7fab0402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a41909ba42c686 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.798+0000 7fab19318640 1 -- 172.21.15.38:0/3115478682 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04037070 con 0x7fab140b9580 2024-04-25T08:32:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.798+0000 7fab19318640 1 -- 172.21.15.38:0/3115478682 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab04032440 con 0x7fab140b9580 2024-04-25T08:32:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.798+0000 7fab19318640 1 -- 172.21.15.38:0/3115478682 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0402ae60 con 0x7fab140b9580 2024-04-25T08:32:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.799+0000 7fab1c57e640 1 -- 172.21.15.38:0/3115478682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab140b9580 msgr2=0x7fab140d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.799+0000 7fab1c57e640 1 --2- 172.21.15.38:0/3115478682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab140b9580 0x7fab140d0e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fab140d1b30 tx=0x7fab0402a110 comp rx=0 tx=0).stop 2024-04-25T08:32:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.799+0000 7fab1c57e640 1 -- 172.21.15.38:0/3115478682 shutdown_connections 2024-04-25T08:32:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.799+0000 7fab1c57e640 1 --2- 172.21.15.38:0/3115478682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab140b9580 0x7fab140d0e10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.799+0000 7fab1c57e640 1 --2- 172.21.15.38:0/3115478682 >> v2:172.21.15.107:3300/0 conn(0x7fab1415dc40 0x7fab140d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.799+0000 7fab1c57e640 1 -- 172.21.15.38:0/3115478682 >> 172.21.15.38:0/3115478682 conn(0x7fab14066ce0 msgr2=0x7fab140670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.799+0000 7fab1c57e640 1 -- 172.21.15.38:0/3115478682 shutdown_connections 2024-04-25T08:32:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.799+0000 7fab1c57e640 1 -- 172.21.15.38:0/3115478682 wait complete. 2024-04-25T08:32:00.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.800+0000 7fab1c57e640 1 Processor -- start 2024-04-25T08:32:00.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.800+0000 7fab1c57e640 1 -- start start 2024-04-25T08:32:00.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab1c57e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab140b9580 0x7fab14158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab1c57e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab140d1e40 0x7fab1415f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab1c57e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1415dc40 0x7fab1415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab1c57e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab140b7bf0 con 0x7fab140d1e40 2024-04-25T08:32:00.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab1c57e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab140b7a70 con 0x7fab1415dc40 2024-04-25T08:32:00.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab1c57e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab140b7d70 con 0x7fab140b9580 2024-04-25T08:32:00.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab19b19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab140d1e40 0x7fab1415f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab19b19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab140d1e40 0x7fab1415f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56662/0 (socket says 172.21.15.38:56662) 2024-04-25T08:32:00.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab19b19640 1 -- 172.21.15.38:0/2131826190 learned_addr learned my addr 172.21.15.38:0/2131826190 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab1ab1b640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1415dc40 0x7fab1415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab1a31a640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab140b9580 0x7fab14158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab19b19640 1 -- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab140b9580 msgr2=0x7fab14158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab19b19640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab140b9580 0x7fab14158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab19b19640 1 -- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1415dc40 msgr2=0x7fab1415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab19b19640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1415dc40 0x7fab1415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab19b19640 1 -- 172.21.15.38:0/2131826190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1415ff00 con 0x7fab140d1e40 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab1ab1b640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1415dc40 0x7fab1415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab1a31a640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab140b9580 0x7fab14158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.801+0000 7fab19b19640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab140d1e40 0x7fab1415f800 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fab04000c00 tx=0x7fab040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.802+0000 7fab0affd640 1 -- 172.21.15.38:0/2131826190 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04037040 con 0x7fab140d1e40 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.802+0000 7fab0affd640 1 -- 172.21.15.38:0/2131826190 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab04042070 con 0x7fab140d1e40 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.802+0000 7fab0affd640 1 -- 172.21.15.38:0/2131826190 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04030bd0 con 0x7fab140d1e40 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.802+0000 7fab1c57e640 1 -- 172.21.15.38:0/2131826190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab141601e0 con 0x7fab140d1e40 2024-04-25T08:32:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.802+0000 7fab1c57e640 1 -- 172.21.15.38:0/2131826190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab141607a0 con 0x7fab140d1e40 2024-04-25T08:32:00.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.803+0000 7fab1c57e640 1 -- 172.21.15.38:0/2131826190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab140b8f80 con 0x7fab140d1e40 2024-04-25T08:32:00.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.803+0000 7fab0affd640 1 -- 172.21.15.38:0/2131826190 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab04047460 con 0x7fab140d1e40 2024-04-25T08:32:00.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.804+0000 7fab0affd640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faaf806b4d0 0x7faaf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:00.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.804+0000 7fab0affd640 1 -- 172.21.15.38:0/2131826190 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fab040ab0b0 con 0x7fab140d1e40 2024-04-25T08:32:00.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.804+0000 7fab1a31a640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faaf806b4d0 0x7faaf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:00.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.804+0000 7fab1a31a640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faaf806b4d0 0x7faaf806d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fab10006fd0 tx=0x7fab10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:00.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.808+0000 7fab0affd640 1 -- 172.21.15.38:0/2131826190 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab140b8f80 con 0x7fab140d1e40 2024-04-25T08:32:00.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.988+0000 7fab1c57e640 1 -- 172.21.15.38:0/2131826190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab14068e80 con 0x7fab140d1e40 2024-04-25T08:32:00.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.989+0000 7fab0affd640 1 -- 172.21.15.38:0/2131826190 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7fab04032e20 con 0x7fab140d1e40 2024-04-25T08:32:00.987 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:00.987 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:00.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.991+0000 7fab1c57e640 1 -- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faaf806b4d0 msgr2=0x7faaf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.991+0000 7fab1c57e640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faaf806b4d0 0x7faaf806d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fab10006fd0 tx=0x7fab10008040 comp rx=0 tx=0).stop 2024-04-25T08:32:00.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.991+0000 7fab1c57e640 1 -- 172.21.15.38:0/2131826190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab140d1e40 msgr2=0x7fab1415f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:00.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.991+0000 7fab1c57e640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab140d1e40 0x7fab1415f800 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fab04000c00 tx=0x7fab040046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.991+0000 7fab1c57e640 1 -- 172.21.15.38:0/2131826190 shutdown_connections 2024-04-25T08:32:00.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.991+0000 7fab1c57e640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1415dc40 0x7fab1415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.991+0000 7fab1c57e640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab140d1e40 0x7fab1415f800 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.991+0000 7fab1c57e640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faaf806b4d0 0x7faaf806d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.991+0000 7fab1c57e640 1 --2- 172.21.15.38:0/2131826190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab140b9580 0x7fab14158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:00.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.991+0000 7fab1c57e640 1 -- 172.21.15.38:0/2131826190 >> 172.21.15.38:0/2131826190 conn(0x7fab14066ce0 msgr2=0x7fab1406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:00.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.991+0000 7fab1c57e640 1 -- 172.21.15.38:0/2131826190 shutdown_connections 2024-04-25T08:32:00.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:00.991+0000 7fab1c57e640 1 -- 172.21.15.38:0/2131826190 wait complete. 2024-04-25T08:32:01.000 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:01.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.104+0000 7f4fe3329640 1 Processor -- start 2024-04-25T08:32:01.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.104+0000 7f4fe3329640 1 -- start start 2024-04-25T08:32:01.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe3329640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4fdc15dc10 0x7f4fdc0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe3329640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4fdc0ced60 0x7f4fdc0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe3329640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fdc0d3900 0x7f4fdc0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe3329640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fdc15dff0 con 0x7f4fdc15dc10 2024-04-25T08:32:01.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe3329640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fdc0b6990 con 0x7f4fdc0ced60 2024-04-25T08:32:01.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe3329640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0d4710 con 0x7f4fdc0d3900 2024-04-25T08:32:01.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe10c5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4fdc15dc10 0x7f4fdc0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe10c5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4fdc15dc10 0x7f4fdc0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56672/0 (socket says 172.21.15.38:56672) 2024-04-25T08:32:01.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe10c5640 1 -- 172.21.15.38:0/1075548308 learned_addr learned my addr 172.21.15.38:0/1075548308 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:01.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe08c4640 1 --2- 172.21.15.38:0/1075548308 >> v2:172.21.15.107:3300/0 conn(0x7f4fdc0ced60 0x7f4fdc0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe18c6640 1 --2- 172.21.15.38:0/1075548308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fdc0d3900 0x7f4fdc0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe10c5640 1 -- 172.21.15.38:0/1075548308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fdc0d3900 msgr2=0x7f4fdc0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe10c5640 1 --2- 172.21.15.38:0/1075548308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fdc0d3900 0x7f4fdc0d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe10c5640 1 -- 172.21.15.38:0/1075548308 >> v2:172.21.15.107:3300/0 conn(0x7f4fdc0ced60 msgr2=0x7f4fdc0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe10c5640 1 --2- 172.21.15.38:0/1075548308 >> v2:172.21.15.107:3300/0 conn(0x7f4fdc0ced60 0x7f4fdc0d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.105+0000 7f4fe10c5640 1 -- 172.21.15.38:0/1075548308 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc0d3e70 con 0x7f4fdc15dc10 2024-04-25T08:32:01.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.106+0000 7f4fe10c5640 1 --2- 172.21.15.38:0/1075548308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fdc15dc10 0x7f4fdc0ce820 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0b9600 tx=0x7f4fd802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5712474a05c4d94d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.106+0000 7f4fd3fff640 1 -- 172.21.15.38:0/1075548308 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8037070 con 0x7f4fdc15dc10 2024-04-25T08:32:01.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.106+0000 7f4fd3fff640 1 -- 172.21.15.38:0/1075548308 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd8032440 con 0x7f4fdc15dc10 2024-04-25T08:32:01.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.106+0000 7f4fd3fff640 1 -- 172.21.15.38:0/1075548308 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd8032810 con 0x7f4fdc15dc10 2024-04-25T08:32:01.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.107+0000 7f4fe3329640 1 -- 172.21.15.38:0/1075548308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fdc15dc10 msgr2=0x7f4fdc0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.107+0000 7f4fe3329640 1 --2- 172.21.15.38:0/1075548308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fdc15dc10 0x7f4fdc0ce820 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0b9600 tx=0x7f4fd802a140 comp rx=0 tx=0).stop 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.107+0000 7f4fe3329640 1 -- 172.21.15.38:0/1075548308 shutdown_connections 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.107+0000 7f4fe3329640 1 --2- 172.21.15.38:0/1075548308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fdc0d3900 0x7f4fdc0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.107+0000 7f4fe3329640 1 --2- 172.21.15.38:0/1075548308 >> v2:172.21.15.107:3300/0 conn(0x7f4fdc0ced60 0x7f4fdc0d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.107+0000 7f4fe3329640 1 --2- 172.21.15.38:0/1075548308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fdc15dc10 0x7f4fdc0ce820 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.107+0000 7f4fe3329640 1 -- 172.21.15.38:0/1075548308 >> 172.21.15.38:0/1075548308 conn(0x7f4fdc066d10 msgr2=0x7f4fdc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.107+0000 7f4fe3329640 1 -- 172.21.15.38:0/1075548308 shutdown_connections 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.107+0000 7f4fe3329640 1 -- 172.21.15.38:0/1075548308 wait complete. 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.108+0000 7f4fe3329640 1 Processor -- start 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.108+0000 7f4fe3329640 1 -- start start 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe3329640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fdc0ced60 0x7f4fdc0aa5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe3329640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fdc0d3900 0x7f4fdc1f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe3329640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fdc15dc10 0x7f4fdc0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe3329640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0b6bc0 con 0x7f4fdc15dc10 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe3329640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0d4430 con 0x7f4fdc0d3900 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe3329640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fdc0b2370 con 0x7f4fdc0ced60 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe18c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fdc15dc10 0x7f4fdc0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe18c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fdc15dc10 0x7f4fdc0c9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56682/0 (socket says 172.21.15.38:56682) 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe18c6640 1 -- 172.21.15.38:0/101055245 learned_addr learned my addr 172.21.15.38:0/101055245 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe08c4640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fdc0d3900 0x7f4fdc1f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe10c5640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fdc0ced60 0x7f4fdc0aa5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe18c6640 1 -- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fdc0ced60 msgr2=0x7f4fdc0aa5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe18c6640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fdc0ced60 0x7f4fdc0aa5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe18c6640 1 -- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fdc0d3900 msgr2=0x7f4fdc1f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe18c6640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fdc0d3900 0x7f4fdc1f3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe18c6640 1 -- 172.21.15.38:0/101055245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fdc1f43c0 con 0x7f4fdc15dc10 2024-04-25T08:32:01.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe10c5640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fdc0ced60 0x7f4fdc0aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:01.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fe18c6640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fdc15dc10 0x7f4fdc0c9b10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400cd70 tx=0x7f4fd400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fd1ffb640 1 -- 172.21.15.38:0/101055245 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4003b90 con 0x7f4fdc15dc10 2024-04-25T08:32:01.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.109+0000 7f4fd1ffb640 1 -- 172.21.15.38:0/101055245 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd4003d30 con 0x7f4fdc15dc10 2024-04-25T08:32:01.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.110+0000 7f4fd1ffb640 1 -- 172.21.15.38:0/101055245 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd4004240 con 0x7f4fdc15dc10 2024-04-25T08:32:01.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.110+0000 7f4fe3329640 1 -- 172.21.15.38:0/101055245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fdc1f3ac0 con 0x7f4fdc15dc10 2024-04-25T08:32:01.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.110+0000 7f4fe3329640 1 -- 172.21.15.38:0/101055245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fdc1f4660 con 0x7f4fdc15dc10 2024-04-25T08:32:01.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.111+0000 7f4fe3329640 1 -- 172.21.15.38:0/101055245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fdc068e80 con 0x7f4fdc15dc10 2024-04-25T08:32:01.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.111+0000 7f4fd1ffb640 1 -- 172.21.15.38:0/101055245 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fd40043e0 con 0x7f4fdc15dc10 2024-04-25T08:32:01.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.112+0000 7f4fd1ffb640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4fc006b4d0 0x7f4fc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.112+0000 7f4fd1ffb640 1 -- 172.21.15.38:0/101055245 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4fd408af50 con 0x7f4fdc15dc10 2024-04-25T08:32:01.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.112+0000 7f4fe10c5640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4fc006b4d0 0x7f4fc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.112+0000 7f4fe10c5640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4fc006b4d0 0x7f4fc006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8034000 tx=0x7f4fd802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.116+0000 7f4fd1ffb640 1 -- 172.21.15.38:0/101055245 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fd4059b20 con 0x7f4fdc15dc10 2024-04-25T08:32:01.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.337+0000 7f4fe3329640 1 -- 172.21.15.38:0/101055245 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fdc1f4830 con 0x7f4fdc15dc10 2024-04-25T08:32:01.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.338+0000 7f4fd1ffb640 1 -- 172.21.15.38:0/101055245 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f4fd4059420 con 0x7f4fdc15dc10 2024-04-25T08:32:01.336 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:01.336 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4242612806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4242612806},{"type":"v1","addr":"172.21.15.38:6835","nonce":4242612806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2154167637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":2154167637},{"type":"v1","addr":"172.21.15.107:6837","nonce":2154167637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/678452936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":678452936},{"type":"v1","addr":"172.21.15.38:6834","nonce":678452936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:31:35.665008+0000","modified":"2024-04-25T08:31:51.799830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6836/1104921666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1104921666},{"type":"v1","addr":"172.21.15.107:6836","nonce":1104921666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:32:01.336 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-04-25T08:32:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.340+0000 7f4fe3329640 1 -- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4fc006b4d0 msgr2=0x7f4fc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.340+0000 7f4fe3329640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4fc006b4d0 0x7f4fc006d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8034000 tx=0x7f4fd802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.340+0000 7f4fe3329640 1 -- 172.21.15.38:0/101055245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fdc15dc10 msgr2=0x7f4fdc0c9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.340+0000 7f4fe3329640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fdc15dc10 0x7f4fdc0c9b10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4fd400cd70 tx=0x7f4fd400a660 comp rx=0 tx=0).stop 2024-04-25T08:32:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.340+0000 7f4fe3329640 1 -- 172.21.15.38:0/101055245 shutdown_connections 2024-04-25T08:32:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.340+0000 7f4fe3329640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fdc15dc10 0x7f4fdc0c9b10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.340+0000 7f4fe3329640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4fdc0d3900 0x7f4fdc1f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.340+0000 7f4fe3329640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4fc006b4d0 0x7f4fc006d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.340+0000 7f4fe3329640 1 --2- 172.21.15.38:0/101055245 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4fdc0ced60 0x7f4fdc0aa5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.340+0000 7f4fe3329640 1 -- 172.21.15.38:0/101055245 >> 172.21.15.38:0/101055245 conn(0x7f4fdc066d10 msgr2=0x7f4fdc0d2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.340+0000 7f4fe3329640 1 -- 172.21.15.38:0/101055245 shutdown_connections 2024-04-25T08:32:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.340+0000 7f4fe3329640 1 -- 172.21.15.38:0/101055245 wait complete. 2024-04-25T08:32:01.751 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:31:35.665008+0000', 'modified': '2024-04-25T08:31:51.799830+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.107:6836/1104921666', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6834', 'nonce': 1104921666}, {'type': 'v1', 'addr': '172.21.15.107:6836', 'nonce': 1104921666}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:32:01.751 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:32:01.752 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T08:32:01.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.859+0000 7f552c9a5640 1 Processor -- start 2024-04-25T08:32:01.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.859+0000 7f552c9a5640 1 -- start start 2024-04-25T08:32:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.860+0000 7f552c9a5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f55281609a0 0x7f5528168b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.860+0000 7f552c9a5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552806d9f0 con 0x7f55280cf7b0 2024-04-25T08:32:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.860+0000 7f552c9a5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552806ce70 con 0x7f5528099a70 2024-04-25T08:32:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.860+0000 7f552c9a5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552806cc30 con 0x7f55281609a0 2024-04-25T08:32:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.860+0000 7f5525d9b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f55280cf7b0 0x7f552806c520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54866/0 (socket says 172.21.15.38:54866) 2024-04-25T08:32:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.860+0000 7f5525d9b640 1 -- 172.21.15.38:0/173709475 learned_addr learned my addr 172.21.15.38:0/173709475 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.860+0000 7f5526d9d640 1 --2- 172.21.15.38:0/173709475 >> v2:172.21.15.107:3300/0 conn(0x7f55281609a0 0x7f5528168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.860+0000 7f552559a640 1 -- 172.21.15.38:0/173709475 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849721322 0 0) 0x7f552806d9f0 con 0x7f55280cf7b0 2024-04-25T08:32:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.860+0000 7f552559a640 1 -- 172.21.15.38:0/173709475 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5510003520 con 0x7f55280cf7b0 2024-04-25T08:32:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.861+0000 7f552559a640 1 -- 172.21.15.38:0/173709475 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 863682535 0 0) 0x7f552806ce70 con 0x7f5528099a70 2024-04-25T08:32:01.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.861+0000 7f552559a640 1 -- 172.21.15.38:0/173709475 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f552806d9f0 con 0x7f5528099a70 2024-04-25T08:32:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.861+0000 7f5526d9d640 1 -- 172.21.15.38:0/173709475 >> v1:172.21.15.107:6789/0 conn(0x7f5528099a70 legacy=0x7f5528099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.861+0000 7f5526d9d640 1 -- 172.21.15.38:0/173709475 >> v1:172.21.15.38:6789/0 conn(0x7f55280cf7b0 legacy=0x7f552806c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.861+0000 7f552559a640 1 -- 172.21.15.38:0/173709475 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4279290045 0 0) 0x7f5510003520 con 0x7f55280cf7b0 2024-04-25T08:32:01.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.861+0000 7f5526d9d640 1 -- 172.21.15.38:0/173709475 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552806d150 con 0x7f55281609a0 2024-04-25T08:32:01.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.861+0000 7f5526d9d640 1 --2- 172.21.15.38:0/173709475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55281609a0 0x7f5528168b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5520009d90 tx=0x7f552002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4623b1d78a53fd56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.862+0000 7f552559a640 1 -- 172.21.15.38:0/173709475 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c001670 con 0x7f55281609a0 2024-04-25T08:32:01.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.862+0000 7f552559a640 1 -- 172.21.15.38:0/173709475 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5520007540 con 0x7f55281609a0 2024-04-25T08:32:01.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.862+0000 7f552559a640 1 -- 172.21.15.38:0/173709475 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520007910 con 0x7f55281609a0 2024-04-25T08:32:01.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.862+0000 7f552c9a5640 1 -- 172.21.15.38:0/173709475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55281609a0 msgr2=0x7f5528168b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.862+0000 7f552c9a5640 1 --2- 172.21.15.38:0/173709475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55281609a0 0x7f5528168b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5520009d90 tx=0x7f552002a5f0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.862+0000 7f552c9a5640 1 -- 172.21.15.38:0/173709475 shutdown_connections 2024-04-25T08:32:01.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.862+0000 7f552c9a5640 1 --2- 172.21.15.38:0/173709475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55281609a0 0x7f5528168b50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.862+0000 7f552c9a5640 1 -- 172.21.15.38:0/173709475 >> 172.21.15.38:0/173709475 conn(0x7f55280b3190 msgr2=0x7f55280b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.862+0000 7f552c9a5640 1 -- 172.21.15.38:0/173709475 shutdown_connections 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.862+0000 7f552c9a5640 1 -- 172.21.15.38:0/173709475 wait complete. 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.864+0000 7f552c9a5640 1 Processor -- start 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.864+0000 7f552c9a5640 1 -- start start 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.864+0000 7f552c9a5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5528099a70 0x7f5528152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.864+0000 7f552c9a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55280cf7b0 0x7f55280c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.864+0000 7f552c9a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55281609a0 0x7f55280ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.864+0000 7f552c9a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55280d08a0 con 0x7f55281609a0 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.864+0000 7f552c9a5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552806d710 con 0x7f55280cf7b0 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.864+0000 7f552c9a5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5528169090 con 0x7f5528099a70 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.864+0000 7f5526d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55281609a0 0x7f55280ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.864+0000 7f5526d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55281609a0 0x7f55280ce120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56688/0 (socket says 172.21.15.38:56688) 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.864+0000 7f5526d9d640 1 -- 172.21.15.38:0/150538067 learned_addr learned my addr 172.21.15.38:0/150538067 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f5525d9b640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55280cf7b0 0x7f55280c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f552659c640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5528099a70 0x7f5528152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f5526d9d640 1 -- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5528099a70 msgr2=0x7f5528152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f5526d9d640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5528099a70 0x7f5528152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f5526d9d640 1 -- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55280cf7b0 msgr2=0x7f55280c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f5526d9d640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55280cf7b0 0x7f55280c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f5526d9d640 1 -- 172.21.15.38:0/150538067 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55280ca8b0 con 0x7f55281609a0 2024-04-25T08:32:01.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f5526d9d640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55281609a0 0x7f55280ce120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5520009d90 tx=0x7f5520004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f5516ffd640 1 -- 172.21.15.38:0/150538067 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552003f070 con 0x7f55281609a0 2024-04-25T08:32:01.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f5516ffd640 1 -- 172.21.15.38:0/150538067 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5520031bc0 con 0x7f55281609a0 2024-04-25T08:32:01.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f552c9a5640 1 -- 172.21.15.38:0/150538067 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55280b4510 con 0x7f55281609a0 2024-04-25T08:32:01.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f5516ffd640 1 -- 172.21.15.38:0/150538067 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520007e90 con 0x7f55281609a0 2024-04-25T08:32:01.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.865+0000 7f552c9a5640 1 -- 172.21.15.38:0/150538067 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55280b4ad0 con 0x7f55281609a0 2024-04-25T08:32:01.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.867+0000 7f5516ffd640 1 -- 172.21.15.38:0/150538067 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f552003b410 con 0x7f55281609a0 2024-04-25T08:32:01.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.867+0000 7f5516ffd640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f550806b500 0x7f550806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.867+0000 7f5516ffd640 1 -- 172.21.15.38:0/150538067 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f55200ab850 con 0x7f55281609a0 2024-04-25T08:32:01.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.867+0000 7f552659c640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f550806b500 0x7f550806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.868+0000 7f552659c640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f550806b500 0x7f550806d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f55180011e0 tx=0x7f5518009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.868+0000 7f552f786740 1 Processor -- start 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.868+0000 7f552f786740 1 -- start start 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f552f786740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29605230 0x557c29605610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f552f786740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 0x557c29623020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f552f786740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c295daf10 0x557c29621190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f552f786740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2953da10 con 0x557c29622ad0 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f552f786740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c294e6b30 con 0x557c29605230 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f552f786740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c29539c50 con 0x557c295daf10 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f5525d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 0x557c29623020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f5525d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 0x557c29623020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56690/0 (socket says 172.21.15.38:56690) 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f552659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29605230 0x557c29605610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f5525d9b640 1 -- 172.21.15.38:0/347432043 learned_addr learned my addr 172.21.15.38:0/347432043 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:01.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f5526d9d640 1 --2- 172.21.15.38:0/347432043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c295daf10 0x557c29621190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f5525d9b640 1 -- 172.21.15.38:0/347432043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c295daf10 msgr2=0x557c29621190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f5525d9b640 1 --2- 172.21.15.38:0/347432043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c295daf10 0x557c29621190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f5525d9b640 1 -- 172.21.15.38:0/347432043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29605230 msgr2=0x557c29605610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f5525d9b640 1 --2- 172.21.15.38:0/347432043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29605230 0x557c29605610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f5525d9b640 1 -- 172.21.15.38:0/347432043 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c294fbe10 con 0x557c29622ad0 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f5525d9b640 1 --2- 172.21.15.38:0/347432043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 0x557c29623020 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f551c001e20 tx=0x7f551c007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b44dc9b1ecdc243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.869+0000 7f5507fff640 1 -- 172.21.15.38:0/347432043 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c018070 con 0x557c29622ad0 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.870+0000 7f5507fff640 1 -- 172.21.15.38:0/347432043 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c005d00 con 0x557c29622ad0 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.870+0000 7f5507fff640 1 -- 172.21.15.38:0/347432043 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c007310 con 0x557c29622ad0 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.870+0000 7f552f786740 1 -- 172.21.15.38:0/347432043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 msgr2=0x557c29623020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.870+0000 7f552f786740 1 --2- 172.21.15.38:0/347432043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 0x557c29623020 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f551c001e20 tx=0x7f551c007d80 comp rx=0 tx=0).stop 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.870+0000 7f552f786740 1 -- 172.21.15.38:0/347432043 shutdown_connections 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.870+0000 7f552f786740 1 --2- 172.21.15.38:0/347432043 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c295daf10 0x557c29621190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.870+0000 7f552f786740 1 --2- 172.21.15.38:0/347432043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 0x557c29623020 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.870+0000 7f552f786740 1 --2- 172.21.15.38:0/347432043 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29605230 0x557c29605610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.870+0000 7f552f786740 1 -- 172.21.15.38:0/347432043 >> 172.21.15.38:0/347432043 conn(0x557c295d7840 msgr2=0x557c29620040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.870+0000 7f552f786740 1 -- 172.21.15.38:0/347432043 shutdown_connections 2024-04-25T08:32:01.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.870+0000 7f552f786740 1 -- 172.21.15.38:0/347432043 wait complete. 2024-04-25T08:32:01.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.871+0000 7f552f786740 1 -- start start 2024-04-25T08:32:01.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.871+0000 7f552f786740 1 Processor -- start 2024-04-25T08:32:01.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f552f786740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29605230 0x557c29601d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f552f786740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 0x557c29630500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f552f786740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c296171a0 0x557c29608ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f552f786740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c295dbcb0 con 0x557c29622ad0 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f552f786740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c294fa0c0 con 0x557c29605230 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f552f786740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c295d5770 con 0x557c296171a0 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f552f786740 10 client.0 _ll_register_callbacks cb 0x557c294edec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f5525d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 0x557c29630500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f5525d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 0x557c29630500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56704/0 (socket says 172.21.15.38:56704) 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f5525d9b640 1 -- 172.21.15.38:0/2024370234 learned_addr learned my addr 172.21.15.38:0/2024370234 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f552659c640 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29605230 0x557c29601d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f5526d9d640 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c296171a0 0x557c29608ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f5525d9b640 1 -- 172.21.15.38:0/2024370234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c296171a0 msgr2=0x557c29608ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f5525d9b640 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c296171a0 0x557c29608ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f5525d9b640 1 -- 172.21.15.38:0/2024370234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29605230 msgr2=0x557c29601d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f5525d9b640 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29605230 0x557c29601d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.872+0000 7f5525d9b640 1 -- 172.21.15.38:0/2024370234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c292522d0 con 0x557c29622ad0 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f5525d9b640 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 0x557c29630500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f551c012000 tx=0x7f551c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1725d2618b970e14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f55057fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f55057fa640 1 -- 172.21.15.38:0/2024370234 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551c018030 con 0x557c29622ad0 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f55057fa640 1 -- 172.21.15.38:0/2024370234 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f551c0075c0 con 0x557c29622ad0 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f552f786740 1 -- 172.21.15.38:0/2024370234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f551c005d00 con 0x557c29622ad0 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f55057fa640 1 -- 172.21.15.38:0/2024370234 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f551c02c050 con 0x557c29622ad0 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f552f786740 10 client.4773 fetch_fsmap learned FSMap version 9 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f552f786740 1 -- 172.21.15.38:0/2024370234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x557c2931c6b0 con 0x557c29622ad0 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f55057fa640 1 -- 172.21.15.38:0/2024370234 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551c005d00 con 0x557c29622ad0 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f55057fa640 1 -- 172.21.15.38:0/2024370234 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f551c02f020 con 0x557c29622ad0 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f552f786740 10 client.4773 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f552f786740 10 client.4773 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:32:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f552f786740 20 client.4773 populate_metadata read hostname 'smithi038' 2024-04-25T08:32:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f552f786740 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x557c29606610 0x557c296069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f552f786740 4 client.4773 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f552f786740 1 -- 172.21.15.38:0/2024370234 --> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c2959c920 con 0x557c29606610 2024-04-25T08:32:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.873+0000 7f552659c640 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x557c29606610 0x557c296069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.874+0000 7f55057fa640 10 client.4773 ms_handle_connect on v2:172.21.15.107:6834/1104921666 2024-04-25T08:32:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.874+0000 7f552659c640 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x557c29606610 0x557c296069f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cc943334f912691 server_cookie=a6aeeb4cb0fae1c7 in_seq=0 out_seq=0 2024-04-25T08:32:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.874+0000 7f55057fa640 1 -- 172.21.15.38:0/2024370234 <== mds.0 v2:172.21.15.107:6834/1104921666 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c2959c920 con 0x557c29606610 2024-04-25T08:32:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.874+0000 7f55057fa640 10 client.4773 handle_command_reply: tid=0 2024-04-25T08:32:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.874+0000 7f552f786740 1 client.4773 shutdown 2024-04-25T08:32:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.875+0000 7f552f786740 1 -- 172.21.15.38:0/2024370234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 msgr2=0x557c29630500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.875+0000 7f552f786740 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 0x557c29630500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f551c012000 tx=0x7f551c000bc0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.875+0000 7f552f786740 1 -- 172.21.15.38:0/2024370234 shutdown_connections 2024-04-25T08:32:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.875+0000 7f552f786740 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x557c29606610 0x557c296069f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.875+0000 7f552f786740 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c296171a0 0x557c29608ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.875+0000 7f552f786740 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29622ad0 0x557c29630500 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.875+0000 7f55057fa640 0 client.4773 ms_handle_reset on v2:172.21.15.107:6834/1104921666 2024-04-25T08:32:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.875+0000 7f552f786740 1 --2- 172.21.15.38:0/2024370234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29605230 0x557c29601d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.875+0000 7f552f786740 1 -- 172.21.15.38:0/2024370234 >> 172.21.15.38:0/2024370234 conn(0x557c295daf10 msgr2=0x557c295d4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.875+0000 7f552f786740 1 -- 172.21.15.38:0/2024370234 shutdown_connections 2024-04-25T08:32:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.875+0000 7f552f786740 1 -- 172.21.15.38:0/2024370234 wait complete. 2024-04-25T08:32:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.875+0000 7f552f786740 20 client.4773 trim_cache size 0 max 16384 2024-04-25T08:32:01.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.891+0000 7f552f786740 1 Processor -- start 2024-04-25T08:32:01.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.891+0000 7f552f786740 1 -- start start 2024-04-25T08:32:01.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552f786740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 0x557c296206b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552f786740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c296171a0 0x557c295db2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552f786740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c29622ad0 0x557c295db830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552f786740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5528200480 con 0x557c29605230 2024-04-25T08:32:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552f786740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55281ff8b0 con 0x557c296171a0 2024-04-25T08:32:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552f786740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55281ffeb0 con 0x557c29622ad0 2024-04-25T08:32:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 0x557c296206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 0x557c296206b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56710/0 (socket says 172.21.15.38:56710) 2024-04-25T08:32:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552659c640 1 -- 172.21.15.38:0/842660051 learned_addr learned my addr 172.21.15.38:0/842660051 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552659c640 1 -- 172.21.15.38:0/842660051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c29622ad0 msgr2=0x557c295db830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552659c640 1 --2- 172.21.15.38:0/842660051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c29622ad0 0x557c295db830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552659c640 1 -- 172.21.15.38:0/842660051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c296171a0 msgr2=0x557c295db2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552659c640 1 --2- 172.21.15.38:0/842660051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c296171a0 0x557c295db2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552659c640 1 -- 172.21.15.38:0/842660051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c296046d0 con 0x557c29605230 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f552659c640 1 --2- 172.21.15.38:0/842660051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 0x557c296206b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x557c29603b70 tx=0x7f5518013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ce05f13299b73b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f55077fe640 1 -- 172.21.15.38:0/842660051 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551802e070 con 0x557c29605230 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f55077fe640 1 -- 172.21.15.38:0/842660051 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5518016c50 con 0x557c29605230 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.892+0000 7f55077fe640 1 -- 172.21.15.38:0/842660051 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518016e20 con 0x557c29605230 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.893+0000 7f552f786740 1 -- 172.21.15.38:0/842660051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 msgr2=0x557c296206b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.893+0000 7f552f786740 1 --2- 172.21.15.38:0/842660051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 0x557c296206b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x557c29603b70 tx=0x7f5518013000 comp rx=0 tx=0).stop 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.893+0000 7f552f786740 1 -- 172.21.15.38:0/842660051 shutdown_connections 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.893+0000 7f552f786740 1 --2- 172.21.15.38:0/842660051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c29622ad0 0x557c295db830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.893+0000 7f552f786740 1 --2- 172.21.15.38:0/842660051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c296171a0 0x557c295db2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.893+0000 7f552f786740 1 --2- 172.21.15.38:0/842660051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 0x557c296206b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.893+0000 7f552f786740 1 -- 172.21.15.38:0/842660051 >> 172.21.15.38:0/842660051 conn(0x557c295daf10 msgr2=0x557c295dc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.893+0000 7f552f786740 1 -- 172.21.15.38:0/842660051 shutdown_connections 2024-04-25T08:32:01.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.893+0000 7f552f786740 1 -- 172.21.15.38:0/842660051 wait complete. 2024-04-25T08:32:01.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.893+0000 7f552f786740 1 -- start start 2024-04-25T08:32:01.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.894+0000 7f552f786740 1 Processor -- start 2024-04-25T08:32:01.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.894+0000 7f552f786740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 0x557c2961f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.894+0000 7f552f786740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c296171a0 0x557c295da740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.894+0000 7f552f786740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29622ad0 0x557c2961f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.894+0000 7f552f786740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c29630c80 con 0x557c29605230 2024-04-25T08:32:01.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.894+0000 7f552f786740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c29608590 con 0x557c29622ad0 2024-04-25T08:32:01.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.894+0000 7f552f786740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c29603920 con 0x557c296171a0 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.894+0000 7f552f786740 10 client.0 _ll_register_callbacks cb 0x557c295d5c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f552659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 0x557c2961f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f552659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 0x557c2961f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56716/0 (socket says 172.21.15.38:56716) 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f552659c640 1 -- 172.21.15.38:0/3920290440 learned_addr learned my addr 172.21.15.38:0/3920290440 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f5525d9b640 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c296171a0 0x557c295da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f5526d9d640 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29622ad0 0x557c2961f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f552659c640 1 -- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c296171a0 msgr2=0x557c295da740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f552659c640 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c296171a0 0x557c295da740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f552659c640 1 -- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29622ad0 msgr2=0x557c2961f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f552659c640 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29622ad0 0x557c2961f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f552659c640 1 -- 172.21.15.38:0/3920290440 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c29623dc0 con 0x557c29605230 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f5525d9b640 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c296171a0 0x557c295da740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f5526d9d640 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29622ad0 0x557c2961f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f552659c640 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 0x557c2961f010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5518024040 tx=0x7f5518010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad6654d17d26baf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f55057fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:32:01.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f55057fa640 1 -- 172.21.15.38:0/3920290440 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518039060 con 0x557c29605230 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f552f786740 1 -- 172.21.15.38:0/3920290440 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5518016c50 con 0x557c29605230 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f55057fa640 1 -- 172.21.15.38:0/3920290440 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5518028250 con 0x557c29605230 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f55057fa640 1 -- 172.21.15.38:0/3920290440 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f551803f050 con 0x557c29605230 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f552f786740 10 client.4779 fetch_fsmap learned FSMap version 9 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f552f786740 1 -- 172.21.15.38:0/3920290440 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x557c29606f10 con 0x557c29605230 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.895+0000 7f55057fa640 1 -- 172.21.15.38:0/3920290440 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5518016c50 con 0x557c29605230 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f55057fa640 1 -- 172.21.15.38:0/3920290440 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f5518042020 con 0x557c29605230 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f552f786740 10 client.4779 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f552f786740 10 client.4779 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f552f786740 20 client.4779 populate_metadata read hostname 'smithi038' 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f552f786740 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x557c296395f0 0x557c296399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f552f786740 4 client.4779 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f552f786740 1 -- 172.21.15.38:0/3920290440 --> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557c2954af50 con 0x557c296395f0 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f5525d9b640 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x557c296395f0 0x557c296399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:01.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f55057fa640 10 client.4779 ms_handle_connect on v2:172.21.15.107:6834/1104921666 2024-04-25T08:32:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f5525d9b640 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x557c296395f0 0x557c296399d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdf4c22aa81ef178 server_cookie=b4dd6cde96ba1a87 in_seq=0 out_seq=0 2024-04-25T08:32:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f55057fa640 1 -- 172.21.15.38:0/3920290440 <== mds.0 v2:172.21.15.107:6834/1104921666 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x557c2954af50 con 0x557c296395f0 2024-04-25T08:32:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f55057fa640 10 client.4779 handle_command_reply: tid=0 2024-04-25T08:32:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f552f786740 1 client.4779 shutdown 2024-04-25T08:32:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.896+0000 7f552f786740 1 -- 172.21.15.38:0/3920290440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 msgr2=0x557c2961f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552f786740 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 0x557c2961f010 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5518024040 tx=0x7f5518010000 comp rx=0 tx=0).stop 2024-04-25T08:32:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552f786740 1 -- 172.21.15.38:0/3920290440 shutdown_connections 2024-04-25T08:32:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552f786740 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:6834/1104921666,v1:172.21.15.107:6836/1104921666] conn(0x557c296395f0 0x557c296399d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552f786740 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557c29622ad0 0x557c2961f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552f786740 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557c296171a0 0x557c295da740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552f786740 1 --2- 172.21.15.38:0/3920290440 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c29605230 0x557c2961f010 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f55057fa640 0 client.4779 ms_handle_reset on v2:172.21.15.107:6834/1104921666 2024-04-25T08:32:01.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552f786740 1 -- 172.21.15.38:0/3920290440 >> 172.21.15.38:0/3920290440 conn(0x557c295daf10 msgr2=0x557c295dc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552f786740 1 -- 172.21.15.38:0/3920290440 shutdown_connections 2024-04-25T08:32:01.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552f786740 1 -- 172.21.15.38:0/3920290440 wait complete. 2024-04-25T08:32:01.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552f786740 20 client.4779 trim_cache size 0 max 16384 2024-04-25T08:32:01.900 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:32:01.901 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:32:01.901 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:32:01.901 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4294, 2024-04-25T08:32:01.901 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:32:01.901 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:32:01.901 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:32:01.902 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 25.599339673999999, 2024-04-25T08:32:01.902 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 9, 2024-04-25T08:32:01.902 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 24, 2024-04-25T08:32:01.902 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T08:32:01.902 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 33.011327342000001 2024-04-25T08:32:01.902 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:32:01.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552c9a5640 1 -- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f550806b500 msgr2=0x7f550806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552c9a5640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f550806b500 0x7f550806d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f55180011e0 tx=0x7f5518009040 comp rx=0 tx=0).stop 2024-04-25T08:32:01.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552c9a5640 1 -- 172.21.15.38:0/150538067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55281609a0 msgr2=0x7f55280ce120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:01.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552c9a5640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55281609a0 0x7f55280ce120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5520009d90 tx=0x7f5520004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:01.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552c9a5640 1 -- 172.21.15.38:0/150538067 shutdown_connections 2024-04-25T08:32:01.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552c9a5640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55281609a0 0x7f55280ce120 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552c9a5640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55280cf7b0 0x7f55280c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552c9a5640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f550806b500 0x7f550806d9c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552c9a5640 1 --2- 172.21.15.38:0/150538067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5528099a70 0x7f5528152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:01.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552c9a5640 1 -- 172.21.15.38:0/150538067 >> 172.21.15.38:0/150538067 conn(0x7f55280b3190 msgr2=0x7f5528099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:01.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552c9a5640 1 -- 172.21.15.38:0/150538067 shutdown_connections 2024-04-25T08:32:01.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:01.897+0000 7f552c9a5640 1 -- 172.21.15.38:0/150538067 wait complete. 2024-04-25T08:32:01.907 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T08:32:01.924 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T08:32:01.924 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T08:32:01.925 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T08:32:01.925 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:32:01.925 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:01.925 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T08:32:01.925 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:32:01.926 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T08:32:01.926 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T08:32:01.926 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:32:01.926 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T08:32:01.926 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T08:32:01.927 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:32:01.927 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T08:32:01.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:01.927 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:32:01.943 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T08:32:01.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:01.943 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:32:02.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:02.008 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-04-25T08:32:02.103 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:32:02.105 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:32:02.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:02.114 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-25T08:32:02.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:02.165 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-04-25T08:32:02.235 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2024-04-25T08:32:02.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:32:02.237 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T08:32:02.237 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:32:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.340+0000 7f1d9a3e0640 1 Processor -- start 2024-04-25T08:32:02.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.340+0000 7f1d9a3e0640 1 -- start start 2024-04-25T08:32:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.341+0000 7f1d9a3e0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1d9415fbf0 0x7f1d940c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.341+0000 7f1d9a3e0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d94160340 con 0x7f1d9415bb10 2024-04-25T08:32:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.341+0000 7f1d9a3e0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d940cf400 con 0x7f1d9409ab90 2024-04-25T08:32:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.341+0000 7f1d9a3e0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d940cffd0 con 0x7f1d9415fbf0 2024-04-25T08:32:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.341+0000 7f1d93fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1d9415bb10 0x7f1d9409a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54870/0 (socket says 172.21.15.38:54870) 2024-04-25T08:32:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.341+0000 7f1d93fff640 1 -- 172.21.15.38:0/4269645187 learned_addr learned my addr 172.21.15.38:0/4269645187 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d9897d640 1 --2- 172.21.15.38:0/4269645187 >> v2:172.21.15.107:3300/0 conn(0x7f1d9415fbf0 0x7f1d940c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3630881449 0 0) 0x7f1d84001110 con 0x7f1d9415bb10 2024-04-25T08:32:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d80003520 con 0x7f1d9415bb10 2024-04-25T08:32:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1524938749 0 0) 0x7f1d80003520 con 0x7f1d9415bb10 2024-04-25T08:32:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 >> v2:172.21.15.107:3300/0 conn(0x7f1d9415fbf0 msgr2=0x7f1d940c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:02.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 --2- 172.21.15.38:0/4269645187 >> v2:172.21.15.107:3300/0 conn(0x7f1d9415fbf0 0x7f1d940c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 >> v1:172.21.15.107:6789/0 conn(0x7f1d9409ab90 legacy=0x7f1d9415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d940cf6a0 con 0x7f1d9415bb10 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f1d84001840 con 0x7f1d9415bb10 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 >> v1:172.21.15.38:6789/0 conn(0x7f1d9415bb10 legacy=0x7f1d9409a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 --2- 172.21.15.38:0/4269645187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d800039b0 0x7f1d80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 --2- 172.21.15.38:0/4269645187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d80004800 0x7f1d80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 --2- 172.21.15.38:0/4269645187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d800051c0 0x7f1d8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940cffd0 con 0x7f1d80004800 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8000c160 con 0x7f1d800039b0 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8000c360 con 0x7f1d800051c0 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d937fe640 1 --2- 172.21.15.38:0/4269645187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d80004800 0x7f1d80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d93fff640 1 --2- 172.21.15.38:0/4269645187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d800039b0 0x7f1d80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.342+0000 7f1d9897d640 1 --2- 172.21.15.38:0/4269645187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d800051c0 0x7f1d8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.343+0000 7f1d937fe640 1 -- 172.21.15.38:0/4269645187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d800051c0 msgr2=0x7f1d8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:02.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.343+0000 7f1d937fe640 1 --2- 172.21.15.38:0/4269645187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d800051c0 0x7f1d8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.343+0000 7f1d937fe640 1 -- 172.21.15.38:0/4269645187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d800039b0 msgr2=0x7f1d80003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:02.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.343+0000 7f1d937fe640 1 --2- 172.21.15.38:0/4269645187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d800039b0 0x7f1d80003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.343+0000 7f1d937fe640 1 -- 172.21.15.38:0/4269645187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d80003520 con 0x7f1d80004800 2024-04-25T08:32:02.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.343+0000 7f1d9897d640 1 -- 172.21.15.38:0/4269645187 reap_dead start 2024-04-25T08:32:02.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.343+0000 7f1d937fe640 1 --2- 172.21.15.38:0/4269645187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d80004800 0x7f1d80004c80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c001750 tx=0x7f1d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb63831416ef77b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:02.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.343+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c001370 con 0x7f1d80004800 2024-04-25T08:32:02.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.343+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c0052d0 con 0x7f1d80004800 2024-04-25T08:32:02.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.343+0000 7f1d92ffd640 1 -- 172.21.15.38:0/4269645187 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c02abc0 con 0x7f1d80004800 2024-04-25T08:32:02.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.344+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/4269645187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d80004800 msgr2=0x7f1d80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.344+0000 7f1d9a3e0640 1 --2- 172.21.15.38:0/4269645187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d80004800 0x7f1d80004c80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c001750 tx=0x7f1d80005830 comp rx=0 tx=0).stop 2024-04-25T08:32:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.344+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/4269645187 shutdown_connections 2024-04-25T08:32:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.344+0000 7f1d9a3e0640 1 --2- 172.21.15.38:0/4269645187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d80004800 0x7f1d80004c80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.344+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/4269645187 >> 172.21.15.38:0/4269645187 conn(0x7f1d940b31f0 msgr2=0x7f1d940b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.344+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/4269645187 shutdown_connections 2024-04-25T08:32:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.344+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/4269645187 wait complete. 2024-04-25T08:32:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.345+0000 7f1d9a3e0640 1 Processor -- start 2024-04-25T08:32:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.345+0000 7f1d9a3e0640 1 -- start start 2024-04-25T08:32:02.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.346+0000 7f1d9a3e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d80004800 0x7f1d941f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.346+0000 7f1d9a3e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d940b4240 0x7f1d940b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.346+0000 7f1d9a3e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d9406c9b0 0x7f1d9406ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.346+0000 7f1d9a3e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940d0460 con 0x7f1d9406c9b0 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.346+0000 7f1d9a3e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940cfc60 con 0x7f1d940b4240 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.346+0000 7f1d9a3e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940d05e0 con 0x7f1d80004800 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.346+0000 7f1d9897d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d9406c9b0 0x7f1d9406ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.346+0000 7f1d9897d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d9406c9b0 0x7f1d9406ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56744/0 (socket says 172.21.15.38:56744) 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.346+0000 7f1d9897d640 1 -- 172.21.15.38:0/1868202013 learned_addr learned my addr 172.21.15.38:0/1868202013 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.346+0000 7f1d93fff640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d80004800 0x7f1d941f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.346+0000 7f1d937fe640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d940b4240 0x7f1d940b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.347+0000 7f1d9897d640 1 -- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d80004800 msgr2=0x7f1d941f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.347+0000 7f1d9897d640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d80004800 0x7f1d941f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.347+0000 7f1d9897d640 1 -- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d940b4240 msgr2=0x7f1d940b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.347+0000 7f1d9897d640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d940b4240 0x7f1d940b46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.347+0000 7f1d9897d640 1 -- 172.21.15.38:0/1868202013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9415c140 con 0x7f1d9406c9b0 2024-04-25T08:32:02.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.347+0000 7f1d9897d640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d9406c9b0 0x7f1d9406ce30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1d94072910 tx=0x7f1d8c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:02.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.347+0000 7f1d90ff9640 1 -- 172.21.15.38:0/1868202013 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00fdb0 con 0x7f1d9406c9b0 2024-04-25T08:32:02.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.347+0000 7f1d90ff9640 1 -- 172.21.15.38:0/1868202013 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c015430 con 0x7f1d9406c9b0 2024-04-25T08:32:02.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.347+0000 7f1d90ff9640 1 -- 172.21.15.38:0/1868202013 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c015820 con 0x7f1d9406c9b0 2024-04-25T08:32:02.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.347+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/1868202013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9415c420 con 0x7f1d9406c9b0 2024-04-25T08:32:02.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.347+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/1868202013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9415c9e0 con 0x7f1d9406c9b0 2024-04-25T08:32:02.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.348+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/1868202013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d94068e80 con 0x7f1d9406c9b0 2024-04-25T08:32:02.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.348+0000 7f1d90ff9640 1 -- 172.21.15.38:0/1868202013 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d8c0159c0 con 0x7f1d9406c9b0 2024-04-25T08:32:02.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.349+0000 7f1d90ff9640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1d7406b4d0 0x7f1d7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:02.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.349+0000 7f1d90ff9640 1 -- 172.21.15.38:0/1868202013 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1d8c08b620 con 0x7f1d9406c9b0 2024-04-25T08:32:02.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.349+0000 7f1d93fff640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1d7406b4d0 0x7f1d7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:02.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.350+0000 7f1d93fff640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1d7406b4d0 0x7f1d7406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1d940692d0 tx=0x7f1d84007dc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:02.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.353+0000 7f1d90ff9640 1 -- 172.21.15.38:0/1868202013 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d8c019030 con 0x7f1d9406c9b0 2024-04-25T08:32:02.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.577+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/1868202013 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1d940cdbf0 con 0x7f1d9406c9b0 2024-04-25T08:32:02.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.578+0000 7f1d90ff9640 1 -- 172.21.15.38:0/1868202013 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f1d8c010040 con 0x7f1d9406c9b0 2024-04-25T08:32:02.576 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:32:02.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.580+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1d7406b4d0 msgr2=0x7f1d7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.580+0000 7f1d9a3e0640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1d7406b4d0 0x7f1d7406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1d940692d0 tx=0x7f1d84007dc0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.580+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/1868202013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d9406c9b0 msgr2=0x7f1d9406ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.580+0000 7f1d9a3e0640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d9406c9b0 0x7f1d9406ce30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1d94072910 tx=0x7f1d8c00f580 comp rx=0 tx=0).stop 2024-04-25T08:32:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.580+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/1868202013 shutdown_connections 2024-04-25T08:32:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.580+0000 7f1d9a3e0640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1d9406c9b0 0x7f1d9406ce30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.580+0000 7f1d9a3e0640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d940b4240 0x7f1d940b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.580+0000 7f1d9a3e0640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1d7406b4d0 0x7f1d7406d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.580+0000 7f1d9a3e0640 1 --2- 172.21.15.38:0/1868202013 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1d80004800 0x7f1d941f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.580+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/1868202013 >> 172.21.15.38:0/1868202013 conn(0x7f1d940b31f0 msgr2=0x7f1d940c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.580+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/1868202013 shutdown_connections 2024-04-25T08:32:02.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:02.580+0000 7f1d9a3e0640 1 -- 172.21.15.38:0/1868202013 wait complete. 2024-04-25T08:32:02.588 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:32:02.588 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:32:02.588 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:32:02.588 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:32:02.588 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:02.588 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:32:02.588 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:32:02.588 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:32:02.588 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:32:02.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:02.589 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:32:02.604 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:32:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:32:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:32:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:32:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:02.604 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:02.604 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42715sec preferred_lft 42715sec 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:02.605 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:32:02.606 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:32:02.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:02.606 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:32:02.606 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:32:02.606 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:32:02.606 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-04-25T08:32:02.606 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-04-25T08:32:02.606 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:32:02.606 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:32:02.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:02.701 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:32:02.724 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:32:02.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:02.726 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:32:02.784 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:32:02.784 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:32:02.784 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:32:02.785 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:32:02.785 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:32:02.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:02.786 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:32:02.786 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:32:02.786 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:32:02.786 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:32:02.786 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:32:02.786 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:32:02.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:02.870 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:32:02.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:02.885 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-04-25T08:32:02.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:02.940 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:32:02.940 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:32:02.940 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:02.940 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T08:32:02.940 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:32:03.016 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T08:32:03.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:03.016 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:32:03.016 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:32:03.016 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T08:32:03.017 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:32:03.017 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T08:32:03.017 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T08:32:03.017 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T08:32:03.017 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:32:03.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:03.127 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:32:03.127 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:32:03.127 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T08:32:03.128 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:32:03.128 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:32:03.163 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:32:03.163 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:32:03.163 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:03.220 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:32:03.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:32:03.220 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:03.275 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:32:03.339 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:32:03.396 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:32:03.396 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:32:03.396 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:32:03.396 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:32:03.396 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:32:03.396 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:32:03.396 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:32:03.396 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:32:03.397 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:32:03.397 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:32:03.397 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:32:03.397 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:32:03.397 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:32:03.397 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:32:03.397 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:32:03.397 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:32:03.397 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:32:03.397 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:32:03.397 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:32:03.397 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:32:03.398 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:32:03.398 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:32:03.398 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:32:03.398 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:32:03.398 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:32:03.398 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:32:03.398 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:32:03.398 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:32:03.398 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:32:03.398 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:32:03.399 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:32:03.399 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:03.399 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:03.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:03.399 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:32:03.451 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:32:03.451 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:32:03.493 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:32:03.518 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:32:03.574 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:32:03.574 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:32:03.574 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:32:03.574 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:32:03.574 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:32:03.574 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:32:03.574 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:32:03.574 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:32:03.574 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:32:03.574 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:32:03.574 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:32:03.574 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:32:03.575 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:32:03.576 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:32:03.576 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:32:03.576 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:32:03.576 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:03.576 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:03.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:03.576 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:32:03.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-04-25T08:32:03.578+0000 7f6e2cf2b480 -1 init, newargv = 0x5616be8b7340 newargc=15 2024-04-25T08:32:03.577 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[44914]: starting ceph client 2024-04-25T08:32:03.580 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[44914]: starting fuse 2024-04-25T08:32:03.592 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:32:03.592 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:32:03.592 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:32:03.592 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:32:03.592 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:32:03.592 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:32:03.592 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:32:03.592 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:03.592 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:32:03.592 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:32:03.592 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:32:03.592 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:03.593 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-04-25T08:32:03.594 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:32:03.711 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.44914.asok 2024-04-25T08:32:03.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:32:03.722 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44914.asok 2024-04-25T08:32:03.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:03.722 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44914.asok status 2024-04-25T08:32:03.858 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:32:03.859 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:32:03.859 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:32:03.859 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:32:03.859 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-04-25T08:32:03.859 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:32:03.859 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:32:03.859 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "44914", 2024-04-25T08:32:03.859 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:32:03.859 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:03.859 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:32:03.859 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:32:03.859 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4791, 2024-04-25T08:32:03.860 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:32:03.860 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:32:03.860 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:32:03.860 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 4791 2024-04-25T08:32:03.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:03.860 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:32:03.860 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:32:03.860 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:32:03.860 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1045110684 2024-04-25T08:32:03.860 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:03.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:03.860 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:32:03.861 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:32:03.861 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:32:03.861 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1045110684 2024-04-25T08:32:03.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:03.861 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.4791 192.168.0.1:0/1045110684", 2024-04-25T08:32:03.861 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/1045110684", 2024-04-25T08:32:03.861 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:32:03.861 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 9, 2024-04-25T08:32:03.862 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 24, 2024-04-25T08:32:03.862 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:32:03.862 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:32:03.862 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:32:03.862 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:32:03.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:32:03.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.969+0000 7fe5e9e93640 1 Processor -- start 2024-04-25T08:32:03.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.969+0000 7fe5e9e93640 1 -- start start 2024-04-25T08:32:03.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.970+0000 7fe5e9e93640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e40b4b50 0x7fe5e40ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.970+0000 7fe5e9e93640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e40b50c0 con 0x7fe5e4144760 2024-04-25T08:32:03.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.970+0000 7fe5e9e93640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5e40b53a0 con 0x7fe5e4144b70 2024-04-25T08:32:03.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.970+0000 7fe5e9e93640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415f380 con 0x7fe5e40b4b50 2024-04-25T08:32:03.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.970+0000 7fe5e37fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe5e4144760 0x7fe5e415faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54886/0 (socket says 172.21.15.38:54886) 2024-04-25T08:32:03.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.970+0000 7fe5e37fe640 1 -- 172.21.15.38:0/4094071042 learned_addr learned my addr 172.21.15.38:0/4094071042 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276273478 0 0) 0x7fe5e40b50c0 con 0x7fe5e4144760 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e2ffd640 1 --2- 172.21.15.38:0/4094071042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e40b4b50 0x7fe5e40ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5cc003520 con 0x7fe5e4144760 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2531379341 0 0) 0x7fe5cc003520 con 0x7fe5e4144760 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e40b4b50 msgr2=0x7fe5e40ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 --2- 172.21.15.38:0/4094071042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e40b4b50 0x7fe5e40ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 >> v1:172.21.15.107:6789/0 conn(0x7fe5e4144b70 legacy=0x7fe5e40b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e40b5680 con 0x7fe5e4144760 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fe5d4001670 con 0x7fe5e4144760 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 >> v1:172.21.15.38:6789/0 conn(0x7fe5e4144760 legacy=0x7fe5e415faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 --2- 172.21.15.38:0/4094071042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5cc0039e0 0x7fe5cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 --2- 172.21.15.38:0/4094071042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5cc004800 0x7fe5cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 --2- 172.21.15.38:0/4094071042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e40b4b50 0x7fe5cc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415f380 con 0x7fe5cc004800 2024-04-25T08:32:03.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc00baf0 con 0x7fe5cc0039e0 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc00bcf0 con 0x7fe5e40b4b50 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e2ffd640 1 --2- 172.21.15.38:0/4094071042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5cc004800 0x7fe5cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e37fe640 1 --2- 172.21.15.38:0/4094071042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5cc0039e0 0x7fe5cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e3fff640 1 --2- 172.21.15.38:0/4094071042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e40b4b50 0x7fe5cc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e2ffd640 1 -- 172.21.15.38:0/4094071042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e40b4b50 msgr2=0x7fe5cc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e2ffd640 1 --2- 172.21.15.38:0/4094071042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e40b4b50 0x7fe5cc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e2ffd640 1 -- 172.21.15.38:0/4094071042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5cc0039e0 msgr2=0x7fe5cc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e2ffd640 1 --2- 172.21.15.38:0/4094071042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5cc0039e0 0x7fe5cc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.971+0000 7fe5e2ffd640 1 -- 172.21.15.38:0/4094071042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc003520 con 0x7fe5cc004800 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.972+0000 7fe5e2ffd640 1 --2- 172.21.15.38:0/4094071042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5cc004800 0x7fe5cc004c80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009ac0 tx=0x7fe5cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69da37194349bbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.972+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8036070 con 0x7fe5cc004800 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.972+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d8031400 con 0x7fe5cc004800 2024-04-25T08:32:03.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.972+0000 7fe5e27fc640 1 -- 172.21.15.38:0/4094071042 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8009690 con 0x7fe5cc004800 2024-04-25T08:32:03.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.973+0000 7fe5e9e93640 1 -- 172.21.15.38:0/4094071042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5cc004800 msgr2=0x7fe5cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:03.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.973+0000 7fe5e9e93640 1 --2- 172.21.15.38:0/4094071042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5cc004800 0x7fe5cc004c80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009ac0 tx=0x7fe5cc005830 comp rx=0 tx=0).stop 2024-04-25T08:32:03.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.973+0000 7fe5e3fff640 1 -- 172.21.15.38:0/4094071042 reap_dead start 2024-04-25T08:32:03.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.973+0000 7fe5e9e93640 1 -- 172.21.15.38:0/4094071042 shutdown_connections 2024-04-25T08:32:03.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.973+0000 7fe5e9e93640 1 -- 172.21.15.38:0/4094071042 >> 172.21.15.38:0/4094071042 conn(0x7fe5e406b210 msgr2=0x7fe5e406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:03.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.973+0000 7fe5e9e93640 1 -- 172.21.15.38:0/4094071042 shutdown_connections 2024-04-25T08:32:03.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.973+0000 7fe5e9e93640 1 -- 172.21.15.38:0/4094071042 wait complete. 2024-04-25T08:32:03.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.974+0000 7fe5e9e93640 1 Processor -- start 2024-04-25T08:32:03.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.974+0000 7fe5e9e93640 1 -- start start 2024-04-25T08:32:03.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e9e93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5cc004800 0x7fe5e41f16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e9e93640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e415d0e0 0x7fe5e4150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e9e93640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5e415df60 0x7fe5e41356d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e9e93640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e4145280 con 0x7fe5cc004800 2024-04-25T08:32:03.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e9e93640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e415f660 con 0x7fe5e415df60 2024-04-25T08:32:03.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e9e93640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e4145400 con 0x7fe5e415d0e0 2024-04-25T08:32:03.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5cc004800 0x7fe5e41f16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5cc004800 0x7fe5e41f16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56768/0 (socket says 172.21.15.38:56768) 2024-04-25T08:32:03.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e37fe640 1 -- 172.21.15.38:0/3773867770 learned_addr learned my addr 172.21.15.38:0/3773867770 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e2ffd640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e415d0e0 0x7fe5e4150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e3fff640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5e415df60 0x7fe5e41356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e37fe640 1 -- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e415d0e0 msgr2=0x7fe5e4150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e37fe640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e415d0e0 0x7fe5e4150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e37fe640 1 -- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5e415df60 msgr2=0x7fe5e41356d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e37fe640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5e415df60 0x7fe5e41356d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e37fe640 1 -- 172.21.15.38:0/3773867770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e415e890 con 0x7fe5cc004800 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.975+0000 7fe5e3fff640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5e415df60 0x7fe5e41356d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.976+0000 7fe5e37fe640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5cc004800 0x7fe5e41f16a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40072c0 tx=0x7fe5d4007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.976+0000 7fe5cbfff640 1 -- 172.21.15.38:0/3773867770 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d401a070 con 0x7fe5cc004800 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.976+0000 7fe5cbfff640 1 -- 172.21.15.38:0/3773867770 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d400f960 con 0x7fe5cc004800 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.976+0000 7fe5e9e93640 1 -- 172.21.15.38:0/3773867770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e4139c20 con 0x7fe5cc004800 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.976+0000 7fe5cbfff640 1 -- 172.21.15.38:0/3773867770 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d400fd10 con 0x7fe5cc004800 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.976+0000 7fe5e9e93640 1 -- 172.21.15.38:0/3773867770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e4151430 con 0x7fe5cc004800 2024-04-25T08:32:03.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.977+0000 7fe5e9e93640 1 -- 172.21.15.38:0/3773867770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e4068e80 con 0x7fe5cc004800 2024-04-25T08:32:03.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.977+0000 7fe5cbfff640 1 -- 172.21.15.38:0/3773867770 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5d401d070 con 0x7fe5cc004800 2024-04-25T08:32:03.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.978+0000 7fe5cbfff640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe5d006b4d0 0x7fe5d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:03.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.978+0000 7fe5cbfff640 1 -- 172.21.15.38:0/3773867770 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe5d4089fd0 con 0x7fe5cc004800 2024-04-25T08:32:03.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.978+0000 7fe5e2ffd640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe5d006b4d0 0x7fe5d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:03.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.978+0000 7fe5e2ffd640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe5d006b4d0 0x7fe5d006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8033000 tx=0x7fe5d8008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:03.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:03.982+0000 7fe5cbfff640 1 -- 172.21.15.38:0/3773867770 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5d4058ba0 con 0x7fe5cc004800 2024-04-25T08:32:04.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.206+0000 7fe5e9e93640 1 -- 172.21.15.38:0/3773867770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe5e406df60 con 0x7fe5cc004800 2024-04-25T08:32:04.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.206+0000 7fe5cbfff640 1 -- 172.21.15.38:0/3773867770 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fe5d40584a0 con 0x7fe5cc004800 2024-04-25T08:32:04.204 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:32:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.208+0000 7fe5e9e93640 1 -- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe5d006b4d0 msgr2=0x7fe5d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.208+0000 7fe5e9e93640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe5d006b4d0 0x7fe5d006d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8033000 tx=0x7fe5d8008030 comp rx=0 tx=0).stop 2024-04-25T08:32:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.208+0000 7fe5e9e93640 1 -- 172.21.15.38:0/3773867770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5cc004800 msgr2=0x7fe5e41f16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.208+0000 7fe5e9e93640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5cc004800 0x7fe5e41f16a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40072c0 tx=0x7fe5d4007780 comp rx=0 tx=0).stop 2024-04-25T08:32:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.208+0000 7fe5e9e93640 1 -- 172.21.15.38:0/3773867770 shutdown_connections 2024-04-25T08:32:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.208+0000 7fe5e9e93640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5e415df60 0x7fe5e41356d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.208+0000 7fe5e9e93640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe5d006b4d0 0x7fe5d006d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.208+0000 7fe5e9e93640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5e415d0e0 0x7fe5e4150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.208+0000 7fe5e9e93640 1 --2- 172.21.15.38:0/3773867770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5cc004800 0x7fe5e41f16a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.208+0000 7fe5e9e93640 1 -- 172.21.15.38:0/3773867770 >> 172.21.15.38:0/3773867770 conn(0x7fe5e406b210 msgr2=0x7fe5e415faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:04.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.208+0000 7fe5e9e93640 1 -- 172.21.15.38:0/3773867770 shutdown_connections 2024-04-25T08:32:04.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:04.208+0000 7fe5e9e93640 1 -- 172.21.15.38:0/3773867770 wait complete. 2024-04-25T08:32:04.216 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:32:04.216 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:32:04.216 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:32:04.216 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T08:32:04.216 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T08:32:04.216 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:32:04.217 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:32:04.217 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:32:04.217 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T08:32:04.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:04.217 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:32:04.232 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:32:04.233 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:04.234 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:32:04.234 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:32:04.234 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42713sec preferred_lft 42713sec 2024-04-25T08:32:04.234 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:32:04.234 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.234 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:04.234 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:04.234 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:32:04.234 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:32:04.234 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ea:23:76:ab:b1:e9 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:04.234 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:32:04.234 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.235 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::440f:97ff:fe63:87b/64 scope link 2024-04-25T08:32:04.235 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.235 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:32:04.235 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ea:23:76:ab:b1:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:04.235 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::e823:76ff:feab:b1e9/64 scope link tentative 2024-04-25T08:32:04.235 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:04.235 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:32:04.296 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:32:04.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:04.297 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:32:04.355 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:32:04.355 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:32:04.355 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:32:04.355 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:32:04.355 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:32:04.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:04.355 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:32:04.355 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:32:04.355 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:32:04.356 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:32:04.356 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:32:04.356 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:32:04.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:04.441 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:32:04.456 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:32:04.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:04.457 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-04-25T08:32:04.512 INFO:teuthology.orchestra.run.smithi038.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T08:32:04.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:04.513 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:32:04.513 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:32:04.513 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:32:04.513 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T08:32:04.513 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:32:04.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:04.590 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T08:32:04.655 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:32:04.655 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:32:04.655 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:32:04.655 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.655 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:32:04.655 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.655 INFO:teuthology.orchestra.run.smithi038.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:32:04.655 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3e:e8:58:c0:dd:53 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:32:04.655 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:32:04.655 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.655 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::3ce8:58ff:fec0:dd53/64 scope link 2024-04-25T08:32:04.655 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:04.657 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T08:32:04.721 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:32:04.721 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:32:04.721 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:32:04.721 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.721 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:32:04.721 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.721 INFO:teuthology.orchestra.run.smithi038.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:32:04.721 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3e:e8:58:c0:dd:53 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:32:04.721 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:32:04.721 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.721 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::3ce8:58ff:fec0:dd53/64 scope link 2024-04-25T08:32:04.721 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:04.723 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T08:32:04.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:04.723 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:32:04.723 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:32:04.723 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T08:32:04.723 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T08:32:04.723 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T08:32:04.723 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T08:32:04.723 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T08:32:04.723 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:32:04.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:04.834 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:32:04.834 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:32:04.834 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.1 up 2024-04-25T08:32:04.834 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T08:32:04.834 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:32:04.870 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T08:32:04.870 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:32:04.870 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T08:32:04.926 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T08:32:04.926 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:32:04.927 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T08:32:04.981 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:32:05.047 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:32:05.103 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:32:05.103 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:32:05.103 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:32:05.103 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:32:05.103 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:32:05.103 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:32:05.103 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:32:05.104 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:32:05.105 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:32:05.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:05.106 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:32:05.158 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:32:05.158 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-04-25T08:32:05.159 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T08:32:05.201 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:32:05.225 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:32:05.282 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:32:05.282 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:32:05.282 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:32:05.282 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:32:05.282 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:32:05.283 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:32:05.284 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:32:05.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:05.285 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:32:05.285 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi038.stderr:2024-04-25T08:32:05.285+0000 7f0bbcd98480 -1 init, newargv = 0x5595bbc52d80 newargc=15 2024-04-25T08:32:05.285 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi038.stderr:ceph-fuse[45408]: starting ceph client 2024-04-25T08:32:05.287 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi038.stderr:ceph-fuse[45408]: starting fuse 2024-04-25T08:32:05.301 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:32:05.301 INFO:teuthology.orchestra.run.smithi038.stdout:44 2024-04-25T08:32:05.301 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:32:05.302 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.2")) 2024-04-25T08:32:05.303 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:32:05.421 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.2.45408.asok 2024-04-25T08:32:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:32:05.432 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.45408.asok 2024-04-25T08:32:05.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:05.432 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.45408.asok status 2024-04-25T08:32:05.568 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:32:05.568 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "2", 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "45408", 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 4806, 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:32:05.569 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 4806 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3977435981 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3977435981 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.4806 192.168.0.2:0/3977435981", 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.2:0/3977435981", 2024-04-25T08:32:05.570 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:32:05.571 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 9, 2024-04-25T08:32:05.571 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 24, 2024-04-25T08:32:05.571 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:32:05.571 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:32:05.571 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:32:05.571 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:32:05.577 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:32:05.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.680+0000 7fd1e41ed640 1 Processor -- start 2024-04-25T08:32:05.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.680+0000 7fd1e41ed640 1 -- start start 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e41ed640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd1dc0d0fe0 0x7fd1dc0b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e41ed640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd1dc0d2e50 0x7fd1dc0b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e41ed640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1dc0b8400 con 0x7fd1dc15dc40 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e41ed640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1dc0d19b0 con 0x7fd1dc0d0fe0 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e41ed640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1dc0b86e0 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e1788640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd1dc0d0fe0 0x7fd1dc0b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e1f89640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd1dc15dc40 0x7fd1dc0d08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54888/0 (socket says 172.21.15.38:54888) 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e1f89640 1 -- 172.21.15.38:0/2119213964 learned_addr learned my addr 172.21.15.38:0/2119213964 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e278a640 1 --2- 172.21.15.38:0/2119213964 >> v2:172.21.15.107:3300/0 conn(0x7fd1dc0d2e50 0x7fd1dc0b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e1788640 1 -- 172.21.15.38:0/2119213964 >> v2:172.21.15.107:3300/0 conn(0x7fd1dc0d2e50 msgr2=0x7fd1dc0b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e1788640 1 --2- 172.21.15.38:0/2119213964 >> v2:172.21.15.107:3300/0 conn(0x7fd1dc0d2e50 0x7fd1dc0b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e1788640 1 -- 172.21.15.38:0/2119213964 >> v1:172.21.15.38:6789/0 conn(0x7fd1dc15dc40 legacy=0x7fd1dc0d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e1788640 1 -- 172.21.15.38:0/2119213964 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc0b89c0 con 0x7fd1dc0d0fe0 2024-04-25T08:32:05.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e0f87640 1 -- 172.21.15.38:0/2119213964 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3945953352 0 0) 0x7fd1dc0b8400 con 0x7fd1dc15dc40 2024-04-25T08:32:05.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.681+0000 7fd1e278a640 1 --2- 172.21.15.38:0/2119213964 >> v2:172.21.15.107:3300/0 conn(0x7fd1dc0d2e50 0x7fd1dc0b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:05.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.682+0000 7fd1e1788640 1 --2- 172.21.15.38:0/2119213964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1dc0d0fe0 0x7fd1dc0b6a60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0c7250 tx=0x7fd1cc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a452e184695e71e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.682+0000 7fd1e0f87640 1 -- 172.21.15.38:0/2119213964 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc037070 con 0x7fd1dc0d0fe0 2024-04-25T08:32:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.682+0000 7fd1e0f87640 1 -- 172.21.15.38:0/2119213964 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc032440 con 0x7fd1dc0d0fe0 2024-04-25T08:32:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.682+0000 7fd1e0f87640 1 -- 172.21.15.38:0/2119213964 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc032810 con 0x7fd1dc0d0fe0 2024-04-25T08:32:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.682+0000 7fd1e41ed640 1 -- 172.21.15.38:0/2119213964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1dc0d0fe0 msgr2=0x7fd1dc0b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.682+0000 7fd1e41ed640 1 --2- 172.21.15.38:0/2119213964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1dc0d0fe0 0x7fd1dc0b6a60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0c7250 tx=0x7fd1cc02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.683+0000 7fd1e41ed640 1 -- 172.21.15.38:0/2119213964 shutdown_connections 2024-04-25T08:32:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.683+0000 7fd1e41ed640 1 --2- 172.21.15.38:0/2119213964 >> v2:172.21.15.107:3300/0 conn(0x7fd1dc0d2e50 0x7fd1dc0b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.683+0000 7fd1e41ed640 1 --2- 172.21.15.38:0/2119213964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1dc0d0fe0 0x7fd1dc0b6a60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.683+0000 7fd1e41ed640 1 -- 172.21.15.38:0/2119213964 >> 172.21.15.38:0/2119213964 conn(0x7fd1dc066ce0 msgr2=0x7fd1dc0670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.683+0000 7fd1e41ed640 1 -- 172.21.15.38:0/2119213964 shutdown_connections 2024-04-25T08:32:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.683+0000 7fd1e41ed640 1 -- 172.21.15.38:0/2119213964 wait complete. 2024-04-25T08:32:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.684+0000 7fd1e41ed640 1 Processor -- start 2024-04-25T08:32:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.684+0000 7fd1e41ed640 1 -- start start 2024-04-25T08:32:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.684+0000 7fd1e41ed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1dc0d0fe0 0x7fd1dc159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.684+0000 7fd1e41ed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1dc0d2e50 0x7fd1dc159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.684+0000 7fd1e41ed640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1dc15dc40 0x7fd1dc0ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e41ed640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0b9700 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e41ed640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0b9580 con 0x7fd1dc0d0fe0 2024-04-25T08:32:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e41ed640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0b9880 con 0x7fd1dc15dc40 2024-04-25T08:32:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e1788640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1dc0d2e50 0x7fd1dc159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e1788640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1dc0d2e50 0x7fd1dc159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56786/0 (socket says 172.21.15.38:56786) 2024-04-25T08:32:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e1788640 1 -- 172.21.15.38:0/1196627080 learned_addr learned my addr 172.21.15.38:0/1196627080 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e1f89640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1dc0d0fe0 0x7fd1dc159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e1788640 1 -- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1dc15dc40 msgr2=0x7fd1dc0ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e1788640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1dc15dc40 0x7fd1dc0ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e1788640 1 -- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1dc0d0fe0 msgr2=0x7fd1dc159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e1788640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1dc0d0fe0 0x7fd1dc159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e1788640 1 -- 172.21.15.38:0/1196627080 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc161010 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e1f89640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1dc0d0fe0 0x7fd1dc159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e1788640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1dc0d2e50 0x7fd1dc159470 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc000c00 tx=0x7fd1cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1d27fc640 1 -- 172.21.15.38:0/1196627080 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc037040 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1d27fc640 1 -- 172.21.15.38:0/1196627080 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc042070 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e41ed640 1 -- 172.21.15.38:0/1196627080 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc15a1d0 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1d27fc640 1 -- 172.21.15.38:0/1196627080 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc030a50 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.685+0000 7fd1e41ed640 1 -- 172.21.15.38:0/1196627080 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1dc0aa300 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.686+0000 7fd1e41ed640 1 -- 172.21.15.38:0/1196627080 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1dc068e80 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.687+0000 7fd1d27fc640 1 -- 172.21.15.38:0/1196627080 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1cc030bf0 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.687+0000 7fd1d27fc640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1c006b4d0 0x7fd1c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:05.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.687+0000 7fd1d27fc640 1 -- 172.21.15.38:0/1196627080 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd1cc0aaf40 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.687+0000 7fd1e1f89640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1c006b4d0 0x7fd1c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:05.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.688+0000 7fd1e1f89640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1c006b4d0 0x7fd1c006d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8000f60 tx=0x7fd1d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.691+0000 7fd1d27fc640 1 -- 172.21.15.38:0/1196627080 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1cc079c30 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.911+0000 7fd1e41ed640 1 -- 172.21.15.38:0/1196627080 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd1dc15c3e0 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.911+0000 7fd1d27fc640 1 -- 172.21.15.38:0/1196627080 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd1cc079530 con 0x7fd1dc0d2e50 2024-04-25T08:32:05.909 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:32:05.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.913+0000 7fd1e41ed640 1 -- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1c006b4d0 msgr2=0x7fd1c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.913+0000 7fd1e41ed640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1c006b4d0 0x7fd1c006d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8000f60 tx=0x7fd1d8009040 comp rx=0 tx=0).stop 2024-04-25T08:32:05.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.913+0000 7fd1e41ed640 1 -- 172.21.15.38:0/1196627080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1dc0d2e50 msgr2=0x7fd1dc159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:05.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.913+0000 7fd1e41ed640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1dc0d2e50 0x7fd1dc159470 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc000c00 tx=0x7fd1cc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.913+0000 7fd1e41ed640 1 -- 172.21.15.38:0/1196627080 shutdown_connections 2024-04-25T08:32:05.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.913+0000 7fd1e41ed640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1c006b4d0 0x7fd1c006d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.913+0000 7fd1e41ed640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1dc15dc40 0x7fd1dc0ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.913+0000 7fd1e41ed640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1dc0d2e50 0x7fd1dc159470 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.913+0000 7fd1e41ed640 1 --2- 172.21.15.38:0/1196627080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1dc0d0fe0 0x7fd1dc159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:05.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.913+0000 7fd1e41ed640 1 -- 172.21.15.38:0/1196627080 >> 172.21.15.38:0/1196627080 conn(0x7fd1dc066ce0 msgr2=0x7fd1dc0d40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:05.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.913+0000 7fd1e41ed640 1 -- 172.21.15.38:0/1196627080 shutdown_connections 2024-04-25T08:32:05.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:05.913+0000 7fd1e41ed640 1 -- 172.21.15.38:0/1196627080 wait complete. 2024-04-25T08:32:05.921 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:32:05.921 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:32:05.921 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-04-25T08:32:05.921 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T08:32:05.921 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T08:32:05.921 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:32:05.921 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:32:05.921 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:32:05.921 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T08:32:05.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:05.922 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-04-25T08:32:05.940 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:32:05.941 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:05.942 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T08:32:05.942 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42725sec preferred_lft 42725sec 2024-04-25T08:32:05.942 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::b49c:9f9e:133f:9f60/64 scope link noprefixroute 2024-04-25T08:32:05.942 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:05.942 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:32:05.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:05.942 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-04-25T08:32:05.942 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T08:32:05.942 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:32:05.942 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2024-04-25T08:32:05.942 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2024-04-25T08:32:05.942 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:32:05.942 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T08:32:06.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:06.088 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:32:06.111 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-04-25T08:32:06.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:06.113 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-04-25T08:32:06.192 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-04-25T08:32:06.192 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:32:06.192 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:32:06.192 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:32:06.192 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:32:06.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:06.193 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-04-25T08:32:06.193 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T08:32:06.193 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:32:06.193 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:32:06.193 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:32:06.193 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T08:32:06.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:06.420 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-25T08:32:06.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:06.435 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-04-25T08:32:06.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:06.490 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-04-25T08:32:06.490 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T08:32:06.490 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:32:06.490 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T08:32:06.490 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T08:32:06.566 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T08:32:06.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:06.566 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-04-25T08:32:06.566 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T08:32:06.566 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T08:32:06.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:32:06.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T08:32:06.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T08:32:06.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T08:32:06.567 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T08:32:06.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:06.686 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-04-25T08:32:06.686 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T08:32:06.687 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-04-25T08:32:06.687 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:32:06.687 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T08:32:06.720 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T08:32:06.720 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:32:06.721 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T08:32:06.774 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T08:32:06.774 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:32:06.774 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T08:32:06.828 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-04-25T08:32:06.894 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:32:06.949 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:32:06.950 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:32:06.951 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:32:06.952 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:32:06.952 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:32:06.952 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-04-25T08:32:06.952 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:32:06.952 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:32:06.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:06.952 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-04-25T08:32:07.004 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:32:07.005 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T08:32:07.048 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-04-25T08:32:07.071 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:32:07.126 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-04-25T08:32:07.126 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-04-25T08:32:07.126 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-04-25T08:32:07.127 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-04-25T08:32:07.128 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:32:07.129 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:32:07.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:07.129 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-04-25T08:32:07.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi107.stderr:2024-04-25T08:32:07.141+0000 7fbcc7f20480 -1 init, newargv = 0x5567fc30b200 newargc=15 2024-04-25T08:32:07.138 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi107.stderr:ceph-fuse[40918]: starting ceph client 2024-04-25T08:32:07.142 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi107.stderr:ceph-fuse[40918]: starting fuse 2024-04-25T08:32:07.152 INFO:teuthology.orchestra.run.smithi107.stdout:43 2024-04-25T08:32:07.152 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:32:07.152 DEBUG:teuthology.orchestra.run.smithi107:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:32:07.152 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> import re 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> import os 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> def _find_admin_socket(client_name): 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:32:07.153 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:32:07.154 DEBUG:teuthology.orchestra.run.smithi107:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:32:07.154 DEBUG:teuthology.orchestra.run.smithi107:> contents = proc_f.read() 2024-04-25T08:32:07.154 DEBUG:teuthology.orchestra.run.smithi107:> if mountpoint in contents: 2024-04-25T08:32:07.154 DEBUG:teuthology.orchestra.run.smithi107:> return f 2024-04-25T08:32:07.154 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:32:07.154 DEBUG:teuthology.orchestra.run.smithi107:> 2024-04-25T08:32:07.154 DEBUG:teuthology.orchestra.run.smithi107:> print(_find_admin_socket("client.1")) 2024-04-25T08:32:07.154 DEBUG:teuthology.orchestra.run.smithi107:> ' 2024-04-25T08:32:07.269 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.1.40918.asok 2024-04-25T08:32:07.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:32:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:32:07.279 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.40918.asok 2024-04-25T08:32:07.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:07.279 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.40918.asok status 2024-04-25T08:32:07.411 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T08:32:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2024-04-25T08:32:07.411 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "1", 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "40918", 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4714, 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 4714 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T08:32:07.412 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 428982566 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 428982566 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.4714 192.168.0.1:0/428982566", 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "192.168.0.1:0/428982566", 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 9, 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 24, 2024-04-25T08:32:07.413 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:32:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklisted": false, 2024-04-25T08:32:07.414 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2024-04-25T08:32:07.414 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T08:32:07.419 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:32:07.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.536+0000 7f7be2eac640 1 Processor -- start 2024-04-25T08:32:07.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.536+0000 7f7be2eac640 1 -- start start 2024-04-25T08:32:07.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.537+0000 7f7be2eac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bdc0d0f60 0x7f7bdc0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.537+0000 7f7be2eac640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bdc0d3e70 con 0x7f7bdc0d3900 2024-04-25T08:32:07.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.537+0000 7f7be2eac640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bdc0d4150 con 0x7f7bdc15dc10 2024-04-25T08:32:07.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.537+0000 7f7be2eac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc15dff0 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.537+0000 7f7bdbfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7bdc0d3900 0x7f7bdc0d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54894/0 (socket says 172.21.15.38:54894) 2024-04-25T08:32:07.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.537+0000 7f7bdbfff640 1 -- 172.21.15.38:0/1177935310 learned_addr learned my addr 172.21.15.38:0/1177935310 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:07.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.537+0000 7f7be1449640 1 --2- 172.21.15.38:0/1177935310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bdc0d0f60 0x7f7bdc0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.537+0000 7f7bdb7fe640 1 -- 172.21.15.38:0/1177935310 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995860398 0 0) 0x7f7bdc0d3e70 con 0x7f7bdc0d3900 2024-04-25T08:32:07.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.537+0000 7f7bdb7fe640 1 -- 172.21.15.38:0/1177935310 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bc4003520 con 0x7f7bdc0d3900 2024-04-25T08:32:07.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.538+0000 7f7bdb7fe640 1 -- 172.21.15.38:0/1177935310 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017703367 0 0) 0x7f7bdc0d4150 con 0x7f7bdc15dc10 2024-04-25T08:32:07.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.538+0000 7f7bdb7fe640 1 -- 172.21.15.38:0/1177935310 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bdc0d3e70 con 0x7f7bdc15dc10 2024-04-25T08:32:07.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.538+0000 7f7be1449640 1 -- 172.21.15.38:0/1177935310 >> v1:172.21.15.107:6789/0 conn(0x7f7bdc15dc10 legacy=0x7f7bdc0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.538+0000 7f7be1449640 1 -- 172.21.15.38:0/1177935310 >> v1:172.21.15.38:6789/0 conn(0x7f7bdc0d3900 legacy=0x7f7bdc0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.538+0000 7f7be1449640 1 -- 172.21.15.38:0/1177935310 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc0d4430 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.538+0000 7f7be1449640 1 --2- 172.21.15.38:0/1177935310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bdc0d0f60 0x7f7bdc0b2370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009dc0 tx=0x7f7bd402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79acddaea16dbc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:07.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.539+0000 7f7bdb7fe640 1 -- 172.21.15.38:0/1177935310 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd402ae60 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.539+0000 7f7bdb7fe640 1 -- 172.21.15.38:0/1177935310 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd4007740 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.539+0000 7f7bdb7fe640 1 -- 172.21.15.38:0/1177935310 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4007b10 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.539+0000 7f7be2eac640 1 -- 172.21.15.38:0/1177935310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bdc0d0f60 msgr2=0x7f7bdc0b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.539+0000 7f7be2eac640 1 --2- 172.21.15.38:0/1177935310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bdc0d0f60 0x7f7bdc0b2370 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7bd4009dc0 tx=0x7f7bd402a640 comp rx=0 tx=0).stop 2024-04-25T08:32:07.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.539+0000 7f7be2eac640 1 -- 172.21.15.38:0/1177935310 shutdown_connections 2024-04-25T08:32:07.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.539+0000 7f7be2eac640 1 --2- 172.21.15.38:0/1177935310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bdc0d0f60 0x7f7bdc0b2370 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.539+0000 7f7be2eac640 1 -- 172.21.15.38:0/1177935310 >> 172.21.15.38:0/1177935310 conn(0x7f7bdc066d10 msgr2=0x7f7bdc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:07.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.539+0000 7f7be2eac640 1 -- 172.21.15.38:0/1177935310 shutdown_connections 2024-04-25T08:32:07.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.539+0000 7f7be2eac640 1 -- 172.21.15.38:0/1177935310 wait complete. 2024-04-25T08:32:07.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.540+0000 7f7be2eac640 1 Processor -- start 2024-04-25T08:32:07.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.540+0000 7f7be2eac640 1 -- start start 2024-04-25T08:32:07.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.541+0000 7f7be2eac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bdc0d0f60 0x7f7bdc0cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.541+0000 7f7be2eac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bdc0d3900 0x7f7bdc0abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.541+0000 7f7be2eac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bdc15dc10 0x7f7bdc1526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.541+0000 7f7be2eac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc0d4710 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.541+0000 7f7be2eac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc0cef30 con 0x7f7bdc15dc10 2024-04-25T08:32:07.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.541+0000 7f7be2eac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc0b80f0 con 0x7f7bdc0d3900 2024-04-25T08:32:07.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7be0c48640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bdc0d0f60 0x7f7bdc0cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7be0c48640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bdc0d0f60 0x7f7bdc0cc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56788/0 (socket says 172.21.15.38:56788) 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7be0c48640 1 -- 172.21.15.38:0/4066072765 learned_addr learned my addr 172.21.15.38:0/4066072765 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7be1449640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bdc15dc10 0x7f7bdc1526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7bdbfff640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bdc0d3900 0x7f7bdc0abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7be0c48640 1 -- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bdc0d3900 msgr2=0x7f7bdc0abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7be0c48640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bdc0d3900 0x7f7bdc0abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7be0c48640 1 -- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bdc15dc10 msgr2=0x7f7bdc1526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7be0c48640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bdc15dc10 0x7f7bdc1526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7be0c48640 1 -- 172.21.15.38:0/4066072765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc0cb7a0 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7be0c48640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bdc0d0f60 0x7f7bdc0cc830 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00c430 tx=0x7f7bcc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7bd97fa640 1 -- 172.21.15.38:0/4066072765 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc017070 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7bd97fa640 1 -- 172.21.15.38:0/4066072765 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bcc0052f0 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7be2eac640 1 -- 172.21.15.38:0/4066072765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bdc0cba80 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7bd97fa640 1 -- 172.21.15.38:0/4066072765 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc003830 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.542+0000 7f7be2eac640 1 -- 172.21.15.38:0/4066072765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc0cc040 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.543+0000 7f7be2eac640 1 -- 172.21.15.38:0/4066072765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bdc068e80 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.544+0000 7f7bd97fa640 1 -- 172.21.15.38:0/4066072765 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7bcc01a070 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.544+0000 7f7bd97fa640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7bc806b4d0 0x7f7bc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:07.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.545+0000 7f7bd97fa640 1 -- 172.21.15.38:0/4066072765 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7bcc089850 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.545+0000 7f7bdbfff640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7bc806b4d0 0x7f7bc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:07.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.545+0000 7f7bdbfff640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7bc806b4d0 0x7f7bc806d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0b7d80 tx=0x7f7bd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:07.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.549+0000 7f7bd97fa640 1 -- 172.21.15.38:0/4066072765 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7bcc057d40 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.768+0000 7f7be2eac640 1 -- 172.21.15.38:0/4066072765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7bdc0b7940 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.768+0000 7f7bd97fa640 1 -- 172.21.15.38:0/4066072765 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f7bcc05bde0 con 0x7f7bdc0d0f60 2024-04-25T08:32:07.765 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:32:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.770+0000 7f7be2eac640 1 -- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7bc806b4d0 msgr2=0x7f7bc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.770+0000 7f7be2eac640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7bc806b4d0 0x7f7bc806d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0b7d80 tx=0x7f7bd0009040 comp rx=0 tx=0).stop 2024-04-25T08:32:07.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.770+0000 7f7be2eac640 1 -- 172.21.15.38:0/4066072765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bdc0d0f60 msgr2=0x7f7bdc0cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:07.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.770+0000 7f7be2eac640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bdc0d0f60 0x7f7bdc0cc830 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc00c430 tx=0x7f7bcc00c8f0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.770+0000 7f7be2eac640 1 -- 172.21.15.38:0/4066072765 shutdown_connections 2024-04-25T08:32:07.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.770+0000 7f7be2eac640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bdc15dc10 0x7f7bdc1526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.770+0000 7f7be2eac640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7bc806b4d0 0x7f7bc806d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.770+0000 7f7be2eac640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bdc0d3900 0x7f7bdc0abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.770+0000 7f7be2eac640 1 --2- 172.21.15.38:0/4066072765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bdc0d0f60 0x7f7bdc0cc830 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:07.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.770+0000 7f7be2eac640 1 -- 172.21.15.38:0/4066072765 >> 172.21.15.38:0/4066072765 conn(0x7f7bdc066d10 msgr2=0x7f7bdc15e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:07.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.770+0000 7f7be2eac640 1 -- 172.21.15.38:0/4066072765 shutdown_connections 2024-04-25T08:32:07.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:07.770+0000 7f7be2eac640 1 -- 172.21.15.38:0/4066072765 wait complete. 2024-04-25T08:32:07.777 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:32:07.778 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:32:07.778 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-04-25T08:32:07.778 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T08:32:07.778 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T08:32:07.778 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:32:07.778 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:32:07.779 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:32:07.779 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T08:32:07.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:07.779 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-04-25T08:32:07.795 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:32:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:32:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:32:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-25T08:32:07.795 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:07.795 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-04-25T08:32:07.796 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-04-25T08:32:07.796 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:07.796 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:32:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T08:32:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42723sec preferred_lft 42723sec 2024-04-25T08:32:07.796 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::b49c:9f9e:133f:9f60/64 scope link noprefixroute 2024-04-25T08:32:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:07.797 INFO:teuthology.orchestra.run.smithi107.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:32:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 3a:67:a4:bc:23:a5 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:32:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::cc98:27ff:fe98:4875/64 scope link tentative 2024-04-25T08:32:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:07.797 INFO:teuthology.orchestra.run.smithi107.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:32:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 3a:67:a4:bc:23:a5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:32:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::3867:a4ff:febc:23a5/64 scope link 2024-04-25T08:32:07.797 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:07.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:07.798 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:32:07.857 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-04-25T08:32:07.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:07.859 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-04-25T08:32:07.914 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-04-25T08:32:07.914 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:32:07.914 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:32:07.914 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:32:07.914 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:32:07.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:07.915 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-04-25T08:32:07.915 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T08:32:07.915 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:32:07.915 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:32:07.915 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:32:07.915 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T08:32:07.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:07.997 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-25T08:32:08.053 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T08:32:08.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:08.053 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-04-25T08:32:08.107 INFO:teuthology.orchestra.run.smithi107.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T08:32:08.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:08.107 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-04-25T08:32:08.107 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T08:32:08.108 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:32:08.108 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T08:32:08.108 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T08:32:08.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:08.182 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T08:32:08.245 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:32:08.246 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:32:08.246 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:32:08.246 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:08.246 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-25T08:32:08.246 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:08.246 INFO:teuthology.orchestra.run.smithi107.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:32:08.246 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 06:2a:71:ee:56:c5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:32:08.246 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:32:08.246 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:08.246 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::42a:71ff:feee:56c5/64 scope link 2024-04-25T08:32:08.247 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:08.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:08.248 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T08:32:08.311 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:32:08.311 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:32:08.311 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:32:08.312 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:08.312 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-04-25T08:32:08.312 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:08.312 INFO:teuthology.orchestra.run.smithi107.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:32:08.312 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 06:2a:71:ee:56:c5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:32:08.312 INFO:teuthology.orchestra.run.smithi107.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:32:08.312 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:08.312 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::42a:71ff:feee:56c5/64 scope link 2024-04-25T08:32:08.312 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:08.313 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T08:32:08.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:08.314 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-04-25T08:32:08.314 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T08:32:08.314 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T08:32:08.315 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T08:32:08.315 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T08:32:08.315 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T08:32:08.315 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T08:32:08.315 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T08:32:08.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:08.421 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec bash -c ' 2024-04-25T08:32:08.422 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-04-25T08:32:08.422 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.1 up 2024-04-25T08:32:08.422 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T08:32:08.422 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-04-25T08:32:08.456 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T08:32:08.456 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:32:08.456 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T08:32:08.510 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T08:32:08.511 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:32:08.511 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T08:32:08.565 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-04-25T08:32:08.629 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:32:08.684 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-04-25T08:32:08.684 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-04-25T08:32:08.684 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-04-25T08:32:08.684 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-04-25T08:32:08.684 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-04-25T08:32:08.684 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-04-25T08:32:08.684 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-04-25T08:32:08.684 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:32:08.685 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:32:08.686 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-04-25T08:32:08.686 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-04-25T08:32:08.686 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:32:08.686 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:32:08.686 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:32:08.686 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:32:08.686 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-04-25T08:32:08.686 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:32:08.686 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:32:08.686 INFO:teuthology.orchestra.run.smithi107.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T08:32:08.686 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:32:08.686 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:32:08.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:08.687 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-04-25T08:32:08.738 INFO:teuthology.orchestra.run.smithi107.stdout:43 2024-04-25T08:32:08.738 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-04-25T08:32:08.738 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T08:32:08.780 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-04-25T08:32:08.803 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:32:08.858 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi107.stderr:2024-04-25T08:32:08.861+0000 7f2c9a930480 -1 init, newargv = 0x55697137bfd0 newargc=15 2024-04-25T08:32:08.858 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi107.stderr:ceph-fuse[41366]: starting ceph client 2024-04-25T08:32:08.859 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-04-25T08:32:08.859 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-04-25T08:32:08.859 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-04-25T08:32:08.859 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-04-25T08:32:08.859 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-04-25T08:32:08.859 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-04-25T08:32:08.859 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-04-25T08:32:08.859 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-04-25T08:32:08.859 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-04-25T08:32:08.859 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-04-25T08:32:08.860 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-04-25T08:32:08.861 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:32:08.861 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:32:08.861 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:32:08.861 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:32:08.861 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-04-25T08:32:08.861 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:32:08.861 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:32:08.861 INFO:teuthology.orchestra.run.smithi107.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T08:32:08.861 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:32:08.861 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:32:08.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:08.861 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-04-25T08:32:08.862 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi107.stderr:ceph-fuse[41366]: starting fuse 2024-04-25T08:32:08.876 INFO:teuthology.orchestra.run.smithi107.stdout:43 2024-04-25T08:32:08.876 INFO:teuthology.orchestra.run.smithi107.stdout:44 2024-04-25T08:32:08.876 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-04-25T08:32:08.876 DEBUG:teuthology.orchestra.run.smithi107:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:32:08.876 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2024-04-25T08:32:08.876 DEBUG:teuthology.orchestra.run.smithi107:> import re 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> import os 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> def _find_admin_socket(client_name): 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:32:08.877 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:32:08.878 DEBUG:teuthology.orchestra.run.smithi107:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:32:08.878 DEBUG:teuthology.orchestra.run.smithi107:> contents = proc_f.read() 2024-04-25T08:32:08.878 DEBUG:teuthology.orchestra.run.smithi107:> if mountpoint in contents: 2024-04-25T08:32:08.878 DEBUG:teuthology.orchestra.run.smithi107:> return f 2024-04-25T08:32:08.878 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:32:08.878 DEBUG:teuthology.orchestra.run.smithi107:> 2024-04-25T08:32:08.878 DEBUG:teuthology.orchestra.run.smithi107:> print(_find_admin_socket("client.3")) 2024-04-25T08:32:08.878 DEBUG:teuthology.orchestra.run.smithi107:> ' 2024-04-25T08:32:08.992 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.3.41366.asok 2024-04-25T08:32:08.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-04-25T08:32:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:32:09.003 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.41366.asok 2024-04-25T08:32:09.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:09.003 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.41366.asok status 2024-04-25T08:32:09.134 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-04-25T08:32:09.135 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2024-04-25T08:32:09.135 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:32:09.135 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:32:09.135 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "3", 2024-04-25T08:32:09.135 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2024-04-25T08:32:09.135 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T08:32:09.135 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "41366", 2024-04-25T08:32:09.135 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/" 2024-04-25T08:32:09.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T08:32:09.135 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2024-04-25T08:32:09.135 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2024-04-25T08:32:09.135 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 4726, 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 4726 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2364240769 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:32:09.136 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2364240769 2024-04-25T08:32:09.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-04-25T08:32:09.137 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.4726 192.168.0.2:0/2364240769", 2024-04-25T08:32:09.137 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "192.168.0.2:0/2364240769", 2024-04-25T08:32:09.137 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 1, 2024-04-25T08:32:09.137 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 9, 2024-04-25T08:32:09.137 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 24, 2024-04-25T08:32:09.137 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:32:09.137 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklisted": false, 2024-04-25T08:32:09.137 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2024-04-25T08:32:09.137 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-04-25T08:32:09.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:09.143 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:32:09.143 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:09.162 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:32:09.162 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:09.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:09.163 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:09.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:09.233 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:32:09.233 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T08:32:09.292 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:32:09.292 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T08:32:09.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:09.293 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T08:32:09.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:09.362 DEBUG:teuthology.orchestra.run.smithi107:> stat --file-system '--printf=%T 2024-04-25T08:32:09.362 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T08:32:09.377 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2024-04-25T08:32:09.377 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T08:32:09.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:09.377 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T08:32:09.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:09.441 DEBUG:teuthology.orchestra.run.smithi107:> stat --file-system '--printf=%T 2024-04-25T08:32:09.441 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T08:32:09.496 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2024-04-25T08:32:09.496 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T08:32:09.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:09.497 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T08:32:09.560 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T08:32:09.578 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:09.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.696+0000 7f8bd9d22640 1 Processor -- start 2024-04-25T08:32:09.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.696+0000 7f8bd9d22640 1 -- start start 2024-04-25T08:32:09.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.696+0000 7f8bd9d22640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bd40d3900 0x7f8bd40d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.696+0000 7f8bd9d22640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd40d3e70 con 0x7f8bd40d0d90 2024-04-25T08:32:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.696+0000 7f8bd9d22640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd40d4150 con 0x7f8bd415dc10 2024-04-25T08:32:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.696+0000 7f8bd9d22640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd415dff0 con 0x7f8bd40d3900 2024-04-25T08:32:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.697+0000 7f8bd3fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f8bd40d0d90 0x7f8bd40b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53990/0 (socket says 172.21.15.38:53990) 2024-04-25T08:32:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.697+0000 7f8bd3fff640 1 -- 172.21.15.38:0/1896452440 learned_addr learned my addr 172.21.15.38:0/1896452440 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.697+0000 7f8bd2ffd640 1 --2- 172.21.15.38:0/1896452440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bd40d3900 0x7f8bd40d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.697+0000 7f8bd27fc640 1 -- 172.21.15.38:0/1896452440 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971182387 0 0) 0x7f8bd40d3e70 con 0x7f8bd40d0d90 2024-04-25T08:32:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.697+0000 7f8bd27fc640 1 -- 172.21.15.38:0/1896452440 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bc0003520 con 0x7f8bd40d0d90 2024-04-25T08:32:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.697+0000 7f8bd27fc640 1 -- 172.21.15.38:0/1896452440 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536666687 0 0) 0x7f8bd40d4150 con 0x7f8bd415dc10 2024-04-25T08:32:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.697+0000 7f8bd27fc640 1 -- 172.21.15.38:0/1896452440 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bd40d3e70 con 0x7f8bd415dc10 2024-04-25T08:32:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.697+0000 7f8bd2ffd640 1 -- 172.21.15.38:0/1896452440 >> v1:172.21.15.107:6789/0 conn(0x7f8bd415dc10 legacy=0x7f8bd40ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.697+0000 7f8bd2ffd640 1 -- 172.21.15.38:0/1896452440 >> v1:172.21.15.38:6789/0 conn(0x7f8bd40d0d90 legacy=0x7f8bd40b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.697+0000 7f8bd2ffd640 1 -- 172.21.15.38:0/1896452440 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd40d4430 con 0x7f8bd40d3900 2024-04-25T08:32:09.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.698+0000 7f8bd2ffd640 1 --2- 172.21.15.38:0/1896452440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bd40d3900 0x7f8bd40d0850 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0097d0 tx=0x7f8bbc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8147469dd7bf522d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:09.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.698+0000 7f8bd27fc640 1 -- 172.21.15.38:0/1896452440 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc037070 con 0x7f8bd40d3900 2024-04-25T08:32:09.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.698+0000 7f8bd27fc640 1 -- 172.21.15.38:0/1896452440 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc032440 con 0x7f8bd40d3900 2024-04-25T08:32:09.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.698+0000 7f8bd27fc640 1 -- 172.21.15.38:0/1896452440 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0327f0 con 0x7f8bd40d3900 2024-04-25T08:32:09.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.698+0000 7f8bd9d22640 1 -- 172.21.15.38:0/1896452440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bd40d3900 msgr2=0x7f8bd40d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.698+0000 7f8bd9d22640 1 --2- 172.21.15.38:0/1896452440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bd40d3900 0x7f8bd40d0850 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0097d0 tx=0x7f8bbc02a180 comp rx=0 tx=0).stop 2024-04-25T08:32:09.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.699+0000 7f8bd9d22640 1 -- 172.21.15.38:0/1896452440 shutdown_connections 2024-04-25T08:32:09.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.699+0000 7f8bd9d22640 1 --2- 172.21.15.38:0/1896452440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bd40d3900 0x7f8bd40d0850 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.699+0000 7f8bd9d22640 1 -- 172.21.15.38:0/1896452440 >> 172.21.15.38:0/1896452440 conn(0x7f8bd4066d10 msgr2=0x7f8bd4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:09.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.699+0000 7f8bd9d22640 1 -- 172.21.15.38:0/1896452440 shutdown_connections 2024-04-25T08:32:09.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.699+0000 7f8bd9d22640 1 -- 172.21.15.38:0/1896452440 wait complete. 2024-04-25T08:32:09.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.700+0000 7f8bd9d22640 1 Processor -- start 2024-04-25T08:32:09.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.700+0000 7f8bd9d22640 1 -- start start 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd9d22640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bd40d0d90 0x7f8bd406c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd9d22640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bd40d3900 0x7f8bd41527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd9d22640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd415dc10 0x7f8bd4152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd9d22640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40d4710 con 0x7f8bd415dc10 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd9d22640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40cef30 con 0x7f8bd40d0d90 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd9d22640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd40b80f0 con 0x7f8bd40d3900 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd415dc10 0x7f8bd4152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd415dc10 0x7f8bd4152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37402/0 (socket says 172.21.15.38:37402) 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd3fff640 1 -- 172.21.15.38:0/4045056650 learned_addr learned my addr 172.21.15.38:0/4045056650 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd37fe640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bd40d0d90 0x7f8bd406c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd2ffd640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bd40d3900 0x7f8bd41527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd3fff640 1 -- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bd40d3900 msgr2=0x7f8bd41527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd3fff640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bd40d3900 0x7f8bd41527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd3fff640 1 -- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bd40d0d90 msgr2=0x7f8bd406c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd3fff640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bd40d0d90 0x7f8bd406c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd3fff640 1 -- 172.21.15.38:0/4045056650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd40c9600 con 0x7f8bd415dc10 2024-04-25T08:32:09.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.701+0000 7f8bd3fff640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd415dc10 0x7f8bd4152d30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00cbd0 tx=0x7f8bcc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:09.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.702+0000 7f8bcbfff640 1 -- 172.21.15.38:0/4045056650 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc017070 con 0x7f8bd415dc10 2024-04-25T08:32:09.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.702+0000 7f8bcbfff640 1 -- 172.21.15.38:0/4045056650 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc00ace0 con 0x7f8bd415dc10 2024-04-25T08:32:09.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.702+0000 7f8bd9d22640 1 -- 172.21.15.38:0/4045056650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd40c98c0 con 0x7f8bd415dc10 2024-04-25T08:32:09.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.702+0000 7f8bcbfff640 1 -- 172.21.15.38:0/4045056650 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc012620 con 0x7f8bd415dc10 2024-04-25T08:32:09.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.702+0000 7f8bd9d22640 1 -- 172.21.15.38:0/4045056650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd40c9e80 con 0x7f8bd415dc10 2024-04-25T08:32:09.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.703+0000 7f8bd9d22640 1 -- 172.21.15.38:0/4045056650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd4068e80 con 0x7f8bd415dc10 2024-04-25T08:32:09.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.703+0000 7f8bcbfff640 1 -- 172.21.15.38:0/4045056650 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bcc01a070 con 0x7f8bd415dc10 2024-04-25T08:32:09.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.704+0000 7f8bcbfff640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8bb406b4d0 0x7f8bb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:09.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.704+0000 7f8bcbfff640 1 -- 172.21.15.38:0/4045056650 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8bcc08a4f0 con 0x7f8bd415dc10 2024-04-25T08:32:09.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.704+0000 7f8bd37fe640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8bb406b4d0 0x7f8bb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:09.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.704+0000 7f8bd37fe640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8bb406b4d0 0x7f8bb406d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4007fd0 tx=0x7f8bc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:09.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.708+0000 7f8bcbfff640 1 -- 172.21.15.38:0/4045056650 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8bcc0589f0 con 0x7f8bd415dc10 2024-04-25T08:32:09.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.889+0000 7f8bd9d22640 1 -- 172.21.15.38:0/4045056650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bd41606e0 con 0x7f8bd415dc10 2024-04-25T08:32:09.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.890+0000 7f8bcbfff640 1 -- 172.21.15.38:0/4045056650 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f8bcc05ca90 con 0x7f8bd415dc10 2024-04-25T08:32:09.886 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:09.886 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:09.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.892+0000 7f8bd9d22640 1 -- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8bb406b4d0 msgr2=0x7f8bb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.892+0000 7f8bd9d22640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8bb406b4d0 0x7f8bb406d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4007fd0 tx=0x7f8bc4009040 comp rx=0 tx=0).stop 2024-04-25T08:32:09.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.892+0000 7f8bd9d22640 1 -- 172.21.15.38:0/4045056650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd415dc10 msgr2=0x7f8bd4152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:09.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.892+0000 7f8bd9d22640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd415dc10 0x7f8bd4152d30 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00cbd0 tx=0x7f8bcc00a430 comp rx=0 tx=0).stop 2024-04-25T08:32:09.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.892+0000 7f8bd9d22640 1 -- 172.21.15.38:0/4045056650 shutdown_connections 2024-04-25T08:32:09.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.892+0000 7f8bd9d22640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bd415dc10 0x7f8bd4152d30 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.892+0000 7f8bd9d22640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8bb406b4d0 0x7f8bb406d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.892+0000 7f8bd9d22640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bd40d3900 0x7f8bd41527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.892+0000 7f8bd9d22640 1 --2- 172.21.15.38:0/4045056650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bd40d0d90 0x7f8bd406c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:09.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.892+0000 7f8bd9d22640 1 -- 172.21.15.38:0/4045056650 >> 172.21.15.38:0/4045056650 conn(0x7f8bd4066d10 msgr2=0x7f8bd40c6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:09.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.892+0000 7f8bd9d22640 1 -- 172.21.15.38:0/4045056650 shutdown_connections 2024-04-25T08:32:09.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:09.892+0000 7f8bd9d22640 1 -- 172.21.15.38:0/4045056650 wait complete. 2024-04-25T08:32:09.899 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:09.899 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.007+0000 7f7a566b3640 1 Processor -- start 2024-04-25T08:32:10.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.007+0000 7f7a566b3640 1 -- start start 2024-04-25T08:32:10.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a566b3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7a500d0320 0x7f7a500d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a566b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a500d0700 0x7f7a500d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a566b3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a500d2330 con 0x7f7a500d1e90 2024-04-25T08:32:10.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a566b3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a500d2610 con 0x7f7a500d0320 2024-04-25T08:32:10.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a566b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500d28f0 con 0x7f7a500d0700 2024-04-25T08:32:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a4ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7a500d0320 0x7f7a500d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a4ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7a500d0320 0x7f7a500d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37410/0 (socket says 172.21.15.38:37410) 2024-04-25T08:32:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a4ffff640 1 -- 172.21.15.38:0/1108511152 learned_addr learned my addr 172.21.15.38:0/1108511152 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a4f7fe640 1 --2- 172.21.15.38:0/1108511152 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a500d0700 0x7f7a500d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a4ffff640 1 -- 172.21.15.38:0/1108511152 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a500d0700 msgr2=0x7f7a500d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a4ffff640 1 --2- 172.21.15.38:0/1108511152 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a500d0700 0x7f7a500d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a4ffff640 1 -- 172.21.15.38:0/1108511152 >> v1:172.21.15.107:6789/0 conn(0x7f7a500d1e90 legacy=0x7f7a500b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.008+0000 7f7a4ffff640 1 -- 172.21.15.38:0/1108511152 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a500d2bd0 con 0x7f7a500d0320 2024-04-25T08:32:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.009+0000 7f7a4ffff640 1 --2- 172.21.15.38:0/1108511152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a500d0320 0x7f7a500d33d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009870 tx=0x7f7a3802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2971b22f2bb036 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.009+0000 7f7a4effd640 1 -- 172.21.15.38:0/1108511152 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38037070 con 0x7f7a500d0320 2024-04-25T08:32:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.009+0000 7f7a4effd640 1 -- 172.21.15.38:0/1108511152 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3802ae50 con 0x7f7a500d0320 2024-04-25T08:32:10.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.009+0000 7f7a4effd640 1 -- 172.21.15.38:0/1108511152 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38032860 con 0x7f7a500d0320 2024-04-25T08:32:10.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.009+0000 7f7a566b3640 1 -- 172.21.15.38:0/1108511152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a500d0320 msgr2=0x7f7a500d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.009+0000 7f7a566b3640 1 --2- 172.21.15.38:0/1108511152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a500d0320 0x7f7a500d33d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7a38009870 tx=0x7f7a3802a300 comp rx=0 tx=0).stop 2024-04-25T08:32:10.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.010+0000 7f7a566b3640 1 -- 172.21.15.38:0/1108511152 shutdown_connections 2024-04-25T08:32:10.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.010+0000 7f7a566b3640 1 --2- 172.21.15.38:0/1108511152 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a500d0700 0x7f7a500d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.010+0000 7f7a566b3640 1 --2- 172.21.15.38:0/1108511152 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a500d0320 0x7f7a500d33d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.010+0000 7f7a566b3640 1 -- 172.21.15.38:0/1108511152 >> 172.21.15.38:0/1108511152 conn(0x7f7a500668f0 msgr2=0x7f7a50066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:10.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.010+0000 7f7a566b3640 1 -- 172.21.15.38:0/1108511152 shutdown_connections 2024-04-25T08:32:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.010+0000 7f7a566b3640 1 -- 172.21.15.38:0/1108511152 wait complete. 2024-04-25T08:32:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.011+0000 7f7a566b3640 1 Processor -- start 2024-04-25T08:32:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.011+0000 7f7a566b3640 1 -- start start 2024-04-25T08:32:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a566b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a500d0320 0x7f7a50155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a566b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a500d0700 0x7f7a50159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a566b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500d1e90 0x7f7a500ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a566b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500d0b40 con 0x7f7a500d0700 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a566b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50161210 con 0x7f7a500d1e90 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a566b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5015a480 con 0x7f7a500d0320 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a500d0700 0x7f7a50159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a500d0700 0x7f7a50159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37416/0 (socket says 172.21.15.38:37416) 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4f7fe640 1 -- 172.21.15.38:0/3404347726 learned_addr learned my addr 172.21.15.38:0/3404347726 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4ffff640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a500d0320 0x7f7a50155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a54c50640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500d1e90 0x7f7a500ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4f7fe640 1 -- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a500d0320 msgr2=0x7f7a50155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4f7fe640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a500d0320 0x7f7a50155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4f7fe640 1 -- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500d1e90 msgr2=0x7f7a500ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4f7fe640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500d1e90 0x7f7a500ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4f7fe640 1 -- 172.21.15.38:0/3404347726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a50155910 con 0x7f7a500d0700 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4f7fe640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a500d0700 0x7f7a50159f40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000e9e0 tx=0x7f7a4000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:10.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4cff9640 1 -- 172.21.15.38:0/3404347726 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40017070 con 0x7f7a500d0700 2024-04-25T08:32:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4cff9640 1 -- 172.21.15.38:0/3404347726 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4000c970 con 0x7f7a500d0700 2024-04-25T08:32:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a566b3640 1 -- 172.21.15.38:0/3404347726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a50155bf0 con 0x7f7a500d0700 2024-04-25T08:32:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.012+0000 7f7a4cff9640 1 -- 172.21.15.38:0/3404347726 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4000cd60 con 0x7f7a500d0700 2024-04-25T08:32:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.013+0000 7f7a566b3640 1 -- 172.21.15.38:0/3404347726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a501561b0 con 0x7f7a500d0700 2024-04-25T08:32:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.014+0000 7f7a566b3640 1 -- 172.21.15.38:0/3404347726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a50068e80 con 0x7f7a500d0700 2024-04-25T08:32:10.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.014+0000 7f7a4cff9640 1 -- 172.21.15.38:0/3404347726 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a4001a070 con 0x7f7a500d0700 2024-04-25T08:32:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.014+0000 7f7a4cff9640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7a3006b4d0 0x7f7a3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.014+0000 7f7a4cff9640 1 -- 172.21.15.38:0/3404347726 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7a4008a7f0 con 0x7f7a500d0700 2024-04-25T08:32:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.014+0000 7f7a4ffff640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7a3006b4d0 0x7f7a3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.015+0000 7f7a4ffff640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7a3006b4d0 0x7f7a3006d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7a38034000 tx=0x7f7a3802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:10.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.018+0000 7f7a4cff9640 1 -- 172.21.15.38:0/3404347726 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a40058cf0 con 0x7f7a500d0700 2024-04-25T08:32:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.197+0000 7f7a566b3640 1 -- 172.21.15.38:0/3404347726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7a500b2550 con 0x7f7a500d0700 2024-04-25T08:32:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.197+0000 7f7a4cff9640 1 -- 172.21.15.38:0/3404347726 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a4005cd90 con 0x7f7a500d0700 2024-04-25T08:32:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:10.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.199+0000 7f7a566b3640 1 -- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7a3006b4d0 msgr2=0x7f7a3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.199+0000 7f7a566b3640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7a3006b4d0 0x7f7a3006d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7a38034000 tx=0x7f7a3802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:10.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.199+0000 7f7a566b3640 1 -- 172.21.15.38:0/3404347726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a500d0700 msgr2=0x7f7a50159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.199+0000 7f7a566b3640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a500d0700 0x7f7a50159f40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000e9e0 tx=0x7f7a4000eea0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.199+0000 7f7a566b3640 1 -- 172.21.15.38:0/3404347726 shutdown_connections 2024-04-25T08:32:10.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.199+0000 7f7a566b3640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a500d1e90 0x7f7a500ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.199+0000 7f7a566b3640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a500d0700 0x7f7a50159f40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.199+0000 7f7a566b3640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7a3006b4d0 0x7f7a3006d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.199+0000 7f7a566b3640 1 --2- 172.21.15.38:0/3404347726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a500d0320 0x7f7a50155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.199+0000 7f7a566b3640 1 -- 172.21.15.38:0/3404347726 >> 172.21.15.38:0/3404347726 conn(0x7f7a500668f0 msgr2=0x7f7a501602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:10.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.199+0000 7f7a566b3640 1 -- 172.21.15.38:0/3404347726 shutdown_connections 2024-04-25T08:32:10.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.199+0000 7f7a566b3640 1 -- 172.21.15.38:0/3404347726 wait complete. 2024-04-25T08:32:10.206 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:10.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.314+0000 7fe34806f640 1 Processor -- start 2024-04-25T08:32:10.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.314+0000 7fe34806f640 1 -- start start 2024-04-25T08:32:10.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.315+0000 7fe34806f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3400d33d0 0x7fe3400d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.315+0000 7fe34806f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3400d2330 con 0x7fe3400d0320 2024-04-25T08:32:10.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.315+0000 7fe34806f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3400d2610 con 0x7fe3400d1e90 2024-04-25T08:32:10.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.315+0000 7fe34806f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400d28f0 con 0x7fe3400d33d0 2024-04-25T08:32:10.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.315+0000 7fe345e0b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe3400d0320 0x7fe3400d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53994/0 (socket says 172.21.15.38:53994) 2024-04-25T08:32:10.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.315+0000 7fe345e0b640 1 -- 172.21.15.38:0/2876339578 learned_addr learned my addr 172.21.15.38:0/2876339578 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:10.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.315+0000 7fe34560a640 1 --2- 172.21.15.38:0/2876339578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3400d33d0 0x7fe3400d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.316+0000 7fe344e09640 1 -- 172.21.15.38:0/2876339578 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023238744 0 0) 0x7fe3400d2330 con 0x7fe3400d0320 2024-04-25T08:32:10.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.316+0000 7fe344e09640 1 -- 172.21.15.38:0/2876339578 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe32c003520 con 0x7fe3400d0320 2024-04-25T08:32:10.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.316+0000 7fe344e09640 1 -- 172.21.15.38:0/2876339578 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497678377 0 0) 0x7fe3400d2610 con 0x7fe3400d1e90 2024-04-25T08:32:10.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.316+0000 7fe344e09640 1 -- 172.21.15.38:0/2876339578 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3400d2330 con 0x7fe3400d1e90 2024-04-25T08:32:10.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.316+0000 7fe34560a640 1 -- 172.21.15.38:0/2876339578 >> v1:172.21.15.107:6789/0 conn(0x7fe3400d1e90 legacy=0x7fe3400b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.316+0000 7fe34560a640 1 -- 172.21.15.38:0/2876339578 >> v1:172.21.15.38:6789/0 conn(0x7fe3400d0320 legacy=0x7fe3400d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.316+0000 7fe34560a640 1 -- 172.21.15.38:0/2876339578 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3400d2bd0 con 0x7fe3400d33d0 2024-04-25T08:32:10.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.316+0000 7fe34560a640 1 --2- 172.21.15.38:0/2876339578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3400d33d0 0x7fe3400d1950 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3300097d0 tx=0x7fe33002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff710300ab383972 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:10.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.317+0000 7fe344e09640 1 -- 172.21.15.38:0/2876339578 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe330037070 con 0x7fe3400d33d0 2024-04-25T08:32:10.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.317+0000 7fe344e09640 1 -- 172.21.15.38:0/2876339578 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe330032440 con 0x7fe3400d33d0 2024-04-25T08:32:10.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.317+0000 7fe344e09640 1 -- 172.21.15.38:0/2876339578 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3300327f0 con 0x7fe3400d33d0 2024-04-25T08:32:10.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.317+0000 7fe34806f640 1 -- 172.21.15.38:0/2876339578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3400d33d0 msgr2=0x7fe3400d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.317+0000 7fe34806f640 1 --2- 172.21.15.38:0/2876339578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3400d33d0 0x7fe3400d1950 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe3300097d0 tx=0x7fe33002a290 comp rx=0 tx=0).stop 2024-04-25T08:32:10.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.317+0000 7fe34806f640 1 -- 172.21.15.38:0/2876339578 shutdown_connections 2024-04-25T08:32:10.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.317+0000 7fe34806f640 1 --2- 172.21.15.38:0/2876339578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3400d33d0 0x7fe3400d1950 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.317+0000 7fe34806f640 1 -- 172.21.15.38:0/2876339578 >> 172.21.15.38:0/2876339578 conn(0x7fe3400668f0 msgr2=0x7fe340066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:10.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.317+0000 7fe34806f640 1 -- 172.21.15.38:0/2876339578 shutdown_connections 2024-04-25T08:32:10.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.317+0000 7fe34806f640 1 -- 172.21.15.38:0/2876339578 wait complete. 2024-04-25T08:32:10.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.318+0000 7fe34806f640 1 Processor -- start 2024-04-25T08:32:10.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.318+0000 7fe34806f640 1 -- start start 2024-04-25T08:32:10.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.319+0000 7fe34806f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3400d0320 0x7fe3400b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.319+0000 7fe34806f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3400d1e90 0x7fe3400b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.319+0000 7fe34806f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3400d33d0 0x7fe3401518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.319+0000 7fe34806f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe340161390 con 0x7fe3400d0320 2024-04-25T08:32:10.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.319+0000 7fe34806f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe340161210 con 0x7fe3400d33d0 2024-04-25T08:32:10.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.319+0000 7fe34806f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400b9720 con 0x7fe3400d1e90 2024-04-25T08:32:10.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.319+0000 7fe345e0b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3400d0320 0x7fe3400b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.319+0000 7fe345e0b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3400d0320 0x7fe3400b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37420/0 (socket says 172.21.15.38:37420) 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.319+0000 7fe345e0b640 1 -- 172.21.15.38:0/569850609 learned_addr learned my addr 172.21.15.38:0/569850609 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.319+0000 7fe34560a640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3400d1e90 0x7fe3400b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.319+0000 7fe34660c640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3400d33d0 0x7fe3401518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.320+0000 7fe345e0b640 1 -- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3400d1e90 msgr2=0x7fe3400b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.320+0000 7fe345e0b640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3400d1e90 0x7fe3400b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.320+0000 7fe345e0b640 1 -- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3400d33d0 msgr2=0x7fe3401518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.320+0000 7fe345e0b640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3400d33d0 0x7fe3401518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.320+0000 7fe345e0b640 1 -- 172.21.15.38:0/569850609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3400aa730 con 0x7fe3400d0320 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.320+0000 7fe34560a640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3400d1e90 0x7fe3400b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.320+0000 7fe345e0b640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3400d0320 0x7fe3400b4c70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00cb50 tx=0x7fe33c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.320+0000 7fe3367fc640 1 -- 172.21.15.38:0/569850609 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c017070 con 0x7fe3400d0320 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.320+0000 7fe34806f640 1 -- 172.21.15.38:0/569850609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3400aaa10 con 0x7fe3400d0320 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.320+0000 7fe3367fc640 1 -- 172.21.15.38:0/569850609 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33c00ad40 con 0x7fe3400d0320 2024-04-25T08:32:10.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.320+0000 7fe3367fc640 1 -- 172.21.15.38:0/569850609 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c012620 con 0x7fe3400d0320 2024-04-25T08:32:10.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.320+0000 7fe34806f640 1 -- 172.21.15.38:0/569850609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3400aafd0 con 0x7fe3400d0320 2024-04-25T08:32:10.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.321+0000 7fe34806f640 1 -- 172.21.15.38:0/569850609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe340068e80 con 0x7fe3400d0320 2024-04-25T08:32:10.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.321+0000 7fe3367fc640 1 -- 172.21.15.38:0/569850609 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe33c01a070 con 0x7fe3400d0320 2024-04-25T08:32:10.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.322+0000 7fe3367fc640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe32406b4d0 0x7fe32406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.322+0000 7fe3367fc640 1 -- 172.21.15.38:0/569850609 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe33c08a360 con 0x7fe3400d0320 2024-04-25T08:32:10.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.322+0000 7fe34560a640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe32406b4d0 0x7fe32406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.323+0000 7fe34560a640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe32406b4d0 0x7fe32406d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe330034000 tx=0x7fe33002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:10.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.326+0000 7fe3367fc640 1 -- 172.21.15.38:0/569850609 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe33c058eb0 con 0x7fe3400d0320 2024-04-25T08:32:10.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.506+0000 7fe34806f640 1 -- 172.21.15.38:0/569850609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3400b2440 con 0x7fe3400d0320 2024-04-25T08:32:10.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.506+0000 7fe3367fc640 1 -- 172.21.15.38:0/569850609 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe33c0587b0 con 0x7fe3400d0320 2024-04-25T08:32:10.503 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.508+0000 7fe34806f640 1 -- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe32406b4d0 msgr2=0x7fe32406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.508+0000 7fe34806f640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe32406b4d0 0x7fe32406d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe330034000 tx=0x7fe33002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.508+0000 7fe34806f640 1 -- 172.21.15.38:0/569850609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3400d0320 msgr2=0x7fe3400b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.508+0000 7fe34806f640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3400d0320 0x7fe3400b4c70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00cb50 tx=0x7fe33c00a430 comp rx=0 tx=0).stop 2024-04-25T08:32:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.508+0000 7fe34806f640 1 -- 172.21.15.38:0/569850609 shutdown_connections 2024-04-25T08:32:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.508+0000 7fe34806f640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3400d33d0 0x7fe3401518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.508+0000 7fe34806f640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe32406b4d0 0x7fe32406d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.508+0000 7fe34806f640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3400d1e90 0x7fe3400b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.508+0000 7fe34806f640 1 --2- 172.21.15.38:0/569850609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3400d0320 0x7fe3400b4c70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.508+0000 7fe34806f640 1 -- 172.21.15.38:0/569850609 >> 172.21.15.38:0/569850609 conn(0x7fe3400668f0 msgr2=0x7fe3400b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.508+0000 7fe34806f640 1 -- 172.21.15.38:0/569850609 shutdown_connections 2024-04-25T08:32:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.508+0000 7fe34806f640 1 -- 172.21.15.38:0/569850609 wait complete. 2024-04-25T08:32:10.515 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:10.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.624+0000 7f83cfdee640 1 Processor -- start 2024-04-25T08:32:10.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.624+0000 7f83cfdee640 1 -- start start 2024-04-25T08:32:10.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83cfdee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c815b990 0x7f83c80cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83cfdee640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f83c80d1d20 0x7f83c80b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83cfdee640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83c815fd00 con 0x7f83c815f5f0 2024-04-25T08:32:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83cfdee640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c8160580 con 0x7f83c80d1d20 2024-04-25T08:32:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83cfdee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c815fa30 con 0x7f83c815b990 2024-04-25T08:32:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83ce38b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f83c80d1d20 0x7f83c80b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83cdb8a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c815b990 0x7f83c80cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83cd389640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f83c815f5f0 0x7f83c80d1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:49114/0 (socket says 172.21.15.38:49114) 2024-04-25T08:32:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83ce38b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f83c80d1d20 0x7f83c80b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:56878/0 (socket says 172.21.15.38:56878) 2024-04-25T08:32:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83cd389640 1 -- 172.21.15.38:0/339932039 learned_addr learned my addr 172.21.15.38:0/339932039 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83ccb88640 1 -- 172.21.15.38:0/339932039 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563878814 0 0) 0x7f83c815fd00 con 0x7f83c815f5f0 2024-04-25T08:32:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83cdb8a640 1 -- 172.21.15.38:0/339932039 >> v2:172.21.15.107:3300/0 conn(0x7f83c80d1d20 msgr2=0x7f83c80b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:10.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83cdb8a640 1 --2- 172.21.15.38:0/339932039 >> v2:172.21.15.107:3300/0 conn(0x7f83c80d1d20 0x7f83c80b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83cdb8a640 1 -- 172.21.15.38:0/339932039 >> v1:172.21.15.107:6789/0 conn(0x7f83c815f5f0 legacy=0x7f83c80d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.625+0000 7f83cdb8a640 1 -- 172.21.15.38:0/339932039 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c815ffc0 con 0x7f83c815b990 2024-04-25T08:32:10.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.626+0000 7f83cdb8a640 1 --2- 172.21.15.38:0/339932039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c815b990 0x7f83c80cf580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f83c40097a0 tx=0x7f83c402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14e065aa5ce3f9da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:10.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.626+0000 7f83ccb88640 1 -- 172.21.15.38:0/339932039 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4037070 con 0x7f83c815b990 2024-04-25T08:32:10.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.626+0000 7f83ccb88640 1 -- 172.21.15.38:0/339932039 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c402ae50 con 0x7f83c815b990 2024-04-25T08:32:10.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.626+0000 7f83ccb88640 1 -- 172.21.15.38:0/339932039 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c4032660 con 0x7f83c815b990 2024-04-25T08:32:10.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.627+0000 7f83cfdee640 1 -- 172.21.15.38:0/339932039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c815b990 msgr2=0x7f83c80cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.627+0000 7f83cfdee640 1 --2- 172.21.15.38:0/339932039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c815b990 0x7f83c80cf580 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f83c40097a0 tx=0x7f83c402a100 comp rx=0 tx=0).stop 2024-04-25T08:32:10.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.627+0000 7f83cfdee640 1 -- 172.21.15.38:0/339932039 shutdown_connections 2024-04-25T08:32:10.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.627+0000 7f83cfdee640 1 --2- 172.21.15.38:0/339932039 >> v2:172.21.15.107:3300/0 conn(0x7f83c80d1d20 0x7f83c80b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.627+0000 7f83cfdee640 1 --2- 172.21.15.38:0/339932039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c815b990 0x7f83c80cf580 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.627+0000 7f83cfdee640 1 -- 172.21.15.38:0/339932039 >> 172.21.15.38:0/339932039 conn(0x7f83c808b000 msgr2=0x7f83c808b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:10.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.627+0000 7f83cfdee640 1 -- 172.21.15.38:0/339932039 shutdown_connections 2024-04-25T08:32:10.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.627+0000 7f83cfdee640 1 -- 172.21.15.38:0/339932039 wait complete. 2024-04-25T08:32:10.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.628+0000 7f83cfdee640 1 Processor -- start 2024-04-25T08:32:10.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.628+0000 7f83cfdee640 1 -- start start 2024-04-25T08:32:10.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cfdee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c80d1d20 0x7f83c809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cfdee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c815b990 0x7f83c815e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cfdee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c815f5f0 0x7f83c80b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cfdee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c815bd70 con 0x7f83c815b990 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cfdee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c80b1ac0 con 0x7f83c815f5f0 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cfdee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c815bef0 con 0x7f83c80d1d20 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cd389640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c815b990 0x7f83c815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cd389640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c815b990 0x7f83c815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37436/0 (socket says 172.21.15.38:37436) 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cd389640 1 -- 172.21.15.38:0/1210691905 learned_addr learned my addr 172.21.15.38:0/1210691905 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83ce38b640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c815f5f0 0x7f83c80b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cdb8a640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c80d1d20 0x7f83c809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cd389640 1 -- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c80d1d20 msgr2=0x7f83c809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cd389640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c80d1d20 0x7f83c809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cd389640 1 -- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c815f5f0 msgr2=0x7f83c80b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cd389640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c815f5f0 0x7f83c80b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.629+0000 7f83cd389640 1 -- 172.21.15.38:0/1210691905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c815d490 con 0x7f83c815b990 2024-04-25T08:32:10.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.630+0000 7f83cd389640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c815b990 0x7f83c815e530 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f83b800c430 tx=0x7f83b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:10.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.630+0000 7f83be7fc640 1 -- 172.21.15.38:0/1210691905 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b80074e0 con 0x7f83c815b990 2024-04-25T08:32:10.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.630+0000 7f83cfdee640 1 -- 172.21.15.38:0/1210691905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c815d770 con 0x7f83c815b990 2024-04-25T08:32:10.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.630+0000 7f83be7fc640 1 -- 172.21.15.38:0/1210691905 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b80047e0 con 0x7f83c815b990 2024-04-25T08:32:10.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.630+0000 7f83be7fc640 1 -- 172.21.15.38:0/1210691905 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8004bd0 con 0x7f83c815b990 2024-04-25T08:32:10.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.630+0000 7f83cfdee640 1 -- 172.21.15.38:0/1210691905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c815dd30 con 0x7f83c815b990 2024-04-25T08:32:10.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.631+0000 7f83cfdee640 1 -- 172.21.15.38:0/1210691905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c8068e80 con 0x7f83c815b990 2024-04-25T08:32:10.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.631+0000 7f83be7fc640 1 -- 172.21.15.38:0/1210691905 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83b8011070 con 0x7f83c815b990 2024-04-25T08:32:10.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.632+0000 7f83be7fc640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83ac06b4d0 0x7f83ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.632+0000 7f83be7fc640 1 -- 172.21.15.38:0/1210691905 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f83b808a0c0 con 0x7f83c815b990 2024-04-25T08:32:10.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.632+0000 7f83cdb8a640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83ac06b4d0 0x7f83ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.632+0000 7f83cdb8a640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83ac06b4d0 0x7f83ac06d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83c4034000 tx=0x7f83c402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:10.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.636+0000 7f83be7fc640 1 -- 172.21.15.38:0/1210691905 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83b8058530 con 0x7f83c815b990 2024-04-25T08:32:10.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.813+0000 7f83cfdee640 1 -- 172.21.15.38:0/1210691905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f83c80b1220 con 0x7f83c815b990 2024-04-25T08:32:10.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.814+0000 7f83be7fc640 1 -- 172.21.15.38:0/1210691905 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f83b80770d0 con 0x7f83c815b990 2024-04-25T08:32:10.810 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:10.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.816+0000 7f83cfdee640 1 -- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83ac06b4d0 msgr2=0x7f83ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.816+0000 7f83cfdee640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83ac06b4d0 0x7f83ac06d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f83c4034000 tx=0x7f83c402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:10.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.816+0000 7f83cfdee640 1 -- 172.21.15.38:0/1210691905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c815b990 msgr2=0x7f83c815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.816+0000 7f83cfdee640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c815b990 0x7f83c815e530 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f83b800c430 tx=0x7f83b800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.816+0000 7f83cfdee640 1 -- 172.21.15.38:0/1210691905 shutdown_connections 2024-04-25T08:32:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.816+0000 7f83cfdee640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83c815f5f0 0x7f83c80b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.816+0000 7f83cfdee640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83c815b990 0x7f83c815e530 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.816+0000 7f83cfdee640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83ac06b4d0 0x7f83ac06d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.816+0000 7f83cfdee640 1 --2- 172.21.15.38:0/1210691905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83c80d1d20 0x7f83c809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.816+0000 7f83cfdee640 1 -- 172.21.15.38:0/1210691905 >> 172.21.15.38:0/1210691905 conn(0x7f83c808b000 msgr2=0x7f83c80b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.816+0000 7f83cfdee640 1 -- 172.21.15.38:0/1210691905 shutdown_connections 2024-04-25T08:32:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.816+0000 7f83cfdee640 1 -- 172.21.15.38:0/1210691905 wait complete. 2024-04-25T08:32:10.823 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.930+0000 7f9de695c640 1 Processor -- start 2024-04-25T08:32:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.930+0000 7f9de695c640 1 -- start start 2024-04-25T08:32:10.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.930+0000 7f9de695c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9de00d0320 0x7f9de00d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.930+0000 7f9de695c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9de00b6980 0x7f9de00b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.930+0000 7f9de695c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9de00d2060 con 0x7f9de00d0700 2024-04-25T08:32:10.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.930+0000 7f9de695c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9de00d2330 con 0x7f9de00d0320 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.930+0000 7f9de695c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de00d2610 con 0x7f9de00b6980 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddf7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9de00d0700 0x7f9de00d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54004/0 (socket says 172.21.15.38:54004) 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddf7fe640 1 -- 172.21.15.38:0/2380405670 learned_addr learned my addr 172.21.15.38:0/2380405670 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9de4ef9640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9de00b6980 0x7f9de00b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddffff640 1 --2- 172.21.15.38:0/2380405670 >> v2:172.21.15.107:3300/0 conn(0x7f9de00d0320 0x7f9de00d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3258946690 0 0) 0x7f9de00d2060 con 0x7f9de00d0700 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dc8003520 con 0x7f9de00d0700 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1768323440 0 0) 0x7f9dc8003520 con 0x7f9de00d0700 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9de00b6980 msgr2=0x7f9de00b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9de00b6980 0x7f9de00b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 >> v2:172.21.15.107:3300/0 conn(0x7f9de00d0320 msgr2=0x7f9de00d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 --2- 172.21.15.38:0/2380405670 >> v2:172.21.15.107:3300/0 conn(0x7f9de00d0320 0x7f9de00d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00d28f0 con 0x7f9de00d0700 2024-04-25T08:32:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f9dd4001670 con 0x7f9de00d0700 2024-04-25T08:32:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9de4ef9640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9de00b6980 0x7f9de00b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 >> v1:172.21.15.38:6789/0 conn(0x7f9de00d0700 legacy=0x7f9de00d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9de00b6980 0x7f9dc80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc8003ef0 0x7f9dc8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dc8004d40 0x7f9dc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de00d2330 con 0x7f9dc8003ef0 2024-04-25T08:32:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de00d2610 con 0x7f9dc8004d40 2024-04-25T08:32:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc800b9c0 con 0x7f9de00b6980 2024-04-25T08:32:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddf7fe640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc8003ef0 0x7f9dc8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9ddffff640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9de00b6980 0x7f9dc80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.931+0000 7f9de4ef9640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dc8004d40 0x7f9dc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.932+0000 7f9ddf7fe640 1 -- 172.21.15.38:0/2380405670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9de00b6980 msgr2=0x7f9dc80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.932+0000 7f9ddf7fe640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9de00b6980 0x7f9dc80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.932+0000 7f9ddf7fe640 1 -- 172.21.15.38:0/2380405670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dc8004d40 msgr2=0x7f9dc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.932+0000 7f9ddf7fe640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9dc8004d40 0x7f9dc80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.932+0000 7f9ddf7fe640 1 -- 172.21.15.38:0/2380405670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc8003520 con 0x7f9dc8003ef0 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.932+0000 7f9ddf7fe640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc8003ef0 0x7f9dc8004800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400c240 tx=0x7f9dc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72d87403f62e9d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.932+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4007d50 con 0x7f9dc8003ef0 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.932+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd402e070 con 0x7f9dc8003ef0 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.932+0000 7f9ddeffd640 1 -- 172.21.15.38:0/2380405670 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4005d70 con 0x7f9dc8003ef0 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.933+0000 7f9de695c640 1 -- 172.21.15.38:0/2380405670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc8003ef0 msgr2=0x7f9dc8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.933+0000 7f9de695c640 1 --2- 172.21.15.38:0/2380405670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9dc8003ef0 0x7f9dc8004800 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9dd400c240 tx=0x7f9dc8005830 comp rx=0 tx=0).stop 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.933+0000 7f9de4ef9640 1 -- 172.21.15.38:0/2380405670 reap_dead start 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.933+0000 7f9de695c640 1 -- 172.21.15.38:0/2380405670 shutdown_connections 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.933+0000 7f9de695c640 1 -- 172.21.15.38:0/2380405670 >> 172.21.15.38:0/2380405670 conn(0x7f9de00668f0 msgr2=0x7f9de0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:10.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.933+0000 7f9de695c640 1 -- 172.21.15.38:0/2380405670 shutdown_connections 2024-04-25T08:32:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.933+0000 7f9de695c640 1 -- 172.21.15.38:0/2380405670 wait complete. 2024-04-25T08:32:10.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.934+0000 7f9de695c640 1 Processor -- start 2024-04-25T08:32:10.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.934+0000 7f9de695c640 1 -- start start 2024-04-25T08:32:10.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.935+0000 7f9de695c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc8003ef0 0x7f9de00c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.935+0000 7f9de695c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de01f7070 0x7f9de01f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.935+0000 7f9de695c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9de0158790 0x7f9de0158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.935+0000 7f9de695c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de00d0b40 con 0x7f9de01f7070 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.935+0000 7f9de695c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9de00d2eb0 con 0x7f9de0158790 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.935+0000 7f9de695c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9de0159f10 con 0x7f9dc8003ef0 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.935+0000 7f9ddf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de01f7070 0x7f9de01f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.935+0000 7f9ddf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de01f7070 0x7f9de01f74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37454/0 (socket says 172.21.15.38:37454) 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.935+0000 7f9ddf7fe640 1 -- 172.21.15.38:0/2079661764 learned_addr learned my addr 172.21.15.38:0/2079661764 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.935+0000 7f9ddffff640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc8003ef0 0x7f9de00c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.935+0000 7f9de4ef9640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9de0158790 0x7f9de0158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.936+0000 7f9ddf7fe640 1 -- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc8003ef0 msgr2=0x7f9de00c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.936+0000 7f9ddf7fe640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc8003ef0 0x7f9de00c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.936+0000 7f9ddf7fe640 1 -- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9de0158790 msgr2=0x7f9de0158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.936+0000 7f9ddf7fe640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9de0158790 0x7f9de0158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.936+0000 7f9ddf7fe640 1 -- 172.21.15.38:0/2079661764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00b5a70 con 0x7f9de01f7070 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.936+0000 7f9ddffff640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc8003ef0 0x7f9de00c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:10.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.936+0000 7f9ddf7fe640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de01f7070 0x7f9de01f74f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9dd402ae80 tx=0x7f9dd4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:10.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.936+0000 7f9ddcff9640 1 -- 172.21.15.38:0/2079661764 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4041020 con 0x7f9de01f7070 2024-04-25T08:32:10.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.936+0000 7f9ddcff9640 1 -- 172.21.15.38:0/2079661764 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd4007280 con 0x7f9de01f7070 2024-04-25T08:32:10.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.936+0000 7f9de695c640 1 -- 172.21.15.38:0/2079661764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9de0146fa0 con 0x7f9de01f7070 2024-04-25T08:32:10.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.936+0000 7f9ddcff9640 1 -- 172.21.15.38:0/2079661764 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd4004e60 con 0x7f9de01f7070 2024-04-25T08:32:10.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.936+0000 7f9de695c640 1 -- 172.21.15.38:0/2079661764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de0159150 con 0x7f9de01f7070 2024-04-25T08:32:10.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.937+0000 7f9de695c640 1 -- 172.21.15.38:0/2079661764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de0146d90 con 0x7f9de01f7070 2024-04-25T08:32:10.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.938+0000 7f9ddcff9640 1 -- 172.21.15.38:0/2079661764 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9dd4044020 con 0x7f9de01f7070 2024-04-25T08:32:10.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.938+0000 7f9ddcff9640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9dcc06b4d0 0x7f9dcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:10.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.938+0000 7f9ddcff9640 1 -- 172.21.15.38:0/2079661764 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9dd40aa520 con 0x7f9de01f7070 2024-04-25T08:32:10.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.938+0000 7f9ddffff640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9dcc06b4d0 0x7f9dcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:10.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.939+0000 7f9ddffff640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9dcc06b4d0 0x7f9dcc06d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009bc0 tx=0x7f9dd0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:10.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:10.942+0000 7f9ddcff9640 1 -- 172.21.15.38:0/2079661764 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9dd403b330 con 0x7f9de01f7070 2024-04-25T08:32:11.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.123+0000 7f9de695c640 1 -- 172.21.15.38:0/2079661764 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9de0068e80 con 0x7f9de01f7070 2024-04-25T08:32:11.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.123+0000 7f9ddcff9640 1 -- 172.21.15.38:0/2079661764 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9dd4079140 con 0x7f9de01f7070 2024-04-25T08:32:11.119 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:11.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.125+0000 7f9de695c640 1 -- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9dcc06b4d0 msgr2=0x7f9dcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.125+0000 7f9de695c640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9dcc06b4d0 0x7f9dcc06d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0009bc0 tx=0x7f9dd0008040 comp rx=0 tx=0).stop 2024-04-25T08:32:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.125+0000 7f9de695c640 1 -- 172.21.15.38:0/2079661764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de01f7070 msgr2=0x7f9de01f74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.125+0000 7f9de695c640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de01f7070 0x7f9de01f74f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9dd402ae80 tx=0x7f9dd4001a40 comp rx=0 tx=0).stop 2024-04-25T08:32:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.125+0000 7f9de695c640 1 -- 172.21.15.38:0/2079661764 shutdown_connections 2024-04-25T08:32:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.125+0000 7f9de695c640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9de0158790 0x7f9de0158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.125+0000 7f9de695c640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9de01f7070 0x7f9de01f74f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.125+0000 7f9de695c640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9dcc06b4d0 0x7f9dcc06d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.125+0000 7f9de695c640 1 --2- 172.21.15.38:0/2079661764 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9dc8003ef0 0x7f9de00c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.125+0000 7f9de695c640 1 -- 172.21.15.38:0/2079661764 >> 172.21.15.38:0/2079661764 conn(0x7f9de00668f0 msgr2=0x7f9de00ca9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.125+0000 7f9de695c640 1 -- 172.21.15.38:0/2079661764 shutdown_connections 2024-04-25T08:32:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.125+0000 7f9de695c640 1 -- 172.21.15.38:0/2079661764 wait complete. 2024-04-25T08:32:11.132 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:11.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.237+0000 7f6ebfdb0640 1 Processor -- start 2024-04-25T08:32:11.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.237+0000 7f6ebfdb0640 1 -- start start 2024-04-25T08:32:11.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebfdb0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6eb815dc10 0x7f6eb80ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebfdb0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6eb80ced60 0x7f6eb80d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebfdb0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb80d3e70 con 0x7f6eb80d3900 2024-04-25T08:32:11.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebfdb0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb815dff0 con 0x7f6eb815dc10 2024-04-25T08:32:11.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebfdb0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb80b6990 con 0x7f6eb80ced60 2024-04-25T08:32:11.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebdb4c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6eb815dc10 0x7f6eb80ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebe34d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6eb80d3900 0x7f6eb80b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54014/0 (socket says 172.21.15.38:54014) 2024-04-25T08:32:11.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebe34d640 1 -- 172.21.15.38:0/233718896 learned_addr learned my addr 172.21.15.38:0/233718896 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:11.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebd34b640 1 --2- 172.21.15.38:0/233718896 >> v2:172.21.15.107:3300/0 conn(0x7f6eb80ced60 0x7f6eb80d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebdb4c640 1 -- 172.21.15.38:0/233718896 >> v2:172.21.15.107:3300/0 conn(0x7f6eb80ced60 msgr2=0x7f6eb80d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:11.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebdb4c640 1 --2- 172.21.15.38:0/233718896 >> v2:172.21.15.107:3300/0 conn(0x7f6eb80ced60 0x7f6eb80d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebdb4c640 1 -- 172.21.15.38:0/233718896 >> v1:172.21.15.38:6789/0 conn(0x7f6eb80d3900 legacy=0x7f6eb80b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebcb4a640 1 -- 172.21.15.38:0/233718896 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949480543 0 0) 0x7f6eb80d3e70 con 0x7f6eb80d3900 2024-04-25T08:32:11.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.238+0000 7f6ebdb4c640 1 -- 172.21.15.38:0/233718896 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb80d4150 con 0x7f6eb815dc10 2024-04-25T08:32:11.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.239+0000 7f6ebdb4c640 1 --2- 172.21.15.38:0/233718896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb815dc10 0x7f6eb80ce820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80b9600 tx=0x7f6eb402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b2d18314e95b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.239+0000 7f6ebcb4a640 1 -- 172.21.15.38:0/233718896 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4037070 con 0x7f6eb815dc10 2024-04-25T08:32:11.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.239+0000 7f6ebcb4a640 1 -- 172.21.15.38:0/233718896 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4032440 con 0x7f6eb815dc10 2024-04-25T08:32:11.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.239+0000 7f6ebcb4a640 1 -- 172.21.15.38:0/233718896 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb402ae60 con 0x7f6eb815dc10 2024-04-25T08:32:11.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.240+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/233718896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb815dc10 msgr2=0x7f6eb80ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.240+0000 7f6ebfdb0640 1 --2- 172.21.15.38:0/233718896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb815dc10 0x7f6eb80ce820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80b9600 tx=0x7f6eb402a110 comp rx=0 tx=0).stop 2024-04-25T08:32:11.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.240+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/233718896 shutdown_connections 2024-04-25T08:32:11.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.240+0000 7f6ebfdb0640 1 --2- 172.21.15.38:0/233718896 >> v2:172.21.15.107:3300/0 conn(0x7f6eb80ced60 0x7f6eb80d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.240+0000 7f6ebfdb0640 1 --2- 172.21.15.38:0/233718896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb815dc10 0x7f6eb80ce820 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.240+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/233718896 >> 172.21.15.38:0/233718896 conn(0x7f6eb8066d10 msgr2=0x7f6eb8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:11.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.240+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/233718896 shutdown_connections 2024-04-25T08:32:11.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.240+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/233718896 wait complete. 2024-04-25T08:32:11.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.241+0000 7f6ebfdb0640 1 Processor -- start 2024-04-25T08:32:11.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.241+0000 7f6ebfdb0640 1 -- start start 2024-04-25T08:32:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebfdb0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6eb80ced60 0x7f6eb8157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebfdb0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6eb80d3900 0x7f6eb80cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebfdb0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb815dc10 0x7f6eb80d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebfdb0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80b6bc0 con 0x7f6eb815dc10 2024-04-25T08:32:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebfdb0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80d4710 con 0x7f6eb80ced60 2024-04-25T08:32:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebfdb0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80d0d90 con 0x7f6eb80d3900 2024-04-25T08:32:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebe34d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb815dc10 0x7f6eb80d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebe34d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb815dc10 0x7f6eb80d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37460/0 (socket says 172.21.15.38:37460) 2024-04-25T08:32:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebe34d640 1 -- 172.21.15.38:0/2631246684 learned_addr learned my addr 172.21.15.38:0/2631246684 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebdb4c640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6eb80ced60 0x7f6eb8157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebd34b640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6eb80d3900 0x7f6eb80cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebe34d640 1 -- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6eb80d3900 msgr2=0x7f6eb80cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebe34d640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6eb80d3900 0x7f6eb80cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebe34d640 1 -- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6eb80ced60 msgr2=0x7f6eb8157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebe34d640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6eb80ced60 0x7f6eb8157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebe34d640 1 -- 172.21.15.38:0/2631246684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb80cc8e0 con 0x7f6eb815dc10 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebd34b640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6eb80d3900 0x7f6eb80cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebdb4c640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6eb80ced60 0x7f6eb8157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebe34d640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb815dc10 0x7f6eb80d1c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000c7d0 tx=0x7f6eb000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6eae7fc640 1 -- 172.21.15.38:0/2631246684 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0018070 con 0x7f6eb815dc10 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6eae7fc640 1 -- 172.21.15.38:0/2631246684 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb000a640 con 0x7f6eb815dc10 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/2631246684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb80ccbc0 con 0x7f6eb815dc10 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6eae7fc640 1 -- 172.21.15.38:0/2631246684 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb000aa70 con 0x7f6eb815dc10 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.242+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/2631246684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb80cd180 con 0x7f6eb815dc10 2024-04-25T08:32:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.243+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/2631246684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb8068e80 con 0x7f6eb815dc10 2024-04-25T08:32:11.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.244+0000 7f6eae7fc640 1 -- 172.21.15.38:0/2631246684 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6eb001b070 con 0x7f6eb815dc10 2024-04-25T08:32:11.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.244+0000 7f6eae7fc640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e9c06b500 0x7f6e9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.245+0000 7f6eae7fc640 1 -- 172.21.15.38:0/2631246684 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6eb0089b30 con 0x7f6eb815dc10 2024-04-25T08:32:11.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.245+0000 7f6ebdb4c640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e9c06b500 0x7f6e9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.245+0000 7f6ebdb4c640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e9c06b500 0x7f6e9c06d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4034000 tx=0x7f6eb402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.249+0000 7f6eae7fc640 1 -- 172.21.15.38:0/2631246684 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6eb0058670 con 0x7f6eb815dc10 2024-04-25T08:32:11.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.429+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/2631246684 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6eb81e9420 con 0x7f6eb815dc10 2024-04-25T08:32:11.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.431+0000 7f6eae7fc640 1 -- 172.21.15.38:0/2631246684 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7f6eb0057f70 con 0x7f6eb815dc10 2024-04-25T08:32:11.427 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:11.427 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:11.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.433+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e9c06b500 msgr2=0x7f6e9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.433+0000 7f6ebfdb0640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e9c06b500 0x7f6e9c06d9c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4034000 tx=0x7f6eb402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:11.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.433+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/2631246684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb815dc10 msgr2=0x7f6eb80d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.433+0000 7f6ebfdb0640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb815dc10 0x7f6eb80d1c20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000c7d0 tx=0x7f6eb000cc90 comp rx=0 tx=0).stop 2024-04-25T08:32:11.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.433+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/2631246684 shutdown_connections 2024-04-25T08:32:11.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.433+0000 7f6ebfdb0640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6eb815dc10 0x7f6eb80d1c20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.433+0000 7f6ebfdb0640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e9c06b500 0x7f6e9c06d9c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.433+0000 7f6ebfdb0640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6eb80d3900 0x7f6eb80cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.433+0000 7f6ebfdb0640 1 --2- 172.21.15.38:0/2631246684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6eb80ced60 0x7f6eb8157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.433+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/2631246684 >> 172.21.15.38:0/2631246684 conn(0x7f6eb8066d10 msgr2=0x7f6eb815ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:11.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.433+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/2631246684 shutdown_connections 2024-04-25T08:32:11.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.433+0000 7f6ebfdb0640 1 -- 172.21.15.38:0/2631246684 wait complete. 2024-04-25T08:32:11.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:11.441 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:11.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.548+0000 7fca68e5d640 1 Processor -- start 2024-04-25T08:32:11.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.548+0000 7fca68e5d640 1 -- start start 2024-04-25T08:32:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca68e5d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fca640d00e0 0x7fca640cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca68e5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca640cd3b0 0x7fca640cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca68e5d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca640cdd80 con 0x7fca640b1d60 2024-04-25T08:32:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca68e5d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6406abe0 con 0x7fca640d00e0 2024-04-25T08:32:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca68e5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6406b480 con 0x7fca640cd3b0 2024-04-25T08:32:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca61d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fca640d00e0 0x7fca640cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca6259c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fca640b1d60 0x7fca640cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54026/0 (socket says 172.21.15.38:54026) 2024-04-25T08:32:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca6259c640 1 -- 172.21.15.38:0/480333783 learned_addr learned my addr 172.21.15.38:0/480333783 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca62d9d640 1 --2- 172.21.15.38:0/480333783 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca640cd3b0 0x7fca640cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca61d9b640 1 -- 172.21.15.38:0/480333783 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca640cd3b0 msgr2=0x7fca640cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:11.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca61d9b640 1 --2- 172.21.15.38:0/480333783 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca640cd3b0 0x7fca640cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca61d9b640 1 -- 172.21.15.38:0/480333783 >> v1:172.21.15.38:6789/0 conn(0x7fca640b1d60 legacy=0x7fca640cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.549+0000 7fca61d9b640 1 -- 172.21.15.38:0/480333783 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6415ef40 con 0x7fca640d00e0 2024-04-25T08:32:11.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.550+0000 7fca61d9b640 1 --2- 172.21.15.38:0/480333783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca640d00e0 0x7fca640cce70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fca540097a0 tx=0x7fca5402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3606ffc93324fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.551+0000 7fca6159a640 1 -- 172.21.15.38:0/480333783 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54037070 con 0x7fca640d00e0 2024-04-25T08:32:11.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.551+0000 7fca6159a640 1 -- 172.21.15.38:0/480333783 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5402ae50 con 0x7fca640d00e0 2024-04-25T08:32:11.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.551+0000 7fca6159a640 1 -- 172.21.15.38:0/480333783 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54032660 con 0x7fca640d00e0 2024-04-25T08:32:11.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.552+0000 7fca68e5d640 1 -- 172.21.15.38:0/480333783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca640d00e0 msgr2=0x7fca640cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.552+0000 7fca68e5d640 1 --2- 172.21.15.38:0/480333783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca640d00e0 0x7fca640cce70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fca540097a0 tx=0x7fca5402a100 comp rx=0 tx=0).stop 2024-04-25T08:32:11.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.552+0000 7fca68e5d640 1 -- 172.21.15.38:0/480333783 shutdown_connections 2024-04-25T08:32:11.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.552+0000 7fca68e5d640 1 --2- 172.21.15.38:0/480333783 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca640cd3b0 0x7fca640cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.552+0000 7fca68e5d640 1 --2- 172.21.15.38:0/480333783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca640d00e0 0x7fca640cce70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.552+0000 7fca68e5d640 1 -- 172.21.15.38:0/480333783 >> 172.21.15.38:0/480333783 conn(0x7fca640d06b0 msgr2=0x7fca640d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:11.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.552+0000 7fca68e5d640 1 -- 172.21.15.38:0/480333783 shutdown_connections 2024-04-25T08:32:11.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.552+0000 7fca68e5d640 1 -- 172.21.15.38:0/480333783 wait complete. 2024-04-25T08:32:11.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.553+0000 7fca68e5d640 1 Processor -- start 2024-04-25T08:32:11.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.554+0000 7fca68e5d640 1 -- start start 2024-04-25T08:32:11.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.554+0000 7fca68e5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca640b1d60 0x7fca640b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.554+0000 7fca68e5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca640cd3b0 0x7fca641530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.554+0000 7fca68e5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca640d00e0 0x7fca641e7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.554+0000 7fca68e5d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6406b910 con 0x7fca640cd3b0 2024-04-25T08:32:11.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.554+0000 7fca68e5d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6406b110 con 0x7fca640b1d60 2024-04-25T08:32:11.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.554+0000 7fca68e5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6406ba90 con 0x7fca640d00e0 2024-04-25T08:32:11.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.554+0000 7fca61d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca640cd3b0 0x7fca641530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.554+0000 7fca61d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca640cd3b0 0x7fca641530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37470/0 (socket says 172.21.15.38:37470) 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.554+0000 7fca61d9b640 1 -- 172.21.15.38:0/2382136722 learned_addr learned my addr 172.21.15.38:0/2382136722 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca6259c640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca640b1d60 0x7fca640b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca62d9d640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca640d00e0 0x7fca641e7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca61d9b640 1 -- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca640d00e0 msgr2=0x7fca641e7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca61d9b640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca640d00e0 0x7fca641e7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca61d9b640 1 -- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca640b1d60 msgr2=0x7fca640b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca61d9b640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca640b1d60 0x7fca640b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca61d9b640 1 -- 172.21.15.38:0/2382136722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca640b5ea0 con 0x7fca640cd3b0 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca6259c640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca640b1d60 0x7fca640b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca61d9b640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca640cd3b0 0x7fca641530c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fca54000c00 tx=0x7fca540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca5affd640 1 -- 172.21.15.38:0/2382136722 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54041070 con 0x7fca640cd3b0 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca5affd640 1 -- 172.21.15.38:0/2382136722 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5403c670 con 0x7fca640cd3b0 2024-04-25T08:32:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca5affd640 1 -- 172.21.15.38:0/2382136722 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca54037040 con 0x7fca640cd3b0 2024-04-25T08:32:11.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca68e5d640 1 -- 172.21.15.38:0/2382136722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca640b3d20 con 0x7fca640cd3b0 2024-04-25T08:32:11.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.555+0000 7fca68e5d640 1 -- 172.21.15.38:0/2382136722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca640b6180 con 0x7fca640cd3b0 2024-04-25T08:32:11.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.556+0000 7fca68e5d640 1 -- 172.21.15.38:0/2382136722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca64068e80 con 0x7fca640cd3b0 2024-04-25T08:32:11.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.556+0000 7fca5affd640 1 -- 172.21.15.38:0/2382136722 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca54044070 con 0x7fca640cd3b0 2024-04-25T08:32:11.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.557+0000 7fca5affd640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fca4406b4d0 0x7fca4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.557+0000 7fca5affd640 1 -- 172.21.15.38:0/2382136722 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fca540aac70 con 0x7fca640cd3b0 2024-04-25T08:32:11.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.557+0000 7fca6259c640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fca4406b4d0 0x7fca4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.557+0000 7fca6259c640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fca4406b4d0 0x7fca4406d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fca4c000f60 tx=0x7fca4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.561+0000 7fca5affd640 1 -- 172.21.15.38:0/2382136722 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca54079960 con 0x7fca640cd3b0 2024-04-25T08:32:11.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.740+0000 7fca68e5d640 1 -- 172.21.15.38:0/2382136722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca64161430 con 0x7fca640cd3b0 2024-04-25T08:32:11.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.741+0000 7fca5affd640 1 -- 172.21.15.38:0/2382136722 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca54079260 con 0x7fca640cd3b0 2024-04-25T08:32:11.737 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.743+0000 7fca68e5d640 1 -- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fca4406b4d0 msgr2=0x7fca4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.743+0000 7fca68e5d640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fca4406b4d0 0x7fca4406d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fca4c000f60 tx=0x7fca4c009040 comp rx=0 tx=0).stop 2024-04-25T08:32:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.743+0000 7fca68e5d640 1 -- 172.21.15.38:0/2382136722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca640cd3b0 msgr2=0x7fca641530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.743+0000 7fca68e5d640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca640cd3b0 0x7fca641530c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fca54000c00 tx=0x7fca540046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.743+0000 7fca68e5d640 1 -- 172.21.15.38:0/2382136722 shutdown_connections 2024-04-25T08:32:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.743+0000 7fca68e5d640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fca4406b4d0 0x7fca4406d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.743+0000 7fca68e5d640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca640d00e0 0x7fca641e7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.743+0000 7fca68e5d640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fca640cd3b0 0x7fca641530c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.743+0000 7fca68e5d640 1 --2- 172.21.15.38:0/2382136722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca640b1d60 0x7fca640b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.743+0000 7fca68e5d640 1 -- 172.21.15.38:0/2382136722 >> 172.21.15.38:0/2382136722 conn(0x7fca640d06b0 msgr2=0x7fca640cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.743+0000 7fca68e5d640 1 -- 172.21.15.38:0/2382136722 shutdown_connections 2024-04-25T08:32:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.743+0000 7fca68e5d640 1 -- 172.21.15.38:0/2382136722 wait complete. 2024-04-25T08:32:11.750 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:11.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.857+0000 7f3bae45d640 1 Processor -- start 2024-04-25T08:32:11.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.858+0000 7f3bae45d640 1 -- start start 2024-04-25T08:32:11.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.858+0000 7f3bae45d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ba815de60 0x7f3ba80d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.858+0000 7f3bae45d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3ba80cf7b0 0x7f3ba80b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.858+0000 7f3bae45d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba80ce710 con 0x7f3ba80d1e00 2024-04-25T08:32:11.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.858+0000 7f3bae45d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba80ce4e0 con 0x7f3ba80cf7b0 2024-04-25T08:32:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.858+0000 7f3bae45d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80cf320 con 0x7f3ba815de60 2024-04-25T08:32:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.858+0000 7f3bac9fa640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3ba80cf7b0 0x7f3ba80b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.859+0000 7f3bac9fa640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3ba80cf7b0 0x7f3ba80b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37484/0 (socket says 172.21.15.38:37484) 2024-04-25T08:32:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.859+0000 7f3bac9fa640 1 -- 172.21.15.38:0/180994813 learned_addr learned my addr 172.21.15.38:0/180994813 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.859+0000 7f3ba77fe640 1 --2- 172.21.15.38:0/180994813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ba815de60 0x7f3ba80d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.859+0000 7f3bac9fa640 1 -- 172.21.15.38:0/180994813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ba815de60 msgr2=0x7f3ba80d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.859+0000 7f3bac9fa640 1 --2- 172.21.15.38:0/180994813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ba815de60 0x7f3ba80d42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.859+0000 7f3bac9fa640 1 -- 172.21.15.38:0/180994813 >> v1:172.21.15.107:6789/0 conn(0x7f3ba80d1e00 legacy=0x7f3ba80d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.859+0000 7f3bac9fa640 1 -- 172.21.15.38:0/180994813 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80ce9f0 con 0x7f3ba80cf7b0 2024-04-25T08:32:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.859+0000 7f3ba77fe640 1 --2- 172.21.15.38:0/180994813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ba815de60 0x7f3ba80d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.860+0000 7f3bac9fa640 1 --2- 172.21.15.38:0/180994813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ba80cf7b0 0x7f3ba80b3180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009e50 tx=0x7f3ba002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bb6a24d7810d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.860+0000 7f3ba6ffd640 1 -- 172.21.15.38:0/180994813 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0036070 con 0x7f3ba80cf7b0 2024-04-25T08:32:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.860+0000 7f3ba6ffd640 1 -- 172.21.15.38:0/180994813 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba0031600 con 0x7f3ba80cf7b0 2024-04-25T08:32:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.860+0000 7f3ba6ffd640 1 -- 172.21.15.38:0/180994813 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00319d0 con 0x7f3ba80cf7b0 2024-04-25T08:32:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.860+0000 7f3bae45d640 1 -- 172.21.15.38:0/180994813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ba80cf7b0 msgr2=0x7f3ba80b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.860+0000 7f3bae45d640 1 --2- 172.21.15.38:0/180994813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ba80cf7b0 0x7f3ba80b3180 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009e50 tx=0x7f3ba002a6c0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.860+0000 7f3bae45d640 1 -- 172.21.15.38:0/180994813 shutdown_connections 2024-04-25T08:32:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.860+0000 7f3bae45d640 1 --2- 172.21.15.38:0/180994813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ba80cf7b0 0x7f3ba80b3180 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.860+0000 7f3bae45d640 1 --2- 172.21.15.38:0/180994813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ba815de60 0x7f3ba80d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.860+0000 7f3bae45d640 1 -- 172.21.15.38:0/180994813 >> 172.21.15.38:0/180994813 conn(0x7f3ba808b000 msgr2=0x7f3ba80d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.860+0000 7f3bae45d640 1 -- 172.21.15.38:0/180994813 shutdown_connections 2024-04-25T08:32:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.860+0000 7f3bae45d640 1 -- 172.21.15.38:0/180994813 wait complete. 2024-04-25T08:32:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.861+0000 7f3bae45d640 1 Processor -- start 2024-04-25T08:32:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.862+0000 7f3bae45d640 1 -- start start 2024-04-25T08:32:11.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.862+0000 7f3bae45d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba80cf7b0 0x7f3ba80cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.862+0000 7f3bae45d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ba80d1e00 0x7f3ba81f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.862+0000 7f3bae45d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ba815de60 0x7f3ba8150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.862+0000 7f3bae45d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80d1600 con 0x7f3ba815de60 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.862+0000 7f3bae45d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80cefb0 con 0x7f3ba80cf7b0 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.862+0000 7f3bae45d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80d1780 con 0x7f3ba80d1e00 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3bac9fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ba815de60 0x7f3ba8150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3bac9fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ba815de60 0x7f3ba8150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37496/0 (socket says 172.21.15.38:37496) 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3bac9fa640 1 -- 172.21.15.38:0/2608050565 learned_addr learned my addr 172.21.15.38:0/2608050565 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3ba7fff640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba80cf7b0 0x7f3ba80cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3ba77fe640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ba80d1e00 0x7f3ba81f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3bac9fa640 1 -- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ba80d1e00 msgr2=0x7f3ba81f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3bac9fa640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ba80d1e00 0x7f3ba81f7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3bac9fa640 1 -- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba80cf7b0 msgr2=0x7f3ba80cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:11.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3bac9fa640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba80cf7b0 0x7f3ba80cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3bac9fa640 1 -- 172.21.15.38:0/2608050565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80caa90 con 0x7f3ba815de60 2024-04-25T08:32:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3bac9fa640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ba815de60 0x7f3ba8150880 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009e50 tx=0x7f3ba0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3ba4ff9640 1 -- 172.21.15.38:0/2608050565 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0036050 con 0x7f3ba815de60 2024-04-25T08:32:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3ba4ff9640 1 -- 172.21.15.38:0/2608050565 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba0042070 con 0x7f3ba815de60 2024-04-25T08:32:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3bae45d640 1 -- 172.21.15.38:0/2608050565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba80cd560 con 0x7f3ba815de60 2024-04-25T08:32:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3ba4ff9640 1 -- 172.21.15.38:0/2608050565 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0009510 con 0x7f3ba815de60 2024-04-25T08:32:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.863+0000 7f3bae45d640 1 -- 172.21.15.38:0/2608050565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba80cb010 con 0x7f3ba815de60 2024-04-25T08:32:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.864+0000 7f3bae45d640 1 -- 172.21.15.38:0/2608050565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba80cde40 con 0x7f3ba815de60 2024-04-25T08:32:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.865+0000 7f3ba4ff9640 1 -- 172.21.15.38:0/2608050565 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ba0007bd0 con 0x7f3ba815de60 2024-04-25T08:32:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.865+0000 7f3ba4ff9640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b9406b4d0 0x7f3b9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.865+0000 7f3ba4ff9640 1 -- 172.21.15.38:0/2608050565 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3ba00ab360 con 0x7f3ba815de60 2024-04-25T08:32:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.865+0000 7f3ba7fff640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b9406b4d0 0x7f3b9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.866+0000 7f3ba7fff640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b9406b4d0 0x7f3b9406d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3b98004f60 tx=0x7f3b98007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:11.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:11.869+0000 7f3ba4ff9640 1 -- 172.21.15.38:0/2608050565 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ba80cde40 con 0x7f3ba815de60 2024-04-25T08:32:12.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.048+0000 7f3bae45d640 1 -- 172.21.15.38:0/2608050565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ba8068e80 con 0x7f3ba815de60 2024-04-25T08:32:12.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.049+0000 7f3ba4ff9640 1 -- 172.21.15.38:0/2608050565 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ba0079850 con 0x7f3ba815de60 2024-04-25T08:32:12.045 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.051+0000 7f3bae45d640 1 -- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b9406b4d0 msgr2=0x7f3b9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.051+0000 7f3bae45d640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b9406b4d0 0x7f3b9406d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3b98004f60 tx=0x7f3b98007e30 comp rx=0 tx=0).stop 2024-04-25T08:32:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.051+0000 7f3bae45d640 1 -- 172.21.15.38:0/2608050565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ba815de60 msgr2=0x7f3ba8150880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.051+0000 7f3bae45d640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ba815de60 0x7f3ba8150880 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009e50 tx=0x7f3ba0004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.051+0000 7f3bae45d640 1 -- 172.21.15.38:0/2608050565 shutdown_connections 2024-04-25T08:32:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.051+0000 7f3bae45d640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3ba815de60 0x7f3ba8150880 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.051+0000 7f3bae45d640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b9406b4d0 0x7f3b9406d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.051+0000 7f3bae45d640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3ba80d1e00 0x7f3ba81f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.051+0000 7f3bae45d640 1 --2- 172.21.15.38:0/2608050565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3ba80cf7b0 0x7f3ba80cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.051+0000 7f3bae45d640 1 -- 172.21.15.38:0/2608050565 >> 172.21.15.38:0/2608050565 conn(0x7f3ba808b000 msgr2=0x7f3ba80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.051+0000 7f3bae45d640 1 -- 172.21.15.38:0/2608050565 shutdown_connections 2024-04-25T08:32:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.051+0000 7f3bae45d640 1 -- 172.21.15.38:0/2608050565 wait complete. 2024-04-25T08:32:12.058 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:12.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.166+0000 7f6d4689e640 1 Processor -- start 2024-04-25T08:32:12.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.166+0000 7f6d4689e640 1 -- start start 2024-04-25T08:32:12.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.167+0000 7f6d4689e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6d40166610 0x7f6d40168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.167+0000 7f6d4689e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d400d2060 con 0x7f6d400d33d0 2024-04-25T08:32:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.167+0000 7f6d4689e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d400d2330 con 0x7f6d400d0320 2024-04-25T08:32:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.167+0000 7f6d4689e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d400d2610 con 0x7f6d40166610 2024-04-25T08:32:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.167+0000 7f6d44e3b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6d40166610 0x7f6d40168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.167+0000 7f6d3f7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6d400d33d0 0x7f6d400d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54032/0 (socket says 172.21.15.38:54032) 2024-04-25T08:32:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.167+0000 7f6d3f7fe640 1 -- 172.21.15.38:0/3979631973 learned_addr learned my addr 172.21.15.38:0/3979631973 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.167+0000 7f6d44e3b640 1 -- 172.21.15.38:0/3979631973 >> v1:172.21.15.107:6789/0 conn(0x7f6d400d0320 legacy=0x7f6d400d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.167+0000 7f6d44e3b640 1 -- 172.21.15.38:0/3979631973 >> v1:172.21.15.38:6789/0 conn(0x7f6d400d33d0 legacy=0x7f6d400d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.167+0000 7f6d44e3b640 1 -- 172.21.15.38:0/3979631973 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d400d28f0 con 0x7f6d40166610 2024-04-25T08:32:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.168+0000 7f6d44e3b640 1 --2- 172.21.15.38:0/3979631973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40166610 0x7f6d40168a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009d90 tx=0x7f6d3802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a2c6f94ee09c65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.168+0000 7f6d3effd640 1 -- 172.21.15.38:0/3979631973 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38007740 con 0x7f6d40166610 2024-04-25T08:32:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.168+0000 7f6d3effd640 1 -- 172.21.15.38:0/3979631973 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d380078e0 con 0x7f6d40166610 2024-04-25T08:32:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.168+0000 7f6d3effd640 1 -- 172.21.15.38:0/3979631973 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38007cb0 con 0x7f6d40166610 2024-04-25T08:32:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.169+0000 7f6d4689e640 1 -- 172.21.15.38:0/3979631973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40166610 msgr2=0x7f6d40168a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.169+0000 7f6d4689e640 1 --2- 172.21.15.38:0/3979631973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40166610 0x7f6d40168a00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009d90 tx=0x7f6d3802a730 comp rx=0 tx=0).stop 2024-04-25T08:32:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.169+0000 7f6d4689e640 1 -- 172.21.15.38:0/3979631973 shutdown_connections 2024-04-25T08:32:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.169+0000 7f6d4689e640 1 --2- 172.21.15.38:0/3979631973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40166610 0x7f6d40168a00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.169+0000 7f6d4689e640 1 -- 172.21.15.38:0/3979631973 >> 172.21.15.38:0/3979631973 conn(0x7f6d400668f0 msgr2=0x7f6d40066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.169+0000 7f6d4689e640 1 -- 172.21.15.38:0/3979631973 shutdown_connections 2024-04-25T08:32:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.169+0000 7f6d4689e640 1 -- 172.21.15.38:0/3979631973 wait complete. 2024-04-25T08:32:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.170+0000 7f6d4689e640 1 Processor -- start 2024-04-25T08:32:12.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.170+0000 7f6d4689e640 1 -- start start 2024-04-25T08:32:12.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d4689e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d400d0320 0x7f6d4014e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d4689e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d400d33d0 0x7f6d4015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d4689e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40166610 0x7f6d40157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d4689e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d40161210 con 0x7f6d40166610 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d4689e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400d2eb0 con 0x7f6d400d33d0 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d4689e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d40161390 con 0x7f6d400d0320 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d44e3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40166610 0x7f6d40157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d44e3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40166610 0x7f6d40157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37512/0 (socket says 172.21.15.38:37512) 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d44e3b640 1 -- 172.21.15.38:0/4052481005 learned_addr learned my addr 172.21.15.38:0/4052481005 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d3f7fe640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d400d33d0 0x7f6d4015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d3ffff640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d400d0320 0x7f6d4014e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d44e3b640 1 -- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d400d0320 msgr2=0x7f6d4014e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d44e3b640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d400d0320 0x7f6d4014e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d44e3b640 1 -- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d400d33d0 msgr2=0x7f6d4015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d44e3b640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d400d33d0 0x7f6d4015ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d44e3b640 1 -- 172.21.15.38:0/4052481005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d400b51a0 con 0x7f6d40166610 2024-04-25T08:32:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.171+0000 7f6d44e3b640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40166610 0x7f6d40157690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009d90 tx=0x7f6d38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.172+0000 7f6d3cff9640 1 -- 172.21.15.38:0/4052481005 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3803f070 con 0x7f6d40166610 2024-04-25T08:32:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.172+0000 7f6d3cff9640 1 -- 172.21.15.38:0/4052481005 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d38031c90 con 0x7f6d40166610 2024-04-25T08:32:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.172+0000 7f6d4689e640 1 -- 172.21.15.38:0/4052481005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d400b5480 con 0x7f6d40166610 2024-04-25T08:32:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.172+0000 7f6d3cff9640 1 -- 172.21.15.38:0/4052481005 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38043630 con 0x7f6d40166610 2024-04-25T08:32:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.172+0000 7f6d4689e640 1 -- 172.21.15.38:0/4052481005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4015f360 con 0x7f6d40166610 2024-04-25T08:32:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.173+0000 7f6d4689e640 1 -- 172.21.15.38:0/4052481005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d400b59a0 con 0x7f6d40166610 2024-04-25T08:32:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.173+0000 7f6d3cff9640 1 -- 172.21.15.38:0/4052481005 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d3803b410 con 0x7f6d40166610 2024-04-25T08:32:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.174+0000 7f6d3cff9640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d2006b4d0 0x7f6d2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.174+0000 7f6d3cff9640 1 -- 172.21.15.38:0/4052481005 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6d380ab8b0 con 0x7f6d40166610 2024-04-25T08:32:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.174+0000 7f6d3ffff640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d2006b4d0 0x7f6d2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.174+0000 7f6d3ffff640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d2006b4d0 0x7f6d2006d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6d28004f60 tx=0x7f6d28007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.178+0000 7f6d3cff9640 1 -- 172.21.15.38:0/4052481005 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d400b59a0 con 0x7f6d40166610 2024-04-25T08:32:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.360+0000 7f6d4689e640 1 -- 172.21.15.38:0/4052481005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6d40068e80 con 0x7f6d40166610 2024-04-25T08:32:12.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.360+0000 7f6d3cff9640 1 -- 172.21.15.38:0/4052481005 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6d38079ef0 con 0x7f6d40166610 2024-04-25T08:32:12.357 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:12.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.362+0000 7f6d4689e640 1 -- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d2006b4d0 msgr2=0x7f6d2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.362+0000 7f6d4689e640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d2006b4d0 0x7f6d2006d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6d28004f60 tx=0x7f6d28007e30 comp rx=0 tx=0).stop 2024-04-25T08:32:12.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.362+0000 7f6d4689e640 1 -- 172.21.15.38:0/4052481005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40166610 msgr2=0x7f6d40157690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.362+0000 7f6d4689e640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40166610 0x7f6d40157690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009d90 tx=0x7f6d38004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:12.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.362+0000 7f6d4689e640 1 -- 172.21.15.38:0/4052481005 shutdown_connections 2024-04-25T08:32:12.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.362+0000 7f6d4689e640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6d40166610 0x7f6d40157690 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.362+0000 7f6d4689e640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d400d33d0 0x7f6d4015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.362+0000 7f6d4689e640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d2006b4d0 0x7f6d2006d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.362+0000 7f6d4689e640 1 --2- 172.21.15.38:0/4052481005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6d400d0320 0x7f6d4014e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.362+0000 7f6d4689e640 1 -- 172.21.15.38:0/4052481005 >> 172.21.15.38:0/4052481005 conn(0x7f6d400668f0 msgr2=0x7f6d4015ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:12.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.363+0000 7f6d4689e640 1 -- 172.21.15.38:0/4052481005 shutdown_connections 2024-04-25T08:32:12.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.363+0000 7f6d4689e640 1 -- 172.21.15.38:0/4052481005 wait complete. 2024-04-25T08:32:12.369 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:12.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.477+0000 7fdc26edb640 1 Processor -- start 2024-04-25T08:32:12.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.477+0000 7fdc26edb640 1 -- start start 2024-04-25T08:32:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc26edb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc200d0320 0x7fdc200d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc26edb640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdc200d0700 0x7fdc200d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc26edb640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc200d2330 con 0x7fdc200d1e90 2024-04-25T08:32:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc26edb640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc200d2610 con 0x7fdc200d0700 2024-04-25T08:32:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc26edb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200d28f0 con 0x7fdc200d0320 2024-04-25T08:32:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc25478640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdc200d1e90 0x7fdc200b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54048/0 (socket says 172.21.15.38:54048) 2024-04-25T08:32:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc25478640 1 -- 172.21.15.38:0/4044103454 learned_addr learned my addr 172.21.15.38:0/4044103454 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc17fff640 1 --2- 172.21.15.38:0/4044103454 >> v2:172.21.15.107:3300/0 conn(0x7fdc200d0700 0x7fdc200d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc24c77640 1 --2- 172.21.15.38:0/4044103454 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc200d0320 0x7fdc200d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc177fe640 1 -- 172.21.15.38:0/4044103454 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338744444 0 0) 0x7fdc200d2330 con 0x7fdc200d1e90 2024-04-25T08:32:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc177fe640 1 -- 172.21.15.38:0/4044103454 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc08003520 con 0x7fdc200d1e90 2024-04-25T08:32:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc177fe640 1 -- 172.21.15.38:0/4044103454 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4034296806 0 0) 0x7fdc08003520 con 0x7fdc200d1e90 2024-04-25T08:32:12.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc17fff640 1 -- 172.21.15.38:0/4044103454 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc200d0320 msgr2=0x7fdc200d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:12.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc17fff640 1 --2- 172.21.15.38:0/4044103454 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc200d0320 0x7fdc200d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc17fff640 1 -- 172.21.15.38:0/4044103454 >> v1:172.21.15.38:6789/0 conn(0x7fdc200d1e90 legacy=0x7fdc200b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.478+0000 7fdc17fff640 1 -- 172.21.15.38:0/4044103454 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc200d2bd0 con 0x7fdc200d0700 2024-04-25T08:32:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.479+0000 7fdc17fff640 1 --2- 172.21.15.38:0/4044103454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc200d0700 0x7fdc200d1950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc100097a0 tx=0x7fdc1002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9edb0608ff68c817 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.479+0000 7fdc177fe640 1 -- 172.21.15.38:0/4044103454 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10037070 con 0x7fdc200d0700 2024-04-25T08:32:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.479+0000 7fdc177fe640 1 -- 172.21.15.38:0/4044103454 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1002ac30 con 0x7fdc200d0700 2024-04-25T08:32:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.479+0000 7fdc177fe640 1 -- 172.21.15.38:0/4044103454 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180019f0 con 0x7fdc200d0700 2024-04-25T08:32:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.480+0000 7fdc26edb640 1 -- 172.21.15.38:0/4044103454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc200d0700 msgr2=0x7fdc200d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.480+0000 7fdc26edb640 1 --2- 172.21.15.38:0/4044103454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc200d0700 0x7fdc200d1950 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdc100097a0 tx=0x7fdc1002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.480+0000 7fdc26edb640 1 -- 172.21.15.38:0/4044103454 shutdown_connections 2024-04-25T08:32:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.480+0000 7fdc26edb640 1 --2- 172.21.15.38:0/4044103454 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc200d0700 0x7fdc200d1950 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.480+0000 7fdc26edb640 1 --2- 172.21.15.38:0/4044103454 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc200d0320 0x7fdc200d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.480+0000 7fdc26edb640 1 -- 172.21.15.38:0/4044103454 >> 172.21.15.38:0/4044103454 conn(0x7fdc200668f0 msgr2=0x7fdc20066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.480+0000 7fdc26edb640 1 -- 172.21.15.38:0/4044103454 shutdown_connections 2024-04-25T08:32:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.480+0000 7fdc26edb640 1 -- 172.21.15.38:0/4044103454 wait complete. 2024-04-25T08:32:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.481+0000 7fdc26edb640 1 Processor -- start 2024-04-25T08:32:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.481+0000 7fdc26edb640 1 -- start start 2024-04-25T08:32:12.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc26edb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc200d0320 0x7fdc200b1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc26edb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc200d0700 0x7fdc200b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc26edb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc200d1e90 0x7fdc200b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc26edb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc200d0b40 con 0x7fdc200d0700 2024-04-25T08:32:12.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc26edb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc20161210 con 0x7fdc200d0320 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc26edb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2014a240 con 0x7fdc200d1e90 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc17fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc200d0700 0x7fdc200b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc17fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc200d0700 0x7fdc200b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37528/0 (socket says 172.21.15.38:37528) 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc17fff640 1 -- 172.21.15.38:0/654712271 learned_addr learned my addr 172.21.15.38:0/654712271 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc25478640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc200d1e90 0x7fdc200b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc24c77640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc200d0320 0x7fdc200b1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc17fff640 1 -- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc200d1e90 msgr2=0x7fdc200b5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc17fff640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc200d1e90 0x7fdc200b5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc17fff640 1 -- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc200d0320 msgr2=0x7fdc200b1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc17fff640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc200d0320 0x7fdc200b1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc17fff640 1 -- 172.21.15.38:0/654712271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2015d4f0 con 0x7fdc200d0700 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc25478640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc200d1e90 0x7fdc200b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:12.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.482+0000 7fdc24c77640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc200d0320 0x7fdc200b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.483+0000 7fdc17fff640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc200d0700 0x7fdc200b4cd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdc10035040 tx=0x7fdc100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.483+0000 7fdc157fa640 1 -- 172.21.15.38:0/654712271 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10041020 con 0x7fdc200d0700 2024-04-25T08:32:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.483+0000 7fdc157fa640 1 -- 172.21.15.38:0/654712271 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1003d620 con 0x7fdc200d0700 2024-04-25T08:32:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.483+0000 7fdc26edb640 1 -- 172.21.15.38:0/654712271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc200b5890 con 0x7fdc200d0700 2024-04-25T08:32:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.483+0000 7fdc157fa640 1 -- 172.21.15.38:0/654712271 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc10037040 con 0x7fdc200d0700 2024-04-25T08:32:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.483+0000 7fdc26edb640 1 -- 172.21.15.38:0/654712271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc20149bf0 con 0x7fdc200d0700 2024-04-25T08:32:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.484+0000 7fdc26edb640 1 -- 172.21.15.38:0/654712271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc20068e80 con 0x7fdc200d0700 2024-04-25T08:32:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.484+0000 7fdc157fa640 1 -- 172.21.15.38:0/654712271 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc10044020 con 0x7fdc200d0700 2024-04-25T08:32:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.485+0000 7fdc157fa640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdc0c06b4d0 0x7fdc0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.485+0000 7fdc157fa640 1 -- 172.21.15.38:0/654712271 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdc100aaaa0 con 0x7fdc200d0700 2024-04-25T08:32:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.485+0000 7fdc24c77640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdc0c06b4d0 0x7fdc0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.485+0000 7fdc24c77640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdc0c06b4d0 0x7fdc0c06d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009b90 tx=0x7fdc1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.489+0000 7fdc157fa640 1 -- 172.21.15.38:0/654712271 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc1004b030 con 0x7fdc200d0700 2024-04-25T08:32:12.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.671+0000 7fdc26edb640 1 -- 172.21.15.38:0/654712271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdc200b3050 con 0x7fdc200d0700 2024-04-25T08:32:12.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.671+0000 7fdc157fa640 1 -- 172.21.15.38:0/654712271 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdc2015d4f0 con 0x7fdc200d0700 2024-04-25T08:32:12.668 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:12.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.673+0000 7fdc26edb640 1 -- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdc0c06b4d0 msgr2=0x7fdc0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.673+0000 7fdc26edb640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdc0c06b4d0 0x7fdc0c06d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009b90 tx=0x7fdc1c008040 comp rx=0 tx=0).stop 2024-04-25T08:32:12.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.673+0000 7fdc26edb640 1 -- 172.21.15.38:0/654712271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc200d0700 msgr2=0x7fdc200b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.673+0000 7fdc26edb640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc200d0700 0x7fdc200b4cd0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fdc10035040 tx=0x7fdc100046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.674+0000 7fdc26edb640 1 -- 172.21.15.38:0/654712271 shutdown_connections 2024-04-25T08:32:12.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.674+0000 7fdc26edb640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdc0c06b4d0 0x7fdc0c06d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.674+0000 7fdc26edb640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc200d1e90 0x7fdc200b5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.674+0000 7fdc26edb640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc200d0700 0x7fdc200b4cd0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.674+0000 7fdc26edb640 1 --2- 172.21.15.38:0/654712271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc200d0320 0x7fdc200b1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.674+0000 7fdc26edb640 1 -- 172.21.15.38:0/654712271 >> 172.21.15.38:0/654712271 conn(0x7fdc200668f0 msgr2=0x7fdc20073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:12.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.674+0000 7fdc26edb640 1 -- 172.21.15.38:0/654712271 shutdown_connections 2024-04-25T08:32:12.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.674+0000 7fdc26edb640 1 -- 172.21.15.38:0/654712271 wait complete. 2024-04-25T08:32:12.681 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.786+0000 7ff5ef109640 1 Processor -- start 2024-04-25T08:32:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.786+0000 7ff5ef109640 1 -- start start 2024-04-25T08:32:12.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5ef109640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff5e815df50 0x7ff5e80ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5ef109640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e80ce2e0 0x7ff5e80d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5ef109640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e80ceff0 con 0x7ff5e806c110 2024-04-25T08:32:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5ef109640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e80cf2d0 con 0x7ff5e815df50 2024-04-25T08:32:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5ef109640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80d1f70 con 0x7ff5e80ce2e0 2024-04-25T08:32:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5e7fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff5e815df50 0x7ff5e80ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5e7fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff5e815df50 0x7ff5e80ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37530/0 (socket says 172.21.15.38:37530) 2024-04-25T08:32:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5e7fff640 1 -- 172.21.15.38:0/2695605296 learned_addr learned my addr 172.21.15.38:0/2695605296 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5ed6a6640 1 --2- 172.21.15.38:0/2695605296 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e80ce2e0 0x7ff5e80d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5e7fff640 1 -- 172.21.15.38:0/2695605296 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e80ce2e0 msgr2=0x7ff5e80d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5e7fff640 1 --2- 172.21.15.38:0/2695605296 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e80ce2e0 0x7ff5e80d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5e7fff640 1 -- 172.21.15.38:0/2695605296 >> v1:172.21.15.107:6789/0 conn(0x7ff5e806c110 legacy=0x7ff5e806c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5e7fff640 1 -- 172.21.15.38:0/2695605296 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e80d2250 con 0x7ff5e815df50 2024-04-25T08:32:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.787+0000 7ff5ed6a6640 1 --2- 172.21.15.38:0/2695605296 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e80ce2e0 0x7ff5e80d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.788+0000 7ff5e7fff640 1 --2- 172.21.15.38:0/2695605296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5e815df50 0x7ff5e80ce7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4009870 tx=0x7ff5d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e7f80e32ff33f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.788+0000 7ff5e77fe640 1 -- 172.21.15.38:0/2695605296 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4037070 con 0x7ff5e815df50 2024-04-25T08:32:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.788+0000 7ff5e77fe640 1 -- 172.21.15.38:0/2695605296 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d4032440 con 0x7ff5e815df50 2024-04-25T08:32:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.788+0000 7ff5e77fe640 1 -- 172.21.15.38:0/2695605296 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d4032810 con 0x7ff5e815df50 2024-04-25T08:32:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.789+0000 7ff5ef109640 1 -- 172.21.15.38:0/2695605296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5e815df50 msgr2=0x7ff5e80ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.789+0000 7ff5ef109640 1 --2- 172.21.15.38:0/2695605296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5e815df50 0x7ff5e80ce7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff5d4009870 tx=0x7ff5d402a270 comp rx=0 tx=0).stop 2024-04-25T08:32:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.789+0000 7ff5ef109640 1 -- 172.21.15.38:0/2695605296 shutdown_connections 2024-04-25T08:32:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.789+0000 7ff5ef109640 1 --2- 172.21.15.38:0/2695605296 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e80ce2e0 0x7ff5e80d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.789+0000 7ff5ef109640 1 --2- 172.21.15.38:0/2695605296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5e815df50 0x7ff5e80ce7a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.789+0000 7ff5ef109640 1 -- 172.21.15.38:0/2695605296 >> 172.21.15.38:0/2695605296 conn(0x7ff5e80d3fb0 msgr2=0x7ff5e80d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.789+0000 7ff5ef109640 1 -- 172.21.15.38:0/2695605296 shutdown_connections 2024-04-25T08:32:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.789+0000 7ff5ef109640 1 -- 172.21.15.38:0/2695605296 wait complete. 2024-04-25T08:32:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.790+0000 7ff5ef109640 1 Processor -- start 2024-04-25T08:32:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.790+0000 7ff5ef109640 1 -- start start 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ef109640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e806c110 0x7ff5e80cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ef109640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e80ce2e0 0x7ff5e8151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ef109640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5e815df50 0x7ff5e814ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ef109640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80d2f90 con 0x7ff5e815df50 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ef109640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80cece0 con 0x7ff5e806c110 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ef109640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80d3110 con 0x7ff5e80ce2e0 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ed6a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5e815df50 0x7ff5e814ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ed6a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5e815df50 0x7ff5e814ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37536/0 (socket says 172.21.15.38:37536) 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ed6a6640 1 -- 172.21.15.38:0/3470856278 learned_addr learned my addr 172.21.15.38:0/3470856278 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5e7fff640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e80ce2e0 0x7ff5e8151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ecea5640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e806c110 0x7ff5e80cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ed6a6640 1 -- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e80ce2e0 msgr2=0x7ff5e8151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ed6a6640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e80ce2e0 0x7ff5e8151250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ed6a6640 1 -- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e806c110 msgr2=0x7ff5e80cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ed6a6640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e806c110 0x7ff5e80cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ed6a6640 1 -- 172.21.15.38:0/3470856278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e814e300 con 0x7ff5e815df50 2024-04-25T08:32:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.791+0000 7ff5ed6a6640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5e815df50 0x7ff5e814ddc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0007e70 tx=0x7ff5e000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.792+0000 7ff5e57fa640 1 -- 172.21.15.38:0/3470856278 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e001a070 con 0x7ff5e815df50 2024-04-25T08:32:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.792+0000 7ff5e57fa640 1 -- 172.21.15.38:0/3470856278 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e0015600 con 0x7ff5e815df50 2024-04-25T08:32:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.792+0000 7ff5ef109640 1 -- 172.21.15.38:0/3470856278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e80cd8e0 con 0x7ff5e815df50 2024-04-25T08:32:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.792+0000 7ff5e57fa640 1 -- 172.21.15.38:0/3470856278 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00159f0 con 0x7ff5e815df50 2024-04-25T08:32:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.792+0000 7ff5ef109640 1 -- 172.21.15.38:0/3470856278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e81f4040 con 0x7ff5e815df50 2024-04-25T08:32:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.793+0000 7ff5ef109640 1 -- 172.21.15.38:0/3470856278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e80c71a0 con 0x7ff5e815df50 2024-04-25T08:32:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.793+0000 7ff5e57fa640 1 -- 172.21.15.38:0/3470856278 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5e001d070 con 0x7ff5e815df50 2024-04-25T08:32:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.794+0000 7ff5e57fa640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.794+0000 7ff5e57fa640 1 -- 172.21.15.38:0/3470856278 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff5e008a7e0 con 0x7ff5e815df50 2024-04-25T08:32:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.794+0000 7ff5ecea5640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.794+0000 7ff5ecea5640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8151200 tx=0x7ff5dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:12.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.798+0000 7ff5e57fa640 1 -- 172.21.15.38:0/3470856278 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5e80c71a0 con 0x7ff5e815df50 2024-04-25T08:32:12.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.980+0000 7ff5ef109640 1 -- 172.21.15.38:0/3470856278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5e8068e80 con 0x7ff5e815df50 2024-04-25T08:32:12.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.982+0000 7ff5e57fa640 1 -- 172.21.15.38:0/3470856278 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7ff5e814e300 con 0x7ff5e815df50 2024-04-25T08:32:12.978 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:12.978 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:12.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.984+0000 7ff5ef109640 1 -- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff5cc06b4d0 msgr2=0x7ff5cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.984+0000 7ff5ef109640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8151200 tx=0x7ff5dc009040 comp rx=0 tx=0).stop 2024-04-25T08:32:12.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.984+0000 7ff5ef109640 1 -- 172.21.15.38:0/3470856278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5e815df50 msgr2=0x7ff5e814ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:12.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.984+0000 7ff5ef109640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5e815df50 0x7ff5e814ddc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0007e70 tx=0x7ff5e000d4a0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.984+0000 7ff5ef109640 1 -- 172.21.15.38:0/3470856278 shutdown_connections 2024-04-25T08:32:12.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.984+0000 7ff5ef109640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5e815df50 0x7ff5e814ddc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.984+0000 7ff5ef109640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.984+0000 7ff5ef109640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5e80ce2e0 0x7ff5e8151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.984+0000 7ff5ef109640 1 --2- 172.21.15.38:0/3470856278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5e806c110 0x7ff5e80cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:12.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.984+0000 7ff5ef109640 1 -- 172.21.15.38:0/3470856278 >> 172.21.15.38:0/3470856278 conn(0x7ff5e80d3fb0 msgr2=0x7ff5e80b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:12.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.984+0000 7ff5ef109640 1 -- 172.21.15.38:0/3470856278 shutdown_connections 2024-04-25T08:32:12.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:12.984+0000 7ff5ef109640 1 -- 172.21.15.38:0/3470856278 wait complete. 2024-04-25T08:32:12.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:12.992 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:13.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.099+0000 7f77ad9d5640 1 Processor -- start 2024-04-25T08:32:13.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.099+0000 7f77ad9d5640 1 -- start start 2024-04-25T08:32:13.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.100+0000 7f77ad9d5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f77a80b1d60 0x7f77a80cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.100+0000 7f77ad9d5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77a80cff10 0x7f77a8161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.100+0000 7f77ad9d5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a815ef40 con 0x7f77a8161c50 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.100+0000 7f77ad9d5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a806abe0 con 0x7f77a80b1d60 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.100+0000 7f77ad9d5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a806b480 con 0x7f77a80cff10 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.100+0000 7f77a6ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f77a80b1d60 0x7f77a80cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.100+0000 7f77a6ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f77a80b1d60 0x7f77a80cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37548/0 (socket says 172.21.15.38:37548) 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.100+0000 7f77a6ffd640 1 -- 172.21.15.38:0/2028133853 learned_addr learned my addr 172.21.15.38:0/2028133853 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.100+0000 7f77a67fc640 1 --2- 172.21.15.38:0/2028133853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77a80cff10 0x7f77a8161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.101+0000 7f77a6ffd640 1 -- 172.21.15.38:0/2028133853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77a80cff10 msgr2=0x7f77a8161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.101+0000 7f77a6ffd640 1 --2- 172.21.15.38:0/2028133853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77a80cff10 0x7f77a8161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.101+0000 7f77a6ffd640 1 -- 172.21.15.38:0/2028133853 >> v1:172.21.15.107:6789/0 conn(0x7f77a8161c50 legacy=0x7f77a80cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.101+0000 7f77a6ffd640 1 -- 172.21.15.38:0/2028133853 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a806ae30 con 0x7f77a80b1d60 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.101+0000 7f77a6ffd640 1 --2- 172.21.15.38:0/2028133853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80b1d60 0x7f77a80cf9d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7798009870 tx=0x7f779802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684cdf92fca28cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:13.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.101+0000 7f77a5ffb640 1 -- 172.21.15.38:0/2028133853 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798037070 con 0x7f77a80b1d60 2024-04-25T08:32:13.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.101+0000 7f77a5ffb640 1 -- 172.21.15.38:0/2028133853 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798032440 con 0x7f77a80b1d60 2024-04-25T08:32:13.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.101+0000 7f77a5ffb640 1 -- 172.21.15.38:0/2028133853 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798032810 con 0x7f77a80b1d60 2024-04-25T08:32:13.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.102+0000 7f77ad9d5640 1 -- 172.21.15.38:0/2028133853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80b1d60 msgr2=0x7f77a80cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.102+0000 7f77ad9d5640 1 --2- 172.21.15.38:0/2028133853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80b1d60 0x7f77a80cf9d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7798009870 tx=0x7f779802a210 comp rx=0 tx=0).stop 2024-04-25T08:32:13.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.102+0000 7f77ad9d5640 1 -- 172.21.15.38:0/2028133853 shutdown_connections 2024-04-25T08:32:13.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.102+0000 7f77ad9d5640 1 --2- 172.21.15.38:0/2028133853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77a80cff10 0x7f77a8161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.102+0000 7f77ad9d5640 1 --2- 172.21.15.38:0/2028133853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80b1d60 0x7f77a80cf9d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.102+0000 7f77ad9d5640 1 -- 172.21.15.38:0/2028133853 >> 172.21.15.38:0/2028133853 conn(0x7f77a80d06b0 msgr2=0x7f77a80d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:13.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.102+0000 7f77ad9d5640 1 -- 172.21.15.38:0/2028133853 shutdown_connections 2024-04-25T08:32:13.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.102+0000 7f77ad9d5640 1 -- 172.21.15.38:0/2028133853 wait complete. 2024-04-25T08:32:13.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.103+0000 7f77ad9d5640 1 Processor -- start 2024-04-25T08:32:13.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.103+0000 7f77ad9d5640 1 -- start start 2024-04-25T08:32:13.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.104+0000 7f77ad9d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80b1d60 0x7f77a81548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.104+0000 7f77ad9d5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77a80cff10 0x7f77a81e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.104+0000 7f77ad9d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a8161c50 0x7f77a8153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.104+0000 7f77ad9d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a806b910 con 0x7f77a80b1d60 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.104+0000 7f77ad9d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a80d0370 con 0x7f77a8161c50 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.104+0000 7f77ad9d5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a806ba90 con 0x7f77a80cff10 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.104+0000 7f77a6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80b1d60 0x7f77a81548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.104+0000 7f77a6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80b1d60 0x7f77a81548d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37554/0 (socket says 172.21.15.38:37554) 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.104+0000 7f77a6ffd640 1 -- 172.21.15.38:0/4020614775 learned_addr learned my addr 172.21.15.38:0/4020614775 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f77a6ffd640 1 -- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77a80cff10 msgr2=0x7f77a81e8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f77a6ffd640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77a80cff10 0x7f77a81e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f77a6ffd640 1 -- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a8161c50 msgr2=0x7f77a8153380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f77a77fe640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a8161c50 0x7f77a8153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f77a6ffd640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a8161c50 0x7f77a8153380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f77a6ffd640 1 -- 172.21.15.38:0/4020614775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a81e9500 con 0x7f77a80b1d60 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f77a77fe640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a8161c50 0x7f77a8153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f77a6ffd640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80b1d60 0x7f77a81548d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7798000c00 tx=0x7f77980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f779ffff640 1 -- 172.21.15.38:0/4020614775 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798041020 con 0x7f77a80b1d60 2024-04-25T08:32:13.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f779ffff640 1 -- 172.21.15.38:0/4020614775 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77980306c0 con 0x7f77a80b1d60 2024-04-25T08:32:13.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f77ad9d5640 1 -- 172.21.15.38:0/4020614775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a81e97e0 con 0x7f77a80b1d60 2024-04-25T08:32:13.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f779ffff640 1 -- 172.21.15.38:0/4020614775 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798037040 con 0x7f77a80b1d60 2024-04-25T08:32:13.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.105+0000 7f77ad9d5640 1 -- 172.21.15.38:0/4020614775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a81e9da0 con 0x7f77a80b1d60 2024-04-25T08:32:13.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.106+0000 7f77ad9d5640 1 -- 172.21.15.38:0/4020614775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a8068e80 con 0x7f77a80b1d60 2024-04-25T08:32:13.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.106+0000 7f779ffff640 1 -- 172.21.15.38:0/4020614775 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7798044020 con 0x7f77a80b1d60 2024-04-25T08:32:13.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.107+0000 7f779ffff640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f778806b4d0 0x7f778806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.107+0000 7f779ffff640 1 -- 172.21.15.38:0/4020614775 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f77980aad60 con 0x7f77a80b1d60 2024-04-25T08:32:13.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.107+0000 7f77a67fc640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f778806b4d0 0x7f778806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.107+0000 7f77a67fc640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f778806b4d0 0x7f778806d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7790006fd0 tx=0x7f7790008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:13.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.111+0000 7f779ffff640 1 -- 172.21.15.38:0/4020614775 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f779804b030 con 0x7f77a80b1d60 2024-04-25T08:32:13.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.290+0000 7f77ad9d5640 1 -- 172.21.15.38:0/4020614775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77a806c9e0 con 0x7f77a80b1d60 2024-04-25T08:32:13.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.290+0000 7f779ffff640 1 -- 172.21.15.38:0/4020614775 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77980798a0 con 0x7f77a80b1d60 2024-04-25T08:32:13.287 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:13.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.292+0000 7f77ad9d5640 1 -- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f778806b4d0 msgr2=0x7f778806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.292+0000 7f77ad9d5640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f778806b4d0 0x7f778806d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7790006fd0 tx=0x7f7790008040 comp rx=0 tx=0).stop 2024-04-25T08:32:13.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.292+0000 7f77ad9d5640 1 -- 172.21.15.38:0/4020614775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80b1d60 msgr2=0x7f77a81548d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.292+0000 7f77ad9d5640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80b1d60 0x7f77a81548d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7798000c00 tx=0x7f77980046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.292+0000 7f77ad9d5640 1 -- 172.21.15.38:0/4020614775 shutdown_connections 2024-04-25T08:32:13.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.292+0000 7f77ad9d5640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f77a8161c50 0x7f77a8153380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.292+0000 7f77ad9d5640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f778806b4d0 0x7f778806d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.292+0000 7f77ad9d5640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f77a80cff10 0x7f77a81e8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.292+0000 7f77ad9d5640 1 --2- 172.21.15.38:0/4020614775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77a80b1d60 0x7f77a81548d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.292+0000 7f77ad9d5640 1 -- 172.21.15.38:0/4020614775 >> 172.21.15.38:0/4020614775 conn(0x7f77a80d06b0 msgr2=0x7f77a8160a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:13.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.292+0000 7f77ad9d5640 1 -- 172.21.15.38:0/4020614775 shutdown_connections 2024-04-25T08:32:13.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.292+0000 7f77ad9d5640 1 -- 172.21.15.38:0/4020614775 wait complete. 2024-04-25T08:32:13.299 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:13.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.407+0000 7f91c501f640 1 Processor -- start 2024-04-25T08:32:13.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.407+0000 7f91c501f640 1 -- start start 2024-04-25T08:32:13.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.408+0000 7f91c501f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f91c00b1d60 0x7f91c00cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.408+0000 7f91c501f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c015ef40 con 0x7f91c0161e20 2024-04-25T08:32:13.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.408+0000 7f91c501f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c006ae30 con 0x7f91c00cff10 2024-04-25T08:32:13.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.408+0000 7f91c501f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c006abe0 con 0x7f91c00b1d60 2024-04-25T08:32:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.408+0000 7f91bed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f91c00b1d60 0x7f91c00cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.408+0000 7f91bf59e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f91c0161e20 0x7f91c015d330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54054/0 (socket says 172.21.15.38:54054) 2024-04-25T08:32:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.408+0000 7f91bf59e640 1 -- 172.21.15.38:0/712039962 learned_addr learned my addr 172.21.15.38:0/712039962 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.408+0000 7f91bed9d640 1 -- 172.21.15.38:0/712039962 >> v1:172.21.15.107:6789/0 conn(0x7f91c00cff10 legacy=0x7f91c0161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.408+0000 7f91bed9d640 1 -- 172.21.15.38:0/712039962 >> v1:172.21.15.38:6789/0 conn(0x7f91c0161e20 legacy=0x7f91c015d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.408+0000 7f91bed9d640 1 -- 172.21.15.38:0/712039962 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c006b110 con 0x7f91c00b1d60 2024-04-25T08:32:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.408+0000 7f91be59c640 1 -- 172.21.15.38:0/712039962 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829700423 0 0) 0x7f91c015ef40 con 0x7f91c0161e20 2024-04-25T08:32:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.409+0000 7f91bed9d640 1 --2- 172.21.15.38:0/712039962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91c00b1d60 0x7f91c00cf9d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f91b00097a0 tx=0x7f91b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1692c91923564508 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.409+0000 7f91be59c640 1 -- 172.21.15.38:0/712039962 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0037070 con 0x7f91c00b1d60 2024-04-25T08:32:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.409+0000 7f91be59c640 1 -- 172.21.15.38:0/712039962 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b0032440 con 0x7f91c00b1d60 2024-04-25T08:32:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.409+0000 7f91be59c640 1 -- 172.21.15.38:0/712039962 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0032810 con 0x7f91c00b1d60 2024-04-25T08:32:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.409+0000 7f91c501f640 1 -- 172.21.15.38:0/712039962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91c00b1d60 msgr2=0x7f91c00cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.409+0000 7f91c501f640 1 --2- 172.21.15.38:0/712039962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91c00b1d60 0x7f91c00cf9d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f91b00097a0 tx=0x7f91b002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.410+0000 7f91c501f640 1 -- 172.21.15.38:0/712039962 shutdown_connections 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.410+0000 7f91c501f640 1 --2- 172.21.15.38:0/712039962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91c00b1d60 0x7f91c00cf9d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.410+0000 7f91c501f640 1 -- 172.21.15.38:0/712039962 >> 172.21.15.38:0/712039962 conn(0x7f91c00d06b0 msgr2=0x7f91c00d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.410+0000 7f91c501f640 1 -- 172.21.15.38:0/712039962 shutdown_connections 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.410+0000 7f91c501f640 1 -- 172.21.15.38:0/712039962 wait complete. 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.411+0000 7f91c501f640 1 Processor -- start 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.411+0000 7f91c501f640 1 -- start start 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91c501f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91c00b1d60 0x7f91c01f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91c501f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91c00cff10 0x7f91c0145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91c501f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0161e20 0x7f91c00adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91c501f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c006b910 con 0x7f91c00b1d60 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91c501f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c006b480 con 0x7f91c0161e20 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91c501f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c006ba90 con 0x7f91c00cff10 2024-04-25T08:32:13.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91bed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91c00b1d60 0x7f91c01f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91bed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91c00b1d60 0x7f91c01f2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37572/0 (socket says 172.21.15.38:37572) 2024-04-25T08:32:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91bed9d640 1 -- 172.21.15.38:0/2843858285 learned_addr learned my addr 172.21.15.38:0/2843858285 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91bf59e640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0161e20 0x7f91c00adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91b659c640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91c00cff10 0x7f91c0145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91bed9d640 1 -- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91c00cff10 msgr2=0x7f91c0145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91bed9d640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91c00cff10 0x7f91c0145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91bed9d640 1 -- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0161e20 msgr2=0x7f91c00adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91bed9d640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0161e20 0x7f91c00adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91bed9d640 1 -- 172.21.15.38:0/2843858285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c0146430 con 0x7f91c00b1d60 2024-04-25T08:32:13.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.412+0000 7f91bed9d640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91c00b1d60 0x7f91c01f2f60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f91b00098c0 tx=0x7f91b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:13.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.413+0000 7f91b7fff640 1 -- 172.21.15.38:0/2843858285 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0041070 con 0x7f91c00b1d60 2024-04-25T08:32:13.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.413+0000 7f91b7fff640 1 -- 172.21.15.38:0/2843858285 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b003c870 con 0x7f91c00b1d60 2024-04-25T08:32:13.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.413+0000 7f91c501f640 1 -- 172.21.15.38:0/2843858285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c0156a70 con 0x7f91c00b1d60 2024-04-25T08:32:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.413+0000 7f91b7fff640 1 -- 172.21.15.38:0/2843858285 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b0037040 con 0x7f91c00b1d60 2024-04-25T08:32:13.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.413+0000 7f91c501f640 1 -- 172.21.15.38:0/2843858285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c0157030 con 0x7f91c00b1d60 2024-04-25T08:32:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.414+0000 7f91c501f640 1 -- 172.21.15.38:0/2843858285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c0068e80 con 0x7f91c00b1d60 2024-04-25T08:32:13.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.414+0000 7f91b7fff640 1 -- 172.21.15.38:0/2843858285 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91b0044070 con 0x7f91c00b1d60 2024-04-25T08:32:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.415+0000 7f91b7fff640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f91a006b4d0 0x7f91a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.415+0000 7f91b7fff640 1 -- 172.21.15.38:0/2843858285 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f91b00ab140 con 0x7f91c00b1d60 2024-04-25T08:32:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.415+0000 7f91b659c640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f91a006b4d0 0x7f91a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.415+0000 7f91b659c640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f91a006b4d0 0x7f91a006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f91c015d7f0 tx=0x7f91a8009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.419+0000 7f91b7fff640 1 -- 172.21.15.38:0/2843858285 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91b00797e0 con 0x7f91c00b1d60 2024-04-25T08:32:13.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.598+0000 7f91c501f640 1 -- 172.21.15.38:0/2843858285 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f91c006c9e0 con 0x7f91c00b1d60 2024-04-25T08:32:13.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.598+0000 7f91b7fff640 1 -- 172.21.15.38:0/2843858285 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f91b007d880 con 0x7f91c00b1d60 2024-04-25T08:32:13.595 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:13.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.600+0000 7f91c501f640 1 -- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f91a006b4d0 msgr2=0x7f91a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.600+0000 7f91c501f640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f91a006b4d0 0x7f91a006d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f91c015d7f0 tx=0x7f91a8009060 comp rx=0 tx=0).stop 2024-04-25T08:32:13.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.600+0000 7f91c501f640 1 -- 172.21.15.38:0/2843858285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91c00b1d60 msgr2=0x7f91c01f2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.600+0000 7f91c501f640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91c00b1d60 0x7f91c01f2f60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f91b00098c0 tx=0x7f91b00046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.600+0000 7f91c501f640 1 -- 172.21.15.38:0/2843858285 shutdown_connections 2024-04-25T08:32:13.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.600+0000 7f91c501f640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f91c0161e20 0x7f91c00adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.600+0000 7f91c501f640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f91a006b4d0 0x7f91a006d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.600+0000 7f91c501f640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f91c00cff10 0x7f91c0145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.600+0000 7f91c501f640 1 --2- 172.21.15.38:0/2843858285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f91c00b1d60 0x7f91c01f2f60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.600+0000 7f91c501f640 1 -- 172.21.15.38:0/2843858285 >> 172.21.15.38:0/2843858285 conn(0x7f91c00d06b0 msgr2=0x7f91c00af270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:13.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.600+0000 7f91c501f640 1 -- 172.21.15.38:0/2843858285 shutdown_connections 2024-04-25T08:32:13.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.600+0000 7f91c501f640 1 -- 172.21.15.38:0/2843858285 wait complete. 2024-04-25T08:32:13.607 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:13.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.715+0000 7fa99b21f640 1 Processor -- start 2024-04-25T08:32:13.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.715+0000 7fa99b21f640 1 -- start start 2024-04-25T08:32:13.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa99b21f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa994166610 0x7fa994168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa99b21f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9940d2060 con 0x7fa9940d0320 2024-04-25T08:32:13.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa99b21f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9940d2330 con 0x7fa9940d33d0 2024-04-25T08:32:13.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa99b21f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940d2610 con 0x7fa994166610 2024-04-25T08:32:13.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa998fbb640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa9940d0320 0x7fa9940d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54058/0 (socket says 172.21.15.38:54058) 2024-04-25T08:32:13.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa998fbb640 1 -- 172.21.15.38:0/321226154 learned_addr learned my addr 172.21.15.38:0/321226154 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:13.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9997bc640 1 --2- 172.21.15.38:0/321226154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa994166610 0x7fa994168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166320543 0 0) 0x7fa9940d2060 con 0x7fa9940d0320 2024-04-25T08:32:13.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa984003520 con 0x7fa9940d0320 2024-04-25T08:32:13.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1556527158 0 0) 0x7fa984003520 con 0x7fa9940d0320 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa994166610 msgr2=0x7fa994168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 --2- 172.21.15.38:0/321226154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa994166610 0x7fa994168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 >> v1:172.21.15.107:6789/0 conn(0x7fa9940d33d0 legacy=0x7fa9940d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9940d28f0 con 0x7fa9940d0320 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fa988001690 con 0x7fa9940d0320 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 >> v1:172.21.15.38:6789/0 conn(0x7fa9940d0320 legacy=0x7fa9940d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 --2- 172.21.15.38:0/321226154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9840039e0 0x7fa984003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 --2- 172.21.15.38:0/321226154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa984004390 0x7fa984004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 --2- 172.21.15.38:0/321226154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa994166610 0x7fa9840050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940d2610 con 0x7fa9840039e0 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98400b9f0 con 0x7fa984004390 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.716+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98400bbf0 con 0x7fa994166610 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.717+0000 7fa998fbb640 1 --2- 172.21.15.38:0/321226154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9840039e0 0x7fa984003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.717+0000 7fa993fff640 1 --2- 172.21.15.38:0/321226154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa984004390 0x7fa984004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.717+0000 7fa9997bc640 1 --2- 172.21.15.38:0/321226154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa994166610 0x7fa9840050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.717+0000 7fa998fbb640 1 -- 172.21.15.38:0/321226154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa994166610 msgr2=0x7fa9840050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.717+0000 7fa998fbb640 1 --2- 172.21.15.38:0/321226154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa994166610 0x7fa9840050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.717+0000 7fa998fbb640 1 -- 172.21.15.38:0/321226154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa984004390 msgr2=0x7fa984004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.717+0000 7fa998fbb640 1 --2- 172.21.15.38:0/321226154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa984004390 0x7fa984004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.717+0000 7fa998fbb640 1 -- 172.21.15.38:0/321226154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984003520 con 0x7fa9840039e0 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.718+0000 7fa998fbb640 1 --2- 172.21.15.38:0/321226154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9840039e0 0x7fa984003dc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa988007fd0 tx=0x7fa984005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e4bb72180acf24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.718+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988007d50 con 0x7fa9840039e0 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.718+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa988005ad0 con 0x7fa9840039e0 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.718+0000 7fa9937fe640 1 -- 172.21.15.38:0/321226154 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa988007070 con 0x7fa9840039e0 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.718+0000 7fa99b21f640 1 -- 172.21.15.38:0/321226154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9840039e0 msgr2=0x7fa984003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.718+0000 7fa99b21f640 1 --2- 172.21.15.38:0/321226154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9840039e0 0x7fa984003dc0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa988007fd0 tx=0x7fa984005830 comp rx=0 tx=0).stop 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.718+0000 7fa9997bc640 1 -- 172.21.15.38:0/321226154 reap_dead start 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.718+0000 7fa99b21f640 1 -- 172.21.15.38:0/321226154 shutdown_connections 2024-04-25T08:32:13.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.718+0000 7fa99b21f640 1 -- 172.21.15.38:0/321226154 >> 172.21.15.38:0/321226154 conn(0x7fa9940668f0 msgr2=0x7fa994066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:13.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.718+0000 7fa99b21f640 1 -- 172.21.15.38:0/321226154 shutdown_connections 2024-04-25T08:32:13.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.718+0000 7fa99b21f640 1 -- 172.21.15.38:0/321226154 wait complete. 2024-04-25T08:32:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.720+0000 7fa99b21f640 1 Processor -- start 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.720+0000 7fa99b21f640 1 -- start start 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.720+0000 7fa99b21f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9840039e0 0x7fa994151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.720+0000 7fa99b21f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9940ca9e0 0x7fa9941f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa99b21f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9940d38a0 0x7fa9940d3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa99b21f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa994161210 con 0x7fa9940d38a0 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa99b21f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9940d2eb0 con 0x7fa9840039e0 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa99b21f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa994161390 con 0x7fa9940ca9e0 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa9997bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9940d38a0 0x7fa9940d3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa9997bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9940d38a0 0x7fa9940d3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37588/0 (socket says 172.21.15.38:37588) 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa9997bc640 1 -- 172.21.15.38:0/3892560546 learned_addr learned my addr 172.21.15.38:0/3892560546 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa9997bc640 1 -- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9940ca9e0 msgr2=0x7fa9941f4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa993fff640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9940ca9e0 0x7fa9941f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa998fbb640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9840039e0 0x7fa994151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa9997bc640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9940ca9e0 0x7fa9941f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa9997bc640 1 -- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9840039e0 msgr2=0x7fa994151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa9997bc640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9840039e0 0x7fa994151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa9997bc640 1 -- 172.21.15.38:0/3892560546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9941f4a90 con 0x7fa9940d38a0 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa998fbb640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9840039e0 0x7fa994151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa9997bc640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9940d38a0 0x7fa9940d3d50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa9940675b0 tx=0x7fa98c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa9917fa640 1 -- 172.21.15.38:0/3892560546 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c015430 con 0x7fa9940d38a0 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa9917fa640 1 -- 172.21.15.38:0/3892560546 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c01d070 con 0x7fa9940d38a0 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa9917fa640 1 -- 172.21.15.38:0/3892560546 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c015830 con 0x7fa9940d38a0 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa99b21f640 1 -- 172.21.15.38:0/3892560546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9940caf30 con 0x7fa9940d38a0 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.721+0000 7fa99b21f640 1 -- 172.21.15.38:0/3892560546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9940cb4f0 con 0x7fa9940d38a0 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.722+0000 7fa99b21f640 1 -- 172.21.15.38:0/3892560546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa994068e80 con 0x7fa9940d38a0 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.723+0000 7fa9917fa640 1 -- 172.21.15.38:0/3892560546 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa98c0159d0 con 0x7fa9940d38a0 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.723+0000 7fa9917fa640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa97806b500 0x7fa97806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.723+0000 7fa9917fa640 1 -- 172.21.15.38:0/3892560546 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa98c08ada0 con 0x7fa9940d38a0 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.724+0000 7fa998fbb640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa97806b500 0x7fa97806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.724+0000 7fa998fbb640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa97806b500 0x7fa97806d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa988033000 tx=0x7fa98802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:13.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.727+0000 7fa9917fa640 1 -- 172.21.15.38:0/3892560546 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa98c010040 con 0x7fa9940d38a0 2024-04-25T08:32:13.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.907+0000 7fa99b21f640 1 -- 172.21.15.38:0/3892560546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa994067dd0 con 0x7fa9940d38a0 2024-04-25T08:32:13.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.907+0000 7fa9917fa640 1 -- 172.21.15.38:0/3892560546 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa98c0598e0 con 0x7fa9940d38a0 2024-04-25T08:32:13.904 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:13.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.909+0000 7fa99b21f640 1 -- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa97806b500 msgr2=0x7fa97806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.909+0000 7fa99b21f640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa97806b500 0x7fa97806d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa988033000 tx=0x7fa98802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:13.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.909+0000 7fa99b21f640 1 -- 172.21.15.38:0/3892560546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9940d38a0 msgr2=0x7fa9940d3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:13.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.909+0000 7fa99b21f640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9940d38a0 0x7fa9940d3d50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa9940675b0 tx=0x7fa98c00f580 comp rx=0 tx=0).stop 2024-04-25T08:32:13.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.910+0000 7fa99b21f640 1 -- 172.21.15.38:0/3892560546 shutdown_connections 2024-04-25T08:32:13.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.910+0000 7fa99b21f640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9940d38a0 0x7fa9940d3d50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.910+0000 7fa99b21f640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa97806b500 0x7fa97806d9c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.910+0000 7fa99b21f640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9940ca9e0 0x7fa9941f4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.910+0000 7fa99b21f640 1 --2- 172.21.15.38:0/3892560546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9840039e0 0x7fa994151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.910+0000 7fa99b21f640 1 -- 172.21.15.38:0/3892560546 >> 172.21.15.38:0/3892560546 conn(0x7fa9940668f0 msgr2=0x7fa9940b8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.910+0000 7fa99b21f640 1 -- 172.21.15.38:0/3892560546 shutdown_connections 2024-04-25T08:32:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:13.910+0000 7fa99b21f640 1 -- 172.21.15.38:0/3892560546 wait complete. 2024-04-25T08:32:13.917 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.024+0000 7fc107563640 1 Processor -- start 2024-04-25T08:32:14.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.024+0000 7fc107563640 1 -- start start 2024-04-25T08:32:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc107563640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc10015baf0 0x7fc1000cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc107563640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc100072660 0x7fc1000acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc107563640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1000b3250 con 0x7fc1000cfb70 2024-04-25T08:32:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc107563640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1000731d0 con 0x7fc10015baf0 2024-04-25T08:32:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc107563640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc100072c10 con 0x7fc100072660 2024-04-25T08:32:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc1052ff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc10015baf0 0x7fc1000cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc104afe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc1000cfb70 0x7fc100071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54074/0 (socket says 172.21.15.38:54074) 2024-04-25T08:32:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc104afe640 1 -- 172.21.15.38:0/3954804811 learned_addr learned my addr 172.21.15.38:0/3954804811 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc105b00640 1 --2- 172.21.15.38:0/3954804811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc100072660 0x7fc1000acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc1052ff640 1 -- 172.21.15.38:0/3954804811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc100072660 msgr2=0x7fc1000acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:14.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc1052ff640 1 --2- 172.21.15.38:0/3954804811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc100072660 0x7fc1000acb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc1052ff640 1 -- 172.21.15.38:0/3954804811 >> v1:172.21.15.38:6789/0 conn(0x7fc1000cfb70 legacy=0x7fc100071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc1052ff640 1 -- 172.21.15.38:0/3954804811 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc100072ef0 con 0x7fc10015baf0 2024-04-25T08:32:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.025+0000 7fc105b00640 1 --2- 172.21.15.38:0/3954804811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc100072660 0x7fc1000acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.026+0000 7fc1052ff640 1 --2- 172.21.15.38:0/3954804811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10015baf0 0x7fc1000cf630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80097a0 tx=0x7fc0f802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30537e8db1e5b245 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.026+0000 7fc0fffff640 1 -- 172.21.15.38:0/3954804811 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8037070 con 0x7fc10015baf0 2024-04-25T08:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.026+0000 7fc0fffff640 1 -- 172.21.15.38:0/3954804811 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f802adb0 con 0x7fc10015baf0 2024-04-25T08:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.026+0000 7fc0fffff640 1 -- 172.21.15.38:0/3954804811 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8032660 con 0x7fc10015baf0 2024-04-25T08:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.027+0000 7fc107563640 1 -- 172.21.15.38:0/3954804811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10015baf0 msgr2=0x7fc1000cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.027+0000 7fc107563640 1 --2- 172.21.15.38:0/3954804811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10015baf0 0x7fc1000cf630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc0f80097a0 tx=0x7fc0f802a060 comp rx=0 tx=0).stop 2024-04-25T08:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.027+0000 7fc107563640 1 -- 172.21.15.38:0/3954804811 shutdown_connections 2024-04-25T08:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.027+0000 7fc107563640 1 --2- 172.21.15.38:0/3954804811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc100072660 0x7fc1000acb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.027+0000 7fc107563640 1 --2- 172.21.15.38:0/3954804811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10015baf0 0x7fc1000cf630 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.027+0000 7fc107563640 1 -- 172.21.15.38:0/3954804811 >> 172.21.15.38:0/3954804811 conn(0x7fc10006adc0 msgr2=0x7fc10006b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.027+0000 7fc107563640 1 -- 172.21.15.38:0/3954804811 shutdown_connections 2024-04-25T08:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.027+0000 7fc107563640 1 -- 172.21.15.38:0/3954804811 wait complete. 2024-04-25T08:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.028+0000 7fc107563640 1 Processor -- start 2024-04-25T08:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.028+0000 7fc107563640 1 -- start start 2024-04-25T08:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc107563640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc100072660 0x7fc1000c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc107563640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1000cfb70 0x7fc100155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc107563640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10015baf0 0x7fc1000bd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc107563640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1000ad1d0 con 0x7fc10015baf0 2024-04-25T08:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc107563640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1000ad050 con 0x7fc100072660 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc107563640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1000ad350 con 0x7fc1000cfb70 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc105b00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10015baf0 0x7fc1000bd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc105b00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10015baf0 0x7fc1000bd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37598/0 (socket says 172.21.15.38:37598) 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc105b00640 1 -- 172.21.15.38:0/835972254 learned_addr learned my addr 172.21.15.38:0/835972254 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc1052ff640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc100072660 0x7fc1000c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc104afe640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1000cfb70 0x7fc100155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc105b00640 1 -- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1000cfb70 msgr2=0x7fc100155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc105b00640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1000cfb70 0x7fc100155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc105b00640 1 -- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc100072660 msgr2=0x7fc1000c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc105b00640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc100072660 0x7fc1000c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc105b00640 1 -- 172.21.15.38:0/835972254 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1000c6a30 con 0x7fc10015baf0 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.029+0000 7fc105b00640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10015baf0 0x7fc1000bd340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000cef0 tx=0x7fc0f000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.030+0000 7fc0fdffb640 1 -- 172.21.15.38:0/835972254 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f001a070 con 0x7fc10015baf0 2024-04-25T08:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.030+0000 7fc0fdffb640 1 -- 172.21.15.38:0/835972254 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f000d860 con 0x7fc10015baf0 2024-04-25T08:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.030+0000 7fc0fdffb640 1 -- 172.21.15.38:0/835972254 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f000dc50 con 0x7fc10015baf0 2024-04-25T08:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.030+0000 7fc107563640 1 -- 172.21.15.38:0/835972254 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1000c6d10 con 0x7fc10015baf0 2024-04-25T08:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.030+0000 7fc107563640 1 -- 172.21.15.38:0/835972254 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1000c72d0 con 0x7fc10015baf0 2024-04-25T08:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.031+0000 7fc107563640 1 -- 172.21.15.38:0/835972254 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc100068e80 con 0x7fc10015baf0 2024-04-25T08:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.031+0000 7fc0fdffb640 1 -- 172.21.15.38:0/835972254 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0f001d070 con 0x7fc10015baf0 2024-04-25T08:32:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.032+0000 7fc0fdffb640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0ec06b4d0 0x7fc0ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.032+0000 7fc0fdffb640 1 -- 172.21.15.38:0/835972254 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc0f008ac90 con 0x7fc10015baf0 2024-04-25T08:32:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.032+0000 7fc1052ff640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0ec06b4d0 0x7fc0ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.032+0000 7fc1052ff640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0ec06b4d0 0x7fc0ec06d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8034000 tx=0x7fc0f802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.036+0000 7fc0fdffb640 1 -- 172.21.15.38:0/835972254 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0f00592d0 con 0x7fc10015baf0 2024-04-25T08:32:14.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.214+0000 7fc107563640 1 -- 172.21.15.38:0/835972254 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc1000c94f0 con 0x7fc10015baf0 2024-04-25T08:32:14.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.214+0000 7fc0fdffb640 1 -- 172.21.15.38:0/835972254 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc0f005d370 con 0x7fc10015baf0 2024-04-25T08:32:14.211 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:14.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.216+0000 7fc107563640 1 -- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0ec06b4d0 msgr2=0x7fc0ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.216+0000 7fc107563640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0ec06b4d0 0x7fc0ec06d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8034000 tx=0x7fc0f802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:14.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.216+0000 7fc107563640 1 -- 172.21.15.38:0/835972254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10015baf0 msgr2=0x7fc1000bd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.216+0000 7fc107563640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10015baf0 0x7fc1000bd340 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc0f000cef0 tx=0x7fc0f000a710 comp rx=0 tx=0).stop 2024-04-25T08:32:14.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.216+0000 7fc107563640 1 -- 172.21.15.38:0/835972254 shutdown_connections 2024-04-25T08:32:14.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.216+0000 7fc107563640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10015baf0 0x7fc1000bd340 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.216+0000 7fc107563640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0ec06b4d0 0x7fc0ec06d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.216+0000 7fc107563640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1000cfb70 0x7fc100155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.216+0000 7fc107563640 1 --2- 172.21.15.38:0/835972254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc100072660 0x7fc1000c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.216+0000 7fc107563640 1 -- 172.21.15.38:0/835972254 >> 172.21.15.38:0/835972254 conn(0x7fc10006adc0 msgr2=0x7fc10015ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:14.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.216+0000 7fc107563640 1 -- 172.21.15.38:0/835972254 shutdown_connections 2024-04-25T08:32:14.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.216+0000 7fc107563640 1 -- 172.21.15.38:0/835972254 wait complete. 2024-04-25T08:32:14.223 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:14.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.328+0000 7effd4553640 1 Processor -- start 2024-04-25T08:32:14.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.328+0000 7effd4553640 1 -- start start 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.329+0000 7effd4553640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7effcc15da90 0x7effcc161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.329+0000 7effd4553640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effcc0d0850 0x7effcc0d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.329+0000 7effd4553640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effcc162160 con 0x7effcc161c50 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.329+0000 7effd4553640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effcc0d19b0 con 0x7effcc15da90 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.329+0000 7effd4553640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc0d1f70 con 0x7effcc0d0850 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.329+0000 7effd22ef640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7effcc15da90 0x7effcc161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.329+0000 7effd1aee640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7effcc161c50 0x7effcc0d0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54080/0 (socket says 172.21.15.38:54080) 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.329+0000 7effd1aee640 1 -- 172.21.15.38:0/144210260 learned_addr learned my addr 172.21.15.38:0/144210260 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.329+0000 7effd22ef640 1 -- 172.21.15.38:0/144210260 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effcc0d0850 msgr2=0x7effcc0d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.329+0000 7effd2af0640 1 --2- 172.21.15.38:0/144210260 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effcc0d0850 0x7effcc0d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.330+0000 7effd22ef640 1 --2- 172.21.15.38:0/144210260 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effcc0d0850 0x7effcc0d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.330+0000 7effd22ef640 1 -- 172.21.15.38:0/144210260 >> v1:172.21.15.38:6789/0 conn(0x7effcc161c50 legacy=0x7effcc0d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.330+0000 7effd12ed640 1 -- 172.21.15.38:0/144210260 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206498478 0 0) 0x7effcc162160 con 0x7effcc161c50 2024-04-25T08:32:14.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.330+0000 7effd22ef640 1 -- 172.21.15.38:0/144210260 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc162440 con 0x7effcc15da90 2024-04-25T08:32:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.330+0000 7effd22ef640 1 --2- 172.21.15.38:0/144210260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effcc15da90 0x7effcc161710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7effc80097a0 tx=0x7effc802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265a588351694981 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.330+0000 7effd12ed640 1 -- 172.21.15.38:0/144210260 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc8037070 con 0x7effcc15da90 2024-04-25T08:32:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.330+0000 7effd12ed640 1 -- 172.21.15.38:0/144210260 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc8032440 con 0x7effcc15da90 2024-04-25T08:32:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.330+0000 7effd12ed640 1 -- 172.21.15.38:0/144210260 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc8032810 con 0x7effcc15da90 2024-04-25T08:32:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.331+0000 7effd4553640 1 -- 172.21.15.38:0/144210260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effcc15da90 msgr2=0x7effcc161710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.331+0000 7effd4553640 1 --2- 172.21.15.38:0/144210260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effcc15da90 0x7effcc161710 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7effc80097a0 tx=0x7effc802a180 comp rx=0 tx=0).stop 2024-04-25T08:32:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.331+0000 7effd4553640 1 -- 172.21.15.38:0/144210260 shutdown_connections 2024-04-25T08:32:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.331+0000 7effd4553640 1 --2- 172.21.15.38:0/144210260 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effcc0d0850 0x7effcc0d2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.331+0000 7effd4553640 1 --2- 172.21.15.38:0/144210260 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effcc15da90 0x7effcc161710 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.331+0000 7effd4553640 1 -- 172.21.15.38:0/144210260 >> 172.21.15.38:0/144210260 conn(0x7effcc06c060 msgr2=0x7effcc06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.331+0000 7effd4553640 1 -- 172.21.15.38:0/144210260 shutdown_connections 2024-04-25T08:32:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.331+0000 7effd4553640 1 -- 172.21.15.38:0/144210260 wait complete. 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.332+0000 7effd4553640 1 Processor -- start 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.332+0000 7effd4553640 1 -- start start 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.333+0000 7effd4553640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effcc0d0850 0x7effcc155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.333+0000 7effd4553640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effcc15da90 0x7effcc15e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.333+0000 7effd4553640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effcc161c50 0x7effcc0ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.333+0000 7effd4553640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc0b9400 con 0x7effcc161c50 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.333+0000 7effd4553640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effcc15de70 con 0x7effcc15da90 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.333+0000 7effd4553640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effcc15f440 con 0x7effcc0d0850 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.333+0000 7effd2af0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effcc161c50 0x7effcc0ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.333+0000 7effd2af0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effcc161c50 0x7effcc0ccd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37620/0 (socket says 172.21.15.38:37620) 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.333+0000 7effd2af0640 1 -- 172.21.15.38:0/366535974 learned_addr learned my addr 172.21.15.38:0/366535974 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.333+0000 7effd22ef640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effcc0d0850 0x7effcc155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.333+0000 7effd1aee640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effcc15da90 0x7effcc15e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.334+0000 7effd2af0640 1 -- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effcc0d0850 msgr2=0x7effcc155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.334+0000 7effd2af0640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effcc0d0850 0x7effcc155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.334+0000 7effd2af0640 1 -- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effcc15da90 msgr2=0x7effcc15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.334+0000 7effd2af0640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effcc15da90 0x7effcc15e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.334+0000 7effd2af0640 1 -- 172.21.15.38:0/366535974 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effcc15eba0 con 0x7effcc161c50 2024-04-25T08:32:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.334+0000 7effd1aee640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effcc15da90 0x7effcc15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.334+0000 7effd2af0640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effcc161c50 0x7effcc0ccd90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7effc400cd20 tx=0x7effc400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.334+0000 7effc2ffd640 1 -- 172.21.15.38:0/366535974 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4003b90 con 0x7effcc161c50 2024-04-25T08:32:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.334+0000 7effc2ffd640 1 -- 172.21.15.38:0/366535974 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc4003d30 con 0x7effcc161c50 2024-04-25T08:32:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.334+0000 7effc2ffd640 1 -- 172.21.15.38:0/366535974 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effc4004240 con 0x7effcc161c50 2024-04-25T08:32:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.334+0000 7effd4553640 1 -- 172.21.15.38:0/366535974 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effcc15ee80 con 0x7effcc161c50 2024-04-25T08:32:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.334+0000 7effd4553640 1 -- 172.21.15.38:0/366535974 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effcc1f3aa0 con 0x7effcc161c50 2024-04-25T08:32:14.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.335+0000 7effd4553640 1 -- 172.21.15.38:0/366535974 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effcc1f3ca0 con 0x7effcc161c50 2024-04-25T08:32:14.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.335+0000 7effc2ffd640 1 -- 172.21.15.38:0/366535974 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7effc40043e0 con 0x7effcc161c50 2024-04-25T08:32:14.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.336+0000 7effc2ffd640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7effb006b4d0 0x7effb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.336+0000 7effc2ffd640 1 -- 172.21.15.38:0/366535974 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7effc408afa0 con 0x7effcc161c50 2024-04-25T08:32:14.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.336+0000 7effd22ef640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7effb006b4d0 0x7effb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.337+0000 7effd22ef640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7effb006b4d0 0x7effb006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7effc8034000 tx=0x7effc802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.340+0000 7effc2ffd640 1 -- 172.21.15.38:0/366535974 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7effcc1f3ca0 con 0x7effcc161c50 2024-04-25T08:32:14.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.520+0000 7effd4553640 1 -- 172.21.15.38:0/366535974 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effcc068e80 con 0x7effcc161c50 2024-04-25T08:32:14.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.521+0000 7effc2ffd640 1 -- 172.21.15.38:0/366535974 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18502 (secure 0 0 0) 0x7effc4059a50 con 0x7effcc161c50 2024-04-25T08:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:14.518 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:31:40.691276+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:14.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.524+0000 7effd4553640 1 -- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7effb006b4d0 msgr2=0x7effb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.524+0000 7effd4553640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7effb006b4d0 0x7effb006d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7effc8034000 tx=0x7effc802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:14.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.524+0000 7effd4553640 1 -- 172.21.15.38:0/366535974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effcc161c50 msgr2=0x7effcc0ccd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.524+0000 7effd4553640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effcc161c50 0x7effcc0ccd90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7effc400cd20 tx=0x7effc400a660 comp rx=0 tx=0).stop 2024-04-25T08:32:14.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.524+0000 7effd4553640 1 -- 172.21.15.38:0/366535974 shutdown_connections 2024-04-25T08:32:14.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.524+0000 7effd4553640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effcc161c50 0x7effcc0ccd90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.524+0000 7effd4553640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7effcc15da90 0x7effcc15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.524+0000 7effd4553640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7effb006b4d0 0x7effb006d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.524+0000 7effd4553640 1 --2- 172.21.15.38:0/366535974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7effcc0d0850 0x7effcc155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.524+0000 7effd4553640 1 -- 172.21.15.38:0/366535974 >> 172.21.15.38:0/366535974 conn(0x7effcc06c060 msgr2=0x7effcc0caa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:14.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.524+0000 7effd4553640 1 -- 172.21.15.38:0/366535974 shutdown_connections 2024-04-25T08:32:14.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.524+0000 7effd4553640 1 -- 172.21.15.38:0/366535974 wait complete. 2024-04-25T08:32:14.531 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:14.531 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:14.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.640+0000 7fa185322640 1 Processor -- start 2024-04-25T08:32:14.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.640+0000 7fa185322640 1 -- start start 2024-04-25T08:32:14.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.641+0000 7fa185322640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa1800d00e0 0x7fa1800cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.641+0000 7fa185322640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa1800cd3b0 0x7fa1800cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.641+0000 7fa185322640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1800cdd80 con 0x7fa1800b1d60 2024-04-25T08:32:14.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.641+0000 7fa185322640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18006abe0 con 0x7fa1800cd3b0 2024-04-25T08:32:14.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.641+0000 7fa185322640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18006b480 con 0x7fa1800d00e0 2024-04-25T08:32:14.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.641+0000 7fa17f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa1800cd3b0 0x7fa1800cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.641+0000 7fa17effd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa1800b1d60 0x7fa1800cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54088/0 (socket says 172.21.15.38:54088) 2024-04-25T08:32:14.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.641+0000 7fa17effd640 1 -- 172.21.15.38:0/1196911690 learned_addr learned my addr 172.21.15.38:0/1196911690 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:14.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.642+0000 7fa17f7fe640 1 -- 172.21.15.38:0/1196911690 >> v2:172.21.15.107:3300/0 conn(0x7fa1800d00e0 msgr2=0x7fa1800cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:14.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.642+0000 7fa17e7fc640 1 --2- 172.21.15.38:0/1196911690 >> v2:172.21.15.107:3300/0 conn(0x7fa1800d00e0 0x7fa1800cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.642+0000 7fa17dffb640 1 -- 172.21.15.38:0/1196911690 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390151613 0 0) 0x7fa1800cdd80 con 0x7fa1800b1d60 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.642+0000 7fa17f7fe640 1 --2- 172.21.15.38:0/1196911690 >> v2:172.21.15.107:3300/0 conn(0x7fa1800d00e0 0x7fa1800cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.642+0000 7fa17f7fe640 1 -- 172.21.15.38:0/1196911690 >> v1:172.21.15.38:6789/0 conn(0x7fa1800b1d60 legacy=0x7fa1800cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.642+0000 7fa17f7fe640 1 -- 172.21.15.38:0/1196911690 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18015ef40 con 0x7fa1800cd3b0 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.642+0000 7fa17e7fc640 1 --2- 172.21.15.38:0/1196911690 >> v2:172.21.15.107:3300/0 conn(0x7fa1800d00e0 0x7fa1800cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.642+0000 7fa17f7fe640 1 --2- 172.21.15.38:0/1196911690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1800cd3b0 0x7fa1800cd840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa1800ce1a0 tx=0x7fa17002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8d3a5405ac3738 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.643+0000 7fa17dffb640 1 -- 172.21.15.38:0/1196911690 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170007740 con 0x7fa1800cd3b0 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.643+0000 7fa17dffb640 1 -- 172.21.15.38:0/1196911690 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1700078e0 con 0x7fa1800cd3b0 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.643+0000 7fa17dffb640 1 -- 172.21.15.38:0/1196911690 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170007cb0 con 0x7fa1800cd3b0 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.643+0000 7fa185322640 1 -- 172.21.15.38:0/1196911690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1800cd3b0 msgr2=0x7fa1800cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.643+0000 7fa185322640 1 --2- 172.21.15.38:0/1196911690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1800cd3b0 0x7fa1800cd840 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa1800ce1a0 tx=0x7fa17002a5f0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.643+0000 7fa185322640 1 -- 172.21.15.38:0/1196911690 shutdown_connections 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.643+0000 7fa185322640 1 --2- 172.21.15.38:0/1196911690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1800cd3b0 0x7fa1800cd840 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.643+0000 7fa185322640 1 --2- 172.21.15.38:0/1196911690 >> v2:172.21.15.107:3300/0 conn(0x7fa1800d00e0 0x7fa1800cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.643+0000 7fa185322640 1 -- 172.21.15.38:0/1196911690 >> 172.21.15.38:0/1196911690 conn(0x7fa1800d06b0 msgr2=0x7fa1800d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:14.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.643+0000 7fa185322640 1 -- 172.21.15.38:0/1196911690 shutdown_connections 2024-04-25T08:32:14.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.643+0000 7fa185322640 1 -- 172.21.15.38:0/1196911690 wait complete. 2024-04-25T08:32:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.644+0000 7fa185322640 1 Processor -- start 2024-04-25T08:32:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.645+0000 7fa185322640 1 -- start start 2024-04-25T08:32:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.645+0000 7fa185322640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1800b1d60 0x7fa180159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.645+0000 7fa185322640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1800cd3b0 0x7fa18015a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.645+0000 7fa185322640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1800d00e0 0x7fa18015d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.645+0000 7fa185322640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18006b910 con 0x7fa1800b1d60 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.645+0000 7fa185322640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18006b110 con 0x7fa1800cd3b0 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.645+0000 7fa185322640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18006ba90 con 0x7fa1800d00e0 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.645+0000 7fa17effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1800b1d60 0x7fa180159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.645+0000 7fa17effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1800b1d60 0x7fa180159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37652/0 (socket says 172.21.15.38:37652) 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.645+0000 7fa17effd640 1 -- 172.21.15.38:0/1348232756 learned_addr learned my addr 172.21.15.38:0/1348232756 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa17effd640 1 -- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1800d00e0 msgr2=0x7fa18015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa17f7fe640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1800d00e0 0x7fa18015d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa17effd640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1800d00e0 0x7fa18015d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa17e7fc640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1800cd3b0 0x7fa18015a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa17effd640 1 -- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1800cd3b0 msgr2=0x7fa18015a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa17effd640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1800cd3b0 0x7fa18015a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa17effd640 1 -- 172.21.15.38:0/1348232756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa180159de0 con 0x7fa1800b1d60 2024-04-25T08:32:14.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa17effd640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1800b1d60 0x7fa180159770 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa17800c430 tx=0x7fa17800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa1777fe640 1 -- 172.21.15.38:0/1348232756 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780074e0 con 0x7fa1800b1d60 2024-04-25T08:32:14.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa1777fe640 1 -- 172.21.15.38:0/1348232756 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1780047e0 con 0x7fa1800b1d60 2024-04-25T08:32:14.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa185322640 1 -- 172.21.15.38:0/1348232756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1800ad5b0 con 0x7fa1800b1d60 2024-04-25T08:32:14.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa1777fe640 1 -- 172.21.15.38:0/1348232756 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa178004bd0 con 0x7fa1800b1d60 2024-04-25T08:32:14.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.646+0000 7fa185322640 1 -- 172.21.15.38:0/1348232756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1800adb70 con 0x7fa1800b1d60 2024-04-25T08:32:14.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.647+0000 7fa185322640 1 -- 172.21.15.38:0/1348232756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa180068e80 con 0x7fa1800b1d60 2024-04-25T08:32:14.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.648+0000 7fa1777fe640 1 -- 172.21.15.38:0/1348232756 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa178011070 con 0x7fa1800b1d60 2024-04-25T08:32:14.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.648+0000 7fa1777fe640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa16006b4d0 0x7fa16006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.648+0000 7fa1777fe640 1 -- 172.21.15.38:0/1348232756 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa17808a200 con 0x7fa1800b1d60 2024-04-25T08:32:14.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.648+0000 7fa17e7fc640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa16006b4d0 0x7fa16006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.649+0000 7fa17e7fc640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa16006b4d0 0x7fa16006d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa18015a3a0 tx=0x7fa168008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.652+0000 7fa1777fe640 1 -- 172.21.15.38:0/1348232756 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa178058d40 con 0x7fa1800b1d60 2024-04-25T08:32:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.834+0000 7fa185322640 1 -- 172.21.15.38:0/1348232756 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1800c46d0 con 0x7fa1800b1d60 2024-04-25T08:32:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.835+0000 7fa1777fe640 1 -- 172.21.15.38:0/1348232756 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa178058640 con 0x7fa1800b1d60 2024-04-25T08:32:14.831 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.837+0000 7fa185322640 1 -- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa16006b4d0 msgr2=0x7fa16006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.837+0000 7fa185322640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa16006b4d0 0x7fa16006d990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa18015a3a0 tx=0x7fa168008040 comp rx=0 tx=0).stop 2024-04-25T08:32:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.837+0000 7fa185322640 1 -- 172.21.15.38:0/1348232756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1800b1d60 msgr2=0x7fa180159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.837+0000 7fa185322640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1800b1d60 0x7fa180159770 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa17800c430 tx=0x7fa17800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.837+0000 7fa185322640 1 -- 172.21.15.38:0/1348232756 shutdown_connections 2024-04-25T08:32:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.837+0000 7fa185322640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa16006b4d0 0x7fa16006d990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.837+0000 7fa185322640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1800d00e0 0x7fa18015d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.837+0000 7fa185322640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1800cd3b0 0x7fa18015a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.837+0000 7fa185322640 1 --2- 172.21.15.38:0/1348232756 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1800b1d60 0x7fa180159770 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.837+0000 7fa185322640 1 -- 172.21.15.38:0/1348232756 >> 172.21.15.38:0/1348232756 conn(0x7fa1800d06b0 msgr2=0x7fa18015f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.837+0000 7fa185322640 1 -- 172.21.15.38:0/1348232756 shutdown_connections 2024-04-25T08:32:14.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.837+0000 7fa185322640 1 -- 172.21.15.38:0/1348232756 wait complete. 2024-04-25T08:32:14.844 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:14.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.952+0000 7fd77e990640 1 Processor -- start 2024-04-25T08:32:14.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.952+0000 7fd77e990640 1 -- start start 2024-04-25T08:32:14.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.953+0000 7fd77e990640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd7780d0320 0x7fd7780d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.953+0000 7fd77e990640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7780d2060 con 0x7fd7780b6980 2024-04-25T08:32:14.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.953+0000 7fd77e990640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7780d2330 con 0x7fd7780d0700 2024-04-25T08:32:14.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.953+0000 7fd77e990640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7780d2610 con 0x7fd7780d0320 2024-04-25T08:32:14.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.953+0000 7fd777fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd7780d0320 0x7fd7780d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.953+0000 7fd777fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd7780d0320 0x7fd7780d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37662/0 (socket says 172.21.15.38:37662) 2024-04-25T08:32:14.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.953+0000 7fd77cf2d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd7780b6980 0x7fd7780b6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54100/0 (socket says 172.21.15.38:54100) 2024-04-25T08:32:14.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.953+0000 7fd777fff640 1 -- 172.21.15.38:0/979056772 learned_addr learned my addr 172.21.15.38:0/979056772 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:14.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.953+0000 7fd777fff640 1 -- 172.21.15.38:0/979056772 >> v1:172.21.15.107:6789/0 conn(0x7fd7780d0700 legacy=0x7fd7780d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:14.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.953+0000 7fd777fff640 1 -- 172.21.15.38:0/979056772 >> v1:172.21.15.38:6789/0 conn(0x7fd7780b6980 legacy=0x7fd7780b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.953+0000 7fd777fff640 1 -- 172.21.15.38:0/979056772 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7780d28f0 con 0x7fd7780d0320 2024-04-25T08:32:14.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.954+0000 7fd777fff640 1 --2- 172.21.15.38:0/979056772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7780d0320 0x7fd7780d33d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009870 tx=0x7fd76c02a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d43f5167c1124bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.954+0000 7fd776ffd640 1 -- 172.21.15.38:0/979056772 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c037070 con 0x7fd7780d0320 2024-04-25T08:32:14.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.954+0000 7fd776ffd640 1 -- 172.21.15.38:0/979056772 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76c032640 con 0x7fd7780d0320 2024-04-25T08:32:14.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.954+0000 7fd776ffd640 1 -- 172.21.15.38:0/979056772 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c032a10 con 0x7fd7780d0320 2024-04-25T08:32:14.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.954+0000 7fd77e990640 1 -- 172.21.15.38:0/979056772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7780d0320 msgr2=0x7fd7780d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.954+0000 7fd77e990640 1 --2- 172.21.15.38:0/979056772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7780d0320 0x7fd7780d33d0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009870 tx=0x7fd76c02a330 comp rx=0 tx=0).stop 2024-04-25T08:32:14.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.955+0000 7fd77e990640 1 -- 172.21.15.38:0/979056772 shutdown_connections 2024-04-25T08:32:14.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.955+0000 7fd77e990640 1 --2- 172.21.15.38:0/979056772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7780d0320 0x7fd7780d33d0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.955+0000 7fd77e990640 1 -- 172.21.15.38:0/979056772 >> 172.21.15.38:0/979056772 conn(0x7fd7780668f0 msgr2=0x7fd778066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:14.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.955+0000 7fd77e990640 1 -- 172.21.15.38:0/979056772 shutdown_connections 2024-04-25T08:32:14.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.955+0000 7fd77e990640 1 -- 172.21.15.38:0/979056772 wait complete. 2024-04-25T08:32:14.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.956+0000 7fd77e990640 1 Processor -- start 2024-04-25T08:32:14.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.956+0000 7fd77e990640 1 -- start start 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd77e990640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7780b6980 0x7fd7780b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd77e990640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7780d0320 0x7fd77815b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd77e990640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7780d0700 0x7fd7781e6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd77e990640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780d0b40 con 0x7fd7780b6980 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd77e990640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780d2eb0 con 0x7fd7780d0700 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd77e990640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7780b4850 con 0x7fd7780d0320 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd777fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7780b6980 0x7fd7780b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd777fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7780b6980 0x7fd7780b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37672/0 (socket says 172.21.15.38:37672) 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd777fff640 1 -- 172.21.15.38:0/2358443784 learned_addr learned my addr 172.21.15.38:0/2358443784 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd77cf2d640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7780d0700 0x7fd7781e6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd7777fe640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7780d0320 0x7fd77815b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd777fff640 1 -- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7780d0320 msgr2=0x7fd77815b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd777fff640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7780d0320 0x7fd77815b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd777fff640 1 -- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7780d0700 msgr2=0x7fd7781e6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:14.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd777fff640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7780d0700 0x7fd7781e6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:14.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd777fff640 1 -- 172.21.15.38:0/2358443784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7780b3c50 con 0x7fd7780b6980 2024-04-25T08:32:14.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd777fff640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7780b6980 0x7fd7780b5e20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009870 tx=0x7fd76c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd774ff9640 1 -- 172.21.15.38:0/2358443784 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c041020 con 0x7fd7780b6980 2024-04-25T08:32:14.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd774ff9640 1 -- 172.21.15.38:0/2358443784 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76c030990 con 0x7fd7780b6980 2024-04-25T08:32:14.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd77e990640 1 -- 172.21.15.38:0/2358443784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7780b3f20 con 0x7fd7780b6980 2024-04-25T08:32:14.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.957+0000 7fd774ff9640 1 -- 172.21.15.38:0/2358443784 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c037040 con 0x7fd7780b6980 2024-04-25T08:32:14.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.958+0000 7fd77e990640 1 -- 172.21.15.38:0/2358443784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7780b44e0 con 0x7fd7780b6980 2024-04-25T08:32:14.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.958+0000 7fd77e990640 1 -- 172.21.15.38:0/2358443784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd778068e80 con 0x7fd7780b6980 2024-04-25T08:32:14.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.959+0000 7fd774ff9640 1 -- 172.21.15.38:0/2358443784 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd76c044020 con 0x7fd7780b6980 2024-04-25T08:32:14.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.959+0000 7fd774ff9640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd76406b4d0 0x7fd76406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:14.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.959+0000 7fd774ff9640 1 -- 172.21.15.38:0/2358443784 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd76c0aaf00 con 0x7fd7780b6980 2024-04-25T08:32:14.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.960+0000 7fd7777fe640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd76406b4d0 0x7fd76406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:14.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.960+0000 7fd7777fe640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd76406b4d0 0x7fd76406d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd768006fd0 tx=0x7fd768008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:14.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:14.964+0000 7fd774ff9640 1 -- 172.21.15.38:0/2358443784 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd76c04b030 con 0x7fd7780b6980 2024-04-25T08:32:15.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.145+0000 7fd77e990640 1 -- 172.21.15.38:0/2358443784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd7780b3050 con 0x7fd7780b6980 2024-04-25T08:32:15.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.145+0000 7fd774ff9640 1 -- 172.21.15.38:0/2358443784 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd76c0799a0 con 0x7fd7780b6980 2024-04-25T08:32:15.141 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:15.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.147+0000 7fd77e990640 1 -- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd76406b4d0 msgr2=0x7fd76406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.147+0000 7fd77e990640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd76406b4d0 0x7fd76406d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd768006fd0 tx=0x7fd768008040 comp rx=0 tx=0).stop 2024-04-25T08:32:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.147+0000 7fd77e990640 1 -- 172.21.15.38:0/2358443784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7780b6980 msgr2=0x7fd7780b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.147+0000 7fd77e990640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7780b6980 0x7fd7780b5e20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009870 tx=0x7fd76c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.147+0000 7fd77e990640 1 -- 172.21.15.38:0/2358443784 shutdown_connections 2024-04-25T08:32:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.147+0000 7fd77e990640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7780d0700 0x7fd7781e6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.147+0000 7fd77e990640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd76406b4d0 0x7fd76406d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.147+0000 7fd77e990640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7780d0320 0x7fd77815b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.147+0000 7fd77e990640 1 --2- 172.21.15.38:0/2358443784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7780b6980 0x7fd7780b5e20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.147+0000 7fd77e990640 1 -- 172.21.15.38:0/2358443784 >> 172.21.15.38:0/2358443784 conn(0x7fd7780668f0 msgr2=0x7fd7780d4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.147+0000 7fd77e990640 1 -- 172.21.15.38:0/2358443784 shutdown_connections 2024-04-25T08:32:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.147+0000 7fd77e990640 1 -- 172.21.15.38:0/2358443784 wait complete. 2024-04-25T08:32:15.155 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:15.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.263+0000 7f5c7d3a5640 1 Processor -- start 2024-04-25T08:32:15.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.263+0000 7f5c7d3a5640 1 -- start start 2024-04-25T08:32:15.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.263+0000 7f5c7d3a5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c780d33d0 0x7f5c780d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.263+0000 7f5c7d3a5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c780d2330 con 0x7f5c780d1e90 2024-04-25T08:32:15.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.264+0000 7f5c7d3a5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c780d2610 con 0x7f5c780d0320 2024-04-25T08:32:15.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.264+0000 7f5c7d3a5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780d28f0 con 0x7f5c780d33d0 2024-04-25T08:32:15.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.264+0000 7f5c777fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5c780d1e90 0x7f5c780b6980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54114/0 (socket says 172.21.15.38:54114) 2024-04-25T08:32:15.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.264+0000 7f5c777fe640 1 -- 172.21.15.38:0/688715766 learned_addr learned my addr 172.21.15.38:0/688715766 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:15.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.264+0000 7f5c767fc640 1 --2- 172.21.15.38:0/688715766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c780d33d0 0x7f5c780d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.264+0000 7f5c75ffb640 1 -- 172.21.15.38:0/688715766 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116578521 0 0) 0x7f5c780d2330 con 0x7f5c780d1e90 2024-04-25T08:32:15.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.264+0000 7f5c75ffb640 1 -- 172.21.15.38:0/688715766 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c60003520 con 0x7f5c780d1e90 2024-04-25T08:32:15.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.264+0000 7f5c75ffb640 1 -- 172.21.15.38:0/688715766 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257110831 0 0) 0x7f5c780d2610 con 0x7f5c780d0320 2024-04-25T08:32:15.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.264+0000 7f5c767fc640 1 -- 172.21.15.38:0/688715766 >> v1:172.21.15.107:6789/0 conn(0x7f5c780d0320 legacy=0x7f5c780d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.264+0000 7f5c767fc640 1 -- 172.21.15.38:0/688715766 >> v1:172.21.15.38:6789/0 conn(0x7f5c780d1e90 legacy=0x7f5c780b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.264+0000 7f5c767fc640 1 -- 172.21.15.38:0/688715766 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780d2bd0 con 0x7f5c780d33d0 2024-04-25T08:32:15.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.265+0000 7f5c767fc640 1 --2- 172.21.15.38:0/688715766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c780d33d0 0x7f5c780d1950 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0097d0 tx=0x7f5c6c02a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebf14274a8c29754 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.265+0000 7f5c75ffb640 1 -- 172.21.15.38:0/688715766 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c037070 con 0x7f5c780d33d0 2024-04-25T08:32:15.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.265+0000 7f5c75ffb640 1 -- 172.21.15.38:0/688715766 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6c032440 con 0x7f5c780d33d0 2024-04-25T08:32:15.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.265+0000 7f5c75ffb640 1 -- 172.21.15.38:0/688715766 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0327f0 con 0x7f5c780d33d0 2024-04-25T08:32:15.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.266+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/688715766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c780d33d0 msgr2=0x7f5c780d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.266+0000 7f5c7d3a5640 1 --2- 172.21.15.38:0/688715766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c780d33d0 0x7f5c780d1950 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0097d0 tx=0x7f5c6c02a290 comp rx=0 tx=0).stop 2024-04-25T08:32:15.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.266+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/688715766 shutdown_connections 2024-04-25T08:32:15.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.266+0000 7f5c7d3a5640 1 --2- 172.21.15.38:0/688715766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c780d33d0 0x7f5c780d1950 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.266+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/688715766 >> 172.21.15.38:0/688715766 conn(0x7f5c780668f0 msgr2=0x7f5c78066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:15.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.266+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/688715766 shutdown_connections 2024-04-25T08:32:15.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.266+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/688715766 wait complete. 2024-04-25T08:32:15.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.267+0000 7f5c7d3a5640 1 Processor -- start 2024-04-25T08:32:15.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.267+0000 7f5c7d3a5640 1 -- start start 2024-04-25T08:32:15.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c7d3a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c780d0320 0x7f5c781f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c7d3a5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c780d1e90 0x7f5c7815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c7d3a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c780d33d0 0x7f5c781f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c7d3a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78161390 con 0x7f5c780d0320 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c7d3a5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78161210 con 0x7f5c780d33d0 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c7d3a5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780b9720 con 0x7f5c780d1e90 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c76ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c780d0320 0x7f5c781f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c76ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c780d0320 0x7f5c781f35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37680/0 (socket says 172.21.15.38:37680) 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c76ffd640 1 -- 172.21.15.38:0/1054849336 learned_addr learned my addr 172.21.15.38:0/1054849336 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c777fe640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c780d33d0 0x7f5c781f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c767fc640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c780d1e90 0x7f5c7815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c76ffd640 1 -- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c780d1e90 msgr2=0x7f5c7815aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c780d1e90 0x7f5c7815aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c76ffd640 1 -- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c780d33d0 msgr2=0x7f5c781f0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c780d33d0 0x7f5c781f0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c76ffd640 1 -- 172.21.15.38:0/1054849336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780b5cf0 con 0x7f5c780d0320 2024-04-25T08:32:15.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.268+0000 7f5c76ffd640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c780d0320 0x7f5c781f35f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800c3f0 tx=0x7f5c6800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.269+0000 7f5c5f7fe640 1 -- 172.21.15.38:0/1054849336 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68017070 con 0x7f5c780d0320 2024-04-25T08:32:15.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.269+0000 7f5c5f7fe640 1 -- 172.21.15.38:0/1054849336 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c680035f0 con 0x7f5c780d0320 2024-04-25T08:32:15.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.269+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/1054849336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c781f1030 con 0x7f5c780d0320 2024-04-25T08:32:15.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.269+0000 7f5c5f7fe640 1 -- 172.21.15.38:0/1054849336 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680039e0 con 0x7f5c780d0320 2024-04-25T08:32:15.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.269+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/1054849336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c781f3f70 con 0x7f5c780d0320 2024-04-25T08:32:15.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.270+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/1054849336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c78068e80 con 0x7f5c780d0320 2024-04-25T08:32:15.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.270+0000 7f5c5f7fe640 1 -- 172.21.15.38:0/1054849336 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c6801a070 con 0x7f5c780d0320 2024-04-25T08:32:15.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.271+0000 7f5c5f7fe640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5c6406b4d0 0x7f5c6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.271+0000 7f5c5f7fe640 1 -- 172.21.15.38:0/1054849336 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5c68089870 con 0x7f5c780d0320 2024-04-25T08:32:15.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.271+0000 7f5c767fc640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5c6406b4d0 0x7f5c6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.271+0000 7f5c767fc640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5c6406b4d0 0x7f5c6406d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c034000 tx=0x7f5c6c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.275+0000 7f5c5f7fe640 1 -- 172.21.15.38:0/1054849336 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c68057d70 con 0x7f5c780d0320 2024-04-25T08:32:15.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.456+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/1054849336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c780d0a80 con 0x7f5c780d0320 2024-04-25T08:32:15.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.457+0000 7f5c5f7fe640 1 -- 172.21.15.38:0/1054849336 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c6805be10 con 0x7f5c780d0320 2024-04-25T08:32:15.453 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.458+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5c6406b4d0 msgr2=0x7f5c6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.458+0000 7f5c7d3a5640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5c6406b4d0 0x7f5c6406d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c034000 tx=0x7f5c6c02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.458+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/1054849336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c780d0320 msgr2=0x7f5c781f35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.458+0000 7f5c7d3a5640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c780d0320 0x7f5c781f35f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800c3f0 tx=0x7f5c6800c8b0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.458+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/1054849336 shutdown_connections 2024-04-25T08:32:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.458+0000 7f5c7d3a5640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c780d33d0 0x7f5c781f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.458+0000 7f5c7d3a5640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5c6406b4d0 0x7f5c6406d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.458+0000 7f5c7d3a5640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c780d1e90 0x7f5c7815aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.458+0000 7f5c7d3a5640 1 --2- 172.21.15.38:0/1054849336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c780d0320 0x7f5c781f35f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.458+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/1054849336 >> 172.21.15.38:0/1054849336 conn(0x7f5c780668f0 msgr2=0x7f5c7815df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.458+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/1054849336 shutdown_connections 2024-04-25T08:32:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.459+0000 7f5c7d3a5640 1 -- 172.21.15.38:0/1054849336 wait complete. 2024-04-25T08:32:15.465 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:15.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.575+0000 7f6e63963640 1 Processor -- start 2024-04-25T08:32:15.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.575+0000 7f6e63963640 1 -- start start 2024-04-25T08:32:15.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e63963640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cc2c0 0x7f6e5c0cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e63963640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6e5c1635e0 0x7f6e5c1659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e63963640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e5c0ccc80 con 0x7f6e5c06abe0 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e63963640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5c15c1b0 con 0x7f6e5c1635e0 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e63963640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0ccf30 con 0x7f6e5c0cc2c0 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e61f00640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6e5c1635e0 0x7f6e5c1659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e61f00640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6e5c1635e0 0x7f6e5c1659d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37692/0 (socket says 172.21.15.38:37692) 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e61f00640 1 -- 172.21.15.38:0/836592945 learned_addr learned my addr 172.21.15.38:0/836592945 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e60efe640 1 --2- 172.21.15.38:0/836592945 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cc2c0 0x7f6e5c0cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e61f00640 1 -- 172.21.15.38:0/836592945 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cc2c0 msgr2=0x7f6e5c0cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e61f00640 1 --2- 172.21.15.38:0/836592945 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cc2c0 0x7f6e5c0cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e61f00640 1 -- 172.21.15.38:0/836592945 >> v1:172.21.15.107:6789/0 conn(0x7f6e5c06abe0 legacy=0x7f6e5c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e61f00640 1 -- 172.21.15.38:0/836592945 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c0cd210 con 0x7f6e5c1635e0 2024-04-25T08:32:15.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.576+0000 7f6e60efe640 1 --2- 172.21.15.38:0/836592945 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cc2c0 0x7f6e5c0cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:15.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.577+0000 7f6e61f00640 1 --2- 172.21.15.38:0/836592945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c1635e0 0x7f6e5c1659d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009870 tx=0x7f6e5402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfff61f02ef33327 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.580+0000 7f6e5bfff640 1 -- 172.21.15.38:0/836592945 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54037070 con 0x7f6e5c1635e0 2024-04-25T08:32:15.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.581+0000 7f6e5bfff640 1 -- 172.21.15.38:0/836592945 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e54032440 con 0x7f6e5c1635e0 2024-04-25T08:32:15.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.581+0000 7f6e5bfff640 1 -- 172.21.15.38:0/836592945 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54032810 con 0x7f6e5c1635e0 2024-04-25T08:32:15.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.582+0000 7f6e63963640 1 -- 172.21.15.38:0/836592945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c1635e0 msgr2=0x7f6e5c1659d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.582+0000 7f6e63963640 1 --2- 172.21.15.38:0/836592945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c1635e0 0x7f6e5c1659d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009870 tx=0x7f6e5402a180 comp rx=0 tx=0).stop 2024-04-25T08:32:15.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.583+0000 7f6e63963640 1 -- 172.21.15.38:0/836592945 shutdown_connections 2024-04-25T08:32:15.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.583+0000 7f6e63963640 1 --2- 172.21.15.38:0/836592945 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c1635e0 0x7f6e5c1659d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.583+0000 7f6e63963640 1 --2- 172.21.15.38:0/836592945 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cc2c0 0x7f6e5c0cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.583+0000 7f6e63963640 1 -- 172.21.15.38:0/836592945 >> 172.21.15.38:0/836592945 conn(0x7f6e5c0d06b0 msgr2=0x7f6e5c0d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:15.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.583+0000 7f6e63963640 1 -- 172.21.15.38:0/836592945 shutdown_connections 2024-04-25T08:32:15.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.583+0000 7f6e63963640 1 -- 172.21.15.38:0/836592945 wait complete. 2024-04-25T08:32:15.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.585+0000 7f6e63963640 1 Processor -- start 2024-04-25T08:32:15.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.585+0000 7f6e63963640 1 -- start start 2024-04-25T08:32:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.586+0000 7f6e63963640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c06abe0 0x7f6e5c0b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.586+0000 7f6e63963640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5c0cc2c0 0x7f6e5c0c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.586+0000 7f6e63963640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c1635e0 0x7f6e5c0c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.586+0000 7f6e63963640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c159790 con 0x7f6e5c06abe0 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.586+0000 7f6e63963640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0cd7d0 con 0x7f6e5c0cc2c0 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.586+0000 7f6e63963640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c159910 con 0x7f6e5c1635e0 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.586+0000 7f6e616ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c06abe0 0x7f6e5c0b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.586+0000 7f6e616ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c06abe0 0x7f6e5c0b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37700/0 (socket says 172.21.15.38:37700) 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.586+0000 7f6e616ff640 1 -- 172.21.15.38:0/173143786 learned_addr learned my addr 172.21.15.38:0/173143786 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.586+0000 7f6e60efe640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5c0cc2c0 0x7f6e5c0c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.586+0000 7f6e61f00640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c1635e0 0x7f6e5c0c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.586+0000 7f6e616ff640 1 -- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c1635e0 msgr2=0x7f6e5c0c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.587+0000 7f6e616ff640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c1635e0 0x7f6e5c0c59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.587+0000 7f6e616ff640 1 -- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5c0cc2c0 msgr2=0x7f6e5c0c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.587+0000 7f6e616ff640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5c0cc2c0 0x7f6e5c0c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.587+0000 7f6e616ff640 1 -- 172.21.15.38:0/173143786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c0c6000 con 0x7f6e5c06abe0 2024-04-25T08:32:15.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.587+0000 7f6e616ff640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c06abe0 0x7f6e5c0b3900 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00c7f0 tx=0x7f6e4c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.587+0000 7f6e59ffb640 1 -- 172.21.15.38:0/173143786 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c016020 con 0x7f6e5c06abe0 2024-04-25T08:32:15.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.587+0000 7f6e59ffb640 1 -- 172.21.15.38:0/173143786 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c00aa50 con 0x7f6e5c06abe0 2024-04-25T08:32:15.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.587+0000 7f6e63963640 1 -- 172.21.15.38:0/173143786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5c0c62e0 con 0x7f6e5c06abe0 2024-04-25T08:32:15.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.587+0000 7f6e59ffb640 1 -- 172.21.15.38:0/173143786 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00ae40 con 0x7f6e5c06abe0 2024-04-25T08:32:15.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.587+0000 7f6e63963640 1 -- 172.21.15.38:0/173143786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5c0c68a0 con 0x7f6e5c06abe0 2024-04-25T08:32:15.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.588+0000 7f6e63963640 1 -- 172.21.15.38:0/173143786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5c068e80 con 0x7f6e5c06abe0 2024-04-25T08:32:15.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.589+0000 7f6e59ffb640 1 -- 172.21.15.38:0/173143786 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e4c019020 con 0x7f6e5c06abe0 2024-04-25T08:32:15.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.589+0000 7f6e59ffb640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e4806b4d0 0x7f6e4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.589+0000 7f6e59ffb640 1 -- 172.21.15.38:0/173143786 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f6e4c08a2f0 con 0x7f6e5c06abe0 2024-04-25T08:32:15.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.589+0000 7f6e60efe640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e4806b4d0 0x7f6e4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.590+0000 7f6e60efe640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e4806b4d0 0x7f6e4806d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0c6e40 tx=0x7f6e50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.593+0000 7f6e59ffb640 1 -- 172.21.15.38:0/173143786 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e4c020070 con 0x7f6e5c06abe0 2024-04-25T08:32:15.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.773+0000 7f6e63963640 1 -- 172.21.15.38:0/173143786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6e5c066cc0 con 0x7f6e5c06abe0 2024-04-25T08:32:15.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.773+0000 7f6e59ffb640 1 -- 172.21.15.38:0/173143786 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6e4c058630 con 0x7f6e5c06abe0 2024-04-25T08:32:15.770 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.775+0000 7f6e63963640 1 -- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e4806b4d0 msgr2=0x7f6e4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.775+0000 7f6e63963640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e4806b4d0 0x7f6e4806d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c0c6e40 tx=0x7f6e50008040 comp rx=0 tx=0).stop 2024-04-25T08:32:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.775+0000 7f6e63963640 1 -- 172.21.15.38:0/173143786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c06abe0 msgr2=0x7f6e5c0b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.775+0000 7f6e63963640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c06abe0 0x7f6e5c0b3900 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00c7f0 tx=0x7f6e4c00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.775+0000 7f6e63963640 1 -- 172.21.15.38:0/173143786 shutdown_connections 2024-04-25T08:32:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.775+0000 7f6e63963640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e4806b4d0 0x7f6e4806d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.775+0000 7f6e63963640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c1635e0 0x7f6e5c0c59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.775+0000 7f6e63963640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5c0cc2c0 0x7f6e5c0c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.775+0000 7f6e63963640 1 --2- 172.21.15.38:0/173143786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c06abe0 0x7f6e5c0b3900 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.775+0000 7f6e63963640 1 -- 172.21.15.38:0/173143786 >> 172.21.15.38:0/173143786 conn(0x7f6e5c0d06b0 msgr2=0x7f6e5c162dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.775+0000 7f6e63963640 1 -- 172.21.15.38:0/173143786 shutdown_connections 2024-04-25T08:32:15.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.775+0000 7f6e63963640 1 -- 172.21.15.38:0/173143786 wait complete. 2024-04-25T08:32:15.793 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-25T08:32:15.793 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T08:32:15.793 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:15.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.898+0000 7fce714ac640 1 Processor -- start 2024-04-25T08:32:15.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.898+0000 7fce714ac640 1 -- start start 2024-04-25T08:32:15.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.899+0000 7fce714ac640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fce6c0cc5d0 0x7fce6c0d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.899+0000 7fce714ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce6c0afae0 0x7fce6c161600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.899+0000 7fce714ac640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fce6c0cf830 0x7fce6c0cfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.899+0000 7fce714ac640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6c0d1470 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.899+0000 7fce714ac640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6c0cc9b0 con 0x7fce6c0cf830 2024-04-25T08:32:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.899+0000 7fce714ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c06abe0 con 0x7fce6c0afae0 2024-04-25T08:32:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.899+0000 7fce6affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fce6c0cc5d0 0x7fce6c0d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.899+0000 7fce6affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fce6c0cc5d0 0x7fce6c0d0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37704/0 (socket says 172.21.15.38:37704) 2024-04-25T08:32:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.899+0000 7fce6affd640 1 -- 172.21.15.38:0/2866751048 learned_addr learned my addr 172.21.15.38:0/2866751048 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.899+0000 7fce6b7fe640 1 --2- 172.21.15.38:0/2866751048 >> v2:172.21.15.107:3300/0 conn(0x7fce6c0cf830 0x7fce6c0cfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.899+0000 7fce6a7fc640 1 --2- 172.21.15.38:0/2866751048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce6c0afae0 0x7fce6c161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.900+0000 7fce6affd640 1 -- 172.21.15.38:0/2866751048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce6c0afae0 msgr2=0x7fce6c161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.900+0000 7fce6affd640 1 --2- 172.21.15.38:0/2866751048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce6c0afae0 0x7fce6c161600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.900+0000 7fce6affd640 1 -- 172.21.15.38:0/2866751048 >> v2:172.21.15.107:3300/0 conn(0x7fce6c0cf830 msgr2=0x7fce6c0cfce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.900+0000 7fce6affd640 1 --2- 172.21.15.38:0/2866751048 >> v2:172.21.15.107:3300/0 conn(0x7fce6c0cf830 0x7fce6c0cfce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.900+0000 7fce6affd640 1 -- 172.21.15.38:0/2866751048 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c06aec0 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.900+0000 7fce6b7fe640 1 --2- 172.21.15.38:0/2866751048 >> v2:172.21.15.107:3300/0 conn(0x7fce6c0cf830 0x7fce6c0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.900+0000 7fce6affd640 1 --2- 172.21.15.38:0/2866751048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce6c0cc5d0 0x7fce6c0d0f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0c4110 tx=0x7fce5c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9101c76bd3cf103 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.900+0000 7fce69ffb640 1 -- 172.21.15.38:0/2866751048 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c036070 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.901+0000 7fce69ffb640 1 -- 172.21.15.38:0/2866751048 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c031600 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.901+0000 7fce69ffb640 1 -- 172.21.15.38:0/2866751048 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c0319d0 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.901+0000 7fce714ac640 1 -- 172.21.15.38:0/2866751048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce6c0cc5d0 msgr2=0x7fce6c0d0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.901+0000 7fce714ac640 1 --2- 172.21.15.38:0/2866751048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce6c0cc5d0 0x7fce6c0d0f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0c4110 tx=0x7fce5c02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.901+0000 7fce714ac640 1 -- 172.21.15.38:0/2866751048 shutdown_connections 2024-04-25T08:32:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.901+0000 7fce714ac640 1 --2- 172.21.15.38:0/2866751048 >> v2:172.21.15.107:3300/0 conn(0x7fce6c0cf830 0x7fce6c0cfce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.901+0000 7fce714ac640 1 --2- 172.21.15.38:0/2866751048 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce6c0afae0 0x7fce6c161600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.901+0000 7fce714ac640 1 --2- 172.21.15.38:0/2866751048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce6c0cc5d0 0x7fce6c0d0f30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.901+0000 7fce714ac640 1 -- 172.21.15.38:0/2866751048 >> 172.21.15.38:0/2866751048 conn(0x7fce6c099a70 msgr2=0x7fce6c0d0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.901+0000 7fce714ac640 1 -- 172.21.15.38:0/2866751048 shutdown_connections 2024-04-25T08:32:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.901+0000 7fce714ac640 1 -- 172.21.15.38:0/2866751048 wait complete. 2024-04-25T08:32:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.902+0000 7fce714ac640 1 Processor -- start 2024-04-25T08:32:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.903+0000 7fce714ac640 1 -- start start 2024-04-25T08:32:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.903+0000 7fce714ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce6c0afae0 0x7fce6c153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.903+0000 7fce714ac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce6c0cc5d0 0x7fce6c0c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.903+0000 7fce714ac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce6c0cf830 0x7fce6c158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.903+0000 7fce714ac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c0d0220 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.903+0000 7fce714ac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c06b480 con 0x7fce6c0cf830 2024-04-25T08:32:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.903+0000 7fce714ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6c161b40 con 0x7fce6c0afae0 2024-04-25T08:32:15.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.903+0000 7fce6a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce6c0cc5d0 0x7fce6c0c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.903+0000 7fce6a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce6c0cc5d0 0x7fce6c0c8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37712/0 (socket says 172.21.15.38:37712) 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.903+0000 7fce6a7fc640 1 -- 172.21.15.38:0/2121181074 learned_addr learned my addr 172.21.15.38:0/2121181074 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.904+0000 7fce6a7fc640 1 -- 172.21.15.38:0/2121181074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce6c0afae0 msgr2=0x7fce6c153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.904+0000 7fce6a7fc640 1 --2- 172.21.15.38:0/2121181074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce6c0afae0 0x7fce6c153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.904+0000 7fce6a7fc640 1 -- 172.21.15.38:0/2121181074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce6c0cf830 msgr2=0x7fce6c158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.904+0000 7fce6a7fc640 1 --2- 172.21.15.38:0/2121181074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce6c0cf830 0x7fce6c158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.904+0000 7fce6a7fc640 1 -- 172.21.15.38:0/2121181074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0ae360 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.904+0000 7fce6a7fc640 1 --2- 172.21.15.38:0/2121181074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce6c0cc5d0 0x7fce6c0c8ec0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fce5400d800 tx=0x7fce5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.904+0000 7fce637fe640 1 -- 172.21.15.38:0/2121181074 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54018020 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.904+0000 7fce637fe640 1 -- 172.21.15.38:0/2121181074 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce540096a0 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.904+0000 7fce714ac640 1 -- 172.21.15.38:0/2121181074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6c0ae640 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.904+0000 7fce637fe640 1 -- 172.21.15.38:0/2121181074 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54009a90 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.904+0000 7fce714ac640 1 -- 172.21.15.38:0/2121181074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6c0aec00 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.905+0000 7fce714ac640 1 -- 172.21.15.38:0/2121181074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6c0b6590 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.906+0000 7fce637fe640 1 -- 172.21.15.38:0/2121181074 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce5401b020 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.906+0000 7fce637fe640 1 --2- 172.21.15.38:0/2121181074 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce4c06b4d0 0x7fce4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:15.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.906+0000 7fce637fe640 1 -- 172.21.15.38:0/2121181074 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fce5408a8f0 con 0x7fce6c0cc5d0 2024-04-25T08:32:15.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.906+0000 7fce6affd640 1 --2- 172.21.15.38:0/2121181074 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce4c06b4d0 0x7fce4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:15.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.907+0000 7fce6affd640 1 --2- 172.21.15.38:0/2121181074 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce4c06b4d0 0x7fce4c06d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce5c033000 tx=0x7fce5c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:15.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:15.910+0000 7fce637fe640 1 -- 172.21.15.38:0/2121181074 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce54022030 con 0x7fce6c0cc5d0 2024-04-25T08:32:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.093+0000 7fce714ac640 1 -- 172.21.15.38:0/2121181074 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce6c06f200 con 0x7fce6c0cc5d0 2024-04-25T08:32:16.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.094+0000 7fce637fe640 1 -- 172.21.15.38:0/2121181074 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18969 (secure 0 0 0) 0x7fce540593a0 con 0x7fce6c0cc5d0 2024-04-25T08:32:16.091 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:16.091 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:15.565567+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.a","mappings":[{"from":4,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.097+0000 7fce714ac640 1 -- 172.21.15.38:0/2121181074 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce4c06b4d0 msgr2=0x7fce4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.097+0000 7fce714ac640 1 --2- 172.21.15.38:0/2121181074 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce4c06b4d0 0x7fce4c06d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fce5c033000 tx=0x7fce5c02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.097+0000 7fce714ac640 1 -- 172.21.15.38:0/2121181074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce6c0cc5d0 msgr2=0x7fce6c0c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.097+0000 7fce714ac640 1 --2- 172.21.15.38:0/2121181074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce6c0cc5d0 0x7fce6c0c8ec0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fce5400d800 tx=0x7fce5400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.097+0000 7fce714ac640 1 -- 172.21.15.38:0/2121181074 shutdown_connections 2024-04-25T08:32:16.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.097+0000 7fce714ac640 1 --2- 172.21.15.38:0/2121181074 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce6c0cf830 0x7fce6c158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.097+0000 7fce714ac640 1 --2- 172.21.15.38:0/2121181074 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce6c0cc5d0 0x7fce6c0c8ec0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.097+0000 7fce714ac640 1 --2- 172.21.15.38:0/2121181074 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce4c06b4d0 0x7fce4c06d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.097+0000 7fce714ac640 1 --2- 172.21.15.38:0/2121181074 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce6c0afae0 0x7fce6c153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.097+0000 7fce714ac640 1 -- 172.21.15.38:0/2121181074 >> 172.21.15.38:0/2121181074 conn(0x7fce6c099a70 msgr2=0x7fce6c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:16.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.097+0000 7fce714ac640 1 -- 172.21.15.38:0/2121181074 shutdown_connections 2024-04-25T08:32:16.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.097+0000 7fce714ac640 1 -- 172.21.15.38:0/2121181074 wait complete. 2024-04-25T08:32:16.104 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:16.105 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:16.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.212+0000 7f674ab34640 1 Processor -- start 2024-04-25T08:32:16.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.212+0000 7f674ab34640 1 -- start start 2024-04-25T08:32:16.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f674ab34640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67440d0320 0x7f67440d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f674ab34640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67440d2060 con 0x7f67440b6980 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f674ab34640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67440d2330 con 0x7f67440d0700 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f674ab34640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67440d2610 con 0x7f67440d0320 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f67490d1640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f67440b6980 0x7f67440b6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54126/0 (socket says 172.21.15.38:54126) 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f67490d1640 1 -- 172.21.15.38:0/1715039479 learned_addr learned my addr 172.21.15.38:0/1715039479 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f67488d0640 1 --2- 172.21.15.38:0/1715039479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67440d0320 0x7f67440d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f67437fe640 1 -- 172.21.15.38:0/1715039479 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3371124313 0 0) 0x7f67440d2060 con 0x7f67440b6980 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f67437fe640 1 -- 172.21.15.38:0/1715039479 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6734003520 con 0x7f67440b6980 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f67437fe640 1 -- 172.21.15.38:0/1715039479 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231349044 0 0) 0x7f67440d2330 con 0x7f67440d0700 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f67437fe640 1 -- 172.21.15.38:0/1715039479 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67440d2060 con 0x7f67440d0700 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f67488d0640 1 -- 172.21.15.38:0/1715039479 >> v1:172.21.15.107:6789/0 conn(0x7f67440d0700 legacy=0x7f67440d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f67488d0640 1 -- 172.21.15.38:0/1715039479 >> v1:172.21.15.38:6789/0 conn(0x7f67440b6980 legacy=0x7f67440b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f67437fe640 1 -- 172.21.15.38:0/1715039479 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3736081917 0 0) 0x7f6734003520 con 0x7f67440b6980 2024-04-25T08:32:16.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.213+0000 7f67488d0640 1 -- 172.21.15.38:0/1715039479 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440d28f0 con 0x7f67440d0320 2024-04-25T08:32:16.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.214+0000 7f67488d0640 1 --2- 172.21.15.38:0/1715039479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67440d0320 0x7f67440d33d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f67380097d0 tx=0x7f673802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d56034d6e8dcb682 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:16.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.214+0000 7f67437fe640 1 -- 172.21.15.38:0/1715039479 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6738037070 con 0x7f67440d0320 2024-04-25T08:32:16.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.214+0000 7f67437fe640 1 -- 172.21.15.38:0/1715039479 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673802acc0 con 0x7f67440d0320 2024-04-25T08:32:16.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.214+0000 7f67437fe640 1 -- 172.21.15.38:0/1715039479 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0019f0 con 0x7f67440d0320 2024-04-25T08:32:16.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.215+0000 7f674ab34640 1 -- 172.21.15.38:0/1715039479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67440d0320 msgr2=0x7f67440d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.215+0000 7f674ab34640 1 --2- 172.21.15.38:0/1715039479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67440d0320 0x7f67440d33d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f67380097d0 tx=0x7f673802a230 comp rx=0 tx=0).stop 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.215+0000 7f674ab34640 1 -- 172.21.15.38:0/1715039479 shutdown_connections 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.215+0000 7f674ab34640 1 --2- 172.21.15.38:0/1715039479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67440d0320 0x7f67440d33d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.215+0000 7f674ab34640 1 -- 172.21.15.38:0/1715039479 >> 172.21.15.38:0/1715039479 conn(0x7f67440668f0 msgr2=0x7f6744066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.215+0000 7f674ab34640 1 -- 172.21.15.38:0/1715039479 shutdown_connections 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.215+0000 7f674ab34640 1 -- 172.21.15.38:0/1715039479 wait complete. 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.216+0000 7f674ab34640 1 Processor -- start 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.216+0000 7f674ab34640 1 -- start start 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.217+0000 7f674ab34640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67440b6980 0x7f6744146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.217+0000 7f674ab34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67440d0320 0x7f6744149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.217+0000 7f674ab34640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67440d0700 0x7f6744149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.217+0000 7f674ab34640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67440d0b40 con 0x7f67440d0320 2024-04-25T08:32:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.217+0000 7f674ab34640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67440d2eb0 con 0x7f67440d0700 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.217+0000 7f674ab34640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67441471b0 con 0x7f67440b6980 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.217+0000 7f6743fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67440d0320 0x7f6744149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.217+0000 7f6743fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67440d0320 0x7f6744149540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37728/0 (socket says 172.21.15.38:37728) 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.217+0000 7f6743fff640 1 -- 172.21.15.38:0/1134993250 learned_addr learned my addr 172.21.15.38:0/1134993250 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.217+0000 7f67488d0640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67440b6980 0x7f6744146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.217+0000 7f67490d1640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67440d0700 0x7f6744149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.217+0000 7f6743fff640 1 -- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67440b6980 msgr2=0x7f6744146c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.218+0000 7f6743fff640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67440b6980 0x7f6744146c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.218+0000 7f6743fff640 1 -- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67440d0700 msgr2=0x7f6744149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.218+0000 7f6743fff640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67440d0700 0x7f6744149a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.218+0000 7f6743fff640 1 -- 172.21.15.38:0/1134993250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67441482e0 con 0x7f67440d0320 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.218+0000 7f6743fff640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67440d0320 0x7f6744149540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f673000c430 tx=0x7f673000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.218+0000 7f67417fa640 1 -- 172.21.15.38:0/1134993250 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6730017070 con 0x7f67440d0320 2024-04-25T08:32:16.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.218+0000 7f67417fa640 1 -- 172.21.15.38:0/1134993250 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67300052f0 con 0x7f67440d0320 2024-04-25T08:32:16.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.218+0000 7f67417fa640 1 -- 172.21.15.38:0/1134993250 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6730003830 con 0x7f67440d0320 2024-04-25T08:32:16.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.218+0000 7f674ab34640 1 -- 172.21.15.38:0/1134993250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67441485c0 con 0x7f67440d0320 2024-04-25T08:32:16.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.218+0000 7f674ab34640 1 -- 172.21.15.38:0/1134993250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6744148b80 con 0x7f67440d0320 2024-04-25T08:32:16.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.219+0000 7f674ab34640 1 -- 172.21.15.38:0/1134993250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6744068e80 con 0x7f67440d0320 2024-04-25T08:32:16.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.219+0000 7f67417fa640 1 -- 172.21.15.38:0/1134993250 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f673001a070 con 0x7f67440d0320 2024-04-25T08:32:16.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.220+0000 7f67417fa640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f672806b4d0 0x7f672806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.220+0000 7f67417fa640 1 -- 172.21.15.38:0/1134993250 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f6730089a70 con 0x7f67440d0320 2024-04-25T08:32:16.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.220+0000 7f67488d0640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f672806b4d0 0x7f672806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.220+0000 7f67488d0640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f672806b4d0 0x7f672806d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6738034000 tx=0x7f673802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:16.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.224+0000 7f67417fa640 1 -- 172.21.15.38:0/1134993250 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6730057da0 con 0x7f67440d0320 2024-04-25T08:32:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.406+0000 7f674ab34640 1 -- 172.21.15.38:0/1134993250 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67440b3050 con 0x7f67440d0320 2024-04-25T08:32:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.406+0000 7f67417fa640 1 -- 172.21.15.38:0/1134993250 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f673005be40 con 0x7f67440d0320 2024-04-25T08:32:16.403 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:16.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.409+0000 7f674ab34640 1 -- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f672806b4d0 msgr2=0x7f672806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.409+0000 7f674ab34640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f672806b4d0 0x7f672806d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6738034000 tx=0x7f673802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:16.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.409+0000 7f674ab34640 1 -- 172.21.15.38:0/1134993250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67440d0320 msgr2=0x7f6744149540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.409+0000 7f674ab34640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67440d0320 0x7f6744149540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f673000c430 tx=0x7f673000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.409+0000 7f674ab34640 1 -- 172.21.15.38:0/1134993250 shutdown_connections 2024-04-25T08:32:16.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.409+0000 7f674ab34640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f67440d0700 0x7f6744149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.409+0000 7f674ab34640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f67440d0320 0x7f6744149540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.409+0000 7f674ab34640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f672806b4d0 0x7f672806d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.409+0000 7f674ab34640 1 --2- 172.21.15.38:0/1134993250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f67440b6980 0x7f6744146c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.409+0000 7f674ab34640 1 -- 172.21.15.38:0/1134993250 >> 172.21.15.38:0/1134993250 conn(0x7f67440668f0 msgr2=0x7f67440d1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:16.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.409+0000 7f674ab34640 1 -- 172.21.15.38:0/1134993250 shutdown_connections 2024-04-25T08:32:16.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.409+0000 7f674ab34640 1 -- 172.21.15.38:0/1134993250 wait complete. 2024-04-25T08:32:16.416 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.523+0000 7f2c80a1f640 1 Processor -- start 2024-04-25T08:32:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.524+0000 7f2c80a1f640 1 -- start start 2024-04-25T08:32:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.524+0000 7f2c80a1f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2c7c0d33d0 0x7f2c7c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.524+0000 7f2c80a1f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2c7c0d1e90 0x7f2c7c0b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.524+0000 7f2c80a1f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7c0d2330 con 0x7f2c7c0d0320 2024-04-25T08:32:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.524+0000 7f2c80a1f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7c0d2610 con 0x7f2c7c0d33d0 2024-04-25T08:32:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.524+0000 7f2c80a1f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7c0d28f0 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.525+0000 7f2c79d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2c7c0d33d0 0x7f2c7c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.525+0000 7f2c79d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2c7c0d33d0 0x7f2c7c0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37734/0 (socket says 172.21.15.38:37734) 2024-04-25T08:32:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.525+0000 7f2c79d9b640 1 -- 172.21.15.38:0/4027546684 learned_addr learned my addr 172.21.15.38:0/4027546684 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.525+0000 7f2c7ad9d640 1 --2- 172.21.15.38:0/4027546684 >> v2:172.21.15.107:3300/0 conn(0x7f2c7c0d1e90 0x7f2c7c0b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.525+0000 7f2c79d9b640 1 -- 172.21.15.38:0/4027546684 >> v2:172.21.15.107:3300/0 conn(0x7f2c7c0d1e90 msgr2=0x7f2c7c0b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.525+0000 7f2c79d9b640 1 --2- 172.21.15.38:0/4027546684 >> v2:172.21.15.107:3300/0 conn(0x7f2c7c0d1e90 0x7f2c7c0b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.525+0000 7f2c79d9b640 1 -- 172.21.15.38:0/4027546684 >> v1:172.21.15.107:6789/0 conn(0x7f2c7c0d0320 legacy=0x7f2c7c0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:16.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.525+0000 7f2c79d9b640 1 -- 172.21.15.38:0/4027546684 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c0d2bd0 con 0x7f2c7c0d33d0 2024-04-25T08:32:16.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.526+0000 7f2c79d9b640 1 --2- 172.21.15.38:0/4027546684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c7c0d33d0 0x7f2c7c0d1950 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0ca800 tx=0x7f2c7002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ec9176ced36192 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:16.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.526+0000 7f2c7959a640 1 -- 172.21.15.38:0/4027546684 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70037070 con 0x7f2c7c0d33d0 2024-04-25T08:32:16.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.526+0000 7f2c7959a640 1 -- 172.21.15.38:0/4027546684 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7002ae50 con 0x7f2c7c0d33d0 2024-04-25T08:32:16.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.526+0000 7f2c7959a640 1 -- 172.21.15.38:0/4027546684 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70032860 con 0x7f2c7c0d33d0 2024-04-25T08:32:16.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.526+0000 7f2c80a1f640 1 -- 172.21.15.38:0/4027546684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c7c0d33d0 msgr2=0x7f2c7c0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.526+0000 7f2c80a1f640 1 --2- 172.21.15.38:0/4027546684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c7c0d33d0 0x7f2c7c0d1950 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0ca800 tx=0x7f2c7002a300 comp rx=0 tx=0).stop 2024-04-25T08:32:16.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.526+0000 7f2c80a1f640 1 -- 172.21.15.38:0/4027546684 shutdown_connections 2024-04-25T08:32:16.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.526+0000 7f2c80a1f640 1 --2- 172.21.15.38:0/4027546684 >> v2:172.21.15.107:3300/0 conn(0x7f2c7c0d1e90 0x7f2c7c0b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.526+0000 7f2c80a1f640 1 --2- 172.21.15.38:0/4027546684 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c7c0d33d0 0x7f2c7c0d1950 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.526+0000 7f2c80a1f640 1 -- 172.21.15.38:0/4027546684 >> 172.21.15.38:0/4027546684 conn(0x7f2c7c0668f0 msgr2=0x7f2c7c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:16.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.527+0000 7f2c80a1f640 1 -- 172.21.15.38:0/4027546684 shutdown_connections 2024-04-25T08:32:16.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.527+0000 7f2c80a1f640 1 -- 172.21.15.38:0/4027546684 wait complete. 2024-04-25T08:32:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.528+0000 7f2c80a1f640 1 Processor -- start 2024-04-25T08:32:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.528+0000 7f2c80a1f640 1 -- start start 2024-04-25T08:32:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c80a1f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c0d0320 0x7f2c7c14eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c80a1f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c7c0d1e90 0x7f2c7c15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c80a1f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c7c0d33d0 0x7f2c7c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c80a1f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c161390 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c80a1f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c161210 con 0x7f2c7c0d0320 2024-04-25T08:32:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c80a1f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7c0b9720 con 0x7f2c7c0d33d0 2024-04-25T08:32:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c79d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c7c0d1e90 0x7f2c7c15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c79d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c7c0d1e90 0x7f2c7c15c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37740/0 (socket says 172.21.15.38:37740) 2024-04-25T08:32:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c79d9b640 1 -- 172.21.15.38:0/1090275584 learned_addr learned my addr 172.21.15.38:0/1090275584 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c7a59c640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c0d0320 0x7f2c7c14eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c7ad9d640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c7c0d33d0 0x7f2c7c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c79d9b640 1 -- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c7c0d33d0 msgr2=0x7f2c7c15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c79d9b640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c7c0d33d0 0x7f2c7c15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c79d9b640 1 -- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c0d0320 msgr2=0x7f2c7c14eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c79d9b640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c0d0320 0x7f2c7c14eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c79d9b640 1 -- 172.21.15.38:0/1090275584 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7c159a90 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c79d9b640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c7c0d1e90 0x7f2c7c15c740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2c7002a340 tx=0x7f2c700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c6affd640 1 -- 172.21.15.38:0/1090275584 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70041070 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c6affd640 1 -- 172.21.15.38:0/1090275584 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7003c660 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c80a1f640 1 -- 172.21.15.38:0/1090275584 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7c1589f0 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.529+0000 7f2c6affd640 1 -- 172.21.15.38:0/1090275584 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70037040 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.530+0000 7f2c80a1f640 1 -- 172.21.15.38:0/1090275584 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7c158fb0 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.531+0000 7f2c80a1f640 1 -- 172.21.15.38:0/1090275584 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7c068e80 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.531+0000 7f2c6affd640 1 -- 172.21.15.38:0/1090275584 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c70044070 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.531+0000 7f2c6affd640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.532+0000 7f2c6affd640 1 -- 172.21.15.38:0/1090275584 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f2c700aacb0 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.532+0000 7f2c7a59c640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.532+0000 7f2c7a59c640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c001780 tx=0x7f2c6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:16.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.536+0000 7f2c6affd640 1 -- 172.21.15.38:0/1090275584 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c70078fe0 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.721+0000 7f2c80a1f640 1 -- 172.21.15.38:0/1090275584 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2c7c15da20 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.722+0000 7f2c6affd640 1 -- 172.21.15.38:0/1090275584 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2c7007d080 con 0x7f2c7c0d1e90 2024-04-25T08:32:16.718 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:16.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.723+0000 7f2c80a1f640 1 -- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2c5c06b4d0 msgr2=0x7f2c5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.723+0000 7f2c80a1f640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c001780 tx=0x7f2c6c009040 comp rx=0 tx=0).stop 2024-04-25T08:32:16.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.723+0000 7f2c80a1f640 1 -- 172.21.15.38:0/1090275584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c7c0d1e90 msgr2=0x7f2c7c15c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.723+0000 7f2c80a1f640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c7c0d1e90 0x7f2c7c15c740 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2c7002a340 tx=0x7f2c700046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.724+0000 7f2c80a1f640 1 -- 172.21.15.38:0/1090275584 shutdown_connections 2024-04-25T08:32:16.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.724+0000 7f2c80a1f640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.724+0000 7f2c80a1f640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2c7c0d33d0 0x7f2c7c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.724+0000 7f2c80a1f640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c7c0d1e90 0x7f2c7c15c740 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.724+0000 7f2c80a1f640 1 --2- 172.21.15.38:0/1090275584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c0d0320 0x7f2c7c14eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.724+0000 7f2c80a1f640 1 -- 172.21.15.38:0/1090275584 >> 172.21.15.38:0/1090275584 conn(0x7f2c7c0668f0 msgr2=0x7f2c7c0cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:16.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.724+0000 7f2c80a1f640 1 -- 172.21.15.38:0/1090275584 shutdown_connections 2024-04-25T08:32:16.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.724+0000 7f2c80a1f640 1 -- 172.21.15.38:0/1090275584 wait complete. 2024-04-25T08:32:16.731 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.838+0000 7fddb12fc640 1 Processor -- start 2024-04-25T08:32:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.838+0000 7fddb12fc640 1 -- start start 2024-04-25T08:32:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.839+0000 7fddb12fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac0cf580 0x7fddac15f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.839+0000 7fddb12fc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddac15fd00 con 0x7fddac0d1d20 2024-04-25T08:32:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.839+0000 7fddb12fc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddac15ffc0 con 0x7fddac15b990 2024-04-25T08:32:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.839+0000 7fddb12fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddac0cfa30 con 0x7fddac0cf580 2024-04-25T08:32:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.839+0000 7fddaa7fc640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fddac0d1d20 0x7fddac0b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54134/0 (socket says 172.21.15.38:54134) 2024-04-25T08:32:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.839+0000 7fddaa7fc640 1 -- 172.21.15.38:0/1012703965 learned_addr learned my addr 172.21.15.38:0/1012703965 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fddab7fe640 1 --2- 172.21.15.38:0/1012703965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac0cf580 0x7fddac15f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900838225 0 0) 0x7fddac15fd00 con 0x7fddac0d1d20 2024-04-25T08:32:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd98003520 con 0x7fddac0d1d20 2024-04-25T08:32:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 834501744 0 0) 0x7fdd98003520 con 0x7fddac0d1d20 2024-04-25T08:32:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac0cf580 msgr2=0x7fddac15f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:16.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 --2- 172.21.15.38:0/1012703965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac0cf580 0x7fddac15f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 >> v1:172.21.15.107:6789/0 conn(0x7fddac15b990 legacy=0x7fddac0d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddac1602a0 con 0x7fddac0d1d20 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fdd94001670 con 0x7fddac0d1d20 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 >> v1:172.21.15.38:6789/0 conn(0x7fddac0d1d20 legacy=0x7fddac0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 --2- 172.21.15.38:0/1012703965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd980039e0 0x7fdd98003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 --2- 172.21.15.38:0/1012703965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac0cf580 0x7fdd98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 --2- 172.21.15.38:0/1012703965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd98004d40 0x7fdd980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddac0cfa30 con 0x7fdd980039e0 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9800ba70 con 0x7fdd98004d40 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9800bc70 con 0x7fddac0cf580 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fddaaffd640 1 --2- 172.21.15.38:0/1012703965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd980039e0 0x7fdd98003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fddaa7fc640 1 --2- 172.21.15.38:0/1012703965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac0cf580 0x7fdd98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fddab7fe640 1 --2- 172.21.15.38:0/1012703965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd98004d40 0x7fdd980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fddaaffd640 1 -- 172.21.15.38:0/1012703965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac0cf580 msgr2=0x7fdd98004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fddaaffd640 1 --2- 172.21.15.38:0/1012703965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fddac0cf580 0x7fdd98004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fddaaffd640 1 -- 172.21.15.38:0/1012703965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd98004d40 msgr2=0x7fdd980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fddaaffd640 1 --2- 172.21.15.38:0/1012703965 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd98004d40 0x7fdd980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.840+0000 7fddaaffd640 1 -- 172.21.15.38:0/1012703965 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd98003520 con 0x7fdd980039e0 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.841+0000 7fddaaffd640 1 --2- 172.21.15.38:0/1012703965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd980039e0 0x7fdd98003dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdda4001860 tx=0x7fdd98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df2a42952271e0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.841+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda4037070 con 0x7fdd980039e0 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.841+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda402ac30 con 0x7fdd980039e0 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.841+0000 7fdda9ffb640 1 -- 172.21.15.38:0/1012703965 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda4001430 con 0x7fdd980039e0 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.842+0000 7fddb12fc640 1 -- 172.21.15.38:0/1012703965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd980039e0 msgr2=0x7fdd98003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.842+0000 7fddb12fc640 1 --2- 172.21.15.38:0/1012703965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd980039e0 0x7fdd98003dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdda4001860 tx=0x7fdd98005830 comp rx=0 tx=0).stop 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.842+0000 7fddab7fe640 1 -- 172.21.15.38:0/1012703965 reap_dead start 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.842+0000 7fddb12fc640 1 -- 172.21.15.38:0/1012703965 shutdown_connections 2024-04-25T08:32:16.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.842+0000 7fddb12fc640 1 -- 172.21.15.38:0/1012703965 >> 172.21.15.38:0/1012703965 conn(0x7fddac08b000 msgr2=0x7fddac08b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:16.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.842+0000 7fddb12fc640 1 -- 172.21.15.38:0/1012703965 shutdown_connections 2024-04-25T08:32:16.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.842+0000 7fddb12fc640 1 -- 172.21.15.38:0/1012703965 wait complete. 2024-04-25T08:32:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.843+0000 7fddb12fc640 1 Processor -- start 2024-04-25T08:32:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.843+0000 7fddb12fc640 1 -- start start 2024-04-25T08:32:16.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddb12fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd980039e0 0x7fddac14fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddb12fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddac15c9b0 0x7fddac15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddb12fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac15c0d0 0x7fddac15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddb12fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddac15bd70 con 0x7fddac15c0d0 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddb12fc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddac160580 con 0x7fddac15c9b0 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddb12fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddac15bef0 con 0x7fdd980039e0 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac15c0d0 0x7fddac15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac15c0d0 0x7fddac15d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37752/0 (socket says 172.21.15.38:37752) 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddab7fe640 1 -- 172.21.15.38:0/3162761088 learned_addr learned my addr 172.21.15.38:0/3162761088 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddaaffd640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd980039e0 0x7fddac14fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddaa7fc640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddac15c9b0 0x7fddac15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddab7fe640 1 -- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd980039e0 msgr2=0x7fddac14fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddab7fe640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd980039e0 0x7fddac14fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddab7fe640 1 -- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddac15c9b0 msgr2=0x7fddac15cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddab7fe640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddac15c9b0 0x7fddac15cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.844+0000 7fddab7fe640 1 -- 172.21.15.38:0/3162761088 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddac06d220 con 0x7fddac15c0d0 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.845+0000 7fddab7fe640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac15c0d0 0x7fddac15d4f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c011bf0 tx=0x7fdd9c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.845+0000 7fdda37fe640 1 -- 172.21.15.38:0/3162761088 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c00fe10 con 0x7fddac15c0d0 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.845+0000 7fdda37fe640 1 -- 172.21.15.38:0/3162761088 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9c01d070 con 0x7fddac15c0d0 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.845+0000 7fdda37fe640 1 -- 172.21.15.38:0/3162761088 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c015670 con 0x7fddac15c0d0 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.845+0000 7fddb12fc640 1 -- 172.21.15.38:0/3162761088 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddac06d500 con 0x7fddac15c0d0 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.845+0000 7fddb12fc640 1 -- 172.21.15.38:0/3162761088 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddac06dac0 con 0x7fddac15c0d0 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.846+0000 7fddb12fc640 1 -- 172.21.15.38:0/3162761088 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddac068e80 con 0x7fddac15c0d0 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.846+0000 7fdda37fe640 1 -- 172.21.15.38:0/3162761088 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd9c015810 con 0x7fddac15c0d0 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.847+0000 7fdda37fe640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd8c06b500 0x7fdd8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.847+0000 7fdda37fe640 1 -- 172.21.15.38:0/3162761088 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fdd9c08acf0 con 0x7fddac15c0d0 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.847+0000 7fddaaffd640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd8c06b500 0x7fdd8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:16.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.847+0000 7fddaaffd640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd8c06b500 0x7fdd8c06d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdda4034000 tx=0x7fdda4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:16.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:16.851+0000 7fdda37fe640 1 -- 172.21.15.38:0/3162761088 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd9c08f070 con 0x7fddac15c0d0 2024-04-25T08:32:17.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.029+0000 7fddb12fc640 1 -- 172.21.15.38:0/3162761088 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fddac159a60 con 0x7fddac15c0d0 2024-04-25T08:32:17.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.030+0000 7fdda37fe640 1 -- 172.21.15.38:0/3162761088 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd9c010040 con 0x7fddac15c0d0 2024-04-25T08:32:17.026 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:17.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.031+0000 7fddb12fc640 1 -- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd8c06b500 msgr2=0x7fdd8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.031+0000 7fddb12fc640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd8c06b500 0x7fdd8c06d9c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdda4034000 tx=0x7fdda4009030 comp rx=0 tx=0).stop 2024-04-25T08:32:17.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.031+0000 7fddb12fc640 1 -- 172.21.15.38:0/3162761088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac15c0d0 msgr2=0x7fddac15d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.031+0000 7fddb12fc640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac15c0d0 0x7fddac15d4f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c011bf0 tx=0x7fdd9c00f580 comp rx=0 tx=0).stop 2024-04-25T08:32:17.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.031+0000 7fddb12fc640 1 -- 172.21.15.38:0/3162761088 shutdown_connections 2024-04-25T08:32:17.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.031+0000 7fddb12fc640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fddac15c0d0 0x7fddac15d4f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.031+0000 7fddb12fc640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fddac15c9b0 0x7fddac15cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.031+0000 7fddb12fc640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd8c06b500 0x7fdd8c06d9c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.031+0000 7fddb12fc640 1 --2- 172.21.15.38:0/3162761088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd980039e0 0x7fddac14fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.031+0000 7fddb12fc640 1 -- 172.21.15.38:0/3162761088 >> 172.21.15.38:0/3162761088 conn(0x7fddac08b000 msgr2=0x7fddac06f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:17.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.032+0000 7fddb12fc640 1 -- 172.21.15.38:0/3162761088 shutdown_connections 2024-04-25T08:32:17.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.032+0000 7fddb12fc640 1 -- 172.21.15.38:0/3162761088 wait complete. 2024-04-25T08:32:17.038 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.146+0000 7f27119b0640 1 Processor -- start 2024-04-25T08:32:17.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.146+0000 7f27119b0640 1 -- start start 2024-04-25T08:32:17.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f27119b0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f270c0d0320 0x7f270c0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f27119b0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f270c0d0700 0x7f270c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f27119b0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f270c0d1e90 0x7f270c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f27119b0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270c0d2330 con 0x7f270c0d1e90 2024-04-25T08:32:17.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f27119b0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270c0d2610 con 0x7f270c0d0320 2024-04-25T08:32:17.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f27119b0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270c0d28f0 con 0x7f270c0d0700 2024-04-25T08:32:17.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f270b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f270c0d1e90 0x7f270c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f270b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f270c0d1e90 0x7f270c0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37760/0 (socket says 172.21.15.38:37760) 2024-04-25T08:32:17.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f270b7fe640 1 -- 172.21.15.38:0/1398043047 learned_addr learned my addr 172.21.15.38:0/1398043047 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:17.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f270affd640 1 --2- 172.21.15.38:0/1398043047 >> v2:172.21.15.107:3300/0 conn(0x7f270c0d0320 0x7f270c0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f270a7fc640 1 --2- 172.21.15.38:0/1398043047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f270c0d0700 0x7f270c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f270b7fe640 1 -- 172.21.15.38:0/1398043047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f270c0d0700 msgr2=0x7f270c0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:17.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f270b7fe640 1 --2- 172.21.15.38:0/1398043047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f270c0d0700 0x7f270c0d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f270b7fe640 1 -- 172.21.15.38:0/1398043047 >> v2:172.21.15.107:3300/0 conn(0x7f270c0d0320 msgr2=0x7f270c0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:17.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f270b7fe640 1 --2- 172.21.15.38:0/1398043047 >> v2:172.21.15.107:3300/0 conn(0x7f270c0d0320 0x7f270c0d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f270b7fe640 1 -- 172.21.15.38:0/1398043047 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c0d2bd0 con 0x7f270c0d1e90 2024-04-25T08:32:17.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f270affd640 1 --2- 172.21.15.38:0/1398043047 >> v2:172.21.15.107:3300/0 conn(0x7f270c0d0320 0x7f270c0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:17.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.147+0000 7f270a7fc640 1 --2- 172.21.15.38:0/1398043047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f270c0d0700 0x7f270c0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:17.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.148+0000 7f270b7fe640 1 --2- 172.21.15.38:0/1398043047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270c0d1e90 0x7f270c0b5390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f270c069100 tx=0x7f270402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66b4f224486a1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.148+0000 7f2709ffb640 1 -- 172.21.15.38:0/1398043047 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704036070 con 0x7f270c0d1e90 2024-04-25T08:32:17.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.148+0000 7f2709ffb640 1 -- 172.21.15.38:0/1398043047 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2704031600 con 0x7f270c0d1e90 2024-04-25T08:32:17.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.148+0000 7f2709ffb640 1 -- 172.21.15.38:0/1398043047 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27040319d0 con 0x7f270c0d1e90 2024-04-25T08:32:17.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.149+0000 7f27119b0640 1 -- 172.21.15.38:0/1398043047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270c0d1e90 msgr2=0x7f270c0b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.149+0000 7f27119b0640 1 --2- 172.21.15.38:0/1398043047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270c0d1e90 0x7f270c0b5390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f270c069100 tx=0x7f270402a730 comp rx=0 tx=0).stop 2024-04-25T08:32:17.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.149+0000 7f27119b0640 1 -- 172.21.15.38:0/1398043047 shutdown_connections 2024-04-25T08:32:17.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.149+0000 7f27119b0640 1 --2- 172.21.15.38:0/1398043047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270c0d1e90 0x7f270c0b5390 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.149+0000 7f27119b0640 1 --2- 172.21.15.38:0/1398043047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f270c0d0700 0x7f270c0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.149+0000 7f27119b0640 1 --2- 172.21.15.38:0/1398043047 >> v2:172.21.15.107:3300/0 conn(0x7f270c0d0320 0x7f270c0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.149+0000 7f27119b0640 1 -- 172.21.15.38:0/1398043047 >> 172.21.15.38:0/1398043047 conn(0x7f270c0668f0 msgr2=0x7f270c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:17.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.149+0000 7f27119b0640 1 -- 172.21.15.38:0/1398043047 shutdown_connections 2024-04-25T08:32:17.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.149+0000 7f27119b0640 1 -- 172.21.15.38:0/1398043047 wait complete. 2024-04-25T08:32:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.150+0000 7f27119b0640 1 Processor -- start 2024-04-25T08:32:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.150+0000 7f27119b0640 1 -- start start 2024-04-25T08:32:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f27119b0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f270c0d0320 0x7f270c154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f27119b0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c0d0700 0x7f270c0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f27119b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270c0d1e90 0x7f270c0b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f27119b0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c0d0b40 con 0x7f270c0d1e90 2024-04-25T08:32:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f27119b0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c161210 con 0x7f270c0d0700 2024-04-25T08:32:17.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f27119b0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270c0b3ca0 con 0x7f270c0d0320 2024-04-25T08:32:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f270b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270c0d1e90 0x7f270c0b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f270b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270c0d1e90 0x7f270c0b55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37770/0 (socket says 172.21.15.38:37770) 2024-04-25T08:32:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f270b7fe640 1 -- 172.21.15.38:0/891007631 learned_addr learned my addr 172.21.15.38:0/891007631 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f270affd640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f270c0d0320 0x7f270c154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f270a7fc640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c0d0700 0x7f270c0b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f270b7fe640 1 -- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f270c0d0320 msgr2=0x7f270c154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f270b7fe640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f270c0d0320 0x7f270c154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f270b7fe640 1 -- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c0d0700 msgr2=0x7f270c0b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f270b7fe640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c0d0700 0x7f270c0b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f270b7fe640 1 -- 172.21.15.38:0/891007631 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c0b3f80 con 0x7f270c0d1e90 2024-04-25T08:32:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.151+0000 7f270b7fe640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270c0d1e90 0x7f270c0b55a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2704009f40 tx=0x7f2704004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.152+0000 7f26f3fff640 1 -- 172.21.15.38:0/891007631 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704036050 con 0x7f270c0d1e90 2024-04-25T08:32:17.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.152+0000 7f26f3fff640 1 -- 172.21.15.38:0/891007631 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2704042070 con 0x7f270c0d1e90 2024-04-25T08:32:17.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.152+0000 7f26f3fff640 1 -- 172.21.15.38:0/891007631 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704009510 con 0x7f270c0d1e90 2024-04-25T08:32:17.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.152+0000 7f27119b0640 1 -- 172.21.15.38:0/891007631 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f270c0b4260 con 0x7f270c0d1e90 2024-04-25T08:32:17.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.152+0000 7f27119b0640 1 -- 172.21.15.38:0/891007631 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f270c0b4820 con 0x7f270c0d1e90 2024-04-25T08:32:17.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.153+0000 7f27119b0640 1 -- 172.21.15.38:0/891007631 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270c0b6510 con 0x7f270c0d1e90 2024-04-25T08:32:17.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.153+0000 7f26f3fff640 1 -- 172.21.15.38:0/891007631 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2704007bf0 con 0x7f270c0d1e90 2024-04-25T08:32:17.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.154+0000 7f26f3fff640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f26f806b4d0 0x7f26f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.154+0000 7f26f3fff640 1 -- 172.21.15.38:0/891007631 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f27040ab3c0 con 0x7f270c0d1e90 2024-04-25T08:32:17.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.154+0000 7f270affd640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f26f806b4d0 0x7f26f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.154+0000 7f270affd640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f26f806b4d0 0x7f26f806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f270c1f7100 tx=0x7f26fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.158+0000 7f26f3fff640 1 -- 172.21.15.38:0/891007631 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f270c0b6510 con 0x7f270c0d1e90 2024-04-25T08:32:17.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.341+0000 7f27119b0640 1 -- 172.21.15.38:0/891007631 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f270c068e80 con 0x7f270c0d1e90 2024-04-25T08:32:17.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.342+0000 7f26f3fff640 1 -- 172.21.15.38:0/891007631 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2704079790 con 0x7f270c0d1e90 2024-04-25T08:32:17.338 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:17.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.344+0000 7f27119b0640 1 -- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f26f806b4d0 msgr2=0x7f26f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.344+0000 7f27119b0640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f26f806b4d0 0x7f26f806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f270c1f7100 tx=0x7f26fc008040 comp rx=0 tx=0).stop 2024-04-25T08:32:17.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.344+0000 7f27119b0640 1 -- 172.21.15.38:0/891007631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270c0d1e90 msgr2=0x7f270c0b55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.344+0000 7f27119b0640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270c0d1e90 0x7f270c0b55a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2704009f40 tx=0x7f2704004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:17.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.344+0000 7f27119b0640 1 -- 172.21.15.38:0/891007631 shutdown_connections 2024-04-25T08:32:17.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.344+0000 7f27119b0640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f270c0d1e90 0x7f270c0b55a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.344+0000 7f27119b0640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c0d0700 0x7f270c0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.344+0000 7f27119b0640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f26f806b4d0 0x7f26f806d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.344+0000 7f27119b0640 1 --2- 172.21.15.38:0/891007631 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f270c0d0320 0x7f270c154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.344+0000 7f27119b0640 1 -- 172.21.15.38:0/891007631 >> 172.21.15.38:0/891007631 conn(0x7f270c0668f0 msgr2=0x7f270c0b7ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:17.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.344+0000 7f27119b0640 1 -- 172.21.15.38:0/891007631 shutdown_connections 2024-04-25T08:32:17.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.344+0000 7f27119b0640 1 -- 172.21.15.38:0/891007631 wait complete. 2024-04-25T08:32:17.351 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T08:32:17.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.456+0000 7f4dbaaac640 1 Processor -- start 2024-04-25T08:32:17.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.456+0000 7f4dbaaac640 1 -- start start 2024-04-25T08:32:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.457+0000 7f4dbaaac640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4db4099a70 0x7f4db4099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.457+0000 7f4dbaaac640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db409a390 con 0x7f4db406cd90 2024-04-25T08:32:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.457+0000 7f4dbaaac640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db40ccd60 con 0x7f4db40cf470 2024-04-25T08:32:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.457+0000 7f4dbaaac640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db41614d0 con 0x7f4db4099a70 2024-04-25T08:32:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.457+0000 7f4db9049640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4db406cd90 0x7f4db4167710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54150/0 (socket says 172.21.15.38:54150) 2024-04-25T08:32:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.457+0000 7f4db9049640 1 -- 172.21.15.38:0/3920179391 learned_addr learned my addr 172.21.15.38:0/3920179391 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:17.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.457+0000 7f4db8848640 1 --2- 172.21.15.38:0/3920179391 >> v2:172.21.15.107:3300/0 conn(0x7f4db4099a70 0x7f4db4099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.457+0000 7f4db37fe640 1 -- 172.21.15.38:0/3920179391 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055007837 0 0) 0x7f4db409a390 con 0x7f4db406cd90 2024-04-25T08:32:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.457+0000 7f4db37fe640 1 -- 172.21.15.38:0/3920179391 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4da0003520 con 0x7f4db406cd90 2024-04-25T08:32:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.458+0000 7f4db37fe640 1 -- 172.21.15.38:0/3920179391 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3448050784 0 0) 0x7f4da0003520 con 0x7f4db406cd90 2024-04-25T08:32:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.458+0000 7f4db8848640 1 -- 172.21.15.38:0/3920179391 >> v1:172.21.15.107:6789/0 conn(0x7f4db40cf470 legacy=0x7f4db40cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.458+0000 7f4db8848640 1 -- 172.21.15.38:0/3920179391 >> v1:172.21.15.38:6789/0 conn(0x7f4db406cd90 legacy=0x7f4db4167710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.458+0000 7f4db8848640 1 -- 172.21.15.38:0/3920179391 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db40cd040 con 0x7f4db4099a70 2024-04-25T08:32:17.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.459+0000 7f4db8848640 1 --2- 172.21.15.38:0/3920179391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4db4099a70 0x7f4db4099e50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0097a0 tx=0x7f4dac02a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e74093b0e453320 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.459+0000 7f4db37fe640 1 -- 172.21.15.38:0/3920179391 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac037070 con 0x7f4db4099a70 2024-04-25T08:32:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.459+0000 7f4db37fe640 1 -- 172.21.15.38:0/3920179391 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac02ab60 con 0x7f4db4099a70 2024-04-25T08:32:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.459+0000 7f4db37fe640 1 -- 172.21.15.38:0/3920179391 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c001670 con 0x7f4db4099a70 2024-04-25T08:32:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.459+0000 7f4dbaaac640 1 -- 172.21.15.38:0/3920179391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4db4099a70 msgr2=0x7f4db4099e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.459+0000 7f4dbaaac640 1 --2- 172.21.15.38:0/3920179391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4db4099a70 0x7f4db4099e50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0097a0 tx=0x7f4dac02a0d0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.459+0000 7f4dbaaac640 1 -- 172.21.15.38:0/3920179391 shutdown_connections 2024-04-25T08:32:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.459+0000 7f4dbaaac640 1 --2- 172.21.15.38:0/3920179391 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4db4099a70 0x7f4db4099e50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.459+0000 7f4dbaaac640 1 -- 172.21.15.38:0/3920179391 >> 172.21.15.38:0/3920179391 conn(0x7f4db40b3180 msgr2=0x7f4db40b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.459+0000 7f4dbaaac640 1 -- 172.21.15.38:0/3920179391 shutdown_connections 2024-04-25T08:32:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.459+0000 7f4dbaaac640 1 -- 172.21.15.38:0/3920179391 wait complete. 2024-04-25T08:32:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.460+0000 7f4dbaaac640 1 Processor -- start 2024-04-25T08:32:17.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.461+0000 7f4dbaaac640 1 -- start start 2024-04-25T08:32:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.461+0000 7f4dbaaac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4db406cd90 0x7f4db4156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.461+0000 7f4dbaaac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4db4099a70 0x7f4db40c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.461+0000 7f4dbaaac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4db40cf470 0x7f4db409d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.461+0000 7f4dbaaac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40cd600 con 0x7f4db4099a70 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.461+0000 7f4dbaaac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40cd970 con 0x7f4db40cf470 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.461+0000 7f4dbaaac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40d0780 con 0x7f4db406cd90 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.461+0000 7f4db3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4db4099a70 0x7f4db40c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4db4099a70 0x7f4db40c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37774/0 (socket says 172.21.15.38:37774) 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db3fff640 1 -- 172.21.15.38:0/1094458364 learned_addr learned my addr 172.21.15.38:0/1094458364 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db8848640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4db406cd90 0x7f4db4156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db9049640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4db40cf470 0x7f4db409d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db3fff640 1 -- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4db406cd90 msgr2=0x7f4db4156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db3fff640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4db406cd90 0x7f4db4156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db3fff640 1 -- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4db40cf470 msgr2=0x7f4db409d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db3fff640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4db40cf470 0x7f4db409d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db3fff640 1 -- 172.21.15.38:0/1094458364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db4156f90 con 0x7f4db4099a70 2024-04-25T08:32:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db8848640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4db406cd90 0x7f4db4156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db9049640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4db40cf470 0x7f4db409d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db3fff640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4db4099a70 0x7f4db40c6470 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4da400c870 tx=0x7f4da400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db17fa640 1 -- 172.21.15.38:0/1094458364 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da4017070 con 0x7f4db4099a70 2024-04-25T08:32:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4dbaaac640 1 -- 172.21.15.38:0/1094458364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db40c50f0 con 0x7f4db4099a70 2024-04-25T08:32:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db17fa640 1 -- 172.21.15.38:0/1094458364 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da4003da0 con 0x7f4db4099a70 2024-04-25T08:32:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4db17fa640 1 -- 172.21.15.38:0/1094458364 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da4005290 con 0x7f4db4099a70 2024-04-25T08:32:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.462+0000 7f4dbaaac640 1 -- 172.21.15.38:0/1094458364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db40c56b0 con 0x7f4db4099a70 2024-04-25T08:32:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.463+0000 7f4dbaaac640 1 -- 172.21.15.38:0/1094458364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db40cee40 con 0x7f4db4099a70 2024-04-25T08:32:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.464+0000 7f4db17fa640 1 -- 172.21.15.38:0/1094458364 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4da401a070 con 0x7f4db4099a70 2024-04-25T08:32:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.464+0000 7f4db17fa640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d9406b4d0 0x7f4d9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.464+0000 7f4db17fa640 1 -- 172.21.15.38:0/1094458364 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f4da4089d50 con 0x7f4db4099a70 2024-04-25T08:32:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.464+0000 7f4db8848640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d9406b4d0 0x7f4d9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.465+0000 7f4db8848640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d9406b4d0 0x7f4d9406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4dac034000 tx=0x7f4dac02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:17.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.468+0000 7f4db17fa640 1 -- 172.21.15.38:0/1094458364 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4db40cee40 con 0x7f4db4099a70 2024-04-25T08:32:17.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:17.723+0000 7f4dbaaac640 1 -- 172.21.15.38:0/1094458364 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f4db409abc0 con 0x7f4db4099a70 2024-04-25T08:32:18.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.603+0000 7f4db17fa640 1 -- 172.21.15.38:0/1094458364 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v63) v1 ==== 151+0+0 (secure 0 0 0) 0x7f4da408d030 con 0x7f4db4099a70 2024-04-25T08:32:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.606+0000 7f4dbaaac640 1 -- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d9406b4d0 msgr2=0x7f4d9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.606+0000 7f4dbaaac640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d9406b4d0 0x7f4d9406d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4dac034000 tx=0x7f4dac02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:18.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.606+0000 7f4dbaaac640 1 -- 172.21.15.38:0/1094458364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4db4099a70 msgr2=0x7f4db40c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.606+0000 7f4dbaaac640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4db4099a70 0x7f4db40c6470 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4da400c870 tx=0x7f4da400cd30 comp rx=0 tx=0).stop 2024-04-25T08:32:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.606+0000 7f4dbaaac640 1 -- 172.21.15.38:0/1094458364 shutdown_connections 2024-04-25T08:32:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.606+0000 7f4dbaaac640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4db40cf470 0x7f4db409d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.606+0000 7f4dbaaac640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4db4099a70 0x7f4db40c6470 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.606+0000 7f4dbaaac640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d9406b4d0 0x7f4d9406d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.606+0000 7f4dbaaac640 1 --2- 172.21.15.38:0/1094458364 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4db406cd90 0x7f4db4156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.606+0000 7f4dbaaac640 1 -- 172.21.15.38:0/1094458364 >> 172.21.15.38:0/1094458364 conn(0x7f4db40b3180 msgr2=0x7f4db40c87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.606+0000 7f4dbaaac640 1 -- 172.21.15.38:0/1094458364 shutdown_connections 2024-04-25T08:32:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.606+0000 7f4dbaaac640 1 -- 172.21.15.38:0/1094458364 wait complete. 2024-04-25T08:32:18.616 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:32:18.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.730+0000 7fd714c1f640 1 Processor -- start 2024-04-25T08:32:18.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.730+0000 7fd714c1f640 1 -- start start 2024-04-25T08:32:18.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.730+0000 7fd714c1f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd7100d0320 0x7fd7100d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.730+0000 7fd714c1f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd7100d0700 0x7fd7100d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.730+0000 7fd714c1f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7100d1e90 0x7fd7100b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.730+0000 7fd714c1f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7100d2330 con 0x7fd7100d0320 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.730+0000 7fd714c1f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7100d2610 con 0x7fd7100d0700 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.730+0000 7fd714c1f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100d28f0 con 0x7fd7100d1e90 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.731+0000 7fd70e59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd7100d0320 0x7fd7100d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.731+0000 7fd70e59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd7100d0320 0x7fd7100d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37782/0 (socket says 172.21.15.38:37782) 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.731+0000 7fd70e59c640 1 -- 172.21.15.38:0/4262769148 learned_addr learned my addr 172.21.15.38:0/4262769148 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.731+0000 7fd707fff640 1 --2- 172.21.15.38:0/4262769148 >> v2:172.21.15.107:3300/0 conn(0x7fd7100d0700 0x7fd7100d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.731+0000 7fd70ed9d640 1 --2- 172.21.15.38:0/4262769148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7100d1e90 0x7fd7100b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.731+0000 7fd70e59c640 1 -- 172.21.15.38:0/4262769148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7100d1e90 msgr2=0x7fd7100b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.731+0000 7fd70e59c640 1 --2- 172.21.15.38:0/4262769148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7100d1e90 0x7fd7100b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.731+0000 7fd70e59c640 1 -- 172.21.15.38:0/4262769148 >> v2:172.21.15.107:3300/0 conn(0x7fd7100d0700 msgr2=0x7fd7100d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.731+0000 7fd70e59c640 1 --2- 172.21.15.38:0/4262769148 >> v2:172.21.15.107:3300/0 conn(0x7fd7100d0700 0x7fd7100d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.731+0000 7fd70e59c640 1 -- 172.21.15.38:0/4262769148 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7100d2bd0 con 0x7fd7100d0320 2024-04-25T08:32:18.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.732+0000 7fd70e59c640 1 --2- 172.21.15.38:0/4262769148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7100d0320 0x7fd7100d33d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd7100b6410 tx=0x7fd70002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e70d449e94364a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:18.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.732+0000 7fd70dd9b640 1 -- 172.21.15.38:0/4262769148 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700037070 con 0x7fd7100d0320 2024-04-25T08:32:18.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.732+0000 7fd70dd9b640 1 -- 172.21.15.38:0/4262769148 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd700032440 con 0x7fd7100d0320 2024-04-25T08:32:18.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.732+0000 7fd70dd9b640 1 -- 172.21.15.38:0/4262769148 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700032810 con 0x7fd7100d0320 2024-04-25T08:32:18.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.732+0000 7fd714c1f640 1 -- 172.21.15.38:0/4262769148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7100d0320 msgr2=0x7fd7100d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.732+0000 7fd714c1f640 1 --2- 172.21.15.38:0/4262769148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7100d0320 0x7fd7100d33d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd7100b6410 tx=0x7fd70002a270 comp rx=0 tx=0).stop 2024-04-25T08:32:18.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.732+0000 7fd714c1f640 1 -- 172.21.15.38:0/4262769148 shutdown_connections 2024-04-25T08:32:18.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.732+0000 7fd714c1f640 1 --2- 172.21.15.38:0/4262769148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7100d1e90 0x7fd7100b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.732+0000 7fd714c1f640 1 --2- 172.21.15.38:0/4262769148 >> v2:172.21.15.107:3300/0 conn(0x7fd7100d0700 0x7fd7100d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.732+0000 7fd714c1f640 1 --2- 172.21.15.38:0/4262769148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7100d0320 0x7fd7100d33d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.732+0000 7fd714c1f640 1 -- 172.21.15.38:0/4262769148 >> 172.21.15.38:0/4262769148 conn(0x7fd7100668f0 msgr2=0x7fd710066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:18.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.732+0000 7fd714c1f640 1 -- 172.21.15.38:0/4262769148 shutdown_connections 2024-04-25T08:32:18.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.733+0000 7fd714c1f640 1 -- 172.21.15.38:0/4262769148 wait complete. 2024-04-25T08:32:18.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.734+0000 7fd714c1f640 1 Processor -- start 2024-04-25T08:32:18.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.734+0000 7fd714c1f640 1 -- start start 2024-04-25T08:32:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd714c1f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7100d0320 0x7fd7101f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd714c1f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7100d0700 0x7fd7100cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd714c1f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7100d1e90 0x7fd7101e9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd714c1f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100d0b40 con 0x7fd7100d0320 2024-04-25T08:32:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd714c1f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd710161210 con 0x7fd7100d1e90 2024-04-25T08:32:18.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd714c1f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100cb930 con 0x7fd7100d0700 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd70e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7100d0320 0x7fd7101f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd70e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7100d0320 0x7fd7101f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37798/0 (socket says 172.21.15.38:37798) 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd70e59c640 1 -- 172.21.15.38:0/3114545816 learned_addr learned my addr 172.21.15.38:0/3114545816 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd707fff640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7100d0700 0x7fd7100cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd70ed9d640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7100d1e90 0x7fd7101e9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd70e59c640 1 -- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7100d0700 msgr2=0x7fd7100cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd70e59c640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7100d0700 0x7fd7100cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd70e59c640 1 -- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7100d1e90 msgr2=0x7fd7101e9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd70e59c640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7100d1e90 0x7fd7101e9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd70e59c640 1 -- 172.21.15.38:0/3114545816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7100b35c0 con 0x7fd7100d0320 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd70e59c640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7100d0320 0x7fd7101f5080 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd700000c00 tx=0x7fd7000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd706ffd640 1 -- 172.21.15.38:0/3114545816 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700041030 con 0x7fd7100d0320 2024-04-25T08:32:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd706ffd640 1 -- 172.21.15.38:0/3114545816 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd700030440 con 0x7fd7100d0320 2024-04-25T08:32:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd706ffd640 1 -- 172.21.15.38:0/3114545816 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700037040 con 0x7fd7100d0320 2024-04-25T08:32:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd714c1f640 1 -- 172.21.15.38:0/3114545816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7100b38a0 con 0x7fd7100d0320 2024-04-25T08:32:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.735+0000 7fd714c1f640 1 -- 172.21.15.38:0/3114545816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7100b3e60 con 0x7fd7100d0320 2024-04-25T08:32:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.736+0000 7fd714c1f640 1 -- 172.21.15.38:0/3114545816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd710068e80 con 0x7fd7100d0320 2024-04-25T08:32:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.737+0000 7fd706ffd640 1 -- 172.21.15.38:0/3114545816 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd700044070 con 0x7fd7100d0320 2024-04-25T08:32:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.737+0000 7fd706ffd640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd6f006b4d0 0x7fd6f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.737+0000 7fd706ffd640 1 -- 172.21.15.38:0/3114545816 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fd7000aad90 con 0x7fd7100d0320 2024-04-25T08:32:18.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.737+0000 7fd707fff640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd6f006b4d0 0x7fd6f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:18.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.738+0000 7fd707fff640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd6f006b4d0 0x7fd6f006d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd7100ca100 tx=0x7fd6f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:18.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.741+0000 7fd706ffd640 1 -- 172.21.15.38:0/3114545816 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7000798c0 con 0x7fd7100d0320 2024-04-25T08:32:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.920+0000 7fd714c1f640 1 -- 172.21.15.38:0/3114545816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd71015fb80 con 0x7fd7100d0320 2024-04-25T08:32:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.927+0000 7fd706ffd640 1 -- 172.21.15.38:0/3114545816 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd7000791c0 con 0x7fd7100d0320 2024-04-25T08:32:18.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.929+0000 7fd714c1f640 1 -- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd6f006b4d0 msgr2=0x7fd6f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.929+0000 7fd714c1f640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd6f006b4d0 0x7fd6f006d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd7100ca100 tx=0x7fd6f8008040 comp rx=0 tx=0).stop 2024-04-25T08:32:18.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.929+0000 7fd714c1f640 1 -- 172.21.15.38:0/3114545816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7100d0320 msgr2=0x7fd7101f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:18.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.929+0000 7fd714c1f640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7100d0320 0x7fd7101f5080 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd700000c00 tx=0x7fd7000046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.930+0000 7fd714c1f640 1 -- 172.21.15.38:0/3114545816 shutdown_connections 2024-04-25T08:32:18.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.930+0000 7fd714c1f640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7100d1e90 0x7fd7101e9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.930+0000 7fd714c1f640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd6f006b4d0 0x7fd6f006d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.930+0000 7fd714c1f640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7100d0700 0x7fd7100cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.930+0000 7fd714c1f640 1 --2- 172.21.15.38:0/3114545816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7100d0320 0x7fd7101f5080 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:18.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.930+0000 7fd714c1f640 1 -- 172.21.15.38:0/3114545816 >> 172.21.15.38:0/3114545816 conn(0x7fd7100668f0 msgr2=0x7fd7100b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:18.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.930+0000 7fd714c1f640 1 -- 172.21.15.38:0/3114545816 shutdown_connections 2024-04-25T08:32:18.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:18.930+0000 7fd714c1f640 1 -- 172.21.15.38:0/3114545816 wait complete. 2024-04-25T08:32:18.937 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:32:18.937 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:32:18.977 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:32:19.028 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:32:19.028 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:32:19.028 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:19.044 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:32:19.044 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:19.100 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:19.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.245+0000 7f4bdebc1640 1 Processor -- start 2024-04-25T08:32:19.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.245+0000 7f4bdebc1640 1 -- start start 2024-04-25T08:32:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bdebc1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4bd806c110 0x7f4bd806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bdebc1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd806ca30 con 0x7f4bd815df50 2024-04-25T08:32:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bdebc1640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd80ceff0 con 0x7f4bd80ce2e0 2024-04-25T08:32:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bdebc1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd80cf2d0 con 0x7f4bd806c110 2024-04-25T08:32:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bd7fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4bd815df50 0x7f4bd80ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34882/0 (socket says 172.21.15.38:34882) 2024-04-25T08:32:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bd7fff640 1 -- 172.21.15.38:0/507936721 learned_addr learned my addr 172.21.15.38:0/507936721 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762289771 0 0) 0x7f4bd806ca30 con 0x7f4bd815df50 2024-04-25T08:32:19.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bdc95d640 1 --2- 172.21.15.38:0/507936721 >> v2:172.21.15.107:3300/0 conn(0x7f4bd806c110 0x7f4bd806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bc8003520 con 0x7f4bd815df50 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2133421724 0 0) 0x7f4bc8003520 con 0x7f4bd815df50 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 >> v2:172.21.15.107:3300/0 conn(0x7f4bd806c110 msgr2=0x7f4bd806c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bd77fe640 1 --2- 172.21.15.38:0/507936721 >> v2:172.21.15.107:3300/0 conn(0x7f4bd806c110 0x7f4bd806c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 >> v1:172.21.15.107:6789/0 conn(0x7f4bd80ce2e0 legacy=0x7f4bd80d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd80d1f70 con 0x7f4bd815df50 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f4bc4001670 con 0x7f4bd815df50 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.246+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 >> v1:172.21.15.38:6789/0 conn(0x7f4bd815df50 legacy=0x7f4bd80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bd77fe640 1 --2- 172.21.15.38:0/507936721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bc80039b0 0x7f4bc8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bd77fe640 1 --2- 172.21.15.38:0/507936721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bc80043b0 0x7f4bc8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bd77fe640 1 --2- 172.21.15.38:0/507936721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 0x7f4bc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80cf2d0 con 0x7f4bc8005120 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc800c060 con 0x7f4bc80039b0 2024-04-25T08:32:19.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc800c260 con 0x7f4bc80043b0 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bdd15e640 1 --2- 172.21.15.38:0/507936721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 0x7f4bc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bdc95d640 1 --2- 172.21.15.38:0/507936721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bc80039b0 0x7f4bc8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bd7fff640 1 --2- 172.21.15.38:0/507936721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bc80043b0 0x7f4bc8004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bdd15e640 1 -- 172.21.15.38:0/507936721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bc80043b0 msgr2=0x7f4bc8004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bdd15e640 1 --2- 172.21.15.38:0/507936721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bc80043b0 0x7f4bc8004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bdd15e640 1 -- 172.21.15.38:0/507936721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bc80039b0 msgr2=0x7f4bc8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bdd15e640 1 --2- 172.21.15.38:0/507936721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bc80039b0 0x7f4bc8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.247+0000 7f4bdd15e640 1 -- 172.21.15.38:0/507936721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc8003520 con 0x7f4bc8005120 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.248+0000 7f4bdd15e640 1 -- 172.21.15.38:0/507936721 reap_dead start 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.248+0000 7f4bdd15e640 1 --2- 172.21.15.38:0/507936721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 0x7f4bc800b9c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0001e50 tx=0x7f4bc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c29852663238f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.248+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0001a70 con 0x7f4bc8005120 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.248+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd0003970 con 0x7f4bc8005120 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.248+0000 7f4bd77fe640 1 -- 172.21.15.38:0/507936721 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0003d60 con 0x7f4bc8005120 2024-04-25T08:32:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.248+0000 7f4bdebc1640 1 -- 172.21.15.38:0/507936721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 msgr2=0x7f4bc800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.248+0000 7f4bdebc1640 1 --2- 172.21.15.38:0/507936721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 0x7f4bc800b9c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0001e50 tx=0x7f4bc8005830 comp rx=0 tx=0).stop 2024-04-25T08:32:19.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.248+0000 7f4bdebc1640 1 -- 172.21.15.38:0/507936721 shutdown_connections 2024-04-25T08:32:19.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.248+0000 7f4bdebc1640 1 --2- 172.21.15.38:0/507936721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 0x7f4bc800b9c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.248+0000 7f4bdebc1640 1 -- 172.21.15.38:0/507936721 >> 172.21.15.38:0/507936721 conn(0x7f4bd80d3fb0 msgr2=0x7f4bd80d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.248+0000 7f4bdebc1640 1 -- 172.21.15.38:0/507936721 shutdown_connections 2024-04-25T08:32:19.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.248+0000 7f4bdebc1640 1 -- 172.21.15.38:0/507936721 wait complete. 2024-04-25T08:32:19.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.250+0000 7f4bdebc1640 1 Processor -- start 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.250+0000 7f4bdebc1640 1 -- start start 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.250+0000 7f4bdebc1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 0x7f4bd80b7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.250+0000 7f4bdebc1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd815f190 0x7f4bd81f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.250+0000 7f4bdebc1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bd80c96c0 0x7f4bd815fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.250+0000 7f4bdebc1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd806c4f0 con 0x7f4bc8005120 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.250+0000 7f4bdebc1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd80d2530 con 0x7f4bd815f190 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.250+0000 7f4bdebc1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd806c670 con 0x7f4bd80c96c0 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bdc95d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 0x7f4bd80b7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bdc95d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 0x7f4bd80b7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48530/0 (socket says 172.21.15.38:48530) 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bdc95d640 1 -- 172.21.15.38:0/889563237 learned_addr learned my addr 172.21.15.38:0/889563237 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bd7fff640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd815f190 0x7f4bd81f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bdd15e640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bd80c96c0 0x7f4bd815fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bdc95d640 1 -- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bd80c96c0 msgr2=0x7f4bd815fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bdc95d640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bd80c96c0 0x7f4bd815fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bdc95d640 1 -- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd815f190 msgr2=0x7f4bd81f4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bdc95d640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd815f190 0x7f4bd81f4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bdc95d640 1 -- 172.21.15.38:0/889563237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd81f4a40 con 0x7f4bc8005120 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bd7fff640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd815f190 0x7f4bd81f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bdc95d640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 0x7f4bd80b7510 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0118e0 tx=0x7f4bcc011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bd57fa640 1 -- 172.21.15.38:0/889563237 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc00fa90 con 0x7f4bc8005120 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bd57fa640 1 -- 172.21.15.38:0/889563237 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bcc00fc30 con 0x7f4bc8005120 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bd57fa640 1 -- 172.21.15.38:0/889563237 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc014440 con 0x7f4bc8005120 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bdebc1640 1 -- 172.21.15.38:0/889563237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd815e3d0 con 0x7f4bc8005120 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.251+0000 7f4bdebc1640 1 -- 172.21.15.38:0/889563237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd815e990 con 0x7f4bc8005120 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.252+0000 7f4bdebc1640 1 -- 172.21.15.38:0/889563237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd8068e80 con 0x7f4bc8005120 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.253+0000 7f4bd57fa640 1 -- 172.21.15.38:0/889563237 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bcc012040 con 0x7f4bc8005120 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.253+0000 7f4bd57fa640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bbc06b4d0 0x7f4bbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.253+0000 7f4bd57fa640 1 -- 172.21.15.38:0/889563237 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f4bcc08acc0 con 0x7f4bc8005120 2024-04-25T08:32:19.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.253+0000 7f4bd7fff640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bbc06b4d0 0x7f4bbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.254+0000 7f4bd7fff640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bbc06b4d0 0x7f4bbc06d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4bd815f820 tx=0x7f4bc400a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.257+0000 7f4bd57fa640 1 -- 172.21.15.38:0/889563237 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4bd81f4a40 con 0x7f4bc8005120 2024-04-25T08:32:19.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.438+0000 7f4bdebc1640 1 -- 172.21.15.38:0/889563237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bd815d220 con 0x7f4bc8005120 2024-04-25T08:32:19.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.439+0000 7f4bd57fa640 1 -- 172.21.15.38:0/889563237 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18969 (secure 0 0 0) 0x7f4bcc08a070 con 0x7f4bc8005120 2024-04-25T08:32:19.435 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:19.435 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:16.572337+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.a","mappings":[{"from":4,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.441+0000 7f4bdebc1640 1 -- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bbc06b4d0 msgr2=0x7f4bbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.441+0000 7f4bdebc1640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bbc06b4d0 0x7f4bbc06d990 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4bd815f820 tx=0x7f4bc400a040 comp rx=0 tx=0).stop 2024-04-25T08:32:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.441+0000 7f4bdebc1640 1 -- 172.21.15.38:0/889563237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 msgr2=0x7f4bd80b7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.441+0000 7f4bdebc1640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 0x7f4bd80b7510 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0118e0 tx=0x7f4bcc011da0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.441+0000 7f4bdebc1640 1 -- 172.21.15.38:0/889563237 shutdown_connections 2024-04-25T08:32:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.441+0000 7f4bdebc1640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bbc06b4d0 0x7f4bbc06d990 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.441+0000 7f4bdebc1640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bd80c96c0 0x7f4bd815fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.441+0000 7f4bdebc1640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bd815f190 0x7f4bd81f4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.441+0000 7f4bdebc1640 1 --2- 172.21.15.38:0/889563237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bc8005120 0x7f4bd80b7510 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.441+0000 7f4bdebc1640 1 -- 172.21.15.38:0/889563237 >> 172.21.15.38:0/889563237 conn(0x7f4bd80d3fb0 msgr2=0x7f4bd8161840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.441+0000 7f4bdebc1640 1 -- 172.21.15.38:0/889563237 shutdown_connections 2024-04-25T08:32:19.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.441+0000 7f4bdebc1640 1 -- 172.21.15.38:0/889563237 wait complete. 2024-04-25T08:32:19.448 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:19.449 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:19.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.556+0000 7feb2844a640 1 Processor -- start 2024-04-25T08:32:19.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.556+0000 7feb2844a640 1 -- start start 2024-04-25T08:32:19.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.557+0000 7feb2844a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb200cd580 0x7feb200cda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.557+0000 7feb2844a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2015ef40 con 0x7feb200b1d60 2024-04-25T08:32:19.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.557+0000 7feb2844a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2006ae30 con 0x7feb200d00e0 2024-04-25T08:32:19.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.557+0000 7feb2844a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2006abe0 con 0x7feb200cd580 2024-04-25T08:32:19.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.557+0000 7feb261e6640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7feb200b1d60 0x7feb200cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34886/0 (socket says 172.21.15.38:34886) 2024-04-25T08:32:19.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.557+0000 7feb261e6640 1 -- 172.21.15.38:0/2570460312 learned_addr learned my addr 172.21.15.38:0/2570460312 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:19.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.558+0000 7feb269e7640 1 --2- 172.21.15.38:0/2570460312 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb200cd580 0x7feb200cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.558+0000 7feb251e4640 1 -- 172.21.15.38:0/2570460312 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945568849 0 0) 0x7feb2015ef40 con 0x7feb200b1d60 2024-04-25T08:32:19.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.558+0000 7feb251e4640 1 -- 172.21.15.38:0/2570460312 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb08003520 con 0x7feb200b1d60 2024-04-25T08:32:19.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.558+0000 7feb251e4640 1 -- 172.21.15.38:0/2570460312 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583692119 0 0) 0x7feb2006ae30 con 0x7feb200d00e0 2024-04-25T08:32:19.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.558+0000 7feb269e7640 1 -- 172.21.15.38:0/2570460312 >> v1:172.21.15.107:6789/0 conn(0x7feb200d00e0 legacy=0x7feb200cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.558+0000 7feb269e7640 1 -- 172.21.15.38:0/2570460312 >> v1:172.21.15.38:6789/0 conn(0x7feb200b1d60 legacy=0x7feb200cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.558+0000 7feb269e7640 1 -- 172.21.15.38:0/2570460312 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2006b110 con 0x7feb200cd580 2024-04-25T08:32:19.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.559+0000 7feb269e7640 1 --2- 172.21.15.38:0/2570460312 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb200cd580 0x7feb200cda10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feb10009dc0 tx=0x7feb1002a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=102946214ad1bc6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.559+0000 7feb251e4640 1 -- 172.21.15.38:0/2570460312 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10035020 con 0x7feb200cd580 2024-04-25T08:32:19.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.559+0000 7feb251e4640 1 -- 172.21.15.38:0/2570460312 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb10007540 con 0x7feb200cd580 2024-04-25T08:32:19.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.559+0000 7feb251e4640 1 -- 172.21.15.38:0/2570460312 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18001670 con 0x7feb200cd580 2024-04-25T08:32:19.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.559+0000 7feb2844a640 1 -- 172.21.15.38:0/2570460312 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb200cd580 msgr2=0x7feb200cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.560+0000 7feb2844a640 1 --2- 172.21.15.38:0/2570460312 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb200cd580 0x7feb200cda10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feb10009dc0 tx=0x7feb1002a5b0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.560+0000 7feb2844a640 1 -- 172.21.15.38:0/2570460312 shutdown_connections 2024-04-25T08:32:19.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.560+0000 7feb2844a640 1 --2- 172.21.15.38:0/2570460312 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb200cd580 0x7feb200cda10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.560+0000 7feb2844a640 1 -- 172.21.15.38:0/2570460312 >> 172.21.15.38:0/2570460312 conn(0x7feb200d06b0 msgr2=0x7feb200d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.560+0000 7feb2844a640 1 -- 172.21.15.38:0/2570460312 shutdown_connections 2024-04-25T08:32:19.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.560+0000 7feb2844a640 1 -- 172.21.15.38:0/2570460312 wait complete. 2024-04-25T08:32:19.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.561+0000 7feb2844a640 1 Processor -- start 2024-04-25T08:32:19.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.561+0000 7feb2844a640 1 -- start start 2024-04-25T08:32:19.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb2844a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb200b1d60 0x7feb201489a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb2844a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb200cd580 0x7feb200c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb2844a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb200d00e0 0x7feb20160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb2844a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2015e340 con 0x7feb200d00e0 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb2844a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2006b480 con 0x7feb200cd580 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb2844a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2015e4c0 con 0x7feb200b1d60 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb269e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb200d00e0 0x7feb20160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb269e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb200d00e0 0x7feb20160460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48532/0 (socket says 172.21.15.38:48532) 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb269e7640 1 -- 172.21.15.38:0/4185592485 learned_addr learned my addr 172.21.15.38:0/4185592485 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb261e6640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb200b1d60 0x7feb201489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb259e5640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb200cd580 0x7feb200c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb269e7640 1 -- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb200b1d60 msgr2=0x7feb201489a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb269e7640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb200b1d60 0x7feb201489a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb269e7640 1 -- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb200cd580 msgr2=0x7feb200c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb269e7640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb200cd580 0x7feb200c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb269e7640 1 -- 172.21.15.38:0/4185592485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb20148ee0 con 0x7feb200d00e0 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb269e7640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb200d00e0 0x7feb20160460 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb10009ee0 tx=0x7feb10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb1effd640 1 -- 172.21.15.38:0/4185592485 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10009450 con 0x7feb200d00e0 2024-04-25T08:32:19.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb1effd640 1 -- 172.21.15.38:0/4185592485 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb10031c40 con 0x7feb200d00e0 2024-04-25T08:32:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb2844a640 1 -- 172.21.15.38:0/4185592485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb200c81d0 con 0x7feb200d00e0 2024-04-25T08:32:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.562+0000 7feb1effd640 1 -- 172.21.15.38:0/4185592485 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10003990 con 0x7feb200d00e0 2024-04-25T08:32:19.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.563+0000 7feb2844a640 1 -- 172.21.15.38:0/4185592485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb200c8790 con 0x7feb200d00e0 2024-04-25T08:32:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.564+0000 7feb2844a640 1 -- 172.21.15.38:0/4185592485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb20068e80 con 0x7feb200d00e0 2024-04-25T08:32:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.564+0000 7feb1effd640 1 -- 172.21.15.38:0/4185592485 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb1002ad10 con 0x7feb200d00e0 2024-04-25T08:32:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.565+0000 7feb1effd640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feb0c06b4d0 0x7feb0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.565+0000 7feb1effd640 1 -- 172.21.15.38:0/4185592485 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7feb100ab4f0 con 0x7feb200d00e0 2024-04-25T08:32:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.565+0000 7feb261e6640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feb0c06b4d0 0x7feb0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.565+0000 7feb261e6640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feb0c06b4d0 0x7feb0c06d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feb18001600 tx=0x7feb18001570 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.569+0000 7feb1effd640 1 -- 172.21.15.38:0/4185592485 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb10041020 con 0x7feb200d00e0 2024-04-25T08:32:19.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.748+0000 7feb2844a640 1 -- 172.21.15.38:0/4185592485 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb2006cfb0 con 0x7feb200d00e0 2024-04-25T08:32:19.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.748+0000 7feb1effd640 1 -- 172.21.15.38:0/4185592485 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb100799e0 con 0x7feb200d00e0 2024-04-25T08:32:19.745 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:19.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.750+0000 7feb2844a640 1 -- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feb0c06b4d0 msgr2=0x7feb0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.750+0000 7feb2844a640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feb0c06b4d0 0x7feb0c06d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7feb18001600 tx=0x7feb18001570 comp rx=0 tx=0).stop 2024-04-25T08:32:19.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.750+0000 7feb2844a640 1 -- 172.21.15.38:0/4185592485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb200d00e0 msgr2=0x7feb20160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.750+0000 7feb2844a640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb200d00e0 0x7feb20160460 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feb10009ee0 tx=0x7feb10004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:19.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.750+0000 7feb2844a640 1 -- 172.21.15.38:0/4185592485 shutdown_connections 2024-04-25T08:32:19.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.750+0000 7feb2844a640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb200d00e0 0x7feb20160460 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.750+0000 7feb2844a640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb200cd580 0x7feb200c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.750+0000 7feb2844a640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feb0c06b4d0 0x7feb0c06d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.750+0000 7feb2844a640 1 --2- 172.21.15.38:0/4185592485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb200b1d60 0x7feb201489a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.750+0000 7feb2844a640 1 -- 172.21.15.38:0/4185592485 >> 172.21.15.38:0/4185592485 conn(0x7feb200d06b0 msgr2=0x7feb2015fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.750+0000 7feb2844a640 1 -- 172.21.15.38:0/4185592485 shutdown_connections 2024-04-25T08:32:19.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.750+0000 7feb2844a640 1 -- 172.21.15.38:0/4185592485 wait complete. 2024-04-25T08:32:19.758 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.865+0000 7f0ec0329640 1 Processor -- start 2024-04-25T08:32:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.865+0000 7f0ec0329640 1 -- start start 2024-04-25T08:32:19.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ec0329640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0eb809a180 0x7f0eb809a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ec0329640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0eb8162390 0x7f0eb80cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ec0329640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb8162820 con 0x7f0eb815df30 2024-04-25T08:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ec0329640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb815e310 con 0x7f0eb8162390 2024-04-25T08:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ec0329640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb809aeb0 con 0x7f0eb809a180 2024-04-25T08:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ebe8c6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0eb8162390 0x7f0eb80cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ebe0c5640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0eb815df30 0x7f0eb8099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34898/0 (socket says 172.21.15.38:34898) 2024-04-25T08:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ebe0c5640 1 -- 172.21.15.38:0/2287384090 learned_addr learned my addr 172.21.15.38:0/2287384090 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ebd8c4640 1 --2- 172.21.15.38:0/2287384090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0eb809a180 0x7f0eb809a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ebe8c6640 1 -- 172.21.15.38:0/2287384090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0eb809a180 msgr2=0x7f0eb809a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ebe8c6640 1 --2- 172.21.15.38:0/2287384090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0eb809a180 0x7f0eb809a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ebe8c6640 1 -- 172.21.15.38:0/2287384090 >> v1:172.21.15.38:6789/0 conn(0x7f0eb815df30 legacy=0x7f0eb8099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ebe8c6640 1 -- 172.21.15.38:0/2287384090 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb815f300 con 0x7f0eb8162390 2024-04-25T08:32:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.866+0000 7f0ebd0c3640 1 -- 172.21.15.38:0/2287384090 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617503354 0 0) 0x7f0eb4001110 con 0x7f0eb815df30 2024-04-25T08:32:19.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.867+0000 7f0ebe8c6640 1 --2- 172.21.15.38:0/2287384090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb8162390 0x7f0eb80cf8f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009d90 tx=0x7f0eb002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e3d6103e68157e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.867+0000 7f0ebd0c3640 1 -- 172.21.15.38:0/2287384090 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0007740 con 0x7f0eb8162390 2024-04-25T08:32:19.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.867+0000 7f0ebd0c3640 1 -- 172.21.15.38:0/2287384090 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb00078e0 con 0x7f0eb8162390 2024-04-25T08:32:19.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.867+0000 7f0ebd0c3640 1 -- 172.21.15.38:0/2287384090 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0007cb0 con 0x7f0eb8162390 2024-04-25T08:32:19.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.868+0000 7f0ec0329640 1 -- 172.21.15.38:0/2287384090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb8162390 msgr2=0x7f0eb80cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.868+0000 7f0ec0329640 1 --2- 172.21.15.38:0/2287384090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb8162390 0x7f0eb80cf8f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0009d90 tx=0x7f0eb002a660 comp rx=0 tx=0).stop 2024-04-25T08:32:19.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.868+0000 7f0ec0329640 1 -- 172.21.15.38:0/2287384090 shutdown_connections 2024-04-25T08:32:19.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.868+0000 7f0ec0329640 1 --2- 172.21.15.38:0/2287384090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb8162390 0x7f0eb80cf8f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.868+0000 7f0ec0329640 1 --2- 172.21.15.38:0/2287384090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0eb809a180 0x7f0eb809a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.868+0000 7f0ec0329640 1 -- 172.21.15.38:0/2287384090 >> 172.21.15.38:0/2287384090 conn(0x7f0eb80b3110 msgr2=0x7f0eb80b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:19.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.868+0000 7f0ec0329640 1 -- 172.21.15.38:0/2287384090 shutdown_connections 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.868+0000 7f0ec0329640 1 -- 172.21.15.38:0/2287384090 wait complete. 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.869+0000 7f0ec0329640 1 Processor -- start 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.869+0000 7f0ec0329640 1 -- start start 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ec0329640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb809a180 0x7f0eb8150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ec0329640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb815df30 0x7f0eb80cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ec0329640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0eb8162390 0x7f0eb815c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ec0329640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb815e820 con 0x7f0eb815df30 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ec0329640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb815ee70 con 0x7f0eb809a180 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ec0329640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb815eb00 con 0x7f0eb8162390 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ebd8c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb815df30 0x7f0eb80cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ebd8c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb815df30 0x7f0eb80cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48552/0 (socket says 172.21.15.38:48552) 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ebd8c4640 1 -- 172.21.15.38:0/2140148569 learned_addr learned my addr 172.21.15.38:0/2140148569 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ebe8c6640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0eb8162390 0x7f0eb815c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ebe0c5640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb809a180 0x7f0eb8150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ebd8c4640 1 -- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0eb8162390 msgr2=0x7f0eb815c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ebd8c4640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0eb8162390 0x7f0eb815c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ebd8c4640 1 -- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb809a180 msgr2=0x7f0eb8150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ebd8c4640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb809a180 0x7f0eb8150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ebd8c4640 1 -- 172.21.15.38:0/2140148569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb80cc210 con 0x7f0eb815df30 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0ebd8c4640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb815df30 0x7f0eb80cd870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800d5f0 tx=0x7f0ea800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.870+0000 7f0eaeffd640 1 -- 172.21.15.38:0/2140148569 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80125f0 con 0x7f0eb815df30 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.871+0000 7f0eaeffd640 1 -- 172.21.15.38:0/2140148569 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea8019070 con 0x7f0eb815df30 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.871+0000 7f0ec0329640 1 -- 172.21.15.38:0/2140148569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb80cc4f0 con 0x7f0eb815df30 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.871+0000 7f0eaeffd640 1 -- 172.21.15.38:0/2140148569 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80129d0 con 0x7f0eb815df30 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.871+0000 7f0ec0329640 1 -- 172.21.15.38:0/2140148569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb80ccab0 con 0x7f0eb815df30 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.871+0000 7f0ec0329640 1 -- 172.21.15.38:0/2140148569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb8068e80 con 0x7f0eb815df30 2024-04-25T08:32:19.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.872+0000 7f0eaeffd640 1 -- 172.21.15.38:0/2140148569 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ea8012b70 con 0x7f0eb815df30 2024-04-25T08:32:19.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.872+0000 7f0eaeffd640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0e9c06b4d0 0x7f0e9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:19.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.872+0000 7f0eaeffd640 1 -- 172.21.15.38:0/2140148569 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f0ea808a980 con 0x7f0eb815df30 2024-04-25T08:32:19.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.873+0000 7f0ebe0c5640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0e9c06b4d0 0x7f0e9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:19.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.873+0000 7f0ebe0c5640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0e9c06b4d0 0x7f0e9c06d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4000f60 tx=0x7f0eb4007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:19.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:19.876+0000 7f0eaeffd640 1 -- 172.21.15.38:0/2140148569 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ea8058d50 con 0x7f0eb815df30 2024-04-25T08:32:20.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.055+0000 7f0ec0329640 1 -- 172.21.15.38:0/2140148569 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0eb80b3de0 con 0x7f0eb815df30 2024-04-25T08:32:20.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.055+0000 7f0eaeffd640 1 -- 172.21.15.38:0/2140148569 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ea805cdf0 con 0x7f0eb815df30 2024-04-25T08:32:20.052 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.057+0000 7f0ec0329640 1 -- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0e9c06b4d0 msgr2=0x7f0e9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.057+0000 7f0ec0329640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0e9c06b4d0 0x7f0e9c06d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4000f60 tx=0x7f0eb4007b50 comp rx=0 tx=0).stop 2024-04-25T08:32:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.057+0000 7f0ec0329640 1 -- 172.21.15.38:0/2140148569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb815df30 msgr2=0x7f0eb80cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.057+0000 7f0ec0329640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb815df30 0x7f0eb80cd870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800d5f0 tx=0x7f0ea800dab0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.057+0000 7f0ec0329640 1 -- 172.21.15.38:0/2140148569 shutdown_connections 2024-04-25T08:32:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.057+0000 7f0ec0329640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0e9c06b4d0 0x7f0e9c06d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.057+0000 7f0ec0329640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0eb8162390 0x7f0eb815c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.057+0000 7f0ec0329640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0eb815df30 0x7f0eb80cd870 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.057+0000 7f0ec0329640 1 --2- 172.21.15.38:0/2140148569 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0eb809a180 0x7f0eb8150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.057+0000 7f0ec0329640 1 -- 172.21.15.38:0/2140148569 >> 172.21.15.38:0/2140148569 conn(0x7f0eb80b3110 msgr2=0x7f0eb8159a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.057+0000 7f0ec0329640 1 -- 172.21.15.38:0/2140148569 shutdown_connections 2024-04-25T08:32:20.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.057+0000 7f0ec0329640 1 -- 172.21.15.38:0/2140148569 wait complete. 2024-04-25T08:32:20.064 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:20.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.172+0000 7ff57aadb640 1 Processor -- start 2024-04-25T08:32:20.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.172+0000 7ff57aadb640 1 -- start start 2024-04-25T08:32:20.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff57aadb640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff5740d33d0 0x7ff5740d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff57aadb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5740d1e90 0x7ff5740b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff57aadb640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5740d2330 con 0x7ff5740d0320 2024-04-25T08:32:20.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff57aadb640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5740d2610 con 0x7ff5740d33d0 2024-04-25T08:32:20.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff57aadb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740d28f0 con 0x7ff5740d1e90 2024-04-25T08:32:20.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff573fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff5740d33d0 0x7ff5740d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff578877640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff5740d0320 0x7ff5740d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34904/0 (socket says 172.21.15.38:34904) 2024-04-25T08:32:20.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff578877640 1 -- 172.21.15.38:0/2669306659 learned_addr learned my addr 172.21.15.38:0/2669306659 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:20.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff579078640 1 --2- 172.21.15.38:0/2669306659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5740d1e90 0x7ff5740b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff573fff640 1 -- 172.21.15.38:0/2669306659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5740d1e90 msgr2=0x7ff5740b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:20.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff573fff640 1 --2- 172.21.15.38:0/2669306659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5740d1e90 0x7ff5740b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff573fff640 1 -- 172.21.15.38:0/2669306659 >> v1:172.21.15.38:6789/0 conn(0x7ff5740d0320 legacy=0x7ff5740d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff5737fe640 1 -- 172.21.15.38:0/2669306659 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922903890 0 0) 0x7ff5740d2330 con 0x7ff5740d0320 2024-04-25T08:32:20.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.173+0000 7ff573fff640 1 -- 172.21.15.38:0/2669306659 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740d2bd0 con 0x7ff5740d33d0 2024-04-25T08:32:20.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.174+0000 7ff573fff640 1 --2- 172.21.15.38:0/2669306659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5740d33d0 0x7ff5740d1950 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0097a0 tx=0x7ff55c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52426e8bc5ff5e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:20.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.174+0000 7ff5737fe640 1 -- 172.21.15.38:0/2669306659 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c037070 con 0x7ff5740d33d0 2024-04-25T08:32:20.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.174+0000 7ff5737fe640 1 -- 172.21.15.38:0/2669306659 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55c032440 con 0x7ff5740d33d0 2024-04-25T08:32:20.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.174+0000 7ff5737fe640 1 -- 172.21.15.38:0/2669306659 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c032810 con 0x7ff5740d33d0 2024-04-25T08:32:20.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.175+0000 7ff57aadb640 1 -- 172.21.15.38:0/2669306659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5740d33d0 msgr2=0x7ff5740d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.175+0000 7ff57aadb640 1 --2- 172.21.15.38:0/2669306659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5740d33d0 0x7ff5740d1950 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0097a0 tx=0x7ff55c02a210 comp rx=0 tx=0).stop 2024-04-25T08:32:20.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.175+0000 7ff57aadb640 1 -- 172.21.15.38:0/2669306659 shutdown_connections 2024-04-25T08:32:20.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.175+0000 7ff57aadb640 1 --2- 172.21.15.38:0/2669306659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5740d1e90 0x7ff5740b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.175+0000 7ff57aadb640 1 --2- 172.21.15.38:0/2669306659 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5740d33d0 0x7ff5740d1950 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.175+0000 7ff57aadb640 1 -- 172.21.15.38:0/2669306659 >> 172.21.15.38:0/2669306659 conn(0x7ff5740668f0 msgr2=0x7ff574066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:20.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.175+0000 7ff57aadb640 1 -- 172.21.15.38:0/2669306659 shutdown_connections 2024-04-25T08:32:20.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.175+0000 7ff57aadb640 1 -- 172.21.15.38:0/2669306659 wait complete. 2024-04-25T08:32:20.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.176+0000 7ff57aadb640 1 Processor -- start 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.176+0000 7ff57aadb640 1 -- start start 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff57aadb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5740d0320 0x7ff57415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff57aadb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5740d1e90 0x7ff5740b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff57aadb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740d33d0 0x7ff57415fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff57aadb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574161390 con 0x7ff5740d0320 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff57aadb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff574161210 con 0x7ff5740d33d0 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff57aadb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5740b9720 con 0x7ff5740d1e90 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff578877640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5740d0320 0x7ff57415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff578877640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5740d0320 0x7ff57415a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48574/0 (socket says 172.21.15.38:48574) 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff578877640 1 -- 172.21.15.38:0/697360538 learned_addr learned my addr 172.21.15.38:0/697360538 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff579078640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740d33d0 0x7ff57415fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff573fff640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5740d1e90 0x7ff5740b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff578877640 1 -- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5740d1e90 msgr2=0x7ff5740b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff578877640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5740d1e90 0x7ff5740b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff578877640 1 -- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740d33d0 msgr2=0x7ff57415fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff578877640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740d33d0 0x7ff57415fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.177+0000 7ff578877640 1 -- 172.21.15.38:0/697360538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5740b4a60 con 0x7ff5740d0320 2024-04-25T08:32:20.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.178+0000 7ff578877640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5740d0320 0x7ff57415a6f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00c7f0 tx=0x7ff56c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:20.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.178+0000 7ff5717fa640 1 -- 172.21.15.38:0/697360538 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c018070 con 0x7ff5740d0320 2024-04-25T08:32:20.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.178+0000 7ff5717fa640 1 -- 172.21.15.38:0/697360538 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56c00a640 con 0x7ff5740d0320 2024-04-25T08:32:20.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.178+0000 7ff57aadb640 1 -- 172.21.15.38:0/697360538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5740b4d40 con 0x7ff5740d0320 2024-04-25T08:32:20.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.178+0000 7ff5717fa640 1 -- 172.21.15.38:0/697360538 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56c00aa30 con 0x7ff5740d0320 2024-04-25T08:32:20.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.178+0000 7ff57aadb640 1 -- 172.21.15.38:0/697360538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5740b5300 con 0x7ff5740d0320 2024-04-25T08:32:20.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.179+0000 7ff57aadb640 1 -- 172.21.15.38:0/697360538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff574068e80 con 0x7ff5740d0320 2024-04-25T08:32:20.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.179+0000 7ff5717fa640 1 -- 172.21.15.38:0/697360538 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff56c01b070 con 0x7ff5740d0320 2024-04-25T08:32:20.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.180+0000 7ff5717fa640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff55406b4d0 0x7ff55406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.180+0000 7ff5717fa640 1 -- 172.21.15.38:0/697360538 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7ff56c089c80 con 0x7ff5740d0320 2024-04-25T08:32:20.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.180+0000 7ff573fff640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff55406b4d0 0x7ff55406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.180+0000 7ff573fff640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff55406b4d0 0x7ff55406d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff55c034000 tx=0x7ff55c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:20.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.184+0000 7ff5717fa640 1 -- 172.21.15.38:0/697360538 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff56c0586a0 con 0x7ff5740d0320 2024-04-25T08:32:20.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.368+0000 7ff57aadb640 1 -- 172.21.15.38:0/697360538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff57415d5e0 con 0x7ff5740d0320 2024-04-25T08:32:20.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.369+0000 7ff5717fa640 1 -- 172.21.15.38:0/697360538 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff56c057fa0 con 0x7ff5740d0320 2024-04-25T08:32:20.365 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:20.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.371+0000 7ff57aadb640 1 -- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff55406b4d0 msgr2=0x7ff55406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.371+0000 7ff57aadb640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff55406b4d0 0x7ff55406d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff55c034000 tx=0x7ff55c02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:20.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.371+0000 7ff57aadb640 1 -- 172.21.15.38:0/697360538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5740d0320 msgr2=0x7ff57415a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.371+0000 7ff57aadb640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5740d0320 0x7ff57415a6f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff56c00c7f0 tx=0x7ff56c00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.371+0000 7ff57aadb640 1 -- 172.21.15.38:0/697360538 shutdown_connections 2024-04-25T08:32:20.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.371+0000 7ff57aadb640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5740d33d0 0x7ff57415fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.371+0000 7ff57aadb640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff55406b4d0 0x7ff55406d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.371+0000 7ff57aadb640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5740d1e90 0x7ff5740b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.371+0000 7ff57aadb640 1 --2- 172.21.15.38:0/697360538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5740d0320 0x7ff57415a6f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.371+0000 7ff57aadb640 1 -- 172.21.15.38:0/697360538 >> 172.21.15.38:0/697360538 conn(0x7ff5740668f0 msgr2=0x7ff5740b8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:20.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.371+0000 7ff57aadb640 1 -- 172.21.15.38:0/697360538 shutdown_connections 2024-04-25T08:32:20.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.371+0000 7ff57aadb640 1 -- 172.21.15.38:0/697360538 wait complete. 2024-04-25T08:32:20.378 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:20.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.485+0000 7f6ff9189640 1 Processor -- start 2024-04-25T08:32:20.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.485+0000 7f6ff9189640 1 -- start start 2024-04-25T08:32:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.486+0000 7f6ff9189640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6ff40d0320 0x7f6ff40d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.486+0000 7f6ff9189640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff40d2060 con 0x7f6ff40b6980 2024-04-25T08:32:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.486+0000 7f6ff9189640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff40d2330 con 0x7f6ff40d0700 2024-04-25T08:32:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.486+0000 7f6ff9189640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff40d2610 con 0x7f6ff40d0320 2024-04-25T08:32:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.486+0000 7f6ff359e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6ff40b6980 0x7f6ff40b6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34916/0 (socket says 172.21.15.38:34916) 2024-04-25T08:32:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.486+0000 7f6ff359e640 1 -- 172.21.15.38:0/2286155316 learned_addr learned my addr 172.21.15.38:0/2286155316 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.486+0000 7f6ff2d9d640 1 --2- 172.21.15.38:0/2286155316 >> v2:172.21.15.107:3300/0 conn(0x7f6ff40d0320 0x7f6ff40d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.486+0000 7f6ff1d9b640 1 -- 172.21.15.38:0/2286155316 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222794305 0 0) 0x7f6ff40d2060 con 0x7f6ff40b6980 2024-04-25T08:32:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.486+0000 7f6ff1d9b640 1 -- 172.21.15.38:0/2286155316 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fdc003520 con 0x7f6ff40b6980 2024-04-25T08:32:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.487+0000 7f6ff2d9d640 1 -- 172.21.15.38:0/2286155316 >> v1:172.21.15.107:6789/0 conn(0x7f6ff40d0700 legacy=0x7f6ff40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.487+0000 7f6ff2d9d640 1 -- 172.21.15.38:0/2286155316 >> v1:172.21.15.38:6789/0 conn(0x7f6ff40b6980 legacy=0x7f6ff40b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.487+0000 7f6ff2d9d640 1 -- 172.21.15.38:0/2286155316 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff40d28f0 con 0x7f6ff40d0320 2024-04-25T08:32:20.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.487+0000 7f6ff2d9d640 1 --2- 172.21.15.38:0/2286155316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff40d0320 0x7f6ff40d33d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009d90 tx=0x7f6fe402a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ab988816a7c7978 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:20.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.487+0000 7f6ff1d9b640 1 -- 172.21.15.38:0/2286155316 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4037070 con 0x7f6ff40d0320 2024-04-25T08:32:20.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.487+0000 7f6ff1d9b640 1 -- 172.21.15.38:0/2286155316 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe4007740 con 0x7f6ff40d0320 2024-04-25T08:32:20.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.488+0000 7f6ff1d9b640 1 -- 172.21.15.38:0/2286155316 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4007af0 con 0x7f6ff40d0320 2024-04-25T08:32:20.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.488+0000 7f6ff9189640 1 -- 172.21.15.38:0/2286155316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff40d0320 msgr2=0x7f6ff40d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.488+0000 7f6ff9189640 1 --2- 172.21.15.38:0/2286155316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff40d0320 0x7f6ff40d33d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009d90 tx=0x7f6fe402a780 comp rx=0 tx=0).stop 2024-04-25T08:32:20.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.488+0000 7f6ff9189640 1 -- 172.21.15.38:0/2286155316 shutdown_connections 2024-04-25T08:32:20.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.488+0000 7f6ff9189640 1 --2- 172.21.15.38:0/2286155316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff40d0320 0x7f6ff40d33d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.488+0000 7f6ff9189640 1 -- 172.21.15.38:0/2286155316 >> 172.21.15.38:0/2286155316 conn(0x7f6ff40668f0 msgr2=0x7f6ff4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:20.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.488+0000 7f6ff9189640 1 -- 172.21.15.38:0/2286155316 shutdown_connections 2024-04-25T08:32:20.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.488+0000 7f6ff9189640 1 -- 172.21.15.38:0/2286155316 wait complete. 2024-04-25T08:32:20.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.489+0000 7f6ff9189640 1 Processor -- start 2024-04-25T08:32:20.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.490+0000 7f6ff9189640 1 -- start start 2024-04-25T08:32:20.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.490+0000 7f6ff9189640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff40b6980 0x7f6ff414abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.490+0000 7f6ff9189640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff40d0320 0x7f6ff40c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.490+0000 7f6ff9189640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ff40d0700 0x7f6ff415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.490+0000 7f6ff9189640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff40d0b40 con 0x7f6ff40b6980 2024-04-25T08:32:20.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.490+0000 7f6ff9189640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff40d2eb0 con 0x7f6ff40d0320 2024-04-25T08:32:20.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.490+0000 7f6ff9189640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff40b5760 con 0x7f6ff40d0700 2024-04-25T08:32:20.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.490+0000 7f6ff2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff40b6980 0x7f6ff414abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.490+0000 7f6ff2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff40b6980 0x7f6ff414abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48586/0 (socket says 172.21.15.38:48586) 2024-04-25T08:32:20.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.490+0000 7f6ff2d9d640 1 -- 172.21.15.38:0/1766110870 learned_addr learned my addr 172.21.15.38:0/1766110870 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:20.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6ff259c640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff40d0320 0x7f6ff40c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6ff359e640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ff40d0700 0x7f6ff415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6ff2d9d640 1 -- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ff40d0700 msgr2=0x7f6ff415ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6ff2d9d640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ff40d0700 0x7f6ff415ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6ff2d9d640 1 -- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff40d0320 msgr2=0x7f6ff40c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6ff2d9d640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff40d0320 0x7f6ff40c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6ff2d9d640 1 -- 172.21.15.38:0/1766110870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff40b5a40 con 0x7f6ff40b6980 2024-04-25T08:32:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6ff359e640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ff40d0700 0x7f6ff415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6ff2d9d640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff40b6980 0x7f6ff414abb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4035040 tx=0x7f6fe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6fdb7fe640 1 -- 172.21.15.38:0/1766110870 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4042070 con 0x7f6ff40b6980 2024-04-25T08:32:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6fdb7fe640 1 -- 172.21.15.38:0/1766110870 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe4003990 con 0x7f6ff40b6980 2024-04-25T08:32:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6ff9189640 1 -- 172.21.15.38:0/1766110870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff40b5d20 con 0x7f6ff40b6980 2024-04-25T08:32:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6fdb7fe640 1 -- 172.21.15.38:0/1766110870 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4037040 con 0x7f6ff40b6980 2024-04-25T08:32:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.491+0000 7f6ff9189640 1 -- 172.21.15.38:0/1766110870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff40c95b0 con 0x7f6ff40b6980 2024-04-25T08:32:20.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.492+0000 7f6ff9189640 1 -- 172.21.15.38:0/1766110870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff41e61c0 con 0x7f6ff40b6980 2024-04-25T08:32:20.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.492+0000 7f6fdb7fe640 1 -- 172.21.15.38:0/1766110870 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fe4032870 con 0x7f6ff40b6980 2024-04-25T08:32:20.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.493+0000 7f6fdb7fe640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6fe006b500 0x7f6fe006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.493+0000 7f6fdb7fe640 1 -- 172.21.15.38:0/1766110870 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f6fe40ab920 con 0x7f6ff40b6980 2024-04-25T08:32:20.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.493+0000 7f6ff259c640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6fe006b500 0x7f6fe006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.494+0000 7f6ff259c640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6fe006b500 0x7f6fe006d9c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8007fd0 tx=0x7f6fe8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:20.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.497+0000 7f6fdb7fe640 1 -- 172.21.15.38:0/1766110870 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fe407a4f0 con 0x7f6ff40b6980 2024-04-25T08:32:20.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.678+0000 7f6ff9189640 1 -- 172.21.15.38:0/1766110870 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6ff4068e80 con 0x7f6ff40b6980 2024-04-25T08:32:20.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.678+0000 7f6fdb7fe640 1 -- 172.21.15.38:0/1766110870 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6fe4079df0 con 0x7f6ff40b6980 2024-04-25T08:32:20.675 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:20.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.680+0000 7f6ff9189640 1 -- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6fe006b500 msgr2=0x7f6fe006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.680+0000 7f6ff9189640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6fe006b500 0x7f6fe006d9c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8007fd0 tx=0x7f6fe8009040 comp rx=0 tx=0).stop 2024-04-25T08:32:20.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.680+0000 7f6ff9189640 1 -- 172.21.15.38:0/1766110870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff40b6980 msgr2=0x7f6ff414abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.680+0000 7f6ff9189640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff40b6980 0x7f6ff414abb0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4035040 tx=0x7f6fe4004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:20.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.680+0000 7f6ff9189640 1 -- 172.21.15.38:0/1766110870 shutdown_connections 2024-04-25T08:32:20.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.680+0000 7f6ff9189640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6fe006b500 0x7f6fe006d9c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.680+0000 7f6ff9189640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ff40d0700 0x7f6ff415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.680+0000 7f6ff9189640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff40d0320 0x7f6ff40c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.680+0000 7f6ff9189640 1 --2- 172.21.15.38:0/1766110870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff40b6980 0x7f6ff414abb0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.680+0000 7f6ff9189640 1 -- 172.21.15.38:0/1766110870 >> 172.21.15.38:0/1766110870 conn(0x7f6ff40668f0 msgr2=0x7f6ff40c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:20.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.680+0000 7f6ff9189640 1 -- 172.21.15.38:0/1766110870 shutdown_connections 2024-04-25T08:32:20.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.680+0000 7f6ff9189640 1 -- 172.21.15.38:0/1766110870 wait complete. 2024-04-25T08:32:20.688 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:32:20.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.793+0000 7fdb4a769640 1 Processor -- start 2024-04-25T08:32:20.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.793+0000 7fdb4a769640 1 -- start start 2024-04-25T08:32:20.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb4a769640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb44160ef0 0x7fdb440d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb4a769640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb440b2ab0 0x7fdb440d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb4a769640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb440b9610 con 0x7fdb440b9180 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb4a769640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb44161cf0 con 0x7fdb440b2ab0 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb4a769640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb441612d0 con 0x7fdb44160ef0 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb48d06640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb440b2ab0 0x7fdb440d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb48d06640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdb440b2ab0 0x7fdb440d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48588/0 (socket says 172.21.15.38:48588) 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb48d06640 1 -- 172.21.15.38:0/337064070 learned_addr learned my addr 172.21.15.38:0/337064070 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb43fff640 1 --2- 172.21.15.38:0/337064070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb44160ef0 0x7fdb440d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb48d06640 1 -- 172.21.15.38:0/337064070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb44160ef0 msgr2=0x7fdb440d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb48d06640 1 --2- 172.21.15.38:0/337064070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb44160ef0 0x7fdb440d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb48d06640 1 -- 172.21.15.38:0/337064070 >> v1:172.21.15.107:6789/0 conn(0x7fdb440b9180 legacy=0x7fdb440d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb48d06640 1 -- 172.21.15.38:0/337064070 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb441616a0 con 0x7fdb440b2ab0 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.794+0000 7fdb43fff640 1 --2- 172.21.15.38:0/337064070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb44160ef0 0x7fdb440d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.795+0000 7fdb48d06640 1 --2- 172.21.15.38:0/337064070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb440b2ab0 0x7fdb440d0cf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009e50 tx=0x7fdb3c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e4d55ee5556505d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.795+0000 7fdb42ffd640 1 -- 172.21.15.38:0/337064070 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c036070 con 0x7fdb440b2ab0 2024-04-25T08:32:20.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.795+0000 7fdb42ffd640 1 -- 172.21.15.38:0/337064070 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c031600 con 0x7fdb440b2ab0 2024-04-25T08:32:20.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.795+0000 7fdb42ffd640 1 -- 172.21.15.38:0/337064070 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c0319d0 con 0x7fdb440b2ab0 2024-04-25T08:32:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.796+0000 7fdb4a769640 1 -- 172.21.15.38:0/337064070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb440b2ab0 msgr2=0x7fdb440d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.796+0000 7fdb4a769640 1 --2- 172.21.15.38:0/337064070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb440b2ab0 0x7fdb440d0cf0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009e50 tx=0x7fdb3c02a630 comp rx=0 tx=0).stop 2024-04-25T08:32:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.796+0000 7fdb4a769640 1 -- 172.21.15.38:0/337064070 shutdown_connections 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.796+0000 7fdb4a769640 1 --2- 172.21.15.38:0/337064070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb440b2ab0 0x7fdb440d0cf0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.796+0000 7fdb4a769640 1 --2- 172.21.15.38:0/337064070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb44160ef0 0x7fdb440d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.796+0000 7fdb4a769640 1 -- 172.21.15.38:0/337064070 >> 172.21.15.38:0/337064070 conn(0x7fdb44066d10 msgr2=0x7fdb44067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.796+0000 7fdb4a769640 1 -- 172.21.15.38:0/337064070 shutdown_connections 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.796+0000 7fdb4a769640 1 -- 172.21.15.38:0/337064070 wait complete. 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.797+0000 7fdb4a769640 1 Processor -- start 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.797+0000 7fdb4a769640 1 -- start start 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb4a769640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb440b2ab0 0x7fdb441e7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb4a769640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb44160ef0 0x7fdb440cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb4a769640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb44155040 0x7fdb4415d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb4a769640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb44162300 con 0x7fdb44155040 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb4a769640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb44162180 con 0x7fdb440b2ab0 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb4a769640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb44162480 con 0x7fdb44160ef0 2024-04-25T08:32:20.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb48d06640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb44155040 0x7fdb4415d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb48d06640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb44155040 0x7fdb4415d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48590/0 (socket says 172.21.15.38:48590) 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb48d06640 1 -- 172.21.15.38:0/1362030172 learned_addr learned my addr 172.21.15.38:0/1362030172 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb43fff640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb440b2ab0 0x7fdb441e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb437fe640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb44160ef0 0x7fdb440cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb48d06640 1 -- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb44160ef0 msgr2=0x7fdb440cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb48d06640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb44160ef0 0x7fdb440cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb48d06640 1 -- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb440b2ab0 msgr2=0x7fdb441e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb48d06640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb440b2ab0 0x7fdb441e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.798+0000 7fdb48d06640 1 -- 172.21.15.38:0/1362030172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb441e8200 con 0x7fdb44155040 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.799+0000 7fdb48d06640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb44155040 0x7fdb4415d710 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009f70 tx=0x7fdb3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.799+0000 7fdb40ff9640 1 -- 172.21.15.38:0/1362030172 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c036050 con 0x7fdb44155040 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.799+0000 7fdb40ff9640 1 -- 172.21.15.38:0/1362030172 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c042070 con 0x7fdb44155040 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.799+0000 7fdb4a769640 1 -- 172.21.15.38:0/1362030172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb441e7c40 con 0x7fdb44155040 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.799+0000 7fdb40ff9640 1 -- 172.21.15.38:0/1362030172 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c009510 con 0x7fdb44155040 2024-04-25T08:32:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.799+0000 7fdb4a769640 1 -- 172.21.15.38:0/1362030172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb440c7c50 con 0x7fdb44155040 2024-04-25T08:32:20.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.800+0000 7fdb4a769640 1 -- 172.21.15.38:0/1362030172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb4404ef90 con 0x7fdb44155040 2024-04-25T08:32:20.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.800+0000 7fdb40ff9640 1 -- 172.21.15.38:0/1362030172 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb3c007bd0 con 0x7fdb44155040 2024-04-25T08:32:20.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.801+0000 7fdb40ff9640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdb2406b4d0 0x7fdb2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:20.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.801+0000 7fdb40ff9640 1 -- 172.21.15.38:0/1362030172 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fdb3c0ab3f0 con 0x7fdb44155040 2024-04-25T08:32:20.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.801+0000 7fdb43fff640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdb2406b4d0 0x7fdb2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:20.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.801+0000 7fdb43fff640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdb2406b4d0 0x7fdb2406d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00aac0 tx=0x7fdb2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:20.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.805+0000 7fdb40ff9640 1 -- 172.21.15.38:0/1362030172 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb3c079970 con 0x7fdb44155040 2024-04-25T08:32:20.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.996+0000 7fdb4a769640 1 -- 172.21.15.38:0/1362030172 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdb440cdc90 con 0x7fdb44155040 2024-04-25T08:32:20.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.997+0000 7fdb40ff9640 1 -- 172.21.15.38:0/1362030172 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdb3c07da10 con 0x7fdb44155040 2024-04-25T08:32:20.993 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.999+0000 7fdb4a769640 1 -- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdb2406b4d0 msgr2=0x7fdb2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.999+0000 7fdb4a769640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdb2406b4d0 0x7fdb2406d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00aac0 tx=0x7fdb2c008040 comp rx=0 tx=0).stop 2024-04-25T08:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.999+0000 7fdb4a769640 1 -- 172.21.15.38:0/1362030172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb44155040 msgr2=0x7fdb4415d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.999+0000 7fdb4a769640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb44155040 0x7fdb4415d710 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009f70 tx=0x7fdb3c004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.999+0000 7fdb4a769640 1 -- 172.21.15.38:0/1362030172 shutdown_connections 2024-04-25T08:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.999+0000 7fdb4a769640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb44155040 0x7fdb4415d710 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.999+0000 7fdb4a769640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdb2406b4d0 0x7fdb2406d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.999+0000 7fdb4a769640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdb44160ef0 0x7fdb440cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.999+0000 7fdb4a769640 1 --2- 172.21.15.38:0/1362030172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb440b2ab0 0x7fdb441e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.999+0000 7fdb4a769640 1 -- 172.21.15.38:0/1362030172 >> 172.21.15.38:0/1362030172 conn(0x7fdb44066d10 msgr2=0x7fdb440d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.999+0000 7fdb4a769640 1 -- 172.21.15.38:0/1362030172 shutdown_connections 2024-04-25T08:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:20.999+0000 7fdb4a769640 1 -- 172.21.15.38:0/1362030172 wait complete. 2024-04-25T08:32:21.006 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:32:21.006 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:21.021 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:21.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.166+0000 7f0a750ac640 1 Processor -- start 2024-04-25T08:32:21.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.166+0000 7f0a750ac640 1 -- start start 2024-04-25T08:32:21.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.167+0000 7f0a750ac640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0a7009ab90 0x7f0a7015f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.167+0000 7f0a750ac640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0a700d0460 0x7f0a7015fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.167+0000 7f0a750ac640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a70160340 con 0x7f0a7015bb10 2024-04-25T08:32:21.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.167+0000 7f0a750ac640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a700cffd0 con 0x7f0a700d0460 2024-04-25T08:32:21.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.167+0000 7f0a750ac640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a700cf400 con 0x7f0a7009ab90 2024-04-25T08:32:21.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.167+0000 7f0a6f59e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0a700d0460 0x7f0a7015fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.167+0000 7f0a6f59e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0a700d0460 0x7f0a7015fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48596/0 (socket says 172.21.15.38:48596) 2024-04-25T08:32:21.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.167+0000 7f0a6f59e640 1 -- 172.21.15.38:0/1236428308 learned_addr learned my addr 172.21.15.38:0/1236428308 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:21.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.167+0000 7f0a6e59c640 1 --2- 172.21.15.38:0/1236428308 >> v2:172.21.15.107:3300/0 conn(0x7f0a7009ab90 0x7f0a7015f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.168+0000 7f0a6f59e640 1 -- 172.21.15.38:0/1236428308 >> v2:172.21.15.107:3300/0 conn(0x7f0a7009ab90 msgr2=0x7f0a7015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:21.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.168+0000 7f0a6f59e640 1 --2- 172.21.15.38:0/1236428308 >> v2:172.21.15.107:3300/0 conn(0x7f0a7009ab90 0x7f0a7015f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.168+0000 7f0a6f59e640 1 -- 172.21.15.38:0/1236428308 >> v1:172.21.15.107:6789/0 conn(0x7f0a7015bb10 legacy=0x7f0a7009a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.168+0000 7f0a6f59e640 1 -- 172.21.15.38:0/1236428308 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a700cf6a0 con 0x7f0a700d0460 2024-04-25T08:32:21.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.168+0000 7f0a6f59e640 1 --2- 172.21.15.38:0/1236428308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a700d0460 0x7f0a7015fa20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0a70067970 tx=0x7f0a6002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197df0a3d08bd735 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:21.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.168+0000 7f0a6dd9b640 1 -- 172.21.15.38:0/1236428308 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60036070 con 0x7f0a700d0460 2024-04-25T08:32:21.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.168+0000 7f0a6dd9b640 1 -- 172.21.15.38:0/1236428308 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a60031600 con 0x7f0a700d0460 2024-04-25T08:32:21.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.169+0000 7f0a6dd9b640 1 -- 172.21.15.38:0/1236428308 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a600319d0 con 0x7f0a700d0460 2024-04-25T08:32:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.169+0000 7f0a750ac640 1 -- 172.21.15.38:0/1236428308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a700d0460 msgr2=0x7f0a7015fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.169+0000 7f0a750ac640 1 --2- 172.21.15.38:0/1236428308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a700d0460 0x7f0a7015fa20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0a70067970 tx=0x7f0a6002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.169+0000 7f0a750ac640 1 -- 172.21.15.38:0/1236428308 shutdown_connections 2024-04-25T08:32:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.169+0000 7f0a750ac640 1 --2- 172.21.15.38:0/1236428308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a700d0460 0x7f0a7015fa20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.169+0000 7f0a750ac640 1 --2- 172.21.15.38:0/1236428308 >> v2:172.21.15.107:3300/0 conn(0x7f0a7009ab90 0x7f0a7015f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.169+0000 7f0a750ac640 1 -- 172.21.15.38:0/1236428308 >> 172.21.15.38:0/1236428308 conn(0x7f0a700b31f0 msgr2=0x7f0a700b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.169+0000 7f0a750ac640 1 -- 172.21.15.38:0/1236428308 shutdown_connections 2024-04-25T08:32:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.169+0000 7f0a750ac640 1 -- 172.21.15.38:0/1236428308 wait complete. 2024-04-25T08:32:21.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.171+0000 7f0a750ac640 1 Processor -- start 2024-04-25T08:32:21.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.171+0000 7f0a750ac640 1 -- start start 2024-04-25T08:32:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.171+0000 7f0a750ac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a7009ab90 0x7f0a70156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.171+0000 7f0a750ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a700d0460 0x7f0a700c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.171+0000 7f0a750ac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a7015bb10 0x7f0a7015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.171+0000 7f0a750ac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a700c5250 con 0x7f0a7009ab90 2024-04-25T08:32:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.171+0000 7f0a750ac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a700cfc60 con 0x7f0a7015bb10 2024-04-25T08:32:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.171+0000 7f0a750ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a700c53d0 con 0x7f0a700d0460 2024-04-25T08:32:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.171+0000 7f0a6ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a7009ab90 0x7f0a70156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a6ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a7009ab90 0x7f0a70156c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48606/0 (socket says 172.21.15.38:48606) 2024-04-25T08:32:21.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a6ed9d640 1 -- 172.21.15.38:0/1606666799 learned_addr learned my addr 172.21.15.38:0/1606666799 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a6e59c640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a700d0460 0x7f0a700c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a6f59e640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a7015bb10 0x7f0a7015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a6ed9d640 1 -- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a700d0460 msgr2=0x7f0a700c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a6ed9d640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a700d0460 0x7f0a700c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a6ed9d640 1 -- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a7015bb10 msgr2=0x7f0a7015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a6ed9d640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a7015bb10 0x7f0a7015d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a6ed9d640 1 -- 172.21.15.38:0/1606666799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a700ca1f0 con 0x7f0a7009ab90 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a6e59c640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a700d0460 0x7f0a700c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a6ed9d640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a7009ab90 0x7f0a70156c80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800c3c0 tx=0x7f0a6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a677fe640 1 -- 172.21.15.38:0/1606666799 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a680052d0 con 0x7f0a7009ab90 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a677fe640 1 -- 172.21.15.38:0/1606666799 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a68005490 con 0x7f0a7009ab90 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a750ac640 1 -- 172.21.15.38:0/1606666799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a700c8ef0 con 0x7f0a7009ab90 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a677fe640 1 -- 172.21.15.38:0/1606666799 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a68007260 con 0x7f0a7009ab90 2024-04-25T08:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.172+0000 7f0a750ac640 1 -- 172.21.15.38:0/1606666799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a700c94b0 con 0x7f0a7009ab90 2024-04-25T08:32:21.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.173+0000 7f0a750ac640 1 -- 172.21.15.38:0/1606666799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a70068e80 con 0x7f0a7009ab90 2024-04-25T08:32:21.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.174+0000 7f0a677fe640 1 -- 172.21.15.38:0/1606666799 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a68007400 con 0x7f0a7009ab90 2024-04-25T08:32:21.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.174+0000 7f0a677fe640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0a5006b4d0 0x7f0a5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.175+0000 7f0a677fe640 1 -- 172.21.15.38:0/1606666799 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f0a68089ab0 con 0x7f0a7009ab90 2024-04-25T08:32:21.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.175+0000 7f0a6e59c640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0a5006b4d0 0x7f0a5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.175+0000 7f0a6e59c640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0a5006b4d0 0x7f0a5006d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0a701e74f0 tx=0x7f0a58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:21.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.179+0000 7f0a677fe640 1 -- 172.21.15.38:0/1606666799 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a70160340 con 0x7f0a7009ab90 2024-04-25T08:32:21.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.360+0000 7f0a750ac640 1 -- 172.21.15.38:0/1606666799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a700cdbf0 con 0x7f0a7009ab90 2024-04-25T08:32:21.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.362+0000 7f0a677fe640 1 -- 172.21.15.38:0/1606666799 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18969 (secure 0 0 0) 0x7f0a68058560 con 0x7f0a7009ab90 2024-04-25T08:32:21.358 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:21.358 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:16.572337+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.a","mappings":[{"from":4,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:21.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.363+0000 7f0a750ac640 1 -- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0a5006b4d0 msgr2=0x7f0a5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.363+0000 7f0a750ac640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0a5006b4d0 0x7f0a5006d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0a701e74f0 tx=0x7f0a58008040 comp rx=0 tx=0).stop 2024-04-25T08:32:21.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.364+0000 7f0a750ac640 1 -- 172.21.15.38:0/1606666799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a7009ab90 msgr2=0x7f0a70156c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.364+0000 7f0a750ac640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a7009ab90 0x7f0a70156c80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0a6800c3c0 tx=0x7f0a6800c880 comp rx=0 tx=0).stop 2024-04-25T08:32:21.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.364+0000 7f0a750ac640 1 -- 172.21.15.38:0/1606666799 shutdown_connections 2024-04-25T08:32:21.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.364+0000 7f0a750ac640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a7015bb10 0x7f0a7015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.364+0000 7f0a750ac640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0a5006b4d0 0x7f0a5006d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.364+0000 7f0a750ac640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a700d0460 0x7f0a700c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.364+0000 7f0a750ac640 1 --2- 172.21.15.38:0/1606666799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a7009ab90 0x7f0a70156c80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.364+0000 7f0a750ac640 1 -- 172.21.15.38:0/1606666799 >> 172.21.15.38:0/1606666799 conn(0x7f0a700b31f0 msgr2=0x7f0a700c5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:21.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.364+0000 7f0a750ac640 1 -- 172.21.15.38:0/1606666799 shutdown_connections 2024-04-25T08:32:21.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.364+0000 7f0a750ac640 1 -- 172.21.15.38:0/1606666799 wait complete. 2024-04-25T08:32:21.371 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:21.371 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:21.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.479+0000 7f3eb54dd640 1 Processor -- start 2024-04-25T08:32:21.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.479+0000 7f3eb54dd640 1 -- start start 2024-04-25T08:32:21.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.480+0000 7f3eb54dd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3eb00b1d60 0x7f3eb00cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.480+0000 7f3eb54dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3eb00cff10 0x7f3eb0161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.480+0000 7f3eb54dd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3eb0161c50 0x7f3eb01620e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.480+0000 7f3eb54dd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb006abe0 con 0x7f3eb0161c50 2024-04-25T08:32:21.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.480+0000 7f3eb54dd640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb006b480 con 0x7f3eb00b1d60 2024-04-25T08:32:21.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.480+0000 7f3eb54dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb015ef40 con 0x7f3eb00cff10 2024-04-25T08:32:21.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.480+0000 7f3eaf7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3eb0161c50 0x7f3eb01620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.480+0000 7f3eaf7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3eb0161c50 0x7f3eb01620e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48616/0 (socket says 172.21.15.38:48616) 2024-04-25T08:32:21.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.480+0000 7f3eaf7fe640 1 -- 172.21.15.38:0/3075226412 learned_addr learned my addr 172.21.15.38:0/3075226412 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:21.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.480+0000 7f3eae7fc640 1 --2- 172.21.15.38:0/3075226412 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3eb00cff10 0x7f3eb0161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.480+0000 7f3eaeffd640 1 --2- 172.21.15.38:0/3075226412 >> v2:172.21.15.107:3300/0 conn(0x7f3eb00b1d60 0x7f3eb00cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.481+0000 7f3eaf7fe640 1 -- 172.21.15.38:0/3075226412 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3eb00cff10 msgr2=0x7f3eb0161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:21.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.481+0000 7f3eaf7fe640 1 --2- 172.21.15.38:0/3075226412 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3eb00cff10 0x7f3eb0161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.481+0000 7f3eaf7fe640 1 -- 172.21.15.38:0/3075226412 >> v2:172.21.15.107:3300/0 conn(0x7f3eb00b1d60 msgr2=0x7f3eb00cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:21.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.481+0000 7f3eaf7fe640 1 --2- 172.21.15.38:0/3075226412 >> v2:172.21.15.107:3300/0 conn(0x7f3eb00b1d60 0x7f3eb00cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.481+0000 7f3eaf7fe640 1 -- 172.21.15.38:0/3075226412 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb006ae30 con 0x7f3eb0161c50 2024-04-25T08:32:21.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.481+0000 7f3eaf7fe640 1 --2- 172.21.15.38:0/3075226412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb0161c50 0x7f3eb01620e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3eb006d780 tx=0x7f3ea002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d858d8fb80bad99b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:21.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.481+0000 7f3eadffb640 1 -- 172.21.15.38:0/3075226412 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0036070 con 0x7f3eb0161c50 2024-04-25T08:32:21.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.482+0000 7f3eadffb640 1 -- 172.21.15.38:0/3075226412 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea0031600 con 0x7f3eb0161c50 2024-04-25T08:32:21.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.482+0000 7f3eadffb640 1 -- 172.21.15.38:0/3075226412 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea00319d0 con 0x7f3eb0161c50 2024-04-25T08:32:21.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.482+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3075226412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb0161c50 msgr2=0x7f3eb01620e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.482+0000 7f3eb54dd640 1 --2- 172.21.15.38:0/3075226412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb0161c50 0x7f3eb01620e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3eb006d780 tx=0x7f3ea002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.482+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3075226412 shutdown_connections 2024-04-25T08:32:21.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.482+0000 7f3eb54dd640 1 --2- 172.21.15.38:0/3075226412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb0161c50 0x7f3eb01620e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.482+0000 7f3eb54dd640 1 --2- 172.21.15.38:0/3075226412 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3eb00cff10 0x7f3eb0161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.482+0000 7f3eb54dd640 1 --2- 172.21.15.38:0/3075226412 >> v2:172.21.15.107:3300/0 conn(0x7f3eb00b1d60 0x7f3eb00cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.482+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3075226412 >> 172.21.15.38:0/3075226412 conn(0x7f3eb00d06b0 msgr2=0x7f3eb00d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:21.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.482+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3075226412 shutdown_connections 2024-04-25T08:32:21.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.482+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3075226412 wait complete. 2024-04-25T08:32:21.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.483+0000 7f3eb54dd640 1 Processor -- start 2024-04-25T08:32:21.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.483+0000 7f3eb54dd640 1 -- start start 2024-04-25T08:32:21.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.484+0000 7f3eb54dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3eb00b1d60 0x7f3eb0151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.484+0000 7f3eb54dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb00cff10 0x7f3eb00c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.484+0000 7f3eb54dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3eb0161c50 0x7f3eb00c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.484+0000 7f3eb54dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb006b910 con 0x7f3eb00cff10 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.484+0000 7f3eb54dd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00d0370 con 0x7f3eb00b1d60 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.484+0000 7f3eb54dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb006ba90 con 0x7f3eb0161c50 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.484+0000 7f3eae7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb00cff10 0x7f3eb00c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.484+0000 7f3eae7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb00cff10 0x7f3eb00c41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48622/0 (socket says 172.21.15.38:48622) 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.484+0000 7f3eae7fc640 1 -- 172.21.15.38:0/3377344834 learned_addr learned my addr 172.21.15.38:0/3377344834 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.485+0000 7f3eae7fc640 1 -- 172.21.15.38:0/3377344834 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3eb0161c50 msgr2=0x7f3eb00c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.485+0000 7f3eae7fc640 1 --2- 172.21.15.38:0/3377344834 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3eb0161c50 0x7f3eb00c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.485+0000 7f3eae7fc640 1 -- 172.21.15.38:0/3377344834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3eb00b1d60 msgr2=0x7f3eb0151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.485+0000 7f3eae7fc640 1 --2- 172.21.15.38:0/3377344834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3eb00b1d60 0x7f3eb0151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.485+0000 7f3eae7fc640 1 -- 172.21.15.38:0/3377344834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb00c4d50 con 0x7f3eb00cff10 2024-04-25T08:32:21.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.485+0000 7f3eae7fc640 1 --2- 172.21.15.38:0/3377344834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb00cff10 0x7f3eb00c41e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800e8a0 tx=0x7f3e9800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:21.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.485+0000 7f3ea77fe640 1 -- 172.21.15.38:0/3377344834 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98017070 con 0x7f3eb00cff10 2024-04-25T08:32:21.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.485+0000 7f3ea77fe640 1 -- 172.21.15.38:0/3377344834 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9800c940 con 0x7f3eb00cff10 2024-04-25T08:32:21.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.485+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3377344834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb00c5030 con 0x7f3eb00cff10 2024-04-25T08:32:21.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.485+0000 7f3ea77fe640 1 -- 172.21.15.38:0/3377344834 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800cd30 con 0x7f3eb00cff10 2024-04-25T08:32:21.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.485+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3377344834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb00c55f0 con 0x7f3eb00cff10 2024-04-25T08:32:21.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.486+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3377344834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb00cfa40 con 0x7f3eb00cff10 2024-04-25T08:32:21.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.486+0000 7f3ea77fe640 1 -- 172.21.15.38:0/3377344834 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e9801a070 con 0x7f3eb00cff10 2024-04-25T08:32:21.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.487+0000 7f3ea77fe640 1 --2- 172.21.15.38:0/3377344834 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e9006b4d0 0x7f3e9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.487+0000 7f3ea77fe640 1 -- 172.21.15.38:0/3377344834 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f3e9808a940 con 0x7f3eb00cff10 2024-04-25T08:32:21.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.487+0000 7f3eaeffd640 1 --2- 172.21.15.38:0/3377344834 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e9006b4d0 0x7f3e9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.487+0000 7f3eaeffd640 1 --2- 172.21.15.38:0/3377344834 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e9006b4d0 0x7f3e9006d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80099a0 tx=0x7f3ea8009460 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:21.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.491+0000 7f3ea77fe640 1 -- 172.21.15.38:0/3377344834 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3eb00cfa40 con 0x7f3eb00cff10 2024-04-25T08:32:21.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.671+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3377344834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3eb0068e80 con 0x7f3eb00cff10 2024-04-25T08:32:21.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.671+0000 7f3ea77fe640 1 -- 172.21.15.38:0/3377344834 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e980592b0 con 0x7f3eb00cff10 2024-04-25T08:32:21.668 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:21.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.673+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3377344834 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e9006b4d0 msgr2=0x7f3e9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.673+0000 7f3eb54dd640 1 --2- 172.21.15.38:0/3377344834 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e9006b4d0 0x7f3e9006d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3ea80099a0 tx=0x7f3ea8009460 comp rx=0 tx=0).stop 2024-04-25T08:32:21.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.673+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3377344834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb00cff10 msgr2=0x7f3eb00c41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.673+0000 7f3eb54dd640 1 --2- 172.21.15.38:0/3377344834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb00cff10 0x7f3eb00c41e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3e9800e8a0 tx=0x7f3e9800ed60 comp rx=0 tx=0).stop 2024-04-25T08:32:21.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.673+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3377344834 shutdown_connections 2024-04-25T08:32:21.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.673+0000 7f3eb54dd640 1 --2- 172.21.15.38:0/3377344834 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e9006b4d0 0x7f3e9006d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.673+0000 7f3eb54dd640 1 --2- 172.21.15.38:0/3377344834 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3eb0161c50 0x7f3eb00c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.673+0000 7f3eb54dd640 1 --2- 172.21.15.38:0/3377344834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3eb00cff10 0x7f3eb00c41e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.673+0000 7f3eb54dd640 1 --2- 172.21.15.38:0/3377344834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3eb00b1d60 0x7f3eb0151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.673+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3377344834 >> 172.21.15.38:0/3377344834 conn(0x7f3eb00d06b0 msgr2=0x7f3eb0160e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:21.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.673+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3377344834 shutdown_connections 2024-04-25T08:32:21.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.673+0000 7f3eb54dd640 1 -- 172.21.15.38:0/3377344834 wait complete. 2024-04-25T08:32:21.680 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:21.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.788+0000 7f68055e7640 1 Processor -- start 2024-04-25T08:32:21.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.788+0000 7f68055e7640 1 -- start start 2024-04-25T08:32:21.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.789+0000 7f68055e7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f68000cf580 0x7f680015f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.789+0000 7f68055e7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68000cf9f0 con 0x7f680015b990 2024-04-25T08:32:21.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.789+0000 7f68055e7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680015fd00 con 0x7f68000d1d20 2024-04-25T08:32:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.789+0000 7f68055e7640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6800160580 con 0x7f68000cf580 2024-04-25T08:32:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.789+0000 7f67feffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f680015b990 0x7f68000d1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34918/0 (socket says 172.21.15.38:34918) 2024-04-25T08:32:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.789+0000 7f67feffd640 1 -- 172.21.15.38:0/2092176336 learned_addr learned my addr 172.21.15.38:0/2092176336 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.789+0000 7f67ff7fe640 1 --2- 172.21.15.38:0/2092176336 >> v2:172.21.15.38:3300/0 conn(0x7f68000cf580 0x7f680015f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.789+0000 7f67fdffb640 1 -- 172.21.15.38:0/2092176336 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168292115 0 0) 0x7f68000cf9f0 con 0x7f680015b990 2024-04-25T08:32:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.789+0000 7f67fdffb640 1 -- 172.21.15.38:0/2092176336 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67e8003520 con 0x7f680015b990 2024-04-25T08:32:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.789+0000 7f67ff7fe640 1 -- 172.21.15.38:0/2092176336 >> v1:172.21.15.107:6789/0 conn(0x7f68000d1d20 legacy=0x7f68000b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.789+0000 7f67ff7fe640 1 -- 172.21.15.38:0/2092176336 >> v1:172.21.15.38:6789/0 conn(0x7f680015b990 legacy=0x7f68000d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.789+0000 7f67ff7fe640 1 -- 172.21.15.38:0/2092176336 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680015ffc0 con 0x7f68000cf580 2024-04-25T08:32:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.790+0000 7f67ff7fe640 1 --2- 172.21.15.38:0/2092176336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000cf580 0x7f680015f5f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009d90 tx=0x7f67f802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=760739ad509c35f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.790+0000 7f67fdffb640 1 -- 172.21.15.38:0/2092176336 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8031600 con 0x7f68000cf580 2024-04-25T08:32:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.790+0000 7f67fdffb640 1 -- 172.21.15.38:0/2092176336 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f80317a0 con 0x7f68000cf580 2024-04-25T08:32:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.790+0000 7f67fdffb640 1 -- 172.21.15.38:0/2092176336 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8031b70 con 0x7f68000cf580 2024-04-25T08:32:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.791+0000 7f68055e7640 1 -- 172.21.15.38:0/2092176336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000cf580 msgr2=0x7f680015f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.791+0000 7f68055e7640 1 --2- 172.21.15.38:0/2092176336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000cf580 0x7f680015f5f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f67f8009d90 tx=0x7f67f802a610 comp rx=0 tx=0).stop 2024-04-25T08:32:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.791+0000 7f68055e7640 1 -- 172.21.15.38:0/2092176336 shutdown_connections 2024-04-25T08:32:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.791+0000 7f68055e7640 1 --2- 172.21.15.38:0/2092176336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000cf580 0x7f680015f5f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.791+0000 7f68055e7640 1 -- 172.21.15.38:0/2092176336 >> 172.21.15.38:0/2092176336 conn(0x7f680008b000 msgr2=0x7f680008b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.791+0000 7f68055e7640 1 -- 172.21.15.38:0/2092176336 shutdown_connections 2024-04-25T08:32:21.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.791+0000 7f68055e7640 1 -- 172.21.15.38:0/2092176336 wait complete. 2024-04-25T08:32:21.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.792+0000 7f68055e7640 1 Processor -- start 2024-04-25T08:32:21.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.792+0000 7f68055e7640 1 -- start start 2024-04-25T08:32:21.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f68055e7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68000cf580 0x7f68001524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f68055e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000d1d20 0x7f680015e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f68055e7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f680015b990 0x7f680015e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f68055e7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680015bd70 con 0x7f68000d1d20 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f68055e7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68000b1ac0 con 0x7f68000cf580 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f68055e7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680015bef0 con 0x7f680015b990 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67fe7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000d1d20 0x7f680015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67fe7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000d1d20 0x7f680015e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48636/0 (socket says 172.21.15.38:48636) 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67fe7fc640 1 -- 172.21.15.38:0/4071738251 learned_addr learned my addr 172.21.15.38:0/4071738251 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67feffd640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68000cf580 0x7f68001524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67ff7fe640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f680015b990 0x7f680015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67fe7fc640 1 -- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f680015b990 msgr2=0x7f680015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67fe7fc640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f680015b990 0x7f680015e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67fe7fc640 1 -- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68000cf580 msgr2=0x7f68001524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67fe7fc640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68000cf580 0x7f68001524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67fe7fc640 1 -- 172.21.15.38:0/4071738251 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68000cdc20 con 0x7f68000d1d20 2024-04-25T08:32:21.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67fe7fc640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000d1d20 0x7f680015e450 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c430 tx=0x7f67f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:21.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67e77fe640 1 -- 172.21.15.38:0/4071738251 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4011790 con 0x7f68000d1d20 2024-04-25T08:32:21.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67e77fe640 1 -- 172.21.15.38:0/4071738251 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f4017070 con 0x7f68000d1d20 2024-04-25T08:32:21.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f68055e7640 1 -- 172.21.15.38:0/4071738251 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680015cc80 con 0x7f68000d1d20 2024-04-25T08:32:21.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.793+0000 7f67e77fe640 1 -- 172.21.15.38:0/4071738251 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4011b70 con 0x7f68000d1d20 2024-04-25T08:32:21.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.794+0000 7f68055e7640 1 -- 172.21.15.38:0/4071738251 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680015d240 con 0x7f68000d1d20 2024-04-25T08:32:21.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.794+0000 7f68055e7640 1 -- 172.21.15.38:0/4071738251 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6800068e80 con 0x7f68000d1d20 2024-04-25T08:32:21.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.795+0000 7f67e77fe640 1 -- 172.21.15.38:0/4071738251 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67f4011d10 con 0x7f68000d1d20 2024-04-25T08:32:21.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.795+0000 7f67e77fe640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f67ec06b4d0 0x7f67ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:21.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.795+0000 7f67e77fe640 1 -- 172.21.15.38:0/4071738251 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f67f408a5c0 con 0x7f68000d1d20 2024-04-25T08:32:21.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.795+0000 7f67feffd640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f67ec06b4d0 0x7f67ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:21.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.796+0000 7f67feffd640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f67ec06b4d0 0x7f67ec06d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f67f0000f60 tx=0x7f67f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:21.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.799+0000 7f67e77fe640 1 -- 172.21.15.38:0/4071738251 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67f4058f40 con 0x7f68000d1d20 2024-04-25T08:32:21.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.983+0000 7f68055e7640 1 -- 172.21.15.38:0/4071738251 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68000b1220 con 0x7f68000d1d20 2024-04-25T08:32:21.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.983+0000 7f67e77fe640 1 -- 172.21.15.38:0/4071738251 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67f4058840 con 0x7f68000d1d20 2024-04-25T08:32:21.979 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:21.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.985+0000 7f68055e7640 1 -- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f67ec06b4d0 msgr2=0x7f67ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.985+0000 7f68055e7640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f67ec06b4d0 0x7f67ec06d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f67f0000f60 tx=0x7f67f0009040 comp rx=0 tx=0).stop 2024-04-25T08:32:21.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.985+0000 7f68055e7640 1 -- 172.21.15.38:0/4071738251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000d1d20 msgr2=0x7f680015e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:21.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.985+0000 7f68055e7640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000d1d20 0x7f680015e450 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f67f400c430 tx=0x7f67f400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.985+0000 7f68055e7640 1 -- 172.21.15.38:0/4071738251 shutdown_connections 2024-04-25T08:32:21.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.985+0000 7f68055e7640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f67ec06b4d0 0x7f67ec06d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.985+0000 7f68055e7640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f680015b990 0x7f680015e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.985+0000 7f68055e7640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68000d1d20 0x7f680015e450 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.985+0000 7f68055e7640 1 --2- 172.21.15.38:0/4071738251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68000cf580 0x7f68001524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:21.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.985+0000 7f68055e7640 1 -- 172.21.15.38:0/4071738251 >> 172.21.15.38:0/4071738251 conn(0x7f680008b000 msgr2=0x7f6800159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:21.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.985+0000 7f68055e7640 1 -- 172.21.15.38:0/4071738251 shutdown_connections 2024-04-25T08:32:21.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:21.985+0000 7f68055e7640 1 -- 172.21.15.38:0/4071738251 wait complete. 2024-04-25T08:32:21.993 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:22.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.100+0000 7fd7a08b3640 1 Processor -- start 2024-04-25T08:32:22.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.100+0000 7fd7a08b3640 1 -- start start 2024-04-25T08:32:22.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.101+0000 7fd7a08b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd79c0d33d0 0x7fd79c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.101+0000 7fd7a08b3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd79c0d1e90 0x7fd79c0b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.101+0000 7fd7a08b3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd79c0d2330 con 0x7fd79c0d0320 2024-04-25T08:32:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.101+0000 7fd7a08b3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd79c0d2610 con 0x7fd79c0d1e90 2024-04-25T08:32:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.101+0000 7fd7a08b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c0d28f0 con 0x7fd79c0d33d0 2024-04-25T08:32:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.101+0000 7fd79ad9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd79c0d1e90 0x7fd79c0b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.101+0000 7fd79a59c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd79c0d0320 0x7fd79c0d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34924/0 (socket says 172.21.15.38:34924) 2024-04-25T08:32:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.101+0000 7fd79a59c640 1 -- 172.21.15.38:0/2073914088 learned_addr learned my addr 172.21.15.38:0/2073914088 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.101+0000 7fd799d9b640 1 --2- 172.21.15.38:0/2073914088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd79c0d33d0 0x7fd79c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.102+0000 7fd79ad9d640 1 -- 172.21.15.38:0/2073914088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd79c0d33d0 msgr2=0x7fd79c0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.102+0000 7fd79ad9d640 1 --2- 172.21.15.38:0/2073914088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd79c0d33d0 0x7fd79c0d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.102+0000 7fd79959a640 1 -- 172.21.15.38:0/2073914088 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060635767 0 0) 0x7fd79c0d2330 con 0x7fd79c0d0320 2024-04-25T08:32:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.102+0000 7fd79ad9d640 1 -- 172.21.15.38:0/2073914088 >> v1:172.21.15.38:6789/0 conn(0x7fd79c0d0320 legacy=0x7fd79c0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.102+0000 7fd79ad9d640 1 -- 172.21.15.38:0/2073914088 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c0d2bd0 con 0x7fd79c0d1e90 2024-04-25T08:32:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.102+0000 7fd79ad9d640 1 --2- 172.21.15.38:0/2073914088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd79c0d1e90 0x7fd79c0b6980 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd794009d90 tx=0x7fd79402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a7190e50200fea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.102+0000 7fd79959a640 1 -- 172.21.15.38:0/2073914088 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794007740 con 0x7fd79c0d1e90 2024-04-25T08:32:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.102+0000 7fd79959a640 1 -- 172.21.15.38:0/2073914088 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7940078e0 con 0x7fd79c0d1e90 2024-04-25T08:32:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.102+0000 7fd79959a640 1 -- 172.21.15.38:0/2073914088 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794007cb0 con 0x7fd79c0d1e90 2024-04-25T08:32:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.103+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2073914088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd79c0d1e90 msgr2=0x7fd79c0b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.103+0000 7fd7a08b3640 1 --2- 172.21.15.38:0/2073914088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd79c0d1e90 0x7fd79c0b6980 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd794009d90 tx=0x7fd79402a6f0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.103+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2073914088 shutdown_connections 2024-04-25T08:32:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.103+0000 7fd7a08b3640 1 --2- 172.21.15.38:0/2073914088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd79c0d1e90 0x7fd79c0b6980 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.103+0000 7fd7a08b3640 1 --2- 172.21.15.38:0/2073914088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd79c0d33d0 0x7fd79c0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.103+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2073914088 >> 172.21.15.38:0/2073914088 conn(0x7fd79c0668f0 msgr2=0x7fd79c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.103+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2073914088 shutdown_connections 2024-04-25T08:32:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.103+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2073914088 wait complete. 2024-04-25T08:32:22.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.104+0000 7fd7a08b3640 1 Processor -- start 2024-04-25T08:32:22.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.104+0000 7fd7a08b3640 1 -- start start 2024-04-25T08:32:22.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.105+0000 7fd7a08b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd79c0d0320 0x7fd79c1f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.105+0000 7fd7a08b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd79c0d1e90 0x7fd79c0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.105+0000 7fd7a08b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd79c0d33d0 0x7fd79c0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.105+0000 7fd7a08b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c161390 con 0x7fd79c0d33d0 2024-04-25T08:32:22.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.105+0000 7fd7a08b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c161210 con 0x7fd79c0d1e90 2024-04-25T08:32:22.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.105+0000 7fd7a08b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79c0b9720 con 0x7fd79c0d0320 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.105+0000 7fd79ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd79c0d33d0 0x7fd79c0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.105+0000 7fd79ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd79c0d33d0 0x7fd79c0b5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48654/0 (socket says 172.21.15.38:48654) 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.105+0000 7fd79ad9d640 1 -- 172.21.15.38:0/2339863800 learned_addr learned my addr 172.21.15.38:0/2339863800 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.105+0000 7fd799d9b640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd79c0d1e90 0x7fd79c0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.105+0000 7fd79a59c640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd79c0d0320 0x7fd79c1f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.106+0000 7fd79ad9d640 1 -- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd79c0d0320 msgr2=0x7fd79c1f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.106+0000 7fd79ad9d640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd79c0d0320 0x7fd79c1f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.106+0000 7fd79ad9d640 1 -- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd79c0d1e90 msgr2=0x7fd79c0b5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.106+0000 7fd79ad9d640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd79c0d1e90 0x7fd79c0b5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.106+0000 7fd79ad9d640 1 -- 172.21.15.38:0/2339863800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd79c1f3700 con 0x7fd79c0d33d0 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.106+0000 7fd79ad9d640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd79c0d33d0 0x7fd79c0b5eb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd794009eb0 tx=0x7fd794004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.106+0000 7fd78affd640 1 -- 172.21.15.38:0/2339863800 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794040070 con 0x7fd79c0d33d0 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.106+0000 7fd78affd640 1 -- 172.21.15.38:0/2339863800 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd794031bc0 con 0x7fd79c0d33d0 2024-04-25T08:32:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.106+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2339863800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79c1f39e0 con 0x7fd79c0d33d0 2024-04-25T08:32:22.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.106+0000 7fd78affd640 1 -- 172.21.15.38:0/2339863800 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794009450 con 0x7fd79c0d33d0 2024-04-25T08:32:22.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.106+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2339863800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79c1f3fa0 con 0x7fd79c0d33d0 2024-04-25T08:32:22.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.107+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2339863800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd79c068e80 con 0x7fd79c0d33d0 2024-04-25T08:32:22.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.108+0000 7fd78affd640 1 -- 172.21.15.38:0/2339863800 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd794039040 con 0x7fd79c0d33d0 2024-04-25T08:32:22.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.108+0000 7fd78affd640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd77c06b500 0x7fd77c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.108+0000 7fd78affd640 1 -- 172.21.15.38:0/2339863800 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fd7940abcd0 con 0x7fd79c0d33d0 2024-04-25T08:32:22.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.108+0000 7fd79a59c640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd77c06b500 0x7fd77c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.109+0000 7fd79a59c640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd77c06b500 0x7fd77c06d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd78c007fd0 tx=0x7fd78c007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.112+0000 7fd78affd640 1 -- 172.21.15.38:0/2339863800 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd79407a0a0 con 0x7fd79c0d33d0 2024-04-25T08:32:22.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.292+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2339863800 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd79c0b3050 con 0x7fd79c0d33d0 2024-04-25T08:32:22.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.292+0000 7fd78affd640 1 -- 172.21.15.38:0/2339863800 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd79407e140 con 0x7fd79c0d33d0 2024-04-25T08:32:22.289 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:22.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.294+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd77c06b500 msgr2=0x7fd77c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.294+0000 7fd7a08b3640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd77c06b500 0x7fd77c06d9c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd78c007fd0 tx=0x7fd78c007de0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.294+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2339863800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd79c0d33d0 msgr2=0x7fd79c0b5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.294+0000 7fd7a08b3640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd79c0d33d0 0x7fd79c0b5eb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd794009eb0 tx=0x7fd794004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:22.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.294+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2339863800 shutdown_connections 2024-04-25T08:32:22.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.294+0000 7fd7a08b3640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd79c0d33d0 0x7fd79c0b5eb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.294+0000 7fd7a08b3640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd79c0d1e90 0x7fd79c0b5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.294+0000 7fd7a08b3640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd77c06b500 0x7fd77c06d9c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.294+0000 7fd7a08b3640 1 --2- 172.21.15.38:0/2339863800 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd79c0d0320 0x7fd79c1f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.294+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2339863800 >> 172.21.15.38:0/2339863800 conn(0x7fd79c0668f0 msgr2=0x7fd79c0b6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:22.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.294+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2339863800 shutdown_connections 2024-04-25T08:32:22.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.294+0000 7fd7a08b3640 1 -- 172.21.15.38:0/2339863800 wait complete. 2024-04-25T08:32:22.301 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:22.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.408+0000 7fa100be2640 1 Processor -- start 2024-04-25T08:32:22.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.408+0000 7fa100be2640 1 -- start start 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.409+0000 7fa100be2640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa0fc06c700 0x7fa0fc162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.409+0000 7fa100be2640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa0fc06cae0 0x7fa0fc15e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.409+0000 7fa100be2640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0fc15ec20 con 0x7fa0fc15e580 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.409+0000 7fa100be2640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0fc15e9f0 con 0x7fa0fc06cae0 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.409+0000 7fa100be2640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0fc161c80 con 0x7fa0fc06c700 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.409+0000 7fa0f9d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa0fc06cae0 0x7fa0fc15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.409+0000 7fa0fad9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa0fc15e580 0x7fa0fc0cdf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34940/0 (socket says 172.21.15.38:34940) 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.409+0000 7fa0fad9d640 1 -- 172.21.15.38:0/3214665001 learned_addr learned my addr 172.21.15.38:0/3214665001 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.410+0000 7fa0fa59c640 1 --2- 172.21.15.38:0/3214665001 >> v2:172.21.15.107:3300/0 conn(0x7fa0fc06c700 0x7fa0fc162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.410+0000 7fa0f9d9b640 1 -- 172.21.15.38:0/3214665001 >> v2:172.21.15.107:3300/0 conn(0x7fa0fc06c700 msgr2=0x7fa0fc162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.410+0000 7fa0f9d9b640 1 --2- 172.21.15.38:0/3214665001 >> v2:172.21.15.107:3300/0 conn(0x7fa0fc06c700 0x7fa0fc162110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.410+0000 7fa0f9d9b640 1 -- 172.21.15.38:0/3214665001 >> v1:172.21.15.38:6789/0 conn(0x7fa0fc15e580 legacy=0x7fa0fc0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.410+0000 7fa0f9d9b640 1 -- 172.21.15.38:0/3214665001 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc15ef00 con 0x7fa0fc06cae0 2024-04-25T08:32:22.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.410+0000 7fa0f959a640 1 -- 172.21.15.38:0/3214665001 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817854731 0 0) 0x7fa0fc15ec20 con 0x7fa0fc15e580 2024-04-25T08:32:22.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.411+0000 7fa0f9d9b640 1 --2- 172.21.15.38:0/3214665001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0fc06cae0 0x7fa0fc15e040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0d25f0 tx=0x7fa0e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5386dac4272c709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:22.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.411+0000 7fa0f959a640 1 -- 172.21.15.38:0/3214665001 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4037070 con 0x7fa0fc06cae0 2024-04-25T08:32:22.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.411+0000 7fa0f959a640 1 -- 172.21.15.38:0/3214665001 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e4032440 con 0x7fa0fc06cae0 2024-04-25T08:32:22.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.411+0000 7fa0f959a640 1 -- 172.21.15.38:0/3214665001 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4032810 con 0x7fa0fc06cae0 2024-04-25T08:32:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.411+0000 7fa100be2640 1 -- 172.21.15.38:0/3214665001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0fc06cae0 msgr2=0x7fa0fc15e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.411+0000 7fa100be2640 1 --2- 172.21.15.38:0/3214665001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0fc06cae0 0x7fa0fc15e040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0d25f0 tx=0x7fa0e402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.411+0000 7fa100be2640 1 -- 172.21.15.38:0/3214665001 shutdown_connections 2024-04-25T08:32:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.411+0000 7fa100be2640 1 --2- 172.21.15.38:0/3214665001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0fc06cae0 0x7fa0fc15e040 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.411+0000 7fa100be2640 1 --2- 172.21.15.38:0/3214665001 >> v2:172.21.15.107:3300/0 conn(0x7fa0fc06c700 0x7fa0fc162110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.411+0000 7fa100be2640 1 -- 172.21.15.38:0/3214665001 >> 172.21.15.38:0/3214665001 conn(0x7fa0fc08b000 msgr2=0x7fa0fc08b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:22.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.411+0000 7fa100be2640 1 -- 172.21.15.38:0/3214665001 shutdown_connections 2024-04-25T08:32:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.411+0000 7fa100be2640 1 -- 172.21.15.38:0/3214665001 wait complete. 2024-04-25T08:32:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.413+0000 7fa100be2640 1 Processor -- start 2024-04-25T08:32:22.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.413+0000 7fa100be2640 1 -- start start 2024-04-25T08:32:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.413+0000 7fa100be2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0fc06c700 0x7fa0fc154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.413+0000 7fa100be2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0fc06cae0 0x7fa0fc1602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.413+0000 7fa100be2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0fc15e580 0x7fa0fc1607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.413+0000 7fa100be2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc06c190 con 0x7fa0fc15e580 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.413+0000 7fa100be2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc161910 con 0x7fa0fc06cae0 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.413+0000 7fa100be2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0fc06cf90 con 0x7fa0fc06c700 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0fad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0fc15e580 0x7fa0fc1607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0fad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0fc15e580 0x7fa0fc1607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48666/0 (socket says 172.21.15.38:48666) 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0fad9d640 1 -- 172.21.15.38:0/3736657326 learned_addr learned my addr 172.21.15.38:0/3736657326 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0f9d9b640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0fc06cae0 0x7fa0fc1602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0fa59c640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0fc06c700 0x7fa0fc154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0fad9d640 1 -- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0fc06c700 msgr2=0x7fa0fc154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0fad9d640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0fc06c700 0x7fa0fc154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0fad9d640 1 -- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0fc06cae0 msgr2=0x7fa0fc1602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0fad9d640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0fc06cae0 0x7fa0fc1602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0fad9d640 1 -- 172.21.15.38:0/3736657326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0fc0af440 con 0x7fa0fc15e580 2024-04-25T08:32:22.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0f9d9b640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0fc06cae0 0x7fa0fc1602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0fa59c640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0fc06c700 0x7fa0fc154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0fad9d640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0fc15e580 0x7fa0fc1607e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400c410 tx=0x7fa0f400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0f2ffd640 1 -- 172.21.15.38:0/3736657326 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40074e0 con 0x7fa0fc15e580 2024-04-25T08:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0f2ffd640 1 -- 172.21.15.38:0/3736657326 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f40047e0 con 0x7fa0fc15e580 2024-04-25T08:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa0f2ffd640 1 -- 172.21.15.38:0/3736657326 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4004bd0 con 0x7fa0fc15e580 2024-04-25T08:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa100be2640 1 -- 172.21.15.38:0/3736657326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0fc0af720 con 0x7fa0fc15e580 2024-04-25T08:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.414+0000 7fa100be2640 1 -- 172.21.15.38:0/3736657326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0fc1599c0 con 0x7fa0fc15e580 2024-04-25T08:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.415+0000 7fa100be2640 1 -- 172.21.15.38:0/3736657326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0fc068e80 con 0x7fa0fc15e580 2024-04-25T08:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.416+0000 7fa0f2ffd640 1 -- 172.21.15.38:0/3736657326 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0f4011070 con 0x7fa0fc15e580 2024-04-25T08:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.416+0000 7fa0f2ffd640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa0dc06b4d0 0x7fa0dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.416+0000 7fa0f2ffd640 1 -- 172.21.15.38:0/3736657326 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fa0f408a170 con 0x7fa0fc15e580 2024-04-25T08:32:22.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.417+0000 7fa0fa59c640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa0dc06b4d0 0x7fa0dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.417+0000 7fa0fa59c640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa0dc06b4d0 0x7fa0dc06d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0c7d50 tx=0x7fa0ec0039f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:22.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.421+0000 7fa0f2ffd640 1 -- 172.21.15.38:0/3736657326 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0f401a2b0 con 0x7fa0fc15e580 2024-04-25T08:32:22.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.598+0000 7fa100be2640 1 -- 172.21.15.38:0/3736657326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa0fc0b1220 con 0x7fa0fc15e580 2024-04-25T08:32:22.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.599+0000 7fa0f2ffd640 1 -- 172.21.15.38:0/3736657326 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa0f4058c20 con 0x7fa0fc15e580 2024-04-25T08:32:22.595 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:22.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.600+0000 7fa100be2640 1 -- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa0dc06b4d0 msgr2=0x7fa0dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.600+0000 7fa100be2640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa0dc06b4d0 0x7fa0dc06d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc0c7d50 tx=0x7fa0ec0039f0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.600+0000 7fa100be2640 1 -- 172.21.15.38:0/3736657326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0fc15e580 msgr2=0x7fa0fc1607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.600+0000 7fa100be2640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0fc15e580 0x7fa0fc1607e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400c410 tx=0x7fa0f400c8d0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.600+0000 7fa100be2640 1 -- 172.21.15.38:0/3736657326 shutdown_connections 2024-04-25T08:32:22.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.600+0000 7fa100be2640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0fc15e580 0x7fa0fc1607e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.600+0000 7fa100be2640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0fc06cae0 0x7fa0fc1602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.600+0000 7fa100be2640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa0dc06b4d0 0x7fa0dc06d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.600+0000 7fa100be2640 1 --2- 172.21.15.38:0/3736657326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0fc06c700 0x7fa0fc154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.600+0000 7fa100be2640 1 -- 172.21.15.38:0/3736657326 >> 172.21.15.38:0/3736657326 conn(0x7fa0fc08b000 msgr2=0x7fa0fc06f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:22.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.601+0000 7fa100be2640 1 -- 172.21.15.38:0/3736657326 shutdown_connections 2024-04-25T08:32:22.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.601+0000 7fa100be2640 1 -- 172.21.15.38:0/3736657326 wait complete. 2024-04-25T08:32:22.608 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:32:22.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.714+0000 7f2ef8037640 1 Processor -- start 2024-04-25T08:32:22.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.715+0000 7f2ef8037640 1 -- start start 2024-04-25T08:32:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.715+0000 7f2ef8037640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ef0099ce0 0x7f2ef009a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.715+0000 7f2ef8037640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2ef009a0c0 0x7f2ef00b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.715+0000 7f2ef8037640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2ef015eb20 0x7f2ef015efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.715+0000 7f2ef8037640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef015f4f0 con 0x7f2ef009a0c0 2024-04-25T08:32:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.715+0000 7f2ef8037640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef00cf6c0 con 0x7f2ef015eb20 2024-04-25T08:32:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.715+0000 7f2ef8037640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00cd120 con 0x7f2ef0099ce0 2024-04-25T08:32:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.715+0000 7f2ef55d2640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2ef009a0c0 0x7f2ef00b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.716+0000 7f2ef55d2640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2ef009a0c0 0x7f2ef00b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48674/0 (socket says 172.21.15.38:48674) 2024-04-25T08:32:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.716+0000 7f2ef55d2640 1 -- 172.21.15.38:0/3908755952 learned_addr learned my addr 172.21.15.38:0/3908755952 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.716+0000 7f2ef65d4640 1 --2- 172.21.15.38:0/3908755952 >> v2:172.21.15.107:3300/0 conn(0x7f2ef015eb20 0x7f2ef015efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.716+0000 7f2ef5dd3640 1 --2- 172.21.15.38:0/3908755952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ef0099ce0 0x7f2ef009a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.716+0000 7f2ef55d2640 1 -- 172.21.15.38:0/3908755952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ef0099ce0 msgr2=0x7f2ef009a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.716+0000 7f2ef55d2640 1 --2- 172.21.15.38:0/3908755952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ef0099ce0 0x7f2ef009a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.716+0000 7f2ef55d2640 1 -- 172.21.15.38:0/3908755952 >> v2:172.21.15.107:3300/0 conn(0x7f2ef015eb20 msgr2=0x7f2ef015efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.716+0000 7f2ef55d2640 1 --2- 172.21.15.38:0/3908755952 >> v2:172.21.15.107:3300/0 conn(0x7f2ef015eb20 0x7f2ef015efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.716+0000 7f2ef55d2640 1 -- 172.21.15.38:0/3908755952 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef00cd400 con 0x7f2ef009a0c0 2024-04-25T08:32:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.716+0000 7f2ef65d4640 1 --2- 172.21.15.38:0/3908755952 >> v2:172.21.15.107:3300/0 conn(0x7f2ef015eb20 0x7f2ef015efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.716+0000 7f2ef55d2640 1 --2- 172.21.15.38:0/3908755952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ef009a0c0 0x7f2ef00b4450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0073100 tx=0x7f2ee002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5afd2b00da315538 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.717+0000 7f2ef4dd1640 1 -- 172.21.15.38:0/3908755952 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0037070 con 0x7f2ef009a0c0 2024-04-25T08:32:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.717+0000 7f2ef4dd1640 1 -- 172.21.15.38:0/3908755952 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee0032440 con 0x7f2ef009a0c0 2024-04-25T08:32:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.717+0000 7f2ef4dd1640 1 -- 172.21.15.38:0/3908755952 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0032810 con 0x7f2ef009a0c0 2024-04-25T08:32:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.717+0000 7f2ef8037640 1 -- 172.21.15.38:0/3908755952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ef009a0c0 msgr2=0x7f2ef00b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.717+0000 7f2ef8037640 1 --2- 172.21.15.38:0/3908755952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ef009a0c0 0x7f2ef00b4450 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0073100 tx=0x7f2ee002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.717+0000 7f2ef8037640 1 -- 172.21.15.38:0/3908755952 shutdown_connections 2024-04-25T08:32:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.717+0000 7f2ef8037640 1 --2- 172.21.15.38:0/3908755952 >> v2:172.21.15.107:3300/0 conn(0x7f2ef015eb20 0x7f2ef015efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.717+0000 7f2ef8037640 1 --2- 172.21.15.38:0/3908755952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ef009a0c0 0x7f2ef00b4450 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.717+0000 7f2ef8037640 1 --2- 172.21.15.38:0/3908755952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ef0099ce0 0x7f2ef009a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.717+0000 7f2ef8037640 1 -- 172.21.15.38:0/3908755952 >> 172.21.15.38:0/3908755952 conn(0x7f2ef00b30f0 msgr2=0x7f2ef00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.717+0000 7f2ef8037640 1 -- 172.21.15.38:0/3908755952 shutdown_connections 2024-04-25T08:32:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.717+0000 7f2ef8037640 1 -- 172.21.15.38:0/3908755952 wait complete. 2024-04-25T08:32:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.718+0000 7f2ef8037640 1 Processor -- start 2024-04-25T08:32:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.719+0000 7f2ef8037640 1 -- start start 2024-04-25T08:32:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.719+0000 7f2ef8037640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ef0099ce0 0x7f2ef015a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.719+0000 7f2ef8037640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ef009a0c0 0x7f2ef009be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.719+0000 7f2ef8037640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ef015eb20 0x7f2ef01f6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.719+0000 7f2ef8037640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00b4990 con 0x7f2ef009a0c0 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.719+0000 7f2ef8037640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef009acf0 con 0x7f2ef015eb20 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.719+0000 7f2ef8037640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00b4b10 con 0x7f2ef0099ce0 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.719+0000 7f2ef55d2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ef009a0c0 0x7f2ef009be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.719+0000 7f2ef55d2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ef009a0c0 0x7f2ef009be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48690/0 (socket says 172.21.15.38:48690) 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.719+0000 7f2ef55d2640 1 -- 172.21.15.38:0/4255245699 learned_addr learned my addr 172.21.15.38:0/4255245699 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.720+0000 7f2ef55d2640 1 -- 172.21.15.38:0/4255245699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ef0099ce0 msgr2=0x7f2ef015a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.720+0000 7f2ef55d2640 1 --2- 172.21.15.38:0/4255245699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ef0099ce0 0x7f2ef015a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.720+0000 7f2ef55d2640 1 -- 172.21.15.38:0/4255245699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ef015eb20 msgr2=0x7f2ef01f6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.720+0000 7f2ef55d2640 1 --2- 172.21.15.38:0/4255245699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ef015eb20 0x7f2ef01f6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.720+0000 7f2ef55d2640 1 -- 172.21.15.38:0/4255245699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef009c770 con 0x7f2ef009a0c0 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.720+0000 7f2ef55d2640 1 --2- 172.21.15.38:0/4255245699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ef009a0c0 0x7f2ef009be90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0000c00 tx=0x7f2ee00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:22.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.720+0000 7f2ee67fc640 1 -- 172.21.15.38:0/4255245699 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0041070 con 0x7f2ef009a0c0 2024-04-25T08:32:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.720+0000 7f2ee67fc640 1 -- 172.21.15.38:0/4255245699 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee003c870 con 0x7f2ef009a0c0 2024-04-25T08:32:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.720+0000 7f2ef8037640 1 -- 172.21.15.38:0/4255245699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef009ca50 con 0x7f2ef009a0c0 2024-04-25T08:32:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.720+0000 7f2ee67fc640 1 -- 172.21.15.38:0/4255245699 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0037040 con 0x7f2ef009a0c0 2024-04-25T08:32:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.720+0000 7f2ef8037640 1 -- 172.21.15.38:0/4255245699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef00ca290 con 0x7f2ef009a0c0 2024-04-25T08:32:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.721+0000 7f2ef8037640 1 -- 172.21.15.38:0/4255245699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef004ef90 con 0x7f2ef009a0c0 2024-04-25T08:32:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.722+0000 7f2ee67fc640 1 -- 172.21.15.38:0/4255245699 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ee0044070 con 0x7f2ef009a0c0 2024-04-25T08:32:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.722+0000 7f2ee67fc640 1 --2- 172.21.15.38:0/4255245699 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ed406b4d0 0x7f2ed406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.722+0000 7f2ee67fc640 1 -- 172.21.15.38:0/4255245699 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f2ee00ab140 con 0x7f2ef009a0c0 2024-04-25T08:32:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.722+0000 7f2ef5dd3640 1 --2- 172.21.15.38:0/4255245699 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ed406b4d0 0x7f2ed406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.723+0000 7f2ef5dd3640 1 --2- 172.21.15.38:0/4255245699 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ed406b4d0 0x7f2ed406d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00a770 tx=0x7f2eec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.726+0000 7f2ee67fc640 1 -- 172.21.15.38:0/4255245699 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ee0079d10 con 0x7f2ef009a0c0 2024-04-25T08:32:22.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.918+0000 7f2ef8037640 1 -- 172.21.15.38:0/4255245699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2ef00b3e30 con 0x7f2ef009a0c0 2024-04-25T08:32:22.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.919+0000 7f2ee67fc640 1 -- 172.21.15.38:0/4255245699 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2ee0079610 con 0x7f2ef009a0c0 2024-04-25T08:32:22.915 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:32:22.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.921+0000 7f2ef8037640 1 -- 172.21.15.38:0/4255245699 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ed406b4d0 msgr2=0x7f2ed406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.921+0000 7f2ef8037640 1 --2- 172.21.15.38:0/4255245699 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ed406b4d0 0x7f2ed406d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00a770 tx=0x7f2eec008040 comp rx=0 tx=0).stop 2024-04-25T08:32:22.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.921+0000 7f2ef8037640 1 -- 172.21.15.38:0/4255245699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ef009a0c0 msgr2=0x7f2ef009be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:22.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.921+0000 7f2ef8037640 1 --2- 172.21.15.38:0/4255245699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ef009a0c0 0x7f2ef009be90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0000c00 tx=0x7f2ee00046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.921+0000 7f2ef8037640 1 -- 172.21.15.38:0/4255245699 shutdown_connections 2024-04-25T08:32:22.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.921+0000 7f2ef8037640 1 --2- 172.21.15.38:0/4255245699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ef015eb20 0x7f2ef01f6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.921+0000 7f2ef8037640 1 --2- 172.21.15.38:0/4255245699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ef009a0c0 0x7f2ef009be90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.921+0000 7f2ef8037640 1 --2- 172.21.15.38:0/4255245699 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ed406b4d0 0x7f2ed406d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.921+0000 7f2ef8037640 1 --2- 172.21.15.38:0/4255245699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ef0099ce0 0x7f2ef015a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:22.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.921+0000 7f2ef8037640 1 -- 172.21.15.38:0/4255245699 >> 172.21.15.38:0/4255245699 conn(0x7f2ef00b30f0 msgr2=0x7f2ef006f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:22.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.921+0000 7f2ef8037640 1 -- 172.21.15.38:0/4255245699 shutdown_connections 2024-04-25T08:32:22.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:22.921+0000 7f2ef8037640 1 -- 172.21.15.38:0/4255245699 wait complete. 2024-04-25T08:32:22.928 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:32:22.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:22.928 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:22.953 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:22.953 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:32:22.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:22.953 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:32:23.008 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:32:23.008 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:23.027 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:32:23.027 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:23.084 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.188+0000 7fa989bb0640 1 Processor -- start 2024-04-25T08:32:23.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.188+0000 7fa989bb0640 1 -- start start 2024-04-25T08:32:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa989bb0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa98415ddf0 0x7fa984161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa989bb0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9840b93f0 0x7fa9840d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa989bb0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9840b98a0 con 0x7fa9840b5710 2024-04-25T08:32:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa989bb0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa984161e00 con 0x7fa98415ddf0 2024-04-25T08:32:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa989bb0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9841620e0 con 0x7fa9840b93f0 2024-04-25T08:32:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa982ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa98415ddf0 0x7fa984161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa9837fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa9840b5710 0x7fa9840b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34950/0 (socket says 172.21.15.38:34950) 2024-04-25T08:32:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa9837fe640 1 -- 172.21.15.38:0/2375697404 learned_addr learned my addr 172.21.15.38:0/2375697404 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:23.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa983fff640 1 --2- 172.21.15.38:0/2375697404 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9840b93f0 0x7fa9840d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa982ffd640 1 -- 172.21.15.38:0/2375697404 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9840b93f0 msgr2=0x7fa9840d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:23.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa982ffd640 1 --2- 172.21.15.38:0/2375697404 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9840b93f0 0x7fa9840d2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa982ffd640 1 -- 172.21.15.38:0/2375697404 >> v1:172.21.15.38:6789/0 conn(0x7fa9840b5710 legacy=0x7fa9840b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa982ffd640 1 -- 172.21.15.38:0/2375697404 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9841623c0 con 0x7fa98415ddf0 2024-04-25T08:32:23.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.189+0000 7fa9827fc640 1 -- 172.21.15.38:0/2375697404 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783354622 0 0) 0x7fa9840b98a0 con 0x7fa9840b5710 2024-04-25T08:32:23.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.190+0000 7fa982ffd640 1 --2- 172.21.15.38:0/2375697404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98415ddf0 0x7fa984161700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa9780097a0 tx=0x7fa97802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a075f7dac00254f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:23.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.190+0000 7fa9827fc640 1 -- 172.21.15.38:0/2375697404 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa978037070 con 0x7fa98415ddf0 2024-04-25T08:32:23.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.190+0000 7fa9827fc640 1 -- 172.21.15.38:0/2375697404 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa978032440 con 0x7fa98415ddf0 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.190+0000 7fa9827fc640 1 -- 172.21.15.38:0/2375697404 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa978032810 con 0x7fa98415ddf0 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.191+0000 7fa989bb0640 1 -- 172.21.15.38:0/2375697404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98415ddf0 msgr2=0x7fa984161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.191+0000 7fa989bb0640 1 --2- 172.21.15.38:0/2375697404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98415ddf0 0x7fa984161700 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa9780097a0 tx=0x7fa97802a180 comp rx=0 tx=0).stop 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.191+0000 7fa989bb0640 1 -- 172.21.15.38:0/2375697404 shutdown_connections 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.191+0000 7fa989bb0640 1 --2- 172.21.15.38:0/2375697404 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9840b93f0 0x7fa9840d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.191+0000 7fa989bb0640 1 --2- 172.21.15.38:0/2375697404 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98415ddf0 0x7fa984161700 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.191+0000 7fa989bb0640 1 -- 172.21.15.38:0/2375697404 >> 172.21.15.38:0/2375697404 conn(0x7fa9840d07c0 msgr2=0x7fa9840d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.191+0000 7fa989bb0640 1 -- 172.21.15.38:0/2375697404 shutdown_connections 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.191+0000 7fa989bb0640 1 -- 172.21.15.38:0/2375697404 wait complete. 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.192+0000 7fa989bb0640 1 Processor -- start 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.192+0000 7fa989bb0640 1 -- start start 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa989bb0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9840b5710 0x7fa9840c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa989bb0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9840b93f0 0x7fa984160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa989bb0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9840c9ad0 0x7fa98415c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa989bb0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840d2c10 con 0x7fa9840c9ad0 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa989bb0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840d2a90 con 0x7fa9840b93f0 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa989bb0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840d2d90 con 0x7fa9840b5710 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa983fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9840c9ad0 0x7fa98415c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa983fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9840c9ad0 0x7fa98415c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48706/0 (socket says 172.21.15.38:48706) 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa983fff640 1 -- 172.21.15.38:0/687323951 learned_addr learned my addr 172.21.15.38:0/687323951 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa9837fe640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9840b5710 0x7fa9840c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa983fff640 1 -- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9840b5710 msgr2=0x7fa9840c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa982ffd640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9840b93f0 0x7fa984160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa983fff640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9840b5710 0x7fa9840c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa983fff640 1 -- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9840b93f0 msgr2=0x7fa984160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa983fff640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9840b93f0 0x7fa984160320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa983fff640 1 -- 172.21.15.38:0/687323951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9840c86b0 con 0x7fa9840c9ad0 2024-04-25T08:32:23.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa9837fe640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9840b5710 0x7fa9840c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:23.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa982ffd640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9840b93f0 0x7fa984160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:23.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.193+0000 7fa983fff640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9840c9ad0 0x7fa98415c540 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00cc30 tx=0x7fa97c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:23.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.194+0000 7fa96bfff640 1 -- 172.21.15.38:0/687323951 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c00ce40 con 0x7fa9840c9ad0 2024-04-25T08:32:23.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.194+0000 7fa96bfff640 1 -- 172.21.15.38:0/687323951 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c003990 con 0x7fa9840c9ad0 2024-04-25T08:32:23.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.194+0000 7fa96bfff640 1 -- 172.21.15.38:0/687323951 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c003d40 con 0x7fa9840c9ad0 2024-04-25T08:32:23.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.194+0000 7fa989bb0640 1 -- 172.21.15.38:0/687323951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9840c8990 con 0x7fa9840c9ad0 2024-04-25T08:32:23.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.194+0000 7fa989bb0640 1 -- 172.21.15.38:0/687323951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9840c83a0 con 0x7fa9840c9ad0 2024-04-25T08:32:23.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.195+0000 7fa989bb0640 1 -- 172.21.15.38:0/687323951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa984068e80 con 0x7fa9840c9ad0 2024-04-25T08:32:23.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.195+0000 7fa96bfff640 1 -- 172.21.15.38:0/687323951 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa97c003ee0 con 0x7fa9840c9ad0 2024-04-25T08:32:23.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.195+0000 7fa96bfff640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa97006b4d0 0x7fa97006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.196+0000 7fa96bfff640 1 -- 172.21.15.38:0/687323951 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fa97c08ab40 con 0x7fa9840c9ad0 2024-04-25T08:32:23.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.196+0000 7fa9837fe640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa97006b4d0 0x7fa97006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.196+0000 7fa9837fe640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa97006b4d0 0x7fa97006d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa974000f60 tx=0x7fa974009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:23.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.200+0000 7fa96bfff640 1 -- 172.21.15.38:0/687323951 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa97c014030 con 0x7fa9840c9ad0 2024-04-25T08:32:23.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.384+0000 7fa989bb0640 1 -- 172.21.15.38:0/687323951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9840b8190 con 0x7fa9840c9ad0 2024-04-25T08:32:23.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.385+0000 7fa96bfff640 1 -- 172.21.15.38:0/687323951 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18969 (secure 0 0 0) 0x7fa97c059560 con 0x7fa9840c9ad0 2024-04-25T08:32:23.382 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:23.382 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:16.572337+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.a","mappings":[{"from":4,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.387+0000 7fa989bb0640 1 -- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa97006b4d0 msgr2=0x7fa97006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.387+0000 7fa989bb0640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa97006b4d0 0x7fa97006d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa974000f60 tx=0x7fa974009040 comp rx=0 tx=0).stop 2024-04-25T08:32:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.387+0000 7fa989bb0640 1 -- 172.21.15.38:0/687323951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9840c9ad0 msgr2=0x7fa98415c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.387+0000 7fa989bb0640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9840c9ad0 0x7fa98415c540 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00cc30 tx=0x7fa97c00a5c0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.387+0000 7fa989bb0640 1 -- 172.21.15.38:0/687323951 shutdown_connections 2024-04-25T08:32:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.387+0000 7fa989bb0640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9840c9ad0 0x7fa98415c540 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.387+0000 7fa989bb0640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9840b93f0 0x7fa984160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.387+0000 7fa989bb0640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa97006b4d0 0x7fa97006d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.387+0000 7fa989bb0640 1 --2- 172.21.15.38:0/687323951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9840b5710 0x7fa9840c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.387+0000 7fa989bb0640 1 -- 172.21.15.38:0/687323951 >> 172.21.15.38:0/687323951 conn(0x7fa9840d07c0 msgr2=0x7fa9840b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.387+0000 7fa989bb0640 1 -- 172.21.15.38:0/687323951 shutdown_connections 2024-04-25T08:32:23.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.387+0000 7fa989bb0640 1 -- 172.21.15.38:0/687323951 wait complete. 2024-04-25T08:32:23.395 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:23.395 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:23.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.502+0000 7f958885d640 1 Processor -- start 2024-04-25T08:32:23.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.503+0000 7f958885d640 1 -- start start 2024-04-25T08:32:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.503+0000 7f958885d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f958415df30 0x7f958409a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.503+0000 7f958885d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9584162390 0x7f9584099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.503+0000 7f958885d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9584099fb0 0x7f958406bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.503+0000 7f958885d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f958415e310 con 0x7f958415df30 2024-04-25T08:32:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.503+0000 7f958885d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f958409aeb0 con 0x7f9584162390 2024-04-25T08:32:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.503+0000 7f958885d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958415ee70 con 0x7f9584099fb0 2024-04-25T08:32:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.503+0000 7f958259c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f958415df30 0x7f958409a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.503+0000 7f958259c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f958415df30 0x7f958409a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48720/0 (socket says 172.21.15.38:48720) 2024-04-25T08:32:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.504+0000 7f958259c640 1 -- 172.21.15.38:0/416535397 learned_addr learned my addr 172.21.15.38:0/416535397 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:23.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.504+0000 7f9582d9d640 1 --2- 172.21.15.38:0/416535397 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9584099fb0 0x7f958406bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.504+0000 7f958259c640 1 -- 172.21.15.38:0/416535397 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9584099fb0 msgr2=0x7f958406bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.504+0000 7f958259c640 1 --2- 172.21.15.38:0/416535397 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9584099fb0 0x7f958406bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.504+0000 7f957bfff640 1 --2- 172.21.15.38:0/416535397 >> v2:172.21.15.107:3300/0 conn(0x7f9584162390 0x7f9584099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.504+0000 7f958259c640 1 -- 172.21.15.38:0/416535397 >> v2:172.21.15.107:3300/0 conn(0x7f9584162390 msgr2=0x7f9584099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.504+0000 7f958259c640 1 --2- 172.21.15.38:0/416535397 >> v2:172.21.15.107:3300/0 conn(0x7f9584162390 0x7f9584099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.504+0000 7f958259c640 1 -- 172.21.15.38:0/416535397 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9584162820 con 0x7f958415df30 2024-04-25T08:32:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.504+0000 7f958259c640 1 --2- 172.21.15.38:0/416535397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f958415df30 0x7f958409a7b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9574009870 tx=0x7f957402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25626f142d7ec7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.505+0000 7f9581d9b640 1 -- 172.21.15.38:0/416535397 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9574037070 con 0x7f958415df30 2024-04-25T08:32:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.505+0000 7f9581d9b640 1 -- 172.21.15.38:0/416535397 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9574032440 con 0x7f958415df30 2024-04-25T08:32:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.505+0000 7f9581d9b640 1 -- 172.21.15.38:0/416535397 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9574032810 con 0x7f958415df30 2024-04-25T08:32:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.505+0000 7f958885d640 1 -- 172.21.15.38:0/416535397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f958415df30 msgr2=0x7f958409a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.505+0000 7f958885d640 1 --2- 172.21.15.38:0/416535397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f958415df30 0x7f958409a7b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9574009870 tx=0x7f957402a210 comp rx=0 tx=0).stop 2024-04-25T08:32:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.505+0000 7f958885d640 1 -- 172.21.15.38:0/416535397 shutdown_connections 2024-04-25T08:32:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.505+0000 7f958885d640 1 --2- 172.21.15.38:0/416535397 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9584099fb0 0x7f958406bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.505+0000 7f958885d640 1 --2- 172.21.15.38:0/416535397 >> v2:172.21.15.107:3300/0 conn(0x7f9584162390 0x7f9584099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.505+0000 7f958885d640 1 --2- 172.21.15.38:0/416535397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f958415df30 0x7f958409a7b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.505+0000 7f958885d640 1 -- 172.21.15.38:0/416535397 >> 172.21.15.38:0/416535397 conn(0x7f95840b3110 msgr2=0x7f95840b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:23.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.505+0000 7f958885d640 1 -- 172.21.15.38:0/416535397 shutdown_connections 2024-04-25T08:32:23.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.505+0000 7f958885d640 1 -- 172.21.15.38:0/416535397 wait complete. 2024-04-25T08:32:23.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.507+0000 7f958885d640 1 Processor -- start 2024-04-25T08:32:23.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.507+0000 7f958885d640 1 -- start start 2024-04-25T08:32:23.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.507+0000 7f958885d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9584099fb0 0x7f9584151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.507+0000 7f958885d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f958415df30 0x7f95840c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.507+0000 7f958885d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9584162390 0x7f958415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.507+0000 7f958885d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958415e820 con 0x7f958415df30 2024-04-25T08:32:23.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.507+0000 7f958885d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958415f300 con 0x7f9584162390 2024-04-25T08:32:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.507+0000 7f958885d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958415eb00 con 0x7f9584099fb0 2024-04-25T08:32:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.507+0000 7f957bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f958415df30 0x7f95840c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f957bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f958415df30 0x7f95840c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48728/0 (socket says 172.21.15.38:48728) 2024-04-25T08:32:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f957bfff640 1 -- 172.21.15.38:0/2887528931 learned_addr learned my addr 172.21.15.38:0/2887528931 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f958259c640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9584099fb0 0x7f9584151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f9582d9d640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9584162390 0x7f958415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f957bfff640 1 -- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9584099fb0 msgr2=0x7f9584151bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f957bfff640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9584099fb0 0x7f9584151bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f957bfff640 1 -- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9584162390 msgr2=0x7f958415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f957bfff640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9584162390 0x7f958415a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f957bfff640 1 -- 172.21.15.38:0/2887528931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95840ca480 con 0x7f958415df30 2024-04-25T08:32:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f9582d9d640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9584162390 0x7f958415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f957bfff640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f958415df30 0x7f95840c9af0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f956c00c7f0 tx=0x7f956c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f957affd640 1 -- 172.21.15.38:0/2887528931 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c015070 con 0x7f958415df30 2024-04-25T08:32:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f957affd640 1 -- 172.21.15.38:0/2887528931 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c004240 con 0x7f958415df30 2024-04-25T08:32:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f958885d640 1 -- 172.21.15.38:0/2887528931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95840ca760 con 0x7f958415df30 2024-04-25T08:32:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f957affd640 1 -- 172.21.15.38:0/2887528931 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c007620 con 0x7f958415df30 2024-04-25T08:32:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.508+0000 7f958885d640 1 -- 172.21.15.38:0/2887528931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95840cad20 con 0x7f958415df30 2024-04-25T08:32:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.509+0000 7f958885d640 1 -- 172.21.15.38:0/2887528931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9584068e80 con 0x7f958415df30 2024-04-25T08:32:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.510+0000 7f957affd640 1 -- 172.21.15.38:0/2887528931 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f956c0079d0 con 0x7f958415df30 2024-04-25T08:32:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.510+0000 7f957affd640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f956406b500 0x7f956406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.510+0000 7f957affd640 1 -- 172.21.15.38:0/2887528931 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f956c08a5e0 con 0x7f958415df30 2024-04-25T08:32:23.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.510+0000 7f958259c640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f956406b500 0x7f956406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.511+0000 7f958259c640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f956406b500 0x7f956406d9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9574034000 tx=0x7f957402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:23.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.514+0000 7f957affd640 1 -- 172.21.15.38:0/2887528931 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f956c058a40 con 0x7f958415df30 2024-04-25T08:32:23.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.695+0000 7f958885d640 1 -- 172.21.15.38:0/2887528931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95840b4020 con 0x7f958415df30 2024-04-25T08:32:23.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.695+0000 7f957affd640 1 -- 172.21.15.38:0/2887528931 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f956c05cae0 con 0x7f958415df30 2024-04-25T08:32:23.691 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:23.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.697+0000 7f958885d640 1 -- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f956406b500 msgr2=0x7f956406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.697+0000 7f958885d640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f956406b500 0x7f956406d9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9574034000 tx=0x7f957402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:23.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.697+0000 7f958885d640 1 -- 172.21.15.38:0/2887528931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f958415df30 msgr2=0x7f95840c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.697+0000 7f958885d640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f958415df30 0x7f95840c9af0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f956c00c7f0 tx=0x7f956c00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.697+0000 7f958885d640 1 -- 172.21.15.38:0/2887528931 shutdown_connections 2024-04-25T08:32:23.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.697+0000 7f958885d640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9584162390 0x7f958415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.697+0000 7f958885d640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f958415df30 0x7f95840c9af0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.697+0000 7f958885d640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f956406b500 0x7f956406d9c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.697+0000 7f958885d640 1 --2- 172.21.15.38:0/2887528931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9584099fb0 0x7f9584151bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.697+0000 7f958885d640 1 -- 172.21.15.38:0/2887528931 >> 172.21.15.38:0/2887528931 conn(0x7f95840b3110 msgr2=0x7f958406fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:23.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.697+0000 7f958885d640 1 -- 172.21.15.38:0/2887528931 shutdown_connections 2024-04-25T08:32:23.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.697+0000 7f958885d640 1 -- 172.21.15.38:0/2887528931 wait complete. 2024-04-25T08:32:23.704 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:23.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.812+0000 7fc8886e1640 1 Processor -- start 2024-04-25T08:32:23.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.812+0000 7fc8886e1640 1 -- start start 2024-04-25T08:32:23.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc8886e1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc8800d0320 0x7fc8800d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc8886e1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8800d0700 0x7fc8800d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc8886e1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8800d2330 con 0x7fc8800d1e90 2024-04-25T08:32:23.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc8886e1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8800d2610 con 0x7fc8800d0320 2024-04-25T08:32:23.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc8886e1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800d28f0 con 0x7fc8800d0700 2024-04-25T08:32:23.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc88647d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc8800d0320 0x7fc8800d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc886c7e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc8800d1e90 0x7fc8800b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34964/0 (socket says 172.21.15.38:34964) 2024-04-25T08:32:23.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc886c7e640 1 -- 172.21.15.38:0/3682864515 learned_addr learned my addr 172.21.15.38:0/3682864515 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:23.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc885c7c640 1 --2- 172.21.15.38:0/3682864515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8800d0700 0x7fc8800d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc88647d640 1 -- 172.21.15.38:0/3682864515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8800d0700 msgr2=0x7fc8800d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:23.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc88647d640 1 --2- 172.21.15.38:0/3682864515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8800d0700 0x7fc8800d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc88647d640 1 -- 172.21.15.38:0/3682864515 >> v1:172.21.15.38:6789/0 conn(0x7fc8800d1e90 legacy=0x7fc8800b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc88547b640 1 -- 172.21.15.38:0/3682864515 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333202219 0 0) 0x7fc8800d2330 con 0x7fc8800d1e90 2024-04-25T08:32:23.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.813+0000 7fc88647d640 1 -- 172.21.15.38:0/3682864515 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8800d2bd0 con 0x7fc8800d0320 2024-04-25T08:32:23.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.814+0000 7fc88647d640 1 --2- 172.21.15.38:0/3682864515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8800d0320 0x7fc8800d33d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0097a0 tx=0x7fc87c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dae5bcb6d5fda2ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:23.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.814+0000 7fc88547b640 1 -- 172.21.15.38:0/3682864515 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c037070 con 0x7fc8800d0320 2024-04-25T08:32:23.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.814+0000 7fc88547b640 1 -- 172.21.15.38:0/3682864515 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c032440 con 0x7fc8800d0320 2024-04-25T08:32:23.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.814+0000 7fc88547b640 1 -- 172.21.15.38:0/3682864515 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c032810 con 0x7fc8800d0320 2024-04-25T08:32:23.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.815+0000 7fc8886e1640 1 -- 172.21.15.38:0/3682864515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8800d0320 msgr2=0x7fc8800d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.815+0000 7fc8886e1640 1 --2- 172.21.15.38:0/3682864515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8800d0320 0x7fc8800d33d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0097a0 tx=0x7fc87c02a210 comp rx=0 tx=0).stop 2024-04-25T08:32:23.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.815+0000 7fc8886e1640 1 -- 172.21.15.38:0/3682864515 shutdown_connections 2024-04-25T08:32:23.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.815+0000 7fc8886e1640 1 --2- 172.21.15.38:0/3682864515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8800d0700 0x7fc8800d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.815+0000 7fc8886e1640 1 --2- 172.21.15.38:0/3682864515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8800d0320 0x7fc8800d33d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.815+0000 7fc8886e1640 1 -- 172.21.15.38:0/3682864515 >> 172.21.15.38:0/3682864515 conn(0x7fc8800668f0 msgr2=0x7fc880066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:23.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.815+0000 7fc8886e1640 1 -- 172.21.15.38:0/3682864515 shutdown_connections 2024-04-25T08:32:23.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.815+0000 7fc8886e1640 1 -- 172.21.15.38:0/3682864515 wait complete. 2024-04-25T08:32:23.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.816+0000 7fc8886e1640 1 Processor -- start 2024-04-25T08:32:23.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.816+0000 7fc8886e1640 1 -- start start 2024-04-25T08:32:23.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc8886e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8800d0320 0x7fc88014bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc8886e1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8800d0700 0x7fc8801f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc8886e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8800d1e90 0x7fc8801f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc8886e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800d0b40 con 0x7fc8800d1e90 2024-04-25T08:32:23.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc8886e1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc880161210 con 0x7fc8800d0320 2024-04-25T08:32:23.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc8886e1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800b4ba0 con 0x7fc8800d0700 2024-04-25T08:32:23.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc886c7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8800d1e90 0x7fc8801f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc886c7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8800d1e90 0x7fc8801f7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48742/0 (socket says 172.21.15.38:48742) 2024-04-25T08:32:23.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc886c7e640 1 -- 172.21.15.38:0/3033078435 learned_addr learned my addr 172.21.15.38:0/3033078435 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:23.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc885c7c640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8800d0700 0x7fc8801f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc88647d640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8800d0320 0x7fc88014bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc886c7e640 1 -- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8800d0700 msgr2=0x7fc8801f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc886c7e640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8800d0700 0x7fc8801f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc886c7e640 1 -- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8800d0320 msgr2=0x7fc88014bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:23.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc886c7e640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8800d0320 0x7fc88014bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:23.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc886c7e640 1 -- 172.21.15.38:0/3033078435 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8800b4e80 con 0x7fc8800d1e90 2024-04-25T08:32:23.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.817+0000 7fc886c7e640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8800d1e90 0x7fc8801f7180 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc87800c770 tx=0x7fc87800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:23.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.818+0000 7fc876ffd640 1 -- 172.21.15.38:0/3033078435 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878018070 con 0x7fc8800d1e90 2024-04-25T08:32:23.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.818+0000 7fc876ffd640 1 -- 172.21.15.38:0/3033078435 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87800a640 con 0x7fc8800d1e90 2024-04-25T08:32:23.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.818+0000 7fc8886e1640 1 -- 172.21.15.38:0/3033078435 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8800b5160 con 0x7fc8800d1e90 2024-04-25T08:32:23.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.818+0000 7fc876ffd640 1 -- 172.21.15.38:0/3033078435 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87800aa30 con 0x7fc8800d1e90 2024-04-25T08:32:23.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.818+0000 7fc8886e1640 1 -- 172.21.15.38:0/3033078435 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc880157360 con 0x7fc8800d1e90 2024-04-25T08:32:23.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.819+0000 7fc8886e1640 1 -- 172.21.15.38:0/3033078435 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc880068e80 con 0x7fc8800d1e90 2024-04-25T08:32:23.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.819+0000 7fc876ffd640 1 -- 172.21.15.38:0/3033078435 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc87801b070 con 0x7fc8800d1e90 2024-04-25T08:32:23.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.819+0000 7fc876ffd640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc86406b4d0 0x7fc86406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:23.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.819+0000 7fc876ffd640 1 -- 172.21.15.38:0/3033078435 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fc878089be0 con 0x7fc8800d1e90 2024-04-25T08:32:23.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.820+0000 7fc88647d640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc86406b4d0 0x7fc86406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:23.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.820+0000 7fc88647d640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc86406b4d0 0x7fc86406d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc87c034000 tx=0x7fc87c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:23.824+0000 7fc876ffd640 1 -- 172.21.15.38:0/3033078435 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc878057fb0 con 0x7fc8800d1e90 2024-04-25T08:32:24.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.005+0000 7fc8886e1640 1 -- 172.21.15.38:0/3033078435 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8800b3050 con 0x7fc8800d1e90 2024-04-25T08:32:24.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.006+0000 7fc876ffd640 1 -- 172.21.15.38:0/3033078435 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc87805c050 con 0x7fc8800d1e90 2024-04-25T08:32:24.002 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:24.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.007+0000 7fc8886e1640 1 -- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc86406b4d0 msgr2=0x7fc86406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.007+0000 7fc8886e1640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc86406b4d0 0x7fc86406d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc87c034000 tx=0x7fc87c02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:24.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.007+0000 7fc8886e1640 1 -- 172.21.15.38:0/3033078435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8800d1e90 msgr2=0x7fc8801f7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.007+0000 7fc8886e1640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8800d1e90 0x7fc8801f7180 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc87800c770 tx=0x7fc87800cc30 comp rx=0 tx=0).stop 2024-04-25T08:32:24.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.007+0000 7fc8886e1640 1 -- 172.21.15.38:0/3033078435 shutdown_connections 2024-04-25T08:32:24.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.007+0000 7fc8886e1640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8800d1e90 0x7fc8801f7180 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.007+0000 7fc8886e1640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc86406b4d0 0x7fc86406d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.007+0000 7fc8886e1640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8800d0700 0x7fc8801f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.007+0000 7fc8886e1640 1 --2- 172.21.15.38:0/3033078435 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8800d0320 0x7fc88014bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.007+0000 7fc8886e1640 1 -- 172.21.15.38:0/3033078435 >> 172.21.15.38:0/3033078435 conn(0x7fc8800668f0 msgr2=0x7fc880073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.008+0000 7fc8886e1640 1 -- 172.21.15.38:0/3033078435 shutdown_connections 2024-04-25T08:32:24.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.008+0000 7fc8886e1640 1 -- 172.21.15.38:0/3033078435 wait complete. 2024-04-25T08:32:24.015 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:24.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.125+0000 7f6597b1d640 1 Processor -- start 2024-04-25T08:32:24.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.126+0000 7f6597b1d640 1 -- start start 2024-04-25T08:32:24.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f6597b1d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f659006c700 0x7f6590162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f6597b1d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f659006cae0 0x7f659015e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f6597b1d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f659015e580 0x7f65900cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f6597b1d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6590161c80 con 0x7f659006cae0 2024-04-25T08:32:24.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f6597b1d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659015ec20 con 0x7f659006c700 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f6597b1d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659015ef00 con 0x7f659015e580 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f65950b8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f659006cae0 0x7f659015e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f65950b8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f659006cae0 0x7f659015e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48748/0 (socket says 172.21.15.38:48748) 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f65950b8640 1 -- 172.21.15.38:0/4053051832 learned_addr learned my addr 172.21.15.38:0/4053051832 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f65958b9640 1 --2- 172.21.15.38:0/4053051832 >> v2:172.21.15.107:3300/0 conn(0x7f659006c700 0x7f6590162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f65960ba640 1 --2- 172.21.15.38:0/4053051832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f659015e580 0x7f65900cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f65950b8640 1 -- 172.21.15.38:0/4053051832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f659015e580 msgr2=0x7f65900cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f65950b8640 1 --2- 172.21.15.38:0/4053051832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f659015e580 0x7f65900cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f65950b8640 1 -- 172.21.15.38:0/4053051832 >> v2:172.21.15.107:3300/0 conn(0x7f659006c700 msgr2=0x7f6590162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f65950b8640 1 --2- 172.21.15.38:0/4053051832 >> v2:172.21.15.107:3300/0 conn(0x7f659006c700 0x7f6590162110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f65950b8640 1 -- 172.21.15.38:0/4053051832 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6590161630 con 0x7f659006cae0 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.127+0000 7f65960ba640 1 --2- 172.21.15.38:0/4053051832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f659015e580 0x7f65900cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.128+0000 7f65950b8640 1 --2- 172.21.15.38:0/4053051832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659006cae0 0x7f659015e040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f65900d25f0 tx=0x7f658402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9550d44386300a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.128+0000 7f65948b7640 1 -- 172.21.15.38:0/4053051832 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584037070 con 0x7f659006cae0 2024-04-25T08:32:24.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.128+0000 7f65948b7640 1 -- 172.21.15.38:0/4053051832 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6584032440 con 0x7f659006cae0 2024-04-25T08:32:24.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.128+0000 7f65948b7640 1 -- 172.21.15.38:0/4053051832 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584032810 con 0x7f659006cae0 2024-04-25T08:32:24.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.129+0000 7f6597b1d640 1 -- 172.21.15.38:0/4053051832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659006cae0 msgr2=0x7f659015e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.129+0000 7f6597b1d640 1 --2- 172.21.15.38:0/4053051832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659006cae0 0x7f659015e040 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f65900d25f0 tx=0x7f658402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.129+0000 7f6597b1d640 1 -- 172.21.15.38:0/4053051832 shutdown_connections 2024-04-25T08:32:24.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.129+0000 7f6597b1d640 1 --2- 172.21.15.38:0/4053051832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f659015e580 0x7f65900cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.129+0000 7f6597b1d640 1 --2- 172.21.15.38:0/4053051832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659006cae0 0x7f659015e040 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.129+0000 7f6597b1d640 1 --2- 172.21.15.38:0/4053051832 >> v2:172.21.15.107:3300/0 conn(0x7f659006c700 0x7f6590162110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.129+0000 7f6597b1d640 1 -- 172.21.15.38:0/4053051832 >> 172.21.15.38:0/4053051832 conn(0x7f659008b000 msgr2=0x7f659008b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.129+0000 7f6597b1d640 1 -- 172.21.15.38:0/4053051832 shutdown_connections 2024-04-25T08:32:24.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.129+0000 7f6597b1d640 1 -- 172.21.15.38:0/4053051832 wait complete. 2024-04-25T08:32:24.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.131+0000 7f6597b1d640 1 Processor -- start 2024-04-25T08:32:24.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.131+0000 7f6597b1d640 1 -- start start 2024-04-25T08:32:24.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f6597b1d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f659006c700 0x7f6590158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f6597b1d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659006cae0 0x7f65900c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f6597b1d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f659015e580 0x7f65900afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f6597b1d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659006cf90 con 0x7f659006cae0 2024-04-25T08:32:24.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f6597b1d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659006c190 con 0x7f659006c700 2024-04-25T08:32:24.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f6597b1d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659015b920 con 0x7f659015e580 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65950b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659006cae0 0x7f65900c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65950b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659006cae0 0x7f65900c63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48764/0 (socket says 172.21.15.38:48764) 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65950b8640 1 -- 172.21.15.38:0/1918992984 learned_addr learned my addr 172.21.15.38:0/1918992984 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65960ba640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f659015e580 0x7f65900afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65958b9640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f659006c700 0x7f6590158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65950b8640 1 -- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f659015e580 msgr2=0x7f65900afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65950b8640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f659015e580 0x7f65900afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65950b8640 1 -- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f659006c700 msgr2=0x7f6590158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65950b8640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f659006c700 0x7f6590158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65950b8640 1 -- 172.21.15.38:0/1918992984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65900c6cf0 con 0x7f659006cae0 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65960ba640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f659015e580 0x7f65900afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65958b9640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f659006c700 0x7f6590158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65950b8640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659006cae0 0x7f65900c63a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f658402a220 tx=0x7f65840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65827fc640 1 -- 172.21.15.38:0/1918992984 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584037040 con 0x7f659006cae0 2024-04-25T08:32:24.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65827fc640 1 -- 172.21.15.38:0/1918992984 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6584042070 con 0x7f659006cae0 2024-04-25T08:32:24.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f65827fc640 1 -- 172.21.15.38:0/1918992984 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584030880 con 0x7f659006cae0 2024-04-25T08:32:24.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.133+0000 7f6597b1d640 1 -- 172.21.15.38:0/1918992984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65900c6fd0 con 0x7f659006cae0 2024-04-25T08:32:24.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.134+0000 7f6597b1d640 1 -- 172.21.15.38:0/1918992984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65900c7590 con 0x7f659006cae0 2024-04-25T08:32:24.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.135+0000 7f6597b1d640 1 -- 172.21.15.38:0/1918992984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6590068e80 con 0x7f659006cae0 2024-04-25T08:32:24.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.135+0000 7f65827fc640 1 -- 172.21.15.38:0/1918992984 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6584030a20 con 0x7f659006cae0 2024-04-25T08:32:24.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.135+0000 7f65827fc640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f657406b4d0 0x7f657406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.136+0000 7f65827fc640 1 -- 172.21.15.38:0/1918992984 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f65840aaf40 con 0x7f659006cae0 2024-04-25T08:32:24.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.136+0000 7f65958b9640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f657406b4d0 0x7f657406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.136+0000 7f65958b9640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f657406b4d0 0x7f657406d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f65900c7b70 tx=0x7f658c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.140+0000 7f65827fc640 1 -- 172.21.15.38:0/1918992984 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65840799e0 con 0x7f659006cae0 2024-04-25T08:32:24.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.321+0000 7f6597b1d640 1 -- 172.21.15.38:0/1918992984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65900b1220 con 0x7f659006cae0 2024-04-25T08:32:24.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.322+0000 7f65827fc640 1 -- 172.21.15.38:0/1918992984 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65840792e0 con 0x7f659006cae0 2024-04-25T08:32:24.318 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:24.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.324+0000 7f6597b1d640 1 -- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f657406b4d0 msgr2=0x7f657406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.324+0000 7f6597b1d640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f657406b4d0 0x7f657406d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f65900c7b70 tx=0x7f658c008040 comp rx=0 tx=0).stop 2024-04-25T08:32:24.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.324+0000 7f6597b1d640 1 -- 172.21.15.38:0/1918992984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659006cae0 msgr2=0x7f65900c63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.324+0000 7f6597b1d640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659006cae0 0x7f65900c63a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f658402a220 tx=0x7f65840046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.324+0000 7f6597b1d640 1 -- 172.21.15.38:0/1918992984 shutdown_connections 2024-04-25T08:32:24.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.324+0000 7f6597b1d640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f657406b4d0 0x7f657406d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.324+0000 7f6597b1d640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f659015e580 0x7f65900afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.324+0000 7f6597b1d640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f659006cae0 0x7f65900c63a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.324+0000 7f6597b1d640 1 --2- 172.21.15.38:0/1918992984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f659006c700 0x7f6590158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.324+0000 7f6597b1d640 1 -- 172.21.15.38:0/1918992984 >> 172.21.15.38:0/1918992984 conn(0x7f659008b000 msgr2=0x7f659015fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.324+0000 7f6597b1d640 1 -- 172.21.15.38:0/1918992984 shutdown_connections 2024-04-25T08:32:24.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.324+0000 7f6597b1d640 1 -- 172.21.15.38:0/1918992984 wait complete. 2024-04-25T08:32:24.332 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:24.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.442+0000 7fdd7d3d5640 1 Processor -- start 2024-04-25T08:32:24.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.443+0000 7fdd7d3d5640 1 -- start start 2024-04-25T08:32:24.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.443+0000 7fdd7d3d5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdd7815de60 0x7fdd780d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.443+0000 7fdd7d3d5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdd780cf7b0 0x7fdd780b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.443+0000 7fdd7d3d5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd780ce710 con 0x7fdd780d1e00 2024-04-25T08:32:24.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.443+0000 7fdd7d3d5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd780ce4e0 con 0x7fdd780cf7b0 2024-04-25T08:32:24.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.443+0000 7fdd7d3d5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd780cf320 con 0x7fdd7815de60 2024-04-25T08:32:24.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.444+0000 7fdd777fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdd780cf7b0 0x7fdd780b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.444+0000 7fdd76ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdd780d1e00 0x7fdd780d21e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34966/0 (socket says 172.21.15.38:34966) 2024-04-25T08:32:24.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.444+0000 7fdd76ffd640 1 -- 172.21.15.38:0/2372140096 learned_addr learned my addr 172.21.15.38:0/2372140096 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:24.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.444+0000 7fdd767fc640 1 --2- 172.21.15.38:0/2372140096 >> v2:172.21.15.107:3300/0 conn(0x7fdd7815de60 0x7fdd780d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.444+0000 7fdd777fe640 1 -- 172.21.15.38:0/2372140096 >> v2:172.21.15.107:3300/0 conn(0x7fdd7815de60 msgr2=0x7fdd780d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.444+0000 7fdd777fe640 1 --2- 172.21.15.38:0/2372140096 >> v2:172.21.15.107:3300/0 conn(0x7fdd7815de60 0x7fdd780d42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.444+0000 7fdd777fe640 1 -- 172.21.15.38:0/2372140096 >> v1:172.21.15.38:6789/0 conn(0x7fdd780d1e00 legacy=0x7fdd780d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.444+0000 7fdd777fe640 1 -- 172.21.15.38:0/2372140096 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd780ce9f0 con 0x7fdd780cf7b0 2024-04-25T08:32:24.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.444+0000 7fdd75ffb640 1 -- 172.21.15.38:0/2372140096 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528256092 0 0) 0x7fdd780ce710 con 0x7fdd780d1e00 2024-04-25T08:32:24.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.444+0000 7fdd767fc640 1 --2- 172.21.15.38:0/2372140096 >> v2:172.21.15.107:3300/0 conn(0x7fdd7815de60 0x7fdd780d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:24.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.445+0000 7fdd777fe640 1 --2- 172.21.15.38:0/2372140096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780cf7b0 0x7fdd780b3180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd780cf4a0 tx=0x7fdd7002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d488f2da83df6d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.445+0000 7fdd75ffb640 1 -- 172.21.15.38:0/2372140096 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70037070 con 0x7fdd780cf7b0 2024-04-25T08:32:24.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.445+0000 7fdd75ffb640 1 -- 172.21.15.38:0/2372140096 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70032440 con 0x7fdd780cf7b0 2024-04-25T08:32:24.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.445+0000 7fdd75ffb640 1 -- 172.21.15.38:0/2372140096 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70032810 con 0x7fdd780cf7b0 2024-04-25T08:32:24.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.446+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/2372140096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780cf7b0 msgr2=0x7fdd780b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.446+0000 7fdd7d3d5640 1 --2- 172.21.15.38:0/2372140096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780cf7b0 0x7fdd780b3180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdd780cf4a0 tx=0x7fdd7002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.446+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/2372140096 shutdown_connections 2024-04-25T08:32:24.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.446+0000 7fdd7d3d5640 1 --2- 172.21.15.38:0/2372140096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd780cf7b0 0x7fdd780b3180 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.446+0000 7fdd7d3d5640 1 --2- 172.21.15.38:0/2372140096 >> v2:172.21.15.107:3300/0 conn(0x7fdd7815de60 0x7fdd780d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.446+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/2372140096 >> 172.21.15.38:0/2372140096 conn(0x7fdd7808b000 msgr2=0x7fdd780d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.446+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/2372140096 shutdown_connections 2024-04-25T08:32:24.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.446+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/2372140096 wait complete. 2024-04-25T08:32:24.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.447+0000 7fdd7d3d5640 1 Processor -- start 2024-04-25T08:32:24.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.447+0000 7fdd7d3d5640 1 -- start start 2024-04-25T08:32:24.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd7d3d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780cf7b0 0x7fdd780b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd7d3d5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd780d1e00 0x7fdd780c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd7d3d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd7815de60 0x7fdd7814ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd7d3d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780d1600 con 0x7fdd7815de60 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd7d3d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780cefb0 con 0x7fdd780cf7b0 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd7d3d5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780d1780 con 0x7fdd780d1e00 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd7815de60 0x7fdd7814ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd7815de60 0x7fdd7814ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48784/0 (socket says 172.21.15.38:48784) 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd777fe640 1 -- 172.21.15.38:0/420385438 learned_addr learned my addr 172.21.15.38:0/420385438 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd767fc640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd780d1e00 0x7fdd780c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd76ffd640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780cf7b0 0x7fdd780b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd777fe640 1 -- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd780d1e00 msgr2=0x7fdd780c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd777fe640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd780d1e00 0x7fdd780c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.448+0000 7fdd777fe640 1 -- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780cf7b0 msgr2=0x7fdd780b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.449+0000 7fdd777fe640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780cf7b0 0x7fdd780b6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.449+0000 7fdd777fe640 1 -- 172.21.15.38:0/420385438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd78158710 con 0x7fdd7815de60 2024-04-25T08:32:24.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.449+0000 7fdd76ffd640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780cf7b0 0x7fdd780b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:24.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.449+0000 7fdd777fe640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd7815de60 0x7fdd7814ec50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdd7002a200 tx=0x7fdd700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.449+0000 7fdd5f7fe640 1 -- 172.21.15.38:0/420385438 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70032e60 con 0x7fdd7815de60 2024-04-25T08:32:24.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.449+0000 7fdd5f7fe640 1 -- 172.21.15.38:0/420385438 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70042070 con 0x7fdd7815de60 2024-04-25T08:32:24.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.449+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/420385438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7815f3b0 con 0x7fdd7815de60 2024-04-25T08:32:24.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.449+0000 7fdd5f7fe640 1 -- 172.21.15.38:0/420385438 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70037040 con 0x7fdd7815de60 2024-04-25T08:32:24.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.449+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/420385438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7815f970 con 0x7fdd7815de60 2024-04-25T08:32:24.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.450+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/420385438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd78068e80 con 0x7fdd7815de60 2024-04-25T08:32:24.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.450+0000 7fdd5f7fe640 1 -- 172.21.15.38:0/420385438 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd70030b50 con 0x7fdd7815de60 2024-04-25T08:32:24.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.451+0000 7fdd5f7fe640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6406b4d0 0x7fdd6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.451+0000 7fdd5f7fe640 1 -- 172.21.15.38:0/420385438 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fdd700ab120 con 0x7fdd7815de60 2024-04-25T08:32:24.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.451+0000 7fdd76ffd640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6406b4d0 0x7fdd6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.452+0000 7fdd76ffd640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6406b4d0 0x7fdd6406d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd680012e0 tx=0x7fdd68007f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.458+0000 7fdd5f7fe640 1 -- 172.21.15.38:0/420385438 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd78158710 con 0x7fdd7815de60 2024-04-25T08:32:24.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.641+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/420385438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdd780d1be0 con 0x7fdd7815de60 2024-04-25T08:32:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.642+0000 7fdd5f7fe640 1 -- 172.21.15.38:0/420385438 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdd70079b40 con 0x7fdd7815de60 2024-04-25T08:32:24.638 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:24.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.644+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6406b4d0 msgr2=0x7fdd6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.644+0000 7fdd7d3d5640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6406b4d0 0x7fdd6406d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd680012e0 tx=0x7fdd68007f90 comp rx=0 tx=0).stop 2024-04-25T08:32:24.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.644+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/420385438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd7815de60 msgr2=0x7fdd7814ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.644+0000 7fdd7d3d5640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd7815de60 0x7fdd7814ec50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdd7002a200 tx=0x7fdd700046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.644+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/420385438 shutdown_connections 2024-04-25T08:32:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.644+0000 7fdd7d3d5640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd7815de60 0x7fdd7814ec50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.644+0000 7fdd7d3d5640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6406b4d0 0x7fdd6406d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.644+0000 7fdd7d3d5640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd780d1e00 0x7fdd780c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.644+0000 7fdd7d3d5640 1 --2- 172.21.15.38:0/420385438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd780cf7b0 0x7fdd780b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.644+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/420385438 >> 172.21.15.38:0/420385438 conn(0x7fdd7808b000 msgr2=0x7fdd7806bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.644+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/420385438 shutdown_connections 2024-04-25T08:32:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.644+0000 7fdd7d3d5640 1 -- 172.21.15.38:0/420385438 wait complete. 2024-04-25T08:32:24.651 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:32:24.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.758+0000 7f9f2924d640 1 Processor -- start 2024-04-25T08:32:24.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.758+0000 7f9f2924d640 1 -- start start 2024-04-25T08:32:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f2924d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9f2415d900 0x7f9f240b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f2924d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9f240ce500 0x7f9f240ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f2924d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f240d1ce0 con 0x7f9f240d1870 2024-04-25T08:32:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f2924d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f240b7b20 con 0x7f9f2415d900 2024-04-25T08:32:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f2924d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f2415dce0 con 0x7f9f240ce500 2024-04-25T08:32:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f22d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9f2415d900 0x7f9f240b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f22d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9f2415d900 0x7f9f240b68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48792/0 (socket says 172.21.15.38:48792) 2024-04-25T08:32:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f22d9d640 1 -- 172.21.15.38:0/1849301371 learned_addr learned my addr 172.21.15.38:0/1849301371 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f2359e640 1 --2- 172.21.15.38:0/1849301371 >> v2:172.21.15.107:3300/0 conn(0x7f9f240ce500 0x7f9f240ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f22d9d640 1 -- 172.21.15.38:0/1849301371 >> v2:172.21.15.107:3300/0 conn(0x7f9f240ce500 msgr2=0x7f9f240ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f22d9d640 1 --2- 172.21.15.38:0/1849301371 >> v2:172.21.15.107:3300/0 conn(0x7f9f240ce500 0x7f9f240ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f22d9d640 1 -- 172.21.15.38:0/1849301371 >> v1:172.21.15.107:6789/0 conn(0x7f9f240d1870 legacy=0x7f9f240b9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:24.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f22d9d640 1 -- 172.21.15.38:0/1849301371 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f240b6f10 con 0x7f9f2415d900 2024-04-25T08:32:24.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.759+0000 7f9f2359e640 1 --2- 172.21.15.38:0/1849301371 >> v2:172.21.15.107:3300/0 conn(0x7f9f240ce500 0x7f9f240ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:24.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.760+0000 7f9f22d9d640 1 --2- 172.21.15.38:0/1849301371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2415d900 0x7f9f240b68a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9f240d1630 tx=0x7f9f1802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a352d9ce1b6a1444 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.760+0000 7f9f21d9b640 1 -- 172.21.15.38:0/1849301371 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18037070 con 0x7f9f2415d900 2024-04-25T08:32:24.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.760+0000 7f9f21d9b640 1 -- 172.21.15.38:0/1849301371 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f18032440 con 0x7f9f2415d900 2024-04-25T08:32:24.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.760+0000 7f9f21d9b640 1 -- 172.21.15.38:0/1849301371 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f18032810 con 0x7f9f2415d900 2024-04-25T08:32:24.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.761+0000 7f9f2924d640 1 -- 172.21.15.38:0/1849301371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2415d900 msgr2=0x7f9f240b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.761+0000 7f9f2924d640 1 --2- 172.21.15.38:0/1849301371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2415d900 0x7f9f240b68a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9f240d1630 tx=0x7f9f1802a230 comp rx=0 tx=0).stop 2024-04-25T08:32:24.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.761+0000 7f9f2924d640 1 -- 172.21.15.38:0/1849301371 shutdown_connections 2024-04-25T08:32:24.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.761+0000 7f9f2924d640 1 --2- 172.21.15.38:0/1849301371 >> v2:172.21.15.107:3300/0 conn(0x7f9f240ce500 0x7f9f240ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.761+0000 7f9f2924d640 1 --2- 172.21.15.38:0/1849301371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f2415d900 0x7f9f240b68a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.761+0000 7f9f2924d640 1 -- 172.21.15.38:0/1849301371 >> 172.21.15.38:0/1849301371 conn(0x7f9f24066d10 msgr2=0x7f9f24067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.761+0000 7f9f2924d640 1 -- 172.21.15.38:0/1849301371 shutdown_connections 2024-04-25T08:32:24.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.761+0000 7f9f2924d640 1 -- 172.21.15.38:0/1849301371 wait complete. 2024-04-25T08:32:24.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.763+0000 7f9f2924d640 1 Processor -- start 2024-04-25T08:32:24.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.763+0000 7f9f2924d640 1 -- start start 2024-04-25T08:32:24.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.763+0000 7f9f2924d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f240ce500 0x7f9f24152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.763+0000 7f9f2924d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f240d1870 0x7f9f240cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.763+0000 7f9f2924d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f2415d900 0x7f9f240c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.763+0000 7f9f2924d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f240b77b0 con 0x7f9f240d1870 2024-04-25T08:32:24.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.763+0000 7f9f2924d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f240b74d0 con 0x7f9f240ce500 2024-04-25T08:32:24.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.763+0000 7f9f2924d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f240cee90 con 0x7f9f2415d900 2024-04-25T08:32:24.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f240d1870 0x7f9f240cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f240d1870 0x7f9f240cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48802/0 (socket says 172.21.15.38:48802) 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2259c640 1 -- 172.21.15.38:0/2058391292 learned_addr learned my addr 172.21.15.38:0/2058391292 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f22d9d640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f240ce500 0x7f9f24152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2359e640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f2415d900 0x7f9f240c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2259c640 1 -- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f2415d900 msgr2=0x7f9f240c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2259c640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f2415d900 0x7f9f240c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2259c640 1 -- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f240ce500 msgr2=0x7f9f24152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2259c640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f240ce500 0x7f9f24152870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2259c640 1 -- 172.21.15.38:0/2058391292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f240cbd70 con 0x7f9f240d1870 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2359e640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f2415d900 0x7f9f240c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2259c640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f240d1870 0x7f9f240cb070 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400c430 tx=0x7f9f1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f0b7fe640 1 -- 172.21.15.38:0/2058391292 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14011790 con 0x7f9f240d1870 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f0b7fe640 1 -- 172.21.15.38:0/2058391292 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f14017070 con 0x7f9f240d1870 2024-04-25T08:32:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f0b7fe640 1 -- 172.21.15.38:0/2058391292 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14011b70 con 0x7f9f240d1870 2024-04-25T08:32:24.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2924d640 1 -- 172.21.15.38:0/2058391292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f240cc050 con 0x7f9f240d1870 2024-04-25T08:32:24.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.764+0000 7f9f2924d640 1 -- 172.21.15.38:0/2058391292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f240cba00 con 0x7f9f240d1870 2024-04-25T08:32:24.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.766+0000 7f9f2924d640 1 -- 172.21.15.38:0/2058391292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2404ef90 con 0x7f9f240d1870 2024-04-25T08:32:24.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.766+0000 7f9f0b7fe640 1 -- 172.21.15.38:0/2058391292 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f14011d10 con 0x7f9f240d1870 2024-04-25T08:32:24.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.766+0000 7f9f0b7fe640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f1006b4d0 0x7f9f1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:24.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.766+0000 7f9f0b7fe640 1 -- 172.21.15.38:0/2058391292 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f9f1408a390 con 0x7f9f240d1870 2024-04-25T08:32:24.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.767+0000 7f9f22d9d640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f1006b4d0 0x7f9f1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:24.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.767+0000 7f9f22d9d640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f1006b4d0 0x7f9f1006d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f18034000 tx=0x7f9f1802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:24.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.774+0000 7f9f0b7fe640 1 -- 172.21.15.38:0/2058391292 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f14058e40 con 0x7f9f240d1870 2024-04-25T08:32:24.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.968+0000 7f9f2924d640 1 -- 172.21.15.38:0/2058391292 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9f240b3610 con 0x7f9f240d1870 2024-04-25T08:32:24.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.968+0000 7f9f0b7fe640 1 -- 172.21.15.38:0/2058391292 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9f14058740 con 0x7f9f240d1870 2024-04-25T08:32:24.964 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:32:24.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.970+0000 7f9f2924d640 1 -- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f1006b4d0 msgr2=0x7f9f1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.970+0000 7f9f2924d640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f1006b4d0 0x7f9f1006d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f18034000 tx=0x7f9f1802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:24.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.970+0000 7f9f2924d640 1 -- 172.21.15.38:0/2058391292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f240d1870 msgr2=0x7f9f240cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:24.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.970+0000 7f9f2924d640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f240d1870 0x7f9f240cb070 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400c430 tx=0x7f9f1400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.970+0000 7f9f2924d640 1 -- 172.21.15.38:0/2058391292 shutdown_connections 2024-04-25T08:32:24.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.970+0000 7f9f2924d640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f1006b4d0 0x7f9f1006d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.970+0000 7f9f2924d640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f2415d900 0x7f9f240c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.970+0000 7f9f2924d640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f240d1870 0x7f9f240cb070 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.970+0000 7f9f2924d640 1 --2- 172.21.15.38:0/2058391292 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f240ce500 0x7f9f24152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:24.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.970+0000 7f9f2924d640 1 -- 172.21.15.38:0/2058391292 >> 172.21.15.38:0/2058391292 conn(0x7f9f24066d10 msgr2=0x7f9f240b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:24.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.970+0000 7f9f2924d640 1 -- 172.21.15.38:0/2058391292 shutdown_connections 2024-04-25T08:32:24.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:24.970+0000 7f9f2924d640 1 -- 172.21.15.38:0/2058391292 wait complete. 2024-04-25T08:32:24.977 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:32:24.978 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:24.997 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:25.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.101+0000 7f43a88a3640 1 Processor -- start 2024-04-25T08:32:25.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.101+0000 7f43a88a3640 1 -- start start 2024-04-25T08:32:25.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.102+0000 7f43a88a3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a415dc10 0x7f43a40ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.102+0000 7f43a88a3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f43a40d0f60 0x7f43a40b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.102+0000 7f43a88a3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43a40d3e70 con 0x7f43a40ced60 2024-04-25T08:32:25.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.102+0000 7f43a88a3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a415dff0 con 0x7f43a40d0f60 2024-04-25T08:32:25.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.102+0000 7f43a88a3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40b6990 con 0x7f43a415dc10 2024-04-25T08:32:25.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.102+0000 7f43a1d9b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f43a40ced60 0x7f43a40d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34974/0 (socket says 172.21.15.38:34974) 2024-04-25T08:32:25.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.102+0000 7f43a1d9b640 1 -- 172.21.15.38:0/1613103012 learned_addr learned my addr 172.21.15.38:0/1613103012 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:25.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.102+0000 7f43a2d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f43a40d0f60 0x7f43a40b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.103+0000 7f43a259c640 1 --2- 172.21.15.38:0/1613103012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a415dc10 0x7f43a40ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.103+0000 7f43a2d9d640 1 -- 172.21.15.38:0/1613103012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a415dc10 msgr2=0x7f43a40ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.103+0000 7f43a2d9d640 1 --2- 172.21.15.38:0/1613103012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a415dc10 0x7f43a40ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.103+0000 7f43a2d9d640 1 -- 172.21.15.38:0/1613103012 >> v1:172.21.15.38:6789/0 conn(0x7f43a40ced60 legacy=0x7f43a40d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.103+0000 7f43a159a640 1 -- 172.21.15.38:0/1613103012 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934061852 0 0) 0x7f43a40d3e70 con 0x7f43a40ced60 2024-04-25T08:32:25.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.103+0000 7f43a2d9d640 1 -- 172.21.15.38:0/1613103012 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a40d4150 con 0x7f43a40d0f60 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.104+0000 7f43a2d9d640 1 --2- 172.21.15.38:0/1613103012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43a40d0f60 0x7f43a40b2370 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f439c009d90 tx=0x7f439c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd1c00967e7b7b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.104+0000 7f43a159a640 1 -- 172.21.15.38:0/1613103012 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c031600 con 0x7f43a40d0f60 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.104+0000 7f43a159a640 1 -- 172.21.15.38:0/1613103012 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c0317a0 con 0x7f43a40d0f60 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.104+0000 7f43a159a640 1 -- 172.21.15.38:0/1613103012 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c031b70 con 0x7f43a40d0f60 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.104+0000 7f43a88a3640 1 -- 172.21.15.38:0/1613103012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43a40d0f60 msgr2=0x7f43a40b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.104+0000 7f43a88a3640 1 --2- 172.21.15.38:0/1613103012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43a40d0f60 0x7f43a40b2370 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f439c009d90 tx=0x7f439c02a5d0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.104+0000 7f43a88a3640 1 -- 172.21.15.38:0/1613103012 shutdown_connections 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.104+0000 7f43a88a3640 1 --2- 172.21.15.38:0/1613103012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43a40d0f60 0x7f43a40b2370 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.104+0000 7f43a88a3640 1 --2- 172.21.15.38:0/1613103012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a415dc10 0x7f43a40ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.104+0000 7f43a88a3640 1 -- 172.21.15.38:0/1613103012 >> 172.21.15.38:0/1613103012 conn(0x7f43a4066d10 msgr2=0x7f43a4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.104+0000 7f43a88a3640 1 -- 172.21.15.38:0/1613103012 shutdown_connections 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.104+0000 7f43a88a3640 1 -- 172.21.15.38:0/1613103012 wait complete. 2024-04-25T08:32:25.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.106+0000 7f43a88a3640 1 Processor -- start 2024-04-25T08:32:25.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.106+0000 7f43a88a3640 1 -- start start 2024-04-25T08:32:25.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.106+0000 7f43a88a3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43a40ced60 0x7f43a40aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a88a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43a40d0f60 0x7f43a4152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a88a3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a415dc10 0x7f43a40abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a88a3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40b6bc0 con 0x7f43a40d0f60 2024-04-25T08:32:25.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a88a3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40d4710 con 0x7f43a40ced60 2024-04-25T08:32:25.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a88a3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a40d3900 con 0x7f43a415dc10 2024-04-25T08:32:25.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43a40d0f60 0x7f43a4152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43a40d0f60 0x7f43a4152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48824/0 (socket says 172.21.15.38:48824) 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a1d9b640 1 -- 172.21.15.38:0/3350042989 learned_addr learned my addr 172.21.15.38:0/3350042989 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a1d9b640 1 -- 172.21.15.38:0/3350042989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a415dc10 msgr2=0x7f43a40abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a1d9b640 1 --2- 172.21.15.38:0/3350042989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a415dc10 0x7f43a40abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a1d9b640 1 -- 172.21.15.38:0/3350042989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43a40ced60 msgr2=0x7f43a40aa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a1d9b640 1 --2- 172.21.15.38:0/3350042989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43a40ced60 0x7f43a40aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a1d9b640 1 -- 172.21.15.38:0/3350042989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a40aabf0 con 0x7f43a40d0f60 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a1d9b640 1 --2- 172.21.15.38:0/3350042989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43a40d0f60 0x7f43a4152c60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f439800c4d0 tx=0x7f439800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f4392ffd640 1 -- 172.21.15.38:0/3350042989 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4398018070 con 0x7f43a40d0f60 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f4392ffd640 1 -- 172.21.15.38:0/3350042989 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4398007500 con 0x7f43a40d0f60 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a88a3640 1 -- 172.21.15.38:0/3350042989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a40aaed0 con 0x7f43a40d0f60 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f4392ffd640 1 -- 172.21.15.38:0/3350042989 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4398004a20 con 0x7f43a40d0f60 2024-04-25T08:32:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.107+0000 7f43a88a3640 1 -- 172.21.15.38:0/3350042989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a40ab490 con 0x7f43a40d0f60 2024-04-25T08:32:25.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.108+0000 7f43a88a3640 1 -- 172.21.15.38:0/3350042989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a4068e80 con 0x7f43a40d0f60 2024-04-25T08:32:25.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.109+0000 7f4392ffd640 1 -- 172.21.15.38:0/3350042989 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f439801b070 con 0x7f43a40d0f60 2024-04-25T08:32:25.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.109+0000 7f4392ffd640 1 --2- 172.21.15.38:0/3350042989 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f438406b4d0 0x7f438406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.109+0000 7f4392ffd640 1 -- 172.21.15.38:0/3350042989 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f4398089930 con 0x7f43a40d0f60 2024-04-25T08:32:25.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.109+0000 7f43a259c640 1 --2- 172.21.15.38:0/3350042989 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f438406b4d0 0x7f438406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.110+0000 7f43a259c640 1 --2- 172.21.15.38:0/3350042989 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f438406b4d0 0x7f438406d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4394006fd0 tx=0x7f4394008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.113+0000 7f4392ffd640 1 -- 172.21.15.38:0/3350042989 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43980583e0 con 0x7f43a40d0f60 2024-04-25T08:32:25.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.296+0000 7f43a88a3640 1 -- 172.21.15.38:0/3350042989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43a40cda00 con 0x7f43a40d0f60 2024-04-25T08:32:25.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.297+0000 7f4392ffd640 1 -- 172.21.15.38:0/3350042989 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18969 (secure 0 0 0) 0x7f4398087e20 con 0x7f43a40d0f60 2024-04-25T08:32:25.293 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:25.293 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:16.572337+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.a","mappings":[{"from":4,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.299+0000 7f43a88a3640 1 -- 172.21.15.38:0/3350042989 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f438406b4d0 msgr2=0x7f438406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.299+0000 7f43a88a3640 1 --2- 172.21.15.38:0/3350042989 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f438406b4d0 0x7f438406d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4394006fd0 tx=0x7f4394008040 comp rx=0 tx=0).stop 2024-04-25T08:32:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.299+0000 7f43a88a3640 1 -- 172.21.15.38:0/3350042989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43a40d0f60 msgr2=0x7f43a4152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.299+0000 7f43a88a3640 1 --2- 172.21.15.38:0/3350042989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43a40d0f60 0x7f43a4152c60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f439800c4d0 tx=0x7f439800c990 comp rx=0 tx=0).stop 2024-04-25T08:32:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.299+0000 7f43a88a3640 1 -- 172.21.15.38:0/3350042989 shutdown_connections 2024-04-25T08:32:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.299+0000 7f43a88a3640 1 --2- 172.21.15.38:0/3350042989 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f438406b4d0 0x7f438406d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.299+0000 7f43a88a3640 1 --2- 172.21.15.38:0/3350042989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43a415dc10 0x7f43a40abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.299+0000 7f43a88a3640 1 --2- 172.21.15.38:0/3350042989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43a40d0f60 0x7f43a4152c60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.299+0000 7f43a88a3640 1 --2- 172.21.15.38:0/3350042989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43a40ced60 0x7f43a40aa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.299+0000 7f43a88a3640 1 -- 172.21.15.38:0/3350042989 >> 172.21.15.38:0/3350042989 conn(0x7f43a4066d10 msgr2=0x7f43a415e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.299+0000 7f43a88a3640 1 -- 172.21.15.38:0/3350042989 shutdown_connections 2024-04-25T08:32:25.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.299+0000 7f43a88a3640 1 -- 172.21.15.38:0/3350042989 wait complete. 2024-04-25T08:32:25.307 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:25.307 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:25.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.414+0000 7f3b583e0640 1 Processor -- start 2024-04-25T08:32:25.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.414+0000 7f3b583e0640 1 -- start start 2024-04-25T08:32:25.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.415+0000 7f3b583e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b500b1d60 0x7f3b500cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.415+0000 7f3b583e0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3b500cff10 0x7f3b50161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.415+0000 7f3b583e0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5015ef40 con 0x7f3b50161c50 2024-04-25T08:32:25.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.415+0000 7f3b583e0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5006abe0 con 0x7f3b500cff10 2024-04-25T08:32:25.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.415+0000 7f3b583e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5006b480 con 0x7f3b500b1d60 2024-04-25T08:32:25.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.416+0000 7f3b5597b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3b500cff10 0x7f3b50161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.416+0000 7f3b5617c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b500b1d60 0x7f3b500cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.416+0000 7f3b5597b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3b500cff10 0x7f3b50161710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:50316/0 (socket says 172.21.15.38:50316) 2024-04-25T08:32:25.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.416+0000 7f3b5597b640 1 -- 172.21.15.38:0/924030956 learned_addr learned my addr 172.21.15.38:0/924030956 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:25.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.416+0000 7f3b5697d640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f3b50161c50 0x7f3b500cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:47202/0 (socket says 172.21.15.38:47202) 2024-04-25T08:32:25.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.416+0000 7f3b5597b640 1 -- 172.21.15.38:0/924030956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b500b1d60 msgr2=0x7f3b500cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.416+0000 7f3b5597b640 1 --2- 172.21.15.38:0/924030956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b500b1d60 0x7f3b500cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.416+0000 7f3b5597b640 1 -- 172.21.15.38:0/924030956 >> v1:172.21.15.107:6789/0 conn(0x7f3b50161c50 legacy=0x7f3b500cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.416+0000 7f3b5597b640 1 -- 172.21.15.38:0/924030956 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5006ae30 con 0x7f3b500cff10 2024-04-25T08:32:25.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.416+0000 7f3b5617c640 1 --2- 172.21.15.38:0/924030956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b500b1d60 0x7f3b500cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:25.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.417+0000 7f3b5597b640 1 --2- 172.21.15.38:0/924030956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b500cff10 0x7f3b50161710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009870 tx=0x7f3b4402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40f10bc41eb3fbde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.417+0000 7f3b5517a640 1 -- 172.21.15.38:0/924030956 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44037070 con 0x7f3b500cff10 2024-04-25T08:32:25.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.417+0000 7f3b5517a640 1 -- 172.21.15.38:0/924030956 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b44032440 con 0x7f3b500cff10 2024-04-25T08:32:25.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.417+0000 7f3b5517a640 1 -- 172.21.15.38:0/924030956 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44032810 con 0x7f3b500cff10 2024-04-25T08:32:25.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.418+0000 7f3b583e0640 1 -- 172.21.15.38:0/924030956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b500cff10 msgr2=0x7f3b50161710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.418+0000 7f3b583e0640 1 --2- 172.21.15.38:0/924030956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b500cff10 0x7f3b50161710 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3b44009870 tx=0x7f3b4402a270 comp rx=0 tx=0).stop 2024-04-25T08:32:25.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.418+0000 7f3b583e0640 1 -- 172.21.15.38:0/924030956 shutdown_connections 2024-04-25T08:32:25.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.418+0000 7f3b583e0640 1 --2- 172.21.15.38:0/924030956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b500cff10 0x7f3b50161710 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.418+0000 7f3b583e0640 1 --2- 172.21.15.38:0/924030956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b500b1d60 0x7f3b500cf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.418+0000 7f3b583e0640 1 -- 172.21.15.38:0/924030956 >> 172.21.15.38:0/924030956 conn(0x7f3b500d06b0 msgr2=0x7f3b500d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.418+0000 7f3b583e0640 1 -- 172.21.15.38:0/924030956 shutdown_connections 2024-04-25T08:32:25.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.418+0000 7f3b583e0640 1 -- 172.21.15.38:0/924030956 wait complete. 2024-04-25T08:32:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.419+0000 7f3b583e0640 1 Processor -- start 2024-04-25T08:32:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.419+0000 7f3b583e0640 1 -- start start 2024-04-25T08:32:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b583e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b500b1d60 0x7f3b501538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b583e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b500cff10 0x7f3b500c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b583e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b50161c50 0x7f3b501e85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b583e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5006b910 con 0x7f3b500b1d60 2024-04-25T08:32:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b583e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500d0370 con 0x7f3b500cff10 2024-04-25T08:32:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b583e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5006ba90 con 0x7f3b50161c50 2024-04-25T08:32:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b5617c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b500b1d60 0x7f3b501538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b5617c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b500b1d60 0x7f3b501538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48840/0 (socket says 172.21.15.38:48840) 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b5617c640 1 -- 172.21.15.38:0/3587395819 learned_addr learned my addr 172.21.15.38:0/3587395819 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b5597b640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b500cff10 0x7f3b500c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b5697d640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b50161c50 0x7f3b501e85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b5617c640 1 -- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b50161c50 msgr2=0x7f3b501e85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b5617c640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b50161c50 0x7f3b501e85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b5617c640 1 -- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b500cff10 msgr2=0x7f3b500c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b5617c640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b500cff10 0x7f3b500c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b5617c640 1 -- 172.21.15.38:0/3587395819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b500c5280 con 0x7f3b500b1d60 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b5617c640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b500b1d60 0x7f3b501538c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000efc0 tx=0x7f3b4000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b4effd640 1 -- 172.21.15.38:0/3587395819 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40018020 con 0x7f3b500b1d60 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.420+0000 7f3b4effd640 1 -- 172.21.15.38:0/3587395819 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b40014680 con 0x7f3b500b1d60 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.421+0000 7f3b583e0640 1 -- 172.21.15.38:0/3587395819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b50159400 con 0x7f3b500b1d60 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.421+0000 7f3b4effd640 1 -- 172.21.15.38:0/3587395819 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40014a50 con 0x7f3b500b1d60 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.421+0000 7f3b583e0640 1 -- 172.21.15.38:0/3587395819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b501599c0 con 0x7f3b500b1d60 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.422+0000 7f3b583e0640 1 -- 172.21.15.38:0/3587395819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b50068e80 con 0x7f3b500b1d60 2024-04-25T08:32:25.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.422+0000 7f3b4effd640 1 -- 172.21.15.38:0/3587395819 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b40005d10 con 0x7f3b500b1d60 2024-04-25T08:32:25.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.422+0000 7f3b4effd640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b3c06b4d0 0x7f3b3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.422+0000 7f3b4effd640 1 -- 172.21.15.38:0/3587395819 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f3b4008b3b0 con 0x7f3b500b1d60 2024-04-25T08:32:25.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.423+0000 7f3b5597b640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b3c06b4d0 0x7f3b3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.423+0000 7f3b5597b640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b3c06b4d0 0x7f3b3c06d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b44034000 tx=0x7f3b4402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.426+0000 7f3b4effd640 1 -- 172.21.15.38:0/3587395819 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b40022030 con 0x7f3b500b1d60 2024-04-25T08:32:25.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.606+0000 7f3b583e0640 1 -- 172.21.15.38:0/3587395819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b500b61e0 con 0x7f3b500b1d60 2024-04-25T08:32:25.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.607+0000 7f3b4effd640 1 -- 172.21.15.38:0/3587395819 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b500c5280 con 0x7f3b500b1d60 2024-04-25T08:32:25.603 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:25.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.609+0000 7f3b583e0640 1 -- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b3c06b4d0 msgr2=0x7f3b3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.609+0000 7f3b583e0640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b3c06b4d0 0x7f3b3c06d990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3b44034000 tx=0x7f3b4402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:25.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.609+0000 7f3b583e0640 1 -- 172.21.15.38:0/3587395819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b500b1d60 msgr2=0x7f3b501538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.609+0000 7f3b583e0640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b500b1d60 0x7f3b501538c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000efc0 tx=0x7f3b4000c490 comp rx=0 tx=0).stop 2024-04-25T08:32:25.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.609+0000 7f3b583e0640 1 -- 172.21.15.38:0/3587395819 shutdown_connections 2024-04-25T08:32:25.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.609+0000 7f3b583e0640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3b3c06b4d0 0x7f3b3c06d990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.609+0000 7f3b583e0640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3b50161c50 0x7f3b501e85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.609+0000 7f3b583e0640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b500cff10 0x7f3b500c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.609+0000 7f3b583e0640 1 --2- 172.21.15.38:0/3587395819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3b500b1d60 0x7f3b501538c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.609+0000 7f3b583e0640 1 -- 172.21.15.38:0/3587395819 >> 172.21.15.38:0/3587395819 conn(0x7f3b500d06b0 msgr2=0x7f3b500cb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.609+0000 7f3b583e0640 1 -- 172.21.15.38:0/3587395819 shutdown_connections 2024-04-25T08:32:25.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.609+0000 7f3b583e0640 1 -- 172.21.15.38:0/3587395819 wait complete. 2024-04-25T08:32:25.616 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:25.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.723+0000 7fb0b9913640 1 Processor -- start 2024-04-25T08:32:25.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.723+0000 7fb0b9913640 1 -- start start 2024-04-25T08:32:25.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b9913640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb0b40d0320 0x7fb0b40d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b9913640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0b40d0700 0x7fb0b40d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b9913640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb0b40d1e90 0x7fb0b40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b9913640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b40d2330 con 0x7fb0b40d0320 2024-04-25T08:32:25.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b9913640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b40d2610 con 0x7fb0b40d1e90 2024-04-25T08:32:25.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b9913640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b40d28f0 con 0x7fb0b40d0700 2024-04-25T08:32:25.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b2ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb0b40d0320 0x7fb0b40d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b2ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb0b40d0320 0x7fb0b40d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48852/0 (socket says 172.21.15.38:48852) 2024-04-25T08:32:25.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b2ffd640 1 -- 172.21.15.38:0/227178896 learned_addr learned my addr 172.21.15.38:0/227178896 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:25.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b37fe640 1 --2- 172.21.15.38:0/227178896 >> v2:172.21.15.107:3300/0 conn(0x7fb0b40d1e90 0x7fb0b40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b27fc640 1 --2- 172.21.15.38:0/227178896 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0b40d0700 0x7fb0b40d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b2ffd640 1 -- 172.21.15.38:0/227178896 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0b40d0700 msgr2=0x7fb0b40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b2ffd640 1 --2- 172.21.15.38:0/227178896 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0b40d0700 0x7fb0b40d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b2ffd640 1 -- 172.21.15.38:0/227178896 >> v2:172.21.15.107:3300/0 conn(0x7fb0b40d1e90 msgr2=0x7fb0b40b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b2ffd640 1 --2- 172.21.15.38:0/227178896 >> v2:172.21.15.107:3300/0 conn(0x7fb0b40d1e90 0x7fb0b40b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b2ffd640 1 -- 172.21.15.38:0/227178896 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b40d2bd0 con 0x7fb0b40d0320 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b37fe640 1 --2- 172.21.15.38:0/227178896 >> v2:172.21.15.107:3300/0 conn(0x7fb0b40d1e90 0x7fb0b40b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.724+0000 7fb0b27fc640 1 --2- 172.21.15.38:0/227178896 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0b40d0700 0x7fb0b40d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.725+0000 7fb0b2ffd640 1 --2- 172.21.15.38:0/227178896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b40d0320 0x7fb0b40d33d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40b68d0 tx=0x7fb0a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9826008bed463414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.725+0000 7fb0b1ffb640 1 -- 172.21.15.38:0/227178896 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4037070 con 0x7fb0b40d0320 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.725+0000 7fb0b1ffb640 1 -- 172.21.15.38:0/227178896 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a4032440 con 0x7fb0b40d0320 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.725+0000 7fb0b1ffb640 1 -- 172.21.15.38:0/227178896 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4032810 con 0x7fb0b40d0320 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.726+0000 7fb0b9913640 1 -- 172.21.15.38:0/227178896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b40d0320 msgr2=0x7fb0b40d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.726+0000 7fb0b9913640 1 --2- 172.21.15.38:0/227178896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b40d0320 0x7fb0b40d33d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb0b40b68d0 tx=0x7fb0a402a270 comp rx=0 tx=0).stop 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.726+0000 7fb0b9913640 1 -- 172.21.15.38:0/227178896 shutdown_connections 2024-04-25T08:32:25.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.726+0000 7fb0b9913640 1 --2- 172.21.15.38:0/227178896 >> v2:172.21.15.107:3300/0 conn(0x7fb0b40d1e90 0x7fb0b40b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.726+0000 7fb0b9913640 1 --2- 172.21.15.38:0/227178896 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0b40d0700 0x7fb0b40d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.726+0000 7fb0b9913640 1 --2- 172.21.15.38:0/227178896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b40d0320 0x7fb0b40d33d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.726+0000 7fb0b9913640 1 -- 172.21.15.38:0/227178896 >> 172.21.15.38:0/227178896 conn(0x7fb0b40668f0 msgr2=0x7fb0b4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.726+0000 7fb0b9913640 1 -- 172.21.15.38:0/227178896 shutdown_connections 2024-04-25T08:32:25.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.726+0000 7fb0b9913640 1 -- 172.21.15.38:0/227178896 wait complete. 2024-04-25T08:32:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.727+0000 7fb0b9913640 1 Processor -- start 2024-04-25T08:32:25.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.727+0000 7fb0b9913640 1 -- start start 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.728+0000 7fb0b9913640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0b40d0320 0x7fb0b40c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.728+0000 7fb0b9913640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0b40d0700 0x7fb0b4157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.728+0000 7fb0b9913640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b40d1e90 0x7fb0b415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.728+0000 7fb0b9913640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b40d0b40 con 0x7fb0b40d1e90 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.728+0000 7fb0b9913640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b4161210 con 0x7fb0b40d0700 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.728+0000 7fb0b9913640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b40cd8e0 con 0x7fb0b40d0320 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.728+0000 7fb0b37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b40d1e90 0x7fb0b415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.728+0000 7fb0b37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b40d1e90 0x7fb0b415e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48866/0 (socket says 172.21.15.38:48866) 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.728+0000 7fb0b37fe640 1 -- 172.21.15.38:0/279098443 learned_addr learned my addr 172.21.15.38:0/279098443 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.728+0000 7fb0b27fc640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0b40d0700 0x7fb0b4157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.728+0000 7fb0b2ffd640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0b40d0320 0x7fb0b40c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.729+0000 7fb0b37fe640 1 -- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0b40d0320 msgr2=0x7fb0b40c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.729+0000 7fb0b37fe640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0b40d0320 0x7fb0b40c9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.729+0000 7fb0b37fe640 1 -- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0b40d0700 msgr2=0x7fb0b4157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.729+0000 7fb0b37fe640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0b40d0700 0x7fb0b4157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.729+0000 7fb0b37fe640 1 -- 172.21.15.38:0/279098443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b4154f80 con 0x7fb0b40d1e90 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.729+0000 7fb0b37fe640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b40d1e90 0x7fb0b415e980 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00cd10 tx=0x7fb0ac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.729+0000 7fb09bfff640 1 -- 172.21.15.38:0/279098443 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac003b90 con 0x7fb0b40d1e90 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.729+0000 7fb09bfff640 1 -- 172.21.15.38:0/279098443 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac003d50 con 0x7fb0b40d1e90 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.729+0000 7fb0b9913640 1 -- 172.21.15.38:0/279098443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b40cd040 con 0x7fb0b40d1e90 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.729+0000 7fb09bfff640 1 -- 172.21.15.38:0/279098443 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac004260 con 0x7fb0b40d1e90 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.729+0000 7fb0b9913640 1 -- 172.21.15.38:0/279098443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b40cd600 con 0x7fb0b40d1e90 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.730+0000 7fb0b9913640 1 -- 172.21.15.38:0/279098443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b4068e80 con 0x7fb0b40d1e90 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.730+0000 7fb09bfff640 1 -- 172.21.15.38:0/279098443 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0ac004400 con 0x7fb0b40d1e90 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.731+0000 7fb09bfff640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb0a006b4d0 0x7fb0a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.731+0000 7fb09bfff640 1 -- 172.21.15.38:0/279098443 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fb0ac08afd0 con 0x7fb0b40d1e90 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.731+0000 7fb0b2ffd640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb0a006b4d0 0x7fb0a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:25.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.732+0000 7fb0b2ffd640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb0a006b4d0 0x7fb0a006d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4034000 tx=0x7fb0a402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:25.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.735+0000 7fb09bfff640 1 -- 172.21.15.38:0/279098443 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0ac014030 con 0x7fb0b40d1e90 2024-04-25T08:32:25.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.915+0000 7fb0b9913640 1 -- 172.21.15.38:0/279098443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb0b40b3050 con 0x7fb0b40d1e90 2024-04-25T08:32:25.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.915+0000 7fb09bfff640 1 -- 172.21.15.38:0/279098443 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb0ac0593a0 con 0x7fb0b40d1e90 2024-04-25T08:32:25.912 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:25.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.917+0000 7fb0b9913640 1 -- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb0a006b4d0 msgr2=0x7fb0a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.917+0000 7fb0b9913640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb0a006b4d0 0x7fb0a006d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4034000 tx=0x7fb0a402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:25.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.917+0000 7fb0b9913640 1 -- 172.21.15.38:0/279098443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b40d1e90 msgr2=0x7fb0b415e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:25.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.917+0000 7fb0b9913640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b40d1e90 0x7fb0b415e980 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac00cd10 tx=0x7fb0ac00a650 comp rx=0 tx=0).stop 2024-04-25T08:32:25.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.917+0000 7fb0b9913640 1 -- 172.21.15.38:0/279098443 shutdown_connections 2024-04-25T08:32:25.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.917+0000 7fb0b9913640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0b40d1e90 0x7fb0b415e980 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.917+0000 7fb0b9913640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0b40d0700 0x7fb0b4157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.917+0000 7fb0b9913640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb0a006b4d0 0x7fb0a006d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.917+0000 7fb0b9913640 1 --2- 172.21.15.38:0/279098443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0b40d0320 0x7fb0b40c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:25.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.917+0000 7fb0b9913640 1 -- 172.21.15.38:0/279098443 >> 172.21.15.38:0/279098443 conn(0x7fb0b40668f0 msgr2=0x7fb0b41602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:25.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.918+0000 7fb0b9913640 1 -- 172.21.15.38:0/279098443 shutdown_connections 2024-04-25T08:32:25.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:25.918+0000 7fb0b9913640 1 -- 172.21.15.38:0/279098443 wait complete. 2024-04-25T08:32:25.924 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:26.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.032+0000 7f7ef1f63640 1 Processor -- start 2024-04-25T08:32:26.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.032+0000 7f7ef1f63640 1 -- start start 2024-04-25T08:32:26.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.033+0000 7f7ef1f63640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7eec09a7b0 0x7f7eec162390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.033+0000 7f7ef1f63640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eec15f300 con 0x7f7eec15df30 2024-04-25T08:32:26.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.033+0000 7f7ef1f63640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eec15e540 con 0x7f7eec09a180 2024-04-25T08:32:26.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.033+0000 7f7ef1f63640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eec15e310 con 0x7f7eec09a7b0 2024-04-25T08:32:26.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.033+0000 7f7eeb7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7eec15df30 0x7f7eec099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34990/0 (socket says 172.21.15.38:34990) 2024-04-25T08:32:26.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.033+0000 7f7eeb7fe640 1 -- 172.21.15.38:0/45353885 learned_addr learned my addr 172.21.15.38:0/45353885 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:26.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.033+0000 7f7eebfff640 1 --2- 172.21.15.38:0/45353885 >> v2:172.21.15.107:3300/0 conn(0x7f7eec09a7b0 0x7f7eec162390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.033+0000 7f7eea7fc640 1 -- 172.21.15.38:0/45353885 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800740958 0 0) 0x7f7eec15f300 con 0x7f7eec15df30 2024-04-25T08:32:26.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.033+0000 7f7eea7fc640 1 -- 172.21.15.38:0/45353885 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ed4003520 con 0x7f7eec15df30 2024-04-25T08:32:26.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.034+0000 7f7eebfff640 1 -- 172.21.15.38:0/45353885 >> v1:172.21.15.107:6789/0 conn(0x7f7eec09a180 legacy=0x7f7eec0cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.034+0000 7f7eea7fc640 1 -- 172.21.15.38:0/45353885 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 221845218 0 0) 0x7f7ed4003520 con 0x7f7eec15df30 2024-04-25T08:32:26.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.034+0000 7f7eebfff640 1 -- 172.21.15.38:0/45353885 >> v1:172.21.15.38:6789/0 conn(0x7f7eec15df30 legacy=0x7f7eec099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.034+0000 7f7eebfff640 1 -- 172.21.15.38:0/45353885 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec15e820 con 0x7f7eec09a7b0 2024-04-25T08:32:26.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.034+0000 7f7eebfff640 1 --2- 172.21.15.38:0/45353885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eec09a7b0 0x7f7eec162390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009d90 tx=0x7f7ee402a5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70a0d8bf5d6a61a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.035+0000 7f7eea7fc640 1 -- 172.21.15.38:0/45353885 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4035020 con 0x7f7eec09a7b0 2024-04-25T08:32:26.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.035+0000 7f7eea7fc640 1 -- 172.21.15.38:0/45353885 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee4007540 con 0x7f7eec09a7b0 2024-04-25T08:32:26.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.035+0000 7f7eea7fc640 1 -- 172.21.15.38:0/45353885 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc001670 con 0x7f7eec09a7b0 2024-04-25T08:32:26.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.035+0000 7f7ef1f63640 1 -- 172.21.15.38:0/45353885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eec09a7b0 msgr2=0x7f7eec162390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.035+0000 7f7ef1f63640 1 --2- 172.21.15.38:0/45353885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eec09a7b0 0x7f7eec162390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009d90 tx=0x7f7ee402a5b0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.035+0000 7f7ef1f63640 1 -- 172.21.15.38:0/45353885 shutdown_connections 2024-04-25T08:32:26.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.035+0000 7f7ef1f63640 1 --2- 172.21.15.38:0/45353885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eec09a7b0 0x7f7eec162390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.035+0000 7f7ef1f63640 1 -- 172.21.15.38:0/45353885 >> 172.21.15.38:0/45353885 conn(0x7f7eec0b3110 msgr2=0x7f7eec0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.035+0000 7f7ef1f63640 1 -- 172.21.15.38:0/45353885 shutdown_connections 2024-04-25T08:32:26.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.035+0000 7f7ef1f63640 1 -- 172.21.15.38:0/45353885 wait complete. 2024-04-25T08:32:26.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.036+0000 7f7ef1f63640 1 Processor -- start 2024-04-25T08:32:26.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.037+0000 7f7ef1f63640 1 -- start start 2024-04-25T08:32:26.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.037+0000 7f7ef1f63640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7eec09a180 0x7f7eec14c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.037+0000 7f7ef1f63640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eec09a7b0 0x7f7eec0c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.037+0000 7f7ef1f63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eec15df30 0x7f7eec1f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.037+0000 7f7ef1f63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec15ee70 con 0x7f7eec15df30 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.037+0000 7f7ef1f63640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec09aeb0 con 0x7f7eec09a7b0 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.037+0000 7f7ef1f63640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec09abc0 con 0x7f7eec09a180 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.037+0000 7f7eebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eec15df30 0x7f7eec1f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.037+0000 7f7eebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eec15df30 0x7f7eec1f3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48872/0 (socket says 172.21.15.38:48872) 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.037+0000 7f7eebfff640 1 -- 172.21.15.38:0/140342996 learned_addr learned my addr 172.21.15.38:0/140342996 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.037+0000 7f7eeaffd640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eec09a7b0 0x7f7eec0c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.037+0000 7f7eeb7fe640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7eec09a180 0x7f7eec14c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7eebfff640 1 -- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7eec09a180 msgr2=0x7f7eec14c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7eebfff640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7eec09a180 0x7f7eec14c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7eebfff640 1 -- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eec09a7b0 msgr2=0x7f7eec0c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7eebfff640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eec09a7b0 0x7f7eec0c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7eebfff640 1 -- 172.21.15.38:0/140342996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec0c5e90 con 0x7f7eec15df30 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7eeaffd640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eec09a7b0 0x7f7eec0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7eeb7fe640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7eec09a180 0x7f7eec14c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:26.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7eebfff640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eec15df30 0x7f7eec1f3840 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009eb0 tx=0x7f7ee4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7ed3fff640 1 -- 172.21.15.38:0/140342996 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee403f070 con 0x7f7eec15df30 2024-04-25T08:32:26.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7ed3fff640 1 -- 172.21.15.38:0/140342996 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee4031bc0 con 0x7f7eec15df30 2024-04-25T08:32:26.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7ef1f63640 1 -- 172.21.15.38:0/140342996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec0c6170 con 0x7f7eec15df30 2024-04-25T08:32:26.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7ed3fff640 1 -- 172.21.15.38:0/140342996 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4035070 con 0x7f7eec15df30 2024-04-25T08:32:26.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.038+0000 7f7ef1f63640 1 -- 172.21.15.38:0/140342996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec0c6730 con 0x7f7eec15df30 2024-04-25T08:32:26.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.039+0000 7f7ef1f63640 1 -- 172.21.15.38:0/140342996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eec068e80 con 0x7f7eec15df30 2024-04-25T08:32:26.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.039+0000 7f7ed3fff640 1 -- 172.21.15.38:0/140342996 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ee4042070 con 0x7f7eec15df30 2024-04-25T08:32:26.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.040+0000 7f7ed3fff640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ed806b4d0 0x7f7ed806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.040+0000 7f7ed3fff640 1 -- 172.21.15.38:0/140342996 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f7ee40ab720 con 0x7f7eec15df30 2024-04-25T08:32:26.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.040+0000 7f7eeb7fe640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ed806b4d0 0x7f7ed806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.041+0000 7f7eeb7fe640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ed806b4d0 0x7f7ed806d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7edc001eb0 tx=0x7f7edc001d00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.044+0000 7f7ed3fff640 1 -- 172.21.15.38:0/140342996 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ee407a0c0 con 0x7f7eec15df30 2024-04-25T08:32:26.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.224+0000 7f7ef1f63640 1 -- 172.21.15.38:0/140342996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7eec0b3de0 con 0x7f7eec15df30 2024-04-25T08:32:26.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.224+0000 7f7ed3fff640 1 -- 172.21.15.38:0/140342996 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ee40799c0 con 0x7f7eec15df30 2024-04-25T08:32:26.221 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:26.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.226+0000 7f7ef1f63640 1 -- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ed806b4d0 msgr2=0x7f7ed806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.226+0000 7f7ef1f63640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ed806b4d0 0x7f7ed806d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7edc001eb0 tx=0x7f7edc001d00 comp rx=0 tx=0).stop 2024-04-25T08:32:26.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.226+0000 7f7ef1f63640 1 -- 172.21.15.38:0/140342996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eec15df30 msgr2=0x7f7eec1f3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.226+0000 7f7ef1f63640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eec15df30 0x7f7eec1f3840 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7ee4009eb0 tx=0x7f7ee4004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:26.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.226+0000 7f7ef1f63640 1 -- 172.21.15.38:0/140342996 shutdown_connections 2024-04-25T08:32:26.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.226+0000 7f7ef1f63640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7eec15df30 0x7f7eec1f3840 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.226+0000 7f7ef1f63640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7eec09a7b0 0x7f7eec0c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.226+0000 7f7ef1f63640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ed806b4d0 0x7f7ed806d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.226+0000 7f7ef1f63640 1 --2- 172.21.15.38:0/140342996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7eec09a180 0x7f7eec14c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.226+0000 7f7ef1f63640 1 -- 172.21.15.38:0/140342996 >> 172.21.15.38:0/140342996 conn(0x7f7eec0b3110 msgr2=0x7f7eec0d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.226+0000 7f7ef1f63640 1 -- 172.21.15.38:0/140342996 shutdown_connections 2024-04-25T08:32:26.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.226+0000 7f7ef1f63640 1 -- 172.21.15.38:0/140342996 wait complete. 2024-04-25T08:32:26.233 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:26.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.340+0000 7fcf48db7640 1 Processor -- start 2024-04-25T08:32:26.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.341+0000 7fcf48db7640 1 -- start start 2024-04-25T08:32:26.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.341+0000 7fcf48db7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcf440d0320 0x7fcf440d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.341+0000 7fcf48db7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf440b6980 0x7fcf440b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.341+0000 7fcf48db7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf440d2060 con 0x7fcf440d0700 2024-04-25T08:32:26.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.341+0000 7fcf48db7640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf440d2330 con 0x7fcf440d0320 2024-04-25T08:32:26.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.341+0000 7fcf48db7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440d2610 con 0x7fcf440b6980 2024-04-25T08:32:26.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.341+0000 7fcf4259c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcf440d0320 0x7fcf440d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.342+0000 7fcf4259c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcf440d0320 0x7fcf440d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48888/0 (socket says 172.21.15.38:48888) 2024-04-25T08:32:26.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.342+0000 7fcf4259c640 1 -- 172.21.15.38:0/4199162984 learned_addr learned my addr 172.21.15.38:0/4199162984 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:26.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.342+0000 7fcf42d9d640 1 --2- 172.21.15.38:0/4199162984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf440b6980 0x7fcf440b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.342+0000 7fcf4259c640 1 -- 172.21.15.38:0/4199162984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf440b6980 msgr2=0x7fcf440b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.342+0000 7fcf4259c640 1 --2- 172.21.15.38:0/4199162984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf440b6980 0x7fcf440b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.342+0000 7fcf4259c640 1 -- 172.21.15.38:0/4199162984 >> v1:172.21.15.107:6789/0 conn(0x7fcf440d0700 legacy=0x7fcf440d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.342+0000 7fcf4259c640 1 -- 172.21.15.38:0/4199162984 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440d28f0 con 0x7fcf440d0320 2024-04-25T08:32:26.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.343+0000 7fcf4259c640 1 --2- 172.21.15.38:0/4199162984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf440d0320 0x7fcf440d33d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009870 tx=0x7fcf3402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c56ea58cb0bb7dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.343+0000 7fcf41d9b640 1 -- 172.21.15.38:0/4199162984 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34037070 con 0x7fcf440d0320 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.343+0000 7fcf41d9b640 1 -- 172.21.15.38:0/4199162984 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3402ae50 con 0x7fcf440d0320 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.343+0000 7fcf41d9b640 1 -- 172.21.15.38:0/4199162984 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34032860 con 0x7fcf440d0320 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.343+0000 7fcf48db7640 1 -- 172.21.15.38:0/4199162984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf440d0320 msgr2=0x7fcf440d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.343+0000 7fcf48db7640 1 --2- 172.21.15.38:0/4199162984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf440d0320 0x7fcf440d33d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009870 tx=0x7fcf3402a300 comp rx=0 tx=0).stop 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.343+0000 7fcf48db7640 1 -- 172.21.15.38:0/4199162984 shutdown_connections 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.343+0000 7fcf48db7640 1 --2- 172.21.15.38:0/4199162984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf440b6980 0x7fcf440b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.343+0000 7fcf48db7640 1 --2- 172.21.15.38:0/4199162984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf440d0320 0x7fcf440d33d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.343+0000 7fcf48db7640 1 -- 172.21.15.38:0/4199162984 >> 172.21.15.38:0/4199162984 conn(0x7fcf440668f0 msgr2=0x7fcf44066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.343+0000 7fcf48db7640 1 -- 172.21.15.38:0/4199162984 shutdown_connections 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.343+0000 7fcf48db7640 1 -- 172.21.15.38:0/4199162984 wait complete. 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.345+0000 7fcf48db7640 1 Processor -- start 2024-04-25T08:32:26.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.345+0000 7fcf48db7640 1 -- start start 2024-04-25T08:32:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.345+0000 7fcf48db7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf440b6980 0x7fcf440b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.345+0000 7fcf48db7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf440d0320 0x7fcf4415ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.345+0000 7fcf48db7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf440d0700 0x7fcf44159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.345+0000 7fcf48db7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440d0b40 con 0x7fcf440d0320 2024-04-25T08:32:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.345+0000 7fcf48db7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440d2eb0 con 0x7fcf440b6980 2024-04-25T08:32:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.345+0000 7fcf48db7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf440b42c0 con 0x7fcf440d0700 2024-04-25T08:32:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.345+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf440d0320 0x7fcf4415ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.345+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf440d0320 0x7fcf4415ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48902/0 (socket says 172.21.15.38:48902) 2024-04-25T08:32:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.345+0000 7fcf3bfff640 1 -- 172.21.15.38:0/398443904 learned_addr learned my addr 172.21.15.38:0/398443904 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:26.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf42d9d640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf440d0700 0x7fcf44159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf4259c640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf440b6980 0x7fcf440b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf3bfff640 1 -- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf440d0700 msgr2=0x7fcf44159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf3bfff640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf440d0700 0x7fcf44159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf3bfff640 1 -- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf440b6980 msgr2=0x7fcf440b5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf3bfff640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf440b6980 0x7fcf440b5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf3bfff640 1 -- 172.21.15.38:0/398443904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf440b4560 con 0x7fcf440d0320 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf42d9d640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf440d0700 0x7fcf44159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf3bfff640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf440d0320 0x7fcf4415ed10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00c3c0 tx=0x7fcf2c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf3affd640 1 -- 172.21.15.38:0/398443904 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c0052d0 con 0x7fcf440d0320 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf3affd640 1 -- 172.21.15.38:0/398443904 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c005490 con 0x7fcf440d0320 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf48db7640 1 -- 172.21.15.38:0/398443904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf440b4840 con 0x7fcf440d0320 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf3affd640 1 -- 172.21.15.38:0/398443904 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c007260 con 0x7fcf440d0320 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.346+0000 7fcf48db7640 1 -- 172.21.15.38:0/398443904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf440b4e00 con 0x7fcf440d0320 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.347+0000 7fcf48db7640 1 -- 172.21.15.38:0/398443904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf440b5000 con 0x7fcf440d0320 2024-04-25T08:32:26.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.347+0000 7fcf3affd640 1 -- 172.21.15.38:0/398443904 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf2c007400 con 0x7fcf440d0320 2024-04-25T08:32:26.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.348+0000 7fcf3affd640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcf2406b4d0 0x7fcf2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.348+0000 7fcf3affd640 1 -- 172.21.15.38:0/398443904 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fcf2c089a50 con 0x7fcf440d0320 2024-04-25T08:32:26.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.348+0000 7fcf4259c640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcf2406b4d0 0x7fcf2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.349+0000 7fcf4259c640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcf2406b4d0 0x7fcf2406d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcf34034000 tx=0x7fcf3402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.352+0000 7fcf3affd640 1 -- 172.21.15.38:0/398443904 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf440b5000 con 0x7fcf440d0320 2024-04-25T08:32:26.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.532+0000 7fcf48db7640 1 -- 172.21.15.38:0/398443904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcf44068e80 con 0x7fcf440d0320 2024-04-25T08:32:26.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.532+0000 7fcf3affd640 1 -- 172.21.15.38:0/398443904 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcf2c057e20 con 0x7fcf440d0320 2024-04-25T08:32:26.528 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:26.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.534+0000 7fcf48db7640 1 -- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcf2406b4d0 msgr2=0x7fcf2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.534+0000 7fcf48db7640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcf2406b4d0 0x7fcf2406d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcf34034000 tx=0x7fcf3402e000 comp rx=0 tx=0).stop 2024-04-25T08:32:26.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.534+0000 7fcf48db7640 1 -- 172.21.15.38:0/398443904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf440d0320 msgr2=0x7fcf4415ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.534+0000 7fcf48db7640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf440d0320 0x7fcf4415ed10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00c3c0 tx=0x7fcf2c00c880 comp rx=0 tx=0).stop 2024-04-25T08:32:26.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.534+0000 7fcf48db7640 1 -- 172.21.15.38:0/398443904 shutdown_connections 2024-04-25T08:32:26.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.534+0000 7fcf48db7640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcf2406b4d0 0x7fcf2406d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.534+0000 7fcf48db7640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf440d0700 0x7fcf44159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.534+0000 7fcf48db7640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf440d0320 0x7fcf4415ed10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.534+0000 7fcf48db7640 1 --2- 172.21.15.38:0/398443904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf440b6980 0x7fcf440b5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.534+0000 7fcf48db7640 1 -- 172.21.15.38:0/398443904 >> 172.21.15.38:0/398443904 conn(0x7fcf440668f0 msgr2=0x7fcf4415fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.535+0000 7fcf48db7640 1 -- 172.21.15.38:0/398443904 shutdown_connections 2024-04-25T08:32:26.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.535+0000 7fcf48db7640 1 -- 172.21.15.38:0/398443904 wait complete. 2024-04-25T08:32:26.542 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:32:26.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.648+0000 7f3a40e3f640 1 Processor -- start 2024-04-25T08:32:26.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.648+0000 7f3a40e3f640 1 -- start start 2024-04-25T08:32:26.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a40e3f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3a3c160ef0 0x7f3a3c0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a40e3f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a3c0b2ab0 0x7f3a3c0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a40e3f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3c0b9610 con 0x7f3a3c0b9180 2024-04-25T08:32:26.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a40e3f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3c161cf0 con 0x7f3a3c160ef0 2024-04-25T08:32:26.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a40e3f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c1612d0 con 0x7f3a3c0b2ab0 2024-04-25T08:32:26.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a3a59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3a3c160ef0 0x7f3a3c0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a33fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f3a3c0b9180 0x7f3a3c0d37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34994/0 (socket says 172.21.15.38:34994) 2024-04-25T08:32:26.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a33fff640 1 -- 172.21.15.38:0/3428987154 learned_addr learned my addr 172.21.15.38:0/3428987154 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:26.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a3ad9d640 1 --2- 172.21.15.38:0/3428987154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a3c0b2ab0 0x7f3a3c0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a3a59c640 1 -- 172.21.15.38:0/3428987154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a3c0b2ab0 msgr2=0x7f3a3c0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:26.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a3a59c640 1 --2- 172.21.15.38:0/3428987154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a3c0b2ab0 0x7f3a3c0d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a3a59c640 1 -- 172.21.15.38:0/3428987154 >> v1:172.21.15.38:6789/0 conn(0x7f3a3c0b9180 legacy=0x7f3a3c0d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a3a59c640 1 -- 172.21.15.38:0/3428987154 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c1616a0 con 0x7f3a3c160ef0 2024-04-25T08:32:26.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a39d9b640 1 -- 172.21.15.38:0/3428987154 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644591345 0 0) 0x7f3a3c0b9610 con 0x7f3a3c0b9180 2024-04-25T08:32:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.649+0000 7f3a3ad9d640 1 --2- 172.21.15.38:0/3428987154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a3c0b2ab0 0x7f3a3c0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:26.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.650+0000 7f3a3a59c640 1 --2- 172.21.15.38:0/3428987154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a3c160ef0 0x7f3a3c0d07b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0097a0 tx=0x7f3a2c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a005ee054e4bce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.650+0000 7f3a39d9b640 1 -- 172.21.15.38:0/3428987154 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c037070 con 0x7f3a3c160ef0 2024-04-25T08:32:26.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.650+0000 7f3a39d9b640 1 -- 172.21.15.38:0/3428987154 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2c02ae50 con 0x7f3a3c160ef0 2024-04-25T08:32:26.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.650+0000 7f3a39d9b640 1 -- 172.21.15.38:0/3428987154 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c032660 con 0x7f3a3c160ef0 2024-04-25T08:32:26.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.651+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3428987154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a3c160ef0 msgr2=0x7f3a3c0d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.651+0000 7f3a40e3f640 1 --2- 172.21.15.38:0/3428987154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a3c160ef0 0x7f3a3c0d07b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c0097a0 tx=0x7f3a2c02a100 comp rx=0 tx=0).stop 2024-04-25T08:32:26.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.651+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3428987154 shutdown_connections 2024-04-25T08:32:26.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.651+0000 7f3a40e3f640 1 --2- 172.21.15.38:0/3428987154 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a3c0b2ab0 0x7f3a3c0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.651+0000 7f3a40e3f640 1 --2- 172.21.15.38:0/3428987154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a3c160ef0 0x7f3a3c0d07b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.651+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3428987154 >> 172.21.15.38:0/3428987154 conn(0x7f3a3c066d10 msgr2=0x7f3a3c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.651+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3428987154 shutdown_connections 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.651+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3428987154 wait complete. 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.652+0000 7f3a40e3f640 1 Processor -- start 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.652+0000 7f3a40e3f640 1 -- start start 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a40e3f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a3c0b2ab0 0x7f3a3c1580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a40e3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a3c160ef0 0x7f3a3c0cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a40e3f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a3c15cd50 0x7f3a3c15d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a40e3f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c162300 con 0x7f3a3c160ef0 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a40e3f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c162180 con 0x7f3a3c15cd50 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a40e3f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c162480 con 0x7f3a3c0b2ab0 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a33fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a3c160ef0 0x7f3a3c0cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a33fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a3c160ef0 0x7f3a3c0cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48926/0 (socket says 172.21.15.38:48926) 2024-04-25T08:32:26.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a33fff640 1 -- 172.21.15.38:0/3746815221 learned_addr learned my addr 172.21.15.38:0/3746815221 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a3a59c640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a3c0b2ab0 0x7f3a3c1580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a3ad9d640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a3c15cd50 0x7f3a3c15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a33fff640 1 -- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a3c0b2ab0 msgr2=0x7f3a3c1580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a33fff640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a3c0b2ab0 0x7f3a3c1580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a33fff640 1 -- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a3c15cd50 msgr2=0x7f3a3c15d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a33fff640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a3c15cd50 0x7f3a3c15d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a33fff640 1 -- 172.21.15.38:0/3746815221 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c0cc9d0 con 0x7f3a3c160ef0 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.653+0000 7f3a33fff640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a3c160ef0 0x7f3a3c0cbf50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c500 tx=0x7f3a2400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.654+0000 7f3a32ffd640 1 -- 172.21.15.38:0/3746815221 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24018070 con 0x7f3a3c160ef0 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.654+0000 7f3a32ffd640 1 -- 172.21.15.38:0/3746815221 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a24007580 con 0x7f3a3c160ef0 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.654+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3746815221 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c0cccb0 con 0x7f3a3c160ef0 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.654+0000 7f3a32ffd640 1 -- 172.21.15.38:0/3746815221 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a24004a20 con 0x7f3a3c160ef0 2024-04-25T08:32:26.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.654+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3746815221 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3c0cd270 con 0x7f3a3c160ef0 2024-04-25T08:32:26.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.654+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3746815221 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3c04ef90 con 0x7f3a3c160ef0 2024-04-25T08:32:26.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.655+0000 7f3a32ffd640 1 -- 172.21.15.38:0/3746815221 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a2401b070 con 0x7f3a3c160ef0 2024-04-25T08:32:26.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.655+0000 7f3a32ffd640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:26.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.656+0000 7f3a32ffd640 1 -- 172.21.15.38:0/3746815221 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f3a24089830 con 0x7f3a3c160ef0 2024-04-25T08:32:26.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.656+0000 7f3a3a59c640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:26.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.656+0000 7f3a3a59c640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c034000 tx=0x7f3a2c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:26.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.660+0000 7f3a32ffd640 1 -- 172.21.15.38:0/3746815221 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a24057c90 con 0x7f3a3c160ef0 2024-04-25T08:32:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.851+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3746815221 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3a3c06ae50 con 0x7f3a3c160ef0 2024-04-25T08:32:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.851+0000 7f3a32ffd640 1 -- 172.21.15.38:0/3746815221 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3a2405bd30 con 0x7f3a3c160ef0 2024-04-25T08:32:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:32:26.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.853+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a1c06b4d0 msgr2=0x7f3a1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.853+0000 7f3a40e3f640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c034000 tx=0x7f3a2c02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:26.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.853+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3746815221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a3c160ef0 msgr2=0x7f3a3c0cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.853+0000 7f3a40e3f640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a3c160ef0 0x7f3a3c0cbf50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3a2400c500 tx=0x7f3a2400c9c0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.853+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3746815221 shutdown_connections 2024-04-25T08:32:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.853+0000 7f3a40e3f640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a3c15cd50 0x7f3a3c15d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.853+0000 7f3a40e3f640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a3c160ef0 0x7f3a3c0cbf50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.853+0000 7f3a40e3f640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a1c06b4d0 0x7f3a1c06d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.853+0000 7f3a40e3f640 1 --2- 172.21.15.38:0/3746815221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a3c0b2ab0 0x7f3a3c1580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.853+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3746815221 >> 172.21.15.38:0/3746815221 conn(0x7f3a3c066d10 msgr2=0x7f3a3c0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.853+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3746815221 shutdown_connections 2024-04-25T08:32:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:26.853+0000 7f3a40e3f640 1 -- 172.21.15.38:0/3746815221 wait complete. 2024-04-25T08:32:26.860 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2024-04-25T08:32:26.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:26.860 DEBUG:teuthology.orchestra.run.smithi107:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T08:32:26.889 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:26.889 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:32:26.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:26.889 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:32:26.952 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:32:26.952 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:26.968 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:32:26.968 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:27.025 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:27.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.170+0000 7f6ae1be0640 1 Processor -- start 2024-04-25T08:32:27.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.170+0000 7f6ae1be0640 1 -- start start 2024-04-25T08:32:27.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.171+0000 7f6ae1be0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6adc15dc10 0x7f6adc0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.171+0000 7f6ae1be0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6adc0ced60 0x7f6adc0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.171+0000 7f6ae1be0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6adc0d3e70 con 0x7f6adc0d3900 2024-04-25T08:32:27.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.171+0000 7f6ae1be0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6adc15dff0 con 0x7f6adc0ced60 2024-04-25T08:32:27.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.171+0000 7f6ae1be0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6adc0b6990 con 0x7f6adc15dc10 2024-04-25T08:32:27.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.171+0000 7f6adaffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6adc0ced60 0x7f6adc0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.171+0000 7f6adaffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6adc0ced60 0x7f6adc0d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48928/0 (socket says 172.21.15.38:48928) 2024-04-25T08:32:27.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.171+0000 7f6adaffd640 1 -- 172.21.15.38:0/1204192885 learned_addr learned my addr 172.21.15.38:0/1204192885 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:27.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.171+0000 7f6adaffd640 1 -- 172.21.15.38:0/1204192885 >> v2:172.21.15.107:3300/0 conn(0x7f6adc15dc10 msgr2=0x7f6adc0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.171+0000 7f6adaffd640 1 --2- 172.21.15.38:0/1204192885 >> v2:172.21.15.107:3300/0 conn(0x7f6adc15dc10 0x7f6adc0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.171+0000 7f6adaffd640 1 -- 172.21.15.38:0/1204192885 >> v1:172.21.15.107:6789/0 conn(0x7f6adc0d3900 legacy=0x7f6adc0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.171+0000 7f6adaffd640 1 -- 172.21.15.38:0/1204192885 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc0d4150 con 0x7f6adc0ced60 2024-04-25T08:32:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.172+0000 7f6adaffd640 1 --2- 172.21.15.38:0/1204192885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6adc0ced60 0x7f6adc0d0850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0b5d50 tx=0x7f6ad002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137791c550288936 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.172+0000 7f6ada7fc640 1 -- 172.21.15.38:0/1204192885 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0037070 con 0x7f6adc0ced60 2024-04-25T08:32:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.172+0000 7f6ada7fc640 1 -- 172.21.15.38:0/1204192885 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad0032440 con 0x7f6adc0ced60 2024-04-25T08:32:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.172+0000 7f6ada7fc640 1 -- 172.21.15.38:0/1204192885 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0032810 con 0x7f6adc0ced60 2024-04-25T08:32:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.173+0000 7f6ae1be0640 1 -- 172.21.15.38:0/1204192885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6adc0ced60 msgr2=0x7f6adc0d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.173+0000 7f6ae1be0640 1 --2- 172.21.15.38:0/1204192885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6adc0ced60 0x7f6adc0d0850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6adc0b5d50 tx=0x7f6ad002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.173+0000 7f6ae1be0640 1 -- 172.21.15.38:0/1204192885 shutdown_connections 2024-04-25T08:32:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.173+0000 7f6ae1be0640 1 --2- 172.21.15.38:0/1204192885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6adc0ced60 0x7f6adc0d0850 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.173+0000 7f6ae1be0640 1 --2- 172.21.15.38:0/1204192885 >> v2:172.21.15.107:3300/0 conn(0x7f6adc15dc10 0x7f6adc0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.173+0000 7f6ae1be0640 1 -- 172.21.15.38:0/1204192885 >> 172.21.15.38:0/1204192885 conn(0x7f6adc066d10 msgr2=0x7f6adc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.173+0000 7f6ae1be0640 1 -- 172.21.15.38:0/1204192885 shutdown_connections 2024-04-25T08:32:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.173+0000 7f6ae1be0640 1 -- 172.21.15.38:0/1204192885 wait complete. 2024-04-25T08:32:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.174+0000 7f6ae1be0640 1 Processor -- start 2024-04-25T08:32:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.174+0000 7f6ae1be0640 1 -- start start 2024-04-25T08:32:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6ae1be0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6adc0ced60 0x7f6adc0c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6ae1be0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6adc0d3900 0x7f6adc1599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6ae1be0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6adc15dc10 0x7f6adc1e62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6ae1be0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0b6bc0 con 0x7f6adc0d3900 2024-04-25T08:32:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6ae1be0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0d4710 con 0x7f6adc0ced60 2024-04-25T08:32:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6ae1be0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc0d0d90 con 0x7f6adc15dc10 2024-04-25T08:32:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6adaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6adc0d3900 0x7f6adc1599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6adaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6adc0d3900 0x7f6adc1599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48932/0 (socket says 172.21.15.38:48932) 2024-04-25T08:32:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6adaffd640 1 -- 172.21.15.38:0/2790307918 learned_addr learned my addr 172.21.15.38:0/2790307918 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6adb7fe640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6adc0ced60 0x7f6adc0c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6adbfff640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6adc15dc10 0x7f6adc1e62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6adaffd640 1 -- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6adc15dc10 msgr2=0x7f6adc1e62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6adaffd640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6adc15dc10 0x7f6adc1e62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6adaffd640 1 -- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6adc0ced60 msgr2=0x7f6adc0c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6adaffd640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6adc0ced60 0x7f6adc0c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6adaffd640 1 -- 172.21.15.38:0/2790307918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6adc147c50 con 0x7f6adc0d3900 2024-04-25T08:32:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.175+0000 7f6adaffd640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6adc0d3900 0x7f6adc1599c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0000c00 tx=0x7f6ad00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.176+0000 7f6ac3fff640 1 -- 172.21.15.38:0/2790307918 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0041070 con 0x7f6adc0d3900 2024-04-25T08:32:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.176+0000 7f6ac3fff640 1 -- 172.21.15.38:0/2790307918 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad003c870 con 0x7f6adc0d3900 2024-04-25T08:32:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.176+0000 7f6ae1be0640 1 -- 172.21.15.38:0/2790307918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc147ef0 con 0x7f6adc0d3900 2024-04-25T08:32:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.176+0000 7f6ac3fff640 1 -- 172.21.15.38:0/2790307918 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0037040 con 0x7f6adc0d3900 2024-04-25T08:32:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.176+0000 7f6ae1be0640 1 -- 172.21.15.38:0/2790307918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc1484b0 con 0x7f6adc0d3900 2024-04-25T08:32:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.177+0000 7f6ae1be0640 1 -- 172.21.15.38:0/2790307918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6adc068e80 con 0x7f6adc0d3900 2024-04-25T08:32:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.177+0000 7f6ac3fff640 1 -- 172.21.15.38:0/2790307918 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ad0044070 con 0x7f6adc0d3900 2024-04-25T08:32:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.178+0000 7f6ac3fff640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ac806b4d0 0x7f6ac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.178+0000 7f6ac3fff640 1 -- 172.21.15.38:0/2790307918 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f6ad00ab0c0 con 0x7f6adc0d3900 2024-04-25T08:32:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.178+0000 7f6adb7fe640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ac806b4d0 0x7f6ac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.178+0000 7f6adb7fe640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ac806b4d0 0x7f6ac806d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0098e0 tx=0x7f6acc009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.182+0000 7f6ac3fff640 1 -- 172.21.15.38:0/2790307918 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ad0079640 con 0x7f6adc0d3900 2024-04-25T08:32:27.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.362+0000 7f6ae1be0640 1 -- 172.21.15.38:0/2790307918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6adc0d1160 con 0x7f6adc0d3900 2024-04-25T08:32:27.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.363+0000 7f6ac3fff640 1 -- 172.21.15.38:0/2790307918 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18969 (secure 0 0 0) 0x7f6ad007d6e0 con 0x7f6adc0d3900 2024-04-25T08:32:27.359 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:27.360 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:16.572337+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.a","mappings":[{"from":4,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.365+0000 7f6ae1be0640 1 -- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ac806b4d0 msgr2=0x7f6ac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.365+0000 7f6ae1be0640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ac806b4d0 0x7f6ac806d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0098e0 tx=0x7f6acc009060 comp rx=0 tx=0).stop 2024-04-25T08:32:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.365+0000 7f6ae1be0640 1 -- 172.21.15.38:0/2790307918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6adc0d3900 msgr2=0x7f6adc1599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.365+0000 7f6ae1be0640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6adc0d3900 0x7f6adc1599c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6ad0000c00 tx=0x7f6ad00046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.365+0000 7f6ae1be0640 1 -- 172.21.15.38:0/2790307918 shutdown_connections 2024-04-25T08:32:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.365+0000 7f6ae1be0640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ac806b4d0 0x7f6ac806d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.365+0000 7f6ae1be0640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6adc15dc10 0x7f6adc1e62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.365+0000 7f6ae1be0640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6adc0d3900 0x7f6adc1599c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.365+0000 7f6ae1be0640 1 --2- 172.21.15.38:0/2790307918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6adc0ced60 0x7f6adc0c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.365+0000 7f6ae1be0640 1 -- 172.21.15.38:0/2790307918 >> 172.21.15.38:0/2790307918 conn(0x7f6adc066d10 msgr2=0x7f6adc15fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.365+0000 7f6ae1be0640 1 -- 172.21.15.38:0/2790307918 shutdown_connections 2024-04-25T08:32:27.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.365+0000 7f6ae1be0640 1 -- 172.21.15.38:0/2790307918 wait complete. 2024-04-25T08:32:27.372 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:27.373 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.480+0000 7fd14ee6f640 1 Processor -- start 2024-04-25T08:32:27.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.480+0000 7fd14ee6f640 1 -- start start 2024-04-25T08:32:27.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.481+0000 7fd14ee6f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd14806c700 0x7fd148162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.481+0000 7fd14ee6f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14815ec20 con 0x7fd14806cae0 2024-04-25T08:32:27.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.481+0000 7fd14ee6f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14815ef00 con 0x7fd14815e750 2024-04-25T08:32:27.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.481+0000 7fd14ee6f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd148161c80 con 0x7fd14806c700 2024-04-25T08:32:27.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.481+0000 7fd147fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd14806cae0 0x7fd14815e040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34996/0 (socket says 172.21.15.38:34996) 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.481+0000 7fd147fff640 1 -- 172.21.15.38:0/1629864843 learned_addr learned my addr 172.21.15.38:0/1629864843 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd14cc0b640 1 --2- 172.21.15.38:0/1629864843 >> v2:172.21.15.107:3300/0 conn(0x7fd14806c700 0x7fd148162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431946087 0 0) 0x7fd14815ec20 con 0x7fd14806cae0 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd134003520 con 0x7fd14806cae0 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1317663813 0 0) 0x7fd134003520 con 0x7fd14806cae0 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 >> v2:172.21.15.107:3300/0 conn(0x7fd14806c700 msgr2=0x7fd148162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 --2- 172.21.15.38:0/1629864843 >> v2:172.21.15.107:3300/0 conn(0x7fd14806c700 0x7fd148162110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 >> v1:172.21.15.107:6789/0 conn(0x7fd14815e750 legacy=0x7fd1480cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd148161630 con 0x7fd14806cae0 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fd130001670 con 0x7fd14806cae0 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 >> v1:172.21.15.38:6789/0 conn(0x7fd14806cae0 legacy=0x7fd14815e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 --2- 172.21.15.38:0/1629864843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340039b0 0x7fd134003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 --2- 172.21.15.38:0/1629864843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd134004800 0x7fd134004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 --2- 172.21.15.38:0/1629864843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1340051c0 0x7fd13400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd148161c80 con 0x7fd1340051c0 2024-04-25T08:32:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13400c060 con 0x7fd134004800 2024-04-25T08:32:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13400c260 con 0x7fd1340039b0 2024-04-25T08:32:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd14d40c640 1 --2- 172.21.15.38:0/1629864843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1340051c0 0x7fd13400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd147fff640 1 --2- 172.21.15.38:0/1629864843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd134004800 0x7fd134004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd14cc0b640 1 --2- 172.21.15.38:0/1629864843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340039b0 0x7fd134003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd14d40c640 1 -- 172.21.15.38:0/1629864843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340039b0 msgr2=0x7fd134003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd14d40c640 1 --2- 172.21.15.38:0/1629864843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340039b0 0x7fd134003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd14d40c640 1 -- 172.21.15.38:0/1629864843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd134004800 msgr2=0x7fd134004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd14d40c640 1 --2- 172.21.15.38:0/1629864843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd134004800 0x7fd134004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.482+0000 7fd14d40c640 1 -- 172.21.15.38:0/1629864843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd134003520 con 0x7fd1340051c0 2024-04-25T08:32:27.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.483+0000 7fd14d40c640 1 -- 172.21.15.38:0/1629864843 reap_dead start 2024-04-25T08:32:27.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.483+0000 7fd14d40c640 1 --2- 172.21.15.38:0/1629864843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1340051c0 0x7fd13400b9c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd138001720 tx=0x7fd134005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb559d31763398c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.483+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138001370 con 0x7fd1340051c0 2024-04-25T08:32:27.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.483+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1380052d0 con 0x7fd1340051c0 2024-04-25T08:32:27.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.483+0000 7fd1477fe640 1 -- 172.21.15.38:0/1629864843 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13802aba0 con 0x7fd1340051c0 2024-04-25T08:32:27.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.484+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1629864843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1340051c0 msgr2=0x7fd13400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.484+0000 7fd14ee6f640 1 --2- 172.21.15.38:0/1629864843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1340051c0 0x7fd13400b9c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fd138001720 tx=0x7fd134005830 comp rx=0 tx=0).stop 2024-04-25T08:32:27.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.484+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1629864843 shutdown_connections 2024-04-25T08:32:27.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.484+0000 7fd14ee6f640 1 --2- 172.21.15.38:0/1629864843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1340051c0 0x7fd13400b9c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.484+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1629864843 >> 172.21.15.38:0/1629864843 conn(0x7fd14808b000 msgr2=0x7fd14808b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.484+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1629864843 shutdown_connections 2024-04-25T08:32:27.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.484+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1629864843 wait complete. 2024-04-25T08:32:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.485+0000 7fd14ee6f640 1 Processor -- start 2024-04-25T08:32:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.485+0000 7fd14ee6f640 1 -- start start 2024-04-25T08:32:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14ee6f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340051c0 0x7fd1481f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14ee6f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1480aec00 0x7fd1480b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14ee6f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd148149950 0x7fd148149db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14ee6f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14806cf90 con 0x7fd148149950 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14ee6f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14806c190 con 0x7fd1480aec00 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14ee6f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14815b920 con 0x7fd1340051c0 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14d40c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd148149950 0x7fd148149db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14d40c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd148149950 0x7fd148149db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48946/0 (socket says 172.21.15.38:48946) 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14d40c640 1 -- 172.21.15.38:0/1159931063 learned_addr learned my addr 172.21.15.38:0/1159931063 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14cc0b640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340051c0 0x7fd1481f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd147fff640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1480aec00 0x7fd1480b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14d40c640 1 -- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340051c0 msgr2=0x7fd1481f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14d40c640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340051c0 0x7fd1481f66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14d40c640 1 -- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1480aec00 msgr2=0x7fd1480b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14d40c640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1480aec00 0x7fd1480b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14d40c640 1 -- 172.21.15.38:0/1159931063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1481f6dc0 con 0x7fd148149950 2024-04-25T08:32:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd14cc0b640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340051c0 0x7fd1481f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.486+0000 7fd147fff640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1480aec00 0x7fd1480b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.487+0000 7fd14d40c640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd148149950 0x7fd148149db0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd13802a570 tx=0x7fd138001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.487+0000 7fd1457fa640 1 -- 172.21.15.38:0/1159931063 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138007a30 con 0x7fd148149950 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.487+0000 7fd1457fa640 1 -- 172.21.15.38:0/1159931063 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd138000ed0 con 0x7fd148149950 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.487+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1159931063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1481f70a0 con 0x7fd148149950 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.487+0000 7fd1457fa640 1 -- 172.21.15.38:0/1159931063 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138005480 con 0x7fd148149950 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.487+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1159931063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1480b0d70 con 0x7fd148149950 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.488+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1159931063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd148068e80 con 0x7fd148149950 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.488+0000 7fd1457fa640 1 -- 172.21.15.38:0/1159931063 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd138043020 con 0x7fd148149950 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.489+0000 7fd1457fa640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd12806b500 0x7fd12806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.489+0000 7fd1457fa640 1 -- 172.21.15.38:0/1159931063 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fd13802ae00 con 0x7fd148149950 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.489+0000 7fd14cc0b640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd12806b500 0x7fd12806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.489+0000 7fd14cc0b640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd12806b500 0x7fd12806d9c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd140007440 tx=0x7fd14000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.493+0000 7fd1457fa640 1 -- 172.21.15.38:0/1159931063 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd138077e70 con 0x7fd148149950 2024-04-25T08:32:27.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.672+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1159931063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1480b1220 con 0x7fd148149950 2024-04-25T08:32:27.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.673+0000 7fd1457fa640 1 -- 172.21.15.38:0/1159931063 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd138077770 con 0x7fd148149950 2024-04-25T08:32:27.669 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:27.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.675+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd12806b500 msgr2=0x7fd12806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.675+0000 7fd14ee6f640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd12806b500 0x7fd12806d9c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd140007440 tx=0x7fd14000b040 comp rx=0 tx=0).stop 2024-04-25T08:32:27.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.675+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1159931063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd148149950 msgr2=0x7fd148149db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.675+0000 7fd14ee6f640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd148149950 0x7fd148149db0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd13802a570 tx=0x7fd138001120 comp rx=0 tx=0).stop 2024-04-25T08:32:27.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.675+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1159931063 shutdown_connections 2024-04-25T08:32:27.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.675+0000 7fd14ee6f640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd148149950 0x7fd148149db0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.675+0000 7fd14ee6f640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1480aec00 0x7fd1480b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.675+0000 7fd14ee6f640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd12806b500 0x7fd12806d9c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.675+0000 7fd14ee6f640 1 --2- 172.21.15.38:0/1159931063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340051c0 0x7fd1481f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.675+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1159931063 >> 172.21.15.38:0/1159931063 conn(0x7fd14808b000 msgr2=0x7fd14808b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.675+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1159931063 shutdown_connections 2024-04-25T08:32:27.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.675+0000 7fd14ee6f640 1 -- 172.21.15.38:0/1159931063 wait complete. 2024-04-25T08:32:27.682 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:27.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.789+0000 7fd5ad46b640 1 Processor -- start 2024-04-25T08:32:27.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.789+0000 7fd5ad46b640 1 -- start start 2024-04-25T08:32:27.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5ad46b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd5a80d0320 0x7fd5a80d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5ad46b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd5a80d0700 0x7fd5a80d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5ad46b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a80d2330 con 0x7fd5a80d1e90 2024-04-25T08:32:27.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5ad46b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a80d2610 con 0x7fd5a80d0320 2024-04-25T08:32:27.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5ad46b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a80d28f0 con 0x7fd5a80d0700 2024-04-25T08:32:27.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5a6ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd5a80d0320 0x7fd5a80d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5a77fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd5a80d1e90 0x7fd5a80b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35002/0 (socket says 172.21.15.38:35002) 2024-04-25T08:32:27.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5a77fe640 1 -- 172.21.15.38:0/3028764237 learned_addr learned my addr 172.21.15.38:0/3028764237 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:27.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5a67fc640 1 --2- 172.21.15.38:0/3028764237 >> v2:172.21.15.107:3300/0 conn(0x7fd5a80d0700 0x7fd5a80d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5a6ffd640 1 -- 172.21.15.38:0/3028764237 >> v2:172.21.15.107:3300/0 conn(0x7fd5a80d0700 msgr2=0x7fd5a80d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:27.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5a6ffd640 1 --2- 172.21.15.38:0/3028764237 >> v2:172.21.15.107:3300/0 conn(0x7fd5a80d0700 0x7fd5a80d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5a6ffd640 1 -- 172.21.15.38:0/3028764237 >> v1:172.21.15.38:6789/0 conn(0x7fd5a80d1e90 legacy=0x7fd5a80b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5a5ffb640 1 -- 172.21.15.38:0/3028764237 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977261738 0 0) 0x7fd5a80d2330 con 0x7fd5a80d1e90 2024-04-25T08:32:27.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5a6ffd640 1 -- 172.21.15.38:0/3028764237 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a80d2bd0 con 0x7fd5a80d0320 2024-04-25T08:32:27.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.790+0000 7fd5a67fc640 1 --2- 172.21.15.38:0/3028764237 >> v2:172.21.15.107:3300/0 conn(0x7fd5a80d0700 0x7fd5a80d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:27.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.791+0000 7fd5a6ffd640 1 --2- 172.21.15.38:0/3028764237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a80d0320 0x7fd5a80d33d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80b6410 tx=0x7fd59802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d8971e87f4158d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.791+0000 7fd5a5ffb640 1 -- 172.21.15.38:0/3028764237 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598037070 con 0x7fd5a80d0320 2024-04-25T08:32:27.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.791+0000 7fd5a5ffb640 1 -- 172.21.15.38:0/3028764237 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd598032440 con 0x7fd5a80d0320 2024-04-25T08:32:27.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.791+0000 7fd5a5ffb640 1 -- 172.21.15.38:0/3028764237 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598032810 con 0x7fd5a80d0320 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.792+0000 7fd5ad46b640 1 -- 172.21.15.38:0/3028764237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a80d0320 msgr2=0x7fd5a80d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.792+0000 7fd5ad46b640 1 --2- 172.21.15.38:0/3028764237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a80d0320 0x7fd5a80d33d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80b6410 tx=0x7fd59802a230 comp rx=0 tx=0).stop 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.792+0000 7fd5ad46b640 1 -- 172.21.15.38:0/3028764237 shutdown_connections 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.792+0000 7fd5ad46b640 1 --2- 172.21.15.38:0/3028764237 >> v2:172.21.15.107:3300/0 conn(0x7fd5a80d0700 0x7fd5a80d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.792+0000 7fd5ad46b640 1 --2- 172.21.15.38:0/3028764237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a80d0320 0x7fd5a80d33d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.792+0000 7fd5ad46b640 1 -- 172.21.15.38:0/3028764237 >> 172.21.15.38:0/3028764237 conn(0x7fd5a80668f0 msgr2=0x7fd5a8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.792+0000 7fd5ad46b640 1 -- 172.21.15.38:0/3028764237 shutdown_connections 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.792+0000 7fd5ad46b640 1 -- 172.21.15.38:0/3028764237 wait complete. 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.793+0000 7fd5ad46b640 1 Processor -- start 2024-04-25T08:32:27.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.793+0000 7fd5ad46b640 1 -- start start 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5ad46b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a80d0320 0x7fd5a8157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5ad46b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a80d0700 0x7fd5a8153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5ad46b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a80d1e90 0x7fd5a815ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5ad46b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a80d0b40 con 0x7fd5a80d0320 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5ad46b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a8161210 con 0x7fd5a80d0700 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5ad46b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a8156650 con 0x7fd5a80d1e90 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5a6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a80d0320 0x7fd5a8157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5a6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a80d0320 0x7fd5a8157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48964/0 (socket says 172.21.15.38:48964) 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5a6ffd640 1 -- 172.21.15.38:0/1802856416 learned_addr learned my addr 172.21.15.38:0/1802856416 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5a67fc640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a80d0700 0x7fd5a8153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5a77fe640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a80d1e90 0x7fd5a815ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5a6ffd640 1 -- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a80d1e90 msgr2=0x7fd5a815ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5a6ffd640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a80d1e90 0x7fd5a815ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5a6ffd640 1 -- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a80d0700 msgr2=0x7fd5a8153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5a6ffd640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a80d0700 0x7fd5a8153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5a6ffd640 1 -- 172.21.15.38:0/1802856416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a80b41d0 con 0x7fd5a80d0320 2024-04-25T08:32:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.794+0000 7fd5a6ffd640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a80d0320 0x7fd5a8157380 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd598000c00 tx=0x7fd5980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.795+0000 7fd58f7fe640 1 -- 172.21.15.38:0/1802856416 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598037050 con 0x7fd5a80d0320 2024-04-25T08:32:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.795+0000 7fd58f7fe640 1 -- 172.21.15.38:0/1802856416 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd598038040 con 0x7fd5a80d0320 2024-04-25T08:32:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.795+0000 7fd5ad46b640 1 -- 172.21.15.38:0/1802856416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a81568a0 con 0x7fd5a80d0320 2024-04-25T08:32:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.795+0000 7fd58f7fe640 1 -- 172.21.15.38:0/1802856416 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598004450 con 0x7fd5a80d0320 2024-04-25T08:32:27.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.795+0000 7fd5ad46b640 1 -- 172.21.15.38:0/1802856416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a815ef40 con 0x7fd5a80d0320 2024-04-25T08:32:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.796+0000 7fd5ad46b640 1 -- 172.21.15.38:0/1802856416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a80d4170 con 0x7fd5a80d0320 2024-04-25T08:32:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.796+0000 7fd58f7fe640 1 -- 172.21.15.38:0/1802856416 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd598030980 con 0x7fd5a80d0320 2024-04-25T08:32:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.796+0000 7fd58f7fe640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd59406b4d0 0x7fd59406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.797+0000 7fd58f7fe640 1 -- 172.21.15.38:0/1802856416 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fd5980ab0c0 con 0x7fd5a80d0320 2024-04-25T08:32:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.797+0000 7fd5a67fc640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd59406b4d0 0x7fd59406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:27.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.797+0000 7fd5a67fc640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd59406b4d0 0x7fd59406d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd5a81565e0 tx=0x7fd59c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:27.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.801+0000 7fd58f7fe640 1 -- 172.21.15.38:0/1802856416 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5a80d4170 con 0x7fd5a80d0320 2024-04-25T08:32:27.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.982+0000 7fd5ad46b640 1 -- 172.21.15.38:0/1802856416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5a8068e80 con 0x7fd5a80d0320 2024-04-25T08:32:27.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.982+0000 7fd58f7fe640 1 -- 172.21.15.38:0/1802856416 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd598079490 con 0x7fd5a80d0320 2024-04-25T08:32:27.978 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:27.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.984+0000 7fd5ad46b640 1 -- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd59406b4d0 msgr2=0x7fd59406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.984+0000 7fd5ad46b640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd59406b4d0 0x7fd59406d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd5a81565e0 tx=0x7fd59c008040 comp rx=0 tx=0).stop 2024-04-25T08:32:27.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.984+0000 7fd5ad46b640 1 -- 172.21.15.38:0/1802856416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a80d0320 msgr2=0x7fd5a8157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:27.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.984+0000 7fd5ad46b640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a80d0320 0x7fd5a8157380 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd598000c00 tx=0x7fd5980046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.984+0000 7fd5ad46b640 1 -- 172.21.15.38:0/1802856416 shutdown_connections 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.984+0000 7fd5ad46b640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd59406b4d0 0x7fd59406d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.984+0000 7fd5ad46b640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a80d1e90 0x7fd5a815ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.984+0000 7fd5ad46b640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a80d0700 0x7fd5a8153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.984+0000 7fd5ad46b640 1 --2- 172.21.15.38:0/1802856416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a80d0320 0x7fd5a8157380 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.984+0000 7fd5ad46b640 1 -- 172.21.15.38:0/1802856416 >> 172.21.15.38:0/1802856416 conn(0x7fd5a80668f0 msgr2=0x7fd5a80ce850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.984+0000 7fd5ad46b640 1 -- 172.21.15.38:0/1802856416 shutdown_connections 2024-04-25T08:32:27.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:27.984+0000 7fd5ad46b640 1 -- 172.21.15.38:0/1802856416 wait complete. 2024-04-25T08:32:27.991 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:28.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.098+0000 7fed175ce640 1 Processor -- start 2024-04-25T08:32:28.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.098+0000 7fed175ce640 1 -- start start 2024-04-25T08:32:28.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.099+0000 7fed175ce640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fed1009a180 0x7fed1009a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.099+0000 7fed175ce640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fed10162390 0x7fed100cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.099+0000 7fed175ce640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1015f300 con 0x7fed1015df30 2024-04-25T08:32:28.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.099+0000 7fed175ce640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1015e310 con 0x7fed10162390 2024-04-25T08:32:28.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.099+0000 7fed175ce640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1009aeb0 con 0x7fed1009a180 2024-04-25T08:32:28.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.099+0000 7fed15b6b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fed10162390 0x7fed100cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.099+0000 7fed1536a640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fed1015df30 0x7fed10099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35006/0 (socket says 172.21.15.38:35006) 2024-04-25T08:32:28.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.099+0000 7fed1536a640 1 -- 172.21.15.38:0/1416356151 learned_addr learned my addr 172.21.15.38:0/1416356151 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:28.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.099+0000 7fed14b69640 1 --2- 172.21.15.38:0/1416356151 >> v2:172.21.15.107:3300/0 conn(0x7fed1009a180 0x7fed1009a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.100+0000 7fed15b6b640 1 -- 172.21.15.38:0/1416356151 >> v2:172.21.15.107:3300/0 conn(0x7fed1009a180 msgr2=0x7fed1009a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:28.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.100+0000 7fed15b6b640 1 --2- 172.21.15.38:0/1416356151 >> v2:172.21.15.107:3300/0 conn(0x7fed1009a180 0x7fed1009a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.100+0000 7fed15b6b640 1 -- 172.21.15.38:0/1416356151 >> v1:172.21.15.38:6789/0 conn(0x7fed1015df30 legacy=0x7fed10099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.100+0000 7fed0ffff640 1 -- 172.21.15.38:0/1416356151 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155989901 0 0) 0x7fed1015f300 con 0x7fed1015df30 2024-04-25T08:32:28.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.100+0000 7fed15b6b640 1 -- 172.21.15.38:0/1416356151 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1015e540 con 0x7fed10162390 2024-04-25T08:32:28.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.100+0000 7fed15b6b640 1 --2- 172.21.15.38:0/1416356151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10162390 0x7fed100cf8f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fed10072410 tx=0x7fed0002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33de3d41b7cb00a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:28.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.100+0000 7fed0ffff640 1 -- 172.21.15.38:0/1416356151 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00007740 con 0x7fed10162390 2024-04-25T08:32:28.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.100+0000 7fed0ffff640 1 -- 172.21.15.38:0/1416356151 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed000078e0 con 0x7fed10162390 2024-04-25T08:32:28.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.100+0000 7fed0ffff640 1 -- 172.21.15.38:0/1416356151 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00007cb0 con 0x7fed10162390 2024-04-25T08:32:28.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.101+0000 7fed175ce640 1 -- 172.21.15.38:0/1416356151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10162390 msgr2=0x7fed100cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.101+0000 7fed175ce640 1 --2- 172.21.15.38:0/1416356151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10162390 0x7fed100cf8f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fed10072410 tx=0x7fed0002a680 comp rx=0 tx=0).stop 2024-04-25T08:32:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.101+0000 7fed175ce640 1 -- 172.21.15.38:0/1416356151 shutdown_connections 2024-04-25T08:32:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.101+0000 7fed175ce640 1 --2- 172.21.15.38:0/1416356151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed10162390 0x7fed100cf8f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.101+0000 7fed175ce640 1 --2- 172.21.15.38:0/1416356151 >> v2:172.21.15.107:3300/0 conn(0x7fed1009a180 0x7fed1009a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.101+0000 7fed175ce640 1 -- 172.21.15.38:0/1416356151 >> 172.21.15.38:0/1416356151 conn(0x7fed100b3110 msgr2=0x7fed100b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.101+0000 7fed175ce640 1 -- 172.21.15.38:0/1416356151 shutdown_connections 2024-04-25T08:32:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.101+0000 7fed175ce640 1 -- 172.21.15.38:0/1416356151 wait complete. 2024-04-25T08:32:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.102+0000 7fed175ce640 1 Processor -- start 2024-04-25T08:32:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.102+0000 7fed175ce640 1 -- start start 2024-04-25T08:32:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.103+0000 7fed175ce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed1009a180 0x7fed10153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.103+0000 7fed175ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed1015df30 0x7fed100b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.103+0000 7fed175ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed10162390 0x7fed10159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.103+0000 7fed175ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1015eb00 con 0x7fed1015df30 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.103+0000 7fed175ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1015ee70 con 0x7fed10162390 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.103+0000 7fed175ce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed101627a0 con 0x7fed1009a180 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.103+0000 7fed14b69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed1015df30 0x7fed100b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.103+0000 7fed14b69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed1015df30 0x7fed100b4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48974/0 (socket says 172.21.15.38:48974) 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.103+0000 7fed14b69640 1 -- 172.21.15.38:0/2712741968 learned_addr learned my addr 172.21.15.38:0/2712741968 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.103+0000 7fed15b6b640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed10162390 0x7fed10159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.103+0000 7fed1536a640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed1009a180 0x7fed10153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.104+0000 7fed14b69640 1 -- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed1009a180 msgr2=0x7fed10153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.104+0000 7fed14b69640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed1009a180 0x7fed10153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.104+0000 7fed14b69640 1 -- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed10162390 msgr2=0x7fed10159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.104+0000 7fed14b69640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed10162390 0x7fed10159360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.104+0000 7fed14b69640 1 -- 172.21.15.38:0/2712741968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed100c54c0 con 0x7fed1015df30 2024-04-25T08:32:28.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.104+0000 7fed14b69640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed1015df30 0x7fed100b4520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fed0400d5f0 tx=0x7fed0400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.104+0000 7fed0dffb640 1 -- 172.21.15.38:0/2712741968 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed040125f0 con 0x7fed1015df30 2024-04-25T08:32:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.104+0000 7fed0dffb640 1 -- 172.21.15.38:0/2712741968 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed04019070 con 0x7fed1015df30 2024-04-25T08:32:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.104+0000 7fed175ce640 1 -- 172.21.15.38:0/2712741968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed100c57a0 con 0x7fed1015df30 2024-04-25T08:32:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.104+0000 7fed0dffb640 1 -- 172.21.15.38:0/2712741968 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed040129d0 con 0x7fed1015df30 2024-04-25T08:32:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.104+0000 7fed175ce640 1 -- 172.21.15.38:0/2712741968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed100c6040 con 0x7fed1015df30 2024-04-25T08:32:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.105+0000 7fed175ce640 1 -- 172.21.15.38:0/2712741968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed100c5970 con 0x7fed1015df30 2024-04-25T08:32:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.105+0000 7fed0dffb640 1 -- 172.21.15.38:0/2712741968 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed04012b70 con 0x7fed1015df30 2024-04-25T08:32:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.106+0000 7fed0dffb640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecfc06b500 0x7fecfc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.106+0000 7fed0dffb640 1 -- 172.21.15.38:0/2712741968 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fed0408a840 con 0x7fed1015df30 2024-04-25T08:32:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.106+0000 7fed1536a640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecfc06b500 0x7fecfc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.107+0000 7fed1536a640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecfc06b500 0x7fecfc06d9c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fed08000f60 tx=0x7fed08007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:28.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.110+0000 7fed0dffb640 1 -- 172.21.15.38:0/2712741968 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed04058ca0 con 0x7fed1015df30 2024-04-25T08:32:28.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.294+0000 7fed175ce640 1 -- 172.21.15.38:0/2712741968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed10068e80 con 0x7fed1015df30 2024-04-25T08:32:28.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.294+0000 7fed0dffb640 1 -- 172.21.15.38:0/2712741968 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fed0405cd40 con 0x7fed1015df30 2024-04-25T08:32:28.290 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:28.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.296+0000 7fed175ce640 1 -- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecfc06b500 msgr2=0x7fecfc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.296+0000 7fed175ce640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecfc06b500 0x7fecfc06d9c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fed08000f60 tx=0x7fed08007d50 comp rx=0 tx=0).stop 2024-04-25T08:32:28.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.296+0000 7fed175ce640 1 -- 172.21.15.38:0/2712741968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed1015df30 msgr2=0x7fed100b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.296+0000 7fed175ce640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed1015df30 0x7fed100b4520 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fed0400d5f0 tx=0x7fed0400dab0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.296+0000 7fed175ce640 1 -- 172.21.15.38:0/2712741968 shutdown_connections 2024-04-25T08:32:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.296+0000 7fed175ce640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed10162390 0x7fed10159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.296+0000 7fed175ce640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed1015df30 0x7fed100b4520 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.296+0000 7fed175ce640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecfc06b500 0x7fecfc06d9c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.296+0000 7fed175ce640 1 --2- 172.21.15.38:0/2712741968 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed1009a180 0x7fed10153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.296+0000 7fed175ce640 1 -- 172.21.15.38:0/2712741968 >> 172.21.15.38:0/2712741968 conn(0x7fed100b3110 msgr2=0x7fed1006f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.296+0000 7fed175ce640 1 -- 172.21.15.38:0/2712741968 shutdown_connections 2024-04-25T08:32:28.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.296+0000 7fed175ce640 1 -- 172.21.15.38:0/2712741968 wait complete. 2024-04-25T08:32:28.303 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:28.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.412+0000 7f47c9fb0640 1 Processor -- start 2024-04-25T08:32:28.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.412+0000 7f47c9fb0640 1 -- start start 2024-04-25T08:32:28.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.413+0000 7f47c9fb0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47c4162110 0x7f47c415e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.413+0000 7f47c9fb0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f47c415e580 0x7f47c40cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.413+0000 7f47c9fb0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47c415ec20 con 0x7f47c406c700 2024-04-25T08:32:28.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.413+0000 7f47c9fb0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c4161c80 con 0x7f47c4162110 2024-04-25T08:32:28.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.413+0000 7f47c9fb0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47c41625c0 con 0x7f47c415e580 2024-04-25T08:32:28.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.413+0000 7f47c2ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47c4162110 0x7f47c415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.413+0000 7f47c2ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47c4162110 0x7f47c415e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48984/0 (socket says 172.21.15.38:48984) 2024-04-25T08:32:28.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.413+0000 7f47c2ffd640 1 -- 172.21.15.38:0/1787804208 learned_addr learned my addr 172.21.15.38:0/1787804208 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:28.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.414+0000 7f47c2ffd640 1 -- 172.21.15.38:0/1787804208 >> v2:172.21.15.107:3300/0 conn(0x7f47c415e580 msgr2=0x7f47c40cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:28.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.414+0000 7f47c3fff640 1 --2- 172.21.15.38:0/1787804208 >> v2:172.21.15.107:3300/0 conn(0x7f47c415e580 0x7f47c40cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.414+0000 7f47c2ffd640 1 --2- 172.21.15.38:0/1787804208 >> v2:172.21.15.107:3300/0 conn(0x7f47c415e580 0x7f47c40cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.414+0000 7f47c2ffd640 1 -- 172.21.15.38:0/1787804208 >> v1:172.21.15.107:6789/0 conn(0x7f47c406c700 legacy=0x7f47c406cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:28.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.414+0000 7f47c2ffd640 1 -- 172.21.15.38:0/1787804208 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c415ef00 con 0x7f47c4162110 2024-04-25T08:32:28.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.414+0000 7f47c3fff640 1 --2- 172.21.15.38:0/1787804208 >> v2:172.21.15.107:3300/0 conn(0x7f47c415e580 0x7f47c40cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:28.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.414+0000 7f47c2ffd640 1 --2- 172.21.15.38:0/1787804208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4162110 0x7f47c415e040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f47c40d2560 tx=0x7f47b802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f64bbb8bf63346 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:28.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.415+0000 7f47c27fc640 1 -- 172.21.15.38:0/1787804208 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8037070 con 0x7f47c4162110 2024-04-25T08:32:28.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.415+0000 7f47c27fc640 1 -- 172.21.15.38:0/1787804208 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b8032440 con 0x7f47c4162110 2024-04-25T08:32:28.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.415+0000 7f47c27fc640 1 -- 172.21.15.38:0/1787804208 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b8032810 con 0x7f47c4162110 2024-04-25T08:32:28.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.415+0000 7f47c9fb0640 1 -- 172.21.15.38:0/1787804208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4162110 msgr2=0x7f47c415e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.415+0000 7f47c9fb0640 1 --2- 172.21.15.38:0/1787804208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4162110 0x7f47c415e040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f47c40d2560 tx=0x7f47b802a230 comp rx=0 tx=0).stop 2024-04-25T08:32:28.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.415+0000 7f47c9fb0640 1 -- 172.21.15.38:0/1787804208 shutdown_connections 2024-04-25T08:32:28.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.415+0000 7f47c9fb0640 1 --2- 172.21.15.38:0/1787804208 >> v2:172.21.15.107:3300/0 conn(0x7f47c415e580 0x7f47c40cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.415+0000 7f47c9fb0640 1 --2- 172.21.15.38:0/1787804208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4162110 0x7f47c415e040 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.415+0000 7f47c9fb0640 1 -- 172.21.15.38:0/1787804208 >> 172.21.15.38:0/1787804208 conn(0x7f47c408b000 msgr2=0x7f47c408b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:28.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.415+0000 7f47c9fb0640 1 -- 172.21.15.38:0/1787804208 shutdown_connections 2024-04-25T08:32:28.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.415+0000 7f47c9fb0640 1 -- 172.21.15.38:0/1787804208 wait complete. 2024-04-25T08:32:28.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.416+0000 7f47c9fb0640 1 Processor -- start 2024-04-25T08:32:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.417+0000 7f47c9fb0640 1 -- start start 2024-04-25T08:32:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.417+0000 7f47c9fb0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c406c700 0x7f47c40b01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.417+0000 7f47c9fb0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47c415e580 0x7f47c4159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.417+0000 7f47c9fb0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47c4162110 0x7f47c40cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.417+0000 7f47c9fb0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c406c190 con 0x7f47c406c700 2024-04-25T08:32:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.417+0000 7f47c9fb0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c4161910 con 0x7f47c4162110 2024-04-25T08:32:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.417+0000 7f47c9fb0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c415b920 con 0x7f47c415e580 2024-04-25T08:32:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.417+0000 7f47c37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c406c700 0x7f47c40b01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.417+0000 7f47c37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c406c700 0x7f47c40b01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48994/0 (socket says 172.21.15.38:48994) 2024-04-25T08:32:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.417+0000 7f47c37fe640 1 -- 172.21.15.38:0/2354212368 learned_addr learned my addr 172.21.15.38:0/2354212368 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.418+0000 7f47c37fe640 1 -- 172.21.15.38:0/2354212368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47c415e580 msgr2=0x7f47c4159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.418+0000 7f47c37fe640 1 --2- 172.21.15.38:0/2354212368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47c415e580 0x7f47c4159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.418+0000 7f47c37fe640 1 -- 172.21.15.38:0/2354212368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47c4162110 msgr2=0x7f47c40cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.418+0000 7f47c37fe640 1 --2- 172.21.15.38:0/2354212368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47c4162110 0x7f47c40cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.418+0000 7f47c37fe640 1 -- 172.21.15.38:0/2354212368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c41596b0 con 0x7f47c406c700 2024-04-25T08:32:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.418+0000 7f47c37fe640 1 --2- 172.21.15.38:0/2354212368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c406c700 0x7f47c40b01a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f47b400b420 tx=0x7f47b400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.418+0000 7f47abfff640 1 -- 172.21.15.38:0/2354212368 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4011830 con 0x7f47c406c700 2024-04-25T08:32:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.418+0000 7f47abfff640 1 -- 172.21.15.38:0/2354212368 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b4016070 con 0x7f47c406c700 2024-04-25T08:32:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.418+0000 7f47abfff640 1 -- 172.21.15.38:0/2354212368 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b4011c10 con 0x7f47c406c700 2024-04-25T08:32:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.418+0000 7f47c9fb0640 1 -- 172.21.15.38:0/2354212368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c40b0840 con 0x7f47c406c700 2024-04-25T08:32:28.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.418+0000 7f47c9fb0640 1 -- 172.21.15.38:0/2354212368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c40b0e00 con 0x7f47c406c700 2024-04-25T08:32:28.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.419+0000 7f47c9fb0640 1 -- 172.21.15.38:0/2354212368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c4068e80 con 0x7f47c406c700 2024-04-25T08:32:28.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.420+0000 7f47abfff640 1 -- 172.21.15.38:0/2354212368 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47b401b460 con 0x7f47c406c700 2024-04-25T08:32:28.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.420+0000 7f47abfff640 1 --2- 172.21.15.38:0/2354212368 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47b006b4d0 0x7f47b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.420+0000 7f47abfff640 1 -- 172.21.15.38:0/2354212368 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f47b408a140 con 0x7f47c406c700 2024-04-25T08:32:28.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.420+0000 7f47c2ffd640 1 --2- 172.21.15.38:0/2354212368 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47b006b4d0 0x7f47b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.421+0000 7f47c2ffd640 1 --2- 172.21.15.38:0/2354212368 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47b006b4d0 0x7f47b006d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f47b8034000 tx=0x7f47b802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.424+0000 7f47abfff640 1 -- 172.21.15.38:0/2354212368 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47b4020020 con 0x7f47c406c700 2024-04-25T08:32:28.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.607+0000 7f47c9fb0640 1 -- 172.21.15.38:0/2354212368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f47c40b1220 con 0x7f47c406c700 2024-04-25T08:32:28.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.607+0000 7f47abfff640 1 -- 172.21.15.38:0/2354212368 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f47b4058bf0 con 0x7f47c406c700 2024-04-25T08:32:28.603 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:28.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.609+0000 7f47c9fb0640 1 -- 172.21.15.38:0/2354212368 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47b006b4d0 msgr2=0x7f47b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.609+0000 7f47c9fb0640 1 --2- 172.21.15.38:0/2354212368 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47b006b4d0 0x7f47b006d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f47b8034000 tx=0x7f47b802c000 comp rx=0 tx=0).stop 2024-04-25T08:32:28.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.609+0000 7f47c9fb0640 1 -- 172.21.15.38:0/2354212368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c406c700 msgr2=0x7f47c40b01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.609+0000 7f47c9fb0640 1 --2- 172.21.15.38:0/2354212368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c406c700 0x7f47c40b01a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f47b400b420 tx=0x7f47b400b8e0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.609+0000 7f47c9fb0640 1 -- 172.21.15.38:0/2354212368 shutdown_connections 2024-04-25T08:32:28.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.609+0000 7f47c9fb0640 1 --2- 172.21.15.38:0/2354212368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47c4162110 0x7f47c40cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.609+0000 7f47c9fb0640 1 --2- 172.21.15.38:0/2354212368 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47b006b4d0 0x7f47b006d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.609+0000 7f47c9fb0640 1 --2- 172.21.15.38:0/2354212368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47c415e580 0x7f47c4159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.609+0000 7f47c9fb0640 1 --2- 172.21.15.38:0/2354212368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c406c700 0x7f47c40b01a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.609+0000 7f47c9fb0640 1 -- 172.21.15.38:0/2354212368 >> 172.21.15.38:0/2354212368 conn(0x7f47c408b000 msgr2=0x7f47c406cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:28.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.609+0000 7f47c9fb0640 1 -- 172.21.15.38:0/2354212368 shutdown_connections 2024-04-25T08:32:28.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.609+0000 7f47c9fb0640 1 -- 172.21.15.38:0/2354212368 wait complete. 2024-04-25T08:32:28.617 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:32:28.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.722+0000 7ffbc9f5c640 1 Processor -- start 2024-04-25T08:32:28.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.723+0000 7ffbc9f5c640 1 -- start start 2024-04-25T08:32:28.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.723+0000 7ffbc9f5c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffbc40d0830 0x7ffbc4168a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.723+0000 7ffbc9f5c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc4162030 con 0x7ffbc40b9600 2024-04-25T08:32:28.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.723+0000 7ffbc9f5c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc4162310 con 0x7ffbc415db80 2024-04-25T08:32:28.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.723+0000 7ffbc9f5c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc4161df0 con 0x7ffbc40d0830 2024-04-25T08:32:28.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.723+0000 7ffbc3fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffbc40d0830 0x7ffbc4168a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.724+0000 7ffbc2ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ffbc40b9600 0x7ffbc40d0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35018/0 (socket says 172.21.15.38:35018) 2024-04-25T08:32:28.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.724+0000 7ffbc2ffd640 1 -- 172.21.15.38:0/1871065431 learned_addr learned my addr 172.21.15.38:0/1871065431 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:28.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.724+0000 7ffbc3fff640 1 -- 172.21.15.38:0/1871065431 >> v1:172.21.15.107:6789/0 conn(0x7ffbc415db80 legacy=0x7ffbc41616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:28.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.724+0000 7ffbc3fff640 1 -- 172.21.15.38:0/1871065431 >> v1:172.21.15.38:6789/0 conn(0x7ffbc40b9600 legacy=0x7ffbc40d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.724+0000 7ffbc27fc640 1 -- 172.21.15.38:0/1871065431 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698423959 0 0) 0x7ffbc4162030 con 0x7ffbc40b9600 2024-04-25T08:32:28.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.724+0000 7ffbc3fff640 1 -- 172.21.15.38:0/1871065431 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40d1ab0 con 0x7ffbc40d0830 2024-04-25T08:32:28.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.724+0000 7ffbc3fff640 1 --2- 172.21.15.38:0/1871065431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc40d0830 0x7ffbc4168a10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009d90 tx=0x7ffbbc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e003f9bccd94a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:28.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.725+0000 7ffbc27fc640 1 -- 172.21.15.38:0/1871065431 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc031600 con 0x7ffbc40d0830 2024-04-25T08:32:28.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.725+0000 7ffbc27fc640 1 -- 172.21.15.38:0/1871065431 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc0317a0 con 0x7ffbc40d0830 2024-04-25T08:32:28.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.725+0000 7ffbc27fc640 1 -- 172.21.15.38:0/1871065431 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc031b70 con 0x7ffbc40d0830 2024-04-25T08:32:28.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.725+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/1871065431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc40d0830 msgr2=0x7ffbc4168a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.725+0000 7ffbc9f5c640 1 --2- 172.21.15.38:0/1871065431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc40d0830 0x7ffbc4168a10 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009d90 tx=0x7ffbbc02a610 comp rx=0 tx=0).stop 2024-04-25T08:32:28.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.725+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/1871065431 shutdown_connections 2024-04-25T08:32:28.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.725+0000 7ffbc9f5c640 1 --2- 172.21.15.38:0/1871065431 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc40d0830 0x7ffbc4168a10 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.725+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/1871065431 >> 172.21.15.38:0/1871065431 conn(0x7ffbc40668f0 msgr2=0x7ffbc4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:28.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.725+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/1871065431 shutdown_connections 2024-04-25T08:32:28.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.725+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/1871065431 wait complete. 2024-04-25T08:32:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.726+0000 7ffbc9f5c640 1 Processor -- start 2024-04-25T08:32:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.727+0000 7ffbc9f5c640 1 -- start start 2024-04-25T08:32:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.727+0000 7ffbc9f5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc40b9600 0x7ffbc40c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.727+0000 7ffbc9f5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbc40d0830 0x7ffbc4154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.727+0000 7ffbc9f5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc415db80 0x7ffbc40c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.727+0000 7ffbc9f5c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40d32c0 con 0x7ffbc40b9600 2024-04-25T08:32:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.727+0000 7ffbc9f5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc41625f0 con 0x7ffbc40d0830 2024-04-25T08:32:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.727+0000 7ffbc9f5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40d3440 con 0x7ffbc415db80 2024-04-25T08:32:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbc37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc40b9600 0x7ffbc40c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbc37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc40b9600 0x7ffbc40c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49012/0 (socket says 172.21.15.38:49012) 2024-04-25T08:32:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbc37fe640 1 -- 172.21.15.38:0/499372424 learned_addr learned my addr 172.21.15.38:0/499372424 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:28.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbc37fe640 1 -- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc415db80 msgr2=0x7ffbc40c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbc37fe640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc415db80 0x7ffbc40c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbc37fe640 1 -- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbc40d0830 msgr2=0x7ffbc4154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbc2ffd640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbc40d0830 0x7ffbc4154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbc37fe640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbc40d0830 0x7ffbc4154480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbc37fe640 1 -- 172.21.15.38:0/499372424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40c7400 con 0x7ffbc40b9600 2024-04-25T08:32:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbc37fe640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc40b9600 0x7ffbc40c8150 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400b3f0 tx=0x7ffbb400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbabfff640 1 -- 172.21.15.38:0/499372424 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4015020 con 0x7ffbc40b9600 2024-04-25T08:32:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbabfff640 1 -- 172.21.15.38:0/499372424 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb4011830 con 0x7ffbc40b9600 2024-04-25T08:32:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/499372424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc40c7670 con 0x7ffbc40b9600 2024-04-25T08:32:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbabfff640 1 -- 172.21.15.38:0/499372424 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4011c20 con 0x7ffbc40b9600 2024-04-25T08:32:28.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.728+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/499372424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc40c8690 con 0x7ffbc40b9600 2024-04-25T08:32:28.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.729+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/499372424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc404ef90 con 0x7ffbc40b9600 2024-04-25T08:32:28.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.730+0000 7ffbabfff640 1 -- 172.21.15.38:0/499372424 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbb4018020 con 0x7ffbc40b9600 2024-04-25T08:32:28.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.730+0000 7ffbabfff640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffbb006b4d0 0x7ffbb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:28.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.730+0000 7ffbabfff640 1 -- 172.21.15.38:0/499372424 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7ffbb408a560 con 0x7ffbc40b9600 2024-04-25T08:32:28.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.730+0000 7ffbc2ffd640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffbb006b4d0 0x7ffbb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:28.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.731+0000 7ffbc2ffd640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffbb006b4d0 0x7ffbb006d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8007fd0 tx=0x7ffbb8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:28.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.734+0000 7ffbabfff640 1 -- 172.21.15.38:0/499372424 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbb401f020 con 0x7ffbc40b9600 2024-04-25T08:32:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.927+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/499372424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffbc40d14a0 con 0x7ffbc40b9600 2024-04-25T08:32:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.927+0000 7ffbabfff640 1 -- 172.21.15.38:0/499372424 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffbb40172c0 con 0x7ffbc40b9600 2024-04-25T08:32:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:32:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.929+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffbb006b4d0 msgr2=0x7ffbb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.929+0000 7ffbc9f5c640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffbb006b4d0 0x7ffbb006d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8007fd0 tx=0x7ffbb8009040 comp rx=0 tx=0).stop 2024-04-25T08:32:28.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.929+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/499372424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc40b9600 msgr2=0x7ffbc40c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.929+0000 7ffbc9f5c640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc40b9600 0x7ffbc40c8150 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400b3f0 tx=0x7ffbb400b8b0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.929+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/499372424 shutdown_connections 2024-04-25T08:32:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.929+0000 7ffbc9f5c640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffbb006b4d0 0x7ffbb006d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.929+0000 7ffbc9f5c640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc415db80 0x7ffbc40c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.929+0000 7ffbc9f5c640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbc40d0830 0x7ffbc4154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.929+0000 7ffbc9f5c640 1 --2- 172.21.15.38:0/499372424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc40b9600 0x7ffbc40c8150 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.929+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/499372424 >> 172.21.15.38:0/499372424 conn(0x7ffbc40668f0 msgr2=0x7ffbc415cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.929+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/499372424 shutdown_connections 2024-04-25T08:32:28.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:28.929+0000 7ffbc9f5c640 1 -- 172.21.15.38:0/499372424 wait complete. 2024-04-25T08:32:28.936 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:32:28.936 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:28.951 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:29.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.096+0000 7f9ee09a5640 1 Processor -- start 2024-04-25T08:32:29.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.096+0000 7f9ee09a5640 1 -- start start 2024-04-25T08:32:29.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9ee09a5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9edc15d930 0x7f9edc15dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9ee09a5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9edc0b4ea0 0x7f9edc0cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9ee09a5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9edc0ccaf0 0x7f9edc0ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9ee09a5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9edc0b4a10 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9ee09a5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9edc0cbee0 con 0x7f9edc15d930 2024-04-25T08:32:29.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9ee09a5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc15e250 con 0x7f9edc0b4ea0 2024-04-25T08:32:29.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9edad9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9edc0ccaf0 0x7f9edc0ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9edad9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9edc0ccaf0 0x7f9edc0ccf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44482/0 (socket says 172.21.15.38:44482) 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9edad9d640 1 -- 172.21.15.38:0/1605555315 learned_addr learned my addr 172.21.15.38:0/1605555315 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9ed3fff640 1 --2- 172.21.15.38:0/1605555315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9edc0b4ea0 0x7f9edc0cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9eda59c640 1 --2- 172.21.15.38:0/1605555315 >> v2:172.21.15.107:3300/0 conn(0x7f9edc15d930 0x7f9edc15dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9edad9d640 1 -- 172.21.15.38:0/1605555315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9edc0b4ea0 msgr2=0x7f9edc0cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9edad9d640 1 --2- 172.21.15.38:0/1605555315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9edc0b4ea0 0x7f9edc0cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9edad9d640 1 -- 172.21.15.38:0/1605555315 >> v2:172.21.15.107:3300/0 conn(0x7f9edc15d930 msgr2=0x7f9edc15dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9edad9d640 1 --2- 172.21.15.38:0/1605555315 >> v2:172.21.15.107:3300/0 conn(0x7f9edc15d930 0x7f9edc15dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9edad9d640 1 -- 172.21.15.38:0/1605555315 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc15e700 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.097+0000 7f9eda59c640 1 --2- 172.21.15.38:0/1605555315 >> v2:172.21.15.107:3300/0 conn(0x7f9edc15d930 0x7f9edc15dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.098+0000 7f9edad9d640 1 --2- 172.21.15.38:0/1605555315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9edc0ccaf0 0x7f9edc0ccf80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9edc06bba0 tx=0x7f9ed402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50eaec679b17380 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.098+0000 7f9ed9d9b640 1 -- 172.21.15.38:0/1605555315 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4037070 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.098+0000 7f9ed9d9b640 1 -- 172.21.15.38:0/1605555315 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed4032600 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.098+0000 7f9ed9d9b640 1 -- 172.21.15.38:0/1605555315 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40329d0 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.099+0000 7f9ee09a5640 1 -- 172.21.15.38:0/1605555315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9edc0ccaf0 msgr2=0x7f9edc0ccf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.099+0000 7f9ee09a5640 1 --2- 172.21.15.38:0/1605555315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9edc0ccaf0 0x7f9edc0ccf80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9edc06bba0 tx=0x7f9ed402b7b0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.099+0000 7f9ee09a5640 1 -- 172.21.15.38:0/1605555315 shutdown_connections 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.099+0000 7f9ee09a5640 1 --2- 172.21.15.38:0/1605555315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9edc0ccaf0 0x7f9edc0ccf80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.099+0000 7f9ee09a5640 1 --2- 172.21.15.38:0/1605555315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9edc0b4ea0 0x7f9edc0cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.099+0000 7f9ee09a5640 1 --2- 172.21.15.38:0/1605555315 >> v2:172.21.15.107:3300/0 conn(0x7f9edc15d930 0x7f9edc15dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.099+0000 7f9ee09a5640 1 -- 172.21.15.38:0/1605555315 >> 172.21.15.38:0/1605555315 conn(0x7f9edc06b210 msgr2=0x7f9edc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.099+0000 7f9ee09a5640 1 -- 172.21.15.38:0/1605555315 shutdown_connections 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.099+0000 7f9ee09a5640 1 -- 172.21.15.38:0/1605555315 wait complete. 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.100+0000 7f9ee09a5640 1 Processor -- start 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.100+0000 7f9ee09a5640 1 -- start start 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ee09a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9edc0b4ea0 0x7f9edc1426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ee09a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9edc0ccaf0 0x7f9edc155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ee09a5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9edc15d930 0x7f9edc144770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ee09a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0b46a0 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ee09a5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0b43c0 con 0x7f9edc0b4ea0 2024-04-25T08:32:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ee09a5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9edc0cb840 con 0x7f9edc15d930 2024-04-25T08:32:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ed3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9edc0ccaf0 0x7f9edc155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ed3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9edc0ccaf0 0x7f9edc155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44492/0 (socket says 172.21.15.38:44492) 2024-04-25T08:32:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ed3fff640 1 -- 172.21.15.38:0/2914785500 learned_addr learned my addr 172.21.15.38:0/2914785500 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9eda59c640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9edc0b4ea0 0x7f9edc1426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9edad9d640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9edc15d930 0x7f9edc144770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ed3fff640 1 -- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9edc15d930 msgr2=0x7f9edc144770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ed3fff640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9edc15d930 0x7f9edc144770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ed3fff640 1 -- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9edc0b4ea0 msgr2=0x7f9edc1426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ed3fff640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9edc0b4ea0 0x7f9edc1426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ed3fff640 1 -- 172.21.15.38:0/2914785500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc156470 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9edad9d640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9edc15d930 0x7f9edc144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ed3fff640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9edc0ccaf0 0x7f9edc155f30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400e8c0 tx=0x7f9ec400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ed2ffd640 1 -- 172.21.15.38:0/2914785500 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4017070 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ed2ffd640 1 -- 172.21.15.38:0/2914785500 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec400c940 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ed2ffd640 1 -- 172.21.15.38:0/2914785500 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec400cd10 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.101+0000 7f9ee09a5640 1 -- 172.21.15.38:0/2914785500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9edc143900 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.102+0000 7f9ee09a5640 1 -- 172.21.15.38:0/2914785500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9edc143060 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.103+0000 7f9ee09a5640 1 -- 172.21.15.38:0/2914785500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9edc068e80 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.103+0000 7f9ed2ffd640 1 -- 172.21.15.38:0/2914785500 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ec401a070 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.103+0000 7f9ed2ffd640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ebc06b4d0 0x7f9ebc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.104+0000 7f9ed2ffd640 1 -- 172.21.15.38:0/2914785500 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f9ec408a5d0 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.104+0000 7f9eda59c640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ebc06b4d0 0x7f9ebc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.104+0000 7f9eda59c640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ebc06b4d0 0x7f9ebc06d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9edc1f2790 tx=0x7f9ecc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.108+0000 7f9ed2ffd640 1 -- 172.21.15.38:0/2914785500 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ec4058ff0 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.290+0000 7f9ee09a5640 1 -- 172.21.15.38:0/2914785500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9edc0cc6a0 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.291+0000 7f9ed2ffd640 1 -- 172.21.15.38:0/2914785500 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18969 (secure 0 0 0) 0x7f9ec40588f0 con 0x7f9edc0ccaf0 2024-04-25T08:32:29.287 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:29.288 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:16.572337+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.a","mappings":[{"from":4,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.293+0000 7f9ee09a5640 1 -- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ebc06b4d0 msgr2=0x7f9ebc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.293+0000 7f9ee09a5640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ebc06b4d0 0x7f9ebc06d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9edc1f2790 tx=0x7f9ecc008040 comp rx=0 tx=0).stop 2024-04-25T08:32:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.293+0000 7f9ee09a5640 1 -- 172.21.15.38:0/2914785500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9edc0ccaf0 msgr2=0x7f9edc155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.293+0000 7f9ee09a5640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9edc0ccaf0 0x7f9edc155f30 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400e8c0 tx=0x7f9ec400ed80 comp rx=0 tx=0).stop 2024-04-25T08:32:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.293+0000 7f9ee09a5640 1 -- 172.21.15.38:0/2914785500 shutdown_connections 2024-04-25T08:32:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.293+0000 7f9ee09a5640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ebc06b4d0 0x7f9ebc06d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.293+0000 7f9ee09a5640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9edc15d930 0x7f9edc144770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.293+0000 7f9ee09a5640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9edc0ccaf0 0x7f9edc155f30 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.293+0000 7f9ee09a5640 1 --2- 172.21.15.38:0/2914785500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9edc0b4ea0 0x7f9edc1426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.293+0000 7f9ee09a5640 1 -- 172.21.15.38:0/2914785500 >> 172.21.15.38:0/2914785500 conn(0x7f9edc06b210 msgr2=0x7f9edc070f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.293+0000 7f9ee09a5640 1 -- 172.21.15.38:0/2914785500 shutdown_connections 2024-04-25T08:32:29.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.293+0000 7f9ee09a5640 1 -- 172.21.15.38:0/2914785500 wait complete. 2024-04-25T08:32:29.301 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:29.301 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:29.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.408+0000 7f840ac9a640 1 Processor -- start 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.408+0000 7f840ac9a640 1 -- start start 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.409+0000 7f840ac9a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f840415b990 0x7f84040cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.409+0000 7f840ac9a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84040d1d20 0x7f84040b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.409+0000 7f840ac9a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840415fd00 con 0x7f840415f5f0 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.409+0000 7f840ac9a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8404160580 con 0x7f840415b990 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.409+0000 7f840ac9a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840415fa30 con 0x7f84040d1d20 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.409+0000 7f8408a36640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f840415b990 0x7f84040cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.409+0000 7f8408a36640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f840415b990 0x7f84040cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44502/0 (socket says 172.21.15.38:44502) 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.409+0000 7f8408a36640 1 -- 172.21.15.38:0/497382934 learned_addr learned my addr 172.21.15.38:0/497382934 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.410+0000 7f8409237640 1 --2- 172.21.15.38:0/497382934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84040d1d20 0x7f84040b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.410+0000 7f8408a36640 1 -- 172.21.15.38:0/497382934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84040d1d20 msgr2=0x7f84040b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.410+0000 7f8408a36640 1 --2- 172.21.15.38:0/497382934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84040d1d20 0x7f84040b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.410+0000 7f8408a36640 1 -- 172.21.15.38:0/497382934 >> v1:172.21.15.107:6789/0 conn(0x7f840415f5f0 legacy=0x7f84040d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.410+0000 7f8408a36640 1 -- 172.21.15.38:0/497382934 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840415ffc0 con 0x7f840415b990 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.410+0000 7f8409237640 1 --2- 172.21.15.38:0/497382934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84040d1d20 0x7f84040b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.410+0000 7f8408a36640 1 --2- 172.21.15.38:0/497382934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f840415b990 0x7f84040cf580 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f84040d15e0 tx=0x7f83fc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21da341d71be8d67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.410+0000 7f84037fe640 1 -- 172.21.15.38:0/497382934 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc036070 con 0x7f840415b990 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.411+0000 7f84037fe640 1 -- 172.21.15.38:0/497382934 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc031400 con 0x7f840415b990 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.411+0000 7f84037fe640 1 -- 172.21.15.38:0/497382934 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc0317d0 con 0x7f840415b990 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.411+0000 7f840ac9a640 1 -- 172.21.15.38:0/497382934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f840415b990 msgr2=0x7f84040cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.411+0000 7f840ac9a640 1 --2- 172.21.15.38:0/497382934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f840415b990 0x7f84040cf580 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f84040d15e0 tx=0x7f83fc02a610 comp rx=0 tx=0).stop 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.411+0000 7f840ac9a640 1 -- 172.21.15.38:0/497382934 shutdown_connections 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.411+0000 7f840ac9a640 1 --2- 172.21.15.38:0/497382934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84040d1d20 0x7f84040b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.411+0000 7f840ac9a640 1 --2- 172.21.15.38:0/497382934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f840415b990 0x7f84040cf580 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.411+0000 7f840ac9a640 1 -- 172.21.15.38:0/497382934 >> 172.21.15.38:0/497382934 conn(0x7f840408b000 msgr2=0x7f840408b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.411+0000 7f840ac9a640 1 -- 172.21.15.38:0/497382934 shutdown_connections 2024-04-25T08:32:29.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.411+0000 7f840ac9a640 1 -- 172.21.15.38:0/497382934 wait complete. 2024-04-25T08:32:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.412+0000 7f840ac9a640 1 Processor -- start 2024-04-25T08:32:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.412+0000 7f840ac9a640 1 -- start start 2024-04-25T08:32:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.413+0000 7f840ac9a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84040d1d20 0x7f840415ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.413+0000 7f840ac9a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f840415b990 0x7f84040c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.413+0000 7f840ac9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f840415f5f0 0x7f840414d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.413+0000 7f840ac9a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840415bd70 con 0x7f840415f5f0 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.413+0000 7f840ac9a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84040b1ac0 con 0x7f84040d1d20 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.413+0000 7f840ac9a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840415bef0 con 0x7f840415b990 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.413+0000 7f8409237640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f840415f5f0 0x7f840414d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.413+0000 7f8409237640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f840415f5f0 0x7f840414d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44516/0 (socket says 172.21.15.38:44516) 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.413+0000 7f8409237640 1 -- 172.21.15.38:0/3410569757 learned_addr learned my addr 172.21.15.38:0/3410569757 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.413+0000 7f8408a36640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84040d1d20 0x7f840415ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.414+0000 7f8403fff640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f840415b990 0x7f84040c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.414+0000 7f8409237640 1 -- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f840415b990 msgr2=0x7f84040c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.414+0000 7f8409237640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f840415b990 0x7f84040c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.414+0000 7f8409237640 1 -- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84040d1d20 msgr2=0x7f840415ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.414+0000 7f8409237640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84040d1d20 0x7f840415ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.414+0000 7f8409237640 1 -- 172.21.15.38:0/3410569757 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840406c1f0 con 0x7f840415f5f0 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.414+0000 7f8409237640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f840415f5f0 0x7f840414d9b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f83f400c820 tx=0x7f83f400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:29.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.414+0000 7f84017fa640 1 -- 172.21.15.38:0/3410569757 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4015070 con 0x7f840415f5f0 2024-04-25T08:32:29.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.414+0000 7f84017fa640 1 -- 172.21.15.38:0/3410569757 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f4007a30 con 0x7f840415f5f0 2024-04-25T08:32:29.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.414+0000 7f840ac9a640 1 -- 172.21.15.38:0/3410569757 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840406c4d0 con 0x7f840415f5f0 2024-04-25T08:32:29.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.414+0000 7f84017fa640 1 -- 172.21.15.38:0/3410569757 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4007e20 con 0x7f840415f5f0 2024-04-25T08:32:29.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.414+0000 7f840ac9a640 1 -- 172.21.15.38:0/3410569757 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840406ca90 con 0x7f840415f5f0 2024-04-25T08:32:29.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.415+0000 7f840ac9a640 1 -- 172.21.15.38:0/3410569757 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8404068e80 con 0x7f840415f5f0 2024-04-25T08:32:29.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.416+0000 7f84017fa640 1 -- 172.21.15.38:0/3410569757 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83f4018070 con 0x7f840415f5f0 2024-04-25T08:32:29.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.416+0000 7f84017fa640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83f006b4d0 0x7f83f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.416+0000 7f84017fa640 1 -- 172.21.15.38:0/3410569757 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f83f408a4e0 con 0x7f840415f5f0 2024-04-25T08:32:29.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.416+0000 7f8408a36640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83f006b4d0 0x7f83f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.417+0000 7f8408a36640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83f006b4d0 0x7f83f006d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f83fc033000 tx=0x7f83fc009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:29.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.420+0000 7f84017fa640 1 -- 172.21.15.38:0/3410569757 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83f40588b0 con 0x7f840415f5f0 2024-04-25T08:32:29.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.604+0000 7f840ac9a640 1 -- 172.21.15.38:0/3410569757 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84040b1220 con 0x7f840415f5f0 2024-04-25T08:32:29.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.605+0000 7f84017fa640 1 -- 172.21.15.38:0/3410569757 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83f405c950 con 0x7f840415f5f0 2024-04-25T08:32:29.601 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.607+0000 7f840ac9a640 1 -- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83f006b4d0 msgr2=0x7f83f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.607+0000 7f840ac9a640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83f006b4d0 0x7f83f006d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f83fc033000 tx=0x7f83fc009000 comp rx=0 tx=0).stop 2024-04-25T08:32:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.607+0000 7f840ac9a640 1 -- 172.21.15.38:0/3410569757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f840415f5f0 msgr2=0x7f840414d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.607+0000 7f840ac9a640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f840415f5f0 0x7f840414d9b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f83f400c820 tx=0x7f83f400cce0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.607+0000 7f840ac9a640 1 -- 172.21.15.38:0/3410569757 shutdown_connections 2024-04-25T08:32:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.607+0000 7f840ac9a640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f840415f5f0 0x7f840414d9b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.607+0000 7f840ac9a640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83f006b4d0 0x7f83f006d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.607+0000 7f840ac9a640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f840415b990 0x7f84040c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.607+0000 7f840ac9a640 1 --2- 172.21.15.38:0/3410569757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84040d1d20 0x7f840415ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.607+0000 7f840ac9a640 1 -- 172.21.15.38:0/3410569757 >> 172.21.15.38:0/3410569757 conn(0x7f840408b000 msgr2=0x7f8404159490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:29.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.607+0000 7f840ac9a640 1 -- 172.21.15.38:0/3410569757 shutdown_connections 2024-04-25T08:32:29.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.607+0000 7f840ac9a640 1 -- 172.21.15.38:0/3410569757 wait complete. 2024-04-25T08:32:29.614 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:29.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.722+0000 7f13cb06b640 1 Processor -- start 2024-04-25T08:32:29.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.722+0000 7f13cb06b640 1 -- start start 2024-04-25T08:32:29.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13cb06b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f13c40d0320 0x7f13c40d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13cb06b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13c40d0700 0x7f13c40d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13cb06b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f13c40d1e90 0x7f13c40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13cb06b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c40d2330 con 0x7f13c40d1e90 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13cb06b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c40d2610 con 0x7f13c40d0320 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13cb06b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40d28f0 con 0x7f13c40d0700 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13c9608640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f13c40d1e90 0x7f13c40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13c9608640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f13c40d1e90 0x7f13c40b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44530/0 (socket says 172.21.15.38:44530) 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13c9608640 1 -- 172.21.15.38:0/2841583401 learned_addr learned my addr 172.21.15.38:0/2841583401 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13c3fff640 1 --2- 172.21.15.38:0/2841583401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13c40d0700 0x7f13c40d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13c9608640 1 -- 172.21.15.38:0/2841583401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13c40d0700 msgr2=0x7f13c40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13c9608640 1 --2- 172.21.15.38:0/2841583401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13c40d0700 0x7f13c40d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13c9608640 1 -- 172.21.15.38:0/2841583401 >> v2:172.21.15.107:3300/0 conn(0x7f13c40d0320 msgr2=0x7f13c40d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13c8e07640 1 --2- 172.21.15.38:0/2841583401 >> v2:172.21.15.107:3300/0 conn(0x7f13c40d0320 0x7f13c40d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13c9608640 1 --2- 172.21.15.38:0/2841583401 >> v2:172.21.15.107:3300/0 conn(0x7f13c40d0320 0x7f13c40d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13c9608640 1 -- 172.21.15.38:0/2841583401 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c40d2bd0 con 0x7f13c40d1e90 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.723+0000 7f13c8e07640 1 --2- 172.21.15.38:0/2841583401 >> v2:172.21.15.107:3300/0 conn(0x7f13c40d0320 0x7f13c40d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.724+0000 7f13c9608640 1 --2- 172.21.15.38:0/2841583401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c40d1e90 0x7f13c40b5390 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f13c4069100 tx=0x7f13bc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890ad59ca8732cda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:29.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.724+0000 7f13c37fe640 1 -- 172.21.15.38:0/2841583401 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc036070 con 0x7f13c40d1e90 2024-04-25T08:32:29.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.724+0000 7f13c37fe640 1 -- 172.21.15.38:0/2841583401 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc031600 con 0x7f13c40d1e90 2024-04-25T08:32:29.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.724+0000 7f13c37fe640 1 -- 172.21.15.38:0/2841583401 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0319d0 con 0x7f13c40d1e90 2024-04-25T08:32:29.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.725+0000 7f13cb06b640 1 -- 172.21.15.38:0/2841583401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c40d1e90 msgr2=0x7f13c40b5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.725+0000 7f13cb06b640 1 --2- 172.21.15.38:0/2841583401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c40d1e90 0x7f13c40b5390 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f13c4069100 tx=0x7f13bc02a730 comp rx=0 tx=0).stop 2024-04-25T08:32:29.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.725+0000 7f13cb06b640 1 -- 172.21.15.38:0/2841583401 shutdown_connections 2024-04-25T08:32:29.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.725+0000 7f13cb06b640 1 --2- 172.21.15.38:0/2841583401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c40d1e90 0x7f13c40b5390 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.725+0000 7f13cb06b640 1 --2- 172.21.15.38:0/2841583401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13c40d0700 0x7f13c40d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.725+0000 7f13cb06b640 1 --2- 172.21.15.38:0/2841583401 >> v2:172.21.15.107:3300/0 conn(0x7f13c40d0320 0x7f13c40d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.725+0000 7f13cb06b640 1 -- 172.21.15.38:0/2841583401 >> 172.21.15.38:0/2841583401 conn(0x7f13c40668f0 msgr2=0x7f13c4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.725+0000 7f13cb06b640 1 -- 172.21.15.38:0/2841583401 shutdown_connections 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.725+0000 7f13cb06b640 1 -- 172.21.15.38:0/2841583401 wait complete. 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.726+0000 7f13cb06b640 1 Processor -- start 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.726+0000 7f13cb06b640 1 -- start start 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.727+0000 7f13cb06b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13c40d0320 0x7f13c415af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.727+0000 7f13cb06b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c40d0700 0x7f13c40b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.727+0000 7f13cb06b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13c40d1e90 0x7f13c40ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.727+0000 7f13cb06b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40d0b40 con 0x7f13c40d0700 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.727+0000 7f13cb06b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c4161210 con 0x7f13c40d0320 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.727+0000 7f13cb06b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c415b4b0 con 0x7f13c40d1e90 2024-04-25T08:32:29.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.727+0000 7f13c3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c40d0700 0x7f13c40b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.727+0000 7f13c3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c40d0700 0x7f13c40b5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44542/0 (socket says 172.21.15.38:44542) 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.727+0000 7f13c3fff640 1 -- 172.21.15.38:0/3700181705 learned_addr learned my addr 172.21.15.38:0/3700181705 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.727+0000 7f13c8e07640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13c40d0320 0x7f13c415af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.727+0000 7f13c9608640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13c40d1e90 0x7f13c40ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.727+0000 7f13c3fff640 1 -- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13c40d1e90 msgr2=0x7f13c40ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.728+0000 7f13c3fff640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13c40d1e90 0x7f13c40ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.728+0000 7f13c3fff640 1 -- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13c40d0320 msgr2=0x7f13c415af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.728+0000 7f13c3fff640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13c40d0320 0x7f13c415af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.728+0000 7f13c3fff640 1 -- 172.21.15.38:0/3700181705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c40aa7d0 con 0x7f13c40d0700 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.728+0000 7f13c3fff640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c40d0700 0x7f13c40b5c50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f13b800c7f0 tx=0x7f13b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.728+0000 7f13c17fa640 1 -- 172.21.15.38:0/3700181705 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8015070 con 0x7f13c40d0700 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.728+0000 7f13c17fa640 1 -- 172.21.15.38:0/3700181705 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b8004240 con 0x7f13c40d0700 2024-04-25T08:32:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.728+0000 7f13c17fa640 1 -- 172.21.15.38:0/3700181705 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b8007620 con 0x7f13c40d0700 2024-04-25T08:32:29.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.728+0000 7f13cb06b640 1 -- 172.21.15.38:0/3700181705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c40aaab0 con 0x7f13c40d0700 2024-04-25T08:32:29.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.728+0000 7f13cb06b640 1 -- 172.21.15.38:0/3700181705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c40ab070 con 0x7f13c40d0700 2024-04-25T08:32:29.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.729+0000 7f13cb06b640 1 -- 172.21.15.38:0/3700181705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c40ab450 con 0x7f13c40d0700 2024-04-25T08:32:29.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.729+0000 7f13c17fa640 1 -- 172.21.15.38:0/3700181705 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13b80079d0 con 0x7f13c40d0700 2024-04-25T08:32:29.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.730+0000 7f13c17fa640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f13b006b4d0 0x7f13b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:29.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.730+0000 7f13c17fa640 1 -- 172.21.15.38:0/3700181705 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f13b808a660 con 0x7f13c40d0700 2024-04-25T08:32:29.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.730+0000 7f13c8e07640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f13b006b4d0 0x7f13b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:29.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.731+0000 7f13c8e07640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f13b006b4d0 0x7f13b006d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f13c40b5160 tx=0x7f13b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:29.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.734+0000 7f13c17fa640 1 -- 172.21.15.38:0/3700181705 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f13c40ab450 con 0x7f13c40d0700 2024-04-25T08:32:29.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.914+0000 7f13cb06b640 1 -- 172.21.15.38:0/3700181705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13c4068e80 con 0x7f13c40d0700 2024-04-25T08:32:29.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.914+0000 7f13c17fa640 1 -- 172.21.15.38:0/3700181705 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f13b8058a30 con 0x7f13c40d0700 2024-04-25T08:32:29.911 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.916+0000 7f13cb06b640 1 -- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f13b006b4d0 msgr2=0x7f13b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.916+0000 7f13cb06b640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f13b006b4d0 0x7f13b006d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f13c40b5160 tx=0x7f13b4008040 comp rx=0 tx=0).stop 2024-04-25T08:32:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.916+0000 7f13cb06b640 1 -- 172.21.15.38:0/3700181705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c40d0700 msgr2=0x7f13c40b5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.916+0000 7f13cb06b640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c40d0700 0x7f13c40b5c50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f13b800c7f0 tx=0x7f13b800ccb0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.916+0000 7f13cb06b640 1 -- 172.21.15.38:0/3700181705 shutdown_connections 2024-04-25T08:32:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.916+0000 7f13cb06b640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f13b006b4d0 0x7f13b006d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.916+0000 7f13cb06b640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13c40d1e90 0x7f13c40ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.916+0000 7f13cb06b640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13c40d0700 0x7f13c40b5c50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.916+0000 7f13cb06b640 1 --2- 172.21.15.38:0/3700181705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13c40d0320 0x7f13c415af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.916+0000 7f13cb06b640 1 -- 172.21.15.38:0/3700181705 >> 172.21.15.38:0/3700181705 conn(0x7f13c40668f0 msgr2=0x7f13c40b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.916+0000 7f13cb06b640 1 -- 172.21.15.38:0/3700181705 shutdown_connections 2024-04-25T08:32:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:29.916+0000 7f13cb06b640 1 -- 172.21.15.38:0/3700181705 wait complete. 2024-04-25T08:32:29.923 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:30.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.031+0000 7f133cb34640 1 Processor -- start 2024-04-25T08:32:30.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.031+0000 7f133cb34640 1 -- start start 2024-04-25T08:32:30.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f133cb34640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f13380d0320 0x7f13380d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f133cb34640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13380d0700 0x7f13380d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f133cb34640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13380d2330 con 0x7f13380d1e90 2024-04-25T08:32:30.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f133cb34640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13380d2610 con 0x7f13380d0320 2024-04-25T08:32:30.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f133cb34640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13380d28f0 con 0x7f13380d0700 2024-04-25T08:32:30.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f133659c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f13380d0320 0x7f13380d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f133659c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f13380d0320 0x7f13380d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44546/0 (socket says 172.21.15.38:44546) 2024-04-25T08:32:30.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f133659c640 1 -- 172.21.15.38:0/4143625940 learned_addr learned my addr 172.21.15.38:0/4143625940 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:30.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f132ffff640 1 --2- 172.21.15.38:0/4143625940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13380d0700 0x7f13380d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f133659c640 1 -- 172.21.15.38:0/4143625940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13380d0700 msgr2=0x7f13380d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:30.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f133659c640 1 --2- 172.21.15.38:0/4143625940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13380d0700 0x7f13380d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f133659c640 1 -- 172.21.15.38:0/4143625940 >> v1:172.21.15.107:6789/0 conn(0x7f13380d1e90 legacy=0x7f13380b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.032+0000 7f133659c640 1 -- 172.21.15.38:0/4143625940 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13380d2bd0 con 0x7f13380d0320 2024-04-25T08:32:30.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.033+0000 7f133659c640 1 --2- 172.21.15.38:0/4143625940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13380d0320 0x7f13380d33d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1328009870 tx=0x7f132802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4a60cd3d0b8fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:30.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.033+0000 7f1335d9b640 1 -- 172.21.15.38:0/4143625940 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328037070 con 0x7f13380d0320 2024-04-25T08:32:30.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.033+0000 7f1335d9b640 1 -- 172.21.15.38:0/4143625940 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132802ae50 con 0x7f13380d0320 2024-04-25T08:32:30.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.033+0000 7f1335d9b640 1 -- 172.21.15.38:0/4143625940 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328032860 con 0x7f13380d0320 2024-04-25T08:32:30.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.034+0000 7f133cb34640 1 -- 172.21.15.38:0/4143625940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13380d0320 msgr2=0x7f13380d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.034+0000 7f133cb34640 1 --2- 172.21.15.38:0/4143625940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13380d0320 0x7f13380d33d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1328009870 tx=0x7f132802a300 comp rx=0 tx=0).stop 2024-04-25T08:32:30.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.034+0000 7f133cb34640 1 -- 172.21.15.38:0/4143625940 shutdown_connections 2024-04-25T08:32:30.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.034+0000 7f133cb34640 1 --2- 172.21.15.38:0/4143625940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13380d0700 0x7f13380d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.034+0000 7f133cb34640 1 --2- 172.21.15.38:0/4143625940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13380d0320 0x7f13380d33d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.034+0000 7f133cb34640 1 -- 172.21.15.38:0/4143625940 >> 172.21.15.38:0/4143625940 conn(0x7f13380668f0 msgr2=0x7f1338066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:30.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.034+0000 7f133cb34640 1 -- 172.21.15.38:0/4143625940 shutdown_connections 2024-04-25T08:32:30.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.034+0000 7f133cb34640 1 -- 172.21.15.38:0/4143625940 wait complete. 2024-04-25T08:32:30.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.035+0000 7f133cb34640 1 Processor -- start 2024-04-25T08:32:30.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.035+0000 7f133cb34640 1 -- start start 2024-04-25T08:32:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f133cb34640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13380d0320 0x7f1338155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f133cb34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13380d0700 0x7f1338159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f133cb34640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13380d1e90 0x7f13380ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f133cb34640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13380d0b40 con 0x7f13380d0700 2024-04-25T08:32:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f133cb34640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1338161210 con 0x7f13380d0320 2024-04-25T08:32:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f133cb34640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133815a480 con 0x7f13380d1e90 2024-04-25T08:32:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f132ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13380d0700 0x7f1338159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f132ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13380d0700 0x7f1338159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44560/0 (socket says 172.21.15.38:44560) 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f132ffff640 1 -- 172.21.15.38:0/1701094608 learned_addr learned my addr 172.21.15.38:0/1701094608 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f133659c640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13380d0320 0x7f1338155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f1336d9d640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13380d1e90 0x7f13380ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f132ffff640 1 -- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13380d1e90 msgr2=0x7f13380ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f132ffff640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13380d1e90 0x7f13380ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f132ffff640 1 -- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13380d0320 msgr2=0x7f1338155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f132ffff640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13380d0320 0x7f1338155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f132ffff640 1 -- 172.21.15.38:0/1701094608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1338155910 con 0x7f13380d0700 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.036+0000 7f132ffff640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13380d0700 0x7f1338159f40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f132000e9e0 tx=0x7f132000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.037+0000 7f132effd640 1 -- 172.21.15.38:0/1701094608 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320017070 con 0x7f13380d0700 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.037+0000 7f132effd640 1 -- 172.21.15.38:0/1701094608 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132000c970 con 0x7f13380d0700 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.037+0000 7f133cb34640 1 -- 172.21.15.38:0/1701094608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1338155bf0 con 0x7f13380d0700 2024-04-25T08:32:30.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.037+0000 7f132effd640 1 -- 172.21.15.38:0/1701094608 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132000cd60 con 0x7f13380d0700 2024-04-25T08:32:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.037+0000 7f133cb34640 1 -- 172.21.15.38:0/1701094608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13381561b0 con 0x7f13380d0700 2024-04-25T08:32:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.038+0000 7f133cb34640 1 -- 172.21.15.38:0/1701094608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1338068e80 con 0x7f13380d0700 2024-04-25T08:32:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.038+0000 7f132effd640 1 -- 172.21.15.38:0/1701094608 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f132001a070 con 0x7f13380d0700 2024-04-25T08:32:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.039+0000 7f132effd640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f131806b4d0 0x7f131806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.039+0000 7f132effd640 1 -- 172.21.15.38:0/1701094608 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f132008a890 con 0x7f13380d0700 2024-04-25T08:32:30.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.039+0000 7f133659c640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f131806b4d0 0x7f131806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.039+0000 7f133659c640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f131806b4d0 0x7f131806d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1328034000 tx=0x7f132802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:30.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.043+0000 7f132effd640 1 -- 172.21.15.38:0/1701094608 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1320058cf0 con 0x7f13380d0700 2024-04-25T08:32:30.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.222+0000 7f133cb34640 1 -- 172.21.15.38:0/1701094608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f13380b2550 con 0x7f13380d0700 2024-04-25T08:32:30.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.223+0000 7f132effd640 1 -- 172.21.15.38:0/1701094608 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f132005cd90 con 0x7f13380d0700 2024-04-25T08:32:30.219 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:30.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.224+0000 7f133cb34640 1 -- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f131806b4d0 msgr2=0x7f131806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.224+0000 7f133cb34640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f131806b4d0 0x7f131806d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1328034000 tx=0x7f132802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:30.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.224+0000 7f133cb34640 1 -- 172.21.15.38:0/1701094608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13380d0700 msgr2=0x7f1338159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.224+0000 7f133cb34640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13380d0700 0x7f1338159f40 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f132000e9e0 tx=0x7f132000eea0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.225+0000 7f133cb34640 1 -- 172.21.15.38:0/1701094608 shutdown_connections 2024-04-25T08:32:30.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.225+0000 7f133cb34640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f131806b4d0 0x7f131806d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.225+0000 7f133cb34640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f13380d1e90 0x7f13380ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.225+0000 7f133cb34640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13380d0700 0x7f1338159f40 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.225+0000 7f133cb34640 1 --2- 172.21.15.38:0/1701094608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13380d0320 0x7f1338155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.225+0000 7f133cb34640 1 -- 172.21.15.38:0/1701094608 >> 172.21.15.38:0/1701094608 conn(0x7f13380668f0 msgr2=0x7f13381602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:30.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.225+0000 7f133cb34640 1 -- 172.21.15.38:0/1701094608 shutdown_connections 2024-04-25T08:32:30.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.225+0000 7f133cb34640 1 -- 172.21.15.38:0/1701094608 wait complete. 2024-04-25T08:32:30.231 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:30.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.339+0000 7f7b5ad7e640 1 Processor -- start 2024-04-25T08:32:30.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.339+0000 7f7b5ad7e640 1 -- start start 2024-04-25T08:32:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.340+0000 7f7b5ad7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540d0320 0x7f7b540d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.340+0000 7f7b5ad7e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7b540b6980 0x7f7b540b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.340+0000 7f7b5ad7e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b540d2060 con 0x7f7b540d0700 2024-04-25T08:32:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.340+0000 7f7b5ad7e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b540d2330 con 0x7f7b540b6980 2024-04-25T08:32:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.340+0000 7f7b5ad7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540d2610 con 0x7f7b540d0320 2024-04-25T08:32:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.340+0000 7f7b53fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7b540d0700 0x7f7b540d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56440/0 (socket says 172.21.15.38:56440) 2024-04-25T08:32:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.340+0000 7f7b53fff640 1 -- 172.21.15.38:0/2267496232 learned_addr learned my addr 172.21.15.38:0/2267496232 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b58b1a640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540d0320 0x7f7b540d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b5931b640 1 --2- 172.21.15.38:0/2267496232 >> v2:172.21.15.107:3300/0 conn(0x7f7b540b6980 0x7f7b540b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604711360 0 0) 0x7f7b540d2060 con 0x7f7b540d0700 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b40003520 con 0x7f7b540d0700 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1285401821 0 0) 0x7f7b40003520 con 0x7f7b540d0700 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540d0320 msgr2=0x7f7b540d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540d0320 0x7f7b540d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 >> v2:172.21.15.107:3300/0 conn(0x7f7b540b6980 msgr2=0x7f7b540b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 --2- 172.21.15.38:0/2267496232 >> v2:172.21.15.107:3300/0 conn(0x7f7b540b6980 0x7f7b540b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b540d28f0 con 0x7f7b540d0700 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f7b44001670 con 0x7f7b540d0700 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 >> v1:172.21.15.38:6789/0 conn(0x7f7b540d0700 legacy=0x7f7b540d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b400039b0 0x7f7b40003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540d0320 0x7f7b40004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b40004d40 0x7f7b400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540d2330 con 0x7f7b400039b0 2024-04-25T08:32:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540d2610 con 0x7f7b40004d40 2024-04-25T08:32:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4000b9f0 con 0x7f7b540d0320 2024-04-25T08:32:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b58b1a640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b400039b0 0x7f7b40003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b53fff640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540d0320 0x7f7b40004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b5931b640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b40004d40 0x7f7b400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b58b1a640 1 -- 172.21.15.38:0/2267496232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540d0320 msgr2=0x7f7b40004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b58b1a640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540d0320 0x7f7b40004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b58b1a640 1 -- 172.21.15.38:0/2267496232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b40004d40 msgr2=0x7f7b400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b58b1a640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b40004d40 0x7f7b400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b58b1a640 1 -- 172.21.15.38:0/2267496232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b40003520 con 0x7f7b400039b0 2024-04-25T08:32:30.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.341+0000 7f7b5931b640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b40004d40 0x7f7b400051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.342+0000 7f7b58b1a640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b400039b0 0x7f7b40003dd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009fc0 tx=0x7f7b40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a06961de4ba219d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.342+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c009c10 con 0x7f7b400039b0 2024-04-25T08:32:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.342+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4c036070 con 0x7f7b400039b0 2024-04-25T08:32:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.342+0000 7f7b537fe640 1 -- 172.21.15.38:0/2267496232 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4c031880 con 0x7f7b400039b0 2024-04-25T08:32:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.343+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/2267496232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b400039b0 msgr2=0x7f7b40003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.343+0000 7f7b5ad7e640 1 --2- 172.21.15.38:0/2267496232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b400039b0 0x7f7b40003dd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c009fc0 tx=0x7f7b40005830 comp rx=0 tx=0).stop 2024-04-25T08:32:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.343+0000 7f7b5931b640 1 -- 172.21.15.38:0/2267496232 reap_dead start 2024-04-25T08:32:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.343+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/2267496232 shutdown_connections 2024-04-25T08:32:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.343+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/2267496232 >> 172.21.15.38:0/2267496232 conn(0x7f7b540668f0 msgr2=0x7f7b54066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.343+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/2267496232 shutdown_connections 2024-04-25T08:32:30.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.343+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/2267496232 wait complete. 2024-04-25T08:32:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.344+0000 7f7b5ad7e640 1 Processor -- start 2024-04-25T08:32:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.344+0000 7f7b5ad7e640 1 -- start start 2024-04-25T08:32:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b5ad7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b400039b0 0x7f7b5415b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b5ad7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b540b5640 0x7f7b540b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b5ad7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540ac820 0x7f7b540acca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b5ad7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540d0b40 con 0x7f7b540b5640 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b5ad7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540d2eb0 con 0x7f7b400039b0 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b5ad7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540b42c0 con 0x7f7b540ac820 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b53fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b540b5640 0x7f7b540b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b53fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b540b5640 0x7f7b540b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44578/0 (socket says 172.21.15.38:44578) 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b53fff640 1 -- 172.21.15.38:0/1849197200 learned_addr learned my addr 172.21.15.38:0/1849197200 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b58b1a640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b400039b0 0x7f7b5415b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b5931b640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540ac820 0x7f7b540acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b53fff640 1 -- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540ac820 msgr2=0x7f7b540acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b53fff640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540ac820 0x7f7b540acca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b53fff640 1 -- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b400039b0 msgr2=0x7f7b5415b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b53fff640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b400039b0 0x7f7b5415b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b53fff640 1 -- 172.21.15.38:0/1849197200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b540b45a0 con 0x7f7b540b5640 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.345+0000 7f7b58b1a640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b400039b0 0x7f7b5415b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:30.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.346+0000 7f7b53fff640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b540b5640 0x7f7b540b5aa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7b44001050 tx=0x7f7b440077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.346+0000 7f7b517fa640 1 -- 172.21.15.38:0/1849197200 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b440016f0 con 0x7f7b540b5640 2024-04-25T08:32:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.346+0000 7f7b517fa640 1 -- 172.21.15.38:0/1849197200 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b44018070 con 0x7f7b540b5640 2024-04-25T08:32:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.346+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/1849197200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b540b4880 con 0x7f7b540b5640 2024-04-25T08:32:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.346+0000 7f7b517fa640 1 -- 172.21.15.38:0/1849197200 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4400fb80 con 0x7f7b540b5640 2024-04-25T08:32:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.346+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/1849197200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b540b4e40 con 0x7f7b540b5640 2024-04-25T08:32:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.347+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/1849197200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b54068e80 con 0x7f7b540b5640 2024-04-25T08:32:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.347+0000 7f7b517fa640 1 -- 172.21.15.38:0/1849197200 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b4400fd20 con 0x7f7b540b5640 2024-04-25T08:32:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.348+0000 7f7b517fa640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7b3406b4d0 0x7f7b3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.348+0000 7f7b517fa640 1 -- 172.21.15.38:0/1849197200 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f7b4408a660 con 0x7f7b540b5640 2024-04-25T08:32:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.348+0000 7f7b58b1a640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7b3406b4d0 0x7f7b3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.348+0000 7f7b58b1a640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7b3406b4d0 0x7f7b3406d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c033040 tx=0x7f7b4c02f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:30.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.352+0000 7f7b517fa640 1 -- 172.21.15.38:0/1849197200 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b44059070 con 0x7f7b540b5640 2024-04-25T08:32:30.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.535+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/1849197200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7b540b3050 con 0x7f7b540b5640 2024-04-25T08:32:30.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.535+0000 7f7b517fa640 1 -- 172.21.15.38:0/1849197200 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7b44058970 con 0x7f7b540b5640 2024-04-25T08:32:30.531 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:30.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.537+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7b3406b4d0 msgr2=0x7f7b3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.537+0000 7f7b5ad7e640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7b3406b4d0 0x7f7b3406d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c033040 tx=0x7f7b4c02f000 comp rx=0 tx=0).stop 2024-04-25T08:32:30.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.537+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/1849197200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b540b5640 msgr2=0x7f7b540b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.537+0000 7f7b5ad7e640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b540b5640 0x7f7b540b5aa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7b44001050 tx=0x7f7b440077a0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.538+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/1849197200 shutdown_connections 2024-04-25T08:32:30.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.538+0000 7f7b5ad7e640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7b3406b4d0 0x7f7b3406d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.538+0000 7f7b5ad7e640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b540ac820 0x7f7b540acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.538+0000 7f7b5ad7e640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b540b5640 0x7f7b540b5aa0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.538+0000 7f7b5ad7e640 1 --2- 172.21.15.38:0/1849197200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b400039b0 0x7f7b5415b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.538+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/1849197200 >> 172.21.15.38:0/1849197200 conn(0x7f7b540668f0 msgr2=0x7f7b540ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.538+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/1849197200 shutdown_connections 2024-04-25T08:32:30.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.538+0000 7f7b5ad7e640 1 -- 172.21.15.38:0/1849197200 wait complete. 2024-04-25T08:32:30.545 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:32:30.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.650+0000 7f76a349e640 1 Processor -- start 2024-04-25T08:32:30.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.651+0000 7f76a349e640 1 -- start start 2024-04-25T08:32:30.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.651+0000 7f76a349e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f769c06b5d0 0x7f769c0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.651+0000 7f76a349e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f769c0b1ad0 0x7f769c0cf910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.651+0000 7f76a349e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f769c0cfe50 0x7f769c1615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.651+0000 7f76a349e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c0b1f30 con 0x7f769c06b5d0 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.651+0000 7f76a349e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769c06b9b0 con 0x7f769c0cfe50 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.651+0000 7f76a349e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c15e270 con 0x7f769c0b1ad0 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.652+0000 7f76a123a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f769c06b5d0 0x7f769c0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.652+0000 7f76a123a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f769c06b5d0 0x7f769c0b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44588/0 (socket says 172.21.15.38:44588) 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.652+0000 7f76a123a640 1 -- 172.21.15.38:0/962074663 learned_addr learned my addr 172.21.15.38:0/962074663 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.652+0000 7f76a0a39640 1 --2- 172.21.15.38:0/962074663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f769c0b1ad0 0x7f769c0cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.652+0000 7f76a1a3b640 1 --2- 172.21.15.38:0/962074663 >> v2:172.21.15.107:3300/0 conn(0x7f769c0cfe50 0x7f769c1615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.652+0000 7f76a123a640 1 -- 172.21.15.38:0/962074663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f769c0b1ad0 msgr2=0x7f769c0cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.652+0000 7f76a123a640 1 --2- 172.21.15.38:0/962074663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f769c0b1ad0 0x7f769c0cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.652+0000 7f76a123a640 1 -- 172.21.15.38:0/962074663 >> v2:172.21.15.107:3300/0 conn(0x7f769c0cfe50 msgr2=0x7f769c1615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.652+0000 7f76a123a640 1 --2- 172.21.15.38:0/962074663 >> v2:172.21.15.107:3300/0 conn(0x7f769c0cfe50 0x7f769c1615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.652+0000 7f76a123a640 1 -- 172.21.15.38:0/962074663 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c15e550 con 0x7f769c06b5d0 2024-04-25T08:32:30.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.652+0000 7f76a0a39640 1 --2- 172.21.15.38:0/962074663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f769c0b1ad0 0x7f769c0cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:30.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.653+0000 7f76a123a640 1 --2- 172.21.15.38:0/962074663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f769c06b5d0 0x7f769c0b1590 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f769c0719c0 tx=0x7f769802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5684fa3b0e7de566 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:30.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.653+0000 7f7693fff640 1 -- 172.21.15.38:0/962074663 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698037070 con 0x7f769c06b5d0 2024-04-25T08:32:30.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.653+0000 7f7693fff640 1 -- 172.21.15.38:0/962074663 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7698032440 con 0x7f769c06b5d0 2024-04-25T08:32:30.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.653+0000 7f7693fff640 1 -- 172.21.15.38:0/962074663 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698032810 con 0x7f769c06b5d0 2024-04-25T08:32:30.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.653+0000 7f76a349e640 1 -- 172.21.15.38:0/962074663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f769c06b5d0 msgr2=0x7f769c0b1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.653+0000 7f76a349e640 1 --2- 172.21.15.38:0/962074663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f769c06b5d0 0x7f769c0b1590 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f769c0719c0 tx=0x7f769802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.654+0000 7f76a349e640 1 -- 172.21.15.38:0/962074663 shutdown_connections 2024-04-25T08:32:30.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.654+0000 7f76a349e640 1 --2- 172.21.15.38:0/962074663 >> v2:172.21.15.107:3300/0 conn(0x7f769c0cfe50 0x7f769c1615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.654+0000 7f76a349e640 1 --2- 172.21.15.38:0/962074663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f769c0b1ad0 0x7f769c0cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.654+0000 7f76a349e640 1 --2- 172.21.15.38:0/962074663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f769c06b5d0 0x7f769c0b1590 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.654+0000 7f76a349e640 1 -- 172.21.15.38:0/962074663 >> 172.21.15.38:0/962074663 conn(0x7f769c099a70 msgr2=0x7f769c0b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:30.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.654+0000 7f76a349e640 1 -- 172.21.15.38:0/962074663 shutdown_connections 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.654+0000 7f76a349e640 1 -- 172.21.15.38:0/962074663 wait complete. 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.655+0000 7f76a349e640 1 Processor -- start 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.655+0000 7f76a349e640 1 -- start start 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.655+0000 7f76a349e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f769c06b5d0 0x7f769c15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.655+0000 7f76a349e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f769c0b1ad0 0x7f769c161140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a349e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f769c0cfe50 0x7f769c161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a349e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c15edf0 con 0x7f769c0cfe50 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a349e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c15eb10 con 0x7f769c0b1ad0 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a349e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0d02a0 con 0x7f769c06b5d0 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a1a3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f769c0cfe50 0x7f769c161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a1a3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f769c0cfe50 0x7f769c161680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44596/0 (socket says 172.21.15.38:44596) 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a1a3b640 1 -- 172.21.15.38:0/2530081661 learned_addr learned my addr 172.21.15.38:0/2530081661 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:30.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a123a640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f769c06b5d0 0x7f769c15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a0a39640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f769c0b1ad0 0x7f769c161140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a1a3b640 1 -- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f769c06b5d0 msgr2=0x7f769c15f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a1a3b640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f769c06b5d0 0x7f769c15f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a1a3b640 1 -- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f769c0b1ad0 msgr2=0x7f769c161140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a1a3b640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f769c0b1ad0 0x7f769c161140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a1a3b640 1 -- 172.21.15.38:0/2530081661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769c1e8ef0 con 0x7f769c0cfe50 2024-04-25T08:32:30.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a1a3b640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f769c0cfe50 0x7f769c161680 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f769400efc0 tx=0x7f769400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:30.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f7691ffb640 1 -- 172.21.15.38:0/2530081661 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7694018020 con 0x7f769c0cfe50 2024-04-25T08:32:30.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f7691ffb640 1 -- 172.21.15.38:0/2530081661 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7694014480 con 0x7f769c0cfe50 2024-04-25T08:32:30.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f76a349e640 1 -- 172.21.15.38:0/2530081661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c1e91d0 con 0x7f769c0cfe50 2024-04-25T08:32:30.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.656+0000 7f7691ffb640 1 -- 172.21.15.38:0/2530081661 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7694014870 con 0x7f769c0cfe50 2024-04-25T08:32:30.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.657+0000 7f76a349e640 1 -- 172.21.15.38:0/2530081661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c1e9790 con 0x7f769c0cfe50 2024-04-25T08:32:30.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.658+0000 7f76a349e640 1 -- 172.21.15.38:0/2530081661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769c04ef90 con 0x7f769c0cfe50 2024-04-25T08:32:30.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.658+0000 7f7691ffb640 1 -- 172.21.15.38:0/2530081661 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f769401b020 con 0x7f769c0cfe50 2024-04-25T08:32:30.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.658+0000 7f7691ffb640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f768006b4d0 0x7f768006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:30.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.659+0000 7f7691ffb640 1 -- 172.21.15.38:0/2530081661 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f769408b410 con 0x7f769c0cfe50 2024-04-25T08:32:30.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.659+0000 7f76a123a640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f768006b4d0 0x7f768006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:30.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.659+0000 7f76a123a640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f768006b4d0 0x7f768006d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7698034000 tx=0x7f769802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:30.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.663+0000 7f7691ffb640 1 -- 172.21.15.38:0/2530081661 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7694022020 con 0x7f769c0cfe50 2024-04-25T08:32:30.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.853+0000 7f76a349e640 1 -- 172.21.15.38:0/2530081661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f769c099fc0 con 0x7f769c0cfe50 2024-04-25T08:32:30.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.854+0000 7f7691ffb640 1 -- 172.21.15.38:0/2530081661 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7694010040 con 0x7f769c0cfe50 2024-04-25T08:32:30.850 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:32:30.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.855+0000 7f76a349e640 1 -- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f768006b4d0 msgr2=0x7f768006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.855+0000 7f76a349e640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f768006b4d0 0x7f768006d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7698034000 tx=0x7f769802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:30.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.855+0000 7f76a349e640 1 -- 172.21.15.38:0/2530081661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f769c0cfe50 msgr2=0x7f769c161680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:30.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.855+0000 7f76a349e640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f769c0cfe50 0x7f769c161680 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f769400efc0 tx=0x7f769400c490 comp rx=0 tx=0).stop 2024-04-25T08:32:30.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.855+0000 7f76a349e640 1 -- 172.21.15.38:0/2530081661 shutdown_connections 2024-04-25T08:32:30.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.855+0000 7f76a349e640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f769c0cfe50 0x7f769c161680 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.855+0000 7f76a349e640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f769c0b1ad0 0x7f769c161140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.855+0000 7f76a349e640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f768006b4d0 0x7f768006d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.855+0000 7f76a349e640 1 --2- 172.21.15.38:0/2530081661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f769c06b5d0 0x7f769c15f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:30.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.855+0000 7f76a349e640 1 -- 172.21.15.38:0/2530081661 >> 172.21.15.38:0/2530081661 conn(0x7f769c099a70 msgr2=0x7f769c0af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:30.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.855+0000 7f76a349e640 1 -- 172.21.15.38:0/2530081661 shutdown_connections 2024-04-25T08:32:30.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:30.856+0000 7f76a349e640 1 -- 172.21.15.38:0/2530081661 wait complete. 2024-04-25T08:32:30.863 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:32:30.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:30.863 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T08:32:30.888 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:30.889 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:32:30.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:30.889 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:32:30.944 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:32:30.944 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:30.963 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:32:30.963 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:31.021 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:31.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.126+0000 7fbd87963640 1 Processor -- start 2024-04-25T08:32:31.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.126+0000 7fbd87963640 1 -- start start 2024-04-25T08:32:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.127+0000 7fbd87963640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd8015a8f0 0x7fbd800cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.127+0000 7fbd87963640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbd800d0950 0x7fbd800c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.127+0000 7fbd87963640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd800cbd20 con 0x7fbd800cb6c0 2024-04-25T08:32:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.127+0000 7fbd87963640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd800cc000 con 0x7fbd8015a8f0 2024-04-25T08:32:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.127+0000 7fbd87963640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd800cc2e0 con 0x7fbd800d0950 2024-04-25T08:32:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.127+0000 7fbd856ff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd8015a8f0 0x7fbd800cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.127+0000 7fbd856ff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd8015a8f0 0x7fbd800cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44598/0 (socket says 172.21.15.38:44598) 2024-04-25T08:32:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.127+0000 7fbd856ff640 1 -- 172.21.15.38:0/2100889312 learned_addr learned my addr 172.21.15.38:0/2100889312 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.128+0000 7fbd856ff640 1 -- 172.21.15.38:0/2100889312 >> v2:172.21.15.107:3300/0 conn(0x7fbd800d0950 msgr2=0x7fbd800c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.128+0000 7fbd856ff640 1 --2- 172.21.15.38:0/2100889312 >> v2:172.21.15.107:3300/0 conn(0x7fbd800d0950 0x7fbd800c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.128+0000 7fbd856ff640 1 -- 172.21.15.38:0/2100889312 >> v1:172.21.15.107:6789/0 conn(0x7fbd800cb6c0 legacy=0x7fbd800d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:31.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.128+0000 7fbd856ff640 1 -- 172.21.15.38:0/2100889312 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd800cc5c0 con 0x7fbd8015a8f0 2024-04-25T08:32:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.128+0000 7fbd856ff640 1 --2- 172.21.15.38:0/2100889312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd8015a8f0 0x7fbd800cb180 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbd800d0de0 tx=0x7fbd7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da47efa5e870a860 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.129+0000 7fbd7ffff640 1 -- 172.21.15.38:0/2100889312 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78037070 con 0x7fbd8015a8f0 2024-04-25T08:32:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.129+0000 7fbd7ffff640 1 -- 172.21.15.38:0/2100889312 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd78032440 con 0x7fbd8015a8f0 2024-04-25T08:32:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.129+0000 7fbd7ffff640 1 -- 172.21.15.38:0/2100889312 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd78032810 con 0x7fbd8015a8f0 2024-04-25T08:32:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.129+0000 7fbd87963640 1 -- 172.21.15.38:0/2100889312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd8015a8f0 msgr2=0x7fbd800cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.129+0000 7fbd87963640 1 --2- 172.21.15.38:0/2100889312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd8015a8f0 0x7fbd800cb180 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbd800d0de0 tx=0x7fbd7802a230 comp rx=0 tx=0).stop 2024-04-25T08:32:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.129+0000 7fbd87963640 1 -- 172.21.15.38:0/2100889312 shutdown_connections 2024-04-25T08:32:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.129+0000 7fbd87963640 1 --2- 172.21.15.38:0/2100889312 >> v2:172.21.15.107:3300/0 conn(0x7fbd800d0950 0x7fbd800c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.129+0000 7fbd87963640 1 --2- 172.21.15.38:0/2100889312 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd8015a8f0 0x7fbd800cb180 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.129+0000 7fbd87963640 1 -- 172.21.15.38:0/2100889312 >> 172.21.15.38:0/2100889312 conn(0x7fbd80072610 msgr2=0x7fbd80099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:31.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.129+0000 7fbd87963640 1 -- 172.21.15.38:0/2100889312 shutdown_connections 2024-04-25T08:32:31.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.129+0000 7fbd87963640 1 -- 172.21.15.38:0/2100889312 wait complete. 2024-04-25T08:32:31.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.130+0000 7fbd87963640 1 Processor -- start 2024-04-25T08:32:31.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.130+0000 7fbd87963640 1 -- start start 2024-04-25T08:32:31.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.131+0000 7fbd87963640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd800cb6c0 0x7fbd8014f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.131+0000 7fbd87963640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd800d0950 0x7fbd801f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.131+0000 7fbd87963640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd8015a8f0 0x7fbd801f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.131+0000 7fbd87963640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8015acd0 con 0x7fbd8015a8f0 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.131+0000 7fbd87963640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd800ce470 con 0x7fbd800cb6c0 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.131+0000 7fbd87963640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8015ae50 con 0x7fbd800d0950 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.131+0000 7fbd85f00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd8015a8f0 0x7fbd801f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.131+0000 7fbd85f00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd8015a8f0 0x7fbd801f51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44612/0 (socket says 172.21.15.38:44612) 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.131+0000 7fbd85f00640 1 -- 172.21.15.38:0/1431060413 learned_addr learned my addr 172.21.15.38:0/1431060413 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.131+0000 7fbd856ff640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd800cb6c0 0x7fbd8014f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd85f00640 1 -- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd800d0950 msgr2=0x7fbd801f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd84efe640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd800d0950 0x7fbd801f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd85f00640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd800d0950 0x7fbd801f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd85f00640 1 -- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd800cb6c0 msgr2=0x7fbd8014f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd85f00640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd800cb6c0 0x7fbd8014f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd85f00640 1 -- 172.21.15.38:0/1431060413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd800b5d40 con 0x7fbd8015a8f0 2024-04-25T08:32:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd856ff640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd800cb6c0 0x7fbd8014f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd84efe640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd800d0950 0x7fbd801f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd85f00640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd8015a8f0 0x7fbd801f51e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000b970 tx=0x7fbd7000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd7dffb640 1 -- 172.21.15.38:0/1431060413 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70019070 con 0x7fbd8015a8f0 2024-04-25T08:32:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd7dffb640 1 -- 172.21.15.38:0/1431060413 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7000cb60 con 0x7fbd8015a8f0 2024-04-25T08:32:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd87963640 1 -- 172.21.15.38:0/1431060413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd800b4000 con 0x7fbd8015a8f0 2024-04-25T08:32:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd7dffb640 1 -- 172.21.15.38:0/1431060413 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70014440 con 0x7fbd8015a8f0 2024-04-25T08:32:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.132+0000 7fbd87963640 1 -- 172.21.15.38:0/1431060413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd800b45c0 con 0x7fbd8015a8f0 2024-04-25T08:32:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.133+0000 7fbd87963640 1 -- 172.21.15.38:0/1431060413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd800b4790 con 0x7fbd8015a8f0 2024-04-25T08:32:31.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.134+0000 7fbd7dffb640 1 -- 172.21.15.38:0/1431060413 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd7001c070 con 0x7fbd8015a8f0 2024-04-25T08:32:31.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.134+0000 7fbd7dffb640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd6c06b4d0 0x7fbd6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.134+0000 7fbd7dffb640 1 -- 172.21.15.38:0/1431060413 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fbd7008a310 con 0x7fbd8015a8f0 2024-04-25T08:32:31.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.134+0000 7fbd856ff640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd6c06b4d0 0x7fbd6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.135+0000 7fbd856ff640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd6c06b4d0 0x7fbd6c06d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbd78034000 tx=0x7fbd7802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:31.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.138+0000 7fbd7dffb640 1 -- 172.21.15.38:0/1431060413 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd70058dc0 con 0x7fbd8015a8f0 2024-04-25T08:32:31.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.319+0000 7fbd87963640 1 -- 172.21.15.38:0/1431060413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd80159c30 con 0x7fbd8015a8f0 2024-04-25T08:32:31.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.320+0000 7fbd7dffb640 1 -- 172.21.15.38:0/1431060413 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18969 (secure 0 0 0) 0x7fbd700586c0 con 0x7fbd8015a8f0 2024-04-25T08:32:31.317 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:31.317 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:16.572337+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.a","mappings":[{"from":4,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.322+0000 7fbd87963640 1 -- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd6c06b4d0 msgr2=0x7fbd6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.322+0000 7fbd87963640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd6c06b4d0 0x7fbd6c06d990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbd78034000 tx=0x7fbd7802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.322+0000 7fbd87963640 1 -- 172.21.15.38:0/1431060413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd8015a8f0 msgr2=0x7fbd801f51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.322+0000 7fbd87963640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd8015a8f0 0x7fbd801f51e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000b970 tx=0x7fbd7000be30 comp rx=0 tx=0).stop 2024-04-25T08:32:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.323+0000 7fbd87963640 1 -- 172.21.15.38:0/1431060413 shutdown_connections 2024-04-25T08:32:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.323+0000 7fbd87963640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd8015a8f0 0x7fbd801f51e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.323+0000 7fbd87963640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd6c06b4d0 0x7fbd6c06d990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.323+0000 7fbd87963640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd800d0950 0x7fbd801f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.323+0000 7fbd87963640 1 --2- 172.21.15.38:0/1431060413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd800cb6c0 0x7fbd8014f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.323+0000 7fbd87963640 1 -- 172.21.15.38:0/1431060413 >> 172.21.15.38:0/1431060413 conn(0x7fbd80072610 msgr2=0x7fbd800c67b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.323+0000 7fbd87963640 1 -- 172.21.15.38:0/1431060413 shutdown_connections 2024-04-25T08:32:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.323+0000 7fbd87963640 1 -- 172.21.15.38:0/1431060413 wait complete. 2024-04-25T08:32:31.330 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:31.330 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:31.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.438+0000 7f834205d640 1 Processor -- start 2024-04-25T08:32:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.438+0000 7f834205d640 1 -- start start 2024-04-25T08:32:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.439+0000 7f834205d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f833c166610 0x7f833c168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.439+0000 7f834205d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c0d2060 con 0x7f833c0d33d0 2024-04-25T08:32:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.439+0000 7f834205d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c0d2330 con 0x7f833c0d0320 2024-04-25T08:32:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.439+0000 7f834205d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833c0d2610 con 0x7f833c166610 2024-04-25T08:32:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.439+0000 7f833affd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f833c0d33d0 0x7f833c0d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56450/0 (socket says 172.21.15.38:56450) 2024-04-25T08:32:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.439+0000 7f833affd640 1 -- 172.21.15.38:0/4269705120 learned_addr learned my addr 172.21.15.38:0/4269705120 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.439+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628624200 0 0) 0x7f833c0d2060 con 0x7f833c0d33d0 2024-04-25T08:32:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.439+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8324003520 con 0x7f833c0d33d0 2024-04-25T08:32:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.439+0000 7f833bfff640 1 --2- 172.21.15.38:0/4269705120 >> v2:172.21.15.107:3300/0 conn(0x7f833c166610 0x7f833c168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4054268341 0 0) 0x7f8324003520 con 0x7f833c0d33d0 2024-04-25T08:32:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 >> v2:172.21.15.107:3300/0 conn(0x7f833c166610 msgr2=0x7f833c168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 --2- 172.21.15.38:0/4269705120 >> v2:172.21.15.107:3300/0 conn(0x7f833c166610 0x7f833c168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 >> v1:172.21.15.107:6789/0 conn(0x7f833c0d0320 legacy=0x7f833c0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c0d28f0 con 0x7f833c0d33d0 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f8330001670 con 0x7f833c0d33d0 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 >> v1:172.21.15.38:6789/0 conn(0x7f833c0d33d0 legacy=0x7f833c0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 --2- 172.21.15.38:0/4269705120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83240039b0 0x7f8324003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 --2- 172.21.15.38:0/4269705120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8324004800 0x7f8324004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 --2- 172.21.15.38:0/4269705120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 0x7f832400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c0d2610 con 0x7f83240051c0 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832400c0f0 con 0x7f83240039b0 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832400c2f0 con 0x7f8324004800 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833bfff640 1 --2- 172.21.15.38:0/4269705120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 0x7f832400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833b7fe640 1 --2- 172.21.15.38:0/4269705120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83240039b0 0x7f8324003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833affd640 1 --2- 172.21.15.38:0/4269705120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8324004800 0x7f8324004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833bfff640 1 -- 172.21.15.38:0/4269705120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8324004800 msgr2=0x7f8324004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833bfff640 1 --2- 172.21.15.38:0/4269705120 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8324004800 0x7f8324004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833bfff640 1 -- 172.21.15.38:0/4269705120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83240039b0 msgr2=0x7f8324003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833bfff640 1 --2- 172.21.15.38:0/4269705120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83240039b0 0x7f8324003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.440+0000 7f833bfff640 1 -- 172.21.15.38:0/4269705120 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8324003520 con 0x7f83240051c0 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.441+0000 7f833bfff640 1 -- 172.21.15.38:0/4269705120 reap_dead start 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.441+0000 7f833bfff640 1 --2- 172.21.15.38:0/4269705120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 0x7f832400b9c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8334007330 tx=0x7f8324005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08e4607b6f8c68c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.441+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334009be0 con 0x7f83240051c0 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.441+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334003b90 con 0x7f83240051c0 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.441+0000 7f833a7fc640 1 -- 172.21.15.38:0/4269705120 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334004020 con 0x7f83240051c0 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.442+0000 7f834205d640 1 -- 172.21.15.38:0/4269705120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 msgr2=0x7f832400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.442+0000 7f834205d640 1 --2- 172.21.15.38:0/4269705120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 0x7f832400b9c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8334007330 tx=0x7f8324005830 comp rx=0 tx=0).stop 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.442+0000 7f834205d640 1 -- 172.21.15.38:0/4269705120 shutdown_connections 2024-04-25T08:32:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.442+0000 7f834205d640 1 --2- 172.21.15.38:0/4269705120 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 0x7f832400b9c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.442+0000 7f834205d640 1 -- 172.21.15.38:0/4269705120 >> 172.21.15.38:0/4269705120 conn(0x7f833c0668f0 msgr2=0x7f833c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:31.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.442+0000 7f834205d640 1 -- 172.21.15.38:0/4269705120 shutdown_connections 2024-04-25T08:32:31.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.442+0000 7f834205d640 1 -- 172.21.15.38:0/4269705120 wait complete. 2024-04-25T08:32:31.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.443+0000 7f834205d640 1 Processor -- start 2024-04-25T08:32:31.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.443+0000 7f834205d640 1 -- start start 2024-04-25T08:32:31.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f834205d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 0x7f833c15ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f834205d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f833c0c9100 0x7f833c0b7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f834205d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f833c0b8280 0x7f833c1537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f834205d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c161210 con 0x7f83240051c0 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f834205d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c0d2eb0 con 0x7f833c0c9100 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f834205d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833c161390 con 0x7f833c0b8280 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f833b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 0x7f833c15ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f833b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 0x7f833c15ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44628/0 (socket says 172.21.15.38:44628) 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f833b7fe640 1 -- 172.21.15.38:0/2785089797 learned_addr learned my addr 172.21.15.38:0/2785089797 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f833affd640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f833c0c9100 0x7f833c0b7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f833bfff640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f833c0b8280 0x7f833c1537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f833b7fe640 1 -- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f833c0b8280 msgr2=0x7f833c1537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f833b7fe640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f833c0b8280 0x7f833c1537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f833b7fe640 1 -- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f833c0c9100 msgr2=0x7f833c0b7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f833b7fe640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f833c0c9100 0x7f833c0b7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f833b7fe640 1 -- 172.21.15.38:0/2785089797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c153300 con 0x7f83240051c0 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f833affd640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f833c0c9100 0x7f833c0b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f833b7fe640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 0x7f833c15ab80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f832c00f620 tx=0x7f832c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f8323fff640 1 -- 172.21.15.38:0/2785089797 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c0052e0 con 0x7f83240051c0 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.444+0000 7f8323fff640 1 -- 172.21.15.38:0/2785089797 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c018070 con 0x7f83240051c0 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.445+0000 7f834205d640 1 -- 172.21.15.38:0/2785089797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833c158f80 con 0x7f83240051c0 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.445+0000 7f8323fff640 1 -- 172.21.15.38:0/2785089797 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c003810 con 0x7f83240051c0 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.445+0000 7f834205d640 1 -- 172.21.15.38:0/2785089797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c157b70 con 0x7f83240051c0 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.446+0000 7f834205d640 1 -- 172.21.15.38:0/2785089797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833c068e80 con 0x7f83240051c0 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.446+0000 7f8323fff640 1 -- 172.21.15.38:0/2785089797 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f832c0039b0 con 0x7f83240051c0 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.446+0000 7f8323fff640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f832806b500 0x7f832806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.447+0000 7f8323fff640 1 -- 172.21.15.38:0/2785089797 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f832c08a3e0 con 0x7f83240051c0 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.447+0000 7f833affd640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f832806b500 0x7f832806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.447+0000 7f833affd640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f832806b500 0x7f832806d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8330001200 tx=0x7f8330000f30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:31.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.451+0000 7f8323fff640 1 -- 172.21.15.38:0/2785089797 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f832c058e00 con 0x7f83240051c0 2024-04-25T08:32:31.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.630+0000 7f834205d640 1 -- 172.21.15.38:0/2785089797 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f833c0b8660 con 0x7f83240051c0 2024-04-25T08:32:31.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.631+0000 7f8323fff640 1 -- 172.21.15.38:0/2785089797 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f832c058700 con 0x7f83240051c0 2024-04-25T08:32:31.627 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.632+0000 7f834205d640 1 -- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f832806b500 msgr2=0x7f832806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.632+0000 7f834205d640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f832806b500 0x7f832806d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8330001200 tx=0x7f8330000f30 comp rx=0 tx=0).stop 2024-04-25T08:32:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.632+0000 7f834205d640 1 -- 172.21.15.38:0/2785089797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 msgr2=0x7f833c15ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.632+0000 7f834205d640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 0x7f833c15ab80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f832c00f620 tx=0x7f832c00fae0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.632+0000 7f834205d640 1 -- 172.21.15.38:0/2785089797 shutdown_connections 2024-04-25T08:32:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.632+0000 7f834205d640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f832806b500 0x7f832806d9c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.632+0000 7f834205d640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f833c0b8280 0x7f833c1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.632+0000 7f834205d640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f833c0c9100 0x7f833c0b7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.632+0000 7f834205d640 1 --2- 172.21.15.38:0/2785089797 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83240051c0 0x7f833c15ab80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.632+0000 7f834205d640 1 -- 172.21.15.38:0/2785089797 >> 172.21.15.38:0/2785089797 conn(0x7f833c0668f0 msgr2=0x7f833c15e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.633+0000 7f834205d640 1 -- 172.21.15.38:0/2785089797 shutdown_connections 2024-04-25T08:32:31.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.633+0000 7f834205d640 1 -- 172.21.15.38:0/2785089797 wait complete. 2024-04-25T08:32:31.640 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.747+0000 7f4b7fbfe640 1 Processor -- start 2024-04-25T08:32:31.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.748+0000 7f4b7fbfe640 1 -- start start 2024-04-25T08:32:31.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.748+0000 7f4b7fbfe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4b7815b990 0x7f4b780cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.748+0000 7f4b7fbfe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4b780d1d20 0x7f4b780b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.748+0000 7f4b7fbfe640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b7815fd00 con 0x7f4b7815f5f0 2024-04-25T08:32:31.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.748+0000 7f4b7fbfe640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b78160580 con 0x7f4b7815b990 2024-04-25T08:32:31.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.748+0000 7f4b7fbfe640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b7815fa30 con 0x7f4b780d1d20 2024-04-25T08:32:31.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.749+0000 7f4b7d99a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4b7815b990 0x7f4b780cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.749+0000 7f4b7d99a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4b7815b990 0x7f4b780cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44638/0 (socket says 172.21.15.38:44638) 2024-04-25T08:32:31.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.749+0000 7f4b7d99a640 1 -- 172.21.15.38:0/1910751738 learned_addr learned my addr 172.21.15.38:0/1910751738 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.749+0000 7f4b7e19b640 1 --2- 172.21.15.38:0/1910751738 >> v2:172.21.15.107:3300/0 conn(0x7f4b780d1d20 0x7f4b780b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.749+0000 7f4b7d99a640 1 -- 172.21.15.38:0/1910751738 >> v2:172.21.15.107:3300/0 conn(0x7f4b780d1d20 msgr2=0x7f4b780b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.749+0000 7f4b7d99a640 1 --2- 172.21.15.38:0/1910751738 >> v2:172.21.15.107:3300/0 conn(0x7f4b780d1d20 0x7f4b780b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.749+0000 7f4b7d99a640 1 -- 172.21.15.38:0/1910751738 >> v1:172.21.15.107:6789/0 conn(0x7f4b7815f5f0 legacy=0x7f4b780d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:31.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.749+0000 7f4b7d99a640 1 -- 172.21.15.38:0/1910751738 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7815ffc0 con 0x7f4b7815b990 2024-04-25T08:32:31.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.750+0000 7f4b7d99a640 1 --2- 172.21.15.38:0/1910751738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b7815b990 0x7f4b780cf580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4b78073db0 tx=0x7f4b6802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7127ac42829f8f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:31.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.750+0000 7f4b7c998640 1 -- 172.21.15.38:0/1910751738 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68036070 con 0x7f4b7815b990 2024-04-25T08:32:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.750+0000 7f4b7c998640 1 -- 172.21.15.38:0/1910751738 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b68031600 con 0x7f4b7815b990 2024-04-25T08:32:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.750+0000 7f4b7c998640 1 -- 172.21.15.38:0/1910751738 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b680319d0 con 0x7f4b7815b990 2024-04-25T08:32:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.750+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/1910751738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b7815b990 msgr2=0x7f4b780cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.750+0000 7f4b7fbfe640 1 --2- 172.21.15.38:0/1910751738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b7815b990 0x7f4b780cf580 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4b78073db0 tx=0x7f4b6802a660 comp rx=0 tx=0).stop 2024-04-25T08:32:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.750+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/1910751738 shutdown_connections 2024-04-25T08:32:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.750+0000 7f4b7fbfe640 1 --2- 172.21.15.38:0/1910751738 >> v2:172.21.15.107:3300/0 conn(0x7f4b780d1d20 0x7f4b780b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.750+0000 7f4b7fbfe640 1 --2- 172.21.15.38:0/1910751738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b7815b990 0x7f4b780cf580 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.750+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/1910751738 >> 172.21.15.38:0/1910751738 conn(0x7f4b7808b000 msgr2=0x7f4b7808b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.750+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/1910751738 shutdown_connections 2024-04-25T08:32:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.750+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/1910751738 wait complete. 2024-04-25T08:32:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.752+0000 7f4b7fbfe640 1 Processor -- start 2024-04-25T08:32:31.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.752+0000 7f4b7fbfe640 1 -- start start 2024-04-25T08:32:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.752+0000 7f4b7fbfe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b780d1d20 0x7f4b780b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.752+0000 7f4b7fbfe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b7815b990 0x7f4b780b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.752+0000 7f4b7fbfe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b7815f5f0 0x7f4b780ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.752+0000 7f4b7fbfe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7815bd70 con 0x7f4b7815f5f0 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.752+0000 7f4b7fbfe640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b780b1ac0 con 0x7f4b780d1d20 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.752+0000 7f4b7fbfe640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7815bef0 con 0x7f4b7815b990 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7e19b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b7815f5f0 0x7f4b780ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7e19b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b7815f5f0 0x7f4b780ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44644/0 (socket says 172.21.15.38:44644) 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7e19b640 1 -- 172.21.15.38:0/771792198 learned_addr learned my addr 172.21.15.38:0/771792198 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7d99a640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b780d1d20 0x7f4b780b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7d199640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b7815b990 0x7f4b780b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7e19b640 1 -- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b7815b990 msgr2=0x7f4b780b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7e19b640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b7815b990 0x7f4b780b62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7e19b640 1 -- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b780d1d20 msgr2=0x7f4b780b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7e19b640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b780d1d20 0x7f4b780b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7e19b640 1 -- 172.21.15.38:0/771792198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7815c540 con 0x7f4b7815f5f0 2024-04-25T08:32:31.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7e19b640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b7815f5f0 0x7f4b780ca9a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000d5d0 tx=0x7f4b7000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b767fc640 1 -- 172.21.15.38:0/771792198 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b700125f0 con 0x7f4b7815f5f0 2024-04-25T08:32:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b767fc640 1 -- 172.21.15.38:0/771792198 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b70019070 con 0x7f4b7815f5f0 2024-04-25T08:32:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b767fc640 1 -- 172.21.15.38:0/771792198 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b700129d0 con 0x7f4b7815f5f0 2024-04-25T08:32:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/771792198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7815c820 con 0x7f4b7815f5f0 2024-04-25T08:32:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.753+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/771792198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7815cde0 con 0x7f4b7815f5f0 2024-04-25T08:32:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.754+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/771792198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7815cfe0 con 0x7f4b7815f5f0 2024-04-25T08:32:31.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.755+0000 7f4b767fc640 1 -- 172.21.15.38:0/771792198 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b70012b70 con 0x7f4b7815f5f0 2024-04-25T08:32:31.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.755+0000 7f4b767fc640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b6406b4d0 0x7f4b6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:31.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.755+0000 7f4b767fc640 1 -- 172.21.15.38:0/771792198 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f4b7008a900 con 0x7f4b7815f5f0 2024-04-25T08:32:31.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.755+0000 7f4b7d99a640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b6406b4d0 0x7f4b6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:31.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.756+0000 7f4b7d99a640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b6406b4d0 0x7f4b6406d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4b68033000 tx=0x7f4b68009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:31.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.759+0000 7f4b767fc640 1 -- 172.21.15.38:0/771792198 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b7815cfe0 con 0x7f4b7815f5f0 2024-04-25T08:32:31.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.938+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/771792198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b78068e80 con 0x7f4b7815f5f0 2024-04-25T08:32:31.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.938+0000 7f4b767fc640 1 -- 172.21.15.38:0/771792198 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b70058cd0 con 0x7f4b7815f5f0 2024-04-25T08:32:31.934 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:31.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.940+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b6406b4d0 msgr2=0x7f4b6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.940+0000 7f4b7fbfe640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b6406b4d0 0x7f4b6406d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4b68033000 tx=0x7f4b68009000 comp rx=0 tx=0).stop 2024-04-25T08:32:31.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.940+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/771792198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b7815f5f0 msgr2=0x7f4b780ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:31.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.940+0000 7f4b7fbfe640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b7815f5f0 0x7f4b780ca9a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000d5d0 tx=0x7f4b7000da90 comp rx=0 tx=0).stop 2024-04-25T08:32:31.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.940+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/771792198 shutdown_connections 2024-04-25T08:32:31.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.940+0000 7f4b7fbfe640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b7815f5f0 0x7f4b780ca9a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.940+0000 7f4b7fbfe640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b6406b4d0 0x7f4b6406d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.940+0000 7f4b7fbfe640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b7815b990 0x7f4b780b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.940+0000 7f4b7fbfe640 1 --2- 172.21.15.38:0/771792198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b780d1d20 0x7f4b780b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:31.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.940+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/771792198 >> 172.21.15.38:0/771792198 conn(0x7f4b7808b000 msgr2=0x7f4b7806f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:31.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.940+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/771792198 shutdown_connections 2024-04-25T08:32:31.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:31.940+0000 7f4b7fbfe640 1 -- 172.21.15.38:0/771792198 wait complete. 2024-04-25T08:32:31.947 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.055+0000 7f043a4c6640 1 Processor -- start 2024-04-25T08:32:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.055+0000 7f043a4c6640 1 -- start start 2024-04-25T08:32:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f043a4c6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f04340d33d0 0x7f04340d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f043a4c6640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f04340d1e90 0x7f04340b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f043a4c6640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04340d2330 con 0x7f04340d0320 2024-04-25T08:32:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f043a4c6640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04340d2610 con 0x7f04340d33d0 2024-04-25T08:32:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f043a4c6640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04340d28f0 con 0x7f04340d1e90 2024-04-25T08:32:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f04337fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f04340d33d0 0x7f04340d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f04337fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f04340d33d0 0x7f04340d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44648/0 (socket says 172.21.15.38:44648) 2024-04-25T08:32:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f04337fe640 1 -- 172.21.15.38:0/1056688962 learned_addr learned my addr 172.21.15.38:0/1056688962 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f0438a63640 1 --2- 172.21.15.38:0/1056688962 >> v2:172.21.15.107:3300/0 conn(0x7f04340d1e90 0x7f04340b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f04337fe640 1 -- 172.21.15.38:0/1056688962 >> v2:172.21.15.107:3300/0 conn(0x7f04340d1e90 msgr2=0x7f04340b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f04337fe640 1 --2- 172.21.15.38:0/1056688962 >> v2:172.21.15.107:3300/0 conn(0x7f04340d1e90 0x7f04340b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f04337fe640 1 -- 172.21.15.38:0/1056688962 >> v1:172.21.15.107:6789/0 conn(0x7f04340d0320 legacy=0x7f04340d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.056+0000 7f04337fe640 1 -- 172.21.15.38:0/1056688962 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04340d2bd0 con 0x7f04340d33d0 2024-04-25T08:32:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.057+0000 7f04337fe640 1 --2- 172.21.15.38:0/1056688962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d33d0 0x7f04340d1950 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f04340ca800 tx=0x7f042402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9695dc93b733e4ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.057+0000 7f0432ffd640 1 -- 172.21.15.38:0/1056688962 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424037070 con 0x7f04340d33d0 2024-04-25T08:32:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.057+0000 7f0432ffd640 1 -- 172.21.15.38:0/1056688962 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042402ae50 con 0x7f04340d33d0 2024-04-25T08:32:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.057+0000 7f0432ffd640 1 -- 172.21.15.38:0/1056688962 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424032860 con 0x7f04340d33d0 2024-04-25T08:32:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.057+0000 7f043a4c6640 1 -- 172.21.15.38:0/1056688962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d33d0 msgr2=0x7f04340d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.057+0000 7f043a4c6640 1 --2- 172.21.15.38:0/1056688962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d33d0 0x7f04340d1950 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f04340ca800 tx=0x7f042402a300 comp rx=0 tx=0).stop 2024-04-25T08:32:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.058+0000 7f043a4c6640 1 -- 172.21.15.38:0/1056688962 shutdown_connections 2024-04-25T08:32:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.058+0000 7f043a4c6640 1 --2- 172.21.15.38:0/1056688962 >> v2:172.21.15.107:3300/0 conn(0x7f04340d1e90 0x7f04340b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.058+0000 7f043a4c6640 1 --2- 172.21.15.38:0/1056688962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d33d0 0x7f04340d1950 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.058+0000 7f043a4c6640 1 -- 172.21.15.38:0/1056688962 >> 172.21.15.38:0/1056688962 conn(0x7f04340668f0 msgr2=0x7f0434066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.058+0000 7f043a4c6640 1 -- 172.21.15.38:0/1056688962 shutdown_connections 2024-04-25T08:32:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.058+0000 7f043a4c6640 1 -- 172.21.15.38:0/1056688962 wait complete. 2024-04-25T08:32:32.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.059+0000 7f043a4c6640 1 Processor -- start 2024-04-25T08:32:32.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.059+0000 7f043a4c6640 1 -- start start 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f043a4c6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04340d0320 0x7f043414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f043a4c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d1e90 0x7f043415c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f043a4c6640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04340d33d0 0x7f043415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f043a4c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0434161390 con 0x7f04340d1e90 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f043a4c6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0434161210 con 0x7f04340d0320 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f043a4c6640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04340b9720 con 0x7f04340d33d0 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f04337fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d1e90 0x7f043415c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f04337fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d1e90 0x7f043415c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44664/0 (socket says 172.21.15.38:44664) 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f04337fe640 1 -- 172.21.15.38:0/2272710484 learned_addr learned my addr 172.21.15.38:0/2272710484 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f0433fff640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04340d0320 0x7f043414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f0438a63640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04340d33d0 0x7f043415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f04337fe640 1 -- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04340d33d0 msgr2=0x7f043415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f04337fe640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04340d33d0 0x7f043415cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f04337fe640 1 -- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04340d0320 msgr2=0x7f043414eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f04337fe640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04340d0320 0x7f043414eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f04337fe640 1 -- 172.21.15.38:0/2272710484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0434159a90 con 0x7f04340d1e90 2024-04-25T08:32:32.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f04337fe640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d1e90 0x7f043415c740 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f042402a340 tx=0x7f04240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:32.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f0430ff9640 1 -- 172.21.15.38:0/2272710484 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424041070 con 0x7f04340d1e90 2024-04-25T08:32:32.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f0430ff9640 1 -- 172.21.15.38:0/2272710484 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f042403c660 con 0x7f04340d1e90 2024-04-25T08:32:32.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f043a4c6640 1 -- 172.21.15.38:0/2272710484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04341589f0 con 0x7f04340d1e90 2024-04-25T08:32:32.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.060+0000 7f0430ff9640 1 -- 172.21.15.38:0/2272710484 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424037040 con 0x7f04340d1e90 2024-04-25T08:32:32.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.061+0000 7f043a4c6640 1 -- 172.21.15.38:0/2272710484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0434158fb0 con 0x7f04340d1e90 2024-04-25T08:32:32.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.061+0000 7f043a4c6640 1 -- 172.21.15.38:0/2272710484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0434068e80 con 0x7f04340d1e90 2024-04-25T08:32:32.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.062+0000 7f0430ff9640 1 -- 172.21.15.38:0/2272710484 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0424044070 con 0x7f04340d1e90 2024-04-25T08:32:32.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.062+0000 7f0430ff9640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f041406b4d0 0x7f041406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.062+0000 7f0430ff9640 1 -- 172.21.15.38:0/2272710484 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f04240aae30 con 0x7f04340d1e90 2024-04-25T08:32:32.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.062+0000 7f0433fff640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f041406b4d0 0x7f041406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.063+0000 7f0433fff640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f041406b4d0 0x7f041406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f041c001780 tx=0x7f041c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:32.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.066+0000 7f0430ff9640 1 -- 172.21.15.38:0/2272710484 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0424079200 con 0x7f04340d1e90 2024-04-25T08:32:32.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.248+0000 7f043a4c6640 1 -- 172.21.15.38:0/2272710484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f043415da20 con 0x7f04340d1e90 2024-04-25T08:32:32.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.249+0000 7f0430ff9640 1 -- 172.21.15.38:0/2272710484 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f042407d2a0 con 0x7f04340d1e90 2024-04-25T08:32:32.245 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.251+0000 7f043a4c6640 1 -- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f041406b4d0 msgr2=0x7f041406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.251+0000 7f043a4c6640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f041406b4d0 0x7f041406d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f041c001780 tx=0x7f041c009040 comp rx=0 tx=0).stop 2024-04-25T08:32:32.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.251+0000 7f043a4c6640 1 -- 172.21.15.38:0/2272710484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d1e90 msgr2=0x7f043415c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.251+0000 7f043a4c6640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d1e90 0x7f043415c740 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f042402a340 tx=0x7f04240046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.251+0000 7f043a4c6640 1 -- 172.21.15.38:0/2272710484 shutdown_connections 2024-04-25T08:32:32.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.251+0000 7f043a4c6640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f041406b4d0 0x7f041406d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.251+0000 7f043a4c6640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04340d33d0 0x7f043415cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.251+0000 7f043a4c6640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04340d1e90 0x7f043415c740 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.251+0000 7f043a4c6640 1 --2- 172.21.15.38:0/2272710484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04340d0320 0x7f043414eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.251+0000 7f043a4c6640 1 -- 172.21.15.38:0/2272710484 >> 172.21.15.38:0/2272710484 conn(0x7f04340668f0 msgr2=0x7f04340cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:32.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.251+0000 7f043a4c6640 1 -- 172.21.15.38:0/2272710484 shutdown_connections 2024-04-25T08:32:32.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.251+0000 7f043a4c6640 1 -- 172.21.15.38:0/2272710484 wait complete. 2024-04-25T08:32:32.258 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:32.298 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:32:32.303 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:32:32.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.369+0000 7f34d584a640 1 Processor -- start 2024-04-25T08:32:32.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.369+0000 7f34d584a640 1 -- start start 2024-04-25T08:32:32.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.369+0000 7f34d584a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f34d00d33d0 0x7f34d00d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.369+0000 7f34d584a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f34d00d1e90 0x7f34d00b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.369+0000 7f34d584a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d00d2330 con 0x7f34d00d0320 2024-04-25T08:32:32.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.369+0000 7f34d584a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d00d2610 con 0x7f34d00d1e90 2024-04-25T08:32:32.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.369+0000 7f34d584a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d00d28f0 con 0x7f34d00d33d0 2024-04-25T08:32:32.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.369+0000 7f34cf7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f34d00d1e90 0x7f34d00b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.370+0000 7f34cf7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f34d00d1e90 0x7f34d00b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44668/0 (socket says 172.21.15.38:44668) 2024-04-25T08:32:32.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.370+0000 7f34cf7fe640 1 -- 172.21.15.38:0/870463415 learned_addr learned my addr 172.21.15.38:0/870463415 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:32.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.370+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/870463415 >> v2:172.21.15.107:3300/0 conn(0x7f34d00d33d0 0x7f34d00d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.370+0000 7f34cf7fe640 1 -- 172.21.15.38:0/870463415 >> v2:172.21.15.107:3300/0 conn(0x7f34d00d33d0 msgr2=0x7f34d00d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:32.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.370+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/870463415 >> v2:172.21.15.107:3300/0 conn(0x7f34d00d33d0 0x7f34d00d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.370+0000 7f34cf7fe640 1 -- 172.21.15.38:0/870463415 >> v1:172.21.15.107:6789/0 conn(0x7f34d00d0320 legacy=0x7f34d00d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.370+0000 7f34cf7fe640 1 -- 172.21.15.38:0/870463415 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d00d2bd0 con 0x7f34d00d1e90 2024-04-25T08:32:32.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.370+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/870463415 >> v2:172.21.15.107:3300/0 conn(0x7f34d00d33d0 0x7f34d00d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:32.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.371+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/870463415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d00d1e90 0x7f34d00b6980 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f34d00b6410 tx=0x7f34c802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66b8fcdc5b1ca73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.371+0000 7f34cdffb640 1 -- 172.21.15.38:0/870463415 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8036070 con 0x7f34d00d1e90 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.371+0000 7f34cdffb640 1 -- 172.21.15.38:0/870463415 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c8031600 con 0x7f34d00d1e90 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.371+0000 7f34cdffb640 1 -- 172.21.15.38:0/870463415 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c80319f0 con 0x7f34d00d1e90 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.371+0000 7f34d584a640 1 -- 172.21.15.38:0/870463415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d00d1e90 msgr2=0x7f34d00b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.371+0000 7f34d584a640 1 --2- 172.21.15.38:0/870463415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d00d1e90 0x7f34d00b6980 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f34d00b6410 tx=0x7f34c802a7c0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.371+0000 7f34d584a640 1 -- 172.21.15.38:0/870463415 shutdown_connections 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.371+0000 7f34d584a640 1 --2- 172.21.15.38:0/870463415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d00d1e90 0x7f34d00b6980 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.371+0000 7f34d584a640 1 --2- 172.21.15.38:0/870463415 >> v2:172.21.15.107:3300/0 conn(0x7f34d00d33d0 0x7f34d00d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.371+0000 7f34d584a640 1 -- 172.21.15.38:0/870463415 >> 172.21.15.38:0/870463415 conn(0x7f34d00668f0 msgr2=0x7f34d0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.371+0000 7f34d584a640 1 -- 172.21.15.38:0/870463415 shutdown_connections 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.371+0000 7f34d584a640 1 -- 172.21.15.38:0/870463415 wait complete. 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.373+0000 7f34d584a640 1 Processor -- start 2024-04-25T08:32:32.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.373+0000 7f34d584a640 1 -- start start 2024-04-25T08:32:32.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.373+0000 7f34d584a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34d00d0320 0x7f34d00c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.373+0000 7f34d584a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d00d1e90 0x7f34d00ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.373+0000 7f34d584a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d00d33d0 0x7f34d00ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.373+0000 7f34d584a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d0161390 con 0x7f34d00d33d0 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.373+0000 7f34d584a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d0161210 con 0x7f34d00d1e90 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.373+0000 7f34d584a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d00b9720 con 0x7f34d00d0320 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d00d33d0 0x7f34d00ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d00d33d0 0x7f34d00ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44682/0 (socket says 172.21.15.38:44682) 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34cf7fe640 1 -- 172.21.15.38:0/798944045 learned_addr learned my addr 172.21.15.38:0/798944045 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d00d1e90 0x7f34d00ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34ceffd640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34d00d0320 0x7f34d00c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34cf7fe640 1 -- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34d00d0320 msgr2=0x7f34d00c93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34d00d0320 0x7f34d00c93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34cf7fe640 1 -- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d00d1e90 msgr2=0x7f34d00ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d00d1e90 0x7f34d00ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34cf7fe640 1 -- 172.21.15.38:0/798944045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d00b43f0 con 0x7f34d00d33d0 2024-04-25T08:32:32.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d00d1e90 0x7f34d00ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:32.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d00d33d0 0x7f34d00ce8b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f34c802a800 tx=0x7f34c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:32.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34b7fff640 1 -- 172.21.15.38:0/798944045 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8036050 con 0x7f34d00d33d0 2024-04-25T08:32:32.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34b7fff640 1 -- 172.21.15.38:0/798944045 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c8042070 con 0x7f34d00d33d0 2024-04-25T08:32:32.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34d584a640 1 -- 172.21.15.38:0/798944045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d00b46d0 con 0x7f34d00d33d0 2024-04-25T08:32:32.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34b7fff640 1 -- 172.21.15.38:0/798944045 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8009510 con 0x7f34d00d33d0 2024-04-25T08:32:32.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.374+0000 7f34d584a640 1 -- 172.21.15.38:0/798944045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d00b4c90 con 0x7f34d00d33d0 2024-04-25T08:32:32.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.375+0000 7f34d584a640 1 -- 172.21.15.38:0/798944045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d0068e80 con 0x7f34d00d33d0 2024-04-25T08:32:32.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.376+0000 7f34b7fff640 1 -- 172.21.15.38:0/798944045 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34c8004470 con 0x7f34d00d33d0 2024-04-25T08:32:32.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.376+0000 7f34b7fff640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f34bc06b4d0 0x7f34bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.376+0000 7f34b7fff640 1 -- 172.21.15.38:0/798944045 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f34c80ab480 con 0x7f34d00d33d0 2024-04-25T08:32:32.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.377+0000 7f34ceffd640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f34bc06b4d0 0x7f34bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.377+0000 7f34ceffd640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f34bc06b4d0 0x7f34bc06d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f34c0001620 tx=0x7f34c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:32.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.380+0000 7f34b7fff640 1 -- 172.21.15.38:0/798944045 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34c807a050 con 0x7f34d00d33d0 2024-04-25T08:32:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.558+0000 7f34d584a640 1 -- 172.21.15.38:0/798944045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f34d00ca0e0 con 0x7f34d00d33d0 2024-04-25T08:32:32.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.558+0000 7f34b7fff640 1 -- 172.21.15.38:0/798944045 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f34c8079950 con 0x7f34d00d33d0 2024-04-25T08:32:32.554 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:32.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.560+0000 7f34d584a640 1 -- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f34bc06b4d0 msgr2=0x7f34bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.560+0000 7f34d584a640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f34bc06b4d0 0x7f34bc06d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f34c0001620 tx=0x7f34c0009040 comp rx=0 tx=0).stop 2024-04-25T08:32:32.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.560+0000 7f34d584a640 1 -- 172.21.15.38:0/798944045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d00d33d0 msgr2=0x7f34d00ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.560+0000 7f34d584a640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d00d33d0 0x7f34d00ce8b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f34c802a800 tx=0x7f34c8004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:32.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.560+0000 7f34d584a640 1 -- 172.21.15.38:0/798944045 shutdown_connections 2024-04-25T08:32:32.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.560+0000 7f34d584a640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d00d33d0 0x7f34d00ce8b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.560+0000 7f34d584a640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34d00d1e90 0x7f34d00ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.560+0000 7f34d584a640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f34bc06b4d0 0x7f34bc06d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.560+0000 7f34d584a640 1 --2- 172.21.15.38:0/798944045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34d00d0320 0x7f34d00c93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.560+0000 7f34d584a640 1 -- 172.21.15.38:0/798944045 >> 172.21.15.38:0/798944045 conn(0x7f34d00668f0 msgr2=0x7f34d015ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:32.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.560+0000 7f34d584a640 1 -- 172.21.15.38:0/798944045 shutdown_connections 2024-04-25T08:32:32.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.560+0000 7f34d584a640 1 -- 172.21.15.38:0/798944045 wait complete. 2024-04-25T08:32:32.567 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:32:32.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.712+0000 7fbdf4fc8640 1 Processor -- start 2024-04-25T08:32:32.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.712+0000 7fbdf4fc8640 1 -- start start 2024-04-25T08:32:32.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.713+0000 7fbdf4fc8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdf0145080 0x7fbdf015cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.713+0000 7fbdf4fc8640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf015c130 con 0x7fbdf0071b30 2024-04-25T08:32:32.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.713+0000 7fbdf4fc8640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdf015c410 con 0x7fbdf00cc520 2024-04-25T08:32:32.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.713+0000 7fbdf4fc8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf0145460 con 0x7fbdf0145080 2024-04-25T08:32:32.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.714+0000 7fbdedd9b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbdf0071b30 0x7fbdf00cbe10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56466/0 (socket says 172.21.15.38:56466) 2024-04-25T08:32:32.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.714+0000 7fbdedd9b640 1 -- 172.21.15.38:0/3959413409 learned_addr learned my addr 172.21.15.38:0/3959413409 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:32.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.714+0000 7fbdee59c640 1 --2- 172.21.15.38:0/3959413409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdf0145080 0x7fbdf015cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.714+0000 7fbded59a640 1 -- 172.21.15.38:0/3959413409 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67416651 0 0) 0x7fbdf015c130 con 0x7fbdf0071b30 2024-04-25T08:32:32.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.714+0000 7fbded59a640 1 -- 172.21.15.38:0/3959413409 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdd8003520 con 0x7fbdf0071b30 2024-04-25T08:32:32.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.714+0000 7fbded59a640 1 -- 172.21.15.38:0/3959413409 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938141092 0 0) 0x7fbdf015c410 con 0x7fbdf00cc520 2024-04-25T08:32:32.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.714+0000 7fbded59a640 1 -- 172.21.15.38:0/3959413409 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdf015c130 con 0x7fbdf00cc520 2024-04-25T08:32:32.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.714+0000 7fbdee59c640 1 -- 172.21.15.38:0/3959413409 >> v1:172.21.15.107:6789/0 conn(0x7fbdf00cc520 legacy=0x7fbdf015f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.714+0000 7fbded59a640 1 -- 172.21.15.38:0/3959413409 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2832837679 0 0) 0x7fbdd8003520 con 0x7fbdf0071b30 2024-04-25T08:32:32.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.714+0000 7fbdee59c640 1 -- 172.21.15.38:0/3959413409 >> v1:172.21.15.38:6789/0 conn(0x7fbdf0071b30 legacy=0x7fbdf00cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.714+0000 7fbdee59c640 1 -- 172.21.15.38:0/3959413409 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf015c6f0 con 0x7fbdf0145080 2024-04-25T08:32:32.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.715+0000 7fbdee59c640 1 --2- 172.21.15.38:0/3959413409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdf0145080 0x7fbdf015cf20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbde00097d0 tx=0x7fbde002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84eb95d2f9c69be1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:32.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.715+0000 7fbded59a640 1 -- 172.21.15.38:0/3959413409 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde0037070 con 0x7fbdf0145080 2024-04-25T08:32:32.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.715+0000 7fbded59a640 1 -- 172.21.15.38:0/3959413409 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde002ac30 con 0x7fbdf0145080 2024-04-25T08:32:32.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.715+0000 7fbded59a640 1 -- 172.21.15.38:0/3959413409 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4001670 con 0x7fbdf0145080 2024-04-25T08:32:32.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.716+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/3959413409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdf0145080 msgr2=0x7fbdf015cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.716+0000 7fbdf4fc8640 1 --2- 172.21.15.38:0/3959413409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdf0145080 0x7fbdf015cf20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbde00097d0 tx=0x7fbde002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.716+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/3959413409 shutdown_connections 2024-04-25T08:32:32.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.716+0000 7fbdf4fc8640 1 --2- 172.21.15.38:0/3959413409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdf0145080 0x7fbdf015cf20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.716+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/3959413409 >> 172.21.15.38:0/3959413409 conn(0x7fbdf006abe0 msgr2=0x7fbdf006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:32.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.716+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/3959413409 shutdown_connections 2024-04-25T08:32:32.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.716+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/3959413409 wait complete. 2024-04-25T08:32:32.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.717+0000 7fbdf4fc8640 1 Processor -- start 2024-04-25T08:32:32.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.717+0000 7fbdf4fc8640 1 -- start start 2024-04-25T08:32:32.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdf4fc8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf0071b30 0x7fbdf013a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdf4fc8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdf00cc520 0x7fbdf01f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdf4fc8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf0145080 0x7fbdf013d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdf4fc8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf015bd60 con 0x7fbdf0145080 2024-04-25T08:32:32.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdf4fc8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf015ccb0 con 0x7fbdf0071b30 2024-04-25T08:32:32.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdf4fc8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf015bee0 con 0x7fbdf00cc520 2024-04-25T08:32:32.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdeed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf0145080 0x7fbdf013d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdeed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf0145080 0x7fbdf013d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44692/0 (socket says 172.21.15.38:44692) 2024-04-25T08:32:32.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdeed9d640 1 -- 172.21.15.38:0/2886140517 learned_addr learned my addr 172.21.15.38:0/2886140517 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:32.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdee59c640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf0071b30 0x7fbdf013a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdedd9b640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdf00cc520 0x7fbdf01f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdeed9d640 1 -- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdf00cc520 msgr2=0x7fbdf01f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdeed9d640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdf00cc520 0x7fbdf01f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdeed9d640 1 -- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf0071b30 msgr2=0x7fbdf013a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdeed9d640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf0071b30 0x7fbdf013a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdeed9d640 1 -- 172.21.15.38:0/2886140517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf013aa10 con 0x7fbdf0145080 2024-04-25T08:32:32.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.718+0000 7fbdeed9d640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf0145080 0x7fbdf013d780 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbde800c790 tx=0x7fbde800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:32.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.719+0000 7fbddeffd640 1 -- 172.21.15.38:0/2886140517 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde8003b70 con 0x7fbdf0145080 2024-04-25T08:32:32.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.719+0000 7fbddeffd640 1 -- 172.21.15.38:0/2886140517 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde8003d10 con 0x7fbdf0145080 2024-04-25T08:32:32.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.719+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/2886140517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf013e160 con 0x7fbdf0145080 2024-04-25T08:32:32.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.719+0000 7fbddeffd640 1 -- 172.21.15.38:0/2886140517 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde800aa50 con 0x7fbdf0145080 2024-04-25T08:32:32.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.719+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/2886140517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf013b4a0 con 0x7fbdf0145080 2024-04-25T08:32:32.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.720+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/2886140517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf004ef90 con 0x7fbdf0145080 2024-04-25T08:32:32.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.720+0000 7fbddeffd640 1 -- 172.21.15.38:0/2886140517 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbde800abf0 con 0x7fbdf0145080 2024-04-25T08:32:32.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.720+0000 7fbddeffd640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbdd006b4d0 0x7fbdd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:32.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.721+0000 7fbddeffd640 1 -- 172.21.15.38:0/2886140517 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fbde808a550 con 0x7fbdf0145080 2024-04-25T08:32:32.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.721+0000 7fbdee59c640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbdd006b4d0 0x7fbdd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:32.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.721+0000 7fbdee59c640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbdd006b4d0 0x7fbdd006d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbde0034000 tx=0x7fbde002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:32.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.725+0000 7fbddeffd640 1 -- 172.21.15.38:0/2886140517 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbde8058890 con 0x7fbdf0145080 2024-04-25T08:32:32.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.919+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/2886140517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbdf00ccd70 con 0x7fbdf0145080 2024-04-25T08:32:32.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.919+0000 7fbddeffd640 1 -- 172.21.15.38:0/2886140517 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbde805c930 con 0x7fbdf0145080 2024-04-25T08:32:32.915 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:32:32.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.921+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbdd006b4d0 msgr2=0x7fbdd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.921+0000 7fbdf4fc8640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbdd006b4d0 0x7fbdd006d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbde0034000 tx=0x7fbde002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:32.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.921+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/2886140517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf0145080 msgr2=0x7fbdf013d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:32.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.921+0000 7fbdf4fc8640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf0145080 0x7fbdf013d780 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbde800c790 tx=0x7fbde800cc50 comp rx=0 tx=0).stop 2024-04-25T08:32:32.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.921+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/2886140517 shutdown_connections 2024-04-25T08:32:32.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.921+0000 7fbdf4fc8640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbdf0145080 0x7fbdf013d780 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.921+0000 7fbdf4fc8640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbdd006b4d0 0x7fbdd006d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.921+0000 7fbdf4fc8640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbdf00cc520 0x7fbdf01f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.921+0000 7fbdf4fc8640 1 --2- 172.21.15.38:0/2886140517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbdf0071b30 0x7fbdf013a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:32.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.921+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/2886140517 >> 172.21.15.38:0/2886140517 conn(0x7fbdf006abe0 msgr2=0x7fbdf00cbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:32.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.921+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/2886140517 shutdown_connections 2024-04-25T08:32:32.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:32.921+0000 7fbdf4fc8640 1 -- 172.21.15.38:0/2886140517 wait complete. 2024-04-25T08:32:32.928 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:32:32.928 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:32.947 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:33.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.052+0000 7f356e488640 1 Processor -- start 2024-04-25T08:32:33.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.052+0000 7f356e488640 1 -- start start 2024-04-25T08:32:33.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.053+0000 7f356e488640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f356815dc10 0x7f35680ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.053+0000 7f356e488640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f35680ced60 0x7f35680d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.053+0000 7f356e488640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35680d3e70 con 0x7f35680d3900 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.053+0000 7f356e488640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f356815dff0 con 0x7f35680ced60 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.053+0000 7f356e488640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680b6990 con 0x7f356815dc10 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.053+0000 7f35677fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f35680ced60 0x7f35680d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.053+0000 7f35677fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f35680ced60 0x7f35680d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44704/0 (socket says 172.21.15.38:44704) 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.053+0000 7f35677fe640 1 -- 172.21.15.38:0/2990864007 learned_addr learned my addr 172.21.15.38:0/2990864007 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.053+0000 7f3567fff640 1 --2- 172.21.15.38:0/2990864007 >> v2:172.21.15.107:3300/0 conn(0x7f356815dc10 0x7f35680ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.054+0000 7f35677fe640 1 -- 172.21.15.38:0/2990864007 >> v2:172.21.15.107:3300/0 conn(0x7f356815dc10 msgr2=0x7f35680ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.054+0000 7f35677fe640 1 --2- 172.21.15.38:0/2990864007 >> v2:172.21.15.107:3300/0 conn(0x7f356815dc10 0x7f35680ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.054+0000 7f35677fe640 1 -- 172.21.15.38:0/2990864007 >> v1:172.21.15.107:6789/0 conn(0x7f35680d3900 legacy=0x7f35680b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.054+0000 7f35677fe640 1 -- 172.21.15.38:0/2990864007 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35680d4150 con 0x7f35680ced60 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.054+0000 7f35677fe640 1 --2- 172.21.15.38:0/2990864007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35680ced60 0x7f35680d0850 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f35680b5d50 tx=0x7f355c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa32637cfb7c74e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.054+0000 7f3566ffd640 1 -- 172.21.15.38:0/2990864007 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c037070 con 0x7f35680ced60 2024-04-25T08:32:33.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.054+0000 7f3566ffd640 1 -- 172.21.15.38:0/2990864007 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c032440 con 0x7f35680ced60 2024-04-25T08:32:33.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.054+0000 7f3566ffd640 1 -- 172.21.15.38:0/2990864007 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c032810 con 0x7f35680ced60 2024-04-25T08:32:33.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.055+0000 7f356e488640 1 -- 172.21.15.38:0/2990864007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35680ced60 msgr2=0x7f35680d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.055+0000 7f356e488640 1 --2- 172.21.15.38:0/2990864007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35680ced60 0x7f35680d0850 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f35680b5d50 tx=0x7f355c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.055+0000 7f356e488640 1 -- 172.21.15.38:0/2990864007 shutdown_connections 2024-04-25T08:32:33.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.055+0000 7f356e488640 1 --2- 172.21.15.38:0/2990864007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35680ced60 0x7f35680d0850 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.055+0000 7f356e488640 1 --2- 172.21.15.38:0/2990864007 >> v2:172.21.15.107:3300/0 conn(0x7f356815dc10 0x7f35680ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.055+0000 7f356e488640 1 -- 172.21.15.38:0/2990864007 >> 172.21.15.38:0/2990864007 conn(0x7f3568066d10 msgr2=0x7f3568067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.055+0000 7f356e488640 1 -- 172.21.15.38:0/2990864007 shutdown_connections 2024-04-25T08:32:33.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.055+0000 7f356e488640 1 -- 172.21.15.38:0/2990864007 wait complete. 2024-04-25T08:32:33.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.056+0000 7f356e488640 1 Processor -- start 2024-04-25T08:32:33.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.056+0000 7f356e488640 1 -- start start 2024-04-25T08:32:33.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.057+0000 7f356e488640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35680ced60 0x7f356814dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.057+0000 7f356e488640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35680d3900 0x7f35681603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.057+0000 7f356e488640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f356815dc10 0x7f35681608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.057+0000 7f356e488640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35680b6bc0 con 0x7f35680d3900 2024-04-25T08:32:33.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.057+0000 7f356e488640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35680d4710 con 0x7f356815dc10 2024-04-25T08:32:33.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.057+0000 7f356e488640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35680d0d90 con 0x7f35680ced60 2024-04-25T08:32:33.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.057+0000 7f35677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35680d3900 0x7f35681603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.057+0000 7f35677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35680d3900 0x7f35681603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44714/0 (socket says 172.21.15.38:44714) 2024-04-25T08:32:33.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.057+0000 7f35677fe640 1 -- 172.21.15.38:0/900539274 learned_addr learned my addr 172.21.15.38:0/900539274 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.057+0000 7f356ca25640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f356815dc10 0x7f35681608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.057+0000 7f3567fff640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35680ced60 0x7f356814dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.058+0000 7f35677fe640 1 -- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35680ced60 msgr2=0x7f356814dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.058+0000 7f35677fe640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35680ced60 0x7f356814dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.058+0000 7f35677fe640 1 -- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f356815dc10 msgr2=0x7f35681608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.058+0000 7f35677fe640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f356815dc10 0x7f35681608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.058+0000 7f35677fe640 1 -- 172.21.15.38:0/900539274 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35681f6d00 con 0x7f35680d3900 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.058+0000 7f35677fe640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35680d3900 0x7f35681603b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f355c02a220 tx=0x7f355c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.058+0000 7f3564ff9640 1 -- 172.21.15.38:0/900539274 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c037040 con 0x7f35680d3900 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.058+0000 7f3564ff9640 1 -- 172.21.15.38:0/900539274 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c042070 con 0x7f35680d3900 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.058+0000 7f356e488640 1 -- 172.21.15.38:0/900539274 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35681f6fe0 con 0x7f35680d3900 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.058+0000 7f3564ff9640 1 -- 172.21.15.38:0/900539274 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c030880 con 0x7f35680d3900 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.058+0000 7f356e488640 1 -- 172.21.15.38:0/900539274 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35681f75a0 con 0x7f35680d3900 2024-04-25T08:32:33.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.059+0000 7f356e488640 1 -- 172.21.15.38:0/900539274 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3568068e80 con 0x7f35680d3900 2024-04-25T08:32:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.059+0000 7f3564ff9640 1 -- 172.21.15.38:0/900539274 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f355c030a20 con 0x7f35680d3900 2024-04-25T08:32:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.060+0000 7f3564ff9640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f355406b500 0x7f355406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.060+0000 7f3564ff9640 1 -- 172.21.15.38:0/900539274 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f355c0aae00 con 0x7f35680d3900 2024-04-25T08:32:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.060+0000 7f3567fff640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f355406b500 0x7f355406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.060+0000 7f3567fff640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f355406b500 0x7f355406d9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3558009aa0 tx=0x7f3558009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.064+0000 7f3564ff9640 1 -- 172.21.15.38:0/900539274 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f355c0791c0 con 0x7f35680d3900 2024-04-25T08:32:33.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.246+0000 7f356e488640 1 -- 172.21.15.38:0/900539274 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f356815d240 con 0x7f35680d3900 2024-04-25T08:32:33.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.247+0000 7f3564ff9640 1 -- 172.21.15.38:0/900539274 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18969 (secure 0 0 0) 0x7f355c07d260 con 0x7f35680d3900 2024-04-25T08:32:33.243 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:33.244 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:16.572337+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.a","mappings":[{"from":4,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:33.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.249+0000 7f356e488640 1 -- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f355406b500 msgr2=0x7f355406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.249+0000 7f356e488640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f355406b500 0x7f355406d9c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3558009aa0 tx=0x7f3558009190 comp rx=0 tx=0).stop 2024-04-25T08:32:33.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.249+0000 7f356e488640 1 -- 172.21.15.38:0/900539274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35680d3900 msgr2=0x7f35681603b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.249+0000 7f356e488640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35680d3900 0x7f35681603b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f355c02a220 tx=0x7f355c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.249+0000 7f356e488640 1 -- 172.21.15.38:0/900539274 shutdown_connections 2024-04-25T08:32:33.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.249+0000 7f356e488640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f356815dc10 0x7f35681608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.249+0000 7f356e488640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f35680d3900 0x7f35681603b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.249+0000 7f356e488640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f355406b500 0x7f355406d9c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.249+0000 7f356e488640 1 --2- 172.21.15.38:0/900539274 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35680ced60 0x7f356814dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.250+0000 7f356e488640 1 -- 172.21.15.38:0/900539274 >> 172.21.15.38:0/900539274 conn(0x7f3568066d10 msgr2=0x7f35680b6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.250+0000 7f356e488640 1 -- 172.21.15.38:0/900539274 shutdown_connections 2024-04-25T08:32:33.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.250+0000 7f356e488640 1 -- 172.21.15.38:0/900539274 wait complete. 2024-04-25T08:32:33.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:33.257 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.364+0000 7f3e4ea31640 1 Processor -- start 2024-04-25T08:32:33.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.364+0000 7f3e4ea31640 1 -- start start 2024-04-25T08:32:33.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.365+0000 7f3e4ea31640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e480d0320 0x7f3e480d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.365+0000 7f3e4ea31640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3e480b6980 0x7f3e480b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.365+0000 7f3e4ea31640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e480d2060 con 0x7f3e480d0700 2024-04-25T08:32:33.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.365+0000 7f3e4ea31640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e480d2330 con 0x7f3e480b6980 2024-04-25T08:32:33.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.365+0000 7f3e4ea31640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480d2610 con 0x7f3e480d0320 2024-04-25T08:32:33.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.365+0000 7f3e4cfce640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3e480b6980 0x7f3e480b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.365+0000 7f3e4cfce640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3e480b6980 0x7f3e480b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44716/0 (socket says 172.21.15.38:44716) 2024-04-25T08:32:33.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.366+0000 7f3e4cfce640 1 -- 172.21.15.38:0/4066007630 learned_addr learned my addr 172.21.15.38:0/4066007630 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:33.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.366+0000 7f3e47fff640 1 --2- 172.21.15.38:0/4066007630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e480d0320 0x7f3e480d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.366+0000 7f3e4cfce640 1 -- 172.21.15.38:0/4066007630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e480d0320 msgr2=0x7f3e480d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.366+0000 7f3e4cfce640 1 --2- 172.21.15.38:0/4066007630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e480d0320 0x7f3e480d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.366+0000 7f3e4cfce640 1 -- 172.21.15.38:0/4066007630 >> v1:172.21.15.107:6789/0 conn(0x7f3e480d0700 legacy=0x7f3e480d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.366+0000 7f3e4cfce640 1 -- 172.21.15.38:0/4066007630 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e480d28f0 con 0x7f3e480b6980 2024-04-25T08:32:33.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.366+0000 7f3e4cfce640 1 --2- 172.21.15.38:0/4066007630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e480b6980 0x7f3e480b6e10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009e50 tx=0x7f3e4002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3184a692b195d3f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.367+0000 7f3e46ffd640 1 -- 172.21.15.38:0/4066007630 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40036070 con 0x7f3e480b6980 2024-04-25T08:32:33.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.367+0000 7f3e46ffd640 1 -- 172.21.15.38:0/4066007630 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e40031600 con 0x7f3e480b6980 2024-04-25T08:32:33.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.367+0000 7f3e46ffd640 1 -- 172.21.15.38:0/4066007630 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e400319d0 con 0x7f3e480b6980 2024-04-25T08:32:33.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.367+0000 7f3e4ea31640 1 -- 172.21.15.38:0/4066007630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e480b6980 msgr2=0x7f3e480b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.367+0000 7f3e4ea31640 1 --2- 172.21.15.38:0/4066007630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e480b6980 0x7f3e480b6e10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009e50 tx=0x7f3e4002a7a0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.367+0000 7f3e4ea31640 1 -- 172.21.15.38:0/4066007630 shutdown_connections 2024-04-25T08:32:33.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.367+0000 7f3e4ea31640 1 --2- 172.21.15.38:0/4066007630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e480b6980 0x7f3e480b6e10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.367+0000 7f3e4ea31640 1 --2- 172.21.15.38:0/4066007630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e480d0320 0x7f3e480d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.367+0000 7f3e4ea31640 1 -- 172.21.15.38:0/4066007630 >> 172.21.15.38:0/4066007630 conn(0x7f3e480668f0 msgr2=0x7f3e48066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.367+0000 7f3e4ea31640 1 -- 172.21.15.38:0/4066007630 shutdown_connections 2024-04-25T08:32:33.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.367+0000 7f3e4ea31640 1 -- 172.21.15.38:0/4066007630 wait complete. 2024-04-25T08:32:33.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.368+0000 7f3e4ea31640 1 Processor -- start 2024-04-25T08:32:33.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.369+0000 7f3e4ea31640 1 -- start start 2024-04-25T08:32:33.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.369+0000 7f3e4ea31640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e480b6980 0x7f3e48154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.369+0000 7f3e4ea31640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e480d0320 0x7f3e480b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.369+0000 7f3e4ea31640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e480d0700 0x7f3e480c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.369+0000 7f3e4ea31640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480d0b40 con 0x7f3e480b6980 2024-04-25T08:32:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.369+0000 7f3e4ea31640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e480d2eb0 con 0x7f3e480d0320 2024-04-25T08:32:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.369+0000 7f3e4ea31640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e48155f00 con 0x7f3e480d0700 2024-04-25T08:32:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.369+0000 7f3e47fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e480b6980 0x7f3e48154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.369+0000 7f3e47fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e480b6980 0x7f3e48154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44724/0 (socket says 172.21.15.38:44724) 2024-04-25T08:32:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.369+0000 7f3e47fff640 1 -- 172.21.15.38:0/2724664703 learned_addr learned my addr 172.21.15.38:0/2724664703 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.369+0000 7f3e477fe640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e480d0320 0x7f3e480b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.369+0000 7f3e4cfce640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e480d0700 0x7f3e480c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.370+0000 7f3e47fff640 1 -- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e480d0700 msgr2=0x7f3e480c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.370+0000 7f3e47fff640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e480d0700 0x7f3e480c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.370+0000 7f3e47fff640 1 -- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e480d0320 msgr2=0x7f3e480b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.370+0000 7f3e47fff640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e480d0320 0x7f3e480b5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.370+0000 7f3e47fff640 1 -- 172.21.15.38:0/2724664703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e481552f0 con 0x7f3e480b6980 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.370+0000 7f3e47fff640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e480b6980 0x7f3e48154cd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800e9b0 tx=0x7f3e3800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.370+0000 7f3e44ff9640 1 -- 172.21.15.38:0/2724664703 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e38017070 con 0x7f3e480b6980 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.370+0000 7f3e44ff9640 1 -- 172.21.15.38:0/2724664703 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3800c9a0 con 0x7f3e480b6980 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.370+0000 7f3e4ea31640 1 -- 172.21.15.38:0/2724664703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e481555d0 con 0x7f3e480b6980 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.370+0000 7f3e44ff9640 1 -- 172.21.15.38:0/2724664703 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3800cd90 con 0x7f3e480b6980 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.370+0000 7f3e4ea31640 1 -- 172.21.15.38:0/2724664703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e48155b90 con 0x7f3e480b6980 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.371+0000 7f3e4ea31640 1 -- 172.21.15.38:0/2724664703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e48068e80 con 0x7f3e480b6980 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.372+0000 7f3e44ff9640 1 -- 172.21.15.38:0/2724664703 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e3801a070 con 0x7f3e480b6980 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.372+0000 7f3e44ff9640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e3406b4d0 0x7f3e3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.372+0000 7f3e44ff9640 1 -- 172.21.15.38:0/2724664703 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f3e3808a950 con 0x7f3e480b6980 2024-04-25T08:32:33.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.372+0000 7f3e477fe640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e3406b4d0 0x7f3e3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.373+0000 7f3e477fe640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e3406b4d0 0x7f3e3406d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3e48160b00 tx=0x7f3e3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.376+0000 7f3e44ff9640 1 -- 172.21.15.38:0/2724664703 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e38058d20 con 0x7f3e480b6980 2024-04-25T08:32:33.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.559+0000 7f3e4ea31640 1 -- 172.21.15.38:0/2724664703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e480b3050 con 0x7f3e480b6980 2024-04-25T08:32:33.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.560+0000 7f3e44ff9640 1 -- 172.21.15.38:0/2724664703 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e3805cdc0 con 0x7f3e480b6980 2024-04-25T08:32:33.556 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:33.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.562+0000 7f3e4ea31640 1 -- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e3406b4d0 msgr2=0x7f3e3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.562+0000 7f3e4ea31640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e3406b4d0 0x7f3e3406d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3e48160b00 tx=0x7f3e3c009040 comp rx=0 tx=0).stop 2024-04-25T08:32:33.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.562+0000 7f3e4ea31640 1 -- 172.21.15.38:0/2724664703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e480b6980 msgr2=0x7f3e48154cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.562+0000 7f3e4ea31640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e480b6980 0x7f3e48154cd0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800e9b0 tx=0x7f3e3800ee70 comp rx=0 tx=0).stop 2024-04-25T08:32:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.562+0000 7f3e4ea31640 1 -- 172.21.15.38:0/2724664703 shutdown_connections 2024-04-25T08:32:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.562+0000 7f3e4ea31640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3e3406b4d0 0x7f3e3406d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.562+0000 7f3e4ea31640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3e480d0700 0x7f3e480c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.562+0000 7f3e4ea31640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e480d0320 0x7f3e480b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.562+0000 7f3e4ea31640 1 --2- 172.21.15.38:0/2724664703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e480b6980 0x7f3e48154cd0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.562+0000 7f3e4ea31640 1 -- 172.21.15.38:0/2724664703 >> 172.21.15.38:0/2724664703 conn(0x7f3e480668f0 msgr2=0x7f3e480b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.562+0000 7f3e4ea31640 1 -- 172.21.15.38:0/2724664703 shutdown_connections 2024-04-25T08:32:33.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.562+0000 7f3e4ea31640 1 -- 172.21.15.38:0/2724664703 wait complete. 2024-04-25T08:32:33.569 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:33.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.676+0000 7fe5cf530640 1 Processor -- start 2024-04-25T08:32:33.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.677+0000 7fe5cf530640 1 -- start start 2024-04-25T08:32:33.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.677+0000 7fe5cf530640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe5c80b1d60 0x7fe5c80cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.677+0000 7fe5cf530640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe5c8161e20 0x7fe5c815d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.677+0000 7fe5cf530640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c815ef40 con 0x7fe5c80cff10 2024-04-25T08:32:33.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.677+0000 7fe5cf530640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c806abe0 con 0x7fe5c80b1d60 2024-04-25T08:32:33.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.677+0000 7fe5cf530640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c806b480 con 0x7fe5c8161e20 2024-04-25T08:32:33.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.678+0000 7fe5cd2cc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe5c80b1d60 0x7fe5c80cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.678+0000 7fe5ccacb640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe5c80cff10 0x7fe5c8161710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56468/0 (socket says 172.21.15.38:56468) 2024-04-25T08:32:33.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.678+0000 7fe5ccacb640 1 -- 172.21.15.38:0/160310674 learned_addr learned my addr 172.21.15.38:0/160310674 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:33.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.678+0000 7fe5cdacd640 1 --2- 172.21.15.38:0/160310674 >> v2:172.21.15.107:3300/0 conn(0x7fe5c8161e20 0x7fe5c815d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.678+0000 7fe5bffff640 1 -- 172.21.15.38:0/160310674 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281770380 0 0) 0x7fe5c815ef40 con 0x7fe5c80cff10 2024-04-25T08:32:33.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.678+0000 7fe5bffff640 1 -- 172.21.15.38:0/160310674 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5b4003520 con 0x7fe5c80cff10 2024-04-25T08:32:33.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.678+0000 7fe5cd2cc640 1 -- 172.21.15.38:0/160310674 >> v2:172.21.15.107:3300/0 conn(0x7fe5c8161e20 msgr2=0x7fe5c815d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.678+0000 7fe5cd2cc640 1 --2- 172.21.15.38:0/160310674 >> v2:172.21.15.107:3300/0 conn(0x7fe5c8161e20 0x7fe5c815d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.678+0000 7fe5cd2cc640 1 -- 172.21.15.38:0/160310674 >> v1:172.21.15.38:6789/0 conn(0x7fe5c80cff10 legacy=0x7fe5c8161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.678+0000 7fe5cd2cc640 1 -- 172.21.15.38:0/160310674 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c806ae30 con 0x7fe5c80b1d60 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.679+0000 7fe5cd2cc640 1 --2- 172.21.15.38:0/160310674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5c80b1d60 0x7fe5c80cf9d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe5c81627d0 tx=0x7fe5c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c879c6cc440746e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.679+0000 7fe5bffff640 1 -- 172.21.15.38:0/160310674 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4037070 con 0x7fe5c80b1d60 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.679+0000 7fe5bffff640 1 -- 172.21.15.38:0/160310674 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c4032440 con 0x7fe5c80b1d60 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.679+0000 7fe5bffff640 1 -- 172.21.15.38:0/160310674 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4032810 con 0x7fe5c80b1d60 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.679+0000 7fe5cf530640 1 -- 172.21.15.38:0/160310674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5c80b1d60 msgr2=0x7fe5c80cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.679+0000 7fe5cf530640 1 --2- 172.21.15.38:0/160310674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5c80b1d60 0x7fe5c80cf9d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe5c81627d0 tx=0x7fe5c402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.679+0000 7fe5cf530640 1 -- 172.21.15.38:0/160310674 shutdown_connections 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.679+0000 7fe5cf530640 1 --2- 172.21.15.38:0/160310674 >> v2:172.21.15.107:3300/0 conn(0x7fe5c8161e20 0x7fe5c815d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.679+0000 7fe5cf530640 1 --2- 172.21.15.38:0/160310674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5c80b1d60 0x7fe5c80cf9d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.679+0000 7fe5cf530640 1 -- 172.21.15.38:0/160310674 >> 172.21.15.38:0/160310674 conn(0x7fe5c80d06b0 msgr2=0x7fe5c80d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.680+0000 7fe5cf530640 1 -- 172.21.15.38:0/160310674 shutdown_connections 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.680+0000 7fe5cf530640 1 -- 172.21.15.38:0/160310674 wait complete. 2024-04-25T08:32:33.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.681+0000 7fe5cf530640 1 Processor -- start 2024-04-25T08:32:33.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.681+0000 7fe5cf530640 1 -- start start 2024-04-25T08:32:33.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.681+0000 7fe5cf530640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5c80b1d60 0x7fe5c8153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.681+0000 7fe5cf530640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c80cff10 0x7fe5c8154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.681+0000 7fe5cf530640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5c8161e20 0x7fe5c80ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.681+0000 7fe5cf530640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c806b910 con 0x7fe5c80b1d60 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.681+0000 7fe5cf530640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c80d03a0 con 0x7fe5c80cff10 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.681+0000 7fe5cf530640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c806ba90 con 0x7fe5c8161e20 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5cd2cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5c80b1d60 0x7fe5c8153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5cd2cc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5c80b1d60 0x7fe5c8153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44738/0 (socket says 172.21.15.38:44738) 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5cd2cc640 1 -- 172.21.15.38:0/1934020397 learned_addr learned my addr 172.21.15.38:0/1934020397 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5ccacb640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c80cff10 0x7fe5c8154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5cdacd640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5c8161e20 0x7fe5c80ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5cd2cc640 1 -- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5c8161e20 msgr2=0x7fe5c80ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5cd2cc640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5c8161e20 0x7fe5c80ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5cd2cc640 1 -- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c80cff10 msgr2=0x7fe5c8154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5cd2cc640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c80cff10 0x7fe5c8154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5cd2cc640 1 -- 172.21.15.38:0/1934020397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c8155650 con 0x7fe5c80b1d60 2024-04-25T08:32:33.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5cd2cc640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5c80b1d60 0x7fe5c8153ca0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009890 tx=0x7fe5c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5bdffb640 1 -- 172.21.15.38:0/1934020397 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4041070 con 0x7fe5c80b1d60 2024-04-25T08:32:33.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5bdffb640 1 -- 172.21.15.38:0/1934020397 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c403c890 con 0x7fe5c80b1d60 2024-04-25T08:32:33.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5bdffb640 1 -- 172.21.15.38:0/1934020397 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4037040 con 0x7fe5c80b1d60 2024-04-25T08:32:33.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5cf530640 1 -- 172.21.15.38:0/1934020397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c8154db0 con 0x7fe5c80b1d60 2024-04-25T08:32:33.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.682+0000 7fe5cf530640 1 -- 172.21.15.38:0/1934020397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c8155370 con 0x7fe5c80b1d60 2024-04-25T08:32:33.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.683+0000 7fe5cf530640 1 -- 172.21.15.38:0/1934020397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c8157b20 con 0x7fe5c80b1d60 2024-04-25T08:32:33.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.684+0000 7fe5bdffb640 1 -- 172.21.15.38:0/1934020397 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5c4044070 con 0x7fe5c80b1d60 2024-04-25T08:32:33.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.684+0000 7fe5bdffb640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe5ac06b4d0 0x7fe5ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.684+0000 7fe5bdffb640 1 -- 172.21.15.38:0/1934020397 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fe5c40ab030 con 0x7fe5c80b1d60 2024-04-25T08:32:33.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.685+0000 7fe5ccacb640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe5ac06b4d0 0x7fe5ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.685+0000 7fe5ccacb640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe5ac06b4d0 0x7fe5ac06d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80b0880 tx=0x7fe5b8007d30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.689+0000 7fe5bdffb640 1 -- 172.21.15.38:0/1934020397 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5c4079520 con 0x7fe5c80b1d60 2024-04-25T08:32:33.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.871+0000 7fe5cf530640 1 -- 172.21.15.38:0/1934020397 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5c8068e80 con 0x7fe5c80b1d60 2024-04-25T08:32:33.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.871+0000 7fe5bdffb640 1 -- 172.21.15.38:0/1934020397 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe5c407d5c0 con 0x7fe5c80b1d60 2024-04-25T08:32:33.867 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:33.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.873+0000 7fe5cf530640 1 -- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe5ac06b4d0 msgr2=0x7fe5ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.873+0000 7fe5cf530640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe5ac06b4d0 0x7fe5ac06d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80b0880 tx=0x7fe5b8007d30 comp rx=0 tx=0).stop 2024-04-25T08:32:33.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.873+0000 7fe5cf530640 1 -- 172.21.15.38:0/1934020397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5c80b1d60 msgr2=0x7fe5c8153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.873+0000 7fe5cf530640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5c80b1d60 0x7fe5c8153ca0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4009890 tx=0x7fe5c40046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.873+0000 7fe5cf530640 1 -- 172.21.15.38:0/1934020397 shutdown_connections 2024-04-25T08:32:33.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.873+0000 7fe5cf530640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe5ac06b4d0 0x7fe5ac06d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.873+0000 7fe5cf530640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe5c8161e20 0x7fe5c80ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.873+0000 7fe5cf530640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5c80cff10 0x7fe5c8154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.873+0000 7fe5cf530640 1 --2- 172.21.15.38:0/1934020397 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe5c80b1d60 0x7fe5c8153ca0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.873+0000 7fe5cf530640 1 -- 172.21.15.38:0/1934020397 >> 172.21.15.38:0/1934020397 conn(0x7fe5c80d06b0 msgr2=0x7fe5c8160c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.873+0000 7fe5cf530640 1 -- 172.21.15.38:0/1934020397 shutdown_connections 2024-04-25T08:32:33.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.873+0000 7fe5cf530640 1 -- 172.21.15.38:0/1934020397 wait complete. 2024-04-25T08:32:33.880 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:33.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.988+0000 7f644f8fc640 1 Processor -- start 2024-04-25T08:32:33.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.988+0000 7f644f8fc640 1 -- start start 2024-04-25T08:32:33.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.988+0000 7f644f8fc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f644815b990 0x7f64480cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f644f8fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f644815f5f0 0x7f64480d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f644f8fc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644815fd00 con 0x7f64480d1b50 2024-04-25T08:32:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f644f8fc640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6448160580 con 0x7f644815b990 2024-04-25T08:32:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f644f8fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644815fa30 con 0x7f644815f5f0 2024-04-25T08:32:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f644ce97640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f644815f5f0 0x7f64480d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f644d698640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f644815b990 0x7f64480cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f644de99640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f64480d1b50 0x7f64480b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:37944/0 (socket says 172.21.15.38:37944) 2024-04-25T08:32:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f644de99640 1 -- 172.21.15.38:0/2626473469 learned_addr learned my addr 172.21.15.38:0/2626473469 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f643ffff640 1 -- 172.21.15.38:0/2626473469 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724616016 0 0) 0x7f644815fd00 con 0x7f64480d1b50 2024-04-25T08:32:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f644ce97640 1 -- 172.21.15.38:0/2626473469 >> v2:172.21.15.107:3300/0 conn(0x7f644815b990 msgr2=0x7f64480cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f644ce97640 1 --2- 172.21.15.38:0/2626473469 >> v2:172.21.15.107:3300/0 conn(0x7f644815b990 0x7f64480cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f644ce97640 1 -- 172.21.15.38:0/2626473469 >> v1:172.21.15.107:6789/0 conn(0x7f64480d1b50 legacy=0x7f64480b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.989+0000 7f644ce97640 1 -- 172.21.15.38:0/2626473469 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644815ffc0 con 0x7f644815f5f0 2024-04-25T08:32:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.990+0000 7f644ce97640 1 --2- 172.21.15.38:0/2626473469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f644815f5f0 0x7f64480d1610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f64380097a0 tx=0x7f643802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f369df9d8a1f01e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.990+0000 7f643ffff640 1 -- 172.21.15.38:0/2626473469 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438037070 con 0x7f644815f5f0 2024-04-25T08:32:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.990+0000 7f643ffff640 1 -- 172.21.15.38:0/2626473469 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643802ae50 con 0x7f644815f5f0 2024-04-25T08:32:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.990+0000 7f643ffff640 1 -- 172.21.15.38:0/2626473469 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438032660 con 0x7f644815f5f0 2024-04-25T08:32:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.991+0000 7f644f8fc640 1 -- 172.21.15.38:0/2626473469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f644815f5f0 msgr2=0x7f64480d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.991+0000 7f644f8fc640 1 --2- 172.21.15.38:0/2626473469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f644815f5f0 0x7f64480d1610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f64380097a0 tx=0x7f643802a100 comp rx=0 tx=0).stop 2024-04-25T08:32:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.991+0000 7f644f8fc640 1 -- 172.21.15.38:0/2626473469 shutdown_connections 2024-04-25T08:32:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.991+0000 7f644f8fc640 1 --2- 172.21.15.38:0/2626473469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f644815f5f0 0x7f64480d1610 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.991+0000 7f644f8fc640 1 --2- 172.21.15.38:0/2626473469 >> v2:172.21.15.107:3300/0 conn(0x7f644815b990 0x7f64480cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.991+0000 7f644f8fc640 1 -- 172.21.15.38:0/2626473469 >> 172.21.15.38:0/2626473469 conn(0x7f644808b000 msgr2=0x7f644808b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.991+0000 7f644f8fc640 1 -- 172.21.15.38:0/2626473469 shutdown_connections 2024-04-25T08:32:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.991+0000 7f644f8fc640 1 -- 172.21.15.38:0/2626473469 wait complete. 2024-04-25T08:32:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.992+0000 7f644f8fc640 1 Processor -- start 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.992+0000 7f644f8fc640 1 -- start start 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644f8fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64480d1b50 0x7f644815c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644f8fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f644815b990 0x7f6448157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644f8fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f644815f5f0 0x7f64480c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644f8fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644815bd70 con 0x7f64480d1b50 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644f8fc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64480b1ac0 con 0x7f644815b990 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644f8fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644815bef0 con 0x7f644815f5f0 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644d698640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64480d1b50 0x7f644815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644d698640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64480d1b50 0x7f644815c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44754/0 (socket says 172.21.15.38:44754) 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644de99640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f644815f5f0 0x7f64480c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644d698640 1 -- 172.21.15.38:0/1006725460 learned_addr learned my addr 172.21.15.38:0/1006725460 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644ce97640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f644815b990 0x7f6448157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644de99640 1 -- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f644815b990 msgr2=0x7f6448157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644de99640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f644815b990 0x7f6448157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644de99640 1 -- 172.21.15.38:0/1006725460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64480d1b50 msgr2=0x7f644815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644de99640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64480d1b50 0x7f644815c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644de99640 1 -- 172.21.15.38:0/1006725460 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6448157e80 con 0x7f644815f5f0 2024-04-25T08:32:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644d698640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64480d1b50 0x7f644815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644de99640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f644815f5f0 0x7f64480c88e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f644000c700 tx=0x7f644000cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f643dffb640 1 -- 172.21.15.38:0/1006725460 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6440018070 con 0x7f644815f5f0 2024-04-25T08:32:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f643dffb640 1 -- 172.21.15.38:0/1006725460 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644000ce10 con 0x7f644815f5f0 2024-04-25T08:32:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f644f8fc640 1 -- 172.21.15.38:0/1006725460 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644815ce60 con 0x7f644815f5f0 2024-04-25T08:32:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.993+0000 7f643dffb640 1 -- 172.21.15.38:0/1006725460 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644000a880 con 0x7f644815f5f0 2024-04-25T08:32:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.994+0000 7f644f8fc640 1 -- 172.21.15.38:0/1006725460 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644815d420 con 0x7f644815f5f0 2024-04-25T08:32:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.995+0000 7f644f8fc640 1 -- 172.21.15.38:0/1006725460 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6448068e80 con 0x7f644815f5f0 2024-04-25T08:32:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.995+0000 7f643dffb640 1 -- 172.21.15.38:0/1006725460 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f644001b070 con 0x7f644815f5f0 2024-04-25T08:32:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.995+0000 7f643dffb640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f642c06b4d0 0x7f642c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.996+0000 7f643dffb640 1 -- 172.21.15.38:0/1006725460 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f6440089b30 con 0x7f644815f5f0 2024-04-25T08:32:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.996+0000 7f644d698640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f642c06b4d0 0x7f642c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:33.996+0000 7f644d698640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f642c06b4d0 0x7f642c06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f64480cf5e0 tx=0x7f6444009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:34.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.004+0000 7f643dffb640 1 -- 172.21.15.38:0/1006725460 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6440057e60 con 0x7f644815f5f0 2024-04-25T08:32:34.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.183+0000 7f644f8fc640 1 -- 172.21.15.38:0/1006725460 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64480b1220 con 0x7f644815f5f0 2024-04-25T08:32:34.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.184+0000 7f643dffb640 1 -- 172.21.15.38:0/1006725460 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f644005ce20 con 0x7f644815f5f0 2024-04-25T08:32:34.180 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:34.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.186+0000 7f644f8fc640 1 -- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f642c06b4d0 msgr2=0x7f642c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.186+0000 7f644f8fc640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f642c06b4d0 0x7f642c06d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f64480cf5e0 tx=0x7f6444009190 comp rx=0 tx=0).stop 2024-04-25T08:32:34.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.186+0000 7f644f8fc640 1 -- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f644815f5f0 msgr2=0x7f64480c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.186+0000 7f644f8fc640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f644815f5f0 0x7f64480c88e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f644000c700 tx=0x7f644000cbc0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.186+0000 7f644f8fc640 1 -- 172.21.15.38:0/1006725460 shutdown_connections 2024-04-25T08:32:34.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.186+0000 7f644f8fc640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f642c06b4d0 0x7f642c06d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.186+0000 7f644f8fc640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f644815f5f0 0x7f64480c88e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.186+0000 7f644f8fc640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f644815b990 0x7f6448157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.186+0000 7f644f8fc640 1 --2- 172.21.15.38:0/1006725460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64480d1b50 0x7f644815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.186+0000 7f644f8fc640 1 -- 172.21.15.38:0/1006725460 >> 172.21.15.38:0/1006725460 conn(0x7f644808b000 msgr2=0x7f644806fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:34.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.186+0000 7f644f8fc640 1 -- 172.21.15.38:0/1006725460 shutdown_connections 2024-04-25T08:32:34.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.186+0000 7f644f8fc640 1 -- 172.21.15.38:0/1006725460 wait complete. 2024-04-25T08:32:34.192 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:34.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.299+0000 7ff755b5c640 1 Processor -- start 2024-04-25T08:32:34.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.299+0000 7ff755b5c640 1 -- start start 2024-04-25T08:32:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.300+0000 7ff755b5c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff750166610 0x7ff750168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.300+0000 7ff755b5c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7500d2060 con 0x7ff7500d0320 2024-04-25T08:32:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.300+0000 7ff755b5c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7500d2330 con 0x7ff7500d33d0 2024-04-25T08:32:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.300+0000 7ff755b5c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7500d2610 con 0x7ff750166610 2024-04-25T08:32:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.300+0000 7ff74f7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff7500d0320 0x7ff7500d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56476/0 (socket says 172.21.15.38:56476) 2024-04-25T08:32:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.300+0000 7ff74f7fe640 1 -- 172.21.15.38:0/4143602268 learned_addr learned my addr 172.21.15.38:0/4143602268 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.301+0000 7ff74ffff640 1 --2- 172.21.15.38:0/4143602268 >> v2:172.21.15.107:3300/0 conn(0x7ff750166610 0x7ff750168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.301+0000 7ff74e7fc640 1 -- 172.21.15.38:0/4143602268 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717030459 0 0) 0x7ff7500d2060 con 0x7ff7500d0320 2024-04-25T08:32:34.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.301+0000 7ff74e7fc640 1 -- 172.21.15.38:0/4143602268 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff738003520 con 0x7ff7500d0320 2024-04-25T08:32:34.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.301+0000 7ff74e7fc640 1 -- 172.21.15.38:0/4143602268 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337401415 0 0) 0x7ff7500d2330 con 0x7ff7500d33d0 2024-04-25T08:32:34.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.301+0000 7ff74e7fc640 1 -- 172.21.15.38:0/4143602268 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7500d2060 con 0x7ff7500d33d0 2024-04-25T08:32:34.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.301+0000 7ff74e7fc640 1 -- 172.21.15.38:0/4143602268 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2545951732 0 0) 0x7ff738003520 con 0x7ff7500d0320 2024-04-25T08:32:34.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.301+0000 7ff74ffff640 1 -- 172.21.15.38:0/4143602268 >> v1:172.21.15.107:6789/0 conn(0x7ff7500d33d0 legacy=0x7ff7500d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.301+0000 7ff74ffff640 1 -- 172.21.15.38:0/4143602268 >> v1:172.21.15.38:6789/0 conn(0x7ff7500d0320 legacy=0x7ff7500d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.301+0000 7ff74ffff640 1 -- 172.21.15.38:0/4143602268 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7500d28f0 con 0x7ff750166610 2024-04-25T08:32:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.302+0000 7ff74ffff640 1 --2- 172.21.15.38:0/4143602268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750166610 0x7ff750168a00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff748009d90 tx=0x7ff74802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c12f36a0009d00e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.302+0000 7ff74e7fc640 1 -- 172.21.15.38:0/4143602268 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740001670 con 0x7ff750166610 2024-04-25T08:32:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.302+0000 7ff74e7fc640 1 -- 172.21.15.38:0/4143602268 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff748007540 con 0x7ff750166610 2024-04-25T08:32:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.302+0000 7ff74e7fc640 1 -- 172.21.15.38:0/4143602268 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7480078f0 con 0x7ff750166610 2024-04-25T08:32:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.302+0000 7ff755b5c640 1 -- 172.21.15.38:0/4143602268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750166610 msgr2=0x7ff750168a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.302+0000 7ff755b5c640 1 --2- 172.21.15.38:0/4143602268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750166610 0x7ff750168a00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff748009d90 tx=0x7ff74802a710 comp rx=0 tx=0).stop 2024-04-25T08:32:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.302+0000 7ff755b5c640 1 -- 172.21.15.38:0/4143602268 shutdown_connections 2024-04-25T08:32:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.302+0000 7ff755b5c640 1 --2- 172.21.15.38:0/4143602268 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750166610 0x7ff750168a00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.302+0000 7ff755b5c640 1 -- 172.21.15.38:0/4143602268 >> 172.21.15.38:0/4143602268 conn(0x7ff7500668f0 msgr2=0x7ff750066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.302+0000 7ff755b5c640 1 -- 172.21.15.38:0/4143602268 shutdown_connections 2024-04-25T08:32:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.302+0000 7ff755b5c640 1 -- 172.21.15.38:0/4143602268 wait complete. 2024-04-25T08:32:34.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.304+0000 7ff755b5c640 1 Processor -- start 2024-04-25T08:32:34.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.304+0000 7ff755b5c640 1 -- start start 2024-04-25T08:32:34.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.304+0000 7ff755b5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7500d0320 0x7ff750157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.304+0000 7ff755b5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7500d33d0 0x7ff7500b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.304+0000 7ff755b5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750166610 0x7ff75015fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.304+0000 7ff755b5c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff750161210 con 0x7ff7500d33d0 2024-04-25T08:32:34.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff755b5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7500d2eb0 con 0x7ff750166610 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff755b5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff750161390 con 0x7ff7500d0320 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff74effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7500d33d0 0x7ff7500b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff74effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7500d33d0 0x7ff7500b3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44766/0 (socket says 172.21.15.38:44766) 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff74effd640 1 -- 172.21.15.38:0/1439942045 learned_addr learned my addr 172.21.15.38:0/1439942045 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff74f7fe640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7500d0320 0x7ff750157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff74ffff640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750166610 0x7ff75015fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff74effd640 1 -- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7500d0320 msgr2=0x7ff750157f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff74effd640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7500d0320 0x7ff750157f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff74effd640 1 -- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750166610 msgr2=0x7ff75015fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff74effd640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750166610 0x7ff75015fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff74effd640 1 -- 172.21.15.38:0/1439942045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7500b47f0 con 0x7ff7500d33d0 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff74effd640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7500d33d0 0x7ff7500b3e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff74400c490 tx=0x7ff74400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff737fff640 1 -- 172.21.15.38:0/1439942045 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7440052d0 con 0x7ff7500d33d0 2024-04-25T08:32:34.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff737fff640 1 -- 172.21.15.38:0/1439942045 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff744005470 con 0x7ff7500d33d0 2024-04-25T08:32:34.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff755b5c640 1 -- 172.21.15.38:0/1439942045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7501f33d0 con 0x7ff7500d33d0 2024-04-25T08:32:34.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff737fff640 1 -- 172.21.15.38:0/1439942045 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744012870 con 0x7ff7500d33d0 2024-04-25T08:32:34.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.305+0000 7ff755b5c640 1 -- 172.21.15.38:0/1439942045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7501f3990 con 0x7ff7500d33d0 2024-04-25T08:32:34.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.307+0000 7ff755b5c640 1 -- 172.21.15.38:0/1439942045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff750068e80 con 0x7ff7500d33d0 2024-04-25T08:32:34.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.307+0000 7ff737fff640 1 -- 172.21.15.38:0/1439942045 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff744012a10 con 0x7ff7500d33d0 2024-04-25T08:32:34.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.307+0000 7ff737fff640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff73c06b4d0 0x7ff73c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.308+0000 7ff737fff640 1 -- 172.21.15.38:0/1439942045 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7ff74408a250 con 0x7ff7500d33d0 2024-04-25T08:32:34.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.308+0000 7ff74f7fe640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff73c06b4d0 0x7ff73c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.308+0000 7ff74f7fe640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff73c06b4d0 0x7ff73c06d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff7400017e0 tx=0x7ff740001610 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:34.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.312+0000 7ff737fff640 1 -- 172.21.15.38:0/1439942045 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff744058580 con 0x7ff7500d33d0 2024-04-25T08:32:34.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.490+0000 7ff755b5c640 1 -- 172.21.15.38:0/1439942045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff7501f3b60 con 0x7ff7500d33d0 2024-04-25T08:32:34.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.490+0000 7ff737fff640 1 -- 172.21.15.38:0/1439942045 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff74405c620 con 0x7ff7500d33d0 2024-04-25T08:32:34.486 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:34.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.492+0000 7ff755b5c640 1 -- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff73c06b4d0 msgr2=0x7ff73c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.492+0000 7ff755b5c640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff73c06b4d0 0x7ff73c06d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff7400017e0 tx=0x7ff740001610 comp rx=0 tx=0).stop 2024-04-25T08:32:34.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.492+0000 7ff755b5c640 1 -- 172.21.15.38:0/1439942045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7500d33d0 msgr2=0x7ff7500b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.492+0000 7ff755b5c640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7500d33d0 0x7ff7500b3e10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff74400c490 tx=0x7ff74400c950 comp rx=0 tx=0).stop 2024-04-25T08:32:34.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.492+0000 7ff755b5c640 1 -- 172.21.15.38:0/1439942045 shutdown_connections 2024-04-25T08:32:34.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.492+0000 7ff755b5c640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff750166610 0x7ff75015fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.492+0000 7ff755b5c640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7500d33d0 0x7ff7500b3e10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.492+0000 7ff755b5c640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff73c06b4d0 0x7ff73c06d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.492+0000 7ff755b5c640 1 --2- 172.21.15.38:0/1439942045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7500d0320 0x7ff750157f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.492+0000 7ff755b5c640 1 -- 172.21.15.38:0/1439942045 >> 172.21.15.38:0/1439942045 conn(0x7ff7500668f0 msgr2=0x7ff7500b79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:34.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.492+0000 7ff755b5c640 1 -- 172.21.15.38:0/1439942045 shutdown_connections 2024-04-25T08:32:34.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.492+0000 7ff755b5c640 1 -- 172.21.15.38:0/1439942045 wait complete. 2024-04-25T08:32:34.499 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:32:34.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.605+0000 7f1bbe90a640 1 Processor -- start 2024-04-25T08:32:34.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.605+0000 7f1bbe90a640 1 -- start start 2024-04-25T08:32:34.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bbe90a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb80b9180 0x7f1bb80d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bbe90a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1bb80b2ab0 0x7f1bb80d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bbe90a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb80b9610 con 0x7f1bb8160ef0 2024-04-25T08:32:34.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bbe90a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb8161cf0 con 0x7f1bb80b2ab0 2024-04-25T08:32:34.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bbe90a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb81612d0 con 0x7f1bb80b9180 2024-04-25T08:32:34.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bb7fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1bb8160ef0 0x7f1bb80d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56488/0 (socket says 172.21.15.38:56488) 2024-04-25T08:32:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bb7fff640 1 -- 172.21.15.38:0/3658626561 learned_addr learned my addr 172.21.15.38:0/3658626561 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bb77fe640 1 --2- 172.21.15.38:0/3658626561 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb80b9180 0x7f1bb80d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bbcea7640 1 --2- 172.21.15.38:0/3658626561 >> v2:172.21.15.107:3300/0 conn(0x7f1bb80b2ab0 0x7f1bb80d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bb6ffd640 1 -- 172.21.15.38:0/3658626561 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350601768 0 0) 0x7f1bb80b9610 con 0x7f1bb8160ef0 2024-04-25T08:32:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bb6ffd640 1 -- 172.21.15.38:0/3658626561 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ba0003520 con 0x7f1bb8160ef0 2024-04-25T08:32:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bbcea7640 1 -- 172.21.15.38:0/3658626561 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb80b9180 msgr2=0x7f1bb80d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bbcea7640 1 --2- 172.21.15.38:0/3658626561 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb80b9180 0x7f1bb80d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bbcea7640 1 -- 172.21.15.38:0/3658626561 >> v1:172.21.15.38:6789/0 conn(0x7f1bb8160ef0 legacy=0x7f1bb80d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bbcea7640 1 -- 172.21.15.38:0/3658626561 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb81616a0 con 0x7f1bb80b2ab0 2024-04-25T08:32:34.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.606+0000 7f1bb77fe640 1 --2- 172.21.15.38:0/3658626561 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb80b9180 0x7f1bb80d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.607+0000 7f1bbcea7640 1 --2- 172.21.15.38:0/3658626561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb80b2ab0 0x7f1bb80d1eb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009d90 tx=0x7f1bb002a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cf3327d512436b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.607+0000 7f1bb6ffd640 1 -- 172.21.15.38:0/3658626561 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0031600 con 0x7f1bb80b2ab0 2024-04-25T08:32:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.607+0000 7f1bb6ffd640 1 -- 172.21.15.38:0/3658626561 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb00317a0 con 0x7f1bb80b2ab0 2024-04-25T08:32:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.607+0000 7f1bb6ffd640 1 -- 172.21.15.38:0/3658626561 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0031b70 con 0x7f1bb80b2ab0 2024-04-25T08:32:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.608+0000 7f1bbe90a640 1 -- 172.21.15.38:0/3658626561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb80b2ab0 msgr2=0x7f1bb80d1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.608+0000 7f1bbe90a640 1 --2- 172.21.15.38:0/3658626561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb80b2ab0 0x7f1bb80d1eb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0009d90 tx=0x7f1bb002a5d0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.608+0000 7f1bbe90a640 1 -- 172.21.15.38:0/3658626561 shutdown_connections 2024-04-25T08:32:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.608+0000 7f1bbe90a640 1 --2- 172.21.15.38:0/3658626561 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb80b2ab0 0x7f1bb80d1eb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.608+0000 7f1bbe90a640 1 --2- 172.21.15.38:0/3658626561 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb80b9180 0x7f1bb80d1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.608+0000 7f1bbe90a640 1 -- 172.21.15.38:0/3658626561 >> 172.21.15.38:0/3658626561 conn(0x7f1bb8066d10 msgr2=0x7f1bb8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.608+0000 7f1bbe90a640 1 -- 172.21.15.38:0/3658626561 shutdown_connections 2024-04-25T08:32:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.608+0000 7f1bbe90a640 1 -- 172.21.15.38:0/3658626561 wait complete. 2024-04-25T08:32:34.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.609+0000 7f1bbe90a640 1 Processor -- start 2024-04-25T08:32:34.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.609+0000 7f1bbe90a640 1 -- start start 2024-04-25T08:32:34.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.610+0000 7f1bbe90a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb80b2ab0 0x7f1bb8154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.610+0000 7f1bbe90a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1bb8160ef0 0x7f1bb81f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.610+0000 7f1bbe90a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb8156290 0x7f1bb80b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.610+0000 7f1bbe90a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb80d1040 con 0x7f1bb8160ef0 2024-04-25T08:32:34.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.610+0000 7f1bbe90a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb80d0ec0 con 0x7f1bb8156290 2024-04-25T08:32:34.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.610+0000 7f1bbe90a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8162180 con 0x7f1bb80b2ab0 2024-04-25T08:32:34.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.610+0000 7f1bb77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1bb8160ef0 0x7f1bb81f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.610+0000 7f1bb77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1bb8160ef0 0x7f1bb81f7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44772/0 (socket says 172.21.15.38:44772) 2024-04-25T08:32:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.610+0000 7f1bb77fe640 1 -- 172.21.15.38:0/2847619660 learned_addr learned my addr 172.21.15.38:0/2847619660 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.610+0000 7f1bbcea7640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb8156290 0x7f1bb80b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.610+0000 7f1bb7fff640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb80b2ab0 0x7f1bb8154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.611+0000 7f1bb77fe640 1 -- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb80b2ab0 msgr2=0x7f1bb8154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.611+0000 7f1bb77fe640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb80b2ab0 0x7f1bb8154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.611+0000 7f1bb77fe640 1 -- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb8156290 msgr2=0x7f1bb80b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.611+0000 7f1bb77fe640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb8156290 0x7f1bb80b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.611+0000 7f1bb77fe640 1 -- 172.21.15.38:0/2847619660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb81f7550 con 0x7f1bb8160ef0 2024-04-25T08:32:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.611+0000 7f1bb77fe640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1bb8160ef0 0x7f1bb81f7010 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800ebe0 tx=0x7f1ba800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.611+0000 7f1bb4ff9640 1 -- 172.21.15.38:0/2847619660 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8017070 con 0x7f1bb8160ef0 2024-04-25T08:32:34.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.611+0000 7f1bb4ff9640 1 -- 172.21.15.38:0/2847619660 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba800cc30 con 0x7f1bb8160ef0 2024-04-25T08:32:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.611+0000 7f1bbe90a640 1 -- 172.21.15.38:0/2847619660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb81f2ec0 con 0x7f1bb8160ef0 2024-04-25T08:32:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.611+0000 7f1bb4ff9640 1 -- 172.21.15.38:0/2847619660 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba8013470 con 0x7f1bb8160ef0 2024-04-25T08:32:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.611+0000 7f1bbe90a640 1 -- 172.21.15.38:0/2847619660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb8156af0 con 0x7f1bb8160ef0 2024-04-25T08:32:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.612+0000 7f1bbe90a640 1 -- 172.21.15.38:0/2847619660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb804ef90 con 0x7f1bb8160ef0 2024-04-25T08:32:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.612+0000 7f1bb4ff9640 1 -- 172.21.15.38:0/2847619660 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ba801a070 con 0x7f1bb8160ef0 2024-04-25T08:32:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.613+0000 7f1bb4ff9640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1ba406b500 0x7f1ba406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.613+0000 7f1bb4ff9640 1 -- 172.21.15.38:0/2847619660 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f1ba808a4f0 con 0x7f1bb8160ef0 2024-04-25T08:32:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.613+0000 7f1bb7fff640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1ba406b500 0x7f1ba406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:34.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.614+0000 7f1bb7fff640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1ba406b500 0x7f1ba406d9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1bac005880 tx=0x7f1bac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:34.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.617+0000 7f1bb4ff9640 1 -- 172.21.15.38:0/2847619660 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1bb81f7550 con 0x7f1bb8160ef0 2024-04-25T08:32:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.819+0000 7f1bbe90a640 1 -- 172.21.15.38:0/2847619660 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1bb80b4010 con 0x7f1bb8160ef0 2024-04-25T08:32:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.819+0000 7f1bb4ff9640 1 -- 172.21.15.38:0/2847619660 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1ba80588c0 con 0x7f1bb8160ef0 2024-04-25T08:32:34.815 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:32:34.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.821+0000 7f1bbe90a640 1 -- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1ba406b500 msgr2=0x7f1ba406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.821+0000 7f1bbe90a640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1ba406b500 0x7f1ba406d9c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1bac005880 tx=0x7f1bac009040 comp rx=0 tx=0).stop 2024-04-25T08:32:34.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.821+0000 7f1bbe90a640 1 -- 172.21.15.38:0/2847619660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1bb8160ef0 msgr2=0x7f1bb81f7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:34.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.821+0000 7f1bbe90a640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1bb8160ef0 0x7f1bb81f7010 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1ba800ebe0 tx=0x7f1ba800c520 comp rx=0 tx=0).stop 2024-04-25T08:32:34.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.821+0000 7f1bbe90a640 1 -- 172.21.15.38:0/2847619660 shutdown_connections 2024-04-25T08:32:34.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.821+0000 7f1bbe90a640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1bb8156290 0x7f1bb80b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.821+0000 7f1bbe90a640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1bb8160ef0 0x7f1bb81f7010 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.821+0000 7f1bbe90a640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1ba406b500 0x7f1ba406d9c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.821+0000 7f1bbe90a640 1 --2- 172.21.15.38:0/2847619660 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1bb80b2ab0 0x7f1bb8154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:34.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.821+0000 7f1bbe90a640 1 -- 172.21.15.38:0/2847619660 >> 172.21.15.38:0/2847619660 conn(0x7f1bb8066d10 msgr2=0x7f1bb80c6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:34.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.822+0000 7f1bbe90a640 1 -- 172.21.15.38:0/2847619660 shutdown_connections 2024-04-25T08:32:34.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:34.822+0000 7f1bbe90a640 1 -- 172.21.15.38:0/2847619660 wait complete. 2024-04-25T08:32:34.830 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2024-04-25T08:32:34.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:34.830 DEBUG:teuthology.orchestra.run.smithi107:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T08:32:34.858 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:34.859 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:32:34.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:34.859 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:32:34.915 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:32:34.915 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:35.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.020+0000 7fcf69c9e640 1 Processor -- start 2024-04-25T08:32:35.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.020+0000 7fcf69c9e640 1 -- start start 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.021+0000 7fcf69c9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf6415ddf0 0x7fcf64161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.021+0000 7fcf69c9e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcf640b93f0 0x7fcf640d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.021+0000 7fcf69c9e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf64161e00 con 0x7fcf640b5710 2024-04-25T08:32:35.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.021+0000 7fcf69c9e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf641620e0 con 0x7fcf640b93f0 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.021+0000 7fcf69c9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf641623c0 con 0x7fcf6415ddf0 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.021+0000 7fcf62ffd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf6415ddf0 0x7fcf64161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.021+0000 7fcf63fff640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcf640b93f0 0x7fcf640d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.021+0000 7fcf62ffd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf6415ddf0 0x7fcf64161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:44410/0 (socket says 172.21.15.38:44410) 2024-04-25T08:32:35.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.021+0000 7fcf62ffd640 1 -- 172.21.15.38:0/290592820 learned_addr learned my addr 172.21.15.38:0/290592820 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.022+0000 7fcf62ffd640 1 -- 172.21.15.38:0/290592820 >> v2:172.21.15.107:3300/0 conn(0x7fcf640b93f0 msgr2=0x7fcf640d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.022+0000 7fcf62ffd640 1 --2- 172.21.15.38:0/290592820 >> v2:172.21.15.107:3300/0 conn(0x7fcf640b93f0 0x7fcf640d2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.022+0000 7fcf62ffd640 1 -- 172.21.15.38:0/290592820 >> v1:172.21.15.107:6789/0 conn(0x7fcf640b5710 legacy=0x7fcf640b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.022+0000 7fcf62ffd640 1 -- 172.21.15.38:0/290592820 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf640b9870 con 0x7fcf6415ddf0 2024-04-25T08:32:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.022+0000 7fcf63fff640 1 --2- 172.21.15.38:0/290592820 >> v2:172.21.15.107:3300/0 conn(0x7fcf640b93f0 0x7fcf640d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.023+0000 7fcf62ffd640 1 --2- 172.21.15.38:0/290592820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf6415ddf0 0x7fcf64161700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009870 tx=0x7fcf5802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77ab88acde87cc01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.023+0000 7fcf627fc640 1 -- 172.21.15.38:0/290592820 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58037070 con 0x7fcf6415ddf0 2024-04-25T08:32:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.023+0000 7fcf627fc640 1 -- 172.21.15.38:0/290592820 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf58032440 con 0x7fcf6415ddf0 2024-04-25T08:32:35.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.023+0000 7fcf627fc640 1 -- 172.21.15.38:0/290592820 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58032810 con 0x7fcf6415ddf0 2024-04-25T08:32:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.023+0000 7fcf69c9e640 1 -- 172.21.15.38:0/290592820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf6415ddf0 msgr2=0x7fcf64161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.023+0000 7fcf69c9e640 1 --2- 172.21.15.38:0/290592820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf6415ddf0 0x7fcf64161700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009870 tx=0x7fcf5802a270 comp rx=0 tx=0).stop 2024-04-25T08:32:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.023+0000 7fcf69c9e640 1 -- 172.21.15.38:0/290592820 shutdown_connections 2024-04-25T08:32:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.023+0000 7fcf69c9e640 1 --2- 172.21.15.38:0/290592820 >> v2:172.21.15.107:3300/0 conn(0x7fcf640b93f0 0x7fcf640d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.023+0000 7fcf69c9e640 1 --2- 172.21.15.38:0/290592820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf6415ddf0 0x7fcf64161700 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.023+0000 7fcf69c9e640 1 -- 172.21.15.38:0/290592820 >> 172.21.15.38:0/290592820 conn(0x7fcf640d07c0 msgr2=0x7fcf640d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.023+0000 7fcf69c9e640 1 -- 172.21.15.38:0/290592820 shutdown_connections 2024-04-25T08:32:35.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.023+0000 7fcf69c9e640 1 -- 172.21.15.38:0/290592820 wait complete. 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.025+0000 7fcf69c9e640 1 Processor -- start 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.025+0000 7fcf69c9e640 1 -- start start 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.025+0000 7fcf69c9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf640b5710 0x7fcf64157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.025+0000 7fcf69c9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf640b93f0 0x7fcf6415f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.025+0000 7fcf69c9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf6415ddf0 0x7fcf6415a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.025+0000 7fcf69c9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf640d2c50 con 0x7fcf640b5710 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.025+0000 7fcf69c9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf640d2ad0 con 0x7fcf6415ddf0 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.025+0000 7fcf69c9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf640d2dd0 con 0x7fcf640b93f0 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf640b5710 0x7fcf64157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf637fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf640b5710 0x7fcf64157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44786/0 (socket says 172.21.15.38:44786) 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf637fe640 1 -- 172.21.15.38:0/1777279109 learned_addr learned my addr 172.21.15.38:0/1777279109 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf63fff640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf6415ddf0 0x7fcf6415a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf62ffd640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf640b93f0 0x7fcf6415f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf637fe640 1 -- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf640b93f0 msgr2=0x7fcf6415f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf637fe640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf640b93f0 0x7fcf6415f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf637fe640 1 -- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf6415ddf0 msgr2=0x7fcf6415a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf637fe640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf6415ddf0 0x7fcf6415a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf637fe640 1 -- 172.21.15.38:0/1777279109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6415fca0 con 0x7fcf640b5710 2024-04-25T08:32:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf62ffd640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf640b93f0 0x7fcf6415f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf63fff640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf6415ddf0 0x7fcf6415a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf637fe640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf640b5710 0x7fcf64157b60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400c400 tx=0x7fcf5400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf4bfff640 1 -- 172.21.15.38:0/1777279109 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf540052d0 con 0x7fcf640b5710 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf4bfff640 1 -- 172.21.15.38:0/1777279109 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf54005490 con 0x7fcf640b5710 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf69c9e640 1 -- 172.21.15.38:0/1777279109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6415ff80 con 0x7fcf640b5710 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf4bfff640 1 -- 172.21.15.38:0/1777279109 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54007260 con 0x7fcf640b5710 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.026+0000 7fcf69c9e640 1 -- 172.21.15.38:0/1777279109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf64160540 con 0x7fcf640b5710 2024-04-25T08:32:35.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.027+0000 7fcf69c9e640 1 -- 172.21.15.38:0/1777279109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf64068e80 con 0x7fcf640b5710 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.028+0000 7fcf4bfff640 1 -- 172.21.15.38:0/1777279109 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf54007400 con 0x7fcf640b5710 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.028+0000 7fcf4bfff640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcf5006b4d0 0x7fcf5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.028+0000 7fcf4bfff640 1 -- 172.21.15.38:0/1777279109 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fcf54089ba0 con 0x7fcf640b5710 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.028+0000 7fcf62ffd640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcf5006b4d0 0x7fcf5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.029+0000 7fcf62ffd640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcf5006b4d0 0x7fcf5006d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcf58034000 tx=0x7fcf5802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.032+0000 7fcf4bfff640 1 -- 172.21.15.38:0/1777279109 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf540585b0 con 0x7fcf640b5710 2024-04-25T08:32:35.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.250+0000 7fcf69c9e640 1 -- 172.21.15.38:0/1777279109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf640d0f90 con 0x7fcf640b5710 2024-04-25T08:32:35.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.250+0000 7fcf4bfff640 1 -- 172.21.15.38:0/1777279109 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7fcf54057eb0 con 0x7fcf640b5710 2024-04-25T08:32:35.247 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:35.247 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4242612806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4242612806},{"type":"v1","addr":"172.21.15.38:6835","nonce":4242612806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2154167637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":2154167637},{"type":"v1","addr":"172.21.15.107:6837","nonce":2154167637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/678452936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":678452936},{"type":"v1","addr":"172.21.15.38:6834","nonce":678452936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:31:35.665008+0000","modified":"2024-04-25T08:31:51.799830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6836/1104921666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1104921666},{"type":"v1","addr":"172.21.15.107:6836","nonce":1104921666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:32:35.247 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-04-25T08:32:35.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.252+0000 7fcf69c9e640 1 -- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcf5006b4d0 msgr2=0x7fcf5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.252+0000 7fcf69c9e640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcf5006b4d0 0x7fcf5006d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcf58034000 tx=0x7fcf5802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:35.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.252+0000 7fcf69c9e640 1 -- 172.21.15.38:0/1777279109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf640b5710 msgr2=0x7fcf64157b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.252+0000 7fcf69c9e640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf640b5710 0x7fcf64157b60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400c400 tx=0x7fcf5400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.253+0000 7fcf69c9e640 1 -- 172.21.15.38:0/1777279109 shutdown_connections 2024-04-25T08:32:35.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.253+0000 7fcf69c9e640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf6415ddf0 0x7fcf6415a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.253+0000 7fcf69c9e640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcf5006b4d0 0x7fcf5006d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.253+0000 7fcf69c9e640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf640b93f0 0x7fcf6415f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.253+0000 7fcf69c9e640 1 --2- 172.21.15.38:0/1777279109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf640b5710 0x7fcf64157b60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.253+0000 7fcf69c9e640 1 -- 172.21.15.38:0/1777279109 >> 172.21.15.38:0/1777279109 conn(0x7fcf640d07c0 msgr2=0x7fcf640b6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.253+0000 7fcf69c9e640 1 -- 172.21.15.38:0/1777279109 shutdown_connections 2024-04-25T08:32:35.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.253+0000 7fcf69c9e640 1 -- 172.21.15.38:0/1777279109 wait complete. 2024-04-25T08:32:35.260 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.367+0000 7feaf7aee640 1 Processor -- start 2024-04-25T08:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.367+0000 7feaf7aee640 1 -- start start 2024-04-25T08:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.368+0000 7feaf7aee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feaf00d0f60 0x7feaf00b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.368+0000 7feaf7aee640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf00d3e70 con 0x7feaf015dc10 2024-04-25T08:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.368+0000 7feaf7aee640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf00d4150 con 0x7feaf00d3900 2024-04-25T08:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.368+0000 7feaf7aee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf015dff0 con 0x7feaf00d0f60 2024-04-25T08:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.368+0000 7feaf588a640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7feaf015dc10 0x7feaf00ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56496/0 (socket says 172.21.15.38:56496) 2024-04-25T08:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.368+0000 7feaf588a640 1 -- 172.21.15.38:0/3353641314 learned_addr learned my addr 172.21.15.38:0/3353641314 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.368+0000 7feaf608b640 1 --2- 172.21.15.38:0/3353641314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feaf00d0f60 0x7feaf00b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.369+0000 7feaf4888640 1 -- 172.21.15.38:0/3353641314 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973029097 0 0) 0x7feaf00d3e70 con 0x7feaf015dc10 2024-04-25T08:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.369+0000 7feaf4888640 1 -- 172.21.15.38:0/3353641314 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead8003520 con 0x7feaf015dc10 2024-04-25T08:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.369+0000 7feaf608b640 1 -- 172.21.15.38:0/3353641314 >> v1:172.21.15.107:6789/0 conn(0x7feaf00d3900 legacy=0x7feaf00d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.369+0000 7feaf608b640 1 -- 172.21.15.38:0/3353641314 >> v1:172.21.15.38:6789/0 conn(0x7feaf015dc10 legacy=0x7feaf00ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.369+0000 7feaf4888640 1 -- 172.21.15.38:0/3353641314 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4261447813 0 0) 0x7feaf00d4150 con 0x7feaf00d3900 2024-04-25T08:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.369+0000 7feaf608b640 1 -- 172.21.15.38:0/3353641314 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf00d4430 con 0x7feaf00d0f60 2024-04-25T08:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.370+0000 7feaf608b640 1 --2- 172.21.15.38:0/3353641314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feaf00d0f60 0x7feaf00b2370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feae0009dc0 tx=0x7feae002a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63c647334dc2a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.370+0000 7feaf4888640 1 -- 172.21.15.38:0/3353641314 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae002ae60 con 0x7feaf00d0f60 2024-04-25T08:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.370+0000 7feaf4888640 1 -- 172.21.15.38:0/3353641314 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae0007740 con 0x7feaf00d0f60 2024-04-25T08:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.370+0000 7feaf4888640 1 -- 172.21.15.38:0/3353641314 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae0007af0 con 0x7feaf00d0f60 2024-04-25T08:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.370+0000 7feaf7aee640 1 -- 172.21.15.38:0/3353641314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feaf00d0f60 msgr2=0x7feaf00b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.370+0000 7feaf7aee640 1 --2- 172.21.15.38:0/3353641314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feaf00d0f60 0x7feaf00b2370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7feae0009dc0 tx=0x7feae002a640 comp rx=0 tx=0).stop 2024-04-25T08:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.370+0000 7feaf7aee640 1 -- 172.21.15.38:0/3353641314 shutdown_connections 2024-04-25T08:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.370+0000 7feaf7aee640 1 --2- 172.21.15.38:0/3353641314 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feaf00d0f60 0x7feaf00b2370 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.370+0000 7feaf7aee640 1 -- 172.21.15.38:0/3353641314 >> 172.21.15.38:0/3353641314 conn(0x7feaf0066d10 msgr2=0x7feaf0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.370+0000 7feaf7aee640 1 -- 172.21.15.38:0/3353641314 shutdown_connections 2024-04-25T08:32:35.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.371+0000 7feaf7aee640 1 -- 172.21.15.38:0/3353641314 wait complete. 2024-04-25T08:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.372+0000 7feaf7aee640 1 Processor -- start 2024-04-25T08:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.372+0000 7feaf7aee640 1 -- start start 2024-04-25T08:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.372+0000 7feaf7aee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feaf00d0f60 0x7feaf00c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.372+0000 7feaf7aee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf00d3900 0x7feaf015f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.372+0000 7feaf7aee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf015dc10 0x7feaf015fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.372+0000 7feaf7aee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf00d4710 con 0x7feaf00d3900 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.372+0000 7feaf7aee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf00cef30 con 0x7feaf015dc10 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.372+0000 7feaf7aee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf00b80f0 con 0x7feaf00d0f60 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.372+0000 7feaf5089640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf00d3900 0x7feaf015f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.372+0000 7feaf5089640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf00d3900 0x7feaf015f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44788/0 (socket says 172.21.15.38:44788) 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.372+0000 7feaf5089640 1 -- 172.21.15.38:0/2700385032 learned_addr learned my addr 172.21.15.38:0/2700385032 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaf608b640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf015dc10 0x7feaf015fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaf588a640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feaf00d0f60 0x7feaf00c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaf5089640 1 -- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feaf00d0f60 msgr2=0x7feaf00c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaf5089640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feaf00d0f60 0x7feaf00c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaf5089640 1 -- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf015dc10 msgr2=0x7feaf015fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaf5089640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf015dc10 0x7feaf015fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaf5089640 1 -- 172.21.15.38:0/2700385032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf014a6a0 con 0x7feaf00d3900 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaf5089640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf00d3900 0x7feaf015f8c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7feae400c4b0 tx=0x7feae400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaee7fc640 1 -- 172.21.15.38:0/2700385032 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4017070 con 0x7feaf00d3900 2024-04-25T08:32:35.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaee7fc640 1 -- 172.21.15.38:0/2700385032 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae4005480 con 0x7feaf00d3900 2024-04-25T08:32:35.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaf7aee640 1 -- 172.21.15.38:0/2700385032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf014a960 con 0x7feaf00d3900 2024-04-25T08:32:35.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaee7fc640 1 -- 172.21.15.38:0/2700385032 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4003830 con 0x7feaf00d3900 2024-04-25T08:32:35.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.373+0000 7feaf7aee640 1 -- 172.21.15.38:0/2700385032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf00c7410 con 0x7feaf00d3900 2024-04-25T08:32:35.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.374+0000 7feaf7aee640 1 -- 172.21.15.38:0/2700385032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf0068e80 con 0x7feaf00d3900 2024-04-25T08:32:35.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.375+0000 7feaee7fc640 1 -- 172.21.15.38:0/2700385032 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feae401a070 con 0x7feaf00d3900 2024-04-25T08:32:35.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.375+0000 7feaee7fc640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feadc06b500 0x7feadc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.375+0000 7feaee7fc640 1 -- 172.21.15.38:0/2700385032 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7feae40899e0 con 0x7feaf00d3900 2024-04-25T08:32:35.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.375+0000 7feaf588a640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feadc06b500 0x7feadc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.376+0000 7feaf588a640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feadc06b500 0x7feadc06d9c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feae8001200 tx=0x7feae8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.379+0000 7feaee7fc640 1 -- 172.21.15.38:0/2700385032 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feae4058400 con 0x7feaf00d3900 2024-04-25T08:32:35.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.560+0000 7feaf7aee640 1 -- 172.21.15.38:0/2700385032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feaf00d0cf0 con 0x7feaf00d3900 2024-04-25T08:32:35.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.561+0000 7feaee7fc640 1 -- 172.21.15.38:0/2700385032 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18969 (secure 0 0 0) 0x7feae4057d00 con 0x7feaf00d3900 2024-04-25T08:32:35.557 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:35.557 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:16.572337+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.a","mappings":[{"from":4,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:35.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.563+0000 7feaf7aee640 1 -- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feadc06b500 msgr2=0x7feadc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.563+0000 7feaf7aee640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feadc06b500 0x7feadc06d9c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7feae8001200 tx=0x7feae8009040 comp rx=0 tx=0).stop 2024-04-25T08:32:35.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.563+0000 7feaf7aee640 1 -- 172.21.15.38:0/2700385032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf00d3900 msgr2=0x7feaf015f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.563+0000 7feaf7aee640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf00d3900 0x7feaf015f8c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7feae400c4b0 tx=0x7feae400c970 comp rx=0 tx=0).stop 2024-04-25T08:32:35.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.563+0000 7feaf7aee640 1 -- 172.21.15.38:0/2700385032 shutdown_connections 2024-04-25T08:32:35.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.563+0000 7feaf7aee640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf015dc10 0x7feaf015fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.563+0000 7feaf7aee640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feaf00d3900 0x7feaf015f8c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.563+0000 7feaf7aee640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feadc06b500 0x7feadc06d9c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.563+0000 7feaf7aee640 1 --2- 172.21.15.38:0/2700385032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feaf00d0f60 0x7feaf00c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.563+0000 7feaf7aee640 1 -- 172.21.15.38:0/2700385032 >> 172.21.15.38:0/2700385032 conn(0x7feaf0066d10 msgr2=0x7feaf015e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.563+0000 7feaf7aee640 1 -- 172.21.15.38:0/2700385032 shutdown_connections 2024-04-25T08:32:35.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.563+0000 7feaf7aee640 1 -- 172.21.15.38:0/2700385032 wait complete. 2024-04-25T08:32:35.570 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:31:33.425500+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:31:34.648941+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:31:38.358207+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:32:35.571 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:35.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.679+0000 7fd38cf22640 1 Processor -- start 2024-04-25T08:32:35.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.679+0000 7fd38cf22640 1 -- start start 2024-04-25T08:32:35.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.680+0000 7fd38cf22640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3880d00e0 0x7fd3880cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.680+0000 7fd38cf22640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd3880cd3b0 0x7fd3880cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.680+0000 7fd38cf22640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3880cdd80 con 0x7fd3880b1d60 2024-04-25T08:32:35.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.680+0000 7fd38cf22640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38806abe0 con 0x7fd3880cd3b0 2024-04-25T08:32:35.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.680+0000 7fd38cf22640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38806b480 con 0x7fd3880d00e0 2024-04-25T08:32:35.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.680+0000 7fd386d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd3880cd3b0 0x7fd3880cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.680+0000 7fd38659c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd3880b1d60 0x7fd3880cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56510/0 (socket says 172.21.15.38:56510) 2024-04-25T08:32:35.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.680+0000 7fd38659c640 1 -- 172.21.15.38:0/3782692640 learned_addr learned my addr 172.21.15.38:0/3782692640 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:35.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.681+0000 7fd386d9d640 1 -- 172.21.15.38:0/3782692640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3880d00e0 msgr2=0x7fd3880cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.681+0000 7fd386d9d640 1 --2- 172.21.15.38:0/3782692640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3880d00e0 0x7fd3880cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.681+0000 7fd386d9d640 1 -- 172.21.15.38:0/3782692640 >> v1:172.21.15.38:6789/0 conn(0x7fd3880b1d60 legacy=0x7fd3880cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.681+0000 7fd386d9d640 1 -- 172.21.15.38:0/3782692640 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38815ef40 con 0x7fd3880cd3b0 2024-04-25T08:32:35.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.681+0000 7fd385d9b640 1 -- 172.21.15.38:0/3782692640 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448836197 0 0) 0x7fd3880cdd80 con 0x7fd3880b1d60 2024-04-25T08:32:35.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.681+0000 7fd386d9d640 1 --2- 172.21.15.38:0/3782692640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3880cd3b0 0x7fd3880cd840 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd380009d90 tx=0x7fd38002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95374bc86dfa366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.682+0000 7fd385d9b640 1 -- 172.21.15.38:0/3782692640 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380007740 con 0x7fd3880cd3b0 2024-04-25T08:32:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.682+0000 7fd385d9b640 1 -- 172.21.15.38:0/3782692640 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3800078e0 con 0x7fd3880cd3b0 2024-04-25T08:32:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.682+0000 7fd385d9b640 1 -- 172.21.15.38:0/3782692640 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd380007cb0 con 0x7fd3880cd3b0 2024-04-25T08:32:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.682+0000 7fd38cf22640 1 -- 172.21.15.38:0/3782692640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3880cd3b0 msgr2=0x7fd3880cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.682+0000 7fd38cf22640 1 --2- 172.21.15.38:0/3782692640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3880cd3b0 0x7fd3880cd840 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd380009d90 tx=0x7fd38002a660 comp rx=0 tx=0).stop 2024-04-25T08:32:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.682+0000 7fd38cf22640 1 -- 172.21.15.38:0/3782692640 shutdown_connections 2024-04-25T08:32:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.682+0000 7fd38cf22640 1 --2- 172.21.15.38:0/3782692640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3880cd3b0 0x7fd3880cd840 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.682+0000 7fd38cf22640 1 --2- 172.21.15.38:0/3782692640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3880d00e0 0x7fd3880cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.682+0000 7fd38cf22640 1 -- 172.21.15.38:0/3782692640 >> 172.21.15.38:0/3782692640 conn(0x7fd3880d06b0 msgr2=0x7fd3880d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.682+0000 7fd38cf22640 1 -- 172.21.15.38:0/3782692640 shutdown_connections 2024-04-25T08:32:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.682+0000 7fd38cf22640 1 -- 172.21.15.38:0/3782692640 wait complete. 2024-04-25T08:32:35.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.684+0000 7fd38cf22640 1 Processor -- start 2024-04-25T08:32:35.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.684+0000 7fd38cf22640 1 -- start start 2024-04-25T08:32:35.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.684+0000 7fd38cf22640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3880b1d60 0x7fd388154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.684+0000 7fd38cf22640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3880cd3b0 0x7fd388154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.684+0000 7fd38cf22640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3880d00e0 0x7fd3881537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.684+0000 7fd38cf22640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38806b910 con 0x7fd3880b1d60 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.684+0000 7fd38cf22640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38806b110 con 0x7fd3880cd3b0 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.684+0000 7fd38cf22640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38806ba90 con 0x7fd3880d00e0 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.684+0000 7fd38659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3880b1d60 0x7fd388154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.684+0000 7fd38659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3880b1d60 0x7fd388154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44800/0 (socket says 172.21.15.38:44800) 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.684+0000 7fd38659c640 1 -- 172.21.15.38:0/1503930665 learned_addr learned my addr 172.21.15.38:0/1503930665 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd37ffff640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3880cd3b0 0x7fd388154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd386d9d640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3880d00e0 0x7fd3881537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd38659c640 1 -- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3880d00e0 msgr2=0x7fd3881537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd38659c640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3880d00e0 0x7fd3881537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd38659c640 1 -- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3880cd3b0 msgr2=0x7fd388154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd38659c640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3880cd3b0 0x7fd388154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd38659c640 1 -- 172.21.15.38:0/1503930665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38815a690 con 0x7fd3880b1d60 2024-04-25T08:32:35.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd386d9d640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3880d00e0 0x7fd3881537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:35.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd38659c640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3880b1d60 0x7fd388154450 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd37800c430 tx=0x7fd37800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd37effd640 1 -- 172.21.15.38:0/1503930665 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3780074e0 con 0x7fd3880b1d60 2024-04-25T08:32:35.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd37effd640 1 -- 172.21.15.38:0/1503930665 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3780047e0 con 0x7fd3880b1d60 2024-04-25T08:32:35.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd37effd640 1 -- 172.21.15.38:0/1503930665 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd378004bd0 con 0x7fd3880b1d60 2024-04-25T08:32:35.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd38cf22640 1 -- 172.21.15.38:0/1503930665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38815a970 con 0x7fd3880b1d60 2024-04-25T08:32:35.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.685+0000 7fd38cf22640 1 -- 172.21.15.38:0/1503930665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd388155590 con 0x7fd3880b1d60 2024-04-25T08:32:35.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.686+0000 7fd38cf22640 1 -- 172.21.15.38:0/1503930665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd388068e80 con 0x7fd3880b1d60 2024-04-25T08:32:35.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.687+0000 7fd37effd640 1 -- 172.21.15.38:0/1503930665 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd378011070 con 0x7fd3880b1d60 2024-04-25T08:32:35.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.687+0000 7fd37effd640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd36806b4d0 0x7fd36806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.687+0000 7fd37effd640 1 -- 172.21.15.38:0/1503930665 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fd37808a240 con 0x7fd3880b1d60 2024-04-25T08:32:35.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.687+0000 7fd37ffff640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd36806b4d0 0x7fd36806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.688+0000 7fd37ffff640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd36806b4d0 0x7fd36806d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd388154990 tx=0x7fd370008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.691+0000 7fd37effd640 1 -- 172.21.15.38:0/1503930665 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd378058cf0 con 0x7fd3880b1d60 2024-04-25T08:32:35.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.871+0000 7fd38cf22640 1 -- 172.21.15.38:0/1503930665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd388160330 con 0x7fd3880b1d60 2024-04-25T08:32:35.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.872+0000 7fd37effd640 1 -- 172.21.15.38:0/1503930665 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd3780585f0 con 0x7fd3880b1d60 2024-04-25T08:32:35.868 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:35.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.874+0000 7fd38cf22640 1 -- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd36806b4d0 msgr2=0x7fd36806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.874+0000 7fd38cf22640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd36806b4d0 0x7fd36806d990 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd388154990 tx=0x7fd370008040 comp rx=0 tx=0).stop 2024-04-25T08:32:35.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.874+0000 7fd38cf22640 1 -- 172.21.15.38:0/1503930665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3880b1d60 msgr2=0x7fd388154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.874+0000 7fd38cf22640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3880b1d60 0x7fd388154450 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd37800c430 tx=0x7fd37800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.874+0000 7fd38cf22640 1 -- 172.21.15.38:0/1503930665 shutdown_connections 2024-04-25T08:32:35.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.874+0000 7fd38cf22640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd36806b4d0 0x7fd36806d990 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.874+0000 7fd38cf22640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3880d00e0 0x7fd3881537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.874+0000 7fd38cf22640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3880cd3b0 0x7fd388154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.874+0000 7fd38cf22640 1 --2- 172.21.15.38:0/1503930665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3880b1d60 0x7fd388154450 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.874+0000 7fd38cf22640 1 -- 172.21.15.38:0/1503930665 >> 172.21.15.38:0/1503930665 conn(0x7fd3880d06b0 msgr2=0x7fd388160dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.874+0000 7fd38cf22640 1 -- 172.21.15.38:0/1503930665 shutdown_connections 2024-04-25T08:32:35.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.874+0000 7fd38cf22640 1 -- 172.21.15.38:0/1503930665 wait complete. 2024-04-25T08:32:35.881 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:32:35.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.988+0000 7f30e7ab3640 1 Processor -- start 2024-04-25T08:32:35.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.989+0000 7f30e7ab3640 1 -- start start 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.989+0000 7f30e7ab3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f30e00d33d0 0x7f30e00d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.989+0000 7f30e7ab3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30e00d1e90 0x7f30e00b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.989+0000 7f30e7ab3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e00d2330 con 0x7f30e00d0320 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.989+0000 7f30e7ab3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e00d2610 con 0x7f30e00d33d0 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.989+0000 7f30e7ab3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00d28f0 con 0x7f30e00d1e90 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.990+0000 7f30e584f640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f30e00d0320 0x7f30e00d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56524/0 (socket says 172.21.15.38:56524) 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.990+0000 7f30e584f640 1 -- 172.21.15.38:0/647217488 learned_addr learned my addr 172.21.15.38:0/647217488 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.990+0000 7f30e504e640 1 --2- 172.21.15.38:0/647217488 >> v2:172.21.15.38:3300/0 conn(0x7f30e00d33d0 0x7f30e00d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.990+0000 7f30e6050640 1 --2- 172.21.15.38:0/647217488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30e00d1e90 0x7f30e00b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.990+0000 7f30e484d640 1 -- 172.21.15.38:0/647217488 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846892584 0 0) 0x7f30e00d2330 con 0x7f30e00d0320 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.990+0000 7f30e484d640 1 -- 172.21.15.38:0/647217488 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30cc003520 con 0x7f30e00d0320 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.990+0000 7f30e504e640 1 -- 172.21.15.38:0/647217488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30e00d1e90 msgr2=0x7f30e00b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:35.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.990+0000 7f30e504e640 1 --2- 172.21.15.38:0/647217488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30e00d1e90 0x7f30e00b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.990+0000 7f30e504e640 1 -- 172.21.15.38:0/647217488 >> v1:172.21.15.38:6789/0 conn(0x7f30e00d0320 legacy=0x7f30e00d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.990+0000 7f30e504e640 1 -- 172.21.15.38:0/647217488 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00d2bd0 con 0x7f30e00d33d0 2024-04-25T08:32:35.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.990+0000 7f30e6050640 1 --2- 172.21.15.38:0/647217488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30e00d1e90 0x7f30e00b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:35.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.991+0000 7f30e504e640 1 --2- 172.21.15.38:0/647217488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30e00d33d0 0x7f30e00d1950 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f30d00097a0 tx=0x7f30d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe57673dbf1c68f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.991+0000 7f30e484d640 1 -- 172.21.15.38:0/647217488 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0037070 con 0x7f30e00d33d0 2024-04-25T08:32:35.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.991+0000 7f30e484d640 1 -- 172.21.15.38:0/647217488 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d0032440 con 0x7f30e00d33d0 2024-04-25T08:32:35.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.991+0000 7f30e484d640 1 -- 172.21.15.38:0/647217488 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0032810 con 0x7f30e00d33d0 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.991+0000 7f30e7ab3640 1 -- 172.21.15.38:0/647217488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30e00d33d0 msgr2=0x7f30e00d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.991+0000 7f30e7ab3640 1 --2- 172.21.15.38:0/647217488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30e00d33d0 0x7f30e00d1950 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f30d00097a0 tx=0x7f30d002a210 comp rx=0 tx=0).stop 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.991+0000 7f30e7ab3640 1 -- 172.21.15.38:0/647217488 shutdown_connections 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.991+0000 7f30e7ab3640 1 --2- 172.21.15.38:0/647217488 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30e00d1e90 0x7f30e00b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.991+0000 7f30e7ab3640 1 --2- 172.21.15.38:0/647217488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30e00d33d0 0x7f30e00d1950 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.991+0000 7f30e7ab3640 1 -- 172.21.15.38:0/647217488 >> 172.21.15.38:0/647217488 conn(0x7f30e00668f0 msgr2=0x7f30e0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.991+0000 7f30e7ab3640 1 -- 172.21.15.38:0/647217488 shutdown_connections 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.991+0000 7f30e7ab3640 1 -- 172.21.15.38:0/647217488 wait complete. 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.993+0000 7f30e7ab3640 1 Processor -- start 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.993+0000 7f30e7ab3640 1 -- start start 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.993+0000 7f30e7ab3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30e00d0320 0x7f30e015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.993+0000 7f30e7ab3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30e00d1e90 0x7f30e00cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.993+0000 7f30e7ab3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e00d33d0 0x7f30e00cc520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.993+0000 7f30e7ab3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e0161390 con 0x7f30e00d1e90 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.993+0000 7f30e7ab3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e0161210 con 0x7f30e00d33d0 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.993+0000 7f30e7ab3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e00b9720 con 0x7f30e00d0320 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.993+0000 7f30e504e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30e00d1e90 0x7f30e00cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.993+0000 7f30e504e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30e00d1e90 0x7f30e00cbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44816/0 (socket says 172.21.15.38:44816) 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.993+0000 7f30e504e640 1 -- 172.21.15.38:0/1977274546 learned_addr learned my addr 172.21.15.38:0/1977274546 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30e584f640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30e00d0320 0x7f30e015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30e6050640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e00d33d0 0x7f30e00cc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30e504e640 1 -- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30e00d0320 msgr2=0x7f30e015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30e504e640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30e00d0320 0x7f30e015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30e504e640 1 -- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e00d33d0 msgr2=0x7f30e00cc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30e504e640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e00d33d0 0x7f30e00cc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30e504e640 1 -- 172.21.15.38:0/1977274546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e00b3310 con 0x7f30e00d1e90 2024-04-25T08:32:35.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30e6050640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e00d33d0 0x7f30e00cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:35.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30e504e640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30e00d1e90 0x7f30e00cbfe0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f30d0000c00 tx=0x7f30d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30d67fc640 1 -- 172.21.15.38:0/1977274546 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0037040 con 0x7f30e00d1e90 2024-04-25T08:32:35.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30d67fc640 1 -- 172.21.15.38:0/1977274546 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d0042070 con 0x7f30e00d1e90 2024-04-25T08:32:35.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30e7ab3640 1 -- 172.21.15.38:0/1977274546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e00b35f0 con 0x7f30e00d1e90 2024-04-25T08:32:35.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30d67fc640 1 -- 172.21.15.38:0/1977274546 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0030920 con 0x7f30e00d1e90 2024-04-25T08:32:35.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.994+0000 7f30e7ab3640 1 -- 172.21.15.38:0/1977274546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e00b3bb0 con 0x7f30e00d1e90 2024-04-25T08:32:35.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.995+0000 7f30e7ab3640 1 -- 172.21.15.38:0/1977274546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e0068e80 con 0x7f30e00d1e90 2024-04-25T08:32:35.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.995+0000 7f30d67fc640 1 -- 172.21.15.38:0/1977274546 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30d0030ac0 con 0x7f30e00d1e90 2024-04-25T08:32:35.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.996+0000 7f30d67fc640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f30c406b4d0 0x7f30c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:35.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.996+0000 7f30d67fc640 1 -- 172.21.15.38:0/1977274546 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f30d00aaf50 con 0x7f30e00d1e90 2024-04-25T08:32:35.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.996+0000 7f30e584f640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f30c406b4d0 0x7f30c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:35.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:35.997+0000 7f30e584f640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f30c406b4d0 0x7f30c406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f30e00ccd00 tx=0x7f30dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:35.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.000+0000 7f30d67fc640 1 -- 172.21.15.38:0/1977274546 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30d0079320 con 0x7f30e00d1e90 2024-04-25T08:32:36.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.183+0000 7f30e7ab3640 1 -- 172.21.15.38:0/1977274546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f30e00d1980 con 0x7f30e00d1e90 2024-04-25T08:32:36.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.184+0000 7f30d67fc640 1 -- 172.21.15.38:0/1977274546 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f30d007d3c0 con 0x7f30e00d1e90 2024-04-25T08:32:36.180 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.185+0000 7f30e7ab3640 1 -- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f30c406b4d0 msgr2=0x7f30c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.185+0000 7f30e7ab3640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f30c406b4d0 0x7f30c406d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f30e00ccd00 tx=0x7f30dc009040 comp rx=0 tx=0).stop 2024-04-25T08:32:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.185+0000 7f30e7ab3640 1 -- 172.21.15.38:0/1977274546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30e00d1e90 msgr2=0x7f30e00cbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.185+0000 7f30e7ab3640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30e00d1e90 0x7f30e00cbfe0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f30d0000c00 tx=0x7f30d00046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.185+0000 7f30e7ab3640 1 -- 172.21.15.38:0/1977274546 shutdown_connections 2024-04-25T08:32:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.185+0000 7f30e7ab3640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30e00d33d0 0x7f30e00cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.185+0000 7f30e7ab3640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30e00d1e90 0x7f30e00cbfe0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.185+0000 7f30e7ab3640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f30c406b4d0 0x7f30c406d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.185+0000 7f30e7ab3640 1 --2- 172.21.15.38:0/1977274546 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30e00d0320 0x7f30e015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.185+0000 7f30e7ab3640 1 -- 172.21.15.38:0/1977274546 >> 172.21.15.38:0/1977274546 conn(0x7f30e00668f0 msgr2=0x7f30e00d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.186+0000 7f30e7ab3640 1 -- 172.21.15.38:0/1977274546 shutdown_connections 2024-04-25T08:32:36.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.186+0000 7f30e7ab3640 1 -- 172.21.15.38:0/1977274546 wait complete. 2024-04-25T08:32:36.192 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:32:36.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.299+0000 7fec8e4f7640 1 Processor -- start 2024-04-25T08:32:36.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.299+0000 7fec8e4f7640 1 -- start start 2024-04-25T08:32:36.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.300+0000 7fec8e4f7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec880d00e0 0x7fec880cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.300+0000 7fec8e4f7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fec880cd3b0 0x7fec880cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.300+0000 7fec8e4f7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec880cdd80 con 0x7fec880b1d60 2024-04-25T08:32:36.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.300+0000 7fec8e4f7640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8806abe0 con 0x7fec880cd3b0 2024-04-25T08:32:36.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.300+0000 7fec8e4f7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8806b480 con 0x7fec880d00e0 2024-04-25T08:32:36.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.300+0000 7fec87fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fec880b1d60 0x7fec880cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56538/0 (socket says 172.21.15.38:56538) 2024-04-25T08:32:36.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.300+0000 7fec87fff640 1 -- 172.21.15.38:0/1343549727 learned_addr learned my addr 172.21.15.38:0/1343549727 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:36.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.301+0000 7fec8ca94640 1 --2- 172.21.15.38:0/1343549727 >> v2:172.21.15.38:3300/0 conn(0x7fec880cd3b0 0x7fec880cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.301+0000 7fec877fe640 1 --2- 172.21.15.38:0/1343549727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec880d00e0 0x7fec880cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.301+0000 7fec8ca94640 1 -- 172.21.15.38:0/1343549727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec880d00e0 msgr2=0x7fec880cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.301+0000 7fec8ca94640 1 --2- 172.21.15.38:0/1343549727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec880d00e0 0x7fec880cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.301+0000 7fec8ca94640 1 -- 172.21.15.38:0/1343549727 >> v1:172.21.15.38:6789/0 conn(0x7fec880b1d60 legacy=0x7fec880cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.301+0000 7fec8ca94640 1 -- 172.21.15.38:0/1343549727 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8815ef40 con 0x7fec880cd3b0 2024-04-25T08:32:36.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.301+0000 7fec86ffd640 1 -- 172.21.15.38:0/1343549727 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1306672619 0 0) 0x7fec880cdd80 con 0x7fec880b1d60 2024-04-25T08:32:36.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.301+0000 7fec877fe640 1 --2- 172.21.15.38:0/1343549727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec880d00e0 0x7fec880cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:36.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.302+0000 7fec8ca94640 1 --2- 172.21.15.38:0/1343549727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec880cd3b0 0x7fec880cd840 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fec80009d90 tx=0x7fec8002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5a39fb91c1280b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.302+0000 7fec86ffd640 1 -- 172.21.15.38:0/1343549727 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80007740 con 0x7fec880cd3b0 2024-04-25T08:32:36.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.302+0000 7fec86ffd640 1 -- 172.21.15.38:0/1343549727 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec800078e0 con 0x7fec880cd3b0 2024-04-25T08:32:36.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.302+0000 7fec86ffd640 1 -- 172.21.15.38:0/1343549727 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80007cb0 con 0x7fec880cd3b0 2024-04-25T08:32:36.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.302+0000 7fec8e4f7640 1 -- 172.21.15.38:0/1343549727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec880cd3b0 msgr2=0x7fec880cd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.302+0000 7fec8e4f7640 1 --2- 172.21.15.38:0/1343549727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec880cd3b0 0x7fec880cd840 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fec80009d90 tx=0x7fec8002a660 comp rx=0 tx=0).stop 2024-04-25T08:32:36.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.302+0000 7fec8e4f7640 1 -- 172.21.15.38:0/1343549727 shutdown_connections 2024-04-25T08:32:36.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.302+0000 7fec8e4f7640 1 --2- 172.21.15.38:0/1343549727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec880cd3b0 0x7fec880cd840 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.302+0000 7fec8e4f7640 1 --2- 172.21.15.38:0/1343549727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec880d00e0 0x7fec880cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.302+0000 7fec8e4f7640 1 -- 172.21.15.38:0/1343549727 >> 172.21.15.38:0/1343549727 conn(0x7fec880d06b0 msgr2=0x7fec880d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.303+0000 7fec8e4f7640 1 -- 172.21.15.38:0/1343549727 shutdown_connections 2024-04-25T08:32:36.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.303+0000 7fec8e4f7640 1 -- 172.21.15.38:0/1343549727 wait complete. 2024-04-25T08:32:36.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.304+0000 7fec8e4f7640 1 Processor -- start 2024-04-25T08:32:36.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.304+0000 7fec8e4f7640 1 -- start start 2024-04-25T08:32:36.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8e4f7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec880b1d60 0x7fec880c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8e4f7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec880cd3b0 0x7fec88154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8e4f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec880d00e0 0x7fec88143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8e4f7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8806b910 con 0x7fec880d00e0 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8e4f7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8806b110 con 0x7fec880b1d60 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8e4f7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8806ba90 con 0x7fec880cd3b0 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8ca94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec880d00e0 0x7fec88143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8ca94640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec880d00e0 0x7fec88143870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44834/0 (socket says 172.21.15.38:44834) 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8ca94640 1 -- 172.21.15.38:0/3959258253 learned_addr learned my addr 172.21.15.38:0/3959258253 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec87fff640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec880b1d60 0x7fec880c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec877fe640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec880cd3b0 0x7fec88154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8ca94640 1 -- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec880cd3b0 msgr2=0x7fec88154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8ca94640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec880cd3b0 0x7fec88154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8ca94640 1 -- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec880b1d60 msgr2=0x7fec880c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8ca94640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec880b1d60 0x7fec880c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8ca94640 1 -- 172.21.15.38:0/3959258253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880c5cb0 con 0x7fec880d00e0 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.305+0000 7fec8ca94640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec880d00e0 0x7fec88143870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fec80009eb0 tx=0x7fec80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.306+0000 7fec84ff9640 1 -- 172.21.15.38:0/3959258253 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8003f070 con 0x7fec880d00e0 2024-04-25T08:32:36.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.306+0000 7fec84ff9640 1 -- 172.21.15.38:0/3959258253 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec80031bc0 con 0x7fec880d00e0 2024-04-25T08:32:36.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.306+0000 7fec8e4f7640 1 -- 172.21.15.38:0/3959258253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec880c5f90 con 0x7fec880d00e0 2024-04-25T08:32:36.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.306+0000 7fec84ff9640 1 -- 172.21.15.38:0/3959258253 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80044460 con 0x7fec880d00e0 2024-04-25T08:32:36.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.306+0000 7fec8e4f7640 1 -- 172.21.15.38:0/3959258253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec880c6550 con 0x7fec880d00e0 2024-04-25T08:32:36.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.307+0000 7fec8e4f7640 1 -- 172.21.15.38:0/3959258253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec88068e80 con 0x7fec880d00e0 2024-04-25T08:32:36.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.307+0000 7fec84ff9640 1 -- 172.21.15.38:0/3959258253 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec80007790 con 0x7fec880d00e0 2024-04-25T08:32:36.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.308+0000 7fec84ff9640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec7406b4d0 0x7fec7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.308+0000 7fec84ff9640 1 -- 172.21.15.38:0/3959258253 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fec800abd00 con 0x7fec880d00e0 2024-04-25T08:32:36.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.308+0000 7fec87fff640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec7406b4d0 0x7fec7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.308+0000 7fec87fff640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec7406b4d0 0x7fec7406d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fec78007fd0 tx=0x7fec78007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.312+0000 7fec84ff9640 1 -- 172.21.15.38:0/3959258253 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec8007a160 con 0x7fec880d00e0 2024-04-25T08:32:36.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.493+0000 7fec8e4f7640 1 -- 172.21.15.38:0/3959258253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec8806c9e0 con 0x7fec880d00e0 2024-04-25T08:32:36.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.494+0000 7fec84ff9640 1 -- 172.21.15.38:0/3959258253 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fec8007e200 con 0x7fec880d00e0 2024-04-25T08:32:36.490 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:32:36.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.495+0000 7fec8e4f7640 1 -- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec7406b4d0 msgr2=0x7fec7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.495+0000 7fec8e4f7640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec7406b4d0 0x7fec7406d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fec78007fd0 tx=0x7fec78007de0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.495+0000 7fec8e4f7640 1 -- 172.21.15.38:0/3959258253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec880d00e0 msgr2=0x7fec88143870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.495+0000 7fec8e4f7640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec880d00e0 0x7fec88143870 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fec80009eb0 tx=0x7fec80004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:36.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.495+0000 7fec8e4f7640 1 -- 172.21.15.38:0/3959258253 shutdown_connections 2024-04-25T08:32:36.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.495+0000 7fec8e4f7640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec880d00e0 0x7fec88143870 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.495+0000 7fec8e4f7640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec7406b4d0 0x7fec7406d990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.495+0000 7fec8e4f7640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec880cd3b0 0x7fec88154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.495+0000 7fec8e4f7640 1 --2- 172.21.15.38:0/3959258253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec880b1d60 0x7fec880c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.495+0000 7fec8e4f7640 1 -- 172.21.15.38:0/3959258253 >> 172.21.15.38:0/3959258253 conn(0x7fec880d06b0 msgr2=0x7fec8815da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.496+0000 7fec8e4f7640 1 -- 172.21.15.38:0/3959258253 shutdown_connections 2024-04-25T08:32:36.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.496+0000 7fec8e4f7640 1 -- 172.21.15.38:0/3959258253 wait complete. 2024-04-25T08:32:36.502 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:32:36.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.611+0000 7f43b1a58640 1 Processor -- start 2024-04-25T08:32:36.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.612+0000 7f43b1a58640 1 -- start start 2024-04-25T08:32:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.612+0000 7f43b1a58640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f43ac0d0320 0x7f43ac0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.612+0000 7f43b1a58640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f43ac0d0700 0x7f43ac0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.612+0000 7f43b1a58640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ac0d2330 con 0x7f43ac0d1e90 2024-04-25T08:32:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.612+0000 7f43b1a58640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ac0d2610 con 0x7f43ac0d0700 2024-04-25T08:32:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.612+0000 7f43b1a58640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ac0d28f0 con 0x7f43ac0d0320 2024-04-25T08:32:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.612+0000 7f43aa7fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f43ac0d0700 0x7f43ac0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.612+0000 7f43aa7fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f43ac0d0700 0x7f43ac0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44838/0 (socket says 172.21.15.38:44838) 2024-04-25T08:32:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.612+0000 7f43aa7fc640 1 -- 172.21.15.38:0/1041072018 learned_addr learned my addr 172.21.15.38:0/1041072018 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.613+0000 7f43aaffd640 1 --2- 172.21.15.38:0/1041072018 >> v2:172.21.15.107:3300/0 conn(0x7f43ac0d0320 0x7f43ac0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.613+0000 7f43aa7fc640 1 -- 172.21.15.38:0/1041072018 >> v2:172.21.15.107:3300/0 conn(0x7f43ac0d0320 msgr2=0x7f43ac0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.613+0000 7f43aa7fc640 1 --2- 172.21.15.38:0/1041072018 >> v2:172.21.15.107:3300/0 conn(0x7f43ac0d0320 0x7f43ac0d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.613+0000 7f43aa7fc640 1 -- 172.21.15.38:0/1041072018 >> v1:172.21.15.107:6789/0 conn(0x7f43ac0d1e90 legacy=0x7f43ac0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.613+0000 7f43aa7fc640 1 -- 172.21.15.38:0/1041072018 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac0d2bd0 con 0x7f43ac0d0700 2024-04-25T08:32:36.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.613+0000 7f43aaffd640 1 --2- 172.21.15.38:0/1041072018 >> v2:172.21.15.107:3300/0 conn(0x7f43ac0d0320 0x7f43ac0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:36.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.613+0000 7f43aa7fc640 1 --2- 172.21.15.38:0/1041072018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43ac0d0700 0x7f43ac0d1950 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f43ac069100 tx=0x7f43a002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6262ebf70f8ee02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.614+0000 7f43a9ffb640 1 -- 172.21.15.38:0/1041072018 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0037070 con 0x7f43ac0d0700 2024-04-25T08:32:36.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.614+0000 7f43a9ffb640 1 -- 172.21.15.38:0/1041072018 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a0032440 con 0x7f43ac0d0700 2024-04-25T08:32:36.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.614+0000 7f43a9ffb640 1 -- 172.21.15.38:0/1041072018 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0032810 con 0x7f43ac0d0700 2024-04-25T08:32:36.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.614+0000 7f43b1a58640 1 -- 172.21.15.38:0/1041072018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43ac0d0700 msgr2=0x7f43ac0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.614+0000 7f43b1a58640 1 --2- 172.21.15.38:0/1041072018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43ac0d0700 0x7f43ac0d1950 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f43ac069100 tx=0x7f43a002a2c0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.614+0000 7f43b1a58640 1 -- 172.21.15.38:0/1041072018 shutdown_connections 2024-04-25T08:32:36.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.614+0000 7f43b1a58640 1 --2- 172.21.15.38:0/1041072018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43ac0d0700 0x7f43ac0d1950 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.614+0000 7f43b1a58640 1 --2- 172.21.15.38:0/1041072018 >> v2:172.21.15.107:3300/0 conn(0x7f43ac0d0320 0x7f43ac0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.614+0000 7f43b1a58640 1 -- 172.21.15.38:0/1041072018 >> 172.21.15.38:0/1041072018 conn(0x7f43ac0668f0 msgr2=0x7f43ac066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.614+0000 7f43b1a58640 1 -- 172.21.15.38:0/1041072018 shutdown_connections 2024-04-25T08:32:36.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.614+0000 7f43b1a58640 1 -- 172.21.15.38:0/1041072018 wait complete. 2024-04-25T08:32:36.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.615+0000 7f43b1a58640 1 Processor -- start 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.616+0000 7f43b1a58640 1 -- start start 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.616+0000 7f43b1a58640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43ac0d0320 0x7f43ac1e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.616+0000 7f43b1a58640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43ac0d0700 0x7f43ac0ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.616+0000 7f43b1a58640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43ac0d1e90 0x7f43ac151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.616+0000 7f43b1a58640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac0d0b40 con 0x7f43ac0d0700 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.616+0000 7f43b1a58640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac161210 con 0x7f43ac0d1e90 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.616+0000 7f43b1a58640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac0ab9c0 con 0x7f43ac0d0320 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.616+0000 7f43aa7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43ac0d0700 0x7f43ac0ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.616+0000 7f43aa7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43ac0d0700 0x7f43ac0ab480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44852/0 (socket says 172.21.15.38:44852) 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.616+0000 7f43aa7fc640 1 -- 172.21.15.38:0/1693667822 learned_addr learned my addr 172.21.15.38:0/1693667822 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f43aaffd640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43ac0d0320 0x7f43ac1e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f43ab7fe640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43ac0d1e90 0x7f43ac151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f43aa7fc640 1 -- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43ac0d0320 msgr2=0x7f43ac1e6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f43aa7fc640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43ac0d0320 0x7f43ac1e6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f43aa7fc640 1 -- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43ac0d1e90 msgr2=0x7f43ac151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f43aa7fc640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43ac0d1e90 0x7f43ac151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f43aa7fc640 1 -- 172.21.15.38:0/1693667822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac1e68f0 con 0x7f43ac0d0700 2024-04-25T08:32:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f43aa7fc640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43ac0d0700 0x7f43ac0ab480 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f43a0000c00 tx=0x7f43a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f4393fff640 1 -- 172.21.15.38:0/1693667822 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0041020 con 0x7f43ac0d0700 2024-04-25T08:32:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f4393fff640 1 -- 172.21.15.38:0/1693667822 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a0030970 con 0x7f43ac0d0700 2024-04-25T08:32:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f43b1a58640 1 -- 172.21.15.38:0/1693667822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ac1e6bd0 con 0x7f43ac0d0700 2024-04-25T08:32:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f4393fff640 1 -- 172.21.15.38:0/1693667822 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0037040 con 0x7f43ac0d0700 2024-04-25T08:32:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.617+0000 7f43b1a58640 1 -- 172.21.15.38:0/1693667822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ac1e7190 con 0x7f43ac0d0700 2024-04-25T08:32:36.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.618+0000 7f43b1a58640 1 -- 172.21.15.38:0/1693667822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ac068e80 con 0x7f43ac0d0700 2024-04-25T08:32:36.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.619+0000 7f4393fff640 1 -- 172.21.15.38:0/1693667822 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43a0044020 con 0x7f43ac0d0700 2024-04-25T08:32:36.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.619+0000 7f4393fff640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f439806b4d0 0x7f439806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.619+0000 7f4393fff640 1 -- 172.21.15.38:0/1693667822 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f43a00ab060 con 0x7f43ac0d0700 2024-04-25T08:32:36.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.619+0000 7f43aaffd640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f439806b4d0 0x7f439806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.620+0000 7f43aaffd640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f439806b4d0 0x7f439806d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f439c00a990 tx=0x7f439c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.623+0000 7f4393fff640 1 -- 172.21.15.38:0/1693667822 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43a004b030 con 0x7f43ac0d0700 2024-04-25T08:32:36.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.805+0000 7f43b1a58640 1 -- 172.21.15.38:0/1693667822 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f43ac0b86c0 con 0x7f43ac0d0700 2024-04-25T08:32:36.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.806+0000 7f4393fff640 1 -- 172.21.15.38:0/1693667822 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f43a0079a80 con 0x7f43ac0d0700 2024-04-25T08:32:36.802 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:32:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.808+0000 7f43b1a58640 1 -- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f439806b4d0 msgr2=0x7f439806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.808+0000 7f43b1a58640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f439806b4d0 0x7f439806d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f439c00a990 tx=0x7f439c008040 comp rx=0 tx=0).stop 2024-04-25T08:32:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.808+0000 7f43b1a58640 1 -- 172.21.15.38:0/1693667822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43ac0d0700 msgr2=0x7f43ac0ab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.808+0000 7f43b1a58640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43ac0d0700 0x7f43ac0ab480 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f43a0000c00 tx=0x7f43a00046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.808+0000 7f43b1a58640 1 -- 172.21.15.38:0/1693667822 shutdown_connections 2024-04-25T08:32:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.808+0000 7f43b1a58640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43ac0d1e90 0x7f43ac151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.808+0000 7f43b1a58640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43ac0d0700 0x7f43ac0ab480 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.808+0000 7f43b1a58640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f439806b4d0 0x7f439806d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.808+0000 7f43b1a58640 1 --2- 172.21.15.38:0/1693667822 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43ac0d0320 0x7f43ac1e6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.808+0000 7f43b1a58640 1 -- 172.21.15.38:0/1693667822 >> 172.21.15.38:0/1693667822 conn(0x7f43ac0668f0 msgr2=0x7f43ac15e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.808+0000 7f43b1a58640 1 -- 172.21.15.38:0/1693667822 shutdown_connections 2024-04-25T08:32:36.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.808+0000 7f43b1a58640 1 -- 172.21.15.38:0/1693667822 wait complete. 2024-04-25T08:32:36.815 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:36.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.920+0000 7fc973dc8640 1 Processor -- start 2024-04-25T08:32:36.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.920+0000 7fc973dc8640 1 -- start start 2024-04-25T08:32:36.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.921+0000 7fc973dc8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc96c0b5710 0x7fc96c161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.921+0000 7fc973dc8640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc96c15ddf0 0x7fc96c0b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.921+0000 7fc973dc8640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc96c161e00 con 0x7fc96c0b5af0 2024-04-25T08:32:36.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.921+0000 7fc973dc8640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c0b9930 con 0x7fc96c0b5710 2024-04-25T08:32:36.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.921+0000 7fc973dc8640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c1620e0 con 0x7fc96c15ddf0 2024-04-25T08:32:36.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.921+0000 7fc971b64640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc96c0b5710 0x7fc96c161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.921+0000 7fc971b64640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc96c0b5710 0x7fc96c161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44854/0 (socket says 172.21.15.38:44854) 2024-04-25T08:32:36.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.921+0000 7fc971b64640 1 -- 172.21.15.38:0/2261029771 learned_addr learned my addr 172.21.15.38:0/2261029771 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:36.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.921+0000 7fc971b64640 1 -- 172.21.15.38:0/2261029771 >> v2:172.21.15.107:3300/0 conn(0x7fc96c15ddf0 msgr2=0x7fc96c0b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.921+0000 7fc971b64640 1 --2- 172.21.15.38:0/2261029771 >> v2:172.21.15.107:3300/0 conn(0x7fc96c15ddf0 0x7fc96c0b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.921+0000 7fc971b64640 1 -- 172.21.15.38:0/2261029771 >> v1:172.21.15.107:6789/0 conn(0x7fc96c0b5af0 legacy=0x7fc96c06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:36.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.921+0000 7fc971b64640 1 -- 172.21.15.38:0/2261029771 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c1623c0 con 0x7fc96c0b5710 2024-04-25T08:32:36.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.922+0000 7fc971b64640 1 --2- 172.21.15.38:0/2261029771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc96c0b5710 0x7fc96c161700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0d0300 tx=0x7fc95c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc253e3b119f8ff2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.922+0000 7fc970b62640 1 -- 172.21.15.38:0/2261029771 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c036070 con 0x7fc96c0b5710 2024-04-25T08:32:36.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.922+0000 7fc970b62640 1 -- 172.21.15.38:0/2261029771 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95c031600 con 0x7fc96c0b5710 2024-04-25T08:32:36.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.922+0000 7fc970b62640 1 -- 172.21.15.38:0/2261029771 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95c0319d0 con 0x7fc96c0b5710 2024-04-25T08:32:36.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.923+0000 7fc973dc8640 1 -- 172.21.15.38:0/2261029771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc96c0b5710 msgr2=0x7fc96c161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.923+0000 7fc973dc8640 1 --2- 172.21.15.38:0/2261029771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc96c0b5710 0x7fc96c161700 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0d0300 tx=0x7fc95c02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.923+0000 7fc973dc8640 1 -- 172.21.15.38:0/2261029771 shutdown_connections 2024-04-25T08:32:36.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.923+0000 7fc973dc8640 1 --2- 172.21.15.38:0/2261029771 >> v2:172.21.15.107:3300/0 conn(0x7fc96c15ddf0 0x7fc96c0b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.923+0000 7fc973dc8640 1 --2- 172.21.15.38:0/2261029771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc96c0b5710 0x7fc96c161700 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.923+0000 7fc973dc8640 1 -- 172.21.15.38:0/2261029771 >> 172.21.15.38:0/2261029771 conn(0x7fc96c0d07c0 msgr2=0x7fc96c0d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:36.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.923+0000 7fc973dc8640 1 -- 172.21.15.38:0/2261029771 shutdown_connections 2024-04-25T08:32:36.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.923+0000 7fc973dc8640 1 -- 172.21.15.38:0/2261029771 wait complete. 2024-04-25T08:32:36.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.924+0000 7fc973dc8640 1 Processor -- start 2024-04-25T08:32:36.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.924+0000 7fc973dc8640 1 -- start start 2024-04-25T08:32:36.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc973dc8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc96c0b5710 0x7fc96c0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc973dc8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc96c0b5af0 0x7fc96c0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc973dc8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc96c15ddf0 0x7fc96c0b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc973dc8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c0d34d0 con 0x7fc96c0b5af0 2024-04-25T08:32:36.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc973dc8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c0d3350 con 0x7fc96c15ddf0 2024-04-25T08:32:36.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc973dc8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c0d3650 con 0x7fc96c0b5710 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc971363640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc96c0b5af0 0x7fc96c0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc971363640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc96c0b5af0 0x7fc96c0b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44868/0 (socket says 172.21.15.38:44868) 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc971363640 1 -- 172.21.15.38:0/638572843 learned_addr learned my addr 172.21.15.38:0/638572843 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc971b64640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc96c0b5710 0x7fc96c0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc972365640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc96c15ddf0 0x7fc96c0b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc971363640 1 -- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc96c0b5710 msgr2=0x7fc96c0cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc971363640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc96c0b5710 0x7fc96c0cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc971363640 1 -- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc96c15ddf0 msgr2=0x7fc96c0b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc971363640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc96c15ddf0 0x7fc96c0b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc971363640 1 -- 172.21.15.38:0/638572843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c0cccd0 con 0x7fc96c0b5af0 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.925+0000 7fc971363640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc96c0b5af0 0x7fc96c0b4450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc96000b410 tx=0x7fc96000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.926+0000 7fc96a7fc640 1 -- 172.21.15.38:0/638572843 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960015020 con 0x7fc96c0b5af0 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.926+0000 7fc96a7fc640 1 -- 172.21.15.38:0/638572843 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc960011850 con 0x7fc96c0b5af0 2024-04-25T08:32:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.926+0000 7fc973dc8640 1 -- 172.21.15.38:0/638572843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96c0ccfb0 con 0x7fc96c0b5af0 2024-04-25T08:32:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.926+0000 7fc96a7fc640 1 -- 172.21.15.38:0/638572843 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960011c40 con 0x7fc96c0b5af0 2024-04-25T08:32:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.926+0000 7fc973dc8640 1 -- 172.21.15.38:0/638572843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c0cd570 con 0x7fc96c0b5af0 2024-04-25T08:32:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.927+0000 7fc973dc8640 1 -- 172.21.15.38:0/638572843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96c068e80 con 0x7fc96c0b5af0 2024-04-25T08:32:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.927+0000 7fc96a7fc640 1 -- 172.21.15.38:0/638572843 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc960018020 con 0x7fc96c0b5af0 2024-04-25T08:32:36.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.928+0000 7fc96a7fc640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc95806b4d0 0x7fc95806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:36.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.928+0000 7fc96a7fc640 1 -- 172.21.15.38:0/638572843 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fc96008a460 con 0x7fc96c0b5af0 2024-04-25T08:32:36.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.928+0000 7fc971b64640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc95806b4d0 0x7fc95806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:36.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.928+0000 7fc971b64640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc95806b4d0 0x7fc95806d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc95c033000 tx=0x7fc95c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:36.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:36.932+0000 7fc96a7fc640 1 -- 172.21.15.38:0/638572843 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc96001f020 con 0x7fc96c0b5af0 2024-04-25T08:32:37.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.148+0000 7fc973dc8640 1 -- 172.21.15.38:0/638572843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc96c0b2c60 con 0x7fc96c0b5af0 2024-04-25T08:32:37.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.149+0000 7fc96a7fc640 1 -- 172.21.15.38:0/638572843 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7fc960058820 con 0x7fc96c0b5af0 2024-04-25T08:32:37.145 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:37.145 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4242612806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4242612806},{"type":"v1","addr":"172.21.15.38:6835","nonce":4242612806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2154167637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":2154167637},{"type":"v1","addr":"172.21.15.107:6837","nonce":2154167637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/678452936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":678452936},{"type":"v1","addr":"172.21.15.38:6834","nonce":678452936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:31:35.665008+0000","modified":"2024-04-25T08:31:51.799830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6836/1104921666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1104921666},{"type":"v1","addr":"172.21.15.107:6836","nonce":1104921666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:32:37.145 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-04-25T08:32:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.151+0000 7fc973dc8640 1 -- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc95806b4d0 msgr2=0x7fc95806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.151+0000 7fc973dc8640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc95806b4d0 0x7fc95806d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fc95c033000 tx=0x7fc95c009000 comp rx=0 tx=0).stop 2024-04-25T08:32:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.151+0000 7fc973dc8640 1 -- 172.21.15.38:0/638572843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc96c0b5af0 msgr2=0x7fc96c0b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.151+0000 7fc973dc8640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc96c0b5af0 0x7fc96c0b4450 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc96000b410 tx=0x7fc96000b8d0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.151+0000 7fc973dc8640 1 -- 172.21.15.38:0/638572843 shutdown_connections 2024-04-25T08:32:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.151+0000 7fc973dc8640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc96c15ddf0 0x7fc96c0b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.151+0000 7fc973dc8640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc96c0b5af0 0x7fc96c0b4450 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.151+0000 7fc973dc8640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc95806b4d0 0x7fc95806d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.151+0000 7fc973dc8640 1 --2- 172.21.15.38:0/638572843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc96c0b5710 0x7fc96c0cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.151+0000 7fc973dc8640 1 -- 172.21.15.38:0/638572843 >> 172.21.15.38:0/638572843 conn(0x7fc96c0d07c0 msgr2=0x7fc96c0d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.151+0000 7fc973dc8640 1 -- 172.21.15.38:0/638572843 shutdown_connections 2024-04-25T08:32:37.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.151+0000 7fc973dc8640 1 -- 172.21.15.38:0/638572843 wait complete. 2024-04-25T08:32:37.158 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:37.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.262+0000 7f42e2377640 1 Processor -- start 2024-04-25T08:32:37.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.262+0000 7f42e2377640 1 -- start start 2024-04-25T08:32:37.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42e2377640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42dc15dc10 0x7f42dc0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42e2377640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f42dc0ced60 0x7f42dc0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42e2377640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42dc0d3e70 con 0x7f42dc0d3900 2024-04-25T08:32:37.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42e2377640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42dc15dff0 con 0x7f42dc0ced60 2024-04-25T08:32:37.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42e2377640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0b6990 con 0x7f42dc15dc10 2024-04-25T08:32:37.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42e0914640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f42dc0d3900 0x7f42dc0b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56550/0 (socket says 172.21.15.38:56550) 2024-04-25T08:32:37.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42e0914640 1 -- 172.21.15.38:0/1507393252 learned_addr learned my addr 172.21.15.38:0/1507393252 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:37.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42db7fe640 1 --2- 172.21.15.38:0/1507393252 >> v2:172.21.15.107:3300/0 conn(0x7f42dc0ced60 0x7f42dc0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42dbfff640 1 --2- 172.21.15.38:0/1507393252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42dc15dc10 0x7f42dc0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42daffd640 1 -- 172.21.15.38:0/1507393252 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3904129088 0 0) 0x7f42dc0d3e70 con 0x7f42dc0d3900 2024-04-25T08:32:37.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42daffd640 1 -- 172.21.15.38:0/1507393252 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42c8003520 con 0x7f42dc0d3900 2024-04-25T08:32:37.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42db7fe640 1 -- 172.21.15.38:0/1507393252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42dc15dc10 msgr2=0x7f42dc0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42db7fe640 1 --2- 172.21.15.38:0/1507393252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42dc15dc10 0x7f42dc0ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42daffd640 1 -- 172.21.15.38:0/1507393252 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 430852460 0 0) 0x7f42c8003520 con 0x7f42dc0d3900 2024-04-25T08:32:37.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.263+0000 7f42db7fe640 1 -- 172.21.15.38:0/1507393252 >> v1:172.21.15.38:6789/0 conn(0x7f42dc0d3900 legacy=0x7f42dc0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.264+0000 7f42db7fe640 1 -- 172.21.15.38:0/1507393252 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc0d4150 con 0x7f42dc0ced60 2024-04-25T08:32:37.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.264+0000 7f42db7fe640 1 --2- 172.21.15.38:0/1507393252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42dc0ced60 0x7f42dc0d0850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0097a0 tx=0x7f42cc02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d83e7ed67cfe2ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.264+0000 7f42daffd640 1 -- 172.21.15.38:0/1507393252 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc037070 con 0x7f42dc0ced60 2024-04-25T08:32:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.265+0000 7f42daffd640 1 -- 172.21.15.38:0/1507393252 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42cc02ab10 con 0x7f42dc0ced60 2024-04-25T08:32:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.265+0000 7f42daffd640 1 -- 172.21.15.38:0/1507393252 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d40019f0 con 0x7f42dc0ced60 2024-04-25T08:32:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.265+0000 7f42e2377640 1 -- 172.21.15.38:0/1507393252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42dc0ced60 msgr2=0x7f42dc0d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.265+0000 7f42e2377640 1 --2- 172.21.15.38:0/1507393252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42dc0ced60 0x7f42dc0d0850 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0097a0 tx=0x7f42cc02a080 comp rx=0 tx=0).stop 2024-04-25T08:32:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.265+0000 7f42e2377640 1 -- 172.21.15.38:0/1507393252 shutdown_connections 2024-04-25T08:32:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.265+0000 7f42e2377640 1 --2- 172.21.15.38:0/1507393252 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42dc0ced60 0x7f42dc0d0850 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.265+0000 7f42e2377640 1 --2- 172.21.15.38:0/1507393252 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42dc15dc10 0x7f42dc0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.265+0000 7f42e2377640 1 -- 172.21.15.38:0/1507393252 >> 172.21.15.38:0/1507393252 conn(0x7f42dc066d10 msgr2=0x7f42dc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.265+0000 7f42e2377640 1 -- 172.21.15.38:0/1507393252 shutdown_connections 2024-04-25T08:32:37.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.265+0000 7f42e2377640 1 -- 172.21.15.38:0/1507393252 wait complete. 2024-04-25T08:32:37.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.266+0000 7f42e2377640 1 Processor -- start 2024-04-25T08:32:37.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.266+0000 7f42e2377640 1 -- start start 2024-04-25T08:32:37.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.267+0000 7f42e2377640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42dc0ced60 0x7f42dc1584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.267+0000 7f42e2377640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42dc0d3900 0x7f42dc0cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.267+0000 7f42e2377640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42dc15dc10 0x7f42dc0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.267+0000 7f42e2377640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0b6bc0 con 0x7f42dc15dc10 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.267+0000 7f42e2377640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0d4710 con 0x7f42dc0ced60 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.267+0000 7f42e2377640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0d0d90 con 0x7f42dc0d3900 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.267+0000 7f42e0914640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42dc15dc10 0x7f42dc0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.267+0000 7f42e0914640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42dc15dc10 0x7f42dc0ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44876/0 (socket says 172.21.15.38:44876) 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.267+0000 7f42e0914640 1 -- 172.21.15.38:0/2210211062 learned_addr learned my addr 172.21.15.38:0/2210211062 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.267+0000 7f42dbfff640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42dc0ced60 0x7f42dc1584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.267+0000 7f42db7fe640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42dc0d3900 0x7f42dc0cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.268+0000 7f42e0914640 1 -- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42dc0d3900 msgr2=0x7f42dc0cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.268+0000 7f42e0914640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42dc0d3900 0x7f42dc0cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.268+0000 7f42e0914640 1 -- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42dc0ced60 msgr2=0x7f42dc1584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.268+0000 7f42e0914640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42dc0ced60 0x7f42dc1584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.268+0000 7f42e0914640 1 -- 172.21.15.38:0/2210211062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc155030 con 0x7f42dc15dc10 2024-04-25T08:32:37.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.268+0000 7f42e0914640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42dc15dc10 0x7f42dc0ccf30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f42d4001b60 tx=0x7f42d400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.268+0000 7f42d8ff9640 1 -- 172.21.15.38:0/2210211062 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4017070 con 0x7f42dc15dc10 2024-04-25T08:32:37.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.268+0000 7f42d8ff9640 1 -- 172.21.15.38:0/2210211062 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d4012400 con 0x7f42dc15dc10 2024-04-25T08:32:37.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.268+0000 7f42d8ff9640 1 -- 172.21.15.38:0/2210211062 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d40127b0 con 0x7f42dc15dc10 2024-04-25T08:32:37.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.268+0000 7f42e2377640 1 -- 172.21.15.38:0/2210211062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc155310 con 0x7f42dc15dc10 2024-04-25T08:32:37.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.268+0000 7f42e2377640 1 -- 172.21.15.38:0/2210211062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc1569f0 con 0x7f42dc15dc10 2024-04-25T08:32:37.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.269+0000 7f42e2377640 1 -- 172.21.15.38:0/2210211062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc0d1380 con 0x7f42dc15dc10 2024-04-25T08:32:37.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.270+0000 7f42d8ff9640 1 -- 172.21.15.38:0/2210211062 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42d401a070 con 0x7f42dc15dc10 2024-04-25T08:32:37.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.270+0000 7f42d8ff9640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42bc06b4d0 0x7f42bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.270+0000 7f42d8ff9640 1 -- 172.21.15.38:0/2210211062 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f42d408a4d0 con 0x7f42dc15dc10 2024-04-25T08:32:37.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.270+0000 7f42dbfff640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42bc06b4d0 0x7f42bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.271+0000 7f42dbfff640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42bc06b4d0 0x7f42bc06d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009aa0 tx=0x7f42c4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.274+0000 7f42d8ff9640 1 -- 172.21.15.38:0/2210211062 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42dc0d1380 con 0x7f42dc15dc10 2024-04-25T08:32:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.458+0000 7f42e2377640 1 -- 172.21.15.38:0/2210211062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42dc068e80 con 0x7f42dc15dc10 2024-04-25T08:32:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.459+0000 7f42d8ff9640 1 -- 172.21.15.38:0/2210211062 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18969 (secure 0 0 0) 0x7f42d40588a0 con 0x7f42dc15dc10 2024-04-25T08:32:37.455 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:37.455 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:16.572337+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.a","mappings":[{"from":4,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:37.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.461+0000 7f42e2377640 1 -- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42bc06b4d0 msgr2=0x7f42bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.461+0000 7f42e2377640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42bc06b4d0 0x7f42bc06d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009aa0 tx=0x7f42c4009190 comp rx=0 tx=0).stop 2024-04-25T08:32:37.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.461+0000 7f42e2377640 1 -- 172.21.15.38:0/2210211062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42dc15dc10 msgr2=0x7f42dc0ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.461+0000 7f42e2377640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42dc15dc10 0x7f42dc0ccf30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f42d4001b60 tx=0x7f42d400a430 comp rx=0 tx=0).stop 2024-04-25T08:32:37.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.461+0000 7f42e2377640 1 -- 172.21.15.38:0/2210211062 shutdown_connections 2024-04-25T08:32:37.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.461+0000 7f42e2377640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42dc15dc10 0x7f42dc0ccf30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.461+0000 7f42e2377640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42bc06b4d0 0x7f42bc06d990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.461+0000 7f42e2377640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42dc0d3900 0x7f42dc0cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.461+0000 7f42e2377640 1 --2- 172.21.15.38:0/2210211062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42dc0ced60 0x7f42dc1584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.461+0000 7f42e2377640 1 -- 172.21.15.38:0/2210211062 >> 172.21.15.38:0/2210211062 conn(0x7f42dc066d10 msgr2=0x7f42dc15ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.461+0000 7f42e2377640 1 -- 172.21.15.38:0/2210211062 shutdown_connections 2024-04-25T08:32:37.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.461+0000 7f42e2377640 1 -- 172.21.15.38:0/2210211062 wait complete. 2024-04-25T08:32:37.468 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:32:37.468 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:37.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.573+0000 7f9d89ae1640 1 Processor -- start 2024-04-25T08:32:37.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.573+0000 7f9d89ae1640 1 -- start start 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.574+0000 7f9d89ae1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9d8406c110 0x7f9d8406c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.574+0000 7f9d89ae1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9d840ce2e0 0x7f9d840d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.574+0000 7f9d89ae1640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8406ca30 con 0x7f9d8415df50 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.574+0000 7f9d89ae1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d840cf2d0 con 0x7f9d8406c110 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.574+0000 7f9d89ae1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d840ceff0 con 0x7f9d840ce2e0 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.574+0000 7f9d837fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9d8406c110 0x7f9d8406c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.574+0000 7f9d837fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9d8406c110 0x7f9d8406c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44888/0 (socket says 172.21.15.38:44888) 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.574+0000 7f9d837fe640 1 -- 172.21.15.38:0/3367740877 learned_addr learned my addr 172.21.15.38:0/3367740877 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.575+0000 7f9d83fff640 1 --2- 172.21.15.38:0/3367740877 >> v2:172.21.15.107:3300/0 conn(0x7f9d840ce2e0 0x7f9d840d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.575+0000 7f9d837fe640 1 -- 172.21.15.38:0/3367740877 >> v2:172.21.15.107:3300/0 conn(0x7f9d840ce2e0 msgr2=0x7f9d840d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.575+0000 7f9d837fe640 1 --2- 172.21.15.38:0/3367740877 >> v2:172.21.15.107:3300/0 conn(0x7f9d840ce2e0 0x7f9d840d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.575+0000 7f9d837fe640 1 -- 172.21.15.38:0/3367740877 >> v1:172.21.15.107:6789/0 conn(0x7f9d8415df50 legacy=0x7f9d840ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.575+0000 7f9d837fe640 1 -- 172.21.15.38:0/3367740877 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d840d1f70 con 0x7f9d8406c110 2024-04-25T08:32:37.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.575+0000 7f9d83fff640 1 --2- 172.21.15.38:0/3367740877 >> v2:172.21.15.107:3300/0 conn(0x7f9d840ce2e0 0x7f9d840d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.575+0000 7f9d837fe640 1 --2- 172.21.15.38:0/3367740877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d8406c110 0x7f9d8406c4f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9d840b5400 tx=0x7f9d7402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e50b15bee4e13a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.575+0000 7f9d827fc640 1 -- 172.21.15.38:0/3367740877 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74037070 con 0x7f9d8406c110 2024-04-25T08:32:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.576+0000 7f9d827fc640 1 -- 172.21.15.38:0/3367740877 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d74032440 con 0x7f9d8406c110 2024-04-25T08:32:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.576+0000 7f9d827fc640 1 -- 172.21.15.38:0/3367740877 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74032810 con 0x7f9d8406c110 2024-04-25T08:32:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.576+0000 7f9d89ae1640 1 -- 172.21.15.38:0/3367740877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d8406c110 msgr2=0x7f9d8406c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.576+0000 7f9d89ae1640 1 --2- 172.21.15.38:0/3367740877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d8406c110 0x7f9d8406c4f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9d840b5400 tx=0x7f9d7402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.576+0000 7f9d89ae1640 1 -- 172.21.15.38:0/3367740877 shutdown_connections 2024-04-25T08:32:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.576+0000 7f9d89ae1640 1 --2- 172.21.15.38:0/3367740877 >> v2:172.21.15.107:3300/0 conn(0x7f9d840ce2e0 0x7f9d840d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.576+0000 7f9d89ae1640 1 --2- 172.21.15.38:0/3367740877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d8406c110 0x7f9d8406c4f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.576+0000 7f9d89ae1640 1 -- 172.21.15.38:0/3367740877 >> 172.21.15.38:0/3367740877 conn(0x7f9d840d3fb0 msgr2=0x7f9d840d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.576+0000 7f9d89ae1640 1 -- 172.21.15.38:0/3367740877 shutdown_connections 2024-04-25T08:32:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.576+0000 7f9d89ae1640 1 -- 172.21.15.38:0/3367740877 wait complete. 2024-04-25T08:32:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.577+0000 7f9d89ae1640 1 Processor -- start 2024-04-25T08:32:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.578+0000 7f9d89ae1640 1 -- start start 2024-04-25T08:32:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.578+0000 7f9d89ae1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d8406c110 0x7f9d840c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.578+0000 7f9d89ae1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d840ce2e0 0x7f9d840cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.578+0000 7f9d89ae1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8415df50 0x7f9d8415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.578+0000 7f9d89ae1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840d2f90 con 0x7f9d8406c110 2024-04-25T08:32:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.578+0000 7f9d89ae1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840d2530 con 0x7f9d8415df50 2024-04-25T08:32:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.578+0000 7f9d89ae1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d840d3110 con 0x7f9d840ce2e0 2024-04-25T08:32:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.578+0000 7f9d837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d8406c110 0x7f9d840c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.578+0000 7f9d837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d8406c110 0x7f9d840c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44890/0 (socket says 172.21.15.38:44890) 2024-04-25T08:32:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.578+0000 7f9d837fe640 1 -- 172.21.15.38:0/1586968119 learned_addr learned my addr 172.21.15.38:0/1586968119 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d82ffd640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d840ce2e0 0x7f9d840cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d83fff640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8415df50 0x7f9d8415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d837fe640 1 -- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d840ce2e0 msgr2=0x7f9d840cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d837fe640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d840ce2e0 0x7f9d840cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d837fe640 1 -- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8415df50 msgr2=0x7f9d8415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d837fe640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8415df50 0x7f9d8415f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d837fe640 1 -- 172.21.15.38:0/1586968119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d840cab60 con 0x7f9d8406c110 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d82ffd640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d840ce2e0 0x7f9d840cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d837fe640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d8406c110 0x7f9d840c8110 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9d7402a1e0 tx=0x7f9d740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d6bfff640 1 -- 172.21.15.38:0/1586968119 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74037040 con 0x7f9d8406c110 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d6bfff640 1 -- 172.21.15.38:0/1586968119 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d74042070 con 0x7f9d8406c110 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d89ae1640 1 -- 172.21.15.38:0/1586968119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d840cd5a0 con 0x7f9d8406c110 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d6bfff640 1 -- 172.21.15.38:0/1586968119 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d74030b90 con 0x7f9d8406c110 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.579+0000 7f9d89ae1640 1 -- 172.21.15.38:0/1586968119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d840cb0e0 con 0x7f9d8406c110 2024-04-25T08:32:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.580+0000 7f9d89ae1640 1 -- 172.21.15.38:0/1586968119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d840c71a0 con 0x7f9d8406c110 2024-04-25T08:32:37.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.581+0000 7f9d6bfff640 1 -- 172.21.15.38:0/1586968119 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d74047460 con 0x7f9d8406c110 2024-04-25T08:32:37.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.581+0000 7f9d6bfff640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9d7006b4d0 0x7f9d7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.581+0000 7f9d6bfff640 1 -- 172.21.15.38:0/1586968119 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f9d740ab220 con 0x7f9d8406c110 2024-04-25T08:32:37.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.581+0000 7f9d82ffd640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9d7006b4d0 0x7f9d7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.582+0000 7f9d82ffd640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9d7006b4d0 0x7f9d7006d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9d840c6f60 tx=0x7f9d78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.586+0000 7f9d6bfff640 1 -- 172.21.15.38:0/1586968119 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d840c71a0 con 0x7f9d8406c110 2024-04-25T08:32:37.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.803+0000 7f9d89ae1640 1 -- 172.21.15.38:0/1586968119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d8406c5f0 con 0x7f9d8406c110 2024-04-25T08:32:37.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.803+0000 7f9d6bfff640 1 -- 172.21.15.38:0/1586968119 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f9d8406c5f0 con 0x7f9d8406c110 2024-04-25T08:32:37.799 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:37.800 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4242612806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4242612806},{"type":"v1","addr":"172.21.15.38:6835","nonce":4242612806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2154167637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":2154167637},{"type":"v1","addr":"172.21.15.107:6837","nonce":2154167637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/678452936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":678452936},{"type":"v1","addr":"172.21.15.38:6834","nonce":678452936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:31:35.665008+0000","modified":"2024-04-25T08:31:51.799830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6836/1104921666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1104921666},{"type":"v1","addr":"172.21.15.107:6836","nonce":1104921666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:32:37.800 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-04-25T08:32:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.805+0000 7f9d89ae1640 1 -- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9d7006b4d0 msgr2=0x7f9d7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.805+0000 7f9d89ae1640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9d7006b4d0 0x7f9d7006d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9d840c6f60 tx=0x7f9d78009040 comp rx=0 tx=0).stop 2024-04-25T08:32:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.805+0000 7f9d89ae1640 1 -- 172.21.15.38:0/1586968119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d8406c110 msgr2=0x7f9d840c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.805+0000 7f9d89ae1640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d8406c110 0x7f9d840c8110 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9d7402a1e0 tx=0x7f9d740046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.805+0000 7f9d89ae1640 1 -- 172.21.15.38:0/1586968119 shutdown_connections 2024-04-25T08:32:37.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.805+0000 7f9d89ae1640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d8415df50 0x7f9d8415f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.805+0000 7f9d89ae1640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9d7006b4d0 0x7f9d7006d990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.805+0000 7f9d89ae1640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9d840ce2e0 0x7f9d840cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.805+0000 7f9d89ae1640 1 --2- 172.21.15.38:0/1586968119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d8406c110 0x7f9d840c8110 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.805+0000 7f9d89ae1640 1 -- 172.21.15.38:0/1586968119 >> 172.21.15.38:0/1586968119 conn(0x7f9d840d3fb0 msgr2=0x7f9d840736e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.805+0000 7f9d89ae1640 1 -- 172.21.15.38:0/1586968119 shutdown_connections 2024-04-25T08:32:37.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.805+0000 7f9d89ae1640 1 -- 172.21.15.38:0/1586968119 wait complete. 2024-04-25T08:32:37.812 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:37.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.916+0000 7f123d6f5640 1 Processor -- start 2024-04-25T08:32:37.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.916+0000 7f123d6f5640 1 -- start start 2024-04-25T08:32:37.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.917+0000 7f123d6f5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f123815dc10 0x7f12380ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.917+0000 7f123d6f5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12380d3e70 con 0x7f12380ced60 2024-04-25T08:32:37.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.917+0000 7f123d6f5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12380d4150 con 0x7f12380d0f60 2024-04-25T08:32:37.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.917+0000 7f123d6f5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f123815dff0 con 0x7f123815dc10 2024-04-25T08:32:37.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.917+0000 7f1236ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f123815dc10 0x7f12380ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.917+0000 7f12367fc640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f12380ced60 0x7f12380d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56560/0 (socket says 172.21.15.38:56560) 2024-04-25T08:32:37.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.917+0000 7f12367fc640 1 -- 172.21.15.38:0/830866721 learned_addr learned my addr 172.21.15.38:0/830866721 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:37.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.917+0000 7f1236ffd640 1 -- 172.21.15.38:0/830866721 >> v1:172.21.15.107:6789/0 conn(0x7f12380d0f60 legacy=0x7f12380b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:37.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.917+0000 7f1236ffd640 1 -- 172.21.15.38:0/830866721 >> v1:172.21.15.38:6789/0 conn(0x7f12380ced60 legacy=0x7f12380d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.917+0000 7f1236ffd640 1 -- 172.21.15.38:0/830866721 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12380d4430 con 0x7f123815dc10 2024-04-25T08:32:37.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.917+0000 7f1235ffb640 1 -- 172.21.15.38:0/830866721 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929805272 0 0) 0x7f12380d3e70 con 0x7f12380ced60 2024-04-25T08:32:37.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.918+0000 7f1236ffd640 1 --2- 172.21.15.38:0/830866721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f123815dc10 0x7f12380ce820 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f12280097a0 tx=0x7f122802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54979b2708392ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.918+0000 7f1235ffb640 1 -- 172.21.15.38:0/830866721 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228037070 con 0x7f123815dc10 2024-04-25T08:32:37.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.918+0000 7f1235ffb640 1 -- 172.21.15.38:0/830866721 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122802ae50 con 0x7f123815dc10 2024-04-25T08:32:37.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.918+0000 7f1235ffb640 1 -- 172.21.15.38:0/830866721 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228032660 con 0x7f123815dc10 2024-04-25T08:32:37.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.919+0000 7f123d6f5640 1 -- 172.21.15.38:0/830866721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f123815dc10 msgr2=0x7f12380ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.919+0000 7f123d6f5640 1 --2- 172.21.15.38:0/830866721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f123815dc10 0x7f12380ce820 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f12280097a0 tx=0x7f122802a140 comp rx=0 tx=0).stop 2024-04-25T08:32:37.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.919+0000 7f123d6f5640 1 -- 172.21.15.38:0/830866721 shutdown_connections 2024-04-25T08:32:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.919+0000 7f123d6f5640 1 --2- 172.21.15.38:0/830866721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f123815dc10 0x7f12380ce820 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.919+0000 7f123d6f5640 1 -- 172.21.15.38:0/830866721 >> 172.21.15.38:0/830866721 conn(0x7f1238066d10 msgr2=0x7f1238067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.919+0000 7f123d6f5640 1 -- 172.21.15.38:0/830866721 shutdown_connections 2024-04-25T08:32:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.919+0000 7f123d6f5640 1 -- 172.21.15.38:0/830866721 wait complete. 2024-04-25T08:32:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.920+0000 7f123d6f5640 1 Processor -- start 2024-04-25T08:32:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.920+0000 7f123d6f5640 1 -- start start 2024-04-25T08:32:37.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f123d6f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12380ced60 0x7f123814ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f123d6f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12380d0f60 0x7f12380c9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f123d6f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f123815dc10 0x7f12380c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f123d6f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12380d4710 con 0x7f12380ced60 2024-04-25T08:32:37.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f123d6f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12380b80f0 con 0x7f12380d0f60 2024-04-25T08:32:37.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f123d6f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12380b8270 con 0x7f123815dc10 2024-04-25T08:32:37.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f1236ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12380ced60 0x7f123814ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f1236ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12380ced60 0x7f123814ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44912/0 (socket says 172.21.15.38:44912) 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f1236ffd640 1 -- 172.21.15.38:0/1335494063 learned_addr learned my addr 172.21.15.38:0/1335494063 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f12367fc640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12380d0f60 0x7f12380c9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f12377fe640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f123815dc10 0x7f12380c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f1236ffd640 1 -- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f123815dc10 msgr2=0x7f12380c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f1236ffd640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f123815dc10 0x7f12380c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f1236ffd640 1 -- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12380d0f60 msgr2=0x7f12380c9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f1236ffd640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12380d0f60 0x7f12380c9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.921+0000 7f1236ffd640 1 -- 172.21.15.38:0/1335494063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12381e62e0 con 0x7f12380ced60 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.922+0000 7f1236ffd640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12380ced60 0x7f123814ec20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f12280097a0 tx=0x7f12280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.922+0000 7f121f7fe640 1 -- 172.21.15.38:0/1335494063 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228037040 con 0x7f12380ced60 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.922+0000 7f121f7fe640 1 -- 172.21.15.38:0/1335494063 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1228004030 con 0x7f12380ced60 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.922+0000 7f123d6f5640 1 -- 172.21.15.38:0/1335494063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f123814f160 con 0x7f12380ced60 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.922+0000 7f121f7fe640 1 -- 172.21.15.38:0/1335494063 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1228006e10 con 0x7f12380ced60 2024-04-25T08:32:37.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.922+0000 7f123d6f5640 1 -- 172.21.15.38:0/1335494063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12381e6580 con 0x7f12380ced60 2024-04-25T08:32:37.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.923+0000 7f123d6f5640 1 -- 172.21.15.38:0/1335494063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1238068e80 con 0x7f12380ced60 2024-04-25T08:32:37.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.923+0000 7f121f7fe640 1 -- 172.21.15.38:0/1335494063 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f122803d650 con 0x7f12380ced60 2024-04-25T08:32:37.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.924+0000 7f121f7fe640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f122406b4d0 0x7f122406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:37.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.924+0000 7f121f7fe640 1 -- 172.21.15.38:0/1335494063 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f12280aaeb0 con 0x7f12380ced60 2024-04-25T08:32:37.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.924+0000 7f12367fc640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f122406b4d0 0x7f122406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:37.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.925+0000 7f12367fc640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f122406b4d0 0x7f122406d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f123814f4d0 tx=0x7f122c007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:37.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:37.928+0000 7f121f7fe640 1 -- 172.21.15.38:0/1335494063 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1228042020 con 0x7f12380ced60 2024-04-25T08:32:38.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.146+0000 7f123d6f5640 1 -- 172.21.15.38:0/1335494063 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1238160e30 con 0x7f12380ced60 2024-04-25T08:32:38.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.146+0000 7f121f7fe640 1 -- 172.21.15.38:0/1335494063 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f12280798d0 con 0x7f12380ced60 2024-04-25T08:32:38.142 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:38.142 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4242612806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4242612806},{"type":"v1","addr":"172.21.15.38:6835","nonce":4242612806}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2154167637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":2154167637},{"type":"v1","addr":"172.21.15.107:6837","nonce":2154167637}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/678452936","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":678452936},{"type":"v1","addr":"172.21.15.38:6834","nonce":678452936}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:31:35.665008+0000","modified":"2024-04-25T08:31:51.799830+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6836/1104921666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":1104921666},{"type":"v1","addr":"172.21.15.107:6836","nonce":1104921666}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:32:38.143 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-04-25T08:32:38.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.148+0000 7f123d6f5640 1 -- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f122406b4d0 msgr2=0x7f122406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.148+0000 7f123d6f5640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f122406b4d0 0x7f122406d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f123814f4d0 tx=0x7f122c007b50 comp rx=0 tx=0).stop 2024-04-25T08:32:38.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.148+0000 7f123d6f5640 1 -- 172.21.15.38:0/1335494063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12380ced60 msgr2=0x7f123814ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.148+0000 7f123d6f5640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12380ced60 0x7f123814ec20 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f12280097a0 tx=0x7f12280046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.148+0000 7f123d6f5640 1 -- 172.21.15.38:0/1335494063 shutdown_connections 2024-04-25T08:32:38.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.148+0000 7f123d6f5640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f122406b4d0 0x7f122406d990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.148+0000 7f123d6f5640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f123815dc10 0x7f12380c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.148+0000 7f123d6f5640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12380d0f60 0x7f12380c9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.148+0000 7f123d6f5640 1 --2- 172.21.15.38:0/1335494063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f12380ced60 0x7f123814ec20 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.148+0000 7f123d6f5640 1 -- 172.21.15.38:0/1335494063 >> 172.21.15.38:0/1335494063 conn(0x7f1238066d10 msgr2=0x7f123815f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:38.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.148+0000 7f123d6f5640 1 -- 172.21.15.38:0/1335494063 shutdown_connections 2024-04-25T08:32:38.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.148+0000 7f123d6f5640 1 -- 172.21.15.38:0/1335494063 wait complete. 2024-04-25T08:32:38.155 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:38.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.259+0000 7fc8ead3b640 1 Processor -- start 2024-04-25T08:32:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.259+0000 7fc8ead3b640 1 -- start start 2024-04-25T08:32:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.260+0000 7fc8ead3b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc8e406c110 0x7fc8e406c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.260+0000 7fc8ead3b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e406ca30 con 0x7fc8e415df50 2024-04-25T08:32:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.260+0000 7fc8ead3b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e40ceff0 con 0x7fc8e40ce2e0 2024-04-25T08:32:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.260+0000 7fc8ead3b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e40cf2d0 con 0x7fc8e406c110 2024-04-25T08:32:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.260+0000 7fc8e8ad7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc8e406c110 0x7fc8e406c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.260+0000 7fc8e3fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc8e415df50 0x7fc8e40ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56570/0 (socket says 172.21.15.38:56570) 2024-04-25T08:32:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.260+0000 7fc8e3fff640 1 -- 172.21.15.38:0/888155232 learned_addr learned my addr 172.21.15.38:0/888155232 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.261+0000 7fc8e8ad7640 1 -- 172.21.15.38:0/888155232 >> v1:172.21.15.107:6789/0 conn(0x7fc8e40ce2e0 legacy=0x7fc8e40d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.261+0000 7fc8e8ad7640 1 -- 172.21.15.38:0/888155232 >> v1:172.21.15.38:6789/0 conn(0x7fc8e415df50 legacy=0x7fc8e40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.261+0000 7fc8e8ad7640 1 -- 172.21.15.38:0/888155232 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e40d1f70 con 0x7fc8e406c110 2024-04-25T08:32:38.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.261+0000 7fc8e8ad7640 1 --2- 172.21.15.38:0/888155232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8e406c110 0x7fc8e406c4f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009d90 tx=0x7fc8dc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6003c5d5b1d52e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:38.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.261+0000 7fc8e37fe640 1 -- 172.21.15.38:0/888155232 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc031600 con 0x7fc8e406c110 2024-04-25T08:32:38.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.262+0000 7fc8e37fe640 1 -- 172.21.15.38:0/888155232 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc0317a0 con 0x7fc8e406c110 2024-04-25T08:32:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.262+0000 7fc8e37fe640 1 -- 172.21.15.38:0/888155232 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc031b70 con 0x7fc8e406c110 2024-04-25T08:32:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.262+0000 7fc8ead3b640 1 -- 172.21.15.38:0/888155232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8e406c110 msgr2=0x7fc8e406c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.262+0000 7fc8ead3b640 1 --2- 172.21.15.38:0/888155232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8e406c110 0x7fc8e406c4f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009d90 tx=0x7fc8dc02a610 comp rx=0 tx=0).stop 2024-04-25T08:32:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.262+0000 7fc8ead3b640 1 -- 172.21.15.38:0/888155232 shutdown_connections 2024-04-25T08:32:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.262+0000 7fc8ead3b640 1 --2- 172.21.15.38:0/888155232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8e406c110 0x7fc8e406c4f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.262+0000 7fc8ead3b640 1 -- 172.21.15.38:0/888155232 >> 172.21.15.38:0/888155232 conn(0x7fc8e40d3fb0 msgr2=0x7fc8e40d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.262+0000 7fc8ead3b640 1 -- 172.21.15.38:0/888155232 shutdown_connections 2024-04-25T08:32:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.262+0000 7fc8ead3b640 1 -- 172.21.15.38:0/888155232 wait complete. 2024-04-25T08:32:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.263+0000 7fc8ead3b640 1 Processor -- start 2024-04-25T08:32:38.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.263+0000 7fc8ead3b640 1 -- start start 2024-04-25T08:32:38.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.264+0000 7fc8ead3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8e406c110 0x7fc8e40c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.264+0000 7fc8ead3b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e40ce2e0 0x7fc8e414dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.264+0000 7fc8ead3b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8e415df50 0x7fc8e415fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.264+0000 7fc8ead3b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40d3160 con 0x7fc8e406c110 2024-04-25T08:32:38.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.264+0000 7fc8ead3b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40d2530 con 0x7fc8e415df50 2024-04-25T08:32:38.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.264+0000 7fc8ead3b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e40d32e0 con 0x7fc8e40ce2e0 2024-04-25T08:32:38.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.264+0000 7fc8e8ad7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8e406c110 0x7fc8e40c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.264+0000 7fc8e8ad7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8e406c110 0x7fc8e40c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44932/0 (socket says 172.21.15.38:44932) 2024-04-25T08:32:38.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.264+0000 7fc8e8ad7640 1 -- 172.21.15.38:0/1408942502 learned_addr learned my addr 172.21.15.38:0/1408942502 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:38.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.264+0000 7fc8e3fff640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e40ce2e0 0x7fc8e414dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8e8ad7640 1 -- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e40ce2e0 msgr2=0x7fc8e414dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8e8ad7640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e40ce2e0 0x7fc8e414dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8e8ad7640 1 -- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8e415df50 msgr2=0x7fc8e415fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8e92d8640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8e415df50 0x7fc8e415fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8e8ad7640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8e415df50 0x7fc8e415fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8e8ad7640 1 -- 172.21.15.38:0/1408942502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e414e400 con 0x7fc8e406c110 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8e3fff640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e40ce2e0 0x7fc8e414dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8e8ad7640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8e406c110 0x7fc8e40c7420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009eb0 tx=0x7fc8dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8e17fa640 1 -- 172.21.15.38:0/1408942502 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc03f070 con 0x7fc8e406c110 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8e17fa640 1 -- 172.21.15.38:0/1408942502 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc030be0 con 0x7fc8e406c110 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8ead3b640 1 -- 172.21.15.38:0/1408942502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e40c7a00 con 0x7fc8e406c110 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8e17fa640 1 -- 172.21.15.38:0/1408942502 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc044460 con 0x7fc8e406c110 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.265+0000 7fc8ead3b640 1 -- 172.21.15.38:0/1408942502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e40c7fc0 con 0x7fc8e406c110 2024-04-25T08:32:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.266+0000 7fc8ead3b640 1 -- 172.21.15.38:0/1408942502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e4068e80 con 0x7fc8e406c110 2024-04-25T08:32:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.267+0000 7fc8e17fa640 1 -- 172.21.15.38:0/1408942502 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8dc03b410 con 0x7fc8e406c110 2024-04-25T08:32:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.267+0000 7fc8e17fa640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8d006b4d0 0x7fc8d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.267+0000 7fc8e17fa640 1 -- 172.21.15.38:0/1408942502 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7fc8dc0abd60 con 0x7fc8e406c110 2024-04-25T08:32:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.267+0000 7fc8e3fff640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8d006b4d0 0x7fc8d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.268+0000 7fc8e3fff640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8d006b4d0 0x7fc8d006d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc8e415b4c0 tx=0x7fc8d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:38.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.271+0000 7fc8e17fa640 1 -- 172.21.15.38:0/1408942502 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8dc07a1f0 con 0x7fc8e406c110 2024-04-25T08:32:38.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.457+0000 7fc8ead3b640 1 -- 172.21.15.38:0/1408942502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8e40d2bf0 con 0x7fc8e406c110 2024-04-25T08:32:38.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.458+0000 7fc8e17fa640 1 -- 172.21.15.38:0/1408942502 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18969 (secure 0 0 0) 0x7fc8dc080030 con 0x7fc8e406c110 2024-04-25T08:32:38.455 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:38.455 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:16.572337+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:31:33.425500+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:31:34.648941+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:31:38.358207+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.a","mappings":[{"from":4,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":6}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.2b","mappings":[{"from":0,"to":2}]},{"pgid":"3.37","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.b","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:38.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.461+0000 7fc8ead3b640 1 -- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8d006b4d0 msgr2=0x7fc8d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.461+0000 7fc8ead3b640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8d006b4d0 0x7fc8d006d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc8e415b4c0 tx=0x7fc8d8009040 comp rx=0 tx=0).stop 2024-04-25T08:32:38.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.461+0000 7fc8ead3b640 1 -- 172.21.15.38:0/1408942502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8e406c110 msgr2=0x7fc8e40c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.461+0000 7fc8ead3b640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8e406c110 0x7fc8e40c7420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009eb0 tx=0x7fc8dc004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:38.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.461+0000 7fc8ead3b640 1 -- 172.21.15.38:0/1408942502 shutdown_connections 2024-04-25T08:32:38.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.461+0000 7fc8ead3b640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8e415df50 0x7fc8e415fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.461+0000 7fc8ead3b640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8d006b4d0 0x7fc8d006d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.461+0000 7fc8ead3b640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8e40ce2e0 0x7fc8e414dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.461+0000 7fc8ead3b640 1 --2- 172.21.15.38:0/1408942502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8e406c110 0x7fc8e40c7420 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.461+0000 7fc8ead3b640 1 -- 172.21.15.38:0/1408942502 >> 172.21.15.38:0/1408942502 conn(0x7fc8e40d3fb0 msgr2=0x7fc8e40d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:38.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.461+0000 7fc8ead3b640 1 -- 172.21.15.38:0/1408942502 shutdown_connections 2024-04-25T08:32:38.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.461+0000 7fc8ead3b640 1 -- 172.21.15.38:0/1408942502 wait complete. 2024-04-25T08:32:38.468 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:32:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.573+0000 7f5921dd5640 1 Processor -- start 2024-04-25T08:32:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.574+0000 7f5921dd5640 1 -- start start 2024-04-25T08:32:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.574+0000 7f5921dd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f591c15db80 0x7f591c1616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.574+0000 7f5921dd5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f591c0b9600 0x7f591c0d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.574+0000 7f5921dd5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591c162030 con 0x7f591c0d0660 2024-04-25T08:32:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.574+0000 7f5921dd5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c0d0ad0 con 0x7f591c0b9600 2024-04-25T08:32:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.574+0000 7f5921dd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c1625f0 con 0x7f591c15db80 2024-04-25T08:32:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.575+0000 7f591b7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f591c15db80 0x7f591c1616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.575+0000 7f591bfff640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f591c0d0660 0x7f591c0b52f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:38018/0 (socket says 172.21.15.38:38018) 2024-04-25T08:32:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.575+0000 7f591bfff640 1 -- 172.21.15.38:0/1436166876 learned_addr learned my addr 172.21.15.38:0/1436166876 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.575+0000 7f591affd640 1 --2- 172.21.15.38:0/1436166876 >> v2:172.21.15.107:3300/0 conn(0x7f591c0b9600 0x7f591c0d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.575+0000 7f591b7fe640 1 -- 172.21.15.38:0/1436166876 >> v2:172.21.15.107:3300/0 conn(0x7f591c0b9600 msgr2=0x7f591c0d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.575+0000 7f591a7fc640 1 -- 172.21.15.38:0/1436166876 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302814036 0 0) 0x7f591c162030 con 0x7f591c0d0660 2024-04-25T08:32:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.575+0000 7f591b7fe640 1 --2- 172.21.15.38:0/1436166876 >> v2:172.21.15.107:3300/0 conn(0x7f591c0b9600 0x7f591c0d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.575+0000 7f591b7fe640 1 -- 172.21.15.38:0/1436166876 >> v1:172.21.15.107:6789/0 conn(0x7f591c0d0660 legacy=0x7f591c0b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.575+0000 7f591b7fe640 1 -- 172.21.15.38:0/1436166876 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c162310 con 0x7f591c15db80 2024-04-25T08:32:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.576+0000 7f591b7fe640 1 --2- 172.21.15.38:0/1436166876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f591c15db80 0x7f591c1616e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f59100097a0 tx=0x7f591002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aad0f8cfcfbcd6e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.576+0000 7f591a7fc640 1 -- 172.21.15.38:0/1436166876 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910037070 con 0x7f591c15db80 2024-04-25T08:32:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.576+0000 7f591a7fc640 1 -- 172.21.15.38:0/1436166876 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591002ae50 con 0x7f591c15db80 2024-04-25T08:32:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.576+0000 7f591a7fc640 1 -- 172.21.15.38:0/1436166876 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5910032660 con 0x7f591c15db80 2024-04-25T08:32:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.576+0000 7f5921dd5640 1 -- 172.21.15.38:0/1436166876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f591c15db80 msgr2=0x7f591c1616e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.576+0000 7f5921dd5640 1 --2- 172.21.15.38:0/1436166876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f591c15db80 0x7f591c1616e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f59100097a0 tx=0x7f591002a100 comp rx=0 tx=0).stop 2024-04-25T08:32:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.576+0000 7f5921dd5640 1 -- 172.21.15.38:0/1436166876 shutdown_connections 2024-04-25T08:32:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.576+0000 7f5921dd5640 1 --2- 172.21.15.38:0/1436166876 >> v2:172.21.15.107:3300/0 conn(0x7f591c0b9600 0x7f591c0d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.576+0000 7f5921dd5640 1 --2- 172.21.15.38:0/1436166876 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f591c15db80 0x7f591c1616e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.576+0000 7f5921dd5640 1 -- 172.21.15.38:0/1436166876 >> 172.21.15.38:0/1436166876 conn(0x7f591c0668f0 msgr2=0x7f591c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.576+0000 7f5921dd5640 1 -- 172.21.15.38:0/1436166876 shutdown_connections 2024-04-25T08:32:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.576+0000 7f5921dd5640 1 -- 172.21.15.38:0/1436166876 wait complete. 2024-04-25T08:32:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.578+0000 7f5921dd5640 1 Processor -- start 2024-04-25T08:32:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.578+0000 7f5921dd5640 1 -- start start 2024-04-25T08:32:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.578+0000 7f5921dd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f591c0b9600 0x7f591c15f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.578+0000 7f5921dd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f591c0d0660 0x7f591c1f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.578+0000 7f5921dd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f591c15db80 0x7f591c0b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.578+0000 7f5921dd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c161c20 con 0x7f591c0d0660 2024-04-25T08:32:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.578+0000 7f5921dd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c15df60 con 0x7f591c15db80 2024-04-25T08:32:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.578+0000 7f5921dd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c161da0 con 0x7f591c0b9600 2024-04-25T08:32:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f591affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f591c0d0660 0x7f591c1f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f591affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f591c0d0660 0x7f591c1f4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44946/0 (socket says 172.21.15.38:44946) 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f591affd640 1 -- 172.21.15.38:0/3815170232 learned_addr learned my addr 172.21.15.38:0/3815170232 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f591b7fe640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f591c0b9600 0x7f591c15f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f591bfff640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f591c15db80 0x7f591c0b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f591affd640 1 -- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f591c0b9600 msgr2=0x7f591c15f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f591affd640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f591c0b9600 0x7f591c15f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f591affd640 1 -- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f591c15db80 msgr2=0x7f591c0b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f591affd640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f591c15db80 0x7f591c0b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f591affd640 1 -- 172.21.15.38:0/3815170232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0b2110 con 0x7f591c0d0660 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f591bfff640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f591c15db80 0x7f591c0b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f591affd640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f591c0d0660 0x7f591c1f4840 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f590c00e900 tx=0x7f590c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f5903fff640 1 -- 172.21.15.38:0/3815170232 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c017070 con 0x7f591c0d0660 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f5903fff640 1 -- 172.21.15.38:0/3815170232 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c00ca00 con 0x7f591c0d0660 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f5921dd5640 1 -- 172.21.15.38:0/3815170232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c15e550 con 0x7f591c0d0660 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f5903fff640 1 -- 172.21.15.38:0/3815170232 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c00cdf0 con 0x7f591c0d0660 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.579+0000 7f5921dd5640 1 -- 172.21.15.38:0/3815170232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c15eb10 con 0x7f591c0d0660 2024-04-25T08:32:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.580+0000 7f5921dd5640 1 -- 172.21.15.38:0/3815170232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c04ef90 con 0x7f591c0d0660 2024-04-25T08:32:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.581+0000 7f5903fff640 1 -- 172.21.15.38:0/3815170232 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f590c01a070 con 0x7f591c0d0660 2024-04-25T08:32:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.581+0000 7f5903fff640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f590806b4d0 0x7f590806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.581+0000 7f5903fff640 1 -- 172.21.15.38:0/3815170232 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6822+0+0 (secure 0 0 0) 0x7f590c08a5e0 con 0x7f591c0d0660 2024-04-25T08:32:38.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.581+0000 7f591b7fe640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f590806b4d0 0x7f590806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.582+0000 7f591b7fe640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f590806b4d0 0x7f590806d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5910034000 tx=0x7f591002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.585+0000 7f5903fff640 1 -- 172.21.15.38:0/3815170232 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f590c0589b0 con 0x7f591c0d0660 2024-04-25T08:32:38.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.804+0000 7f5921dd5640 1 -- 172.21.15.38:0/3815170232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f591c0cdc30 con 0x7f591c0d0660 2024-04-25T08:32:38.801 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:32:38.805+0000 7ff7937f0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:32:38.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.807+0000 7f5903fff640 1 -- 172.21.15.38:0/3815170232 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7f590c05ca50 con 0x7f591c0d0660 2024-04-25T08:32:38.802 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:32:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.808+0000 7f5921dd5640 1 -- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f590806b4d0 msgr2=0x7f590806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.808+0000 7f5921dd5640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f590806b4d0 0x7f590806d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5910034000 tx=0x7f591002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.808+0000 7f5921dd5640 1 -- 172.21.15.38:0/3815170232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f591c0d0660 msgr2=0x7f591c1f4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.808+0000 7f5921dd5640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f591c0d0660 0x7f591c1f4840 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f590c00e900 tx=0x7f590c00edc0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.808+0000 7f5921dd5640 1 -- 172.21.15.38:0/3815170232 shutdown_connections 2024-04-25T08:32:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.808+0000 7f5921dd5640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f591c15db80 0x7f591c0b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.809+0000 7f5921dd5640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f591c0d0660 0x7f591c1f4840 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.809+0000 7f5921dd5640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f590806b4d0 0x7f590806d990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.809+0000 7f5921dd5640 1 --2- 172.21.15.38:0/3815170232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f591c0b9600 0x7f591c15f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.809+0000 7f5921dd5640 1 -- 172.21.15.38:0/3815170232 >> 172.21.15.38:0/3815170232 conn(0x7f591c0668f0 msgr2=0x7f591c0d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:38.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.809+0000 7f5921dd5640 1 -- 172.21.15.38:0/3815170232 shutdown_connections 2024-04-25T08:32:38.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.809+0000 7f5921dd5640 1 -- 172.21.15.38:0/3815170232 wait complete. 2024-04-25T08:32:38.810 INFO:tasks.ceph.mds.b.smithi107.stderr: -4345> 2024-04-25T08:31:39.887+0000 7fa78f396640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:32:38.816 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:32:38.840 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-25T08:32:38.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.922+0000 7fcae28dd640 1 Processor -- start 2024-04-25T08:32:38.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.922+0000 7fcae28dd640 1 -- start start 2024-04-25T08:32:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcae28dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcadc0d33d0 0x7fcadc0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcae28dd640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc0d2330 con 0x7fcadc0d0320 2024-04-25T08:32:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcae28dd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc0d2610 con 0x7fcadc0d1e90 2024-04-25T08:32:38.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcae28dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0d28f0 con 0x7fcadc0d33d0 2024-04-25T08:32:38.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcadbfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcadc0d0320 0x7fcadc0d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49538/0 (socket says 172.21.15.38:49538) 2024-04-25T08:32:38.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcadbfff640 1 -- 172.21.15.38:0/1080075800 learned_addr learned my addr 172.21.15.38:0/1080075800 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:38.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcadb7fe640 1 --2- 172.21.15.38:0/1080075800 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcadc0d33d0 0x7fcadc0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcadaffd640 1 -- 172.21.15.38:0/1080075800 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321538931 0 0) 0x7fcadc0d2330 con 0x7fcadc0d0320 2024-04-25T08:32:38.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcadaffd640 1 -- 172.21.15.38:0/1080075800 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcac4003520 con 0x7fcadc0d0320 2024-04-25T08:32:38.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcadaffd640 1 -- 172.21.15.38:0/1080075800 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446403719 0 0) 0x7fcadc0d2610 con 0x7fcadc0d1e90 2024-04-25T08:32:38.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcadb7fe640 1 -- 172.21.15.38:0/1080075800 >> v1:172.21.15.107:6789/0 conn(0x7fcadc0d1e90 legacy=0x7fcadc0b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcadb7fe640 1 -- 172.21.15.38:0/1080075800 >> v1:172.21.15.38:6789/0 conn(0x7fcadc0d0320 legacy=0x7fcadc0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.923+0000 7fcadb7fe640 1 -- 172.21.15.38:0/1080075800 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc0d2bd0 con 0x7fcadc0d33d0 2024-04-25T08:32:38.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.924+0000 7fcadb7fe640 1 --2- 172.21.15.38:0/1080075800 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcadc0d33d0 0x7fcadc0d1950 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcad00097d0 tx=0x7fcad002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6db13b8966b6584 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:38.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.924+0000 7fcadaffd640 1 -- 172.21.15.38:0/1080075800 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0037070 con 0x7fcadc0d33d0 2024-04-25T08:32:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.924+0000 7fcadaffd640 1 -- 172.21.15.38:0/1080075800 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad0032440 con 0x7fcadc0d33d0 2024-04-25T08:32:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.924+0000 7fcadaffd640 1 -- 172.21.15.38:0/1080075800 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad00327f0 con 0x7fcadc0d33d0 2024-04-25T08:32:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.925+0000 7fcae28dd640 1 -- 172.21.15.38:0/1080075800 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcadc0d33d0 msgr2=0x7fcadc0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.925+0000 7fcae28dd640 1 --2- 172.21.15.38:0/1080075800 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcadc0d33d0 0x7fcadc0d1950 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcad00097d0 tx=0x7fcad002a290 comp rx=0 tx=0).stop 2024-04-25T08:32:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.925+0000 7fcae28dd640 1 -- 172.21.15.38:0/1080075800 shutdown_connections 2024-04-25T08:32:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.925+0000 7fcae28dd640 1 --2- 172.21.15.38:0/1080075800 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcadc0d33d0 0x7fcadc0d1950 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.925+0000 7fcae28dd640 1 -- 172.21.15.38:0/1080075800 >> 172.21.15.38:0/1080075800 conn(0x7fcadc0668f0 msgr2=0x7fcadc066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.925+0000 7fcae28dd640 1 -- 172.21.15.38:0/1080075800 shutdown_connections 2024-04-25T08:32:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.925+0000 7fcae28dd640 1 -- 172.21.15.38:0/1080075800 wait complete. 2024-04-25T08:32:38.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.926+0000 7fcae28dd640 1 Processor -- start 2024-04-25T08:32:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.926+0000 7fcae28dd640 1 -- start start 2024-04-25T08:32:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcae28dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcadc0d0320 0x7fcadc0b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcae28dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcadc0d1e90 0x7fcadc0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcae28dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcadc0d33d0 0x7fcadc0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcae28dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc161390 con 0x7fcadc0d0320 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcae28dd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc161210 con 0x7fcadc0d33d0 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcae28dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0b9720 con 0x7fcadc0d1e90 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcadbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcadc0d0320 0x7fcadc0b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcadbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcadc0d0320 0x7fcadc0b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41698/0 (socket says 172.21.15.38:41698) 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcadbfff640 1 -- 172.21.15.38:0/2883743330 learned_addr learned my addr 172.21.15.38:0/2883743330 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcae0e7a640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcadc0d33d0 0x7fcadc0b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcadb7fe640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcadc0d1e90 0x7fcadc0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcadbfff640 1 -- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcadc0d1e90 msgr2=0x7fcadc0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcadbfff640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcadc0d1e90 0x7fcadc0b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcadbfff640 1 -- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcadc0d33d0 msgr2=0x7fcadc0b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcadbfff640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcadc0d33d0 0x7fcadc0b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcadbfff640 1 -- 172.21.15.38:0/2883743330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc0b4c80 con 0x7fcadc0d0320 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcadbfff640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcadc0d0320 0x7fcadc0b4200 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00c430 tx=0x7fcacc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.927+0000 7fcad8ff9640 1 -- 172.21.15.38:0/2883743330 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc017070 con 0x7fcadc0d0320 2024-04-25T08:32:38.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.928+0000 7fcad8ff9640 1 -- 172.21.15.38:0/2883743330 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc005ce0 con 0x7fcadc0d0320 2024-04-25T08:32:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.928+0000 7fcae28dd640 1 -- 172.21.15.38:0/2883743330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc158a00 con 0x7fcadc0d0320 2024-04-25T08:32:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.928+0000 7fcad8ff9640 1 -- 172.21.15.38:0/2883743330 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0072f0 con 0x7fcadc0d0320 2024-04-25T08:32:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.928+0000 7fcae28dd640 1 -- 172.21.15.38:0/2883743330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc158fc0 con 0x7fcadc0d0320 2024-04-25T08:32:38.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.928+0000 7fcae28dd640 1 -- 172.21.15.38:0/2883743330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc068e80 con 0x7fcadc0d0320 2024-04-25T08:32:38.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.929+0000 7fcad8ff9640 1 -- 172.21.15.38:0/2883743330 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcacc01a070 con 0x7fcadc0d0320 2024-04-25T08:32:38.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.929+0000 7fcad8ff9640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcac806b4d0 0x7fcac806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:38.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.929+0000 7fcad8ff9640 1 -- 172.21.15.38:0/2883743330 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6908+0+0 (secure 0 0 0) 0x7fcacc089ab0 con 0x7fcadc0d0320 2024-04-25T08:32:38.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.930+0000 7fcadb7fe640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcac806b4d0 0x7fcac806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:38.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.930+0000 7fcadb7fe640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcac806b4d0 0x7fcac806d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcad0034000 tx=0x7fcad002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:38.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:38.933+0000 7fcad8ff9640 1 -- 172.21.15.38:0/2883743330 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcadc0b4c80 con 0x7fcadc0d0320 2024-04-25T08:32:39.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.151+0000 7fcae28dd640 1 -- 172.21.15.38:0/2883743330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcadc0c9d50 con 0x7fcadc0d0320 2024-04-25T08:32:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.813+0000 7fcad8ff9640 1 -- 172.21.15.38:0/2883743330 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcacc057e30 con 0x7fcadc0d0320 2024-04-25T08:32:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.816+0000 7fcae28dd640 1 -- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcac806b4d0 msgr2=0x7fcac806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.816+0000 7fcae28dd640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcac806b4d0 0x7fcac806d990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcad0034000 tx=0x7fcad002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.816+0000 7fcae28dd640 1 -- 172.21.15.38:0/2883743330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcadc0d0320 msgr2=0x7fcadc0b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.816+0000 7fcae28dd640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcadc0d0320 0x7fcadc0b4200 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00c430 tx=0x7fcacc00c8f0 comp rx=0 tx=0).stop 2024-04-25T08:32:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.816+0000 7fcae28dd640 1 -- 172.21.15.38:0/2883743330 shutdown_connections 2024-04-25T08:32:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.816+0000 7fcae28dd640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcadc0d33d0 0x7fcadc0b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.816+0000 7fcae28dd640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcac806b4d0 0x7fcac806d990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.816+0000 7fcae28dd640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcadc0d1e90 0x7fcadc0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.816+0000 7fcae28dd640 1 --2- 172.21.15.38:0/2883743330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcadc0d0320 0x7fcadc0b4200 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.816+0000 7fcae28dd640 1 -- 172.21.15.38:0/2883743330 >> 172.21.15.38:0/2883743330 conn(0x7fcadc0668f0 msgr2=0x7fcadc0b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.816+0000 7fcae28dd640 1 -- 172.21.15.38:0/2883743330 shutdown_connections 2024-04-25T08:32:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.816+0000 7fcae28dd640 1 -- 172.21.15.38:0/2883743330 wait complete. 2024-04-25T08:32:39.825 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:32:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.939+0000 7f8c724a1640 1 Processor -- start 2024-04-25T08:32:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.939+0000 7f8c724a1640 1 -- start start 2024-04-25T08:32:39.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c724a1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c6c1612c0 0x7f8c6c0b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:39.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c724a1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8c6c0b6a00 0x7f8c6c0d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:39.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c724a1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6c0b7200 con 0x7f8c6c0d0870 2024-04-25T08:32:39.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c724a1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b74e0 con 0x7f8c6c0b6a00 2024-04-25T08:32:39.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c724a1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b77c0 con 0x7f8c6c1612c0 2024-04-25T08:32:39.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c6b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8c6c0b6a00 0x7f8c6c0d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:39.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c70a3e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f8c6c0d0870 0x7f8c6c0b82a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49540/0 (socket says 172.21.15.38:49540) 2024-04-25T08:32:39.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c70a3e640 1 -- 172.21.15.38:0/405468648 learned_addr learned my addr 172.21.15.38:0/405468648 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:39.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c6bfff640 1 --2- 172.21.15.38:0/405468648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c6c1612c0 0x7f8c6c0b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:39.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c6b7fe640 1 -- 172.21.15.38:0/405468648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c6c1612c0 msgr2=0x7f8c6c0b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:39.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c6b7fe640 1 --2- 172.21.15.38:0/405468648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c6c1612c0 0x7f8c6c0b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:39.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c6b7fe640 1 -- 172.21.15.38:0/405468648 >> v1:172.21.15.38:6789/0 conn(0x7f8c6c0d0870 legacy=0x7f8c6c0b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:39.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c6b7fe640 1 -- 172.21.15.38:0/405468648 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c0b7aa0 con 0x7f8c6c0b6a00 2024-04-25T08:32:39.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.940+0000 7f8c6bfff640 1 --2- 172.21.15.38:0/405468648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c6c1612c0 0x7f8c6c0b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:39.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.941+0000 7f8c6b7fe640 1 --2- 172.21.15.38:0/405468648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6c0b6a00 0x7f8c6c0d1c30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8c540097a0 tx=0x7f8c5402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a2775bbc0ae856 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:39.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.941+0000 7f8c6affd640 1 -- 172.21.15.38:0/405468648 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54037070 con 0x7f8c6c0b6a00 2024-04-25T08:32:39.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.941+0000 7f8c6affd640 1 -- 172.21.15.38:0/405468648 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5402ae50 con 0x7f8c6c0b6a00 2024-04-25T08:32:39.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.941+0000 7f8c6affd640 1 -- 172.21.15.38:0/405468648 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54032660 con 0x7f8c6c0b6a00 2024-04-25T08:32:39.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.942+0000 7f8c724a1640 1 -- 172.21.15.38:0/405468648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6c0b6a00 msgr2=0x7f8c6c0d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:39.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.942+0000 7f8c724a1640 1 --2- 172.21.15.38:0/405468648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6c0b6a00 0x7f8c6c0d1c30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8c540097a0 tx=0x7f8c5402a100 comp rx=0 tx=0).stop 2024-04-25T08:32:39.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.942+0000 7f8c724a1640 1 -- 172.21.15.38:0/405468648 shutdown_connections 2024-04-25T08:32:39.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.942+0000 7f8c724a1640 1 --2- 172.21.15.38:0/405468648 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6c0b6a00 0x7f8c6c0d1c30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:39.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.942+0000 7f8c724a1640 1 --2- 172.21.15.38:0/405468648 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c6c1612c0 0x7f8c6c0b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:39.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.942+0000 7f8c724a1640 1 -- 172.21.15.38:0/405468648 >> 172.21.15.38:0/405468648 conn(0x7f8c6c066d10 msgr2=0x7f8c6c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:39.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.942+0000 7f8c724a1640 1 -- 172.21.15.38:0/405468648 shutdown_connections 2024-04-25T08:32:39.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.942+0000 7f8c724a1640 1 -- 172.21.15.38:0/405468648 wait complete. 2024-04-25T08:32:39.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.943+0000 7f8c724a1640 1 Processor -- start 2024-04-25T08:32:39.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.943+0000 7f8c724a1640 1 -- start start 2024-04-25T08:32:39.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c724a1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c6c0b6a00 0x7f8c6c13ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:39.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c724a1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0d0870 0x7f8c6c154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:39.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c724a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6c1612c0 0x7f8c6c13f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:39.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c724a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b6e60 con 0x7f8c6c1612c0 2024-04-25T08:32:39.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c724a1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0d2170 con 0x7f8c6c0d0870 2024-04-25T08:32:39.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c724a1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b6fe0 con 0x7f8c6c0b6a00 2024-04-25T08:32:39.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c70a3e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6c1612c0 0x7f8c6c13f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:39.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c70a3e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6c1612c0 0x7f8c6c13f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41726/0 (socket says 172.21.15.38:41726) 2024-04-25T08:32:39.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c70a3e640 1 -- 172.21.15.38:0/2696388739 learned_addr learned my addr 172.21.15.38:0/2696388739 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c6bfff640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c6c0b6a00 0x7f8c6c13ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c6b7fe640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0d0870 0x7f8c6c154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c70a3e640 1 -- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c6c0b6a00 msgr2=0x7f8c6c13ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c70a3e640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c6c0b6a00 0x7f8c6c13ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c70a3e640 1 -- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0d0870 msgr2=0x7f8c6c154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c70a3e640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0d0870 0x7f8c6c154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.944+0000 7f8c70a3e640 1 -- 172.21.15.38:0/2696388739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c0d2e30 con 0x7f8c6c1612c0 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.945+0000 7f8c70a3e640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6c1612c0 0x7f8c6c13f7d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400cbc0 tx=0x7f8c6400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.945+0000 7f8c68ff9640 1 -- 172.21.15.38:0/2696388739 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64017070 con 0x7f8c6c1612c0 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.945+0000 7f8c68ff9640 1 -- 172.21.15.38:0/2696388739 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6400ad40 con 0x7f8c6c1612c0 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.945+0000 7f8c68ff9640 1 -- 172.21.15.38:0/2696388739 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64012600 con 0x7f8c6c1612c0 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.945+0000 7f8c724a1640 1 -- 172.21.15.38:0/2696388739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6c0d3110 con 0x7f8c6c1612c0 2024-04-25T08:32:39.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.945+0000 7f8c724a1640 1 -- 172.21.15.38:0/2696388739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c0d36d0 con 0x7f8c6c1612c0 2024-04-25T08:32:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.946+0000 7f8c724a1640 1 -- 172.21.15.38:0/2696388739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6c068e80 con 0x7f8c6c1612c0 2024-04-25T08:32:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.946+0000 7f8c68ff9640 1 -- 172.21.15.38:0/2696388739 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c6401a070 con 0x7f8c6c1612c0 2024-04-25T08:32:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.947+0000 7f8c68ff9640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c4c06b4d0 0x7f8c4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.947+0000 7f8c68ff9640 1 -- 172.21.15.38:0/2696388739 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6908+0+0 (secure 0 0 0) 0x7f8c6408a360 con 0x7f8c6c1612c0 2024-04-25T08:32:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.947+0000 7f8c6bfff640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c4c06b4d0 0x7f8c4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.947+0000 7f8c6bfff640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c4c06b4d0 0x7f8c4c06d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00aac0 tx=0x7f8c5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:39.951+0000 7f8c68ff9640 1 -- 172.21.15.38:0/2696388739 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c640586e0 con 0x7f8c6c1612c0 2024-04-25T08:32:40.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.130+0000 7f8c724a1640 1 -- 172.21.15.38:0/2696388739 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c6c15c4a0 con 0x7f8c6c1612c0 2024-04-25T08:32:40.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.816+0000 7f8c68ff9640 1 -- 172.21.15.38:0/2696388739 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8c6405c780 con 0x7f8c6c1612c0 2024-04-25T08:32:40.813 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:32:40.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.819+0000 7f8c724a1640 1 -- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c4c06b4d0 msgr2=0x7f8c4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:40.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.819+0000 7f8c724a1640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c4c06b4d0 0x7f8c4c06d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c00aac0 tx=0x7f8c5c008040 comp rx=0 tx=0).stop 2024-04-25T08:32:40.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.819+0000 7f8c724a1640 1 -- 172.21.15.38:0/2696388739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6c1612c0 msgr2=0x7f8c6c13f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:40.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.819+0000 7f8c724a1640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6c1612c0 0x7f8c6c13f7d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400cbc0 tx=0x7f8c6400a430 comp rx=0 tx=0).stop 2024-04-25T08:32:40.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.819+0000 7f8c724a1640 1 -- 172.21.15.38:0/2696388739 shutdown_connections 2024-04-25T08:32:40.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.819+0000 7f8c724a1640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c6c1612c0 0x7f8c6c13f7d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:40.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.819+0000 7f8c724a1640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c6c0d0870 0x7f8c6c154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:40.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.819+0000 7f8c724a1640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c4c06b4d0 0x7f8c4c06d990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:40.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.819+0000 7f8c724a1640 1 --2- 172.21.15.38:0/2696388739 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c6c0b6a00 0x7f8c6c13ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:40.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.819+0000 7f8c724a1640 1 -- 172.21.15.38:0/2696388739 >> 172.21.15.38:0/2696388739 conn(0x7f8c6c066d10 msgr2=0x7f8c6c15c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:40.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.819+0000 7f8c724a1640 1 -- 172.21.15.38:0/2696388739 shutdown_connections 2024-04-25T08:32:40.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.819+0000 7f8c724a1640 1 -- 172.21.15.38:0/2696388739 wait complete. 2024-04-25T08:32:40.828 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:32:40.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.944+0000 7fcef4891640 1 Processor -- start 2024-04-25T08:32:40.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.944+0000 7fcef4891640 1 -- start start 2024-04-25T08:32:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.945+0000 7fcef4891640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcef00d0860 0x7fcef00d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.945+0000 7fcef4891640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcef00d19e0 0x7fcef00d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.945+0000 7fcef4891640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcef00d3860 0x7fcef00d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.945+0000 7fcef4891640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef015e2b0 con 0x7fcef00d19e0 2024-04-25T08:32:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.945+0000 7fcef4891640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef00d1180 con 0x7fcef00d0860 2024-04-25T08:32:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.945+0000 7fcef4891640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00d24a0 con 0x7fcef00d3860 2024-04-25T08:32:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.945+0000 7fceedd9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcef00d19e0 0x7fcef00d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.945+0000 7fceedd9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcef00d19e0 0x7fcef00d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41732/0 (socket says 172.21.15.38:41732) 2024-04-25T08:32:40.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.945+0000 7fceedd9b640 1 -- 172.21.15.38:0/883033540 learned_addr learned my addr 172.21.15.38:0/883033540 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:40.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.946+0000 7fceee59c640 1 --2- 172.21.15.38:0/883033540 >> v2:172.21.15.107:3300/0 conn(0x7fcef00d0860 0x7fcef00d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:40.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.946+0000 7fceeed9d640 1 --2- 172.21.15.38:0/883033540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcef00d3860 0x7fcef00d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:40.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.946+0000 7fceedd9b640 1 -- 172.21.15.38:0/883033540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcef00d3860 msgr2=0x7fcef00d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:40.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.946+0000 7fceedd9b640 1 --2- 172.21.15.38:0/883033540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcef00d3860 0x7fcef00d3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:40.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.946+0000 7fceedd9b640 1 -- 172.21.15.38:0/883033540 >> v2:172.21.15.107:3300/0 conn(0x7fcef00d0860 msgr2=0x7fcef00d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:40.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.946+0000 7fceedd9b640 1 --2- 172.21.15.38:0/883033540 >> v2:172.21.15.107:3300/0 conn(0x7fcef00d0860 0x7fcef00d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:40.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.946+0000 7fceedd9b640 1 -- 172.21.15.38:0/883033540 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef00d2780 con 0x7fcef00d19e0 2024-04-25T08:32:40.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.946+0000 7fceeed9d640 1 --2- 172.21.15.38:0/883033540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcef00d3860 0x7fcef00d3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:40.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.946+0000 7fceedd9b640 1 --2- 172.21.15.38:0/883033540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef00d19e0 0x7fcef00d1e50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcef00ce120 tx=0x7fced802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b675248baf92fcd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.947+0000 7fceed59a640 1 -- 172.21.15.38:0/883033540 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8037070 con 0x7fcef00d19e0 2024-04-25T08:32:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.947+0000 7fceed59a640 1 -- 172.21.15.38:0/883033540 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced8032440 con 0x7fcef00d19e0 2024-04-25T08:32:40.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.947+0000 7fceed59a640 1 -- 172.21.15.38:0/883033540 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced8032810 con 0x7fcef00d19e0 2024-04-25T08:32:40.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.947+0000 7fcef4891640 1 -- 172.21.15.38:0/883033540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef00d19e0 msgr2=0x7fcef00d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:40.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.947+0000 7fcef4891640 1 --2- 172.21.15.38:0/883033540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef00d19e0 0x7fcef00d1e50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fcef00ce120 tx=0x7fced802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:40.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.947+0000 7fcef4891640 1 -- 172.21.15.38:0/883033540 shutdown_connections 2024-04-25T08:32:40.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.947+0000 7fcef4891640 1 --2- 172.21.15.38:0/883033540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcef00d3860 0x7fcef00d3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:40.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.947+0000 7fcef4891640 1 --2- 172.21.15.38:0/883033540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef00d19e0 0x7fcef00d1e50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:40.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.947+0000 7fcef4891640 1 --2- 172.21.15.38:0/883033540 >> v2:172.21.15.107:3300/0 conn(0x7fcef00d0860 0x7fcef00d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:40.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.947+0000 7fcef4891640 1 -- 172.21.15.38:0/883033540 >> 172.21.15.38:0/883033540 conn(0x7fcef0066d10 msgr2=0x7fcef0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:40.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.947+0000 7fcef4891640 1 -- 172.21.15.38:0/883033540 shutdown_connections 2024-04-25T08:32:40.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.947+0000 7fcef4891640 1 -- 172.21.15.38:0/883033540 wait complete. 2024-04-25T08:32:40.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.949+0000 7fcef4891640 1 Processor -- start 2024-04-25T08:32:40.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.949+0000 7fcef4891640 1 -- start start 2024-04-25T08:32:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.949+0000 7fcef4891640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcef00d0860 0x7fcef014b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.949+0000 7fcef4891640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcef00d19e0 0x7fcef01f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.949+0000 7fcef4891640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef00d3860 0x7fcef0149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.949+0000 7fcef4891640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00d3540 con 0x7fcef00d3860 2024-04-25T08:32:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.949+0000 7fcef4891640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00d2d40 con 0x7fcef00d0860 2024-04-25T08:32:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.949+0000 7fcef4891640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef00b9770 con 0x7fcef00d19e0 2024-04-25T08:32:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.949+0000 7fceeed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef00d3860 0x7fcef0149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.949+0000 7fceeed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef00d3860 0x7fcef0149a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41746/0 (socket says 172.21.15.38:41746) 2024-04-25T08:32:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.949+0000 7fceeed9d640 1 -- 172.21.15.38:0/3747606873 learned_addr learned my addr 172.21.15.38:0/3747606873 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fceee59c640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcef00d0860 0x7fcef014b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fceedd9b640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcef00d19e0 0x7fcef01f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fceeed9d640 1 -- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcef00d19e0 msgr2=0x7fcef01f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fceeed9d640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcef00d19e0 0x7fcef01f3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fceeed9d640 1 -- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcef00d0860 msgr2=0x7fcef014b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fceeed9d640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcef00d0860 0x7fcef014b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fceeed9d640 1 -- 172.21.15.38:0/3747606873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef014a350 con 0x7fcef00d3860 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fceee59c640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcef00d0860 0x7fcef014b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fceeed9d640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef00d3860 0x7fcef0149a20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcee000cd50 tx=0x7fcee000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fcee6ffd640 1 -- 172.21.15.38:0/3747606873 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee000f680 con 0x7fcef00d3860 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fcee6ffd640 1 -- 172.21.15.38:0/3747606873 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee000f840 con 0x7fcef00d3860 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fcef4891640 1 -- 172.21.15.38:0/3747606873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef014a630 con 0x7fcef00d3860 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fcee6ffd640 1 -- 172.21.15.38:0/3747606873 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee000fc30 con 0x7fcef00d3860 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.950+0000 7fcef4891640 1 -- 172.21.15.38:0/3747606873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef014abf0 con 0x7fcef00d3860 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.951+0000 7fcef4891640 1 -- 172.21.15.38:0/3747606873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef0068e80 con 0x7fcef00d3860 2024-04-25T08:32:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.952+0000 7fcee6ffd640 1 -- 172.21.15.38:0/3747606873 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcee00213f0 con 0x7fcef00d3860 2024-04-25T08:32:40.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.952+0000 7fcee6ffd640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fced006b4d0 0x7fced006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:40.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.952+0000 7fcee6ffd640 1 -- 172.21.15.38:0/3747606873 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6470+0+0 (secure 0 0 0) 0x7fcee008b2e0 con 0x7fcef00d3860 2024-04-25T08:32:40.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.952+0000 7fceee59c640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fced006b4d0 0x7fced006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:40.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.953+0000 7fceee59c640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fced006b4d0 0x7fced006d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcee8006fd0 tx=0x7fcee8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:40.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:40.956+0000 7fcee6ffd640 1 -- 172.21.15.38:0/3747606873 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcee0014030 con 0x7fcef00d3860 2024-04-25T08:32:41.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.139+0000 7fcef4891640 1 -- 172.21.15.38:0/3747606873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcef006af00 con 0x7fcef00d3860 2024-04-25T08:32:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.822+0000 7fcee6ffd640 1 -- 172.21.15.38:0/3747606873 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcee0059810 con 0x7fcef00d3860 2024-04-25T08:32:41.818 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-04-25T08:32:41.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.825+0000 7fcef4891640 1 -- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fced006b4d0 msgr2=0x7fced006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:41.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.825+0000 7fcef4891640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fced006b4d0 0x7fced006d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcee8006fd0 tx=0x7fcee8008040 comp rx=0 tx=0).stop 2024-04-25T08:32:41.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.825+0000 7fcef4891640 1 -- 172.21.15.38:0/3747606873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef00d3860 msgr2=0x7fcef0149a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:41.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.825+0000 7fcef4891640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef00d3860 0x7fcef0149a20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcee000cd50 tx=0x7fcee000a650 comp rx=0 tx=0).stop 2024-04-25T08:32:41.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.825+0000 7fcef4891640 1 -- 172.21.15.38:0/3747606873 shutdown_connections 2024-04-25T08:32:41.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.825+0000 7fcef4891640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcef00d3860 0x7fcef0149a20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:41.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.825+0000 7fcef4891640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fced006b4d0 0x7fced006d990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:41.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.825+0000 7fcef4891640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcef00d19e0 0x7fcef01f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:41.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.825+0000 7fcef4891640 1 --2- 172.21.15.38:0/3747606873 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcef00d0860 0x7fcef014b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:41.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.825+0000 7fcef4891640 1 -- 172.21.15.38:0/3747606873 >> 172.21.15.38:0/3747606873 conn(0x7fcef0066d10 msgr2=0x7fcef00670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:41.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.825+0000 7fcef4891640 1 -- 172.21.15.38:0/3747606873 shutdown_connections 2024-04-25T08:32:41.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.825+0000 7fcef4891640 1 -- 172.21.15.38:0/3747606873 wait complete. 2024-04-25T08:32:41.837 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T08:32:41.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.952+0000 7fc836163640 1 Processor -- start 2024-04-25T08:32:41.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.952+0000 7fc836163640 1 -- start start 2024-04-25T08:32:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc836163640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc8300d0860 0x7fc8300d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc836163640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc8300d19e0 0x7fc8300d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc836163640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8300d1180 con 0x7fc8300d3860 2024-04-25T08:32:41.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc836163640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83015e2b0 con 0x7fc8300d19e0 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc836163640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8300d24a0 con 0x7fc8300d0860 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc82effd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc8300d19e0 0x7fc8300d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc82ffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc8300d3860 0x7fc8300d3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49546/0 (socket says 172.21.15.38:49546) 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc82ffff640 1 -- 172.21.15.38:0/3144765364 learned_addr learned my addr 172.21.15.38:0/3144765364 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc82f7fe640 1 --2- 172.21.15.38:0/3144765364 >> v2:172.21.15.107:3300/0 conn(0x7fc8300d0860 0x7fc8300d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc82effd640 1 -- 172.21.15.38:0/3144765364 >> v2:172.21.15.107:3300/0 conn(0x7fc8300d0860 msgr2=0x7fc8300d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc82effd640 1 --2- 172.21.15.38:0/3144765364 >> v2:172.21.15.107:3300/0 conn(0x7fc8300d0860 0x7fc8300d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc82effd640 1 -- 172.21.15.38:0/3144765364 >> v1:172.21.15.38:6789/0 conn(0x7fc8300d3860 legacy=0x7fc8300d3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc82effd640 1 -- 172.21.15.38:0/3144765364 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8300d2780 con 0x7fc8300d19e0 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc82e7fc640 1 -- 172.21.15.38:0/3144765364 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469906831 0 0) 0x7fc8300d1180 con 0x7fc8300d3860 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.953+0000 7fc82f7fe640 1 --2- 172.21.15.38:0/3144765364 >> v2:172.21.15.107:3300/0 conn(0x7fc8300d0860 0x7fc8300d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.954+0000 7fc82effd640 1 --2- 172.21.15.38:0/3144765364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8300d19e0 0x7fc8300d1e50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc8300ce120 tx=0x7fc82402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffe413ebae436cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.954+0000 7fc82e7fc640 1 -- 172.21.15.38:0/3144765364 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824037070 con 0x7fc8300d19e0 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.954+0000 7fc82e7fc640 1 -- 172.21.15.38:0/3144765364 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc824032440 con 0x7fc8300d19e0 2024-04-25T08:32:41.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.954+0000 7fc82e7fc640 1 -- 172.21.15.38:0/3144765364 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc824032810 con 0x7fc8300d19e0 2024-04-25T08:32:41.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.955+0000 7fc836163640 1 -- 172.21.15.38:0/3144765364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8300d19e0 msgr2=0x7fc8300d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:41.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.955+0000 7fc836163640 1 --2- 172.21.15.38:0/3144765364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8300d19e0 0x7fc8300d1e50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc8300ce120 tx=0x7fc82402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:41.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.955+0000 7fc836163640 1 -- 172.21.15.38:0/3144765364 shutdown_connections 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.955+0000 7fc836163640 1 --2- 172.21.15.38:0/3144765364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8300d19e0 0x7fc8300d1e50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.955+0000 7fc836163640 1 --2- 172.21.15.38:0/3144765364 >> v2:172.21.15.107:3300/0 conn(0x7fc8300d0860 0x7fc8300d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.955+0000 7fc836163640 1 -- 172.21.15.38:0/3144765364 >> 172.21.15.38:0/3144765364 conn(0x7fc830066d10 msgr2=0x7fc830067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.955+0000 7fc836163640 1 -- 172.21.15.38:0/3144765364 shutdown_connections 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.955+0000 7fc836163640 1 -- 172.21.15.38:0/3144765364 wait complete. 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.956+0000 7fc836163640 1 Processor -- start 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.956+0000 7fc836163640 1 -- start start 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.957+0000 7fc836163640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8300d0860 0x7fc8300c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.957+0000 7fc836163640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8300d19e0 0x7fc8301e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.957+0000 7fc836163640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8300d3860 0x7fc83015b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.957+0000 7fc836163640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300d3540 con 0x7fc8300d3860 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.957+0000 7fc836163640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300d2d40 con 0x7fc8300d0860 2024-04-25T08:32:41.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.957+0000 7fc836163640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300b9770 con 0x7fc8300d19e0 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.957+0000 7fc82ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8300d3860 0x7fc83015b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.957+0000 7fc82ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8300d3860 0x7fc83015b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41760/0 (socket says 172.21.15.38:41760) 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.957+0000 7fc82ffff640 1 -- 172.21.15.38:0/3252420760 learned_addr learned my addr 172.21.15.38:0/3252420760 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.957+0000 7fc82f7fe640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8300d0860 0x7fc8300c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.957+0000 7fc82effd640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8300d19e0 0x7fc8301e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.958+0000 7fc82ffff640 1 -- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8300d19e0 msgr2=0x7fc8301e71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.958+0000 7fc82ffff640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8300d19e0 0x7fc8301e71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.958+0000 7fc82ffff640 1 -- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8300d0860 msgr2=0x7fc8300c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.958+0000 7fc82ffff640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8300d0860 0x7fc8300c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.958+0000 7fc82ffff640 1 -- 172.21.15.38:0/3252420760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8300ca2d0 con 0x7fc8300d3860 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.958+0000 7fc82ffff640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8300d3860 0x7fc83015b180 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc82800c700 tx=0x7fc82800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.958+0000 7fc817fff640 1 -- 172.21.15.38:0/3252420760 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828018070 con 0x7fc8300d3860 2024-04-25T08:32:41.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.958+0000 7fc817fff640 1 -- 172.21.15.38:0/3252420760 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82800ce10 con 0x7fc8300d3860 2024-04-25T08:32:41.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.958+0000 7fc836163640 1 -- 172.21.15.38:0/3252420760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8300c9ff0 con 0x7fc8300d3860 2024-04-25T08:32:41.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.958+0000 7fc817fff640 1 -- 172.21.15.38:0/3252420760 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800a880 con 0x7fc8300d3860 2024-04-25T08:32:41.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.958+0000 7fc836163640 1 -- 172.21.15.38:0/3252420760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8300c7810 con 0x7fc8300d3860 2024-04-25T08:32:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.959+0000 7fc836163640 1 -- 172.21.15.38:0/3252420760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc830068e80 con 0x7fc8300d3860 2024-04-25T08:32:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.959+0000 7fc817fff640 1 -- 172.21.15.38:0/3252420760 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc82801b070 con 0x7fc8300d3860 2024-04-25T08:32:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.960+0000 7fc817fff640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc81c06b4d0 0x7fc81c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.960+0000 7fc817fff640 1 -- 172.21.15.38:0/3252420760 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc828089790 con 0x7fc8300d3860 2024-04-25T08:32:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.960+0000 7fc82f7fe640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc81c06b4d0 0x7fc81c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:41.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.961+0000 7fc82f7fe640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc81c06b4d0 0x7fc81c06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc820006fd0 tx=0x7fc820008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:41.964+0000 7fc817fff640 1 -- 172.21.15.38:0/3252420760 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc828058610 con 0x7fc8300d3860 2024-04-25T08:32:42.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.145+0000 7fc836163640 1 -- 172.21.15.38:0/3252420760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc8300cf330 con 0x7fc8300d3860 2024-04-25T08:32:42.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.823+0000 7fc817fff640 1 -- 172.21.15.38:0/3252420760 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7fc828057f10 con 0x7fc8300d3860 2024-04-25T08:32:42.820 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data_ec' removed 2024-04-25T08:32:42.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.826+0000 7fc836163640 1 -- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc81c06b4d0 msgr2=0x7fc81c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.826+0000 7fc836163640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc81c06b4d0 0x7fc81c06d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc820006fd0 tx=0x7fc820008040 comp rx=0 tx=0).stop 2024-04-25T08:32:42.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.826+0000 7fc836163640 1 -- 172.21.15.38:0/3252420760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8300d3860 msgr2=0x7fc83015b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:42.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.826+0000 7fc836163640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8300d3860 0x7fc83015b180 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc82800c700 tx=0x7fc82800cbc0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.826+0000 7fc836163640 1 -- 172.21.15.38:0/3252420760 shutdown_connections 2024-04-25T08:32:42.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.826+0000 7fc836163640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8300d3860 0x7fc83015b180 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.826+0000 7fc836163640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc81c06b4d0 0x7fc81c06d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.826+0000 7fc836163640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8300d19e0 0x7fc8301e71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.826+0000 7fc836163640 1 --2- 172.21.15.38:0/3252420760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8300d0860 0x7fc8300c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:42.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.826+0000 7fc836163640 1 -- 172.21.15.38:0/3252420760 >> 172.21.15.38:0/3252420760 conn(0x7fc830066d10 msgr2=0x7fc8300b4b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:42.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.826+0000 7fc836163640 1 -- 172.21.15.38:0/3252420760 shutdown_connections 2024-04-25T08:32:42.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:42.826+0000 7fc836163640 1 -- 172.21.15.38:0/3252420760 wait complete. 2024-04-25T08:32:42.835 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:32:42.836 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:32:42.836 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:32:42.836 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:42.836 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:32:42.836 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:32:42.836 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:32:42.837 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:42.837 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:32:42.837 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:32:42.837 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:32:42.837 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:42.837 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:32:42.837 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:32:42.838 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:32:42.838 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:32:42.847 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:32:42.847 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:32:42.848 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:32:42.848 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:32:42.849 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:32:42.849 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:32:42.850 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:32:42.850 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:32:42.889 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:32:42.889 DEBUG:teuthology.parallel:result is None 2024-04-25T08:32:42.890 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:32:42.891 DEBUG:teuthology.parallel:result is None 2024-04-25T08:32:42.891 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:32:42.892 DEBUG:teuthology.parallel:result is None 2024-04-25T08:32:42.892 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:32:42.892 DEBUG:teuthology.parallel:result is None 2024-04-25T08:32:42.892 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:32:42.968 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-25T08:32:42.969 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:32:42.969 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:32:42.970 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:32:43.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.006+0000 7f09dab70640 1 Processor -- start 2024-04-25T08:32:43.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.007+0000 7f09dab70640 1 -- start start 2024-04-25T08:32:43.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.007+0000 7f09dab70640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09d406b4d0 0x7f09d40b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.007+0000 7f09dab70640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f09d40b1a90 0x7f09d40cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.007+0000 7f09dab70640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f09d40cfc90 0x7f09d41616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.007+0000 7f09dab70640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d40afc20 con 0x7f09d40b1a90 2024-04-25T08:32:43.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.007+0000 7f09dab70640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d40aff00 con 0x7f09d40cfc90 2024-04-25T08:32:43.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.007+0000 7f09dab70640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d40b01e0 con 0x7f09d406b4d0 2024-04-25T08:32:43.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.007+0000 7f09cbfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f09d40b1a90 0x7f09d40cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.008+0000 7f09cbfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f09d40b1a90 0x7f09d40cf750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41804/0 (socket says 172.21.15.38:41804) 2024-04-25T08:32:43.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.008+0000 7f09cbfff640 1 -- 172.21.15.38:0/2154785448 learned_addr learned my addr 172.21.15.38:0/2154785448 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:43.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.008+0000 7f09d890c640 1 --2- 172.21.15.38:0/2154785448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09d406b4d0 0x7f09d40b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.008+0000 7f09d910d640 1 --2- 172.21.15.38:0/2154785448 >> v2:172.21.15.107:3300/0 conn(0x7f09d40cfc90 0x7f09d41616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.008+0000 7f09cbfff640 1 -- 172.21.15.38:0/2154785448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09d406b4d0 msgr2=0x7f09d40b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.008+0000 7f09cbfff640 1 --2- 172.21.15.38:0/2154785448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09d406b4d0 0x7f09d40b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.008+0000 7f09cbfff640 1 -- 172.21.15.38:0/2154785448 >> v2:172.21.15.107:3300/0 conn(0x7f09d40cfc90 msgr2=0x7f09d41616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:43.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.008+0000 7f09cbfff640 1 --2- 172.21.15.38:0/2154785448 >> v2:172.21.15.107:3300/0 conn(0x7f09d40cfc90 0x7f09d41616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.008+0000 7f09cbfff640 1 -- 172.21.15.38:0/2154785448 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d40b04c0 con 0x7f09d40b1a90 2024-04-25T08:32:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.008+0000 7f09cbfff640 1 --2- 172.21.15.38:0/2154785448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d40b1a90 0x7f09d40cf750 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f09d41615f0 tx=0x7f09c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ea78fae5b0ca55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.009+0000 7f09cb7fe640 1 -- 172.21.15.38:0/2154785448 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4037070 con 0x7f09d40b1a90 2024-04-25T08:32:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.009+0000 7f09cb7fe640 1 -- 172.21.15.38:0/2154785448 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c4032440 con 0x7f09d40b1a90 2024-04-25T08:32:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.009+0000 7f09cb7fe640 1 -- 172.21.15.38:0/2154785448 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c4032810 con 0x7f09d40b1a90 2024-04-25T08:32:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.009+0000 7f09dab70640 1 -- 172.21.15.38:0/2154785448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d40b1a90 msgr2=0x7f09d40cf750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.009+0000 7f09dab70640 1 --2- 172.21.15.38:0/2154785448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d40b1a90 0x7f09d40cf750 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f09d41615f0 tx=0x7f09c402a270 comp rx=0 tx=0).stop 2024-04-25T08:32:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.009+0000 7f09dab70640 1 -- 172.21.15.38:0/2154785448 shutdown_connections 2024-04-25T08:32:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.009+0000 7f09dab70640 1 --2- 172.21.15.38:0/2154785448 >> v2:172.21.15.107:3300/0 conn(0x7f09d40cfc90 0x7f09d41616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.009+0000 7f09dab70640 1 --2- 172.21.15.38:0/2154785448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d40b1a90 0x7f09d40cf750 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.009+0000 7f09dab70640 1 --2- 172.21.15.38:0/2154785448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09d406b4d0 0x7f09d40b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.009+0000 7f09dab70640 1 -- 172.21.15.38:0/2154785448 >> 172.21.15.38:0/2154785448 conn(0x7f09d40b1ff0 msgr2=0x7f09d40b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.009+0000 7f09dab70640 1 -- 172.21.15.38:0/2154785448 shutdown_connections 2024-04-25T08:32:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.009+0000 7f09dab70640 1 -- 172.21.15.38:0/2154785448 wait complete. 2024-04-25T08:32:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.010+0000 7f09dab70640 1 Processor -- start 2024-04-25T08:32:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.011+0000 7f09dab70640 1 -- start start 2024-04-25T08:32:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.011+0000 7f09dab70640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09d406b4d0 0x7f09d40b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.011+0000 7f09dab70640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09d40b1a90 0x7f09d415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.011+0000 7f09dab70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d40cfc90 0x7f09d40c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.011+0000 7f09dab70640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d40d0120 con 0x7f09d40cfc90 2024-04-25T08:32:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.011+0000 7f09dab70640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d40b0a80 con 0x7f09d40b1a90 2024-04-25T08:32:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.011+0000 7f09dab70640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d40b0fa0 con 0x7f09d406b4d0 2024-04-25T08:32:43.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09d910d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d40cfc90 0x7f09d40c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09d910d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d40cfc90 0x7f09d40c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41814/0 (socket says 172.21.15.38:41814) 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09d910d640 1 -- 172.21.15.38:0/399428475 learned_addr learned my addr 172.21.15.38:0/399428475 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09cbfff640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09d40b1a90 0x7f09d415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09d890c640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09d406b4d0 0x7f09d40b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09d910d640 1 -- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09d406b4d0 msgr2=0x7f09d40b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09d910d640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09d406b4d0 0x7f09d40b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09d910d640 1 -- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09d40b1a90 msgr2=0x7f09d415a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09d910d640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09d40b1a90 0x7f09d415a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09d910d640 1 -- 172.21.15.38:0/399428475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d40b4af0 con 0x7f09d40cfc90 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09cbfff640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09d40b1a90 0x7f09d415a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09d910d640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d40cfc90 0x7f09d40c6050 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00efc0 tx=0x7f09cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09c97fa640 1 -- 172.21.15.38:0/399428475 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc018020 con 0x7f09d40cfc90 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09c97fa640 1 -- 172.21.15.38:0/399428475 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc014480 con 0x7f09d40cfc90 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09c97fa640 1 -- 172.21.15.38:0/399428475 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc014870 con 0x7f09d40cfc90 2024-04-25T08:32:43.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09dab70640 1 -- 172.21.15.38:0/399428475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d40b6080 con 0x7f09d40cfc90 2024-04-25T08:32:43.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.012+0000 7f09dab70640 1 -- 172.21.15.38:0/399428475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d40b5050 con 0x7f09d40cfc90 2024-04-25T08:32:43.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.014+0000 7f09dab70640 1 -- 172.21.15.38:0/399428475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d404ef90 con 0x7f09d40cfc90 2024-04-25T08:32:43.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.014+0000 7f09c97fa640 1 -- 172.21.15.38:0/399428475 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09cc005d10 con 0x7f09d40cfc90 2024-04-25T08:32:43.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.014+0000 7f09c97fa640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f09c006b4d0 0x7f09c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.015+0000 7f09c97fa640 1 -- 172.21.15.38:0/399428475 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f09cc08aa00 con 0x7f09d40cfc90 2024-04-25T08:32:43.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.015+0000 7f09d890c640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f09c006b4d0 0x7f09c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.015+0000 7f09d890c640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f09c006b4d0 0x7f09c006d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009ac0 tx=0x7f09d0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.019+0000 7f09c97fa640 1 -- 172.21.15.38:0/399428475 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09cc022070 con 0x7f09d40cfc90 2024-04-25T08:32:43.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.220+0000 7f09dab70640 1 -- 172.21.15.38:0/399428475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f09d40b6460 con 0x7f09d40cfc90 2024-04-25T08:32:43.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.858+0000 7f09c97fa640 1 -- 172.21.15.38:0/399428475 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7f09d40b4af0 con 0x7f09d40cfc90 2024-04-25T08:32:43.854 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-04-25T08:32:43.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.860+0000 7f09dab70640 1 -- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f09c006b4d0 msgr2=0x7f09c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.860+0000 7f09dab70640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f09c006b4d0 0x7f09c006d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009ac0 tx=0x7f09d0009190 comp rx=0 tx=0).stop 2024-04-25T08:32:43.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.860+0000 7f09dab70640 1 -- 172.21.15.38:0/399428475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d40cfc90 msgr2=0x7f09d40c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.860+0000 7f09dab70640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d40cfc90 0x7f09d40c6050 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f09cc00efc0 tx=0x7f09cc00c490 comp rx=0 tx=0).stop 2024-04-25T08:32:43.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.860+0000 7f09dab70640 1 -- 172.21.15.38:0/399428475 shutdown_connections 2024-04-25T08:32:43.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.860+0000 7f09dab70640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09d40cfc90 0x7f09d40c6050 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.860+0000 7f09dab70640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09d40b1a90 0x7f09d415a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.860+0000 7f09dab70640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f09c006b4d0 0x7f09c006d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.860+0000 7f09dab70640 1 --2- 172.21.15.38:0/399428475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f09d406b4d0 0x7f09d40b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.860+0000 7f09dab70640 1 -- 172.21.15.38:0/399428475 >> 172.21.15.38:0/399428475 conn(0x7f09d40b1ff0 msgr2=0x7f09d406f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:43.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.860+0000 7f09dab70640 1 -- 172.21.15.38:0/399428475 shutdown_connections 2024-04-25T08:32:43.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.860+0000 7f09dab70640 1 -- 172.21.15.38:0/399428475 wait complete. 2024-04-25T08:32:43.872 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:32:43.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.983+0000 7fea02c9e640 1 Processor -- start 2024-04-25T08:32:43.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.983+0000 7fea02c9e640 1 -- start start 2024-04-25T08:32:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fea02c9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc15ddf0 0x7fe9fc161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fea02c9e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9fc161e00 con 0x7fe9fc0b5710 2024-04-25T08:32:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fea02c9e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9fc1620e0 con 0x7fe9fc0b93f0 2024-04-25T08:32:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fea02c9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc1623c0 con 0x7fe9fc15ddf0 2024-04-25T08:32:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fea00a3a640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe9fc0b5710 0x7fe9fc0b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49574/0 (socket says 172.21.15.38:49574) 2024-04-25T08:32:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fea00a3a640 1 -- 172.21.15.38:0/3839347659 learned_addr learned my addr 172.21.15.38:0/3839347659 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fe9fbfff640 1 --2- 172.21.15.38:0/3839347659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc15ddf0 0x7fe9fc161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fe9fb7fe640 1 -- 172.21.15.38:0/3839347659 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725421482 0 0) 0x7fe9fc161e00 con 0x7fe9fc0b5710 2024-04-25T08:32:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fe9fb7fe640 1 -- 172.21.15.38:0/3839347659 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9ec003520 con 0x7fe9fc0b5710 2024-04-25T08:32:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fe9fb7fe640 1 -- 172.21.15.38:0/3839347659 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557949931 0 0) 0x7fe9fc1620e0 con 0x7fe9fc0b93f0 2024-04-25T08:32:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fe9fbfff640 1 -- 172.21.15.38:0/3839347659 >> v1:172.21.15.107:6789/0 conn(0x7fe9fc0b93f0 legacy=0x7fe9fc0d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fe9fbfff640 1 -- 172.21.15.38:0/3839347659 >> v1:172.21.15.38:6789/0 conn(0x7fe9fc0b5710 legacy=0x7fe9fc0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.984+0000 7fe9fbfff640 1 -- 172.21.15.38:0/3839347659 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc0b9870 con 0x7fe9fc15ddf0 2024-04-25T08:32:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.985+0000 7fe9fbfff640 1 --2- 172.21.15.38:0/3839347659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc15ddf0 0x7fe9fc161700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80097d0 tx=0x7fe9e802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcf883260af5ce22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.985+0000 7fe9fb7fe640 1 -- 172.21.15.38:0/3839347659 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8037070 con 0x7fe9fc15ddf0 2024-04-25T08:32:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.985+0000 7fe9fb7fe640 1 -- 172.21.15.38:0/3839347659 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e8032440 con 0x7fe9fc15ddf0 2024-04-25T08:32:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.985+0000 7fe9fb7fe640 1 -- 172.21.15.38:0/3839347659 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8032810 con 0x7fe9fc15ddf0 2024-04-25T08:32:43.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.986+0000 7fea02c9e640 1 -- 172.21.15.38:0/3839347659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc15ddf0 msgr2=0x7fe9fc161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.986+0000 7fea02c9e640 1 --2- 172.21.15.38:0/3839347659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc15ddf0 0x7fe9fc161700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9e80097d0 tx=0x7fe9e802a210 comp rx=0 tx=0).stop 2024-04-25T08:32:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.986+0000 7fea02c9e640 1 -- 172.21.15.38:0/3839347659 shutdown_connections 2024-04-25T08:32:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.986+0000 7fea02c9e640 1 --2- 172.21.15.38:0/3839347659 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc15ddf0 0x7fe9fc161700 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.986+0000 7fea02c9e640 1 -- 172.21.15.38:0/3839347659 >> 172.21.15.38:0/3839347659 conn(0x7fe9fc0d07c0 msgr2=0x7fe9fc0d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.986+0000 7fea02c9e640 1 -- 172.21.15.38:0/3839347659 shutdown_connections 2024-04-25T08:32:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.986+0000 7fea02c9e640 1 -- 172.21.15.38:0/3839347659 wait complete. 2024-04-25T08:32:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.987+0000 7fea02c9e640 1 Processor -- start 2024-04-25T08:32:43.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.987+0000 7fea02c9e640 1 -- start start 2024-04-25T08:32:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.988+0000 7fea02c9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0b5710 0x7fe9fc0aa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.988+0000 7fea02c9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b93f0 0x7fe9fc0aa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.988+0000 7fea02c9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc15ddf0 0x7fe9fc15faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.988+0000 7fea02c9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0d2e60 con 0x7fe9fc0b93f0 2024-04-25T08:32:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.988+0000 7fea02c9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0d2ce0 con 0x7fe9fc15ddf0 2024-04-25T08:32:43.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.988+0000 7fea02c9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0d2fe0 con 0x7fe9fc0b5710 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.988+0000 7fe9fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b93f0 0x7fe9fc0aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.988+0000 7fe9fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b93f0 0x7fe9fc0aa840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41816/0 (socket says 172.21.15.38:41816) 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.988+0000 7fe9fbfff640 1 -- 172.21.15.38:0/3357293758 learned_addr learned my addr 172.21.15.38:0/3357293758 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.988+0000 7fea00a3a640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0b5710 0x7fe9fc0aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.988+0000 7fea0123b640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc15ddf0 0x7fe9fc15faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.989+0000 7fe9fbfff640 1 -- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0b5710 msgr2=0x7fe9fc0aa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.989+0000 7fe9fbfff640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0b5710 0x7fe9fc0aa300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.989+0000 7fe9fbfff640 1 -- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc15ddf0 msgr2=0x7fe9fc15faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.989+0000 7fe9fbfff640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc15ddf0 0x7fe9fc15faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.989+0000 7fe9fbfff640 1 -- 172.21.15.38:0/3357293758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc0cb030 con 0x7fe9fc0b93f0 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.989+0000 7fe9fbfff640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b93f0 0x7fe9fc0aa840 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000c00 tx=0x7fe9e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.989+0000 7fe9f97fa640 1 -- 172.21.15.38:0/3357293758 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8037050 con 0x7fe9fc0b93f0 2024-04-25T08:32:43.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.989+0000 7fe9f97fa640 1 -- 172.21.15.38:0/3357293758 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e8043070 con 0x7fe9fc0b93f0 2024-04-25T08:32:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.989+0000 7fea02c9e640 1 -- 172.21.15.38:0/3357293758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc0cb310 con 0x7fe9fc0b93f0 2024-04-25T08:32:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.989+0000 7fe9f97fa640 1 -- 172.21.15.38:0/3357293758 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e803cb10 con 0x7fe9fc0b93f0 2024-04-25T08:32:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.989+0000 7fea02c9e640 1 -- 172.21.15.38:0/3357293758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc0cb8d0 con 0x7fe9fc0b93f0 2024-04-25T08:32:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.990+0000 7fea02c9e640 1 -- 172.21.15.38:0/3357293758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9fc072e40 con 0x7fe9fc0b93f0 2024-04-25T08:32:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.991+0000 7fe9f97fa640 1 -- 172.21.15.38:0/3357293758 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9e803ccb0 con 0x7fe9fc0b93f0 2024-04-25T08:32:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.991+0000 7fe9f97fa640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9e006b4d0 0x7fe9e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.991+0000 7fe9f97fa640 1 -- 172.21.15.38:0/3357293758 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe9e80aac40 con 0x7fe9fc0b93f0 2024-04-25T08:32:43.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.991+0000 7fea00a3a640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9e006b4d0 0x7fe9e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:43.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.992+0000 7fea00a3a640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9e006b4d0 0x7fe9e006d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0005880 tx=0x7fe9f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:43.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:43.996+0000 7fe9f97fa640 1 -- 172.21.15.38:0/3357293758 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9fc072e40 con 0x7fe9fc0b93f0 2024-04-25T08:32:44.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.245+0000 7fea02c9e640 1 -- 172.21.15.38:0/3357293758 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe9fc068e80 con 0x7fe9fc0b93f0 2024-04-25T08:32:44.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.245+0000 7fe9f97fa640 1 -- 172.21.15.38:0/3357293758 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fe9e80796d0 con 0x7fe9fc0b93f0 2024-04-25T08:32:44.241 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:44.241 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:32:44.241 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-04-25T08:32:44.241 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.241 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-04-25T08:32:44.241 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmwq8CDBAA+cjd+QXtFgwZ3kxdABRWlA==", 2024-04-25T08:32:44.242 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.242 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:32:44.242 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:32:44.242 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:32:44.242 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.242 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.242 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.242 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.242 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-04-25T08:32:44.242 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmPRFbFxAAojn2r/62vvf1qTy9QVuyIw==", 2024-04-25T08:32:44.242 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmGLjqEhAAyPRgyy5iMgtccapydv4uOQ==", 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.243 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmt0AUHhAAoQyQ7wXBMGE+7Hi84tUfAg==", 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCxFCpmv+iTExAAEJihmsvBbp9xXxPnj49D6g==", 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.244 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC0FCpmdnQKFBAADLAsYZEUhoJx292jIYZroQ==", 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.245 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC3FCpm4f5BExAAMVSPkqsrnuLcO88AXKrGHQ==", 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC6FCpmHNN7EhAABPhmnxGo70ZOd8Oex62Mdw==", 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.246 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDAFCpmiThGFhAAG5Pyvn/JzL9NUoxHJkPMvg==", 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDDFCpmb/FHFBAA2cMhMqQQc4Fk6j8FpjhnaQ==", 2024-04-25T08:32:44.247 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDGFCpmAZsSEhAAnGIqqcvuHMVSc8Zbk309kA==", 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.248 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDJFCpmqdVUEBAAb9abblyUztrjwxxe4jsAwQ==", 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw==", 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:32:44.249 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmL7eKKhAASeQEdHDBInfozdLVzP4DQw==", 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.250 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmH22HJxAAWyrooyJn5hRs/bb0Rn1HHw==", 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpm6Zr/LxAAxh9Y2gvvqj1jIaVVrQAvgw==", 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.251 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCpFCpmlj/fLBAAvxLluFQFR8uhppGlNpaygA==", 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.252 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-mds", 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDPFCpmcU9YBRAA07+HhoASVnBQBXcTfhSV4Q==", 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDPFCpmcldYBRAArzjIC+SfXn5IZufJ3iqqYg==", 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.253 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-osd", 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDPFCpmcl9YBRAAVNv7UnX2ZC2efIupCJ5xrg==", 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDPFCpmrmZYBRAArCLflp6iyPNG7lioBj7N4g==", 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.254 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDPFCpmD25YBRAAQatA7SyXA1j2cqR0tsYDCg==", 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDPFCpmeXVYBRAAznkCfXX3G6ypRDFhEI6fwA==", 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.255 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcXHhCBAAvI5zmWidVTvOmEcVRWasEA==", 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcr96BRAAQwUPxILJukYERAiEXeur+g==", 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:32:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:32:44.257 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:32:44.257 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.257 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:44.257 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T08:32:44.257 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:32:44.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.247+0000 7fea02c9e640 1 -- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9e006b4d0 msgr2=0x7fe9e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.247+0000 7fea02c9e640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9e006b4d0 0x7fe9e006d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0005880 tx=0x7fe9f0009040 comp rx=0 tx=0).stop 2024-04-25T08:32:44.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.247+0000 7fea02c9e640 1 -- 172.21.15.38:0/3357293758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b93f0 msgr2=0x7fe9fc0aa840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.247+0000 7fea02c9e640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b93f0 0x7fe9fc0aa840 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000c00 tx=0x7fe9e80046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.247+0000 7fea02c9e640 1 -- 172.21.15.38:0/3357293758 shutdown_connections 2024-04-25T08:32:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.247+0000 7fea02c9e640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc15ddf0 0x7fe9fc15faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.247+0000 7fea02c9e640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b93f0 0x7fe9fc0aa840 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.247+0000 7fea02c9e640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9e006b4d0 0x7fe9e006d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.247+0000 7fea02c9e640 1 --2- 172.21.15.38:0/3357293758 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0b5710 0x7fe9fc0aa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.247+0000 7fea02c9e640 1 -- 172.21.15.38:0/3357293758 >> 172.21.15.38:0/3357293758 conn(0x7fe9fc0d07c0 msgr2=0x7fe9fc0d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.247+0000 7fea02c9e640 1 -- 172.21.15.38:0/3357293758 shutdown_connections 2024-04-25T08:32:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.247+0000 7fea02c9e640 1 -- 172.21.15.38:0/3357293758 wait complete. 2024-04-25T08:32:44.259 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T08:32:44.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.365+0000 7f2410d22640 1 Processor -- start 2024-04-25T08:32:44.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.365+0000 7f2410d22640 1 -- start start 2024-04-25T08:32:44.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f2410d22640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f240c0d01e0 0x7f240c0d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f2410d22640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f240c0b6990 0x7f240c0b6e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f2410d22640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240c0d24a0 con 0x7f240c15dcc0 2024-04-25T08:32:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f2410d22640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240c0d0b80 con 0x7f240c0b6990 2024-04-25T08:32:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f2410d22640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0d3540 con 0x7f240c0d01e0 2024-04-25T08:32:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f240ad9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f240c0b6990 0x7f240c0b6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f240a59c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f240c15dcc0 0x7f240c0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49580/0 (socket says 172.21.15.38:49580) 2024-04-25T08:32:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f240a59c640 1 -- 172.21.15.38:0/1751795880 learned_addr learned my addr 172.21.15.38:0/1751795880 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f2403fff640 1 --2- 172.21.15.38:0/1751795880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f240c0d01e0 0x7f240c0d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f240ad9d640 1 -- 172.21.15.38:0/1751795880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f240c0d01e0 msgr2=0x7f240c0d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f240ad9d640 1 --2- 172.21.15.38:0/1751795880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f240c0d01e0 0x7f240c0d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f240ad9d640 1 -- 172.21.15.38:0/1751795880 >> v1:172.21.15.38:6789/0 conn(0x7f240c15dcc0 legacy=0x7f240c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.366+0000 7f240ad9d640 1 -- 172.21.15.38:0/1751795880 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c0d2780 con 0x7f240c0b6990 2024-04-25T08:32:44.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.367+0000 7f240ad9d640 1 --2- 172.21.15.38:0/1751795880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f240c0b6990 0x7f240c0b6e20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2404009d90 tx=0x7f240402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233367e77b0826d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.367+0000 7f2409d9b640 1 -- 172.21.15.38:0/1751795880 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404007740 con 0x7f240c0b6990 2024-04-25T08:32:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.367+0000 7f2409d9b640 1 -- 172.21.15.38:0/1751795880 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24040078e0 con 0x7f240c0b6990 2024-04-25T08:32:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.367+0000 7f2409d9b640 1 -- 172.21.15.38:0/1751795880 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404007cb0 con 0x7f240c0b6990 2024-04-25T08:32:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.367+0000 7f2410d22640 1 -- 172.21.15.38:0/1751795880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f240c0b6990 msgr2=0x7f240c0b6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.367+0000 7f2410d22640 1 --2- 172.21.15.38:0/1751795880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f240c0b6990 0x7f240c0b6e20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2404009d90 tx=0x7f240402a6f0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.368+0000 7f2410d22640 1 -- 172.21.15.38:0/1751795880 shutdown_connections 2024-04-25T08:32:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.368+0000 7f2410d22640 1 --2- 172.21.15.38:0/1751795880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f240c0b6990 0x7f240c0b6e20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.368+0000 7f2410d22640 1 --2- 172.21.15.38:0/1751795880 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f240c0d01e0 0x7f240c0d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.368+0000 7f2410d22640 1 -- 172.21.15.38:0/1751795880 >> 172.21.15.38:0/1751795880 conn(0x7f240c06bde0 msgr2=0x7f240c06c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.368+0000 7f2410d22640 1 -- 172.21.15.38:0/1751795880 shutdown_connections 2024-04-25T08:32:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.368+0000 7f2410d22640 1 -- 172.21.15.38:0/1751795880 wait complete. 2024-04-25T08:32:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.369+0000 7f2410d22640 1 Processor -- start 2024-04-25T08:32:44.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.369+0000 7f2410d22640 1 -- start start 2024-04-25T08:32:44.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f2410d22640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f240c0b6990 0x7f240c0b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f2410d22640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f240c0d01e0 0x7f240c160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f2410d22640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f240c15dcc0 0x7f240c0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f2410d22640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0d3020 con 0x7f240c15dcc0 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f2410d22640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0d2d40 con 0x7f240c0b6990 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f2410d22640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240c0b9720 con 0x7f240c0d01e0 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f240ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f240c15dcc0 0x7f240c0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f240ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f240c15dcc0 0x7f240c0b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41844/0 (socket says 172.21.15.38:41844) 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f240ad9d640 1 -- 172.21.15.38:0/3851941058 learned_addr learned my addr 172.21.15.38:0/3851941058 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f2403fff640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f240c0d01e0 0x7f240c160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f240a59c640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f240c0b6990 0x7f240c0b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f240ad9d640 1 -- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f240c0d01e0 msgr2=0x7f240c160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f240ad9d640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f240c0d01e0 0x7f240c160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f240ad9d640 1 -- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f240c0b6990 msgr2=0x7f240c0b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f240ad9d640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f240c0b6990 0x7f240c0b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f240ad9d640 1 -- 172.21.15.38:0/3851941058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c0cbc00 con 0x7f240c15dcc0 2024-04-25T08:32:44.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f240ad9d640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f240c15dcc0 0x7f240c0b4e30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f240402ace0 tx=0x7f2404004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f2402ffd640 1 -- 172.21.15.38:0/3851941058 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404040070 con 0x7f240c15dcc0 2024-04-25T08:32:44.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f2402ffd640 1 -- 172.21.15.38:0/3851941058 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2404031bc0 con 0x7f240c15dcc0 2024-04-25T08:32:44.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f2410d22640 1 -- 172.21.15.38:0/3851941058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240c0cbee0 con 0x7f240c15dcc0 2024-04-25T08:32:44.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.370+0000 7f2402ffd640 1 -- 172.21.15.38:0/3851941058 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404045460 con 0x7f240c15dcc0 2024-04-25T08:32:44.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.371+0000 7f2410d22640 1 -- 172.21.15.38:0/3851941058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f240c0cc4a0 con 0x7f240c15dcc0 2024-04-25T08:32:44.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.371+0000 7f2410d22640 1 -- 172.21.15.38:0/3851941058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f240c0cc6a0 con 0x7f240c15dcc0 2024-04-25T08:32:44.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.372+0000 7f2402ffd640 1 -- 172.21.15.38:0/3851941058 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2404039040 con 0x7f240c15dcc0 2024-04-25T08:32:44.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.372+0000 7f2402ffd640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f23ec06b4d0 0x7f23ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.372+0000 7f2402ffd640 1 -- 172.21.15.38:0/3851941058 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f24040aba60 con 0x7f240c15dcc0 2024-04-25T08:32:44.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.373+0000 7f240a59c640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f23ec06b4d0 0x7f23ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.373+0000 7f240a59c640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f23ec06b4d0 0x7f23ec06d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f23fc000f60 tx=0x7f23fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.376+0000 7f2402ffd640 1 -- 172.21.15.38:0/3851941058 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f240c0cc6a0 con 0x7f240c15dcc0 2024-04-25T08:32:44.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.624+0000 7f2410d22640 1 -- 172.21.15.38:0/3851941058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f240c04ef90 con 0x7f240c15dcc0 2024-04-25T08:32:44.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.627+0000 7f2402ffd640 1 -- 172.21.15.38:0/3851941058 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f240407ab20 con 0x7f240c15dcc0 2024-04-25T08:32:44.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.629+0000 7f2410d22640 1 -- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f23ec06b4d0 msgr2=0x7f23ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.629+0000 7f2410d22640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f23ec06b4d0 0x7f23ec06d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f23fc000f60 tx=0x7f23fc009040 comp rx=0 tx=0).stop 2024-04-25T08:32:44.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.629+0000 7f2410d22640 1 -- 172.21.15.38:0/3851941058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f240c15dcc0 msgr2=0x7f240c0b4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.629+0000 7f2410d22640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f240c15dcc0 0x7f240c0b4e30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f240402ace0 tx=0x7f2404004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:44.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.629+0000 7f2410d22640 1 -- 172.21.15.38:0/3851941058 shutdown_connections 2024-04-25T08:32:44.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.629+0000 7f2410d22640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f240c15dcc0 0x7f240c0b4e30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.629+0000 7f2410d22640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f23ec06b4d0 0x7f23ec06d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.629+0000 7f2410d22640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f240c0d01e0 0x7f240c160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.629+0000 7f2410d22640 1 --2- 172.21.15.38:0/3851941058 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f240c0b6990 0x7f240c0b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.629+0000 7f2410d22640 1 -- 172.21.15.38:0/3851941058 >> 172.21.15.38:0/3851941058 conn(0x7f240c06bde0 msgr2=0x7f240c0b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:44.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.630+0000 7f2410d22640 1 -- 172.21.15.38:0/3851941058 shutdown_connections 2024-04-25T08:32:44.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.630+0000 7f2410d22640 1 -- 172.21.15.38:0/3851941058 wait complete. 2024-04-25T08:32:44.636 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T08:32:44.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.742+0000 7f481b56a640 1 Processor -- start 2024-04-25T08:32:44.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.742+0000 7f481b56a640 1 -- start start 2024-04-25T08:32:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.743+0000 7f481b56a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f48140b9600 0x7f48140d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.743+0000 7f481b56a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48140d0a70 con 0x7f48140d0660 2024-04-25T08:32:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.743+0000 7f481b56a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4814162030 con 0x7f481415db80 2024-04-25T08:32:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.743+0000 7f481b56a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4814161df0 con 0x7f48140b9600 2024-04-25T08:32:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.743+0000 7f4818b05640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f48140b9600 0x7f48140d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.743+0000 7f4819b07640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f48140d0660 0x7f48140d22b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49586/0 (socket says 172.21.15.38:49586) 2024-04-25T08:32:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.743+0000 7f4819b07640 1 -- 172.21.15.38:0/2959081954 learned_addr learned my addr 172.21.15.38:0/2959081954 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.743+0000 7f4818b05640 1 -- 172.21.15.38:0/2959081954 >> v1:172.21.15.107:6789/0 conn(0x7f481415db80 legacy=0x7f48141616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.743+0000 7f4818b05640 1 -- 172.21.15.38:0/2959081954 >> v1:172.21.15.38:6789/0 conn(0x7f48140d0660 legacy=0x7f48140d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.743+0000 7f480bfff640 1 -- 172.21.15.38:0/2959081954 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245432003 0 0) 0x7f48140d0a70 con 0x7f48140d0660 2024-04-25T08:32:44.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.743+0000 7f4818b05640 1 -- 172.21.15.38:0/2959081954 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4814162310 con 0x7f48140b9600 2024-04-25T08:32:44.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.744+0000 7f4818b05640 1 --2- 172.21.15.38:0/2959081954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48140b9600 0x7f48140d0120 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f48040097a0 tx=0x7f480402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dec510f13ccca84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.744+0000 7f480bfff640 1 -- 172.21.15.38:0/2959081954 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804037070 con 0x7f48140b9600 2024-04-25T08:32:44.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.744+0000 7f480bfff640 1 -- 172.21.15.38:0/2959081954 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4804032440 con 0x7f48140b9600 2024-04-25T08:32:44.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.744+0000 7f480bfff640 1 -- 172.21.15.38:0/2959081954 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804032810 con 0x7f48140b9600 2024-04-25T08:32:44.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.745+0000 7f481b56a640 1 -- 172.21.15.38:0/2959081954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48140b9600 msgr2=0x7f48140d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.745+0000 7f481b56a640 1 --2- 172.21.15.38:0/2959081954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48140b9600 0x7f48140d0120 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f48040097a0 tx=0x7f480402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.745+0000 7f481b56a640 1 -- 172.21.15.38:0/2959081954 shutdown_connections 2024-04-25T08:32:44.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.745+0000 7f481b56a640 1 --2- 172.21.15.38:0/2959081954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48140b9600 0x7f48140d0120 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.745+0000 7f481b56a640 1 -- 172.21.15.38:0/2959081954 >> 172.21.15.38:0/2959081954 conn(0x7f48140668f0 msgr2=0x7f4814066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:44.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.745+0000 7f481b56a640 1 -- 172.21.15.38:0/2959081954 shutdown_connections 2024-04-25T08:32:44.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.745+0000 7f481b56a640 1 -- 172.21.15.38:0/2959081954 wait complete. 2024-04-25T08:32:44.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.746+0000 7f481b56a640 1 Processor -- start 2024-04-25T08:32:44.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.746+0000 7f481b56a640 1 -- start start 2024-04-25T08:32:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f481b56a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48140b9600 0x7f48141532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f481b56a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48140d0660 0x7f48140ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f481b56a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f481415db80 0x7f48140cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f481b56a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481415df60 con 0x7f48140d0660 2024-04-25T08:32:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f481b56a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48141625f0 con 0x7f48140b9600 2024-04-25T08:32:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f481b56a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48140d29c0 con 0x7f481415db80 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f4818b05640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48140d0660 0x7f48140ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f4818b05640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48140d0660 0x7f48140ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41856/0 (socket says 172.21.15.38:41856) 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f4818b05640 1 -- 172.21.15.38:0/103817940 learned_addr learned my addr 172.21.15.38:0/103817940 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f4819306640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48140b9600 0x7f48141532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f4819b07640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f481415db80 0x7f48140cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f4818b05640 1 -- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f481415db80 msgr2=0x7f48140cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f4818b05640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f481415db80 0x7f48140cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f4818b05640 1 -- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48140b9600 msgr2=0x7f48141532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f4818b05640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48140b9600 0x7f48141532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f4818b05640 1 -- 172.21.15.38:0/103817940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48140c9290 con 0x7f48140d0660 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f4819b07640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f481415db80 0x7f48140cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.747+0000 7f4818b05640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48140d0660 0x7f48140ca940 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f48040098c0 tx=0x7f48040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.748+0000 7f4809ffb640 1 -- 172.21.15.38:0/103817940 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804037040 con 0x7f48140d0660 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.748+0000 7f4809ffb640 1 -- 172.21.15.38:0/103817940 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4804042070 con 0x7f48140d0660 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.748+0000 7f4809ffb640 1 -- 172.21.15.38:0/103817940 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804030a70 con 0x7f48140d0660 2024-04-25T08:32:44.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.748+0000 7f481b56a640 1 -- 172.21.15.38:0/103817940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48140c9570 con 0x7f48140d0660 2024-04-25T08:32:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.748+0000 7f481b56a640 1 -- 172.21.15.38:0/103817940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48140cbc10 con 0x7f48140d0660 2024-04-25T08:32:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.749+0000 7f481b56a640 1 -- 172.21.15.38:0/103817940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481404ef90 con 0x7f48140d0660 2024-04-25T08:32:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.749+0000 7f4809ffb640 1 -- 172.21.15.38:0/103817940 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4804030c10 con 0x7f48140d0660 2024-04-25T08:32:44.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.750+0000 7f4809ffb640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47f806b4d0 0x7f47f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:44.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.750+0000 7f4809ffb640 1 -- 172.21.15.38:0/103817940 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f48040aaad0 con 0x7f48140d0660 2024-04-25T08:32:44.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.750+0000 7f4819306640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47f806b4d0 0x7f47f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:44.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.751+0000 7f4819306640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47f806b4d0 0x7f47f806d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4810000f60 tx=0x7f4810009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:44.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:44.754+0000 7f4809ffb640 1 -- 172.21.15.38:0/103817940 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4804079a80 con 0x7f48140d0660 2024-04-25T08:32:45.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.008+0000 7f481b56a640 1 -- 172.21.15.38:0/103817940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f481415d310 con 0x7f48140d0660 2024-04-25T08:32:45.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.010+0000 7f4809ffb640 1 -- 172.21.15.38:0/103817940 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4804079380 con 0x7f48140d0660 2024-04-25T08:32:45.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.012+0000 7f481b56a640 1 -- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47f806b4d0 msgr2=0x7f47f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.012+0000 7f481b56a640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47f806b4d0 0x7f47f806d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4810000f60 tx=0x7f4810009040 comp rx=0 tx=0).stop 2024-04-25T08:32:45.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.012+0000 7f481b56a640 1 -- 172.21.15.38:0/103817940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48140d0660 msgr2=0x7f48140ca940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.012+0000 7f481b56a640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48140d0660 0x7f48140ca940 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f48040098c0 tx=0x7f48040046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.012+0000 7f481b56a640 1 -- 172.21.15.38:0/103817940 shutdown_connections 2024-04-25T08:32:45.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.012+0000 7f481b56a640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47f806b4d0 0x7f47f806d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.012+0000 7f481b56a640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f481415db80 0x7f48140cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.012+0000 7f481b56a640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f48140d0660 0x7f48140ca940 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.012+0000 7f481b56a640 1 --2- 172.21.15.38:0/103817940 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f48140b9600 0x7f48141532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.012+0000 7f481b56a640 1 -- 172.21.15.38:0/103817940 >> 172.21.15.38:0/103817940 conn(0x7f48140668f0 msgr2=0x7f48140b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:45.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.012+0000 7f481b56a640 1 -- 172.21.15.38:0/103817940 shutdown_connections 2024-04-25T08:32:45.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.013+0000 7f481b56a640 1 -- 172.21.15.38:0/103817940 wait complete. 2024-04-25T08:32:45.019 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T08:32:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.125+0000 7fbe8f815640 1 Processor -- start 2024-04-25T08:32:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.125+0000 7fbe8f815640 1 -- start start 2024-04-25T08:32:45.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8f815640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbe8815db80 0x7fbe881616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8f815640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe880d0830 0x7fbe880d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8f815640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe88162030 con 0x7fbe880b9600 2024-04-25T08:32:45.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8f815640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe881625f0 con 0x7fbe8815db80 2024-04-25T08:32:45.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8f815640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe88162310 con 0x7fbe880d0830 2024-04-25T08:32:45.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8d5b1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbe8815db80 0x7fbe881616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8d5b1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbe8815db80 0x7fbe881616e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41870/0 (socket says 172.21.15.38:41870) 2024-04-25T08:32:45.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8d5b1640 1 -- 172.21.15.38:0/2877297536 learned_addr learned my addr 172.21.15.38:0/2877297536 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:45.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8ddb2640 1 --2- 172.21.15.38:0/2877297536 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe880d0830 0x7fbe880d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8d5b1640 1 -- 172.21.15.38:0/2877297536 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe880d0830 msgr2=0x7fbe880d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:45.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8d5b1640 1 --2- 172.21.15.38:0/2877297536 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe880d0830 0x7fbe880d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8d5b1640 1 -- 172.21.15.38:0/2877297536 >> v1:172.21.15.107:6789/0 conn(0x7fbe880b9600 legacy=0x7fbe880d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.126+0000 7fbe8d5b1640 1 -- 172.21.15.38:0/2877297536 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe880d1ab0 con 0x7fbe8815db80 2024-04-25T08:32:45.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.127+0000 7fbe8d5b1640 1 --2- 172.21.15.38:0/2877297536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe8815db80 0x7fbe881616e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009870 tx=0x7fbe8002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9366ff9fd09726fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:45.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.127+0000 7fbe7ffff640 1 -- 172.21.15.38:0/2877297536 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80037070 con 0x7fbe8815db80 2024-04-25T08:32:45.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.127+0000 7fbe7ffff640 1 -- 172.21.15.38:0/2877297536 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe80032440 con 0x7fbe8815db80 2024-04-25T08:32:45.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.127+0000 7fbe7ffff640 1 -- 172.21.15.38:0/2877297536 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80032810 con 0x7fbe8815db80 2024-04-25T08:32:45.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.128+0000 7fbe8f815640 1 -- 172.21.15.38:0/2877297536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe8815db80 msgr2=0x7fbe881616e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.128+0000 7fbe8f815640 1 --2- 172.21.15.38:0/2877297536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe8815db80 0x7fbe881616e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbe80009870 tx=0x7fbe8002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.128+0000 7fbe8f815640 1 -- 172.21.15.38:0/2877297536 shutdown_connections 2024-04-25T08:32:45.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.128+0000 7fbe8f815640 1 --2- 172.21.15.38:0/2877297536 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe880d0830 0x7fbe880d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.128+0000 7fbe8f815640 1 --2- 172.21.15.38:0/2877297536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe8815db80 0x7fbe881616e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.128+0000 7fbe8f815640 1 -- 172.21.15.38:0/2877297536 >> 172.21.15.38:0/2877297536 conn(0x7fbe880668f0 msgr2=0x7fbe88066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:45.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.128+0000 7fbe8f815640 1 -- 172.21.15.38:0/2877297536 shutdown_connections 2024-04-25T08:32:45.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.128+0000 7fbe8f815640 1 -- 172.21.15.38:0/2877297536 wait complete. 2024-04-25T08:32:45.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.129+0000 7fbe8f815640 1 Processor -- start 2024-04-25T08:32:45.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.129+0000 7fbe8f815640 1 -- start start 2024-04-25T08:32:45.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8f815640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe880b9600 0x7fbe88157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8f815640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe880d0830 0x7fbe8815fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8f815640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe8815db80 0x7fbe8815e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8f815640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe88161da0 con 0x7fbe8815db80 2024-04-25T08:32:45.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8f815640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe88161c20 con 0x7fbe880b9600 2024-04-25T08:32:45.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8f815640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe880d27f0 con 0x7fbe880d0830 2024-04-25T08:32:45.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8ddb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe8815db80 0x7fbe8815e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8ddb2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe8815db80 0x7fbe8815e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41884/0 (socket says 172.21.15.38:41884) 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8ddb2640 1 -- 172.21.15.38:0/3067887994 learned_addr learned my addr 172.21.15.38:0/3067887994 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8cdb0640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe880d0830 0x7fbe8815fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8d5b1640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe880b9600 0x7fbe88157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8ddb2640 1 -- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe880d0830 msgr2=0x7fbe8815fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8ddb2640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe880d0830 0x7fbe8815fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8ddb2640 1 -- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe880b9600 msgr2=0x7fbe88157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8ddb2640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe880b9600 0x7fbe88157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8ddb2640 1 -- 172.21.15.38:0/3067887994 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe880cc8a0 con 0x7fbe8815db80 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.130+0000 7fbe8ddb2640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe8815db80 0x7fbe8815e1e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400ef90 tx=0x7fbe8400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.131+0000 7fbe7dffb640 1 -- 172.21.15.38:0/3067887994 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84018020 con 0x7fbe8815db80 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.131+0000 7fbe7dffb640 1 -- 172.21.15.38:0/3067887994 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe84014480 con 0x7fbe8815db80 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.131+0000 7fbe8f815640 1 -- 172.21.15.38:0/3067887994 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8815ed10 con 0x7fbe8815db80 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.131+0000 7fbe7dffb640 1 -- 172.21.15.38:0/3067887994 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84014870 con 0x7fbe8815db80 2024-04-25T08:32:45.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.131+0000 7fbe8f815640 1 -- 172.21.15.38:0/3067887994 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8815f5b0 con 0x7fbe8815db80 2024-04-25T08:32:45.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.132+0000 7fbe8f815640 1 -- 172.21.15.38:0/3067887994 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe880cf530 con 0x7fbe8815db80 2024-04-25T08:32:45.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.132+0000 7fbe7dffb640 1 -- 172.21.15.38:0/3067887994 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe84005d10 con 0x7fbe8815db80 2024-04-25T08:32:45.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.133+0000 7fbe7dffb640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.133+0000 7fbe7dffb640 1 -- 172.21.15.38:0/3067887994 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbe8408ac80 con 0x7fbe8815db80 2024-04-25T08:32:45.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.133+0000 7fbe8d5b1640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.133+0000 7fbe8d5b1640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbe80034000 tx=0x7fbe8002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:45.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.137+0000 7fbe7dffb640 1 -- 172.21.15.38:0/3067887994 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe880cf530 con 0x7fbe8815db80 2024-04-25T08:32:45.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.384+0000 7fbe8f815640 1 -- 172.21.15.38:0/3067887994 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fbe8804ef90 con 0x7fbe8815db80 2024-04-25T08:32:45.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.388+0000 7fbe7dffb640 1 -- 172.21.15.38:0/3067887994 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbe84022030 con 0x7fbe8815db80 2024-04-25T08:32:45.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.390+0000 7fbe8f815640 1 -- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbe6c06b4d0 msgr2=0x7fbe6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.390+0000 7fbe8f815640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbe80034000 tx=0x7fbe8002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:45.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.390+0000 7fbe8f815640 1 -- 172.21.15.38:0/3067887994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe8815db80 msgr2=0x7fbe8815e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.390+0000 7fbe8f815640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe8815db80 0x7fbe8815e1e0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400ef90 tx=0x7fbe8400c490 comp rx=0 tx=0).stop 2024-04-25T08:32:45.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.390+0000 7fbe8f815640 1 -- 172.21.15.38:0/3067887994 shutdown_connections 2024-04-25T08:32:45.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.390+0000 7fbe8f815640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe8815db80 0x7fbe8815e1e0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.390+0000 7fbe8f815640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.390+0000 7fbe8f815640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe880d0830 0x7fbe8815fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.390+0000 7fbe8f815640 1 --2- 172.21.15.38:0/3067887994 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe880b9600 0x7fbe88157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.390+0000 7fbe8f815640 1 -- 172.21.15.38:0/3067887994 >> 172.21.15.38:0/3067887994 conn(0x7fbe880668f0 msgr2=0x7fbe880b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:45.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.390+0000 7fbe8f815640 1 -- 172.21.15.38:0/3067887994 shutdown_connections 2024-04-25T08:32:45.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.390+0000 7fbe8f815640 1 -- 172.21.15.38:0/3067887994 wait complete. 2024-04-25T08:32:45.397 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T08:32:45.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.503+0000 7f6e27f70640 1 Processor -- start 2024-04-25T08:32:45.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.503+0000 7f6e27f70640 1 -- start start 2024-04-25T08:32:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.504+0000 7f6e27f70640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e20145080 0x7f6e2015cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.504+0000 7f6e27f70640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6e20071b30 0x7f6e200cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.504+0000 7f6e27f70640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2015c130 con 0x7f6e20071f90 2024-04-25T08:32:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.504+0000 7f6e27f70640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e20145460 con 0x7f6e20071b30 2024-04-25T08:32:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.504+0000 7f6e27f70640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015c410 con 0x7f6e20145080 2024-04-25T08:32:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.504+0000 7f6e2650d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6e20071f90 0x7f6e2015f630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49594/0 (socket says 172.21.15.38:49594) 2024-04-25T08:32:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.504+0000 7f6e2650d640 1 -- 172.21.15.38:0/3557925820 learned_addr learned my addr 172.21.15.38:0/3557925820 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e2550b640 1 --2- 172.21.15.38:0/3557925820 >> v2:172.21.15.107:3300/0 conn(0x7f6e20071b30 0x7f6e200cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e25d0c640 1 --2- 172.21.15.38:0/3557925820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e20145080 0x7f6e2015cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894227319 0 0) 0x7f6e2015c130 con 0x7f6e20071f90 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e08003520 con 0x7f6e20071f90 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3362854612 0 0) 0x7f6e08003520 con 0x7f6e20071f90 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e20145080 msgr2=0x7f6e2015cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 --2- 172.21.15.38:0/3557925820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e20145080 0x7f6e2015cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 >> v2:172.21.15.107:3300/0 conn(0x7f6e20071b30 msgr2=0x7f6e200cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 --2- 172.21.15.38:0/3557925820 >> v2:172.21.15.107:3300/0 conn(0x7f6e20071b30 0x7f6e200cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2015c6f0 con 0x7f6e20071f90 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f6e18001670 con 0x7f6e20071f90 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 >> v1:172.21.15.38:6789/0 conn(0x7f6e20071f90 legacy=0x7f6e2015f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 --2- 172.21.15.38:0/3557925820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e20145080 0x7f6e080039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 --2- 172.21.15.38:0/3557925820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e08003f30 0x7f6e08004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 --2- 172.21.15.38:0/3557925820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e08004d40 0x7f6e080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015c410 con 0x7f6e08004d40 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0800b9c0 con 0x7f6e08003f30 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0800bb40 con 0x7f6e20145080 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e25d0c640 1 --2- 172.21.15.38:0/3557925820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e20145080 0x7f6e080039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e2650d640 1 --2- 172.21.15.38:0/3557925820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e08004d40 0x7f6e080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e2550b640 1 --2- 172.21.15.38:0/3557925820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e08003f30 0x7f6e08004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e2650d640 1 -- 172.21.15.38:0/3557925820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e20145080 msgr2=0x7f6e080039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e2650d640 1 --2- 172.21.15.38:0/3557925820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e20145080 0x7f6e080039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e2650d640 1 -- 172.21.15.38:0/3557925820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e08003f30 msgr2=0x7f6e08004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e2650d640 1 --2- 172.21.15.38:0/3557925820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e08003f30 0x7f6e08004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.505+0000 7f6e2650d640 1 -- 172.21.15.38:0/3557925820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e08003520 con 0x7f6e08004d40 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.506+0000 7f6e2650d640 1 --2- 172.21.15.38:0/3557925820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e08004d40 0x7f6e080051c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6e20072420 tx=0x7f6e08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8291bb6c5381dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.506+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18007d50 con 0x7f6e08004d40 2024-04-25T08:32:45.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.506+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e180058d0 con 0x7f6e08004d40 2024-04-25T08:32:45.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.506+0000 7f6e24d0a640 1 -- 172.21.15.38:0/3557925820 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18005cc0 con 0x7f6e08004d40 2024-04-25T08:32:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.507+0000 7f6e27f70640 1 -- 172.21.15.38:0/3557925820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e08004d40 msgr2=0x7f6e080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.507+0000 7f6e27f70640 1 --2- 172.21.15.38:0/3557925820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e08004d40 0x7f6e080051c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6e20072420 tx=0x7f6e08005830 comp rx=0 tx=0).stop 2024-04-25T08:32:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.507+0000 7f6e2650d640 1 -- 172.21.15.38:0/3557925820 reap_dead start 2024-04-25T08:32:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.507+0000 7f6e27f70640 1 -- 172.21.15.38:0/3557925820 shutdown_connections 2024-04-25T08:32:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.507+0000 7f6e27f70640 1 -- 172.21.15.38:0/3557925820 >> 172.21.15.38:0/3557925820 conn(0x7f6e2006abe0 msgr2=0x7f6e2006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.507+0000 7f6e27f70640 1 -- 172.21.15.38:0/3557925820 shutdown_connections 2024-04-25T08:32:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.507+0000 7f6e27f70640 1 -- 172.21.15.38:0/3557925820 wait complete. 2024-04-25T08:32:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.508+0000 7f6e27f70640 1 Processor -- start 2024-04-25T08:32:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.508+0000 7f6e27f70640 1 -- start start 2024-04-25T08:32:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e27f70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e2015ae80 0x7f6e2013c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e27f70640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e2013c930 0x7f6e200ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e27f70640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e200adc10 0x7f6e201f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e27f70640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015bd60 con 0x7f6e2015ae80 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e27f70640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015ccb0 con 0x7f6e200adc10 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e27f70640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2015bee0 con 0x7f6e2013c930 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e25d0c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e2015ae80 0x7f6e2013c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e25d0c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e2015ae80 0x7f6e2013c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41896/0 (socket says 172.21.15.38:41896) 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e25d0c640 1 -- 172.21.15.38:0/548616040 learned_addr learned my addr 172.21.15.38:0/548616040 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e2650d640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e200adc10 0x7f6e201f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e2550b640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e2013c930 0x7f6e200ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e25d0c640 1 -- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e2013c930 msgr2=0x7f6e200ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e25d0c640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e2013c930 0x7f6e200ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e25d0c640 1 -- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e200adc10 msgr2=0x7f6e201f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e25d0c640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e200adc10 0x7f6e201f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e25d0c640 1 -- 172.21.15.38:0/548616040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e200b3330 con 0x7f6e2015ae80 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e2650d640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e200adc10 0x7f6e201f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e2550b640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e2013c930 0x7f6e200ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.509+0000 7f6e25d0c640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e2015ae80 0x7f6e2013c3f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6e10011e60 tx=0x7f6e1000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.510+0000 7f6e1e7fc640 1 -- 172.21.15.38:0/548616040 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e10015630 con 0x7f6e2015ae80 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.510+0000 7f6e1e7fc640 1 -- 172.21.15.38:0/548616040 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e100157d0 con 0x7f6e2015ae80 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.510+0000 7f6e1e7fc640 1 -- 172.21.15.38:0/548616040 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e10015ba0 con 0x7f6e2015ae80 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.510+0000 7f6e27f70640 1 -- 172.21.15.38:0/548616040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e200b2290 con 0x7f6e2015ae80 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.510+0000 7f6e27f70640 1 -- 172.21.15.38:0/548616040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e200b2850 con 0x7f6e2015ae80 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.511+0000 7f6e27f70640 1 -- 172.21.15.38:0/548616040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2004ef90 con 0x7f6e2015ae80 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.511+0000 7f6e1e7fc640 1 -- 172.21.15.38:0/548616040 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e1001d400 con 0x7f6e2015ae80 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.511+0000 7f6e1e7fc640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e0c06b4d0 0x7f6e0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.512+0000 7f6e1e7fc640 1 -- 172.21.15.38:0/548616040 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6e1008b1c0 con 0x7f6e2015ae80 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.512+0000 7f6e2550b640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e0c06b4d0 0x7f6e0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.512+0000 7f6e2550b640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e0c06b4d0 0x7f6e0c06d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6e200b35b0 tx=0x7f6e14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:45.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.516+0000 7f6e1e7fc640 1 -- 172.21.15.38:0/548616040 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e10019030 con 0x7f6e2015ae80 2024-04-25T08:32:45.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.765+0000 7f6e27f70640 1 -- 172.21.15.38:0/548616040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f6e200ccd70 con 0x7f6e2015ae80 2024-04-25T08:32:45.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.768+0000 7f6e1e7fc640 1 -- 172.21.15.38:0/548616040 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6e10010040 con 0x7f6e2015ae80 2024-04-25T08:32:45.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.770+0000 7f6e27f70640 1 -- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e0c06b4d0 msgr2=0x7f6e0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.770+0000 7f6e27f70640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e0c06b4d0 0x7f6e0c06d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6e200b35b0 tx=0x7f6e14008040 comp rx=0 tx=0).stop 2024-04-25T08:32:45.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.770+0000 7f6e27f70640 1 -- 172.21.15.38:0/548616040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e2015ae80 msgr2=0x7f6e2013c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.770+0000 7f6e27f70640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e2015ae80 0x7f6e2013c3f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6e10011e60 tx=0x7f6e1000f580 comp rx=0 tx=0).stop 2024-04-25T08:32:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.770+0000 7f6e27f70640 1 -- 172.21.15.38:0/548616040 shutdown_connections 2024-04-25T08:32:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.770+0000 7f6e27f70640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e200adc10 0x7f6e201f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.770+0000 7f6e27f70640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e0c06b4d0 0x7f6e0c06d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.770+0000 7f6e27f70640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e2013c930 0x7f6e200ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.770+0000 7f6e27f70640 1 --2- 172.21.15.38:0/548616040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e2015ae80 0x7f6e2013c3f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.770+0000 7f6e27f70640 1 -- 172.21.15.38:0/548616040 >> 172.21.15.38:0/548616040 conn(0x7f6e2006abe0 msgr2=0x7f6e200b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.770+0000 7f6e27f70640 1 -- 172.21.15.38:0/548616040 shutdown_connections 2024-04-25T08:32:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.770+0000 7f6e27f70640 1 -- 172.21.15.38:0/548616040 wait complete. 2024-04-25T08:32:45.777 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T08:32:45.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.883+0000 7fd2b1b6a640 1 Processor -- start 2024-04-25T08:32:45.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.883+0000 7fd2b1b6a640 1 -- start start 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2b1b6a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd2ac0b9180 0x7fd2ac0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2b1b6a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2ac0b2ab0 0x7fd2ac0d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2b1b6a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac0b9610 con 0x7fd2ac160ef0 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2b1b6a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ac161cf0 con 0x7fd2ac0b9180 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2b1b6a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac1612d0 con 0x7fd2ac0b2ab0 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2aaffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd2ac0b9180 0x7fd2ac0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2ab7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd2ac160ef0 0x7fd2ac0d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49596/0 (socket says 172.21.15.38:49596) 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2aaffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd2ac0b9180 0x7fd2ac0d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41910/0 (socket says 172.21.15.38:41910) 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2ab7fe640 1 -- 172.21.15.38:0/4201106653 learned_addr learned my addr 172.21.15.38:0/4201106653 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2abfff640 1 --2- 172.21.15.38:0/4201106653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2ac0b2ab0 0x7fd2ac0d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2aaffd640 1 -- 172.21.15.38:0/4201106653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2ac0b2ab0 msgr2=0x7fd2ac0d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2aaffd640 1 --2- 172.21.15.38:0/4201106653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2ac0b2ab0 0x7fd2ac0d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2aaffd640 1 -- 172.21.15.38:0/4201106653 >> v1:172.21.15.38:6789/0 conn(0x7fd2ac160ef0 legacy=0x7fd2ac0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2aa7fc640 1 -- 172.21.15.38:0/4201106653 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2066487831 0 0) 0x7fd2ac0b9610 con 0x7fd2ac160ef0 2024-04-25T08:32:45.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2aaffd640 1 -- 172.21.15.38:0/4201106653 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac1616a0 con 0x7fd2ac0b9180 2024-04-25T08:32:45.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.884+0000 7fd2abfff640 1 --2- 172.21.15.38:0/4201106653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2ac0b2ab0 0x7fd2ac0d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:45.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.885+0000 7fd2aaffd640 1 --2- 172.21.15.38:0/4201106653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2ac0b9180 0x7fd2ac0d1970 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00097a0 tx=0x7fd2a002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ac3684efb044e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:45.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.885+0000 7fd2aa7fc640 1 -- 172.21.15.38:0/4201106653 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0037070 con 0x7fd2ac0b9180 2024-04-25T08:32:45.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.885+0000 7fd2aa7fc640 1 -- 172.21.15.38:0/4201106653 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a002ae50 con 0x7fd2ac0b9180 2024-04-25T08:32:45.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.885+0000 7fd2aa7fc640 1 -- 172.21.15.38:0/4201106653 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0032660 con 0x7fd2ac0b9180 2024-04-25T08:32:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.886+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/4201106653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2ac0b9180 msgr2=0x7fd2ac0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.886+0000 7fd2b1b6a640 1 --2- 172.21.15.38:0/4201106653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2ac0b9180 0x7fd2ac0d1970 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd2a00097a0 tx=0x7fd2a002a100 comp rx=0 tx=0).stop 2024-04-25T08:32:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.886+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/4201106653 shutdown_connections 2024-04-25T08:32:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.886+0000 7fd2b1b6a640 1 --2- 172.21.15.38:0/4201106653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2ac0b2ab0 0x7fd2ac0d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.886+0000 7fd2b1b6a640 1 --2- 172.21.15.38:0/4201106653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2ac0b9180 0x7fd2ac0d1970 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.886+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/4201106653 >> 172.21.15.38:0/4201106653 conn(0x7fd2ac066d10 msgr2=0x7fd2ac067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.886+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/4201106653 shutdown_connections 2024-04-25T08:32:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.886+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/4201106653 wait complete. 2024-04-25T08:32:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.887+0000 7fd2b1b6a640 1 Processor -- start 2024-04-25T08:32:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.887+0000 7fd2b1b6a640 1 -- start start 2024-04-25T08:32:45.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2b1b6a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2ac0b2ab0 0x7fd2ac1f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2b1b6a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2ac160ef0 0x7fd2ac158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2b1b6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2ac159120 0x7fd2ac0cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2b1b6a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0d1040 con 0x7fd2ac159120 2024-04-25T08:32:45.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2b1b6a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0d0ec0 con 0x7fd2ac0b2ab0 2024-04-25T08:32:45.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2b1b6a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac162180 con 0x7fd2ac160ef0 2024-04-25T08:32:45.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2abfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2ac159120 0x7fd2ac0cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2abfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2ac159120 0x7fd2ac0cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41912/0 (socket says 172.21.15.38:41912) 2024-04-25T08:32:45.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2abfff640 1 -- 172.21.15.38:0/1678651090 learned_addr learned my addr 172.21.15.38:0/1678651090 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:45.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2ab7fe640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2ac0b2ab0 0x7fd2ac1f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2aaffd640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2ac160ef0 0x7fd2ac158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2abfff640 1 -- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2ac160ef0 msgr2=0x7fd2ac158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2abfff640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2ac160ef0 0x7fd2ac158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2abfff640 1 -- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2ac0b2ab0 msgr2=0x7fd2ac1f2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2abfff640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2ac0b2ab0 0x7fd2ac1f2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2abfff640 1 -- 172.21.15.38:0/1678651090 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac159910 con 0x7fd2ac159120 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.888+0000 7fd2aaffd640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2ac160ef0 0x7fd2ac158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.889+0000 7fd2abfff640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2ac159120 0x7fd2ac0cc350 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400ced0 tx=0x7fd2a400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.889+0000 7fd293fff640 1 -- 172.21.15.38:0/1678651090 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4003b90 con 0x7fd2ac159120 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.889+0000 7fd293fff640 1 -- 172.21.15.38:0/1678651090 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a4003d50 con 0x7fd2ac159120 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.889+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/1678651090 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac159bf0 con 0x7fd2ac159120 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.889+0000 7fd293fff640 1 -- 172.21.15.38:0/1678651090 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a4004260 con 0x7fd2ac159120 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.889+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/1678651090 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac15a1b0 con 0x7fd2ac159120 2024-04-25T08:32:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.890+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/1678651090 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ac04ef90 con 0x7fd2ac159120 2024-04-25T08:32:45.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.890+0000 7fd293fff640 1 -- 172.21.15.38:0/1678651090 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2a4004400 con 0x7fd2ac159120 2024-04-25T08:32:45.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.891+0000 7fd293fff640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd29806b500 0x7fd29806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:45.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.891+0000 7fd293fff640 1 -- 172.21.15.38:0/1678651090 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd2a408a9d0 con 0x7fd2ac159120 2024-04-25T08:32:45.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.891+0000 7fd2ab7fe640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd29806b500 0x7fd29806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:45.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.892+0000 7fd2ab7fe640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd29806b500 0x7fd29806d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd29c000f60 tx=0x7fd29c007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:45.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:45.895+0000 7fd293fff640 1 -- 172.21.15.38:0/1678651090 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2a4059ad0 con 0x7fd2ac159120 2024-04-25T08:32:46.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.147+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/1678651090 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fd2ac06ae50 con 0x7fd2ac159120 2024-04-25T08:32:46.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.148+0000 7fd293fff640 1 -- 172.21.15.38:0/1678651090 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd2a40593d0 con 0x7fd2ac159120 2024-04-25T08:32:46.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.151+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd29806b500 msgr2=0x7fd29806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.151+0000 7fd2b1b6a640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd29806b500 0x7fd29806d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd29c000f60 tx=0x7fd29c007c00 comp rx=0 tx=0).stop 2024-04-25T08:32:46.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.151+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/1678651090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2ac159120 msgr2=0x7fd2ac0cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.151+0000 7fd2b1b6a640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2ac159120 0x7fd2ac0cc350 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400ced0 tx=0x7fd2a400a710 comp rx=0 tx=0).stop 2024-04-25T08:32:46.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.151+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/1678651090 shutdown_connections 2024-04-25T08:32:46.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.151+0000 7fd2b1b6a640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2ac159120 0x7fd2ac0cc350 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.151+0000 7fd2b1b6a640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd29806b500 0x7fd29806d9c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.151+0000 7fd2b1b6a640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2ac160ef0 0x7fd2ac158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.151+0000 7fd2b1b6a640 1 --2- 172.21.15.38:0/1678651090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2ac0b2ab0 0x7fd2ac1f2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.151+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/1678651090 >> 172.21.15.38:0/1678651090 conn(0x7fd2ac066d10 msgr2=0x7fd2ac0d0780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:46.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.151+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/1678651090 shutdown_connections 2024-04-25T08:32:46.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.151+0000 7fd2b1b6a640 1 -- 172.21.15.38:0/1678651090 wait complete. 2024-04-25T08:32:46.158 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T08:32:46.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.264+0000 7f84ec688640 1 Processor -- start 2024-04-25T08:32:46.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.264+0000 7f84ec688640 1 -- start start 2024-04-25T08:32:46.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84ec688640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f84e415a780 0x7f84e40cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84ec688640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f84e40b3a40 0x7f84e415e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84ec688640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e40cde90 con 0x7f84e40cd670 2024-04-25T08:32:46.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84ec688640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e40cb190 con 0x7f84e415a780 2024-04-25T08:32:46.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84ec688640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e40cb470 con 0x7f84e40b3a40 2024-04-25T08:32:46.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84ea424640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f84e415a780 0x7f84e40cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84ea424640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f84e415a780 0x7f84e40cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41920/0 (socket says 172.21.15.38:41920) 2024-04-25T08:32:46.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84ea424640 1 -- 172.21.15.38:0/2926137794 learned_addr learned my addr 172.21.15.38:0/2926137794 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:46.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84eac25640 1 --2- 172.21.15.38:0/2926137794 >> v2:172.21.15.107:3300/0 conn(0x7f84e40b3a40 0x7f84e415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84ea424640 1 -- 172.21.15.38:0/2926137794 >> v2:172.21.15.107:3300/0 conn(0x7f84e40b3a40 msgr2=0x7f84e415e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:46.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84ea424640 1 --2- 172.21.15.38:0/2926137794 >> v2:172.21.15.107:3300/0 conn(0x7f84e40b3a40 0x7f84e415e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84ea424640 1 -- 172.21.15.38:0/2926137794 >> v1:172.21.15.107:6789/0 conn(0x7f84e40cd670 legacy=0x7f84e40b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:46.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.265+0000 7f84ea424640 1 -- 172.21.15.38:0/2926137794 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e40cb750 con 0x7f84e415a780 2024-04-25T08:32:46.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.266+0000 7f84ea424640 1 --2- 172.21.15.38:0/2926137794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84e415a780 0x7f84e40cbf50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f84e415f3f0 tx=0x7f84e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b015e85e200f86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:46.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.266+0000 7f84e9422640 1 -- 172.21.15.38:0/2926137794 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0037070 con 0x7f84e415a780 2024-04-25T08:32:46.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.266+0000 7f84e9422640 1 -- 172.21.15.38:0/2926137794 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e0032440 con 0x7f84e415a780 2024-04-25T08:32:46.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.266+0000 7f84e9422640 1 -- 172.21.15.38:0/2926137794 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0032810 con 0x7f84e415a780 2024-04-25T08:32:46.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.266+0000 7f84ec688640 1 -- 172.21.15.38:0/2926137794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84e415a780 msgr2=0x7f84e40cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.266+0000 7f84ec688640 1 --2- 172.21.15.38:0/2926137794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84e415a780 0x7f84e40cbf50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f84e415f3f0 tx=0x7f84e002a230 comp rx=0 tx=0).stop 2024-04-25T08:32:46.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.267+0000 7f84ec688640 1 -- 172.21.15.38:0/2926137794 shutdown_connections 2024-04-25T08:32:46.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.267+0000 7f84ec688640 1 --2- 172.21.15.38:0/2926137794 >> v2:172.21.15.107:3300/0 conn(0x7f84e40b3a40 0x7f84e415e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.267+0000 7f84ec688640 1 --2- 172.21.15.38:0/2926137794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84e415a780 0x7f84e40cbf50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.267+0000 7f84ec688640 1 -- 172.21.15.38:0/2926137794 >> 172.21.15.38:0/2926137794 conn(0x7f84e40b1ff0 msgr2=0x7f84e40b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:46.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.267+0000 7f84ec688640 1 -- 172.21.15.38:0/2926137794 shutdown_connections 2024-04-25T08:32:46.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.267+0000 7f84ec688640 1 -- 172.21.15.38:0/2926137794 wait complete. 2024-04-25T08:32:46.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.268+0000 7f84ec688640 1 Processor -- start 2024-04-25T08:32:46.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.268+0000 7f84ec688640 1 -- start start 2024-04-25T08:32:46.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.268+0000 7f84ec688640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e40b3a40 0x7f84e40c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.268+0000 7f84ec688640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84e40cd670 0x7f84e415b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.268+0000 7f84ec688640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84e415a780 0x7f84e415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84ec688640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40cda90 con 0x7f84e415a780 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84ec688640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e415ab60 con 0x7f84e40b3a40 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84ec688640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40cdc10 con 0x7f84e40cd670 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84eac25640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84e415a780 0x7f84e415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84eac25640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84e415a780 0x7f84e415bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41934/0 (socket says 172.21.15.38:41934) 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84eac25640 1 -- 172.21.15.38:0/619235356 learned_addr learned my addr 172.21.15.38:0/619235356 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84e9c23640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84e40cd670 0x7f84e415b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84ea424640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e40b3a40 0x7f84e40c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84eac25640 1 -- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84e40cd670 msgr2=0x7f84e415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84eac25640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84e40cd670 0x7f84e415b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84eac25640 1 -- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e40b3a40 msgr2=0x7f84e40c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84eac25640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e40b3a40 0x7f84e40c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84eac25640 1 -- 172.21.15.38:0/619235356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e415db20 con 0x7f84e415a780 2024-04-25T08:32:46.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84ea424640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e40b3a40 0x7f84e40c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:46.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84eac25640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84e415a780 0x7f84e415bda0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00cc60 tx=0x7f84dc00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:46.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84daffd640 1 -- 172.21.15.38:0/619235356 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc003990 con 0x7f84e415a780 2024-04-25T08:32:46.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84daffd640 1 -- 172.21.15.38:0/619235356 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc003b30 con 0x7f84e415a780 2024-04-25T08:32:46.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84ec688640 1 -- 172.21.15.38:0/619235356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e415d8a0 con 0x7f84e415a780 2024-04-25T08:32:46.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84daffd640 1 -- 172.21.15.38:0/619235356 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc004020 con 0x7f84e415a780 2024-04-25T08:32:46.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.269+0000 7f84ec688640 1 -- 172.21.15.38:0/619235356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e415b340 con 0x7f84e415a780 2024-04-25T08:32:46.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.270+0000 7f84ec688640 1 -- 172.21.15.38:0/619235356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e404ef90 con 0x7f84e415a780 2024-04-25T08:32:46.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.271+0000 7f84daffd640 1 -- 172.21.15.38:0/619235356 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84dc0041c0 con 0x7f84e415a780 2024-04-25T08:32:46.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.271+0000 7f84daffd640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f84c806b4d0 0x7f84c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.271+0000 7f84daffd640 1 -- 172.21.15.38:0/619235356 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f84dc08a3f0 con 0x7f84e415a780 2024-04-25T08:32:46.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.271+0000 7f84ea424640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f84c806b4d0 0x7f84c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.272+0000 7f84ea424640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f84c806b4d0 0x7f84c806d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f84e0034000 tx=0x7f84e002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:46.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.275+0000 7f84daffd640 1 -- 172.21.15.38:0/619235356 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84e415d8a0 con 0x7f84e415a780 2024-04-25T08:32:46.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.526+0000 7f84ec688640 1 -- 172.21.15.38:0/619235356 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f84e4159f80 con 0x7f84e415a780 2024-04-25T08:32:46.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.528+0000 7f84daffd640 1 -- 172.21.15.38:0/619235356 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f84dc014030 con 0x7f84e415a780 2024-04-25T08:32:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.530+0000 7f84ec688640 1 -- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f84c806b4d0 msgr2=0x7f84c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.530+0000 7f84ec688640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f84c806b4d0 0x7f84c806d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f84e0034000 tx=0x7f84e002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.530+0000 7f84ec688640 1 -- 172.21.15.38:0/619235356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84e415a780 msgr2=0x7f84e415bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.530+0000 7f84ec688640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84e415a780 0x7f84e415bda0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f84dc00cc60 tx=0x7f84dc00a5c0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.530+0000 7f84ec688640 1 -- 172.21.15.38:0/619235356 shutdown_connections 2024-04-25T08:32:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.530+0000 7f84ec688640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84e415a780 0x7f84e415bda0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.530+0000 7f84ec688640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f84c806b4d0 0x7f84c806d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.530+0000 7f84ec688640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84e40cd670 0x7f84e415b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.530+0000 7f84ec688640 1 --2- 172.21.15.38:0/619235356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84e40b3a40 0x7f84e40c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.530+0000 7f84ec688640 1 -- 172.21.15.38:0/619235356 >> 172.21.15.38:0/619235356 conn(0x7f84e40b1ff0 msgr2=0x7f84e406e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.530+0000 7f84ec688640 1 -- 172.21.15.38:0/619235356 shutdown_connections 2024-04-25T08:32:46.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.530+0000 7f84ec688640 1 -- 172.21.15.38:0/619235356 wait complete. 2024-04-25T08:32:46.537 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:46.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.642+0000 7ff441c07640 1 Processor -- start 2024-04-25T08:32:46.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.642+0000 7ff441c07640 1 -- start start 2024-04-25T08:32:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.643+0000 7ff441c07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff43c15dc10 0x7ff43c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.643+0000 7ff441c07640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff43c0ced60 0x7ff43c0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.643+0000 7ff441c07640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff43c0d3e70 con 0x7ff43c0d3900 2024-04-25T08:32:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.643+0000 7ff441c07640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff43c15dff0 con 0x7ff43c0ced60 2024-04-25T08:32:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.643+0000 7ff441c07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0b6990 con 0x7ff43c15dc10 2024-04-25T08:32:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.643+0000 7ff43bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff43c0d3900 0x7ff43c0b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49608/0 (socket says 172.21.15.38:49608) 2024-04-25T08:32:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.643+0000 7ff43bfff640 1 -- 172.21.15.38:0/4174327633 learned_addr learned my addr 172.21.15.38:0/4174327633 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.644+0000 7ff43affd640 1 --2- 172.21.15.38:0/4174327633 >> v2:172.21.15.107:3300/0 conn(0x7ff43c0ced60 0x7ff43c0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.644+0000 7ff43b7fe640 1 --2- 172.21.15.38:0/4174327633 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff43c15dc10 0x7ff43c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.644+0000 7ff43a7fc640 1 -- 172.21.15.38:0/4174327633 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1682668021 0 0) 0x7ff43c0d3e70 con 0x7ff43c0d3900 2024-04-25T08:32:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.644+0000 7ff43a7fc640 1 -- 172.21.15.38:0/4174327633 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff424003520 con 0x7ff43c0d3900 2024-04-25T08:32:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.644+0000 7ff43affd640 1 -- 172.21.15.38:0/4174327633 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff43c15dc10 msgr2=0x7ff43c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.644+0000 7ff43affd640 1 --2- 172.21.15.38:0/4174327633 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff43c15dc10 0x7ff43c0ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.644+0000 7ff43affd640 1 -- 172.21.15.38:0/4174327633 >> v1:172.21.15.38:6789/0 conn(0x7ff43c0d3900 legacy=0x7ff43c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.644+0000 7ff43affd640 1 -- 172.21.15.38:0/4174327633 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0d4150 con 0x7ff43c0ced60 2024-04-25T08:32:46.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.645+0000 7ff43affd640 1 --2- 172.21.15.38:0/4174327633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c0ced60 0x7ff43c0d0850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff4300097a0 tx=0x7ff43002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6a6668cf2979ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.645+0000 7ff43a7fc640 1 -- 172.21.15.38:0/4174327633 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff430037070 con 0x7ff43c0ced60 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.645+0000 7ff43a7fc640 1 -- 172.21.15.38:0/4174327633 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43002ae50 con 0x7ff43c0ced60 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.645+0000 7ff43a7fc640 1 -- 172.21.15.38:0/4174327633 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff430032660 con 0x7ff43c0ced60 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.645+0000 7ff441c07640 1 -- 172.21.15.38:0/4174327633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c0ced60 msgr2=0x7ff43c0d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.645+0000 7ff441c07640 1 --2- 172.21.15.38:0/4174327633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c0ced60 0x7ff43c0d0850 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff4300097a0 tx=0x7ff43002a100 comp rx=0 tx=0).stop 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.645+0000 7ff441c07640 1 -- 172.21.15.38:0/4174327633 shutdown_connections 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.645+0000 7ff441c07640 1 --2- 172.21.15.38:0/4174327633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c0ced60 0x7ff43c0d0850 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.645+0000 7ff441c07640 1 --2- 172.21.15.38:0/4174327633 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff43c15dc10 0x7ff43c0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.645+0000 7ff441c07640 1 -- 172.21.15.38:0/4174327633 >> 172.21.15.38:0/4174327633 conn(0x7ff43c066d10 msgr2=0x7ff43c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.645+0000 7ff441c07640 1 -- 172.21.15.38:0/4174327633 shutdown_connections 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.645+0000 7ff441c07640 1 -- 172.21.15.38:0/4174327633 wait complete. 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.647+0000 7ff441c07640 1 Processor -- start 2024-04-25T08:32:46.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.647+0000 7ff441c07640 1 -- start start 2024-04-25T08:32:46.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.647+0000 7ff441c07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff43c0ced60 0x7ff43c15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.647+0000 7ff441c07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c0d3900 0x7ff43c0ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.647+0000 7ff441c07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff43c15dc10 0x7ff43c15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.647+0000 7ff441c07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0b6bc0 con 0x7ff43c15dc10 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.647+0000 7ff441c07640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0d4710 con 0x7ff43c0d3900 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.647+0000 7ff441c07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c0d0d90 con 0x7ff43c0ced60 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff43bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff43c15dc10 0x7ff43c15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff43bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff43c15dc10 0x7ff43c15e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41936/0 (socket says 172.21.15.38:41936) 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff43bfff640 1 -- 172.21.15.38:0/2176112934 learned_addr learned my addr 172.21.15.38:0/2176112934 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff43affd640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c0d3900 0x7ff43c0ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff43b7fe640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff43c0ced60 0x7ff43c15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff43bfff640 1 -- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff43c0ced60 msgr2=0x7ff43c15c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff43bfff640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff43c0ced60 0x7ff43c15c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff43bfff640 1 -- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c0d3900 msgr2=0x7ff43c0ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff43bfff640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c0d3900 0x7ff43c0ac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff43bfff640 1 -- 172.21.15.38:0/2176112934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff43c0aa390 con 0x7ff43c15dc10 2024-04-25T08:32:46.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff43affd640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c0d3900 0x7ff43c0ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:46.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff43bfff640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff43c15dc10 0x7ff43c15e3a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff43400cb40 tx=0x7ff43400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:46.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff423fff640 1 -- 172.21.15.38:0/2176112934 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434017070 con 0x7ff43c15dc10 2024-04-25T08:32:46.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff423fff640 1 -- 172.21.15.38:0/2176112934 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43400aca0 con 0x7ff43c15dc10 2024-04-25T08:32:46.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff441c07640 1 -- 172.21.15.38:0/2176112934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43c0aa670 con 0x7ff43c15dc10 2024-04-25T08:32:46.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff423fff640 1 -- 172.21.15.38:0/2176112934 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434012620 con 0x7ff43c15dc10 2024-04-25T08:32:46.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.648+0000 7ff441c07640 1 -- 172.21.15.38:0/2176112934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c0aac30 con 0x7ff43c15dc10 2024-04-25T08:32:46.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.650+0000 7ff441c07640 1 -- 172.21.15.38:0/2176112934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c068e80 con 0x7ff43c15dc10 2024-04-25T08:32:46.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.650+0000 7ff423fff640 1 -- 172.21.15.38:0/2176112934 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff43401a070 con 0x7ff43c15dc10 2024-04-25T08:32:46.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.650+0000 7ff423fff640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff42806b500 0x7ff42806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.651+0000 7ff423fff640 1 -- 172.21.15.38:0/2176112934 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff434089f30 con 0x7ff43c15dc10 2024-04-25T08:32:46.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.651+0000 7ff43b7fe640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff42806b500 0x7ff42806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.651+0000 7ff43b7fe640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff42806b500 0x7ff42806d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009a30 tx=0x7ff42c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:46.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.655+0000 7ff423fff640 1 -- 172.21.15.38:0/2176112934 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff434058ee0 con 0x7ff43c15dc10 2024-04-25T08:32:46.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.837+0000 7ff441c07640 1 -- 172.21.15.38:0/2176112934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff43c0d2840 con 0x7ff43c15dc10 2024-04-25T08:32:46.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.838+0000 7ff423fff640 1 -- 172.21.15.38:0/2176112934 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13152 (secure 0 0 0) 0x7ff4340587e0 con 0x7ff43c15dc10 2024-04-25T08:32:46.834 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:46.834 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":34,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:43.875698+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:46.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.839+0000 7ff441c07640 1 -- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff42806b500 msgr2=0x7ff42806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.839+0000 7ff441c07640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff42806b500 0x7ff42806d9c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009a30 tx=0x7ff42c008040 comp rx=0 tx=0).stop 2024-04-25T08:32:46.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.839+0000 7ff441c07640 1 -- 172.21.15.38:0/2176112934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff43c15dc10 msgr2=0x7ff43c15e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.839+0000 7ff441c07640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff43c15dc10 0x7ff43c15e3a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff43400cb40 tx=0x7ff43400a060 comp rx=0 tx=0).stop 2024-04-25T08:32:46.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.839+0000 7ff441c07640 1 -- 172.21.15.38:0/2176112934 shutdown_connections 2024-04-25T08:32:46.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.839+0000 7ff441c07640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff43c15dc10 0x7ff43c15e3a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.839+0000 7ff441c07640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff43c0d3900 0x7ff43c0ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.839+0000 7ff441c07640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff42806b500 0x7ff42806d9c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.840+0000 7ff441c07640 1 --2- 172.21.15.38:0/2176112934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff43c0ced60 0x7ff43c15c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.840+0000 7ff441c07640 1 -- 172.21.15.38:0/2176112934 >> 172.21.15.38:0/2176112934 conn(0x7ff43c066d10 msgr2=0x7ff43c0d3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:46.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.840+0000 7ff441c07640 1 -- 172.21.15.38:0/2176112934 shutdown_connections 2024-04-25T08:32:46.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.840+0000 7ff441c07640 1 -- 172.21.15.38:0/2176112934 wait complete. 2024-04-25T08:32:46.847 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:32:46.847 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:32:46.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.954+0000 7f878b815640 1 Processor -- start 2024-04-25T08:32:46.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.954+0000 7f878b815640 1 -- start start 2024-04-25T08:32:46.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.955+0000 7f878b815640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f878406c700 0x7f8784162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.955+0000 7f878b815640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f878406cae0 0x7f878415e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.955+0000 7f878b815640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f878415e580 0x7f87840cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.955+0000 7f878b815640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8784161c80 con 0x7f878406c700 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.955+0000 7f878b815640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878415ec20 con 0x7f878406cae0 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.955+0000 7f878b815640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878415ef00 con 0x7f878415e580 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.955+0000 7f87895b1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f878406c700 0x7f8784162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.955+0000 7f87895b1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f878406c700 0x7f8784162110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41944/0 (socket says 172.21.15.38:41944) 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.955+0000 7f87895b1640 1 -- 172.21.15.38:0/1861563997 learned_addr learned my addr 172.21.15.38:0/1861563997 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.956+0000 7f8788db0640 1 --2- 172.21.15.38:0/1861563997 >> v2:172.21.15.107:3300/0 conn(0x7f878406cae0 0x7f878415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.956+0000 7f8789db2640 1 --2- 172.21.15.38:0/1861563997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f878415e580 0x7f87840cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.956+0000 7f87895b1640 1 -- 172.21.15.38:0/1861563997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f878415e580 msgr2=0x7f87840cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.956+0000 7f87895b1640 1 --2- 172.21.15.38:0/1861563997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f878415e580 0x7f87840cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.956+0000 7f87895b1640 1 -- 172.21.15.38:0/1861563997 >> v2:172.21.15.107:3300/0 conn(0x7f878406cae0 msgr2=0x7f878415e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.956+0000 7f87895b1640 1 --2- 172.21.15.38:0/1861563997 >> v2:172.21.15.107:3300/0 conn(0x7f878406cae0 0x7f878415e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.956+0000 7f87895b1640 1 -- 172.21.15.38:0/1861563997 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8784161630 con 0x7f878406c700 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f87895b1640 1 --2- 172.21.15.38:0/1861563997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f878406c700 0x7f8784162110 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f87840b3050 tx=0x7f877c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39edebbc7e6858b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f8783fff640 1 -- 172.21.15.38:0/1861563997 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c037070 con 0x7f878406c700 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f8783fff640 1 -- 172.21.15.38:0/1861563997 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877c032440 con 0x7f878406c700 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f8783fff640 1 -- 172.21.15.38:0/1861563997 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877c032810 con 0x7f878406c700 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f878b815640 1 -- 172.21.15.38:0/1861563997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f878406c700 msgr2=0x7f8784162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f878b815640 1 --2- 172.21.15.38:0/1861563997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f878406c700 0x7f8784162110 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f87840b3050 tx=0x7f877c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f878b815640 1 -- 172.21.15.38:0/1861563997 shutdown_connections 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f878b815640 1 --2- 172.21.15.38:0/1861563997 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f878415e580 0x7f87840cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f878b815640 1 --2- 172.21.15.38:0/1861563997 >> v2:172.21.15.107:3300/0 conn(0x7f878406cae0 0x7f878415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f878b815640 1 --2- 172.21.15.38:0/1861563997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f878406c700 0x7f8784162110 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f878b815640 1 -- 172.21.15.38:0/1861563997 >> 172.21.15.38:0/1861563997 conn(0x7f878408b000 msgr2=0x7f878408b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f878b815640 1 -- 172.21.15.38:0/1861563997 shutdown_connections 2024-04-25T08:32:46.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.957+0000 7f878b815640 1 -- 172.21.15.38:0/1861563997 wait complete. 2024-04-25T08:32:46.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.959+0000 7f878b815640 1 Processor -- start 2024-04-25T08:32:46.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.959+0000 7f878b815640 1 -- start start 2024-04-25T08:32:46.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.959+0000 7f878b815640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f878406c700 0x7f8784151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.959+0000 7f878b815640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f878406cae0 0x7f87840b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.959+0000 7f878b815640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878415e580 0x7f878406f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.959+0000 7f878b815640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878406cf90 con 0x7f878406cae0 2024-04-25T08:32:46.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.959+0000 7f878b815640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878406c190 con 0x7f878415e580 2024-04-25T08:32:46.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.959+0000 7f878b815640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878415b920 con 0x7f878406c700 2024-04-25T08:32:46.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.959+0000 7f8788db0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f878406cae0 0x7f87840b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.959+0000 7f8788db0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f878406cae0 0x7f87840b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41952/0 (socket says 172.21.15.38:41952) 2024-04-25T08:32:46.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.959+0000 7f8788db0640 1 -- 172.21.15.38:0/967702554 learned_addr learned my addr 172.21.15.38:0/967702554 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:46.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f87895b1640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f878406c700 0x7f8784151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f8789db2640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878415e580 0x7f878406f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f8788db0640 1 -- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f878406c700 msgr2=0x7f8784151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f8788db0640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f878406c700 0x7f8784151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f8788db0640 1 -- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878415e580 msgr2=0x7f878406f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:46.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f8788db0640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878415e580 0x7f878406f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:46.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f8788db0640 1 -- 172.21.15.38:0/967702554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87840aedd0 con 0x7f878406cae0 2024-04-25T08:32:46.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f8789db2640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878415e580 0x7f878406f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:46.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f87895b1640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f878406c700 0x7f8784151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:46.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f8788db0640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f878406cae0 0x7f87840b4240 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f877800e890 tx=0x7f877800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:46.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f8781ffb640 1 -- 172.21.15.38:0/967702554 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8778017070 con 0x7f878406cae0 2024-04-25T08:32:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f8781ffb640 1 -- 172.21.15.38:0/967702554 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f877800c940 con 0x7f878406cae0 2024-04-25T08:32:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f878b815640 1 -- 172.21.15.38:0/967702554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87840af0b0 con 0x7f878406cae0 2024-04-25T08:32:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f8781ffb640 1 -- 172.21.15.38:0/967702554 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f877800cd10 con 0x7f878406cae0 2024-04-25T08:32:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.960+0000 7f878b815640 1 -- 172.21.15.38:0/967702554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87840af670 con 0x7f878406cae0 2024-04-25T08:32:46.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.961+0000 7f878b815640 1 -- 172.21.15.38:0/967702554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8784068e80 con 0x7f878406cae0 2024-04-25T08:32:46.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.962+0000 7f8781ffb640 1 -- 172.21.15.38:0/967702554 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f877801a070 con 0x7f878406cae0 2024-04-25T08:32:46.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.962+0000 7f8781ffb640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f877006b4d0 0x7f877006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:46.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.962+0000 7f8781ffb640 1 -- 172.21.15.38:0/967702554 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f877808a430 con 0x7f878406cae0 2024-04-25T08:32:46.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.963+0000 7f87895b1640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f877006b4d0 0x7f877006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:46.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.963+0000 7f87895b1640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f877006b4d0 0x7f877006d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f877c034000 tx=0x7f877c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:46.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:46.967+0000 7f8781ffb640 1 -- 172.21.15.38:0/967702554 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87780593d0 con 0x7f878406cae0 2024-04-25T08:32:47.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.146+0000 7f878b815640 1 -- 172.21.15.38:0/967702554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87840b1220 con 0x7f878406cae0 2024-04-25T08:32:47.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.146+0000 7f8781ffb640 1 -- 172.21.15.38:0/967702554 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v34) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8778058cd0 con 0x7f878406cae0 2024-04-25T08:32:47.142 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:32:47.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.148+0000 7f878b815640 1 -- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f877006b4d0 msgr2=0x7f877006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.148+0000 7f878b815640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f877006b4d0 0x7f877006d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f877c034000 tx=0x7f877c02e000 comp rx=0 tx=0).stop 2024-04-25T08:32:47.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.148+0000 7f878b815640 1 -- 172.21.15.38:0/967702554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f878406cae0 msgr2=0x7f87840b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.148+0000 7f878b815640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f878406cae0 0x7f87840b4240 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f877800e890 tx=0x7f877800ed50 comp rx=0 tx=0).stop 2024-04-25T08:32:47.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.148+0000 7f878b815640 1 -- 172.21.15.38:0/967702554 shutdown_connections 2024-04-25T08:32:47.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.148+0000 7f878b815640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f878415e580 0x7f878406f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.148+0000 7f878b815640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f878406cae0 0x7f87840b4240 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.148+0000 7f878b815640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f877006b4d0 0x7f877006d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.148+0000 7f878b815640 1 --2- 172.21.15.38:0/967702554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f878406c700 0x7f8784151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.148+0000 7f878b815640 1 -- 172.21.15.38:0/967702554 >> 172.21.15.38:0/967702554 conn(0x7f878408b000 msgr2=0x7f87840b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:47.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.148+0000 7f878b815640 1 -- 172.21.15.38:0/967702554 shutdown_connections 2024-04-25T08:32:47.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.148+0000 7f878b815640 1 -- 172.21.15.38:0/967702554 wait complete. 2024-04-25T08:32:47.155 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:32:47.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.261+0000 7f5583d29640 1 Processor -- start 2024-04-25T08:32:47.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.261+0000 7f5583d29640 1 -- start start 2024-04-25T08:32:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f5583d29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557c06c110 0x7f557c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f5583d29640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f557c15df50 0x7f557c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f5583d29640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f557c0ce2e0 0x7f557c0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f5583d29640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f557c0cf2d0 con 0x7f557c0ce2e0 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f5583d29640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f557c06ca30 con 0x7f557c15df50 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f5583d29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557c0ceff0 con 0x7f557c06c110 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f55822c6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f557c0ce2e0 0x7f557c0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f55822c6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f557c0ce2e0 0x7f557c0d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41956/0 (socket says 172.21.15.38:41956) 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f55822c6640 1 -- 172.21.15.38:0/3086995813 learned_addr learned my addr 172.21.15.38:0/3086995813 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f5581ac5640 1 --2- 172.21.15.38:0/3086995813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557c06c110 0x7f557c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f55822c6640 1 -- 172.21.15.38:0/3086995813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557c06c110 msgr2=0x7f557c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f55812c4640 1 --2- 172.21.15.38:0/3086995813 >> v2:172.21.15.107:3300/0 conn(0x7f557c15df50 0x7f557c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f55822c6640 1 --2- 172.21.15.38:0/3086995813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557c06c110 0x7f557c06c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f55822c6640 1 -- 172.21.15.38:0/3086995813 >> v2:172.21.15.107:3300/0 conn(0x7f557c15df50 msgr2=0x7f557c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f55822c6640 1 --2- 172.21.15.38:0/3086995813 >> v2:172.21.15.107:3300/0 conn(0x7f557c15df50 0x7f557c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f55822c6640 1 -- 172.21.15.38:0/3086995813 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c0d1f70 con 0x7f557c0ce2e0 2024-04-25T08:32:47.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.262+0000 7f55812c4640 1 --2- 172.21.15.38:0/3086995813 >> v2:172.21.15.107:3300/0 conn(0x7f557c15df50 0x7f557c0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:47.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.263+0000 7f55822c6640 1 --2- 172.21.15.38:0/3086995813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f557c0ce2e0 0x7f557c0d2a50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f557c08c2f0 tx=0x7f556c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1578200aad52c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.263+0000 7f5580ac3640 1 -- 172.21.15.38:0/3086995813 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c036070 con 0x7f557c0ce2e0 2024-04-25T08:32:47.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.263+0000 7f5580ac3640 1 -- 172.21.15.38:0/3086995813 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556c031600 con 0x7f557c0ce2e0 2024-04-25T08:32:47.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.263+0000 7f5580ac3640 1 -- 172.21.15.38:0/3086995813 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c0319d0 con 0x7f557c0ce2e0 2024-04-25T08:32:47.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.264+0000 7f5583d29640 1 -- 172.21.15.38:0/3086995813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f557c0ce2e0 msgr2=0x7f557c0d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.264+0000 7f5583d29640 1 --2- 172.21.15.38:0/3086995813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f557c0ce2e0 0x7f557c0d2a50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f557c08c2f0 tx=0x7f556c02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.264+0000 7f5583d29640 1 -- 172.21.15.38:0/3086995813 shutdown_connections 2024-04-25T08:32:47.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.264+0000 7f5583d29640 1 --2- 172.21.15.38:0/3086995813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f557c0ce2e0 0x7f557c0d2a50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.264+0000 7f5583d29640 1 --2- 172.21.15.38:0/3086995813 >> v2:172.21.15.107:3300/0 conn(0x7f557c15df50 0x7f557c0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.264+0000 7f5583d29640 1 --2- 172.21.15.38:0/3086995813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557c06c110 0x7f557c06c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.264+0000 7f5583d29640 1 -- 172.21.15.38:0/3086995813 >> 172.21.15.38:0/3086995813 conn(0x7f557c0d3fb0 msgr2=0x7f557c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:47.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.264+0000 7f5583d29640 1 -- 172.21.15.38:0/3086995813 shutdown_connections 2024-04-25T08:32:47.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.264+0000 7f5583d29640 1 -- 172.21.15.38:0/3086995813 wait complete. 2024-04-25T08:32:47.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.265+0000 7f5583d29640 1 Processor -- start 2024-04-25T08:32:47.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.265+0000 7f5583d29640 1 -- start start 2024-04-25T08:32:47.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f5583d29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557c06c110 0x7f557c160bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f5583d29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f557c0ce2e0 0x7f557c0cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f5583d29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f557c15df50 0x7f557c0c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f5583d29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c0cece0 con 0x7f557c15df50 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f5583d29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557c0d2530 con 0x7f557c0ce2e0 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f5583d29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557c0d2f90 con 0x7f557c06c110 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f55822c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f557c15df50 0x7f557c0c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f55822c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f557c15df50 0x7f557c0c88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41964/0 (socket says 172.21.15.38:41964) 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f55822c6640 1 -- 172.21.15.38:0/779432658 learned_addr learned my addr 172.21.15.38:0/779432658 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f5581ac5640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557c06c110 0x7f557c160bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f55812c4640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f557c0ce2e0 0x7f557c0cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f55822c6640 1 -- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557c06c110 msgr2=0x7f557c160bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f55822c6640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557c06c110 0x7f557c160bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f55822c6640 1 -- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f557c0ce2e0 msgr2=0x7f557c0cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f55822c6640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f557c0ce2e0 0x7f557c0cb2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f55822c6640 1 -- 172.21.15.38:0/779432658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c15e3d0 con 0x7f557c15df50 2024-04-25T08:32:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.266+0000 7f55822c6640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f557c15df50 0x7f557c0c88b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f556c0093e0 tx=0x7f556c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.267+0000 7f557a7fc640 1 -- 172.21.15.38:0/779432658 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c041020 con 0x7f557c15df50 2024-04-25T08:32:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.267+0000 7f557a7fc640 1 -- 172.21.15.38:0/779432658 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556c007720 con 0x7f557c15df50 2024-04-25T08:32:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.267+0000 7f5583d29640 1 -- 172.21.15.38:0/779432658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557c0cbb20 con 0x7f557c15df50 2024-04-25T08:32:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.267+0000 7f557a7fc640 1 -- 172.21.15.38:0/779432658 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556c036040 con 0x7f557c15df50 2024-04-25T08:32:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.267+0000 7f5583d29640 1 -- 172.21.15.38:0/779432658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557c161100 con 0x7f557c15df50 2024-04-25T08:32:47.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.268+0000 7f5583d29640 1 -- 172.21.15.38:0/779432658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f557c068e80 con 0x7f557c15df50 2024-04-25T08:32:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.268+0000 7f557a7fc640 1 -- 172.21.15.38:0/779432658 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f556c044020 con 0x7f557c15df50 2024-04-25T08:32:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.269+0000 7f557a7fc640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f556806b500 0x7f556806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.269+0000 7f557a7fc640 1 -- 172.21.15.38:0/779432658 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f556c0aaca0 con 0x7f557c15df50 2024-04-25T08:32:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.269+0000 7f5581ac5640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f556806b500 0x7f556806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.269+0000 7f5581ac5640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f556806b500 0x7f556806d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f557400aaf0 tx=0x7f5574008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.273+0000 7f557a7fc640 1 -- 172.21.15.38:0/779432658 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f556c04b030 con 0x7f557c15df50 2024-04-25T08:32:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.457+0000 7f5583d29640 1 -- 172.21.15.38:0/779432658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f557c0cb810 con 0x7f557c15df50 2024-04-25T08:32:47.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.458+0000 7f557a7fc640 1 -- 172.21.15.38:0/779432658 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v34) v1 ==== 84+0+65 (secure 0 0 0) 0x7f556c079720 con 0x7f557c15df50 2024-04-25T08:32:47.454 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:47.454 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T08:32:47.454 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:32:47.455 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-04-25T08:32:47.455 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-04-25T08:32:47.455 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:47.455 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T08:32:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.460+0000 7f5583d29640 1 -- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f556806b500 msgr2=0x7f556806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.460+0000 7f5583d29640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f556806b500 0x7f556806d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f557400aaf0 tx=0x7f5574008040 comp rx=0 tx=0).stop 2024-04-25T08:32:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.460+0000 7f5583d29640 1 -- 172.21.15.38:0/779432658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f557c15df50 msgr2=0x7f557c0c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.460+0000 7f5583d29640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f557c15df50 0x7f557c0c88b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f556c0093e0 tx=0x7f556c004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.460+0000 7f5583d29640 1 -- 172.21.15.38:0/779432658 shutdown_connections 2024-04-25T08:32:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.460+0000 7f5583d29640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f557c15df50 0x7f557c0c88b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.460+0000 7f5583d29640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f557c0ce2e0 0x7f557c0cb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.460+0000 7f5583d29640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f556806b500 0x7f556806d9c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.460+0000 7f5583d29640 1 --2- 172.21.15.38:0/779432658 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557c06c110 0x7f557c160bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.460+0000 7f5583d29640 1 -- 172.21.15.38:0/779432658 >> 172.21.15.38:0/779432658 conn(0x7f557c0d3fb0 msgr2=0x7f557c15ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:47.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.460+0000 7f5583d29640 1 -- 172.21.15.38:0/779432658 shutdown_connections 2024-04-25T08:32:47.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.460+0000 7f5583d29640 1 -- 172.21.15.38:0/779432658 wait complete. 2024-04-25T08:32:47.467 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:32:47.467 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:32:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.575+0000 7f1a0d1f9640 1 Processor -- start 2024-04-25T08:32:47.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.575+0000 7f1a0d1f9640 1 -- start start 2024-04-25T08:32:47.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.576+0000 7f1a0d1f9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a080d0860 0x7f1a080d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.576+0000 7f1a0d1f9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1a080d1e20 0x7f1a080d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.576+0000 7f1a0d1f9640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a080d1180 con 0x7f1a080d19e0 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.576+0000 7f1a0d1f9640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0815e2b0 con 0x7f1a080d1e20 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.576+0000 7f1a0d1f9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080d2250 con 0x7f1a080d0860 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.576+0000 7f1a0759e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1a080d1e20 0x7f1a080d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.576+0000 7f1a0659c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1a080d19e0 0x7f1a080d3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49622/0 (socket says 172.21.15.38:49622) 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.576+0000 7f1a0659c640 1 -- 172.21.15.38:0/981809965 learned_addr learned my addr 172.21.15.38:0/981809965 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.577+0000 7f1a06d9d640 1 --2- 172.21.15.38:0/981809965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a080d0860 0x7f1a080d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.577+0000 7f1a0759e640 1 -- 172.21.15.38:0/981809965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a080d0860 msgr2=0x7f1a080d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.577+0000 7f1a0759e640 1 --2- 172.21.15.38:0/981809965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a080d0860 0x7f1a080d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.577+0000 7f1a0759e640 1 -- 172.21.15.38:0/981809965 >> v1:172.21.15.38:6789/0 conn(0x7f1a080d19e0 legacy=0x7f1a080d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.577+0000 7f1a0759e640 1 -- 172.21.15.38:0/981809965 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a080d24a0 con 0x7f1a080d1e20 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.577+0000 7f1a05d9b640 1 -- 172.21.15.38:0/981809965 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889121346 0 0) 0x7f1a080d1180 con 0x7f1a080d19e0 2024-04-25T08:32:47.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.577+0000 7f1a06d9d640 1 --2- 172.21.15.38:0/981809965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a080d0860 0x7f1a080d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:47.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.577+0000 7f1a0759e640 1 --2- 172.21.15.38:0/981809965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a080d1e20 0x7f1a080d3f70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009d90 tx=0x7f19f802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef06aa6601233acd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.578+0000 7f1a05d9b640 1 -- 172.21.15.38:0/981809965 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8007740 con 0x7f1a080d1e20 2024-04-25T08:32:47.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.578+0000 7f1a05d9b640 1 -- 172.21.15.38:0/981809965 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19f80078e0 con 0x7f1a080d1e20 2024-04-25T08:32:47.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.578+0000 7f1a05d9b640 1 -- 172.21.15.38:0/981809965 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8007cb0 con 0x7f1a080d1e20 2024-04-25T08:32:47.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.578+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/981809965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a080d1e20 msgr2=0x7f1a080d3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.578+0000 7f1a0d1f9640 1 --2- 172.21.15.38:0/981809965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a080d1e20 0x7f1a080d3f70 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009d90 tx=0x7f19f802a660 comp rx=0 tx=0).stop 2024-04-25T08:32:47.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.578+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/981809965 shutdown_connections 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.578+0000 7f1a0d1f9640 1 --2- 172.21.15.38:0/981809965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a080d1e20 0x7f1a080d3f70 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.578+0000 7f1a0d1f9640 1 --2- 172.21.15.38:0/981809965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a080d0860 0x7f1a080d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.578+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/981809965 >> 172.21.15.38:0/981809965 conn(0x7f1a08066d10 msgr2=0x7f1a08067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.578+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/981809965 shutdown_connections 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.578+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/981809965 wait complete. 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.580+0000 7f1a0d1f9640 1 Processor -- start 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.580+0000 7f1a0d1f9640 1 -- start start 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.580+0000 7f1a0d1f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a080d0860 0x7f1a080c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.580+0000 7f1a0d1f9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a080d19e0 0x7f1a081f3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.580+0000 7f1a0d1f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a080d1e20 0x7f1a080cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.580+0000 7f1a0d1f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080d2d40 con 0x7f1a080d0860 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.580+0000 7f1a0d1f9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080d2a60 con 0x7f1a080d1e20 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.580+0000 7f1a0d1f9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a080d3540 con 0x7f1a080d19e0 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.580+0000 7f1a06d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a080d0860 0x7f1a080c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.580+0000 7f1a06d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a080d0860 0x7f1a080c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41982/0 (socket says 172.21.15.38:41982) 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.580+0000 7f1a06d9d640 1 -- 172.21.15.38:0/113714354 learned_addr learned my addr 172.21.15.38:0/113714354 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f1a0659c640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a080d19e0 0x7f1a081f3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f1a0759e640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a080d1e20 0x7f1a080cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f1a06d9d640 1 -- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a080d19e0 msgr2=0x7f1a081f3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f1a06d9d640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a080d19e0 0x7f1a081f3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f1a06d9d640 1 -- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a080d1e20 msgr2=0x7f1a080cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f1a06d9d640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a080d1e20 0x7f1a080cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f1a06d9d640 1 -- 172.21.15.38:0/113714354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a081f4290 con 0x7f1a080d0860 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f1a0659c640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a080d19e0 0x7f1a081f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f1a06d9d640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a080d0860 0x7f1a080c9e20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000c7f0 tx=0x7f1a0000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f19ff7fe640 1 -- 172.21.15.38:0/113714354 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00015070 con 0x7f1a080d0860 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f19ff7fe640 1 -- 172.21.15.38:0/113714354 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a00007a30 con 0x7f1a080d0860 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/113714354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a080cbc90 con 0x7f1a080d0860 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f19ff7fe640 1 -- 172.21.15.38:0/113714354 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00007e00 con 0x7f1a080d0860 2024-04-25T08:32:47.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.581+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/113714354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a080caed0 con 0x7f1a080d0860 2024-04-25T08:32:47.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.582+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/113714354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a080ca980 con 0x7f1a080d0860 2024-04-25T08:32:47.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.583+0000 7f19ff7fe640 1 -- 172.21.15.38:0/113714354 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a00018070 con 0x7f1a080d0860 2024-04-25T08:32:47.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.583+0000 7f19ff7fe640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f19e806b500 0x7f19e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:47.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.583+0000 7f19ff7fe640 1 -- 172.21.15.38:0/113714354 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1a00089f90 con 0x7f1a080d0860 2024-04-25T08:32:47.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.583+0000 7f1a0659c640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f19e806b500 0x7f19e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:47.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.584+0000 7f1a0659c640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f19e806b500 0x7f19e806d9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1a080cda30 tx=0x7f19f0007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:47.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.587+0000 7f19ff7fe640 1 -- 172.21.15.38:0/113714354 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a081f4290 con 0x7f1a080d0860 2024-04-25T08:32:47.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:47.767+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/113714354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1a08068e80 con 0x7f1a080d0860 2024-04-25T08:32:48.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.022+0000 7f19ff7fe640 1 -- 172.21.15.38:0/113714354 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1a00058fd0 con 0x7f1a080d0860 2024-04-25T08:32:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-04-25T08:32:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.024+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f19e806b500 msgr2=0x7f19e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.024+0000 7f1a0d1f9640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f19e806b500 0x7f19e806d9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1a080cda30 tx=0x7f19f0007c10 comp rx=0 tx=0).stop 2024-04-25T08:32:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.024+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/113714354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a080d0860 msgr2=0x7f1a080c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.024+0000 7f1a0d1f9640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a080d0860 0x7f1a080c9e20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000c7f0 tx=0x7f1a0000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.024+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/113714354 shutdown_connections 2024-04-25T08:32:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.024+0000 7f1a0d1f9640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a080d1e20 0x7f1a080cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.024+0000 7f1a0d1f9640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f19e806b500 0x7f19e806d9c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.024+0000 7f1a0d1f9640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a080d19e0 0x7f1a081f3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.024+0000 7f1a0d1f9640 1 --2- 172.21.15.38:0/113714354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a080d0860 0x7f1a080c9e20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.024+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/113714354 >> 172.21.15.38:0/113714354 conn(0x7f1a08066d10 msgr2=0x7f1a080b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:48.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.024+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/113714354 shutdown_connections 2024-04-25T08:32:48.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.024+0000 7f1a0d1f9640 1 -- 172.21.15.38:0/113714354 wait complete. 2024-04-25T08:32:48.035 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:32:48.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.148+0000 7fade9a6f640 1 Processor -- start 2024-04-25T08:32:48.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.148+0000 7fade9a6f640 1 -- start start 2024-04-25T08:32:48.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade9a6f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fade40d33d0 0x7fade40d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade9a6f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade40d1e90 0x7fade40b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade9a6f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade40d2330 con 0x7fade40d0320 2024-04-25T08:32:48.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade9a6f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade40d2610 con 0x7fade40d33d0 2024-04-25T08:32:48.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade9a6f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade40d28f0 con 0x7fade40d1e90 2024-04-25T08:32:48.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade2ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fade40d33d0 0x7fade40d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade2ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fade40d33d0 0x7fade40d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41984/0 (socket says 172.21.15.38:41984) 2024-04-25T08:32:48.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade2ffd640 1 -- 172.21.15.38:0/3275015903 learned_addr learned my addr 172.21.15.38:0/3275015903 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:48.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade3fff640 1 --2- 172.21.15.38:0/3275015903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade40d1e90 0x7fade40b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade2ffd640 1 -- 172.21.15.38:0/3275015903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade40d1e90 msgr2=0x7fade40b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:48.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade2ffd640 1 --2- 172.21.15.38:0/3275015903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade40d1e90 0x7fade40b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade2ffd640 1 -- 172.21.15.38:0/3275015903 >> v1:172.21.15.107:6789/0 conn(0x7fade40d0320 legacy=0x7fade40d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.149+0000 7fade2ffd640 1 -- 172.21.15.38:0/3275015903 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40d2bd0 con 0x7fade40d33d0 2024-04-25T08:32:48.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.150+0000 7fade2ffd640 1 --2- 172.21.15.38:0/3275015903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade40d33d0 0x7fade40d1950 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009870 tx=0x7fadd802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd86389cf7c783fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:48.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.150+0000 7fade27fc640 1 -- 172.21.15.38:0/3275015903 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8037070 con 0x7fade40d33d0 2024-04-25T08:32:48.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.150+0000 7fade27fc640 1 -- 172.21.15.38:0/3275015903 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd802ae50 con 0x7fade40d33d0 2024-04-25T08:32:48.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.150+0000 7fade27fc640 1 -- 172.21.15.38:0/3275015903 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8032860 con 0x7fade40d33d0 2024-04-25T08:32:48.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.150+0000 7fade9a6f640 1 -- 172.21.15.38:0/3275015903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade40d33d0 msgr2=0x7fade40d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.150+0000 7fade9a6f640 1 --2- 172.21.15.38:0/3275015903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade40d33d0 0x7fade40d1950 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009870 tx=0x7fadd802a300 comp rx=0 tx=0).stop 2024-04-25T08:32:48.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.151+0000 7fade9a6f640 1 -- 172.21.15.38:0/3275015903 shutdown_connections 2024-04-25T08:32:48.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.151+0000 7fade9a6f640 1 --2- 172.21.15.38:0/3275015903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade40d1e90 0x7fade40b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.151+0000 7fade9a6f640 1 --2- 172.21.15.38:0/3275015903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade40d33d0 0x7fade40d1950 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.151+0000 7fade9a6f640 1 -- 172.21.15.38:0/3275015903 >> 172.21.15.38:0/3275015903 conn(0x7fade40668f0 msgr2=0x7fade4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:48.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.151+0000 7fade9a6f640 1 -- 172.21.15.38:0/3275015903 shutdown_connections 2024-04-25T08:32:48.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.151+0000 7fade9a6f640 1 -- 172.21.15.38:0/3275015903 wait complete. 2024-04-25T08:32:48.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.152+0000 7fade9a6f640 1 Processor -- start 2024-04-25T08:32:48.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.152+0000 7fade9a6f640 1 -- start start 2024-04-25T08:32:48.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade9a6f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade40d0320 0x7fade40b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade9a6f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade40d1e90 0x7fade40b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade9a6f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade40d33d0 0x7fade40ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade9a6f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade4161390 con 0x7fade40d0320 2024-04-25T08:32:48.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade9a6f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade4161210 con 0x7fade40d1e90 2024-04-25T08:32:48.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade9a6f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade40b9720 con 0x7fade40d33d0 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade40d0320 0x7fade40b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade40d0320 0x7fade40b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41986/0 (socket says 172.21.15.38:41986) 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade37fe640 1 -- 172.21.15.38:0/2820412085 learned_addr learned my addr 172.21.15.38:0/2820412085 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade2ffd640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade40d1e90 0x7fade40b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade3fff640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade40d33d0 0x7fade40ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade37fe640 1 -- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade40d33d0 msgr2=0x7fade40ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade37fe640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade40d33d0 0x7fade40ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade37fe640 1 -- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade40d1e90 msgr2=0x7fade40b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade37fe640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade40d1e90 0x7fade40b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade37fe640 1 -- 172.21.15.38:0/2820412085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40aa2a0 con 0x7fade40d0320 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade37fe640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade40d0320 0x7fade40b3c80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fadd400c430 tx=0x7fadd400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fadcbfff640 1 -- 172.21.15.38:0/2820412085 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd40052d0 con 0x7fade40d0320 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fadcbfff640 1 -- 172.21.15.38:0/2820412085 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd4005490 con 0x7fade40d0320 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fade9a6f640 1 -- 172.21.15.38:0/2820412085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade40aa580 con 0x7fade40d0320 2024-04-25T08:32:48.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.153+0000 7fadcbfff640 1 -- 172.21.15.38:0/2820412085 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd4007260 con 0x7fade40d0320 2024-04-25T08:32:48.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.154+0000 7fade9a6f640 1 -- 172.21.15.38:0/2820412085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade40aab40 con 0x7fade40d0320 2024-04-25T08:32:48.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.154+0000 7fade9a6f640 1 -- 172.21.15.38:0/2820412085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade40b2b50 con 0x7fade40d0320 2024-04-25T08:32:48.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.155+0000 7fadcbfff640 1 -- 172.21.15.38:0/2820412085 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadd4007400 con 0x7fade40d0320 2024-04-25T08:32:48.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.156+0000 7fadcbfff640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fadd006b500 0x7fadd006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:48.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.156+0000 7fadcbfff640 1 -- 172.21.15.38:0/2820412085 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fadd4089b90 con 0x7fade40d0320 2024-04-25T08:32:48.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.156+0000 7fade2ffd640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fadd006b500 0x7fadd006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:48.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.156+0000 7fade2ffd640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fadd006b500 0x7fadd006d9c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fadd8034000 tx=0x7fadd802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:48.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.160+0000 7fadcbfff640 1 -- 172.21.15.38:0/2820412085 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fade40b2b50 con 0x7fade40d0320 2024-04-25T08:32:48.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:48.340+0000 7fade9a6f640 1 -- 172.21.15.38:0/2820412085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fade4068e80 con 0x7fade40d0320 2024-04-25T08:32:49.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.024+0000 7fadcbfff640 1 -- 172.21.15.38:0/2820412085 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v36) v1 ==== 170+0+0 (secure 0 0 0) 0x7fadd40583f0 con 0x7fade40d0320 2024-04-25T08:32:49.020 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-04-25T08:32:49.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.028+0000 7fade9a6f640 1 -- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fadd006b500 msgr2=0x7fadd006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.028+0000 7fade9a6f640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fadd006b500 0x7fadd006d9c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fadd8034000 tx=0x7fadd802e000 comp rx=0 tx=0).stop 2024-04-25T08:32:49.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.028+0000 7fade9a6f640 1 -- 172.21.15.38:0/2820412085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade40d0320 msgr2=0x7fade40b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.028+0000 7fade9a6f640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade40d0320 0x7fade40b3c80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fadd400c430 tx=0x7fadd400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.028+0000 7fade9a6f640 1 -- 172.21.15.38:0/2820412085 shutdown_connections 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.028+0000 7fade9a6f640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fadd006b500 0x7fadd006d9c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.028+0000 7fade9a6f640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fade40d33d0 0x7fade40ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.028+0000 7fade9a6f640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fade40d1e90 0x7fade40b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.028+0000 7fade9a6f640 1 --2- 172.21.15.38:0/2820412085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fade40d0320 0x7fade40b3c80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.028+0000 7fade9a6f640 1 -- 172.21.15.38:0/2820412085 >> 172.21.15.38:0/2820412085 conn(0x7fade40668f0 msgr2=0x7fade40b8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.028+0000 7fade9a6f640 1 -- 172.21.15.38:0/2820412085 shutdown_connections 2024-04-25T08:32:49.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.028+0000 7fade9a6f640 1 -- 172.21.15.38:0/2820412085 wait complete. 2024-04-25T08:32:49.047 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:32:49.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.160+0000 7f30b0bd5640 1 Processor -- start 2024-04-25T08:32:49.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.160+0000 7f30b0bd5640 1 -- start start 2024-04-25T08:32:49.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.161+0000 7f30b0bd5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f30ac0d1d20 0x7f30ac0cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.161+0000 7f30b0bd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30ac15f5f0 0x7f30ac0b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.161+0000 7f30b0bd5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30ac15fd00 con 0x7f30ac15b990 2024-04-25T08:32:49.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.161+0000 7f30b0bd5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ac160580 con 0x7f30ac0d1d20 2024-04-25T08:32:49.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.161+0000 7f30b0bd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac15faa0 con 0x7f30ac15f5f0 2024-04-25T08:32:49.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.162+0000 7f30a9d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f30ac0d1d20 0x7f30ac0cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.162+0000 7f30a9d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f30ac0d1d20 0x7f30ac0cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57758/0 (socket says 172.21.15.38:57758) 2024-04-25T08:32:49.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.162+0000 7f30a9d9b640 1 -- 172.21.15.38:0/4041680205 learned_addr learned my addr 172.21.15.38:0/4041680205 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:49.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.162+0000 7f30aad9d640 1 --2- 172.21.15.38:0/4041680205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30ac15f5f0 0x7f30ac0b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.162+0000 7f30a9d9b640 1 -- 172.21.15.38:0/4041680205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30ac15f5f0 msgr2=0x7f30ac0b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:49.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.162+0000 7f30a9d9b640 1 --2- 172.21.15.38:0/4041680205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30ac15f5f0 0x7f30ac0b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.162+0000 7f30a9d9b640 1 -- 172.21.15.38:0/4041680205 >> v1:172.21.15.107:6789/0 conn(0x7f30ac15b990 legacy=0x7f30ac0d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.162+0000 7f30a9d9b640 1 -- 172.21.15.38:0/4041680205 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac15ffc0 con 0x7f30ac0d1d20 2024-04-25T08:32:49.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.163+0000 7f30a9d9b640 1 --2- 172.21.15.38:0/4041680205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ac0d1d20 0x7f30ac0cf580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3094009870 tx=0x7f309402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8278a01b70ce6a2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:49.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.163+0000 7f30a959a640 1 -- 172.21.15.38:0/4041680205 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094037070 con 0x7f30ac0d1d20 2024-04-25T08:32:49.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.163+0000 7f30a959a640 1 -- 172.21.15.38:0/4041680205 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3094032440 con 0x7f30ac0d1d20 2024-04-25T08:32:49.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.163+0000 7f30a959a640 1 -- 172.21.15.38:0/4041680205 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094032810 con 0x7f30ac0d1d20 2024-04-25T08:32:49.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.163+0000 7f30b0bd5640 1 -- 172.21.15.38:0/4041680205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ac0d1d20 msgr2=0x7f30ac0cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.163+0000 7f30b0bd5640 1 --2- 172.21.15.38:0/4041680205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ac0d1d20 0x7f30ac0cf580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3094009870 tx=0x7f309402a1c0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.163+0000 7f30b0bd5640 1 -- 172.21.15.38:0/4041680205 shutdown_connections 2024-04-25T08:32:49.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.163+0000 7f30b0bd5640 1 --2- 172.21.15.38:0/4041680205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30ac15f5f0 0x7f30ac0b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.163+0000 7f30b0bd5640 1 --2- 172.21.15.38:0/4041680205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ac0d1d20 0x7f30ac0cf580 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.163+0000 7f30b0bd5640 1 -- 172.21.15.38:0/4041680205 >> 172.21.15.38:0/4041680205 conn(0x7f30ac08b000 msgr2=0x7f30ac08b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:49.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.163+0000 7f30b0bd5640 1 -- 172.21.15.38:0/4041680205 shutdown_connections 2024-04-25T08:32:49.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.163+0000 7f30b0bd5640 1 -- 172.21.15.38:0/4041680205 wait complete. 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.165+0000 7f30b0bd5640 1 Processor -- start 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.165+0000 7f30b0bd5640 1 -- start start 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30b0bd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30ac0d1d20 0x7f30ac152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30b0bd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac15b990 0x7f30ac15c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30b0bd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ac15f5f0 0x7f30ac0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30b0bd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac15bd70 con 0x7f30ac15f5f0 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30b0bd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac0b1ac0 con 0x7f30ac15b990 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30b0bd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac15bef0 con 0x7f30ac0d1d20 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30aad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ac15f5f0 0x7f30ac0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30aad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ac15f5f0 0x7f30ac0ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57764/0 (socket says 172.21.15.38:57764) 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30aad9d640 1 -- 172.21.15.38:0/947145687 learned_addr learned my addr 172.21.15.38:0/947145687 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30a9d9b640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac15b990 0x7f30ac15c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30aa59c640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30ac0d1d20 0x7f30ac152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30aad9d640 1 -- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30ac0d1d20 msgr2=0x7f30ac152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30aad9d640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30ac0d1d20 0x7f30ac152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30aad9d640 1 -- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac15b990 msgr2=0x7f30ac15c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:49.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30aad9d640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac15b990 0x7f30ac15c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:49.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30aad9d640 1 -- 172.21.15.38:0/947145687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac06c730 con 0x7f30ac15f5f0 2024-04-25T08:32:49.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30a9d9b640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac15b990 0x7f30ac15c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:49.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30aad9d640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ac15f5f0 0x7f30ac0ccca0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f30a4007e90 tx=0x7f30a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:49.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.166+0000 7f30a2ffd640 1 -- 172.21.15.38:0/947145687 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4014420 con 0x7f30ac15f5f0 2024-04-25T08:32:49.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.167+0000 7f30a2ffd640 1 -- 172.21.15.38:0/947145687 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a4019070 con 0x7f30ac15f5f0 2024-04-25T08:32:49.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.167+0000 7f30b0bd5640 1 -- 172.21.15.38:0/947145687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ac06ca10 con 0x7f30ac15f5f0 2024-04-25T08:32:49.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.167+0000 7f30a2ffd640 1 -- 172.21.15.38:0/947145687 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4014800 con 0x7f30ac15f5f0 2024-04-25T08:32:49.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.167+0000 7f30b0bd5640 1 -- 172.21.15.38:0/947145687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac06cfd0 con 0x7f30ac15f5f0 2024-04-25T08:32:49.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.168+0000 7f30b0bd5640 1 -- 172.21.15.38:0/947145687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ac068e80 con 0x7f30ac15f5f0 2024-04-25T08:32:49.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.168+0000 7f30a2ffd640 1 -- 172.21.15.38:0/947145687 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30a40149a0 con 0x7f30ac15f5f0 2024-04-25T08:32:49.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.169+0000 7f30a2ffd640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f308c06b4d0 0x7f308c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.169+0000 7f30a2ffd640 1 -- 172.21.15.38:0/947145687 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f30a408ab00 con 0x7f30ac15f5f0 2024-04-25T08:32:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.169+0000 7f30aa59c640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f308c06b4d0 0x7f308c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:49.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.169+0000 7f30aa59c640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f308c06b4d0 0x7f308c06d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f309c001640 tx=0x7f309c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:49.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.173+0000 7f30a2ffd640 1 -- 172.21.15.38:0/947145687 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30a40598e0 con 0x7f30ac15f5f0 2024-04-25T08:32:49.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:49.390+0000 7f30b0bd5640 1 -- 172.21.15.38:0/947145687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f30ac0b1220 con 0x7f30ac15f5f0 2024-04-25T08:32:50.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.061+0000 7f30a2ffd640 1 -- 172.21.15.38:0/947145687 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f30a40591e0 con 0x7f30ac15f5f0 2024-04-25T08:32:50.060 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T08:32:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.066+0000 7f30b0bd5640 1 -- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f308c06b4d0 msgr2=0x7f308c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.066+0000 7f30b0bd5640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f308c06b4d0 0x7f308c06d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f309c001640 tx=0x7f309c009040 comp rx=0 tx=0).stop 2024-04-25T08:32:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.066+0000 7f30b0bd5640 1 -- 172.21.15.38:0/947145687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ac15f5f0 msgr2=0x7f30ac0ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.066+0000 7f30b0bd5640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ac15f5f0 0x7f30ac0ccca0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f30a4007e90 tx=0x7f30a400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.066+0000 7f30b0bd5640 1 -- 172.21.15.38:0/947145687 shutdown_connections 2024-04-25T08:32:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.066+0000 7f30b0bd5640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30ac15f5f0 0x7f30ac0ccca0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.066+0000 7f30b0bd5640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30ac15b990 0x7f30ac15c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.066+0000 7f30b0bd5640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f308c06b4d0 0x7f308c06d990 secure :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f309c001640 tx=0x7f309c009040 comp rx=0 tx=0).stop 2024-04-25T08:32:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.066+0000 7f30b0bd5640 1 --2- 172.21.15.38:0/947145687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30ac0d1d20 0x7f30ac152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.066+0000 7f30b0bd5640 1 -- 172.21.15.38:0/947145687 >> 172.21.15.38:0/947145687 conn(0x7f30ac08b000 msgr2=0x7f30ac0d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.068+0000 7f30b0bd5640 1 -- 172.21.15.38:0/947145687 shutdown_connections 2024-04-25T08:32:50.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.070+0000 7f30b0bd5640 1 -- 172.21.15.38:0/947145687 wait complete. 2024-04-25T08:32:50.087 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:50.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.197+0000 7f04bbbf4640 1 Processor -- start 2024-04-25T08:32:50.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.197+0000 7f04bbbf4640 1 -- start start 2024-04-25T08:32:50.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04bbbf4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04b406c110 0x7f04b406c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04bbbf4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b406ca30 con 0x7f04b40ce2e0 2024-04-25T08:32:50.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04bbbf4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b40ceff0 con 0x7f04b415df50 2024-04-25T08:32:50.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04bbbf4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b40cf2d0 con 0x7f04b406c110 2024-04-25T08:32:50.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04ba191640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f04b40ce2e0 0x7f04b40d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47602/0 (socket says 172.21.15.38:47602) 2024-04-25T08:32:50.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04ba191640 1 -- 172.21.15.38:0/2266099040 learned_addr learned my addr 172.21.15.38:0/2266099040 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:50.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04b9990640 1 --2- 172.21.15.38:0/2266099040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04b406c110 0x7f04b406c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04b898e640 1 -- 172.21.15.38:0/2266099040 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036645867 0 0) 0x7f04b406ca30 con 0x7f04b40ce2e0 2024-04-25T08:32:50.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04b898e640 1 -- 172.21.15.38:0/2266099040 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04a0003520 con 0x7f04b40ce2e0 2024-04-25T08:32:50.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04b898e640 1 -- 172.21.15.38:0/2266099040 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290848381 0 0) 0x7f04b40ceff0 con 0x7f04b415df50 2024-04-25T08:32:50.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04b898e640 1 -- 172.21.15.38:0/2266099040 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04b406ca30 con 0x7f04b415df50 2024-04-25T08:32:50.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04b9990640 1 -- 172.21.15.38:0/2266099040 >> v1:172.21.15.107:6789/0 conn(0x7f04b415df50 legacy=0x7f04b40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04b9990640 1 -- 172.21.15.38:0/2266099040 >> v1:172.21.15.38:6789/0 conn(0x7f04b40ce2e0 legacy=0x7f04b40d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.198+0000 7f04b9990640 1 -- 172.21.15.38:0/2266099040 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b40d1f70 con 0x7f04b406c110 2024-04-25T08:32:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.199+0000 7f04b9990640 1 --2- 172.21.15.38:0/2266099040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04b406c110 0x7f04b406c4f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009dc0 tx=0x7f04b002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f370f00aa97e8e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.199+0000 7f04b898e640 1 -- 172.21.15.38:0/2266099040 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0007740 con 0x7f04b406c110 2024-04-25T08:32:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.199+0000 7f04b898e640 1 -- 172.21.15.38:0/2266099040 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b00078e0 con 0x7f04b406c110 2024-04-25T08:32:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.199+0000 7f04b898e640 1 -- 172.21.15.38:0/2266099040 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0007c90 con 0x7f04b406c110 2024-04-25T08:32:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.200+0000 7f04bbbf4640 1 -- 172.21.15.38:0/2266099040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04b406c110 msgr2=0x7f04b406c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.200+0000 7f04bbbf4640 1 --2- 172.21.15.38:0/2266099040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04b406c110 0x7f04b406c4f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009dc0 tx=0x7f04b002a6d0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.200+0000 7f04bbbf4640 1 -- 172.21.15.38:0/2266099040 shutdown_connections 2024-04-25T08:32:50.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.200+0000 7f04bbbf4640 1 --2- 172.21.15.38:0/2266099040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04b406c110 0x7f04b406c4f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.200+0000 7f04bbbf4640 1 -- 172.21.15.38:0/2266099040 >> 172.21.15.38:0/2266099040 conn(0x7f04b40d3fb0 msgr2=0x7f04b40d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.200+0000 7f04bbbf4640 1 -- 172.21.15.38:0/2266099040 shutdown_connections 2024-04-25T08:32:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.200+0000 7f04bbbf4640 1 -- 172.21.15.38:0/2266099040 wait complete. 2024-04-25T08:32:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.201+0000 7f04bbbf4640 1 Processor -- start 2024-04-25T08:32:50.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.201+0000 7f04bbbf4640 1 -- start start 2024-04-25T08:32:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04bbbf4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04b406c110 0x7f04b415e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04bbbf4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04b40ce2e0 0x7f04b40b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04bbbf4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04b415df50 0x7f04b4155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04bbbf4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b40d3160 con 0x7f04b406c110 2024-04-25T08:32:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04bbbf4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b40d2530 con 0x7f04b415df50 2024-04-25T08:32:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04bbbf4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b40d32e0 con 0x7f04b40ce2e0 2024-04-25T08:32:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04b9990640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04b406c110 0x7f04b415e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04b9990640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04b406c110 0x7f04b415e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57770/0 (socket says 172.21.15.38:57770) 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04b9990640 1 -- 172.21.15.38:0/4248237036 learned_addr learned my addr 172.21.15.38:0/4248237036 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04b918f640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04b40ce2e0 0x7f04b40b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04ba191640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04b415df50 0x7f04b4155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04b9990640 1 -- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04b40ce2e0 msgr2=0x7f04b40b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04b9990640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04b40ce2e0 0x7f04b40b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04b9990640 1 -- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04b415df50 msgr2=0x7f04b4155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04b9990640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04b415df50 0x7f04b4155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04b9990640 1 -- 172.21.15.38:0/4248237036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b415eec0 con 0x7f04b406c110 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.202+0000 7f04b9990640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04b406c110 0x7f04b415e7a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009ee0 tx=0x7f04b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.203+0000 7f04aa7fc640 1 -- 172.21.15.38:0/4248237036 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0040070 con 0x7f04b406c110 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.203+0000 7f04aa7fc640 1 -- 172.21.15.38:0/4248237036 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b0031c40 con 0x7f04b406c110 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.203+0000 7f04bbbf4640 1 -- 172.21.15.38:0/4248237036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b415f1a0 con 0x7f04b406c110 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.203+0000 7f04aa7fc640 1 -- 172.21.15.38:0/4248237036 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0009450 con 0x7f04b406c110 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.203+0000 7f04bbbf4640 1 -- 172.21.15.38:0/4248237036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b415f760 con 0x7f04b406c110 2024-04-25T08:32:50.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.204+0000 7f04bbbf4640 1 -- 172.21.15.38:0/4248237036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b4068e80 con 0x7f04b406c110 2024-04-25T08:32:50.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.204+0000 7f04aa7fc640 1 -- 172.21.15.38:0/4248237036 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04b0039040 con 0x7f04b406c110 2024-04-25T08:32:50.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.205+0000 7f04aa7fc640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f049806b4d0 0x7f049806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.205+0000 7f04aa7fc640 1 -- 172.21.15.38:0/4248237036 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04b00ab9d0 con 0x7f04b406c110 2024-04-25T08:32:50.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.205+0000 7f04b918f640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f049806b4d0 0x7f049806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.206+0000 7f04b918f640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f049806b4d0 0x7f049806d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04a4007fd0 tx=0x7f04a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.209+0000 7f04aa7fc640 1 -- 172.21.15.38:0/4248237036 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04b007a070 con 0x7f04b406c110 2024-04-25T08:32:50.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.392+0000 7f04bbbf4640 1 -- 172.21.15.38:0/4248237036 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04b406b050 con 0x7f04b406c110 2024-04-25T08:32:50.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.394+0000 7f04aa7fc640 1 -- 172.21.15.38:0/4248237036 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f04b007e110 con 0x7f04b406c110 2024-04-25T08:32:50.390 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:50.390 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":37,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:49.391964+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:47.798666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:48.371717+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.395+0000 7f04bbbf4640 1 -- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f049806b4d0 msgr2=0x7f049806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.395+0000 7f04bbbf4640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f049806b4d0 0x7f049806d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f04a4007fd0 tx=0x7f04a4009040 comp rx=0 tx=0).stop 2024-04-25T08:32:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.395+0000 7f04bbbf4640 1 -- 172.21.15.38:0/4248237036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04b406c110 msgr2=0x7f04b415e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.395+0000 7f04bbbf4640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04b406c110 0x7f04b415e7a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009ee0 tx=0x7f04b0004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.396+0000 7f04bbbf4640 1 -- 172.21.15.38:0/4248237036 shutdown_connections 2024-04-25T08:32:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.396+0000 7f04bbbf4640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f04b415df50 0x7f04b4155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.396+0000 7f04bbbf4640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f049806b4d0 0x7f049806d990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.396+0000 7f04bbbf4640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f04b40ce2e0 0x7f04b40b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.396+0000 7f04bbbf4640 1 --2- 172.21.15.38:0/4248237036 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f04b406c110 0x7f04b415e7a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.396+0000 7f04bbbf4640 1 -- 172.21.15.38:0/4248237036 >> 172.21.15.38:0/4248237036 conn(0x7f04b40d3fb0 msgr2=0x7f04b415bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.396+0000 7f04bbbf4640 1 -- 172.21.15.38:0/4248237036 shutdown_connections 2024-04-25T08:32:50.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.396+0000 7f04bbbf4640 1 -- 172.21.15.38:0/4248237036 wait complete. 2024-04-25T08:32:50.403 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:50.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.507+0000 7fbd9fb3b640 1 Processor -- start 2024-04-25T08:32:50.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.507+0000 7fbd9fb3b640 1 -- start start 2024-04-25T08:32:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9fb3b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd98144760 0x7fbd9815faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9fb3b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbd98160030 0x7fbd980ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9fb3b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd980b4b50 0x7fbd980ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9fb3b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9815f380 con 0x7fbd98144760 2024-04-25T08:32:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9fb3b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9815f660 con 0x7fbd98160030 2024-04-25T08:32:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9fb3b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd980b50c0 con 0x7fbd980b4b50 2024-04-25T08:32:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9d8d7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd98144760 0x7fbd9815faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9d8d7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd98144760 0x7fbd9815faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57778/0 (socket says 172.21.15.38:57778) 2024-04-25T08:32:50.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9d8d7640 1 -- 172.21.15.38:0/2482455557 learned_addr learned my addr 172.21.15.38:0/2482455557 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9d0d6640 1 --2- 172.21.15.38:0/2482455557 >> v2:172.21.15.107:3300/0 conn(0x7fbd98160030 0x7fbd980ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9e0d8640 1 --2- 172.21.15.38:0/2482455557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd980b4b50 0x7fbd980ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9d8d7640 1 -- 172.21.15.38:0/2482455557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd980b4b50 msgr2=0x7fbd980ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9d8d7640 1 --2- 172.21.15.38:0/2482455557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd980b4b50 0x7fbd980ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9d8d7640 1 -- 172.21.15.38:0/2482455557 >> v2:172.21.15.107:3300/0 conn(0x7fbd98160030 msgr2=0x7fbd980ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9d8d7640 1 --2- 172.21.15.38:0/2482455557 >> v2:172.21.15.107:3300/0 conn(0x7fbd98160030 0x7fbd980ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9d8d7640 1 -- 172.21.15.38:0/2482455557 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd980b53a0 con 0x7fbd98144760 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9d0d6640 1 --2- 172.21.15.38:0/2482455557 >> v2:172.21.15.107:3300/0 conn(0x7fbd98160030 0x7fbd980ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.508+0000 7fbd9e0d8640 1 --2- 172.21.15.38:0/2482455557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd980b4b50 0x7fbd980ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.509+0000 7fbd9d8d7640 1 --2- 172.21.15.38:0/2482455557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd98144760 0x7fbd9815faf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbd980c74a0 tx=0x7fbd9002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b25df2c1a2860fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.509+0000 7fbd9c8d5640 1 -- 172.21.15.38:0/2482455557 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90036070 con 0x7fbd98144760 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.509+0000 7fbd9c8d5640 1 -- 172.21.15.38:0/2482455557 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd90031600 con 0x7fbd98144760 2024-04-25T08:32:50.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.509+0000 7fbd9c8d5640 1 -- 172.21.15.38:0/2482455557 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd900319d0 con 0x7fbd98144760 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.510+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/2482455557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd98144760 msgr2=0x7fbd9815faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.510+0000 7fbd9fb3b640 1 --2- 172.21.15.38:0/2482455557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd98144760 0x7fbd9815faf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fbd980c74a0 tx=0x7fbd9002a610 comp rx=0 tx=0).stop 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.510+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/2482455557 shutdown_connections 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.510+0000 7fbd9fb3b640 1 --2- 172.21.15.38:0/2482455557 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd980b4b50 0x7fbd980ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.510+0000 7fbd9fb3b640 1 --2- 172.21.15.38:0/2482455557 >> v2:172.21.15.107:3300/0 conn(0x7fbd98160030 0x7fbd980ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.510+0000 7fbd9fb3b640 1 --2- 172.21.15.38:0/2482455557 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd98144760 0x7fbd9815faf0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.510+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/2482455557 >> 172.21.15.38:0/2482455557 conn(0x7fbd9806b210 msgr2=0x7fbd9806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.510+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/2482455557 shutdown_connections 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.510+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/2482455557 wait complete. 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.511+0000 7fbd9fb3b640 1 Processor -- start 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.511+0000 7fbd9fb3b640 1 -- start start 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9fb3b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd980b4b50 0x7fbd9813a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9fb3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd98160030 0x7fbd9813c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9fb3b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd9815ace0 0x7fbd981380f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9fb3b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd98145400 con 0x7fbd98160030 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9fb3b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd98145280 con 0x7fbd9815ace0 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9fb3b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd98144b70 con 0x7fbd980b4b50 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9d0d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd98160030 0x7fbd9813c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9d0d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd98160030 0x7fbd9813c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57788/0 (socket says 172.21.15.38:57788) 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9d0d6640 1 -- 172.21.15.38:0/3013695148 learned_addr learned my addr 172.21.15.38:0/3013695148 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9d8d7640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd980b4b50 0x7fbd9813a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9e0d8640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd9815ace0 0x7fbd981380f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9d0d6640 1 -- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd980b4b50 msgr2=0x7fbd9813a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9d0d6640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd980b4b50 0x7fbd9813a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9d0d6640 1 -- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd9815ace0 msgr2=0x7fbd981380f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9d0d6640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd9815ace0 0x7fbd981380f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9d0d6640 1 -- 172.21.15.38:0/3013695148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9813acc0 con 0x7fbd98160030 2024-04-25T08:32:50.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9e0d8640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd9815ace0 0x7fbd981380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:50.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.512+0000 7fbd9d0d6640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd98160030 0x7fbd9813c140 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00d7d0 tx=0x7fbd8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.513+0000 7fbd967fc640 1 -- 172.21.15.38:0/3013695148 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c004030 con 0x7fbd98160030 2024-04-25T08:32:50.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.513+0000 7fbd967fc640 1 -- 172.21.15.38:0/3013695148 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c0041d0 con 0x7fbd98160030 2024-04-25T08:32:50.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.513+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/3013695148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9813a700 con 0x7fbd98160030 2024-04-25T08:32:50.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.513+0000 7fbd967fc640 1 -- 172.21.15.38:0/3013695148 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0096a0 con 0x7fbd98160030 2024-04-25T08:32:50.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.513+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/3013695148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9815d2b0 con 0x7fbd98160030 2024-04-25T08:32:50.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.514+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/3013695148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd98068e80 con 0x7fbd98160030 2024-04-25T08:32:50.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.514+0000 7fbd967fc640 1 -- 172.21.15.38:0/3013695148 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd8c009840 con 0x7fbd98160030 2024-04-25T08:32:50.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.515+0000 7fbd967fc640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd8406b4d0 0x7fbd8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.515+0000 7fbd967fc640 1 -- 172.21.15.38:0/3013695148 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbd8c08a560 con 0x7fbd98160030 2024-04-25T08:32:50.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.515+0000 7fbd9d8d7640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd8406b4d0 0x7fbd8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.515+0000 7fbd9d8d7640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd8406b4d0 0x7fbd8406d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbd90033000 tx=0x7fbd90009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.519+0000 7fbd967fc640 1 -- 172.21.15.38:0/3013695148 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd8c059260 con 0x7fbd98160030 2024-04-25T08:32:50.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.700+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/3013695148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd9806df60 con 0x7fbd98160030 2024-04-25T08:32:50.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.701+0000 7fbd967fc640 1 -- 172.21.15.38:0/3013695148 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+16943 (secure 0 0 0) 0x7fbd8c058b60 con 0x7fbd98160030 2024-04-25T08:32:50.697 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:50.697 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":37,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:49.391964+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:47.798666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:48.371717+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:50.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.703+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd8406b4d0 msgr2=0x7fbd8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.703+0000 7fbd9fb3b640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd8406b4d0 0x7fbd8406d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbd90033000 tx=0x7fbd90009000 comp rx=0 tx=0).stop 2024-04-25T08:32:50.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.703+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/3013695148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd98160030 msgr2=0x7fbd9813c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.703+0000 7fbd9fb3b640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd98160030 0x7fbd9813c140 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00d7d0 tx=0x7fbd8c00dc90 comp rx=0 tx=0).stop 2024-04-25T08:32:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.703+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/3013695148 shutdown_connections 2024-04-25T08:32:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.703+0000 7fbd9fb3b640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd9815ace0 0x7fbd981380f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.703+0000 7fbd9fb3b640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd98160030 0x7fbd9813c140 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.703+0000 7fbd9fb3b640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd8406b4d0 0x7fbd8406d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.703+0000 7fbd9fb3b640 1 --2- 172.21.15.38:0/3013695148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd980b4b50 0x7fbd9813a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.703+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/3013695148 >> 172.21.15.38:0/3013695148 conn(0x7fbd9806b210 msgr2=0x7fbd9813e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.703+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/3013695148 shutdown_connections 2024-04-25T08:32:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.703+0000 7fbd9fb3b640 1 -- 172.21.15.38:0/3013695148 wait complete. 2024-04-25T08:32:50.711 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:32:50.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.819+0000 7fdee4815640 1 Processor -- start 2024-04-25T08:32:50.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.819+0000 7fdee4815640 1 -- start start 2024-04-25T08:32:50.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.819+0000 7fdee4815640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee00d1d20 0x7fdee00cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.819+0000 7fdee4815640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdee015f5f0 0x7fdee00b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.819+0000 7fdee4815640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee015fd00 con 0x7fdee015b990 2024-04-25T08:32:50.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.819+0000 7fdee4815640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee0160580 con 0x7fdee015f5f0 2024-04-25T08:32:50.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.819+0000 7fdee4815640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee015faa0 con 0x7fdee00d1d20 2024-04-25T08:32:50.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.820+0000 7fdede59c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdee015b990 0x7fdee00d1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47618/0 (socket says 172.21.15.38:47618) 2024-04-25T08:32:50.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.820+0000 7fdede59c640 1 -- 172.21.15.38:0/1356792206 learned_addr learned my addr 172.21.15.38:0/1356792206 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:50.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.820+0000 7fded7fff640 1 --2- 172.21.15.38:0/1356792206 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee00d1d20 0x7fdee00cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.820+0000 7fdeded9d640 1 --2- 172.21.15.38:0/1356792206 >> v2:172.21.15.107:3300/0 conn(0x7fdee015f5f0 0x7fdee00b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.820+0000 7fdeddd9b640 1 -- 172.21.15.38:0/1356792206 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824332629 0 0) 0x7fdee015fd00 con 0x7fdee015b990 2024-04-25T08:32:50.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.820+0000 7fdeddd9b640 1 -- 172.21.15.38:0/1356792206 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdecc003520 con 0x7fdee015b990 2024-04-25T08:32:50.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.820+0000 7fded7fff640 1 -- 172.21.15.38:0/1356792206 >> v2:172.21.15.107:3300/0 conn(0x7fdee015f5f0 msgr2=0x7fdee00b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:50.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.820+0000 7fded7fff640 1 --2- 172.21.15.38:0/1356792206 >> v2:172.21.15.107:3300/0 conn(0x7fdee015f5f0 0x7fdee00b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.820+0000 7fded7fff640 1 -- 172.21.15.38:0/1356792206 >> v1:172.21.15.38:6789/0 conn(0x7fdee015b990 legacy=0x7fdee00d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.820+0000 7fded7fff640 1 -- 172.21.15.38:0/1356792206 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee015ffc0 con 0x7fdee00d1d20 2024-04-25T08:32:50.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.821+0000 7fded7fff640 1 --2- 172.21.15.38:0/1356792206 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee00d1d20 0x7fdee00cf580 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdec80097a0 tx=0x7fdec802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdbbc87b0fe24813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.821+0000 7fdeddd9b640 1 -- 172.21.15.38:0/1356792206 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8037070 con 0x7fdee00d1d20 2024-04-25T08:32:50.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.821+0000 7fdeddd9b640 1 -- 172.21.15.38:0/1356792206 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec802ae50 con 0x7fdee00d1d20 2024-04-25T08:32:50.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.821+0000 7fdeddd9b640 1 -- 172.21.15.38:0/1356792206 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8032660 con 0x7fdee00d1d20 2024-04-25T08:32:50.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.821+0000 7fdee4815640 1 -- 172.21.15.38:0/1356792206 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee00d1d20 msgr2=0x7fdee00cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.821+0000 7fdee4815640 1 --2- 172.21.15.38:0/1356792206 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee00d1d20 0x7fdee00cf580 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdec80097a0 tx=0x7fdec802a100 comp rx=0 tx=0).stop 2024-04-25T08:32:50.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.821+0000 7fdee4815640 1 -- 172.21.15.38:0/1356792206 shutdown_connections 2024-04-25T08:32:50.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.821+0000 7fdee4815640 1 --2- 172.21.15.38:0/1356792206 >> v2:172.21.15.107:3300/0 conn(0x7fdee015f5f0 0x7fdee00b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.822+0000 7fdee4815640 1 --2- 172.21.15.38:0/1356792206 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee00d1d20 0x7fdee00cf580 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.822+0000 7fdee4815640 1 -- 172.21.15.38:0/1356792206 >> 172.21.15.38:0/1356792206 conn(0x7fdee008b000 msgr2=0x7fdee008b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:50.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.822+0000 7fdee4815640 1 -- 172.21.15.38:0/1356792206 shutdown_connections 2024-04-25T08:32:50.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.822+0000 7fdee4815640 1 -- 172.21.15.38:0/1356792206 wait complete. 2024-04-25T08:32:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.823+0000 7fdee4815640 1 Processor -- start 2024-04-25T08:32:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.823+0000 7fdee4815640 1 -- start start 2024-04-25T08:32:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdee4815640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee00d1d20 0x7fdee015c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdee4815640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee015b990 0x7fdee0153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdee4815640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee015f5f0 0x7fdee0158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdee4815640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee015bd70 con 0x7fdee015f5f0 2024-04-25T08:32:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdee4815640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00b1ac0 con 0x7fdee015b990 2024-04-25T08:32:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdee4815640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee015bef0 con 0x7fdee00d1d20 2024-04-25T08:32:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdeded9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee015f5f0 0x7fdee0158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdeded9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee015f5f0 0x7fdee0158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57796/0 (socket says 172.21.15.38:57796) 2024-04-25T08:32:50.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdeded9d640 1 -- 172.21.15.38:0/1502216198 learned_addr learned my addr 172.21.15.38:0/1502216198 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdede59c640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee00d1d20 0x7fdee015c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fded7fff640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee015b990 0x7fdee0153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdeded9d640 1 -- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee00d1d20 msgr2=0x7fdee015c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdeded9d640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee00d1d20 0x7fdee015c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdeded9d640 1 -- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee015b990 msgr2=0x7fdee0153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdeded9d640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee015b990 0x7fdee0153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdeded9d640 1 -- 172.21.15.38:0/1502216198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee0153850 con 0x7fdee015f5f0 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdeded9d640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee015f5f0 0x7fdee0158b10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fded800efc0 tx=0x7fded800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fded6ffd640 1 -- 172.21.15.38:0/1502216198 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8018020 con 0x7fdee015f5f0 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fded6ffd640 1 -- 172.21.15.38:0/1502216198 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded8014480 con 0x7fdee015f5f0 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fded6ffd640 1 -- 172.21.15.38:0/1502216198 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8014870 con 0x7fdee015f5f0 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.824+0000 7fdee4815640 1 -- 172.21.15.38:0/1502216198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee015c6b0 con 0x7fdee015f5f0 2024-04-25T08:32:50.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.825+0000 7fdee4815640 1 -- 172.21.15.38:0/1502216198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee015cc70 con 0x7fdee015f5f0 2024-04-25T08:32:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.825+0000 7fdee4815640 1 -- 172.21.15.38:0/1502216198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee0068e80 con 0x7fdee015f5f0 2024-04-25T08:32:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.826+0000 7fded6ffd640 1 -- 172.21.15.38:0/1502216198 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fded8005d10 con 0x7fdee015f5f0 2024-04-25T08:32:50.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.826+0000 7fded6ffd640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdec006b500 0x7fdec006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.826+0000 7fded6ffd640 1 -- 172.21.15.38:0/1502216198 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fded808af30 con 0x7fdee015f5f0 2024-04-25T08:32:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.827+0000 7fdede59c640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdec006b500 0x7fdec006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:50.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.827+0000 7fdede59c640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdec006b500 0x7fdec006d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fded0005880 tx=0x7fded0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:50.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:50.830+0000 7fded6ffd640 1 -- 172.21.15.38:0/1502216198 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fded8022030 con 0x7fdee015f5f0 2024-04-25T08:32:51.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:51.047+0000 7fdee4815640 1 -- 172.21.15.38:0/1502216198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdee00d1a20 con 0x7fdee015f5f0 2024-04-25T08:32:52.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.053+0000 7fded6ffd640 1 -- 172.21.15.38:0/1502216198 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fded8010040 con 0x7fdee015f5f0 2024-04-25T08:32:52.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.056+0000 7fdee4815640 1 -- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdec006b500 msgr2=0x7fdec006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:52.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.056+0000 7fdee4815640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdec006b500 0x7fdec006d9c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fded0005880 tx=0x7fded0009040 comp rx=0 tx=0).stop 2024-04-25T08:32:52.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.056+0000 7fdee4815640 1 -- 172.21.15.38:0/1502216198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee015f5f0 msgr2=0x7fdee0158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:52.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.056+0000 7fdee4815640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee015f5f0 0x7fdee0158b10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fded800efc0 tx=0x7fded800c490 comp rx=0 tx=0).stop 2024-04-25T08:32:52.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.056+0000 7fdee4815640 1 -- 172.21.15.38:0/1502216198 shutdown_connections 2024-04-25T08:32:52.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.056+0000 7fdee4815640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee015f5f0 0x7fdee0158b10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.056+0000 7fdee4815640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee015b990 0x7fdee0153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.056+0000 7fdee4815640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdec006b500 0x7fdec006d9c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.056+0000 7fdee4815640 1 --2- 172.21.15.38:0/1502216198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee00d1d20 0x7fdee015c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.056+0000 7fdee4815640 1 -- 172.21.15.38:0/1502216198 >> 172.21.15.38:0/1502216198 conn(0x7fdee008b000 msgr2=0x7fdee00cf200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:52.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.056+0000 7fdee4815640 1 -- 172.21.15.38:0/1502216198 shutdown_connections 2024-04-25T08:32:52.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.056+0000 7fdee4815640 1 -- 172.21.15.38:0/1502216198 wait complete. 2024-04-25T08:32:52.065 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:32:52.066 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:32:52.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.180+0000 7f691c4fc640 1 Processor -- start 2024-04-25T08:32:52.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.180+0000 7f691c4fc640 1 -- start start 2024-04-25T08:32:52.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.181+0000 7f691c4fc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f69140d0320 0x7f69140d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.181+0000 7f691c4fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69140b6980 0x7f69140b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.181+0000 7f691c4fc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69140d2060 con 0x7f69140d0700 2024-04-25T08:32:52.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.181+0000 7f691c4fc640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69140d2330 con 0x7f69140d0320 2024-04-25T08:32:52.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.181+0000 7f691c4fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69140d2610 con 0x7f69140b6980 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.181+0000 7f6919a97640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f69140d0700 0x7f69140d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47620/0 (socket says 172.21.15.38:47620) 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.181+0000 7f6919a97640 1 -- 172.21.15.38:0/3438012542 learned_addr learned my addr 172.21.15.38:0/3438012542 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 255152118 0 0) 0x7f69140d2060 con 0x7f69140d0700 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f691aa99640 1 --2- 172.21.15.38:0/3438012542 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69140b6980 0x7f69140b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68fc003520 con 0x7f69140d0700 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f691a298640 1 --2- 172.21.15.38:0/3438012542 >> v2:172.21.15.107:3300/0 conn(0x7f69140d0320 0x7f69140d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1559648471 0 0) 0x7f68fc003520 con 0x7f69140d0700 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69140b6980 msgr2=0x7f69140b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 --2- 172.21.15.38:0/3438012542 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69140b6980 0x7f69140b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 >> v2:172.21.15.107:3300/0 conn(0x7f69140d0320 msgr2=0x7f69140d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 --2- 172.21.15.38:0/3438012542 >> v2:172.21.15.107:3300/0 conn(0x7f69140d0320 0x7f69140d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69140d28f0 con 0x7f69140d0700 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f6908001670 con 0x7f69140d0700 2024-04-25T08:32:52.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f691a298640 1 --2- 172.21.15.38:0/3438012542 >> v2:172.21.15.107:3300/0 conn(0x7f69140d0320 0x7f69140d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 >> v1:172.21.15.38:6789/0 conn(0x7f69140d0700 legacy=0x7f69140d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 --2- 172.21.15.38:0/3438012542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68fc0039b0 0x7f68fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 --2- 172.21.15.38:0/3438012542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68fc004800 0x7f68fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 --2- 172.21.15.38:0/3438012542 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69140b6980 0x7f68fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140d2330 con 0x7f68fc004800 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140d2610 con 0x7f68fc0039b0 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc00ba70 con 0x7f69140b6980 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f6919a97640 1 --2- 172.21.15.38:0/3438012542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68fc004800 0x7f68fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f691a298640 1 --2- 172.21.15.38:0/3438012542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68fc0039b0 0x7f68fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.182+0000 7f691aa99640 1 --2- 172.21.15.38:0/3438012542 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69140b6980 0x7f68fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.183+0000 7f6919a97640 1 -- 172.21.15.38:0/3438012542 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69140b6980 msgr2=0x7f68fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.183+0000 7f6919a97640 1 --2- 172.21.15.38:0/3438012542 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69140b6980 0x7f68fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.183+0000 7f6919a97640 1 -- 172.21.15.38:0/3438012542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68fc0039b0 msgr2=0x7f68fc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:52.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.183+0000 7f6919a97640 1 --2- 172.21.15.38:0/3438012542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68fc0039b0 0x7f68fc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.183+0000 7f6919a97640 1 -- 172.21.15.38:0/3438012542 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc003520 con 0x7f68fc004800 2024-04-25T08:32:52.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.183+0000 7f6919a97640 1 --2- 172.21.15.38:0/3438012542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68fc004800 0x7f68fc004c80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6908001050 tx=0x7f68fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5520eba87655d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:52.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.183+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690802e070 con 0x7f68fc004800 2024-04-25T08:32:52.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.183+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6908005af0 con 0x7f68fc004800 2024-04-25T08:32:52.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.183+0000 7f6919296640 1 -- 172.21.15.38:0/3438012542 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908007d70 con 0x7f68fc004800 2024-04-25T08:32:52.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.184+0000 7f691c4fc640 1 -- 172.21.15.38:0/3438012542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68fc004800 msgr2=0x7f68fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:52.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.184+0000 7f691c4fc640 1 --2- 172.21.15.38:0/3438012542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68fc004800 0x7f68fc004c80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6908001050 tx=0x7f68fc005830 comp rx=0 tx=0).stop 2024-04-25T08:32:52.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.184+0000 7f691aa99640 1 -- 172.21.15.38:0/3438012542 reap_dead start 2024-04-25T08:32:52.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.184+0000 7f691c4fc640 1 -- 172.21.15.38:0/3438012542 shutdown_connections 2024-04-25T08:32:52.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.184+0000 7f691c4fc640 1 -- 172.21.15.38:0/3438012542 >> 172.21.15.38:0/3438012542 conn(0x7f69140668f0 msgr2=0x7f6914066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.184+0000 7f691c4fc640 1 -- 172.21.15.38:0/3438012542 shutdown_connections 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.184+0000 7f691c4fc640 1 -- 172.21.15.38:0/3438012542 wait complete. 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.185+0000 7f691c4fc640 1 Processor -- start 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.185+0000 7f691c4fc640 1 -- start start 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f691c4fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc004800 0x7f69140b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f691c4fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69141f3810 0x7f6914158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f691c4fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69141f3c70 0x7f6914158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f691c4fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140d0b40 con 0x7f69141f3810 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f691c4fc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140d2eb0 con 0x7f69141f3c70 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f691c4fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69140b4c10 con 0x7f68fc004800 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f6919a97640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69141f3810 0x7f6914158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f6919a97640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69141f3810 0x7f6914158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57816/0 (socket says 172.21.15.38:57816) 2024-04-25T08:32:52.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f6919a97640 1 -- 172.21.15.38:0/2618293575 learned_addr learned my addr 172.21.15.38:0/2618293575 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f691a298640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc004800 0x7f69140b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f691aa99640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69141f3c70 0x7f6914158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f6919a97640 1 -- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc004800 msgr2=0x7f69140b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f6919a97640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc004800 0x7f69140b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f6919a97640 1 -- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69141f3c70 msgr2=0x7f6914158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f6919a97640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69141f3c70 0x7f6914158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f6919a97640 1 -- 172.21.15.38:0/2618293575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69140b4ef0 con 0x7f69141f3810 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.186+0000 7f691aa99640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69141f3c70 0x7f6914158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.187+0000 7f6919a97640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69141f3810 0x7f6914158950 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f690800c300 tx=0x7f6908000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.187+0000 7f6912ffd640 1 -- 172.21.15.38:0/2618293575 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908042070 con 0x7f69141f3810 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.187+0000 7f6912ffd640 1 -- 172.21.15.38:0/2618293575 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6908007480 con 0x7f69141f3810 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.187+0000 7f6912ffd640 1 -- 172.21.15.38:0/2618293575 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908007d50 con 0x7f69141f3810 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.187+0000 7f691c4fc640 1 -- 172.21.15.38:0/2618293575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69140b51d0 con 0x7f69141f3810 2024-04-25T08:32:52.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.187+0000 7f691c4fc640 1 -- 172.21.15.38:0/2618293575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69141593d0 con 0x7f69141f3810 2024-04-25T08:32:52.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.188+0000 7f691c4fc640 1 -- 172.21.15.38:0/2618293575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6914068e80 con 0x7f69141f3810 2024-04-25T08:32:52.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.188+0000 7f6912ffd640 1 -- 172.21.15.38:0/2618293575 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6908045070 con 0x7f69141f3810 2024-04-25T08:32:52.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.189+0000 7f6912ffd640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f690006b4d0 0x7f690006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:52.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.189+0000 7f6912ffd640 1 -- 172.21.15.38:0/2618293575 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f69080ab030 con 0x7f69141f3810 2024-04-25T08:32:52.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.189+0000 7f691a298640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f690006b4d0 0x7f690006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:52.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.189+0000 7f691a298640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f690006b4d0 0x7f690006d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f690c006fd0 tx=0x7f690c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:52.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.193+0000 7f6912ffd640 1 -- 172.21.15.38:0/2618293575 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6908005ac0 con 0x7f69141f3810 2024-04-25T08:32:52.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:52.411+0000 7f691c4fc640 1 -- 172.21.15.38:0/2618293575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f69140b3050 con 0x7f69141f3810 2024-04-25T08:32:53.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.064+0000 7f6912ffd640 1 -- 172.21.15.38:0/2618293575 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6908079760 con 0x7f69141f3810 2024-04-25T08:32:53.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.066+0000 7f691c4fc640 1 -- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f690006b4d0 msgr2=0x7f690006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.066+0000 7f691c4fc640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f690006b4d0 0x7f690006d990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f690c006fd0 tx=0x7f690c008040 comp rx=0 tx=0).stop 2024-04-25T08:32:53.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.066+0000 7f691c4fc640 1 -- 172.21.15.38:0/2618293575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69141f3810 msgr2=0x7f6914158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.066+0000 7f691c4fc640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69141f3810 0x7f6914158950 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f690800c300 tx=0x7f6908000fe0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.066+0000 7f691c4fc640 1 -- 172.21.15.38:0/2618293575 shutdown_connections 2024-04-25T08:32:53.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.066+0000 7f691c4fc640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69141f3c70 0x7f6914158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.066+0000 7f691c4fc640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69141f3810 0x7f6914158950 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.066+0000 7f691c4fc640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f690006b4d0 0x7f690006d990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.066+0000 7f691c4fc640 1 --2- 172.21.15.38:0/2618293575 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc004800 0x7f69140b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.066+0000 7f691c4fc640 1 -- 172.21.15.38:0/2618293575 >> 172.21.15.38:0/2618293575 conn(0x7f69140668f0 msgr2=0x7f69140ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:53.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.066+0000 7f691c4fc640 1 -- 172.21.15.38:0/2618293575 shutdown_connections 2024-04-25T08:32:53.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.066+0000 7f691c4fc640 1 -- 172.21.15.38:0/2618293575 wait complete. 2024-04-25T08:32:53.074 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:32:53.074 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:53.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.182+0000 7f365fb70640 1 Processor -- start 2024-04-25T08:32:53.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.182+0000 7f365fb70640 1 -- start start 2024-04-25T08:32:53.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.183+0000 7f365fb70640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f36580b5710 0x7f3658161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.183+0000 7f365fb70640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36580b93f0 0x7f36580d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.183+0000 7f365fb70640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36580b98a0 con 0x7f365815ddf0 2024-04-25T08:32:53.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.183+0000 7f365fb70640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3658161e00 con 0x7f36580b5710 2024-04-25T08:32:53.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.183+0000 7f365fb70640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36581620e0 con 0x7f36580b93f0 2024-04-25T08:32:53.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.183+0000 7f365e10d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36580b93f0 0x7f36580d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.183+0000 7f365e10d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36580b93f0 0x7f36580d2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:44996/0 (socket says 172.21.15.38:44996) 2024-04-25T08:32:53.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.183+0000 7f365e10d640 1 -- 172.21.15.38:0/2751943699 learned_addr learned my addr 172.21.15.38:0/2751943699 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:53.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.183+0000 7f365d90c640 1 --2- 172.21.15.38:0/2751943699 >> v2:172.21.15.107:3300/0 conn(0x7f36580b5710 0x7f3658161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.184+0000 7f365c90a640 1 -- 172.21.15.38:0/2751943699 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13256894 0 0) 0x7f36580b98a0 con 0x7f365815ddf0 2024-04-25T08:32:53.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.184+0000 7f365e10d640 1 -- 172.21.15.38:0/2751943699 >> v2:172.21.15.107:3300/0 conn(0x7f36580b5710 msgr2=0x7f3658161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:53.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.184+0000 7f365e10d640 1 --2- 172.21.15.38:0/2751943699 >> v2:172.21.15.107:3300/0 conn(0x7f36580b5710 0x7f3658161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.184+0000 7f365e10d640 1 -- 172.21.15.38:0/2751943699 >> v1:172.21.15.107:6789/0 conn(0x7f365815ddf0 legacy=0x7f36580b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.184+0000 7f365e10d640 1 -- 172.21.15.38:0/2751943699 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36581623c0 con 0x7f36580b93f0 2024-04-25T08:32:53.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.184+0000 7f365e10d640 1 --2- 172.21.15.38:0/2751943699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36580b93f0 0x7f36580d2340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3650009e50 tx=0x7f365002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=859c7839b0c281b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.185+0000 7f365c90a640 1 -- 172.21.15.38:0/2751943699 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3650036070 con 0x7f36580b93f0 2024-04-25T08:32:53.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.185+0000 7f365c90a640 1 -- 172.21.15.38:0/2751943699 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3650031600 con 0x7f36580b93f0 2024-04-25T08:32:53.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.185+0000 7f365c90a640 1 -- 172.21.15.38:0/2751943699 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36500319d0 con 0x7f36580b93f0 2024-04-25T08:32:53.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.185+0000 7f365fb70640 1 -- 172.21.15.38:0/2751943699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36580b93f0 msgr2=0x7f36580d2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.185+0000 7f365fb70640 1 --2- 172.21.15.38:0/2751943699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36580b93f0 0x7f36580d2340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3650009e50 tx=0x7f365002a710 comp rx=0 tx=0).stop 2024-04-25T08:32:53.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.185+0000 7f365fb70640 1 -- 172.21.15.38:0/2751943699 shutdown_connections 2024-04-25T08:32:53.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.185+0000 7f365fb70640 1 --2- 172.21.15.38:0/2751943699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36580b93f0 0x7f36580d2340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.185+0000 7f365fb70640 1 --2- 172.21.15.38:0/2751943699 >> v2:172.21.15.107:3300/0 conn(0x7f36580b5710 0x7f3658161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.185+0000 7f365fb70640 1 -- 172.21.15.38:0/2751943699 >> 172.21.15.38:0/2751943699 conn(0x7f36580d07c0 msgr2=0x7f36580d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:53.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.185+0000 7f365fb70640 1 -- 172.21.15.38:0/2751943699 shutdown_connections 2024-04-25T08:32:53.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.185+0000 7f365fb70640 1 -- 172.21.15.38:0/2751943699 wait complete. 2024-04-25T08:32:53.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.186+0000 7f365fb70640 1 Processor -- start 2024-04-25T08:32:53.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.186+0000 7f365fb70640 1 -- start start 2024-04-25T08:32:53.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.187+0000 7f365fb70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36580b5710 0x7f365814fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.187+0000 7f365fb70640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36580b93f0 0x7f36580caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.187+0000 7f365fb70640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f365815ddf0 0x7f36580b3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.187+0000 7f365fb70640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36580d2c10 con 0x7f36580b5710 2024-04-25T08:32:53.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.187+0000 7f365fb70640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36580d2a90 con 0x7f36580b93f0 2024-04-25T08:32:53.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.187+0000 7f365fb70640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36580d2d90 con 0x7f365815ddf0 2024-04-25T08:32:53.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.187+0000 7f365d90c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36580b5710 0x7f365814fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.187+0000 7f365d90c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36580b5710 0x7f365814fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57828/0 (socket says 172.21.15.38:57828) 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.187+0000 7f365d90c640 1 -- 172.21.15.38:0/1695295149 learned_addr learned my addr 172.21.15.38:0/1695295149 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.187+0000 7f365e10d640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f365815ddf0 0x7f36580b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.187+0000 7f365d10b640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36580b93f0 0x7f36580caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.188+0000 7f365d90c640 1 -- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f365815ddf0 msgr2=0x7f36580b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.188+0000 7f365d90c640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f365815ddf0 0x7f36580b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.188+0000 7f365d90c640 1 -- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36580b93f0 msgr2=0x7f36580caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.188+0000 7f365d90c640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36580b93f0 0x7f36580caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.188+0000 7f365d90c640 1 -- 172.21.15.38:0/1695295149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36580cb0e0 con 0x7f36580b5710 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.188+0000 7f365d10b640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36580b93f0 0x7f36580caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.188+0000 7f365d90c640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36580b5710 0x7f365814fe80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f365400e9b0 tx=0x7f365400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.188+0000 7f364e7fc640 1 -- 172.21.15.38:0/1695295149 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654017070 con 0x7f36580b5710 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.188+0000 7f364e7fc640 1 -- 172.21.15.38:0/1695295149 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365400c9a0 con 0x7f36580b5710 2024-04-25T08:32:53.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.188+0000 7f365fb70640 1 -- 172.21.15.38:0/1695295149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36581e7550 con 0x7f36580b5710 2024-04-25T08:32:53.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.188+0000 7f364e7fc640 1 -- 172.21.15.38:0/1695295149 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365400cd90 con 0x7f36580b5710 2024-04-25T08:32:53.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.188+0000 7f365fb70640 1 -- 172.21.15.38:0/1695295149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36580cc190 con 0x7f36580b5710 2024-04-25T08:32:53.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.189+0000 7f365fb70640 1 -- 172.21.15.38:0/1695295149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3658068e80 con 0x7f36580b5710 2024-04-25T08:32:53.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.189+0000 7f364e7fc640 1 -- 172.21.15.38:0/1695295149 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f365401a070 con 0x7f36580b5710 2024-04-25T08:32:53.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.190+0000 7f364e7fc640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f363c06b4d0 0x7f363c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.190+0000 7f364e7fc640 1 -- 172.21.15.38:0/1695295149 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f365408a370 con 0x7f36580b5710 2024-04-25T08:32:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.190+0000 7f365d10b640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f363c06b4d0 0x7f363c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.191+0000 7f365d10b640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f363c06b4d0 0x7f363c06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f365815d5b0 tx=0x7f3648009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.194+0000 7f364e7fc640 1 -- 172.21.15.38:0/1695295149 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3654059050 con 0x7f36580b5710 2024-04-25T08:32:53.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.413+0000 7f365fb70640 1 -- 172.21.15.38:0/1695295149 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36580b2e70 con 0x7f36580b5710 2024-04-25T08:32:53.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.413+0000 7f364e7fc640 1 -- 172.21.15.38:0/1695295149 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f3654058950 con 0x7f36580b5710 2024-04-25T08:32:53.409 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:53.409 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/422749749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":422749749},{"type":"v1","addr":"172.21.15.38:6834","nonce":422749749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5143,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/4194720752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":4194720752},{"type":"v1","addr":"172.21.15.107:6836","nonce":4194720752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3379884609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3379884609},{"type":"v1","addr":"172.21.15.107:6837","nonce":3379884609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:49.392279+0000","modified":"2024-04-25T08:32:52.412335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5149":{"gid":5149,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/234860294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":234860294},{"type":"v1","addr":"172.21.15.38:6835","nonce":234860294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:32:53.410 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 17 2024-04-25T08:32:53.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.415+0000 7f365fb70640 1 -- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f363c06b4d0 msgr2=0x7f363c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.415+0000 7f365fb70640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f363c06b4d0 0x7f363c06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f365815d5b0 tx=0x7f3648009040 comp rx=0 tx=0).stop 2024-04-25T08:32:53.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.415+0000 7f365fb70640 1 -- 172.21.15.38:0/1695295149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36580b5710 msgr2=0x7f365814fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.415+0000 7f365fb70640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36580b5710 0x7f365814fe80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f365400e9b0 tx=0x7f365400ee70 comp rx=0 tx=0).stop 2024-04-25T08:32:53.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.415+0000 7f365fb70640 1 -- 172.21.15.38:0/1695295149 shutdown_connections 2024-04-25T08:32:53.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.415+0000 7f365fb70640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f363c06b4d0 0x7f363c06d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.415+0000 7f365fb70640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f365815ddf0 0x7f36580b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.415+0000 7f365fb70640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36580b93f0 0x7f36580caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.415+0000 7f365fb70640 1 --2- 172.21.15.38:0/1695295149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36580b5710 0x7f365814fe80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.415+0000 7f365fb70640 1 -- 172.21.15.38:0/1695295149 >> 172.21.15.38:0/1695295149 conn(0x7f36580d07c0 msgr2=0x7f36580b4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:53.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.416+0000 7f365fb70640 1 -- 172.21.15.38:0/1695295149 shutdown_connections 2024-04-25T08:32:53.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.416+0000 7f365fb70640 1 -- 172.21.15.38:0/1695295149 wait complete. 2024-04-25T08:32:53.423 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:32:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.528+0000 7f7cca0f7640 1 Processor -- start 2024-04-25T08:32:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.528+0000 7f7cca0f7640 1 -- start start 2024-04-25T08:32:53.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cca0f7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7cc40b34e0 0x7f7cc40acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cca0f7640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7cc406c240 0x7f7cc40ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cca0f7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc406b1a0 con 0x7f7cc415da40 2024-04-25T08:32:53.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cca0f7640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc406b480 con 0x7f7cc40b34e0 2024-04-25T08:32:53.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cca0f7640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cc406b760 con 0x7f7cc406c240 2024-04-25T08:32:53.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cbbfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7cc40b34e0 0x7f7cc40acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cc37fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7cc415da40 0x7f7cc40b2dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47634/0 (socket says 172.21.15.38:47634) 2024-04-25T08:32:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cc37fe640 1 -- 172.21.15.38:0/3025936850 learned_addr learned my addr 172.21.15.38:0/3025936850 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cc3fff640 1 --2- 172.21.15.38:0/3025936850 >> v2:172.21.15.107:3300/0 conn(0x7f7cc406c240 0x7f7cc40ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cbbfff640 1 -- 172.21.15.38:0/3025936850 >> v2:172.21.15.107:3300/0 conn(0x7f7cc406c240 msgr2=0x7f7cc40ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cbbfff640 1 --2- 172.21.15.38:0/3025936850 >> v2:172.21.15.107:3300/0 conn(0x7f7cc406c240 0x7f7cc40ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cbbfff640 1 -- 172.21.15.38:0/3025936850 >> v1:172.21.15.38:6789/0 conn(0x7f7cc415da40 legacy=0x7f7cc40b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cbbfff640 1 -- 172.21.15.38:0/3025936850 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc406ba40 con 0x7f7cc40b34e0 2024-04-25T08:32:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.529+0000 7f7cc2ffd640 1 -- 172.21.15.38:0/3025936850 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087593587 0 0) 0x7f7cc406b1a0 con 0x7f7cc415da40 2024-04-25T08:32:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.530+0000 7f7cbbfff640 1 --2- 172.21.15.38:0/3025936850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc40b34e0 0x7f7cc40acdb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40b4660 tx=0x7f7cac02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67177bf4d176ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.530+0000 7f7cc2ffd640 1 -- 172.21.15.38:0/3025936850 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac037070 con 0x7f7cc40b34e0 2024-04-25T08:32:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.530+0000 7f7cc2ffd640 1 -- 172.21.15.38:0/3025936850 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac032440 con 0x7f7cc40b34e0 2024-04-25T08:32:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.530+0000 7f7cc2ffd640 1 -- 172.21.15.38:0/3025936850 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac02ae60 con 0x7f7cc40b34e0 2024-04-25T08:32:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.531+0000 7f7cca0f7640 1 -- 172.21.15.38:0/3025936850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc40b34e0 msgr2=0x7f7cc40acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.531+0000 7f7cca0f7640 1 --2- 172.21.15.38:0/3025936850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc40b34e0 0x7f7cc40acdb0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40b4660 tx=0x7f7cac02a110 comp rx=0 tx=0).stop 2024-04-25T08:32:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.531+0000 7f7cca0f7640 1 -- 172.21.15.38:0/3025936850 shutdown_connections 2024-04-25T08:32:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.531+0000 7f7cca0f7640 1 --2- 172.21.15.38:0/3025936850 >> v2:172.21.15.107:3300/0 conn(0x7f7cc406c240 0x7f7cc40ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.531+0000 7f7cca0f7640 1 --2- 172.21.15.38:0/3025936850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc40b34e0 0x7f7cc40acdb0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.531+0000 7f7cca0f7640 1 -- 172.21.15.38:0/3025936850 >> 172.21.15.38:0/3025936850 conn(0x7f7cc40cfdc0 msgr2=0x7f7cc40d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.531+0000 7f7cca0f7640 1 -- 172.21.15.38:0/3025936850 shutdown_connections 2024-04-25T08:32:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.531+0000 7f7cca0f7640 1 -- 172.21.15.38:0/3025936850 wait complete. 2024-04-25T08:32:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.532+0000 7f7cca0f7640 1 Processor -- start 2024-04-25T08:32:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.532+0000 7f7cca0f7640 1 -- start start 2024-04-25T08:32:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.532+0000 7f7cca0f7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc406c240 0x7f7cc415f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.532+0000 7f7cca0f7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc40b34e0 0x7f7cc40cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.532+0000 7f7cca0f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc415da40 0x7f7cc41f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cca0f7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40b3ac0 con 0x7f7cc415da40 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cca0f7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc415de20 con 0x7f7cc40b34e0 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cca0f7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40b3c40 con 0x7f7cc406c240 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc415da40 0x7f7cc41f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc415da40 0x7f7cc41f3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57856/0 (socket says 172.21.15.38:57856) 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc3fff640 1 -- 172.21.15.38:0/374554193 learned_addr learned my addr 172.21.15.38:0/374554193 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cbbfff640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc40b34e0 0x7f7cc40cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc37fe640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc406c240 0x7f7cc415f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc3fff640 1 -- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc406c240 msgr2=0x7f7cc415f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc3fff640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc406c240 0x7f7cc415f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc3fff640 1 -- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc40b34e0 msgr2=0x7f7cc40cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc3fff640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc40b34e0 0x7f7cc40cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc3fff640 1 -- 172.21.15.38:0/374554193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc415e340 con 0x7f7cc415da40 2024-04-25T08:32:53.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cbbfff640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc40b34e0 0x7f7cc40cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:53.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc3fff640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc415da40 0x7f7cc41f3ba0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc007e40 tx=0x7f7cbc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc0ff9640 1 -- 172.21.15.38:0/374554193 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc014620 con 0x7f7cc415da40 2024-04-25T08:32:53.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc0ff9640 1 -- 172.21.15.38:0/374554193 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cbc019070 con 0x7f7cc415da40 2024-04-25T08:32:53.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cc0ff9640 1 -- 172.21.15.38:0/374554193 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc014a20 con 0x7f7cc415da40 2024-04-25T08:32:53.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cca0f7640 1 -- 172.21.15.38:0/374554193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc415e620 con 0x7f7cc415da40 2024-04-25T08:32:53.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.533+0000 7f7cca0f7640 1 -- 172.21.15.38:0/374554193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc415ebe0 con 0x7f7cc415da40 2024-04-25T08:32:53.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.534+0000 7f7cca0f7640 1 -- 172.21.15.38:0/374554193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc41613c0 con 0x7f7cc415da40 2024-04-25T08:32:53.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.535+0000 7f7cc0ff9640 1 -- 172.21.15.38:0/374554193 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7cbc014bc0 con 0x7f7cc415da40 2024-04-25T08:32:53.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.535+0000 7f7cc0ff9640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ca406b4d0 0x7f7ca406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.535+0000 7f7cc0ff9640 1 -- 172.21.15.38:0/374554193 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7cbc08ac20 con 0x7f7cc415da40 2024-04-25T08:32:53.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.535+0000 7f7cc37fe640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ca406b4d0 0x7f7ca406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.536+0000 7f7cc37fe640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ca406b4d0 0x7f7ca406d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4000f60 tx=0x7f7cb4007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.539+0000 7f7cc0ff9640 1 -- 172.21.15.38:0/374554193 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7cbc059900 con 0x7f7cc415da40 2024-04-25T08:32:53.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.723+0000 7f7cca0f7640 1 -- 172.21.15.38:0/374554193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cc4066b50 con 0x7f7cc415da40 2024-04-25T08:32:53.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.724+0000 7f7cc0ff9640 1 -- 172.21.15.38:0/374554193 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16930 (secure 0 0 0) 0x7f7cbc059200 con 0x7f7cc415da40 2024-04-25T08:32:53.720 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:53.720 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":38,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:32:51.043321+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:47.798666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:48.371717+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:32:53.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.726+0000 7f7cca0f7640 1 -- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ca406b4d0 msgr2=0x7f7ca406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.726+0000 7f7cca0f7640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ca406b4d0 0x7f7ca406d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7cb4000f60 tx=0x7f7cb4007c00 comp rx=0 tx=0).stop 2024-04-25T08:32:53.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.726+0000 7f7cca0f7640 1 -- 172.21.15.38:0/374554193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc415da40 msgr2=0x7f7cc41f3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.726+0000 7f7cca0f7640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc415da40 0x7f7cc41f3ba0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc007e40 tx=0x7f7cbc00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.726+0000 7f7cca0f7640 1 -- 172.21.15.38:0/374554193 shutdown_connections 2024-04-25T08:32:53.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.726+0000 7f7cca0f7640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7cc415da40 0x7f7cc41f3ba0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.726+0000 7f7cca0f7640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc40b34e0 0x7f7cc40cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.726+0000 7f7cca0f7640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ca406b4d0 0x7f7ca406d990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.726+0000 7f7cca0f7640 1 --2- 172.21.15.38:0/374554193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc406c240 0x7f7cc415f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.726+0000 7f7cca0f7640 1 -- 172.21.15.38:0/374554193 >> 172.21.15.38:0/374554193 conn(0x7f7cc40cfdc0 msgr2=0x7f7cc406dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:53.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.726+0000 7f7cca0f7640 1 -- 172.21.15.38:0/374554193 shutdown_connections 2024-04-25T08:32:53.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.726+0000 7f7cca0f7640 1 -- 172.21.15.38:0/374554193 wait complete. 2024-04-25T08:32:53.733 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:32:53.733 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:32:53.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.837+0000 7f2e8609e640 1 Processor -- start 2024-04-25T08:32:53.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.837+0000 7f2e8609e640 1 -- start start 2024-04-25T08:32:53.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.838+0000 7f2e8609e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e8015fbf0 0x7f2e800c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.838+0000 7f2e8609e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e80160340 con 0x7f2e8015bb10 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.838+0000 7f2e8609e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e800cf400 con 0x7f2e8009ab90 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.838+0000 7f2e8609e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800cffd0 con 0x7f2e8015fbf0 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.838+0000 7f2e7f7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2e8015bb10 0x7f2e8009a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47646/0 (socket says 172.21.15.38:47646) 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.838+0000 7f2e7f7fe640 1 -- 172.21.15.38:0/2844137372 learned_addr learned my addr 172.21.15.38:0/2844137372 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.839+0000 7f2e7ffff640 1 --2- 172.21.15.38:0/2844137372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e8015fbf0 0x7f2e800c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.839+0000 7f2e7e7fc640 1 -- 172.21.15.38:0/2844137372 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 95719908 0 0) 0x7f2e70001110 con 0x7f2e8015bb10 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.839+0000 7f2e7e7fc640 1 -- 172.21.15.38:0/2844137372 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e68003520 con 0x7f2e8015bb10 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.839+0000 7f2e7e7fc640 1 -- 172.21.15.38:0/2844137372 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576617666 0 0) 0x7f2e800cf400 con 0x7f2e8009ab90 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.839+0000 7f2e7e7fc640 1 -- 172.21.15.38:0/2844137372 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e70001110 con 0x7f2e8009ab90 2024-04-25T08:32:53.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.839+0000 7f2e7ffff640 1 -- 172.21.15.38:0/2844137372 >> v1:172.21.15.107:6789/0 conn(0x7f2e8009ab90 legacy=0x7f2e8015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.839+0000 7f2e7ffff640 1 -- 172.21.15.38:0/2844137372 >> v1:172.21.15.38:6789/0 conn(0x7f2e8015bb10 legacy=0x7f2e8009a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.839+0000 7f2e7e7fc640 1 -- 172.21.15.38:0/2844137372 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3951358622 0 0) 0x7f2e68003520 con 0x7f2e8015bb10 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.839+0000 7f2e7ffff640 1 -- 172.21.15.38:0/2844137372 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e800cf6a0 con 0x7f2e8015fbf0 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.840+0000 7f2e7ffff640 1 --2- 172.21.15.38:0/2844137372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e8015fbf0 0x7f2e800c5250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009dc0 tx=0x7f2e7802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ed0f80ed758c931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.840+0000 7f2e7e7fc640 1 -- 172.21.15.38:0/2844137372 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70001840 con 0x7f2e8015fbf0 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.840+0000 7f2e7e7fc640 1 -- 172.21.15.38:0/2844137372 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e78007540 con 0x7f2e8015fbf0 2024-04-25T08:32:53.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.840+0000 7f2e7e7fc640 1 -- 172.21.15.38:0/2844137372 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e780078f0 con 0x7f2e8015fbf0 2024-04-25T08:32:53.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.840+0000 7f2e8609e640 1 -- 172.21.15.38:0/2844137372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e8015fbf0 msgr2=0x7f2e800c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.840+0000 7f2e8609e640 1 --2- 172.21.15.38:0/2844137372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e8015fbf0 0x7f2e800c5250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009dc0 tx=0x7f2e7802a5d0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.840+0000 7f2e8609e640 1 -- 172.21.15.38:0/2844137372 shutdown_connections 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.840+0000 7f2e8609e640 1 --2- 172.21.15.38:0/2844137372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e8015fbf0 0x7f2e800c5250 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.840+0000 7f2e8609e640 1 -- 172.21.15.38:0/2844137372 >> 172.21.15.38:0/2844137372 conn(0x7f2e800b31f0 msgr2=0x7f2e800b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.840+0000 7f2e8609e640 1 -- 172.21.15.38:0/2844137372 shutdown_connections 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.841+0000 7f2e8609e640 1 -- 172.21.15.38:0/2844137372 wait complete. 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.842+0000 7f2e8609e640 1 Processor -- start 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.842+0000 7f2e8609e640 1 -- start start 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.842+0000 7f2e8609e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e8009ab90 0x7f2e8006d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.842+0000 7f2e8609e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e8015bb10 0x7f2e8006c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.842+0000 7f2e8609e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e8015fbf0 0x7f2e80156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e8609e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800d0460 con 0x7f2e8009ab90 2024-04-25T08:32:53.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e8609e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800cfc60 con 0x7f2e8015fbf0 2024-04-25T08:32:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e8609e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e800d05e0 con 0x7f2e8015bb10 2024-04-25T08:32:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e8009ab90 0x7f2e8006d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e8009ab90 0x7f2e8006d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57872/0 (socket says 172.21.15.38:57872) 2024-04-25T08:32:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7f7fe640 1 -- 172.21.15.38:0/2774876368 learned_addr learned my addr 172.21.15.38:0/2774876368 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7f7fe640 1 -- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e8015bb10 msgr2=0x7f2e8006c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7ffff640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e8015fbf0 0x7f2e80156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7effd640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e8015bb10 0x7f2e8006c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7f7fe640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e8015bb10 0x7f2e8006c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7f7fe640 1 -- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e8015fbf0 msgr2=0x7f2e80156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7f7fe640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e8015fbf0 0x7f2e80156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:53.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7f7fe640 1 -- 172.21.15.38:0/2774876368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e80156d70 con 0x7f2e8009ab90 2024-04-25T08:32:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7ffff640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e8015fbf0 0x7f2e80156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7effd640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e8015bb10 0x7f2e8006c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e7f7fe640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e8009ab90 0x7f2e8006d880 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2e700019b0 tx=0x7f2e7000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e67fff640 1 -- 172.21.15.38:0/2774876368 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70017070 con 0x7f2e8009ab90 2024-04-25T08:32:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e67fff640 1 -- 172.21.15.38:0/2774876368 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e70005af0 con 0x7f2e8009ab90 2024-04-25T08:32:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e67fff640 1 -- 172.21.15.38:0/2774876368 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70007070 con 0x7f2e8009ab90 2024-04-25T08:32:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e8609e640 1 -- 172.21.15.38:0/2774876368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e800b5af0 con 0x7f2e8009ab90 2024-04-25T08:32:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.843+0000 7f2e8609e640 1 -- 172.21.15.38:0/2774876368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e800b60b0 con 0x7f2e8009ab90 2024-04-25T08:32:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.845+0000 7f2e8609e640 1 -- 172.21.15.38:0/2774876368 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e80068e80 con 0x7f2e8009ab90 2024-04-25T08:32:53.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.845+0000 7f2e67fff640 1 -- 172.21.15.38:0/2774876368 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e7001a070 con 0x7f2e8009ab90 2024-04-25T08:32:53.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.845+0000 7f2e67fff640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2e6c06b4d0 0x7f2e6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:53.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.845+0000 7f2e67fff640 1 -- 172.21.15.38:0/2774876368 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2e70089b20 con 0x7f2e8009ab90 2024-04-25T08:32:53.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.846+0000 7f2e7effd640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2e6c06b4d0 0x7f2e6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:53.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.846+0000 7f2e7effd640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2e6c06b4d0 0x7f2e6c06d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2e800cb430 tx=0x7f2e74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:53.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:53.850+0000 7f2e67fff640 1 -- 172.21.15.38:0/2774876368 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e80160340 con 0x7f2e8009ab90 2024-04-25T08:32:54.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.026+0000 7f2e8609e640 1 -- 172.21.15.38:0/2774876368 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e800cdbf0 con 0x7f2e6c06b4d0 2024-04-25T08:32:54.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.033+0000 7f2e67fff640 1 -- 172.21.15.38:0/2774876368 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251959 (secure 0 0 0) 0x7f2e800cdbf0 con 0x7f2e6c06b4d0 2024-04-25T08:32:54.032 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:54.033 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:32:54.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.036+0000 7f2e8609e640 1 -- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2e6c06b4d0 msgr2=0x7f2e6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.036+0000 7f2e8609e640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2e6c06b4d0 0x7f2e6c06d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2e800cb430 tx=0x7f2e74009040 comp rx=0 tx=0).stop 2024-04-25T08:32:54.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.036+0000 7f2e8609e640 1 -- 172.21.15.38:0/2774876368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e8009ab90 msgr2=0x7f2e8006d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.036+0000 7f2e8609e640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e8009ab90 0x7f2e8006d880 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2e700019b0 tx=0x7f2e7000cb90 comp rx=0 tx=0).stop 2024-04-25T08:32:54.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.036+0000 7f2e8609e640 1 -- 172.21.15.38:0/2774876368 shutdown_connections 2024-04-25T08:32:54.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.036+0000 7f2e8609e640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e8015fbf0 0x7f2e80156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.036+0000 7f2e8609e640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2e6c06b4d0 0x7f2e6c06d990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.036+0000 7f2e8609e640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e8015bb10 0x7f2e8006c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.036+0000 7f2e8609e640 1 --2- 172.21.15.38:0/2774876368 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e8009ab90 0x7f2e8006d880 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.036+0000 7f2e8609e640 1 -- 172.21.15.38:0/2774876368 >> 172.21.15.38:0/2774876368 conn(0x7f2e800b31f0 msgr2=0x7f2e800c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:54.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.036+0000 7f2e8609e640 1 -- 172.21.15.38:0/2774876368 shutdown_connections 2024-04-25T08:32:54.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.036+0000 7f2e8609e640 1 -- 172.21.15.38:0/2774876368 wait complete. 2024-04-25T08:32:54.043 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-04-25T08:32:53.223264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233288,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741432,"statfs":{"total":773094113280,"available":772855226368,"internally_reserved":0,"allocated":4759552,"data_stored":2709796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.199463"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136837+0000","last_change":"2024-04-25T08:32:50.067495+0000","last_active":"2024-04-25T08:32:51.136837+0000","last_peered":"2024-04-25T08:32:51.136837+0000","last_clean":"2024-04-25T08:32:51.136837+0000","last_became_active":"2024-04-25T08:32:50.065158+0000","last_became_peered":"2024-04-25T08:32:50.065158+0000","last_unstale":"2024-04-25T08:32:51.136837+0000","last_undegraded":"2024-04-25T08:32:51.136837+0000","last_fullsized":"2024-04-25T08:32:51.136837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:26.628616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136383+0000","last_change":"2024-04-25T08:32:50.069418+0000","last_active":"2024-04-25T08:32:51.136383+0000","last_peered":"2024-04-25T08:32:51.136383+0000","last_clean":"2024-04-25T08:32:51.136383+0000","last_became_active":"2024-04-25T08:32:50.069106+0000","last_became_peered":"2024-04-25T08:32:50.069106+0000","last_unstale":"2024-04-25T08:32:51.136383+0000","last_undegraded":"2024-04-25T08:32:51.136383+0000","last_fullsized":"2024-04-25T08:32:51.136383+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:03.755425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136319+0000","last_change":"2024-04-25T08:32:50.063645+0000","last_active":"2024-04-25T08:32:51.136319+0000","last_peered":"2024-04-25T08:32:51.136319+0000","last_clean":"2024-04-25T08:32:51.136319+0000","last_became_active":"2024-04-25T08:32:50.063347+0000","last_became_peered":"2024-04-25T08:32:50.063347+0000","last_unstale":"2024-04-25T08:32:51.136319+0000","last_undegraded":"2024-04-25T08:32:51.136319+0000","last_fullsized":"2024-04-25T08:32:51.136319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:04.174094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069583+0000","last_change":"2024-04-25T08:32:50.060924+0000","last_active":"2024-04-25T08:32:51.069583+0000","last_peered":"2024-04-25T08:32:51.069583+0000","last_clean":"2024-04-25T08:32:51.069583+0000","last_became_active":"2024-04-25T08:32:50.060443+0000","last_became_peered":"2024-04-25T08:32:50.060443+0000","last_unstale":"2024-04-25T08:32:51.069583+0000","last_undegraded":"2024-04-25T08:32:51.069583+0000","last_fullsized":"2024-04-25T08:32:51.069583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:03.187855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069540+0000","last_change":"2024-04-25T08:32:50.061268+0000","last_active":"2024-04-25T08:32:51.069540+0000","last_peered":"2024-04-25T08:32:51.069540+0000","last_clean":"2024-04-25T08:32:51.069540+0000","last_became_active":"2024-04-25T08:32:50.060925+0000","last_became_peered":"2024-04-25T08:32:50.060925+0000","last_unstale":"2024-04-25T08:32:51.069540+0000","last_undegraded":"2024-04-25T08:32:51.069540+0000","last_fullsized":"2024-04-25T08:32:51.069540+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:52.238653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135921+0000","last_change":"2024-04-25T08:32:50.071338+0000","last_active":"2024-04-25T08:32:51.135921+0000","last_peered":"2024-04-25T08:32:51.135921+0000","last_clean":"2024-04-25T08:32:51.135921+0000","last_became_active":"2024-04-25T08:32:50.070938+0000","last_became_peered":"2024-04-25T08:32:50.070938+0000","last_unstale":"2024-04-25T08:32:51.135921+0000","last_undegraded":"2024-04-25T08:32:51.135921+0000","last_fullsized":"2024-04-25T08:32:51.135921+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:47.788345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071631+0000","last_change":"2024-04-25T08:32:50.065520+0000","last_active":"2024-04-25T08:32:51.071631+0000","last_peered":"2024-04-25T08:32:51.071631+0000","last_clean":"2024-04-25T08:32:51.071631+0000","last_became_active":"2024-04-25T08:32:50.065195+0000","last_became_peered":"2024-04-25T08:32:50.065195+0000","last_unstale":"2024-04-25T08:32:51.071631+0000","last_undegraded":"2024-04-25T08:32:51.071631+0000","last_fullsized":"2024-04-25T08:32:51.071631+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:18.339044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135645+0000","last_change":"2024-04-25T08:32:50.073686+0000","last_active":"2024-04-25T08:32:51.135645+0000","last_peered":"2024-04-25T08:32:51.135645+0000","last_clean":"2024-04-25T08:32:51.135645+0000","last_became_active":"2024-04-25T08:32:50.073393+0000","last_became_peered":"2024-04-25T08:32:50.073393+0000","last_unstale":"2024-04-25T08:32:51.135645+0000","last_undegraded":"2024-04-25T08:32:51.135645+0000","last_fullsized":"2024-04-25T08:32:51.135645+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:44.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136649+0000","last_change":"2024-04-25T08:32:50.074775+0000","last_active":"2024-04-25T08:32:51.136649+0000","last_peered":"2024-04-25T08:32:51.136649+0000","last_clean":"2024-04-25T08:32:51.136649+0000","last_became_active":"2024-04-25T08:32:50.074467+0000","last_became_peered":"2024-04-25T08:32:50.074467+0000","last_unstale":"2024-04-25T08:32:51.136649+0000","last_undegraded":"2024-04-25T08:32:51.136649+0000","last_fullsized":"2024-04-25T08:32:51.136649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:47.867299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072092+0000","last_change":"2024-04-25T08:32:50.067608+0000","last_active":"2024-04-25T08:32:51.072092+0000","last_peered":"2024-04-25T08:32:51.072092+0000","last_clean":"2024-04-25T08:32:51.072092+0000","last_became_active":"2024-04-25T08:32:50.067046+0000","last_became_peered":"2024-04-25T08:32:50.067046+0000","last_unstale":"2024-04-25T08:32:51.072092+0000","last_undegraded":"2024-04-25T08:32:51.072092+0000","last_fullsized":"2024-04-25T08:32:51.072092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:03.755425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071982+0000","last_change":"2024-04-25T08:32:50.061486+0000","last_active":"2024-04-25T08:32:51.071982+0000","last_peered":"2024-04-25T08:32:51.071982+0000","last_clean":"2024-04-25T08:32:51.071982+0000","last_became_active":"2024-04-25T08:32:50.060917+0000","last_became_peered":"2024-04-25T08:32:50.060917+0000","last_unstale":"2024-04-25T08:32:51.071982+0000","last_undegraded":"2024-04-25T08:32:51.071982+0000","last_fullsized":"2024-04-25T08:32:51.071982+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:11.216952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136842+0000","last_change":"2024-04-25T08:32:50.066619+0000","last_active":"2024-04-25T08:32:51.136842+0000","last_peered":"2024-04-25T08:32:51.136842+0000","last_clean":"2024-04-25T08:32:51.136842+0000","last_became_active":"2024-04-25T08:32:50.066282+0000","last_became_peered":"2024-04-25T08:32:50.066282+0000","last_unstale":"2024-04-25T08:32:51.136842+0000","last_undegraded":"2024-04-25T08:32:51.136842+0000","last_fullsized":"2024-04-25T08:32:51.136842+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:35.594853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071877+0000","last_change":"2024-04-25T08:32:50.066834+0000","last_active":"2024-04-25T08:32:51.071877+0000","last_peered":"2024-04-25T08:32:51.071877+0000","last_clean":"2024-04-25T08:32:51.071877+0000","last_became_active":"2024-04-25T08:32:50.066587+0000","last_became_peered":"2024-04-25T08:32:50.066587+0000","last_unstale":"2024-04-25T08:32:51.071877+0000","last_undegraded":"2024-04-25T08:32:51.071877+0000","last_fullsized":"2024-04-25T08:32:51.071877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:53.965629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.053980+0000","last_change":"2024-04-25T08:32:50.052380+0000","last_active":"2024-04-25T08:32:51.053980+0000","last_peered":"2024-04-25T08:32:51.053980+0000","last_clean":"2024-04-25T08:32:51.053980+0000","last_became_active":"2024-04-25T08:32:50.052043+0000","last_became_peered":"2024-04-25T08:32:50.052043+0000","last_unstale":"2024-04-25T08:32:51.053980+0000","last_undegraded":"2024-04-25T08:32:51.053980+0000","last_fullsized":"2024-04-25T08:32:51.053980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:45.592577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071727+0000","last_change":"2024-04-25T08:32:50.064118+0000","last_active":"2024-04-25T08:32:51.071727+0000","last_peered":"2024-04-25T08:32:51.071727+0000","last_clean":"2024-04-25T08:32:51.071727+0000","last_became_active":"2024-04-25T08:32:50.063831+0000","last_became_peered":"2024-04-25T08:32:50.063831+0000","last_unstale":"2024-04-25T08:32:51.071727+0000","last_undegraded":"2024-04-25T08:32:51.071727+0000","last_fullsized":"2024-04-25T08:32:51.071727+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:35.594853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056040+0000","last_change":"2024-04-25T08:32:50.069948+0000","last_active":"2024-04-25T08:32:51.056040+0000","last_peered":"2024-04-25T08:32:51.056040+0000","last_clean":"2024-04-25T08:32:51.056040+0000","last_became_active":"2024-04-25T08:32:50.069658+0000","last_became_peered":"2024-04-25T08:32:50.069658+0000","last_unstale":"2024-04-25T08:32:51.056040+0000","last_undegraded":"2024-04-25T08:32:51.056040+0000","last_fullsized":"2024-04-25T08:32:51.056040+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:47.867299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056590+0000","last_change":"2024-04-25T08:32:50.070130+0000","last_active":"2024-04-25T08:32:51.056590+0000","last_peered":"2024-04-25T08:32:51.056590+0000","last_clean":"2024-04-25T08:32:51.056590+0000","last_became_active":"2024-04-25T08:32:50.069824+0000","last_became_peered":"2024-04-25T08:32:50.069824+0000","last_unstale":"2024-04-25T08:32:51.056590+0000","last_undegraded":"2024-04-25T08:32:51.056590+0000","last_fullsized":"2024-04-25T08:32:51.056590+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:03.755425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.053894+0000","last_change":"2024-04-25T08:32:50.053689+0000","last_active":"2024-04-25T08:32:51.053894+0000","last_peered":"2024-04-25T08:32:51.053894+0000","last_clean":"2024-04-25T08:32:51.053894+0000","last_became_active":"2024-04-25T08:32:50.053256+0000","last_became_peered":"2024-04-25T08:32:50.053256+0000","last_unstale":"2024-04-25T08:32:51.053894+0000","last_undegraded":"2024-04-25T08:32:51.053894+0000","last_fullsized":"2024-04-25T08:32:51.053894+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:45.592577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069724+0000","last_change":"2024-04-25T08:32:50.072264+0000","last_active":"2024-04-25T08:32:51.069724+0000","last_peered":"2024-04-25T08:32:51.069724+0000","last_clean":"2024-04-25T08:32:51.069724+0000","last_became_active":"2024-04-25T08:32:50.072012+0000","last_became_peered":"2024-04-25T08:32:50.072012+0000","last_unstale":"2024-04-25T08:32:51.069724+0000","last_undegraded":"2024-04-25T08:32:51.069724+0000","last_fullsized":"2024-04-25T08:32:51.069724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:29.161357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054084+0000","last_change":"2024-04-25T08:32:50.054361+0000","last_active":"2024-04-25T08:32:51.054084+0000","last_peered":"2024-04-25T08:32:51.054084+0000","last_clean":"2024-04-25T08:32:51.054084+0000","last_became_active":"2024-04-25T08:32:50.054090+0000","last_became_peered":"2024-04-25T08:32:50.054090+0000","last_unstale":"2024-04-25T08:32:51.054084+0000","last_undegraded":"2024-04-25T08:32:51.054084+0000","last_fullsized":"2024-04-25T08:32:51.054084+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:17.697869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071779+0000","last_change":"2024-04-25T08:32:50.069110+0000","last_active":"2024-04-25T08:32:51.071779+0000","last_peered":"2024-04-25T08:32:51.071779+0000","last_clean":"2024-04-25T08:32:51.071779+0000","last_became_active":"2024-04-25T08:32:50.068821+0000","last_became_peered":"2024-04-25T08:32:50.068821+0000","last_unstale":"2024-04-25T08:32:51.071779+0000","last_undegraded":"2024-04-25T08:32:51.071779+0000","last_fullsized":"2024-04-25T08:32:51.071779+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.005731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054430+0000","last_change":"2024-04-25T08:32:50.055696+0000","last_active":"2024-04-25T08:32:51.054430+0000","last_peered":"2024-04-25T08:32:51.054430+0000","last_clean":"2024-04-25T08:32:51.054430+0000","last_became_active":"2024-04-25T08:32:50.055393+0000","last_became_peered":"2024-04-25T08:32:50.055393+0000","last_unstale":"2024-04-25T08:32:51.054430+0000","last_undegraded":"2024-04-25T08:32:51.054430+0000","last_fullsized":"2024-04-25T08:32:51.054430+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:10.466200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056024+0000","last_change":"2024-04-25T08:32:50.068059+0000","last_active":"2024-04-25T08:32:51.056024+0000","last_peered":"2024-04-25T08:32:51.056024+0000","last_clean":"2024-04-25T08:32:51.056024+0000","last_became_active":"2024-04-25T08:32:50.067779+0000","last_became_peered":"2024-04-25T08:32:50.067779+0000","last_unstale":"2024-04-25T08:32:51.056024+0000","last_undegraded":"2024-04-25T08:32:51.056024+0000","last_fullsized":"2024-04-25T08:32:51.056024+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:18.339044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069678+0000","last_change":"2024-04-25T08:32:50.062730+0000","last_active":"2024-04-25T08:32:51.069678+0000","last_peered":"2024-04-25T08:32:51.069678+0000","last_clean":"2024-04-25T08:32:51.069678+0000","last_became_active":"2024-04-25T08:32:50.062400+0000","last_became_peered":"2024-04-25T08:32:50.062400+0000","last_unstale":"2024-04-25T08:32:51.069678+0000","last_undegraded":"2024-04-25T08:32:51.069678+0000","last_fullsized":"2024-04-25T08:32:51.069678+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:32.807198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054061+0000","last_change":"2024-04-25T08:32:50.062554+0000","last_active":"2024-04-25T08:32:51.054061+0000","last_peered":"2024-04-25T08:32:51.054061+0000","last_clean":"2024-04-25T08:32:51.054061+0000","last_became_active":"2024-04-25T08:32:50.062270+0000","last_became_peered":"2024-04-25T08:32:50.062270+0000","last_unstale":"2024-04-25T08:32:51.054061+0000","last_undegraded":"2024-04-25T08:32:51.054061+0000","last_fullsized":"2024-04-25T08:32:51.054061+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:23.561947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136182+0000","last_change":"2024-04-25T08:32:50.068420+0000","last_active":"2024-04-25T08:32:51.136182+0000","last_peered":"2024-04-25T08:32:51.136182+0000","last_clean":"2024-04-25T08:32:51.136182+0000","last_became_active":"2024-04-25T08:32:50.068045+0000","last_became_peered":"2024-04-25T08:32:50.068045+0000","last_unstale":"2024-04-25T08:32:51.136182+0000","last_undegraded":"2024-04-25T08:32:51.136182+0000","last_fullsized":"2024-04-25T08:32:51.136182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:28.896687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054932+0000","last_change":"2024-04-25T08:32:50.063195+0000","last_active":"2024-04-25T08:32:51.054932+0000","last_peered":"2024-04-25T08:32:51.054932+0000","last_clean":"2024-04-25T08:32:51.054932+0000","last_became_active":"2024-04-25T08:32:50.062936+0000","last_became_peered":"2024-04-25T08:32:50.062936+0000","last_unstale":"2024-04-25T08:32:51.054932+0000","last_undegraded":"2024-04-25T08:32:51.054932+0000","last_fullsized":"2024-04-25T08:32:51.054932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:18.150285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071954+0000","last_change":"2024-04-25T08:32:50.065701+0000","last_active":"2024-04-25T08:32:51.071954+0000","last_peered":"2024-04-25T08:32:51.071954+0000","last_clean":"2024-04-25T08:32:51.071954+0000","last_became_active":"2024-04-25T08:32:50.065299+0000","last_became_peered":"2024-04-25T08:32:50.065299+0000","last_unstale":"2024-04-25T08:32:51.071954+0000","last_undegraded":"2024-04-25T08:32:51.071954+0000","last_fullsized":"2024-04-25T08:32:51.071954+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:19.209886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.053811+0000","last_change":"2024-04-25T08:32:50.069330+0000","last_active":"2024-04-25T08:32:51.053811+0000","last_peered":"2024-04-25T08:32:51.053811+0000","last_clean":"2024-04-25T08:32:51.053811+0000","last_became_active":"2024-04-25T08:32:50.069042+0000","last_became_peered":"2024-04-25T08:32:50.069042+0000","last_unstale":"2024-04-25T08:32:51.053811+0000","last_undegraded":"2024-04-25T08:32:51.053811+0000","last_fullsized":"2024-04-25T08:32:51.053811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:47.572903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054934+0000","last_change":"2024-04-25T08:32:50.064837+0000","last_active":"2024-04-25T08:32:51.054934+0000","last_peered":"2024-04-25T08:32:51.054934+0000","last_clean":"2024-04-25T08:32:51.054934+0000","last_became_active":"2024-04-25T08:32:50.064467+0000","last_became_peered":"2024-04-25T08:32:50.064467+0000","last_unstale":"2024-04-25T08:32:51.054934+0000","last_undegraded":"2024-04-25T08:32:51.054934+0000","last_fullsized":"2024-04-25T08:32:51.054934+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:07.978309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056111+0000","last_change":"2024-04-25T08:32:50.068096+0000","last_active":"2024-04-25T08:32:51.056111+0000","last_peered":"2024-04-25T08:32:51.056111+0000","last_clean":"2024-04-25T08:32:51.056111+0000","last_became_active":"2024-04-25T08:32:50.067835+0000","last_became_peered":"2024-04-25T08:32:50.067835+0000","last_unstale":"2024-04-25T08:32:51.056111+0000","last_undegraded":"2024-04-25T08:32:51.056111+0000","last_fullsized":"2024-04-25T08:32:51.056111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.005731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135795+0000","last_change":"2024-04-25T08:32:50.068970+0000","last_active":"2024-04-25T08:32:51.135795+0000","last_peered":"2024-04-25T08:32:51.135795+0000","last_clean":"2024-04-25T08:32:51.135795+0000","last_became_active":"2024-04-25T08:32:50.068640+0000","last_became_peered":"2024-04-25T08:32:50.068640+0000","last_unstale":"2024-04-25T08:32:51.135795+0000","last_undegraded":"2024-04-25T08:32:51.135795+0000","last_fullsized":"2024-04-25T08:32:51.135795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:07.978309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056465+0000","last_change":"2024-04-25T08:32:50.066656+0000","last_active":"2024-04-25T08:32:51.056465+0000","last_peered":"2024-04-25T08:32:51.056465+0000","last_clean":"2024-04-25T08:32:51.056465+0000","last_became_active":"2024-04-25T08:32:50.066377+0000","last_became_peered":"2024-04-25T08:32:50.066377+0000","last_unstale":"2024-04-25T08:32:51.056465+0000","last_undegraded":"2024-04-25T08:32:51.056465+0000","last_fullsized":"2024-04-25T08:32:51.056465+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:53.965629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054941+0000","last_change":"2024-04-25T08:32:49.038030+0000","last_active":"2024-04-25T08:32:51.054941+0000","last_peered":"2024-04-25T08:32:51.054941+0000","last_clean":"2024-04-25T08:32:51.054941+0000","last_became_active":"2024-04-25T08:32:49.037752+0000","last_became_peered":"2024-04-25T08:32:49.037752+0000","last_unstale":"2024-04-25T08:32:51.054941+0000","last_undegraded":"2024-04-25T08:32:51.054941+0000","last_fullsized":"2024-04-25T08:32:51.054941+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:53.243334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136275+0000","last_change":"2024-04-25T08:32:50.068978+0000","last_active":"2024-04-25T08:32:51.136275+0000","last_peered":"2024-04-25T08:32:51.136275+0000","last_clean":"2024-04-25T08:32:51.136275+0000","last_became_active":"2024-04-25T08:32:50.068572+0000","last_became_peered":"2024-04-25T08:32:50.068572+0000","last_unstale":"2024-04-25T08:32:51.136275+0000","last_undegraded":"2024-04-25T08:32:51.136275+0000","last_fullsized":"2024-04-25T08:32:51.136275+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:19.436283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.137057+0000","last_change":"2024-04-25T08:32:49.049574+0000","last_active":"2024-04-25T08:32:51.137057+0000","last_peered":"2024-04-25T08:32:51.137057+0000","last_clean":"2024-04-25T08:32:51.137057+0000","last_became_active":"2024-04-25T08:32:49.049273+0000","last_became_peered":"2024-04-25T08:32:49.049273+0000","last_unstale":"2024-04-25T08:32:51.137057+0000","last_undegraded":"2024-04-25T08:32:51.137057+0000","last_fullsized":"2024-04-25T08:32:51.137057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:46.566748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054445+0000","last_change":"2024-04-25T08:32:50.052385+0000","last_active":"2024-04-25T08:32:51.054445+0000","last_peered":"2024-04-25T08:32:51.054445+0000","last_clean":"2024-04-25T08:32:51.054445+0000","last_became_active":"2024-04-25T08:32:50.052079+0000","last_became_peered":"2024-04-25T08:32:50.052079+0000","last_unstale":"2024-04-25T08:32:51.054445+0000","last_undegraded":"2024-04-25T08:32:51.054445+0000","last_fullsized":"2024-04-25T08:32:51.054445+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:47.788345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136451+0000","last_change":"2024-04-25T08:32:49.046231+0000","last_active":"2024-04-25T08:32:51.136451+0000","last_peered":"2024-04-25T08:32:51.136451+0000","last_clean":"2024-04-25T08:32:51.136451+0000","last_became_active":"2024-04-25T08:32:49.045431+0000","last_became_peered":"2024-04-25T08:32:49.045431+0000","last_unstale":"2024-04-25T08:32:51.136451+0000","last_undegraded":"2024-04-25T08:32:51.136451+0000","last_fullsized":"2024-04-25T08:32:51.136451+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:02.821888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136833+0000","last_change":"2024-04-25T08:32:50.062068+0000","last_active":"2024-04-25T08:32:51.136833+0000","last_peered":"2024-04-25T08:32:51.136833+0000","last_clean":"2024-04-25T08:32:51.136833+0000","last_became_active":"2024-04-25T08:32:50.061730+0000","last_became_peered":"2024-04-25T08:32:50.061730+0000","last_unstale":"2024-04-25T08:32:51.136833+0000","last_undegraded":"2024-04-25T08:32:51.136833+0000","last_fullsized":"2024-04-25T08:32:51.136833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:24.060540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054756+0000","last_change":"2024-04-25T08:32:49.038775+0000","last_active":"2024-04-25T08:32:51.054756+0000","last_peered":"2024-04-25T08:32:51.054756+0000","last_clean":"2024-04-25T08:32:51.054756+0000","last_became_active":"2024-04-25T08:32:49.038511+0000","last_became_peered":"2024-04-25T08:32:49.038511+0000","last_unstale":"2024-04-25T08:32:51.054756+0000","last_undegraded":"2024-04-25T08:32:51.054756+0000","last_fullsized":"2024-04-25T08:32:51.054756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:14.175021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136051+0000","last_change":"2024-04-25T08:32:50.075106+0000","last_active":"2024-04-25T08:32:51.136051+0000","last_peered":"2024-04-25T08:32:51.136051+0000","last_clean":"2024-04-25T08:32:51.136051+0000","last_became_active":"2024-04-25T08:32:50.074628+0000","last_became_peered":"2024-04-25T08:32:50.074628+0000","last_unstale":"2024-04-25T08:32:51.136051+0000","last_undegraded":"2024-04-25T08:32:51.136051+0000","last_fullsized":"2024-04-25T08:32:51.136051+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:19.436283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071960+0000","last_change":"2024-04-25T08:32:49.051766+0000","last_active":"2024-04-25T08:32:51.071960+0000","last_peered":"2024-04-25T08:32:51.071960+0000","last_clean":"2024-04-25T08:32:51.071960+0000","last_became_active":"2024-04-25T08:32:49.051469+0000","last_became_peered":"2024-04-25T08:32:49.051469+0000","last_unstale":"2024-04-25T08:32:51.071960+0000","last_undegraded":"2024-04-25T08:32:51.071960+0000","last_fullsized":"2024-04-25T08:32:51.071960+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:01.628880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071816+0000","last_change":"2024-04-25T08:32:50.070609+0000","last_active":"2024-04-25T08:32:51.071816+0000","last_peered":"2024-04-25T08:32:51.071816+0000","last_clean":"2024-04-25T08:32:51.071816+0000","last_became_active":"2024-04-25T08:32:50.070323+0000","last_became_peered":"2024-04-25T08:32:50.070323+0000","last_unstale":"2024-04-25T08:32:51.071816+0000","last_undegraded":"2024-04-25T08:32:51.071816+0000","last_fullsized":"2024-04-25T08:32:51.071816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:45.058682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.055996+0000","last_change":"2024-04-25T08:32:49.040573+0000","last_active":"2024-04-25T08:32:51.055996+0000","last_peered":"2024-04-25T08:32:51.055996+0000","last_clean":"2024-04-25T08:32:51.055996+0000","last_became_active":"2024-04-25T08:32:49.040312+0000","last_became_peered":"2024-04-25T08:32:49.040312+0000","last_unstale":"2024-04-25T08:32:51.055996+0000","last_undegraded":"2024-04-25T08:32:51.055996+0000","last_fullsized":"2024-04-25T08:32:51.055996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:16.691715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054329+0000","last_change":"2024-04-25T08:32:50.058024+0000","last_active":"2024-04-25T08:32:51.054329+0000","last_peered":"2024-04-25T08:32:51.054329+0000","last_clean":"2024-04-25T08:32:51.054329+0000","last_became_active":"2024-04-25T08:32:50.057747+0000","last_became_peered":"2024-04-25T08:32:50.057747+0000","last_unstale":"2024-04-25T08:32:51.054329+0000","last_undegraded":"2024-04-25T08:32:51.054329+0000","last_fullsized":"2024-04-25T08:32:51.054329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:18.150285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135966+0000","last_change":"2024-04-25T08:32:49.047169+0000","last_active":"2024-04-25T08:32:51.135966+0000","last_peered":"2024-04-25T08:32:51.135966+0000","last_clean":"2024-04-25T08:32:51.135966+0000","last_became_active":"2024-04-25T08:32:49.046692+0000","last_became_peered":"2024-04-25T08:32:49.046692+0000","last_unstale":"2024-04-25T08:32:51.135966+0000","last_undegraded":"2024-04-25T08:32:51.135966+0000","last_fullsized":"2024-04-25T08:32:51.135966+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:08.321886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135984+0000","last_change":"2024-04-25T08:32:50.070574+0000","last_active":"2024-04-25T08:32:51.135984+0000","last_peered":"2024-04-25T08:32:51.135984+0000","last_clean":"2024-04-25T08:32:51.135984+0000","last_became_active":"2024-04-25T08:32:50.070281+0000","last_became_peered":"2024-04-25T08:32:50.070281+0000","last_unstale":"2024-04-25T08:32:51.135984+0000","last_undegraded":"2024-04-25T08:32:51.135984+0000","last_fullsized":"2024-04-25T08:32:51.135984+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:45.058682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.053598+0000","last_change":"2024-04-25T08:32:49.037257+0000","last_active":"2024-04-25T08:32:51.053598+0000","last_peered":"2024-04-25T08:32:51.053598+0000","last_clean":"2024-04-25T08:32:51.053598+0000","last_became_active":"2024-04-25T08:32:49.036954+0000","last_became_peered":"2024-04-25T08:32:49.036954+0000","last_unstale":"2024-04-25T08:32:51.053598+0000","last_undegraded":"2024-04-25T08:32:51.053598+0000","last_fullsized":"2024-04-25T08:32:51.053598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:22.613342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069744+0000","last_change":"2024-04-25T08:32:50.061697+0000","last_active":"2024-04-25T08:32:51.069744+0000","last_peered":"2024-04-25T08:32:51.069744+0000","last_clean":"2024-04-25T08:32:51.069744+0000","last_became_active":"2024-04-25T08:32:50.061412+0000","last_became_peered":"2024-04-25T08:32:50.061412+0000","last_unstale":"2024-04-25T08:32:51.069744+0000","last_undegraded":"2024-04-25T08:32:51.069744+0000","last_fullsized":"2024-04-25T08:32:51.069744+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:15.297135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136904+0000","last_change":"2024-04-25T08:32:49.052317+0000","last_active":"2024-04-25T08:32:51.136904+0000","last_peered":"2024-04-25T08:32:51.136904+0000","last_clean":"2024-04-25T08:32:51.136904+0000","last_became_active":"2024-04-25T08:32:49.051952+0000","last_became_peered":"2024-04-25T08:32:49.051952+0000","last_unstale":"2024-04-25T08:32:51.136904+0000","last_undegraded":"2024-04-25T08:32:51.136904+0000","last_fullsized":"2024-04-25T08:32:51.136904+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:06.972154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056519+0000","last_change":"2024-04-25T08:32:50.070910+0000","last_active":"2024-04-25T08:32:51.056519+0000","last_peered":"2024-04-25T08:32:51.056519+0000","last_clean":"2024-04-25T08:32:51.056519+0000","last_became_active":"2024-04-25T08:32:50.070658+0000","last_became_peered":"2024-04-25T08:32:50.070658+0000","last_unstale":"2024-04-25T08:32:51.056519+0000","last_undegraded":"2024-04-25T08:32:51.056519+0000","last_fullsized":"2024-04-25T08:32:51.056519+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:19.209886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072222+0000","last_change":"2024-04-25T08:32:49.044181+0000","last_active":"2024-04-25T08:32:51.072222+0000","last_peered":"2024-04-25T08:32:51.072222+0000","last_clean":"2024-04-25T08:32:51.072222+0000","last_became_active":"2024-04-25T08:32:49.043680+0000","last_became_peered":"2024-04-25T08:32:49.043680+0000","last_unstale":"2024-04-25T08:32:51.072222+0000","last_undegraded":"2024-04-25T08:32:51.072222+0000","last_fullsized":"2024-04-25T08:32:51.072222+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:46.566748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.137105+0000","last_change":"2024-04-25T08:32:50.076163+0000","last_active":"2024-04-25T08:32:51.137105+0000","last_peered":"2024-04-25T08:32:51.137105+0000","last_clean":"2024-04-25T08:32:51.137105+0000","last_became_active":"2024-04-25T08:32:50.075848+0000","last_became_peered":"2024-04-25T08:32:50.075848+0000","last_unstale":"2024-04-25T08:32:51.137105+0000","last_undegraded":"2024-04-25T08:32:51.137105+0000","last_fullsized":"2024-04-25T08:32:51.137105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.005731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072068+0000","last_change":"2024-04-25T08:32:49.052395+0000","last_active":"2024-04-25T08:32:51.072068+0000","last_peered":"2024-04-25T08:32:51.072068+0000","last_clean":"2024-04-25T08:32:51.072068+0000","last_became_active":"2024-04-25T08:32:49.052190+0000","last_became_peered":"2024-04-25T08:32:49.052190+0000","last_unstale":"2024-04-25T08:32:51.072068+0000","last_undegraded":"2024-04-25T08:32:51.072068+0000","last_fullsized":"2024-04-25T08:32:51.072068+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:44.586423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054138+0000","last_change":"2024-04-25T08:32:50.055661+0000","last_active":"2024-04-25T08:32:51.054138+0000","last_peered":"2024-04-25T08:32:51.054138+0000","last_clean":"2024-04-25T08:32:51.054138+0000","last_became_active":"2024-04-25T08:32:50.055366+0000","last_became_peered":"2024-04-25T08:32:50.055366+0000","last_unstale":"2024-04-25T08:32:51.054138+0000","last_undegraded":"2024-04-25T08:32:51.054138+0000","last_fullsized":"2024-04-25T08:32:51.054138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:23.561947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136636+0000","last_change":"2024-04-25T08:32:49.040086+0000","last_active":"2024-04-25T08:32:51.136636+0000","last_peered":"2024-04-25T08:32:51.136636+0000","last_clean":"2024-04-25T08:32:51.136636+0000","last_became_active":"2024-04-25T08:32:49.039793+0000","last_became_peered":"2024-04-25T08:32:49.039793+0000","last_unstale":"2024-04-25T08:32:51.136636+0000","last_undegraded":"2024-04-25T08:32:51.136636+0000","last_fullsized":"2024-04-25T08:32:51.136636+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:09.460046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136192+0000","last_change":"2024-04-25T08:32:50.067236+0000","last_active":"2024-04-25T08:32:51.136192+0000","last_peered":"2024-04-25T08:32:51.136192+0000","last_clean":"2024-04-25T08:32:51.136192+0000","last_became_active":"2024-04-25T08:32:50.066931+0000","last_became_peered":"2024-04-25T08:32:50.066931+0000","last_unstale":"2024-04-25T08:32:51.136192+0000","last_undegraded":"2024-04-25T08:32:51.136192+0000","last_fullsized":"2024-04-25T08:32:51.136192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:44.651552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136413+0000","last_change":"2024-04-25T08:32:49.050784+0000","last_active":"2024-04-25T08:32:51.136413+0000","last_peered":"2024-04-25T08:32:51.136413+0000","last_clean":"2024-04-25T08:32:51.136413+0000","last_became_active":"2024-04-25T08:32:49.050521+0000","last_became_peered":"2024-04-25T08:32:49.050521+0000","last_unstale":"2024-04-25T08:32:51.136413+0000","last_undegraded":"2024-04-25T08:32:51.136413+0000","last_fullsized":"2024-04-25T08:32:51.136413+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:17.144131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136174+0000","last_change":"2024-04-25T08:32:50.070474+0000","last_active":"2024-04-25T08:32:51.136174+0000","last_peered":"2024-04-25T08:32:51.136174+0000","last_clean":"2024-04-25T08:32:51.136174+0000","last_became_active":"2024-04-25T08:32:50.069709+0000","last_became_peered":"2024-04-25T08:32:50.069709+0000","last_unstale":"2024-04-25T08:32:51.136174+0000","last_undegraded":"2024-04-25T08:32:51.136174+0000","last_fullsized":"2024-04-25T08:32:51.136174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:19.209886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135938+0000","last_change":"2024-04-25T08:32:49.049178+0000","last_active":"2024-04-25T08:32:51.135938+0000","last_peered":"2024-04-25T08:32:51.135938+0000","last_clean":"2024-04-25T08:32:51.135938+0000","last_became_active":"2024-04-25T08:32:49.048880+0000","last_became_peered":"2024-04-25T08:32:49.048880+0000","last_unstale":"2024-04-25T08:32:51.135938+0000","last_undegraded":"2024-04-25T08:32:51.135938+0000","last_fullsized":"2024-04-25T08:32:51.135938+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:50.208918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056029+0000","last_change":"2024-04-25T08:32:50.067331+0000","last_active":"2024-04-25T08:32:51.056029+0000","last_peered":"2024-04-25T08:32:51.056029+0000","last_clean":"2024-04-25T08:32:51.056029+0000","last_became_active":"2024-04-25T08:32:50.066994+0000","last_became_peered":"2024-04-25T08:32:50.066994+0000","last_unstale":"2024-04-25T08:32:51.056029+0000","last_undegraded":"2024-04-25T08:32:51.056029+0000","last_fullsized":"2024-04-25T08:32:51.056029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:35.594853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054701+0000","last_change":"2024-04-25T08:32:49.038492+0000","last_active":"2024-04-25T08:32:51.054701+0000","last_peered":"2024-04-25T08:32:51.054701+0000","last_clean":"2024-04-25T08:32:51.054701+0000","last_became_active":"2024-04-25T08:32:49.038194+0000","last_became_peered":"2024-04-25T08:32:49.038194+0000","last_unstale":"2024-04-25T08:32:51.054701+0000","last_undegraded":"2024-04-25T08:32:51.054701+0000","last_fullsized":"2024-04-25T08:32:51.054701+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:57.806567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136133+0000","last_change":"2024-04-25T08:32:50.069229+0000","last_active":"2024-04-25T08:32:51.136133+0000","last_peered":"2024-04-25T08:32:51.136133+0000","last_clean":"2024-04-25T08:32:51.136133+0000","last_became_active":"2024-04-25T08:32:50.068948+0000","last_became_peered":"2024-04-25T08:32:50.068948+0000","last_unstale":"2024-04-25T08:32:51.136133+0000","last_undegraded":"2024-04-25T08:32:51.136133+0000","last_fullsized":"2024-04-25T08:32:51.136133+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:53.965629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.055971+0000","last_change":"2024-04-25T08:32:49.039869+0000","last_active":"2024-04-25T08:32:51.055971+0000","last_peered":"2024-04-25T08:32:51.055971+0000","last_clean":"2024-04-25T08:32:51.055971+0000","last_became_active":"2024-04-25T08:32:49.039581+0000","last_became_peered":"2024-04-25T08:32:49.039581+0000","last_unstale":"2024-04-25T08:32:51.055971+0000","last_undegraded":"2024-04-25T08:32:51.055971+0000","last_fullsized":"2024-04-25T08:32:51.055971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:46.566748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136475+0000","last_change":"2024-04-25T08:32:50.063996+0000","last_active":"2024-04-25T08:32:51.136475+0000","last_peered":"2024-04-25T08:32:51.136475+0000","last_clean":"2024-04-25T08:32:51.136475+0000","last_became_active":"2024-04-25T08:32:50.063578+0000","last_became_peered":"2024-04-25T08:32:50.063578+0000","last_unstale":"2024-04-25T08:32:51.136475+0000","last_undegraded":"2024-04-25T08:32:51.136475+0000","last_fullsized":"2024-04-25T08:32:51.136475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:24.436448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072279+0000","last_change":"2024-04-25T08:32:49.046367+0000","last_active":"2024-04-25T08:32:51.072279+0000","last_peered":"2024-04-25T08:32:51.072279+0000","last_clean":"2024-04-25T08:32:51.072279+0000","last_became_active":"2024-04-25T08:32:49.046049+0000","last_became_peered":"2024-04-25T08:32:49.046049+0000","last_unstale":"2024-04-25T08:32:51.072279+0000","last_undegraded":"2024-04-25T08:32:51.072279+0000","last_fullsized":"2024-04-25T08:32:51.072279+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:22.555793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072118+0000","last_change":"2024-04-25T08:32:50.070743+0000","last_active":"2024-04-25T08:32:51.072118+0000","last_peered":"2024-04-25T08:32:51.072118+0000","last_clean":"2024-04-25T08:32:51.072118+0000","last_became_active":"2024-04-25T08:32:50.070406+0000","last_became_peered":"2024-04-25T08:32:50.070406+0000","last_unstale":"2024-04-25T08:32:51.072118+0000","last_undegraded":"2024-04-25T08:32:51.072118+0000","last_fullsized":"2024-04-25T08:32:51.072118+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:24.436448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054512+0000","last_change":"2024-04-25T08:32:49.043219+0000","last_active":"2024-04-25T08:32:51.054512+0000","last_peered":"2024-04-25T08:32:51.054512+0000","last_clean":"2024-04-25T08:32:51.054512+0000","last_became_active":"2024-04-25T08:32:49.042951+0000","last_became_peered":"2024-04-25T08:32:49.042951+0000","last_unstale":"2024-04-25T08:32:51.054512+0000","last_undegraded":"2024-04-25T08:32:51.054512+0000","last_fullsized":"2024-04-25T08:32:51.054512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:57.806567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054394+0000","last_change":"2024-04-25T08:32:50.069239+0000","last_active":"2024-04-25T08:32:51.054394+0000","last_peered":"2024-04-25T08:32:51.054394+0000","last_clean":"2024-04-25T08:32:51.054394+0000","last_became_active":"2024-04-25T08:32:50.068950+0000","last_became_peered":"2024-04-25T08:32:50.068950+0000","last_unstale":"2024-04-25T08:32:51.054394+0000","last_undegraded":"2024-04-25T08:32:51.054394+0000","last_fullsized":"2024-04-25T08:32:51.054394+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:07.978309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069823+0000","last_change":"2024-04-25T08:32:49.041981+0000","last_active":"2024-04-25T08:32:51.069823+0000","last_peered":"2024-04-25T08:32:51.069823+0000","last_clean":"2024-04-25T08:32:51.069823+0000","last_became_active":"2024-04-25T08:32:49.041659+0000","last_became_peered":"2024-04-25T08:32:49.041659+0000","last_unstale":"2024-04-25T08:32:51.069823+0000","last_undegraded":"2024-04-25T08:32:51.069823+0000","last_fullsized":"2024-04-25T08:32:51.069823+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:40.785082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069853+0000","last_change":"2024-04-25T08:32:50.071895+0000","last_active":"2024-04-25T08:32:51.069853+0000","last_peered":"2024-04-25T08:32:51.069853+0000","last_clean":"2024-04-25T08:32:51.069853+0000","last_became_active":"2024-04-25T08:32:50.071599+0000","last_became_peered":"2024-04-25T08:32:50.071599+0000","last_unstale":"2024-04-25T08:32:51.069853+0000","last_undegraded":"2024-04-25T08:32:51.069853+0000","last_fullsized":"2024-04-25T08:32:51.069853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:58.486734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136076+0000","last_change":"2024-04-25T08:32:49.042385+0000","last_active":"2024-04-25T08:32:51.136076+0000","last_peered":"2024-04-25T08:32:51.136076+0000","last_clean":"2024-04-25T08:32:51.136076+0000","last_became_active":"2024-04-25T08:32:49.042123+0000","last_became_peered":"2024-04-25T08:32:49.042123+0000","last_unstale":"2024-04-25T08:32:51.136076+0000","last_undegraded":"2024-04-25T08:32:51.136076+0000","last_fullsized":"2024-04-25T08:32:51.136076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:31.238532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056214+0000","last_change":"2024-04-25T08:32:50.068639+0000","last_active":"2024-04-25T08:32:51.056214+0000","last_peered":"2024-04-25T08:32:51.056214+0000","last_clean":"2024-04-25T08:32:51.056214+0000","last_became_active":"2024-04-25T08:32:50.068361+0000","last_became_peered":"2024-04-25T08:32:50.068361+0000","last_unstale":"2024-04-25T08:32:51.056214+0000","last_undegraded":"2024-04-25T08:32:51.056214+0000","last_fullsized":"2024-04-25T08:32:51.056214+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:44.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069522+0000","last_change":"2024-04-25T08:32:49.050532+0000","last_active":"2024-04-25T08:32:51.069522+0000","last_peered":"2024-04-25T08:32:51.069522+0000","last_clean":"2024-04-25T08:32:51.069522+0000","last_became_active":"2024-04-25T08:32:49.049717+0000","last_became_peered":"2024-04-25T08:32:49.049717+0000","last_unstale":"2024-04-25T08:32:51.069522+0000","last_undegraded":"2024-04-25T08:32:51.069522+0000","last_fullsized":"2024-04-25T08:32:51.069522+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:32.433112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136693+0000","last_change":"2024-04-25T08:32:50.063464+0000","last_active":"2024-04-25T08:32:51.136693+0000","last_peered":"2024-04-25T08:32:51.136693+0000","last_clean":"2024-04-25T08:32:51.136693+0000","last_became_active":"2024-04-25T08:32:50.063180+0000","last_became_peered":"2024-04-25T08:32:50.063180+0000","last_unstale":"2024-04-25T08:32:51.136693+0000","last_undegraded":"2024-04-25T08:32:51.136693+0000","last_fullsized":"2024-04-25T08:32:51.136693+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:18.339044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069536+0000","last_change":"2024-04-25T08:32:49.047513+0000","last_active":"2024-04-25T08:32:51.069536+0000","last_peered":"2024-04-25T08:32:51.069536+0000","last_clean":"2024-04-25T08:32:51.069536+0000","last_became_active":"2024-04-25T08:32:49.047185+0000","last_became_peered":"2024-04-25T08:32:49.047185+0000","last_unstale":"2024-04-25T08:32:51.069536+0000","last_undegraded":"2024-04-25T08:32:51.069536+0000","last_fullsized":"2024-04-25T08:32:51.069536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:20.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071817+0000","last_change":"2024-04-25T08:32:50.066103+0000","last_active":"2024-04-25T08:32:51.071817+0000","last_peered":"2024-04-25T08:32:51.071817+0000","last_clean":"2024-04-25T08:32:51.071817+0000","last_became_active":"2024-04-25T08:32:50.065792+0000","last_became_peered":"2024-04-25T08:32:50.065792+0000","last_unstale":"2024-04-25T08:32:51.071817+0000","last_undegraded":"2024-04-25T08:32:51.071817+0000","last_fullsized":"2024-04-25T08:32:51.071817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:44.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056177+0000","last_change":"2024-04-25T08:32:49.038521+0000","last_active":"2024-04-25T08:32:51.056177+0000","last_peered":"2024-04-25T08:32:51.056177+0000","last_clean":"2024-04-25T08:32:51.056177+0000","last_became_active":"2024-04-25T08:32:49.037788+0000","last_became_peered":"2024-04-25T08:32:49.037788+0000","last_unstale":"2024-04-25T08:32:51.056177+0000","last_undegraded":"2024-04-25T08:32:51.056177+0000","last_fullsized":"2024-04-25T08:32:51.056177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:22.555793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136302+0000","last_change":"2024-04-25T08:32:50.066619+0000","last_active":"2024-04-25T08:32:51.136302+0000","last_peered":"2024-04-25T08:32:51.136302+0000","last_clean":"2024-04-25T08:32:51.136302+0000","last_became_active":"2024-04-25T08:32:50.065788+0000","last_became_peered":"2024-04-25T08:32:50.065788+0000","last_unstale":"2024-04-25T08:32:51.136302+0000","last_undegraded":"2024-04-25T08:32:51.136302+0000","last_fullsized":"2024-04-25T08:32:51.136302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:11.216952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069561+0000","last_change":"2024-04-25T08:32:49.050402+0000","last_active":"2024-04-25T08:32:51.069561+0000","last_peered":"2024-04-25T08:32:51.069561+0000","last_clean":"2024-04-25T08:32:51.069561+0000","last_became_active":"2024-04-25T08:32:49.050116+0000","last_became_peered":"2024-04-25T08:32:49.050116+0000","last_unstale":"2024-04-25T08:32:51.069561+0000","last_undegraded":"2024-04-25T08:32:51.069561+0000","last_fullsized":"2024-04-25T08:32:51.069561+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:08.943644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135922+0000","last_change":"2024-04-25T08:32:50.063465+0000","last_active":"2024-04-25T08:32:51.135922+0000","last_peered":"2024-04-25T08:32:51.135922+0000","last_clean":"2024-04-25T08:32:51.135922+0000","last_became_active":"2024-04-25T08:32:50.063142+0000","last_became_peered":"2024-04-25T08:32:50.063142+0000","last_unstale":"2024-04-25T08:32:51.135922+0000","last_undegraded":"2024-04-25T08:32:51.135922+0000","last_fullsized":"2024-04-25T08:32:51.135922+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:32.331468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072357+0000","last_change":"2024-04-25T08:32:49.053278+0000","last_active":"2024-04-25T08:32:51.072357+0000","last_peered":"2024-04-25T08:32:51.072357+0000","last_clean":"2024-04-25T08:32:51.072357+0000","last_became_active":"2024-04-25T08:32:49.053046+0000","last_became_peered":"2024-04-25T08:32:49.053046+0000","last_unstale":"2024-04-25T08:32:51.072357+0000","last_undegraded":"2024-04-25T08:32:51.072357+0000","last_fullsized":"2024-04-25T08:32:51.072357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:16.691715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056575+0000","last_change":"2024-04-25T08:32:50.071575+0000","last_active":"2024-04-25T08:32:51.056575+0000","last_peered":"2024-04-25T08:32:51.056575+0000","last_clean":"2024-04-25T08:32:51.056575+0000","last_became_active":"2024-04-25T08:32:50.071246+0000","last_became_peered":"2024-04-25T08:32:50.071246+0000","last_unstale":"2024-04-25T08:32:51.056575+0000","last_undegraded":"2024-04-25T08:32:51.056575+0000","last_fullsized":"2024-04-25T08:32:51.056575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:11.216952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071071+0000","last_change":"2024-04-25T08:32:49.048263+0000","last_active":"2024-04-25T08:32:51.071071+0000","last_peered":"2024-04-25T08:32:51.071071+0000","last_clean":"2024-04-25T08:32:51.071071+0000","last_became_active":"2024-04-25T08:32:49.047976+0000","last_became_peered":"2024-04-25T08:32:49.047976+0000","last_unstale":"2024-04-25T08:32:51.071071+0000","last_undegraded":"2024-04-25T08:32:51.071071+0000","last_fullsized":"2024-04-25T08:32:51.071071+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:18.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136574+0000","last_change":"2024-04-25T08:32:50.069993+0000","last_active":"2024-04-25T08:32:51.136574+0000","last_peered":"2024-04-25T08:32:51.136574+0000","last_clean":"2024-04-25T08:32:51.136574+0000","last_became_active":"2024-04-25T08:32:50.069614+0000","last_became_peered":"2024-04-25T08:32:50.069614+0000","last_unstale":"2024-04-25T08:32:51.136574+0000","last_undegraded":"2024-04-25T08:32:51.136574+0000","last_fullsized":"2024-04-25T08:32:51.136574+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.438733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136410+0000","last_change":"2024-04-25T08:32:49.052616+0000","last_active":"2024-04-25T08:32:51.136410+0000","last_peered":"2024-04-25T08:32:51.136410+0000","last_clean":"2024-04-25T08:32:51.136410+0000","last_became_active":"2024-04-25T08:32:49.052316+0000","last_became_peered":"2024-04-25T08:32:49.052316+0000","last_unstale":"2024-04-25T08:32:51.136410+0000","last_undegraded":"2024-04-25T08:32:51.136410+0000","last_fullsized":"2024-04-25T08:32:51.136410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:38.738549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136334+0000","last_change":"2024-04-25T08:32:50.071805+0000","last_active":"2024-04-25T08:32:51.136334+0000","last_peered":"2024-04-25T08:32:51.136334+0000","last_clean":"2024-04-25T08:32:51.136334+0000","last_became_active":"2024-04-25T08:32:50.071541+0000","last_became_peered":"2024-04-25T08:32:50.071541+0000","last_unstale":"2024-04-25T08:32:51.136334+0000","last_undegraded":"2024-04-25T08:32:51.136334+0000","last_fullsized":"2024-04-25T08:32:51.136334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:51.810579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136683+0000","last_change":"2024-04-25T08:32:49.049702+0000","last_active":"2024-04-25T08:32:51.136683+0000","last_peered":"2024-04-25T08:32:51.136683+0000","last_clean":"2024-04-25T08:32:51.136683+0000","last_became_active":"2024-04-25T08:32:49.049411+0000","last_became_peered":"2024-04-25T08:32:49.049411+0000","last_unstale":"2024-04-25T08:32:51.136683+0000","last_undegraded":"2024-04-25T08:32:51.136683+0000","last_fullsized":"2024-04-25T08:32:51.136683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:01.628880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135857+0000","last_change":"2024-04-25T08:32:50.068961+0000","last_active":"2024-04-25T08:32:51.135857+0000","last_peered":"2024-04-25T08:32:51.135857+0000","last_clean":"2024-04-25T08:32:51.135857+0000","last_became_active":"2024-04-25T08:32:50.068618+0000","last_became_peered":"2024-04-25T08:32:50.068618+0000","last_unstale":"2024-04-25T08:32:51.135857+0000","last_undegraded":"2024-04-25T08:32:51.135857+0000","last_fullsized":"2024-04-25T08:32:51.135857+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:10.466200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054806+0000","last_change":"2024-04-25T08:32:49.049879+0000","last_active":"2024-04-25T08:32:51.054806+0000","last_peered":"2024-04-25T08:32:51.054806+0000","last_clean":"2024-04-25T08:32:51.054806+0000","last_became_active":"2024-04-25T08:32:49.049595+0000","last_became_peered":"2024-04-25T08:32:49.049595+0000","last_unstale":"2024-04-25T08:32:51.054806+0000","last_undegraded":"2024-04-25T08:32:51.054806+0000","last_fullsized":"2024-04-25T08:32:51.054806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:14.175021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054157+0000","last_change":"2024-04-25T08:32:50.063895+0000","last_active":"2024-04-25T08:32:51.054157+0000","last_peered":"2024-04-25T08:32:51.054157+0000","last_clean":"2024-04-25T08:32:51.054157+0000","last_became_active":"2024-04-25T08:32:50.063636+0000","last_became_peered":"2024-04-25T08:32:50.063636+0000","last_unstale":"2024-04-25T08:32:51.054157+0000","last_undegraded":"2024-04-25T08:32:51.054157+0000","last_fullsized":"2024-04-25T08:32:51.054157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:17.697869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.053690+0000","last_change":"2024-04-25T08:32:49.041583+0000","last_active":"2024-04-25T08:32:51.053690+0000","last_peered":"2024-04-25T08:32:51.053690+0000","last_clean":"2024-04-25T08:32:51.053690+0000","last_became_active":"2024-04-25T08:32:49.041316+0000","last_became_peered":"2024-04-25T08:32:49.041316+0000","last_unstale":"2024-04-25T08:32:51.053690+0000","last_undegraded":"2024-04-25T08:32:51.053690+0000","last_fullsized":"2024-04-25T08:32:51.053690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:53.243334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056248+0000","last_change":"2024-04-25T08:32:50.067107+0000","last_active":"2024-04-25T08:32:51.056248+0000","last_peered":"2024-04-25T08:32:51.056248+0000","last_clean":"2024-04-25T08:32:51.056248+0000","last_became_active":"2024-04-25T08:32:50.066800+0000","last_became_peered":"2024-04-25T08:32:50.066800+0000","last_unstale":"2024-04-25T08:32:51.056248+0000","last_undegraded":"2024-04-25T08:32:51.056248+0000","last_fullsized":"2024-04-25T08:32:51.056248+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:45.058682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135770+0000","last_change":"2024-04-25T08:32:49.046564+0000","last_active":"2024-04-25T08:32:51.135770+0000","last_peered":"2024-04-25T08:32:51.135770+0000","last_clean":"2024-04-25T08:32:51.135770+0000","last_became_active":"2024-04-25T08:32:49.045854+0000","last_became_peered":"2024-04-25T08:32:49.045854+0000","last_unstale":"2024-04-25T08:32:51.135770+0000","last_undegraded":"2024-04-25T08:32:51.135770+0000","last_fullsized":"2024-04-25T08:32:51.135770+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:16.691715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072238+0000","last_change":"2024-04-25T08:32:50.076660+0000","last_active":"2024-04-25T08:32:51.072238+0000","last_peered":"2024-04-25T08:32:51.072238+0000","last_clean":"2024-04-25T08:32:51.072238+0000","last_became_active":"2024-04-25T08:32:50.075600+0000","last_became_peered":"2024-04-25T08:32:50.075600+0000","last_unstale":"2024-04-25T08:32:51.072238+0000","last_undegraded":"2024-04-25T08:32:51.072238+0000","last_fullsized":"2024-04-25T08:32:51.072238+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.438733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056373+0000","last_change":"2024-04-25T08:32:49.043943+0000","last_active":"2024-04-25T08:32:51.056373+0000","last_peered":"2024-04-25T08:32:51.056373+0000","last_clean":"2024-04-25T08:32:51.056373+0000","last_became_active":"2024-04-25T08:32:49.043659+0000","last_became_peered":"2024-04-25T08:32:49.043659+0000","last_unstale":"2024-04-25T08:32:51.056373+0000","last_undegraded":"2024-04-25T08:32:51.056373+0000","last_fullsized":"2024-04-25T08:32:51.056373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:44.586423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056108+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:32:51.056108+0000","last_peered":"2024-04-25T08:32:51.056108+0000","last_clean":"2024-04-25T08:32:51.056108+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:32:51.056108+0000","last_undegraded":"2024-04-25T08:32:51.056108+0000","last_fullsized":"2024-04-25T08:32:51.056108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":782580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":191112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":191134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":192176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":190620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":780932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:32:54.047 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:32:54.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.189+0000 7f079724d640 1 Processor -- start 2024-04-25T08:32:54.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.189+0000 7f079724d640 1 -- start start 2024-04-25T08:32:54.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.190+0000 7f079724d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0790143f40 0x7f07900b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.190+0000 7f079724d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07900cdd40 0x7f079015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.190+0000 7f079724d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07900ce360 con 0x7f07900b4e60 2024-04-25T08:32:54.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.190+0000 7f079724d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07900ce640 con 0x7f0790143f40 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.190+0000 7f079724d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07900ce920 con 0x7f07900cdd40 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.190+0000 7f0794fe9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0790143f40 0x7f07900b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.190+0000 7f0794fe9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0790143f40 0x7f07900b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57874/0 (socket says 172.21.15.38:57874) 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.190+0000 7f0794fe9640 1 -- 172.21.15.38:0/3635865257 learned_addr learned my addr 172.21.15.38:0/3635865257 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.191+0000 7f07957ea640 1 --2- 172.21.15.38:0/3635865257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07900cdd40 0x7f079015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.191+0000 7f0794fe9640 1 -- 172.21.15.38:0/3635865257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07900cdd40 msgr2=0x7f079015abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.191+0000 7f0794fe9640 1 --2- 172.21.15.38:0/3635865257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07900cdd40 0x7f079015abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.191+0000 7f0794fe9640 1 -- 172.21.15.38:0/3635865257 >> v1:172.21.15.107:6789/0 conn(0x7f07900b4e60 legacy=0x7f07900cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.191+0000 7f0794fe9640 1 -- 172.21.15.38:0/3635865257 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07900cec00 con 0x7f0790143f40 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.192+0000 7f0794fe9640 1 --2- 172.21.15.38:0/3635865257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0790143f40 0x7f07900b4920 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0784009870 tx=0x7f078402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92eba18d0c015e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.192+0000 7f078f7fe640 1 -- 172.21.15.38:0/3635865257 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784037070 con 0x7f0790143f40 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.192+0000 7f078f7fe640 1 -- 172.21.15.38:0/3635865257 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0784032440 con 0x7f0790143f40 2024-04-25T08:32:54.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.192+0000 7f078f7fe640 1 -- 172.21.15.38:0/3635865257 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784032810 con 0x7f0790143f40 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.192+0000 7f079724d640 1 -- 172.21.15.38:0/3635865257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0790143f40 msgr2=0x7f07900b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.192+0000 7f079724d640 1 --2- 172.21.15.38:0/3635865257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0790143f40 0x7f07900b4920 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0784009870 tx=0x7f078402a270 comp rx=0 tx=0).stop 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.192+0000 7f079724d640 1 -- 172.21.15.38:0/3635865257 shutdown_connections 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.192+0000 7f079724d640 1 --2- 172.21.15.38:0/3635865257 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07900cdd40 0x7f079015abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.192+0000 7f079724d640 1 --2- 172.21.15.38:0/3635865257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0790143f40 0x7f07900b4920 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.192+0000 7f079724d640 1 -- 172.21.15.38:0/3635865257 >> 172.21.15.38:0/3635865257 conn(0x7f079006b210 msgr2=0x7f079006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.192+0000 7f079724d640 1 -- 172.21.15.38:0/3635865257 shutdown_connections 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.193+0000 7f079724d640 1 -- 172.21.15.38:0/3635865257 wait complete. 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.194+0000 7f079724d640 1 Processor -- start 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.194+0000 7f079724d640 1 -- start start 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f079724d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07900b4e60 0x7f079015b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f079724d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07900cdd40 0x7f0790141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f079724d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0790143f40 0x7f079013f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f079724d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07901444a0 con 0x7f07900b4e60 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f079724d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0790144320 con 0x7f07900cdd40 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f079724d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079015b130 con 0x7f0790143f40 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f0794fe9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07900b4e60 0x7f079015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f0794fe9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07900b4e60 0x7f079015b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57878/0 (socket says 172.21.15.38:57878) 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f0794fe9640 1 -- 172.21.15.38:0/389918198 learned_addr learned my addr 172.21.15.38:0/389918198 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f078ffff640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07900cdd40 0x7f0790141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f07957ea640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0790143f40 0x7f079013f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f0794fe9640 1 -- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0790143f40 msgr2=0x7f079013f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f0794fe9640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0790143f40 0x7f079013f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f0794fe9640 1 -- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07900cdd40 msgr2=0x7f0790141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f0794fe9640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07900cdd40 0x7f0790141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f0794fe9640 1 -- 172.21.15.38:0/389918198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07901f6dc0 con 0x7f07900b4e60 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f0794fe9640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07900b4e60 0x7f079015b590 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0784000c00 tx=0x7f07840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f078d7fa640 1 -- 172.21.15.38:0/389918198 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784041020 con 0x7f07900b4e60 2024-04-25T08:32:54.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f078d7fa640 1 -- 172.21.15.38:0/389918198 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07840306c0 con 0x7f07900b4e60 2024-04-25T08:32:54.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f079724d640 1 -- 172.21.15.38:0/389918198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07901f70a0 con 0x7f07900b4e60 2024-04-25T08:32:54.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f078d7fa640 1 -- 172.21.15.38:0/389918198 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784037040 con 0x7f07900b4e60 2024-04-25T08:32:54.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.195+0000 7f079724d640 1 -- 172.21.15.38:0/389918198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07901f7660 con 0x7f07900b4e60 2024-04-25T08:32:54.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.196+0000 7f079724d640 1 -- 172.21.15.38:0/389918198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0790068e80 con 0x7f07900b4e60 2024-04-25T08:32:54.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.197+0000 7f078d7fa640 1 -- 172.21.15.38:0/389918198 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0784044020 con 0x7f07900b4e60 2024-04-25T08:32:54.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.197+0000 7f078d7fa640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f077406b4d0 0x7f077406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.197+0000 7f078d7fa640 1 -- 172.21.15.38:0/389918198 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07840aac50 con 0x7f07900b4e60 2024-04-25T08:32:54.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.197+0000 7f078ffff640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f077406b4d0 0x7f077406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.198+0000 7f078ffff640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f077406b4d0 0x7f077406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f079006d420 tx=0x7f077c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.202+0000 7f078d7fa640 1 -- 172.21.15.38:0/389918198 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f078404b030 con 0x7f07900b4e60 2024-04-25T08:32:54.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.380+0000 7f079724d640 1 -- 172.21.15.38:0/389918198 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07901f6b70 con 0x7f077406b4d0 2024-04-25T08:32:54.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.386+0000 7f078d7fa640 1 -- 172.21.15.38:0/389918198 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251959 (secure 0 0 0) 0x7f07901f6b70 con 0x7f077406b4d0 2024-04-25T08:32:54.385 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:54.386 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:32:54.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.389+0000 7f079724d640 1 -- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f077406b4d0 msgr2=0x7f077406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.389+0000 7f079724d640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f077406b4d0 0x7f077406d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f079006d420 tx=0x7f077c009040 comp rx=0 tx=0).stop 2024-04-25T08:32:54.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.389+0000 7f079724d640 1 -- 172.21.15.38:0/389918198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07900b4e60 msgr2=0x7f079015b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.389+0000 7f079724d640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07900b4e60 0x7f079015b590 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0784000c00 tx=0x7f07840046c0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.390+0000 7f079724d640 1 -- 172.21.15.38:0/389918198 shutdown_connections 2024-04-25T08:32:54.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.390+0000 7f079724d640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f077406b4d0 0x7f077406d990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.390+0000 7f079724d640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0790143f40 0x7f079013f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.390+0000 7f079724d640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07900cdd40 0x7f0790141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.390+0000 7f079724d640 1 --2- 172.21.15.38:0/389918198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07900b4e60 0x7f079015b590 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.390+0000 7f079724d640 1 -- 172.21.15.38:0/389918198 >> 172.21.15.38:0/389918198 conn(0x7f079006b210 msgr2=0x7f0790072060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:54.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.390+0000 7f079724d640 1 -- 172.21.15.38:0/389918198 shutdown_connections 2024-04-25T08:32:54.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.390+0000 7f079724d640 1 -- 172.21.15.38:0/389918198 wait complete. 2024-04-25T08:32:54.396 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-04-25T08:32:53.223264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":233288,"kb_used_data":4648,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754741432,"statfs":{"total":773094113280,"available":772855226368,"internally_reserved":0,"allocated":4759552,"data_stored":2709796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.199463"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136837+0000","last_change":"2024-04-25T08:32:50.067495+0000","last_active":"2024-04-25T08:32:51.136837+0000","last_peered":"2024-04-25T08:32:51.136837+0000","last_clean":"2024-04-25T08:32:51.136837+0000","last_became_active":"2024-04-25T08:32:50.065158+0000","last_became_peered":"2024-04-25T08:32:50.065158+0000","last_unstale":"2024-04-25T08:32:51.136837+0000","last_undegraded":"2024-04-25T08:32:51.136837+0000","last_fullsized":"2024-04-25T08:32:51.136837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:26.628616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136383+0000","last_change":"2024-04-25T08:32:50.069418+0000","last_active":"2024-04-25T08:32:51.136383+0000","last_peered":"2024-04-25T08:32:51.136383+0000","last_clean":"2024-04-25T08:32:51.136383+0000","last_became_active":"2024-04-25T08:32:50.069106+0000","last_became_peered":"2024-04-25T08:32:50.069106+0000","last_unstale":"2024-04-25T08:32:51.136383+0000","last_undegraded":"2024-04-25T08:32:51.136383+0000","last_fullsized":"2024-04-25T08:32:51.136383+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:03.755425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136319+0000","last_change":"2024-04-25T08:32:50.063645+0000","last_active":"2024-04-25T08:32:51.136319+0000","last_peered":"2024-04-25T08:32:51.136319+0000","last_clean":"2024-04-25T08:32:51.136319+0000","last_became_active":"2024-04-25T08:32:50.063347+0000","last_became_peered":"2024-04-25T08:32:50.063347+0000","last_unstale":"2024-04-25T08:32:51.136319+0000","last_undegraded":"2024-04-25T08:32:51.136319+0000","last_fullsized":"2024-04-25T08:32:51.136319+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:04.174094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069583+0000","last_change":"2024-04-25T08:32:50.060924+0000","last_active":"2024-04-25T08:32:51.069583+0000","last_peered":"2024-04-25T08:32:51.069583+0000","last_clean":"2024-04-25T08:32:51.069583+0000","last_became_active":"2024-04-25T08:32:50.060443+0000","last_became_peered":"2024-04-25T08:32:50.060443+0000","last_unstale":"2024-04-25T08:32:51.069583+0000","last_undegraded":"2024-04-25T08:32:51.069583+0000","last_fullsized":"2024-04-25T08:32:51.069583+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:03.187855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069540+0000","last_change":"2024-04-25T08:32:50.061268+0000","last_active":"2024-04-25T08:32:51.069540+0000","last_peered":"2024-04-25T08:32:51.069540+0000","last_clean":"2024-04-25T08:32:51.069540+0000","last_became_active":"2024-04-25T08:32:50.060925+0000","last_became_peered":"2024-04-25T08:32:50.060925+0000","last_unstale":"2024-04-25T08:32:51.069540+0000","last_undegraded":"2024-04-25T08:32:51.069540+0000","last_fullsized":"2024-04-25T08:32:51.069540+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:52.238653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135921+0000","last_change":"2024-04-25T08:32:50.071338+0000","last_active":"2024-04-25T08:32:51.135921+0000","last_peered":"2024-04-25T08:32:51.135921+0000","last_clean":"2024-04-25T08:32:51.135921+0000","last_became_active":"2024-04-25T08:32:50.070938+0000","last_became_peered":"2024-04-25T08:32:50.070938+0000","last_unstale":"2024-04-25T08:32:51.135921+0000","last_undegraded":"2024-04-25T08:32:51.135921+0000","last_fullsized":"2024-04-25T08:32:51.135921+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:47.788345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071631+0000","last_change":"2024-04-25T08:32:50.065520+0000","last_active":"2024-04-25T08:32:51.071631+0000","last_peered":"2024-04-25T08:32:51.071631+0000","last_clean":"2024-04-25T08:32:51.071631+0000","last_became_active":"2024-04-25T08:32:50.065195+0000","last_became_peered":"2024-04-25T08:32:50.065195+0000","last_unstale":"2024-04-25T08:32:51.071631+0000","last_undegraded":"2024-04-25T08:32:51.071631+0000","last_fullsized":"2024-04-25T08:32:51.071631+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:18.339044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135645+0000","last_change":"2024-04-25T08:32:50.073686+0000","last_active":"2024-04-25T08:32:51.135645+0000","last_peered":"2024-04-25T08:32:51.135645+0000","last_clean":"2024-04-25T08:32:51.135645+0000","last_became_active":"2024-04-25T08:32:50.073393+0000","last_became_peered":"2024-04-25T08:32:50.073393+0000","last_unstale":"2024-04-25T08:32:51.135645+0000","last_undegraded":"2024-04-25T08:32:51.135645+0000","last_fullsized":"2024-04-25T08:32:51.135645+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:44.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136649+0000","last_change":"2024-04-25T08:32:50.074775+0000","last_active":"2024-04-25T08:32:51.136649+0000","last_peered":"2024-04-25T08:32:51.136649+0000","last_clean":"2024-04-25T08:32:51.136649+0000","last_became_active":"2024-04-25T08:32:50.074467+0000","last_became_peered":"2024-04-25T08:32:50.074467+0000","last_unstale":"2024-04-25T08:32:51.136649+0000","last_undegraded":"2024-04-25T08:32:51.136649+0000","last_fullsized":"2024-04-25T08:32:51.136649+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:47.867299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072092+0000","last_change":"2024-04-25T08:32:50.067608+0000","last_active":"2024-04-25T08:32:51.072092+0000","last_peered":"2024-04-25T08:32:51.072092+0000","last_clean":"2024-04-25T08:32:51.072092+0000","last_became_active":"2024-04-25T08:32:50.067046+0000","last_became_peered":"2024-04-25T08:32:50.067046+0000","last_unstale":"2024-04-25T08:32:51.072092+0000","last_undegraded":"2024-04-25T08:32:51.072092+0000","last_fullsized":"2024-04-25T08:32:51.072092+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:03.755425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071982+0000","last_change":"2024-04-25T08:32:50.061486+0000","last_active":"2024-04-25T08:32:51.071982+0000","last_peered":"2024-04-25T08:32:51.071982+0000","last_clean":"2024-04-25T08:32:51.071982+0000","last_became_active":"2024-04-25T08:32:50.060917+0000","last_became_peered":"2024-04-25T08:32:50.060917+0000","last_unstale":"2024-04-25T08:32:51.071982+0000","last_undegraded":"2024-04-25T08:32:51.071982+0000","last_fullsized":"2024-04-25T08:32:51.071982+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:11.216952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136842+0000","last_change":"2024-04-25T08:32:50.066619+0000","last_active":"2024-04-25T08:32:51.136842+0000","last_peered":"2024-04-25T08:32:51.136842+0000","last_clean":"2024-04-25T08:32:51.136842+0000","last_became_active":"2024-04-25T08:32:50.066282+0000","last_became_peered":"2024-04-25T08:32:50.066282+0000","last_unstale":"2024-04-25T08:32:51.136842+0000","last_undegraded":"2024-04-25T08:32:51.136842+0000","last_fullsized":"2024-04-25T08:32:51.136842+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:35.594853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071877+0000","last_change":"2024-04-25T08:32:50.066834+0000","last_active":"2024-04-25T08:32:51.071877+0000","last_peered":"2024-04-25T08:32:51.071877+0000","last_clean":"2024-04-25T08:32:51.071877+0000","last_became_active":"2024-04-25T08:32:50.066587+0000","last_became_peered":"2024-04-25T08:32:50.066587+0000","last_unstale":"2024-04-25T08:32:51.071877+0000","last_undegraded":"2024-04-25T08:32:51.071877+0000","last_fullsized":"2024-04-25T08:32:51.071877+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:53.965629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.053980+0000","last_change":"2024-04-25T08:32:50.052380+0000","last_active":"2024-04-25T08:32:51.053980+0000","last_peered":"2024-04-25T08:32:51.053980+0000","last_clean":"2024-04-25T08:32:51.053980+0000","last_became_active":"2024-04-25T08:32:50.052043+0000","last_became_peered":"2024-04-25T08:32:50.052043+0000","last_unstale":"2024-04-25T08:32:51.053980+0000","last_undegraded":"2024-04-25T08:32:51.053980+0000","last_fullsized":"2024-04-25T08:32:51.053980+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:45.592577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071727+0000","last_change":"2024-04-25T08:32:50.064118+0000","last_active":"2024-04-25T08:32:51.071727+0000","last_peered":"2024-04-25T08:32:51.071727+0000","last_clean":"2024-04-25T08:32:51.071727+0000","last_became_active":"2024-04-25T08:32:50.063831+0000","last_became_peered":"2024-04-25T08:32:50.063831+0000","last_unstale":"2024-04-25T08:32:51.071727+0000","last_undegraded":"2024-04-25T08:32:51.071727+0000","last_fullsized":"2024-04-25T08:32:51.071727+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:35.594853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056040+0000","last_change":"2024-04-25T08:32:50.069948+0000","last_active":"2024-04-25T08:32:51.056040+0000","last_peered":"2024-04-25T08:32:51.056040+0000","last_clean":"2024-04-25T08:32:51.056040+0000","last_became_active":"2024-04-25T08:32:50.069658+0000","last_became_peered":"2024-04-25T08:32:50.069658+0000","last_unstale":"2024-04-25T08:32:51.056040+0000","last_undegraded":"2024-04-25T08:32:51.056040+0000","last_fullsized":"2024-04-25T08:32:51.056040+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:47.867299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056590+0000","last_change":"2024-04-25T08:32:50.070130+0000","last_active":"2024-04-25T08:32:51.056590+0000","last_peered":"2024-04-25T08:32:51.056590+0000","last_clean":"2024-04-25T08:32:51.056590+0000","last_became_active":"2024-04-25T08:32:50.069824+0000","last_became_peered":"2024-04-25T08:32:50.069824+0000","last_unstale":"2024-04-25T08:32:51.056590+0000","last_undegraded":"2024-04-25T08:32:51.056590+0000","last_fullsized":"2024-04-25T08:32:51.056590+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:03.755425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.053894+0000","last_change":"2024-04-25T08:32:50.053689+0000","last_active":"2024-04-25T08:32:51.053894+0000","last_peered":"2024-04-25T08:32:51.053894+0000","last_clean":"2024-04-25T08:32:51.053894+0000","last_became_active":"2024-04-25T08:32:50.053256+0000","last_became_peered":"2024-04-25T08:32:50.053256+0000","last_unstale":"2024-04-25T08:32:51.053894+0000","last_undegraded":"2024-04-25T08:32:51.053894+0000","last_fullsized":"2024-04-25T08:32:51.053894+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:45.592577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069724+0000","last_change":"2024-04-25T08:32:50.072264+0000","last_active":"2024-04-25T08:32:51.069724+0000","last_peered":"2024-04-25T08:32:51.069724+0000","last_clean":"2024-04-25T08:32:51.069724+0000","last_became_active":"2024-04-25T08:32:50.072012+0000","last_became_peered":"2024-04-25T08:32:50.072012+0000","last_unstale":"2024-04-25T08:32:51.069724+0000","last_undegraded":"2024-04-25T08:32:51.069724+0000","last_fullsized":"2024-04-25T08:32:51.069724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:29.161357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054084+0000","last_change":"2024-04-25T08:32:50.054361+0000","last_active":"2024-04-25T08:32:51.054084+0000","last_peered":"2024-04-25T08:32:51.054084+0000","last_clean":"2024-04-25T08:32:51.054084+0000","last_became_active":"2024-04-25T08:32:50.054090+0000","last_became_peered":"2024-04-25T08:32:50.054090+0000","last_unstale":"2024-04-25T08:32:51.054084+0000","last_undegraded":"2024-04-25T08:32:51.054084+0000","last_fullsized":"2024-04-25T08:32:51.054084+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:17.697869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071779+0000","last_change":"2024-04-25T08:32:50.069110+0000","last_active":"2024-04-25T08:32:51.071779+0000","last_peered":"2024-04-25T08:32:51.071779+0000","last_clean":"2024-04-25T08:32:51.071779+0000","last_became_active":"2024-04-25T08:32:50.068821+0000","last_became_peered":"2024-04-25T08:32:50.068821+0000","last_unstale":"2024-04-25T08:32:51.071779+0000","last_undegraded":"2024-04-25T08:32:51.071779+0000","last_fullsized":"2024-04-25T08:32:51.071779+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.005731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054430+0000","last_change":"2024-04-25T08:32:50.055696+0000","last_active":"2024-04-25T08:32:51.054430+0000","last_peered":"2024-04-25T08:32:51.054430+0000","last_clean":"2024-04-25T08:32:51.054430+0000","last_became_active":"2024-04-25T08:32:50.055393+0000","last_became_peered":"2024-04-25T08:32:50.055393+0000","last_unstale":"2024-04-25T08:32:51.054430+0000","last_undegraded":"2024-04-25T08:32:51.054430+0000","last_fullsized":"2024-04-25T08:32:51.054430+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:10.466200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056024+0000","last_change":"2024-04-25T08:32:50.068059+0000","last_active":"2024-04-25T08:32:51.056024+0000","last_peered":"2024-04-25T08:32:51.056024+0000","last_clean":"2024-04-25T08:32:51.056024+0000","last_became_active":"2024-04-25T08:32:50.067779+0000","last_became_peered":"2024-04-25T08:32:50.067779+0000","last_unstale":"2024-04-25T08:32:51.056024+0000","last_undegraded":"2024-04-25T08:32:51.056024+0000","last_fullsized":"2024-04-25T08:32:51.056024+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:18.339044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069678+0000","last_change":"2024-04-25T08:32:50.062730+0000","last_active":"2024-04-25T08:32:51.069678+0000","last_peered":"2024-04-25T08:32:51.069678+0000","last_clean":"2024-04-25T08:32:51.069678+0000","last_became_active":"2024-04-25T08:32:50.062400+0000","last_became_peered":"2024-04-25T08:32:50.062400+0000","last_unstale":"2024-04-25T08:32:51.069678+0000","last_undegraded":"2024-04-25T08:32:51.069678+0000","last_fullsized":"2024-04-25T08:32:51.069678+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:32.807198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054061+0000","last_change":"2024-04-25T08:32:50.062554+0000","last_active":"2024-04-25T08:32:51.054061+0000","last_peered":"2024-04-25T08:32:51.054061+0000","last_clean":"2024-04-25T08:32:51.054061+0000","last_became_active":"2024-04-25T08:32:50.062270+0000","last_became_peered":"2024-04-25T08:32:50.062270+0000","last_unstale":"2024-04-25T08:32:51.054061+0000","last_undegraded":"2024-04-25T08:32:51.054061+0000","last_fullsized":"2024-04-25T08:32:51.054061+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:23.561947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136182+0000","last_change":"2024-04-25T08:32:50.068420+0000","last_active":"2024-04-25T08:32:51.136182+0000","last_peered":"2024-04-25T08:32:51.136182+0000","last_clean":"2024-04-25T08:32:51.136182+0000","last_became_active":"2024-04-25T08:32:50.068045+0000","last_became_peered":"2024-04-25T08:32:50.068045+0000","last_unstale":"2024-04-25T08:32:51.136182+0000","last_undegraded":"2024-04-25T08:32:51.136182+0000","last_fullsized":"2024-04-25T08:32:51.136182+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:28.896687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054932+0000","last_change":"2024-04-25T08:32:50.063195+0000","last_active":"2024-04-25T08:32:51.054932+0000","last_peered":"2024-04-25T08:32:51.054932+0000","last_clean":"2024-04-25T08:32:51.054932+0000","last_became_active":"2024-04-25T08:32:50.062936+0000","last_became_peered":"2024-04-25T08:32:50.062936+0000","last_unstale":"2024-04-25T08:32:51.054932+0000","last_undegraded":"2024-04-25T08:32:51.054932+0000","last_fullsized":"2024-04-25T08:32:51.054932+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:18.150285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071954+0000","last_change":"2024-04-25T08:32:50.065701+0000","last_active":"2024-04-25T08:32:51.071954+0000","last_peered":"2024-04-25T08:32:51.071954+0000","last_clean":"2024-04-25T08:32:51.071954+0000","last_became_active":"2024-04-25T08:32:50.065299+0000","last_became_peered":"2024-04-25T08:32:50.065299+0000","last_unstale":"2024-04-25T08:32:51.071954+0000","last_undegraded":"2024-04-25T08:32:51.071954+0000","last_fullsized":"2024-04-25T08:32:51.071954+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:19.209886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.053811+0000","last_change":"2024-04-25T08:32:50.069330+0000","last_active":"2024-04-25T08:32:51.053811+0000","last_peered":"2024-04-25T08:32:51.053811+0000","last_clean":"2024-04-25T08:32:51.053811+0000","last_became_active":"2024-04-25T08:32:50.069042+0000","last_became_peered":"2024-04-25T08:32:50.069042+0000","last_unstale":"2024-04-25T08:32:51.053811+0000","last_undegraded":"2024-04-25T08:32:51.053811+0000","last_fullsized":"2024-04-25T08:32:51.053811+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:47.572903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054934+0000","last_change":"2024-04-25T08:32:50.064837+0000","last_active":"2024-04-25T08:32:51.054934+0000","last_peered":"2024-04-25T08:32:51.054934+0000","last_clean":"2024-04-25T08:32:51.054934+0000","last_became_active":"2024-04-25T08:32:50.064467+0000","last_became_peered":"2024-04-25T08:32:50.064467+0000","last_unstale":"2024-04-25T08:32:51.054934+0000","last_undegraded":"2024-04-25T08:32:51.054934+0000","last_fullsized":"2024-04-25T08:32:51.054934+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:07.978309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056111+0000","last_change":"2024-04-25T08:32:50.068096+0000","last_active":"2024-04-25T08:32:51.056111+0000","last_peered":"2024-04-25T08:32:51.056111+0000","last_clean":"2024-04-25T08:32:51.056111+0000","last_became_active":"2024-04-25T08:32:50.067835+0000","last_became_peered":"2024-04-25T08:32:50.067835+0000","last_unstale":"2024-04-25T08:32:51.056111+0000","last_undegraded":"2024-04-25T08:32:51.056111+0000","last_fullsized":"2024-04-25T08:32:51.056111+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.005731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135795+0000","last_change":"2024-04-25T08:32:50.068970+0000","last_active":"2024-04-25T08:32:51.135795+0000","last_peered":"2024-04-25T08:32:51.135795+0000","last_clean":"2024-04-25T08:32:51.135795+0000","last_became_active":"2024-04-25T08:32:50.068640+0000","last_became_peered":"2024-04-25T08:32:50.068640+0000","last_unstale":"2024-04-25T08:32:51.135795+0000","last_undegraded":"2024-04-25T08:32:51.135795+0000","last_fullsized":"2024-04-25T08:32:51.135795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:07.978309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056465+0000","last_change":"2024-04-25T08:32:50.066656+0000","last_active":"2024-04-25T08:32:51.056465+0000","last_peered":"2024-04-25T08:32:51.056465+0000","last_clean":"2024-04-25T08:32:51.056465+0000","last_became_active":"2024-04-25T08:32:50.066377+0000","last_became_peered":"2024-04-25T08:32:50.066377+0000","last_unstale":"2024-04-25T08:32:51.056465+0000","last_undegraded":"2024-04-25T08:32:51.056465+0000","last_fullsized":"2024-04-25T08:32:51.056465+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:53.965629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"37'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054941+0000","last_change":"2024-04-25T08:32:49.038030+0000","last_active":"2024-04-25T08:32:51.054941+0000","last_peered":"2024-04-25T08:32:51.054941+0000","last_clean":"2024-04-25T08:32:51.054941+0000","last_became_active":"2024-04-25T08:32:49.037752+0000","last_became_peered":"2024-04-25T08:32:49.037752+0000","last_unstale":"2024-04-25T08:32:51.054941+0000","last_undegraded":"2024-04-25T08:32:51.054941+0000","last_fullsized":"2024-04-25T08:32:51.054941+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:53.243334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136275+0000","last_change":"2024-04-25T08:32:50.068978+0000","last_active":"2024-04-25T08:32:51.136275+0000","last_peered":"2024-04-25T08:32:51.136275+0000","last_clean":"2024-04-25T08:32:51.136275+0000","last_became_active":"2024-04-25T08:32:50.068572+0000","last_became_peered":"2024-04-25T08:32:50.068572+0000","last_unstale":"2024-04-25T08:32:51.136275+0000","last_undegraded":"2024-04-25T08:32:51.136275+0000","last_fullsized":"2024-04-25T08:32:51.136275+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:19.436283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.137057+0000","last_change":"2024-04-25T08:32:49.049574+0000","last_active":"2024-04-25T08:32:51.137057+0000","last_peered":"2024-04-25T08:32:51.137057+0000","last_clean":"2024-04-25T08:32:51.137057+0000","last_became_active":"2024-04-25T08:32:49.049273+0000","last_became_peered":"2024-04-25T08:32:49.049273+0000","last_unstale":"2024-04-25T08:32:51.137057+0000","last_undegraded":"2024-04-25T08:32:51.137057+0000","last_fullsized":"2024-04-25T08:32:51.137057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:46.566748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054445+0000","last_change":"2024-04-25T08:32:50.052385+0000","last_active":"2024-04-25T08:32:51.054445+0000","last_peered":"2024-04-25T08:32:51.054445+0000","last_clean":"2024-04-25T08:32:51.054445+0000","last_became_active":"2024-04-25T08:32:50.052079+0000","last_became_peered":"2024-04-25T08:32:50.052079+0000","last_unstale":"2024-04-25T08:32:51.054445+0000","last_undegraded":"2024-04-25T08:32:51.054445+0000","last_fullsized":"2024-04-25T08:32:51.054445+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:47.788345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136451+0000","last_change":"2024-04-25T08:32:49.046231+0000","last_active":"2024-04-25T08:32:51.136451+0000","last_peered":"2024-04-25T08:32:51.136451+0000","last_clean":"2024-04-25T08:32:51.136451+0000","last_became_active":"2024-04-25T08:32:49.045431+0000","last_became_peered":"2024-04-25T08:32:49.045431+0000","last_unstale":"2024-04-25T08:32:51.136451+0000","last_undegraded":"2024-04-25T08:32:51.136451+0000","last_fullsized":"2024-04-25T08:32:51.136451+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:02.821888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136833+0000","last_change":"2024-04-25T08:32:50.062068+0000","last_active":"2024-04-25T08:32:51.136833+0000","last_peered":"2024-04-25T08:32:51.136833+0000","last_clean":"2024-04-25T08:32:51.136833+0000","last_became_active":"2024-04-25T08:32:50.061730+0000","last_became_peered":"2024-04-25T08:32:50.061730+0000","last_unstale":"2024-04-25T08:32:51.136833+0000","last_undegraded":"2024-04-25T08:32:51.136833+0000","last_fullsized":"2024-04-25T08:32:51.136833+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:24.060540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054756+0000","last_change":"2024-04-25T08:32:49.038775+0000","last_active":"2024-04-25T08:32:51.054756+0000","last_peered":"2024-04-25T08:32:51.054756+0000","last_clean":"2024-04-25T08:32:51.054756+0000","last_became_active":"2024-04-25T08:32:49.038511+0000","last_became_peered":"2024-04-25T08:32:49.038511+0000","last_unstale":"2024-04-25T08:32:51.054756+0000","last_undegraded":"2024-04-25T08:32:51.054756+0000","last_fullsized":"2024-04-25T08:32:51.054756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:14.175021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136051+0000","last_change":"2024-04-25T08:32:50.075106+0000","last_active":"2024-04-25T08:32:51.136051+0000","last_peered":"2024-04-25T08:32:51.136051+0000","last_clean":"2024-04-25T08:32:51.136051+0000","last_became_active":"2024-04-25T08:32:50.074628+0000","last_became_peered":"2024-04-25T08:32:50.074628+0000","last_unstale":"2024-04-25T08:32:51.136051+0000","last_undegraded":"2024-04-25T08:32:51.136051+0000","last_fullsized":"2024-04-25T08:32:51.136051+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:19.436283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071960+0000","last_change":"2024-04-25T08:32:49.051766+0000","last_active":"2024-04-25T08:32:51.071960+0000","last_peered":"2024-04-25T08:32:51.071960+0000","last_clean":"2024-04-25T08:32:51.071960+0000","last_became_active":"2024-04-25T08:32:49.051469+0000","last_became_peered":"2024-04-25T08:32:49.051469+0000","last_unstale":"2024-04-25T08:32:51.071960+0000","last_undegraded":"2024-04-25T08:32:51.071960+0000","last_fullsized":"2024-04-25T08:32:51.071960+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:01.628880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071816+0000","last_change":"2024-04-25T08:32:50.070609+0000","last_active":"2024-04-25T08:32:51.071816+0000","last_peered":"2024-04-25T08:32:51.071816+0000","last_clean":"2024-04-25T08:32:51.071816+0000","last_became_active":"2024-04-25T08:32:50.070323+0000","last_became_peered":"2024-04-25T08:32:50.070323+0000","last_unstale":"2024-04-25T08:32:51.071816+0000","last_undegraded":"2024-04-25T08:32:51.071816+0000","last_fullsized":"2024-04-25T08:32:51.071816+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:45.058682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.055996+0000","last_change":"2024-04-25T08:32:49.040573+0000","last_active":"2024-04-25T08:32:51.055996+0000","last_peered":"2024-04-25T08:32:51.055996+0000","last_clean":"2024-04-25T08:32:51.055996+0000","last_became_active":"2024-04-25T08:32:49.040312+0000","last_became_peered":"2024-04-25T08:32:49.040312+0000","last_unstale":"2024-04-25T08:32:51.055996+0000","last_undegraded":"2024-04-25T08:32:51.055996+0000","last_fullsized":"2024-04-25T08:32:51.055996+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:16.691715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054329+0000","last_change":"2024-04-25T08:32:50.058024+0000","last_active":"2024-04-25T08:32:51.054329+0000","last_peered":"2024-04-25T08:32:51.054329+0000","last_clean":"2024-04-25T08:32:51.054329+0000","last_became_active":"2024-04-25T08:32:50.057747+0000","last_became_peered":"2024-04-25T08:32:50.057747+0000","last_unstale":"2024-04-25T08:32:51.054329+0000","last_undegraded":"2024-04-25T08:32:51.054329+0000","last_fullsized":"2024-04-25T08:32:51.054329+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:18.150285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135966+0000","last_change":"2024-04-25T08:32:49.047169+0000","last_active":"2024-04-25T08:32:51.135966+0000","last_peered":"2024-04-25T08:32:51.135966+0000","last_clean":"2024-04-25T08:32:51.135966+0000","last_became_active":"2024-04-25T08:32:49.046692+0000","last_became_peered":"2024-04-25T08:32:49.046692+0000","last_unstale":"2024-04-25T08:32:51.135966+0000","last_undegraded":"2024-04-25T08:32:51.135966+0000","last_fullsized":"2024-04-25T08:32:51.135966+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:08.321886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135984+0000","last_change":"2024-04-25T08:32:50.070574+0000","last_active":"2024-04-25T08:32:51.135984+0000","last_peered":"2024-04-25T08:32:51.135984+0000","last_clean":"2024-04-25T08:32:51.135984+0000","last_became_active":"2024-04-25T08:32:50.070281+0000","last_became_peered":"2024-04-25T08:32:50.070281+0000","last_unstale":"2024-04-25T08:32:51.135984+0000","last_undegraded":"2024-04-25T08:32:51.135984+0000","last_fullsized":"2024-04-25T08:32:51.135984+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:45.058682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.053598+0000","last_change":"2024-04-25T08:32:49.037257+0000","last_active":"2024-04-25T08:32:51.053598+0000","last_peered":"2024-04-25T08:32:51.053598+0000","last_clean":"2024-04-25T08:32:51.053598+0000","last_became_active":"2024-04-25T08:32:49.036954+0000","last_became_peered":"2024-04-25T08:32:49.036954+0000","last_unstale":"2024-04-25T08:32:51.053598+0000","last_undegraded":"2024-04-25T08:32:51.053598+0000","last_fullsized":"2024-04-25T08:32:51.053598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:22.613342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069744+0000","last_change":"2024-04-25T08:32:50.061697+0000","last_active":"2024-04-25T08:32:51.069744+0000","last_peered":"2024-04-25T08:32:51.069744+0000","last_clean":"2024-04-25T08:32:51.069744+0000","last_became_active":"2024-04-25T08:32:50.061412+0000","last_became_peered":"2024-04-25T08:32:50.061412+0000","last_unstale":"2024-04-25T08:32:51.069744+0000","last_undegraded":"2024-04-25T08:32:51.069744+0000","last_fullsized":"2024-04-25T08:32:51.069744+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:15.297135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136904+0000","last_change":"2024-04-25T08:32:49.052317+0000","last_active":"2024-04-25T08:32:51.136904+0000","last_peered":"2024-04-25T08:32:51.136904+0000","last_clean":"2024-04-25T08:32:51.136904+0000","last_became_active":"2024-04-25T08:32:49.051952+0000","last_became_peered":"2024-04-25T08:32:49.051952+0000","last_unstale":"2024-04-25T08:32:51.136904+0000","last_undegraded":"2024-04-25T08:32:51.136904+0000","last_fullsized":"2024-04-25T08:32:51.136904+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:06.972154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056519+0000","last_change":"2024-04-25T08:32:50.070910+0000","last_active":"2024-04-25T08:32:51.056519+0000","last_peered":"2024-04-25T08:32:51.056519+0000","last_clean":"2024-04-25T08:32:51.056519+0000","last_became_active":"2024-04-25T08:32:50.070658+0000","last_became_peered":"2024-04-25T08:32:50.070658+0000","last_unstale":"2024-04-25T08:32:51.056519+0000","last_undegraded":"2024-04-25T08:32:51.056519+0000","last_fullsized":"2024-04-25T08:32:51.056519+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:19.209886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072222+0000","last_change":"2024-04-25T08:32:49.044181+0000","last_active":"2024-04-25T08:32:51.072222+0000","last_peered":"2024-04-25T08:32:51.072222+0000","last_clean":"2024-04-25T08:32:51.072222+0000","last_became_active":"2024-04-25T08:32:49.043680+0000","last_became_peered":"2024-04-25T08:32:49.043680+0000","last_unstale":"2024-04-25T08:32:51.072222+0000","last_undegraded":"2024-04-25T08:32:51.072222+0000","last_fullsized":"2024-04-25T08:32:51.072222+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:46.566748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.137105+0000","last_change":"2024-04-25T08:32:50.076163+0000","last_active":"2024-04-25T08:32:51.137105+0000","last_peered":"2024-04-25T08:32:51.137105+0000","last_clean":"2024-04-25T08:32:51.137105+0000","last_became_active":"2024-04-25T08:32:50.075848+0000","last_became_peered":"2024-04-25T08:32:50.075848+0000","last_unstale":"2024-04-25T08:32:51.137105+0000","last_undegraded":"2024-04-25T08:32:51.137105+0000","last_fullsized":"2024-04-25T08:32:51.137105+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.005731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"37'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072068+0000","last_change":"2024-04-25T08:32:49.052395+0000","last_active":"2024-04-25T08:32:51.072068+0000","last_peered":"2024-04-25T08:32:51.072068+0000","last_clean":"2024-04-25T08:32:51.072068+0000","last_became_active":"2024-04-25T08:32:49.052190+0000","last_became_peered":"2024-04-25T08:32:49.052190+0000","last_unstale":"2024-04-25T08:32:51.072068+0000","last_undegraded":"2024-04-25T08:32:51.072068+0000","last_fullsized":"2024-04-25T08:32:51.072068+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:44.586423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054138+0000","last_change":"2024-04-25T08:32:50.055661+0000","last_active":"2024-04-25T08:32:51.054138+0000","last_peered":"2024-04-25T08:32:51.054138+0000","last_clean":"2024-04-25T08:32:51.054138+0000","last_became_active":"2024-04-25T08:32:50.055366+0000","last_became_peered":"2024-04-25T08:32:50.055366+0000","last_unstale":"2024-04-25T08:32:51.054138+0000","last_undegraded":"2024-04-25T08:32:51.054138+0000","last_fullsized":"2024-04-25T08:32:51.054138+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:23.561947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"37'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136636+0000","last_change":"2024-04-25T08:32:49.040086+0000","last_active":"2024-04-25T08:32:51.136636+0000","last_peered":"2024-04-25T08:32:51.136636+0000","last_clean":"2024-04-25T08:32:51.136636+0000","last_became_active":"2024-04-25T08:32:49.039793+0000","last_became_peered":"2024-04-25T08:32:49.039793+0000","last_unstale":"2024-04-25T08:32:51.136636+0000","last_undegraded":"2024-04-25T08:32:51.136636+0000","last_fullsized":"2024-04-25T08:32:51.136636+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:09.460046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136192+0000","last_change":"2024-04-25T08:32:50.067236+0000","last_active":"2024-04-25T08:32:51.136192+0000","last_peered":"2024-04-25T08:32:51.136192+0000","last_clean":"2024-04-25T08:32:51.136192+0000","last_became_active":"2024-04-25T08:32:50.066931+0000","last_became_peered":"2024-04-25T08:32:50.066931+0000","last_unstale":"2024-04-25T08:32:51.136192+0000","last_undegraded":"2024-04-25T08:32:51.136192+0000","last_fullsized":"2024-04-25T08:32:51.136192+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:44.651552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136413+0000","last_change":"2024-04-25T08:32:49.050784+0000","last_active":"2024-04-25T08:32:51.136413+0000","last_peered":"2024-04-25T08:32:51.136413+0000","last_clean":"2024-04-25T08:32:51.136413+0000","last_became_active":"2024-04-25T08:32:49.050521+0000","last_became_peered":"2024-04-25T08:32:49.050521+0000","last_unstale":"2024-04-25T08:32:51.136413+0000","last_undegraded":"2024-04-25T08:32:51.136413+0000","last_fullsized":"2024-04-25T08:32:51.136413+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:33:17.144131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136174+0000","last_change":"2024-04-25T08:32:50.070474+0000","last_active":"2024-04-25T08:32:51.136174+0000","last_peered":"2024-04-25T08:32:51.136174+0000","last_clean":"2024-04-25T08:32:51.136174+0000","last_became_active":"2024-04-25T08:32:50.069709+0000","last_became_peered":"2024-04-25T08:32:50.069709+0000","last_unstale":"2024-04-25T08:32:51.136174+0000","last_undegraded":"2024-04-25T08:32:51.136174+0000","last_fullsized":"2024-04-25T08:32:51.136174+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:19.209886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135938+0000","last_change":"2024-04-25T08:32:49.049178+0000","last_active":"2024-04-25T08:32:51.135938+0000","last_peered":"2024-04-25T08:32:51.135938+0000","last_clean":"2024-04-25T08:32:51.135938+0000","last_became_active":"2024-04-25T08:32:49.048880+0000","last_became_peered":"2024-04-25T08:32:49.048880+0000","last_unstale":"2024-04-25T08:32:51.135938+0000","last_undegraded":"2024-04-25T08:32:51.135938+0000","last_fullsized":"2024-04-25T08:32:51.135938+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:50.208918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056029+0000","last_change":"2024-04-25T08:32:50.067331+0000","last_active":"2024-04-25T08:32:51.056029+0000","last_peered":"2024-04-25T08:32:51.056029+0000","last_clean":"2024-04-25T08:32:51.056029+0000","last_became_active":"2024-04-25T08:32:50.066994+0000","last_became_peered":"2024-04-25T08:32:50.066994+0000","last_unstale":"2024-04-25T08:32:51.056029+0000","last_undegraded":"2024-04-25T08:32:51.056029+0000","last_fullsized":"2024-04-25T08:32:51.056029+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:35.594853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054701+0000","last_change":"2024-04-25T08:32:49.038492+0000","last_active":"2024-04-25T08:32:51.054701+0000","last_peered":"2024-04-25T08:32:51.054701+0000","last_clean":"2024-04-25T08:32:51.054701+0000","last_became_active":"2024-04-25T08:32:49.038194+0000","last_became_peered":"2024-04-25T08:32:49.038194+0000","last_unstale":"2024-04-25T08:32:51.054701+0000","last_undegraded":"2024-04-25T08:32:51.054701+0000","last_fullsized":"2024-04-25T08:32:51.054701+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:57.806567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136133+0000","last_change":"2024-04-25T08:32:50.069229+0000","last_active":"2024-04-25T08:32:51.136133+0000","last_peered":"2024-04-25T08:32:51.136133+0000","last_clean":"2024-04-25T08:32:51.136133+0000","last_became_active":"2024-04-25T08:32:50.068948+0000","last_became_peered":"2024-04-25T08:32:50.068948+0000","last_unstale":"2024-04-25T08:32:51.136133+0000","last_undegraded":"2024-04-25T08:32:51.136133+0000","last_fullsized":"2024-04-25T08:32:51.136133+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:53.965629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"37'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.055971+0000","last_change":"2024-04-25T08:32:49.039869+0000","last_active":"2024-04-25T08:32:51.055971+0000","last_peered":"2024-04-25T08:32:51.055971+0000","last_clean":"2024-04-25T08:32:51.055971+0000","last_became_active":"2024-04-25T08:32:49.039581+0000","last_became_peered":"2024-04-25T08:32:49.039581+0000","last_unstale":"2024-04-25T08:32:51.055971+0000","last_undegraded":"2024-04-25T08:32:51.055971+0000","last_fullsized":"2024-04-25T08:32:51.055971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:46.566748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136475+0000","last_change":"2024-04-25T08:32:50.063996+0000","last_active":"2024-04-25T08:32:51.136475+0000","last_peered":"2024-04-25T08:32:51.136475+0000","last_clean":"2024-04-25T08:32:51.136475+0000","last_became_active":"2024-04-25T08:32:50.063578+0000","last_became_peered":"2024-04-25T08:32:50.063578+0000","last_unstale":"2024-04-25T08:32:51.136475+0000","last_undegraded":"2024-04-25T08:32:51.136475+0000","last_fullsized":"2024-04-25T08:32:51.136475+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:24.436448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072279+0000","last_change":"2024-04-25T08:32:49.046367+0000","last_active":"2024-04-25T08:32:51.072279+0000","last_peered":"2024-04-25T08:32:51.072279+0000","last_clean":"2024-04-25T08:32:51.072279+0000","last_became_active":"2024-04-25T08:32:49.046049+0000","last_became_peered":"2024-04-25T08:32:49.046049+0000","last_unstale":"2024-04-25T08:32:51.072279+0000","last_undegraded":"2024-04-25T08:32:51.072279+0000","last_fullsized":"2024-04-25T08:32:51.072279+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:22.555793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072118+0000","last_change":"2024-04-25T08:32:50.070743+0000","last_active":"2024-04-25T08:32:51.072118+0000","last_peered":"2024-04-25T08:32:51.072118+0000","last_clean":"2024-04-25T08:32:51.072118+0000","last_became_active":"2024-04-25T08:32:50.070406+0000","last_became_peered":"2024-04-25T08:32:50.070406+0000","last_unstale":"2024-04-25T08:32:51.072118+0000","last_undegraded":"2024-04-25T08:32:51.072118+0000","last_fullsized":"2024-04-25T08:32:51.072118+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:24.436448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054512+0000","last_change":"2024-04-25T08:32:49.043219+0000","last_active":"2024-04-25T08:32:51.054512+0000","last_peered":"2024-04-25T08:32:51.054512+0000","last_clean":"2024-04-25T08:32:51.054512+0000","last_became_active":"2024-04-25T08:32:49.042951+0000","last_became_peered":"2024-04-25T08:32:49.042951+0000","last_unstale":"2024-04-25T08:32:51.054512+0000","last_undegraded":"2024-04-25T08:32:51.054512+0000","last_fullsized":"2024-04-25T08:32:51.054512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:57.806567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054394+0000","last_change":"2024-04-25T08:32:50.069239+0000","last_active":"2024-04-25T08:32:51.054394+0000","last_peered":"2024-04-25T08:32:51.054394+0000","last_clean":"2024-04-25T08:32:51.054394+0000","last_became_active":"2024-04-25T08:32:50.068950+0000","last_became_peered":"2024-04-25T08:32:50.068950+0000","last_unstale":"2024-04-25T08:32:51.054394+0000","last_undegraded":"2024-04-25T08:32:51.054394+0000","last_fullsized":"2024-04-25T08:32:51.054394+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:07.978309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"37'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069823+0000","last_change":"2024-04-25T08:32:49.041981+0000","last_active":"2024-04-25T08:32:51.069823+0000","last_peered":"2024-04-25T08:32:51.069823+0000","last_clean":"2024-04-25T08:32:51.069823+0000","last_became_active":"2024-04-25T08:32:49.041659+0000","last_became_peered":"2024-04-25T08:32:49.041659+0000","last_unstale":"2024-04-25T08:32:51.069823+0000","last_undegraded":"2024-04-25T08:32:51.069823+0000","last_fullsized":"2024-04-25T08:32:51.069823+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:40.785082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069853+0000","last_change":"2024-04-25T08:32:50.071895+0000","last_active":"2024-04-25T08:32:51.069853+0000","last_peered":"2024-04-25T08:32:51.069853+0000","last_clean":"2024-04-25T08:32:51.069853+0000","last_became_active":"2024-04-25T08:32:50.071599+0000","last_became_peered":"2024-04-25T08:32:50.071599+0000","last_unstale":"2024-04-25T08:32:51.069853+0000","last_undegraded":"2024-04-25T08:32:51.069853+0000","last_fullsized":"2024-04-25T08:32:51.069853+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:58.486734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136076+0000","last_change":"2024-04-25T08:32:49.042385+0000","last_active":"2024-04-25T08:32:51.136076+0000","last_peered":"2024-04-25T08:32:51.136076+0000","last_clean":"2024-04-25T08:32:51.136076+0000","last_became_active":"2024-04-25T08:32:49.042123+0000","last_became_peered":"2024-04-25T08:32:49.042123+0000","last_unstale":"2024-04-25T08:32:51.136076+0000","last_undegraded":"2024-04-25T08:32:51.136076+0000","last_fullsized":"2024-04-25T08:32:51.136076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:31.238532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056214+0000","last_change":"2024-04-25T08:32:50.068639+0000","last_active":"2024-04-25T08:32:51.056214+0000","last_peered":"2024-04-25T08:32:51.056214+0000","last_clean":"2024-04-25T08:32:51.056214+0000","last_became_active":"2024-04-25T08:32:50.068361+0000","last_became_peered":"2024-04-25T08:32:50.068361+0000","last_unstale":"2024-04-25T08:32:51.056214+0000","last_undegraded":"2024-04-25T08:32:51.056214+0000","last_fullsized":"2024-04-25T08:32:51.056214+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:44.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069522+0000","last_change":"2024-04-25T08:32:49.050532+0000","last_active":"2024-04-25T08:32:51.069522+0000","last_peered":"2024-04-25T08:32:51.069522+0000","last_clean":"2024-04-25T08:32:51.069522+0000","last_became_active":"2024-04-25T08:32:49.049717+0000","last_became_peered":"2024-04-25T08:32:49.049717+0000","last_unstale":"2024-04-25T08:32:51.069522+0000","last_undegraded":"2024-04-25T08:32:51.069522+0000","last_fullsized":"2024-04-25T08:32:51.069522+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:32.433112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136693+0000","last_change":"2024-04-25T08:32:50.063464+0000","last_active":"2024-04-25T08:32:51.136693+0000","last_peered":"2024-04-25T08:32:51.136693+0000","last_clean":"2024-04-25T08:32:51.136693+0000","last_became_active":"2024-04-25T08:32:50.063180+0000","last_became_peered":"2024-04-25T08:32:50.063180+0000","last_unstale":"2024-04-25T08:32:51.136693+0000","last_undegraded":"2024-04-25T08:32:51.136693+0000","last_fullsized":"2024-04-25T08:32:51.136693+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:18.339044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069536+0000","last_change":"2024-04-25T08:32:49.047513+0000","last_active":"2024-04-25T08:32:51.069536+0000","last_peered":"2024-04-25T08:32:51.069536+0000","last_clean":"2024-04-25T08:32:51.069536+0000","last_became_active":"2024-04-25T08:32:49.047185+0000","last_became_peered":"2024-04-25T08:32:49.047185+0000","last_unstale":"2024-04-25T08:32:51.069536+0000","last_undegraded":"2024-04-25T08:32:51.069536+0000","last_fullsized":"2024-04-25T08:32:51.069536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:20.574438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071817+0000","last_change":"2024-04-25T08:32:50.066103+0000","last_active":"2024-04-25T08:32:51.071817+0000","last_peered":"2024-04-25T08:32:51.071817+0000","last_clean":"2024-04-25T08:32:51.071817+0000","last_became_active":"2024-04-25T08:32:50.065792+0000","last_became_peered":"2024-04-25T08:32:50.065792+0000","last_unstale":"2024-04-25T08:32:51.071817+0000","last_undegraded":"2024-04-25T08:32:51.071817+0000","last_fullsized":"2024-04-25T08:32:51.071817+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:44.613294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056177+0000","last_change":"2024-04-25T08:32:49.038521+0000","last_active":"2024-04-25T08:32:51.056177+0000","last_peered":"2024-04-25T08:32:51.056177+0000","last_clean":"2024-04-25T08:32:51.056177+0000","last_became_active":"2024-04-25T08:32:49.037788+0000","last_became_peered":"2024-04-25T08:32:49.037788+0000","last_unstale":"2024-04-25T08:32:51.056177+0000","last_undegraded":"2024-04-25T08:32:51.056177+0000","last_fullsized":"2024-04-25T08:32:51.056177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:22.555793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136302+0000","last_change":"2024-04-25T08:32:50.066619+0000","last_active":"2024-04-25T08:32:51.136302+0000","last_peered":"2024-04-25T08:32:51.136302+0000","last_clean":"2024-04-25T08:32:51.136302+0000","last_became_active":"2024-04-25T08:32:50.065788+0000","last_became_peered":"2024-04-25T08:32:50.065788+0000","last_unstale":"2024-04-25T08:32:51.136302+0000","last_undegraded":"2024-04-25T08:32:51.136302+0000","last_fullsized":"2024-04-25T08:32:51.136302+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:11.216952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.069561+0000","last_change":"2024-04-25T08:32:49.050402+0000","last_active":"2024-04-25T08:32:51.069561+0000","last_peered":"2024-04-25T08:32:51.069561+0000","last_clean":"2024-04-25T08:32:51.069561+0000","last_became_active":"2024-04-25T08:32:49.050116+0000","last_became_peered":"2024-04-25T08:32:49.050116+0000","last_unstale":"2024-04-25T08:32:51.069561+0000","last_undegraded":"2024-04-25T08:32:51.069561+0000","last_fullsized":"2024-04-25T08:32:51.069561+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:08.943644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135922+0000","last_change":"2024-04-25T08:32:50.063465+0000","last_active":"2024-04-25T08:32:51.135922+0000","last_peered":"2024-04-25T08:32:51.135922+0000","last_clean":"2024-04-25T08:32:51.135922+0000","last_became_active":"2024-04-25T08:32:50.063142+0000","last_became_peered":"2024-04-25T08:32:50.063142+0000","last_unstale":"2024-04-25T08:32:51.135922+0000","last_undegraded":"2024-04-25T08:32:51.135922+0000","last_fullsized":"2024-04-25T08:32:51.135922+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:32.331468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"37'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072357+0000","last_change":"2024-04-25T08:32:49.053278+0000","last_active":"2024-04-25T08:32:51.072357+0000","last_peered":"2024-04-25T08:32:51.072357+0000","last_clean":"2024-04-25T08:32:51.072357+0000","last_became_active":"2024-04-25T08:32:49.053046+0000","last_became_peered":"2024-04-25T08:32:49.053046+0000","last_unstale":"2024-04-25T08:32:51.072357+0000","last_undegraded":"2024-04-25T08:32:51.072357+0000","last_fullsized":"2024-04-25T08:32:51.072357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:16.691715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056575+0000","last_change":"2024-04-25T08:32:50.071575+0000","last_active":"2024-04-25T08:32:51.056575+0000","last_peered":"2024-04-25T08:32:51.056575+0000","last_clean":"2024-04-25T08:32:51.056575+0000","last_became_active":"2024-04-25T08:32:50.071246+0000","last_became_peered":"2024-04-25T08:32:50.071246+0000","last_unstale":"2024-04-25T08:32:51.056575+0000","last_undegraded":"2024-04-25T08:32:51.056575+0000","last_fullsized":"2024-04-25T08:32:51.056575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:11.216952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"38'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.071071+0000","last_change":"2024-04-25T08:32:49.048263+0000","last_active":"2024-04-25T08:32:51.071071+0000","last_peered":"2024-04-25T08:32:51.071071+0000","last_clean":"2024-04-25T08:32:51.071071+0000","last_became_active":"2024-04-25T08:32:49.047976+0000","last_became_peered":"2024-04-25T08:32:49.047976+0000","last_unstale":"2024-04-25T08:32:51.071071+0000","last_undegraded":"2024-04-25T08:32:51.071071+0000","last_fullsized":"2024-04-25T08:32:51.071071+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:18.141737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136574+0000","last_change":"2024-04-25T08:32:50.069993+0000","last_active":"2024-04-25T08:32:51.136574+0000","last_peered":"2024-04-25T08:32:51.136574+0000","last_clean":"2024-04-25T08:32:51.136574+0000","last_became_active":"2024-04-25T08:32:50.069614+0000","last_became_peered":"2024-04-25T08:32:50.069614+0000","last_unstale":"2024-04-25T08:32:51.136574+0000","last_undegraded":"2024-04-25T08:32:51.136574+0000","last_fullsized":"2024-04-25T08:32:51.136574+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.438733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136410+0000","last_change":"2024-04-25T08:32:49.052616+0000","last_active":"2024-04-25T08:32:51.136410+0000","last_peered":"2024-04-25T08:32:51.136410+0000","last_clean":"2024-04-25T08:32:51.136410+0000","last_became_active":"2024-04-25T08:32:49.052316+0000","last_became_peered":"2024-04-25T08:32:49.052316+0000","last_unstale":"2024-04-25T08:32:51.136410+0000","last_undegraded":"2024-04-25T08:32:51.136410+0000","last_fullsized":"2024-04-25T08:32:51.136410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:38.738549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136334+0000","last_change":"2024-04-25T08:32:50.071805+0000","last_active":"2024-04-25T08:32:51.136334+0000","last_peered":"2024-04-25T08:32:51.136334+0000","last_clean":"2024-04-25T08:32:51.136334+0000","last_became_active":"2024-04-25T08:32:50.071541+0000","last_became_peered":"2024-04-25T08:32:50.071541+0000","last_unstale":"2024-04-25T08:32:51.136334+0000","last_undegraded":"2024-04-25T08:32:51.136334+0000","last_fullsized":"2024-04-25T08:32:51.136334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:51.810579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.136683+0000","last_change":"2024-04-25T08:32:49.049702+0000","last_active":"2024-04-25T08:32:51.136683+0000","last_peered":"2024-04-25T08:32:51.136683+0000","last_clean":"2024-04-25T08:32:51.136683+0000","last_became_active":"2024-04-25T08:32:49.049411+0000","last_became_peered":"2024-04-25T08:32:49.049411+0000","last_unstale":"2024-04-25T08:32:51.136683+0000","last_undegraded":"2024-04-25T08:32:51.136683+0000","last_fullsized":"2024-04-25T08:32:51.136683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:01.628880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135857+0000","last_change":"2024-04-25T08:32:50.068961+0000","last_active":"2024-04-25T08:32:51.135857+0000","last_peered":"2024-04-25T08:32:51.135857+0000","last_clean":"2024-04-25T08:32:51.135857+0000","last_became_active":"2024-04-25T08:32:50.068618+0000","last_became_peered":"2024-04-25T08:32:50.068618+0000","last_unstale":"2024-04-25T08:32:51.135857+0000","last_undegraded":"2024-04-25T08:32:51.135857+0000","last_fullsized":"2024-04-25T08:32:51.135857+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:10.466200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054806+0000","last_change":"2024-04-25T08:32:49.049879+0000","last_active":"2024-04-25T08:32:51.054806+0000","last_peered":"2024-04-25T08:32:51.054806+0000","last_clean":"2024-04-25T08:32:51.054806+0000","last_became_active":"2024-04-25T08:32:49.049595+0000","last_became_peered":"2024-04-25T08:32:49.049595+0000","last_unstale":"2024-04-25T08:32:51.054806+0000","last_undegraded":"2024-04-25T08:32:51.054806+0000","last_fullsized":"2024-04-25T08:32:51.054806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:14.175021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.054157+0000","last_change":"2024-04-25T08:32:50.063895+0000","last_active":"2024-04-25T08:32:51.054157+0000","last_peered":"2024-04-25T08:32:51.054157+0000","last_clean":"2024-04-25T08:32:51.054157+0000","last_became_active":"2024-04-25T08:32:50.063636+0000","last_became_peered":"2024-04-25T08:32:50.063636+0000","last_unstale":"2024-04-25T08:32:51.054157+0000","last_undegraded":"2024-04-25T08:32:51.054157+0000","last_fullsized":"2024-04-25T08:32:51.054157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:17.697869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.053690+0000","last_change":"2024-04-25T08:32:49.041583+0000","last_active":"2024-04-25T08:32:51.053690+0000","last_peered":"2024-04-25T08:32:51.053690+0000","last_clean":"2024-04-25T08:32:51.053690+0000","last_became_active":"2024-04-25T08:32:49.041316+0000","last_became_peered":"2024-04-25T08:32:49.041316+0000","last_unstale":"2024-04-25T08:32:51.053690+0000","last_undegraded":"2024-04-25T08:32:51.053690+0000","last_fullsized":"2024-04-25T08:32:51.053690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:53.243334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056248+0000","last_change":"2024-04-25T08:32:50.067107+0000","last_active":"2024-04-25T08:32:51.056248+0000","last_peered":"2024-04-25T08:32:51.056248+0000","last_clean":"2024-04-25T08:32:51.056248+0000","last_became_active":"2024-04-25T08:32:50.066800+0000","last_became_peered":"2024-04-25T08:32:50.066800+0000","last_unstale":"2024-04-25T08:32:51.056248+0000","last_undegraded":"2024-04-25T08:32:51.056248+0000","last_fullsized":"2024-04-25T08:32:51.056248+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:45.058682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.135770+0000","last_change":"2024-04-25T08:32:49.046564+0000","last_active":"2024-04-25T08:32:51.135770+0000","last_peered":"2024-04-25T08:32:51.135770+0000","last_clean":"2024-04-25T08:32:51.135770+0000","last_became_active":"2024-04-25T08:32:49.045854+0000","last_became_peered":"2024-04-25T08:32:49.045854+0000","last_unstale":"2024-04-25T08:32:51.135770+0000","last_undegraded":"2024-04-25T08:32:51.135770+0000","last_fullsized":"2024-04-25T08:32:51.135770+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:16.691715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.072238+0000","last_change":"2024-04-25T08:32:50.076660+0000","last_active":"2024-04-25T08:32:51.072238+0000","last_peered":"2024-04-25T08:32:51.072238+0000","last_clean":"2024-04-25T08:32:51.072238+0000","last_became_active":"2024-04-25T08:32:50.075600+0000","last_became_peered":"2024-04-25T08:32:50.075600+0000","last_unstale":"2024-04-25T08:32:51.072238+0000","last_undegraded":"2024-04-25T08:32:51.072238+0000","last_fullsized":"2024-04-25T08:32:51.072238+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:49.022905+0000","last_clean_scrub_stamp":"2024-04-25T08:32:49.022905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.438733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056373+0000","last_change":"2024-04-25T08:32:49.043943+0000","last_active":"2024-04-25T08:32:51.056373+0000","last_peered":"2024-04-25T08:32:51.056373+0000","last_clean":"2024-04-25T08:32:51.056373+0000","last_became_active":"2024-04-25T08:32:49.043659+0000","last_became_peered":"2024-04-25T08:32:49.043659+0000","last_unstale":"2024-04-25T08:32:51.056373+0000","last_undegraded":"2024-04-25T08:32:51.056373+0000","last_fullsized":"2024-04-25T08:32:51.056373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:32:48.016751+0000","last_clean_scrub_stamp":"2024-04-25T08:32:48.016751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:44.586423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":203,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T08:32:51.056108+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:32:51.056108+0000","last_peered":"2024-04-25T08:32:51.056108+0000","last_clean":"2024-04-25T08:32:51.056108+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:32:51.056108+0000","last_undegraded":"2024-04-25T08:32:51.056108+0000","last_fullsized":"2024-04-25T08:32:51.056108+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":782580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":191112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":190642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":191134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":192176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":190600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":190620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":780932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:32:54.400 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:32:54.400 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:32:54.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.550+0000 7f0c22f3b640 1 Processor -- start 2024-04-25T08:32:54.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.550+0000 7f0c22f3b640 1 -- start start 2024-04-25T08:32:54.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.551+0000 7f0c22f3b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0c1c1451a0 0x7f0c1c0cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.551+0000 7f0c22f3b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0c1c15f570 0x7f0c1c15f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.551+0000 7f0c22f3b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1c0cfa00 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.551+0000 7f0c22f3b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1c15c200 con 0x7f0c1c15f570 2024-04-25T08:32:54.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.551+0000 7f0c22f3b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1c0cfce0 con 0x7f0c1c1451a0 2024-04-25T08:32:54.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.551+0000 7f0c214d8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0c1c15f570 0x7f0c1c15f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.551+0000 7f0c214d8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0c1c15f570 0x7f0c1c15f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57886/0 (socket says 172.21.15.38:57886) 2024-04-25T08:32:54.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.551+0000 7f0c214d8640 1 -- 172.21.15.38:0/3452683565 learned_addr learned my addr 172.21.15.38:0/3452683565 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:54.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.551+0000 7f0c20cd7640 1 --2- 172.21.15.38:0/3452683565 >> v2:172.21.15.107:3300/0 conn(0x7f0c1c1451a0 0x7f0c1c0cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.551+0000 7f0c214d8640 1 -- 172.21.15.38:0/3452683565 >> v2:172.21.15.107:3300/0 conn(0x7f0c1c1451a0 msgr2=0x7f0c1c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:54.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.552+0000 7f0c214d8640 1 --2- 172.21.15.38:0/3452683565 >> v2:172.21.15.107:3300/0 conn(0x7f0c1c1451a0 0x7f0c1c0cf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.552+0000 7f0c214d8640 1 -- 172.21.15.38:0/3452683565 >> v1:172.21.15.107:6789/0 conn(0x7f0c1c06e8f0 legacy=0x7f0c1c15c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:54.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.552+0000 7f0c214d8640 1 -- 172.21.15.38:0/3452683565 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c0cffc0 con 0x7f0c1c15f570 2024-04-25T08:32:54.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.553+0000 7f0c214d8640 1 --2- 172.21.15.38:0/3452683565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c1c15f570 0x7f0c1c15f9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0cd260 tx=0x7f0c0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad87941a8a040ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.553+0000 7f0c1b7fe640 1 -- 172.21.15.38:0/3452683565 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04037070 con 0x7f0c1c15f570 2024-04-25T08:32:54.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.553+0000 7f0c1b7fe640 1 -- 172.21.15.38:0/3452683565 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04032440 con 0x7f0c1c15f570 2024-04-25T08:32:54.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.553+0000 7f0c1b7fe640 1 -- 172.21.15.38:0/3452683565 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04032810 con 0x7f0c1c15f570 2024-04-25T08:32:54.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.553+0000 7f0c22f3b640 1 -- 172.21.15.38:0/3452683565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c1c15f570 msgr2=0x7f0c1c15f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.553+0000 7f0c22f3b640 1 --2- 172.21.15.38:0/3452683565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c1c15f570 0x7f0c1c15f9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0cd260 tx=0x7f0c0402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.554+0000 7f0c22f3b640 1 -- 172.21.15.38:0/3452683565 shutdown_connections 2024-04-25T08:32:54.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.554+0000 7f0c22f3b640 1 --2- 172.21.15.38:0/3452683565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c1c15f570 0x7f0c1c15f9c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.554+0000 7f0c22f3b640 1 --2- 172.21.15.38:0/3452683565 >> v2:172.21.15.107:3300/0 conn(0x7f0c1c1451a0 0x7f0c1c0cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.554+0000 7f0c22f3b640 1 -- 172.21.15.38:0/3452683565 >> 172.21.15.38:0/3452683565 conn(0x7f0c1c0b5080 msgr2=0x7f0c1c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:54.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.554+0000 7f0c22f3b640 1 -- 172.21.15.38:0/3452683565 shutdown_connections 2024-04-25T08:32:54.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.554+0000 7f0c22f3b640 1 -- 172.21.15.38:0/3452683565 wait complete. 2024-04-25T08:32:54.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.555+0000 7f0c22f3b640 1 Processor -- start 2024-04-25T08:32:54.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.555+0000 7f0c22f3b640 1 -- start start 2024-04-25T08:32:54.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c22f3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c1c06e8f0 0x7f0c1c137c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c22f3b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c1c1451a0 0x7f0c1c1f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c22f3b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c1c15f570 0x7f0c1c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c22f3b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c15cda0 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c22f3b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c06ed80 con 0x7f0c1c15f570 2024-04-25T08:32:54.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c22f3b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1c071220 con 0x7f0c1c1451a0 2024-04-25T08:32:54.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c20cd7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c1c06e8f0 0x7f0c1c137c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c20cd7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c1c06e8f0 0x7f0c1c137c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57892/0 (socket says 172.21.15.38:57892) 2024-04-25T08:32:54.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c20cd7640 1 -- 172.21.15.38:0/2797108159 learned_addr learned my addr 172.21.15.38:0/2797108159 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:54.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c214d8640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c1c15f570 0x7f0c1c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c1bfff640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c1c1451a0 0x7f0c1c1f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c20cd7640 1 -- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c1c1451a0 msgr2=0x7f0c1c1f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c20cd7640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c1c1451a0 0x7f0c1c1f39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c20cd7640 1 -- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c1c15f570 msgr2=0x7f0c1c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c20cd7640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c1c15f570 0x7f0c1c0b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c20cd7640 1 -- 172.21.15.38:0/2797108159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c0b2ac0 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c1bfff640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c1c1451a0 0x7f0c1c1f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:54.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c20cd7640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c1c06e8f0 0x7f0c1c137c10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0c14007ef0 tx=0x7f0c1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.556+0000 7f0c197fa640 1 -- 172.21.15.38:0/2797108159 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14014620 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.557+0000 7f0c197fa640 1 -- 172.21.15.38:0/2797108159 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c14019070 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.557+0000 7f0c22f3b640 1 -- 172.21.15.38:0/2797108159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1c0b2da0 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.557+0000 7f0c197fa640 1 -- 172.21.15.38:0/2797108159 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c14014a00 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.557+0000 7f0c22f3b640 1 -- 172.21.15.38:0/2797108159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1c0b3640 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.558+0000 7f0c22f3b640 1 -- 172.21.15.38:0/2797108159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1c0b3190 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.558+0000 7f0c197fa640 1 -- 172.21.15.38:0/2797108159 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c14014ba0 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.559+0000 7f0c197fa640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0bfc06b4d0 0x7f0bfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.559+0000 7f0c197fa640 1 -- 172.21.15.38:0/2797108159 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c1408abb0 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.559+0000 7f0c1bfff640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0bfc06b4d0 0x7f0bfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.559+0000 7f0c1bfff640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0bfc06b4d0 0x7f0bfc06d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0ad910 tx=0x7f0c0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.563+0000 7f0c197fa640 1 -- 172.21.15.38:0/2797108159 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c14059930 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.813+0000 7f0c22f3b640 1 -- 172.21.15.38:0/2797108159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0c1c068e80 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.815+0000 7f0c197fa640 1 -- 172.21.15.38:0/2797108159 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0c14059230 con 0x7f0c1c06e8f0 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw== 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:32:54.811 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-04-25T08:32:54.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.817+0000 7f0c22f3b640 1 -- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0bfc06b4d0 msgr2=0x7f0bfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.817+0000 7f0c22f3b640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0bfc06b4d0 0x7f0bfc06d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0ad910 tx=0x7f0c0c009040 comp rx=0 tx=0).stop 2024-04-25T08:32:54.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.817+0000 7f0c22f3b640 1 -- 172.21.15.38:0/2797108159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c1c06e8f0 msgr2=0x7f0c1c137c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.817+0000 7f0c22f3b640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c1c06e8f0 0x7f0c1c137c10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0c14007ef0 tx=0x7f0c1400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.817+0000 7f0c22f3b640 1 -- 172.21.15.38:0/2797108159 shutdown_connections 2024-04-25T08:32:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.817+0000 7f0c22f3b640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0c1c15f570 0x7f0c1c0b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.817+0000 7f0c22f3b640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0bfc06b4d0 0x7f0bfc06d990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.817+0000 7f0c22f3b640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0c1c1451a0 0x7f0c1c1f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.817+0000 7f0c22f3b640 1 --2- 172.21.15.38:0/2797108159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0c1c06e8f0 0x7f0c1c137c10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.817+0000 7f0c22f3b640 1 -- 172.21.15.38:0/2797108159 >> 172.21.15.38:0/2797108159 conn(0x7f0c1c0b5080 msgr2=0x7f0c1c159ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.817+0000 7f0c22f3b640 1 -- 172.21.15.38:0/2797108159 shutdown_connections 2024-04-25T08:32:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.817+0000 7f0c22f3b640 1 -- 172.21.15.38:0/2797108159 wait complete. 2024-04-25T08:32:54.824 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:32:54.853 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-04-25T08:32:54.857+0000 7f4a16662640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:32:54.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.929+0000 7ff53745d640 1 Processor -- start 2024-04-25T08:32:54.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.930+0000 7ff53745d640 1 -- start start 2024-04-25T08:32:54.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.930+0000 7ff53745d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff53015da90 0x7ff5300d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.930+0000 7ff53745d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff5300d3fb0 0x7ff5300b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.930+0000 7ff53745d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5300d1b60 con 0x7ff5300ceca0 2024-04-25T08:32:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.930+0000 7ff53745d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5300d2bc0 con 0x7ff53015da90 2024-04-25T08:32:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.930+0000 7ff53745d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5300d1e00 con 0x7ff5300d3fb0 2024-04-25T08:32:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.930+0000 7ff5351f9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff53015da90 0x7ff5300d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.930+0000 7ff5351f9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff53015da90 0x7ff5300d2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57906/0 (socket says 172.21.15.38:57906) 2024-04-25T08:32:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.930+0000 7ff5351f9640 1 -- 172.21.15.38:0/2633132616 learned_addr learned my addr 172.21.15.38:0/2633132616 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.931+0000 7ff5351f9640 1 -- 172.21.15.38:0/2633132616 >> v2:172.21.15.107:3300/0 conn(0x7ff5300d3fb0 msgr2=0x7ff5300b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.931+0000 7ff5351f9640 1 --2- 172.21.15.38:0/2633132616 >> v2:172.21.15.107:3300/0 conn(0x7ff5300d3fb0 0x7ff5300b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.931+0000 7ff5351f9640 1 -- 172.21.15.38:0/2633132616 >> v1:172.21.15.107:6789/0 conn(0x7ff5300ceca0 legacy=0x7ff5300d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.931+0000 7ff5351f9640 1 -- 172.21.15.38:0/2633132616 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5300d20e0 con 0x7ff53015da90 2024-04-25T08:32:54.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.931+0000 7ff5351f9640 1 --2- 172.21.15.38:0/2633132616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff53015da90 0x7ff5300d2f70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff5300c7410 tx=0x7ff52002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e99839686d2f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.932+0000 7ff527fff640 1 -- 172.21.15.38:0/2633132616 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520037070 con 0x7ff53015da90 2024-04-25T08:32:54.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.932+0000 7ff527fff640 1 -- 172.21.15.38:0/2633132616 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff520032440 con 0x7ff53015da90 2024-04-25T08:32:54.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.932+0000 7ff527fff640 1 -- 172.21.15.38:0/2633132616 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff520032810 con 0x7ff53015da90 2024-04-25T08:32:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.932+0000 7ff53745d640 1 -- 172.21.15.38:0/2633132616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff53015da90 msgr2=0x7ff5300d2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.932+0000 7ff53745d640 1 --2- 172.21.15.38:0/2633132616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff53015da90 0x7ff5300d2f70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff5300c7410 tx=0x7ff52002a230 comp rx=0 tx=0).stop 2024-04-25T08:32:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.932+0000 7ff53745d640 1 -- 172.21.15.38:0/2633132616 shutdown_connections 2024-04-25T08:32:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.932+0000 7ff53745d640 1 --2- 172.21.15.38:0/2633132616 >> v2:172.21.15.107:3300/0 conn(0x7ff5300d3fb0 0x7ff5300b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.932+0000 7ff53745d640 1 --2- 172.21.15.38:0/2633132616 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff53015da90 0x7ff5300d2f70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.932+0000 7ff53745d640 1 -- 172.21.15.38:0/2633132616 >> 172.21.15.38:0/2633132616 conn(0x7ff53006bed0 msgr2=0x7ff53006c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.932+0000 7ff53745d640 1 -- 172.21.15.38:0/2633132616 shutdown_connections 2024-04-25T08:32:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.932+0000 7ff53745d640 1 -- 172.21.15.38:0/2633132616 wait complete. 2024-04-25T08:32:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.933+0000 7ff53745d640 1 Processor -- start 2024-04-25T08:32:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.934+0000 7ff53745d640 1 -- start start 2024-04-25T08:32:54.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.934+0000 7ff53745d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5300d3fb0 0x7ff5300b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.934+0000 7ff53745d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff53015da90 0x7ff530159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.934+0000 7ff53745d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5301598d0 0x7ff5300cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.934+0000 7ff53745d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53015dff0 con 0x7ff53015da90 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.934+0000 7ff53745d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53015de70 con 0x7ff5300d3fb0 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.934+0000 7ff53745d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5300d34b0 con 0x7ff5301598d0 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.934+0000 7ff5349f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff53015da90 0x7ff530159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.934+0000 7ff5349f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff53015da90 0x7ff530159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57914/0 (socket says 172.21.15.38:57914) 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.934+0000 7ff5349f8640 1 -- 172.21.15.38:0/346386482 learned_addr learned my addr 172.21.15.38:0/346386482 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff5359fa640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5301598d0 0x7ff5300cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff5351f9640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5300d3fb0 0x7ff5300b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff5349f8640 1 -- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5301598d0 msgr2=0x7ff5300cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff5349f8640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5301598d0 0x7ff5300cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff5349f8640 1 -- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5300d3fb0 msgr2=0x7ff5300b7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff5349f8640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5300d3fb0 0x7ff5300b7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff5349f8640 1 -- 172.21.15.38:0/346386482 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff530157a80 con 0x7ff53015da90 2024-04-25T08:32:54.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff5349f8640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff53015da90 0x7ff530159390 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00c440 tx=0x7ff52c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff525ffb640 1 -- 172.21.15.38:0/346386482 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c017070 con 0x7ff53015da90 2024-04-25T08:32:54.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff525ffb640 1 -- 172.21.15.38:0/346386482 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c0035f0 con 0x7ff53015da90 2024-04-25T08:32:54.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff53745d640 1 -- 172.21.15.38:0/346386482 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff530157d60 con 0x7ff53015da90 2024-04-25T08:32:54.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff525ffb640 1 -- 172.21.15.38:0/346386482 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0039e0 con 0x7ff53015da90 2024-04-25T08:32:54.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.935+0000 7ff53745d640 1 -- 172.21.15.38:0/346386482 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff530158320 con 0x7ff53015da90 2024-04-25T08:32:54.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.936+0000 7ff53745d640 1 -- 172.21.15.38:0/346386482 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff530068e80 con 0x7ff53015da90 2024-04-25T08:32:54.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.937+0000 7ff525ffb640 1 -- 172.21.15.38:0/346386482 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff52c01a070 con 0x7ff53015da90 2024-04-25T08:32:54.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.937+0000 7ff525ffb640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff51406b4d0 0x7ff51406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:54.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.937+0000 7ff525ffb640 1 -- 172.21.15.38:0/346386482 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff52c089720 con 0x7ff53015da90 2024-04-25T08:32:54.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.938+0000 7ff5351f9640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff51406b4d0 0x7ff51406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:54.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.938+0000 7ff5351f9640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff51406b4d0 0x7ff51406d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff520034000 tx=0x7ff52002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:54.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:54.942+0000 7ff525ffb640 1 -- 172.21.15.38:0/346386482 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff52c058400 con 0x7ff53015da90 2024-04-25T08:32:55.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.161+0000 7ff53745d640 1 -- 172.21.15.38:0/346386482 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5300b3e40 con 0x7ff53015da90 2024-04-25T08:32:55.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.162+0000 7ff525ffb640 1 -- 172.21.15.38:0/346386482 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7ff52c0010c0 con 0x7ff53015da90 2024-04-25T08:32:55.158 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:32:55.158 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/422749749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":422749749},{"type":"v1","addr":"172.21.15.38:6834","nonce":422749749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5143,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/4194720752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":4194720752},{"type":"v1","addr":"172.21.15.107:6836","nonce":4194720752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3379884609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3379884609},{"type":"v1","addr":"172.21.15.107:6837","nonce":3379884609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:32:49.392279+0000","modified":"2024-04-25T08:32:52.412335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5149":{"gid":5149,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/234860294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":234860294},{"type":"v1","addr":"172.21.15.38:6835","nonce":234860294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:32:55.158 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 17 2024-04-25T08:32:55.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.164+0000 7ff53745d640 1 -- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff51406b4d0 msgr2=0x7ff51406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.164+0000 7ff53745d640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff51406b4d0 0x7ff51406d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff520034000 tx=0x7ff52002e000 comp rx=0 tx=0).stop 2024-04-25T08:32:55.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.164+0000 7ff53745d640 1 -- 172.21.15.38:0/346386482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff53015da90 msgr2=0x7ff530159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.164+0000 7ff53745d640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff53015da90 0x7ff530159390 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00c440 tx=0x7ff52c00c900 comp rx=0 tx=0).stop 2024-04-25T08:32:55.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.164+0000 7ff53745d640 1 -- 172.21.15.38:0/346386482 shutdown_connections 2024-04-25T08:32:55.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.164+0000 7ff53745d640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff51406b4d0 0x7ff51406d990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.164+0000 7ff53745d640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5301598d0 0x7ff5300cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.164+0000 7ff53745d640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff53015da90 0x7ff530159390 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.164+0000 7ff53745d640 1 --2- 172.21.15.38:0/346386482 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5300d3fb0 0x7ff5300b7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.164+0000 7ff53745d640 1 -- 172.21.15.38:0/346386482 >> 172.21.15.38:0/346386482 conn(0x7ff53006bed0 msgr2=0x7ff5300b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:55.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.164+0000 7ff53745d640 1 -- 172.21.15.38:0/346386482 shutdown_connections 2024-04-25T08:32:55.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.164+0000 7ff53745d640 1 -- 172.21.15.38:0/346386482 wait complete. 2024-04-25T08:32:55.573 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:32:49.392279+0000', 'modified': '2024-04-25T08:32:52.412335+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5149}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5149': {'gid': 5149, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6835/234860294', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 234860294}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 234860294}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:32:55.574 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:32:55.574 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:32:55.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.682+0000 7f00b5f61640 1 Processor -- start 2024-04-25T08:32:55.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.682+0000 7f00b5f61640 1 -- start start 2024-04-25T08:32:55.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.683+0000 7f00b5f61640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f00b015eb20 0x7f00b015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.683+0000 7f00b5f61640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b015f490 con 0x7f00b0099ce0 2024-04-25T08:32:55.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.683+0000 7f00b5f61640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b00cd120 con 0x7f00b009a0c0 2024-04-25T08:32:55.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.683+0000 7f00b5f61640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b00cf6c0 con 0x7f00b015eb20 2024-04-25T08:32:55.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.683+0000 7f00af7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f00b0099ce0 0x7f00b009a7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47648/0 (socket says 172.21.15.38:47648) 2024-04-25T08:32:55.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.683+0000 7f00af7fe640 1 -- 172.21.15.38:0/1098364318 learned_addr learned my addr 172.21.15.38:0/1098364318 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:55.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.683+0000 7f00affff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f00b015eb20 0x7f00b015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.684+0000 7f00affff640 1 -- 172.21.15.38:0/1098364318 >> v1:172.21.15.107:6789/0 conn(0x7f00b009a0c0 legacy=0x7f00b00b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.684+0000 7f00affff640 1 -- 172.21.15.38:0/1098364318 >> v1:172.21.15.38:6789/0 conn(0x7f00b0099ce0 legacy=0x7f00b009a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.684+0000 7f00affff640 1 -- 172.21.15.38:0/1098364318 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00cd400 con 0x7f00b015eb20 2024-04-25T08:32:55.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.684+0000 7f00affff640 1 --2- 172.21.15.38:0/1098364318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b015eb20 0x7f00b015ef50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009d90 tx=0x7f00a802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b4cefbfa3032a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.685+0000 7f00ae7fc640 1 -- 172.21.15.38:0/1098364318 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8007740 con 0x7f00b015eb20 2024-04-25T08:32:55.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.685+0000 7f00ae7fc640 1 -- 172.21.15.38:0/1098364318 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a80078e0 con 0x7f00b015eb20 2024-04-25T08:32:55.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.685+0000 7f00ae7fc640 1 -- 172.21.15.38:0/1098364318 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8007cb0 con 0x7f00b015eb20 2024-04-25T08:32:55.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.685+0000 7f00b5f61640 1 -- 172.21.15.38:0/1098364318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b015eb20 msgr2=0x7f00b015ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.685+0000 7f00b5f61640 1 --2- 172.21.15.38:0/1098364318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b015eb20 0x7f00b015ef50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009d90 tx=0x7f00a802a6f0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.685+0000 7f00b5f61640 1 -- 172.21.15.38:0/1098364318 shutdown_connections 2024-04-25T08:32:55.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.685+0000 7f00b5f61640 1 --2- 172.21.15.38:0/1098364318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b015eb20 0x7f00b015ef50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.685+0000 7f00b5f61640 1 -- 172.21.15.38:0/1098364318 >> 172.21.15.38:0/1098364318 conn(0x7f00b00b30f0 msgr2=0x7f00b00b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:55.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.685+0000 7f00b5f61640 1 -- 172.21.15.38:0/1098364318 shutdown_connections 2024-04-25T08:32:55.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.685+0000 7f00b5f61640 1 -- 172.21.15.38:0/1098364318 wait complete. 2024-04-25T08:32:55.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.686+0000 7f00b5f61640 1 Processor -- start 2024-04-25T08:32:55.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.686+0000 7f00b5f61640 1 -- start start 2024-04-25T08:32:55.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.687+0000 7f00b5f61640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00b0099ce0 0x7f00b0153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.687+0000 7f00b5f61640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b009a0c0 0x7f00b00b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.687+0000 7f00b5f61640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b015eb20 0x7f00b00b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.687+0000 7f00b5f61640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b00cce30 con 0x7f00b009a0c0 2024-04-25T08:32:55.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.687+0000 7f00b5f61640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b00cccb0 con 0x7f00b015eb20 2024-04-25T08:32:55.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.687+0000 7f00b5f61640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b00cfb50 con 0x7f00b0099ce0 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.687+0000 7f00aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b009a0c0 0x7f00b00b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.687+0000 7f00aeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b009a0c0 0x7f00b00b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57936/0 (socket says 172.21.15.38:57936) 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.687+0000 7f00aeffd640 1 -- 172.21.15.38:0/1790710845 learned_addr learned my addr 172.21.15.38:0/1790710845 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f00affff640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b015eb20 0x7f00b00b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f00af7fe640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00b0099ce0 0x7f00b0153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f00aeffd640 1 -- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00b0099ce0 msgr2=0x7f00b0153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f00aeffd640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00b0099ce0 0x7f00b0153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f00aeffd640 1 -- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b015eb20 msgr2=0x7f00b00b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f00aeffd640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b015eb20 0x7f00b00b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f00aeffd640 1 -- 172.21.15.38:0/1790710845 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b00b4590 con 0x7f00b009a0c0 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f00affff640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b015eb20 0x7f00b00b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f00aeffd640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b009a0c0 0x7f00b00b4050 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f00a400c430 tx=0x7f00a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f0097fff640 1 -- 172.21.15.38:0/1790710845 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a40052d0 con 0x7f00b009a0c0 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f0097fff640 1 -- 172.21.15.38:0/1790710845 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a4005490 con 0x7f00b009a0c0 2024-04-25T08:32:55.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f00b5f61640 1 -- 172.21.15.38:0/1790710845 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b0157890 con 0x7f00b009a0c0 2024-04-25T08:32:55.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f0097fff640 1 -- 172.21.15.38:0/1790710845 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4007260 con 0x7f00b009a0c0 2024-04-25T08:32:55.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.688+0000 7f00b5f61640 1 -- 172.21.15.38:0/1790710845 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b0156db0 con 0x7f00b009a0c0 2024-04-25T08:32:55.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.690+0000 7f0097fff640 1 -- 172.21.15.38:0/1790710845 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f00a4007400 con 0x7f00b009a0c0 2024-04-25T08:32:55.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.690+0000 7f0097fff640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f009c06b4d0 0x7f009c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.690+0000 7f0097fff640 1 -- 172.21.15.38:0/1790710845 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f00a40897f0 con 0x7f00b009a0c0 2024-04-25T08:32:55.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.690+0000 7f00af7fe640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f009c06b4d0 0x7f009c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.691+0000 7f00af7fe640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f009c06b4d0 0x7f009c06d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f00a0005880 tx=0x7f00a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.691+0000 7f00b8c5d740 1 Processor -- start 2024-04-25T08:32:55.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.691+0000 7f00b8c5d740 1 -- start start 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00b8c5d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 0x55810a90b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00b8c5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a928ad0 0x55810a929020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00b8c5d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a8e0f10 0x55810a927190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00b8c5d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810a843a10 con 0x55810a90b230 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00b8c5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810a7ecb30 con 0x55810a928ad0 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00b8c5d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55810a83fc50 con 0x55810a8e0f10 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 0x55810a90b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 0x55810a90b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57940/0 (socket says 172.21.15.38:57940) 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00af7fe640 1 -- 172.21.15.38:0/992092704 learned_addr learned my addr 172.21.15.38:0/992092704 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00aeffd640 1 --2- 172.21.15.38:0/992092704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a928ad0 0x55810a929020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00affff640 1 --2- 172.21.15.38:0/992092704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a8e0f10 0x55810a927190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00af7fe640 1 -- 172.21.15.38:0/992092704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a8e0f10 msgr2=0x55810a927190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00af7fe640 1 --2- 172.21.15.38:0/992092704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a8e0f10 0x55810a927190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00af7fe640 1 -- 172.21.15.38:0/992092704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a928ad0 msgr2=0x55810a929020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00af7fe640 1 --2- 172.21.15.38:0/992092704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a928ad0 0x55810a929020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00af7fe640 1 -- 172.21.15.38:0/992092704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55810a801e10 con 0x55810a90b230 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.692+0000 7f00affff640 1 --2- 172.21.15.38:0/992092704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a8e0f10 0x55810a927190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.693+0000 7f00af7fe640 1 --2- 172.21.15.38:0/992092704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 0x55810a90b610 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f00a0005b60 tx=0x7f00a0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff561280da8a14d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.693+0000 7f00957fa640 1 -- 172.21.15.38:0/992092704 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000d9a0 con 0x55810a90b230 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.693+0000 7f00957fa640 1 -- 172.21.15.38:0/992092704 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a000db40 con 0x55810a90b230 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.693+0000 7f00957fa640 1 -- 172.21.15.38:0/992092704 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0020450 con 0x55810a90b230 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.693+0000 7f00b8c5d740 1 -- 172.21.15.38:0/992092704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 msgr2=0x55810a90b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.693+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/992092704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 0x55810a90b610 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f00a0005b60 tx=0x7f00a0010000 comp rx=0 tx=0).stop 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.693+0000 7f00b8c5d740 1 -- 172.21.15.38:0/992092704 shutdown_connections 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.693+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/992092704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a8e0f10 0x55810a927190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.693+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/992092704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a928ad0 0x55810a929020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.693+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/992092704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 0x55810a90b610 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.693+0000 7f00b8c5d740 1 -- 172.21.15.38:0/992092704 >> 172.21.15.38:0/992092704 conn(0x55810a8dd840 msgr2=0x55810a926040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.694+0000 7f00b8c5d740 1 -- 172.21.15.38:0/992092704 shutdown_connections 2024-04-25T08:32:55.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.694+0000 7f00b8c5d740 1 -- 172.21.15.38:0/992092704 wait complete. 2024-04-25T08:32:55.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.694+0000 7f00b8c5d740 1 -- start start 2024-04-25T08:32:55.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.694+0000 7f00b8c5d740 1 Processor -- start 2024-04-25T08:32:55.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00b8c5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90b230 0x55810a936fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00b8c5d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a928ad0 0x55810a8f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00b8c5d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90e850 0x55810a9274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00b8c5d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810a8e1cb0 con 0x55810a90e850 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00b8c5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810a8000c0 con 0x55810a90b230 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00b8c5d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55810a8db770 con 0x55810a928ad0 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00b8c5d740 10 client.0 _ll_register_callbacks cb 0x55810a7f3ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90e850 0x55810a9274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90e850 0x55810a9274c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57946/0 (socket says 172.21.15.38:57946) 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00affff640 1 -- 172.21.15.38:0/517274790 learned_addr learned my addr 172.21.15.38:0/517274790 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00aeffd640 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a928ad0 0x55810a8f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00af7fe640 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90b230 0x55810a936fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00affff640 1 -- 172.21.15.38:0/517274790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a928ad0 msgr2=0x55810a8f5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00affff640 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a928ad0 0x55810a8f5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00affff640 1 -- 172.21.15.38:0/517274790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90b230 msgr2=0x55810a936fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00affff640 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90b230 0x55810a936fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.695+0000 7f00affff640 1 -- 172.21.15.38:0/517274790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55810a5582d0 con 0x55810a90e850 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f00affff640 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90e850 0x55810a9274c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f00a802a730 tx=0x7f00a8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd88d4176e18b89c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f008e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f008e7fc640 1 -- 172.21.15.38:0/517274790 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8046430 con 0x55810a90e850 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f008e7fc640 1 -- 172.21.15.38:0/517274790 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f00a803d430 con 0x55810a90e850 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f00b8c5d740 1 -- 172.21.15.38:0/517274790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00a000db40 con 0x55810a90e850 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f008e7fc640 1 -- 172.21.15.38:0/517274790 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00a803d630 con 0x55810a90e850 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f00b8c5d740 10 client.5646 fetch_fsmap learned FSMap version 17 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f00b8c5d740 1 -- 172.21.15.38:0/517274790 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55810a6226b0 con 0x55810a90e850 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f008e7fc640 1 -- 172.21.15.38:0/517274790 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a000db40 con 0x55810a90e850 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f008e7fc640 1 -- 172.21.15.38:0/517274790 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f00a804e050 con 0x55810a90e850 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f00b8c5d740 10 client.5646 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f00b8c5d740 10 client.5646 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f00b8c5d740 20 client.5646 populate_metadata read hostname 'smithi038' 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55810a90c540 0x55810a91dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f00b8c5d740 4 client.5646 mds_command: new command op to 5149 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f00b8c5d740 1 -- 172.21.15.38:0/517274790 --> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55810a8a2920 con 0x55810a90c540 2024-04-25T08:32:55.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.696+0000 7f00af7fe640 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55810a90c540 0x55810a91dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f008e7fc640 10 client.5646 ms_handle_connect on v2:172.21.15.38:6832/234860294 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f00af7fe640 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55810a90c540 0x55810a91dff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77ffd3ea9966049e server_cookie=7d91ac70d75b74b6 in_seq=0 out_seq=0 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f008e7fc640 1 -- 172.21.15.38:0/517274790 <== mds.0 v2:172.21.15.38:6832/234860294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55810a8a2920 con 0x55810a90c540 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f008e7fc640 10 client.5646 handle_command_reply: tid=0 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f00b8c5d740 1 client.5646 shutdown 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f00b8c5d740 1 -- 172.21.15.38:0/517274790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90e850 msgr2=0x55810a9274c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90e850 0x55810a9274c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f00a802a730 tx=0x7f00a8004ab0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f00b8c5d740 1 -- 172.21.15.38:0/517274790 shutdown_connections 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55810a90c540 0x55810a91dff0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90e850 0x55810a9274c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a928ad0 0x55810a8f5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f008e7fc640 0 client.5646 ms_handle_reset on v2:172.21.15.38:6832/234860294 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/517274790 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90b230 0x55810a936fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f00b8c5d740 1 -- 172.21.15.38:0/517274790 >> 172.21.15.38:0/517274790 conn(0x55810a8e0f10 msgr2=0x55810a9295c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:55.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.697+0000 7f00b8c5d740 1 -- 172.21.15.38:0/517274790 shutdown_connections 2024-04-25T08:32:55.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.698+0000 7f00b8c5d740 1 -- 172.21.15.38:0/517274790 wait complete. 2024-04-25T08:32:55.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.698+0000 7f00b8c5d740 20 client.5646 trim_cache size 0 max 16384 2024-04-25T08:32:55.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.713+0000 7f00b8c5d740 1 Processor -- start 2024-04-25T08:32:55.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.713+0000 7f00b8c5d740 1 -- start start 2024-04-25T08:32:55.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.714+0000 7f00b8c5d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 0x55810a90d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.714+0000 7f00b8c5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90e850 0x55810a8e12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.714+0000 7f00b8c5d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a928ad0 0x55810a8e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.714+0000 7f00b8c5d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b01fd340 con 0x55810a90b230 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.714+0000 7f00b8c5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b0159910 con 0x55810a90e850 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.714+0000 7f00b8c5d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b0159ee0 con 0x55810a928ad0 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.714+0000 7f00af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 0x55810a90d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.714+0000 7f00af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 0x55810a90d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57958/0 (socket says 172.21.15.38:57958) 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.714+0000 7f00af7fe640 1 -- 172.21.15.38:0/1681143919 learned_addr learned my addr 172.21.15.38:0/1681143919 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.715+0000 7f00af7fe640 1 -- 172.21.15.38:0/1681143919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a928ad0 msgr2=0x55810a8e1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.715+0000 7f00af7fe640 1 --2- 172.21.15.38:0/1681143919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a928ad0 0x55810a8e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.715+0000 7f00af7fe640 1 -- 172.21.15.38:0/1681143919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90e850 msgr2=0x55810a8e12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.715+0000 7f00af7fe640 1 --2- 172.21.15.38:0/1681143919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90e850 0x55810a8e12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.715+0000 7f00af7fe640 1 -- 172.21.15.38:0/1681143919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55810a91cf20 con 0x55810a90b230 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.715+0000 7f00af7fe640 1 --2- 172.21.15.38:0/1681143919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 0x55810a90d510 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f00a0001eb0 tx=0x7f00a0007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bf3594afb2da71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.715+0000 7f0094ff9640 1 -- 172.21.15.38:0/1681143919 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a0033070 con 0x55810a90b230 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.715+0000 7f0094ff9640 1 -- 172.21.15.38:0/1681143919 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a000d9a0 con 0x55810a90b230 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.715+0000 7f0094ff9640 1 -- 172.21.15.38:0/1681143919 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000db50 con 0x55810a90b230 2024-04-25T08:32:55.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.715+0000 7f00b8c5d740 1 -- 172.21.15.38:0/1681143919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 msgr2=0x55810a90d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.715+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/1681143919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 0x55810a90d510 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f00a0001eb0 tx=0x7f00a0007000 comp rx=0 tx=0).stop 2024-04-25T08:32:55.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.716+0000 7f00b8c5d740 1 -- 172.21.15.38:0/1681143919 shutdown_connections 2024-04-25T08:32:55.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.716+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/1681143919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a928ad0 0x55810a8e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.716+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/1681143919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90e850 0x55810a8e12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.716+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/1681143919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a90b230 0x55810a90d510 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.716+0000 7f00b8c5d740 1 -- 172.21.15.38:0/1681143919 >> 172.21.15.38:0/1681143919 conn(0x55810a8e0f10 msgr2=0x55810a9262f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:55.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.716+0000 7f00b8c5d740 1 -- 172.21.15.38:0/1681143919 shutdown_connections 2024-04-25T08:32:55.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.716+0000 7f00b8c5d740 1 -- 172.21.15.38:0/1681143919 wait complete. 2024-04-25T08:32:55.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.716+0000 7f00b8c5d740 1 -- start start 2024-04-25T08:32:55.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.716+0000 7f00b8c5d740 1 Processor -- start 2024-04-25T08:32:55.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00b8c5d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90b230 0x55810a990920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00b8c5d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a90e850 0x55810a8dfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00b8c5d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a928ad0 0x55810a8e03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00b8c5d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810a90c3c0 con 0x55810a928ad0 2024-04-25T08:32:55.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00b8c5d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55810a91d4d0 con 0x55810a90b230 2024-04-25T08:32:55.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00b8c5d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55810a90c9c0 con 0x55810a90e850 2024-04-25T08:32:55.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00b8c5d740 10 client.0 _ll_register_callbacks cb 0x55810a8dbc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:32:55.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a928ad0 0x55810a8e03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a928ad0 0x55810a8e03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57968/0 (socket says 172.21.15.38:57968) 2024-04-25T08:32:55.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00affff640 1 -- 172.21.15.38:0/2904177664 learned_addr learned my addr 172.21.15.38:0/2904177664 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:55.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00aeffd640 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a90e850 0x55810a8dfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00af7fe640 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90b230 0x55810a990920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00affff640 1 -- 172.21.15.38:0/2904177664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a90e850 msgr2=0x55810a8dfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:55.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00affff640 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a90e850 0x55810a8dfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00affff640 1 -- 172.21.15.38:0/2904177664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90b230 msgr2=0x55810a990920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00affff640 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90b230 0x55810a990920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.717+0000 7f00affff640 1 -- 172.21.15.38:0/2904177664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55810a8f2fe0 con 0x55810a928ad0 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f00affff640 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a928ad0 0x55810a8e03b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f00a802ace0 tx=0x7f00a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b31eb61e7e18de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f008e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f008e7fc640 1 -- 172.21.15.38:0/2904177664 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8054070 con 0x55810a928ad0 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f008e7fc640 1 -- 172.21.15.38:0/2904177664 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f00a803dc70 con 0x55810a928ad0 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f00b8c5d740 1 -- 172.21.15.38:0/2904177664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00a000d9a0 con 0x55810a928ad0 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f008e7fc640 1 -- 172.21.15.38:0/2904177664 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00a8009a30 con 0x55810a928ad0 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f00b8c5d740 10 client.5652 fetch_fsmap learned FSMap version 17 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f00b8c5d740 1 -- 172.21.15.38:0/2904177664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55810a91c480 con 0x55810a928ad0 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f008e7fc640 1 -- 172.21.15.38:0/2904177664 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a000d9a0 con 0x55810a928ad0 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f008e7fc640 1 -- 172.21.15.38:0/2904177664 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f00a804e040 con 0x55810a928ad0 2024-04-25T08:32:55.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f00b8c5d740 10 client.5652 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f00b8c5d740 10 client.5652 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f00b8c5d740 20 client.5652 populate_metadata read hostname 'smithi038' 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55810a8df4e0 0x55810a8da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f00b8c5d740 4 client.5652 mds_command: new command op to 5149 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f00b8c5d740 1 -- 172.21.15.38:0/2904177664 --> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55810a91ff50 con 0x55810a8df4e0 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.718+0000 7f00af7fe640 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55810a8df4e0 0x55810a8da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f008e7fc640 10 client.5652 ms_handle_connect on v2:172.21.15.38:6832/234860294 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00af7fe640 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55810a8df4e0 0x55810a8da4a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d4fe31b5a6d5ab server_cookie=c5cab90d694ec1c7 in_seq=0 out_seq=0 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f008e7fc640 1 -- 172.21.15.38:0/2904177664 <== mds.0 v2:172.21.15.38:6832/234860294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55810a91ff50 con 0x55810a8df4e0 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f008e7fc640 10 client.5652 handle_command_reply: tid=0 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00b8c5d740 1 client.5652 shutdown 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00b8c5d740 1 -- 172.21.15.38:0/2904177664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a928ad0 msgr2=0x55810a8e03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a928ad0 0x55810a8e03b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f00a802ace0 tx=0x7f00a8004a40 comp rx=0 tx=0).stop 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00b8c5d740 1 -- 172.21.15.38:0/2904177664 shutdown_connections 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55810a8df4e0 0x55810a8da4a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55810a928ad0 0x55810a8e03b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55810a90e850 0x55810a8dfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00b8c5d740 1 --2- 172.21.15.38:0/2904177664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55810a90b230 0x55810a990920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f008e7fc640 0 client.5652 ms_handle_reset on v2:172.21.15.38:6832/234860294 2024-04-25T08:32:55.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00b8c5d740 1 -- 172.21.15.38:0/2904177664 >> 172.21.15.38:0/2904177664 conn(0x55810a8e0f10 msgr2=0x55810a90d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:55.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00b8c5d740 1 -- 172.21.15.38:0/2904177664 shutdown_connections 2024-04-25T08:32:55.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00b8c5d740 1 -- 172.21.15.38:0/2904177664 wait complete. 2024-04-25T08:32:55.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.719+0000 7f00b8c5d740 20 client.5652 trim_cache size 0 max 16384 2024-04-25T08:32:55.721 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:32:55.721 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:32:55.721 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:32:55.721 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 5149, 2024-04-25T08:32:55.722 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:32:55.722 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:32:55.722 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:32:55.722 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 5.6585955820000002, 2024-04-25T08:32:55.722 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 17, 2024-04-25T08:32:55.722 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 38, 2024-04-25T08:32:55.722 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 38, 2024-04-25T08:32:55.722 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 12.743088006000001 2024-04-25T08:32:55.722 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:32:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.720+0000 7f00b5f61640 1 -- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f009c06b4d0 msgr2=0x7f009c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.720+0000 7f00b5f61640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f009c06b4d0 0x7f009c06d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f00a0005880 tx=0x7f00a0009040 comp rx=0 tx=0).stop 2024-04-25T08:32:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.720+0000 7f00b5f61640 1 -- 172.21.15.38:0/1790710845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b009a0c0 msgr2=0x7f00b00b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.720+0000 7f00b5f61640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b009a0c0 0x7f00b00b4050 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f00a400c430 tx=0x7f00a400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.720+0000 7f00b5f61640 1 -- 172.21.15.38:0/1790710845 shutdown_connections 2024-04-25T08:32:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.720+0000 7f00b5f61640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b015eb20 0x7f00b00b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.720+0000 7f00b5f61640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b009a0c0 0x7f00b00b4050 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.720+0000 7f00b5f61640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f009c06b4d0 0x7f009c06d990 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.720+0000 7f00b5f61640 1 --2- 172.21.15.38:0/1790710845 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f00b0099ce0 0x7f00b0153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.720+0000 7f00b5f61640 1 -- 172.21.15.38:0/1790710845 >> 172.21.15.38:0/1790710845 conn(0x7f00b00b30f0 msgr2=0x7f00b00cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:55.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.720+0000 7f00b5f61640 1 -- 172.21.15.38:0/1790710845 shutdown_connections 2024-04-25T08:32:55.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.720+0000 7f00b5f61640 1 -- 172.21.15.38:0/1790710845 wait complete. 2024-04-25T08:32:55.726 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:32:55.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.829+0000 7fb14c732640 1 Processor -- start 2024-04-25T08:32:55.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.830+0000 7fb14c732640 1 -- start start 2024-04-25T08:32:55.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.830+0000 7fb14c732640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb14415bb10 0x7fb14409a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.830+0000 7fb14c732640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb1440d0460 0x7fb14409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.830+0000 7fb14c732640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb144160340 con 0x7fb14415f4e0 2024-04-25T08:32:55.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.830+0000 7fb14c732640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1440cffd0 con 0x7fb1440d0460 2024-04-25T08:32:55.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.830+0000 7fb14c732640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1440cf400 con 0x7fb14415bb10 2024-04-25T08:32:55.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.831+0000 7fb149ccd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb1440d0460 0x7fb14409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.831+0000 7fb14accf640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb14415f4e0 0x7fb1440c5250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47660/0 (socket says 172.21.15.38:47660) 2024-04-25T08:32:55.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.831+0000 7fb14accf640 1 -- 172.21.15.38:0/716353928 learned_addr learned my addr 172.21.15.38:0/716353928 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:55.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.831+0000 7fb14a4ce640 1 --2- 172.21.15.38:0/716353928 >> v2:172.21.15.107:3300/0 conn(0x7fb14415bb10 0x7fb14409a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.831+0000 7fb149ccd640 1 -- 172.21.15.38:0/716353928 >> v2:172.21.15.107:3300/0 conn(0x7fb14415bb10 msgr2=0x7fb14409a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:55.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.831+0000 7fb149ccd640 1 --2- 172.21.15.38:0/716353928 >> v2:172.21.15.107:3300/0 conn(0x7fb14415bb10 0x7fb14409a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.831+0000 7fb149ccd640 1 -- 172.21.15.38:0/716353928 >> v1:172.21.15.38:6789/0 conn(0x7fb14415f4e0 legacy=0x7fb1440c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.831+0000 7fb149ccd640 1 -- 172.21.15.38:0/716353928 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1440cf6a0 con 0x7fb1440d0460 2024-04-25T08:32:55.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.831+0000 7fb1494cc640 1 -- 172.21.15.38:0/716353928 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205959389 0 0) 0x7fb13c001110 con 0x7fb14415f4e0 2024-04-25T08:32:55.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.831+0000 7fb14a4ce640 1 --2- 172.21.15.38:0/716353928 >> v2:172.21.15.107:3300/0 conn(0x7fb14415bb10 0x7fb14409a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:55.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.832+0000 7fb149ccd640 1 --2- 172.21.15.38:0/716353928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1440d0460 0x7fb14409a9c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb1440cf280 tx=0x7fb13802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b434635ce71bb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.832+0000 7fb1494cc640 1 -- 172.21.15.38:0/716353928 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb138037070 con 0x7fb1440d0460 2024-04-25T08:32:55.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.832+0000 7fb1494cc640 1 -- 172.21.15.38:0/716353928 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb138032440 con 0x7fb1440d0460 2024-04-25T08:32:55.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.832+0000 7fb1494cc640 1 -- 172.21.15.38:0/716353928 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13802ae60 con 0x7fb1440d0460 2024-04-25T08:32:55.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.832+0000 7fb14c732640 1 -- 172.21.15.38:0/716353928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1440d0460 msgr2=0x7fb14409a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.832+0000 7fb14c732640 1 --2- 172.21.15.38:0/716353928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1440d0460 0x7fb14409a9c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb1440cf280 tx=0x7fb13802a110 comp rx=0 tx=0).stop 2024-04-25T08:32:55.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.832+0000 7fb14c732640 1 -- 172.21.15.38:0/716353928 shutdown_connections 2024-04-25T08:32:55.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.832+0000 7fb14c732640 1 --2- 172.21.15.38:0/716353928 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1440d0460 0x7fb14409a9c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.832+0000 7fb14c732640 1 --2- 172.21.15.38:0/716353928 >> v2:172.21.15.107:3300/0 conn(0x7fb14415bb10 0x7fb14409a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.832+0000 7fb14c732640 1 -- 172.21.15.38:0/716353928 >> 172.21.15.38:0/716353928 conn(0x7fb1440b31f0 msgr2=0x7fb1440b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:55.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.832+0000 7fb14c732640 1 -- 172.21.15.38:0/716353928 shutdown_connections 2024-04-25T08:32:55.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.832+0000 7fb14c732640 1 -- 172.21.15.38:0/716353928 wait complete. 2024-04-25T08:32:55.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.834+0000 7fb14c732640 1 Processor -- start 2024-04-25T08:32:55.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.834+0000 7fb14c732640 1 -- start start 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.834+0000 7fb14c732640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1440d0460 0x7fb1441f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.834+0000 7fb14c732640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb14415bb10 0x7fb14414fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.834+0000 7fb14c732640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb14415f4e0 0x7fb14415e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.834+0000 7fb14c732640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14415f950 con 0x7fb1440d0460 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.834+0000 7fb14c732640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1440cfc60 con 0x7fb14415f4e0 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.834+0000 7fb14c732640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14415fad0 con 0x7fb14415bb10 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.834+0000 7fb14a4ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1440d0460 0x7fb1441f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.834+0000 7fb14a4ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1440d0460 0x7fb1441f7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57984/0 (socket says 172.21.15.38:57984) 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.834+0000 7fb14a4ce640 1 -- 172.21.15.38:0/3363130753 learned_addr learned my addr 172.21.15.38:0/3363130753 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.835+0000 7fb14a4ce640 1 -- 172.21.15.38:0/3363130753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb14415bb10 msgr2=0x7fb14414fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.835+0000 7fb14a4ce640 1 --2- 172.21.15.38:0/3363130753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb14415bb10 0x7fb14414fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.835+0000 7fb14a4ce640 1 -- 172.21.15.38:0/3363130753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb14415f4e0 msgr2=0x7fb14415e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.835+0000 7fb14a4ce640 1 --2- 172.21.15.38:0/3363130753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb14415f4e0 0x7fb14415e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:55.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.835+0000 7fb14a4ce640 1 -- 172.21.15.38:0/3363130753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1441f83c0 con 0x7fb1440d0460 2024-04-25T08:32:55.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.835+0000 7fb14a4ce640 1 --2- 172.21.15.38:0/3363130753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1440d0460 0x7fb1441f7e80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb134009fc0 tx=0x7fb13400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.835+0000 7fb142ffd640 1 -- 172.21.15.38:0/3363130753 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13401a070 con 0x7fb1440d0460 2024-04-25T08:32:55.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.835+0000 7fb142ffd640 1 -- 172.21.15.38:0/3363130753 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb134003bb0 con 0x7fb1440d0460 2024-04-25T08:32:55.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.835+0000 7fb14c732640 1 -- 172.21.15.38:0/3363130753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1441500c0 con 0x7fb1440d0460 2024-04-25T08:32:55.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.835+0000 7fb142ffd640 1 -- 172.21.15.38:0/3363130753 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb134004030 con 0x7fb1440d0460 2024-04-25T08:32:55.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.835+0000 7fb14c732640 1 -- 172.21.15.38:0/3363130753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14415da00 con 0x7fb1440d0460 2024-04-25T08:32:55.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.836+0000 7fb14c732640 1 -- 172.21.15.38:0/3363130753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb144068e80 con 0x7fb1440d0460 2024-04-25T08:32:55.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.836+0000 7fb142ffd640 1 -- 172.21.15.38:0/3363130753 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb13401d070 con 0x7fb1440d0460 2024-04-25T08:32:55.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.837+0000 7fb142ffd640 1 --2- 172.21.15.38:0/3363130753 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb13006b500 0x7fb13006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:55.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.837+0000 7fb142ffd640 1 -- 172.21.15.38:0/3363130753 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb13408a560 con 0x7fb1440d0460 2024-04-25T08:32:55.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.837+0000 7fb149ccd640 1 --2- 172.21.15.38:0/3363130753 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb13006b500 0x7fb13006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:55.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.838+0000 7fb149ccd640 1 --2- 172.21.15.38:0/3363130753 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb13006b500 0x7fb13006d9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb138034000 tx=0x7fb13802c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:55.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:55.841+0000 7fb142ffd640 1 -- 172.21.15.38:0/3363130753 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1441500c0 con 0x7fb1440d0460 2024-04-25T08:32:56.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.059+0000 7fb14c732640 1 -- 172.21.15.38:0/3363130753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb1440cdbf0 con 0x7fb1440d0460 2024-04-25T08:32:56.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.059+0000 7fb142ffd640 1 -- 172.21.15.38:0/3363130753 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb1441f83c0 con 0x7fb1440d0460 2024-04-25T08:32:56.057 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:32:56.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.061+0000 7fb14c732640 1 -- 172.21.15.38:0/3363130753 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb13006b500 msgr2=0x7fb13006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:56.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.061+0000 7fb14c732640 1 --2- 172.21.15.38:0/3363130753 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb13006b500 0x7fb13006d9c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb138034000 tx=0x7fb13802c000 comp rx=0 tx=0).stop 2024-04-25T08:32:56.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.061+0000 7fb14c732640 1 -- 172.21.15.38:0/3363130753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1440d0460 msgr2=0x7fb1441f7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:56.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.061+0000 7fb14c732640 1 --2- 172.21.15.38:0/3363130753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1440d0460 0x7fb1441f7e80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fb134009fc0 tx=0x7fb13400d560 comp rx=0 tx=0).stop 2024-04-25T08:32:56.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.061+0000 7fb14c732640 1 -- 172.21.15.38:0/3363130753 shutdown_connections 2024-04-25T08:32:56.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.061+0000 7fb14c732640 1 --2- 172.21.15.38:0/3363130753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb14415f4e0 0x7fb14415e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.061+0000 7fb14c732640 1 --2- 172.21.15.38:0/3363130753 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb13006b500 0x7fb13006d9c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.061+0000 7fb14c732640 1 --2- 172.21.15.38:0/3363130753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb14415bb10 0x7fb14414fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.061+0000 7fb14c732640 1 --2- 172.21.15.38:0/3363130753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1440d0460 0x7fb1441f7e80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:56.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.061+0000 7fb14c732640 1 -- 172.21.15.38:0/3363130753 >> 172.21.15.38:0/3363130753 conn(0x7fb1440b31f0 msgr2=0x7fb1440c5810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:56.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.062+0000 7fb14c732640 1 -- 172.21.15.38:0/3363130753 shutdown_connections 2024-04-25T08:32:56.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56.062+0000 7fb14c732640 1 -- 172.21.15.38:0/3363130753 wait complete. 2024-04-25T08:32:56.069 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:32:56.069 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:32:56.069 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:32:56.069 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:32:56.069 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:56.069 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:32:56.069 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:32:56.069 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:32:56.069 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:32:56.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:56.070 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:32:56.085 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:32:56.085 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:32:56.085 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:32:56.085 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:56.085 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:32:56.085 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42661sec preferred_lft 42661sec 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:32:56.086 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::440f:97ff:fe63:87b/64 scope link 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ea:23:76:ab:b1:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::e823:76ff:feab:b1e9/64 scope link 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::38c7:38ff:fe2e:6c92/64 scope link 2024-04-25T08:32:56.087 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:32:56.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:56.088 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:32:56.149 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:32:56.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:56.151 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:32:56.206 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:32:56.206 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:32:56.207 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:32:56.207 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:32:56.207 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:32:56.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:56.207 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:32:56.207 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:32:56.207 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:32:56.207 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:32:56.207 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:32:56.207 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:32:56.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:56.292 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:32:56.306 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:32:56.306 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:32:56.307 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:32:56.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:56.307 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T08:32:56.372 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:32:56.372 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:32:56.372 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:56.427 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:32:56.427 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:32:56.427 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:56.482 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:32:56.547 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:32:56.603 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:32:56.603 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:32:56.603 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:32:56.603 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:32:56.603 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:32:56.603 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:32:56.603 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:32:56.604 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:32:56.604 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:32:56.604 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:32:56.604 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:32:56.604 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:32:56.604 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:32:56.604 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:32:56.604 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:32:56.604 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:32:56.604 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:32:56.604 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:56.605 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:56.606 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:32:56.606 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:32:56.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:56.607 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:32:56.661 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:32:56.662 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:32:56.704 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:32:56.727 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:32:56.784 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:32:56.784 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:32:56.784 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:32:56.784 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:32:56.784 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:32:56.784 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:32:56.784 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:32:56.784 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:32:56.784 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:32:56.785 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:32:56.786 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:32:56.786 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:32:56.786 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:32:56.786 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:32:56.786 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:32:56.786 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:32:56.786 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:32:56.786 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:32:56.786 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:56.786 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:32:56.787 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:32:56.787 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:32:56.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:56.787 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:32:56.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-04-25T08:32:56.788+0000 7fe06c9b8480 -1 init, newargv = 0x55871773e5c0 newargc=15 2024-04-25T08:32:56.788 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[51161]: starting ceph client 2024-04-25T08:32:56.790 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[51161]: starting fuse 2024-04-25T08:32:56.803 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:32:56.803 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:32:56.803 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:32:56.803 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:32:56.803 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:32:56.803 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:32:56.804 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:32:56.805 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:32:56.805 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:32:56.805 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:32:56.805 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:32:56.805 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:32:56.805 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:32:56.805 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:56.805 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-04-25T08:32:56.805 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:32:56.922 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.51161.asok 2024-04-25T08:32:56.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:32:56.933 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51161.asok 2024-04-25T08:32:56.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:56.933 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51161.asok status 2024-04-25T08:32:57.070 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:32:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:32:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:32:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:32:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-04-25T08:32:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:32:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:32:57.070 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "51161", 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 5670, 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 5670 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 951657270 2024-04-25T08:32:57.071 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 951657270 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.5670 192.168.0.1:0/951657270", 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/951657270", 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 17, 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 38, 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:32:57.072 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:32:57.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:57.078 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:32:57.078 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:57.094 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:32:57.094 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:57.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:57.094 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:32:57.158 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T08:32:57.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.306+0000 7f3d98d5c640 1 Processor -- start 2024-04-25T08:32:57.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.306+0000 7f3d98d5c640 1 -- start start 2024-04-25T08:32:57.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.307+0000 7f3d98d5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d9415b940 0x7f3d9409a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.307+0000 7f3d98d5c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d940cc590 con 0x7f3d940cd350 2024-04-25T08:32:57.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.307+0000 7f3d98d5c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d940cc870 con 0x7f3d9409a180 2024-04-25T08:32:57.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.307+0000 7f3d98d5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940ccb50 con 0x7f3d9415b940 2024-04-25T08:32:57.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.307+0000 7f3d91d9b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f3d940cd350 0x7f3d94099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47676/0 (socket says 172.21.15.38:47676) 2024-04-25T08:32:57.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.307+0000 7f3d91d9b640 1 -- 172.21.15.38:0/170020705 learned_addr learned my addr 172.21.15.38:0/170020705 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:32:57.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9259c640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d9415b940 0x7f3d9409a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806026600 0 0) 0x7f3d940cc590 con 0x7f3d940cd350 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d7c003520 con 0x7f3d940cd350 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309970190 0 0) 0x7f3d940cc870 con 0x7f3d9409a180 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d940cc590 con 0x7f3d9409a180 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3620955863 0 0) 0x7f3d7c003520 con 0x7f3d940cd350 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d9415b940 msgr2=0x7f3d9409a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d9415b940 0x7f3d9409a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 >> v1:172.21.15.107:6789/0 conn(0x7f3d9409a180 legacy=0x7f3d940b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940cce30 con 0x7f3d940cd350 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f3d88001670 con 0x7f3d940cd350 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 >> v1:172.21.15.38:6789/0 conn(0x7f3d940cd350 legacy=0x7f3d94099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d7c003b70 0x7f3d7c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d7c004800 0x7f3d7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d7c005210 0x7f3d7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940ccb50 con 0x7f3d7c004800 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c00c140 con 0x7f3d7c005210 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c00c340 con 0x7f3d7c003b70 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9259c640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d9415b940 0x7f3d9409a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d91d9b640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d7c004800 0x7f3d7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d92d9d640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d7c005210 0x7f3d7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d9259c640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d7c003b70 0x7f3d7c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d91d9b640 1 -- 172.21.15.38:0/170020705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d7c003b70 msgr2=0x7f3d7c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d91d9b640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d7c003b70 0x7f3d7c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d91d9b640 1 -- 172.21.15.38:0/170020705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d7c005210 msgr2=0x7f3d7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d91d9b640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d7c005210 0x7f3d7c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.308+0000 7f3d91d9b640 1 -- 172.21.15.38:0/170020705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c003520 con 0x7f3d7c004800 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.309+0000 7f3d91d9b640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d7c004800 0x7f3d7c004c80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3d940cd7b0 tx=0x7f3d7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee1af1309d407e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.309+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88007c40 con 0x7f3d7c004800 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.309+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d88005ad0 con 0x7f3d7c004800 2024-04-25T08:32:57.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.309+0000 7f3d9159a640 1 -- 172.21.15.38:0/170020705 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88007070 con 0x7f3d7c004800 2024-04-25T08:32:57.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.310+0000 7f3d98d5c640 1 -- 172.21.15.38:0/170020705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d7c004800 msgr2=0x7f3d7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:57.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.310+0000 7f3d98d5c640 1 --2- 172.21.15.38:0/170020705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d7c004800 0x7f3d7c004c80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3d940cd7b0 tx=0x7f3d7c005830 comp rx=0 tx=0).stop 2024-04-25T08:32:57.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.310+0000 7f3d92d9d640 1 -- 172.21.15.38:0/170020705 reap_dead start 2024-04-25T08:32:57.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.310+0000 7f3d98d5c640 1 -- 172.21.15.38:0/170020705 shutdown_connections 2024-04-25T08:32:57.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.310+0000 7f3d98d5c640 1 -- 172.21.15.38:0/170020705 >> 172.21.15.38:0/170020705 conn(0x7f3d940b3190 msgr2=0x7f3d940b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:57.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.310+0000 7f3d98d5c640 1 -- 172.21.15.38:0/170020705 shutdown_connections 2024-04-25T08:32:57.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.310+0000 7f3d98d5c640 1 -- 172.21.15.38:0/170020705 wait complete. 2024-04-25T08:32:57.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.311+0000 7f3d98d5c640 1 Processor -- start 2024-04-25T08:32:57.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.311+0000 7f3d98d5c640 1 -- start start 2024-04-25T08:32:57.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d98d5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d7c004800 0x7f3d94148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d98d5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d9415e860 0x7f3d94159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d98d5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94159fd0 0x7f3d940af320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d98d5c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9415bea0 con 0x7f3d94159fd0 2024-04-25T08:32:57.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d98d5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9415bd20 con 0x7f3d7c004800 2024-04-25T08:32:57.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d98d5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d940cf480 con 0x7f3d9415e860 2024-04-25T08:32:57.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d92d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94159fd0 0x7f3d940af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d92d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94159fd0 0x7f3d940af320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58002/0 (socket says 172.21.15.38:58002) 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d92d9d640 1 -- 172.21.15.38:0/2153662372 learned_addr learned my addr 172.21.15.38:0/2153662372 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d9259c640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d7c004800 0x7f3d94148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d91d9b640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d9415e860 0x7f3d94159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d92d9d640 1 -- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d9415e860 msgr2=0x7f3d94159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d92d9d640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d9415e860 0x7f3d94159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d92d9d640 1 -- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d7c004800 msgr2=0x7f3d94148b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d92d9d640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d7c004800 0x7f3d94148b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d92d9d640 1 -- 172.21.15.38:0/2153662372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d940b6530 con 0x7f3d94159fd0 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d91d9b640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d9415e860 0x7f3d94159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.312+0000 7f3d92d9d640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94159fd0 0x7f3d940af320 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400f8d0 tx=0x7f3d8400fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.313+0000 7f3d82ffd640 1 -- 172.21.15.38:0/2153662372 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84019070 con 0x7f3d94159fd0 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.313+0000 7f3d82ffd640 1 -- 172.21.15.38:0/2153662372 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d84006d60 con 0x7f3d94159fd0 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.313+0000 7f3d98d5c640 1 -- 172.21.15.38:0/2153662372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d940b6810 con 0x7f3d94159fd0 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.313+0000 7f3d82ffd640 1 -- 172.21.15.38:0/2153662372 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d84003bb0 con 0x7f3d94159fd0 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.313+0000 7f3d98d5c640 1 -- 172.21.15.38:0/2153662372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d940b6dd0 con 0x7f3d94159fd0 2024-04-25T08:32:57.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.314+0000 7f3d98d5c640 1 -- 172.21.15.38:0/2153662372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9415a450 con 0x7f3d94159fd0 2024-04-25T08:32:57.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.314+0000 7f3d82ffd640 1 -- 172.21.15.38:0/2153662372 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d8401c070 con 0x7f3d94159fd0 2024-04-25T08:32:57.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.315+0000 7f3d82ffd640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3d7406b4d0 0x7f3d7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:32:57.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.315+0000 7f3d82ffd640 1 -- 172.21.15.38:0/2153662372 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d84089590 con 0x7f3d94159fd0 2024-04-25T08:32:57.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.315+0000 7f3d9259c640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3d7406b4d0 0x7f3d7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:32:57.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.315+0000 7f3d9259c640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3d7406b4d0 0x7f3d7406d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3d94072740 tx=0x7f3d8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:32:57.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.319+0000 7f3d82ffd640 1 -- 172.21.15.38:0/2153662372 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d840581f0 con 0x7f3d94159fd0 2024-04-25T08:32:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:57.537+0000 7f3d98d5c640 1 -- 172.21.15.38:0/2153662372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3d94068e80 con 0x7f3d94159fd0 2024-04-25T08:32:58.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.088+0000 7f3d82ffd640 1 -- 172.21.15.38:0/2153662372 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v18) v1 ==== 167+0+0 (secure 0 0 0) 0x7f3d84057af0 con 0x7f3d94159fd0 2024-04-25T08:32:58.085 INFO:teuthology.orchestra.run.smithi038.stderr:enabled new snapshots 2024-04-25T08:32:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.091+0000 7f3d98d5c640 1 -- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3d7406b4d0 msgr2=0x7f3d7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.091+0000 7f3d98d5c640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3d7406b4d0 0x7f3d7406d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3d94072740 tx=0x7f3d8c008040 comp rx=0 tx=0).stop 2024-04-25T08:32:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.091+0000 7f3d98d5c640 1 -- 172.21.15.38:0/2153662372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94159fd0 msgr2=0x7f3d940af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:32:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.091+0000 7f3d98d5c640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94159fd0 0x7f3d940af320 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3d8400f8d0 tx=0x7f3d8400fcd0 comp rx=0 tx=0).stop 2024-04-25T08:32:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.091+0000 7f3d98d5c640 1 -- 172.21.15.38:0/2153662372 shutdown_connections 2024-04-25T08:32:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.091+0000 7f3d98d5c640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d94159fd0 0x7f3d940af320 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.091+0000 7f3d98d5c640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3d7406b4d0 0x7f3d7406d990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.091+0000 7f3d98d5c640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d9415e860 0x7f3d94159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.091+0000 7f3d98d5c640 1 --2- 172.21.15.38:0/2153662372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d7c004800 0x7f3d94148b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:32:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.091+0000 7f3d98d5c640 1 -- 172.21.15.38:0/2153662372 >> 172.21.15.38:0/2153662372 conn(0x7f3d940b3190 msgr2=0x7f3d94099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:32:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.091+0000 7f3d98d5c640 1 -- 172.21.15.38:0/2153662372 shutdown_connections 2024-04-25T08:32:58.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:32:58.091+0000 7f3d98d5c640 1 -- 172.21.15.38:0/2153662372 wait complete. 2024-04-25T08:32:58.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:32:58.100 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T08:32:58.121 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:32:58.121 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:32:58.179 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-04-25T08:32:58.179 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:32:58.179 DEBUG:teuthology.orchestra.run.smithi038:> import uuid 2024-04-25T08:32:58.179 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:58.179 DEBUG:teuthology.orchestra.run.smithi038:> n = 2000 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> create_hard_links = False 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> if 3 > 0: 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> create_hard_links = True 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> dpath = os.path.dirname(path) 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> fnameprefix = os.path.basename(path) 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> os.makedirs(dpath, exist_ok=True) 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> try: 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> for i in range(n): 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> f.write(f"{i}") 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> if False: 2024-04-25T08:32:58.180 DEBUG:teuthology.orchestra.run.smithi038:> f.flush() 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> os.fsync(f.fileno()) 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> if False: 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> os.unlink(fpath) 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> if False: 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> os.fsync(dirfd) 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> if create_hard_links: 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> for j in range(3): 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> if False: 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> os.fsync(dirfd) 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> finally: 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> os.close(dirfd) 2024-04-25T08:32:58.181 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:33:02.363 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:33:02.366 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:33:26.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:33:26.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:26.233 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T08:33:26.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:26.282 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T08:33:32.417 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:33:32.420 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:33:55.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:55.788 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T08:33:55.868 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:33:55.869 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T08:33:55.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.977+0000 7f53c7dd5640 1 Processor -- start 2024-04-25T08:33:55.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.977+0000 7f53c7dd5640 1 -- start start 2024-04-25T08:33:55.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.978+0000 7f53c7dd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c00d0320 0x7f53c00d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.978+0000 7f53c7dd5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f53c00d0700 0x7f53c00d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.978+0000 7f53c7dd5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c00d2330 con 0x7f53c00d1e90 2024-04-25T08:33:55.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.978+0000 7f53c7dd5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c00d2610 con 0x7f53c00d0700 2024-04-25T08:33:55.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.978+0000 7f53c7dd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c00d28f0 con 0x7f53c00d0320 2024-04-25T08:33:55.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.978+0000 7f53c5b71640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c00d0320 0x7f53c00d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.978+0000 7f53c5b71640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c00d0320 0x7f53c00d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:42862/0 (socket says 172.21.15.38:42862) 2024-04-25T08:33:55.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.978+0000 7f53c5b71640 1 -- 172.21.15.38:0/1709847530 learned_addr learned my addr 172.21.15.38:0/1709847530 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:55.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.978+0000 7f53c5370640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f53c00d0700 0x7f53c00d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.979+0000 7f53c5b71640 1 -- 172.21.15.38:0/1709847530 >> v2:172.21.15.107:3300/0 conn(0x7f53c00d0700 msgr2=0x7f53c00d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:55.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.979+0000 7f53c5b71640 1 --2- 172.21.15.38:0/1709847530 >> v2:172.21.15.107:3300/0 conn(0x7f53c00d0700 0x7f53c00d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.979+0000 7f53c5b71640 1 -- 172.21.15.38:0/1709847530 >> v1:172.21.15.107:6789/0 conn(0x7f53c00d1e90 legacy=0x7f53c00b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:55.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.979+0000 7f53c5b71640 1 -- 172.21.15.38:0/1709847530 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c00d2bd0 con 0x7f53c00d0320 2024-04-25T08:33:55.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.979+0000 7f53c5b71640 1 --2- 172.21.15.38:0/1709847530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c00d0320 0x7f53c00d33d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009870 tx=0x7f53bc02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9deff9da743b7635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:55.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.980+0000 7f53c4b6f640 1 -- 172.21.15.38:0/1709847530 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc037070 con 0x7f53c00d0320 2024-04-25T08:33:55.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.980+0000 7f53c4b6f640 1 -- 172.21.15.38:0/1709847530 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc02ae50 con 0x7f53c00d0320 2024-04-25T08:33:55.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.980+0000 7f53c4b6f640 1 -- 172.21.15.38:0/1709847530 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc032860 con 0x7f53c00d0320 2024-04-25T08:33:55.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.980+0000 7f53c7dd5640 1 -- 172.21.15.38:0/1709847530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c00d0320 msgr2=0x7f53c00d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:55.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.980+0000 7f53c7dd5640 1 --2- 172.21.15.38:0/1709847530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c00d0320 0x7f53c00d33d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f53bc009870 tx=0x7f53bc02a300 comp rx=0 tx=0).stop 2024-04-25T08:33:55.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.980+0000 7f53c7dd5640 1 -- 172.21.15.38:0/1709847530 shutdown_connections 2024-04-25T08:33:55.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.980+0000 7f53c7dd5640 1 --2- 172.21.15.38:0/1709847530 >> v2:172.21.15.107:3300/0 conn(0x7f53c00d0700 0x7f53c00d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.980+0000 7f53c7dd5640 1 --2- 172.21.15.38:0/1709847530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c00d0320 0x7f53c00d33d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.980+0000 7f53c7dd5640 1 -- 172.21.15.38:0/1709847530 >> 172.21.15.38:0/1709847530 conn(0x7f53c00668f0 msgr2=0x7f53c0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:55.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.980+0000 7f53c7dd5640 1 -- 172.21.15.38:0/1709847530 shutdown_connections 2024-04-25T08:33:55.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.980+0000 7f53c7dd5640 1 -- 172.21.15.38:0/1709847530 wait complete. 2024-04-25T08:33:55.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.981+0000 7f53c7dd5640 1 Processor -- start 2024-04-25T08:33:55.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.981+0000 7f53c7dd5640 1 -- start start 2024-04-25T08:33:55.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.982+0000 7f53c7dd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53c00d0320 0x7f53c0152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.982+0000 7f53c7dd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c00d0700 0x7f53c00b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.982+0000 7f53c7dd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c00d1e90 0x7f53c00b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.982+0000 7f53c7dd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c00d0b40 con 0x7f53c00d1e90 2024-04-25T08:33:55.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.982+0000 7f53c7dd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c0161210 con 0x7f53c00d0320 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.982+0000 7f53c7dd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c0153350 con 0x7f53c00d0700 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.982+0000 7f53c6372640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c00d1e90 0x7f53c00b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.982+0000 7f53c6372640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c00d1e90 0x7f53c00b5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32942/0 (socket says 172.21.15.38:32942) 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.982+0000 7f53c6372640 1 -- 172.21.15.38:0/482647976 learned_addr learned my addr 172.21.15.38:0/482647976 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.982+0000 7f53c5b71640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53c00d0320 0x7f53c0152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.982+0000 7f53c5370640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c00d0700 0x7f53c00b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.983+0000 7f53c6372640 1 -- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c00d0700 msgr2=0x7f53c00b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.983+0000 7f53c6372640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c00d0700 0x7f53c00b3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.983+0000 7f53c6372640 1 -- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53c00d0320 msgr2=0x7f53c0152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.983+0000 7f53c6372640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53c00d0320 0x7f53c0152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.983+0000 7f53c6372640 1 -- 172.21.15.38:0/482647976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c00cb5e0 con 0x7f53c00d1e90 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.983+0000 7f53c6372640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c00d1e90 0x7f53c00b5700 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f53b800c730 tx=0x7f53b800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.983+0000 7f53b67fc640 1 -- 172.21.15.38:0/482647976 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8003b70 con 0x7f53c00d1e90 2024-04-25T08:33:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.983+0000 7f53b67fc640 1 -- 172.21.15.38:0/482647976 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b8003d10 con 0x7f53c00d1e90 2024-04-25T08:33:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.983+0000 7f53c7dd5640 1 -- 172.21.15.38:0/482647976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c00cb8c0 con 0x7f53c00d1e90 2024-04-25T08:33:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.983+0000 7f53b67fc640 1 -- 172.21.15.38:0/482647976 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8005310 con 0x7f53c00d1e90 2024-04-25T08:33:55.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.983+0000 7f53c7dd5640 1 -- 172.21.15.38:0/482647976 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c00cbe80 con 0x7f53c00d1e90 2024-04-25T08:33:55.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.985+0000 7f53b67fc640 1 -- 172.21.15.38:0/482647976 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53b80056c0 con 0x7f53c00d1e90 2024-04-25T08:33:55.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.985+0000 7f53b67fc640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f53a406b4d0 0x7f53a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.985+0000 7f53b67fc640 1 -- 172.21.15.38:0/482647976 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f53b808a2b0 con 0x7f53c00d1e90 2024-04-25T08:33:55.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.985+0000 7f53c5b71640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f53a406b4d0 0x7f53a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.986+0000 7f53c5b71640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f53a406b4d0 0x7f53a406d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f53bc034000 tx=0x7f53bc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:55.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.986+0000 7f53cabba740 1 Processor -- start 2024-04-25T08:33:55.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.987+0000 7f53cabba740 1 -- start start 2024-04-25T08:33:55.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.987+0000 7f53cabba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 0x55bf0409ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.987+0000 7f53cabba740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 0x55bf04099080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.987+0000 7f53cabba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 0x55bf040c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.987+0000 7f53cabba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf03ffbb90 con 0x55bf040c3760 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.987+0000 7f53cabba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf03fa4e40 con 0x55bf04098ca0 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.987+0000 7f53cabba740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf03ff7c30 con 0x55bf040d5370 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.987+0000 7f53c6372640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 0x55bf040c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.987+0000 7f53c6372640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 0x55bf040c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32946/0 (socket says 172.21.15.38:32946) 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.987+0000 7f53c6372640 1 -- 172.21.15.38:0/3471410770 learned_addr learned my addr 172.21.15.38:0/3471410770 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.987+0000 7f53c5b71640 1 --2- 172.21.15.38:0/3471410770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 0x55bf0409ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.987+0000 7f53c5370640 1 --2- 172.21.15.38:0/3471410770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 0x55bf04099080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.988+0000 7f53c6372640 1 -- 172.21.15.38:0/3471410770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 msgr2=0x55bf04099080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.988+0000 7f53c6372640 1 --2- 172.21.15.38:0/3471410770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 0x55bf04099080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.988+0000 7f53c6372640 1 -- 172.21.15.38:0/3471410770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 msgr2=0x55bf0409ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:55.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.988+0000 7f53c6372640 1 --2- 172.21.15.38:0/3471410770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 0x55bf0409ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.988+0000 7f53c6372640 1 -- 172.21.15.38:0/3471410770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf03dda7d0 con 0x55bf040c3760 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.988+0000 7f53c5370640 1 --2- 172.21.15.38:0/3471410770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 0x55bf04099080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.988+0000 7f53c6372640 1 --2- 172.21.15.38:0/3471410770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 0x55bf040c3be0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f53b8060a70 tx=0x7f53b8058940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e10cba5c911307a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.988+0000 7f53a37fe640 1 -- 172.21.15.38:0/3471410770 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b80569e0 con 0x55bf040c3760 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.988+0000 7f53a37fe640 1 -- 172.21.15.38:0/3471410770 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b8056c40 con 0x55bf040c3760 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.988+0000 7f53a37fe640 1 -- 172.21.15.38:0/3471410770 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8059bc0 con 0x55bf040c3760 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.989+0000 7f53cabba740 1 -- 172.21.15.38:0/3471410770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 msgr2=0x55bf040c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.989+0000 7f53cabba740 1 --2- 172.21.15.38:0/3471410770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 0x55bf040c3be0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f53b8060a70 tx=0x7f53b8058940 comp rx=0 tx=0).stop 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.989+0000 7f53cabba740 1 -- 172.21.15.38:0/3471410770 shutdown_connections 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.989+0000 7f53cabba740 1 --2- 172.21.15.38:0/3471410770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 0x55bf040c3be0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.989+0000 7f53cabba740 1 --2- 172.21.15.38:0/3471410770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 0x55bf04099080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.989+0000 7f53cabba740 1 --2- 172.21.15.38:0/3471410770 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 0x55bf0409ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.989+0000 7f53cabba740 1 -- 172.21.15.38:0/3471410770 >> 172.21.15.38:0/3471410770 conn(0x55bf04060870 msgr2=0x55bf040de760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:55.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.989+0000 7f53cabba740 1 -- 172.21.15.38:0/3471410770 shutdown_connections 2024-04-25T08:33:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.989+0000 7f53cabba740 1 -- 172.21.15.38:0/3471410770 wait complete. 2024-04-25T08:33:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.989+0000 7f53cabba740 1 -- start start 2024-04-25T08:33:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.989+0000 7f53cabba740 1 Processor -- start 2024-04-25T08:33:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.990+0000 7f53cabba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 0x55bf040c6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.990+0000 7f53cabba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 0x55bf040c6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.990+0000 7f53cabba740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 0x55bf040c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.990+0000 7f53cabba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf03fb83e0 con 0x55bf040c3760 2024-04-25T08:33:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.990+0000 7f53cabba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf040c12b0 con 0x55bf04098ca0 2024-04-25T08:33:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.990+0000 7f53cabba740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf040da290 con 0x55bf040d5370 2024-04-25T08:33:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.990+0000 7f53cabba740 10 client.0 _ll_register_callbacks cb 0x55bf03fb7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:33:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.990+0000 7f53c5370640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 0x55bf040c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.990+0000 7f53c5370640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 0x55bf040c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32962/0 (socket says 172.21.15.38:32962) 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.990+0000 7f53c5370640 1 -- 172.21.15.38:0/2298542848 learned_addr learned my addr 172.21.15.38:0/2298542848 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.990+0000 7f53c5b71640 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 0x55bf040c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53c6372640 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 0x55bf040c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53c5370640 1 -- 172.21.15.38:0/2298542848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 msgr2=0x55bf040c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53c5370640 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 0x55bf040c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53c5370640 1 -- 172.21.15.38:0/2298542848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 msgr2=0x55bf040c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53c5370640 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 0x55bf040c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53c5370640 1 -- 172.21.15.38:0/2298542848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf03cd00f0 con 0x55bf040c3760 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53c5370640 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 0x55bf040c6760 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f53b0014600 tx=0x7f53b0014ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc69b5492e5d527 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53a0ff9640 1 -- 172.21.15.38:0/2298542848 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0004300 con 0x55bf040c3760 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53a0ff9640 1 -- 172.21.15.38:0/2298542848 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f53b0003610 con 0x55bf040c3760 2024-04-25T08:33:55.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53cabba740 1 -- 172.21.15.38:0/2298542848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53b8056c40 con 0x55bf040c3760 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53a0ff9640 1 -- 172.21.15.38:0/2298542848 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53b0027050 con 0x55bf040c3760 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53cabba740 10 client.5685 fetch_fsmap learned FSMap version 18 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53cabba740 1 -- 172.21.15.38:0/2298542848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55bf03e3af60 con 0x55bf040c3760 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.991+0000 7f53a0ff9640 1 -- 172.21.15.38:0/2298542848 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b8056c40 con 0x55bf040c3760 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.992+0000 7f53a0ff9640 1 -- 172.21.15.38:0/2298542848 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53b002a020 con 0x55bf040c3760 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.992+0000 7f53cabba740 10 client.5685 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.992+0000 7f53cabba740 10 client.5685 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.992+0000 7f53cabba740 20 client.5685 populate_metadata read hostname 'smithi038' 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.992+0000 7f53cabba740 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bf040b3e70 0x55bf040d9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.992+0000 7f53cabba740 4 client.5685 mds_command: new command op to 5149 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.992+0000 7f53cabba740 1 -- 172.21.15.38:0/2298542848 --> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf040090d0 con 0x55bf040b3e70 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.992+0000 7f53c5b71640 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bf040b3e70 0x55bf040d9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.992+0000 7f53a0ff9640 10 client.5685 ms_handle_connect on v2:172.21.15.38:6832/234860294 2024-04-25T08:33:55.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.992+0000 7f53c5b71640 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bf040b3e70 0x55bf040d9420 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3203673a33092ff server_cookie=ac5694c81f2c17c7 in_seq=0 out_seq=0 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53a0ff9640 1 -- 172.21.15.38:0/2298542848 <== mds.0 v2:172.21.15.38:6832/234860294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf040090d0 con 0x55bf040b3e70 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53a0ff9640 10 client.5685 handle_command_reply: tid=0 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53cabba740 1 client.5685 shutdown 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53cabba740 1 -- 172.21.15.38:0/2298542848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 msgr2=0x55bf040c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53cabba740 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 0x55bf040c6760 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f53b0014600 tx=0x7f53b0014ac0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53cabba740 1 -- 172.21.15.38:0/2298542848 shutdown_connections 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53cabba740 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bf040b3e70 0x55bf040d9420 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53cabba740 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 0x55bf040c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53cabba740 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040c3760 0x55bf040c6760 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53cabba740 1 --2- 172.21.15.38:0/2298542848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 0x55bf040c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53cabba740 1 -- 172.21.15.38:0/2298542848 >> 172.21.15.38:0/2298542848 conn(0x55bf04060870 msgr2=0x55bf040c1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53a0ff9640 0 client.5685 ms_handle_reset on v2:172.21.15.38:6832/234860294 2024-04-25T08:33:55.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53cabba740 1 -- 172.21.15.38:0/2298542848 shutdown_connections 2024-04-25T08:33:55.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53cabba740 1 -- 172.21.15.38:0/2298542848 wait complete. 2024-04-25T08:33:55.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:55.993+0000 7f53cabba740 20 client.5685 trim_cache size 0 max 16384 2024-04-25T08:33:56.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.007+0000 7f53cabba740 1 Processor -- start 2024-04-25T08:33:56.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.007+0000 7f53cabba740 1 -- start start 2024-04-25T08:33:56.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53cabba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf04098ca0 0x55bf040bea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53cabba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf040c3760 0x55bf040bef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53cabba740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 0x55bf040d40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53cabba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c0200570 con 0x55bf04098ca0 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53cabba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c01ff770 con 0x55bf040c3760 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53cabba740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c00ccf90 con 0x55bf040d5370 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53c5b71640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf04098ca0 0x55bf040bea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53c5b71640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf04098ca0 0x55bf040bea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32978/0 (socket says 172.21.15.38:32978) 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53c5b71640 1 -- 172.21.15.38:0/4193803710 learned_addr learned my addr 172.21.15.38:0/4193803710 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53c5b71640 1 -- 172.21.15.38:0/4193803710 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 msgr2=0x55bf040d40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53c5b71640 1 --2- 172.21.15.38:0/4193803710 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 0x55bf040d40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53c5b71640 1 -- 172.21.15.38:0/4193803710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf040c3760 msgr2=0x55bf040bef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53c5b71640 1 --2- 172.21.15.38:0/4193803710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf040c3760 0x55bf040bef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.008+0000 7f53c5b71640 1 -- 172.21.15.38:0/4193803710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf03ca91b0 con 0x55bf04098ca0 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53c5b71640 1 --2- 172.21.15.38:0/4193803710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf04098ca0 0x55bf040bea30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f53bc02a340 tx=0x7f53bc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41ece249467ae1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53a2ffd640 1 -- 172.21.15.38:0/4193803710 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc037040 con 0x55bf04098ca0 2024-04-25T08:33:56.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53a2ffd640 1 -- 172.21.15.38:0/4193803710 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc054070 con 0x55bf04098ca0 2024-04-25T08:33:56.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53a2ffd640 1 -- 172.21.15.38:0/4193803710 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc032d90 con 0x55bf04098ca0 2024-04-25T08:33:56.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53cabba740 1 -- 172.21.15.38:0/4193803710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf04098ca0 msgr2=0x55bf040bea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53cabba740 1 --2- 172.21.15.38:0/4193803710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf04098ca0 0x55bf040bea30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f53bc02a340 tx=0x7f53bc02c000 comp rx=0 tx=0).stop 2024-04-25T08:33:56.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53cabba740 1 -- 172.21.15.38:0/4193803710 shutdown_connections 2024-04-25T08:33:56.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53cabba740 1 --2- 172.21.15.38:0/4193803710 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040d5370 0x55bf040d40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53cabba740 1 --2- 172.21.15.38:0/4193803710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf040c3760 0x55bf040bef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53cabba740 1 --2- 172.21.15.38:0/4193803710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf04098ca0 0x55bf040bea30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53cabba740 1 -- 172.21.15.38:0/4193803710 >> 172.21.15.38:0/4193803710 conn(0x55bf04060870 msgr2=0x55bf04099b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53cabba740 1 -- 172.21.15.38:0/4193803710 shutdown_connections 2024-04-25T08:33:56.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.009+0000 7f53cabba740 1 -- 172.21.15.38:0/4193803710 wait complete. 2024-04-25T08:33:56.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.010+0000 7f53cabba740 1 -- start start 2024-04-25T08:33:56.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.010+0000 7f53cabba740 1 Processor -- start 2024-04-25T08:33:56.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53cabba740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 0x55bf040bee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53cabba740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040c3760 0x55bf04143270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53cabba740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040d5370 0x55bf041437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53cabba740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf040069b0 con 0x55bf040d5370 2024-04-25T08:33:56.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53cabba740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c00cd590 con 0x55bf04098ca0 2024-04-25T08:33:56.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53cabba740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf040db970 con 0x55bf040c3760 2024-04-25T08:33:56.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53cabba740 10 client.0 _ll_register_callbacks cb 0x7f53b00097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:33:56.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53c6372640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040d5370 0x55bf041437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53c6372640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040d5370 0x55bf041437b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32984/0 (socket says 172.21.15.38:32984) 2024-04-25T08:33:56.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53c6372640 1 -- 172.21.15.38:0/2589043112 learned_addr learned my addr 172.21.15.38:0/2589043112 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:56.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53c5b71640 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 0x55bf040bee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53c5370640 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040c3760 0x55bf04143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53c6372640 1 -- 172.21.15.38:0/2589043112 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040c3760 msgr2=0x55bf04143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53c6372640 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040c3760 0x55bf04143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53c6372640 1 -- 172.21.15.38:0/2589043112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 msgr2=0x55bf040bee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53c6372640 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 0x55bf040bee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53c6372640 1 -- 172.21.15.38:0/2589043112 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf040d4bc0 con 0x55bf040d5370 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53c6372640 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040d5370 0x55bf041437b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f53b8054d50 tx=0x7f53b8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8640c5df73fce65c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53a0ff9640 1 -- 172.21.15.38:0/2589043112 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8086060 con 0x55bf040d5370 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53a0ff9640 1 -- 172.21.15.38:0/2589043112 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f53b8054fb0 con 0x55bf040d5370 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.011+0000 7f53cabba740 1 -- 172.21.15.38:0/2589043112 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53bc054070 con 0x55bf040d5370 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53a0ff9640 1 -- 172.21.15.38:0/2589043112 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53b8059bc0 con 0x55bf040d5370 2024-04-25T08:33:56.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53cabba740 10 client.5691 fetch_fsmap learned FSMap version 18 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53cabba740 1 -- 172.21.15.38:0/2589043112 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55bf0414acd0 con 0x55bf040d5370 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53a0ff9640 1 -- 172.21.15.38:0/2589043112 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc054070 con 0x55bf040d5370 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53a0ff9640 1 -- 172.21.15.38:0/2589043112 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53b8070170 con 0x55bf040d5370 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53cabba740 10 client.5691 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53cabba740 10 client.5691 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53cabba740 20 client.5691 populate_metadata read hostname 'smithi038' 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53cabba740 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bf040db0b0 0x55bf040d7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53cabba740 4 client.5691 mds_command: new command op to 5149 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53cabba740 1 -- 172.21.15.38:0/2589043112 --> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55bf04003420 con 0x55bf040db0b0 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53c5b71640 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bf040db0b0 0x55bf040d7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53a0ff9640 10 client.5691 ms_handle_connect on v2:172.21.15.38:6832/234860294 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.012+0000 7f53c5b71640 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bf040db0b0 0x55bf040d7760 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7efc4d85d0c79f2 server_cookie=61696d041b443b52 in_seq=0 out_seq=0 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53a0ff9640 1 -- 172.21.15.38:0/2589043112 <== mds.0 v2:172.21.15.38:6832/234860294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17925 (crc 0 0 0) 0x55bf04003420 con 0x55bf040db0b0 2024-04-25T08:33:56.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53a0ff9640 10 client.5691 handle_command_reply: tid=0 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53cabba740 1 client.5691 shutdown 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53cabba740 1 -- 172.21.15.38:0/2589043112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040d5370 msgr2=0x55bf041437b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53cabba740 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040d5370 0x55bf041437b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f53b8054d50 tx=0x7f53b8001250 comp rx=0 tx=0).stop 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53cabba740 1 -- 172.21.15.38:0/2589043112 shutdown_connections 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53cabba740 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bf040db0b0 0x55bf040d7760 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53a0ff9640 0 client.5691 ms_handle_reset on v2:172.21.15.38:6832/234860294 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53cabba740 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf040d5370 0x55bf041437b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53cabba740 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bf040c3760 0x55bf04143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53cabba740 1 --2- 172.21.15.38:0/2589043112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bf04098ca0 0x55bf040bee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53cabba740 1 -- 172.21.15.38:0/2589043112 >> 172.21.15.38:0/2589043112 conn(0x55bf04060870 msgr2=0x55bf041011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53cabba740 1 -- 172.21.15.38:0/2589043112 shutdown_connections 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53cabba740 1 -- 172.21.15.38:0/2589043112 wait complete. 2024-04-25T08:33:56.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.013+0000 7f53cabba740 20 client.5691 trim_cache size 0 max 16384 2024-04-25T08:33:56.017 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:33:56.017 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T08:33:56.017 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 4032, 2024-04-25T08:33:56.017 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 4032, 2024-04-25T08:33:56.017 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 998471, 2024-04-25T08:33:56.017 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 39073670, 2024-04-25T08:33:56.017 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 9, 2024-04-25T08:33:56.017 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-04-25T08:33:56.017 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.347086109, 2024-04-25T08:33:56.017 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.157522086, 2024-04-25T08:33:56.017 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.066208776, 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.101509535, 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 4031, 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.263322557, 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000065324 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2, 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000224, 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000112 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T08:33:56.018 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 2217, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 2234, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 549852, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 17604450, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 6, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.193362934, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.088140161, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.038691377, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.055250433, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2232, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.243554401, 2024-04-25T08:33:56.019 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000109119 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 110304 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 34136, 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 60057, 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 12136702, 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 54320622, 2024-04-25T08:33:56.020 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 7, 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 2.941201617, 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 1.462024492, 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 1.740223805, 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.052774081, 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 60057, 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.039243329, 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000017304 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 88300, 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.013193931, 2024-04-25T08:33:56.021 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000149 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 117488, 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 17792 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 9080, 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 12.604478058, 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001388158 2024-04-25T08:33:56.022 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1579, 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.550023585, 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000348336 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 28175, 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 28175, 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 28175, 2024-04-25T08:33:56.023 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 258.454011423, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.009173168 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 12, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 19845, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 19845, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-04-25T08:33:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 19845, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 647, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 647, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 2, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 46529, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 46131, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 140, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-04-25T08:33:56.025 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T08:33:56.026 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 6166, 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 6007, 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 6166, 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 24, 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 17858 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-04-25T08:33:56.027 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 2000, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 319, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 4121, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 1355, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 2211, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-04-25T08:33:56.028 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 21548, 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-04-25T08:33:56.029 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 21548, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 29, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 29, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 3, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 80883965, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 0, 2024-04-25T08:33:56.030 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6460, 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 79.241555500, 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.012266494 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 0 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 2014, 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 2014, 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 0, 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 13, 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 13, 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 0, 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 19845, 2024-04-25T08:33:56.031 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 19845, 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 0, 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 330, 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 2002, 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 1672, 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 103904, 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215396 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 37455, 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 28175, 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 5, 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-04-25T08:33:56.032 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2000, 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.348468024, 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000674234 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1927, 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.581732819, 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000301885 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.033 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6000, 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 9.325768037, 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001554294 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 8024, 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 6.845976495, 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000853187 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.035 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3, 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.052661596, 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.017553865 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.002772077, 2024-04-25T08:33:56.036 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002772077 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.005103231, 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.005103231 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.037 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6, 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.092864459, 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.015477409 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2211, 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 224.863278792, 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.101702070 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.038 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.003313507, 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.003313507 2024-04-25T08:33:56.039 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000225571, 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000225571 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.040 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.041 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 8000, 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 15.331846815, 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001916480 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.042 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 1, 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 0, 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 19561, 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 19561, 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 56, 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-04-25T08:33:56.043 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-04-25T08:33:56.044 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T08:33:56.045 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-04-25T08:33:56.046 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-04-25T08:33:56.046 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-04-25T08:33:56.046 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-04-25T08:33:56.046 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T08:33:56.046 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-04-25T08:33:56.046 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T08:33:56.046 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-04-25T08:33:56.046 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 16842033, 2024-04-25T08:33:56.046 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 4253, 2024-04-25T08:33:56.046 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-04-25T08:33:56.046 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-04-25T08:33:56.046 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-04-25T08:33:56.047 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-04-25T08:33:56.047 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-04-25T08:33:56.047 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-04-25T08:33:56.047 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-04-25T08:33:56.047 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-04-25T08:33:56.047 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 5520, 2024-04-25T08:33:56.047 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 78, 2024-04-25T08:33:56.047 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T08:33:56.047 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-04-25T08:33:56.047 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-04-25T08:33:56.047 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-04-25T08:33:56.047 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 20485853, 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 803203, 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 8059, 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 76855655, 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 20, 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 8059, 2024-04-25T08:33:56.048 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 8059, 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 23.674582110, 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002937657 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 8059, 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 8127 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 8059, 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 55, 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 8004, 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-04-25T08:33:56.049 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 0, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 11, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 55, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 6515, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 35, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 1355, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 30, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 0, 2024-04-25T08:33:56.050 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 22, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-04-25T08:33:56.051 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 40, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-04-25T08:33:56.052 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 3, 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 8, 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 8, 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 90, 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 0, 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 14 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 1, 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-04-25T08:33:56.053 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 32, 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 1355, 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 1355, 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:33:56.054 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 40380, 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 10046404, 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 40380, 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 40380, 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 10046404, 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:33:56.055 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 8059, 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 77003350, 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 8059, 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 77003350, 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:33:56.056 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 8059, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 8059, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 8059, 2024-04-25T08:33:56.057 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 8059, 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 1355, 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 147695, 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 1355, 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T08:33:56.058 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 55, 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 147695, 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x557bda42c6e0": { 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 84, 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 21548, 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 76689661, 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:33:56.059 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 21548, 2024-04-25T08:33:56.060 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T08:33:56.060 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T08:33:56.060 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 6460, 2024-04-25T08:33:56.060 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 76689577, 2024-04-25T08:33:56.060 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:33:56.060 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:33:56.060 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:33:56.060 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:33:56.060 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:33:56.060 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:33:56.060 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:33:56.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.014+0000 7f53c7dd5640 1 -- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f53a406b4d0 msgr2=0x7f53a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.014+0000 7f53c7dd5640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f53a406b4d0 0x7f53a406d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f53bc034000 tx=0x7f53bc02e000 comp rx=0 tx=0).stop 2024-04-25T08:33:56.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.014+0000 7f53c7dd5640 1 -- 172.21.15.38:0/482647976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c00d1e90 msgr2=0x7f53c00b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.014+0000 7f53c7dd5640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c00d1e90 0x7f53c00b5700 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f53b800c730 tx=0x7f53b800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.014+0000 7f53c7dd5640 1 -- 172.21.15.38:0/482647976 shutdown_connections 2024-04-25T08:33:56.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.014+0000 7f53c7dd5640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c00d1e90 0x7f53c00b5700 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.014+0000 7f53c7dd5640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f53a406b4d0 0x7f53a406d990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.014+0000 7f53c7dd5640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c00d0700 0x7f53c00b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.014+0000 7f53c7dd5640 1 --2- 172.21.15.38:0/482647976 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53c00d0320 0x7f53c0152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.014+0000 7f53c7dd5640 1 -- 172.21.15.38:0/482647976 >> 172.21.15.38:0/482647976 conn(0x7f53c00668f0 msgr2=0x7f53c0160230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.014+0000 7f53c7dd5640 1 -- 172.21.15.38:0/482647976 shutdown_connections 2024-04-25T08:33:56.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.014+0000 7f53c7dd5640 1 -- 172.21.15.38:0/482647976 wait complete. 2024-04-25T08:33:56.063 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T08:33:56.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:56.063 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-25T08:33:56.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.172+0000 7fb967458640 1 Processor -- start 2024-04-25T08:33:56.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.172+0000 7fb967458640 1 -- start start 2024-04-25T08:33:56.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.173+0000 7fb967458640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb96015bd00 0x7fb9600ae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.173+0000 7fb967458640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9600cd1f0 con 0x7fb9600ccca0 2024-04-25T08:33:56.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.173+0000 7fb967458640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb960099b10 con 0x7fb9600cf980 2024-04-25T08:33:56.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.173+0000 7fb967458640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb96009a160 con 0x7fb96015bd00 2024-04-25T08:33:56.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.173+0000 7fb9649f3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb96015bd00 0x7fb9600ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.173+0000 7fb9659f5640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb9600ccca0 0x7fb96009a5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34628/0 (socket says 172.21.15.38:34628) 2024-04-25T08:33:56.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.173+0000 7fb9659f5640 1 -- 172.21.15.38:0/3445527063 learned_addr learned my addr 172.21.15.38:0/3445527063 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:33:56.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.173+0000 7fb9649f3640 1 -- 172.21.15.38:0/3445527063 >> v1:172.21.15.107:6789/0 conn(0x7fb9600cf980 legacy=0x7fb9600cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.173+0000 7fb9649f3640 1 -- 172.21.15.38:0/3445527063 >> v1:172.21.15.38:6789/0 conn(0x7fb9600ccca0 legacy=0x7fb96009a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.173+0000 7fb9649f3640 1 -- 172.21.15.38:0/3445527063 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb960099df0 con 0x7fb96015bd00 2024-04-25T08:33:56.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.174+0000 7fb9649f3640 1 --2- 172.21.15.38:0/3445527063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96015bd00 0x7fb9600ae360 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb9540097a0 tx=0x7fb95402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b64f389b3cf78a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.174+0000 7fb953fff640 1 -- 172.21.15.38:0/3445527063 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954037070 con 0x7fb96015bd00 2024-04-25T08:33:56.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.174+0000 7fb953fff640 1 -- 172.21.15.38:0/3445527063 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb954032440 con 0x7fb96015bd00 2024-04-25T08:33:56.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.174+0000 7fb953fff640 1 -- 172.21.15.38:0/3445527063 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954032810 con 0x7fb96015bd00 2024-04-25T08:33:56.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.175+0000 7fb967458640 1 -- 172.21.15.38:0/3445527063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96015bd00 msgr2=0x7fb9600ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.175+0000 7fb967458640 1 --2- 172.21.15.38:0/3445527063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96015bd00 0x7fb9600ae360 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fb9540097a0 tx=0x7fb95402a230 comp rx=0 tx=0).stop 2024-04-25T08:33:56.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.175+0000 7fb967458640 1 -- 172.21.15.38:0/3445527063 shutdown_connections 2024-04-25T08:33:56.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.175+0000 7fb967458640 1 --2- 172.21.15.38:0/3445527063 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb96015bd00 0x7fb9600ae360 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.175+0000 7fb967458640 1 -- 172.21.15.38:0/3445527063 >> 172.21.15.38:0/3445527063 conn(0x7fb9600b31b0 msgr2=0x7fb9600b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.175+0000 7fb967458640 1 -- 172.21.15.38:0/3445527063 shutdown_connections 2024-04-25T08:33:56.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.175+0000 7fb967458640 1 -- 172.21.15.38:0/3445527063 wait complete. 2024-04-25T08:33:56.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.176+0000 7fb967458640 1 Processor -- start 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.176+0000 7fb967458640 1 -- start start 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb967458640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9600ccca0 0x7fb960155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb967458640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9600cf980 0x7fb96006d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb967458640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb96015bd00 0x7fb96006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb967458640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600cf770 con 0x7fb9600cf980 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb967458640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600cf5f0 con 0x7fb96015bd00 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb967458640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96009ad00 con 0x7fb9600ccca0 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb9649f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9600cf980 0x7fb96006d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb9649f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9600cf980 0x7fb96006d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33006/0 (socket says 172.21.15.38:33006) 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb9649f3640 1 -- 172.21.15.38:0/3274867409 learned_addr learned my addr 172.21.15.38:0/3274867409 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb9659f5640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb96015bd00 0x7fb96006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb9651f4640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9600ccca0 0x7fb960155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb9649f3640 1 -- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9600ccca0 msgr2=0x7fb960155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb9649f3640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9600ccca0 0x7fb960155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb9649f3640 1 -- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb96015bd00 msgr2=0x7fb96006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb9649f3640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb96015bd00 0x7fb96006d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb9649f3640 1 -- 172.21.15.38:0/3274867409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9601f3a90 con 0x7fb9600cf980 2024-04-25T08:33:56.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb9649f3640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9600cf980 0x7fb96006d3f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb954035040 tx=0x7fb9540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb951ffb640 1 -- 172.21.15.38:0/3274867409 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954037040 con 0x7fb9600cf980 2024-04-25T08:33:56.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb951ffb640 1 -- 172.21.15.38:0/3274867409 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb954030770 con 0x7fb9600cf980 2024-04-25T08:33:56.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb967458640 1 -- 172.21.15.38:0/3274867409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9601f3d70 con 0x7fb9600cf980 2024-04-25T08:33:56.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.177+0000 7fb951ffb640 1 -- 172.21.15.38:0/3274867409 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954030b40 con 0x7fb9600cf980 2024-04-25T08:33:56.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.178+0000 7fb967458640 1 -- 172.21.15.38:0/3274867409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9601f4330 con 0x7fb9600cf980 2024-04-25T08:33:56.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.179+0000 7fb951ffb640 1 -- 172.21.15.38:0/3274867409 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb954044020 con 0x7fb9600cf980 2024-04-25T08:33:56.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.179+0000 7fb951ffb640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb94406b4d0 0x7fb94406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.180+0000 7fb951ffb640 1 -- 172.21.15.38:0/3274867409 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fb9540aa830 con 0x7fb9600cf980 2024-04-25T08:33:56.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.180+0000 7fb9651f4640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb94406b4d0 0x7fb94406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.180+0000 7fb9651f4640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb94406b4d0 0x7fb94406d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb95c001780 tx=0x7fb95c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.181+0000 7fb96a237740 1 Processor -- start 2024-04-25T08:33:56.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.181+0000 7fb96a237740 1 -- start start 2024-04-25T08:33:56.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.181+0000 7fb96a237740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03f710 0x55e9db041f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.181+0000 7fb96a237740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 0x55e9db042460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.181+0000 7fb96a237740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db012370 0x55e9db015b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.181+0000 7fb96a237740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9daf7abc0 con 0x55e9db03faf0 2024-04-25T08:33:56.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.181+0000 7fb96a237740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9daf23e70 con 0x55e9db03f710 2024-04-25T08:33:56.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.181+0000 7fb96a237740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9daf76c60 con 0x55e9db012370 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.181+0000 7fb9649f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 0x55e9db042460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.181+0000 7fb9649f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 0x55e9db042460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33018/0 (socket says 172.21.15.38:33018) 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.181+0000 7fb9649f3640 1 -- 172.21.15.38:0/2877534500 learned_addr learned my addr 172.21.15.38:0/2877534500 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.182+0000 7fb9651f4640 1 --2- 172.21.15.38:0/2877534500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03f710 0x55e9db041f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.182+0000 7fb9659f5640 1 --2- 172.21.15.38:0/2877534500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db012370 0x55e9db015b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.182+0000 7fb9649f3640 1 -- 172.21.15.38:0/2877534500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db012370 msgr2=0x55e9db015b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.182+0000 7fb9649f3640 1 --2- 172.21.15.38:0/2877534500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db012370 0x55e9db015b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.182+0000 7fb9649f3640 1 -- 172.21.15.38:0/2877534500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03f710 msgr2=0x55e9db041f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.182+0000 7fb9649f3640 1 --2- 172.21.15.38:0/2877534500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03f710 0x55e9db041f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.182+0000 7fb9649f3640 1 -- 172.21.15.38:0/2877534500 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9daf44ad0 con 0x55e9db03faf0 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.182+0000 7fb9649f3640 1 --2- 172.21.15.38:0/2877534500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 0x55e9db042460 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb954075300 tx=0x7fb954078800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c355b85356980c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.182+0000 7fb94affd640 1 -- 172.21.15.38:0/2877534500 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95407de60 con 0x55e9db03faf0 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.182+0000 7fb94affd640 1 -- 172.21.15.38:0/2877534500 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb954081500 con 0x55e9db03faf0 2024-04-25T08:33:56.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.182+0000 7fb94affd640 1 -- 172.21.15.38:0/2877534500 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954030ce0 con 0x55e9db03faf0 2024-04-25T08:33:56.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.183+0000 7fb96a237740 1 -- 172.21.15.38:0/2877534500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 msgr2=0x55e9db042460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.183+0000 7fb96a237740 1 --2- 172.21.15.38:0/2877534500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 0x55e9db042460 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb954075300 tx=0x7fb954078800 comp rx=0 tx=0).stop 2024-04-25T08:33:56.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.183+0000 7fb96a237740 1 -- 172.21.15.38:0/2877534500 shutdown_connections 2024-04-25T08:33:56.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.183+0000 7fb96a237740 1 --2- 172.21.15.38:0/2877534500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db012370 0x55e9db015b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.183+0000 7fb96a237740 1 --2- 172.21.15.38:0/2877534500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 0x55e9db042460 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.183+0000 7fb96a237740 1 --2- 172.21.15.38:0/2877534500 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03f710 0x55e9db041f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.183+0000 7fb96a237740 1 -- 172.21.15.38:0/2877534500 >> 172.21.15.38:0/2877534500 conn(0x55e9db0145e0 msgr2=0x55e9db05cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.183+0000 7fb96a237740 1 -- 172.21.15.38:0/2877534500 shutdown_connections 2024-04-25T08:33:56.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.183+0000 7fb96a237740 1 -- 172.21.15.38:0/2877534500 wait complete. 2024-04-25T08:33:56.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.183+0000 7fb96a237740 1 -- start start 2024-04-25T08:33:56.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.183+0000 7fb96a237740 1 Processor -- start 2024-04-25T08:33:56.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.184+0000 7fb96a237740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db03f710 0x55e9db038c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.184+0000 7fb96a237740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 0x55e9db039150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.184+0000 7fb96a237740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db070010 0x55e9db05e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.184+0000 7fb96a237740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9db017790 con 0x55e9db03faf0 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.184+0000 7fb96a237740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9daf194e0 con 0x55e9db070010 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.184+0000 7fb96a237740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9db017a70 con 0x55e9db03f710 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.184+0000 7fb96a237740 10 client.0 _ll_register_callbacks cb 0x55e9daf36de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb9649f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 0x55e9db039150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb9649f3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 0x55e9db039150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33032/0 (socket says 172.21.15.38:33032) 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb9649f3640 1 -- 172.21.15.38:0/952990151 learned_addr learned my addr 172.21.15.38:0/952990151 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb9651f4640 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db03f710 0x55e9db038c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb9659f5640 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db070010 0x55e9db05e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb9649f3640 1 -- 172.21.15.38:0/952990151 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db03f710 msgr2=0x55e9db038c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb9649f3640 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db03f710 0x55e9db038c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb9649f3640 1 -- 172.21.15.38:0/952990151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db070010 msgr2=0x55e9db05e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb9649f3640 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db070010 0x55e9db05e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb9649f3640 1 -- 172.21.15.38:0/952990151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9dad5b940 con 0x55e9db03faf0 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb9649f3640 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 0x55e9db039150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb9540a8bf0 tx=0x7fb954009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3fee2a175d0294 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb93ffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb93ffff640 1 -- 172.21.15.38:0/952990151 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954095060 con 0x55e9db03faf0 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb93ffff640 1 -- 172.21.15.38:0/952990151 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fb954075510 con 0x55e9db03faf0 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb96a237740 1 -- 172.21.15.38:0/952990151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb954081500 con 0x55e9db03faf0 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb93ffff640 1 -- 172.21.15.38:0/952990151 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9540a6070 con 0x55e9db03faf0 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb96a237740 10 client.5706 fetch_fsmap learned FSMap version 18 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb96a237740 1 -- 172.21.15.38:0/952990151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55e9dadbc0d0 con 0x55e9db03faf0 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.185+0000 7fb93ffff640 1 -- 172.21.15.38:0/952990151 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb954081500 con 0x55e9db03faf0 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.186+0000 7fb93ffff640 1 -- 172.21.15.38:0/952990151 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb95403a030 con 0x55e9db03faf0 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.186+0000 7fb96a237740 10 client.5706 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.186+0000 7fb96a237740 10 client.5706 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.186+0000 7fb96a237740 20 client.5706 populate_metadata read hostname 'smithi038' 2024-04-25T08:33:56.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.186+0000 7fb96a237740 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e9db077760 0x55e9db079b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.186+0000 7fb96a237740 4 client.5706 mds_command: new command op to 5149 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.186+0000 7fb96a237740 1 -- 172.21.15.38:0/952990151 --> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9daf88100 con 0x55e9db077760 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.186+0000 7fb9651f4640 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e9db077760 0x55e9db079b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.186+0000 7fb93ffff640 10 client.5706 ms_handle_connect on v2:172.21.15.38:6832/234860294 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.186+0000 7fb9651f4640 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e9db077760 0x55e9db079b50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fefacf460cb8bc6 server_cookie=d0de22860be3f378 in_seq=0 out_seq=0 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb93ffff640 1 -- 172.21.15.38:0/952990151 <== mds.0 v2:172.21.15.38:6832/234860294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e9daf88100 con 0x55e9db077760 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb93ffff640 10 client.5706 handle_command_reply: tid=0 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb96a237740 1 client.5706 shutdown 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb96a237740 1 -- 172.21.15.38:0/952990151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 msgr2=0x55e9db039150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb96a237740 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 0x55e9db039150 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb9540a8bf0 tx=0x7fb954009190 comp rx=0 tx=0).stop 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb96a237740 1 -- 172.21.15.38:0/952990151 shutdown_connections 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb96a237740 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e9db077760 0x55e9db079b50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb96a237740 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db070010 0x55e9db05e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb93ffff640 0 client.5706 ms_handle_reset on v2:172.21.15.38:6832/234860294 2024-04-25T08:33:56.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb96a237740 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03faf0 0x55e9db039150 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb96a237740 1 --2- 172.21.15.38:0/952990151 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db03f710 0x55e9db038c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb96a237740 1 -- 172.21.15.38:0/952990151 >> 172.21.15.38:0/952990151 conn(0x55e9db012370 msgr2=0x55e9db03eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb96a237740 1 -- 172.21.15.38:0/952990151 shutdown_connections 2024-04-25T08:33:56.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb96a237740 1 -- 172.21.15.38:0/952990151 wait complete. 2024-04-25T08:33:56.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.187+0000 7fb96a237740 20 client.5706 trim_cache size 0 max 16384 2024-04-25T08:33:56.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.202+0000 7fb96a237740 1 Processor -- start 2024-04-25T08:33:56.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.202+0000 7fb96a237740 1 -- start start 2024-04-25T08:33:56.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb96a237740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03f710 0x55e9db03e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb96a237740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db03faf0 0x55e9db016fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb96a237740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db070010 0x55e9db0602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb96a237740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9602004b0 con 0x55e9db070010 2024-04-25T08:33:56.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb96a237740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9601ff630 con 0x55e9db03f710 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb96a237740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9600cab90 con 0x55e9db03faf0 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb9659f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db070010 0x55e9db0602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb9659f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db070010 0x55e9db0602b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33042/0 (socket says 172.21.15.38:33042) 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb9659f5640 1 -- 172.21.15.38:0/3353116271 learned_addr learned my addr 172.21.15.38:0/3353116271 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb9649f3640 1 --2- 172.21.15.38:0/3353116271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db03faf0 0x55e9db016fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb9651f4640 1 --2- 172.21.15.38:0/3353116271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03f710 0x55e9db03e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb9659f5640 1 -- 172.21.15.38:0/3353116271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db03faf0 msgr2=0x55e9db016fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb9659f5640 1 --2- 172.21.15.38:0/3353116271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db03faf0 0x55e9db016fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb9659f5640 1 -- 172.21.15.38:0/3353116271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03f710 msgr2=0x55e9db03e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb9659f5640 1 --2- 172.21.15.38:0/3353116271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03f710 0x55e9db03e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.203+0000 7fb9659f5640 1 -- 172.21.15.38:0/3353116271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9dac4f230 con 0x55e9db070010 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb9659f5640 1 --2- 172.21.15.38:0/3353116271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db070010 0x55e9db0602b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb958015580 tx=0x7fb9580019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65931f17a571b3ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb94a7fc640 1 -- 172.21.15.38:0/3353116271 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb958016750 con 0x55e9db070010 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb94a7fc640 1 -- 172.21.15.38:0/3353116271 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9580168f0 con 0x55e9db070010 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb94a7fc640 1 -- 172.21.15.38:0/3353116271 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb958016cc0 con 0x55e9db070010 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb96a237740 1 -- 172.21.15.38:0/3353116271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db070010 msgr2=0x55e9db0602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb96a237740 1 --2- 172.21.15.38:0/3353116271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db070010 0x55e9db0602b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb958015580 tx=0x7fb9580019f0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb96a237740 1 -- 172.21.15.38:0/3353116271 shutdown_connections 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb96a237740 1 --2- 172.21.15.38:0/3353116271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db070010 0x55e9db0602b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb96a237740 1 --2- 172.21.15.38:0/3353116271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db03faf0 0x55e9db016fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb96a237740 1 --2- 172.21.15.38:0/3353116271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03f710 0x55e9db03e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb96a237740 1 -- 172.21.15.38:0/3353116271 >> 172.21.15.38:0/3353116271 conn(0x55e9db012370 msgr2=0x55e9db045c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb96a237740 1 -- 172.21.15.38:0/3353116271 shutdown_connections 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.204+0000 7fb96a237740 1 -- 172.21.15.38:0/3353116271 wait complete. 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.205+0000 7fb96a237740 1 -- start start 2024-04-25T08:33:56.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.205+0000 7fb96a237740 1 Processor -- start 2024-04-25T08:33:56.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb96a237740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03f710 0x55e9db059be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb96a237740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03faf0 0x55e9db0602b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb96a237740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db070010 0x55e9db0607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb96a237740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9601ff090 con 0x55e9db03f710 2024-04-25T08:33:56.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb96a237740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9db039f80 con 0x55e9db03faf0 2024-04-25T08:33:56.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb96a237740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9db073c40 con 0x55e9db070010 2024-04-25T08:33:56.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb96a237740 10 client.0 _ll_register_callbacks cb 0x7fb95408f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:33:56.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9651f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03f710 0x55e9db059be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9651f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03f710 0x55e9db059be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33044/0 (socket says 172.21.15.38:33044) 2024-04-25T08:33:56.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9651f4640 1 -- 172.21.15.38:0/3077600116 learned_addr learned my addr 172.21.15.38:0/3077600116 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:56.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9649f3640 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03faf0 0x55e9db0602b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9651f4640 1 -- 172.21.15.38:0/3077600116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db070010 msgr2=0x55e9db0607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9659f5640 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db070010 0x55e9db0607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9651f4640 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db070010 0x55e9db0607f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9651f4640 1 -- 172.21.15.38:0/3077600116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03faf0 msgr2=0x55e9db0602b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9651f4640 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03faf0 0x55e9db0602b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9651f4640 1 -- 172.21.15.38:0/3077600116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9db073770 con 0x55e9db03f710 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9649f3640 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03faf0 0x55e9db0602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9659f5640 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db070010 0x55e9db0607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.206+0000 7fb9651f4640 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03f710 0x55e9db059be0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55e9db06f640 tx=0x7fb95c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5f502bc3961db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb93ffff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb93ffff640 1 -- 172.21.15.38:0/3077600116 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95c007030 con 0x55e9db03f710 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb93ffff640 1 -- 172.21.15.38:0/3077600116 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fb95c0071d0 con 0x55e9db03f710 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb96a237740 1 -- 172.21.15.38:0/3077600116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9580168f0 con 0x55e9db03f710 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb93ffff640 1 -- 172.21.15.38:0/3077600116 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb95c037050 con 0x55e9db03f710 2024-04-25T08:33:56.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb96a237740 10 client.5712 fetch_fsmap learned FSMap version 18 2024-04-25T08:33:56.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb96a237740 1 -- 172.21.15.38:0/3077600116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55e9db045590 con 0x55e9db03f710 2024-04-25T08:33:56.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb93ffff640 1 -- 172.21.15.38:0/3077600116 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9580168f0 con 0x55e9db03f710 2024-04-25T08:33:56.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb93ffff640 1 -- 172.21.15.38:0/3077600116 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb95c03a020 con 0x55e9db03f710 2024-04-25T08:33:56.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb96a237740 10 client.5712 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T08:33:56.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb96a237740 10 client.5712 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:33:56.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb96a237740 20 client.5712 populate_metadata read hostname 'smithi038' 2024-04-25T08:33:56.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb96a237740 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e9db0c77e0 0x55e9db0361f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb96a237740 4 client.5712 mds_command: new command op to 5149 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-25T08:33:56.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb96a237740 1 -- 172.21.15.38:0/3077600116 --> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x55e9daf82450 con 0x55e9db0c77e0 2024-04-25T08:33:56.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.207+0000 7fb9649f3640 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e9db0c77e0 0x55e9db0361f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.208+0000 7fb93ffff640 10 client.5712 ms_handle_connect on v2:172.21.15.38:6832/234860294 2024-04-25T08:33:56.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.208+0000 7fb9649f3640 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e9db0c77e0 0x55e9db0361f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c56886a68d217a9c server_cookie=baaffd5905869369 in_seq=0 out_seq=0 2024-04-25T08:33:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.225+0000 7fb93ffff640 1 -- 172.21.15.38:0/3077600116 <== mds.0 v2:172.21.15.38:6832/234860294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e9daf82450 con 0x55e9db0c77e0 2024-04-25T08:33:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.225+0000 7fb93ffff640 10 client.5712 handle_command_reply: tid=0 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.225+0000 7fb96a237740 1 client.5712 shutdown 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.225+0000 7fb96a237740 1 -- 172.21.15.38:0/3077600116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03f710 msgr2=0x55e9db059be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.225+0000 7fb96a237740 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03f710 0x55e9db059be0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x55e9db06f640 tx=0x7fb95c000ed0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.226+0000 7fb96a237740 1 -- 172.21.15.38:0/3077600116 shutdown_connections 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.226+0000 7fb96a237740 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e9db0c77e0 0x55e9db0361f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.226+0000 7fb96a237740 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9db070010 0x55e9db0607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.226+0000 7fb96a237740 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9db03faf0 0x55e9db0602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.226+0000 7fb96a237740 1 --2- 172.21.15.38:0/3077600116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9db03f710 0x55e9db059be0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.226+0000 7fb96a237740 1 -- 172.21.15.38:0/3077600116 >> 172.21.15.38:0/3077600116 conn(0x55e9db012370 msgr2=0x55e9db05cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.226+0000 7fb93ffff640 0 client.5712 ms_handle_reset on v2:172.21.15.38:6832/234860294 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.226+0000 7fb96a237740 1 -- 172.21.15.38:0/3077600116 shutdown_connections 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.226+0000 7fb96a237740 1 -- 172.21.15.38:0/3077600116 wait complete. 2024-04-25T08:33:56.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.226+0000 7fb96a237740 20 client.5712 trim_cache size 0 max 16384 2024-04-25T08:33:56.224 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:33:56.224 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:33:56.224 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "a6c2efe4-d678-4a89-9c74-b3b55c81d8ed", 2024-04-25T08:33:56.224 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:33:56.224 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:33:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.227+0000 7fb967458640 1 -- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb94406b4d0 msgr2=0x7fb94406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.227+0000 7fb967458640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb94406b4d0 0x7fb94406d990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb95c001780 tx=0x7fb95c009040 comp rx=0 tx=0).stop 2024-04-25T08:33:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.227+0000 7fb967458640 1 -- 172.21.15.38:0/3274867409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9600cf980 msgr2=0x7fb96006d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.227+0000 7fb967458640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9600cf980 0x7fb96006d3f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb954035040 tx=0x7fb9540046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.227+0000 7fb967458640 1 -- 172.21.15.38:0/3274867409 shutdown_connections 2024-04-25T08:33:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.227+0000 7fb967458640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb96015bd00 0x7fb96006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.227+0000 7fb967458640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9600cf980 0x7fb96006d3f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.227+0000 7fb967458640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb94406b4d0 0x7fb94406d990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.227+0000 7fb967458640 1 --2- 172.21.15.38:0/3274867409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb9600ccca0 0x7fb960155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.227+0000 7fb967458640 1 -- 172.21.15.38:0/3274867409 >> 172.21.15.38:0/3274867409 conn(0x7fb9600b31b0 msgr2=0x7fb96015e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.227+0000 7fb967458640 1 -- 172.21.15.38:0/3274867409 shutdown_connections 2024-04-25T08:33:56.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.227+0000 7fb967458640 1 -- 172.21.15.38:0/3274867409 wait complete. 2024-04-25T08:33:56.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:33:56.234 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T08:33:56.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.348+0000 7efc98f9e640 1 Processor -- start 2024-04-25T08:33:56.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.348+0000 7efc98f9e640 1 -- start start 2024-04-25T08:33:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.349+0000 7efc98f9e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7efc940ccfa0 0x7efc940cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.349+0000 7efc98f9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940ae050 0x7efc940ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.349+0000 7efc98f9e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc940cd8c0 con 0x7efc940cdc90 2024-04-25T08:33:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.349+0000 7efc98f9e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc940cc4b0 con 0x7efc940ccfa0 2024-04-25T08:33:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.349+0000 7efc98f9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc940ad570 con 0x7efc940ae050 2024-04-25T08:33:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.350+0000 7efc9259c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7efc940ccfa0 0x7efc940cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.350+0000 7efc91d9b640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7efc940cdc90 0x7efc940cbc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:37532/0 (socket says 172.21.15.38:37532) 2024-04-25T08:33:56.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.350+0000 7efc91d9b640 1 -- 172.21.15.38:0/2074011196 learned_addr learned my addr 172.21.15.38:0/2074011196 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:33:56.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.350+0000 7efc92d9d640 1 --2- 172.21.15.38:0/2074011196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940ae050 0x7efc940ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.350+0000 7efc92d9d640 1 -- 172.21.15.38:0/2074011196 >> v2:172.21.15.107:3300/0 conn(0x7efc940ccfa0 msgr2=0x7efc940cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.350+0000 7efc92d9d640 1 --2- 172.21.15.38:0/2074011196 >> v2:172.21.15.107:3300/0 conn(0x7efc940ccfa0 0x7efc940cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.350+0000 7efc92d9d640 1 -- 172.21.15.38:0/2074011196 >> v1:172.21.15.107:6789/0 conn(0x7efc940cdc90 legacy=0x7efc940cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.350+0000 7efc92d9d640 1 -- 172.21.15.38:0/2074011196 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc940ad850 con 0x7efc940ae050 2024-04-25T08:33:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.350+0000 7efc9259c640 1 --2- 172.21.15.38:0/2074011196 >> v2:172.21.15.107:3300/0 conn(0x7efc940ccfa0 0x7efc940cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.351+0000 7efc92d9d640 1 --2- 172.21.15.38:0/2074011196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940ae050 0x7efc940ae4e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009d90 tx=0x7efc8c02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=278293f73d76d639 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.351+0000 7efc9159a640 1 -- 172.21.15.38:0/2074011196 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c007740 con 0x7efc940ae050 2024-04-25T08:33:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.352+0000 7efc9159a640 1 -- 172.21.15.38:0/2074011196 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c0078e0 con 0x7efc940ae050 2024-04-25T08:33:56.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.352+0000 7efc9159a640 1 -- 172.21.15.38:0/2074011196 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c007cb0 con 0x7efc940ae050 2024-04-25T08:33:56.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.352+0000 7efc98f9e640 1 -- 172.21.15.38:0/2074011196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940ae050 msgr2=0x7efc940ae4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.352+0000 7efc98f9e640 1 --2- 172.21.15.38:0/2074011196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940ae050 0x7efc940ae4e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009d90 tx=0x7efc8c02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.353+0000 7efc98f9e640 1 -- 172.21.15.38:0/2074011196 shutdown_connections 2024-04-25T08:33:56.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.353+0000 7efc98f9e640 1 --2- 172.21.15.38:0/2074011196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940ae050 0x7efc940ae4e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.353+0000 7efc98f9e640 1 --2- 172.21.15.38:0/2074011196 >> v2:172.21.15.107:3300/0 conn(0x7efc940ccfa0 0x7efc940cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.353+0000 7efc98f9e640 1 -- 172.21.15.38:0/2074011196 >> 172.21.15.38:0/2074011196 conn(0x7efc940d06b0 msgr2=0x7efc940d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.353+0000 7efc98f9e640 1 -- 172.21.15.38:0/2074011196 shutdown_connections 2024-04-25T08:33:56.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.353+0000 7efc98f9e640 1 -- 172.21.15.38:0/2074011196 wait complete. 2024-04-25T08:33:56.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.355+0000 7efc98f9e640 1 Processor -- start 2024-04-25T08:33:56.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.355+0000 7efc98f9e640 1 -- start start 2024-04-25T08:33:56.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc98f9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc940ae050 0x7efc940b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc98f9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940ccfa0 0x7efc940b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc98f9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc940cdc90 0x7efc940b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc98f9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc940aea20 con 0x7efc940cdc90 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc98f9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9415ad80 con 0x7efc940ae050 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc98f9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc940aeba0 con 0x7efc940ccfa0 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc92d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc940cdc90 0x7efc940b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc91d9b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940ccfa0 0x7efc940b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc92d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc940cdc90 0x7efc940b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33048/0 (socket says 172.21.15.38:33048) 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc92d9d640 1 -- 172.21.15.38:0/1456053333 learned_addr learned my addr 172.21.15.38:0/1456053333 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc9259c640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc940ae050 0x7efc940b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc92d9d640 1 -- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940ccfa0 msgr2=0x7efc940b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc92d9d640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940ccfa0 0x7efc940b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc92d9d640 1 -- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc940ae050 msgr2=0x7efc940b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc92d9d640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc940ae050 0x7efc940b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc92d9d640 1 -- 172.21.15.38:0/1456053333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc94151ed0 con 0x7efc940cdc90 2024-04-25T08:33:56.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.356+0000 7efc91d9b640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940ccfa0 0x7efc940b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:56.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.357+0000 7efc92d9d640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc940cdc90 0x7efc940b5e20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009eb0 tx=0x7efc8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.357+0000 7efc82ffd640 1 -- 172.21.15.38:0/1456053333 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c040070 con 0x7efc940cdc90 2024-04-25T08:33:56.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.357+0000 7efc82ffd640 1 -- 172.21.15.38:0/1456053333 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c031bc0 con 0x7efc940cdc90 2024-04-25T08:33:56.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.357+0000 7efc98f9e640 1 -- 172.21.15.38:0/1456053333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc941521b0 con 0x7efc940cdc90 2024-04-25T08:33:56.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.357+0000 7efc82ffd640 1 -- 172.21.15.38:0/1456053333 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c009450 con 0x7efc940cdc90 2024-04-25T08:33:56.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.357+0000 7efc98f9e640 1 -- 172.21.15.38:0/1456053333 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc94152770 con 0x7efc940cdc90 2024-04-25T08:33:56.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.359+0000 7efc82ffd640 1 -- 172.21.15.38:0/1456053333 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc8c039040 con 0x7efc940cdc90 2024-04-25T08:33:56.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.360+0000 7efc82ffd640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efc7406b4d0 0x7efc7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.360+0000 7efc82ffd640 1 -- 172.21.15.38:0/1456053333 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7efc8c0aba70 con 0x7efc940cdc90 2024-04-25T08:33:56.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.360+0000 7efc9259c640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efc7406b4d0 0x7efc7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.361+0000 7efc9bd7d740 1 Processor -- start 2024-04-25T08:33:56.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.361+0000 7efc9bd7d740 1 -- start start 2024-04-25T08:33:56.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.361+0000 7efc9259c640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efc7406b4d0 0x7efc7406d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efc84009560 tx=0x7efc84009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc9bd7d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 0x55e31fdcc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc9bd7d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9ef0 0x55e31fdcc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc9bd7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fd9c770 0x55e31fd9ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc9bd7d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31fd05da0 con 0x55e31fdc9b10 2024-04-25T08:33:56.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc9bd7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31fcaeb80 con 0x55e31fd9c770 2024-04-25T08:33:56.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc9bd7d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e31fd01e40 con 0x55e31fdc9ef0 2024-04-25T08:33:56.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc9259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 0x55e31fdcc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc9259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 0x55e31fdcc320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33056/0 (socket says 172.21.15.38:33056) 2024-04-25T08:33:56.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc9259c640 1 -- 172.21.15.38:0/1717120279 learned_addr learned my addr 172.21.15.38:0/1717120279 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:56.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc91d9b640 1 --2- 172.21.15.38:0/1717120279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9ef0 0x55e31fdcc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc92d9d640 1 --2- 172.21.15.38:0/1717120279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fd9c770 0x55e31fd9ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc91d9b640 1 -- 172.21.15.38:0/1717120279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fd9c770 msgr2=0x55e31fd9ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc91d9b640 1 --2- 172.21.15.38:0/1717120279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fd9c770 0x55e31fd9ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc91d9b640 1 -- 172.21.15.38:0/1717120279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 msgr2=0x55e31fdcc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc91d9b640 1 --2- 172.21.15.38:0/1717120279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 0x55e31fdcc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.362+0000 7efc91d9b640 1 -- 172.21.15.38:0/1717120279 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e31fcb8f10 con 0x55e31fdc9ef0 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.363+0000 7efc9259c640 1 --2- 172.21.15.38:0/1717120279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 0x55e31fdcc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.363+0000 7efc92d9d640 1 --2- 172.21.15.38:0/1717120279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fd9c770 0x55e31fd9ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.363+0000 7efc91d9b640 1 --2- 172.21.15.38:0/1717120279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9ef0 0x55e31fdcc860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efc94067d40 tx=0x7efc88007430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a54c7bcd6b78a8a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.363+0000 7efc73fff640 1 -- 172.21.15.38:0/1717120279 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc88001b20 con 0x55e31fdc9ef0 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.363+0000 7efc73fff640 1 -- 172.21.15.38:0/1717120279 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc88018070 con 0x55e31fdc9ef0 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.363+0000 7efc73fff640 1 -- 172.21.15.38:0/1717120279 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc880133f0 con 0x55e31fdc9ef0 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.364+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1717120279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9ef0 msgr2=0x55e31fdcc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.364+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1717120279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9ef0 0x55e31fdcc860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efc94067d40 tx=0x7efc88007430 comp rx=0 tx=0).stop 2024-04-25T08:33:56.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.364+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1717120279 shutdown_connections 2024-04-25T08:33:56.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.364+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1717120279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fd9c770 0x55e31fd9ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.364+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1717120279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9ef0 0x55e31fdcc860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.364+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1717120279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 0x55e31fdcc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.364+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1717120279 >> 172.21.15.38:0/1717120279 conn(0x55e31fd9e9e0 msgr2=0x55e31fde7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.364+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1717120279 shutdown_connections 2024-04-25T08:33:56.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.364+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1717120279 wait complete. 2024-04-25T08:33:56.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.365+0000 7efc9bd7d740 1 -- start start 2024-04-25T08:33:56.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.365+0000 7efc9bd7d740 1 Processor -- start 2024-04-25T08:33:56.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9bd7d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 0x55e31fdea070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9bd7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9ef0 0x55e31fdcd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9bd7d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdcf870 0x55e31fdb3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 0x55e31fdea070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 0x55e31fdea070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33072/0 (socket says 172.21.15.38:33072) 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9259c640 1 -- 172.21.15.38:0/3333620728 learned_addr learned my addr 172.21.15.38:0/3333620728 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9bd7d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31fda1b90 con 0x55e31fdc9b10 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9bd7d740 1 -- 172.21.15.38:0/3333620728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31fca44b0 con 0x55e31fdc9ef0 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9bd7d740 1 -- 172.21.15.38:0/3333620728 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e31fda1e70 con 0x55e31fdcf870 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9bd7d740 10 client.0 _ll_register_callbacks cb 0x55e31fce4180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc91d9b640 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9ef0 0x55e31fdcd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc92d9d640 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdcf870 0x55e31fdb3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9259c640 1 -- 172.21.15.38:0/3333620728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdcf870 msgr2=0x55e31fdb3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9259c640 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdcf870 0x55e31fdb3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9259c640 1 -- 172.21.15.38:0/3333620728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9ef0 msgr2=0x55e31fdcd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9259c640 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9ef0 0x55e31fdcd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc9259c640 1 -- 172.21.15.38:0/3333620728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e31fae47d0 con 0x55e31fdc9b10 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.367+0000 7efc91d9b640 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9ef0 0x55e31fdcd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.368+0000 7efc9259c640 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 0x55e31fdea070 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efc84003890 tx=0x7efc840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f23302101d72e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.368+0000 7efc717fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.368+0000 7efc717fa640 1 -- 172.21.15.38:0/3333620728 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84027070 con 0x55e31fdc9b10 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.368+0000 7efc717fa640 1 -- 172.21.15.38:0/3333620728 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7efc84014650 con 0x55e31fdc9b10 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.368+0000 7efc9bd7d740 1 -- 172.21.15.38:0/3333620728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc88018070 con 0x55e31fdc9b10 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.368+0000 7efc717fa640 1 -- 172.21.15.38:0/3333620728 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc84032050 con 0x55e31fdc9b10 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.368+0000 7efc9bd7d740 10 client.5721 fetch_fsmap learned FSMap version 18 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.368+0000 7efc9bd7d740 1 -- 172.21.15.38:0/3333620728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55e31fa1a3f0 con 0x55e31fdc9b10 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.369+0000 7efc717fa640 1 -- 172.21.15.38:0/3333620728 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc88018070 con 0x55e31fdc9b10 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.369+0000 7efc717fa640 1 -- 172.21.15.38:0/3333620728 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efc84035020 con 0x55e31fdc9b10 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.369+0000 7efc9bd7d740 10 client.5721 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.369+0000 7efc9bd7d740 10 client.5721 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.369+0000 7efc9bd7d740 20 client.5721 populate_metadata read hostname 'smithi038' 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.369+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e31fdb5e20 0x55e31fdb6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.369+0000 7efc9bd7d740 4 client.5721 mds_command: new command op to 5149 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:33:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.369+0000 7efc9bd7d740 1 -- 172.21.15.38:0/3333620728 --> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e31fd132e0 con 0x55e31fdb5e20 2024-04-25T08:33:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.369+0000 7efc91d9b640 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e31fdb5e20 0x55e31fdb6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.370+0000 7efc717fa640 10 client.5721 ms_handle_connect on v2:172.21.15.38:6832/234860294 2024-04-25T08:33:56.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.370+0000 7efc91d9b640 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e31fdb5e20 0x55e31fdb6240 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a4bb7dc98fb0293 server_cookie=47f3218eb674ef17 in_seq=0 out_seq=0 2024-04-25T08:33:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc717fa640 1 -- 172.21.15.38:0/3333620728 <== mds.0 v2:172.21.15.38:6832/234860294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e31fd132e0 con 0x55e31fdb5e20 2024-04-25T08:33:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc717fa640 10 client.5721 handle_command_reply: tid=0 2024-04-25T08:33:56.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc9bd7d740 1 client.5721 shutdown 2024-04-25T08:33:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc9bd7d740 1 -- 172.21.15.38:0/3333620728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 msgr2=0x55e31fdea070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 0x55e31fdea070 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efc84003890 tx=0x7efc840046c0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc9bd7d740 1 -- 172.21.15.38:0/3333620728 shutdown_connections 2024-04-25T08:33:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e31fdb5e20 0x55e31fdb6240 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc717fa640 0 client.5721 ms_handle_reset on v2:172.21.15.38:6832/234860294 2024-04-25T08:33:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdcf870 0x55e31fdb3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9ef0 0x55e31fdcd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/3333620728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9b10 0x55e31fdea070 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc9bd7d740 1 -- 172.21.15.38:0/3333620728 >> 172.21.15.38:0/3333620728 conn(0x55e31fd9c770 msgr2=0x55e31fdcc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc9bd7d740 1 -- 172.21.15.38:0/3333620728 shutdown_connections 2024-04-25T08:33:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.371+0000 7efc9bd7d740 1 -- 172.21.15.38:0/3333620728 wait complete. 2024-04-25T08:33:56.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.372+0000 7efc9bd7d740 20 client.5721 trim_cache size 0 max 16384 2024-04-25T08:33:56.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.397+0000 7efc9bd7d740 1 Processor -- start 2024-04-25T08:33:56.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.397+0000 7efc9bd7d740 1 -- start start 2024-04-25T08:33:56.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc9bd7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9b10 0x55e31fd97b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc9bd7d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9ef0 0x55e31fdc8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc9bd7d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdcf870 0x55e31fde64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc9bd7d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9414fe90 con 0x55e31fdcf870 2024-04-25T08:33:56.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc9bd7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc94167eb0 con 0x55e31fdc9b10 2024-04-25T08:33:56.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc9bd7d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc941672b0 con 0x55e31fdc9ef0 2024-04-25T08:33:56.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc9259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9b10 0x55e31fd97b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc92d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdcf870 0x55e31fde64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc9259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9b10 0x55e31fd97b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:51194/0 (socket says 172.21.15.38:51194) 2024-04-25T08:33:56.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc9259c640 1 -- 172.21.15.38:0/1814614734 learned_addr learned my addr 172.21.15.38:0/1814614734 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:56.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc91d9b640 1 --2- 172.21.15.38:0/1814614734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9ef0 0x55e31fdc8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc9259c640 1 -- 172.21.15.38:0/1814614734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9ef0 msgr2=0x55e31fdc8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.398+0000 7efc9259c640 1 --2- 172.21.15.38:0/1814614734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9ef0 0x55e31fdc8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.399+0000 7efc9259c640 1 -- 172.21.15.38:0/1814614734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdcf870 msgr2=0x55e31fde64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.399+0000 7efc9259c640 1 --2- 172.21.15.38:0/1814614734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdcf870 0x55e31fde64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.399+0000 7efc9259c640 1 -- 172.21.15.38:0/1814614734 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e31f9da0f0 con 0x55e31fdc9b10 2024-04-25T08:33:56.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.399+0000 7efc9259c640 1 --2- 172.21.15.38:0/1814614734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9b10 0x55e31fd97b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efc84011730 tx=0x7efc84024000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=624280d2ef26a161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.399+0000 7efc737fe640 1 -- 172.21.15.38:0/1814614734 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84027050 con 0x55e31fdc9b10 2024-04-25T08:33:56.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.399+0000 7efc737fe640 1 -- 172.21.15.38:0/1814614734 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc84014d50 con 0x55e31fdc9b10 2024-04-25T08:33:56.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.399+0000 7efc737fe640 1 -- 172.21.15.38:0/1814614734 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc84014850 con 0x55e31fdc9b10 2024-04-25T08:33:56.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.400+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1814614734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9b10 msgr2=0x55e31fd97b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.400+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1814614734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9b10 0x55e31fd97b80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efc84011730 tx=0x7efc84024000 comp rx=0 tx=0).stop 2024-04-25T08:33:56.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.400+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1814614734 shutdown_connections 2024-04-25T08:33:56.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.400+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1814614734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdcf870 0x55e31fde64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.400+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1814614734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9ef0 0x55e31fdc8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.400+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1814614734 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdc9b10 0x55e31fd97b80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.400+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1814614734 >> 172.21.15.38:0/1814614734 conn(0x55e31fd9c770 msgr2=0x55e31fdcd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.400+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1814614734 shutdown_connections 2024-04-25T08:33:56.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.400+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1814614734 wait complete. 2024-04-25T08:33:56.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.401+0000 7efc9bd7d740 1 -- start start 2024-04-25T08:33:56.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.401+0000 7efc9bd7d740 1 Processor -- start 2024-04-25T08:33:56.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc9bd7d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9b10 0x55e31fd9c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc9bd7d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9ef0 0x55e31fe524f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc9bd7d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdcf870 0x55e31fd9d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc9bd7d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31fdcf2c0 con 0x55e31fdc9ef0 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc9bd7d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc941678b0 con 0x55e31fdcf870 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc9bd7d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e31fdcc530 con 0x55e31fdc9b10 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc9bd7d740 10 client.0 _ll_register_callbacks cb 0x7efc88004900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc91d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9ef0 0x55e31fe524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc91d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9ef0 0x55e31fe524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33092/0 (socket says 172.21.15.38:33092) 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc91d9b640 1 -- 172.21.15.38:0/1024582821 learned_addr learned my addr 172.21.15.38:0/1024582821 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc9259c640 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9b10 0x55e31fd9c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc92d9d640 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdcf870 0x55e31fd9d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc92d9d640 1 -- 172.21.15.38:0/1024582821 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9b10 msgr2=0x55e31fd9c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc92d9d640 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9b10 0x55e31fd9c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc92d9d640 1 -- 172.21.15.38:0/1024582821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9ef0 msgr2=0x55e31fe524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc92d9d640 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9ef0 0x55e31fe524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc92d9d640 1 -- 172.21.15.38:0/1024582821 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e31fdddc80 con 0x55e31fdcf870 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc91d9b640 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9ef0 0x55e31fe524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.403+0000 7efc9259c640 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9b10 0x55e31fd9c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.404+0000 7efc92d9d640 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdcf870 0x55e31fd9d8a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0a9e50 tx=0x7efc8c02abf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a68e5fdb0058d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.404+0000 7efc717fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.404+0000 7efc717fa640 1 -- 172.21.15.38:0/1024582821 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc8c091e60 con 0x55e31fdcf870 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.404+0000 7efc717fa640 1 -- 172.21.15.38:0/1024582821 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7efc8c082550 con 0x55e31fdcf870 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.404+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1024582821 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efc84014d50 con 0x55e31fdcf870 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.404+0000 7efc717fa640 1 -- 172.21.15.38:0/1024582821 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc8c0a7070 con 0x55e31fdcf870 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.404+0000 7efc9bd7d740 10 client.5338 fetch_fsmap learned FSMap version 18 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.404+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1024582821 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55e31fe55a70 con 0x55e31fdcf870 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.404+0000 7efc717fa640 1 -- 172.21.15.38:0/1024582821 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc84014d50 con 0x55e31fdcf870 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.405+0000 7efc717fa640 1 -- 172.21.15.38:0/1024582821 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efc8c04d030 con 0x55e31fdcf870 2024-04-25T08:33:56.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.405+0000 7efc9bd7d740 10 client.5338 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T08:33:56.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.405+0000 7efc9bd7d740 10 client.5338 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:33:56.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.405+0000 7efc9bd7d740 20 client.5338 populate_metadata read hostname 'smithi038' 2024-04-25T08:33:56.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.405+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e31fdcfef0 0x55e31fdd0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:33:56.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.405+0000 7efc9bd7d740 4 client.5338 mds_command: new command op to 5149 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:33:56.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.405+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1024582821 --> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e31fd0d630 con 0x55e31fdcfef0 2024-04-25T08:33:56.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.405+0000 7efc9259c640 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e31fdcfef0 0x55e31fdd0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:33:56.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.405+0000 7efc717fa640 10 client.5338 ms_handle_connect on v2:172.21.15.38:6832/234860294 2024-04-25T08:33:56.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.405+0000 7efc9259c640 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e31fdcfef0 0x55e31fdd0300 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c2bd7a2f4071971 server_cookie=fc4da164280ae97d in_seq=0 out_seq=0 2024-04-25T08:33:56.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.406+0000 7efc717fa640 1 -- 172.21.15.38:0/1024582821 <== mds.0 v2:172.21.15.38:6832/234860294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+275 (crc 0 0 0) 0x55e31fd0d630 con 0x55e31fdcfef0 2024-04-25T08:33:56.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.406+0000 7efc717fa640 10 client.5338 handle_command_reply: tid=0 2024-04-25T08:33:56.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.406+0000 7efc9bd7d740 1 client.5338 shutdown 2024-04-25T08:33:56.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.406+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1024582821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdcf870 msgr2=0x55e31fd9d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.406+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdcf870 0x55e31fd9d8a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0a9e50 tx=0x7efc8c02abf0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.406+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1024582821 shutdown_connections 2024-04-25T08:33:56.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.406+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55e31fdcfef0 0x55e31fdd0300 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.406+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e31fdcf870 0x55e31fd9d8a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.406+0000 7efc717fa640 0 client.5338 ms_handle_reset on v2:172.21.15.38:6832/234860294 2024-04-25T08:33:56.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.406+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e31fdc9ef0 0x55e31fe524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.406+0000 7efc9bd7d740 1 --2- 172.21.15.38:0/1024582821 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e31fdc9b10 0x55e31fd9c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.406+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1024582821 >> 172.21.15.38:0/1024582821 conn(0x55e31fd9c770 msgr2=0x55e31fde8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.407+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1024582821 shutdown_connections 2024-04-25T08:33:56.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.407+0000 7efc9bd7d740 1 -- 172.21.15.38:0/1024582821 wait complete. 2024-04-25T08:33:56.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.407+0000 7efc9bd7d740 20 client.5338 trim_cache size 0 max 16384 2024-04-25T08:33:56.407 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:33:56.407 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "scrub active (1264 inodes in the stack)", 2024-04-25T08:33:56.407 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-04-25T08:33:56.407 INFO:teuthology.orchestra.run.smithi038.stdout: "a6c2efe4-d678-4a89-9c74-b3b55c81d8ed": { 2024-04-25T08:33:56.407 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-04-25T08:33:56.407 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "a6c2efe4-d678-4a89-9c74-b3b55c81d8ed", 2024-04-25T08:33:56.407 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,scrub_mdsdir" 2024-04-25T08:33:56.407 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:33:56.408 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:33:56.408 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:33:56.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.408+0000 7efc98f9e640 1 -- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efc7406b4d0 msgr2=0x7efc7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.408+0000 7efc98f9e640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efc7406b4d0 0x7efc7406d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efc84009560 tx=0x7efc84009190 comp rx=0 tx=0).stop 2024-04-25T08:33:56.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.408+0000 7efc98f9e640 1 -- 172.21.15.38:0/1456053333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc940cdc90 msgr2=0x7efc940b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:33:56.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.408+0000 7efc98f9e640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc940cdc90 0x7efc940b5e20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009eb0 tx=0x7efc8c004a40 comp rx=0 tx=0).stop 2024-04-25T08:33:56.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.408+0000 7efc98f9e640 1 -- 172.21.15.38:0/1456053333 shutdown_connections 2024-04-25T08:33:56.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.408+0000 7efc98f9e640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc940cdc90 0x7efc940b5e20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.408+0000 7efc98f9e640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efc7406b4d0 0x7efc7406d990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.408+0000 7efc98f9e640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efc940ccfa0 0x7efc940b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.408+0000 7efc98f9e640 1 --2- 172.21.15.38:0/1456053333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efc940ae050 0x7efc940b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:33:56.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.408+0000 7efc98f9e640 1 -- 172.21.15.38:0/1456053333 >> 172.21.15.38:0/1456053333 conn(0x7efc940d06b0 msgr2=0x7efc941546c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:33:56.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.408+0000 7efc98f9e640 1 -- 172.21.15.38:0/1456053333 shutdown_connections 2024-04-25T08:33:56.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:33:56.408+0000 7efc98f9e640 1 -- 172.21.15.38:0/1456053333 wait complete. 2024-04-25T08:33:56.418 INFO:tasks.cephfs.filesystem:scrub status for tag:a6c2efe4-d678-4a89-9c74-b3b55c81d8ed - {'path': '/', 'tag': 'a6c2efe4-d678-4a89-9c74-b3b55c81d8ed', 'options': 'recursive,scrub_mdsdir'} 2024-04-25T08:34:02.762 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:34:02.765 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:34:26.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:26.420 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T08:34:26.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.540+0000 7fa989734640 1 Processor -- start 2024-04-25T08:34:26.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.541+0000 7fa989734640 1 -- start start 2024-04-25T08:34:26.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.541+0000 7fa989734640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa98415df30 0x7fa98409a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.541+0000 7fa989734640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa984162390 0x7fa984099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.541+0000 7fa989734640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa984099fb0 0x7fa98406bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.541+0000 7fa989734640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98415e310 con 0x7fa984099fb0 2024-04-25T08:34:26.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.541+0000 7fa989734640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa98409aeb0 con 0x7fa984162390 2024-04-25T08:34:26.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.541+0000 7fa989734640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98415ee70 con 0x7fa98415df30 2024-04-25T08:34:26.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.541+0000 7fa9837fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa984099fb0 0x7fa98406bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.542+0000 7fa9837fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa984099fb0 0x7fa98406bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35444/0 (socket says 172.21.15.38:35444) 2024-04-25T08:34:26.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.542+0000 7fa9837fe640 1 -- 172.21.15.38:0/1011362453 learned_addr learned my addr 172.21.15.38:0/1011362453 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:26.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.542+0000 7fa982ffd640 1 --2- 172.21.15.38:0/1011362453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa98415df30 0x7fa98409a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.542+0000 7fa9827fc640 1 --2- 172.21.15.38:0/1011362453 >> v2:172.21.15.107:3300/0 conn(0x7fa984162390 0x7fa984099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.542+0000 7fa9837fe640 1 -- 172.21.15.38:0/1011362453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa98415df30 msgr2=0x7fa98409a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.542+0000 7fa9837fe640 1 --2- 172.21.15.38:0/1011362453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa98415df30 0x7fa98409a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.542+0000 7fa9837fe640 1 -- 172.21.15.38:0/1011362453 >> v2:172.21.15.107:3300/0 conn(0x7fa984162390 msgr2=0x7fa984099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.542+0000 7fa9837fe640 1 --2- 172.21.15.38:0/1011362453 >> v2:172.21.15.107:3300/0 conn(0x7fa984162390 0x7fa984099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.542+0000 7fa9837fe640 1 -- 172.21.15.38:0/1011362453 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98415f300 con 0x7fa984099fb0 2024-04-25T08:34:26.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.542+0000 7fa9827fc640 1 --2- 172.21.15.38:0/1011362453 >> v2:172.21.15.107:3300/0 conn(0x7fa984162390 0x7fa984099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:26.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa9837fe640 1 --2- 172.21.15.38:0/1011362453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa984099fb0 0x7fa98406bf90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa984072cd0 tx=0x7fa97402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ea2410a6254396 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa981ffb640 1 -- 172.21.15.38:0/1011362453 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974036070 con 0x7fa984099fb0 2024-04-25T08:34:26.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa981ffb640 1 -- 172.21.15.38:0/1011362453 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa974031600 con 0x7fa984099fb0 2024-04-25T08:34:26.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa981ffb640 1 -- 172.21.15.38:0/1011362453 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9740319d0 con 0x7fa984099fb0 2024-04-25T08:34:26.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa989734640 1 -- 172.21.15.38:0/1011362453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa984099fb0 msgr2=0x7fa98406bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa989734640 1 --2- 172.21.15.38:0/1011362453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa984099fb0 0x7fa98406bf90 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa984072cd0 tx=0x7fa97402a6a0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa989734640 1 -- 172.21.15.38:0/1011362453 shutdown_connections 2024-04-25T08:34:26.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa989734640 1 --2- 172.21.15.38:0/1011362453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa984099fb0 0x7fa98406bf90 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa989734640 1 --2- 172.21.15.38:0/1011362453 >> v2:172.21.15.107:3300/0 conn(0x7fa984162390 0x7fa984099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa989734640 1 --2- 172.21.15.38:0/1011362453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa98415df30 0x7fa98409a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa989734640 1 -- 172.21.15.38:0/1011362453 >> 172.21.15.38:0/1011362453 conn(0x7fa9840b3110 msgr2=0x7fa9840b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:26.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa989734640 1 -- 172.21.15.38:0/1011362453 shutdown_connections 2024-04-25T08:34:26.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.543+0000 7fa989734640 1 -- 172.21.15.38:0/1011362453 wait complete. 2024-04-25T08:34:26.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.545+0000 7fa989734640 1 Processor -- start 2024-04-25T08:34:26.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.545+0000 7fa989734640 1 -- start start 2024-04-25T08:34:26.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa989734640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa984099fb0 0x7fa9840b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa989734640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98415df30 0x7fa984156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa989734640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa984162390 0x7fa9840c9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa989734640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98415e820 con 0x7fa98415df30 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa989734640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98415e540 con 0x7fa984162390 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa989734640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98415eb00 con 0x7fa984099fb0 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa9827fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98415df30 0x7fa984156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa9827fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98415df30 0x7fa984156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35456/0 (socket says 172.21.15.38:35456) 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa9827fc640 1 -- 172.21.15.38:0/4006746846 learned_addr learned my addr 172.21.15.38:0/4006746846 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa982ffd640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa984099fb0 0x7fa9840b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa9837fe640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa984162390 0x7fa9840c9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa9827fc640 1 -- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa984099fb0 msgr2=0x7fa9840b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa9827fc640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa984099fb0 0x7fa9840b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa9827fc640 1 -- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa984162390 msgr2=0x7fa9840c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa9827fc640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa984162390 0x7fa9840c9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa9827fc640 1 -- 172.21.15.38:0/4006746846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa984157530 con 0x7fa98415df30 2024-04-25T08:34:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa9837fe640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa984162390 0x7fa9840c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa9827fc640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98415df30 0x7fa984156ff0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00c820 tx=0x7fa96c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa97b7fe640 1 -- 172.21.15.38:0/4006746846 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c015070 con 0x7fa98415df30 2024-04-25T08:34:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa97b7fe640 1 -- 172.21.15.38:0/4006746846 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa96c007a30 con 0x7fa98415df30 2024-04-25T08:34:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa97b7fe640 1 -- 172.21.15.38:0/4006746846 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96c007e00 con 0x7fa98415df30 2024-04-25T08:34:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.546+0000 7fa989734640 1 -- 172.21.15.38:0/4006746846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9840c9f70 con 0x7fa98415df30 2024-04-25T08:34:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.547+0000 7fa989734640 1 -- 172.21.15.38:0/4006746846 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9840cbcf0 con 0x7fa98415df30 2024-04-25T08:34:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.548+0000 7fa97b7fe640 1 -- 172.21.15.38:0/4006746846 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa96c018070 con 0x7fa98415df30 2024-04-25T08:34:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.548+0000 7fa97b7fe640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa96406b4d0 0x7fa96406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.548+0000 7fa97b7fe640 1 -- 172.21.15.38:0/4006746846 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa96c08a3a0 con 0x7fa98415df30 2024-04-25T08:34:26.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.549+0000 7fa982ffd640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa96406b4d0 0x7fa96406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.549+0000 7fa982ffd640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa96406b4d0 0x7fa96406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa97c006fd0 tx=0x7fa97c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.550+0000 7fa98c43a740 1 Processor -- start 2024-04-25T08:34:26.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.550+0000 7fa98c43a740 1 -- start start 2024-04-25T08:34:26.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.550+0000 7fa98c43a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd98795320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.550+0000 7fa98c43a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98795860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.550+0000 7fa98c43a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98765770 0x55cd98768f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.550+0000 7fa98c43a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd986ceda0 con 0x55cd98792b10 2024-04-25T08:34:26.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.550+0000 7fa98c43a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd98677b80 con 0x55cd98765770 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.550+0000 7fa98c43a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd986cae40 con 0x55cd98792ef0 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa982ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd98795320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa982ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd98795320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35462/0 (socket says 172.21.15.38:35462) 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa982ffd640 1 -- 172.21.15.38:0/1134618224 learned_addr learned my addr 172.21.15.38:0/1134618224 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa9827fc640 1 --2- 172.21.15.38:0/1134618224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98795860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa9837fe640 1 --2- 172.21.15.38:0/1134618224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98765770 0x55cd98768f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa982ffd640 1 -- 172.21.15.38:0/1134618224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 msgr2=0x55cd98795860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa982ffd640 1 --2- 172.21.15.38:0/1134618224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98795860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa982ffd640 1 -- 172.21.15.38:0/1134618224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98765770 msgr2=0x55cd98768f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa982ffd640 1 --2- 172.21.15.38:0/1134618224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98765770 0x55cd98768f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa982ffd640 1 -- 172.21.15.38:0/1134618224 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd98681f10 con 0x55cd98792b10 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa982ffd640 1 --2- 172.21.15.38:0/1134618224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd98795320 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00fad0 tx=0x7fa97c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6ae2bd06f05b69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa978ff9640 1 -- 172.21.15.38:0/1134618224 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c01b050 con 0x55cd98792b10 2024-04-25T08:34:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa978ff9640 1 -- 172.21.15.38:0/1134618224 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c025070 con 0x55cd98792b10 2024-04-25T08:34:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.551+0000 7fa978ff9640 1 -- 172.21.15.38:0/1134618224 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c020440 con 0x55cd98792b10 2024-04-25T08:34:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.552+0000 7fa98c43a740 1 -- 172.21.15.38:0/1134618224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 msgr2=0x55cd98795320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.552+0000 7fa98c43a740 1 --2- 172.21.15.38:0/1134618224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd98795320 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00fad0 tx=0x7fa97c012000 comp rx=0 tx=0).stop 2024-04-25T08:34:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.552+0000 7fa98c43a740 1 -- 172.21.15.38:0/1134618224 shutdown_connections 2024-04-25T08:34:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.552+0000 7fa98c43a740 1 --2- 172.21.15.38:0/1134618224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98765770 0x55cd98768f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.552+0000 7fa98c43a740 1 --2- 172.21.15.38:0/1134618224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98795860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.552+0000 7fa98c43a740 1 --2- 172.21.15.38:0/1134618224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd98795320 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.552+0000 7fa98c43a740 1 -- 172.21.15.38:0/1134618224 >> 172.21.15.38:0/1134618224 conn(0x55cd987679e0 msgr2=0x55cd987b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.552+0000 7fa98c43a740 1 -- 172.21.15.38:0/1134618224 shutdown_connections 2024-04-25T08:34:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.552+0000 7fa98c43a740 1 -- 172.21.15.38:0/1134618224 wait complete. 2024-04-25T08:34:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.553+0000 7fa98c43a740 1 -- start start 2024-04-25T08:34:26.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.553+0000 7fa98c43a740 1 Processor -- start 2024-04-25T08:34:26.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa98c43a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd987880c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa98c43a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98788680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa98c43a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98796860 0x55cd98797bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa98c43a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9876ab90 con 0x55cd98792b10 2024-04-25T08:34:26.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa98c43a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9866d4b0 con 0x55cd98796860 2024-04-25T08:34:26.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa98c43a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd9876ae70 con 0x55cd98792ef0 2024-04-25T08:34:26.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa98c43a740 10 client.0 _ll_register_callbacks cb 0x55cd986ad180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:26.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa982ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd987880c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa982ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd987880c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35472/0 (socket says 172.21.15.38:35472) 2024-04-25T08:34:26.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa982ffd640 1 -- 172.21.15.38:0/2486406317 learned_addr learned my addr 172.21.15.38:0/2486406317 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:26.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa9827fc640 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98788680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa9837fe640 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98796860 0x55cd98797bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa982ffd640 1 -- 172.21.15.38:0/2486406317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 msgr2=0x55cd98788680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa982ffd640 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98788680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa982ffd640 1 -- 172.21.15.38:0/2486406317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98796860 msgr2=0x55cd98797bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa982ffd640 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98796860 0x55cd98797bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa982ffd640 1 -- 172.21.15.38:0/2486406317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd984ad7d0 con 0x55cd98792b10 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa9827fc640 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98788680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa982ffd640 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd987880c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa97c008010 tx=0x7fa97c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1a45837b2c1aab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa961ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.554+0000 7fa961ffb640 1 -- 172.21.15.38:0/2486406317 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c020c80 con 0x55cd98792b10 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa961ffb640 1 -- 172.21.15.38:0/2486406317 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa97c034cb0 con 0x55cd98792b10 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa98c43a740 1 -- 172.21.15.38:0/2486406317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa97c025070 con 0x55cd98792b10 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa961ffb640 1 -- 172.21.15.38:0/2486406317 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa97c01b020 con 0x55cd98792b10 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa98c43a740 10 client.5739 fetch_fsmap learned FSMap version 18 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa98c43a740 1 -- 172.21.15.38:0/2486406317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55cd983e33f0 con 0x55cd98792b10 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa961ffb640 1 -- 172.21.15.38:0/2486406317 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa97c025070 con 0x55cd98792b10 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa961ffb640 1 -- 172.21.15.38:0/2486406317 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa97c031070 con 0x55cd98792b10 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa98c43a740 10 client.5739 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T08:34:26.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa98c43a740 10 client.5739 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:34:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa98c43a740 20 client.5739 populate_metadata read hostname 'smithi038' 2024-04-25T08:34:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa98c43a740 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55cd98798100 0x55cd98795510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa98c43a740 4 client.5739 mds_command: new command op to 5149 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:34:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa98c43a740 1 -- 172.21.15.38:0/2486406317 --> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd986dc2e0 con 0x55cd98798100 2024-04-25T08:34:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa9827fc640 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55cd98798100 0x55cd98795510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa961ffb640 10 client.5739 ms_handle_connect on v2:172.21.15.38:6832/234860294 2024-04-25T08:34:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.555+0000 7fa9827fc640 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55cd98798100 0x55cd98795510 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffeafab3b83a6cad server_cookie=27ab4f9aa141be26 in_seq=0 out_seq=0 2024-04-25T08:34:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa961ffb640 1 -- 172.21.15.38:0/2486406317 <== mds.0 v2:172.21.15.38:6832/234860294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cd986dc2e0 con 0x55cd98798100 2024-04-25T08:34:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa961ffb640 10 client.5739 handle_command_reply: tid=0 2024-04-25T08:34:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa98c43a740 1 client.5739 shutdown 2024-04-25T08:34:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa98c43a740 1 -- 172.21.15.38:0/2486406317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 msgr2=0x55cd987880c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa98c43a740 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd987880c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa97c008010 tx=0x7fa97c004730 comp rx=0 tx=0).stop 2024-04-25T08:34:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa98c43a740 1 -- 172.21.15.38:0/2486406317 shutdown_connections 2024-04-25T08:34:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa98c43a740 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55cd98798100 0x55cd98795510 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa98c43a740 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98796860 0x55cd98797bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa98c43a740 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98788680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa961ffb640 0 client.5739 ms_handle_reset on v2:172.21.15.38:6832/234860294 2024-04-25T08:34:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa98c43a740 1 --2- 172.21.15.38:0/2486406317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd987880c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa98c43a740 1 -- 172.21.15.38:0/2486406317 >> 172.21.15.38:0/2486406317 conn(0x55cd98765770 msgr2=0x55cd987667a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa98c43a740 1 -- 172.21.15.38:0/2486406317 shutdown_connections 2024-04-25T08:34:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa98c43a740 1 -- 172.21.15.38:0/2486406317 wait complete. 2024-04-25T08:34:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.556+0000 7fa98c43a740 20 client.5739 trim_cache size 0 max 16384 2024-04-25T08:34:26.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.572+0000 7fa98c43a740 1 Processor -- start 2024-04-25T08:34:26.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.572+0000 7fa98c43a740 1 -- start start 2024-04-25T08:34:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.572+0000 7fa98c43a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98792b10 0x55cd98760db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.572+0000 7fa98c43a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98789640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.572+0000 7fa98c43a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98796860 0x55cd98787ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.572+0000 7fa98c43a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9842004b0 con 0x55cd98796860 2024-04-25T08:34:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.572+0000 7fa98c43a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840cd0d0 con 0x55cd98792b10 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.572+0000 7fa98c43a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840cc4d0 con 0x55cd98792ef0 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.572+0000 7fa9837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98796860 0x55cd98787ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa9837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98796860 0x55cd98787ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35480/0 (socket says 172.21.15.38:35480) 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa9837fe640 1 -- 172.21.15.38:0/1998171451 learned_addr learned my addr 172.21.15.38:0/1998171451 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa9827fc640 1 --2- 172.21.15.38:0/1998171451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98789640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa982ffd640 1 --2- 172.21.15.38:0/1998171451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98792b10 0x55cd98760db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa9837fe640 1 -- 172.21.15.38:0/1998171451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 msgr2=0x55cd98789640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa9837fe640 1 --2- 172.21.15.38:0/1998171451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98789640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa9837fe640 1 -- 172.21.15.38:0/1998171451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98792b10 msgr2=0x55cd98760db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa9837fe640 1 --2- 172.21.15.38:0/1998171451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98792b10 0x55cd98760db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa9837fe640 1 -- 172.21.15.38:0/1998171451 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd983a30f0 con 0x55cd98796860 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa982ffd640 1 --2- 172.21.15.38:0/1998171451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98792b10 0x55cd98760db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa9837fe640 1 --2- 172.21.15.38:0/1998171451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98796860 0x55cd98787ed0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa9740093e0 tx=0x7fa97402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c0e6b449d47a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa963fff640 1 -- 172.21.15.38:0/1998171451 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974036040 con 0x55cd98796860 2024-04-25T08:34:26.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa963fff640 1 -- 172.21.15.38:0/1998171451 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa974031700 con 0x55cd98796860 2024-04-25T08:34:26.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.573+0000 7fa963fff640 1 -- 172.21.15.38:0/1998171451 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa974031ab0 con 0x55cd98796860 2024-04-25T08:34:26.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.574+0000 7fa98c43a740 1 -- 172.21.15.38:0/1998171451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98796860 msgr2=0x55cd98787ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.574+0000 7fa98c43a740 1 --2- 172.21.15.38:0/1998171451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98796860 0x55cd98787ed0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa9740093e0 tx=0x7fa97402e000 comp rx=0 tx=0).stop 2024-04-25T08:34:26.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.574+0000 7fa98c43a740 1 -- 172.21.15.38:0/1998171451 shutdown_connections 2024-04-25T08:34:26.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.574+0000 7fa98c43a740 1 --2- 172.21.15.38:0/1998171451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98796860 0x55cd98787ed0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.574+0000 7fa98c43a740 1 --2- 172.21.15.38:0/1998171451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98792ef0 0x55cd98789640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.574+0000 7fa98c43a740 1 --2- 172.21.15.38:0/1998171451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98792b10 0x55cd98760db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.574+0000 7fa98c43a740 1 -- 172.21.15.38:0/1998171451 >> 172.21.15.38:0/1998171451 conn(0x55cd98765770 msgr2=0x55cd98765c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:26.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.574+0000 7fa98c43a740 1 -- 172.21.15.38:0/1998171451 shutdown_connections 2024-04-25T08:34:26.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.574+0000 7fa98c43a740 1 -- 172.21.15.38:0/1998171451 wait complete. 2024-04-25T08:34:26.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.574+0000 7fa98c43a740 1 -- start start 2024-04-25T08:34:26.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.574+0000 7fa98c43a740 1 Processor -- start 2024-04-25T08:34:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.575+0000 7fa98c43a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd98819950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.575+0000 7fa98c43a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98792ef0 0x55cd987948d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.575+0000 7fa98c43a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98796860 0x55cd98791190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.575+0000 7fa98c43a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd98798a50 con 0x55cd98792b10 2024-04-25T08:34:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.575+0000 7fa98c43a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9840ccad0 con 0x55cd98792ef0 2024-04-25T08:34:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.575+0000 7fa98c43a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cd98789b80 con 0x55cd98796860 2024-04-25T08:34:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.575+0000 7fa98c43a740 10 client.0 _ll_register_callbacks cb 0x7fa97c00b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.575+0000 7fa982ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd98819950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.575+0000 7fa982ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd98819950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35496/0 (socket says 172.21.15.38:35496) 2024-04-25T08:34:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.575+0000 7fa982ffd640 1 -- 172.21.15.38:0/3794509298 learned_addr learned my addr 172.21.15.38:0/3794509298 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa9837fe640 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98796860 0x55cd98791190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa982ffd640 1 -- 172.21.15.38:0/3794509298 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98796860 msgr2=0x55cd98791190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa982ffd640 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98796860 0x55cd98791190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa9827fc640 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98792ef0 0x55cd987948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa982ffd640 1 -- 172.21.15.38:0/3794509298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98792ef0 msgr2=0x55cd987948d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa982ffd640 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98792ef0 0x55cd987948d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa982ffd640 1 -- 172.21.15.38:0/3794509298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd987916d0 con 0x55cd98792b10 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa982ffd640 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd98819950 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00b830 tx=0x7fa97c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab71912a2c1f3130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa961ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa961ffb640 1 -- 172.21.15.38:0/3794509298 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97c044060 con 0x55cd98792b10 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa961ffb640 1 -- 172.21.15.38:0/3794509298 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa97c033850 con 0x55cd98792b10 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa98c43a740 1 -- 172.21.15.38:0/3794509298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa974031700 con 0x55cd98792b10 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa961ffb640 1 -- 172.21.15.38:0/3794509298 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa97c00b140 con 0x55cd98792b10 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa98c43a740 10 client.5745 fetch_fsmap learned FSMap version 18 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa98c43a740 1 -- 172.21.15.38:0/3794509298 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55cd98790b60 con 0x55cd98792b10 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa961ffb640 1 -- 172.21.15.38:0/3794509298 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa974031700 con 0x55cd98792b10 2024-04-25T08:34:26.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa961ffb640 1 -- 172.21.15.38:0/3794509298 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa97c04d080 con 0x55cd98792b10 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa98c43a740 10 client.5745 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa98c43a740 10 client.5745 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa98c43a740 20 client.5745 populate_metadata read hostname 'smithi038' 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa98c43a740 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55cd98790380 0x55cd98787ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa98c43a740 4 client.5745 mds_command: new command op to 5149 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.576+0000 7fa98c43a740 1 -- 172.21.15.38:0/3794509298 --> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cd986d6630 con 0x55cd98790380 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa9827fc640 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55cd98790380 0x55cd98787ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa9827fc640 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55cd98790380 0x55cd98787ed0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74a0985e646626b1 server_cookie=cfd4dcfb009fb73a in_seq=0 out_seq=0 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa961ffb640 10 client.5745 ms_handle_connect on v2:172.21.15.38:6832/234860294 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa961ffb640 1 -- 172.21.15.38:0/3794509298 <== mds.0 v2:172.21.15.38:6832/234860294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cd986d6630 con 0x55cd98790380 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa961ffb640 10 client.5745 handle_command_reply: tid=0 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa98c43a740 1 client.5745 shutdown 2024-04-25T08:34:26.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa98c43a740 1 -- 172.21.15.38:0/3794509298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 msgr2=0x55cd98819950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa98c43a740 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd98819950 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa97c00b830 tx=0x7fa97c015000 comp rx=0 tx=0).stop 2024-04-25T08:34:26.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa98c43a740 1 -- 172.21.15.38:0/3794509298 shutdown_connections 2024-04-25T08:34:26.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa98c43a740 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55cd98790380 0x55cd98787ed0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa98c43a740 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cd98796860 0x55cd98791190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa98c43a740 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cd98792ef0 0x55cd987948d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa98c43a740 1 --2- 172.21.15.38:0/3794509298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd98792b10 0x55cd98819950 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa98c43a740 1 -- 172.21.15.38:0/3794509298 >> 172.21.15.38:0/3794509298 conn(0x55cd98765770 msgr2=0x55cd987b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:26.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa961ffb640 0 client.5745 ms_handle_reset on v2:172.21.15.38:6832/234860294 2024-04-25T08:34:26.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa98c43a740 1 -- 172.21.15.38:0/3794509298 shutdown_connections 2024-04-25T08:34:26.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa98c43a740 1 -- 172.21.15.38:0/3794509298 wait complete. 2024-04-25T08:34:26.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.577+0000 7fa98c43a740 20 client.5745 trim_cache size 0 max 16384 2024-04-25T08:34:26.579 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:34:26.579 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-04-25T08:34:26.579 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-04-25T08:34:26.579 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:34:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.578+0000 7fa989734640 1 -- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa96406b4d0 msgr2=0x7fa96406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.578+0000 7fa989734640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa96406b4d0 0x7fa96406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa97c006fd0 tx=0x7fa97c008040 comp rx=0 tx=0).stop 2024-04-25T08:34:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.578+0000 7fa989734640 1 -- 172.21.15.38:0/4006746846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98415df30 msgr2=0x7fa984156ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.578+0000 7fa989734640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98415df30 0x7fa984156ff0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa96c00c820 tx=0x7fa96c00cc20 comp rx=0 tx=0).stop 2024-04-25T08:34:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.578+0000 7fa989734640 1 -- 172.21.15.38:0/4006746846 shutdown_connections 2024-04-25T08:34:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.578+0000 7fa989734640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa984162390 0x7fa9840c9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.578+0000 7fa989734640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa98415df30 0x7fa984156ff0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.578+0000 7fa989734640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa96406b4d0 0x7fa96406d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.578+0000 7fa989734640 1 --2- 172.21.15.38:0/4006746846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa984099fb0 0x7fa9840b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.578+0000 7fa989734640 1 -- 172.21.15.38:0/4006746846 >> 172.21.15.38:0/4006746846 conn(0x7fa9840b3110 msgr2=0x7fa984161bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.578+0000 7fa989734640 1 -- 172.21.15.38:0/4006746846 shutdown_connections 2024-04-25T08:34:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.578+0000 7fa989734640 1 -- 172.21.15.38:0/4006746846 wait complete. 2024-04-25T08:34:26.583 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:34:26.584 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:34:26.584 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T08:34:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.691+0000 7f158f0e1640 1 Processor -- start 2024-04-25T08:34:26.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.691+0000 7f158f0e1640 1 -- start start 2024-04-25T08:34:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158f0e1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f15880d0320 0x7f15880d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158f0e1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f15880b6980 0x7f15880b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158f0e1640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15880d2060 con 0x7f15880d0700 2024-04-25T08:34:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158f0e1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15880d2330 con 0x7f15880b6980 2024-04-25T08:34:26.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158f0e1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15880d2610 con 0x7f15880d0320 2024-04-25T08:34:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158d67e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f15880b6980 0x7f15880b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158d67e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f15880b6980 0x7f15880b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35506/0 (socket says 172.21.15.38:35506) 2024-04-25T08:34:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158d67e640 1 -- 172.21.15.38:0/3428929314 learned_addr learned my addr 172.21.15.38:0/3428929314 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158ce7d640 1 --2- 172.21.15.38:0/3428929314 >> v2:172.21.15.107:3300/0 conn(0x7f15880d0320 0x7f15880d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158d67e640 1 -- 172.21.15.38:0/3428929314 >> v2:172.21.15.107:3300/0 conn(0x7f15880d0320 msgr2=0x7f15880d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158d67e640 1 --2- 172.21.15.38:0/3428929314 >> v2:172.21.15.107:3300/0 conn(0x7f15880d0320 0x7f15880d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158d67e640 1 -- 172.21.15.38:0/3428929314 >> v1:172.21.15.107:6789/0 conn(0x7f15880d0700 legacy=0x7f15880d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.692+0000 7f158d67e640 1 -- 172.21.15.38:0/3428929314 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15880d28f0 con 0x7f15880b6980 2024-04-25T08:34:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.693+0000 7f158d67e640 1 --2- 172.21.15.38:0/3428929314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880b6980 0x7f15880b6dd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1588069100 tx=0x7f158002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fcd1f3ee874609 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.693+0000 7f15877fe640 1 -- 172.21.15.38:0/3428929314 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580036070 con 0x7f15880b6980 2024-04-25T08:34:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.693+0000 7f15877fe640 1 -- 172.21.15.38:0/3428929314 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1580031600 con 0x7f15880b6980 2024-04-25T08:34:26.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.693+0000 7f15877fe640 1 -- 172.21.15.38:0/3428929314 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15800319d0 con 0x7f15880b6980 2024-04-25T08:34:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.694+0000 7f158f0e1640 1 -- 172.21.15.38:0/3428929314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880b6980 msgr2=0x7f15880b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.694+0000 7f158f0e1640 1 --2- 172.21.15.38:0/3428929314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880b6980 0x7f15880b6dd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1588069100 tx=0x7f158002a780 comp rx=0 tx=0).stop 2024-04-25T08:34:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.694+0000 7f158f0e1640 1 -- 172.21.15.38:0/3428929314 shutdown_connections 2024-04-25T08:34:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.694+0000 7f158f0e1640 1 --2- 172.21.15.38:0/3428929314 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880b6980 0x7f15880b6dd0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.694+0000 7f158f0e1640 1 --2- 172.21.15.38:0/3428929314 >> v2:172.21.15.107:3300/0 conn(0x7f15880d0320 0x7f15880d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.694+0000 7f158f0e1640 1 -- 172.21.15.38:0/3428929314 >> 172.21.15.38:0/3428929314 conn(0x7f15880668f0 msgr2=0x7f1588066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.694+0000 7f158f0e1640 1 -- 172.21.15.38:0/3428929314 shutdown_connections 2024-04-25T08:34:26.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.694+0000 7f158f0e1640 1 -- 172.21.15.38:0/3428929314 wait complete. 2024-04-25T08:34:26.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.695+0000 7f158f0e1640 1 Processor -- start 2024-04-25T08:34:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.695+0000 7f158f0e1640 1 -- start start 2024-04-25T08:34:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f158f0e1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b6980 0x7f1588149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f158f0e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0320 0x7f1588145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f158f0e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15880d0700 0x7f15880b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f158f0e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880d0b40 con 0x7f15880d0320 2024-04-25T08:34:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f158f0e1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15880d2eb0 con 0x7f15880d0700 2024-04-25T08:34:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f158f0e1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15880b59a0 con 0x7f15880b6980 2024-04-25T08:34:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f1587fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0320 0x7f1588145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f1587fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0320 0x7f1588145cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35508/0 (socket says 172.21.15.38:35508) 2024-04-25T08:34:26.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f1587fff640 1 -- 172.21.15.38:0/3816734959 learned_addr learned my addr 172.21.15.38:0/3816734959 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:26.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f158ce7d640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b6980 0x7f1588149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f158d67e640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15880d0700 0x7f15880b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f1587fff640 1 -- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b6980 msgr2=0x7f1588149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f1587fff640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b6980 0x7f1588149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f1587fff640 1 -- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15880d0700 msgr2=0x7f15880b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f1587fff640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15880d0700 0x7f15880b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.696+0000 7f1587fff640 1 -- 172.21.15.38:0/3816734959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15880b4d90 con 0x7f15880d0320 2024-04-25T08:34:26.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.697+0000 7f1587fff640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0320 0x7f1588145cd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f157c00b3f0 tx=0x7f157c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.697+0000 7f15857fa640 1 -- 172.21.15.38:0/3816734959 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c015020 con 0x7f15880d0320 2024-04-25T08:34:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.697+0000 7f15857fa640 1 -- 172.21.15.38:0/3816734959 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157c011830 con 0x7f15880d0320 2024-04-25T08:34:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.697+0000 7f158f0e1640 1 -- 172.21.15.38:0/3816734959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15880b5070 con 0x7f15880d0320 2024-04-25T08:34:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.697+0000 7f15857fa640 1 -- 172.21.15.38:0/3816734959 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c011c20 con 0x7f15880d0320 2024-04-25T08:34:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.697+0000 7f158f0e1640 1 -- 172.21.15.38:0/3816734959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15880b5630 con 0x7f15880d0320 2024-04-25T08:34:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.698+0000 7f15857fa640 1 -- 172.21.15.38:0/3816734959 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f157c018020 con 0x7f15880d0320 2024-04-25T08:34:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.699+0000 7f15857fa640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f157406b500 0x7f157406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.699+0000 7f15857fa640 1 -- 172.21.15.38:0/3816734959 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f157c089e50 con 0x7f15880d0320 2024-04-25T08:34:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.699+0000 7f158ce7d640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f157406b500 0x7f157406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.699+0000 7f158ce7d640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f157406b500 0x7f157406d9c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1578006fd0 tx=0x7f1578008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.700+0000 7f1591de8740 1 Processor -- start 2024-04-25T08:34:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.700+0000 7f1591de8740 1 -- start start 2024-04-25T08:34:26.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.700+0000 7f1591de8740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83a4ca0 0x55bae83a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.700+0000 7f1591de8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83e1370 0x55bae83a5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.700+0000 7f1591de8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae83cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.700+0000 7f1591de8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae8307b90 con 0x55bae83cf760 2024-04-25T08:34:26.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.700+0000 7f1591de8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae82b0e40 con 0x55bae83e1370 2024-04-25T08:34:26.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.700+0000 7f1591de8740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bae8303c30 con 0x55bae83a4ca0 2024-04-25T08:34:26.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.700+0000 7f158d67e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae83cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.701+0000 7f158ce7d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83a4ca0 0x55bae83a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.700+0000 7f158d67e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae83cfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35522/0 (socket says 172.21.15.38:35522) 2024-04-25T08:34:26.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.701+0000 7f158d67e640 1 -- 172.21.15.38:0/3817086737 learned_addr learned my addr 172.21.15.38:0/3817086737 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:26.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.701+0000 7f1587fff640 1 --2- 172.21.15.38:0/3817086737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83e1370 0x55bae83a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.701+0000 7f158d67e640 1 -- 172.21.15.38:0/3817086737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83a4ca0 msgr2=0x55bae83a6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.701+0000 7f158d67e640 1 --2- 172.21.15.38:0/3817086737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83a4ca0 0x55bae83a6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.701+0000 7f158d67e640 1 -- 172.21.15.38:0/3817086737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83e1370 msgr2=0x55bae83a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.701+0000 7f158d67e640 1 --2- 172.21.15.38:0/3817086737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83e1370 0x55bae83a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.701+0000 7f158d67e640 1 -- 172.21.15.38:0/3817086737 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bae80e67d0 con 0x55bae83cf760 2024-04-25T08:34:26.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.701+0000 7f158d67e640 1 --2- 172.21.15.38:0/3817086737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae83cfbe0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1580009f40 tx=0x7f158002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23c80f37b7ca4e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.701+0000 7f156e7fc640 1 -- 172.21.15.38:0/3817086737 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580036040 con 0x55bae83cf760 2024-04-25T08:34:26.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.701+0000 7f156e7fc640 1 -- 172.21.15.38:0/3817086737 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1580037040 con 0x55bae83cf760 2024-04-25T08:34:26.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.701+0000 7f156e7fc640 1 -- 172.21.15.38:0/3817086737 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15800319c0 con 0x55bae83cf760 2024-04-25T08:34:26.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.702+0000 7f1591de8740 1 -- 172.21.15.38:0/3817086737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 msgr2=0x55bae83cfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.702+0000 7f1591de8740 1 --2- 172.21.15.38:0/3817086737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae83cfbe0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1580009f40 tx=0x7f158002e000 comp rx=0 tx=0).stop 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.702+0000 7f1591de8740 1 -- 172.21.15.38:0/3817086737 shutdown_connections 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.702+0000 7f1591de8740 1 --2- 172.21.15.38:0/3817086737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae83cfbe0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.702+0000 7f1591de8740 1 --2- 172.21.15.38:0/3817086737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83e1370 0x55bae83a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.702+0000 7f1591de8740 1 --2- 172.21.15.38:0/3817086737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83a4ca0 0x55bae83a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.702+0000 7f1591de8740 1 -- 172.21.15.38:0/3817086737 >> 172.21.15.38:0/3817086737 conn(0x55bae836c870 msgr2=0x55bae83ea760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.702+0000 7f1591de8740 1 -- 172.21.15.38:0/3817086737 shutdown_connections 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.702+0000 7f1591de8740 1 -- 172.21.15.38:0/3817086737 wait complete. 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.702+0000 7f1591de8740 1 -- start start 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.703+0000 7f1591de8740 1 Processor -- start 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.703+0000 7f1591de8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83a4ca0 0x55bae83e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.703+0000 7f1591de8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae83bb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.703+0000 7f1591de8740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83e1370 0x55bae83d27c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.703+0000 7f1591de8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae82c43e0 con 0x55bae83cf760 2024-04-25T08:34:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.703+0000 7f1591de8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae83cd2b0 con 0x55bae83a4ca0 2024-04-25T08:34:26.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.703+0000 7f1591de8740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bae83bc1b0 con 0x55bae83e1370 2024-04-25T08:34:26.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.703+0000 7f1591de8740 10 client.0 _ll_register_callbacks cb 0x55bae82c3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:26.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.703+0000 7f1587fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae83bb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1587fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae83bb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35534/0 (socket says 172.21.15.38:35534) 2024-04-25T08:34:26.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1587fff640 1 -- 172.21.15.38:0/1809497162 learned_addr learned my addr 172.21.15.38:0/1809497162 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:26.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f158d67e640 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83e1370 0x55bae83d27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f158ce7d640 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83a4ca0 0x55bae83e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1587fff640 1 -- 172.21.15.38:0/1809497162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83e1370 msgr2=0x55bae83d27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1587fff640 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83e1370 0x55bae83d27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1587fff640 1 -- 172.21.15.38:0/1809497162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83a4ca0 msgr2=0x55bae83e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1587fff640 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83a4ca0 0x55bae83e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1587fff640 1 -- 172.21.15.38:0/1809497162 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bae7fdc0f0 con 0x55bae83cf760 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1587fff640 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae83bb770 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f157c058450 tx=0x7f157c060c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50c4e20018cb255 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f15677fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f15677fe640 1 -- 172.21.15.38:0/1809497162 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c05c350 con 0x55bae83cf760 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f15677fe640 1 -- 172.21.15.38:0/1809497162 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f157c057910 con 0x55bae83cf760 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1591de8740 1 -- 172.21.15.38:0/1809497162 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1580037040 con 0x55bae83cf760 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f15677fe640 1 -- 172.21.15.38:0/1809497162 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f157c057b10 con 0x55bae83cf760 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1591de8740 10 client.5757 fetch_fsmap learned FSMap version 18 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1591de8740 1 -- 172.21.15.38:0/1809497162 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55bae8146f60 con 0x55bae83cf760 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f15677fe640 1 -- 172.21.15.38:0/1809497162 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1580037040 con 0x55bae83cf760 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f15677fe640 1 -- 172.21.15.38:0/1809497162 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f157c081b20 con 0x55bae83cf760 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1591de8740 10 client.5757 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1591de8740 10 client.5757 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:34:26.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.704+0000 7f1591de8740 20 client.5757 populate_metadata read hostname 'smithi038' 2024-04-25T08:34:26.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.705+0000 7f1591de8740 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bae83d0e90 0x55bae83baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.705+0000 7f1591de8740 4 client.5757 mds_command: new command op to 5149 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:34:26.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.705+0000 7f1591de8740 1 -- 172.21.15.38:0/1809497162 --> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bae83150d0 con 0x55bae83d0e90 2024-04-25T08:34:26.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.705+0000 7f158ce7d640 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bae83d0e90 0x55bae83baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.705+0000 7f15677fe640 10 client.5757 ms_handle_connect on v2:172.21.15.38:6832/234860294 2024-04-25T08:34:26.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.705+0000 7f158ce7d640 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bae83d0e90 0x55bae83baab0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67655765bf05e624 server_cookie=cd76b64f239e8314 in_seq=0 out_seq=0 2024-04-25T08:34:26.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.705+0000 7f15677fe640 1 -- 172.21.15.38:0/1809497162 <== mds.0 v2:172.21.15.38:6832/234860294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bae83150d0 con 0x55bae83d0e90 2024-04-25T08:34:26.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.705+0000 7f15677fe640 10 client.5757 handle_command_reply: tid=0 2024-04-25T08:34:26.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.705+0000 7f1591de8740 1 client.5757 shutdown 2024-04-25T08:34:26.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.706+0000 7f1591de8740 1 -- 172.21.15.38:0/1809497162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 msgr2=0x55bae83bb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.706+0000 7f1591de8740 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae83bb770 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f157c058450 tx=0x7f157c060c60 comp rx=0 tx=0).stop 2024-04-25T08:34:26.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.706+0000 7f1591de8740 1 -- 172.21.15.38:0/1809497162 shutdown_connections 2024-04-25T08:34:26.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.706+0000 7f1591de8740 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bae83d0e90 0x55bae83baab0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.706+0000 7f1591de8740 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83e1370 0x55bae83d27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.706+0000 7f1591de8740 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae83bb770 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.706+0000 7f1591de8740 1 --2- 172.21.15.38:0/1809497162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83a4ca0 0x55bae83e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.706+0000 7f15677fe640 0 client.5757 ms_handle_reset on v2:172.21.15.38:6832/234860294 2024-04-25T08:34:26.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.706+0000 7f1591de8740 1 -- 172.21.15.38:0/1809497162 >> 172.21.15.38:0/1809497162 conn(0x55bae836c870 msgr2=0x55bae83cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:26.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.706+0000 7f1591de8740 1 -- 172.21.15.38:0/1809497162 shutdown_connections 2024-04-25T08:34:26.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.706+0000 7f1591de8740 1 -- 172.21.15.38:0/1809497162 wait complete. 2024-04-25T08:34:26.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.706+0000 7f1591de8740 20 client.5757 trim_cache size 0 max 16384 2024-04-25T08:34:26.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.720+0000 7f1591de8740 1 Processor -- start 2024-04-25T08:34:26.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.720+0000 7f1591de8740 1 -- start start 2024-04-25T08:34:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f1591de8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83a4ca0 0x55bae83cd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f1591de8740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83cf760 0x55bae83e00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f1591de8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83e1370 0x55bae83e0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f1591de8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15882003a0 con 0x55bae83e1370 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f1591de8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15881568a0 con 0x55bae83a4ca0 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f1591de8740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15881470c0 con 0x55bae83cf760 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f158d67e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83e1370 0x55bae83e0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f158d67e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83e1370 0x55bae83e0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35546/0 (socket says 172.21.15.38:35546) 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f158d67e640 1 -- 172.21.15.38:0/1701558006 learned_addr learned my addr 172.21.15.38:0/1701558006 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f1587fff640 1 --2- 172.21.15.38:0/1701558006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83cf760 0x55bae83e00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f158ce7d640 1 --2- 172.21.15.38:0/1701558006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83a4ca0 0x55bae83cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f158d67e640 1 -- 172.21.15.38:0/1701558006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83cf760 msgr2=0x55bae83e00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f158d67e640 1 --2- 172.21.15.38:0/1701558006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83cf760 0x55bae83e00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f158d67e640 1 -- 172.21.15.38:0/1701558006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83a4ca0 msgr2=0x55bae83cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f158d67e640 1 --2- 172.21.15.38:0/1701558006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83a4ca0 0x55bae83cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f158d67e640 1 -- 172.21.15.38:0/1701558006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bae83e5490 con 0x55bae83e1370 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f1587fff640 1 --2- 172.21.15.38:0/1701558006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83cf760 0x55bae83e00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f158ce7d640 1 --2- 172.21.15.38:0/1701558006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83a4ca0 0x55bae83cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.721+0000 7f158d67e640 1 --2- 172.21.15.38:0/1701558006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83e1370 0x55bae83e0630 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f158002a7c0 tx=0x7f1580009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99f272f99116dd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.722+0000 7f156dffb640 1 -- 172.21.15.38:0/1701558006 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580036070 con 0x55bae83e1370 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.722+0000 7f156dffb640 1 -- 172.21.15.38:0/1701558006 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1580037040 con 0x55bae83e1370 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.722+0000 7f156dffb640 1 -- 172.21.15.38:0/1701558006 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158002adc0 con 0x55bae83e1370 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.722+0000 7f1591de8740 1 -- 172.21.15.38:0/1701558006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83e1370 msgr2=0x55bae83e0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.722+0000 7f1591de8740 1 --2- 172.21.15.38:0/1701558006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83e1370 0x55bae83e0630 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f158002a7c0 tx=0x7f1580009810 comp rx=0 tx=0).stop 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.722+0000 7f1591de8740 1 -- 172.21.15.38:0/1701558006 shutdown_connections 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.722+0000 7f1591de8740 1 --2- 172.21.15.38:0/1701558006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83e1370 0x55bae83e0630 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.722+0000 7f1591de8740 1 --2- 172.21.15.38:0/1701558006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83cf760 0x55bae83e00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.722+0000 7f1591de8740 1 --2- 172.21.15.38:0/1701558006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83a4ca0 0x55bae83cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.722+0000 7f1591de8740 1 -- 172.21.15.38:0/1701558006 >> 172.21.15.38:0/1701558006 conn(0x55bae836c870 msgr2=0x55bae839af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.722+0000 7f1591de8740 1 -- 172.21.15.38:0/1701558006 shutdown_connections 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.722+0000 7f1591de8740 1 -- 172.21.15.38:0/1701558006 wait complete. 2024-04-25T08:34:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.723+0000 7f1591de8740 1 -- start start 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.723+0000 7f1591de8740 1 Processor -- start 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1591de8740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83a4ca0 0x55bae8457f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1591de8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae8403790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1591de8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83e1370 0x55bae8403cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1591de8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bae83e52f0 con 0x55bae83cf760 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1591de8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15881562a0 con 0x55bae83e1370 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1591de8740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bae83cc5b0 con 0x55bae83a4ca0 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1591de8740 10 client.0 _ll_register_callbacks cb 0x7f157c057ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1587fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae8403790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1587fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae8403790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35562/0 (socket says 172.21.15.38:35562) 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1587fff640 1 -- 172.21.15.38:0/1816680193 learned_addr learned my addr 172.21.15.38:0/1816680193 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f158ce7d640 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83a4ca0 0x55bae8457f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f158d67e640 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83e1370 0x55bae8403cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1587fff640 1 -- 172.21.15.38:0/1816680193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83a4ca0 msgr2=0x55bae8457f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1587fff640 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83a4ca0 0x55bae8457f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1587fff640 1 -- 172.21.15.38:0/1816680193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83e1370 msgr2=0x55bae8403cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:26.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1587fff640 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83e1370 0x55bae8403cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1587fff640 1 -- 172.21.15.38:0/1816680193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bae7fb51b0 con 0x55bae83cf760 2024-04-25T08:34:26.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f1587fff640 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae8403790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f157c00bdb0 tx=0x7f157c088090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe791bf988a513 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:26.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f15677fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:34:26.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.724+0000 7f15677fe640 1 -- 172.21.15.38:0/1816680193 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157c054420 con 0x55bae83cf760 2024-04-25T08:34:26.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f15677fe640 1 -- 172.21.15.38:0/1816680193 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f157c01bc70 con 0x55bae83cf760 2024-04-25T08:34:26.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f1591de8740 1 -- 172.21.15.38:0/1816680193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1580037040 con 0x55bae83cf760 2024-04-25T08:34:26.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f15677fe640 1 -- 172.21.15.38:0/1816680193 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f157c01be70 con 0x55bae83cf760 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f1591de8740 10 client.5763 fetch_fsmap learned FSMap version 18 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f1591de8740 1 -- 172.21.15.38:0/1816680193 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x55bae83e39b0 con 0x55bae83cf760 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f15677fe640 1 -- 172.21.15.38:0/1816680193 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1580037040 con 0x55bae83cf760 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f15677fe640 1 -- 172.21.15.38:0/1816680193 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f157c057380 con 0x55bae83cf760 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f1591de8740 10 client.5763 fetch_fsmap finished waiting for FSMap version 18 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f1591de8740 10 client.5763 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f1591de8740 20 client.5763 populate_metadata read hostname 'smithi038' 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f1591de8740 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bae83a5f50 0x55bae84588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f1591de8740 4 client.5763 mds_command: new command op to 5149 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f1591de8740 1 -- 172.21.15.38:0/1816680193 --> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55bae830f420 con 0x55bae83a5f50 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f158ce7d640 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bae83a5f50 0x55bae84588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f15677fe640 10 client.5763 ms_handle_connect on v2:172.21.15.38:6832/234860294 2024-04-25T08:34:26.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.725+0000 7f158ce7d640 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bae83a5f50 0x55bae84588a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ae8dd8610b2300 server_cookie=5d6cdb1db599171a in_seq=0 out_seq=0 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f15677fe640 1 -- 172.21.15.38:0/1816680193 <== mds.0 v2:172.21.15.38:6832/234860294 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17947 (crc 0 0 0) 0x55bae830f420 con 0x55bae83a5f50 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f15677fe640 10 client.5763 handle_command_reply: tid=0 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f1591de8740 1 client.5763 shutdown 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f1591de8740 1 -- 172.21.15.38:0/1816680193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 msgr2=0x55bae8403790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f1591de8740 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae8403790 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f157c00bdb0 tx=0x7f157c088090 comp rx=0 tx=0).stop 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f1591de8740 1 -- 172.21.15.38:0/1816680193 shutdown_connections 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f1591de8740 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.38:6832/234860294,v1:172.21.15.38:6835/234860294] conn(0x55bae83a5f50 0x55bae84588a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f1591de8740 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bae83e1370 0x55bae8403cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f1591de8740 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bae83cf760 0x55bae8403790 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f1591de8740 1 --2- 172.21.15.38:0/1816680193 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bae83a4ca0 0x55bae8457f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f15677fe640 0 client.5763 ms_handle_reset on v2:172.21.15.38:6832/234860294 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f1591de8740 1 -- 172.21.15.38:0/1816680193 >> 172.21.15.38:0/1816680193 conn(0x55bae836c870 msgr2=0x55bae83d2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f1591de8740 1 -- 172.21.15.38:0/1816680193 shutdown_connections 2024-04-25T08:34:26.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f1591de8740 1 -- 172.21.15.38:0/1816680193 wait complete. 2024-04-25T08:34:26.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.726+0000 7f1591de8740 20 client.5763 trim_cache size 0 max 16384 2024-04-25T08:34:26.730 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:34:26.730 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T08:34:26.730 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 5648, 2024-04-25T08:34:26.730 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 5648, 2024-04-25T08:34:26.730 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1409111, 2024-04-25T08:34:26.730 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 39698582, 2024-04-25T08:34:26.730 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 17, 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 6, 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.446996131, 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.193969286, 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.090812505, 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.133579722, 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 5647, 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.368804435, 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000065309 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 10, 2024-04-25T08:34:26.731 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000001128, 2024-04-25T08:34:26.732 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000112 2024-04-25T08:34:26.732 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.732 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T08:34:26.732 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T08:34:26.732 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.732 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T08:34:26.732 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 3820, 2024-04-25T08:34:26.732 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 3843, 2024-04-25T08:34:26.732 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 957160, 2024-04-25T08:34:26.732 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 18142732, 2024-04-25T08:34:26.732 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 6, 2024-04-25T08:34:26.732 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.289980250, 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.125143286, 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.059875195, 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.086040827, 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3841, 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.341910099, 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000089015 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.733 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.734 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.734 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T08:34:26.734 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 121056 2024-04-25T08:34:26.734 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.734 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T08:34:26.734 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 35003, 2024-04-25T08:34:26.734 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 60894, 2024-04-25T08:34:26.734 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 12572568, 2024-04-25T08:34:26.734 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 56002203, 2024-04-25T08:34:26.734 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 7, 2024-04-25T08:34:26.734 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 3.003459568, 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 1.484579615, 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 1.755523166, 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.073700459, 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 60894, 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.094414850, 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000017972 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 88335, 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.013200959, 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000149 2024-04-25T08:34:26.735 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 121456, 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 24640 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 10466, 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 13.015085568, 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001243558 2024-04-25T08:34:26.736 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:26.737 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.737 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-04-25T08:34:26.737 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-25T08:34:26.737 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-25T08:34:26.737 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3699, 2024-04-25T08:34:26.737 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.758821846, 2024-04-25T08:34:26.737 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000205142 2024-04-25T08:34:26.737 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:26.737 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.737 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-04-25T08:34:26.737 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 28175, 2024-04-25T08:34:26.737 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 28175, 2024-04-25T08:34:26.738 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-04-25T08:34:26.738 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 28175, 2024-04-25T08:34:26.738 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 258.454011423, 2024-04-25T08:34:26.738 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.009173168 2024-04-25T08:34:26.738 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.738 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-04-25T08:34:26.738 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-04-25T08:34:26.738 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-04-25T08:34:26.738 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-04-25T08:34:26.738 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 12, 2024-04-25T08:34:26.738 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-04-25T08:34:26.738 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 17845, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 17845, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 17845, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 2, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 2, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 2, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 46530, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 46132, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-04-25T08:34:26.739 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 140, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 2013, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-04-25T08:34:26.740 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 2013, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 13, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 2, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 11, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 2000, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 6166, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 6007, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T08:34:26.741 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 6166, 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 25, 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 17858 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-04-25T08:34:26.742 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 4121, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 3355, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 2211, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-04-25T08:34:26.743 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 23554, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 23554, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 31, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-04-25T08:34:26.744 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 31, 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 3, 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 82086409, 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 0, 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6464, 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 79.275983313, 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.012264230 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 0 2024-04-25T08:34:26.745 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 14, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 2016, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 2002, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 13, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 13, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 0, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 17845, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 19845, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 2000, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 2, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 2002, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 2000, 2024-04-25T08:34:26.746 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 92176, 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215396 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 37455, 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 28175, 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 7, 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2000, 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.348468024, 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000674234 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-04-25T08:34:26.747 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1927, 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.581732819, 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000301885 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6000, 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 9.325768037, 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001554294 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-04-25T08:34:26.748 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 8024, 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 6.845976495, 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000853187 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.749 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.750 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.750 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.750 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-04-25T08:34:26.750 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.750 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.750 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.750 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.750 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-04-25T08:34:26.750 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3, 2024-04-25T08:34:26.750 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.052661596, 2024-04-25T08:34:26.750 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.017553865 2024-04-25T08:34:26.750 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.002772077, 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002772077 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.751 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.005103231, 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.005103231 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6, 2024-04-25T08:34:26.752 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.092864459, 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.015477409 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2211, 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 224.863278792, 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.101702070 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-04-25T08:34:26.753 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.754 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.754 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.754 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.754 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-04-25T08:34:26.754 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.754 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.754 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.754 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.754 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-04-25T08:34:26.754 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:34:26.754 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.003313507, 2024-04-25T08:34:26.755 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.003313507 2024-04-25T08:34:26.755 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.755 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-04-25T08:34:26.755 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.755 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.755 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.755 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.755 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-04-25T08:34:26.755 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:34:26.755 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000225571, 2024-04-25T08:34:26.755 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000225571 2024-04-25T08:34:26.755 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.756 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 8000, 2024-04-25T08:34:26.757 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 15.331846815, 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001916480 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-04-25T08:34:26.758 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 1, 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 0, 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 19561, 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 19561, 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 86, 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T08:34:26.759 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T08:34:26.760 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T08:34:26.761 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 251345, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 86, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 5624, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 80, 2024-04-25T08:34:26.762 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 13437260, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 660952, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 12091, 2024-04-25T08:34:26.763 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 78272791, 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 20, 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 12091, 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 12091, 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 34.086398101, 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002819154 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 12091, 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 12163 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 12091, 2024-04-25T08:34:26.764 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 2075, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 10016, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 0, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 11, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 60, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 6522, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 37, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 1355, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 2030, 2024-04-25T08:34:26.765 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 2015, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 22, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-04-25T08:34:26.766 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-04-25T08:34:26.767 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 42, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 5, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 8, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 8, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 94, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 0, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 17 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-04-25T08:34:26.768 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 1, 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 32, 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 3355, 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 3355, 2024-04-25T08:34:26.769 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-04-25T08:34:26.770 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.770 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T08:34:26.770 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:34:26.770 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T08:34:26.770 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:34:26.770 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 44466, 2024-04-25T08:34:26.770 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 10931845, 2024-04-25T08:34:26.770 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:34:26.770 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 44466, 2024-04-25T08:34:26.770 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T08:34:26.770 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T08:34:26.770 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 44466, 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 10931845, 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:34:26.771 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 12091, 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 78622486, 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 12091, 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 78622486, 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:34:26.772 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 12091, 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 12091, 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 12091, 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 12091, 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.773 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 3355, 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 349695, 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 3355, 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 58, 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 349695, 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:34:26.774 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.775 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.775 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.775 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:26.775 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:26.775 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x557bda42c6e0": { 2024-04-25T08:34:26.775 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:34:26.775 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-25T08:34:26.775 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:34:26.775 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 23554, 2024-04-25T08:34:26.775 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 77892105, 2024-04-25T08:34:26.775 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:34:26.776 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 23554, 2024-04-25T08:34:26.776 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T08:34:26.776 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T08:34:26.776 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 6464, 2024-04-25T08:34:26.776 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 77892105, 2024-04-25T08:34:26.776 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:34:26.776 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:34:26.776 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:34:26.776 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:34:26.776 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:26.776 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:26.776 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:34:26.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.727+0000 7f158f0e1640 1 -- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f157406b500 msgr2=0x7f157406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.727+0000 7f158f0e1640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f157406b500 0x7f157406d9c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1578006fd0 tx=0x7f1578008040 comp rx=0 tx=0).stop 2024-04-25T08:34:26.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.727+0000 7f158f0e1640 1 -- 172.21.15.38:0/3816734959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0320 msgr2=0x7f1588145cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:26.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.727+0000 7f158f0e1640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0320 0x7f1588145cd0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f157c00b3f0 tx=0x7f157c00b8b0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.727+0000 7f158f0e1640 1 -- 172.21.15.38:0/3816734959 shutdown_connections 2024-04-25T08:34:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.727+0000 7f158f0e1640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15880d0700 0x7f15880b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.727+0000 7f158f0e1640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15880d0320 0x7f1588145cd0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.727+0000 7f158f0e1640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f157406b500 0x7f157406d9c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.727+0000 7f158f0e1640 1 --2- 172.21.15.38:0/3816734959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15880b6980 0x7f1588149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.727+0000 7f158f0e1640 1 -- 172.21.15.38:0/3816734959 >> 172.21.15.38:0/3816734959 conn(0x7f15880668f0 msgr2=0x7f15880ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.727+0000 7f158f0e1640 1 -- 172.21.15.38:0/3816734959 shutdown_connections 2024-04-25T08:34:26.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:26.727+0000 7f158f0e1640 1 -- 172.21.15.38:0/3816734959 wait complete. 2024-04-25T08:34:26.780 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:34:26.780 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:34:26.815 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:34:26.856 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:34:26.856 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:34:26.856 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:34:26.872 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:34:27.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.017+0000 7fcbd16bd640 1 Processor -- start 2024-04-25T08:34:27.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.017+0000 7fcbd16bd640 1 -- start start 2024-04-25T08:34:27.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.018+0000 7fcbd16bd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbcc15df50 0x7fcbcc0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.018+0000 7fcbd16bd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcbcc0ce2e0 0x7fcbcc0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.018+0000 7fcbd16bd640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbcc0ceff0 con 0x7fcbcc06c110 2024-04-25T08:34:27.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.018+0000 7fcbd16bd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbcc0cf2d0 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.018+0000 7fcbd16bd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0d1f70 con 0x7fcbcc15df50 2024-04-25T08:34:27.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.018+0000 7fcbcaffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcbcc06c110 0x7fcbcc06c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55186/0 (socket says 172.21.15.38:55186) 2024-04-25T08:34:27.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.018+0000 7fcbcaffd640 1 -- 172.21.15.38:0/2084243901 learned_addr learned my addr 172.21.15.38:0/2084243901 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:27.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.018+0000 7fcbcb7fe640 1 --2- 172.21.15.38:0/2084243901 >> v2:172.21.15.38:3300/0 conn(0x7fcbcc0ce2e0 0x7fcbcc0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.019+0000 7fcbc3fff640 1 --2- 172.21.15.38:0/2084243901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbcc15df50 0x7fcbcc0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.019+0000 7fcbca7fc640 1 -- 172.21.15.38:0/2084243901 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146556102 0 0) 0x7fcbcc0ceff0 con 0x7fcbcc06c110 2024-04-25T08:34:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.019+0000 7fcbca7fc640 1 -- 172.21.15.38:0/2084243901 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbb8003520 con 0x7fcbcc06c110 2024-04-25T08:34:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.019+0000 7fcbcb7fe640 1 -- 172.21.15.38:0/2084243901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbcc15df50 msgr2=0x7fcbcc0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.019+0000 7fcbcb7fe640 1 --2- 172.21.15.38:0/2084243901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbcc15df50 0x7fcbcc0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.019+0000 7fcbcb7fe640 1 -- 172.21.15.38:0/2084243901 >> v1:172.21.15.38:6789/0 conn(0x7fcbcc06c110 legacy=0x7fcbcc06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.019+0000 7fcbcb7fe640 1 -- 172.21.15.38:0/2084243901 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc0d2250 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.019+0000 7fcbcb7fe640 1 --2- 172.21.15.38:0/2084243901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbcc0ce2e0 0x7fcbcc0d2a50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009d90 tx=0x7fcbc402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfa8d0ab69be328 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.020+0000 7fcbca7fc640 1 -- 172.21.15.38:0/2084243901 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4007740 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.020+0000 7fcbca7fc640 1 -- 172.21.15.38:0/2084243901 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc40078e0 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.020+0000 7fcbca7fc640 1 -- 172.21.15.38:0/2084243901 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4007cb0 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.020+0000 7fcbd16bd640 1 -- 172.21.15.38:0/2084243901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbcc0ce2e0 msgr2=0x7fcbcc0d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.020+0000 7fcbd16bd640 1 --2- 172.21.15.38:0/2084243901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbcc0ce2e0 0x7fcbcc0d2a50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4009d90 tx=0x7fcbc402a660 comp rx=0 tx=0).stop 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.020+0000 7fcbd16bd640 1 -- 172.21.15.38:0/2084243901 shutdown_connections 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.020+0000 7fcbd16bd640 1 --2- 172.21.15.38:0/2084243901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbcc0ce2e0 0x7fcbcc0d2a50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.020+0000 7fcbd16bd640 1 --2- 172.21.15.38:0/2084243901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbcc15df50 0x7fcbcc0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.020+0000 7fcbd16bd640 1 -- 172.21.15.38:0/2084243901 >> 172.21.15.38:0/2084243901 conn(0x7fcbcc0d3fb0 msgr2=0x7fcbcc0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.020+0000 7fcbd16bd640 1 -- 172.21.15.38:0/2084243901 shutdown_connections 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.020+0000 7fcbd16bd640 1 -- 172.21.15.38:0/2084243901 wait complete. 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.021+0000 7fcbd16bd640 1 Processor -- start 2024-04-25T08:34:27.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.021+0000 7fcbd16bd640 1 -- start start 2024-04-25T08:34:27.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.022+0000 7fcbd16bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbcc06c110 0x7fcbcc156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.022+0000 7fcbd16bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbcc0ce2e0 0x7fcbcc154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.022+0000 7fcbd16bd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbcc15df50 0x7fcbcc0b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.022+0000 7fcbd16bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0d2f90 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.022+0000 7fcbd16bd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0cece0 con 0x7fcbcc06c110 2024-04-25T08:34:27.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.022+0000 7fcbd16bd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0d3110 con 0x7fcbcc15df50 2024-04-25T08:34:27.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.022+0000 7fcbc3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbcc0ce2e0 0x7fcbcc154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.022+0000 7fcbc3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbcc0ce2e0 0x7fcbcc154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35580/0 (socket says 172.21.15.38:35580) 2024-04-25T08:34:27.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.022+0000 7fcbc3fff640 1 -- 172.21.15.38:0/3944771326 learned_addr learned my addr 172.21.15.38:0/3944771326 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:27.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.022+0000 7fcbcb7fe640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbcc15df50 0x7fcbcc0b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.022+0000 7fcbcaffd640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbcc06c110 0x7fcbcc156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.023+0000 7fcbc3fff640 1 -- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbcc15df50 msgr2=0x7fcbcc0b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.023+0000 7fcbc3fff640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbcc15df50 0x7fcbcc0b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.023+0000 7fcbc3fff640 1 -- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbcc06c110 msgr2=0x7fcbcc156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.023+0000 7fcbc3fff640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbcc06c110 0x7fcbcc156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.023+0000 7fcbc3fff640 1 -- 172.21.15.38:0/3944771326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc15f420 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.023+0000 7fcbc3fff640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbcc0ce2e0 0x7fcbcc154ae0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400d800 tx=0x7fcbb400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:27.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.023+0000 7fcbc37fe640 1 -- 172.21.15.38:0/3944771326 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400b6c0 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.023+0000 7fcbc37fe640 1 -- 172.21.15.38:0/3944771326 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb4019070 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.023+0000 7fcbd16bd640 1 -- 172.21.15.38:0/3944771326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbcc155020 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.023+0000 7fcbc37fe640 1 -- 172.21.15.38:0/3944771326 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400ba80 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.023+0000 7fcbd16bd640 1 -- 172.21.15.38:0/3944771326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbcc15f6c0 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.024+0000 7fcbd16bd640 1 -- 172.21.15.38:0/3944771326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbcc068e80 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.025+0000 7fcbc37fe640 1 -- 172.21.15.38:0/3944771326 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbb400bc20 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.025+0000 7fcbc37fe640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcbac06b4d0 0x7fcbac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.025+0000 7fcbc37fe640 1 -- 172.21.15.38:0/3944771326 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fcbb408a7c0 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.025+0000 7fcbcaffd640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcbac06b4d0 0x7fcbac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.026+0000 7fcbcaffd640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcbac06b4d0 0x7fcbac06d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc005880 tx=0x7fcbbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:27.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.029+0000 7fcbc37fe640 1 -- 172.21.15.38:0/3944771326 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcbb4059470 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.211+0000 7fcbd16bd640 1 -- 172.21.15.38:0/3944771326 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbcc0b8cc0 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.212+0000 7fcbc37fe640 1 -- 172.21.15.38:0/3944771326 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17044 (secure 0 0 0) 0x7fcbb4058d70 con 0x7fcbcc0ce2e0 2024-04-25T08:34:27.207 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:27.208 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":42,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:34:05.884436+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:47.798666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:48.371717+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:34:27.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.214+0000 7fcbd16bd640 1 -- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcbac06b4d0 msgr2=0x7fcbac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.214+0000 7fcbd16bd640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcbac06b4d0 0x7fcbac06d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc005880 tx=0x7fcbbc009040 comp rx=0 tx=0).stop 2024-04-25T08:34:27.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.214+0000 7fcbd16bd640 1 -- 172.21.15.38:0/3944771326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbcc0ce2e0 msgr2=0x7fcbcc154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.214+0000 7fcbd16bd640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbcc0ce2e0 0x7fcbcc154ae0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400d800 tx=0x7fcbb400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.214+0000 7fcbd16bd640 1 -- 172.21.15.38:0/3944771326 shutdown_connections 2024-04-25T08:34:27.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.214+0000 7fcbd16bd640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcbac06b4d0 0x7fcbac06d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.214+0000 7fcbd16bd640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcbcc15df50 0x7fcbcc0b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.214+0000 7fcbd16bd640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcbcc0ce2e0 0x7fcbcc154ae0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.214+0000 7fcbd16bd640 1 --2- 172.21.15.38:0/3944771326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcbcc06c110 0x7fcbcc156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.214+0000 7fcbd16bd640 1 -- 172.21.15.38:0/3944771326 >> 172.21.15.38:0/3944771326 conn(0x7fcbcc0d3fb0 msgr2=0x7fcbcc0d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:27.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.214+0000 7fcbd16bd640 1 -- 172.21.15.38:0/3944771326 shutdown_connections 2024-04-25T08:34:27.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.214+0000 7fcbd16bd640 1 -- 172.21.15.38:0/3944771326 wait complete. 2024-04-25T08:34:27.220 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:32:47.798666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:32:48.371717+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 41, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:34:27.221 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:34:27.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.329+0000 7ffbec31d640 1 Processor -- start 2024-04-25T08:34:27.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.329+0000 7ffbec31d640 1 -- start start 2024-04-25T08:34:27.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.330+0000 7ffbec31d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ffbe415db80 0x7ffbe40cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.330+0000 7ffbec31d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffbe40cfb70 0x7ffbe40ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.330+0000 7ffbec31d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe40ac430 con 0x7ffbe40ccfc0 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.330+0000 7ffbec31d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe40cffe0 con 0x7ffbe40cfb70 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.330+0000 7ffbec31d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe40ac1e0 con 0x7ffbe415db80 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.331+0000 7ffbea8ba640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffbe40cfb70 0x7ffbe40ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.331+0000 7ffbe98b8640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ffbe40ccfc0 0x7ffbe40c9ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55196/0 (socket says 172.21.15.38:55196) 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.331+0000 7ffbe98b8640 1 -- 172.21.15.38:0/2290713062 learned_addr learned my addr 172.21.15.38:0/2290713062 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.331+0000 7ffbea8ba640 1 -- 172.21.15.38:0/2290713062 >> v2:172.21.15.107:3300/0 conn(0x7ffbe415db80 msgr2=0x7ffbe40cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.331+0000 7ffbea0b9640 1 --2- 172.21.15.38:0/2290713062 >> v2:172.21.15.107:3300/0 conn(0x7ffbe415db80 0x7ffbe40cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.331+0000 7ffbea8ba640 1 --2- 172.21.15.38:0/2290713062 >> v2:172.21.15.107:3300/0 conn(0x7ffbe415db80 0x7ffbe40cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.331+0000 7ffbe90b7640 1 -- 172.21.15.38:0/2290713062 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410373524 0 0) 0x7ffbe40ac430 con 0x7ffbe40ccfc0 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.331+0000 7ffbea8ba640 1 -- 172.21.15.38:0/2290713062 >> v1:172.21.15.38:6789/0 conn(0x7ffbe40ccfc0 legacy=0x7ffbe40c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.331+0000 7ffbea8ba640 1 -- 172.21.15.38:0/2290713062 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe40ac710 con 0x7ffbe40cfb70 2024-04-25T08:34:27.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.331+0000 7ffbea0b9640 1 --2- 172.21.15.38:0/2290713062 >> v2:172.21.15.107:3300/0 conn(0x7ffbe415db80 0x7ffbe40cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:27.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.332+0000 7ffbea8ba640 1 --2- 172.21.15.38:0/2290713062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe40cfb70 0x7ffbe40ca1e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffbe406bbc0 tx=0x7ffbdc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70d6ea4e983c75d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:27.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.332+0000 7ffbe90b7640 1 -- 172.21.15.38:0/2290713062 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc007740 con 0x7ffbe40cfb70 2024-04-25T08:34:27.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.332+0000 7ffbe90b7640 1 -- 172.21.15.38:0/2290713062 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc0078e0 con 0x7ffbe40cfb70 2024-04-25T08:34:27.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.332+0000 7ffbe90b7640 1 -- 172.21.15.38:0/2290713062 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc007cb0 con 0x7ffbe40cfb70 2024-04-25T08:34:27.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.332+0000 7ffbec31d640 1 -- 172.21.15.38:0/2290713062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe40cfb70 msgr2=0x7ffbe40ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.332+0000 7ffbec31d640 1 --2- 172.21.15.38:0/2290713062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe40cfb70 0x7ffbe40ca1e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ffbe406bbc0 tx=0x7ffbdc02a680 comp rx=0 tx=0).stop 2024-04-25T08:34:27.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.332+0000 7ffbec31d640 1 -- 172.21.15.38:0/2290713062 shutdown_connections 2024-04-25T08:34:27.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.332+0000 7ffbec31d640 1 --2- 172.21.15.38:0/2290713062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe40cfb70 0x7ffbe40ca1e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.332+0000 7ffbec31d640 1 --2- 172.21.15.38:0/2290713062 >> v2:172.21.15.107:3300/0 conn(0x7ffbe415db80 0x7ffbe40cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.332+0000 7ffbec31d640 1 -- 172.21.15.38:0/2290713062 >> 172.21.15.38:0/2290713062 conn(0x7ffbe406adc0 msgr2=0x7ffbe406b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:27.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.332+0000 7ffbec31d640 1 -- 172.21.15.38:0/2290713062 shutdown_connections 2024-04-25T08:34:27.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.332+0000 7ffbec31d640 1 -- 172.21.15.38:0/2290713062 wait complete. 2024-04-25T08:34:27.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.334+0000 7ffbec31d640 1 Processor -- start 2024-04-25T08:34:27.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.334+0000 7ffbec31d640 1 -- start start 2024-04-25T08:34:27.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.334+0000 7ffbec31d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbe40ccfc0 0x7ffbe40c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.334+0000 7ffbec31d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbe40cfb70 0x7ffbe40afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.334+0000 7ffbec31d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe415db80 0x7ffbe40aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.334+0000 7ffbec31d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe415df60 con 0x7ffbe415db80 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.334+0000 7ffbec31d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe40accd0 con 0x7ffbe40cfb70 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.334+0000 7ffbec31d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe415e2c0 con 0x7ffbe40ccfc0 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.334+0000 7ffbea8ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe415db80 0x7ffbe40aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.334+0000 7ffbea8ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe415db80 0x7ffbe40aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35610/0 (socket says 172.21.15.38:35610) 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.334+0000 7ffbea8ba640 1 -- 172.21.15.38:0/3774568617 learned_addr learned my addr 172.21.15.38:0/3774568617 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbe98b8640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbe40cfb70 0x7ffbe40afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbea0b9640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbe40ccfc0 0x7ffbe40c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbea8ba640 1 -- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbe40ccfc0 msgr2=0x7ffbe40c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbea8ba640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbe40ccfc0 0x7ffbe40c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbea8ba640 1 -- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbe40cfb70 msgr2=0x7ffbe40afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbea8ba640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbe40cfb70 0x7ffbe40afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbea8ba640 1 -- 172.21.15.38:0/3774568617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe415c870 con 0x7ffbe415db80 2024-04-25T08:34:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbea0b9640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbe40ccfc0 0x7ffbe40c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:27.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbea8ba640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe415db80 0x7ffbe40aff60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc02ac70 tx=0x7ffbdc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:27.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbdaffd640 1 -- 172.21.15.38:0/3774568617 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc040070 con 0x7ffbe415db80 2024-04-25T08:34:27.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbdaffd640 1 -- 172.21.15.38:0/3774568617 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbdc031c90 con 0x7ffbe415db80 2024-04-25T08:34:27.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbec31d640 1 -- 172.21.15.38:0/3774568617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe40c53e0 con 0x7ffbe415db80 2024-04-25T08:34:27.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbdaffd640 1 -- 172.21.15.38:0/3774568617 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc045680 con 0x7ffbe415db80 2024-04-25T08:34:27.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.335+0000 7ffbec31d640 1 -- 172.21.15.38:0/3774568617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe40c56c0 con 0x7ffbe415db80 2024-04-25T08:34:27.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.336+0000 7ffbec31d640 1 -- 172.21.15.38:0/3774568617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe4068e80 con 0x7ffbe415db80 2024-04-25T08:34:27.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.337+0000 7ffbdaffd640 1 -- 172.21.15.38:0/3774568617 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbdc039040 con 0x7ffbe415db80 2024-04-25T08:34:27.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.337+0000 7ffbdaffd640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffbc806b4d0 0x7ffbc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.337+0000 7ffbdaffd640 1 -- 172.21.15.38:0/3774568617 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7ffbdc0abdb0 con 0x7ffbe415db80 2024-04-25T08:34:27.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.337+0000 7ffbea0b9640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffbc806b4d0 0x7ffbc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.338+0000 7ffbea0b9640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffbc806b4d0 0x7ffbc806d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0006fd0 tx=0x7ffbe0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:27.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.341+0000 7ffbdaffd640 1 -- 172.21.15.38:0/3774568617 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbdc043360 con 0x7ffbe415db80 2024-04-25T08:34:27.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.519+0000 7ffbec31d640 1 -- 172.21.15.38:0/3774568617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbe40b4230 con 0x7ffbe415db80 2024-04-25T08:34:27.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.520+0000 7ffbdaffd640 1 -- 172.21.15.38:0/3774568617 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffbdc07ab30 con 0x7ffbe415db80 2024-04-25T08:34:27.515 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:34:27.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.522+0000 7ffbec31d640 1 -- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffbc806b4d0 msgr2=0x7ffbc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.522+0000 7ffbec31d640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffbc806b4d0 0x7ffbc806d990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0006fd0 tx=0x7ffbe0008040 comp rx=0 tx=0).stop 2024-04-25T08:34:27.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.522+0000 7ffbec31d640 1 -- 172.21.15.38:0/3774568617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe415db80 msgr2=0x7ffbe40aff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.522+0000 7ffbec31d640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe415db80 0x7ffbe40aff60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc02ac70 tx=0x7ffbdc004a40 comp rx=0 tx=0).stop 2024-04-25T08:34:27.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.522+0000 7ffbec31d640 1 -- 172.21.15.38:0/3774568617 shutdown_connections 2024-04-25T08:34:27.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.522+0000 7ffbec31d640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbe415db80 0x7ffbe40aff60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.522+0000 7ffbec31d640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbe40cfb70 0x7ffbe40afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.522+0000 7ffbec31d640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffbc806b4d0 0x7ffbc806d990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.522+0000 7ffbec31d640 1 --2- 172.21.15.38:0/3774568617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbe40ccfc0 0x7ffbe40c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.522+0000 7ffbec31d640 1 -- 172.21.15.38:0/3774568617 >> 172.21.15.38:0/3774568617 conn(0x7ffbe406adc0 msgr2=0x7ffbe40cbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:27.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.522+0000 7ffbec31d640 1 -- 172.21.15.38:0/3774568617 shutdown_connections 2024-04-25T08:34:27.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.522+0000 7ffbec31d640 1 -- 172.21.15.38:0/3774568617 wait complete. 2024-04-25T08:34:27.528 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:34:27.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.636+0000 7fdc18329640 1 Processor -- start 2024-04-25T08:34:27.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.636+0000 7fdc18329640 1 -- start start 2024-04-25T08:34:27.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc18329640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc100d0320 0x7fdc100d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc18329640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc100d2060 con 0x7fdc100d0700 2024-04-25T08:34:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc18329640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc100d2330 con 0x7fdc100b6980 2024-04-25T08:34:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc18329640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc100d2610 con 0x7fdc100d0320 2024-04-25T08:34:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc158c4640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdc100d0700 0x7fdc100d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55208/0 (socket says 172.21.15.38:55208) 2024-04-25T08:34:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc158c4640 1 -- 172.21.15.38:0/1515039514 learned_addr learned my addr 172.21.15.38:0/1515039514 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc160c5640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc100d0320 0x7fdc100d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868058721 0 0) 0x7fdc100d2060 con 0x7fdc100d0700 2024-04-25T08:34:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbfc003520 con 0x7fdc100d0700 2024-04-25T08:34:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632823061 0 0) 0x7fdc100d2330 con 0x7fdc100b6980 2024-04-25T08:34:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc100d2060 con 0x7fdc100b6980 2024-04-25T08:34:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1724073422 0 0) 0x7fdbfc003520 con 0x7fdc100d0700 2024-04-25T08:34:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc100d0320 msgr2=0x7fdc100d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc150c3640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc100d0320 0x7fdc100d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 >> v1:172.21.15.107:6789/0 conn(0x7fdc100b6980 legacy=0x7fdc100b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc100d28f0 con 0x7fdc100d0700 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fdc00001670 con 0x7fdc100d0700 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.637+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 >> v1:172.21.15.38:6789/0 conn(0x7fdc100d0700 legacy=0x7fdc100d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc150c3640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbfc003b70 0x7fdbfc003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc150c3640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbfc004800 0x7fdbfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc150c3640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbfc0051c0 0x7fdbfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc100d2610 con 0x7fdbfc0051c0 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc00bf60 con 0x7fdbfc003b70 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc00c160 con 0x7fdbfc004800 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc160c5640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc100d0320 0x7fdc100d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc168c6640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbfc0051c0 0x7fdbfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc158c4640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbfc004800 0x7fdbfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc160c5640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbfc003b70 0x7fdbfc003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc168c6640 1 -- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbfc004800 msgr2=0x7fdbfc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc168c6640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbfc004800 0x7fdbfc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc168c6640 1 -- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbfc003b70 msgr2=0x7fdbfc003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc168c6640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbfc003b70 0x7fdbfc003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc168c6640 1 -- 172.21.15.38:0/1515039514 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc003520 con 0x7fdbfc0051c0 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.638+0000 7fdc160c5640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbfc003b70 0x7fdbfc003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.639+0000 7fdc168c6640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbfc0051c0 0x7fdbfc00b9c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdc08001fd0 tx=0x7fdbfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ad4536d08129a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.639+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc08001bf0 con 0x7fdbfc0051c0 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.639+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc08003b70 con 0x7fdbfc0051c0 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.639+0000 7fdc150c3640 1 -- 172.21.15.38:0/1515039514 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc080050d0 con 0x7fdbfc0051c0 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.639+0000 7fdc18329640 1 -- 172.21.15.38:0/1515039514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbfc0051c0 msgr2=0x7fdbfc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.639+0000 7fdc18329640 1 --2- 172.21.15.38:0/1515039514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbfc0051c0 0x7fdbfc00b9c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdc08001fd0 tx=0x7fdbfc005830 comp rx=0 tx=0).stop 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.639+0000 7fdc168c6640 1 -- 172.21.15.38:0/1515039514 reap_dead start 2024-04-25T08:34:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.639+0000 7fdc18329640 1 -- 172.21.15.38:0/1515039514 shutdown_connections 2024-04-25T08:34:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.639+0000 7fdc18329640 1 -- 172.21.15.38:0/1515039514 >> 172.21.15.38:0/1515039514 conn(0x7fdc100668f0 msgr2=0x7fdc10066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.640+0000 7fdc18329640 1 -- 172.21.15.38:0/1515039514 shutdown_connections 2024-04-25T08:34:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.640+0000 7fdc18329640 1 -- 172.21.15.38:0/1515039514 wait complete. 2024-04-25T08:34:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.641+0000 7fdc18329640 1 Processor -- start 2024-04-25T08:34:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.641+0000 7fdc18329640 1 -- start start 2024-04-25T08:34:27.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.641+0000 7fdc18329640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbfc0051c0 0x7fdc100b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.641+0000 7fdc18329640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc101e6470 0x7fdc100b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.641+0000 7fdc18329640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc1015ab00 0x7fdc1015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.641+0000 7fdc18329640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc100d0b40 con 0x7fdc101e6470 2024-04-25T08:34:27.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.641+0000 7fdc18329640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc100d2eb0 con 0x7fdc1015ab00 2024-04-25T08:34:27.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.641+0000 7fdc18329640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc101e8960 con 0x7fdbfc0051c0 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc158c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc101e6470 0x7fdc100b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc158c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc101e6470 0x7fdc100b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35636/0 (socket says 172.21.15.38:35636) 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc158c4640 1 -- 172.21.15.38:0/3822728610 learned_addr learned my addr 172.21.15.38:0/3822728610 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc160c5640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbfc0051c0 0x7fdc100b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc158c4640 1 -- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbfc0051c0 msgr2=0x7fdc100b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc158c4640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbfc0051c0 0x7fdc100b80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc158c4640 1 -- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc1015ab00 msgr2=0x7fdc1015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc158c4640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc1015ab00 0x7fdc1015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc158c4640 1 -- 172.21.15.38:0/3822728610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc101e8660 con 0x7fdc101e6470 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc158c4640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc101e6470 0x7fdc100b50d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdc00007290 tx=0x7fdc00007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc06ffd640 1 -- 172.21.15.38:0/3822728610 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00001670 con 0x7fdc101e6470 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc06ffd640 1 -- 172.21.15.38:0/3822728610 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc00018070 con 0x7fdc101e6470 2024-04-25T08:34:27.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc18329640 1 -- 172.21.15.38:0/3822728610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc101e6930 con 0x7fdc101e6470 2024-04-25T08:34:27.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc06ffd640 1 -- 172.21.15.38:0/3822728610 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000fb60 con 0x7fdc101e6470 2024-04-25T08:34:27.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.642+0000 7fdc18329640 1 -- 172.21.15.38:0/3822728610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc101e6ef0 con 0x7fdc101e6470 2024-04-25T08:34:27.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.643+0000 7fdc18329640 1 -- 172.21.15.38:0/3822728610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc10068e80 con 0x7fdc101e6470 2024-04-25T08:34:27.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.644+0000 7fdc06ffd640 1 -- 172.21.15.38:0/3822728610 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc0000fd00 con 0x7fdc101e6470 2024-04-25T08:34:27.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.644+0000 7fdc06ffd640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdbf406b4d0 0x7fdbf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.644+0000 7fdc06ffd640 1 -- 172.21.15.38:0/3822728610 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fdc0008a3d0 con 0x7fdc101e6470 2024-04-25T08:34:27.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.644+0000 7fdc160c5640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdbf406b4d0 0x7fdbf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.645+0000 7fdc160c5640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdbf406b4d0 0x7fdbf406d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc10069100 tx=0x7fdc0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:27.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.648+0000 7fdc06ffd640 1 -- 172.21.15.38:0/3822728610 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc000589a0 con 0x7fdc101e6470 2024-04-25T08:34:27.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.826+0000 7fdc18329640 1 -- 172.21.15.38:0/3822728610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdc101e8ae0 con 0x7fdc101e6470 2024-04-25T08:34:27.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.826+0000 7fdc06ffd640 1 -- 172.21.15.38:0/3822728610 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdc0005ca40 con 0x7fdc101e6470 2024-04-25T08:34:27.822 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:34:27.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.828+0000 7fdc18329640 1 -- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdbf406b4d0 msgr2=0x7fdbf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.828+0000 7fdc18329640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdbf406b4d0 0x7fdbf406d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc10069100 tx=0x7fdc0c008040 comp rx=0 tx=0).stop 2024-04-25T08:34:27.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.828+0000 7fdc18329640 1 -- 172.21.15.38:0/3822728610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc101e6470 msgr2=0x7fdc100b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.828+0000 7fdc18329640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc101e6470 0x7fdc100b50d0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdc00007290 tx=0x7fdc00007750 comp rx=0 tx=0).stop 2024-04-25T08:34:27.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.829+0000 7fdc18329640 1 -- 172.21.15.38:0/3822728610 shutdown_connections 2024-04-25T08:34:27.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.829+0000 7fdc18329640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc1015ab00 0x7fdc1015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.829+0000 7fdc18329640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc101e6470 0x7fdc100b50d0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.829+0000 7fdc18329640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdbf406b4d0 0x7fdbf406d990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.829+0000 7fdc18329640 1 --2- 172.21.15.38:0/3822728610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbfc0051c0 0x7fdc100b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.829+0000 7fdc18329640 1 -- 172.21.15.38:0/3822728610 >> 172.21.15.38:0/3822728610 conn(0x7fdc100668f0 msgr2=0x7fdc100b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:27.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.829+0000 7fdc18329640 1 -- 172.21.15.38:0/3822728610 shutdown_connections 2024-04-25T08:34:27.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.829+0000 7fdc18329640 1 -- 172.21.15.38:0/3822728610 wait complete. 2024-04-25T08:34:27.834 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:34:27.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.944+0000 7f2168bab640 1 Processor -- start 2024-04-25T08:34:27.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.944+0000 7f2168bab640 1 -- start start 2024-04-25T08:34:27.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2168bab640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f216415db80 0x7f21640cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2168bab640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21640ccfc0 0x7f21640cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2168bab640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f21640c9ad0 0x7f21640c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2168bab640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21640ac1e0 con 0x7f21640c9ad0 2024-04-25T08:34:27.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2168bab640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21640ac430 con 0x7f216415db80 2024-04-25T08:34:27.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2168bab640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21640ac710 con 0x7f21640ccfc0 2024-04-25T08:34:27.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2162d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f21640c9ad0 0x7f21640c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2162d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f21640c9ad0 0x7f21640c9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35640/0 (socket says 172.21.15.38:35640) 2024-04-25T08:34:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2162d9d640 1 -- 172.21.15.38:0/3229808625 learned_addr learned my addr 172.21.15.38:0/3229808625 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f215bfff640 1 --2- 172.21.15.38:0/3229808625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21640ccfc0 0x7f21640cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f216259c640 1 --2- 172.21.15.38:0/3229808625 >> v2:172.21.15.107:3300/0 conn(0x7f216415db80 0x7f21640cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2162d9d640 1 -- 172.21.15.38:0/3229808625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21640ccfc0 msgr2=0x7f21640cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2162d9d640 1 --2- 172.21.15.38:0/3229808625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21640ccfc0 0x7f21640cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2162d9d640 1 -- 172.21.15.38:0/3229808625 >> v2:172.21.15.107:3300/0 conn(0x7f216415db80 msgr2=0x7f21640cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2162d9d640 1 --2- 172.21.15.38:0/3229808625 >> v2:172.21.15.107:3300/0 conn(0x7f216415db80 0x7f21640cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.945+0000 7f2162d9d640 1 -- 172.21.15.38:0/3229808625 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21640ac9f0 con 0x7f21640c9ad0 2024-04-25T08:34:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.946+0000 7f2162d9d640 1 --2- 172.21.15.38:0/3229808625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21640c9ad0 0x7f21640c9f60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f216406bbc0 tx=0x7f215c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16b170c15c3973d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.946+0000 7f2161d9b640 1 -- 172.21.15.38:0/3229808625 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c036070 con 0x7f21640c9ad0 2024-04-25T08:34:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.946+0000 7f2161d9b640 1 -- 172.21.15.38:0/3229808625 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c031600 con 0x7f21640c9ad0 2024-04-25T08:34:27.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.946+0000 7f2161d9b640 1 -- 172.21.15.38:0/3229808625 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c0319d0 con 0x7f21640c9ad0 2024-04-25T08:34:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.947+0000 7f2168bab640 1 -- 172.21.15.38:0/3229808625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21640c9ad0 msgr2=0x7f21640c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.947+0000 7f2168bab640 1 --2- 172.21.15.38:0/3229808625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21640c9ad0 0x7f21640c9f60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f216406bbc0 tx=0x7f215c02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.947+0000 7f2168bab640 1 -- 172.21.15.38:0/3229808625 shutdown_connections 2024-04-25T08:34:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.947+0000 7f2168bab640 1 --2- 172.21.15.38:0/3229808625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21640c9ad0 0x7f21640c9f60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.947+0000 7f2168bab640 1 --2- 172.21.15.38:0/3229808625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21640ccfc0 0x7f21640cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.947+0000 7f2168bab640 1 --2- 172.21.15.38:0/3229808625 >> v2:172.21.15.107:3300/0 conn(0x7f216415db80 0x7f21640cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.947+0000 7f2168bab640 1 -- 172.21.15.38:0/3229808625 >> 172.21.15.38:0/3229808625 conn(0x7f216406adc0 msgr2=0x7f216406b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.947+0000 7f2168bab640 1 -- 172.21.15.38:0/3229808625 shutdown_connections 2024-04-25T08:34:27.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.947+0000 7f2168bab640 1 -- 172.21.15.38:0/3229808625 wait complete. 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.948+0000 7f2168bab640 1 Processor -- start 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.948+0000 7f2168bab640 1 -- start start 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2168bab640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21640c9ad0 0x7f216415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2168bab640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21640ccfc0 0x7f21641f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2168bab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f216415db80 0x7f21640cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2168bab640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21640ca4a0 con 0x7f216415db80 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2168bab640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216415df60 con 0x7f21640ccfc0 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2168bab640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21640ca620 con 0x7f21640c9ad0 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2162d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f216415db80 0x7f21640cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2162d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f216415db80 0x7f21640cd430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35642/0 (socket says 172.21.15.38:35642) 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2162d9d640 1 -- 172.21.15.38:0/3552278089 learned_addr learned my addr 172.21.15.38:0/3552278089 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f216259c640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21640c9ad0 0x7f216415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f215bfff640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21640ccfc0 0x7f21641f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2162d9d640 1 -- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21640c9ad0 msgr2=0x7f216415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2162d9d640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21640c9ad0 0x7f216415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2162d9d640 1 -- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21640ccfc0 msgr2=0x7f21641f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2162d9d640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21640ccfc0 0x7f21641f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2162d9d640 1 -- 172.21.15.38:0/3552278089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21640c7470 con 0x7f216415db80 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f215bfff640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21640ccfc0 0x7f21641f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f2162d9d640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f216415db80 0x7f21640cd430 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f215c009f40 tx=0x7f215c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.949+0000 7f215affd640 1 -- 172.21.15.38:0/3552278089 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c036050 con 0x7f216415db80 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.950+0000 7f215affd640 1 -- 172.21.15.38:0/3552278089 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215c042070 con 0x7f216415db80 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.950+0000 7f215affd640 1 -- 172.21.15.38:0/3552278089 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215c009510 con 0x7f216415db80 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.950+0000 7f2168bab640 1 -- 172.21.15.38:0/3552278089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21640c7750 con 0x7f216415db80 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.950+0000 7f2168bab640 1 -- 172.21.15.38:0/3552278089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21640c7d10 con 0x7f216415db80 2024-04-25T08:34:27.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.951+0000 7f2168bab640 1 -- 172.21.15.38:0/3552278089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2164067f50 con 0x7f216415db80 2024-04-25T08:34:27.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.951+0000 7f215affd640 1 -- 172.21.15.38:0/3552278089 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f215c007bf0 con 0x7f216415db80 2024-04-25T08:34:27.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.951+0000 7f215affd640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f214406b4d0 0x7f214406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:27.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.951+0000 7f215affd640 1 -- 172.21.15.38:0/3552278089 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f215c0ab1f0 con 0x7f216415db80 2024-04-25T08:34:27.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.952+0000 7f216259c640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f214406b4d0 0x7f214406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:27.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.952+0000 7f216259c640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f214406b4d0 0x7f214406d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f21540099b0 tx=0x7f2154008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:27.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:27.956+0000 7f215affd640 1 -- 172.21.15.38:0/3552278089 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2164067f50 con 0x7f216415db80 2024-04-25T08:34:28.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.134+0000 7f2168bab640 1 -- 172.21.15.38:0/3552278089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2164068e80 con 0x7f216415db80 2024-04-25T08:34:28.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.134+0000 7f215affd640 1 -- 172.21.15.38:0/3552278089 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f215c045360 con 0x7f216415db80 2024-04-25T08:34:28.129 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:34:28.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.136+0000 7f2168bab640 1 -- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f214406b4d0 msgr2=0x7f214406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:28.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.136+0000 7f2168bab640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f214406b4d0 0x7f214406d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f21540099b0 tx=0x7f2154008040 comp rx=0 tx=0).stop 2024-04-25T08:34:28.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.136+0000 7f2168bab640 1 -- 172.21.15.38:0/3552278089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f216415db80 msgr2=0x7f21640cd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:28.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.136+0000 7f2168bab640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f216415db80 0x7f21640cd430 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f215c009f40 tx=0x7f215c004a40 comp rx=0 tx=0).stop 2024-04-25T08:34:28.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.136+0000 7f2168bab640 1 -- 172.21.15.38:0/3552278089 shutdown_connections 2024-04-25T08:34:28.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.136+0000 7f2168bab640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f216415db80 0x7f21640cd430 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.136+0000 7f2168bab640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21640ccfc0 0x7f21641f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.136+0000 7f2168bab640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f214406b4d0 0x7f214406d990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.136+0000 7f2168bab640 1 --2- 172.21.15.38:0/3552278089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21640c9ad0 0x7f216415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.136+0000 7f2168bab640 1 -- 172.21.15.38:0/3552278089 >> 172.21.15.38:0/3552278089 conn(0x7f216406adc0 msgr2=0x7f216406d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:28.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.136+0000 7f2168bab640 1 -- 172.21.15.38:0/3552278089 shutdown_connections 2024-04-25T08:34:28.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.136+0000 7f2168bab640 1 -- 172.21.15.38:0/3552278089 wait complete. 2024-04-25T08:34:28.142 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:34:28.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.249+0000 7f2db7190640 1 Processor -- start 2024-04-25T08:34:28.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.249+0000 7f2db7190640 1 -- start start 2024-04-25T08:34:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2db7190640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2db015d900 0x7f2db00b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2db7190640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2db00d1870 0x7f2db00b9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2db7190640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2db015dce0 con 0x7f2db00ce500 2024-04-25T08:34:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2db7190640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2db00b7b20 con 0x7f2db015d900 2024-04-25T08:34:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2db7190640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00d1cb0 con 0x7f2db00d1870 2024-04-25T08:34:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2daffff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2db00d1870 0x7f2db00b9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2db4f2c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2db015d900 0x7f2db00b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2db572d640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f2db00ce500 0x7f2db00ce950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:59888/0 (socket says 172.21.15.38:59888) 2024-04-25T08:34:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2db572d640 1 -- 172.21.15.38:0/2328781477 learned_addr learned my addr 172.21.15.38:0/2328781477 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2daffff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2db00d1870 0x7f2db00b9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:37088/0 (socket says 172.21.15.38:37088) 2024-04-25T08:34:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2db4f2c640 1 -- 172.21.15.38:0/2328781477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2db00d1870 msgr2=0x7f2db00b9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:28.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2db4f2c640 1 --2- 172.21.15.38:0/2328781477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2db00d1870 0x7f2db00b9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2db4f2c640 1 -- 172.21.15.38:0/2328781477 >> v1:172.21.15.107:6789/0 conn(0x7f2db00ce500 legacy=0x7f2db00ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:28.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2db4f2c640 1 -- 172.21.15.38:0/2328781477 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00b6f10 con 0x7f2db015d900 2024-04-25T08:34:28.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.250+0000 7f2daf7fe640 1 -- 172.21.15.38:0/2328781477 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339424348 0 0) 0x7f2db015dce0 con 0x7f2db00ce500 2024-04-25T08:34:28.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.251+0000 7f2db4f2c640 1 --2- 172.21.15.38:0/2328781477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2db015d900 0x7f2db00b68a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2db00b7ca0 tx=0x7f2da402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8104723491f28e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:28.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.251+0000 7f2daf7fe640 1 -- 172.21.15.38:0/2328781477 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4037070 con 0x7f2db015d900 2024-04-25T08:34:28.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.251+0000 7f2daf7fe640 1 -- 172.21.15.38:0/2328781477 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da4032440 con 0x7f2db015d900 2024-04-25T08:34:28.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.251+0000 7f2daf7fe640 1 -- 172.21.15.38:0/2328781477 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da4032810 con 0x7f2db015d900 2024-04-25T08:34:28.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.252+0000 7f2db7190640 1 -- 172.21.15.38:0/2328781477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2db015d900 msgr2=0x7f2db00b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:28.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.252+0000 7f2db7190640 1 --2- 172.21.15.38:0/2328781477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2db015d900 0x7f2db00b68a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2db00b7ca0 tx=0x7f2da402a180 comp rx=0 tx=0).stop 2024-04-25T08:34:28.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.252+0000 7f2db7190640 1 -- 172.21.15.38:0/2328781477 shutdown_connections 2024-04-25T08:34:28.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.252+0000 7f2db7190640 1 --2- 172.21.15.38:0/2328781477 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2db00d1870 0x7f2db00b9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.252+0000 7f2db7190640 1 --2- 172.21.15.38:0/2328781477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2db015d900 0x7f2db00b68a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.252+0000 7f2db7190640 1 -- 172.21.15.38:0/2328781477 >> 172.21.15.38:0/2328781477 conn(0x7f2db0066d10 msgr2=0x7f2db0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:28.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.252+0000 7f2db7190640 1 -- 172.21.15.38:0/2328781477 shutdown_connections 2024-04-25T08:34:28.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.252+0000 7f2db7190640 1 -- 172.21.15.38:0/2328781477 wait complete. 2024-04-25T08:34:28.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.253+0000 7f2db7190640 1 Processor -- start 2024-04-25T08:34:28.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.253+0000 7f2db7190640 1 -- start start 2024-04-25T08:34:28.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2db7190640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2db00ce500 0x7f2db00c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:28.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2db7190640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2db00d1870 0x7f2db0152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:28.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2db7190640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2db015d900 0x7f2db0153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:28.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2db7190640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00b77b0 con 0x7f2db00d1870 2024-04-25T08:34:28.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2db7190640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00b74d0 con 0x7f2db015d900 2024-04-25T08:34:28.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2db7190640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db00b96b0 con 0x7f2db00ce500 2024-04-25T08:34:28.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2daffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2db00d1870 0x7f2db0152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:28.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2daffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2db00d1870 0x7f2db0152a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35648/0 (socket says 172.21.15.38:35648) 2024-04-25T08:34:28.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2daffff640 1 -- 172.21.15.38:0/4163437825 learned_addr learned my addr 172.21.15.38:0/4163437825 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:28.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2db4f2c640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2db00ce500 0x7f2db00c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2db572d640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2db015d900 0x7f2db0153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2daffff640 1 -- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2db00ce500 msgr2=0x7f2db00c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2daffff640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2db00ce500 0x7f2db00c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2daffff640 1 -- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2db015d900 msgr2=0x7f2db0153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2daffff640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2db015d900 0x7f2db0153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2daffff640 1 -- 172.21.15.38:0/4163437825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db00c9890 con 0x7f2db00d1870 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.254+0000 7f2db572d640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2db015d900 0x7f2db0153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.255+0000 7f2daffff640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2db00d1870 0x7f2db0152a80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00e8a0 tx=0x7f2d9c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.255+0000 7f2dad7fa640 1 -- 172.21.15.38:0/4163437825 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c017070 con 0x7f2db00d1870 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.255+0000 7f2dad7fa640 1 -- 172.21.15.38:0/4163437825 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9c00c940 con 0x7f2db00d1870 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.255+0000 7f2db7190640 1 -- 172.21.15.38:0/4163437825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db00c9b70 con 0x7f2db00d1870 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.255+0000 7f2dad7fa640 1 -- 172.21.15.38:0/4163437825 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9c00cd30 con 0x7f2db00d1870 2024-04-25T08:34:28.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.255+0000 7f2db7190640 1 -- 172.21.15.38:0/4163437825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2db00cb660 con 0x7f2db00d1870 2024-04-25T08:34:28.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.256+0000 7f2db7190640 1 -- 172.21.15.38:0/4163437825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2db004ef90 con 0x7f2db00d1870 2024-04-25T08:34:28.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.256+0000 7f2dad7fa640 1 -- 172.21.15.38:0/4163437825 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d9c01a070 con 0x7f2db00d1870 2024-04-25T08:34:28.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.257+0000 7f2dad7fa640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2d9406b4d0 0x7f2d9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:28.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.257+0000 7f2dad7fa640 1 -- 172.21.15.38:0/4163437825 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f2d9c08a640 con 0x7f2db00d1870 2024-04-25T08:34:28.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.257+0000 7f2db4f2c640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2d9406b4d0 0x7f2d9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:28.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.258+0000 7f2db4f2c640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2d9406b4d0 0x7f2d9406d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2da4034000 tx=0x7f2da402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:28.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.261+0000 7f2dad7fa640 1 -- 172.21.15.38:0/4163437825 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d9c059300 con 0x7f2db00d1870 2024-04-25T08:34:28.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.455+0000 7f2db7190640 1 -- 172.21.15.38:0/4163437825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2db0158e40 con 0x7f2db00d1870 2024-04-25T08:34:28.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.456+0000 7f2dad7fa640 1 -- 172.21.15.38:0/4163437825 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2d9c058c00 con 0x7f2db00d1870 2024-04-25T08:34:28.454 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:34:28.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.458+0000 7f2db7190640 1 -- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2d9406b4d0 msgr2=0x7f2d9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:28.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.458+0000 7f2db7190640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2d9406b4d0 0x7f2d9406d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2da4034000 tx=0x7f2da402e000 comp rx=0 tx=0).stop 2024-04-25T08:34:28.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.458+0000 7f2db7190640 1 -- 172.21.15.38:0/4163437825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2db00d1870 msgr2=0x7f2db0152a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.458+0000 7f2db7190640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2db00d1870 0x7f2db0152a80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c00e8a0 tx=0x7f2d9c00ed60 comp rx=0 tx=0).stop 2024-04-25T08:34:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.458+0000 7f2db7190640 1 -- 172.21.15.38:0/4163437825 shutdown_connections 2024-04-25T08:34:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.458+0000 7f2db7190640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2db015d900 0x7f2db0153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.458+0000 7f2db7190640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2db00d1870 0x7f2db0152a80 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.458+0000 7f2db7190640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2d9406b4d0 0x7f2d9406d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.458+0000 7f2db7190640 1 --2- 172.21.15.38:0/4163437825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2db00ce500 0x7f2db00c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.458+0000 7f2db7190640 1 -- 172.21.15.38:0/4163437825 >> 172.21.15.38:0/4163437825 conn(0x7f2db0066d10 msgr2=0x7f2db00d36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.458+0000 7f2db7190640 1 -- 172.21.15.38:0/4163437825 shutdown_connections 2024-04-25T08:34:28.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.458+0000 7f2db7190640 1 -- 172.21.15.38:0/4163437825 wait complete. 2024-04-25T08:34:28.464 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:34:28.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:28.464 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:34:28.487 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[51161]: fuse finished with error 0 and tester_r 0 2024-04-25T08:34:28.491 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:34:28.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:28.491 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:34:28.660 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:34:28.660 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:34:28.679 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:34:28.680 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:34:28.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:28.680 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:34:28.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:34:28.737 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:34:28.738 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:34:28.738 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:34:28.753 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:34:28.753 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:34:28.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:28.753 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:34:28.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:34:28.808 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:34:28.808 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:34:28.808 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:34:28.826 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:34:28.826 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:34:28.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:28.827 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:34:28.884 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:34:28.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:34:28.885 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:34:28.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.991+0000 7fe6190a1640 1 Processor -- start 2024-04-25T08:34:28.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.991+0000 7fe6190a1640 1 -- start start 2024-04-25T08:34:28.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe6190a1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61415be80 0x7fe6140b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:28.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe6190a1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe6140b4e60 0x7fe6140ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:28.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe6190a1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6140cd690 con 0x7fe614154cb0 2024-04-25T08:34:28.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe6190a1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6140b52d0 con 0x7fe6140b4e60 2024-04-25T08:34:28.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe6190a1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140cd440 con 0x7fe61415be80 2024-04-25T08:34:28.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe61359e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe6140b4e60 0x7fe6140ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:28.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe612d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe614154cb0 0x7fe6140c4ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58322/0 (socket says 172.21.15.38:58322) 2024-04-25T08:34:28.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe612d9d640 1 -- 172.21.15.38:0/1022002149 learned_addr learned my addr 172.21.15.38:0/1022002149 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:28.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe61259c640 1 --2- 172.21.15.38:0/1022002149 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61415be80 0x7fe6140b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:28.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe61359e640 1 -- 172.21.15.38:0/1022002149 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61415be80 msgr2=0x7fe6140b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:28.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe61359e640 1 --2- 172.21.15.38:0/1022002149 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61415be80 0x7fe6140b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe61359e640 1 -- 172.21.15.38:0/1022002149 >> v1:172.21.15.38:6789/0 conn(0x7fe614154cb0 legacy=0x7fe6140c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:28.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe611d9b640 1 -- 172.21.15.38:0/1022002149 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969151190 0 0) 0x7fe6140cd690 con 0x7fe614154cb0 2024-04-25T08:34:28.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe61359e640 1 -- 172.21.15.38:0/1022002149 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6140cd970 con 0x7fe6140b4e60 2024-04-25T08:34:28.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.992+0000 7fe61259c640 1 --2- 172.21.15.38:0/1022002149 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61415be80 0x7fe6140b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:28.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.993+0000 7fe61359e640 1 --2- 172.21.15.38:0/1022002149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6140b4e60 0x7fe6140ce730 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe604009d90 tx=0x7fe60402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c34644ed67f5e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:28.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.993+0000 7fe611d9b640 1 -- 172.21.15.38:0/1022002149 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604007740 con 0x7fe6140b4e60 2024-04-25T08:34:28.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.993+0000 7fe611d9b640 1 -- 172.21.15.38:0/1022002149 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6040078e0 con 0x7fe6140b4e60 2024-04-25T08:34:28.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.993+0000 7fe611d9b640 1 -- 172.21.15.38:0/1022002149 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604007cb0 con 0x7fe6140b4e60 2024-04-25T08:34:28.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.994+0000 7fe6190a1640 1 -- 172.21.15.38:0/1022002149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6140b4e60 msgr2=0x7fe6140ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:28.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.994+0000 7fe6190a1640 1 --2- 172.21.15.38:0/1022002149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6140b4e60 0x7fe6140ce730 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe604009d90 tx=0x7fe60402a660 comp rx=0 tx=0).stop 2024-04-25T08:34:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.994+0000 7fe6190a1640 1 -- 172.21.15.38:0/1022002149 shutdown_connections 2024-04-25T08:34:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.994+0000 7fe6190a1640 1 --2- 172.21.15.38:0/1022002149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6140b4e60 0x7fe6140ce730 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.994+0000 7fe6190a1640 1 --2- 172.21.15.38:0/1022002149 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61415be80 0x7fe6140b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.994+0000 7fe6190a1640 1 -- 172.21.15.38:0/1022002149 >> 172.21.15.38:0/1022002149 conn(0x7fe61406b210 msgr2=0x7fe61406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.994+0000 7fe6190a1640 1 -- 172.21.15.38:0/1022002149 shutdown_connections 2024-04-25T08:34:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.994+0000 7fe6190a1640 1 -- 172.21.15.38:0/1022002149 wait complete. 2024-04-25T08:34:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.995+0000 7fe6190a1640 1 Processor -- start 2024-04-25T08:34:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.995+0000 7fe6190a1640 1 -- start start 2024-04-25T08:34:28.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe6190a1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6140b4e60 0x7fe6141f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:28.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe6190a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe614154cb0 0x7fe6140b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:28.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe6190a1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61415be80 0x7fe61414b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:28.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe6190a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140cdf30 con 0x7fe614154cb0 2024-04-25T08:34:28.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe6190a1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61415c330 con 0x7fe6140b4e60 2024-04-25T08:34:28.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe6190a1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6140ce210 con 0x7fe61415be80 2024-04-25T08:34:28.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe61259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe614154cb0 0x7fe6140b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:28.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe61259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe614154cb0 0x7fe6140b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52846/0 (socket says 172.21.15.38:52846) 2024-04-25T08:34:28.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe61259c640 1 -- 172.21.15.38:0/598532599 learned_addr learned my addr 172.21.15.38:0/598532599 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:28.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe61359e640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61415be80 0x7fe61414b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe612d9d640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6140b4e60 0x7fe6141f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe61259c640 1 -- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61415be80 msgr2=0x7fe61414b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe61259c640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61415be80 0x7fe61414b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe61259c640 1 -- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6140b4e60 msgr2=0x7fe6141f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe61259c640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6140b4e60 0x7fe6141f3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe61259c640 1 -- 172.21.15.38:0/598532599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6140b2700 con 0x7fe614154cb0 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.996+0000 7fe61259c640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe614154cb0 0x7fe6140b4200 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe60800c820 tx=0x7fe60800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.997+0000 7fe5fb7fe640 1 -- 172.21.15.38:0/598532599 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608015070 con 0x7fe614154cb0 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.997+0000 7fe5fb7fe640 1 -- 172.21.15.38:0/598532599 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe608007a30 con 0x7fe614154cb0 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.997+0000 7fe6190a1640 1 -- 172.21.15.38:0/598532599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6141f4490 con 0x7fe614154cb0 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.997+0000 7fe5fb7fe640 1 -- 172.21.15.38:0/598532599 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608007de0 con 0x7fe614154cb0 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.997+0000 7fe6190a1640 1 -- 172.21.15.38:0/598532599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6141f4a50 con 0x7fe614154cb0 2024-04-25T08:34:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.998+0000 7fe6190a1640 1 -- 172.21.15.38:0/598532599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe614068e80 con 0x7fe614154cb0 2024-04-25T08:34:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.998+0000 7fe5fb7fe640 1 -- 172.21.15.38:0/598532599 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe608018070 con 0x7fe614154cb0 2024-04-25T08:34:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.999+0000 7fe5fb7fe640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe60006b4d0 0x7fe60006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.999+0000 7fe5fb7fe640 1 -- 172.21.15.38:0/598532599 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fe60808a250 con 0x7fe614154cb0 2024-04-25T08:34:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.999+0000 7fe612d9d640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe60006b4d0 0x7fe60006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:28.999+0000 7fe612d9d640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe60006b4d0 0x7fe60006d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe60c007fd0 tx=0x7fe60c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.003+0000 7fe5fb7fe640 1 -- 172.21.15.38:0/598532599 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe608058f20 con 0x7fe614154cb0 2024-04-25T08:34:29.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.219+0000 7fe6190a1640 1 -- 172.21.15.38:0/598532599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6140b4c40 con 0x7fe614154cb0 2024-04-25T08:34:29.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.219+0000 7fe5fb7fe640 1 -- 172.21.15.38:0/598532599 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fe608058820 con 0x7fe614154cb0 2024-04-25T08:34:29.215 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:29.215 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/422749749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":422749749},{"type":"v1","addr":"172.21.15.38:6834","nonce":422749749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5143,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/4194720752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":4194720752},{"type":"v1","addr":"172.21.15.107:6836","nonce":4194720752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3379884609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3379884609},{"type":"v1","addr":"172.21.15.107:6837","nonce":3379884609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:32:49.392279+0000","modified":"2024-04-25T08:32:57.539857+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5149":{"gid":5149,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/234860294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":234860294},{"type":"v1","addr":"172.21.15.38:6835","nonce":234860294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:34:29.215 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 18 2024-04-25T08:34:29.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.221+0000 7fe6190a1640 1 -- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe60006b4d0 msgr2=0x7fe60006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.221+0000 7fe6190a1640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe60006b4d0 0x7fe60006d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe60c007fd0 tx=0x7fe60c009040 comp rx=0 tx=0).stop 2024-04-25T08:34:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.221+0000 7fe6190a1640 1 -- 172.21.15.38:0/598532599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe614154cb0 msgr2=0x7fe6140b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.221+0000 7fe6190a1640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe614154cb0 0x7fe6140b4200 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe60800c820 tx=0x7fe60800cce0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.221+0000 7fe6190a1640 1 -- 172.21.15.38:0/598532599 shutdown_connections 2024-04-25T08:34:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.221+0000 7fe6190a1640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe60006b4d0 0x7fe60006d990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.221+0000 7fe6190a1640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61415be80 0x7fe61414b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.221+0000 7fe6190a1640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe614154cb0 0x7fe6140b4200 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.221+0000 7fe6190a1640 1 --2- 172.21.15.38:0/598532599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6140b4e60 0x7fe6141f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.221+0000 7fe6190a1640 1 -- 172.21.15.38:0/598532599 >> 172.21.15.38:0/598532599 conn(0x7fe61406b210 msgr2=0x7fe61415ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.221+0000 7fe6190a1640 1 -- 172.21.15.38:0/598532599 shutdown_connections 2024-04-25T08:34:29.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.221+0000 7fe6190a1640 1 -- 172.21.15.38:0/598532599 wait complete. 2024-04-25T08:34:29.228 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:34:29.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.333+0000 7f892089e640 1 Processor -- start 2024-04-25T08:34:29.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.333+0000 7f892089e640 1 -- start start 2024-04-25T08:34:29.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.334+0000 7f892089e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f891c15dc10 0x7f891c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.334+0000 7f892089e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f891c0ced60 0x7f891c0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.334+0000 7f892089e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891c0d3e70 con 0x7f891c0d3900 2024-04-25T08:34:29.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.334+0000 7f892089e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f891c15dff0 con 0x7f891c15dc10 2024-04-25T08:34:29.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.334+0000 7f892089e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0b6990 con 0x7f891c0ced60 2024-04-25T08:34:29.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.335+0000 7f891a59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f891c15dc10 0x7f891c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.335+0000 7f891a59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f891c15dc10 0x7f891c0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52862/0 (socket says 172.21.15.38:52862) 2024-04-25T08:34:29.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.335+0000 7f891a59c640 1 -- 172.21.15.38:0/3667643396 learned_addr learned my addr 172.21.15.38:0/3667643396 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:29.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.335+0000 7f8919d9b640 1 --2- 172.21.15.38:0/3667643396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f891c0ced60 0x7f891c0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.335+0000 7f891a59c640 1 -- 172.21.15.38:0/3667643396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f891c0ced60 msgr2=0x7f891c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:29.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.335+0000 7f891a59c640 1 --2- 172.21.15.38:0/3667643396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f891c0ced60 0x7f891c0d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.335+0000 7f891a59c640 1 -- 172.21.15.38:0/3667643396 >> v1:172.21.15.107:6789/0 conn(0x7f891c0d3900 legacy=0x7f891c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:29.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.335+0000 7f891a59c640 1 -- 172.21.15.38:0/3667643396 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c0d4150 con 0x7f891c15dc10 2024-04-25T08:34:29.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.336+0000 7f891a59c640 1 --2- 172.21.15.38:0/3667643396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f891c15dc10 0x7f891c0ce820 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8914009870 tx=0x7f891402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58c4db28e50eb681 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:29.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.336+0000 7f891959a640 1 -- 172.21.15.38:0/3667643396 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914037070 con 0x7f891c15dc10 2024-04-25T08:34:29.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.336+0000 7f891959a640 1 -- 172.21.15.38:0/3667643396 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8914032440 con 0x7f891c15dc10 2024-04-25T08:34:29.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.336+0000 7f891959a640 1 -- 172.21.15.38:0/3667643396 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8914032810 con 0x7f891c15dc10 2024-04-25T08:34:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.336+0000 7f892089e640 1 -- 172.21.15.38:0/3667643396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f891c15dc10 msgr2=0x7f891c0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.336+0000 7f892089e640 1 --2- 172.21.15.38:0/3667643396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f891c15dc10 0x7f891c0ce820 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8914009870 tx=0x7f891402a1c0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.336+0000 7f892089e640 1 -- 172.21.15.38:0/3667643396 shutdown_connections 2024-04-25T08:34:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.336+0000 7f892089e640 1 --2- 172.21.15.38:0/3667643396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f891c0ced60 0x7f891c0d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.336+0000 7f892089e640 1 --2- 172.21.15.38:0/3667643396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f891c15dc10 0x7f891c0ce820 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.336+0000 7f892089e640 1 -- 172.21.15.38:0/3667643396 >> 172.21.15.38:0/3667643396 conn(0x7f891c066d10 msgr2=0x7f891c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.336+0000 7f892089e640 1 -- 172.21.15.38:0/3667643396 shutdown_connections 2024-04-25T08:34:29.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.337+0000 7f892089e640 1 -- 172.21.15.38:0/3667643396 wait complete. 2024-04-25T08:34:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.338+0000 7f892089e640 1 Processor -- start 2024-04-25T08:34:29.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.338+0000 7f892089e640 1 -- start start 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.338+0000 7f892089e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f891c0ced60 0x7f891c15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.338+0000 7f892089e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f891c0d3900 0x7f891c0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.338+0000 7f892089e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f891c15dc10 0x7f891c0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.338+0000 7f892089e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0b6bc0 con 0x7f891c15dc10 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.338+0000 7f892089e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0d4710 con 0x7f891c0d3900 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.338+0000 7f892089e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f891c0d0d90 con 0x7f891c0ced60 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f891ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f891c15dc10 0x7f891c0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f891ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f891c15dc10 0x7f891c0c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52872/0 (socket says 172.21.15.38:52872) 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f891ad9d640 1 -- 172.21.15.38:0/3376347233 learned_addr learned my addr 172.21.15.38:0/3376347233 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f8919d9b640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f891c0d3900 0x7f891c0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f891a59c640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f891c0ced60 0x7f891c15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f891ad9d640 1 -- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f891c0ced60 msgr2=0x7f891c15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f891ad9d640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f891c0ced60 0x7f891c15bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f891ad9d640 1 -- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f891c0d3900 msgr2=0x7f891c0c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f891ad9d640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f891c0d3900 0x7f891c0c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f891ad9d640 1 -- 172.21.15.38:0/3376347233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c15c460 con 0x7f891c15dc10 2024-04-25T08:34:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f891a59c640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f891c0ced60 0x7f891c15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f891ad9d640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f891c15dc10 0x7f891c0c9d20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f890c00c7a0 tx=0x7f890c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f8912ffd640 1 -- 172.21.15.38:0/3376347233 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c003b70 con 0x7f891c15dc10 2024-04-25T08:34:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f8912ffd640 1 -- 172.21.15.38:0/3376347233 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890c003d30 con 0x7f891c15dc10 2024-04-25T08:34:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f8912ffd640 1 -- 172.21.15.38:0/3376347233 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890c00aa90 con 0x7f891c15dc10 2024-04-25T08:34:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f892089e640 1 -- 172.21.15.38:0/3376347233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f891c0aaed0 con 0x7f891c15dc10 2024-04-25T08:34:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.339+0000 7f892089e640 1 -- 172.21.15.38:0/3376347233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f891c0c7680 con 0x7f891c15dc10 2024-04-25T08:34:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.340+0000 7f892089e640 1 -- 172.21.15.38:0/3376347233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891c068e80 con 0x7f891c15dc10 2024-04-25T08:34:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.341+0000 7f8912ffd640 1 -- 172.21.15.38:0/3376347233 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f890c00ac30 con 0x7f891c15dc10 2024-04-25T08:34:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.341+0000 7f8912ffd640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f88fc06b4d0 0x7f88fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.341+0000 7f8912ffd640 1 -- 172.21.15.38:0/3376347233 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f890c08a310 con 0x7f891c15dc10 2024-04-25T08:34:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.342+0000 7f891a59c640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f88fc06b4d0 0x7f88fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.342+0000 7f891a59c640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f88fc06b4d0 0x7f88fc06d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8914034000 tx=0x7f891402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:29.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.346+0000 7f8912ffd640 1 -- 172.21.15.38:0/3376347233 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f891c15c460 con 0x7f891c15dc10 2024-04-25T08:34:29.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.527+0000 7f892089e640 1 -- 172.21.15.38:0/3376347233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f891c0cda00 con 0x7f891c15dc10 2024-04-25T08:34:29.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.529+0000 7f8912ffd640 1 -- 172.21.15.38:0/3376347233 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17044 (secure 0 0 0) 0x7f890c058f30 con 0x7f891c15dc10 2024-04-25T08:34:29.524 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:29.524 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":42,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:34:05.884436+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:47.798666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:48.371717+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:34:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.530+0000 7f892089e640 1 -- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f88fc06b4d0 msgr2=0x7f88fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.530+0000 7f892089e640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f88fc06b4d0 0x7f88fc06d990 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8914034000 tx=0x7f891402e000 comp rx=0 tx=0).stop 2024-04-25T08:34:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.530+0000 7f892089e640 1 -- 172.21.15.38:0/3376347233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f891c15dc10 msgr2=0x7f891c0c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.530+0000 7f892089e640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f891c15dc10 0x7f891c0c9d20 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f890c00c7a0 tx=0x7f890c00cc60 comp rx=0 tx=0).stop 2024-04-25T08:34:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.531+0000 7f892089e640 1 -- 172.21.15.38:0/3376347233 shutdown_connections 2024-04-25T08:34:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.531+0000 7f892089e640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f891c15dc10 0x7f891c0c9d20 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.531+0000 7f892089e640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f891c0d3900 0x7f891c0c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.531+0000 7f892089e640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f88fc06b4d0 0x7f88fc06d990 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.531+0000 7f892089e640 1 --2- 172.21.15.38:0/3376347233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f891c0ced60 0x7f891c15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.531+0000 7f892089e640 1 -- 172.21.15.38:0/3376347233 >> 172.21.15.38:0/3376347233 conn(0x7f891c066d10 msgr2=0x7f891c15ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:29.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.531+0000 7f892089e640 1 -- 172.21.15.38:0/3376347233 shutdown_connections 2024-04-25T08:34:29.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.531+0000 7f892089e640 1 -- 172.21.15.38:0/3376347233 wait complete. 2024-04-25T08:34:29.537 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:32:47.798666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:32:48.371717+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 41, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:34:29.538 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:34:29.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.646+0000 7fd2641b0640 1 Processor -- start 2024-04-25T08:34:29.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.646+0000 7fd2641b0640 1 -- start start 2024-04-25T08:34:29.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd2641b0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd25c15df30 0x7fd25c09a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd2641b0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd25c162390 0x7fd25c099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd2641b0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd25c099fb0 0x7fd25c06bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd2641b0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25c15e310 con 0x7fd25c162390 2024-04-25T08:34:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd2641b0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25c09aeb0 con 0x7fd25c099fb0 2024-04-25T08:34:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd2641b0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c15ee70 con 0x7fd25c15df30 2024-04-25T08:34:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd26174b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd25c162390 0x7fd25c099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd26174b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd25c162390 0x7fd25c099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52878/0 (socket says 172.21.15.38:52878) 2024-04-25T08:34:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd26174b640 1 -- 172.21.15.38:0/735640156 learned_addr learned my addr 172.21.15.38:0/735640156 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd26274d640 1 --2- 172.21.15.38:0/735640156 >> v2:172.21.15.107:3300/0 conn(0x7fd25c099fb0 0x7fd25c06bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd261f4c640 1 --2- 172.21.15.38:0/735640156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd25c15df30 0x7fd25c09a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd26174b640 1 -- 172.21.15.38:0/735640156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd25c15df30 msgr2=0x7fd25c09a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:29.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd26174b640 1 --2- 172.21.15.38:0/735640156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd25c15df30 0x7fd25c09a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd26174b640 1 -- 172.21.15.38:0/735640156 >> v2:172.21.15.107:3300/0 conn(0x7fd25c099fb0 msgr2=0x7fd25c06bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:29.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd26174b640 1 --2- 172.21.15.38:0/735640156 >> v2:172.21.15.107:3300/0 conn(0x7fd25c099fb0 0x7fd25c06bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd26174b640 1 -- 172.21.15.38:0/735640156 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c15f300 con 0x7fd25c162390 2024-04-25T08:34:29.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.647+0000 7fd26274d640 1 --2- 172.21.15.38:0/735640156 >> v2:172.21.15.107:3300/0 conn(0x7fd25c099fb0 0x7fd25c06bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:29.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.648+0000 7fd26174b640 1 --2- 172.21.15.38:0/735640156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c162390 0x7fd25c099a70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0d2680 tx=0x7fd25002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682a4722bea0ea90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:29.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.648+0000 7fd260f4a640 1 -- 172.21.15.38:0/735640156 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250037070 con 0x7fd25c162390 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.648+0000 7fd260f4a640 1 -- 172.21.15.38:0/735640156 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250032440 con 0x7fd25c162390 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.648+0000 7fd260f4a640 1 -- 172.21.15.38:0/735640156 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250032810 con 0x7fd25c162390 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.649+0000 7fd2641b0640 1 -- 172.21.15.38:0/735640156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c162390 msgr2=0x7fd25c099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.649+0000 7fd2641b0640 1 --2- 172.21.15.38:0/735640156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c162390 0x7fd25c099a70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0d2680 tx=0x7fd25002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.649+0000 7fd2641b0640 1 -- 172.21.15.38:0/735640156 shutdown_connections 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.649+0000 7fd2641b0640 1 --2- 172.21.15.38:0/735640156 >> v2:172.21.15.107:3300/0 conn(0x7fd25c099fb0 0x7fd25c06bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.649+0000 7fd2641b0640 1 --2- 172.21.15.38:0/735640156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c162390 0x7fd25c099a70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.649+0000 7fd2641b0640 1 --2- 172.21.15.38:0/735640156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd25c15df30 0x7fd25c09a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.649+0000 7fd2641b0640 1 -- 172.21.15.38:0/735640156 >> 172.21.15.38:0/735640156 conn(0x7fd25c0b3110 msgr2=0x7fd25c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.649+0000 7fd2641b0640 1 -- 172.21.15.38:0/735640156 shutdown_connections 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.649+0000 7fd2641b0640 1 -- 172.21.15.38:0/735640156 wait complete. 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.650+0000 7fd2641b0640 1 Processor -- start 2024-04-25T08:34:29.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.650+0000 7fd2641b0640 1 -- start start 2024-04-25T08:34:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd2641b0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd25c099fb0 0x7fd25c0b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd2641b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c15df30 0x7fd25c1e8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd2641b0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd25c162390 0x7fd25c160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd2641b0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c15e820 con 0x7fd25c15df30 2024-04-25T08:34:29.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd2641b0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c15e540 con 0x7fd25c099fb0 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd2641b0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c15eb00 con 0x7fd25c162390 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd26174b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c15df30 0x7fd25c1e8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd26174b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c15df30 0x7fd25c1e8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52886/0 (socket says 172.21.15.38:52886) 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd26174b640 1 -- 172.21.15.38:0/3687004959 learned_addr learned my addr 172.21.15.38:0/3687004959 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd261f4c640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd25c099fb0 0x7fd25c0b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd26274d640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd25c162390 0x7fd25c160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd26174b640 1 -- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd25c162390 msgr2=0x7fd25c160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd26174b640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd25c162390 0x7fd25c160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd26174b640 1 -- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd25c099fb0 msgr2=0x7fd25c0b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd26174b640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd25c099fb0 0x7fd25c0b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd26174b640 1 -- 172.21.15.38:0/3687004959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0c8c20 con 0x7fd25c15df30 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd261f4c640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd25c099fb0 0x7fd25c0b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:29.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.651+0000 7fd26274d640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd25c162390 0x7fd25c160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.652+0000 7fd26174b640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c15df30 0x7fd25c1e8120 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd25002a220 tx=0x7fd2500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.652+0000 7fd25a7fc640 1 -- 172.21.15.38:0/3687004959 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250041020 con 0x7fd25c15df30 2024-04-25T08:34:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.652+0000 7fd25a7fc640 1 -- 172.21.15.38:0/3687004959 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250030970 con 0x7fd25c15df30 2024-04-25T08:34:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.652+0000 7fd2641b0640 1 -- 172.21.15.38:0/3687004959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c1e8870 con 0x7fd25c15df30 2024-04-25T08:34:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.652+0000 7fd25a7fc640 1 -- 172.21.15.38:0/3687004959 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250037040 con 0x7fd25c15df30 2024-04-25T08:34:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.652+0000 7fd2641b0640 1 -- 172.21.15.38:0/3687004959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c0c9170 con 0x7fd25c15df30 2024-04-25T08:34:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.653+0000 7fd2641b0640 1 -- 172.21.15.38:0/3687004959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c068e80 con 0x7fd25c15df30 2024-04-25T08:34:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.653+0000 7fd25a7fc640 1 -- 172.21.15.38:0/3687004959 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd250044020 con 0x7fd25c15df30 2024-04-25T08:34:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.653+0000 7fd25a7fc640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd24806b500 0x7fd24806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.654+0000 7fd25a7fc640 1 -- 172.21.15.38:0/3687004959 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fd2500aad90 con 0x7fd25c15df30 2024-04-25T08:34:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.654+0000 7fd261f4c640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd24806b500 0x7fd24806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.654+0000 7fd261f4c640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd24806b500 0x7fd24806d9c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd25c06f030 tx=0x7fd254008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.658+0000 7fd25a7fc640 1 -- 172.21.15.38:0/3687004959 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd25004b030 con 0x7fd25c15df30 2024-04-25T08:34:29.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.837+0000 7fd2641b0640 1 -- 172.21.15.38:0/3687004959 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd25c06e9a0 con 0x7fd25c15df30 2024-04-25T08:34:29.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.837+0000 7fd25a7fc640 1 -- 172.21.15.38:0/3687004959 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd250079ae0 con 0x7fd25c15df30 2024-04-25T08:34:29.832 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:34:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.839+0000 7fd2641b0640 1 -- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd24806b500 msgr2=0x7fd24806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.839+0000 7fd2641b0640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd24806b500 0x7fd24806d9c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd25c06f030 tx=0x7fd254008040 comp rx=0 tx=0).stop 2024-04-25T08:34:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.839+0000 7fd2641b0640 1 -- 172.21.15.38:0/3687004959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c15df30 msgr2=0x7fd25c1e8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.839+0000 7fd2641b0640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c15df30 0x7fd25c1e8120 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fd25002a220 tx=0x7fd2500046c0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.839+0000 7fd2641b0640 1 -- 172.21.15.38:0/3687004959 shutdown_connections 2024-04-25T08:34:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.839+0000 7fd2641b0640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd24806b500 0x7fd24806d9c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.839+0000 7fd2641b0640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd25c162390 0x7fd25c160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.839+0000 7fd2641b0640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd25c15df30 0x7fd25c1e8120 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.839+0000 7fd2641b0640 1 --2- 172.21.15.38:0/3687004959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd25c099fb0 0x7fd25c0b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.839+0000 7fd2641b0640 1 -- 172.21.15.38:0/3687004959 >> 172.21.15.38:0/3687004959 conn(0x7fd25c0b3110 msgr2=0x7fd25c0709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.839+0000 7fd2641b0640 1 -- 172.21.15.38:0/3687004959 shutdown_connections 2024-04-25T08:34:29.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.839+0000 7fd2641b0640 1 -- 172.21.15.38:0/3687004959 wait complete. 2024-04-25T08:34:29.845 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:34:29.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.954+0000 7fdd45d09640 1 Processor -- start 2024-04-25T08:34:29.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.954+0000 7fdd45d09640 1 -- start start 2024-04-25T08:34:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd45d09640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdd400d0320 0x7fdd400d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd45d09640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd400d2060 con 0x7fdd400d0700 2024-04-25T08:34:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd45d09640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd400d2330 con 0x7fdd400b6980 2024-04-25T08:34:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd45d09640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd400d2610 con 0x7fdd400d0320 2024-04-25T08:34:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd3effd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdd400d0700 0x7fdd400d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58334/0 (socket says 172.21.15.38:58334) 2024-04-25T08:34:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd3effd640 1 -- 172.21.15.38:0/3232077841 learned_addr learned my addr 172.21.15.38:0/3232077841 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd3f7fe640 1 --2- 172.21.15.38:0/3232077841 >> v2:172.21.15.107:3300/0 conn(0x7fdd400d0320 0x7fdd400d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd3e7fc640 1 -- 172.21.15.38:0/3232077841 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4132108460 0 0) 0x7fdd400d2060 con 0x7fdd400d0700 2024-04-25T08:34:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd3e7fc640 1 -- 172.21.15.38:0/3232077841 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd28003520 con 0x7fdd400d0700 2024-04-25T08:34:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd3f7fe640 1 -- 172.21.15.38:0/3232077841 >> v1:172.21.15.107:6789/0 conn(0x7fdd400b6980 legacy=0x7fdd400b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd3e7fc640 1 -- 172.21.15.38:0/3232077841 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4078310393 0 0) 0x7fdd28003520 con 0x7fdd400d0700 2024-04-25T08:34:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd3f7fe640 1 -- 172.21.15.38:0/3232077841 >> v1:172.21.15.38:6789/0 conn(0x7fdd400d0700 legacy=0x7fdd400d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.955+0000 7fdd3f7fe640 1 -- 172.21.15.38:0/3232077841 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd400d28f0 con 0x7fdd400d0320 2024-04-25T08:34:29.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.956+0000 7fdd3f7fe640 1 --2- 172.21.15.38:0/3232077841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd400d0320 0x7fdd400d33d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdd300097a0 tx=0x7fdd3002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73598d762b4804ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:29.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.956+0000 7fdd3e7fc640 1 -- 172.21.15.38:0/3232077841 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30037070 con 0x7fdd400d0320 2024-04-25T08:34:29.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.956+0000 7fdd3e7fc640 1 -- 172.21.15.38:0/3232077841 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3002ace0 con 0x7fdd400d0320 2024-04-25T08:34:29.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.956+0000 7fdd3e7fc640 1 -- 172.21.15.38:0/3232077841 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34001670 con 0x7fdd400d0320 2024-04-25T08:34:29.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.957+0000 7fdd45d09640 1 -- 172.21.15.38:0/3232077841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd400d0320 msgr2=0x7fdd400d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.957+0000 7fdd45d09640 1 --2- 172.21.15.38:0/3232077841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd400d0320 0x7fdd400d33d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdd300097a0 tx=0x7fdd3002a250 comp rx=0 tx=0).stop 2024-04-25T08:34:29.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.957+0000 7fdd45d09640 1 -- 172.21.15.38:0/3232077841 shutdown_connections 2024-04-25T08:34:29.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.957+0000 7fdd45d09640 1 --2- 172.21.15.38:0/3232077841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd400d0320 0x7fdd400d33d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.957+0000 7fdd45d09640 1 -- 172.21.15.38:0/3232077841 >> 172.21.15.38:0/3232077841 conn(0x7fdd400668f0 msgr2=0x7fdd40066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:29.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.957+0000 7fdd45d09640 1 -- 172.21.15.38:0/3232077841 shutdown_connections 2024-04-25T08:34:29.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.957+0000 7fdd45d09640 1 -- 172.21.15.38:0/3232077841 wait complete. 2024-04-25T08:34:29.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.958+0000 7fdd45d09640 1 Processor -- start 2024-04-25T08:34:29.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.958+0000 7fdd45d09640 1 -- start start 2024-04-25T08:34:29.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.959+0000 7fdd45d09640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd400b6980 0x7fdd4015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.959+0000 7fdd45d09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd400d0320 0x7fdd401f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.959+0000 7fdd45d09640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd400d0700 0x7fdd400b7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.959+0000 7fdd45d09640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd400d0b40 con 0x7fdd400d0320 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.959+0000 7fdd45d09640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd400d2eb0 con 0x7fdd400b6980 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.959+0000 7fdd45d09640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd400b4680 con 0x7fdd400d0700 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.959+0000 7fdd3effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd400d0320 0x7fdd401f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.959+0000 7fdd3effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd400d0320 0x7fdd401f3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52888/0 (socket says 172.21.15.38:52888) 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.959+0000 7fdd3effd640 1 -- 172.21.15.38:0/159241374 learned_addr learned my addr 172.21.15.38:0/159241374 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.959+0000 7fdd3f7fe640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd400b6980 0x7fdd4015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.959+0000 7fdd3ffff640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd400d0700 0x7fdd400b7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.959+0000 7fdd3effd640 1 -- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd400d0700 msgr2=0x7fdd400b7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.960+0000 7fdd3effd640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd400d0700 0x7fdd400b7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.960+0000 7fdd3effd640 1 -- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd400b6980 msgr2=0x7fdd4015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.960+0000 7fdd3effd640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd400b6980 0x7fdd4015a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.960+0000 7fdd3effd640 1 -- 172.21.15.38:0/159241374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd400b48b0 con 0x7fdd400d0320 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.960+0000 7fdd3f7fe640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd400b6980 0x7fdd4015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:29.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.960+0000 7fdd3effd640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd400d0320 0x7fdd401f3740 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400c580 tx=0x7fdd3400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:29.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.960+0000 7fdd27fff640 1 -- 172.21.15.38:0/159241374 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34017070 con 0x7fdd400d0320 2024-04-25T08:34:29.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.960+0000 7fdd27fff640 1 -- 172.21.15.38:0/159241374 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd34005ce0 con 0x7fdd400d0320 2024-04-25T08:34:29.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.960+0000 7fdd27fff640 1 -- 172.21.15.38:0/159241374 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd34007330 con 0x7fdd400d0320 2024-04-25T08:34:29.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.960+0000 7fdd45d09640 1 -- 172.21.15.38:0/159241374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd400b4b90 con 0x7fdd400d0320 2024-04-25T08:34:29.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.960+0000 7fdd45d09640 1 -- 172.21.15.38:0/159241374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd400b5150 con 0x7fdd400d0320 2024-04-25T08:34:29.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.961+0000 7fdd45d09640 1 -- 172.21.15.38:0/159241374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd40068e80 con 0x7fdd400d0320 2024-04-25T08:34:29.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.961+0000 7fdd27fff640 1 -- 172.21.15.38:0/159241374 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd3401a070 con 0x7fdd400d0320 2024-04-25T08:34:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.962+0000 7fdd27fff640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd2c06b4d0 0x7fdd2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.962+0000 7fdd27fff640 1 -- 172.21.15.38:0/159241374 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fdd34089870 con 0x7fdd400d0320 2024-04-25T08:34:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.962+0000 7fdd3f7fe640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd2c06b4d0 0x7fdd2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.963+0000 7fdd3f7fe640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd2c06b4d0 0x7fdd2c06d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd30034000 tx=0x7fdd3002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:29.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:29.966+0000 7fdd27fff640 1 -- 172.21.15.38:0/159241374 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd34058540 con 0x7fdd400d0320 2024-04-25T08:34:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.149+0000 7fdd45d09640 1 -- 172.21.15.38:0/159241374 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd401603e0 con 0x7fdd400d0320 2024-04-25T08:34:30.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.150+0000 7fdd27fff640 1 -- 172.21.15.38:0/159241374 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd34057e40 con 0x7fdd400d0320 2024-04-25T08:34:30.145 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.152+0000 7fdd45d09640 1 -- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd2c06b4d0 msgr2=0x7fdd2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.152+0000 7fdd45d09640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd2c06b4d0 0x7fdd2c06d990 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdd30034000 tx=0x7fdd3002e000 comp rx=0 tx=0).stop 2024-04-25T08:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.152+0000 7fdd45d09640 1 -- 172.21.15.38:0/159241374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd400d0320 msgr2=0x7fdd401f3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.152+0000 7fdd45d09640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd400d0320 0x7fdd401f3740 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fdd3400c580 tx=0x7fdd3400ca40 comp rx=0 tx=0).stop 2024-04-25T08:34:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.152+0000 7fdd45d09640 1 -- 172.21.15.38:0/159241374 shutdown_connections 2024-04-25T08:34:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.152+0000 7fdd45d09640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd2c06b4d0 0x7fdd2c06d990 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.152+0000 7fdd45d09640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd400d0700 0x7fdd400b7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.152+0000 7fdd45d09640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd400d0320 0x7fdd401f3740 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.152+0000 7fdd45d09640 1 --2- 172.21.15.38:0/159241374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd400b6980 0x7fdd4015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.152+0000 7fdd45d09640 1 -- 172.21.15.38:0/159241374 >> 172.21.15.38:0/159241374 conn(0x7fdd400668f0 msgr2=0x7fdd4015d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.152+0000 7fdd45d09640 1 -- 172.21.15.38:0/159241374 shutdown_connections 2024-04-25T08:34:30.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.152+0000 7fdd45d09640 1 -- 172.21.15.38:0/159241374 wait complete. 2024-04-25T08:34:30.158 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:34:30.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.267+0000 7f342297e640 1 Processor -- start 2024-04-25T08:34:30.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.267+0000 7f342297e640 1 -- start start 2024-04-25T08:34:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.268+0000 7f342297e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f341c0d33d0 0x7f341c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.268+0000 7f342297e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c0d2330 con 0x7f341c0d1e90 2024-04-25T08:34:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.268+0000 7f342297e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341c0d2610 con 0x7f341c0d0320 2024-04-25T08:34:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.268+0000 7f342297e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341c0d28f0 con 0x7f341c0d33d0 2024-04-25T08:34:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.268+0000 7f341b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f341c0d33d0 0x7f341c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.268+0000 7f341b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f341c0d33d0 0x7f341c0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52898/0 (socket says 172.21.15.38:52898) 2024-04-25T08:34:30.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.268+0000 7f3420f1b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f341c0d1e90 0x7f341c0b6980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58338/0 (socket says 172.21.15.38:58338) 2024-04-25T08:34:30.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.268+0000 7f341b7fe640 1 -- 172.21.15.38:0/2316964857 learned_addr learned my addr 172.21.15.38:0/2316964857 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:30.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.268+0000 7f341b7fe640 1 -- 172.21.15.38:0/2316964857 >> v1:172.21.15.107:6789/0 conn(0x7f341c0d0320 legacy=0x7f341c0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:30.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.268+0000 7f341b7fe640 1 -- 172.21.15.38:0/2316964857 >> v1:172.21.15.38:6789/0 conn(0x7f341c0d1e90 legacy=0x7f341c0b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.268+0000 7f341b7fe640 1 -- 172.21.15.38:0/2316964857 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c0d2bd0 con 0x7f341c0d33d0 2024-04-25T08:34:30.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.268+0000 7f341affd640 1 -- 172.21.15.38:0/2316964857 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844485840 0 0) 0x7f341c0d2330 con 0x7f341c0d1e90 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.269+0000 7f341b7fe640 1 --2- 172.21.15.38:0/2316964857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c0d33d0 0x7f341c0d1950 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3410009870 tx=0x7f341002a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6258c25bd91eae12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.269+0000 7f341affd640 1 -- 172.21.15.38:0/2316964857 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410037070 con 0x7f341c0d33d0 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.269+0000 7f341affd640 1 -- 172.21.15.38:0/2316964857 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3410032640 con 0x7f341c0d33d0 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.269+0000 7f341affd640 1 -- 172.21.15.38:0/2316964857 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3410032a10 con 0x7f341c0d33d0 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.269+0000 7f342297e640 1 -- 172.21.15.38:0/2316964857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c0d33d0 msgr2=0x7f341c0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.269+0000 7f342297e640 1 --2- 172.21.15.38:0/2316964857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c0d33d0 0x7f341c0d1950 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3410009870 tx=0x7f341002a330 comp rx=0 tx=0).stop 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.270+0000 7f342297e640 1 -- 172.21.15.38:0/2316964857 shutdown_connections 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.270+0000 7f342297e640 1 --2- 172.21.15.38:0/2316964857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c0d33d0 0x7f341c0d1950 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.270+0000 7f342297e640 1 -- 172.21.15.38:0/2316964857 >> 172.21.15.38:0/2316964857 conn(0x7f341c0668f0 msgr2=0x7f341c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.270+0000 7f342297e640 1 -- 172.21.15.38:0/2316964857 shutdown_connections 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.270+0000 7f342297e640 1 -- 172.21.15.38:0/2316964857 wait complete. 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.271+0000 7f342297e640 1 Processor -- start 2024-04-25T08:34:30.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.271+0000 7f342297e640 1 -- start start 2024-04-25T08:34:30.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f342297e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c0d0320 0x7f341c14a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f342297e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f341c0d1e90 0x7f341c15b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f342297e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f341c0d33d0 0x7f341c0c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f342297e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c161390 con 0x7f341c0d0320 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f342297e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341c161210 con 0x7f341c0d1e90 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f342297e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341c0b9720 con 0x7f341c0d33d0 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f341bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c0d0320 0x7f341c14a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f341bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c0d0320 0x7f341c14a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52906/0 (socket says 172.21.15.38:52906) 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f341bfff640 1 -- 172.21.15.38:0/1668258597 learned_addr learned my addr 172.21.15.38:0/1668258597 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f3420f1b640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f341c0d33d0 0x7f341c0c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f341b7fe640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f341c0d1e90 0x7f341c15b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f341bfff640 1 -- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f341c0d33d0 msgr2=0x7f341c0c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f341bfff640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f341c0d33d0 0x7f341c0c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f341bfff640 1 -- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f341c0d1e90 msgr2=0x7f341c15b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f341bfff640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f341c0d1e90 0x7f341c15b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f341bfff640 1 -- 172.21.15.38:0/1668258597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c15be60 con 0x7f341c0d0320 2024-04-25T08:34:30.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f341bfff640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c0d0320 0x7f341c14a4c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c460 tx=0x7f340c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f3418ff9640 1 -- 172.21.15.38:0/1668258597 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c016020 con 0x7f341c0d0320 2024-04-25T08:34:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f3418ff9640 1 -- 172.21.15.38:0/1668258597 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c012870 con 0x7f341c0d0320 2024-04-25T08:34:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f3418ff9640 1 -- 172.21.15.38:0/1668258597 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c012c60 con 0x7f341c0d0320 2024-04-25T08:34:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f342297e640 1 -- 172.21.15.38:0/1668258597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341c15c140 con 0x7f341c0d0320 2024-04-25T08:34:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.272+0000 7f342297e640 1 -- 172.21.15.38:0/1668258597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341c15c700 con 0x7f341c0d0320 2024-04-25T08:34:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.273+0000 7f342297e640 1 -- 172.21.15.38:0/1668258597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341c068e80 con 0x7f341c0d0320 2024-04-25T08:34:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.274+0000 7f3418ff9640 1 -- 172.21.15.38:0/1668258597 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f340c019020 con 0x7f341c0d0320 2024-04-25T08:34:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.274+0000 7f3418ff9640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f340806b4d0 0x7f340806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.275+0000 7f3418ff9640 1 -- 172.21.15.38:0/1668258597 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f340c08a070 con 0x7f341c0d0320 2024-04-25T08:34:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.275+0000 7f341b7fe640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f340806b4d0 0x7f340806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.275+0000 7f341b7fe640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f340806b4d0 0x7f340806d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3410034000 tx=0x7f341002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:30.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.279+0000 7f3418ff9640 1 -- 172.21.15.38:0/1668258597 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f340c020030 con 0x7f341c0d0320 2024-04-25T08:34:30.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.457+0000 7f342297e640 1 -- 172.21.15.38:0/1668258597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f341c15b4b0 con 0x7f341c0d0320 2024-04-25T08:34:30.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.458+0000 7f3418ff9640 1 -- 172.21.15.38:0/1668258597 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f340c019310 con 0x7f341c0d0320 2024-04-25T08:34:30.453 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.460+0000 7f342297e640 1 -- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f340806b4d0 msgr2=0x7f340806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.460+0000 7f342297e640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f340806b4d0 0x7f340806d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3410034000 tx=0x7f341002e000 comp rx=0 tx=0).stop 2024-04-25T08:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.460+0000 7f342297e640 1 -- 172.21.15.38:0/1668258597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c0d0320 msgr2=0x7f341c14a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.460+0000 7f342297e640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c0d0320 0x7f341c14a4c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f340c00c460 tx=0x7f340c00c920 comp rx=0 tx=0).stop 2024-04-25T08:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.460+0000 7f342297e640 1 -- 172.21.15.38:0/1668258597 shutdown_connections 2024-04-25T08:34:30.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.460+0000 7f342297e640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f340806b4d0 0x7f340806d990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.460+0000 7f342297e640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f341c0d33d0 0x7f341c0c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.460+0000 7f342297e640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f341c0d1e90 0x7f341c15b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.460+0000 7f342297e640 1 --2- 172.21.15.38:0/1668258597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f341c0d0320 0x7f341c14a4c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.460+0000 7f342297e640 1 -- 172.21.15.38:0/1668258597 >> 172.21.15.38:0/1668258597 conn(0x7f341c0668f0 msgr2=0x7f341c0b7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.460+0000 7f342297e640 1 -- 172.21.15.38:0/1668258597 shutdown_connections 2024-04-25T08:34:30.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.460+0000 7f342297e640 1 -- 172.21.15.38:0/1668258597 wait complete. 2024-04-25T08:34:30.466 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:34:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.572+0000 7f4511b5c640 1 Processor -- start 2024-04-25T08:34:30.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.572+0000 7f4511b5c640 1 -- start start 2024-04-25T08:34:30.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f4511b5c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f450c15df50 0x7f450c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f4511b5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f450c0ce2e0 0x7f450c0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f4511b5c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f450c0ceff0 con 0x7f450c06c110 2024-04-25T08:34:30.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f4511b5c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450c0cf2d0 con 0x7f450c15df50 2024-04-25T08:34:30.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f4511b5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0d1f70 con 0x7f450c0ce2e0 2024-04-25T08:34:30.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f450affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f450c15df50 0x7f450c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f450affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f450c15df50 0x7f450c0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52910/0 (socket says 172.21.15.38:52910) 2024-04-25T08:34:30.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f450affd640 1 -- 172.21.15.38:0/4162788489 learned_addr learned my addr 172.21.15.38:0/4162788489 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f450b7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f450c06c110 0x7f450c06c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58346/0 (socket says 172.21.15.38:58346) 2024-04-25T08:34:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f450bfff640 1 --2- 172.21.15.38:0/4162788489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f450c0ce2e0 0x7f450c0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f450affd640 1 -- 172.21.15.38:0/4162788489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f450c0ce2e0 msgr2=0x7f450c0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f450affd640 1 --2- 172.21.15.38:0/4162788489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f450c0ce2e0 0x7f450c0d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f450affd640 1 -- 172.21.15.38:0/4162788489 >> v1:172.21.15.38:6789/0 conn(0x7f450c06c110 legacy=0x7f450c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f450a7fc640 1 -- 172.21.15.38:0/4162788489 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559914215 0 0) 0x7f450c0ceff0 con 0x7f450c06c110 2024-04-25T08:34:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.573+0000 7f450affd640 1 -- 172.21.15.38:0/4162788489 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c0d2250 con 0x7f450c15df50 2024-04-25T08:34:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.574+0000 7f450affd640 1 --2- 172.21.15.38:0/4162788489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c15df50 0x7f450c0ce7a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4500009870 tx=0x7f450002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524cc1d59161a33b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.574+0000 7f450a7fc640 1 -- 172.21.15.38:0/4162788489 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500037070 con 0x7f450c15df50 2024-04-25T08:34:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.574+0000 7f450a7fc640 1 -- 172.21.15.38:0/4162788489 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4500032440 con 0x7f450c15df50 2024-04-25T08:34:30.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.574+0000 7f450a7fc640 1 -- 172.21.15.38:0/4162788489 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500032810 con 0x7f450c15df50 2024-04-25T08:34:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.575+0000 7f4511b5c640 1 -- 172.21.15.38:0/4162788489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c15df50 msgr2=0x7f450c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.575+0000 7f4511b5c640 1 --2- 172.21.15.38:0/4162788489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c15df50 0x7f450c0ce7a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4500009870 tx=0x7f450002a270 comp rx=0 tx=0).stop 2024-04-25T08:34:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.575+0000 7f4511b5c640 1 -- 172.21.15.38:0/4162788489 shutdown_connections 2024-04-25T08:34:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.575+0000 7f4511b5c640 1 --2- 172.21.15.38:0/4162788489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f450c0ce2e0 0x7f450c0d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.575+0000 7f4511b5c640 1 --2- 172.21.15.38:0/4162788489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c15df50 0x7f450c0ce7a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.575+0000 7f4511b5c640 1 -- 172.21.15.38:0/4162788489 >> 172.21.15.38:0/4162788489 conn(0x7f450c0d3fb0 msgr2=0x7f450c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.575+0000 7f4511b5c640 1 -- 172.21.15.38:0/4162788489 shutdown_connections 2024-04-25T08:34:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.575+0000 7f4511b5c640 1 -- 172.21.15.38:0/4162788489 wait complete. 2024-04-25T08:34:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.576+0000 7f4511b5c640 1 Processor -- start 2024-04-25T08:34:30.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.576+0000 7f4511b5c640 1 -- start start 2024-04-25T08:34:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f4511b5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f450c06c110 0x7f450c0cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f4511b5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c0ce2e0 0x7f450c1610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f4511b5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f450c15df50 0x7f450c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f4511b5c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0d2f90 con 0x7f450c0ce2e0 2024-04-25T08:34:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f4511b5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0cece0 con 0x7f450c15df50 2024-04-25T08:34:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f4511b5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f450c0d3110 con 0x7f450c06c110 2024-04-25T08:34:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f450affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c0ce2e0 0x7f450c1610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f450affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c0ce2e0 0x7f450c1610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52916/0 (socket says 172.21.15.38:52916) 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f450affd640 1 -- 172.21.15.38:0/368042796 learned_addr learned my addr 172.21.15.38:0/368042796 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f450bfff640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f450c15df50 0x7f450c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f450b7fe640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f450c06c110 0x7f450c0cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f450affd640 1 -- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f450c06c110 msgr2=0x7f450c0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f450affd640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f450c06c110 0x7f450c0cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f450affd640 1 -- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f450c15df50 msgr2=0x7f450c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f450affd640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f450c15df50 0x7f450c0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.577+0000 7f450affd640 1 -- 172.21.15.38:0/368042796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c15e7c0 con 0x7f450c0ce2e0 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.578+0000 7f450affd640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c0ce2e0 0x7f450c1610e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4500000c00 tx=0x7f45000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.578+0000 7f44f3fff640 1 -- 172.21.15.38:0/368042796 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500041020 con 0x7f450c0ce2e0 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.578+0000 7f44f3fff640 1 -- 172.21.15.38:0/368042796 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4500030640 con 0x7f450c0ce2e0 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.578+0000 7f44f3fff640 1 -- 172.21.15.38:0/368042796 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500037040 con 0x7f450c0ce2e0 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.578+0000 7f4511b5c640 1 -- 172.21.15.38:0/368042796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450c15eec0 con 0x7f450c0ce2e0 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.578+0000 7f4511b5c640 1 -- 172.21.15.38:0/368042796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450c156230 con 0x7f450c0ce2e0 2024-04-25T08:34:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.579+0000 7f4511b5c640 1 -- 172.21.15.38:0/368042796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450c0c71a0 con 0x7f450c0ce2e0 2024-04-25T08:34:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.579+0000 7f44f3fff640 1 -- 172.21.15.38:0/368042796 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4500044020 con 0x7f450c0ce2e0 2024-04-25T08:34:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.580+0000 7f44f3fff640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44f806b4d0 0x7f44f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.580+0000 7f44f3fff640 1 -- 172.21.15.38:0/368042796 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f45000aab20 con 0x7f450c0ce2e0 2024-04-25T08:34:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.580+0000 7f450b7fe640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44f806b4d0 0x7f44f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.580+0000 7f450b7fe640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44f806b4d0 0x7f44f806d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f44fc000f60 tx=0x7f44fc007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:30.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.584+0000 7f44f3fff640 1 -- 172.21.15.38:0/368042796 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f450c0c71a0 con 0x7f450c0ce2e0 2024-04-25T08:34:30.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.799+0000 7f4511b5c640 1 -- 172.21.15.38:0/368042796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f450c072e40 con 0x7f450c0ce2e0 2024-04-25T08:34:30.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.799+0000 7f44f3fff640 1 -- 172.21.15.38:0/368042796 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f450c072e40 con 0x7f450c0ce2e0 2024-04-25T08:34:30.795 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:30.795 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/422749749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":422749749},{"type":"v1","addr":"172.21.15.38:6834","nonce":422749749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5143,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/4194720752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":4194720752},{"type":"v1","addr":"172.21.15.107:6836","nonce":4194720752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3379884609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3379884609},{"type":"v1","addr":"172.21.15.107:6837","nonce":3379884609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:32:49.392279+0000","modified":"2024-04-25T08:32:57.539857+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5149":{"gid":5149,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/234860294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":234860294},{"type":"v1","addr":"172.21.15.38:6835","nonce":234860294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:34:30.795 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 18 2024-04-25T08:34:30.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.801+0000 7f4511b5c640 1 -- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44f806b4d0 msgr2=0x7f44f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.801+0000 7f4511b5c640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44f806b4d0 0x7f44f806d990 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f44fc000f60 tx=0x7f44fc007c00 comp rx=0 tx=0).stop 2024-04-25T08:34:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.801+0000 7f4511b5c640 1 -- 172.21.15.38:0/368042796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c0ce2e0 msgr2=0x7f450c1610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.801+0000 7f4511b5c640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c0ce2e0 0x7f450c1610e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4500000c00 tx=0x7f45000046c0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.801+0000 7f4511b5c640 1 -- 172.21.15.38:0/368042796 shutdown_connections 2024-04-25T08:34:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.801+0000 7f4511b5c640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f450c15df50 0x7f450c0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.801+0000 7f4511b5c640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f450c0ce2e0 0x7f450c1610e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.801+0000 7f4511b5c640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44f806b4d0 0x7f44f806d990 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.801+0000 7f4511b5c640 1 --2- 172.21.15.38:0/368042796 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f450c06c110 0x7f450c0cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.801+0000 7f4511b5c640 1 -- 172.21.15.38:0/368042796 >> 172.21.15.38:0/368042796 conn(0x7f450c0d3fb0 msgr2=0x7f450c073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.801+0000 7f4511b5c640 1 -- 172.21.15.38:0/368042796 shutdown_connections 2024-04-25T08:34:30.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.801+0000 7f4511b5c640 1 -- 172.21.15.38:0/368042796 wait complete. 2024-04-25T08:34:30.808 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:34:30.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.914+0000 7f83f0516640 1 Processor -- start 2024-04-25T08:34:30.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.914+0000 7f83f0516640 1 -- start start 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.915+0000 7f83f0516640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83e806c110 0x7f83e806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.915+0000 7f83f0516640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f83e80ce2e0 0x7f83e80d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.915+0000 7f83f0516640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e806ca30 con 0x7f83e815df50 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.915+0000 7f83f0516640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e80cf2d0 con 0x7f83e80ce2e0 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.915+0000 7f83f0516640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80ceff0 con 0x7f83e806c110 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.915+0000 7f83edab1640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f83e815df50 0x7f83e80ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58354/0 (socket says 172.21.15.38:58354) 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.915+0000 7f83edab1640 1 -- 172.21.15.38:0/3356844239 learned_addr learned my addr 172.21.15.38:0/3356844239 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.915+0000 7f83ee2b2640 1 --2- 172.21.15.38:0/3356844239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83e806c110 0x7f83e806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.915+0000 7f83eeab3640 1 --2- 172.21.15.38:0/3356844239 >> v2:172.21.15.107:3300/0 conn(0x7f83e80ce2e0 0x7f83e80d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.915+0000 7f83ed2b0640 1 -- 172.21.15.38:0/3356844239 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195058903 0 0) 0x7f83e806ca30 con 0x7f83e815df50 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.915+0000 7f83ed2b0640 1 -- 172.21.15.38:0/3356844239 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83d4003520 con 0x7f83e815df50 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.916+0000 7f83ee2b2640 1 -- 172.21.15.38:0/3356844239 >> v2:172.21.15.107:3300/0 conn(0x7f83e80ce2e0 msgr2=0x7f83e80d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.916+0000 7f83ee2b2640 1 --2- 172.21.15.38:0/3356844239 >> v2:172.21.15.107:3300/0 conn(0x7f83e80ce2e0 0x7f83e80d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.916+0000 7f83ee2b2640 1 -- 172.21.15.38:0/3356844239 >> v1:172.21.15.38:6789/0 conn(0x7f83e815df50 legacy=0x7f83e80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.916+0000 7f83ee2b2640 1 -- 172.21.15.38:0/3356844239 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e80d1f70 con 0x7f83e806c110 2024-04-25T08:34:30.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.916+0000 7f83eeab3640 1 --2- 172.21.15.38:0/3356844239 >> v2:172.21.15.107:3300/0 conn(0x7f83e80ce2e0 0x7f83e80d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:30.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.917+0000 7f83ee2b2640 1 --2- 172.21.15.38:0/3356844239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83e806c110 0x7f83e806c4f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f83e40097a0 tx=0x7f83e402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d782c8823872940 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:30.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.917+0000 7f83ed2b0640 1 -- 172.21.15.38:0/3356844239 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4037070 con 0x7f83e806c110 2024-04-25T08:34:30.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.917+0000 7f83ed2b0640 1 -- 172.21.15.38:0/3356844239 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e4032440 con 0x7f83e806c110 2024-04-25T08:34:30.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.917+0000 7f83ed2b0640 1 -- 172.21.15.38:0/3356844239 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4032810 con 0x7f83e806c110 2024-04-25T08:34:30.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.917+0000 7f83f0516640 1 -- 172.21.15.38:0/3356844239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83e806c110 msgr2=0x7f83e806c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.917+0000 7f83f0516640 1 --2- 172.21.15.38:0/3356844239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83e806c110 0x7f83e806c4f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f83e40097a0 tx=0x7f83e402a180 comp rx=0 tx=0).stop 2024-04-25T08:34:30.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.917+0000 7f83f0516640 1 -- 172.21.15.38:0/3356844239 shutdown_connections 2024-04-25T08:34:30.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.917+0000 7f83f0516640 1 --2- 172.21.15.38:0/3356844239 >> v2:172.21.15.107:3300/0 conn(0x7f83e80ce2e0 0x7f83e80d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.917+0000 7f83f0516640 1 --2- 172.21.15.38:0/3356844239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83e806c110 0x7f83e806c4f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.917+0000 7f83f0516640 1 -- 172.21.15.38:0/3356844239 >> 172.21.15.38:0/3356844239 conn(0x7f83e80d3fb0 msgr2=0x7f83e80d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:30.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.917+0000 7f83f0516640 1 -- 172.21.15.38:0/3356844239 shutdown_connections 2024-04-25T08:34:30.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.917+0000 7f83f0516640 1 -- 172.21.15.38:0/3356844239 wait complete. 2024-04-25T08:34:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.919+0000 7f83f0516640 1 Processor -- start 2024-04-25T08:34:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.919+0000 7f83f0516640 1 -- start start 2024-04-25T08:34:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.919+0000 7f83f0516640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83e806c110 0x7f83e80ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.919+0000 7f83f0516640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83e80ce2e0 0x7f83e81604c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.919+0000 7f83f0516640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83e815df50 0x7f83e814a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.919+0000 7f83f0516640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80d2f90 con 0x7f83e806c110 2024-04-25T08:34:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.919+0000 7f83f0516640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80d2530 con 0x7f83e815df50 2024-04-25T08:34:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.919+0000 7f83f0516640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e80d3110 con 0x7f83e80ce2e0 2024-04-25T08:34:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83ee2b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83e806c110 0x7f83e80ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83ee2b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83e806c110 0x7f83e80ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52918/0 (socket says 172.21.15.38:52918) 2024-04-25T08:34:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83ee2b2640 1 -- 172.21.15.38:0/980278137 learned_addr learned my addr 172.21.15.38:0/980278137 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83eeab3640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83e815df50 0x7f83e814a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83edab1640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83e80ce2e0 0x7f83e81604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83ee2b2640 1 -- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83e80ce2e0 msgr2=0x7f83e81604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83ee2b2640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83e80ce2e0 0x7f83e81604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83ee2b2640 1 -- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83e815df50 msgr2=0x7f83e814a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83ee2b2640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83e815df50 0x7f83e814a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83ee2b2640 1 -- 172.21.15.38:0/980278137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e815f420 con 0x7f83e806c110 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83eeab3640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83e815df50 0x7f83e814a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83ee2b2640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83e806c110 0x7f83e80ccaf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f83e40096f0 tx=0x7f83e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83deffd640 1 -- 172.21.15.38:0/980278137 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4041020 con 0x7f83e806c110 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83deffd640 1 -- 172.21.15.38:0/980278137 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e403d700 con 0x7f83e806c110 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83deffd640 1 -- 172.21.15.38:0/980278137 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4037040 con 0x7f83e806c110 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83f0516640 1 -- 172.21.15.38:0/980278137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e815f700 con 0x7f83e806c110 2024-04-25T08:34:30.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.920+0000 7f83f0516640 1 -- 172.21.15.38:0/980278137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e815ffa0 con 0x7f83e806c110 2024-04-25T08:34:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.921+0000 7f83f0516640 1 -- 172.21.15.38:0/980278137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e806c4f0 con 0x7f83e806c110 2024-04-25T08:34:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.922+0000 7f83deffd640 1 -- 172.21.15.38:0/980278137 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83e4044020 con 0x7f83e806c110 2024-04-25T08:34:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.922+0000 7f83deffd640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83cc06b4d0 0x7f83cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.922+0000 7f83deffd640 1 -- 172.21.15.38:0/980278137 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f83e40aab40 con 0x7f83e806c110 2024-04-25T08:34:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.923+0000 7f83edab1640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83cc06b4d0 0x7f83cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.923+0000 7f83edab1640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83cc06b4d0 0x7f83cc06d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f83e8158b10 tx=0x7f83d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:30.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:30.927+0000 7f83deffd640 1 -- 172.21.15.38:0/980278137 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83e806c4f0 con 0x7f83e806c110 2024-04-25T08:34:31.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.110+0000 7f83f0516640 1 -- 172.21.15.38:0/980278137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83e815f8d0 con 0x7f83e806c110 2024-04-25T08:34:31.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.111+0000 7f83deffd640 1 -- 172.21.15.38:0/980278137 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17044 (secure 0 0 0) 0x7f83e404b030 con 0x7f83e806c110 2024-04-25T08:34:31.106 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:31.106 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":42,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:34:05.884436+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:47.798666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:48.371717+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:34:31.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.113+0000 7f83f0516640 1 -- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83cc06b4d0 msgr2=0x7f83cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.113+0000 7f83f0516640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83cc06b4d0 0x7f83cc06d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f83e8158b10 tx=0x7f83d8009040 comp rx=0 tx=0).stop 2024-04-25T08:34:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.113+0000 7f83f0516640 1 -- 172.21.15.38:0/980278137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83e806c110 msgr2=0x7f83e80ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.113+0000 7f83f0516640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83e806c110 0x7f83e80ccaf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f83e40096f0 tx=0x7f83e40046c0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.113+0000 7f83f0516640 1 -- 172.21.15.38:0/980278137 shutdown_connections 2024-04-25T08:34:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.113+0000 7f83f0516640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f83e815df50 0x7f83e814a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.113+0000 7f83f0516640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f83cc06b4d0 0x7f83cc06d990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.113+0000 7f83f0516640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f83e80ce2e0 0x7f83e81604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.113+0000 7f83f0516640 1 --2- 172.21.15.38:0/980278137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f83e806c110 0x7f83e80ccaf0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.113+0000 7f83f0516640 1 -- 172.21.15.38:0/980278137 >> 172.21.15.38:0/980278137 conn(0x7f83e80d3fb0 msgr2=0x7f83e8073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.113+0000 7f83f0516640 1 -- 172.21.15.38:0/980278137 shutdown_connections 2024-04-25T08:34:31.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.113+0000 7f83f0516640 1 -- 172.21.15.38:0/980278137 wait complete. 2024-04-25T08:34:31.119 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:34:31.120 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:34:31.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.225+0000 7fcb1449e640 1 Processor -- start 2024-04-25T08:34:31.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.225+0000 7fcb1449e640 1 -- start start 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb1449e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcb0c0aeeb0 0x7fcb0c0ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb1449e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcb0c06b2f0 0x7fcb0c0d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb1449e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb0c0cfa40 con 0x7fcb0c0cd450 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb1449e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0c0cd9b0 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb1449e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0c0cdc90 con 0x7fcb0c0aeeb0 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb12a3b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcb0c06b2f0 0x7fcb0c0d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb11a39640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcb0c0cd450 0x7fcb0c06abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58370/0 (socket says 172.21.15.38:58370) 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb11a39640 1 -- 172.21.15.38:0/782045740 learned_addr learned my addr 172.21.15.38:0/782045740 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb1223a640 1 --2- 172.21.15.38:0/782045740 >> v2:172.21.15.107:3300/0 conn(0x7fcb0c0aeeb0 0x7fcb0c0ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb12a3b640 1 -- 172.21.15.38:0/782045740 >> v2:172.21.15.107:3300/0 conn(0x7fcb0c0aeeb0 msgr2=0x7fcb0c0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb12a3b640 1 --2- 172.21.15.38:0/782045740 >> v2:172.21.15.107:3300/0 conn(0x7fcb0c0aeeb0 0x7fcb0c0ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb12a3b640 1 -- 172.21.15.38:0/782045740 >> v1:172.21.15.38:6789/0 conn(0x7fcb0c0cd450 legacy=0x7fcb0c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb12a3b640 1 -- 172.21.15.38:0/782045740 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c15e390 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.226+0000 7fcb11238640 1 -- 172.21.15.38:0/782045740 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404353619 0 0) 0x7fcb0c0cfa40 con 0x7fcb0c0cd450 2024-04-25T08:34:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.227+0000 7fcb12a3b640 1 --2- 172.21.15.38:0/782045740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb0c06b2f0 0x7fcb0c0d01d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0d16e0 tx=0x7fcb0402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc105f165d6761d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.227+0000 7fcb11238640 1 -- 172.21.15.38:0/782045740 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04007740 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.227+0000 7fcb11238640 1 -- 172.21.15.38:0/782045740 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb040078e0 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.227+0000 7fcb11238640 1 -- 172.21.15.38:0/782045740 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb04007cb0 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.228+0000 7fcb1449e640 1 -- 172.21.15.38:0/782045740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb0c06b2f0 msgr2=0x7fcb0c0d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.228+0000 7fcb1449e640 1 --2- 172.21.15.38:0/782045740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb0c06b2f0 0x7fcb0c0d01d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0d16e0 tx=0x7fcb0402a680 comp rx=0 tx=0).stop 2024-04-25T08:34:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.228+0000 7fcb1449e640 1 -- 172.21.15.38:0/782045740 shutdown_connections 2024-04-25T08:34:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.228+0000 7fcb1449e640 1 --2- 172.21.15.38:0/782045740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb0c06b2f0 0x7fcb0c0d01d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.228+0000 7fcb1449e640 1 --2- 172.21.15.38:0/782045740 >> v2:172.21.15.107:3300/0 conn(0x7fcb0c0aeeb0 0x7fcb0c0ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.228+0000 7fcb1449e640 1 -- 172.21.15.38:0/782045740 >> 172.21.15.38:0/782045740 conn(0x7fcb0c072610 msgr2=0x7fcb0c099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.228+0000 7fcb1449e640 1 -- 172.21.15.38:0/782045740 shutdown_connections 2024-04-25T08:34:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.228+0000 7fcb1449e640 1 -- 172.21.15.38:0/782045740 wait complete. 2024-04-25T08:34:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.229+0000 7fcb1449e640 1 Processor -- start 2024-04-25T08:34:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.229+0000 7fcb1449e640 1 -- start start 2024-04-25T08:34:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1449e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb0c06b2f0 0x7fcb0c1f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1449e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb0c0aeeb0 0x7fcb0c0c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1449e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb0c0cd450 0x7fcb0c0b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1449e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c0d0890 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1449e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c0d0710 con 0x7fcb0c0aeeb0 2024-04-25T08:34:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1449e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0c0d0a10 con 0x7fcb0c0cd450 2024-04-25T08:34:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1223a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb0c06b2f0 0x7fcb0c1f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1223a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb0c06b2f0 0x7fcb0c1f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52940/0 (socket says 172.21.15.38:52940) 2024-04-25T08:34:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1223a640 1 -- 172.21.15.38:0/3935784638 learned_addr learned my addr 172.21.15.38:0/3935784638 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb11a39640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb0c0aeeb0 0x7fcb0c0c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb12a3b640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb0c0cd450 0x7fcb0c0b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1223a640 1 -- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb0c0cd450 msgr2=0x7fcb0c0b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1223a640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb0c0cd450 0x7fcb0c0b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1223a640 1 -- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb0c0aeeb0 msgr2=0x7fcb0c0c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1223a640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb0c0aeeb0 0x7fcb0c0c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb1223a640 1 -- 172.21.15.38:0/3935784638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0c1f3320 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb12a3b640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb0c0cd450 0x7fcb0c0b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.230+0000 7fcb11a39640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb0c0aeeb0 0x7fcb0c0c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.231+0000 7fcb1223a640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb0c06b2f0 0x7fcb0c1f2de0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800d800 tx=0x7fcb0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.231+0000 7fcb02ffd640 1 -- 172.21.15.38:0/3935784638 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800b6c0 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.231+0000 7fcb02ffd640 1 -- 172.21.15.38:0/3935784638 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb08019070 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.231+0000 7fcb1449e640 1 -- 172.21.15.38:0/3935784638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0c0b53d0 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.231+0000 7fcb02ffd640 1 -- 172.21.15.38:0/3935784638 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0800ba80 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.231+0000 7fcb1449e640 1 -- 172.21.15.38:0/3935784638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0c0b5950 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.232+0000 7fcb1449e640 1 -- 172.21.15.38:0/3935784638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0c0d1460 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.232+0000 7fcb02ffd640 1 -- 172.21.15.38:0/3935784638 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb0800bc20 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.232+0000 7fcb02ffd640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcaf006b4d0 0x7fcaf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.233+0000 7fcb02ffd640 1 -- 172.21.15.38:0/3935784638 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fcb0808a870 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.233+0000 7fcb11a39640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcaf006b4d0 0x7fcaf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.233+0000 7fcb11a39640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcaf006b4d0 0x7fcaf006d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0b20b0 tx=0x7fcafc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:31.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.237+0000 7fcb02ffd640 1 -- 172.21.15.38:0/3935784638 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb08059530 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.454+0000 7fcb1449e640 1 -- 172.21.15.38:0/3935784638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb0c06d090 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.454+0000 7fcb02ffd640 1 -- 172.21.15.38:0/3935784638 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fcb08058e30 con 0x7fcb0c06b2f0 2024-04-25T08:34:31.450 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:31.450 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/422749749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":422749749},{"type":"v1","addr":"172.21.15.38:6834","nonce":422749749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5143,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/4194720752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":4194720752},{"type":"v1","addr":"172.21.15.107:6836","nonce":4194720752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3379884609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3379884609},{"type":"v1","addr":"172.21.15.107:6837","nonce":3379884609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:32:49.392279+0000","modified":"2024-04-25T08:32:57.539857+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5149":{"gid":5149,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/234860294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":234860294},{"type":"v1","addr":"172.21.15.38:6835","nonce":234860294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:34:31.450 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 18 2024-04-25T08:34:31.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.456+0000 7fcb1449e640 1 -- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcaf006b4d0 msgr2=0x7fcaf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.456+0000 7fcb1449e640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcaf006b4d0 0x7fcaf006d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0b20b0 tx=0x7fcafc009040 comp rx=0 tx=0).stop 2024-04-25T08:34:31.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.456+0000 7fcb1449e640 1 -- 172.21.15.38:0/3935784638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb0c06b2f0 msgr2=0x7fcb0c1f2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.456+0000 7fcb1449e640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb0c06b2f0 0x7fcb0c1f2de0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800d800 tx=0x7fcb0800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.456+0000 7fcb1449e640 1 -- 172.21.15.38:0/3935784638 shutdown_connections 2024-04-25T08:34:31.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.456+0000 7fcb1449e640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcaf006b4d0 0x7fcaf006d990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.456+0000 7fcb1449e640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb0c0cd450 0x7fcb0c0b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.456+0000 7fcb1449e640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb0c0aeeb0 0x7fcb0c0c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.456+0000 7fcb1449e640 1 --2- 172.21.15.38:0/3935784638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb0c06b2f0 0x7fcb0c1f2de0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.456+0000 7fcb1449e640 1 -- 172.21.15.38:0/3935784638 >> 172.21.15.38:0/3935784638 conn(0x7fcb0c072610 msgr2=0x7fcb0c0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:31.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.456+0000 7fcb1449e640 1 -- 172.21.15.38:0/3935784638 shutdown_connections 2024-04-25T08:34:31.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.456+0000 7fcb1449e640 1 -- 172.21.15.38:0/3935784638 wait complete. 2024-04-25T08:34:31.463 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:34:31.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.568+0000 7fecd0970640 1 Processor -- start 2024-04-25T08:34:31.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.568+0000 7fecd0970640 1 -- start start 2024-04-25T08:34:31.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.569+0000 7fecd0970640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feccc06b2f0 0x7feccc0b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.569+0000 7fecd0970640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7feccc0b1930 0x7feccc15d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.569+0000 7fecd0970640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feccc0cdf00 con 0x7feccc15ba00 2024-04-25T08:34:31.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.569+0000 7fecd0970640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feccc15cf70 con 0x7feccc0b1930 2024-04-25T08:34:31.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.569+0000 7fecd0970640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feccc15cae0 con 0x7feccc06b2f0 2024-04-25T08:34:31.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.569+0000 7fecca59c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7feccc15ba00 0x7feccc06abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58382/0 (socket says 172.21.15.38:58382) 2024-04-25T08:34:31.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.569+0000 7fecca59c640 1 -- 172.21.15.38:0/1078714276 learned_addr learned my addr 172.21.15.38:0/1078714276 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:31.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.569+0000 7feccad9d640 1 --2- 172.21.15.38:0/1078714276 >> v2:172.21.15.38:3300/0 conn(0x7feccc0b1930 0x7feccc15d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.570+0000 7fecc9d9b640 1 --2- 172.21.15.38:0/1078714276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feccc06b2f0 0x7feccc0b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.570+0000 7fecc959a640 1 -- 172.21.15.38:0/1078714276 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402575110 0 0) 0x7feccc0cdf00 con 0x7feccc15ba00 2024-04-25T08:34:31.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.570+0000 7fecc959a640 1 -- 172.21.15.38:0/1078714276 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb8003520 con 0x7feccc15ba00 2024-04-25T08:34:31.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.570+0000 7feccad9d640 1 -- 172.21.15.38:0/1078714276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feccc06b2f0 msgr2=0x7feccc0b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:31.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.570+0000 7feccad9d640 1 --2- 172.21.15.38:0/1078714276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feccc06b2f0 0x7feccc0b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.570+0000 7feccad9d640 1 -- 172.21.15.38:0/1078714276 >> v1:172.21.15.38:6789/0 conn(0x7feccc15ba00 legacy=0x7feccc06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.570+0000 7feccad9d640 1 -- 172.21.15.38:0/1078714276 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc15c1b0 con 0x7feccc0b1930 2024-04-25T08:34:31.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.570+0000 7fecc9d9b640 1 --2- 172.21.15.38:0/1078714276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feccc06b2f0 0x7feccc0b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:31.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.571+0000 7feccad9d640 1 --2- 172.21.15.38:0/1078714276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feccc0b1930 0x7feccc15d140 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fecbc009d90 tx=0x7fecbc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e8f30095b33b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:31.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.571+0000 7fecc959a640 1 -- 172.21.15.38:0/1078714276 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc031600 con 0x7feccc0b1930 2024-04-25T08:34:31.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.571+0000 7fecc959a640 1 -- 172.21.15.38:0/1078714276 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc0317a0 con 0x7feccc0b1930 2024-04-25T08:34:31.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.571+0000 7fecc959a640 1 -- 172.21.15.38:0/1078714276 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc031b70 con 0x7feccc0b1930 2024-04-25T08:34:31.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.571+0000 7fecd0970640 1 -- 172.21.15.38:0/1078714276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feccc0b1930 msgr2=0x7feccc15d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.571+0000 7fecd0970640 1 --2- 172.21.15.38:0/1078714276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feccc0b1930 0x7feccc15d140 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fecbc009d90 tx=0x7fecbc02a5d0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.571+0000 7fecd0970640 1 -- 172.21.15.38:0/1078714276 shutdown_connections 2024-04-25T08:34:31.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.571+0000 7fecd0970640 1 --2- 172.21.15.38:0/1078714276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feccc0b1930 0x7feccc15d140 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.571+0000 7fecd0970640 1 --2- 172.21.15.38:0/1078714276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feccc06b2f0 0x7feccc0b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.571+0000 7fecd0970640 1 -- 172.21.15.38:0/1078714276 >> 172.21.15.38:0/1078714276 conn(0x7feccc072610 msgr2=0x7feccc099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:31.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.571+0000 7fecd0970640 1 -- 172.21.15.38:0/1078714276 shutdown_connections 2024-04-25T08:34:31.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.571+0000 7fecd0970640 1 -- 172.21.15.38:0/1078714276 wait complete. 2024-04-25T08:34:31.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.573+0000 7fecd0970640 1 Processor -- start 2024-04-25T08:34:31.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.573+0000 7fecd0970640 1 -- start start 2024-04-25T08:34:31.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecd0970640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feccc06b2f0 0x7feccc151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecd0970640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feccc0b1930 0x7feccc1f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecd0970640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feccc15ba00 0x7feccc0c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecd0970640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc15c770 con 0x7feccc0b1930 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecd0970640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feccc0b1dc0 con 0x7feccc06b2f0 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecd0970640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feccc15bde0 con 0x7feccc15ba00 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecc9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feccc0b1930 0x7feccc1f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecc9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feccc0b1930 0x7feccc1f4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52962/0 (socket says 172.21.15.38:52962) 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecc9d9b640 1 -- 172.21.15.38:0/3354694493 learned_addr learned my addr 172.21.15.38:0/3354694493 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7feccad9d640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feccc15ba00 0x7feccc0c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecca59c640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feccc06b2f0 0x7feccc151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecc9d9b640 1 -- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feccc15ba00 msgr2=0x7feccc0c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecc9d9b640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feccc15ba00 0x7feccc0c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecc9d9b640 1 -- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feccc06b2f0 msgr2=0x7feccc151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecc9d9b640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feccc06b2f0 0x7feccc151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecc9d9b640 1 -- 172.21.15.38:0/3354694493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc146280 con 0x7feccc0b1930 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7feccad9d640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feccc15ba00 0x7feccc0c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecca59c640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feccc06b2f0 0x7feccc151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecc9d9b640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feccc0b1930 0x7feccc1f4a40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fecb400e7e0 tx=0x7fecb400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecc2ffd640 1 -- 172.21.15.38:0/3354694493 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb400f040 con 0x7feccc0b1930 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecc2ffd640 1 -- 172.21.15.38:0/3354694493 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb4004030 con 0x7feccc0b1930 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecd0970640 1 -- 172.21.15.38:0/3354694493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feccc0c6590 con 0x7feccc0b1930 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.574+0000 7fecc2ffd640 1 -- 172.21.15.38:0/3354694493 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb4004400 con 0x7feccc0b1930 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.575+0000 7fecd0970640 1 -- 172.21.15.38:0/3354694493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feccc0c6b50 con 0x7feccc0b1930 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.576+0000 7fecd0970640 1 -- 172.21.15.38:0/3354694493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feccc15b220 con 0x7feccc0b1930 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.576+0000 7fecc2ffd640 1 -- 172.21.15.38:0/3354694493 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecb400c6d0 con 0x7feccc0b1930 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.576+0000 7fecc2ffd640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecac06b4d0 0x7fecac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.576+0000 7fecc2ffd640 1 -- 172.21.15.38:0/3354694493 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fecb408a1d0 con 0x7feccc0b1930 2024-04-25T08:34:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.577+0000 7fecca59c640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecac06b4d0 0x7fecac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.577+0000 7fecca59c640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecac06b4d0 0x7fecac06d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fecc4007fd0 tx=0x7fecc4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:31.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.580+0000 7fecc2ffd640 1 -- 172.21.15.38:0/3354694493 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fecb4017020 con 0x7feccc0b1930 2024-04-25T08:34:31.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.796+0000 7fecd0970640 1 -- 172.21.15.38:0/3354694493 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feccc0cab80 con 0x7feccc0b1930 2024-04-25T08:34:31.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.797+0000 7fecc2ffd640 1 -- 172.21.15.38:0/3354694493 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fecb4058e90 con 0x7feccc0b1930 2024-04-25T08:34:31.792 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:31.792 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/422749749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":422749749},{"type":"v1","addr":"172.21.15.38:6834","nonce":422749749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5143,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/4194720752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":4194720752},{"type":"v1","addr":"172.21.15.107:6836","nonce":4194720752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3379884609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3379884609},{"type":"v1","addr":"172.21.15.107:6837","nonce":3379884609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:32:49.392279+0000","modified":"2024-04-25T08:32:57.539857+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5149},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5149":{"gid":5149,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/234860294","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":234860294},{"type":"v1","addr":"172.21.15.38:6835","nonce":234860294}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:34:31.793 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 18 2024-04-25T08:34:31.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.799+0000 7fecd0970640 1 -- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecac06b4d0 msgr2=0x7fecac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.799+0000 7fecd0970640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecac06b4d0 0x7fecac06d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fecc4007fd0 tx=0x7fecc4009040 comp rx=0 tx=0).stop 2024-04-25T08:34:31.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.799+0000 7fecd0970640 1 -- 172.21.15.38:0/3354694493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feccc0b1930 msgr2=0x7feccc1f4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.799+0000 7fecd0970640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feccc0b1930 0x7feccc1f4a40 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fecb400e7e0 tx=0x7fecb400eca0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.799+0000 7fecd0970640 1 -- 172.21.15.38:0/3354694493 shutdown_connections 2024-04-25T08:34:31.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.799+0000 7fecd0970640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecac06b4d0 0x7fecac06d990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.799+0000 7fecd0970640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feccc15ba00 0x7feccc0c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.799+0000 7fecd0970640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feccc0b1930 0x7feccc1f4a40 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.799+0000 7fecd0970640 1 --2- 172.21.15.38:0/3354694493 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feccc06b2f0 0x7feccc151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.799+0000 7fecd0970640 1 -- 172.21.15.38:0/3354694493 >> 172.21.15.38:0/3354694493 conn(0x7feccc072610 msgr2=0x7feccc06f9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:31.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.799+0000 7fecd0970640 1 -- 172.21.15.38:0/3354694493 shutdown_connections 2024-04-25T08:34:31.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.799+0000 7fecd0970640 1 -- 172.21.15.38:0/3354694493 wait complete. 2024-04-25T08:34:31.805 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:34:31.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.911+0000 7fad850dd640 1 Processor -- start 2024-04-25T08:34:31.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.911+0000 7fad850dd640 1 -- start start 2024-04-25T08:34:31.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.911+0000 7fad850dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d0140 0x7fad800d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.911+0000 7fad850dd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fad800d2400 0x7fad800d2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.911+0000 7fad850dd640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad80162160 con 0x7fad8015da90 2024-04-25T08:34:31.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.911+0000 7fad850dd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad800d0b00 con 0x7fad800d2400 2024-04-25T08:34:31.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.911+0000 7fad850dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad800d19b0 con 0x7fad800d0140 2024-04-25T08:34:31.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.912+0000 7fad7ed9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fad8015da90 0x7fad80161710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58392/0 (socket says 172.21.15.38:58392) 2024-04-25T08:34:31.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.912+0000 7fad7f59e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fad800d2400 0x7fad800d2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.912+0000 7fad7ed9d640 1 -- 172.21.15.38:0/3925622992 learned_addr learned my addr 172.21.15.38:0/3925622992 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:31.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.912+0000 7fad7e59c640 1 --2- 172.21.15.38:0/3925622992 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d0140 0x7fad800d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.912+0000 7fad7f59e640 1 -- 172.21.15.38:0/3925622992 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d0140 msgr2=0x7fad800d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:31.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.912+0000 7fad7f59e640 1 --2- 172.21.15.38:0/3925622992 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d0140 0x7fad800d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.912+0000 7fad7f59e640 1 -- 172.21.15.38:0/3925622992 >> v1:172.21.15.38:6789/0 conn(0x7fad8015da90 legacy=0x7fad80161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.912+0000 7fad7dd9b640 1 -- 172.21.15.38:0/3925622992 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284149525 0 0) 0x7fad80162160 con 0x7fad8015da90 2024-04-25T08:34:31.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.912+0000 7fad7f59e640 1 -- 172.21.15.38:0/3925622992 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad80162440 con 0x7fad800d2400 2024-04-25T08:34:31.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.913+0000 7fad7f59e640 1 --2- 172.21.15.38:0/3925622992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad800d2400 0x7fad800d2890 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fad70009dc0 tx=0x7fad7002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5daf6421d69962 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:31.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.913+0000 7fad7dd9b640 1 -- 172.21.15.38:0/3925622992 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7002ae60 con 0x7fad800d2400 2024-04-25T08:34:31.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.913+0000 7fad7dd9b640 1 -- 172.21.15.38:0/3925622992 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad70007740 con 0x7fad800d2400 2024-04-25T08:34:31.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.913+0000 7fad7dd9b640 1 -- 172.21.15.38:0/3925622992 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70007b10 con 0x7fad800d2400 2024-04-25T08:34:31.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.913+0000 7fad850dd640 1 -- 172.21.15.38:0/3925622992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad800d2400 msgr2=0x7fad800d2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.913+0000 7fad850dd640 1 --2- 172.21.15.38:0/3925622992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad800d2400 0x7fad800d2890 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fad70009dc0 tx=0x7fad7002a680 comp rx=0 tx=0).stop 2024-04-25T08:34:31.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.913+0000 7fad850dd640 1 -- 172.21.15.38:0/3925622992 shutdown_connections 2024-04-25T08:34:31.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.913+0000 7fad850dd640 1 --2- 172.21.15.38:0/3925622992 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad800d2400 0x7fad800d2890 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.913+0000 7fad850dd640 1 --2- 172.21.15.38:0/3925622992 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d0140 0x7fad800d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.913+0000 7fad850dd640 1 -- 172.21.15.38:0/3925622992 >> 172.21.15.38:0/3925622992 conn(0x7fad8006c060 msgr2=0x7fad8006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:31.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.913+0000 7fad850dd640 1 -- 172.21.15.38:0/3925622992 shutdown_connections 2024-04-25T08:34:31.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.913+0000 7fad850dd640 1 -- 172.21.15.38:0/3925622992 wait complete. 2024-04-25T08:34:31.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.915+0000 7fad850dd640 1 Processor -- start 2024-04-25T08:34:31.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.915+0000 7fad850dd640 1 -- start start 2024-04-25T08:34:31.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad850dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d0140 0x7fad8015e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad850dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad800d2400 0x7fad8015b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad850dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad8015da90 0x7fad8014e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad850dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad80161e20 con 0x7fad8015da90 2024-04-25T08:34:31.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad850dd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad800d1f70 con 0x7fad800d2400 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad850dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad800b9400 con 0x7fad800d0140 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad7f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad8015da90 0x7fad8014e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad7f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad8015da90 0x7fad8014e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52982/0 (socket says 172.21.15.38:52982) 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad7f59e640 1 -- 172.21.15.38:0/3670252415 learned_addr learned my addr 172.21.15.38:0/3670252415 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad7ed9d640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d0140 0x7fad8015e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad7e59c640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad800d2400 0x7fad8015b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad7f59e640 1 -- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d0140 msgr2=0x7fad8015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad7f59e640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d0140 0x7fad8015e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad7f59e640 1 -- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad800d2400 msgr2=0x7fad8015b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad7f59e640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad800d2400 0x7fad8015b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad7f59e640 1 -- 172.21.15.38:0/3670252415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad800c75e0 con 0x7fad8015da90 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.916+0000 7fad7f59e640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad8015da90 0x7fad8014e400 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fad70009ee0 tx=0x7fad70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.917+0000 7fad777fe640 1 -- 172.21.15.38:0/3670252415 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70040070 con 0x7fad8015da90 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.917+0000 7fad777fe640 1 -- 172.21.15.38:0/3670252415 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad70031d40 con 0x7fad8015da90 2024-04-25T08:34:31.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.917+0000 7fad850dd640 1 -- 172.21.15.38:0/3670252415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad800c78c0 con 0x7fad8015da90 2024-04-25T08:34:31.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.917+0000 7fad777fe640 1 -- 172.21.15.38:0/3670252415 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70009450 con 0x7fad8015da90 2024-04-25T08:34:31.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.917+0000 7fad850dd640 1 -- 172.21.15.38:0/3670252415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8015e970 con 0x7fad8015da90 2024-04-25T08:34:31.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.918+0000 7fad850dd640 1 -- 172.21.15.38:0/3670252415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad80068e80 con 0x7fad8015da90 2024-04-25T08:34:31.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.918+0000 7fad777fe640 1 -- 172.21.15.38:0/3670252415 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad70039040 con 0x7fad8015da90 2024-04-25T08:34:31.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.918+0000 7fad777fe640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad6006b500 0x7fad6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:31.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.918+0000 7fad777fe640 1 -- 172.21.15.38:0/3670252415 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fad700ab510 con 0x7fad8015da90 2024-04-25T08:34:31.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.919+0000 7fad7ed9d640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad6006b500 0x7fad6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:31.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.919+0000 7fad7ed9d640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad6006b500 0x7fad6006d9c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fad78007fd0 tx=0x7fad78007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:31.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:31.923+0000 7fad777fe640 1 -- 172.21.15.38:0/3670252415 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad70079d30 con 0x7fad8015da90 2024-04-25T08:34:32.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.104+0000 7fad850dd640 1 -- 172.21.15.38:0/3670252415 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad800d1560 con 0x7fad8015da90 2024-04-25T08:34:32.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.105+0000 7fad777fe640 1 -- 172.21.15.38:0/3670252415 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+17044 (secure 0 0 0) 0x7fad7007ddd0 con 0x7fad8015da90 2024-04-25T08:34:32.100 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:32.100 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":42,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:34:05.884436+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:32:47.798666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:32:48.371717+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":41,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.38","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:34:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.107+0000 7fad850dd640 1 -- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad6006b500 msgr2=0x7fad6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.107+0000 7fad850dd640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad6006b500 0x7fad6006d9c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fad78007fd0 tx=0x7fad78007de0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.107+0000 7fad850dd640 1 -- 172.21.15.38:0/3670252415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad8015da90 msgr2=0x7fad8014e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.107+0000 7fad850dd640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad8015da90 0x7fad8014e400 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fad70009ee0 tx=0x7fad70004a40 comp rx=0 tx=0).stop 2024-04-25T08:34:32.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.107+0000 7fad850dd640 1 -- 172.21.15.38:0/3670252415 shutdown_connections 2024-04-25T08:34:32.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.107+0000 7fad850dd640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad8015da90 0x7fad8014e400 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.107+0000 7fad850dd640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad800d2400 0x7fad8015b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.107+0000 7fad850dd640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad6006b500 0x7fad6006d9c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.107+0000 7fad850dd640 1 --2- 172.21.15.38:0/3670252415 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad800d0140 0x7fad8015e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.107+0000 7fad850dd640 1 -- 172.21.15.38:0/3670252415 >> 172.21.15.38:0/3670252415 conn(0x7fad8006c060 msgr2=0x7fad800b6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:32.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.107+0000 7fad850dd640 1 -- 172.21.15.38:0/3670252415 shutdown_connections 2024-04-25T08:34:32.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.107+0000 7fad850dd640 1 -- 172.21.15.38:0/3670252415 wait complete. 2024-04-25T08:34:32.114 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:34:32.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.220+0000 7fdeec342640 1 Processor -- start 2024-04-25T08:34:32.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.221+0000 7fdeec342640 1 -- start start 2024-04-25T08:34:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.221+0000 7fdeec342640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdee40d0d10 0x7fdee4099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.221+0000 7fdeec342640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee4099fb0 0x7fdee40cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.221+0000 7fdeec342640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee40cfc70 con 0x7fdee415def0 2024-04-25T08:34:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.221+0000 7fdeec342640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee409a420 con 0x7fdee40d0d10 2024-04-25T08:34:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.221+0000 7fdeec342640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee415e2d0 con 0x7fdee4099fb0 2024-04-25T08:34:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.222+0000 7fdee98dd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdee40d0d10 0x7fdee4099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.222+0000 7fdeea0de640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdee415def0 0x7fdee406c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58394/0 (socket says 172.21.15.38:58394) 2024-04-25T08:34:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.222+0000 7fdeea0de640 1 -- 172.21.15.38:0/2085975735 learned_addr learned my addr 172.21.15.38:0/2085975735 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.222+0000 7fdee98dd640 1 -- 172.21.15.38:0/2085975735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee4099fb0 msgr2=0x7fdee40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.222+0000 7fdeea8df640 1 --2- 172.21.15.38:0/2085975735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee4099fb0 0x7fdee40cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.222+0000 7fdee90dc640 1 -- 172.21.15.38:0/2085975735 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193508887 0 0) 0x7fdee40cfc70 con 0x7fdee415def0 2024-04-25T08:34:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.222+0000 7fdee98dd640 1 --2- 172.21.15.38:0/2085975735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee4099fb0 0x7fdee40cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.222+0000 7fdee98dd640 1 -- 172.21.15.38:0/2085975735 >> v1:172.21.15.38:6789/0 conn(0x7fdee415def0 legacy=0x7fdee406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.222+0000 7fdee98dd640 1 -- 172.21.15.38:0/2085975735 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee40cff50 con 0x7fdee40d0d10 2024-04-25T08:34:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.222+0000 7fdeea8df640 1 --2- 172.21.15.38:0/2085975735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee4099fb0 0x7fdee40cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.223+0000 7fdee98dd640 1 --2- 172.21.15.38:0/2085975735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40d0d10 0x7fdee4099a70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fded80097a0 tx=0x7fded802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4b2624823de9b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.223+0000 7fdee90dc640 1 -- 172.21.15.38:0/2085975735 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8037070 con 0x7fdee40d0d10 2024-04-25T08:34:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.223+0000 7fdee90dc640 1 -- 172.21.15.38:0/2085975735 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded8032440 con 0x7fdee40d0d10 2024-04-25T08:34:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.223+0000 7fdee90dc640 1 -- 172.21.15.38:0/2085975735 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8032810 con 0x7fdee40d0d10 2024-04-25T08:34:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.223+0000 7fdeec342640 1 -- 172.21.15.38:0/2085975735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40d0d10 msgr2=0x7fdee4099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.223+0000 7fdeec342640 1 --2- 172.21.15.38:0/2085975735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40d0d10 0x7fdee4099a70 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fded80097a0 tx=0x7fded802a180 comp rx=0 tx=0).stop 2024-04-25T08:34:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.223+0000 7fdeec342640 1 -- 172.21.15.38:0/2085975735 shutdown_connections 2024-04-25T08:34:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.223+0000 7fdeec342640 1 --2- 172.21.15.38:0/2085975735 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee4099fb0 0x7fdee40cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.223+0000 7fdeec342640 1 --2- 172.21.15.38:0/2085975735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee40d0d10 0x7fdee4099a70 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.223+0000 7fdeec342640 1 -- 172.21.15.38:0/2085975735 >> 172.21.15.38:0/2085975735 conn(0x7fdee40b30f0 msgr2=0x7fdee40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.223+0000 7fdeec342640 1 -- 172.21.15.38:0/2085975735 shutdown_connections 2024-04-25T08:34:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.223+0000 7fdeec342640 1 -- 172.21.15.38:0/2085975735 wait complete. 2024-04-25T08:34:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.225+0000 7fdeec342640 1 Processor -- start 2024-04-25T08:34:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.225+0000 7fdeec342640 1 -- start start 2024-04-25T08:34:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.225+0000 7fdeec342640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4099fb0 0x7fdee409dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.225+0000 7fdeec342640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee40d0d10 0x7fdee409e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.225+0000 7fdeec342640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee415def0 0x7fdee414aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.225+0000 7fdeec342640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40d07f0 con 0x7fdee4099fb0 2024-04-25T08:34:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.225+0000 7fdeec342640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40d0510 con 0x7fdee415def0 2024-04-25T08:34:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.225+0000 7fdeec342640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee40d1170 con 0x7fdee40d0d10 2024-04-25T08:34:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.225+0000 7fdeea0de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4099fb0 0x7fdee409dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:32.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.225+0000 7fdeea0de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4099fb0 0x7fdee409dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52992/0 (socket says 172.21.15.38:52992) 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.225+0000 7fdeea0de640 1 -- 172.21.15.38:0/99769944 learned_addr learned my addr 172.21.15.38:0/99769944 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdee98dd640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee40d0d10 0x7fdee409e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdeea8df640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee415def0 0x7fdee414aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdeea0de640 1 -- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee40d0d10 msgr2=0x7fdee409e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdeea0de640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee40d0d10 0x7fdee409e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdeea0de640 1 -- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee415def0 msgr2=0x7fdee414aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdeea0de640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee415def0 0x7fdee414aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdeea0de640 1 -- 172.21.15.38:0/99769944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee40b5ef0 con 0x7fdee4099fb0 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdeea0de640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4099fb0 0x7fdee409dd60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00c4d0 tx=0x7fdedc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdee2ffd640 1 -- 172.21.15.38:0/99769944 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc018070 con 0x7fdee4099fb0 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdee2ffd640 1 -- 172.21.15.38:0/99769944 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdedc007500 con 0x7fdee4099fb0 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdeec342640 1 -- 172.21.15.38:0/99769944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee4148af0 con 0x7fdee4099fb0 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdee2ffd640 1 -- 172.21.15.38:0/99769944 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc004a20 con 0x7fdee4099fb0 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.226+0000 7fdeec342640 1 -- 172.21.15.38:0/99769944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee414af90 con 0x7fdee4099fb0 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.227+0000 7fdeec342640 1 -- 172.21.15.38:0/99769944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee404ef90 con 0x7fdee4099fb0 2024-04-25T08:34:32.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.227+0000 7fdee2ffd640 1 -- 172.21.15.38:0/99769944 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdedc01b070 con 0x7fdee4099fb0 2024-04-25T08:34:32.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.228+0000 7fdee2ffd640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fded006b4d0 0x7fded006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:32.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.228+0000 7fdee2ffd640 1 -- 172.21.15.38:0/99769944 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fdedc089680 con 0x7fdee4099fb0 2024-04-25T08:34:32.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.228+0000 7fdee98dd640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fded006b4d0 0x7fded006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:32.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.229+0000 7fdee98dd640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fded006b4d0 0x7fded006d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fded8034000 tx=0x7fded802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:32.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.232+0000 7fdee2ffd640 1 -- 172.21.15.38:0/99769944 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdedc0583d0 con 0x7fdee4099fb0 2024-04-25T08:34:32.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.455+0000 7fdeec342640 1 -- 172.21.15.38:0/99769944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdee409af90 con 0x7fdee4099fb0 2024-04-25T08:34:32.450 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:34:32.455+0000 7ff7937f0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:34:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.458+0000 7fdee2ffd640 1 -- 172.21.15.38:0/99769944 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdedc05e0d0 con 0x7fdee4099fb0 2024-04-25T08:34:32.453 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:34:32.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.461+0000 7fdeec342640 1 -- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fded006b4d0 msgr2=0x7fded006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:32.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.461+0000 7fdeec342640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fded006b4d0 0x7fded006d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fded8034000 tx=0x7fded802e000 comp rx=0 tx=0).stop 2024-04-25T08:34:32.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.461+0000 7fdeec342640 1 -- 172.21.15.38:0/99769944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4099fb0 msgr2=0x7fdee409dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:32.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.461+0000 7fdeec342640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4099fb0 0x7fdee409dd60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdedc00c4d0 tx=0x7fdedc00c990 comp rx=0 tx=0).stop 2024-04-25T08:34:32.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.461+0000 7fdeec342640 1 -- 172.21.15.38:0/99769944 shutdown_connections 2024-04-25T08:34:32.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.461+0000 7fdeec342640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdee415def0 0x7fdee414aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.461+0000 7fdeec342640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fded006b4d0 0x7fded006d990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.461+0000 7fdeec342640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdee40d0d10 0x7fdee409e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.461+0000 7fdeec342640 1 --2- 172.21.15.38:0/99769944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdee4099fb0 0x7fdee409dd60 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.461+0000 7fdeec342640 1 -- 172.21.15.38:0/99769944 >> 172.21.15.38:0/99769944 conn(0x7fdee40b30f0 msgr2=0x7fdee415b940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.461+0000 7fdeec342640 1 -- 172.21.15.38:0/99769944 shutdown_connections 2024-04-25T08:34:32.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.461+0000 7fdeec342640 1 -- 172.21.15.38:0/99769944 wait complete. 2024-04-25T08:34:32.468 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:34:32.496 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:34:32.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.577+0000 7f0b80dd5640 1 Processor -- start 2024-04-25T08:34:32.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.577+0000 7f0b80dd5640 1 -- start start 2024-04-25T08:34:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.578+0000 7f0b80dd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c0d33d0 0x7f0b7c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:32.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.578+0000 7f0b80dd5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0b7c0d1e90 0x7f0b7c0b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.578+0000 7f0b80dd5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b7c0d2330 con 0x7f0b7c0d0320 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.578+0000 7f0b80dd5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7c0d2610 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.578+0000 7f0b80dd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c0d28f0 con 0x7f0b7c0d33d0 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.578+0000 7f0b7ad9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0b7c0d1e90 0x7f0b7c0b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.578+0000 7f0b7ad9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0b7c0d1e90 0x7f0b7c0b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53006/0 (socket says 172.21.15.38:53006) 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.578+0000 7f0b7ad9d640 1 -- 172.21.15.38:0/4043248822 learned_addr learned my addr 172.21.15.38:0/4043248822 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.578+0000 7f0b79d9b640 1 --2- 172.21.15.38:0/4043248822 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c0d33d0 0x7f0b7c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.579+0000 7f0b7ad9d640 1 -- 172.21.15.38:0/4043248822 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c0d33d0 msgr2=0x7f0b7c0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.579+0000 7f0b7ad9d640 1 --2- 172.21.15.38:0/4043248822 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c0d33d0 0x7f0b7c0d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.579+0000 7f0b7ad9d640 1 -- 172.21.15.38:0/4043248822 >> v1:172.21.15.107:6789/0 conn(0x7f0b7c0d0320 legacy=0x7f0b7c0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.579+0000 7f0b7ad9d640 1 -- 172.21.15.38:0/4043248822 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c0d2bd0 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.579+0000 7f0b79d9b640 1 --2- 172.21.15.38:0/4043248822 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c0d33d0 0x7f0b7c0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.579+0000 7f0b7ad9d640 1 --2- 172.21.15.38:0/4043248822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0d1e90 0x7f0b7c0b6980 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009e50 tx=0x7f0b7402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2184c2371fc2ee94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.580+0000 7f0b7959a640 1 -- 172.21.15.38:0/4043248822 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74036070 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.580+0000 7f0b7959a640 1 -- 172.21.15.38:0/4043248822 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b74031600 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.580+0000 7f0b7959a640 1 -- 172.21.15.38:0/4043248822 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b740319d0 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.580+0000 7f0b80dd5640 1 -- 172.21.15.38:0/4043248822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0d1e90 msgr2=0x7f0b7c0b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.580+0000 7f0b80dd5640 1 --2- 172.21.15.38:0/4043248822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0d1e90 0x7f0b7c0b6980 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009e50 tx=0x7f0b7402a7a0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.580+0000 7f0b80dd5640 1 -- 172.21.15.38:0/4043248822 shutdown_connections 2024-04-25T08:34:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.580+0000 7f0b80dd5640 1 --2- 172.21.15.38:0/4043248822 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0d1e90 0x7f0b7c0b6980 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.580+0000 7f0b80dd5640 1 --2- 172.21.15.38:0/4043248822 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c0d33d0 0x7f0b7c0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.580+0000 7f0b80dd5640 1 -- 172.21.15.38:0/4043248822 >> 172.21.15.38:0/4043248822 conn(0x7f0b7c0668f0 msgr2=0x7f0b7c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.580+0000 7f0b80dd5640 1 -- 172.21.15.38:0/4043248822 shutdown_connections 2024-04-25T08:34:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.580+0000 7f0b80dd5640 1 -- 172.21.15.38:0/4043248822 wait complete. 2024-04-25T08:34:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.581+0000 7f0b80dd5640 1 Processor -- start 2024-04-25T08:34:32.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.582+0000 7f0b80dd5640 1 -- start start 2024-04-25T08:34:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.582+0000 7f0b80dd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0d0320 0x7f0b7c1f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.582+0000 7f0b80dd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0d1e90 0x7f0b7c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.582+0000 7f0b80dd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c0d33d0 0x7f0b7c0cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.582+0000 7f0b80dd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c161390 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.582+0000 7f0b80dd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c161210 con 0x7f0b7c0d0320 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.582+0000 7f0b80dd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c0b9720 con 0x7f0b7c0d33d0 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.582+0000 7f0b79d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0d1e90 0x7f0b7c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.582+0000 7f0b79d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0d1e90 0x7f0b7c0ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53018/0 (socket says 172.21.15.38:53018) 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.582+0000 7f0b79d9b640 1 -- 172.21.15.38:0/3377479171 learned_addr learned my addr 172.21.15.38:0/3377479171 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.582+0000 7f0b7a59c640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0d0320 0x7f0b7c1f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.583+0000 7f0b7ad9d640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c0d33d0 0x7f0b7c0cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.583+0000 7f0b79d9b640 1 -- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c0d33d0 msgr2=0x7f0b7c0cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.583+0000 7f0b79d9b640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c0d33d0 0x7f0b7c0cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.583+0000 7f0b79d9b640 1 -- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0d0320 msgr2=0x7f0b7c1f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.583+0000 7f0b79d9b640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0d0320 0x7f0b7c1f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.583+0000 7f0b79d9b640 1 -- 172.21.15.38:0/3377479171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c15a5d0 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.583+0000 7f0b79d9b640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0d1e90 0x7f0b7c0ccc50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000d7d0 tx=0x7f0b7000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:32.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.583+0000 7f0b6affd640 1 -- 172.21.15.38:0/3377479171 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70004030 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.583+0000 7f0b6affd640 1 -- 172.21.15.38:0/3377479171 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b700041d0 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.583+0000 7f0b80dd5640 1 -- 172.21.15.38:0/3377479171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7c1599c0 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.583+0000 7f0b6affd640 1 -- 172.21.15.38:0/3377479171 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700096a0 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.583+0000 7f0b80dd5640 1 -- 172.21.15.38:0/3377479171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7c159f80 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.584+0000 7f0b80dd5640 1 -- 172.21.15.38:0/3377479171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7c0b27e0 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.584+0000 7f0b6affd640 1 -- 172.21.15.38:0/3377479171 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b70009840 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.585+0000 7f0b6affd640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b5c06b4d0 0x7f0b5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:32.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.585+0000 7f0b6affd640 1 -- 172.21.15.38:0/3377479171 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f0b7008a5c0 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.585+0000 7f0b7a59c640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b5c06b4d0 0x7f0b5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:32.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.586+0000 7f0b7a59c640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b5c06b4d0 0x7f0b5c06d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c001780 tx=0x7f0b6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:32.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.589+0000 7f0b6affd640 1 -- 172.21.15.38:0/3377479171 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b70058bd0 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:32.806+0000 7f0b80dd5640 1 -- 172.21.15.38:0/3377479171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0b7c068e80 con 0x7f0b7c0d1e90 2024-04-25T08:34:32.906 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:34:32.909 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:34:33.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.460+0000 7f0b6affd640 1 -- 172.21.15.38:0/3377479171 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0b7005cc70 con 0x7f0b7c0d1e90 2024-04-25T08:34:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.462+0000 7f0b80dd5640 1 -- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b5c06b4d0 msgr2=0x7f0b5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.462+0000 7f0b80dd5640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b5c06b4d0 0x7f0b5c06d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c001780 tx=0x7f0b6c009040 comp rx=0 tx=0).stop 2024-04-25T08:34:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.462+0000 7f0b80dd5640 1 -- 172.21.15.38:0/3377479171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0d1e90 msgr2=0x7f0b7c0ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.462+0000 7f0b80dd5640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0d1e90 0x7f0b7c0ccc50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0b7000d7d0 tx=0x7f0b7000dc90 comp rx=0 tx=0).stop 2024-04-25T08:34:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.462+0000 7f0b80dd5640 1 -- 172.21.15.38:0/3377479171 shutdown_connections 2024-04-25T08:34:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.462+0000 7f0b80dd5640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b5c06b4d0 0x7f0b5c06d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.462+0000 7f0b80dd5640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c0d33d0 0x7f0b7c0cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.462+0000 7f0b80dd5640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0d1e90 0x7f0b7c0ccc50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.462+0000 7f0b80dd5640 1 --2- 172.21.15.38:0/3377479171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b7c0d0320 0x7f0b7c1f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.462+0000 7f0b80dd5640 1 -- 172.21.15.38:0/3377479171 >> 172.21.15.38:0/3377479171 conn(0x7f0b7c0668f0 msgr2=0x7f0b7c0b8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.462+0000 7f0b80dd5640 1 -- 172.21.15.38:0/3377479171 shutdown_connections 2024-04-25T08:34:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.462+0000 7f0b80dd5640 1 -- 172.21.15.38:0/3377479171 wait complete. 2024-04-25T08:34:33.471 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:34:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.583+0000 7f11649cf640 1 Processor -- start 2024-04-25T08:34:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.583+0000 7f11649cf640 1 -- start start 2024-04-25T08:34:33.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.584+0000 7f11649cf640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11600d0860 0x7f11600d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:33.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.584+0000 7f11649cf640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f11600d19e0 0x7f11600d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:33.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.584+0000 7f11649cf640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f11600d3860 0x7f11600d3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:33.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.584+0000 7f11649cf640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116015e2b0 con 0x7f11600d19e0 2024-04-25T08:34:33.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.584+0000 7f11649cf640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11600d1180 con 0x7f11600d3860 2024-04-25T08:34:33.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.584+0000 7f11649cf640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11600d24a0 con 0x7f11600d0860 2024-04-25T08:34:33.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.584+0000 7f115dd9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f11600d19e0 0x7f11600d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.584+0000 7f115dd9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f11600d19e0 0x7f11600d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53032/0 (socket says 172.21.15.38:53032) 2024-04-25T08:34:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.584+0000 7f115dd9b640 1 -- 172.21.15.38:0/3444779500 learned_addr learned my addr 172.21.15.38:0/3444779500 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.585+0000 7f115e59c640 1 --2- 172.21.15.38:0/3444779500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11600d0860 0x7f11600d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.585+0000 7f115ed9d640 1 --2- 172.21.15.38:0/3444779500 >> v2:172.21.15.107:3300/0 conn(0x7f11600d3860 0x7f11600d3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.585+0000 7f115dd9b640 1 -- 172.21.15.38:0/3444779500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11600d0860 msgr2=0x7f11600d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.585+0000 7f115dd9b640 1 --2- 172.21.15.38:0/3444779500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11600d0860 0x7f11600d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.585+0000 7f115dd9b640 1 -- 172.21.15.38:0/3444779500 >> v2:172.21.15.107:3300/0 conn(0x7f11600d3860 msgr2=0x7f11600d3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.585+0000 7f115dd9b640 1 --2- 172.21.15.38:0/3444779500 >> v2:172.21.15.107:3300/0 conn(0x7f11600d3860 0x7f11600d3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.585+0000 7f115dd9b640 1 -- 172.21.15.38:0/3444779500 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11600d2780 con 0x7f11600d19e0 2024-04-25T08:34:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.585+0000 7f115dd9b640 1 --2- 172.21.15.38:0/3444779500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11600d19e0 0x7f11600d1e20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f11600ce0d0 tx=0x7f114802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef7d4f1059be633 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:33.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.586+0000 7f115d59a640 1 -- 172.21.15.38:0/3444779500 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148037070 con 0x7f11600d19e0 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.586+0000 7f115d59a640 1 -- 172.21.15.38:0/3444779500 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148032440 con 0x7f11600d19e0 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.586+0000 7f115d59a640 1 -- 172.21.15.38:0/3444779500 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148032810 con 0x7f11600d19e0 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.586+0000 7f11649cf640 1 -- 172.21.15.38:0/3444779500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11600d19e0 msgr2=0x7f11600d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.586+0000 7f11649cf640 1 --2- 172.21.15.38:0/3444779500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11600d19e0 0x7f11600d1e20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f11600ce0d0 tx=0x7f114802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.586+0000 7f11649cf640 1 -- 172.21.15.38:0/3444779500 shutdown_connections 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.586+0000 7f11649cf640 1 --2- 172.21.15.38:0/3444779500 >> v2:172.21.15.107:3300/0 conn(0x7f11600d3860 0x7f11600d3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.586+0000 7f11649cf640 1 --2- 172.21.15.38:0/3444779500 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11600d19e0 0x7f11600d1e20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.586+0000 7f11649cf640 1 --2- 172.21.15.38:0/3444779500 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11600d0860 0x7f11600d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.586+0000 7f11649cf640 1 -- 172.21.15.38:0/3444779500 >> 172.21.15.38:0/3444779500 conn(0x7f1160066d10 msgr2=0x7f1160067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.586+0000 7f11649cf640 1 -- 172.21.15.38:0/3444779500 shutdown_connections 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.586+0000 7f11649cf640 1 -- 172.21.15.38:0/3444779500 wait complete. 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.588+0000 7f11649cf640 1 Processor -- start 2024-04-25T08:34:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.588+0000 7f11649cf640 1 -- start start 2024-04-25T08:34:33.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.588+0000 7f11649cf640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11600d0860 0x7f11601f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:33.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.588+0000 7f11649cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11600d19e0 0x7f11601e3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:33.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.588+0000 7f11649cf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11600d3860 0x7f11600c95d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.588+0000 7f11649cf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11600d3540 con 0x7f11600d3860 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.588+0000 7f11649cf640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11600d2d40 con 0x7f11600d19e0 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.588+0000 7f11649cf640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11600b9770 con 0x7f11600d0860 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.588+0000 7f115ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11600d3860 0x7f11600c95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.588+0000 7f115ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11600d3860 0x7f11600c95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53042/0 (socket says 172.21.15.38:53042) 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.588+0000 7f115ed9d640 1 -- 172.21.15.38:0/2559192983 learned_addr learned my addr 172.21.15.38:0/2559192983 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.589+0000 7f115ed9d640 1 -- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11600d0860 msgr2=0x7f11601f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.589+0000 7f115ed9d640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11600d0860 0x7f11601f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.589+0000 7f115ed9d640 1 -- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11600d19e0 msgr2=0x7f11601e3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.589+0000 7f115dd9b640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11600d19e0 0x7f11601e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.589+0000 7f115ed9d640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11600d19e0 0x7f11601e3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.589+0000 7f115ed9d640 1 -- 172.21.15.38:0/2559192983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11601e3ad0 con 0x7f11600d3860 2024-04-25T08:34:33.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.589+0000 7f115ed9d640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11600d3860 0x7f11600c95d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1150007e60 tx=0x7f115000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:33.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.589+0000 7f1156ffd640 1 -- 172.21.15.38:0/2559192983 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150014420 con 0x7f11600d3860 2024-04-25T08:34:33.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.589+0000 7f1156ffd640 1 -- 172.21.15.38:0/2559192983 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1150019070 con 0x7f11600d3860 2024-04-25T08:34:33.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.589+0000 7f11649cf640 1 -- 172.21.15.38:0/2559192983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11601e3db0 con 0x7f11600d3860 2024-04-25T08:34:33.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.589+0000 7f1156ffd640 1 -- 172.21.15.38:0/2559192983 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150014820 con 0x7f11600d3860 2024-04-25T08:34:33.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.589+0000 7f11649cf640 1 -- 172.21.15.38:0/2559192983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11601e46c0 con 0x7f11600d3860 2024-04-25T08:34:33.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.590+0000 7f11649cf640 1 -- 172.21.15.38:0/2559192983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1160068e80 con 0x7f11600d3860 2024-04-25T08:34:33.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.590+0000 7f1156ffd640 1 -- 172.21.15.38:0/2559192983 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11500149c0 con 0x7f11600d3860 2024-04-25T08:34:33.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.591+0000 7f1156ffd640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f114006b4d0 0x7f114006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:33.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.591+0000 7f1156ffd640 1 -- 172.21.15.38:0/2559192983 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f115008abb0 con 0x7f11600d3860 2024-04-25T08:34:33.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.591+0000 7f115e59c640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f114006b4d0 0x7f114006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:33.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.592+0000 7f115e59c640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f114006b4d0 0x7f114006d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f11600b8640 tx=0x7f1158008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:33.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.595+0000 7f1156ffd640 1 -- 172.21.15.38:0/2559192983 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11500598c0 con 0x7f11600d3860 2024-04-25T08:34:33.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:33.779+0000 7f11649cf640 1 -- 172.21.15.38:0/2559192983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f11601e48c0 con 0x7f11600d3860 2024-04-25T08:34:34.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.462+0000 7f1156ffd640 1 -- 172.21.15.38:0/2559192983 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7f11500591c0 con 0x7f11600d3860 2024-04-25T08:34:34.457 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:34:34.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.464+0000 7f11649cf640 1 -- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f114006b4d0 msgr2=0x7f114006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:34.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.464+0000 7f11649cf640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f114006b4d0 0x7f114006d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f11600b8640 tx=0x7f1158008040 comp rx=0 tx=0).stop 2024-04-25T08:34:34.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.464+0000 7f11649cf640 1 -- 172.21.15.38:0/2559192983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11600d3860 msgr2=0x7f11600c95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:34.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.464+0000 7f11649cf640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11600d3860 0x7f11600c95d0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f1150007e60 tx=0x7f115000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:34:34.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.464+0000 7f11649cf640 1 -- 172.21.15.38:0/2559192983 shutdown_connections 2024-04-25T08:34:34.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.464+0000 7f11649cf640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11600d3860 0x7f11600c95d0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:34.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.464+0000 7f11649cf640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11600d19e0 0x7f11601e3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:34.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.464+0000 7f11649cf640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f114006b4d0 0x7f114006d990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:34.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.464+0000 7f11649cf640 1 --2- 172.21.15.38:0/2559192983 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11600d0860 0x7f11601f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:34.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.464+0000 7f11649cf640 1 -- 172.21.15.38:0/2559192983 >> 172.21.15.38:0/2559192983 conn(0x7f1160066d10 msgr2=0x7f11600cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:34.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.465+0000 7f11649cf640 1 -- 172.21.15.38:0/2559192983 shutdown_connections 2024-04-25T08:34:34.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.465+0000 7f11649cf640 1 -- 172.21.15.38:0/2559192983 wait complete. 2024-04-25T08:34:34.473 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:34:34.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.600+0000 7ffa9225d640 1 Processor -- start 2024-04-25T08:34:34.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.600+0000 7ffa9225d640 1 -- start start 2024-04-25T08:34:34.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa9225d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ffa8c1612c0 0x7ffa8c0b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa9225d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffa8c0d0870 0x7ffa8c0d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa9225d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa8c0b7200 con 0x7ffa8c0b6a00 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa9225d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8c0d11e0 con 0x7ffa8c0d0870 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa9225d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa8c0b74e0 con 0x7ffa8c1612c0 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa8bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffa8c0d0870 0x7ffa8c0d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa8affd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ffa8c0b6a00 0x7ffa8c0d1c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58414/0 (socket says 172.21.15.38:58414) 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa8affd640 1 -- 172.21.15.38:0/3715301723 learned_addr learned my addr 172.21.15.38:0/3715301723 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa8b7fe640 1 --2- 172.21.15.38:0/3715301723 >> v2:172.21.15.107:3300/0 conn(0x7ffa8c1612c0 0x7ffa8c0b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa8bfff640 1 -- 172.21.15.38:0/3715301723 >> v2:172.21.15.107:3300/0 conn(0x7ffa8c1612c0 msgr2=0x7ffa8c0b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa8bfff640 1 --2- 172.21.15.38:0/3715301723 >> v2:172.21.15.107:3300/0 conn(0x7ffa8c1612c0 0x7ffa8c0b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa8bfff640 1 -- 172.21.15.38:0/3715301723 >> v1:172.21.15.38:6789/0 conn(0x7ffa8c0b6a00 legacy=0x7ffa8c0d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa8bfff640 1 -- 172.21.15.38:0/3715301723 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c0b77c0 con 0x7ffa8c0d0870 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa8a7fc640 1 -- 172.21.15.38:0/3715301723 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020395006 0 0) 0x7ffa8c0b7200 con 0x7ffa8c0b6a00 2024-04-25T08:34:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.601+0000 7ffa8b7fe640 1 --2- 172.21.15.38:0/3715301723 >> v2:172.21.15.107:3300/0 conn(0x7ffa8c1612c0 0x7ffa8c0b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:34.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.602+0000 7ffa8bfff640 1 --2- 172.21.15.38:0/3715301723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa8c0d0870 0x7ffa8c0d0ca0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0d49a0 tx=0x7ffa8402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15bf119b0f67093 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:34.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.602+0000 7ffa8a7fc640 1 -- 172.21.15.38:0/3715301723 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84007740 con 0x7ffa8c0d0870 2024-04-25T08:34:34.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.602+0000 7ffa8a7fc640 1 -- 172.21.15.38:0/3715301723 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa840078e0 con 0x7ffa8c0d0870 2024-04-25T08:34:34.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.602+0000 7ffa8a7fc640 1 -- 172.21.15.38:0/3715301723 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84007cb0 con 0x7ffa8c0d0870 2024-04-25T08:34:34.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.603+0000 7ffa9225d640 1 -- 172.21.15.38:0/3715301723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa8c0d0870 msgr2=0x7ffa8c0d0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:34.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.603+0000 7ffa9225d640 1 --2- 172.21.15.38:0/3715301723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa8c0d0870 0x7ffa8c0d0ca0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c0d49a0 tx=0x7ffa8402a5f0 comp rx=0 tx=0).stop 2024-04-25T08:34:34.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.603+0000 7ffa9225d640 1 -- 172.21.15.38:0/3715301723 shutdown_connections 2024-04-25T08:34:34.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.603+0000 7ffa9225d640 1 --2- 172.21.15.38:0/3715301723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa8c0d0870 0x7ffa8c0d0ca0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:34.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.603+0000 7ffa9225d640 1 --2- 172.21.15.38:0/3715301723 >> v2:172.21.15.107:3300/0 conn(0x7ffa8c1612c0 0x7ffa8c0b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:34.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.603+0000 7ffa9225d640 1 -- 172.21.15.38:0/3715301723 >> 172.21.15.38:0/3715301723 conn(0x7ffa8c066d10 msgr2=0x7ffa8c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:34.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.603+0000 7ffa9225d640 1 -- 172.21.15.38:0/3715301723 shutdown_connections 2024-04-25T08:34:34.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.603+0000 7ffa9225d640 1 -- 172.21.15.38:0/3715301723 wait complete. 2024-04-25T08:34:34.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.604+0000 7ffa9225d640 1 Processor -- start 2024-04-25T08:34:34.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.604+0000 7ffa9225d640 1 -- start start 2024-04-25T08:34:34.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa9225d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0b6a00 0x7ffa8c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:34.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa9225d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa8c0d0870 0x7ffa8c0c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:34.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa9225d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa8c1612c0 0x7ffa8c0c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:34.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa9225d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0b6e90 con 0x7ffa8c0d0870 2024-04-25T08:34:34.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa9225d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0b7d80 con 0x7ffa8c1612c0 2024-04-25T08:34:34.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa9225d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8c0b82a0 con 0x7ffa8c0b6a00 2024-04-25T08:34:34.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa8affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa8c0d0870 0x7ffa8c0c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:34.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa8affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa8c0d0870 0x7ffa8c0c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53064/0 (socket says 172.21.15.38:53064) 2024-04-25T08:34:34.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa8affd640 1 -- 172.21.15.38:0/364943024 learned_addr learned my addr 172.21.15.38:0/364943024 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:34.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa8bfff640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa8c1612c0 0x7ffa8c0c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa8b7fe640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0b6a00 0x7ffa8c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa8affd640 1 -- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0b6a00 msgr2=0x7ffa8c0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa8affd640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0b6a00 0x7ffa8c0cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa8affd640 1 -- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa8c1612c0 msgr2=0x7ffa8c0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa8affd640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa8c1612c0 0x7ffa8c0c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa8affd640 1 -- 172.21.15.38:0/364943024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c15ad00 con 0x7ffa8c0d0870 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa8bfff640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa8c1612c0 0x7ffa8c0c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.605+0000 7ffa8affd640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa8c0d0870 0x7ffa8c0c9af0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00c4d0 tx=0x7ffa7c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.606+0000 7ffa73fff640 1 -- 172.21.15.38:0/364943024 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c018070 con 0x7ffa8c0d0870 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.606+0000 7ffa73fff640 1 -- 172.21.15.38:0/364943024 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7c007500 con 0x7ffa8c0d0870 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.606+0000 7ffa9225d640 1 -- 172.21.15.38:0/364943024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8c15afe0 con 0x7ffa8c0d0870 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.606+0000 7ffa73fff640 1 -- 172.21.15.38:0/364943024 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c004a20 con 0x7ffa8c0d0870 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.606+0000 7ffa9225d640 1 -- 172.21.15.38:0/364943024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8c0cc110 con 0x7ffa8c0d0870 2024-04-25T08:34:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.607+0000 7ffa9225d640 1 -- 172.21.15.38:0/364943024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa8c0cc310 con 0x7ffa8c0d0870 2024-04-25T08:34:34.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.607+0000 7ffa73fff640 1 -- 172.21.15.38:0/364943024 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa7c01b070 con 0x7ffa8c0d0870 2024-04-25T08:34:34.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.607+0000 7ffa73fff640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffa7806b4d0 0x7ffa7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:34.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.608+0000 7ffa73fff640 1 -- 172.21.15.38:0/364943024 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5778+0+0 (secure 0 0 0) 0x7ffa7c089590 con 0x7ffa8c0d0870 2024-04-25T08:34:34.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.608+0000 7ffa8b7fe640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffa7806b4d0 0x7ffa7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:34.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.608+0000 7ffa8b7fe640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffa7806b4d0 0x7ffa7806d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000b840 tx=0x7ffa80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.612+0000 7ffa73fff640 1 -- 172.21.15.38:0/364943024 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa8c0cc310 con 0x7ffa8c0d0870 2024-04-25T08:34:34.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:34.794+0000 7ffa9225d640 1 -- 172.21.15.38:0/364943024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffa8c068e80 con 0x7ffa8c0d0870 2024-04-25T08:34:35.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.464+0000 7ffa73fff640 1 -- 172.21.15.38:0/364943024 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7ffa7c0583c0 con 0x7ffa8c0d0870 2024-04-25T08:34:35.459 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-04-25T08:34:35.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.467+0000 7ffa9225d640 1 -- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffa7806b4d0 msgr2=0x7ffa7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.467+0000 7ffa9225d640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffa7806b4d0 0x7ffa7806d990 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000b840 tx=0x7ffa80008040 comp rx=0 tx=0).stop 2024-04-25T08:34:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.467+0000 7ffa9225d640 1 -- 172.21.15.38:0/364943024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa8c0d0870 msgr2=0x7ffa8c0c9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.467+0000 7ffa9225d640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa8c0d0870 0x7ffa8c0c9af0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00c4d0 tx=0x7ffa7c00c990 comp rx=0 tx=0).stop 2024-04-25T08:34:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.467+0000 7ffa9225d640 1 -- 172.21.15.38:0/364943024 shutdown_connections 2024-04-25T08:34:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.467+0000 7ffa9225d640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa8c1612c0 0x7ffa8c0c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.467+0000 7ffa9225d640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa8c0d0870 0x7ffa8c0c9af0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.467+0000 7ffa9225d640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffa7806b4d0 0x7ffa7806d990 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.467+0000 7ffa9225d640 1 --2- 172.21.15.38:0/364943024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8c0b6a00 0x7ffa8c0cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.467+0000 7ffa9225d640 1 -- 172.21.15.38:0/364943024 >> 172.21.15.38:0/364943024 conn(0x7ffa8c066d10 msgr2=0x7ffa8c15c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.467+0000 7ffa9225d640 1 -- 172.21.15.38:0/364943024 shutdown_connections 2024-04-25T08:34:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.468+0000 7ffa9225d640 1 -- 172.21.15.38:0/364943024 wait complete. 2024-04-25T08:34:35.479 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:34:35.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.593+0000 7f719355a640 1 Processor -- start 2024-04-25T08:34:35.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.593+0000 7f719355a640 1 -- start start 2024-04-25T08:34:35.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.594+0000 7f719355a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f718c15df30 0x7f718c09a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.594+0000 7f719355a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718c15f300 con 0x7f718c09a180 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.594+0000 7f719355a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718c1627f0 con 0x7f718c162390 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.594+0000 7f719355a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718c15e310 con 0x7f718c15df30 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.594+0000 7f7191af7640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f718c09a180 0x7f718c0cf8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58416/0 (socket says 172.21.15.38:58416) 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.594+0000 7f7191af7640 1 -- 172.21.15.38:0/4247263508 learned_addr learned my addr 172.21.15.38:0/4247263508 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.594+0000 7f71912f6640 1 --2- 172.21.15.38:0/4247263508 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f718c15df30 0x7f718c09a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.594+0000 7f7183fff640 1 -- 172.21.15.38:0/4247263508 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3068635539 0 0) 0x7f718c15f300 con 0x7f718c09a180 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.594+0000 7f7183fff640 1 -- 172.21.15.38:0/4247263508 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7178003520 con 0x7f718c09a180 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.594+0000 7f7183fff640 1 -- 172.21.15.38:0/4247263508 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1340014375 0 0) 0x7f718c1627f0 con 0x7f718c162390 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.594+0000 7f7183fff640 1 -- 172.21.15.38:0/4247263508 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f718c15f300 con 0x7f718c162390 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.595+0000 7f7183fff640 1 -- 172.21.15.38:0/4247263508 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1769296752 0 0) 0x7f7178003520 con 0x7f718c09a180 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.595+0000 7f71912f6640 1 -- 172.21.15.38:0/4247263508 >> v1:172.21.15.107:6789/0 conn(0x7f718c162390 legacy=0x7f718c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:35.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.595+0000 7f71912f6640 1 -- 172.21.15.38:0/4247263508 >> v1:172.21.15.38:6789/0 conn(0x7f718c09a180 legacy=0x7f718c0cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:35.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.595+0000 7f71912f6640 1 -- 172.21.15.38:0/4247263508 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c15e540 con 0x7f718c15df30 2024-04-25T08:34:35.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.595+0000 7f71912f6640 1 --2- 172.21.15.38:0/4247263508 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f718c15df30 0x7f718c09a7b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f71880097d0 tx=0x7f718802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7564365a36f76bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:35.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.596+0000 7f7183fff640 1 -- 172.21.15.38:0/4247263508 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7188037070 con 0x7f718c15df30 2024-04-25T08:34:35.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.596+0000 7f7183fff640 1 -- 172.21.15.38:0/4247263508 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718802ac30 con 0x7f718c15df30 2024-04-25T08:34:35.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.596+0000 7f7183fff640 1 -- 172.21.15.38:0/4247263508 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71840019f0 con 0x7f718c15df30 2024-04-25T08:34:35.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.596+0000 7f719355a640 1 -- 172.21.15.38:0/4247263508 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f718c15df30 msgr2=0x7f718c09a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:35.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.596+0000 7f719355a640 1 --2- 172.21.15.38:0/4247263508 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f718c15df30 0x7f718c09a7b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f71880097d0 tx=0x7f718802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.596+0000 7f719355a640 1 -- 172.21.15.38:0/4247263508 shutdown_connections 2024-04-25T08:34:35.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.596+0000 7f719355a640 1 --2- 172.21.15.38:0/4247263508 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f718c15df30 0x7f718c09a7b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.596+0000 7f719355a640 1 -- 172.21.15.38:0/4247263508 >> 172.21.15.38:0/4247263508 conn(0x7f718c0b3110 msgr2=0x7f718c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:35.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.596+0000 7f719355a640 1 -- 172.21.15.38:0/4247263508 shutdown_connections 2024-04-25T08:34:35.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.596+0000 7f719355a640 1 -- 172.21.15.38:0/4247263508 wait complete. 2024-04-25T08:34:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.597+0000 7f719355a640 1 Processor -- start 2024-04-25T08:34:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.597+0000 7f719355a640 1 -- start start 2024-04-25T08:34:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.598+0000 7f719355a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f718c09a180 0x7f718c0c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.598+0000 7f719355a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f718c15df30 0x7f718c0b5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.598+0000 7f719355a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f718c162390 0x7f718c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.598+0000 7f719355a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c15ee70 con 0x7f718c15df30 2024-04-25T08:34:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.598+0000 7f719355a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c09aeb0 con 0x7f718c162390 2024-04-25T08:34:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.598+0000 7f719355a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718c15eb00 con 0x7f718c09a180 2024-04-25T08:34:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.598+0000 7f7190af5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f718c15df30 0x7f718c0b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.598+0000 7f7190af5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f718c15df30 0x7f718c0b5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53070/0 (socket says 172.21.15.38:53070) 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.598+0000 7f7190af5640 1 -- 172.21.15.38:0/2882531273 learned_addr learned my addr 172.21.15.38:0/2882531273 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.598+0000 7f7191af7640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f718c162390 0x7f718c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.598+0000 7f71912f6640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f718c09a180 0x7f718c0c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.599+0000 7f7190af5640 1 -- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f718c09a180 msgr2=0x7f718c0c7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.599+0000 7f7190af5640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f718c09a180 0x7f718c0c7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.599+0000 7f7190af5640 1 -- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f718c162390 msgr2=0x7f718c0cd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.599+0000 7f7190af5640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f718c162390 0x7f718c0cd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.599+0000 7f7190af5640 1 -- 172.21.15.38:0/2882531273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c0b42e0 con 0x7f718c15df30 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.599+0000 7f7190af5640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f718c15df30 0x7f718c0b5310 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f717c00c490 tx=0x7f717c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.599+0000 7f7181ffb640 1 -- 172.21.15.38:0/2882531273 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c0052d0 con 0x7f718c15df30 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.599+0000 7f7181ffb640 1 -- 172.21.15.38:0/2882531273 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c005470 con 0x7f718c15df30 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.599+0000 7f7181ffb640 1 -- 172.21.15.38:0/2882531273 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c012850 con 0x7f718c15df30 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.599+0000 7f719355a640 1 -- 172.21.15.38:0/2882531273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718c0b4550 con 0x7f718c15df30 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.599+0000 7f719355a640 1 -- 172.21.15.38:0/2882531273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718c0b4b10 con 0x7f718c15df30 2024-04-25T08:34:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.600+0000 7f719355a640 1 -- 172.21.15.38:0/2882531273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718c068e80 con 0x7f718c15df30 2024-04-25T08:34:35.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.600+0000 7f7181ffb640 1 -- 172.21.15.38:0/2882531273 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f717c0129f0 con 0x7f718c15df30 2024-04-25T08:34:35.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.601+0000 7f7181ffb640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f717006b4d0 0x7f717006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:35.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.601+0000 7f7181ffb640 1 -- 172.21.15.38:0/2882531273 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f717c089d40 con 0x7f718c15df30 2024-04-25T08:34:35.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.601+0000 7f71912f6640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f717006b4d0 0x7f717006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:35.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.602+0000 7f71912f6640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f717006b4d0 0x7f717006d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7188034000 tx=0x7f718802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:35.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.605+0000 7f7181ffb640 1 -- 172.21.15.38:0/2882531273 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f717c058d40 con 0x7f718c15df30 2024-04-25T08:34:35.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.792+0000 7f719355a640 1 -- 172.21.15.38:0/2882531273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f718c0b4020 con 0x7f718c15df30 2024-04-25T08:34:35.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.795+0000 7f7181ffb640 1 -- 172.21.15.38:0/2882531273 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f717c058640 con 0x7f718c15df30 2024-04-25T08:34:35.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.797+0000 7f719355a640 1 -- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f717006b4d0 msgr2=0x7f717006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.797+0000 7f719355a640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f717006b4d0 0x7f717006d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7188034000 tx=0x7f718802e000 comp rx=0 tx=0).stop 2024-04-25T08:34:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.797+0000 7f719355a640 1 -- 172.21.15.38:0/2882531273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f718c15df30 msgr2=0x7f718c0b5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.797+0000 7f719355a640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f718c15df30 0x7f718c0b5310 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f717c00c490 tx=0x7f717c00c950 comp rx=0 tx=0).stop 2024-04-25T08:34:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.797+0000 7f719355a640 1 -- 172.21.15.38:0/2882531273 shutdown_connections 2024-04-25T08:34:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.797+0000 7f719355a640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f718c162390 0x7f718c0cd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.797+0000 7f719355a640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f718c15df30 0x7f718c0b5310 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.797+0000 7f719355a640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f717006b4d0 0x7f717006d990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.797+0000 7f719355a640 1 --2- 172.21.15.38:0/2882531273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f718c09a180 0x7f718c0c7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.797+0000 7f719355a640 1 -- 172.21.15.38:0/2882531273 >> 172.21.15.38:0/2882531273 conn(0x7f718c0b3110 msgr2=0x7f718c15b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.797+0000 7f719355a640 1 -- 172.21.15.38:0/2882531273 shutdown_connections 2024-04-25T08:34:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.797+0000 7f719355a640 1 -- 172.21.15.38:0/2882531273 wait complete. 2024-04-25T08:34:35.803 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T08:34:35.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.909+0000 7f1ac4aac640 1 Processor -- start 2024-04-25T08:34:35.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.910+0000 7f1ac4aac640 1 -- start start 2024-04-25T08:34:35.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.910+0000 7f1ac4aac640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1ac015ece0 0x7f1ac00ae200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:35.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.910+0000 7f1ac4aac640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1ac006bbf0 0x7f1ac006c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:35.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.910+0000 7f1ac4aac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ac00b2df0 0x7f1ac00b3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:35.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.910+0000 7f1ac4aac640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac01584d0 con 0x7f1ac006bbf0 2024-04-25T08:34:35.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.910+0000 7f1ac4aac640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac00add70 con 0x7f1ac015ece0 2024-04-25T08:34:35.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.910+0000 7f1ac4aac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00b3b70 con 0x7f1ac00b2df0 2024-04-25T08:34:35.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.911+0000 7f1abdd9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1ac006bbf0 0x7f1ac006c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:35.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.911+0000 7f1abdd9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1ac006bbf0 0x7f1ac006c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53078/0 (socket says 172.21.15.38:53078) 2024-04-25T08:34:35.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.911+0000 7f1abdd9b640 1 -- 172.21.15.38:0/1755045009 learned_addr learned my addr 172.21.15.38:0/1755045009 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:35.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.911+0000 7f1abdd9b640 1 -- 172.21.15.38:0/1755045009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ac00b2df0 msgr2=0x7f1ac00b3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:35.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.911+0000 7f1abdd9b640 1 --2- 172.21.15.38:0/1755045009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ac00b2df0 0x7f1ac00b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.911+0000 7f1abdd9b640 1 -- 172.21.15.38:0/1755045009 >> v2:172.21.15.107:3300/0 conn(0x7f1ac015ece0 msgr2=0x7f1ac00ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:35.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.911+0000 7f1abdd9b640 1 --2- 172.21.15.38:0/1755045009 >> v2:172.21.15.107:3300/0 conn(0x7f1ac015ece0 0x7f1ac00ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.911+0000 7f1abdd9b640 1 -- 172.21.15.38:0/1755045009 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac00ad720 con 0x7f1ac006bbf0 2024-04-25T08:34:35.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.912+0000 7f1abdd9b640 1 --2- 172.21.15.38:0/1755045009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ac006bbf0 0x7f1ac006c070 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00715d0 tx=0x7f1aa802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287cbe23095b8bf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:35.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.912+0000 7f1abd59a640 1 -- 172.21.15.38:0/1755045009 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8037070 con 0x7f1ac006bbf0 2024-04-25T08:34:35.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.912+0000 7f1abd59a640 1 -- 172.21.15.38:0/1755045009 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa8032440 con 0x7f1ac006bbf0 2024-04-25T08:34:35.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.912+0000 7f1abd59a640 1 -- 172.21.15.38:0/1755045009 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8032810 con 0x7f1ac006bbf0 2024-04-25T08:34:35.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.912+0000 7f1ac4aac640 1 -- 172.21.15.38:0/1755045009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ac006bbf0 msgr2=0x7f1ac006c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:35.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.912+0000 7f1ac4aac640 1 --2- 172.21.15.38:0/1755045009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ac006bbf0 0x7f1ac006c070 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f1ac00715d0 tx=0x7f1aa802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.912+0000 7f1ac4aac640 1 -- 172.21.15.38:0/1755045009 shutdown_connections 2024-04-25T08:34:35.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.912+0000 7f1ac4aac640 1 --2- 172.21.15.38:0/1755045009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ac00b2df0 0x7f1ac00b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.912+0000 7f1ac4aac640 1 --2- 172.21.15.38:0/1755045009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ac006bbf0 0x7f1ac006c070 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.912+0000 7f1ac4aac640 1 --2- 172.21.15.38:0/1755045009 >> v2:172.21.15.107:3300/0 conn(0x7f1ac015ece0 0x7f1ac00ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.912+0000 7f1ac4aac640 1 -- 172.21.15.38:0/1755045009 >> 172.21.15.38:0/1755045009 conn(0x7f1ac0071920 msgr2=0x7f1ac0071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:35.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.913+0000 7f1ac4aac640 1 -- 172.21.15.38:0/1755045009 shutdown_connections 2024-04-25T08:34:35.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.913+0000 7f1ac4aac640 1 -- 172.21.15.38:0/1755045009 wait complete. 2024-04-25T08:34:35.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.914+0000 7f1ac4aac640 1 Processor -- start 2024-04-25T08:34:35.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.914+0000 7f1ac4aac640 1 -- start start 2024-04-25T08:34:35.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1ac4aac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ac006bbf0 0x7f1ac0141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:35.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1ac4aac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ac00b2df0 0x7f1ac0144930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:35.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1ac4aac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac015ece0 0x7f1ac0144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:35.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1ac4aac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00b3900 con 0x7f1ac006bbf0 2024-04-25T08:34:35.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1ac4aac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00b3780 con 0x7f1ac015ece0 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1ac4aac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac015f160 con 0x7f1ac00b2df0 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1abe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ac006bbf0 0x7f1ac0141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1abe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ac006bbf0 0x7f1ac0141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53092/0 (socket says 172.21.15.38:53092) 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1abe59c640 1 -- 172.21.15.38:0/2120505042 learned_addr learned my addr 172.21.15.38:0/2120505042 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1abdd9b640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ac00b2df0 0x7f1ac0144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1abed9d640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac015ece0 0x7f1ac0144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1abe59c640 1 -- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ac00b2df0 msgr2=0x7f1ac0144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1abe59c640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ac00b2df0 0x7f1ac0144930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1abe59c640 1 -- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac015ece0 msgr2=0x7f1ac0144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1abe59c640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac015ece0 0x7f1ac0144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1abe59c640 1 -- 172.21.15.38:0/2120505042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac0159fd0 con 0x7f1ac006bbf0 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1abe59c640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ac006bbf0 0x7f1ac0141f20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800b440 tx=0x7f1ab800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:35.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1ab6ffd640 1 -- 172.21.15.38:0/2120505042 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8015020 con 0x7f1ac006bbf0 2024-04-25T08:34:35.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1ab6ffd640 1 -- 172.21.15.38:0/2120505042 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab8011890 con 0x7f1ac006bbf0 2024-04-25T08:34:35.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1ac4aac640 1 -- 172.21.15.38:0/2120505042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac0159450 con 0x7f1ac006bbf0 2024-04-25T08:34:35.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.915+0000 7f1ab6ffd640 1 -- 172.21.15.38:0/2120505042 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8011c80 con 0x7f1ac006bbf0 2024-04-25T08:34:35.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.916+0000 7f1ac4aac640 1 -- 172.21.15.38:0/2120505042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac0159a10 con 0x7f1ac006bbf0 2024-04-25T08:34:35.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.916+0000 7f1ac4aac640 1 -- 172.21.15.38:0/2120505042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac0157f60 con 0x7f1ac006bbf0 2024-04-25T08:34:35.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.917+0000 7f1ab6ffd640 1 -- 172.21.15.38:0/2120505042 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ab8018020 con 0x7f1ac006bbf0 2024-04-25T08:34:35.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.917+0000 7f1ab6ffd640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1aa006b4d0 0x7f1aa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:35.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.917+0000 7f1ab6ffd640 1 -- 172.21.15.38:0/2120505042 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1ab808a150 con 0x7f1ac006bbf0 2024-04-25T08:34:35.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.917+0000 7f1abdd9b640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1aa006b4d0 0x7f1aa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:35.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.918+0000 7f1abdd9b640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1aa006b4d0 0x7f1aa006d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8034000 tx=0x7f1aa802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:35.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:35.921+0000 7f1ab6ffd640 1 -- 172.21.15.38:0/2120505042 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ab801f020 con 0x7f1ac006bbf0 2024-04-25T08:34:36.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.174+0000 7f1ac4aac640 1 -- 172.21.15.38:0/2120505042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f1ac00b0520 con 0x7f1ac006bbf0 2024-04-25T08:34:36.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.465+0000 7f1ab6ffd640 1 -- 172.21.15.38:0/2120505042 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v158) v1 ==== 148+0+0 (secure 0 0 0) 0x7f1ab800d040 con 0x7f1ac006bbf0 2024-04-25T08:34:36.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.468+0000 7f1ac4aac640 1 -- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1aa006b4d0 msgr2=0x7f1aa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:36.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.468+0000 7f1ac4aac640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1aa006b4d0 0x7f1aa006d990 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8034000 tx=0x7f1aa802e000 comp rx=0 tx=0).stop 2024-04-25T08:34:36.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.468+0000 7f1ac4aac640 1 -- 172.21.15.38:0/2120505042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ac006bbf0 msgr2=0x7f1ac0141f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:36.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.468+0000 7f1ac4aac640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ac006bbf0 0x7f1ac0141f20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800b440 tx=0x7f1ab800b900 comp rx=0 tx=0).stop 2024-04-25T08:34:36.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.468+0000 7f1ac4aac640 1 -- 172.21.15.38:0/2120505042 shutdown_connections 2024-04-25T08:34:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.468+0000 7f1ac4aac640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ac015ece0 0x7f1ac0144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.468+0000 7f1ac4aac640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1aa006b4d0 0x7f1aa006d990 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.468+0000 7f1ac4aac640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1ac00b2df0 0x7f1ac0144930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.468+0000 7f1ac4aac640 1 --2- 172.21.15.38:0/2120505042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1ac006bbf0 0x7f1ac0141f20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.468+0000 7f1ac4aac640 1 -- 172.21.15.38:0/2120505042 >> 172.21.15.38:0/2120505042 conn(0x7f1ac0071920 msgr2=0x7f1ac015d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.468+0000 7f1ac4aac640 1 -- 172.21.15.38:0/2120505042 shutdown_connections 2024-04-25T08:34:36.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.468+0000 7f1ac4aac640 1 -- 172.21.15.38:0/2120505042 wait complete. 2024-04-25T08:34:36.475 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-25T08:34:36.476 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T08:34:36.476 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:34:36.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.592+0000 7fdd902db640 1 Processor -- start 2024-04-25T08:34:36.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.592+0000 7fdd902db640 1 -- start start 2024-04-25T08:34:36.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.593+0000 7fdd902db640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdd8815df50 0x7fdd880ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:36.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.593+0000 7fdd902db640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd880ce2e0 0x7fdd880d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:36.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.593+0000 7fdd902db640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd880ceff0 con 0x7fdd8806c110 2024-04-25T08:34:36.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.593+0000 7fdd902db640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd880cf2d0 con 0x7fdd8815df50 2024-04-25T08:34:36.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.593+0000 7fdd902db640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880d1f70 con 0x7fdd880ce2e0 2024-04-25T08:34:36.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.593+0000 7fdd8e878640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd880ce2e0 0x7fdd880d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:36.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.593+0000 7fdd8e878640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd880ce2e0 0x7fdd880d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:56802/0 (socket says 172.21.15.38:56802) 2024-04-25T08:34:36.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.593+0000 7fdd8e878640 1 -- 172.21.15.38:0/1671465583 learned_addr learned my addr 172.21.15.38:0/1671465583 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:36.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.593+0000 7fdd8d876640 1 --2- 172.21.15.38:0/1671465583 >> v2:172.21.15.107:3300/0 conn(0x7fdd8815df50 0x7fdd880ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.593+0000 7fdd8d075640 1 -- 172.21.15.38:0/1671465583 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539640104 0 0) 0x7fdd880ceff0 con 0x7fdd8806c110 2024-04-25T08:34:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.593+0000 7fdd8d075640 1 -- 172.21.15.38:0/1671465583 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd74003520 con 0x7fdd8806c110 2024-04-25T08:34:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.593+0000 7fdd8e878640 1 -- 172.21.15.38:0/1671465583 >> v2:172.21.15.107:3300/0 conn(0x7fdd8815df50 msgr2=0x7fdd880ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.594+0000 7fdd8e878640 1 --2- 172.21.15.38:0/1671465583 >> v2:172.21.15.107:3300/0 conn(0x7fdd8815df50 0x7fdd880ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.594+0000 7fdd8e878640 1 -- 172.21.15.38:0/1671465583 >> v1:172.21.15.107:6789/0 conn(0x7fdd8806c110 legacy=0x7fdd8806c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.594+0000 7fdd8e878640 1 -- 172.21.15.38:0/1671465583 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd880d2250 con 0x7fdd880ce2e0 2024-04-25T08:34:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.594+0000 7fdd8d876640 1 --2- 172.21.15.38:0/1671465583 >> v2:172.21.15.107:3300/0 conn(0x7fdd8815df50 0x7fdd880ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.594+0000 7fdd8e878640 1 --2- 172.21.15.38:0/1671465583 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd880ce2e0 0x7fdd880d2a50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009e50 tx=0x7fdd8002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86cf6b371bd478b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.594+0000 7fdd8d075640 1 -- 172.21.15.38:0/1671465583 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80036070 con 0x7fdd880ce2e0 2024-04-25T08:34:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.595+0000 7fdd8d075640 1 -- 172.21.15.38:0/1671465583 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd80031600 con 0x7fdd880ce2e0 2024-04-25T08:34:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.595+0000 7fdd8d075640 1 -- 172.21.15.38:0/1671465583 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd800319b0 con 0x7fdd880ce2e0 2024-04-25T08:34:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.595+0000 7fdd902db640 1 -- 172.21.15.38:0/1671465583 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd880ce2e0 msgr2=0x7fdd880d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.595+0000 7fdd902db640 1 --2- 172.21.15.38:0/1671465583 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd880ce2e0 0x7fdd880d2a50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009e50 tx=0x7fdd8002a710 comp rx=0 tx=0).stop 2024-04-25T08:34:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.595+0000 7fdd902db640 1 -- 172.21.15.38:0/1671465583 shutdown_connections 2024-04-25T08:34:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.595+0000 7fdd902db640 1 --2- 172.21.15.38:0/1671465583 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd880ce2e0 0x7fdd880d2a50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.595+0000 7fdd902db640 1 --2- 172.21.15.38:0/1671465583 >> v2:172.21.15.107:3300/0 conn(0x7fdd8815df50 0x7fdd880ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.595+0000 7fdd902db640 1 -- 172.21.15.38:0/1671465583 >> 172.21.15.38:0/1671465583 conn(0x7fdd880d3fb0 msgr2=0x7fdd880d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.595+0000 7fdd902db640 1 -- 172.21.15.38:0/1671465583 shutdown_connections 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.595+0000 7fdd902db640 1 -- 172.21.15.38:0/1671465583 wait complete. 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.596+0000 7fdd902db640 1 Processor -- start 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.596+0000 7fdd902db640 1 -- start start 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd902db640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd8806c110 0x7fdd8814d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd902db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd880ce2e0 0x7fdd8814ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd902db640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd8815df50 0x7fdd880b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd902db640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880d2f90 con 0x7fdd8806c110 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd902db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880cece0 con 0x7fdd880ce2e0 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd902db640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880d3110 con 0x7fdd8815df50 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd8e077640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd8806c110 0x7fdd8814d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd8e077640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd8806c110 0x7fdd8814d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53108/0 (socket says 172.21.15.38:53108) 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd8e077640 1 -- 172.21.15.38:0/244401014 learned_addr learned my addr 172.21.15.38:0/244401014 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:36.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd8e878640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd8815df50 0x7fdd880b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd8d876640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd880ce2e0 0x7fdd8814ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd8e077640 1 -- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd8815df50 msgr2=0x7fdd880b7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd8e077640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd8815df50 0x7fdd880b7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd8e077640 1 -- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd880ce2e0 msgr2=0x7fdd8814ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd8e077640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd880ce2e0 0x7fdd8814ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.597+0000 7fdd8e077640 1 -- 172.21.15.38:0/244401014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8814d3d0 con 0x7fdd8806c110 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.598+0000 7fdd8e077640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd8806c110 0x7fdd8814d920 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400c530 tx=0x7fdd8400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.598+0000 7fdd7effd640 1 -- 172.21.15.38:0/244401014 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd840052d0 con 0x7fdd8806c110 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.598+0000 7fdd7effd640 1 -- 172.21.15.38:0/244401014 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd84005490 con 0x7fdd8806c110 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.598+0000 7fdd7effd640 1 -- 172.21.15.38:0/244401014 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84012870 con 0x7fdd8806c110 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.598+0000 7fdd902db640 1 -- 172.21.15.38:0/244401014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8815f060 con 0x7fdd8806c110 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.598+0000 7fdd902db640 1 -- 172.21.15.38:0/244401014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8815ffe0 con 0x7fdd8806c110 2024-04-25T08:34:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.599+0000 7fdd902db640 1 -- 172.21.15.38:0/244401014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd88068e80 con 0x7fdd8806c110 2024-04-25T08:34:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.599+0000 7fdd7effd640 1 -- 172.21.15.38:0/244401014 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd84012a10 con 0x7fdd8806c110 2024-04-25T08:34:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.600+0000 7fdd7effd640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6c06b4d0 0x7fdd6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.600+0000 7fdd7effd640 1 -- 172.21.15.38:0/244401014 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fdd84089c10 con 0x7fdd8806c110 2024-04-25T08:34:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.600+0000 7fdd8d876640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6c06b4d0 0x7fdd6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.600+0000 7fdd8d876640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6c06b4d0 0x7fdd6c06d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009560 tx=0x7fdd78009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:36.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.604+0000 7fdd7effd640 1 -- 172.21.15.38:0/244401014 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd84058530 con 0x7fdd8806c110 2024-04-25T08:34:36.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.784+0000 7fdd902db640 1 -- 172.21.15.38:0/244401014 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd880ca440 con 0x7fdd8806c110 2024-04-25T08:34:36.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.785+0000 7fdd7effd640 1 -- 172.21.15.38:0/244401014 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+13279 (secure 0 0 0) 0x7fdd8405c5d0 con 0x7fdd8806c110 2024-04-25T08:34:36.780 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:36.780 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":45,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:34:35.463033+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/234860294":"2024-04-26T08:34:32.456301+0000","172.21.15.38:6832/234860294":"2024-04-26T08:34:32.456301+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:34:36.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.787+0000 7fdd902db640 1 -- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6c06b4d0 msgr2=0x7fdd6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:36.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.787+0000 7fdd902db640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6c06b4d0 0x7fdd6c06d990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdd78009560 tx=0x7fdd78009190 comp rx=0 tx=0).stop 2024-04-25T08:34:36.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.787+0000 7fdd902db640 1 -- 172.21.15.38:0/244401014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd8806c110 msgr2=0x7fdd8814d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:36.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.787+0000 7fdd902db640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd8806c110 0x7fdd8814d920 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400c530 tx=0x7fdd8400c9f0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.787+0000 7fdd902db640 1 -- 172.21.15.38:0/244401014 shutdown_connections 2024-04-25T08:34:36.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.787+0000 7fdd902db640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6c06b4d0 0x7fdd6c06d990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.787+0000 7fdd902db640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd8815df50 0x7fdd880b7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.787+0000 7fdd902db640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd880ce2e0 0x7fdd8814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.787+0000 7fdd902db640 1 --2- 172.21.15.38:0/244401014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd8806c110 0x7fdd8814d920 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.787+0000 7fdd902db640 1 -- 172.21.15.38:0/244401014 >> 172.21.15.38:0/244401014 conn(0x7fdd880d3fb0 msgr2=0x7fdd880b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:36.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.787+0000 7fdd902db640 1 -- 172.21.15.38:0/244401014 shutdown_connections 2024-04-25T08:34:36.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.787+0000 7fdd902db640 1 -- 172.21.15.38:0/244401014 wait complete. 2024-04-25T08:34:36.793 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:34:36.794 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:34:36.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.902+0000 7fac7b2d8640 1 Processor -- start 2024-04-25T08:34:36.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.903+0000 7fac7b2d8640 1 -- start start 2024-04-25T08:34:36.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.903+0000 7fac7b2d8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac740d33d0 0x7fac740d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:36.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.903+0000 7fac7b2d8640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fac740d1e90 0x7fac740b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:36.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.903+0000 7fac7b2d8640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac740d2330 con 0x7fac740d0320 2024-04-25T08:34:36.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.903+0000 7fac7b2d8640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac740d2610 con 0x7fac740d1e90 2024-04-25T08:34:36.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.903+0000 7fac7b2d8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac740d28f0 con 0x7fac740d33d0 2024-04-25T08:34:36.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.904+0000 7fac79074640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fac740d0320 0x7fac740d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58418/0 (socket says 172.21.15.38:58418) 2024-04-25T08:34:36.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.904+0000 7fac79074640 1 -- 172.21.15.38:0/2374373259 learned_addr learned my addr 172.21.15.38:0/2374373259 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:36.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.904+0000 7fac78873640 1 --2- 172.21.15.38:0/2374373259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac740d33d0 0x7fac740d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:36.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.904+0000 7fac79875640 1 --2- 172.21.15.38:0/2374373259 >> v2:172.21.15.107:3300/0 conn(0x7fac740d1e90 0x7fac740b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:36.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.904+0000 7fac6bfff640 1 -- 172.21.15.38:0/2374373259 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447847130 0 0) 0x7fac740d2330 con 0x7fac740d0320 2024-04-25T08:34:36.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.904+0000 7fac6bfff640 1 -- 172.21.15.38:0/2374373259 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac60003520 con 0x7fac740d0320 2024-04-25T08:34:36.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.904+0000 7fac6bfff640 1 -- 172.21.15.38:0/2374373259 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1218855630 0 0) 0x7fac60003520 con 0x7fac740d0320 2024-04-25T08:34:36.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.904+0000 7fac78873640 1 -- 172.21.15.38:0/2374373259 >> v2:172.21.15.107:3300/0 conn(0x7fac740d1e90 msgr2=0x7fac740b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:36.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.904+0000 7fac78873640 1 --2- 172.21.15.38:0/2374373259 >> v2:172.21.15.107:3300/0 conn(0x7fac740d1e90 0x7fac740b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.904+0000 7fac78873640 1 -- 172.21.15.38:0/2374373259 >> v1:172.21.15.38:6789/0 conn(0x7fac740d0320 legacy=0x7fac740d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:36.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.904+0000 7fac78873640 1 -- 172.21.15.38:0/2374373259 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac740d2bd0 con 0x7fac740d33d0 2024-04-25T08:34:36.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.905+0000 7fac78873640 1 --2- 172.21.15.38:0/2374373259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac740d33d0 0x7fac740d1950 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fac640097a0 tx=0x7fac6402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3d36f154cf9346d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:36.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.905+0000 7fac6bfff640 1 -- 172.21.15.38:0/2374373259 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64037070 con 0x7fac740d33d0 2024-04-25T08:34:36.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.905+0000 7fac6bfff640 1 -- 172.21.15.38:0/2374373259 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6402ac30 con 0x7fac740d33d0 2024-04-25T08:34:36.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.905+0000 7fac6bfff640 1 -- 172.21.15.38:0/2374373259 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac70001670 con 0x7fac740d33d0 2024-04-25T08:34:36.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.905+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2374373259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac740d33d0 msgr2=0x7fac740d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:36.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.905+0000 7fac7b2d8640 1 --2- 172.21.15.38:0/2374373259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac740d33d0 0x7fac740d1950 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fac640097a0 tx=0x7fac6402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.906+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2374373259 shutdown_connections 2024-04-25T08:34:36.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.906+0000 7fac7b2d8640 1 --2- 172.21.15.38:0/2374373259 >> v2:172.21.15.107:3300/0 conn(0x7fac740d1e90 0x7fac740b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.906+0000 7fac7b2d8640 1 --2- 172.21.15.38:0/2374373259 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac740d33d0 0x7fac740d1950 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.906+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2374373259 >> 172.21.15.38:0/2374373259 conn(0x7fac740668f0 msgr2=0x7fac74066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.906+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2374373259 shutdown_connections 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.906+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2374373259 wait complete. 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.907+0000 7fac7b2d8640 1 Processor -- start 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.907+0000 7fac7b2d8640 1 -- start start 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.907+0000 7fac7b2d8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac740d0320 0x7fac74159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.907+0000 7fac7b2d8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac740d1e90 0x7fac741e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.907+0000 7fac7b2d8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac740d33d0 0x7fac740aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac7b2d8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac74161390 con 0x7fac740d0320 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac7b2d8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac74161210 con 0x7fac740d1e90 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac7b2d8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac740b9720 con 0x7fac740d33d0 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac79074640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac740d0320 0x7fac74159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac79074640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac740d0320 0x7fac74159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53118/0 (socket says 172.21.15.38:53118) 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac79074640 1 -- 172.21.15.38:0/2241728650 learned_addr learned my addr 172.21.15.38:0/2241728650 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:36.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac79875640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac740d33d0 0x7fac740aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac78873640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac740d1e90 0x7fac741e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac79074640 1 -- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac740d33d0 msgr2=0x7fac740aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac79074640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac740d33d0 0x7fac740aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac79074640 1 -- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac740d1e90 msgr2=0x7fac741e72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac79074640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac740d1e90 0x7fac741e72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac79074640 1 -- 172.21.15.38:0/2241728650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7415a000 con 0x7fac740d0320 2024-04-25T08:34:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac79074640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac740d0320 0x7fac74159ac0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fac7000c780 tx=0x7fac7000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac69ffb640 1 -- 172.21.15.38:0/2241728650 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac70017070 con 0x7fac740d0320 2024-04-25T08:34:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac69ffb640 1 -- 172.21.15.38:0/2241728650 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac70005ad0 con 0x7fac740d0320 2024-04-25T08:34:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac69ffb640 1 -- 172.21.15.38:0/2241728650 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac70007070 con 0x7fac740d0320 2024-04-25T08:34:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.908+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2241728650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac741e7c40 con 0x7fac740d0320 2024-04-25T08:34:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.909+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2241728650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac740b4d10 con 0x7fac740d0320 2024-04-25T08:34:36.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.910+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2241728650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac74068e80 con 0x7fac740d0320 2024-04-25T08:34:36.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.910+0000 7fac69ffb640 1 -- 172.21.15.38:0/2241728650 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac7001a070 con 0x7fac740d0320 2024-04-25T08:34:36.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.910+0000 7fac69ffb640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fac5806b4d0 0x7fac5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:36.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.910+0000 7fac69ffb640 1 -- 172.21.15.38:0/2241728650 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fac700895f0 con 0x7fac740d0320 2024-04-25T08:34:36.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.911+0000 7fac78873640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fac5806b4d0 0x7fac5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:36.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.911+0000 7fac78873640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fac5806b4d0 0x7fac5806d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fac64034000 tx=0x7fac6402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:36.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:36.914+0000 7fac69ffb640 1 -- 172.21.15.38:0/2241728650 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac70057ef0 con 0x7fac740d0320 2024-04-25T08:34:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.093+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2241728650 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac740d1ad0 con 0x7fac740d0320 2024-04-25T08:34:37.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.093+0000 7fac69ffb640 1 -- 172.21.15.38:0/2241728650 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac7005bf90 con 0x7fac740d0320 2024-04-25T08:34:37.089 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:34:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.095+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fac5806b4d0 msgr2=0x7fac5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.095+0000 7fac7b2d8640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fac5806b4d0 0x7fac5806d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fac64034000 tx=0x7fac6402e000 comp rx=0 tx=0).stop 2024-04-25T08:34:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.095+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2241728650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac740d0320 msgr2=0x7fac74159ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.095+0000 7fac7b2d8640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac740d0320 0x7fac74159ac0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fac7000c780 tx=0x7fac7000cc40 comp rx=0 tx=0).stop 2024-04-25T08:34:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.095+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2241728650 shutdown_connections 2024-04-25T08:34:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.095+0000 7fac7b2d8640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fac5806b4d0 0x7fac5806d990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.095+0000 7fac7b2d8640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac740d33d0 0x7fac740aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.095+0000 7fac7b2d8640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac740d1e90 0x7fac741e72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.095+0000 7fac7b2d8640 1 --2- 172.21.15.38:0/2241728650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac740d0320 0x7fac74159ac0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.095+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2241728650 >> 172.21.15.38:0/2241728650 conn(0x7fac740668f0 msgr2=0x7fac740b7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.095+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2241728650 shutdown_connections 2024-04-25T08:34:37.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.095+0000 7fac7b2d8640 1 -- 172.21.15.38:0/2241728650 wait complete. 2024-04-25T08:34:37.101 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T08:34:37.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.207+0000 7f3811f96640 1 Processor -- start 2024-04-25T08:34:37.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.207+0000 7f3811f96640 1 -- start start 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.208+0000 7f3811f96640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f380c06c110 0x7f380c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.208+0000 7f3811f96640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f380c15df50 0x7f380c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.208+0000 7f3811f96640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f380c0ce2e0 0x7f380c0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.208+0000 7f3811f96640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380c0cf2d0 con 0x7f380c15df50 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.208+0000 7f3811f96640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380c06ca30 con 0x7f380c06c110 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.208+0000 7f3811f96640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380c0ceff0 con 0x7f380c0ce2e0 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.208+0000 7f380affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f380c15df50 0x7f380c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.208+0000 7f380affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f380c15df50 0x7f380c0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53126/0 (socket says 172.21.15.38:53126) 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.208+0000 7f380affd640 1 -- 172.21.15.38:0/867337706 learned_addr learned my addr 172.21.15.38:0/867337706 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.208+0000 7f380bfff640 1 --2- 172.21.15.38:0/867337706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f380c0ce2e0 0x7f380c0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.208+0000 7f380b7fe640 1 --2- 172.21.15.38:0/867337706 >> v2:172.21.15.107:3300/0 conn(0x7f380c06c110 0x7f380c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.209+0000 7f380affd640 1 -- 172.21.15.38:0/867337706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f380c0ce2e0 msgr2=0x7f380c0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:37.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.209+0000 7f380affd640 1 --2- 172.21.15.38:0/867337706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f380c0ce2e0 0x7f380c0d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:37.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.209+0000 7f380affd640 1 -- 172.21.15.38:0/867337706 >> v2:172.21.15.107:3300/0 conn(0x7f380c06c110 msgr2=0x7f380c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:37.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.209+0000 7f380affd640 1 --2- 172.21.15.38:0/867337706 >> v2:172.21.15.107:3300/0 conn(0x7f380c06c110 0x7f380c06c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:37.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.209+0000 7f380affd640 1 -- 172.21.15.38:0/867337706 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c0d1f70 con 0x7f380c15df50 2024-04-25T08:34:37.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.209+0000 7f380affd640 1 --2- 172.21.15.38:0/867337706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f380c15df50 0x7f380c0ce7a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f380c08c550 tx=0x7f380002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ac73f44d320846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:37.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.209+0000 7f380a7fc640 1 -- 172.21.15.38:0/867337706 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800037070 con 0x7f380c15df50 2024-04-25T08:34:37.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.209+0000 7f380a7fc640 1 -- 172.21.15.38:0/867337706 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3800032440 con 0x7f380c15df50 2024-04-25T08:34:37.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.209+0000 7f380a7fc640 1 -- 172.21.15.38:0/867337706 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800032810 con 0x7f380c15df50 2024-04-25T08:34:37.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.210+0000 7f3811f96640 1 -- 172.21.15.38:0/867337706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f380c15df50 msgr2=0x7f380c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:37.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.210+0000 7f3811f96640 1 --2- 172.21.15.38:0/867337706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f380c15df50 0x7f380c0ce7a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f380c08c550 tx=0x7f380002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:34:37.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.210+0000 7f3811f96640 1 -- 172.21.15.38:0/867337706 shutdown_connections 2024-04-25T08:34:37.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.210+0000 7f3811f96640 1 --2- 172.21.15.38:0/867337706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f380c0ce2e0 0x7f380c0d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:37.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.210+0000 7f3811f96640 1 --2- 172.21.15.38:0/867337706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f380c15df50 0x7f380c0ce7a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:37.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.210+0000 7f3811f96640 1 --2- 172.21.15.38:0/867337706 >> v2:172.21.15.107:3300/0 conn(0x7f380c06c110 0x7f380c06c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:37.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.210+0000 7f3811f96640 1 -- 172.21.15.38:0/867337706 >> 172.21.15.38:0/867337706 conn(0x7f380c0d3fb0 msgr2=0x7f380c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:37.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.210+0000 7f3811f96640 1 -- 172.21.15.38:0/867337706 shutdown_connections 2024-04-25T08:34:37.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.210+0000 7f3811f96640 1 -- 172.21.15.38:0/867337706 wait complete. 2024-04-25T08:34:37.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.211+0000 7f3811f96640 1 Processor -- start 2024-04-25T08:34:37.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.212+0000 7f3811f96640 1 -- start start 2024-04-25T08:34:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.212+0000 7f3811f96640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f380c06c110 0x7f380c150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.212+0000 7f3811f96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f380c0ce2e0 0x7f380c1609a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.212+0000 7f3811f96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f380c15df50 0x7f380c0c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.212+0000 7f3811f96640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c0cece0 con 0x7f380c15df50 2024-04-25T08:34:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.212+0000 7f3811f96640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c0d2530 con 0x7f380c0ce2e0 2024-04-25T08:34:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.212+0000 7f3811f96640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380c0d2f90 con 0x7f380c06c110 2024-04-25T08:34:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.212+0000 7f380bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f380c15df50 0x7f380c0c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.212+0000 7f380bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f380c15df50 0x7f380c0c8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53130/0 (socket says 172.21.15.38:53130) 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.212+0000 7f380bfff640 1 -- 172.21.15.38:0/995345261 learned_addr learned my addr 172.21.15.38:0/995345261 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f380affd640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f380c0ce2e0 0x7f380c1609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f380b7fe640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f380c06c110 0x7f380c150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f380bfff640 1 -- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f380c06c110 msgr2=0x7f380c150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f380bfff640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f380c06c110 0x7f380c150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f380bfff640 1 -- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f380c0ce2e0 msgr2=0x7f380c1609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f380bfff640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f380c0ce2e0 0x7f380c1609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f380bfff640 1 -- 172.21.15.38:0/995345261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c15f920 con 0x7f380c15df50 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f380b7fe640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f380c06c110 0x7f380c150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f380bfff640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f380c15df50 0x7f380c0c8f20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f380400efc0 tx=0x7f380400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f37f3fff640 1 -- 172.21.15.38:0/995345261 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804018020 con 0x7f380c15df50 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f37f3fff640 1 -- 172.21.15.38:0/995345261 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3804014480 con 0x7f380c15df50 2024-04-25T08:34:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f3811f96640 1 -- 172.21.15.38:0/995345261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380c15fbe0 con 0x7f380c15df50 2024-04-25T08:34:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f37f3fff640 1 -- 172.21.15.38:0/995345261 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3804014870 con 0x7f380c15df50 2024-04-25T08:34:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.213+0000 7f3811f96640 1 -- 172.21.15.38:0/995345261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380c1601a0 con 0x7f380c15df50 2024-04-25T08:34:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.214+0000 7f3811f96640 1 -- 172.21.15.38:0/995345261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380c068e80 con 0x7f380c15df50 2024-04-25T08:34:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.215+0000 7f37f3fff640 1 -- 172.21.15.38:0/995345261 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3804005d10 con 0x7f380c15df50 2024-04-25T08:34:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.215+0000 7f37f3fff640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f37f806b4d0 0x7f37f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.215+0000 7f37f3fff640 1 -- 172.21.15.38:0/995345261 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f380408adf0 con 0x7f380c15df50 2024-04-25T08:34:37.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.215+0000 7f380b7fe640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f37f806b4d0 0x7f37f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:37.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.216+0000 7f380b7fe640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f37f806b4d0 0x7f37f806d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009b10 tx=0x7f37fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:37.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.219+0000 7f37f3fff640 1 -- 172.21.15.38:0/995345261 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3804022030 con 0x7f380c15df50 2024-04-25T08:34:37.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:37.470+0000 7f3811f96640 1 -- 172.21.15.38:0/995345261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f380c0d32c0 con 0x7f380c15df50 2024-04-25T08:34:38.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.482+0000 7f37f3fff640 1 -- 172.21.15.38:0/995345261 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v160) v1 ==== 146+0+0 (secure 0 0 0) 0x7f3804010040 con 0x7f380c15df50 2024-04-25T08:34:38.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.485+0000 7f3811f96640 1 -- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f37f806b4d0 msgr2=0x7f37f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:38.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.485+0000 7f3811f96640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f37f806b4d0 0x7f37f806d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009b10 tx=0x7f37fc008040 comp rx=0 tx=0).stop 2024-04-25T08:34:38.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.485+0000 7f3811f96640 1 -- 172.21.15.38:0/995345261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f380c15df50 msgr2=0x7f380c0c8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:38.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.485+0000 7f3811f96640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f380c15df50 0x7f380c0c8f20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f380400efc0 tx=0x7f380400c490 comp rx=0 tx=0).stop 2024-04-25T08:34:38.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.485+0000 7f3811f96640 1 -- 172.21.15.38:0/995345261 shutdown_connections 2024-04-25T08:34:38.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.485+0000 7f3811f96640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f380c15df50 0x7f380c0c8f20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.485+0000 7f3811f96640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f380c0ce2e0 0x7f380c1609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.485+0000 7f3811f96640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f37f806b4d0 0x7f37f806d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.485+0000 7f3811f96640 1 --2- 172.21.15.38:0/995345261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f380c06c110 0x7f380c150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.485+0000 7f3811f96640 1 -- 172.21.15.38:0/995345261 >> 172.21.15.38:0/995345261 conn(0x7f380c0d3fb0 msgr2=0x7f380c161840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:38.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.485+0000 7f3811f96640 1 -- 172.21.15.38:0/995345261 shutdown_connections 2024-04-25T08:34:38.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.485+0000 7f3811f96640 1 -- 172.21.15.38:0/995345261 wait complete. 2024-04-25T08:34:38.494 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:34:38.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.610+0000 7f0745a3f640 1 Processor -- start 2024-04-25T08:34:38.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.610+0000 7f0745a3f640 1 -- start start 2024-04-25T08:34:38.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f0745a3f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07400d33d0 0x7f07400d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f0745a3f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07400d2330 con 0x7f07400d1e90 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f0745a3f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07400d2610 con 0x7f07400d0320 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f0745a3f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07400d28f0 con 0x7f07400d33d0 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073f7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f07400d1e90 0x7f07400b6980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:58422/0 (socket says 172.21.15.38:58422) 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073f7fe640 1 -- 172.21.15.38:0/496401239 learned_addr learned my addr 172.21.15.38:0/496401239 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073e7fc640 1 --2- 172.21.15.38:0/496401239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07400d33d0 0x7f07400d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330776028 0 0) 0x7f07400d2330 con 0x7f07400d1e90 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0728003520 con 0x7f07400d1e90 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 402151830 0 0) 0x7f0728003520 con 0x7f07400d1e90 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07400d33d0 msgr2=0x7f07400d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 --2- 172.21.15.38:0/496401239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07400d33d0 0x7f07400d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 >> v1:172.21.15.107:6789/0 conn(0x7f07400d0320 legacy=0x7f07400d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07400d2bd0 con 0x7f07400d1e90 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f07380019f0 con 0x7f07400d1e90 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 >> v1:172.21.15.38:6789/0 conn(0x7f07400d1e90 legacy=0x7f07400b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 --2- 172.21.15.38:0/496401239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07280039e0 0x7f0728003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 --2- 172.21.15.38:0/496401239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0728004800 0x7f0728004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 --2- 172.21.15.38:0/496401239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07400d33d0 0x7f07280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07400d28f0 con 0x7f07280039e0 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072800bb30 con 0x7f0728004800 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.611+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072800bd30 con 0x7f07400d33d0 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.612+0000 7f073effd640 1 --2- 172.21.15.38:0/496401239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07280039e0 0x7f0728003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.612+0000 7f073e7fc640 1 --2- 172.21.15.38:0/496401239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0728004800 0x7f0728004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.612+0000 7f073f7fe640 1 --2- 172.21.15.38:0/496401239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07400d33d0 0x7f07280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.612+0000 7f073effd640 1 -- 172.21.15.38:0/496401239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07400d33d0 msgr2=0x7f07280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.612+0000 7f073effd640 1 --2- 172.21.15.38:0/496401239 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07400d33d0 0x7f07280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.612+0000 7f073effd640 1 -- 172.21.15.38:0/496401239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0728004800 msgr2=0x7f0728004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:38.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.612+0000 7f073effd640 1 --2- 172.21.15.38:0/496401239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0728004800 0x7f0728004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.612+0000 7f073effd640 1 -- 172.21.15.38:0/496401239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0728003520 con 0x7f07280039e0 2024-04-25T08:34:38.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.613+0000 7f073effd640 1 --2- 172.21.15.38:0/496401239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07280039e0 0x7f0728003e00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0730007fd0 tx=0x7f0728005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c5b74225f177c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:38.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.613+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300129f0 con 0x7f07280039e0 2024-04-25T08:34:38.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.613+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07300052d0 con 0x7f07280039e0 2024-04-25T08:34:38.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.613+0000 7f073dffb640 1 -- 172.21.15.38:0/496401239 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730012310 con 0x7f07280039e0 2024-04-25T08:34:38.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.613+0000 7f0745a3f640 1 -- 172.21.15.38:0/496401239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07280039e0 msgr2=0x7f0728003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.613+0000 7f0745a3f640 1 --2- 172.21.15.38:0/496401239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07280039e0 0x7f0728003e00 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f0730007fd0 tx=0x7f0728005830 comp rx=0 tx=0).stop 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.613+0000 7f073f7fe640 1 -- 172.21.15.38:0/496401239 reap_dead start 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.613+0000 7f0745a3f640 1 -- 172.21.15.38:0/496401239 shutdown_connections 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.613+0000 7f0745a3f640 1 -- 172.21.15.38:0/496401239 >> 172.21.15.38:0/496401239 conn(0x7f07400668f0 msgr2=0x7f0740066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.613+0000 7f0745a3f640 1 -- 172.21.15.38:0/496401239 shutdown_connections 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.613+0000 7f0745a3f640 1 -- 172.21.15.38:0/496401239 wait complete. 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.615+0000 7f0745a3f640 1 Processor -- start 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.615+0000 7f0745a3f640 1 -- start start 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.615+0000 7f0745a3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07280039e0 0x7f0740157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.615+0000 7f0745a3f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f074006f260 0x7f074015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.615+0000 7f0745a3f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f074006f6e0 0x7f074015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.615+0000 7f0745a3f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0740161390 con 0x7f07280039e0 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.615+0000 7f0745a3f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0740161210 con 0x7f074006f6e0 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.615+0000 7f0745a3f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07400b9720 con 0x7f074006f260 2024-04-25T08:34:38.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f073effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07280039e0 0x7f0740157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f073effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07280039e0 0x7f0740157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53148/0 (socket says 172.21.15.38:53148) 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f073effd640 1 -- 172.21.15.38:0/3433776448 learned_addr learned my addr 172.21.15.38:0/3433776448 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f073e7fc640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f074006f260 0x7f074015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f073effd640 1 -- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f074006f260 msgr2=0x7f074015b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f073effd640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f074006f260 0x7f074015b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f073effd640 1 -- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f074006f6e0 msgr2=0x7f074015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f073f7fe640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f074006f6e0 0x7f074015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f073effd640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f074006f6e0 0x7f074015bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f073effd640 1 -- 172.21.15.38:0/3433776448 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074015d750 con 0x7f07280039e0 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f073effd640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07280039e0 0x7f0740157ef0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0730012de0 tx=0x7f0730001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f0727fff640 1 -- 172.21.15.38:0/3433776448 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073004a070 con 0x7f07280039e0 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f0727fff640 1 -- 172.21.15.38:0/3433776448 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07300129d0 con 0x7f07280039e0 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f0727fff640 1 -- 172.21.15.38:0/3433776448 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730007a30 con 0x7f07280039e0 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f0745a3f640 1 -- 172.21.15.38:0/3433776448 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07400b4320 con 0x7f07280039e0 2024-04-25T08:34:38.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.616+0000 7f0745a3f640 1 -- 172.21.15.38:0/3433776448 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0740158430 con 0x7f07280039e0 2024-04-25T08:34:38.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.617+0000 7f0745a3f640 1 -- 172.21.15.38:0/3433776448 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07400b4840 con 0x7f07280039e0 2024-04-25T08:34:38.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.618+0000 7f0727fff640 1 -- 172.21.15.38:0/3433776448 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f073004d070 con 0x7f07280039e0 2024-04-25T08:34:38.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.618+0000 7f0727fff640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f072c06b500 0x7f072c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:38.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.619+0000 7f0727fff640 1 -- 172.21.15.38:0/3433776448 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f07300b1fc0 con 0x7f07280039e0 2024-04-25T08:34:38.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.619+0000 7f073e7fc640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f072c06b500 0x7f072c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:38.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.619+0000 7f073e7fc640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f072c06b500 0x7f072c06d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f07400cccd0 tx=0x7f0734008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:38.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.623+0000 7f0727fff640 1 -- 172.21.15.38:0/3433776448 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07400b4840 con 0x7f07280039e0 2024-04-25T08:34:38.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.807+0000 7f0745a3f640 1 -- 172.21.15.38:0/3433776448 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0740068e80 con 0x7f07280039e0 2024-04-25T08:34:38.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.814+0000 7f0727fff640 1 -- 172.21.15.38:0/3433776448 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0730081040 con 0x7f07280039e0 2024-04-25T08:34:38.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.817+0000 7f0745a3f640 1 -- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f072c06b500 msgr2=0x7f072c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:38.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.817+0000 7f0745a3f640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f072c06b500 0x7f072c06d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f07400cccd0 tx=0x7f0734008040 comp rx=0 tx=0).stop 2024-04-25T08:34:38.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.817+0000 7f0745a3f640 1 -- 172.21.15.38:0/3433776448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07280039e0 msgr2=0x7f0740157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:38.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.817+0000 7f0745a3f640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07280039e0 0x7f0740157ef0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0730012de0 tx=0x7f0730001120 comp rx=0 tx=0).stop 2024-04-25T08:34:38.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.817+0000 7f0745a3f640 1 -- 172.21.15.38:0/3433776448 shutdown_connections 2024-04-25T08:34:38.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.817+0000 7f0745a3f640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f074006f6e0 0x7f074015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.817+0000 7f0745a3f640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f072c06b500 0x7f072c06d9c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.817+0000 7f0745a3f640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f074006f260 0x7f074015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.817+0000 7f0745a3f640 1 --2- 172.21.15.38:0/3433776448 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07280039e0 0x7f0740157ef0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:38.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.817+0000 7f0745a3f640 1 -- 172.21.15.38:0/3433776448 >> 172.21.15.38:0/3433776448 conn(0x7f07400668f0 msgr2=0x7f074015fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:38.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.817+0000 7f0745a3f640 1 -- 172.21.15.38:0/3433776448 shutdown_connections 2024-04-25T08:34:38.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:38.817+0000 7f0745a3f640 1 -- 172.21.15.38:0/3433776448 wait complete. 2024-04-25T08:34:38.823 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:34:38.823 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:34:38.862 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:34:38.905 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:34:38.905 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:34:38.905 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:34:38.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:34:38.921 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:34:38.921 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:34:38.963 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:34:38.963 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:34:38.963 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:34:38.978 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:34:38.978 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:34:38.978 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:34:39.020 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:34:39.020 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:34:39.020 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:34:39.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.126+0000 7f8dfdb2b640 1 Processor -- start 2024-04-25T08:34:39.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.126+0000 7f8dfdb2b640 1 -- start start 2024-04-25T08:34:39.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.127+0000 7f8dfdb2b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8df8144760 0x7f8df815faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:39.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.127+0000 7f8dfdb2b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8df8160030 0x7f8df80ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:39.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.127+0000 7f8dfdb2b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df80b50c0 con 0x7f8df80b4b50 2024-04-25T08:34:39.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.127+0000 7f8dfdb2b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df815f380 con 0x7f8df8144760 2024-04-25T08:34:39.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.127+0000 7f8dfdb2b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df815f660 con 0x7f8df8160030 2024-04-25T08:34:39.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.127+0000 7f8df7fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f8df80b4b50 0x7f8df80b3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33190/0 (socket says 172.21.15.38:33190) 2024-04-25T08:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.127+0000 7f8df7fff640 1 -- 172.21.15.38:0/2000478980 learned_addr learned my addr 172.21.15.38:0/2000478980 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.127+0000 7f8df6ffd640 1 --2- 172.21.15.38:0/2000478980 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8df8160030 0x7f8df80ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.127+0000 7f8df77fe640 1 --2- 172.21.15.38:0/2000478980 >> v2:172.21.15.107:3300/0 conn(0x7f8df8144760 0x7f8df815faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.128+0000 7f8df67fc640 1 -- 172.21.15.38:0/2000478980 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363697178 0 0) 0x7f8df80b50c0 con 0x7f8df80b4b50 2024-04-25T08:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.128+0000 7f8df67fc640 1 -- 172.21.15.38:0/2000478980 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8de0003520 con 0x7f8df80b4b50 2024-04-25T08:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.128+0000 7f8df6ffd640 1 -- 172.21.15.38:0/2000478980 >> v2:172.21.15.107:3300/0 conn(0x7f8df8144760 msgr2=0x7f8df815faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.128+0000 7f8df6ffd640 1 --2- 172.21.15.38:0/2000478980 >> v2:172.21.15.107:3300/0 conn(0x7f8df8144760 0x7f8df815faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.128+0000 7f8df6ffd640 1 -- 172.21.15.38:0/2000478980 >> v1:172.21.15.38:6789/0 conn(0x7f8df80b4b50 legacy=0x7f8df80b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.128+0000 7f8df6ffd640 1 -- 172.21.15.38:0/2000478980 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df80b53a0 con 0x7f8df8160030 2024-04-25T08:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.128+0000 7f8df77fe640 1 --2- 172.21.15.38:0/2000478980 >> v2:172.21.15.107:3300/0 conn(0x7f8df8144760 0x7f8df815faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:39.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.128+0000 7f8df6ffd640 1 --2- 172.21.15.38:0/2000478980 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8df8160030 0x7f8df80ce360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0097a0 tx=0x7f8dec02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74d195a20428e6e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:39.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.129+0000 7f8df67fc640 1 -- 172.21.15.38:0/2000478980 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec037070 con 0x7f8df8160030 2024-04-25T08:34:39.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.129+0000 7f8df67fc640 1 -- 172.21.15.38:0/2000478980 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec02ae50 con 0x7f8df8160030 2024-04-25T08:34:39.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.129+0000 7f8df67fc640 1 -- 172.21.15.38:0/2000478980 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec032660 con 0x7f8df8160030 2024-04-25T08:34:39.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.129+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/2000478980 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8df8160030 msgr2=0x7f8df80ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:39.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.129+0000 7f8dfdb2b640 1 --2- 172.21.15.38:0/2000478980 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8df8160030 0x7f8df80ce360 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8dec0097a0 tx=0x7f8dec02a100 comp rx=0 tx=0).stop 2024-04-25T08:34:39.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.129+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/2000478980 shutdown_connections 2024-04-25T08:34:39.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.129+0000 7f8dfdb2b640 1 --2- 172.21.15.38:0/2000478980 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8df8160030 0x7f8df80ce360 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.129+0000 7f8dfdb2b640 1 --2- 172.21.15.38:0/2000478980 >> v2:172.21.15.107:3300/0 conn(0x7f8df8144760 0x7f8df815faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.129+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/2000478980 >> 172.21.15.38:0/2000478980 conn(0x7f8df806b210 msgr2=0x7f8df806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:39.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.129+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/2000478980 shutdown_connections 2024-04-25T08:34:39.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.129+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/2000478980 wait complete. 2024-04-25T08:34:39.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.130+0000 7f8dfdb2b640 1 Processor -- start 2024-04-25T08:34:39.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.131+0000 7f8dfdb2b640 1 -- start start 2024-04-25T08:34:39.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.131+0000 7f8dfdb2b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8df80b4b50 0x7f8df80cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:39.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.131+0000 7f8dfdb2b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df8160030 0x7f8df80c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:39.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.131+0000 7f8dfdb2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df80c5f30 0x7f8df81f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:39.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.131+0000 7f8dfdb2b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8145400 con 0x7f8df80c5f30 2024-04-25T08:34:39.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.131+0000 7f8dfdb2b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8145280 con 0x7f8df8160030 2024-04-25T08:34:39.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.131+0000 7f8dfdb2b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80b3f20 con 0x7f8df80b4b50 2024-04-25T08:34:39.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df80c5f30 0x7f8df81f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df80c5f30 0x7f8df81f7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50784/0 (socket says 172.21.15.38:50784) 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df7fff640 1 -- 172.21.15.38:0/3540807385 learned_addr learned my addr 172.21.15.38:0/3540807385 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df6ffd640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df8160030 0x7f8df80c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df77fe640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8df80b4b50 0x7f8df80cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df7fff640 1 -- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8df80b4b50 msgr2=0x7f8df80cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df7fff640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8df80b4b50 0x7f8df80cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df7fff640 1 -- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df8160030 msgr2=0x7f8df80c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df7fff640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df8160030 0x7f8df80c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df7fff640 1 -- 172.21.15.38:0/3540807385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df8141c90 con 0x7f8df80c5f30 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df6ffd640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df8160030 0x7f8df80c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df77fe640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8df80b4b50 0x7f8df80cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8df7fff640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df80c5f30 0x7f8df81f7250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8df000cb40 tx=0x7f8df000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8ddffff640 1 -- 172.21.15.38:0/3540807385 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0017070 con 0x7f8df80c5f30 2024-04-25T08:34:39.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8ddffff640 1 -- 172.21.15.38:0/3540807385 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df000aca0 con 0x7f8df80c5f30 2024-04-25T08:34:39.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/3540807385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df8141f70 con 0x7f8df80c5f30 2024-04-25T08:34:39.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8ddffff640 1 -- 172.21.15.38:0/3540807385 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0012620 con 0x7f8df80c5f30 2024-04-25T08:34:39.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.132+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/3540807385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df8142530 con 0x7f8df80c5f30 2024-04-25T08:34:39.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.133+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/3540807385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df8068e80 con 0x7f8df80c5f30 2024-04-25T08:34:39.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.134+0000 7f8ddffff640 1 -- 172.21.15.38:0/3540807385 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8df001a070 con 0x7f8df80c5f30 2024-04-25T08:34:39.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.134+0000 7f8ddffff640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8de406b4d0 0x7f8de406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:39.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.134+0000 7f8ddffff640 1 -- 172.21.15.38:0/3540807385 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8df0089e90 con 0x7f8df80c5f30 2024-04-25T08:34:39.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.135+0000 7f8df77fe640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8de406b4d0 0x7f8de406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:39.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.135+0000 7f8df77fe640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8de406b4d0 0x7f8de406d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8df81419f0 tx=0x7f8de8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:39.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.139+0000 7f8ddffff640 1 -- 172.21.15.38:0/3540807385 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8df0058f10 con 0x7f8df80c5f30 2024-04-25T08:34:39.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.357+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/3540807385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8df806df60 con 0x7f8df80c5f30 2024-04-25T08:34:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.358+0000 7f8ddffff640 1 -- 172.21.15.38:0/3540807385 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f8df0058810 con 0x7f8df80c5f30 2024-04-25T08:34:39.353 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:39.353 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4910,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/422749749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":422749749},{"type":"v1","addr":"172.21.15.38:6834","nonce":422749749}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5143,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/4194720752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":4194720752},{"type":"v1","addr":"172.21.15.107:6836","nonce":4194720752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5146,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3379884609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3379884609},{"type":"v1","addr":"172.21.15.107:6837","nonce":3379884609}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3655197328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3655197328},{"type":"v1","addr":"172.21.15.38:6835","nonce":3655197328}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[]} 2024-04-25T08:34:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 20 2024-04-25T08:34:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.360+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8de406b4d0 msgr2=0x7f8de406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.360+0000 7f8dfdb2b640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8de406b4d0 0x7f8de406d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8df81419f0 tx=0x7f8de8008040 comp rx=0 tx=0).stop 2024-04-25T08:34:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.360+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/3540807385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df80c5f30 msgr2=0x7f8df81f7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.360+0000 7f8dfdb2b640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df80c5f30 0x7f8df81f7250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f8df000cb40 tx=0x7f8df000a060 comp rx=0 tx=0).stop 2024-04-25T08:34:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.360+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/3540807385 shutdown_connections 2024-04-25T08:34:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.360+0000 7f8dfdb2b640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8df80c5f30 0x7f8df81f7250 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.360+0000 7f8dfdb2b640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8df8160030 0x7f8df80c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.360+0000 7f8dfdb2b640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8de406b4d0 0x7f8de406d990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.360+0000 7f8dfdb2b640 1 --2- 172.21.15.38:0/3540807385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8df80b4b50 0x7f8df80cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.360+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/3540807385 >> 172.21.15.38:0/3540807385 conn(0x7f8df806b210 msgr2=0x7f8df80b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:39.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.360+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/3540807385 shutdown_connections 2024-04-25T08:34:39.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.360+0000 7f8dfdb2b640 1 -- 172.21.15.38:0/3540807385 wait complete. 2024-04-25T08:34:39.367 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:34:39.367 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:34:39.367 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:34:39.367 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:34:39.367 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:34:39.367 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:34:39.367 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:34:39.368 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:34:39.368 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:34:39.368 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:34:39.368 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:34:39.368 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:34:39.369 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:34:39.369 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:34:39.369 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:34:39.369 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:34:39.376 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:34:39.376 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:34:39.378 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:34:39.378 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:34:39.380 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:34:39.380 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:34:39.381 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:34:39.381 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:34:39.419 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:34:39.419 DEBUG:teuthology.parallel:result is None 2024-04-25T08:34:39.419 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:34:39.420 DEBUG:teuthology.parallel:result is None 2024-04-25T08:34:39.422 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:34:39.422 DEBUG:teuthology.parallel:result is None 2024-04-25T08:34:39.423 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:34:39.423 DEBUG:teuthology.parallel:result is None 2024-04-25T08:34:39.423 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:34:39.496 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:34:39.496 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:34:39.502 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:34:39.503 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-25T08:34:39.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.534+0000 7fcf182b3640 1 Processor -- start 2024-04-25T08:34:39.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.534+0000 7fcf182b3640 1 -- start start 2024-04-25T08:34:39.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.535+0000 7fcf182b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf1015def0 0x7fcf100d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:39.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.535+0000 7fcf182b3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcf1009a180 0x7fcf100cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:39.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.535+0000 7fcf182b3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf100cfc70 con 0x7fcf1006c8c0 2024-04-25T08:34:39.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.535+0000 7fcf182b3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1015e2d0 con 0x7fcf1009a180 2024-04-25T08:34:39.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.535+0000 7fcf182b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100d1250 con 0x7fcf1015def0 2024-04-25T08:34:39.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.536+0000 7fcf1584e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcf1006c8c0 0x7fcf10099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33200/0 (socket says 172.21.15.38:33200) 2024-04-25T08:34:39.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.536+0000 7fcf1584e640 1 -- 172.21.15.38:0/857081940 learned_addr learned my addr 172.21.15.38:0/857081940 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.536+0000 7fcf16850640 1 --2- 172.21.15.38:0/857081940 >> v2:172.21.15.107:3300/0 conn(0x7fcf1009a180 0x7fcf100cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.536+0000 7fcf1604f640 1 --2- 172.21.15.38:0/857081940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf1015def0 0x7fcf100d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.536+0000 7fcf1504d640 1 -- 172.21.15.38:0/857081940 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189030445 0 0) 0x7fcf100cfc70 con 0x7fcf1006c8c0 2024-04-25T08:34:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.536+0000 7fcf1504d640 1 -- 172.21.15.38:0/857081940 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcefc003520 con 0x7fcf1006c8c0 2024-04-25T08:34:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.536+0000 7fcf1604f640 1 -- 172.21.15.38:0/857081940 >> v2:172.21.15.107:3300/0 conn(0x7fcf1009a180 msgr2=0x7fcf100cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.536+0000 7fcf1604f640 1 --2- 172.21.15.38:0/857081940 >> v2:172.21.15.107:3300/0 conn(0x7fcf1009a180 0x7fcf100cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.536+0000 7fcf1604f640 1 -- 172.21.15.38:0/857081940 >> v1:172.21.15.38:6789/0 conn(0x7fcf1006c8c0 legacy=0x7fcf10099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.536+0000 7fcf1604f640 1 -- 172.21.15.38:0/857081940 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf100cff50 con 0x7fcf1015def0 2024-04-25T08:34:39.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.536+0000 7fcf16850640 1 --2- 172.21.15.38:0/857081940 >> v2:172.21.15.107:3300/0 conn(0x7fcf1009a180 0x7fcf100cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.537+0000 7fcf1604f640 1 --2- 172.21.15.38:0/857081940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf1015def0 0x7fcf100d0d10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0097a0 tx=0x7fcf0c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5dc17da11d772a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.537+0000 7fcf1504d640 1 -- 172.21.15.38:0/857081940 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c037070 con 0x7fcf1015def0 2024-04-25T08:34:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.537+0000 7fcf1504d640 1 -- 172.21.15.38:0/857081940 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c032440 con 0x7fcf1015def0 2024-04-25T08:34:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.537+0000 7fcf1504d640 1 -- 172.21.15.38:0/857081940 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c032810 con 0x7fcf1015def0 2024-04-25T08:34:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.537+0000 7fcf182b3640 1 -- 172.21.15.38:0/857081940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf1015def0 msgr2=0x7fcf100d0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.537+0000 7fcf182b3640 1 --2- 172.21.15.38:0/857081940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf1015def0 0x7fcf100d0d10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0097a0 tx=0x7fcf0c02a180 comp rx=0 tx=0).stop 2024-04-25T08:34:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.537+0000 7fcf182b3640 1 -- 172.21.15.38:0/857081940 shutdown_connections 2024-04-25T08:34:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.537+0000 7fcf182b3640 1 --2- 172.21.15.38:0/857081940 >> v2:172.21.15.107:3300/0 conn(0x7fcf1009a180 0x7fcf100cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.537+0000 7fcf182b3640 1 --2- 172.21.15.38:0/857081940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf1015def0 0x7fcf100d0d10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.538+0000 7fcf182b3640 1 -- 172.21.15.38:0/857081940 >> 172.21.15.38:0/857081940 conn(0x7fcf100b30f0 msgr2=0x7fcf100b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.538+0000 7fcf182b3640 1 -- 172.21.15.38:0/857081940 shutdown_connections 2024-04-25T08:34:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.538+0000 7fcf182b3640 1 -- 172.21.15.38:0/857081940 wait complete. 2024-04-25T08:34:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.539+0000 7fcf182b3640 1 Processor -- start 2024-04-25T08:34:39.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.539+0000 7fcf182b3640 1 -- start start 2024-04-25T08:34:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.539+0000 7fcf182b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf1006c8c0 0x7fcf10153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.539+0000 7fcf182b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf1009a180 0x7fcf1009ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf182b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf1015def0 0x7fcf1009d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf182b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100d07f0 con 0x7fcf1009a180 2024-04-25T08:34:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf182b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100d0510 con 0x7fcf1015def0 2024-04-25T08:34:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf182b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100cf8d0 con 0x7fcf1006c8c0 2024-04-25T08:34:39.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf1584e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf1009a180 0x7fcf1009ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf1584e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf1009a180 0x7fcf1009ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50808/0 (socket says 172.21.15.38:50808) 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf1584e640 1 -- 172.21.15.38:0/3186196774 learned_addr learned my addr 172.21.15.38:0/3186196774 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf1604f640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf1006c8c0 0x7fcf10153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf16850640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf1015def0 0x7fcf1009d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf1584e640 1 -- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf1006c8c0 msgr2=0x7fcf10153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf1584e640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf1006c8c0 0x7fcf10153b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf1584e640 1 -- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf1015def0 msgr2=0x7fcf1009d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf1584e640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf1015def0 0x7fcf1009d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf1584e640 1 -- 172.21.15.38:0/3186196774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1009c0d0 con 0x7fcf1009a180 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf1584e640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf1009a180 0x7fcf1009ceb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000c440 tx=0x7fcf0000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf06ffd640 1 -- 172.21.15.38:0/3186196774 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00017070 con 0x7fcf1009a180 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf06ffd640 1 -- 172.21.15.38:0/3186196774 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf00005ce0 con 0x7fcf1009a180 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf182b3640 1 -- 172.21.15.38:0/3186196774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1009c3b0 con 0x7fcf1009a180 2024-04-25T08:34:39.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf06ffd640 1 -- 172.21.15.38:0/3186196774 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf000072b0 con 0x7fcf1009a180 2024-04-25T08:34:39.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.540+0000 7fcf182b3640 1 -- 172.21.15.38:0/3186196774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1009b3e0 con 0x7fcf1009a180 2024-04-25T08:34:39.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.541+0000 7fcf182b3640 1 -- 172.21.15.38:0/3186196774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1004ef90 con 0x7fcf1009a180 2024-04-25T08:34:39.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.542+0000 7fcf06ffd640 1 -- 172.21.15.38:0/3186196774 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf0001a070 con 0x7fcf1009a180 2024-04-25T08:34:39.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.542+0000 7fcf06ffd640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcef406b4d0 0x7fcef406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:39.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.542+0000 7fcf06ffd640 1 -- 172.21.15.38:0/3186196774 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcf00089430 con 0x7fcf1009a180 2024-04-25T08:34:39.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.542+0000 7fcf1604f640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcef406b4d0 0x7fcef406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:39.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.543+0000 7fcf1604f640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcef406b4d0 0x7fcef406d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0097a0 tx=0x7fcf0c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:39.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.547+0000 7fcf06ffd640 1 -- 172.21.15.38:0/3186196774 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf00058450 con 0x7fcf1009a180 2024-04-25T08:34:39.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:39.737+0000 7fcf182b3640 1 -- 172.21.15.38:0/3186196774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcf1009af90 con 0x7fcf1009a180 2024-04-25T08:34:40.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.847+0000 7fcf06ffd640 1 -- 172.21.15.38:0/3186196774 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcf00057d50 con 0x7fcf1009a180 2024-04-25T08:34:40.842 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-04-25T08:34:40.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.850+0000 7fcf182b3640 1 -- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcef406b4d0 msgr2=0x7fcef406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:40.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.850+0000 7fcf182b3640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcef406b4d0 0x7fcef406d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0097a0 tx=0x7fcf0c02e000 comp rx=0 tx=0).stop 2024-04-25T08:34:40.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.850+0000 7fcf182b3640 1 -- 172.21.15.38:0/3186196774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf1009a180 msgr2=0x7fcf1009ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:40.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.850+0000 7fcf182b3640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf1009a180 0x7fcf1009ceb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000c440 tx=0x7fcf0000c900 comp rx=0 tx=0).stop 2024-04-25T08:34:40.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.850+0000 7fcf182b3640 1 -- 172.21.15.38:0/3186196774 shutdown_connections 2024-04-25T08:34:40.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.850+0000 7fcf182b3640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcf1015def0 0x7fcf1009d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:40.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.850+0000 7fcf182b3640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf1009a180 0x7fcf1009ceb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:40.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.850+0000 7fcf182b3640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcef406b4d0 0x7fcef406d990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:40.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.850+0000 7fcf182b3640 1 --2- 172.21.15.38:0/3186196774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcf1006c8c0 0x7fcf10153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:40.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.850+0000 7fcf182b3640 1 -- 172.21.15.38:0/3186196774 >> 172.21.15.38:0/3186196774 conn(0x7fcf100b30f0 msgr2=0x7fcf10073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:40.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.850+0000 7fcf182b3640 1 -- 172.21.15.38:0/3186196774 shutdown_connections 2024-04-25T08:34:40.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.850+0000 7fcf182b3640 1 -- 172.21.15.38:0/3186196774 wait complete. 2024-04-25T08:34:40.859 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:34:40.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.971+0000 7f85e32bc640 1 Processor -- start 2024-04-25T08:34:40.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.971+0000 7f85e32bc640 1 -- start start 2024-04-25T08:34:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.972+0000 7f85e32bc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f85dc0d0140 0x7f85dc0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.972+0000 7f85e32bc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f85dc0d2400 0x7f85dc0d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.972+0000 7f85e32bc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85dc162160 con 0x7f85dc15da90 2024-04-25T08:34:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.972+0000 7f85e32bc640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85dc0d0b00 con 0x7f85dc0d2400 2024-04-25T08:34:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.972+0000 7f85e32bc640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85dc0d19b0 con 0x7f85dc0d0140 2024-04-25T08:34:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.972+0000 7f85e1859640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f85dc0d2400 0x7f85dc0d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.972+0000 7f85e1058640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f85dc15da90 0x7f85dc161710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33210/0 (socket says 172.21.15.38:33210) 2024-04-25T08:34:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.972+0000 7f85e1058640 1 -- 172.21.15.38:0/3144447225 learned_addr learned my addr 172.21.15.38:0/3144447225 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.973+0000 7f85e0857640 1 --2- 172.21.15.38:0/3144447225 >> v2:172.21.15.107:3300/0 conn(0x7f85dc0d0140 0x7f85dc0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.973+0000 7f85e1859640 1 -- 172.21.15.38:0/3144447225 >> v2:172.21.15.107:3300/0 conn(0x7f85dc0d0140 msgr2=0x7f85dc0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.973+0000 7f85e1859640 1 --2- 172.21.15.38:0/3144447225 >> v2:172.21.15.107:3300/0 conn(0x7f85dc0d0140 0x7f85dc0d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:40.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.973+0000 7f85e1859640 1 -- 172.21.15.38:0/3144447225 >> v1:172.21.15.38:6789/0 conn(0x7f85dc15da90 legacy=0x7f85dc161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:40.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.973+0000 7f85e1859640 1 -- 172.21.15.38:0/3144447225 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc162440 con 0x7f85dc0d2400 2024-04-25T08:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.973+0000 7f85e1859640 1 --2- 172.21.15.38:0/3144447225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85dc0d2400 0x7f85dc0d2850 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f85dc08c2f0 tx=0x7f85cc02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07e3a857c72b1da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.973+0000 7f85dbfff640 1 -- 172.21.15.38:0/3144447225 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc007740 con 0x7f85dc0d2400 2024-04-25T08:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.974+0000 7f85dbfff640 1 -- 172.21.15.38:0/3144447225 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85cc0078e0 con 0x7f85dc0d2400 2024-04-25T08:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.974+0000 7f85dbfff640 1 -- 172.21.15.38:0/3144447225 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc007cb0 con 0x7f85dc0d2400 2024-04-25T08:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.974+0000 7f85e32bc640 1 -- 172.21.15.38:0/3144447225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85dc0d2400 msgr2=0x7f85dc0d2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.974+0000 7f85e32bc640 1 --2- 172.21.15.38:0/3144447225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85dc0d2400 0x7f85dc0d2850 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f85dc08c2f0 tx=0x7f85cc02a5f0 comp rx=0 tx=0).stop 2024-04-25T08:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.974+0000 7f85e32bc640 1 -- 172.21.15.38:0/3144447225 shutdown_connections 2024-04-25T08:34:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.974+0000 7f85e32bc640 1 --2- 172.21.15.38:0/3144447225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85dc0d2400 0x7f85dc0d2850 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:40.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.974+0000 7f85e32bc640 1 --2- 172.21.15.38:0/3144447225 >> v2:172.21.15.107:3300/0 conn(0x7f85dc0d0140 0x7f85dc0d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:40.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.974+0000 7f85e32bc640 1 -- 172.21.15.38:0/3144447225 >> 172.21.15.38:0/3144447225 conn(0x7f85dc06c060 msgr2=0x7f85dc06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:40.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.974+0000 7f85e32bc640 1 -- 172.21.15.38:0/3144447225 shutdown_connections 2024-04-25T08:34:40.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.974+0000 7f85e32bc640 1 -- 172.21.15.38:0/3144447225 wait complete. 2024-04-25T08:34:40.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.975+0000 7f85e32bc640 1 Processor -- start 2024-04-25T08:34:40.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.975+0000 7f85e32bc640 1 -- start start 2024-04-25T08:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.976+0000 7f85e32bc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85dc0d0140 0x7f85dc151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.976+0000 7f85e32bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85dc0d2400 0x7f85dc0c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.976+0000 7f85e32bc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85dc15da90 0x7f85dc15ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.976+0000 7f85e32bc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc161e20 con 0x7f85dc0d2400 2024-04-25T08:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.976+0000 7f85e32bc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0d1f70 con 0x7f85dc15da90 2024-04-25T08:34:40.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.976+0000 7f85e32bc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85dc0b9400 con 0x7f85dc0d0140 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.976+0000 7f85e0857640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85dc0d2400 0x7f85dc0c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.976+0000 7f85e0857640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85dc0d2400 0x7f85dc0c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50820/0 (socket says 172.21.15.38:50820) 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.976+0000 7f85e0857640 1 -- 172.21.15.38:0/605946629 learned_addr learned my addr 172.21.15.38:0/605946629 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.976+0000 7f85e1859640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85dc15da90 0x7f85dc15ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.976+0000 7f85e0857640 1 -- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85dc0d0140 msgr2=0x7f85dc151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.976+0000 7f85e1058640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85dc0d0140 0x7f85dc151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.977+0000 7f85e0857640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85dc0d0140 0x7f85dc151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.977+0000 7f85e0857640 1 -- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85dc15da90 msgr2=0x7f85dc15ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.977+0000 7f85e0857640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85dc15da90 0x7f85dc15ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.977+0000 7f85e0857640 1 -- 172.21.15.38:0/605946629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc15e880 con 0x7f85dc0d2400 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.977+0000 7f85e1859640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85dc15da90 0x7f85dc15ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.977+0000 7f85e0857640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85dc0d2400 0x7f85dc0c9db0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f85d000c820 tx=0x7f85d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:40.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.977+0000 7f85d9ffb640 1 -- 172.21.15.38:0/605946629 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0015070 con 0x7f85dc0d2400 2024-04-25T08:34:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.977+0000 7f85d9ffb640 1 -- 172.21.15.38:0/605946629 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d0004020 con 0x7f85dc0d2400 2024-04-25T08:34:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.977+0000 7f85d9ffb640 1 -- 172.21.15.38:0/605946629 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d00043f0 con 0x7f85dc0d2400 2024-04-25T08:34:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.977+0000 7f85e32bc640 1 -- 172.21.15.38:0/605946629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85dc15eb60 con 0x7f85dc0d2400 2024-04-25T08:34:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.977+0000 7f85e32bc640 1 -- 172.21.15.38:0/605946629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85dc15f120 con 0x7f85dc0d2400 2024-04-25T08:34:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.978+0000 7f85e32bc640 1 -- 172.21.15.38:0/605946629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85dc068e80 con 0x7f85dc0d2400 2024-04-25T08:34:40.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.978+0000 7f85d9ffb640 1 -- 172.21.15.38:0/605946629 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85d0007830 con 0x7f85dc0d2400 2024-04-25T08:34:40.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.979+0000 7f85d9ffb640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f85c806b4d0 0x7f85c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:40.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.979+0000 7f85d9ffb640 1 -- 172.21.15.38:0/605946629 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f85d0089dd0 con 0x7f85dc0d2400 2024-04-25T08:34:40.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.979+0000 7f85e1058640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f85c806b4d0 0x7f85c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:40.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.979+0000 7f85e1058640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f85c806b4d0 0x7f85c806d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f85d4007fd0 tx=0x7f85d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:40.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:40.983+0000 7f85d9ffb640 1 -- 172.21.15.38:0/605946629 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85d0058ea0 con 0x7f85dc0d2400 2024-04-25T08:34:41.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.230+0000 7f85e32bc640 1 -- 172.21.15.38:0/605946629 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f85dc0d1560 con 0x7f85dc0d2400 2024-04-25T08:34:41.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.230+0000 7f85d9ffb640 1 -- 172.21.15.38:0/605946629 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f85d008b090 con 0x7f85dc0d2400 2024-04-25T08:34:41.226 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:41.226 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:34:41.226 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-04-25T08:34:41.226 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.226 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-04-25T08:34:41.226 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmwq8CDBAA+cjd+QXtFgwZ3kxdABRWlA==", 2024-04-25T08:34:41.226 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.226 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:34:41.226 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:34:41.226 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:34:41.226 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.226 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmPRFbFxAAojn2r/62vvf1qTy9QVuyIw==", 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmGLjqEhAAyPRgyy5iMgtccapydv4uOQ==", 2024-04-25T08:34:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmt0AUHhAAoQyQ7wXBMGE+7Hi84tUfAg==", 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.228 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCxFCpmv+iTExAAEJihmsvBbp9xXxPnj49D6g==", 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC0FCpmdnQKFBAADLAsYZEUhoJx292jIYZroQ==", 2024-04-25T08:34:41.229 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC3FCpm4f5BExAAMVSPkqsrnuLcO88AXKrGHQ==", 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.230 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC6FCpmHNN7EhAABPhmnxGo70ZOd8Oex62Mdw==", 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDAFCpmiThGFhAAG5Pyvn/JzL9NUoxHJkPMvg==", 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:34:41.231 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDDFCpmb/FHFBAA2cMhMqQQc4Fk6j8FpjhnaQ==", 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-04-25T08:34:41.232 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDGFCpmAZsSEhAAnGIqqcvuHMVSc8Zbk309kA==", 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDJFCpmqdVUEBAAb9abblyUztrjwxxe4jsAwQ==", 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.233 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw==", 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmL7eKKhAASeQEdHDBInfozdLVzP4DQw==", 2024-04-25T08:34:41.234 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmH22HJxAAWyrooyJn5hRs/bb0Rn1HHw==", 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:34:41.235 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpm6Zr/LxAAxh9Y2gvvqj1jIaVVrQAvgw==", 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCpFCpmlj/fLBAAvxLluFQFR8uhppGlNpaygA==", 2024-04-25T08:34:41.236 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcXHhCBAAvI5zmWidVTvOmEcVRWasEA==", 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.237 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:41.238 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:41.238 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-04-25T08:34:41.238 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcr96BRAAQwUPxILJukYERAiEXeur+g==", 2024-04-25T08:34:41.238 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:34:41.238 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:34:41.238 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:34:41.238 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:34:41.238 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.238 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:41.238 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T08:34:41.238 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:34:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.233+0000 7f85e32bc640 1 -- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f85c806b4d0 msgr2=0x7f85c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.233+0000 7f85e32bc640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f85c806b4d0 0x7f85c806d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f85d4007fd0 tx=0x7f85d4009040 comp rx=0 tx=0).stop 2024-04-25T08:34:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.233+0000 7f85e32bc640 1 -- 172.21.15.38:0/605946629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85dc0d2400 msgr2=0x7f85dc0c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.233+0000 7f85e32bc640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85dc0d2400 0x7f85dc0c9db0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f85d000c820 tx=0x7f85d000cce0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.233+0000 7f85e32bc640 1 -- 172.21.15.38:0/605946629 shutdown_connections 2024-04-25T08:34:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.233+0000 7f85e32bc640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85dc15da90 0x7f85dc15ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.233+0000 7f85e32bc640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85dc0d2400 0x7f85dc0c9db0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.233+0000 7f85e32bc640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f85c806b4d0 0x7f85c806d990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.233+0000 7f85e32bc640 1 --2- 172.21.15.38:0/605946629 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85dc0d0140 0x7f85dc151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.233+0000 7f85e32bc640 1 -- 172.21.15.38:0/605946629 >> 172.21.15.38:0/605946629 conn(0x7f85dc06c060 msgr2=0x7f85dc0ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.233+0000 7f85e32bc640 1 -- 172.21.15.38:0/605946629 shutdown_connections 2024-04-25T08:34:41.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.233+0000 7f85e32bc640 1 -- 172.21.15.38:0/605946629 wait complete. 2024-04-25T08:34:41.241 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:34:41.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.346+0000 7f49e859b640 1 Processor -- start 2024-04-25T08:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.346+0000 7f49e859b640 1 -- start start 2024-04-25T08:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.347+0000 7f49e859b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f49e015ddf0 0x7f49e0161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.347+0000 7f49e859b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e00b98a0 con 0x7f49e00b93f0 2024-04-25T08:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.347+0000 7f49e859b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e0161e00 con 0x7f49e00b5710 2024-04-25T08:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.347+0000 7f49e859b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e01620e0 con 0x7f49e015ddf0 2024-04-25T08:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e6b38640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f49e00b93f0 0x7f49e00d2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33224/0 (socket says 172.21.15.38:33224) 2024-04-25T08:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e6b38640 1 -- 172.21.15.38:0/1880578330 learned_addr learned my addr 172.21.15.38:0/1880578330 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5b36640 1 --2- 172.21.15.38:0/1880578330 >> v2:172.21.15.107:3300/0 conn(0x7f49e015ddf0 0x7f49e0161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017337745 0 0) 0x7f49e00b98a0 con 0x7f49e00b93f0 2024-04-25T08:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49cc003520 con 0x7f49e00b93f0 2024-04-25T08:34:41.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2799613311 0 0) 0x7f49cc003520 con 0x7f49e00b93f0 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 >> v2:172.21.15.107:3300/0 conn(0x7f49e015ddf0 msgr2=0x7f49e0161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 --2- 172.21.15.38:0/1880578330 >> v2:172.21.15.107:3300/0 conn(0x7f49e015ddf0 0x7f49e0161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 >> v1:172.21.15.107:6789/0 conn(0x7f49e00b5710 legacy=0x7f49e00b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e01623c0 con 0x7f49e00b93f0 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f49d80019f0 con 0x7f49e00b93f0 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 >> v1:172.21.15.38:6789/0 conn(0x7f49e00b93f0 legacy=0x7f49e00d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 --2- 172.21.15.38:0/1880578330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49cc0039b0 0x7f49cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 --2- 172.21.15.38:0/1880578330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49cc0043b0 0x7f49cc004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 --2- 172.21.15.38:0/1880578330 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49cc005080 0x7f49cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e01620e0 con 0x7f49cc0043b0 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc00bfe0 con 0x7f49cc0039b0 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc00c1e0 con 0x7f49cc005080 2024-04-25T08:34:41.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.348+0000 7f49e5b36640 1 --2- 172.21.15.38:0/1880578330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49cc0043b0 0x7f49cc004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.349+0000 7f49e6337640 1 --2- 172.21.15.38:0/1880578330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49cc0039b0 0x7f49cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.349+0000 7f49e6b38640 1 --2- 172.21.15.38:0/1880578330 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49cc005080 0x7f49cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.349+0000 7f49e5b36640 1 -- 172.21.15.38:0/1880578330 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49cc005080 msgr2=0x7f49cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.349+0000 7f49e5b36640 1 --2- 172.21.15.38:0/1880578330 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49cc005080 0x7f49cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.349+0000 7f49e5b36640 1 -- 172.21.15.38:0/1880578330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49cc0039b0 msgr2=0x7f49cc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.349+0000 7f49e5b36640 1 --2- 172.21.15.38:0/1880578330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49cc0039b0 0x7f49cc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.349+0000 7f49e5b36640 1 -- 172.21.15.38:0/1880578330 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc003520 con 0x7f49cc0043b0 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.349+0000 7f49e6b38640 1 -- 172.21.15.38:0/1880578330 reap_dead start 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.349+0000 7f49e5b36640 1 --2- 172.21.15.38:0/1880578330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49cc0043b0 0x7f49cc004880 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009a90 tx=0x7f49cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2031f46adeb7560 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.350+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0036070 con 0x7f49cc0043b0 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.350+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d0031400 con 0x7f49cc0043b0 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.350+0000 7f49e5335640 1 -- 172.21.15.38:0/1880578330 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0009690 con 0x7f49cc0043b0 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.350+0000 7f49e859b640 1 -- 172.21.15.38:0/1880578330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49cc0043b0 msgr2=0x7f49cc004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.350+0000 7f49e859b640 1 --2- 172.21.15.38:0/1880578330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49cc0043b0 0x7f49cc004880 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009a90 tx=0x7f49cc005830 comp rx=0 tx=0).stop 2024-04-25T08:34:41.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.350+0000 7f49e859b640 1 -- 172.21.15.38:0/1880578330 shutdown_connections 2024-04-25T08:34:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.350+0000 7f49e859b640 1 --2- 172.21.15.38:0/1880578330 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49cc0043b0 0x7f49cc004880 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.350+0000 7f49e859b640 1 -- 172.21.15.38:0/1880578330 >> 172.21.15.38:0/1880578330 conn(0x7f49e00d07c0 msgr2=0x7f49e00d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.350+0000 7f49e859b640 1 -- 172.21.15.38:0/1880578330 shutdown_connections 2024-04-25T08:34:41.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.350+0000 7f49e859b640 1 -- 172.21.15.38:0/1880578330 wait complete. 2024-04-25T08:34:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.351+0000 7f49e859b640 1 Processor -- start 2024-04-25T08:34:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.351+0000 7f49e859b640 1 -- start start 2024-04-25T08:34:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.352+0000 7f49e859b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49cc0043b0 0x7f49e00ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.352+0000 7f49e859b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e00b7a70 0x7f49e0160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.352+0000 7f49e859b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e00c9960 0x7f49e00c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.352+0000 7f49e859b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e0161a90 con 0x7f49e00c9960 2024-04-25T08:34:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.352+0000 7f49e859b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e0161910 con 0x7f49e00b7a70 2024-04-25T08:34:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.352+0000 7f49e859b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00b5af0 con 0x7f49cc0043b0 2024-04-25T08:34:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.352+0000 7f49e6b38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e00c9960 0x7f49e00c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.352+0000 7f49e6b38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e00c9960 0x7f49e00c9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50840/0 (socket says 172.21.15.38:50840) 2024-04-25T08:34:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.352+0000 7f49e6b38640 1 -- 172.21.15.38:0/1753174689 learned_addr learned my addr 172.21.15.38:0/1753174689 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:41.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49e5b36640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e00b7a70 0x7f49e0160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49e6337640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49cc0043b0 0x7f49e00ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49e6b38640 1 -- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49cc0043b0 msgr2=0x7f49e00ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49e6b38640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49cc0043b0 0x7f49e00ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49e6b38640 1 -- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e00b7a70 msgr2=0x7f49e0160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49e6b38640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e00b7a70 0x7f49e0160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49e6b38640 1 -- 172.21.15.38:0/1753174689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0158750 con 0x7f49e00c9960 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49e6337640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49cc0043b0 0x7f49e00ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49e6b38640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e00c9960 0x7f49e00c9de0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f49e006d2f0 tx=0x7f49d800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49d6ffd640 1 -- 172.21.15.38:0/1753174689 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d800eb90 con 0x7f49e00c9960 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49d6ffd640 1 -- 172.21.15.38:0/1753174689 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d8018070 con 0x7f49e00c9960 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49e859b640 1 -- 172.21.15.38:0/1753174689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e00cd370 con 0x7f49e00c9960 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49d6ffd640 1 -- 172.21.15.38:0/1753174689 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d8013420 con 0x7f49e00c9960 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.353+0000 7f49e859b640 1 -- 172.21.15.38:0/1753174689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e00cd930 con 0x7f49e00c9960 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.354+0000 7f49e859b640 1 -- 172.21.15.38:0/1753174689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e0068e80 con 0x7f49e00c9960 2024-04-25T08:34:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.354+0000 7f49d6ffd640 1 -- 172.21.15.38:0/1753174689 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49d80135c0 con 0x7f49e00c9960 2024-04-25T08:34:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.355+0000 7f49d6ffd640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f49c406b4d0 0x7f49c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.355+0000 7f49d6ffd640 1 -- 172.21.15.38:0/1753174689 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f49d808a3b0 con 0x7f49e00c9960 2024-04-25T08:34:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.355+0000 7f49e6337640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f49c406b4d0 0x7f49c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.356+0000 7f49e6337640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f49c406b4d0 0x7f49c406d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0017e0 tx=0x7f49dc007b70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.359+0000 7f49d6ffd640 1 -- 172.21.15.38:0/1753174689 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49d80593f0 con 0x7f49e00c9960 2024-04-25T08:34:41.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.539+0000 7f49e859b640 1 -- 172.21.15.38:0/1753174689 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49e00b98a0 con 0x7f49e00c9960 2024-04-25T08:34:41.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.540+0000 7f49d6ffd640 1 -- 172.21.15.38:0/1753174689 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13152 (secure 0 0 0) 0x7f49d8058cf0 con 0x7f49e00c9960 2024-04-25T08:34:41.535 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:41.536 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":50,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:34:40.842767+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:34:41.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.542+0000 7f49e859b640 1 -- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f49c406b4d0 msgr2=0x7f49c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.542+0000 7f49e859b640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f49c406b4d0 0x7f49c406d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0017e0 tx=0x7f49dc007b70 comp rx=0 tx=0).stop 2024-04-25T08:34:41.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.542+0000 7f49e859b640 1 -- 172.21.15.38:0/1753174689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e00c9960 msgr2=0x7f49e00c9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.542+0000 7f49e859b640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e00c9960 0x7f49e00c9de0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f49e006d2f0 tx=0x7f49d800e530 comp rx=0 tx=0).stop 2024-04-25T08:34:41.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.542+0000 7f49e859b640 1 -- 172.21.15.38:0/1753174689 shutdown_connections 2024-04-25T08:34:41.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.542+0000 7f49e859b640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49e00c9960 0x7f49e00c9de0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.542+0000 7f49e859b640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49e00b7a70 0x7f49e0160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.542+0000 7f49e859b640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f49c406b4d0 0x7f49c406d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.542+0000 7f49e859b640 1 --2- 172.21.15.38:0/1753174689 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49cc0043b0 0x7f49e00ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.542+0000 7f49e859b640 1 -- 172.21.15.38:0/1753174689 >> 172.21.15.38:0/1753174689 conn(0x7f49e00d07c0 msgr2=0x7f49e00d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:41.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.542+0000 7f49e859b640 1 -- 172.21.15.38:0/1753174689 shutdown_connections 2024-04-25T08:34:41.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.542+0000 7f49e859b640 1 -- 172.21.15.38:0/1753174689 wait complete. 2024-04-25T08:34:41.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:34:41.549 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:34:41.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.657+0000 7f517c5b7640 1 Processor -- start 2024-04-25T08:34:41.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.657+0000 7f517c5b7640 1 -- start start 2024-04-25T08:34:41.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.658+0000 7f517c5b7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5174166610 0x7f5174168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.658+0000 7f517c5b7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51740d2060 con 0x7f51740d0320 2024-04-25T08:34:41.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.658+0000 7f517c5b7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51740d2330 con 0x7f51740d33d0 2024-04-25T08:34:41.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.658+0000 7f517c5b7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51740d2610 con 0x7f5174166610 2024-04-25T08:34:41.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.658+0000 7f517a353640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f51740d0320 0x7f51740d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33228/0 (socket says 172.21.15.38:33228) 2024-04-25T08:34:41.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.658+0000 7f517a353640 1 -- 172.21.15.38:0/1771979304 learned_addr learned my addr 172.21.15.38:0/1771979304 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:41.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.658+0000 7f517ab54640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5174166610 0x7f5174168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.658+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426773568 0 0) 0x7f51740d2060 con 0x7f51740d0320 2024-04-25T08:34:41.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.658+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5160003520 con 0x7f51740d0320 2024-04-25T08:34:41.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018179776 0 0) 0x7f51740d2330 con 0x7f51740d33d0 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51740d2060 con 0x7f51740d33d0 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2075145612 0 0) 0x7f5160003520 con 0x7f51740d0320 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5174166610 msgr2=0x7f5174168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5174166610 0x7f5174168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 >> v1:172.21.15.107:6789/0 conn(0x7f51740d33d0 legacy=0x7f51740d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51740d28f0 con 0x7f51740d0320 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f516c001690 con 0x7f51740d0320 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f517ab54640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5174166610 0x7f5174168a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 >> v1:172.21.15.38:6789/0 conn(0x7f51740d0320 legacy=0x7f51740d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5160003b70 0x7f5160003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5174166610 0x7f5160004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5160004d40 0x7f51600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51740d2610 con 0x7f5160003b70 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516000ba70 con 0x7f5160004d40 2024-04-25T08:34:41.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516000bc20 con 0x7f5174166610 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f517a353640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5160003b70 0x7f5160003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f517ab54640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5160004d40 0x7f51600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f5179b52640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5174166610 0x7f5160004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f517a353640 1 -- 172.21.15.38:0/1771979304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5174166610 msgr2=0x7f5160004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f517a353640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5174166610 0x7f5160004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f517a353640 1 -- 172.21.15.38:0/1771979304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5160004d40 msgr2=0x7f51600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f517a353640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5160004d40 0x7f51600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.659+0000 7f517a353640 1 -- 172.21.15.38:0/1771979304 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160003520 con 0x7f5160003b70 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.660+0000 7f517a353640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5160003b70 0x7f5160003f70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f516c00c210 tx=0x7f5160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8a46b6ce2b347f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.660+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c007ce0 con 0x7f5160003b70 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.660+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c02e070 con 0x7f5160003b70 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.660+0000 7f5179351640 1 -- 172.21.15.38:0/1771979304 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c005d30 con 0x7f5160003b70 2024-04-25T08:34:41.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.661+0000 7f517c5b7640 1 -- 172.21.15.38:0/1771979304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5160003b70 msgr2=0x7f5160003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.661+0000 7f517c5b7640 1 --2- 172.21.15.38:0/1771979304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5160003b70 0x7f5160003f70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f516c00c210 tx=0x7f5160005830 comp rx=0 tx=0).stop 2024-04-25T08:34:41.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.661+0000 7f517ab54640 1 -- 172.21.15.38:0/1771979304 reap_dead start 2024-04-25T08:34:41.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.661+0000 7f517c5b7640 1 -- 172.21.15.38:0/1771979304 shutdown_connections 2024-04-25T08:34:41.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.661+0000 7f517c5b7640 1 -- 172.21.15.38:0/1771979304 >> 172.21.15.38:0/1771979304 conn(0x7f51740668f0 msgr2=0x7f5174066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:41.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.661+0000 7f517c5b7640 1 -- 172.21.15.38:0/1771979304 shutdown_connections 2024-04-25T08:34:41.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.661+0000 7f517c5b7640 1 -- 172.21.15.38:0/1771979304 wait complete. 2024-04-25T08:34:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.662+0000 7f517c5b7640 1 Processor -- start 2024-04-25T08:34:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.662+0000 7f517c5b7640 1 -- start start 2024-04-25T08:34:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517c5b7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5160003b70 0x7f5174149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517c5b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51740b5020 0x7f517415fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517c5b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51741602e0 0x7f51740b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517c5b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5174161210 con 0x7f51741602e0 2024-04-25T08:34:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517c5b7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51740d2eb0 con 0x7f51740b5020 2024-04-25T08:34:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517c5b7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5174161390 con 0x7f5160003b70 2024-04-25T08:34:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517ab54640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51741602e0 0x7f51740b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517ab54640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51741602e0 0x7f51740b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50854/0 (socket says 172.21.15.38:50854) 2024-04-25T08:34:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517ab54640 1 -- 172.21.15.38:0/3333130461 learned_addr learned my addr 172.21.15.38:0/3333130461 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517a353640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5160003b70 0x7f5174149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f5179b52640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51740b5020 0x7f517415fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517ab54640 1 -- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5160003b70 msgr2=0x7f5174149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517ab54640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5160003b70 0x7f5174149e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517ab54640 1 -- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51740b5020 msgr2=0x7f517415fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517ab54640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51740b5020 0x7f517415fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517ab54640 1 -- 172.21.15.38:0/3333130461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5174159ae0 con 0x7f51741602e0 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.663+0000 7f517ab54640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51741602e0 0x7f51740b3270 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f51740675b0 tx=0x7f517000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.664+0000 7f516affd640 1 -- 172.21.15.38:0/3333130461 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517000fb80 con 0x7f51741602e0 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.664+0000 7f516affd640 1 -- 172.21.15.38:0/3333130461 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517000fd60 con 0x7f51741602e0 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.664+0000 7f517c5b7640 1 -- 172.21.15.38:0/3333130461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5174159dc0 con 0x7f51741602e0 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.664+0000 7f516affd640 1 -- 172.21.15.38:0/3333130461 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170003bd0 con 0x7f51741602e0 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.664+0000 7f517c5b7640 1 -- 172.21.15.38:0/3333130461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517415a380 con 0x7f51741602e0 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.665+0000 7f517c5b7640 1 -- 172.21.15.38:0/3333130461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5174068e80 con 0x7f51741602e0 2024-04-25T08:34:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.665+0000 7f516affd640 1 -- 172.21.15.38:0/3333130461 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f517001f020 con 0x7f51741602e0 2024-04-25T08:34:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.665+0000 7f516affd640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f515806b500 0x7f515806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.666+0000 7f516affd640 1 -- 172.21.15.38:0/3333130461 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f517008a220 con 0x7f51741602e0 2024-04-25T08:34:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.666+0000 7f517a353640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f515806b500 0x7f515806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.666+0000 7f517a353640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f515806b500 0x7f515806d9c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f516c032000 tx=0x7f516c007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:41.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.670+0000 7f516affd640 1 -- 172.21.15.38:0/3333130461 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5170010040 con 0x7f51741602e0 2024-04-25T08:34:41.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.854+0000 7f517c5b7640 1 -- 172.21.15.38:0/3333130461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51740b3050 con 0x7f51741602e0 2024-04-25T08:34:41.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.855+0000 7f516affd640 1 -- 172.21.15.38:0/3333130461 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5170059260 con 0x7f51741602e0 2024-04-25T08:34:41.850 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:34:41.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.857+0000 7f517c5b7640 1 -- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f515806b500 msgr2=0x7f515806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.857+0000 7f517c5b7640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f515806b500 0x7f515806d9c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f516c032000 tx=0x7f516c007000 comp rx=0 tx=0).stop 2024-04-25T08:34:41.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.857+0000 7f517c5b7640 1 -- 172.21.15.38:0/3333130461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51741602e0 msgr2=0x7f51740b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.857+0000 7f517c5b7640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51741602e0 0x7f51740b3270 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f51740675b0 tx=0x7f517000f580 comp rx=0 tx=0).stop 2024-04-25T08:34:41.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.857+0000 7f517c5b7640 1 -- 172.21.15.38:0/3333130461 shutdown_connections 2024-04-25T08:34:41.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.857+0000 7f517c5b7640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51741602e0 0x7f51740b3270 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.857+0000 7f517c5b7640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51740b5020 0x7f517415fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.857+0000 7f517c5b7640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f515806b500 0x7f515806d9c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.857+0000 7f517c5b7640 1 --2- 172.21.15.38:0/3333130461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5160003b70 0x7f5174149e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.857+0000 7f517c5b7640 1 -- 172.21.15.38:0/3333130461 >> 172.21.15.38:0/3333130461 conn(0x7f51740668f0 msgr2=0x7f51740d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:41.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.857+0000 7f517c5b7640 1 -- 172.21.15.38:0/3333130461 shutdown_connections 2024-04-25T08:34:41.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.857+0000 7f517c5b7640 1 -- 172.21.15.38:0/3333130461 wait complete. 2024-04-25T08:34:41.863 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:34:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.969+0000 7f4f4d95c640 1 Processor -- start 2024-04-25T08:34:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.969+0000 7f4f4d95c640 1 -- start start 2024-04-25T08:34:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f4d95c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f4815df50 0x7f4f480ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f4d95c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4f480ce2e0 0x7f4f480d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f4d95c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f480ceff0 con 0x7f4f4806c110 2024-04-25T08:34:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f4d95c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f480cf2d0 con 0x7f4f480ce2e0 2024-04-25T08:34:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f4d95c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480d1f70 con 0x7f4f4815df50 2024-04-25T08:34:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f477fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4f480ce2e0 0x7f4f480d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f477fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4f480ce2e0 0x7f4f480d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50866/0 (socket says 172.21.15.38:50866) 2024-04-25T08:34:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f477fe640 1 -- 172.21.15.38:0/1766226995 learned_addr learned my addr 172.21.15.38:0/1766226995 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f3ffff640 1 --2- 172.21.15.38:0/1766226995 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f4815df50 0x7f4f480ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f477fe640 1 -- 172.21.15.38:0/1766226995 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f4815df50 msgr2=0x7f4f480ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f477fe640 1 --2- 172.21.15.38:0/1766226995 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f4815df50 0x7f4f480ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f477fe640 1 -- 172.21.15.38:0/1766226995 >> v1:172.21.15.107:6789/0 conn(0x7f4f4806c110 legacy=0x7f4f4806c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.970+0000 7f4f477fe640 1 -- 172.21.15.38:0/1766226995 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f480d2250 con 0x7f4f480ce2e0 2024-04-25T08:34:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.971+0000 7f4f477fe640 1 --2- 172.21.15.38:0/1766226995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f480ce2e0 0x7f4f480d2a50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009e50 tx=0x7f4f4002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178f968049859c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.971+0000 7f4f467fc640 1 -- 172.21.15.38:0/1766226995 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f40036070 con 0x7f4f480ce2e0 2024-04-25T08:34:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.971+0000 7f4f467fc640 1 -- 172.21.15.38:0/1766226995 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f40031600 con 0x7f4f480ce2e0 2024-04-25T08:34:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.971+0000 7f4f467fc640 1 -- 172.21.15.38:0/1766226995 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f400319d0 con 0x7f4f480ce2e0 2024-04-25T08:34:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.972+0000 7f4f4d95c640 1 -- 172.21.15.38:0/1766226995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f480ce2e0 msgr2=0x7f4f480d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.972+0000 7f4f4d95c640 1 --2- 172.21.15.38:0/1766226995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f480ce2e0 0x7f4f480d2a50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009e50 tx=0x7f4f4002a710 comp rx=0 tx=0).stop 2024-04-25T08:34:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.972+0000 7f4f4d95c640 1 -- 172.21.15.38:0/1766226995 shutdown_connections 2024-04-25T08:34:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.972+0000 7f4f4d95c640 1 --2- 172.21.15.38:0/1766226995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f480ce2e0 0x7f4f480d2a50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.972+0000 7f4f4d95c640 1 --2- 172.21.15.38:0/1766226995 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f4815df50 0x7f4f480ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.972+0000 7f4f4d95c640 1 -- 172.21.15.38:0/1766226995 >> 172.21.15.38:0/1766226995 conn(0x7f4f480d3fb0 msgr2=0x7f4f480d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.972+0000 7f4f4d95c640 1 -- 172.21.15.38:0/1766226995 shutdown_connections 2024-04-25T08:34:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.972+0000 7f4f4d95c640 1 -- 172.21.15.38:0/1766226995 wait complete. 2024-04-25T08:34:41.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.973+0000 7f4f4d95c640 1 Processor -- start 2024-04-25T08:34:41.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.973+0000 7f4f4d95c640 1 -- start start 2024-04-25T08:34:41.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f4d95c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f4806c110 0x7f4f48150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f4d95c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f480ce2e0 0x7f4f4815ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f4d95c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f4815df50 0x7f4f480b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f4d95c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480d2f90 con 0x7f4f4806c110 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f4d95c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480cece0 con 0x7f4f480ce2e0 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f4d95c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480d3110 con 0x7f4f4815df50 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f4806c110 0x7f4f48150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f46ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f4806c110 0x7f4f48150d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50872/0 (socket says 172.21.15.38:50872) 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f46ffd640 1 -- 172.21.15.38:0/2298853584 learned_addr learned my addr 172.21.15.38:0/2298853584 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f3ffff640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f480ce2e0 0x7f4f4815ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f477fe640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f4815df50 0x7f4f480b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f46ffd640 1 -- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f4815df50 msgr2=0x7f4f480b7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f46ffd640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f4815df50 0x7f4f480b7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f46ffd640 1 -- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f480ce2e0 msgr2=0x7f4f4815ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f46ffd640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f480ce2e0 0x7f4f4815ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f46ffd640 1 -- 172.21.15.38:0/2298853584 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f481608d0 con 0x7f4f4806c110 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f3ffff640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f480ce2e0 0x7f4f4815ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:41.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f477fe640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f4815df50 0x7f4f480b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:41.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f46ffd640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f4806c110 0x7f4f48150d80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800c3f0 tx=0x7f4f3800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:41.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.974+0000 7f4f44ff9640 1 -- 172.21.15.38:0/2298853584 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f380052d0 con 0x7f4f4806c110 2024-04-25T08:34:41.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.975+0000 7f4f44ff9640 1 -- 172.21.15.38:0/2298853584 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f38005490 con 0x7f4f4806c110 2024-04-25T08:34:41.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.975+0000 7f4f4d95c640 1 -- 172.21.15.38:0/2298853584 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f48160bb0 con 0x7f4f4806c110 2024-04-25T08:34:41.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.975+0000 7f4f44ff9640 1 -- 172.21.15.38:0/2298853584 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38007260 con 0x7f4f4806c110 2024-04-25T08:34:41.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.975+0000 7f4f4d95c640 1 -- 172.21.15.38:0/2298853584 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f480c73e0 con 0x7f4f4806c110 2024-04-25T08:34:41.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.975+0000 7f4f4d95c640 1 -- 172.21.15.38:0/2298853584 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f48068e80 con 0x7f4f4806c110 2024-04-25T08:34:41.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.976+0000 7f4f44ff9640 1 -- 172.21.15.38:0/2298853584 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f38007400 con 0x7f4f4806c110 2024-04-25T08:34:41.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.976+0000 7f4f44ff9640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f2806b500 0x7f4f2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:41.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.976+0000 7f4f44ff9640 1 -- 172.21.15.38:0/2298853584 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4f380895b0 con 0x7f4f4806c110 2024-04-25T08:34:41.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.977+0000 7f4f3ffff640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f2806b500 0x7f4f2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:41.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.977+0000 7f4f3ffff640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f2806b500 0x7f4f2806d9c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4f4815fae0 tx=0x7f4f30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:41.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:41.981+0000 7f4f44ff9640 1 -- 172.21.15.38:0/2298853584 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f38058560 con 0x7f4f4806c110 2024-04-25T08:34:42.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.164+0000 7f4f4d95c640 1 -- 172.21.15.38:0/2298853584 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4f480b5fd0 con 0x7f4f4806c110 2024-04-25T08:34:42.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.164+0000 7f4f44ff9640 1 -- 172.21.15.38:0/2298853584 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4f38057e60 con 0x7f4f4806c110 2024-04-25T08:34:42.159 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:42.159 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T08:34:42.159 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:34:42.159 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-04-25T08:34:42.159 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-04-25T08:34:42.160 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:42.160 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T08:34:42.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.166+0000 7f4f4d95c640 1 -- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f2806b500 msgr2=0x7f4f2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.166+0000 7f4f4d95c640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f2806b500 0x7f4f2806d9c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4f4815fae0 tx=0x7f4f30008040 comp rx=0 tx=0).stop 2024-04-25T08:34:42.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.166+0000 7f4f4d95c640 1 -- 172.21.15.38:0/2298853584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f4806c110 msgr2=0x7f4f48150d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.166+0000 7f4f4d95c640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f4806c110 0x7f4f48150d80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800c3f0 tx=0x7f4f3800c8b0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.166+0000 7f4f4d95c640 1 -- 172.21.15.38:0/2298853584 shutdown_connections 2024-04-25T08:34:42.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.166+0000 7f4f4d95c640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f2806b500 0x7f4f2806d9c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.166+0000 7f4f4d95c640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f4815df50 0x7f4f480b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.166+0000 7f4f4d95c640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f480ce2e0 0x7f4f4815ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.166+0000 7f4f4d95c640 1 --2- 172.21.15.38:0/2298853584 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f4806c110 0x7f4f48150d80 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.166+0000 7f4f4d95c640 1 -- 172.21.15.38:0/2298853584 >> 172.21.15.38:0/2298853584 conn(0x7f4f480d3fb0 msgr2=0x7f4f4815d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:42.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.166+0000 7f4f4d95c640 1 -- 172.21.15.38:0/2298853584 shutdown_connections 2024-04-25T08:34:42.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.166+0000 7f4f4d95c640 1 -- 172.21.15.38:0/2298853584 wait complete. 2024-04-25T08:34:42.172 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:34:42.172 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:34:42.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.282+0000 7fb0010c4640 1 Processor -- start 2024-04-25T08:34:42.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.282+0000 7fb0010c4640 1 -- start start 2024-04-25T08:34:42.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.283+0000 7fb0010c4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7faffc0d0860 0x7faffc0d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.283+0000 7fb0010c4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faffc0d4680 con 0x7faffc0d19e0 2024-04-25T08:34:42.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.283+0000 7fb0010c4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faffc0d1180 con 0x7faffc0d1e50 2024-04-25T08:34:42.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.283+0000 7fb0010c4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faffc15e2b0 con 0x7faffc0d0860 2024-04-25T08:34:42.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.283+0000 7faff3fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7faffc0d19e0 0x7faffc0d3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33232/0 (socket says 172.21.15.38:33232) 2024-04-25T08:34:42.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.283+0000 7faff3fff640 1 -- 172.21.15.38:0/979246371 learned_addr learned my addr 172.21.15.38:0/979246371 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:42.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.283+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148393809 0 0) 0x7faffc0d4680 con 0x7faffc0d19e0 2024-04-25T08:34:42.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.283+0000 7faffad9d640 1 --2- 172.21.15.38:0/979246371 >> v2:172.21.15.107:3300/0 conn(0x7faffc0d0860 0x7faffc0d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.283+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafe8003520 con 0x7faffc0d19e0 2024-04-25T08:34:42.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1067487714 0 0) 0x7fafe8003520 con 0x7faffc0d19e0 2024-04-25T08:34:42.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 >> v2:172.21.15.107:3300/0 conn(0x7faffc0d0860 msgr2=0x7faffc0d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 --2- 172.21.15.38:0/979246371 >> v2:172.21.15.107:3300/0 conn(0x7faffc0d0860 0x7faffc0d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 >> v1:172.21.15.107:6789/0 conn(0x7faffc0d1e50 legacy=0x7faffc0d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc0d24a0 con 0x7faffc0d19e0 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fafe4001670 con 0x7faffc0d19e0 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 >> v1:172.21.15.38:6789/0 conn(0x7faffc0d19e0 legacy=0x7faffc0d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 --2- 172.21.15.38:0/979246371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafe80039b0 0x7fafe8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 --2- 172.21.15.38:0/979246371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 0x7fafe8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 --2- 172.21.15.38:0/979246371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafe80051c0 0x7fafe800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc15e2b0 con 0x7fafe8004800 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe800c060 con 0x7fafe80039b0 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe800c260 con 0x7fafe80051c0 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faff3fff640 1 --2- 172.21.15.38:0/979246371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 0x7fafe8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffad9d640 1 --2- 172.21.15.38:0/979246371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafe80039b0 0x7fafe8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffb59e640 1 --2- 172.21.15.38:0/979246371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafe80051c0 0x7fafe800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faff3fff640 1 -- 172.21.15.38:0/979246371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafe80051c0 msgr2=0x7fafe800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:42.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faff3fff640 1 --2- 172.21.15.38:0/979246371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafe80051c0 0x7fafe800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faff3fff640 1 -- 172.21.15.38:0/979246371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafe80039b0 msgr2=0x7fafe8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:42.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faff3fff640 1 --2- 172.21.15.38:0/979246371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafe80039b0 0x7fafe8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faffb59e640 1 -- 172.21.15.38:0/979246371 reap_dead start 2024-04-25T08:34:42.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.284+0000 7faff3fff640 1 -- 172.21.15.38:0/979246371 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe8003520 con 0x7fafe8004800 2024-04-25T08:34:42.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.285+0000 7faff3fff640 1 --2- 172.21.15.38:0/979246371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 0x7fafe8004c80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c240 tx=0x7fafe8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdce5966159d40b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:42.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.285+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4007cf0 con 0x7fafe8004800 2024-04-25T08:34:42.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.285+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe4005af0 con 0x7fafe8004800 2024-04-25T08:34:42.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.285+0000 7faffa59c640 1 -- 172.21.15.38:0/979246371 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4007070 con 0x7fafe8004800 2024-04-25T08:34:42.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.286+0000 7fb0010c4640 1 -- 172.21.15.38:0/979246371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 msgr2=0x7fafe8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.286+0000 7fb0010c4640 1 --2- 172.21.15.38:0/979246371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 0x7fafe8004c80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fafe400c240 tx=0x7fafe8005830 comp rx=0 tx=0).stop 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.286+0000 7fb0010c4640 1 -- 172.21.15.38:0/979246371 shutdown_connections 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.286+0000 7fb0010c4640 1 --2- 172.21.15.38:0/979246371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 0x7fafe8004c80 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.286+0000 7fb0010c4640 1 -- 172.21.15.38:0/979246371 >> 172.21.15.38:0/979246371 conn(0x7faffc066d10 msgr2=0x7faffc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.286+0000 7fb0010c4640 1 -- 172.21.15.38:0/979246371 shutdown_connections 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.286+0000 7fb0010c4640 1 -- 172.21.15.38:0/979246371 wait complete. 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.287+0000 7fb0010c4640 1 Processor -- start 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.287+0000 7fb0010c4640 1 -- start start 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7fb0010c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 0x7faffc1e3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7fb0010c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faffc1e54c0 0x7faffc159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7fb0010c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faffc1e78c0 0x7faffc1e7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7fb0010c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc0d2a60 con 0x7fafe8004800 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7fb0010c4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc0d2260 con 0x7faffc1e54c0 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7fb0010c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faffc0d2d40 con 0x7faffc1e78c0 2024-04-25T08:34:42.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7faffad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 0x7faffc1e3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7faffad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 0x7faffc1e3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50898/0 (socket says 172.21.15.38:50898) 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7faffad9d640 1 -- 172.21.15.38:0/2435547352 learned_addr learned my addr 172.21.15.38:0/2435547352 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7faffb59e640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faffc1e78c0 0x7faffc1e7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7faff3fff640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faffc1e54c0 0x7faffc159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7faffad9d640 1 -- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faffc1e78c0 msgr2=0x7faffc1e7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7faffad9d640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faffc1e78c0 0x7faffc1e7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7faffad9d640 1 -- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faffc1e54c0 msgr2=0x7faffc159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7faffad9d640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faffc1e54c0 0x7faffc159fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7faffad9d640 1 -- 172.21.15.38:0/2435547352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc1e4430 con 0x7fafe8004800 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.288+0000 7faffad9d640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 0x7faffc1e3b50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fafec0118c0 tx=0x7fafec011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.289+0000 7faff37fe640 1 -- 172.21.15.38:0/2435547352 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec019070 con 0x7fafe8004800 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.289+0000 7faff37fe640 1 -- 172.21.15.38:0/2435547352 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec004240 con 0x7fafe8004800 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.289+0000 7fb0010c4640 1 -- 172.21.15.38:0/2435547352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faffc1e4710 con 0x7fafe8004800 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.289+0000 7faff37fe640 1 -- 172.21.15.38:0/2435547352 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec00f820 con 0x7fafe8004800 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.289+0000 7fb0010c4640 1 -- 172.21.15.38:0/2435547352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faffc0aa440 con 0x7fafe8004800 2024-04-25T08:34:42.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.290+0000 7fb0010c4640 1 -- 172.21.15.38:0/2435547352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faffc1e4c30 con 0x7fafe8004800 2024-04-25T08:34:42.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.290+0000 7faff37fe640 1 -- 172.21.15.38:0/2435547352 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafec01c070 con 0x7fafe8004800 2024-04-25T08:34:42.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.291+0000 7faff37fe640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fafdc06b500 0x7fafdc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.291+0000 7faff37fe640 1 -- 172.21.15.38:0/2435547352 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fafec08a330 con 0x7fafe8004800 2024-04-25T08:34:42.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.291+0000 7faff3fff640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fafdc06b500 0x7fafdc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.291+0000 7faff3fff640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fafdc06b500 0x7fafdc06d9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fafe4032000 tx=0x7fafe4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:42.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.295+0000 7faff37fe640 1 -- 172.21.15.38:0/2435547352 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faffc1e4c30 con 0x7fafe8004800 2024-04-25T08:34:42.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.476+0000 7fb0010c4640 1 -- 172.21.15.38:0/2435547352 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7faffc068e80 con 0x7fafe8004800 2024-04-25T08:34:42.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.854+0000 7faff37fe640 1 -- 172.21.15.38:0/2435547352 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) v1 ==== 138+0+0 (secure 0 0 0) 0x7fafec05f100 con 0x7fafe8004800 2024-04-25T08:34:42.849 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-04-25T08:34:42.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.856+0000 7fb0010c4640 1 -- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fafdc06b500 msgr2=0x7fafdc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.856+0000 7fb0010c4640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fafdc06b500 0x7fafdc06d9c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fafe4032000 tx=0x7fafe4009030 comp rx=0 tx=0).stop 2024-04-25T08:34:42.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.856+0000 7fb0010c4640 1 -- 172.21.15.38:0/2435547352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 msgr2=0x7faffc1e3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.856+0000 7fb0010c4640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 0x7faffc1e3b50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fafec0118c0 tx=0x7fafec011d80 comp rx=0 tx=0).stop 2024-04-25T08:34:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.856+0000 7fb0010c4640 1 -- 172.21.15.38:0/2435547352 shutdown_connections 2024-04-25T08:34:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.856+0000 7fb0010c4640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fafdc06b500 0x7fafdc06d9c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.856+0000 7fb0010c4640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faffc1e78c0 0x7faffc1e7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.856+0000 7fb0010c4640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faffc1e54c0 0x7faffc159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.856+0000 7fb0010c4640 1 --2- 172.21.15.38:0/2435547352 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafe8004800 0x7faffc1e3b50 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.856+0000 7fb0010c4640 1 -- 172.21.15.38:0/2435547352 >> 172.21.15.38:0/2435547352 conn(0x7faffc066d10 msgr2=0x7faffc0b7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.856+0000 7fb0010c4640 1 -- 172.21.15.38:0/2435547352 shutdown_connections 2024-04-25T08:34:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.856+0000 7fb0010c4640 1 -- 172.21.15.38:0/2435547352 wait complete. 2024-04-25T08:34:42.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:34:42.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.982+0000 7f5a47e2c640 1 Processor -- start 2024-04-25T08:34:42.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.982+0000 7f5a47e2c640 1 -- start start 2024-04-25T08:34:42.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.983+0000 7f5a47e2c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5a400d0320 0x7f5a400d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.983+0000 7f5a47e2c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a400d0700 0x7f5a400d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.983+0000 7f5a47e2c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a400d2330 con 0x7f5a400d1e90 2024-04-25T08:34:42.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.983+0000 7f5a47e2c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a400d2610 con 0x7f5a400d0320 2024-04-25T08:34:42.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.983+0000 7f5a47e2c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a400d28f0 con 0x7f5a400d0700 2024-04-25T08:34:42.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.983+0000 7f5a45bc8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5a400d0320 0x7f5a400d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.983+0000 7f5a463c9640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5a400d1e90 0x7f5a400b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33248/0 (socket says 172.21.15.38:33248) 2024-04-25T08:34:42.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.983+0000 7f5a463c9640 1 -- 172.21.15.38:0/2675462323 learned_addr learned my addr 172.21.15.38:0/2675462323 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:42.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.983+0000 7f5a45bc8640 1 -- 172.21.15.38:0/2675462323 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a400d0700 msgr2=0x7f5a400d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:42.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.983+0000 7f5a45bc8640 1 --2- 172.21.15.38:0/2675462323 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a400d0700 0x7f5a400d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.983+0000 7f5a45bc8640 1 -- 172.21.15.38:0/2675462323 >> v1:172.21.15.38:6789/0 conn(0x7f5a400d1e90 legacy=0x7f5a400b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.983+0000 7f5a45bc8640 1 -- 172.21.15.38:0/2675462323 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a400d2bd0 con 0x7f5a400d0320 2024-04-25T08:34:42.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.984+0000 7f5a45bc8640 1 --2- 172.21.15.38:0/2675462323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a400d0320 0x7f5a400d33d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009d90 tx=0x7f5a3c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80261bb48b5aa94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:42.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.984+0000 7f5a44bc6640 1 -- 172.21.15.38:0/2675462323 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c007740 con 0x7f5a400d0320 2024-04-25T08:34:42.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.984+0000 7f5a44bc6640 1 -- 172.21.15.38:0/2675462323 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c0078e0 con 0x7f5a400d0320 2024-04-25T08:34:42.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.984+0000 7f5a44bc6640 1 -- 172.21.15.38:0/2675462323 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c007cb0 con 0x7f5a400d0320 2024-04-25T08:34:42.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.985+0000 7f5a47e2c640 1 -- 172.21.15.38:0/2675462323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a400d0320 msgr2=0x7f5a400d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.985+0000 7f5a47e2c640 1 --2- 172.21.15.38:0/2675462323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a400d0320 0x7f5a400d33d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009d90 tx=0x7f5a3c02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.985+0000 7f5a47e2c640 1 -- 172.21.15.38:0/2675462323 shutdown_connections 2024-04-25T08:34:42.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.985+0000 7f5a47e2c640 1 --2- 172.21.15.38:0/2675462323 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a400d0700 0x7f5a400d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.985+0000 7f5a47e2c640 1 --2- 172.21.15.38:0/2675462323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a400d0320 0x7f5a400d33d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.985+0000 7f5a47e2c640 1 -- 172.21.15.38:0/2675462323 >> 172.21.15.38:0/2675462323 conn(0x7f5a400668f0 msgr2=0x7f5a40066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:42.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.985+0000 7f5a47e2c640 1 -- 172.21.15.38:0/2675462323 shutdown_connections 2024-04-25T08:34:42.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.985+0000 7f5a47e2c640 1 -- 172.21.15.38:0/2675462323 wait complete. 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.986+0000 7f5a47e2c640 1 Processor -- start 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.986+0000 7f5a47e2c640 1 -- start start 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a47e2c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a400d0320 0x7f5a400cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a47e2c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a400d0700 0x7f5a40156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a47e2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a400d1e90 0x7f5a401571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a47e2c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a400d0b40 con 0x7f5a400d1e90 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a47e2c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a40161210 con 0x7f5a400d0700 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a47e2c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a400c9bb0 con 0x7f5a400d0320 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a463c9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a400d1e90 0x7f5a401571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a463c9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a400d1e90 0x7f5a401571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50916/0 (socket says 172.21.15.38:50916) 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a463c9640 1 -- 172.21.15.38:0/1173051089 learned_addr learned my addr 172.21.15.38:0/1173051089 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a45bc8640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a400d0320 0x7f5a400cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a453c7640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a400d0700 0x7f5a40156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a463c9640 1 -- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a400d0320 msgr2=0x7f5a400cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a463c9640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a400d0320 0x7f5a400cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a463c9640 1 -- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a400d0700 msgr2=0x7f5a40156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a463c9640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a400d0700 0x7f5a40156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a463c9640 1 -- 172.21.15.38:0/1173051089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a400c9e90 con 0x7f5a400d1e90 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a453c7640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a400d0700 0x7f5a40156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.987+0000 7f5a463c9640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a400d1e90 0x7f5a401571e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c310 tx=0x7f5a3800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.988+0000 7f5a367fc640 1 -- 172.21.15.38:0/1173051089 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38017070 con 0x7f5a400d1e90 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.988+0000 7f5a367fc640 1 -- 172.21.15.38:0/1173051089 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a38005ce0 con 0x7f5a400d1e90 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.988+0000 7f5a47e2c640 1 -- 172.21.15.38:0/1173051089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a400b3d20 con 0x7f5a400d1e90 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.988+0000 7f5a367fc640 1 -- 172.21.15.38:0/1173051089 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380072d0 con 0x7f5a400d1e90 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.988+0000 7f5a47e2c640 1 -- 172.21.15.38:0/1173051089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a400cd650 con 0x7f5a400d1e90 2024-04-25T08:34:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.989+0000 7f5a47e2c640 1 -- 172.21.15.38:0/1173051089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a40068e80 con 0x7f5a400d1e90 2024-04-25T08:34:42.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.989+0000 7f5a367fc640 1 -- 172.21.15.38:0/1173051089 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a3801a070 con 0x7f5a400d1e90 2024-04-25T08:34:42.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.990+0000 7f5a367fc640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a2406b4d0 0x7f5a2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:42.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.990+0000 7f5a367fc640 1 -- 172.21.15.38:0/1173051089 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f5a38089890 con 0x7f5a400d1e90 2024-04-25T08:34:42.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.990+0000 7f5a45bc8640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a2406b4d0 0x7f5a2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:42.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.991+0000 7f5a45bc8640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a2406b4d0 0x7f5a2406d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c034000 tx=0x7f5a3c02f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:42.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:42.994+0000 7f5a367fc640 1 -- 172.21.15.38:0/1173051089 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a38058180 con 0x7f5a400d1e90 2024-04-25T08:34:43.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.177+0000 7f5a47e2c640 1 -- 172.21.15.38:0/1173051089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5a400ccef0 con 0x7f5a400d1e90 2024-04-25T08:34:43.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.860+0000 7f5a367fc640 1 -- 172.21.15.38:0/1173051089 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5a3805c220 con 0x7f5a400d1e90 2024-04-25T08:34:43.855 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-04-25T08:34:43.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.863+0000 7f5a47e2c640 1 -- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a2406b4d0 msgr2=0x7f5a2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:43.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.863+0000 7f5a47e2c640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a2406b4d0 0x7f5a2406d990 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c034000 tx=0x7f5a3c02f000 comp rx=0 tx=0).stop 2024-04-25T08:34:43.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.863+0000 7f5a47e2c640 1 -- 172.21.15.38:0/1173051089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a400d1e90 msgr2=0x7f5a401571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:43.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.863+0000 7f5a47e2c640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a400d1e90 0x7f5a401571e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c310 tx=0x7f5a3800c7d0 comp rx=0 tx=0).stop 2024-04-25T08:34:43.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.863+0000 7f5a47e2c640 1 -- 172.21.15.38:0/1173051089 shutdown_connections 2024-04-25T08:34:43.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.863+0000 7f5a47e2c640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a400d1e90 0x7f5a401571e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:43.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.863+0000 7f5a47e2c640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a400d0700 0x7f5a40156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:43.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.863+0000 7f5a47e2c640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a2406b4d0 0x7f5a2406d990 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:43.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.863+0000 7f5a47e2c640 1 --2- 172.21.15.38:0/1173051089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a400d0320 0x7f5a400cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:43.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.863+0000 7f5a47e2c640 1 -- 172.21.15.38:0/1173051089 >> 172.21.15.38:0/1173051089 conn(0x7f5a400668f0 msgr2=0x7f5a40073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:43.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.863+0000 7f5a47e2c640 1 -- 172.21.15.38:0/1173051089 shutdown_connections 2024-04-25T08:34:43.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:43.863+0000 7f5a47e2c640 1 -- 172.21.15.38:0/1173051089 wait complete. 2024-04-25T08:34:43.887 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:34:43.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.001+0000 7fa32631d640 1 Processor -- start 2024-04-25T08:34:43.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.001+0000 7fa32631d640 1 -- start start 2024-04-25T08:34:43.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.002+0000 7fa32631d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa3200d0320 0x7fa3200d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:43.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.002+0000 7fa32631d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3200d0700 0x7fa3200d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:43.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.002+0000 7fa32631d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3200d2330 con 0x7fa3200d1e90 2024-04-25T08:34:43.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.002+0000 7fa32631d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3200d2610 con 0x7fa3200d0320 2024-04-25T08:34:43.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.002+0000 7fa32631d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3200d28f0 con 0x7fa3200d0700 2024-04-25T08:34:43.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.002+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3200d0700 0x7fa3200d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:43.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.002+0000 7fa31ffff640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa3200d0320 0x7fa3200d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:43.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.002+0000 7fa3248ba640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fa3200d1e90 0x7fa3200b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:42418/0 (socket says 172.21.15.38:42418) 2024-04-25T08:34:43.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.002+0000 7fa3248ba640 1 -- 172.21.15.38:0/3865996724 learned_addr learned my addr 172.21.15.38:0/3865996724 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:43.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.003+0000 7fa31f7fe640 1 -- 172.21.15.38:0/3865996724 >> v2:172.21.15.107:3300/0 conn(0x7fa3200d0320 msgr2=0x7fa3200d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:43.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.003+0000 7fa31f7fe640 1 --2- 172.21.15.38:0/3865996724 >> v2:172.21.15.107:3300/0 conn(0x7fa3200d0320 0x7fa3200d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:43.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.003+0000 7fa31f7fe640 1 -- 172.21.15.38:0/3865996724 >> v1:172.21.15.107:6789/0 conn(0x7fa3200d1e90 legacy=0x7fa3200b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:43.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.003+0000 7fa31f7fe640 1 -- 172.21.15.38:0/3865996724 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3200d2bd0 con 0x7fa3200d0700 2024-04-25T08:34:43.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.003+0000 7fa31ffff640 1 --2- 172.21.15.38:0/3865996724 >> v2:172.21.15.107:3300/0 conn(0x7fa3200d0320 0x7fa3200d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:43.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.003+0000 7fa31f7fe640 1 --2- 172.21.15.38:0/3865996724 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3200d0700 0x7fa3200d1950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa3140097a0 tx=0x7fa31402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe563dec26a1c2e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:44.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.003+0000 7fa31effd640 1 -- 172.21.15.38:0/3865996724 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa314037070 con 0x7fa3200d0700 2024-04-25T08:34:44.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.004+0000 7fa31effd640 1 -- 172.21.15.38:0/3865996724 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa314032440 con 0x7fa3200d0700 2024-04-25T08:34:44.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.004+0000 7fa31effd640 1 -- 172.21.15.38:0/3865996724 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa314032810 con 0x7fa3200d0700 2024-04-25T08:34:44.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.004+0000 7fa32631d640 1 -- 172.21.15.38:0/3865996724 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3200d0700 msgr2=0x7fa3200d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:44.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.004+0000 7fa32631d640 1 --2- 172.21.15.38:0/3865996724 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3200d0700 0x7fa3200d1950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa3140097a0 tx=0x7fa31402a210 comp rx=0 tx=0).stop 2024-04-25T08:34:44.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.004+0000 7fa32631d640 1 -- 172.21.15.38:0/3865996724 shutdown_connections 2024-04-25T08:34:44.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.004+0000 7fa32631d640 1 --2- 172.21.15.38:0/3865996724 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3200d0700 0x7fa3200d1950 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.004+0000 7fa32631d640 1 --2- 172.21.15.38:0/3865996724 >> v2:172.21.15.107:3300/0 conn(0x7fa3200d0320 0x7fa3200d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.004+0000 7fa32631d640 1 -- 172.21.15.38:0/3865996724 >> 172.21.15.38:0/3865996724 conn(0x7fa3200668f0 msgr2=0x7fa320066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:44.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.004+0000 7fa32631d640 1 -- 172.21.15.38:0/3865996724 shutdown_connections 2024-04-25T08:34:44.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.004+0000 7fa32631d640 1 -- 172.21.15.38:0/3865996724 wait complete. 2024-04-25T08:34:44.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.005+0000 7fa32631d640 1 Processor -- start 2024-04-25T08:34:44.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.005+0000 7fa32631d640 1 -- start start 2024-04-25T08:34:44.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.006+0000 7fa32631d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3200d0320 0x7fa3201f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:44.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.006+0000 7fa32631d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3200d0700 0x7fa3200b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.006+0000 7fa32631d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3200d1e90 0x7fa3200b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.006+0000 7fa32631d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3200d0b40 con 0x7fa3200d0700 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.006+0000 7fa32631d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa320161210 con 0x7fa3200d1e90 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.006+0000 7fa32631d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa320155ed0 con 0x7fa3200d0320 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.006+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3200d0700 0x7fa3200b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.006+0000 7fa31f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3200d0700 0x7fa3200b5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50930/0 (socket says 172.21.15.38:50930) 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.006+0000 7fa31f7fe640 1 -- 172.21.15.38:0/1951811628 learned_addr learned my addr 172.21.15.38:0/1951811628 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.006+0000 7fa3248ba640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3200d1e90 0x7fa3200b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.006+0000 7fa31ffff640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3200d0320 0x7fa3201f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.007+0000 7fa31f7fe640 1 -- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3200d0320 msgr2=0x7fa3201f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.007+0000 7fa31f7fe640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3200d0320 0x7fa3201f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.007+0000 7fa31f7fe640 1 -- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3200d1e90 msgr2=0x7fa3200b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.007+0000 7fa31f7fe640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3200d1e90 0x7fa3200b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.007+0000 7fa31f7fe640 1 -- 172.21.15.38:0/1951811628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3201f7cc0 con 0x7fa3200d0700 2024-04-25T08:34:44.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.007+0000 7fa31f7fe640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3200d0700 0x7fa3200b5b90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa3140098c0 tx=0x7fa3140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:44.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.007+0000 7fa31cff9640 1 -- 172.21.15.38:0/1951811628 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa314037040 con 0x7fa3200d0700 2024-04-25T08:34:44.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.007+0000 7fa31cff9640 1 -- 172.21.15.38:0/1951811628 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa314030500 con 0x7fa3200d0700 2024-04-25T08:34:44.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.007+0000 7fa32631d640 1 -- 172.21.15.38:0/1951811628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3201f7fa0 con 0x7fa3200d0700 2024-04-25T08:34:44.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.007+0000 7fa31cff9640 1 -- 172.21.15.38:0/1951811628 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3140308d0 con 0x7fa3200d0700 2024-04-25T08:34:44.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.007+0000 7fa32631d640 1 -- 172.21.15.38:0/1951811628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32014a800 con 0x7fa3200d0700 2024-04-25T08:34:44.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.008+0000 7fa32631d640 1 -- 172.21.15.38:0/1951811628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa320068e80 con 0x7fa3200d0700 2024-04-25T08:34:44.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.008+0000 7fa31cff9640 1 -- 172.21.15.38:0/1951811628 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa314044020 con 0x7fa3200d0700 2024-04-25T08:34:44.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.009+0000 7fa31cff9640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa30c06b4d0 0x7fa30c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:44.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.009+0000 7fa31cff9640 1 -- 172.21.15.38:0/1951811628 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fa3140aa610 con 0x7fa3200d0700 2024-04-25T08:34:44.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.009+0000 7fa31ffff640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa30c06b4d0 0x7fa30c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:44.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.010+0000 7fa31ffff640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa30c06b4d0 0x7fa30c06d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa310009560 tx=0x7fa310008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:44.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.014+0000 7fa31cff9640 1 -- 172.21.15.38:0/1951811628 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa314079360 con 0x7fa3200d0700 2024-04-25T08:34:44.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.233+0000 7fa32631d640 1 -- 172.21.15.38:0/1951811628 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa3200b3050 con 0x7fa3200d0700 2024-04-25T08:34:44.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.862+0000 7fa31cff9640 1 -- 172.21.15.38:0/1951811628 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa314078c60 con 0x7fa3200d0700 2024-04-25T08:34:44.857 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T08:34:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.865+0000 7fa32631d640 1 -- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa30c06b4d0 msgr2=0x7fa30c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.865+0000 7fa32631d640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa30c06b4d0 0x7fa30c06d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa310009560 tx=0x7fa310008040 comp rx=0 tx=0).stop 2024-04-25T08:34:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.865+0000 7fa32631d640 1 -- 172.21.15.38:0/1951811628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3200d0700 msgr2=0x7fa3200b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.865+0000 7fa32631d640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3200d0700 0x7fa3200b5b90 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fa3140098c0 tx=0x7fa3140046c0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.865+0000 7fa32631d640 1 -- 172.21.15.38:0/1951811628 shutdown_connections 2024-04-25T08:34:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.865+0000 7fa32631d640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa3200d1e90 0x7fa3200b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.865+0000 7fa32631d640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa3200d0700 0x7fa3200b5b90 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.865+0000 7fa32631d640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa30c06b4d0 0x7fa30c06d990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.865+0000 7fa32631d640 1 --2- 172.21.15.38:0/1951811628 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa3200d0320 0x7fa3201f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.865+0000 7fa32631d640 1 -- 172.21.15.38:0/1951811628 >> 172.21.15.38:0/1951811628 conn(0x7fa3200668f0 msgr2=0x7fa3200d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.865+0000 7fa32631d640 1 -- 172.21.15.38:0/1951811628 shutdown_connections 2024-04-25T08:34:44.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.865+0000 7fa32631d640 1 -- 172.21.15.38:0/1951811628 wait complete. 2024-04-25T08:34:44.878 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:34:44.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.987+0000 7f3dd4e4a640 1 Processor -- start 2024-04-25T08:34:44.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.987+0000 7f3dd4e4a640 1 -- start start 2024-04-25T08:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.988+0000 7f3dd4e4a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3dd006c110 0x7f3dd006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.988+0000 7f3dd4e4a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dd00ce2e0 0x7f3dd00d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.988+0000 7f3dd4e4a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd006ca30 con 0x7f3dd015df50 2024-04-25T08:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.988+0000 7f3dd4e4a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd00cf2d0 con 0x7f3dd006c110 2024-04-25T08:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.988+0000 7f3dd4e4a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd00ceff0 con 0x7f3dd00ce2e0 2024-04-25T08:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.988+0000 7f3dced9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dd00ce2e0 0x7f3dd00d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.988+0000 7f3dce59c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3dd006c110 0x7f3dd006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.988+0000 7f3dced9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dd00ce2e0 0x7f3dd00d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:45766/0 (socket says 172.21.15.38:45766) 2024-04-25T08:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.988+0000 7f3dce59c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3dd006c110 0x7f3dd006c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:41226/0 (socket says 172.21.15.38:41226) 2024-04-25T08:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.988+0000 7f3dced9d640 1 -- 172.21.15.38:0/1772471071 learned_addr learned my addr 172.21.15.38:0/1772471071 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:44.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.988+0000 7f3dc7fff640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f3dd015df50 0x7f3dd00ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:42426/0 (socket says 172.21.15.38:42426) 2024-04-25T08:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.989+0000 7f3dce59c640 1 -- 172.21.15.38:0/1772471071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dd00ce2e0 msgr2=0x7f3dd00d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.989+0000 7f3dce59c640 1 --2- 172.21.15.38:0/1772471071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dd00ce2e0 0x7f3dd00d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.989+0000 7f3dce59c640 1 -- 172.21.15.38:0/1772471071 >> v1:172.21.15.107:6789/0 conn(0x7f3dd015df50 legacy=0x7f3dd00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.989+0000 7f3dcdd9b640 1 -- 172.21.15.38:0/1772471071 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592294969 0 0) 0x7f3dd006ca30 con 0x7f3dd015df50 2024-04-25T08:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.989+0000 7f3dce59c640 1 -- 172.21.15.38:0/1772471071 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd00d1f70 con 0x7f3dd006c110 2024-04-25T08:34:44.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.989+0000 7f3dced9d640 1 --2- 172.21.15.38:0/1772471071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dd00ce2e0 0x7f3dd00d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:44.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.989+0000 7f3dce59c640 1 --2- 172.21.15.38:0/1772471071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd006c110 0x7f3dd006c4f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00097a0 tx=0x7f3dc002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d857a31361ec1df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.990+0000 7f3dcdd9b640 1 -- 172.21.15.38:0/1772471071 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0037070 con 0x7f3dd006c110 2024-04-25T08:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.990+0000 7f3dcdd9b640 1 -- 172.21.15.38:0/1772471071 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc0032440 con 0x7f3dd006c110 2024-04-25T08:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.990+0000 7f3dcdd9b640 1 -- 172.21.15.38:0/1772471071 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0032810 con 0x7f3dd006c110 2024-04-25T08:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.990+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/1772471071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd006c110 msgr2=0x7f3dd006c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.990+0000 7f3dd4e4a640 1 --2- 172.21.15.38:0/1772471071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd006c110 0x7f3dd006c4f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3dc00097a0 tx=0x7f3dc002a180 comp rx=0 tx=0).stop 2024-04-25T08:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.990+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/1772471071 shutdown_connections 2024-04-25T08:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.990+0000 7f3dd4e4a640 1 --2- 172.21.15.38:0/1772471071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dd00ce2e0 0x7f3dd00d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.990+0000 7f3dd4e4a640 1 --2- 172.21.15.38:0/1772471071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd006c110 0x7f3dd006c4f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.990+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/1772471071 >> 172.21.15.38:0/1772471071 conn(0x7f3dd00d3fb0 msgr2=0x7f3dd00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.990+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/1772471071 shutdown_connections 2024-04-25T08:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.990+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/1772471071 wait complete. 2024-04-25T08:34:44.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.991+0000 7f3dd4e4a640 1 Processor -- start 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.991+0000 7f3dd4e4a640 1 -- start start 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.992+0000 7f3dd4e4a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dd006c110 0x7f3dd01f4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.992+0000 7f3dd4e4a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dd00ce2e0 0x7f3dd01f0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.992+0000 7f3dd4e4a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd015df50 0x7f3dd00c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.992+0000 7f3dd4e4a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd00d2f90 con 0x7f3dd00ce2e0 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.992+0000 7f3dd4e4a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd00d2530 con 0x7f3dd015df50 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.992+0000 7f3dd4e4a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd00d3110 con 0x7f3dd006c110 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.992+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dd00ce2e0 0x7f3dd01f0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.992+0000 7f3dc7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dd00ce2e0 0x7f3dd01f0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50944/0 (socket says 172.21.15.38:50944) 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.992+0000 7f3dc7fff640 1 -- 172.21.15.38:0/2471106610 learned_addr learned my addr 172.21.15.38:0/2471106610 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dce59c640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dd006c110 0x7f3dd01f4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dced9d640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd015df50 0x7f3dd00c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:44.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dc7fff640 1 -- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dd006c110 msgr2=0x7f3dd01f4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dc7fff640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dd006c110 0x7f3dd01f4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dc7fff640 1 -- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd015df50 msgr2=0x7f3dd00c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dc7fff640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd015df50 0x7f3dd00c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dc7fff640 1 -- 172.21.15.38:0/2471106610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd015f700 con 0x7f3dd00ce2e0 2024-04-25T08:34:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dc7fff640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dd00ce2e0 0x7f3dd01f0e40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3db800c430 tx=0x7f3db800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dc6ffd640 1 -- 172.21.15.38:0/2471106610 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db80074e0 con 0x7f3dd00ce2e0 2024-04-25T08:34:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dc6ffd640 1 -- 172.21.15.38:0/2471106610 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db80047e0 con 0x7f3dd00ce2e0 2024-04-25T08:34:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dc6ffd640 1 -- 172.21.15.38:0/2471106610 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8004bd0 con 0x7f3dd00ce2e0 2024-04-25T08:34:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/2471106610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd01f4980 con 0x7f3dd00ce2e0 2024-04-25T08:34:44.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.993+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/2471106610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd015e940 con 0x7f3dd00ce2e0 2024-04-25T08:34:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.994+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/2471106610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd0068e80 con 0x7f3dd00ce2e0 2024-04-25T08:34:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.995+0000 7f3dc6ffd640 1 -- 172.21.15.38:0/2471106610 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3db8011070 con 0x7f3dd00ce2e0 2024-04-25T08:34:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.995+0000 7f3dc6ffd640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3db006b4d0 0x7f3db006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:44.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.995+0000 7f3dc6ffd640 1 -- 172.21.15.38:0/2471106610 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3db808a000 con 0x7f3dd00ce2e0 2024-04-25T08:34:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.995+0000 7f3dce59c640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3db006b4d0 0x7f3db006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:44.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:44.996+0000 7f3dce59c640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3db006b4d0 0x7f3db006d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0034000 tx=0x7f3dc002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:44.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.000+0000 7f3dc6ffd640 1 -- 172.21.15.38:0/2471106610 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3db8058cf0 con 0x7f3dd00ce2e0 2024-04-25T08:34:45.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.181+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/2471106610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dd00ce920 con 0x7f3dd00ce2e0 2024-04-25T08:34:45.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.182+0000 7f3dc6ffd640 1 -- 172.21.15.38:0/2471106610 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f3db80585f0 con 0x7f3dd00ce2e0 2024-04-25T08:34:45.177 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:45.177 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":53,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:34:44.235150+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:34:42.507595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:34:43.209205+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:34:45.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.184+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3db006b4d0 msgr2=0x7f3db006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.184+0000 7f3dd4e4a640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3db006b4d0 0x7f3db006d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0034000 tx=0x7f3dc002e000 comp rx=0 tx=0).stop 2024-04-25T08:34:45.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.184+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/2471106610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dd00ce2e0 msgr2=0x7f3dd01f0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.184+0000 7f3dd4e4a640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dd00ce2e0 0x7f3dd01f0e40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f3db800c430 tx=0x7f3db800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.184+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/2471106610 shutdown_connections 2024-04-25T08:34:45.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.184+0000 7f3dd4e4a640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dd015df50 0x7f3dd00c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.184+0000 7f3dd4e4a640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dd00ce2e0 0x7f3dd01f0e40 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.184+0000 7f3dd4e4a640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3db006b4d0 0x7f3db006d990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.184+0000 7f3dd4e4a640 1 --2- 172.21.15.38:0/2471106610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dd006c110 0x7f3dd01f4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.184+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/2471106610 >> 172.21.15.38:0/2471106610 conn(0x7f3dd00d3fb0 msgr2=0x7f3dd00b6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:45.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.184+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/2471106610 shutdown_connections 2024-04-25T08:34:45.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.184+0000 7f3dd4e4a640 1 -- 172.21.15.38:0/2471106610 wait complete. 2024-04-25T08:34:45.191 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:34:45.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.296+0000 7f242728f640 1 Processor -- start 2024-04-25T08:34:45.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.296+0000 7f242728f640 1 -- start start 2024-04-25T08:34:45.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.297+0000 7f242728f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f242015bb10 0x7f242009a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.297+0000 7f242728f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24200d0460 0x7f242009a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.297+0000 7f242728f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f242015f4e0 0x7f242015f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.297+0000 7f242728f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24200cffd0 con 0x7f242015f4e0 2024-04-25T08:34:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.297+0000 7f242728f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2420160340 con 0x7f242015bb10 2024-04-25T08:34:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.297+0000 7f242728f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24200cf400 con 0x7f24200d0460 2024-04-25T08:34:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.297+0000 7f242582c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f242015f4e0 0x7f242015f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.297+0000 7f242582c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f242015f4e0 0x7f242015f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50952/0 (socket says 172.21.15.38:50952) 2024-04-25T08:34:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.297+0000 7f242582c640 1 -- 172.21.15.38:0/1289258233 learned_addr learned my addr 172.21.15.38:0/1289258233 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.298+0000 7f242482a640 1 --2- 172.21.15.38:0/1289258233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24200d0460 0x7f242009a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.298+0000 7f242502b640 1 --2- 172.21.15.38:0/1289258233 >> v2:172.21.15.107:3300/0 conn(0x7f242015bb10 0x7f242009a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.298+0000 7f242582c640 1 -- 172.21.15.38:0/1289258233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24200d0460 msgr2=0x7f242009a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.298+0000 7f242582c640 1 --2- 172.21.15.38:0/1289258233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24200d0460 0x7f242009a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.298+0000 7f242582c640 1 -- 172.21.15.38:0/1289258233 >> v2:172.21.15.107:3300/0 conn(0x7f242015bb10 msgr2=0x7f242009a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.298+0000 7f242582c640 1 --2- 172.21.15.38:0/1289258233 >> v2:172.21.15.107:3300/0 conn(0x7f242015bb10 0x7f242009a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.298+0000 7f242582c640 1 -- 172.21.15.38:0/1289258233 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24200cf6a0 con 0x7f242015f4e0 2024-04-25T08:34:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.298+0000 7f242482a640 1 --2- 172.21.15.38:0/1289258233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24200d0460 0x7f242009a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.298+0000 7f242502b640 1 --2- 172.21.15.38:0/1289258233 >> v2:172.21.15.107:3300/0 conn(0x7f242015bb10 0x7f242009a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:45.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.298+0000 7f242582c640 1 --2- 172.21.15.38:0/1289258233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f242015f4e0 0x7f242015f970 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f24200cf280 tx=0x7f241802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebac8e716d00ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.299+0000 7f241ffff640 1 -- 172.21.15.38:0/1289258233 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418037070 con 0x7f242015f4e0 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.299+0000 7f241ffff640 1 -- 172.21.15.38:0/1289258233 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418032440 con 0x7f242015f4e0 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.299+0000 7f241ffff640 1 -- 172.21.15.38:0/1289258233 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418032810 con 0x7f242015f4e0 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.299+0000 7f242728f640 1 -- 172.21.15.38:0/1289258233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f242015f4e0 msgr2=0x7f242015f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.299+0000 7f242728f640 1 --2- 172.21.15.38:0/1289258233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f242015f4e0 0x7f242015f970 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f24200cf280 tx=0x7f241802a140 comp rx=0 tx=0).stop 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.299+0000 7f242728f640 1 -- 172.21.15.38:0/1289258233 shutdown_connections 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.299+0000 7f242728f640 1 --2- 172.21.15.38:0/1289258233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f242015f4e0 0x7f242015f970 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.299+0000 7f242728f640 1 --2- 172.21.15.38:0/1289258233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24200d0460 0x7f242009a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.299+0000 7f242728f640 1 --2- 172.21.15.38:0/1289258233 >> v2:172.21.15.107:3300/0 conn(0x7f242015bb10 0x7f242009a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.299+0000 7f242728f640 1 -- 172.21.15.38:0/1289258233 >> 172.21.15.38:0/1289258233 conn(0x7f24200b31f0 msgr2=0x7f24200b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.299+0000 7f242728f640 1 -- 172.21.15.38:0/1289258233 shutdown_connections 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.299+0000 7f242728f640 1 -- 172.21.15.38:0/1289258233 wait complete. 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.300+0000 7f242728f640 1 Processor -- start 2024-04-25T08:34:45.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.300+0000 7f242728f640 1 -- start start 2024-04-25T08:34:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.301+0000 7f242728f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24200d0460 0x7f242014cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.301+0000 7f242728f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f242015bb10 0x7f242015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.301+0000 7f242728f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f242015f4e0 0x7f242015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.301+0000 7f242728f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24200c5250 con 0x7f242015f4e0 2024-04-25T08:34:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.301+0000 7f242728f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24200cfc60 con 0x7f242015bb10 2024-04-25T08:34:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.301+0000 7f242728f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24200c53d0 con 0x7f24200d0460 2024-04-25T08:34:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.301+0000 7f242582c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f242015f4e0 0x7f242015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.301+0000 7f242582c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f242015f4e0 0x7f242015c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50960/0 (socket says 172.21.15.38:50960) 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.301+0000 7f242582c640 1 -- 172.21.15.38:0/487276233 learned_addr learned my addr 172.21.15.38:0/487276233 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.301+0000 7f242482a640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f242015bb10 0x7f242015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.301+0000 7f242502b640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24200d0460 0x7f242014cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f242582c640 1 -- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24200d0460 msgr2=0x7f242014cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f242582c640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24200d0460 0x7f242014cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f242582c640 1 -- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f242015bb10 msgr2=0x7f242015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f242582c640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f242015bb10 0x7f242015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f242582c640 1 -- 172.21.15.38:0/487276233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242015cd30 con 0x7f242015f4e0 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f242482a640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f242015bb10 0x7f242015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f242502b640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24200d0460 0x7f242014cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f242582c640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f242015f4e0 0x7f242015c6d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f24180090b0 tx=0x7f24180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f241dffb640 1 -- 172.21.15.38:0/487276233 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418041020 con 0x7f242015f4e0 2024-04-25T08:34:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f241dffb640 1 -- 172.21.15.38:0/487276233 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2418030990 con 0x7f242015f4e0 2024-04-25T08:34:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f241dffb640 1 -- 172.21.15.38:0/487276233 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2418037040 con 0x7f242015f4e0 2024-04-25T08:34:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f242728f640 1 -- 172.21.15.38:0/487276233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242015d010 con 0x7f242015f4e0 2024-04-25T08:34:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.302+0000 7f242728f640 1 -- 172.21.15.38:0/487276233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242015d5d0 con 0x7f242015f4e0 2024-04-25T08:34:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.303+0000 7f242728f640 1 -- 172.21.15.38:0/487276233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2420068e80 con 0x7f242015f4e0 2024-04-25T08:34:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.304+0000 7f241dffb640 1 -- 172.21.15.38:0/487276233 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2418044020 con 0x7f242015f4e0 2024-04-25T08:34:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.304+0000 7f241dffb640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f240c06b4d0 0x7f240c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.304+0000 7f241dffb640 1 -- 172.21.15.38:0/487276233 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f24180aad30 con 0x7f242015f4e0 2024-04-25T08:34:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.304+0000 7f242502b640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f240c06b4d0 0x7f240c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.305+0000 7f242502b640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f240c06b4d0 0x7f240c06d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f241000d430 tx=0x7f2410007530 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:45.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.308+0000 7f241dffb640 1 -- 172.21.15.38:0/487276233 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f241804b030 con 0x7f242015f4e0 2024-04-25T08:34:45.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.492+0000 7f242728f640 1 -- 172.21.15.38:0/487276233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24200cdbf0 con 0x7f242015f4e0 2024-04-25T08:34:45.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.493+0000 7f241dffb640 1 -- 172.21.15.38:0/487276233 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f2418079a20 con 0x7f242015f4e0 2024-04-25T08:34:45.489 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:45.489 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":53,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:34:44.235150+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:34:42.507595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:34:43.209205+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:34:45.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.495+0000 7f242728f640 1 -- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f240c06b4d0 msgr2=0x7f240c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.495+0000 7f242728f640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f240c06b4d0 0x7f240c06d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f241000d430 tx=0x7f2410007530 comp rx=0 tx=0).stop 2024-04-25T08:34:45.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.495+0000 7f242728f640 1 -- 172.21.15.38:0/487276233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f242015f4e0 msgr2=0x7f242015c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.495+0000 7f242728f640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f242015f4e0 0x7f242015c6d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f24180090b0 tx=0x7f24180046c0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.495+0000 7f242728f640 1 -- 172.21.15.38:0/487276233 shutdown_connections 2024-04-25T08:34:45.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.495+0000 7f242728f640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f242015f4e0 0x7f242015c6d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.495+0000 7f242728f640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f242015bb10 0x7f242015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.495+0000 7f242728f640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f240c06b4d0 0x7f240c06d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.495+0000 7f242728f640 1 --2- 172.21.15.38:0/487276233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24200d0460 0x7f242014cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.495+0000 7f242728f640 1 -- 172.21.15.38:0/487276233 >> 172.21.15.38:0/487276233 conn(0x7f24200b31f0 msgr2=0x7f2420159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:45.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.496+0000 7f242728f640 1 -- 172.21.15.38:0/487276233 shutdown_connections 2024-04-25T08:34:45.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.496+0000 7f242728f640 1 -- 172.21.15.38:0/487276233 wait complete. 2024-04-25T08:34:45.502 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:34:45.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.611+0000 7feffcbc4640 1 Processor -- start 2024-04-25T08:34:45.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.612+0000 7feffcbc4640 1 -- start start 2024-04-25T08:34:45.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.612+0000 7feffcbc4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7feff80ccfc0 0x7feff80ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.612+0000 7feffcbc4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff80ac430 con 0x7feff815db80 2024-04-25T08:34:45.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.612+0000 7feffcbc4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff80ac710 con 0x7feff80cfd40 2024-04-25T08:34:45.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.612+0000 7feffcbc4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff80ac1e0 con 0x7feff80ccfc0 2024-04-25T08:34:45.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff659c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7feff815db80 0x7feff80cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33250/0 (socket says 172.21.15.38:33250) 2024-04-25T08:34:45.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff659c640 1 -- 172.21.15.38:0/1997782055 learned_addr learned my addr 172.21.15.38:0/1997782055 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:45.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff6d9d640 1 --2- 172.21.15.38:0/1997782055 >> v2:172.21.15.107:3300/0 conn(0x7feff80ccfc0 0x7feff80ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654353577 0 0) 0x7feff80ac430 con 0x7feff815db80 2024-04-25T08:34:45.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefe4003520 con 0x7feff815db80 2024-04-25T08:34:45.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3704436220 0 0) 0x7fefe4003520 con 0x7feff815db80 2024-04-25T08:34:45.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 >> v2:172.21.15.107:3300/0 conn(0x7feff80ccfc0 msgr2=0x7feff80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:45.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 --2- 172.21.15.38:0/1997782055 >> v2:172.21.15.107:3300/0 conn(0x7feff80ccfc0 0x7feff80ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 >> v1:172.21.15.107:6789/0 conn(0x7feff80cfd40 legacy=0x7feff80c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff80ac9f0 con 0x7feff815db80 2024-04-25T08:34:45.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fefe80019f0 con 0x7feff815db80 2024-04-25T08:34:45.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 >> v1:172.21.15.38:6789/0 conn(0x7feff815db80 legacy=0x7feff80cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 --2- 172.21.15.38:0/1997782055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefe40039b0 0x7fefe4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 --2- 172.21.15.38:0/1997782055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefe4004800 0x7fefe4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 --2- 172.21.15.38:0/1997782055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 0x7fefe400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff80ac1e0 con 0x7fefe40051c0 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe400c060 con 0x7fefe4004800 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe400c260 con 0x7fefe40039b0 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff6d9d640 1 --2- 172.21.15.38:0/1997782055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 0x7fefe400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff659c640 1 --2- 172.21.15.38:0/1997782055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefe40039b0 0x7fefe4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.613+0000 7feff5d9b640 1 --2- 172.21.15.38:0/1997782055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefe4004800 0x7fefe4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.614+0000 7feff6d9d640 1 -- 172.21.15.38:0/1997782055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefe40039b0 msgr2=0x7fefe4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.614+0000 7feff6d9d640 1 --2- 172.21.15.38:0/1997782055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefe40039b0 0x7fefe4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.614+0000 7feff6d9d640 1 -- 172.21.15.38:0/1997782055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefe4004800 msgr2=0x7fefe4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.614+0000 7feff6d9d640 1 --2- 172.21.15.38:0/1997782055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fefe4004800 0x7fefe4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.614+0000 7feff6d9d640 1 -- 172.21.15.38:0/1997782055 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe4003520 con 0x7fefe40051c0 2024-04-25T08:34:45.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.614+0000 7feff659c640 1 --2- 172.21.15.38:0/1997782055 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fefe40039b0 0x7fefe4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:45.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.614+0000 7feff6d9d640 1 -- 172.21.15.38:0/1997782055 reap_dead start 2024-04-25T08:34:45.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.614+0000 7feff6d9d640 1 --2- 172.21.15.38:0/1997782055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 0x7fefe400b9c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7feff0009ac0 tx=0x7fefe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263aa30f0e6479d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:45.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.614+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0009690 con 0x7fefe40051c0 2024-04-25T08:34:45.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.615+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff0004020 con 0x7fefe40051c0 2024-04-25T08:34:45.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.615+0000 7feff559a640 1 -- 172.21.15.38:0/1997782055 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0004410 con 0x7fefe40051c0 2024-04-25T08:34:45.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.615+0000 7feffcbc4640 1 -- 172.21.15.38:0/1997782055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 msgr2=0x7fefe400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.615+0000 7feffcbc4640 1 --2- 172.21.15.38:0/1997782055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 0x7fefe400b9c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7feff0009ac0 tx=0x7fefe4005830 comp rx=0 tx=0).stop 2024-04-25T08:34:45.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.615+0000 7feffcbc4640 1 -- 172.21.15.38:0/1997782055 shutdown_connections 2024-04-25T08:34:45.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.615+0000 7feffcbc4640 1 --2- 172.21.15.38:0/1997782055 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 0x7fefe400b9c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.615+0000 7feffcbc4640 1 -- 172.21.15.38:0/1997782055 >> 172.21.15.38:0/1997782055 conn(0x7feff806adc0 msgr2=0x7feff806b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:45.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.615+0000 7feffcbc4640 1 -- 172.21.15.38:0/1997782055 shutdown_connections 2024-04-25T08:34:45.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.615+0000 7feffcbc4640 1 -- 172.21.15.38:0/1997782055 wait complete. 2024-04-25T08:34:45.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.616+0000 7feffcbc4640 1 Processor -- start 2024-04-25T08:34:45.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.616+0000 7feffcbc4640 1 -- start start 2024-04-25T08:34:45.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.617+0000 7feffcbc4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 0x7feff80c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.617+0000 7feffcbc4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feff81ef1a0 0x7feff80c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.617+0000 7feffcbc4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feff80cbe10 0x7feff80cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.617+0000 7feffcbc4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff80cf630 con 0x7fefe40051c0 2024-04-25T08:34:45.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.617+0000 7feffcbc4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff815df60 con 0x7feff80cbe10 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.617+0000 7feffcbc4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff80cf7b0 con 0x7feff81ef1a0 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.617+0000 7feff659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 0x7feff80c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.617+0000 7feff659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 0x7feff80c5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50976/0 (socket says 172.21.15.38:50976) 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.617+0000 7feff659c640 1 -- 172.21.15.38:0/3266024999 learned_addr learned my addr 172.21.15.38:0/3266024999 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.617+0000 7feff5d9b640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feff81ef1a0 0x7feff80c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.617+0000 7feff6d9d640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feff80cbe10 0x7feff80cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.618+0000 7feff659c640 1 -- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feff81ef1a0 msgr2=0x7feff80c8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.618+0000 7feff659c640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feff81ef1a0 0x7feff80c8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.618+0000 7feff659c640 1 -- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feff80cbe10 msgr2=0x7feff80cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.618+0000 7feff659c640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feff80cbe10 0x7feff80cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.618+0000 7feff659c640 1 -- 172.21.15.38:0/3266024999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff81ef6f0 con 0x7fefe40051c0 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.618+0000 7feff659c640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 0x7feff80c5110 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fefe8001370 tx=0x7fefe800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.618+0000 7fefeeffd640 1 -- 172.21.15.38:0/3266024999 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe800edf0 con 0x7fefe40051c0 2024-04-25T08:34:45.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.618+0000 7fefeeffd640 1 -- 172.21.15.38:0/3266024999 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe8018070 con 0x7fefe40051c0 2024-04-25T08:34:45.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.618+0000 7feffcbc4640 1 -- 172.21.15.38:0/3266024999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff81ef9d0 con 0x7fefe40051c0 2024-04-25T08:34:45.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.618+0000 7fefeeffd640 1 -- 172.21.15.38:0/3266024999 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe8013620 con 0x7fefe40051c0 2024-04-25T08:34:45.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.618+0000 7feffcbc4640 1 -- 172.21.15.38:0/3266024999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff81eff90 con 0x7fefe40051c0 2024-04-25T08:34:45.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.619+0000 7feffcbc4640 1 -- 172.21.15.38:0/3266024999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff8068e80 con 0x7fefe40051c0 2024-04-25T08:34:45.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.620+0000 7fefeeffd640 1 -- 172.21.15.38:0/3266024999 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefe80137c0 con 0x7fefe40051c0 2024-04-25T08:34:45.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.620+0000 7fefeeffd640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fefd806b500 0x7fefd806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.620+0000 7fefeeffd640 1 -- 172.21.15.38:0/3266024999 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fefe808a760 con 0x7fefe40051c0 2024-04-25T08:34:45.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.620+0000 7feff5d9b640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fefd806b500 0x7fefd806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.621+0000 7feff5d9b640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fefd806b500 0x7fefd806d9c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fefe0000c00 tx=0x7fefe0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:45.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.624+0000 7fefeeffd640 1 -- 172.21.15.38:0/3266024999 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fefe8058e00 con 0x7fefe40051c0 2024-04-25T08:34:45.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.844+0000 7feffcbc4640 1 -- 172.21.15.38:0/3266024999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feff80b4230 con 0x7fefe40051c0 2024-04-25T08:34:45.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.874+0000 7fefeeffd640 1 -- 172.21.15.38:0/3266024999 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fefe805cea0 con 0x7fefe40051c0 2024-04-25T08:34:45.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.876+0000 7feffcbc4640 1 -- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fefd806b500 msgr2=0x7fefd806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.876+0000 7feffcbc4640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fefd806b500 0x7fefd806d9c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fefe0000c00 tx=0x7fefe0009040 comp rx=0 tx=0).stop 2024-04-25T08:34:45.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.876+0000 7feffcbc4640 1 -- 172.21.15.38:0/3266024999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 msgr2=0x7feff80c5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.876+0000 7feffcbc4640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 0x7feff80c5110 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fefe8001370 tx=0x7fefe800e530 comp rx=0 tx=0).stop 2024-04-25T08:34:45.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.876+0000 7feffcbc4640 1 -- 172.21.15.38:0/3266024999 shutdown_connections 2024-04-25T08:34:45.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.876+0000 7feffcbc4640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feff80cbe10 0x7feff80cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.876+0000 7feffcbc4640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fefd806b500 0x7fefd806d9c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.876+0000 7feffcbc4640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feff81ef1a0 0x7feff80c8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.876+0000 7feffcbc4640 1 --2- 172.21.15.38:0/3266024999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefe40051c0 0x7feff80c5110 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.876+0000 7feffcbc4640 1 -- 172.21.15.38:0/3266024999 >> 172.21.15.38:0/3266024999 conn(0x7feff806adc0 msgr2=0x7feff80c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:45.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.876+0000 7feffcbc4640 1 -- 172.21.15.38:0/3266024999 shutdown_connections 2024-04-25T08:34:45.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.876+0000 7feffcbc4640 1 -- 172.21.15.38:0/3266024999 wait complete. 2024-04-25T08:34:45.882 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:34:45.882 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:34:45.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.990+0000 7fb716789640 1 Processor -- start 2024-04-25T08:34:45.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.990+0000 7fb716789640 1 -- start start 2024-04-25T08:34:45.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.991+0000 7fb716789640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb7100d0320 0x7fb7100d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.991+0000 7fb716789640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb7100d0700 0x7fb7100d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.991+0000 7fb716789640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7100d2330 con 0x7fb7100d1e90 2024-04-25T08:34:45.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.991+0000 7fb716789640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7100d2610 con 0x7fb7100d0320 2024-04-25T08:34:45.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.991+0000 7fb716789640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7100d28f0 con 0x7fb7100d0700 2024-04-25T08:34:45.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.991+0000 7fb70ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb7100d0320 0x7fb7100d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.991+0000 7fb714d26640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb7100d1e90 0x7fb7100b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33264/0 (socket says 172.21.15.38:33264) 2024-04-25T08:34:45.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.991+0000 7fb714d26640 1 -- 172.21.15.38:0/1805157155 learned_addr learned my addr 172.21.15.38:0/1805157155 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:45.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.991+0000 7fb70f7fe640 1 --2- 172.21.15.38:0/1805157155 >> v2:172.21.15.107:3300/0 conn(0x7fb7100d0700 0x7fb7100d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.992+0000 7fb70ffff640 1 -- 172.21.15.38:0/1805157155 >> v2:172.21.15.107:3300/0 conn(0x7fb7100d0700 msgr2=0x7fb7100d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:45.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.992+0000 7fb70ffff640 1 --2- 172.21.15.38:0/1805157155 >> v2:172.21.15.107:3300/0 conn(0x7fb7100d0700 0x7fb7100d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.992+0000 7fb70ffff640 1 -- 172.21.15.38:0/1805157155 >> v1:172.21.15.38:6789/0 conn(0x7fb7100d1e90 legacy=0x7fb7100b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.992+0000 7fb70effd640 1 -- 172.21.15.38:0/1805157155 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232449262 0 0) 0x7fb7100d2330 con 0x7fb7100d1e90 2024-04-25T08:34:45.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.992+0000 7fb70ffff640 1 -- 172.21.15.38:0/1805157155 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7100d2bd0 con 0x7fb7100d0320 2024-04-25T08:34:45.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.992+0000 7fb70ffff640 1 --2- 172.21.15.38:0/1805157155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7100d0320 0x7fb7100d33d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb7100b6410 tx=0x7fb70402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ceed09551c6941c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:45.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.992+0000 7fb70effd640 1 -- 172.21.15.38:0/1805157155 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704037070 con 0x7fb7100d0320 2024-04-25T08:34:45.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.992+0000 7fb70effd640 1 -- 172.21.15.38:0/1805157155 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb704032440 con 0x7fb7100d0320 2024-04-25T08:34:45.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.992+0000 7fb70effd640 1 -- 172.21.15.38:0/1805157155 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704032810 con 0x7fb7100d0320 2024-04-25T08:34:45.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.993+0000 7fb716789640 1 -- 172.21.15.38:0/1805157155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7100d0320 msgr2=0x7fb7100d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.993+0000 7fb716789640 1 --2- 172.21.15.38:0/1805157155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7100d0320 0x7fb7100d33d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb7100b6410 tx=0x7fb70402a230 comp rx=0 tx=0).stop 2024-04-25T08:34:45.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.993+0000 7fb716789640 1 -- 172.21.15.38:0/1805157155 shutdown_connections 2024-04-25T08:34:45.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.993+0000 7fb716789640 1 --2- 172.21.15.38:0/1805157155 >> v2:172.21.15.107:3300/0 conn(0x7fb7100d0700 0x7fb7100d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.993+0000 7fb716789640 1 --2- 172.21.15.38:0/1805157155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7100d0320 0x7fb7100d33d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.993+0000 7fb716789640 1 -- 172.21.15.38:0/1805157155 >> 172.21.15.38:0/1805157155 conn(0x7fb7100668f0 msgr2=0x7fb710066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:45.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.993+0000 7fb716789640 1 -- 172.21.15.38:0/1805157155 shutdown_connections 2024-04-25T08:34:45.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.993+0000 7fb716789640 1 -- 172.21.15.38:0/1805157155 wait complete. 2024-04-25T08:34:45.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.994+0000 7fb716789640 1 Processor -- start 2024-04-25T08:34:45.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.994+0000 7fb716789640 1 -- start start 2024-04-25T08:34:45.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.995+0000 7fb716789640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7100d0320 0x7fb7101f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.995+0000 7fb716789640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7100d0700 0x7fb710155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.995+0000 7fb716789640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7100d1e90 0x7fb7100d37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.995+0000 7fb716789640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7100d0b40 con 0x7fb7100d1e90 2024-04-25T08:34:45.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.995+0000 7fb716789640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb710161210 con 0x7fb7100d0700 2024-04-25T08:34:45.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.995+0000 7fb716789640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb710156850 con 0x7fb7100d0320 2024-04-25T08:34:45.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.995+0000 7fb714d26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7100d1e90 0x7fb7100d37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.995+0000 7fb714d26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7100d1e90 0x7fb7100d37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51004/0 (socket says 172.21.15.38:51004) 2024-04-25T08:34:45.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.995+0000 7fb714d26640 1 -- 172.21.15.38:0/821121956 learned_addr learned my addr 172.21.15.38:0/821121956 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:45.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.995+0000 7fb70f7fe640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7100d0700 0x7fb710155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.995+0000 7fb70ffff640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7100d0320 0x7fb7101f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.996+0000 7fb714d26640 1 -- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7100d0320 msgr2=0x7fb7101f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.996+0000 7fb714d26640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7100d0320 0x7fb7101f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.996+0000 7fb714d26640 1 -- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7100d0700 msgr2=0x7fb710155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:45.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.996+0000 7fb714d26640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7100d0700 0x7fb710155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:45.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.996+0000 7fb714d26640 1 -- 172.21.15.38:0/821121956 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb710156b30 con 0x7fb7100d1e90 2024-04-25T08:34:45.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.996+0000 7fb714d26640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7100d1e90 0x7fb7100d37a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c700 tx=0x7fb70800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:45.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.996+0000 7fb70cff9640 1 -- 172.21.15.38:0/821121956 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708018070 con 0x7fb7100d1e90 2024-04-25T08:34:45.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.996+0000 7fb70cff9640 1 -- 172.21.15.38:0/821121956 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70800ce10 con 0x7fb7100d1e90 2024-04-25T08:34:45.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.996+0000 7fb716789640 1 -- 172.21.15.38:0/821121956 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb710156e10 con 0x7fb7100d1e90 2024-04-25T08:34:45.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.996+0000 7fb70cff9640 1 -- 172.21.15.38:0/821121956 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70800a860 con 0x7fb7100d1e90 2024-04-25T08:34:45.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.996+0000 7fb716789640 1 -- 172.21.15.38:0/821121956 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7100b3500 con 0x7fb7100d1e90 2024-04-25T08:34:45.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.997+0000 7fb716789640 1 -- 172.21.15.38:0/821121956 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb710068e80 con 0x7fb7100d1e90 2024-04-25T08:34:45.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.997+0000 7fb70cff9640 1 -- 172.21.15.38:0/821121956 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb70801b070 con 0x7fb7100d1e90 2024-04-25T08:34:45.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.998+0000 7fb70cff9640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb6fc06b4d0 0x7fb6fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:45.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.998+0000 7fb70cff9640 1 -- 172.21.15.38:0/821121956 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb7080898f0 con 0x7fb7100d1e90 2024-04-25T08:34:45.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.998+0000 7fb70ffff640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb6fc06b4d0 0x7fb6fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:45.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:45.999+0000 7fb70ffff640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb6fc06b4d0 0x7fb6fc06d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb704034000 tx=0x7fb70402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:45.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.002+0000 7fb70cff9640 1 -- 172.21.15.38:0/821121956 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb708057ee0 con 0x7fb7100d1e90 2024-04-25T08:34:46.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.221+0000 7fb716789640 1 -- 172.21.15.38:0/821121956 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb7100cf5f0 con 0x7fb7100d1e90 2024-04-25T08:34:46.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.879+0000 7fb70cff9640 1 -- 172.21.15.38:0/821121956 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb70805bf80 con 0x7fb7100d1e90 2024-04-25T08:34:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.882+0000 7fb716789640 1 -- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb6fc06b4d0 msgr2=0x7fb6fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.882+0000 7fb716789640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb6fc06b4d0 0x7fb6fc06d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb704034000 tx=0x7fb70402e000 comp rx=0 tx=0).stop 2024-04-25T08:34:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.882+0000 7fb716789640 1 -- 172.21.15.38:0/821121956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7100d1e90 msgr2=0x7fb7100d37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.882+0000 7fb716789640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7100d1e90 0x7fb7100d37a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c700 tx=0x7fb70800cbc0 comp rx=0 tx=0).stop 2024-04-25T08:34:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.882+0000 7fb716789640 1 -- 172.21.15.38:0/821121956 shutdown_connections 2024-04-25T08:34:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.882+0000 7fb716789640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7100d1e90 0x7fb7100d37a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:46.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.882+0000 7fb716789640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7100d0700 0x7fb710155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.882+0000 7fb716789640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb6fc06b4d0 0x7fb6fc06d990 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.882+0000 7fb716789640 1 --2- 172.21.15.38:0/821121956 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7100d0320 0x7fb7101f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.882+0000 7fb716789640 1 -- 172.21.15.38:0/821121956 >> 172.21.15.38:0/821121956 conn(0x7fb7100668f0 msgr2=0x7fb7100cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.882+0000 7fb716789640 1 -- 172.21.15.38:0/821121956 shutdown_connections 2024-04-25T08:34:46.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:46.882+0000 7fb716789640 1 -- 172.21.15.38:0/821121956 wait complete. 2024-04-25T08:34:46.891 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:34:46.891 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:34:46.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.003+0000 7f27564b3640 1 Processor -- start 2024-04-25T08:34:46.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.003+0000 7f27564b3640 1 -- start start 2024-04-25T08:34:46.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f27564b3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f27500b5710 0x7f2750161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:46.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f27564b3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f275015ddf0 0x7f27500b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:46.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f27564b3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2750161e00 con 0x7f27500b5af0 2024-04-25T08:34:47.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f27564b3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27500b9930 con 0x7f27500b5710 2024-04-25T08:34:47.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f27564b3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27501620e0 con 0x7f275015ddf0 2024-04-25T08:34:47.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f274ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f27500b5710 0x7f2750161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f274ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f27500b5710 0x7f2750161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51006/0 (socket says 172.21.15.38:51006) 2024-04-25T08:34:47.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f274ffff640 1 -- 172.21.15.38:0/828318888 learned_addr learned my addr 172.21.15.38:0/828318888 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:47.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f274f7fe640 1 --2- 172.21.15.38:0/828318888 >> v2:172.21.15.107:3300/0 conn(0x7f275015ddf0 0x7f27500b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f274ffff640 1 -- 172.21.15.38:0/828318888 >> v2:172.21.15.107:3300/0 conn(0x7f275015ddf0 msgr2=0x7f27500b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:47.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f274ffff640 1 --2- 172.21.15.38:0/828318888 >> v2:172.21.15.107:3300/0 conn(0x7f275015ddf0 0x7f27500b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f274ffff640 1 -- 172.21.15.38:0/828318888 >> v1:172.21.15.107:6789/0 conn(0x7f27500b5af0 legacy=0x7f275006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:47.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.004+0000 7f274ffff640 1 -- 172.21.15.38:0/828318888 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27501623c0 con 0x7f27500b5710 2024-04-25T08:34:47.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.005+0000 7f274ffff640 1 --2- 172.21.15.38:0/828318888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27500b5710 0x7f2750161700 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f27500d0300 tx=0x7f274402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ec20e3ff19fe59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:47.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.005+0000 7f274effd640 1 -- 172.21.15.38:0/828318888 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744037070 con 0x7f27500b5710 2024-04-25T08:34:47.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.005+0000 7f274effd640 1 -- 172.21.15.38:0/828318888 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2744032440 con 0x7f27500b5710 2024-04-25T08:34:47.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.005+0000 7f274effd640 1 -- 172.21.15.38:0/828318888 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744032810 con 0x7f27500b5710 2024-04-25T08:34:47.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.006+0000 7f27564b3640 1 -- 172.21.15.38:0/828318888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27500b5710 msgr2=0x7f2750161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.006+0000 7f27564b3640 1 --2- 172.21.15.38:0/828318888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27500b5710 0x7f2750161700 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f27500d0300 tx=0x7f274402a270 comp rx=0 tx=0).stop 2024-04-25T08:34:47.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.006+0000 7f27564b3640 1 -- 172.21.15.38:0/828318888 shutdown_connections 2024-04-25T08:34:47.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.006+0000 7f27564b3640 1 --2- 172.21.15.38:0/828318888 >> v2:172.21.15.107:3300/0 conn(0x7f275015ddf0 0x7f27500b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.006+0000 7f27564b3640 1 --2- 172.21.15.38:0/828318888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27500b5710 0x7f2750161700 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.006+0000 7f27564b3640 1 -- 172.21.15.38:0/828318888 >> 172.21.15.38:0/828318888 conn(0x7f27500d07c0 msgr2=0x7f27500d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:47.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.006+0000 7f27564b3640 1 -- 172.21.15.38:0/828318888 shutdown_connections 2024-04-25T08:34:47.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.006+0000 7f27564b3640 1 -- 172.21.15.38:0/828318888 wait complete. 2024-04-25T08:34:47.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.007+0000 7f27564b3640 1 Processor -- start 2024-04-25T08:34:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.007+0000 7f27564b3640 1 -- start start 2024-04-25T08:34:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f27564b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27500b5710 0x7f275015f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f27564b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27500b5af0 0x7f27500c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f27564b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f275015ddf0 0x7f27500c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f27564b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27500d34d0 con 0x7f27500b5af0 2024-04-25T08:34:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f27564b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27500d3350 con 0x7f275015ddf0 2024-04-25T08:34:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f27564b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27500d3650 con 0x7f27500b5710 2024-04-25T08:34:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27500b5af0 0x7f27500c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27500b5af0 0x7f27500c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51020/0 (socket says 172.21.15.38:51020) 2024-04-25T08:34:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f274f7fe640 1 -- 172.21.15.38:0/986718451 learned_addr learned my addr 172.21.15.38:0/986718451 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:47.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f2754a50640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f275015ddf0 0x7f27500c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f274ffff640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27500b5710 0x7f275015f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f274f7fe640 1 -- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27500b5710 msgr2=0x7f275015f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f274f7fe640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27500b5710 0x7f275015f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f274f7fe640 1 -- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f275015ddf0 msgr2=0x7f27500c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f274f7fe640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f275015ddf0 0x7f27500c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f274f7fe640 1 -- 172.21.15.38:0/986718451 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27500ca280 con 0x7f27500b5af0 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f2754a50640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f275015ddf0 0x7f27500c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.008+0000 7f274ffff640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27500b5710 0x7f275015f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.009+0000 7f274f7fe640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27500b5af0 0x7f27500c9800 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f274000d5c0 tx=0x7f274000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.009+0000 7f274cff9640 1 -- 172.21.15.38:0/986718451 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27400125f0 con 0x7f27500b5af0 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.009+0000 7f274cff9640 1 -- 172.21.15.38:0/986718451 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2740019070 con 0x7f27500b5af0 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.009+0000 7f27564b3640 1 -- 172.21.15.38:0/986718451 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275015fd20 con 0x7f27500b5af0 2024-04-25T08:34:47.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.009+0000 7f274cff9640 1 -- 172.21.15.38:0/986718451 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27400129d0 con 0x7f27500b5af0 2024-04-25T08:34:47.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.009+0000 7f27564b3640 1 -- 172.21.15.38:0/986718451 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27501602e0 con 0x7f27500b5af0 2024-04-25T08:34:47.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.010+0000 7f27564b3640 1 -- 172.21.15.38:0/986718451 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2750068e80 con 0x7f27500b5af0 2024-04-25T08:34:47.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.010+0000 7f274cff9640 1 -- 172.21.15.38:0/986718451 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2740012b70 con 0x7f27500b5af0 2024-04-25T08:34:47.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.011+0000 7f274cff9640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f273c06b4d0 0x7f273c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.011+0000 7f274cff9640 1 -- 172.21.15.38:0/986718451 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f274008a7f0 con 0x7f27500b5af0 2024-04-25T08:34:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.011+0000 7f274ffff640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f273c06b4d0 0x7f273c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.011+0000 7f274ffff640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f273c06b4d0 0x7f273c06d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2744034000 tx=0x7f274402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:47.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.015+0000 7f274cff9640 1 -- 172.21.15.38:0/986718451 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2740059450 con 0x7f27500b5af0 2024-04-25T08:34:47.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.231+0000 7f27564b3640 1 -- 172.21.15.38:0/986718451 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2750073c00 con 0x7f27500b5af0 2024-04-25T08:34:47.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.232+0000 7f274cff9640 1 -- 172.21.15.38:0/986718451 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f2740058d50 con 0x7f27500b5af0 2024-04-25T08:34:47.227 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:47.227 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:34:46.222808+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:34:47.227 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 24 2024-04-25T08:34:47.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.234+0000 7f27564b3640 1 -- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f273c06b4d0 msgr2=0x7f273c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.234+0000 7f27564b3640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f273c06b4d0 0x7f273c06d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2744034000 tx=0x7f274402e000 comp rx=0 tx=0).stop 2024-04-25T08:34:47.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.234+0000 7f27564b3640 1 -- 172.21.15.38:0/986718451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27500b5af0 msgr2=0x7f27500c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.234+0000 7f27564b3640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27500b5af0 0x7f27500c9800 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f274000d5c0 tx=0x7f274000da80 comp rx=0 tx=0).stop 2024-04-25T08:34:47.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.234+0000 7f27564b3640 1 -- 172.21.15.38:0/986718451 shutdown_connections 2024-04-25T08:34:47.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.234+0000 7f27564b3640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f275015ddf0 0x7f27500c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.234+0000 7f27564b3640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27500b5af0 0x7f27500c9800 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.234+0000 7f27564b3640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f273c06b4d0 0x7f273c06d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.234+0000 7f27564b3640 1 --2- 172.21.15.38:0/986718451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f27500b5710 0x7f275015f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.234+0000 7f27564b3640 1 -- 172.21.15.38:0/986718451 >> 172.21.15.38:0/986718451 conn(0x7f27500d07c0 msgr2=0x7f27500d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:47.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.234+0000 7f27564b3640 1 -- 172.21.15.38:0/986718451 shutdown_connections 2024-04-25T08:34:47.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.234+0000 7f27564b3640 1 -- 172.21.15.38:0/986718451 wait complete. 2024-04-25T08:34:47.240 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:34:47.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.346+0000 7fec2cbf5640 1 Processor -- start 2024-04-25T08:34:47.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.346+0000 7fec2cbf5640 1 -- start start 2024-04-25T08:34:47.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.347+0000 7fec2cbf5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fec280b5710 0x7fec28161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.347+0000 7fec2cbf5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2815ddf0 0x7fec280b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.347+0000 7fec2cbf5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fec280b5af0 0x7fec2806bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.347+0000 7fec2cbf5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec280b9930 con 0x7fec280b5710 2024-04-25T08:34:47.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.347+0000 7fec2cbf5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec28161e00 con 0x7fec280b5af0 2024-04-25T08:34:47.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.347+0000 7fec2cbf5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec281620e0 con 0x7fec2815ddf0 2024-04-25T08:34:47.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.347+0000 7fec2659c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fec280b5710 0x7fec28161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.347+0000 7fec2659c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fec280b5710 0x7fec28161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51024/0 (socket says 172.21.15.38:51024) 2024-04-25T08:34:47.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.347+0000 7fec2659c640 1 -- 172.21.15.38:0/2995819060 learned_addr learned my addr 172.21.15.38:0/2995819060 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:47.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.348+0000 7fec26d9d640 1 --2- 172.21.15.38:0/2995819060 >> v2:172.21.15.107:3300/0 conn(0x7fec280b5af0 0x7fec2806bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.348+0000 7fec2659c640 1 -- 172.21.15.38:0/2995819060 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2815ddf0 msgr2=0x7fec280b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:47.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.348+0000 7fec2659c640 1 --2- 172.21.15.38:0/2995819060 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2815ddf0 0x7fec280b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.348+0000 7fec2659c640 1 -- 172.21.15.38:0/2995819060 >> v2:172.21.15.107:3300/0 conn(0x7fec280b5af0 msgr2=0x7fec2806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:47.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.348+0000 7fec2659c640 1 --2- 172.21.15.38:0/2995819060 >> v2:172.21.15.107:3300/0 conn(0x7fec280b5af0 0x7fec2806bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.348+0000 7fec2659c640 1 -- 172.21.15.38:0/2995819060 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec281623c0 con 0x7fec280b5710 2024-04-25T08:34:47.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.348+0000 7fec26d9d640 1 --2- 172.21.15.38:0/2995819060 >> v2:172.21.15.107:3300/0 conn(0x7fec280b5af0 0x7fec2806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:47.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.348+0000 7fec2659c640 1 --2- 172.21.15.38:0/2995819060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec280b5710 0x7fec28161700 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fec280d1bd0 tx=0x7fec2002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe8b64d669128d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:47.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.348+0000 7fec2559a640 1 -- 172.21.15.38:0/2995819060 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20037070 con 0x7fec280b5710 2024-04-25T08:34:47.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.348+0000 7fec2559a640 1 -- 172.21.15.38:0/2995819060 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec20032440 con 0x7fec280b5710 2024-04-25T08:34:47.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.348+0000 7fec2559a640 1 -- 172.21.15.38:0/2995819060 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20032810 con 0x7fec280b5710 2024-04-25T08:34:47.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.349+0000 7fec2cbf5640 1 -- 172.21.15.38:0/2995819060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec280b5710 msgr2=0x7fec28161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.349+0000 7fec2cbf5640 1 --2- 172.21.15.38:0/2995819060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec280b5710 0x7fec28161700 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fec280d1bd0 tx=0x7fec2002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.349+0000 7fec2cbf5640 1 -- 172.21.15.38:0/2995819060 shutdown_connections 2024-04-25T08:34:47.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.349+0000 7fec2cbf5640 1 --2- 172.21.15.38:0/2995819060 >> v2:172.21.15.107:3300/0 conn(0x7fec280b5af0 0x7fec2806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.349+0000 7fec2cbf5640 1 --2- 172.21.15.38:0/2995819060 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2815ddf0 0x7fec280b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.349+0000 7fec2cbf5640 1 --2- 172.21.15.38:0/2995819060 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec280b5710 0x7fec28161700 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.349+0000 7fec2cbf5640 1 -- 172.21.15.38:0/2995819060 >> 172.21.15.38:0/2995819060 conn(0x7fec280d07c0 msgr2=0x7fec280d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:47.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.349+0000 7fec2cbf5640 1 -- 172.21.15.38:0/2995819060 shutdown_connections 2024-04-25T08:34:47.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.349+0000 7fec2cbf5640 1 -- 172.21.15.38:0/2995819060 wait complete. 2024-04-25T08:34:47.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.350+0000 7fec2cbf5640 1 Processor -- start 2024-04-25T08:34:47.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.350+0000 7fec2cbf5640 1 -- start start 2024-04-25T08:34:47.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.351+0000 7fec2cbf5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec280b5710 0x7fec28160c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.351+0000 7fec2cbf5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec280b5af0 0x7fec281f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.351+0000 7fec2cbf5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2815ddf0 0x7fec280cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.351+0000 7fec2cbf5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2806c6b0 con 0x7fec280b5710 2024-04-25T08:34:47.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.351+0000 7fec2cbf5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2806c530 con 0x7fec280b5af0 2024-04-25T08:34:47.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.351+0000 7fec2cbf5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2806c830 con 0x7fec2815ddf0 2024-04-25T08:34:47.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.351+0000 7fec2659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec280b5710 0x7fec28160c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.351+0000 7fec2659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec280b5710 0x7fec28160c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51036/0 (socket says 172.21.15.38:51036) 2024-04-25T08:34:47.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.351+0000 7fec2659c640 1 -- 172.21.15.38:0/1403490016 learned_addr learned my addr 172.21.15.38:0/1403490016 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:47.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec25d9b640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec280b5af0 0x7fec281f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec26d9d640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2815ddf0 0x7fec280cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec2659c640 1 -- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2815ddf0 msgr2=0x7fec280cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec2659c640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2815ddf0 0x7fec280cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec2659c640 1 -- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec280b5af0 msgr2=0x7fec281f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec2659c640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec280b5af0 0x7fec281f7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec2659c640 1 -- 172.21.15.38:0/1403490016 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec280ccf80 con 0x7fec280b5710 2024-04-25T08:34:47.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec25d9b640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec280b5af0 0x7fec281f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:47.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec26d9d640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2815ddf0 0x7fec280cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:47.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec2659c640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec280b5710 0x7fec28160c00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fec200097c0 tx=0x7fec200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:47.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec1effd640 1 -- 172.21.15.38:0/1403490016 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20037040 con 0x7fec280b5710 2024-04-25T08:34:47.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec1effd640 1 -- 172.21.15.38:0/1403490016 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec20042070 con 0x7fec280b5710 2024-04-25T08:34:47.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec2cbf5640 1 -- 172.21.15.38:0/1403490016 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec280cd220 con 0x7fec280b5710 2024-04-25T08:34:47.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec1effd640 1 -- 172.21.15.38:0/1403490016 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec20030b90 con 0x7fec280b5710 2024-04-25T08:34:47.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.352+0000 7fec2cbf5640 1 -- 172.21.15.38:0/1403490016 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec281f7130 con 0x7fec280b5710 2024-04-25T08:34:47.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.353+0000 7fec2cbf5640 1 -- 172.21.15.38:0/1403490016 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec28068e80 con 0x7fec280b5710 2024-04-25T08:34:47.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.353+0000 7fec1effd640 1 -- 172.21.15.38:0/1403490016 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec20047460 con 0x7fec280b5710 2024-04-25T08:34:47.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.354+0000 7fec1effd640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec0806b4d0 0x7fec0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.354+0000 7fec1effd640 1 -- 172.21.15.38:0/1403490016 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fec200aaf60 con 0x7fec280b5710 2024-04-25T08:34:47.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.354+0000 7fec25d9b640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec0806b4d0 0x7fec0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.355+0000 7fec25d9b640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec0806b4d0 0x7fec0806d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fec281f7090 tx=0x7fec10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:47.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.358+0000 7fec1effd640 1 -- 172.21.15.38:0/1403490016 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec20045360 con 0x7fec280b5710 2024-04-25T08:34:47.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.539+0000 7fec2cbf5640 1 -- 172.21.15.38:0/1403490016 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec280b6ab0 con 0x7fec280b5710 2024-04-25T08:34:47.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.540+0000 7fec1effd640 1 -- 172.21.15.38:0/1403490016 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16930 (secure 0 0 0) 0x7fec20079ce0 con 0x7fec280b5710 2024-04-25T08:34:47.536 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:47.536 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":54,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:34:45.512274+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:34:42.507595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:34:43.209205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:34:47.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.543+0000 7fec2cbf5640 1 -- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec0806b4d0 msgr2=0x7fec0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.543+0000 7fec2cbf5640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec0806b4d0 0x7fec0806d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fec281f7090 tx=0x7fec10008040 comp rx=0 tx=0).stop 2024-04-25T08:34:47.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.543+0000 7fec2cbf5640 1 -- 172.21.15.38:0/1403490016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec280b5710 msgr2=0x7fec28160c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.543+0000 7fec2cbf5640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec280b5710 0x7fec28160c00 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fec200097c0 tx=0x7fec200046c0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.543+0000 7fec2cbf5640 1 -- 172.21.15.38:0/1403490016 shutdown_connections 2024-04-25T08:34:47.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.543+0000 7fec2cbf5640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec0806b4d0 0x7fec0806d990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.543+0000 7fec2cbf5640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec2815ddf0 0x7fec280cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.543+0000 7fec2cbf5640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec280b5af0 0x7fec281f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.543+0000 7fec2cbf5640 1 --2- 172.21.15.38:0/1403490016 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec280b5710 0x7fec28160c00 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.543+0000 7fec2cbf5640 1 -- 172.21.15.38:0/1403490016 >> 172.21.15.38:0/1403490016 conn(0x7fec280d07c0 msgr2=0x7fec280b7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:47.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.543+0000 7fec2cbf5640 1 -- 172.21.15.38:0/1403490016 shutdown_connections 2024-04-25T08:34:47.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.543+0000 7fec2cbf5640 1 -- 172.21.15.38:0/1403490016 wait complete. 2024-04-25T08:34:47.549 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:34:47.549 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:34:47.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.656+0000 7ff1a3609640 1 Processor -- start 2024-04-25T08:34:47.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.656+0000 7ff1a3609640 1 -- start start 2024-04-25T08:34:47.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.657+0000 7ff1a3609640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff19c15df50 0x7ff19c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.657+0000 7ff1a3609640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff19c0ce2e0 0x7ff19c0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.657+0000 7ff1a3609640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c0ceff0 con 0x7ff19c06c110 2024-04-25T08:34:47.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.657+0000 7ff1a3609640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c0cf2d0 con 0x7ff19c15df50 2024-04-25T08:34:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.657+0000 7ff1a3609640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c0d1f70 con 0x7ff19c0ce2e0 2024-04-25T08:34:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.657+0000 7ff1a0ba4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff19c15df50 0x7ff19c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.657+0000 7ff1a13a5640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff19c06c110 0x7ff19c06c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33268/0 (socket says 172.21.15.38:33268) 2024-04-25T08:34:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.657+0000 7ff1a13a5640 1 -- 172.21.15.38:0/877275692 learned_addr learned my addr 172.21.15.38:0/877275692 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.657+0000 7ff1a1ba6640 1 --2- 172.21.15.38:0/877275692 >> v2:172.21.15.107:3300/0 conn(0x7ff19c0ce2e0 0x7ff19c0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.657+0000 7ff1a0ba4640 1 -- 172.21.15.38:0/877275692 >> v2:172.21.15.107:3300/0 conn(0x7ff19c0ce2e0 msgr2=0x7ff19c0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.657+0000 7ff1a0ba4640 1 --2- 172.21.15.38:0/877275692 >> v2:172.21.15.107:3300/0 conn(0x7ff19c0ce2e0 0x7ff19c0d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.658+0000 7ff1a0ba4640 1 -- 172.21.15.38:0/877275692 >> v1:172.21.15.38:6789/0 conn(0x7ff19c06c110 legacy=0x7ff19c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.658+0000 7ff1a0ba4640 1 -- 172.21.15.38:0/877275692 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0d2250 con 0x7ff19c15df50 2024-04-25T08:34:47.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.658+0000 7ff19bfff640 1 -- 172.21.15.38:0/877275692 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651250926 0 0) 0x7ff19c0ceff0 con 0x7ff19c06c110 2024-04-25T08:34:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.658+0000 7ff1a0ba4640 1 --2- 172.21.15.38:0/877275692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff19c15df50 0x7ff19c0ce7a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0b6520 tx=0x7ff19002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee6ccc3897fde02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:47.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.658+0000 7ff19bfff640 1 -- 172.21.15.38:0/877275692 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190037070 con 0x7ff19c15df50 2024-04-25T08:34:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.658+0000 7ff19bfff640 1 -- 172.21.15.38:0/877275692 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff190032440 con 0x7ff19c15df50 2024-04-25T08:34:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.658+0000 7ff19bfff640 1 -- 172.21.15.38:0/877275692 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff190032810 con 0x7ff19c15df50 2024-04-25T08:34:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.659+0000 7ff1a3609640 1 -- 172.21.15.38:0/877275692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff19c15df50 msgr2=0x7ff19c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.659+0000 7ff1a3609640 1 --2- 172.21.15.38:0/877275692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff19c15df50 0x7ff19c0ce7a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0b6520 tx=0x7ff19002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.659+0000 7ff1a3609640 1 -- 172.21.15.38:0/877275692 shutdown_connections 2024-04-25T08:34:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.659+0000 7ff1a3609640 1 --2- 172.21.15.38:0/877275692 >> v2:172.21.15.107:3300/0 conn(0x7ff19c0ce2e0 0x7ff19c0d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.659+0000 7ff1a3609640 1 --2- 172.21.15.38:0/877275692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff19c15df50 0x7ff19c0ce7a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.659+0000 7ff1a3609640 1 -- 172.21.15.38:0/877275692 >> 172.21.15.38:0/877275692 conn(0x7ff19c0d3fb0 msgr2=0x7ff19c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.659+0000 7ff1a3609640 1 -- 172.21.15.38:0/877275692 shutdown_connections 2024-04-25T08:34:47.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.659+0000 7ff1a3609640 1 -- 172.21.15.38:0/877275692 wait complete. 2024-04-25T08:34:47.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.660+0000 7ff1a3609640 1 Processor -- start 2024-04-25T08:34:47.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.660+0000 7ff1a3609640 1 -- start start 2024-04-25T08:34:47.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a3609640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff19c06c110 0x7ff19c0b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a3609640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff19c0ce2e0 0x7ff19c160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a3609640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c15df50 0x7ff19c14cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a3609640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0d2f90 con 0x7ff19c06c110 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a3609640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0cece0 con 0x7ff19c15df50 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a3609640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0d3110 con 0x7ff19c0ce2e0 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a13a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff19c06c110 0x7ff19c0b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a13a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff19c06c110 0x7ff19c0b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51042/0 (socket says 172.21.15.38:51042) 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a13a5640 1 -- 172.21.15.38:0/1231476432 learned_addr learned my addr 172.21.15.38:0/1231476432 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a1ba6640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c15df50 0x7ff19c14cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a0ba4640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff19c0ce2e0 0x7ff19c160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a13a5640 1 -- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff19c0ce2e0 msgr2=0x7ff19c160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a13a5640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff19c0ce2e0 0x7ff19c160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a13a5640 1 -- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c15df50 msgr2=0x7ff19c14cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a13a5640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c15df50 0x7ff19c14cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.661+0000 7ff1a13a5640 1 -- 172.21.15.38:0/1231476432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0b86e0 con 0x7ff19c06c110 2024-04-25T08:34:47.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.662+0000 7ff1a13a5640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff19c06c110 0x7ff19c0b7760 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff19400c4d0 tx=0x7ff19400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.662+0000 7ff199ffb640 1 -- 172.21.15.38:0/1231476432 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194018070 con 0x7ff19c06c110 2024-04-25T08:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.662+0000 7ff199ffb640 1 -- 172.21.15.38:0/1231476432 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff194007500 con 0x7ff19c06c110 2024-04-25T08:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.662+0000 7ff1a3609640 1 -- 172.21.15.38:0/1231476432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c0cbc40 con 0x7ff19c06c110 2024-04-25T08:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.662+0000 7ff199ffb640 1 -- 172.21.15.38:0/1231476432 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff194004a20 con 0x7ff19c06c110 2024-04-25T08:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.662+0000 7ff1a3609640 1 -- 172.21.15.38:0/1231476432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c0b8120 con 0x7ff19c06c110 2024-04-25T08:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.663+0000 7ff1a3609640 1 -- 172.21.15.38:0/1231476432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c068e80 con 0x7ff19c06c110 2024-04-25T08:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.663+0000 7ff199ffb640 1 -- 172.21.15.38:0/1231476432 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff19401b070 con 0x7ff19c06c110 2024-04-25T08:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.664+0000 7ff199ffb640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff18806b4d0 0x7ff18806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.664+0000 7ff199ffb640 1 -- 172.21.15.38:0/1231476432 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff1940896b0 con 0x7ff19c06c110 2024-04-25T08:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.664+0000 7ff1a0ba4640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff18806b4d0 0x7ff18806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:47.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.665+0000 7ff1a0ba4640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff18806b4d0 0x7ff18806d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff190034000 tx=0x7ff19002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.668+0000 7ff199ffb640 1 -- 172.21.15.38:0/1231476432 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff194058390 con 0x7ff19c06c110 2024-04-25T08:34:47.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.845+0000 7ff1a3609640 1 -- 172.21.15.38:0/1231476432 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff19c0cdc40 con 0x7ff18806b4d0 2024-04-25T08:34:47.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.851+0000 7ff199ffb640 1 -- 172.21.15.38:0/1231476432 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246879 (secure 0 0 0) 0x7ff19c0cdc40 con 0x7ff18806b4d0 2024-04-25T08:34:47.849 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:47.851 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:34:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.854+0000 7ff1a3609640 1 -- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff18806b4d0 msgr2=0x7ff18806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.854+0000 7ff1a3609640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff18806b4d0 0x7ff18806d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff190034000 tx=0x7ff19002e000 comp rx=0 tx=0).stop 2024-04-25T08:34:47.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.854+0000 7ff1a3609640 1 -- 172.21.15.38:0/1231476432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff19c06c110 msgr2=0x7ff19c0b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.854+0000 7ff1a3609640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff19c06c110 0x7ff19c0b7760 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff19400c4d0 tx=0x7ff19400c990 comp rx=0 tx=0).stop 2024-04-25T08:34:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.854+0000 7ff1a3609640 1 -- 172.21.15.38:0/1231476432 shutdown_connections 2024-04-25T08:34:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.854+0000 7ff1a3609640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff19c15df50 0x7ff19c14cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.854+0000 7ff1a3609640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff18806b4d0 0x7ff18806d990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.854+0000 7ff1a3609640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff19c0ce2e0 0x7ff19c160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.854+0000 7ff1a3609640 1 --2- 172.21.15.38:0/1231476432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff19c06c110 0x7ff19c0b7760 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.854+0000 7ff1a3609640 1 -- 172.21.15.38:0/1231476432 >> 172.21.15.38:0/1231476432 conn(0x7ff19c0d3fb0 msgr2=0x7ff19c0d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.854+0000 7ff1a3609640 1 -- 172.21.15.38:0/1231476432 shutdown_connections 2024-04-25T08:34:47.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:47.854+0000 7ff1a3609640 1 -- 172.21.15.38:0/1231476432 wait complete. 2024-04-25T08:34:47.861 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-04-25T08:34:47.249503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591076,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":179,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":367916,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":361715,"kb_avail":754606804,"statfs":{"total":773094113280,"available":772717367296,"internally_reserved":0,"allocated":6201344,"data_stored":3414158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":370396720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":708,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.393179"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086305+0000","last_change":"2024-04-25T08:34:44.527955+0000","last_active":"2024-04-25T08:34:46.086305+0000","last_peered":"2024-04-25T08:34:46.086305+0000","last_clean":"2024-04-25T08:34:46.086305+0000","last_became_active":"2024-04-25T08:34:44.527662+0000","last_became_peered":"2024-04-25T08:34:44.527662+0000","last_unstale":"2024-04-25T08:34:46.086305+0000","last_undegraded":"2024-04-25T08:34:46.086305+0000","last_fullsized":"2024-04-25T08:34:46.086305+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:09.895753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093741+0000","last_change":"2024-04-25T08:34:44.534121+0000","last_active":"2024-04-25T08:34:46.093741+0000","last_peered":"2024-04-25T08:34:46.093741+0000","last_clean":"2024-04-25T08:34:46.093741+0000","last_became_active":"2024-04-25T08:34:44.533765+0000","last_became_peered":"2024-04-25T08:34:44.533765+0000","last_unstale":"2024-04-25T08:34:46.093741+0000","last_undegraded":"2024-04-25T08:34:46.093741+0000","last_fullsized":"2024-04-25T08:34:46.093741+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.333087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531042+0000","last_change":"2024-04-25T08:34:44.520910+0000","last_active":"2024-04-25T08:34:45.531042+0000","last_peered":"2024-04-25T08:34:45.531042+0000","last_clean":"2024-04-25T08:34:45.531042+0000","last_became_active":"2024-04-25T08:34:44.520607+0000","last_became_peered":"2024-04-25T08:34:44.520607+0000","last_unstale":"2024-04-25T08:34:45.531042+0000","last_undegraded":"2024-04-25T08:34:45.531042+0000","last_fullsized":"2024-04-25T08:34:45.531042+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:43.933652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086256+0000","last_change":"2024-04-25T08:34:44.527358+0000","last_active":"2024-04-25T08:34:46.086256+0000","last_peered":"2024-04-25T08:34:46.086256+0000","last_clean":"2024-04-25T08:34:46.086256+0000","last_became_active":"2024-04-25T08:34:44.527098+0000","last_became_peered":"2024-04-25T08:34:44.527098+0000","last_unstale":"2024-04-25T08:34:46.086256+0000","last_undegraded":"2024-04-25T08:34:46.086256+0000","last_fullsized":"2024-04-25T08:34:46.086256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:47.550727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094066+0000","last_change":"2024-04-25T08:34:44.536974+0000","last_active":"2024-04-25T08:34:46.094066+0000","last_peered":"2024-04-25T08:34:46.094066+0000","last_clean":"2024-04-25T08:34:46.094066+0000","last_became_active":"2024-04-25T08:34:44.536690+0000","last_became_peered":"2024-04-25T08:34:44.536690+0000","last_unstale":"2024-04-25T08:34:46.094066+0000","last_undegraded":"2024-04-25T08:34:46.094066+0000","last_fullsized":"2024-04-25T08:34:46.094066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:27.300505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086137+0000","last_change":"2024-04-25T08:34:44.533732+0000","last_active":"2024-04-25T08:34:46.086137+0000","last_peered":"2024-04-25T08:34:46.086137+0000","last_clean":"2024-04-25T08:34:46.086137+0000","last_became_active":"2024-04-25T08:34:44.533441+0000","last_became_peered":"2024-04-25T08:34:44.533441+0000","last_unstale":"2024-04-25T08:34:46.086137+0000","last_undegraded":"2024-04-25T08:34:46.086137+0000","last_fullsized":"2024-04-25T08:34:46.086137+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:11.733375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086171+0000","last_change":"2024-04-25T08:34:44.531394+0000","last_active":"2024-04-25T08:34:46.086171+0000","last_peered":"2024-04-25T08:34:46.086171+0000","last_clean":"2024-04-25T08:34:46.086171+0000","last_became_active":"2024-04-25T08:34:44.531057+0000","last_became_peered":"2024-04-25T08:34:44.531057+0000","last_unstale":"2024-04-25T08:34:46.086171+0000","last_undegraded":"2024-04-25T08:34:46.086171+0000","last_fullsized":"2024-04-25T08:34:46.086171+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:18.730106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094348+0000","last_change":"2024-04-25T08:34:44.533220+0000","last_active":"2024-04-25T08:34:46.094348+0000","last_peered":"2024-04-25T08:34:46.094348+0000","last_clean":"2024-04-25T08:34:46.094348+0000","last_became_active":"2024-04-25T08:34:44.532647+0000","last_became_peered":"2024-04-25T08:34:44.532647+0000","last_unstale":"2024-04-25T08:34:46.094348+0000","last_undegraded":"2024-04-25T08:34:46.094348+0000","last_fullsized":"2024-04-25T08:34:46.094348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:00.701022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086067+0000","last_change":"2024-04-25T08:34:44.525673+0000","last_active":"2024-04-25T08:34:46.086067+0000","last_peered":"2024-04-25T08:34:46.086067+0000","last_clean":"2024-04-25T08:34:46.086067+0000","last_became_active":"2024-04-25T08:34:44.525384+0000","last_became_peered":"2024-04-25T08:34:44.525384+0000","last_unstale":"2024-04-25T08:34:46.086067+0000","last_undegraded":"2024-04-25T08:34:46.086067+0000","last_fullsized":"2024-04-25T08:34:46.086067+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.499854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095168+0000","last_change":"2024-04-25T08:34:44.533259+0000","last_active":"2024-04-25T08:34:46.095168+0000","last_peered":"2024-04-25T08:34:46.095168+0000","last_clean":"2024-04-25T08:34:46.095168+0000","last_became_active":"2024-04-25T08:34:44.531589+0000","last_became_peered":"2024-04-25T08:34:44.531589+0000","last_unstale":"2024-04-25T08:34:46.095168+0000","last_undegraded":"2024-04-25T08:34:46.095168+0000","last_fullsized":"2024-04-25T08:34:46.095168+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:01.549992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093885+0000","last_change":"2024-04-25T08:34:44.530163+0000","last_active":"2024-04-25T08:34:46.093885+0000","last_peered":"2024-04-25T08:34:46.093885+0000","last_clean":"2024-04-25T08:34:46.093885+0000","last_became_active":"2024-04-25T08:34:44.529905+0000","last_became_peered":"2024-04-25T08:34:44.529905+0000","last_unstale":"2024-04-25T08:34:46.093885+0000","last_undegraded":"2024-04-25T08:34:46.093885+0000","last_fullsized":"2024-04-25T08:34:46.093885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:18.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529970+0000","last_change":"2024-04-25T08:34:44.520405+0000","last_active":"2024-04-25T08:34:45.529970+0000","last_peered":"2024-04-25T08:34:45.529970+0000","last_clean":"2024-04-25T08:34:45.529970+0000","last_became_active":"2024-04-25T08:34:44.520122+0000","last_became_peered":"2024-04-25T08:34:44.520122+0000","last_unstale":"2024-04-25T08:34:45.529970+0000","last_undegraded":"2024-04-25T08:34:45.529970+0000","last_fullsized":"2024-04-25T08:34:45.529970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:09.811944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530067+0000","last_change":"2024-04-25T08:34:44.522344+0000","last_active":"2024-04-25T08:34:45.530067+0000","last_peered":"2024-04-25T08:34:45.530067+0000","last_clean":"2024-04-25T08:34:45.530067+0000","last_became_active":"2024-04-25T08:34:44.522063+0000","last_became_peered":"2024-04-25T08:34:44.522063+0000","last_unstale":"2024-04-25T08:34:45.530067+0000","last_undegraded":"2024-04-25T08:34:45.530067+0000","last_fullsized":"2024-04-25T08:34:45.530067+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:25.590051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093628+0000","last_change":"2024-04-25T08:34:44.536670+0000","last_active":"2024-04-25T08:34:46.093628+0000","last_peered":"2024-04-25T08:34:46.093628+0000","last_clean":"2024-04-25T08:34:46.093628+0000","last_became_active":"2024-04-25T08:34:44.536110+0000","last_became_peered":"2024-04-25T08:34:44.536110+0000","last_unstale":"2024-04-25T08:34:46.093628+0000","last_undegraded":"2024-04-25T08:34:46.093628+0000","last_fullsized":"2024-04-25T08:34:46.093628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:43.198107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095058+0000","last_change":"2024-04-25T08:34:44.533241+0000","last_active":"2024-04-25T08:34:46.095058+0000","last_peered":"2024-04-25T08:34:46.095058+0000","last_clean":"2024-04-25T08:34:46.095058+0000","last_became_active":"2024-04-25T08:34:44.531910+0000","last_became_peered":"2024-04-25T08:34:44.531910+0000","last_unstale":"2024-04-25T08:34:46.095058+0000","last_undegraded":"2024-04-25T08:34:46.095058+0000","last_fullsized":"2024-04-25T08:34:46.095058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.963128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086493+0000","last_change":"2024-04-25T08:34:44.538453+0000","last_active":"2024-04-25T08:34:46.086493+0000","last_peered":"2024-04-25T08:34:46.086493+0000","last_clean":"2024-04-25T08:34:46.086493+0000","last_became_active":"2024-04-25T08:34:44.538165+0000","last_became_peered":"2024-04-25T08:34:44.538165+0000","last_unstale":"2024-04-25T08:34:46.086493+0000","last_undegraded":"2024-04-25T08:34:46.086493+0000","last_fullsized":"2024-04-25T08:34:46.086493+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:46.558329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530135+0000","last_change":"2024-04-25T08:34:44.528517+0000","last_active":"2024-04-25T08:34:45.530135+0000","last_peered":"2024-04-25T08:34:45.530135+0000","last_clean":"2024-04-25T08:34:45.530135+0000","last_became_active":"2024-04-25T08:34:44.528229+0000","last_became_peered":"2024-04-25T08:34:44.528229+0000","last_unstale":"2024-04-25T08:34:45.530135+0000","last_undegraded":"2024-04-25T08:34:45.530135+0000","last_fullsized":"2024-04-25T08:34:45.530135+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:41.585740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094613+0000","last_change":"2024-04-25T08:34:44.528023+0000","last_active":"2024-04-25T08:34:46.094613+0000","last_peered":"2024-04-25T08:34:46.094613+0000","last_clean":"2024-04-25T08:34:46.094613+0000","last_became_active":"2024-04-25T08:34:44.527668+0000","last_became_peered":"2024-04-25T08:34:44.527668+0000","last_unstale":"2024-04-25T08:34:46.094613+0000","last_undegraded":"2024-04-25T08:34:46.094613+0000","last_fullsized":"2024-04-25T08:34:46.094613+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:06.085151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094200+0000","last_change":"2024-04-25T08:34:44.533947+0000","last_active":"2024-04-25T08:34:46.094200+0000","last_peered":"2024-04-25T08:34:46.094200+0000","last_clean":"2024-04-25T08:34:46.094200+0000","last_became_active":"2024-04-25T08:34:44.533609+0000","last_became_peered":"2024-04-25T08:34:44.533609+0000","last_unstale":"2024-04-25T08:34:46.094200+0000","last_undegraded":"2024-04-25T08:34:46.094200+0000","last_fullsized":"2024-04-25T08:34:46.094200+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:44.889254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093558+0000","last_change":"2024-04-25T08:34:44.532723+0000","last_active":"2024-04-25T08:34:46.093558+0000","last_peered":"2024-04-25T08:34:46.093558+0000","last_clean":"2024-04-25T08:34:46.093558+0000","last_became_active":"2024-04-25T08:34:44.531850+0000","last_became_peered":"2024-04-25T08:34:44.531850+0000","last_unstale":"2024-04-25T08:34:46.093558+0000","last_undegraded":"2024-04-25T08:34:46.093558+0000","last_fullsized":"2024-04-25T08:34:46.093558+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:20.979454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529837+0000","last_change":"2024-04-25T08:34:43.873145+0000","last_active":"2024-04-25T08:34:45.529837+0000","last_peered":"2024-04-25T08:34:45.529837+0000","last_clean":"2024-04-25T08:34:45.529837+0000","last_became_active":"2024-04-25T08:34:43.872860+0000","last_became_peered":"2024-04-25T08:34:43.872860+0000","last_unstale":"2024-04-25T08:34:45.529837+0000","last_undegraded":"2024-04-25T08:34:45.529837+0000","last_fullsized":"2024-04-25T08:34:45.529837+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:39.042852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093598+0000","last_change":"2024-04-25T08:34:44.530047+0000","last_active":"2024-04-25T08:34:46.093598+0000","last_peered":"2024-04-25T08:34:46.093598+0000","last_clean":"2024-04-25T08:34:46.093598+0000","last_became_active":"2024-04-25T08:34:44.529781+0000","last_became_peered":"2024-04-25T08:34:44.529781+0000","last_unstale":"2024-04-25T08:34:46.093598+0000","last_undegraded":"2024-04-25T08:34:46.093598+0000","last_fullsized":"2024-04-25T08:34:46.093598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:36.970964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086257+0000","last_change":"2024-04-25T08:34:43.884678+0000","last_active":"2024-04-25T08:34:46.086257+0000","last_peered":"2024-04-25T08:34:46.086257+0000","last_clean":"2024-04-25T08:34:46.086257+0000","last_became_active":"2024-04-25T08:34:43.884357+0000","last_became_peered":"2024-04-25T08:34:43.884357+0000","last_unstale":"2024-04-25T08:34:46.086257+0000","last_undegraded":"2024-04-25T08:34:46.086257+0000","last_fullsized":"2024-04-25T08:34:46.086257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:04.802285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530601+0000","last_change":"2024-04-25T08:34:44.519962+0000","last_active":"2024-04-25T08:34:45.530601+0000","last_peered":"2024-04-25T08:34:45.530601+0000","last_clean":"2024-04-25T08:34:45.530601+0000","last_became_active":"2024-04-25T08:34:44.519671+0000","last_became_peered":"2024-04-25T08:34:44.519671+0000","last_unstale":"2024-04-25T08:34:45.530601+0000","last_undegraded":"2024-04-25T08:34:45.530601+0000","last_fullsized":"2024-04-25T08:34:45.530601+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:49.239506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086556+0000","last_change":"2024-04-25T08:34:43.878220+0000","last_active":"2024-04-25T08:34:46.086556+0000","last_peered":"2024-04-25T08:34:46.086556+0000","last_clean":"2024-04-25T08:34:46.086556+0000","last_became_active":"2024-04-25T08:34:43.877915+0000","last_became_peered":"2024-04-25T08:34:43.877915+0000","last_unstale":"2024-04-25T08:34:46.086556+0000","last_undegraded":"2024-04-25T08:34:46.086556+0000","last_fullsized":"2024-04-25T08:34:46.086556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:55.485664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093702+0000","last_change":"2024-04-25T08:34:43.882922+0000","last_active":"2024-04-25T08:34:46.093702+0000","last_peered":"2024-04-25T08:34:46.093702+0000","last_clean":"2024-04-25T08:34:46.093702+0000","last_became_active":"2024-04-25T08:34:43.882609+0000","last_became_peered":"2024-04-25T08:34:43.882609+0000","last_unstale":"2024-04-25T08:34:46.093702+0000","last_undegraded":"2024-04-25T08:34:46.093702+0000","last_fullsized":"2024-04-25T08:34:46.093702+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:17.727985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531083+0000","last_change":"2024-04-25T08:34:44.532481+0000","last_active":"2024-04-25T08:34:45.531083+0000","last_peered":"2024-04-25T08:34:45.531083+0000","last_clean":"2024-04-25T08:34:45.531083+0000","last_became_active":"2024-04-25T08:34:44.532204+0000","last_became_peered":"2024-04-25T08:34:44.532204+0000","last_unstale":"2024-04-25T08:34:45.531083+0000","last_undegraded":"2024-04-25T08:34:45.531083+0000","last_fullsized":"2024-04-25T08:34:45.531083+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:19.590018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094711+0000","last_change":"2024-04-25T08:34:43.883536+0000","last_active":"2024-04-25T08:34:46.094711+0000","last_peered":"2024-04-25T08:34:46.094711+0000","last_clean":"2024-04-25T08:34:46.094711+0000","last_became_active":"2024-04-25T08:34:43.883232+0000","last_became_peered":"2024-04-25T08:34:43.883232+0000","last_unstale":"2024-04-25T08:34:46.094711+0000","last_undegraded":"2024-04-25T08:34:46.094711+0000","last_fullsized":"2024-04-25T08:34:46.094711+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:10.731254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531305+0000","last_change":"2024-04-25T08:34:43.875476+0000","last_active":"2024-04-25T08:34:45.531305+0000","last_peered":"2024-04-25T08:34:45.531305+0000","last_clean":"2024-04-25T08:34:45.531305+0000","last_became_active":"2024-04-25T08:34:43.874920+0000","last_became_peered":"2024-04-25T08:34:43.874920+0000","last_unstale":"2024-04-25T08:34:45.531305+0000","last_undegraded":"2024-04-25T08:34:45.531305+0000","last_fullsized":"2024-04-25T08:34:45.531305+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:54.880133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530427+0000","last_change":"2024-04-25T08:34:43.869200+0000","last_active":"2024-04-25T08:34:45.530427+0000","last_peered":"2024-04-25T08:34:45.530427+0000","last_clean":"2024-04-25T08:34:45.530427+0000","last_became_active":"2024-04-25T08:34:43.868895+0000","last_became_peered":"2024-04-25T08:34:43.868895+0000","last_unstale":"2024-04-25T08:34:45.530427+0000","last_undegraded":"2024-04-25T08:34:45.530427+0000","last_fullsized":"2024-04-25T08:34:45.530427+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:36.714740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093548+0000","last_change":"2024-04-25T08:34:43.881571+0000","last_active":"2024-04-25T08:34:46.093548+0000","last_peered":"2024-04-25T08:34:46.093548+0000","last_clean":"2024-04-25T08:34:46.093548+0000","last_became_active":"2024-04-25T08:34:43.881272+0000","last_became_peered":"2024-04-25T08:34:43.881272+0000","last_unstale":"2024-04-25T08:34:46.093548+0000","last_undegraded":"2024-04-25T08:34:46.093548+0000","last_fullsized":"2024-04-25T08:34:46.093548+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:52.868065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530959+0000","last_change":"2024-04-25T08:34:44.531396+0000","last_active":"2024-04-25T08:34:45.530959+0000","last_peered":"2024-04-25T08:34:45.530959+0000","last_clean":"2024-04-25T08:34:45.530959+0000","last_became_active":"2024-04-25T08:34:44.531134+0000","last_became_peered":"2024-04-25T08:34:44.531134+0000","last_unstale":"2024-04-25T08:34:45.530959+0000","last_undegraded":"2024-04-25T08:34:45.530959+0000","last_fullsized":"2024-04-25T08:34:45.530959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:12.856976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093479+0000","last_change":"2024-04-25T08:34:44.525749+0000","last_active":"2024-04-25T08:34:46.093479+0000","last_peered":"2024-04-25T08:34:46.093479+0000","last_clean":"2024-04-25T08:34:46.093479+0000","last_became_active":"2024-04-25T08:34:44.525470+0000","last_became_peered":"2024-04-25T08:34:44.525470+0000","last_unstale":"2024-04-25T08:34:46.093479+0000","last_undegraded":"2024-04-25T08:34:46.093479+0000","last_fullsized":"2024-04-25T08:34:46.093479+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:40.781389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530997+0000","last_change":"2024-04-25T08:34:43.876071+0000","last_active":"2024-04-25T08:34:45.530997+0000","last_peered":"2024-04-25T08:34:45.530997+0000","last_clean":"2024-04-25T08:34:45.530997+0000","last_became_active":"2024-04-25T08:34:43.875782+0000","last_became_peered":"2024-04-25T08:34:43.875782+0000","last_unstale":"2024-04-25T08:34:45.530997+0000","last_undegraded":"2024-04-25T08:34:45.530997+0000","last_fullsized":"2024-04-25T08:34:45.530997+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:39.042852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094436+0000","last_change":"2024-04-25T08:34:44.528331+0000","last_active":"2024-04-25T08:34:46.094436+0000","last_peered":"2024-04-25T08:34:46.094436+0000","last_clean":"2024-04-25T08:34:46.094436+0000","last_became_active":"2024-04-25T08:34:44.528080+0000","last_became_peered":"2024-04-25T08:34:44.528080+0000","last_unstale":"2024-04-25T08:34:46.094436+0000","last_undegraded":"2024-04-25T08:34:46.094436+0000","last_fullsized":"2024-04-25T08:34:46.094436+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:25.030847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531206+0000","last_change":"2024-04-25T08:34:44.529717+0000","last_active":"2024-04-25T08:34:45.531206+0000","last_peered":"2024-04-25T08:34:45.531206+0000","last_clean":"2024-04-25T08:34:45.531206+0000","last_became_active":"2024-04-25T08:34:44.529442+0000","last_became_peered":"2024-04-25T08:34:44.529442+0000","last_unstale":"2024-04-25T08:34:45.531206+0000","last_undegraded":"2024-04-25T08:34:45.531206+0000","last_fullsized":"2024-04-25T08:34:45.531206+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:57.569222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093571+0000","last_change":"2024-04-25T08:34:44.532534+0000","last_active":"2024-04-25T08:34:46.093571+0000","last_peered":"2024-04-25T08:34:46.093571+0000","last_clean":"2024-04-25T08:34:46.093571+0000","last_became_active":"2024-04-25T08:34:44.532250+0000","last_became_peered":"2024-04-25T08:34:44.532250+0000","last_unstale":"2024-04-25T08:34:46.093571+0000","last_undegraded":"2024-04-25T08:34:46.093571+0000","last_fullsized":"2024-04-25T08:34:46.093571+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:11.760981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094802+0000","last_change":"2024-04-25T08:34:43.883681+0000","last_active":"2024-04-25T08:34:46.094802+0000","last_peered":"2024-04-25T08:34:46.094802+0000","last_clean":"2024-04-25T08:34:46.094802+0000","last_became_active":"2024-04-25T08:34:43.883385+0000","last_became_peered":"2024-04-25T08:34:43.883385+0000","last_unstale":"2024-04-25T08:34:46.094802+0000","last_undegraded":"2024-04-25T08:34:46.094802+0000","last_fullsized":"2024-04-25T08:34:46.094802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:17.727985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094016+0000","last_change":"2024-04-25T08:34:44.532945+0000","last_active":"2024-04-25T08:34:46.094016+0000","last_peered":"2024-04-25T08:34:46.094016+0000","last_clean":"2024-04-25T08:34:46.094016+0000","last_became_active":"2024-04-25T08:34:44.532563+0000","last_became_peered":"2024-04-25T08:34:44.532563+0000","last_unstale":"2024-04-25T08:34:46.094016+0000","last_undegraded":"2024-04-25T08:34:46.094016+0000","last_fullsized":"2024-04-25T08:34:46.094016+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:18.966389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530710+0000","last_change":"2024-04-25T08:34:43.882768+0000","last_active":"2024-04-25T08:34:45.530710+0000","last_peered":"2024-04-25T08:34:45.530710+0000","last_clean":"2024-04-25T08:34:45.530710+0000","last_became_active":"2024-04-25T08:34:43.882459+0000","last_became_peered":"2024-04-25T08:34:43.882459+0000","last_unstale":"2024-04-25T08:34:45.530710+0000","last_undegraded":"2024-04-25T08:34:45.530710+0000","last_fullsized":"2024-04-25T08:34:45.530710+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:46.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094908+0000","last_change":"2024-04-25T08:34:44.533491+0000","last_active":"2024-04-25T08:34:46.094908+0000","last_peered":"2024-04-25T08:34:46.094908+0000","last_clean":"2024-04-25T08:34:46.094908+0000","last_became_active":"2024-04-25T08:34:44.532257+0000","last_became_peered":"2024-04-25T08:34:44.532257+0000","last_unstale":"2024-04-25T08:34:46.094908+0000","last_undegraded":"2024-04-25T08:34:46.094908+0000","last_fullsized":"2024-04-25T08:34:46.094908+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:41.495853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095013+0000","last_change":"2024-04-25T08:34:43.889631+0000","last_active":"2024-04-25T08:34:46.095013+0000","last_peered":"2024-04-25T08:34:46.095013+0000","last_clean":"2024-04-25T08:34:46.095013+0000","last_became_active":"2024-04-25T08:34:43.889354+0000","last_became_peered":"2024-04-25T08:34:43.889354+0000","last_unstale":"2024-04-25T08:34:46.095013+0000","last_undegraded":"2024-04-25T08:34:46.095013+0000","last_fullsized":"2024-04-25T08:34:46.095013+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:02.862950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086134+0000","last_change":"2024-04-25T08:34:44.533026+0000","last_active":"2024-04-25T08:34:46.086134+0000","last_peered":"2024-04-25T08:34:46.086134+0000","last_clean":"2024-04-25T08:34:46.086134+0000","last_became_active":"2024-04-25T08:34:44.532759+0000","last_became_peered":"2024-04-25T08:34:44.532759+0000","last_unstale":"2024-04-25T08:34:46.086134+0000","last_undegraded":"2024-04-25T08:34:46.086134+0000","last_fullsized":"2024-04-25T08:34:46.086134+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:53.870186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531275+0000","last_change":"2024-04-25T08:34:44.532873+0000","last_active":"2024-04-25T08:34:45.531275+0000","last_peered":"2024-04-25T08:34:45.531275+0000","last_clean":"2024-04-25T08:34:45.531275+0000","last_became_active":"2024-04-25T08:34:44.532594+0000","last_became_peered":"2024-04-25T08:34:44.532594+0000","last_unstale":"2024-04-25T08:34:45.531275+0000","last_undegraded":"2024-04-25T08:34:45.531275+0000","last_fullsized":"2024-04-25T08:34:45.531275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:52.822815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530597+0000","last_change":"2024-04-25T08:34:43.877431+0000","last_active":"2024-04-25T08:34:45.530597+0000","last_peered":"2024-04-25T08:34:45.530597+0000","last_clean":"2024-04-25T08:34:45.530597+0000","last_became_active":"2024-04-25T08:34:43.877115+0000","last_became_peered":"2024-04-25T08:34:43.877115+0000","last_unstale":"2024-04-25T08:34:45.530597+0000","last_undegraded":"2024-04-25T08:34:45.530597+0000","last_fullsized":"2024-04-25T08:34:45.530597+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:55.485664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086192+0000","last_change":"2024-04-25T08:34:44.527817+0000","last_active":"2024-04-25T08:34:46.086192+0000","last_peered":"2024-04-25T08:34:46.086192+0000","last_clean":"2024-04-25T08:34:46.086192+0000","last_became_active":"2024-04-25T08:34:44.527550+0000","last_became_peered":"2024-04-25T08:34:44.527550+0000","last_unstale":"2024-04-25T08:34:46.086192+0000","last_undegraded":"2024-04-25T08:34:46.086192+0000","last_fullsized":"2024-04-25T08:34:46.086192+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:13.210050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530234+0000","last_change":"2024-04-25T08:34:43.883793+0000","last_active":"2024-04-25T08:34:45.530234+0000","last_peered":"2024-04-25T08:34:45.530234+0000","last_clean":"2024-04-25T08:34:45.530234+0000","last_became_active":"2024-04-25T08:34:43.882987+0000","last_became_peered":"2024-04-25T08:34:43.882987+0000","last_unstale":"2024-04-25T08:34:45.530234+0000","last_undegraded":"2024-04-25T08:34:45.530234+0000","last_fullsized":"2024-04-25T08:34:45.530234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:04.802285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093760+0000","last_change":"2024-04-25T08:34:44.528347+0000","last_active":"2024-04-25T08:34:46.093760+0000","last_peered":"2024-04-25T08:34:46.093760+0000","last_clean":"2024-04-25T08:34:46.093760+0000","last_became_active":"2024-04-25T08:34:44.528059+0000","last_became_peered":"2024-04-25T08:34:44.528059+0000","last_unstale":"2024-04-25T08:34:46.093760+0000","last_undegraded":"2024-04-25T08:34:46.093760+0000","last_fullsized":"2024-04-25T08:34:46.093760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.919973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086286+0000","last_change":"2024-04-25T08:34:43.878609+0000","last_active":"2024-04-25T08:34:46.086286+0000","last_peered":"2024-04-25T08:34:46.086286+0000","last_clean":"2024-04-25T08:34:46.086286+0000","last_became_active":"2024-04-25T08:34:43.878322+0000","last_became_peered":"2024-04-25T08:34:43.878322+0000","last_unstale":"2024-04-25T08:34:46.086286+0000","last_undegraded":"2024-04-25T08:34:46.086286+0000","last_fullsized":"2024-04-25T08:34:46.086286+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:46.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530698+0000","last_change":"2024-04-25T08:34:44.519833+0000","last_active":"2024-04-25T08:34:45.530698+0000","last_peered":"2024-04-25T08:34:45.530698+0000","last_clean":"2024-04-25T08:34:45.530698+0000","last_became_active":"2024-04-25T08:34:44.519535+0000","last_became_peered":"2024-04-25T08:34:44.519535+0000","last_unstale":"2024-04-25T08:34:45.530698+0000","last_undegraded":"2024-04-25T08:34:45.530698+0000","last_fullsized":"2024-04-25T08:34:45.530698+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.499854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094705+0000","last_change":"2024-04-25T08:34:44.527976+0000","last_active":"2024-04-25T08:34:46.094705+0000","last_peered":"2024-04-25T08:34:46.094705+0000","last_clean":"2024-04-25T08:34:46.094705+0000","last_became_active":"2024-04-25T08:34:44.527199+0000","last_became_peered":"2024-04-25T08:34:44.527199+0000","last_unstale":"2024-04-25T08:34:46.094705+0000","last_undegraded":"2024-04-25T08:34:46.094705+0000","last_fullsized":"2024-04-25T08:34:46.094705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:26.675283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094290+0000","last_change":"2024-04-25T08:34:43.880717+0000","last_active":"2024-04-25T08:34:46.094290+0000","last_peered":"2024-04-25T08:34:46.094290+0000","last_clean":"2024-04-25T08:34:46.094290+0000","last_became_active":"2024-04-25T08:34:43.880417+0000","last_became_peered":"2024-04-25T08:34:43.880417+0000","last_unstale":"2024-04-25T08:34:46.094290+0000","last_undegraded":"2024-04-25T08:34:46.094290+0000","last_fullsized":"2024-04-25T08:34:46.094290+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:29.445107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086364+0000","last_change":"2024-04-25T08:34:44.532688+0000","last_active":"2024-04-25T08:34:46.086364+0000","last_peered":"2024-04-25T08:34:46.086364+0000","last_clean":"2024-04-25T08:34:46.086364+0000","last_became_active":"2024-04-25T08:34:44.532430+0000","last_became_peered":"2024-04-25T08:34:44.532430+0000","last_unstale":"2024-04-25T08:34:46.086364+0000","last_undegraded":"2024-04-25T08:34:46.086364+0000","last_fullsized":"2024-04-25T08:34:46.086364+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:19.629450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093707+0000","last_change":"2024-04-25T08:34:44.532922+0000","last_active":"2024-04-25T08:34:46.093707+0000","last_peered":"2024-04-25T08:34:46.093707+0000","last_clean":"2024-04-25T08:34:46.093707+0000","last_became_active":"2024-04-25T08:34:44.532361+0000","last_became_peered":"2024-04-25T08:34:44.532361+0000","last_unstale":"2024-04-25T08:34:46.093707+0000","last_undegraded":"2024-04-25T08:34:46.093707+0000","last_fullsized":"2024-04-25T08:34:46.093707+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:05.622445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530587+0000","last_change":"2024-04-25T08:34:43.881524+0000","last_active":"2024-04-25T08:34:45.530587+0000","last_peered":"2024-04-25T08:34:45.530587+0000","last_clean":"2024-04-25T08:34:45.530587+0000","last_became_active":"2024-04-25T08:34:43.881273+0000","last_became_peered":"2024-04-25T08:34:43.881273+0000","last_unstale":"2024-04-25T08:34:45.530587+0000","last_undegraded":"2024-04-25T08:34:45.530587+0000","last_fullsized":"2024-04-25T08:34:45.530587+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:54.880133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086187+0000","last_change":"2024-04-25T08:34:44.531176+0000","last_active":"2024-04-25T08:34:46.086187+0000","last_peered":"2024-04-25T08:34:46.086187+0000","last_clean":"2024-04-25T08:34:46.086187+0000","last_became_active":"2024-04-25T08:34:44.529858+0000","last_became_peered":"2024-04-25T08:34:44.529858+0000","last_unstale":"2024-04-25T08:34:46.086187+0000","last_undegraded":"2024-04-25T08:34:46.086187+0000","last_fullsized":"2024-04-25T08:34:46.086187+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:03.865071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093840+0000","last_change":"2024-04-25T08:34:43.875641+0000","last_active":"2024-04-25T08:34:46.093840+0000","last_peered":"2024-04-25T08:34:46.093840+0000","last_clean":"2024-04-25T08:34:46.093840+0000","last_became_active":"2024-04-25T08:34:43.875344+0000","last_became_peered":"2024-04-25T08:34:43.875344+0000","last_unstale":"2024-04-25T08:34:46.093840+0000","last_undegraded":"2024-04-25T08:34:46.093840+0000","last_fullsized":"2024-04-25T08:34:46.093840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:44.497733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093679+0000","last_change":"2024-04-25T08:34:44.532615+0000","last_active":"2024-04-25T08:34:46.093679+0000","last_peered":"2024-04-25T08:34:46.093679+0000","last_clean":"2024-04-25T08:34:46.093679+0000","last_became_active":"2024-04-25T08:34:44.532335+0000","last_became_peered":"2024-04-25T08:34:44.532335+0000","last_unstale":"2024-04-25T08:34:46.093679+0000","last_undegraded":"2024-04-25T08:34:46.093679+0000","last_fullsized":"2024-04-25T08:34:46.093679+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:05.252545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093705+0000","last_change":"2024-04-25T08:34:44.529316+0000","last_active":"2024-04-25T08:34:46.093705+0000","last_peered":"2024-04-25T08:34:46.093705+0000","last_clean":"2024-04-25T08:34:46.093705+0000","last_became_active":"2024-04-25T08:34:44.529026+0000","last_became_peered":"2024-04-25T08:34:44.529026+0000","last_unstale":"2024-04-25T08:34:46.093705+0000","last_undegraded":"2024-04-25T08:34:46.093705+0000","last_fullsized":"2024-04-25T08:34:46.093705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:33.006310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094563+0000","last_change":"2024-04-25T08:34:44.536436+0000","last_active":"2024-04-25T08:34:46.094563+0000","last_peered":"2024-04-25T08:34:46.094563+0000","last_clean":"2024-04-25T08:34:46.094563+0000","last_became_active":"2024-04-25T08:34:44.534485+0000","last_became_peered":"2024-04-25T08:34:44.534485+0000","last_unstale":"2024-04-25T08:34:46.094563+0000","last_undegraded":"2024-04-25T08:34:46.094563+0000","last_fullsized":"2024-04-25T08:34:46.094563+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:31.481085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531115+0000","last_change":"2024-04-25T08:34:43.883072+0000","last_active":"2024-04-25T08:34:45.531115+0000","last_peered":"2024-04-25T08:34:45.531115+0000","last_clean":"2024-04-25T08:34:45.531115+0000","last_became_active":"2024-04-25T08:34:43.882688+0000","last_became_peered":"2024-04-25T08:34:43.882688+0000","last_unstale":"2024-04-25T08:34:45.531115+0000","last_undegraded":"2024-04-25T08:34:45.531115+0000","last_fullsized":"2024-04-25T08:34:45.531115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:16.493814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093700+0000","last_change":"2024-04-25T08:34:43.883730+0000","last_active":"2024-04-25T08:34:46.093700+0000","last_peered":"2024-04-25T08:34:46.093700+0000","last_clean":"2024-04-25T08:34:46.093700+0000","last_became_active":"2024-04-25T08:34:43.883435+0000","last_became_peered":"2024-04-25T08:34:43.883435+0000","last_unstale":"2024-04-25T08:34:46.093700+0000","last_undegraded":"2024-04-25T08:34:46.093700+0000","last_fullsized":"2024-04-25T08:34:46.093700+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:10.731254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530395+0000","last_change":"2024-04-25T08:34:44.526203+0000","last_active":"2024-04-25T08:34:45.530395+0000","last_peered":"2024-04-25T08:34:45.530395+0000","last_clean":"2024-04-25T08:34:45.530395+0000","last_became_active":"2024-04-25T08:34:44.525945+0000","last_became_peered":"2024-04-25T08:34:44.525945+0000","last_unstale":"2024-04-25T08:34:45.530395+0000","last_undegraded":"2024-04-25T08:34:45.530395+0000","last_fullsized":"2024-04-25T08:34:45.530395+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.447228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.869817+0000","last_change":"2024-04-25T08:34:43.881671+0000","last_active":"2024-04-25T08:34:46.869817+0000","last_peered":"2024-04-25T08:34:46.869817+0000","last_clean":"2024-04-25T08:34:46.869817+0000","last_became_active":"2024-04-25T08:34:43.881375+0000","last_became_peered":"2024-04-25T08:34:43.881375+0000","last_unstale":"2024-04-25T08:34:46.869817+0000","last_undegraded":"2024-04-25T08:34:46.869817+0000","last_fullsized":"2024-04-25T08:34:46.869817+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:48.237385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":227,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T08:34:42.092335+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:34:42.092335+0000","last_peered":"2024-04-25T08:34:42.092335+0000","last_clean":"2024-04-25T08:34:42.092335+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:34:42.092335+0000","last_undegraded":"2024-04-25T08:34:42.092335+0000","last_fullsized":"2024-04-25T08:34:42.092335+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":86,"acting":86,"num_store_stats":5},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":708,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":42,"acting":42,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58704,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313136,"statfs":{"total":96636764160,"available":96576651264,"internally_reserved":0,"allocated":1175552,"data_stored":847826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40144,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331696,"statfs":{"total":96636764160,"available":96595656704,"internally_reserved":0,"allocated":651264,"data_stored":290364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35972,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335868,"statfs":{"total":96636764160,"available":96599928832,"internally_reserved":0,"allocated":643072,"data_stored":289712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58184,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313656,"statfs":{"total":96636764160,"available":96577183744,"internally_reserved":0,"allocated":647168,"data_stored":290294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44244,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327596,"statfs":{"total":96636764160,"available":96591458304,"internally_reserved":0,"allocated":593920,"data_stored":262977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40084,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331756,"statfs":{"total":96636764160,"available":96595718144,"internally_reserved":0,"allocated":593920,"data_stored":262977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":651264,"data_stored":289872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50444,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321396,"statfs":{"total":96636764160,"available":96585109504,"internally_reserved":0,"allocated":1245184,"data_stored":880136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:34:47.868 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:34:48.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.007+0000 7f3c36f29640 1 Processor -- start 2024-04-25T08:34:48.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.007+0000 7f3c36f29640 1 -- start start 2024-04-25T08:34:48.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.008+0000 7f3c36f29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c3015be80 0x7f3c300b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.008+0000 7f3c36f29640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c300cd690 con 0x7f3c30154cb0 2024-04-25T08:34:48.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.008+0000 7f3c36f29640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c300cd970 con 0x7f3c300b4e60 2024-04-25T08:34:48.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.008+0000 7f3c36f29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300b52d0 con 0x7f3c3015be80 2024-04-25T08:34:48.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.008+0000 7f3c34cc5640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f3c30154cb0 0x7f3c300c4ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33272/0 (socket says 172.21.15.38:33272) 2024-04-25T08:34:48.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.008+0000 7f3c34cc5640 1 -- 172.21.15.38:0/3650828445 learned_addr learned my addr 172.21.15.38:0/3650828445 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:48.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.008+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243361234 0 0) 0x7f3c300cd690 con 0x7f3c30154cb0 2024-04-25T08:34:48.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c18003520 con 0x7f3c30154cb0 2024-04-25T08:34:48.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2ffff640 1 --2- 172.21.15.38:0/3650828445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c3015be80 0x7f3c300b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3880440644 0 0) 0x7f3c18003520 con 0x7f3c30154cb0 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c3015be80 msgr2=0x7f3c300b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 --2- 172.21.15.38:0/3650828445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c3015be80 0x7f3c300b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 >> v1:172.21.15.107:6789/0 conn(0x7f3c300b4e60 legacy=0x7f3c300ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c300cdc50 con 0x7f3c30154cb0 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f3c280019f0 con 0x7f3c30154cb0 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 >> v1:172.21.15.38:6789/0 conn(0x7f3c30154cb0 legacy=0x7f3c300c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 --2- 172.21.15.38:0/3650828445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c180039e0 0x7f3c18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 --2- 172.21.15.38:0/3650828445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c3015be80 0x7f3c18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 --2- 172.21.15.38:0/3650828445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c18004d40 0x7f3c180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300b52d0 con 0x7f3c18004d40 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1800baa0 con 0x7f3c180039e0 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1800deb0 con 0x7f3c3015be80 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c354c6640 1 --2- 172.21.15.38:0/3650828445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c18004d40 0x7f3c180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c34cc5640 1 --2- 172.21.15.38:0/3650828445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c180039e0 0x7f3c18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c2ffff640 1 --2- 172.21.15.38:0/3650828445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c3015be80 0x7f3c18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c354c6640 1 -- 172.21.15.38:0/3650828445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c3015be80 msgr2=0x7f3c18004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c354c6640 1 --2- 172.21.15.38:0/3650828445 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c3015be80 0x7f3c18004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c354c6640 1 -- 172.21.15.38:0/3650828445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c180039e0 msgr2=0x7f3c18003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c354c6640 1 --2- 172.21.15.38:0/3650828445 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c180039e0 0x7f3c18003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.009+0000 7f3c354c6640 1 -- 172.21.15.38:0/3650828445 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c18003520 con 0x7f3c18004d40 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.010+0000 7f3c354c6640 1 --2- 172.21.15.38:0/3650828445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c18004d40 0x7f3c180051c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3c20001720 tx=0x7f3c18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c945129a86959b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.010+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20001370 con 0x7f3c18004d40 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.010+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c200052d0 con 0x7f3c18004d40 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.010+0000 7f3c2f7fe640 1 -- 172.21.15.38:0/3650828445 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2002abc0 con 0x7f3c18004d40 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.011+0000 7f3c36f29640 1 -- 172.21.15.38:0/3650828445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c18004d40 msgr2=0x7f3c180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.011+0000 7f3c36f29640 1 --2- 172.21.15.38:0/3650828445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c18004d40 0x7f3c180051c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f3c20001720 tx=0x7f3c18005830 comp rx=0 tx=0).stop 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.011+0000 7f3c354c6640 1 -- 172.21.15.38:0/3650828445 reap_dead start 2024-04-25T08:34:48.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.011+0000 7f3c36f29640 1 -- 172.21.15.38:0/3650828445 shutdown_connections 2024-04-25T08:34:48.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.011+0000 7f3c36f29640 1 -- 172.21.15.38:0/3650828445 >> 172.21.15.38:0/3650828445 conn(0x7f3c3006b210 msgr2=0x7f3c3006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:48.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.011+0000 7f3c36f29640 1 -- 172.21.15.38:0/3650828445 shutdown_connections 2024-04-25T08:34:48.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.011+0000 7f3c36f29640 1 -- 172.21.15.38:0/3650828445 wait complete. 2024-04-25T08:34:48.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.012+0000 7f3c36f29640 1 Processor -- start 2024-04-25T08:34:48.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.012+0000 7f3c36f29640 1 -- start start 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c36f29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c18004d40 0x7f3c3014b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c36f29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c30159020 0x7f3c301594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c36f29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c301ee9f0 0x7f3c301eee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c36f29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3015c330 con 0x7f3c18004d40 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c36f29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300cd440 con 0x7f3c30159020 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c36f29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300ce210 con 0x7f3c301ee9f0 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c34cc5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c18004d40 0x7f3c3014b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c34cc5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c18004d40 0x7f3c3014b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51056/0 (socket says 172.21.15.38:51056) 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c34cc5640 1 -- 172.21.15.38:0/2997357340 learned_addr learned my addr 172.21.15.38:0/2997357340 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c2ffff640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c30159020 0x7f3c301594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c354c6640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c301ee9f0 0x7f3c301eee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c34cc5640 1 -- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c301ee9f0 msgr2=0x7f3c301eee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c34cc5640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c301ee9f0 0x7f3c301eee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c34cc5640 1 -- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c30159020 msgr2=0x7f3c301594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c34cc5640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c30159020 0x7f3c301594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c34cc5640 1 -- 172.21.15.38:0/2997357340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c301ef770 con 0x7f3c18004d40 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c2ffff640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c30159020 0x7f3c301594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.013+0000 7f3c34cc5640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c18004d40 0x7f3c3014b0e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3c280079d0 tx=0x7f3c2800a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.014+0000 7f3c2d7fa640 1 -- 172.21.15.38:0/2997357340 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c28017020 con 0x7f3c18004d40 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.014+0000 7f3c2d7fa640 1 -- 172.21.15.38:0/2997357340 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c28007ac0 con 0x7f3c18004d40 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.014+0000 7f3c36f29640 1 -- 172.21.15.38:0/2997357340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c301efa50 con 0x7f3c18004d40 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.014+0000 7f3c2d7fa640 1 -- 172.21.15.38:0/2997357340 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c28013610 con 0x7f3c18004d40 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.014+0000 7f3c36f29640 1 -- 172.21.15.38:0/2997357340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c301f0010 con 0x7f3c18004d40 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.015+0000 7f3c36f29640 1 -- 172.21.15.38:0/2997357340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c30068e80 con 0x7f3c18004d40 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.015+0000 7f3c2d7fa640 1 -- 172.21.15.38:0/2997357340 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3c2801a020 con 0x7f3c18004d40 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.016+0000 7f3c2d7fa640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3c1c06b500 0x7f3c1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.016+0000 7f3c2d7fa640 1 -- 172.21.15.38:0/2997357340 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3c2808b010 con 0x7f3c18004d40 2024-04-25T08:34:48.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.016+0000 7f3c2ffff640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3c1c06b500 0x7f3c1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.016+0000 7f3c2ffff640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3c1c06b500 0x7f3c1c06d9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3c3006e3c0 tx=0x7f3c24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:48.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.020+0000 7f3c2d7fa640 1 -- 172.21.15.38:0/2997357340 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3c28021020 con 0x7f3c18004d40 2024-04-25T08:34:48.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.196+0000 7f3c36f29640 1 -- 172.21.15.38:0/2997357340 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3c300cd010 con 0x7f3c1c06b500 2024-04-25T08:34:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.203+0000 7f3c2d7fa640 1 -- 172.21.15.38:0/2997357340 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246879 (secure 0 0 0) 0x7f3c300cd010 con 0x7f3c1c06b500 2024-04-25T08:34:48.200 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:48.201 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:34:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.205+0000 7f3c36f29640 1 -- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3c1c06b500 msgr2=0x7f3c1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.205+0000 7f3c36f29640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3c1c06b500 0x7f3c1c06d9c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3c3006e3c0 tx=0x7f3c24008040 comp rx=0 tx=0).stop 2024-04-25T08:34:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.205+0000 7f3c36f29640 1 -- 172.21.15.38:0/2997357340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c18004d40 msgr2=0x7f3c3014b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.205+0000 7f3c36f29640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c18004d40 0x7f3c3014b0e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f3c280079d0 tx=0x7f3c2800a730 comp rx=0 tx=0).stop 2024-04-25T08:34:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.206+0000 7f3c36f29640 1 -- 172.21.15.38:0/2997357340 shutdown_connections 2024-04-25T08:34:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.206+0000 7f3c36f29640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3c1c06b500 0x7f3c1c06d9c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.206+0000 7f3c36f29640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3c301ee9f0 0x7f3c301eee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.206+0000 7f3c36f29640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3c30159020 0x7f3c301594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.206+0000 7f3c36f29640 1 --2- 172.21.15.38:0/2997357340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3c18004d40 0x7f3c3014b0e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.206+0000 7f3c36f29640 1 -- 172.21.15.38:0/2997357340 >> 172.21.15.38:0/2997357340 conn(0x7f3c3006b210 msgr2=0x7f3c300b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.206+0000 7f3c36f29640 1 -- 172.21.15.38:0/2997357340 shutdown_connections 2024-04-25T08:34:48.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.206+0000 7f3c36f29640 1 -- 172.21.15.38:0/2997357340 wait complete. 2024-04-25T08:34:48.211 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-04-25T08:34:47.249503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591076,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":179,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":367916,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":361715,"kb_avail":754606804,"statfs":{"total":773094113280,"available":772717367296,"internally_reserved":0,"allocated":6201344,"data_stored":3414158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":370396720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":708,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.393179"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086305+0000","last_change":"2024-04-25T08:34:44.527955+0000","last_active":"2024-04-25T08:34:46.086305+0000","last_peered":"2024-04-25T08:34:46.086305+0000","last_clean":"2024-04-25T08:34:46.086305+0000","last_became_active":"2024-04-25T08:34:44.527662+0000","last_became_peered":"2024-04-25T08:34:44.527662+0000","last_unstale":"2024-04-25T08:34:46.086305+0000","last_undegraded":"2024-04-25T08:34:46.086305+0000","last_fullsized":"2024-04-25T08:34:46.086305+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:09.895753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093741+0000","last_change":"2024-04-25T08:34:44.534121+0000","last_active":"2024-04-25T08:34:46.093741+0000","last_peered":"2024-04-25T08:34:46.093741+0000","last_clean":"2024-04-25T08:34:46.093741+0000","last_became_active":"2024-04-25T08:34:44.533765+0000","last_became_peered":"2024-04-25T08:34:44.533765+0000","last_unstale":"2024-04-25T08:34:46.093741+0000","last_undegraded":"2024-04-25T08:34:46.093741+0000","last_fullsized":"2024-04-25T08:34:46.093741+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.333087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531042+0000","last_change":"2024-04-25T08:34:44.520910+0000","last_active":"2024-04-25T08:34:45.531042+0000","last_peered":"2024-04-25T08:34:45.531042+0000","last_clean":"2024-04-25T08:34:45.531042+0000","last_became_active":"2024-04-25T08:34:44.520607+0000","last_became_peered":"2024-04-25T08:34:44.520607+0000","last_unstale":"2024-04-25T08:34:45.531042+0000","last_undegraded":"2024-04-25T08:34:45.531042+0000","last_fullsized":"2024-04-25T08:34:45.531042+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:43.933652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086256+0000","last_change":"2024-04-25T08:34:44.527358+0000","last_active":"2024-04-25T08:34:46.086256+0000","last_peered":"2024-04-25T08:34:46.086256+0000","last_clean":"2024-04-25T08:34:46.086256+0000","last_became_active":"2024-04-25T08:34:44.527098+0000","last_became_peered":"2024-04-25T08:34:44.527098+0000","last_unstale":"2024-04-25T08:34:46.086256+0000","last_undegraded":"2024-04-25T08:34:46.086256+0000","last_fullsized":"2024-04-25T08:34:46.086256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:47.550727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094066+0000","last_change":"2024-04-25T08:34:44.536974+0000","last_active":"2024-04-25T08:34:46.094066+0000","last_peered":"2024-04-25T08:34:46.094066+0000","last_clean":"2024-04-25T08:34:46.094066+0000","last_became_active":"2024-04-25T08:34:44.536690+0000","last_became_peered":"2024-04-25T08:34:44.536690+0000","last_unstale":"2024-04-25T08:34:46.094066+0000","last_undegraded":"2024-04-25T08:34:46.094066+0000","last_fullsized":"2024-04-25T08:34:46.094066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:27.300505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086137+0000","last_change":"2024-04-25T08:34:44.533732+0000","last_active":"2024-04-25T08:34:46.086137+0000","last_peered":"2024-04-25T08:34:46.086137+0000","last_clean":"2024-04-25T08:34:46.086137+0000","last_became_active":"2024-04-25T08:34:44.533441+0000","last_became_peered":"2024-04-25T08:34:44.533441+0000","last_unstale":"2024-04-25T08:34:46.086137+0000","last_undegraded":"2024-04-25T08:34:46.086137+0000","last_fullsized":"2024-04-25T08:34:46.086137+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:11.733375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086171+0000","last_change":"2024-04-25T08:34:44.531394+0000","last_active":"2024-04-25T08:34:46.086171+0000","last_peered":"2024-04-25T08:34:46.086171+0000","last_clean":"2024-04-25T08:34:46.086171+0000","last_became_active":"2024-04-25T08:34:44.531057+0000","last_became_peered":"2024-04-25T08:34:44.531057+0000","last_unstale":"2024-04-25T08:34:46.086171+0000","last_undegraded":"2024-04-25T08:34:46.086171+0000","last_fullsized":"2024-04-25T08:34:46.086171+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:18.730106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094348+0000","last_change":"2024-04-25T08:34:44.533220+0000","last_active":"2024-04-25T08:34:46.094348+0000","last_peered":"2024-04-25T08:34:46.094348+0000","last_clean":"2024-04-25T08:34:46.094348+0000","last_became_active":"2024-04-25T08:34:44.532647+0000","last_became_peered":"2024-04-25T08:34:44.532647+0000","last_unstale":"2024-04-25T08:34:46.094348+0000","last_undegraded":"2024-04-25T08:34:46.094348+0000","last_fullsized":"2024-04-25T08:34:46.094348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:00.701022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086067+0000","last_change":"2024-04-25T08:34:44.525673+0000","last_active":"2024-04-25T08:34:46.086067+0000","last_peered":"2024-04-25T08:34:46.086067+0000","last_clean":"2024-04-25T08:34:46.086067+0000","last_became_active":"2024-04-25T08:34:44.525384+0000","last_became_peered":"2024-04-25T08:34:44.525384+0000","last_unstale":"2024-04-25T08:34:46.086067+0000","last_undegraded":"2024-04-25T08:34:46.086067+0000","last_fullsized":"2024-04-25T08:34:46.086067+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.499854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095168+0000","last_change":"2024-04-25T08:34:44.533259+0000","last_active":"2024-04-25T08:34:46.095168+0000","last_peered":"2024-04-25T08:34:46.095168+0000","last_clean":"2024-04-25T08:34:46.095168+0000","last_became_active":"2024-04-25T08:34:44.531589+0000","last_became_peered":"2024-04-25T08:34:44.531589+0000","last_unstale":"2024-04-25T08:34:46.095168+0000","last_undegraded":"2024-04-25T08:34:46.095168+0000","last_fullsized":"2024-04-25T08:34:46.095168+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:01.549992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093885+0000","last_change":"2024-04-25T08:34:44.530163+0000","last_active":"2024-04-25T08:34:46.093885+0000","last_peered":"2024-04-25T08:34:46.093885+0000","last_clean":"2024-04-25T08:34:46.093885+0000","last_became_active":"2024-04-25T08:34:44.529905+0000","last_became_peered":"2024-04-25T08:34:44.529905+0000","last_unstale":"2024-04-25T08:34:46.093885+0000","last_undegraded":"2024-04-25T08:34:46.093885+0000","last_fullsized":"2024-04-25T08:34:46.093885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:18.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529970+0000","last_change":"2024-04-25T08:34:44.520405+0000","last_active":"2024-04-25T08:34:45.529970+0000","last_peered":"2024-04-25T08:34:45.529970+0000","last_clean":"2024-04-25T08:34:45.529970+0000","last_became_active":"2024-04-25T08:34:44.520122+0000","last_became_peered":"2024-04-25T08:34:44.520122+0000","last_unstale":"2024-04-25T08:34:45.529970+0000","last_undegraded":"2024-04-25T08:34:45.529970+0000","last_fullsized":"2024-04-25T08:34:45.529970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:09.811944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530067+0000","last_change":"2024-04-25T08:34:44.522344+0000","last_active":"2024-04-25T08:34:45.530067+0000","last_peered":"2024-04-25T08:34:45.530067+0000","last_clean":"2024-04-25T08:34:45.530067+0000","last_became_active":"2024-04-25T08:34:44.522063+0000","last_became_peered":"2024-04-25T08:34:44.522063+0000","last_unstale":"2024-04-25T08:34:45.530067+0000","last_undegraded":"2024-04-25T08:34:45.530067+0000","last_fullsized":"2024-04-25T08:34:45.530067+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:25.590051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093628+0000","last_change":"2024-04-25T08:34:44.536670+0000","last_active":"2024-04-25T08:34:46.093628+0000","last_peered":"2024-04-25T08:34:46.093628+0000","last_clean":"2024-04-25T08:34:46.093628+0000","last_became_active":"2024-04-25T08:34:44.536110+0000","last_became_peered":"2024-04-25T08:34:44.536110+0000","last_unstale":"2024-04-25T08:34:46.093628+0000","last_undegraded":"2024-04-25T08:34:46.093628+0000","last_fullsized":"2024-04-25T08:34:46.093628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:43.198107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095058+0000","last_change":"2024-04-25T08:34:44.533241+0000","last_active":"2024-04-25T08:34:46.095058+0000","last_peered":"2024-04-25T08:34:46.095058+0000","last_clean":"2024-04-25T08:34:46.095058+0000","last_became_active":"2024-04-25T08:34:44.531910+0000","last_became_peered":"2024-04-25T08:34:44.531910+0000","last_unstale":"2024-04-25T08:34:46.095058+0000","last_undegraded":"2024-04-25T08:34:46.095058+0000","last_fullsized":"2024-04-25T08:34:46.095058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.963128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086493+0000","last_change":"2024-04-25T08:34:44.538453+0000","last_active":"2024-04-25T08:34:46.086493+0000","last_peered":"2024-04-25T08:34:46.086493+0000","last_clean":"2024-04-25T08:34:46.086493+0000","last_became_active":"2024-04-25T08:34:44.538165+0000","last_became_peered":"2024-04-25T08:34:44.538165+0000","last_unstale":"2024-04-25T08:34:46.086493+0000","last_undegraded":"2024-04-25T08:34:46.086493+0000","last_fullsized":"2024-04-25T08:34:46.086493+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:46.558329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530135+0000","last_change":"2024-04-25T08:34:44.528517+0000","last_active":"2024-04-25T08:34:45.530135+0000","last_peered":"2024-04-25T08:34:45.530135+0000","last_clean":"2024-04-25T08:34:45.530135+0000","last_became_active":"2024-04-25T08:34:44.528229+0000","last_became_peered":"2024-04-25T08:34:44.528229+0000","last_unstale":"2024-04-25T08:34:45.530135+0000","last_undegraded":"2024-04-25T08:34:45.530135+0000","last_fullsized":"2024-04-25T08:34:45.530135+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:41.585740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094613+0000","last_change":"2024-04-25T08:34:44.528023+0000","last_active":"2024-04-25T08:34:46.094613+0000","last_peered":"2024-04-25T08:34:46.094613+0000","last_clean":"2024-04-25T08:34:46.094613+0000","last_became_active":"2024-04-25T08:34:44.527668+0000","last_became_peered":"2024-04-25T08:34:44.527668+0000","last_unstale":"2024-04-25T08:34:46.094613+0000","last_undegraded":"2024-04-25T08:34:46.094613+0000","last_fullsized":"2024-04-25T08:34:46.094613+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:06.085151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094200+0000","last_change":"2024-04-25T08:34:44.533947+0000","last_active":"2024-04-25T08:34:46.094200+0000","last_peered":"2024-04-25T08:34:46.094200+0000","last_clean":"2024-04-25T08:34:46.094200+0000","last_became_active":"2024-04-25T08:34:44.533609+0000","last_became_peered":"2024-04-25T08:34:44.533609+0000","last_unstale":"2024-04-25T08:34:46.094200+0000","last_undegraded":"2024-04-25T08:34:46.094200+0000","last_fullsized":"2024-04-25T08:34:46.094200+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:44.889254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093558+0000","last_change":"2024-04-25T08:34:44.532723+0000","last_active":"2024-04-25T08:34:46.093558+0000","last_peered":"2024-04-25T08:34:46.093558+0000","last_clean":"2024-04-25T08:34:46.093558+0000","last_became_active":"2024-04-25T08:34:44.531850+0000","last_became_peered":"2024-04-25T08:34:44.531850+0000","last_unstale":"2024-04-25T08:34:46.093558+0000","last_undegraded":"2024-04-25T08:34:46.093558+0000","last_fullsized":"2024-04-25T08:34:46.093558+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:20.979454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529837+0000","last_change":"2024-04-25T08:34:43.873145+0000","last_active":"2024-04-25T08:34:45.529837+0000","last_peered":"2024-04-25T08:34:45.529837+0000","last_clean":"2024-04-25T08:34:45.529837+0000","last_became_active":"2024-04-25T08:34:43.872860+0000","last_became_peered":"2024-04-25T08:34:43.872860+0000","last_unstale":"2024-04-25T08:34:45.529837+0000","last_undegraded":"2024-04-25T08:34:45.529837+0000","last_fullsized":"2024-04-25T08:34:45.529837+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:39.042852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093598+0000","last_change":"2024-04-25T08:34:44.530047+0000","last_active":"2024-04-25T08:34:46.093598+0000","last_peered":"2024-04-25T08:34:46.093598+0000","last_clean":"2024-04-25T08:34:46.093598+0000","last_became_active":"2024-04-25T08:34:44.529781+0000","last_became_peered":"2024-04-25T08:34:44.529781+0000","last_unstale":"2024-04-25T08:34:46.093598+0000","last_undegraded":"2024-04-25T08:34:46.093598+0000","last_fullsized":"2024-04-25T08:34:46.093598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:36.970964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086257+0000","last_change":"2024-04-25T08:34:43.884678+0000","last_active":"2024-04-25T08:34:46.086257+0000","last_peered":"2024-04-25T08:34:46.086257+0000","last_clean":"2024-04-25T08:34:46.086257+0000","last_became_active":"2024-04-25T08:34:43.884357+0000","last_became_peered":"2024-04-25T08:34:43.884357+0000","last_unstale":"2024-04-25T08:34:46.086257+0000","last_undegraded":"2024-04-25T08:34:46.086257+0000","last_fullsized":"2024-04-25T08:34:46.086257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:04.802285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530601+0000","last_change":"2024-04-25T08:34:44.519962+0000","last_active":"2024-04-25T08:34:45.530601+0000","last_peered":"2024-04-25T08:34:45.530601+0000","last_clean":"2024-04-25T08:34:45.530601+0000","last_became_active":"2024-04-25T08:34:44.519671+0000","last_became_peered":"2024-04-25T08:34:44.519671+0000","last_unstale":"2024-04-25T08:34:45.530601+0000","last_undegraded":"2024-04-25T08:34:45.530601+0000","last_fullsized":"2024-04-25T08:34:45.530601+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:49.239506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086556+0000","last_change":"2024-04-25T08:34:43.878220+0000","last_active":"2024-04-25T08:34:46.086556+0000","last_peered":"2024-04-25T08:34:46.086556+0000","last_clean":"2024-04-25T08:34:46.086556+0000","last_became_active":"2024-04-25T08:34:43.877915+0000","last_became_peered":"2024-04-25T08:34:43.877915+0000","last_unstale":"2024-04-25T08:34:46.086556+0000","last_undegraded":"2024-04-25T08:34:46.086556+0000","last_fullsized":"2024-04-25T08:34:46.086556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:55.485664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093702+0000","last_change":"2024-04-25T08:34:43.882922+0000","last_active":"2024-04-25T08:34:46.093702+0000","last_peered":"2024-04-25T08:34:46.093702+0000","last_clean":"2024-04-25T08:34:46.093702+0000","last_became_active":"2024-04-25T08:34:43.882609+0000","last_became_peered":"2024-04-25T08:34:43.882609+0000","last_unstale":"2024-04-25T08:34:46.093702+0000","last_undegraded":"2024-04-25T08:34:46.093702+0000","last_fullsized":"2024-04-25T08:34:46.093702+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:17.727985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531083+0000","last_change":"2024-04-25T08:34:44.532481+0000","last_active":"2024-04-25T08:34:45.531083+0000","last_peered":"2024-04-25T08:34:45.531083+0000","last_clean":"2024-04-25T08:34:45.531083+0000","last_became_active":"2024-04-25T08:34:44.532204+0000","last_became_peered":"2024-04-25T08:34:44.532204+0000","last_unstale":"2024-04-25T08:34:45.531083+0000","last_undegraded":"2024-04-25T08:34:45.531083+0000","last_fullsized":"2024-04-25T08:34:45.531083+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:19.590018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094711+0000","last_change":"2024-04-25T08:34:43.883536+0000","last_active":"2024-04-25T08:34:46.094711+0000","last_peered":"2024-04-25T08:34:46.094711+0000","last_clean":"2024-04-25T08:34:46.094711+0000","last_became_active":"2024-04-25T08:34:43.883232+0000","last_became_peered":"2024-04-25T08:34:43.883232+0000","last_unstale":"2024-04-25T08:34:46.094711+0000","last_undegraded":"2024-04-25T08:34:46.094711+0000","last_fullsized":"2024-04-25T08:34:46.094711+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:10.731254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531305+0000","last_change":"2024-04-25T08:34:43.875476+0000","last_active":"2024-04-25T08:34:45.531305+0000","last_peered":"2024-04-25T08:34:45.531305+0000","last_clean":"2024-04-25T08:34:45.531305+0000","last_became_active":"2024-04-25T08:34:43.874920+0000","last_became_peered":"2024-04-25T08:34:43.874920+0000","last_unstale":"2024-04-25T08:34:45.531305+0000","last_undegraded":"2024-04-25T08:34:45.531305+0000","last_fullsized":"2024-04-25T08:34:45.531305+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:54.880133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530427+0000","last_change":"2024-04-25T08:34:43.869200+0000","last_active":"2024-04-25T08:34:45.530427+0000","last_peered":"2024-04-25T08:34:45.530427+0000","last_clean":"2024-04-25T08:34:45.530427+0000","last_became_active":"2024-04-25T08:34:43.868895+0000","last_became_peered":"2024-04-25T08:34:43.868895+0000","last_unstale":"2024-04-25T08:34:45.530427+0000","last_undegraded":"2024-04-25T08:34:45.530427+0000","last_fullsized":"2024-04-25T08:34:45.530427+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:36.714740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093548+0000","last_change":"2024-04-25T08:34:43.881571+0000","last_active":"2024-04-25T08:34:46.093548+0000","last_peered":"2024-04-25T08:34:46.093548+0000","last_clean":"2024-04-25T08:34:46.093548+0000","last_became_active":"2024-04-25T08:34:43.881272+0000","last_became_peered":"2024-04-25T08:34:43.881272+0000","last_unstale":"2024-04-25T08:34:46.093548+0000","last_undegraded":"2024-04-25T08:34:46.093548+0000","last_fullsized":"2024-04-25T08:34:46.093548+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:52.868065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530959+0000","last_change":"2024-04-25T08:34:44.531396+0000","last_active":"2024-04-25T08:34:45.530959+0000","last_peered":"2024-04-25T08:34:45.530959+0000","last_clean":"2024-04-25T08:34:45.530959+0000","last_became_active":"2024-04-25T08:34:44.531134+0000","last_became_peered":"2024-04-25T08:34:44.531134+0000","last_unstale":"2024-04-25T08:34:45.530959+0000","last_undegraded":"2024-04-25T08:34:45.530959+0000","last_fullsized":"2024-04-25T08:34:45.530959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:12.856976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093479+0000","last_change":"2024-04-25T08:34:44.525749+0000","last_active":"2024-04-25T08:34:46.093479+0000","last_peered":"2024-04-25T08:34:46.093479+0000","last_clean":"2024-04-25T08:34:46.093479+0000","last_became_active":"2024-04-25T08:34:44.525470+0000","last_became_peered":"2024-04-25T08:34:44.525470+0000","last_unstale":"2024-04-25T08:34:46.093479+0000","last_undegraded":"2024-04-25T08:34:46.093479+0000","last_fullsized":"2024-04-25T08:34:46.093479+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:40.781389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530997+0000","last_change":"2024-04-25T08:34:43.876071+0000","last_active":"2024-04-25T08:34:45.530997+0000","last_peered":"2024-04-25T08:34:45.530997+0000","last_clean":"2024-04-25T08:34:45.530997+0000","last_became_active":"2024-04-25T08:34:43.875782+0000","last_became_peered":"2024-04-25T08:34:43.875782+0000","last_unstale":"2024-04-25T08:34:45.530997+0000","last_undegraded":"2024-04-25T08:34:45.530997+0000","last_fullsized":"2024-04-25T08:34:45.530997+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:39.042852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094436+0000","last_change":"2024-04-25T08:34:44.528331+0000","last_active":"2024-04-25T08:34:46.094436+0000","last_peered":"2024-04-25T08:34:46.094436+0000","last_clean":"2024-04-25T08:34:46.094436+0000","last_became_active":"2024-04-25T08:34:44.528080+0000","last_became_peered":"2024-04-25T08:34:44.528080+0000","last_unstale":"2024-04-25T08:34:46.094436+0000","last_undegraded":"2024-04-25T08:34:46.094436+0000","last_fullsized":"2024-04-25T08:34:46.094436+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:25.030847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531206+0000","last_change":"2024-04-25T08:34:44.529717+0000","last_active":"2024-04-25T08:34:45.531206+0000","last_peered":"2024-04-25T08:34:45.531206+0000","last_clean":"2024-04-25T08:34:45.531206+0000","last_became_active":"2024-04-25T08:34:44.529442+0000","last_became_peered":"2024-04-25T08:34:44.529442+0000","last_unstale":"2024-04-25T08:34:45.531206+0000","last_undegraded":"2024-04-25T08:34:45.531206+0000","last_fullsized":"2024-04-25T08:34:45.531206+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:57.569222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093571+0000","last_change":"2024-04-25T08:34:44.532534+0000","last_active":"2024-04-25T08:34:46.093571+0000","last_peered":"2024-04-25T08:34:46.093571+0000","last_clean":"2024-04-25T08:34:46.093571+0000","last_became_active":"2024-04-25T08:34:44.532250+0000","last_became_peered":"2024-04-25T08:34:44.532250+0000","last_unstale":"2024-04-25T08:34:46.093571+0000","last_undegraded":"2024-04-25T08:34:46.093571+0000","last_fullsized":"2024-04-25T08:34:46.093571+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:11.760981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094802+0000","last_change":"2024-04-25T08:34:43.883681+0000","last_active":"2024-04-25T08:34:46.094802+0000","last_peered":"2024-04-25T08:34:46.094802+0000","last_clean":"2024-04-25T08:34:46.094802+0000","last_became_active":"2024-04-25T08:34:43.883385+0000","last_became_peered":"2024-04-25T08:34:43.883385+0000","last_unstale":"2024-04-25T08:34:46.094802+0000","last_undegraded":"2024-04-25T08:34:46.094802+0000","last_fullsized":"2024-04-25T08:34:46.094802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:17.727985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094016+0000","last_change":"2024-04-25T08:34:44.532945+0000","last_active":"2024-04-25T08:34:46.094016+0000","last_peered":"2024-04-25T08:34:46.094016+0000","last_clean":"2024-04-25T08:34:46.094016+0000","last_became_active":"2024-04-25T08:34:44.532563+0000","last_became_peered":"2024-04-25T08:34:44.532563+0000","last_unstale":"2024-04-25T08:34:46.094016+0000","last_undegraded":"2024-04-25T08:34:46.094016+0000","last_fullsized":"2024-04-25T08:34:46.094016+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:18.966389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530710+0000","last_change":"2024-04-25T08:34:43.882768+0000","last_active":"2024-04-25T08:34:45.530710+0000","last_peered":"2024-04-25T08:34:45.530710+0000","last_clean":"2024-04-25T08:34:45.530710+0000","last_became_active":"2024-04-25T08:34:43.882459+0000","last_became_peered":"2024-04-25T08:34:43.882459+0000","last_unstale":"2024-04-25T08:34:45.530710+0000","last_undegraded":"2024-04-25T08:34:45.530710+0000","last_fullsized":"2024-04-25T08:34:45.530710+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:46.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094908+0000","last_change":"2024-04-25T08:34:44.533491+0000","last_active":"2024-04-25T08:34:46.094908+0000","last_peered":"2024-04-25T08:34:46.094908+0000","last_clean":"2024-04-25T08:34:46.094908+0000","last_became_active":"2024-04-25T08:34:44.532257+0000","last_became_peered":"2024-04-25T08:34:44.532257+0000","last_unstale":"2024-04-25T08:34:46.094908+0000","last_undegraded":"2024-04-25T08:34:46.094908+0000","last_fullsized":"2024-04-25T08:34:46.094908+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:41.495853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095013+0000","last_change":"2024-04-25T08:34:43.889631+0000","last_active":"2024-04-25T08:34:46.095013+0000","last_peered":"2024-04-25T08:34:46.095013+0000","last_clean":"2024-04-25T08:34:46.095013+0000","last_became_active":"2024-04-25T08:34:43.889354+0000","last_became_peered":"2024-04-25T08:34:43.889354+0000","last_unstale":"2024-04-25T08:34:46.095013+0000","last_undegraded":"2024-04-25T08:34:46.095013+0000","last_fullsized":"2024-04-25T08:34:46.095013+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:02.862950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086134+0000","last_change":"2024-04-25T08:34:44.533026+0000","last_active":"2024-04-25T08:34:46.086134+0000","last_peered":"2024-04-25T08:34:46.086134+0000","last_clean":"2024-04-25T08:34:46.086134+0000","last_became_active":"2024-04-25T08:34:44.532759+0000","last_became_peered":"2024-04-25T08:34:44.532759+0000","last_unstale":"2024-04-25T08:34:46.086134+0000","last_undegraded":"2024-04-25T08:34:46.086134+0000","last_fullsized":"2024-04-25T08:34:46.086134+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:53.870186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531275+0000","last_change":"2024-04-25T08:34:44.532873+0000","last_active":"2024-04-25T08:34:45.531275+0000","last_peered":"2024-04-25T08:34:45.531275+0000","last_clean":"2024-04-25T08:34:45.531275+0000","last_became_active":"2024-04-25T08:34:44.532594+0000","last_became_peered":"2024-04-25T08:34:44.532594+0000","last_unstale":"2024-04-25T08:34:45.531275+0000","last_undegraded":"2024-04-25T08:34:45.531275+0000","last_fullsized":"2024-04-25T08:34:45.531275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:52.822815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530597+0000","last_change":"2024-04-25T08:34:43.877431+0000","last_active":"2024-04-25T08:34:45.530597+0000","last_peered":"2024-04-25T08:34:45.530597+0000","last_clean":"2024-04-25T08:34:45.530597+0000","last_became_active":"2024-04-25T08:34:43.877115+0000","last_became_peered":"2024-04-25T08:34:43.877115+0000","last_unstale":"2024-04-25T08:34:45.530597+0000","last_undegraded":"2024-04-25T08:34:45.530597+0000","last_fullsized":"2024-04-25T08:34:45.530597+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:55.485664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086192+0000","last_change":"2024-04-25T08:34:44.527817+0000","last_active":"2024-04-25T08:34:46.086192+0000","last_peered":"2024-04-25T08:34:46.086192+0000","last_clean":"2024-04-25T08:34:46.086192+0000","last_became_active":"2024-04-25T08:34:44.527550+0000","last_became_peered":"2024-04-25T08:34:44.527550+0000","last_unstale":"2024-04-25T08:34:46.086192+0000","last_undegraded":"2024-04-25T08:34:46.086192+0000","last_fullsized":"2024-04-25T08:34:46.086192+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:13.210050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530234+0000","last_change":"2024-04-25T08:34:43.883793+0000","last_active":"2024-04-25T08:34:45.530234+0000","last_peered":"2024-04-25T08:34:45.530234+0000","last_clean":"2024-04-25T08:34:45.530234+0000","last_became_active":"2024-04-25T08:34:43.882987+0000","last_became_peered":"2024-04-25T08:34:43.882987+0000","last_unstale":"2024-04-25T08:34:45.530234+0000","last_undegraded":"2024-04-25T08:34:45.530234+0000","last_fullsized":"2024-04-25T08:34:45.530234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:04.802285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093760+0000","last_change":"2024-04-25T08:34:44.528347+0000","last_active":"2024-04-25T08:34:46.093760+0000","last_peered":"2024-04-25T08:34:46.093760+0000","last_clean":"2024-04-25T08:34:46.093760+0000","last_became_active":"2024-04-25T08:34:44.528059+0000","last_became_peered":"2024-04-25T08:34:44.528059+0000","last_unstale":"2024-04-25T08:34:46.093760+0000","last_undegraded":"2024-04-25T08:34:46.093760+0000","last_fullsized":"2024-04-25T08:34:46.093760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.919973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086286+0000","last_change":"2024-04-25T08:34:43.878609+0000","last_active":"2024-04-25T08:34:46.086286+0000","last_peered":"2024-04-25T08:34:46.086286+0000","last_clean":"2024-04-25T08:34:46.086286+0000","last_became_active":"2024-04-25T08:34:43.878322+0000","last_became_peered":"2024-04-25T08:34:43.878322+0000","last_unstale":"2024-04-25T08:34:46.086286+0000","last_undegraded":"2024-04-25T08:34:46.086286+0000","last_fullsized":"2024-04-25T08:34:46.086286+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:46.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530698+0000","last_change":"2024-04-25T08:34:44.519833+0000","last_active":"2024-04-25T08:34:45.530698+0000","last_peered":"2024-04-25T08:34:45.530698+0000","last_clean":"2024-04-25T08:34:45.530698+0000","last_became_active":"2024-04-25T08:34:44.519535+0000","last_became_peered":"2024-04-25T08:34:44.519535+0000","last_unstale":"2024-04-25T08:34:45.530698+0000","last_undegraded":"2024-04-25T08:34:45.530698+0000","last_fullsized":"2024-04-25T08:34:45.530698+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.499854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094705+0000","last_change":"2024-04-25T08:34:44.527976+0000","last_active":"2024-04-25T08:34:46.094705+0000","last_peered":"2024-04-25T08:34:46.094705+0000","last_clean":"2024-04-25T08:34:46.094705+0000","last_became_active":"2024-04-25T08:34:44.527199+0000","last_became_peered":"2024-04-25T08:34:44.527199+0000","last_unstale":"2024-04-25T08:34:46.094705+0000","last_undegraded":"2024-04-25T08:34:46.094705+0000","last_fullsized":"2024-04-25T08:34:46.094705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:26.675283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094290+0000","last_change":"2024-04-25T08:34:43.880717+0000","last_active":"2024-04-25T08:34:46.094290+0000","last_peered":"2024-04-25T08:34:46.094290+0000","last_clean":"2024-04-25T08:34:46.094290+0000","last_became_active":"2024-04-25T08:34:43.880417+0000","last_became_peered":"2024-04-25T08:34:43.880417+0000","last_unstale":"2024-04-25T08:34:46.094290+0000","last_undegraded":"2024-04-25T08:34:46.094290+0000","last_fullsized":"2024-04-25T08:34:46.094290+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:29.445107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086364+0000","last_change":"2024-04-25T08:34:44.532688+0000","last_active":"2024-04-25T08:34:46.086364+0000","last_peered":"2024-04-25T08:34:46.086364+0000","last_clean":"2024-04-25T08:34:46.086364+0000","last_became_active":"2024-04-25T08:34:44.532430+0000","last_became_peered":"2024-04-25T08:34:44.532430+0000","last_unstale":"2024-04-25T08:34:46.086364+0000","last_undegraded":"2024-04-25T08:34:46.086364+0000","last_fullsized":"2024-04-25T08:34:46.086364+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:19.629450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093707+0000","last_change":"2024-04-25T08:34:44.532922+0000","last_active":"2024-04-25T08:34:46.093707+0000","last_peered":"2024-04-25T08:34:46.093707+0000","last_clean":"2024-04-25T08:34:46.093707+0000","last_became_active":"2024-04-25T08:34:44.532361+0000","last_became_peered":"2024-04-25T08:34:44.532361+0000","last_unstale":"2024-04-25T08:34:46.093707+0000","last_undegraded":"2024-04-25T08:34:46.093707+0000","last_fullsized":"2024-04-25T08:34:46.093707+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:05.622445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530587+0000","last_change":"2024-04-25T08:34:43.881524+0000","last_active":"2024-04-25T08:34:45.530587+0000","last_peered":"2024-04-25T08:34:45.530587+0000","last_clean":"2024-04-25T08:34:45.530587+0000","last_became_active":"2024-04-25T08:34:43.881273+0000","last_became_peered":"2024-04-25T08:34:43.881273+0000","last_unstale":"2024-04-25T08:34:45.530587+0000","last_undegraded":"2024-04-25T08:34:45.530587+0000","last_fullsized":"2024-04-25T08:34:45.530587+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:54.880133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086187+0000","last_change":"2024-04-25T08:34:44.531176+0000","last_active":"2024-04-25T08:34:46.086187+0000","last_peered":"2024-04-25T08:34:46.086187+0000","last_clean":"2024-04-25T08:34:46.086187+0000","last_became_active":"2024-04-25T08:34:44.529858+0000","last_became_peered":"2024-04-25T08:34:44.529858+0000","last_unstale":"2024-04-25T08:34:46.086187+0000","last_undegraded":"2024-04-25T08:34:46.086187+0000","last_fullsized":"2024-04-25T08:34:46.086187+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:03.865071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093840+0000","last_change":"2024-04-25T08:34:43.875641+0000","last_active":"2024-04-25T08:34:46.093840+0000","last_peered":"2024-04-25T08:34:46.093840+0000","last_clean":"2024-04-25T08:34:46.093840+0000","last_became_active":"2024-04-25T08:34:43.875344+0000","last_became_peered":"2024-04-25T08:34:43.875344+0000","last_unstale":"2024-04-25T08:34:46.093840+0000","last_undegraded":"2024-04-25T08:34:46.093840+0000","last_fullsized":"2024-04-25T08:34:46.093840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:44.497733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093679+0000","last_change":"2024-04-25T08:34:44.532615+0000","last_active":"2024-04-25T08:34:46.093679+0000","last_peered":"2024-04-25T08:34:46.093679+0000","last_clean":"2024-04-25T08:34:46.093679+0000","last_became_active":"2024-04-25T08:34:44.532335+0000","last_became_peered":"2024-04-25T08:34:44.532335+0000","last_unstale":"2024-04-25T08:34:46.093679+0000","last_undegraded":"2024-04-25T08:34:46.093679+0000","last_fullsized":"2024-04-25T08:34:46.093679+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:05.252545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093705+0000","last_change":"2024-04-25T08:34:44.529316+0000","last_active":"2024-04-25T08:34:46.093705+0000","last_peered":"2024-04-25T08:34:46.093705+0000","last_clean":"2024-04-25T08:34:46.093705+0000","last_became_active":"2024-04-25T08:34:44.529026+0000","last_became_peered":"2024-04-25T08:34:44.529026+0000","last_unstale":"2024-04-25T08:34:46.093705+0000","last_undegraded":"2024-04-25T08:34:46.093705+0000","last_fullsized":"2024-04-25T08:34:46.093705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:33.006310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094563+0000","last_change":"2024-04-25T08:34:44.536436+0000","last_active":"2024-04-25T08:34:46.094563+0000","last_peered":"2024-04-25T08:34:46.094563+0000","last_clean":"2024-04-25T08:34:46.094563+0000","last_became_active":"2024-04-25T08:34:44.534485+0000","last_became_peered":"2024-04-25T08:34:44.534485+0000","last_unstale":"2024-04-25T08:34:46.094563+0000","last_undegraded":"2024-04-25T08:34:46.094563+0000","last_fullsized":"2024-04-25T08:34:46.094563+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:31.481085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531115+0000","last_change":"2024-04-25T08:34:43.883072+0000","last_active":"2024-04-25T08:34:45.531115+0000","last_peered":"2024-04-25T08:34:45.531115+0000","last_clean":"2024-04-25T08:34:45.531115+0000","last_became_active":"2024-04-25T08:34:43.882688+0000","last_became_peered":"2024-04-25T08:34:43.882688+0000","last_unstale":"2024-04-25T08:34:45.531115+0000","last_undegraded":"2024-04-25T08:34:45.531115+0000","last_fullsized":"2024-04-25T08:34:45.531115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:16.493814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093700+0000","last_change":"2024-04-25T08:34:43.883730+0000","last_active":"2024-04-25T08:34:46.093700+0000","last_peered":"2024-04-25T08:34:46.093700+0000","last_clean":"2024-04-25T08:34:46.093700+0000","last_became_active":"2024-04-25T08:34:43.883435+0000","last_became_peered":"2024-04-25T08:34:43.883435+0000","last_unstale":"2024-04-25T08:34:46.093700+0000","last_undegraded":"2024-04-25T08:34:46.093700+0000","last_fullsized":"2024-04-25T08:34:46.093700+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:10.731254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530395+0000","last_change":"2024-04-25T08:34:44.526203+0000","last_active":"2024-04-25T08:34:45.530395+0000","last_peered":"2024-04-25T08:34:45.530395+0000","last_clean":"2024-04-25T08:34:45.530395+0000","last_became_active":"2024-04-25T08:34:44.525945+0000","last_became_peered":"2024-04-25T08:34:44.525945+0000","last_unstale":"2024-04-25T08:34:45.530395+0000","last_undegraded":"2024-04-25T08:34:45.530395+0000","last_fullsized":"2024-04-25T08:34:45.530395+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.447228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.869817+0000","last_change":"2024-04-25T08:34:43.881671+0000","last_active":"2024-04-25T08:34:46.869817+0000","last_peered":"2024-04-25T08:34:46.869817+0000","last_clean":"2024-04-25T08:34:46.869817+0000","last_became_active":"2024-04-25T08:34:43.881375+0000","last_became_peered":"2024-04-25T08:34:43.881375+0000","last_unstale":"2024-04-25T08:34:46.869817+0000","last_undegraded":"2024-04-25T08:34:46.869817+0000","last_fullsized":"2024-04-25T08:34:46.869817+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:48.237385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":227,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T08:34:42.092335+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:34:42.092335+0000","last_peered":"2024-04-25T08:34:42.092335+0000","last_clean":"2024-04-25T08:34:42.092335+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:34:42.092335+0000","last_undegraded":"2024-04-25T08:34:42.092335+0000","last_fullsized":"2024-04-25T08:34:42.092335+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":86,"acting":86,"num_store_stats":5},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":708,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":42,"acting":42,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58704,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313136,"statfs":{"total":96636764160,"available":96576651264,"internally_reserved":0,"allocated":1175552,"data_stored":847826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40144,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331696,"statfs":{"total":96636764160,"available":96595656704,"internally_reserved":0,"allocated":651264,"data_stored":290364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35972,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335868,"statfs":{"total":96636764160,"available":96599928832,"internally_reserved":0,"allocated":643072,"data_stored":289712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58184,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313656,"statfs":{"total":96636764160,"available":96577183744,"internally_reserved":0,"allocated":647168,"data_stored":290294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44244,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327596,"statfs":{"total":96636764160,"available":96591458304,"internally_reserved":0,"allocated":593920,"data_stored":262977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40084,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331756,"statfs":{"total":96636764160,"available":96595718144,"internally_reserved":0,"allocated":593920,"data_stored":262977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":651264,"data_stored":289872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50444,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321396,"statfs":{"total":96636764160,"available":96585109504,"internally_reserved":0,"allocated":1245184,"data_stored":880136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:34:48.215 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:34:48.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.358+0000 7f52d2df9640 1 Processor -- start 2024-04-25T08:34:48.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.358+0000 7f52d2df9640 1 -- start start 2024-04-25T08:34:48.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52d2df9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0b5710 0x7f52cc161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52d2df9640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f52cc15ddf0 0x7f52cc0b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52d2df9640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc161e00 con 0x7f52cc0b5af0 2024-04-25T08:34:48.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52d2df9640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc0b9930 con 0x7f52cc15ddf0 2024-04-25T08:34:48.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52d2df9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc1620e0 con 0x7f52cc0b5710 2024-04-25T08:34:48.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52d1396640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f52cc0b5af0 0x7f52cc06bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33274/0 (socket says 172.21.15.38:33274) 2024-04-25T08:34:48.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52d1396640 1 -- 172.21.15.38:0/297280931 learned_addr learned my addr 172.21.15.38:0/297280931 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:48.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52cbfff640 1 --2- 172.21.15.38:0/297280931 >> v2:172.21.15.107:3300/0 conn(0x7f52cc15ddf0 0x7f52cc0b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52d0b95640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0b5710 0x7f52cc161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956061208 0 0) 0x7f52cc161e00 con 0x7f52cc0b5af0 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52bc003520 con 0x7f52cc0b5af0 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2732234927 0 0) 0x7f52bc003520 con 0x7f52cc0b5af0 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0b5710 msgr2=0x7f52cc161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52cb7fe640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0b5710 0x7f52cc161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 >> v2:172.21.15.107:3300/0 conn(0x7f52cc15ddf0 msgr2=0x7f52cc0b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52cb7fe640 1 --2- 172.21.15.38:0/297280931 >> v2:172.21.15.107:3300/0 conn(0x7f52cc15ddf0 0x7f52cc0b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc1623c0 con 0x7f52cc0b5af0 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f52c40019f0 con 0x7f52cc0b5af0 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52cbfff640 1 --2- 172.21.15.38:0/297280931 >> v2:172.21.15.107:3300/0 conn(0x7f52cc15ddf0 0x7f52cc0b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.359+0000 7f52d0b95640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0b5710 0x7f52cc161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 >> v1:172.21.15.38:6789/0 conn(0x7f52cc0b5af0 legacy=0x7f52cc06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cb7fe640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc0039b0 0x7f52bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cb7fe640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52bc004800 0x7f52bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cb7fe640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0b5710 0x7f52bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc1620e0 con 0x7f52bc004800 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc00b9c0 con 0x7f52bc0039b0 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52bc00bb40 con 0x7f52cc0b5710 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cbfff640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52bc004800 0x7f52bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52d0b95640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc0039b0 0x7f52bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52d1396640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0b5710 0x7f52bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cbfff640 1 -- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0b5710 msgr2=0x7f52bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cbfff640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0b5710 0x7f52bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cbfff640 1 -- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc0039b0 msgr2=0x7f52bc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cbfff640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc0039b0 0x7f52bc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52cbfff640 1 -- 172.21.15.38:0/297280931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc003520 con 0x7f52bc004800 2024-04-25T08:34:48.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.360+0000 7f52d0b95640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc0039b0 0x7f52bc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:48.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.361+0000 7f52cbfff640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52bc004800 0x7f52bc004c80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009c60 tx=0x7f52bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ec1a0c4f975d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:48.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.361+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8036070 con 0x7f52bc004800 2024-04-25T08:34:48.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.361+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b8031400 con 0x7f52bc004800 2024-04-25T08:34:48.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.361+0000 7f52cb7fe640 1 -- 172.21.15.38:0/297280931 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8009860 con 0x7f52bc004800 2024-04-25T08:34:48.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.361+0000 7f52d2df9640 1 -- 172.21.15.38:0/297280931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52bc004800 msgr2=0x7f52bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.361+0000 7f52d2df9640 1 --2- 172.21.15.38:0/297280931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52bc004800 0x7f52bc004c80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009c60 tx=0x7f52bc005830 comp rx=0 tx=0).stop 2024-04-25T08:34:48.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.361+0000 7f52d1396640 1 -- 172.21.15.38:0/297280931 reap_dead start 2024-04-25T08:34:48.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.361+0000 7f52d2df9640 1 -- 172.21.15.38:0/297280931 shutdown_connections 2024-04-25T08:34:48.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.361+0000 7f52d2df9640 1 -- 172.21.15.38:0/297280931 >> 172.21.15.38:0/297280931 conn(0x7f52cc0d07c0 msgr2=0x7f52cc0d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:48.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.361+0000 7f52d2df9640 1 -- 172.21.15.38:0/297280931 shutdown_connections 2024-04-25T08:34:48.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.361+0000 7f52d2df9640 1 -- 172.21.15.38:0/297280931 wait complete. 2024-04-25T08:34:48.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.363+0000 7f52d2df9640 1 Processor -- start 2024-04-25T08:34:48.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.363+0000 7f52d2df9640 1 -- start start 2024-04-25T08:34:48.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.363+0000 7f52d2df9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc004800 0x7f52cc0ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.363+0000 7f52d2df9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52cc0af4a0 0x7f52cc0abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52d2df9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0af920 0x7f52cc0ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52d2df9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0d34d0 con 0x7f52cc0af4a0 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52d2df9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0d3350 con 0x7f52bc004800 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52d2df9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0d3650 con 0x7f52cc0af920 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52cbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52cc0af4a0 0x7f52cc0abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52cbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52cc0af4a0 0x7f52cc0abe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51086/0 (socket says 172.21.15.38:51086) 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52cbfff640 1 -- 172.21.15.38:0/2447022834 learned_addr learned my addr 172.21.15.38:0/2447022834 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52d1396640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0af920 0x7f52cc0ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52d0b95640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc004800 0x7f52cc0ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52cbfff640 1 -- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0af920 msgr2=0x7f52cc0ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52cbfff640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0af920 0x7f52cc0ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52cbfff640 1 -- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc004800 msgr2=0x7f52cc0ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52cbfff640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc004800 0x7f52cc0ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52cbfff640 1 -- 172.21.15.38:0/2447022834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc0aadb0 con 0x7f52cc0af4a0 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52d0b95640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc004800 0x7f52cc0ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52d1396640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0af920 0x7f52cc0ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:48.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52cbfff640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52cc0af4a0 0x7f52cc0abe10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009d80 tx=0x7f52b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:48.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52c97fa640 1 -- 172.21.15.38:0/2447022834 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8040070 con 0x7f52cc0af4a0 2024-04-25T08:34:48.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52c97fa640 1 -- 172.21.15.38:0/2447022834 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b803b660 con 0x7f52cc0af4a0 2024-04-25T08:34:48.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52c97fa640 1 -- 172.21.15.38:0/2447022834 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b8036040 con 0x7f52cc0af4a0 2024-04-25T08:34:48.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52d2df9640 1 -- 172.21.15.38:0/2447022834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc0ab050 con 0x7f52cc0af4a0 2024-04-25T08:34:48.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.364+0000 7f52d2df9640 1 -- 172.21.15.38:0/2447022834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc0ab610 con 0x7f52cc0af4a0 2024-04-25T08:34:48.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.365+0000 7f52d2df9640 1 -- 172.21.15.38:0/2447022834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc1607e0 con 0x7f52cc0af4a0 2024-04-25T08:34:48.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.366+0000 7f52c97fa640 1 -- 172.21.15.38:0/2447022834 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52b8043070 con 0x7f52cc0af4a0 2024-04-25T08:34:48.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.366+0000 7f52c97fa640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52b006b4d0 0x7f52b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.366+0000 7f52c97fa640 1 -- 172.21.15.38:0/2447022834 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f52b80aa8e0 con 0x7f52cc0af4a0 2024-04-25T08:34:48.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.367+0000 7f52d0b95640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52b006b4d0 0x7f52b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.367+0000 7f52d0b95640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52b006b4d0 0x7f52b006d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009560 tx=0x7f52c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:48.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.370+0000 7f52c97fa640 1 -- 172.21.15.38:0/2447022834 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f52cc1607e0 con 0x7f52cc0af4a0 2024-04-25T08:34:48.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.553+0000 7f52d2df9640 1 -- 172.21.15.38:0/2447022834 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52cc0ac890 con 0x7f52b006b4d0 2024-04-25T08:34:48.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.560+0000 7f52c97fa640 1 -- 172.21.15.38:0/2447022834 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246879 (secure 0 0 0) 0x7f52c0012920 con 0x7f52b006b4d0 2024-04-25T08:34:48.557 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:48.558 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:34:48.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.563+0000 7f52d2df9640 1 -- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52b006b4d0 msgr2=0x7f52b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.563+0000 7f52d2df9640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52b006b4d0 0x7f52b006d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009560 tx=0x7f52c0008040 comp rx=0 tx=0).stop 2024-04-25T08:34:48.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.563+0000 7f52d2df9640 1 -- 172.21.15.38:0/2447022834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52cc0af4a0 msgr2=0x7f52cc0abe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.563+0000 7f52d2df9640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52cc0af4a0 0x7f52cc0abe10 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009d80 tx=0x7f52b8004730 comp rx=0 tx=0).stop 2024-04-25T08:34:48.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.563+0000 7f52d2df9640 1 -- 172.21.15.38:0/2447022834 shutdown_connections 2024-04-25T08:34:48.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.563+0000 7f52d2df9640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52b006b4d0 0x7f52b006d990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.563+0000 7f52d2df9640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52cc0af920 0x7f52cc0ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.563+0000 7f52d2df9640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52cc0af4a0 0x7f52cc0abe10 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.563+0000 7f52d2df9640 1 --2- 172.21.15.38:0/2447022834 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52bc004800 0x7f52cc0ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.563+0000 7f52d2df9640 1 -- 172.21.15.38:0/2447022834 >> 172.21.15.38:0/2447022834 conn(0x7f52cc0d07c0 msgr2=0x7f52cc0b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:48.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.563+0000 7f52d2df9640 1 -- 172.21.15.38:0/2447022834 shutdown_connections 2024-04-25T08:34:48.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.563+0000 7f52d2df9640 1 -- 172.21.15.38:0/2447022834 wait complete. 2024-04-25T08:34:48.569 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-04-25T08:34:47.249503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591076,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":179,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":367916,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":361715,"kb_avail":754606804,"statfs":{"total":773094113280,"available":772717367296,"internally_reserved":0,"allocated":6201344,"data_stored":3414158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":370396720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":708,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.393179"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086305+0000","last_change":"2024-04-25T08:34:44.527955+0000","last_active":"2024-04-25T08:34:46.086305+0000","last_peered":"2024-04-25T08:34:46.086305+0000","last_clean":"2024-04-25T08:34:46.086305+0000","last_became_active":"2024-04-25T08:34:44.527662+0000","last_became_peered":"2024-04-25T08:34:44.527662+0000","last_unstale":"2024-04-25T08:34:46.086305+0000","last_undegraded":"2024-04-25T08:34:46.086305+0000","last_fullsized":"2024-04-25T08:34:46.086305+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:09.895753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093741+0000","last_change":"2024-04-25T08:34:44.534121+0000","last_active":"2024-04-25T08:34:46.093741+0000","last_peered":"2024-04-25T08:34:46.093741+0000","last_clean":"2024-04-25T08:34:46.093741+0000","last_became_active":"2024-04-25T08:34:44.533765+0000","last_became_peered":"2024-04-25T08:34:44.533765+0000","last_unstale":"2024-04-25T08:34:46.093741+0000","last_undegraded":"2024-04-25T08:34:46.093741+0000","last_fullsized":"2024-04-25T08:34:46.093741+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.333087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531042+0000","last_change":"2024-04-25T08:34:44.520910+0000","last_active":"2024-04-25T08:34:45.531042+0000","last_peered":"2024-04-25T08:34:45.531042+0000","last_clean":"2024-04-25T08:34:45.531042+0000","last_became_active":"2024-04-25T08:34:44.520607+0000","last_became_peered":"2024-04-25T08:34:44.520607+0000","last_unstale":"2024-04-25T08:34:45.531042+0000","last_undegraded":"2024-04-25T08:34:45.531042+0000","last_fullsized":"2024-04-25T08:34:45.531042+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:43.933652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086256+0000","last_change":"2024-04-25T08:34:44.527358+0000","last_active":"2024-04-25T08:34:46.086256+0000","last_peered":"2024-04-25T08:34:46.086256+0000","last_clean":"2024-04-25T08:34:46.086256+0000","last_became_active":"2024-04-25T08:34:44.527098+0000","last_became_peered":"2024-04-25T08:34:44.527098+0000","last_unstale":"2024-04-25T08:34:46.086256+0000","last_undegraded":"2024-04-25T08:34:46.086256+0000","last_fullsized":"2024-04-25T08:34:46.086256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:47.550727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094066+0000","last_change":"2024-04-25T08:34:44.536974+0000","last_active":"2024-04-25T08:34:46.094066+0000","last_peered":"2024-04-25T08:34:46.094066+0000","last_clean":"2024-04-25T08:34:46.094066+0000","last_became_active":"2024-04-25T08:34:44.536690+0000","last_became_peered":"2024-04-25T08:34:44.536690+0000","last_unstale":"2024-04-25T08:34:46.094066+0000","last_undegraded":"2024-04-25T08:34:46.094066+0000","last_fullsized":"2024-04-25T08:34:46.094066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:27.300505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086137+0000","last_change":"2024-04-25T08:34:44.533732+0000","last_active":"2024-04-25T08:34:46.086137+0000","last_peered":"2024-04-25T08:34:46.086137+0000","last_clean":"2024-04-25T08:34:46.086137+0000","last_became_active":"2024-04-25T08:34:44.533441+0000","last_became_peered":"2024-04-25T08:34:44.533441+0000","last_unstale":"2024-04-25T08:34:46.086137+0000","last_undegraded":"2024-04-25T08:34:46.086137+0000","last_fullsized":"2024-04-25T08:34:46.086137+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:11.733375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086171+0000","last_change":"2024-04-25T08:34:44.531394+0000","last_active":"2024-04-25T08:34:46.086171+0000","last_peered":"2024-04-25T08:34:46.086171+0000","last_clean":"2024-04-25T08:34:46.086171+0000","last_became_active":"2024-04-25T08:34:44.531057+0000","last_became_peered":"2024-04-25T08:34:44.531057+0000","last_unstale":"2024-04-25T08:34:46.086171+0000","last_undegraded":"2024-04-25T08:34:46.086171+0000","last_fullsized":"2024-04-25T08:34:46.086171+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:18.730106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094348+0000","last_change":"2024-04-25T08:34:44.533220+0000","last_active":"2024-04-25T08:34:46.094348+0000","last_peered":"2024-04-25T08:34:46.094348+0000","last_clean":"2024-04-25T08:34:46.094348+0000","last_became_active":"2024-04-25T08:34:44.532647+0000","last_became_peered":"2024-04-25T08:34:44.532647+0000","last_unstale":"2024-04-25T08:34:46.094348+0000","last_undegraded":"2024-04-25T08:34:46.094348+0000","last_fullsized":"2024-04-25T08:34:46.094348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:00.701022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086067+0000","last_change":"2024-04-25T08:34:44.525673+0000","last_active":"2024-04-25T08:34:46.086067+0000","last_peered":"2024-04-25T08:34:46.086067+0000","last_clean":"2024-04-25T08:34:46.086067+0000","last_became_active":"2024-04-25T08:34:44.525384+0000","last_became_peered":"2024-04-25T08:34:44.525384+0000","last_unstale":"2024-04-25T08:34:46.086067+0000","last_undegraded":"2024-04-25T08:34:46.086067+0000","last_fullsized":"2024-04-25T08:34:46.086067+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.499854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095168+0000","last_change":"2024-04-25T08:34:44.533259+0000","last_active":"2024-04-25T08:34:46.095168+0000","last_peered":"2024-04-25T08:34:46.095168+0000","last_clean":"2024-04-25T08:34:46.095168+0000","last_became_active":"2024-04-25T08:34:44.531589+0000","last_became_peered":"2024-04-25T08:34:44.531589+0000","last_unstale":"2024-04-25T08:34:46.095168+0000","last_undegraded":"2024-04-25T08:34:46.095168+0000","last_fullsized":"2024-04-25T08:34:46.095168+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:01.549992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093885+0000","last_change":"2024-04-25T08:34:44.530163+0000","last_active":"2024-04-25T08:34:46.093885+0000","last_peered":"2024-04-25T08:34:46.093885+0000","last_clean":"2024-04-25T08:34:46.093885+0000","last_became_active":"2024-04-25T08:34:44.529905+0000","last_became_peered":"2024-04-25T08:34:44.529905+0000","last_unstale":"2024-04-25T08:34:46.093885+0000","last_undegraded":"2024-04-25T08:34:46.093885+0000","last_fullsized":"2024-04-25T08:34:46.093885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:18.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529970+0000","last_change":"2024-04-25T08:34:44.520405+0000","last_active":"2024-04-25T08:34:45.529970+0000","last_peered":"2024-04-25T08:34:45.529970+0000","last_clean":"2024-04-25T08:34:45.529970+0000","last_became_active":"2024-04-25T08:34:44.520122+0000","last_became_peered":"2024-04-25T08:34:44.520122+0000","last_unstale":"2024-04-25T08:34:45.529970+0000","last_undegraded":"2024-04-25T08:34:45.529970+0000","last_fullsized":"2024-04-25T08:34:45.529970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:09.811944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530067+0000","last_change":"2024-04-25T08:34:44.522344+0000","last_active":"2024-04-25T08:34:45.530067+0000","last_peered":"2024-04-25T08:34:45.530067+0000","last_clean":"2024-04-25T08:34:45.530067+0000","last_became_active":"2024-04-25T08:34:44.522063+0000","last_became_peered":"2024-04-25T08:34:44.522063+0000","last_unstale":"2024-04-25T08:34:45.530067+0000","last_undegraded":"2024-04-25T08:34:45.530067+0000","last_fullsized":"2024-04-25T08:34:45.530067+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:25.590051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093628+0000","last_change":"2024-04-25T08:34:44.536670+0000","last_active":"2024-04-25T08:34:46.093628+0000","last_peered":"2024-04-25T08:34:46.093628+0000","last_clean":"2024-04-25T08:34:46.093628+0000","last_became_active":"2024-04-25T08:34:44.536110+0000","last_became_peered":"2024-04-25T08:34:44.536110+0000","last_unstale":"2024-04-25T08:34:46.093628+0000","last_undegraded":"2024-04-25T08:34:46.093628+0000","last_fullsized":"2024-04-25T08:34:46.093628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:43.198107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095058+0000","last_change":"2024-04-25T08:34:44.533241+0000","last_active":"2024-04-25T08:34:46.095058+0000","last_peered":"2024-04-25T08:34:46.095058+0000","last_clean":"2024-04-25T08:34:46.095058+0000","last_became_active":"2024-04-25T08:34:44.531910+0000","last_became_peered":"2024-04-25T08:34:44.531910+0000","last_unstale":"2024-04-25T08:34:46.095058+0000","last_undegraded":"2024-04-25T08:34:46.095058+0000","last_fullsized":"2024-04-25T08:34:46.095058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.963128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086493+0000","last_change":"2024-04-25T08:34:44.538453+0000","last_active":"2024-04-25T08:34:46.086493+0000","last_peered":"2024-04-25T08:34:46.086493+0000","last_clean":"2024-04-25T08:34:46.086493+0000","last_became_active":"2024-04-25T08:34:44.538165+0000","last_became_peered":"2024-04-25T08:34:44.538165+0000","last_unstale":"2024-04-25T08:34:46.086493+0000","last_undegraded":"2024-04-25T08:34:46.086493+0000","last_fullsized":"2024-04-25T08:34:46.086493+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:46.558329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530135+0000","last_change":"2024-04-25T08:34:44.528517+0000","last_active":"2024-04-25T08:34:45.530135+0000","last_peered":"2024-04-25T08:34:45.530135+0000","last_clean":"2024-04-25T08:34:45.530135+0000","last_became_active":"2024-04-25T08:34:44.528229+0000","last_became_peered":"2024-04-25T08:34:44.528229+0000","last_unstale":"2024-04-25T08:34:45.530135+0000","last_undegraded":"2024-04-25T08:34:45.530135+0000","last_fullsized":"2024-04-25T08:34:45.530135+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:41.585740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094613+0000","last_change":"2024-04-25T08:34:44.528023+0000","last_active":"2024-04-25T08:34:46.094613+0000","last_peered":"2024-04-25T08:34:46.094613+0000","last_clean":"2024-04-25T08:34:46.094613+0000","last_became_active":"2024-04-25T08:34:44.527668+0000","last_became_peered":"2024-04-25T08:34:44.527668+0000","last_unstale":"2024-04-25T08:34:46.094613+0000","last_undegraded":"2024-04-25T08:34:46.094613+0000","last_fullsized":"2024-04-25T08:34:46.094613+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:06.085151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094200+0000","last_change":"2024-04-25T08:34:44.533947+0000","last_active":"2024-04-25T08:34:46.094200+0000","last_peered":"2024-04-25T08:34:46.094200+0000","last_clean":"2024-04-25T08:34:46.094200+0000","last_became_active":"2024-04-25T08:34:44.533609+0000","last_became_peered":"2024-04-25T08:34:44.533609+0000","last_unstale":"2024-04-25T08:34:46.094200+0000","last_undegraded":"2024-04-25T08:34:46.094200+0000","last_fullsized":"2024-04-25T08:34:46.094200+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:44.889254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093558+0000","last_change":"2024-04-25T08:34:44.532723+0000","last_active":"2024-04-25T08:34:46.093558+0000","last_peered":"2024-04-25T08:34:46.093558+0000","last_clean":"2024-04-25T08:34:46.093558+0000","last_became_active":"2024-04-25T08:34:44.531850+0000","last_became_peered":"2024-04-25T08:34:44.531850+0000","last_unstale":"2024-04-25T08:34:46.093558+0000","last_undegraded":"2024-04-25T08:34:46.093558+0000","last_fullsized":"2024-04-25T08:34:46.093558+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:20.979454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529837+0000","last_change":"2024-04-25T08:34:43.873145+0000","last_active":"2024-04-25T08:34:45.529837+0000","last_peered":"2024-04-25T08:34:45.529837+0000","last_clean":"2024-04-25T08:34:45.529837+0000","last_became_active":"2024-04-25T08:34:43.872860+0000","last_became_peered":"2024-04-25T08:34:43.872860+0000","last_unstale":"2024-04-25T08:34:45.529837+0000","last_undegraded":"2024-04-25T08:34:45.529837+0000","last_fullsized":"2024-04-25T08:34:45.529837+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:39.042852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093598+0000","last_change":"2024-04-25T08:34:44.530047+0000","last_active":"2024-04-25T08:34:46.093598+0000","last_peered":"2024-04-25T08:34:46.093598+0000","last_clean":"2024-04-25T08:34:46.093598+0000","last_became_active":"2024-04-25T08:34:44.529781+0000","last_became_peered":"2024-04-25T08:34:44.529781+0000","last_unstale":"2024-04-25T08:34:46.093598+0000","last_undegraded":"2024-04-25T08:34:46.093598+0000","last_fullsized":"2024-04-25T08:34:46.093598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:36.970964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086257+0000","last_change":"2024-04-25T08:34:43.884678+0000","last_active":"2024-04-25T08:34:46.086257+0000","last_peered":"2024-04-25T08:34:46.086257+0000","last_clean":"2024-04-25T08:34:46.086257+0000","last_became_active":"2024-04-25T08:34:43.884357+0000","last_became_peered":"2024-04-25T08:34:43.884357+0000","last_unstale":"2024-04-25T08:34:46.086257+0000","last_undegraded":"2024-04-25T08:34:46.086257+0000","last_fullsized":"2024-04-25T08:34:46.086257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:04.802285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530601+0000","last_change":"2024-04-25T08:34:44.519962+0000","last_active":"2024-04-25T08:34:45.530601+0000","last_peered":"2024-04-25T08:34:45.530601+0000","last_clean":"2024-04-25T08:34:45.530601+0000","last_became_active":"2024-04-25T08:34:44.519671+0000","last_became_peered":"2024-04-25T08:34:44.519671+0000","last_unstale":"2024-04-25T08:34:45.530601+0000","last_undegraded":"2024-04-25T08:34:45.530601+0000","last_fullsized":"2024-04-25T08:34:45.530601+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:49.239506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086556+0000","last_change":"2024-04-25T08:34:43.878220+0000","last_active":"2024-04-25T08:34:46.086556+0000","last_peered":"2024-04-25T08:34:46.086556+0000","last_clean":"2024-04-25T08:34:46.086556+0000","last_became_active":"2024-04-25T08:34:43.877915+0000","last_became_peered":"2024-04-25T08:34:43.877915+0000","last_unstale":"2024-04-25T08:34:46.086556+0000","last_undegraded":"2024-04-25T08:34:46.086556+0000","last_fullsized":"2024-04-25T08:34:46.086556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:55.485664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093702+0000","last_change":"2024-04-25T08:34:43.882922+0000","last_active":"2024-04-25T08:34:46.093702+0000","last_peered":"2024-04-25T08:34:46.093702+0000","last_clean":"2024-04-25T08:34:46.093702+0000","last_became_active":"2024-04-25T08:34:43.882609+0000","last_became_peered":"2024-04-25T08:34:43.882609+0000","last_unstale":"2024-04-25T08:34:46.093702+0000","last_undegraded":"2024-04-25T08:34:46.093702+0000","last_fullsized":"2024-04-25T08:34:46.093702+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:17.727985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531083+0000","last_change":"2024-04-25T08:34:44.532481+0000","last_active":"2024-04-25T08:34:45.531083+0000","last_peered":"2024-04-25T08:34:45.531083+0000","last_clean":"2024-04-25T08:34:45.531083+0000","last_became_active":"2024-04-25T08:34:44.532204+0000","last_became_peered":"2024-04-25T08:34:44.532204+0000","last_unstale":"2024-04-25T08:34:45.531083+0000","last_undegraded":"2024-04-25T08:34:45.531083+0000","last_fullsized":"2024-04-25T08:34:45.531083+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:19.590018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094711+0000","last_change":"2024-04-25T08:34:43.883536+0000","last_active":"2024-04-25T08:34:46.094711+0000","last_peered":"2024-04-25T08:34:46.094711+0000","last_clean":"2024-04-25T08:34:46.094711+0000","last_became_active":"2024-04-25T08:34:43.883232+0000","last_became_peered":"2024-04-25T08:34:43.883232+0000","last_unstale":"2024-04-25T08:34:46.094711+0000","last_undegraded":"2024-04-25T08:34:46.094711+0000","last_fullsized":"2024-04-25T08:34:46.094711+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:10.731254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531305+0000","last_change":"2024-04-25T08:34:43.875476+0000","last_active":"2024-04-25T08:34:45.531305+0000","last_peered":"2024-04-25T08:34:45.531305+0000","last_clean":"2024-04-25T08:34:45.531305+0000","last_became_active":"2024-04-25T08:34:43.874920+0000","last_became_peered":"2024-04-25T08:34:43.874920+0000","last_unstale":"2024-04-25T08:34:45.531305+0000","last_undegraded":"2024-04-25T08:34:45.531305+0000","last_fullsized":"2024-04-25T08:34:45.531305+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:54.880133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530427+0000","last_change":"2024-04-25T08:34:43.869200+0000","last_active":"2024-04-25T08:34:45.530427+0000","last_peered":"2024-04-25T08:34:45.530427+0000","last_clean":"2024-04-25T08:34:45.530427+0000","last_became_active":"2024-04-25T08:34:43.868895+0000","last_became_peered":"2024-04-25T08:34:43.868895+0000","last_unstale":"2024-04-25T08:34:45.530427+0000","last_undegraded":"2024-04-25T08:34:45.530427+0000","last_fullsized":"2024-04-25T08:34:45.530427+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:36.714740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093548+0000","last_change":"2024-04-25T08:34:43.881571+0000","last_active":"2024-04-25T08:34:46.093548+0000","last_peered":"2024-04-25T08:34:46.093548+0000","last_clean":"2024-04-25T08:34:46.093548+0000","last_became_active":"2024-04-25T08:34:43.881272+0000","last_became_peered":"2024-04-25T08:34:43.881272+0000","last_unstale":"2024-04-25T08:34:46.093548+0000","last_undegraded":"2024-04-25T08:34:46.093548+0000","last_fullsized":"2024-04-25T08:34:46.093548+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:52.868065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530959+0000","last_change":"2024-04-25T08:34:44.531396+0000","last_active":"2024-04-25T08:34:45.530959+0000","last_peered":"2024-04-25T08:34:45.530959+0000","last_clean":"2024-04-25T08:34:45.530959+0000","last_became_active":"2024-04-25T08:34:44.531134+0000","last_became_peered":"2024-04-25T08:34:44.531134+0000","last_unstale":"2024-04-25T08:34:45.530959+0000","last_undegraded":"2024-04-25T08:34:45.530959+0000","last_fullsized":"2024-04-25T08:34:45.530959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:12.856976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093479+0000","last_change":"2024-04-25T08:34:44.525749+0000","last_active":"2024-04-25T08:34:46.093479+0000","last_peered":"2024-04-25T08:34:46.093479+0000","last_clean":"2024-04-25T08:34:46.093479+0000","last_became_active":"2024-04-25T08:34:44.525470+0000","last_became_peered":"2024-04-25T08:34:44.525470+0000","last_unstale":"2024-04-25T08:34:46.093479+0000","last_undegraded":"2024-04-25T08:34:46.093479+0000","last_fullsized":"2024-04-25T08:34:46.093479+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:40.781389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530997+0000","last_change":"2024-04-25T08:34:43.876071+0000","last_active":"2024-04-25T08:34:45.530997+0000","last_peered":"2024-04-25T08:34:45.530997+0000","last_clean":"2024-04-25T08:34:45.530997+0000","last_became_active":"2024-04-25T08:34:43.875782+0000","last_became_peered":"2024-04-25T08:34:43.875782+0000","last_unstale":"2024-04-25T08:34:45.530997+0000","last_undegraded":"2024-04-25T08:34:45.530997+0000","last_fullsized":"2024-04-25T08:34:45.530997+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:39.042852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094436+0000","last_change":"2024-04-25T08:34:44.528331+0000","last_active":"2024-04-25T08:34:46.094436+0000","last_peered":"2024-04-25T08:34:46.094436+0000","last_clean":"2024-04-25T08:34:46.094436+0000","last_became_active":"2024-04-25T08:34:44.528080+0000","last_became_peered":"2024-04-25T08:34:44.528080+0000","last_unstale":"2024-04-25T08:34:46.094436+0000","last_undegraded":"2024-04-25T08:34:46.094436+0000","last_fullsized":"2024-04-25T08:34:46.094436+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:25.030847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531206+0000","last_change":"2024-04-25T08:34:44.529717+0000","last_active":"2024-04-25T08:34:45.531206+0000","last_peered":"2024-04-25T08:34:45.531206+0000","last_clean":"2024-04-25T08:34:45.531206+0000","last_became_active":"2024-04-25T08:34:44.529442+0000","last_became_peered":"2024-04-25T08:34:44.529442+0000","last_unstale":"2024-04-25T08:34:45.531206+0000","last_undegraded":"2024-04-25T08:34:45.531206+0000","last_fullsized":"2024-04-25T08:34:45.531206+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:57.569222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093571+0000","last_change":"2024-04-25T08:34:44.532534+0000","last_active":"2024-04-25T08:34:46.093571+0000","last_peered":"2024-04-25T08:34:46.093571+0000","last_clean":"2024-04-25T08:34:46.093571+0000","last_became_active":"2024-04-25T08:34:44.532250+0000","last_became_peered":"2024-04-25T08:34:44.532250+0000","last_unstale":"2024-04-25T08:34:46.093571+0000","last_undegraded":"2024-04-25T08:34:46.093571+0000","last_fullsized":"2024-04-25T08:34:46.093571+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:11.760981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094802+0000","last_change":"2024-04-25T08:34:43.883681+0000","last_active":"2024-04-25T08:34:46.094802+0000","last_peered":"2024-04-25T08:34:46.094802+0000","last_clean":"2024-04-25T08:34:46.094802+0000","last_became_active":"2024-04-25T08:34:43.883385+0000","last_became_peered":"2024-04-25T08:34:43.883385+0000","last_unstale":"2024-04-25T08:34:46.094802+0000","last_undegraded":"2024-04-25T08:34:46.094802+0000","last_fullsized":"2024-04-25T08:34:46.094802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:17.727985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094016+0000","last_change":"2024-04-25T08:34:44.532945+0000","last_active":"2024-04-25T08:34:46.094016+0000","last_peered":"2024-04-25T08:34:46.094016+0000","last_clean":"2024-04-25T08:34:46.094016+0000","last_became_active":"2024-04-25T08:34:44.532563+0000","last_became_peered":"2024-04-25T08:34:44.532563+0000","last_unstale":"2024-04-25T08:34:46.094016+0000","last_undegraded":"2024-04-25T08:34:46.094016+0000","last_fullsized":"2024-04-25T08:34:46.094016+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:18.966389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530710+0000","last_change":"2024-04-25T08:34:43.882768+0000","last_active":"2024-04-25T08:34:45.530710+0000","last_peered":"2024-04-25T08:34:45.530710+0000","last_clean":"2024-04-25T08:34:45.530710+0000","last_became_active":"2024-04-25T08:34:43.882459+0000","last_became_peered":"2024-04-25T08:34:43.882459+0000","last_unstale":"2024-04-25T08:34:45.530710+0000","last_undegraded":"2024-04-25T08:34:45.530710+0000","last_fullsized":"2024-04-25T08:34:45.530710+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:46.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094908+0000","last_change":"2024-04-25T08:34:44.533491+0000","last_active":"2024-04-25T08:34:46.094908+0000","last_peered":"2024-04-25T08:34:46.094908+0000","last_clean":"2024-04-25T08:34:46.094908+0000","last_became_active":"2024-04-25T08:34:44.532257+0000","last_became_peered":"2024-04-25T08:34:44.532257+0000","last_unstale":"2024-04-25T08:34:46.094908+0000","last_undegraded":"2024-04-25T08:34:46.094908+0000","last_fullsized":"2024-04-25T08:34:46.094908+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:41.495853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095013+0000","last_change":"2024-04-25T08:34:43.889631+0000","last_active":"2024-04-25T08:34:46.095013+0000","last_peered":"2024-04-25T08:34:46.095013+0000","last_clean":"2024-04-25T08:34:46.095013+0000","last_became_active":"2024-04-25T08:34:43.889354+0000","last_became_peered":"2024-04-25T08:34:43.889354+0000","last_unstale":"2024-04-25T08:34:46.095013+0000","last_undegraded":"2024-04-25T08:34:46.095013+0000","last_fullsized":"2024-04-25T08:34:46.095013+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:02.862950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086134+0000","last_change":"2024-04-25T08:34:44.533026+0000","last_active":"2024-04-25T08:34:46.086134+0000","last_peered":"2024-04-25T08:34:46.086134+0000","last_clean":"2024-04-25T08:34:46.086134+0000","last_became_active":"2024-04-25T08:34:44.532759+0000","last_became_peered":"2024-04-25T08:34:44.532759+0000","last_unstale":"2024-04-25T08:34:46.086134+0000","last_undegraded":"2024-04-25T08:34:46.086134+0000","last_fullsized":"2024-04-25T08:34:46.086134+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:53.870186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531275+0000","last_change":"2024-04-25T08:34:44.532873+0000","last_active":"2024-04-25T08:34:45.531275+0000","last_peered":"2024-04-25T08:34:45.531275+0000","last_clean":"2024-04-25T08:34:45.531275+0000","last_became_active":"2024-04-25T08:34:44.532594+0000","last_became_peered":"2024-04-25T08:34:44.532594+0000","last_unstale":"2024-04-25T08:34:45.531275+0000","last_undegraded":"2024-04-25T08:34:45.531275+0000","last_fullsized":"2024-04-25T08:34:45.531275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:52.822815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530597+0000","last_change":"2024-04-25T08:34:43.877431+0000","last_active":"2024-04-25T08:34:45.530597+0000","last_peered":"2024-04-25T08:34:45.530597+0000","last_clean":"2024-04-25T08:34:45.530597+0000","last_became_active":"2024-04-25T08:34:43.877115+0000","last_became_peered":"2024-04-25T08:34:43.877115+0000","last_unstale":"2024-04-25T08:34:45.530597+0000","last_undegraded":"2024-04-25T08:34:45.530597+0000","last_fullsized":"2024-04-25T08:34:45.530597+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:55.485664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086192+0000","last_change":"2024-04-25T08:34:44.527817+0000","last_active":"2024-04-25T08:34:46.086192+0000","last_peered":"2024-04-25T08:34:46.086192+0000","last_clean":"2024-04-25T08:34:46.086192+0000","last_became_active":"2024-04-25T08:34:44.527550+0000","last_became_peered":"2024-04-25T08:34:44.527550+0000","last_unstale":"2024-04-25T08:34:46.086192+0000","last_undegraded":"2024-04-25T08:34:46.086192+0000","last_fullsized":"2024-04-25T08:34:46.086192+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:13.210050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530234+0000","last_change":"2024-04-25T08:34:43.883793+0000","last_active":"2024-04-25T08:34:45.530234+0000","last_peered":"2024-04-25T08:34:45.530234+0000","last_clean":"2024-04-25T08:34:45.530234+0000","last_became_active":"2024-04-25T08:34:43.882987+0000","last_became_peered":"2024-04-25T08:34:43.882987+0000","last_unstale":"2024-04-25T08:34:45.530234+0000","last_undegraded":"2024-04-25T08:34:45.530234+0000","last_fullsized":"2024-04-25T08:34:45.530234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:04.802285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093760+0000","last_change":"2024-04-25T08:34:44.528347+0000","last_active":"2024-04-25T08:34:46.093760+0000","last_peered":"2024-04-25T08:34:46.093760+0000","last_clean":"2024-04-25T08:34:46.093760+0000","last_became_active":"2024-04-25T08:34:44.528059+0000","last_became_peered":"2024-04-25T08:34:44.528059+0000","last_unstale":"2024-04-25T08:34:46.093760+0000","last_undegraded":"2024-04-25T08:34:46.093760+0000","last_fullsized":"2024-04-25T08:34:46.093760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.919973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086286+0000","last_change":"2024-04-25T08:34:43.878609+0000","last_active":"2024-04-25T08:34:46.086286+0000","last_peered":"2024-04-25T08:34:46.086286+0000","last_clean":"2024-04-25T08:34:46.086286+0000","last_became_active":"2024-04-25T08:34:43.878322+0000","last_became_peered":"2024-04-25T08:34:43.878322+0000","last_unstale":"2024-04-25T08:34:46.086286+0000","last_undegraded":"2024-04-25T08:34:46.086286+0000","last_fullsized":"2024-04-25T08:34:46.086286+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:46.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530698+0000","last_change":"2024-04-25T08:34:44.519833+0000","last_active":"2024-04-25T08:34:45.530698+0000","last_peered":"2024-04-25T08:34:45.530698+0000","last_clean":"2024-04-25T08:34:45.530698+0000","last_became_active":"2024-04-25T08:34:44.519535+0000","last_became_peered":"2024-04-25T08:34:44.519535+0000","last_unstale":"2024-04-25T08:34:45.530698+0000","last_undegraded":"2024-04-25T08:34:45.530698+0000","last_fullsized":"2024-04-25T08:34:45.530698+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.499854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094705+0000","last_change":"2024-04-25T08:34:44.527976+0000","last_active":"2024-04-25T08:34:46.094705+0000","last_peered":"2024-04-25T08:34:46.094705+0000","last_clean":"2024-04-25T08:34:46.094705+0000","last_became_active":"2024-04-25T08:34:44.527199+0000","last_became_peered":"2024-04-25T08:34:44.527199+0000","last_unstale":"2024-04-25T08:34:46.094705+0000","last_undegraded":"2024-04-25T08:34:46.094705+0000","last_fullsized":"2024-04-25T08:34:46.094705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:26.675283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094290+0000","last_change":"2024-04-25T08:34:43.880717+0000","last_active":"2024-04-25T08:34:46.094290+0000","last_peered":"2024-04-25T08:34:46.094290+0000","last_clean":"2024-04-25T08:34:46.094290+0000","last_became_active":"2024-04-25T08:34:43.880417+0000","last_became_peered":"2024-04-25T08:34:43.880417+0000","last_unstale":"2024-04-25T08:34:46.094290+0000","last_undegraded":"2024-04-25T08:34:46.094290+0000","last_fullsized":"2024-04-25T08:34:46.094290+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:29.445107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086364+0000","last_change":"2024-04-25T08:34:44.532688+0000","last_active":"2024-04-25T08:34:46.086364+0000","last_peered":"2024-04-25T08:34:46.086364+0000","last_clean":"2024-04-25T08:34:46.086364+0000","last_became_active":"2024-04-25T08:34:44.532430+0000","last_became_peered":"2024-04-25T08:34:44.532430+0000","last_unstale":"2024-04-25T08:34:46.086364+0000","last_undegraded":"2024-04-25T08:34:46.086364+0000","last_fullsized":"2024-04-25T08:34:46.086364+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:19.629450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093707+0000","last_change":"2024-04-25T08:34:44.532922+0000","last_active":"2024-04-25T08:34:46.093707+0000","last_peered":"2024-04-25T08:34:46.093707+0000","last_clean":"2024-04-25T08:34:46.093707+0000","last_became_active":"2024-04-25T08:34:44.532361+0000","last_became_peered":"2024-04-25T08:34:44.532361+0000","last_unstale":"2024-04-25T08:34:46.093707+0000","last_undegraded":"2024-04-25T08:34:46.093707+0000","last_fullsized":"2024-04-25T08:34:46.093707+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:05.622445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530587+0000","last_change":"2024-04-25T08:34:43.881524+0000","last_active":"2024-04-25T08:34:45.530587+0000","last_peered":"2024-04-25T08:34:45.530587+0000","last_clean":"2024-04-25T08:34:45.530587+0000","last_became_active":"2024-04-25T08:34:43.881273+0000","last_became_peered":"2024-04-25T08:34:43.881273+0000","last_unstale":"2024-04-25T08:34:45.530587+0000","last_undegraded":"2024-04-25T08:34:45.530587+0000","last_fullsized":"2024-04-25T08:34:45.530587+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:54.880133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086187+0000","last_change":"2024-04-25T08:34:44.531176+0000","last_active":"2024-04-25T08:34:46.086187+0000","last_peered":"2024-04-25T08:34:46.086187+0000","last_clean":"2024-04-25T08:34:46.086187+0000","last_became_active":"2024-04-25T08:34:44.529858+0000","last_became_peered":"2024-04-25T08:34:44.529858+0000","last_unstale":"2024-04-25T08:34:46.086187+0000","last_undegraded":"2024-04-25T08:34:46.086187+0000","last_fullsized":"2024-04-25T08:34:46.086187+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:03.865071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093840+0000","last_change":"2024-04-25T08:34:43.875641+0000","last_active":"2024-04-25T08:34:46.093840+0000","last_peered":"2024-04-25T08:34:46.093840+0000","last_clean":"2024-04-25T08:34:46.093840+0000","last_became_active":"2024-04-25T08:34:43.875344+0000","last_became_peered":"2024-04-25T08:34:43.875344+0000","last_unstale":"2024-04-25T08:34:46.093840+0000","last_undegraded":"2024-04-25T08:34:46.093840+0000","last_fullsized":"2024-04-25T08:34:46.093840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:44.497733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093679+0000","last_change":"2024-04-25T08:34:44.532615+0000","last_active":"2024-04-25T08:34:46.093679+0000","last_peered":"2024-04-25T08:34:46.093679+0000","last_clean":"2024-04-25T08:34:46.093679+0000","last_became_active":"2024-04-25T08:34:44.532335+0000","last_became_peered":"2024-04-25T08:34:44.532335+0000","last_unstale":"2024-04-25T08:34:46.093679+0000","last_undegraded":"2024-04-25T08:34:46.093679+0000","last_fullsized":"2024-04-25T08:34:46.093679+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:05.252545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093705+0000","last_change":"2024-04-25T08:34:44.529316+0000","last_active":"2024-04-25T08:34:46.093705+0000","last_peered":"2024-04-25T08:34:46.093705+0000","last_clean":"2024-04-25T08:34:46.093705+0000","last_became_active":"2024-04-25T08:34:44.529026+0000","last_became_peered":"2024-04-25T08:34:44.529026+0000","last_unstale":"2024-04-25T08:34:46.093705+0000","last_undegraded":"2024-04-25T08:34:46.093705+0000","last_fullsized":"2024-04-25T08:34:46.093705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:33.006310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:42.853277+0000","last_change":"2024-04-25T08:34:42.853277+0000","last_active":"2024-04-25T08:34:42.853277+0000","last_peered":"2024-04-25T08:34:42.853277+0000","last_clean":"2024-04-25T08:34:42.853277+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:42.853277+0000","last_undegraded":"2024-04-25T08:34:42.853277+0000","last_fullsized":"2024-04-25T08:34:42.853277+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094563+0000","last_change":"2024-04-25T08:34:44.536436+0000","last_active":"2024-04-25T08:34:46.094563+0000","last_peered":"2024-04-25T08:34:46.094563+0000","last_clean":"2024-04-25T08:34:46.094563+0000","last_became_active":"2024-04-25T08:34:44.534485+0000","last_became_peered":"2024-04-25T08:34:44.534485+0000","last_unstale":"2024-04-25T08:34:46.094563+0000","last_undegraded":"2024-04-25T08:34:46.094563+0000","last_fullsized":"2024-04-25T08:34:46.094563+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:31.481085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531115+0000","last_change":"2024-04-25T08:34:43.883072+0000","last_active":"2024-04-25T08:34:45.531115+0000","last_peered":"2024-04-25T08:34:45.531115+0000","last_clean":"2024-04-25T08:34:45.531115+0000","last_became_active":"2024-04-25T08:34:43.882688+0000","last_became_peered":"2024-04-25T08:34:43.882688+0000","last_unstale":"2024-04-25T08:34:45.531115+0000","last_undegraded":"2024-04-25T08:34:45.531115+0000","last_fullsized":"2024-04-25T08:34:45.531115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:16.493814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:34:43.855398+0000","last_change":"2024-04-25T08:34:43.855398+0000","last_active":"2024-04-25T08:34:43.855398+0000","last_peered":"2024-04-25T08:34:43.855398+0000","last_clean":"2024-04-25T08:34:43.855398+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:34:43.855398+0000","last_undegraded":"2024-04-25T08:34:43.855398+0000","last_fullsized":"2024-04-25T08:34:43.855398+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093700+0000","last_change":"2024-04-25T08:34:43.883730+0000","last_active":"2024-04-25T08:34:46.093700+0000","last_peered":"2024-04-25T08:34:46.093700+0000","last_clean":"2024-04-25T08:34:46.093700+0000","last_became_active":"2024-04-25T08:34:43.883435+0000","last_became_peered":"2024-04-25T08:34:43.883435+0000","last_unstale":"2024-04-25T08:34:46.093700+0000","last_undegraded":"2024-04-25T08:34:46.093700+0000","last_fullsized":"2024-04-25T08:34:46.093700+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:10.731254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530395+0000","last_change":"2024-04-25T08:34:44.526203+0000","last_active":"2024-04-25T08:34:45.530395+0000","last_peered":"2024-04-25T08:34:45.530395+0000","last_clean":"2024-04-25T08:34:45.530395+0000","last_became_active":"2024-04-25T08:34:44.525945+0000","last_became_peered":"2024-04-25T08:34:44.525945+0000","last_unstale":"2024-04-25T08:34:45.530395+0000","last_undegraded":"2024-04-25T08:34:45.530395+0000","last_fullsized":"2024-04-25T08:34:45.530395+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.447228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.869817+0000","last_change":"2024-04-25T08:34:43.881671+0000","last_active":"2024-04-25T08:34:46.869817+0000","last_peered":"2024-04-25T08:34:46.869817+0000","last_clean":"2024-04-25T08:34:46.869817+0000","last_became_active":"2024-04-25T08:34:43.881375+0000","last_became_peered":"2024-04-25T08:34:43.881375+0000","last_unstale":"2024-04-25T08:34:46.869817+0000","last_undegraded":"2024-04-25T08:34:46.869817+0000","last_fullsized":"2024-04-25T08:34:46.869817+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:48.237385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":227,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-25T08:34:42.092335+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:34:42.092335+0000","last_peered":"2024-04-25T08:34:42.092335+0000","last_clean":"2024-04-25T08:34:42.092335+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:34:42.092335+0000","last_undegraded":"2024-04-25T08:34:42.092335+0000","last_fullsized":"2024-04-25T08:34:42.092335+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":86,"acting":86,"num_store_stats":5},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":708,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":42,"acting":42,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58704,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313136,"statfs":{"total":96636764160,"available":96576651264,"internally_reserved":0,"allocated":1175552,"data_stored":847826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40144,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331696,"statfs":{"total":96636764160,"available":96595656704,"internally_reserved":0,"allocated":651264,"data_stored":290364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35972,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335868,"statfs":{"total":96636764160,"available":96599928832,"internally_reserved":0,"allocated":643072,"data_stored":289712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58184,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313656,"statfs":{"total":96636764160,"available":96577183744,"internally_reserved":0,"allocated":647168,"data_stored":290294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":44244,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327596,"statfs":{"total":96636764160,"available":96591458304,"internally_reserved":0,"allocated":593920,"data_stored":262977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40084,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331756,"statfs":{"total":96636764160,"available":96595718144,"internally_reserved":0,"allocated":593920,"data_stored":262977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":651264,"data_stored":289872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50444,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321396,"statfs":{"total":96636764160,"available":96585109504,"internally_reserved":0,"allocated":1245184,"data_stored":880136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:34:48.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2024-04-25T08:34:48.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2024-04-25T08:34:48.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-04-25T08:34:48.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-04-25T08:34:48.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2024-04-25T08:34:48.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-04-25T08:34:48.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2024-04-25T08:34:48.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2024-04-25T08:34:48.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-04-25T08:34:48.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-04-25T08:34:48.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-04-25T08:34:48.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2024-04-25T08:34:48.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-04-25T08:34:48.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-04-25T08:34:48.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-04-25T08:34:48.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-04-25T08:34:48.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-04-25T08:34:48.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-25T08:34:48.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:42.853277+0000', 'last_change': '2024-04-25T08:34:42.853277+0000', 'last_active': '2024-04-25T08:34:42.853277+0000', 'last_peered': '2024-04-25T08:34:42.853277+0000', 'last_clean': '2024-04-25T08:34:42.853277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:42.853277+0000', 'last_undegraded': '2024-04-25T08:34:42.853277+0000', 'last_fullsized': '2024-04-25T08:34:42.853277+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-25T08:34:48.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:42.853277+0000', 'last_change': '2024-04-25T08:34:42.853277+0000', 'last_active': '2024-04-25T08:34:42.853277+0000', 'last_peered': '2024-04-25T08:34:42.853277+0000', 'last_clean': '2024-04-25T08:34:42.853277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:42.853277+0000', 'last_undegraded': '2024-04-25T08:34:42.853277+0000', 'last_fullsized': '2024-04-25T08:34:42.853277+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-25T08:34:48.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:42.853277+0000', 'last_change': '2024-04-25T08:34:42.853277+0000', 'last_active': '2024-04-25T08:34:42.853277+0000', 'last_peered': '2024-04-25T08:34:42.853277+0000', 'last_clean': '2024-04-25T08:34:42.853277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:42.853277+0000', 'last_undegraded': '2024-04-25T08:34:42.853277+0000', 'last_fullsized': '2024-04-25T08:34:42.853277+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-04-25T08:34:48.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-25T08:34:48.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:42.853277+0000', 'last_change': '2024-04-25T08:34:42.853277+0000', 'last_active': '2024-04-25T08:34:42.853277+0000', 'last_peered': '2024-04-25T08:34:42.853277+0000', 'last_clean': '2024-04-25T08:34:42.853277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:42.853277+0000', 'last_undegraded': '2024-04-25T08:34:42.853277+0000', 'last_fullsized': '2024-04-25T08:34:42.853277+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-25T08:34:48.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:42.853277+0000', 'last_change': '2024-04-25T08:34:42.853277+0000', 'last_active': '2024-04-25T08:34:42.853277+0000', 'last_peered': '2024-04-25T08:34:42.853277+0000', 'last_clean': '2024-04-25T08:34:42.853277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:42.853277+0000', 'last_undegraded': '2024-04-25T08:34:42.853277+0000', 'last_fullsized': '2024-04-25T08:34:42.853277+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-04-25T08:34:48.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-04-25T08:34:48.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:42.853277+0000', 'last_change': '2024-04-25T08:34:42.853277+0000', 'last_active': '2024-04-25T08:34:42.853277+0000', 'last_peered': '2024-04-25T08:34:42.853277+0000', 'last_clean': '2024-04-25T08:34:42.853277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:42.853277+0000', 'last_undegraded': '2024-04-25T08:34:42.853277+0000', 'last_fullsized': '2024-04-25T08:34:42.853277+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-25T08:34:48.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:42.853277+0000', 'last_change': '2024-04-25T08:34:42.853277+0000', 'last_active': '2024-04-25T08:34:42.853277+0000', 'last_peered': '2024-04-25T08:34:42.853277+0000', 'last_clean': '2024-04-25T08:34:42.853277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:42.853277+0000', 'last_undegraded': '2024-04-25T08:34:42.853277+0000', 'last_fullsized': '2024-04-25T08:34:42.853277+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-25T08:34:48.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:42.853277+0000', 'last_change': '2024-04-25T08:34:42.853277+0000', 'last_active': '2024-04-25T08:34:42.853277+0000', 'last_peered': '2024-04-25T08:34:42.853277+0000', 'last_clean': '2024-04-25T08:34:42.853277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:42.853277+0000', 'last_undegraded': '2024-04-25T08:34:42.853277+0000', 'last_fullsized': '2024-04-25T08:34:42.853277+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-25T08:34:48.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:42.853277+0000', 'last_change': '2024-04-25T08:34:42.853277+0000', 'last_active': '2024-04-25T08:34:42.853277+0000', 'last_peered': '2024-04-25T08:34:42.853277+0000', 'last_clean': '2024-04-25T08:34:42.853277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:42.853277+0000', 'last_undegraded': '2024-04-25T08:34:42.853277+0000', 'last_fullsized': '2024-04-25T08:34:42.853277+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-04-25T08:34:48.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-25T08:34:48.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:42.853277+0000', 'last_change': '2024-04-25T08:34:42.853277+0000', 'last_active': '2024-04-25T08:34:42.853277+0000', 'last_peered': '2024-04-25T08:34:42.853277+0000', 'last_clean': '2024-04-25T08:34:42.853277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:42.853277+0000', 'last_undegraded': '2024-04-25T08:34:42.853277+0000', 'last_fullsized': '2024-04-25T08:34:42.853277+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-25T08:34:48.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:42.853277+0000', 'last_change': '2024-04-25T08:34:42.853277+0000', 'last_active': '2024-04-25T08:34:42.853277+0000', 'last_peered': '2024-04-25T08:34:42.853277+0000', 'last_clean': '2024-04-25T08:34:42.853277+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:42.853277+0000', 'last_undegraded': '2024-04-25T08:34:42.853277+0000', 'last_fullsized': '2024-04-25T08:34:42.853277+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:42.853277+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-04-25T08:34:48.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:34:43.855398+0000', 'last_change': '2024-04-25T08:34:43.855398+0000', 'last_active': '2024-04-25T08:34:43.855398+0000', 'last_peered': '2024-04-25T08:34:43.855398+0000', 'last_clean': '2024-04-25T08:34:43.855398+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:34:43.855398+0000', 'last_undegraded': '2024-04-25T08:34:43.855398+0000', 'last_fullsized': '2024-04-25T08:34:43.855398+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'last_clean_scrub_stamp': '2024-04-25T08:34:43.855398+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:34:48.579 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:34:48.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.714+0000 7fb7895c1640 1 Processor -- start 2024-04-25T08:34:48.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.714+0000 7fb7895c1640 1 -- start start 2024-04-25T08:34:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.715+0000 7fb7895c1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb78406c110 0x7fb78406c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.715+0000 7fb7895c1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb78415df50 0x7fb7840ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.715+0000 7fb7895c1640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78406ca30 con 0x7fb7840ce2e0 2024-04-25T08:34:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.715+0000 7fb7895c1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7840cf2d0 con 0x7fb78406c110 2024-04-25T08:34:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.715+0000 7fb7895c1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7840ceff0 con 0x7fb78415df50 2024-04-25T08:34:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.715+0000 7fb782ffd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb78406c110 0x7fb78406c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.715+0000 7fb7837fe640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fb7840ce2e0 0x7fb7840d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:42452/0 (socket says 172.21.15.38:42452) 2024-04-25T08:34:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.715+0000 7fb7827fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb78415df50 0x7fb7840ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.715+0000 7fb7837fe640 1 -- 172.21.15.38:0/1848565173 learned_addr learned my addr 172.21.15.38:0/1848565173 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.715+0000 7fb782ffd640 1 -- 172.21.15.38:0/1848565173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb78415df50 msgr2=0x7fb7840ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.715+0000 7fb782ffd640 1 --2- 172.21.15.38:0/1848565173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb78415df50 0x7fb7840ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.716+0000 7fb782ffd640 1 -- 172.21.15.38:0/1848565173 >> v1:172.21.15.107:6789/0 conn(0x7fb7840ce2e0 legacy=0x7fb7840d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.716+0000 7fb782ffd640 1 -- 172.21.15.38:0/1848565173 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7840d1f70 con 0x7fb78406c110 2024-04-25T08:34:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.716+0000 7fb7827fc640 1 --2- 172.21.15.38:0/1848565173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb78415df50 0x7fb7840ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.716+0000 7fb782ffd640 1 --2- 172.21.15.38:0/1848565173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb78406c110 0x7fb78406c4f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097a0 tx=0x7fb77402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab24a65c39bbe755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.716+0000 7fb781ffb640 1 -- 172.21.15.38:0/1848565173 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774037070 con 0x7fb78406c110 2024-04-25T08:34:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.717+0000 7fb781ffb640 1 -- 172.21.15.38:0/1848565173 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb774032440 con 0x7fb78406c110 2024-04-25T08:34:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.717+0000 7fb781ffb640 1 -- 172.21.15.38:0/1848565173 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774032810 con 0x7fb78406c110 2024-04-25T08:34:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.717+0000 7fb7895c1640 1 -- 172.21.15.38:0/1848565173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb78406c110 msgr2=0x7fb78406c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.717+0000 7fb7895c1640 1 --2- 172.21.15.38:0/1848565173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb78406c110 0x7fb78406c4f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097a0 tx=0x7fb77402a180 comp rx=0 tx=0).stop 2024-04-25T08:34:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.717+0000 7fb7895c1640 1 -- 172.21.15.38:0/1848565173 shutdown_connections 2024-04-25T08:34:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.717+0000 7fb7895c1640 1 --2- 172.21.15.38:0/1848565173 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb78415df50 0x7fb7840ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.717+0000 7fb7895c1640 1 --2- 172.21.15.38:0/1848565173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb78406c110 0x7fb78406c4f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.717+0000 7fb7895c1640 1 -- 172.21.15.38:0/1848565173 >> 172.21.15.38:0/1848565173 conn(0x7fb7840d3fb0 msgr2=0x7fb7840d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.717+0000 7fb7895c1640 1 -- 172.21.15.38:0/1848565173 shutdown_connections 2024-04-25T08:34:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.717+0000 7fb7895c1640 1 -- 172.21.15.38:0/1848565173 wait complete. 2024-04-25T08:34:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.718+0000 7fb7895c1640 1 Processor -- start 2024-04-25T08:34:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.718+0000 7fb7895c1640 1 -- start start 2024-04-25T08:34:48.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.719+0000 7fb7895c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb78406c110 0x7fb7840c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.719+0000 7fb7895c1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7840ce2e0 0x7fb7840cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.719+0000 7fb7895c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb78415df50 0x7fb7840c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.719+0000 7fb7895c1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7840cece0 con 0x7fb78406c110 2024-04-25T08:34:48.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.719+0000 7fb7895c1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7840d2530 con 0x7fb78415df50 2024-04-25T08:34:48.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.719+0000 7fb7895c1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7840d3160 con 0x7fb7840ce2e0 2024-04-25T08:34:48.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.719+0000 7fb782ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb78406c110 0x7fb7840c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.719+0000 7fb782ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb78406c110 0x7fb7840c6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51102/0 (socket says 172.21.15.38:51102) 2024-04-25T08:34:48.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.719+0000 7fb782ffd640 1 -- 172.21.15.38:0/1833833109 learned_addr learned my addr 172.21.15.38:0/1833833109 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:48.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb7837fe640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb78415df50 0x7fb7840c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb7827fc640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7840ce2e0 0x7fb7840cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb782ffd640 1 -- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7840ce2e0 msgr2=0x7fb7840cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb782ffd640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7840ce2e0 0x7fb7840cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb782ffd640 1 -- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb78415df50 msgr2=0x7fb7840c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb782ffd640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb78415df50 0x7fb7840c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb782ffd640 1 -- 172.21.15.38:0/1833833109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7840cb860 con 0x7fb78406c110 2024-04-25T08:34:48.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb782ffd640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb78406c110 0x7fb7840c6460 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb7740096f0 tx=0x7fb7740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:48.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb76b7fe640 1 -- 172.21.15.38:0/1833833109 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774037040 con 0x7fb78406c110 2024-04-25T08:34:48.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb76b7fe640 1 -- 172.21.15.38:0/1833833109 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb774030500 con 0x7fb78406c110 2024-04-25T08:34:48.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb7895c1640 1 -- 172.21.15.38:0/1833833109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7840cbb40 con 0x7fb78406c110 2024-04-25T08:34:48.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb76b7fe640 1 -- 172.21.15.38:0/1833833109 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7740308d0 con 0x7fb78406c110 2024-04-25T08:34:48.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.720+0000 7fb7895c1640 1 -- 172.21.15.38:0/1833833109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7840cc100 con 0x7fb78406c110 2024-04-25T08:34:48.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.721+0000 7fb7895c1640 1 -- 172.21.15.38:0/1833833109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb784068e80 con 0x7fb78406c110 2024-04-25T08:34:48.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.722+0000 7fb76b7fe640 1 -- 172.21.15.38:0/1833833109 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb774044020 con 0x7fb78406c110 2024-04-25T08:34:48.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.722+0000 7fb76b7fe640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb77006b4d0 0x7fb77006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:48.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.722+0000 7fb76b7fe640 1 -- 172.21.15.38:0/1833833109 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb7740aa380 con 0x7fb78406c110 2024-04-25T08:34:48.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.722+0000 7fb7827fc640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb77006b4d0 0x7fb77006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:48.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.723+0000 7fb7827fc640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb77006b4d0 0x7fb77006d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb7840b3570 tx=0x7fb778008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:48.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.726+0000 7fb76b7fe640 1 -- 172.21.15.38:0/1833833109 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb77404b020 con 0x7fb78406c110 2024-04-25T08:34:48.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.979+0000 7fb7895c1640 1 -- 172.21.15.38:0/1833833109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb78415d500 con 0x7fb78406c110 2024-04-25T08:34:48.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.979+0000 7fb76b7fe640 1 -- 172.21.15.38:0/1833833109 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1089 (secure 0 0 0) 0x7fb774079100 con 0x7fb78406c110 2024-04-25T08:34:48.976 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:48.976 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":217,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":65},{"state_name":"unknown","count":32}],"num_pgs":97,"num_pools":3,"num_objects":13,"data_bytes":591076,"bytes_used":376745984,"bytes_avail":772717367296,"bytes_total":773094113280,"unknown_pgs_ratio":0.32989689707756042,"write_bytes_sec":2330,"read_op_per_sec":0,"write_op_per_sec":4},"fsmap":{"epoch":24,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5964}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-04-25T08:33:55.236880+0000","services":{}},"progress_events":{}} 2024-04-25T08:34:48.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.981+0000 7fb7895c1640 1 -- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb77006b4d0 msgr2=0x7fb77006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.981+0000 7fb7895c1640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb77006b4d0 0x7fb77006d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb7840b3570 tx=0x7fb778008040 comp rx=0 tx=0).stop 2024-04-25T08:34:48.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.981+0000 7fb7895c1640 1 -- 172.21.15.38:0/1833833109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb78406c110 msgr2=0x7fb7840c6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:48.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.981+0000 7fb7895c1640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb78406c110 0x7fb7840c6460 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fb7740096f0 tx=0x7fb7740046c0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.981+0000 7fb7895c1640 1 -- 172.21.15.38:0/1833833109 shutdown_connections 2024-04-25T08:34:48.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.981+0000 7fb7895c1640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb78415df50 0x7fb7840c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.981+0000 7fb7895c1640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb77006b4d0 0x7fb77006d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.981+0000 7fb7895c1640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7840ce2e0 0x7fb7840cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.981+0000 7fb7895c1640 1 --2- 172.21.15.38:0/1833833109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb78406c110 0x7fb7840c6460 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:48.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.981+0000 7fb7895c1640 1 -- 172.21.15.38:0/1833833109 >> 172.21.15.38:0/1833833109 conn(0x7fb7840d3fb0 msgr2=0x7fb7840d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:48.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.981+0000 7fb7895c1640 1 -- 172.21.15.38:0/1833833109 shutdown_connections 2024-04-25T08:34:48.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:48.981+0000 7fb7895c1640 1 -- 172.21.15.38:0/1833833109 wait complete. 2024-04-25T08:34:48.988 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:34:48.989 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:34:49.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.093+0000 7fd5ab438640 1 Processor -- start 2024-04-25T08:34:49.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.093+0000 7fd5ab438640 1 -- start start 2024-04-25T08:34:49.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.094+0000 7fd5ab438640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd5a415da90 0x7fd5a40d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:49.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.094+0000 7fd5ab438640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a40d3fb0 0x7fd5a40b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:49.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.094+0000 7fd5ab438640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a40d1b60 con 0x7fd5a40ceca0 2024-04-25T08:34:49.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.094+0000 7fd5ab438640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a40d2bc0 con 0x7fd5a415da90 2024-04-25T08:34:49.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.094+0000 7fd5ab438640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40d1e00 con 0x7fd5a40d3fb0 2024-04-25T08:34:49.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.094+0000 7fd5a89d3640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd5a40ceca0 0x7fd5a40d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54250/0 (socket says 172.21.15.38:54250) 2024-04-25T08:34:49.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.094+0000 7fd5a89d3640 1 -- 172.21.15.38:0/2231522217 learned_addr learned my addr 172.21.15.38:0/2231522217 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:49.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.095+0000 7fd5a99d5640 1 --2- 172.21.15.38:0/2231522217 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a40d3fb0 0x7fd5a40b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:49.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.095+0000 7fd5a91d4640 1 --2- 172.21.15.38:0/2231522217 >> v2:172.21.15.107:3300/0 conn(0x7fd5a415da90 0x7fd5a40d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:49.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.095+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2231522217 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574065552 0 0) 0x7fd5a40d1b60 con 0x7fd5a40ceca0 2024-04-25T08:34:49.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.095+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2231522217 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd58c003520 con 0x7fd5a40ceca0 2024-04-25T08:34:49.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.095+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2231522217 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3642456517 0 0) 0x7fd58c003520 con 0x7fd5a40ceca0 2024-04-25T08:34:49.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.095+0000 7fd5a99d5640 1 -- 172.21.15.38:0/2231522217 >> v2:172.21.15.107:3300/0 conn(0x7fd5a415da90 msgr2=0x7fd5a40d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.095+0000 7fd5a99d5640 1 --2- 172.21.15.38:0/2231522217 >> v2:172.21.15.107:3300/0 conn(0x7fd5a415da90 0x7fd5a40d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.095+0000 7fd5a99d5640 1 -- 172.21.15.38:0/2231522217 >> v1:172.21.15.38:6789/0 conn(0x7fd5a40ceca0 legacy=0x7fd5a40d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.095+0000 7fd5a99d5640 1 -- 172.21.15.38:0/2231522217 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a40d20e0 con 0x7fd5a40d3fb0 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.095+0000 7fd5a91d4640 1 --2- 172.21.15.38:0/2231522217 >> v2:172.21.15.107:3300/0 conn(0x7fd5a415da90 0x7fd5a40d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.096+0000 7fd5a99d5640 1 --2- 172.21.15.38:0/2231522217 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a40d3fb0 0x7fd5a40b69a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd594009d90 tx=0x7fd59402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac061617571326f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.096+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2231522217 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598001690 con 0x7fd5a40d3fb0 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.096+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2231522217 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd594007540 con 0x7fd5a40d3fb0 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.096+0000 7fd5a3fff640 1 -- 172.21.15.38:0/2231522217 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5940078f0 con 0x7fd5a40d3fb0 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.096+0000 7fd5ab438640 1 -- 172.21.15.38:0/2231522217 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a40d3fb0 msgr2=0x7fd5a40b69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.096+0000 7fd5ab438640 1 --2- 172.21.15.38:0/2231522217 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a40d3fb0 0x7fd5a40b69a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd594009d90 tx=0x7fd59402a5f0 comp rx=0 tx=0).stop 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.096+0000 7fd5ab438640 1 -- 172.21.15.38:0/2231522217 shutdown_connections 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.096+0000 7fd5ab438640 1 --2- 172.21.15.38:0/2231522217 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a40d3fb0 0x7fd5a40b69a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.096+0000 7fd5ab438640 1 --2- 172.21.15.38:0/2231522217 >> v2:172.21.15.107:3300/0 conn(0x7fd5a415da90 0x7fd5a40d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:49.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.096+0000 7fd5ab438640 1 -- 172.21.15.38:0/2231522217 >> 172.21.15.38:0/2231522217 conn(0x7fd5a406bed0 msgr2=0x7fd5a406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.097+0000 7fd5ab438640 1 -- 172.21.15.38:0/2231522217 shutdown_connections 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.097+0000 7fd5ab438640 1 -- 172.21.15.38:0/2231522217 wait complete. 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.098+0000 7fd5ab438640 1 Processor -- start 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.098+0000 7fd5ab438640 1 -- start start 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.098+0000 7fd5ab438640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a40d3fb0 0x7fd5a40b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.098+0000 7fd5ab438640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a415da90 0x7fd5a4154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.098+0000 7fd5ab438640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a41545c0 0x7fd5a40b4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.098+0000 7fd5ab438640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a415dff0 con 0x7fd5a415da90 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.098+0000 7fd5ab438640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a415de70 con 0x7fd5a41545c0 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.098+0000 7fd5ab438640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40d34b0 con 0x7fd5a40d3fb0 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.098+0000 7fd5a89d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a415da90 0x7fd5a4154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a89d3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a415da90 0x7fd5a4154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52720/0 (socket says 172.21.15.38:52720) 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a89d3640 1 -- 172.21.15.38:0/393387538 learned_addr learned my addr 172.21.15.38:0/393387538 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a91d4640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a40d3fb0 0x7fd5a40b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:49.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a99d5640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a41545c0 0x7fd5a40b4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a89d3640 1 -- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a40d3fb0 msgr2=0x7fd5a40b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a89d3640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a40d3fb0 0x7fd5a40b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a89d3640 1 -- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a41545c0 msgr2=0x7fd5a40b4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a89d3640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a41545c0 0x7fd5a40b4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a89d3640 1 -- 172.21.15.38:0/393387538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a40b3d30 con 0x7fd5a415da90 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a89d3640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a415da90 0x7fd5a4154080 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd59800c780 tx=0x7fd59800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a1ffb640 1 -- 172.21.15.38:0/393387538 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598018070 con 0x7fd5a415da90 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a1ffb640 1 -- 172.21.15.38:0/393387538 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5980072d0 con 0x7fd5a415da90 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5ab438640 1 -- 172.21.15.38:0/393387538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a40b4010 con 0x7fd5a415da90 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5a1ffb640 1 -- 172.21.15.38:0/393387538 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5980047e0 con 0x7fd5a415da90 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.099+0000 7fd5ab438640 1 -- 172.21.15.38:0/393387538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a4153880 con 0x7fd5a415da90 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.100+0000 7fd5ab438640 1 -- 172.21.15.38:0/393387538 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a4068e80 con 0x7fd5a415da90 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.101+0000 7fd5a1ffb640 1 -- 172.21.15.38:0/393387538 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd59801b070 con 0x7fd5a415da90 2024-04-25T08:34:49.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.101+0000 7fd5a1ffb640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd59006b4d0 0x7fd59006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:49.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.101+0000 7fd5a1ffb640 1 -- 172.21.15.38:0/393387538 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd598089940 con 0x7fd5a415da90 2024-04-25T08:34:49.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.101+0000 7fd5a91d4640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd59006b4d0 0x7fd59006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:49.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.102+0000 7fd5a91d4640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd59006b4d0 0x7fd59006d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009950 tx=0x7fd59c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:49.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.106+0000 7fd5a1ffb640 1 -- 172.21.15.38:0/393387538 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd598058630 con 0x7fd5a415da90 2024-04-25T08:34:49.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.283+0000 7fd5ab438640 1 -- 172.21.15.38:0/393387538 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd5a4068540 con 0x7fd59006b4d0 2024-04-25T08:34:49.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.289+0000 7fd5a1ffb640 1 -- 172.21.15.38:0/393387538 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251952 (secure 0 0 0) 0x7fd5a4068540 con 0x7fd59006b4d0 2024-04-25T08:34:49.288 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:49.289 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:34:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.292+0000 7fd5ab438640 1 -- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd59006b4d0 msgr2=0x7fd59006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.292+0000 7fd5ab438640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd59006b4d0 0x7fd59006d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd59c009950 tx=0x7fd59c008040 comp rx=0 tx=0).stop 2024-04-25T08:34:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.292+0000 7fd5ab438640 1 -- 172.21.15.38:0/393387538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a415da90 msgr2=0x7fd5a4154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.292+0000 7fd5ab438640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a415da90 0x7fd5a4154080 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd59800c780 tx=0x7fd59800cc40 comp rx=0 tx=0).stop 2024-04-25T08:34:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.292+0000 7fd5ab438640 1 -- 172.21.15.38:0/393387538 shutdown_connections 2024-04-25T08:34:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.292+0000 7fd5ab438640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd5a41545c0 0x7fd5a40b4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.292+0000 7fd5ab438640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5a415da90 0x7fd5a4154080 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.292+0000 7fd5ab438640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd59006b4d0 0x7fd59006d990 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.292+0000 7fd5ab438640 1 --2- 172.21.15.38:0/393387538 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd5a40d3fb0 0x7fd5a40b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.292+0000 7fd5ab438640 1 -- 172.21.15.38:0/393387538 >> 172.21.15.38:0/393387538 conn(0x7fd5a406bed0 msgr2=0x7fd5a406c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.292+0000 7fd5ab438640 1 -- 172.21.15.38:0/393387538 shutdown_connections 2024-04-25T08:34:49.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:49.292+0000 7fd5ab438640 1 -- 172.21.15.38:0/393387538 wait complete. 2024-04-25T08:34:49.302 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-04-25T08:34:49.249968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":372256,"kb_used_data":6248,"kb_used_omap":12,"kb_used_meta":365875,"kb_avail":754602464,"statfs":{"total":773094113280,"available":772712923136,"internally_reserved":0,"allocated":6397952,"data_stored":3503252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":374656560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001468"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086305+0000","last_change":"2024-04-25T08:34:44.527955+0000","last_active":"2024-04-25T08:34:46.086305+0000","last_peered":"2024-04-25T08:34:46.086305+0000","last_clean":"2024-04-25T08:34:46.086305+0000","last_became_active":"2024-04-25T08:34:44.527662+0000","last_became_peered":"2024-04-25T08:34:44.527662+0000","last_unstale":"2024-04-25T08:34:46.086305+0000","last_undegraded":"2024-04-25T08:34:46.086305+0000","last_fullsized":"2024-04-25T08:34:46.086305+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:09.895753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529700+0000","last_change":"2024-04-25T08:34:44.528374+0000","last_active":"2024-04-25T08:34:45.529700+0000","last_peered":"2024-04-25T08:34:45.529700+0000","last_clean":"2024-04-25T08:34:45.529700+0000","last_became_active":"2024-04-25T08:34:44.528107+0000","last_became_peered":"2024-04-25T08:34:44.528107+0000","last_unstale":"2024-04-25T08:34:45.529700+0000","last_undegraded":"2024-04-25T08:34:45.529700+0000","last_fullsized":"2024-04-25T08:34:45.529700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:18.254832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093741+0000","last_change":"2024-04-25T08:34:44.534121+0000","last_active":"2024-04-25T08:34:46.093741+0000","last_peered":"2024-04-25T08:34:46.093741+0000","last_clean":"2024-04-25T08:34:46.093741+0000","last_became_active":"2024-04-25T08:34:44.533765+0000","last_became_peered":"2024-04-25T08:34:44.533765+0000","last_unstale":"2024-04-25T08:34:46.093741+0000","last_undegraded":"2024-04-25T08:34:46.093741+0000","last_fullsized":"2024-04-25T08:34:46.093741+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.333087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529869+0000","last_change":"2024-04-25T08:34:44.535500+0000","last_active":"2024-04-25T08:34:45.529869+0000","last_peered":"2024-04-25T08:34:45.529869+0000","last_clean":"2024-04-25T08:34:45.529869+0000","last_became_active":"2024-04-25T08:34:44.535220+0000","last_became_peered":"2024-04-25T08:34:44.535220+0000","last_unstale":"2024-04-25T08:34:45.529869+0000","last_undegraded":"2024-04-25T08:34:45.529869+0000","last_fullsized":"2024-04-25T08:34:45.529869+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:43.198107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531042+0000","last_change":"2024-04-25T08:34:44.520910+0000","last_active":"2024-04-25T08:34:45.531042+0000","last_peered":"2024-04-25T08:34:45.531042+0000","last_clean":"2024-04-25T08:34:45.531042+0000","last_became_active":"2024-04-25T08:34:44.520607+0000","last_became_peered":"2024-04-25T08:34:44.520607+0000","last_unstale":"2024-04-25T08:34:45.531042+0000","last_undegraded":"2024-04-25T08:34:45.531042+0000","last_fullsized":"2024-04-25T08:34:45.531042+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:43.933652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086256+0000","last_change":"2024-04-25T08:34:44.527358+0000","last_active":"2024-04-25T08:34:46.086256+0000","last_peered":"2024-04-25T08:34:46.086256+0000","last_clean":"2024-04-25T08:34:46.086256+0000","last_became_active":"2024-04-25T08:34:44.527098+0000","last_became_peered":"2024-04-25T08:34:44.527098+0000","last_unstale":"2024-04-25T08:34:46.086256+0000","last_undegraded":"2024-04-25T08:34:46.086256+0000","last_fullsized":"2024-04-25T08:34:46.086256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:47.550727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094066+0000","last_change":"2024-04-25T08:34:44.536974+0000","last_active":"2024-04-25T08:34:46.094066+0000","last_peered":"2024-04-25T08:34:46.094066+0000","last_clean":"2024-04-25T08:34:46.094066+0000","last_became_active":"2024-04-25T08:34:44.536690+0000","last_became_peered":"2024-04-25T08:34:44.536690+0000","last_unstale":"2024-04-25T08:34:46.094066+0000","last_undegraded":"2024-04-25T08:34:46.094066+0000","last_fullsized":"2024-04-25T08:34:46.094066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:27.300505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094429+0000","last_change":"2024-04-25T08:34:44.530573+0000","last_active":"2024-04-25T08:34:46.094429+0000","last_peered":"2024-04-25T08:34:46.094429+0000","last_clean":"2024-04-25T08:34:46.094429+0000","last_became_active":"2024-04-25T08:34:44.530307+0000","last_became_peered":"2024-04-25T08:34:44.530307+0000","last_unstale":"2024-04-25T08:34:46.094429+0000","last_undegraded":"2024-04-25T08:34:46.094429+0000","last_fullsized":"2024-04-25T08:34:46.094429+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:31.481085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094299+0000","last_change":"2024-04-25T08:34:44.537554+0000","last_active":"2024-04-25T08:34:46.094299+0000","last_peered":"2024-04-25T08:34:46.094299+0000","last_clean":"2024-04-25T08:34:46.094299+0000","last_became_active":"2024-04-25T08:34:44.537285+0000","last_became_peered":"2024-04-25T08:34:44.537285+0000","last_unstale":"2024-04-25T08:34:46.094299+0000","last_undegraded":"2024-04-25T08:34:46.094299+0000","last_fullsized":"2024-04-25T08:34:46.094299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:25.030847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086137+0000","last_change":"2024-04-25T08:34:44.533732+0000","last_active":"2024-04-25T08:34:46.086137+0000","last_peered":"2024-04-25T08:34:46.086137+0000","last_clean":"2024-04-25T08:34:46.086137+0000","last_became_active":"2024-04-25T08:34:44.533441+0000","last_became_peered":"2024-04-25T08:34:44.533441+0000","last_unstale":"2024-04-25T08:34:46.086137+0000","last_undegraded":"2024-04-25T08:34:46.086137+0000","last_fullsized":"2024-04-25T08:34:46.086137+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:11.733375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093123+0000","last_change":"2024-04-25T08:34:44.530709+0000","last_active":"2024-04-25T08:34:46.093123+0000","last_peered":"2024-04-25T08:34:46.093123+0000","last_clean":"2024-04-25T08:34:46.093123+0000","last_became_active":"2024-04-25T08:34:44.530416+0000","last_became_peered":"2024-04-25T08:34:44.530416+0000","last_unstale":"2024-04-25T08:34:46.093123+0000","last_undegraded":"2024-04-25T08:34:46.093123+0000","last_fullsized":"2024-04-25T08:34:46.093123+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:32.618322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093910+0000","last_change":"2024-04-25T08:34:44.529095+0000","last_active":"2024-04-25T08:34:46.093910+0000","last_peered":"2024-04-25T08:34:46.093910+0000","last_clean":"2024-04-25T08:34:46.093910+0000","last_became_active":"2024-04-25T08:34:44.528821+0000","last_became_peered":"2024-04-25T08:34:44.528821+0000","last_unstale":"2024-04-25T08:34:46.093910+0000","last_undegraded":"2024-04-25T08:34:46.093910+0000","last_fullsized":"2024-04-25T08:34:46.093910+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:18.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086171+0000","last_change":"2024-04-25T08:34:44.531394+0000","last_active":"2024-04-25T08:34:46.086171+0000","last_peered":"2024-04-25T08:34:46.086171+0000","last_clean":"2024-04-25T08:34:46.086171+0000","last_became_active":"2024-04-25T08:34:44.531057+0000","last_became_peered":"2024-04-25T08:34:44.531057+0000","last_unstale":"2024-04-25T08:34:46.086171+0000","last_undegraded":"2024-04-25T08:34:46.086171+0000","last_fullsized":"2024-04-25T08:34:46.086171+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:18.730106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094348+0000","last_change":"2024-04-25T08:34:44.533220+0000","last_active":"2024-04-25T08:34:46.094348+0000","last_peered":"2024-04-25T08:34:46.094348+0000","last_clean":"2024-04-25T08:34:46.094348+0000","last_became_active":"2024-04-25T08:34:44.532647+0000","last_became_peered":"2024-04-25T08:34:44.532647+0000","last_unstale":"2024-04-25T08:34:46.094348+0000","last_undegraded":"2024-04-25T08:34:46.094348+0000","last_fullsized":"2024-04-25T08:34:46.094348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:00.701022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529308+0000","last_change":"2024-04-25T08:34:44.521532+0000","last_active":"2024-04-25T08:34:45.529308+0000","last_peered":"2024-04-25T08:34:45.529308+0000","last_clean":"2024-04-25T08:34:45.529308+0000","last_became_active":"2024-04-25T08:34:44.521274+0000","last_became_peered":"2024-04-25T08:34:44.521274+0000","last_unstale":"2024-04-25T08:34:45.529308+0000","last_undegraded":"2024-04-25T08:34:45.529308+0000","last_fullsized":"2024-04-25T08:34:45.529308+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.919973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086067+0000","last_change":"2024-04-25T08:34:44.525673+0000","last_active":"2024-04-25T08:34:46.086067+0000","last_peered":"2024-04-25T08:34:46.086067+0000","last_clean":"2024-04-25T08:34:46.086067+0000","last_became_active":"2024-04-25T08:34:44.525384+0000","last_became_peered":"2024-04-25T08:34:44.525384+0000","last_unstale":"2024-04-25T08:34:46.086067+0000","last_undegraded":"2024-04-25T08:34:46.086067+0000","last_fullsized":"2024-04-25T08:34:46.086067+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.499854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095168+0000","last_change":"2024-04-25T08:34:44.533259+0000","last_active":"2024-04-25T08:34:46.095168+0000","last_peered":"2024-04-25T08:34:46.095168+0000","last_clean":"2024-04-25T08:34:46.095168+0000","last_became_active":"2024-04-25T08:34:44.531589+0000","last_became_peered":"2024-04-25T08:34:44.531589+0000","last_unstale":"2024-04-25T08:34:46.095168+0000","last_undegraded":"2024-04-25T08:34:46.095168+0000","last_fullsized":"2024-04-25T08:34:46.095168+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:01.549992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529955+0000","last_change":"2024-04-25T08:34:44.527725+0000","last_active":"2024-04-25T08:34:45.529955+0000","last_peered":"2024-04-25T08:34:45.529955+0000","last_clean":"2024-04-25T08:34:45.529955+0000","last_became_active":"2024-04-25T08:34:44.526928+0000","last_became_peered":"2024-04-25T08:34:44.526928+0000","last_unstale":"2024-04-25T08:34:45.529955+0000","last_undegraded":"2024-04-25T08:34:45.529955+0000","last_fullsized":"2024-04-25T08:34:45.529955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.333087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093885+0000","last_change":"2024-04-25T08:34:44.530163+0000","last_active":"2024-04-25T08:34:46.093885+0000","last_peered":"2024-04-25T08:34:46.093885+0000","last_clean":"2024-04-25T08:34:46.093885+0000","last_became_active":"2024-04-25T08:34:44.529905+0000","last_became_peered":"2024-04-25T08:34:44.529905+0000","last_unstale":"2024-04-25T08:34:46.093885+0000","last_undegraded":"2024-04-25T08:34:46.093885+0000","last_fullsized":"2024-04-25T08:34:46.093885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:18.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529970+0000","last_change":"2024-04-25T08:34:44.520405+0000","last_active":"2024-04-25T08:34:45.529970+0000","last_peered":"2024-04-25T08:34:45.529970+0000","last_clean":"2024-04-25T08:34:45.529970+0000","last_became_active":"2024-04-25T08:34:44.520122+0000","last_became_peered":"2024-04-25T08:34:44.520122+0000","last_unstale":"2024-04-25T08:34:45.529970+0000","last_undegraded":"2024-04-25T08:34:45.529970+0000","last_fullsized":"2024-04-25T08:34:45.529970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:09.811944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530067+0000","last_change":"2024-04-25T08:34:44.522344+0000","last_active":"2024-04-25T08:34:45.530067+0000","last_peered":"2024-04-25T08:34:45.530067+0000","last_clean":"2024-04-25T08:34:45.530067+0000","last_became_active":"2024-04-25T08:34:44.522063+0000","last_became_peered":"2024-04-25T08:34:44.522063+0000","last_unstale":"2024-04-25T08:34:45.530067+0000","last_undegraded":"2024-04-25T08:34:45.530067+0000","last_fullsized":"2024-04-25T08:34:45.530067+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:25.590051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093745+0000","last_change":"2024-04-25T08:34:44.530629+0000","last_active":"2024-04-25T08:34:46.093745+0000","last_peered":"2024-04-25T08:34:46.093745+0000","last_clean":"2024-04-25T08:34:46.093745+0000","last_became_active":"2024-04-25T08:34:44.530358+0000","last_became_peered":"2024-04-25T08:34:44.530358+0000","last_unstale":"2024-04-25T08:34:46.093745+0000","last_undegraded":"2024-04-25T08:34:46.093745+0000","last_fullsized":"2024-04-25T08:34:46.093745+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.333087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093078+0000","last_change":"2024-04-25T08:34:44.529987+0000","last_active":"2024-04-25T08:34:46.093078+0000","last_peered":"2024-04-25T08:34:46.093078+0000","last_clean":"2024-04-25T08:34:46.093078+0000","last_became_active":"2024-04-25T08:34:44.529724+0000","last_became_peered":"2024-04-25T08:34:44.529724+0000","last_unstale":"2024-04-25T08:34:46.093078+0000","last_undegraded":"2024-04-25T08:34:46.093078+0000","last_fullsized":"2024-04-25T08:34:46.093078+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:11.876226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093628+0000","last_change":"2024-04-25T08:34:44.536670+0000","last_active":"2024-04-25T08:34:46.093628+0000","last_peered":"2024-04-25T08:34:46.093628+0000","last_clean":"2024-04-25T08:34:46.093628+0000","last_became_active":"2024-04-25T08:34:44.536110+0000","last_became_peered":"2024-04-25T08:34:44.536110+0000","last_unstale":"2024-04-25T08:34:46.093628+0000","last_undegraded":"2024-04-25T08:34:46.093628+0000","last_fullsized":"2024-04-25T08:34:46.093628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:43.198107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095058+0000","last_change":"2024-04-25T08:34:44.533241+0000","last_active":"2024-04-25T08:34:46.095058+0000","last_peered":"2024-04-25T08:34:46.095058+0000","last_clean":"2024-04-25T08:34:46.095058+0000","last_became_active":"2024-04-25T08:34:44.531910+0000","last_became_peered":"2024-04-25T08:34:44.531910+0000","last_unstale":"2024-04-25T08:34:46.095058+0000","last_undegraded":"2024-04-25T08:34:46.095058+0000","last_fullsized":"2024-04-25T08:34:46.095058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.963128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093922+0000","last_change":"2024-04-25T08:34:44.535827+0000","last_active":"2024-04-25T08:34:46.093922+0000","last_peered":"2024-04-25T08:34:46.093922+0000","last_clean":"2024-04-25T08:34:46.093922+0000","last_became_active":"2024-04-25T08:34:44.535273+0000","last_became_peered":"2024-04-25T08:34:44.535273+0000","last_unstale":"2024-04-25T08:34:46.093922+0000","last_undegraded":"2024-04-25T08:34:46.093922+0000","last_fullsized":"2024-04-25T08:34:46.093922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:18.966389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086493+0000","last_change":"2024-04-25T08:34:44.538453+0000","last_active":"2024-04-25T08:34:46.086493+0000","last_peered":"2024-04-25T08:34:46.086493+0000","last_clean":"2024-04-25T08:34:46.086493+0000","last_became_active":"2024-04-25T08:34:44.538165+0000","last_became_peered":"2024-04-25T08:34:44.538165+0000","last_unstale":"2024-04-25T08:34:46.086493+0000","last_undegraded":"2024-04-25T08:34:46.086493+0000","last_fullsized":"2024-04-25T08:34:46.086493+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:46.558329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530135+0000","last_change":"2024-04-25T08:34:44.528517+0000","last_active":"2024-04-25T08:34:45.530135+0000","last_peered":"2024-04-25T08:34:45.530135+0000","last_clean":"2024-04-25T08:34:45.530135+0000","last_became_active":"2024-04-25T08:34:44.528229+0000","last_became_peered":"2024-04-25T08:34:44.528229+0000","last_unstale":"2024-04-25T08:34:45.530135+0000","last_undegraded":"2024-04-25T08:34:45.530135+0000","last_fullsized":"2024-04-25T08:34:45.530135+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:41.585740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094613+0000","last_change":"2024-04-25T08:34:44.528023+0000","last_active":"2024-04-25T08:34:46.094613+0000","last_peered":"2024-04-25T08:34:46.094613+0000","last_clean":"2024-04-25T08:34:46.094613+0000","last_became_active":"2024-04-25T08:34:44.527668+0000","last_became_peered":"2024-04-25T08:34:44.527668+0000","last_unstale":"2024-04-25T08:34:46.094613+0000","last_undegraded":"2024-04-25T08:34:46.094613+0000","last_fullsized":"2024-04-25T08:34:46.094613+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:06.085151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529886+0000","last_change":"2024-04-25T08:34:44.527591+0000","last_active":"2024-04-25T08:34:45.529886+0000","last_peered":"2024-04-25T08:34:45.529886+0000","last_clean":"2024-04-25T08:34:45.529886+0000","last_became_active":"2024-04-25T08:34:44.527285+0000","last_became_peered":"2024-04-25T08:34:44.527285+0000","last_unstale":"2024-04-25T08:34:45.529886+0000","last_undegraded":"2024-04-25T08:34:45.529886+0000","last_fullsized":"2024-04-25T08:34:45.529886+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:05.252545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094200+0000","last_change":"2024-04-25T08:34:44.533947+0000","last_active":"2024-04-25T08:34:46.094200+0000","last_peered":"2024-04-25T08:34:46.094200+0000","last_clean":"2024-04-25T08:34:46.094200+0000","last_became_active":"2024-04-25T08:34:44.533609+0000","last_became_peered":"2024-04-25T08:34:44.533609+0000","last_unstale":"2024-04-25T08:34:46.094200+0000","last_undegraded":"2024-04-25T08:34:46.094200+0000","last_fullsized":"2024-04-25T08:34:46.094200+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:44.889254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093558+0000","last_change":"2024-04-25T08:34:44.532723+0000","last_active":"2024-04-25T08:34:46.093558+0000","last_peered":"2024-04-25T08:34:46.093558+0000","last_clean":"2024-04-25T08:34:46.093558+0000","last_became_active":"2024-04-25T08:34:44.531850+0000","last_became_peered":"2024-04-25T08:34:44.531850+0000","last_unstale":"2024-04-25T08:34:46.093558+0000","last_undegraded":"2024-04-25T08:34:46.093558+0000","last_fullsized":"2024-04-25T08:34:46.093558+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:20.979454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529253+0000","last_change":"2024-04-25T08:34:44.527874+0000","last_active":"2024-04-25T08:34:45.529253+0000","last_peered":"2024-04-25T08:34:45.529253+0000","last_clean":"2024-04-25T08:34:45.529253+0000","last_became_active":"2024-04-25T08:34:44.527594+0000","last_became_peered":"2024-04-25T08:34:44.527594+0000","last_unstale":"2024-04-25T08:34:45.529253+0000","last_undegraded":"2024-04-25T08:34:45.529253+0000","last_fullsized":"2024-04-25T08:34:45.529253+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:05.622445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529837+0000","last_change":"2024-04-25T08:34:43.873145+0000","last_active":"2024-04-25T08:34:45.529837+0000","last_peered":"2024-04-25T08:34:45.529837+0000","last_clean":"2024-04-25T08:34:45.529837+0000","last_became_active":"2024-04-25T08:34:43.872860+0000","last_became_peered":"2024-04-25T08:34:43.872860+0000","last_unstale":"2024-04-25T08:34:45.529837+0000","last_undegraded":"2024-04-25T08:34:45.529837+0000","last_fullsized":"2024-04-25T08:34:45.529837+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:39.042852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093598+0000","last_change":"2024-04-25T08:34:44.530047+0000","last_active":"2024-04-25T08:34:46.093598+0000","last_peered":"2024-04-25T08:34:46.093598+0000","last_clean":"2024-04-25T08:34:46.093598+0000","last_became_active":"2024-04-25T08:34:44.529781+0000","last_became_peered":"2024-04-25T08:34:44.529781+0000","last_unstale":"2024-04-25T08:34:46.093598+0000","last_undegraded":"2024-04-25T08:34:46.093598+0000","last_fullsized":"2024-04-25T08:34:46.093598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:36.970964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086257+0000","last_change":"2024-04-25T08:34:43.884678+0000","last_active":"2024-04-25T08:34:46.086257+0000","last_peered":"2024-04-25T08:34:46.086257+0000","last_clean":"2024-04-25T08:34:46.086257+0000","last_became_active":"2024-04-25T08:34:43.884357+0000","last_became_peered":"2024-04-25T08:34:43.884357+0000","last_unstale":"2024-04-25T08:34:46.086257+0000","last_undegraded":"2024-04-25T08:34:46.086257+0000","last_fullsized":"2024-04-25T08:34:46.086257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:04.802285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530601+0000","last_change":"2024-04-25T08:34:44.519962+0000","last_active":"2024-04-25T08:34:45.530601+0000","last_peered":"2024-04-25T08:34:45.530601+0000","last_clean":"2024-04-25T08:34:45.530601+0000","last_became_active":"2024-04-25T08:34:44.519671+0000","last_became_peered":"2024-04-25T08:34:44.519671+0000","last_unstale":"2024-04-25T08:34:45.530601+0000","last_undegraded":"2024-04-25T08:34:45.530601+0000","last_fullsized":"2024-04-25T08:34:45.530601+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:49.239506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086556+0000","last_change":"2024-04-25T08:34:43.878220+0000","last_active":"2024-04-25T08:34:46.086556+0000","last_peered":"2024-04-25T08:34:46.086556+0000","last_clean":"2024-04-25T08:34:46.086556+0000","last_became_active":"2024-04-25T08:34:43.877915+0000","last_became_peered":"2024-04-25T08:34:43.877915+0000","last_unstale":"2024-04-25T08:34:46.086556+0000","last_undegraded":"2024-04-25T08:34:46.086556+0000","last_fullsized":"2024-04-25T08:34:46.086556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:55.485664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094296+0000","last_change":"2024-04-25T08:34:44.535169+0000","last_active":"2024-04-25T08:34:46.094296+0000","last_peered":"2024-04-25T08:34:46.094296+0000","last_clean":"2024-04-25T08:34:46.094296+0000","last_became_active":"2024-04-25T08:34:44.534858+0000","last_became_peered":"2024-04-25T08:34:44.534858+0000","last_unstale":"2024-04-25T08:34:46.094296+0000","last_undegraded":"2024-04-25T08:34:46.094296+0000","last_fullsized":"2024-04-25T08:34:46.094296+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:00.701022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093702+0000","last_change":"2024-04-25T08:34:43.882922+0000","last_active":"2024-04-25T08:34:46.093702+0000","last_peered":"2024-04-25T08:34:46.093702+0000","last_clean":"2024-04-25T08:34:46.093702+0000","last_became_active":"2024-04-25T08:34:43.882609+0000","last_became_peered":"2024-04-25T08:34:43.882609+0000","last_unstale":"2024-04-25T08:34:46.093702+0000","last_undegraded":"2024-04-25T08:34:46.093702+0000","last_fullsized":"2024-04-25T08:34:46.093702+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:17.727985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531083+0000","last_change":"2024-04-25T08:34:44.532481+0000","last_active":"2024-04-25T08:34:45.531083+0000","last_peered":"2024-04-25T08:34:45.531083+0000","last_clean":"2024-04-25T08:34:45.531083+0000","last_became_active":"2024-04-25T08:34:44.532204+0000","last_became_peered":"2024-04-25T08:34:44.532204+0000","last_unstale":"2024-04-25T08:34:45.531083+0000","last_undegraded":"2024-04-25T08:34:45.531083+0000","last_fullsized":"2024-04-25T08:34:45.531083+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:19.590018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094711+0000","last_change":"2024-04-25T08:34:43.883536+0000","last_active":"2024-04-25T08:34:46.094711+0000","last_peered":"2024-04-25T08:34:46.094711+0000","last_clean":"2024-04-25T08:34:46.094711+0000","last_became_active":"2024-04-25T08:34:43.883232+0000","last_became_peered":"2024-04-25T08:34:43.883232+0000","last_unstale":"2024-04-25T08:34:46.094711+0000","last_undegraded":"2024-04-25T08:34:46.094711+0000","last_fullsized":"2024-04-25T08:34:46.094711+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:10.731254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530119+0000","last_change":"2024-04-25T08:34:44.529487+0000","last_active":"2024-04-25T08:34:45.530119+0000","last_peered":"2024-04-25T08:34:45.530119+0000","last_clean":"2024-04-25T08:34:45.530119+0000","last_became_active":"2024-04-25T08:34:44.529217+0000","last_became_peered":"2024-04-25T08:34:44.529217+0000","last_unstale":"2024-04-25T08:34:45.530119+0000","last_undegraded":"2024-04-25T08:34:45.530119+0000","last_fullsized":"2024-04-25T08:34:45.530119+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:18.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531305+0000","last_change":"2024-04-25T08:34:43.875476+0000","last_active":"2024-04-25T08:34:45.531305+0000","last_peered":"2024-04-25T08:34:45.531305+0000","last_clean":"2024-04-25T08:34:45.531305+0000","last_became_active":"2024-04-25T08:34:43.874920+0000","last_became_peered":"2024-04-25T08:34:43.874920+0000","last_unstale":"2024-04-25T08:34:45.531305+0000","last_undegraded":"2024-04-25T08:34:45.531305+0000","last_fullsized":"2024-04-25T08:34:45.531305+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:54.880133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094086+0000","last_change":"2024-04-25T08:34:44.536131+0000","last_active":"2024-04-25T08:34:46.094086+0000","last_peered":"2024-04-25T08:34:46.094086+0000","last_clean":"2024-04-25T08:34:46.094086+0000","last_became_active":"2024-04-25T08:34:44.535860+0000","last_became_peered":"2024-04-25T08:34:44.535860+0000","last_unstale":"2024-04-25T08:34:46.094086+0000","last_undegraded":"2024-04-25T08:34:46.094086+0000","last_fullsized":"2024-04-25T08:34:46.094086+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:44.889254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530427+0000","last_change":"2024-04-25T08:34:43.869200+0000","last_active":"2024-04-25T08:34:45.530427+0000","last_peered":"2024-04-25T08:34:45.530427+0000","last_clean":"2024-04-25T08:34:45.530427+0000","last_became_active":"2024-04-25T08:34:43.868895+0000","last_became_peered":"2024-04-25T08:34:43.868895+0000","last_unstale":"2024-04-25T08:34:45.530427+0000","last_undegraded":"2024-04-25T08:34:45.530427+0000","last_fullsized":"2024-04-25T08:34:45.530427+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:36.714740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529600+0000","last_change":"2024-04-25T08:34:44.528869+0000","last_active":"2024-04-25T08:34:45.529600+0000","last_peered":"2024-04-25T08:34:45.529600+0000","last_clean":"2024-04-25T08:34:45.529600+0000","last_became_active":"2024-04-25T08:34:44.528619+0000","last_became_peered":"2024-04-25T08:34:44.528619+0000","last_unstale":"2024-04-25T08:34:45.529600+0000","last_undegraded":"2024-04-25T08:34:45.529600+0000","last_fullsized":"2024-04-25T08:34:45.529600+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:36.250986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093548+0000","last_change":"2024-04-25T08:34:43.881571+0000","last_active":"2024-04-25T08:34:46.093548+0000","last_peered":"2024-04-25T08:34:46.093548+0000","last_clean":"2024-04-25T08:34:46.093548+0000","last_became_active":"2024-04-25T08:34:43.881272+0000","last_became_peered":"2024-04-25T08:34:43.881272+0000","last_unstale":"2024-04-25T08:34:46.093548+0000","last_undegraded":"2024-04-25T08:34:46.093548+0000","last_fullsized":"2024-04-25T08:34:46.093548+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:52.868065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530959+0000","last_change":"2024-04-25T08:34:44.531396+0000","last_active":"2024-04-25T08:34:45.530959+0000","last_peered":"2024-04-25T08:34:45.530959+0000","last_clean":"2024-04-25T08:34:45.530959+0000","last_became_active":"2024-04-25T08:34:44.531134+0000","last_became_peered":"2024-04-25T08:34:44.531134+0000","last_unstale":"2024-04-25T08:34:45.530959+0000","last_undegraded":"2024-04-25T08:34:45.530959+0000","last_fullsized":"2024-04-25T08:34:45.530959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:12.856976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529630+0000","last_change":"2024-04-25T08:34:43.874727+0000","last_active":"2024-04-25T08:34:45.529630+0000","last_peered":"2024-04-25T08:34:45.529630+0000","last_clean":"2024-04-25T08:34:45.529630+0000","last_became_active":"2024-04-25T08:34:43.874477+0000","last_became_peered":"2024-04-25T08:34:43.874477+0000","last_unstale":"2024-04-25T08:34:45.529630+0000","last_undegraded":"2024-04-25T08:34:45.529630+0000","last_fullsized":"2024-04-25T08:34:45.529630+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:12.207929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093479+0000","last_change":"2024-04-25T08:34:44.525749+0000","last_active":"2024-04-25T08:34:46.093479+0000","last_peered":"2024-04-25T08:34:46.093479+0000","last_clean":"2024-04-25T08:34:46.093479+0000","last_became_active":"2024-04-25T08:34:44.525470+0000","last_became_peered":"2024-04-25T08:34:44.525470+0000","last_unstale":"2024-04-25T08:34:46.093479+0000","last_undegraded":"2024-04-25T08:34:46.093479+0000","last_fullsized":"2024-04-25T08:34:46.093479+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:40.781389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530997+0000","last_change":"2024-04-25T08:34:43.876071+0000","last_active":"2024-04-25T08:34:45.530997+0000","last_peered":"2024-04-25T08:34:45.530997+0000","last_clean":"2024-04-25T08:34:45.530997+0000","last_became_active":"2024-04-25T08:34:43.875782+0000","last_became_peered":"2024-04-25T08:34:43.875782+0000","last_unstale":"2024-04-25T08:34:45.530997+0000","last_undegraded":"2024-04-25T08:34:45.530997+0000","last_fullsized":"2024-04-25T08:34:45.530997+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:39.042852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094436+0000","last_change":"2024-04-25T08:34:44.528331+0000","last_active":"2024-04-25T08:34:46.094436+0000","last_peered":"2024-04-25T08:34:46.094436+0000","last_clean":"2024-04-25T08:34:46.094436+0000","last_became_active":"2024-04-25T08:34:44.528080+0000","last_became_peered":"2024-04-25T08:34:44.528080+0000","last_unstale":"2024-04-25T08:34:46.094436+0000","last_undegraded":"2024-04-25T08:34:46.094436+0000","last_fullsized":"2024-04-25T08:34:46.094436+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:25.030847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530237+0000","last_change":"2024-04-25T08:34:43.877334+0000","last_active":"2024-04-25T08:34:45.530237+0000","last_peered":"2024-04-25T08:34:45.530237+0000","last_clean":"2024-04-25T08:34:45.530237+0000","last_became_active":"2024-04-25T08:34:43.877084+0000","last_became_peered":"2024-04-25T08:34:43.877084+0000","last_unstale":"2024-04-25T08:34:45.530237+0000","last_undegraded":"2024-04-25T08:34:45.530237+0000","last_fullsized":"2024-04-25T08:34:45.530237+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:46.548606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531206+0000","last_change":"2024-04-25T08:34:44.529717+0000","last_active":"2024-04-25T08:34:45.531206+0000","last_peered":"2024-04-25T08:34:45.531206+0000","last_clean":"2024-04-25T08:34:45.531206+0000","last_became_active":"2024-04-25T08:34:44.529442+0000","last_became_peered":"2024-04-25T08:34:44.529442+0000","last_unstale":"2024-04-25T08:34:45.531206+0000","last_undegraded":"2024-04-25T08:34:45.531206+0000","last_fullsized":"2024-04-25T08:34:45.531206+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:57.569222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094077+0000","last_change":"2024-04-25T08:34:43.887845+0000","last_active":"2024-04-25T08:34:46.094077+0000","last_peered":"2024-04-25T08:34:46.094077+0000","last_clean":"2024-04-25T08:34:46.094077+0000","last_became_active":"2024-04-25T08:34:43.887544+0000","last_became_peered":"2024-04-25T08:34:43.887544+0000","last_unstale":"2024-04-25T08:34:46.094077+0000","last_undegraded":"2024-04-25T08:34:46.094077+0000","last_fullsized":"2024-04-25T08:34:46.094077+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:37.118342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093571+0000","last_change":"2024-04-25T08:34:44.532534+0000","last_active":"2024-04-25T08:34:46.093571+0000","last_peered":"2024-04-25T08:34:46.093571+0000","last_clean":"2024-04-25T08:34:46.093571+0000","last_became_active":"2024-04-25T08:34:44.532250+0000","last_became_peered":"2024-04-25T08:34:44.532250+0000","last_unstale":"2024-04-25T08:34:46.093571+0000","last_undegraded":"2024-04-25T08:34:46.093571+0000","last_fullsized":"2024-04-25T08:34:46.093571+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:11.760981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094802+0000","last_change":"2024-04-25T08:34:43.883681+0000","last_active":"2024-04-25T08:34:46.094802+0000","last_peered":"2024-04-25T08:34:46.094802+0000","last_clean":"2024-04-25T08:34:46.094802+0000","last_became_active":"2024-04-25T08:34:43.883385+0000","last_became_peered":"2024-04-25T08:34:43.883385+0000","last_unstale":"2024-04-25T08:34:46.094802+0000","last_undegraded":"2024-04-25T08:34:46.094802+0000","last_fullsized":"2024-04-25T08:34:46.094802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:17.727985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094016+0000","last_change":"2024-04-25T08:34:44.532945+0000","last_active":"2024-04-25T08:34:46.094016+0000","last_peered":"2024-04-25T08:34:46.094016+0000","last_clean":"2024-04-25T08:34:46.094016+0000","last_became_active":"2024-04-25T08:34:44.532563+0000","last_became_peered":"2024-04-25T08:34:44.532563+0000","last_unstale":"2024-04-25T08:34:46.094016+0000","last_undegraded":"2024-04-25T08:34:46.094016+0000","last_fullsized":"2024-04-25T08:34:46.094016+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:18.966389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530710+0000","last_change":"2024-04-25T08:34:43.882768+0000","last_active":"2024-04-25T08:34:45.530710+0000","last_peered":"2024-04-25T08:34:45.530710+0000","last_clean":"2024-04-25T08:34:45.530710+0000","last_became_active":"2024-04-25T08:34:43.882459+0000","last_became_peered":"2024-04-25T08:34:43.882459+0000","last_unstale":"2024-04-25T08:34:45.530710+0000","last_undegraded":"2024-04-25T08:34:45.530710+0000","last_fullsized":"2024-04-25T08:34:45.530710+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:46.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094076+0000","last_change":"2024-04-25T08:34:44.534966+0000","last_active":"2024-04-25T08:34:46.094076+0000","last_peered":"2024-04-25T08:34:46.094076+0000","last_clean":"2024-04-25T08:34:46.094076+0000","last_became_active":"2024-04-25T08:34:44.534697+0000","last_became_peered":"2024-04-25T08:34:44.534697+0000","last_unstale":"2024-04-25T08:34:46.094076+0000","last_undegraded":"2024-04-25T08:34:46.094076+0000","last_fullsized":"2024-04-25T08:34:46.094076+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:27.300505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529417+0000","last_change":"2024-04-25T08:34:43.873249+0000","last_active":"2024-04-25T08:34:45.529417+0000","last_peered":"2024-04-25T08:34:45.529417+0000","last_clean":"2024-04-25T08:34:45.529417+0000","last_became_active":"2024-04-25T08:34:43.872980+0000","last_became_peered":"2024-04-25T08:34:43.872980+0000","last_unstale":"2024-04-25T08:34:45.529417+0000","last_undegraded":"2024-04-25T08:34:45.529417+0000","last_fullsized":"2024-04-25T08:34:45.529417+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:02.862950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094908+0000","last_change":"2024-04-25T08:34:44.533491+0000","last_active":"2024-04-25T08:34:46.094908+0000","last_peered":"2024-04-25T08:34:46.094908+0000","last_clean":"2024-04-25T08:34:46.094908+0000","last_became_active":"2024-04-25T08:34:44.532257+0000","last_became_peered":"2024-04-25T08:34:44.532257+0000","last_unstale":"2024-04-25T08:34:46.094908+0000","last_undegraded":"2024-04-25T08:34:46.094908+0000","last_fullsized":"2024-04-25T08:34:46.094908+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:41.495853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094668+0000","last_change":"2024-04-25T08:34:43.886395+0000","last_active":"2024-04-25T08:34:46.094668+0000","last_peered":"2024-04-25T08:34:46.094668+0000","last_clean":"2024-04-25T08:34:46.094668+0000","last_became_active":"2024-04-25T08:34:43.886111+0000","last_became_peered":"2024-04-25T08:34:43.886111+0000","last_unstale":"2024-04-25T08:34:46.094668+0000","last_undegraded":"2024-04-25T08:34:46.094668+0000","last_fullsized":"2024-04-25T08:34:46.094668+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:18.627329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094572+0000","last_change":"2024-04-25T08:34:44.530937+0000","last_active":"2024-04-25T08:34:46.094572+0000","last_peered":"2024-04-25T08:34:46.094572+0000","last_clean":"2024-04-25T08:34:46.094572+0000","last_became_active":"2024-04-25T08:34:44.529251+0000","last_became_peered":"2024-04-25T08:34:44.529251+0000","last_unstale":"2024-04-25T08:34:46.094572+0000","last_undegraded":"2024-04-25T08:34:46.094572+0000","last_fullsized":"2024-04-25T08:34:46.094572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:41.495853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095013+0000","last_change":"2024-04-25T08:34:43.889631+0000","last_active":"2024-04-25T08:34:46.095013+0000","last_peered":"2024-04-25T08:34:46.095013+0000","last_clean":"2024-04-25T08:34:46.095013+0000","last_became_active":"2024-04-25T08:34:43.889354+0000","last_became_peered":"2024-04-25T08:34:43.889354+0000","last_unstale":"2024-04-25T08:34:46.095013+0000","last_undegraded":"2024-04-25T08:34:46.095013+0000","last_fullsized":"2024-04-25T08:34:46.095013+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:02.862950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086134+0000","last_change":"2024-04-25T08:34:44.533026+0000","last_active":"2024-04-25T08:34:46.086134+0000","last_peered":"2024-04-25T08:34:46.086134+0000","last_clean":"2024-04-25T08:34:46.086134+0000","last_became_active":"2024-04-25T08:34:44.532759+0000","last_became_peered":"2024-04-25T08:34:44.532759+0000","last_unstale":"2024-04-25T08:34:46.086134+0000","last_undegraded":"2024-04-25T08:34:46.086134+0000","last_fullsized":"2024-04-25T08:34:46.086134+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:53.870186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093300+0000","last_change":"2024-04-25T08:34:43.874603+0000","last_active":"2024-04-25T08:34:46.093300+0000","last_peered":"2024-04-25T08:34:46.093300+0000","last_clean":"2024-04-25T08:34:46.093300+0000","last_became_active":"2024-04-25T08:34:43.874330+0000","last_became_peered":"2024-04-25T08:34:43.874330+0000","last_unstale":"2024-04-25T08:34:46.093300+0000","last_undegraded":"2024-04-25T08:34:46.093300+0000","last_fullsized":"2024-04-25T08:34:46.093300+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:34.309763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531275+0000","last_change":"2024-04-25T08:34:44.532873+0000","last_active":"2024-04-25T08:34:45.531275+0000","last_peered":"2024-04-25T08:34:45.531275+0000","last_clean":"2024-04-25T08:34:45.531275+0000","last_became_active":"2024-04-25T08:34:44.532594+0000","last_became_peered":"2024-04-25T08:34:44.532594+0000","last_unstale":"2024-04-25T08:34:45.531275+0000","last_undegraded":"2024-04-25T08:34:45.531275+0000","last_fullsized":"2024-04-25T08:34:45.531275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:52.822815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530597+0000","last_change":"2024-04-25T08:34:43.877431+0000","last_active":"2024-04-25T08:34:45.530597+0000","last_peered":"2024-04-25T08:34:45.530597+0000","last_clean":"2024-04-25T08:34:45.530597+0000","last_became_active":"2024-04-25T08:34:43.877115+0000","last_became_peered":"2024-04-25T08:34:43.877115+0000","last_unstale":"2024-04-25T08:34:45.530597+0000","last_undegraded":"2024-04-25T08:34:45.530597+0000","last_fullsized":"2024-04-25T08:34:45.530597+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:55.485664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086192+0000","last_change":"2024-04-25T08:34:44.527817+0000","last_active":"2024-04-25T08:34:46.086192+0000","last_peered":"2024-04-25T08:34:46.086192+0000","last_clean":"2024-04-25T08:34:46.086192+0000","last_became_active":"2024-04-25T08:34:44.527550+0000","last_became_peered":"2024-04-25T08:34:44.527550+0000","last_unstale":"2024-04-25T08:34:46.086192+0000","last_undegraded":"2024-04-25T08:34:46.086192+0000","last_fullsized":"2024-04-25T08:34:46.086192+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:13.210050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530234+0000","last_change":"2024-04-25T08:34:43.883793+0000","last_active":"2024-04-25T08:34:45.530234+0000","last_peered":"2024-04-25T08:34:45.530234+0000","last_clean":"2024-04-25T08:34:45.530234+0000","last_became_active":"2024-04-25T08:34:43.882987+0000","last_became_peered":"2024-04-25T08:34:43.882987+0000","last_unstale":"2024-04-25T08:34:45.530234+0000","last_undegraded":"2024-04-25T08:34:45.530234+0000","last_fullsized":"2024-04-25T08:34:45.530234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:04.802285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093760+0000","last_change":"2024-04-25T08:34:44.528347+0000","last_active":"2024-04-25T08:34:46.093760+0000","last_peered":"2024-04-25T08:34:46.093760+0000","last_clean":"2024-04-25T08:34:46.093760+0000","last_became_active":"2024-04-25T08:34:44.528059+0000","last_became_peered":"2024-04-25T08:34:44.528059+0000","last_unstale":"2024-04-25T08:34:46.093760+0000","last_undegraded":"2024-04-25T08:34:46.093760+0000","last_fullsized":"2024-04-25T08:34:46.093760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.919973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086286+0000","last_change":"2024-04-25T08:34:43.878609+0000","last_active":"2024-04-25T08:34:46.086286+0000","last_peered":"2024-04-25T08:34:46.086286+0000","last_clean":"2024-04-25T08:34:46.086286+0000","last_became_active":"2024-04-25T08:34:43.878322+0000","last_became_peered":"2024-04-25T08:34:43.878322+0000","last_unstale":"2024-04-25T08:34:46.086286+0000","last_undegraded":"2024-04-25T08:34:46.086286+0000","last_fullsized":"2024-04-25T08:34:46.086286+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:46.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530698+0000","last_change":"2024-04-25T08:34:44.519833+0000","last_active":"2024-04-25T08:34:45.530698+0000","last_peered":"2024-04-25T08:34:45.530698+0000","last_clean":"2024-04-25T08:34:45.530698+0000","last_became_active":"2024-04-25T08:34:44.519535+0000","last_became_peered":"2024-04-25T08:34:44.519535+0000","last_unstale":"2024-04-25T08:34:45.530698+0000","last_undegraded":"2024-04-25T08:34:45.530698+0000","last_fullsized":"2024-04-25T08:34:45.530698+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.499854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093837+0000","last_change":"2024-04-25T08:34:43.885348+0000","last_active":"2024-04-25T08:34:46.093837+0000","last_peered":"2024-04-25T08:34:46.093837+0000","last_clean":"2024-04-25T08:34:46.093837+0000","last_became_active":"2024-04-25T08:34:43.882976+0000","last_became_peered":"2024-04-25T08:34:43.882976+0000","last_unstale":"2024-04-25T08:34:46.093837+0000","last_undegraded":"2024-04-25T08:34:46.093837+0000","last_fullsized":"2024-04-25T08:34:46.093837+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:46.548606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094705+0000","last_change":"2024-04-25T08:34:44.527976+0000","last_active":"2024-04-25T08:34:46.094705+0000","last_peered":"2024-04-25T08:34:46.094705+0000","last_clean":"2024-04-25T08:34:46.094705+0000","last_became_active":"2024-04-25T08:34:44.527199+0000","last_became_peered":"2024-04-25T08:34:44.527199+0000","last_unstale":"2024-04-25T08:34:46.094705+0000","last_undegraded":"2024-04-25T08:34:46.094705+0000","last_fullsized":"2024-04-25T08:34:46.094705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:26.675283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094290+0000","last_change":"2024-04-25T08:34:43.880717+0000","last_active":"2024-04-25T08:34:46.094290+0000","last_peered":"2024-04-25T08:34:46.094290+0000","last_clean":"2024-04-25T08:34:46.094290+0000","last_became_active":"2024-04-25T08:34:43.880417+0000","last_became_peered":"2024-04-25T08:34:43.880417+0000","last_unstale":"2024-04-25T08:34:46.094290+0000","last_undegraded":"2024-04-25T08:34:46.094290+0000","last_fullsized":"2024-04-25T08:34:46.094290+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:29.445107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086364+0000","last_change":"2024-04-25T08:34:44.532688+0000","last_active":"2024-04-25T08:34:46.086364+0000","last_peered":"2024-04-25T08:34:46.086364+0000","last_clean":"2024-04-25T08:34:46.086364+0000","last_became_active":"2024-04-25T08:34:44.532430+0000","last_became_peered":"2024-04-25T08:34:44.532430+0000","last_unstale":"2024-04-25T08:34:46.086364+0000","last_undegraded":"2024-04-25T08:34:46.086364+0000","last_fullsized":"2024-04-25T08:34:46.086364+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:19.629450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.092950+0000","last_change":"2024-04-25T08:34:43.880044+0000","last_active":"2024-04-25T08:34:46.092950+0000","last_peered":"2024-04-25T08:34:46.092950+0000","last_clean":"2024-04-25T08:34:46.092950+0000","last_became_active":"2024-04-25T08:34:43.879758+0000","last_became_peered":"2024-04-25T08:34:43.879758+0000","last_unstale":"2024-04-25T08:34:46.092950+0000","last_undegraded":"2024-04-25T08:34:46.092950+0000","last_fullsized":"2024-04-25T08:34:46.092950+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:27.565543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093707+0000","last_change":"2024-04-25T08:34:44.532922+0000","last_active":"2024-04-25T08:34:46.093707+0000","last_peered":"2024-04-25T08:34:46.093707+0000","last_clean":"2024-04-25T08:34:46.093707+0000","last_became_active":"2024-04-25T08:34:44.532361+0000","last_became_peered":"2024-04-25T08:34:44.532361+0000","last_unstale":"2024-04-25T08:34:46.093707+0000","last_undegraded":"2024-04-25T08:34:46.093707+0000","last_fullsized":"2024-04-25T08:34:46.093707+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:05.622445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530587+0000","last_change":"2024-04-25T08:34:43.881524+0000","last_active":"2024-04-25T08:34:45.530587+0000","last_peered":"2024-04-25T08:34:45.530587+0000","last_clean":"2024-04-25T08:34:45.530587+0000","last_became_active":"2024-04-25T08:34:43.881273+0000","last_became_peered":"2024-04-25T08:34:43.881273+0000","last_unstale":"2024-04-25T08:34:45.530587+0000","last_undegraded":"2024-04-25T08:34:45.530587+0000","last_fullsized":"2024-04-25T08:34:45.530587+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:54.880133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086187+0000","last_change":"2024-04-25T08:34:44.531176+0000","last_active":"2024-04-25T08:34:46.086187+0000","last_peered":"2024-04-25T08:34:46.086187+0000","last_clean":"2024-04-25T08:34:46.086187+0000","last_became_active":"2024-04-25T08:34:44.529858+0000","last_became_peered":"2024-04-25T08:34:44.529858+0000","last_unstale":"2024-04-25T08:34:46.086187+0000","last_undegraded":"2024-04-25T08:34:46.086187+0000","last_fullsized":"2024-04-25T08:34:46.086187+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:03.865071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530234+0000","last_change":"2024-04-25T08:34:43.875552+0000","last_active":"2024-04-25T08:34:45.530234+0000","last_peered":"2024-04-25T08:34:45.530234+0000","last_clean":"2024-04-25T08:34:45.530234+0000","last_became_active":"2024-04-25T08:34:43.875282+0000","last_became_peered":"2024-04-25T08:34:43.875282+0000","last_unstale":"2024-04-25T08:34:45.530234+0000","last_undegraded":"2024-04-25T08:34:45.530234+0000","last_fullsized":"2024-04-25T08:34:45.530234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:08.893632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094542+0000","last_change":"2024-04-25T08:34:44.534163+0000","last_active":"2024-04-25T08:34:46.094542+0000","last_peered":"2024-04-25T08:34:46.094542+0000","last_clean":"2024-04-25T08:34:46.094542+0000","last_became_active":"2024-04-25T08:34:44.533860+0000","last_became_peered":"2024-04-25T08:34:44.533860+0000","last_unstale":"2024-04-25T08:34:46.094542+0000","last_undegraded":"2024-04-25T08:34:46.094542+0000","last_fullsized":"2024-04-25T08:34:46.094542+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:26.675283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093840+0000","last_change":"2024-04-25T08:34:43.875641+0000","last_active":"2024-04-25T08:34:46.093840+0000","last_peered":"2024-04-25T08:34:46.093840+0000","last_clean":"2024-04-25T08:34:46.093840+0000","last_became_active":"2024-04-25T08:34:43.875344+0000","last_became_peered":"2024-04-25T08:34:43.875344+0000","last_unstale":"2024-04-25T08:34:46.093840+0000","last_undegraded":"2024-04-25T08:34:46.093840+0000","last_fullsized":"2024-04-25T08:34:46.093840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:44.497733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093679+0000","last_change":"2024-04-25T08:34:44.532615+0000","last_active":"2024-04-25T08:34:46.093679+0000","last_peered":"2024-04-25T08:34:46.093679+0000","last_clean":"2024-04-25T08:34:46.093679+0000","last_became_active":"2024-04-25T08:34:44.532335+0000","last_became_peered":"2024-04-25T08:34:44.532335+0000","last_unstale":"2024-04-25T08:34:46.093679+0000","last_undegraded":"2024-04-25T08:34:46.093679+0000","last_fullsized":"2024-04-25T08:34:46.093679+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:05.252545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094428+0000","last_change":"2024-04-25T08:34:43.889528+0000","last_active":"2024-04-25T08:34:46.094428+0000","last_peered":"2024-04-25T08:34:46.094428+0000","last_clean":"2024-04-25T08:34:46.094428+0000","last_became_active":"2024-04-25T08:34:43.889232+0000","last_became_peered":"2024-04-25T08:34:43.889232+0000","last_unstale":"2024-04-25T08:34:46.094428+0000","last_undegraded":"2024-04-25T08:34:46.094428+0000","last_fullsized":"2024-04-25T08:34:46.094428+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:45.556208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093705+0000","last_change":"2024-04-25T08:34:44.529316+0000","last_active":"2024-04-25T08:34:46.093705+0000","last_peered":"2024-04-25T08:34:46.093705+0000","last_clean":"2024-04-25T08:34:46.093705+0000","last_became_active":"2024-04-25T08:34:44.529026+0000","last_became_peered":"2024-04-25T08:34:44.529026+0000","last_unstale":"2024-04-25T08:34:46.093705+0000","last_undegraded":"2024-04-25T08:34:46.093705+0000","last_fullsized":"2024-04-25T08:34:46.093705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:33.006310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093931+0000","last_change":"2024-04-25T08:34:43.883046+0000","last_active":"2024-04-25T08:34:46.093931+0000","last_peered":"2024-04-25T08:34:46.093931+0000","last_clean":"2024-04-25T08:34:46.093931+0000","last_became_active":"2024-04-25T08:34:43.882748+0000","last_became_peered":"2024-04-25T08:34:43.882748+0000","last_unstale":"2024-04-25T08:34:46.093931+0000","last_undegraded":"2024-04-25T08:34:46.093931+0000","last_fullsized":"2024-04-25T08:34:46.093931+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:08.893632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094563+0000","last_change":"2024-04-25T08:34:44.536436+0000","last_active":"2024-04-25T08:34:46.094563+0000","last_peered":"2024-04-25T08:34:46.094563+0000","last_clean":"2024-04-25T08:34:46.094563+0000","last_became_active":"2024-04-25T08:34:44.534485+0000","last_became_peered":"2024-04-25T08:34:44.534485+0000","last_unstale":"2024-04-25T08:34:46.094563+0000","last_undegraded":"2024-04-25T08:34:46.094563+0000","last_fullsized":"2024-04-25T08:34:46.094563+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:31.481085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531115+0000","last_change":"2024-04-25T08:34:43.883072+0000","last_active":"2024-04-25T08:34:45.531115+0000","last_peered":"2024-04-25T08:34:45.531115+0000","last_clean":"2024-04-25T08:34:45.531115+0000","last_became_active":"2024-04-25T08:34:43.882688+0000","last_became_peered":"2024-04-25T08:34:43.882688+0000","last_unstale":"2024-04-25T08:34:45.531115+0000","last_undegraded":"2024-04-25T08:34:45.531115+0000","last_fullsized":"2024-04-25T08:34:45.531115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:16.493814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094446+0000","last_change":"2024-04-25T08:34:44.535560+0000","last_active":"2024-04-25T08:34:46.094446+0000","last_peered":"2024-04-25T08:34:46.094446+0000","last_clean":"2024-04-25T08:34:46.094446+0000","last_became_active":"2024-04-25T08:34:44.535149+0000","last_became_peered":"2024-04-25T08:34:44.535149+0000","last_unstale":"2024-04-25T08:34:46.094446+0000","last_undegraded":"2024-04-25T08:34:46.094446+0000","last_fullsized":"2024-04-25T08:34:46.094446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:06.085151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093700+0000","last_change":"2024-04-25T08:34:43.883730+0000","last_active":"2024-04-25T08:34:46.093700+0000","last_peered":"2024-04-25T08:34:46.093700+0000","last_clean":"2024-04-25T08:34:46.093700+0000","last_became_active":"2024-04-25T08:34:43.883435+0000","last_became_peered":"2024-04-25T08:34:43.883435+0000","last_unstale":"2024-04-25T08:34:46.093700+0000","last_undegraded":"2024-04-25T08:34:46.093700+0000","last_fullsized":"2024-04-25T08:34:46.093700+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:10.731254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530395+0000","last_change":"2024-04-25T08:34:44.526203+0000","last_active":"2024-04-25T08:34:45.530395+0000","last_peered":"2024-04-25T08:34:45.530395+0000","last_clean":"2024-04-25T08:34:45.530395+0000","last_became_active":"2024-04-25T08:34:44.525945+0000","last_became_peered":"2024-04-25T08:34:44.525945+0000","last_unstale":"2024-04-25T08:34:45.530395+0000","last_undegraded":"2024-04-25T08:34:45.530395+0000","last_fullsized":"2024-04-25T08:34:45.530395+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.447228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.869817+0000","last_change":"2024-04-25T08:34:43.881671+0000","last_active":"2024-04-25T08:34:46.869817+0000","last_peered":"2024-04-25T08:34:46.869817+0000","last_clean":"2024-04-25T08:34:46.869817+0000","last_became_active":"2024-04-25T08:34:43.881375+0000","last_became_peered":"2024-04-25T08:34:43.881375+0000","last_unstale":"2024-04-25T08:34:46.869817+0000","last_undegraded":"2024-04-25T08:34:46.869817+0000","last_fullsized":"2024-04-25T08:34:46.869817+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:48.237385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":235,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529016+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:34:45.529016+0000","last_peered":"2024-04-25T08:34:45.529016+0000","last_clean":"2024-04-25T08:34:45.529016+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:34:45.529016+0000","last_undegraded":"2024-04-25T08:34:45.529016+0000","last_fullsized":"2024-04-25T08:34:45.529016+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62924,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308916,"statfs":{"total":96636764160,"available":96572329984,"internally_reserved":0,"allocated":1245184,"data_stored":880204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40144,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331696,"statfs":{"total":96636764160,"available":96595656704,"internally_reserved":0,"allocated":651264,"data_stored":290364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35972,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335868,"statfs":{"total":96636764160,"available":96599928832,"internally_reserved":0,"allocated":643072,"data_stored":289712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58184,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313656,"statfs":{"total":96636764160,"available":96577183744,"internally_reserved":0,"allocated":647168,"data_stored":290294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44308,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327532,"statfs":{"total":96636764160,"available":96591392768,"internally_reserved":0,"allocated":659456,"data_stored":291346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":655360,"data_stored":291324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":651264,"data_stored":289872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50444,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321396,"statfs":{"total":96636764160,"available":96585109504,"internally_reserved":0,"allocated":1245184,"data_stored":880136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:34:50.497 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-04-25T08:34:50.501+0000 7f2637940640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:34:52.306 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:34:52.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.425+0000 7f21d054f640 1 Processor -- start 2024-04-25T08:34:52.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.425+0000 7f21d054f640 1 -- start start 2024-04-25T08:34:52.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21d054f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21c8144760 0x7f21c815faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:52.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21d054f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f21c8160030 0x7f21c80ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:52.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21d054f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c80b50c0 con 0x7f21c80b4b50 2024-04-25T08:34:52.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21d054f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c815f380 con 0x7f21c8160030 2024-04-25T08:34:52.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21d054f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c815f660 con 0x7f21c8144760 2024-04-25T08:34:52.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21cdaea640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f21c8160030 0x7f21c80ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:52.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21cdaea640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f21c8160030 0x7f21c80ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52730/0 (socket says 172.21.15.38:52730) 2024-04-25T08:34:52.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21cdaea640 1 -- 172.21.15.38:0/578516579 learned_addr learned my addr 172.21.15.38:0/578516579 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:52.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21ce2eb640 1 --2- 172.21.15.38:0/578516579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21c8144760 0x7f21c815faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:52.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21cdaea640 1 -- 172.21.15.38:0/578516579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21c8144760 msgr2=0x7f21c815faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:52.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21cdaea640 1 --2- 172.21.15.38:0/578516579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21c8144760 0x7f21c815faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21cdaea640 1 -- 172.21.15.38:0/578516579 >> v1:172.21.15.107:6789/0 conn(0x7f21c80b4b50 legacy=0x7f21c80b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:52.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.426+0000 7f21cdaea640 1 -- 172.21.15.38:0/578516579 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c80b53a0 con 0x7f21c8160030 2024-04-25T08:34:52.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.427+0000 7f21cdaea640 1 --2- 172.21.15.38:0/578516579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c8160030 0x7f21c80ce360 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009870 tx=0x7f21b802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4423721f55a62673 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:52.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.427+0000 7f21cd2e9640 1 -- 172.21.15.38:0/578516579 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8037070 con 0x7f21c8160030 2024-04-25T08:34:52.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.427+0000 7f21cd2e9640 1 -- 172.21.15.38:0/578516579 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b8032440 con 0x7f21c8160030 2024-04-25T08:34:52.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.427+0000 7f21cd2e9640 1 -- 172.21.15.38:0/578516579 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b8032810 con 0x7f21c8160030 2024-04-25T08:34:52.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.428+0000 7f21d054f640 1 -- 172.21.15.38:0/578516579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c8160030 msgr2=0x7f21c80ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:52.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.428+0000 7f21d054f640 1 --2- 172.21.15.38:0/578516579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c8160030 0x7f21c80ce360 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009870 tx=0x7f21b802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.428+0000 7f21d054f640 1 -- 172.21.15.38:0/578516579 shutdown_connections 2024-04-25T08:34:52.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.428+0000 7f21d054f640 1 --2- 172.21.15.38:0/578516579 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c8160030 0x7f21c80ce360 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.428+0000 7f21d054f640 1 --2- 172.21.15.38:0/578516579 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21c8144760 0x7f21c815faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.428+0000 7f21d054f640 1 -- 172.21.15.38:0/578516579 >> 172.21.15.38:0/578516579 conn(0x7f21c806b210 msgr2=0x7f21c806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:52.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.428+0000 7f21d054f640 1 -- 172.21.15.38:0/578516579 shutdown_connections 2024-04-25T08:34:52.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.428+0000 7f21d054f640 1 -- 172.21.15.38:0/578516579 wait complete. 2024-04-25T08:34:52.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.429+0000 7f21d054f640 1 Processor -- start 2024-04-25T08:34:52.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.429+0000 7f21d054f640 1 -- start start 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.430+0000 7f21d054f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80b4b50 0x7f21c80cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.430+0000 7f21d054f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21c8160030 0x7f21c8150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.430+0000 7f21d054f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c80c8890 0x7f21c815d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.430+0000 7f21d054f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c8145400 con 0x7f21c80b4b50 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.430+0000 7f21d054f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c8145280 con 0x7f21c80c8890 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.430+0000 7f21d054f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c80b3f20 con 0x7f21c8160030 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.430+0000 7f21ce2eb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80b4b50 0x7f21c80cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.430+0000 7f21ce2eb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80b4b50 0x7f21c80cae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52740/0 (socket says 172.21.15.38:52740) 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.430+0000 7f21ce2eb640 1 -- 172.21.15.38:0/1015759116 learned_addr learned my addr 172.21.15.38:0/1015759116 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.430+0000 7f21ceaec640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c80c8890 0x7f21c815d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21cdaea640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21c8160030 0x7f21c8150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21ce2eb640 1 -- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21c8160030 msgr2=0x7f21c8150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:52.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21ce2eb640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21c8160030 0x7f21c8150fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21ce2eb640 1 -- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c80c8890 msgr2=0x7f21c815d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:52.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21ce2eb640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c80c8890 0x7f21c815d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21ce2eb640 1 -- 172.21.15.38:0/1015759116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c81422d0 con 0x7f21c80b4b50 2024-04-25T08:34:52.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21ceaec640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c80c8890 0x7f21c815d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:52.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21ce2eb640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80b4b50 0x7f21c80cae00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f21c400d800 tx=0x7f21c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:52.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21beffd640 1 -- 172.21.15.38:0/1015759116 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c400b6c0 con 0x7f21c80b4b50 2024-04-25T08:34:52.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21beffd640 1 -- 172.21.15.38:0/1015759116 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c4019070 con 0x7f21c80b4b50 2024-04-25T08:34:52.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21d054f640 1 -- 172.21.15.38:0/1015759116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c80cb470 con 0x7f21c80b4b50 2024-04-25T08:34:52.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21beffd640 1 -- 172.21.15.38:0/1015759116 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c400ba80 con 0x7f21c80b4b50 2024-04-25T08:34:52.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.431+0000 7f21d054f640 1 -- 172.21.15.38:0/1015759116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c8142590 con 0x7f21c80b4b50 2024-04-25T08:34:52.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.432+0000 7f21d054f640 1 -- 172.21.15.38:0/1015759116 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c8068e80 con 0x7f21c80b4b50 2024-04-25T08:34:52.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.433+0000 7f21beffd640 1 -- 172.21.15.38:0/1015759116 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21c400bc20 con 0x7f21c80b4b50 2024-04-25T08:34:52.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.433+0000 7f21beffd640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f21ac06b4d0 0x7f21ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:52.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.433+0000 7f21beffd640 1 -- 172.21.15.38:0/1015759116 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f21c408a7e0 con 0x7f21c80b4b50 2024-04-25T08:34:52.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.433+0000 7f21cdaea640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f21ac06b4d0 0x7f21ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:52.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.434+0000 7f21cdaea640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f21ac06b4d0 0x7f21ac06d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f21b8034000 tx=0x7f21b802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:52.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.437+0000 7f21beffd640 1 -- 172.21.15.38:0/1015759116 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21c401c360 con 0x7f21c80b4b50 2024-04-25T08:34:52.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.616+0000 7f21d054f640 1 -- 172.21.15.38:0/1015759116 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21c81514e0 con 0x7f21ac06b4d0 2024-04-25T08:34:52.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.623+0000 7f21beffd640 1 -- 172.21.15.38:0/1015759116 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251952 (secure 0 0 0) 0x7f21b803c460 con 0x7f21ac06b4d0 2024-04-25T08:34:52.621 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:52.623 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:34:52.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.626+0000 7f21d054f640 1 -- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f21ac06b4d0 msgr2=0x7f21ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:52.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.626+0000 7f21d054f640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f21ac06b4d0 0x7f21ac06d990 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f21b8034000 tx=0x7f21b802e000 comp rx=0 tx=0).stop 2024-04-25T08:34:52.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.626+0000 7f21d054f640 1 -- 172.21.15.38:0/1015759116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80b4b50 msgr2=0x7f21c80cae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:52.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.626+0000 7f21d054f640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80b4b50 0x7f21c80cae00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f21c400d800 tx=0x7f21c400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.626+0000 7f21d054f640 1 -- 172.21.15.38:0/1015759116 shutdown_connections 2024-04-25T08:34:52.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.626+0000 7f21d054f640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21c80c8890 0x7f21c815d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.626+0000 7f21d054f640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f21ac06b4d0 0x7f21ac06d990 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.626+0000 7f21d054f640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f21c8160030 0x7f21c8150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.626+0000 7f21d054f640 1 --2- 172.21.15.38:0/1015759116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f21c80b4b50 0x7f21c80cae00 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.626+0000 7f21d054f640 1 -- 172.21.15.38:0/1015759116 >> 172.21.15.38:0/1015759116 conn(0x7f21c806b210 msgr2=0x7f21c80718a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:52.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.626+0000 7f21d054f640 1 -- 172.21.15.38:0/1015759116 shutdown_connections 2024-04-25T08:34:52.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.626+0000 7f21d054f640 1 -- 172.21.15.38:0/1015759116 wait complete. 2024-04-25T08:34:52.633 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-04-25T08:34:51.250550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":372256,"kb_used_data":6248,"kb_used_omap":12,"kb_used_meta":365875,"kb_avail":754602464,"statfs":{"total":773094113280,"available":772712923136,"internally_reserved":0,"allocated":6397952,"data_stored":3503252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":374656560},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.387968"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086305+0000","last_change":"2024-04-25T08:34:44.527955+0000","last_active":"2024-04-25T08:34:46.086305+0000","last_peered":"2024-04-25T08:34:46.086305+0000","last_clean":"2024-04-25T08:34:46.086305+0000","last_became_active":"2024-04-25T08:34:44.527662+0000","last_became_peered":"2024-04-25T08:34:44.527662+0000","last_unstale":"2024-04-25T08:34:46.086305+0000","last_undegraded":"2024-04-25T08:34:46.086305+0000","last_fullsized":"2024-04-25T08:34:46.086305+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:09.895753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529700+0000","last_change":"2024-04-25T08:34:44.528374+0000","last_active":"2024-04-25T08:34:45.529700+0000","last_peered":"2024-04-25T08:34:45.529700+0000","last_clean":"2024-04-25T08:34:45.529700+0000","last_became_active":"2024-04-25T08:34:44.528107+0000","last_became_peered":"2024-04-25T08:34:44.528107+0000","last_unstale":"2024-04-25T08:34:45.529700+0000","last_undegraded":"2024-04-25T08:34:45.529700+0000","last_fullsized":"2024-04-25T08:34:45.529700+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:18.254832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093741+0000","last_change":"2024-04-25T08:34:44.534121+0000","last_active":"2024-04-25T08:34:46.093741+0000","last_peered":"2024-04-25T08:34:46.093741+0000","last_clean":"2024-04-25T08:34:46.093741+0000","last_became_active":"2024-04-25T08:34:44.533765+0000","last_became_peered":"2024-04-25T08:34:44.533765+0000","last_unstale":"2024-04-25T08:34:46.093741+0000","last_undegraded":"2024-04-25T08:34:46.093741+0000","last_fullsized":"2024-04-25T08:34:46.093741+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.333087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529869+0000","last_change":"2024-04-25T08:34:44.535500+0000","last_active":"2024-04-25T08:34:45.529869+0000","last_peered":"2024-04-25T08:34:45.529869+0000","last_clean":"2024-04-25T08:34:45.529869+0000","last_became_active":"2024-04-25T08:34:44.535220+0000","last_became_peered":"2024-04-25T08:34:44.535220+0000","last_unstale":"2024-04-25T08:34:45.529869+0000","last_undegraded":"2024-04-25T08:34:45.529869+0000","last_fullsized":"2024-04-25T08:34:45.529869+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:43.198107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531042+0000","last_change":"2024-04-25T08:34:44.520910+0000","last_active":"2024-04-25T08:34:45.531042+0000","last_peered":"2024-04-25T08:34:45.531042+0000","last_clean":"2024-04-25T08:34:45.531042+0000","last_became_active":"2024-04-25T08:34:44.520607+0000","last_became_peered":"2024-04-25T08:34:44.520607+0000","last_unstale":"2024-04-25T08:34:45.531042+0000","last_undegraded":"2024-04-25T08:34:45.531042+0000","last_fullsized":"2024-04-25T08:34:45.531042+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:43.933652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086256+0000","last_change":"2024-04-25T08:34:44.527358+0000","last_active":"2024-04-25T08:34:46.086256+0000","last_peered":"2024-04-25T08:34:46.086256+0000","last_clean":"2024-04-25T08:34:46.086256+0000","last_became_active":"2024-04-25T08:34:44.527098+0000","last_became_peered":"2024-04-25T08:34:44.527098+0000","last_unstale":"2024-04-25T08:34:46.086256+0000","last_undegraded":"2024-04-25T08:34:46.086256+0000","last_fullsized":"2024-04-25T08:34:46.086256+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:47.550727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094066+0000","last_change":"2024-04-25T08:34:44.536974+0000","last_active":"2024-04-25T08:34:46.094066+0000","last_peered":"2024-04-25T08:34:46.094066+0000","last_clean":"2024-04-25T08:34:46.094066+0000","last_became_active":"2024-04-25T08:34:44.536690+0000","last_became_peered":"2024-04-25T08:34:44.536690+0000","last_unstale":"2024-04-25T08:34:46.094066+0000","last_undegraded":"2024-04-25T08:34:46.094066+0000","last_fullsized":"2024-04-25T08:34:46.094066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:27.300505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094429+0000","last_change":"2024-04-25T08:34:44.530573+0000","last_active":"2024-04-25T08:34:46.094429+0000","last_peered":"2024-04-25T08:34:46.094429+0000","last_clean":"2024-04-25T08:34:46.094429+0000","last_became_active":"2024-04-25T08:34:44.530307+0000","last_became_peered":"2024-04-25T08:34:44.530307+0000","last_unstale":"2024-04-25T08:34:46.094429+0000","last_undegraded":"2024-04-25T08:34:46.094429+0000","last_fullsized":"2024-04-25T08:34:46.094429+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:31.481085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094299+0000","last_change":"2024-04-25T08:34:44.537554+0000","last_active":"2024-04-25T08:34:46.094299+0000","last_peered":"2024-04-25T08:34:46.094299+0000","last_clean":"2024-04-25T08:34:46.094299+0000","last_became_active":"2024-04-25T08:34:44.537285+0000","last_became_peered":"2024-04-25T08:34:44.537285+0000","last_unstale":"2024-04-25T08:34:46.094299+0000","last_undegraded":"2024-04-25T08:34:46.094299+0000","last_fullsized":"2024-04-25T08:34:46.094299+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:25.030847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086137+0000","last_change":"2024-04-25T08:34:44.533732+0000","last_active":"2024-04-25T08:34:46.086137+0000","last_peered":"2024-04-25T08:34:46.086137+0000","last_clean":"2024-04-25T08:34:46.086137+0000","last_became_active":"2024-04-25T08:34:44.533441+0000","last_became_peered":"2024-04-25T08:34:44.533441+0000","last_unstale":"2024-04-25T08:34:46.086137+0000","last_undegraded":"2024-04-25T08:34:46.086137+0000","last_fullsized":"2024-04-25T08:34:46.086137+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:11.733375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093123+0000","last_change":"2024-04-25T08:34:44.530709+0000","last_active":"2024-04-25T08:34:46.093123+0000","last_peered":"2024-04-25T08:34:46.093123+0000","last_clean":"2024-04-25T08:34:46.093123+0000","last_became_active":"2024-04-25T08:34:44.530416+0000","last_became_peered":"2024-04-25T08:34:44.530416+0000","last_unstale":"2024-04-25T08:34:46.093123+0000","last_undegraded":"2024-04-25T08:34:46.093123+0000","last_fullsized":"2024-04-25T08:34:46.093123+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:32.618322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093910+0000","last_change":"2024-04-25T08:34:44.529095+0000","last_active":"2024-04-25T08:34:46.093910+0000","last_peered":"2024-04-25T08:34:46.093910+0000","last_clean":"2024-04-25T08:34:46.093910+0000","last_became_active":"2024-04-25T08:34:44.528821+0000","last_became_peered":"2024-04-25T08:34:44.528821+0000","last_unstale":"2024-04-25T08:34:46.093910+0000","last_undegraded":"2024-04-25T08:34:46.093910+0000","last_fullsized":"2024-04-25T08:34:46.093910+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:18.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086171+0000","last_change":"2024-04-25T08:34:44.531394+0000","last_active":"2024-04-25T08:34:46.086171+0000","last_peered":"2024-04-25T08:34:46.086171+0000","last_clean":"2024-04-25T08:34:46.086171+0000","last_became_active":"2024-04-25T08:34:44.531057+0000","last_became_peered":"2024-04-25T08:34:44.531057+0000","last_unstale":"2024-04-25T08:34:46.086171+0000","last_undegraded":"2024-04-25T08:34:46.086171+0000","last_fullsized":"2024-04-25T08:34:46.086171+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:18.730106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094348+0000","last_change":"2024-04-25T08:34:44.533220+0000","last_active":"2024-04-25T08:34:46.094348+0000","last_peered":"2024-04-25T08:34:46.094348+0000","last_clean":"2024-04-25T08:34:46.094348+0000","last_became_active":"2024-04-25T08:34:44.532647+0000","last_became_peered":"2024-04-25T08:34:44.532647+0000","last_unstale":"2024-04-25T08:34:46.094348+0000","last_undegraded":"2024-04-25T08:34:46.094348+0000","last_fullsized":"2024-04-25T08:34:46.094348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:00.701022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529308+0000","last_change":"2024-04-25T08:34:44.521532+0000","last_active":"2024-04-25T08:34:45.529308+0000","last_peered":"2024-04-25T08:34:45.529308+0000","last_clean":"2024-04-25T08:34:45.529308+0000","last_became_active":"2024-04-25T08:34:44.521274+0000","last_became_peered":"2024-04-25T08:34:44.521274+0000","last_unstale":"2024-04-25T08:34:45.529308+0000","last_undegraded":"2024-04-25T08:34:45.529308+0000","last_fullsized":"2024-04-25T08:34:45.529308+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.919973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086067+0000","last_change":"2024-04-25T08:34:44.525673+0000","last_active":"2024-04-25T08:34:46.086067+0000","last_peered":"2024-04-25T08:34:46.086067+0000","last_clean":"2024-04-25T08:34:46.086067+0000","last_became_active":"2024-04-25T08:34:44.525384+0000","last_became_peered":"2024-04-25T08:34:44.525384+0000","last_unstale":"2024-04-25T08:34:46.086067+0000","last_undegraded":"2024-04-25T08:34:46.086067+0000","last_fullsized":"2024-04-25T08:34:46.086067+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.499854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095168+0000","last_change":"2024-04-25T08:34:44.533259+0000","last_active":"2024-04-25T08:34:46.095168+0000","last_peered":"2024-04-25T08:34:46.095168+0000","last_clean":"2024-04-25T08:34:46.095168+0000","last_became_active":"2024-04-25T08:34:44.531589+0000","last_became_peered":"2024-04-25T08:34:44.531589+0000","last_unstale":"2024-04-25T08:34:46.095168+0000","last_undegraded":"2024-04-25T08:34:46.095168+0000","last_fullsized":"2024-04-25T08:34:46.095168+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:01.549992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529955+0000","last_change":"2024-04-25T08:34:44.527725+0000","last_active":"2024-04-25T08:34:45.529955+0000","last_peered":"2024-04-25T08:34:45.529955+0000","last_clean":"2024-04-25T08:34:45.529955+0000","last_became_active":"2024-04-25T08:34:44.526928+0000","last_became_peered":"2024-04-25T08:34:44.526928+0000","last_unstale":"2024-04-25T08:34:45.529955+0000","last_undegraded":"2024-04-25T08:34:45.529955+0000","last_fullsized":"2024-04-25T08:34:45.529955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.333087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093885+0000","last_change":"2024-04-25T08:34:44.530163+0000","last_active":"2024-04-25T08:34:46.093885+0000","last_peered":"2024-04-25T08:34:46.093885+0000","last_clean":"2024-04-25T08:34:46.093885+0000","last_became_active":"2024-04-25T08:34:44.529905+0000","last_became_peered":"2024-04-25T08:34:44.529905+0000","last_unstale":"2024-04-25T08:34:46.093885+0000","last_undegraded":"2024-04-25T08:34:46.093885+0000","last_fullsized":"2024-04-25T08:34:46.093885+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:18.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529970+0000","last_change":"2024-04-25T08:34:44.520405+0000","last_active":"2024-04-25T08:34:45.529970+0000","last_peered":"2024-04-25T08:34:45.529970+0000","last_clean":"2024-04-25T08:34:45.529970+0000","last_became_active":"2024-04-25T08:34:44.520122+0000","last_became_peered":"2024-04-25T08:34:44.520122+0000","last_unstale":"2024-04-25T08:34:45.529970+0000","last_undegraded":"2024-04-25T08:34:45.529970+0000","last_fullsized":"2024-04-25T08:34:45.529970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:09.811944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530067+0000","last_change":"2024-04-25T08:34:44.522344+0000","last_active":"2024-04-25T08:34:45.530067+0000","last_peered":"2024-04-25T08:34:45.530067+0000","last_clean":"2024-04-25T08:34:45.530067+0000","last_became_active":"2024-04-25T08:34:44.522063+0000","last_became_peered":"2024-04-25T08:34:44.522063+0000","last_unstale":"2024-04-25T08:34:45.530067+0000","last_undegraded":"2024-04-25T08:34:45.530067+0000","last_fullsized":"2024-04-25T08:34:45.530067+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:25.590051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093745+0000","last_change":"2024-04-25T08:34:44.530629+0000","last_active":"2024-04-25T08:34:46.093745+0000","last_peered":"2024-04-25T08:34:46.093745+0000","last_clean":"2024-04-25T08:34:46.093745+0000","last_became_active":"2024-04-25T08:34:44.530358+0000","last_became_peered":"2024-04-25T08:34:44.530358+0000","last_unstale":"2024-04-25T08:34:46.093745+0000","last_undegraded":"2024-04-25T08:34:46.093745+0000","last_fullsized":"2024-04-25T08:34:46.093745+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:18.333087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093078+0000","last_change":"2024-04-25T08:34:44.529987+0000","last_active":"2024-04-25T08:34:46.093078+0000","last_peered":"2024-04-25T08:34:46.093078+0000","last_clean":"2024-04-25T08:34:46.093078+0000","last_became_active":"2024-04-25T08:34:44.529724+0000","last_became_peered":"2024-04-25T08:34:44.529724+0000","last_unstale":"2024-04-25T08:34:46.093078+0000","last_undegraded":"2024-04-25T08:34:46.093078+0000","last_fullsized":"2024-04-25T08:34:46.093078+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:11.876226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093628+0000","last_change":"2024-04-25T08:34:44.536670+0000","last_active":"2024-04-25T08:34:46.093628+0000","last_peered":"2024-04-25T08:34:46.093628+0000","last_clean":"2024-04-25T08:34:46.093628+0000","last_became_active":"2024-04-25T08:34:44.536110+0000","last_became_peered":"2024-04-25T08:34:44.536110+0000","last_unstale":"2024-04-25T08:34:46.093628+0000","last_undegraded":"2024-04-25T08:34:46.093628+0000","last_fullsized":"2024-04-25T08:34:46.093628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:43.198107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095058+0000","last_change":"2024-04-25T08:34:44.533241+0000","last_active":"2024-04-25T08:34:46.095058+0000","last_peered":"2024-04-25T08:34:46.095058+0000","last_clean":"2024-04-25T08:34:46.095058+0000","last_became_active":"2024-04-25T08:34:44.531910+0000","last_became_peered":"2024-04-25T08:34:44.531910+0000","last_unstale":"2024-04-25T08:34:46.095058+0000","last_undegraded":"2024-04-25T08:34:46.095058+0000","last_fullsized":"2024-04-25T08:34:46.095058+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:33.963128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093922+0000","last_change":"2024-04-25T08:34:44.535827+0000","last_active":"2024-04-25T08:34:46.093922+0000","last_peered":"2024-04-25T08:34:46.093922+0000","last_clean":"2024-04-25T08:34:46.093922+0000","last_became_active":"2024-04-25T08:34:44.535273+0000","last_became_peered":"2024-04-25T08:34:44.535273+0000","last_unstale":"2024-04-25T08:34:46.093922+0000","last_undegraded":"2024-04-25T08:34:46.093922+0000","last_fullsized":"2024-04-25T08:34:46.093922+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:18.966389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086493+0000","last_change":"2024-04-25T08:34:44.538453+0000","last_active":"2024-04-25T08:34:46.086493+0000","last_peered":"2024-04-25T08:34:46.086493+0000","last_clean":"2024-04-25T08:34:46.086493+0000","last_became_active":"2024-04-25T08:34:44.538165+0000","last_became_peered":"2024-04-25T08:34:44.538165+0000","last_unstale":"2024-04-25T08:34:46.086493+0000","last_undegraded":"2024-04-25T08:34:46.086493+0000","last_fullsized":"2024-04-25T08:34:46.086493+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:46.558329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530135+0000","last_change":"2024-04-25T08:34:44.528517+0000","last_active":"2024-04-25T08:34:45.530135+0000","last_peered":"2024-04-25T08:34:45.530135+0000","last_clean":"2024-04-25T08:34:45.530135+0000","last_became_active":"2024-04-25T08:34:44.528229+0000","last_became_peered":"2024-04-25T08:34:44.528229+0000","last_unstale":"2024-04-25T08:34:45.530135+0000","last_undegraded":"2024-04-25T08:34:45.530135+0000","last_fullsized":"2024-04-25T08:34:45.530135+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:41.585740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094613+0000","last_change":"2024-04-25T08:34:44.528023+0000","last_active":"2024-04-25T08:34:46.094613+0000","last_peered":"2024-04-25T08:34:46.094613+0000","last_clean":"2024-04-25T08:34:46.094613+0000","last_became_active":"2024-04-25T08:34:44.527668+0000","last_became_peered":"2024-04-25T08:34:44.527668+0000","last_unstale":"2024-04-25T08:34:46.094613+0000","last_undegraded":"2024-04-25T08:34:46.094613+0000","last_fullsized":"2024-04-25T08:34:46.094613+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:06.085151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529886+0000","last_change":"2024-04-25T08:34:44.527591+0000","last_active":"2024-04-25T08:34:45.529886+0000","last_peered":"2024-04-25T08:34:45.529886+0000","last_clean":"2024-04-25T08:34:45.529886+0000","last_became_active":"2024-04-25T08:34:44.527285+0000","last_became_peered":"2024-04-25T08:34:44.527285+0000","last_unstale":"2024-04-25T08:34:45.529886+0000","last_undegraded":"2024-04-25T08:34:45.529886+0000","last_fullsized":"2024-04-25T08:34:45.529886+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:05.252545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094200+0000","last_change":"2024-04-25T08:34:44.533947+0000","last_active":"2024-04-25T08:34:46.094200+0000","last_peered":"2024-04-25T08:34:46.094200+0000","last_clean":"2024-04-25T08:34:46.094200+0000","last_became_active":"2024-04-25T08:34:44.533609+0000","last_became_peered":"2024-04-25T08:34:44.533609+0000","last_unstale":"2024-04-25T08:34:46.094200+0000","last_undegraded":"2024-04-25T08:34:46.094200+0000","last_fullsized":"2024-04-25T08:34:46.094200+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:44.889254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093558+0000","last_change":"2024-04-25T08:34:44.532723+0000","last_active":"2024-04-25T08:34:46.093558+0000","last_peered":"2024-04-25T08:34:46.093558+0000","last_clean":"2024-04-25T08:34:46.093558+0000","last_became_active":"2024-04-25T08:34:44.531850+0000","last_became_peered":"2024-04-25T08:34:44.531850+0000","last_unstale":"2024-04-25T08:34:46.093558+0000","last_undegraded":"2024-04-25T08:34:46.093558+0000","last_fullsized":"2024-04-25T08:34:46.093558+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:20.979454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529253+0000","last_change":"2024-04-25T08:34:44.527874+0000","last_active":"2024-04-25T08:34:45.529253+0000","last_peered":"2024-04-25T08:34:45.529253+0000","last_clean":"2024-04-25T08:34:45.529253+0000","last_became_active":"2024-04-25T08:34:44.527594+0000","last_became_peered":"2024-04-25T08:34:44.527594+0000","last_unstale":"2024-04-25T08:34:45.529253+0000","last_undegraded":"2024-04-25T08:34:45.529253+0000","last_fullsized":"2024-04-25T08:34:45.529253+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:05.622445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529837+0000","last_change":"2024-04-25T08:34:43.873145+0000","last_active":"2024-04-25T08:34:45.529837+0000","last_peered":"2024-04-25T08:34:45.529837+0000","last_clean":"2024-04-25T08:34:45.529837+0000","last_became_active":"2024-04-25T08:34:43.872860+0000","last_became_peered":"2024-04-25T08:34:43.872860+0000","last_unstale":"2024-04-25T08:34:45.529837+0000","last_undegraded":"2024-04-25T08:34:45.529837+0000","last_fullsized":"2024-04-25T08:34:45.529837+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:39.042852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093598+0000","last_change":"2024-04-25T08:34:44.530047+0000","last_active":"2024-04-25T08:34:46.093598+0000","last_peered":"2024-04-25T08:34:46.093598+0000","last_clean":"2024-04-25T08:34:46.093598+0000","last_became_active":"2024-04-25T08:34:44.529781+0000","last_became_peered":"2024-04-25T08:34:44.529781+0000","last_unstale":"2024-04-25T08:34:46.093598+0000","last_undegraded":"2024-04-25T08:34:46.093598+0000","last_fullsized":"2024-04-25T08:34:46.093598+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:36.970964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086257+0000","last_change":"2024-04-25T08:34:43.884678+0000","last_active":"2024-04-25T08:34:46.086257+0000","last_peered":"2024-04-25T08:34:46.086257+0000","last_clean":"2024-04-25T08:34:46.086257+0000","last_became_active":"2024-04-25T08:34:43.884357+0000","last_became_peered":"2024-04-25T08:34:43.884357+0000","last_unstale":"2024-04-25T08:34:46.086257+0000","last_undegraded":"2024-04-25T08:34:46.086257+0000","last_fullsized":"2024-04-25T08:34:46.086257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:04.802285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530601+0000","last_change":"2024-04-25T08:34:44.519962+0000","last_active":"2024-04-25T08:34:45.530601+0000","last_peered":"2024-04-25T08:34:45.530601+0000","last_clean":"2024-04-25T08:34:45.530601+0000","last_became_active":"2024-04-25T08:34:44.519671+0000","last_became_peered":"2024-04-25T08:34:44.519671+0000","last_unstale":"2024-04-25T08:34:45.530601+0000","last_undegraded":"2024-04-25T08:34:45.530601+0000","last_fullsized":"2024-04-25T08:34:45.530601+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:49.239506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086556+0000","last_change":"2024-04-25T08:34:43.878220+0000","last_active":"2024-04-25T08:34:46.086556+0000","last_peered":"2024-04-25T08:34:46.086556+0000","last_clean":"2024-04-25T08:34:46.086556+0000","last_became_active":"2024-04-25T08:34:43.877915+0000","last_became_peered":"2024-04-25T08:34:43.877915+0000","last_unstale":"2024-04-25T08:34:46.086556+0000","last_undegraded":"2024-04-25T08:34:46.086556+0000","last_fullsized":"2024-04-25T08:34:46.086556+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:55.485664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094296+0000","last_change":"2024-04-25T08:34:44.535169+0000","last_active":"2024-04-25T08:34:46.094296+0000","last_peered":"2024-04-25T08:34:46.094296+0000","last_clean":"2024-04-25T08:34:46.094296+0000","last_became_active":"2024-04-25T08:34:44.534858+0000","last_became_peered":"2024-04-25T08:34:44.534858+0000","last_unstale":"2024-04-25T08:34:46.094296+0000","last_undegraded":"2024-04-25T08:34:46.094296+0000","last_fullsized":"2024-04-25T08:34:46.094296+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:00.701022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093702+0000","last_change":"2024-04-25T08:34:43.882922+0000","last_active":"2024-04-25T08:34:46.093702+0000","last_peered":"2024-04-25T08:34:46.093702+0000","last_clean":"2024-04-25T08:34:46.093702+0000","last_became_active":"2024-04-25T08:34:43.882609+0000","last_became_peered":"2024-04-25T08:34:43.882609+0000","last_unstale":"2024-04-25T08:34:46.093702+0000","last_undegraded":"2024-04-25T08:34:46.093702+0000","last_fullsized":"2024-04-25T08:34:46.093702+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:17.727985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531083+0000","last_change":"2024-04-25T08:34:44.532481+0000","last_active":"2024-04-25T08:34:45.531083+0000","last_peered":"2024-04-25T08:34:45.531083+0000","last_clean":"2024-04-25T08:34:45.531083+0000","last_became_active":"2024-04-25T08:34:44.532204+0000","last_became_peered":"2024-04-25T08:34:44.532204+0000","last_unstale":"2024-04-25T08:34:45.531083+0000","last_undegraded":"2024-04-25T08:34:45.531083+0000","last_fullsized":"2024-04-25T08:34:45.531083+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:19.590018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094711+0000","last_change":"2024-04-25T08:34:43.883536+0000","last_active":"2024-04-25T08:34:46.094711+0000","last_peered":"2024-04-25T08:34:46.094711+0000","last_clean":"2024-04-25T08:34:46.094711+0000","last_became_active":"2024-04-25T08:34:43.883232+0000","last_became_peered":"2024-04-25T08:34:43.883232+0000","last_unstale":"2024-04-25T08:34:46.094711+0000","last_undegraded":"2024-04-25T08:34:46.094711+0000","last_fullsized":"2024-04-25T08:34:46.094711+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:10.731254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530119+0000","last_change":"2024-04-25T08:34:44.529487+0000","last_active":"2024-04-25T08:34:45.530119+0000","last_peered":"2024-04-25T08:34:45.530119+0000","last_clean":"2024-04-25T08:34:45.530119+0000","last_became_active":"2024-04-25T08:34:44.529217+0000","last_became_peered":"2024-04-25T08:34:44.529217+0000","last_unstale":"2024-04-25T08:34:45.530119+0000","last_undegraded":"2024-04-25T08:34:45.530119+0000","last_fullsized":"2024-04-25T08:34:45.530119+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:18.932728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531305+0000","last_change":"2024-04-25T08:34:43.875476+0000","last_active":"2024-04-25T08:34:45.531305+0000","last_peered":"2024-04-25T08:34:45.531305+0000","last_clean":"2024-04-25T08:34:45.531305+0000","last_became_active":"2024-04-25T08:34:43.874920+0000","last_became_peered":"2024-04-25T08:34:43.874920+0000","last_unstale":"2024-04-25T08:34:45.531305+0000","last_undegraded":"2024-04-25T08:34:45.531305+0000","last_fullsized":"2024-04-25T08:34:45.531305+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:54.880133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094086+0000","last_change":"2024-04-25T08:34:44.536131+0000","last_active":"2024-04-25T08:34:46.094086+0000","last_peered":"2024-04-25T08:34:46.094086+0000","last_clean":"2024-04-25T08:34:46.094086+0000","last_became_active":"2024-04-25T08:34:44.535860+0000","last_became_peered":"2024-04-25T08:34:44.535860+0000","last_unstale":"2024-04-25T08:34:46.094086+0000","last_undegraded":"2024-04-25T08:34:46.094086+0000","last_fullsized":"2024-04-25T08:34:46.094086+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:44.889254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530427+0000","last_change":"2024-04-25T08:34:43.869200+0000","last_active":"2024-04-25T08:34:45.530427+0000","last_peered":"2024-04-25T08:34:45.530427+0000","last_clean":"2024-04-25T08:34:45.530427+0000","last_became_active":"2024-04-25T08:34:43.868895+0000","last_became_peered":"2024-04-25T08:34:43.868895+0000","last_unstale":"2024-04-25T08:34:45.530427+0000","last_undegraded":"2024-04-25T08:34:45.530427+0000","last_fullsized":"2024-04-25T08:34:45.530427+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:36.714740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529600+0000","last_change":"2024-04-25T08:34:44.528869+0000","last_active":"2024-04-25T08:34:45.529600+0000","last_peered":"2024-04-25T08:34:45.529600+0000","last_clean":"2024-04-25T08:34:45.529600+0000","last_became_active":"2024-04-25T08:34:44.528619+0000","last_became_peered":"2024-04-25T08:34:44.528619+0000","last_unstale":"2024-04-25T08:34:45.529600+0000","last_undegraded":"2024-04-25T08:34:45.529600+0000","last_fullsized":"2024-04-25T08:34:45.529600+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:36.250986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093548+0000","last_change":"2024-04-25T08:34:43.881571+0000","last_active":"2024-04-25T08:34:46.093548+0000","last_peered":"2024-04-25T08:34:46.093548+0000","last_clean":"2024-04-25T08:34:46.093548+0000","last_became_active":"2024-04-25T08:34:43.881272+0000","last_became_peered":"2024-04-25T08:34:43.881272+0000","last_unstale":"2024-04-25T08:34:46.093548+0000","last_undegraded":"2024-04-25T08:34:46.093548+0000","last_fullsized":"2024-04-25T08:34:46.093548+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:52.868065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530959+0000","last_change":"2024-04-25T08:34:44.531396+0000","last_active":"2024-04-25T08:34:45.530959+0000","last_peered":"2024-04-25T08:34:45.530959+0000","last_clean":"2024-04-25T08:34:45.530959+0000","last_became_active":"2024-04-25T08:34:44.531134+0000","last_became_peered":"2024-04-25T08:34:44.531134+0000","last_unstale":"2024-04-25T08:34:45.530959+0000","last_undegraded":"2024-04-25T08:34:45.530959+0000","last_fullsized":"2024-04-25T08:34:45.530959+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:12.856976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529630+0000","last_change":"2024-04-25T08:34:43.874727+0000","last_active":"2024-04-25T08:34:45.529630+0000","last_peered":"2024-04-25T08:34:45.529630+0000","last_clean":"2024-04-25T08:34:45.529630+0000","last_became_active":"2024-04-25T08:34:43.874477+0000","last_became_peered":"2024-04-25T08:34:43.874477+0000","last_unstale":"2024-04-25T08:34:45.529630+0000","last_undegraded":"2024-04-25T08:34:45.529630+0000","last_fullsized":"2024-04-25T08:34:45.529630+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:12.207929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093479+0000","last_change":"2024-04-25T08:34:44.525749+0000","last_active":"2024-04-25T08:34:46.093479+0000","last_peered":"2024-04-25T08:34:46.093479+0000","last_clean":"2024-04-25T08:34:46.093479+0000","last_became_active":"2024-04-25T08:34:44.525470+0000","last_became_peered":"2024-04-25T08:34:44.525470+0000","last_unstale":"2024-04-25T08:34:46.093479+0000","last_undegraded":"2024-04-25T08:34:46.093479+0000","last_fullsized":"2024-04-25T08:34:46.093479+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:40.781389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530997+0000","last_change":"2024-04-25T08:34:43.876071+0000","last_active":"2024-04-25T08:34:45.530997+0000","last_peered":"2024-04-25T08:34:45.530997+0000","last_clean":"2024-04-25T08:34:45.530997+0000","last_became_active":"2024-04-25T08:34:43.875782+0000","last_became_peered":"2024-04-25T08:34:43.875782+0000","last_unstale":"2024-04-25T08:34:45.530997+0000","last_undegraded":"2024-04-25T08:34:45.530997+0000","last_fullsized":"2024-04-25T08:34:45.530997+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:39.042852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094436+0000","last_change":"2024-04-25T08:34:44.528331+0000","last_active":"2024-04-25T08:34:46.094436+0000","last_peered":"2024-04-25T08:34:46.094436+0000","last_clean":"2024-04-25T08:34:46.094436+0000","last_became_active":"2024-04-25T08:34:44.528080+0000","last_became_peered":"2024-04-25T08:34:44.528080+0000","last_unstale":"2024-04-25T08:34:46.094436+0000","last_undegraded":"2024-04-25T08:34:46.094436+0000","last_fullsized":"2024-04-25T08:34:46.094436+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:25.030847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530237+0000","last_change":"2024-04-25T08:34:43.877334+0000","last_active":"2024-04-25T08:34:45.530237+0000","last_peered":"2024-04-25T08:34:45.530237+0000","last_clean":"2024-04-25T08:34:45.530237+0000","last_became_active":"2024-04-25T08:34:43.877084+0000","last_became_peered":"2024-04-25T08:34:43.877084+0000","last_unstale":"2024-04-25T08:34:45.530237+0000","last_undegraded":"2024-04-25T08:34:45.530237+0000","last_fullsized":"2024-04-25T08:34:45.530237+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:46.548606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531206+0000","last_change":"2024-04-25T08:34:44.529717+0000","last_active":"2024-04-25T08:34:45.531206+0000","last_peered":"2024-04-25T08:34:45.531206+0000","last_clean":"2024-04-25T08:34:45.531206+0000","last_became_active":"2024-04-25T08:34:44.529442+0000","last_became_peered":"2024-04-25T08:34:44.529442+0000","last_unstale":"2024-04-25T08:34:45.531206+0000","last_undegraded":"2024-04-25T08:34:45.531206+0000","last_fullsized":"2024-04-25T08:34:45.531206+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:57.569222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094077+0000","last_change":"2024-04-25T08:34:43.887845+0000","last_active":"2024-04-25T08:34:46.094077+0000","last_peered":"2024-04-25T08:34:46.094077+0000","last_clean":"2024-04-25T08:34:46.094077+0000","last_became_active":"2024-04-25T08:34:43.887544+0000","last_became_peered":"2024-04-25T08:34:43.887544+0000","last_unstale":"2024-04-25T08:34:46.094077+0000","last_undegraded":"2024-04-25T08:34:46.094077+0000","last_fullsized":"2024-04-25T08:34:46.094077+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:37.118342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093571+0000","last_change":"2024-04-25T08:34:44.532534+0000","last_active":"2024-04-25T08:34:46.093571+0000","last_peered":"2024-04-25T08:34:46.093571+0000","last_clean":"2024-04-25T08:34:46.093571+0000","last_became_active":"2024-04-25T08:34:44.532250+0000","last_became_peered":"2024-04-25T08:34:44.532250+0000","last_unstale":"2024-04-25T08:34:46.093571+0000","last_undegraded":"2024-04-25T08:34:46.093571+0000","last_fullsized":"2024-04-25T08:34:46.093571+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:11.760981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094802+0000","last_change":"2024-04-25T08:34:43.883681+0000","last_active":"2024-04-25T08:34:46.094802+0000","last_peered":"2024-04-25T08:34:46.094802+0000","last_clean":"2024-04-25T08:34:46.094802+0000","last_became_active":"2024-04-25T08:34:43.883385+0000","last_became_peered":"2024-04-25T08:34:43.883385+0000","last_unstale":"2024-04-25T08:34:46.094802+0000","last_undegraded":"2024-04-25T08:34:46.094802+0000","last_fullsized":"2024-04-25T08:34:46.094802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:17.727985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094016+0000","last_change":"2024-04-25T08:34:44.532945+0000","last_active":"2024-04-25T08:34:46.094016+0000","last_peered":"2024-04-25T08:34:46.094016+0000","last_clean":"2024-04-25T08:34:46.094016+0000","last_became_active":"2024-04-25T08:34:44.532563+0000","last_became_peered":"2024-04-25T08:34:44.532563+0000","last_unstale":"2024-04-25T08:34:46.094016+0000","last_undegraded":"2024-04-25T08:34:46.094016+0000","last_fullsized":"2024-04-25T08:34:46.094016+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:18.966389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530710+0000","last_change":"2024-04-25T08:34:43.882768+0000","last_active":"2024-04-25T08:34:45.530710+0000","last_peered":"2024-04-25T08:34:45.530710+0000","last_clean":"2024-04-25T08:34:45.530710+0000","last_became_active":"2024-04-25T08:34:43.882459+0000","last_became_peered":"2024-04-25T08:34:43.882459+0000","last_unstale":"2024-04-25T08:34:45.530710+0000","last_undegraded":"2024-04-25T08:34:45.530710+0000","last_fullsized":"2024-04-25T08:34:45.530710+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:46.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094076+0000","last_change":"2024-04-25T08:34:44.534966+0000","last_active":"2024-04-25T08:34:46.094076+0000","last_peered":"2024-04-25T08:34:46.094076+0000","last_clean":"2024-04-25T08:34:46.094076+0000","last_became_active":"2024-04-25T08:34:44.534697+0000","last_became_peered":"2024-04-25T08:34:44.534697+0000","last_unstale":"2024-04-25T08:34:46.094076+0000","last_undegraded":"2024-04-25T08:34:46.094076+0000","last_fullsized":"2024-04-25T08:34:46.094076+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:27.300505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529417+0000","last_change":"2024-04-25T08:34:43.873249+0000","last_active":"2024-04-25T08:34:45.529417+0000","last_peered":"2024-04-25T08:34:45.529417+0000","last_clean":"2024-04-25T08:34:45.529417+0000","last_became_active":"2024-04-25T08:34:43.872980+0000","last_became_peered":"2024-04-25T08:34:43.872980+0000","last_unstale":"2024-04-25T08:34:45.529417+0000","last_undegraded":"2024-04-25T08:34:45.529417+0000","last_fullsized":"2024-04-25T08:34:45.529417+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:02.862950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094908+0000","last_change":"2024-04-25T08:34:44.533491+0000","last_active":"2024-04-25T08:34:46.094908+0000","last_peered":"2024-04-25T08:34:46.094908+0000","last_clean":"2024-04-25T08:34:46.094908+0000","last_became_active":"2024-04-25T08:34:44.532257+0000","last_became_peered":"2024-04-25T08:34:44.532257+0000","last_unstale":"2024-04-25T08:34:46.094908+0000","last_undegraded":"2024-04-25T08:34:46.094908+0000","last_fullsized":"2024-04-25T08:34:46.094908+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:41.495853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094668+0000","last_change":"2024-04-25T08:34:43.886395+0000","last_active":"2024-04-25T08:34:46.094668+0000","last_peered":"2024-04-25T08:34:46.094668+0000","last_clean":"2024-04-25T08:34:46.094668+0000","last_became_active":"2024-04-25T08:34:43.886111+0000","last_became_peered":"2024-04-25T08:34:43.886111+0000","last_unstale":"2024-04-25T08:34:46.094668+0000","last_undegraded":"2024-04-25T08:34:46.094668+0000","last_fullsized":"2024-04-25T08:34:46.094668+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:18.627329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094572+0000","last_change":"2024-04-25T08:34:44.530937+0000","last_active":"2024-04-25T08:34:46.094572+0000","last_peered":"2024-04-25T08:34:46.094572+0000","last_clean":"2024-04-25T08:34:46.094572+0000","last_became_active":"2024-04-25T08:34:44.529251+0000","last_became_peered":"2024-04-25T08:34:44.529251+0000","last_unstale":"2024-04-25T08:34:46.094572+0000","last_undegraded":"2024-04-25T08:34:46.094572+0000","last_fullsized":"2024-04-25T08:34:46.094572+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:41.495853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.095013+0000","last_change":"2024-04-25T08:34:43.889631+0000","last_active":"2024-04-25T08:34:46.095013+0000","last_peered":"2024-04-25T08:34:46.095013+0000","last_clean":"2024-04-25T08:34:46.095013+0000","last_became_active":"2024-04-25T08:34:43.889354+0000","last_became_peered":"2024-04-25T08:34:43.889354+0000","last_unstale":"2024-04-25T08:34:46.095013+0000","last_undegraded":"2024-04-25T08:34:46.095013+0000","last_fullsized":"2024-04-25T08:34:46.095013+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:02.862950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086134+0000","last_change":"2024-04-25T08:34:44.533026+0000","last_active":"2024-04-25T08:34:46.086134+0000","last_peered":"2024-04-25T08:34:46.086134+0000","last_clean":"2024-04-25T08:34:46.086134+0000","last_became_active":"2024-04-25T08:34:44.532759+0000","last_became_peered":"2024-04-25T08:34:44.532759+0000","last_unstale":"2024-04-25T08:34:46.086134+0000","last_undegraded":"2024-04-25T08:34:46.086134+0000","last_fullsized":"2024-04-25T08:34:46.086134+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:53.870186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093300+0000","last_change":"2024-04-25T08:34:43.874603+0000","last_active":"2024-04-25T08:34:46.093300+0000","last_peered":"2024-04-25T08:34:46.093300+0000","last_clean":"2024-04-25T08:34:46.093300+0000","last_became_active":"2024-04-25T08:34:43.874330+0000","last_became_peered":"2024-04-25T08:34:43.874330+0000","last_unstale":"2024-04-25T08:34:46.093300+0000","last_undegraded":"2024-04-25T08:34:46.093300+0000","last_fullsized":"2024-04-25T08:34:46.093300+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:34.309763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531275+0000","last_change":"2024-04-25T08:34:44.532873+0000","last_active":"2024-04-25T08:34:45.531275+0000","last_peered":"2024-04-25T08:34:45.531275+0000","last_clean":"2024-04-25T08:34:45.531275+0000","last_became_active":"2024-04-25T08:34:44.532594+0000","last_became_peered":"2024-04-25T08:34:44.532594+0000","last_unstale":"2024-04-25T08:34:45.531275+0000","last_undegraded":"2024-04-25T08:34:45.531275+0000","last_fullsized":"2024-04-25T08:34:45.531275+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:52.822815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530597+0000","last_change":"2024-04-25T08:34:43.877431+0000","last_active":"2024-04-25T08:34:45.530597+0000","last_peered":"2024-04-25T08:34:45.530597+0000","last_clean":"2024-04-25T08:34:45.530597+0000","last_became_active":"2024-04-25T08:34:43.877115+0000","last_became_peered":"2024-04-25T08:34:43.877115+0000","last_unstale":"2024-04-25T08:34:45.530597+0000","last_undegraded":"2024-04-25T08:34:45.530597+0000","last_fullsized":"2024-04-25T08:34:45.530597+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:55.485664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086192+0000","last_change":"2024-04-25T08:34:44.527817+0000","last_active":"2024-04-25T08:34:46.086192+0000","last_peered":"2024-04-25T08:34:46.086192+0000","last_clean":"2024-04-25T08:34:46.086192+0000","last_became_active":"2024-04-25T08:34:44.527550+0000","last_became_peered":"2024-04-25T08:34:44.527550+0000","last_unstale":"2024-04-25T08:34:46.086192+0000","last_undegraded":"2024-04-25T08:34:46.086192+0000","last_fullsized":"2024-04-25T08:34:46.086192+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:13.210050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530234+0000","last_change":"2024-04-25T08:34:43.883793+0000","last_active":"2024-04-25T08:34:45.530234+0000","last_peered":"2024-04-25T08:34:45.530234+0000","last_clean":"2024-04-25T08:34:45.530234+0000","last_became_active":"2024-04-25T08:34:43.882987+0000","last_became_peered":"2024-04-25T08:34:43.882987+0000","last_unstale":"2024-04-25T08:34:45.530234+0000","last_undegraded":"2024-04-25T08:34:45.530234+0000","last_fullsized":"2024-04-25T08:34:45.530234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:04.802285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093760+0000","last_change":"2024-04-25T08:34:44.528347+0000","last_active":"2024-04-25T08:34:46.093760+0000","last_peered":"2024-04-25T08:34:46.093760+0000","last_clean":"2024-04-25T08:34:46.093760+0000","last_became_active":"2024-04-25T08:34:44.528059+0000","last_became_peered":"2024-04-25T08:34:44.528059+0000","last_unstale":"2024-04-25T08:34:46.093760+0000","last_undegraded":"2024-04-25T08:34:46.093760+0000","last_fullsized":"2024-04-25T08:34:46.093760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:58.919973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086286+0000","last_change":"2024-04-25T08:34:43.878609+0000","last_active":"2024-04-25T08:34:46.086286+0000","last_peered":"2024-04-25T08:34:46.086286+0000","last_clean":"2024-04-25T08:34:46.086286+0000","last_became_active":"2024-04-25T08:34:43.878322+0000","last_became_peered":"2024-04-25T08:34:43.878322+0000","last_unstale":"2024-04-25T08:34:46.086286+0000","last_undegraded":"2024-04-25T08:34:46.086286+0000","last_fullsized":"2024-04-25T08:34:46.086286+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:46.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530698+0000","last_change":"2024-04-25T08:34:44.519833+0000","last_active":"2024-04-25T08:34:45.530698+0000","last_peered":"2024-04-25T08:34:45.530698+0000","last_clean":"2024-04-25T08:34:45.530698+0000","last_became_active":"2024-04-25T08:34:44.519535+0000","last_became_peered":"2024-04-25T08:34:44.519535+0000","last_unstale":"2024-04-25T08:34:45.530698+0000","last_undegraded":"2024-04-25T08:34:45.530698+0000","last_fullsized":"2024-04-25T08:34:45.530698+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:45.499854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093837+0000","last_change":"2024-04-25T08:34:43.885348+0000","last_active":"2024-04-25T08:34:46.093837+0000","last_peered":"2024-04-25T08:34:46.093837+0000","last_clean":"2024-04-25T08:34:46.093837+0000","last_became_active":"2024-04-25T08:34:43.882976+0000","last_became_peered":"2024-04-25T08:34:43.882976+0000","last_unstale":"2024-04-25T08:34:46.093837+0000","last_undegraded":"2024-04-25T08:34:46.093837+0000","last_fullsized":"2024-04-25T08:34:46.093837+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:46.548606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094705+0000","last_change":"2024-04-25T08:34:44.527976+0000","last_active":"2024-04-25T08:34:46.094705+0000","last_peered":"2024-04-25T08:34:46.094705+0000","last_clean":"2024-04-25T08:34:46.094705+0000","last_became_active":"2024-04-25T08:34:44.527199+0000","last_became_peered":"2024-04-25T08:34:44.527199+0000","last_unstale":"2024-04-25T08:34:46.094705+0000","last_undegraded":"2024-04-25T08:34:46.094705+0000","last_fullsized":"2024-04-25T08:34:46.094705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:26.675283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094290+0000","last_change":"2024-04-25T08:34:43.880717+0000","last_active":"2024-04-25T08:34:46.094290+0000","last_peered":"2024-04-25T08:34:46.094290+0000","last_clean":"2024-04-25T08:34:46.094290+0000","last_became_active":"2024-04-25T08:34:43.880417+0000","last_became_peered":"2024-04-25T08:34:43.880417+0000","last_unstale":"2024-04-25T08:34:46.094290+0000","last_undegraded":"2024-04-25T08:34:46.094290+0000","last_fullsized":"2024-04-25T08:34:46.094290+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:29.445107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086364+0000","last_change":"2024-04-25T08:34:44.532688+0000","last_active":"2024-04-25T08:34:46.086364+0000","last_peered":"2024-04-25T08:34:46.086364+0000","last_clean":"2024-04-25T08:34:46.086364+0000","last_became_active":"2024-04-25T08:34:44.532430+0000","last_became_peered":"2024-04-25T08:34:44.532430+0000","last_unstale":"2024-04-25T08:34:46.086364+0000","last_undegraded":"2024-04-25T08:34:46.086364+0000","last_fullsized":"2024-04-25T08:34:46.086364+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:19.629450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.092950+0000","last_change":"2024-04-25T08:34:43.880044+0000","last_active":"2024-04-25T08:34:46.092950+0000","last_peered":"2024-04-25T08:34:46.092950+0000","last_clean":"2024-04-25T08:34:46.092950+0000","last_became_active":"2024-04-25T08:34:43.879758+0000","last_became_peered":"2024-04-25T08:34:43.879758+0000","last_unstale":"2024-04-25T08:34:46.092950+0000","last_undegraded":"2024-04-25T08:34:46.092950+0000","last_fullsized":"2024-04-25T08:34:46.092950+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:27.565543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093707+0000","last_change":"2024-04-25T08:34:44.532922+0000","last_active":"2024-04-25T08:34:46.093707+0000","last_peered":"2024-04-25T08:34:46.093707+0000","last_clean":"2024-04-25T08:34:46.093707+0000","last_became_active":"2024-04-25T08:34:44.532361+0000","last_became_peered":"2024-04-25T08:34:44.532361+0000","last_unstale":"2024-04-25T08:34:46.093707+0000","last_undegraded":"2024-04-25T08:34:46.093707+0000","last_fullsized":"2024-04-25T08:34:46.093707+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:05.622445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530587+0000","last_change":"2024-04-25T08:34:43.881524+0000","last_active":"2024-04-25T08:34:45.530587+0000","last_peered":"2024-04-25T08:34:45.530587+0000","last_clean":"2024-04-25T08:34:45.530587+0000","last_became_active":"2024-04-25T08:34:43.881273+0000","last_became_peered":"2024-04-25T08:34:43.881273+0000","last_unstale":"2024-04-25T08:34:45.530587+0000","last_undegraded":"2024-04-25T08:34:45.530587+0000","last_fullsized":"2024-04-25T08:34:45.530587+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:54.880133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.086187+0000","last_change":"2024-04-25T08:34:44.531176+0000","last_active":"2024-04-25T08:34:46.086187+0000","last_peered":"2024-04-25T08:34:46.086187+0000","last_clean":"2024-04-25T08:34:46.086187+0000","last_became_active":"2024-04-25T08:34:44.529858+0000","last_became_peered":"2024-04-25T08:34:44.529858+0000","last_unstale":"2024-04-25T08:34:46.086187+0000","last_undegraded":"2024-04-25T08:34:46.086187+0000","last_fullsized":"2024-04-25T08:34:46.086187+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:03.865071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530234+0000","last_change":"2024-04-25T08:34:43.875552+0000","last_active":"2024-04-25T08:34:45.530234+0000","last_peered":"2024-04-25T08:34:45.530234+0000","last_clean":"2024-04-25T08:34:45.530234+0000","last_became_active":"2024-04-25T08:34:43.875282+0000","last_became_peered":"2024-04-25T08:34:43.875282+0000","last_unstale":"2024-04-25T08:34:45.530234+0000","last_undegraded":"2024-04-25T08:34:45.530234+0000","last_fullsized":"2024-04-25T08:34:45.530234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:08.893632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094542+0000","last_change":"2024-04-25T08:34:44.534163+0000","last_active":"2024-04-25T08:34:46.094542+0000","last_peered":"2024-04-25T08:34:46.094542+0000","last_clean":"2024-04-25T08:34:46.094542+0000","last_became_active":"2024-04-25T08:34:44.533860+0000","last_became_peered":"2024-04-25T08:34:44.533860+0000","last_unstale":"2024-04-25T08:34:46.094542+0000","last_undegraded":"2024-04-25T08:34:46.094542+0000","last_fullsized":"2024-04-25T08:34:46.094542+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:26.675283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093840+0000","last_change":"2024-04-25T08:34:43.875641+0000","last_active":"2024-04-25T08:34:46.093840+0000","last_peered":"2024-04-25T08:34:46.093840+0000","last_clean":"2024-04-25T08:34:46.093840+0000","last_became_active":"2024-04-25T08:34:43.875344+0000","last_became_peered":"2024-04-25T08:34:43.875344+0000","last_unstale":"2024-04-25T08:34:46.093840+0000","last_undegraded":"2024-04-25T08:34:46.093840+0000","last_fullsized":"2024-04-25T08:34:46.093840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:57:44.497733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093679+0000","last_change":"2024-04-25T08:34:44.532615+0000","last_active":"2024-04-25T08:34:46.093679+0000","last_peered":"2024-04-25T08:34:46.093679+0000","last_clean":"2024-04-25T08:34:46.093679+0000","last_became_active":"2024-04-25T08:34:44.532335+0000","last_became_peered":"2024-04-25T08:34:44.532335+0000","last_unstale":"2024-04-25T08:34:46.093679+0000","last_undegraded":"2024-04-25T08:34:46.093679+0000","last_fullsized":"2024-04-25T08:34:46.093679+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:05.252545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094428+0000","last_change":"2024-04-25T08:34:43.889528+0000","last_active":"2024-04-25T08:34:46.094428+0000","last_peered":"2024-04-25T08:34:46.094428+0000","last_clean":"2024-04-25T08:34:46.094428+0000","last_became_active":"2024-04-25T08:34:43.889232+0000","last_became_peered":"2024-04-25T08:34:43.889232+0000","last_unstale":"2024-04-25T08:34:46.094428+0000","last_undegraded":"2024-04-25T08:34:46.094428+0000","last_fullsized":"2024-04-25T08:34:46.094428+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:45.556208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093705+0000","last_change":"2024-04-25T08:34:44.529316+0000","last_active":"2024-04-25T08:34:46.093705+0000","last_peered":"2024-04-25T08:34:46.093705+0000","last_clean":"2024-04-25T08:34:46.093705+0000","last_became_active":"2024-04-25T08:34:44.529026+0000","last_became_peered":"2024-04-25T08:34:44.529026+0000","last_unstale":"2024-04-25T08:34:46.093705+0000","last_undegraded":"2024-04-25T08:34:46.093705+0000","last_fullsized":"2024-04-25T08:34:46.093705+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:33.006310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093931+0000","last_change":"2024-04-25T08:34:43.883046+0000","last_active":"2024-04-25T08:34:46.093931+0000","last_peered":"2024-04-25T08:34:46.093931+0000","last_clean":"2024-04-25T08:34:46.093931+0000","last_became_active":"2024-04-25T08:34:43.882748+0000","last_became_peered":"2024-04-25T08:34:43.882748+0000","last_unstale":"2024-04-25T08:34:46.093931+0000","last_undegraded":"2024-04-25T08:34:46.093931+0000","last_fullsized":"2024-04-25T08:34:46.093931+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:08.893632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094563+0000","last_change":"2024-04-25T08:34:44.536436+0000","last_active":"2024-04-25T08:34:46.094563+0000","last_peered":"2024-04-25T08:34:46.094563+0000","last_clean":"2024-04-25T08:34:46.094563+0000","last_became_active":"2024-04-25T08:34:44.534485+0000","last_became_peered":"2024-04-25T08:34:44.534485+0000","last_unstale":"2024-04-25T08:34:46.094563+0000","last_undegraded":"2024-04-25T08:34:46.094563+0000","last_fullsized":"2024-04-25T08:34:46.094563+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:31.481085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.531115+0000","last_change":"2024-04-25T08:34:43.883072+0000","last_active":"2024-04-25T08:34:45.531115+0000","last_peered":"2024-04-25T08:34:45.531115+0000","last_clean":"2024-04-25T08:34:45.531115+0000","last_became_active":"2024-04-25T08:34:43.882688+0000","last_became_peered":"2024-04-25T08:34:43.882688+0000","last_unstale":"2024-04-25T08:34:45.531115+0000","last_undegraded":"2024-04-25T08:34:45.531115+0000","last_fullsized":"2024-04-25T08:34:45.531115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:16.493814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.094446+0000","last_change":"2024-04-25T08:34:44.535560+0000","last_active":"2024-04-25T08:34:46.094446+0000","last_peered":"2024-04-25T08:34:46.094446+0000","last_clean":"2024-04-25T08:34:46.094446+0000","last_became_active":"2024-04-25T08:34:44.535149+0000","last_became_peered":"2024-04-25T08:34:44.535149+0000","last_unstale":"2024-04-25T08:34:46.094446+0000","last_undegraded":"2024-04-25T08:34:46.094446+0000","last_fullsized":"2024-04-25T08:34:46.094446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:06.085151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.093700+0000","last_change":"2024-04-25T08:34:43.883730+0000","last_active":"2024-04-25T08:34:46.093700+0000","last_peered":"2024-04-25T08:34:46.093700+0000","last_clean":"2024-04-25T08:34:46.093700+0000","last_became_active":"2024-04-25T08:34:43.883435+0000","last_became_peered":"2024-04-25T08:34:43.883435+0000","last_unstale":"2024-04-25T08:34:46.093700+0000","last_undegraded":"2024-04-25T08:34:46.093700+0000","last_fullsized":"2024-04-25T08:34:46.093700+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:10.731254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.530395+0000","last_change":"2024-04-25T08:34:44.526203+0000","last_active":"2024-04-25T08:34:45.530395+0000","last_peered":"2024-04-25T08:34:45.530395+0000","last_clean":"2024-04-25T08:34:45.530395+0000","last_became_active":"2024-04-25T08:34:44.525945+0000","last_became_peered":"2024-04-25T08:34:44.525945+0000","last_unstale":"2024-04-25T08:34:45.530395+0000","last_undegraded":"2024-04-25T08:34:45.530395+0000","last_fullsized":"2024-04-25T08:34:45.530395+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:43.855398+0000","last_clean_scrub_stamp":"2024-04-25T08:34:43.855398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:30.447228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:46.869817+0000","last_change":"2024-04-25T08:34:43.881671+0000","last_active":"2024-04-25T08:34:46.869817+0000","last_peered":"2024-04-25T08:34:46.869817+0000","last_clean":"2024-04-25T08:34:46.869817+0000","last_became_active":"2024-04-25T08:34:43.881375+0000","last_became_peered":"2024-04-25T08:34:43.881375+0000","last_unstale":"2024-04-25T08:34:46.869817+0000","last_undegraded":"2024-04-25T08:34:46.869817+0000","last_fullsized":"2024-04-25T08:34:46.869817+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:34:42.853277+0000","last_clean_scrub_stamp":"2024-04-25T08:34:42.853277+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:48.237385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":235,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T08:34:45.529016+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:34:45.529016+0000","last_peered":"2024-04-25T08:34:45.529016+0000","last_clean":"2024-04-25T08:34:45.529016+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:34:45.529016+0000","last_undegraded":"2024-04-25T08:34:45.529016+0000","last_fullsized":"2024-04-25T08:34:45.529016+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":62924,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94308916,"statfs":{"total":96636764160,"available":96572329984,"internally_reserved":0,"allocated":1245184,"data_stored":880204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":63175114},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40144,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331696,"statfs":{"total":96636764160,"available":96595656704,"internally_reserved":0,"allocated":651264,"data_stored":290364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35972,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335868,"statfs":{"total":96636764160,"available":96599928832,"internally_reserved":0,"allocated":643072,"data_stored":289712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58184,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313656,"statfs":{"total":96636764160,"available":96577183744,"internally_reserved":0,"allocated":647168,"data_stored":290294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44308,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327532,"statfs":{"total":96636764160,"available":96591392768,"internally_reserved":0,"allocated":659456,"data_stored":291346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":655360,"data_stored":291324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40140,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331700,"statfs":{"total":96636764160,"available":96595660800,"internally_reserved":0,"allocated":651264,"data_stored":289872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50444,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321396,"statfs":{"total":96636764160,"available":96585109504,"internally_reserved":0,"allocated":1245184,"data_stored":880136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:34:52.636 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:34:52.636 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:34:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.784+0000 7f0d4d23f640 1 Processor -- start 2024-04-25T08:34:52.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.784+0000 7f0d4d23f640 1 -- start start 2024-04-25T08:34:52.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d4d23f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0d4815de60 0x7f0d480b9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:52.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d4d23f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0d480d0060 0x7f0d480d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d4d23f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d480d1a80 con 0x7f0d4815da80 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d4d23f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d48161fa0 con 0x7f0d4815de60 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d4d23f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d48162280 con 0x7f0d480d0060 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d46d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0d4815da80 0x7f0d48161770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54264/0 (socket says 172.21.15.38:54264) 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d46d9d640 1 -- 172.21.15.38:0/2754897560 learned_addr learned my addr 172.21.15.38:0/2754897560 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d4659c640 1 --2- 172.21.15.38:0/2754897560 >> v2:172.21.15.38:3300/0 conn(0x7f0d4815de60 0x7f0d480b9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d45d9b640 1 -- 172.21.15.38:0/2754897560 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994388490 0 0) 0x7f0d480d1a80 con 0x7f0d4815da80 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d4759e640 1 --2- 172.21.15.38:0/2754897560 >> v2:172.21.15.107:3300/0 conn(0x7f0d480d0060 0x7f0d480d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d45d9b640 1 -- 172.21.15.38:0/2754897560 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d34003520 con 0x7f0d4815da80 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d4659c640 1 -- 172.21.15.38:0/2754897560 >> v2:172.21.15.107:3300/0 conn(0x7f0d480d0060 msgr2=0x7f0d480d04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d4659c640 1 --2- 172.21.15.38:0/2754897560 >> v2:172.21.15.107:3300/0 conn(0x7f0d480d0060 0x7f0d480d04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.785+0000 7f0d4659c640 1 -- 172.21.15.38:0/2754897560 >> v1:172.21.15.38:6789/0 conn(0x7f0d4815da80 legacy=0x7f0d48161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:52.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.786+0000 7f0d4659c640 1 -- 172.21.15.38:0/2754897560 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d48162560 con 0x7f0d4815de60 2024-04-25T08:34:52.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.786+0000 7f0d4659c640 1 --2- 172.21.15.38:0/2754897560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d4815de60 0x7f0d480b9530 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0d480ca950 tx=0x7f0d3002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806b77e39d6c6188 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:52.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.786+0000 7f0d45d9b640 1 -- 172.21.15.38:0/2754897560 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30037070 con 0x7f0d4815de60 2024-04-25T08:34:52.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.786+0000 7f0d45d9b640 1 -- 172.21.15.38:0/2754897560 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d30032440 con 0x7f0d4815de60 2024-04-25T08:34:52.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.786+0000 7f0d45d9b640 1 -- 172.21.15.38:0/2754897560 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30032810 con 0x7f0d4815de60 2024-04-25T08:34:52.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.787+0000 7f0d4d23f640 1 -- 172.21.15.38:0/2754897560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d4815de60 msgr2=0x7f0d480b9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:52.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.787+0000 7f0d4d23f640 1 --2- 172.21.15.38:0/2754897560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d4815de60 0x7f0d480b9530 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f0d480ca950 tx=0x7f0d3002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.787+0000 7f0d4d23f640 1 -- 172.21.15.38:0/2754897560 shutdown_connections 2024-04-25T08:34:52.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.787+0000 7f0d4d23f640 1 --2- 172.21.15.38:0/2754897560 >> v2:172.21.15.107:3300/0 conn(0x7f0d480d0060 0x7f0d480d04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.787+0000 7f0d4d23f640 1 --2- 172.21.15.38:0/2754897560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d4815de60 0x7f0d480b9530 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.787+0000 7f0d4d23f640 1 -- 172.21.15.38:0/2754897560 >> 172.21.15.38:0/2754897560 conn(0x7f0d4806be60 msgr2=0x7f0d4806c240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:52.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.787+0000 7f0d4d23f640 1 -- 172.21.15.38:0/2754897560 shutdown_connections 2024-04-25T08:34:52.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.787+0000 7f0d4d23f640 1 -- 172.21.15.38:0/2754897560 wait complete. 2024-04-25T08:34:52.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.788+0000 7f0d4d23f640 1 Processor -- start 2024-04-25T08:34:52.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.789+0000 7f0d4d23f640 1 -- start start 2024-04-25T08:34:52.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.789+0000 7f0d4d23f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d480d0060 0x7f0d4815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:52.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.789+0000 7f0d4d23f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d4815da80 0x7f0d481f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:52.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.789+0000 7f0d4d23f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d4815de60 0x7f0d481f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:52.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.789+0000 7f0d4d23f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480d23e0 con 0x7f0d4815da80 2024-04-25T08:34:52.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.789+0000 7f0d4d23f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480d2260 con 0x7f0d480d0060 2024-04-25T08:34:52.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.789+0000 7f0d4d23f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d480d2560 con 0x7f0d4815de60 2024-04-25T08:34:52.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.789+0000 7f0d4659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d4815da80 0x7f0d481f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:52.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.789+0000 7f0d4659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d4815da80 0x7f0d481f3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52764/0 (socket says 172.21.15.38:52764) 2024-04-25T08:34:52.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.789+0000 7f0d4659c640 1 -- 172.21.15.38:0/4089837405 learned_addr learned my addr 172.21.15.38:0/4089837405 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d46d9d640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d480d0060 0x7f0d4815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d4759e640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d4815de60 0x7f0d481f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d4659c640 1 -- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d4815de60 msgr2=0x7f0d481f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d4659c640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d4815de60 0x7f0d481f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d4659c640 1 -- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d480d0060 msgr2=0x7f0d4815b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d4659c640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d480d0060 0x7f0d4815b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d4659c640 1 -- 172.21.15.38:0/4089837405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d480cc510 con 0x7f0d4815da80 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d4759e640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d4815de60 0x7f0d481f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d4659c640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d4815da80 0x7f0d481f3280 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0d30000c00 tx=0x7f0d300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d3f7fe640 1 -- 172.21.15.38:0/4089837405 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30037040 con 0x7f0d4815da80 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d3f7fe640 1 -- 172.21.15.38:0/4089837405 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d30042070 con 0x7f0d4815da80 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d4d23f640 1 -- 172.21.15.38:0/4089837405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d480cc7f0 con 0x7f0d4815da80 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d3f7fe640 1 -- 172.21.15.38:0/4089837405 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30030960 con 0x7f0d4815da80 2024-04-25T08:34:52.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.790+0000 7f0d4d23f640 1 -- 172.21.15.38:0/4089837405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d480ccdb0 con 0x7f0d4815da80 2024-04-25T08:34:52.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.791+0000 7f0d4d23f640 1 -- 172.21.15.38:0/4089837405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d48068e80 con 0x7f0d4815da80 2024-04-25T08:34:52.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.791+0000 7f0d3f7fe640 1 -- 172.21.15.38:0/4089837405 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d30030b00 con 0x7f0d4815da80 2024-04-25T08:34:52.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.792+0000 7f0d3f7fe640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d2806b4d0 0x7f0d2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:52.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.792+0000 7f0d3f7fe640 1 -- 172.21.15.38:0/4089837405 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0d300aac50 con 0x7f0d4815da80 2024-04-25T08:34:52.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.792+0000 7f0d46d9d640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d2806b4d0 0x7f0d2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:52.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.792+0000 7f0d46d9d640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d2806b4d0 0x7f0d2806d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d40005880 tx=0x7f0d40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:52.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:52.796+0000 7f0d3f7fe640 1 -- 172.21.15.38:0/4089837405 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d300b0020 con 0x7f0d4815da80 2024-04-25T08:34:53.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.044+0000 7f0d4d23f640 1 -- 172.21.15.38:0/4089837405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0d480d15a0 con 0x7f0d4815da80 2024-04-25T08:34:53.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.046+0000 7f0d3f7fe640 1 -- 172.21.15.38:0/4089837405 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0d30045360 con 0x7f0d4815da80 2024-04-25T08:34:53.042 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-04-25T08:34:53.042 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw== 2024-04-25T08:34:53.042 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-04-25T08:34:53.042 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-04-25T08:34:53.042 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:34:53.042 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-04-25T08:34:53.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.048+0000 7f0d4d23f640 1 -- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d2806b4d0 msgr2=0x7f0d2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.048+0000 7f0d4d23f640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d2806b4d0 0x7f0d2806d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0d40005880 tx=0x7f0d40009040 comp rx=0 tx=0).stop 2024-04-25T08:34:53.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.048+0000 7f0d4d23f640 1 -- 172.21.15.38:0/4089837405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d4815da80 msgr2=0x7f0d481f3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.048+0000 7f0d4d23f640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d4815da80 0x7f0d481f3280 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0d30000c00 tx=0x7f0d300046c0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.048+0000 7f0d4d23f640 1 -- 172.21.15.38:0/4089837405 shutdown_connections 2024-04-25T08:34:53.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.048+0000 7f0d4d23f640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d2806b4d0 0x7f0d2806d990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.048+0000 7f0d4d23f640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d4815de60 0x7f0d481f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.048+0000 7f0d4d23f640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d4815da80 0x7f0d481f3280 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.048+0000 7f0d4d23f640 1 --2- 172.21.15.38:0/4089837405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d480d0060 0x7f0d4815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.048+0000 7f0d4d23f640 1 -- 172.21.15.38:0/4089837405 >> 172.21.15.38:0/4089837405 conn(0x7f0d4806be60 msgr2=0x7f0d480b8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:53.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.048+0000 7f0d4d23f640 1 -- 172.21.15.38:0/4089837405 shutdown_connections 2024-04-25T08:34:53.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.048+0000 7f0d4d23f640 1 -- 172.21.15.38:0/4089837405 wait complete. 2024-04-25T08:34:53.055 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:34:53.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.160+0000 7fc1d1d53640 1 Processor -- start 2024-04-25T08:34:53.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.160+0000 7fc1d1d53640 1 -- start start 2024-04-25T08:34:53.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.160+0000 7fc1d1d53640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 0x7fc1cc0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1d1d53640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc1cc0d0f60 0x7fc1cc0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1d1d53640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1cc0d3e70 con 0x7fc1cc0ced60 2024-04-25T08:34:53.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1d1d53640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1cc15dff0 con 0x7fc1cc0d0f60 2024-04-25T08:34:53.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1d1d53640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc0b6990 con 0x7fc1cc15dc10 2024-04-25T08:34:53.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1cb7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 0x7fc1cc0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1cbfff640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc1cc0d0f60 0x7fc1cc0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1cb7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 0x7fc1cc0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:40682/0 (socket says 172.21.15.38:40682) 2024-04-25T08:34:53.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1cbfff640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc1cc0d0f60 0x7fc1cc0b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:38600/0 (socket says 172.21.15.38:38600) 2024-04-25T08:34:53.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1cb7fe640 1 -- 172.21.15.38:0/298220906 learned_addr learned my addr 172.21.15.38:0/298220906 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:53.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1caffd640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fc1cc0ced60 0x7fc1cc0d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:59354/0 (socket says 172.21.15.38:59354) 2024-04-25T08:34:53.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1cb7fe640 1 -- 172.21.15.38:0/298220906 >> v2:172.21.15.107:3300/0 conn(0x7fc1cc0d0f60 msgr2=0x7fc1cc0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:53.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1cb7fe640 1 --2- 172.21.15.38:0/298220906 >> v2:172.21.15.107:3300/0 conn(0x7fc1cc0d0f60 0x7fc1cc0b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1cb7fe640 1 -- 172.21.15.38:0/298220906 >> v1:172.21.15.107:6789/0 conn(0x7fc1cc0ced60 legacy=0x7fc1cc0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1ca7fc640 1 -- 172.21.15.38:0/298220906 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150410247 0 0) 0x7fc1cc0d3e70 con 0x7fc1cc0ced60 2024-04-25T08:34:53.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1cb7fe640 1 -- 172.21.15.38:0/298220906 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc0d4150 con 0x7fc1cc15dc10 2024-04-25T08:34:53.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.161+0000 7fc1cbfff640 1 --2- 172.21.15.38:0/298220906 >> v2:172.21.15.107:3300/0 conn(0x7fc1cc0d0f60 0x7fc1cc0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:53.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.162+0000 7fc1cb7fe640 1 --2- 172.21.15.38:0/298220906 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 0x7fc1cc0ce820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009870 tx=0x7fc1bc02a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5a0bba7e080501f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:53.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.162+0000 7fc1ca7fc640 1 -- 172.21.15.38:0/298220906 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc037070 con 0x7fc1cc15dc10 2024-04-25T08:34:53.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.162+0000 7fc1ca7fc640 1 -- 172.21.15.38:0/298220906 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc032440 con 0x7fc1cc15dc10 2024-04-25T08:34:53.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.162+0000 7fc1ca7fc640 1 -- 172.21.15.38:0/298220906 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc032810 con 0x7fc1cc15dc10 2024-04-25T08:34:53.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.163+0000 7fc1d1d53640 1 -- 172.21.15.38:0/298220906 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 msgr2=0x7fc1cc0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.163+0000 7fc1d1d53640 1 --2- 172.21.15.38:0/298220906 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 0x7fc1cc0ce820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009870 tx=0x7fc1bc02a1c0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.163+0000 7fc1d1d53640 1 -- 172.21.15.38:0/298220906 shutdown_connections 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.163+0000 7fc1d1d53640 1 --2- 172.21.15.38:0/298220906 >> v2:172.21.15.107:3300/0 conn(0x7fc1cc0d0f60 0x7fc1cc0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.163+0000 7fc1d1d53640 1 --2- 172.21.15.38:0/298220906 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 0x7fc1cc0ce820 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.163+0000 7fc1d1d53640 1 -- 172.21.15.38:0/298220906 >> 172.21.15.38:0/298220906 conn(0x7fc1cc066d10 msgr2=0x7fc1cc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.163+0000 7fc1d1d53640 1 -- 172.21.15.38:0/298220906 shutdown_connections 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.163+0000 7fc1d1d53640 1 -- 172.21.15.38:0/298220906 wait complete. 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.164+0000 7fc1d1d53640 1 Processor -- start 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.164+0000 7fc1d1d53640 1 -- start start 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1d1d53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1cc0ced60 0x7fc1cc0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1d1d53640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1cc0d0f60 0x7fc1cc0c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1d1d53640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 0x7fc1cc0c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1d1d53640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc0b6bc0 con 0x7fc1cc0ced60 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1d1d53640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc0d4710 con 0x7fc1cc0d0f60 2024-04-25T08:34:53.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1d1d53640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc0d3900 con 0x7fc1cc15dc10 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1cb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1cc0ced60 0x7fc1cc0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1cb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1cc0ced60 0x7fc1cc0cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52768/0 (socket says 172.21.15.38:52768) 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1cb7fe640 1 -- 172.21.15.38:0/1792468661 learned_addr learned my addr 172.21.15.38:0/1792468661 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1caffd640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1cc0d0f60 0x7fc1cc0c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1cbfff640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 0x7fc1cc0c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1cb7fe640 1 -- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 msgr2=0x7fc1cc0c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1cb7fe640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 0x7fc1cc0c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1cb7fe640 1 -- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1cc0d0f60 msgr2=0x7fc1cc0c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1cb7fe640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1cc0d0f60 0x7fc1cc0c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1cb7fe640 1 -- 172.21.15.38:0/1792468661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc15c220 con 0x7fc1cc0ced60 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1caffd640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1cc0d0f60 0x7fc1cc0c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.165+0000 7fc1cbfff640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 0x7fc1cc0c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.166+0000 7fc1cb7fe640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1cc0ced60 0x7fc1cc0cac70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc000c00 tx=0x7fc1bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:53.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.166+0000 7fc1b3fff640 1 -- 172.21.15.38:0/1792468661 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc037040 con 0x7fc1cc0ced60 2024-04-25T08:34:53.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.166+0000 7fc1b3fff640 1 -- 172.21.15.38:0/1792468661 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc042070 con 0x7fc1cc0ced60 2024-04-25T08:34:53.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.166+0000 7fc1d1d53640 1 -- 172.21.15.38:0/1792468661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc15c500 con 0x7fc1cc0ced60 2024-04-25T08:34:53.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.166+0000 7fc1b3fff640 1 -- 172.21.15.38:0/1792468661 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc030880 con 0x7fc1cc0ced60 2024-04-25T08:34:53.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.166+0000 7fc1d1d53640 1 -- 172.21.15.38:0/1792468661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc0c9030 con 0x7fc1cc0ced60 2024-04-25T08:34:53.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.167+0000 7fc1d1d53640 1 -- 172.21.15.38:0/1792468661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc068e80 con 0x7fc1cc0ced60 2024-04-25T08:34:53.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.167+0000 7fc1b3fff640 1 -- 172.21.15.38:0/1792468661 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1bc02ad10 con 0x7fc1cc0ced60 2024-04-25T08:34:53.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.168+0000 7fc1b3fff640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1b806b4d0 0x7fc1b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.168+0000 7fc1b3fff640 1 -- 172.21.15.38:0/1792468661 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc1bc0aa930 con 0x7fc1cc0ced60 2024-04-25T08:34:53.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.168+0000 7fc1caffd640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1b806b4d0 0x7fc1b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.169+0000 7fc1caffd640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1b806b4d0 0x7fc1b806d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc15a6f0 tx=0x7fc1c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:53.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.172+0000 7fc1b3fff640 1 -- 172.21.15.38:0/1792468661 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1bc079620 con 0x7fc1cc0ced60 2024-04-25T08:34:53.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.390+0000 7fc1d1d53640 1 -- 172.21.15.38:0/1792468661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1cc1e91f0 con 0x7fc1cc0ced60 2024-04-25T08:34:53.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.390+0000 7fc1b3fff640 1 -- 172.21.15.38:0/1792468661 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fc1bc078f20 con 0x7fc1cc0ced60 2024-04-25T08:34:53.386 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:34:53.386 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:34:46.222808+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:34:53.387 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 24 2024-04-25T08:34:53.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.392+0000 7fc1d1d53640 1 -- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1b806b4d0 msgr2=0x7fc1b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.392+0000 7fc1d1d53640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1b806b4d0 0x7fc1b806d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc15a6f0 tx=0x7fc1c0009040 comp rx=0 tx=0).stop 2024-04-25T08:34:53.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.392+0000 7fc1d1d53640 1 -- 172.21.15.38:0/1792468661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1cc0ced60 msgr2=0x7fc1cc0cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.392+0000 7fc1d1d53640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1cc0ced60 0x7fc1cc0cac70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc000c00 tx=0x7fc1bc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.392+0000 7fc1d1d53640 1 -- 172.21.15.38:0/1792468661 shutdown_connections 2024-04-25T08:34:53.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.392+0000 7fc1d1d53640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1b806b4d0 0x7fc1b806d990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.392+0000 7fc1d1d53640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1cc15dc10 0x7fc1cc0c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.392+0000 7fc1d1d53640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1cc0d0f60 0x7fc1cc0c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.392+0000 7fc1d1d53640 1 --2- 172.21.15.38:0/1792468661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1cc0ced60 0x7fc1cc0cac70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.392+0000 7fc1d1d53640 1 -- 172.21.15.38:0/1792468661 >> 172.21.15.38:0/1792468661 conn(0x7fc1cc066d10 msgr2=0x7fc1cc15cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:53.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.392+0000 7fc1d1d53640 1 -- 172.21.15.38:0/1792468661 shutdown_connections 2024-04-25T08:34:53.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.392+0000 7fc1d1d53640 1 -- 172.21.15.38:0/1792468661 wait complete. 2024-04-25T08:34:53.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:34:44.235451+0000', 'modified': '2024-04-25T08:34:46.222808+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5964}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5964': {'gid': 5964, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6834/226686241', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 226686241}, {'type': 'v1', 'addr': '172.21.15.38:6834', 'nonce': 226686241}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:34:53.803 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:34:53.803 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:34:53.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.912+0000 7fc21c858640 1 Processor -- start 2024-04-25T08:34:53.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.912+0000 7fc21c858640 1 -- start start 2024-04-25T08:34:53.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.912+0000 7fc21c858640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc21815db70 0x7fc2180d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.912+0000 7fc21c858640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc2180b92c0 0x7fc2180ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.912+0000 7fc21c858640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2180ceb20 0x7fc218161680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.912+0000 7fc21c858640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2180d1d10 con 0x7fc2180b92c0 2024-04-25T08:34:53.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.912+0000 7fc21c858640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21815df50 con 0x7fc21815db70 2024-04-25T08:34:53.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.912+0000 7fc21c858640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180d0750 con 0x7fc2180ceb20 2024-04-25T08:34:53.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.913+0000 7fc20ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc2180b92c0 0x7fc2180ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.913+0000 7fc20ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc2180b92c0 0x7fc2180ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52774/0 (socket says 172.21.15.38:52774) 2024-04-25T08:34:53.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.913+0000 7fc20ffff640 1 -- 172.21.15.38:0/3099784014 learned_addr learned my addr 172.21.15.38:0/3099784014 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:53.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.913+0000 7fc21659c640 1 --2- 172.21.15.38:0/3099784014 >> v2:172.21.15.107:3300/0 conn(0x7fc21815db70 0x7fc2180d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.913+0000 7fc216d9d640 1 --2- 172.21.15.38:0/3099784014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2180ceb20 0x7fc218161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.913+0000 7fc20ffff640 1 -- 172.21.15.38:0/3099784014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2180ceb20 msgr2=0x7fc218161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:53.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.913+0000 7fc20ffff640 1 --2- 172.21.15.38:0/3099784014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2180ceb20 0x7fc218161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.913+0000 7fc20ffff640 1 -- 172.21.15.38:0/3099784014 >> v2:172.21.15.107:3300/0 conn(0x7fc21815db70 msgr2=0x7fc2180d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:53.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.913+0000 7fc20ffff640 1 --2- 172.21.15.38:0/3099784014 >> v2:172.21.15.107:3300/0 conn(0x7fc21815db70 0x7fc2180d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.913+0000 7fc20ffff640 1 -- 172.21.15.38:0/3099784014 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180d0a30 con 0x7fc2180b92c0 2024-04-25T08:34:53.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.913+0000 7fc216d9d640 1 --2- 172.21.15.38:0/3099784014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2180ceb20 0x7fc218161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:34:53.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc20ffff640 1 --2- 172.21.15.38:0/3099784014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2180b92c0 0x7fc2180ce5e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc2180b63e0 tx=0x7fc20002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d4d093c2406d30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:53.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc215d9b640 1 -- 172.21.15.38:0/3099784014 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200037070 con 0x7fc2180b92c0 2024-04-25T08:34:53.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc215d9b640 1 -- 172.21.15.38:0/3099784014 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc200032440 con 0x7fc2180b92c0 2024-04-25T08:34:53.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc215d9b640 1 -- 172.21.15.38:0/3099784014 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc200032810 con 0x7fc2180b92c0 2024-04-25T08:34:53.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc21c858640 1 -- 172.21.15.38:0/3099784014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2180b92c0 msgr2=0x7fc2180ce5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc21c858640 1 --2- 172.21.15.38:0/3099784014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2180b92c0 0x7fc2180ce5e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fc2180b63e0 tx=0x7fc20002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc21c858640 1 -- 172.21.15.38:0/3099784014 shutdown_connections 2024-04-25T08:34:53.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc21c858640 1 --2- 172.21.15.38:0/3099784014 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2180ceb20 0x7fc218161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc21c858640 1 --2- 172.21.15.38:0/3099784014 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2180b92c0 0x7fc2180ce5e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc21c858640 1 --2- 172.21.15.38:0/3099784014 >> v2:172.21.15.107:3300/0 conn(0x7fc21815db70 0x7fc2180d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc21c858640 1 -- 172.21.15.38:0/3099784014 >> 172.21.15.38:0/3099784014 conn(0x7fc21806bed0 msgr2=0x7fc21806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:53.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc21c858640 1 -- 172.21.15.38:0/3099784014 shutdown_connections 2024-04-25T08:34:53.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.914+0000 7fc21c858640 1 -- 172.21.15.38:0/3099784014 wait complete. 2024-04-25T08:34:53.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.916+0000 7fc21c858640 1 Processor -- start 2024-04-25T08:34:53.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.916+0000 7fc21c858640 1 -- start start 2024-04-25T08:34:53.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.916+0000 7fc21c858640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2180b92c0 0x7fc218152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.916+0000 7fc21c858640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2180ceb20 0x7fc2180c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.916+0000 7fc21c858640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21815db70 0x7fc2180c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.916+0000 7fc21c858640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180d0ff0 con 0x7fc2180b92c0 2024-04-25T08:34:53.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.916+0000 7fc21c858640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2180b9740 con 0x7fc21815db70 2024-04-25T08:34:53.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.916+0000 7fc21c858640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc218161bc0 con 0x7fc2180ceb20 2024-04-25T08:34:53.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc21659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2180b92c0 0x7fc218152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc21659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2180b92c0 0x7fc218152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52776/0 (socket says 172.21.15.38:52776) 2024-04-25T08:34:53.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc21659c640 1 -- 172.21.15.38:0/2553162814 learned_addr learned my addr 172.21.15.38:0/2553162814 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:53.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc216d9d640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21815db70 0x7fc2180c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc20ffff640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2180ceb20 0x7fc2180c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc21659c640 1 -- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2180ceb20 msgr2=0x7fc2180c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc21659c640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2180ceb20 0x7fc2180c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc21659c640 1 -- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21815db70 msgr2=0x7fc2180c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc21659c640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21815db70 0x7fc2180c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc21659c640 1 -- 172.21.15.38:0/2553162814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21815f1a0 con 0x7fc2180b92c0 2024-04-25T08:34:53.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc21659c640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2180b92c0 0x7fc218152b70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc20800d5f0 tx=0x7fc20800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:53.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc20effd640 1 -- 172.21.15.38:0/2553162814 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080125f0 con 0x7fc2180b92c0 2024-04-25T08:34:53.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc20effd640 1 -- 172.21.15.38:0/2553162814 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208019070 con 0x7fc2180b92c0 2024-04-25T08:34:53.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc21c858640 1 -- 172.21.15.38:0/2553162814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2181e8b60 con 0x7fc2180b92c0 2024-04-25T08:34:53.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc20effd640 1 -- 172.21.15.38:0/2553162814 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080129d0 con 0x7fc2180b92c0 2024-04-25T08:34:53.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.917+0000 7fc21c858640 1 -- 172.21.15.38:0/2553162814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21815f420 con 0x7fc2180b92c0 2024-04-25T08:34:53.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.919+0000 7fc20effd640 1 -- 172.21.15.38:0/2553162814 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc208012b70 con 0x7fc2180b92c0 2024-04-25T08:34:53.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.920+0000 7fc20effd640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1f806b4d0 0x7fc1f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.920+0000 7fc20effd640 1 -- 172.21.15.38:0/2553162814 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc20808a6a0 con 0x7fc2180b92c0 2024-04-25T08:34:53.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.920+0000 7fc20ffff640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1f806b4d0 0x7fc1f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.920+0000 7fc20ffff640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1f806b4d0 0x7fc1f806d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc200034000 tx=0x7fc20002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:53.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.921+0000 7fc21f632740 1 Processor -- start 2024-04-25T08:34:53.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.921+0000 7fc21f632740 1 -- start start 2024-04-25T08:34:53.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.921+0000 7fc21f632740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.921+0000 7fc21f632740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621c5ad0 0x55e9621c6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.921+0000 7fc21f632740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e96217df10 0x55e9621c4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.921+0000 7fc21f632740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9620e0a10 con 0x55e9621a8230 2024-04-25T08:34:53.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.921+0000 7fc21f632740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e962089b30 con 0x55e96217df10 2024-04-25T08:34:53.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.921+0000 7fc21f632740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9620dcc50 con 0x55e9621c5ad0 2024-04-25T08:34:53.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc21659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc21659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621a8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52788/0 (socket says 172.21.15.38:52788) 2024-04-25T08:34:53.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc21659c640 1 -- 172.21.15.38:0/1623746774 learned_addr learned my addr 172.21.15.38:0/1623746774 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:53.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc216d9d640 1 --2- 172.21.15.38:0/1623746774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e96217df10 0x55e9621c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc20ffff640 1 --2- 172.21.15.38:0/1623746774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621c5ad0 0x55e9621c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc21659c640 1 -- 172.21.15.38:0/1623746774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621c5ad0 msgr2=0x55e9621c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:53.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc21659c640 1 --2- 172.21.15.38:0/1623746774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621c5ad0 0x55e9621c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc21659c640 1 -- 172.21.15.38:0/1623746774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e96217df10 msgr2=0x55e9621c4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:53.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc21659c640 1 --2- 172.21.15.38:0/1623746774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e96217df10 0x55e9621c4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc21659c640 1 -- 172.21.15.38:0/1623746774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e96209ee10 con 0x55e9621a8230 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc21659c640 1 --2- 172.21.15.38:0/1623746774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621a8610 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc208060e70 tx=0x7fc208058df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09c0e83288ca156 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc1f7fff640 1 -- 172.21.15.38:0/1623746774 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208070070 con 0x55e9621a8230 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc1f7fff640 1 -- 172.21.15.38:0/1623746774 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208056fb0 con 0x55e9621a8230 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.922+0000 7fc1f7fff640 1 -- 172.21.15.38:0/1623746774 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208088e60 con 0x55e9621a8230 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.923+0000 7fc21f632740 1 -- 172.21.15.38:0/1623746774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 msgr2=0x55e9621a8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.923+0000 7fc21f632740 1 --2- 172.21.15.38:0/1623746774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621a8610 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc208060e70 tx=0x7fc208058df0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.923+0000 7fc21f632740 1 -- 172.21.15.38:0/1623746774 shutdown_connections 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.923+0000 7fc21f632740 1 --2- 172.21.15.38:0/1623746774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e96217df10 0x55e9621c4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.923+0000 7fc21f632740 1 --2- 172.21.15.38:0/1623746774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621c5ad0 0x55e9621c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.923+0000 7fc21f632740 1 --2- 172.21.15.38:0/1623746774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621a8610 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.923+0000 7fc21f632740 1 -- 172.21.15.38:0/1623746774 >> 172.21.15.38:0/1623746774 conn(0x55e96217a840 msgr2=0x55e9621c3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.923+0000 7fc21f632740 1 -- 172.21.15.38:0/1623746774 shutdown_connections 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.923+0000 7fc21f632740 1 -- 172.21.15.38:0/1623746774 wait complete. 2024-04-25T08:34:53.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.924+0000 7fc21f632740 1 -- start start 2024-04-25T08:34:53.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.924+0000 7fc21f632740 1 Processor -- start 2024-04-25T08:34:53.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21f632740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621a8230 0x55e9621a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21f632740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621c5ad0 0x55e9621ab070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21f632740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 0x55e9621ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21f632740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96217ecb0 con 0x55e9621c5ad0 2024-04-25T08:34:53.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21f632740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96209d0c0 con 0x55e9621a8230 2024-04-25T08:34:53.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21f632740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e962178770 con 0x55e9621a9e10 2024-04-25T08:34:53.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21f632740 10 client.0 _ll_register_callbacks cb 0x55e962090ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:53.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621a8230 0x55e9621a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621a8230 0x55e9621a98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:38634/0 (socket says 172.21.15.38:38634) 2024-04-25T08:34:53.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21659c640 1 -- 172.21.15.38:0/138727099 learned_addr learned my addr 172.21.15.38:0/138727099 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:53.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc20ffff640 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621c5ad0 0x55e9621ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc216d9d640 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 0x55e9621ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21659c640 1 -- 172.21.15.38:0/138727099 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 msgr2=0x55e9621ab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:53.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21659c640 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 0x55e9621ab5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21659c640 1 -- 172.21.15.38:0/138727099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621c5ad0 msgr2=0x55e9621ab070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:53.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21659c640 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621c5ad0 0x55e9621ab070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21659c640 1 -- 172.21.15.38:0/138727099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e961df52d0 con 0x55e9621a8230 2024-04-25T08:34:53.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc20ffff640 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621c5ad0 0x55e9621ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:34:53.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc21659c640 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621a8230 0x55e9621a98d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc208088a90 tx=0x7fc208006f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=749f542bbe229897 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:53.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.925+0000 7fc1f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T08:34:53.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.928+0000 7fc1f57fa640 1 -- 172.21.15.38:0/138727099 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208074020 con 0x55e9621a8230 2024-04-25T08:34:53.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.928+0000 7fc1f57fa640 1 -- 172.21.15.38:0/138727099 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc208077b60 con 0x55e9621a8230 2024-04-25T08:34:53.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.928+0000 7fc21f632740 1 -- 172.21.15.38:0/138727099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc208056fb0 con 0x55e9621a8230 2024-04-25T08:34:53.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.931+0000 7fc1f57fa640 1 -- 172.21.15.38:0/138727099 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc208070040 con 0x55e9621a8230 2024-04-25T08:34:53.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.931+0000 7fc21f632740 10 client.5602 fetch_fsmap learned FSMap version 24 2024-04-25T08:34:53.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.931+0000 7fc21f632740 1 -- 172.21.15.38:0/138727099 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e961ebf6b0 con 0x55e9621a8230 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.932+0000 7fc1f57fa640 1 -- 172.21.15.38:0/138727099 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208056fb0 con 0x55e9621a8230 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.932+0000 7fc1f57fa640 1 -- 172.21.15.38:0/138727099 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc208013070 con 0x55e9621a8230 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.932+0000 7fc21f632740 10 client.5602 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.932+0000 7fc21f632740 10 client.5602 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.932+0000 7fc21f632740 20 client.5602 populate_metadata read hostname 'smithi038' 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.932+0000 7fc21f632740 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e9621bb0d0 0x55e9621bb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.932+0000 7fc21f632740 4 client.5602 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.932+0000 7fc21f632740 1 -- 172.21.15.38:0/138727099 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e96213f920 con 0x55e9621bb0d0 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.932+0000 7fc20ffff640 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e9621bb0d0 0x55e9621bb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.932+0000 7fc1f57fa640 10 client.5602 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.932+0000 7fc20ffff640 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e9621bb0d0 0x55e9621bb4b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a1691e7b73051e3 server_cookie=5811be8509a08261 in_seq=0 out_seq=0 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc1f57fa640 1 -- 172.21.15.38:0/138727099 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e96213f920 con 0x55e9621bb0d0 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc1f57fa640 10 client.5602 handle_command_reply: tid=0 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc21f632740 1 client.5602 shutdown 2024-04-25T08:34:53.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc21f632740 1 -- 172.21.15.38:0/138727099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621a8230 msgr2=0x55e9621a98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc21f632740 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621a8230 0x55e9621a98d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc208088a90 tx=0x7fc208006f30 comp rx=0 tx=0).stop 2024-04-25T08:34:53.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc21f632740 1 -- 172.21.15.38:0/138727099 shutdown_connections 2024-04-25T08:34:53.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc21f632740 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e9621bb0d0 0x55e9621bb4b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc21f632740 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 0x55e9621ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc1f57fa640 0 client.5602 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:34:53.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc21f632740 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621c5ad0 0x55e9621ab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc21f632740 1 --2- 172.21.15.38:0/138727099 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621a8230 0x55e9621a98d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc21f632740 1 -- 172.21.15.38:0/138727099 >> 172.21.15.38:0/138727099 conn(0x55e96217df10 msgr2=0x55e9621c65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:53.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc21f632740 1 -- 172.21.15.38:0/138727099 shutdown_connections 2024-04-25T08:34:53.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc21f632740 1 -- 172.21.15.38:0/138727099 wait complete. 2024-04-25T08:34:53.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.933+0000 7fc21f632740 20 client.5602 trim_cache size 0 max 16384 2024-04-25T08:34:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.949+0000 7fc21f632740 1 Processor -- start 2024-04-25T08:34:53.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.949+0000 7fc21f632740 1 -- start start 2024-04-25T08:34:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21f632740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621ab070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21f632740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 0x55e9621ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21f632740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621c5ad0 0x55e9621d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21f632740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2181ff600 con 0x55e9621a8230 2024-04-25T08:34:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21f632740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc218160830 con 0x55e9621c5ad0 2024-04-25T08:34:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21f632740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc218160e30 con 0x55e9621a9e10 2024-04-25T08:34:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52810/0 (socket says 172.21.15.38:52810) 2024-04-25T08:34:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21659c640 1 -- 172.21.15.38:0/1677236382 learned_addr learned my addr 172.21.15.38:0/1677236382 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:53.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21659c640 1 -- 172.21.15.38:0/1677236382 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 msgr2=0x55e9621ab5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21659c640 1 --2- 172.21.15.38:0/1677236382 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 0x55e9621ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21659c640 1 -- 172.21.15.38:0/1677236382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621c5ad0 msgr2=0x55e9621d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21659c640 1 --2- 172.21.15.38:0/1677236382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621c5ad0 0x55e9621d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21659c640 1 -- 172.21.15.38:0/1677236382 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9621e6350 con 0x55e9621a8230 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.950+0000 7fc21659c640 1 --2- 172.21.15.38:0/1677236382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621ab070 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc20808a9c0 tx=0x7fc208015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf635989c9d223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.951+0000 7fc1f77fe640 1 -- 172.21.15.38:0/1677236382 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208074070 con 0x55e9621a8230 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.951+0000 7fc1f77fe640 1 -- 172.21.15.38:0/1677236382 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208058100 con 0x55e9621a8230 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.951+0000 7fc1f77fe640 1 -- 172.21.15.38:0/1677236382 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208088e60 con 0x55e9621a8230 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.951+0000 7fc21f632740 1 -- 172.21.15.38:0/1677236382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 msgr2=0x55e9621ab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.951+0000 7fc21f632740 1 --2- 172.21.15.38:0/1677236382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621ab070 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc20808a9c0 tx=0x7fc208015000 comp rx=0 tx=0).stop 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.951+0000 7fc21f632740 1 -- 172.21.15.38:0/1677236382 shutdown_connections 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.951+0000 7fc21f632740 1 --2- 172.21.15.38:0/1677236382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621c5ad0 0x55e9621d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.951+0000 7fc21f632740 1 --2- 172.21.15.38:0/1677236382 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 0x55e9621ab5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.951+0000 7fc21f632740 1 --2- 172.21.15.38:0/1677236382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621ab070 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.951+0000 7fc21f632740 1 -- 172.21.15.38:0/1677236382 >> 172.21.15.38:0/1677236382 conn(0x55e96217df10 msgr2=0x55e9621bb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:53.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.951+0000 7fc21f632740 1 -- 172.21.15.38:0/1677236382 shutdown_connections 2024-04-25T08:34:53.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.951+0000 7fc21f632740 1 -- 172.21.15.38:0/1677236382 wait complete. 2024-04-25T08:34:53.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.952+0000 7fc21f632740 1 -- start start 2024-04-25T08:34:53.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.952+0000 7fc21f632740 1 Processor -- start 2024-04-25T08:34:53.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21f632740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621ab470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21f632740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 0x55e96222c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21f632740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621c5ad0 0x55e96222cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21f632740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96209d0c0 con 0x55e9621a8230 2024-04-25T08:34:53.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21f632740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e96217ecb0 con 0x55e9621c5ad0 2024-04-25T08:34:53.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21f632740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9621ba190 con 0x55e9621a9e10 2024-04-25T08:34:53.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21f632740 10 client.0 _ll_register_callbacks cb 0x55e962178c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:34:53.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621ab470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52826/0 (socket says 172.21.15.38:52826) 2024-04-25T08:34:53.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21659c640 1 -- 172.21.15.38:0/3283727080 learned_addr learned my addr 172.21.15.38:0/3283727080 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:53.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc216d9d640 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621c5ad0 0x55e96222cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc20ffff640 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 0x55e96222c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21659c640 1 -- 172.21.15.38:0/3283727080 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 msgr2=0x55e96222c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:53.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21659c640 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 0x55e96222c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21659c640 1 -- 172.21.15.38:0/3283727080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621c5ad0 msgr2=0x55e96222cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:53.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21659c640 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621c5ad0 0x55e96222cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21659c640 1 -- 172.21.15.38:0/3283727080 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9621c6530 con 0x55e9621a8230 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc216d9d640 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621c5ad0 0x55e96222cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc21659c640 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621ab470 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc208056f60 tx=0x7fc208055080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35241ed8d86effe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.953+0000 7fc1f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc1f57fa640 1 -- 172.21.15.38:0/3283727080 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208088e60 con 0x55e9621a8230 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc1f57fa640 1 -- 172.21.15.38:0/3283727080 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc208054c00 con 0x55e9621a8230 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc21f632740 1 -- 172.21.15.38:0/3283727080 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc208058100 con 0x55e9621a8230 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc1f57fa640 1 -- 172.21.15.38:0/3283727080 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc208074040 con 0x55e9621a8230 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc21f632740 10 client.6138 fetch_fsmap learned FSMap version 24 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc21f632740 1 -- 172.21.15.38:0/3283727080 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e9621c6530 con 0x55e9621a8230 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc1f57fa640 1 -- 172.21.15.38:0/3283727080 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208058100 con 0x55e9621a8230 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc1f57fa640 1 -- 172.21.15.38:0/3283727080 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc208057b70 con 0x55e9621a8230 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc21f632740 10 client.6138 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:34:53.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc21f632740 10 client.6138 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:34:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc21f632740 20 client.6138 populate_metadata read hostname 'smithi038' 2024-04-25T08:34:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc21f632740 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e9621a8ef0 0x55e9621a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc21f632740 4 client.6138 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:34:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc21f632740 1 -- 172.21.15.38:0/3283727080 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e9621bc600 con 0x55e9621a8ef0 2024-04-25T08:34:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc20ffff640 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e9621a8ef0 0x55e9621a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc1f57fa640 10 client.6138 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:34:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.954+0000 7fc20ffff640 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e9621a8ef0 0x55e9621a92d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8c2b8c3ace3d85 server_cookie=e819e39956f6d570 in_seq=0 out_seq=0 2024-04-25T08:34:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc1f57fa640 1 -- 172.21.15.38:0/3283727080 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e9621bc600 con 0x55e9621a8ef0 2024-04-25T08:34:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc1f57fa640 10 client.6138 handle_command_reply: tid=0 2024-04-25T08:34:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21f632740 1 client.6138 shutdown 2024-04-25T08:34:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21f632740 1 -- 172.21.15.38:0/3283727080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 msgr2=0x55e9621ab470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21f632740 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621ab470 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc208056f60 tx=0x7fc208055080 comp rx=0 tx=0).stop 2024-04-25T08:34:53.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21f632740 1 -- 172.21.15.38:0/3283727080 shutdown_connections 2024-04-25T08:34:53.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21f632740 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e9621a8ef0 0x55e9621a92d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21f632740 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e9621c5ad0 0x55e96222cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc1f57fa640 0 client.6138 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:34:53.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21f632740 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e9621a9e10 0x55e96222c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21f632740 1 --2- 172.21.15.38:0/3283727080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e9621a8230 0x55e9621ab470 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21f632740 1 -- 172.21.15.38:0/3283727080 >> 172.21.15.38:0/3283727080 conn(0x55e96217df10 msgr2=0x55e9621a75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:53.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21f632740 1 -- 172.21.15.38:0/3283727080 shutdown_connections 2024-04-25T08:34:53.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21f632740 1 -- 172.21.15.38:0/3283727080 wait complete. 2024-04-25T08:34:53.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21f632740 20 client.6138 trim_cache size 0 max 16384 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 5964, 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 9.0933693210000008, 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 24, 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 54, 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 54, 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 14.453284964 2024-04-25T08:34:53.959 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:34:53.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21c858640 1 -- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1f806b4d0 msgr2=0x7fc1f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.955+0000 7fc21c858640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1f806b4d0 0x7fc1f806d990 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc200034000 tx=0x7fc20002e000 comp rx=0 tx=0).stop 2024-04-25T08:34:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.956+0000 7fc21c858640 1 -- 172.21.15.38:0/2553162814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2180b92c0 msgr2=0x7fc218152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.956+0000 7fc21c858640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2180b92c0 0x7fc218152b70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fc20800d5f0 tx=0x7fc20800dab0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.956+0000 7fc21c858640 1 -- 172.21.15.38:0/2553162814 shutdown_connections 2024-04-25T08:34:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.956+0000 7fc21c858640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc21815db70 0x7fc2180c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.956+0000 7fc21c858640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1f806b4d0 0x7fc1f806d990 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.956+0000 7fc21c858640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2180ceb20 0x7fc2180c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.956+0000 7fc21c858640 1 --2- 172.21.15.38:0/2553162814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2180b92c0 0x7fc218152b70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.956+0000 7fc21c858640 1 -- 172.21.15.38:0/2553162814 >> 172.21.15.38:0/2553162814 conn(0x7fc21806bed0 msgr2=0x7fc2180cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.956+0000 7fc21c858640 1 -- 172.21.15.38:0/2553162814 shutdown_connections 2024-04-25T08:34:53.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:53.956+0000 7fc21c858640 1 -- 172.21.15.38:0/2553162814 wait complete. 2024-04-25T08:34:53.963 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:34:54.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.068+0000 7f53b625d640 1 Processor -- start 2024-04-25T08:34:54.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.068+0000 7f53b625d640 1 -- start start 2024-04-25T08:34:54.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53b625d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f53b006abe0 0x7f53b006c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53b625d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f53b015e5c0 0x7f53b015e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53b625d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b006b0a0 con 0x7f53b00b2d60 2024-04-25T08:34:54.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53b625d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b006b380 con 0x7f53b015e5c0 2024-04-25T08:34:54.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53b625d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b006b660 con 0x7f53b006abe0 2024-04-25T08:34:54.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53af7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f53b00b2d60 0x7f53b00b3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54278/0 (socket says 172.21.15.38:54278) 2024-04-25T08:34:54.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53af7fe640 1 -- 172.21.15.38:0/2592328415 learned_addr learned my addr 172.21.15.38:0/2592328415 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:34:54.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53affff640 1 --2- 172.21.15.38:0/2592328415 >> v2:172.21.15.38:3300/0 conn(0x7f53b015e5c0 0x7f53b015e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53ae7fc640 1 -- 172.21.15.38:0/2592328415 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144079193 0 0) 0x7f53b006b0a0 con 0x7f53b00b2d60 2024-04-25T08:34:54.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53aeffd640 1 --2- 172.21.15.38:0/2592328415 >> v2:172.21.15.107:3300/0 conn(0x7f53b006abe0 0x7f53b006c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53ae7fc640 1 -- 172.21.15.38:0/2592328415 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5398003520 con 0x7f53b00b2d60 2024-04-25T08:34:54.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53affff640 1 -- 172.21.15.38:0/2592328415 >> v2:172.21.15.107:3300/0 conn(0x7f53b006abe0 msgr2=0x7f53b006c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:34:54.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53affff640 1 --2- 172.21.15.38:0/2592328415 >> v2:172.21.15.107:3300/0 conn(0x7f53b006abe0 0x7f53b006c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53affff640 1 -- 172.21.15.38:0/2592328415 >> v1:172.21.15.38:6789/0 conn(0x7f53b00b2d60 legacy=0x7f53b00b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.069+0000 7f53affff640 1 -- 172.21.15.38:0/2592328415 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b006b940 con 0x7f53b015e5c0 2024-04-25T08:34:54.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.070+0000 7f53affff640 1 --2- 172.21.15.38:0/2592328415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53b015e5c0 0x7f53b015e9e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f53b0071500 tx=0x7f53a802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b77fe7ef7ec4431 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.070+0000 7f53ae7fc640 1 -- 172.21.15.38:0/2592328415 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8007740 con 0x7f53b015e5c0 2024-04-25T08:34:54.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.070+0000 7f53ae7fc640 1 -- 172.21.15.38:0/2592328415 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a80078e0 con 0x7f53b015e5c0 2024-04-25T08:34:54.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.070+0000 7f53ae7fc640 1 -- 172.21.15.38:0/2592328415 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8007cb0 con 0x7f53b015e5c0 2024-04-25T08:34:54.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.071+0000 7f53b625d640 1 -- 172.21.15.38:0/2592328415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53b015e5c0 msgr2=0x7f53b015e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.071+0000 7f53b625d640 1 --2- 172.21.15.38:0/2592328415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53b015e5c0 0x7f53b015e9e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f53b0071500 tx=0x7f53a802a680 comp rx=0 tx=0).stop 2024-04-25T08:34:54.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.071+0000 7f53b625d640 1 -- 172.21.15.38:0/2592328415 shutdown_connections 2024-04-25T08:34:54.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.071+0000 7f53b625d640 1 --2- 172.21.15.38:0/2592328415 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53b015e5c0 0x7f53b015e9e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.071+0000 7f53b625d640 1 --2- 172.21.15.38:0/2592328415 >> v2:172.21.15.107:3300/0 conn(0x7f53b006abe0 0x7f53b006c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.071+0000 7f53b625d640 1 -- 172.21.15.38:0/2592328415 >> 172.21.15.38:0/2592328415 conn(0x7f53b00cfd30 msgr2=0x7f53b00d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.071+0000 7f53b625d640 1 -- 172.21.15.38:0/2592328415 shutdown_connections 2024-04-25T08:34:54.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.071+0000 7f53b625d640 1 -- 172.21.15.38:0/2592328415 wait complete. 2024-04-25T08:34:54.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.072+0000 7f53b625d640 1 Processor -- start 2024-04-25T08:34:54.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.072+0000 7f53b625d640 1 -- start start 2024-04-25T08:34:54.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53b625d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53b006abe0 0x7f53b01f6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53b625d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53b00b2d60 0x7f53b01f0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53b625d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53b015e5c0 0x7f53b01f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53b625d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b015f1b0 con 0x7f53b006abe0 2024-04-25T08:34:54.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53b625d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b015f030 con 0x7f53b015e5c0 2024-04-25T08:34:54.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53b625d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b015f330 con 0x7f53b00b2d60 2024-04-25T08:34:54.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53b006abe0 0x7f53b01f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53b006abe0 0x7f53b01f6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52842/0 (socket says 172.21.15.38:52842) 2024-04-25T08:34:54.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53af7fe640 1 -- 172.21.15.38:0/285848925 learned_addr learned my addr 172.21.15.38:0/285848925 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:34:54.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53aeffd640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53b00b2d60 0x7f53b01f0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53affff640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53b015e5c0 0x7f53b01f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53af7fe640 1 -- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53b00b2d60 msgr2=0x7f53b01f0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53af7fe640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53b00b2d60 0x7f53b01f0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53af7fe640 1 -- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53b015e5c0 msgr2=0x7f53b01f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53af7fe640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53b015e5c0 0x7f53b01f3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53af7fe640 1 -- 172.21.15.38:0/285848925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b01f3370 con 0x7f53b006abe0 2024-04-25T08:34:54.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53affff640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53b015e5c0 0x7f53b01f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:34:54.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.073+0000 7f53af7fe640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53b006abe0 0x7f53b01f6950 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c430 tx=0x7f53a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.074+0000 7f5397fff640 1 -- 172.21.15.38:0/285848925 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0017070 con 0x7f53b006abe0 2024-04-25T08:34:54.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.074+0000 7f5397fff640 1 -- 172.21.15.38:0/285848925 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a0005ce0 con 0x7f53b006abe0 2024-04-25T08:34:54.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.074+0000 7f5397fff640 1 -- 172.21.15.38:0/285848925 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a00072f0 con 0x7f53b006abe0 2024-04-25T08:34:54.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.074+0000 7f53b625d640 1 -- 172.21.15.38:0/285848925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b01f6e90 con 0x7f53b006abe0 2024-04-25T08:34:54.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.074+0000 7f53b625d640 1 -- 172.21.15.38:0/285848925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b01f7130 con 0x7f53b006abe0 2024-04-25T08:34:54.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.075+0000 7f53b625d640 1 -- 172.21.15.38:0/285848925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b0068e80 con 0x7f53b006abe0 2024-04-25T08:34:54.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.075+0000 7f5397fff640 1 -- 172.21.15.38:0/285848925 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53a001a070 con 0x7f53b006abe0 2024-04-25T08:34:54.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.075+0000 7f5397fff640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f539c06b4d0 0x7f539c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:34:54.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.076+0000 7f5397fff640 1 -- 172.21.15.38:0/285848925 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53a0089870 con 0x7f53b006abe0 2024-04-25T08:34:54.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.076+0000 7f53aeffd640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f539c06b4d0 0x7f539c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:34:54.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.076+0000 7f53aeffd640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f539c06b4d0 0x7f539c06d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f53b015e300 tx=0x7f53a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:34:54.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.080+0000 7f5397fff640 1 -- 172.21.15.38:0/285848925 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53a0058560 con 0x7f53b006abe0 2024-04-25T08:34:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.300+0000 7f53b625d640 1 -- 172.21.15.38:0/285848925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f53b015ece0 con 0x7f53b006abe0 2024-04-25T08:34:54.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.301+0000 7f5397fff640 1 -- 172.21.15.38:0/285848925 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f53a0057e60 con 0x7f53b006abe0 2024-04-25T08:34:54.297 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:34:54.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.303+0000 7f53b625d640 1 -- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f539c06b4d0 msgr2=0x7f539c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.303+0000 7f53b625d640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f539c06b4d0 0x7f539c06d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f53b015e300 tx=0x7f53a4008040 comp rx=0 tx=0).stop 2024-04-25T08:34:54.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.303+0000 7f53b625d640 1 -- 172.21.15.38:0/285848925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53b006abe0 msgr2=0x7f53b01f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:34:54.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.303+0000 7f53b625d640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53b006abe0 0x7f53b01f6950 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f53a000c430 tx=0x7f53a000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.303+0000 7f53b625d640 1 -- 172.21.15.38:0/285848925 shutdown_connections 2024-04-25T08:34:54.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.303+0000 7f53b625d640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53b015e5c0 0x7f53b01f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.303+0000 7f53b625d640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f539c06b4d0 0x7f539c06d990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.303+0000 7f53b625d640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53b00b2d60 0x7f53b01f0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.303+0000 7f53b625d640 1 --2- 172.21.15.38:0/285848925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53b006abe0 0x7f53b01f6950 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:34:54.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.303+0000 7f53b625d640 1 -- 172.21.15.38:0/285848925 >> 172.21.15.38:0/285848925 conn(0x7f53b00cfd30 msgr2=0x7f53b015d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:34:54.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.303+0000 7f53b625d640 1 -- 172.21.15.38:0/285848925 shutdown_connections 2024-04-25T08:34:54.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:54.303+0000 7f53b625d640 1 -- 172.21.15.38:0/285848925 wait complete. 2024-04-25T08:34:54.310 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:34:54.310 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:34:54.310 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:34:54.310 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:34:54.310 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:34:54.310 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:34:54.310 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:34:54.310 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:34:54.311 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:34:54.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:54.311 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:34:54.327 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:34:54.327 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:34:54.327 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:34:54.327 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:54.327 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:34:54.327 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:54.328 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:34:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:34:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:34:54.328 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:34:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:34:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:34:54.328 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:34:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:34:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:34:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:34:54.328 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42543sec preferred_lft 42543sec 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::440f:97ff:fe63:87b/64 scope link 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:34:54.329 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ea:23:76:ab:b1:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::e823:76ff:feab:b1e9/64 scope link 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::38c7:38ff:fe2e:6c92/64 scope link 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:34:54.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:54.331 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:34:54.391 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:34:54.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:54.392 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:34:54.448 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:34:54.449 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:34:54.449 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:34:54.449 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:34:54.449 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:34:54.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:54.449 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:34:54.449 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:34:54.449 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:34:54.450 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:34:54.450 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:34:54.450 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:34:54.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:54.535 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:34:54.550 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:34:54.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:54.551 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T08:34:54.618 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:34:54.618 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:34:54.619 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:34:54.675 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:34:54.675 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:34:54.675 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:34:54.729 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:34:54.794 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:34:54.851 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:34:54.851 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:34:54.851 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:34:54.851 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:34:54.851 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:34:54.851 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:34:54.851 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:34:54.851 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:34:54.851 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:34:54.852 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:34:54.853 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:34:54.853 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:34:54.853 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:34:54.853 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:34:54.853 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:34:54.853 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:34:54.853 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:34:54.853 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:34:54.853 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:34:54.853 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:34:54.853 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:34:54.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:54.854 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:34:54.907 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:34:54.907 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:34:54.950 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:34:54.974 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:34:55.031 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:34:55.031 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:34:55.031 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:34:55.031 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:34:55.031 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:34:55.031 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:34:55.031 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:34:55.031 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:34:55.032 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:34:55.033 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:34:55.034 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-04-25T08:34:55.035+0000 7f74c8444480 -1 init, newargv = 0x564fe45b5320 newargc=15 2024-04-25T08:34:55.034 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[60477]: starting ceph client 2024-04-25T08:34:55.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:55.035 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:34:55.036 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[60477]: starting fuse 2024-04-25T08:34:55.091 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:34:55.091 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:34:55.092 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:34:55.092 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:34:55.092 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:34:55.092 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:34:55.092 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:34:55.092 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:34:55.092 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:34:55.092 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:34:55.092 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:34:55.092 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:34:55.092 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-04-25T08:34:55.093 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:34:55.211 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.60477.asok 2024-04-25T08:34:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:34:55.221 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.60477.asok 2024-04-25T08:34:55.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:55.221 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.60477.asok status 2024-04-25T08:34:55.357 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:34:55.357 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "60477", 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 6153, 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:34:55.358 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 6153 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1156583766 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1156583766 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.6153 192.168.0.1:0/1156583766", 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/1156583766", 2024-04-25T08:34:55.359 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:34:55.360 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 24, 2024-04-25T08:34:55.360 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 54, 2024-04-25T08:34:55.360 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:34:55.360 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:34:55.360 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:34:55.360 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:34:55.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:55.366 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:34:55.366 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:34:55.381 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:34:55.381 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:34:55.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:55.381 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:34:55.446 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T08:34:55.447 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T08:34:55.447 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T08:34:55.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:55.447 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> import json 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> import sys 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> try: 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> except OSError as e: 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> sys.exit(e.errno) 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T08:34:55.507 DEBUG:teuthology.orchestra.run.smithi038:> print(json.dumps( 2024-04-25T08:34:55.508 DEBUG:teuthology.orchestra.run.smithi038:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T08:34:55.508 DEBUG:teuthology.orchestra.run.smithi038:> indent=2)) 2024-04-25T08:34:55.508 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:34:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:34:55 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T08:34:55.624 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T08:34:55.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:34:55.625 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-25T08:34:55.686 INFO:teuthology.orchestra.run.smithi038.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-25T08:34:55.929 INFO:teuthology.orchestra.run.smithi038.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-25T08:35:02.968 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:35:02.971 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:35:03.153 INFO:teuthology.orchestra.run.smithi038.stderr:Updating files: 22% (317/1398) Updating files: 23% (322/1398) Updating files: 24% (336/1398) Updating files: 25% (350/1398) Updating files: 26% (364/1398) Updating files: 27% (378/1398) Updating files: 28% (392/1398) Updating files: 29% (406/1398) Updating files: 30% (420/1398) Updating files: 31% (434/1398) Updating files: 32% (448/1398) Updating files: 33% (462/1398) Updating files: 34% (476/1398) Updating files: 35% (490/1398) Updating files: 36% (504/1398) Updating files: 37% (518/1398) Updating files: 38% (532/1398) Updating files: 39% (546/1398) Updating files: 40% (560/1398) Updating files: 41% (574/1398) Updating files: 42% (588/1398) Updating files: 43% (602/1398) Updating files: 44% (616/1398) Updating files: 45% (630/1398) Updating files: 45% (631/1398) Updating files: 46% (644/1398) Updating files: 47% (658/1398) Updating files: 48% (672/1398) Updating files: 49% (686/1398) Updating files: 50% (699/1398) Updating files: 51% (713/1398) Updating files: 52% (727/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 65% (921/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 81% (1141/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-25T08:35:03.162 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-25T08:35:03.162 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T08:35:03.162 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:03.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.309+0000 7f84a978b640 1 Processor -- start 2024-04-25T08:35:03.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.309+0000 7f84a978b640 1 -- start start 2024-04-25T08:35:03.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a978b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a415dbc0 0x7f84a4072b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a978b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a415dfa0 con 0x7f84a40b2220 2024-04-25T08:35:03.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a978b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84a406ba00 con 0x7f84a40ca920 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a978b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a40b3970 con 0x7f84a415dbc0 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f849bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f84a40b2220 0x7f84a406abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60594/0 (socket says 172.21.15.38:60594) 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f849bfff640 1 -- 172.21.15.38:0/595455424 learned_addr learned my addr 172.21.15.38:0/595455424 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a2ffd640 1 --2- 172.21.15.38:0/595455424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a415dbc0 0x7f84a4072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2444202684 0 0) 0x7f84a415dfa0 con 0x7f84a40b2220 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8490003520 con 0x7f84a40b2220 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 946345884 0 0) 0x7f8490003520 con 0x7f84a40b2220 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a415dbc0 msgr2=0x7f84a4072b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 --2- 172.21.15.38:0/595455424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a415dbc0 0x7f84a4072b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 >> v1:172.21.15.107:6789/0 conn(0x7f84a40ca920 legacy=0x7f84a406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a40b3c50 con 0x7f84a40b2220 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f848c001670 con 0x7f84a40b2220 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 >> v1:172.21.15.38:6789/0 conn(0x7f84a40b2220 legacy=0x7f84a406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 --2- 172.21.15.38:0/595455424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a415dbc0 0x7f84900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 --2- 172.21.15.38:0/595455424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8490003f20 0x7f8490004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 --2- 172.21.15.38:0/595455424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8490004d40 0x7f84900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a40b3970 con 0x7f8490003f20 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849000ba20 con 0x7f8490004d40 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.310+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849000bc20 con 0x7f84a415dbc0 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.311+0000 7f849bfff640 1 --2- 172.21.15.38:0/595455424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8490003f20 0x7f8490004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.311+0000 7f84a2ffd640 1 --2- 172.21.15.38:0/595455424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a415dbc0 0x7f84900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.311+0000 7f84a37fe640 1 --2- 172.21.15.38:0/595455424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8490004d40 0x7f84900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.311+0000 7f849bfff640 1 -- 172.21.15.38:0/595455424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a415dbc0 msgr2=0x7f84900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.311+0000 7f849bfff640 1 --2- 172.21.15.38:0/595455424 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a415dbc0 0x7f84900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.311+0000 7f849bfff640 1 -- 172.21.15.38:0/595455424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8490004d40 msgr2=0x7f84900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.311+0000 7f849bfff640 1 --2- 172.21.15.38:0/595455424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8490004d40 0x7f84900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.311+0000 7f849bfff640 1 -- 172.21.15.38:0/595455424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8490003520 con 0x7f8490003f20 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.312+0000 7f849bfff640 1 --2- 172.21.15.38:0/595455424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8490003f20 0x7f8490004800 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f848c00c290 tx=0x7f8490005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c467d6d7e900b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.312+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c007d50 con 0x7f8490003f20 2024-04-25T08:35:03.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.312+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c005af0 con 0x7f8490003f20 2024-04-25T08:35:03.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.312+0000 7f84a27fc640 1 -- 172.21.15.38:0/595455424 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c007070 con 0x7f8490003f20 2024-04-25T08:35:03.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.312+0000 7f84a978b640 1 -- 172.21.15.38:0/595455424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8490003f20 msgr2=0x7f8490004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.312+0000 7f84a978b640 1 --2- 172.21.15.38:0/595455424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8490003f20 0x7f8490004800 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f848c00c290 tx=0x7f8490005830 comp rx=0 tx=0).stop 2024-04-25T08:35:03.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.312+0000 7f84a37fe640 1 -- 172.21.15.38:0/595455424 reap_dead start 2024-04-25T08:35:03.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.312+0000 7f84a978b640 1 -- 172.21.15.38:0/595455424 shutdown_connections 2024-04-25T08:35:03.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.312+0000 7f84a978b640 1 -- 172.21.15.38:0/595455424 >> 172.21.15.38:0/595455424 conn(0x7f84a40b5c90 msgr2=0x7f84a40b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:03.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.312+0000 7f84a978b640 1 -- 172.21.15.38:0/595455424 shutdown_connections 2024-04-25T08:35:03.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.312+0000 7f84a978b640 1 -- 172.21.15.38:0/595455424 wait complete. 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.313+0000 7f84a978b640 1 Processor -- start 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.313+0000 7f84a978b640 1 -- start start 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.314+0000 7f84a978b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8490003f20 0x7f84a4142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.314+0000 7f84a978b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a4145d50 0x7f84a4143010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.314+0000 7f84a978b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a41609c0 0x7f84a4072110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.314+0000 7f84a978b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a40b44f0 con 0x7f8490003f20 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.314+0000 7f84a978b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a40b4210 con 0x7f84a4145d50 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.314+0000 7f84a978b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84a40b3590 con 0x7f84a41609c0 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.314+0000 7f84a2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8490003f20 0x7f84a4142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.314+0000 7f84a2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8490003f20 0x7f84a4142950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42530/0 (socket says 172.21.15.38:42530) 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.314+0000 7f84a2ffd640 1 -- 172.21.15.38:0/2132643418 learned_addr learned my addr 172.21.15.38:0/2132643418 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.314+0000 7f849bfff640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a4145d50 0x7f84a4143010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.314+0000 7f84a37fe640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a41609c0 0x7f84a4072110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.315+0000 7f84a2ffd640 1 -- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a41609c0 msgr2=0x7f84a4072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.315+0000 7f84a2ffd640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a41609c0 0x7f84a4072110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.315+0000 7f84a2ffd640 1 -- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a4145d50 msgr2=0x7f84a4143010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.315+0000 7f84a2ffd640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a4145d50 0x7f84a4143010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.315+0000 7f84a2ffd640 1 -- 172.21.15.38:0/2132643418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a40ceb80 con 0x7f8490003f20 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.315+0000 7f84a2ffd640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8490003f20 0x7f84a4142950 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f84940119a0 tx=0x7f8494011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.315+0000 7f849b7fe640 1 -- 172.21.15.38:0/2132643418 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849400faf0 con 0x7f8490003f20 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.315+0000 7f849b7fe640 1 -- 172.21.15.38:0/2132643418 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849400fc90 con 0x7f8490003f20 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.315+0000 7f84a978b640 1 -- 172.21.15.38:0/2132643418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a415e140 con 0x7f8490003f20 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.315+0000 7f849b7fe640 1 -- 172.21.15.38:0/2132643418 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8494014440 con 0x7f8490003f20 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.315+0000 7f84a978b640 1 -- 172.21.15.38:0/2132643418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a40cee20 con 0x7f8490003f20 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.316+0000 7f84a978b640 1 -- 172.21.15.38:0/2132643418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a4068e80 con 0x7f8490003f20 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.317+0000 7f849b7fe640 1 -- 172.21.15.38:0/2132643418 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8494012040 con 0x7f8490003f20 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.317+0000 7f849b7fe640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f848406b500 0x7f848406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.317+0000 7f849b7fe640 1 -- 172.21.15.38:0/2132643418 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f849408a840 con 0x7f8490003f20 2024-04-25T08:35:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.317+0000 7f849bfff640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f848406b500 0x7f848406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.318+0000 7f849bfff640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f848406b500 0x7f848406d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f848c032000 tx=0x7f848c034040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.321+0000 7f849b7fe640 1 -- 172.21.15.38:0/2132643418 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8494019030 con 0x7f8490003f20 2024-04-25T08:35:03.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.541+0000 7f84a978b640 1 -- 172.21.15.38:0/2132643418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84a40cc460 con 0x7f8490003f20 2024-04-25T08:35:03.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.542+0000 7f849b7fe640 1 -- 172.21.15.38:0/2132643418 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f8494058f70 con 0x7f8490003f20 2024-04-25T08:35:03.537 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:03.537 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:34:46.222808+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:03.537 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 24 2024-04-25T08:35:03.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.544+0000 7f84a978b640 1 -- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f848406b500 msgr2=0x7f848406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.544+0000 7f84a978b640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f848406b500 0x7f848406d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f848c032000 tx=0x7f848c034040 comp rx=0 tx=0).stop 2024-04-25T08:35:03.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.544+0000 7f84a978b640 1 -- 172.21.15.38:0/2132643418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8490003f20 msgr2=0x7f84a4142950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.544+0000 7f84a978b640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8490003f20 0x7f84a4142950 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f84940119a0 tx=0x7f8494011e60 comp rx=0 tx=0).stop 2024-04-25T08:35:03.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.544+0000 7f84a978b640 1 -- 172.21.15.38:0/2132643418 shutdown_connections 2024-04-25T08:35:03.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.544+0000 7f84a978b640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f848406b500 0x7f848406d9c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.544+0000 7f84a978b640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f84a41609c0 0x7f84a4072110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.544+0000 7f84a978b640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84a4145d50 0x7f84a4143010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.544+0000 7f84a978b640 1 --2- 172.21.15.38:0/2132643418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8490003f20 0x7f84a4142950 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.544+0000 7f84a978b640 1 -- 172.21.15.38:0/2132643418 >> 172.21.15.38:0/2132643418 conn(0x7f84a40b5c90 msgr2=0x7f84a40b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:03.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.544+0000 7f84a978b640 1 -- 172.21.15.38:0/2132643418 shutdown_connections 2024-04-25T08:35:03.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.544+0000 7f84a978b640 1 -- 172.21.15.38:0/2132643418 wait complete. 2024-04-25T08:35:03.549 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T08:35:03.655 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:03.656 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": -2 2024-04-25T08:35:03.656 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:03.664 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T08:35:03.664 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T08:35:03.664 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:03.664 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T08:35:03.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.774+0000 7f7d09e3a640 1 Processor -- start 2024-04-25T08:35:03.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.774+0000 7f7d09e3a640 1 -- start start 2024-04-25T08:35:03.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.774+0000 7f7d09e3a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d040d0320 0x7f7d040d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.774+0000 7f7d09e3a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7d040d0700 0x7f7d040d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.774+0000 7f7d09e3a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d040d2330 con 0x7f7d040d1e90 2024-04-25T08:35:03.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.774+0000 7f7d09e3a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d040d2610 con 0x7f7d040d0700 2024-04-25T08:35:03.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.774+0000 7f7d09e3a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040d28f0 con 0x7f7d040d0320 2024-04-25T08:35:03.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.775+0000 7f7d03fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7d040d1e90 0x7f7d040b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60604/0 (socket says 172.21.15.38:60604) 2024-04-25T08:35:03.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.775+0000 7f7d03fff640 1 -- 172.21.15.38:0/3668525381 learned_addr learned my addr 172.21.15.38:0/3668525381 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:03.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.775+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/3668525381 >> v2:172.21.15.107:3300/0 conn(0x7f7d040d0700 0x7f7d040d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.775+0000 7f7d037fe640 1 --2- 172.21.15.38:0/3668525381 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d040d0320 0x7f7d040d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.775+0000 7f7d027fc640 1 -- 172.21.15.38:0/3668525381 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841391805 0 0) 0x7f7d040d2330 con 0x7f7d040d1e90 2024-04-25T08:35:03.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.775+0000 7f7d027fc640 1 -- 172.21.15.38:0/3668525381 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cec003520 con 0x7f7d040d1e90 2024-04-25T08:35:03.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.775+0000 7f7d02ffd640 1 -- 172.21.15.38:0/3668525381 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d040d0320 msgr2=0x7f7d040d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.775+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/3668525381 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d040d0320 0x7f7d040d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.775+0000 7f7d02ffd640 1 -- 172.21.15.38:0/3668525381 >> v1:172.21.15.38:6789/0 conn(0x7f7d040d1e90 legacy=0x7f7d040b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.775+0000 7f7d02ffd640 1 -- 172.21.15.38:0/3668525381 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d040d2bd0 con 0x7f7d040d0700 2024-04-25T08:35:03.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.776+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/3668525381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d040d0700 0x7f7d040d1950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80097a0 tx=0x7f7cf802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ea96b740ef60e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.776+0000 7f7d027fc640 1 -- 172.21.15.38:0/3668525381 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8037070 con 0x7f7d040d0700 2024-04-25T08:35:03.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.776+0000 7f7d027fc640 1 -- 172.21.15.38:0/3668525381 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf8032440 con 0x7f7d040d0700 2024-04-25T08:35:03.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.776+0000 7f7d027fc640 1 -- 172.21.15.38:0/3668525381 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8032810 con 0x7f7d040d0700 2024-04-25T08:35:03.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.776+0000 7f7d09e3a640 1 -- 172.21.15.38:0/3668525381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d040d0700 msgr2=0x7f7d040d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.776+0000 7f7d09e3a640 1 --2- 172.21.15.38:0/3668525381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d040d0700 0x7f7d040d1950 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80097a0 tx=0x7f7cf802a210 comp rx=0 tx=0).stop 2024-04-25T08:35:03.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.776+0000 7f7d09e3a640 1 -- 172.21.15.38:0/3668525381 shutdown_connections 2024-04-25T08:35:03.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.776+0000 7f7d09e3a640 1 --2- 172.21.15.38:0/3668525381 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d040d0700 0x7f7d040d1950 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.776+0000 7f7d09e3a640 1 --2- 172.21.15.38:0/3668525381 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d040d0320 0x7f7d040d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.776+0000 7f7d09e3a640 1 -- 172.21.15.38:0/3668525381 >> 172.21.15.38:0/3668525381 conn(0x7f7d040668f0 msgr2=0x7f7d04066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:03.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.776+0000 7f7d09e3a640 1 -- 172.21.15.38:0/3668525381 shutdown_connections 2024-04-25T08:35:03.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.776+0000 7f7d09e3a640 1 -- 172.21.15.38:0/3668525381 wait complete. 2024-04-25T08:35:03.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.778+0000 7f7d09e3a640 1 Processor -- start 2024-04-25T08:35:03.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.778+0000 7f7d09e3a640 1 -- start start 2024-04-25T08:35:03.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.778+0000 7f7d09e3a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d040d0320 0x7f7d0414ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.778+0000 7f7d09e3a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d040d0700 0x7f7d041e7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.778+0000 7f7d09e3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d040d1e90 0x7f7d040cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.778+0000 7f7d09e3a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d040d0b40 con 0x7f7d040d0700 2024-04-25T08:35:03.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.778+0000 7f7d09e3a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d04161210 con 0x7f7d040d1e90 2024-04-25T08:35:03.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.778+0000 7f7d09e3a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d041e8f40 con 0x7f7d040d0320 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d02ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d040d0700 0x7f7d041e7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d02ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d040d0700 0x7f7d041e7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42538/0 (socket says 172.21.15.38:42538) 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d02ffd640 1 -- 172.21.15.38:0/433630156 learned_addr learned my addr 172.21.15.38:0/433630156 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d037fe640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d040d0320 0x7f7d0414ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d03fff640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d040d1e90 0x7f7d040cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d02ffd640 1 -- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d040d0320 msgr2=0x7f7d0414ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d040d0320 0x7f7d0414ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d02ffd640 1 -- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d040d1e90 msgr2=0x7f7d040cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d040d1e90 0x7f7d040cc9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d02ffd640 1 -- 172.21.15.38:0/433630156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d041e86a0 con 0x7f7d040d0700 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d040d0700 0x7f7d041e7c50 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80098c0 tx=0x7f7cf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7cebfff640 1 -- 172.21.15.38:0/433630156 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8041030 con 0x7f7d040d0700 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7cebfff640 1 -- 172.21.15.38:0/433630156 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf803d4e0 con 0x7f7d040d0700 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d09e3a640 1 -- 172.21.15.38:0/433630156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d041e8980 con 0x7f7d040d0700 2024-04-25T08:35:03.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7cebfff640 1 -- 172.21.15.38:0/433630156 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8037040 con 0x7f7d040d0700 2024-04-25T08:35:03.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.779+0000 7f7d09e3a640 1 -- 172.21.15.38:0/433630156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0415d2c0 con 0x7f7d040d0700 2024-04-25T08:35:03.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.781+0000 7f7cebfff640 1 -- 172.21.15.38:0/433630156 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7cf8044070 con 0x7f7d040d0700 2024-04-25T08:35:03.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.781+0000 7f7cebfff640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7cf006b500 0x7f7cf006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.782+0000 7f7d037fe640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7cf006b500 0x7f7cf006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.782+0000 7f7cebfff640 1 -- 172.21.15.38:0/433630156 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7cf80aa790 con 0x7f7d040d0700 2024-04-25T08:35:03.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.782+0000 7f7d037fe640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7cf006b500 0x7f7cf006d9c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7d041e9110 tx=0x7f7cf4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.783+0000 7f7d0cc10740 1 Processor -- start 2024-04-25T08:35:03.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.783+0000 7f7d0cc10740 1 -- start start 2024-04-25T08:35:03.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.783+0000 7f7d0cc10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c9436b0 0x56355c943a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d0cc10740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 0x56355c9614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d0cc10740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c919390 0x56355c95f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d0cc10740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56355c87bc60 con 0x56355c919390 2024-04-25T08:35:03.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d0cc10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56355c824ee0 con 0x56355c9436b0 2024-04-25T08:35:03.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d0cc10740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56355c877d00 con 0x56355c960f50 2024-04-25T08:35:03.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c919390 0x56355c95f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c919390 0x56355c95f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42542/0 (socket says 172.21.15.38:42542) 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d037fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c9436b0 0x56355c943a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d03fff640 1 -- 172.21.15.38:0/2671923971 learned_addr learned my addr 172.21.15.38:0/2671923971 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/2671923971 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 0x56355c9614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d03fff640 1 -- 172.21.15.38:0/2671923971 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 msgr2=0x56355c9614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d03fff640 1 --2- 172.21.15.38:0/2671923971 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 0x56355c9614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d03fff640 1 -- 172.21.15.38:0/2671923971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c9436b0 msgr2=0x56355c943a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d03fff640 1 --2- 172.21.15.38:0/2671923971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c9436b0 0x56355c943a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d03fff640 1 -- 172.21.15.38:0/2671923971 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56355c65c9f0 con 0x56355c919390 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d037fe640 1 --2- 172.21.15.38:0/2671923971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c9436b0 0x56355c943a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7d03fff640 1 --2- 172.21.15.38:0/2671923971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c919390 0x56355c95f610 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc001fc0 tx=0x7f7cfc007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12e4671456ee22f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7ce97fa640 1 -- 172.21.15.38:0/2671923971 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0133f0 con 0x56355c919390 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7ce97fa640 1 -- 172.21.15.38:0/2671923971 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc018070 con 0x56355c919390 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.784+0000 7f7ce97fa640 1 -- 172.21.15.38:0/2671923971 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0137b0 con 0x56355c919390 2024-04-25T08:35:03.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.785+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2671923971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c919390 msgr2=0x56355c95f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.785+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/2671923971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c919390 0x56355c95f610 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc001fc0 tx=0x7f7cfc007410 comp rx=0 tx=0).stop 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.785+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2671923971 shutdown_connections 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.785+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/2671923971 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c919390 0x56355c95f610 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.785+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/2671923971 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 0x56355c9614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.785+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/2671923971 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c9436b0 0x56355c943a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.785+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2671923971 >> 172.21.15.38:0/2671923971 conn(0x56355c915cc0 msgr2=0x56355c95e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.785+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2671923971 shutdown_connections 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.785+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2671923971 wait complete. 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.786+0000 7f7d0cc10740 1 -- start start 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.786+0000 7f7d0cc10740 1 Processor -- start 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.786+0000 7f7d0cc10740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 0x56355c96f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.786+0000 7f7d0cc10740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 0x56355c956f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.786+0000 7f7d0cc10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c95cf00 0x56355c955570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.786+0000 7f7d0cc10740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56355c91a130 con 0x56355c9436b0 2024-04-25T08:35:03.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.786+0000 7f7d0cc10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56355c838470 con 0x56355c95cf00 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.786+0000 7f7d0cc10740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56355c913bf0 con 0x56355c960f50 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d0cc10740 10 client.0 _ll_register_callbacks cb 0x56355c8047d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 0x56355c96f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 0x56355c96f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42552/0 (socket says 172.21.15.38:42552) 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d037fe640 1 -- 172.21.15.38:0/3696402728 learned_addr learned my addr 172.21.15.38:0/3696402728 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 0x56355c956f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d03fff640 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c95cf00 0x56355c955570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d037fe640 1 -- 172.21.15.38:0/3696402728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 msgr2=0x56355c956f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d037fe640 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 0x56355c956f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d037fe640 1 -- 172.21.15.38:0/3696402728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c95cf00 msgr2=0x56355c955570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d037fe640 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c95cf00 0x56355c955570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d037fe640 1 -- 172.21.15.38:0/3696402728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56355c5502e0 con 0x56355c9436b0 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d037fe640 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 0x56355c96f450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4003640 tx=0x7f7cf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a6af0aabc4882 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7ce27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7ce27fc640 1 -- 172.21.15.38:0/3696402728 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4027070 con 0x56355c9436b0 2024-04-25T08:35:03.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7ce27fc640 1 -- 172.21.15.38:0/3696402728 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7cf4014440 con 0x56355c9436b0 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d0cc10740 1 -- 172.21.15.38:0/3696402728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cfc018070 con 0x56355c9436b0 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7ce27fc640 1 -- 172.21.15.38:0/3696402728 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cf4014640 con 0x56355c9436b0 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d0cc10740 10 client.6171 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.787+0000 7f7d0cc10740 1 -- 172.21.15.38:0/3696402728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56355c6bd180 con 0x56355c9436b0 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.788+0000 7f7ce27fc640 1 -- 172.21.15.38:0/3696402728 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc018070 con 0x56355c9436b0 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.788+0000 7f7ce27fc640 1 -- 172.21.15.38:0/3696402728 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7cf4035080 con 0x56355c9436b0 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.788+0000 7f7d0cc10740 10 client.6171 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.788+0000 7f7d0cc10740 10 client.6171 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.788+0000 7f7d0cc10740 20 client.6171 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.788+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56355c955ab0 0x56355c912920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.788+0000 7f7d0cc10740 4 client.6171 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.788+0000 7f7d0cc10740 1 -- 172.21.15.38:0/3696402728 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56355c8891a0 con 0x56355c955ab0 2024-04-25T08:35:03.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.788+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56355c955ab0 0x56355c912920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.788+0000 7f7ce27fc640 10 client.6171 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.788+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56355c955ab0 0x56355c912920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ae953434d0e09ae server_cookie=229b89bf987376a7 in_seq=0 out_seq=0 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7ce27fc640 1 -- 172.21.15.38:0/3696402728 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56355c8891a0 con 0x56355c955ab0 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7ce27fc640 10 client.6171 handle_command_reply: tid=0 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7d0cc10740 1 client.6171 shutdown 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7d0cc10740 1 -- 172.21.15.38:0/3696402728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 msgr2=0x56355c96f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 0x56355c96f450 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4003640 tx=0x7f7cf40046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7d0cc10740 1 -- 172.21.15.38:0/3696402728 shutdown_connections 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56355c955ab0 0x56355c912920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c95cf00 0x56355c955570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 0x56355c956f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7ce27fc640 0 client.6171 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:03.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/3696402728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 0x56355c96f450 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7d0cc10740 1 -- 172.21.15.38:0/3696402728 >> 172.21.15.38:0/3696402728 conn(0x56355c919390 msgr2=0x56355c9622d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:03.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7d0cc10740 1 -- 172.21.15.38:0/3696402728 shutdown_connections 2024-04-25T08:35:03.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7d0cc10740 1 -- 172.21.15.38:0/3696402728 wait complete. 2024-04-25T08:35:03.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.789+0000 7f7d0cc10740 20 client.6171 trim_cache size 0 max 16384 2024-04-25T08:35:03.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.804+0000 7f7d0cc10740 1 Processor -- start 2024-04-25T08:35:03.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.804+0000 7f7d0cc10740 1 -- start start 2024-04-25T08:35:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d0cc10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c9436b0 0x56355c95dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d0cc10740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c95cf00 0x56355c919960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d0cc10740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 0x56355c917260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d0cc10740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d042002c0 con 0x56355c95cf00 2024-04-25T08:35:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d0cc10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d041ff460 con 0x56355c9436b0 2024-04-25T08:35:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d0cc10740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0414e8b0 con 0x56355c960f50 2024-04-25T08:35:03.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d02ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c95cf00 0x56355c919960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d02ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c95cf00 0x56355c919960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42556/0 (socket says 172.21.15.38:42556) 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d02ffd640 1 -- 172.21.15.38:0/4217233787 learned_addr learned my addr 172.21.15.38:0/4217233787 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d02ffd640 1 -- 172.21.15.38:0/4217233787 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 msgr2=0x56355c917260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/4217233787 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 0x56355c917260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d02ffd640 1 -- 172.21.15.38:0/4217233787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c9436b0 msgr2=0x56355c95dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d037fe640 1 --2- 172.21.15.38:0/4217233787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c9436b0 0x56355c95dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.805+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/4217233787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c9436b0 0x56355c95dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7d02ffd640 1 -- 172.21.15.38:0/4217233787 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56355c5293a0 con 0x56355c95cf00 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7d037fe640 1 --2- 172.21.15.38:0/4217233787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c9436b0 0x56355c95dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/4217233787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c95cf00 0x56355c919960 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8078dd0 tx=0x7f7cf807f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42e4b9826145e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7ce8ff9640 1 -- 172.21.15.38:0/4217233787 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf80814a0 con 0x56355c95cf00 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7ce8ff9640 1 -- 172.21.15.38:0/4217233787 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf8095070 con 0x56355c95cf00 2024-04-25T08:35:03.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7ce8ff9640 1 -- 172.21.15.38:0/4217233787 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8076ee0 con 0x56355c95cf00 2024-04-25T08:35:03.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7d0cc10740 1 -- 172.21.15.38:0/4217233787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c95cf00 msgr2=0x56355c919960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/4217233787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c95cf00 0x56355c919960 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8078dd0 tx=0x7f7cf807f040 comp rx=0 tx=0).stop 2024-04-25T08:35:03.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7d0cc10740 1 -- 172.21.15.38:0/4217233787 shutdown_connections 2024-04-25T08:35:03.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/4217233787 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c960f50 0x56355c917260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/4217233787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c95cf00 0x56355c919960 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/4217233787 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c9436b0 0x56355c95dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.806+0000 7f7d0cc10740 1 -- 172.21.15.38:0/4217233787 >> 172.21.15.38:0/4217233787 conn(0x56355c919390 msgr2=0x56355c978690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:03.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.807+0000 7f7d0cc10740 1 -- 172.21.15.38:0/4217233787 shutdown_connections 2024-04-25T08:35:03.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.807+0000 7f7d0cc10740 1 -- 172.21.15.38:0/4217233787 wait complete. 2024-04-25T08:35:03.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.807+0000 7f7d0cc10740 1 -- start start 2024-04-25T08:35:03.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.807+0000 7f7d0cc10740 1 Processor -- start 2024-04-25T08:35:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d0cc10740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 0x56355c956c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d0cc10740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c95cf00 0x56355c917260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d0cc10740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c960f50 0x56355c9177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d0cc10740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56355c913090 con 0x56355c9436b0 2024-04-25T08:35:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d0cc10740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0414eeb0 con 0x56355c960f50 2024-04-25T08:35:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d0cc10740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56355c960240 con 0x56355c95cf00 2024-04-25T08:35:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d0cc10740 10 client.0 _ll_register_callbacks cb 0x56355c9140d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 0x56355c956c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 0x56355c956c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42572/0 (socket says 172.21.15.38:42572) 2024-04-25T08:35:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d037fe640 1 -- 172.21.15.38:0/2429016957 learned_addr learned my addr 172.21.15.38:0/2429016957 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c95cf00 0x56355c917260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d03fff640 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c960f50 0x56355c9177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d037fe640 1 -- 172.21.15.38:0/2429016957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c95cf00 msgr2=0x56355c917260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d037fe640 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c95cf00 0x56355c917260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d037fe640 1 -- 172.21.15.38:0/2429016957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c960f50 msgr2=0x56355c9177a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d037fe640 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c960f50 0x56355c9177a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d037fe640 1 -- 172.21.15.38:0/2429016957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56355c917e30 con 0x56355c9436b0 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7d037fe640 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 0x56355c956c10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400a010 tx=0x7f7cf4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4361abe0f5311dc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.808+0000 7f7ce27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7ce27fc640 1 -- 172.21.15.38:0/2429016957 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4041060 con 0x56355c9436b0 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7ce27fc640 1 -- 172.21.15.38:0/2429016957 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7cf402c650 con 0x56355c9436b0 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2429016957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7cf8095070 con 0x56355c9436b0 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7ce27fc640 1 -- 172.21.15.38:0/2429016957 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cf4027050 con 0x56355c9436b0 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7d0cc10740 10 client.6177 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2429016957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56355c957150 con 0x56355c9436b0 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7ce27fc640 1 -- 172.21.15.38:0/2429016957 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf8095070 con 0x56355c9436b0 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7ce27fc640 1 -- 172.21.15.38:0/2429016957 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7cf4005990 con 0x56355c9436b0 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7d0cc10740 10 client.6177 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7d0cc10740 10 client.6177 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7d0cc10740 20 client.6177 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56355c96ee70 0x56355c9c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7d0cc10740 4 client.6177 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2429016957 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x56355c8834f0 con 0x56355c96ee70 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.809+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56355c96ee70 0x56355c9c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7ce27fc640 10 client.6177 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d02ffd640 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56355c96ee70 0x56355c9c8680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c046b57c43523f8f server_cookie=1926ea96af4b701a in_seq=0 out_seq=0 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7ce27fc640 1 -- 172.21.15.38:0/2429016957 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x56355c8834f0 con 0x56355c96ee70 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7ce27fc640 10 client.6177 handle_command_reply: tid=0 2024-04-25T08:35:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d0cc10740 1 client.6177 shutdown 2024-04-25T08:35:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2429016957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 msgr2=0x56355c956c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 0x56355c956c10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400a010 tx=0x7f7cf4019000 comp rx=0 tx=0).stop 2024-04-25T08:35:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2429016957 shutdown_connections 2024-04-25T08:35:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56355c96ee70 0x56355c9c8680 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56355c960f50 0x56355c9177a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56355c95cf00 0x56355c917260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7ce27fc640 0 client.6177 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d0cc10740 1 --2- 172.21.15.38:0/2429016957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56355c9436b0 0x56355c956c10 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2429016957 >> 172.21.15.38:0/2429016957 conn(0x56355c919390 msgr2=0x56355c95c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2429016957 shutdown_connections 2024-04-25T08:35:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d0cc10740 1 -- 172.21.15.38:0/2429016957 wait complete. 2024-04-25T08:35:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.810+0000 7f7d0cc10740 20 client.6177 trim_cache size 0 max 16384 2024-04-25T08:35:03.812 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:03.812 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": -2 2024-04-25T08:35:03.812 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.811+0000 7f7d09e3a640 1 -- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7cf006b500 msgr2=0x7f7cf006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.811+0000 7f7d09e3a640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7cf006b500 0x7f7cf006d9c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7d041e9110 tx=0x7f7cf4009190 comp rx=0 tx=0).stop 2024-04-25T08:35:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.811+0000 7f7d09e3a640 1 -- 172.21.15.38:0/433630156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d040d0700 msgr2=0x7f7d041e7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.811+0000 7f7d09e3a640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d040d0700 0x7f7d041e7c50 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f7cf80098c0 tx=0x7f7cf80046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.811+0000 7f7d09e3a640 1 -- 172.21.15.38:0/433630156 shutdown_connections 2024-04-25T08:35:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.811+0000 7f7d09e3a640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d040d1e90 0x7f7d040cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.811+0000 7f7d09e3a640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d040d0700 0x7f7d041e7c50 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.811+0000 7f7d09e3a640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7cf006b500 0x7f7cf006d9c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.811+0000 7f7d09e3a640 1 --2- 172.21.15.38:0/433630156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d040d0320 0x7f7d0414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.811+0000 7f7d09e3a640 1 -- 172.21.15.38:0/433630156 >> 172.21.15.38:0/433630156 conn(0x7f7d040668f0 msgr2=0x7f7d040d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.811+0000 7f7d09e3a640 1 -- 172.21.15.38:0/433630156 shutdown_connections 2024-04-25T08:35:03.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.811+0000 7f7d09e3a640 1 -- 172.21.15.38:0/433630156 wait complete. 2024-04-25T08:35:03.816 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T08:35:03.816 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-25T08:35:03.816 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T08:35:03.816 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:03.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.922+0000 7f763c15c640 1 Processor -- start 2024-04-25T08:35:03.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.922+0000 7f763c15c640 1 -- start start 2024-04-25T08:35:03.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.923+0000 7f763c15c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f76340aeeb0 0x7f76340ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.923+0000 7f763c15c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f76340cd450 0x7f763406abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.923+0000 7f763c15c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f763406b120 0x7f76341616d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.923+0000 7f763c15c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763406b550 con 0x7f76340cd450 2024-04-25T08:35:03.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.923+0000 7f763c15c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76340cfa40 con 0x7f76340aeeb0 2024-04-25T08:35:03.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.923+0000 7f763c15c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76340cd9b0 con 0x7f763406b120 2024-04-25T08:35:03.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.923+0000 7f76396f7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f76340cd450 0x7f763406abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.923+0000 7f76396f7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f76340cd450 0x7f763406abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42588/0 (socket says 172.21.15.38:42588) 2024-04-25T08:35:03.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.923+0000 7f76396f7640 1 -- 172.21.15.38:0/251904451 learned_addr learned my addr 172.21.15.38:0/251904451 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:03.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.924+0000 7f7639ef8640 1 --2- 172.21.15.38:0/251904451 >> v2:172.21.15.107:3300/0 conn(0x7f76340aeeb0 0x7f76340ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.924+0000 7f763a6f9640 1 --2- 172.21.15.38:0/251904451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f763406b120 0x7f76341616d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.924+0000 7f76396f7640 1 -- 172.21.15.38:0/251904451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f763406b120 msgr2=0x7f76341616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.924+0000 7f76396f7640 1 --2- 172.21.15.38:0/251904451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f763406b120 0x7f76341616d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.924+0000 7f76396f7640 1 -- 172.21.15.38:0/251904451 >> v2:172.21.15.107:3300/0 conn(0x7f76340aeeb0 msgr2=0x7f76340ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.924+0000 7f76396f7640 1 --2- 172.21.15.38:0/251904451 >> v2:172.21.15.107:3300/0 conn(0x7f76340aeeb0 0x7f76340ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.924+0000 7f76396f7640 1 -- 172.21.15.38:0/251904451 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76340cdc90 con 0x7f76340cd450 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.924+0000 7f763a6f9640 1 --2- 172.21.15.38:0/251904451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f763406b120 0x7f76341616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.924+0000 7f76396f7640 1 --2- 172.21.15.38:0/251904451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76340cd450 0x7f763406abe0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f76340d16e0 tx=0x7f762802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee4b6e8d7b87878 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.925+0000 7f7638ef6640 1 -- 172.21.15.38:0/251904451 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628037070 con 0x7f76340cd450 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.925+0000 7f7638ef6640 1 -- 172.21.15.38:0/251904451 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7628032440 con 0x7f76340cd450 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.925+0000 7f7638ef6640 1 -- 172.21.15.38:0/251904451 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7628032810 con 0x7f76340cd450 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.925+0000 7f763c15c640 1 -- 172.21.15.38:0/251904451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76340cd450 msgr2=0x7f763406abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.925+0000 7f763c15c640 1 --2- 172.21.15.38:0/251904451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76340cd450 0x7f763406abe0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f76340d16e0 tx=0x7f762802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.925+0000 7f763c15c640 1 -- 172.21.15.38:0/251904451 shutdown_connections 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.925+0000 7f763c15c640 1 --2- 172.21.15.38:0/251904451 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f763406b120 0x7f76341616d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.925+0000 7f763c15c640 1 --2- 172.21.15.38:0/251904451 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76340cd450 0x7f763406abe0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.925+0000 7f763c15c640 1 --2- 172.21.15.38:0/251904451 >> v2:172.21.15.107:3300/0 conn(0x7f76340aeeb0 0x7f76340ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.925+0000 7f763c15c640 1 -- 172.21.15.38:0/251904451 >> 172.21.15.38:0/251904451 conn(0x7f7634072610 msgr2=0x7f7634099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:03.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.925+0000 7f763c15c640 1 -- 172.21.15.38:0/251904451 shutdown_connections 2024-04-25T08:35:03.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.925+0000 7f763c15c640 1 -- 172.21.15.38:0/251904451 wait complete. 2024-04-25T08:35:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.926+0000 7f763c15c640 1 Processor -- start 2024-04-25T08:35:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.926+0000 7f763c15c640 1 -- start start 2024-04-25T08:35:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.927+0000 7f763c15c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f763406b120 0x7f76340c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.927+0000 7f763c15c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76340aeeb0 0x7f76340b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.927+0000 7f763c15c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76340cd450 0x7f76340b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.927+0000 7f763c15c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7634161c10 con 0x7f76340cd450 2024-04-25T08:35:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.927+0000 7f763c15c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76340cdf70 con 0x7f76340aeeb0 2024-04-25T08:35:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.927+0000 7f763c15c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7634161d90 con 0x7f763406b120 2024-04-25T08:35:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.927+0000 7f763a6f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76340cd450 0x7f76340b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.927+0000 7f763a6f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76340cd450 0x7f76340b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42594/0 (socket says 172.21.15.38:42594) 2024-04-25T08:35:03.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.927+0000 7f763a6f9640 1 -- 172.21.15.38:0/2323988228 learned_addr learned my addr 172.21.15.38:0/2323988228 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.927+0000 7f76396f7640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76340aeeb0 0x7f76340b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.927+0000 7f7639ef8640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f763406b120 0x7f76340c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.927+0000 7f763a6f9640 1 -- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f763406b120 msgr2=0x7f76340c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.928+0000 7f763a6f9640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f763406b120 0x7f76340c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.928+0000 7f763a6f9640 1 -- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76340aeeb0 msgr2=0x7f76340b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.928+0000 7f763a6f9640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76340aeeb0 0x7f76340b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.928+0000 7f763a6f9640 1 -- 172.21.15.38:0/2323988228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76341f31a0 con 0x7f76340cd450 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.928+0000 7f763a6f9640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76340cd450 0x7f76340b3ab0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f762c007e40 tx=0x7f762c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.928+0000 7f76267fc640 1 -- 172.21.15.38:0/2323988228 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c01a070 con 0x7f76340cd450 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.928+0000 7f76267fc640 1 -- 172.21.15.38:0/2323988228 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762c003b90 con 0x7f76340cd450 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.928+0000 7f763c15c640 1 -- 172.21.15.38:0/2323988228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76340c5400 con 0x7f76340cd450 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.928+0000 7f76267fc640 1 -- 172.21.15.38:0/2323988228 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c00de80 con 0x7f76340cd450 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.928+0000 7f763c15c640 1 -- 172.21.15.38:0/2323988228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76340c5ca0 con 0x7f76340cd450 2024-04-25T08:35:03.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.929+0000 7f763c15c640 1 -- 172.21.15.38:0/2323988228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76340c57f0 con 0x7f76340cd450 2024-04-25T08:35:03.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.929+0000 7f76267fc640 1 -- 172.21.15.38:0/2323988228 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f762c01d070 con 0x7f76340cd450 2024-04-25T08:35:03.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.930+0000 7f76267fc640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f761806b4d0 0x7f761806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:03.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.930+0000 7f76267fc640 1 -- 172.21.15.38:0/2323988228 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f762c08a620 con 0x7f76340cd450 2024-04-25T08:35:03.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.930+0000 7f7639ef8640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f761806b4d0 0x7f761806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:03.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.931+0000 7f7639ef8640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f761806b4d0 0x7f761806d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7630009ad0 tx=0x7f7630008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:03.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:03.934+0000 7f76267fc640 1 -- 172.21.15.38:0/2323988228 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76341f31a0 con 0x7f76340cd450 2024-04-25T08:35:04.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.157+0000 7f763c15c640 1 -- 172.21.15.38:0/2323988228 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f763406d090 con 0x7f76340cd450 2024-04-25T08:35:04.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.158+0000 7f76267fc640 1 -- 172.21.15.38:0/2323988228 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f762c010070 con 0x7f76340cd450 2024-04-25T08:35:04.154 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:04.154 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:34:46.222808+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:04.155 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 24 2024-04-25T08:35:04.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.161+0000 7f763c15c640 1 -- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f761806b4d0 msgr2=0x7f761806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.161+0000 7f763c15c640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f761806b4d0 0x7f761806d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7630009ad0 tx=0x7f7630008040 comp rx=0 tx=0).stop 2024-04-25T08:35:04.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.161+0000 7f763c15c640 1 -- 172.21.15.38:0/2323988228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76340cd450 msgr2=0x7f76340b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.161+0000 7f763c15c640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76340cd450 0x7f76340b3ab0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f762c007e40 tx=0x7f762c00d4a0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.161+0000 7f763c15c640 1 -- 172.21.15.38:0/2323988228 shutdown_connections 2024-04-25T08:35:04.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.161+0000 7f763c15c640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f76340cd450 0x7f76340b3ab0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.161+0000 7f763c15c640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76340aeeb0 0x7f76340b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.161+0000 7f763c15c640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f761806b4d0 0x7f761806d990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.161+0000 7f763c15c640 1 --2- 172.21.15.38:0/2323988228 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f763406b120 0x7f76340c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.161+0000 7f763c15c640 1 -- 172.21.15.38:0/2323988228 >> 172.21.15.38:0/2323988228 conn(0x7f7634072610 msgr2=0x7f76340b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:04.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.161+0000 7f763c15c640 1 -- 172.21.15.38:0/2323988228 shutdown_connections 2024-04-25T08:35:04.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.161+0000 7f763c15c640 1 -- 172.21.15.38:0/2323988228 wait complete. 2024-04-25T08:35:04.167 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-25T08:35:04.359 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:04.359 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:35:04.359 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:04.367 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:35:04.368 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T08:35:04.368 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:04.368 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T08:35:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.477+0000 7f392977e640 1 Processor -- start 2024-04-25T08:35:04.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.477+0000 7f392977e640 1 -- start start 2024-04-25T08:35:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.477+0000 7f392977e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f39240d33d0 0x7f39240d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.477+0000 7f392977e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f39240d1e90 0x7f39240b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.477+0000 7f392977e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39240d2330 con 0x7f39240d0320 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.477+0000 7f392977e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39240d2610 con 0x7f39240d1e90 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.477+0000 7f392977e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39240d28f0 con 0x7f39240d33d0 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.478+0000 7f3922ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f39240d0320 0x7f39240d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60616/0 (socket says 172.21.15.38:60616) 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.478+0000 7f3922ffd640 1 -- 172.21.15.38:0/2191898154 learned_addr learned my addr 172.21.15.38:0/2191898154 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.478+0000 7f39237fe640 1 --2- 172.21.15.38:0/2191898154 >> v2:172.21.15.38:3300/0 conn(0x7f39240d1e90 0x7f39240b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.478+0000 7f39227fc640 1 --2- 172.21.15.38:0/2191898154 >> v2:172.21.15.107:3300/0 conn(0x7f39240d33d0 0x7f39240d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.478+0000 7f39237fe640 1 -- 172.21.15.38:0/2191898154 >> v2:172.21.15.107:3300/0 conn(0x7f39240d33d0 msgr2=0x7f39240d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.478+0000 7f39237fe640 1 --2- 172.21.15.38:0/2191898154 >> v2:172.21.15.107:3300/0 conn(0x7f39240d33d0 0x7f39240d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.478+0000 7f39237fe640 1 -- 172.21.15.38:0/2191898154 >> v1:172.21.15.38:6789/0 conn(0x7f39240d0320 legacy=0x7f39240d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.478+0000 7f3921ffb640 1 -- 172.21.15.38:0/2191898154 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516493768 0 0) 0x7f39240d2330 con 0x7f39240d0320 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.478+0000 7f39237fe640 1 -- 172.21.15.38:0/2191898154 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39240d2bd0 con 0x7f39240d1e90 2024-04-25T08:35:04.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.479+0000 7f39237fe640 1 --2- 172.21.15.38:0/2191898154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39240d1e90 0x7f39240b6980 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f39240b6410 tx=0x7f391c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b7e6c0ec3e3237 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:04.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.479+0000 7f3921ffb640 1 -- 172.21.15.38:0/2191898154 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c037070 con 0x7f39240d1e90 2024-04-25T08:35:04.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.479+0000 7f3921ffb640 1 -- 172.21.15.38:0/2191898154 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c032440 con 0x7f39240d1e90 2024-04-25T08:35:04.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.479+0000 7f3921ffb640 1 -- 172.21.15.38:0/2191898154 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c032810 con 0x7f39240d1e90 2024-04-25T08:35:04.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.479+0000 7f392977e640 1 -- 172.21.15.38:0/2191898154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39240d1e90 msgr2=0x7f39240b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.479+0000 7f392977e640 1 --2- 172.21.15.38:0/2191898154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39240d1e90 0x7f39240b6980 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f39240b6410 tx=0x7f391c02a230 comp rx=0 tx=0).stop 2024-04-25T08:35:04.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.479+0000 7f392977e640 1 -- 172.21.15.38:0/2191898154 shutdown_connections 2024-04-25T08:35:04.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.479+0000 7f392977e640 1 --2- 172.21.15.38:0/2191898154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39240d1e90 0x7f39240b6980 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.479+0000 7f392977e640 1 --2- 172.21.15.38:0/2191898154 >> v2:172.21.15.107:3300/0 conn(0x7f39240d33d0 0x7f39240d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.479+0000 7f392977e640 1 -- 172.21.15.38:0/2191898154 >> 172.21.15.38:0/2191898154 conn(0x7f39240668f0 msgr2=0x7f3924066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:04.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.479+0000 7f392977e640 1 -- 172.21.15.38:0/2191898154 shutdown_connections 2024-04-25T08:35:04.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.479+0000 7f392977e640 1 -- 172.21.15.38:0/2191898154 wait complete. 2024-04-25T08:35:04.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.481+0000 7f392977e640 1 Processor -- start 2024-04-25T08:35:04.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.481+0000 7f392977e640 1 -- start start 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.481+0000 7f392977e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39240d0320 0x7f3924159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.481+0000 7f392977e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39240d1e90 0x7f39240b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.481+0000 7f392977e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39240d33d0 0x7f392414edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.481+0000 7f392977e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3924161390 con 0x7f39240d33d0 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.481+0000 7f392977e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3924161210 con 0x7f39240d1e90 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.481+0000 7f392977e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39240b9720 con 0x7f39240d0320 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f39237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39240d33d0 0x7f392414edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f39237fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39240d33d0 0x7f392414edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42614/0 (socket says 172.21.15.38:42614) 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f39237fe640 1 -- 172.21.15.38:0/1299340560 learned_addr learned my addr 172.21.15.38:0/1299340560 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f39227fc640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39240d1e90 0x7f39240b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f3922ffd640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39240d0320 0x7f3924159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f39237fe640 1 -- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39240d0320 msgr2=0x7f3924159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f39237fe640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39240d0320 0x7f3924159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f39237fe640 1 -- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39240d1e90 msgr2=0x7f39240b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f39237fe640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39240d1e90 0x7f39240b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f39237fe640 1 -- 172.21.15.38:0/1299340560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392414f330 con 0x7f39240d33d0 2024-04-25T08:35:04.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f39227fc640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39240d1e90 0x7f39240b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:04.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f39237fe640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39240d33d0 0x7f392414edf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f391c035040 tx=0x7f391c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:04.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f390b7fe640 1 -- 172.21.15.38:0/1299340560 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c037040 con 0x7f39240d33d0 2024-04-25T08:35:04.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f390b7fe640 1 -- 172.21.15.38:0/1299340560 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391c042070 con 0x7f39240d33d0 2024-04-25T08:35:04.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f392977e640 1 -- 172.21.15.38:0/1299340560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39240b3a10 con 0x7f39240d33d0 2024-04-25T08:35:04.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f390b7fe640 1 -- 172.21.15.38:0/1299340560 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c030b30 con 0x7f39240d33d0 2024-04-25T08:35:04.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.482+0000 7f392977e640 1 -- 172.21.15.38:0/1299340560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39240b3fd0 con 0x7f39240d33d0 2024-04-25T08:35:04.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.484+0000 7f390b7fe640 1 -- 172.21.15.38:0/1299340560 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f391c030cd0 con 0x7f39240d33d0 2024-04-25T08:35:04.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.484+0000 7f390b7fe640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f391006b4d0 0x7f391006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.484+0000 7f390b7fe640 1 -- 172.21.15.38:0/1299340560 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f391c0aaef0 con 0x7f39240d33d0 2024-04-25T08:35:04.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.484+0000 7f3922ffd640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f391006b4d0 0x7f391006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.485+0000 7f3922ffd640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f391006b4d0 0x7f391006d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f39140012e0 tx=0x7f3914007ef0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:04.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.485+0000 7f392c122740 1 Processor -- start 2024-04-25T08:35:04.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.485+0000 7f392c122740 1 -- start start 2024-04-25T08:35:04.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f392c122740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925aaf10 0x5573925ad070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f392c122740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925e75e0 0x5573925ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f392c122740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925d59d0 0x5573925d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f392c122740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573924dcfe0 con 0x5573925e75e0 2024-04-25T08:35:04.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f392c122740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739250ed20 con 0x5573925d59d0 2024-04-25T08:35:04.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f392c122740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573924b7fa0 con 0x5573925aaf10 2024-04-25T08:35:04.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f39227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925e75e0 0x5573925ab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f39227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925e75e0 0x5573925ab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42624/0 (socket says 172.21.15.38:42624) 2024-04-25T08:35:04.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f39227fc640 1 -- 172.21.15.38:0/1266905387 learned_addr learned my addr 172.21.15.38:0/1266905387 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:04.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f3922ffd640 1 --2- 172.21.15.38:0/1266905387 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925aaf10 0x5573925ad070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f39237fe640 1 --2- 172.21.15.38:0/1266905387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925d59d0 0x5573925d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f39227fc640 1 -- 172.21.15.38:0/1266905387 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925aaf10 msgr2=0x5573925ad070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:04.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f39227fc640 1 --2- 172.21.15.38:0/1266905387 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925aaf10 0x5573925ad070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f39227fc640 1 -- 172.21.15.38:0/1266905387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925d59d0 msgr2=0x5573925d5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:04.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f39227fc640 1 --2- 172.21.15.38:0/1266905387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925d59d0 0x5573925d5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f39227fc640 1 -- 172.21.15.38:0/1266905387 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573922efb00 con 0x5573925e75e0 2024-04-25T08:35:04.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.486+0000 7f39227fc640 1 --2- 172.21.15.38:0/1266905387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925e75e0 0x5573925ab2f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f391800b980 tx=0x7f391800be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a92b36f2066d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.487+0000 7f3908ff9640 1 -- 172.21.15.38:0/1266905387 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918018020 con 0x5573925e75e0 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.487+0000 7f3908ff9640 1 -- 172.21.15.38:0/1266905387 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391800fa20 con 0x5573925e75e0 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.487+0000 7f3908ff9640 1 -- 172.21.15.38:0/1266905387 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391800fdf0 con 0x5573925e75e0 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.487+0000 7f392c122740 1 -- 172.21.15.38:0/1266905387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925e75e0 msgr2=0x5573925ab2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.487+0000 7f392c122740 1 --2- 172.21.15.38:0/1266905387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925e75e0 0x5573925ab2f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f391800b980 tx=0x7f391800be40 comp rx=0 tx=0).stop 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.487+0000 7f392c122740 1 -- 172.21.15.38:0/1266905387 shutdown_connections 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.487+0000 7f392c122740 1 --2- 172.21.15.38:0/1266905387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925d59d0 0x5573925d5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.487+0000 7f392c122740 1 --2- 172.21.15.38:0/1266905387 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925e75e0 0x5573925ab2f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.487+0000 7f392c122740 1 --2- 172.21.15.38:0/1266905387 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925aaf10 0x5573925ad070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.487+0000 7f392c122740 1 -- 172.21.15.38:0/1266905387 >> 172.21.15.38:0/1266905387 conn(0x557392573100 msgr2=0x5573925f09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.487+0000 7f392c122740 1 -- 172.21.15.38:0/1266905387 shutdown_connections 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.487+0000 7f392c122740 1 -- 172.21.15.38:0/1266905387 wait complete. 2024-04-25T08:35:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.488+0000 7f392c122740 1 -- start start 2024-04-25T08:35:04.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.488+0000 7f392c122740 1 Processor -- start 2024-04-25T08:35:04.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f392c122740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925aaf10 0x5573925c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f392c122740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 0x557392600790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f392c122740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f392c122740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573924ad750 con 0x5573925d59d0 2024-04-25T08:35:04.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f392c122740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55739250adc0 con 0x5573925aaf10 2024-04-25T08:35:04.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f392c122740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573926011a0 con 0x5573925e75e0 2024-04-25T08:35:04.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f392c122740 10 client.0 _ll_register_callbacks cb 0x5573924c5ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:04.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f39227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 0x557392600790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f39227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 0x557392600790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42626/0 (socket says 172.21.15.38:42626) 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f39227fc640 1 -- 172.21.15.38:0/3390243255 learned_addr learned my addr 172.21.15.38:0/3390243255 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f39237fe640 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f3922ffd640 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925aaf10 0x5573925c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f39227fc640 1 -- 172.21.15.38:0/3390243255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 msgr2=0x5573925d1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f39227fc640 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f39227fc640 1 -- 172.21.15.38:0/3390243255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925aaf10 msgr2=0x5573925c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f39227fc640 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925aaf10 0x5573925c51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f39227fc640 1 -- 172.21.15.38:0/3390243255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573921e33f0 con 0x5573925d59d0 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f39227fc640 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 0x557392600790 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3918007010 tx=0x7f3918004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cae91a83f0a88f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f3901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f3901ffb640 1 -- 172.21.15.38:0/3390243255 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918018070 con 0x5573925d59d0 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f3901ffb640 1 -- 172.21.15.38:0/3390243255 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f391800f820 con 0x5573925d59d0 2024-04-25T08:35:04.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.489+0000 7f392c122740 1 -- 172.21.15.38:0/3390243255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f391800fa20 con 0x5573925d59d0 2024-04-25T08:35:04.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f3901ffb640 1 -- 172.21.15.38:0/3390243255 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f391802f050 con 0x5573925d59d0 2024-04-25T08:35:04.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f392c122740 10 client.6198 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:04.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f392c122740 1 -- 172.21.15.38:0/3390243255 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x557392350290 con 0x5573925d59d0 2024-04-25T08:35:04.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f3901ffb640 1 -- 172.21.15.38:0/3390243255 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391800fa20 con 0x5573925d59d0 2024-04-25T08:35:04.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f3901ffb640 1 -- 172.21.15.38:0/3390243255 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3918032020 con 0x5573925d59d0 2024-04-25T08:35:04.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f392c122740 10 client.6198 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:04.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f392c122740 10 client.6198 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:04.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f392c122740 20 client.6198 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:04.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f392c122740 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5573925d3b60 0x5573925f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f392c122740 4 client.6198 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:04.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f392c122740 1 -- 172.21.15.38:0/3390243255 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55739251c260 con 0x5573925d3b60 2024-04-25T08:35:04.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f3922ffd640 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5573925d3b60 0x5573925f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f3922ffd640 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5573925d3b60 0x5573925f2e30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac958aa9e931d15c server_cookie=4ce86bf52a6ee94e in_seq=0 out_seq=0 2024-04-25T08:35:04.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.490+0000 7f3901ffb640 10 client.6198 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:04.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f3901ffb640 1 -- 172.21.15.38:0/3390243255 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55739251c260 con 0x5573925d3b60 2024-04-25T08:35:04.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f3901ffb640 10 client.6198 handle_command_reply: tid=0 2024-04-25T08:35:04.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f392c122740 1 client.6198 shutdown 2024-04-25T08:35:04.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f392c122740 1 -- 172.21.15.38:0/3390243255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 msgr2=0x557392600790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f392c122740 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 0x557392600790 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3918007010 tx=0x7f3918004730 comp rx=0 tx=0).stop 2024-04-25T08:35:04.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f392c122740 1 -- 172.21.15.38:0/3390243255 shutdown_connections 2024-04-25T08:35:04.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f392c122740 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5573925d3b60 0x5573925f2e30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f392c122740 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f3901ffb640 0 client.6198 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:04.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f392c122740 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 0x557392600790 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f392c122740 1 --2- 172.21.15.38:0/3390243255 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925aaf10 0x5573925c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f392c122740 1 -- 172.21.15.38:0/3390243255 >> 172.21.15.38:0/3390243255 conn(0x557392573100 msgr2=0x5573925f15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:04.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f392c122740 1 -- 172.21.15.38:0/3390243255 shutdown_connections 2024-04-25T08:35:04.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f392c122740 1 -- 172.21.15.38:0/3390243255 wait complete. 2024-04-25T08:35:04.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.491+0000 7f392c122740 20 client.6198 trim_cache size 0 max 16384 2024-04-25T08:35:04.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.507+0000 7f392c122740 1 Processor -- start 2024-04-25T08:35:04.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.507+0000 7f392c122740 1 -- start start 2024-04-25T08:35:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.508+0000 7f392c122740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925aaf10 0x5573925f2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.508+0000 7f392c122740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 0x5573925d8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.508+0000 7f392c122740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.508+0000 7f392c122740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3924200150 con 0x5573925d59d0 2024-04-25T08:35:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.508+0000 7f392c122740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241ff2d0 con 0x5573925aaf10 2024-04-25T08:35:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.508+0000 7f392c122740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39241f41e0 con 0x5573925e75e0 2024-04-25T08:35:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.509+0000 7f39227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 0x5573925d8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.509+0000 7f39227fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 0x5573925d8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42632/0 (socket says 172.21.15.38:42632) 2024-04-25T08:35:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.509+0000 7f39227fc640 1 -- 172.21.15.38:0/1079330757 learned_addr learned my addr 172.21.15.38:0/1079330757 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.509+0000 7f3922ffd640 1 --2- 172.21.15.38:0/1079330757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925aaf10 0x5573925f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.509+0000 7f39237fe640 1 --2- 172.21.15.38:0/1079330757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.509+0000 7f39227fc640 1 -- 172.21.15.38:0/1079330757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 msgr2=0x5573925d8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.509+0000 7f39227fc640 1 --2- 172.21.15.38:0/1079330757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.509+0000 7f39227fc640 1 -- 172.21.15.38:0/1079330757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925aaf10 msgr2=0x5573925f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.509+0000 7f39227fc640 1 --2- 172.21.15.38:0/1079330757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925aaf10 0x5573925f2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.509+0000 7f39227fc640 1 -- 172.21.15.38:0/1079330757 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573921bc4b0 con 0x5573925d59d0 2024-04-25T08:35:04.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.509+0000 7f39227fc640 1 --2- 172.21.15.38:0/1079330757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 0x5573925d8890 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f391800b6f0 tx=0x7f3918020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccba3c847afc04e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:04.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.510+0000 7f3903fff640 1 -- 172.21.15.38:0/1079330757 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391800f820 con 0x5573925d59d0 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.510+0000 7f3903fff640 1 -- 172.21.15.38:0/1079330757 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391800f9c0 con 0x5573925d59d0 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.510+0000 7f3903fff640 1 -- 172.21.15.38:0/1079330757 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39180038e0 con 0x5573925d59d0 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.510+0000 7f392c122740 1 -- 172.21.15.38:0/1079330757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 msgr2=0x5573925d8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.510+0000 7f392c122740 1 --2- 172.21.15.38:0/1079330757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 0x5573925d8890 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f391800b6f0 tx=0x7f3918020040 comp rx=0 tx=0).stop 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.510+0000 7f392c122740 1 -- 172.21.15.38:0/1079330757 shutdown_connections 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.510+0000 7f392c122740 1 --2- 172.21.15.38:0/1079330757 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.511+0000 7f392c122740 1 --2- 172.21.15.38:0/1079330757 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925d59d0 0x5573925d8890 secure :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f391800b6f0 tx=0x7f3918020040 comp rx=0 tx=0).stop 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.511+0000 7f392c122740 1 --2- 172.21.15.38:0/1079330757 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5573925aaf10 0x5573925f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.511+0000 7f392c122740 1 -- 172.21.15.38:0/1079330757 >> 172.21.15.38:0/1079330757 conn(0x557392573100 msgr2=0x5573925f3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.511+0000 7f392c122740 1 -- 172.21.15.38:0/1079330757 shutdown_connections 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.511+0000 7f392c122740 1 -- 172.21.15.38:0/1079330757 wait complete. 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.511+0000 7f392c122740 1 -- start start 2024-04-25T08:35:04.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.512+0000 7f392c122740 1 Processor -- start 2024-04-25T08:35:04.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f392c122740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925aaf10 0x5573925a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f392c122740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f392c122740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557392644390 0x5573925d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f392c122740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573925d1fa0 con 0x5573925aaf10 2024-04-25T08:35:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f392c122740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39241fee20 con 0x557392644390 2024-04-25T08:35:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f392c122740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573925f31f0 con 0x5573925e75e0 2024-04-25T08:35:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f392c122740 10 client.0 _ll_register_callbacks cb 0x7f391800b380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f3922ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925aaf10 0x5573925a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f3922ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925aaf10 0x5573925a89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42646/0 (socket says 172.21.15.38:42646) 2024-04-25T08:35:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f3922ffd640 1 -- 172.21.15.38:0/2429186587 learned_addr learned my addr 172.21.15.38:0/2429186587 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f39237fe640 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557392644390 0x5573925d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f39227fc640 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f3922ffd640 1 -- 172.21.15.38:0/2429186587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 msgr2=0x5573925d8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f3922ffd640 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f3922ffd640 1 -- 172.21.15.38:0/2429186587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557392644390 msgr2=0x5573925d8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f3922ffd640 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557392644390 0x5573925d8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.513+0000 7f3922ffd640 1 -- 172.21.15.38:0/2429186587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55739265b300 con 0x5573925aaf10 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.514+0000 7f39227fc640 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.514+0000 7f3922ffd640 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925aaf10 0x5573925a89e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3914005fd0 tx=0x7f3914011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50f00cf83c33164 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.514+0000 7f3901ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.514+0000 7f3901ffb640 1 -- 172.21.15.38:0/2429186587 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914026700 con 0x5573925aaf10 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.514+0000 7f3901ffb640 1 -- 172.21.15.38:0/2429186587 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3914010040 con 0x5573925aaf10 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.514+0000 7f392c122740 1 -- 172.21.15.38:0/2429186587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f391800f9c0 con 0x5573925aaf10 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.515+0000 7f3901ffb640 1 -- 172.21.15.38:0/2429186587 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3914034050 con 0x5573925aaf10 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.515+0000 7f392c122740 10 client.6204 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.515+0000 7f392c122740 1 -- 172.21.15.38:0/2429186587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55739265bc00 con 0x5573925aaf10 2024-04-25T08:35:04.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.515+0000 7f3901ffb640 1 -- 172.21.15.38:0/2429186587 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391800f9c0 con 0x5573925aaf10 2024-04-25T08:35:04.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.515+0000 7f3901ffb640 1 -- 172.21.15.38:0/2429186587 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3914025900 con 0x5573925aaf10 2024-04-25T08:35:04.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.515+0000 7f392c122740 10 client.6204 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:04.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.515+0000 7f392c122740 10 client.6204 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:04.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.516+0000 7f392c122740 20 client.6204 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:04.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.516+0000 7f392c122740 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5573925c3b30 0x5573925c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.516+0000 7f392c122740 4 client.6204 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T08:35:04.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.516+0000 7f392c122740 1 -- 172.21.15.38:0/2429186587 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x5573925165b0 con 0x5573925c3b30 2024-04-25T08:35:04.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.516+0000 7f39227fc640 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5573925c3b30 0x5573925c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.517+0000 7f3901ffb640 10 client.6204 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:04.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.517+0000 7f39227fc640 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5573925c3b30 0x5573925c3f10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08263413e7efe46 server_cookie=213b257cdf4830df in_seq=0 out_seq=0 2024-04-25T08:35:04.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f3901ffb640 1 -- 172.21.15.38:0/2429186587 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5573925165b0 con 0x5573925c3b30 2024-04-25T08:35:04.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f3901ffb640 10 client.6204 handle_command_reply: tid=0 2024-04-25T08:35:04.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f392c122740 1 client.6204 shutdown 2024-04-25T08:35:04.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f392c122740 1 -- 172.21.15.38:0/2429186587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925aaf10 msgr2=0x5573925a89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f392c122740 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925aaf10 0x5573925a89e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3914005fd0 tx=0x7f3914011000 comp rx=0 tx=0).stop 2024-04-25T08:35:04.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f392c122740 1 -- 172.21.15.38:0/2429186587 shutdown_connections 2024-04-25T08:35:04.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f392c122740 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5573925c3b30 0x5573925c3f10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f392c122740 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557392644390 0x5573925d8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f392c122740 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5573925e75e0 0x5573925d8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f392c122740 1 --2- 172.21.15.38:0/2429186587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5573925aaf10 0x5573925a89e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f392c122740 1 -- 172.21.15.38:0/2429186587 >> 172.21.15.38:0/2429186587 conn(0x557392573100 msgr2=0x5573925f3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:04.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f3901ffb640 0 client.6204 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:04.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f392c122740 1 -- 172.21.15.38:0/2429186587 shutdown_connections 2024-04-25T08:35:04.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f392c122740 1 -- 172.21.15.38:0/2429186587 wait complete. 2024-04-25T08:35:04.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.561+0000 7f392c122740 20 client.6204 trim_cache size 0 max 16384 2024-04-25T08:35:04.557 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:04.557 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:35:04.557 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "303693eb-e4c0-466c-9152-905ca1863bed", 2024-04-25T08:35:04.558 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:35:04.558 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:04.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.562+0000 7f392977e640 1 -- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f391006b4d0 msgr2=0x7f391006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.562+0000 7f392977e640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f391006b4d0 0x7f391006d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f39140012e0 tx=0x7f3914007ef0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.562+0000 7f392977e640 1 -- 172.21.15.38:0/1299340560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39240d33d0 msgr2=0x7f392414edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.562+0000 7f392977e640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39240d33d0 0x7f392414edf0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f391c035040 tx=0x7f391c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.562+0000 7f392977e640 1 -- 172.21.15.38:0/1299340560 shutdown_connections 2024-04-25T08:35:04.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.562+0000 7f392977e640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39240d33d0 0x7f392414edf0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.562+0000 7f392977e640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f39240d1e90 0x7f39240b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.562+0000 7f392977e640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f391006b4d0 0x7f391006d990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.562+0000 7f392977e640 1 --2- 172.21.15.38:0/1299340560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f39240d0320 0x7f3924159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.562+0000 7f392977e640 1 -- 172.21.15.38:0/1299340560 >> 172.21.15.38:0/1299340560 conn(0x7f39240668f0 msgr2=0x7f39240ccf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:04.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.563+0000 7f392977e640 1 -- 172.21.15.38:0/1299340560 shutdown_connections 2024-04-25T08:35:04.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.563+0000 7f392977e640 1 -- 172.21.15.38:0/1299340560 wait complete. 2024-04-25T08:35:04.569 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '303693eb-e4c0-466c-9152-905ca1863bed', 'mode': 'asynchronous'}' 2024-04-25T08:35:04.569 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T08:35:04.569 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T08:35:04.569 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.675+0000 7fc25d693640 1 Processor -- start 2024-04-25T08:35:04.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.675+0000 7fc25d693640 1 -- start start 2024-04-25T08:35:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc25d693640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc2580d3900 0x7fc2580d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc25d693640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2580d0d90 0x7fc2580b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc25d693640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2580d3e70 con 0x7fc25815dc10 2024-04-25T08:35:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc25d693640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc25815dff0 con 0x7fc2580d3900 2024-04-25T08:35:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc25d693640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580b6990 con 0x7fc2580d0d90 2024-04-25T08:35:04.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc256ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc25815dc10 0x7fc2580ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60626/0 (socket says 172.21.15.38:60626) 2024-04-25T08:35:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc256ffd640 1 -- 172.21.15.38:0/4133710977 learned_addr learned my addr 172.21.15.38:0/4133710977 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc2567fc640 1 --2- 172.21.15.38:0/4133710977 >> v2:172.21.15.38:3300/0 conn(0x7fc2580d3900 0x7fc2580d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc2577fe640 1 --2- 172.21.15.38:0/4133710977 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2580d0d90 0x7fc2580b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc2567fc640 1 -- 172.21.15.38:0/4133710977 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2580d0d90 msgr2=0x7fc2580b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc2567fc640 1 --2- 172.21.15.38:0/4133710977 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2580d0d90 0x7fc2580b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc2567fc640 1 -- 172.21.15.38:0/4133710977 >> v1:172.21.15.38:6789/0 conn(0x7fc25815dc10 legacy=0x7fc2580ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc255ffb640 1 -- 172.21.15.38:0/4133710977 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781370197 0 0) 0x7fc2580d3e70 con 0x7fc25815dc10 2024-04-25T08:35:04.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.676+0000 7fc2567fc640 1 -- 172.21.15.38:0/4133710977 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2580d4150 con 0x7fc2580d3900 2024-04-25T08:35:04.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.677+0000 7fc2567fc640 1 --2- 172.21.15.38:0/4133710977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2580d3900 0x7fc2580d0850 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0097a0 tx=0x7fc24c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a896d3f46a121c4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.677+0000 7fc255ffb640 1 -- 172.21.15.38:0/4133710977 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c037070 con 0x7fc2580d3900 2024-04-25T08:35:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.677+0000 7fc255ffb640 1 -- 172.21.15.38:0/4133710977 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24c02ae50 con 0x7fc2580d3900 2024-04-25T08:35:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.677+0000 7fc255ffb640 1 -- 172.21.15.38:0/4133710977 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24c032660 con 0x7fc2580d3900 2024-04-25T08:35:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.678+0000 7fc25d693640 1 -- 172.21.15.38:0/4133710977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2580d3900 msgr2=0x7fc2580d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.678+0000 7fc25d693640 1 --2- 172.21.15.38:0/4133710977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2580d3900 0x7fc2580d0850 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0097a0 tx=0x7fc24c02a100 comp rx=0 tx=0).stop 2024-04-25T08:35:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.678+0000 7fc25d693640 1 -- 172.21.15.38:0/4133710977 shutdown_connections 2024-04-25T08:35:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.678+0000 7fc25d693640 1 --2- 172.21.15.38:0/4133710977 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2580d0d90 0x7fc2580b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.678+0000 7fc25d693640 1 --2- 172.21.15.38:0/4133710977 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2580d3900 0x7fc2580d0850 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.678+0000 7fc25d693640 1 -- 172.21.15.38:0/4133710977 >> 172.21.15.38:0/4133710977 conn(0x7fc258066d10 msgr2=0x7fc258067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.678+0000 7fc25d693640 1 -- 172.21.15.38:0/4133710977 shutdown_connections 2024-04-25T08:35:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.678+0000 7fc25d693640 1 -- 172.21.15.38:0/4133710977 wait complete. 2024-04-25T08:35:04.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.679+0000 7fc25d693640 1 Processor -- start 2024-04-25T08:35:04.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.679+0000 7fc25d693640 1 -- start start 2024-04-25T08:35:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc25d693640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2580d0d90 0x7fc25815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc25d693640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2580d3900 0x7fc2580c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc25d693640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25815dc10 0x7fc2580c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc25d693640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580b6bc0 con 0x7fc25815dc10 2024-04-25T08:35:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc25d693640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580d4710 con 0x7fc2580d0d90 2024-04-25T08:35:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc25d693640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2580cef30 con 0x7fc2580d3900 2024-04-25T08:35:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc2577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25815dc10 0x7fc2580c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc2577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25815dc10 0x7fc2580c6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42668/0 (socket says 172.21.15.38:42668) 2024-04-25T08:35:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc2577fe640 1 -- 172.21.15.38:0/3323328512 learned_addr learned my addr 172.21.15.38:0/3323328512 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc256ffd640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2580d0d90 0x7fc25815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc2567fc640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2580d3900 0x7fc2580c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc2577fe640 1 -- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2580d3900 msgr2=0x7fc2580c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc2577fe640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2580d3900 0x7fc2580c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc2577fe640 1 -- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2580d0d90 msgr2=0x7fc25815b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc2577fe640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2580d0d90 0x7fc25815b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc2577fe640 1 -- 172.21.15.38:0/3323328512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2580cca60 con 0x7fc25815dc10 2024-04-25T08:35:04.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.680+0000 7fc2577fe640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25815dc10 0x7fc2580c6d90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc25000d5d0 tx=0x7fc25000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:04.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.681+0000 7fc23f7fe640 1 -- 172.21.15.38:0/3323328512 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2500125f0 con 0x7fc25815dc10 2024-04-25T08:35:04.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.681+0000 7fc23f7fe640 1 -- 172.21.15.38:0/3323328512 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc250019070 con 0x7fc25815dc10 2024-04-25T08:35:04.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.681+0000 7fc25d693640 1 -- 172.21.15.38:0/3323328512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2580ccd40 con 0x7fc25815dc10 2024-04-25T08:35:04.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.681+0000 7fc23f7fe640 1 -- 172.21.15.38:0/3323328512 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2500129f0 con 0x7fc25815dc10 2024-04-25T08:35:04.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.681+0000 7fc25d693640 1 -- 172.21.15.38:0/3323328512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2580cd300 con 0x7fc25815dc10 2024-04-25T08:35:04.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.682+0000 7fc25d693640 1 -- 172.21.15.38:0/3323328512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc258068e80 con 0x7fc25815dc10 2024-04-25T08:35:04.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.682+0000 7fc23f7fe640 1 -- 172.21.15.38:0/3323328512 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc250012b90 con 0x7fc25815dc10 2024-04-25T08:35:04.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.682+0000 7fc23f7fe640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc24406b4d0 0x7fc24406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:04.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.683+0000 7fc23f7fe640 1 -- 172.21.15.38:0/3323328512 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc25008a620 con 0x7fc25815dc10 2024-04-25T08:35:04.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.683+0000 7fc256ffd640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc24406b4d0 0x7fc24406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:04.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.683+0000 7fc256ffd640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc24406b4d0 0x7fc24406d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc2480012e0 tx=0x7fc248001700 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:04.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.687+0000 7fc23f7fe640 1 -- 172.21.15.38:0/3323328512 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc250058cd0 con 0x7fc25815dc10 2024-04-25T08:35:04.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.905+0000 7fc25d693640 1 -- 172.21.15.38:0/3323328512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc25815faf0 con 0x7fc25815dc10 2024-04-25T08:35:04.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.905+0000 7fc23f7fe640 1 -- 172.21.15.38:0/3323328512 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fc25005cd70 con 0x7fc25815dc10 2024-04-25T08:35:04.900 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:04.900 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:34:46.222808+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:04.901 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 24 2024-04-25T08:35:04.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.907+0000 7fc25d693640 1 -- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc24406b4d0 msgr2=0x7fc24406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.907+0000 7fc25d693640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc24406b4d0 0x7fc24406d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc2480012e0 tx=0x7fc248001700 comp rx=0 tx=0).stop 2024-04-25T08:35:04.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.907+0000 7fc25d693640 1 -- 172.21.15.38:0/3323328512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25815dc10 msgr2=0x7fc2580c6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:04.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.907+0000 7fc25d693640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25815dc10 0x7fc2580c6d90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc25000d5d0 tx=0x7fc25000da90 comp rx=0 tx=0).stop 2024-04-25T08:35:04.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.907+0000 7fc25d693640 1 -- 172.21.15.38:0/3323328512 shutdown_connections 2024-04-25T08:35:04.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.907+0000 7fc25d693640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc25815dc10 0x7fc2580c6d90 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.907+0000 7fc25d693640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc24406b4d0 0x7fc24406d990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.907+0000 7fc25d693640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2580d3900 0x7fc2580c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.907+0000 7fc25d693640 1 --2- 172.21.15.38:0/3323328512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc2580d0d90 0x7fc25815b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:04.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.907+0000 7fc25d693640 1 -- 172.21.15.38:0/3323328512 >> 172.21.15.38:0/3323328512 conn(0x7fc258066d10 msgr2=0x7fc2580b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:04.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.907+0000 7fc25d693640 1 -- 172.21.15.38:0/3323328512 shutdown_connections 2024-04-25T08:35:04.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:04.907+0000 7fc25d693640 1 -- 172.21.15.38:0/3323328512 wait complete. 2024-04-25T08:35:04.913 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T08:35:05.022 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:05.022 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:35:05.022 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:05.031 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:35:05.031 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T08:35:05.031 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:05.031 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T08:35:05.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.160+0000 7f63bce46640 1 Processor -- start 2024-04-25T08:35:05.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.160+0000 7f63bce46640 1 -- start start 2024-04-25T08:35:05.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.161+0000 7f63bce46640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b8166610 0x7f63b8168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.161+0000 7f63bce46640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b80d2060 con 0x7f63b80d33d0 2024-04-25T08:35:05.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.161+0000 7f63bce46640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b80d2330 con 0x7f63b80d0320 2024-04-25T08:35:05.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.161+0000 7f63bce46640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b80d2610 con 0x7f63b8166610 2024-04-25T08:35:05.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b5d9b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f63b80d33d0 0x7f63b80d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60632/0 (socket says 172.21.15.38:60632) 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b5d9b640 1 -- 172.21.15.38:0/3170340708 learned_addr learned my addr 172.21.15.38:0/3170340708 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b6d9d640 1 --2- 172.21.15.38:0/3170340708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b8166610 0x7f63b8168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202301758 0 0) 0x7f63b80d2060 con 0x7f63b80d33d0 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63a4003520 con 0x7f63b80d33d0 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1909784211 0 0) 0x7f63a4003520 con 0x7f63b80d33d0 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b8166610 msgr2=0x7f63b8168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 --2- 172.21.15.38:0/3170340708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b8166610 0x7f63b8168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 >> v1:172.21.15.107:6789/0 conn(0x7f63b80d0320 legacy=0x7f63b80d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b80d28f0 con 0x7f63b80d33d0 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f63a8001690 con 0x7f63b80d33d0 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 >> v1:172.21.15.38:6789/0 conn(0x7f63b80d33d0 legacy=0x7f63b80d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 --2- 172.21.15.38:0/3170340708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a40039e0 0x7f63a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 --2- 172.21.15.38:0/3170340708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a4004390 0x7f63a4004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 --2- 172.21.15.38:0/3170340708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b8166610 0x7f63a40050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b80d2610 con 0x7f63a40039e0 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a400b9f0 con 0x7f63a4004390 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a400bbf0 con 0x7f63b8166610 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b659c640 1 --2- 172.21.15.38:0/3170340708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a40039e0 0x7f63a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b5d9b640 1 --2- 172.21.15.38:0/3170340708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a4004390 0x7f63a4004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.162+0000 7f63b6d9d640 1 --2- 172.21.15.38:0/3170340708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b8166610 0x7f63a40050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.163+0000 7f63b659c640 1 -- 172.21.15.38:0/3170340708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b8166610 msgr2=0x7f63a40050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.163+0000 7f63b659c640 1 --2- 172.21.15.38:0/3170340708 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b8166610 0x7f63a40050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.163+0000 7f63b659c640 1 -- 172.21.15.38:0/3170340708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a4004390 msgr2=0x7f63a4004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.163+0000 7f63b659c640 1 --2- 172.21.15.38:0/3170340708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63a4004390 0x7f63a4004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.163+0000 7f63b659c640 1 -- 172.21.15.38:0/3170340708 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a4003520 con 0x7f63a40039e0 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.163+0000 7f63b659c640 1 --2- 172.21.15.38:0/3170340708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a40039e0 0x7f63a4003dc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f63a0001860 tx=0x7f63a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9becad7f2f689de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.163+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0037070 con 0x7f63a40039e0 2024-04-25T08:35:05.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.163+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a002ac30 con 0x7f63a40039e0 2024-04-25T08:35:05.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.163+0000 7f63b559a640 1 -- 172.21.15.38:0/3170340708 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0001430 con 0x7f63a40039e0 2024-04-25T08:35:05.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.164+0000 7f63bce46640 1 -- 172.21.15.38:0/3170340708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a40039e0 msgr2=0x7f63a4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.164+0000 7f63bce46640 1 --2- 172.21.15.38:0/3170340708 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a40039e0 0x7f63a4003dc0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f63a0001860 tx=0x7f63a4005830 comp rx=0 tx=0).stop 2024-04-25T08:35:05.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.164+0000 7f63b6d9d640 1 -- 172.21.15.38:0/3170340708 reap_dead start 2024-04-25T08:35:05.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.164+0000 7f63bce46640 1 -- 172.21.15.38:0/3170340708 shutdown_connections 2024-04-25T08:35:05.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.164+0000 7f63bce46640 1 -- 172.21.15.38:0/3170340708 >> 172.21.15.38:0/3170340708 conn(0x7f63b80668f0 msgr2=0x7f63b8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.164+0000 7f63bce46640 1 -- 172.21.15.38:0/3170340708 shutdown_connections 2024-04-25T08:35:05.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.164+0000 7f63bce46640 1 -- 172.21.15.38:0/3170340708 wait complete. 2024-04-25T08:35:05.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.165+0000 7f63bce46640 1 Processor -- start 2024-04-25T08:35:05.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.166+0000 7f63bce46640 1 -- start start 2024-04-25T08:35:05.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.166+0000 7f63bce46640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a40039e0 0x7f63b814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.166+0000 7f63bce46640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b814e500 0x7f63b815cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.166+0000 7f63bce46640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b80b3f30 0x7f63b8156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.166+0000 7f63bce46640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b8161210 con 0x7f63a40039e0 2024-04-25T08:35:05.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.166+0000 7f63bce46640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b80d2eb0 con 0x7f63b814e500 2024-04-25T08:35:05.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.166+0000 7f63bce46640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b8161390 con 0x7f63b80b3f30 2024-04-25T08:35:05.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.166+0000 7f63b659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a40039e0 0x7f63b814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.166+0000 7f63b659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a40039e0 0x7f63b814dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42680/0 (socket says 172.21.15.38:42680) 2024-04-25T08:35:05.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.166+0000 7f63b659c640 1 -- 172.21.15.38:0/4014836192 learned_addr learned my addr 172.21.15.38:0/4014836192 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.166+0000 7f63b5d9b640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b814e500 0x7f63b815cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.166+0000 7f63b6d9d640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b80b3f30 0x7f63b8156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63b659c640 1 -- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b80b3f30 msgr2=0x7f63b8156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63b659c640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b80b3f30 0x7f63b8156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63b659c640 1 -- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b814e500 msgr2=0x7f63b815cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63b659c640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b814e500 0x7f63b815cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63b659c640 1 -- 172.21.15.38:0/4014836192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b815d3f0 con 0x7f63a40039e0 2024-04-25T08:35:05.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63b5d9b640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b814e500 0x7f63b815cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:05.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63b6d9d640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b80b3f30 0x7f63b8156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:05.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63b659c640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a40039e0 0x7f63b814dd50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f63a0001980 tx=0x7f63a0001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63aeffd640 1 -- 172.21.15.38:0/4014836192 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0040020 con 0x7f63a40039e0 2024-04-25T08:35:05.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63aeffd640 1 -- 172.21.15.38:0/4014836192 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a0005270 con 0x7f63a40039e0 2024-04-25T08:35:05.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63bce46640 1 -- 172.21.15.38:0/4014836192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b815d6d0 con 0x7f63a40039e0 2024-04-25T08:35:05.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63aeffd640 1 -- 172.21.15.38:0/4014836192 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0037040 con 0x7f63a40039e0 2024-04-25T08:35:05.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.167+0000 7f63bce46640 1 -- 172.21.15.38:0/4014836192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b80cc230 con 0x7f63a40039e0 2024-04-25T08:35:05.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.168+0000 7f63aeffd640 1 -- 172.21.15.38:0/4014836192 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63a0043020 con 0x7f63a40039e0 2024-04-25T08:35:05.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.169+0000 7f63aeffd640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f639806b500 0x7f639806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.169+0000 7f63aeffd640 1 -- 172.21.15.38:0/4014836192 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63a00aa340 con 0x7f63a40039e0 2024-04-25T08:35:05.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.169+0000 7f63b5d9b640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f639806b500 0x7f639806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.170+0000 7f63b5d9b640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f639806b500 0x7f639806d9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f63b815e610 tx=0x7f63a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.170+0000 7f63bfc22740 1 Processor -- start 2024-04-25T08:35:05.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.170+0000 7f63bfc22740 1 -- start start 2024-04-25T08:35:05.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63bfc22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63bfc22740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b17f130 0x56384b1c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63bfc22740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 0x56384b1c2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63bfc22740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384b0e1e60 con 0x56384b1a5af0 2024-04-25T08:35:05.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63bfc22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384b08b120 con 0x56384b1c4f60 2024-04-25T08:35:05.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63bfc22740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56384b0ddf00 con 0x56384b17f130 2024-04-25T08:35:05.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63b6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 0x56384b1c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63b6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 0x56384b1c2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42688/0 (socket says 172.21.15.38:42688) 2024-04-25T08:35:05.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63b6d9d640 1 -- 172.21.15.38:0/3973875311 learned_addr learned my addr 172.21.15.38:0/3973875311 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63b5d9b640 1 --2- 172.21.15.38:0/3973875311 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b17f130 0x56384b1c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63b659c640 1 --2- 172.21.15.38:0/3973875311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63b6d9d640 1 -- 172.21.15.38:0/3973875311 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b17f130 msgr2=0x56384b1c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63b6d9d640 1 --2- 172.21.15.38:0/3973875311 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b17f130 0x56384b1c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63b6d9d640 1 -- 172.21.15.38:0/3973875311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 msgr2=0x56384b1c31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63b6d9d640 1 --2- 172.21.15.38:0/3973875311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1c31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63b6d9d640 1 -- 172.21.15.38:0/3973875311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56384aec2f00 con 0x56384b1a5af0 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.171+0000 7f63b6d9d640 1 --2- 172.21.15.38:0/3973875311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 0x56384b1c2590 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f63b80675b0 tx=0x7f63b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194f6400f11fc6e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.172+0000 7f6397fff640 1 -- 172.21.15.38:0/3973875311 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0016840 con 0x56384b1a5af0 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.172+0000 7f6397fff640 1 -- 172.21.15.38:0/3973875311 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b001f070 con 0x56384b1a5af0 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.172+0000 7f6397fff640 1 -- 172.21.15.38:0/3973875311 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0016c20 con 0x56384b1a5af0 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.172+0000 7f63bfc22740 1 -- 172.21.15.38:0/3973875311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 msgr2=0x56384b1c2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.172+0000 7f63bfc22740 1 --2- 172.21.15.38:0/3973875311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 0x56384b1c2590 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f63b80675b0 tx=0x7f63b0009750 comp rx=0 tx=0).stop 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.172+0000 7f63bfc22740 1 -- 172.21.15.38:0/3973875311 shutdown_connections 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.172+0000 7f63bfc22740 1 --2- 172.21.15.38:0/3973875311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 0x56384b1c2590 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.172+0000 7f63bfc22740 1 --2- 172.21.15.38:0/3973875311 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b17f130 0x56384b1c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.172+0000 7f63bfc22740 1 --2- 172.21.15.38:0/3973875311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1c31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.172+0000 7f63bfc22740 1 -- 172.21.15.38:0/3973875311 >> 172.21.15.38:0/3973875311 conn(0x56384b145f70 msgr2=0x56384b17c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.172+0000 7f63bfc22740 1 -- 172.21.15.38:0/3973875311 shutdown_connections 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.172+0000 7f63bfc22740 1 -- 172.21.15.38:0/3973875311 wait complete. 2024-04-25T08:35:05.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.173+0000 7f63bfc22740 1 -- start start 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.173+0000 7f63bfc22740 1 Processor -- start 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63bfc22740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 0x56384b191b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63bfc22740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b1a5af0 0x56384b1c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63bfc22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1c3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63bfc22740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384b1c4760 con 0x56384b17f130 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63bfc22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384b080cd0 con 0x56384b1c4f60 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63bfc22740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56384b1c4480 con 0x56384b1a5af0 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63bfc22740 10 client.0 _ll_register_callbacks cb 0x56384adf62e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63b659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 0x56384b191b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63b659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 0x56384b191b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42700/0 (socket says 172.21.15.38:42700) 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63b659c640 1 -- 172.21.15.38:0/3621087564 learned_addr learned my addr 172.21.15.38:0/3621087564 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63b5d9b640 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b1a5af0 0x56384b1c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63b6d9d640 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63b659c640 1 -- 172.21.15.38:0/3621087564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b1a5af0 msgr2=0x56384b1c2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63b659c640 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b1a5af0 0x56384b1c2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63b659c640 1 -- 172.21.15.38:0/3621087564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 msgr2=0x56384b1c3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63b659c640 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1c3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63b659c640 1 -- 172.21.15.38:0/3621087564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56384adb67f0 con 0x56384b17f130 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63b6d9d640 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63b659c640 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 0x56384b191b70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f63a0074eb0 tx=0x7f63a00791d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99bb57e2ce22f9da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.174+0000 7f63957fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63957fa640 1 -- 172.21.15.38:0/3621087564 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0081160 con 0x56384b17f130 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63957fa640 1 -- 172.21.15.38:0/3621087564 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63a0078cb0 con 0x56384b17f130 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63bfc22740 1 -- 172.21.15.38:0/3621087564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63b001f070 con 0x56384b17f130 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63957fa640 1 -- 172.21.15.38:0/3621087564 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63a0078ed0 con 0x56384b17f130 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63bfc22740 10 client.6225 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63bfc22740 1 -- 172.21.15.38:0/3621087564 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56384af23690 con 0x56384b17f130 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63957fa640 1 -- 172.21.15.38:0/3621087564 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b001f070 con 0x56384b17f130 2024-04-25T08:35:05.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63957fa640 1 -- 172.21.15.38:0/3621087564 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f63a0080cd0 con 0x56384b17f130 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63bfc22740 10 client.6225 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63bfc22740 10 client.6225 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63bfc22740 20 client.6225 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63bfc22740 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56384b1a7700 0x56384b1a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63bfc22740 4 client.6225 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63bfc22740 1 -- 172.21.15.38:0/3621087564 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56384b0ef3a0 con 0x56384b1a7700 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.175+0000 7f63b5d9b640 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56384b1a7700 0x56384b1a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63957fa640 10 client.6225 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63b5d9b640 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56384b1a7700 0x56384b1a7b30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e6b4bb272c0fc6c server_cookie=9e25a4ea7a48190c in_seq=0 out_seq=0 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63957fa640 1 -- 172.21.15.38:0/3621087564 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56384b0ef3a0 con 0x56384b1a7700 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63957fa640 10 client.6225 handle_command_reply: tid=0 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63bfc22740 1 client.6225 shutdown 2024-04-25T08:35:05.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63bfc22740 1 -- 172.21.15.38:0/3621087564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 msgr2=0x56384b191b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63bfc22740 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 0x56384b191b70 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f63a0074eb0 tx=0x7f63a00791d0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63bfc22740 1 -- 172.21.15.38:0/3621087564 shutdown_connections 2024-04-25T08:35:05.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63bfc22740 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56384b1a7700 0x56384b1a7b30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63bfc22740 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63bfc22740 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b1a5af0 0x56384b1c2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63bfc22740 1 --2- 172.21.15.38:0/3621087564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 0x56384b191b70 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63957fa640 0 client.6225 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:05.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63bfc22740 1 -- 172.21.15.38:0/3621087564 >> 172.21.15.38:0/3621087564 conn(0x56384b145f70 msgr2=0x56384b1a4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63bfc22740 1 -- 172.21.15.38:0/3621087564 shutdown_connections 2024-04-25T08:35:05.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.176+0000 7f63bfc22740 1 -- 172.21.15.38:0/3621087564 wait complete. 2024-04-25T08:35:05.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.177+0000 7f63bfc22740 20 client.6225 trim_cache size 0 max 16384 2024-04-25T08:35:05.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.192+0000 7f63bfc22740 1 Processor -- start 2024-04-25T08:35:05.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.192+0000 7f63bfc22740 1 -- start start 2024-04-25T08:35:05.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.192+0000 7f63bfc22740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 0x56384b1915a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.192+0000 7f63bfc22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1a5af0 0x56384b1c6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.192+0000 7f63bfc22740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b1c4f60 0x56384b1c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.192+0000 7f63bfc22740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56384b17e3d0 con 0x56384b17f130 2024-04-25T08:35:05.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.192+0000 7f63bfc22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b82008e0 con 0x56384b1a5af0 2024-04-25T08:35:05.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.192+0000 7f63bfc22740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b81ffae0 con 0x56384b1c4f60 2024-04-25T08:35:05.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.192+0000 7f63b659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 0x56384b1915a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.192+0000 7f63b659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 0x56384b1915a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42708/0 (socket says 172.21.15.38:42708) 2024-04-25T08:35:05.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.192+0000 7f63b659c640 1 -- 172.21.15.38:0/1963172636 learned_addr learned my addr 172.21.15.38:0/1963172636 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63b659c640 1 -- 172.21.15.38:0/1963172636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b1c4f60 msgr2=0x56384b1c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63b659c640 1 --2- 172.21.15.38:0/1963172636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b1c4f60 0x56384b1c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63b659c640 1 -- 172.21.15.38:0/1963172636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1a5af0 msgr2=0x56384b1c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63b659c640 1 --2- 172.21.15.38:0/1963172636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1a5af0 0x56384b1c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63b659c640 1 -- 172.21.15.38:0/1963172636 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56384ad8f8b0 con 0x56384b17f130 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63b659c640 1 --2- 172.21.15.38:0/1963172636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 0x56384b1915a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f63a002a6c0 tx=0x7f63a003e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a472bd564034599 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63977fe640 1 -- 172.21.15.38:0/1963172636 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0081160 con 0x56384b17f130 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63977fe640 1 -- 172.21.15.38:0/1963172636 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a00aa0b0 con 0x56384b17f130 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63977fe640 1 -- 172.21.15.38:0/1963172636 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0078820 con 0x56384b17f130 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63bfc22740 1 -- 172.21.15.38:0/1963172636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 msgr2=0x56384b1915a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63bfc22740 1 --2- 172.21.15.38:0/1963172636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 0x56384b1915a0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f63a002a6c0 tx=0x7f63a003e070 comp rx=0 tx=0).stop 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63bfc22740 1 -- 172.21.15.38:0/1963172636 shutdown_connections 2024-04-25T08:35:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63bfc22740 1 --2- 172.21.15.38:0/1963172636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b1c4f60 0x56384b1c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63bfc22740 1 --2- 172.21.15.38:0/1963172636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1a5af0 0x56384b1c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.193+0000 7f63bfc22740 1 --2- 172.21.15.38:0/1963172636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b17f130 0x56384b1915a0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.194+0000 7f63bfc22740 1 -- 172.21.15.38:0/1963172636 >> 172.21.15.38:0/1963172636 conn(0x56384b145f70 msgr2=0x56384b1a5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.194+0000 7f63bfc22740 1 -- 172.21.15.38:0/1963172636 shutdown_connections 2024-04-25T08:35:05.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.194+0000 7f63bfc22740 1 -- 172.21.15.38:0/1963172636 wait complete. 2024-04-25T08:35:05.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.194+0000 7f63bfc22740 1 -- start start 2024-04-25T08:35:05.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.194+0000 7f63bfc22740 1 Processor -- start 2024-04-25T08:35:05.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63bfc22740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b17f130 0x56384b191240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63bfc22740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 0x56384b22d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63bfc22740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63bfc22740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b81ff4e0 con 0x56384b1a5af0 2024-04-25T08:35:05.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63bfc22740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b81feee0 con 0x56384b1c4f60 2024-04-25T08:35:05.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63bfc22740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56384b17e3d0 con 0x56384b17f130 2024-04-25T08:35:05.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63bfc22740 10 client.0 _ll_register_callbacks cb 0x7f63b00092d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63b5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 0x56384b22d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63b5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 0x56384b22d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42710/0 (socket says 172.21.15.38:42710) 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63b5d9b640 1 -- 172.21.15.38:0/2976723171 learned_addr learned my addr 172.21.15.38:0/2976723171 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63b6d9d640 1 --2- 172.21.15.38:0/2976723171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63b5d9b640 1 -- 172.21.15.38:0/2976723171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b17f130 msgr2=0x56384b191240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63b5d9b640 1 --2- 172.21.15.38:0/2976723171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b17f130 0x56384b191240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63b5d9b640 1 -- 172.21.15.38:0/2976723171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 msgr2=0x56384b1a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63b5d9b640 1 --2- 172.21.15.38:0/2976723171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1a8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.195+0000 7f63b5d9b640 1 -- 172.21.15.38:0/2976723171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56384b1c0d30 con 0x56384b1a5af0 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63b5d9b640 1 --2- 172.21.15.38:0/2976723171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 0x56384b22d280 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f63a80016a0 tx=0x7f63a80078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d829cf7c61461db5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63957fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63957fa640 1 -- 172.21.15.38:0/2976723171 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a802ab70 con 0x56384b1a5af0 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63957fa640 1 -- 172.21.15.38:0/2976723171 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63a802ad10 con 0x56384b1a5af0 2024-04-25T08:35:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63bfc22740 1 -- 172.21.15.38:0/2976723171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63a00aa0b0 con 0x56384b1a5af0 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63957fa640 1 -- 172.21.15.38:0/2976723171 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63a8037050 con 0x56384b1a5af0 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63bfc22740 10 client.6231 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63bfc22740 1 -- 172.21.15.38:0/2976723171 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56384b22db10 con 0x56384b1a5af0 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63957fa640 1 -- 172.21.15.38:0/2976723171 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a00aa0b0 con 0x56384b1a5af0 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63957fa640 1 -- 172.21.15.38:0/2976723171 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f63a803a020 con 0x56384b1a5af0 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63bfc22740 10 client.6231 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63bfc22740 10 client.6231 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63bfc22740 20 client.6231 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63bfc22740 1 --2- 172.21.15.38:0/2976723171 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56384b1d2040 0x56384b1a8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63bfc22740 4 client.6231 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63bfc22740 1 -- 172.21.15.38:0/2976723171 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x56384b1c5f50 con 0x56384b1d2040 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.196+0000 7f63b659c640 1 --2- 172.21.15.38:0/2976723171 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56384b1d2040 0x56384b1a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.197+0000 7f63957fa640 10 client.6231 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:05.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.197+0000 7f63b659c640 1 --2- 172.21.15.38:0/2976723171 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56384b1d2040 0x56384b1a8540 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6864c44c3d95554b server_cookie=15d34f9df0d335b6 in_seq=0 out_seq=0 2024-04-25T08:35:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.197+0000 7f63957fa640 1 -- 172.21.15.38:0/2976723171 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56384b1c5f50 con 0x56384b1d2040 2024-04-25T08:35:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.197+0000 7f63957fa640 10 client.6231 handle_command_reply: tid=0 2024-04-25T08:35:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.197+0000 7f63bfc22740 1 client.6231 shutdown 2024-04-25T08:35:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bfc22740 1 -- 172.21.15.38:0/2976723171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 msgr2=0x56384b22d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bfc22740 1 --2- 172.21.15.38:0/2976723171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 0x56384b22d280 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f63a80016a0 tx=0x7f63a80078f0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bfc22740 1 -- 172.21.15.38:0/2976723171 shutdown_connections 2024-04-25T08:35:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bfc22740 1 --2- 172.21.15.38:0/2976723171 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x56384b1d2040 0x56384b1a8540 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bfc22740 1 --2- 172.21.15.38:0/2976723171 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56384b1c4f60 0x56384b1a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bfc22740 1 --2- 172.21.15.38:0/2976723171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56384b1a5af0 0x56384b22d280 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bfc22740 1 --2- 172.21.15.38:0/2976723171 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56384b17f130 0x56384b191240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63957fa640 0 client.6231 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bfc22740 1 -- 172.21.15.38:0/2976723171 >> 172.21.15.38:0/2976723171 conn(0x56384b145f70 msgr2=0x56384b1e4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bfc22740 1 -- 172.21.15.38:0/2976723171 shutdown_connections 2024-04-25T08:35:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bfc22740 1 -- 172.21.15.38:0/2976723171 wait complete. 2024-04-25T08:35:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bfc22740 20 client.6231 trim_cache size 0 max 16384 2024-04-25T08:35:05.199 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:05.199 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:35:05.199 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "f83cb3ff-e58e-4574-a66c-e80b3b7d2da7", 2024-04-25T08:35:05.199 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:35:05.200 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:05.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bce46640 1 -- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f639806b500 msgr2=0x7f639806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bce46640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f639806b500 0x7f639806d9c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f63b815e610 tx=0x7f63a8009040 comp rx=0 tx=0).stop 2024-04-25T08:35:05.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bce46640 1 -- 172.21.15.38:0/4014836192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a40039e0 msgr2=0x7f63b814dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bce46640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a40039e0 0x7f63b814dd50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f63a0001980 tx=0x7f63a0001230 comp rx=0 tx=0).stop 2024-04-25T08:35:05.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bce46640 1 -- 172.21.15.38:0/4014836192 shutdown_connections 2024-04-25T08:35:05.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bce46640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f639806b500 0x7f639806d9c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bce46640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63b80b3f30 0x7f63b8156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bce46640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63b814e500 0x7f63b815cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bce46640 1 --2- 172.21.15.38:0/4014836192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a40039e0 0x7f63b814dd50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bce46640 1 -- 172.21.15.38:0/4014836192 >> 172.21.15.38:0/4014836192 conn(0x7f63b80668f0 msgr2=0x7f63b81604d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bce46640 1 -- 172.21.15.38:0/4014836192 shutdown_connections 2024-04-25T08:35:05.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.198+0000 7f63bce46640 1 -- 172.21.15.38:0/4014836192 wait complete. 2024-04-25T08:35:05.204 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'f83cb3ff-e58e-4574-a66c-e80b3b7d2da7', 'mode': 'asynchronous'}' 2024-04-25T08:35:05.204 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-25T08:35:05.204 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T08:35:05.204 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:05.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.311+0000 7f1832d5c640 1 Processor -- start 2024-04-25T08:35:05.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.312+0000 7f1832d5c640 1 -- start start 2024-04-25T08:35:05.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.312+0000 7f1832d5c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f182c15da90 0x7f182c0d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.312+0000 7f1832d5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f182c0ceca0 0x7f182c0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.312+0000 7f1832d5c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c0d1b60 con 0x7f182c0d3de0 2024-04-25T08:35:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.312+0000 7f1832d5c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c0d2bc0 con 0x7f182c15da90 2024-04-25T08:35:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.312+0000 7f1832d5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0d1e00 con 0x7f182c0ceca0 2024-04-25T08:35:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.312+0000 7f1830af8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f182c15da90 0x7f182c0d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.312+0000 7f18312f9640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f182c0d3de0 0x7f182c0d0320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60638/0 (socket says 172.21.15.38:60638) 2024-04-25T08:35:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.312+0000 7f18312f9640 1 -- 172.21.15.38:0/1122001572 learned_addr learned my addr 172.21.15.38:0/1122001572 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.313+0000 7f182bfff640 1 --2- 172.21.15.38:0/1122001572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f182c0ceca0 0x7f182c0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.313+0000 7f1830af8640 1 -- 172.21.15.38:0/1122001572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f182c0ceca0 msgr2=0x7f182c0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.313+0000 7f1830af8640 1 --2- 172.21.15.38:0/1122001572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f182c0ceca0 0x7f182c0d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.313+0000 7f1830af8640 1 -- 172.21.15.38:0/1122001572 >> v1:172.21.15.38:6789/0 conn(0x7f182c0d3de0 legacy=0x7f182c0d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.313+0000 7f1830af8640 1 -- 172.21.15.38:0/1122001572 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0d20e0 con 0x7f182c15da90 2024-04-25T08:35:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.313+0000 7f182b7fe640 1 -- 172.21.15.38:0/1122001572 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907451149 0 0) 0x7f182c0d1b60 con 0x7f182c0d3de0 2024-04-25T08:35:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.313+0000 7f1830af8640 1 --2- 172.21.15.38:0/1122001572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f182c15da90 0x7f182c0d2f70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f18140097a0 tx=0x7f181402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6336faecd04d3be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.314+0000 7f182b7fe640 1 -- 172.21.15.38:0/1122001572 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814037070 con 0x7f182c15da90 2024-04-25T08:35:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.314+0000 7f182b7fe640 1 -- 172.21.15.38:0/1122001572 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1814032440 con 0x7f182c15da90 2024-04-25T08:35:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.314+0000 7f182b7fe640 1 -- 172.21.15.38:0/1122001572 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1814032810 con 0x7f182c15da90 2024-04-25T08:35:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.314+0000 7f1832d5c640 1 -- 172.21.15.38:0/1122001572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f182c15da90 msgr2=0x7f182c0d2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.314+0000 7f1832d5c640 1 --2- 172.21.15.38:0/1122001572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f182c15da90 0x7f182c0d2f70 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f18140097a0 tx=0x7f181402a180 comp rx=0 tx=0).stop 2024-04-25T08:35:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.314+0000 7f1832d5c640 1 -- 172.21.15.38:0/1122001572 shutdown_connections 2024-04-25T08:35:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.314+0000 7f1832d5c640 1 --2- 172.21.15.38:0/1122001572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f182c0ceca0 0x7f182c0d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.314+0000 7f1832d5c640 1 --2- 172.21.15.38:0/1122001572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f182c15da90 0x7f182c0d2f70 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.314+0000 7f1832d5c640 1 -- 172.21.15.38:0/1122001572 >> 172.21.15.38:0/1122001572 conn(0x7f182c06bed0 msgr2=0x7f182c06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.314+0000 7f1832d5c640 1 -- 172.21.15.38:0/1122001572 shutdown_connections 2024-04-25T08:35:05.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.314+0000 7f1832d5c640 1 -- 172.21.15.38:0/1122001572 wait complete. 2024-04-25T08:35:05.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.315+0000 7f1832d5c640 1 Processor -- start 2024-04-25T08:35:05.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.315+0000 7f1832d5c640 1 -- start start 2024-04-25T08:35:05.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.316+0000 7f1832d5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f182c0d3de0 0x7f182c14f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.316+0000 7f1832d5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f182c15da90 0x7f182c0cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.316+0000 7f1832d5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f182c14c450 0x7f182c0b85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.316+0000 7f1832d5c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c15dff0 con 0x7f182c14c450 2024-04-25T08:35:05.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.316+0000 7f1832d5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c15de70 con 0x7f182c0d3de0 2024-04-25T08:35:05.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.316+0000 7f1832d5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0d34b0 con 0x7f182c15da90 2024-04-25T08:35:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.316+0000 7f18312f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f182c14c450 0x7f182c0b85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.316+0000 7f18312f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f182c14c450 0x7f182c0b85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42726/0 (socket says 172.21.15.38:42726) 2024-04-25T08:35:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.316+0000 7f18312f9640 1 -- 172.21.15.38:0/1055541006 learned_addr learned my addr 172.21.15.38:0/1055541006 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f1830af8640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f182c0d3de0 0x7f182c14f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f182bfff640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f182c15da90 0x7f182c0cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f18312f9640 1 -- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f182c15da90 msgr2=0x7f182c0cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f18312f9640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f182c15da90 0x7f182c0cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f18312f9640 1 -- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f182c0d3de0 msgr2=0x7f182c14f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f18312f9640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f182c0d3de0 0x7f182c14f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f18312f9640 1 -- 172.21.15.38:0/1055541006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0cdb60 con 0x7f182c14c450 2024-04-25T08:35:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f1830af8640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f182c0d3de0 0x7f182c14f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f18312f9640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f182c14c450 0x7f182c0b85d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f182400cb20 tx=0x7f182400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f18297fa640 1 -- 172.21.15.38:0/1055541006 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400aca0 con 0x7f182c14c450 2024-04-25T08:35:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f18297fa640 1 -- 172.21.15.38:0/1055541006 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1824012400 con 0x7f182c14c450 2024-04-25T08:35:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f1832d5c640 1 -- 172.21.15.38:0/1055541006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c14ce00 con 0x7f182c14c450 2024-04-25T08:35:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f18297fa640 1 -- 172.21.15.38:0/1055541006 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400ae60 con 0x7f182c14c450 2024-04-25T08:35:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.317+0000 7f1832d5c640 1 -- 172.21.15.38:0/1055541006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c14d3c0 con 0x7f182c14c450 2024-04-25T08:35:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.318+0000 7f1832d5c640 1 -- 172.21.15.38:0/1055541006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c068e80 con 0x7f182c14c450 2024-04-25T08:35:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.319+0000 7f18297fa640 1 -- 172.21.15.38:0/1055541006 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18240127f0 con 0x7f182c14c450 2024-04-25T08:35:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.319+0000 7f18297fa640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f180c06b4d0 0x7f180c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.319+0000 7f18297fa640 1 -- 172.21.15.38:0/1055541006 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f182408a9b0 con 0x7f182c14c450 2024-04-25T08:35:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.319+0000 7f1830af8640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f180c06b4d0 0x7f180c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.320+0000 7f1830af8640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f180c06b4d0 0x7f180c06d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1814034000 tx=0x7f181402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.323+0000 7f18297fa640 1 -- 172.21.15.38:0/1055541006 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1824059730 con 0x7f182c14c450 2024-04-25T08:35:05.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.546+0000 7f1832d5c640 1 -- 172.21.15.38:0/1055541006 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f182c160fb0 con 0x7f182c14c450 2024-04-25T08:35:05.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.547+0000 7f18297fa640 1 -- 172.21.15.38:0/1055541006 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f1824059030 con 0x7f182c14c450 2024-04-25T08:35:05.542 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:05.542 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:34:46.222808+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:05.542 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 24 2024-04-25T08:35:05.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.549+0000 7f1832d5c640 1 -- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f180c06b4d0 msgr2=0x7f180c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.549+0000 7f1832d5c640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f180c06b4d0 0x7f180c06d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1814034000 tx=0x7f181402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:05.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.549+0000 7f1832d5c640 1 -- 172.21.15.38:0/1055541006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f182c14c450 msgr2=0x7f182c0b85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.549+0000 7f1832d5c640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f182c14c450 0x7f182c0b85d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f182400cb20 tx=0x7f182400a060 comp rx=0 tx=0).stop 2024-04-25T08:35:05.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.549+0000 7f1832d5c640 1 -- 172.21.15.38:0/1055541006 shutdown_connections 2024-04-25T08:35:05.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.549+0000 7f1832d5c640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f182c14c450 0x7f182c0b85d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.549+0000 7f1832d5c640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f180c06b4d0 0x7f180c06d990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.549+0000 7f1832d5c640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f182c15da90 0x7f182c0cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.549+0000 7f1832d5c640 1 --2- 172.21.15.38:0/1055541006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f182c0d3de0 0x7f182c14f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.549+0000 7f1832d5c640 1 -- 172.21.15.38:0/1055541006 >> 172.21.15.38:0/1055541006 conn(0x7f182c06bed0 msgr2=0x7f182c15cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.549+0000 7f1832d5c640 1 -- 172.21.15.38:0/1055541006 shutdown_connections 2024-04-25T08:35:05.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.549+0000 7f1832d5c640 1 -- 172.21.15.38:0/1055541006 wait complete. 2024-04-25T08:35:05.555 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T08:35:05.663 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:05.664 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:35:05.664 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:05.672 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:35:05.672 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T08:35:05.672 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:05.672 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T08:35:05.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.781+0000 7fb849968640 1 Processor -- start 2024-04-25T08:35:05.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.781+0000 7fb849968640 1 -- start start 2024-04-25T08:35:05.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.782+0000 7fb849968640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb8440cc2c0 0x7fb8440cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.782+0000 7fb849968640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb8441635e0 0x7fb8441659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.782+0000 7fb849968640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8440ccc80 con 0x7fb84406abe0 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.782+0000 7fb849968640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84415c1b0 con 0x7fb8441635e0 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.782+0000 7fb849968640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8440ccf30 con 0x7fb8440cc2c0 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.782+0000 7fb8437fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb8441635e0 0x7fb8441659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.782+0000 7fb8437fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb8441635e0 0x7fb8441659d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42742/0 (socket says 172.21.15.38:42742) 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.782+0000 7fb8437fe640 1 -- 172.21.15.38:0/1657620353 learned_addr learned my addr 172.21.15.38:0/1657620353 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.782+0000 7fb8427fc640 1 --2- 172.21.15.38:0/1657620353 >> v2:172.21.15.107:3300/0 conn(0x7fb8440cc2c0 0x7fb8440cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.783+0000 7fb8437fe640 1 -- 172.21.15.38:0/1657620353 >> v2:172.21.15.107:3300/0 conn(0x7fb8440cc2c0 msgr2=0x7fb8440cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.783+0000 7fb8437fe640 1 --2- 172.21.15.38:0/1657620353 >> v2:172.21.15.107:3300/0 conn(0x7fb8440cc2c0 0x7fb8440cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.783+0000 7fb8437fe640 1 -- 172.21.15.38:0/1657620353 >> v1:172.21.15.107:6789/0 conn(0x7fb84406abe0 legacy=0x7fb84406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.783+0000 7fb8437fe640 1 -- 172.21.15.38:0/1657620353 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8440cd210 con 0x7fb8441635e0 2024-04-25T08:35:05.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.783+0000 7fb8427fc640 1 --2- 172.21.15.38:0/1657620353 >> v2:172.21.15.107:3300/0 conn(0x7fb8440cc2c0 0x7fb8440cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.783+0000 7fb8437fe640 1 --2- 172.21.15.38:0/1657620353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8441635e0 0x7fb8441659d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb844067620 tx=0x7fb83c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ebef8139c3e2c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.783+0000 7fb841ffb640 1 -- 172.21.15.38:0/1657620353 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c037070 con 0x7fb8441635e0 2024-04-25T08:35:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.783+0000 7fb841ffb640 1 -- 172.21.15.38:0/1657620353 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c032440 con 0x7fb8441635e0 2024-04-25T08:35:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.783+0000 7fb841ffb640 1 -- 172.21.15.38:0/1657620353 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c032810 con 0x7fb8441635e0 2024-04-25T08:35:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.784+0000 7fb849968640 1 -- 172.21.15.38:0/1657620353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8441635e0 msgr2=0x7fb8441659d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.784+0000 7fb849968640 1 --2- 172.21.15.38:0/1657620353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8441635e0 0x7fb8441659d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb844067620 tx=0x7fb83c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.784+0000 7fb849968640 1 -- 172.21.15.38:0/1657620353 shutdown_connections 2024-04-25T08:35:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.784+0000 7fb849968640 1 --2- 172.21.15.38:0/1657620353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8441635e0 0x7fb8441659d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.784+0000 7fb849968640 1 --2- 172.21.15.38:0/1657620353 >> v2:172.21.15.107:3300/0 conn(0x7fb8440cc2c0 0x7fb8440cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.784+0000 7fb849968640 1 -- 172.21.15.38:0/1657620353 >> 172.21.15.38:0/1657620353 conn(0x7fb8440d06b0 msgr2=0x7fb8440d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.784+0000 7fb849968640 1 -- 172.21.15.38:0/1657620353 shutdown_connections 2024-04-25T08:35:05.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.784+0000 7fb849968640 1 -- 172.21.15.38:0/1657620353 wait complete. 2024-04-25T08:35:05.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.785+0000 7fb849968640 1 Processor -- start 2024-04-25T08:35:05.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.785+0000 7fb849968640 1 -- start start 2024-04-25T08:35:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.786+0000 7fb849968640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb84406abe0 0x7fb844152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.786+0000 7fb849968640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8440cc2c0 0x7fb844161d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.786+0000 7fb849968640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8441635e0 0x7fb8441622a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.786+0000 7fb849968640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb844159790 con 0x7fb8441635e0 2024-04-25T08:35:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.786+0000 7fb849968640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440cd7d0 con 0x7fb8440cc2c0 2024-04-25T08:35:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.786+0000 7fb849968640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb844159910 con 0x7fb84406abe0 2024-04-25T08:35:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.786+0000 7fb8437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8441635e0 0x7fb8441622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.786+0000 7fb8437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8441635e0 0x7fb8441622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42746/0 (socket says 172.21.15.38:42746) 2024-04-25T08:35:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.786+0000 7fb8437fe640 1 -- 172.21.15.38:0/798816200 learned_addr learned my addr 172.21.15.38:0/798816200 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.786+0000 7fb8427fc640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8440cc2c0 0x7fb844161d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb8437fe640 1 -- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb84406abe0 msgr2=0x7fb844152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb842ffd640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb84406abe0 0x7fb844152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb8437fe640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb84406abe0 0x7fb844152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb8437fe640 1 -- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8440cc2c0 msgr2=0x7fb844161d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb8437fe640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8440cc2c0 0x7fb844161d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb8437fe640 1 -- 172.21.15.38:0/798816200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84415d290 con 0x7fb8441635e0 2024-04-25T08:35:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb8427fc640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8440cc2c0 0x7fb844161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb8437fe640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8441635e0 0x7fb8441622a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0090b0 tx=0x7fb83c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb83bfff640 1 -- 172.21.15.38:0/798816200 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c037040 con 0x7fb8441635e0 2024-04-25T08:35:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb83bfff640 1 -- 172.21.15.38:0/798816200 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c042070 con 0x7fb8441635e0 2024-04-25T08:35:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb849968640 1 -- 172.21.15.38:0/798816200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84415d9f0 con 0x7fb8441635e0 2024-04-25T08:35:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb83bfff640 1 -- 172.21.15.38:0/798816200 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c030920 con 0x7fb8441635e0 2024-04-25T08:35:05.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.787+0000 7fb849968640 1 -- 172.21.15.38:0/798816200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb844153590 con 0x7fb8441635e0 2024-04-25T08:35:05.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.788+0000 7fb83bfff640 1 -- 172.21.15.38:0/798816200 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb83c030ac0 con 0x7fb8441635e0 2024-04-25T08:35:05.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.789+0000 7fb83bfff640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb82406b4d0 0x7fb82406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.789+0000 7fb83bfff640 1 -- 172.21.15.38:0/798816200 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb83c0aacd0 con 0x7fb8441635e0 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.789+0000 7fb842ffd640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb82406b4d0 0x7fb82406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.790+0000 7fb842ffd640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb82406b4d0 0x7fb82406d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb834007fd0 tx=0x7fb834009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.790+0000 7fb84c662740 1 Processor -- start 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.790+0000 7fb84c662740 1 -- start start 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb84c662740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129b4fe0 0x5606129b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb84c662740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb84c662740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560612987c40 0x56061298b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb84c662740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606128f0a50 con 0x560612987c40 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb84c662740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x560612899d40 con 0x5606129b4fe0 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb84c662740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606128ecaf0 con 0x5606129b53c0 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb8437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560612987c40 0x56061298b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb8437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560612987c40 0x56061298b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42750/0 (socket says 172.21.15.38:42750) 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb8437fe640 1 -- 172.21.15.38:0/1491459908 learned_addr learned my addr 172.21.15.38:0/1491459908 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb842ffd640 1 --2- 172.21.15.38:0/1491459908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129b4fe0 0x5606129b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb8427fc640 1 --2- 172.21.15.38:0/1491459908 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb8437fe640 1 -- 172.21.15.38:0/1491459908 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 msgr2=0x5606129b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb8437fe640 1 --2- 172.21.15.38:0/1491459908 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb8437fe640 1 -- 172.21.15.38:0/1491459908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129b4fe0 msgr2=0x5606129b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb8437fe640 1 --2- 172.21.15.38:0/1491459908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129b4fe0 0x5606129b77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb8437fe640 1 -- 172.21.15.38:0/1491459908 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606126d1690 con 0x560612987c40 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.791+0000 7fb8437fe640 1 --2- 172.21.15.38:0/1491459908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560612987c40 0x56061298b450 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0814b0 tx=0x7fb83c078da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda0afecca7c9555 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.792+0000 7fb8397fa640 1 -- 172.21.15.38:0/1491459908 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c07d420 con 0x560612987c40 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.792+0000 7fb8397fa640 1 -- 172.21.15.38:0/1491459908 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c0819e0 con 0x560612987c40 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.792+0000 7fb8397fa640 1 -- 172.21.15.38:0/1491459908 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c07d120 con 0x560612987c40 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.792+0000 7fb84c662740 1 -- 172.21.15.38:0/1491459908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560612987c40 msgr2=0x56061298b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.792+0000 7fb84c662740 1 --2- 172.21.15.38:0/1491459908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560612987c40 0x56061298b450 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0814b0 tx=0x7fb83c078da0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.792+0000 7fb84c662740 1 -- 172.21.15.38:0/1491459908 shutdown_connections 2024-04-25T08:35:05.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.792+0000 7fb84c662740 1 --2- 172.21.15.38:0/1491459908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560612987c40 0x56061298b450 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.792+0000 7fb84c662740 1 --2- 172.21.15.38:0/1491459908 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.792+0000 7fb84c662740 1 --2- 172.21.15.38:0/1491459908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129b4fe0 0x5606129b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.792+0000 7fb84c662740 1 -- 172.21.15.38:0/1491459908 >> 172.21.15.38:0/1491459908 conn(0x560612989eb0 msgr2=0x5606129d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.792+0000 7fb84c662740 1 -- 172.21.15.38:0/1491459908 shutdown_connections 2024-04-25T08:35:05.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.792+0000 7fb84c662740 1 -- 172.21.15.38:0/1491459908 wait complete. 2024-04-25T08:35:05.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.793+0000 7fb84c662740 1 -- start start 2024-04-25T08:35:05.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.793+0000 7fb84c662740 1 Processor -- start 2024-04-25T08:35:05.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb84c662740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x5606129ac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb84c662740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129b53c0 0x5606129aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb84c662740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129d58d0 0x5606129b86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb84c662740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606128f0a50 con 0x5606129b4fe0 2024-04-25T08:35:05.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb84c662740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56061288f340 con 0x5606129b53c0 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb84c662740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56061298d060 con 0x5606129d58d0 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb84c662740 10 client.0 _ll_register_callbacks cb 0x5606128accb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb842ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x5606129ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb842ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x5606129ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42762/0 (socket says 172.21.15.38:42762) 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb842ffd640 1 -- 172.21.15.38:0/2176809850 learned_addr learned my addr 172.21.15.38:0/2176809850 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb8427fc640 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129b53c0 0x5606129aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb8437fe640 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129d58d0 0x5606129b86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb842ffd640 1 -- 172.21.15.38:0/2176809850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129d58d0 msgr2=0x5606129b86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb842ffd640 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129d58d0 0x5606129b86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb842ffd640 1 -- 172.21.15.38:0/2176809850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129b53c0 msgr2=0x5606129aefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb842ffd640 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129b53c0 0x5606129aefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb842ffd640 1 -- 172.21.15.38:0/2176809850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560612731e20 con 0x5606129b4fe0 2024-04-25T08:35:05.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb8437fe640 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129d58d0 0x5606129b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb842ffd640 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x5606129ac520 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb83400db10 tx=0x7fb834010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab33203aa685473 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.794+0000 7fb8227fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb8227fc640 1 -- 172.21.15.38:0/2176809850 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834024640 con 0x5606129b4fe0 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb8227fc640 1 -- 172.21.15.38:0/2176809850 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb834021b20 con 0x5606129b4fe0 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb84c662740 1 -- 172.21.15.38:0/2176809850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb83c0819e0 con 0x5606129b4fe0 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb8227fc640 1 -- 172.21.15.38:0/2176809850 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb834021d20 con 0x5606129b4fe0 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb84c662740 10 client.6252 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb84c662740 1 -- 172.21.15.38:0/2176809850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5606125c4f80 con 0x5606129b4fe0 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb8227fc640 1 -- 172.21.15.38:0/2176809850 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c0819e0 con 0x5606129b4fe0 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb8227fc640 1 -- 172.21.15.38:0/2176809850 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb834022020 con 0x5606129b4fe0 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb84c662740 10 client.6252 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb84c662740 10 client.6252 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb84c662740 20 client.6252 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb84c662740 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5606129b8e60 0x5606129b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb84c662740 4 client.6252 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb84c662740 1 -- 172.21.15.38:0/2176809850 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606128fdf90 con 0x5606129b8e60 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb8427fc640 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5606129b8e60 0x5606129b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb8427fc640 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5606129b8e60 0x5606129b3d70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54e5a9bb103f1ebc server_cookie=a46848a16352175e in_seq=0 out_seq=0 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.795+0000 7fb8227fc640 10 client.6252 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb8227fc640 1 -- 172.21.15.38:0/2176809850 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5606128fdf90 con 0x5606129b8e60 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb8227fc640 10 client.6252 handle_command_reply: tid=0 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb84c662740 1 client.6252 shutdown 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb84c662740 1 -- 172.21.15.38:0/2176809850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 msgr2=0x5606129ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb84c662740 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x5606129ac520 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb83400db10 tx=0x7fb834010000 comp rx=0 tx=0).stop 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb84c662740 1 -- 172.21.15.38:0/2176809850 shutdown_connections 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb84c662740 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5606129b8e60 0x5606129b3d70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb84c662740 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129d58d0 0x5606129b86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb84c662740 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129b53c0 0x5606129aefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb8227fc640 0 client.6252 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:05.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb84c662740 1 --2- 172.21.15.38:0/2176809850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x5606129ac520 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb84c662740 1 -- 172.21.15.38:0/2176809850 >> 172.21.15.38:0/2176809850 conn(0x560612987c40 msgr2=0x5606129b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb84c662740 1 -- 172.21.15.38:0/2176809850 shutdown_connections 2024-04-25T08:35:05.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb84c662740 1 -- 172.21.15.38:0/2176809850 wait complete. 2024-04-25T08:35:05.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.796+0000 7fb84c662740 20 client.6252 trim_cache size 0 max 16384 2024-04-25T08:35:05.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.812+0000 7fb84c662740 1 Processor -- start 2024-04-25T08:35:05.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.812+0000 7fb84c662740 1 -- start start 2024-04-25T08:35:05.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.812+0000 7fb84c662740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x5606129f5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.812+0000 7fb84c662740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.812+0000 7fb84c662740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129d58d0 0x5606129ad140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb84c662740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8441fe330 con 0x5606129b4fe0 2024-04-25T08:35:05.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb84c662740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8441fd560 con 0x5606129d58d0 2024-04-25T08:35:05.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb84c662740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8440c98c0 con 0x5606129b53c0 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb842ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x5606129f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb842ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x5606129f5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42776/0 (socket says 172.21.15.38:42776) 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb842ffd640 1 -- 172.21.15.38:0/3195664926 learned_addr learned my addr 172.21.15.38:0/3195664926 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb8437fe640 1 --2- 172.21.15.38:0/3195664926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129d58d0 0x5606129ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb8427fc640 1 --2- 172.21.15.38:0/3195664926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb842ffd640 1 -- 172.21.15.38:0/3195664926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 msgr2=0x5606129d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb842ffd640 1 --2- 172.21.15.38:0/3195664926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb842ffd640 1 -- 172.21.15.38:0/3195664926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129d58d0 msgr2=0x5606129ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb842ffd640 1 --2- 172.21.15.38:0/3195664926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129d58d0 0x5606129ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb842ffd640 1 -- 172.21.15.38:0/3195664926 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606129d4a70 con 0x5606129b4fe0 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb8427fc640 1 --2- 172.21.15.38:0/3195664926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb842ffd640 1 --2- 172.21.15.38:0/3195664926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x5606129f5690 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb834003cd0 tx=0x7fb834011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b8ce44d0f95330 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb838ff9640 1 -- 172.21.15.38:0/3195664926 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834021710 con 0x5606129b4fe0 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb838ff9640 1 -- 172.21.15.38:0/3195664926 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8340218b0 con 0x5606129b4fe0 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.813+0000 7fb838ff9640 1 -- 172.21.15.38:0/3195664926 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83402e480 con 0x5606129b4fe0 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.814+0000 7fb84c662740 1 -- 172.21.15.38:0/3195664926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 msgr2=0x5606129f5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.814+0000 7fb84c662740 1 --2- 172.21.15.38:0/3195664926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x5606129f5690 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb834003cd0 tx=0x7fb834011000 comp rx=0 tx=0).stop 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.814+0000 7fb84c662740 1 -- 172.21.15.38:0/3195664926 shutdown_connections 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.814+0000 7fb84c662740 1 --2- 172.21.15.38:0/3195664926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129d58d0 0x5606129ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.814+0000 7fb84c662740 1 --2- 172.21.15.38:0/3195664926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.814+0000 7fb84c662740 1 --2- 172.21.15.38:0/3195664926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x5606129f5690 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.814+0000 7fb84c662740 1 -- 172.21.15.38:0/3195664926 >> 172.21.15.38:0/3195664926 conn(0x560612987c40 msgr2=0x56061298d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.814+0000 7fb84c662740 1 -- 172.21.15.38:0/3195664926 shutdown_connections 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.814+0000 7fb84c662740 1 -- 172.21.15.38:0/3195664926 wait complete. 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.814+0000 7fb84c662740 1 -- start start 2024-04-25T08:35:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.815+0000 7fb84c662740 1 Processor -- start 2024-04-25T08:35:05.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.815+0000 7fb84c662740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x56061298b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.815+0000 7fb84c662740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129ac330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.815+0000 7fb84c662740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129d58d0 0x5606129ac870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.815+0000 7fb84c662740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8441fd090 con 0x5606129b4fe0 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.815+0000 7fb84c662740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606129daed0 con 0x5606129d58d0 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.815+0000 7fb84c662740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x560612a42820 con 0x5606129b53c0 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.815+0000 7fb84c662740 10 client.0 _ll_register_callbacks cb 0x7fb8340039b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.815+0000 7fb842ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x56061298b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.815+0000 7fb842ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x56061298b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42788/0 (socket says 172.21.15.38:42788) 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb842ffd640 1 -- 172.21.15.38:0/4278457047 learned_addr learned my addr 172.21.15.38:0/4278457047 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb8437fe640 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129d58d0 0x5606129ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb8427fc640 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb842ffd640 1 -- 172.21.15.38:0/4278457047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 msgr2=0x5606129ac330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb842ffd640 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129ac330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb842ffd640 1 -- 172.21.15.38:0/4278457047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129d58d0 msgr2=0x5606129ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb842ffd640 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129d58d0 0x5606129ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb842ffd640 1 -- 172.21.15.38:0/4278457047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606129c9200 con 0x5606129b4fe0 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb842ffd640 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x56061298b350 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb83400dae0 tx=0x7fb834000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ad1c49d78683f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb8227fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb8227fc640 1 -- 172.21.15.38:0/4278457047 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83403c030 con 0x5606129b4fe0 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb8227fc640 1 -- 172.21.15.38:0/4278457047 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb834021510 con 0x5606129b4fe0 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb84c662740 1 -- 172.21.15.38:0/4278457047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8340218b0 con 0x5606129b4fe0 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb8227fc640 1 -- 172.21.15.38:0/4278457047 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb834048050 con 0x5606129b4fe0 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb84c662740 10 client.6258 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb84c662740 1 -- 172.21.15.38:0/4278457047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x560612a41ee0 con 0x5606129b4fe0 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.816+0000 7fb8227fc640 1 -- 172.21.15.38:0/4278457047 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8340218b0 con 0x5606129b4fe0 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb8227fc640 1 -- 172.21.15.38:0/4278457047 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb83404b020 con 0x5606129b4fe0 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb84c662740 10 client.6258 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb84c662740 10 client.6258 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb84c662740 20 client.6258 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb84c662740 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x560612a27310 0x5606129c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb84c662740 4 client.6258 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb84c662740 1 -- 172.21.15.38:0/4278457047 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x5606128f82e0 con 0x560612a27310 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb8427fc640 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x560612a27310 0x5606129c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb8227fc640 10 client.6258 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb8427fc640 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x560612a27310 0x5606129c8180 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c894612fe8a596a server_cookie=37619172efa2a059 in_seq=0 out_seq=0 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb8227fc640 1 -- 172.21.15.38:0/4278457047 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5606128f82e0 con 0x560612a27310 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb8227fc640 10 client.6258 handle_command_reply: tid=0 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.817+0000 7fb84c662740 1 client.6258 shutdown 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb84c662740 1 -- 172.21.15.38:0/4278457047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 msgr2=0x56061298b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb84c662740 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x56061298b350 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb83400dae0 tx=0x7fb834000f40 comp rx=0 tx=0).stop 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb84c662740 1 -- 172.21.15.38:0/4278457047 shutdown_connections 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb84c662740 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x560612a27310 0x5606129c8180 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb84c662740 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5606129d58d0 0x5606129ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb84c662740 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5606129b53c0 0x5606129ac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb8227fc640 0 client.6258 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:05.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb84c662740 1 --2- 172.21.15.38:0/4278457047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5606129b4fe0 0x56061298b350 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb84c662740 1 -- 172.21.15.38:0/4278457047 >> 172.21.15.38:0/4278457047 conn(0x560612987c40 msgr2=0x5606129d27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb84c662740 1 -- 172.21.15.38:0/4278457047 shutdown_connections 2024-04-25T08:35:05.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb84c662740 1 -- 172.21.15.38:0/4278457047 wait complete. 2024-04-25T08:35:05.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb84c662740 20 client.6258 trim_cache size 0 max 16384 2024-04-25T08:35:05.819 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:05.819 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:35:05.819 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "5197916b-a7b0-4971-a5d9-35913e715c2a", 2024-04-25T08:35:05.819 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:35:05.819 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:05.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb849968640 1 -- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb82406b4d0 msgr2=0x7fb82406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb849968640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb82406b4d0 0x7fb82406d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb834007fd0 tx=0x7fb834009040 comp rx=0 tx=0).stop 2024-04-25T08:35:05.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb849968640 1 -- 172.21.15.38:0/798816200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8441635e0 msgr2=0x7fb8441622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb849968640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8441635e0 0x7fb8441622a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb83c0090b0 tx=0x7fb83c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb849968640 1 -- 172.21.15.38:0/798816200 shutdown_connections 2024-04-25T08:35:05.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb849968640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8441635e0 0x7fb8441622a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb849968640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8440cc2c0 0x7fb844161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb849968640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb82406b4d0 0x7fb82406d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb849968640 1 --2- 172.21.15.38:0/798816200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb84406abe0 0x7fb844152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.818+0000 7fb849968640 1 -- 172.21.15.38:0/798816200 >> 172.21.15.38:0/798816200 conn(0x7fb8440d06b0 msgr2=0x7fb84415c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.819+0000 7fb849968640 1 -- 172.21.15.38:0/798816200 shutdown_connections 2024-04-25T08:35:05.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.819+0000 7fb849968640 1 -- 172.21.15.38:0/798816200 wait complete. 2024-04-25T08:35:05.825 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '5197916b-a7b0-4971-a5d9-35913e715c2a', 'mode': 'asynchronous'}' 2024-04-25T08:35:05.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:05.825 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-25T08:35:05.845 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-25T08:35:05.845 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:05.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.991+0000 7fe204cfc640 1 Processor -- start 2024-04-25T08:35:05.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.991+0000 7fe204cfc640 1 -- start start 2024-04-25T08:35:05.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.991+0000 7fe204cfc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20015ddf0 0x7fe200161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.991+0000 7fe204cfc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe2000b93f0 0x7fe2000d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe204cfc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe200161e00 con 0x7fe2000b5710 2024-04-25T08:35:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe204cfc640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2001620e0 con 0x7fe2000b93f0 2024-04-25T08:35:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe204cfc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2001623c0 con 0x7fe20015ddf0 2024-04-25T08:35:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe1fed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe2000b93f0 0x7fe2000d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe1fed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe2000b93f0 0x7fe2000d2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42792/0 (socket says 172.21.15.38:42792) 2024-04-25T08:35:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe1fe59c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe2000b5710 0x7fe2000b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60640/0 (socket says 172.21.15.38:60640) 2024-04-25T08:35:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe1fed9d640 1 -- 172.21.15.38:0/254603551 learned_addr learned my addr 172.21.15.38:0/254603551 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe1f7fff640 1 --2- 172.21.15.38:0/254603551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20015ddf0 0x7fe200161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe1fed9d640 1 -- 172.21.15.38:0/254603551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20015ddf0 msgr2=0x7fe200161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe1fed9d640 1 --2- 172.21.15.38:0/254603551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20015ddf0 0x7fe200161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe1fed9d640 1 -- 172.21.15.38:0/254603551 >> v1:172.21.15.38:6789/0 conn(0x7fe2000b5710 legacy=0x7fe2000b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe1fed9d640 1 -- 172.21.15.38:0/254603551 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000b9870 con 0x7fe2000b93f0 2024-04-25T08:35:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.992+0000 7fe1fdd9b640 1 -- 172.21.15.38:0/254603551 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203729743 0 0) 0x7fe200161e00 con 0x7fe2000b5710 2024-04-25T08:35:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.993+0000 7fe1fed9d640 1 --2- 172.21.15.38:0/254603551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2000b93f0 0x7fe2000d2340 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009e50 tx=0x7fe1f802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c53e6d65f9e056 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.993+0000 7fe1fdd9b640 1 -- 172.21.15.38:0/254603551 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8036070 con 0x7fe2000b93f0 2024-04-25T08:35:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.993+0000 7fe1fdd9b640 1 -- 172.21.15.38:0/254603551 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f8031600 con 0x7fe2000b93f0 2024-04-25T08:35:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.993+0000 7fe1fdd9b640 1 -- 172.21.15.38:0/254603551 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f80319d0 con 0x7fe2000b93f0 2024-04-25T08:35:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.993+0000 7fe204cfc640 1 -- 172.21.15.38:0/254603551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2000b93f0 msgr2=0x7fe2000d2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.993+0000 7fe204cfc640 1 --2- 172.21.15.38:0/254603551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2000b93f0 0x7fe2000d2340 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009e50 tx=0x7fe1f802a710 comp rx=0 tx=0).stop 2024-04-25T08:35:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.993+0000 7fe204cfc640 1 -- 172.21.15.38:0/254603551 shutdown_connections 2024-04-25T08:35:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.993+0000 7fe204cfc640 1 --2- 172.21.15.38:0/254603551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2000b93f0 0x7fe2000d2340 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.993+0000 7fe204cfc640 1 --2- 172.21.15.38:0/254603551 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe20015ddf0 0x7fe200161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.993+0000 7fe204cfc640 1 -- 172.21.15.38:0/254603551 >> 172.21.15.38:0/254603551 conn(0x7fe2000d07c0 msgr2=0x7fe2000d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.993+0000 7fe204cfc640 1 -- 172.21.15.38:0/254603551 shutdown_connections 2024-04-25T08:35:05.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.993+0000 7fe204cfc640 1 -- 172.21.15.38:0/254603551 wait complete. 2024-04-25T08:35:05.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.995+0000 7fe204cfc640 1 Processor -- start 2024-04-25T08:35:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.995+0000 7fe204cfc640 1 -- start start 2024-04-25T08:35:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe204cfc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2000b5710 0x7fe2000cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe204cfc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2000b93f0 0x7fe20015f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe204cfc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe20015ddf0 0x7fe2000ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe204cfc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000d2c50 con 0x7fe20015ddf0 2024-04-25T08:35:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe204cfc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000d2ad0 con 0x7fe2000b93f0 2024-04-25T08:35:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe204cfc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000d2dd0 con 0x7fe2000b5710 2024-04-25T08:35:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1fed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe20015ddf0 0x7fe2000ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1fed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe20015ddf0 0x7fe2000ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42804/0 (socket says 172.21.15.38:42804) 2024-04-25T08:35:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1fed9d640 1 -- 172.21.15.38:0/2380307952 learned_addr learned my addr 172.21.15.38:0/2380307952 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:05.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1f7fff640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2000b93f0 0x7fe20015f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1fe59c640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2000b5710 0x7fe2000cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1fed9d640 1 -- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2000b5710 msgr2=0x7fe2000cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1fed9d640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2000b5710 0x7fe2000cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1fed9d640 1 -- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2000b93f0 msgr2=0x7fe20015f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:05.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1fed9d640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2000b93f0 0x7fe20015f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:05.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1fed9d640 1 -- 172.21.15.38:0/2380307952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20015fcf0 con 0x7fe20015ddf0 2024-04-25T08:35:05.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1fed9d640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe20015ddf0 0x7fe2000ccb80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8034040 tx=0x7fe1f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1f6ffd640 1 -- 172.21.15.38:0/2380307952 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8036050 con 0x7fe20015ddf0 2024-04-25T08:35:05.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.996+0000 7fe1f6ffd640 1 -- 172.21.15.38:0/2380307952 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f8042070 con 0x7fe20015ddf0 2024-04-25T08:35:05.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.997+0000 7fe204cfc640 1 -- 172.21.15.38:0/2380307952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20015ffd0 con 0x7fe20015ddf0 2024-04-25T08:35:05.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.997+0000 7fe1f6ffd640 1 -- 172.21.15.38:0/2380307952 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8009510 con 0x7fe20015ddf0 2024-04-25T08:35:05.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.997+0000 7fe204cfc640 1 -- 172.21.15.38:0/2380307952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe200160590 con 0x7fe20015ddf0 2024-04-25T08:35:05.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.998+0000 7fe204cfc640 1 -- 172.21.15.38:0/2380307952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe200068e80 con 0x7fe20015ddf0 2024-04-25T08:35:05.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.998+0000 7fe1f6ffd640 1 -- 172.21.15.38:0/2380307952 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1f8007bd0 con 0x7fe20015ddf0 2024-04-25T08:35:05.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.998+0000 7fe1f6ffd640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe1e006b4d0 0x7fe1e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:05.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.999+0000 7fe1f6ffd640 1 -- 172.21.15.38:0/2380307952 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1f80ab050 con 0x7fe20015ddf0 2024-04-25T08:35:05.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.999+0000 7fe1fe59c640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe1e006b4d0 0x7fe1e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:05.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:05.999+0000 7fe1fe59c640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe1e006b4d0 0x7fe1e006d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0000f60 tx=0x7fe1f0007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:05.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.003+0000 7fe1f6ffd640 1 -- 172.21.15.38:0/2380307952 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1f80798a0 con 0x7fe20015ddf0 2024-04-25T08:35:06.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.233+0000 7fe204cfc640 1 -- 172.21.15.38:0/2380307952 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2000c6b70 con 0x7fe20015ddf0 2024-04-25T08:35:06.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.233+0000 7fe1f6ffd640 1 -- 172.21.15.38:0/2380307952 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fe1f807d940 con 0x7fe20015ddf0 2024-04-25T08:35:06.228 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:06.229 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:34:46.222808+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:06.229 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 24 2024-04-25T08:35:06.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.236+0000 7fe204cfc640 1 -- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe1e006b4d0 msgr2=0x7fe1e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:06.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.236+0000 7fe204cfc640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe1e006b4d0 0x7fe1e006d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0000f60 tx=0x7fe1f0007d50 comp rx=0 tx=0).stop 2024-04-25T08:35:06.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.236+0000 7fe204cfc640 1 -- 172.21.15.38:0/2380307952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe20015ddf0 msgr2=0x7fe2000ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:06.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.236+0000 7fe204cfc640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe20015ddf0 0x7fe2000ccb80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8034040 tx=0x7fe1f8004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:06.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.236+0000 7fe204cfc640 1 -- 172.21.15.38:0/2380307952 shutdown_connections 2024-04-25T08:35:06.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.236+0000 7fe204cfc640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe20015ddf0 0x7fe2000ccb80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.236+0000 7fe204cfc640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2000b93f0 0x7fe20015f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.236+0000 7fe204cfc640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe1e006b4d0 0x7fe1e006d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.236+0000 7fe204cfc640 1 --2- 172.21.15.38:0/2380307952 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2000b5710 0x7fe2000cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.236+0000 7fe204cfc640 1 -- 172.21.15.38:0/2380307952 >> 172.21.15.38:0/2380307952 conn(0x7fe2000d07c0 msgr2=0x7fe2000b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:06.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.236+0000 7fe204cfc640 1 -- 172.21.15.38:0/2380307952 shutdown_connections 2024-04-25T08:35:06.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.236+0000 7fe204cfc640 1 -- 172.21.15.38:0/2380307952 wait complete. 2024-04-25T08:35:06.241 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-25T08:35:06.350 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:06.350 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:35:06.350 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:06.358 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:35:06.359 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:35:06.359 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:06.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:06.375 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T08:35:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:35:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:35:06.441 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00923689 s, 114 MB/s 2024-04-25T08:35:06.442 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T08:35:06.442 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:06.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.589+0000 7fda62db7640 1 Processor -- start 2024-04-25T08:35:06.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.589+0000 7fda62db7640 1 -- start start 2024-04-25T08:35:06.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.589+0000 7fda62db7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fda5c06abe0 0x7fda5c06b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:06.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.589+0000 7fda62db7640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fda5c0d01d0 0x7fda5c0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:06.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.589+0000 7fda62db7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5c0cfa40 con 0x7fda5c0aeeb0 2024-04-25T08:35:06.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.589+0000 7fda62db7640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0cd9b0 con 0x7fda5c06abe0 2024-04-25T08:35:06.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.589+0000 7fda62db7640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0cdc90 con 0x7fda5c0d01d0 2024-04-25T08:35:06.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.590+0000 7fda5bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fda5c06abe0 0x7fda5c06b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:06.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.590+0000 7fda5bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fda5c06abe0 0x7fda5c06b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42818/0 (socket says 172.21.15.38:42818) 2024-04-25T08:35:06.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.590+0000 7fda5bfff640 1 -- 172.21.15.38:0/1479278984 learned_addr learned my addr 172.21.15.38:0/1479278984 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:06.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.590+0000 7fda5bfff640 1 -- 172.21.15.38:0/1479278984 >> v2:172.21.15.107:3300/0 conn(0x7fda5c0d01d0 msgr2=0x7fda5c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:06.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.590+0000 7fda5bfff640 1 --2- 172.21.15.38:0/1479278984 >> v2:172.21.15.107:3300/0 conn(0x7fda5c0d01d0 0x7fda5c0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.590+0000 7fda5bfff640 1 -- 172.21.15.38:0/1479278984 >> v1:172.21.15.107:6789/0 conn(0x7fda5c0aeeb0 legacy=0x7fda5c0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:06.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.590+0000 7fda5bfff640 1 -- 172.21.15.38:0/1479278984 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c15e390 con 0x7fda5c06abe0 2024-04-25T08:35:06.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.590+0000 7fda5bfff640 1 --2- 172.21.15.38:0/1479278984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c06abe0 0x7fda5c06b090 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0b66f0 tx=0x7fda5002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeef6fbd976705e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:06.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.591+0000 7fda5b7fe640 1 -- 172.21.15.38:0/1479278984 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50037070 con 0x7fda5c06abe0 2024-04-25T08:35:06.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.591+0000 7fda5b7fe640 1 -- 172.21.15.38:0/1479278984 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda50032440 con 0x7fda5c06abe0 2024-04-25T08:35:06.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.591+0000 7fda5b7fe640 1 -- 172.21.15.38:0/1479278984 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50032810 con 0x7fda5c06abe0 2024-04-25T08:35:06.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.591+0000 7fda62db7640 1 -- 172.21.15.38:0/1479278984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c06abe0 msgr2=0x7fda5c06b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:06.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.591+0000 7fda62db7640 1 --2- 172.21.15.38:0/1479278984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c06abe0 0x7fda5c06b090 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0b66f0 tx=0x7fda5002a230 comp rx=0 tx=0).stop 2024-04-25T08:35:06.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.591+0000 7fda62db7640 1 -- 172.21.15.38:0/1479278984 shutdown_connections 2024-04-25T08:35:06.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.591+0000 7fda62db7640 1 --2- 172.21.15.38:0/1479278984 >> v2:172.21.15.107:3300/0 conn(0x7fda5c0d01d0 0x7fda5c0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.591+0000 7fda62db7640 1 --2- 172.21.15.38:0/1479278984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c06abe0 0x7fda5c06b090 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.591+0000 7fda62db7640 1 -- 172.21.15.38:0/1479278984 >> 172.21.15.38:0/1479278984 conn(0x7fda5c072610 msgr2=0x7fda5c099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:06.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.591+0000 7fda62db7640 1 -- 172.21.15.38:0/1479278984 shutdown_connections 2024-04-25T08:35:06.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.591+0000 7fda62db7640 1 -- 172.21.15.38:0/1479278984 wait complete. 2024-04-25T08:35:06.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.592+0000 7fda62db7640 1 Processor -- start 2024-04-25T08:35:06.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.593+0000 7fda62db7640 1 -- start start 2024-04-25T08:35:06.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.593+0000 7fda62db7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c06abe0 0x7fda5c0c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:06.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.593+0000 7fda62db7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c0aeeb0 0x7fda5c0c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:06.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.593+0000 7fda62db7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0d01d0 0x7fda5c0b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:06.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.593+0000 7fda62db7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0d0af0 con 0x7fda5c06abe0 2024-04-25T08:35:06.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.593+0000 7fda62db7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0cd620 con 0x7fda5c0aeeb0 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.593+0000 7fda62db7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0d0c70 con 0x7fda5c0d01d0 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.593+0000 7fda60b53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c06abe0 0x7fda5c0c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.593+0000 7fda60b53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c06abe0 0x7fda5c0c6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42826/0 (socket says 172.21.15.38:42826) 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.593+0000 7fda60b53640 1 -- 172.21.15.38:0/1451810429 learned_addr learned my addr 172.21.15.38:0/1451810429 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda5bfff640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c0aeeb0 0x7fda5c0c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda61354640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0d01d0 0x7fda5c0b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda60b53640 1 -- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0d01d0 msgr2=0x7fda5c0b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda60b53640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0d01d0 0x7fda5c0b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda60b53640 1 -- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c0aeeb0 msgr2=0x7fda5c0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda60b53640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c0aeeb0 0x7fda5c0c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda60b53640 1 -- 172.21.15.38:0/1451810429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c0c9a80 con 0x7fda5c06abe0 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda61354640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0d01d0 0x7fda5c0b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:06.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda60b53640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c06abe0 0x7fda5c0c6a10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fda5400c7a0 tx=0x7fda5400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:06.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda597fa640 1 -- 172.21.15.38:0/1451810429 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54017070 con 0x7fda5c06abe0 2024-04-25T08:35:06.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda597fa640 1 -- 172.21.15.38:0/1451810429 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda54003da0 con 0x7fda5c06abe0 2024-04-25T08:35:06.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda62db7640 1 -- 172.21.15.38:0/1451810429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c0c9d60 con 0x7fda5c06abe0 2024-04-25T08:35:06.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda597fa640 1 -- 172.21.15.38:0/1451810429 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda540052b0 con 0x7fda5c06abe0 2024-04-25T08:35:06.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.594+0000 7fda62db7640 1 -- 172.21.15.38:0/1451810429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c0c48f0 con 0x7fda5c06abe0 2024-04-25T08:35:06.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.595+0000 7fda62db7640 1 -- 172.21.15.38:0/1451810429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c0ca8e0 con 0x7fda5c06abe0 2024-04-25T08:35:06.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.596+0000 7fda597fa640 1 -- 172.21.15.38:0/1451810429 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda5401a070 con 0x7fda5c06abe0 2024-04-25T08:35:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.596+0000 7fda597fa640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda4806b4d0 0x7fda4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.596+0000 7fda597fa640 1 -- 172.21.15.38:0/1451810429 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda54089aa0 con 0x7fda5c06abe0 2024-04-25T08:35:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.596+0000 7fda5bfff640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda4806b4d0 0x7fda4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:06.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.597+0000 7fda5bfff640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda4806b4d0 0x7fda4806d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fda50034000 tx=0x7fda5002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:06.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.600+0000 7fda597fa640 1 -- 172.21.15.38:0/1451810429 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda54058790 con 0x7fda5c06abe0 2024-04-25T08:35:06.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.816+0000 7fda62db7640 1 -- 172.21.15.38:0/1451810429 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda5c06fad0 con 0x7fda5c06abe0 2024-04-25T08:35:06.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.817+0000 7fda597fa640 1 -- 172.21.15.38:0/1451810429 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fda54058090 con 0x7fda5c06abe0 2024-04-25T08:35:06.811 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:06.812 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:34:46.222808+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:06.812 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 24 2024-04-25T08:35:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.818+0000 7fda62db7640 1 -- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda4806b4d0 msgr2=0x7fda4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.818+0000 7fda62db7640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda4806b4d0 0x7fda4806d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fda50034000 tx=0x7fda5002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.818+0000 7fda62db7640 1 -- 172.21.15.38:0/1451810429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c06abe0 msgr2=0x7fda5c0c6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.818+0000 7fda62db7640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c06abe0 0x7fda5c0c6a10 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fda5400c7a0 tx=0x7fda5400cc60 comp rx=0 tx=0).stop 2024-04-25T08:35:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.819+0000 7fda62db7640 1 -- 172.21.15.38:0/1451810429 shutdown_connections 2024-04-25T08:35:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.819+0000 7fda62db7640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda4806b4d0 0x7fda4806d990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.819+0000 7fda62db7640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda5c0d01d0 0x7fda5c0b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.819+0000 7fda62db7640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda5c0aeeb0 0x7fda5c0c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.819+0000 7fda62db7640 1 --2- 172.21.15.38:0/1451810429 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda5c06abe0 0x7fda5c0c6a10 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.819+0000 7fda62db7640 1 -- 172.21.15.38:0/1451810429 >> 172.21.15.38:0/1451810429 conn(0x7fda5c072610 msgr2=0x7fda5c1e4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.819+0000 7fda62db7640 1 -- 172.21.15.38:0/1451810429 shutdown_connections 2024-04-25T08:35:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:06.819+0000 7fda62db7640 1 -- 172.21.15.38:0/1451810429 wait complete. 2024-04-25T08:35:06.824 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T08:35:06.989 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:06.989 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:35:06.989 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:06.998 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:35:06.999 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-04-25T08:35:06.999 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:35:06.999 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-04-25T08:35:06.999 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:06.999 DEBUG:teuthology.orchestra.run.smithi038:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-25T08:35:06.999 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:35:07.067 INFO:teuthology.orchestra.run.smithi038.stdout:1099511630412 2024-04-25T08:35:07.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:35:07.076 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:07.076 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T08:35:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.225+0000 7f5904aee640 1 Processor -- start 2024-04-25T08:35:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.225+0000 7f5904aee640 1 -- start start 2024-04-25T08:35:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.226+0000 7f5904aee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590015df30 0x7f590009a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.226+0000 7f5904aee640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5900162390 0x7f5900099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.226+0000 7f5904aee640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590015f300 con 0x7f5900099fb0 2024-04-25T08:35:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.226+0000 7f5904aee640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590015e310 con 0x7f5900162390 2024-04-25T08:35:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.226+0000 7f5904aee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590009aeb0 con 0x7f590015df30 2024-04-25T08:35:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.226+0000 7f58fed9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5900099fb0 0x7f590006bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60652/0 (socket says 172.21.15.38:60652) 2024-04-25T08:35:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.226+0000 7f58fed9d640 1 -- 172.21.15.38:0/3522134230 learned_addr learned my addr 172.21.15.38:0/3522134230 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721505111 0 0) 0x7f590015f300 con 0x7f5900099fb0 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58ec003520 con 0x7f5900099fb0 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/3522134230 >> v2:172.21.15.107:3300/0 conn(0x7f5900162390 0x7f5900099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fe59c640 1 --2- 172.21.15.38:0/3522134230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590015df30 0x7f590009a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2007241491 0 0) 0x7f58ec003520 con 0x7f5900099fb0 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590015df30 msgr2=0x7f590009a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 --2- 172.21.15.38:0/3522134230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590015df30 0x7f590009a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 >> v2:172.21.15.107:3300/0 conn(0x7f5900162390 msgr2=0x7f5900099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 --2- 172.21.15.38:0/3522134230 >> v2:172.21.15.107:3300/0 conn(0x7f5900162390 0x7f5900099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59001627f0 con 0x7f5900099fb0 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f58f80019f0 con 0x7f5900099fb0 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 >> v1:172.21.15.38:6789/0 conn(0x7f5900099fb0 legacy=0x7f590006bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 --2- 172.21.15.38:0/3522134230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec0039b0 0x7f58ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 --2- 172.21.15.38:0/3522134230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58ec004800 0x7f58ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 --2- 172.21.15.38:0/3522134230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590015df30 0x7f58ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590009aeb0 con 0x7f58ec004800 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec00b9c0 con 0x7f58ec0039b0 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec00bb40 con 0x7f590015df30 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/3522134230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58ec004800 0x7f58ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fe59c640 1 --2- 172.21.15.38:0/3522134230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec0039b0 0x7f58ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.227+0000 7f58fed9d640 1 --2- 172.21.15.38:0/3522134230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590015df30 0x7f58ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.228+0000 7f58fdd9b640 1 -- 172.21.15.38:0/3522134230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590015df30 msgr2=0x7f58ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.228+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/3522134230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f590015df30 0x7f58ec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.228+0000 7f58fdd9b640 1 -- 172.21.15.38:0/3522134230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec0039b0 msgr2=0x7f58ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.228+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/3522134230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58ec0039b0 0x7f58ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.228+0000 7f58fdd9b640 1 -- 172.21.15.38:0/3522134230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec003520 con 0x7f58ec004800 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.228+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/3522134230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58ec004800 0x7f58ec004c80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009ac0 tx=0x7f58ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1276f1ce379cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.228+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0036070 con 0x7f58ec004800 2024-04-25T08:35:07.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.228+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0031400 con 0x7f58ec004800 2024-04-25T08:35:07.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.228+0000 7f58fd59a640 1 -- 172.21.15.38:0/3522134230 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0009690 con 0x7f58ec004800 2024-04-25T08:35:07.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.229+0000 7f5904aee640 1 -- 172.21.15.38:0/3522134230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58ec004800 msgr2=0x7f58ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.229+0000 7f5904aee640 1 --2- 172.21.15.38:0/3522134230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58ec004800 0x7f58ec004c80 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009ac0 tx=0x7f58ec005830 comp rx=0 tx=0).stop 2024-04-25T08:35:07.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.229+0000 7f58fed9d640 1 -- 172.21.15.38:0/3522134230 reap_dead start 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.229+0000 7f5904aee640 1 -- 172.21.15.38:0/3522134230 shutdown_connections 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.229+0000 7f5904aee640 1 -- 172.21.15.38:0/3522134230 >> 172.21.15.38:0/3522134230 conn(0x7f59000b3110 msgr2=0x7f59000b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.229+0000 7f5904aee640 1 -- 172.21.15.38:0/3522134230 shutdown_connections 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.229+0000 7f5904aee640 1 -- 172.21.15.38:0/3522134230 wait complete. 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.230+0000 7f5904aee640 1 Processor -- start 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.230+0000 7f5904aee640 1 -- start start 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.231+0000 7f5904aee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec004800 0x7f5900156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.231+0000 7f5904aee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5900160380 0x7f5900160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.231+0000 7f5904aee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59001f3720 0x7f59001f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.231+0000 7f5904aee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590015e820 con 0x7f59001f3720 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.231+0000 7f5904aee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590015ee70 con 0x7f5900160380 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.231+0000 7f5904aee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590015eb00 con 0x7f58ec004800 2024-04-25T08:35:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.231+0000 7f58fed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59001f3720 0x7f59001f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.231+0000 7f58fed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59001f3720 0x7f59001f3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42838/0 (socket says 172.21.15.38:42838) 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.231+0000 7f58fed9d640 1 -- 172.21.15.38:0/203647504 learned_addr learned my addr 172.21.15.38:0/203647504 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.231+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5900160380 0x7f5900160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.231+0000 7f58fe59c640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec004800 0x7f5900156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f58fed9d640 1 -- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec004800 msgr2=0x7f5900156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f58fed9d640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec004800 0x7f5900156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f58fed9d640 1 -- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5900160380 msgr2=0x7f5900160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f58fed9d640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5900160380 0x7f5900160800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f58fed9d640 1 -- 172.21.15.38:0/203647504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59001f4450 con 0x7f59001f3720 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5900160380 0x7f5900160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f58fe59c640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec004800 0x7f5900156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f58fed9d640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59001f3720 0x7f59001f3b60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5900072150 tx=0x7f58f800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f58f6ffd640 1 -- 172.21.15.38:0/203647504 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f800ebb0 con 0x7f59001f3720 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f58f6ffd640 1 -- 172.21.15.38:0/203647504 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f800ed50 con 0x7f59001f3720 2024-04-25T08:35:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f5904aee640 1 -- 172.21.15.38:0/203647504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59000cbf50 con 0x7f59001f3720 2024-04-25T08:35:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f58f6ffd640 1 -- 172.21.15.38:0/203647504 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8014670 con 0x7f59001f3720 2024-04-25T08:35:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.232+0000 7f5904aee640 1 -- 172.21.15.38:0/203647504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59001f40a0 con 0x7f59001f3720 2024-04-25T08:35:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.233+0000 7f58f6ffd640 1 -- 172.21.15.38:0/203647504 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58f8014810 con 0x7f59001f3720 2024-04-25T08:35:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.234+0000 7f58f6ffd640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58e006b4d0 0x7f58e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.234+0000 7f58f6ffd640 1 -- 172.21.15.38:0/203647504 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58f808a0c0 con 0x7f59001f3720 2024-04-25T08:35:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.234+0000 7f58fe59c640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58e006b4d0 0x7f58e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.234+0000 7f58fe59c640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58e006b4d0 0x7f58e006d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f58e80099e0 tx=0x7f58e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.235+0000 7f59077ed740 1 Processor -- start 2024-04-25T08:35:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.235+0000 7f59077ed740 1 -- start start 2024-04-25T08:35:07.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.235+0000 7f59077ed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc7aa0 0x555fa3dca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.235+0000 7f59077ed740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 0x555fa3dca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.235+0000 7f59077ed740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3d9a700 0x555fa3d9df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.235+0000 7f59077ed740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa3d03e00 con 0x555fa3dc7e80 2024-04-25T08:35:07.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.235+0000 7f59077ed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa3cacd20 con 0x555fa3dc7aa0 2024-04-25T08:35:07.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.235+0000 7f59077ed740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x555fa3cffea0 con 0x555fa3d9a700 2024-04-25T08:35:07.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58fdd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 0x555fa3dca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58fdd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 0x555fa3dca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42848/0 (socket says 172.21.15.38:42848) 2024-04-25T08:35:07.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58fdd9b640 1 -- 172.21.15.38:0/2749612840 learned_addr learned my addr 172.21.15.38:0/2749612840 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58fe59c640 1 --2- 172.21.15.38:0/2749612840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc7aa0 0x555fa3dca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58fed9d640 1 --2- 172.21.15.38:0/2749612840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3d9a700 0x555fa3d9df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58fdd9b640 1 -- 172.21.15.38:0/2749612840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3d9a700 msgr2=0x555fa3d9df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/2749612840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3d9a700 0x555fa3d9df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58fdd9b640 1 -- 172.21.15.38:0/2749612840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc7aa0 msgr2=0x555fa3dca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/2749612840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc7aa0 0x555fa3dca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58fdd9b640 1 -- 172.21.15.38:0/2749612840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fa3ccedf0 con 0x555fa3dc7e80 2024-04-25T08:35:07.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/2749612840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 0x555fa3dca7f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009ac0 tx=0x7f58f002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadc67c34cd2dc78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58dffff640 1 -- 172.21.15.38:0/2749612840 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0036070 con 0x555fa3dc7e80 2024-04-25T08:35:07.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58dffff640 1 -- 172.21.15.38:0/2749612840 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0042070 con 0x555fa3dc7e80 2024-04-25T08:35:07.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.236+0000 7f58dffff640 1 -- 172.21.15.38:0/2749612840 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0004440 con 0x555fa3dc7e80 2024-04-25T08:35:07.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.237+0000 7f59077ed740 1 -- 172.21.15.38:0/2749612840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 msgr2=0x555fa3dca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.237+0000 7f59077ed740 1 --2- 172.21.15.38:0/2749612840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 0x555fa3dca7f0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f58f0009ac0 tx=0x7f58f002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:07.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.237+0000 7f59077ed740 1 -- 172.21.15.38:0/2749612840 shutdown_connections 2024-04-25T08:35:07.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.237+0000 7f59077ed740 1 --2- 172.21.15.38:0/2749612840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3d9a700 0x555fa3d9df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.237+0000 7f59077ed740 1 --2- 172.21.15.38:0/2749612840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 0x555fa3dca7f0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.237+0000 7f59077ed740 1 --2- 172.21.15.38:0/2749612840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc7aa0 0x555fa3dca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.237+0000 7f59077ed740 1 -- 172.21.15.38:0/2749612840 >> 172.21.15.38:0/2749612840 conn(0x555fa3d9c970 msgr2=0x555fa3de52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.237+0000 7f59077ed740 1 -- 172.21.15.38:0/2749612840 shutdown_connections 2024-04-25T08:35:07.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.237+0000 7f59077ed740 1 -- 172.21.15.38:0/2749612840 wait complete. 2024-04-25T08:35:07.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.238+0000 7f59077ed740 1 -- start start 2024-04-25T08:35:07.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.238+0000 7f59077ed740 1 Processor -- start 2024-04-25T08:35:07.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f59077ed740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7aa0 0x555fa3dce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f59077ed740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7e80 0x555fa3dbf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f59077ed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc1a40 0x555fa3d99710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f59077ed740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa3d9fb20 con 0x555fa3dc7aa0 2024-04-25T08:35:07.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f59077ed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa3ca2590 con 0x555fa3dc1a40 2024-04-25T08:35:07.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f59077ed740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x555fa3d9fe00 con 0x555fa3dc7e80 2024-04-25T08:35:07.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f59077ed740 10 client.0 _ll_register_callbacks cb 0x555fa3caee30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:07.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58fe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7aa0 0x555fa3dce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58fe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7aa0 0x555fa3dce450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42858/0 (socket says 172.21.15.38:42858) 2024-04-25T08:35:07.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58fe59c640 1 -- 172.21.15.38:0/3993008011 learned_addr learned my addr 172.21.15.38:0/3993008011 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7e80 0x555fa3dbf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58fed9d640 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc1a40 0x555fa3d99710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58fe59c640 1 -- 172.21.15.38:0/3993008011 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7e80 msgr2=0x555fa3dbf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58fe59c640 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7e80 0x555fa3dbf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58fe59c640 1 -- 172.21.15.38:0/3993008011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc1a40 msgr2=0x555fa3d99710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58fe59c640 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc1a40 0x555fa3d99710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58fe59c640 1 -- 172.21.15.38:0/3993008011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fa3a183f0 con 0x555fa3dc7aa0 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58fe59c640 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7aa0 0x555fa3dce450 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f58e8016770 tx=0x7f58e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98da3da8dca5b62f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58dd7fa640 1 -- 172.21.15.38:0/3993008011 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8015500 con 0x555fa3dc7aa0 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f58dd7fa640 1 -- 172.21.15.38:0/3993008011 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58e80156a0 con 0x555fa3dc7aa0 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.239+0000 7f59077ed740 1 -- 172.21.15.38:0/3993008011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58f0042070 con 0x555fa3dc7aa0 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f58dd7fa640 1 -- 172.21.15.38:0/3993008011 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58e8034050 con 0x555fa3dc7aa0 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f59077ed740 10 client.6285 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f59077ed740 1 -- 172.21.15.38:0/3993008011 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x555fa3ae27d0 con 0x555fa3dc7aa0 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f58dd7fa640 1 -- 172.21.15.38:0/3993008011 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0042070 con 0x555fa3dc7aa0 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f58dd7fa640 1 -- 172.21.15.38:0/3993008011 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f58e80158a0 con 0x555fa3dc7aa0 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f59077ed740 10 client.6285 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f59077ed740 10 client.6285 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:07.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f59077ed740 20 client.6285 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f59077ed740 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x555fa3de7900 0x555fa3de7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f59077ed740 4 client.6285 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f59077ed740 1 -- 172.21.15.38:0/3993008011 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fa3d11340 con 0x555fa3de7900 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x555fa3de7900 0x555fa3de7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f58dd7fa640 10 client.6285 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.240+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x555fa3de7900 0x555fa3de7d10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c28b6c2b94c64f8 server_cookie=a332e0ad35b02d99 in_seq=0 out_seq=0 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f58dd7fa640 1 -- 172.21.15.38:0/3993008011 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555fa3d11340 con 0x555fa3de7900 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f58dd7fa640 10 client.6285 handle_command_reply: tid=0 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f59077ed740 1 client.6285 shutdown 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f59077ed740 1 -- 172.21.15.38:0/3993008011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7aa0 msgr2=0x555fa3dce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f59077ed740 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7aa0 0x555fa3dce450 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f58e8016770 tx=0x7f58e8004730 comp rx=0 tx=0).stop 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f59077ed740 1 -- 172.21.15.38:0/3993008011 shutdown_connections 2024-04-25T08:35:07.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f59077ed740 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x555fa3de7900 0x555fa3de7d10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f59077ed740 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc1a40 0x555fa3d99710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f59077ed740 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7e80 0x555fa3dbf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f59077ed740 1 --2- 172.21.15.38:0/3993008011 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7aa0 0x555fa3dce450 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f58dd7fa640 0 client.6285 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:07.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f59077ed740 1 -- 172.21.15.38:0/3993008011 >> 172.21.15.38:0/3993008011 conn(0x555fa3d9a700 msgr2=0x555fa3d9b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f59077ed740 1 -- 172.21.15.38:0/3993008011 shutdown_connections 2024-04-25T08:35:07.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f59077ed740 1 -- 172.21.15.38:0/3993008011 wait complete. 2024-04-25T08:35:07.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.241+0000 7f59077ed740 20 client.6285 trim_cache size 0 max 16384 2024-04-25T08:35:07.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.253+0000 7f59077ed740 1 Processor -- start 2024-04-25T08:35:07.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.253+0000 7f59077ed740 1 -- start start 2024-04-25T08:35:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.254+0000 7f59077ed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc1a40 0x555fa3de8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.254+0000 7f59077ed740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7aa0 0x555fa3d95df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.254+0000 7f59077ed740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 0x555fa3e4b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.254+0000 7f59077ed740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59002007e0 con 0x555fa3dc7e80 2024-04-25T08:35:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.254+0000 7f59077ed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59001ff9e0 con 0x555fa3dc1a40 2024-04-25T08:35:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.254+0000 7f59077ed740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59001fef60 con 0x555fa3dc7aa0 2024-04-25T08:35:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.254+0000 7f58fed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 0x555fa3e4b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.254+0000 7f58fed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 0x555fa3e4b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42870/0 (socket says 172.21.15.38:42870) 2024-04-25T08:35:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.254+0000 7f58fed9d640 1 -- 172.21.15.38:0/1410000166 learned_addr learned my addr 172.21.15.38:0/1410000166 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.254+0000 7f58fe59c640 1 --2- 172.21.15.38:0/1410000166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc1a40 0x555fa3de8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.254+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/1410000166 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7aa0 0x555fa3d95df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f58fed9d640 1 -- 172.21.15.38:0/1410000166 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7aa0 msgr2=0x555fa3d95df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f58fed9d640 1 --2- 172.21.15.38:0/1410000166 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7aa0 0x555fa3d95df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f58fed9d640 1 -- 172.21.15.38:0/1410000166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc1a40 msgr2=0x555fa3de8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f58fed9d640 1 --2- 172.21.15.38:0/1410000166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc1a40 0x555fa3de8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f58fed9d640 1 -- 172.21.15.38:0/1410000166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fa3df5150 con 0x555fa3dc7e80 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/1410000166 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7aa0 0x555fa3d95df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f58fed9d640 1 --2- 172.21.15.38:0/1410000166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 0x555fa3e4b060 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f58f8058440 tx=0x7f58f801f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69d15738b6505fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f58df7fe640 1 -- 172.21.15.38:0/1410000166 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8056a40 con 0x555fa3dc7e80 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f58df7fe640 1 -- 172.21.15.38:0/1410000166 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f8081e50 con 0x555fa3dc7e80 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f58df7fe640 1 -- 172.21.15.38:0/1410000166 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f8054db0 con 0x555fa3dc7e80 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f59077ed740 1 -- 172.21.15.38:0/1410000166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 msgr2=0x555fa3e4b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f59077ed740 1 --2- 172.21.15.38:0/1410000166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 0x555fa3e4b060 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f58f8058440 tx=0x7f58f801f040 comp rx=0 tx=0).stop 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f59077ed740 1 -- 172.21.15.38:0/1410000166 shutdown_connections 2024-04-25T08:35:07.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f59077ed740 1 --2- 172.21.15.38:0/1410000166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc7e80 0x555fa3e4b060 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f59077ed740 1 --2- 172.21.15.38:0/1410000166 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7aa0 0x555fa3d95df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.255+0000 7f59077ed740 1 --2- 172.21.15.38:0/1410000166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc1a40 0x555fa3de8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.256+0000 7f59077ed740 1 -- 172.21.15.38:0/1410000166 >> 172.21.15.38:0/1410000166 conn(0x555fa3d9a700 msgr2=0x555fa3dbf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.256+0000 7f59077ed740 1 -- 172.21.15.38:0/1410000166 shutdown_connections 2024-04-25T08:35:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.256+0000 7f59077ed740 1 -- 172.21.15.38:0/1410000166 wait complete. 2024-04-25T08:35:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.256+0000 7f59077ed740 1 -- start start 2024-04-25T08:35:07.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.256+0000 7f59077ed740 1 Processor -- start 2024-04-25T08:35:07.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f59077ed740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc1a40 0x555fa3e44f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f59077ed740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7aa0 0x555fa3dcd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f59077ed740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc7e80 0x555fa3d9dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f59077ed740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa3dce3b0 con 0x555fa3dc1a40 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f59077ed740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59001ff3e0 con 0x555fa3dc7e80 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f59077ed740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x555fa3d96690 con 0x555fa3dc7aa0 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f59077ed740 10 client.0 _ll_register_callbacks cb 0x7f58f0034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f58fe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc1a40 0x555fa3e44f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f58fe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc1a40 0x555fa3e44f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42872/0 (socket says 172.21.15.38:42872) 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f58fe59c640 1 -- 172.21.15.38:0/460140670 learned_addr learned my addr 172.21.15.38:0/460140670 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f58fed9d640 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc7e80 0x555fa3d9dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7aa0 0x555fa3dcd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f58fe59c640 1 -- 172.21.15.38:0/460140670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7aa0 msgr2=0x555fa3dcd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f58fe59c640 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7aa0 0x555fa3dcd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f58fe59c640 1 -- 172.21.15.38:0/460140670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc7e80 msgr2=0x555fa3d9dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f58fe59c640 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc7e80 0x555fa3d9dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.257+0000 7f58fe59c640 1 -- 172.21.15.38:0/460140670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fa3e49050 con 0x555fa3dc1a40 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f58fe59c640 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc1a40 0x555fa3e44f60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f58e80136b0 tx=0x7f58e8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80357d1e6eaa2a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f58dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f58dd7fa640 1 -- 172.21.15.38:0/460140670 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8047060 con 0x555fa3dc1a40 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f58dd7fa640 1 -- 172.21.15.38:0/460140670 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58e8015d50 con 0x555fa3dc1a40 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f59077ed740 1 -- 172.21.15.38:0/460140670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58f8081e50 con 0x555fa3dc1a40 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f58dd7fa640 1 -- 172.21.15.38:0/460140670 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58e804d050 con 0x555fa3dc1a40 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f59077ed740 10 client.6291 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f59077ed740 1 -- 172.21.15.38:0/460140670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x555fa3e49260 con 0x555fa3dc1a40 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f58dd7fa640 1 -- 172.21.15.38:0/460140670 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f8081e50 con 0x555fa3dc1a40 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f58dd7fa640 1 -- 172.21.15.38:0/460140670 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f58e8050020 con 0x555fa3dc1a40 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f59077ed740 10 client.6291 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f59077ed740 10 client.6291 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:07.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f59077ed740 20 client.6291 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f59077ed740 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x555fa3d9e2c0 0x555fa3dde430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f59077ed740 4 client.6291 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f59077ed740 1 -- 172.21.15.38:0/460140670 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555fa3d0b690 con 0x555fa3d9e2c0 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.258+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x555fa3d9e2c0 0x555fa3dde430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f58dd7fa640 10 client.6291 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f58fdd9b640 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x555fa3d9e2c0 0x555fa3dde430 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4ed259b86e53c3 server_cookie=beeeb9a75a0531f5 in_seq=0 out_seq=0 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f58dd7fa640 1 -- 172.21.15.38:0/460140670 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555fa3d0b690 con 0x555fa3d9e2c0 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f58dd7fa640 10 client.6291 handle_command_reply: tid=0 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f59077ed740 1 client.6291 shutdown 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f59077ed740 1 -- 172.21.15.38:0/460140670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc1a40 msgr2=0x555fa3e44f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f59077ed740 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc1a40 0x555fa3e44f60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f58e80136b0 tx=0x7f58e8018000 comp rx=0 tx=0).stop 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f59077ed740 1 -- 172.21.15.38:0/460140670 shutdown_connections 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f59077ed740 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x555fa3d9e2c0 0x555fa3dde430 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f59077ed740 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fa3dc7e80 0x555fa3d9dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f59077ed740 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555fa3dc7aa0 0x555fa3dcd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f59077ed740 1 --2- 172.21.15.38:0/460140670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555fa3dc1a40 0x555fa3e44f60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f58dd7fa640 0 client.6291 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f59077ed740 1 -- 172.21.15.38:0/460140670 >> 172.21.15.38:0/460140670 conn(0x555fa3d9a700 msgr2=0x555fa3dbf100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f59077ed740 1 -- 172.21.15.38:0/460140670 shutdown_connections 2024-04-25T08:35:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f59077ed740 1 -- 172.21.15.38:0/460140670 wait complete. 2024-04-25T08:35:07.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.259+0000 7f59077ed740 20 client.6291 trim_cache size 0 max 16384 2024-04-25T08:35:07.260 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-04-25T08:35:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.260+0000 7f5904aee640 1 -- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58e006b4d0 msgr2=0x7f58e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.260+0000 7f5904aee640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58e006b4d0 0x7f58e006d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f58e80099e0 tx=0x7f58e8008040 comp rx=0 tx=0).stop 2024-04-25T08:35:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.260+0000 7f5904aee640 1 -- 172.21.15.38:0/203647504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59001f3720 msgr2=0x7f59001f3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.260+0000 7f5904aee640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59001f3720 0x7f59001f3b60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5900072150 tx=0x7f58f800e530 comp rx=0 tx=0).stop 2024-04-25T08:35:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.260+0000 7f5904aee640 1 -- 172.21.15.38:0/203647504 shutdown_connections 2024-04-25T08:35:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.260+0000 7f5904aee640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59001f3720 0x7f59001f3b60 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.260+0000 7f5904aee640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5900160380 0x7f5900160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.260+0000 7f5904aee640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58e006b4d0 0x7f58e006d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.260+0000 7f5904aee640 1 --2- 172.21.15.38:0/203647504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58ec004800 0x7f5900156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.260+0000 7f5904aee640 1 -- 172.21.15.38:0/203647504 >> 172.21.15.38:0/203647504 conn(0x7f59000b3110 msgr2=0x7f5900161bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.260+0000 7f5904aee640 1 -- 172.21.15.38:0/203647504 shutdown_connections 2024-04-25T08:35:07.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.260+0000 7f5904aee640 1 -- 172.21.15.38:0/203647504 wait complete. 2024-04-25T08:35:07.265 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4c.00000000 parent 2024-04-25T08:35:07.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.313+0000 7f9383047f40 1 Processor -- start 2024-04-25T08:35:07.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.313+0000 7f9383047f40 1 -- start start 2024-04-25T08:35:07.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9383047f40 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55c2594965f0 0x55c2595675a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9383047f40 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c25949adb0 0x55c2594c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9383047f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 0x55c2595710c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9383047f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2595745b0 con 0x55c259552e90 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9383047f40 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c25943fdf0 con 0x55c25949adb0 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9383047f40 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c259573a60 con 0x55c2594965f0 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9381349640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 0x55c2595710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9381349640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 0x55c2595710c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42892/0 (socket says 172.21.15.38:42892) 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9381349640 1 -- 172.21.15.38:0/3630387889 learned_addr learned my addr 172.21.15.38:0/3630387889 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9380347640 1 --2- 172.21.15.38:0/3630387889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c25949adb0 0x55c2594c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9380b48640 1 --2- 172.21.15.38:0/3630387889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55c2594965f0 0x55c2595675a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9381349640 1 -- 172.21.15.38:0/3630387889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55c2594965f0 msgr2=0x55c2595675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9381349640 1 --2- 172.21.15.38:0/3630387889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55c2594965f0 0x55c2595675a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9381349640 1 -- 172.21.15.38:0/3630387889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c25949adb0 msgr2=0x55c2594c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9381349640 1 --2- 172.21.15.38:0/3630387889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c25949adb0 0x55c2594c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9381349640 1 -- 172.21.15.38:0/3630387889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2593c5c50 con 0x55c259552e90 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f9381349640 1 --2- 172.21.15.38:0/3630387889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 0x55c2595710c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9370001680 tx=0x7f9370001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233bd9d14fc390da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f937f345640 1 -- 172.21.15.38:0/3630387889 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370016070 con 0x55c259552e90 2024-04-25T08:35:07.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f937f345640 1 -- 172.21.15.38:0/3630387889 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937000d690 con 0x55c259552e90 2024-04-25T08:35:07.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.314+0000 7f937f345640 1 -- 172.21.15.38:0/3630387889 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937000da40 con 0x55c259552e90 2024-04-25T08:35:07.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.315+0000 7f9383047f40 1 -- 172.21.15.38:0/3630387889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 msgr2=0x55c2595710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.315+0000 7f9383047f40 1 --2- 172.21.15.38:0/3630387889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 0x55c2595710c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9370001680 tx=0x7f9370001b40 comp rx=0 tx=0).stop 2024-04-25T08:35:07.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.315+0000 7f9383047f40 1 -- 172.21.15.38:0/3630387889 shutdown_connections 2024-04-25T08:35:07.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.315+0000 7f9383047f40 1 --2- 172.21.15.38:0/3630387889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 0x55c2595710c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.315+0000 7f9383047f40 1 --2- 172.21.15.38:0/3630387889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c25949adb0 0x55c2594c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.315+0000 7f9383047f40 1 --2- 172.21.15.38:0/3630387889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55c2594965f0 0x55c2595675a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.315+0000 7f9383047f40 1 -- 172.21.15.38:0/3630387889 >> 172.21.15.38:0/3630387889 conn(0x55c259498900 msgr2=0x55c259499690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.315+0000 7f9383047f40 1 -- 172.21.15.38:0/3630387889 shutdown_connections 2024-04-25T08:35:07.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.315+0000 7f9383047f40 1 -- 172.21.15.38:0/3630387889 wait complete. 2024-04-25T08:35:07.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9383047f40 1 Processor -- start 2024-04-25T08:35:07.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9383047f40 1 -- start start 2024-04-25T08:35:07.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9383047f40 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55c2594965f0 0x55c2595ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9383047f40 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c25949adb0 0x55c2594de7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9383047f40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 0x55c259565f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9383047f40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c259499e40 con 0x55c259552e90 2024-04-25T08:35:07.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9383047f40 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c259570510 con 0x55c25949adb0 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9383047f40 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c25949a0e0 con 0x55c2594965f0 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9381349640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 0x55c259565f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9381349640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 0x55c259565f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42896/0 (socket says 172.21.15.38:42896) 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9381349640 1 -- 172.21.15.38:0/2682492653 learned_addr learned my addr 172.21.15.38:0/2682492653 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9380347640 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c25949adb0 0x55c2594de7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.316+0000 7f9380b48640 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55c2594965f0 0x55c2595ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.317+0000 7f9381349640 1 -- 172.21.15.38:0/2682492653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55c2594965f0 msgr2=0x55c2595ff550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.317+0000 7f9381349640 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55c2594965f0 0x55c2595ff550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.317+0000 7f9381349640 1 -- 172.21.15.38:0/2682492653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c25949adb0 msgr2=0x55c2594de7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.317+0000 7f9381349640 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c25949adb0 0x55c2594de7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.317+0000 7f9381349640 1 -- 172.21.15.38:0/2682492653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2595ff200 con 0x55c259552e90 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.317+0000 7f9381349640 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 0x55c259565f80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9370014040 tx=0x7f9370001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.317+0000 7f937e343640 1 -- 172.21.15.38:0/2682492653 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370016040 con 0x55c259552e90 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.317+0000 7f937e343640 1 -- 172.21.15.38:0/2682492653 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93700117b0 con 0x55c259552e90 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.317+0000 7f9383047f40 1 -- 172.21.15.38:0/2682492653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c259601ae0 con 0x55c259552e90 2024-04-25T08:35:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.317+0000 7f937e343640 1 -- 172.21.15.38:0/2682492653 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93700119e0 con 0x55c259552e90 2024-04-25T08:35:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.317+0000 7f9383047f40 1 -- 172.21.15.38:0/2682492653 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c259572e70 con 0x55c259552e90 2024-04-25T08:35:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.319+0000 7f937e343640 1 -- 172.21.15.38:0/2682492653 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9370011b80 con 0x55c259552e90 2024-04-25T08:35:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.319+0000 7f937e343640 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9360069eb0 0x7f936006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.319+0000 7f937e343640 1 -- 172.21.15.38:0/2682492653 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f937008a620 con 0x55c259552e90 2024-04-25T08:35:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.319+0000 7f9380b48640 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9360069eb0 0x7f936006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.319+0000 7f9383047f40 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x55c25954e350 0x55c25960a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.319+0000 7f9383047f40 1 -- 172.21.15.38:0/2682492653 --> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] -- osd_op(unknown.0.0:1 8.3a 8:5fa3b718:::10000000a4c.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54) v9 -- 0x55c2594decf0 con 0x55c25954e350 2024-04-25T08:35:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.319+0000 7f9380347640 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x55c25954e350 0x55c25960a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.320+0000 7f9380b48640 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9360069eb0 0x7f936006c370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9378005fd0 tx=0x7f937800e040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.320+0000 7f9380347640 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x55c25954e350 0x55c25960a8b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.321+0000 7f9380347640 1 -- 172.21.15.38:0/2682492653 <== osd.1 v2:172.21.15.38:6802/3579687234 1 ==== osd_op_reply(1 10000000a4c.00000000 [rmxattr] v54'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9368051070 con 0x55c25954e350 2024-04-25T08:35:07.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.321+0000 7f9383047f40 1 -- 172.21.15.38:0/2682492653 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x55c25954e350 msgr2=0x55c25960a8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.321+0000 7f9383047f40 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x55c25954e350 0x55c25960a8b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.321+0000 7f9383047f40 1 -- 172.21.15.38:0/2682492653 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9360069eb0 msgr2=0x7f936006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.321+0000 7f9383047f40 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9360069eb0 0x7f936006c370 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9378005fd0 tx=0x7f937800e040 comp rx=0 tx=0).stop 2024-04-25T08:35:07.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.321+0000 7f9383047f40 1 -- 172.21.15.38:0/2682492653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 msgr2=0x55c259565f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.321+0000 7f9383047f40 1 --2- 172.21.15.38:0/2682492653 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55c259552e90 0x55c259565f80 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9370014040 tx=0x7f9370001250 comp rx=0 tx=0).stop 2024-04-25T08:35:07.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.321+0000 7f9381349640 1 -- 172.21.15.38:0/2682492653 reap_dead start 2024-04-25T08:35:07.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.321+0000 7f9383047f40 1 -- 172.21.15.38:0/2682492653 shutdown_connections 2024-04-25T08:35:07.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.321+0000 7f9383047f40 1 -- 172.21.15.38:0/2682492653 >> 172.21.15.38:0/2682492653 conn(0x55c259498900 msgr2=0x55c2594dcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.321+0000 7f9383047f40 1 -- 172.21.15.38:0/2682492653 shutdown_connections 2024-04-25T08:35:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.321+0000 7f9383047f40 1 -- 172.21.15.38:0/2682492653 wait complete. 2024-04-25T08:35:07.320 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T08:35:07.320 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:07.320 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T08:35:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.469+0000 7f0e3dc3f640 1 Processor -- start 2024-04-25T08:35:07.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.469+0000 7f0e3dc3f640 1 -- start start 2024-04-25T08:35:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.470+0000 7f0e3dc3f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0e380d33d0 0x7f0e380d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.470+0000 7f0e3dc3f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e380d1e90 0x7f0e380b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.470+0000 7f0e3dc3f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e380d2330 con 0x7f0e380d0320 2024-04-25T08:35:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.470+0000 7f0e3dc3f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e380d2610 con 0x7f0e380d33d0 2024-04-25T08:35:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.470+0000 7f0e3dc3f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380d28f0 con 0x7f0e380d1e90 2024-04-25T08:35:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.470+0000 7f0e36ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0e380d33d0 0x7f0e380d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.470+0000 7f0e36ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0e380d33d0 0x7f0e380d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42908/0 (socket says 172.21.15.38:42908) 2024-04-25T08:35:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.470+0000 7f0e36ffd640 1 -- 172.21.15.38:0/1978476116 learned_addr learned my addr 172.21.15.38:0/1978476116 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.470+0000 7f0e37fff640 1 --2- 172.21.15.38:0/1978476116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e380d1e90 0x7f0e380b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.471+0000 7f0e36ffd640 1 -- 172.21.15.38:0/1978476116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e380d1e90 msgr2=0x7f0e380b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.471+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/1978476116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e380d1e90 0x7f0e380b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.471+0000 7f0e36ffd640 1 -- 172.21.15.38:0/1978476116 >> v1:172.21.15.107:6789/0 conn(0x7f0e380d0320 legacy=0x7f0e380d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.471+0000 7f0e36ffd640 1 -- 172.21.15.38:0/1978476116 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e380d2bd0 con 0x7f0e380d33d0 2024-04-25T08:35:07.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.471+0000 7f0e37fff640 1 --2- 172.21.15.38:0/1978476116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e380d1e90 0x7f0e380b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.471+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/1978476116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e380d33d0 0x7f0e380d1950 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009870 tx=0x7f0e2c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f487c190a74eb23b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.471+0000 7f0e367fc640 1 -- 172.21.15.38:0/1978476116 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c037070 con 0x7f0e380d33d0 2024-04-25T08:35:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.471+0000 7f0e367fc640 1 -- 172.21.15.38:0/1978476116 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c02ae50 con 0x7f0e380d33d0 2024-04-25T08:35:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.471+0000 7f0e367fc640 1 -- 172.21.15.38:0/1978476116 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c032860 con 0x7f0e380d33d0 2024-04-25T08:35:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.472+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1978476116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e380d33d0 msgr2=0x7f0e380d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.472+0000 7f0e3dc3f640 1 --2- 172.21.15.38:0/1978476116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e380d33d0 0x7f0e380d1950 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009870 tx=0x7f0e2c02a300 comp rx=0 tx=0).stop 2024-04-25T08:35:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.472+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1978476116 shutdown_connections 2024-04-25T08:35:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.472+0000 7f0e3dc3f640 1 --2- 172.21.15.38:0/1978476116 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e380d1e90 0x7f0e380b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.472+0000 7f0e3dc3f640 1 --2- 172.21.15.38:0/1978476116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e380d33d0 0x7f0e380d1950 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.472+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1978476116 >> 172.21.15.38:0/1978476116 conn(0x7f0e380668f0 msgr2=0x7f0e38066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.472+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1978476116 shutdown_connections 2024-04-25T08:35:07.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.472+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1978476116 wait complete. 2024-04-25T08:35:07.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.473+0000 7f0e3dc3f640 1 Processor -- start 2024-04-25T08:35:07.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.473+0000 7f0e3dc3f640 1 -- start start 2024-04-25T08:35:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e3dc3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e380d0320 0x7f0e380c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e3dc3f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e380d1e90 0x7f0e380ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e3dc3f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e380d33d0 0x7f0e380cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e3dc3f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38161390 con 0x7f0e380d0320 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e3dc3f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38161210 con 0x7f0e380d1e90 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e3dc3f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e380b9720 con 0x7f0e380d33d0 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e377fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e380d0320 0x7f0e380c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e377fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e380d0320 0x7f0e380c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42918/0 (socket says 172.21.15.38:42918) 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e377fe640 1 -- 172.21.15.38:0/1757805503 learned_addr learned my addr 172.21.15.38:0/1757805503 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e380d1e90 0x7f0e380ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e37fff640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e380d33d0 0x7f0e380cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e377fe640 1 -- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e380d33d0 msgr2=0x7f0e380cd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e377fe640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e380d33d0 0x7f0e380cd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e377fe640 1 -- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e380d1e90 msgr2=0x7f0e380ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e377fe640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e380d1e90 0x7f0e380ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e377fe640 1 -- 172.21.15.38:0/1757805503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e380c9a70 con 0x7f0e380d0320 2024-04-25T08:35:07.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e380d1e90 0x7f0e380ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.474+0000 7f0e37fff640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e380d33d0 0x7f0e380cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.475+0000 7f0e377fe640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e380d0320 0x7f0e380c9530 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800c7c0 tx=0x7f0e2800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.475+0000 7f0e1ffff640 1 -- 172.21.15.38:0/1757805503 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28016020 con 0x7f0e380d0320 2024-04-25T08:35:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.475+0000 7f0e1ffff640 1 -- 172.21.15.38:0/1757805503 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e28006d50 con 0x7f0e380d0320 2024-04-25T08:35:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.475+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1757805503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e380b5a80 con 0x7f0e380d0320 2024-04-25T08:35:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.475+0000 7f0e1ffff640 1 -- 172.21.15.38:0/1757805503 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28003bb0 con 0x7f0e380d0320 2024-04-25T08:35:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.475+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1757805503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e381e63c0 con 0x7f0e380d0320 2024-04-25T08:35:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.476+0000 7f0e1ffff640 1 -- 172.21.15.38:0/1757805503 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e2800a850 con 0x7f0e380d0320 2024-04-25T08:35:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.477+0000 7f0e1ffff640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0e2406b4d0 0x7f0e2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.477+0000 7f0e1ffff640 1 -- 172.21.15.38:0/1757805503 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e2808a030 con 0x7f0e380d0320 2024-04-25T08:35:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.477+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0e2406b4d0 0x7f0e2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.478+0000 7f0e40a1e740 1 Processor -- start 2024-04-25T08:35:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.478+0000 7f0e40a1e740 1 -- start start 2024-04-25T08:35:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.478+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0e2406b4d0 0x7f0e2406d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c034000 tx=0x7f0e2c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.478+0000 7f0e40a1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e19360 0x5617f0e397d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.478+0000 7f0e40a1e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0e20c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.478+0000 7f0e40a1e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e21140 0x5617f0df67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.478+0000 7f0e40a1e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f0d59ee0 con 0x5617f0df8360 2024-04-25T08:35:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.478+0000 7f0e40a1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f0d02c50 con 0x5617f0e19360 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.478+0000 7f0e40a1e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f0d55f80 con 0x5617f0e21140 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e36ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0e20c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e36ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0e20c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42930/0 (socket says 172.21.15.38:42930) 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e36ffd640 1 -- 172.21.15.38:0/616003192 learned_addr learned my addr 172.21.15.38:0/616003192 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e377fe640 1 --2- 172.21.15.38:0/616003192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e19360 0x5617f0e397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e37fff640 1 --2- 172.21.15.38:0/616003192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e21140 0x5617f0df67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e36ffd640 1 -- 172.21.15.38:0/616003192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e21140 msgr2=0x5617f0df67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/616003192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e21140 0x5617f0df67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e36ffd640 1 -- 172.21.15.38:0/616003192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e19360 msgr2=0x5617f0e397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/616003192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e19360 0x5617f0e397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e36ffd640 1 -- 172.21.15.38:0/616003192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f0b3ac00 con 0x5617f0df8360 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e37fff640 1 --2- 172.21.15.38:0/616003192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e21140 0x5617f0df67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/616003192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0e20c00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c02a340 tx=0x7f0e2c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf7c3d73ed7fc43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e1d7fa640 1 -- 172.21.15.38:0/616003192 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c04d070 con 0x5617f0df8360 2024-04-25T08:35:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e1d7fa640 1 -- 172.21.15.38:0/616003192 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c02ae50 con 0x5617f0df8360 2024-04-25T08:35:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.479+0000 7f0e1d7fa640 1 -- 172.21.15.38:0/616003192 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c037040 con 0x5617f0df8360 2024-04-25T08:35:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.480+0000 7f0e40a1e740 1 -- 172.21.15.38:0/616003192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 msgr2=0x5617f0e20c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.480+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/616003192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0e20c00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c02a340 tx=0x7f0e2c02c000 comp rx=0 tx=0).stop 2024-04-25T08:35:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.480+0000 7f0e40a1e740 1 -- 172.21.15.38:0/616003192 shutdown_connections 2024-04-25T08:35:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.480+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/616003192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e21140 0x5617f0df67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.480+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/616003192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0e20c00 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.480+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/616003192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e19360 0x5617f0e397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.480+0000 7f0e40a1e740 1 -- 172.21.15.38:0/616003192 >> 172.21.15.38:0/616003192 conn(0x5617f0dbd810 msgr2=0x5617f0e3bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.480+0000 7f0e40a1e740 1 -- 172.21.15.38:0/616003192 shutdown_connections 2024-04-25T08:35:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.480+0000 7f0e40a1e740 1 -- 172.21.15.38:0/616003192 wait complete. 2024-04-25T08:35:07.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.481+0000 7f0e40a1e740 1 -- start start 2024-04-25T08:35:07.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.481+0000 7f0e40a1e740 1 Processor -- start 2024-04-25T08:35:07.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.481+0000 7f0e40a1e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0e17c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.481+0000 7f0e40a1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e19360 0x5617f0e4bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.481+0000 7f0e40a1e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e21140 0x5617f0e33e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.481+0000 7f0e40a1e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f0df6330 con 0x5617f0df8360 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.481+0000 7f0e40a1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f0cf8340 con 0x5617f0e19360 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.481+0000 7f0e40a1e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f0df11a0 con 0x5617f0e21140 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.481+0000 7f0e40a1e740 10 client.0 _ll_register_callbacks cb 0x5617f0a6dfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.481+0000 7f0e377fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0e17c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e377fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0e17c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42934/0 (socket says 172.21.15.38:42934) 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e377fe640 1 -- 172.21.15.38:0/2308741763 learned_addr learned my addr 172.21.15.38:0/2308741763 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e37fff640 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e21140 0x5617f0e33e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e19360 0x5617f0e4bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e377fe640 1 -- 172.21.15.38:0/2308741763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e21140 msgr2=0x5617f0e33e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e377fe640 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e21140 0x5617f0e33e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e377fe640 1 -- 172.21.15.38:0/2308741763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e19360 msgr2=0x5617f0e4bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e377fe640 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e19360 0x5617f0e4bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e377fe640 1 -- 172.21.15.38:0/2308741763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617f0a2e4f0 con 0x5617f0df8360 2024-04-25T08:35:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e377fe640 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0e17c00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0e28054b10 tx=0x7f0e28058ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd81b80ae8b047e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e167fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e167fc640 1 -- 172.21.15.38:0/2308741763 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28060dc0 con 0x5617f0df8360 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e167fc640 1 -- 172.21.15.38:0/2308741763 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e2800ab40 con 0x5617f0df8360 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e40a1e740 1 -- 172.21.15.38:0/2308741763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e2c02ae50 con 0x5617f0df8360 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e167fc640 1 -- 172.21.15.38:0/2308741763 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e28076050 con 0x5617f0df8360 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e40a1e740 10 client.6312 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e40a1e740 1 -- 172.21.15.38:0/2308741763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5617f0b9b390 con 0x5617f0df8360 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.482+0000 7f0e167fc640 1 -- 172.21.15.38:0/2308741763 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c02ae50 con 0x5617f0df8360 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e167fc640 1 -- 172.21.15.38:0/2308741763 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0e28057b00 con 0x5617f0df8360 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e40a1e740 10 client.6312 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e40a1e740 10 client.6312 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e40a1e740 20 client.6312 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5617f0e3dd30 0x5617f0e351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e40a1e740 4 client.6312 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e40a1e740 1 -- 172.21.15.38:0/2308741763 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617f0d67420 con 0x5617f0e3dd30 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5617f0e3dd30 0x5617f0e351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5617f0e3dd30 0x5617f0e351c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=321dc564389e3342 server_cookie=96f822be9d83ecf7 in_seq=0 out_seq=0 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e167fc640 10 client.6312 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e167fc640 1 -- 172.21.15.38:0/2308741763 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5617f0d67420 con 0x5617f0e3dd30 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e167fc640 10 client.6312 handle_command_reply: tid=0 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.483+0000 7f0e40a1e740 1 client.6312 shutdown 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.484+0000 7f0e40a1e740 1 -- 172.21.15.38:0/2308741763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 msgr2=0x5617f0e17c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.484+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0e17c00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f0e28054b10 tx=0x7f0e28058ee0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.484+0000 7f0e40a1e740 1 -- 172.21.15.38:0/2308741763 shutdown_connections 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.484+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5617f0e3dd30 0x5617f0e351c0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.484+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e21140 0x5617f0e33e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.484+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e19360 0x5617f0e4bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.484+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/2308741763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0e17c00 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.484+0000 7f0e167fc640 0 client.6312 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:07.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.484+0000 7f0e40a1e740 1 -- 172.21.15.38:0/2308741763 >> 172.21.15.38:0/2308741763 conn(0x5617f0dbd810 msgr2=0x5617f0df0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.484+0000 7f0e40a1e740 1 -- 172.21.15.38:0/2308741763 shutdown_connections 2024-04-25T08:35:07.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.484+0000 7f0e40a1e740 1 -- 172.21.15.38:0/2308741763 wait complete. 2024-04-25T08:35:07.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.484+0000 7f0e40a1e740 20 client.6312 trim_cache size 0 max 16384 2024-04-25T08:35:07.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.499+0000 7f0e40a1e740 1 Processor -- start 2024-04-25T08:35:07.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.499+0000 7f0e40a1e740 1 -- start start 2024-04-25T08:35:07.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e40a1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0df8360 0x5617f0e33e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e40a1e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e19360 0x5617f0df4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e40a1e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0e21140 0x5617f0e17c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e40a1e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e382003b0 con 0x5617f0e21140 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e40a1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e381ff5a0 con 0x5617f0df8360 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e40a1e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e381e7710 con 0x5617f0e19360 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e37fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0e21140 0x5617f0e17c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e37fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0e21140 0x5617f0e17c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42940/0 (socket says 172.21.15.38:42940) 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e37fff640 1 -- 172.21.15.38:0/350280589 learned_addr learned my addr 172.21.15.38:0/350280589 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/350280589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e19360 0x5617f0df4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e377fe640 1 --2- 172.21.15.38:0/350280589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0df8360 0x5617f0e33e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e37fff640 1 -- 172.21.15.38:0/350280589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e19360 msgr2=0x5617f0df4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e37fff640 1 --2- 172.21.15.38:0/350280589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e19360 0x5617f0df4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e37fff640 1 -- 172.21.15.38:0/350280589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0df8360 msgr2=0x5617f0e33e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e37fff640 1 --2- 172.21.15.38:0/350280589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0df8360 0x5617f0e33e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e37fff640 1 -- 172.21.15.38:0/350280589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f0a07670 con 0x5617f0e21140 2024-04-25T08:35:07.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/350280589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e19360 0x5617f0df4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.500+0000 7f0e377fe640 1 --2- 172.21.15.38:0/350280589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0df8360 0x5617f0e33e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.501+0000 7f0e37fff640 1 --2- 172.21.15.38:0/350280589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0e21140 0x5617f0e17c90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0e300098e0 tx=0x7f0e30009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcaede8e4672b38c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.501+0000 7f0e1cff9640 1 -- 172.21.15.38:0/350280589 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3000f040 con 0x5617f0e21140 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.501+0000 7f0e1cff9640 1 -- 172.21.15.38:0/350280589 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e300042f0 con 0x5617f0e21140 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.501+0000 7f0e1cff9640 1 -- 172.21.15.38:0/350280589 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30015b40 con 0x5617f0e21140 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.501+0000 7f0e40a1e740 1 -- 172.21.15.38:0/350280589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0e21140 msgr2=0x5617f0e17c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.501+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/350280589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0e21140 0x5617f0e17c90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0e300098e0 tx=0x7f0e30009da0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.501+0000 7f0e40a1e740 1 -- 172.21.15.38:0/350280589 shutdown_connections 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.501+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/350280589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0e21140 0x5617f0e17c90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.501+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/350280589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e19360 0x5617f0df4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.501+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/350280589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0df8360 0x5617f0e33e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.501+0000 7f0e40a1e740 1 -- 172.21.15.38:0/350280589 >> 172.21.15.38:0/350280589 conn(0x5617f0dbd810 msgr2=0x5617f0df6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.502+0000 7f0e40a1e740 1 -- 172.21.15.38:0/350280589 shutdown_connections 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.502+0000 7f0e40a1e740 1 -- 172.21.15.38:0/350280589 wait complete. 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.502+0000 7f0e40a1e740 1 -- start start 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.502+0000 7f0e40a1e740 1 Processor -- start 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e40a1e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0ea7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e40a1e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e19360 0x5617f0e24420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e40a1e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e21140 0x5617f0e1eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e40a1e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e381e7d10 con 0x5617f0df8360 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e40a1e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f0e1cf70 con 0x5617f0e21140 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e40a1e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f0e1f8f0 con 0x5617f0e19360 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e40a1e740 10 client.0 _ll_register_callbacks cb 0x7f0e2c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e377fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0ea7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e377fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0ea7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42948/0 (socket says 172.21.15.38:42948) 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e377fe640 1 -- 172.21.15.38:0/3338060801 learned_addr learned my addr 172.21.15.38:0/3338060801 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e19360 0x5617f0e24420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e37fff640 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e21140 0x5617f0e1eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e377fe640 1 -- 172.21.15.38:0/3338060801 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e19360 msgr2=0x5617f0e24420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e377fe640 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e19360 0x5617f0e24420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e377fe640 1 -- 172.21.15.38:0/3338060801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e21140 msgr2=0x5617f0e1eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e377fe640 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e21140 0x5617f0e1eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.503+0000 7f0e377fe640 1 -- 172.21.15.38:0/3338060801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617f0e1f610 con 0x5617f0df8360 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e377fe640 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0ea7040 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800cfd0 tx=0x7f0e28088330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f5a96d24b91b07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e167fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e167fc640 1 -- 172.21.15.38:0/3338060801 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2807b070 con 0x5617f0df8360 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e167fc640 1 -- 172.21.15.38:0/3338060801 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e2800ad40 con 0x5617f0df8360 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e40a1e740 1 -- 172.21.15.38:0/3338060801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e300042f0 con 0x5617f0df8360 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e167fc640 1 -- 172.21.15.38:0/3338060801 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e28060dc0 con 0x5617f0df8360 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e40a1e740 10 client.6318 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:07.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e40a1e740 1 -- 172.21.15.38:0/3338060801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5617f0ea3c40 con 0x5617f0df8360 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e167fc640 1 -- 172.21.15.38:0/3338060801 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e300042f0 con 0x5617f0df8360 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e167fc640 1 -- 172.21.15.38:0/3338060801 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0e2807e020 con 0x5617f0df8360 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e40a1e740 10 client.6318 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e40a1e740 10 client.6318 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e40a1e740 20 client.6318 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5617f0e1c230 0x5617f0e1c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e40a1e740 4 client.6318 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e40a1e740 1 -- 172.21.15.38:0/3338060801 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x5617f0d61770 con 0x5617f0e1c230 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.504+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5617f0e1c230 0x5617f0e1c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e167fc640 10 client.6318 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e36ffd640 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5617f0e1c230 0x5617f0e1c640 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46b4d885fb1477dc server_cookie=7a708b1ffbbb5d16 in_seq=0 out_seq=0 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e167fc640 1 -- 172.21.15.38:0/3338060801 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5617f0d61770 con 0x5617f0e1c230 2024-04-25T08:35:07.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e167fc640 10 client.6318 handle_command_reply: tid=0 2024-04-25T08:35:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e40a1e740 1 client.6318 shutdown 2024-04-25T08:35:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e40a1e740 1 -- 172.21.15.38:0/3338060801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 msgr2=0x5617f0ea7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0ea7040 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800cfd0 tx=0x7f0e28088330 comp rx=0 tx=0).stop 2024-04-25T08:35:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e40a1e740 1 -- 172.21.15.38:0/3338060801 shutdown_connections 2024-04-25T08:35:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5617f0e1c230 0x5617f0e1c640 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5617f0e21140 0x5617f0e1eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5617f0e19360 0x5617f0e24420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e40a1e740 1 --2- 172.21.15.38:0/3338060801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5617f0df8360 0x5617f0ea7040 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e40a1e740 1 -- 172.21.15.38:0/3338060801 >> 172.21.15.38:0/3338060801 conn(0x5617f0dbd810 msgr2=0x5617f0e18a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e167fc640 0 client.6318 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e40a1e740 1 -- 172.21.15.38:0/3338060801 shutdown_connections 2024-04-25T08:35:07.507 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f2633938640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4c [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v545 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6153=0-4194304@1} caps={6153=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6153 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5590ce61d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000a4c:[<0x10000000001/new_file_0 v545>,<0x10000000000/ceph-qa-suite v57>,<0x1/scrub_test_path v26>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T08:35:07.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e40a1e740 1 -- 172.21.15.38:0/3338060801 wait complete. 2024-04-25T08:35:07.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.505+0000 7f0e40a1e740 20 client.6318 trim_cache size 0 max 16384 2024-04-25T08:35:07.508 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:07.508 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:35:07.509 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "0572eb7f-a143-4b1d-b5ea-39b7d7647297", 2024-04-25T08:35:07.509 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:35:07.509 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:07.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.506+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0e2406b4d0 msgr2=0x7f0e2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.506+0000 7f0e3dc3f640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0e2406b4d0 0x7f0e2406d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c034000 tx=0x7f0e2c02e000 comp rx=0 tx=0).stop 2024-04-25T08:35:07.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.506+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1757805503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e380d0320 msgr2=0x7f0e380c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.506+0000 7f0e3dc3f640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e380d0320 0x7f0e380c9530 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0e2800c7c0 tx=0x7f0e2800cc80 comp rx=0 tx=0).stop 2024-04-25T08:35:07.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.506+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1757805503 shutdown_connections 2024-04-25T08:35:07.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.506+0000 7f0e3dc3f640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0e2406b4d0 0x7f0e2406d990 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.506+0000 7f0e3dc3f640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0e380d33d0 0x7f0e380cd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.506+0000 7f0e3dc3f640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0e380d1e90 0x7f0e380ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.506+0000 7f0e3dc3f640 1 --2- 172.21.15.38:0/1757805503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e380d0320 0x7f0e380c9530 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.506+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1757805503 >> 172.21.15.38:0/1757805503 conn(0x7f0e380668f0 msgr2=0x7f0e38160140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.506+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1757805503 shutdown_connections 2024-04-25T08:35:07.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.506+0000 7f0e3dc3f640 1 -- 172.21.15.38:0/1757805503 wait complete. 2024-04-25T08:35:07.511 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': '0572eb7f-a143-4b1d-b5ea-39b7d7647297', 'mode': 'asynchronous'}' 2024-04-25T08:35:07.512 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:35:07.512 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:35:07.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.619+0000 7fedc7529640 1 Processor -- start 2024-04-25T08:35:07.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.619+0000 7fedc7529640 1 -- start start 2024-04-25T08:35:07.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.620+0000 7fedc7529640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fedc015db80 0x7fedc00cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.620+0000 7fedc7529640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fedc00cfb70 0x7fedc00ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.620+0000 7fedc7529640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc00ac430 con 0x7fedc00ccfc0 2024-04-25T08:35:07.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.620+0000 7fedc7529640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc00cffe0 con 0x7fedc00cfb70 2024-04-25T08:35:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.620+0000 7fedc7529640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc00ac1e0 con 0x7fedc015db80 2024-04-25T08:35:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.620+0000 7fedc5ac6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fedc00cfb70 0x7fedc00ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.620+0000 7fedc5ac6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fedc00cfb70 0x7fedc00ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42952/0 (socket says 172.21.15.38:42952) 2024-04-25T08:35:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.620+0000 7fedc5ac6640 1 -- 172.21.15.38:0/2546599605 learned_addr learned my addr 172.21.15.38:0/2546599605 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.620+0000 7fedc52c5640 1 --2- 172.21.15.38:0/2546599605 >> v2:172.21.15.107:3300/0 conn(0x7fedc015db80 0x7fedc00cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.620+0000 7fedc5ac6640 1 -- 172.21.15.38:0/2546599605 >> v2:172.21.15.107:3300/0 conn(0x7fedc015db80 msgr2=0x7fedc00cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.620+0000 7fedc5ac6640 1 --2- 172.21.15.38:0/2546599605 >> v2:172.21.15.107:3300/0 conn(0x7fedc015db80 0x7fedc00cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.621+0000 7fedc5ac6640 1 -- 172.21.15.38:0/2546599605 >> v1:172.21.15.107:6789/0 conn(0x7fedc00ccfc0 legacy=0x7fedc00c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.621+0000 7fedc5ac6640 1 -- 172.21.15.38:0/2546599605 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00ac710 con 0x7fedc00cfb70 2024-04-25T08:35:07.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.621+0000 7fedc5ac6640 1 --2- 172.21.15.38:0/2546599605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedc00cfb70 0x7fedc00ca1e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fedc006bbc0 tx=0x7fedb802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbf6efec3265ac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.621+0000 7fedb7fff640 1 -- 172.21.15.38:0/2546599605 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8036070 con 0x7fedc00cfb70 2024-04-25T08:35:07.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.621+0000 7fedb7fff640 1 -- 172.21.15.38:0/2546599605 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb8031600 con 0x7fedc00cfb70 2024-04-25T08:35:07.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.621+0000 7fedb7fff640 1 -- 172.21.15.38:0/2546599605 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb80319d0 con 0x7fedc00cfb70 2024-04-25T08:35:07.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.622+0000 7fedc7529640 1 -- 172.21.15.38:0/2546599605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedc00cfb70 msgr2=0x7fedc00ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.622+0000 7fedc7529640 1 --2- 172.21.15.38:0/2546599605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedc00cfb70 0x7fedc00ca1e0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fedc006bbc0 tx=0x7fedb802a730 comp rx=0 tx=0).stop 2024-04-25T08:35:07.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.622+0000 7fedc7529640 1 -- 172.21.15.38:0/2546599605 shutdown_connections 2024-04-25T08:35:07.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.622+0000 7fedc7529640 1 --2- 172.21.15.38:0/2546599605 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedc00cfb70 0x7fedc00ca1e0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.622+0000 7fedc7529640 1 --2- 172.21.15.38:0/2546599605 >> v2:172.21.15.107:3300/0 conn(0x7fedc015db80 0x7fedc00cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.622+0000 7fedc7529640 1 -- 172.21.15.38:0/2546599605 >> 172.21.15.38:0/2546599605 conn(0x7fedc006adc0 msgr2=0x7fedc006b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.622+0000 7fedc7529640 1 -- 172.21.15.38:0/2546599605 shutdown_connections 2024-04-25T08:35:07.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.622+0000 7fedc7529640 1 -- 172.21.15.38:0/2546599605 wait complete. 2024-04-25T08:35:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.623+0000 7fedc7529640 1 Processor -- start 2024-04-25T08:35:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.623+0000 7fedc7529640 1 -- start start 2024-04-25T08:35:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc7529640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc00ccfc0 0x7fedc015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc7529640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedc00cfb70 0x7fedc015b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc7529640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fedc015db80 0x7fedc0155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc7529640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc015df60 con 0x7fedc00cfb70 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc7529640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00accd0 con 0x7fedc00ccfc0 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc7529640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc015e2c0 con 0x7fedc015db80 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc4ac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedc00cfb70 0x7fedc015b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc4ac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedc00cfb70 0x7fedc015b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42958/0 (socket says 172.21.15.38:42958) 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc4ac4640 1 -- 172.21.15.38:0/1034477520 learned_addr learned my addr 172.21.15.38:0/1034477520 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc52c5640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc00ccfc0 0x7fedc015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc5ac6640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fedc015db80 0x7fedc0155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc4ac4640 1 -- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fedc015db80 msgr2=0x7fedc0155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fedc015db80 0x7fedc0155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc4ac4640 1 -- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc00ccfc0 msgr2=0x7fedc015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc00ccfc0 0x7fedc015cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.624+0000 7fedc4ac4640 1 -- 172.21.15.38:0/1034477520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00c6ae0 con 0x7fedc00cfb70 2024-04-25T08:35:07.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.625+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedc00cfb70 0x7fedc015b170 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fedb000e710 tx=0x7fedb000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.625+0000 7fedb5ffb640 1 -- 172.21.15.38:0/1034477520 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00052d0 con 0x7fedc00cfb70 2024-04-25T08:35:07.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.625+0000 7fedb5ffb640 1 -- 172.21.15.38:0/1034477520 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb0005470 con 0x7fedc00cfb70 2024-04-25T08:35:07.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.625+0000 7fedb5ffb640 1 -- 172.21.15.38:0/1034477520 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0001b50 con 0x7fedc00cfb70 2024-04-25T08:35:07.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.625+0000 7fedc7529640 1 -- 172.21.15.38:0/1034477520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc00c6560 con 0x7fedc00cfb70 2024-04-25T08:35:07.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.625+0000 7fedc7529640 1 -- 172.21.15.38:0/1034477520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc00726b0 con 0x7fedc00cfb70 2024-04-25T08:35:07.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.626+0000 7fedb5ffb640 1 -- 172.21.15.38:0/1034477520 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fedb00035f0 con 0x7fedc00cfb70 2024-04-25T08:35:07.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.627+0000 7fedb5ffb640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feda406b500 0x7feda406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.627+0000 7fedb5ffb640 1 -- 172.21.15.38:0/1034477520 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fedb008c4a0 con 0x7fedc00cfb70 2024-04-25T08:35:07.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.627+0000 7fedc52c5640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feda406b500 0x7feda406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.628+0000 7fedc52c5640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feda406b500 0x7feda406d9c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009aa0 tx=0x7fedbc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.628+0000 7fedca22d740 1 Processor -- start 2024-04-25T08:35:07.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.628+0000 7fedca22d740 1 -- start start 2024-04-25T08:35:07.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedca22d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedca22d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8a5770 0x55df9d8ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedca22d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8f1470 0x55df9d8f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedca22d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9d80eda0 con 0x55df9d8a5770 2024-04-25T08:35:07.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedca22d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9d7b7b80 con 0x55df9d8d5510 2024-04-25T08:35:07.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedca22d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df9d80ae40 con 0x55df9d8f1470 2024-04-25T08:35:07.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedc4ac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8a5770 0x55df9d8ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedc4ac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8a5770 0x55df9d8ab7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42960/0 (socket says 172.21.15.38:42960) 2024-04-25T08:35:07.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedc4ac4640 1 -- 172.21.15.38:0/1292324726 learned_addr learned my addr 172.21.15.38:0/1292324726 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedc5ac6640 1 --2- 172.21.15.38:0/1292324726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8f1470 0x55df9d8f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedc52c5640 1 --2- 172.21.15.38:0/1292324726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedc4ac4640 1 -- 172.21.15.38:0/1292324726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8f1470 msgr2=0x55df9d8f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/1292324726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8f1470 0x55df9d8f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedc4ac4640 1 -- 172.21.15.38:0/1292324726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 msgr2=0x55df9d8d58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/1292324726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedc4ac4640 1 -- 172.21.15.38:0/1292324726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df9d7c1f10 con 0x55df9d8a5770 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.629+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/1292324726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8a5770 0x55df9d8ab7a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fedb008a840 tx=0x7fedb0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e3a270adef3009 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.630+0000 7feda2ffd640 1 -- 172.21.15.38:0/1292324726 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb001f040 con 0x55df9d8a5770 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.630+0000 7feda2ffd640 1 -- 172.21.15.38:0/1292324726 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb005be50 con 0x55df9d8a5770 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.630+0000 7feda2ffd640 1 -- 172.21.15.38:0/1292324726 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0063230 con 0x55df9d8a5770 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.630+0000 7fedca22d740 1 -- 172.21.15.38:0/1292324726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8a5770 msgr2=0x55df9d8ab7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.630+0000 7fedca22d740 1 --2- 172.21.15.38:0/1292324726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8a5770 0x55df9d8ab7a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fedb008a840 tx=0x7fedb0000bc0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.630+0000 7fedca22d740 1 -- 172.21.15.38:0/1292324726 shutdown_connections 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.630+0000 7fedca22d740 1 --2- 172.21.15.38:0/1292324726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8f1470 0x55df9d8f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.630+0000 7fedca22d740 1 --2- 172.21.15.38:0/1292324726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8a5770 0x55df9d8ab7a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.630+0000 7fedca22d740 1 --2- 172.21.15.38:0/1292324726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.630+0000 7fedca22d740 1 -- 172.21.15.38:0/1292324726 >> 172.21.15.38:0/1292324726 conn(0x55df9d8a79e0 msgr2=0x55df9d8f0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.630+0000 7fedca22d740 1 -- 172.21.15.38:0/1292324726 shutdown_connections 2024-04-25T08:35:07.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.630+0000 7fedca22d740 1 -- 172.21.15.38:0/1292324726 wait complete. 2024-04-25T08:35:07.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.631+0000 7fedca22d740 1 -- start start 2024-04-25T08:35:07.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.631+0000 7fedca22d740 1 Processor -- start 2024-04-25T08:35:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedca22d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d5510 0x55df9d8d8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedca22d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8f1470 0x55df9d8d91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedca22d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8d6b50 0x55df9d8c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedca22d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9d8abce0 con 0x55df9d8d5510 2024-04-25T08:35:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedca22d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9d7ad4b0 con 0x55df9d8f1470 2024-04-25T08:35:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedca22d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df9d8ab310 con 0x55df9d8d6b50 2024-04-25T08:35:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedca22d740 10 client.0 _ll_register_callbacks cb 0x55df9d7ed180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedc52c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d5510 0x55df9d8d8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedc52c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d5510 0x55df9d8d8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42968/0 (socket says 172.21.15.38:42968) 2024-04-25T08:35:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedc52c5640 1 -- 172.21.15.38:0/888596122 learned_addr learned my addr 172.21.15.38:0/888596122 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8f1470 0x55df9d8d91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedc5ac6640 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8d6b50 0x55df9d8c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedc52c5640 1 -- 172.21.15.38:0/888596122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8d6b50 msgr2=0x55df9d8c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedc52c5640 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8d6b50 0x55df9d8c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedc52c5640 1 -- 172.21.15.38:0/888596122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8f1470 msgr2=0x55df9d8d91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedc52c5640 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8f1470 0x55df9d8d91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.632+0000 7fedc52c5640 1 -- 172.21.15.38:0/888596122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df9d5ed7d0 con 0x55df9d8d5510 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fedc52c5640 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d5510 0x55df9d8d8260 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009310 tx=0x7fedbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1658844cc149fdd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fed9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fed9bfff640 1 -- 172.21.15.38:0/888596122 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc027020 con 0x55df9d8d5510 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fed9bfff640 1 -- 172.21.15.38:0/888596122 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fedbc023450 con 0x55df9d8d5510 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fedca22d740 1 -- 172.21.15.38:0/888596122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedb005be50 con 0x55df9d8d5510 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fed9bfff640 1 -- 172.21.15.38:0/888596122 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedbc031050 con 0x55df9d8d5510 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fedca22d740 10 client.6330 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fedca22d740 1 -- 172.21.15.38:0/888596122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55df9d5233f0 con 0x55df9d8d5510 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fed9bfff640 1 -- 172.21.15.38:0/888596122 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb005be50 con 0x55df9d8d5510 2024-04-25T08:35:07.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fed9bfff640 1 -- 172.21.15.38:0/888596122 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fedbc034020 con 0x55df9d8d5510 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fedca22d740 10 client.6330 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fedca22d740 10 client.6330 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fedca22d740 20 client.6330 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fedca22d740 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55df9d8d4370 0x55df9d8d4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fedca22d740 4 client.6330 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fedca22d740 1 -- 172.21.15.38:0/888596122 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df9d81c2e0 con 0x55df9d8d4370 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.633+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55df9d8d4370 0x55df9d8d4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fed9bfff640 10 client.6330 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55df9d8d4370 0x55df9d8d4790 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf924c675d6671d3 server_cookie=290eb8d566d75ba1 in_seq=0 out_seq=0 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fed9bfff640 1 -- 172.21.15.38:0/888596122 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55df9d81c2e0 con 0x55df9d8d4370 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fed9bfff640 10 client.6330 handle_command_reply: tid=0 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fedca22d740 1 client.6330 shutdown 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fedca22d740 1 -- 172.21.15.38:0/888596122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d5510 msgr2=0x55df9d8d8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fedca22d740 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d5510 0x55df9d8d8260 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009310 tx=0x7fedbc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fedca22d740 1 -- 172.21.15.38:0/888596122 shutdown_connections 2024-04-25T08:35:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fedca22d740 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55df9d8d4370 0x55df9d8d4790 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fedca22d740 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8d6b50 0x55df9d8c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fedca22d740 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8f1470 0x55df9d8d91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fed9bfff640 0 client.6330 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fedca22d740 1 --2- 172.21.15.38:0/888596122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d5510 0x55df9d8d8260 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fedca22d740 1 -- 172.21.15.38:0/888596122 >> 172.21.15.38:0/888596122 conn(0x55df9d8a5770 msgr2=0x55df9d8d1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fedca22d740 1 -- 172.21.15.38:0/888596122 shutdown_connections 2024-04-25T08:35:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.634+0000 7fedca22d740 1 -- 172.21.15.38:0/888596122 wait complete. 2024-04-25T08:35:07.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.635+0000 7fedca22d740 20 client.6330 trim_cache size 0 max 16384 2024-04-25T08:35:07.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.650+0000 7fedca22d740 1 Processor -- start 2024-04-25T08:35:07.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.650+0000 7fedca22d740 1 -- start start 2024-04-25T08:35:07.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.650+0000 7fedca22d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.650+0000 7fedca22d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8d6b50 0x55df9d8ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.650+0000 7fedca22d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8f1470 0x55df9d8f0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.650+0000 7fedca22d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc01fc060 con 0x55df9d8f1470 2024-04-25T08:35:07.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.650+0000 7fedca22d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00cc1f0 con 0x55df9d8d5510 2024-04-25T08:35:07.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.650+0000 7fedca22d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00c97e0 con 0x55df9d8d6b50 2024-04-25T08:35:07.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.650+0000 7fedc5ac6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8f1470 0x55df9d8f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7fedc5ac6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8f1470 0x55df9d8f0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42982/0 (socket says 172.21.15.38:42982) 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7fedc5ac6640 1 -- 172.21.15.38:0/2763898317 learned_addr learned my addr 172.21.15.38:0/2763898317 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/2763898317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8d6b50 0x55df9d8ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7fedc5ac6640 1 -- 172.21.15.38:0/2763898317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8d6b50 msgr2=0x55df9d8ab240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7fedc52c5640 1 --2- 172.21.15.38:0/2763898317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7fedc5ac6640 1 --2- 172.21.15.38:0/2763898317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8d6b50 0x55df9d8ab240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7fedc5ac6640 1 -- 172.21.15.38:0/2763898317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 msgr2=0x55df9d8d0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7fedc5ac6640 1 --2- 172.21.15.38:0/2763898317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7fedc5ac6640 1 -- 172.21.15.38:0/2763898317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df9d4e30f0 con 0x55df9d8f1470 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/2763898317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8d6b50 0x55df9d8ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7fedc52c5640 1 --2- 172.21.15.38:0/2763898317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7fedc5ac6640 1 --2- 172.21.15.38:0/2763898317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8f1470 0x55df9d8f0010 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009f40 tx=0x7fedb802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244a5231058b7db0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7feda27fc640 1 -- 172.21.15.38:0/2763898317 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8036040 con 0x55df9d8f1470 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7feda27fc640 1 -- 172.21.15.38:0/2763898317 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb8037040 con 0x55df9d8f1470 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.651+0000 7feda27fc640 1 -- 172.21.15.38:0/2763898317 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8031a00 con 0x55df9d8f1470 2024-04-25T08:35:07.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.652+0000 7fedca22d740 1 -- 172.21.15.38:0/2763898317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8f1470 msgr2=0x55df9d8f0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.652+0000 7fedca22d740 1 --2- 172.21.15.38:0/2763898317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8f1470 0x55df9d8f0010 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fedb8009f40 tx=0x7fedb802e000 comp rx=0 tx=0).stop 2024-04-25T08:35:07.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.652+0000 7fedca22d740 1 -- 172.21.15.38:0/2763898317 shutdown_connections 2024-04-25T08:35:07.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.652+0000 7fedca22d740 1 --2- 172.21.15.38:0/2763898317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8f1470 0x55df9d8f0010 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.652+0000 7fedca22d740 1 --2- 172.21.15.38:0/2763898317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8d6b50 0x55df9d8ab240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.652+0000 7fedca22d740 1 --2- 172.21.15.38:0/2763898317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.652+0000 7fedca22d740 1 -- 172.21.15.38:0/2763898317 >> 172.21.15.38:0/2763898317 conn(0x55df9d8a5770 msgr2=0x55df9d8f2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.652+0000 7fedca22d740 1 -- 172.21.15.38:0/2763898317 shutdown_connections 2024-04-25T08:35:07.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.652+0000 7fedca22d740 1 -- 172.21.15.38:0/2763898317 wait complete. 2024-04-25T08:35:07.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.652+0000 7fedca22d740 1 -- start start 2024-04-25T08:35:07.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.652+0000 7fedca22d740 1 Processor -- start 2024-04-25T08:35:07.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.653+0000 7fedca22d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.653+0000 7fedca22d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d6b50 0x55df9d947fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.653+0000 7fedca22d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8f1470 0x55df9d8a8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.653+0000 7fedca22d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df9d8d2b10 con 0x55df9d8d6b50 2024-04-25T08:35:07.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.653+0000 7fedca22d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc0071ba0 con 0x55df9d8d5510 2024-04-25T08:35:07.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.653+0000 7fedca22d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55df9d8ac160 con 0x55df9d8f1470 2024-04-25T08:35:07.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.653+0000 7fedca22d740 10 client.0 _ll_register_callbacks cb 0x7fedb8034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:07.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.653+0000 7fedc4ac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d6b50 0x55df9d947fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.653+0000 7fedc4ac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d6b50 0x55df9d947fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42986/0 (socket says 172.21.15.38:42986) 2024-04-25T08:35:07.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.653+0000 7fedc4ac4640 1 -- 172.21.15.38:0/3763724025 learned_addr learned my addr 172.21.15.38:0/3763724025 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:07.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.653+0000 7fedc52c5640 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.653+0000 7fedc5ac6640 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8f1470 0x55df9d8a8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedc4ac4640 1 -- 172.21.15.38:0/3763724025 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8f1470 msgr2=0x55df9d8a8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8f1470 0x55df9d8a8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedc4ac4640 1 -- 172.21.15.38:0/3763724025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 msgr2=0x55df9d8d3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedc4ac4640 1 -- 172.21.15.38:0/3763724025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df9d8a5540 con 0x55df9d8d6b50 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedc4ac4640 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d6b50 0x55df9d947fa0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fedb00014f0 tx=0x7fedb008c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955e381c63f5327f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fed9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fed9bfff640 1 -- 172.21.15.38:0/3763724025 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb001f050 con 0x55df9d8d6b50 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fed9bfff640 1 -- 172.21.15.38:0/3763724025 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fedb0057240 con 0x55df9d8d6b50 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedca22d740 1 -- 172.21.15.38:0/3763724025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedb8037040 con 0x55df9d8d6b50 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fed9bfff640 1 -- 172.21.15.38:0/3763724025 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedb008c0c0 con 0x55df9d8d6b50 2024-04-25T08:35:07.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedca22d740 10 client.6336 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedca22d740 1 -- 172.21.15.38:0/3763724025 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55df9d8d2f10 con 0x55df9d8d6b50 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fed9bfff640 1 -- 172.21.15.38:0/3763724025 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb8037040 con 0x55df9d8d6b50 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fed9bfff640 1 -- 172.21.15.38:0/3763724025 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fedb000f040 con 0x55df9d8d6b50 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedca22d740 10 client.6336 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedca22d740 10 client.6336 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedca22d740 20 client.6336 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedca22d740 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55df9d8a99b0 0x55df9d8a9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedca22d740 4 client.6336 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.654+0000 7fedca22d740 1 -- 172.21.15.38:0/3763724025 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55df9d816630 con 0x55df9d8a99b0 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedc52c5640 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55df9d8a99b0 0x55df9d8a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fed9bfff640 10 client.6336 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedc52c5640 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55df9d8a99b0 0x55df9d8a9230 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e86ead17d2acead server_cookie=e20640ea090a3b9f in_seq=0 out_seq=0 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fed9bfff640 1 -- 172.21.15.38:0/3763724025 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+903 (crc 0 0 0) 0x55df9d816630 con 0x55df9d8a99b0 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fed9bfff640 10 client.6336 handle_command_reply: tid=0 2024-04-25T08:35:07.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedca22d740 1 client.6336 shutdown 2024-04-25T08:35:07.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedca22d740 1 -- 172.21.15.38:0/3763724025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d6b50 msgr2=0x55df9d947fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedca22d740 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d6b50 0x55df9d947fa0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fedb00014f0 tx=0x7fedb008c050 comp rx=0 tx=0).stop 2024-04-25T08:35:07.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedca22d740 1 -- 172.21.15.38:0/3763724025 shutdown_connections 2024-04-25T08:35:07.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedca22d740 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55df9d8a99b0 0x55df9d8a9230 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedca22d740 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55df9d8f1470 0x55df9d8a8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedca22d740 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55df9d8d6b50 0x55df9d947fa0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fed9bfff640 0 client.6336 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:07.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedca22d740 1 --2- 172.21.15.38:0/3763724025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55df9d8d5510 0x55df9d8d3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedca22d740 1 -- 172.21.15.38:0/3763724025 >> 172.21.15.38:0/3763724025 conn(0x55df9d8a5770 msgr2=0x55df9d8aa450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedca22d740 1 -- 172.21.15.38:0/3763724025 shutdown_connections 2024-04-25T08:35:07.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedca22d740 1 -- 172.21.15.38:0/3763724025 wait complete. 2024-04-25T08:35:07.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.655+0000 7fedca22d740 20 client.6336 trim_cache size 0 max 16384 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: "0572eb7f-a143-4b1d-b5ea-39b7d7647297": { 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "0572eb7f-a143-4b1d-b5ea-39b7d7647297", 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "" 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: "303693eb-e4c0-466c-9152-905ca1863bed": { 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites", 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "303693eb-e4c0-466c-9152-905ca1863bed", 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "" 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:07.657 INFO:teuthology.orchestra.run.smithi038.stdout: "5197916b-a7b0-4971-a5d9-35913e715c2a": { 2024-04-25T08:35:07.658 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-04-25T08:35:07.658 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "5197916b-a7b0-4971-a5d9-35913e715c2a", 2024-04-25T08:35:07.658 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "" 2024-04-25T08:35:07.658 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:07.658 INFO:teuthology.orchestra.run.smithi038.stdout: "f83cb3ff-e58e-4574-a66c-e80b3b7d2da7": { 2024-04-25T08:35:07.658 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T08:35:07.658 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "f83cb3ff-e58e-4574-a66c-e80b3b7d2da7", 2024-04-25T08:35:07.658 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "" 2024-04-25T08:35:07.658 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:07.658 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:07.658 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:07.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.656+0000 7fedc7529640 1 -- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feda406b500 msgr2=0x7feda406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.656+0000 7fedc7529640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feda406b500 0x7feda406d9c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009aa0 tx=0x7fedbc009190 comp rx=0 tx=0).stop 2024-04-25T08:35:07.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.656+0000 7fedc7529640 1 -- 172.21.15.38:0/1034477520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedc00cfb70 msgr2=0x7fedc015b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:07.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.656+0000 7fedc7529640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedc00cfb70 0x7fedc015b170 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fedb000e710 tx=0x7fedb000ebd0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.656+0000 7fedc7529640 1 -- 172.21.15.38:0/1034477520 shutdown_connections 2024-04-25T08:35:07.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.656+0000 7fedc7529640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feda406b500 0x7feda406d9c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.656+0000 7fedc7529640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fedc015db80 0x7fedc0155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.656+0000 7fedc7529640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fedc00cfb70 0x7fedc015b170 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.656+0000 7fedc7529640 1 --2- 172.21.15.38:0/1034477520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fedc00ccfc0 0x7fedc015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:07.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.656+0000 7fedc7529640 1 -- 172.21.15.38:0/1034477520 >> 172.21.15.38:0/1034477520 conn(0x7fedc006adc0 msgr2=0x7fedc00cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:07.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.656+0000 7fedc7529640 1 -- 172.21.15.38:0/1034477520 shutdown_connections 2024-04-25T08:35:07.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:07.656+0000 7fedc7529640 1 -- 172.21.15.38:0/1034477520 wait complete. 2024-04-25T08:35:12.662 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:35:12.662 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:35:12.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.783+0000 7f530c25d640 1 Processor -- start 2024-04-25T08:35:12.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.783+0000 7f530c25d640 1 -- start start 2024-04-25T08:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.784+0000 7f530c25d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53040d33d0 0x7f53040d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.784+0000 7f530c25d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f53040d1e90 0x7f53040b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.784+0000 7f530c25d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53040d2330 con 0x7f53040d0320 2024-04-25T08:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.784+0000 7f530c25d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53040d2610 con 0x7f53040d1e90 2024-04-25T08:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.784+0000 7f530c25d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53040d28f0 con 0x7f53040d33d0 2024-04-25T08:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.784+0000 7f530a7fa640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f53040d1e90 0x7f53040b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.784+0000 7f530a7fa640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f53040d1e90 0x7f53040b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43782/0 (socket says 172.21.15.38:43782) 2024-04-25T08:35:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.784+0000 7f530a7fa640 1 -- 172.21.15.38:0/2394155697 learned_addr learned my addr 172.21.15.38:0/2394155697 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.785+0000 7f53097f8640 1 --2- 172.21.15.38:0/2394155697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53040d33d0 0x7f53040d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.785+0000 7f530a7fa640 1 -- 172.21.15.38:0/2394155697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53040d33d0 msgr2=0x7f53040d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.785+0000 7f530a7fa640 1 --2- 172.21.15.38:0/2394155697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53040d33d0 0x7f53040d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.785+0000 7f530a7fa640 1 -- 172.21.15.38:0/2394155697 >> v1:172.21.15.107:6789/0 conn(0x7f53040d0320 legacy=0x7f53040d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.785+0000 7f530a7fa640 1 -- 172.21.15.38:0/2394155697 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53040d2bd0 con 0x7f53040d1e90 2024-04-25T08:35:12.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.785+0000 7f530a7fa640 1 --2- 172.21.15.38:0/2394155697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53040d1e90 0x7f53040b6980 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009e50 tx=0x7f52fc02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4213a0e3a50236a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.785+0000 7f5308ff7640 1 -- 172.21.15.38:0/2394155697 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc036070 con 0x7f53040d1e90 2024-04-25T08:35:12.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.785+0000 7f5308ff7640 1 -- 172.21.15.38:0/2394155697 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc031600 con 0x7f53040d1e90 2024-04-25T08:35:12.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.786+0000 7f5308ff7640 1 -- 172.21.15.38:0/2394155697 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc0319d0 con 0x7f53040d1e90 2024-04-25T08:35:12.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.786+0000 7f530c25d640 1 -- 172.21.15.38:0/2394155697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53040d1e90 msgr2=0x7f53040b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.786+0000 7f530c25d640 1 --2- 172.21.15.38:0/2394155697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53040d1e90 0x7f53040b6980 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009e50 tx=0x7f52fc02a750 comp rx=0 tx=0).stop 2024-04-25T08:35:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.786+0000 7f530c25d640 1 -- 172.21.15.38:0/2394155697 shutdown_connections 2024-04-25T08:35:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.786+0000 7f530c25d640 1 --2- 172.21.15.38:0/2394155697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53040d1e90 0x7f53040b6980 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.786+0000 7f530c25d640 1 --2- 172.21.15.38:0/2394155697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53040d33d0 0x7f53040d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.786+0000 7f530c25d640 1 -- 172.21.15.38:0/2394155697 >> 172.21.15.38:0/2394155697 conn(0x7f53040668f0 msgr2=0x7f5304066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.786+0000 7f530c25d640 1 -- 172.21.15.38:0/2394155697 shutdown_connections 2024-04-25T08:35:12.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.786+0000 7f530c25d640 1 -- 172.21.15.38:0/2394155697 wait complete. 2024-04-25T08:35:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.787+0000 7f530c25d640 1 Processor -- start 2024-04-25T08:35:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.787+0000 7f530c25d640 1 -- start start 2024-04-25T08:35:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.788+0000 7f530c25d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53040d0320 0x7f53041e7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.788+0000 7f530c25d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53040d1e90 0x7f53040b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.788+0000 7f530c25d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53040d33d0 0x7f53040cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.788+0000 7f530c25d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5304161390 con 0x7f53040d0320 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.788+0000 7f530c25d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5304161210 con 0x7f53040d1e90 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.788+0000 7f530c25d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53040b9720 con 0x7f53040d33d0 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.788+0000 7f5309ff9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53040d0320 0x7f53041e7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.788+0000 7f5309ff9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53040d0320 0x7f53041e7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43790/0 (socket says 172.21.15.38:43790) 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.788+0000 7f5309ff9640 1 -- 172.21.15.38:0/2755495146 learned_addr learned my addr 172.21.15.38:0/2755495146 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.788+0000 7f53097f8640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53040d1e90 0x7f53040b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.788+0000 7f530a7fa640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53040d33d0 0x7f53040cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.789+0000 7f5309ff9640 1 -- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53040d33d0 msgr2=0x7f53040cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.789+0000 7f5309ff9640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53040d33d0 0x7f53040cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.789+0000 7f5309ff9640 1 -- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53040d1e90 msgr2=0x7f53040b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.789+0000 7f5309ff9640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53040d1e90 0x7f53040b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.789+0000 7f5309ff9640 1 -- 172.21.15.38:0/2755495146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53041f46f0 con 0x7f53040d0320 2024-04-25T08:35:12.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.789+0000 7f53097f8640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53040d1e90 0x7f53040b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.789+0000 7f5309ff9640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53040d0320 0x7f53041e7960 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f530000c430 tx=0x7f530000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.789+0000 7f52fa7fc640 1 -- 172.21.15.38:0/2755495146 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300011790 con 0x7f53040d0320 2024-04-25T08:35:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.789+0000 7f52fa7fc640 1 -- 172.21.15.38:0/2755495146 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5300017070 con 0x7f53040d0320 2024-04-25T08:35:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.789+0000 7f530c25d640 1 -- 172.21.15.38:0/2755495146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53041f3b70 con 0x7f53040d0320 2024-04-25T08:35:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.789+0000 7f52fa7fc640 1 -- 172.21.15.38:0/2755495146 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300011bb0 con 0x7f53040d0320 2024-04-25T08:35:12.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.789+0000 7f530c25d640 1 -- 172.21.15.38:0/2755495146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53041f4410 con 0x7f53040d0320 2024-04-25T08:35:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.791+0000 7f52fa7fc640 1 -- 172.21.15.38:0/2755495146 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f530001c460 con 0x7f53040d0320 2024-04-25T08:35:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.791+0000 7f52fa7fc640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52e806b4d0 0x7f52e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.791+0000 7f52fa7fc640 1 -- 172.21.15.38:0/2755495146 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f530008a1e0 con 0x7f53040d0320 2024-04-25T08:35:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.791+0000 7f53097f8640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52e806b4d0 0x7f52e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.792+0000 7f53097f8640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52e806b4d0 0x7f52e806d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f52f4006fd0 tx=0x7f52f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.792+0000 7f530ef64740 1 Processor -- start 2024-04-25T08:35:12.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.792+0000 7f530ef64740 1 -- start start 2024-04-25T08:35:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f530ef64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acb10 0x55f2570af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f530ef64740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acef0 0x55f2570af860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f530ef64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f25707f770 0x55f257082f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f530ef64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f256fe8da0 con 0x55f2570acb10 2024-04-25T08:35:12.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f530ef64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f256f91b80 con 0x55f25707f770 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f530ef64740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f256fe4e40 con 0x55f2570acef0 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f5309ff9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acb10 0x55f2570af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f5309ff9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acb10 0x55f2570af320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43794/0 (socket says 172.21.15.38:43794) 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f5309ff9640 1 -- 172.21.15.38:0/2499716432 learned_addr learned my addr 172.21.15.38:0/2499716432 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f53097f8640 1 --2- 172.21.15.38:0/2499716432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acef0 0x55f2570af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f530a7fa640 1 --2- 172.21.15.38:0/2499716432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f25707f770 0x55f257082f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f5309ff9640 1 -- 172.21.15.38:0/2499716432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acef0 msgr2=0x55f2570af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f5309ff9640 1 --2- 172.21.15.38:0/2499716432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acef0 0x55f2570af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f5309ff9640 1 -- 172.21.15.38:0/2499716432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f25707f770 msgr2=0x55f257082f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f5309ff9640 1 --2- 172.21.15.38:0/2499716432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f25707f770 0x55f257082f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.793+0000 7f5309ff9640 1 -- 172.21.15.38:0/2499716432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f256f9bf10 con 0x55f2570acb10 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.794+0000 7f5309ff9640 1 --2- 172.21.15.38:0/2499716432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acb10 0x55f2570af320 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f530000a040 tx=0x7f5300059020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfaf155bc08db687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.794+0000 7f52e77fe640 1 -- 172.21.15.38:0/2499716432 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300070070 con 0x55f2570acb10 2024-04-25T08:35:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.794+0000 7f52e77fe640 1 -- 172.21.15.38:0/2499716432 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5300060f70 con 0x55f2570acb10 2024-04-25T08:35:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.794+0000 7f52e77fe640 1 -- 172.21.15.38:0/2499716432 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5300011d50 con 0x55f2570acb10 2024-04-25T08:35:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.794+0000 7f530ef64740 1 -- 172.21.15.38:0/2499716432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acb10 msgr2=0x55f2570af320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.794+0000 7f530ef64740 1 --2- 172.21.15.38:0/2499716432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acb10 0x55f2570af320 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f530000a040 tx=0x7f5300059020 comp rx=0 tx=0).stop 2024-04-25T08:35:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.794+0000 7f530ef64740 1 -- 172.21.15.38:0/2499716432 shutdown_connections 2024-04-25T08:35:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.795+0000 7f530ef64740 1 --2- 172.21.15.38:0/2499716432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f25707f770 0x55f257082f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.795+0000 7f530ef64740 1 --2- 172.21.15.38:0/2499716432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acef0 0x55f2570af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.795+0000 7f530ef64740 1 --2- 172.21.15.38:0/2499716432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acb10 0x55f2570af320 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.795+0000 7f530ef64740 1 -- 172.21.15.38:0/2499716432 >> 172.21.15.38:0/2499716432 conn(0x55f2570819e0 msgr2=0x55f2570ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.795+0000 7f530ef64740 1 -- 172.21.15.38:0/2499716432 shutdown_connections 2024-04-25T08:35:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.795+0000 7f530ef64740 1 -- 172.21.15.38:0/2499716432 wait complete. 2024-04-25T08:35:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.795+0000 7f530ef64740 1 -- start start 2024-04-25T08:35:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.795+0000 7f530ef64740 1 Processor -- start 2024-04-25T08:35:12.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.796+0000 7f530ef64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570acb10 0x55f2570b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.796+0000 7f530ef64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 0x55f2570cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.796+0000 7f530ef64740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570b01f0 0x55f2570d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.796+0000 7f530ef64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f257084b90 con 0x55f2570acef0 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.796+0000 7f530ef64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f256f874b0 con 0x55f2570acb10 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.796+0000 7f530ef64740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f257084e70 con 0x55f2570b01f0 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.796+0000 7f530ef64740 10 client.0 _ll_register_callbacks cb 0x55f256fc7180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.796+0000 7f53097f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 0x55f2570cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.796+0000 7f53097f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 0x55f2570cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43804/0 (socket says 172.21.15.38:43804) 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.796+0000 7f53097f8640 1 -- 172.21.15.38:0/1521152042 learned_addr learned my addr 172.21.15.38:0/1521152042 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.796+0000 7f5309ff9640 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570acb10 0x55f2570b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f530a7fa640 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570b01f0 0x55f2570d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f53097f8640 1 -- 172.21.15.38:0/1521152042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570b01f0 msgr2=0x55f2570d24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f53097f8640 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570b01f0 0x55f2570d24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f53097f8640 1 -- 172.21.15.38:0/1521152042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570acb10 msgr2=0x55f2570b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f53097f8640 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570acb10 0x55f2570b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f53097f8640 1 -- 172.21.15.38:0/1521152042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f256dc77d0 con 0x55f2570acef0 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f530a7fa640 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570b01f0 0x55f2570d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f53097f8640 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 0x55f2570cd2a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f52f40136d0 tx=0x7f52f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7710d486079fc31c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f52e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f52e4ff9640 1 -- 172.21.15.38:0/1521152042 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4024020 con 0x55f2570acef0 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f530ef64740 1 -- 172.21.15.38:0/1521152042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5300060f70 con 0x55f2570acef0 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f52e4ff9640 1 -- 172.21.15.38:0/1521152042 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f52f40147d0 con 0x55f2570acef0 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f52e4ff9640 1 -- 172.21.15.38:0/1521152042 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52f402e050 con 0x55f2570acef0 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f530ef64740 10 client.6348 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f530ef64740 1 -- 172.21.15.38:0/1521152042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55f256cfd3f0 con 0x55f2570acef0 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.797+0000 7f52e4ff9640 1 -- 172.21.15.38:0/1521152042 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5300060f70 con 0x55f2570acef0 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.798+0000 7f52e4ff9640 1 -- 172.21.15.38:0/1521152042 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f52f4031020 con 0x55f2570acef0 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.798+0000 7f530ef64740 10 client.6348 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.798+0000 7f530ef64740 10 client.6348 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:12.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.798+0000 7f530ef64740 20 client.6348 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.798+0000 7f530ef64740 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55f2570af380 0x55f2570cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.798+0000 7f530ef64740 4 client.6348 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.798+0000 7f530ef64740 1 -- 172.21.15.38:0/1521152042 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f256ff62e0 con 0x55f2570af380 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.798+0000 7f5309ff9640 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55f2570af380 0x55f2570cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.798+0000 7f5309ff9640 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55f2570af380 0x55f2570cb450 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2195a18f47365b server_cookie=70eb7bdda475e299 in_seq=0 out_seq=0 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.798+0000 7f52e4ff9640 10 client.6348 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.798+0000 7f52e4ff9640 1 -- 172.21.15.38:0/1521152042 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f256ff62e0 con 0x55f2570af380 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.798+0000 7f52e4ff9640 10 client.6348 handle_command_reply: tid=0 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f530ef64740 1 client.6348 shutdown 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f530ef64740 1 -- 172.21.15.38:0/1521152042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 msgr2=0x55f2570cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f530ef64740 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 0x55f2570cd2a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f52f40136d0 tx=0x7f52f4004730 comp rx=0 tx=0).stop 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f530ef64740 1 -- 172.21.15.38:0/1521152042 shutdown_connections 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f530ef64740 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55f2570af380 0x55f2570cb450 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f530ef64740 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570b01f0 0x55f2570d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f530ef64740 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 0x55f2570cd2a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f530ef64740 1 --2- 172.21.15.38:0/1521152042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570acb10 0x55f2570b1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f52e4ff9640 0 client.6348 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:12.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f530ef64740 1 -- 172.21.15.38:0/1521152042 >> 172.21.15.38:0/1521152042 conn(0x55f25707f770 msgr2=0x55f2570aa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:12.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f530ef64740 1 -- 172.21.15.38:0/1521152042 shutdown_connections 2024-04-25T08:35:12.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f530ef64740 1 -- 172.21.15.38:0/1521152042 wait complete. 2024-04-25T08:35:12.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.799+0000 7f530ef64740 20 client.6348 trim_cache size 0 max 16384 2024-04-25T08:35:12.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.814+0000 7f530ef64740 1 Processor -- start 2024-04-25T08:35:12.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.814+0000 7f530ef64740 1 -- start start 2024-04-25T08:35:12.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.815+0000 7f530ef64740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acb10 0x55f2570ed0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.815+0000 7f530ef64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570acef0 0x55f257085810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.815+0000 7f530ef64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570b01f0 0x55f2570ae700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.815+0000 7f530ef64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53042003a0 con 0x55f2570b01f0 2024-04-25T08:35:12.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.815+0000 7f530ef64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53041ff520 con 0x55f2570acef0 2024-04-25T08:35:12.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.815+0000 7f530ef64740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53040b3b00 con 0x55f2570acb10 2024-04-25T08:35:12.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.815+0000 7f530a7fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570b01f0 0x55f2570ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.815+0000 7f530a7fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570b01f0 0x55f2570ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43806/0 (socket says 172.21.15.38:43806) 2024-04-25T08:35:12.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.815+0000 7f530a7fa640 1 -- 172.21.15.38:0/233456059 learned_addr learned my addr 172.21.15.38:0/233456059 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:12.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.815+0000 7f53097f8640 1 --2- 172.21.15.38:0/233456059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570acef0 0x55f257085810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.815+0000 7f5309ff9640 1 --2- 172.21.15.38:0/233456059 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acb10 0x55f2570ed0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.816+0000 7f530a7fa640 1 -- 172.21.15.38:0/233456059 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acb10 msgr2=0x55f2570ed0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.816+0000 7f530a7fa640 1 --2- 172.21.15.38:0/233456059 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acb10 0x55f2570ed0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.816+0000 7f530a7fa640 1 -- 172.21.15.38:0/233456059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570acef0 msgr2=0x55f257085810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.816+0000 7f530a7fa640 1 --2- 172.21.15.38:0/233456059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570acef0 0x55f257085810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.816+0000 7f530a7fa640 1 -- 172.21.15.38:0/233456059 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f256cbd0f0 con 0x55f2570b01f0 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.816+0000 7f530a7fa640 1 --2- 172.21.15.38:0/233456059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570b01f0 0x55f2570ae700 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009f70 tx=0x7f52fc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f348a18e7dae6a60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.816+0000 7f52e6ffd640 1 -- 172.21.15.38:0/233456059 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc036040 con 0x55f2570b01f0 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.816+0000 7f52e6ffd640 1 -- 172.21.15.38:0/233456059 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc0317d0 con 0x55f2570b01f0 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.816+0000 7f52e6ffd640 1 -- 172.21.15.38:0/233456059 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc031bc0 con 0x55f2570b01f0 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.816+0000 7f530ef64740 1 -- 172.21.15.38:0/233456059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570b01f0 msgr2=0x55f2570ae700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.816+0000 7f530ef64740 1 --2- 172.21.15.38:0/233456059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570b01f0 0x55f2570ae700 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f52fc009f70 tx=0x7f52fc02e000 comp rx=0 tx=0).stop 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.817+0000 7f530ef64740 1 -- 172.21.15.38:0/233456059 shutdown_connections 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.817+0000 7f530ef64740 1 --2- 172.21.15.38:0/233456059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570b01f0 0x55f2570ae700 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.817+0000 7f530ef64740 1 --2- 172.21.15.38:0/233456059 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570acef0 0x55f257085810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.817+0000 7f530ef64740 1 --2- 172.21.15.38:0/233456059 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acb10 0x55f2570ed0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.817+0000 7f530ef64740 1 -- 172.21.15.38:0/233456059 >> 172.21.15.38:0/233456059 conn(0x55f25707f770 msgr2=0x55f2570c6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:12.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.817+0000 7f530ef64740 1 -- 172.21.15.38:0/233456059 shutdown_connections 2024-04-25T08:35:12.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.817+0000 7f530ef64740 1 -- 172.21.15.38:0/233456059 wait complete. 2024-04-25T08:35:12.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.817+0000 7f530ef64740 1 -- start start 2024-04-25T08:35:12.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.817+0000 7f530ef64740 1 Processor -- start 2024-04-25T08:35:12.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f530ef64740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acb10 0x55f2570aa590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f530ef64740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 0x55f2570aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f530ef64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570b01f0 0x55f2570cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f530ef64740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2570dd970 con 0x55f2570acef0 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f530ef64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53040b4100 con 0x55f2570b01f0 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f530ef64740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2570d2240 con 0x55f2570acb10 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f530ef64740 10 client.0 _ll_register_callbacks cb 0x7f5300058630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f53097f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 0x55f2570aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f53097f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 0x55f2570aaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43808/0 (socket says 172.21.15.38:43808) 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f53097f8640 1 -- 172.21.15.38:0/2411676683 learned_addr learned my addr 172.21.15.38:0/2411676683 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f530a7fa640 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570b01f0 0x55f2570cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f5309ff9640 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acb10 0x55f2570aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f53097f8640 1 -- 172.21.15.38:0/2411676683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acb10 msgr2=0x55f2570aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f53097f8640 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acb10 0x55f2570aa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f53097f8640 1 -- 172.21.15.38:0/2411676683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570b01f0 msgr2=0x55f2570cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f53097f8640 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570b01f0 0x55f2570cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.818+0000 7f53097f8640 1 -- 172.21.15.38:0/2411676683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2570ca320 con 0x55f2570acef0 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f5309ff9640 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acb10 0x55f2570aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f53097f8640 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 0x55f2570aaad0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f52f4006b10 tx=0x7f52f4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29dfb3ee93bdbcb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f52e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f52e4ff9640 1 -- 172.21.15.38:0/2411676683 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4024040 con 0x55f2570acef0 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f52e4ff9640 1 -- 172.21.15.38:0/2411676683 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f52f4005da0 con 0x55f2570acef0 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f530ef64740 1 -- 172.21.15.38:0/2411676683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52fc0317d0 con 0x55f2570acef0 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f52e4ff9640 1 -- 172.21.15.38:0/2411676683 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52f403f050 con 0x55f2570acef0 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f530ef64740 10 client.6354 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f530ef64740 1 -- 172.21.15.38:0/2411676683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55f2570c0240 con 0x55f2570acef0 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f52e4ff9640 1 -- 172.21.15.38:0/2411676683 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc0317d0 con 0x55f2570acef0 2024-04-25T08:35:12.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f52e4ff9640 1 -- 172.21.15.38:0/2411676683 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f52f4042020 con 0x55f2570acef0 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f530ef64740 10 client.6354 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f530ef64740 10 client.6354 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f530ef64740 20 client.6354 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f530ef64740 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55f2570dc740 0x55f2570cba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f530ef64740 4 client.6354 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f530ef64740 1 -- 172.21.15.38:0/2411676683 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f256ff0630 con 0x55f2570dc740 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.819+0000 7f5309ff9640 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55f2570dc740 0x55f2570cba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f52e4ff9640 10 client.6354 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f5309ff9640 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55f2570dc740 0x55f2570cba20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663d94fb110b32ac server_cookie=114926eeee11e178 in_seq=0 out_seq=0 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f52e4ff9640 1 -- 172.21.15.38:0/2411676683 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f256ff0630 con 0x55f2570dc740 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f52e4ff9640 10 client.6354 handle_command_reply: tid=0 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f530ef64740 1 client.6354 shutdown 2024-04-25T08:35:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f530ef64740 1 -- 172.21.15.38:0/2411676683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 msgr2=0x55f2570aaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f530ef64740 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 0x55f2570aaad0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f52f4006b10 tx=0x7f52f4015000 comp rx=0 tx=0).stop 2024-04-25T08:35:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f530ef64740 1 -- 172.21.15.38:0/2411676683 shutdown_connections 2024-04-25T08:35:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f530ef64740 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55f2570dc740 0x55f2570cba20 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f530ef64740 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2570b01f0 0x55f2570cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f530ef64740 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2570acef0 0x55f2570aaad0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f52e4ff9640 0 client.6354 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f530ef64740 1 --2- 172.21.15.38:0/2411676683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2570acb10 0x55f2570aa590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f530ef64740 1 -- 172.21.15.38:0/2411676683 >> 172.21.15.38:0/2411676683 conn(0x55f25707f770 msgr2=0x55f2570857a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f530ef64740 1 -- 172.21.15.38:0/2411676683 shutdown_connections 2024-04-25T08:35:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f530ef64740 1 -- 172.21.15.38:0/2411676683 wait complete. 2024-04-25T08:35:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.820+0000 7f530ef64740 20 client.6354 trim_cache size 0 max 16384 2024-04-25T08:35:12.823 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:12.823 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-04-25T08:35:12.823 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-04-25T08:35:12.823 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.821+0000 7f530c25d640 1 -- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52e806b4d0 msgr2=0x7f52e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.821+0000 7f530c25d640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52e806b4d0 0x7f52e806d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f52f4006fd0 tx=0x7f52f4008040 comp rx=0 tx=0).stop 2024-04-25T08:35:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.821+0000 7f530c25d640 1 -- 172.21.15.38:0/2755495146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53040d0320 msgr2=0x7f53041e7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.821+0000 7f530c25d640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53040d0320 0x7f53041e7960 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f530000c430 tx=0x7f530000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.821+0000 7f530c25d640 1 -- 172.21.15.38:0/2755495146 shutdown_connections 2024-04-25T08:35:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.821+0000 7f530c25d640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52e806b4d0 0x7f52e806d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.821+0000 7f530c25d640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53040d33d0 0x7f53040cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.821+0000 7f530c25d640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53040d1e90 0x7f53040b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.821+0000 7f530c25d640 1 --2- 172.21.15.38:0/2755495146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53040d0320 0x7f53041e7960 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.821+0000 7f530c25d640 1 -- 172.21.15.38:0/2755495146 >> 172.21.15.38:0/2755495146 conn(0x7f53040668f0 msgr2=0x7f5304160350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.821+0000 7f530c25d640 1 -- 172.21.15.38:0/2755495146 shutdown_connections 2024-04-25T08:35:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.821+0000 7f530c25d640 1 -- 172.21.15.38:0/2755495146 wait complete. 2024-04-25T08:35:12.826 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:35:12.826 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:12.827 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T08:35:12.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.935+0000 7fda08963640 1 Processor -- start 2024-04-25T08:35:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.935+0000 7fda08963640 1 -- start start 2024-04-25T08:35:12.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda08963640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fda0415b830 0x7fda040cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda08963640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda040727e0 0x7fda040cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda08963640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fda0406bbf0 0x7fda0406c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda08963640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda0406c5a0 con 0x7fda0415b830 2024-04-25T08:35:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda08963640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda040ce1b0 con 0x7fda0406bbf0 2024-04-25T08:35:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda08963640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda040ce490 con 0x7fda040727e0 2024-04-25T08:35:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda0259c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fda0415b830 0x7fda040cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda0259c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fda0415b830 0x7fda040cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43820/0 (socket says 172.21.15.38:43820) 2024-04-25T08:35:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda0259c640 1 -- 172.21.15.38:0/58062539 learned_addr learned my addr 172.21.15.38:0/58062539 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda02d9d640 1 --2- 172.21.15.38:0/58062539 >> v2:172.21.15.107:3300/0 conn(0x7fda0406bbf0 0x7fda0406c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda01d9b640 1 --2- 172.21.15.38:0/58062539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda040727e0 0x7fda040cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda0259c640 1 -- 172.21.15.38:0/58062539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda040727e0 msgr2=0x7fda040cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda0259c640 1 --2- 172.21.15.38:0/58062539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda040727e0 0x7fda040cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda0259c640 1 -- 172.21.15.38:0/58062539 >> v2:172.21.15.107:3300/0 conn(0x7fda0406bbf0 msgr2=0x7fda0406c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda0259c640 1 --2- 172.21.15.38:0/58062539 >> v2:172.21.15.107:3300/0 conn(0x7fda0406bbf0 0x7fda0406c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.936+0000 7fda0259c640 1 -- 172.21.15.38:0/58062539 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040ce770 con 0x7fda0415b830 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.937+0000 7fda0259c640 1 --2- 172.21.15.38:0/58062539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda0415b830 0x7fda040cf250 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fda04070e00 tx=0x7fd9f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b532fa0d7803a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.937+0000 7fda0159a640 1 -- 172.21.15.38:0/58062539 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4037070 con 0x7fda0415b830 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.937+0000 7fda0159a640 1 -- 172.21.15.38:0/58062539 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f4032440 con 0x7fda0415b830 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.937+0000 7fda0159a640 1 -- 172.21.15.38:0/58062539 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4032810 con 0x7fda0415b830 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.937+0000 7fda08963640 1 -- 172.21.15.38:0/58062539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda0415b830 msgr2=0x7fda040cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.937+0000 7fda08963640 1 --2- 172.21.15.38:0/58062539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda0415b830 0x7fda040cf250 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fda04070e00 tx=0x7fd9f402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.938+0000 7fda08963640 1 -- 172.21.15.38:0/58062539 shutdown_connections 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.938+0000 7fda08963640 1 --2- 172.21.15.38:0/58062539 >> v2:172.21.15.107:3300/0 conn(0x7fda0406bbf0 0x7fda0406c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.938+0000 7fda08963640 1 --2- 172.21.15.38:0/58062539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda040727e0 0x7fda040cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.938+0000 7fda08963640 1 --2- 172.21.15.38:0/58062539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda0415b830 0x7fda040cf250 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.938+0000 7fda08963640 1 -- 172.21.15.38:0/58062539 >> 172.21.15.38:0/58062539 conn(0x7fda040717f0 msgr2=0x7fda04071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.938+0000 7fda08963640 1 -- 172.21.15.38:0/58062539 shutdown_connections 2024-04-25T08:35:12.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.938+0000 7fda08963640 1 -- 172.21.15.38:0/58062539 wait complete. 2024-04-25T08:35:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.939+0000 7fda08963640 1 Processor -- start 2024-04-25T08:35:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.939+0000 7fda08963640 1 -- start start 2024-04-25T08:35:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda08963640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda0406bbf0 0x7fda040c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda08963640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda040727e0 0x7fda040c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda08963640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda0415b830 0x7fda0415d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda08963640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda040cde10 con 0x7fda040727e0 2024-04-25T08:35:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda08963640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda040ced30 con 0x7fda0406bbf0 2024-04-25T08:35:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda08963640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda040cdf90 con 0x7fda0415b830 2024-04-25T08:35:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda01d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda040727e0 0x7fda040c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda01d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda040727e0 0x7fda040c59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43828/0 (socket says 172.21.15.38:43828) 2024-04-25T08:35:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda01d9b640 1 -- 172.21.15.38:0/4198168938 learned_addr learned my addr 172.21.15.38:0/4198168938 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda0259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda0406bbf0 0x7fda040c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda02d9d640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda0415b830 0x7fda0415d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda01d9b640 1 -- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda0415b830 msgr2=0x7fda0415d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda01d9b640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda0415b830 0x7fda0415d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda01d9b640 1 -- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda0406bbf0 msgr2=0x7fda040c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda01d9b640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda0406bbf0 0x7fda040c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda01d9b640 1 -- 172.21.15.38:0/4198168938 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda040c5ee0 con 0x7fda040727e0 2024-04-25T08:35:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda0259c640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda0406bbf0 0x7fda040c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.940+0000 7fda01d9b640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda040727e0 0x7fda040c59a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800d830 tx=0x7fd9f800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.941+0000 7fd9f2ffd640 1 -- 172.21.15.38:0/4198168938 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f800b6c0 con 0x7fda040727e0 2024-04-25T08:35:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.941+0000 7fda08963640 1 -- 172.21.15.38:0/4198168938 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda040c9c00 con 0x7fda040727e0 2024-04-25T08:35:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.941+0000 7fd9f2ffd640 1 -- 172.21.15.38:0/4198168938 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f8019070 con 0x7fda040727e0 2024-04-25T08:35:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.941+0000 7fd9f2ffd640 1 -- 172.21.15.38:0/4198168938 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f800ba80 con 0x7fda040727e0 2024-04-25T08:35:12.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.941+0000 7fda08963640 1 -- 172.21.15.38:0/4198168938 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda040c98b0 con 0x7fda040727e0 2024-04-25T08:35:12.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.943+0000 7fd9f2ffd640 1 -- 172.21.15.38:0/4198168938 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9f801e460 con 0x7fda040727e0 2024-04-25T08:35:12.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.943+0000 7fd9f2ffd640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd9e406b4d0 0x7fd9e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.943+0000 7fd9f2ffd640 1 -- 172.21.15.38:0/4198168938 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd9f808a860 con 0x7fda040727e0 2024-04-25T08:35:12.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.943+0000 7fda0259c640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd9e406b4d0 0x7fd9e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.944+0000 7fda0b66e740 1 Processor -- start 2024-04-25T08:35:12.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.944+0000 7fda0259c640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd9e406b4d0 0x7fd9e406d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4034000 tx=0x7fd9f402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.944+0000 7fda0b66e740 1 -- start start 2024-04-25T08:35:12.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda0b66e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 0x55b0639bf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda0b66e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639bce80 0x55b0639bf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda0b66e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b06398f700 0x55b063992f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda0b66e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0638f8e00 con 0x55b06398f700 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda0b66e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0638a1d20 con 0x55b0639bcaa0 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda0b66e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0638f4ea0 con 0x55b0639bce80 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda02d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b06398f700 0x55b063992f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda02d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b06398f700 0x55b063992f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43836/0 (socket says 172.21.15.38:43836) 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda02d9d640 1 -- 172.21.15.38:0/1001303159 learned_addr learned my addr 172.21.15.38:0/1001303159 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda01d9b640 1 --2- 172.21.15.38:0/1001303159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639bce80 0x55b0639bf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda0259c640 1 --2- 172.21.15.38:0/1001303159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 0x55b0639bf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda02d9d640 1 -- 172.21.15.38:0/1001303159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639bce80 msgr2=0x55b0639bf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda02d9d640 1 --2- 172.21.15.38:0/1001303159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639bce80 0x55b0639bf7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda02d9d640 1 -- 172.21.15.38:0/1001303159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 msgr2=0x55b0639bf2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda02d9d640 1 --2- 172.21.15.38:0/1001303159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 0x55b0639bf2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda02d9d640 1 -- 172.21.15.38:0/1001303159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0638c3df0 con 0x55b06398f700 2024-04-25T08:35:12.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fda02d9d640 1 --2- 172.21.15.38:0/1001303159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b06398f700 0x55b063992f10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc011550 tx=0x7fd9fc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fc25aace2bc7c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fd9e3fff640 1 -- 172.21.15.38:0/1001303159 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc015840 con 0x55b06398f700 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fd9e3fff640 1 -- 172.21.15.38:0/1001303159 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc0159e0 con 0x55b06398f700 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.945+0000 7fd9e3fff640 1 -- 172.21.15.38:0/1001303159 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc015d90 con 0x55b06398f700 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.946+0000 7fda0b66e740 1 -- 172.21.15.38:0/1001303159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b06398f700 msgr2=0x55b063992f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.946+0000 7fda0b66e740 1 --2- 172.21.15.38:0/1001303159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b06398f700 0x55b063992f10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc011550 tx=0x7fd9fc009750 comp rx=0 tx=0).stop 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.946+0000 7fda0b66e740 1 -- 172.21.15.38:0/1001303159 shutdown_connections 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.946+0000 7fda0b66e740 1 --2- 172.21.15.38:0/1001303159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b06398f700 0x55b063992f10 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.946+0000 7fda0b66e740 1 --2- 172.21.15.38:0/1001303159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639bce80 0x55b0639bf7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.946+0000 7fda0b66e740 1 --2- 172.21.15.38:0/1001303159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 0x55b0639bf2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.946+0000 7fda0b66e740 1 -- 172.21.15.38:0/1001303159 >> 172.21.15.38:0/1001303159 conn(0x55b063991970 msgr2=0x55b0639da2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.946+0000 7fda0b66e740 1 -- 172.21.15.38:0/1001303159 shutdown_connections 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.946+0000 7fda0b66e740 1 -- 172.21.15.38:0/1001303159 wait complete. 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.947+0000 7fda0b66e740 1 -- start start 2024-04-25T08:35:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.947+0000 7fda0b66e740 1 Processor -- start 2024-04-25T08:35:12.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda0b66e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 0x55b0639b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda0b66e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 0x55b0639ea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda0b66e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639eab90 0x55b063993f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda0b66e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b063994b20 con 0x55b0639bce80 2024-04-25T08:35:12.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda0b66e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b063897590 con 0x55b0639bcaa0 2024-04-25T08:35:12.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda0b66e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b063994e00 con 0x55b0639eab90 2024-04-25T08:35:12.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda0b66e740 10 client.0 _ll_register_callbacks cb 0x55b0638a3e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda01d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 0x55b0639ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda01d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 0x55b0639ea650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43850/0 (socket says 172.21.15.38:43850) 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda01d9b640 1 -- 172.21.15.38:0/3253537299 learned_addr learned my addr 172.21.15.38:0/3253537299 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda0259c640 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 0x55b0639b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda02d9d640 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639eab90 0x55b063993f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda01d9b640 1 -- 172.21.15.38:0/3253537299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639eab90 msgr2=0x55b063993f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda01d9b640 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639eab90 0x55b063993f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda01d9b640 1 -- 172.21.15.38:0/3253537299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 msgr2=0x55b0639b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda01d9b640 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 0x55b0639b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda01d9b640 1 -- 172.21.15.38:0/3253537299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b06360d3f0 con 0x55b0639bce80 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fda01d9b640 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 0x55b0639ea650 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8088c90 tx=0x7fd9f8058f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6810cc052d433b6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.948+0000 7fd9e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fd9e17fa640 1 -- 172.21.15.38:0/3253537299 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8075060 con 0x55b0639bce80 2024-04-25T08:35:12.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fd9e17fa640 1 -- 172.21.15.38:0/3253537299 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd9f80610e0 con 0x55b0639bce80 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fda0b66e740 1 -- 172.21.15.38:0/3253537299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9fc0159e0 con 0x55b0639bce80 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fd9e17fa640 1 -- 172.21.15.38:0/3253537299 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9f8070070 con 0x55b0639bce80 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fda0b66e740 10 client.6366 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fda0b66e740 1 -- 172.21.15.38:0/3253537299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55b0636d77d0 con 0x55b0639bce80 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fd9e17fa640 1 -- 172.21.15.38:0/3253537299 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9fc0159e0 con 0x55b0639bce80 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fd9e17fa640 1 -- 172.21.15.38:0/3253537299 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd9f8013070 con 0x55b0639bce80 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fda0b66e740 10 client.6366 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fda0b66e740 10 client.6366 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fda0b66e740 20 client.6366 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fda0b66e740 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55b0639db8b0 0x55b0639dbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fda0b66e740 4 client.6366 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fda0b66e740 1 -- 172.21.15.38:0/3253537299 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b063906340 con 0x55b0639db8b0 2024-04-25T08:35:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fda0259c640 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55b0639db8b0 0x55b0639dbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.949+0000 7fd9e17fa640 10 client.6366 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fda0259c640 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55b0639db8b0 0x55b0639dbcd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8e6272d18c0e7c server_cookie=bf6f161eae711384 in_seq=0 out_seq=0 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fd9e17fa640 1 -- 172.21.15.38:0/3253537299 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b063906340 con 0x55b0639db8b0 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fd9e17fa640 10 client.6366 handle_command_reply: tid=0 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fda0b66e740 1 client.6366 shutdown 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fda0b66e740 1 -- 172.21.15.38:0/3253537299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 msgr2=0x55b0639ea650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fda0b66e740 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 0x55b0639ea650 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8088c90 tx=0x7fd9f8058f10 comp rx=0 tx=0).stop 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fda0b66e740 1 -- 172.21.15.38:0/3253537299 shutdown_connections 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fda0b66e740 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55b0639db8b0 0x55b0639dbcd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fda0b66e740 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639eab90 0x55b063993f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fda0b66e740 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 0x55b0639ea650 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fda0b66e740 1 --2- 172.21.15.38:0/3253537299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 0x55b0639b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fd9e17fa640 0 client.6366 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:12.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fda0b66e740 1 -- 172.21.15.38:0/3253537299 >> 172.21.15.38:0/3253537299 conn(0x55b06398f700 msgr2=0x55b0639bbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:12.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fda0b66e740 1 -- 172.21.15.38:0/3253537299 shutdown_connections 2024-04-25T08:35:12.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.950+0000 7fda0b66e740 1 -- 172.21.15.38:0/3253537299 wait complete. 2024-04-25T08:35:12.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.951+0000 7fda0b66e740 20 client.6366 trim_cache size 0 max 16384 2024-04-25T08:35:12.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.962+0000 7fda0b66e740 1 Processor -- start 2024-04-25T08:35:12.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.962+0000 7fda0b66e740 1 -- start start 2024-04-25T08:35:12.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0b66e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bcaa0 0x55b0639bf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0b66e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639bce80 0x55b0639bf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0b66e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639eab90 0x55b063990110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0b66e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b06398ff50 con 0x55b0639bcaa0 2024-04-25T08:35:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0b66e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda041fe140 con 0x55b0639eab90 2024-04-25T08:35:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0b66e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda041fd320 con 0x55b0639bce80 2024-04-25T08:35:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bcaa0 0x55b0639bf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bcaa0 0x55b0639bf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43860/0 (socket says 172.21.15.38:43860) 2024-04-25T08:35:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0259c640 1 -- 172.21.15.38:0/1600393258 learned_addr learned my addr 172.21.15.38:0/1600393258 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda01d9b640 1 --2- 172.21.15.38:0/1600393258 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639bce80 0x55b0639bf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0259c640 1 -- 172.21.15.38:0/1600393258 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639bce80 msgr2=0x55b0639bf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0259c640 1 --2- 172.21.15.38:0/1600393258 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639bce80 0x55b0639bf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda02d9d640 1 --2- 172.21.15.38:0/1600393258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639eab90 0x55b063990110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0259c640 1 -- 172.21.15.38:0/1600393258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639eab90 msgr2=0x55b063990110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0259c640 1 --2- 172.21.15.38:0/1600393258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639eab90 0x55b063990110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0259c640 1 -- 172.21.15.38:0/1600393258 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0639960f0 con 0x55b0639bcaa0 2024-04-25T08:35:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.963+0000 7fda0259c640 1 --2- 172.21.15.38:0/1600393258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bcaa0 0x55b0639bf270 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40090b0 tx=0x7fd9f402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37aee396170447c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.964+0000 7fd9e37fe640 1 -- 172.21.15.38:0/1600393258 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4037040 con 0x55b0639bcaa0 2024-04-25T08:35:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.964+0000 7fd9e37fe640 1 -- 172.21.15.38:0/1600393258 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f4052030 con 0x55b0639bcaa0 2024-04-25T08:35:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.964+0000 7fd9e37fe640 1 -- 172.21.15.38:0/1600393258 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f404e570 con 0x55b0639bcaa0 2024-04-25T08:35:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.964+0000 7fda0b66e740 1 -- 172.21.15.38:0/1600393258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bcaa0 msgr2=0x55b0639bf270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.964+0000 7fda0b66e740 1 --2- 172.21.15.38:0/1600393258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bcaa0 0x55b0639bf270 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40090b0 tx=0x7fd9f402c000 comp rx=0 tx=0).stop 2024-04-25T08:35:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.964+0000 7fda0b66e740 1 -- 172.21.15.38:0/1600393258 shutdown_connections 2024-04-25T08:35:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.964+0000 7fda0b66e740 1 --2- 172.21.15.38:0/1600393258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639eab90 0x55b063990110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.964+0000 7fda0b66e740 1 --2- 172.21.15.38:0/1600393258 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639bce80 0x55b0639bf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.964+0000 7fda0b66e740 1 --2- 172.21.15.38:0/1600393258 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bcaa0 0x55b0639bf270 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.964+0000 7fda0b66e740 1 -- 172.21.15.38:0/1600393258 >> 172.21.15.38:0/1600393258 conn(0x55b06398f700 msgr2=0x55b0639c31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.964+0000 7fda0b66e740 1 -- 172.21.15.38:0/1600393258 shutdown_connections 2024-04-25T08:35:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.964+0000 7fda0b66e740 1 -- 172.21.15.38:0/1600393258 wait complete. 2024-04-25T08:35:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.965+0000 7fda0b66e740 1 -- start start 2024-04-25T08:35:12.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.965+0000 7fda0b66e740 1 Processor -- start 2024-04-25T08:35:12.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda0b66e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 0x55b0639da510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda0b66e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 0x55b063a42330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda0b66e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639eab90 0x55b063990110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda0b66e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda041fcd50 con 0x55b0639bce80 2024-04-25T08:35:12.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda0b66e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda040b4710 con 0x55b0639bcaa0 2024-04-25T08:35:12.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda0b66e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0639e9710 con 0x55b0639eab90 2024-04-25T08:35:12.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda0b66e740 10 client.0 _ll_register_callbacks cb 0x7fd9fc009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:12.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda01d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 0x55b063a42330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda01d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 0x55b063a42330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43868/0 (socket says 172.21.15.38:43868) 2024-04-25T08:35:12.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda01d9b640 1 -- 172.21.15.38:0/3265832672 learned_addr learned my addr 172.21.15.38:0/3265832672 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:12.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda0259c640 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 0x55b0639da510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda02d9d640 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639eab90 0x55b063990110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda01d9b640 1 -- 172.21.15.38:0/3265832672 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639eab90 msgr2=0x55b063990110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda01d9b640 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639eab90 0x55b063990110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda01d9b640 1 -- 172.21.15.38:0/3265832672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 msgr2=0x55b0639da510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda01d9b640 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 0x55b0639da510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda01d9b640 1 -- 172.21.15.38:0/3265832672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0639b70f0 con 0x55b0639bce80 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda02d9d640 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639eab90 0x55b063990110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda01d9b640 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 0x55b063a42330 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8058c10 tx=0x7fd9f8088b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd40821eb761acd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fd9e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fd9e17fa640 1 -- 172.21.15.38:0/3265832672 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8061330 con 0x55b0639bce80 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fd9e17fa640 1 -- 172.21.15.38:0/3265832672 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd9f80610e0 con 0x55b0639bce80 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.966+0000 7fda0b66e740 1 -- 172.21.15.38:0/3265832672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9f4052030 con 0x55b0639bce80 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fd9e17fa640 1 -- 172.21.15.38:0/3265832672 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9f8058570 con 0x55b0639bce80 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fda0b66e740 10 client.6372 fetch_fsmap learned FSMap version 24 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fda0b66e740 1 -- 172.21.15.38:0/3265832672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55b0639e9120 con 0x55b0639bce80 2024-04-25T08:35:12.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fd9e17fa640 1 -- 172.21.15.38:0/3265832672 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f4052030 con 0x55b0639bce80 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fd9e17fa640 1 -- 172.21.15.38:0/3265832672 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd9f8082170 con 0x55b0639bce80 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fda0b66e740 10 client.6372 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fda0b66e740 10 client.6372 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fda0b66e740 20 client.6372 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fda0b66e740 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55b063990650 0x55b063a45160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fda0b66e740 4 client.6372 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fda0b66e740 1 -- 172.21.15.38:0/3265832672 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b063900690 con 0x55b063990650 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fda0259c640 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55b063990650 0x55b063a45160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fd9e17fa640 10 client.6372 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fda0259c640 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55b063990650 0x55b063a45160 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=542ba11e2810499b server_cookie=ebb44b9264cd93c2 in_seq=0 out_seq=0 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fd9e17fa640 1 -- 172.21.15.38:0/3265832672 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x55b063900690 con 0x55b063990650 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fd9e17fa640 10 client.6372 handle_command_reply: tid=0 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.967+0000 7fda0b66e740 1 client.6372 shutdown 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda0b66e740 1 -- 172.21.15.38:0/3265832672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 msgr2=0x55b063a42330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda0b66e740 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 0x55b063a42330 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8058c10 tx=0x7fd9f8088b60 comp rx=0 tx=0).stop 2024-04-25T08:35:12.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda0b66e740 1 -- 172.21.15.38:0/3265832672 shutdown_connections 2024-04-25T08:35:12.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda0b66e740 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55b063990650 0x55b063a45160 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda0b66e740 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b0639eab90 0x55b063990110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda0b66e740 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b0639bce80 0x55b063a42330 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fd9e17fa640 0 client.6372 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:12.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda0b66e740 1 --2- 172.21.15.38:0/3265832672 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b0639bcaa0 0x55b0639da510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda0b66e740 1 -- 172.21.15.38:0/3265832672 >> 172.21.15.38:0/3265832672 conn(0x55b06398f700 msgr2=0x55b0639fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:12.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda0b66e740 1 -- 172.21.15.38:0/3265832672 shutdown_connections 2024-04-25T08:35:12.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda0b66e740 1 -- 172.21.15.38:0/3265832672 wait complete. 2024-04-25T08:35:12.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda0b66e740 20 client.6372 trim_cache size 0 max 16384 2024-04-25T08:35:12.969 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T08:35:12.969 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:12.970 INFO:teuthology.orchestra.run.smithi038.stdout: "damage_type": "backtrace", 2024-04-25T08:35:12.970 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 2356467559, 2024-04-25T08:35:12.970 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 1099511630412, 2024-04-25T08:35:12.970 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-25T08:35:12.970 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:12.970 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T08:35:12.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda08963640 1 -- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd9e406b4d0 msgr2=0x7fd9e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda08963640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd9e406b4d0 0x7fd9e406d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4034000 tx=0x7fd9f402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:12.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda08963640 1 -- 172.21.15.38:0/4198168938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda040727e0 msgr2=0x7fda040c59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:12.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda08963640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda040727e0 0x7fda040c59a0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd9f800d830 tx=0x7fd9f800dcf0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda08963640 1 -- 172.21.15.38:0/4198168938 shutdown_connections 2024-04-25T08:35:12.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda08963640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd9e406b4d0 0x7fd9e406d990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda08963640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda0415b830 0x7fda0415d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda08963640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda040727e0 0x7fda040c59a0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda08963640 1 --2- 172.21.15.38:0/4198168938 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda0406bbf0 0x7fda040c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:12.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda08963640 1 -- 172.21.15.38:0/4198168938 >> 172.21.15.38:0/4198168938 conn(0x7fda040717f0 msgr2=0x7fda04071bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:12.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda08963640 1 -- 172.21.15.38:0/4198168938 shutdown_connections 2024-04-25T08:35:12.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:12.968+0000 7fda08963640 1 -- 172.21.15.38:0/4198168938 wait complete. 2024-04-25T08:35:12.974 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:13.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.080+0000 7f58a659e640 1 Processor -- start 2024-04-25T08:35:13.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.080+0000 7f58a659e640 1 -- start start 2024-04-25T08:35:13.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f58a659e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f58a0160da0 0x7f58a00d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f58a659e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a00b64f0 0x7f58a00d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f58a659e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a00cef10 con 0x7f58a00d42f0 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f58a659e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a00cf740 con 0x7f58a0160da0 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f58a659e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00cf180 con 0x7f58a00b64f0 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f58a5d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f58a00d42f0 0x7f58a01616b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56662/0 (socket says 172.21.15.38:56662) 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f58a5d9d640 1 -- 172.21.15.38:0/3145836358 learned_addr learned my addr 172.21.15.38:0/3145836358 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914201574 0 0) 0x7f58a00cef10 con 0x7f58a00d42f0 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f588c003520 con 0x7f58a00d42f0 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f58a4d9b640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a00b64f0 0x7f58a00d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f58a559c640 1 --2- 172.21.15.38:0/3145836358 >> v2:172.21.15.107:3300/0 conn(0x7f58a0160da0 0x7f58a00d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3058111159 0 0) 0x7f588c003520 con 0x7f58a00d42f0 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a00b64f0 msgr2=0x7f58a00d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f5897fff640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a00b64f0 0x7f58a00d0020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 >> v2:172.21.15.107:3300/0 conn(0x7f58a0160da0 msgr2=0x7f58a00d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f5897fff640 1 --2- 172.21.15.38:0/3145836358 >> v2:172.21.15.107:3300/0 conn(0x7f58a0160da0 0x7f58a00d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00cf460 con 0x7f58a00d42f0 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.081+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f58980019f0 con 0x7f58a00d42f0 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 >> v1:172.21.15.38:6789/0 conn(0x7f58a00d42f0 legacy=0x7f58a01616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f5897fff640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a00b64f0 0x7f588c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f5897fff640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f588c003f80 0x7f588c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f5897fff640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f588c004d40 0x7f588c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00cf740 con 0x7f588c003f80 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00cf180 con 0x7f588c004d40 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c00ba70 con 0x7f58a00b64f0 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f58a4d9b640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f588c003f80 0x7f588c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f58a559c640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a00b64f0 0x7f588c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f58a5d9d640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f588c004d40 0x7f588c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f58a4d9b640 1 -- 172.21.15.38:0/3145836358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a00b64f0 msgr2=0x7f588c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f58a4d9b640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a00b64f0 0x7f588c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f58a4d9b640 1 -- 172.21.15.38:0/3145836358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f588c004d40 msgr2=0x7f588c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f58a4d9b640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f588c004d40 0x7f588c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f58a4d9b640 1 -- 172.21.15.38:0/3145836358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c003520 con 0x7f588c003f80 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.082+0000 7f58a5d9d640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f588c004d40 0x7f588c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.083+0000 7f58a4d9b640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f588c003f80 0x7f588c004800 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5890009a90 tx=0x7f588c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee4c9cca167b49b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.083+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890036070 con 0x7f588c003f80 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.083+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5890031400 con 0x7f588c003f80 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.083+0000 7f5897fff640 1 -- 172.21.15.38:0/3145836358 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890009690 con 0x7f588c003f80 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.083+0000 7f58a659e640 1 -- 172.21.15.38:0/3145836358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f588c003f80 msgr2=0x7f588c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.083+0000 7f58a659e640 1 --2- 172.21.15.38:0/3145836358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f588c003f80 0x7f588c004800 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5890009a90 tx=0x7f588c005830 comp rx=0 tx=0).stop 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.083+0000 7f58a5d9d640 1 -- 172.21.15.38:0/3145836358 reap_dead start 2024-04-25T08:35:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.083+0000 7f58a659e640 1 -- 172.21.15.38:0/3145836358 shutdown_connections 2024-04-25T08:35:13.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.083+0000 7f58a659e640 1 -- 172.21.15.38:0/3145836358 >> 172.21.15.38:0/3145836358 conn(0x7f58a008b000 msgr2=0x7f58a00d36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:13.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.083+0000 7f58a659e640 1 -- 172.21.15.38:0/3145836358 shutdown_connections 2024-04-25T08:35:13.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.084+0000 7f58a659e640 1 -- 172.21.15.38:0/3145836358 wait complete. 2024-04-25T08:35:13.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.085+0000 7f58a659e640 1 Processor -- start 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.085+0000 7f58a659e640 1 -- start start 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.085+0000 7f58a659e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f588c003f80 0x7f58a00b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.085+0000 7f58a659e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a0155af0 0x7f58a0153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a659e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a0154390 0x7f58a00c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a659e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a0161300 con 0x7f58a0155af0 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a659e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a0161180 con 0x7f588c003f80 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a659e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a0161480 con 0x7f58a0154390 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a4d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a0155af0 0x7f58a0153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a4d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a0155af0 0x7f58a0153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43880/0 (socket says 172.21.15.38:43880) 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a4d9b640 1 -- 172.21.15.38:0/1272650205 learned_addr learned my addr 172.21.15.38:0/1272650205 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a559c640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f588c003f80 0x7f58a00b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a5d9d640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a0154390 0x7f58a00c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a4d9b640 1 -- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a0154390 msgr2=0x7f58a00c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a4d9b640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a0154390 0x7f58a00c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a4d9b640 1 -- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f588c003f80 msgr2=0x7f58a00b83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a4d9b640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f588c003f80 0x7f58a00b83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a4d9b640 1 -- 172.21.15.38:0/1272650205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00ca910 con 0x7f58a0155af0 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a559c640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f588c003f80 0x7f58a00b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a5d9d640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a0154390 0x7f58a00c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a4d9b640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a0155af0 0x7f58a0153e50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5890000c00 tx=0x7f5890004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f5895ffb640 1 -- 172.21.15.38:0/1272650205 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890040020 con 0x7f58a0155af0 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f5895ffb640 1 -- 172.21.15.38:0/1272650205 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5890003c60 con 0x7f58a0155af0 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f58a659e640 1 -- 172.21.15.38:0/1272650205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a00cabf0 con 0x7f58a0155af0 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.086+0000 7f5895ffb640 1 -- 172.21.15.38:0/1272650205 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5890036040 con 0x7f58a0155af0 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.087+0000 7f58a659e640 1 -- 172.21.15.38:0/1272650205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a00cb1b0 con 0x7f58a0155af0 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.088+0000 7f58a659e640 1 -- 172.21.15.38:0/1272650205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a004ef20 con 0x7f58a0155af0 2024-04-25T08:35:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.088+0000 7f5895ffb640 1 -- 172.21.15.38:0/1272650205 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5890043020 con 0x7f58a0155af0 2024-04-25T08:35:13.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.089+0000 7f5895ffb640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f588406b4d0 0x7f588406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.089+0000 7f5895ffb640 1 -- 172.21.15.38:0/1272650205 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58900aa4b0 con 0x7f58a0155af0 2024-04-25T08:35:13.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.089+0000 7f58a559c640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f588406b4d0 0x7f588406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.089+0000 7f58a559c640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f588406b4d0 0x7f588406d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f589c009b30 tx=0x7f589c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.093+0000 7f5895ffb640 1 -- 172.21.15.38:0/1272650205 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f589004a070 con 0x7f58a0155af0 2024-04-25T08:35:13.093 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:35:13.099+0000 7ff795ff5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T08:35:13.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.309+0000 7f58a659e640 1 -- 172.21.15.38:0/1272650205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58a00b6940 con 0x7f58a0155af0 2024-04-25T08:35:13.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.310+0000 7f5895ffb640 1 -- 172.21.15.38:0/1272650205 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f58900044e0 con 0x7f58a0155af0 2024-04-25T08:35:13.305 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:13.305 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:35:12.877254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:13.305 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-04-25T08:35:13.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.312+0000 7f58a659e640 1 -- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f588406b4d0 msgr2=0x7f588406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.312+0000 7f58a659e640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f588406b4d0 0x7f588406d990 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f589c009b30 tx=0x7f589c008040 comp rx=0 tx=0).stop 2024-04-25T08:35:13.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.312+0000 7f58a659e640 1 -- 172.21.15.38:0/1272650205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a0155af0 msgr2=0x7f58a0153e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.312+0000 7f58a659e640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a0155af0 0x7f58a0153e50 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5890000c00 tx=0x7f5890004730 comp rx=0 tx=0).stop 2024-04-25T08:35:13.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.312+0000 7f58a659e640 1 -- 172.21.15.38:0/1272650205 shutdown_connections 2024-04-25T08:35:13.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.312+0000 7f58a659e640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f588406b4d0 0x7f588406d990 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.312+0000 7f58a659e640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58a0154390 0x7f58a00c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.312+0000 7f58a659e640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58a0155af0 0x7f58a0153e50 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.312+0000 7f58a659e640 1 --2- 172.21.15.38:0/1272650205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f588c003f80 0x7f58a00b83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.312+0000 7f58a659e640 1 -- 172.21.15.38:0/1272650205 >> 172.21.15.38:0/1272650205 conn(0x7f58a008b000 msgr2=0x7f58a015f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:13.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.312+0000 7f58a659e640 1 -- 172.21.15.38:0/1272650205 shutdown_connections 2024-04-25T08:35:13.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.312+0000 7f58a659e640 1 -- 172.21.15.38:0/1272650205 wait complete. 2024-04-25T08:35:13.317 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 2356467559 2024-04-25T08:35:13.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.426+0000 7f9b000cb640 1 Processor -- start 2024-04-25T08:35:13.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.426+0000 7f9b000cb640 1 -- start start 2024-04-25T08:35:13.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.427+0000 7f9b000cb640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9af80ccfa0 0x7f9af80cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.427+0000 7f9b000cb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af80cdc90 0x7f9af80cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.427+0000 7f9b000cb640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80cd8c0 con 0x7f9af80ce280 2024-04-25T08:35:13.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.427+0000 7f9b000cb640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af80cc4b0 con 0x7f9af80ccfa0 2024-04-25T08:35:13.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.427+0000 7f9b000cb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80ad570 con 0x7f9af80cdc90 2024-04-25T08:35:13.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.427+0000 7f9afe668640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9af80ce280 0x7f9af80ce710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56672/0 (socket says 172.21.15.38:56672) 2024-04-25T08:35:13.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.427+0000 7f9afe668640 1 -- 172.21.15.38:0/593090624 learned_addr learned my addr 172.21.15.38:0/593090624 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afde67640 1 --2- 172.21.15.38:0/593090624 >> v2:172.21.15.107:3300/0 conn(0x7f9af80ccfa0 0x7f9af80cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afd666640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af80cdc90 0x7f9af80cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899598645 0 0) 0x7f9af80cd8c0 con 0x7f9af80ce280 2024-04-25T08:35:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae0003520 con 0x7f9af80ce280 2024-04-25T08:35:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 196415141 0 0) 0x7f9ae0003520 con 0x7f9af80ce280 2024-04-25T08:35:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af80cdc90 msgr2=0x7f9af80cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af80cdc90 0x7f9af80cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 >> v2:172.21.15.107:3300/0 conn(0x7f9af80ccfa0 msgr2=0x7f9af80cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 --2- 172.21.15.38:0/593090624 >> v2:172.21.15.107:3300/0 conn(0x7f9af80ccfa0 0x7f9af80cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80ad850 con 0x7f9af80ce280 2024-04-25T08:35:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f9ae80019f0 con 0x7f9af80ce280 2024-04-25T08:35:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 >> v1:172.21.15.38:6789/0 conn(0x7f9af80ce280 legacy=0x7f9af80ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af80cdc90 0x7f9ae00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ae0003f30 0x7f9ae0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae0004d40 0x7f9ae00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80cc4b0 con 0x7f9ae0003f30 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80ad570 con 0x7f9ae0004d40 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae000b9f0 con 0x7f9af80cdc90 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afd666640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ae0003f30 0x7f9ae0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afde67640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af80cdc90 0x7f9ae00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afe668640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae0004d40 0x7f9ae00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afd666640 1 -- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af80cdc90 msgr2=0x7f9ae00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afd666640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af80cdc90 0x7f9ae00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afd666640 1 -- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae0004d40 msgr2=0x7f9ae00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afd666640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae0004d40 0x7f9ae00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.428+0000 7f9afd666640 1 -- 172.21.15.38:0/593090624 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae0003520 con 0x7f9ae0003f30 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.429+0000 7f9afde67640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af80cdc90 0x7f9ae00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.429+0000 7f9afe668640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae0004d40 0x7f9ae00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.429+0000 7f9afd666640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ae0003f30 0x7f9ae0004800 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009ac0 tx=0x7f9ae0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e98e99277bad311 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.429+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec036070 con 0x7f9ae0003f30 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.429+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec031400 con 0x7f9ae0003f30 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.429+0000 7f9afce65640 1 -- 172.21.15.38:0/593090624 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec009690 con 0x7f9ae0003f30 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.430+0000 7f9b000cb640 1 -- 172.21.15.38:0/593090624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ae0003f30 msgr2=0x7f9ae0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.430+0000 7f9b000cb640 1 --2- 172.21.15.38:0/593090624 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ae0003f30 0x7f9ae0004800 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9aec009ac0 tx=0x7f9ae0005830 comp rx=0 tx=0).stop 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.430+0000 7f9afe668640 1 -- 172.21.15.38:0/593090624 reap_dead start 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.430+0000 7f9b000cb640 1 -- 172.21.15.38:0/593090624 shutdown_connections 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.430+0000 7f9b000cb640 1 -- 172.21.15.38:0/593090624 >> 172.21.15.38:0/593090624 conn(0x7f9af80d06b0 msgr2=0x7f9af80d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.430+0000 7f9b000cb640 1 -- 172.21.15.38:0/593090624 shutdown_connections 2024-04-25T08:35:13.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.430+0000 7f9b000cb640 1 -- 172.21.15.38:0/593090624 wait complete. 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.431+0000 7f9b000cb640 1 Processor -- start 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.431+0000 7f9b000cb640 1 -- start start 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9b000cb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae0003f30 0x7f9af80b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9b000cb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af8150240 0x7f9af80c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9b000cb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af80ca4c0 0x7f9af80c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9b000cb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80cee20 con 0x7f9af80ca4c0 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9b000cb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af815ad80 con 0x7f9ae0003f30 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9b000cb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80cefa0 con 0x7f9af8150240 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9afe668640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af80ca4c0 0x7f9af80c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9afe668640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af80ca4c0 0x7f9af80c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43896/0 (socket says 172.21.15.38:43896) 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9afe668640 1 -- 172.21.15.38:0/2658424578 learned_addr learned my addr 172.21.15.38:0/2658424578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9afde67640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae0003f30 0x7f9af80b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9afd666640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af8150240 0x7f9af80c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9afe668640 1 -- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af8150240 msgr2=0x7f9af80c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9afe668640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af8150240 0x7f9af80c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9afe668640 1 -- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae0003f30 msgr2=0x7f9af80b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9afe668640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae0003f30 0x7f9af80b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.432+0000 7f9afe668640 1 -- 172.21.15.38:0/2658424578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af8150a20 con 0x7f9af80ca4c0 2024-04-25T08:35:13.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.433+0000 7f9afe668640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af80ca4c0 0x7f9af80c7f20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80013d0 tx=0x7f9ae8004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.433+0000 7f9af67fc640 1 -- 172.21.15.38:0/2658424578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae800a4e0 con 0x7f9af80ca4c0 2024-04-25T08:35:13.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.433+0000 7f9af67fc640 1 -- 172.21.15.38:0/2658424578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae80078d0 con 0x7f9af80ca4c0 2024-04-25T08:35:13.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.433+0000 7f9af67fc640 1 -- 172.21.15.38:0/2658424578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0009690 con 0x7f9af80ca4c0 2024-04-25T08:35:13.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.433+0000 7f9b000cb640 1 -- 172.21.15.38:0/2658424578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af8150d00 con 0x7f9af80ca4c0 2024-04-25T08:35:13.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.433+0000 7f9b000cb640 1 -- 172.21.15.38:0/2658424578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af81506c0 con 0x7f9af80ca4c0 2024-04-25T08:35:13.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.434+0000 7f9af67fc640 1 -- 172.21.15.38:0/2658424578 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ae8013630 con 0x7f9af80ca4c0 2024-04-25T08:35:13.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.435+0000 7f9af67fc640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ae406b500 0x7f9ae406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.435+0000 7f9af67fc640 1 -- 172.21.15.38:0/2658424578 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ae808a4b0 con 0x7f9af80ca4c0 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.435+0000 7f9afde67640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ae406b500 0x7f9ae406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.435+0000 7f9afde67640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ae406b500 0x7f9ae406d9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009de0 tx=0x7f9af0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.436+0000 7f9b02dce740 1 Processor -- start 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.436+0000 7f9b02dce740 1 -- start start 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.436+0000 7f9b02dce740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 0x55ec126c9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.436+0000 7f9b02dce740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c7360 0x55ec126c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9b02dce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec12699be0 0x55ec1269d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9b02dce740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec12602ac0 con 0x55ec126c6f80 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9b02dce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec125abdb0 con 0x55ec12699be0 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9b02dce740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec125feb60 con 0x55ec126c7360 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9afde67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 0x55ec126c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9afde67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 0x55ec126c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43900/0 (socket says 172.21.15.38:43900) 2024-04-25T08:35:13.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9afde67640 1 -- 172.21.15.38:0/524845436 learned_addr learned my addr 172.21.15.38:0/524845436 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9afe668640 1 --2- 172.21.15.38:0/524845436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec12699be0 0x55ec1269d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9afd666640 1 --2- 172.21.15.38:0/524845436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c7360 0x55ec126c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9afde67640 1 -- 172.21.15.38:0/524845436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c7360 msgr2=0x55ec126c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9afde67640 1 --2- 172.21.15.38:0/524845436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c7360 0x55ec126c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9afde67640 1 -- 172.21.15.38:0/524845436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec12699be0 msgr2=0x55ec1269d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9afde67640 1 --2- 172.21.15.38:0/524845436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec12699be0 0x55ec1269d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9afde67640 1 -- 172.21.15.38:0/524845436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec123e3710 con 0x55ec126c6f80 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9afde67640 1 --2- 172.21.15.38:0/524845436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 0x55ec126c9790 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9af0004380 tx=0x7f9af0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8245737134bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9adf7fe640 1 -- 172.21.15.38:0/524845436 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af001b070 con 0x55ec126c6f80 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9adf7fe640 1 -- 172.21.15.38:0/524845436 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af0019070 con 0x55ec126c6f80 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.437+0000 7f9adf7fe640 1 -- 172.21.15.38:0/524845436 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0015620 con 0x55ec126c6f80 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.438+0000 7f9b02dce740 1 -- 172.21.15.38:0/524845436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 msgr2=0x55ec126c9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.438+0000 7f9b02dce740 1 --2- 172.21.15.38:0/524845436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 0x55ec126c9790 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9af0004380 tx=0x7f9af0017000 comp rx=0 tx=0).stop 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.438+0000 7f9b02dce740 1 -- 172.21.15.38:0/524845436 shutdown_connections 2024-04-25T08:35:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.438+0000 7f9b02dce740 1 --2- 172.21.15.38:0/524845436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec12699be0 0x55ec1269d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.438+0000 7f9b02dce740 1 --2- 172.21.15.38:0/524845436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c7360 0x55ec126c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.438+0000 7f9b02dce740 1 --2- 172.21.15.38:0/524845436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 0x55ec126c9790 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.438+0000 7f9b02dce740 1 -- 172.21.15.38:0/524845436 >> 172.21.15.38:0/524845436 conn(0x55ec1269be50 msgr2=0x55ec126e4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.438+0000 7f9b02dce740 1 -- 172.21.15.38:0/524845436 shutdown_connections 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.438+0000 7f9b02dce740 1 -- 172.21.15.38:0/524845436 wait complete. 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.439+0000 7f9b02dce740 1 -- start start 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.439+0000 7f9b02dce740 1 Processor -- start 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.439+0000 7f9b02dce740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c6f80 0x55ec126f7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9b02dce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 0x55ec126bd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9b02dce740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 0x55ec126ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9b02dce740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec1269f000 con 0x55ec126e58c0 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9b02dce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec125a14e0 con 0x55ec126c7360 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9b02dce740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec1269f2e0 con 0x55ec126c6f80 2024-04-25T08:35:13.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9b02dce740 10 client.0 _ll_register_callbacks cb 0x55ec12316af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9afe668640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 0x55ec126ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9afe668640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 0x55ec126ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43906/0 (socket says 172.21.15.38:43906) 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9afe668640 1 -- 172.21.15.38:0/4184379411 learned_addr learned my addr 172.21.15.38:0/4184379411 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9afde67640 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c6f80 0x55ec126f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9afd666640 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 0x55ec126bd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9afe668640 1 -- 172.21.15.38:0/4184379411 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c6f80 msgr2=0x55ec126f7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9afe668640 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c6f80 0x55ec126f7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9afe668640 1 -- 172.21.15.38:0/4184379411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 msgr2=0x55ec126bd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9afe668640 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 0x55ec126bd590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9afe668640 1 -- 172.21.15.38:0/4184379411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec12443ea0 con 0x55ec126e58c0 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9afe668640 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 0x55ec126ca660 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80074f0 tx=0x7f9ae8058b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3532ff567b42fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9adcff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9adcff9640 1 -- 172.21.15.38:0/4184379411 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8059e60 con 0x55ec126e58c0 2024-04-25T08:35:13.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9adcff9640 1 -- 172.21.15.38:0/4184379411 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ae805dba0 con 0x55ec126e58c0 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9b02dce740 1 -- 172.21.15.38:0/4184379411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9af0019070 con 0x55ec126e58c0 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9adcff9640 1 -- 172.21.15.38:0/4184379411 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ae8007a80 con 0x55ec126e58c0 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9b02dce740 10 client.6390 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.440+0000 7f9b02dce740 1 -- 172.21.15.38:0/4184379411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55ec122d7000 con 0x55ec126e58c0 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.441+0000 7f9adcff9640 1 -- 172.21.15.38:0/4184379411 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af0019070 con 0x55ec126e58c0 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.441+0000 7f9adcff9640 1 -- 172.21.15.38:0/4184379411 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9ae8057f80 con 0x55ec126e58c0 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.441+0000 7f9b02dce740 10 client.6390 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.441+0000 7f9b02dce740 10 client.6390 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.441+0000 7f9b02dce740 20 client.6390 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.441+0000 7f9b02dce740 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ec126e7a90 0x55ec126e7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.441+0000 7f9b02dce740 4 client.6390 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.441+0000 7f9b02dce740 1 -- 172.21.15.38:0/4184379411 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec12610000 con 0x55ec126e7a90 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.441+0000 7f9afde67640 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ec126e7a90 0x55ec126e7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.441+0000 7f9adcff9640 10 client.6390 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.441+0000 7f9afde67640 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ec126e7a90 0x55ec126e7e70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb363fb58086e8b server_cookie=6a5fa69631e69eb7 in_seq=0 out_seq=0 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9adcff9640 1 -- 172.21.15.38:0/4184379411 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ec12610000 con 0x55ec126e7a90 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9adcff9640 10 client.6390 handle_command_reply: tid=0 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9b02dce740 1 client.6390 shutdown 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9b02dce740 1 -- 172.21.15.38:0/4184379411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 msgr2=0x55ec126ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9b02dce740 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 0x55ec126ca660 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80074f0 tx=0x7f9ae8058b30 comp rx=0 tx=0).stop 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9b02dce740 1 -- 172.21.15.38:0/4184379411 shutdown_connections 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9b02dce740 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ec126e7a90 0x55ec126e7e70 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9b02dce740 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 0x55ec126ca660 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9b02dce740 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 0x55ec126bd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9b02dce740 1 --2- 172.21.15.38:0/4184379411 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c6f80 0x55ec126f7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9b02dce740 1 -- 172.21.15.38:0/4184379411 >> 172.21.15.38:0/4184379411 conn(0x55ec12699be0 msgr2=0x55ec126c5600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:13.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9adcff9640 0 client.6390 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:13.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9b02dce740 1 -- 172.21.15.38:0/4184379411 shutdown_connections 2024-04-25T08:35:13.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9b02dce740 1 -- 172.21.15.38:0/4184379411 wait complete. 2024-04-25T08:35:13.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.442+0000 7f9b02dce740 20 client.6390 trim_cache size 0 max 16384 2024-04-25T08:35:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.454+0000 7f9b02dce740 1 Processor -- start 2024-04-25T08:35:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.454+0000 7f9b02dce740 1 -- start start 2024-04-25T08:35:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9b02dce740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c6f80 0x55ec126f7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9b02dce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 0x55ec126f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9b02dce740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 0x55ec126e6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9b02dce740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af81fd940 con 0x55ec126e58c0 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9b02dce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af80b02a0 con 0x55ec126c7360 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9b02dce740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af81fd370 con 0x55ec126c6f80 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9afe668640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 0x55ec126e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9afe668640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 0x55ec126e6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43914/0 (socket says 172.21.15.38:43914) 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9afe668640 1 -- 172.21.15.38:0/3796970989 learned_addr learned my addr 172.21.15.38:0/3796970989 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9afe668640 1 -- 172.21.15.38:0/3796970989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c6f80 msgr2=0x55ec126f7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9afe668640 1 --2- 172.21.15.38:0/3796970989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c6f80 0x55ec126f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9afe668640 1 -- 172.21.15.38:0/3796970989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 msgr2=0x55ec126f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9afd666640 1 --2- 172.21.15.38:0/3796970989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 0x55ec126f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9afe668640 1 --2- 172.21.15.38:0/3796970989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 0x55ec126f7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9afe668640 1 -- 172.21.15.38:0/3796970989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec126e7770 con 0x55ec126e58c0 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9afe668640 1 --2- 172.21.15.38:0/3796970989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 0x55ec126e6150 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8060d10 tx=0x7f9ae808a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9405c9c93ff0d420 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.455+0000 7f9adeffd640 1 -- 172.21.15.38:0/3796970989 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8059e60 con 0x55ec126e58c0 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.456+0000 7f9adeffd640 1 -- 172.21.15.38:0/3796970989 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae808a0c0 con 0x55ec126e58c0 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.456+0000 7f9adeffd640 1 -- 172.21.15.38:0/3796970989 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8054dd0 con 0x55ec126e58c0 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.456+0000 7f9b02dce740 1 -- 172.21.15.38:0/3796970989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 msgr2=0x55ec126e6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.456+0000 7f9b02dce740 1 --2- 172.21.15.38:0/3796970989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 0x55ec126e6150 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8060d10 tx=0x7f9ae808a050 comp rx=0 tx=0).stop 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.456+0000 7f9b02dce740 1 -- 172.21.15.38:0/3796970989 shutdown_connections 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.456+0000 7f9b02dce740 1 --2- 172.21.15.38:0/3796970989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126e58c0 0x55ec126e6150 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.456+0000 7f9b02dce740 1 --2- 172.21.15.38:0/3796970989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 0x55ec126f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.456+0000 7f9b02dce740 1 --2- 172.21.15.38:0/3796970989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126c6f80 0x55ec126f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.456+0000 7f9b02dce740 1 -- 172.21.15.38:0/3796970989 >> 172.21.15.38:0/3796970989 conn(0x55ec12699be0 msgr2=0x55ec126e4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.456+0000 7f9b02dce740 1 -- 172.21.15.38:0/3796970989 shutdown_connections 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.456+0000 7f9b02dce740 1 -- 172.21.15.38:0/3796970989 wait complete. 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.457+0000 7f9b02dce740 1 -- start start 2024-04-25T08:35:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.457+0000 7f9b02dce740 1 Processor -- start 2024-04-25T08:35:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9b02dce740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 0x55ec126f7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9b02dce740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 0x55ec126f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9b02dce740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126e58c0 0x55ec1269dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9b02dce740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec126bdb70 con 0x55ec126c6f80 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9b02dce740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec126e5d00 con 0x55ec126c7360 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9b02dce740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec126cc2e0 con 0x55ec126e58c0 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9b02dce740 10 client.0 _ll_register_callbacks cb 0x7f9af0009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9afde67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 0x55ec126f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9afde67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 0x55ec126f7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43924/0 (socket says 172.21.15.38:43924) 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9afde67640 1 -- 172.21.15.38:0/995780416 learned_addr learned my addr 172.21.15.38:0/995780416 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9afe668640 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126e58c0 0x55ec1269dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9afd666640 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 0x55ec126f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9afde67640 1 -- 172.21.15.38:0/995780416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126e58c0 msgr2=0x55ec1269dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9afde67640 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126e58c0 0x55ec1269dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9afde67640 1 -- 172.21.15.38:0/995780416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 msgr2=0x55ec126f7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9afde67640 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 0x55ec126f7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9afde67640 1 -- 172.21.15.38:0/995780416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec12738920 con 0x55ec126c6f80 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9afde67640 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 0x55ec126f7690 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9af0004380 tx=0x7f9af0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bad9d6257cb7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9adcff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9adcff9640 1 -- 172.21.15.38:0/995780416 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0036480 con 0x55ec126c6f80 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9adcff9640 1 -- 172.21.15.38:0/995780416 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9af0036620 con 0x55ec126c6f80 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.458+0000 7f9b02dce740 1 -- 172.21.15.38:0/995780416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9ae808a0c0 con 0x55ec126c6f80 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9adcff9640 1 -- 172.21.15.38:0/995780416 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9af0043050 con 0x55ec126c6f80 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9b02dce740 10 client.6396 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9b02dce740 1 -- 172.21.15.38:0/995780416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55ec126e6350 con 0x55ec126c6f80 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9adcff9640 1 -- 172.21.15.38:0/995780416 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae808a0c0 con 0x55ec126c6f80 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9adcff9640 1 -- 172.21.15.38:0/995780416 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9af0046020 con 0x55ec126c6f80 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9b02dce740 10 client.6396 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9b02dce740 10 client.6396 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9b02dce740 20 client.6396 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9b02dce740 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ec1269e1f0 0x55ec126c4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9b02dce740 4 client.6396 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2356467559}] 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9b02dce740 1 -- 172.21.15.38:0/995780416 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2356467559}) v1 -- 0x55ec1260a350 con 0x55ec1269e1f0 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9afd666640 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ec1269e1f0 0x55ec126c4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9afd666640 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ec1269e1f0 0x55ec126c4b30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b853c41da594d4f0 server_cookie=c6668e96549f555b in_seq=0 out_seq=0 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9adcff9640 10 client.6396 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9adcff9640 1 -- 172.21.15.38:0/995780416 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55ec1260a350 con 0x55ec1269e1f0 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9adcff9640 10 client.6396 handle_command_reply: tid=0 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.459+0000 7f9b02dce740 1 client.6396 shutdown 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b02dce740 1 -- 172.21.15.38:0/995780416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 msgr2=0x55ec126f7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b02dce740 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 0x55ec126f7690 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9af0004380 tx=0x7f9af0004730 comp rx=0 tx=0).stop 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b02dce740 1 -- 172.21.15.38:0/995780416 shutdown_connections 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b02dce740 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ec1269e1f0 0x55ec126c4b30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b02dce740 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ec126e58c0 0x55ec1269dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b02dce740 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ec126c7360 0x55ec126f7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b02dce740 1 --2- 172.21.15.38:0/995780416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ec126c6f80 0x55ec126f7690 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9adcff9640 0 client.6396 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b02dce740 1 -- 172.21.15.38:0/995780416 >> 172.21.15.38:0/995780416 conn(0x55ec12699be0 msgr2=0x55ec12698c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b02dce740 1 -- 172.21.15.38:0/995780416 shutdown_connections 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b02dce740 1 -- 172.21.15.38:0/995780416 wait complete. 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b02dce740 20 client.6396 trim_cache size 0 max 16384 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b000cb640 1 -- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ae406b500 msgr2=0x7f9ae406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b000cb640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ae406b500 0x7f9ae406d9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9af0009de0 tx=0x7f9af0009190 comp rx=0 tx=0).stop 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b000cb640 1 -- 172.21.15.38:0/2658424578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af80ca4c0 msgr2=0x7f9af80c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b000cb640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af80ca4c0 0x7f9af80c7f20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9ae80013d0 tx=0x7f9ae8004f10 comp rx=0 tx=0).stop 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b000cb640 1 -- 172.21.15.38:0/2658424578 shutdown_connections 2024-04-25T08:35:13.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b000cb640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9af80ca4c0 0x7f9af80c7f20 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b000cb640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ae406b500 0x7f9ae406d9c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b000cb640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9af8150240 0x7f9af80c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b000cb640 1 --2- 172.21.15.38:0/2658424578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ae0003f30 0x7f9af80b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b000cb640 1 -- 172.21.15.38:0/2658424578 >> 172.21.15.38:0/2658424578 conn(0x7f9af80d06b0 msgr2=0x7f9af80d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b000cb640 1 -- 172.21.15.38:0/2658424578 shutdown_connections 2024-04-25T08:35:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.460+0000 7f9b000cb640 1 -- 172.21.15.38:0/2658424578 wait complete. 2024-04-25T08:35:13.465 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T08:35:13.466 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:13.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.571+0000 7fcadfb29640 1 Processor -- start 2024-04-25T08:35:13.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.572+0000 7fcadfb29640 1 -- start start 2024-04-25T08:35:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.572+0000 7fcadfb29640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcad806cc10 0x7fcad80cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.572+0000 7fcadfb29640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcad806cff0 0x7fcad80cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.572+0000 7fcadfb29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcad8099a70 0x7fcad8099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.572+0000 7fcadfb29640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad809a3e0 con 0x7fcad806cc10 2024-04-25T08:35:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.572+0000 7fcadfb29640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad80d05c0 con 0x7fcad806cff0 2024-04-25T08:35:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.572+0000 7fcadfb29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80d08a0 con 0x7fcad8099a70 2024-04-25T08:35:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.573+0000 7fcadd8c5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcad806cc10 0x7fcad80cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.573+0000 7fcadd8c5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcad806cc10 0x7fcad80cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43926/0 (socket says 172.21.15.38:43926) 2024-04-25T08:35:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.573+0000 7fcadd8c5640 1 -- 172.21.15.38:0/2200061017 learned_addr learned my addr 172.21.15.38:0/2200061017 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.573+0000 7fcadd0c4640 1 --2- 172.21.15.38:0/2200061017 >> v2:172.21.15.107:3300/0 conn(0x7fcad806cff0 0x7fcad80cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.573+0000 7fcade0c6640 1 --2- 172.21.15.38:0/2200061017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcad8099a70 0x7fcad8099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.573+0000 7fcadd8c5640 1 -- 172.21.15.38:0/2200061017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcad8099a70 msgr2=0x7fcad8099ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.573+0000 7fcadd8c5640 1 --2- 172.21.15.38:0/2200061017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcad8099a70 0x7fcad8099ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.573+0000 7fcadd8c5640 1 -- 172.21.15.38:0/2200061017 >> v2:172.21.15.107:3300/0 conn(0x7fcad806cff0 msgr2=0x7fcad80cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:13.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.573+0000 7fcadd8c5640 1 --2- 172.21.15.38:0/2200061017 >> v2:172.21.15.107:3300/0 conn(0x7fcad806cff0 0x7fcad80cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.573+0000 7fcadd8c5640 1 -- 172.21.15.38:0/2200061017 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad80d0b80 con 0x7fcad806cc10 2024-04-25T08:35:13.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadd8c5640 1 --2- 172.21.15.38:0/2200061017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcad806cc10 0x7fcad80cfa70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcad806db60 tx=0x7fcad002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4e85987b0fe46aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadc8c3640 1 -- 172.21.15.38:0/2200061017 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0037070 con 0x7fcad806cc10 2024-04-25T08:35:13.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadc8c3640 1 -- 172.21.15.38:0/2200061017 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad0032440 con 0x7fcad806cc10 2024-04-25T08:35:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadc8c3640 1 -- 172.21.15.38:0/2200061017 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0032810 con 0x7fcad806cc10 2024-04-25T08:35:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadfb29640 1 -- 172.21.15.38:0/2200061017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcad806cc10 msgr2=0x7fcad80cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadfb29640 1 --2- 172.21.15.38:0/2200061017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcad806cc10 0x7fcad80cfa70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcad806db60 tx=0x7fcad002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadfb29640 1 -- 172.21.15.38:0/2200061017 shutdown_connections 2024-04-25T08:35:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadfb29640 1 --2- 172.21.15.38:0/2200061017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcad8099a70 0x7fcad8099ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadfb29640 1 --2- 172.21.15.38:0/2200061017 >> v2:172.21.15.107:3300/0 conn(0x7fcad806cff0 0x7fcad80cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadfb29640 1 --2- 172.21.15.38:0/2200061017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcad806cc10 0x7fcad80cfa70 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadfb29640 1 -- 172.21.15.38:0/2200061017 >> 172.21.15.38:0/2200061017 conn(0x7fcad80b31f0 msgr2=0x7fcad80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadfb29640 1 -- 172.21.15.38:0/2200061017 shutdown_connections 2024-04-25T08:35:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.574+0000 7fcadfb29640 1 -- 172.21.15.38:0/2200061017 wait complete. 2024-04-25T08:35:13.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.576+0000 7fcadfb29640 1 Processor -- start 2024-04-25T08:35:13.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.576+0000 7fcadfb29640 1 -- start start 2024-04-25T08:35:13.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.576+0000 7fcadfb29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcad806cc10 0x7fcad80ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.576+0000 7fcadfb29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcad806cff0 0x7fcad81f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.576+0000 7fcadfb29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcad8099a70 0x7fcad80c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.576+0000 7fcadfb29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad806c150 con 0x7fcad806cff0 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.576+0000 7fcadfb29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80d1140 con 0x7fcad8099a70 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.576+0000 7fcadfb29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad806c2d0 con 0x7fcad806cc10 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.576+0000 7fcadd0c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcad806cff0 0x7fcad81f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.576+0000 7fcadd0c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcad806cff0 0x7fcad81f5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43940/0 (socket says 172.21.15.38:43940) 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.576+0000 7fcadd0c4640 1 -- 172.21.15.38:0/1141932187 learned_addr learned my addr 172.21.15.38:0/1141932187 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcade0c6640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcad8099a70 0x7fcad80c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcadd8c5640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcad806cc10 0x7fcad80ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcadd0c4640 1 -- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcad806cc10 msgr2=0x7fcad80ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcadd0c4640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcad806cc10 0x7fcad80ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcadd0c4640 1 -- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcad8099a70 msgr2=0x7fcad80c6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcadd0c4640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcad8099a70 0x7fcad80c6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcadd0c4640 1 -- 172.21.15.38:0/1141932187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad81f5790 con 0x7fcad806cff0 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcadd8c5640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcad806cc10 0x7fcad80ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:13.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcadd0c4640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcad806cff0 0x7fcad81f5250 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00d800 tx=0x7fcacc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcad67fc640 1 -- 172.21.15.38:0/1141932187 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc004030 con 0x7fcad806cff0 2024-04-25T08:35:13.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcad67fc640 1 -- 172.21.15.38:0/1141932187 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc0041f0 con 0x7fcad806cff0 2024-04-25T08:35:13.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcad67fc640 1 -- 172.21.15.38:0/1141932187 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0096a0 con 0x7fcad806cff0 2024-04-25T08:35:13.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcadfb29640 1 -- 172.21.15.38:0/1141932187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad80c5230 con 0x7fcad806cff0 2024-04-25T08:35:13.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.577+0000 7fcadfb29640 1 -- 172.21.15.38:0/1141932187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad80c57f0 con 0x7fcad806cff0 2024-04-25T08:35:13.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.578+0000 7fcadfb29640 1 -- 172.21.15.38:0/1141932187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad8068e80 con 0x7fcad806cff0 2024-04-25T08:35:13.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.579+0000 7fcad67fc640 1 -- 172.21.15.38:0/1141932187 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcacc009840 con 0x7fcad806cff0 2024-04-25T08:35:13.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.579+0000 7fcad67fc640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcac406b4d0 0x7fcac406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:13.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.579+0000 7fcad67fc640 1 -- 172.21.15.38:0/1141932187 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcacc08a5a0 con 0x7fcad806cff0 2024-04-25T08:35:13.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.579+0000 7fcadd8c5640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcac406b4d0 0x7fcac406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:13.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.580+0000 7fcadd8c5640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcac406b4d0 0x7fcac406d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcad0034000 tx=0x7fcad002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:13.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.583+0000 7fcad67fc640 1 -- 172.21.15.38:0/1141932187 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcacc059290 con 0x7fcad806cff0 2024-04-25T08:35:13.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.801+0000 7fcadfb29640 1 -- 172.21.15.38:0/1141932187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcad80cba30 con 0x7fcad806cff0 2024-04-25T08:35:13.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.801+0000 7fcad67fc640 1 -- 172.21.15.38:0/1141932187 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fcacc058b90 con 0x7fcad806cff0 2024-04-25T08:35:13.796 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:13.796 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:35:12.877254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:13.797 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-04-25T08:35:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.803+0000 7fcadfb29640 1 -- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcac406b4d0 msgr2=0x7fcac406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.803+0000 7fcadfb29640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcac406b4d0 0x7fcac406d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcad0034000 tx=0x7fcad002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.803+0000 7fcadfb29640 1 -- 172.21.15.38:0/1141932187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcad806cff0 msgr2=0x7fcad81f5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.803+0000 7fcadfb29640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcad806cff0 0x7fcad81f5250 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00d800 tx=0x7fcacc00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.803+0000 7fcadfb29640 1 -- 172.21.15.38:0/1141932187 shutdown_connections 2024-04-25T08:35:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.803+0000 7fcadfb29640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcad8099a70 0x7fcad80c6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.803+0000 7fcadfb29640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcad806cff0 0x7fcad81f5250 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.803+0000 7fcadfb29640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcac406b4d0 0x7fcac406d990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.803+0000 7fcadfb29640 1 --2- 172.21.15.38:0/1141932187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcad806cc10 0x7fcad80ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.803+0000 7fcadfb29640 1 -- 172.21.15.38:0/1141932187 >> 172.21.15.38:0/1141932187 conn(0x7fcad80b31f0 msgr2=0x7fcad80b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.803+0000 7fcadfb29640 1 -- 172.21.15.38:0/1141932187 shutdown_connections 2024-04-25T08:35:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:13.803+0000 7fcadfb29640 1 -- 172.21.15.38:0/1141932187 wait complete. 2024-04-25T08:35:13.809 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T08:35:13.916 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:13.916 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:35:13.916 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:13.925 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:35:13.925 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:13.925 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T08:35:13.925 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T08:35:13.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:13.926 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T08:35:13.941 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-04-25T08:35:13.941 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> import json 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> import sys 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> try: 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> except OSError as e: 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> sys.exit(e.errno) 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> print(json.dumps( 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> indent=2)) 2024-04-25T08:35:13.942 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:35:14.050 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:14.050 INFO:teuthology.orchestra.run.smithi038.stdout: "st_mode": 16877, 2024-04-25T08:35:14.050 INFO:teuthology.orchestra.run.smithi038.stdout: "st_ino": 1099511627777, 2024-04-25T08:35:14.050 INFO:teuthology.orchestra.run.smithi038.stdout: "st_dev": 43, 2024-04-25T08:35:14.050 INFO:teuthology.orchestra.run.smithi038.stdout: "st_nlink": 16, 2024-04-25T08:35:14.050 INFO:teuthology.orchestra.run.smithi038.stdout: "st_uid": 1000, 2024-04-25T08:35:14.050 INFO:teuthology.orchestra.run.smithi038.stdout: "st_gid": 1316, 2024-04-25T08:35:14.051 INFO:teuthology.orchestra.run.smithi038.stdout: "st_size": 9362323, 2024-04-25T08:35:14.051 INFO:teuthology.orchestra.run.smithi038.stdout: "st_atime": 1714034095.6913378, 2024-04-25T08:35:14.051 INFO:teuthology.orchestra.run.smithi038.stdout: "st_mtime": 1714034106.437428, 2024-04-25T08:35:14.051 INFO:teuthology.orchestra.run.smithi038.stdout: "st_ctime": 1714034106.437428 2024-04-25T08:35:14.051 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:14.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:35:14.059 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-25T08:35:14.059 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T08:35:14.059 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:14.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.205+0000 7fe3b1a88640 1 Processor -- start 2024-04-25T08:35:14.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.205+0000 7fe3b1a88640 1 -- start start 2024-04-25T08:35:14.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3b1a88640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3ac144760 0x7fe3ac15faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3b1a88640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe3ac160030 0x7fe3ac0ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3b1a88640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3ac0b50c0 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3b1a88640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3ac15f380 con 0x7fe3ac160030 2024-04-25T08:35:14.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3b1a88640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac15f660 con 0x7fe3ac144760 2024-04-25T08:35:14.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3aaffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe3ac160030 0x7fe3ac0ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3aaffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe3ac160030 0x7fe3ac0ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43956/0 (socket says 172.21.15.38:43956) 2024-04-25T08:35:14.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3aaffd640 1 -- 172.21.15.38:0/2929858909 learned_addr learned my addr 172.21.15.38:0/2929858909 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3ab7fe640 1 --2- 172.21.15.38:0/2929858909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3ac144760 0x7fe3ac15faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3aaffd640 1 -- 172.21.15.38:0/2929858909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3ac144760 msgr2=0x7fe3ac15faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3aaffd640 1 --2- 172.21.15.38:0/2929858909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3ac144760 0x7fe3ac15faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3aaffd640 1 -- 172.21.15.38:0/2929858909 >> v1:172.21.15.107:6789/0 conn(0x7fe3ac0b4b50 legacy=0x7fe3ac0b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.206+0000 7fe3aaffd640 1 -- 172.21.15.38:0/2929858909 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac0b53a0 con 0x7fe3ac160030 2024-04-25T08:35:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.207+0000 7fe3aaffd640 1 --2- 172.21.15.38:0/2929858909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3ac160030 0x7fe3ac0ce360 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009870 tx=0x7fe3a002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e7a5206c76b67b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.207+0000 7fe3aa7fc640 1 -- 172.21.15.38:0/2929858909 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0037070 con 0x7fe3ac160030 2024-04-25T08:35:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.207+0000 7fe3aa7fc640 1 -- 172.21.15.38:0/2929858909 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a0032440 con 0x7fe3ac160030 2024-04-25T08:35:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.207+0000 7fe3aa7fc640 1 -- 172.21.15.38:0/2929858909 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0032810 con 0x7fe3ac160030 2024-04-25T08:35:14.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.208+0000 7fe3b1a88640 1 -- 172.21.15.38:0/2929858909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3ac160030 msgr2=0x7fe3ac0ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.208+0000 7fe3b1a88640 1 --2- 172.21.15.38:0/2929858909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3ac160030 0x7fe3ac0ce360 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009870 tx=0x7fe3a002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.208+0000 7fe3b1a88640 1 -- 172.21.15.38:0/2929858909 shutdown_connections 2024-04-25T08:35:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.208+0000 7fe3b1a88640 1 --2- 172.21.15.38:0/2929858909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3ac160030 0x7fe3ac0ce360 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.208+0000 7fe3b1a88640 1 --2- 172.21.15.38:0/2929858909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3ac144760 0x7fe3ac15faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.208+0000 7fe3b1a88640 1 -- 172.21.15.38:0/2929858909 >> 172.21.15.38:0/2929858909 conn(0x7fe3ac06b210 msgr2=0x7fe3ac06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.208+0000 7fe3b1a88640 1 -- 172.21.15.38:0/2929858909 shutdown_connections 2024-04-25T08:35:14.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.208+0000 7fe3b1a88640 1 -- 172.21.15.38:0/2929858909 wait complete. 2024-04-25T08:35:14.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.209+0000 7fe3b1a88640 1 Processor -- start 2024-04-25T08:35:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.209+0000 7fe3b1a88640 1 -- start start 2024-04-25T08:35:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3b1a88640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3ac0b4b50 0x7fe3ac0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3b1a88640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac160030 0x7fe3ac15d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3b1a88640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3ac1422e0 0x7fe3ac15ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3b1a88640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac145400 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3b1a88640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac145280 con 0x7fe3ac160030 2024-04-25T08:35:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3b1a88640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac0b3f20 con 0x7fe3ac1422e0 2024-04-25T08:35:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3ac0b4b50 0x7fe3ac0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3ac0b4b50 0x7fe3ac0ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43964/0 (socket says 172.21.15.38:43964) 2024-04-25T08:35:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3ab7fe640 1 -- 172.21.15.38:0/1308408089 learned_addr learned my addr 172.21.15.38:0/1308408089 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3aaffd640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac160030 0x7fe3ac15d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3abfff640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3ac1422e0 0x7fe3ac15ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3ab7fe640 1 -- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3ac1422e0 msgr2=0x7fe3ac15ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3ab7fe640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3ac1422e0 0x7fe3ac15ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3ab7fe640 1 -- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac160030 msgr2=0x7fe3ac15d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3ab7fe640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac160030 0x7fe3ac15d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3ab7fe640 1 -- 172.21.15.38:0/1308408089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac142d30 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3aaffd640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac160030 0x7fe3ac15d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.210+0000 7fe3ab7fe640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3ac0b4b50 0x7fe3ac0ccd40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00d830 tx=0x7fe39c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.211+0000 7fe393fff640 1 -- 172.21.15.38:0/1308408089 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00b6c0 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.211+0000 7fe393fff640 1 -- 172.21.15.38:0/1308408089 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39c019070 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.211+0000 7fe393fff640 1 -- 172.21.15.38:0/1308408089 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00bac0 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.211+0000 7fe3b1a88640 1 -- 172.21.15.38:0/1308408089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ac0cc260 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.211+0000 7fe3b1a88640 1 -- 172.21.15.38:0/1308408089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ac0cc820 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.212+0000 7fe3b1a88640 1 -- 172.21.15.38:0/1308408089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ac068e80 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.212+0000 7fe393fff640 1 -- 172.21.15.38:0/1308408089 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe39c01e460 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.213+0000 7fe393fff640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe39806b4d0 0x7fe39806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.213+0000 7fe393fff640 1 -- 172.21.15.38:0/1308408089 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe39c08aaf0 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.213+0000 7fe3aaffd640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe39806b4d0 0x7fe39806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.213+0000 7fe3aaffd640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe39806b4d0 0x7fe39806d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0034000 tx=0x7fe3a002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.217+0000 7fe393fff640 1 -- 172.21.15.38:0/1308408089 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe39c0597e0 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.434+0000 7fe3b1a88640 1 -- 172.21.15.38:0/1308408089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3ac06df60 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.434+0000 7fe393fff640 1 -- 172.21.15.38:0/1308408089 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fe39c0590e0 con 0x7fe3ac0b4b50 2024-04-25T08:35:14.429 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:14.429 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:35:12.877254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:14.430 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-04-25T08:35:14.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.436+0000 7fe3b1a88640 1 -- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe39806b4d0 msgr2=0x7fe39806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.436+0000 7fe3b1a88640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe39806b4d0 0x7fe39806d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0034000 tx=0x7fe3a002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:14.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.436+0000 7fe3b1a88640 1 -- 172.21.15.38:0/1308408089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3ac0b4b50 msgr2=0x7fe3ac0ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.436+0000 7fe3b1a88640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3ac0b4b50 0x7fe3ac0ccd40 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00d830 tx=0x7fe39c00dcf0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.436+0000 7fe3b1a88640 1 -- 172.21.15.38:0/1308408089 shutdown_connections 2024-04-25T08:35:14.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.436+0000 7fe3b1a88640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe39806b4d0 0x7fe39806d990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.436+0000 7fe3b1a88640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3ac1422e0 0x7fe3ac15ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.436+0000 7fe3b1a88640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3ac160030 0x7fe3ac15d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.436+0000 7fe3b1a88640 1 --2- 172.21.15.38:0/1308408089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3ac0b4b50 0x7fe3ac0ccd40 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.436+0000 7fe3b1a88640 1 -- 172.21.15.38:0/1308408089 >> 172.21.15.38:0/1308408089 conn(0x7fe3ac06b210 msgr2=0x7fe3ac0b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.436+0000 7fe3b1a88640 1 -- 172.21.15.38:0/1308408089 shutdown_connections 2024-04-25T08:35:14.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.436+0000 7fe3b1a88640 1 -- 172.21.15.38:0/1308408089 wait complete. 2024-04-25T08:35:14.442 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T08:35:14.555 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:14.555 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": -2 2024-04-25T08:35:14.555 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:14.563 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T08:35:14.563 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T08:35:14.564 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:14.564 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T08:35:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.672+0000 7f8244ae1640 1 Processor -- start 2024-04-25T08:35:14.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.672+0000 7f8244ae1640 1 -- start start 2024-04-25T08:35:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.673+0000 7f8244ae1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f824015df30 0x7f824009a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.673+0000 7f8244ae1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f824009a180 0x7f82400cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.673+0000 7f8244ae1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8240162820 con 0x7f8240162390 2024-04-25T08:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.673+0000 7f8244ae1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824015e310 con 0x7f824009a180 2024-04-25T08:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.673+0000 7f8244ae1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824009aeb0 con 0x7f824015df30 2024-04-25T08:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.674+0000 7f823ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f824009a180 0x7f82400cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.674+0000 7f823dd9b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f8240162390 0x7f8240099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56674/0 (socket says 172.21.15.38:56674) 2024-04-25T08:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.674+0000 7f823ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f824009a180 0x7f82400cf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43968/0 (socket says 172.21.15.38:43968) 2024-04-25T08:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.674+0000 7f823dd9b640 1 -- 172.21.15.38:0/1806949961 learned_addr learned my addr 172.21.15.38:0/1806949961 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.674+0000 7f823e59c640 1 --2- 172.21.15.38:0/1806949961 >> v2:172.21.15.107:3300/0 conn(0x7f824015df30 0x7f824009a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.674+0000 7f823ed9d640 1 -- 172.21.15.38:0/1806949961 >> v2:172.21.15.107:3300/0 conn(0x7f824015df30 msgr2=0x7f824009a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.674+0000 7f823ed9d640 1 --2- 172.21.15.38:0/1806949961 >> v2:172.21.15.107:3300/0 conn(0x7f824015df30 0x7f824009a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.674+0000 7f823ed9d640 1 -- 172.21.15.38:0/1806949961 >> v1:172.21.15.38:6789/0 conn(0x7f8240162390 legacy=0x7f8240099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.674+0000 7f823d59a640 1 -- 172.21.15.38:0/1806949961 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999756487 0 0) 0x7f8234001110 con 0x7f8240162390 2024-04-25T08:35:14.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.674+0000 7f823ed9d640 1 -- 172.21.15.38:0/1806949961 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f824015f300 con 0x7f824009a180 2024-04-25T08:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.675+0000 7f823ed9d640 1 --2- 172.21.15.38:0/1806949961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f824009a180 0x7f82400cf8f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8238009d90 tx=0x7f823802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b594bab7988ef5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.675+0000 7f823d59a640 1 -- 172.21.15.38:0/1806949961 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238007740 con 0x7f824009a180 2024-04-25T08:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.675+0000 7f823d59a640 1 -- 172.21.15.38:0/1806949961 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82380078e0 con 0x7f824009a180 2024-04-25T08:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.675+0000 7f823d59a640 1 -- 172.21.15.38:0/1806949961 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238007cb0 con 0x7f824009a180 2024-04-25T08:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.675+0000 7f8244ae1640 1 -- 172.21.15.38:0/1806949961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f824009a180 msgr2=0x7f82400cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.675+0000 7f8244ae1640 1 --2- 172.21.15.38:0/1806949961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f824009a180 0x7f82400cf8f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f8238009d90 tx=0x7f823802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.675+0000 7f8244ae1640 1 -- 172.21.15.38:0/1806949961 shutdown_connections 2024-04-25T08:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.675+0000 7f8244ae1640 1 --2- 172.21.15.38:0/1806949961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f824009a180 0x7f82400cf8f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.675+0000 7f8244ae1640 1 --2- 172.21.15.38:0/1806949961 >> v2:172.21.15.107:3300/0 conn(0x7f824015df30 0x7f824009a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.675+0000 7f8244ae1640 1 -- 172.21.15.38:0/1806949961 >> 172.21.15.38:0/1806949961 conn(0x7f82400b3110 msgr2=0x7f82400b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.675+0000 7f8244ae1640 1 -- 172.21.15.38:0/1806949961 shutdown_connections 2024-04-25T08:35:14.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.675+0000 7f8244ae1640 1 -- 172.21.15.38:0/1806949961 wait complete. 2024-04-25T08:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.677+0000 7f8244ae1640 1 Processor -- start 2024-04-25T08:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.677+0000 7f8244ae1640 1 -- start start 2024-04-25T08:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.677+0000 7f8244ae1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824009a180 0x7f82400c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.677+0000 7f8244ae1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f824015df30 0x7f82400c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.677+0000 7f8244ae1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8240162390 0x7f82400ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.677+0000 7f8244ae1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824015e820 con 0x7f8240162390 2024-04-25T08:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.677+0000 7f8244ae1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824015ee70 con 0x7f824009a180 2024-04-25T08:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.677+0000 7f8244ae1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824015eb00 con 0x7f824015df30 2024-04-25T08:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f823ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8240162390 0x7f82400ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f823ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8240162390 0x7f82400ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43970/0 (socket says 172.21.15.38:43970) 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f823ed9d640 1 -- 172.21.15.38:0/4113836720 learned_addr learned my addr 172.21.15.38:0/4113836720 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f823dd9b640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f824015df30 0x7f82400c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f823e59c640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824009a180 0x7f82400c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f823ed9d640 1 -- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f824015df30 msgr2=0x7f82400c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f823ed9d640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f824015df30 0x7f82400c9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f823ed9d640 1 -- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824009a180 msgr2=0x7f82400c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f823ed9d640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824009a180 0x7f82400c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f823ed9d640 1 -- 172.21.15.38:0/4113836720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82400ca2c0 con 0x7f8240162390 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f823ed9d640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8240162390 0x7f82400ccca0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8238009eb0 tx=0x7f8238004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f822effd640 1 -- 172.21.15.38:0/4113836720 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823803f070 con 0x7f8240162390 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f822effd640 1 -- 172.21.15.38:0/4113836720 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8238031bc0 con 0x7f8240162390 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f822effd640 1 -- 172.21.15.38:0/4113836720 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238044460 con 0x7f8240162390 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f8244ae1640 1 -- 172.21.15.38:0/4113836720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82400ca5a0 con 0x7f8240162390 2024-04-25T08:35:14.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.678+0000 7f8244ae1640 1 -- 172.21.15.38:0/4113836720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82400cab60 con 0x7f8240162390 2024-04-25T08:35:14.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.680+0000 7f822effd640 1 -- 172.21.15.38:0/4113836720 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8238007790 con 0x7f8240162390 2024-04-25T08:35:14.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.680+0000 7f822effd640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f822006b4d0 0x7f822006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.680+0000 7f822effd640 1 -- 172.21.15.38:0/4113836720 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f82380ab9e0 con 0x7f8240162390 2024-04-25T08:35:14.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.681+0000 7f823e59c640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f822006b4d0 0x7f822006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.681+0000 7f823e59c640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f822006b4d0 0x7f822006d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8230006fd0 tx=0x7f8230008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.681+0000 7f82477e9740 1 Processor -- start 2024-04-25T08:35:14.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.681+0000 7f82477e9740 1 -- start start 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.682+0000 7f82477e9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e666b0 0x562d74e66a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.682+0000 7f82477e9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e844a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.682+0000 7f82477e9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e3c390 0x562d74e82610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.682+0000 7f82477e9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d74d9ec60 con 0x562d74e3c390 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.682+0000 7f82477e9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d74d47ee0 con 0x562d74e666b0 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.682+0000 7f82477e9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d74d9ad00 con 0x562d74e83f50 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.682+0000 7f823ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e3c390 0x562d74e82610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.682+0000 7f823ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e3c390 0x562d74e82610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43976/0 (socket says 172.21.15.38:43976) 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.682+0000 7f823ed9d640 1 -- 172.21.15.38:0/870070634 learned_addr learned my addr 172.21.15.38:0/870070634 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.682+0000 7f823e59c640 1 --2- 172.21.15.38:0/870070634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e666b0 0x562d74e66a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.682+0000 7f823dd9b640 1 --2- 172.21.15.38:0/870070634 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e844a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.682+0000 7f823ed9d640 1 -- 172.21.15.38:0/870070634 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 msgr2=0x562d74e844a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f823ed9d640 1 --2- 172.21.15.38:0/870070634 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e844a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f823ed9d640 1 -- 172.21.15.38:0/870070634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e666b0 msgr2=0x562d74e66a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f823ed9d640 1 --2- 172.21.15.38:0/870070634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e666b0 0x562d74e66a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f823ed9d640 1 -- 172.21.15.38:0/870070634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d74b7f9f0 con 0x562d74e3c390 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f823e59c640 1 --2- 172.21.15.38:0/870070634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e666b0 0x562d74e66a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f823ed9d640 1 --2- 172.21.15.38:0/870070634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e3c390 0x562d74e82610 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8238008000 tx=0x7f82380763b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13cf08b8ab84187 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f821ffff640 1 -- 172.21.15.38:0/870070634 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238082770 con 0x562d74e3c390 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f821ffff640 1 -- 172.21.15.38:0/870070634 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8238078410 con 0x562d74e3c390 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f821ffff640 1 -- 172.21.15.38:0/870070634 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823807a370 con 0x562d74e3c390 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f82477e9740 1 -- 172.21.15.38:0/870070634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e3c390 msgr2=0x562d74e82610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f82477e9740 1 --2- 172.21.15.38:0/870070634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e3c390 0x562d74e82610 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8238008000 tx=0x7f82380763b0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f82477e9740 1 -- 172.21.15.38:0/870070634 shutdown_connections 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f82477e9740 1 --2- 172.21.15.38:0/870070634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e3c390 0x562d74e82610 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f82477e9740 1 --2- 172.21.15.38:0/870070634 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e844a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f82477e9740 1 --2- 172.21.15.38:0/870070634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e666b0 0x562d74e66a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.683+0000 7f82477e9740 1 -- 172.21.15.38:0/870070634 >> 172.21.15.38:0/870070634 conn(0x562d74e38cc0 msgr2=0x562d74e814c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.684+0000 7f82477e9740 1 -- 172.21.15.38:0/870070634 shutdown_connections 2024-04-25T08:35:14.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.684+0000 7f82477e9740 1 -- 172.21.15.38:0/870070634 wait complete. 2024-04-25T08:35:14.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.684+0000 7f82477e9740 1 -- start start 2024-04-25T08:35:14.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.684+0000 7f82477e9740 1 Processor -- start 2024-04-25T08:35:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f82477e9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 0x562d74e7a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f82477e9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e69d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f82477e9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e95780 0x562d74e35920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f82477e9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d74d9ec60 con 0x562d74e666b0 2024-04-25T08:35:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f82477e9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d74d5b470 con 0x562d74e95780 2024-04-25T08:35:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f82477e9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d74e3d130 con 0x562d74e83f50 2024-04-25T08:35:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f82477e9740 10 client.0 _ll_register_callbacks cb 0x562d74d277d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f823e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 0x562d74e7a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f823e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 0x562d74e7a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43984/0 (socket says 172.21.15.38:43984) 2024-04-25T08:35:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f823e59c640 1 -- 172.21.15.38:0/4204730515 learned_addr learned my addr 172.21.15.38:0/4204730515 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f823ed9d640 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e95780 0x562d74e35920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f823dd9b640 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e69d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f823e59c640 1 -- 172.21.15.38:0/4204730515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 msgr2=0x562d74e69d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f823e59c640 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e69d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f823e59c640 1 -- 172.21.15.38:0/4204730515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e95780 msgr2=0x562d74e35920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f823e59c640 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e95780 0x562d74e35920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f823e59c640 1 -- 172.21.15.38:0/4204730515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d74a732e0 con 0x562d74e666b0 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.685+0000 7f823dd9b640 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e69d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f823e59c640 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 0x562d74e7a820 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8230016a00 tx=0x7f8230004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e74101211f140a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f821d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f821d7fa640 1 -- 172.21.15.38:0/4204730515 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230010e60 con 0x562d74e666b0 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f821d7fa640 1 -- 172.21.15.38:0/4204730515 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8230014bc0 con 0x562d74e666b0 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f82477e9740 1 -- 172.21.15.38:0/4204730515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8238078410 con 0x562d74e666b0 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f821d7fa640 1 -- 172.21.15.38:0/4204730515 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f823002f050 con 0x562d74e666b0 2024-04-25T08:35:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f82477e9740 10 client.6423 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f82477e9740 1 -- 172.21.15.38:0/4204730515 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x562d74be0180 con 0x562d74e666b0 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f821d7fa640 1 -- 172.21.15.38:0/4204730515 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8238078410 con 0x562d74e666b0 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f821d7fa640 1 -- 172.21.15.38:0/4204730515 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8230032020 con 0x562d74e666b0 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f82477e9740 10 client.6423 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f82477e9740 10 client.6423 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f82477e9740 20 client.6423 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f82477e9740 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x562d74e78570 0x562d74e789b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f82477e9740 4 client.6423 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f82477e9740 1 -- 172.21.15.38:0/4204730515 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d74dac1a0 con 0x562d74e78570 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.686+0000 7f823dd9b640 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x562d74e78570 0x562d74e789b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f821d7fa640 10 client.6423 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f823dd9b640 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x562d74e78570 0x562d74e789b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d5b8d2eb5b6f0e3 server_cookie=3b75a2e0e25af854 in_seq=0 out_seq=0 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f821d7fa640 1 -- 172.21.15.38:0/4204730515 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d74dac1a0 con 0x562d74e78570 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f821d7fa640 10 client.6423 handle_command_reply: tid=0 2024-04-25T08:35:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f82477e9740 1 client.6423 shutdown 2024-04-25T08:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f82477e9740 1 -- 172.21.15.38:0/4204730515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 msgr2=0x562d74e7a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f82477e9740 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 0x562d74e7a820 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8230016a00 tx=0x7f8230004730 comp rx=0 tx=0).stop 2024-04-25T08:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f82477e9740 1 -- 172.21.15.38:0/4204730515 shutdown_connections 2024-04-25T08:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f82477e9740 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x562d74e78570 0x562d74e789b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f82477e9740 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e95780 0x562d74e35920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f82477e9740 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e69d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f82477e9740 1 --2- 172.21.15.38:0/4204730515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 0x562d74e7a820 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f821d7fa640 0 client.6423 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.687+0000 7f82477e9740 1 -- 172.21.15.38:0/4204730515 >> 172.21.15.38:0/4204730515 conn(0x562d74e3c390 msgr2=0x562d74e84a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.688+0000 7f82477e9740 1 -- 172.21.15.38:0/4204730515 shutdown_connections 2024-04-25T08:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.688+0000 7f82477e9740 1 -- 172.21.15.38:0/4204730515 wait complete. 2024-04-25T08:35:14.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.688+0000 7f82477e9740 20 client.6423 trim_cache size 0 max 16384 2024-04-25T08:35:14.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.703+0000 7f82477e9740 1 Processor -- start 2024-04-25T08:35:14.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.703+0000 7f82477e9740 1 -- start start 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f82477e9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 0x562d74e3d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f82477e9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e3a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f82477e9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e95780 0x562d74e56350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f82477e9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8240200150 con 0x562d74e666b0 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f82477e9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82401ff350 con 0x562d74e95780 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f82477e9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82401fe750 con 0x562d74e83f50 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f823e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 0x562d74e3d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f823e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 0x562d74e3d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43994/0 (socket says 172.21.15.38:43994) 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f823e59c640 1 -- 172.21.15.38:0/4238724219 learned_addr learned my addr 172.21.15.38:0/4238724219 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f823ed9d640 1 --2- 172.21.15.38:0/4238724219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e95780 0x562d74e56350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f823e59c640 1 -- 172.21.15.38:0/4238724219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 msgr2=0x562d74e3a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f823dd9b640 1 --2- 172.21.15.38:0/4238724219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e3a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f823e59c640 1 --2- 172.21.15.38:0/4238724219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e3a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f823e59c640 1 -- 172.21.15.38:0/4238724219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e95780 msgr2=0x562d74e56350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f823e59c640 1 --2- 172.21.15.38:0/4238724219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e95780 0x562d74e56350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f823e59c640 1 -- 172.21.15.38:0/4238724219 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d74e35c90 con 0x562d74e666b0 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f823dd9b640 1 --2- 172.21.15.38:0/4238724219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e3a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.704+0000 7f823e59c640 1 --2- 172.21.15.38:0/4238724219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 0x562d74e3d760 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8230007860 tx=0x7f8230020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7103c37c9820f3a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.705+0000 7f821f7fe640 1 -- 172.21.15.38:0/4238724219 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230038070 con 0x562d74e666b0 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.705+0000 7f821f7fe640 1 -- 172.21.15.38:0/4238724219 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8230027070 con 0x562d74e666b0 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.705+0000 7f821f7fe640 1 -- 172.21.15.38:0/4238724219 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8230014800 con 0x562d74e666b0 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.705+0000 7f82477e9740 1 -- 172.21.15.38:0/4238724219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 msgr2=0x562d74e3d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.705+0000 7f82477e9740 1 --2- 172.21.15.38:0/4238724219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 0x562d74e3d760 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8230007860 tx=0x7f8230020000 comp rx=0 tx=0).stop 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.705+0000 7f82477e9740 1 -- 172.21.15.38:0/4238724219 shutdown_connections 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.705+0000 7f82477e9740 1 --2- 172.21.15.38:0/4238724219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e95780 0x562d74e56350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.705+0000 7f82477e9740 1 --2- 172.21.15.38:0/4238724219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e3a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.705+0000 7f82477e9740 1 --2- 172.21.15.38:0/4238724219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e666b0 0x562d74e3d760 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.705+0000 7f82477e9740 1 -- 172.21.15.38:0/4238724219 >> 172.21.15.38:0/4238724219 conn(0x562d74e3c390 msgr2=0x562d74e793e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.705+0000 7f82477e9740 1 -- 172.21.15.38:0/4238724219 shutdown_connections 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.705+0000 7f82477e9740 1 -- 172.21.15.38:0/4238724219 wait complete. 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.706+0000 7f82477e9740 1 -- start start 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.706+0000 7f82477e9740 1 Processor -- start 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f82477e9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e666b0 0x562d74eeb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f82477e9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e7a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f82477e9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e95780 0x562d74e7ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f82477e9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82401fed50 con 0x562d74e95780 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f82477e9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d74e3dca0 con 0x562d74e666b0 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f82477e9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d74eec020 con 0x562d74e83f50 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f82477e9740 10 client.0 _ll_register_callbacks cb 0x562d74e370d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f823ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e95780 0x562d74e7ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f823ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e95780 0x562d74e7ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43998/0 (socket says 172.21.15.38:43998) 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f823ed9d640 1 -- 172.21.15.38:0/792826443 learned_addr learned my addr 172.21.15.38:0/792826443 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f823e59c640 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e666b0 0x562d74eeb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f823dd9b640 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e7a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f823ed9d640 1 -- 172.21.15.38:0/792826443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 msgr2=0x562d74e7a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f823ed9d640 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e7a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f823ed9d640 1 -- 172.21.15.38:0/792826443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e666b0 msgr2=0x562d74eeb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f823ed9d640 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e666b0 0x562d74eeb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f823ed9d640 1 -- 172.21.15.38:0/792826443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d74a4c3a0 con 0x562d74e95780 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f823ed9d640 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e95780 0x562d74e7ab70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f823807a0a0 tx=0x7f8238046070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8b0ff875618375 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f821d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f821d7fa640 1 -- 172.21.15.38:0/792826443 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238082770 con 0x562d74e95780 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f821d7fa640 1 -- 172.21.15.38:0/792826443 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8238097050 con 0x562d74e95780 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f82477e9740 1 -- 172.21.15.38:0/792826443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8230027070 con 0x562d74e95780 2024-04-25T08:35:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f821d7fa640 1 -- 172.21.15.38:0/792826443 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82380a3050 con 0x562d74e95780 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f82477e9740 10 client.6429 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.707+0000 7f82477e9740 1 -- 172.21.15.38:0/792826443 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x562d74eebe20 con 0x562d74e95780 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f821d7fa640 1 -- 172.21.15.38:0/792826443 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8230027070 con 0x562d74e95780 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f821d7fa640 1 -- 172.21.15.38:0/792826443 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f823807a370 con 0x562d74e95780 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f82477e9740 10 client.6429 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f82477e9740 10 client.6429 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f82477e9740 20 client.6429 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f82477e9740 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x562d74e789b0 0x562d74e7eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f82477e9740 4 client.6429 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f82477e9740 1 -- 172.21.15.38:0/792826443 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x562d74da64f0 con 0x562d74e789b0 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f823e59c640 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x562d74e789b0 0x562d74e7eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f823e59c640 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x562d74e789b0 0x562d74e7eed0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aef2fe92ea8b347 server_cookie=cca2762f497398fa in_seq=0 out_seq=0 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f821d7fa640 10 client.6429 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:14.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f821d7fa640 1 -- 172.21.15.38:0/792826443 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x562d74da64f0 con 0x562d74e789b0 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f821d7fa640 10 client.6429 handle_command_reply: tid=0 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f82477e9740 1 client.6429 shutdown 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f82477e9740 1 -- 172.21.15.38:0/792826443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e95780 msgr2=0x562d74e7ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.708+0000 7f82477e9740 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e95780 0x562d74e7ab70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f823807a0a0 tx=0x7f8238046070 comp rx=0 tx=0).stop 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f82477e9740 1 -- 172.21.15.38:0/792826443 shutdown_connections 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f82477e9740 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x562d74e789b0 0x562d74e7eed0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f82477e9740 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d74e95780 0x562d74e7ab70 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f82477e9740 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d74e83f50 0x562d74e7a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f821d7fa640 0 client.6429 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f82477e9740 1 --2- 172.21.15.38:0/792826443 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d74e666b0 0x562d74eeb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f82477e9740 1 -- 172.21.15.38:0/792826443 >> 172.21.15.38:0/792826443 conn(0x562d74e3c390 msgr2=0x562d74e57580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f82477e9740 1 -- 172.21.15.38:0/792826443 shutdown_connections 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f82477e9740 1 -- 172.21.15.38:0/792826443 wait complete. 2024-04-25T08:35:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f82477e9740 20 client.6429 trim_cache size 0 max 16384 2024-04-25T08:35:14.711 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:14.711 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": -2 2024-04-25T08:35:14.711 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f8244ae1640 1 -- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f822006b4d0 msgr2=0x7f822006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f8244ae1640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f822006b4d0 0x7f822006d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8230006fd0 tx=0x7f8230008040 comp rx=0 tx=0).stop 2024-04-25T08:35:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f8244ae1640 1 -- 172.21.15.38:0/4113836720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8240162390 msgr2=0x7f82400ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f8244ae1640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8240162390 0x7f82400ccca0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8238009eb0 tx=0x7f8238004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f8244ae1640 1 -- 172.21.15.38:0/4113836720 shutdown_connections 2024-04-25T08:35:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f8244ae1640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8240162390 0x7f82400ccca0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f8244ae1640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f822006b4d0 0x7f822006d990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f8244ae1640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f824015df30 0x7f82400c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f8244ae1640 1 --2- 172.21.15.38:0/4113836720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f824009a180 0x7f82400c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f8244ae1640 1 -- 172.21.15.38:0/4113836720 >> 172.21.15.38:0/4113836720 conn(0x7f82400b3110 msgr2=0x7f8240070780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f8244ae1640 1 -- 172.21.15.38:0/4113836720 shutdown_connections 2024-04-25T08:35:14.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.709+0000 7f8244ae1640 1 -- 172.21.15.38:0/4113836720 wait complete. 2024-04-25T08:35:14.715 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T08:35:14.715 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T08:35:14.715 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:14.715 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T08:35:14.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.824+0000 7f013fd09640 1 Processor -- start 2024-04-25T08:35:14.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.824+0000 7f013fd09640 1 -- start start 2024-04-25T08:35:14.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.824+0000 7f013fd09640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01380d33d0 0x7f01380d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013fd09640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01380d2330 con 0x7f01380d1e90 2024-04-25T08:35:14.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013fd09640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01380d2610 con 0x7f01380d0320 2024-04-25T08:35:14.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013fd09640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01380d28f0 con 0x7f01380d33d0 2024-04-25T08:35:14.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013e2a6640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f01380d1e90 0x7f01380b6980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56684/0 (socket says 172.21.15.38:56684) 2024-04-25T08:35:14.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013e2a6640 1 -- 172.21.15.38:0/12323426 learned_addr learned my addr 172.21.15.38:0/12323426 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013d2a4640 1 --2- 172.21.15.38:0/12323426 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01380d33d0 0x7f01380d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104334291 0 0) 0x7f01380d2330 con 0x7f01380d1e90 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0124003520 con 0x7f01380d1e90 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 874673343 0 0) 0x7f0124003520 con 0x7f01380d1e90 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01380d33d0 msgr2=0x7f01380d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 --2- 172.21.15.38:0/12323426 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01380d33d0 0x7f01380d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 >> v1:172.21.15.107:6789/0 conn(0x7f01380d0320 legacy=0x7f01380d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01380d2bd0 con 0x7f01380d1e90 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f0130001670 con 0x7f01380d1e90 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 >> v1:172.21.15.38:6789/0 conn(0x7f01380d1e90 legacy=0x7f01380b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 --2- 172.21.15.38:0/12323426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01240039e0 0x7f0124003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 --2- 172.21.15.38:0/12323426 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01380d33d0 0x7f0124004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 --2- 172.21.15.38:0/12323426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0124004d40 0x7f01240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01380d28f0 con 0x7f01240039e0 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012400b9c0 con 0x7f0124004d40 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.825+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012400bbc0 con 0x7f01380d33d0 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.826+0000 7f013daa5640 1 --2- 172.21.15.38:0/12323426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01240039e0 0x7f0124003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.826+0000 7f013e2a6640 1 --2- 172.21.15.38:0/12323426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0124004d40 0x7f01240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.826+0000 7f013d2a4640 1 --2- 172.21.15.38:0/12323426 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01380d33d0 0x7f0124004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.826+0000 7f013daa5640 1 -- 172.21.15.38:0/12323426 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01380d33d0 msgr2=0x7f0124004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.826+0000 7f013daa5640 1 --2- 172.21.15.38:0/12323426 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01380d33d0 0x7f0124004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.826+0000 7f013daa5640 1 -- 172.21.15.38:0/12323426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0124004d40 msgr2=0x7f01240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.826+0000 7f013daa5640 1 --2- 172.21.15.38:0/12323426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0124004d40 0x7f01240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.826+0000 7f013daa5640 1 -- 172.21.15.38:0/12323426 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0124003520 con 0x7f01240039e0 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.826+0000 7f013daa5640 1 --2- 172.21.15.38:0/12323426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01240039e0 0x7f0124003dc0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0134001f60 tx=0x7f0124005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3bbf8a864baa5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.827+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134037070 con 0x7f01240039e0 2024-04-25T08:35:14.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.827+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0134032440 con 0x7f01240039e0 2024-04-25T08:35:14.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.827+0000 7f013caa3640 1 -- 172.21.15.38:0/12323426 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134001b30 con 0x7f01240039e0 2024-04-25T08:35:14.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.827+0000 7f013fd09640 1 -- 172.21.15.38:0/12323426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01240039e0 msgr2=0x7f0124003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.827+0000 7f013fd09640 1 --2- 172.21.15.38:0/12323426 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01240039e0 0x7f0124003dc0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f0134001f60 tx=0x7f0124005830 comp rx=0 tx=0).stop 2024-04-25T08:35:14.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.827+0000 7f013e2a6640 1 -- 172.21.15.38:0/12323426 reap_dead start 2024-04-25T08:35:14.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.827+0000 7f013fd09640 1 -- 172.21.15.38:0/12323426 shutdown_connections 2024-04-25T08:35:14.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.827+0000 7f013fd09640 1 -- 172.21.15.38:0/12323426 >> 172.21.15.38:0/12323426 conn(0x7f01380668f0 msgr2=0x7f0138066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.827+0000 7f013fd09640 1 -- 172.21.15.38:0/12323426 shutdown_connections 2024-04-25T08:35:14.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.827+0000 7f013fd09640 1 -- 172.21.15.38:0/12323426 wait complete. 2024-04-25T08:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.828+0000 7f013fd09640 1 Processor -- start 2024-04-25T08:35:14.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.828+0000 7f013fd09640 1 -- start start 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.829+0000 7f013fd09640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240039e0 0x7f0138153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.829+0000 7f013fd09640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0138158030 0x7f01380b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.829+0000 7f013fd09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01380ccbb0 0x7f01380cd030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.829+0000 7f013fd09640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0138161390 con 0x7f01380ccbb0 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.829+0000 7f013fd09640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0138161210 con 0x7f01240039e0 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.829+0000 7f013fd09640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01380b9720 con 0x7f0138158030 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.829+0000 7f013e2a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01380ccbb0 0x7f01380cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.829+0000 7f013e2a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01380ccbb0 0x7f01380cd030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44012/0 (socket says 172.21.15.38:44012) 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.829+0000 7f013e2a6640 1 -- 172.21.15.38:0/3687709236 learned_addr learned my addr 172.21.15.38:0/3687709236 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.829+0000 7f013daa5640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240039e0 0x7f0138153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f013d2a4640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0138158030 0x7f01380b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f013e2a6640 1 -- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0138158030 msgr2=0x7f01380b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f013e2a6640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0138158030 0x7f01380b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f013e2a6640 1 -- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240039e0 msgr2=0x7f0138153eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f013e2a6640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240039e0 0x7f0138153eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f013e2a6640 1 -- 172.21.15.38:0/3687709236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01380b34c0 con 0x7f01380ccbb0 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f013d2a4640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0138158030 0x7f01380b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f013e2a6640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01380ccbb0 0x7f01380cd030 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0130000c00 tx=0x7f0130007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f012e7fc640 1 -- 172.21.15.38:0/3687709236 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130001640 con 0x7f01380ccbb0 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f012e7fc640 1 -- 172.21.15.38:0/3687709236 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0130018070 con 0x7f01380ccbb0 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f013fd09640 1 -- 172.21.15.38:0/3687709236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01380b37a0 con 0x7f01380ccbb0 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f012e7fc640 1 -- 172.21.15.38:0/3687709236 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013000fb10 con 0x7f01380ccbb0 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.830+0000 7f013fd09640 1 -- 172.21.15.38:0/3687709236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01380b3d60 con 0x7f01380ccbb0 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.831+0000 7f012e7fc640 1 -- 172.21.15.38:0/3687709236 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f013000fcb0 con 0x7f01380ccbb0 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.832+0000 7f012e7fc640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f011c06b4d0 0x7f011c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.832+0000 7f012e7fc640 1 -- 172.21.15.38:0/3687709236 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f013008a230 con 0x7f01380ccbb0 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.832+0000 7f013daa5640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f011c06b4d0 0x7f011c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.833+0000 7f013daa5640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f011c06b4d0 0x7f011c06d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0134034000 tx=0x7f0134009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.833+0000 7f0142a11740 1 Processor -- start 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.833+0000 7f0142a11740 1 -- start start 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f0142a11740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe80fff10 0x55afe8102070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f0142a11740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe813c5e0 0x55afe81002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f0142a11740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe812a9d0 0x55afe812ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f0142a11740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8031fe0 con 0x55afe80fff10 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f0142a11740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8063d20 con 0x55afe812a9d0 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f0142a11740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55afe800cfa0 con 0x55afe813c5e0 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f013daa5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe80fff10 0x55afe8102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f013daa5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe80fff10 0x55afe8102070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44014/0 (socket says 172.21.15.38:44014) 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f013daa5640 1 -- 172.21.15.38:0/695744573 learned_addr learned my addr 172.21.15.38:0/695744573 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f013e2a6640 1 --2- 172.21.15.38:0/695744573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe812a9d0 0x55afe812ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f013daa5640 1 -- 172.21.15.38:0/695744573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe813c5e0 msgr2=0x55afe81002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f013daa5640 1 --2- 172.21.15.38:0/695744573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe813c5e0 0x55afe81002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f013daa5640 1 -- 172.21.15.38:0/695744573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe812a9d0 msgr2=0x55afe812ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f013daa5640 1 --2- 172.21.15.38:0/695744573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe812a9d0 0x55afe812ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f013daa5640 1 -- 172.21.15.38:0/695744573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afe7e44b00 con 0x55afe80fff10 2024-04-25T08:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f013e2a6640 1 --2- 172.21.15.38:0/695744573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe812a9d0 0x55afe812ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.834+0000 7f013daa5640 1 --2- 172.21.15.38:0/695744573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe80fff10 0x55afe8102070 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f013402a970 tx=0x7f013402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8e1062d173c7a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.835+0000 7f011b7fe640 1 -- 172.21.15.38:0/695744573 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0134037040 con 0x55afe80fff10 2024-04-25T08:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.835+0000 7f011b7fe640 1 -- 172.21.15.38:0/695744573 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0134032db0 con 0x55afe80fff10 2024-04-25T08:35:14.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.835+0000 7f011b7fe640 1 -- 172.21.15.38:0/695744573 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013403d640 con 0x55afe80fff10 2024-04-25T08:35:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.835+0000 7f0142a11740 1 -- 172.21.15.38:0/695744573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe80fff10 msgr2=0x55afe8102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.835+0000 7f0142a11740 1 --2- 172.21.15.38:0/695744573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe80fff10 0x55afe8102070 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f013402a970 tx=0x7f013402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.835+0000 7f0142a11740 1 -- 172.21.15.38:0/695744573 shutdown_connections 2024-04-25T08:35:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.835+0000 7f0142a11740 1 --2- 172.21.15.38:0/695744573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe812a9d0 0x55afe812ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.835+0000 7f0142a11740 1 --2- 172.21.15.38:0/695744573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe813c5e0 0x55afe81002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.835+0000 7f0142a11740 1 --2- 172.21.15.38:0/695744573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe80fff10 0x55afe8102070 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.835+0000 7f0142a11740 1 -- 172.21.15.38:0/695744573 >> 172.21.15.38:0/695744573 conn(0x55afe80c8100 msgr2=0x55afe81459d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.835+0000 7f0142a11740 1 -- 172.21.15.38:0/695744573 shutdown_connections 2024-04-25T08:35:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.835+0000 7f0142a11740 1 -- 172.21.15.38:0/695744573 wait complete. 2024-04-25T08:35:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.836+0000 7f0142a11740 1 -- start start 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.836+0000 7f0142a11740 1 Processor -- start 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f0142a11740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe80fff10 0x55afe812ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f0142a11740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe812a9d0 0x55afe8114200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f0142a11740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 0x55afe8127c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f0142a11740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8002750 con 0x55afe813c5e0 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f0142a11740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe805fdc0 con 0x55afe80fff10 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f0142a11740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55afe8128660 con 0x55afe812a9d0 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f0142a11740 10 client.0 _ll_register_callbacks cb 0x55afe801aee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f013e2a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 0x55afe8127c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f013e2a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 0x55afe8127c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44018/0 (socket says 172.21.15.38:44018) 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f013e2a6640 1 -- 172.21.15.38:0/1580333593 learned_addr learned my addr 172.21.15.38:0/1580333593 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f013daa5640 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe80fff10 0x55afe812ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f013d2a4640 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe812a9d0 0x55afe8114200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f013e2a6640 1 -- 172.21.15.38:0/1580333593 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe812a9d0 msgr2=0x55afe8114200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f013e2a6640 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe812a9d0 0x55afe8114200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f013e2a6640 1 -- 172.21.15.38:0/1580333593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe80fff10 msgr2=0x55afe812ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f013e2a6640 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe80fff10 0x55afe812ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f013e2a6640 1 -- 172.21.15.38:0/1580333593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afe7d383f0 con 0x55afe813c5e0 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f013daa5640 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe80fff10 0x55afe812ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f013e2a6640 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 0x55afe8127c50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f01300884e0 tx=0x7f0130059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56a77ccac17f3a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.837+0000 7f0118ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0118ff9640 1 -- 172.21.15.38:0/1580333593 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130070300 con 0x55afe813c5e0 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0118ff9640 1 -- 172.21.15.38:0/1580333593 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0130057c10 con 0x55afe813c5e0 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0142a11740 1 -- 172.21.15.38:0/1580333593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0134032db0 con 0x55afe813c5e0 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0118ff9640 1 -- 172.21.15.38:0/1580333593 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0130057e10 con 0x55afe813c5e0 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0142a11740 10 client.6441 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0142a11740 1 -- 172.21.15.38:0/1580333593 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55afe7ea5290 con 0x55afe813c5e0 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0118ff9640 1 -- 172.21.15.38:0/1580333593 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0134032db0 con 0x55afe813c5e0 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0118ff9640 1 -- 172.21.15.38:0/1580333593 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0130059af0 con 0x55afe813c5e0 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0142a11740 10 client.6441 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:14.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0142a11740 10 client.6441 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0142a11740 20 client.6441 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0142a11740 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55afe812cc10 0x55afe812d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0142a11740 4 client.6441 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f0142a11740 1 -- 172.21.15.38:0/1580333593 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afe8071260 con 0x55afe812cc10 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.838+0000 7f013daa5640 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55afe812cc10 0x55afe812d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0118ff9640 10 client.6441 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f013daa5640 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55afe812cc10 0x55afe812d690 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51671257462f6de4 server_cookie=b6ec9f50f5ef493d in_seq=0 out_seq=0 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0118ff9640 1 -- 172.21.15.38:0/1580333593 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55afe8071260 con 0x55afe812cc10 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0118ff9640 10 client.6441 handle_command_reply: tid=0 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0142a11740 1 client.6441 shutdown 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0142a11740 1 -- 172.21.15.38:0/1580333593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 msgr2=0x55afe8127c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0142a11740 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 0x55afe8127c50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f01300884e0 tx=0x7f0130059000 comp rx=0 tx=0).stop 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0142a11740 1 -- 172.21.15.38:0/1580333593 shutdown_connections 2024-04-25T08:35:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0142a11740 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55afe812cc10 0x55afe812d690 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0142a11740 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 0x55afe8127c50 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0142a11740 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe812a9d0 0x55afe8114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0142a11740 1 --2- 172.21.15.38:0/1580333593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe80fff10 0x55afe812ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0118ff9640 0 client.6441 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.839+0000 7f0142a11740 1 -- 172.21.15.38:0/1580333593 >> 172.21.15.38:0/1580333593 conn(0x55afe80c8100 msgr2=0x55afe8146b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.840+0000 7f0142a11740 1 -- 172.21.15.38:0/1580333593 shutdown_connections 2024-04-25T08:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.840+0000 7f0142a11740 1 -- 172.21.15.38:0/1580333593 wait complete. 2024-04-25T08:35:14.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.840+0000 7f0142a11740 20 client.6441 trim_cache size 0 max 16384 2024-04-25T08:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.855+0000 7f0142a11740 1 Processor -- start 2024-04-25T08:35:14.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.855+0000 7f0142a11740 1 -- start start 2024-04-25T08:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f0142a11740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe80fff10 0x55afe8148de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f0142a11740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe812a9d0 0x55afe812d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f0142a11740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 0x55afe8129920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f0142a11740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0138200760 con 0x55afe813c5e0 2024-04-25T08:35:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f0142a11740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381ff920 con 0x55afe80fff10 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f0142a11740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01381fed20 con 0x55afe812a9d0 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f013e2a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 0x55afe8129920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f013e2a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 0x55afe8129920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44032/0 (socket says 172.21.15.38:44032) 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f013e2a6640 1 -- 172.21.15.38:0/2487529715 learned_addr learned my addr 172.21.15.38:0/2487529715 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f013daa5640 1 --2- 172.21.15.38:0/2487529715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe80fff10 0x55afe8148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f013d2a4640 1 --2- 172.21.15.38:0/2487529715 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe812a9d0 0x55afe812d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f013e2a6640 1 -- 172.21.15.38:0/2487529715 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe812a9d0 msgr2=0x55afe812d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f013e2a6640 1 --2- 172.21.15.38:0/2487529715 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe812a9d0 0x55afe812d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f013e2a6640 1 -- 172.21.15.38:0/2487529715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe80fff10 msgr2=0x55afe8148de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f013e2a6640 1 --2- 172.21.15.38:0/2487529715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe80fff10 0x55afe8148de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f013e2a6640 1 -- 172.21.15.38:0/2487529715 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afe7d114b0 con 0x55afe813c5e0 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f013e2a6640 1 --2- 172.21.15.38:0/2487529715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 0x55afe8129920 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0130000ea0 tx=0x7f013006d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9add2805ef6b0e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f011affd640 1 -- 172.21.15.38:0/2487529715 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130088270 con 0x55afe813c5e0 2024-04-25T08:35:14.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f011affd640 1 -- 172.21.15.38:0/2487529715 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013005c910 con 0x55afe813c5e0 2024-04-25T08:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.856+0000 7f011affd640 1 -- 172.21.15.38:0/2487529715 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0130070100 con 0x55afe813c5e0 2024-04-25T08:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.857+0000 7f0142a11740 1 -- 172.21.15.38:0/2487529715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 msgr2=0x55afe8129920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.857+0000 7f0142a11740 1 --2- 172.21.15.38:0/2487529715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 0x55afe8129920 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0130000ea0 tx=0x7f013006d000 comp rx=0 tx=0).stop 2024-04-25T08:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.857+0000 7f0142a11740 1 -- 172.21.15.38:0/2487529715 shutdown_connections 2024-04-25T08:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.857+0000 7f0142a11740 1 --2- 172.21.15.38:0/2487529715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe813c5e0 0x55afe8129920 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.857+0000 7f0142a11740 1 --2- 172.21.15.38:0/2487529715 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe812a9d0 0x55afe812d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.857+0000 7f0142a11740 1 --2- 172.21.15.38:0/2487529715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe80fff10 0x55afe8148de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.857+0000 7f0142a11740 1 -- 172.21.15.38:0/2487529715 >> 172.21.15.38:0/2487529715 conn(0x55afe80c8100 msgr2=0x55afe80f6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.857+0000 7f0142a11740 1 -- 172.21.15.38:0/2487529715 shutdown_connections 2024-04-25T08:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.857+0000 7f0142a11740 1 -- 172.21.15.38:0/2487529715 wait complete. 2024-04-25T08:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.857+0000 7f0142a11740 1 -- start start 2024-04-25T08:35:14.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.858+0000 7f0142a11740 1 Processor -- start 2024-04-25T08:35:14.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.858+0000 7f0142a11740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe80fff10 0x55afe812dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.858+0000 7f0142a11740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe812a9d0 0x55afe813b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.858+0000 7f0142a11740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe813c5e0 0x55afe812d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.858+0000 7f0142a11740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afe8126960 con 0x55afe812a9d0 2024-04-25T08:35:14.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.858+0000 7f0142a11740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01381ff320 con 0x55afe813c5e0 2024-04-25T08:35:14.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.858+0000 7f0142a11740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55afe811ec50 con 0x55afe80fff10 2024-04-25T08:35:14.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f0142a11740 10 client.0 _ll_register_callbacks cb 0x7f0134035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:14.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f013d2a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe812a9d0 0x55afe813b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f013d2a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe812a9d0 0x55afe813b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44044/0 (socket says 172.21.15.38:44044) 2024-04-25T08:35:14.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f013d2a4640 1 -- 172.21.15.38:0/989207498 learned_addr learned my addr 172.21.15.38:0/989207498 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f013daa5640 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe80fff10 0x55afe812dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f013e2a6640 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe813c5e0 0x55afe812d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f013d2a4640 1 -- 172.21.15.38:0/989207498 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe80fff10 msgr2=0x55afe812dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f013d2a4640 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe80fff10 0x55afe812dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f013d2a4640 1 -- 172.21.15.38:0/989207498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe813c5e0 msgr2=0x55afe812d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f013d2a4640 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe813c5e0 0x55afe812d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f013d2a4640 1 -- 172.21.15.38:0/989207498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afe813bca0 con 0x55afe812a9d0 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f013e2a6640 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe813c5e0 0x55afe812d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f013d2a4640 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe812a9d0 0x55afe813b760 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f01380b85b0 tx=0x7f0128016a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d20a4da37cb883b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f0118ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f0118ff9640 1 -- 172.21.15.38:0/989207498 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128004020 con 0x55afe812a9d0 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f0118ff9640 1 -- 172.21.15.38:0/989207498 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f01280041c0 con 0x55afe812a9d0 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f0142a11740 1 -- 172.21.15.38:0/989207498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f013005c910 con 0x55afe812a9d0 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f0118ff9640 1 -- 172.21.15.38:0/989207498 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0128028050 con 0x55afe812a9d0 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f0142a11740 10 client.6447 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f0142a11740 1 -- 172.21.15.38:0/989207498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55afe80fea30 con 0x55afe812a9d0 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.859+0000 7f0118ff9640 1 -- 172.21.15.38:0/989207498 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f013005c910 con 0x55afe812a9d0 2024-04-25T08:35:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.860+0000 7f0118ff9640 1 -- 172.21.15.38:0/989207498 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f012802b020 con 0x55afe812a9d0 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.860+0000 7f0142a11740 10 client.6447 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.860+0000 7f0142a11740 10 client.6447 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.860+0000 7f0142a11740 20 client.6447 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.860+0000 7f0142a11740 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55afe812d7a0 0x55afe81b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.860+0000 7f0142a11740 4 client.6447 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.860+0000 7f0142a11740 1 -- 172.21.15.38:0/989207498 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55afe806b5b0 con 0x55afe812d7a0 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.860+0000 7f013daa5640 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55afe812d7a0 0x55afe81b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.860+0000 7f0118ff9640 10 client.6447 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.860+0000 7f013daa5640 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55afe812d7a0 0x55afe81b6ea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=563bc36082423a72 server_cookie=d99b1fd6c825a8d9 in_seq=0 out_seq=0 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0118ff9640 1 -- 172.21.15.38:0/989207498 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55afe806b5b0 con 0x55afe812d7a0 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0118ff9640 10 client.6447 handle_command_reply: tid=0 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0142a11740 1 client.6447 shutdown 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0142a11740 1 -- 172.21.15.38:0/989207498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe812a9d0 msgr2=0x55afe813b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0142a11740 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe812a9d0 0x55afe813b760 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f01380b85b0 tx=0x7f0128016a80 comp rx=0 tx=0).stop 2024-04-25T08:35:14.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0142a11740 1 -- 172.21.15.38:0/989207498 shutdown_connections 2024-04-25T08:35:14.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0142a11740 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55afe812d7a0 0x55afe81b6ea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0142a11740 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55afe813c5e0 0x55afe812d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0118ff9640 0 client.6447 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:14.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0142a11740 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55afe812a9d0 0x55afe813b760 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0142a11740 1 --2- 172.21.15.38:0/989207498 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55afe80fff10 0x55afe812dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0142a11740 1 -- 172.21.15.38:0/989207498 >> 172.21.15.38:0/989207498 conn(0x55afe80c8100 msgr2=0x55afe8148c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0142a11740 1 -- 172.21.15.38:0/989207498 shutdown_connections 2024-04-25T08:35:14.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0142a11740 1 -- 172.21.15.38:0/989207498 wait complete. 2024-04-25T08:35:14.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f0142a11740 20 client.6447 trim_cache size 0 max 16384 2024-04-25T08:35:14.862 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:14.863 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:35:14.863 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "9eb1a0a6-6ec5-41b5-baec-ef0d2fcbf0ae", 2024-04-25T08:35:14.863 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:35:14.863 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:14.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f013fd09640 1 -- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f011c06b4d0 msgr2=0x7f011c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f013fd09640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f011c06b4d0 0x7f011c06d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0134034000 tx=0x7f0134009030 comp rx=0 tx=0).stop 2024-04-25T08:35:14.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f013fd09640 1 -- 172.21.15.38:0/3687709236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01380ccbb0 msgr2=0x7f01380cd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.861+0000 7f013fd09640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01380ccbb0 0x7f01380cd030 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0130000c00 tx=0x7f0130007720 comp rx=0 tx=0).stop 2024-04-25T08:35:14.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.862+0000 7f013fd09640 1 -- 172.21.15.38:0/3687709236 shutdown_connections 2024-04-25T08:35:14.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.862+0000 7f013fd09640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01380ccbb0 0x7f01380cd030 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.862+0000 7f013fd09640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f011c06b4d0 0x7f011c06d990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.862+0000 7f013fd09640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0138158030 0x7f01380b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.862+0000 7f013fd09640 1 --2- 172.21.15.38:0/3687709236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01240039e0 0x7f0138153eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.862+0000 7f013fd09640 1 -- 172.21.15.38:0/3687709236 >> 172.21.15.38:0/3687709236 conn(0x7f01380668f0 msgr2=0x7f013815f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.862+0000 7f013fd09640 1 -- 172.21.15.38:0/3687709236 shutdown_connections 2024-04-25T08:35:14.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.862+0000 7f013fd09640 1 -- 172.21.15.38:0/3687709236 wait complete. 2024-04-25T08:35:14.867 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '9eb1a0a6-6ec5-41b5-baec-ef0d2fcbf0ae', 'mode': 'asynchronous'}' 2024-04-25T08:35:14.867 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T08:35:14.867 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:14.867 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T08:35:14.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.976+0000 7f0f5d437640 1 Processor -- start 2024-04-25T08:35:14.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.976+0000 7f0f5d437640 1 -- start start 2024-04-25T08:35:14.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.977+0000 7f0f5d437640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f58166610 0x7f0f58168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.977+0000 7f0f5d437640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f580d2060 con 0x7f0f580d33d0 2024-04-25T08:35:14.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.977+0000 7f0f5d437640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f580d2330 con 0x7f0f580d0320 2024-04-25T08:35:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.977+0000 7f0f5d437640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580d2610 con 0x7f0f58166610 2024-04-25T08:35:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.977+0000 7f0f567fc640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0f580d33d0 0x7f0f580d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56692/0 (socket says 172.21.15.38:56692) 2024-04-25T08:35:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.977+0000 7f0f567fc640 1 -- 172.21.15.38:0/3076687283 learned_addr learned my addr 172.21.15.38:0/3076687283 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.977+0000 7f0f577fe640 1 --2- 172.21.15.38:0/3076687283 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f58166610 0x7f0f58168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.977+0000 7f0f55ffb640 1 -- 172.21.15.38:0/3076687283 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895181367 0 0) 0x7f0f580d2060 con 0x7f0f580d33d0 2024-04-25T08:35:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.977+0000 7f0f55ffb640 1 -- 172.21.15.38:0/3076687283 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f44003520 con 0x7f0f580d33d0 2024-04-25T08:35:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.978+0000 7f0f55ffb640 1 -- 172.21.15.38:0/3076687283 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287116926 0 0) 0x7f0f580d2330 con 0x7f0f580d0320 2024-04-25T08:35:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.978+0000 7f0f55ffb640 1 -- 172.21.15.38:0/3076687283 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f580d2060 con 0x7f0f580d0320 2024-04-25T08:35:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.978+0000 7f0f55ffb640 1 -- 172.21.15.38:0/3076687283 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1911387317 0 0) 0x7f0f44003520 con 0x7f0f580d33d0 2024-04-25T08:35:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.978+0000 7f0f577fe640 1 -- 172.21.15.38:0/3076687283 >> v1:172.21.15.107:6789/0 conn(0x7f0f580d0320 legacy=0x7f0f580d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.978+0000 7f0f577fe640 1 -- 172.21.15.38:0/3076687283 >> v1:172.21.15.38:6789/0 conn(0x7f0f580d33d0 legacy=0x7f0f580d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.978+0000 7f0f577fe640 1 -- 172.21.15.38:0/3076687283 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580d28f0 con 0x7f0f58166610 2024-04-25T08:35:14.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.978+0000 7f0f577fe640 1 --2- 172.21.15.38:0/3076687283 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f58166610 0x7f0f58168a00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009dc0 tx=0x7f0f4802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f8de8529ce8b78c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.979+0000 7f0f55ffb640 1 -- 172.21.15.38:0/3076687283 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40001690 con 0x7f0f58166610 2024-04-25T08:35:14.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.979+0000 7f0f55ffb640 1 -- 172.21.15.38:0/3076687283 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4802ae50 con 0x7f0f58166610 2024-04-25T08:35:14.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.979+0000 7f0f55ffb640 1 -- 172.21.15.38:0/3076687283 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48007740 con 0x7f0f58166610 2024-04-25T08:35:14.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.979+0000 7f0f5d437640 1 -- 172.21.15.38:0/3076687283 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f58166610 msgr2=0x7f0f58168a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.979+0000 7f0f5d437640 1 --2- 172.21.15.38:0/3076687283 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f58166610 0x7f0f58168a00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009dc0 tx=0x7f0f4802a6f0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.979+0000 7f0f5d437640 1 -- 172.21.15.38:0/3076687283 shutdown_connections 2024-04-25T08:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.979+0000 7f0f5d437640 1 --2- 172.21.15.38:0/3076687283 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f58166610 0x7f0f58168a00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.979+0000 7f0f5d437640 1 -- 172.21.15.38:0/3076687283 >> 172.21.15.38:0/3076687283 conn(0x7f0f580668f0 msgr2=0x7f0f58066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.979+0000 7f0f5d437640 1 -- 172.21.15.38:0/3076687283 shutdown_connections 2024-04-25T08:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.979+0000 7f0f5d437640 1 -- 172.21.15.38:0/3076687283 wait complete. 2024-04-25T08:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.980+0000 7f0f5d437640 1 Processor -- start 2024-04-25T08:35:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.980+0000 7f0f5d437640 1 -- start start 2024-04-25T08:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.981+0000 7f0f5d437640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f580d0320 0x7f0f5814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.981+0000 7f0f5d437640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f580d33d0 0x7f0f58160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.981+0000 7f0f5d437640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f58166610 0x7f0f580ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.981+0000 7f0f5d437640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58161210 con 0x7f0f580d33d0 2024-04-25T08:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.981+0000 7f0f5d437640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580d2eb0 con 0x7f0f58166610 2024-04-25T08:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.981+0000 7f0f5d437640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58161390 con 0x7f0f580d0320 2024-04-25T08:35:14.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.981+0000 7f0f567fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f580d33d0 0x7f0f58160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.981+0000 7f0f567fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f580d33d0 0x7f0f58160250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44052/0 (socket says 172.21.15.38:44052) 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.981+0000 7f0f567fc640 1 -- 172.21.15.38:0/2691582416 learned_addr learned my addr 172.21.15.38:0/2691582416 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.981+0000 7f0f577fe640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f58166610 0x7f0f580ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f580d0320 0x7f0f5814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f567fc640 1 -- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f580d0320 msgr2=0x7f0f5814edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f567fc640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f580d0320 0x7f0f5814edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f567fc640 1 -- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f58166610 msgr2=0x7f0f580ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f567fc640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f58166610 0x7f0f580ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f567fc640 1 -- 172.21.15.38:0/2691582416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580b56a0 con 0x7f0f580d33d0 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f577fe640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f58166610 0x7f0f580ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f580d0320 0x7f0f5814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f567fc640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f580d33d0 0x7f0f58160250 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0f40001200 tx=0x7f0f4000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f4f7fe640 1 -- 172.21.15.38:0/2691582416 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40018070 con 0x7f0f580d33d0 2024-04-25T08:35:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f4f7fe640 1 -- 172.21.15.38:0/2691582416 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f400072d0 con 0x7f0f580d33d0 2024-04-25T08:35:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f5d437640 1 -- 172.21.15.38:0/2691582416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f580b5980 con 0x7f0f580d33d0 2024-04-25T08:35:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f4f7fe640 1 -- 172.21.15.38:0/2691582416 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400047e0 con 0x7f0f580d33d0 2024-04-25T08:35:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.982+0000 7f0f5d437640 1 -- 172.21.15.38:0/2691582416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f580b5f40 con 0x7f0f580d33d0 2024-04-25T08:35:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.984+0000 7f0f4f7fe640 1 -- 172.21.15.38:0/2691582416 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f4001b070 con 0x7f0f580d33d0 2024-04-25T08:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.984+0000 7f0f4f7fe640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f3806b4d0 0x7f0f3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.984+0000 7f0f4f7fe640 1 -- 172.21.15.38:0/2691582416 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f400897f0 con 0x7f0f580d33d0 2024-04-25T08:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.984+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f3806b4d0 0x7f0f3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.985+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f3806b4d0 0x7f0f3806d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0f500011e0 tx=0x7f0f50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.985+0000 7f0f6020c740 1 Processor -- start 2024-04-25T08:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.985+0000 7f0f6020c740 1 -- start start 2024-04-25T08:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f6020c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 0x5653d90351b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f6020c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d9037340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f6020c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9034590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f6020c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d8f53e60 con 0x5653d9036f60 2024-04-25T08:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f6020c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d8efd120 con 0x5653d9017af0 2024-04-25T08:35:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f6020c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653d8f4ff00 con 0x5653d8ff1130 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f56ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 0x5653d90351b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f56ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 0x5653d90351b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44056/0 (socket says 172.21.15.38:44056) 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f56ffd640 1 -- 172.21.15.38:0/468304847 learned_addr learned my addr 172.21.15.38:0/468304847 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f577fe640 1 --2- 172.21.15.38:0/468304847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9034590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f567fc640 1 --2- 172.21.15.38:0/468304847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d9037340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f56ffd640 1 -- 172.21.15.38:0/468304847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 msgr2=0x5653d9037340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/468304847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d9037340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f56ffd640 1 -- 172.21.15.38:0/468304847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 msgr2=0x5653d9034590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/468304847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9034590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f56ffd640 1 -- 172.21.15.38:0/468304847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d8d34f00 con 0x5653d9036f60 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f577fe640 1 --2- 172.21.15.38:0/468304847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9034590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.986+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/468304847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 0x5653d90351b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000dac0 tx=0x7f0f50010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c911c4abb6c9facf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.987+0000 7f0f4cff9640 1 -- 172.21.15.38:0/468304847 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50024070 con 0x5653d9036f60 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.987+0000 7f0f4cff9640 1 -- 172.21.15.38:0/468304847 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f50014db0 con 0x5653d9036f60 2024-04-25T08:35:14.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.987+0000 7f0f4cff9640 1 -- 172.21.15.38:0/468304847 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5001f600 con 0x5653d9036f60 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.987+0000 7f0f6020c740 1 -- 172.21.15.38:0/468304847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 msgr2=0x5653d90351b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.987+0000 7f0f6020c740 1 --2- 172.21.15.38:0/468304847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 0x5653d90351b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f0f5000dac0 tx=0x7f0f50010000 comp rx=0 tx=0).stop 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.987+0000 7f0f6020c740 1 -- 172.21.15.38:0/468304847 shutdown_connections 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.987+0000 7f0f6020c740 1 --2- 172.21.15.38:0/468304847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9034590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.987+0000 7f0f6020c740 1 --2- 172.21.15.38:0/468304847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d9037340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.987+0000 7f0f6020c740 1 --2- 172.21.15.38:0/468304847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 0x5653d90351b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.987+0000 7f0f6020c740 1 -- 172.21.15.38:0/468304847 >> 172.21.15.38:0/468304847 conn(0x5653d8fb7f70 msgr2=0x5653d8fee150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.987+0000 7f0f6020c740 1 -- 172.21.15.38:0/468304847 shutdown_connections 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.987+0000 7f0f6020c740 1 -- 172.21.15.38:0/468304847 wait complete. 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.988+0000 7f0f6020c740 1 -- start start 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.988+0000 7f0f6020c740 1 Processor -- start 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f6020c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d90188a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f6020c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9017af0 0x5653d9018de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f6020c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9036f60 0x5653d901c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f6020c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d9036760 con 0x5653d9017af0 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f6020c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d8ef2cd0 con 0x5653d9036f60 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f6020c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653d9036480 con 0x5653d8ff1130 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f6020c740 10 client.0 _ll_register_callbacks cb 0x5653d8c682e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f567fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9017af0 0x5653d9018de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f567fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9017af0 0x5653d9018de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44062/0 (socket says 172.21.15.38:44062) 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f567fc640 1 -- 172.21.15.38:0/3298781361 learned_addr learned my addr 172.21.15.38:0/3298781361 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f577fe640 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9036f60 0x5653d901c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d90188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f567fc640 1 -- 172.21.15.38:0/3298781361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 msgr2=0x5653d90188a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f567fc640 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d90188a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f567fc640 1 -- 172.21.15.38:0/3298781361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9036f60 msgr2=0x5653d901c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f567fc640 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9036f60 0x5653d901c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f567fc640 1 -- 172.21.15.38:0/3298781361 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653d8c287f0 con 0x5653d9017af0 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d90188a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f567fc640 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9017af0 0x5653d9018de0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0f40001800 tx=0x7f0f40058620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ae4ef280cbb22c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.989+0000 7f0f35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f35ffb640 1 -- 172.21.15.38:0/3298781361 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40057230 con 0x5653d9017af0 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f35ffb640 1 -- 172.21.15.38:0/3298781361 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f400573d0 con 0x5653d9017af0 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f6020c740 1 -- 172.21.15.38:0/3298781361 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f50014db0 con 0x5653d9017af0 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f35ffb640 1 -- 172.21.15.38:0/3298781361 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f4006fb70 con 0x5653d9017af0 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f6020c740 10 client.6459 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f6020c740 1 -- 172.21.15.38:0/3298781361 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5653d8d95690 con 0x5653d9017af0 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f35ffb640 1 -- 172.21.15.38:0/3298781361 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f50014db0 con 0x5653d9017af0 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f35ffb640 1 -- 172.21.15.38:0/3298781361 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0f40053b30 con 0x5653d9017af0 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f6020c740 10 client.6459 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f6020c740 10 client.6459 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:14.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f6020c740 20 client.6459 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f6020c740 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5653d901d290 0x5653d9053000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f6020c740 4 client.6459 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f6020c740 1 -- 172.21.15.38:0/3298781361 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653d8f613a0 con 0x5653d901d290 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5653d901d290 0x5653d9053000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5653d901d290 0x5653d9053000 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=824faa09ef0c5474 server_cookie=7817a2014ebd5c5d in_seq=0 out_seq=0 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.990+0000 7f0f35ffb640 10 client.6459 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f35ffb640 1 -- 172.21.15.38:0/3298781361 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653d8f613a0 con 0x5653d901d290 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f35ffb640 10 client.6459 handle_command_reply: tid=0 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f6020c740 1 client.6459 shutdown 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f6020c740 1 -- 172.21.15.38:0/3298781361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9017af0 msgr2=0x5653d9018de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f6020c740 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9017af0 0x5653d9018de0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f0f40001800 tx=0x7f0f40058620 comp rx=0 tx=0).stop 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f6020c740 1 -- 172.21.15.38:0/3298781361 shutdown_connections 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f6020c740 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5653d901d290 0x5653d9053000 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f6020c740 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9036f60 0x5653d901c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f6020c740 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9017af0 0x5653d9018de0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f35ffb640 0 client.6459 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:14.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f6020c740 1 --2- 172.21.15.38:0/3298781361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d90188a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:14.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f6020c740 1 -- 172.21.15.38:0/3298781361 >> 172.21.15.38:0/3298781361 conn(0x5653d8fb7f70 msgr2=0x5653d90166b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:14.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f6020c740 1 -- 172.21.15.38:0/3298781361 shutdown_connections 2024-04-25T08:35:14.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f6020c740 1 -- 172.21.15.38:0/3298781361 wait complete. 2024-04-25T08:35:14.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:14.991+0000 7f0f6020c740 20 client.6459 trim_cache size 0 max 16384 2024-04-25T08:35:15.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.006+0000 7f0f6020c740 1 Processor -- start 2024-04-25T08:35:15.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.007+0000 7f0f6020c740 1 -- start start 2024-04-25T08:35:15.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.007+0000 7f0f6020c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d8ff1130 0x5653d8fec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.007+0000 7f0f6020c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9016f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.007+0000 7f0f6020c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d9036f60 0x5653d9017470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.007+0000 7f0f6020c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58200210 con 0x5653d8ff1130 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.007+0000 7f0f6020c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581ff480 con 0x5653d9017af0 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.007+0000 7f0f6020c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580cb980 con 0x5653d9036f60 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.007+0000 7f0f56ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d8ff1130 0x5653d8fec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.007+0000 7f0f56ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d8ff1130 0x5653d8fec6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44066/0 (socket says 172.21.15.38:44066) 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.007+0000 7f0f56ffd640 1 -- 172.21.15.38:0/708932197 learned_addr learned my addr 172.21.15.38:0/708932197 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f56ffd640 1 -- 172.21.15.38:0/708932197 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d9036f60 msgr2=0x5653d9017470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f577fe640 1 --2- 172.21.15.38:0/708932197 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d9036f60 0x5653d9017470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f567fc640 1 --2- 172.21.15.38:0/708932197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9016f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/708932197 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d9036f60 0x5653d9017470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f56ffd640 1 -- 172.21.15.38:0/708932197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 msgr2=0x5653d9016f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/708932197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9016f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f56ffd640 1 -- 172.21.15.38:0/708932197 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653d8c018b0 con 0x5653d8ff1130 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f577fe640 1 --2- 172.21.15.38:0/708932197 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d9036f60 0x5653d9017470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f567fc640 1 --2- 172.21.15.38:0/708932197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9016f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/708932197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d8ff1130 0x5653d8fec6c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009010 tx=0x7f0f50012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e253ccec50dbb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f37fff640 1 -- 172.21.15.38:0/708932197 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50035070 con 0x5653d8ff1130 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f37fff640 1 -- 172.21.15.38:0/708932197 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5002f030 con 0x5653d8ff1130 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f37fff640 1 -- 172.21.15.38:0/708932197 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f50024040 con 0x5653d8ff1130 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f6020c740 1 -- 172.21.15.38:0/708932197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d8ff1130 msgr2=0x5653d8fec6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f6020c740 1 --2- 172.21.15.38:0/708932197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d8ff1130 0x5653d8fec6c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009010 tx=0x7f0f50012000 comp rx=0 tx=0).stop 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f6020c740 1 -- 172.21.15.38:0/708932197 shutdown_connections 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f6020c740 1 --2- 172.21.15.38:0/708932197 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d9036f60 0x5653d9017470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f6020c740 1 --2- 172.21.15.38:0/708932197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9016f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f6020c740 1 --2- 172.21.15.38:0/708932197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d8ff1130 0x5653d8fec6c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.008+0000 7f0f6020c740 1 -- 172.21.15.38:0/708932197 >> 172.21.15.38:0/708932197 conn(0x5653d8fb7f70 msgr2=0x5653d8fe6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.009+0000 7f0f6020c740 1 -- 172.21.15.38:0/708932197 shutdown_connections 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.009+0000 7f0f6020c740 1 -- 172.21.15.38:0/708932197 wait complete. 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.009+0000 7f0f6020c740 1 -- start start 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.009+0000 7f0f6020c740 1 Processor -- start 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f6020c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d90323e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f6020c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9016330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f6020c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 0x5653d9038c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f6020c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653d90191d0 con 0x5653d9036f60 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f6020c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f580cbf80 con 0x5653d9017af0 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f6020c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653d901bf70 con 0x5653d8ff1130 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f6020c740 10 client.0 _ll_register_callbacks cb 0x7f0f50001470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 0x5653d9038c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 0x5653d9038c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44076/0 (socket says 172.21.15.38:44076) 2024-04-25T08:35:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f577fe640 1 -- 172.21.15.38:0/3377056651 learned_addr learned my addr 172.21.15.38:0/3377056651 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f567fc640 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9016330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d90323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f577fe640 1 -- 172.21.15.38:0/3377056651 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 msgr2=0x5653d90323e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f577fe640 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d90323e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f577fe640 1 -- 172.21.15.38:0/3377056651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 msgr2=0x5653d9016330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f577fe640 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9016330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f577fe640 1 -- 172.21.15.38:0/3377056651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653d9055190 con 0x5653d9036f60 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f577fe640 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 0x5653d9038c80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0f48003e20 tx=0x7f0f48004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410b4e7b166ee9df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.010+0000 7f0f35ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f35ffb640 1 -- 172.21.15.38:0/3377056651 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f48009450 con 0x5653d9036f60 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f35ffb640 1 -- 172.21.15.38:0/3377056651 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f4804ac60 con 0x5653d9036f60 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f6020c740 1 -- 172.21.15.38:0/3377056651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0f5002f030 con 0x5653d9036f60 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f35ffb640 1 -- 172.21.15.38:0/3377056651 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f4804ae60 con 0x5653d9036f60 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f6020c740 10 client.6465 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f6020c740 1 -- 172.21.15.38:0/3377056651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5653d901bb30 con 0x5653d9036f60 2024-04-25T08:35:15.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f35ffb640 1 -- 172.21.15.38:0/3377056651 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5002f030 con 0x5653d9036f60 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f35ffb640 1 -- 172.21.15.38:0/3377056651 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0f48004250 con 0x5653d9036f60 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f6020c740 10 client.6465 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f6020c740 10 client.6465 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f6020c740 20 client.6465 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f6020c740 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5653d8fed3e0 0x5653d90391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f6020c740 4 client.6465 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f6020c740 1 -- 172.21.15.38:0/3377056651 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x5653d8f5b6f0 con 0x5653d8fed3e0 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5653d8fed3e0 0x5653d90391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f35ffb640 10 client.6465 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.011+0000 7f0f56ffd640 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5653d8fed3e0 0x5653d90391c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ca8ed55da80770 server_cookie=d87d12d614ce6269 in_seq=0 out_seq=0 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f35ffb640 1 -- 172.21.15.38:0/3377056651 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5653d8f5b6f0 con 0x5653d8fed3e0 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f35ffb640 10 client.6465 handle_command_reply: tid=0 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f6020c740 1 client.6465 shutdown 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f6020c740 1 -- 172.21.15.38:0/3377056651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 msgr2=0x5653d9038c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f6020c740 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 0x5653d9038c80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0f48003e20 tx=0x7f0f48004ab0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f6020c740 1 -- 172.21.15.38:0/3377056651 shutdown_connections 2024-04-25T08:35:15.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f6020c740 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5653d8fed3e0 0x5653d90391c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f6020c740 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653d9036f60 0x5653d9038c80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f6020c740 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5653d9017af0 0x5653d9016330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f6020c740 1 --2- 172.21.15.38:0/3377056651 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5653d8ff1130 0x5653d90323e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f6020c740 1 -- 172.21.15.38:0/3377056651 >> 172.21.15.38:0/3377056651 conn(0x5653d8fb7f70 msgr2=0x5653d8fec6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:15.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f35ffb640 0 client.6465 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:15.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f6020c740 1 -- 172.21.15.38:0/3377056651 shutdown_connections 2024-04-25T08:35:15.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f6020c740 1 -- 172.21.15.38:0/3377056651 wait complete. 2024-04-25T08:35:15.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.012+0000 7f0f6020c740 20 client.6465 trim_cache size 0 max 16384 2024-04-25T08:35:15.013 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:15.014 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:35:15.014 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "3b6f1285-ec00-44c1-9790-b85ccb89d320", 2024-04-25T08:35:15.014 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:35:15.014 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.013+0000 7f0f5d437640 1 -- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f3806b4d0 msgr2=0x7f0f3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.013+0000 7f0f5d437640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f3806b4d0 0x7f0f3806d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0f500011e0 tx=0x7f0f50009040 comp rx=0 tx=0).stop 2024-04-25T08:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.013+0000 7f0f5d437640 1 -- 172.21.15.38:0/2691582416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f580d33d0 msgr2=0x7f0f58160250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.013+0000 7f0f5d437640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f580d33d0 0x7f0f58160250 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f0f40001200 tx=0x7f0f4000cbe0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.013+0000 7f0f5d437640 1 -- 172.21.15.38:0/2691582416 shutdown_connections 2024-04-25T08:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.013+0000 7f0f5d437640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f58166610 0x7f0f580ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.013+0000 7f0f5d437640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f580d33d0 0x7f0f58160250 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.013+0000 7f0f5d437640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f3806b4d0 0x7f0f3806d990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.013+0000 7f0f5d437640 1 --2- 172.21.15.38:0/2691582416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f580d0320 0x7f0f5814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.013+0000 7f0f5d437640 1 -- 172.21.15.38:0/2691582416 >> 172.21.15.38:0/2691582416 conn(0x7f0f580668f0 msgr2=0x7f0f580b8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.013+0000 7f0f5d437640 1 -- 172.21.15.38:0/2691582416 shutdown_connections 2024-04-25T08:35:15.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.013+0000 7f0f5d437640 1 -- 172.21.15.38:0/2691582416 wait complete. 2024-04-25T08:35:15.018 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '3b6f1285-ec00-44c1-9790-b85ccb89d320', 'mode': 'asynchronous'}' 2024-04-25T08:35:15.018 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T08:35:15.018 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:15.019 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T08:35:15.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.128+0000 7f58fbb29640 1 Processor -- start 2024-04-25T08:35:15.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.128+0000 7f58fbb29640 1 -- start start 2024-04-25T08:35:15.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.129+0000 7f58fbb29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40d33d0 0x7f58f40d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.129+0000 7f58fbb29640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f58f40d1e90 0x7f58f40b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.129+0000 7f58fbb29640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f40d2330 con 0x7f58f40d0320 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.129+0000 7f58fbb29640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f40d2610 con 0x7f58f40d1e90 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.129+0000 7f58fbb29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40d28f0 con 0x7f58f40d33d0 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.129+0000 7f58f98c5640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f58f40d0320 0x7f58f40d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56694/0 (socket says 172.21.15.38:56694) 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.129+0000 7f58f98c5640 1 -- 172.21.15.38:0/2412225762 learned_addr learned my addr 172.21.15.38:0/2412225762 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.129+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/2412225762 >> v2:172.21.15.107:3300/0 conn(0x7f58f40d1e90 0x7f58f40b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.129+0000 7f58f90c4640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40d33d0 0x7f58f40d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.129+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364562320 0 0) 0x7f58f40d2330 con 0x7f58f40d0320 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.129+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58e0003520 con 0x7f58f40d0320 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1609505802 0 0) 0x7f58e0003520 con 0x7f58f40d0320 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40d33d0 msgr2=0x7f58f40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40d33d0 0x7f58f40d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 >> v2:172.21.15.107:3300/0 conn(0x7f58f40d1e90 msgr2=0x7f58f40b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 --2- 172.21.15.38:0/2412225762 >> v2:172.21.15.107:3300/0 conn(0x7f58f40d1e90 0x7f58f40b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f40d2bd0 con 0x7f58f40d0320 2024-04-25T08:35:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f58f0001670 con 0x7f58f40d0320 2024-04-25T08:35:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 >> v1:172.21.15.38:6789/0 conn(0x7f58f40d0320 legacy=0x7f58f40d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e00039b0 0x7f58e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0004800 0x7f58e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40d33d0 0x7f58e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40d2610 con 0x7f58e0004800 2024-04-25T08:35:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40d28f0 con 0x7f58e00039b0 2024-04-25T08:35:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e000b9c0 con 0x7f58f40d33d0 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f90c4640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0004800 0x7f58e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40d33d0 0x7f58e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f98c5640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e00039b0 0x7f58e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f90c4640 1 -- 172.21.15.38:0/2412225762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40d33d0 msgr2=0x7f58e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f90c4640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f40d33d0 0x7f58e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f90c4640 1 -- 172.21.15.38:0/2412225762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e00039b0 msgr2=0x7f58e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f90c4640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e00039b0 0x7f58e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f90c4640 1 -- 172.21.15.38:0/2412225762 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e0003520 con 0x7f58e0004800 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.130+0000 7f58f98c5640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e00039b0 0x7f58e0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.131+0000 7f58f90c4640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0004800 0x7f58e0004c80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009ac0 tx=0x7f58e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef4e5e86471222c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.131+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4036070 con 0x7f58e0004800 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.131+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e4031400 con 0x7f58e0004800 2024-04-25T08:35:15.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.131+0000 7f58f88c3640 1 -- 172.21.15.38:0/2412225762 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e4009690 con 0x7f58e0004800 2024-04-25T08:35:15.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.132+0000 7f58fbb29640 1 -- 172.21.15.38:0/2412225762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0004800 msgr2=0x7f58e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.132+0000 7f58fbb29640 1 --2- 172.21.15.38:0/2412225762 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58e0004800 0x7f58e0004c80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009ac0 tx=0x7f58e0005830 comp rx=0 tx=0).stop 2024-04-25T08:35:15.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.132+0000 7f58fa0c6640 1 -- 172.21.15.38:0/2412225762 reap_dead start 2024-04-25T08:35:15.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.132+0000 7f58fbb29640 1 -- 172.21.15.38:0/2412225762 shutdown_connections 2024-04-25T08:35:15.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.132+0000 7f58fbb29640 1 -- 172.21.15.38:0/2412225762 >> 172.21.15.38:0/2412225762 conn(0x7f58f40668f0 msgr2=0x7f58f4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:15.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.132+0000 7f58fbb29640 1 -- 172.21.15.38:0/2412225762 shutdown_connections 2024-04-25T08:35:15.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.132+0000 7f58fbb29640 1 -- 172.21.15.38:0/2412225762 wait complete. 2024-04-25T08:35:15.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.133+0000 7f58fbb29640 1 Processor -- start 2024-04-25T08:35:15.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.133+0000 7f58fbb29640 1 -- start start 2024-04-25T08:35:15.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fbb29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e0004800 0x7f58f415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fbb29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f41e6c50 0x7f58f40cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fbb29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40b4ef0 0x7f58f40b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fbb29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f4161390 con 0x7f58f40b4ef0 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fbb29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f4161210 con 0x7f58e0004800 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fbb29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f40b9720 con 0x7f58f41e6c50 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fa0c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40b4ef0 0x7f58f40b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fa0c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40b4ef0 0x7f58f40b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44106/0 (socket says 172.21.15.38:44106) 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fa0c6640 1 -- 172.21.15.38:0/1223072710 learned_addr learned my addr 172.21.15.38:0/1223072710 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58f98c5640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e0004800 0x7f58f415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58f90c4640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f41e6c50 0x7f58f40cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fa0c6640 1 -- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f41e6c50 msgr2=0x7f58f40cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f41e6c50 0x7f58f40cc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fa0c6640 1 -- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e0004800 msgr2=0x7f58f415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e0004800 0x7f58f415b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fa0c6640 1 -- 172.21.15.38:0/1223072710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f415b570 con 0x7f58f40b4ef0 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40b4ef0 0x7f58f40b5370 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f58ec011c80 tx=0x7f58ec00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58ea7fc640 1 -- 172.21.15.38:0/1223072710 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec003990 con 0x7f58f40b4ef0 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58ea7fc640 1 -- 172.21.15.38:0/1223072710 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec01c070 con 0x7f58f40b4ef0 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.134+0000 7f58fbb29640 1 -- 172.21.15.38:0/1223072710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f41e71e0 con 0x7f58f40b4ef0 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.135+0000 7f58ea7fc640 1 -- 172.21.15.38:0/1223072710 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec003d50 con 0x7f58f40b4ef0 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.135+0000 7f58fbb29640 1 -- 172.21.15.38:0/1223072710 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f41e77a0 con 0x7f58f40b4ef0 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.136+0000 7f58ea7fc640 1 -- 172.21.15.38:0/1223072710 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58ec004030 con 0x7f58f40b4ef0 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.137+0000 7f58ea7fc640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58d806b4d0 0x7f58d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.137+0000 7f58ea7fc640 1 -- 172.21.15.38:0/1223072710 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58ec08ac30 con 0x7f58f40b4ef0 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.137+0000 7f58f98c5640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58d806b4d0 0x7f58d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.137+0000 7f58f98c5640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58d806b4d0 0x7f58d806d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58f0005b50 tx=0x7f58f0007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.138+0000 7f58fe82e740 1 Processor -- start 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.138+0000 7f58fe82e740 1 -- start start 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58fe82e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 0x5625f4f297f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58fe82e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f273c0 0x5625f4f29d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58fe82e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4ef9c40 0x5625f4efd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58fe82e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f4e62a50 con 0x5625f4f26fe0 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58fe82e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f4e0bd40 con 0x5625f4ef9c40 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58fe82e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625f4e5eaf0 con 0x5625f4f273c0 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58f98c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 0x5625f4f297f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58f98c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 0x5625f4f297f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44108/0 (socket says 172.21.15.38:44108) 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58f98c5640 1 -- 172.21.15.38:0/925844413 learned_addr learned my addr 172.21.15.38:0/925844413 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/925844413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4ef9c40 0x5625f4efd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58f90c4640 1 --2- 172.21.15.38:0/925844413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f273c0 0x5625f4f29d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58f98c5640 1 -- 172.21.15.38:0/925844413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f273c0 msgr2=0x5625f4f29d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58f98c5640 1 --2- 172.21.15.38:0/925844413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f273c0 0x5625f4f29d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58f98c5640 1 -- 172.21.15.38:0/925844413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4ef9c40 msgr2=0x5625f4efd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58f98c5640 1 --2- 172.21.15.38:0/925844413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4ef9c40 0x5625f4efd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58f98c5640 1 -- 172.21.15.38:0/925844413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625f4c43690 con 0x5625f4f26fe0 2024-04-25T08:35:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58f98c5640 1 --2- 172.21.15.38:0/925844413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 0x5625f4f297f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f58f0011c60 tx=0x7f58f0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c94a07ea9176a83f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58d77fe640 1 -- 172.21.15.38:0/925844413 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0027070 con 0x5625f4f26fe0 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58d77fe640 1 -- 172.21.15.38:0/925844413 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0022420 con 0x5625f4f26fe0 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.139+0000 7f58d77fe640 1 -- 172.21.15.38:0/925844413 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0011ea0 con 0x5625f4f26fe0 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.140+0000 7f58fe82e740 1 -- 172.21.15.38:0/925844413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 msgr2=0x5625f4f297f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.140+0000 7f58fe82e740 1 --2- 172.21.15.38:0/925844413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 0x5625f4f297f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f58f0011c60 tx=0x7f58f0014000 comp rx=0 tx=0).stop 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.140+0000 7f58fe82e740 1 -- 172.21.15.38:0/925844413 shutdown_connections 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.140+0000 7f58fe82e740 1 --2- 172.21.15.38:0/925844413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4ef9c40 0x5625f4efd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.140+0000 7f58fe82e740 1 --2- 172.21.15.38:0/925844413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f273c0 0x5625f4f29d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.140+0000 7f58fe82e740 1 --2- 172.21.15.38:0/925844413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 0x5625f4f297f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.140+0000 7f58fe82e740 1 -- 172.21.15.38:0/925844413 >> 172.21.15.38:0/925844413 conn(0x5625f4efbeb0 msgr2=0x5625f4f447f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.140+0000 7f58fe82e740 1 -- 172.21.15.38:0/925844413 shutdown_connections 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.140+0000 7f58fe82e740 1 -- 172.21.15.38:0/925844413 wait complete. 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.141+0000 7f58fe82e740 1 -- start start 2024-04-25T08:35:15.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.141+0000 7f58fe82e740 1 Processor -- start 2024-04-25T08:35:15.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fe82e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f26fe0 0x5625f4f26680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fe82e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f273c0 0x5625f4f4c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fe82e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 0x5625f4f47ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fe82e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f4eff060 con 0x5625f4f47a40 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fe82e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f4e01340 con 0x5625f4f26fe0 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fe82e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625f4eff340 con 0x5625f4f273c0 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fe82e740 10 client.0 _ll_register_callbacks cb 0x5625f4e1ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fa0c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 0x5625f4f47ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fa0c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 0x5625f4f47ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44114/0 (socket says 172.21.15.38:44114) 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fa0c6640 1 -- 172.21.15.38:0/401340030 learned_addr learned my addr 172.21.15.38:0/401340030 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58f98c5640 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f26fe0 0x5625f4f26680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58f90c4640 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f273c0 0x5625f4f4c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fa0c6640 1 -- 172.21.15.38:0/401340030 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f273c0 msgr2=0x5625f4f4c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f273c0 0x5625f4f4c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fa0c6640 1 -- 172.21.15.38:0/401340030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f26fe0 msgr2=0x5625f4f26680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f26fe0 0x5625f4f26680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fa0c6640 1 -- 172.21.15.38:0/401340030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625f4ca3e20 con 0x5625f4f47a40 2024-04-25T08:35:15.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 0x5625f4f47ec0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f58ec055730 tx=0x7f58ec05d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d1e7e7505935d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58d4ff9640 1 -- 172.21.15.38:0/401340030 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec05f530 con 0x5625f4f47a40 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58d4ff9640 1 -- 172.21.15.38:0/401340030 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58ec058720 con 0x5625f4f47a40 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.142+0000 7f58fe82e740 1 -- 172.21.15.38:0/401340030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58f0022420 con 0x5625f4f47a40 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58d4ff9640 1 -- 172.21.15.38:0/401340030 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58ec058920 con 0x5625f4f47a40 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58fe82e740 10 client.6477 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58fe82e740 1 -- 172.21.15.38:0/401340030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5625f4b36f80 con 0x5625f4f47a40 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58d4ff9640 1 -- 172.21.15.38:0/401340030 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0022420 con 0x5625f4f47a40 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58d4ff9640 1 -- 172.21.15.38:0/401340030 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f58ec055020 con 0x5625f4f47a40 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58fe82e740 10 client.6477 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58fe82e740 10 client.6477 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58fe82e740 20 client.6477 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58fe82e740 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5625f4f2ab90 0x5625f4f2afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58fe82e740 4 client.6477 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58fe82e740 1 -- 172.21.15.38:0/401340030 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5625f4e6ff90 con 0x5625f4f2ab90 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58f98c5640 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5625f4f2ab90 0x5625f4f2afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58d4ff9640 10 client.6477 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.143+0000 7f58f98c5640 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5625f4f2ab90 0x5625f4f2afa0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1976e20832080bbb server_cookie=369171d5334be37d in_seq=0 out_seq=0 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58d4ff9640 1 -- 172.21.15.38:0/401340030 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5625f4e6ff90 con 0x5625f4f2ab90 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58d4ff9640 10 client.6477 handle_command_reply: tid=0 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58fe82e740 1 client.6477 shutdown 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58fe82e740 1 -- 172.21.15.38:0/401340030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 msgr2=0x5625f4f47ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58fe82e740 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 0x5625f4f47ec0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f58ec055730 tx=0x7f58ec05d370 comp rx=0 tx=0).stop 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58fe82e740 1 -- 172.21.15.38:0/401340030 shutdown_connections 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58fe82e740 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5625f4f2ab90 0x5625f4f2afa0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58fe82e740 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 0x5625f4f47ec0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58fe82e740 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f273c0 0x5625f4f4c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58d4ff9640 0 client.6477 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:15.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58fe82e740 1 --2- 172.21.15.38:0/401340030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f26fe0 0x5625f4f26680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58fe82e740 1 -- 172.21.15.38:0/401340030 >> 172.21.15.38:0/401340030 conn(0x5625f4ef9c40 msgr2=0x5625f4efd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:15.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58fe82e740 1 -- 172.21.15.38:0/401340030 shutdown_connections 2024-04-25T08:35:15.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58fe82e740 1 -- 172.21.15.38:0/401340030 wait complete. 2024-04-25T08:35:15.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.144+0000 7f58fe82e740 20 client.6477 trim_cache size 0 max 16384 2024-04-25T08:35:15.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.160+0000 7f58fe82e740 1 Processor -- start 2024-04-25T08:35:15.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.160+0000 7f58fe82e740 1 -- start start 2024-04-25T08:35:15.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.160+0000 7f58fe82e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f26fe0 0x5625f4efde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.160+0000 7f58fe82e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f273c0 0x5625f4efe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.160+0000 7f58fe82e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 0x5625f4f258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.160+0000 7f58fe82e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f4200930 con 0x5625f4f47a40 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.160+0000 7f58fe82e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41ffa90 con 0x5625f4f273c0 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.160+0000 7f58fe82e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41fee90 con 0x5625f4f26fe0 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.160+0000 7f58fa0c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 0x5625f4f258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.160+0000 7f58fa0c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 0x5625f4f258d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44116/0 (socket says 172.21.15.38:44116) 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58fa0c6640 1 -- 172.21.15.38:0/2934011899 learned_addr learned my addr 172.21.15.38:0/2934011899 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58f90c4640 1 --2- 172.21.15.38:0/2934011899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f273c0 0x5625f4efe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58f98c5640 1 --2- 172.21.15.38:0/2934011899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f26fe0 0x5625f4efde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58fa0c6640 1 -- 172.21.15.38:0/2934011899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f26fe0 msgr2=0x5625f4efde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/2934011899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f26fe0 0x5625f4efde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58fa0c6640 1 -- 172.21.15.38:0/2934011899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f273c0 msgr2=0x5625f4efe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/2934011899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f273c0 0x5625f4efe580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58fa0c6640 1 -- 172.21.15.38:0/2934011899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625f4f2ba10 con 0x5625f4f47a40 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58f90c4640 1 --2- 172.21.15.38:0/2934011899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f273c0 0x5625f4efe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/2934011899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 0x5625f4f258d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f58ec08af50 tx=0x7f58ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d4c05a99ef463f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58d6ffd640 1 -- 172.21.15.38:0/2934011899 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec055570 con 0x5625f4f47a40 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58d6ffd640 1 -- 172.21.15.38:0/2934011899 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec055020 con 0x5625f4f47a40 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58d6ffd640 1 -- 172.21.15.38:0/2934011899 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec0590a0 con 0x5625f4f47a40 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.161+0000 7f58fe82e740 1 -- 172.21.15.38:0/2934011899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 msgr2=0x5625f4f258d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.162+0000 7f58fe82e740 1 --2- 172.21.15.38:0/2934011899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 0x5625f4f258d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f58ec08af50 tx=0x7f58ec017000 comp rx=0 tx=0).stop 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.162+0000 7f58fe82e740 1 -- 172.21.15.38:0/2934011899 shutdown_connections 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.162+0000 7f58fe82e740 1 --2- 172.21.15.38:0/2934011899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f47a40 0x5625f4f258d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.162+0000 7f58fe82e740 1 --2- 172.21.15.38:0/2934011899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f273c0 0x5625f4efe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.162+0000 7f58fe82e740 1 --2- 172.21.15.38:0/2934011899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f26fe0 0x5625f4efde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.162+0000 7f58fe82e740 1 -- 172.21.15.38:0/2934011899 >> 172.21.15.38:0/2934011899 conn(0x5625f4ef9c40 msgr2=0x5625f4efa120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.162+0000 7f58fe82e740 1 -- 172.21.15.38:0/2934011899 shutdown_connections 2024-04-25T08:35:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.162+0000 7f58fe82e740 1 -- 172.21.15.38:0/2934011899 wait complete. 2024-04-25T08:35:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.162+0000 7f58fe82e740 1 -- start start 2024-04-25T08:35:15.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.162+0000 7f58fe82e740 1 Processor -- start 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.163+0000 7f58fe82e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 0x5625f4f574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.163+0000 7f58fe82e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f273c0 0x5625f4f579f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.163+0000 7f58fe82e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f47a40 0x5625f4f42cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.163+0000 7f58fe82e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625f4f68460 con 0x5625f4f26fe0 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.163+0000 7f58fe82e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f41ff490 con 0x5625f4f273c0 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.163+0000 7f58fe82e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625f4f67c70 con 0x5625f4f47a40 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.163+0000 7f58fe82e740 10 client.0 _ll_register_callbacks cb 0x7f58ec058e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.163+0000 7f58f98c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 0x5625f4f574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.163+0000 7f58f98c5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 0x5625f4f574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44132/0 (socket says 172.21.15.38:44132) 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.163+0000 7f58f98c5640 1 -- 172.21.15.38:0/2149932474 learned_addr learned my addr 172.21.15.38:0/2149932474 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.163+0000 7f58f90c4640 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f273c0 0x5625f4f579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.163+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f47a40 0x5625f4f42cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58f98c5640 1 -- 172.21.15.38:0/2149932474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f47a40 msgr2=0x5625f4f42cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58f98c5640 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f47a40 0x5625f4f42cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58f98c5640 1 -- 172.21.15.38:0/2149932474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f273c0 msgr2=0x5625f4f579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58f98c5640 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f273c0 0x5625f4f579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58f98c5640 1 -- 172.21.15.38:0/2149932474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625f4f4a700 con 0x5625f4f26fe0 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58fa0c6640 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f47a40 0x5625f4f42cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58f98c5640 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 0x5625f4f574b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f58f0007010 tx=0x7f58f0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af259f8d61884840 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58d4ff9640 1 -- 172.21.15.38:0/2149932474 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f003e060 con 0x5625f4f26fe0 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58d4ff9640 1 -- 172.21.15.38:0/2149932474 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58f00228e0 con 0x5625f4f26fe0 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58fe82e740 1 -- 172.21.15.38:0/2149932474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58ec055020 con 0x5625f4f26fe0 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58d4ff9640 1 -- 172.21.15.38:0/2149932474 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58f0044050 con 0x5625f4f26fe0 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58fe82e740 10 client.6483 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58fe82e740 1 -- 172.21.15.38:0/2149932474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5625f4efa6d0 con 0x5625f4f26fe0 2024-04-25T08:35:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58d4ff9640 1 -- 172.21.15.38:0/2149932474 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec055020 con 0x5625f4f26fe0 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58d4ff9640 1 -- 172.21.15.38:0/2149932474 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f58f0047020 con 0x5625f4f26fe0 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58fe82e740 10 client.6483 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58fe82e740 10 client.6483 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58fe82e740 20 client.6483 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58fe82e740 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5625f4faa140 0x5625f4f43210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58fe82e740 4 client.6483 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58fe82e740 1 -- 172.21.15.38:0/2149932474 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x5625f4e6a2e0 con 0x5625f4faa140 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.164+0000 7f58f90c4640 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5625f4faa140 0x5625f4f43210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58f90c4640 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5625f4faa140 0x5625f4f43210 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63f4867905aac8c server_cookie=f3567fcd1a6aa3ff in_seq=0 out_seq=0 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58d4ff9640 10 client.6483 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58d4ff9640 1 -- 172.21.15.38:0/2149932474 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5625f4e6a2e0 con 0x5625f4faa140 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58d4ff9640 10 client.6483 handle_command_reply: tid=0 2024-04-25T08:35:15.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58fe82e740 1 client.6483 shutdown 2024-04-25T08:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58fe82e740 1 -- 172.21.15.38:0/2149932474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 msgr2=0x5625f4f574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58fe82e740 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 0x5625f4f574b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f58f0007010 tx=0x7f58f0001a40 comp rx=0 tx=0).stop 2024-04-25T08:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58fe82e740 1 -- 172.21.15.38:0/2149932474 shutdown_connections 2024-04-25T08:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58fe82e740 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5625f4faa140 0x5625f4f43210 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58fe82e740 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5625f4f47a40 0x5625f4f42cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58fe82e740 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5625f4f273c0 0x5625f4f579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58d4ff9640 0 client.6483 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58fe82e740 1 --2- 172.21.15.38:0/2149932474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5625f4f26fe0 0x5625f4f574b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.165+0000 7f58fe82e740 1 -- 172.21.15.38:0/2149932474 >> 172.21.15.38:0/2149932474 conn(0x5625f4ef9c40 msgr2=0x5625f4efd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fe82e740 1 -- 172.21.15.38:0/2149932474 shutdown_connections 2024-04-25T08:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fe82e740 1 -- 172.21.15.38:0/2149932474 wait complete. 2024-04-25T08:35:15.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fe82e740 20 client.6483 trim_cache size 0 max 16384 2024-04-25T08:35:15.167 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:15.167 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:35:15.167 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "0a91dd8a-7080-4ed1-b0f2-ee25e8c00593", 2024-04-25T08:35:15.167 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:35:15.167 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:15.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fbb29640 1 -- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58d806b4d0 msgr2=0x7f58d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fbb29640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58d806b4d0 0x7f58d806d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f58f0005b50 tx=0x7f58f0007040 comp rx=0 tx=0).stop 2024-04-25T08:35:15.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fbb29640 1 -- 172.21.15.38:0/1223072710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40b4ef0 msgr2=0x7f58f40b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fbb29640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40b4ef0 0x7f58f40b5370 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f58ec011c80 tx=0x7f58ec00f580 comp rx=0 tx=0).stop 2024-04-25T08:35:15.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fbb29640 1 -- 172.21.15.38:0/1223072710 shutdown_connections 2024-04-25T08:35:15.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fbb29640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f58f40b4ef0 0x7f58f40b5370 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fbb29640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f58d806b4d0 0x7f58d806d990 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fbb29640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f58f41e6c50 0x7f58f40cc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fbb29640 1 --2- 172.21.15.38:0/1223072710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f58e0004800 0x7f58f415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fbb29640 1 -- 172.21.15.38:0/1223072710 >> 172.21.15.38:0/1223072710 conn(0x7f58f40668f0 msgr2=0x7f58f40b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:15.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fbb29640 1 -- 172.21.15.38:0/1223072710 shutdown_connections 2024-04-25T08:35:15.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.166+0000 7f58fbb29640 1 -- 172.21.15.38:0/1223072710 wait complete. 2024-04-25T08:35:15.171 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '0a91dd8a-7080-4ed1-b0f2-ee25e8c00593', 'mode': 'asynchronous'}' 2024-04-25T08:35:15.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:15.172 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-25T08:35:15.189 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-25T08:35:15.189 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:15.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.336+0000 7f355f153640 1 Processor -- start 2024-04-25T08:35:15.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.336+0000 7f355f153640 1 -- start start 2024-04-25T08:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f355f153640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35580b4b50 0x7f35580ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f355f153640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3558144b70 0x7f35580ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f355f153640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35580b50c0 con 0x7f3558144760 2024-04-25T08:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f355f153640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f355815f380 con 0x7f3558144b70 2024-04-25T08:35:15.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f355f153640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355815f660 con 0x7f35580b4b50 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f355ceef640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f3558144760 0x7f355815faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56704/0 (socket says 172.21.15.38:56704) 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f355ceef640 1 -- 172.21.15.38:0/506740761 learned_addr learned my addr 172.21.15.38:0/506740761 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f3557fff640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35580b4b50 0x7f35580ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f355d6f0640 1 --2- 172.21.15.38:0/506740761 >> v2:172.21.15.107:3300/0 conn(0x7f3558144b70 0x7f35580ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266454104 0 0) 0x7f35580b50c0 con 0x7f3558144760 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3548003520 con 0x7f3558144760 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2031180748 0 0) 0x7f3548003520 con 0x7f3558144760 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35580b4b50 msgr2=0x7f35580ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35580b4b50 0x7f35580ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 >> v2:172.21.15.107:3300/0 conn(0x7f3558144b70 msgr2=0x7f35580ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 --2- 172.21.15.38:0/506740761 >> v2:172.21.15.107:3300/0 conn(0x7f3558144b70 0x7f35580ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35580b53a0 con 0x7f3558144760 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f354c001670 con 0x7f3558144760 2024-04-25T08:35:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f3557fff640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35580b4b50 0x7f35580ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f355d6f0640 1 --2- 172.21.15.38:0/506740761 >> v2:172.21.15.107:3300/0 conn(0x7f3558144b70 0x7f35580ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 >> v1:172.21.15.38:6789/0 conn(0x7f3558144760 legacy=0x7f355815faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35580b4b50 0x7f35480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3548003ef0 0x7f3548004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548004d40 0x7f35480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355815f380 con 0x7f3548004d40 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355815f660 con 0x7f3548003ef0 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.337+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354800b9c0 con 0x7f35580b4b50 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.338+0000 7f355d6f0640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548004d40 0x7f35480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.338+0000 7f3557fff640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3548003ef0 0x7f3548004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.338+0000 7f355ceef640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35580b4b50 0x7f35480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.338+0000 7f355d6f0640 1 -- 172.21.15.38:0/506740761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35580b4b50 msgr2=0x7f35480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.338+0000 7f355d6f0640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f35580b4b50 0x7f35480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.338+0000 7f355d6f0640 1 -- 172.21.15.38:0/506740761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3548003ef0 msgr2=0x7f3548004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.338+0000 7f355d6f0640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3548003ef0 0x7f3548004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.338+0000 7f355d6f0640 1 -- 172.21.15.38:0/506740761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3548003520 con 0x7f3548004d40 2024-04-25T08:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.339+0000 7f355d6f0640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548004d40 0x7f35480051c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f355000e350 tx=0x7f3548005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2817dc71cf27fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.339+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550009de0 con 0x7f3548004d40 2024-04-25T08:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.339+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355002e660 con 0x7f3548004d40 2024-04-25T08:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.339+0000 7f35577fe640 1 -- 172.21.15.38:0/506740761 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355002ea10 con 0x7f3548004d40 2024-04-25T08:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.339+0000 7f355f153640 1 -- 172.21.15.38:0/506740761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548004d40 msgr2=0x7f35480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.339+0000 7f355f153640 1 --2- 172.21.15.38:0/506740761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3548004d40 0x7f35480051c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f355000e350 tx=0x7f3548005830 comp rx=0 tx=0).stop 2024-04-25T08:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.339+0000 7f355d6f0640 1 -- 172.21.15.38:0/506740761 reap_dead start 2024-04-25T08:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.339+0000 7f355f153640 1 -- 172.21.15.38:0/506740761 shutdown_connections 2024-04-25T08:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.339+0000 7f355f153640 1 -- 172.21.15.38:0/506740761 >> 172.21.15.38:0/506740761 conn(0x7f355806b210 msgr2=0x7f355806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.339+0000 7f355f153640 1 -- 172.21.15.38:0/506740761 shutdown_connections 2024-04-25T08:35:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.339+0000 7f355f153640 1 -- 172.21.15.38:0/506740761 wait complete. 2024-04-25T08:35:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.340+0000 7f355f153640 1 Processor -- start 2024-04-25T08:35:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.340+0000 7f355f153640 1 -- start start 2024-04-25T08:35:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.341+0000 7f355f153640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f355815b0c0 0x7f355813b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.341+0000 7f355f153640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3558138b40 0x7f3558138fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.341+0000 7f355f153640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f355813d540 0x7f355813d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.341+0000 7f355f153640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3558145280 con 0x7f355813d540 2024-04-25T08:35:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.341+0000 7f355f153640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3558160200 con 0x7f355815b0c0 2024-04-25T08:35:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.341+0000 7f355f153640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3558145400 con 0x7f3558138b40 2024-04-25T08:35:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.341+0000 7f355d6f0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f355813d540 0x7f355813d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.341+0000 7f355d6f0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f355813d540 0x7f355813d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44148/0 (socket says 172.21.15.38:44148) 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.341+0000 7f355d6f0640 1 -- 172.21.15.38:0/2639592436 learned_addr learned my addr 172.21.15.38:0/2639592436 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f355ceef640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f355815b0c0 0x7f355813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f3557fff640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3558138b40 0x7f3558138fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f355d6f0640 1 -- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3558138b40 msgr2=0x7f3558138fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f355d6f0640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3558138b40 0x7f3558138fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f355d6f0640 1 -- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f355815b0c0 msgr2=0x7f355813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f355d6f0640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f355815b0c0 0x7f355813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f355d6f0640 1 -- 172.21.15.38:0/2639592436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35581395c0 con 0x7f355813d540 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f355d6f0640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f355813d540 0x7f355813d9c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f355000e470 tx=0x7f3550009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f35557fa640 1 -- 172.21.15.38:0/2639592436 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550009de0 con 0x7f355813d540 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f35557fa640 1 -- 172.21.15.38:0/2639592436 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355002e460 con 0x7f355813d540 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f355f153640 1 -- 172.21.15.38:0/2639592436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355813a3a0 con 0x7f355813d540 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f35557fa640 1 -- 172.21.15.38:0/2639592436 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550003bd0 con 0x7f355813d540 2024-04-25T08:35:15.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.342+0000 7f355f153640 1 -- 172.21.15.38:0/2639592436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355813a960 con 0x7f355813d540 2024-04-25T08:35:15.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.343+0000 7f355f153640 1 -- 172.21.15.38:0/2639592436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3558068e80 con 0x7f355813d540 2024-04-25T08:35:15.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.344+0000 7f35557fa640 1 -- 172.21.15.38:0/2639592436 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3550048030 con 0x7f355813d540 2024-04-25T08:35:15.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.344+0000 7f35557fa640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f353c06b4d0 0x7f353c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.344+0000 7f35557fa640 1 -- 172.21.15.38:0/2639592436 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f35500ab4b0 con 0x7f355813d540 2024-04-25T08:35:15.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.344+0000 7f355ceef640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f353c06b4d0 0x7f353c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.345+0000 7f355ceef640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f353c06b4d0 0x7f353c06d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f355815f350 tx=0x7f354c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.348+0000 7f35557fa640 1 -- 172.21.15.38:0/2639592436 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f355004f020 con 0x7f355813d540 2024-04-25T08:35:15.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.567+0000 7f355f153640 1 -- 172.21.15.38:0/2639592436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35581386d0 con 0x7f355813d540 2024-04-25T08:35:15.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.567+0000 7f35557fa640 1 -- 172.21.15.38:0/2639592436 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f3550032030 con 0x7f355813d540 2024-04-25T08:35:15.562 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:15.562 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:35:12.877254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:15.562 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-04-25T08:35:15.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.569+0000 7f355f153640 1 -- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f353c06b4d0 msgr2=0x7f353c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.569+0000 7f355f153640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f353c06b4d0 0x7f353c06d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f355815f350 tx=0x7f354c009040 comp rx=0 tx=0).stop 2024-04-25T08:35:15.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.569+0000 7f355f153640 1 -- 172.21.15.38:0/2639592436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f355813d540 msgr2=0x7f355813d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.569+0000 7f355f153640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f355813d540 0x7f355813d9c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f355000e470 tx=0x7f3550009750 comp rx=0 tx=0).stop 2024-04-25T08:35:15.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.569+0000 7f355f153640 1 -- 172.21.15.38:0/2639592436 shutdown_connections 2024-04-25T08:35:15.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.569+0000 7f355f153640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f355813d540 0x7f355813d9c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.569+0000 7f355f153640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f353c06b4d0 0x7f353c06d990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.569+0000 7f355f153640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3558138b40 0x7f3558138fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.569+0000 7f355f153640 1 --2- 172.21.15.38:0/2639592436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f355815b0c0 0x7f355813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.569+0000 7f355f153640 1 -- 172.21.15.38:0/2639592436 >> 172.21.15.38:0/2639592436 conn(0x7f355806b210 msgr2=0x7f35580b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:15.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.569+0000 7f355f153640 1 -- 172.21.15.38:0/2639592436 shutdown_connections 2024-04-25T08:35:15.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.569+0000 7f355f153640 1 -- 172.21.15.38:0/2639592436 wait complete. 2024-04-25T08:35:15.575 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-25T08:35:15.685 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:15.685 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:35:15.686 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:15.694 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:35:15.695 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:35:15.695 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:15.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:15.710 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T08:35:15.777 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:35:15.777 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:35:15.777 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939081 s, 112 MB/s 2024-04-25T08:35:15.778 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T08:35:15.778 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:15.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.926+0000 7f6ff7d90640 1 Processor -- start 2024-04-25T08:35:15.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.926+0000 7f6ff7d90640 1 -- start start 2024-04-25T08:35:15.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.927+0000 7f6ff7d90640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6ff009acb0 0x7f6ff00cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.927+0000 7f6ff7d90640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff0161540 con 0x7f6ff006cdf0 2024-04-25T08:35:15.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.927+0000 7f6ff7d90640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff009a520 con 0x7f6ff00cd8d0 2024-04-25T08:35:15.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.927+0000 7f6ff7d90640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff009a820 con 0x7f6ff009acb0 2024-04-25T08:35:15.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.927+0000 7f6ff532b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6ff009acb0 0x7f6ff00cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.927+0000 7f6ff632d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6ff006cdf0 0x7f6ff00cc200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56712/0 (socket says 172.21.15.38:56712) 2024-04-25T08:35:15.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.927+0000 7f6ff632d640 1 -- 172.21.15.38:0/3761256430 learned_addr learned my addr 172.21.15.38:0/3761256430 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:15.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.927+0000 7f6ff532b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6ff009acb0 0x7f6ff00cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44164/0 (socket says 172.21.15.38:44164) 2024-04-25T08:35:15.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.927+0000 7f6ff532b640 1 -- 172.21.15.38:0/3761256430 >> v1:172.21.15.107:6789/0 conn(0x7f6ff00cd8d0 legacy=0x7f6ff006c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:15.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.927+0000 7f6ff532b640 1 -- 172.21.15.38:0/3761256430 >> v1:172.21.15.38:6789/0 conn(0x7f6ff006cdf0 legacy=0x7f6ff00cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.927+0000 7f6ff532b640 1 -- 172.21.15.38:0/3761256430 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff006be90 con 0x7f6ff009acb0 2024-04-25T08:35:15.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.927+0000 7f6ff4b2a640 1 -- 172.21.15.38:0/3761256430 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231543172 0 0) 0x7f6ff0161540 con 0x7f6ff006cdf0 2024-04-25T08:35:15.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.928+0000 7f6ff532b640 1 --2- 172.21.15.38:0/3761256430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff009acb0 0x7f6ff00cbcc0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6fe00097a0 tx=0x7f6fe002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6226e5da46deabf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.928+0000 7f6ff4b2a640 1 -- 172.21.15.38:0/3761256430 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0037070 con 0x7f6ff009acb0 2024-04-25T08:35:15.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.928+0000 7f6ff4b2a640 1 -- 172.21.15.38:0/3761256430 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe0032440 con 0x7f6ff009acb0 2024-04-25T08:35:15.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.928+0000 7f6ff4b2a640 1 -- 172.21.15.38:0/3761256430 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0032810 con 0x7f6ff009acb0 2024-04-25T08:35:15.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.929+0000 7f6ff7d90640 1 -- 172.21.15.38:0/3761256430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff009acb0 msgr2=0x7f6ff00cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.929+0000 7f6ff7d90640 1 --2- 172.21.15.38:0/3761256430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff009acb0 0x7f6ff00cbcc0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6fe00097a0 tx=0x7f6fe002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.929+0000 7f6ff7d90640 1 -- 172.21.15.38:0/3761256430 shutdown_connections 2024-04-25T08:35:15.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.929+0000 7f6ff7d90640 1 --2- 172.21.15.38:0/3761256430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff009acb0 0x7f6ff00cbcc0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.929+0000 7f6ff7d90640 1 -- 172.21.15.38:0/3761256430 >> 172.21.15.38:0/3761256430 conn(0x7f6ff00b3160 msgr2=0x7f6ff00b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:15.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.929+0000 7f6ff7d90640 1 -- 172.21.15.38:0/3761256430 shutdown_connections 2024-04-25T08:35:15.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.929+0000 7f6ff7d90640 1 -- 172.21.15.38:0/3761256430 wait complete. 2024-04-25T08:35:15.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.930+0000 7f6ff7d90640 1 Processor -- start 2024-04-25T08:35:15.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.930+0000 7f6ff7d90640 1 -- start start 2024-04-25T08:35:15.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff7d90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff006cdf0 0x7f6ff0152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff7d90640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ff009acb0 0x7f6ff00ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff7d90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff00cd8d0 0x7f6ff00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff7d90640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff006c410 con 0x7f6ff006cdf0 2024-04-25T08:35:15.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff7d90640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00d0ad0 con 0x7f6ff00cd8d0 2024-04-25T08:35:15.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff7d90640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff00cc910 con 0x7f6ff009acb0 2024-04-25T08:35:15.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff5b2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff006cdf0 0x7f6ff0152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff5b2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff006cdf0 0x7f6ff0152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44168/0 (socket says 172.21.15.38:44168) 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff5b2c640 1 -- 172.21.15.38:0/1537871942 learned_addr learned my addr 172.21.15.38:0/1537871942 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff632d640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff00cd8d0 0x7f6ff00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff532b640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ff009acb0 0x7f6ff00ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff5b2c640 1 -- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ff009acb0 msgr2=0x7f6ff00ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff5b2c640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ff009acb0 0x7f6ff00ca870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff5b2c640 1 -- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff00cd8d0 msgr2=0x7f6ff00cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff5b2c640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff00cd8d0 0x7f6ff00cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff5b2c640 1 -- 172.21.15.38:0/1537871942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff0153410 con 0x7f6ff006cdf0 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff5b2c640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff006cdf0 0x7f6ff0152ed0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00b420 tx=0x7f6fec00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6fe67fc640 1 -- 172.21.15.38:0/1537871942 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec011830 con 0x7f6ff006cdf0 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6fe67fc640 1 -- 172.21.15.38:0/1537871942 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec016070 con 0x7f6ff006cdf0 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6fe67fc640 1 -- 172.21.15.38:0/1537871942 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec011c10 con 0x7f6ff006cdf0 2024-04-25T08:35:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff7d90640 1 -- 172.21.15.38:0/1537871942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff0150540 con 0x7f6ff006cdf0 2024-04-25T08:35:15.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.931+0000 7f6ff7d90640 1 -- 172.21.15.38:0/1537871942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff00af080 con 0x7f6ff006cdf0 2024-04-25T08:35:15.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.932+0000 7f6ff7d90640 1 -- 172.21.15.38:0/1537871942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff0068e80 con 0x7f6ff006cdf0 2024-04-25T08:35:15.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.933+0000 7f6fe67fc640 1 -- 172.21.15.38:0/1537871942 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fec01b460 con 0x7f6ff006cdf0 2024-04-25T08:35:15.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.933+0000 7f6fe67fc640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6fd406b4d0 0x7f6fd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:15.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.933+0000 7f6fe67fc640 1 -- 172.21.15.38:0/1537871942 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6fec089ff0 con 0x7f6ff006cdf0 2024-04-25T08:35:15.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.933+0000 7f6ff532b640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6fd406b4d0 0x7f6fd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:15.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.934+0000 7f6ff532b640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6fd406b4d0 0x7f6fd406d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0034000 tx=0x7f6fe002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:15.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:15.937+0000 7f6fe67fc640 1 -- 172.21.15.38:0/1537871942 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fec020020 con 0x7f6ff006cdf0 2024-04-25T08:35:16.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.159+0000 7f6ff7d90640 1 -- 172.21.15.38:0/1537871942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ff006c630 con 0x7f6ff006cdf0 2024-04-25T08:35:16.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.160+0000 7f6fe67fc640 1 -- 172.21.15.38:0/1537871942 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f6fec058bb0 con 0x7f6ff006cdf0 2024-04-25T08:35:16.155 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:16.155 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:35:12.877254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":10,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:16.155 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-04-25T08:35:16.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.161+0000 7f6ff7d90640 1 -- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6fd406b4d0 msgr2=0x7f6fd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.161+0000 7f6ff7d90640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6fd406b4d0 0x7f6fd406d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0034000 tx=0x7f6fe002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:16.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.161+0000 7f6ff7d90640 1 -- 172.21.15.38:0/1537871942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff006cdf0 msgr2=0x7f6ff0152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.161+0000 7f6ff7d90640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff006cdf0 0x7f6ff0152ed0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6fec00b420 tx=0x7f6fec00b8e0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.162+0000 7f6ff7d90640 1 -- 172.21.15.38:0/1537871942 shutdown_connections 2024-04-25T08:35:16.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.162+0000 7f6ff7d90640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ff00cd8d0 0x7f6ff00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.162+0000 7f6ff7d90640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6fd406b4d0 0x7f6fd406d990 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.162+0000 7f6ff7d90640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ff009acb0 0x7f6ff00ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.162+0000 7f6ff7d90640 1 --2- 172.21.15.38:0/1537871942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ff006cdf0 0x7f6ff0152ed0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.162+0000 7f6ff7d90640 1 -- 172.21.15.38:0/1537871942 >> 172.21.15.38:0/1537871942 conn(0x7f6ff00b3160 msgr2=0x7f6ff006f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.162+0000 7f6ff7d90640 1 -- 172.21.15.38:0/1537871942 shutdown_connections 2024-04-25T08:35:16.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.162+0000 7f6ff7d90640 1 -- 172.21.15.38:0/1537871942 wait complete. 2024-04-25T08:35:16.167 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T08:35:16.277 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:16.277 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:35:16.277 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:16.286 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:35:16.286 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-04-25T08:35:16.286 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:35:16.286 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-04-25T08:35:16.286 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:16.286 DEBUG:teuthology.orchestra.run.smithi038:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-25T08:35:16.287 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:35:16.354 INFO:teuthology.orchestra.run.smithi038.stdout:1099511630414 2024-04-25T08:35:16.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:35:16.363 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:16.363 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T08:35:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.511+0000 7f610faee640 1 Processor -- start 2024-04-25T08:35:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.511+0000 7f610faee640 1 -- start start 2024-04-25T08:35:16.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.512+0000 7f610faee640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f610815de60 0x7f61080d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.512+0000 7f610faee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61080cf7b0 0x7f61080b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.512+0000 7f610faee640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61080ce710 con 0x7f61080d1e00 2024-04-25T08:35:16.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.512+0000 7f610faee640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61080ce4e0 con 0x7f610815de60 2024-04-25T08:35:16.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.512+0000 7f610faee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61080cf320 con 0x7f61080cf7b0 2024-04-25T08:35:16.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.512+0000 7f610d089640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f610815de60 0x7f61080d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.512+0000 7f610d089640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f610815de60 0x7f61080d42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44176/0 (socket says 172.21.15.38:44176) 2024-04-25T08:35:16.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.512+0000 7f610d089640 1 -- 172.21.15.38:0/2008647172 learned_addr learned my addr 172.21.15.38:0/2008647172 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.513+0000 7f610e08b640 1 --2- 172.21.15.38:0/2008647172 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61080cf7b0 0x7f61080b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.513+0000 7f610d089640 1 -- 172.21.15.38:0/2008647172 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61080cf7b0 msgr2=0x7f61080b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.513+0000 7f610d089640 1 --2- 172.21.15.38:0/2008647172 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61080cf7b0 0x7f61080b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.513+0000 7f610d089640 1 -- 172.21.15.38:0/2008647172 >> v1:172.21.15.107:6789/0 conn(0x7f61080d1e00 legacy=0x7f61080d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.513+0000 7f610d089640 1 -- 172.21.15.38:0/2008647172 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61080ce9f0 con 0x7f610815de60 2024-04-25T08:35:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.513+0000 7f610d089640 1 --2- 172.21.15.38:0/2008647172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f610815de60 0x7f61080d42a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009870 tx=0x7f60fc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5eb5f5247d43144 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.514+0000 7f610c888640 1 -- 172.21.15.38:0/2008647172 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc037070 con 0x7f610815de60 2024-04-25T08:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.514+0000 7f610c888640 1 -- 172.21.15.38:0/2008647172 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60fc032440 con 0x7f610815de60 2024-04-25T08:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.514+0000 7f610c888640 1 -- 172.21.15.38:0/2008647172 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc032810 con 0x7f610815de60 2024-04-25T08:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.514+0000 7f610faee640 1 -- 172.21.15.38:0/2008647172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f610815de60 msgr2=0x7f61080d42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.514+0000 7f610faee640 1 --2- 172.21.15.38:0/2008647172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f610815de60 0x7f61080d42a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f60fc009870 tx=0x7f60fc02a210 comp rx=0 tx=0).stop 2024-04-25T08:35:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.514+0000 7f610faee640 1 -- 172.21.15.38:0/2008647172 shutdown_connections 2024-04-25T08:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.514+0000 7f610faee640 1 --2- 172.21.15.38:0/2008647172 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61080cf7b0 0x7f61080b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.514+0000 7f610faee640 1 --2- 172.21.15.38:0/2008647172 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f610815de60 0x7f61080d42a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.514+0000 7f610faee640 1 -- 172.21.15.38:0/2008647172 >> 172.21.15.38:0/2008647172 conn(0x7f610808b000 msgr2=0x7f61080d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.514+0000 7f610faee640 1 -- 172.21.15.38:0/2008647172 shutdown_connections 2024-04-25T08:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.514+0000 7f610faee640 1 -- 172.21.15.38:0/2008647172 wait complete. 2024-04-25T08:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.515+0000 7f610faee640 1 Processor -- start 2024-04-25T08:35:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.515+0000 7f610faee640 1 -- start start 2024-04-25T08:35:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.516+0000 7f610faee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61080cf7b0 0x7f610815f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.516+0000 7f610faee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61080d1e00 0x7f610815bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.516+0000 7f610faee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f610815de60 0x7f610815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.516+0000 7f610faee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61080d1600 con 0x7f61080d1e00 2024-04-25T08:35:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.516+0000 7f610faee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61080cefb0 con 0x7f610815de60 2024-04-25T08:35:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.516+0000 7f610faee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61080d1780 con 0x7f61080cf7b0 2024-04-25T08:35:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.516+0000 7f610d089640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61080d1e00 0x7f610815bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.516+0000 7f610d089640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61080d1e00 0x7f610815bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44190/0 (socket says 172.21.15.38:44190) 2024-04-25T08:35:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.516+0000 7f610d089640 1 -- 172.21.15.38:0/89271736 learned_addr learned my addr 172.21.15.38:0/89271736 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.517+0000 7f610e08b640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f610815de60 0x7f610815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.517+0000 7f610d089640 1 -- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61080cf7b0 msgr2=0x7f610815f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.517+0000 7f610d089640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61080cf7b0 0x7f610815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.517+0000 7f610d089640 1 -- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f610815de60 msgr2=0x7f610815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.517+0000 7f610d089640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f610815de60 0x7f610815c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.517+0000 7f610d089640 1 -- 172.21.15.38:0/89271736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61081555b0 con 0x7f61080d1e00 2024-04-25T08:35:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.517+0000 7f610d089640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61080d1e00 0x7f610815bf60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0090b0 tx=0x7f60fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.517+0000 7f61067fc640 1 -- 172.21.15.38:0/89271736 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc041020 con 0x7f61080d1e00 2024-04-25T08:35:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.517+0000 7f61067fc640 1 -- 172.21.15.38:0/89271736 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60fc0306c0 con 0x7f61080d1e00 2024-04-25T08:35:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.517+0000 7f61067fc640 1 -- 172.21.15.38:0/89271736 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc037040 con 0x7f61080d1e00 2024-04-25T08:35:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.517+0000 7f610faee640 1 -- 172.21.15.38:0/89271736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f610815fd50 con 0x7f61080d1e00 2024-04-25T08:35:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.517+0000 7f610faee640 1 -- 172.21.15.38:0/89271736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6108160310 con 0x7f61080d1e00 2024-04-25T08:35:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.518+0000 7f61067fc640 1 -- 172.21.15.38:0/89271736 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60fc044020 con 0x7f61080d1e00 2024-04-25T08:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.519+0000 7f61067fc640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60f406b4d0 0x7f60f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.519+0000 7f61067fc640 1 -- 172.21.15.38:0/89271736 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f60fc0aab50 con 0x7f61080d1e00 2024-04-25T08:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.519+0000 7f610d88a640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60f406b4d0 0x7f60f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.520+0000 7f610d88a640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60f406b4d0 0x7f60f406d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6100000f60 tx=0x7f61000079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.520+0000 7f61127eb740 1 Processor -- start 2024-04-25T08:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.520+0000 7f61127eb740 1 -- start start 2024-04-25T08:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f61127eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea682b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f61127eb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea687f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f61127eb740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea38700 0x55ae2ea3bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f61127eb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e9a1e00 con 0x55ae2ea65e80 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f61127eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e94ad20 con 0x55ae2ea65aa0 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f61127eb740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e99dea0 con 0x55ae2ea38700 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f610d089640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea687f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f610d089640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea687f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44206/0 (socket says 172.21.15.38:44206) 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f610d089640 1 -- 172.21.15.38:0/2821538586 learned_addr learned my addr 172.21.15.38:0/2821538586 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f610d88a640 1 --2- 172.21.15.38:0/2821538586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f610e08b640 1 --2- 172.21.15.38:0/2821538586 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea38700 0x55ae2ea3bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f610d089640 1 -- 172.21.15.38:0/2821538586 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea38700 msgr2=0x55ae2ea3bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f610d089640 1 --2- 172.21.15.38:0/2821538586 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea38700 0x55ae2ea3bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f610d089640 1 -- 172.21.15.38:0/2821538586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 msgr2=0x55ae2ea682b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f610d089640 1 --2- 172.21.15.38:0/2821538586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea682b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f610d089640 1 -- 172.21.15.38:0/2821538586 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae2e96cdf0 con 0x55ae2ea65e80 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.521+0000 7f610e08b640 1 --2- 172.21.15.38:0/2821538586 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea38700 0x55ae2ea3bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.522+0000 7f610d089640 1 --2- 172.21.15.38:0/2821538586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea687f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0813b0 tx=0x7f60fc078b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f841cd4171965b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.522+0000 7f60ef7fe640 1 -- 172.21.15.38:0/2821538586 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc0a7070 con 0x55ae2ea65e80 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.522+0000 7f60ef7fe640 1 -- 172.21.15.38:0/2821538586 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60fc0791d0 con 0x55ae2ea65e80 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.522+0000 7f60ef7fe640 1 -- 172.21.15.38:0/2821538586 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60fc07d320 con 0x55ae2ea65e80 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.522+0000 7f61127eb740 1 -- 172.21.15.38:0/2821538586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 msgr2=0x55ae2ea687f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.522+0000 7f61127eb740 1 --2- 172.21.15.38:0/2821538586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea687f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0813b0 tx=0x7f60fc078b30 comp rx=0 tx=0).stop 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.522+0000 7f61127eb740 1 -- 172.21.15.38:0/2821538586 shutdown_connections 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.522+0000 7f61127eb740 1 --2- 172.21.15.38:0/2821538586 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea38700 0x55ae2ea3bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.522+0000 7f61127eb740 1 --2- 172.21.15.38:0/2821538586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea687f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.522+0000 7f61127eb740 1 --2- 172.21.15.38:0/2821538586 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea682b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.522+0000 7f61127eb740 1 -- 172.21.15.38:0/2821538586 >> 172.21.15.38:0/2821538586 conn(0x55ae2ea3a970 msgr2=0x55ae2ea832b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.523+0000 7f61127eb740 1 -- 172.21.15.38:0/2821538586 shutdown_connections 2024-04-25T08:35:16.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.523+0000 7f61127eb740 1 -- 172.21.15.38:0/2821538586 wait complete. 2024-04-25T08:35:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.523+0000 7f61127eb740 1 -- start start 2024-04-25T08:35:16.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.523+0000 7f61127eb740 1 Processor -- start 2024-04-25T08:35:16.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f61127eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea87220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f61127eb740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea65e80 0x55ae2ea7f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f61127eb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea641f0 0x55ae2ea7f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f61127eb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2ea3db20 con 0x55ae2ea641f0 2024-04-25T08:35:16.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f61127eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2e940590 con 0x55ae2ea65aa0 2024-04-25T08:35:16.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f61127eb740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae2ea3de00 con 0x55ae2ea65e80 2024-04-25T08:35:16.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f61127eb740 10 client.0 _ll_register_callbacks cb 0x55ae2e94ce30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:16.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f610e08b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea641f0 0x55ae2ea7f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f610e08b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea641f0 0x55ae2ea7f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44212/0 (socket says 172.21.15.38:44212) 2024-04-25T08:35:16.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f610e08b640 1 -- 172.21.15.38:0/1678846803 learned_addr learned my addr 172.21.15.38:0/1678846803 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f610d089640 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea65e80 0x55ae2ea7f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f610d88a640 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea87220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f610e08b640 1 -- 172.21.15.38:0/1678846803 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea65e80 msgr2=0x55ae2ea7f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f610e08b640 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea65e80 0x55ae2ea7f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f610e08b640 1 -- 172.21.15.38:0/1678846803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 msgr2=0x55ae2ea87220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f610e08b640 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea87220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.524+0000 7f610e08b640 1 -- 172.21.15.38:0/1678846803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae2e6b63f0 con 0x55ae2ea641f0 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f610e08b640 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea641f0 0x55ae2ea7f7b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f60f80079d0 tx=0x7f60f8007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f4a5943172dc6f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f60ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f60ecff9640 1 -- 172.21.15.38:0/1678846803 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f801a020 con 0x55ae2ea641f0 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f60ecff9640 1 -- 172.21.15.38:0/1678846803 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f60f8011a20 con 0x55ae2ea641f0 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f61127eb740 1 -- 172.21.15.38:0/1678846803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60fc0791d0 con 0x55ae2ea641f0 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f60ecff9640 1 -- 172.21.15.38:0/1678846803 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60f8024050 con 0x55ae2ea641f0 2024-04-25T08:35:16.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f61127eb740 10 client.6510 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f61127eb740 1 -- 172.21.15.38:0/1678846803 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55ae2e7807d0 con 0x55ae2ea641f0 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f60ecff9640 1 -- 172.21.15.38:0/1678846803 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60fc0791d0 con 0x55ae2ea641f0 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f60ecff9640 1 -- 172.21.15.38:0/1678846803 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f60f8027020 con 0x55ae2ea641f0 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f61127eb740 10 client.6510 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f61127eb740 10 client.6510 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f61127eb740 20 client.6510 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f61127eb740 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ae2ea85230 0x55ae2ea85610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f61127eb740 4 client.6510 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f61127eb740 1 -- 172.21.15.38:0/1678846803 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae2e9af340 con 0x55ae2ea85230 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.525+0000 7f610d88a640 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ae2ea85230 0x55ae2ea85610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.526+0000 7f60ecff9640 10 client.6510 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.526+0000 7f610d88a640 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ae2ea85230 0x55ae2ea85610 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7588d1437ca3c8d1 server_cookie=698b558e4f10198d in_seq=0 out_seq=0 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.526+0000 7f60ecff9640 1 -- 172.21.15.38:0/1678846803 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ae2e9af340 con 0x55ae2ea85230 2024-04-25T08:35:16.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.526+0000 7f60ecff9640 10 client.6510 handle_command_reply: tid=0 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.526+0000 7f61127eb740 1 client.6510 shutdown 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.526+0000 7f61127eb740 1 -- 172.21.15.38:0/1678846803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea641f0 msgr2=0x55ae2ea7f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.526+0000 7f61127eb740 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea641f0 0x55ae2ea7f7b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f60f80079d0 tx=0x7f60f8007dd0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.526+0000 7f61127eb740 1 -- 172.21.15.38:0/1678846803 shutdown_connections 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.527+0000 7f61127eb740 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ae2ea85230 0x55ae2ea85610 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.527+0000 7f61127eb740 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea641f0 0x55ae2ea7f7b0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.527+0000 7f61127eb740 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea65e80 0x55ae2ea7f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.527+0000 7f60ecff9640 0 client.6510 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.527+0000 7f61127eb740 1 --2- 172.21.15.38:0/1678846803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea87220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.527+0000 7f61127eb740 1 -- 172.21.15.38:0/1678846803 >> 172.21.15.38:0/1678846803 conn(0x55ae2ea38700 msgr2=0x55ae2ea68270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.527+0000 7f61127eb740 1 -- 172.21.15.38:0/1678846803 shutdown_connections 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.527+0000 7f61127eb740 1 -- 172.21.15.38:0/1678846803 wait complete. 2024-04-25T08:35:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.527+0000 7f61127eb740 20 client.6510 trim_cache size 0 max 16384 2024-04-25T08:35:16.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.541+0000 7f61127eb740 1 Processor -- start 2024-04-25T08:35:16.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.541+0000 7f61127eb740 1 -- start start 2024-04-25T08:35:16.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f61127eb740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea641f0 0x55ae2ea3e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f61127eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea68dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f61127eb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea69300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f61127eb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6108200320 con 0x55ae2ea65e80 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f61127eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61081ff4c0 con 0x55ae2ea65aa0 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f61127eb740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61081fe8c0 con 0x55ae2ea641f0 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f610e08b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea69300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f610e08b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea69300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44226/0 (socket says 172.21.15.38:44226) 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f610e08b640 1 -- 172.21.15.38:0/4119822759 learned_addr learned my addr 172.21.15.38:0/4119822759 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f610d88a640 1 --2- 172.21.15.38:0/4119822759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea641f0 0x55ae2ea3e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f610d089640 1 --2- 172.21.15.38:0/4119822759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea68dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f610e08b640 1 -- 172.21.15.38:0/4119822759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea641f0 msgr2=0x55ae2ea3e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f610e08b640 1 --2- 172.21.15.38:0/4119822759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea641f0 0x55ae2ea3e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f610e08b640 1 -- 172.21.15.38:0/4119822759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 msgr2=0x55ae2ea68dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f610e08b640 1 --2- 172.21.15.38:0/4119822759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea68dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f610e08b640 1 -- 172.21.15.38:0/4119822759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae2ea645d0 con 0x55ae2ea65e80 2024-04-25T08:35:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f610e08b640 1 --2- 172.21.15.38:0/4119822759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea69300 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009090 tx=0x7f60f800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8e0edf08adbd8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f60eeffd640 1 -- 172.21.15.38:0/4119822759 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f802c070 con 0x55ae2ea65e80 2024-04-25T08:35:16.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f60eeffd640 1 -- 172.21.15.38:0/4119822759 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f8011c20 con 0x55ae2ea65e80 2024-04-25T08:35:16.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.542+0000 7f60eeffd640 1 -- 172.21.15.38:0/4119822759 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f801a030 con 0x55ae2ea65e80 2024-04-25T08:35:16.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.543+0000 7f61127eb740 1 -- 172.21.15.38:0/4119822759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 msgr2=0x55ae2ea69300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.543+0000 7f61127eb740 1 --2- 172.21.15.38:0/4119822759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea69300 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009090 tx=0x7f60f800f000 comp rx=0 tx=0).stop 2024-04-25T08:35:16.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.543+0000 7f61127eb740 1 -- 172.21.15.38:0/4119822759 shutdown_connections 2024-04-25T08:35:16.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.543+0000 7f61127eb740 1 --2- 172.21.15.38:0/4119822759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea69300 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.543+0000 7f61127eb740 1 --2- 172.21.15.38:0/4119822759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea68dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.543+0000 7f61127eb740 1 --2- 172.21.15.38:0/4119822759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea641f0 0x55ae2ea3e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.543+0000 7f61127eb740 1 -- 172.21.15.38:0/4119822759 >> 172.21.15.38:0/4119822759 conn(0x55ae2ea38700 msgr2=0x55ae2ea39190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.543+0000 7f61127eb740 1 -- 172.21.15.38:0/4119822759 shutdown_connections 2024-04-25T08:35:16.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.543+0000 7f61127eb740 1 -- 172.21.15.38:0/4119822759 wait complete. 2024-04-25T08:35:16.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.544+0000 7f61127eb740 1 -- start start 2024-04-25T08:35:16.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.544+0000 7f61127eb740 1 Processor -- start 2024-04-25T08:35:16.544 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f61127eb740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea641f0 0x55ae2ea5e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f61127eb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea3bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f61127eb740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea7bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f61127eb740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae2ea85680 con 0x55ae2ea65e80 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f61127eb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61081feec0 con 0x55ae2ea65aa0 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f61127eb740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae2ea3cc30 con 0x55ae2ea641f0 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f61127eb740 10 client.0 _ll_register_callbacks cb 0x7f60fc009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f610e08b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea7bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f610e08b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea7bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44230/0 (socket says 172.21.15.38:44230) 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f610d089640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea3bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f610e08b640 1 -- 172.21.15.38:0/4056526145 learned_addr learned my addr 172.21.15.38:0/4056526145 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f610d88a640 1 --2- 172.21.15.38:0/4056526145 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea641f0 0x55ae2ea5e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f610e08b640 1 -- 172.21.15.38:0/4056526145 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea641f0 msgr2=0x55ae2ea5e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f610e08b640 1 --2- 172.21.15.38:0/4056526145 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea641f0 0x55ae2ea5e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f610e08b640 1 -- 172.21.15.38:0/4056526145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 msgr2=0x55ae2ea3bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f610e08b640 1 --2- 172.21.15.38:0/4056526145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea3bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f610e08b640 1 -- 172.21.15.38:0/4056526145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae2ea5b640 con 0x55ae2ea65e80 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f610e08b640 1 --2- 172.21.15.38:0/4056526145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea7bfc0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f60f800b010 tx=0x7f60f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57858cfaa3f3586 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f60ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.545+0000 7f60ecff9640 1 -- 172.21.15.38:0/4056526145 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f8035070 con 0x55ae2ea65e80 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f60ecff9640 1 -- 172.21.15.38:0/4056526145 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f60f8011a20 con 0x55ae2ea65e80 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f61127eb740 1 -- 172.21.15.38:0/4056526145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f60f8011c20 con 0x55ae2ea65e80 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f60ecff9640 1 -- 172.21.15.38:0/4056526145 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60f8040050 con 0x55ae2ea65e80 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f61127eb740 10 client.6516 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f61127eb740 1 -- 172.21.15.38:0/4056526145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55ae2ea5b310 con 0x55ae2ea65e80 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f60ecff9640 1 -- 172.21.15.38:0/4056526145 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f8011c20 con 0x55ae2ea65e80 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f60ecff9640 1 -- 172.21.15.38:0/4056526145 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f60f8043020 con 0x55ae2ea65e80 2024-04-25T08:35:16.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f61127eb740 10 client.6516 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f61127eb740 10 client.6516 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f61127eb740 20 client.6516 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f61127eb740 1 --2- 172.21.15.38:0/4056526145 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ae2ea3c450 0x55ae2ea7c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f61127eb740 4 client.6516 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f61127eb740 1 -- 172.21.15.38:0/4056526145 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ae2ea86d90 con 0x55ae2ea3c450 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f610d88a640 1 --2- 172.21.15.38:0/4056526145 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ae2ea3c450 0x55ae2ea7c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.546+0000 7f60ecff9640 10 client.6516 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f610d88a640 1 --2- 172.21.15.38:0/4056526145 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ae2ea3c450 0x55ae2ea7c500 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be7953c024f087b7 server_cookie=3ea4116210533187 in_seq=0 out_seq=0 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f60ecff9640 1 -- 172.21.15.38:0/4056526145 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ae2ea86d90 con 0x55ae2ea3c450 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f60ecff9640 10 client.6516 handle_command_reply: tid=0 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f61127eb740 1 client.6516 shutdown 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f61127eb740 1 -- 172.21.15.38:0/4056526145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 msgr2=0x55ae2ea7bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f61127eb740 1 --2- 172.21.15.38:0/4056526145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea7bfc0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f60f800b010 tx=0x7f60f8004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:16.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f61127eb740 1 -- 172.21.15.38:0/4056526145 shutdown_connections 2024-04-25T08:35:16.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f61127eb740 1 --2- 172.21.15.38:0/4056526145 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55ae2ea3c450 0x55ae2ea7c500 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f61127eb740 1 --2- 172.21.15.38:0/4056526145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ae2ea65e80 0x55ae2ea7bfc0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f61127eb740 1 --2- 172.21.15.38:0/4056526145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ae2ea65aa0 0x55ae2ea3bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f60ecff9640 0 client.6516 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:16.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f61127eb740 1 --2- 172.21.15.38:0/4056526145 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55ae2ea641f0 0x55ae2ea5e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f61127eb740 1 -- 172.21.15.38:0/4056526145 >> 172.21.15.38:0/4056526145 conn(0x55ae2ea38700 msgr2=0x55ae2ea82940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f61127eb740 1 -- 172.21.15.38:0/4056526145 shutdown_connections 2024-04-25T08:35:16.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f61127eb740 1 -- 172.21.15.38:0/4056526145 wait complete. 2024-04-25T08:35:16.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.547+0000 7f61127eb740 20 client.6516 trim_cache size 0 max 16384 2024-04-25T08:35:16.548 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-04-25T08:35:16.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.548+0000 7f610faee640 1 -- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60f406b4d0 msgr2=0x7f60f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.548+0000 7f610faee640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60f406b4d0 0x7f60f406d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6100000f60 tx=0x7f61000079b0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.548+0000 7f610faee640 1 -- 172.21.15.38:0/89271736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61080d1e00 msgr2=0x7f610815bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.548+0000 7f610faee640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61080d1e00 0x7f610815bf60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f60fc0090b0 tx=0x7f60fc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.548+0000 7f610faee640 1 -- 172.21.15.38:0/89271736 shutdown_connections 2024-04-25T08:35:16.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.548+0000 7f610faee640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f610815de60 0x7f610815c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.548+0000 7f610faee640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f61080d1e00 0x7f610815bf60 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.548+0000 7f610faee640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60f406b4d0 0x7f60f406d990 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.548+0000 7f610faee640 1 --2- 172.21.15.38:0/89271736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f61080cf7b0 0x7f610815f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.548+0000 7f610faee640 1 -- 172.21.15.38:0/89271736 >> 172.21.15.38:0/89271736 conn(0x7f610808b000 msgr2=0x7f61080d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.548+0000 7f610faee640 1 -- 172.21.15.38:0/89271736 shutdown_connections 2024-04-25T08:35:16.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.548+0000 7f610faee640 1 -- 172.21.15.38:0/89271736 wait complete. 2024-04-25T08:35:16.553 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4e.00000000 parent 2024-04-25T08:35:16.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.600+0000 7f848932ff40 1 Processor -- start 2024-04-25T08:35:16.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.600+0000 7f848932ff40 1 -- start start 2024-04-25T08:35:16.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f848932ff40 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56121a0335f0 0x56121a0ebd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f848932ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a037db0 0x56121a0ec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f848932ff40 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56121a0efe90 0x56121a10f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f848932ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121a1115b0 con 0x56121a037db0 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f848932ff40 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561219fdcdf0 con 0x56121a0335f0 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f848932ff40 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56121a110a60 con 0x56121a0efe90 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f8486631640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a037db0 0x56121a0ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f8486631640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a037db0 0x56121a0ec340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44240/0 (socket says 172.21.15.38:44240) 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f8486631640 1 -- 172.21.15.38:0/4226345849 learned_addr learned my addr 172.21.15.38:0/4226345849 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f8486e32640 1 --2- 172.21.15.38:0/4226345849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56121a0335f0 0x56121a0ebd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f8487633640 1 --2- 172.21.15.38:0/4226345849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56121a0efe90 0x56121a10f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f8486631640 1 -- 172.21.15.38:0/4226345849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56121a0efe90 msgr2=0x56121a10f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f8486631640 1 --2- 172.21.15.38:0/4226345849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56121a0efe90 0x56121a10f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f8486631640 1 -- 172.21.15.38:0/4226345849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56121a0335f0 msgr2=0x56121a0ebd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f8486631640 1 --2- 172.21.15.38:0/4226345849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56121a0335f0 0x56121a0ebd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f8486631640 1 -- 172.21.15.38:0/4226345849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561219f62c50 con 0x56121a037db0 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f8487633640 1 --2- 172.21.15.38:0/4226345849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56121a0efe90 0x56121a10f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.601+0000 7f8486631640 1 --2- 172.21.15.38:0/4226345849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a037db0 0x56121a0ec340 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8470045e10 tx=0x7f847004e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90017372de819780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.602+0000 7f848562f640 1 -- 172.21.15.38:0/4226345849 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470065070 con 0x56121a037db0 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.602+0000 7f848562f640 1 -- 172.21.15.38:0/4226345849 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84700549f0 con 0x56121a037db0 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.602+0000 7f848562f640 1 -- 172.21.15.38:0/4226345849 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470059040 con 0x56121a037db0 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.602+0000 7f848932ff40 1 -- 172.21.15.38:0/4226345849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a037db0 msgr2=0x56121a0ec340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.602+0000 7f848932ff40 1 --2- 172.21.15.38:0/4226345849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a037db0 0x56121a0ec340 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f8470045e10 tx=0x7f847004e070 comp rx=0 tx=0).stop 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.602+0000 7f848932ff40 1 -- 172.21.15.38:0/4226345849 shutdown_connections 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.602+0000 7f848932ff40 1 --2- 172.21.15.38:0/4226345849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56121a0efe90 0x56121a10f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.602+0000 7f848932ff40 1 --2- 172.21.15.38:0/4226345849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a037db0 0x56121a0ec340 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.602+0000 7f848932ff40 1 --2- 172.21.15.38:0/4226345849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56121a0335f0 0x56121a0ebd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.602+0000 7f848932ff40 1 -- 172.21.15.38:0/4226345849 >> 172.21.15.38:0/4226345849 conn(0x56121a035900 msgr2=0x56121a036690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.602+0000 7f848932ff40 1 -- 172.21.15.38:0/4226345849 shutdown_connections 2024-04-25T08:35:16.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.602+0000 7f848932ff40 1 -- 172.21.15.38:0/4226345849 wait complete. 2024-04-25T08:35:16.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.603+0000 7f848932ff40 1 Processor -- start 2024-04-25T08:35:16.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.603+0000 7f848932ff40 1 -- start start 2024-04-25T08:35:16.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.603+0000 7f848932ff40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a0335f0 0x56121a10d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.603+0000 7f848932ff40 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56121a037db0 0x56121a10a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.603+0000 7f848932ff40 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56121a0efe90 0x56121a1a3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.603+0000 7f848932ff40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121a10df80 con 0x56121a0335f0 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.603+0000 7f848932ff40 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56121a0370b0 con 0x56121a037db0 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.603+0000 7f848932ff40 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56121a0ec880 con 0x56121a0efe90 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f8486e32640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a0335f0 0x56121a10d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f8486e32640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a0335f0 0x56121a10d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44244/0 (socket says 172.21.15.38:44244) 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f8486e32640 1 -- 172.21.15.38:0/1828799427 learned_addr learned my addr 172.21.15.38:0/1828799427 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f8487633640 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56121a0efe90 0x56121a1a3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f8486631640 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56121a037db0 0x56121a10a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f8486e32640 1 -- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56121a0efe90 msgr2=0x56121a1a3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f8486e32640 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56121a0efe90 0x56121a1a3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f8486e32640 1 -- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56121a037db0 msgr2=0x56121a10a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f8486e32640 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56121a037db0 0x56121a10a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f8486e32640 1 -- 172.21.15.38:0/1828799427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56121a10ab60 con 0x56121a0335f0 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f8487633640 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56121a0efe90 0x56121a1a3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f8486e32640 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a0335f0 0x56121a10d890 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8480009e20 tx=0x7f8480009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f847ffff640 1 -- 172.21.15.38:0/1828799427 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800143f0 con 0x56121a0335f0 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f847ffff640 1 -- 172.21.15.38:0/1828799427 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8480019070 con 0x56121a0335f0 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f848932ff40 1 -- 172.21.15.38:0/1828799427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56121a10ae40 con 0x56121a0335f0 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f847ffff640 1 -- 172.21.15.38:0/1828799427 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800147d0 con 0x56121a0335f0 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.604+0000 7f848932ff40 1 -- 172.21.15.38:0/1828799427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56121a10a760 con 0x56121a0335f0 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.606+0000 7f847ffff640 1 -- 172.21.15.38:0/1828799427 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8480014970 con 0x56121a0335f0 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.606+0000 7f847ffff640 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8468069eb0 0x7f846806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.606+0000 7f847ffff640 1 -- 172.21.15.38:0/1828799427 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8480088e10 con 0x56121a0335f0 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.606+0000 7f8486631640 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8468069eb0 0x7f846806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.606+0000 7f848932ff40 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x56121a1a38a0 0x56121a10fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.606+0000 7f848932ff40 1 -- 172.21.15.38:0/1828799427 --> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] -- osd_op(unknown.0.0:1 8.2 8:43adcc81:::10000000a4e.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54) v9 -- 0x56121a064250 con 0x56121a1a38a0 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.606+0000 7f8487633640 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x56121a1a38a0 0x56121a10fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.607+0000 7f8486631640 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8468069eb0 0x7f846806c370 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8470045e10 tx=0x7f8470050000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.607+0000 7f8487633640 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x56121a1a38a0 0x56121a10fb70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.608+0000 7f8487633640 1 -- 172.21.15.38:0/1828799427 <== osd.4 v2:172.21.15.107:6800/592699329 1 ==== osd_op_reply(1 10000000a4e.00000000 [rmxattr] v54'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f84780017f0 con 0x56121a1a38a0 2024-04-25T08:35:16.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.608+0000 7f848932ff40 1 -- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x56121a1a38a0 msgr2=0x56121a10fb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.608+0000 7f848932ff40 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x56121a1a38a0 0x56121a10fb70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.608+0000 7f848932ff40 1 -- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8468069eb0 msgr2=0x7f846806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.608+0000 7f848932ff40 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8468069eb0 0x7f846806c370 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8470045e10 tx=0x7f8470050000 comp rx=0 tx=0).stop 2024-04-25T08:35:16.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.608+0000 7f848932ff40 1 -- 172.21.15.38:0/1828799427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a0335f0 msgr2=0x56121a10d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.608+0000 7f848932ff40 1 --2- 172.21.15.38:0/1828799427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56121a0335f0 0x56121a10d890 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8480009e20 tx=0x7f8480009f10 comp rx=0 tx=0).stop 2024-04-25T08:35:16.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.608+0000 7f8487633640 1 -- 172.21.15.38:0/1828799427 reap_dead start 2024-04-25T08:35:16.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.608+0000 7f848932ff40 1 -- 172.21.15.38:0/1828799427 shutdown_connections 2024-04-25T08:35:16.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.608+0000 7f848932ff40 1 -- 172.21.15.38:0/1828799427 >> 172.21.15.38:0/1828799427 conn(0x56121a035900 msgr2=0x56121a036690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.608+0000 7f848932ff40 1 -- 172.21.15.38:0/1828799427 shutdown_connections 2024-04-25T08:35:16.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.608+0000 7f848932ff40 1 -- 172.21.15.38:0/1828799427 wait complete. 2024-04-25T08:35:16.607 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T08:35:16.607 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:16.607 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T08:35:16.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.756+0000 7f5513c93640 1 Processor -- start 2024-04-25T08:35:16.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.756+0000 7f5513c93640 1 -- start start 2024-04-25T08:35:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f5513c93640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f550c0d0320 0x7f550c0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f5513c93640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f550c0b6980 0x7f550c0b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f5513c93640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f550c0d2060 con 0x7f550c0d0700 2024-04-25T08:35:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f5513c93640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f550c0d2330 con 0x7f550c0b6980 2024-04-25T08:35:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f5513c93640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0d2610 con 0x7f550c0d0320 2024-04-25T08:35:16.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f5512230640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f550c0b6980 0x7f550c0b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f551122e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f550c0d0700 0x7f550c0d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56732/0 (socket says 172.21.15.38:56732) 2024-04-25T08:35:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f551122e640 1 -- 172.21.15.38:0/2108170677 learned_addr learned my addr 172.21.15.38:0/2108170677 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f5511a2f640 1 --2- 172.21.15.38:0/2108170677 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f550c0d0320 0x7f550c0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f5512230640 1 -- 172.21.15.38:0/2108170677 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f550c0d0320 msgr2=0x7f550c0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f5512230640 1 --2- 172.21.15.38:0/2108170677 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f550c0d0320 0x7f550c0d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f5512230640 1 -- 172.21.15.38:0/2108170677 >> v1:172.21.15.38:6789/0 conn(0x7f550c0d0700 legacy=0x7f550c0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f5512230640 1 -- 172.21.15.38:0/2108170677 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c0d28f0 con 0x7f550c0b6980 2024-04-25T08:35:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.757+0000 7f5510a2d640 1 -- 172.21.15.38:0/2108170677 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097207910 0 0) 0x7f550c0d2060 con 0x7f550c0d0700 2024-04-25T08:35:16.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.758+0000 7f5512230640 1 --2- 172.21.15.38:0/2108170677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c0b6980 0x7f550c0b6e10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5504009d90 tx=0x7f550402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fa359151c00559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.758+0000 7f5510a2d640 1 -- 172.21.15.38:0/2108170677 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504007740 con 0x7f550c0b6980 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.758+0000 7f5510a2d640 1 -- 172.21.15.38:0/2108170677 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55040078e0 con 0x7f550c0b6980 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.758+0000 7f5510a2d640 1 -- 172.21.15.38:0/2108170677 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504007cb0 con 0x7f550c0b6980 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.759+0000 7f5513c93640 1 -- 172.21.15.38:0/2108170677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c0b6980 msgr2=0x7f550c0b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.759+0000 7f5513c93640 1 --2- 172.21.15.38:0/2108170677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c0b6980 0x7f550c0b6e10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f5504009d90 tx=0x7f550402a6f0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.759+0000 7f5513c93640 1 -- 172.21.15.38:0/2108170677 shutdown_connections 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.759+0000 7f5513c93640 1 --2- 172.21.15.38:0/2108170677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c0b6980 0x7f550c0b6e10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.759+0000 7f5513c93640 1 --2- 172.21.15.38:0/2108170677 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f550c0d0320 0x7f550c0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.759+0000 7f5513c93640 1 -- 172.21.15.38:0/2108170677 >> 172.21.15.38:0/2108170677 conn(0x7f550c0668f0 msgr2=0x7f550c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.759+0000 7f5513c93640 1 -- 172.21.15.38:0/2108170677 shutdown_connections 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.759+0000 7f5513c93640 1 -- 172.21.15.38:0/2108170677 wait complete. 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.760+0000 7f5513c93640 1 Processor -- start 2024-04-25T08:35:16.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.760+0000 7f5513c93640 1 -- start start 2024-04-25T08:35:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5513c93640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f550c0b6980 0x7f550c0c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5513c93640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f550c0d0320 0x7f550c14b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5513c93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c0d0700 0x7f550c147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5513c93640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0d0b40 con 0x7f550c0d0700 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5513c93640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0d2eb0 con 0x7f550c0d0320 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5513c93640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c0b5190 con 0x7f550c0b6980 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5512230640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c0d0700 0x7f550c147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5512230640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c0d0700 0x7f550c147a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44262/0 (socket says 172.21.15.38:44262) 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5512230640 1 -- 172.21.15.38:0/269499926 learned_addr learned my addr 172.21.15.38:0/269499926 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5511a2f640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f550c0b6980 0x7f550c0c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f551122e640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f550c0d0320 0x7f550c14b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5512230640 1 -- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f550c0b6980 msgr2=0x7f550c0c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5512230640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f550c0b6980 0x7f550c0c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5512230640 1 -- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f550c0d0320 msgr2=0x7f550c14b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5512230640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f550c0d0320 0x7f550c14b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5512230640 1 -- 172.21.15.38:0/269499926 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c0b5470 con 0x7f550c0d0700 2024-04-25T08:35:16.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.761+0000 7f5511a2f640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f550c0b6980 0x7f550c0c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:16.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.762+0000 7f5512230640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c0d0700 0x7f550c147a50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5504009eb0 tx=0x7f5504004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.762+0000 7f55027fc640 1 -- 172.21.15.38:0/269499926 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504040070 con 0x7f550c0d0700 2024-04-25T08:35:16.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.762+0000 7f55027fc640 1 -- 172.21.15.38:0/269499926 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5504031bc0 con 0x7f550c0d0700 2024-04-25T08:35:16.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.762+0000 7f5513c93640 1 -- 172.21.15.38:0/269499926 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f550c0b5750 con 0x7f550c0d0700 2024-04-25T08:35:16.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.762+0000 7f55027fc640 1 -- 172.21.15.38:0/269499926 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504009450 con 0x7f550c0d0700 2024-04-25T08:35:16.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.762+0000 7f5513c93640 1 -- 172.21.15.38:0/269499926 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550c156d10 con 0x7f550c0d0700 2024-04-25T08:35:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.763+0000 7f55027fc640 1 -- 172.21.15.38:0/269499926 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5504039040 con 0x7f550c0d0700 2024-04-25T08:35:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.764+0000 7f55027fc640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f54f006b4d0 0x7f54f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.764+0000 7f55027fc640 1 -- 172.21.15.38:0/269499926 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f55040abbe0 con 0x7f550c0d0700 2024-04-25T08:35:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.764+0000 7f5511a2f640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f54f006b4d0 0x7f54f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.764+0000 7f5511a2f640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f54f006b4d0 0x7f54f006d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5508009ac0 tx=0x7f5508008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f5516990740 1 Processor -- start 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f5516990740 1 -- start start 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f5516990740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514ea360 0x564a5150a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f5516990740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f5516990740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514f2140 0x564a514c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f5516990740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5142aee0 con 0x564a514c9360 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f5516990740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a513d3c50 con 0x564a514f2140 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f5516990740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a51426f80 con 0x564a514ea360 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f551122e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f551122e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514f1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44278/0 (socket says 172.21.15.38:44278) 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f551122e640 1 -- 172.21.15.38:0/2483402195 learned_addr learned my addr 172.21.15.38:0/2483402195 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f5511a2f640 1 --2- 172.21.15.38:0/2483402195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514ea360 0x564a5150a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.765+0000 7f5512230640 1 --2- 172.21.15.38:0/2483402195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514f2140 0x564a514c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.766+0000 7f551122e640 1 -- 172.21.15.38:0/2483402195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514ea360 msgr2=0x564a5150a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.766+0000 7f551122e640 1 --2- 172.21.15.38:0/2483402195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514ea360 0x564a5150a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.766+0000 7f551122e640 1 -- 172.21.15.38:0/2483402195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514f2140 msgr2=0x564a514c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.766+0000 7f551122e640 1 --2- 172.21.15.38:0/2483402195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514f2140 0x564a514c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.766+0000 7f551122e640 1 -- 172.21.15.38:0/2483402195 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a5120bc00 con 0x564a514c9360 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.766+0000 7f5512230640 1 --2- 172.21.15.38:0/2483402195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514f2140 0x564a514c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.766+0000 7f551122e640 1 --2- 172.21.15.38:0/2483402195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514f1c00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f54fc001630 tx=0x7f54fc001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7471f6c57a3a2547 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.766+0000 7f54ef7fe640 1 -- 172.21.15.38:0/2483402195 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc001d40 con 0x564a514c9360 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.766+0000 7f54ef7fe640 1 -- 172.21.15.38:0/2483402195 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc0074e0 con 0x564a514c9360 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.766+0000 7f54ef7fe640 1 -- 172.21.15.38:0/2483402195 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54fc004a00 con 0x564a514c9360 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.767+0000 7f5516990740 1 -- 172.21.15.38:0/2483402195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 msgr2=0x564a514f1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.767+0000 7f5516990740 1 --2- 172.21.15.38:0/2483402195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514f1c00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f54fc001630 tx=0x7f54fc001af0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.767+0000 7f5516990740 1 -- 172.21.15.38:0/2483402195 shutdown_connections 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.767+0000 7f5516990740 1 --2- 172.21.15.38:0/2483402195 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514f2140 0x564a514c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.767+0000 7f5516990740 1 --2- 172.21.15.38:0/2483402195 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514f1c00 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.767+0000 7f5516990740 1 --2- 172.21.15.38:0/2483402195 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514ea360 0x564a5150a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.767+0000 7f5516990740 1 -- 172.21.15.38:0/2483402195 >> 172.21.15.38:0/2483402195 conn(0x564a5148e810 msgr2=0x564a5150cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.767+0000 7f5516990740 1 -- 172.21.15.38:0/2483402195 shutdown_connections 2024-04-25T08:35:16.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.767+0000 7f5516990740 1 -- 172.21.15.38:0/2483402195 wait complete. 2024-04-25T08:35:16.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.767+0000 7f5516990740 1 -- start start 2024-04-25T08:35:16.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.767+0000 7f5516990740 1 Processor -- start 2024-04-25T08:35:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.768+0000 7f5516990740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514e9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.768+0000 7f5516990740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514ea360 0x564a51505520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.768+0000 7f5516990740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514f2140 0x564a5150ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.768+0000 7f5516990740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a514c7330 con 0x564a514c9360 2024-04-25T08:35:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.768+0000 7f5516990740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a513c9340 con 0x564a514f2140 2024-04-25T08:35:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.768+0000 7f5516990740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a514c21a0 con 0x564a514ea360 2024-04-25T08:35:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.768+0000 7f5516990740 10 client.0 _ll_register_callbacks cb 0x564a5113efe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.768+0000 7f5511a2f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.768+0000 7f5511a2f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514e9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44284/0 (socket says 172.21.15.38:44284) 2024-04-25T08:35:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.768+0000 7f5511a2f640 1 -- 172.21.15.38:0/2441204692 learned_addr learned my addr 172.21.15.38:0/2441204692 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.768+0000 7f5512230640 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514f2140 0x564a5150ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.768+0000 7f551122e640 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514ea360 0x564a51505520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f5511a2f640 1 -- 172.21.15.38:0/2441204692 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514ea360 msgr2=0x564a51505520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f5511a2f640 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514ea360 0x564a51505520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f5511a2f640 1 -- 172.21.15.38:0/2441204692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514f2140 msgr2=0x564a5150ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f5511a2f640 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514f2140 0x564a5150ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f5511a2f640 1 -- 172.21.15.38:0/2441204692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a510ff4f0 con 0x564a514c9360 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f5511a2f640 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514e9110 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5508015a20 tx=0x7f55080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d785092561be42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f54ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f54ecff9640 1 -- 172.21.15.38:0/2441204692 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508026070 con 0x564a514c9360 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f54ecff9640 1 -- 172.21.15.38:0/2441204692 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5508016d30 con 0x564a514c9360 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f5516990740 1 -- 172.21.15.38:0/2441204692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54fc0074e0 con 0x564a514c9360 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f54ecff9640 1 -- 172.21.15.38:0/2441204692 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5508031050 con 0x564a514c9360 2024-04-25T08:35:16.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f5516990740 10 client.6543 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f5516990740 1 -- 172.21.15.38:0/2441204692 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x564a5126c390 con 0x564a514c9360 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.769+0000 7f54ecff9640 1 -- 172.21.15.38:0/2441204692 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc0074e0 con 0x564a514c9360 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.770+0000 7f54ecff9640 1 -- 172.21.15.38:0/2441204692 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5508034020 con 0x564a514c9360 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.770+0000 7f5516990740 10 client.6543 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.770+0000 7f5516990740 10 client.6543 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.770+0000 7f5516990740 20 client.6543 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.770+0000 7f5516990740 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564a5151f300 0x564a5151f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.770+0000 7f5516990740 4 client.6543 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.770+0000 7f5516990740 1 -- 172.21.15.38:0/2441204692 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a51438420 con 0x564a5151f300 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.770+0000 7f551122e640 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564a5151f300 0x564a5151f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.770+0000 7f54ecff9640 10 client.6543 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.770+0000 7f551122e640 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564a5151f300 0x564a5151f700 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b19838043acdd39 server_cookie=c2370783ccc3ce37 in_seq=0 out_seq=0 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.770+0000 7f54ecff9640 1 -- 172.21.15.38:0/2441204692 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564a51438420 con 0x564a5151f300 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.770+0000 7f54ecff9640 10 client.6543 handle_command_reply: tid=0 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f5516990740 1 client.6543 shutdown 2024-04-25T08:35:16.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f5516990740 1 -- 172.21.15.38:0/2441204692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 msgr2=0x564a514e9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f5516990740 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514e9110 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5508015a20 tx=0x7f55080046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f5516990740 1 -- 172.21.15.38:0/2441204692 shutdown_connections 2024-04-25T08:35:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f5516990740 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564a5151f300 0x564a5151f700 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f5516990740 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514f2140 0x564a5150ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f5516990740 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514ea360 0x564a51505520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f54ecff9640 0 client.6543 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f5516990740 1 --2- 172.21.15.38:0/2441204692 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514e9110 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f5516990740 1 -- 172.21.15.38:0/2441204692 >> 172.21.15.38:0/2441204692 conn(0x564a5148e810 msgr2=0x564a514c1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f5516990740 1 -- 172.21.15.38:0/2441204692 shutdown_connections 2024-04-25T08:35:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f5516990740 1 -- 172.21.15.38:0/2441204692 wait complete. 2024-04-25T08:35:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.771+0000 7f5516990740 20 client.6543 trim_cache size 0 max 16384 2024-04-25T08:35:16.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.786+0000 7f5516990740 1 Processor -- start 2024-04-25T08:35:16.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.786+0000 7f5516990740 1 -- start start 2024-04-25T08:35:16.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5516990740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514c9360 0x564a5150caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5516990740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514ea360 0x564a514ee460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5516990740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514f2140 0x564a514ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5516990740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c200110 con 0x564a514f2140 2024-04-25T08:35:16.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5516990740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c1ff2f0 con 0x564a514ea360 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5516990740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c158210 con 0x564a514c9360 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5512230640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514f2140 0x564a514ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5512230640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514f2140 0x564a514ee9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44290/0 (socket says 172.21.15.38:44290) 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5512230640 1 -- 172.21.15.38:0/408429148 learned_addr learned my addr 172.21.15.38:0/408429148 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5511a2f640 1 --2- 172.21.15.38:0/408429148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514c9360 0x564a5150caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f551122e640 1 --2- 172.21.15.38:0/408429148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514ea360 0x564a514ee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5512230640 1 -- 172.21.15.38:0/408429148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514c9360 msgr2=0x564a5150caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5512230640 1 --2- 172.21.15.38:0/408429148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514c9360 0x564a5150caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5512230640 1 -- 172.21.15.38:0/408429148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514ea360 msgr2=0x564a514ee460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5512230640 1 --2- 172.21.15.38:0/408429148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514ea360 0x564a514ee460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5512230640 1 -- 172.21.15.38:0/408429148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a515102f0 con 0x564a514f2140 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.787+0000 7f5512230640 1 --2- 172.21.15.38:0/408429148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514f2140 0x564a514ee9a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5504009e80 tx=0x7f5504007fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ae90b3da835bda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.788+0000 7f54eeffd640 1 -- 172.21.15.38:0/408429148 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504043e10 con 0x564a514f2140 2024-04-25T08:35:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.788+0000 7f54eeffd640 1 -- 172.21.15.38:0/408429148 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5504078270 con 0x564a514f2140 2024-04-25T08:35:16.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.788+0000 7f54eeffd640 1 -- 172.21.15.38:0/408429148 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55040784d0 con 0x564a514f2140 2024-04-25T08:35:16.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.788+0000 7f5516990740 1 -- 172.21.15.38:0/408429148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514f2140 msgr2=0x564a514ee9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.788+0000 7f5516990740 1 --2- 172.21.15.38:0/408429148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514f2140 0x564a514ee9a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5504009e80 tx=0x7f5504007fc0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.788+0000 7f5516990740 1 -- 172.21.15.38:0/408429148 shutdown_connections 2024-04-25T08:35:16.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.788+0000 7f5516990740 1 --2- 172.21.15.38:0/408429148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514f2140 0x564a514ee9a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.788+0000 7f5516990740 1 --2- 172.21.15.38:0/408429148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514ea360 0x564a514ee460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.788+0000 7f5516990740 1 --2- 172.21.15.38:0/408429148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514c9360 0x564a5150caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.788+0000 7f5516990740 1 -- 172.21.15.38:0/408429148 >> 172.21.15.38:0/408429148 conn(0x564a5148e810 msgr2=0x564a514f3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.788+0000 7f5516990740 1 -- 172.21.15.38:0/408429148 shutdown_connections 2024-04-25T08:35:16.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.788+0000 7f5516990740 1 -- 172.21.15.38:0/408429148 wait complete. 2024-04-25T08:35:16.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.789+0000 7f5516990740 1 -- start start 2024-04-25T08:35:16.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.789+0000 7f5516990740 1 Processor -- start 2024-04-25T08:35:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5516990740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514f3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5516990740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514ea360 0x564a5151b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5516990740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514f2140 0x564a5151b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5516990740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a5151cb80 con 0x564a514c9360 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5516990740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c1fed20 con 0x564a514ea360 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5516990740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x564a514eeee0 con 0x564a514f2140 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5516990740 10 client.0 _ll_register_callbacks cb 0x7f54fc007ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5511a2f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514f3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5511a2f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514f3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44304/0 (socket says 172.21.15.38:44304) 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5511a2f640 1 -- 172.21.15.38:0/408787042 learned_addr learned my addr 172.21.15.38:0/408787042 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5512230640 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514f2140 0x564a5151b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f551122e640 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514ea360 0x564a5151b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5511a2f640 1 -- 172.21.15.38:0/408787042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514f2140 msgr2=0x564a5151b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5511a2f640 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514f2140 0x564a5151b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5511a2f640 1 -- 172.21.15.38:0/408787042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514ea360 msgr2=0x564a5151b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5511a2f640 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514ea360 0x564a5151b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5511a2f640 1 -- 172.21.15.38:0/408787042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a510d8670 con 0x564a514c9360 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f551122e640 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514ea360 0x564a5151b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5511a2f640 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514f3330 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f55080098a0 tx=0x7f5508015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7945da61cd26131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f54ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f54ecff9640 1 -- 172.21.15.38:0/408787042 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508026040 con 0x564a514c9360 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f54ecff9640 1 -- 172.21.15.38:0/408787042 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f550803fc80 con 0x564a514c9360 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.790+0000 7f5516990740 1 -- 172.21.15.38:0/408787042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5504078270 con 0x564a514c9360 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f54ecff9640 1 -- 172.21.15.38:0/408787042 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f550801c040 con 0x564a514c9360 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f5516990740 10 client.6549 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f5516990740 1 -- 172.21.15.38:0/408787042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x564a5157b8e0 con 0x564a514c9360 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f54ecff9640 1 -- 172.21.15.38:0/408787042 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5504078270 con 0x564a514c9360 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f54ecff9640 1 -- 172.21.15.38:0/408787042 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f550803c070 con 0x564a514c9360 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f5516990740 10 client.6549 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f5516990740 10 client.6549 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f5516990740 20 client.6549 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:16.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f5516990740 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564a51530cf0 0x564a5151bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f5516990740 4 client.6549 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f5516990740 1 -- 172.21.15.38:0/408787042 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x564a51432770 con 0x564a51530cf0 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f551122e640 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564a51530cf0 0x564a5151bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f54ecff9640 10 client.6549 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.791+0000 7f551122e640 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564a51530cf0 0x564a5151bd80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43ede5565f20993 server_cookie=10ed9252768cd65d in_seq=0 out_seq=0 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f54ecff9640 1 -- 172.21.15.38:0/408787042 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564a51432770 con 0x564a51530cf0 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f54ecff9640 10 client.6549 handle_command_reply: tid=0 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f5516990740 1 client.6549 shutdown 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f5516990740 1 -- 172.21.15.38:0/408787042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 msgr2=0x564a514f3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f5516990740 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514f3330 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f55080098a0 tx=0x7f5508015000 comp rx=0 tx=0).stop 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f5516990740 1 -- 172.21.15.38:0/408787042 shutdown_connections 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f5516990740 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564a51530cf0 0x564a5151bd80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f5516990740 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564a514f2140 0x564a5151b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f54ecff9640 0 client.6549 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f5516990740 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564a514ea360 0x564a5151b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f5516990740 1 --2- 172.21.15.38:0/408787042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564a514c9360 0x564a514f3330 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f5516990740 1 -- 172.21.15.38:0/408787042 >> 172.21.15.38:0/408787042 conn(0x564a5148e810 msgr2=0x564a514c5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f5516990740 1 -- 172.21.15.38:0/408787042 shutdown_connections 2024-04-25T08:35:16.793 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f2633938640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4e [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v751 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6153=0-4194304@1} caps={6153=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=6153 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5590cdfce580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000a4e:[<0x10000000001/new_file_1 v751>,<0x10000000000/ceph-qa-suite v78>,<0x1/scrub_test_path v38>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T08:35:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f5516990740 1 -- 172.21.15.38:0/408787042 wait complete. 2024-04-25T08:35:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.792+0000 7f5516990740 20 client.6549 trim_cache size 0 max 16384 2024-04-25T08:35:16.794 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:16.794 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:35:16.794 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "18947434-e437-41c2-a01e-6cae6e0bb481", 2024-04-25T08:35:16.794 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:35:16.794 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.793+0000 7f5513c93640 1 -- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f54f006b4d0 msgr2=0x7f54f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.793+0000 7f5513c93640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f54f006b4d0 0x7f54f006d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5508009ac0 tx=0x7f5508008040 comp rx=0 tx=0).stop 2024-04-25T08:35:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.793+0000 7f5513c93640 1 -- 172.21.15.38:0/269499926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c0d0700 msgr2=0x7f550c147a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.793+0000 7f5513c93640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c0d0700 0x7f550c147a50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f5504009eb0 tx=0x7f5504004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.793+0000 7f5513c93640 1 -- 172.21.15.38:0/269499926 shutdown_connections 2024-04-25T08:35:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.793+0000 7f5513c93640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f550c0d0700 0x7f550c147a50 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.793+0000 7f5513c93640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f550c0d0320 0x7f550c14b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.793+0000 7f5513c93640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f54f006b4d0 0x7f54f006d990 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.793+0000 7f5513c93640 1 --2- 172.21.15.38:0/269499926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f550c0b6980 0x7f550c0c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.793+0000 7f5513c93640 1 -- 172.21.15.38:0/269499926 >> 172.21.15.38:0/269499926 conn(0x7f550c0668f0 msgr2=0x7f550c0ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.793+0000 7f5513c93640 1 -- 172.21.15.38:0/269499926 shutdown_connections 2024-04-25T08:35:16.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.793+0000 7f5513c93640 1 -- 172.21.15.38:0/269499926 wait complete. 2024-04-25T08:35:16.798 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '18947434-e437-41c2-a01e-6cae6e0bb481', 'mode': 'asynchronous'}' 2024-04-25T08:35:16.798 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:35:16.798 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:35:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.906+0000 7f19baabd640 1 Processor -- start 2024-04-25T08:35:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.906+0000 7f19baabd640 1 -- start start 2024-04-25T08:35:16.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.907+0000 7f19baabd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f19b40d33d0 0x7f19b40d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.907+0000 7f19baabd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19b40d1e90 0x7f19b40b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.907+0000 7f19baabd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b40d2330 con 0x7f19b40d0320 2024-04-25T08:35:16.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.907+0000 7f19baabd640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b40d2610 con 0x7f19b40d33d0 2024-04-25T08:35:16.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.907+0000 7f19baabd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b40d28f0 con 0x7f19b40d1e90 2024-04-25T08:35:16.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.908+0000 7f19b905a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19b40d1e90 0x7f19b40b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.908+0000 7f19b3fff640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f19b40d33d0 0x7f19b40d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.908+0000 7f19b8859640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f19b40d0320 0x7f19b40d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:57708/0 (socket says 172.21.15.38:57708) 2024-04-25T08:35:16.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.908+0000 7f19b8859640 1 -- 172.21.15.38:0/266796679 learned_addr learned my addr 172.21.15.38:0/266796679 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:16.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.908+0000 7f19b905a640 1 -- 172.21.15.38:0/266796679 >> v2:172.21.15.107:3300/0 conn(0x7f19b40d33d0 msgr2=0x7f19b40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.908+0000 7f19b905a640 1 --2- 172.21.15.38:0/266796679 >> v2:172.21.15.107:3300/0 conn(0x7f19b40d33d0 0x7f19b40d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.908+0000 7f19b905a640 1 -- 172.21.15.38:0/266796679 >> v1:172.21.15.107:6789/0 conn(0x7f19b40d0320 legacy=0x7f19b40d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.908+0000 7f19b905a640 1 -- 172.21.15.38:0/266796679 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b40d2bd0 con 0x7f19b40d1e90 2024-04-25T08:35:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.909+0000 7f19b905a640 1 --2- 172.21.15.38:0/266796679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19b40d1e90 0x7f19b40b6980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19a40097a0 tx=0x7f19a402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61ab11fe45a9d9d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.909+0000 7f19b37fe640 1 -- 172.21.15.38:0/266796679 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4037070 con 0x7f19b40d1e90 2024-04-25T08:35:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.909+0000 7f19b37fe640 1 -- 172.21.15.38:0/266796679 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4032440 con 0x7f19b40d1e90 2024-04-25T08:35:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.909+0000 7f19b37fe640 1 -- 172.21.15.38:0/266796679 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4032810 con 0x7f19b40d1e90 2024-04-25T08:35:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.909+0000 7f19baabd640 1 -- 172.21.15.38:0/266796679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19b40d1e90 msgr2=0x7f19b40b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.909+0000 7f19baabd640 1 --2- 172.21.15.38:0/266796679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19b40d1e90 0x7f19b40b6980 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f19a40097a0 tx=0x7f19a402a210 comp rx=0 tx=0).stop 2024-04-25T08:35:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.910+0000 7f19baabd640 1 -- 172.21.15.38:0/266796679 shutdown_connections 2024-04-25T08:35:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.910+0000 7f19baabd640 1 --2- 172.21.15.38:0/266796679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19b40d1e90 0x7f19b40b6980 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.910+0000 7f19baabd640 1 --2- 172.21.15.38:0/266796679 >> v2:172.21.15.107:3300/0 conn(0x7f19b40d33d0 0x7f19b40d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.910+0000 7f19baabd640 1 -- 172.21.15.38:0/266796679 >> 172.21.15.38:0/266796679 conn(0x7f19b40668f0 msgr2=0x7f19b4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.910+0000 7f19baabd640 1 -- 172.21.15.38:0/266796679 shutdown_connections 2024-04-25T08:35:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.910+0000 7f19baabd640 1 -- 172.21.15.38:0/266796679 wait complete. 2024-04-25T08:35:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.911+0000 7f19baabd640 1 Processor -- start 2024-04-25T08:35:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.911+0000 7f19baabd640 1 -- start start 2024-04-25T08:35:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19baabd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19b40d0320 0x7f19b415f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19baabd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b40d1e90 0x7f19b4156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19baabd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b40d33d0 0x7f19b40c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19baabd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b4161390 con 0x7f19b40d1e90 2024-04-25T08:35:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19baabd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b4161210 con 0x7f19b40d33d0 2024-04-25T08:35:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19baabd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b40b9720 con 0x7f19b40d0320 2024-04-25T08:35:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b40d1e90 0x7f19b4156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b40d1e90 0x7f19b4156810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44308/0 (socket says 172.21.15.38:44308) 2024-04-25T08:35:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19b3fff640 1 -- 172.21.15.38:0/1805506625 learned_addr learned my addr 172.21.15.38:0/1805506625 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19b8859640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19b40d0320 0x7f19b415f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19b905a640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b40d33d0 0x7f19b40c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19b3fff640 1 -- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19b40d0320 msgr2=0x7f19b415f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19b3fff640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19b40d0320 0x7f19b415f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19b3fff640 1 -- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b40d33d0 msgr2=0x7f19b40c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19b3fff640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b40d33d0 0x7f19b40c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19b3fff640 1 -- 172.21.15.38:0/1805506625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b4159290 con 0x7f19b40d1e90 2024-04-25T08:35:16.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.912+0000 7f19b3fff640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b40d1e90 0x7f19b4156810 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f199c00e870 tx=0x7f199c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.913+0000 7f19b17fa640 1 -- 172.21.15.38:0/1805506625 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c017070 con 0x7f19b40d1e90 2024-04-25T08:35:16.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.913+0000 7f19b17fa640 1 -- 172.21.15.38:0/1805506625 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c00c940 con 0x7f19b40d1e90 2024-04-25T08:35:16.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.913+0000 7f19baabd640 1 -- 172.21.15.38:0/1805506625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b4159570 con 0x7f19b40d1e90 2024-04-25T08:35:16.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.913+0000 7f19b17fa640 1 -- 172.21.15.38:0/1805506625 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c00cd30 con 0x7f19b40d1e90 2024-04-25T08:35:16.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.913+0000 7f19baabd640 1 -- 172.21.15.38:0/1805506625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b4159b30 con 0x7f19b40d1e90 2024-04-25T08:35:16.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.914+0000 7f19b17fa640 1 -- 172.21.15.38:0/1805506625 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f199c01a070 con 0x7f19b40d1e90 2024-04-25T08:35:16.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.915+0000 7f19b17fa640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f199406b4d0 0x7f199406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.915+0000 7f19b17fa640 1 -- 172.21.15.38:0/1805506625 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f199c08a5f0 con 0x7f19b40d1e90 2024-04-25T08:35:16.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.915+0000 7f19b8859640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f199406b4d0 0x7f199406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.915+0000 7f19b8859640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f199406b4d0 0x7f199406d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0012e0 tx=0x7f19ac001a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.916+0000 7f19bd7c2740 1 Processor -- start 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.916+0000 7f19bd7c2740 1 -- start start 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.916+0000 7f19bd7c2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 0x564d3cacf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.916+0000 7f19bd7c2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3caccef0 0x564d3cacf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.916+0000 7f19bd7c2740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3ca9f770 0x564d3caa2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.916+0000 7f19bd7c2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3ca08da0 con 0x564d3caccb10 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.916+0000 7f19bd7c2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c9b1b80 con 0x564d3caccef0 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.916+0000 7f19bd7c2740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d3ca04e40 con 0x564d3ca9f770 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.916+0000 7f19b8859640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 0x564d3cacf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.916+0000 7f19b8859640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 0x564d3cacf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44320/0 (socket says 172.21.15.38:44320) 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.916+0000 7f19b8859640 1 -- 172.21.15.38:0/2829893738 learned_addr learned my addr 172.21.15.38:0/2829893738 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.917+0000 7f19b3fff640 1 --2- 172.21.15.38:0/2829893738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3caccef0 0x564d3cacf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.917+0000 7f19b905a640 1 --2- 172.21.15.38:0/2829893738 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3ca9f770 0x564d3caa2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.917+0000 7f19b8859640 1 -- 172.21.15.38:0/2829893738 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3ca9f770 msgr2=0x564d3caa2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.917+0000 7f19b8859640 1 --2- 172.21.15.38:0/2829893738 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3ca9f770 0x564d3caa2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.917+0000 7f19b8859640 1 -- 172.21.15.38:0/2829893738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3caccef0 msgr2=0x564d3cacf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.917+0000 7f19b8859640 1 --2- 172.21.15.38:0/2829893738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3caccef0 0x564d3cacf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.917+0000 7f19b8859640 1 -- 172.21.15.38:0/2829893738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d3c9bbf10 con 0x564d3caccb10 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.917+0000 7f19b905a640 1 --2- 172.21.15.38:0/2829893738 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3ca9f770 0x564d3caa2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.917+0000 7f19b8859640 1 --2- 172.21.15.38:0/2829893738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 0x564d3cacf320 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f19ac004d10 tx=0x7f19ac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afda4b1589a3636 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.917+0000 7f19aa7fc640 1 -- 172.21.15.38:0/2829893738 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac026070 con 0x564d3caccb10 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.917+0000 7f19aa7fc640 1 -- 172.21.15.38:0/2829893738 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac012350 con 0x564d3caccb10 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.917+0000 7f19aa7fc640 1 -- 172.21.15.38:0/2829893738 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac012740 con 0x564d3caccb10 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.918+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2829893738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 msgr2=0x564d3cacf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.918+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/2829893738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 0x564d3cacf320 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f19ac004d10 tx=0x7f19ac013000 comp rx=0 tx=0).stop 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.918+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2829893738 shutdown_connections 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.918+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/2829893738 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3ca9f770 0x564d3caa2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.918+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/2829893738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3caccef0 0x564d3cacf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.918+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/2829893738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 0x564d3cacf320 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.918+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2829893738 >> 172.21.15.38:0/2829893738 conn(0x564d3caa19e0 msgr2=0x564d3caea320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.918+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2829893738 shutdown_connections 2024-04-25T08:35:16.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.918+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2829893738 wait complete. 2024-04-25T08:35:16.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.918+0000 7f19bd7c2740 1 -- start start 2024-04-25T08:35:16.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.919+0000 7f19bd7c2740 1 Processor -- start 2024-04-25T08:35:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.919+0000 7f19bd7c2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3caccb10 0x564d3cac29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.919+0000 7f19bd7c2740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccef0 0x564d3cacfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.919+0000 7f19bd7c2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3cab15c0 0x564d3cad0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.919+0000 7f19bd7c2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3caa4b90 con 0x564d3cab15c0 2024-04-25T08:35:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.919+0000 7f19bd7c2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3c9a74b0 con 0x564d3caccb10 2024-04-25T08:35:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.919+0000 7f19bd7c2740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d3caa4e70 con 0x564d3caccef0 2024-04-25T08:35:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.919+0000 7f19bd7c2740 10 client.0 _ll_register_callbacks cb 0x564d3c9e7180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.919+0000 7f19b905a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3cab15c0 0x564d3cad0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19b905a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3cab15c0 0x564d3cad0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44332/0 (socket says 172.21.15.38:44332) 2024-04-25T08:35:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19b905a640 1 -- 172.21.15.38:0/1180514414 learned_addr learned my addr 172.21.15.38:0/1180514414 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19b3fff640 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccef0 0x564d3cacfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19b8859640 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3caccb10 0x564d3cac29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19b905a640 1 -- 172.21.15.38:0/1180514414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccef0 msgr2=0x564d3cacfe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19b905a640 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccef0 0x564d3cacfe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19b905a640 1 -- 172.21.15.38:0/1180514414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3caccb10 msgr2=0x564d3cac29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19b905a640 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3caccb10 0x564d3cac29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19b905a640 1 -- 172.21.15.38:0/1180514414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d3c7e77d0 con 0x564d3cab15c0 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19b905a640 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3cab15c0 0x564d3cad0390 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f19a40097a0 tx=0x7f19a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe5cc86e81a088b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19937fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19937fe640 1 -- 172.21.15.38:0/1180514414 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4037040 con 0x564d3cab15c0 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19937fe640 1 -- 172.21.15.38:0/1180514414 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f19a4047c30 con 0x564d3cab15c0 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19bd7c2740 1 -- 172.21.15.38:0/1180514414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19ac012350 con 0x564d3cab15c0 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19937fe640 1 -- 172.21.15.38:0/1180514414 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19a4047e30 con 0x564d3cab15c0 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19bd7c2740 10 client.6558 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19bd7c2740 1 -- 172.21.15.38:0/1180514414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x564d3c71d3f0 con 0x564d3cab15c0 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.920+0000 7f19937fe640 1 -- 172.21.15.38:0/1180514414 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac012350 con 0x564d3cab15c0 2024-04-25T08:35:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19937fe640 1 -- 172.21.15.38:0/1180514414 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f19a404e050 con 0x564d3cab15c0 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19bd7c2740 10 client.6558 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19bd7c2740 10 client.6558 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19bd7c2740 20 client.6558 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564d3caecf30 0x564d3caed360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19bd7c2740 4 client.6558 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19bd7c2740 1 -- 172.21.15.38:0/1180514414 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d3ca162e0 con 0x564d3caecf30 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19b8859640 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564d3caecf30 0x564d3caed360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19b8859640 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564d3caecf30 0x564d3caed360 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f63c9d7c3586e9 server_cookie=3dd0449af14fe882 in_seq=0 out_seq=0 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19937fe640 10 client.6558 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19937fe640 1 -- 172.21.15.38:0/1180514414 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564d3ca162e0 con 0x564d3caecf30 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19937fe640 10 client.6558 handle_command_reply: tid=0 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.921+0000 7f19bd7c2740 1 client.6558 shutdown 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.922+0000 7f19bd7c2740 1 -- 172.21.15.38:0/1180514414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3cab15c0 msgr2=0x564d3cad0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.922+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3cab15c0 0x564d3cad0390 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f19a40097a0 tx=0x7f19a4004730 comp rx=0 tx=0).stop 2024-04-25T08:35:16.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.922+0000 7f19bd7c2740 1 -- 172.21.15.38:0/1180514414 shutdown_connections 2024-04-25T08:35:16.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.922+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564d3caecf30 0x564d3caed360 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.922+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3cab15c0 0x564d3cad0390 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.922+0000 7f19937fe640 0 client.6558 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:16.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.922+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccef0 0x564d3cacfe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.922+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/1180514414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3caccb10 0x564d3cac29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.922+0000 7f19bd7c2740 1 -- 172.21.15.38:0/1180514414 >> 172.21.15.38:0/1180514414 conn(0x564d3ca9f770 msgr2=0x564d3cacf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.922+0000 7f19bd7c2740 1 -- 172.21.15.38:0/1180514414 shutdown_connections 2024-04-25T08:35:16.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.922+0000 7f19bd7c2740 1 -- 172.21.15.38:0/1180514414 wait complete. 2024-04-25T08:35:16.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.922+0000 7f19bd7c2740 20 client.6558 trim_cache size 0 max 16384 2024-04-25T08:35:16.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.937+0000 7f19bd7c2740 1 Processor -- start 2024-04-25T08:35:16.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.937+0000 7f19bd7c2740 1 -- start start 2024-04-25T08:35:16.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19bd7c2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3cab15c0 0x564d3cad25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19bd7c2740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccb10 0x564d3cacfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19bd7c2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccef0 0x564d3cad0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19bd7c2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b4200360 con 0x564d3caccef0 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19bd7c2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b41ff540 con 0x564d3cab15c0 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19bd7c2740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b40b4920 con 0x564d3caccb10 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19b905a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccef0 0x564d3cad0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19b905a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccef0 0x564d3cad0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44334/0 (socket says 172.21.15.38:44334) 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19b905a640 1 -- 172.21.15.38:0/3366988789 learned_addr learned my addr 172.21.15.38:0/3366988789 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19b8859640 1 --2- 172.21.15.38:0/3366988789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3cab15c0 0x564d3cad25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19b3fff640 1 --2- 172.21.15.38:0/3366988789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccb10 0x564d3cacfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19b905a640 1 -- 172.21.15.38:0/3366988789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccb10 msgr2=0x564d3cacfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19b905a640 1 --2- 172.21.15.38:0/3366988789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccb10 0x564d3cacfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19b905a640 1 -- 172.21.15.38:0/3366988789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3cab15c0 msgr2=0x564d3cad25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19b905a640 1 --2- 172.21.15.38:0/3366988789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3cab15c0 0x564d3cad25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19b905a640 1 -- 172.21.15.38:0/3366988789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d3c6dd0f0 con 0x564d3caccef0 2024-04-25T08:35:16.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19b3fff640 1 --2- 172.21.15.38:0/3366988789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccb10 0x564d3cacfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19b905a640 1 --2- 172.21.15.38:0/3366988789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccef0 0x564d3cad0370 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009680 tx=0x7f19a403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2a34cebd3ca329 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19a9ffb640 1 -- 172.21.15.38:0/3366988789 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a4037070 con 0x564d3caccef0 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19a9ffb640 1 -- 172.21.15.38:0/3366988789 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4004040 con 0x564d3caccef0 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.938+0000 7f19a9ffb640 1 -- 172.21.15.38:0/3366988789 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a40043f0 con 0x564d3caccef0 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.939+0000 7f19bd7c2740 1 -- 172.21.15.38:0/3366988789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccef0 msgr2=0x564d3cad0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.939+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/3366988789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccef0 0x564d3cad0370 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f19a4009680 tx=0x7f19a403d000 comp rx=0 tx=0).stop 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.939+0000 7f19bd7c2740 1 -- 172.21.15.38:0/3366988789 shutdown_connections 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.939+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/3366988789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccef0 0x564d3cad0370 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.939+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/3366988789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccb10 0x564d3cacfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.939+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/3366988789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3cab15c0 0x564d3cad25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.939+0000 7f19bd7c2740 1 -- 172.21.15.38:0/3366988789 >> 172.21.15.38:0/3366988789 conn(0x564d3ca9f770 msgr2=0x564d3caa4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.939+0000 7f19bd7c2740 1 -- 172.21.15.38:0/3366988789 shutdown_connections 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.939+0000 7f19bd7c2740 1 -- 172.21.15.38:0/3366988789 wait complete. 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.940+0000 7f19bd7c2740 1 -- start start 2024-04-25T08:35:16.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.940+0000 7f19bd7c2740 1 Processor -- start 2024-04-25T08:35:16.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19bd7c2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3cab15c0 0x564d3caa4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19bd7c2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 0x564d3caeabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19bd7c2740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccef0 0x564d3cae8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19bd7c2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d3caf9fc0 con 0x564d3caccb10 2024-04-25T08:35:16.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19bd7c2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b40b4f20 con 0x564d3cab15c0 2024-04-25T08:35:16.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19bd7c2740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d3cac29d0 con 0x564d3caccef0 2024-04-25T08:35:16.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19bd7c2740 10 client.0 _ll_register_callbacks cb 0x7f19ac004e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:16.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 0x564d3caeabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 0x564d3caeabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44342/0 (socket says 172.21.15.38:44342) 2024-04-25T08:35:16.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b3fff640 1 -- 172.21.15.38:0/2653964573 learned_addr learned my addr 172.21.15.38:0/2653964573 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:16.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b905a640 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccef0 0x564d3cae8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b3fff640 1 -- 172.21.15.38:0/2653964573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccef0 msgr2=0x564d3cae8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b8859640 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3cab15c0 0x564d3caa4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b3fff640 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccef0 0x564d3cae8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b3fff640 1 -- 172.21.15.38:0/2653964573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3cab15c0 msgr2=0x564d3caa4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:16.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b3fff640 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3cab15c0 0x564d3caa4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b3fff640 1 -- 172.21.15.38:0/2653964573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d3cac1c70 con 0x564d3caccb10 2024-04-25T08:35:16.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b905a640 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccef0 0x564d3cae8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:16.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b8859640 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3cab15c0 0x564d3caa4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:16.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19b3fff640 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 0x564d3caeabc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f199c055060 tx=0x7f199c058670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903fcc8f5820ef7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19937fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19937fe640 1 -- 172.21.15.38:0/2653964573 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c021040 con 0x564d3caccb10 2024-04-25T08:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19937fe640 1 -- 172.21.15.38:0/2653964573 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f199c06cda0 con 0x564d3caccb10 2024-04-25T08:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2653964573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19a4004040 con 0x564d3caccb10 2024-04-25T08:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19937fe640 1 -- 172.21.15.38:0/2653964573 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f199c077050 con 0x564d3caccb10 2024-04-25T08:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19bd7c2740 10 client.6564 fetch_fsmap learned FSMap version 25 2024-04-25T08:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.941+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2653964573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x564d3caeb160 con 0x564d3caccb10 2024-04-25T08:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19937fe640 1 -- 172.21.15.38:0/2653964573 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a4004040 con 0x564d3caccb10 2024-04-25T08:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19937fe640 1 -- 172.21.15.38:0/2653964573 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f199c083070 con 0x564d3caccb10 2024-04-25T08:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19bd7c2740 10 client.6564 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19bd7c2740 10 client.6564 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19bd7c2740 20 client.6564 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:16.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564d3cae93d0 0x564d3cae79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:16.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19bd7c2740 4 client.6564 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:35:16.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2653964573 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564d3ca10630 con 0x564d3cae93d0 2024-04-25T08:35:16.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19b8859640 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564d3cae93d0 0x564d3cae79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:16.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19b8859640 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564d3cae93d0 0x564d3cae79a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b077a21c3aa2836 server_cookie=ddbdeeb84a14271 in_seq=0 out_seq=0 2024-04-25T08:35:16.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19937fe640 10 client.6564 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:16.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19937fe640 1 -- 172.21.15.38:0/2653964573 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+903 (crc 0 0 0) 0x564d3ca10630 con 0x564d3cae93d0 2024-04-25T08:35:16.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19937fe640 10 client.6564 handle_command_reply: tid=0 2024-04-25T08:35:16.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.942+0000 7f19bd7c2740 1 client.6564 shutdown 2024-04-25T08:35:16.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2653964573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 msgr2=0x564d3caeabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 0x564d3caeabc0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f199c055060 tx=0x7f199c058670 comp rx=0 tx=0).stop 2024-04-25T08:35:16.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2653964573 shutdown_connections 2024-04-25T08:35:16.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x564d3cae93d0 0x564d3cae79a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x564d3caccef0 0x564d3cae8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564d3caccb10 0x564d3caeabc0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19937fe640 0 client.6564 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:16.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19bd7c2740 1 --2- 172.21.15.38:0/2653964573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564d3cab15c0 0x564d3caa4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2653964573 >> 172.21.15.38:0/2653964573 conn(0x564d3ca9f770 msgr2=0x564d3cac2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2653964573 shutdown_connections 2024-04-25T08:35:16.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19bd7c2740 1 -- 172.21.15.38:0/2653964573 wait complete. 2024-04-25T08:35:16.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19bd7c2740 20 client.6564 trim_cache size 0 max 16384 2024-04-25T08:35:16.946 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:16.946 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:35:16.946 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-04-25T08:35:16.946 INFO:teuthology.orchestra.run.smithi038.stdout: "0a91dd8a-7080-4ed1-b0f2-ee25e8c00593": { 2024-04-25T08:35:16.946 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-04-25T08:35:16.946 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "0a91dd8a-7080-4ed1-b0f2-ee25e8c00593", 2024-04-25T08:35:16.946 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "" 2024-04-25T08:35:16.946 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:16.946 INFO:teuthology.orchestra.run.smithi038.stdout: "18947434-e437-41c2-a01e-6cae6e0bb481": { 2024-04-25T08:35:16.946 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-25T08:35:16.946 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "18947434-e437-41c2-a01e-6cae6e0bb481", 2024-04-25T08:35:16.946 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "" 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: "3b6f1285-ec00-44c1-9790-b85ccb89d320": { 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "3b6f1285-ec00-44c1-9790-b85ccb89d320", 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "" 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: "9eb1a0a6-6ec5-41b5-baec-ef0d2fcbf0ae": { 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites", 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "9eb1a0a6-6ec5-41b5-baec-ef0d2fcbf0ae", 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "" 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:16.947 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:16.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19baabd640 1 -- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f199406b4d0 msgr2=0x7f199406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19baabd640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f199406b4d0 0x7f199406d990 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f19ac0012e0 tx=0x7f19ac001a30 comp rx=0 tx=0).stop 2024-04-25T08:35:16.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19baabd640 1 -- 172.21.15.38:0/1805506625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b40d1e90 msgr2=0x7f19b4156810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:16.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19baabd640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b40d1e90 0x7f19b4156810 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f199c00e870 tx=0x7f199c00ed30 comp rx=0 tx=0).stop 2024-04-25T08:35:16.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19baabd640 1 -- 172.21.15.38:0/1805506625 shutdown_connections 2024-04-25T08:35:16.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19baabd640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19b40d33d0 0x7f19b40c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19baabd640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19b40d1e90 0x7f19b4156810 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19baabd640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f199406b4d0 0x7f199406d990 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19baabd640 1 --2- 172.21.15.38:0/1805506625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19b40d0320 0x7f19b415f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:16.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19baabd640 1 -- 172.21.15.38:0/1805506625 >> 172.21.15.38:0/1805506625 conn(0x7f19b40668f0 msgr2=0x7f19b415e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:16.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.943+0000 7f19baabd640 1 -- 172.21.15.38:0/1805506625 shutdown_connections 2024-04-25T08:35:16.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:16.944+0000 7f19baabd640 1 -- 172.21.15.38:0/1805506625 wait complete. 2024-04-25T08:35:21.131 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:35:21.136+0000 7ff795ff5640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T08:35:21.951 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:35:21.951 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:35:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.072+0000 7f290ef0a640 1 Processor -- start 2024-04-25T08:35:22.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.072+0000 7f290ef0a640 1 -- start start 2024-04-25T08:35:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.073+0000 7f290ef0a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29080d0320 0x7f29080d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.073+0000 7f290ef0a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f29080d0700 0x7f29080d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.073+0000 7f290ef0a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29080d2330 con 0x7f29080d1e90 2024-04-25T08:35:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.073+0000 7f290ef0a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29080d2610 con 0x7f29080d0700 2024-04-25T08:35:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.073+0000 7f290ef0a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29080d28f0 con 0x7f29080d0320 2024-04-25T08:35:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.073+0000 7f290cca6640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29080d0320 0x7f29080d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.073+0000 7f2907fff640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f29080d0700 0x7f29080d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.073+0000 7f290d4a7640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f29080d1e90 0x7f29080b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:35938/0 (socket says 172.21.15.38:35938) 2024-04-25T08:35:22.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.073+0000 7f290d4a7640 1 -- 172.21.15.38:0/248967898 learned_addr learned my addr 172.21.15.38:0/248967898 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.074+0000 7f2907fff640 1 -- 172.21.15.38:0/248967898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29080d0320 msgr2=0x7f29080d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.074+0000 7f2907fff640 1 --2- 172.21.15.38:0/248967898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29080d0320 0x7f29080d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.074+0000 7f2907fff640 1 -- 172.21.15.38:0/248967898 >> v1:172.21.15.107:6789/0 conn(0x7f29080d1e90 legacy=0x7f29080b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.074+0000 7f2907fff640 1 -- 172.21.15.38:0/248967898 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29080d2bd0 con 0x7f29080d0700 2024-04-25T08:35:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.074+0000 7f29077fe640 1 -- 172.21.15.38:0/248967898 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971483027 0 0) 0x7f29080d2330 con 0x7f29080d1e90 2024-04-25T08:35:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.074+0000 7f290cca6640 1 --2- 172.21.15.38:0/248967898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29080d0320 0x7f29080d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.075+0000 7f2907fff640 1 --2- 172.21.15.38:0/248967898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29080d0700 0x7f29080d1950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f28f40097a0 tx=0x7f28f402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb75b6d34c9d750 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.075+0000 7f29077fe640 1 -- 172.21.15.38:0/248967898 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4037070 con 0x7f29080d0700 2024-04-25T08:35:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.075+0000 7f29077fe640 1 -- 172.21.15.38:0/248967898 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4032440 con 0x7f29080d0700 2024-04-25T08:35:22.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.075+0000 7f29077fe640 1 -- 172.21.15.38:0/248967898 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4032810 con 0x7f29080d0700 2024-04-25T08:35:22.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.075+0000 7f290ef0a640 1 -- 172.21.15.38:0/248967898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29080d0700 msgr2=0x7f29080d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.075+0000 7f290ef0a640 1 --2- 172.21.15.38:0/248967898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29080d0700 0x7f29080d1950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f28f40097a0 tx=0x7f28f402a210 comp rx=0 tx=0).stop 2024-04-25T08:35:22.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.075+0000 7f290ef0a640 1 -- 172.21.15.38:0/248967898 shutdown_connections 2024-04-25T08:35:22.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.075+0000 7f290ef0a640 1 --2- 172.21.15.38:0/248967898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29080d0700 0x7f29080d1950 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.075+0000 7f290ef0a640 1 --2- 172.21.15.38:0/248967898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29080d0320 0x7f29080d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.075+0000 7f290ef0a640 1 -- 172.21.15.38:0/248967898 >> 172.21.15.38:0/248967898 conn(0x7f29080668f0 msgr2=0x7f2908066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.075+0000 7f290ef0a640 1 -- 172.21.15.38:0/248967898 shutdown_connections 2024-04-25T08:35:22.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.075+0000 7f290ef0a640 1 -- 172.21.15.38:0/248967898 wait complete. 2024-04-25T08:35:22.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.077+0000 7f290ef0a640 1 Processor -- start 2024-04-25T08:35:22.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.077+0000 7f290ef0a640 1 -- start start 2024-04-25T08:35:22.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.077+0000 7f290ef0a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29080d0320 0x7f29080b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.077+0000 7f290ef0a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29080d0700 0x7f290815c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.077+0000 7f290ef0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29080d1e90 0x7f29080b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.077+0000 7f290ef0a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29080d0b40 con 0x7f29080d1e90 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.077+0000 7f290ef0a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2908161210 con 0x7f29080d0700 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.077+0000 7f290ef0a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290815c310 con 0x7f29080d0320 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f290d4a7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29080d1e90 0x7f29080b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f290d4a7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29080d1e90 0x7f29080b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55882/0 (socket says 172.21.15.38:55882) 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f290d4a7640 1 -- 172.21.15.38:0/335523400 learned_addr learned my addr 172.21.15.38:0/335523400 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f2907fff640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29080d0700 0x7f290815c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f290cca6640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29080d0320 0x7f29080b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f290d4a7640 1 -- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29080d0320 msgr2=0x7f29080b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f290d4a7640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29080d0320 0x7f29080b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f290d4a7640 1 -- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29080d0700 msgr2=0x7f290815c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f290d4a7640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29080d0700 0x7f290815c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f290d4a7640 1 -- 172.21.15.38:0/335523400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2908156f30 con 0x7f29080d1e90 2024-04-25T08:35:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f290d4a7640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29080d1e90 0x7f29080b3e70 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f290000c780 tx=0x7f290000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f29057fa640 1 -- 172.21.15.38:0/335523400 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900018070 con 0x7f29080d1e90 2024-04-25T08:35:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f29057fa640 1 -- 172.21.15.38:0/335523400 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290000ce30 con 0x7f29080d1e90 2024-04-25T08:35:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f290ef0a640 1 -- 172.21.15.38:0/335523400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290815bfe0 con 0x7f29080d1e90 2024-04-25T08:35:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f29057fa640 1 -- 172.21.15.38:0/335523400 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290000a840 con 0x7f29080d1e90 2024-04-25T08:35:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.078+0000 7f290ef0a640 1 -- 172.21.15.38:0/335523400 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29081574b0 con 0x7f29080d1e90 2024-04-25T08:35:22.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.080+0000 7f29057fa640 1 -- 172.21.15.38:0/335523400 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f290000a9e0 con 0x7f29080d1e90 2024-04-25T08:35:22.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.080+0000 7f29057fa640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f28ec06b500 0x7f28ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.081+0000 7f29057fa640 1 -- 172.21.15.38:0/335523400 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2900089800 con 0x7f29080d1e90 2024-04-25T08:35:22.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.081+0000 7f290cca6640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f28ec06b500 0x7f28ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.081+0000 7f290cca6640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f28ec06b500 0x7f28ec06d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009560 tx=0x7f28fc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.081+0000 7f2911c0a740 1 Processor -- start 2024-04-25T08:35:22.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.081+0000 7f2911c0a740 1 -- start start 2024-04-25T08:35:22.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.082+0000 7f2911c0a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eb10 0x55e7d4981320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.082+0000 7f2911c0a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eef0 0x55e7d4981860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.082+0000 7f2911c0a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4951770 0x55e7d4954f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.082+0000 7f2911c0a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7d48bada0 con 0x55e7d4951770 2024-04-25T08:35:22.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.082+0000 7f2911c0a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7d4863b80 con 0x55e7d497eef0 2024-04-25T08:35:22.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.082+0000 7f2911c0a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7d48b6e40 con 0x55e7d497eb10 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.082+0000 7f290d4a7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4951770 0x55e7d4954f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.082+0000 7f290cca6640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eb10 0x55e7d4981320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.082+0000 7f290d4a7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4951770 0x55e7d4954f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55888/0 (socket says 172.21.15.38:55888) 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.082+0000 7f290d4a7640 1 -- 172.21.15.38:0/1942774820 learned_addr learned my addr 172.21.15.38:0/1942774820 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.082+0000 7f2907fff640 1 --2- 172.21.15.38:0/1942774820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eef0 0x55e7d4981860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.083+0000 7f290d4a7640 1 -- 172.21.15.38:0/1942774820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eb10 msgr2=0x55e7d4981320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.083+0000 7f290d4a7640 1 --2- 172.21.15.38:0/1942774820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eb10 0x55e7d4981320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.083+0000 7f290d4a7640 1 -- 172.21.15.38:0/1942774820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eef0 msgr2=0x55e7d4981860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.083+0000 7f290d4a7640 1 --2- 172.21.15.38:0/1942774820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eef0 0x55e7d4981860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.083+0000 7f290d4a7640 1 -- 172.21.15.38:0/1942774820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7d486df10 con 0x55e7d4951770 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.083+0000 7f290d4a7640 1 --2- 172.21.15.38:0/1942774820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4951770 0x55e7d4954f80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f290000c750 tx=0x7f2900057eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f3af53eb2aff3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.083+0000 7f28f27fc640 1 -- 172.21.15.38:0/1942774820 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29000560f0 con 0x55e7d4951770 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.083+0000 7f28f27fc640 1 -- 172.21.15.38:0/1942774820 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290005e070 con 0x55e7d4951770 2024-04-25T08:35:22.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.083+0000 7f28f27fc640 1 -- 172.21.15.38:0/1942774820 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900054460 con 0x55e7d4951770 2024-04-25T08:35:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.083+0000 7f2911c0a740 1 -- 172.21.15.38:0/1942774820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4951770 msgr2=0x55e7d4954f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.083+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1942774820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4951770 0x55e7d4954f80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f290000c750 tx=0x7f2900057eb0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.084+0000 7f2911c0a740 1 -- 172.21.15.38:0/1942774820 shutdown_connections 2024-04-25T08:35:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.084+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1942774820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4951770 0x55e7d4954f80 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.084+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1942774820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eef0 0x55e7d4981860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.084+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1942774820 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eb10 0x55e7d4981320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.084+0000 7f2911c0a740 1 -- 172.21.15.38:0/1942774820 >> 172.21.15.38:0/1942774820 conn(0x55e7d49539e0 msgr2=0x55e7d499c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.084+0000 7f2911c0a740 1 -- 172.21.15.38:0/1942774820 shutdown_connections 2024-04-25T08:35:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.084+0000 7f2911c0a740 1 -- 172.21.15.38:0/1942774820 wait complete. 2024-04-25T08:35:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.084+0000 7f2911c0a740 1 -- start start 2024-04-25T08:35:22.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.084+0000 7f2911c0a740 1 Processor -- start 2024-04-25T08:35:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.085+0000 7f2911c0a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eb10 0x55e7d499f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.085+0000 7f2911c0a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eef0 0x55e7d499ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.085+0000 7f2911c0a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4975cc0 0x55e7d49778b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.085+0000 7f2911c0a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7d4956b90 con 0x55e7d4975cc0 2024-04-25T08:35:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.085+0000 7f2911c0a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7d48594b0 con 0x55e7d497eb10 2024-04-25T08:35:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.085+0000 7f2911c0a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7d4956e70 con 0x55e7d497eef0 2024-04-25T08:35:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.085+0000 7f2911c0a740 10 client.0 _ll_register_callbacks cb 0x55e7d4899180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.085+0000 7f290d4a7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4975cc0 0x55e7d49778b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.085+0000 7f290cca6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eb10 0x55e7d499f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.085+0000 7f290d4a7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4975cc0 0x55e7d49778b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55900/0 (socket says 172.21.15.38:55900) 2024-04-25T08:35:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.085+0000 7f290d4a7640 1 -- 172.21.15.38:0/1136578118 learned_addr learned my addr 172.21.15.38:0/1136578118 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.085+0000 7f2907fff640 1 --2- 172.21.15.38:0/1136578118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eef0 0x55e7d499ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f290d4a7640 1 -- 172.21.15.38:0/1136578118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eef0 msgr2=0x55e7d499ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f290d4a7640 1 --2- 172.21.15.38:0/1136578118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eef0 0x55e7d499ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f290d4a7640 1 -- 172.21.15.38:0/1136578118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eb10 msgr2=0x55e7d499f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f290d4a7640 1 --2- 172.21.15.38:0/1136578118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eb10 0x55e7d499f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f290d4a7640 1 -- 172.21.15.38:0/1136578118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7d46997d0 con 0x55e7d4975cc0 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f290d4a7640 1 --2- 172.21.15.38:0/1136578118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4975cc0 0x55e7d49778b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2900087b40 tx=0x7f29000541c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7512fa148d7687c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f28e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f28e77fe640 1 -- 172.21.15.38:0/1136578118 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900054460 con 0x55e7d4975cc0 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f28e77fe640 1 -- 172.21.15.38:0/1136578118 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2900057240 con 0x55e7d4975cc0 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f2911c0a740 1 -- 172.21.15.38:0/1136578118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f290005e070 con 0x55e7d4975cc0 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f28e77fe640 1 -- 172.21.15.38:0/1136578118 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2900054600 con 0x55e7d4975cc0 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f2911c0a740 10 client.6573 fetch_fsmap learned FSMap version 27 2024-04-25T08:35:22.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f2911c0a740 1 -- 172.21.15.38:0/1136578118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55e7d45cf3f0 con 0x55e7d4975cc0 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f28e77fe640 1 -- 172.21.15.38:0/1136578118 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290005e070 con 0x55e7d4975cc0 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f28e77fe640 1 -- 172.21.15.38:0/1136578118 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2900053b40 con 0x55e7d4975cc0 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.086+0000 7f2911c0a740 10 client.6573 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.087+0000 7f2911c0a740 10 client.6573 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.087+0000 7f2911c0a740 20 client.6573 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.087+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1136578118 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e7d499e590 0x55e7d499e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.087+0000 7f2911c0a740 4 client.6573 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.087+0000 7f2911c0a740 1 -- 172.21.15.38:0/1136578118 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7d48c82e0 con 0x55e7d499e590 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.087+0000 7f290cca6640 1 --2- 172.21.15.38:0/1136578118 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e7d499e590 0x55e7d499e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.087+0000 7f290cca6640 1 --2- 172.21.15.38:0/1136578118 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e7d499e590 0x55e7d499e9a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e323f3ed0e5e29d server_cookie=d890184856178d16 in_seq=0 out_seq=0 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.087+0000 7f28e77fe640 10 client.6573 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.087+0000 7f28e77fe640 1 -- 172.21.15.38:0/1136578118 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e7d48c82e0 con 0x55e7d499e590 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.087+0000 7f28e77fe640 10 client.6573 handle_command_reply: tid=0 2024-04-25T08:35:22.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f2911c0a740 1 client.6573 shutdown 2024-04-25T08:35:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f2911c0a740 1 -- 172.21.15.38:0/1136578118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4975cc0 msgr2=0x55e7d49778b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1136578118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4975cc0 0x55e7d49778b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2900087b40 tx=0x7f29000541c0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f2911c0a740 1 -- 172.21.15.38:0/1136578118 shutdown_connections 2024-04-25T08:35:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1136578118 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e7d499e590 0x55e7d499e9a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1136578118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d4975cc0 0x55e7d49778b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1136578118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eef0 0x55e7d499ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f28e77fe640 0 client.6573 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1136578118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eb10 0x55e7d499f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f2911c0a740 1 -- 172.21.15.38:0/1136578118 >> 172.21.15.38:0/1136578118 conn(0x55e7d4951770 msgr2=0x55e7d497d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f2911c0a740 1 -- 172.21.15.38:0/1136578118 shutdown_connections 2024-04-25T08:35:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f2911c0a740 1 -- 172.21.15.38:0/1136578118 wait complete. 2024-04-25T08:35:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.088+0000 7f2911c0a740 20 client.6573 trim_cache size 0 max 16384 2024-04-25T08:35:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.103+0000 7f2911c0a740 1 Processor -- start 2024-04-25T08:35:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.103+0000 7f2911c0a740 1 -- start start 2024-04-25T08:35:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2911c0a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d4975cc0 0x55e7d499cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2911c0a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 0x55e7d4950600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2911c0a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eef0 0x55e7d4950b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2911c0a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7d499bf30 con 0x55e7d497eb10 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2911c0a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2908200320 con 0x55e7d497eef0 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2911c0a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29081ff530 con 0x55e7d4975cc0 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2907fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 0x55e7d4950600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2907fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 0x55e7d4950600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55906/0 (socket says 172.21.15.38:55906) 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2907fff640 1 -- 172.21.15.38:0/586666518 learned_addr learned my addr 172.21.15.38:0/586666518 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f290d4a7640 1 --2- 172.21.15.38:0/586666518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eef0 0x55e7d4950b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f290cca6640 1 --2- 172.21.15.38:0/586666518 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d4975cc0 0x55e7d499cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2907fff640 1 -- 172.21.15.38:0/586666518 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d4975cc0 msgr2=0x55e7d499cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2907fff640 1 --2- 172.21.15.38:0/586666518 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d4975cc0 0x55e7d499cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2907fff640 1 -- 172.21.15.38:0/586666518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eef0 msgr2=0x55e7d4950b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2907fff640 1 --2- 172.21.15.38:0/586666518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eef0 0x55e7d4950b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2907fff640 1 -- 172.21.15.38:0/586666518 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7d458f0f0 con 0x55e7d497eb10 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f290d4a7640 1 --2- 172.21.15.38:0/586666518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eef0 0x55e7d4950b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:22.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.104+0000 7f2907fff640 1 --2- 172.21.15.38:0/586666518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 0x55e7d4950600 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000c00 tx=0x7f28f402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a740274e2cbe7ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.105+0000 7f28f1ffb640 1 -- 172.21.15.38:0/586666518 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f404b070 con 0x55e7d497eb10 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.105+0000 7f28f1ffb640 1 -- 172.21.15.38:0/586666518 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f40328d0 con 0x55e7d497eb10 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.105+0000 7f28f1ffb640 1 -- 172.21.15.38:0/586666518 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4037040 con 0x55e7d497eb10 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.105+0000 7f2911c0a740 1 -- 172.21.15.38:0/586666518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 msgr2=0x55e7d4950600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.105+0000 7f2911c0a740 1 --2- 172.21.15.38:0/586666518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 0x55e7d4950600 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000c00 tx=0x7f28f402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.105+0000 7f2911c0a740 1 -- 172.21.15.38:0/586666518 shutdown_connections 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.105+0000 7f2911c0a740 1 --2- 172.21.15.38:0/586666518 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d497eef0 0x55e7d4950b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.105+0000 7f2911c0a740 1 --2- 172.21.15.38:0/586666518 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 0x55e7d4950600 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.105+0000 7f2911c0a740 1 --2- 172.21.15.38:0/586666518 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d4975cc0 0x55e7d499cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.105+0000 7f2911c0a740 1 -- 172.21.15.38:0/586666518 >> 172.21.15.38:0/586666518 conn(0x55e7d4951770 msgr2=0x55e7d499c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.105+0000 7f2911c0a740 1 -- 172.21.15.38:0/586666518 shutdown_connections 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.105+0000 7f2911c0a740 1 -- 172.21.15.38:0/586666518 wait complete. 2024-04-25T08:35:22.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.106+0000 7f2911c0a740 1 -- start start 2024-04-25T08:35:22.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.106+0000 7f2911c0a740 1 Processor -- start 2024-04-25T08:35:22.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2911c0a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d4975cc0 0x55e7d4978980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2911c0a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 0x55e7d497b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2911c0a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eef0 0x55e7d499c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2911c0a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29081596b0 con 0x55e7d497eb10 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2911c0a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29081f7e50 con 0x55e7d4975cc0 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2911c0a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e7d4985660 con 0x55e7d497eef0 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2911c0a740 10 client.0 _ll_register_callbacks cb 0x7f28fc009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2907fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 0x55e7d497b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2907fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 0x55e7d497b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55908/0 (socket says 172.21.15.38:55908) 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2907fff640 1 -- 172.21.15.38:0/1762646693 learned_addr learned my addr 172.21.15.38:0/1762646693 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f290cca6640 1 --2- 172.21.15.38:0/1762646693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d4975cc0 0x55e7d4978980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2907fff640 1 -- 172.21.15.38:0/1762646693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eef0 msgr2=0x55e7d499c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2907fff640 1 --2- 172.21.15.38:0/1762646693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eef0 0x55e7d499c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2907fff640 1 -- 172.21.15.38:0/1762646693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d4975cc0 msgr2=0x55e7d4978980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2907fff640 1 --2- 172.21.15.38:0/1762646693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d4975cc0 0x55e7d4978980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2907fff640 1 -- 172.21.15.38:0/1762646693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7d4999400 con 0x55e7d497eb10 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f2907fff640 1 --2- 172.21.15.38:0/1762646693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 0x55e7d497b020 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f28f4049040 tx=0x7f28f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142ecc15c7f696b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.107+0000 7f28e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f28e77fe640 1 -- 172.21.15.38:0/1762646693 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4037070 con 0x55e7d497eb10 2024-04-25T08:35:22.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f28e77fe640 1 -- 172.21.15.38:0/1762646693 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f28f4004230 con 0x55e7d497eb10 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f2911c0a740 1 -- 172.21.15.38:0/1762646693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28f40328d0 con 0x55e7d497eb10 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f28e77fe640 1 -- 172.21.15.38:0/1762646693 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28f4059050 con 0x55e7d497eb10 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f2911c0a740 10 client.6579 fetch_fsmap learned FSMap version 27 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f2911c0a740 1 -- 172.21.15.38:0/1762646693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x55e7d4991df0 con 0x55e7d497eb10 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f28e77fe640 1 -- 172.21.15.38:0/1762646693 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f40328d0 con 0x55e7d497eb10 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f28e77fe640 1 -- 172.21.15.38:0/1762646693 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f28f4003610 con 0x55e7d497eb10 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f2911c0a740 10 client.6579 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f2911c0a740 10 client.6579 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f2911c0a740 20 client.6579 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1762646693 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e7d49812e0 0x55e7d499c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f2911c0a740 4 client.6579 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f2911c0a740 1 -- 172.21.15.38:0/1762646693 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e7d48c2630 con 0x55e7d49812e0 2024-04-25T08:35:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.108+0000 7f290cca6640 1 --2- 172.21.15.38:0/1762646693 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e7d49812e0 0x55e7d499c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f28e77fe640 10 client.6579 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f290cca6640 1 --2- 172.21.15.38:0/1762646693 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e7d49812e0 0x55e7d499c860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f1388a77a95a5a2 server_cookie=3582005efdc572a5 in_seq=0 out_seq=0 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f28e77fe640 1 -- 172.21.15.38:0/1762646693 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e7d48c2630 con 0x55e7d49812e0 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f28e77fe640 10 client.6579 handle_command_reply: tid=0 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f2911c0a740 1 client.6579 shutdown 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f2911c0a740 1 -- 172.21.15.38:0/1762646693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 msgr2=0x55e7d497b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1762646693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 0x55e7d497b020 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f28f4049040 tx=0x7f28f4004730 comp rx=0 tx=0).stop 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f2911c0a740 1 -- 172.21.15.38:0/1762646693 shutdown_connections 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1762646693 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x55e7d49812e0 0x55e7d499c860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1762646693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e7d497eef0 0x55e7d499c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1762646693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e7d497eb10 0x55e7d497b020 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f28e77fe640 0 client.6579 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:22.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f2911c0a740 1 --2- 172.21.15.38:0/1762646693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7d4975cc0 0x55e7d4978980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f2911c0a740 1 -- 172.21.15.38:0/1762646693 >> 172.21.15.38:0/1762646693 conn(0x55e7d4951770 msgr2=0x55e7d4950600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f2911c0a740 1 -- 172.21.15.38:0/1762646693 shutdown_connections 2024-04-25T08:35:22.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f2911c0a740 1 -- 172.21.15.38:0/1762646693 wait complete. 2024-04-25T08:35:22.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.109+0000 7f2911c0a740 20 client.6579 trim_cache size 0 max 16384 2024-04-25T08:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-04-25T08:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-04-25T08:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:22.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.110+0000 7f290ef0a640 1 -- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f28ec06b500 msgr2=0x7f28ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.110+0000 7f290ef0a640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f28ec06b500 0x7f28ec06d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009560 tx=0x7f28fc009190 comp rx=0 tx=0).stop 2024-04-25T08:35:22.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.110+0000 7f290ef0a640 1 -- 172.21.15.38:0/335523400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29080d1e90 msgr2=0x7f29080b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.110+0000 7f290ef0a640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29080d1e90 0x7f29080b3e70 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f290000c780 tx=0x7f290000cc40 comp rx=0 tx=0).stop 2024-04-25T08:35:22.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.110+0000 7f290ef0a640 1 -- 172.21.15.38:0/335523400 shutdown_connections 2024-04-25T08:35:22.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.110+0000 7f290ef0a640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29080d1e90 0x7f29080b3e70 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.110+0000 7f290ef0a640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29080d0700 0x7f290815c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.110+0000 7f290ef0a640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f28ec06b500 0x7f28ec06d9c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.110+0000 7f290ef0a640 1 --2- 172.21.15.38:0/335523400 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29080d0320 0x7f29080b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.110+0000 7f290ef0a640 1 -- 172.21.15.38:0/335523400 >> 172.21.15.38:0/335523400 conn(0x7f29080668f0 msgr2=0x7f29080b8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.110+0000 7f290ef0a640 1 -- 172.21.15.38:0/335523400 shutdown_connections 2024-04-25T08:35:22.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.110+0000 7f290ef0a640 1 -- 172.21.15.38:0/335523400 wait complete. 2024-04-25T08:35:22.115 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:35:22.115 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:22.115 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T08:35:22.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.224+0000 7f599e693640 1 Processor -- start 2024-04-25T08:35:22.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.224+0000 7f599e693640 1 -- start start 2024-04-25T08:35:22.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.225+0000 7f599e693640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f59980727e0 0x7f59980cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.225+0000 7f599e693640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59980ce1b0 con 0x7f599806bbf0 2024-04-25T08:35:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.225+0000 7f599e693640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59980ce490 con 0x7f599815b830 2024-04-25T08:35:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.225+0000 7f599e693640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59980ce770 con 0x7f59980727e0 2024-04-25T08:35:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.225+0000 7f59977fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f59980727e0 0x7f59980cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.225+0000 7f599cc30640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f599806bbf0 0x7f599806c000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53274/0 (socket says 172.21.15.38:53274) 2024-04-25T08:35:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.225+0000 7f59977fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f59980727e0 0x7f59980cf250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55918/0 (socket says 172.21.15.38:55918) 2024-04-25T08:35:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.225+0000 7f599cc30640 1 -- 172.21.15.38:0/1031204526 learned_addr learned my addr 172.21.15.38:0/1031204526 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.226+0000 7f59977fe640 1 -- 172.21.15.38:0/1031204526 >> v1:172.21.15.107:6789/0 conn(0x7f599815b830 legacy=0x7f59980cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.226+0000 7f59977fe640 1 -- 172.21.15.38:0/1031204526 >> v1:172.21.15.38:6789/0 conn(0x7f599806bbf0 legacy=0x7f599806c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.226+0000 7f59977fe640 1 -- 172.21.15.38:0/1031204526 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59980cea50 con 0x7f59980727e0 2024-04-25T08:35:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.226+0000 7f59977fe640 1 --2- 172.21.15.38:0/1031204526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59980727e0 0x7f59980cf250 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f59880097a0 tx=0x7f598802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57376fa09451994 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.226+0000 7f5996ffd640 1 -- 172.21.15.38:0/1031204526 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988037070 con 0x7f59980727e0 2024-04-25T08:35:22.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.226+0000 7f5996ffd640 1 -- 172.21.15.38:0/1031204526 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5988032440 con 0x7f59980727e0 2024-04-25T08:35:22.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.226+0000 7f5996ffd640 1 -- 172.21.15.38:0/1031204526 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988032810 con 0x7f59980727e0 2024-04-25T08:35:22.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.227+0000 7f599e693640 1 -- 172.21.15.38:0/1031204526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59980727e0 msgr2=0x7f59980cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.227+0000 7f599e693640 1 --2- 172.21.15.38:0/1031204526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59980727e0 0x7f59980cf250 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f59880097a0 tx=0x7f598802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.227+0000 7f599e693640 1 -- 172.21.15.38:0/1031204526 shutdown_connections 2024-04-25T08:35:22.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.227+0000 7f599e693640 1 --2- 172.21.15.38:0/1031204526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59980727e0 0x7f59980cf250 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.227+0000 7f599e693640 1 -- 172.21.15.38:0/1031204526 >> 172.21.15.38:0/1031204526 conn(0x7f59980717f0 msgr2=0x7f5998071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.227+0000 7f599e693640 1 -- 172.21.15.38:0/1031204526 shutdown_connections 2024-04-25T08:35:22.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.227+0000 7f599e693640 1 -- 172.21.15.38:0/1031204526 wait complete. 2024-04-25T08:35:22.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.228+0000 7f599e693640 1 Processor -- start 2024-04-25T08:35:22.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.228+0000 7f599e693640 1 -- start start 2024-04-25T08:35:22.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.229+0000 7f599e693640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f599806bbf0 0x7f59980cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.229+0000 7f599e693640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59980727e0 0x7f59981583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.229+0000 7f599e693640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f599815b830 0x7f59980ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.229+0000 7f599e693640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599815d030 con 0x7f59980727e0 2024-04-25T08:35:22.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.229+0000 7f599e693640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599815ceb0 con 0x7f599806bbf0 2024-04-25T08:35:22.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.229+0000 7f599e693640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599815d1b0 con 0x7f599815b830 2024-04-25T08:35:22.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.229+0000 7f59977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59980727e0 0x7f59981583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.229+0000 7f59977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59980727e0 0x7f59981583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55924/0 (socket says 172.21.15.38:55924) 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.229+0000 7f59977fe640 1 -- 172.21.15.38:0/632084609 learned_addr learned my addr 172.21.15.38:0/632084609 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.229+0000 7f5997fff640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f599806bbf0 0x7f59980cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.229+0000 7f59977fe640 1 -- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f599815b830 msgr2=0x7f59980ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.229+0000 7f599cc30640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f599815b830 0x7f59980ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.230+0000 7f59977fe640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f599815b830 0x7f59980ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.230+0000 7f59977fe640 1 -- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f599806bbf0 msgr2=0x7f59980cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.230+0000 7f59977fe640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f599806bbf0 0x7f59980cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.230+0000 7f59977fe640 1 -- 172.21.15.38:0/632084609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59980ae730 con 0x7f59980727e0 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.230+0000 7f5997fff640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f599806bbf0 0x7f59980cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.230+0000 7f59977fe640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59980727e0 0x7f59981583b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f59880098c0 tx=0x7f59880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.230+0000 7f5994ff9640 1 -- 172.21.15.38:0/632084609 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988037040 con 0x7f59980727e0 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.230+0000 7f5994ff9640 1 -- 172.21.15.38:0/632084609 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5988042070 con 0x7f59980727e0 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.230+0000 7f599e693640 1 -- 172.21.15.38:0/632084609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59980ae9d0 con 0x7f59980727e0 2024-04-25T08:35:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.230+0000 7f5994ff9640 1 -- 172.21.15.38:0/632084609 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988030a70 con 0x7f59980727e0 2024-04-25T08:35:22.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.230+0000 7f599e693640 1 -- 172.21.15.38:0/632084609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59980aef90 con 0x7f59980727e0 2024-04-25T08:35:22.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.231+0000 7f5994ff9640 1 -- 172.21.15.38:0/632084609 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5988030c10 con 0x7f59980727e0 2024-04-25T08:35:22.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.232+0000 7f5994ff9640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f598406b4d0 0x7f598406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.232+0000 7f5994ff9640 1 -- 172.21.15.38:0/632084609 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f59880aae50 con 0x7f59980727e0 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.232+0000 7f5997fff640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f598406b4d0 0x7f598406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.233+0000 7f5997fff640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f598406b4d0 0x7f598406d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f598c00a830 tx=0x7f598c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.233+0000 7f59a1393740 1 Processor -- start 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.233+0000 7f59a1393740 1 -- start start 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59a1393740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de4632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59a1393740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 0x5588de4637f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59a1393740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de433700 0x5588de436f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59a1393740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588de39ce00 con 0x5588de460e80 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59a1393740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588de345d20 con 0x5588de433700 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59a1393740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588de398ea0 con 0x5588de460aa0 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 0x5588de4637f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 0x5588de4637f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55936/0 (socket says 172.21.15.38:55936) 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59977fe640 1 -- 172.21.15.38:0/2984683996 learned_addr learned my addr 172.21.15.38:0/2984683996 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f599cc30640 1 --2- 172.21.15.38:0/2984683996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de433700 0x5588de436f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f5997fff640 1 --2- 172.21.15.38:0/2984683996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de4632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59977fe640 1 -- 172.21.15.38:0/2984683996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 msgr2=0x5588de4632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59977fe640 1 --2- 172.21.15.38:0/2984683996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de4632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59977fe640 1 -- 172.21.15.38:0/2984683996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de433700 msgr2=0x5588de436f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59977fe640 1 --2- 172.21.15.38:0/2984683996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de433700 0x5588de436f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59977fe640 1 -- 172.21.15.38:0/2984683996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588de367df0 con 0x5588de460e80 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f59977fe640 1 --2- 172.21.15.38:0/2984683996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 0x5588de4637f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5988079110 tx=0x7f59880757e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726d169d661cdfa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f597dffb640 1 -- 172.21.15.38:0/2984683996 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988079410 con 0x5588de460e80 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f597dffb640 1 -- 172.21.15.38:0/2984683996 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598807d560 con 0x5588de460e80 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.234+0000 7f597dffb640 1 -- 172.21.15.38:0/2984683996 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5988081b20 con 0x5588de460e80 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.235+0000 7f59a1393740 1 -- 172.21.15.38:0/2984683996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 msgr2=0x5588de4637f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.235+0000 7f59a1393740 1 --2- 172.21.15.38:0/2984683996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 0x5588de4637f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5988079110 tx=0x7f59880757e0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.235+0000 7f59a1393740 1 -- 172.21.15.38:0/2984683996 shutdown_connections 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.235+0000 7f59a1393740 1 --2- 172.21.15.38:0/2984683996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de433700 0x5588de436f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.235+0000 7f59a1393740 1 --2- 172.21.15.38:0/2984683996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 0x5588de4637f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.235+0000 7f59a1393740 1 --2- 172.21.15.38:0/2984683996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de4632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.235+0000 7f59a1393740 1 -- 172.21.15.38:0/2984683996 >> 172.21.15.38:0/2984683996 conn(0x5588de435970 msgr2=0x5588de47e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.235+0000 7f59a1393740 1 -- 172.21.15.38:0/2984683996 shutdown_connections 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.235+0000 7f59a1393740 1 -- 172.21.15.38:0/2984683996 wait complete. 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.235+0000 7f59a1393740 1 -- start start 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.236+0000 7f59a1393740 1 Processor -- start 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.236+0000 7f59a1393740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de465e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.236+0000 7f59a1393740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460e80 0x5588de485560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.236+0000 7f59a1393740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 0x5588de485aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.236+0000 7f59a1393740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588de438b20 con 0x5588de45f5a0 2024-04-25T08:35:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.236+0000 7f59a1393740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588de33b590 con 0x5588de460e80 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f59a1393740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588de438e00 con 0x5588de460aa0 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f59a1393740 10 client.0 _ll_register_callbacks cb 0x5588de347e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f599cc30640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 0x5588de485aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f5997fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de465e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f599cc30640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 0x5588de485aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55944/0 (socket says 172.21.15.38:55944) 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f599cc30640 1 -- 172.21.15.38:0/817494201 learned_addr learned my addr 172.21.15.38:0/817494201 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f5997fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de465e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:35916/0 (socket says 172.21.15.38:35916) 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f59977fe640 1 --2- 172.21.15.38:0/817494201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460e80 0x5588de485560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f599cc30640 1 -- 172.21.15.38:0/817494201 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 msgr2=0x5588de465e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f599cc30640 1 --2- 172.21.15.38:0/817494201 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de465e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f599cc30640 1 -- 172.21.15.38:0/817494201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460e80 msgr2=0x5588de485560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f599cc30640 1 --2- 172.21.15.38:0/817494201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460e80 0x5588de485560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f599cc30640 1 -- 172.21.15.38:0/817494201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588de0b13f0 con 0x5588de45f5a0 2024-04-25T08:35:22.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f599cc30640 1 --2- 172.21.15.38:0/817494201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 0x5588de485aa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f5990010880 tx=0x7f5990010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bd453c7724b576 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f5976ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f5976ffd640 1 -- 172.21.15.38:0/817494201 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599000eb50 con 0x5588de45f5a0 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f5976ffd640 1 -- 172.21.15.38:0/817494201 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f599000ecf0 con 0x5588de45f5a0 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f59a1393740 1 -- 172.21.15.38:0/817494201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f598807d560 con 0x5588de45f5a0 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.237+0000 7f5976ffd640 1 -- 172.21.15.38:0/817494201 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5990021050 con 0x5588de45f5a0 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f59a1393740 10 client.6594 fetch_fsmap learned FSMap version 27 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f59a1393740 1 -- 172.21.15.38:0/817494201 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5588de17b7d0 con 0x5588de45f5a0 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f5976ffd640 1 -- 172.21.15.38:0/817494201 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598807d560 con 0x5588de45f5a0 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f5976ffd640 1 -- 172.21.15.38:0/817494201 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5990024020 con 0x5588de45f5a0 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f59a1393740 10 client.6594 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f59a1393740 10 client.6594 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f59a1393740 20 client.6594 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f59a1393740 1 --2- 172.21.15.38:0/817494201 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5588de480b40 0x5588de480f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f59a1393740 4 client.6594 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f59a1393740 1 -- 172.21.15.38:0/817494201 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588de3aa340 con 0x5588de480b40 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f5997fff640 1 --2- 172.21.15.38:0/817494201 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5588de480b40 0x5588de480f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f5976ffd640 10 client.6594 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.238+0000 7f5997fff640 1 --2- 172.21.15.38:0/817494201 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5588de480b40 0x5588de480f20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa265adccfcdf0d server_cookie=e08119bf7581e6a1 in_seq=0 out_seq=0 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f5976ffd640 1 -- 172.21.15.38:0/817494201 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588de3aa340 con 0x5588de480b40 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f5976ffd640 10 client.6594 handle_command_reply: tid=0 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f59a1393740 1 client.6594 shutdown 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f59a1393740 1 -- 172.21.15.38:0/817494201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 msgr2=0x5588de485aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f59a1393740 1 --2- 172.21.15.38:0/817494201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 0x5588de485aa0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f5990010880 tx=0x7f5990010d40 comp rx=0 tx=0).stop 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f59a1393740 1 -- 172.21.15.38:0/817494201 shutdown_connections 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f59a1393740 1 --2- 172.21.15.38:0/817494201 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5588de480b40 0x5588de480f20 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f59a1393740 1 --2- 172.21.15.38:0/817494201 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 0x5588de485aa0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f59a1393740 1 --2- 172.21.15.38:0/817494201 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460e80 0x5588de485560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f59a1393740 1 --2- 172.21.15.38:0/817494201 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de465e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f5976ffd640 0 client.6594 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:22.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f59a1393740 1 -- 172.21.15.38:0/817494201 >> 172.21.15.38:0/817494201 conn(0x5588de433700 msgr2=0x5588de47be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f59a1393740 1 -- 172.21.15.38:0/817494201 shutdown_connections 2024-04-25T08:35:22.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f59a1393740 1 -- 172.21.15.38:0/817494201 wait complete. 2024-04-25T08:35:22.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.239+0000 7f59a1393740 20 client.6594 trim_cache size 0 max 16384 2024-04-25T08:35:22.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.251+0000 7f59a1393740 1 Processor -- start 2024-04-25T08:35:22.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.251+0000 7f59a1393740 1 -- start start 2024-04-25T08:35:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f59a1393740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 0x5588de438f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f59a1393740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de47bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f59a1393740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460e80 0x5588de47c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f59a1393740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59982005c0 con 0x5588de45f5a0 2024-04-25T08:35:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f59a1393740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59981ff7b0 con 0x5588de460e80 2024-04-25T08:35:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f59a1393740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59980c9170 con 0x5588de460aa0 2024-04-25T08:35:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f5997fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 0x5588de438f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f5997fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 0x5588de438f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55946/0 (socket says 172.21.15.38:55946) 2024-04-25T08:35:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f5997fff640 1 -- 172.21.15.38:0/3399730830 learned_addr learned my addr 172.21.15.38:0/3399730830 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f599cc30640 1 --2- 172.21.15.38:0/3399730830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460e80 0x5588de47c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f59977fe640 1 --2- 172.21.15.38:0/3399730830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de47bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f5997fff640 1 -- 172.21.15.38:0/3399730830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 msgr2=0x5588de47bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f5997fff640 1 --2- 172.21.15.38:0/3399730830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de47bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f5997fff640 1 -- 172.21.15.38:0/3399730830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460e80 msgr2=0x5588de47c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f5997fff640 1 --2- 172.21.15.38:0/3399730830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460e80 0x5588de47c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f5997fff640 1 -- 172.21.15.38:0/3399730830 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588de42f690 con 0x5588de45f5a0 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f599cc30640 1 --2- 172.21.15.38:0/3399730830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460e80 0x5588de47c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f59977fe640 1 --2- 172.21.15.38:0/3399730830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de47bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.252+0000 7f5997fff640 1 --2- 172.21.15.38:0/3399730830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 0x5588de438f20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f598c00c880 tx=0x7f598c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8b1d628e41f2b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.253+0000 7f597d7fa640 1 -- 172.21.15.38:0/3399730830 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c004020 con 0x5588de45f5a0 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.253+0000 7f597d7fa640 1 -- 172.21.15.38:0/3399730830 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c0041c0 con 0x5588de45f5a0 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.253+0000 7f597d7fa640 1 -- 172.21.15.38:0/3399730830 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c0035f0 con 0x5588de45f5a0 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.253+0000 7f59a1393740 1 -- 172.21.15.38:0/3399730830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 msgr2=0x5588de438f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.253+0000 7f59a1393740 1 --2- 172.21.15.38:0/3399730830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 0x5588de438f20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f598c00c880 tx=0x7f598c012000 comp rx=0 tx=0).stop 2024-04-25T08:35:22.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.253+0000 7f59a1393740 1 -- 172.21.15.38:0/3399730830 shutdown_connections 2024-04-25T08:35:22.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.253+0000 7f59a1393740 1 --2- 172.21.15.38:0/3399730830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460e80 0x5588de47c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.253+0000 7f59a1393740 1 --2- 172.21.15.38:0/3399730830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de460aa0 0x5588de47bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.253+0000 7f59a1393740 1 --2- 172.21.15.38:0/3399730830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de45f5a0 0x5588de438f20 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.253+0000 7f59a1393740 1 -- 172.21.15.38:0/3399730830 >> 172.21.15.38:0/3399730830 conn(0x5588de433700 msgr2=0x5588de42edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.253+0000 7f59a1393740 1 -- 172.21.15.38:0/3399730830 shutdown_connections 2024-04-25T08:35:22.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.253+0000 7f59a1393740 1 -- 172.21.15.38:0/3399730830 wait complete. 2024-04-25T08:35:22.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.254+0000 7f59a1393740 1 -- start start 2024-04-25T08:35:22.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.254+0000 7f59a1393740 1 Processor -- start 2024-04-25T08:35:22.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f59a1393740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de45f5a0 0x5588de47d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f59a1393740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460aa0 0x5588de47be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f59a1393740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 0x5588de481e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f59a1393740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588de45f420 con 0x5588de460e80 2024-04-25T08:35:22.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f59a1393740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59980c9770 con 0x5588de460aa0 2024-04-25T08:35:22.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f59a1393740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588de4602e0 con 0x5588de45f5a0 2024-04-25T08:35:22.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f59a1393740 10 client.0 _ll_register_callbacks cb 0x7f5988006c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:22.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f599cc30640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 0x5588de481e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f599cc30640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 0x5588de481e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55954/0 (socket says 172.21.15.38:55954) 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f599cc30640 1 -- 172.21.15.38:0/1909758241 learned_addr learned my addr 172.21.15.38:0/1909758241 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f59977fe640 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460aa0 0x5588de47be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f5997fff640 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de45f5a0 0x5588de47d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f599cc30640 1 -- 172.21.15.38:0/1909758241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de45f5a0 msgr2=0x5588de47d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f599cc30640 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de45f5a0 0x5588de47d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f599cc30640 1 -- 172.21.15.38:0/1909758241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460aa0 msgr2=0x5588de47be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f599cc30640 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460aa0 0x5588de47be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f599cc30640 1 -- 172.21.15.38:0/1909758241 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588de47d4e0 con 0x5588de460e80 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f5997fff640 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de45f5a0 0x5588de47d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f599cc30640 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 0x5588de481e80 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f59900013d0 tx=0x7f5990005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e79da5329f33e25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.255+0000 7f5976ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f5976ffd640 1 -- 172.21.15.38:0/1909758241 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5990032060 con 0x5588de460e80 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f5976ffd640 1 -- 172.21.15.38:0/1909758241 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5990014a80 con 0x5588de460e80 2024-04-25T08:35:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f59a1393740 1 -- 172.21.15.38:0/1909758241 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f598c0041c0 con 0x5588de460e80 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f5976ffd640 1 -- 172.21.15.38:0/1909758241 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f599000e590 con 0x5588de460e80 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f59a1393740 10 client.6600 fetch_fsmap learned FSMap version 27 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f59a1393740 1 -- 172.21.15.38:0/1909758241 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x5588de45ea90 con 0x5588de460e80 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f5976ffd640 1 -- 172.21.15.38:0/1909758241 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c0041c0 con 0x5588de460e80 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f5976ffd640 1 -- 172.21.15.38:0/1909758241 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f599002c030 con 0x5588de460e80 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f59a1393740 10 client.6600 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f59a1393740 10 client.6600 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f59a1393740 20 client.6600 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f59a1393740 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5588de4823c0 0x5588de4e9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f59a1393740 4 client.6600 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f59a1393740 1 -- 172.21.15.38:0/1909758241 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5588de3a4690 con 0x5588de4823c0 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.256+0000 7f5997fff640 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5588de4823c0 0x5588de4e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f5976ffd640 10 client.6600 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f5997fff640 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5588de4823c0 0x5588de4e9650 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4899f833468528cf server_cookie=fd958a14d2e9518 in_seq=0 out_seq=0 2024-04-25T08:35:22.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f5976ffd640 1 -- 172.21.15.38:0/1909758241 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x5588de3a4690 con 0x5588de4823c0 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f5976ffd640 10 client.6600 handle_command_reply: tid=0 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f59a1393740 1 client.6600 shutdown 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f59a1393740 1 -- 172.21.15.38:0/1909758241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 msgr2=0x5588de481e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f59a1393740 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 0x5588de481e80 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f59900013d0 tx=0x7f5990005b70 comp rx=0 tx=0).stop 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f59a1393740 1 -- 172.21.15.38:0/1909758241 shutdown_connections 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f59a1393740 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x5588de4823c0 0x5588de4e9650 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f59a1393740 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5588de460e80 0x5588de481e80 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f59a1393740 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5588de460aa0 0x5588de47be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f59a1393740 1 --2- 172.21.15.38:0/1909758241 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5588de45f5a0 0x5588de47d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f5976ffd640 0 client.6600 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f59a1393740 1 -- 172.21.15.38:0/1909758241 >> 172.21.15.38:0/1909758241 conn(0x5588de433700 msgr2=0x5588de4858e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f59a1393740 1 -- 172.21.15.38:0/1909758241 shutdown_connections 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f59a1393740 1 -- 172.21.15.38:0/1909758241 wait complete. 2024-04-25T08:35:22.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.257+0000 7f59a1393740 20 client.6600 trim_cache size 0 max 16384 2024-04-25T08:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T08:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "damage_type": "backtrace", 2024-04-25T08:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 3876340424, 2024-04-25T08:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 1099511630414, 2024-04-25T08:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-25T08:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T08:35:22.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.258+0000 7f599e693640 1 -- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f598406b4d0 msgr2=0x7f598406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.258+0000 7f599e693640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f598406b4d0 0x7f598406d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f598c00a830 tx=0x7f598c008040 comp rx=0 tx=0).stop 2024-04-25T08:35:22.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.258+0000 7f599e693640 1 -- 172.21.15.38:0/632084609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59980727e0 msgr2=0x7f59981583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.258+0000 7f599e693640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59980727e0 0x7f59981583b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f59880098c0 tx=0x7f59880046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.258+0000 7f599e693640 1 -- 172.21.15.38:0/632084609 shutdown_connections 2024-04-25T08:35:22.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.258+0000 7f599e693640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f598406b4d0 0x7f598406d990 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.258+0000 7f599e693640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f599815b830 0x7f59980ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.258+0000 7f599e693640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59980727e0 0x7f59981583b0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.258+0000 7f599e693640 1 --2- 172.21.15.38:0/632084609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f599806bbf0 0x7f59980cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.258+0000 7f599e693640 1 -- 172.21.15.38:0/632084609 >> 172.21.15.38:0/632084609 conn(0x7f59980717f0 msgr2=0x7f5998073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.258+0000 7f599e693640 1 -- 172.21.15.38:0/632084609 shutdown_connections 2024-04-25T08:35:22.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.258+0000 7f599e693640 1 -- 172.21.15.38:0/632084609 wait complete. 2024-04-25T08:35:22.263 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.369+0000 7f015bba5640 1 Processor -- start 2024-04-25T08:35:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.369+0000 7f015bba5640 1 -- start start 2024-04-25T08:35:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f015bba5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f015415df50 0x7f01540ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f015bba5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f01540ce2e0 0x7f01540d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f015bba5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01540ceff0 con 0x7f015406c110 2024-04-25T08:35:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f015bba5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01540cf2d0 con 0x7f015415df50 2024-04-25T08:35:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f015bba5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01540d1f70 con 0x7f01540ce2e0 2024-04-25T08:35:22.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f0159140640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f015415df50 0x7f01540ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f0159140640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f015415df50 0x7f01540ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55968/0 (socket says 172.21.15.38:55968) 2024-04-25T08:35:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f0159140640 1 -- 172.21.15.38:0/2883528770 learned_addr learned my addr 172.21.15.38:0/2883528770 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f015a142640 1 --2- 172.21.15.38:0/2883528770 >> v2:172.21.15.107:3300/0 conn(0x7f01540ce2e0 0x7f01540d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f0159140640 1 -- 172.21.15.38:0/2883528770 >> v2:172.21.15.107:3300/0 conn(0x7f01540ce2e0 msgr2=0x7f01540d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f0159140640 1 --2- 172.21.15.38:0/2883528770 >> v2:172.21.15.107:3300/0 conn(0x7f01540ce2e0 0x7f01540d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f0159140640 1 -- 172.21.15.38:0/2883528770 >> v1:172.21.15.107:6789/0 conn(0x7f015406c110 legacy=0x7f015406c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.370+0000 7f0159140640 1 -- 172.21.15.38:0/2883528770 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01540d2250 con 0x7f015415df50 2024-04-25T08:35:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.371+0000 7f0159140640 1 --2- 172.21.15.38:0/2883528770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015415df50 0x7f01540ce7a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f01540b6520 tx=0x7f014402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9770ee53109bc3e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.371+0000 7f015893f640 1 -- 172.21.15.38:0/2883528770 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144037070 con 0x7f015415df50 2024-04-25T08:35:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.371+0000 7f015893f640 1 -- 172.21.15.38:0/2883528770 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0144032440 con 0x7f015415df50 2024-04-25T08:35:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.371+0000 7f015893f640 1 -- 172.21.15.38:0/2883528770 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144032810 con 0x7f015415df50 2024-04-25T08:35:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.372+0000 7f015bba5640 1 -- 172.21.15.38:0/2883528770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015415df50 msgr2=0x7f01540ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.372+0000 7f015bba5640 1 --2- 172.21.15.38:0/2883528770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015415df50 0x7f01540ce7a0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f01540b6520 tx=0x7f014402a270 comp rx=0 tx=0).stop 2024-04-25T08:35:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.372+0000 7f015bba5640 1 -- 172.21.15.38:0/2883528770 shutdown_connections 2024-04-25T08:35:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.372+0000 7f015bba5640 1 --2- 172.21.15.38:0/2883528770 >> v2:172.21.15.107:3300/0 conn(0x7f01540ce2e0 0x7f01540d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.372+0000 7f015bba5640 1 --2- 172.21.15.38:0/2883528770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015415df50 0x7f01540ce7a0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.372+0000 7f015bba5640 1 -- 172.21.15.38:0/2883528770 >> 172.21.15.38:0/2883528770 conn(0x7f01540d3fb0 msgr2=0x7f01540d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.372+0000 7f015bba5640 1 -- 172.21.15.38:0/2883528770 shutdown_connections 2024-04-25T08:35:22.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.372+0000 7f015bba5640 1 -- 172.21.15.38:0/2883528770 wait complete. 2024-04-25T08:35:22.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.373+0000 7f015bba5640 1 Processor -- start 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.373+0000 7f015bba5640 1 -- start start 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f015bba5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015406c110 0x7f01540b5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f015bba5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01540ce2e0 0x7f01541f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f015bba5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f015415df50 0x7f01540c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f015bba5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01540d2f90 con 0x7f015406c110 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f015bba5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01540cece0 con 0x7f01540ce2e0 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f015bba5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01540d3110 con 0x7f015415df50 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f0159941640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015406c110 0x7f01540b5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f0159941640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015406c110 0x7f01540b5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55972/0 (socket says 172.21.15.38:55972) 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f0159941640 1 -- 172.21.15.38:0/2249879747 learned_addr learned my addr 172.21.15.38:0/2249879747 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f015a142640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f015415df50 0x7f01540c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f0159140640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01540ce2e0 0x7f01541f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f0159941640 1 -- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f015415df50 msgr2=0x7f01540c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f0159941640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f015415df50 0x7f01540c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f0159941640 1 -- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01540ce2e0 msgr2=0x7f01541f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f0159941640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01540ce2e0 0x7f01541f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f0159941640 1 -- 172.21.15.38:0/2249879747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f015415e7c0 con 0x7f015406c110 2024-04-25T08:35:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f0159140640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01540ce2e0 0x7f01541f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.374+0000 7f0159941640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015406c110 0x7f01540b5ee0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f015000c430 tx=0x7f015000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.375+0000 7f014a7fc640 1 -- 172.21.15.38:0/2249879747 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500052d0 con 0x7f015406c110 2024-04-25T08:35:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.375+0000 7f014a7fc640 1 -- 172.21.15.38:0/2249879747 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0150005490 con 0x7f015406c110 2024-04-25T08:35:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.375+0000 7f014a7fc640 1 -- 172.21.15.38:0/2249879747 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150007260 con 0x7f015406c110 2024-04-25T08:35:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.375+0000 7f015bba5640 1 -- 172.21.15.38:0/2249879747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015415eaa0 con 0x7f015406c110 2024-04-25T08:35:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.375+0000 7f015bba5640 1 -- 172.21.15.38:0/2249879747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015415f060 con 0x7f015406c110 2024-04-25T08:35:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.376+0000 7f015bba5640 1 -- 172.21.15.38:0/2249879747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0154068e80 con 0x7f015406c110 2024-04-25T08:35:22.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.376+0000 7f014a7fc640 1 -- 172.21.15.38:0/2249879747 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0150007400 con 0x7f015406c110 2024-04-25T08:35:22.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.377+0000 7f014a7fc640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f013806b4d0 0x7f013806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.377+0000 7f014a7fc640 1 -- 172.21.15.38:0/2249879747 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0150089820 con 0x7f015406c110 2024-04-25T08:35:22.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.377+0000 7f0159140640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f013806b4d0 0x7f013806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.377+0000 7f0159140640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f013806b4d0 0x7f013806d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0144034000 tx=0x7f014402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.381+0000 7f014a7fc640 1 -- 172.21.15.38:0/2249879747 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0150058570 con 0x7f015406c110 2024-04-25T08:35:22.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.597+0000 7f015bba5640 1 -- 172.21.15.38:0/2249879747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01540d14a0 con 0x7f015406c110 2024-04-25T08:35:22.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.598+0000 7f014a7fc640 1 -- 172.21.15.38:0/2249879747 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f0150057e70 con 0x7f015406c110 2024-04-25T08:35:22.592 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:22.593 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:35:20.877703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:22.593 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-04-25T08:35:22.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.599+0000 7f015bba5640 1 -- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f013806b4d0 msgr2=0x7f013806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.599+0000 7f015bba5640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f013806b4d0 0x7f013806d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0144034000 tx=0x7f014402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:22.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.599+0000 7f015bba5640 1 -- 172.21.15.38:0/2249879747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015406c110 msgr2=0x7f01540b5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.599+0000 7f015bba5640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015406c110 0x7f01540b5ee0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f015000c430 tx=0x7f015000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.599+0000 7f015bba5640 1 -- 172.21.15.38:0/2249879747 shutdown_connections 2024-04-25T08:35:22.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.599+0000 7f015bba5640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f013806b4d0 0x7f013806d990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.599+0000 7f015bba5640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f015415df50 0x7f01540c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.599+0000 7f015bba5640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01540ce2e0 0x7f01541f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.599+0000 7f015bba5640 1 --2- 172.21.15.38:0/2249879747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015406c110 0x7f01540b5ee0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.599+0000 7f015bba5640 1 -- 172.21.15.38:0/2249879747 >> 172.21.15.38:0/2249879747 conn(0x7f01540d3fb0 msgr2=0x7f015415d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.600+0000 7f015bba5640 1 -- 172.21.15.38:0/2249879747 shutdown_connections 2024-04-25T08:35:22.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.600+0000 7f015bba5640 1 -- 172.21.15.38:0/2249879747 wait complete. 2024-04-25T08:35:22.605 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage rm 3876340424 2024-04-25T08:35:22.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.714+0000 7f33f30bd640 1 Processor -- start 2024-04-25T08:35:22.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.714+0000 7f33f30bd640 1 -- start start 2024-04-25T08:35:22.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33f30bd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33ec0cfd40 0x7f33ec0c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33f30bd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33ec0ccfc0 0x7f33ec0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33f30bd640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33ec0ac430 con 0x7f33ec15db80 2024-04-25T08:35:22.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33f30bd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ec0ac1e0 con 0x7f33ec0cfd40 2024-04-25T08:35:22.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33f30bd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0ac710 con 0x7f33ec0ccfc0 2024-04-25T08:35:22.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33ebfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33ec0cfd40 0x7f33ec0c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33f0e59640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f33ec15db80 0x7f33ec0cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53290/0 (socket says 172.21.15.38:53290) 2024-04-25T08:35:22.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33f0e59640 1 -- 172.21.15.38:0/1158557775 learned_addr learned my addr 172.21.15.38:0/1158557775 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:22.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33f165a640 1 --2- 172.21.15.38:0/1158557775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33ec0ccfc0 0x7f33ec0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33ebfff640 1 -- 172.21.15.38:0/1158557775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33ec0ccfc0 msgr2=0x7f33ec0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33ebfff640 1 --2- 172.21.15.38:0/1158557775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33ec0ccfc0 0x7f33ec0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33ebfff640 1 -- 172.21.15.38:0/1158557775 >> v1:172.21.15.38:6789/0 conn(0x7f33ec15db80 legacy=0x7f33ec0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33ebfff640 1 -- 172.21.15.38:0/1158557775 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec0ac9f0 con 0x7f33ec0cfd40 2024-04-25T08:35:22.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.715+0000 7f33eb7fe640 1 -- 172.21.15.38:0/1158557775 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2341400817 0 0) 0x7f33ec0ac430 con 0x7f33ec15db80 2024-04-25T08:35:22.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.716+0000 7f33ebfff640 1 --2- 172.21.15.38:0/1158557775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ec0cfd40 0x7f33ec0c9ad0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f33e00097a0 tx=0x7f33e002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a140c7c942f84db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.716+0000 7f33eb7fe640 1 -- 172.21.15.38:0/1158557775 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0037070 con 0x7f33ec0cfd40 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.716+0000 7f33eb7fe640 1 -- 172.21.15.38:0/1158557775 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0032440 con 0x7f33ec0cfd40 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.716+0000 7f33eb7fe640 1 -- 172.21.15.38:0/1158557775 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0032810 con 0x7f33ec0cfd40 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.717+0000 7f33f30bd640 1 -- 172.21.15.38:0/1158557775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ec0cfd40 msgr2=0x7f33ec0c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.717+0000 7f33f30bd640 1 --2- 172.21.15.38:0/1158557775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ec0cfd40 0x7f33ec0c9ad0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f33e00097a0 tx=0x7f33e002a180 comp rx=0 tx=0).stop 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.717+0000 7f33f30bd640 1 -- 172.21.15.38:0/1158557775 shutdown_connections 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.717+0000 7f33f30bd640 1 --2- 172.21.15.38:0/1158557775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33ec0ccfc0 0x7f33ec0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.717+0000 7f33f30bd640 1 --2- 172.21.15.38:0/1158557775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ec0cfd40 0x7f33ec0c9ad0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.717+0000 7f33f30bd640 1 -- 172.21.15.38:0/1158557775 >> 172.21.15.38:0/1158557775 conn(0x7f33ec06adc0 msgr2=0x7f33ec06b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.717+0000 7f33f30bd640 1 -- 172.21.15.38:0/1158557775 shutdown_connections 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.717+0000 7f33f30bd640 1 -- 172.21.15.38:0/1158557775 wait complete. 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.718+0000 7f33f30bd640 1 Processor -- start 2024-04-25T08:35:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.718+0000 7f33f30bd640 1 -- start start 2024-04-25T08:35:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33f30bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33ec0ccfc0 0x7f33ec159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33f30bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ec0cfd40 0x7f33ec071e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33f30bd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33ec15db80 0x7f33ec0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33f30bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec15df60 con 0x7f33ec0cfd40 2024-04-25T08:35:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33f30bd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0ca550 con 0x7f33ec0ccfc0 2024-04-25T08:35:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33f30bd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec15e2c0 con 0x7f33ec15db80 2024-04-25T08:35:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ec0cfd40 0x7f33ec071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ec0cfd40 0x7f33ec071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55990/0 (socket says 172.21.15.38:55990) 2024-04-25T08:35:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33ebfff640 1 -- 172.21.15.38:0/2537283563 learned_addr learned my addr 172.21.15.38:0/2537283563 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33f165a640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33ec15db80 0x7f33ec0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33f0e59640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33ec0ccfc0 0x7f33ec159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33ebfff640 1 -- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33ec15db80 msgr2=0x7f33ec0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33ebfff640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33ec15db80 0x7f33ec0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33ebfff640 1 -- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33ec0ccfc0 msgr2=0x7f33ec159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33ebfff640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33ec0ccfc0 0x7f33ec159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.719+0000 7f33ebfff640 1 -- 172.21.15.38:0/2537283563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec098530 con 0x7f33ec0cfd40 2024-04-25T08:35:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.720+0000 7f33ebfff640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ec0cfd40 0x7f33ec071e60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000c00 tx=0x7f33e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.720+0000 7f33e97fa640 1 -- 172.21.15.38:0/2537283563 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0037040 con 0x7f33ec0cfd40 2024-04-25T08:35:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.720+0000 7f33e97fa640 1 -- 172.21.15.38:0/2537283563 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0042070 con 0x7f33ec0cfd40 2024-04-25T08:35:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.720+0000 7f33f30bd640 1 -- 172.21.15.38:0/2537283563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ec098810 con 0x7f33ec0cfd40 2024-04-25T08:35:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.720+0000 7f33e97fa640 1 -- 172.21.15.38:0/2537283563 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0030940 con 0x7f33ec0cfd40 2024-04-25T08:35:22.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.720+0000 7f33f30bd640 1 -- 172.21.15.38:0/2537283563 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ec0c5670 con 0x7f33ec0cfd40 2024-04-25T08:35:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.722+0000 7f33e97fa640 1 -- 172.21.15.38:0/2537283563 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33e0030ae0 con 0x7f33ec0cfd40 2024-04-25T08:35:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.722+0000 7f33e97fa640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33d806b4d0 0x7f33d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.722+0000 7f33e97fa640 1 -- 172.21.15.38:0/2537283563 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f33e00aad70 con 0x7f33ec0cfd40 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.722+0000 7f33f0e59640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33d806b4d0 0x7f33d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.723+0000 7f33f0e59640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33d806b4d0 0x7f33d806d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f33dc000f60 tx=0x7f33dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.723+0000 7f33f5dc2740 1 Processor -- start 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.723+0000 7f33f5dc2740 1 -- start start 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.723+0000 7f33f5dc2740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557829131f80 0x557829134790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f5dc2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 0x557829134cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f5dc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829104be0 0x5578291083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f5dc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782906dac0 con 0x557829104be0 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f5dc2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557829016db0 con 0x557829132360 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f5dc2740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x557829069b60 con 0x557829131f80 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f165a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829104be0 0x5578291083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f165a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829104be0 0x5578291083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55994/0 (socket says 172.21.15.38:55994) 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f165a640 1 -- 172.21.15.38:0/2713250848 learned_addr learned my addr 172.21.15.38:0/2713250848 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f0e59640 1 --2- 172.21.15.38:0/2713250848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557829131f80 0x557829134790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33ebfff640 1 --2- 172.21.15.38:0/2713250848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 0x557829134cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f165a640 1 -- 172.21.15.38:0/2713250848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557829131f80 msgr2=0x557829134790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f165a640 1 --2- 172.21.15.38:0/2713250848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557829131f80 0x557829134790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f165a640 1 -- 172.21.15.38:0/2713250848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 msgr2=0x557829134cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f165a640 1 --2- 172.21.15.38:0/2713250848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 0x557829134cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f165a640 1 -- 172.21.15.38:0/2713250848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557828e4e710 con 0x557829104be0 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33f165a640 1 --2- 172.21.15.38:0/2713250848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829104be0 0x5578291083f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f33e40074c0 tx=0x7f33e4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df2cf66c85405ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33d27fc640 1 -- 172.21.15.38:0/2713250848 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4019020 con 0x557829104be0 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33d27fc640 1 -- 172.21.15.38:0/2713250848 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e4015600 con 0x557829104be0 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.724+0000 7f33d27fc640 1 -- 172.21.15.38:0/2713250848 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e40159d0 con 0x557829104be0 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.725+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2713250848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829104be0 msgr2=0x5578291083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.725+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/2713250848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829104be0 0x5578291083f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f33e40074c0 tx=0x7f33e4009750 comp rx=0 tx=0).stop 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.725+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2713250848 shutdown_connections 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.725+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/2713250848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829104be0 0x5578291083f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.725+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/2713250848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 0x557829134cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.725+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/2713250848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557829131f80 0x557829134790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.725+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2713250848 >> 172.21.15.38:0/2713250848 conn(0x557829106e50 msgr2=0x55782914f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.725+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2713250848 shutdown_connections 2024-04-25T08:35:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.725+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2713250848 wait complete. 2024-04-25T08:35:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.725+0000 7f33f5dc2740 1 -- start start 2024-04-25T08:35:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.726+0000 7f33f5dc2740 1 Processor -- start 2024-04-25T08:35:22.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.726+0000 7f33f5dc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 0x55782911d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.726+0000 7f33f5dc2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 0x55782912f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.726+0000 7f33f5dc2740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55782911d6a0 0x55782912f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f5dc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782910a000 con 0x557829131f80 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f5dc2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782900c4e0 con 0x557829132360 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f5dc2740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55782910a2e0 con 0x55782911d6a0 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f0e59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 0x55782911d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f5dc2740 10 client.0 _ll_register_callbacks cb 0x557828d81af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f0e59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 0x55782911d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55998/0 (socket says 172.21.15.38:55998) 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f0e59640 1 -- 172.21.15.38:0/4149305978 learned_addr learned my addr 172.21.15.38:0/4149305978 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f165a640 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55782911d6a0 0x55782912f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33ebfff640 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 0x55782912f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f0e59640 1 -- 172.21.15.38:0/4149305978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55782911d6a0 msgr2=0x55782912f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f0e59640 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55782911d6a0 0x55782912f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f0e59640 1 -- 172.21.15.38:0/4149305978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 msgr2=0x55782912f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f0e59640 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 0x55782912f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f0e59640 1 -- 172.21.15.38:0/4149305978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557828eaeea0 con 0x557829131f80 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33ebfff640 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 0x55782912f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f0e59640 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 0x55782911d160 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f33dc005f20 tx=0x7f33dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc164b5f0729b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33cb7fe640 1 -- 172.21.15.38:0/4149305978 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc027060 con 0x557829131f80 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33cb7fe640 1 -- 172.21.15.38:0/4149305978 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f33dc010040 con 0x557829131f80 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f5dc2740 1 -- 172.21.15.38:0/4149305978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33e4015600 con 0x557829131f80 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33cb7fe640 1 -- 172.21.15.38:0/4149305978 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33dc007560 con 0x557829131f80 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f5dc2740 10 client.6621 fetch_fsmap learned FSMap version 27 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.727+0000 7f33f5dc2740 1 -- 172.21.15.38:0/4149305978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x557828d42000 con 0x557829131f80 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.728+0000 7f33cb7fe640 1 -- 172.21.15.38:0/4149305978 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e4015600 con 0x557829131f80 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.728+0000 7f33cb7fe640 1 -- 172.21.15.38:0/4149305978 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f33dc013420 con 0x557829131f80 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.728+0000 7f33f5dc2740 10 client.6621 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.728+0000 7f33f5dc2740 10 client.6621 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.728+0000 7f33f5dc2740 20 client.6621 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.728+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x557829117bb0 0x557829117f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.728+0000 7f33f5dc2740 4 client.6621 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.728+0000 7f33f5dc2740 1 -- 172.21.15.38:0/4149305978 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55782907b000 con 0x557829117bb0 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.728+0000 7f33ebfff640 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x557829117bb0 0x557829117f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.728+0000 7f33cb7fe640 10 client.6621 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.728+0000 7f33ebfff640 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x557829117bb0 0x557829117f90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf193f9a89187cf server_cookie=9e90b9ea64fb2b7a in_seq=0 out_seq=0 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33cb7fe640 1 -- 172.21.15.38:0/4149305978 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55782907b000 con 0x557829117bb0 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33cb7fe640 10 client.6621 handle_command_reply: tid=0 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33f5dc2740 1 client.6621 shutdown 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33f5dc2740 1 -- 172.21.15.38:0/4149305978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 msgr2=0x55782911d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 0x55782911d160 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f33dc005f20 tx=0x7f33dc0057f0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33f5dc2740 1 -- 172.21.15.38:0/4149305978 shutdown_connections 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x557829117bb0 0x557829117f90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55782911d6a0 0x55782912f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 0x55782912f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33cb7fe640 0 client.6621 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/4149305978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 0x55782911d160 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33f5dc2740 1 -- 172.21.15.38:0/4149305978 >> 172.21.15.38:0/4149305978 conn(0x557829104be0 msgr2=0x557829109850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33f5dc2740 1 -- 172.21.15.38:0/4149305978 shutdown_connections 2024-04-25T08:35:22.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33f5dc2740 1 -- 172.21.15.38:0/4149305978 wait complete. 2024-04-25T08:35:22.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.729+0000 7f33f5dc2740 20 client.6621 trim_cache size 0 max 16384 2024-04-25T08:35:22.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.741+0000 7f33f5dc2740 1 Processor -- start 2024-04-25T08:35:22.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.741+0000 7f33f5dc2740 1 -- start start 2024-04-25T08:35:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33f5dc2740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55782911d6a0 0x557829138360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33f5dc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 0x5578291388a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33f5dc2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 0x55782912eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33f5dc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec1fd370 con 0x557829131f80 2024-04-25T08:35:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33f5dc2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec0afdb0 con 0x557829132360 2024-04-25T08:35:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33f5dc2740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec1fce20 con 0x55782911d6a0 2024-04-25T08:35:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 0x5578291388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33ebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 0x5578291388a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56004/0 (socket says 172.21.15.38:56004) 2024-04-25T08:35:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33ebfff640 1 -- 172.21.15.38:0/698369395 learned_addr learned my addr 172.21.15.38:0/698369395 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33ebfff640 1 -- 172.21.15.38:0/698369395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55782911d6a0 msgr2=0x557829138360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33ebfff640 1 --2- 172.21.15.38:0/698369395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55782911d6a0 0x557829138360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33ebfff640 1 -- 172.21.15.38:0/698369395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 msgr2=0x55782912eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33ebfff640 1 --2- 172.21.15.38:0/698369395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 0x55782912eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33ebfff640 1 -- 172.21.15.38:0/698369395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557829136150 con 0x557829131f80 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33ebfff640 1 --2- 172.21.15.38:0/698369395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 0x5578291388a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f33e0079230 tx=0x7f33e0049040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b27b37baa6b51a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33d1ffb640 1 -- 172.21.15.38:0/698369395 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e007d3f0 con 0x557829131f80 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33d1ffb640 1 -- 172.21.15.38:0/698369395 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0081a40 con 0x557829131f80 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.742+0000 7f33d1ffb640 1 -- 172.21.15.38:0/698369395 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0077340 con 0x557829131f80 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.743+0000 7f33f5dc2740 1 -- 172.21.15.38:0/698369395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 msgr2=0x5578291388a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.743+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/698369395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 0x5578291388a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f33e0079230 tx=0x7f33e0049040 comp rx=0 tx=0).stop 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.743+0000 7f33f5dc2740 1 -- 172.21.15.38:0/698369395 shutdown_connections 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.743+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/698369395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829132360 0x55782912eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.743+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/698369395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557829131f80 0x5578291388a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.743+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/698369395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55782911d6a0 0x557829138360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.743+0000 7f33f5dc2740 1 -- 172.21.15.38:0/698369395 >> 172.21.15.38:0/698369395 conn(0x557829104be0 msgr2=0x557829105660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.743+0000 7f33f5dc2740 1 -- 172.21.15.38:0/698369395 shutdown_connections 2024-04-25T08:35:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.743+0000 7f33f5dc2740 1 -- 172.21.15.38:0/698369395 wait complete. 2024-04-25T08:35:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.743+0000 7f33f5dc2740 1 -- start start 2024-04-25T08:35:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.744+0000 7f33f5dc2740 1 Processor -- start 2024-04-25T08:35:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.744+0000 7f33f5dc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55782911d6a0 0x5578291523e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.744+0000 7f33f5dc2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829131f80 0x5578291be140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.744+0000 7f33f5dc2740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557829132360 0x5578291040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.744+0000 7f33f5dc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782912fb20 con 0x55782911d6a0 2024-04-25T08:35:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.744+0000 7f33f5dc2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55782911dd90 con 0x557829131f80 2024-04-25T08:35:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.744+0000 7f33f5dc2740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55782912c110 con 0x557829132360 2024-04-25T08:35:22.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.744+0000 7f33f5dc2740 10 client.0 _ll_register_callbacks cb 0x7f33e4009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.744+0000 7f33f0e59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55782911d6a0 0x5578291523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.744+0000 7f33f0e59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55782911d6a0 0x5578291523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56016/0 (socket says 172.21.15.38:56016) 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.744+0000 7f33f0e59640 1 -- 172.21.15.38:0/2451676687 learned_addr learned my addr 172.21.15.38:0/2451676687 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f0e59640 1 -- 172.21.15.38:0/2451676687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557829132360 msgr2=0x5578291040c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f165a640 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557829132360 0x5578291040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f0e59640 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557829132360 0x5578291040c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33ebfff640 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829131f80 0x5578291be140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f0e59640 1 -- 172.21.15.38:0/2451676687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829131f80 msgr2=0x5578291be140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f0e59640 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829131f80 0x5578291be140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f0e59640 1 -- 172.21.15.38:0/2451676687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55782912f200 con 0x55782911d6a0 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f0e59640 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55782911d6a0 0x5578291523e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f33dc001eb0 tx=0x7f33dc007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743f8d4d1faefebe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33cb7fe640 1 -- 172.21.15.38:0/2451676687 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc007700 con 0x55782911d6a0 2024-04-25T08:35:22.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33cb7fe640 1 -- 172.21.15.38:0/2451676687 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f33dc010070 con 0x55782911d6a0 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2451676687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33e0081a40 con 0x55782911d6a0 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33cb7fe640 1 -- 172.21.15.38:0/2451676687 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33dc03d050 con 0x55782911d6a0 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f5dc2740 10 client.6627 fetch_fsmap learned FSMap version 27 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2451676687 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x557829117f90 con 0x55782911d6a0 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33cb7fe640 1 -- 172.21.15.38:0/2451676687 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0081a40 con 0x55782911d6a0 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33cb7fe640 1 -- 172.21.15.38:0/2451676687 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 27) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f33dc040020 con 0x55782911d6a0 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f5dc2740 10 client.6627 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f5dc2740 10 client.6627 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f5dc2740 20 client.6627 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x557829138a50 0x5578291ca030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f5dc2740 4 client.6627 mds_command: new command op to 5964 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3876340424}] 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.745+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2451676687 --> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3876340424}) v1 -- 0x557829075350 con 0x557829138a50 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33ebfff640 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x557829138a50 0x5578291ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33cb7fe640 10 client.6627 ms_handle_connect on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33ebfff640 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x557829138a50 0x5578291ca030 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8263371275de727d server_cookie=c68b973452bcad2d in_seq=0 out_seq=0 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33cb7fe640 1 -- 172.21.15.38:0/2451676687 <== mds.0 v2:172.21.15.38:6832/226686241 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x557829075350 con 0x557829138a50 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33cb7fe640 10 client.6627 handle_command_reply: tid=0 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33f5dc2740 1 client.6627 shutdown 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2451676687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55782911d6a0 msgr2=0x5578291523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55782911d6a0 0x5578291523e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f33dc001eb0 tx=0x7f33dc007000 comp rx=0 tx=0).stop 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2451676687 shutdown_connections 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.38:6832/226686241,v1:172.21.15.38:6834/226686241] conn(0x557829138a50 0x5578291ca030 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557829132360 0x5578291040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557829131f80 0x5578291be140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33cb7fe640 0 client.6627 ms_handle_reset on v2:172.21.15.38:6832/226686241 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33f5dc2740 1 --2- 172.21.15.38:0/2451676687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55782911d6a0 0x5578291523e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2451676687 >> 172.21.15.38:0/2451676687 conn(0x557829104be0 msgr2=0x557829150a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2451676687 shutdown_connections 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33f5dc2740 1 -- 172.21.15.38:0/2451676687 wait complete. 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.746+0000 7f33f5dc2740 20 client.6627 trim_cache size 0 max 16384 2024-04-25T08:35:22.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.747+0000 7f33f30bd640 1 -- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33d806b4d0 msgr2=0x7f33d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.747+0000 7f33f30bd640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33d806b4d0 0x7f33d806d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f33dc000f60 tx=0x7f33dc009040 comp rx=0 tx=0).stop 2024-04-25T08:35:22.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.747+0000 7f33f30bd640 1 -- 172.21.15.38:0/2537283563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ec0cfd40 msgr2=0x7f33ec071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.747+0000 7f33f30bd640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ec0cfd40 0x7f33ec071e60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000c00 tx=0x7f33e00046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.747+0000 7f33f30bd640 1 -- 172.21.15.38:0/2537283563 shutdown_connections 2024-04-25T08:35:22.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.747+0000 7f33f30bd640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33d806b4d0 0x7f33d806d990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.747+0000 7f33f30bd640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33ec15db80 0x7f33ec0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.747+0000 7f33f30bd640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33ec0cfd40 0x7f33ec071e60 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.747+0000 7f33f30bd640 1 --2- 172.21.15.38:0/2537283563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33ec0ccfc0 0x7f33ec159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.747+0000 7f33f30bd640 1 -- 172.21.15.38:0/2537283563 >> 172.21.15.38:0/2537283563 conn(0x7f33ec06adc0 msgr2=0x7f33ec15a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.747+0000 7f33f30bd640 1 -- 172.21.15.38:0/2537283563 shutdown_connections 2024-04-25T08:35:22.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.747+0000 7f33f30bd640 1 -- 172.21.15.38:0/2537283563 wait complete. 2024-04-25T08:35:22.752 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T08:35:22.752 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:22.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.858+0000 7f19f2d30640 1 Processor -- start 2024-04-25T08:35:22.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.858+0000 7f19f2d30640 1 -- start start 2024-04-25T08:35:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.859+0000 7f19f2d30640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f19ec0d3900 0x7f19ec0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.859+0000 7f19f2d30640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f19ec0d0d90 0x7f19ec0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.859+0000 7f19f2d30640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec0d3e70 con 0x7f19ec15dc10 2024-04-25T08:35:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.859+0000 7f19f2d30640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec15dff0 con 0x7f19ec0d0d90 2024-04-25T08:35:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.859+0000 7f19f2d30640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec0b6990 con 0x7f19ec0d3900 2024-04-25T08:35:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.859+0000 7f19f12cd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f19ec0d0d90 0x7f19ec0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.859+0000 7f19f0acc640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f19ec15dc10 0x7f19ec0ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53302/0 (socket says 172.21.15.38:53302) 2024-04-25T08:35:22.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.859+0000 7f19f12cd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f19ec0d0d90 0x7f19ec0b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56026/0 (socket says 172.21.15.38:56026) 2024-04-25T08:35:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.859+0000 7f19f0acc640 1 -- 172.21.15.38:0/141300319 learned_addr learned my addr 172.21.15.38:0/141300319 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.859+0000 7f19ebfff640 1 --2- 172.21.15.38:0/141300319 >> v2:172.21.15.107:3300/0 conn(0x7f19ec0d3900 0x7f19ec0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.860+0000 7f19f12cd640 1 -- 172.21.15.38:0/141300319 >> v2:172.21.15.107:3300/0 conn(0x7f19ec0d3900 msgr2=0x7f19ec0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.860+0000 7f19f12cd640 1 --2- 172.21.15.38:0/141300319 >> v2:172.21.15.107:3300/0 conn(0x7f19ec0d3900 0x7f19ec0d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.860+0000 7f19f12cd640 1 -- 172.21.15.38:0/141300319 >> v1:172.21.15.38:6789/0 conn(0x7f19ec15dc10 legacy=0x7f19ec0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.860+0000 7f19eb7fe640 1 -- 172.21.15.38:0/141300319 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225774715 0 0) 0x7f19ec0d3e70 con 0x7f19ec15dc10 2024-04-25T08:35:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.860+0000 7f19f12cd640 1 -- 172.21.15.38:0/141300319 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0d4150 con 0x7f19ec0d0d90 2024-04-25T08:35:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.860+0000 7f19f12cd640 1 --2- 172.21.15.38:0/141300319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19ec0d0d90 0x7f19ec0b2370 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0b9600 tx=0x7f19dc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bac55b5cfdf589 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.860+0000 7f19eb7fe640 1 -- 172.21.15.38:0/141300319 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc037070 con 0x7f19ec0d0d90 2024-04-25T08:35:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.860+0000 7f19eb7fe640 1 -- 172.21.15.38:0/141300319 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc032440 con 0x7f19ec0d0d90 2024-04-25T08:35:22.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.860+0000 7f19eb7fe640 1 -- 172.21.15.38:0/141300319 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc02ae60 con 0x7f19ec0d0d90 2024-04-25T08:35:22.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.861+0000 7f19f2d30640 1 -- 172.21.15.38:0/141300319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19ec0d0d90 msgr2=0x7f19ec0b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.861+0000 7f19f2d30640 1 --2- 172.21.15.38:0/141300319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19ec0d0d90 0x7f19ec0b2370 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0b9600 tx=0x7f19dc02a110 comp rx=0 tx=0).stop 2024-04-25T08:35:22.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.861+0000 7f19f2d30640 1 -- 172.21.15.38:0/141300319 shutdown_connections 2024-04-25T08:35:22.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.861+0000 7f19f2d30640 1 --2- 172.21.15.38:0/141300319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19ec0d0d90 0x7f19ec0b2370 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.861+0000 7f19f2d30640 1 --2- 172.21.15.38:0/141300319 >> v2:172.21.15.107:3300/0 conn(0x7f19ec0d3900 0x7f19ec0d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.861+0000 7f19f2d30640 1 -- 172.21.15.38:0/141300319 >> 172.21.15.38:0/141300319 conn(0x7f19ec066d10 msgr2=0x7f19ec067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:22.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.861+0000 7f19f2d30640 1 -- 172.21.15.38:0/141300319 shutdown_connections 2024-04-25T08:35:22.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.861+0000 7f19f2d30640 1 -- 172.21.15.38:0/141300319 wait complete. 2024-04-25T08:35:22.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.862+0000 7f19f2d30640 1 Processor -- start 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.862+0000 7f19f2d30640 1 -- start start 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.863+0000 7f19f2d30640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19ec0d0d90 0x7f19ec15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.863+0000 7f19f2d30640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19ec0d3900 0x7f19ec0cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.863+0000 7f19f2d30640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19ec15dc10 0x7f19ec0c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.863+0000 7f19f2d30640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0d4710 con 0x7f19ec0d3900 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.863+0000 7f19f2d30640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0d11e0 con 0x7f19ec0d0d90 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.863+0000 7f19f2d30640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0b6bc0 con 0x7f19ec15dc10 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.863+0000 7f19ebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19ec0d3900 0x7f19ec0cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.863+0000 7f19ebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19ec0d3900 0x7f19ec0cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56030/0 (socket says 172.21.15.38:56030) 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.863+0000 7f19ebfff640 1 -- 172.21.15.38:0/2762156540 learned_addr learned my addr 172.21.15.38:0/2762156540 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.863+0000 7f19f0acc640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19ec0d0d90 0x7f19ec15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.863+0000 7f19f12cd640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19ec15dc10 0x7f19ec0c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.864+0000 7f19ebfff640 1 -- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19ec15dc10 msgr2=0x7f19ec0c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.864+0000 7f19ebfff640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19ec15dc10 0x7f19ec0c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.864+0000 7f19ebfff640 1 -- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19ec0d0d90 msgr2=0x7f19ec15b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:22.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.864+0000 7f19ebfff640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19ec0d0d90 0x7f19ec15b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:22.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.864+0000 7f19ebfff640 1 -- 172.21.15.38:0/2762156540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec15b5e0 con 0x7f19ec0d3900 2024-04-25T08:35:22.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.864+0000 7f19f12cd640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19ec15dc10 0x7f19ec0c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:22.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.864+0000 7f19ebfff640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19ec0d3900 0x7f19ec0cb3e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f19d400d830 tx=0x7f19d400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.864+0000 7f19e97fa640 1 -- 172.21.15.38:0/2762156540 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d400b6c0 con 0x7f19ec0d3900 2024-04-25T08:35:22.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.864+0000 7f19e97fa640 1 -- 172.21.15.38:0/2762156540 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4019070 con 0x7f19ec0d3900 2024-04-25T08:35:22.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.864+0000 7f19f2d30640 1 -- 172.21.15.38:0/2762156540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec0cb920 con 0x7f19ec0d3900 2024-04-25T08:35:22.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.864+0000 7f19e97fa640 1 -- 172.21.15.38:0/2762156540 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d400ba80 con 0x7f19ec0d3900 2024-04-25T08:35:22.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.864+0000 7f19f2d30640 1 -- 172.21.15.38:0/2762156540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec0c7b80 con 0x7f19ec0d3900 2024-04-25T08:35:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.865+0000 7f19f2d30640 1 -- 172.21.15.38:0/2762156540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec068e80 con 0x7f19ec0d3900 2024-04-25T08:35:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.865+0000 7f19e97fa640 1 -- 172.21.15.38:0/2762156540 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19d400bc20 con 0x7f19ec0d3900 2024-04-25T08:35:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.866+0000 7f19e97fa640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f19cc06b4d0 0x7f19cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.866+0000 7f19e97fa640 1 -- 172.21.15.38:0/2762156540 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f19d408a8a0 con 0x7f19ec0d3900 2024-04-25T08:35:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.866+0000 7f19f0acc640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f19cc06b4d0 0x7f19cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:22.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.866+0000 7f19f0acc640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f19cc06b4d0 0x7f19cc06d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f19e4007ed0 tx=0x7f19e4007e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:22.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:22.870+0000 7f19e97fa640 1 -- 172.21.15.38:0/2762156540 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19d401c360 con 0x7f19ec0d3900 2024-04-25T08:35:23.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.086+0000 7f19f2d30640 1 -- 172.21.15.38:0/2762156540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19ec0b8c00 con 0x7f19ec0d3900 2024-04-25T08:35:23.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.087+0000 7f19e97fa640 1 -- 172.21.15.38:0/2762156540 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f19d4059560 con 0x7f19ec0d3900 2024-04-25T08:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":27,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:35:20.877703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:23.082 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 27 2024-04-25T08:35:23.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.089+0000 7f19f2d30640 1 -- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f19cc06b4d0 msgr2=0x7f19cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.089+0000 7f19f2d30640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f19cc06b4d0 0x7f19cc06d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f19e4007ed0 tx=0x7f19e4007e60 comp rx=0 tx=0).stop 2024-04-25T08:35:23.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.089+0000 7f19f2d30640 1 -- 172.21.15.38:0/2762156540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19ec0d3900 msgr2=0x7f19ec0cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.089+0000 7f19f2d30640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19ec0d3900 0x7f19ec0cb3e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f19d400d830 tx=0x7f19d400dcf0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.089+0000 7f19f2d30640 1 -- 172.21.15.38:0/2762156540 shutdown_connections 2024-04-25T08:35:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.089+0000 7f19f2d30640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f19cc06b4d0 0x7f19cc06d990 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.089+0000 7f19f2d30640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f19ec15dc10 0x7f19ec0c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.089+0000 7f19f2d30640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19ec0d3900 0x7f19ec0cb3e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.089+0000 7f19f2d30640 1 --2- 172.21.15.38:0/2762156540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f19ec0d0d90 0x7f19ec15b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.089+0000 7f19f2d30640 1 -- 172.21.15.38:0/2762156540 >> 172.21.15.38:0/2762156540 conn(0x7f19ec066d10 msgr2=0x7f19ec15ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.089+0000 7f19f2d30640 1 -- 172.21.15.38:0/2762156540 shutdown_connections 2024-04-25T08:35:23.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.089+0000 7f19f2d30640 1 -- 172.21.15.38:0/2762156540 wait complete. 2024-04-25T08:35:23.095 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok flush_path / 2024-04-25T08:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:23.212 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:35:23.212 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:35:23.212 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:35:23.253 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:35:23.291 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:35:23.291 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:35:23.291 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:23.309 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:23.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.454+0000 7fb0aceb5640 1 Processor -- start 2024-04-25T08:35:23.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.454+0000 7fb0aceb5640 1 -- start start 2024-04-25T08:35:23.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.455+0000 7fb0aceb5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a815dc10 0x7fb0a80ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.455+0000 7fb0aceb5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a80d3e70 con 0x7fb0a80d0f60 2024-04-25T08:35:23.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.455+0000 7fb0aceb5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a80d4150 con 0x7fb0a80ced60 2024-04-25T08:35:23.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.455+0000 7fb0aceb5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a815dff0 con 0x7fb0a815dc10 2024-04-25T08:35:23.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.455+0000 7fb0a6d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb0a80d0f60 0x7fb0a80b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53316/0 (socket says 172.21.15.38:53316) 2024-04-25T08:35:23.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.455+0000 7fb0a6d9d640 1 -- 172.21.15.38:0/1404707913 learned_addr learned my addr 172.21.15.38:0/1404707913 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:23.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a659c640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a815dc10 0x7fb0a80ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305010088 0 0) 0x7fb0a80d3e70 con 0x7fb0a80d0f60 2024-04-25T08:35:23.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb094003520 con 0x7fb0a80d0f60 2024-04-25T08:35:23.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 759836079 0 0) 0x7fb0a80d4150 con 0x7fb0a80ced60 2024-04-25T08:35:23.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0a80d3e70 con 0x7fb0a80ced60 2024-04-25T08:35:23.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234947661 0 0) 0x7fb094003520 con 0x7fb0a80d0f60 2024-04-25T08:35:23.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a815dc10 msgr2=0x7fb0a80ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:23.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a815dc10 0x7fb0a80ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 >> v1:172.21.15.107:6789/0 conn(0x7fb0a80ced60 legacy=0x7fb0a80d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a80d4430 con 0x7fb0a80d0f60 2024-04-25T08:35:23.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a659c640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a815dc10 0x7fb0a80ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fb0980019f0 con 0x7fb0a80d0f60 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 >> v1:172.21.15.38:6789/0 conn(0x7fb0a80d0f60 legacy=0x7fb0a80b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a815dc10 0x7fb094003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0940040b0 0x7fb094004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb094004d40 0x7fb0940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a815dff0 con 0x7fb0940040b0 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09400bb00 con 0x7fb094004d40 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09400bd00 con 0x7fb0a815dc10 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a5d9b640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0940040b0 0x7fb094004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a659c640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a815dc10 0x7fb094003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.456+0000 7fb0a6d9d640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb094004d40 0x7fb0940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.457+0000 7fb0a5d9b640 1 -- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a815dc10 msgr2=0x7fb094003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:23.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.457+0000 7fb0a5d9b640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a815dc10 0x7fb094003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.457+0000 7fb0a5d9b640 1 -- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb094004d40 msgr2=0x7fb0940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.457+0000 7fb0a5d9b640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb094004d40 0x7fb0940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.457+0000 7fb0a5d9b640 1 -- 172.21.15.38:0/1404707913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb094003520 con 0x7fb0940040b0 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.457+0000 7fb0a659c640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a815dc10 0x7fb094003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.457+0000 7fb0a5d9b640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0940040b0 0x7fb094004800 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb0900018d0 tx=0x7fb094005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2807005f2b1c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.457+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0900014f0 con 0x7fb0940040b0 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.457+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0900052d0 con 0x7fb0940040b0 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.458+0000 7fb0a559a640 1 -- 172.21.15.38:0/1404707913 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09002aca0 con 0x7fb0940040b0 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.458+0000 7fb0aceb5640 1 -- 172.21.15.38:0/1404707913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0940040b0 msgr2=0x7fb094004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.458+0000 7fb0aceb5640 1 --2- 172.21.15.38:0/1404707913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0940040b0 0x7fb094004800 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb0900018d0 tx=0x7fb094005830 comp rx=0 tx=0).stop 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.458+0000 7fb0a6d9d640 1 -- 172.21.15.38:0/1404707913 reap_dead start 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.458+0000 7fb0aceb5640 1 -- 172.21.15.38:0/1404707913 shutdown_connections 2024-04-25T08:35:23.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.458+0000 7fb0aceb5640 1 -- 172.21.15.38:0/1404707913 >> 172.21.15.38:0/1404707913 conn(0x7fb0a8066d10 msgr2=0x7fb0a8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:23.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.458+0000 7fb0aceb5640 1 -- 172.21.15.38:0/1404707913 shutdown_connections 2024-04-25T08:35:23.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.458+0000 7fb0aceb5640 1 -- 172.21.15.38:0/1404707913 wait complete. 2024-04-25T08:35:23.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.459+0000 7fb0aceb5640 1 Processor -- start 2024-04-25T08:35:23.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.459+0000 7fb0aceb5640 1 -- start start 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.460+0000 7fb0aceb5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0940040b0 0x7fb0a8153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.460+0000 7fb0aceb5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a8156f10 0x7fb0a81f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.460+0000 7fb0aceb5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0a81f7720 0x7fb0a80cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.460+0000 7fb0aceb5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a80d4710 con 0x7fb0940040b0 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.460+0000 7fb0aceb5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a80b80f0 con 0x7fb0a81f7720 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.460+0000 7fb0aceb5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a80b8270 con 0x7fb0a8156f10 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.460+0000 7fb0a659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0940040b0 0x7fb0a8153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.460+0000 7fb0a659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0940040b0 0x7fb0a8153020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56040/0 (socket says 172.21.15.38:56040) 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.460+0000 7fb0a659c640 1 -- 172.21.15.38:0/3561914683 learned_addr learned my addr 172.21.15.38:0/3561914683 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.460+0000 7fb0a6d9d640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0a81f7720 0x7fb0a80cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.460+0000 7fb0a5d9b640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a8156f10 0x7fb0a81f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb0a659c640 1 -- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a8156f10 msgr2=0x7fb0a81f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb0a659c640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a8156f10 0x7fb0a81f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb0a659c640 1 -- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0a81f7720 msgr2=0x7fb0a80cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb0a659c640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0a81f7720 0x7fb0a80cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb0a659c640 1 -- 172.21.15.38:0/3561914683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a815a280 con 0x7fb0940040b0 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb0a6d9d640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0a81f7720 0x7fb0a80cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb0a5d9b640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a8156f10 0x7fb0a81f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb0a659c640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0940040b0 0x7fb0a8153020 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0011d00 tx=0x7fb0a000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb09effd640 1 -- 172.21.15.38:0/3561914683 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a000fd20 con 0x7fb0940040b0 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb09effd640 1 -- 172.21.15.38:0/3561914683 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a0014440 con 0x7fb0940040b0 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb0aceb5640 1 -- 172.21.15.38:0/3561914683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a81f6d00 con 0x7fb0940040b0 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb09effd640 1 -- 172.21.15.38:0/3561914683 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a00147f0 con 0x7fb0940040b0 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.461+0000 7fb0aceb5640 1 -- 172.21.15.38:0/3561914683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a8159cc0 con 0x7fb0940040b0 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.462+0000 7fb0aceb5640 1 -- 172.21.15.38:0/3561914683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a81f7b00 con 0x7fb0940040b0 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.462+0000 7fb09effd640 1 -- 172.21.15.38:0/3561914683 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0a0012040 con 0x7fb0940040b0 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.463+0000 7fb09effd640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb08806b4d0 0x7fb08806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.463+0000 7fb09effd640 1 -- 172.21.15.38:0/3561914683 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb0a008ab00 con 0x7fb0940040b0 2024-04-25T08:35:23.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.463+0000 7fb0a5d9b640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb08806b4d0 0x7fb08806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.464+0000 7fb0a5d9b640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb08806b4d0 0x7fb08806d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb090032000 tx=0x7fb090009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:23.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.467+0000 7fb09effd640 1 -- 172.21.15.38:0/3561914683 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0a0019030 con 0x7fb0940040b0 2024-04-25T08:35:23.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.647+0000 7fb0aceb5640 1 -- 172.21.15.38:0/3561914683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0a8068e80 con 0x7fb0940040b0 2024-04-25T08:35:23.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.648+0000 7fb09effd640 1 -- 172.21.15.38:0/3561914683 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+17026 (secure 0 0 0) 0x7fb0a0059850 con 0x7fb0940040b0 2024-04-25T08:35:23.643 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:23.643 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":56,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:18.124795+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:34:42.507595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:34:43.209205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.28","mappings":[{"from":0,"to":6}]},{"pgid":"8.2f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:23.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.650+0000 7fb0aceb5640 1 -- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb08806b4d0 msgr2=0x7fb08806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.650+0000 7fb0aceb5640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb08806b4d0 0x7fb08806d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb090032000 tx=0x7fb090009030 comp rx=0 tx=0).stop 2024-04-25T08:35:23.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.650+0000 7fb0aceb5640 1 -- 172.21.15.38:0/3561914683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0940040b0 msgr2=0x7fb0a8153020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.650+0000 7fb0aceb5640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0940040b0 0x7fb0a8153020 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb0a0011d00 tx=0x7fb0a000f610 comp rx=0 tx=0).stop 2024-04-25T08:35:23.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.650+0000 7fb0aceb5640 1 -- 172.21.15.38:0/3561914683 shutdown_connections 2024-04-25T08:35:23.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.650+0000 7fb0aceb5640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0a81f7720 0x7fb0a80cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.650+0000 7fb0aceb5640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb08806b4d0 0x7fb08806d990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.650+0000 7fb0aceb5640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0a8156f10 0x7fb0a81f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.650+0000 7fb0aceb5640 1 --2- 172.21.15.38:0/3561914683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0940040b0 0x7fb0a8153020 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.650+0000 7fb0aceb5640 1 -- 172.21.15.38:0/3561914683 >> 172.21.15.38:0/3561914683 conn(0x7fb0a8066d10 msgr2=0x7fb0a80d0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:23.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.650+0000 7fb0aceb5640 1 -- 172.21.15.38:0/3561914683 shutdown_connections 2024-04-25T08:35:23.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.650+0000 7fb0aceb5640 1 -- 172.21.15.38:0/3561914683 wait complete. 2024-04-25T08:35:23.657 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:34:42.507595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:34:43.209205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:35:23.657 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:35:23.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.766+0000 7f42d646b640 1 Processor -- start 2024-04-25T08:35:23.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.766+0000 7f42d646b640 1 -- start start 2024-04-25T08:35:23.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.766+0000 7f42d646b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f42d015db80 0x7f42d00cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.766+0000 7f42d646b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42d00cfb70 0x7f42d00ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.766+0000 7f42d646b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d00ac430 con 0x7f42d00ccfc0 2024-04-25T08:35:23.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.766+0000 7f42d646b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d00ac1e0 con 0x7f42d015db80 2024-04-25T08:35:23.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.766+0000 7f42d646b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d00ac710 con 0x7f42d00cfb70 2024-04-25T08:35:23.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.767+0000 7f42cf7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f42d00ccfc0 0x7f42d00c9ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53332/0 (socket says 172.21.15.38:53332) 2024-04-25T08:35:23.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.767+0000 7f42cf7fe640 1 -- 172.21.15.38:0/94705519 learned_addr learned my addr 172.21.15.38:0/94705519 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:23.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.767+0000 7f42cffff640 1 --2- 172.21.15.38:0/94705519 >> v2:172.21.15.107:3300/0 conn(0x7f42d015db80 0x7f42d00cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.767+0000 7f42d4a08640 1 --2- 172.21.15.38:0/94705519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42d00cfb70 0x7f42d00ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.767+0000 7f42ceffd640 1 -- 172.21.15.38:0/94705519 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225535505 0 0) 0x7f42d00ac430 con 0x7f42d00ccfc0 2024-04-25T08:35:23.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.767+0000 7f42ceffd640 1 -- 172.21.15.38:0/94705519 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42b8003520 con 0x7f42d00ccfc0 2024-04-25T08:35:23.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.767+0000 7f42cffff640 1 -- 172.21.15.38:0/94705519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42d00cfb70 msgr2=0x7f42d00ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:23.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.767+0000 7f42cffff640 1 --2- 172.21.15.38:0/94705519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42d00cfb70 0x7f42d00ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.767+0000 7f42cffff640 1 -- 172.21.15.38:0/94705519 >> v1:172.21.15.38:6789/0 conn(0x7f42d00ccfc0 legacy=0x7f42d00c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.767+0000 7f42cffff640 1 -- 172.21.15.38:0/94705519 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d00ac9f0 con 0x7f42d015db80 2024-04-25T08:35:23.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.767+0000 7f42d4a08640 1 --2- 172.21.15.38:0/94705519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42d00cfb70 0x7f42d00ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.768+0000 7f42cffff640 1 --2- 172.21.15.38:0/94705519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d015db80 0x7f42d00cf630 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f42c00097a0 tx=0x7f42c002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f910b11d897cb3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.768+0000 7f42ceffd640 1 -- 172.21.15.38:0/94705519 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0037070 con 0x7f42d015db80 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.768+0000 7f42ceffd640 1 -- 172.21.15.38:0/94705519 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c0032440 con 0x7f42d015db80 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.768+0000 7f42ceffd640 1 -- 172.21.15.38:0/94705519 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c00327f0 con 0x7f42d015db80 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.769+0000 7f42d646b640 1 -- 172.21.15.38:0/94705519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d015db80 msgr2=0x7f42d00cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.769+0000 7f42d646b640 1 --2- 172.21.15.38:0/94705519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d015db80 0x7f42d00cf630 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f42c00097a0 tx=0x7f42c002a180 comp rx=0 tx=0).stop 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.769+0000 7f42d646b640 1 -- 172.21.15.38:0/94705519 shutdown_connections 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.769+0000 7f42d646b640 1 --2- 172.21.15.38:0/94705519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42d00cfb70 0x7f42d00ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.769+0000 7f42d646b640 1 --2- 172.21.15.38:0/94705519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d015db80 0x7f42d00cf630 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.769+0000 7f42d646b640 1 -- 172.21.15.38:0/94705519 >> 172.21.15.38:0/94705519 conn(0x7f42d006adc0 msgr2=0x7f42d006b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.769+0000 7f42d646b640 1 -- 172.21.15.38:0/94705519 shutdown_connections 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.769+0000 7f42d646b640 1 -- 172.21.15.38:0/94705519 wait complete. 2024-04-25T08:35:23.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.770+0000 7f42d646b640 1 Processor -- start 2024-04-25T08:35:23.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.770+0000 7f42d646b640 1 -- start start 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d646b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42d00ccfc0 0x7f42d00cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d646b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d00cfb70 0x7f42d0071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d646b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d015db80 0x7f42d00c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d646b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d015e2c0 con 0x7f42d015db80 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d646b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d015df60 con 0x7f42d00cfb70 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d646b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d015e440 con 0x7f42d00ccfc0 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d4a08640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d015db80 0x7f42d00c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d4a08640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d015db80 0x7f42d00c4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56046/0 (socket says 172.21.15.38:56046) 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d4a08640 1 -- 172.21.15.38:0/1329653874 learned_addr learned my addr 172.21.15.38:0/1329653874 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42cf7fe640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d00cfb70 0x7f42d0071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42cffff640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42d00ccfc0 0x7f42d00cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d4a08640 1 -- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42d00ccfc0 msgr2=0x7f42d00cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d4a08640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42d00ccfc0 0x7f42d00cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d4a08640 1 -- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d00cfb70 msgr2=0x7f42d0071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d4a08640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d00cfb70 0x7f42d0071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d4a08640 1 -- 172.21.15.38:0/1329653874 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d01f1730 con 0x7f42d015db80 2024-04-25T08:35:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.771+0000 7f42d4a08640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d015db80 0x7f42d00c4f00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009f00 tx=0x7f42c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.772+0000 7f42ccff9640 1 -- 172.21.15.38:0/1329653874 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8018030 con 0x7f42d015db80 2024-04-25T08:35:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.772+0000 7f42ccff9640 1 -- 172.21.15.38:0/1329653874 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c800ce50 con 0x7f42d015db80 2024-04-25T08:35:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.772+0000 7f42d646b640 1 -- 172.21.15.38:0/1329653874 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d00cc450 con 0x7f42d015db80 2024-04-25T08:35:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.772+0000 7f42ccff9640 1 -- 172.21.15.38:0/1329653874 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8003bb0 con 0x7f42d015db80 2024-04-25T08:35:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.772+0000 7f42d646b640 1 -- 172.21.15.38:0/1329653874 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d00cca10 con 0x7f42d015db80 2024-04-25T08:35:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.773+0000 7f42d646b640 1 -- 172.21.15.38:0/1329653874 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d0068e80 con 0x7f42d015db80 2024-04-25T08:35:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.773+0000 7f42ccff9640 1 -- 172.21.15.38:0/1329653874 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42c801b070 con 0x7f42d015db80 2024-04-25T08:35:23.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.774+0000 7f42ccff9640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42bc06b4d0 0x7f42bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:23.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.774+0000 7f42ccff9640 1 -- 172.21.15.38:0/1329653874 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f42c808adb0 con 0x7f42d015db80 2024-04-25T08:35:23.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.774+0000 7f42cffff640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42bc06b4d0 0x7f42bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:23.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.774+0000 7f42cffff640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42bc06b4d0 0x7f42bc06d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f42c0034000 tx=0x7f42c002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:23.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.778+0000 7f42ccff9640 1 -- 172.21.15.38:0/1329653874 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42c8010040 con 0x7f42d015db80 2024-04-25T08:35:23.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.963+0000 7f42d646b640 1 -- 172.21.15.38:0/1329653874 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42d00b4230 con 0x7f42d015db80 2024-04-25T08:35:23.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.963+0000 7f42ccff9640 1 -- 172.21.15.38:0/1329653874 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42c8012340 con 0x7f42d015db80 2024-04-25T08:35:23.958 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:35:23.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.966+0000 7f42d646b640 1 -- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42bc06b4d0 msgr2=0x7f42bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.966+0000 7f42d646b640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42bc06b4d0 0x7f42bc06d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f42c0034000 tx=0x7f42c002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.966+0000 7f42d646b640 1 -- 172.21.15.38:0/1329653874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d015db80 msgr2=0x7f42d00c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.966+0000 7f42d646b640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d015db80 0x7f42d00c4f00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f42c8009f00 tx=0x7f42c800c490 comp rx=0 tx=0).stop 2024-04-25T08:35:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.966+0000 7f42d646b640 1 -- 172.21.15.38:0/1329653874 shutdown_connections 2024-04-25T08:35:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.966+0000 7f42d646b640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42d015db80 0x7f42d00c4f00 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.966+0000 7f42d646b640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42d00cfb70 0x7f42d0071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.966+0000 7f42d646b640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42bc06b4d0 0x7f42bc06d990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.966+0000 7f42d646b640 1 --2- 172.21.15.38:0/1329653874 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42d00ccfc0 0x7f42d00cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.966+0000 7f42d646b640 1 -- 172.21.15.38:0/1329653874 >> 172.21.15.38:0/1329653874 conn(0x7f42d006adc0 msgr2=0x7f42d006cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.966+0000 7f42d646b640 1 -- 172.21.15.38:0/1329653874 shutdown_connections 2024-04-25T08:35:23.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:23.966+0000 7f42d646b640 1 -- 172.21.15.38:0/1329653874 wait complete. 2024-04-25T08:35:23.971 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:35:24.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.079+0000 7f16800bc640 1 Processor -- start 2024-04-25T08:35:24.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.080+0000 7f16800bc640 1 -- start start 2024-04-25T08:35:24.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.080+0000 7f16800bc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16780d0320 0x7f16780d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.080+0000 7f16800bc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f16780b6980 0x7f16780b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.080+0000 7f16800bc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780d2060 con 0x7f16780d0700 2024-04-25T08:35:24.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.080+0000 7f16800bc640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16780d2330 con 0x7f16780b6980 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.080+0000 7f16800bc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16780d2610 con 0x7f16780d0320 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.081+0000 7f167e659640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f16780b6980 0x7f16780b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.081+0000 7f167e659640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f16780b6980 0x7f16780b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56060/0 (socket says 172.21.15.38:56060) 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.081+0000 7f167e659640 1 -- 172.21.15.38:0/1908799396 learned_addr learned my addr 172.21.15.38:0/1908799396 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.081+0000 7f167de58640 1 --2- 172.21.15.38:0/1908799396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16780d0320 0x7f16780d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.081+0000 7f167e659640 1 -- 172.21.15.38:0/1908799396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16780d0320 msgr2=0x7f16780d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.081+0000 7f167e659640 1 --2- 172.21.15.38:0/1908799396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16780d0320 0x7f16780d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.081+0000 7f167e659640 1 -- 172.21.15.38:0/1908799396 >> v1:172.21.15.107:6789/0 conn(0x7f16780d0700 legacy=0x7f16780d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.081+0000 7f167e659640 1 -- 172.21.15.38:0/1908799396 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780d28f0 con 0x7f16780b6980 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.081+0000 7f167de58640 1 --2- 172.21.15.38:0/1908799396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16780d0320 0x7f16780d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.082+0000 7f167e659640 1 --2- 172.21.15.38:0/1908799396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16780b6980 0x7f16780b6e10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1670009e50 tx=0x7f167002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f590ac3c03fdae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.082+0000 7f167ce56640 1 -- 172.21.15.38:0/1908799396 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670036070 con 0x7f16780b6980 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.082+0000 7f167ce56640 1 -- 172.21.15.38:0/1908799396 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1670031600 con 0x7f16780b6980 2024-04-25T08:35:24.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.082+0000 7f167ce56640 1 -- 172.21.15.38:0/1908799396 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16700319d0 con 0x7f16780b6980 2024-04-25T08:35:24.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.082+0000 7f16800bc640 1 -- 172.21.15.38:0/1908799396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16780b6980 msgr2=0x7f16780b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.082+0000 7f16800bc640 1 --2- 172.21.15.38:0/1908799396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16780b6980 0x7f16780b6e10 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1670009e50 tx=0x7f167002a750 comp rx=0 tx=0).stop 2024-04-25T08:35:24.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.082+0000 7f16800bc640 1 -- 172.21.15.38:0/1908799396 shutdown_connections 2024-04-25T08:35:24.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.082+0000 7f16800bc640 1 --2- 172.21.15.38:0/1908799396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16780b6980 0x7f16780b6e10 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.082+0000 7f16800bc640 1 --2- 172.21.15.38:0/1908799396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16780d0320 0x7f16780d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.082+0000 7f16800bc640 1 -- 172.21.15.38:0/1908799396 >> 172.21.15.38:0/1908799396 conn(0x7f16780668f0 msgr2=0x7f1678066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:24.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.082+0000 7f16800bc640 1 -- 172.21.15.38:0/1908799396 shutdown_connections 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.082+0000 7f16800bc640 1 -- 172.21.15.38:0/1908799396 wait complete. 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.084+0000 7f16800bc640 1 Processor -- start 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.084+0000 7f16800bc640 1 -- start start 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.084+0000 7f16800bc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16780b6980 0x7f16780b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.084+0000 7f16800bc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16780d0320 0x7f1678155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.084+0000 7f16800bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16780d0700 0x7f1678156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.084+0000 7f16800bc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780d0b40 con 0x7f16780d0700 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.084+0000 7f16800bc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780d2eb0 con 0x7f16780b6980 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.084+0000 7f16800bc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16780b5c80 con 0x7f16780d0320 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.084+0000 7f167e659640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16780d0700 0x7f1678156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.084+0000 7f167e659640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16780d0700 0x7f1678156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56066/0 (socket says 172.21.15.38:56066) 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.084+0000 7f167e659640 1 -- 172.21.15.38:0/137893367 learned_addr learned my addr 172.21.15.38:0/137893367 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:24.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f167d657640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16780d0320 0x7f1678155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f167de58640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16780b6980 0x7f16780b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f167e659640 1 -- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16780d0320 msgr2=0x7f1678155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f167e659640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16780d0320 0x7f1678155f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f167e659640 1 -- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16780b6980 msgr2=0x7f16780b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f167e659640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16780b6980 0x7f16780b4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f167e659640 1 -- 172.21.15.38:0/137893367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780b6110 con 0x7f16780d0700 2024-04-25T08:35:24.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f167e659640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16780d0700 0x7f1678156450 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1670009e50 tx=0x7f1670004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f166e7fc640 1 -- 172.21.15.38:0/137893367 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670036050 con 0x7f16780d0700 2024-04-25T08:35:24.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f166e7fc640 1 -- 172.21.15.38:0/137893367 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1670042070 con 0x7f16780d0700 2024-04-25T08:35:24.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f166e7fc640 1 -- 172.21.15.38:0/137893367 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670009510 con 0x7f16780d0700 2024-04-25T08:35:24.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f16800bc640 1 -- 172.21.15.38:0/137893367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16780b5070 con 0x7f16780d0700 2024-04-25T08:35:24.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.085+0000 7f16800bc640 1 -- 172.21.15.38:0/137893367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16780b5630 con 0x7f16780d0700 2024-04-25T08:35:24.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.086+0000 7f16800bc640 1 -- 172.21.15.38:0/137893367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1678068e80 con 0x7f16780d0700 2024-04-25T08:35:24.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.087+0000 7f166e7fc640 1 -- 172.21.15.38:0/137893367 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1670007bd0 con 0x7f16780d0700 2024-04-25T08:35:24.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.087+0000 7f166e7fc640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f165c06b500 0x7f165c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.087+0000 7f166e7fc640 1 -- 172.21.15.38:0/137893367 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f16700ab250 con 0x7f16780d0700 2024-04-25T08:35:24.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.088+0000 7f167de58640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f165c06b500 0x7f165c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.088+0000 7f167de58640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f165c06b500 0x7f165c06d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1674008850 tx=0x7f1674008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.091+0000 7f166e7fc640 1 -- 172.21.15.38:0/137893367 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1670079fa0 con 0x7f16780d0700 2024-04-25T08:35:24.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.276+0000 7f16800bc640 1 -- 172.21.15.38:0/137893367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f16780b3050 con 0x7f16780d0700 2024-04-25T08:35:24.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.276+0000 7f166e7fc640 1 -- 172.21.15.38:0/137893367 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f16700798a0 con 0x7f16780d0700 2024-04-25T08:35:24.271 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:35:24.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.278+0000 7f16800bc640 1 -- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f165c06b500 msgr2=0x7f165c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.278+0000 7f16800bc640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f165c06b500 0x7f165c06d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f1674008850 tx=0x7f1674008040 comp rx=0 tx=0).stop 2024-04-25T08:35:24.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.278+0000 7f16800bc640 1 -- 172.21.15.38:0/137893367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16780d0700 msgr2=0x7f1678156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.278+0000 7f16800bc640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16780d0700 0x7f1678156450 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1670009e50 tx=0x7f1670004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.278+0000 7f16800bc640 1 -- 172.21.15.38:0/137893367 shutdown_connections 2024-04-25T08:35:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.278+0000 7f16800bc640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f16780d0700 0x7f1678156450 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.278+0000 7f16800bc640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f165c06b500 0x7f165c06d9c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.278+0000 7f16800bc640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f16780d0320 0x7f1678155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.278+0000 7f16800bc640 1 --2- 172.21.15.38:0/137893367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16780b6980 0x7f16780b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.278+0000 7f16800bc640 1 -- 172.21.15.38:0/137893367 >> 172.21.15.38:0/137893367 conn(0x7f16780668f0 msgr2=0x7f167815f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.279+0000 7f16800bc640 1 -- 172.21.15.38:0/137893367 shutdown_connections 2024-04-25T08:35:24.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.279+0000 7f16800bc640 1 -- 172.21.15.38:0/137893367 wait complete. 2024-04-25T08:35:24.284 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:35:24.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.393+0000 7fbb855a5640 1 Processor -- start 2024-04-25T08:35:24.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.393+0000 7fbb855a5640 1 -- start start 2024-04-25T08:35:24.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb855a5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800d33d0 0x7fbb800d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb855a5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbb800d1e90 0x7fbb800b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb855a5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb800d2330 con 0x7fbb800d0320 2024-04-25T08:35:24.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb855a5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb800d2610 con 0x7fbb800d1e90 2024-04-25T08:35:24.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb855a5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800d28f0 con 0x7fbb800d33d0 2024-04-25T08:35:24.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb7f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbb800d1e90 0x7fbb800b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb7f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbb800d1e90 0x7fbb800b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56082/0 (socket says 172.21.15.38:56082) 2024-04-25T08:35:24.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb7f7fe640 1 -- 172.21.15.38:0/3363914724 learned_addr learned my addr 172.21.15.38:0/3363914724 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:24.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb7e7fc640 1 --2- 172.21.15.38:0/3363914724 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800d33d0 0x7fbb800d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb7f7fe640 1 -- 172.21.15.38:0/3363914724 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800d33d0 msgr2=0x7fbb800d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb7f7fe640 1 --2- 172.21.15.38:0/3363914724 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800d33d0 0x7fbb800d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb7f7fe640 1 -- 172.21.15.38:0/3363914724 >> v1:172.21.15.107:6789/0 conn(0x7fbb800d0320 legacy=0x7fbb800d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb7f7fe640 1 -- 172.21.15.38:0/3363914724 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb800d2bd0 con 0x7fbb800d1e90 2024-04-25T08:35:24.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.394+0000 7fbb7e7fc640 1 --2- 172.21.15.38:0/3363914724 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800d33d0 0x7fbb800d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:24.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.395+0000 7fbb7f7fe640 1 --2- 172.21.15.38:0/3363914724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800d1e90 0x7fbb800b6980 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009e50 tx=0x7fbb7802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a2f03018021f5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.395+0000 7fbb7dffb640 1 -- 172.21.15.38:0/3363914724 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb78036070 con 0x7fbb800d1e90 2024-04-25T08:35:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.395+0000 7fbb7dffb640 1 -- 172.21.15.38:0/3363914724 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb78031600 con 0x7fbb800d1e90 2024-04-25T08:35:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.395+0000 7fbb7dffb640 1 -- 172.21.15.38:0/3363914724 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb780319d0 con 0x7fbb800d1e90 2024-04-25T08:35:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.396+0000 7fbb855a5640 1 -- 172.21.15.38:0/3363914724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800d1e90 msgr2=0x7fbb800b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.396+0000 7fbb855a5640 1 --2- 172.21.15.38:0/3363914724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800d1e90 0x7fbb800b6980 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbb78009e50 tx=0x7fbb7802a750 comp rx=0 tx=0).stop 2024-04-25T08:35:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.396+0000 7fbb855a5640 1 -- 172.21.15.38:0/3363914724 shutdown_connections 2024-04-25T08:35:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.396+0000 7fbb855a5640 1 --2- 172.21.15.38:0/3363914724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800d1e90 0x7fbb800b6980 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.396+0000 7fbb855a5640 1 --2- 172.21.15.38:0/3363914724 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800d33d0 0x7fbb800d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.396+0000 7fbb855a5640 1 -- 172.21.15.38:0/3363914724 >> 172.21.15.38:0/3363914724 conn(0x7fbb800668f0 msgr2=0x7fbb80066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.396+0000 7fbb855a5640 1 -- 172.21.15.38:0/3363914724 shutdown_connections 2024-04-25T08:35:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.396+0000 7fbb855a5640 1 -- 172.21.15.38:0/3363914724 wait complete. 2024-04-25T08:35:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.397+0000 7fbb855a5640 1 Processor -- start 2024-04-25T08:35:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.397+0000 7fbb855a5640 1 -- start start 2024-04-25T08:35:24.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb855a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800d0320 0x7fbb8014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb855a5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800d1e90 0x7fbb801e7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb855a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb800d33d0 0x7fbb8015c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb855a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80161390 con 0x7fbb800d0320 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb855a5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80161210 con 0x7fbb800d33d0 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb855a5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb800b9720 con 0x7fbb800d1e90 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800d0320 0x7fbb8014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb7effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800d0320 0x7fbb8014de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56094/0 (socket says 172.21.15.38:56094) 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb7effd640 1 -- 172.21.15.38:0/3383143638 learned_addr learned my addr 172.21.15.38:0/3383143638 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb7f7fe640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb800d33d0 0x7fbb8015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb7e7fc640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800d1e90 0x7fbb801e7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb7effd640 1 -- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800d1e90 msgr2=0x7fbb801e7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb7effd640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800d1e90 0x7fbb801e7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb7effd640 1 -- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb800d33d0 msgr2=0x7fbb8015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb7effd640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb800d33d0 0x7fbb8015c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb7effd640 1 -- 172.21.15.38:0/3383143638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb801f7650 con 0x7fbb800d0320 2024-04-25T08:35:24.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb7f7fe640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb800d33d0 0x7fbb8015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:24.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.398+0000 7fbb7effd640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800d0320 0x7fbb8014de30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000c430 tx=0x7fbb7000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.399+0000 7fbb677fe640 1 -- 172.21.15.38:0/3383143638 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb70011790 con 0x7fbb800d0320 2024-04-25T08:35:24.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.399+0000 7fbb677fe640 1 -- 172.21.15.38:0/3383143638 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb70017070 con 0x7fbb800d0320 2024-04-25T08:35:24.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.399+0000 7fbb677fe640 1 -- 172.21.15.38:0/3383143638 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb70011bb0 con 0x7fbb800d0320 2024-04-25T08:35:24.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.399+0000 7fbb855a5640 1 -- 172.21.15.38:0/3383143638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb801f7930 con 0x7fbb800d0320 2024-04-25T08:35:24.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.399+0000 7fbb855a5640 1 -- 172.21.15.38:0/3383143638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb801f7ef0 con 0x7fbb800d0320 2024-04-25T08:35:24.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.400+0000 7fbb855a5640 1 -- 172.21.15.38:0/3383143638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb80068e80 con 0x7fbb800d0320 2024-04-25T08:35:24.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.400+0000 7fbb677fe640 1 -- 172.21.15.38:0/3383143638 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb7001c460 con 0x7fbb800d0320 2024-04-25T08:35:24.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.401+0000 7fbb677fe640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb6c06b4d0 0x7fbb6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.401+0000 7fbb677fe640 1 -- 172.21.15.38:0/3383143638 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fbb7008a290 con 0x7fbb800d0320 2024-04-25T08:35:24.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.401+0000 7fbb7e7fc640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb6c06b4d0 0x7fbb6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.401+0000 7fbb7e7fc640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb6c06b4d0 0x7fbb6c06d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbb800680b0 tx=0x7fbb74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.405+0000 7fbb677fe640 1 -- 172.21.15.38:0/3383143638 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb70058f40 con 0x7fbb800d0320 2024-04-25T08:35:24.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.589+0000 7fbb855a5640 1 -- 172.21.15.38:0/3383143638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb8015e450 con 0x7fbb800d0320 2024-04-25T08:35:24.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.590+0000 7fbb677fe640 1 -- 172.21.15.38:0/3383143638 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbb70058840 con 0x7fbb800d0320 2024-04-25T08:35:24.584 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:35:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.592+0000 7fbb855a5640 1 -- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb6c06b4d0 msgr2=0x7fbb6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.592+0000 7fbb855a5640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb6c06b4d0 0x7fbb6c06d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fbb800680b0 tx=0x7fbb74008040 comp rx=0 tx=0).stop 2024-04-25T08:35:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.592+0000 7fbb855a5640 1 -- 172.21.15.38:0/3383143638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800d0320 msgr2=0x7fbb8014de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.592+0000 7fbb855a5640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800d0320 0x7fbb8014de30 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000c430 tx=0x7fbb7000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.592+0000 7fbb855a5640 1 -- 172.21.15.38:0/3383143638 shutdown_connections 2024-04-25T08:35:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.592+0000 7fbb855a5640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb800d33d0 0x7fbb8015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.592+0000 7fbb855a5640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb6c06b4d0 0x7fbb6c06d990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.592+0000 7fbb855a5640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800d1e90 0x7fbb801e7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.592+0000 7fbb855a5640 1 --2- 172.21.15.38:0/3383143638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800d0320 0x7fbb8014de30 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.592+0000 7fbb855a5640 1 -- 172.21.15.38:0/3383143638 >> 172.21.15.38:0/3383143638 conn(0x7fbb800668f0 msgr2=0x7fbb800ccc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.592+0000 7fbb855a5640 1 -- 172.21.15.38:0/3383143638 shutdown_connections 2024-04-25T08:35:24.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.592+0000 7fbb855a5640 1 -- 172.21.15.38:0/3383143638 wait complete. 2024-04-25T08:35:24.597 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:35:24.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.704+0000 7fa02f9f5640 1 Processor -- start 2024-04-25T08:35:24.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.704+0000 7fa02f9f5640 1 -- start start 2024-04-25T08:35:24.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.705+0000 7fa02f9f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa0280b9600 0x7fa0280d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.705+0000 7fa02f9f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0280d0660 0x7fa0280d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.705+0000 7fa02f9f5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa028162030 con 0x7fa02815db80 2024-04-25T08:35:24.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.705+0000 7fa02f9f5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa028161df0 con 0x7fa0280b9600 2024-04-25T08:35:24.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.705+0000 7fa02f9f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0281625f0 con 0x7fa0280d0660 2024-04-25T08:35:24.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.705+0000 7fa02d791640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa02815db80 0x7fa0281616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53338/0 (socket says 172.21.15.38:53338) 2024-04-25T08:35:24.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.705+0000 7fa02d791640 1 -- 172.21.15.38:0/2455912042 learned_addr learned my addr 172.21.15.38:0/2455912042 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:24.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.705+0000 7fa02df92640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0280d0660 0x7fa0280d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.705+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 329263046 0 0) 0x7fa028162030 con 0x7fa02815db80 2024-04-25T08:35:24.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.705+0000 7fa02cf90640 1 --2- 172.21.15.38:0/2455912042 >> v2:172.21.15.107:3300/0 conn(0x7fa0280b9600 0x7fa0280d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.705+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa014003520 con 0x7fa02815db80 2024-04-25T08:35:24.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3820725621 0 0) 0x7fa014003520 con 0x7fa02815db80 2024-04-25T08:35:24.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0280d0660 msgr2=0x7fa0280d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0280d0660 0x7fa0280d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 >> v2:172.21.15.107:3300/0 conn(0x7fa0280b9600 msgr2=0x7fa0280d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 --2- 172.21.15.38:0/2455912042 >> v2:172.21.15.107:3300/0 conn(0x7fa0280b9600 0x7fa0280d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa028162310 con 0x7fa02815db80 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fa024001670 con 0x7fa02815db80 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 >> v1:172.21.15.38:6789/0 conn(0x7fa02815db80 legacy=0x7fa0281616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0140039b0 0x7fa014003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0280d0660 0x7fa014004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa014004d40 0x7fa0140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028161df0 con 0x7fa014004d40 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0281625f0 con 0x7fa0140039b0 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01400b9c0 con 0x7fa0280d0660 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa02df92640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa014004d40 0x7fa0140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa02d791640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0140039b0 0x7fa014003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa02cf90640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0280d0660 0x7fa014004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa02d791640 1 -- 172.21.15.38:0/2455912042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0280d0660 msgr2=0x7fa014004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa02d791640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0280d0660 0x7fa014004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa02d791640 1 -- 172.21.15.38:0/2455912042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa014004d40 msgr2=0x7fa0140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:24.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa02d791640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa014004d40 0x7fa0140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa02d791640 1 -- 172.21.15.38:0/2455912042 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa014003520 con 0x7fa0140039b0 2024-04-25T08:35:24.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.706+0000 7fa02df92640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa014004d40 0x7fa0140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:24.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.707+0000 7fa02d791640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0140039b0 0x7fa014003dd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa02400c210 tx=0x7fa014005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3264e67333182d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.707+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024007d50 con 0x7fa0140039b0 2024-04-25T08:35:24.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.707+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa024005af0 con 0x7fa0140039b0 2024-04-25T08:35:24.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.707+0000 7fa01ffff640 1 -- 172.21.15.38:0/2455912042 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa024007070 con 0x7fa0140039b0 2024-04-25T08:35:24.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.708+0000 7fa02f9f5640 1 -- 172.21.15.38:0/2455912042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0140039b0 msgr2=0x7fa014003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.708+0000 7fa02f9f5640 1 --2- 172.21.15.38:0/2455912042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0140039b0 0x7fa014003dd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa02400c210 tx=0x7fa014005830 comp rx=0 tx=0).stop 2024-04-25T08:35:24.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.708+0000 7fa02df92640 1 -- 172.21.15.38:0/2455912042 reap_dead start 2024-04-25T08:35:24.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.708+0000 7fa02f9f5640 1 -- 172.21.15.38:0/2455912042 shutdown_connections 2024-04-25T08:35:24.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.708+0000 7fa02f9f5640 1 -- 172.21.15.38:0/2455912042 >> 172.21.15.38:0/2455912042 conn(0x7fa0280668f0 msgr2=0x7fa028066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:24.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.708+0000 7fa02f9f5640 1 -- 172.21.15.38:0/2455912042 shutdown_connections 2024-04-25T08:35:24.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.708+0000 7fa02f9f5640 1 -- 172.21.15.38:0/2455912042 wait complete. 2024-04-25T08:35:24.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.709+0000 7fa02f9f5640 1 Processor -- start 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.709+0000 7fa02f9f5640 1 -- start start 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02f9f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0140039b0 0x7fa028147b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02f9f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa02815e0e0 0x7fa02815e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02f9f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa028160440 0x7fa0281608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02f9f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02815df60 con 0x7fa02815e0e0 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02f9f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280d0ab0 con 0x7fa028160440 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02f9f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0280d22b0 con 0x7fa0140039b0 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02cf90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa02815e0e0 0x7fa02815e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02cf90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa02815e0e0 0x7fa02815e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56120/0 (socket says 172.21.15.38:56120) 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02cf90640 1 -- 172.21.15.38:0/3149098879 learned_addr learned my addr 172.21.15.38:0/3149098879 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02df92640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa028160440 0x7fa0281608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02d791640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0140039b0 0x7fa028147b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02cf90640 1 -- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0140039b0 msgr2=0x7fa028147b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02cf90640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0140039b0 0x7fa028147b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02cf90640 1 -- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa028160440 msgr2=0x7fa0281608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02cf90640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa028160440 0x7fa0281608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02cf90640 1 -- 172.21.15.38:0/3149098879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa028148060 con 0x7fa02815e0e0 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02d791640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0140039b0 0x7fa028147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02df92640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa028160440 0x7fa0281608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02cf90640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa02815e0e0 0x7fa02815e540 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa018011940 tx=0x7fa018011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa01dffb640 1 -- 172.21.15.38:0/3149098879 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018019070 con 0x7fa02815e0e0 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa01dffb640 1 -- 172.21.15.38:0/3149098879 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa018004240 con 0x7fa02815e0e0 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa02f9f5640 1 -- 172.21.15.38:0/3149098879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02815eb30 con 0x7fa02815e0e0 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.710+0000 7fa01dffb640 1 -- 172.21.15.38:0/3149098879 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01800f8e0 con 0x7fa02815e0e0 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.711+0000 7fa02f9f5640 1 -- 172.21.15.38:0/3149098879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02815f0f0 con 0x7fa02815e0e0 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.711+0000 7fa02f9f5640 1 -- 172.21.15.38:0/3149098879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02804ef90 con 0x7fa02815e0e0 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.712+0000 7fa01dffb640 1 -- 172.21.15.38:0/3149098879 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa01801c070 con 0x7fa02815e0e0 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.712+0000 7fa01dffb640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa00c06b4d0 0x7fa00c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.713+0000 7fa01dffb640 1 -- 172.21.15.38:0/3149098879 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa01808a860 con 0x7fa02815e0e0 2024-04-25T08:35:24.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.713+0000 7fa02d791640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa00c06b4d0 0x7fa00c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:24.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.713+0000 7fa02d791640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa00c06b4d0 0x7fa00c06d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa024033000 tx=0x7fa024009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:24.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.716+0000 7fa01dffb640 1 -- 172.21.15.38:0/3149098879 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa018059480 con 0x7fa02815e0e0 2024-04-25T08:35:24.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.909+0000 7fa02f9f5640 1 -- 172.21.15.38:0/3149098879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa0280d14a0 con 0x7fa02815e0e0 2024-04-25T08:35:24.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.910+0000 7fa01dffb640 1 -- 172.21.15.38:0/3149098879 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa018058d80 con 0x7fa02815e0e0 2024-04-25T08:35:24.904 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:35:24.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.912+0000 7fa02f9f5640 1 -- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa00c06b4d0 msgr2=0x7fa00c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.912+0000 7fa02f9f5640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa00c06b4d0 0x7fa00c06d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa024033000 tx=0x7fa024009030 comp rx=0 tx=0).stop 2024-04-25T08:35:24.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.912+0000 7fa02f9f5640 1 -- 172.21.15.38:0/3149098879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa02815e0e0 msgr2=0x7fa02815e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:24.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.912+0000 7fa02f9f5640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa02815e0e0 0x7fa02815e540 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fa018011940 tx=0x7fa018011e00 comp rx=0 tx=0).stop 2024-04-25T08:35:24.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.912+0000 7fa02f9f5640 1 -- 172.21.15.38:0/3149098879 shutdown_connections 2024-04-25T08:35:24.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.912+0000 7fa02f9f5640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa028160440 0x7fa0281608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.912+0000 7fa02f9f5640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa02815e0e0 0x7fa02815e540 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.912+0000 7fa02f9f5640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa00c06b4d0 0x7fa00c06d990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.912+0000 7fa02f9f5640 1 --2- 172.21.15.38:0/3149098879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0140039b0 0x7fa028147b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:24.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.912+0000 7fa02f9f5640 1 -- 172.21.15.38:0/3149098879 >> 172.21.15.38:0/3149098879 conn(0x7fa0280668f0 msgr2=0x7fa0280c9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:24.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.912+0000 7fa02f9f5640 1 -- 172.21.15.38:0/3149098879 shutdown_connections 2024-04-25T08:35:24.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:24.912+0000 7fa02f9f5640 1 -- 172.21.15.38:0/3149098879 wait complete. 2024-04-25T08:35:24.917 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:35:24.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:24.917 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:24.940 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[60477]: fuse finished with error 0 and tester_r 0 2024-04-25T08:35:24.945 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:35:24.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:24.946 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:35:25.122 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:35:25.122 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:25.139 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:35:25.139 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:35:25.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:25.139 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:35:25.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:35:25.195 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:35:25.195 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:35:25.195 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:25.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:35:25.211 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:35:25.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:25.211 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:35:25.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:35:25.266 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:35:25.267 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:35:25.267 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:25.283 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:35:25.283 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:35:25.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:25.283 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:35:25.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:35:25.339 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:35:25.339 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.446+0000 7f070cb16640 1 Processor -- start 2024-04-25T08:35:25.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.446+0000 7f070cb16640 1 -- start start 2024-04-25T08:35:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.447+0000 7f070cb16640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f070815b920 0x7f07080cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.447+0000 7f070cb16640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0708099a70 0x7f07080cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.447+0000 7f070cb16640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f07080d02a0 0x7f07080b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.447+0000 7f070cb16640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07080cc2b0 con 0x7f07080d02a0 2024-04-25T08:35:25.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.447+0000 7f070cb16640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f070809a1e0 con 0x7f0708099a70 2024-04-25T08:35:25.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.447+0000 7f070cb16640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070809a4c0 con 0x7f070815b920 2024-04-25T08:35:25.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.447+0000 7f0706d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f07080d02a0 0x7f07080b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.447+0000 7f0706d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f07080d02a0 0x7f07080b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56126/0 (socket says 172.21.15.38:56126) 2024-04-25T08:35:25.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.447+0000 7f0706d9d640 1 -- 172.21.15.38:0/1583287473 learned_addr learned my addr 172.21.15.38:0/1583287473 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:25.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.448+0000 7f06fffff640 1 --2- 172.21.15.38:0/1583287473 >> v2:172.21.15.107:3300/0 conn(0x7f0708099a70 0x7f07080cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.448+0000 7f070659c640 1 --2- 172.21.15.38:0/1583287473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f070815b920 0x7f07080cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.448+0000 7f0706d9d640 1 -- 172.21.15.38:0/1583287473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f070815b920 msgr2=0x7f07080cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.448+0000 7f0706d9d640 1 --2- 172.21.15.38:0/1583287473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f070815b920 0x7f07080cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.448+0000 7f0706d9d640 1 -- 172.21.15.38:0/1583287473 >> v2:172.21.15.107:3300/0 conn(0x7f0708099a70 msgr2=0x7f07080cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.448+0000 7f0706d9d640 1 --2- 172.21.15.38:0/1583287473 >> v2:172.21.15.107:3300/0 conn(0x7f0708099a70 0x7f07080cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.448+0000 7f0706d9d640 1 -- 172.21.15.38:0/1583287473 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070809a7a0 con 0x7f07080d02a0 2024-04-25T08:35:25.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.448+0000 7f0706d9d640 1 --2- 172.21.15.38:0/1583287473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07080d02a0 0x7f07080b1930 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f07080b42f0 tx=0x7f070002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f3d221c703318f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.449+0000 7f0705d9b640 1 -- 172.21.15.38:0/1583287473 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700036070 con 0x7f07080d02a0 2024-04-25T08:35:25.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.449+0000 7f0705d9b640 1 -- 172.21.15.38:0/1583287473 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0700031600 con 0x7f07080d02a0 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.449+0000 7f0705d9b640 1 -- 172.21.15.38:0/1583287473 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000319d0 con 0x7f07080d02a0 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.449+0000 7f070cb16640 1 -- 172.21.15.38:0/1583287473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07080d02a0 msgr2=0x7f07080b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.449+0000 7f070cb16640 1 --2- 172.21.15.38:0/1583287473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07080d02a0 0x7f07080b1930 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f07080b42f0 tx=0x7f070002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.449+0000 7f070cb16640 1 -- 172.21.15.38:0/1583287473 shutdown_connections 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.449+0000 7f070cb16640 1 --2- 172.21.15.38:0/1583287473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07080d02a0 0x7f07080b1930 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.449+0000 7f070cb16640 1 --2- 172.21.15.38:0/1583287473 >> v2:172.21.15.107:3300/0 conn(0x7f0708099a70 0x7f07080cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.449+0000 7f070cb16640 1 --2- 172.21.15.38:0/1583287473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f070815b920 0x7f07080cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.449+0000 7f070cb16640 1 -- 172.21.15.38:0/1583287473 >> 172.21.15.38:0/1583287473 conn(0x7f07080d17a0 msgr2=0x7f07080d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.449+0000 7f070cb16640 1 -- 172.21.15.38:0/1583287473 shutdown_connections 2024-04-25T08:35:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.449+0000 7f070cb16640 1 -- 172.21.15.38:0/1583287473 wait complete. 2024-04-25T08:35:25.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.451+0000 7f070cb16640 1 Processor -- start 2024-04-25T08:35:25.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.451+0000 7f070cb16640 1 -- start start 2024-04-25T08:35:25.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.451+0000 7f070cb16640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0708099a70 0x7f0708157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.451+0000 7f070cb16640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07080d02a0 0x7f07081507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.451+0000 7f070cb16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f070815b920 0x7f070815c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.451+0000 7f070cb16640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0708099ed0 con 0x7f070815b920 2024-04-25T08:35:25.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.451+0000 7f070cb16640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f070815bd00 con 0x7f0708099a70 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.451+0000 7f070cb16640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07080b1e70 con 0x7f07080d02a0 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.451+0000 7f0706d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f070815b920 0x7f070815c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.451+0000 7f0706d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f070815b920 0x7f070815c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56134/0 (socket says 172.21.15.38:56134) 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.451+0000 7f0706d9d640 1 -- 172.21.15.38:0/116055180 learned_addr learned my addr 172.21.15.38:0/116055180 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f070659c640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0708099a70 0x7f0708157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f06fffff640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07080d02a0 0x7f07081507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f0706d9d640 1 -- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07080d02a0 msgr2=0x7f07081507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f0706d9d640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07080d02a0 0x7f07081507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f0706d9d640 1 -- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0708099a70 msgr2=0x7f0708157dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f0706d9d640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0708099a70 0x7f0708157dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f0706d9d640 1 -- 172.21.15.38:0/116055180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07080b5ea0 con 0x7f070815b920 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f0706d9d640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f070815b920 0x7f070815c920 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0700009f40 tx=0x7f0700004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f06feffd640 1 -- 172.21.15.38:0/116055180 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700036050 con 0x7f070815b920 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f06feffd640 1 -- 172.21.15.38:0/116055180 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0700042070 con 0x7f070815b920 2024-04-25T08:35:25.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f070cb16640 1 -- 172.21.15.38:0/116055180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07080b6140 con 0x7f070815b920 2024-04-25T08:35:25.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f06feffd640 1 -- 172.21.15.38:0/116055180 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700009510 con 0x7f070815b920 2024-04-25T08:35:25.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.452+0000 7f070cb16640 1 -- 172.21.15.38:0/116055180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07080b6700 con 0x7f070815b920 2024-04-25T08:35:25.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.453+0000 7f070cb16640 1 -- 172.21.15.38:0/116055180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07080b0e80 con 0x7f070815b920 2024-04-25T08:35:25.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.454+0000 7f06feffd640 1 -- 172.21.15.38:0/116055180 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0700007bf0 con 0x7f070815b920 2024-04-25T08:35:25.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.454+0000 7f06feffd640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06e806b4d0 0x7f06e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.454+0000 7f06feffd640 1 -- 172.21.15.38:0/116055180 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f07000aaf80 con 0x7f070815b920 2024-04-25T08:35:25.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.454+0000 7f070659c640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06e806b4d0 0x7f06e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.455+0000 7f070659c640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06e806b4d0 0x7f06e806d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009aa0 tx=0x7f06f8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.458+0000 7f06feffd640 1 -- 172.21.15.38:0/116055180 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0700079680 con 0x7f070815b920 2024-04-25T08:35:25.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.676+0000 7f070cb16640 1 -- 172.21.15.38:0/116055180 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f070806f4e0 con 0x7f070815b920 2024-04-25T08:35:25.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.676+0000 7f06feffd640 1 -- 172.21.15.38:0/116055180 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f070007d720 con 0x7f070815b920 2024-04-25T08:35:25.671 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:25.671 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:35:24.877680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:25.671 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-04-25T08:35:25.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.678+0000 7f070cb16640 1 -- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06e806b4d0 msgr2=0x7f06e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.678+0000 7f070cb16640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06e806b4d0 0x7f06e806d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009aa0 tx=0x7f06f8009190 comp rx=0 tx=0).stop 2024-04-25T08:35:25.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.678+0000 7f070cb16640 1 -- 172.21.15.38:0/116055180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f070815b920 msgr2=0x7f070815c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.678+0000 7f070cb16640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f070815b920 0x7f070815c920 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f0700009f40 tx=0x7f0700004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:25.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.678+0000 7f070cb16640 1 -- 172.21.15.38:0/116055180 shutdown_connections 2024-04-25T08:35:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.678+0000 7f070cb16640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f070815b920 0x7f070815c920 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.678+0000 7f070cb16640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06e806b4d0 0x7f06e806d990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.678+0000 7f070cb16640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07080d02a0 0x7f07081507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.678+0000 7f070cb16640 1 --2- 172.21.15.38:0/116055180 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0708099a70 0x7f0708157dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.678+0000 7f070cb16640 1 -- 172.21.15.38:0/116055180 >> 172.21.15.38:0/116055180 conn(0x7f07080d17a0 msgr2=0x7f0708070390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.678+0000 7f070cb16640 1 -- 172.21.15.38:0/116055180 shutdown_connections 2024-04-25T08:35:25.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.678+0000 7f070cb16640 1 -- 172.21.15.38:0/116055180 wait complete. 2024-04-25T08:35:25.684 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:25.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.791+0000 7f2eef046640 1 Processor -- start 2024-04-25T08:35:25.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.791+0000 7f2eef046640 1 -- start start 2024-04-25T08:35:25.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.792+0000 7f2eef046640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2ee80d0fe0 0x7f2ee80b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.792+0000 7f2eef046640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee80d2e50 0x7f2ee80b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.792+0000 7f2eef046640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee80b8400 con 0x7f2ee815dc40 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.792+0000 7f2eef046640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee80d19b0 con 0x7f2ee80d0fe0 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.792+0000 7f2eef046640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80b86e0 con 0x7f2ee80d2e50 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.792+0000 7f2eecde2640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2ee815dc40 0x7f2ee80d08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53344/0 (socket says 172.21.15.38:53344) 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.792+0000 7f2eecde2640 1 -- 172.21.15.38:0/3435887989 learned_addr learned my addr 172.21.15.38:0/3435887989 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.792+0000 7f2ee7fff640 1 --2- 172.21.15.38:0/3435887989 >> v2:172.21.15.107:3300/0 conn(0x7f2ee80d0fe0 0x7f2ee80b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.792+0000 7f2eed5e3640 1 --2- 172.21.15.38:0/3435887989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee80d2e50 0x7f2ee80b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.792+0000 7f2ee77fe640 1 -- 172.21.15.38:0/3435887989 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929085122 0 0) 0x7f2ee80b8400 con 0x7f2ee815dc40 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.792+0000 7f2ee77fe640 1 -- 172.21.15.38:0/3435887989 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ed4003520 con 0x7f2ee815dc40 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.793+0000 7f2ee7fff640 1 -- 172.21.15.38:0/3435887989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee80d2e50 msgr2=0x7f2ee80b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.793+0000 7f2ee7fff640 1 --2- 172.21.15.38:0/3435887989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee80d2e50 0x7f2ee80b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.793+0000 7f2ee7fff640 1 -- 172.21.15.38:0/3435887989 >> v1:172.21.15.38:6789/0 conn(0x7f2ee815dc40 legacy=0x7f2ee80d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.793+0000 7f2ee77fe640 1 -- 172.21.15.38:0/3435887989 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 170322864 0 0) 0x7f2ed4003520 con 0x7f2ee815dc40 2024-04-25T08:35:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.793+0000 7f2ee7fff640 1 -- 172.21.15.38:0/3435887989 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee80b89c0 con 0x7f2ee80d0fe0 2024-04-25T08:35:25.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.793+0000 7f2ee7fff640 1 --2- 172.21.15.38:0/3435887989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee80d0fe0 0x7f2ee80b6a60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00097a0 tx=0x7f2ed002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3738c1cc019424e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.794+0000 7f2ee77fe640 1 -- 172.21.15.38:0/3435887989 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0037070 con 0x7f2ee80d0fe0 2024-04-25T08:35:25.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.794+0000 7f2ee77fe640 1 -- 172.21.15.38:0/3435887989 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed002ab10 con 0x7f2ee80d0fe0 2024-04-25T08:35:25.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.794+0000 7f2ee77fe640 1 -- 172.21.15.38:0/3435887989 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0001a10 con 0x7f2ee80d0fe0 2024-04-25T08:35:25.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.794+0000 7f2eef046640 1 -- 172.21.15.38:0/3435887989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee80d0fe0 msgr2=0x7f2ee80b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.794+0000 7f2eef046640 1 --2- 172.21.15.38:0/3435887989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee80d0fe0 0x7f2ee80b6a60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00097a0 tx=0x7f2ed002a080 comp rx=0 tx=0).stop 2024-04-25T08:35:25.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.794+0000 7f2eef046640 1 -- 172.21.15.38:0/3435887989 shutdown_connections 2024-04-25T08:35:25.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.794+0000 7f2eef046640 1 --2- 172.21.15.38:0/3435887989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee80d2e50 0x7f2ee80b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.794+0000 7f2eef046640 1 --2- 172.21.15.38:0/3435887989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee80d0fe0 0x7f2ee80b6a60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.794+0000 7f2eef046640 1 -- 172.21.15.38:0/3435887989 >> 172.21.15.38:0/3435887989 conn(0x7f2ee8066ce0 msgr2=0x7f2ee80670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.794+0000 7f2eef046640 1 -- 172.21.15.38:0/3435887989 shutdown_connections 2024-04-25T08:35:25.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.794+0000 7f2eef046640 1 -- 172.21.15.38:0/3435887989 wait complete. 2024-04-25T08:35:25.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.795+0000 7f2eef046640 1 Processor -- start 2024-04-25T08:35:25.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.796+0000 7f2eef046640 1 -- start start 2024-04-25T08:35:25.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.796+0000 7f2eef046640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee80d0fe0 0x7f2ee814b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.796+0000 7f2eef046640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee80d2e50 0x7f2ee815eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.796+0000 7f2eef046640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ee815dc40 0x7f2ee80c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.796+0000 7f2eef046640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80b9700 con 0x7f2ee815dc40 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.796+0000 7f2eef046640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80b9580 con 0x7f2ee80d0fe0 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.796+0000 7f2eef046640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80b9880 con 0x7f2ee80d2e50 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.796+0000 7f2eed5e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ee815dc40 0x7f2ee80c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2eed5e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ee815dc40 0x7f2ee80c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56142/0 (socket says 172.21.15.38:56142) 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2eed5e3640 1 -- 172.21.15.38:0/3002904901 learned_addr learned my addr 172.21.15.38:0/3002904901 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2ee7fff640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee80d2e50 0x7f2ee815eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2eecde2640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee80d0fe0 0x7f2ee814b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2eed5e3640 1 -- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee80d2e50 msgr2=0x7f2ee815eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2eed5e3640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee80d2e50 0x7f2ee815eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2eed5e3640 1 -- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee80d0fe0 msgr2=0x7f2ee814b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2eed5e3640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee80d0fe0 0x7f2ee814b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2eed5e3640 1 -- 172.21.15.38:0/3002904901 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee814b7d0 con 0x7f2ee815dc40 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2eed5e3640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ee815dc40 0x7f2ee80c8d70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8003c00 tx=0x7f2ed80096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2ee57fa640 1 -- 172.21.15.38:0/3002904901 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed801c620 con 0x7f2ee815dc40 2024-04-25T08:35:25.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2ee57fa640 1 -- 172.21.15.38:0/3002904901 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed8021070 con 0x7f2ee815dc40 2024-04-25T08:35:25.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2ee57fa640 1 -- 172.21.15.38:0/3002904901 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed801ca00 con 0x7f2ee815dc40 2024-04-25T08:35:25.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2eef046640 1 -- 172.21.15.38:0/3002904901 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee8155f80 con 0x7f2ee815dc40 2024-04-25T08:35:25.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.797+0000 7f2eef046640 1 -- 172.21.15.38:0/3002904901 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee8156540 con 0x7f2ee815dc40 2024-04-25T08:35:25.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.798+0000 7f2eef046640 1 -- 172.21.15.38:0/3002904901 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee8068e80 con 0x7f2ee815dc40 2024-04-25T08:35:25.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.799+0000 7f2ee57fa640 1 -- 172.21.15.38:0/3002904901 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ed801cba0 con 0x7f2ee815dc40 2024-04-25T08:35:25.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.799+0000 7f2ee57fa640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ec806b4d0 0x7f2ec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:25.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.799+0000 7f2ee57fa640 1 -- 172.21.15.38:0/3002904901 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2ed8092800 con 0x7f2ee815dc40 2024-04-25T08:35:25.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.799+0000 7f2eecde2640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ec806b4d0 0x7f2ec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:25.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.800+0000 7f2eecde2640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ec806b4d0 0x7f2ec806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0001330 tx=0x7f2ee0001d40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:25.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.804+0000 7f2ee57fa640 1 -- 172.21.15.38:0/3002904901 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ed802b020 con 0x7f2ee815dc40 2024-04-25T08:35:25.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.984+0000 7f2eef046640 1 -- 172.21.15.38:0/3002904901 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ee8073a10 con 0x7f2ee815dc40 2024-04-25T08:35:25.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.985+0000 7f2ee57fa640 1 -- 172.21.15.38:0/3002904901 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+17026 (secure 0 0 0) 0x7f2ed8060f00 con 0x7f2ee815dc40 2024-04-25T08:35:25.980 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:25.980 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":56,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:18.124795+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:34:42.507595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:34:43.209205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.28","mappings":[{"from":0,"to":6}]},{"pgid":"8.2f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:25.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.987+0000 7f2eef046640 1 -- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ec806b4d0 msgr2=0x7f2ec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.987+0000 7f2eef046640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ec806b4d0 0x7f2ec806d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0001330 tx=0x7f2ee0001d40 comp rx=0 tx=0).stop 2024-04-25T08:35:25.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.987+0000 7f2eef046640 1 -- 172.21.15.38:0/3002904901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ee815dc40 msgr2=0x7f2ee80c8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:25.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.987+0000 7f2eef046640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ee815dc40 0x7f2ee80c8d70 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f2ed8003c00 tx=0x7f2ed80096b0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.987+0000 7f2eef046640 1 -- 172.21.15.38:0/3002904901 shutdown_connections 2024-04-25T08:35:25.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.987+0000 7f2eef046640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ee815dc40 0x7f2ee80c8d70 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.987+0000 7f2eef046640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ec806b4d0 0x7f2ec806d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.987+0000 7f2eef046640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ee80d2e50 0x7f2ee815eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.987+0000 7f2eef046640 1 --2- 172.21.15.38:0/3002904901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ee80d0fe0 0x7f2ee814b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:25.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.987+0000 7f2eef046640 1 -- 172.21.15.38:0/3002904901 >> 172.21.15.38:0/3002904901 conn(0x7f2ee8066ce0 msgr2=0x7f2ee806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:25.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.987+0000 7f2eef046640 1 -- 172.21.15.38:0/3002904901 shutdown_connections 2024-04-25T08:35:25.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:25.987+0000 7f2eef046640 1 -- 172.21.15.38:0/3002904901 wait complete. 2024-04-25T08:35:25.993 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:34:42.507595+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:34:43.209205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:35:25.993 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:35:26.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.102+0000 7fc4942fc640 1 Processor -- start 2024-04-25T08:35:26.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.102+0000 7fc4942fc640 1 -- start start 2024-04-25T08:35:26.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.103+0000 7fc4942fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc48c0b1d60 0x7fc48c0cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.103+0000 7fc4942fc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48c15ef40 con 0x7fc48c0cff10 2024-04-25T08:35:26.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.103+0000 7fc4942fc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc48c06ae30 con 0x7fc48c161e20 2024-04-25T08:35:26.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.103+0000 7fc4942fc640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48c06abe0 con 0x7fc48c0b1d60 2024-04-25T08:35:26.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.103+0000 7fc492098640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc48c0b1d60 0x7fc48c0cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.103+0000 7fc491897640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc48c0cff10 0x7fc48c161710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53346/0 (socket says 172.21.15.38:53346) 2024-04-25T08:35:26.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.103+0000 7fc491897640 1 -- 172.21.15.38:0/2497118162 learned_addr learned my addr 172.21.15.38:0/2497118162 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:26.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.104+0000 7fc492098640 1 -- 172.21.15.38:0/2497118162 >> v1:172.21.15.107:6789/0 conn(0x7fc48c161e20 legacy=0x7fc48c15d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.104+0000 7fc492098640 1 -- 172.21.15.38:0/2497118162 >> v1:172.21.15.38:6789/0 conn(0x7fc48c0cff10 legacy=0x7fc48c161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.104+0000 7fc492098640 1 -- 172.21.15.38:0/2497118162 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c06b110 con 0x7fc48c0b1d60 2024-04-25T08:35:26.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.104+0000 7fc491096640 1 -- 172.21.15.38:0/2497118162 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105193671 0 0) 0x7fc48c15ef40 con 0x7fc48c0cff10 2024-04-25T08:35:26.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.104+0000 7fc492098640 1 --2- 172.21.15.38:0/2497118162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc48c0b1d60 0x7fc48c0cf9d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009d90 tx=0x7fc47c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3449114a812d69f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.105+0000 7fc491096640 1 -- 172.21.15.38:0/2497118162 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c007740 con 0x7fc48c0b1d60 2024-04-25T08:35:26.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.105+0000 7fc491096640 1 -- 172.21.15.38:0/2497118162 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47c0078e0 con 0x7fc48c0b1d60 2024-04-25T08:35:26.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.105+0000 7fc491096640 1 -- 172.21.15.38:0/2497118162 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c007cb0 con 0x7fc48c0b1d60 2024-04-25T08:35:26.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.105+0000 7fc4942fc640 1 -- 172.21.15.38:0/2497118162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc48c0b1d60 msgr2=0x7fc48c0cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.105+0000 7fc4942fc640 1 --2- 172.21.15.38:0/2497118162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc48c0b1d60 0x7fc48c0cf9d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009d90 tx=0x7fc47c02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.105+0000 7fc4942fc640 1 -- 172.21.15.38:0/2497118162 shutdown_connections 2024-04-25T08:35:26.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.105+0000 7fc4942fc640 1 --2- 172.21.15.38:0/2497118162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc48c0b1d60 0x7fc48c0cf9d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.105+0000 7fc4942fc640 1 -- 172.21.15.38:0/2497118162 >> 172.21.15.38:0/2497118162 conn(0x7fc48c0d06b0 msgr2=0x7fc48c0d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:26.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.105+0000 7fc4942fc640 1 -- 172.21.15.38:0/2497118162 shutdown_connections 2024-04-25T08:35:26.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.105+0000 7fc4942fc640 1 -- 172.21.15.38:0/2497118162 wait complete. 2024-04-25T08:35:26.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.106+0000 7fc4942fc640 1 Processor -- start 2024-04-25T08:35:26.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.106+0000 7fc4942fc640 1 -- start start 2024-04-25T08:35:26.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc4942fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc48c0b1d60 0x7fc48c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc4942fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc48c0cff10 0x7fc48c1563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc4942fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c161e20 0x7fc48c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc4942fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c06b910 con 0x7fc48c0cff10 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc4942fc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c06b480 con 0x7fc48c161e20 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc4942fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c06ba90 con 0x7fc48c0b1d60 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc491897640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc48c0cff10 0x7fc48c1563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc491897640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc48c0cff10 0x7fc48c1563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56162/0 (socket says 172.21.15.38:56162) 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc491897640 1 -- 172.21.15.38:0/97540082 learned_addr learned my addr 172.21.15.38:0/97540082 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc491897640 1 -- 172.21.15.38:0/97540082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc48c0b1d60 msgr2=0x7fc48c1539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc491897640 1 --2- 172.21.15.38:0/97540082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc48c0b1d60 0x7fc48c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc491897640 1 -- 172.21.15.38:0/97540082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c161e20 msgr2=0x7fc48c1568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc491897640 1 --2- 172.21.15.38:0/97540082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c161e20 0x7fc48c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.107+0000 7fc491897640 1 -- 172.21.15.38:0/97540082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c153f00 con 0x7fc48c0cff10 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.108+0000 7fc491897640 1 --2- 172.21.15.38:0/97540082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc48c0cff10 0x7fc48c1563a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c3f0 tx=0x7fc48000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.108+0000 7fc48affd640 1 -- 172.21.15.38:0/97540082 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800074e0 con 0x7fc48c0cff10 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.108+0000 7fc48affd640 1 -- 172.21.15.38:0/97540082 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4800047e0 con 0x7fc48c0cff10 2024-04-25T08:35:26.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.108+0000 7fc4942fc640 1 -- 172.21.15.38:0/97540082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48c1f78d0 con 0x7fc48c0cff10 2024-04-25T08:35:26.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.108+0000 7fc48affd640 1 -- 172.21.15.38:0/97540082 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480004bd0 con 0x7fc48c0cff10 2024-04-25T08:35:26.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.108+0000 7fc4942fc640 1 -- 172.21.15.38:0/97540082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48c1f7e90 con 0x7fc48c0cff10 2024-04-25T08:35:26.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.109+0000 7fc4942fc640 1 -- 172.21.15.38:0/97540082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48c068e80 con 0x7fc48c0cff10 2024-04-25T08:35:26.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.109+0000 7fc48affd640 1 -- 172.21.15.38:0/97540082 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc480011070 con 0x7fc48c0cff10 2024-04-25T08:35:26.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.110+0000 7fc48affd640 1 --2- 172.21.15.38:0/97540082 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc47806b4d0 0x7fc47806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.110+0000 7fc48affd640 1 -- 172.21.15.38:0/97540082 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc480089eb0 con 0x7fc48c0cff10 2024-04-25T08:35:26.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.110+0000 7fc492098640 1 --2- 172.21.15.38:0/97540082 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc47806b4d0 0x7fc47806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.110+0000 7fc492098640 1 --2- 172.21.15.38:0/97540082 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc47806b4d0 0x7fc47806d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc47c034000 tx=0x7fc47c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.114+0000 7fc48affd640 1 -- 172.21.15.38:0/97540082 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc48001a2b0 con 0x7fc48c0cff10 2024-04-25T08:35:26.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.295+0000 7fc4942fc640 1 -- 172.21.15.38:0/97540082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc48c06c9e0 con 0x7fc48c0cff10 2024-04-25T08:35:26.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.296+0000 7fc48affd640 1 -- 172.21.15.38:0/97540082 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc480058c00 con 0x7fc48c0cff10 2024-04-25T08:35:26.291 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:35:26.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.298+0000 7fc4942fc640 1 -- 172.21.15.38:0/97540082 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc47806b4d0 msgr2=0x7fc47806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.298+0000 7fc4942fc640 1 --2- 172.21.15.38:0/97540082 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc47806b4d0 0x7fc47806d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc47c034000 tx=0x7fc47c02e000 comp rx=0 tx=0).stop 2024-04-25T08:35:26.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.298+0000 7fc4942fc640 1 -- 172.21.15.38:0/97540082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc48c0cff10 msgr2=0x7fc48c1563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.298+0000 7fc4942fc640 1 --2- 172.21.15.38:0/97540082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc48c0cff10 0x7fc48c1563a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc48000c3f0 tx=0x7fc48000c8b0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.298+0000 7fc4942fc640 1 -- 172.21.15.38:0/97540082 shutdown_connections 2024-04-25T08:35:26.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.298+0000 7fc4942fc640 1 --2- 172.21.15.38:0/97540082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc48c161e20 0x7fc48c1568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.298+0000 7fc4942fc640 1 --2- 172.21.15.38:0/97540082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc48c0cff10 0x7fc48c1563a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.298+0000 7fc4942fc640 1 --2- 172.21.15.38:0/97540082 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc47806b4d0 0x7fc47806d990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.298+0000 7fc4942fc640 1 --2- 172.21.15.38:0/97540082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc48c0b1d60 0x7fc48c1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.298+0000 7fc4942fc640 1 -- 172.21.15.38:0/97540082 >> 172.21.15.38:0/97540082 conn(0x7fc48c0d06b0 msgr2=0x7fc48c160d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:26.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.298+0000 7fc4942fc640 1 -- 172.21.15.38:0/97540082 shutdown_connections 2024-04-25T08:35:26.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.298+0000 7fc4942fc640 1 -- 172.21.15.38:0/97540082 wait complete. 2024-04-25T08:35:26.303 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:35:26.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.413+0000 7f1a47d30640 1 Processor -- start 2024-04-25T08:35:26.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.413+0000 7f1a47d30640 1 -- start start 2024-04-25T08:35:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.414+0000 7f1a47d30640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1a400d0320 0x7f1a400d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.414+0000 7f1a47d30640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a400d2060 con 0x7f1a400b6980 2024-04-25T08:35:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.414+0000 7f1a47d30640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a400d2330 con 0x7f1a400d0700 2024-04-25T08:35:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.414+0000 7f1a47d30640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a400d2610 con 0x7f1a400d0320 2024-04-25T08:35:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.414+0000 7f1a462cd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1a400b6980 0x7f1a400b6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53362/0 (socket says 172.21.15.38:53362) 2024-04-25T08:35:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.414+0000 7f1a462cd640 1 -- 172.21.15.38:0/3861935254 learned_addr learned my addr 172.21.15.38:0/3861935254 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.414+0000 7f1a45acc640 1 --2- 172.21.15.38:0/3861935254 >> v2:172.21.15.107:3300/0 conn(0x7f1a400d0320 0x7f1a400d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.415+0000 7f1a44aca640 1 -- 172.21.15.38:0/3861935254 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588590049 0 0) 0x7f1a400d2060 con 0x7f1a400b6980 2024-04-25T08:35:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.415+0000 7f1a44aca640 1 -- 172.21.15.38:0/3861935254 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a28003520 con 0x7f1a400b6980 2024-04-25T08:35:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.415+0000 7f1a44aca640 1 -- 172.21.15.38:0/3861935254 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300753922 0 0) 0x7f1a400d2330 con 0x7f1a400d0700 2024-04-25T08:35:26.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.415+0000 7f1a44aca640 1 -- 172.21.15.38:0/3861935254 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a400d2060 con 0x7f1a400d0700 2024-04-25T08:35:26.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.415+0000 7f1a45acc640 1 -- 172.21.15.38:0/3861935254 >> v1:172.21.15.107:6789/0 conn(0x7f1a400d0700 legacy=0x7f1a400d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.415+0000 7f1a45acc640 1 -- 172.21.15.38:0/3861935254 >> v1:172.21.15.38:6789/0 conn(0x7f1a400b6980 legacy=0x7f1a400b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.415+0000 7f1a45acc640 1 -- 172.21.15.38:0/3861935254 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a400d28f0 con 0x7f1a400d0320 2024-04-25T08:35:26.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.415+0000 7f1a45acc640 1 --2- 172.21.15.38:0/3861935254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a400d0320 0x7f1a400d33d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a380097a0 tx=0x7f1a3802a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=222cc2d9c8a645b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.416+0000 7f1a44aca640 1 -- 172.21.15.38:0/3861935254 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38037070 con 0x7f1a400d0320 2024-04-25T08:35:26.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.416+0000 7f1a44aca640 1 -- 172.21.15.38:0/3861935254 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a38032640 con 0x7f1a400d0320 2024-04-25T08:35:26.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.416+0000 7f1a44aca640 1 -- 172.21.15.38:0/3861935254 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38032a10 con 0x7f1a400d0320 2024-04-25T08:35:26.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.416+0000 7f1a47d30640 1 -- 172.21.15.38:0/3861935254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a400d0320 msgr2=0x7f1a400d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.416+0000 7f1a47d30640 1 --2- 172.21.15.38:0/3861935254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a400d0320 0x7f1a400d33d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1a380097a0 tx=0x7f1a3802a2c0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.416+0000 7f1a47d30640 1 -- 172.21.15.38:0/3861935254 shutdown_connections 2024-04-25T08:35:26.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.416+0000 7f1a47d30640 1 --2- 172.21.15.38:0/3861935254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a400d0320 0x7f1a400d33d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.416+0000 7f1a47d30640 1 -- 172.21.15.38:0/3861935254 >> 172.21.15.38:0/3861935254 conn(0x7f1a400668f0 msgr2=0x7f1a40066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:26.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.416+0000 7f1a47d30640 1 -- 172.21.15.38:0/3861935254 shutdown_connections 2024-04-25T08:35:26.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.416+0000 7f1a47d30640 1 -- 172.21.15.38:0/3861935254 wait complete. 2024-04-25T08:35:26.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.417+0000 7f1a47d30640 1 Processor -- start 2024-04-25T08:35:26.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.418+0000 7f1a47d30640 1 -- start start 2024-04-25T08:35:26.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.418+0000 7f1a47d30640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a400b6980 0x7f1a401e8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.418+0000 7f1a47d30640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a400d0320 0x7f1a4015a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.418+0000 7f1a47d30640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a400d0700 0x7f1a401f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.418+0000 7f1a47d30640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400d0b40 con 0x7f1a400d0320 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.418+0000 7f1a47d30640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a400d2eb0 con 0x7f1a400b6980 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.418+0000 7f1a47d30640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a401e8c60 con 0x7f1a400d0700 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.418+0000 7f1a452cb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a400d0320 0x7f1a4015a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.418+0000 7f1a452cb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a400d0320 0x7f1a4015a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56170/0 (socket says 172.21.15.38:56170) 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.418+0000 7f1a452cb640 1 -- 172.21.15.38:0/3448408272 learned_addr learned my addr 172.21.15.38:0/3448408272 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.419+0000 7f1a45acc640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a400b6980 0x7f1a401e8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.419+0000 7f1a452cb640 1 -- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a400d0700 msgr2=0x7f1a401f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.419+0000 7f1a452cb640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a400d0700 0x7f1a401f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.419+0000 7f1a452cb640 1 -- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a400b6980 msgr2=0x7f1a401e8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.419+0000 7f1a452cb640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a400b6980 0x7f1a401e8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.419+0000 7f1a452cb640 1 -- 172.21.15.38:0/3448408272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a401e8e80 con 0x7f1a400d0320 2024-04-25T08:35:26.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.419+0000 7f1a452cb640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a400d0320 0x7f1a4015a9d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400c430 tx=0x7f1a3400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.419+0000 7f1a3e7fc640 1 -- 172.21.15.38:0/3448408272 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34017070 con 0x7f1a400d0320 2024-04-25T08:35:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.419+0000 7f1a3e7fc640 1 -- 172.21.15.38:0/3448408272 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a340052f0 con 0x7f1a400d0320 2024-04-25T08:35:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.419+0000 7f1a3e7fc640 1 -- 172.21.15.38:0/3448408272 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34003830 con 0x7f1a400d0320 2024-04-25T08:35:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.419+0000 7f1a47d30640 1 -- 172.21.15.38:0/3448408272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a400b3310 con 0x7f1a400d0320 2024-04-25T08:35:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.419+0000 7f1a47d30640 1 -- 172.21.15.38:0/3448408272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a400b38d0 con 0x7f1a400d0320 2024-04-25T08:35:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.420+0000 7f1a47d30640 1 -- 172.21.15.38:0/3448408272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a400b3ad0 con 0x7f1a400d0320 2024-04-25T08:35:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.420+0000 7f1a3e7fc640 1 -- 172.21.15.38:0/3448408272 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a3401a070 con 0x7f1a400d0320 2024-04-25T08:35:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.421+0000 7f1a3e7fc640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.421+0000 7f1a3e7fc640 1 -- 172.21.15.38:0/3448408272 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1a34089760 con 0x7f1a400d0320 2024-04-25T08:35:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.421+0000 7f1a45acc640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.422+0000 7f1a45acc640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1a38034000 tx=0x7f1a3802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.425+0000 7f1a3e7fc640 1 -- 172.21.15.38:0/3448408272 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a400b3ad0 con 0x7f1a400d0320 2024-04-25T08:35:26.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.604+0000 7f1a47d30640 1 -- 172.21.15.38:0/3448408272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a40068e80 con 0x7f1a400d0320 2024-04-25T08:35:26.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.604+0000 7f1a3e7fc640 1 -- 172.21.15.38:0/3448408272 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1a34058420 con 0x7f1a400d0320 2024-04-25T08:35:26.599 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:35:26.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.606+0000 7f1a47d30640 1 -- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1a2c06b4d0 msgr2=0x7f1a2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.606+0000 7f1a47d30640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1a38034000 tx=0x7f1a3802e000 comp rx=0 tx=0).stop 2024-04-25T08:35:26.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.606+0000 7f1a47d30640 1 -- 172.21.15.38:0/3448408272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a400d0320 msgr2=0x7f1a4015a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.606+0000 7f1a47d30640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a400d0320 0x7f1a4015a9d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400c430 tx=0x7f1a3400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.606+0000 7f1a47d30640 1 -- 172.21.15.38:0/3448408272 shutdown_connections 2024-04-25T08:35:26.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.606+0000 7f1a47d30640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.606+0000 7f1a47d30640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a400d0700 0x7f1a401f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.606+0000 7f1a47d30640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a400d0320 0x7f1a4015a9d0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.606+0000 7f1a47d30640 1 --2- 172.21.15.38:0/3448408272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a400b6980 0x7f1a401e8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.606+0000 7f1a47d30640 1 -- 172.21.15.38:0/3448408272 >> 172.21.15.38:0/3448408272 conn(0x7f1a400668f0 msgr2=0x7f1a4015e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:26.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.606+0000 7f1a47d30640 1 -- 172.21.15.38:0/3448408272 shutdown_connections 2024-04-25T08:35:26.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.606+0000 7f1a47d30640 1 -- 172.21.15.38:0/3448408272 wait complete. 2024-04-25T08:35:26.612 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:35:26.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.721+0000 7f06ed4f7640 1 Processor -- start 2024-04-25T08:35:26.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.721+0000 7f06ed4f7640 1 -- start start 2024-04-25T08:35:26.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06ed4f7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f06e815bb90 0x7f06e80b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06ed4f7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06e80cc130 0x7f06e80b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06ed4f7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e80c3e40 con 0x7f06e80b13f0 2024-04-25T08:35:26.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06ed4f7640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e806b8d0 con 0x7f06e815bb90 2024-04-25T08:35:26.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06ed4f7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e80b00b0 con 0x7f06e80cc130 2024-04-25T08:35:26.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06e67fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f06e815bb90 0x7f06e80b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06e6ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f06e80b13f0 0x7f06e815c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53366/0 (socket says 172.21.15.38:53366) 2024-04-25T08:35:26.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06e6ffd640 1 -- 172.21.15.38:0/3398171373 learned_addr learned my addr 172.21.15.38:0/3398171373 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:26.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06e77fe640 1 --2- 172.21.15.38:0/3398171373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06e80cc130 0x7f06e80b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06e67fc640 1 -- 172.21.15.38:0/3398171373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06e80cc130 msgr2=0x7f06e80b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:26.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06e67fc640 1 --2- 172.21.15.38:0/3398171373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06e80cc130 0x7f06e80b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06e67fc640 1 -- 172.21.15.38:0/3398171373 >> v1:172.21.15.38:6789/0 conn(0x7f06e80b13f0 legacy=0x7f06e815c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06e5ffb640 1 -- 172.21.15.38:0/3398171373 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076687140 0 0) 0x7f06e80c3e40 con 0x7f06e80b13f0 2024-04-25T08:35:26.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.722+0000 7f06e67fc640 1 -- 172.21.15.38:0/3398171373 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80b1d10 con 0x7f06e815bb90 2024-04-25T08:35:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.723+0000 7f06e67fc640 1 --2- 172.21.15.38:0/3398171373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e815bb90 0x7f06e80b17d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0097a0 tx=0x7f06dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3ce47d9089a9d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.723+0000 7f06e5ffb640 1 -- 172.21.15.38:0/3398171373 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc037070 con 0x7f06e815bb90 2024-04-25T08:35:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.723+0000 7f06e5ffb640 1 -- 172.21.15.38:0/3398171373 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06dc032440 con 0x7f06e815bb90 2024-04-25T08:35:26.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.723+0000 7f06e5ffb640 1 -- 172.21.15.38:0/3398171373 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06dc032810 con 0x7f06e815bb90 2024-04-25T08:35:26.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.724+0000 7f06ed4f7640 1 -- 172.21.15.38:0/3398171373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e815bb90 msgr2=0x7f06e80b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.724+0000 7f06ed4f7640 1 --2- 172.21.15.38:0/3398171373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e815bb90 0x7f06e80b17d0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f06dc0097a0 tx=0x7f06dc02a180 comp rx=0 tx=0).stop 2024-04-25T08:35:26.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.724+0000 7f06ed4f7640 1 -- 172.21.15.38:0/3398171373 shutdown_connections 2024-04-25T08:35:26.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.724+0000 7f06ed4f7640 1 --2- 172.21.15.38:0/3398171373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06e80cc130 0x7f06e80b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.724+0000 7f06ed4f7640 1 --2- 172.21.15.38:0/3398171373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e815bb90 0x7f06e80b17d0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.724+0000 7f06ed4f7640 1 -- 172.21.15.38:0/3398171373 >> 172.21.15.38:0/3398171373 conn(0x7f06e80d0520 msgr2=0x7f06e80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:26.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.724+0000 7f06ed4f7640 1 -- 172.21.15.38:0/3398171373 shutdown_connections 2024-04-25T08:35:26.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.724+0000 7f06ed4f7640 1 -- 172.21.15.38:0/3398171373 wait complete. 2024-04-25T08:35:26.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.725+0000 7f06ed4f7640 1 Processor -- start 2024-04-25T08:35:26.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.725+0000 7f06ed4f7640 1 -- start start 2024-04-25T08:35:26.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06ed4f7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06e80b13f0 0x7f06e8151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06ed4f7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06e80cc130 0x7f06e80b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06ed4f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e815bb90 0x7f06e81500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06ed4f7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e806b5f0 con 0x7f06e815bb90 2024-04-25T08:35:26.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06ed4f7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e806b310 con 0x7f06e80b13f0 2024-04-25T08:35:26.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06ed4f7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e806abe0 con 0x7f06e80cc130 2024-04-25T08:35:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e815bb90 0x7f06e81500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e815bb90 0x7f06e81500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56180/0 (socket says 172.21.15.38:56180) 2024-04-25T08:35:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06e77fe640 1 -- 172.21.15.38:0/1794273275 learned_addr learned my addr 172.21.15.38:0/1794273275 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06e67fc640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06e80cc130 0x7f06e80b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06e6ffd640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06e80b13f0 0x7f06e8151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06e77fe640 1 -- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06e80cc130 msgr2=0x7f06e80b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06e77fe640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06e80cc130 0x7f06e80b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06e77fe640 1 -- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06e80b13f0 msgr2=0x7f06e8151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.726+0000 7f06e77fe640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06e80b13f0 0x7f06e8151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.727+0000 7f06e77fe640 1 -- 172.21.15.38:0/1794273275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e80b6320 con 0x7f06e815bb90 2024-04-25T08:35:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.727+0000 7f06e77fe640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e815bb90 0x7f06e81500d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f06e000efc0 tx=0x7f06e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.727+0000 7f06cf7fe640 1 -- 172.21.15.38:0/1794273275 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0018020 con 0x7f06e815bb90 2024-04-25T08:35:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.727+0000 7f06cf7fe640 1 -- 172.21.15.38:0/1794273275 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e0014480 con 0x7f06e815bb90 2024-04-25T08:35:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.727+0000 7f06ed4f7640 1 -- 172.21.15.38:0/1794273275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e80b4490 con 0x7f06e815bb90 2024-04-25T08:35:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.727+0000 7f06cf7fe640 1 -- 172.21.15.38:0/1794273275 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e0014870 con 0x7f06e815bb90 2024-04-25T08:35:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.727+0000 7f06ed4f7640 1 -- 172.21.15.38:0/1794273275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e815d360 con 0x7f06e815bb90 2024-04-25T08:35:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.728+0000 7f06ed4f7640 1 -- 172.21.15.38:0/1794273275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e8068e80 con 0x7f06e815bb90 2024-04-25T08:35:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.728+0000 7f06cf7fe640 1 -- 172.21.15.38:0/1794273275 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06e0005d10 con 0x7f06e815bb90 2024-04-25T08:35:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.729+0000 7f06cf7fe640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06d406b4d0 0x7f06d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.729+0000 7f06cf7fe640 1 -- 172.21.15.38:0/1794273275 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f06e008ad30 con 0x7f06e815bb90 2024-04-25T08:35:26.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.729+0000 7f06e6ffd640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06d406b4d0 0x7f06d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:26.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.730+0000 7f06e6ffd640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06d406b4d0 0x7f06d406d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f06d8000f60 tx=0x7f06d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:26.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.733+0000 7f06cf7fe640 1 -- 172.21.15.38:0/1794273275 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06e0022070 con 0x7f06e815bb90 2024-04-25T08:35:26.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.915+0000 7f06ed4f7640 1 -- 172.21.15.38:0/1794273275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06e8066da0 con 0x7f06e815bb90 2024-04-25T08:35:26.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.915+0000 7f06cf7fe640 1 -- 172.21.15.38:0/1794273275 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06e0010040 con 0x7f06e815bb90 2024-04-25T08:35:26.910 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:35:26.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.917+0000 7f06ed4f7640 1 -- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06d406b4d0 msgr2=0x7f06d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.917+0000 7f06ed4f7640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06d406b4d0 0x7f06d406d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f06d8000f60 tx=0x7f06d8009040 comp rx=0 tx=0).stop 2024-04-25T08:35:26.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.917+0000 7f06ed4f7640 1 -- 172.21.15.38:0/1794273275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e815bb90 msgr2=0x7f06e81500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:26.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.917+0000 7f06ed4f7640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e815bb90 0x7f06e81500d0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f06e000efc0 tx=0x7f06e000c490 comp rx=0 tx=0).stop 2024-04-25T08:35:26.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.917+0000 7f06ed4f7640 1 -- 172.21.15.38:0/1794273275 shutdown_connections 2024-04-25T08:35:26.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.917+0000 7f06ed4f7640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06e815bb90 0x7f06e81500d0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.917+0000 7f06ed4f7640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06d406b4d0 0x7f06d406d990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.917+0000 7f06ed4f7640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06e80cc130 0x7f06e80b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.917+0000 7f06ed4f7640 1 --2- 172.21.15.38:0/1794273275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06e80b13f0 0x7f06e8151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:26.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.917+0000 7f06ed4f7640 1 -- 172.21.15.38:0/1794273275 >> 172.21.15.38:0/1794273275 conn(0x7f06e80d0520 msgr2=0x7f06e806f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:26.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.917+0000 7f06ed4f7640 1 -- 172.21.15.38:0/1794273275 shutdown_connections 2024-04-25T08:35:26.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:26.917+0000 7f06ed4f7640 1 -- 172.21.15.38:0/1794273275 wait complete. 2024-04-25T08:35:26.923 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.030+0000 7ff79aab3640 1 Processor -- start 2024-04-25T08:35:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.030+0000 7ff79aab3640 1 -- start start 2024-04-25T08:35:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff79aab3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff79415dc10 0x7ff7940ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff79aab3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff7940d0f60 0x7ff7940b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff79aab3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7940d3e70 con 0x7ff7940ced60 2024-04-25T08:35:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff79aab3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79415dff0 con 0x7ff7940d0f60 2024-04-25T08:35:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff79aab3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7940b6990 con 0x7ff79415dc10 2024-04-25T08:35:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff799050640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff7940d0f60 0x7ff7940b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff793fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff7940ced60 0x7ff7940d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53372/0 (socket says 172.21.15.38:53372) 2024-04-25T08:35:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff793fff640 1 -- 172.21.15.38:0/992640385 learned_addr learned my addr 172.21.15.38:0/992640385 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff79884f640 1 --2- 172.21.15.38:0/992640385 >> v2:172.21.15.107:3300/0 conn(0x7ff79415dc10 0x7ff7940ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff799050640 1 -- 172.21.15.38:0/992640385 >> v2:172.21.15.107:3300/0 conn(0x7ff79415dc10 msgr2=0x7ff7940ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff799050640 1 --2- 172.21.15.38:0/992640385 >> v2:172.21.15.107:3300/0 conn(0x7ff79415dc10 0x7ff7940ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff799050640 1 -- 172.21.15.38:0/992640385 >> v1:172.21.15.38:6789/0 conn(0x7ff7940ced60 legacy=0x7ff7940d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff7937fe640 1 -- 172.21.15.38:0/992640385 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998072824 0 0) 0x7ff7940d3e70 con 0x7ff7940ced60 2024-04-25T08:35:27.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.031+0000 7ff799050640 1 -- 172.21.15.38:0/992640385 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7940d4150 con 0x7ff7940d0f60 2024-04-25T08:35:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.032+0000 7ff799050640 1 --2- 172.21.15.38:0/992640385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7940d0f60 0x7ff7940b2370 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff7940b5d50 tx=0x7ff78c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6aa1e313aa38be7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.032+0000 7ff7937fe640 1 -- 172.21.15.38:0/992640385 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c007740 con 0x7ff7940d0f60 2024-04-25T08:35:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.032+0000 7ff7937fe640 1 -- 172.21.15.38:0/992640385 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c0078e0 con 0x7ff7940d0f60 2024-04-25T08:35:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.032+0000 7ff7937fe640 1 -- 172.21.15.38:0/992640385 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c007cb0 con 0x7ff7940d0f60 2024-04-25T08:35:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.033+0000 7ff79aab3640 1 -- 172.21.15.38:0/992640385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7940d0f60 msgr2=0x7ff7940b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.033+0000 7ff79aab3640 1 --2- 172.21.15.38:0/992640385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7940d0f60 0x7ff7940b2370 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7ff7940b5d50 tx=0x7ff78c02a5f0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.033+0000 7ff79aab3640 1 -- 172.21.15.38:0/992640385 shutdown_connections 2024-04-25T08:35:27.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.033+0000 7ff79aab3640 1 --2- 172.21.15.38:0/992640385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7940d0f60 0x7ff7940b2370 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.033+0000 7ff79aab3640 1 --2- 172.21.15.38:0/992640385 >> v2:172.21.15.107:3300/0 conn(0x7ff79415dc10 0x7ff7940ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.033+0000 7ff79aab3640 1 -- 172.21.15.38:0/992640385 >> 172.21.15.38:0/992640385 conn(0x7ff794066d10 msgr2=0x7ff794067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:27.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.033+0000 7ff79aab3640 1 -- 172.21.15.38:0/992640385 shutdown_connections 2024-04-25T08:35:27.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.033+0000 7ff79aab3640 1 -- 172.21.15.38:0/992640385 wait complete. 2024-04-25T08:35:27.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.034+0000 7ff79aab3640 1 Processor -- start 2024-04-25T08:35:27.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.034+0000 7ff79aab3640 1 -- start start 2024-04-25T08:35:27.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff79aab3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7940ced60 0x7ff7940ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff79aab3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7940d0f60 0x7ff79415bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff79aab3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79415dc10 0x7ff79415c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff79aab3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940d3900 con 0x7ff7940d0f60 2024-04-25T08:35:27.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff79aab3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940b6bc0 con 0x7ff79415dc10 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff79aab3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7940d3a80 con 0x7ff7940ced60 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff793fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7940d0f60 0x7ff79415bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff793fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7940d0f60 0x7ff79415bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56196/0 (socket says 172.21.15.38:56196) 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff793fff640 1 -- 172.21.15.38:0/2525701711 learned_addr learned my addr 172.21.15.38:0/2525701711 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff799050640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79415dc10 0x7ff79415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff79884f640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7940ced60 0x7ff7940ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff793fff640 1 -- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7940ced60 msgr2=0x7ff7940ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff793fff640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7940ced60 0x7ff7940ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff793fff640 1 -- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79415dc10 msgr2=0x7ff79415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff793fff640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79415dc10 0x7ff79415c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.035+0000 7ff793fff640 1 -- 172.21.15.38:0/2525701711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7941f4370 con 0x7ff7940d0f60 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.036+0000 7ff799050640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79415dc10 0x7ff79415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:27.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.036+0000 7ff793fff640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7940d0f60 0x7ff79415bdd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff78800c430 tx=0x7ff78800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.036+0000 7ff7917fa640 1 -- 172.21.15.38:0/2525701711 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7880074e0 con 0x7ff7940d0f60 2024-04-25T08:35:27.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.036+0000 7ff7917fa640 1 -- 172.21.15.38:0/2525701711 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7880047e0 con 0x7ff7940d0f60 2024-04-25T08:35:27.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.036+0000 7ff79aab3640 1 -- 172.21.15.38:0/2525701711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7941f4650 con 0x7ff7940d0f60 2024-04-25T08:35:27.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.036+0000 7ff7917fa640 1 -- 172.21.15.38:0/2525701711 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788004bd0 con 0x7ff7940d0f60 2024-04-25T08:35:27.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.036+0000 7ff79aab3640 1 -- 172.21.15.38:0/2525701711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7941f4c10 con 0x7ff7940d0f60 2024-04-25T08:35:27.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.037+0000 7ff79aab3640 1 -- 172.21.15.38:0/2525701711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff794068e80 con 0x7ff7940d0f60 2024-04-25T08:35:27.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.037+0000 7ff7917fa640 1 -- 172.21.15.38:0/2525701711 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff788011070 con 0x7ff7940d0f60 2024-04-25T08:35:27.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.038+0000 7ff7917fa640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff78006b4d0 0x7ff78006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.038+0000 7ff7917fa640 1 -- 172.21.15.38:0/2525701711 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff78808a0a0 con 0x7ff7940d0f60 2024-04-25T08:35:27.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.038+0000 7ff79884f640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff78006b4d0 0x7ff78006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.039+0000 7ff79884f640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff78006b4d0 0x7ff78006d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff784009a10 tx=0x7ff784008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.042+0000 7ff7917fa640 1 -- 172.21.15.38:0/2525701711 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff788058d40 con 0x7ff7940d0f60 2024-04-25T08:35:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.263+0000 7ff79aab3640 1 -- 172.21.15.38:0/2525701711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7941f4de0 con 0x7ff7940d0f60 2024-04-25T08:35:27.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.263+0000 7ff7917fa640 1 -- 172.21.15.38:0/2525701711 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4939 (secure 0 0 0) 0x7ff788058640 con 0x7ff7940d0f60 2024-04-25T08:35:27.258 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:27.258 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:35:24.877680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:27.259 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-04-25T08:35:27.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.265+0000 7ff79aab3640 1 -- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff78006b4d0 msgr2=0x7ff78006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.265+0000 7ff79aab3640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff78006b4d0 0x7ff78006d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7ff784009a10 tx=0x7ff784008040 comp rx=0 tx=0).stop 2024-04-25T08:35:27.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.265+0000 7ff79aab3640 1 -- 172.21.15.38:0/2525701711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7940d0f60 msgr2=0x7ff79415bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.265+0000 7ff79aab3640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7940d0f60 0x7ff79415bdd0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7ff78800c430 tx=0x7ff78800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.265+0000 7ff79aab3640 1 -- 172.21.15.38:0/2525701711 shutdown_connections 2024-04-25T08:35:27.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.265+0000 7ff79aab3640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79415dc10 0x7ff79415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.265+0000 7ff79aab3640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7940d0f60 0x7ff79415bdd0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.265+0000 7ff79aab3640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff78006b4d0 0x7ff78006d990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.265+0000 7ff79aab3640 1 --2- 172.21.15.38:0/2525701711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7940ced60 0x7ff7940ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.265+0000 7ff79aab3640 1 -- 172.21.15.38:0/2525701711 >> 172.21.15.38:0/2525701711 conn(0x7ff794066d10 msgr2=0x7ff79415e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:27.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.265+0000 7ff79aab3640 1 -- 172.21.15.38:0/2525701711 shutdown_connections 2024-04-25T08:35:27.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.265+0000 7ff79aab3640 1 -- 172.21.15.38:0/2525701711 wait complete. 2024-04-25T08:35:27.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:27.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.377+0000 7f32d3a37640 1 Processor -- start 2024-04-25T08:35:27.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.377+0000 7f32d3a37640 1 -- start start 2024-04-25T08:35:27.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.378+0000 7f32d3a37640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f32cc0b34e0 0x7f32cc0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.378+0000 7f32d3a37640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32cc06b1a0 con 0x7f32cc06c240 2024-04-25T08:35:27.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.378+0000 7f32d3a37640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32cc06b480 con 0x7f32cc15da40 2024-04-25T08:35:27.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.378+0000 7f32d3a37640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32cc06b760 con 0x7f32cc0b34e0 2024-04-25T08:35:27.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.378+0000 7f32d0fd2640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f32cc0b34e0 0x7f32cc0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.378+0000 7f32d1fd4640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f32cc06c240 0x7f32cc0ad2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53374/0 (socket says 172.21.15.38:53374) 2024-04-25T08:35:27.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.378+0000 7f32d1fd4640 1 -- 172.21.15.38:0/2753693287 learned_addr learned my addr 172.21.15.38:0/2753693287 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:27.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.379+0000 7f32d0fd2640 1 -- 172.21.15.38:0/2753693287 >> v1:172.21.15.107:6789/0 conn(0x7f32cc15da40 legacy=0x7f32cc0b2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:27.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.379+0000 7f32d0fd2640 1 -- 172.21.15.38:0/2753693287 >> v1:172.21.15.38:6789/0 conn(0x7f32cc06c240 legacy=0x7f32cc0ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.379+0000 7f32d0fd2640 1 -- 172.21.15.38:0/2753693287 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc06ba40 con 0x7f32cc0b34e0 2024-04-25T08:35:27.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.379+0000 7f32d0fd2640 1 --2- 172.21.15.38:0/2753693287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32cc0b34e0 0x7f32cc0acdb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f32c00097a0 tx=0x7f32c002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c596195aec604078 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.380+0000 7f32cbfff640 1 -- 172.21.15.38:0/2753693287 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0037070 con 0x7f32cc0b34e0 2024-04-25T08:35:27.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.380+0000 7f32cbfff640 1 -- 172.21.15.38:0/2753693287 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c002ae50 con 0x7f32cc0b34e0 2024-04-25T08:35:27.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.380+0000 7f32cbfff640 1 -- 172.21.15.38:0/2753693287 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0032660 con 0x7f32cc0b34e0 2024-04-25T08:35:27.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.380+0000 7f32d3a37640 1 -- 172.21.15.38:0/2753693287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32cc0b34e0 msgr2=0x7f32cc0acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.380+0000 7f32d3a37640 1 --2- 172.21.15.38:0/2753693287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32cc0b34e0 0x7f32cc0acdb0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f32c00097a0 tx=0x7f32c002a140 comp rx=0 tx=0).stop 2024-04-25T08:35:27.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.380+0000 7f32d3a37640 1 -- 172.21.15.38:0/2753693287 shutdown_connections 2024-04-25T08:35:27.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.380+0000 7f32d3a37640 1 --2- 172.21.15.38:0/2753693287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32cc0b34e0 0x7f32cc0acdb0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.380+0000 7f32d3a37640 1 -- 172.21.15.38:0/2753693287 >> 172.21.15.38:0/2753693287 conn(0x7f32cc0cfdc0 msgr2=0x7f32cc0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:27.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.380+0000 7f32d3a37640 1 -- 172.21.15.38:0/2753693287 shutdown_connections 2024-04-25T08:35:27.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.380+0000 7f32d3a37640 1 -- 172.21.15.38:0/2753693287 wait complete. 2024-04-25T08:35:27.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.381+0000 7f32d3a37640 1 Processor -- start 2024-04-25T08:35:27.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.381+0000 7f32d3a37640 1 -- start start 2024-04-25T08:35:27.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.382+0000 7f32d3a37640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc06c240 0x7f32cc1e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.382+0000 7f32d3a37640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32cc0b34e0 0x7f32cc154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.382+0000 7f32d3a37640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32cc15da40 0x7f32cc0ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.382+0000 7f32d3a37640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0b3ac0 con 0x7f32cc0b34e0 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.382+0000 7f32d3a37640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc15de20 con 0x7f32cc06c240 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.382+0000 7f32d3a37640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc0b3c40 con 0x7f32cc15da40 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.382+0000 7f32d0fd2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32cc0b34e0 0x7f32cc154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.382+0000 7f32d0fd2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32cc0b34e0 0x7f32cc154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56214/0 (socket says 172.21.15.38:56214) 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.382+0000 7f32d0fd2640 1 -- 172.21.15.38:0/3852332018 learned_addr learned my addr 172.21.15.38:0/3852332018 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.382+0000 7f32d17d3640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc06c240 0x7f32cc1e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.382+0000 7f32d1fd4640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32cc15da40 0x7f32cc0ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.383+0000 7f32d0fd2640 1 -- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32cc15da40 msgr2=0x7f32cc0ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.383+0000 7f32d0fd2640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32cc15da40 0x7f32cc0ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.383+0000 7f32d0fd2640 1 -- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc06c240 msgr2=0x7f32cc1e7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.383+0000 7f32d0fd2640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc06c240 0x7f32cc1e7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.383+0000 7f32d0fd2640 1 -- 172.21.15.38:0/3852332018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc0cde00 con 0x7f32cc0b34e0 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.383+0000 7f32d17d3640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc06c240 0x7f32cc1e7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.383+0000 7f32d0fd2640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32cc0b34e0 0x7f32cc154bb0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f32c0000c00 tx=0x7f32c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.383+0000 7f32c9ffb640 1 -- 172.21.15.38:0/3852332018 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0041070 con 0x7f32cc0b34e0 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.383+0000 7f32c9ffb640 1 -- 172.21.15.38:0/3852332018 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c003c670 con 0x7f32cc0b34e0 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.383+0000 7f32c9ffb640 1 -- 172.21.15.38:0/3852332018 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0037040 con 0x7f32cc0b34e0 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.383+0000 7f32d3a37640 1 -- 172.21.15.38:0/3852332018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32cc1e8710 con 0x7f32cc0b34e0 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.383+0000 7f32d3a37640 1 -- 172.21.15.38:0/3852332018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32cc1e8cd0 con 0x7f32cc0b34e0 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.384+0000 7f32d3a37640 1 -- 172.21.15.38:0/3852332018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32cc1613c0 con 0x7f32cc0b34e0 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.384+0000 7f32c9ffb640 1 -- 172.21.15.38:0/3852332018 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32c0044070 con 0x7f32cc0b34e0 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.385+0000 7f32c9ffb640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32b806b4d0 0x7f32b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.385+0000 7f32c9ffb640 1 -- 172.21.15.38:0/3852332018 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f32c00aaab0 con 0x7f32cc0b34e0 2024-04-25T08:35:27.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.385+0000 7f32d17d3640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32b806b4d0 0x7f32b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.386+0000 7f32d17d3640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32b806b4d0 0x7f32b806d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009fc0 tx=0x7f32c4009550 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.389+0000 7f32c9ffb640 1 -- 172.21.15.38:0/3852332018 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32cc0cde00 con 0x7f32cc0b34e0 2024-04-25T08:35:27.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.570+0000 7f32d3a37640 1 -- 172.21.15.38:0/3852332018 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32cc0af150 con 0x7f32cc0b34e0 2024-04-25T08:35:27.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.571+0000 7f32c9ffb640 1 -- 172.21.15.38:0/3852332018 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+17026 (secure 0 0 0) 0x7f32c0079800 con 0x7f32cc0b34e0 2024-04-25T08:35:27.566 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:27.567 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":56,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:18.124795+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:34:42.507595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:34:43.209205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.28","mappings":[{"from":0,"to":6}]},{"pgid":"8.2f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.573+0000 7f32d3a37640 1 -- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32b806b4d0 msgr2=0x7f32b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.573+0000 7f32d3a37640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32b806b4d0 0x7f32b806d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009fc0 tx=0x7f32c4009550 comp rx=0 tx=0).stop 2024-04-25T08:35:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.573+0000 7f32d3a37640 1 -- 172.21.15.38:0/3852332018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32cc0b34e0 msgr2=0x7f32cc154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.573+0000 7f32d3a37640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32cc0b34e0 0x7f32cc154bb0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f32c0000c00 tx=0x7f32c00046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.574+0000 7f32d3a37640 1 -- 172.21.15.38:0/3852332018 shutdown_connections 2024-04-25T08:35:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.574+0000 7f32d3a37640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32b806b4d0 0x7f32b806d990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.574+0000 7f32d3a37640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32cc15da40 0x7f32cc0ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.574+0000 7f32d3a37640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32cc0b34e0 0x7f32cc154bb0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.574+0000 7f32d3a37640 1 --2- 172.21.15.38:0/3852332018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32cc06c240 0x7f32cc1e7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.574+0000 7f32d3a37640 1 -- 172.21.15.38:0/3852332018 >> 172.21.15.38:0/3852332018 conn(0x7f32cc0cfdc0 msgr2=0x7f32cc15c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:27.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.574+0000 7f32d3a37640 1 -- 172.21.15.38:0/3852332018 shutdown_connections 2024-04-25T08:35:27.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.574+0000 7f32d3a37640 1 -- 172.21.15.38:0/3852332018 wait complete. 2024-04-25T08:35:27.579 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:35:27.580 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:27.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.686+0000 7fbb9c6fc640 1 Processor -- start 2024-04-25T08:35:27.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.686+0000 7fbb9c6fc640 1 -- start start 2024-04-25T08:35:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.687+0000 7fbb9c6fc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbb940b4ea0 0x7fbb940cedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.687+0000 7fbb9c6fc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb9415e700 con 0x7fbb9415d930 2024-04-25T08:35:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.687+0000 7fbb9c6fc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb940cbc00 con 0x7fbb9415a340 2024-04-25T08:35:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.687+0000 7fbb9c6fc640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb940b4a10 con 0x7fbb940b4ea0 2024-04-25T08:35:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.687+0000 7fbb9a498640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbb9415d930 0x7fbb9415dd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53386/0 (socket says 172.21.15.38:53386) 2024-04-25T08:35:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.687+0000 7fbb9a498640 1 -- 172.21.15.38:0/1299508774 learned_addr learned my addr 172.21.15.38:0/1299508774 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.687+0000 7fbb99c97640 1 --2- 172.21.15.38:0/1299508774 >> v2:172.21.15.107:3300/0 conn(0x7fbb940b4ea0 0x7fbb940cedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192032229 0 0) 0x7fbb9415e700 con 0x7fbb9415d930 2024-04-25T08:35:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb80003520 con 0x7fbb9415d930 2024-04-25T08:35:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3730895726 0 0) 0x7fbb80003520 con 0x7fbb9415d930 2024-04-25T08:35:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 >> v2:172.21.15.107:3300/0 conn(0x7fbb940b4ea0 msgr2=0x7fbb940cedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 --2- 172.21.15.38:0/1299508774 >> v2:172.21.15.107:3300/0 conn(0x7fbb940b4ea0 0x7fbb940cedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 >> v1:172.21.15.107:6789/0 conn(0x7fbb9415a340 legacy=0x7fbb9415a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb940b43c0 con 0x7fbb9415d930 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fbb8c001670 con 0x7fbb9415d930 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 >> v1:172.21.15.38:6789/0 conn(0x7fbb9415d930 legacy=0x7fbb9415dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 --2- 172.21.15.38:0/1299508774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800039b0 0x7fbb80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 --2- 172.21.15.38:0/1299508774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb80004800 0x7fbb80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 --2- 172.21.15.38:0/1299508774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800051c0 0x7fbb8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940b4a10 con 0x7fbb800039b0 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8000c060 con 0x7fbb80004800 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8000c260 con 0x7fbb800051c0 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99c97640 1 --2- 172.21.15.38:0/1299508774 >> v2:172.21.15.107:3300/0 conn(0x7fbb940b4ea0 0x7fbb940cedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:27.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb9a498640 1 --2- 172.21.15.38:0/1299508774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800039b0 0x7fbb80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb9ac99640 1 --2- 172.21.15.38:0/1299508774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800051c0 0x7fbb8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb99c97640 1 --2- 172.21.15.38:0/1299508774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb80004800 0x7fbb80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb9a498640 1 -- 172.21.15.38:0/1299508774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800051c0 msgr2=0x7fbb8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb9a498640 1 --2- 172.21.15.38:0/1299508774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800051c0 0x7fbb8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb9a498640 1 -- 172.21.15.38:0/1299508774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb80004800 msgr2=0x7fbb80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb9a498640 1 --2- 172.21.15.38:0/1299508774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb80004800 0x7fbb80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb9a498640 1 -- 172.21.15.38:0/1299508774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb80003520 con 0x7fbb800039b0 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.688+0000 7fbb9ac99640 1 -- 172.21.15.38:0/1299508774 reap_dead start 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.689+0000 7fbb9a498640 1 --2- 172.21.15.38:0/1299508774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800039b0 0x7fbb80003dd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00c240 tx=0x7fbb80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ca53f92d771210 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.689+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c007cf0 con 0x7fbb800039b0 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.689+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c0058d0 con 0x7fbb800039b0 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.689+0000 7fbb99496640 1 -- 172.21.15.38:0/1299508774 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c005d20 con 0x7fbb800039b0 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.690+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/1299508774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800039b0 msgr2=0x7fbb80003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.690+0000 7fbb9c6fc640 1 --2- 172.21.15.38:0/1299508774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800039b0 0x7fbb80003dd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c00c240 tx=0x7fbb80005830 comp rx=0 tx=0).stop 2024-04-25T08:35:27.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.690+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/1299508774 shutdown_connections 2024-04-25T08:35:27.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.690+0000 7fbb9c6fc640 1 --2- 172.21.15.38:0/1299508774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb800039b0 0x7fbb80003dd0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.690+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/1299508774 >> 172.21.15.38:0/1299508774 conn(0x7fbb9406b210 msgr2=0x7fbb9406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:27.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.690+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/1299508774 shutdown_connections 2024-04-25T08:35:27.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.690+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/1299508774 wait complete. 2024-04-25T08:35:27.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.691+0000 7fbb9c6fc640 1 Processor -- start 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.691+0000 7fbb9c6fc640 1 -- start start 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9c6fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800039b0 0x7fbb941405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9c6fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb9413dd40 0x7fbb940c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9c6fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb94070fd0 0x7fbb94071450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9c6fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940cb840 con 0x7fbb94070fd0 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9c6fc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940cbee0 con 0x7fbb9413dd40 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9c6fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940cb9c0 con 0x7fbb800039b0 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9ac99640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb94070fd0 0x7fbb94071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9ac99640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb94070fd0 0x7fbb94071450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56236/0 (socket says 172.21.15.38:56236) 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9ac99640 1 -- 172.21.15.38:0/3305971163 learned_addr learned my addr 172.21.15.38:0/3305971163 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb99c97640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb9413dd40 0x7fbb940c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9a498640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800039b0 0x7fbb941405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9ac99640 1 -- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800039b0 msgr2=0x7fbb941405d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9ac99640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800039b0 0x7fbb941405d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9ac99640 1 -- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb9413dd40 msgr2=0x7fbb940c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9ac99640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb9413dd40 0x7fbb940c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9ac99640 1 -- 172.21.15.38:0/3305971163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb940c6850 con 0x7fbb94070fd0 2024-04-25T08:35:27.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9a498640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800039b0 0x7fbb941405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:27.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9ac99640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb94070fd0 0x7fbb94071450 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbb90007fd0 tx=0x7fbb9000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb8affd640 1 -- 172.21.15.38:0/3305971163 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90003970 con 0x7fbb94070fd0 2024-04-25T08:35:27.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb8affd640 1 -- 172.21.15.38:0/3305971163 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb90018070 con 0x7fbb94070fd0 2024-04-25T08:35:27.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/3305971163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb94140e40 con 0x7fbb94070fd0 2024-04-25T08:35:27.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb8affd640 1 -- 172.21.15.38:0/3305971163 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90003d70 con 0x7fbb94070fd0 2024-04-25T08:35:27.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.692+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/3305971163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb941435f0 con 0x7fbb94070fd0 2024-04-25T08:35:27.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.694+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/3305971163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb94068e80 con 0x7fbb94070fd0 2024-04-25T08:35:27.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.694+0000 7fbb8affd640 1 -- 172.21.15.38:0/3305971163 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb900050d0 con 0x7fbb94070fd0 2024-04-25T08:35:27.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.694+0000 7fbb8affd640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb7806b4d0 0x7fbb7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:27.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.694+0000 7fbb8affd640 1 -- 172.21.15.38:0/3305971163 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fbb90089fa0 con 0x7fbb94070fd0 2024-04-25T08:35:27.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.694+0000 7fbb9a498640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb7806b4d0 0x7fbb7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:27.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.695+0000 7fbb9a498640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb7806b4d0 0x7fbb7806d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c033000 tx=0x7fbb8c009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:27.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.698+0000 7fbb8affd640 1 -- 172.21.15.38:0/3305971163 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb940c6850 con 0x7fbb94070fd0 2024-04-25T08:35:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.915+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/3305971163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb94141010 con 0x7fbb94070fd0 2024-04-25T08:35:27.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.916+0000 7fbb8affd640 1 -- 172.21.15.38:0/3305971163 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fbb90058bc0 con 0x7fbb94070fd0 2024-04-25T08:35:27.911 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:27.911 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:35:24.877680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:27.911 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-04-25T08:35:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.918+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb7806b4d0 msgr2=0x7fbb7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.918+0000 7fbb9c6fc640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb7806b4d0 0x7fbb7806d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c033000 tx=0x7fbb8c009030 comp rx=0 tx=0).stop 2024-04-25T08:35:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.918+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/3305971163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb94070fd0 msgr2=0x7fbb94071450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.918+0000 7fbb9c6fc640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb94070fd0 0x7fbb94071450 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbb90007fd0 tx=0x7fbb9000eb90 comp rx=0 tx=0).stop 2024-04-25T08:35:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.918+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/3305971163 shutdown_connections 2024-04-25T08:35:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.918+0000 7fbb9c6fc640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb94070fd0 0x7fbb94071450 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.918+0000 7fbb9c6fc640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbb9413dd40 0x7fbb940c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.918+0000 7fbb9c6fc640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb7806b4d0 0x7fbb7806d990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.918+0000 7fbb9c6fc640 1 --2- 172.21.15.38:0/3305971163 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbb800039b0 0x7fbb941405d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.918+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/3305971163 >> 172.21.15.38:0/3305971163 conn(0x7fbb9406b210 msgr2=0x7fbb940b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:27.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.918+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/3305971163 shutdown_connections 2024-04-25T08:35:27.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:27.918+0000 7fbb9c6fc640 1 -- 172.21.15.38:0/3305971163 wait complete. 2024-04-25T08:35:27.924 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:28.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.030+0000 7f971485a640 1 Processor -- start 2024-04-25T08:35:28.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.030+0000 7f971485a640 1 -- start start 2024-04-25T08:35:28.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.030+0000 7f971485a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f971015dbc0 0x7f9710072b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f971485a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f97100ca920 0x7f97100b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f971485a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971015dfa0 con 0x7f97100b2220 2024-04-25T08:35:28.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f971485a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97100b2bc0 con 0x7f97100ca920 2024-04-25T08:35:28.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f971485a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97100b3970 con 0x7f971015dbc0 2024-04-25T08:35:28.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f970ed9d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f97100ca920 0x7f97100b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f970e59c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f971015dbc0 0x7f9710072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f970ed9d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f97100ca920 0x7f97100b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:41594/0 (socket says 172.21.15.38:41594) 2024-04-25T08:35:28.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f970ed9d640 1 -- 172.21.15.38:0/1969714331 learned_addr learned my addr 172.21.15.38:0/1969714331 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:28.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f9705d9b640 1 --1- 172.21.15.38:0/1969714331 >> v1:172.21.15.107:6789/0 conn(0x7f97100b2220 0x7f971006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:36042/0 (socket says 172.21.15.38:36042) 2024-04-25T08:35:28.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f970ed9d640 1 -- 172.21.15.38:0/1969714331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f971015dbc0 msgr2=0x7f9710072b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:28.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f970ed9d640 1 --2- 172.21.15.38:0/1969714331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f971015dbc0 0x7f9710072b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f970ed9d640 1 -- 172.21.15.38:0/1969714331 >> v1:172.21.15.107:6789/0 conn(0x7f97100b2220 legacy=0x7f971006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.031+0000 7f970ed9d640 1 -- 172.21.15.38:0/1969714331 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97100b3c50 con 0x7f97100ca920 2024-04-25T08:35:28.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.032+0000 7f970ed9d640 1 --2- 172.21.15.38:0/1969714331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97100ca920 0x7f97100b2680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9708009e50 tx=0x7f970802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87793d403365cb01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.032+0000 7f970dd9b640 1 -- 172.21.15.38:0/1969714331 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9708036070 con 0x7f97100ca920 2024-04-25T08:35:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.032+0000 7f970dd9b640 1 -- 172.21.15.38:0/1969714331 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9708031600 con 0x7f97100ca920 2024-04-25T08:35:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.032+0000 7f970dd9b640 1 -- 172.21.15.38:0/1969714331 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97080319d0 con 0x7f97100ca920 2024-04-25T08:35:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.033+0000 7f971485a640 1 -- 172.21.15.38:0/1969714331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97100ca920 msgr2=0x7f97100b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.033+0000 7f971485a640 1 --2- 172.21.15.38:0/1969714331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97100ca920 0x7f97100b2680 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9708009e50 tx=0x7f970802a730 comp rx=0 tx=0).stop 2024-04-25T08:35:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.033+0000 7f971485a640 1 -- 172.21.15.38:0/1969714331 shutdown_connections 2024-04-25T08:35:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.033+0000 7f971485a640 1 --2- 172.21.15.38:0/1969714331 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97100ca920 0x7f97100b2680 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.033+0000 7f971485a640 1 --2- 172.21.15.38:0/1969714331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f971015dbc0 0x7f9710072b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.033+0000 7f971485a640 1 -- 172.21.15.38:0/1969714331 >> 172.21.15.38:0/1969714331 conn(0x7f97100b5c90 msgr2=0x7f97100b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.033+0000 7f971485a640 1 -- 172.21.15.38:0/1969714331 shutdown_connections 2024-04-25T08:35:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.033+0000 7f971485a640 1 -- 172.21.15.38:0/1969714331 wait complete. 2024-04-25T08:35:28.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.034+0000 7f971485a640 1 Processor -- start 2024-04-25T08:35:28.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.034+0000 7f971485a640 1 -- start start 2024-04-25T08:35:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f971485a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97100b2220 0x7f971014f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f971485a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97100ca920 0x7f97100cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f971485a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f971015dbc0 0x7f971014cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f971485a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97100b44f0 con 0x7f97100ca920 2024-04-25T08:35:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f971485a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97100b4210 con 0x7f97100b2220 2024-04-25T08:35:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f971485a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97100b3590 con 0x7f971015dbc0 2024-04-25T08:35:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f9705d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97100ca920 0x7f97100cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f9705d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97100ca920 0x7f97100cf5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56244/0 (socket says 172.21.15.38:56244) 2024-04-25T08:35:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f9705d9b640 1 -- 172.21.15.38:0/2520710718 learned_addr learned my addr 172.21.15.38:0/2520710718 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f970ed9d640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f971015dbc0 0x7f971014cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f970e59c640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97100b2220 0x7f971014f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f9705d9b640 1 -- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f971015dbc0 msgr2=0x7f971014cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f9705d9b640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f971015dbc0 0x7f971014cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f9705d9b640 1 -- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97100b2220 msgr2=0x7f971014f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f9705d9b640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97100b2220 0x7f971014f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.035+0000 7f9705d9b640 1 -- 172.21.15.38:0/2520710718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97100cff40 con 0x7f97100ca920 2024-04-25T08:35:28.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.036+0000 7f9705d9b640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97100ca920 0x7f97100cf5a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c430 tx=0x7f96f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.036+0000 7f97077fe640 1 -- 172.21.15.38:0/2520710718 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f80074e0 con 0x7f97100ca920 2024-04-25T08:35:28.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.036+0000 7f97077fe640 1 -- 172.21.15.38:0/2520710718 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f8007680 con 0x7f97100ca920 2024-04-25T08:35:28.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.036+0000 7f971485a640 1 -- 172.21.15.38:0/2520710718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97100d01e0 con 0x7f97100ca920 2024-04-25T08:35:28.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.036+0000 7f97077fe640 1 -- 172.21.15.38:0/2520710718 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f8004a00 con 0x7f97100ca920 2024-04-25T08:35:28.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.036+0000 7f971485a640 1 -- 172.21.15.38:0/2520710718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97101608a0 con 0x7f97100ca920 2024-04-25T08:35:28.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.037+0000 7f971485a640 1 -- 172.21.15.38:0/2520710718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97100afce0 con 0x7f97100ca920 2024-04-25T08:35:28.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.037+0000 7f97077fe640 1 -- 172.21.15.38:0/2520710718 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96f801a020 con 0x7f97100ca920 2024-04-25T08:35:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.038+0000 7f97077fe640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f96f006b4d0 0x7f96f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.038+0000 7f97077fe640 1 -- 172.21.15.38:0/2520710718 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f96f8089a70 con 0x7f97100ca920 2024-04-25T08:35:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.038+0000 7f970e59c640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f96f006b4d0 0x7f96f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.038+0000 7f970e59c640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f96f006b4d0 0x7f96f006d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9700009aa0 tx=0x7f9700009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.042+0000 7f97077fe640 1 -- 172.21.15.38:0/2520710718 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97100afce0 con 0x7f97100ca920 2024-04-25T08:35:28.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.261+0000 7f971485a640 1 -- 172.21.15.38:0/2520710718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97100d03b0 con 0x7f97100ca920 2024-04-25T08:35:28.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.262+0000 7f97077fe640 1 -- 172.21.15.38:0/2520710718 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f97100d03b0 con 0x7f97100ca920 2024-04-25T08:35:28.256 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:28.257 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:34:44.235451+0000","modified":"2024-04-25T08:35:24.877680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5964},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5964":{"gid":5964,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":13,"addr":"172.21.15.38:6834/226686241","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":226686241},{"type":"v1","addr":"172.21.15.38:6834","nonce":226686241}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:35:28.257 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 28 2024-04-25T08:35:28.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.263+0000 7f971485a640 1 -- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f96f006b4d0 msgr2=0x7f96f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.263+0000 7f971485a640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f96f006b4d0 0x7f96f006d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9700009aa0 tx=0x7f9700009190 comp rx=0 tx=0).stop 2024-04-25T08:35:28.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.263+0000 7f971485a640 1 -- 172.21.15.38:0/2520710718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97100ca920 msgr2=0x7f97100cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.263+0000 7f971485a640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97100ca920 0x7f97100cf5a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f96f800c430 tx=0x7f96f800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.263+0000 7f971485a640 1 -- 172.21.15.38:0/2520710718 shutdown_connections 2024-04-25T08:35:28.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.263+0000 7f971485a640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f96f006b4d0 0x7f96f006d990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.263+0000 7f971485a640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f971015dbc0 0x7f971014cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.263+0000 7f971485a640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97100ca920 0x7f97100cf5a0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.263+0000 7f971485a640 1 --2- 172.21.15.38:0/2520710718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97100b2220 0x7f971014f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.264+0000 7f971485a640 1 -- 172.21.15.38:0/2520710718 >> 172.21.15.38:0/2520710718 conn(0x7f97100b5c90 msgr2=0x7f97100b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:28.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.264+0000 7f971485a640 1 -- 172.21.15.38:0/2520710718 shutdown_connections 2024-04-25T08:35:28.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.264+0000 7f971485a640 1 -- 172.21.15.38:0/2520710718 wait complete. 2024-04-25T08:35:28.269 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:28.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.375+0000 7f47e4e3f640 1 Processor -- start 2024-04-25T08:35:28.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.375+0000 7f47e4e3f640 1 -- start start 2024-04-25T08:35:28.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.376+0000 7f47e4e3f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47e006cff0 0x7f47e0099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.376+0000 7f47e4e3f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f47e0099fb0 0x7f47e015d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.376+0000 7f47e4e3f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e009a3c0 con 0x7f47e006cc10 2024-04-25T08:35:28.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.376+0000 7f47e4e3f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e00d05c0 con 0x7f47e006cff0 2024-04-25T08:35:28.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.376+0000 7f47e4e3f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47e00d08a0 con 0x7f47e0099fb0 2024-04-25T08:35:28.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.376+0000 7f47d7fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47e006cff0 0x7f47e0099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.376+0000 7f47d7fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47e006cff0 0x7f47e0099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56246/0 (socket says 172.21.15.38:56246) 2024-04-25T08:35:28.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.376+0000 7f47d7fff640 1 -- 172.21.15.38:0/2712556030 learned_addr learned my addr 172.21.15.38:0/2712556030 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:28.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.377+0000 7f47ded9d640 1 --2- 172.21.15.38:0/2712556030 >> v2:172.21.15.107:3300/0 conn(0x7f47e0099fb0 0x7f47e015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.377+0000 7f47d7fff640 1 -- 172.21.15.38:0/2712556030 >> v2:172.21.15.107:3300/0 conn(0x7f47e0099fb0 msgr2=0x7f47e015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:28.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.377+0000 7f47d7fff640 1 --2- 172.21.15.38:0/2712556030 >> v2:172.21.15.107:3300/0 conn(0x7f47e0099fb0 0x7f47e015d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.377+0000 7f47d7fff640 1 -- 172.21.15.38:0/2712556030 >> v1:172.21.15.107:6789/0 conn(0x7f47e006cc10 legacy=0x7f47e00cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:28.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.377+0000 7f47d7fff640 1 -- 172.21.15.38:0/2712556030 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e00d0b80 con 0x7f47e006cff0 2024-04-25T08:35:28.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.377+0000 7f47d7fff640 1 --2- 172.21.15.38:0/2712556030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e006cff0 0x7f47e0099a70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f47e0161700 tx=0x7f47c802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13aa34eec849efe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.378+0000 7f47ddd9b640 1 -- 172.21.15.38:0/2712556030 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8037070 con 0x7f47e006cff0 2024-04-25T08:35:28.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.378+0000 7f47ddd9b640 1 -- 172.21.15.38:0/2712556030 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c8032440 con 0x7f47e006cff0 2024-04-25T08:35:28.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.378+0000 7f47ddd9b640 1 -- 172.21.15.38:0/2712556030 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8032810 con 0x7f47e006cff0 2024-04-25T08:35:28.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.378+0000 7f47e4e3f640 1 -- 172.21.15.38:0/2712556030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e006cff0 msgr2=0x7f47e0099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.378+0000 7f47e4e3f640 1 --2- 172.21.15.38:0/2712556030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e006cff0 0x7f47e0099a70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f47e0161700 tx=0x7f47c802a230 comp rx=0 tx=0).stop 2024-04-25T08:35:28.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.378+0000 7f47e4e3f640 1 -- 172.21.15.38:0/2712556030 shutdown_connections 2024-04-25T08:35:28.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.378+0000 7f47e4e3f640 1 --2- 172.21.15.38:0/2712556030 >> v2:172.21.15.107:3300/0 conn(0x7f47e0099fb0 0x7f47e015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.378+0000 7f47e4e3f640 1 --2- 172.21.15.38:0/2712556030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e006cff0 0x7f47e0099a70 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.378+0000 7f47e4e3f640 1 -- 172.21.15.38:0/2712556030 >> 172.21.15.38:0/2712556030 conn(0x7f47e00b31f0 msgr2=0x7f47e00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:28.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.378+0000 7f47e4e3f640 1 -- 172.21.15.38:0/2712556030 shutdown_connections 2024-04-25T08:35:28.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.378+0000 7f47e4e3f640 1 -- 172.21.15.38:0/2712556030 wait complete. 2024-04-25T08:35:28.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.380+0000 7f47e4e3f640 1 Processor -- start 2024-04-25T08:35:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.380+0000 7f47e4e3f640 1 -- start start 2024-04-25T08:35:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.380+0000 7f47e4e3f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47e006cc10 0x7f47e0158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.380+0000 7f47e4e3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e006cff0 0x7f47e00b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.380+0000 7f47e4e3f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47e0099fb0 0x7f47e00b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.380+0000 7f47e4e3f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e006c150 con 0x7f47e006cff0 2024-04-25T08:35:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.380+0000 7f47e4e3f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e00d1140 con 0x7f47e006cc10 2024-04-25T08:35:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.380+0000 7f47e4e3f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e006c2d0 con 0x7f47e0099fb0 2024-04-25T08:35:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.380+0000 7f47d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e006cff0 0x7f47e00b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.380+0000 7f47d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e006cff0 0x7f47e00b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56260/0 (socket says 172.21.15.38:56260) 2024-04-25T08:35:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.380+0000 7f47d7fff640 1 -- 172.21.15.38:0/498683909 learned_addr learned my addr 172.21.15.38:0/498683909 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47de59c640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47e006cc10 0x7f47e0158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47ded9d640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47e0099fb0 0x7f47e00b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47d7fff640 1 -- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47e0099fb0 msgr2=0x7f47e00b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47d7fff640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47e0099fb0 0x7f47e00b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47d7fff640 1 -- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47e006cc10 msgr2=0x7f47e0158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47d7fff640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47e006cc10 0x7f47e0158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47d7fff640 1 -- 172.21.15.38:0/498683909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0158d70 con 0x7f47e006cff0 2024-04-25T08:35:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47d7fff640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e006cff0 0x7f47e00b1d40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f47c8000c00 tx=0x7f47c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47d6ffd640 1 -- 172.21.15.38:0/498683909 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8041020 con 0x7f47e006cff0 2024-04-25T08:35:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47d6ffd640 1 -- 172.21.15.38:0/498683909 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c80306c0 con 0x7f47e006cff0 2024-04-25T08:35:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47d6ffd640 1 -- 172.21.15.38:0/498683909 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8037040 con 0x7f47e006cff0 2024-04-25T08:35:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47e4e3f640 1 -- 172.21.15.38:0/498683909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e00b7120 con 0x7f47e006cff0 2024-04-25T08:35:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.381+0000 7f47e4e3f640 1 -- 172.21.15.38:0/498683909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e00b6b60 con 0x7f47e006cff0 2024-04-25T08:35:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.382+0000 7f47e4e3f640 1 -- 172.21.15.38:0/498683909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e0068e80 con 0x7f47e006cff0 2024-04-25T08:35:28.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.383+0000 7f47d6ffd640 1 -- 172.21.15.38:0/498683909 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47c8044020 con 0x7f47e006cff0 2024-04-25T08:35:28.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.383+0000 7f47d6ffd640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47c006b500 0x7f47c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.383+0000 7f47d6ffd640 1 -- 172.21.15.38:0/498683909 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f47c80aaa80 con 0x7f47e006cff0 2024-04-25T08:35:28.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.383+0000 7f47de59c640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47c006b500 0x7f47c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.384+0000 7f47de59c640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47c006b500 0x7f47c006d9c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f47d0004f60 tx=0x7f47d0007e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.387+0000 7f47d6ffd640 1 -- 172.21.15.38:0/498683909 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47c80ace20 con 0x7f47e006cff0 2024-04-25T08:35:28.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.572+0000 7f47e4e3f640 1 -- 172.21.15.38:0/498683909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47e00b6f00 con 0x7f47e006cff0 2024-04-25T08:35:28.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.573+0000 7f47d6ffd640 1 -- 172.21.15.38:0/498683909 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+17026 (secure 0 0 0) 0x7f47c804b030 con 0x7f47e006cff0 2024-04-25T08:35:28.568 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:28.568 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":56,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:18.124795+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:34:42.507595+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:34:43.209205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.28","mappings":[{"from":0,"to":6}]},{"pgid":"8.2f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:28.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.575+0000 7f47e4e3f640 1 -- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47c006b500 msgr2=0x7f47c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.575+0000 7f47e4e3f640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47c006b500 0x7f47c006d9c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f47d0004f60 tx=0x7f47d0007e30 comp rx=0 tx=0).stop 2024-04-25T08:35:28.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.575+0000 7f47e4e3f640 1 -- 172.21.15.38:0/498683909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e006cff0 msgr2=0x7f47e00b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.575+0000 7f47e4e3f640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e006cff0 0x7f47e00b1d40 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f47c8000c00 tx=0x7f47c80046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.575+0000 7f47e4e3f640 1 -- 172.21.15.38:0/498683909 shutdown_connections 2024-04-25T08:35:28.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.575+0000 7f47e4e3f640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f47c006b500 0x7f47c006d9c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.575+0000 7f47e4e3f640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47e0099fb0 0x7f47e00b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.575+0000 7f47e4e3f640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e006cff0 0x7f47e00b1d40 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.575+0000 7f47e4e3f640 1 --2- 172.21.15.38:0/498683909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47e006cc10 0x7f47e0158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.575+0000 7f47e4e3f640 1 -- 172.21.15.38:0/498683909 >> 172.21.15.38:0/498683909 conn(0x7f47e00b31f0 msgr2=0x7f47e00b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.575+0000 7f47e4e3f640 1 -- 172.21.15.38:0/498683909 shutdown_connections 2024-04-25T08:35:28.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.575+0000 7f47e4e3f640 1 -- 172.21.15.38:0/498683909 wait complete. 2024-04-25T08:35:28.581 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:35:28.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.688+0000 7f970672b640 1 Processor -- start 2024-04-25T08:35:28.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.688+0000 7f970672b640 1 -- start start 2024-04-25T08:35:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f970672b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f970015db00 0x7f9700071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f970672b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f970006f080 0x7f97000ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f970672b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97000cdc10 con 0x7f9700071d80 2024-04-25T08:35:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f970672b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970015dee0 con 0x7f970015db00 2024-04-25T08:35:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f970672b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970006bad0 con 0x7f970006f080 2024-04-25T08:35:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f96fffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f970015db00 0x7f9700071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f96ff7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9700071d80 0x7f970006e970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53394/0 (socket says 172.21.15.38:53394) 2024-04-25T08:35:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f96ff7fe640 1 -- 172.21.15.38:0/4136474156 learned_addr learned my addr 172.21.15.38:0/4136474156 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f96fffff640 1 -- 172.21.15.38:0/4136474156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f970006f080 msgr2=0x7f97000ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f9704cc8640 1 --2- 172.21.15.38:0/4136474156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f970006f080 0x7f97000ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f96fffff640 1 --2- 172.21.15.38:0/4136474156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f970006f080 0x7f97000ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f96fffff640 1 -- 172.21.15.38:0/4136474156 >> v1:172.21.15.38:6789/0 conn(0x7f9700071d80 legacy=0x7f970006e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f96fffff640 1 -- 172.21.15.38:0/4136474156 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970006f580 con 0x7f970015db00 2024-04-25T08:35:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.689+0000 7f96feffd640 1 -- 172.21.15.38:0/4136474156 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 45571570 0 0) 0x7f97000cdc10 con 0x7f9700071d80 2024-04-25T08:35:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.690+0000 7f96fffff640 1 --2- 172.21.15.38:0/4136474156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f970015db00 0x7f9700071840 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f96f40097a0 tx=0x7f96f402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1177e6d3e7c788da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.690+0000 7f96feffd640 1 -- 172.21.15.38:0/4136474156 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4037070 con 0x7f970015db00 2024-04-25T08:35:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.690+0000 7f96feffd640 1 -- 172.21.15.38:0/4136474156 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f4032440 con 0x7f970015db00 2024-04-25T08:35:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.690+0000 7f96feffd640 1 -- 172.21.15.38:0/4136474156 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4032810 con 0x7f970015db00 2024-04-25T08:35:28.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.691+0000 7f970672b640 1 -- 172.21.15.38:0/4136474156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f970015db00 msgr2=0x7f9700071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.691+0000 7f970672b640 1 --2- 172.21.15.38:0/4136474156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f970015db00 0x7f9700071840 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f96f40097a0 tx=0x7f96f402a180 comp rx=0 tx=0).stop 2024-04-25T08:35:28.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.691+0000 7f970672b640 1 -- 172.21.15.38:0/4136474156 shutdown_connections 2024-04-25T08:35:28.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.691+0000 7f970672b640 1 --2- 172.21.15.38:0/4136474156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f970006f080 0x7f97000ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.691+0000 7f970672b640 1 --2- 172.21.15.38:0/4136474156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f970015db00 0x7f9700071840 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.691+0000 7f970672b640 1 -- 172.21.15.38:0/4136474156 >> 172.21.15.38:0/4136474156 conn(0x7f97000b24c0 msgr2=0x7f97000cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:28.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.691+0000 7f970672b640 1 -- 172.21.15.38:0/4136474156 shutdown_connections 2024-04-25T08:35:28.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.691+0000 7f970672b640 1 -- 172.21.15.38:0/4136474156 wait complete. 2024-04-25T08:35:28.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.692+0000 7f970672b640 1 Processor -- start 2024-04-25T08:35:28.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.692+0000 7f970672b640 1 -- start start 2024-04-25T08:35:28.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f970672b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f970006f080 0x7f9700152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f970672b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9700071d80 0x7f97001f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f970672b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f970015db00 0x7f97001557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f970672b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000ce470 con 0x7f970006f080 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f970672b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000ce190 con 0x7f970015db00 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f970672b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97000ceed0 con 0x7f9700071d80 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f96fffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f970006f080 0x7f9700152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f96fffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f970006f080 0x7f9700152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56278/0 (socket says 172.21.15.38:56278) 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f96fffff640 1 -- 172.21.15.38:0/705543456 learned_addr learned my addr 172.21.15.38:0/705543456 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f9704cc8640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f970015db00 0x7f97001557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f96ff7fe640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9700071d80 0x7f97001f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f96fffff640 1 -- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9700071d80 msgr2=0x7f97001f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f96fffff640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9700071d80 0x7f97001f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f96fffff640 1 -- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f970015db00 msgr2=0x7f97001557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f96fffff640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f970015db00 0x7f97001557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f96fffff640 1 -- 172.21.15.38:0/705543456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970015e8f0 con 0x7f970006f080 2024-04-25T08:35:28.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.693+0000 7f96fffff640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f970006f080 0x7f9700152980 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f96f4000c00 tx=0x7f96f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.694+0000 7f96fcff9640 1 -- 172.21.15.38:0/705543456 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4037050 con 0x7f970006f080 2024-04-25T08:35:28.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.694+0000 7f96fcff9640 1 -- 172.21.15.38:0/705543456 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f4038040 con 0x7f970006f080 2024-04-25T08:35:28.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.694+0000 7f970672b640 1 -- 172.21.15.38:0/705543456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970015ebd0 con 0x7f970006f080 2024-04-25T08:35:28.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.694+0000 7f96fcff9640 1 -- 172.21.15.38:0/705543456 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4004470 con 0x7f970006f080 2024-04-25T08:35:28.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.694+0000 7f970672b640 1 -- 172.21.15.38:0/705543456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97001f7390 con 0x7f970006f080 2024-04-25T08:35:28.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.695+0000 7f970672b640 1 -- 172.21.15.38:0/705543456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970004ef90 con 0x7f970006f080 2024-04-25T08:35:28.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.695+0000 7f96fcff9640 1 -- 172.21.15.38:0/705543456 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96f4030980 con 0x7f970006f080 2024-04-25T08:35:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.696+0000 7f96fcff9640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f96ec06b4d0 0x7f96ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.696+0000 7f96fcff9640 1 -- 172.21.15.38:0/705543456 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f96f40aaf00 con 0x7f970006f080 2024-04-25T08:35:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.696+0000 7f96ff7fe640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f96ec06b4d0 0x7f96ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:28.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.696+0000 7f96ff7fe640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f96ec06b4d0 0x7f96ec06d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9700152930 tx=0x7f96f0007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:28.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.700+0000 7f96fcff9640 1 -- 172.21.15.38:0/705543456 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96f4079d70 con 0x7f970006f080 2024-04-25T08:35:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.919+0000 7f970672b640 1 -- 172.21.15.38:0/705543456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9700072440 con 0x7f970006f080 2024-04-25T08:35:28.914 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:35:28.920+0000 7ff7937f0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:35:28.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.925+0000 7f96fcff9640 1 -- 172.21.15.38:0/705543456 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 161+0+0 (secure 0 0 0) 0x7f96f4079670 con 0x7f970006f080 2024-04-25T08:35:28.920 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:35:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.928+0000 7f970672b640 1 -- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f96ec06b4d0 msgr2=0x7f96ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.928+0000 7f970672b640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f96ec06b4d0 0x7f96ec06d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9700152930 tx=0x7f96f0007b50 comp rx=0 tx=0).stop 2024-04-25T08:35:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.928+0000 7f970672b640 1 -- 172.21.15.38:0/705543456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f970006f080 msgr2=0x7f9700152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.928+0000 7f970672b640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f970006f080 0x7f9700152980 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f96f4000c00 tx=0x7f96f40046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.928+0000 7f970672b640 1 -- 172.21.15.38:0/705543456 shutdown_connections 2024-04-25T08:35:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.928+0000 7f970672b640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f970015db00 0x7f97001557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.928+0000 7f970672b640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f96ec06b4d0 0x7f96ec06d990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.928+0000 7f970672b640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9700071d80 0x7f97001f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.928+0000 7f970672b640 1 --2- 172.21.15.38:0/705543456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f970006f080 0x7f9700152980 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.928+0000 7f970672b640 1 -- 172.21.15.38:0/705543456 >> 172.21.15.38:0/705543456 conn(0x7f97000b24c0 msgr2=0x7f97000cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:28.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.928+0000 7f970672b640 1 -- 172.21.15.38:0/705543456 shutdown_connections 2024-04-25T08:35:28.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:28.928+0000 7f970672b640 1 -- 172.21.15.38:0/705543456 wait complete. 2024-04-25T08:35:28.935 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:35:28.961 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:35:29.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.044+0000 7f961ac1b640 1 Processor -- start 2024-04-25T08:35:29.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.044+0000 7f961ac1b640 1 -- start start 2024-04-25T08:35:29.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f961ac1b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9614166610 0x7f9614168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f961ac1b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140d2060 con 0x7f96140d33d0 2024-04-25T08:35:29.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f961ac1b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140d2330 con 0x7f96140d0320 2024-04-25T08:35:29.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f961ac1b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96140d2610 con 0x7f9614166610 2024-04-25T08:35:29.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f9613fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f96140d33d0 0x7f96140d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45122/0 (socket says 172.21.15.38:45122) 2024-04-25T08:35:29.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f9613fff640 1 -- 172.21.15.38:0/2701957094 learned_addr learned my addr 172.21.15.38:0/2701957094 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:29.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f96191b8640 1 --2- 172.21.15.38:0/2701957094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9614166610 0x7f9614168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f96137fe640 1 -- 172.21.15.38:0/2701957094 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663754563 0 0) 0x7f96140d2060 con 0x7f96140d33d0 2024-04-25T08:35:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f96137fe640 1 -- 172.21.15.38:0/2701957094 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9604003520 con 0x7f96140d33d0 2024-04-25T08:35:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f96137fe640 1 -- 172.21.15.38:0/2701957094 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108333128 0 0) 0x7f96140d2330 con 0x7f96140d0320 2024-04-25T08:35:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f96137fe640 1 -- 172.21.15.38:0/2701957094 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96140d2060 con 0x7f96140d0320 2024-04-25T08:35:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f96191b8640 1 -- 172.21.15.38:0/2701957094 >> v1:172.21.15.107:6789/0 conn(0x7f96140d0320 legacy=0x7f96140d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f96191b8640 1 -- 172.21.15.38:0/2701957094 >> v1:172.21.15.38:6789/0 conn(0x7f96140d33d0 legacy=0x7f96140d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.045+0000 7f96191b8640 1 -- 172.21.15.38:0/2701957094 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96140d28f0 con 0x7f9614166610 2024-04-25T08:35:29.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.046+0000 7f96191b8640 1 --2- 172.21.15.38:0/2701957094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9614166610 0x7f9614168a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f960c009dc0 tx=0x7f960c02a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c015d35cb19ff2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.046+0000 7f96137fe640 1 -- 172.21.15.38:0/2701957094 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c007740 con 0x7f9614166610 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.046+0000 7f96137fe640 1 -- 172.21.15.38:0/2701957094 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c0078e0 con 0x7f9614166610 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.046+0000 7f96137fe640 1 -- 172.21.15.38:0/2701957094 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c007c90 con 0x7f9614166610 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.047+0000 7f961ac1b640 1 -- 172.21.15.38:0/2701957094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9614166610 msgr2=0x7f9614168a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.047+0000 7f961ac1b640 1 --2- 172.21.15.38:0/2701957094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9614166610 0x7f9614168a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f960c009dc0 tx=0x7f960c02a760 comp rx=0 tx=0).stop 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.047+0000 7f961ac1b640 1 -- 172.21.15.38:0/2701957094 shutdown_connections 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.047+0000 7f961ac1b640 1 --2- 172.21.15.38:0/2701957094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9614166610 0x7f9614168a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.047+0000 7f961ac1b640 1 -- 172.21.15.38:0/2701957094 >> 172.21.15.38:0/2701957094 conn(0x7f96140668f0 msgr2=0x7f9614066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.047+0000 7f961ac1b640 1 -- 172.21.15.38:0/2701957094 shutdown_connections 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.047+0000 7f961ac1b640 1 -- 172.21.15.38:0/2701957094 wait complete. 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.048+0000 7f961ac1b640 1 Processor -- start 2024-04-25T08:35:29.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.048+0000 7f961ac1b640 1 -- start start 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f961ac1b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96140d0320 0x7f961414e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f961ac1b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140d33d0 0x7f96140b6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f961ac1b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9614166610 0x7f96140b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f961ac1b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9614161210 con 0x7f9614166610 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f961ac1b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140d2eb0 con 0x7f96140d33d0 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f961ac1b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9614161390 con 0x7f96140d0320 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f96191b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9614166610 0x7f96140b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f96191b8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9614166610 0x7f96140b6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42920/0 (socket says 172.21.15.38:42920) 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f96191b8640 1 -- 172.21.15.38:0/253084184 learned_addr learned my addr 172.21.15.38:0/253084184 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f96189b7640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96140d0320 0x7f961414e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f9613fff640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140d33d0 0x7f96140b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f96191b8640 1 -- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96140d0320 msgr2=0x7f961414e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f96191b8640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96140d0320 0x7f961414e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f96191b8640 1 -- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140d33d0 msgr2=0x7f96140b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f96191b8640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140d33d0 0x7f96140b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.049+0000 7f96191b8640 1 -- 172.21.15.38:0/253084184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961414e6b0 con 0x7f9614166610 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.050+0000 7f96191b8640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9614166610 0x7f96140b6ec0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f960c02ad50 tx=0x7f960c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.050+0000 7f96117fa640 1 -- 172.21.15.38:0/253084184 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c03f070 con 0x7f9614166610 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.050+0000 7f96117fa640 1 -- 172.21.15.38:0/253084184 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c031c40 con 0x7f9614166610 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.050+0000 7f961ac1b640 1 -- 172.21.15.38:0/253084184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96140ccbf0 con 0x7f9614166610 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.050+0000 7f96117fa640 1 -- 172.21.15.38:0/253084184 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c009450 con 0x7f9614166610 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.050+0000 7f961ac1b640 1 -- 172.21.15.38:0/253084184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96141598e0 con 0x7f9614166610 2024-04-25T08:35:29.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.051+0000 7f961ac1b640 1 -- 172.21.15.38:0/253084184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9614068e80 con 0x7f9614166610 2024-04-25T08:35:29.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.051+0000 7f96117fa640 1 -- 172.21.15.38:0/253084184 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f960c044460 con 0x7f9614166610 2024-04-25T08:35:29.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.052+0000 7f96117fa640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f806b4d0 0x7f95f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:29.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.052+0000 7f96117fa640 1 -- 172.21.15.38:0/253084184 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f960c0ab900 con 0x7f9614166610 2024-04-25T08:35:29.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.052+0000 7f96189b7640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f806b4d0 0x7f95f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:29.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.052+0000 7f96189b7640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f806b4d0 0x7f95f806d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9608007fd0 tx=0x7f9608009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:29.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.056+0000 7f96117fa640 1 -- 172.21.15.38:0/253084184 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f960c04b020 con 0x7f9614166610 2024-04-25T08:35:29.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.280+0000 7f961ac1b640 1 -- 172.21.15.38:0/253084184 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f96140d44c0 con 0x7f9614166610 2024-04-25T08:35:29.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.927+0000 7f96117fa640 1 -- 172.21.15.38:0/253084184 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 104+0+0 (secure 0 0 0) 0x7f960c079e90 con 0x7f9614166610 2024-04-25T08:35:29.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.930+0000 7f961ac1b640 1 -- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f806b4d0 msgr2=0x7f95f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.930+0000 7f961ac1b640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f806b4d0 0x7f95f806d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9608007fd0 tx=0x7f9608009040 comp rx=0 tx=0).stop 2024-04-25T08:35:29.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.930+0000 7f961ac1b640 1 -- 172.21.15.38:0/253084184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9614166610 msgr2=0x7f96140b6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:29.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.930+0000 7f961ac1b640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9614166610 0x7f96140b6ec0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f960c02ad50 tx=0x7f960c004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:29.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.930+0000 7f961ac1b640 1 -- 172.21.15.38:0/253084184 shutdown_connections 2024-04-25T08:35:29.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.930+0000 7f961ac1b640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9614166610 0x7f96140b6ec0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.930+0000 7f961ac1b640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140d33d0 0x7f96140b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.930+0000 7f961ac1b640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f806b4d0 0x7f95f806d990 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.930+0000 7f961ac1b640 1 --2- 172.21.15.38:0/253084184 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96140d0320 0x7f961414e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:29.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.930+0000 7f961ac1b640 1 -- 172.21.15.38:0/253084184 >> 172.21.15.38:0/253084184 conn(0x7f96140668f0 msgr2=0x7f961415f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:29.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.930+0000 7f961ac1b640 1 -- 172.21.15.38:0/253084184 shutdown_connections 2024-04-25T08:35:29.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:29.931+0000 7f961ac1b640 1 -- 172.21.15.38:0/253084184 wait complete. 2024-04-25T08:35:29.939 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:35:30.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.100+0000 7f755ab9d640 1 Processor -- start 2024-04-25T08:35:30.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.100+0000 7f755ab9d640 1 -- start start 2024-04-25T08:35:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.100+0000 7f755ab9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75540d0860 0x7f75540d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.100+0000 7f755ab9d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f75540d1e20 0x7f75540d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.100+0000 7f755ab9d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75540d1180 con 0x7f75540d19e0 2024-04-25T08:35:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.100+0000 7f755ab9d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f755415e2b0 con 0x7f75540d1e20 2024-04-25T08:35:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.100+0000 7f755ab9d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75540d2250 con 0x7f75540d0860 2024-04-25T08:35:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f7553fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f75540d19e0 0x7f75540d3860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45138/0 (socket says 172.21.15.38:45138) 2024-04-25T08:35:30.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f7553fff640 1 -- 172.21.15.38:0/100081889 learned_addr learned my addr 172.21.15.38:0/100081889 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f755913a640 1 --2- 172.21.15.38:0/100081889 >> v2:172.21.15.107:3300/0 conn(0x7f75540d1e20 0x7f75540d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f7558939640 1 --2- 172.21.15.38:0/100081889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75540d0860 0x7f75540d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1681356951 0 0) 0x7f75540d1180 con 0x7f75540d19e0 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7544003520 con 0x7f75540d19e0 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1339308455 0 0) 0x7f7544003520 con 0x7f75540d19e0 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75540d0860 msgr2=0x7f75540d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 --2- 172.21.15.38:0/100081889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75540d0860 0x7f75540d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 >> v2:172.21.15.107:3300/0 conn(0x7f75540d1e20 msgr2=0x7f75540d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 --2- 172.21.15.38:0/100081889 >> v2:172.21.15.107:3300/0 conn(0x7f75540d1e20 0x7f75540d3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75540d24a0 con 0x7f75540d19e0 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f7540001670 con 0x7f75540d19e0 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 >> v1:172.21.15.38:6789/0 conn(0x7f75540d19e0 legacy=0x7f75540d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 --2- 172.21.15.38:0/100081889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75440039b0 0x7f7544003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 --2- 172.21.15.38:0/100081889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7544004800 0x7f7544004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 --2- 172.21.15.38:0/100081889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75540d0860 0x7f75440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f755415e2b0 con 0x7f7544004800 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75540d2250 con 0x7f75440039b0 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f754400b9c0 con 0x7f75540d0860 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f755913a640 1 --2- 172.21.15.38:0/100081889 >> v2:172.21.15.107:3300/0 conn(0x7f75540d1e20 0x7f75540d3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f7553fff640 1 --2- 172.21.15.38:0/100081889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7544004800 0x7f7544004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.101+0000 7f7558939640 1 --2- 172.21.15.38:0/100081889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75440039b0 0x7f7544003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.102+0000 7f755913a640 1 --2- 172.21.15.38:0/100081889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75540d0860 0x7f75440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.102+0000 7f7553fff640 1 -- 172.21.15.38:0/100081889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75540d0860 msgr2=0x7f75440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.102+0000 7f7553fff640 1 --2- 172.21.15.38:0/100081889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75540d0860 0x7f75440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.102+0000 7f7553fff640 1 -- 172.21.15.38:0/100081889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75440039b0 msgr2=0x7f7544003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.102+0000 7f7553fff640 1 --2- 172.21.15.38:0/100081889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f75440039b0 0x7f7544003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.102+0000 7f7553fff640 1 -- 172.21.15.38:0/100081889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7544003520 con 0x7f7544004800 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.102+0000 7f7553fff640 1 --2- 172.21.15.38:0/100081889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7544004800 0x7f7544004c80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f754000c240 tx=0x7f7544005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=372ed1aee8d5e57d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:30.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.103+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754002e070 con 0x7f7544004800 2024-04-25T08:35:30.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.103+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7540005af0 con 0x7f7544004800 2024-04-25T08:35:30.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.103+0000 7f75537fe640 1 -- 172.21.15.38:0/100081889 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7540007d30 con 0x7f7544004800 2024-04-25T08:35:30.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.103+0000 7f755ab9d640 1 -- 172.21.15.38:0/100081889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7544004800 msgr2=0x7f7544004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.103+0000 7f755ab9d640 1 --2- 172.21.15.38:0/100081889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7544004800 0x7f7544004c80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f754000c240 tx=0x7f7544005830 comp rx=0 tx=0).stop 2024-04-25T08:35:30.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.103+0000 7f755913a640 1 -- 172.21.15.38:0/100081889 reap_dead start 2024-04-25T08:35:30.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.103+0000 7f755ab9d640 1 -- 172.21.15.38:0/100081889 shutdown_connections 2024-04-25T08:35:30.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.103+0000 7f755ab9d640 1 -- 172.21.15.38:0/100081889 >> 172.21.15.38:0/100081889 conn(0x7f7554066d10 msgr2=0x7f7554067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:30.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.103+0000 7f755ab9d640 1 -- 172.21.15.38:0/100081889 shutdown_connections 2024-04-25T08:35:30.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.103+0000 7f755ab9d640 1 -- 172.21.15.38:0/100081889 wait complete. 2024-04-25T08:35:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.104+0000 7f755ab9d640 1 Processor -- start 2024-04-25T08:35:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.104+0000 7f755ab9d640 1 -- start start 2024-04-25T08:35:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.105+0000 7f755ab9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7544004800 0x7f7554157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.105+0000 7f755ab9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75541597e0 0x7f755415a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.105+0000 7f755ab9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7554149ee0 0x7f755414aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.105+0000 7f755ab9d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75540d2d40 con 0x7f7554149ee0 2024-04-25T08:35:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.105+0000 7f755ab9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75540d2a60 con 0x7f7544004800 2024-04-25T08:35:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.105+0000 7f755ab9d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75540d3540 con 0x7f75541597e0 2024-04-25T08:35:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.105+0000 7f755913a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7554149ee0 0x7f755414aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.105+0000 7f755913a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7554149ee0 0x7f755414aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42936/0 (socket says 172.21.15.38:42936) 2024-04-25T08:35:30.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.105+0000 7f755913a640 1 -- 172.21.15.38:0/870285030 learned_addr learned my addr 172.21.15.38:0/870285030 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f7558939640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7544004800 0x7f7554157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f755913a640 1 -- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75541597e0 msgr2=0x7f755415a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f755913a640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75541597e0 0x7f755415a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f755913a640 1 -- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7544004800 msgr2=0x7f7554157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f755913a640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7544004800 0x7f7554157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f755913a640 1 -- 172.21.15.38:0/870285030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755415a5f0 con 0x7f7554149ee0 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f7558939640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7544004800 0x7f7554157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f755913a640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7554149ee0 0x7f755414aa50 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f75540b54a0 tx=0x7f754c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f75517fa640 1 -- 172.21.15.38:0/870285030 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c003990 con 0x7f7554149ee0 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f75517fa640 1 -- 172.21.15.38:0/870285030 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f754c01b070 con 0x7f7554149ee0 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f755ab9d640 1 -- 172.21.15.38:0/870285030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7554157d30 con 0x7f7554149ee0 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f75517fa640 1 -- 172.21.15.38:0/870285030 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f754c003b50 con 0x7f7554149ee0 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.106+0000 7f755ab9d640 1 -- 172.21.15.38:0/870285030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7554158fe0 con 0x7f7554149ee0 2024-04-25T08:35:30.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.107+0000 7f755ab9d640 1 -- 172.21.15.38:0/870285030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7554068e80 con 0x7f7554149ee0 2024-04-25T08:35:30.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.108+0000 7f75517fa640 1 -- 172.21.15.38:0/870285030 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f754c005d10 con 0x7f7554149ee0 2024-04-25T08:35:30.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.108+0000 7f75517fa640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f753806b500 0x7f753806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:30.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.108+0000 7f75517fa640 1 -- 172.21.15.38:0/870285030 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f754c08aaa0 con 0x7f7554149ee0 2024-04-25T08:35:30.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.108+0000 7f7558939640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f753806b500 0x7f753806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:30.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.109+0000 7f7558939640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f753806b500 0x7f753806d9c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7548006d40 tx=0x7f7548008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:30.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.112+0000 7f75517fa640 1 -- 172.21.15.38:0/870285030 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f754c059700 con 0x7f7554149ee0 2024-04-25T08:35:30.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:30.299+0000 7f755ab9d640 1 -- 172.21.15.38:0/870285030 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f75540b6e20 con 0x7f7554149ee0 2024-04-25T08:35:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.185+0000 7f75517fa640 1 -- 172.21.15.38:0/870285030 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) v1 ==== 181+0+0 (secure 0 0 0) 0x7f754c059000 con 0x7f7554149ee0 2024-04-25T08:35:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:35:31.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.188+0000 7f755ab9d640 1 -- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f753806b500 msgr2=0x7f753806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.188+0000 7f755ab9d640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f753806b500 0x7f753806d9c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7548006d40 tx=0x7f7548008040 comp rx=0 tx=0).stop 2024-04-25T08:35:31.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.188+0000 7f755ab9d640 1 -- 172.21.15.38:0/870285030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7554149ee0 msgr2=0x7f755414aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.188+0000 7f755ab9d640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7554149ee0 0x7f755414aa50 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f75540b54a0 tx=0x7f754c009f90 comp rx=0 tx=0).stop 2024-04-25T08:35:31.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.188+0000 7f755ab9d640 1 -- 172.21.15.38:0/870285030 shutdown_connections 2024-04-25T08:35:31.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.188+0000 7f755ab9d640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7554149ee0 0x7f755414aa50 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.188+0000 7f755ab9d640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f753806b500 0x7f753806d9c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.188+0000 7f755ab9d640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f75541597e0 0x7f755415a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.188+0000 7f755ab9d640 1 --2- 172.21.15.38:0/870285030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7544004800 0x7f7554157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.188+0000 7f755ab9d640 1 -- 172.21.15.38:0/870285030 >> 172.21.15.38:0/870285030 conn(0x7f7554066d10 msgr2=0x7f755415a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:31.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.188+0000 7f755ab9d640 1 -- 172.21.15.38:0/870285030 shutdown_connections 2024-04-25T08:35:31.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.188+0000 7f755ab9d640 1 -- 172.21.15.38:0/870285030 wait complete. 2024-04-25T08:35:31.197 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:35:31.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.315+0000 7f26418b3640 1 Processor -- start 2024-04-25T08:35:31.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.315+0000 7f26418b3640 1 -- start start 2024-04-25T08:35:31.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f26418b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c1612c0 0x7f263c0b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f26418b3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f263c0b6a00 0x7f263c0d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f26418b3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263c0b7200 con 0x7f263c0d0870 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f26418b3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f263c0b74e0 con 0x7f263c0b6a00 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f26418b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0b77c0 con 0x7f263c1612c0 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263b7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f263c0d0870 0x7f263c0b82a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45148/0 (socket says 172.21.15.38:45148) 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263b7fe640 1 -- 172.21.15.38:0/1508010919 learned_addr learned my addr 172.21.15.38:0/1508010919 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263affd640 1 --2- 172.21.15.38:0/1508010919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c1612c0 0x7f263c0b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f2633fff640 1 --2- 172.21.15.38:0/1508010919 >> v2:172.21.15.107:3300/0 conn(0x7f263c0b6a00 0x7f263c0d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810423950 0 0) 0x7f263c0b7200 con 0x7f263c0d0870 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2628003520 con 0x7f263c0d0870 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 527924949 0 0) 0x7f2628003520 con 0x7f263c0d0870 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c1612c0 msgr2=0x7f263c0b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263a7fc640 1 --2- 172.21.15.38:0/1508010919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c1612c0 0x7f263c0b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 >> v2:172.21.15.107:3300/0 conn(0x7f263c0b6a00 msgr2=0x7f263c0d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263a7fc640 1 --2- 172.21.15.38:0/1508010919 >> v2:172.21.15.107:3300/0 conn(0x7f263c0b6a00 0x7f263c0d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c0b7aa0 con 0x7f263c0d0870 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f26340019f0 con 0x7f263c0d0870 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.316+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 >> v1:172.21.15.38:6789/0 conn(0x7f263c0d0870 legacy=0x7f263c0b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263a7fc640 1 --2- 172.21.15.38:0/1508010919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c1612c0 0x7f26280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263a7fc640 1 --2- 172.21.15.38:0/1508010919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2628003f30 0x7f2628004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263a7fc640 1 --2- 172.21.15.38:0/1508010919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2628004d40 0x7f26280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0b74e0 con 0x7f2628004d40 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0b77c0 con 0x7f2628003f30 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262800ba10 con 0x7f263c1612c0 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263b7fe640 1 --2- 172.21.15.38:0/1508010919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2628004d40 0x7f26280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f2633fff640 1 --2- 172.21.15.38:0/1508010919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2628003f30 0x7f2628004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263affd640 1 --2- 172.21.15.38:0/1508010919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c1612c0 0x7f26280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263b7fe640 1 -- 172.21.15.38:0/1508010919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c1612c0 msgr2=0x7f26280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:31.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263b7fe640 1 --2- 172.21.15.38:0/1508010919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c1612c0 0x7f26280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263b7fe640 1 -- 172.21.15.38:0/1508010919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2628003f30 msgr2=0x7f2628004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:31.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263b7fe640 1 --2- 172.21.15.38:0/1508010919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2628003f30 0x7f2628004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.317+0000 7f263b7fe640 1 -- 172.21.15.38:0/1508010919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2628003520 con 0x7f2628004d40 2024-04-25T08:35:31.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.318+0000 7f263b7fe640 1 --2- 172.21.15.38:0/1508010919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2628004d40 0x7f26280051c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f263400a8a0 tx=0x7f2628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daabe445ab833f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:31.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.318+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263400a4c0 con 0x7f2628004d40 2024-04-25T08:35:31.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.318+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263402f070 con 0x7f2628004d40 2024-04-25T08:35:31.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.318+0000 7f263a7fc640 1 -- 172.21.15.38:0/1508010919 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263402bcd0 con 0x7f2628004d40 2024-04-25T08:35:31.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.318+0000 7f26418b3640 1 -- 172.21.15.38:0/1508010919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2628004d40 msgr2=0x7f26280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.318+0000 7f26418b3640 1 --2- 172.21.15.38:0/1508010919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2628004d40 0x7f26280051c0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f263400a8a0 tx=0x7f2628005830 comp rx=0 tx=0).stop 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.318+0000 7f263b7fe640 1 -- 172.21.15.38:0/1508010919 reap_dead start 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.319+0000 7f26418b3640 1 -- 172.21.15.38:0/1508010919 shutdown_connections 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.319+0000 7f26418b3640 1 -- 172.21.15.38:0/1508010919 >> 172.21.15.38:0/1508010919 conn(0x7f263c066d10 msgr2=0x7f263c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.319+0000 7f26418b3640 1 -- 172.21.15.38:0/1508010919 shutdown_connections 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.319+0000 7f26418b3640 1 -- 172.21.15.38:0/1508010919 wait complete. 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.320+0000 7f26418b3640 1 Processor -- start 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.320+0000 7f26418b3640 1 -- start start 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f26418b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2628004d40 0x7f263c159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f26418b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f263c152d50 0x7f263c149e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f26418b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c0cd250 0x7f263c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f26418b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0b6e60 con 0x7f263c152d50 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f26418b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0d2170 con 0x7f2628004d40 2024-04-25T08:35:31.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f26418b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263c0b6fe0 con 0x7f263c0cd250 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f2633fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f263c152d50 0x7f263c149e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f2633fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f263c152d50 0x7f263c149e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42954/0 (socket says 172.21.15.38:42954) 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f2633fff640 1 -- 172.21.15.38:0/3894273726 learned_addr learned my addr 172.21.15.38:0/3894273726 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f263affd640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2628004d40 0x7f263c159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f263b7fe640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c0cd250 0x7f263c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f2633fff640 1 -- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c0cd250 msgr2=0x7f263c155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f2633fff640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c0cd250 0x7f263c155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f2633fff640 1 -- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2628004d40 msgr2=0x7f263c159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f2633fff640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2628004d40 0x7f263c159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f2633fff640 1 -- 172.21.15.38:0/3894273726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263c0cc1b0 con 0x7f263c152d50 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f2633fff640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f263c152d50 0x7f263c149e70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2624011960 tx=0x7f2624011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f2638ff9640 1 -- 172.21.15.38:0/3894273726 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262400fa30 con 0x7f263c152d50 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f2638ff9640 1 -- 172.21.15.38:0/3894273726 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262400fbd0 con 0x7f263c152d50 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f2638ff9640 1 -- 172.21.15.38:0/3894273726 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2624014440 con 0x7f263c152d50 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.321+0000 7f26418b3640 1 -- 172.21.15.38:0/3894273726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263c0cc490 con 0x7f263c152d50 2024-04-25T08:35:31.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.322+0000 7f26418b3640 1 -- 172.21.15.38:0/3894273726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263c0cca50 con 0x7f263c152d50 2024-04-25T08:35:31.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.323+0000 7f26418b3640 1 -- 172.21.15.38:0/3894273726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263c068e80 con 0x7f263c152d50 2024-04-25T08:35:31.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.323+0000 7f2638ff9640 1 -- 172.21.15.38:0/3894273726 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2624012040 con 0x7f263c152d50 2024-04-25T08:35:31.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.323+0000 7f2638ff9640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f261c06b4d0 0x7f261c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:31.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.323+0000 7f2638ff9640 1 -- 172.21.15.38:0/3894273726 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5807+0+0 (secure 0 0 0) 0x7f262408a720 con 0x7f263c152d50 2024-04-25T08:35:31.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.324+0000 7f263affd640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f261c06b4d0 0x7f261c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:31.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.324+0000 7f263affd640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f261c06b4d0 0x7f261c06d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f262c009bc0 tx=0x7f262c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:31.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.328+0000 7f2638ff9640 1 -- 172.21.15.38:0/3894273726 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2624019030 con 0x7f263c152d50 2024-04-25T08:35:31.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:31.510+0000 7f26418b3640 1 -- 172.21.15.38:0/3894273726 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f263c153550 con 0x7f263c152d50 2024-04-25T08:35:32.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.192+0000 7f2638ff9640 1 -- 172.21.15.38:0/3894273726 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) v1 ==== 169+0+0 (secure 0 0 0) 0x7f26240595d0 con 0x7f263c152d50 2024-04-25T08:35:32.187 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-04-25T08:35:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.195+0000 7f26418b3640 1 -- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f261c06b4d0 msgr2=0x7f261c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.195+0000 7f26418b3640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f261c06b4d0 0x7f261c06d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f262c009bc0 tx=0x7f262c009190 comp rx=0 tx=0).stop 2024-04-25T08:35:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.195+0000 7f26418b3640 1 -- 172.21.15.38:0/3894273726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f263c152d50 msgr2=0x7f263c149e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.195+0000 7f26418b3640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f263c152d50 0x7f263c149e70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f2624011960 tx=0x7f2624011d60 comp rx=0 tx=0).stop 2024-04-25T08:35:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.195+0000 7f26418b3640 1 -- 172.21.15.38:0/3894273726 shutdown_connections 2024-04-25T08:35:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.195+0000 7f26418b3640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f261c06b4d0 0x7f261c06d990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.195+0000 7f26418b3640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f263c0cd250 0x7f263c155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.195+0000 7f26418b3640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f263c152d50 0x7f263c149e70 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.195+0000 7f26418b3640 1 --2- 172.21.15.38:0/3894273726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2628004d40 0x7f263c159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.195+0000 7f26418b3640 1 -- 172.21.15.38:0/3894273726 >> 172.21.15.38:0/3894273726 conn(0x7f263c066d10 msgr2=0x7f263c0c8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:32.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.195+0000 7f26418b3640 1 -- 172.21.15.38:0/3894273726 shutdown_connections 2024-04-25T08:35:32.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.195+0000 7f26418b3640 1 -- 172.21.15.38:0/3894273726 wait complete. 2024-04-25T08:35:32.204 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:35:32.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.318+0000 7f98e654c640 1 Processor -- start 2024-04-25T08:35:32.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.318+0000 7f98e654c640 1 -- start start 2024-04-25T08:35:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.319+0000 7f98e654c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f98e00d0320 0x7f98e00d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.319+0000 7f98e654c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98e00d0700 0x7f98e00d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.319+0000 7f98e654c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e00d2330 con 0x7f98e00d1e90 2024-04-25T08:35:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.319+0000 7f98e654c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e00d2610 con 0x7f98e00d0320 2024-04-25T08:35:32.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.319+0000 7f98e654c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00d28f0 con 0x7f98e00d0700 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.319+0000 7f98dffff640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f98e00d0320 0x7f98e00d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.319+0000 7f98df7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98e00d0700 0x7f98e00d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.319+0000 7f98e4ae9640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f98e00d1e90 0x7f98e00b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:39034/0 (socket says 172.21.15.38:39034) 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.319+0000 7f98e4ae9640 1 -- 172.21.15.38:0/2426047548 learned_addr learned my addr 172.21.15.38:0/2426047548 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.320+0000 7f98deffd640 1 -- 172.21.15.38:0/2426047548 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162518084 0 0) 0x7f98e00d2330 con 0x7f98e00d1e90 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.320+0000 7f98deffd640 1 -- 172.21.15.38:0/2426047548 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c8003520 con 0x7f98e00d1e90 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.320+0000 7f98dffff640 1 -- 172.21.15.38:0/2426047548 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98e00d0700 msgr2=0x7f98e00d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.320+0000 7f98dffff640 1 --2- 172.21.15.38:0/2426047548 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98e00d0700 0x7f98e00d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.320+0000 7f98dffff640 1 -- 172.21.15.38:0/2426047548 >> v1:172.21.15.107:6789/0 conn(0x7f98e00d1e90 legacy=0x7f98e00b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.320+0000 7f98dffff640 1 -- 172.21.15.38:0/2426047548 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e00d2bd0 con 0x7f98e00d0320 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.320+0000 7f98df7fe640 1 --2- 172.21.15.38:0/2426047548 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98e00d0700 0x7f98e00d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.321+0000 7f98dffff640 1 --2- 172.21.15.38:0/2426047548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98e00d0320 0x7f98e00d33d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98d40097a0 tx=0x7f98d402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44478ed85ac91f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.321+0000 7f98deffd640 1 -- 172.21.15.38:0/2426047548 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4037070 con 0x7f98e00d0320 2024-04-25T08:35:32.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.321+0000 7f98deffd640 1 -- 172.21.15.38:0/2426047548 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d4032440 con 0x7f98e00d0320 2024-04-25T08:35:32.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.321+0000 7f98deffd640 1 -- 172.21.15.38:0/2426047548 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d40327f0 con 0x7f98e00d0320 2024-04-25T08:35:32.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.321+0000 7f98e654c640 1 -- 172.21.15.38:0/2426047548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98e00d0320 msgr2=0x7f98e00d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.321+0000 7f98e654c640 1 --2- 172.21.15.38:0/2426047548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98e00d0320 0x7f98e00d33d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f98d40097a0 tx=0x7f98d402a210 comp rx=0 tx=0).stop 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.321+0000 7f98e654c640 1 -- 172.21.15.38:0/2426047548 shutdown_connections 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.321+0000 7f98e654c640 1 --2- 172.21.15.38:0/2426047548 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98e00d0700 0x7f98e00d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.321+0000 7f98e654c640 1 --2- 172.21.15.38:0/2426047548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98e00d0320 0x7f98e00d33d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.321+0000 7f98e654c640 1 -- 172.21.15.38:0/2426047548 >> 172.21.15.38:0/2426047548 conn(0x7f98e00668f0 msgr2=0x7f98e0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.321+0000 7f98e654c640 1 -- 172.21.15.38:0/2426047548 shutdown_connections 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.321+0000 7f98e654c640 1 -- 172.21.15.38:0/2426047548 wait complete. 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.323+0000 7f98e654c640 1 Processor -- start 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.323+0000 7f98e654c640 1 -- start start 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.323+0000 7f98e654c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98e00d0320 0x7f98e0159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.323+0000 7f98e654c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98e00d0700 0x7f98e0157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.323+0000 7f98e654c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98e00d1e90 0x7f98e0154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.323+0000 7f98e654c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00d0b40 con 0x7f98e00d0320 2024-04-25T08:35:32.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98e654c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0161210 con 0x7f98e00d0700 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98e654c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0157940 con 0x7f98e00d1e90 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98e00d0320 0x7f98e0159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98e00d0320 0x7f98e0159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42964/0 (socket says 172.21.15.38:42964) 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98dffff640 1 -- 172.21.15.38:0/3459504071 learned_addr learned my addr 172.21.15.38:0/3459504071 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98df7fe640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98e00d0700 0x7f98e0157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98e4ae9640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98e00d1e90 0x7f98e0154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98dffff640 1 -- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98e00d1e90 msgr2=0x7f98e0154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98dffff640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98e00d1e90 0x7f98e0154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98dffff640 1 -- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98e00d0700 msgr2=0x7f98e0157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98dffff640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98e00d0700 0x7f98e0157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98dffff640 1 -- 172.21.15.38:0/3459504071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e0157ba0 con 0x7f98e00d0320 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98dffff640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98e00d0320 0x7f98e0159e20 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f98d402a250 tx=0x7f98d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:32.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98dcff9640 1 -- 172.21.15.38:0/3459504071 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4037040 con 0x7f98e00d0320 2024-04-25T08:35:32.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98dcff9640 1 -- 172.21.15.38:0/3459504071 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d4042070 con 0x7f98e00d0320 2024-04-25T08:35:32.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98e654c640 1 -- 172.21.15.38:0/3459504071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e0157e80 con 0x7f98e00d0320 2024-04-25T08:35:32.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98dcff9640 1 -- 172.21.15.38:0/3459504071 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d403b930 con 0x7f98e00d0320 2024-04-25T08:35:32.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.324+0000 7f98e654c640 1 -- 172.21.15.38:0/3459504071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e015a690 con 0x7f98e00d0320 2024-04-25T08:35:32.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.325+0000 7f98e654c640 1 -- 172.21.15.38:0/3459504071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e0068e80 con 0x7f98e00d0320 2024-04-25T08:35:32.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.326+0000 7f98dcff9640 1 -- 172.21.15.38:0/3459504071 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98d403bad0 con 0x7f98e00d0320 2024-04-25T08:35:32.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.326+0000 7f98dcff9640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98cc06b4d0 0x7f98cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.326+0000 7f98dcff9640 1 -- 172.21.15.38:0/3459504071 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f98d40aa800 con 0x7f98e00d0320 2024-04-25T08:35:32.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.327+0000 7f98df7fe640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98cc06b4d0 0x7f98cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.327+0000 7f98df7fe640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98cc06b4d0 0x7f98cc06d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009560 tx=0x7f98d0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:32.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.331+0000 7f98dcff9640 1 -- 172.21.15.38:0/3459504071 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98d40791a0 con 0x7f98e00d0320 2024-04-25T08:35:32.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.526+0000 7f98e654c640 1 -- 172.21.15.38:0/3459504071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f98e00b3050 con 0x7f98e00d0320 2024-04-25T08:35:32.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.529+0000 7f98dcff9640 1 -- 172.21.15.38:0/3459504071 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f98d407d240 con 0x7f98e00d0320 2024-04-25T08:35:32.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.531+0000 7f98e654c640 1 -- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98cc06b4d0 msgr2=0x7f98cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.531+0000 7f98e654c640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98cc06b4d0 0x7f98cc06d990 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f98d0009560 tx=0x7f98d0009190 comp rx=0 tx=0).stop 2024-04-25T08:35:32.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.531+0000 7f98e654c640 1 -- 172.21.15.38:0/3459504071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98e00d0320 msgr2=0x7f98e0159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.531+0000 7f98e654c640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98e00d0320 0x7f98e0159e20 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f98d402a250 tx=0x7f98d40046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.531+0000 7f98e654c640 1 -- 172.21.15.38:0/3459504071 shutdown_connections 2024-04-25T08:35:32.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.531+0000 7f98e654c640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98cc06b4d0 0x7f98cc06d990 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.531+0000 7f98e654c640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98e00d1e90 0x7f98e0154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.531+0000 7f98e654c640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98e00d0700 0x7f98e0157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.531+0000 7f98e654c640 1 --2- 172.21.15.38:0/3459504071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98e00d0320 0x7f98e0159e20 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.531+0000 7f98e654c640 1 -- 172.21.15.38:0/3459504071 >> 172.21.15.38:0/3459504071 conn(0x7f98e00668f0 msgr2=0x7f98e00b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:32.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.531+0000 7f98e654c640 1 -- 172.21.15.38:0/3459504071 shutdown_connections 2024-04-25T08:35:32.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.531+0000 7f98e654c640 1 -- 172.21.15.38:0/3459504071 wait complete. 2024-04-25T08:35:32.537 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T08:35:32.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.643+0000 7f0615142640 1 Processor -- start 2024-04-25T08:35:32.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.643+0000 7f0615142640 1 -- start start 2024-04-25T08:35:32.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.644+0000 7f0615142640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06100aeeb0 0x7f06100ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.644+0000 7f0615142640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f06100cd450 0x7f061006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.644+0000 7f0615142640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06100cfa40 con 0x7f061006b120 2024-04-25T08:35:32.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.644+0000 7f0615142640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06100cd9b0 con 0x7f06100cd450 2024-04-25T08:35:32.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.644+0000 7f0615142640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100cdc90 con 0x7f06100aeeb0 2024-04-25T08:35:32.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.644+0000 7f060659c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f06100cd450 0x7f061006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.644+0000 7f060ed9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06100aeeb0 0x7f06100ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.644+0000 7f060f59e640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f061006b120 0x7f06101616d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:39044/0 (socket says 172.21.15.38:39044) 2024-04-25T08:35:32.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.644+0000 7f060f59e640 1 -- 172.21.15.38:0/2435426028 learned_addr learned my addr 172.21.15.38:0/2435426028 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:32.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.645+0000 7f060e59c640 1 -- 172.21.15.38:0/2435426028 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 839933273 0 0) 0x7f06100cfa40 con 0x7f061006b120 2024-04-25T08:35:32.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.645+0000 7f060ed9d640 1 -- 172.21.15.38:0/2435426028 >> v2:172.21.15.107:3300/0 conn(0x7f06100cd450 msgr2=0x7f061006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:32.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.645+0000 7f060ed9d640 1 --2- 172.21.15.38:0/2435426028 >> v2:172.21.15.107:3300/0 conn(0x7f06100cd450 0x7f061006abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.645+0000 7f060ed9d640 1 -- 172.21.15.38:0/2435426028 >> v1:172.21.15.107:6789/0 conn(0x7f061006b120 legacy=0x7f06101616d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.645+0000 7f060ed9d640 1 -- 172.21.15.38:0/2435426028 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061015e390 con 0x7f06100aeeb0 2024-04-25T08:35:32.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.646+0000 7f060ed9d640 1 --2- 172.21.15.38:0/2435426028 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06100aeeb0 0x7f06100ccf10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f06000097a0 tx=0x7f060002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738e17ae9d32e2b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:32.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.646+0000 7f060e59c640 1 -- 172.21.15.38:0/2435426028 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600037070 con 0x7f06100aeeb0 2024-04-25T08:35:32.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.646+0000 7f060e59c640 1 -- 172.21.15.38:0/2435426028 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0600032440 con 0x7f06100aeeb0 2024-04-25T08:35:32.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.646+0000 7f060e59c640 1 -- 172.21.15.38:0/2435426028 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0600032810 con 0x7f06100aeeb0 2024-04-25T08:35:32.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.646+0000 7f0615142640 1 -- 172.21.15.38:0/2435426028 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06100aeeb0 msgr2=0x7f06100ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.646+0000 7f0615142640 1 --2- 172.21.15.38:0/2435426028 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06100aeeb0 0x7f06100ccf10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f06000097a0 tx=0x7f060002a180 comp rx=0 tx=0).stop 2024-04-25T08:35:32.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.646+0000 7f0615142640 1 -- 172.21.15.38:0/2435426028 shutdown_connections 2024-04-25T08:35:32.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.646+0000 7f0615142640 1 --2- 172.21.15.38:0/2435426028 >> v2:172.21.15.107:3300/0 conn(0x7f06100cd450 0x7f061006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.646+0000 7f0615142640 1 --2- 172.21.15.38:0/2435426028 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06100aeeb0 0x7f06100ccf10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.646+0000 7f0615142640 1 -- 172.21.15.38:0/2435426028 >> 172.21.15.38:0/2435426028 conn(0x7f0610072610 msgr2=0x7f0610099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:32.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.646+0000 7f0615142640 1 -- 172.21.15.38:0/2435426028 shutdown_connections 2024-04-25T08:35:32.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.646+0000 7f0615142640 1 -- 172.21.15.38:0/2435426028 wait complete. 2024-04-25T08:35:32.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.648+0000 7f0615142640 1 Processor -- start 2024-04-25T08:35:32.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.648+0000 7f0615142640 1 -- start start 2024-04-25T08:35:32.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f0615142640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f061006b120 0x7f06100c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f0615142640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100aeeb0 0x7f06100c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f0615142640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06100cd450 0x7f06101f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f0615142640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06100d1360 con 0x7f06100cd450 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f0615142640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06100d11e0 con 0x7f06100aeeb0 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f0615142640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100d14e0 con 0x7f061006b120 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f060f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06100cd450 0x7f06101f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f060f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06100cd450 0x7f06101f28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42980/0 (socket says 172.21.15.38:42980) 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f060f59e640 1 -- 172.21.15.38:0/3273339408 learned_addr learned my addr 172.21.15.38:0/3273339408 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f060659c640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100aeeb0 0x7f06100c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f060ed9d640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f061006b120 0x7f06100c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f060659c640 1 -- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f061006b120 msgr2=0x7f06100c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f060659c640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f061006b120 0x7f06100c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f060659c640 1 -- 172.21.15.38:0/3273339408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06100cd450 msgr2=0x7f06101f28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:32.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f060659c640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06100cd450 0x7f06101f28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:32.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f060659c640 1 -- 172.21.15.38:0/3273339408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06100c4900 con 0x7f06100aeeb0 2024-04-25T08:35:32.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f060f59e640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06100cd450 0x7f06101f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:32.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f060659c640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100aeeb0 0x7f06100c8f10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f05f800e870 tx=0x7f05f800ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:32.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.649+0000 7f0607fff640 1 -- 172.21.15.38:0/3273339408 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f8017070 con 0x7f06100aeeb0 2024-04-25T08:35:32.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.650+0000 7f0607fff640 1 -- 172.21.15.38:0/3273339408 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f800c730 con 0x7f06100aeeb0 2024-04-25T08:35:32.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.650+0000 7f0607fff640 1 -- 172.21.15.38:0/3273339408 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f800cb00 con 0x7f06100aeeb0 2024-04-25T08:35:32.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.650+0000 7f0615142640 1 -- 172.21.15.38:0/3273339408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06100c4be0 con 0x7f06100aeeb0 2024-04-25T08:35:32.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.650+0000 7f0615142640 1 -- 172.21.15.38:0/3273339408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06100c51a0 con 0x7f06100aeeb0 2024-04-25T08:35:32.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.650+0000 7f0615142640 1 -- 172.21.15.38:0/3273339408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06100b2d60 con 0x7f06100aeeb0 2024-04-25T08:35:32.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.651+0000 7f0607fff640 1 -- 172.21.15.38:0/3273339408 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05f801a070 con 0x7f06100aeeb0 2024-04-25T08:35:32.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.652+0000 7f0607fff640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f05f006b4d0 0x7f05f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:32.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.652+0000 7f0607fff640 1 -- 172.21.15.38:0/3273339408 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f05f808a780 con 0x7f06100aeeb0 2024-04-25T08:35:32.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.652+0000 7f060ed9d640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f05f006b4d0 0x7f05f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:32.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.652+0000 7f060ed9d640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f05f006b4d0 0x7f05f006d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0600034000 tx=0x7f060002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.656+0000 7f0607fff640 1 -- 172.21.15.38:0/3273339408 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05f8059100 con 0x7f06100aeeb0 2024-04-25T08:35:32.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:32.919+0000 7f0615142640 1 -- 172.21.15.38:0/3273339408 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f061006d090 con 0x7f06100aeeb0 2024-04-25T08:35:33.072 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:35:33.075 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:35:33.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.196+0000 7f0607fff640 1 -- 172.21.15.38:0/3273339408 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v207) v1 ==== 143+0+0 (secure 0 0 0) 0x7f05f805d1a0 con 0x7f06100aeeb0 2024-04-25T08:35:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.198+0000 7f0615142640 1 -- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f05f006b4d0 msgr2=0x7f05f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.198+0000 7f0615142640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f05f006b4d0 0x7f05f006d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0600034000 tx=0x7f060002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.198+0000 7f0615142640 1 -- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100aeeb0 msgr2=0x7f06100c8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.198+0000 7f0615142640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100aeeb0 0x7f06100c8f10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f05f800e870 tx=0x7f05f800ec70 comp rx=0 tx=0).stop 2024-04-25T08:35:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.198+0000 7f0615142640 1 -- 172.21.15.38:0/3273339408 shutdown_connections 2024-04-25T08:35:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.198+0000 7f0615142640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06100cd450 0x7f06101f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.198+0000 7f0615142640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06100aeeb0 0x7f06100c8f10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.198+0000 7f0615142640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f05f006b4d0 0x7f05f006d990 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.198+0000 7f0615142640 1 --2- 172.21.15.38:0/3273339408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f061006b120 0x7f06100c40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.198+0000 7f0615142640 1 -- 172.21.15.38:0/3273339408 >> 172.21.15.38:0/3273339408 conn(0x7f0610072610 msgr2=0x7f06100caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:33.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.198+0000 7f0615142640 1 -- 172.21.15.38:0/3273339408 shutdown_connections 2024-04-25T08:35:33.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.198+0000 7f0615142640 1 -- 172.21.15.38:0/3273339408 wait complete. 2024-04-25T08:35:33.204 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T08:35:33.204 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T08:35:33.205 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:33.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.311+0000 7f882cbd5640 1 Processor -- start 2024-04-25T08:35:33.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.311+0000 7f882cbd5640 1 -- start start 2024-04-25T08:35:33.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.312+0000 7f882cbd5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f882806b090 0x7f88280d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.312+0000 7f882cbd5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88280cfa40 con 0x7f882806abe0 2024-04-25T08:35:33.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.312+0000 7f882cbd5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88280cd9b0 con 0x7f88280aeeb0 2024-04-25T08:35:33.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.312+0000 7f882cbd5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88280cdc90 con 0x7f882806b090 2024-04-25T08:35:33.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.312+0000 7f8825d9b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f882806abe0 0x7f88280d01d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45150/0 (socket says 172.21.15.38:45150) 2024-04-25T08:35:33.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.312+0000 7f8825d9b640 1 -- 172.21.15.38:0/322399670 learned_addr learned my addr 172.21.15.38:0/322399670 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:33.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2434760413 0 0) 0x7f88280cfa40 con 0x7f882806abe0 2024-04-25T08:35:33.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8814003520 con 0x7f882806abe0 2024-04-25T08:35:33.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f8826d9d640 1 --2- 172.21.15.38:0/322399670 >> v2:172.21.15.107:3300/0 conn(0x7f882806b090 0x7f88280d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1313094432 0 0) 0x7f8814003520 con 0x7f882806abe0 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 >> v2:172.21.15.107:3300/0 conn(0x7f882806b090 msgr2=0x7f88280d08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 --2- 172.21.15.38:0/322399670 >> v2:172.21.15.107:3300/0 conn(0x7f882806b090 0x7f88280d08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 >> v1:172.21.15.107:6789/0 conn(0x7f88280aeeb0 legacy=0x7f88280ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882815e390 con 0x7f882806abe0 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f8810001670 con 0x7f882806abe0 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 >> v1:172.21.15.38:6789/0 conn(0x7f882806abe0 legacy=0x7f88280d01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 --2- 172.21.15.38:0/322399670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88140039b0 0x7f8814003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 --2- 172.21.15.38:0/322399670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 0x7f8814004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 --2- 172.21.15.38:0/322399670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88140051c0 0x7f881400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88280cdc90 con 0x7f8814004800 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881400c060 con 0x7f88140051c0 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881400c260 con 0x7f88140039b0 2024-04-25T08:35:33.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f8825d9b640 1 --2- 172.21.15.38:0/322399670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 0x7f8814004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f8826d9d640 1 --2- 172.21.15.38:0/322399670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88140051c0 0x7f881400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f882659c640 1 --2- 172.21.15.38:0/322399670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88140039b0 0x7f8814003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f8825d9b640 1 -- 172.21.15.38:0/322399670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88140039b0 msgr2=0x7f8814003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f8825d9b640 1 --2- 172.21.15.38:0/322399670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88140039b0 0x7f8814003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f8825d9b640 1 -- 172.21.15.38:0/322399670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88140051c0 msgr2=0x7f881400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.313+0000 7f8825d9b640 1 --2- 172.21.15.38:0/322399670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88140051c0 0x7f881400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.314+0000 7f8825d9b640 1 -- 172.21.15.38:0/322399670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8814003520 con 0x7f8814004800 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.314+0000 7f8826d9d640 1 -- 172.21.15.38:0/322399670 reap_dead start 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.314+0000 7f882659c640 1 --2- 172.21.15.38:0/322399670 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88140039b0 0x7f8814003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.314+0000 7f8825d9b640 1 --2- 172.21.15.38:0/322399670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 0x7f8814004c80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f881000c210 tx=0x7f8814005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91150d803600c75c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.314+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810007d30 con 0x7f8814004800 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.314+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8810005af0 con 0x7f8814004800 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.314+0000 7f882559a640 1 -- 172.21.15.38:0/322399670 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8810007070 con 0x7f8814004800 2024-04-25T08:35:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.315+0000 7f882cbd5640 1 -- 172.21.15.38:0/322399670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 msgr2=0x7f8814004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.315+0000 7f882cbd5640 1 --2- 172.21.15.38:0/322399670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 0x7f8814004c80 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f881000c210 tx=0x7f8814005830 comp rx=0 tx=0).stop 2024-04-25T08:35:33.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.315+0000 7f882cbd5640 1 -- 172.21.15.38:0/322399670 shutdown_connections 2024-04-25T08:35:33.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.315+0000 7f882cbd5640 1 --2- 172.21.15.38:0/322399670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 0x7f8814004c80 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.315+0000 7f882cbd5640 1 -- 172.21.15.38:0/322399670 >> 172.21.15.38:0/322399670 conn(0x7f8828072610 msgr2=0x7f8828099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:33.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.315+0000 7f882cbd5640 1 -- 172.21.15.38:0/322399670 shutdown_connections 2024-04-25T08:35:33.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.315+0000 7f882cbd5640 1 -- 172.21.15.38:0/322399670 wait complete. 2024-04-25T08:35:33.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.316+0000 7f882cbd5640 1 Processor -- start 2024-04-25T08:35:33.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.316+0000 7f882cbd5640 1 -- start start 2024-04-25T08:35:33.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.317+0000 7f882cbd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 0x7f88280b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.317+0000 7f882cbd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f882814c440 0x7f882814bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.317+0000 7f882cbd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280c53b0 0x7f88280c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.317+0000 7f882cbd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88280ccf10 con 0x7f8814004800 2024-04-25T08:35:33.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.317+0000 7f882cbd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88280cdf70 con 0x7f882814c440 2024-04-25T08:35:33.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.317+0000 7f882cbd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88280cd090 con 0x7f88280c53b0 2024-04-25T08:35:33.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.317+0000 7f882659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 0x7f88280b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.317+0000 7f882659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 0x7f88280b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42988/0 (socket says 172.21.15.38:42988) 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.317+0000 7f882659c640 1 -- 172.21.15.38:0/2607611755 learned_addr learned my addr 172.21.15.38:0/2607611755 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.317+0000 7f8825d9b640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f882814c440 0x7f882814bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.317+0000 7f8826d9d640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280c53b0 0x7f88280c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.318+0000 7f882659c640 1 -- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280c53b0 msgr2=0x7f88280c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.318+0000 7f882659c640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280c53b0 0x7f88280c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.318+0000 7f882659c640 1 -- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f882814c440 msgr2=0x7f882814bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.318+0000 7f882659c640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f882814c440 0x7f882814bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.318+0000 7f882659c640 1 -- 172.21.15.38:0/2607611755 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88281f3990 con 0x7f8814004800 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.318+0000 7f882659c640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 0x7f88280b3840 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f882000e4e0 tx=0x7f882000e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.318+0000 7f881effd640 1 -- 172.21.15.38:0/2607611755 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88200052e0 con 0x7f8814004800 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.318+0000 7f881effd640 1 -- 172.21.15.38:0/2607611755 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8820007020 con 0x7f8814004800 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.318+0000 7f882cbd5640 1 -- 172.21.15.38:0/2607611755 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88281506e0 con 0x7f8814004800 2024-04-25T08:35:33.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.318+0000 7f881effd640 1 -- 172.21.15.38:0/2607611755 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820007210 con 0x7f8814004800 2024-04-25T08:35:33.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.318+0000 7f882cbd5640 1 -- 172.21.15.38:0/2607611755 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882814d440 con 0x7f8814004800 2024-04-25T08:35:33.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.319+0000 7f882cbd5640 1 -- 172.21.15.38:0/2607611755 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882806b860 con 0x7f8814004800 2024-04-25T08:35:33.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.319+0000 7f881effd640 1 -- 172.21.15.38:0/2607611755 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f882001a020 con 0x7f8814004800 2024-04-25T08:35:33.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.320+0000 7f881effd640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f880806b4d0 0x7f880806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.320+0000 7f881effd640 1 -- 172.21.15.38:0/2607611755 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8820089620 con 0x7f8814004800 2024-04-25T08:35:33.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.320+0000 7f8825d9b640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f880806b4d0 0x7f880806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.320+0000 7f8825d9b640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f880806b4d0 0x7f880806d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8810032000 tx=0x7f881002d000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:33.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.324+0000 7f881effd640 1 -- 172.21.15.38:0/2607611755 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8820058570 con 0x7f8814004800 2024-04-25T08:35:33.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.508+0000 7f882cbd5640 1 -- 172.21.15.38:0/2607611755 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f882806ec60 con 0x7f8814004800 2024-04-25T08:35:33.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.508+0000 7f881effd640 1 -- 172.21.15.38:0/2607611755 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+13279 (secure 0 0 0) 0x7f8820057e70 con 0x7f8814004800 2024-04-25T08:35:33.503 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:33.503 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":59,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:32.186368+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6834/226686241":"2024-04-26T08:35:28.920796+0000","172.21.15.38:6832/226686241":"2024-04-26T08:35:28.920796+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:33.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.510+0000 7f882cbd5640 1 -- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f880806b4d0 msgr2=0x7f880806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.510+0000 7f882cbd5640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f880806b4d0 0x7f880806d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8810032000 tx=0x7f881002d000 comp rx=0 tx=0).stop 2024-04-25T08:35:33.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.510+0000 7f882cbd5640 1 -- 172.21.15.38:0/2607611755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 msgr2=0x7f88280b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.510+0000 7f882cbd5640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 0x7f88280b3840 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f882000e4e0 tx=0x7f882000e9a0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.510+0000 7f882cbd5640 1 -- 172.21.15.38:0/2607611755 shutdown_connections 2024-04-25T08:35:33.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.510+0000 7f882cbd5640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f880806b4d0 0x7f880806d990 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.510+0000 7f882cbd5640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f88280c53b0 0x7f88280c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.510+0000 7f882cbd5640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f882814c440 0x7f882814bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.510+0000 7f882cbd5640 1 --2- 172.21.15.38:0/2607611755 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8814004800 0x7f88280b3840 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.510+0000 7f882cbd5640 1 -- 172.21.15.38:0/2607611755 >> 172.21.15.38:0/2607611755 conn(0x7f8828072610 msgr2=0x7f88280afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:33.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.511+0000 7f882cbd5640 1 -- 172.21.15.38:0/2607611755 shutdown_connections 2024-04-25T08:35:33.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.511+0000 7f882cbd5640 1 -- 172.21.15.38:0/2607611755 wait complete. 2024-04-25T08:35:33.516 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:35:33.517 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:35:33.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.625+0000 7fd13966b640 1 Processor -- start 2024-04-25T08:35:33.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.625+0000 7fd13966b640 1 -- start start 2024-04-25T08:35:33.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.626+0000 7fd13966b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd13406abe0 0x7fd13406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.626+0000 7fd13966b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd1341635e0 0x7fd1341659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.626+0000 7fd13966b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13406b500 con 0x7fd1340cc2c0 2024-04-25T08:35:33.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.626+0000 7fd13966b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd13415c1b0 con 0x7fd1341635e0 2024-04-25T08:35:33.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.626+0000 7fd13966b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340ccf30 con 0x7fd13406abe0 2024-04-25T08:35:33.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.627+0000 7fd1337fe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd1341635e0 0x7fd1341659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.627+0000 7fd132ffd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd13406abe0 0x7fd13406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.627+0000 7fd1337fe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd1341635e0 0x7fd1341659d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:51352/0 (socket says 172.21.15.38:51352) 2024-04-25T08:35:33.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.627+0000 7fd1337fe640 1 -- 172.21.15.38:0/2393975768 learned_addr learned my addr 172.21.15.38:0/2393975768 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:33.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.627+0000 7fd1327fc640 1 --1- 172.21.15.38:0/2393975768 >> v1:172.21.15.107:6789/0 conn(0x7fd1340cc2c0 0x7fd1340cc740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:39054/0 (socket says 172.21.15.38:39054) 2024-04-25T08:35:33.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.627+0000 7fd131ffb640 1 -- 172.21.15.38:0/2393975768 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992610053 0 0) 0x7fd13406b500 con 0x7fd1340cc2c0 2024-04-25T08:35:33.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.627+0000 7fd1337fe640 1 -- 172.21.15.38:0/2393975768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd13406abe0 msgr2=0x7fd13406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:33.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.627+0000 7fd1337fe640 1 --2- 172.21.15.38:0/2393975768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd13406abe0 0x7fd13406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.627+0000 7fd1337fe640 1 -- 172.21.15.38:0/2393975768 >> v1:172.21.15.107:6789/0 conn(0x7fd1340cc2c0 legacy=0x7fd1340cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.627+0000 7fd1337fe640 1 -- 172.21.15.38:0/2393975768 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1340cd210 con 0x7fd1341635e0 2024-04-25T08:35:33.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.627+0000 7fd132ffd640 1 --2- 172.21.15.38:0/2393975768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd13406abe0 0x7fd13406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:33.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.628+0000 7fd1337fe640 1 --2- 172.21.15.38:0/2393975768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1341635e0 0x7fd1341659d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd124009e50 tx=0x7fd12402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfe1fb5c3af62251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:33.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.628+0000 7fd131ffb640 1 -- 172.21.15.38:0/2393975768 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124036070 con 0x7fd1341635e0 2024-04-25T08:35:33.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.628+0000 7fd131ffb640 1 -- 172.21.15.38:0/2393975768 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd124031600 con 0x7fd1341635e0 2024-04-25T08:35:33.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.628+0000 7fd131ffb640 1 -- 172.21.15.38:0/2393975768 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1240319d0 con 0x7fd1341635e0 2024-04-25T08:35:33.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.628+0000 7fd13966b640 1 -- 172.21.15.38:0/2393975768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1341635e0 msgr2=0x7fd1341659d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.628+0000 7fd13966b640 1 --2- 172.21.15.38:0/2393975768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1341635e0 0x7fd1341659d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd124009e50 tx=0x7fd12402a680 comp rx=0 tx=0).stop 2024-04-25T08:35:33.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.629+0000 7fd13966b640 1 -- 172.21.15.38:0/2393975768 shutdown_connections 2024-04-25T08:35:33.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.629+0000 7fd13966b640 1 --2- 172.21.15.38:0/2393975768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1341635e0 0x7fd1341659d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.629+0000 7fd13966b640 1 --2- 172.21.15.38:0/2393975768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd13406abe0 0x7fd13406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.629+0000 7fd13966b640 1 -- 172.21.15.38:0/2393975768 >> 172.21.15.38:0/2393975768 conn(0x7fd1340d06b0 msgr2=0x7fd1340d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:33.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.629+0000 7fd13966b640 1 -- 172.21.15.38:0/2393975768 shutdown_connections 2024-04-25T08:35:33.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.629+0000 7fd13966b640 1 -- 172.21.15.38:0/2393975768 wait complete. 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.630+0000 7fd13966b640 1 Processor -- start 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.630+0000 7fd13966b640 1 -- start start 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd13966b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13406abe0 0x7fd134159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd13966b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340cc2c0 0x7fd13415e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd13966b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1341635e0 0x7fd13415e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd13966b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd134159790 con 0x7fd1341635e0 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd13966b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340cd7d0 con 0x7fd13406abe0 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd13966b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd134159910 con 0x7fd1340cc2c0 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd1337fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1341635e0 0x7fd13415e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd1337fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1341635e0 0x7fd13415e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42990/0 (socket says 172.21.15.38:42990) 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd1337fe640 1 -- 172.21.15.38:0/1993439512 learned_addr learned my addr 172.21.15.38:0/1993439512 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd132ffd640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13406abe0 0x7fd134159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd1327fc640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340cc2c0 0x7fd13415e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd1337fe640 1 -- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340cc2c0 msgr2=0x7fd13415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd1337fe640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340cc2c0 0x7fd13415e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd1337fe640 1 -- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13406abe0 msgr2=0x7fd134159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd1337fe640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13406abe0 0x7fd134159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd1337fe640 1 -- 172.21.15.38:0/1993439512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1340c6490 con 0x7fd1341635e0 2024-04-25T08:35:33.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd1327fc640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340cc2c0 0x7fd13415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:33.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd1337fe640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1341635e0 0x7fd13415e6e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd124034040 tx=0x7fd124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:33.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd11b7fe640 1 -- 172.21.15.38:0/1993439512 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124041020 con 0x7fd1341635e0 2024-04-25T08:35:33.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd11b7fe640 1 -- 172.21.15.38:0/1993439512 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd124037040 con 0x7fd1341635e0 2024-04-25T08:35:33.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.631+0000 7fd13966b640 1 -- 172.21.15.38:0/1993439512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1340c6770 con 0x7fd1341635e0 2024-04-25T08:35:33.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.632+0000 7fd11b7fe640 1 -- 172.21.15.38:0/1993439512 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124036050 con 0x7fd1341635e0 2024-04-25T08:35:33.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.632+0000 7fd13966b640 1 -- 172.21.15.38:0/1993439512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1340c7010 con 0x7fd1341635e0 2024-04-25T08:35:33.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.633+0000 7fd13966b640 1 -- 172.21.15.38:0/1993439512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd134068e80 con 0x7fd1341635e0 2024-04-25T08:35:33.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.633+0000 7fd11b7fe640 1 -- 172.21.15.38:0/1993439512 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd124007bd0 con 0x7fd1341635e0 2024-04-25T08:35:33.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.633+0000 7fd11b7fe640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd12006b4d0 0x7fd12006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.634+0000 7fd11b7fe640 1 -- 172.21.15.38:0/1993439512 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd1240aa610 con 0x7fd1341635e0 2024-04-25T08:35:33.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.634+0000 7fd132ffd640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd12006b4d0 0x7fd12006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.634+0000 7fd132ffd640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd12006b4d0 0x7fd12006d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd1340ca780 tx=0x7fd12c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:33.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.638+0000 7fd11b7fe640 1 -- 172.21.15.38:0/1993439512 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd12404b060 con 0x7fd1341635e0 2024-04-25T08:35:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.817+0000 7fd13966b640 1 -- 172.21.15.38:0/1993439512 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd1340c6940 con 0x7fd1341635e0 2024-04-25T08:35:33.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.818+0000 7fd11b7fe640 1 -- 172.21.15.38:0/1993439512 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd124079690 con 0x7fd1341635e0 2024-04-25T08:35:33.812 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:35:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.820+0000 7fd13966b640 1 -- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd12006b4d0 msgr2=0x7fd12006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.820+0000 7fd13966b640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd12006b4d0 0x7fd12006d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd1340ca780 tx=0x7fd12c009190 comp rx=0 tx=0).stop 2024-04-25T08:35:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.820+0000 7fd13966b640 1 -- 172.21.15.38:0/1993439512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1341635e0 msgr2=0x7fd13415e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.820+0000 7fd13966b640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1341635e0 0x7fd13415e6e0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd124034040 tx=0x7fd124004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.820+0000 7fd13966b640 1 -- 172.21.15.38:0/1993439512 shutdown_connections 2024-04-25T08:35:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.820+0000 7fd13966b640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1341635e0 0x7fd13415e6e0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.820+0000 7fd13966b640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd12006b4d0 0x7fd12006d990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.820+0000 7fd13966b640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340cc2c0 0x7fd13415e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.820+0000 7fd13966b640 1 --2- 172.21.15.38:0/1993439512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13406abe0 0x7fd134159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.820+0000 7fd13966b640 1 -- 172.21.15.38:0/1993439512 >> 172.21.15.38:0/1993439512 conn(0x7fd1340d06b0 msgr2=0x7fd1340d0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.820+0000 7fd13966b640 1 -- 172.21.15.38:0/1993439512 shutdown_connections 2024-04-25T08:35:33.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.820+0000 7fd13966b640 1 -- 172.21.15.38:0/1993439512 wait complete. 2024-04-25T08:35:33.826 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T08:35:33.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.932+0000 7fb36450a640 1 Processor -- start 2024-04-25T08:35:33.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.932+0000 7fb36450a640 1 -- start start 2024-04-25T08:35:33.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.933+0000 7fb36450a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb35c099a70 0x7fb35c099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.933+0000 7fb36450a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb35c06cd90 0x7fb35c167710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.933+0000 7fb36450a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35c09a390 con 0x7fb35c0cf470 2024-04-25T08:35:33.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.933+0000 7fb36450a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35c1614d0 con 0x7fb35c099a70 2024-04-25T08:35:33.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.933+0000 7fb36450a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0cd970 con 0x7fb35c06cd90 2024-04-25T08:35:33.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.933+0000 7fb3622a6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb35c099a70 0x7fb35c099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.933+0000 7fb3622a6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb35c099a70 0x7fb35c099e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42992/0 (socket says 172.21.15.38:42992) 2024-04-25T08:35:33.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.933+0000 7fb3622a6640 1 -- 172.21.15.38:0/754119606 learned_addr learned my addr 172.21.15.38:0/754119606 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:33.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.933+0000 7fb362aa7640 1 --2- 172.21.15.38:0/754119606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb35c06cd90 0x7fb35c167710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.934+0000 7fb3622a6640 1 -- 172.21.15.38:0/754119606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb35c06cd90 msgr2=0x7fb35c167710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:33.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.934+0000 7fb3622a6640 1 --2- 172.21.15.38:0/754119606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb35c06cd90 0x7fb35c167710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.934+0000 7fb3622a6640 1 -- 172.21.15.38:0/754119606 >> v1:172.21.15.107:6789/0 conn(0x7fb35c0cf470 legacy=0x7fb35c0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.934+0000 7fb3622a6640 1 -- 172.21.15.38:0/754119606 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c0ccd60 con 0x7fb35c099a70 2024-04-25T08:35:33.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.934+0000 7fb362aa7640 1 --2- 172.21.15.38:0/754119606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb35c06cd90 0x7fb35c167710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:33.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.934+0000 7fb3622a6640 1 --2- 172.21.15.38:0/754119606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb35c099a70 0x7fb35c099e50 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb358009870 tx=0x7fb35802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b900860bbec1ae6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:33.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.934+0000 7fb3612a4640 1 -- 172.21.15.38:0/754119606 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358037070 con 0x7fb35c099a70 2024-04-25T08:35:33.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.935+0000 7fb3612a4640 1 -- 172.21.15.38:0/754119606 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb358032440 con 0x7fb35c099a70 2024-04-25T08:35:33.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.935+0000 7fb3612a4640 1 -- 172.21.15.38:0/754119606 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358032810 con 0x7fb35c099a70 2024-04-25T08:35:33.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.935+0000 7fb36450a640 1 -- 172.21.15.38:0/754119606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb35c099a70 msgr2=0x7fb35c099e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.935+0000 7fb36450a640 1 --2- 172.21.15.38:0/754119606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb35c099a70 0x7fb35c099e50 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fb358009870 tx=0x7fb35802a250 comp rx=0 tx=0).stop 2024-04-25T08:35:33.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.935+0000 7fb36450a640 1 -- 172.21.15.38:0/754119606 shutdown_connections 2024-04-25T08:35:33.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.935+0000 7fb36450a640 1 --2- 172.21.15.38:0/754119606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb35c06cd90 0x7fb35c167710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.935+0000 7fb36450a640 1 --2- 172.21.15.38:0/754119606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb35c099a70 0x7fb35c099e50 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.935+0000 7fb36450a640 1 -- 172.21.15.38:0/754119606 >> 172.21.15.38:0/754119606 conn(0x7fb35c0b3180 msgr2=0x7fb35c0b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:33.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.935+0000 7fb36450a640 1 -- 172.21.15.38:0/754119606 shutdown_connections 2024-04-25T08:35:33.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.935+0000 7fb36450a640 1 -- 172.21.15.38:0/754119606 wait complete. 2024-04-25T08:35:33.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.936+0000 7fb36450a640 1 Processor -- start 2024-04-25T08:35:33.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.936+0000 7fb36450a640 1 -- start start 2024-04-25T08:35:33.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.937+0000 7fb36450a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb35c06cd90 0x7fb35c1f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.937+0000 7fb36450a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb35c099a70 0x7fb35c156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.937+0000 7fb36450a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb35c0cf470 0x7fb35c0b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.937+0000 7fb36450a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0cd600 con 0x7fb35c06cd90 2024-04-25T08:35:33.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.937+0000 7fb36450a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0cd320 con 0x7fb35c0cf470 2024-04-25T08:35:33.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.937+0000 7fb36450a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0d0780 con 0x7fb35c099a70 2024-04-25T08:35:33.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.937+0000 7fb3622a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb35c06cd90 0x7fb35c1f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.937+0000 7fb3622a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb35c06cd90 0x7fb35c1f73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43006/0 (socket says 172.21.15.38:43006) 2024-04-25T08:35:33.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.937+0000 7fb3622a6640 1 -- 172.21.15.38:0/1236916774 learned_addr learned my addr 172.21.15.38:0/1236916774 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:33.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.937+0000 7fb362aa7640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb35c0cf470 0x7fb35c0b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.937+0000 7fb361aa5640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb35c099a70 0x7fb35c156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.938+0000 7fb3622a6640 1 -- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb35c099a70 msgr2=0x7fb35c156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.938+0000 7fb3622a6640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb35c099a70 0x7fb35c156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.938+0000 7fb3622a6640 1 -- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb35c0cf470 msgr2=0x7fb35c0b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.938+0000 7fb3622a6640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb35c0cf470 0x7fb35c0b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.938+0000 7fb3622a6640 1 -- 172.21.15.38:0/1236916774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c1f7990 con 0x7fb35c06cd90 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.938+0000 7fb362aa7640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb35c0cf470 0x7fb35c0b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.938+0000 7fb3622a6640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb35c06cd90 0x7fb35c1f73b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb3580097c0 tx=0x7fb3580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.938+0000 7fb352ffd640 1 -- 172.21.15.38:0/1236916774 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358037040 con 0x7fb35c06cd90 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.938+0000 7fb352ffd640 1 -- 172.21.15.38:0/1236916774 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb358042070 con 0x7fb35c06cd90 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.938+0000 7fb36450a640 1 -- 172.21.15.38:0/1236916774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35c1f7c70 con 0x7fb35c06cd90 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.938+0000 7fb352ffd640 1 -- 172.21.15.38:0/1236916774 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358030bb0 con 0x7fb35c06cd90 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.938+0000 7fb36450a640 1 -- 172.21.15.38:0/1236916774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35c1f8230 con 0x7fb35c06cd90 2024-04-25T08:35:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.939+0000 7fb36450a640 1 -- 172.21.15.38:0/1236916774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35c068e80 con 0x7fb35c06cd90 2024-04-25T08:35:33.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.939+0000 7fb352ffd640 1 -- 172.21.15.38:0/1236916774 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb358047460 con 0x7fb35c06cd90 2024-04-25T08:35:33.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.940+0000 7fb352ffd640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb34006b4d0 0x7fb34006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:33.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.940+0000 7fb352ffd640 1 -- 172.21.15.38:0/1236916774 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb3580aaca0 con 0x7fb35c06cd90 2024-04-25T08:35:33.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.940+0000 7fb361aa5640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb34006b4d0 0x7fb34006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:33.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.940+0000 7fb361aa5640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb34006b4d0 0x7fb34006d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb35c1572f0 tx=0x7fb34c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:33.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:33.944+0000 7fb352ffd640 1 -- 172.21.15.38:0/1236916774 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb358045360 con 0x7fb35c06cd90 2024-04-25T08:35:34.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:34.198+0000 7fb36450a640 1 -- 172.21.15.38:0/1236916774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7fb35c0cf960 con 0x7fb35c06cd90 2024-04-25T08:35:35.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.201+0000 7fb352ffd640 1 -- 172.21.15.38:0/1236916774 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v209) v1 ==== 146+0+0 (secure 0 0 0) 0x7fb358079c80 con 0x7fb35c06cd90 2024-04-25T08:35:35.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.204+0000 7fb36450a640 1 -- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb34006b4d0 msgr2=0x7fb34006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.204+0000 7fb36450a640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb34006b4d0 0x7fb34006d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb35c1572f0 tx=0x7fb34c009040 comp rx=0 tx=0).stop 2024-04-25T08:35:35.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.204+0000 7fb36450a640 1 -- 172.21.15.38:0/1236916774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb35c06cd90 msgr2=0x7fb35c1f73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.204+0000 7fb36450a640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb35c06cd90 0x7fb35c1f73b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fb3580097c0 tx=0x7fb3580046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.204+0000 7fb36450a640 1 -- 172.21.15.38:0/1236916774 shutdown_connections 2024-04-25T08:35:35.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.204+0000 7fb36450a640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb35c0cf470 0x7fb35c0b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.204+0000 7fb36450a640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb34006b4d0 0x7fb34006d990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.204+0000 7fb36450a640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb35c099a70 0x7fb35c156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.204+0000 7fb36450a640 1 --2- 172.21.15.38:0/1236916774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb35c06cd90 0x7fb35c1f73b0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.204+0000 7fb36450a640 1 -- 172.21.15.38:0/1236916774 >> 172.21.15.38:0/1236916774 conn(0x7fb35c0b3180 msgr2=0x7fb35c0b08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:35.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.204+0000 7fb36450a640 1 -- 172.21.15.38:0/1236916774 shutdown_connections 2024-04-25T08:35:35.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.204+0000 7fb36450a640 1 -- 172.21.15.38:0/1236916774 wait complete. 2024-04-25T08:35:35.212 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:35:35.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.329+0000 7f5ea562a640 1 Processor -- start 2024-04-25T08:35:35.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.329+0000 7f5ea562a640 1 -- start start 2024-04-25T08:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.330+0000 7f5ea562a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ea00d1d20 0x7f5ea00cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.330+0000 7f5ea562a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5ea015f5f0 0x7f5ea00b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.330+0000 7f5ea562a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea015fd00 con 0x7f5ea015b990 2024-04-25T08:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.330+0000 7f5ea562a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea0160580 con 0x7f5ea015f5f0 2024-04-25T08:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.330+0000 7f5ea562a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea015faa0 con 0x7f5ea00d1d20 2024-04-25T08:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.330+0000 7f5e9f7fe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5ea015f5f0 0x7f5ea00b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.330+0000 7f5e9e7fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ea00d1d20 0x7f5ea00cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.330+0000 7f5e9effd640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f5ea015b990 0x7f5ea00d1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:39078/0 (socket says 172.21.15.38:39078) 2024-04-25T08:35:35.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.330+0000 7f5e9effd640 1 -- 172.21.15.38:0/3324868970 learned_addr learned my addr 172.21.15.38:0/3324868970 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:35.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.331+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/3324868970 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ea00d1d20 msgr2=0x7f5ea00cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:35.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.331+0000 7f5e9f7fe640 1 --2- 172.21.15.38:0/3324868970 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ea00d1d20 0x7f5ea00cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.331+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/3324868970 >> v1:172.21.15.107:6789/0 conn(0x7f5ea015b990 legacy=0x7f5ea00d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.331+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/3324868970 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea015ffc0 con 0x7f5ea015f5f0 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.331+0000 7f5e9f7fe640 1 --2- 172.21.15.38:0/3324868970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ea015f5f0 0x7f5ea00b1fe0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009dc0 tx=0x7f5e9802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=256fb51cfbf8db15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.332+0000 7f5e9dffb640 1 -- 172.21.15.38:0/3324868970 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98007740 con 0x7f5ea015f5f0 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.332+0000 7f5e9dffb640 1 -- 172.21.15.38:0/3324868970 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e980078e0 con 0x7f5ea015f5f0 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.332+0000 7f5e9dffb640 1 -- 172.21.15.38:0/3324868970 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98007cb0 con 0x7f5ea015f5f0 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.332+0000 7f5ea562a640 1 -- 172.21.15.38:0/3324868970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ea015f5f0 msgr2=0x7f5ea00b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.332+0000 7f5ea562a640 1 --2- 172.21.15.38:0/3324868970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ea015f5f0 0x7f5ea00b1fe0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009dc0 tx=0x7f5e9802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.332+0000 7f5ea562a640 1 -- 172.21.15.38:0/3324868970 shutdown_connections 2024-04-25T08:35:35.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.332+0000 7f5ea562a640 1 --2- 172.21.15.38:0/3324868970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ea015f5f0 0x7f5ea00b1fe0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.332+0000 7f5ea562a640 1 --2- 172.21.15.38:0/3324868970 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ea00d1d20 0x7f5ea00cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.332+0000 7f5ea562a640 1 -- 172.21.15.38:0/3324868970 >> 172.21.15.38:0/3324868970 conn(0x7f5ea008b000 msgr2=0x7f5ea008b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:35.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.332+0000 7f5ea562a640 1 -- 172.21.15.38:0/3324868970 shutdown_connections 2024-04-25T08:35:35.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.332+0000 7f5ea562a640 1 -- 172.21.15.38:0/3324868970 wait complete. 2024-04-25T08:35:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.333+0000 7f5ea562a640 1 Processor -- start 2024-04-25T08:35:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.334+0000 7f5ea562a640 1 -- start start 2024-04-25T08:35:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.334+0000 7f5ea562a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ea00d1d20 0x7f5ea01f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.334+0000 7f5ea562a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ea015b990 0x7f5ea00c51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.334+0000 7f5ea562a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ea015f5f0 0x7f5ea00c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.334+0000 7f5ea562a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea015bd70 con 0x7f5ea015f5f0 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.334+0000 7f5ea562a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00b1ac0 con 0x7f5ea015b990 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.334+0000 7f5ea562a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea015bef0 con 0x7f5ea00d1d20 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.334+0000 7f5e9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ea015f5f0 0x7f5ea00c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.334+0000 7f5e9f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ea015f5f0 0x7f5ea00c56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43022/0 (socket says 172.21.15.38:43022) 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.334+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/2289866763 learned_addr learned my addr 172.21.15.38:0/2289866763 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5e9effd640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ea00d1d20 0x7f5ea01f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5e9e7fc640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ea015b990 0x7f5ea00c51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ea00d1d20 msgr2=0x7f5ea01f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5e9f7fe640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ea00d1d20 0x7f5ea01f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ea015b990 msgr2=0x7f5ea00c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5e9f7fe640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ea015b990 0x7f5ea00c51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5e9f7fe640 1 -- 172.21.15.38:0/2289866763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea00c7d10 con 0x7f5ea015f5f0 2024-04-25T08:35:35.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5e9f7fe640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ea015f5f0 0x7f5ea00c56f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009ee0 tx=0x7f5e98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:35.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5e877fe640 1 -- 172.21.15.38:0/2289866763 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9803f070 con 0x7f5ea015f5f0 2024-04-25T08:35:35.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5e877fe640 1 -- 172.21.15.38:0/2289866763 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e98031bc0 con 0x7f5ea015f5f0 2024-04-25T08:35:35.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5ea562a640 1 -- 172.21.15.38:0/2289866763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea00c7ff0 con 0x7f5ea015f5f0 2024-04-25T08:35:35.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5e877fe640 1 -- 172.21.15.38:0/2289866763 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e98009450 con 0x7f5ea015f5f0 2024-04-25T08:35:35.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.335+0000 7f5ea562a640 1 -- 172.21.15.38:0/2289866763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea015d3d0 con 0x7f5ea015f5f0 2024-04-25T08:35:35.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.336+0000 7f5ea562a640 1 -- 172.21.15.38:0/2289866763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea015d5d0 con 0x7f5ea015f5f0 2024-04-25T08:35:35.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.337+0000 7f5e877fe640 1 -- 172.21.15.38:0/2289866763 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e9803b410 con 0x7f5ea015f5f0 2024-04-25T08:35:35.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.337+0000 7f5e877fe640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e8c06b500 0x7f5e8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.337+0000 7f5e877fe640 1 -- 172.21.15.38:0/2289866763 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5e980ab630 con 0x7f5ea015f5f0 2024-04-25T08:35:35.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.337+0000 7f5e9effd640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e8c06b500 0x7f5e8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.338+0000 7f5e9effd640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e8c06b500 0x7f5e8c06d9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0158450 tx=0x7f5e90007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:35.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.341+0000 7f5e877fe640 1 -- 172.21.15.38:0/2289866763 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ea015d5d0 con 0x7f5ea015f5f0 2024-04-25T08:35:35.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.521+0000 7f5ea562a640 1 -- 172.21.15.38:0/2289866763 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5ea0068e80 con 0x7f5ea015f5f0 2024-04-25T08:35:35.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.526+0000 7f5e877fe640 1 -- 172.21.15.38:0/2289866763 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5e98079fd0 con 0x7f5ea015f5f0 2024-04-25T08:35:35.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.527+0000 7f5ea562a640 1 -- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e8c06b500 msgr2=0x7f5e8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.528+0000 7f5ea562a640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e8c06b500 0x7f5e8c06d9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0158450 tx=0x7f5e90007e80 comp rx=0 tx=0).stop 2024-04-25T08:35:35.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.528+0000 7f5ea562a640 1 -- 172.21.15.38:0/2289866763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ea015f5f0 msgr2=0x7f5ea00c56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.528+0000 7f5ea562a640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ea015f5f0 0x7f5ea00c56f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f5e98009ee0 tx=0x7f5e98004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:35.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.528+0000 7f5ea562a640 1 -- 172.21.15.38:0/2289866763 shutdown_connections 2024-04-25T08:35:35.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.528+0000 7f5ea562a640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ea015f5f0 0x7f5ea00c56f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.528+0000 7f5ea562a640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ea015b990 0x7f5ea00c51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.528+0000 7f5ea562a640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e8c06b500 0x7f5e8c06d9c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.528+0000 7f5ea562a640 1 --2- 172.21.15.38:0/2289866763 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ea00d1d20 0x7f5ea01f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.528+0000 7f5ea562a640 1 -- 172.21.15.38:0/2289866763 >> 172.21.15.38:0/2289866763 conn(0x7f5ea008b000 msgr2=0x7f5ea00cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:35.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.528+0000 7f5ea562a640 1 -- 172.21.15.38:0/2289866763 shutdown_connections 2024-04-25T08:35:35.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.528+0000 7f5ea562a640 1 -- 172.21.15.38:0/2289866763 wait complete. 2024-04-25T08:35:35.533 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:35:35.534 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:35:35.573 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:35:35.618 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:35:35.618 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:35:35.618 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:35.634 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:35:35.634 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:35:35.634 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:35.651 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:35:35.652 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:35:35.652 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:35.689 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:35:35.690 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:35:35.690 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:35.710 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:35:35.710 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:35:35.710 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:35.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.836+0000 7f9f9d2dd640 1 Processor -- start 2024-04-25T08:35:35.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.836+0000 7f9f9d2dd640 1 -- start start 2024-04-25T08:35:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.837+0000 7f9f9d2dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f9809ab90 0x7f9f9815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.837+0000 7f9f9d2dd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9f980d0460 0x7f9f9815fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.837+0000 7f9f9d2dd640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f98160340 con 0x7f9f9815bb10 2024-04-25T08:35:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.837+0000 7f9f9d2dd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f980cffd0 con 0x7f9f980d0460 2024-04-25T08:35:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.837+0000 7f9f9d2dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980cf400 con 0x7f9f9809ab90 2024-04-25T08:35:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.837+0000 7f9f96ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9f9815bb10 0x7f9f9809a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45164/0 (socket says 172.21.15.38:45164) 2024-04-25T08:35:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.837+0000 7f9f96ffd640 1 -- 172.21.15.38:0/3459391690 learned_addr learned my addr 172.21.15.38:0/3459391690 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:35.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.837+0000 7f9f977fe640 1 --2- 172.21.15.38:0/3459391690 >> v2:172.21.15.38:3300/0 conn(0x7f9f980d0460 0x7f9f9815fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.838+0000 7f9f967fc640 1 --2- 172.21.15.38:0/3459391690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f9809ab90 0x7f9f9815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.838+0000 7f9f95ffb640 1 -- 172.21.15.38:0/3459391690 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616902098 0 0) 0x7f9f90001110 con 0x7f9f9815bb10 2024-04-25T08:35:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.838+0000 7f9f95ffb640 1 -- 172.21.15.38:0/3459391690 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f80003520 con 0x7f9f9815bb10 2024-04-25T08:35:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.838+0000 7f9f977fe640 1 -- 172.21.15.38:0/3459391690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f9809ab90 msgr2=0x7f9f9815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.838+0000 7f9f977fe640 1 --2- 172.21.15.38:0/3459391690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f9809ab90 0x7f9f9815f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.838+0000 7f9f977fe640 1 -- 172.21.15.38:0/3459391690 >> v1:172.21.15.38:6789/0 conn(0x7f9f9815bb10 legacy=0x7f9f9809a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.838+0000 7f9f977fe640 1 -- 172.21.15.38:0/3459391690 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f980cf6a0 con 0x7f9f980d0460 2024-04-25T08:35:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.838+0000 7f9f977fe640 1 --2- 172.21.15.38:0/3459391690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f980d0460 0x7f9f9815fa20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009d90 tx=0x7f9f8802a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0db339faaca3ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.839+0000 7f9f95ffb640 1 -- 172.21.15.38:0/3459391690 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88007740 con 0x7f9f980d0460 2024-04-25T08:35:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.839+0000 7f9f95ffb640 1 -- 172.21.15.38:0/3459391690 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f880078e0 con 0x7f9f980d0460 2024-04-25T08:35:35.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.839+0000 7f9f95ffb640 1 -- 172.21.15.38:0/3459391690 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88007cb0 con 0x7f9f980d0460 2024-04-25T08:35:35.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.839+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3459391690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f980d0460 msgr2=0x7f9f9815fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.839+0000 7f9f9d2dd640 1 --2- 172.21.15.38:0/3459391690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f980d0460 0x7f9f9815fa20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009d90 tx=0x7f9f8802a530 comp rx=0 tx=0).stop 2024-04-25T08:35:35.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.839+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3459391690 shutdown_connections 2024-04-25T08:35:35.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.839+0000 7f9f9d2dd640 1 --2- 172.21.15.38:0/3459391690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f980d0460 0x7f9f9815fa20 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.839+0000 7f9f9d2dd640 1 --2- 172.21.15.38:0/3459391690 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f9809ab90 0x7f9f9815f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.839+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3459391690 >> 172.21.15.38:0/3459391690 conn(0x7f9f980b31f0 msgr2=0x7f9f980b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:35.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.839+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3459391690 shutdown_connections 2024-04-25T08:35:35.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.839+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3459391690 wait complete. 2024-04-25T08:35:35.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.840+0000 7f9f9d2dd640 1 Processor -- start 2024-04-25T08:35:35.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.841+0000 7f9f9d2dd640 1 -- start start 2024-04-25T08:35:35.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.841+0000 7f9f9d2dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f9809ab90 0x7f9f98156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.841+0000 7f9f9d2dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f980d0460 0x7f9f9815bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.841+0000 7f9f9d2dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f9815bb10 0x7f9f9815d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.841+0000 7f9f9d2dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980c5250 con 0x7f9f9815bb10 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.841+0000 7f9f9d2dd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980cfc60 con 0x7f9f9809ab90 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.841+0000 7f9f9d2dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980c53d0 con 0x7f9f980d0460 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.841+0000 7f9f977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f9815bb10 0x7f9f9815d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.841+0000 7f9f977fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f9815bb10 0x7f9f9815d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43044/0 (socket says 172.21.15.38:43044) 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.841+0000 7f9f977fe640 1 -- 172.21.15.38:0/3074306136 learned_addr learned my addr 172.21.15.38:0/3074306136 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f967fc640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f980d0460 0x7f9f9815bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f96ffd640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f9809ab90 0x7f9f98156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f977fe640 1 -- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f980d0460 msgr2=0x7f9f9815bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f977fe640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f980d0460 0x7f9f9815bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f977fe640 1 -- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f9809ab90 msgr2=0x7f9f98156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f977fe640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f9809ab90 0x7f9f98156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:35.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f977fe640 1 -- 172.21.15.38:0/3074306136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9815c5b0 con 0x7f9f9815bb10 2024-04-25T08:35:35.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f977fe640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f9815bb10 0x7f9f9815d660 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009eb0 tx=0x7f9f88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:35.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f7f7fe640 1 -- 172.21.15.38:0/3074306136 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88040070 con 0x7f9f9815bb10 2024-04-25T08:35:35.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f7f7fe640 1 -- 172.21.15.38:0/3074306136 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f88031bc0 con 0x7f9f9815bb10 2024-04-25T08:35:35.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3074306136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9815c890 con 0x7f9f9815bb10 2024-04-25T08:35:35.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f7f7fe640 1 -- 172.21.15.38:0/3074306136 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88009450 con 0x7f9f9815bb10 2024-04-25T08:35:35.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.842+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3074306136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9815ce50 con 0x7f9f9815bb10 2024-04-25T08:35:35.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.843+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3074306136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9815d230 con 0x7f9f9815bb10 2024-04-25T08:35:35.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.843+0000 7f9f7f7fe640 1 -- 172.21.15.38:0/3074306136 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f88039040 con 0x7f9f9815bb10 2024-04-25T08:35:35.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.844+0000 7f9f7f7fe640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f8406b4d0 0x7f9f8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:35.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.844+0000 7f9f7f7fe640 1 -- 172.21.15.38:0/3074306136 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9f880ab8b0 con 0x7f9f9815bb10 2024-04-25T08:35:35.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.844+0000 7f9f96ffd640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f8406b4d0 0x7f9f8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:35.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.845+0000 7f9f96ffd640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f8406b4d0 0x7f9f8406d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9f90007fd0 tx=0x7f9f90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:35.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:35.848+0000 7f9f7f7fe640 1 -- 172.21.15.38:0/3074306136 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f88043360 con 0x7f9f9815bb10 2024-04-25T08:35:36.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.065+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3074306136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f98068e80 con 0x7f9f9815bb10 2024-04-25T08:35:36.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.065+0000 7f9f7f7fe640 1 -- 172.21.15.38:0/3074306136 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f9f8807a930 con 0x7f9f9815bb10 2024-04-25T08:35:36.060 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:36.060 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5219,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4156279972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4156279972},{"type":"v1","addr":"172.21.15.38:6835","nonce":4156279972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5476,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/704742056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":704742056},{"type":"v1","addr":"172.21.15.107:6836","nonce":704742056}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3500005511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3500005511},{"type":"v1","addr":"172.21.15.107:6837","nonce":3500005511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6753,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/2851156615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2851156615},{"type":"v1","addr":"172.21.15.38:6834","nonce":2851156615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[]} 2024-04-25T08:35:36.060 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 30 2024-04-25T08:35:36.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.067+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f8406b4d0 msgr2=0x7f9f8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.067+0000 7f9f9d2dd640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f8406b4d0 0x7f9f8406d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9f90007fd0 tx=0x7f9f90009040 comp rx=0 tx=0).stop 2024-04-25T08:35:36.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.067+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3074306136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f9815bb10 msgr2=0x7f9f9815d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.067+0000 7f9f9d2dd640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f9815bb10 0x7f9f9815d660 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009eb0 tx=0x7f9f88004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:36.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.067+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3074306136 shutdown_connections 2024-04-25T08:35:36.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.067+0000 7f9f9d2dd640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f9815bb10 0x7f9f9815d660 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.067+0000 7f9f9d2dd640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f8406b4d0 0x7f9f8406d990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.067+0000 7f9f9d2dd640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f980d0460 0x7f9f9815bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.067+0000 7f9f9d2dd640 1 --2- 172.21.15.38:0/3074306136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f9809ab90 0x7f9f98156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.067+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3074306136 >> 172.21.15.38:0/3074306136 conn(0x7f9f980b31f0 msgr2=0x7f9f980c6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:36.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.067+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3074306136 shutdown_connections 2024-04-25T08:35:36.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.067+0000 7f9f9d2dd640 1 -- 172.21.15.38:0/3074306136 wait complete. 2024-04-25T08:35:36.073 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:35:36.074 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:35:36.074 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:35:36.074 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:36.074 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:35:36.074 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:35:36.074 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:35:36.074 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:36.075 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:35:36.075 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:35:36.075 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:35:36.075 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:36.075 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:35:36.075 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:35:36.075 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:35:36.075 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:36.083 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:35:36.084 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:35:36.084 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:35:36.085 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:35:36.087 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:35:36.087 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:35:36.088 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:35:36.088 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:35:36.126 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:35:36.126 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:36.126 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:35:36.127 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:36.130 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:35:36.130 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:36.131 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:35:36.131 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:36.131 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:35:36.202 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:35:36.202 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:35:36.214 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:35:36.214 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-25T08:35:36.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.244+0000 7fd1a07f4640 1 Processor -- start 2024-04-25T08:35:36.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.244+0000 7fd1a07f4640 1 -- start start 2024-04-25T08:35:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd1a07f4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd19815db70 0x7fd1980d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd1a07f4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 0x7fd1980ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd1a07f4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19815df50 con 0x7fd1980ceb20 2024-04-25T08:35:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd1a07f4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1980d1d10 con 0x7fd19815db70 2024-04-25T08:35:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd1a07f4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980d0750 con 0x7fd1980b92c0 2024-04-25T08:35:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd19dd8f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 0x7fd1980ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd19dd8f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 0x7fd1980ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:53886/0 (socket says 172.21.15.38:53886) 2024-04-25T08:35:36.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd19dd8f640 1 -- 172.21.15.38:0/4239990713 learned_addr learned my addr 172.21.15.38:0/4239990713 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:36.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd19e590640 1 --2- 172.21.15.38:0/4239990713 >> v2:172.21.15.107:3300/0 conn(0x7fd19815db70 0x7fd1980d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd19d58e640 1 -- 172.21.15.38:0/4239990713 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462713613 0 0) 0x7fd19815df50 con 0x7fd1980ceb20 2024-04-25T08:35:36.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd19e590640 1 -- 172.21.15.38:0/4239990713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 msgr2=0x7fd1980ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:36.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd19e590640 1 --2- 172.21.15.38:0/4239990713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 0x7fd1980ce5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd19e590640 1 -- 172.21.15.38:0/4239990713 >> v1:172.21.15.107:6789/0 conn(0x7fd1980ceb20 legacy=0x7fd198161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd19e590640 1 -- 172.21.15.38:0/4239990713 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1980d0a30 con 0x7fd19815db70 2024-04-25T08:35:36.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.245+0000 7fd19dd8f640 1 --2- 172.21.15.38:0/4239990713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 0x7fd1980ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:36.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.246+0000 7fd19e590640 1 --2- 172.21.15.38:0/4239990713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19815db70 0x7fd1980d00c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd1940097a0 tx=0x7fd19402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9b9ff52dd6e12b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:36.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.246+0000 7fd19d58e640 1 -- 172.21.15.38:0/4239990713 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194037070 con 0x7fd19815db70 2024-04-25T08:35:36.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.247+0000 7fd19d58e640 1 -- 172.21.15.38:0/4239990713 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd194032440 con 0x7fd19815db70 2024-04-25T08:35:36.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.247+0000 7fd19d58e640 1 -- 172.21.15.38:0/4239990713 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194032810 con 0x7fd19815db70 2024-04-25T08:35:36.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.247+0000 7fd1a07f4640 1 -- 172.21.15.38:0/4239990713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19815db70 msgr2=0x7fd1980d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.247+0000 7fd1a07f4640 1 --2- 172.21.15.38:0/4239990713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19815db70 0x7fd1980d00c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd1940097a0 tx=0x7fd19402a180 comp rx=0 tx=0).stop 2024-04-25T08:35:36.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.247+0000 7fd1a07f4640 1 -- 172.21.15.38:0/4239990713 shutdown_connections 2024-04-25T08:35:36.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.247+0000 7fd1a07f4640 1 --2- 172.21.15.38:0/4239990713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 0x7fd1980ce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.247+0000 7fd1a07f4640 1 --2- 172.21.15.38:0/4239990713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19815db70 0x7fd1980d00c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.247+0000 7fd1a07f4640 1 -- 172.21.15.38:0/4239990713 >> 172.21.15.38:0/4239990713 conn(0x7fd19806bed0 msgr2=0x7fd19806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:36.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.247+0000 7fd1a07f4640 1 -- 172.21.15.38:0/4239990713 shutdown_connections 2024-04-25T08:35:36.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.247+0000 7fd1a07f4640 1 -- 172.21.15.38:0/4239990713 wait complete. 2024-04-25T08:35:36.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.248+0000 7fd1a07f4640 1 Processor -- start 2024-04-25T08:35:36.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.248+0000 7fd1a07f4640 1 -- start start 2024-04-25T08:35:36.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.249+0000 7fd1a07f4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 0x7fd1981f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.249+0000 7fd1a07f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1980ceb20 0x7fd1981f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.249+0000 7fd1a07f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19815db70 0x7fd1980c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.249+0000 7fd1a07f4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980d0ff0 con 0x7fd1980ceb20 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.249+0000 7fd1a07f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980b9740 con 0x7fd19815db70 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.249+0000 7fd1a07f4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980d31b0 con 0x7fd1980b92c0 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.249+0000 7fd19dd8f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1980ceb20 0x7fd1981f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.249+0000 7fd19dd8f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1980ceb20 0x7fd1981f4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43070/0 (socket says 172.21.15.38:43070) 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.249+0000 7fd19dd8f640 1 -- 172.21.15.38:0/3341754607 learned_addr learned my addr 172.21.15.38:0/3341754607 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.249+0000 7fd19ed91640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19815db70 0x7fd1980c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.249+0000 7fd19e590640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 0x7fd1981f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.250+0000 7fd19dd8f640 1 -- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 msgr2=0x7fd1981f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.250+0000 7fd19dd8f640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 0x7fd1981f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.250+0000 7fd19dd8f640 1 -- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19815db70 msgr2=0x7fd1980c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.250+0000 7fd19dd8f640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19815db70 0x7fd1980c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:36.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.250+0000 7fd19dd8f640 1 -- 172.21.15.38:0/3341754607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1981f4fc0 con 0x7fd1980ceb20 2024-04-25T08:35:36.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.250+0000 7fd19e590640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 0x7fd1981f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:36.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.250+0000 7fd19dd8f640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1980ceb20 0x7fd1981f4a80 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd18800e9d0 tx=0x7fd18800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:36.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.250+0000 7fd18effd640 1 -- 172.21.15.38:0/3341754607 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd188017070 con 0x7fd1980ceb20 2024-04-25T08:35:36.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.250+0000 7fd18effd640 1 -- 172.21.15.38:0/3341754607 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18800ca60 con 0x7fd1980ceb20 2024-04-25T08:35:36.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.250+0000 7fd18effd640 1 -- 172.21.15.38:0/3341754607 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18800ce10 con 0x7fd1980ceb20 2024-04-25T08:35:36.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.250+0000 7fd1a07f4640 1 -- 172.21.15.38:0/3341754607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19815f280 con 0x7fd1980ceb20 2024-04-25T08:35:36.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.250+0000 7fd1a07f4640 1 -- 172.21.15.38:0/3341754607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1980c8b40 con 0x7fd1980ceb20 2024-04-25T08:35:36.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.251+0000 7fd1a07f4640 1 -- 172.21.15.38:0/3341754607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19804ef90 con 0x7fd1980ceb20 2024-04-25T08:35:36.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.251+0000 7fd18effd640 1 -- 172.21.15.38:0/3341754607 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd18801a070 con 0x7fd1980ceb20 2024-04-25T08:35:36.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.252+0000 7fd18effd640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd17c06b4d0 0x7fd17c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:36.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.252+0000 7fd18effd640 1 -- 172.21.15.38:0/3341754607 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd188089f40 con 0x7fd1980ceb20 2024-04-25T08:35:36.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.252+0000 7fd19e590640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd17c06b4d0 0x7fd17c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:36.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.252+0000 7fd19e590640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd17c06b4d0 0x7fd17c06d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd194034000 tx=0x7fd19402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:36.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.256+0000 7fd18effd640 1 -- 172.21.15.38:0/3341754607 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1880588e0 con 0x7fd1980ceb20 2024-04-25T08:35:36.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:36.447+0000 7fd1a07f4640 1 -- 172.21.15.38:0/3341754607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd1980b7390 con 0x7fd1980ceb20 2024-04-25T08:35:37.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.220+0000 7fd18effd640 1 -- 172.21.15.38:0/3341754607 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd18805c980 con 0x7fd1980ceb20 2024-04-25T08:35:37.215 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-04-25T08:35:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.223+0000 7fd1a07f4640 1 -- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd17c06b4d0 msgr2=0x7fd17c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.223+0000 7fd1a07f4640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd17c06b4d0 0x7fd17c06d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd194034000 tx=0x7fd19402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.223+0000 7fd1a07f4640 1 -- 172.21.15.38:0/3341754607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1980ceb20 msgr2=0x7fd1981f4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.223+0000 7fd1a07f4640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1980ceb20 0x7fd1981f4a80 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd18800e9d0 tx=0x7fd18800ee90 comp rx=0 tx=0).stop 2024-04-25T08:35:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.224+0000 7fd1a07f4640 1 -- 172.21.15.38:0/3341754607 shutdown_connections 2024-04-25T08:35:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.224+0000 7fd1a07f4640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd19815db70 0x7fd1980c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.224+0000 7fd1a07f4640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1980ceb20 0x7fd1981f4a80 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.224+0000 7fd1a07f4640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd17c06b4d0 0x7fd17c06d990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.224+0000 7fd1a07f4640 1 --2- 172.21.15.38:0/3341754607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1980b92c0 0x7fd1981f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.224+0000 7fd1a07f4640 1 -- 172.21.15.38:0/3341754607 >> 172.21.15.38:0/3341754607 conn(0x7fd19806bed0 msgr2=0x7fd1980b8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.224+0000 7fd1a07f4640 1 -- 172.21.15.38:0/3341754607 shutdown_connections 2024-04-25T08:35:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.224+0000 7fd1a07f4640 1 -- 172.21.15.38:0/3341754607 wait complete. 2024-04-25T08:35:37.233 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:35:37.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.347+0000 7f70b6aa1640 1 Processor -- start 2024-04-25T08:35:37.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.347+0000 7f70b6aa1640 1 -- start start 2024-04-25T08:35:37.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70b6aa1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f70b015df50 0x7f70b00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70b6aa1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70b00ce2e0 0x7f70b00d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70b6aa1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b00ceff0 con 0x7f70b006c110 2024-04-25T08:35:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70b6aa1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b00cf2d0 con 0x7f70b015df50 2024-04-25T08:35:37.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70b6aa1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00d1f70 con 0x7f70b00ce2e0 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70affff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f70b015df50 0x7f70b00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70b483d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f70b006c110 0x7f70b006c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45192/0 (socket says 172.21.15.38:45192) 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70b483d640 1 -- 172.21.15.38:0/2190808944 learned_addr learned my addr 172.21.15.38:0/2190808944 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70b503e640 1 --2- 172.21.15.38:0/2190808944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70b00ce2e0 0x7f70b00d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70affff640 1 -- 172.21.15.38:0/2190808944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70b00ce2e0 msgr2=0x7f70b00d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70affff640 1 --2- 172.21.15.38:0/2190808944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70b00ce2e0 0x7f70b00d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70affff640 1 -- 172.21.15.38:0/2190808944 >> v1:172.21.15.38:6789/0 conn(0x7f70b006c110 legacy=0x7f70b006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70af7fe640 1 -- 172.21.15.38:0/2190808944 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829784097 0 0) 0x7f70b00ceff0 con 0x7f70b006c110 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.348+0000 7f70affff640 1 -- 172.21.15.38:0/2190808944 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b00d2250 con 0x7f70b015df50 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.349+0000 7f70affff640 1 --2- 172.21.15.38:0/2190808944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b015df50 0x7f70b00ce7a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f709c0097a0 tx=0x7f709c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3f2bdf5bb98297 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.349+0000 7f70af7fe640 1 -- 172.21.15.38:0/2190808944 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c037070 con 0x7f70b015df50 2024-04-25T08:35:37.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.349+0000 7f70af7fe640 1 -- 172.21.15.38:0/2190808944 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709c032440 con 0x7f70b015df50 2024-04-25T08:35:37.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.349+0000 7f70af7fe640 1 -- 172.21.15.38:0/2190808944 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c032810 con 0x7f70b015df50 2024-04-25T08:35:37.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.350+0000 7f70b6aa1640 1 -- 172.21.15.38:0/2190808944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b015df50 msgr2=0x7f70b00ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.350+0000 7f70b6aa1640 1 --2- 172.21.15.38:0/2190808944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b015df50 0x7f70b00ce7a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f709c0097a0 tx=0x7f709c02a180 comp rx=0 tx=0).stop 2024-04-25T08:35:37.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.350+0000 7f70b6aa1640 1 -- 172.21.15.38:0/2190808944 shutdown_connections 2024-04-25T08:35:37.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.350+0000 7f70b6aa1640 1 --2- 172.21.15.38:0/2190808944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70b00ce2e0 0x7f70b00d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.350+0000 7f70b6aa1640 1 --2- 172.21.15.38:0/2190808944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b015df50 0x7f70b00ce7a0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.350+0000 7f70b6aa1640 1 -- 172.21.15.38:0/2190808944 >> 172.21.15.38:0/2190808944 conn(0x7f70b00d3fb0 msgr2=0x7f70b00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.350+0000 7f70b6aa1640 1 -- 172.21.15.38:0/2190808944 shutdown_connections 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.350+0000 7f70b6aa1640 1 -- 172.21.15.38:0/2190808944 wait complete. 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.351+0000 7f70b6aa1640 1 Processor -- start 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.351+0000 7f70b6aa1640 1 -- start start 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b6aa1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b006c110 0x7f70b015eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b6aa1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b00ce2e0 0x7f70b014a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b6aa1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70b015df50 0x7f70b014aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b6aa1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00d2f90 con 0x7f70b006c110 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b6aa1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00cece0 con 0x7f70b00ce2e0 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b6aa1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00d3110 con 0x7f70b015df50 2024-04-25T08:35:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b483d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b006c110 0x7f70b015eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b483d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b006c110 0x7f70b015eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43092/0 (socket says 172.21.15.38:43092) 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b483d640 1 -- 172.21.15.38:0/902815002 learned_addr learned my addr 172.21.15.38:0/902815002 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70affff640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b00ce2e0 0x7f70b014a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b503e640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70b015df50 0x7f70b014aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b483d640 1 -- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70b015df50 msgr2=0x7f70b014aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b483d640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70b015df50 0x7f70b014aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b483d640 1 -- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b00ce2e0 msgr2=0x7f70b014a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b483d640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b00ce2e0 0x7f70b014a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.352+0000 7f70b483d640 1 -- 172.21.15.38:0/902815002 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b014b430 con 0x7f70b006c110 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.353+0000 7f70b483d640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b006c110 0x7f70b015eab0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c430 tx=0x7f70a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.353+0000 7f70ad7fa640 1 -- 172.21.15.38:0/902815002 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a40074e0 con 0x7f70b006c110 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.353+0000 7f70ad7fa640 1 -- 172.21.15.38:0/902815002 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a40047e0 con 0x7f70b006c110 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.353+0000 7f70b6aa1640 1 -- 172.21.15.38:0/902815002 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b00cab60 con 0x7f70b006c110 2024-04-25T08:35:37.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.353+0000 7f70ad7fa640 1 -- 172.21.15.38:0/902815002 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4004bd0 con 0x7f70b006c110 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.353+0000 7f70b6aa1640 1 -- 172.21.15.38:0/902815002 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b00cae00 con 0x7f70b006c110 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.354+0000 7f70b6aa1640 1 -- 172.21.15.38:0/902815002 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b0068e80 con 0x7f70b006c110 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.354+0000 7f70ad7fa640 1 -- 172.21.15.38:0/902815002 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70a4011070 con 0x7f70b006c110 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.355+0000 7f70ad7fa640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f709406b4d0 0x7f709406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.355+0000 7f70ad7fa640 1 -- 172.21.15.38:0/902815002 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f70a4089ae0 con 0x7f70b006c110 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.355+0000 7f70affff640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f709406b4d0 0x7f709406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.355+0000 7f70affff640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f709406b4d0 0x7f709406d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f709c034000 tx=0x7f709c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:37.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.359+0000 7f70ad7fa640 1 -- 172.21.15.38:0/902815002 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70a40584d0 con 0x7f70b006c110 2024-04-25T08:35:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.608+0000 7f70b6aa1640 1 -- 172.21.15.38:0/902815002 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f70b01615c0 con 0x7f70b006c110 2024-04-25T08:35:37.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.608+0000 7f70ad7fa640 1 -- 172.21.15.38:0/902815002 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f70a405c570 con 0x7f70b006c110 2024-04-25T08:35:37.603 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:37.603 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:37.603 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-04-25T08:35:37.603 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.603 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-04-25T08:35:37.603 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmwq8CDBAA+cjd+QXtFgwZ3kxdABRWlA==", 2024-04-25T08:35:37.603 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmPRFbFxAAojn2r/62vvf1qTy9QVuyIw==", 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:35:37.604 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmGLjqEhAAyPRgyy5iMgtccapydv4uOQ==", 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.605 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmt0AUHhAAoQyQ7wXBMGE+7Hi84tUfAg==", 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCxFCpmv+iTExAAEJihmsvBbp9xXxPnj49D6g==", 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.606 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC0FCpmdnQKFBAADLAsYZEUhoJx292jIYZroQ==", 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.607 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC3FCpm4f5BExAAMVSPkqsrnuLcO88AXKrGHQ==", 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC6FCpmHNN7EhAABPhmnxGo70ZOd8Oex62Mdw==", 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:37.608 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDAFCpmiThGFhAAG5Pyvn/JzL9NUoxHJkPMvg==", 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDDFCpmb/FHFBAA2cMhMqQQc4Fk6j8FpjhnaQ==", 2024-04-25T08:35:37.609 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDGFCpmAZsSEhAAnGIqqcvuHMVSc8Zbk309kA==", 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.610 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDJFCpmqdVUEBAAb9abblyUztrjwxxe4jsAwQ==", 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw==", 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.611 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmL7eKKhAASeQEdHDBInfozdLVzP4DQw==", 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.612 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmH22HJxAAWyrooyJn5hRs/bb0Rn1HHw==", 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpm6Zr/LxAAxh9Y2gvvqj1jIaVVrQAvgw==", 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.613 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCpFCpmlj/fLBAAvxLluFQFR8uhppGlNpaygA==", 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcXHhCBAAvI5zmWidVTvOmEcVRWasEA==", 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcr96BRAAQwUPxILJukYERAiEXeur+g==", 2024-04-25T08:35:37.615 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:35:37.616 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:35:37.616 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:35:37.616 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:35:37.616 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.616 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:37.616 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T08:35:37.616 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.610+0000 7f70b6aa1640 1 -- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f709406b4d0 msgr2=0x7f709406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.610+0000 7f70b6aa1640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f709406b4d0 0x7f709406d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f709c034000 tx=0x7f709c02e000 comp rx=0 tx=0).stop 2024-04-25T08:35:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.610+0000 7f70b6aa1640 1 -- 172.21.15.38:0/902815002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b006c110 msgr2=0x7f70b015eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.610+0000 7f70b6aa1640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b006c110 0x7f70b015eab0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c430 tx=0x7f70a400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.610+0000 7f70b6aa1640 1 -- 172.21.15.38:0/902815002 shutdown_connections 2024-04-25T08:35:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.610+0000 7f70b6aa1640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f709406b4d0 0x7f709406d990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.610+0000 7f70b6aa1640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f70b015df50 0x7f70b014aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.610+0000 7f70b6aa1640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70b00ce2e0 0x7f70b014a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.610+0000 7f70b6aa1640 1 --2- 172.21.15.38:0/902815002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70b006c110 0x7f70b015eab0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.610+0000 7f70b6aa1640 1 -- 172.21.15.38:0/902815002 >> 172.21.15.38:0/902815002 conn(0x7f70b00d3fb0 msgr2=0x7f70b00b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.610+0000 7f70b6aa1640 1 -- 172.21.15.38:0/902815002 shutdown_connections 2024-04-25T08:35:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.610+0000 7f70b6aa1640 1 -- 172.21.15.38:0/902815002 wait complete. 2024-04-25T08:35:37.619 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:37.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.724+0000 7f2e14ff9640 1 Processor -- start 2024-04-25T08:35:37.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.724+0000 7f2e14ff9640 1 -- start start 2024-04-25T08:35:37.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.725+0000 7f2e14ff9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2e1006cc10 0x7f2e100cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.725+0000 7f2e14ff9640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2e1009a180 0x7f2e100cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.725+0000 7f2e14ff9640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e100d05c0 con 0x7f2e1006cff0 2024-04-25T08:35:37.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.725+0000 7f2e14ff9640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e100d08a0 con 0x7f2e1006cc10 2024-04-25T08:35:37.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.725+0000 7f2e14ff9640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e100d0b80 con 0x7f2e1009a180 2024-04-25T08:35:37.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.725+0000 7f2e0e59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2e1006cc10 0x7f2e100cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.725+0000 7f2e0dd9b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2e1006cff0 0x7f2e10099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45202/0 (socket says 172.21.15.38:45202) 2024-04-25T08:35:37.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.725+0000 7f2e0dd9b640 1 -- 172.21.15.38:0/4234038502 learned_addr learned my addr 172.21.15.38:0/4234038502 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:37.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.726+0000 7f2e0ed9d640 1 --2- 172.21.15.38:0/4234038502 >> v2:172.21.15.107:3300/0 conn(0x7f2e1009a180 0x7f2e100cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.726+0000 7f2e0e59c640 1 -- 172.21.15.38:0/4234038502 >> v2:172.21.15.107:3300/0 conn(0x7f2e1009a180 msgr2=0x7f2e100cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:37.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.726+0000 7f2e0e59c640 1 --2- 172.21.15.38:0/4234038502 >> v2:172.21.15.107:3300/0 conn(0x7f2e1009a180 0x7f2e100cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.726+0000 7f2e0e59c640 1 -- 172.21.15.38:0/4234038502 >> v1:172.21.15.38:6789/0 conn(0x7f2e1006cff0 legacy=0x7f2e10099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.726+0000 7f2e0e59c640 1 -- 172.21.15.38:0/4234038502 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e100d0e60 con 0x7f2e1006cc10 2024-04-25T08:35:37.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.726+0000 7f2e0e59c640 1 --2- 172.21.15.38:0/4234038502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1006cc10 0x7f2e100cfa70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2e10161700 tx=0x7f2e0802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7144e39b020677 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:37.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.727+0000 7f2e0d59a640 1 -- 172.21.15.38:0/4234038502 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e08037070 con 0x7f2e1006cc10 2024-04-25T08:35:37.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.727+0000 7f2e0d59a640 1 -- 172.21.15.38:0/4234038502 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e08032440 con 0x7f2e1006cc10 2024-04-25T08:35:37.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.727+0000 7f2e0d59a640 1 -- 172.21.15.38:0/4234038502 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e08032810 con 0x7f2e1006cc10 2024-04-25T08:35:37.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.727+0000 7f2e14ff9640 1 -- 172.21.15.38:0/4234038502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1006cc10 msgr2=0x7f2e100cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.727+0000 7f2e14ff9640 1 --2- 172.21.15.38:0/4234038502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1006cc10 0x7f2e100cfa70 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2e10161700 tx=0x7f2e0802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.727+0000 7f2e14ff9640 1 -- 172.21.15.38:0/4234038502 shutdown_connections 2024-04-25T08:35:37.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.727+0000 7f2e14ff9640 1 --2- 172.21.15.38:0/4234038502 >> v2:172.21.15.107:3300/0 conn(0x7f2e1009a180 0x7f2e100cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.727+0000 7f2e14ff9640 1 --2- 172.21.15.38:0/4234038502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1006cc10 0x7f2e100cfa70 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.727+0000 7f2e14ff9640 1 -- 172.21.15.38:0/4234038502 >> 172.21.15.38:0/4234038502 conn(0x7f2e100b31f0 msgr2=0x7f2e100b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:37.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.727+0000 7f2e14ff9640 1 -- 172.21.15.38:0/4234038502 shutdown_connections 2024-04-25T08:35:37.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.727+0000 7f2e14ff9640 1 -- 172.21.15.38:0/4234038502 wait complete. 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.728+0000 7f2e14ff9640 1 Processor -- start 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.729+0000 7f2e14ff9640 1 -- start start 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.729+0000 7f2e14ff9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e1006cc10 0x7f2e100afb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.729+0000 7f2e14ff9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e1006cff0 0x7f2e10159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.729+0000 7f2e14ff9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1009a180 0x7f2e1015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.729+0000 7f2e14ff9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1006c2d0 con 0x7f2e1009a180 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.729+0000 7f2e14ff9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1006c150 con 0x7f2e1006cc10 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.729+0000 7f2e14ff9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1015d370 con 0x7f2e1006cff0 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.729+0000 7f2e0ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1009a180 0x7f2e1015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.729+0000 7f2e0ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1009a180 0x7f2e1015a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43100/0 (socket says 172.21.15.38:43100) 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.729+0000 7f2e0ed9d640 1 -- 172.21.15.38:0/2120835134 learned_addr learned my addr 172.21.15.38:0/2120835134 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e0e59c640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e1006cc10 0x7f2e100afb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e0dd9b640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e1006cff0 0x7f2e10159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e0ed9d640 1 -- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e1006cff0 msgr2=0x7f2e10159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e0ed9d640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e1006cff0 0x7f2e10159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e0ed9d640 1 -- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e1006cc10 msgr2=0x7f2e100afb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e0ed9d640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e1006cc10 0x7f2e100afb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e0ed9d640 1 -- 172.21.15.38:0/2120835134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e100b5270 con 0x7f2e1009a180 2024-04-25T08:35:37.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e0e59c640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e1006cc10 0x7f2e100afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:37.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e0ed9d640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1009a180 0x7f2e1015a080 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000cc80 tx=0x7f2e0000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:37.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e06ffd640 1 -- 172.21.15.38:0/2120835134 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00003990 con 0x7f2e1009a180 2024-04-25T08:35:37.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e06ffd640 1 -- 172.21.15.38:0/2120835134 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e00003b50 con 0x7f2e1009a180 2024-04-25T08:35:37.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e14ff9640 1 -- 172.21.15.38:0/2120835134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e100b5550 con 0x7f2e1009a180 2024-04-25T08:35:37.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e06ffd640 1 -- 172.21.15.38:0/2120835134 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00004020 con 0x7f2e1009a180 2024-04-25T08:35:37.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.730+0000 7f2e14ff9640 1 -- 172.21.15.38:0/2120835134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e100b5b10 con 0x7f2e1009a180 2024-04-25T08:35:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.731+0000 7f2e14ff9640 1 -- 172.21.15.38:0/2120835134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e10068e80 con 0x7f2e1009a180 2024-04-25T08:35:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.732+0000 7f2e06ffd640 1 -- 172.21.15.38:0/2120835134 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e000041c0 con 0x7f2e1009a180 2024-04-25T08:35:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.732+0000 7f2e06ffd640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2df006b4d0 0x7f2df006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.732+0000 7f2e06ffd640 1 -- 172.21.15.38:0/2120835134 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2e0008a550 con 0x7f2e1009a180 2024-04-25T08:35:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.732+0000 7f2e0e59c640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2df006b4d0 0x7f2df006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.733+0000 7f2e0e59c640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2df006b4d0 0x7f2df006d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2e08034000 tx=0x7f2e0802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:37.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.736+0000 7f2e06ffd640 1 -- 172.21.15.38:0/2120835134 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e00014030 con 0x7f2e1009a180 2024-04-25T08:35:37.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.918+0000 7f2e14ff9640 1 -- 172.21.15.38:0/2120835134 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e1006d450 con 0x7f2e1009a180 2024-04-25T08:35:37.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.919+0000 7f2e06ffd640 1 -- 172.21.15.38:0/2120835134 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+13152 (secure 0 0 0) 0x7f2e00059510 con 0x7f2e1009a180 2024-04-25T08:35:37.914 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:37.914 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:37.228630+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.921+0000 7f2e14ff9640 1 -- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2df006b4d0 msgr2=0x7f2df006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.921+0000 7f2e14ff9640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2df006b4d0 0x7f2df006d990 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f2e08034000 tx=0x7f2e0802e000 comp rx=0 tx=0).stop 2024-04-25T08:35:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.921+0000 7f2e14ff9640 1 -- 172.21.15.38:0/2120835134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1009a180 msgr2=0x7f2e1015a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.921+0000 7f2e14ff9640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1009a180 0x7f2e1015a080 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000cc80 tx=0x7f2e0000a5c0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.921+0000 7f2e14ff9640 1 -- 172.21.15.38:0/2120835134 shutdown_connections 2024-04-25T08:35:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.921+0000 7f2e14ff9640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e1009a180 0x7f2e1015a080 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.921+0000 7f2e14ff9640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2df006b4d0 0x7f2df006d990 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.921+0000 7f2e14ff9640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e1006cff0 0x7f2e10159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.921+0000 7f2e14ff9640 1 --2- 172.21.15.38:0/2120835134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e1006cc10 0x7f2e100afb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.921+0000 7f2e14ff9640 1 -- 172.21.15.38:0/2120835134 >> 172.21.15.38:0/2120835134 conn(0x7f2e100b31f0 msgr2=0x7f2e1006f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.921+0000 7f2e14ff9640 1 -- 172.21.15.38:0/2120835134 shutdown_connections 2024-04-25T08:35:37.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:37.921+0000 7f2e14ff9640 1 -- 172.21.15.38:0/2120835134 wait complete. 2024-04-25T08:35:37.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:35:37.927 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:35:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.037+0000 7fecdcb29640 1 Processor -- start 2024-04-25T08:35:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.037+0000 7fecdcb29640 1 -- start start 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.037+0000 7fecdcb29640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fecd80d33d0 0x7fecd80d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.037+0000 7fecdcb29640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd80d2330 con 0x7fecd80d1e90 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.037+0000 7fecdcb29640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd80d2610 con 0x7fecd80d0320 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.037+0000 7fecdcb29640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd80d28f0 con 0x7fecd80d33d0 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd6d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fecd80d1e90 0x7fecd80b6980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45214/0 (socket says 172.21.15.38:45214) 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd6d9d640 1 -- 172.21.15.38:0/4124682162 learned_addr learned my addr 172.21.15.38:0/4124682162 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7feccffff640 1 --2- 172.21.15.38:0/4124682162 >> v2:172.21.15.107:3300/0 conn(0x7fecd80d33d0 0x7fecd80d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342189816 0 0) 0x7fecd80d2330 con 0x7fecd80d1e90 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecc4003520 con 0x7fecd80d1e90 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356089472 0 0) 0x7fecd80d2610 con 0x7fecd80d0320 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd80d2330 con 0x7fecd80d0320 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 185404367 0 0) 0x7fecc4003520 con 0x7fecd80d1e90 2024-04-25T08:35:38.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 >> v2:172.21.15.107:3300/0 conn(0x7fecd80d33d0 msgr2=0x7fecd80d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 --2- 172.21.15.38:0/4124682162 >> v2:172.21.15.107:3300/0 conn(0x7fecd80d33d0 0x7fecd80d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 >> v1:172.21.15.107:6789/0 conn(0x7fecd80d0320 legacy=0x7fecd80d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd80d2bd0 con 0x7fecd80d1e90 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fecd00019f0 con 0x7fecd80d1e90 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 >> v1:172.21.15.38:6789/0 conn(0x7fecd80d1e90 legacy=0x7fecd80b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 --2- 172.21.15.38:0/4124682162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecc4003b40 0x7fecc4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 --2- 172.21.15.38:0/4124682162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc4004800 0x7fecc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 --2- 172.21.15.38:0/4124682162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecc40051c0 0x7fecc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd80d28f0 con 0x7fecc40051c0 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc400bfb0 con 0x7fecc4004800 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc400c1b0 con 0x7fecc4003b40 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd6d9d640 1 --2- 172.21.15.38:0/4124682162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecc40051c0 0x7fecc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7feccffff640 1 --2- 172.21.15.38:0/4124682162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc4004800 0x7fecc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.038+0000 7fecd659c640 1 --2- 172.21.15.38:0/4124682162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecc4003b40 0x7fecc4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.039+0000 7fecd6d9d640 1 -- 172.21.15.38:0/4124682162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecc4003b40 msgr2=0x7fecc4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.039+0000 7fecd6d9d640 1 --2- 172.21.15.38:0/4124682162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecc4003b40 0x7fecc4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.039+0000 7fecd6d9d640 1 -- 172.21.15.38:0/4124682162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc4004800 msgr2=0x7fecc4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.039+0000 7fecd6d9d640 1 --2- 172.21.15.38:0/4124682162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc4004800 0x7fecc4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.039+0000 7fecd6d9d640 1 -- 172.21.15.38:0/4124682162 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc4003520 con 0x7fecc40051c0 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.039+0000 7fecd6d9d640 1 -- 172.21.15.38:0/4124682162 reap_dead start 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.039+0000 7fecd6d9d640 1 --2- 172.21.15.38:0/4124682162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecc40051c0 0x7fecc400b9c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fecd806b9f0 tx=0x7fecc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efba78abb7af09d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.039+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd000a410 con 0x7fecc40051c0 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.040+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd002f070 con 0x7fecc40051c0 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.040+0000 7fecd5d9b640 1 -- 172.21.15.38:0/4124682162 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd002bbe0 con 0x7fecc40051c0 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.040+0000 7fecdcb29640 1 -- 172.21.15.38:0/4124682162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecc40051c0 msgr2=0x7fecc400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.040+0000 7fecdcb29640 1 --2- 172.21.15.38:0/4124682162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecc40051c0 0x7fecc400b9c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fecd806b9f0 tx=0x7fecc4005830 comp rx=0 tx=0).stop 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.040+0000 7fecdcb29640 1 -- 172.21.15.38:0/4124682162 shutdown_connections 2024-04-25T08:35:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.040+0000 7fecdcb29640 1 --2- 172.21.15.38:0/4124682162 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecc40051c0 0x7fecc400b9c0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.040+0000 7fecdcb29640 1 -- 172.21.15.38:0/4124682162 >> 172.21.15.38:0/4124682162 conn(0x7fecd80668f0 msgr2=0x7fecd8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:38.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.040+0000 7fecdcb29640 1 -- 172.21.15.38:0/4124682162 shutdown_connections 2024-04-25T08:35:38.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.040+0000 7fecdcb29640 1 -- 172.21.15.38:0/4124682162 wait complete. 2024-04-25T08:35:38.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.041+0000 7fecdcb29640 1 Processor -- start 2024-04-25T08:35:38.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.041+0000 7fecdcb29640 1 -- start start 2024-04-25T08:35:38.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.042+0000 7fecdcb29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc40051c0 0x7fecd815f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.042+0000 7fecdcb29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecd81f7c60 0x7fecd80b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.042+0000 7fecdcb29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecd80b5920 0x7fecd814b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.042+0000 7fecdcb29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd8161390 con 0x7fecd81f7c60 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.042+0000 7fecdcb29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd8161210 con 0x7fecc40051c0 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.042+0000 7fecdcb29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd80b9720 con 0x7fecd80b5920 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.042+0000 7feccffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecd81f7c60 0x7fecd80b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.042+0000 7feccffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecd81f7c60 0x7fecd80b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43124/0 (socket says 172.21.15.38:43124) 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.042+0000 7feccffff640 1 -- 172.21.15.38:0/4169704495 learned_addr learned my addr 172.21.15.38:0/4169704495 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.042+0000 7fecd6d9d640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecd80b5920 0x7fecd814b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.042+0000 7fecd659c640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc40051c0 0x7fecd815f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.043+0000 7feccffff640 1 -- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecd80b5920 msgr2=0x7fecd814b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.043+0000 7feccffff640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecd80b5920 0x7fecd814b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.043+0000 7feccffff640 1 -- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc40051c0 msgr2=0x7fecd815f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.043+0000 7feccffff640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc40051c0 0x7fecd815f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.043+0000 7feccffff640 1 -- 172.21.15.38:0/4169704495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd80b5ff0 con 0x7fecd81f7c60 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.043+0000 7feccffff640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecd81f7c60 0x7fecd80b53e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fecc00118e0 tx=0x7fecc0011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.043+0000 7fecceffd640 1 -- 172.21.15.38:0/4169704495 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc000faf0 con 0x7fecd81f7c60 2024-04-25T08:35:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.043+0000 7fecceffd640 1 -- 172.21.15.38:0/4169704495 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc000fc90 con 0x7fecd81f7c60 2024-04-25T08:35:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.043+0000 7fecdcb29640 1 -- 172.21.15.38:0/4169704495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd81f7180 con 0x7fecd81f7c60 2024-04-25T08:35:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.043+0000 7fecceffd640 1 -- 172.21.15.38:0/4169704495 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0014440 con 0x7fecd81f7c60 2024-04-25T08:35:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.043+0000 7fecdcb29640 1 -- 172.21.15.38:0/4169704495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd815f7c0 con 0x7fecd81f7c60 2024-04-25T08:35:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.044+0000 7fecdcb29640 1 -- 172.21.15.38:0/4169704495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd8068e80 con 0x7fecd81f7c60 2024-04-25T08:35:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.044+0000 7fecceffd640 1 -- 172.21.15.38:0/4169704495 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecc0012040 con 0x7fecd81f7c60 2024-04-25T08:35:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.045+0000 7fecceffd640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecb806b4d0 0x7fecb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.045+0000 7fecceffd640 1 -- 172.21.15.38:0/4169704495 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fecc008a780 con 0x7fecd81f7c60 2024-04-25T08:35:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.045+0000 7fecd659c640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecb806b4d0 0x7fecb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.045+0000 7fecd659c640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecb806b4d0 0x7fecb806d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fecc8000c00 tx=0x7fecc8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:38.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.049+0000 7fecceffd640 1 -- 172.21.15.38:0/4169704495 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fecc0019030 con 0x7fecd81f7c60 2024-04-25T08:35:38.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.230+0000 7fecdcb29640 1 -- 172.21.15.38:0/4169704495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fecd80b5d30 con 0x7fecd81f7c60 2024-04-25T08:35:38.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.230+0000 7fecceffd640 1 -- 172.21.15.38:0/4169704495 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7fecc00597c0 con 0x7fecd81f7c60 2024-04-25T08:35:38.225 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:35:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.232+0000 7fecdcb29640 1 -- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecb806b4d0 msgr2=0x7fecb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.232+0000 7fecdcb29640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecb806b4d0 0x7fecb806d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fecc8000c00 tx=0x7fecc8009040 comp rx=0 tx=0).stop 2024-04-25T08:35:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.232+0000 7fecdcb29640 1 -- 172.21.15.38:0/4169704495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecd81f7c60 msgr2=0x7fecd80b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.232+0000 7fecdcb29640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecd81f7c60 0x7fecd80b53e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fecc00118e0 tx=0x7fecc0011da0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.232+0000 7fecdcb29640 1 -- 172.21.15.38:0/4169704495 shutdown_connections 2024-04-25T08:35:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.233+0000 7fecdcb29640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecb806b4d0 0x7fecb806d990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.233+0000 7fecdcb29640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecd80b5920 0x7fecd814b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.233+0000 7fecdcb29640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fecd81f7c60 0x7fecd80b53e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.233+0000 7fecdcb29640 1 --2- 172.21.15.38:0/4169704495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc40051c0 0x7fecd815f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.233+0000 7fecdcb29640 1 -- 172.21.15.38:0/4169704495 >> 172.21.15.38:0/4169704495 conn(0x7fecd80668f0 msgr2=0x7fecd80b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.233+0000 7fecdcb29640 1 -- 172.21.15.38:0/4169704495 shutdown_connections 2024-04-25T08:35:38.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.233+0000 7fecdcb29640 1 -- 172.21.15.38:0/4169704495 wait complete. 2024-04-25T08:35:38.239 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:35:38.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.345+0000 7f6632190640 1 Processor -- start 2024-04-25T08:35:38.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.345+0000 7f6632190640 1 -- start start 2024-04-25T08:35:38.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.345+0000 7f6632190640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f662c15dc10 0x7f662c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.345+0000 7f6632190640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f662c0ced60 0x7f662c0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.345+0000 7f6632190640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662c0d3e70 con 0x7f662c0d3900 2024-04-25T08:35:38.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.345+0000 7f6632190640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f662c15dff0 con 0x7f662c15dc10 2024-04-25T08:35:38.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.345+0000 7f6632190640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662c0b6990 con 0x7f662c0ced60 2024-04-25T08:35:38.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f662c0d3900 0x7f662c0b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45230/0 (socket says 172.21.15.38:45230) 2024-04-25T08:35:38.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662bfff640 1 -- 172.21.15.38:0/3143420215 learned_addr learned my addr 172.21.15.38:0/3143420215 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:38.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662b7fe640 1 --2- 172.21.15.38:0/3143420215 >> v2:172.21.15.107:3300/0 conn(0x7f662c15dc10 0x7f662c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662affd640 1 --2- 172.21.15.38:0/3143420215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f662c0ced60 0x7f662c0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539384438 0 0) 0x7f662c0d3e70 con 0x7f662c0d3900 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6614003520 con 0x7f662c0d3900 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1036623294 0 0) 0x7f6614003520 con 0x7f662c0d3900 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f662c0ced60 msgr2=0x7f662c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 --2- 172.21.15.38:0/3143420215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f662c0ced60 0x7f662c0d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 >> v2:172.21.15.107:3300/0 conn(0x7f662c15dc10 msgr2=0x7f662c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 --2- 172.21.15.38:0/3143420215 >> v2:172.21.15.107:3300/0 conn(0x7f662c15dc10 0x7f662c0ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c0d4150 con 0x7f662c0d3900 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f66240019f0 con 0x7f662c0d3900 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 >> v1:172.21.15.38:6789/0 conn(0x7f662c0d3900 legacy=0x7f662c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 --2- 172.21.15.38:0/3143420215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f662c0ced60 0x7f66140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 --2- 172.21.15.38:0/3143420215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6614003f30 0x7f6614004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 --2- 172.21.15.38:0/3143420215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6614004d40 0x7f66140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c15dff0 con 0x7f6614003f30 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661400b9c0 con 0x7f6614004d40 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661400bb90 con 0x7f662c0ced60 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662affd640 1 --2- 172.21.15.38:0/3143420215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6614003f30 0x7f6614004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662b7fe640 1 --2- 172.21.15.38:0/3143420215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f662c0ced60 0x7f66140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.346+0000 7f662bfff640 1 --2- 172.21.15.38:0/3143420215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6614004d40 0x7f66140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.347+0000 7f662affd640 1 -- 172.21.15.38:0/3143420215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f662c0ced60 msgr2=0x7f66140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.347+0000 7f662affd640 1 --2- 172.21.15.38:0/3143420215 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f662c0ced60 0x7f66140039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.347+0000 7f662affd640 1 -- 172.21.15.38:0/3143420215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6614004d40 msgr2=0x7f66140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.347+0000 7f662affd640 1 --2- 172.21.15.38:0/3143420215 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6614004d40 0x7f66140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.347+0000 7f662affd640 1 -- 172.21.15.38:0/3143420215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614003520 con 0x7f6614003f30 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.347+0000 7f662affd640 1 --2- 172.21.15.38:0/3143420215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6614003f30 0x7f6614004800 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6620009af0 tx=0x7f6614005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4614e590f3876df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.347+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620036070 con 0x7f6614003f30 2024-04-25T08:35:38.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.348+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6620031400 con 0x7f6614003f30 2024-04-25T08:35:38.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.348+0000 7f662a7fc640 1 -- 172.21.15.38:0/3143420215 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620009690 con 0x7f6614003f30 2024-04-25T08:35:38.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.348+0000 7f6632190640 1 -- 172.21.15.38:0/3143420215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6614003f30 msgr2=0x7f6614004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.348+0000 7f6632190640 1 --2- 172.21.15.38:0/3143420215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6614003f30 0x7f6614004800 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6620009af0 tx=0x7f6614005830 comp rx=0 tx=0).stop 2024-04-25T08:35:38.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.348+0000 7f662bfff640 1 -- 172.21.15.38:0/3143420215 reap_dead start 2024-04-25T08:35:38.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.348+0000 7f6632190640 1 -- 172.21.15.38:0/3143420215 shutdown_connections 2024-04-25T08:35:38.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.348+0000 7f6632190640 1 -- 172.21.15.38:0/3143420215 >> 172.21.15.38:0/3143420215 conn(0x7f662c066d10 msgr2=0x7f662c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:38.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.348+0000 7f6632190640 1 -- 172.21.15.38:0/3143420215 shutdown_connections 2024-04-25T08:35:38.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.348+0000 7f6632190640 1 -- 172.21.15.38:0/3143420215 wait complete. 2024-04-25T08:35:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.349+0000 7f6632190640 1 Processor -- start 2024-04-25T08:35:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.349+0000 7f6632190640 1 -- start start 2024-04-25T08:35:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.350+0000 7f6632190640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6614003f30 0x7f662c0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.350+0000 7f6632190640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f662c15ecf0 0x7f662c160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.350+0000 7f6632190640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f662c1544e0 0x7f662c1f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.350+0000 7f6632190640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c0b6bc0 con 0x7f662c15ecf0 2024-04-25T08:35:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.350+0000 7f6632190640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c0d4710 con 0x7f662c1544e0 2024-04-25T08:35:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.350+0000 7f6632190640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f662c0b2a80 con 0x7f6614003f30 2024-04-25T08:35:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.350+0000 7f662affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f662c15ecf0 0x7f662c160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.350+0000 7f662affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f662c15ecf0 0x7f662c160060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43132/0 (socket says 172.21.15.38:43132) 2024-04-25T08:35:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.350+0000 7f662affd640 1 -- 172.21.15.38:0/3851302322 learned_addr learned my addr 172.21.15.38:0/3851302322 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f662bfff640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f662c1544e0 0x7f662c1f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f662b7fe640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6614003f30 0x7f662c0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f662affd640 1 -- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6614003f30 msgr2=0x7f662c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f662affd640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6614003f30 0x7f662c0cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f662affd640 1 -- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f662c1544e0 msgr2=0x7f662c1f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f662affd640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f662c1544e0 0x7f662c1f2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f662affd640 1 -- 172.21.15.38:0/3851302322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f662c15f170 con 0x7f662c15ecf0 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f662bfff640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f662c1544e0 0x7f662c1f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f662affd640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f662c15ecf0 0x7f662c160060 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6620034040 tx=0x7f6620004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f6613fff640 1 -- 172.21.15.38:0/3851302322 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620040020 con 0x7f662c15ecf0 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f6613fff640 1 -- 172.21.15.38:0/3851302322 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66200038d0 con 0x7f662c15ecf0 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f6613fff640 1 -- 172.21.15.38:0/3851302322 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620036040 con 0x7f662c15ecf0 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f6632190640 1 -- 172.21.15.38:0/3851302322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662c15f3f0 con 0x7f662c15ecf0 2024-04-25T08:35:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.351+0000 7f6632190640 1 -- 172.21.15.38:0/3851302322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662c0cb030 con 0x7f662c15ecf0 2024-04-25T08:35:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.352+0000 7f6632190640 1 -- 172.21.15.38:0/3851302322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662c068e80 con 0x7f662c15ecf0 2024-04-25T08:35:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.352+0000 7f6613fff640 1 -- 172.21.15.38:0/3851302322 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6620043020 con 0x7f662c15ecf0 2024-04-25T08:35:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.353+0000 7f6613fff640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f661806b4d0 0x7f661806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.353+0000 7f6613fff640 1 -- 172.21.15.38:0/3851302322 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f66200aa0a0 con 0x7f662c15ecf0 2024-04-25T08:35:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.353+0000 7f662b7fe640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f661806b4d0 0x7f661806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.354+0000 7f662b7fe640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f661806b4d0 0x7f661806d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f661c009a60 tx=0x7f661c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.357+0000 7f6613fff640 1 -- 172.21.15.38:0/3851302322 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f662004a070 con 0x7f662c15ecf0 2024-04-25T08:35:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.541+0000 7f6632190640 1 -- 172.21.15.38:0/3851302322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f662c0cab20 con 0x7f662c15ecf0 2024-04-25T08:35:38.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.541+0000 7f6613fff640 1 -- 172.21.15.38:0/3851302322 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) v1 ==== 84+0+65 (secure 0 0 0) 0x7f662c15f170 con 0x7f662c15ecf0 2024-04-25T08:35:38.536 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:38.536 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T08:35:38.536 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:35:38.536 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-04-25T08:35:38.536 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-04-25T08:35:38.537 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:38.537 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T08:35:38.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.543+0000 7f6632190640 1 -- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f661806b4d0 msgr2=0x7f661806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.543+0000 7f6632190640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f661806b4d0 0x7f661806d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f661c009a60 tx=0x7f661c008040 comp rx=0 tx=0).stop 2024-04-25T08:35:38.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.543+0000 7f6632190640 1 -- 172.21.15.38:0/3851302322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f662c15ecf0 msgr2=0x7f662c160060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.543+0000 7f6632190640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f662c15ecf0 0x7f662c160060 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6620034040 tx=0x7f6620004730 comp rx=0 tx=0).stop 2024-04-25T08:35:38.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.543+0000 7f6632190640 1 -- 172.21.15.38:0/3851302322 shutdown_connections 2024-04-25T08:35:38.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.543+0000 7f6632190640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f662c1544e0 0x7f662c1f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.543+0000 7f6632190640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f662c15ecf0 0x7f662c160060 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.543+0000 7f6632190640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f661806b4d0 0x7f661806d990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.543+0000 7f6632190640 1 --2- 172.21.15.38:0/3851302322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6614003f30 0x7f662c0cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.543+0000 7f6632190640 1 -- 172.21.15.38:0/3851302322 >> 172.21.15.38:0/3851302322 conn(0x7f662c066d10 msgr2=0x7f662c152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:38.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.543+0000 7f6632190640 1 -- 172.21.15.38:0/3851302322 shutdown_connections 2024-04-25T08:35:38.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.543+0000 7f6632190640 1 -- 172.21.15.38:0/3851302322 wait complete. 2024-04-25T08:35:38.549 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:35:38.549 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:35:38.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.659+0000 7f6db8e5d640 1 Processor -- start 2024-04-25T08:35:38.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.659+0000 7f6db8e5d640 1 -- start start 2024-04-25T08:35:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db8e5d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6db41612c0 0x7f6db40b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db8e5d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6db40d0870 0x7f6db40d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db8e5d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db40b7200 con 0x7f6db40b6a00 2024-04-25T08:35:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db8e5d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db40d11e0 con 0x7f6db41612c0 2024-04-25T08:35:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db8e5d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db40b74e0 con 0x7f6db40d0870 2024-04-25T08:35:38.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db259c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6db41612c0 0x7f6db40b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db259c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6db41612c0 0x7f6db40b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43142/0 (socket says 172.21.15.38:43142) 2024-04-25T08:35:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db259c640 1 -- 172.21.15.38:0/545545613 learned_addr learned my addr 172.21.15.38:0/545545613 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db2d9d640 1 --2- 172.21.15.38:0/545545613 >> v2:172.21.15.107:3300/0 conn(0x7f6db40d0870 0x7f6db40d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db259c640 1 -- 172.21.15.38:0/545545613 >> v2:172.21.15.107:3300/0 conn(0x7f6db40d0870 msgr2=0x7f6db40d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db259c640 1 --2- 172.21.15.38:0/545545613 >> v2:172.21.15.107:3300/0 conn(0x7f6db40d0870 0x7f6db40d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db259c640 1 -- 172.21.15.38:0/545545613 >> v1:172.21.15.107:6789/0 conn(0x7f6db40b6a00 legacy=0x7f6db40d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.660+0000 7f6db259c640 1 -- 172.21.15.38:0/545545613 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db40b77c0 con 0x7f6db41612c0 2024-04-25T08:35:38.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.661+0000 7f6db259c640 1 --2- 172.21.15.38:0/545545613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db41612c0 0x7f6db40b93f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f6db40d1990 tx=0x7f6dac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5268e9cb822a974 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:38.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.661+0000 7f6db159a640 1 -- 172.21.15.38:0/545545613 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac037070 con 0x7f6db41612c0 2024-04-25T08:35:38.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.661+0000 7f6db159a640 1 -- 172.21.15.38:0/545545613 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac032440 con 0x7f6db41612c0 2024-04-25T08:35:38.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.661+0000 7f6db159a640 1 -- 172.21.15.38:0/545545613 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac032810 con 0x7f6db41612c0 2024-04-25T08:35:38.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.662+0000 7f6db8e5d640 1 -- 172.21.15.38:0/545545613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db41612c0 msgr2=0x7f6db40b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.662+0000 7f6db8e5d640 1 --2- 172.21.15.38:0/545545613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db41612c0 0x7f6db40b93f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f6db40d1990 tx=0x7f6dac02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.662+0000 7f6db8e5d640 1 -- 172.21.15.38:0/545545613 shutdown_connections 2024-04-25T08:35:38.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.662+0000 7f6db8e5d640 1 --2- 172.21.15.38:0/545545613 >> v2:172.21.15.107:3300/0 conn(0x7f6db40d0870 0x7f6db40d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.662+0000 7f6db8e5d640 1 --2- 172.21.15.38:0/545545613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db41612c0 0x7f6db40b93f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.662+0000 7f6db8e5d640 1 -- 172.21.15.38:0/545545613 >> 172.21.15.38:0/545545613 conn(0x7f6db4066d10 msgr2=0x7f6db4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:38.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.662+0000 7f6db8e5d640 1 -- 172.21.15.38:0/545545613 shutdown_connections 2024-04-25T08:35:38.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.662+0000 7f6db8e5d640 1 -- 172.21.15.38:0/545545613 wait complete. 2024-04-25T08:35:38.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.663+0000 7f6db8e5d640 1 Processor -- start 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.663+0000 7f6db8e5d640 1 -- start start 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db8e5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6db40b6a00 0x7f6db4150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db8e5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40d0870 0x7f6db415f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db8e5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db41612c0 0x7f6db40d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db8e5d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40b6e90 con 0x7f6db41612c0 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db8e5d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40b7d80 con 0x7f6db40b6a00 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db8e5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40b82a0 con 0x7f6db40d0870 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db41612c0 0x7f6db40d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db41612c0 0x7f6db40d2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43158/0 (socket says 172.21.15.38:43158) 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db2d9d640 1 -- 172.21.15.38:0/1847013580 learned_addr learned my addr 172.21.15.38:0/1847013580 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db259c640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6db40b6a00 0x7f6db4150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db1d9b640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40d0870 0x7f6db415f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db2d9d640 1 -- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40d0870 msgr2=0x7f6db415f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db2d9d640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40d0870 0x7f6db415f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db2d9d640 1 -- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6db40b6a00 msgr2=0x7f6db4150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db2d9d640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6db40b6a00 0x7f6db4150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.664+0000 7f6db2d9d640 1 -- 172.21.15.38:0/1847013580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db41604e0 con 0x7f6db41612c0 2024-04-25T08:35:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.665+0000 7f6db2d9d640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db41612c0 0x7f6db40d2c80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6da400efc0 tx=0x7f6da400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.665+0000 7f6daaffd640 1 -- 172.21.15.38:0/1847013580 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4018020 con 0x7f6db41612c0 2024-04-25T08:35:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.665+0000 7f6daaffd640 1 -- 172.21.15.38:0/1847013580 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da4014480 con 0x7f6db41612c0 2024-04-25T08:35:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.665+0000 7f6daaffd640 1 -- 172.21.15.38:0/1847013580 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da4014870 con 0x7f6db41612c0 2024-04-25T08:35:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.665+0000 7f6db8e5d640 1 -- 172.21.15.38:0/1847013580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db4160a70 con 0x7f6db41612c0 2024-04-25T08:35:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.665+0000 7f6db8e5d640 1 -- 172.21.15.38:0/1847013580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db4152cb0 con 0x7f6db41612c0 2024-04-25T08:35:38.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.666+0000 7f6db8e5d640 1 -- 172.21.15.38:0/1847013580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db4068e80 con 0x7f6db41612c0 2024-04-25T08:35:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.667+0000 7f6daaffd640 1 -- 172.21.15.38:0/1847013580 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6da4005d10 con 0x7f6db41612c0 2024-04-25T08:35:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.667+0000 7f6daaffd640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d9406b4d0 0x7f6d9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.667+0000 7f6daaffd640 1 -- 172.21.15.38:0/1847013580 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6da408ad20 con 0x7f6db41612c0 2024-04-25T08:35:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.667+0000 7f6db259c640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d9406b4d0 0x7f6d9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:38.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.668+0000 7f6db259c640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d9406b4d0 0x7f6d9406d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6dac034000 tx=0x7f6dac02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.671+0000 7f6daaffd640 1 -- 172.21.15.38:0/1847013580 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6da4022030 con 0x7f6db41612c0 2024-04-25T08:35:38.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:38.853+0000 7f6db8e5d640 1 -- 172.21.15.38:0/1847013580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6db40c6fd0 con 0x7f6db41612c0 2024-04-25T08:35:39.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.235+0000 7f6daaffd640 1 -- 172.21.15.38:0/1847013580 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6da4010040 con 0x7f6db41612c0 2024-04-25T08:35:39.231 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-04-25T08:35:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.237+0000 7f6db8e5d640 1 -- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d9406b4d0 msgr2=0x7f6d9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.237+0000 7f6db8e5d640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d9406b4d0 0x7f6d9406d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6dac034000 tx=0x7f6dac02e000 comp rx=0 tx=0).stop 2024-04-25T08:35:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.237+0000 7f6db8e5d640 1 -- 172.21.15.38:0/1847013580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db41612c0 msgr2=0x7f6db40d2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.237+0000 7f6db8e5d640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db41612c0 0x7f6db40d2c80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f6da400efc0 tx=0x7f6da400c490 comp rx=0 tx=0).stop 2024-04-25T08:35:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.237+0000 7f6db8e5d640 1 -- 172.21.15.38:0/1847013580 shutdown_connections 2024-04-25T08:35:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.237+0000 7f6db8e5d640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db41612c0 0x7f6db40d2c80 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.237+0000 7f6db8e5d640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d9406b4d0 0x7f6d9406d990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.237+0000 7f6db8e5d640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40d0870 0x7f6db415f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.237+0000 7f6db8e5d640 1 --2- 172.21.15.38:0/1847013580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6db40b6a00 0x7f6db4150f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.237+0000 7f6db8e5d640 1 -- 172.21.15.38:0/1847013580 >> 172.21.15.38:0/1847013580 conn(0x7f6db4066d10 msgr2=0x7f6db40c8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.237+0000 7f6db8e5d640 1 -- 172.21.15.38:0/1847013580 shutdown_connections 2024-04-25T08:35:39.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.237+0000 7f6db8e5d640 1 -- 172.21.15.38:0/1847013580 wait complete. 2024-04-25T08:35:39.247 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:35:39.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.361+0000 7f95660c4640 1 Processor -- start 2024-04-25T08:35:39.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.361+0000 7f95660c4640 1 -- start start 2024-04-25T08:35:39.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.362+0000 7f95660c4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f95600d0320 0x7f95600d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.362+0000 7f95660c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95600d0700 0x7f95600d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.362+0000 7f95660c4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95600d2330 con 0x7f95600d1e90 2024-04-25T08:35:39.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.362+0000 7f95660c4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95600d2610 con 0x7f95600d0320 2024-04-25T08:35:39.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.362+0000 7f95660c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95600d28f0 con 0x7f95600d0700 2024-04-25T08:35:39.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.362+0000 7f955f7fe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f95600d0320 0x7f95600d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.362+0000 7f955effd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95600d0700 0x7f95600d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.362+0000 7f955ffff640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f95600d1e90 0x7f95600b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:40484/0 (socket says 172.21.15.38:40484) 2024-04-25T08:35:39.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.362+0000 7f955ffff640 1 -- 172.21.15.38:0/2900698250 learned_addr learned my addr 172.21.15.38:0/2900698250 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:39.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.363+0000 7f955f7fe640 1 -- 172.21.15.38:0/2900698250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95600d0700 msgr2=0x7f95600d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:39.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.363+0000 7f955f7fe640 1 --2- 172.21.15.38:0/2900698250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95600d0700 0x7f95600d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.363+0000 7f955f7fe640 1 -- 172.21.15.38:0/2900698250 >> v1:172.21.15.107:6789/0 conn(0x7f95600d1e90 legacy=0x7f95600b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.363+0000 7f955f7fe640 1 -- 172.21.15.38:0/2900698250 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600d2bd0 con 0x7f95600d0320 2024-04-25T08:35:39.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.364+0000 7f955f7fe640 1 --2- 172.21.15.38:0/2900698250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95600d0320 0x7f95600d33d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f95500097a0 tx=0x7f955002a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef00cd347f40f5f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:39.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.364+0000 7f955e7fc640 1 -- 172.21.15.38:0/2900698250 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550037070 con 0x7f95600d0320 2024-04-25T08:35:39.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.364+0000 7f955e7fc640 1 -- 172.21.15.38:0/2900698250 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9550032440 con 0x7f95600d0320 2024-04-25T08:35:39.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.364+0000 7f955e7fc640 1 -- 172.21.15.38:0/2900698250 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550032810 con 0x7f95600d0320 2024-04-25T08:35:39.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.364+0000 7f95660c4640 1 -- 172.21.15.38:0/2900698250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95600d0320 msgr2=0x7f95600d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.364+0000 7f95660c4640 1 --2- 172.21.15.38:0/2900698250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95600d0320 0x7f95600d33d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f95500097a0 tx=0x7f955002a210 comp rx=0 tx=0).stop 2024-04-25T08:35:39.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.364+0000 7f95660c4640 1 -- 172.21.15.38:0/2900698250 shutdown_connections 2024-04-25T08:35:39.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.364+0000 7f95660c4640 1 --2- 172.21.15.38:0/2900698250 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95600d0700 0x7f95600d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.364+0000 7f95660c4640 1 --2- 172.21.15.38:0/2900698250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95600d0320 0x7f95600d33d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.364+0000 7f95660c4640 1 -- 172.21.15.38:0/2900698250 >> 172.21.15.38:0/2900698250 conn(0x7f95600668f0 msgr2=0x7f9560066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:39.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.364+0000 7f95660c4640 1 -- 172.21.15.38:0/2900698250 shutdown_connections 2024-04-25T08:35:39.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.364+0000 7f95660c4640 1 -- 172.21.15.38:0/2900698250 wait complete. 2024-04-25T08:35:39.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.366+0000 7f95660c4640 1 Processor -- start 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.366+0000 7f95660c4640 1 -- start start 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.366+0000 7f95660c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95600d0320 0x7f95601f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.366+0000 7f95660c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95600d0700 0x7f956015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.366+0000 7f95660c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95600d1e90 0x7f956015c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f95660c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600d0b40 con 0x7f95600d1e90 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f95660c4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9560161210 con 0x7f95600d0320 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f95660c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95601f7fa0 con 0x7f95600d0700 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f955ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95600d1e90 0x7f956015c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f955ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95600d1e90 0x7f956015c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56968/0 (socket says 172.21.15.38:56968) 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f955ffff640 1 -- 172.21.15.38:0/1009661988 learned_addr learned my addr 172.21.15.38:0/1009661988 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f955f7fe640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95600d0320 0x7f95601f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f955effd640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95600d0700 0x7f956015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f955f7fe640 1 -- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95600d0700 msgr2=0x7f956015bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f955f7fe640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95600d0700 0x7f956015bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f955f7fe640 1 -- 172.21.15.38:0/1009661988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95600d1e90 msgr2=0x7f956015c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f955f7fe640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95600d1e90 0x7f956015c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f955f7fe640 1 -- 172.21.15.38:0/1009661988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95600ccab0 con 0x7f95600d0320 2024-04-25T08:35:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f955ffff640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95600d1e90 0x7f956015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f955f7fe640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95600d0320 0x7f95601f7a60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9550031000 tx=0x7f95500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.367+0000 7f9547fff640 1 -- 172.21.15.38:0/1009661988 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550041030 con 0x7f95600d0320 2024-04-25T08:35:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.368+0000 7f9547fff640 1 -- 172.21.15.38:0/1009661988 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9550030500 con 0x7f95600d0320 2024-04-25T08:35:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.368+0000 7f9547fff640 1 -- 172.21.15.38:0/1009661988 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550037040 con 0x7f95600d0320 2024-04-25T08:35:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.368+0000 7f95660c4640 1 -- 172.21.15.38:0/1009661988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95600cd270 con 0x7f95600d0320 2024-04-25T08:35:39.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.368+0000 7f95660c4640 1 -- 172.21.15.38:0/1009661988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95600cc7d0 con 0x7f95600d0320 2024-04-25T08:35:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.368+0000 7f95660c4640 1 -- 172.21.15.38:0/1009661988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9560068e80 con 0x7f95600d0320 2024-04-25T08:35:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.369+0000 7f9547fff640 1 -- 172.21.15.38:0/1009661988 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9550044070 con 0x7f95600d0320 2024-04-25T08:35:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.370+0000 7f9547fff640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f954c06b4d0 0x7f954c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:39.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.370+0000 7f9547fff640 1 -- 172.21.15.38:0/1009661988 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f95500ab460 con 0x7f95600d0320 2024-04-25T08:35:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.370+0000 7f955effd640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f954c06b4d0 0x7f954c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:39.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.370+0000 7f955effd640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f954c06b4d0 0x7f954c06d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f95600cf5a0 tx=0x7f9554009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:39.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.374+0000 7f9547fff640 1 -- 172.21.15.38:0/1009661988 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9550079cc0 con 0x7f95600d0320 2024-04-25T08:35:39.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:39.557+0000 7f95660c4640 1 -- 172.21.15.38:0/1009661988 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9560073aa0 con 0x7f95600d0320 2024-04-25T08:35:40.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.255+0000 7f9547fff640 1 -- 172.21.15.38:0/1009661988 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) v1 ==== 170+0+0 (secure 0 0 0) 0x7f955007dd60 con 0x7f95600d0320 2024-04-25T08:35:40.250 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-04-25T08:35:40.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.260+0000 7f95660c4640 1 -- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f954c06b4d0 msgr2=0x7f954c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.261+0000 7f95660c4640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f954c06b4d0 0x7f954c06d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f95600cf5a0 tx=0x7f9554009190 comp rx=0 tx=0).stop 2024-04-25T08:35:40.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.261+0000 7f95660c4640 1 -- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95600d0320 msgr2=0x7f95601f7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.261+0000 7f95660c4640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95600d0320 0x7f95601f7a60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9550031000 tx=0x7f95500046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.265+0000 7f95660c4640 1 -- 172.21.15.38:0/1009661988 shutdown_connections 2024-04-25T08:35:40.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.265+0000 7f95660c4640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f95600d1e90 0x7f956015c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.265+0000 7f95660c4640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f954c06b4d0 0x7f954c06d990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.265+0000 7f95660c4640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f95600d0700 0x7f956015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.265+0000 7f95660c4640 1 --2- 172.21.15.38:0/1009661988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95600d0320 0x7f95601f7a60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.265+0000 7f95660c4640 1 -- 172.21.15.38:0/1009661988 >> 172.21.15.38:0/1009661988 conn(0x7f95600668f0 msgr2=0x7f95600b6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:40.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.265+0000 7f95660c4640 1 -- 172.21.15.38:0/1009661988 shutdown_connections 2024-04-25T08:35:40.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.265+0000 7f95660c4640 1 -- 172.21.15.38:0/1009661988 wait complete. 2024-04-25T08:35:40.274 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:35:40.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.388+0000 7f5ba9eb3640 1 Processor -- start 2024-04-25T08:35:40.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.388+0000 7f5ba9eb3640 1 -- start start 2024-04-25T08:35:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba9eb3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5ba4166610 0x7f5ba4168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba9eb3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba40d2060 con 0x7f5ba40d0320 2024-04-25T08:35:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba9eb3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba40d2330 con 0x7f5ba40d33d0 2024-04-25T08:35:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba9eb3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba40d2610 con 0x7f5ba4166610 2024-04-25T08:35:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba37fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5ba40d0320 0x7f5ba40d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:39858/0 (socket says 172.21.15.38:39858) 2024-04-25T08:35:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba37fe640 1 -- 172.21.15.38:0/3490115127 learned_addr learned my addr 172.21.15.38:0/3490115127 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba3fff640 1 --2- 172.21.15.38:0/3490115127 >> v2:172.21.15.107:3300/0 conn(0x7f5ba4166610 0x7f5ba4168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3355007976 0 0) 0x7f5ba40d2060 con 0x7f5ba40d0320 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b8c003520 con 0x7f5ba40d0320 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2909730532 0 0) 0x7f5b8c003520 con 0x7f5ba40d0320 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 >> v2:172.21.15.107:3300/0 conn(0x7f5ba4166610 msgr2=0x7f5ba4168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba27fc640 1 --2- 172.21.15.38:0/3490115127 >> v2:172.21.15.107:3300/0 conn(0x7f5ba4166610 0x7f5ba4168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 >> v1:172.21.15.107:6789/0 conn(0x7f5ba40d33d0 legacy=0x7f5ba40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba40d28f0 con 0x7f5ba40d0320 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.389+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f5b94001670 con 0x7f5ba40d0320 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 >> v1:172.21.15.38:6789/0 conn(0x7f5ba40d0320 legacy=0x7f5ba40d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba27fc640 1 --2- 172.21.15.38:0/3490115127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b8c0039b0 0x7f5b8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba27fc640 1 --2- 172.21.15.38:0/3490115127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b8c004800 0x7f5b8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba27fc640 1 --2- 172.21.15.38:0/3490115127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b8c0051c0 0x7f5b8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40d2610 con 0x7f5b8c004800 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c00c070 con 0x7f5b8c0051c0 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8c00c270 con 0x7f5b8c0039b0 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba3fff640 1 --2- 172.21.15.38:0/3490115127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b8c0051c0 0x7f5b8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba2ffd640 1 --2- 172.21.15.38:0/3490115127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b8c004800 0x7f5b8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba37fe640 1 --2- 172.21.15.38:0/3490115127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b8c0039b0 0x7f5b8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba3fff640 1 -- 172.21.15.38:0/3490115127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b8c0039b0 msgr2=0x7f5b8c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba3fff640 1 --2- 172.21.15.38:0/3490115127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b8c0039b0 0x7f5b8c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba3fff640 1 -- 172.21.15.38:0/3490115127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b8c004800 msgr2=0x7f5b8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba3fff640 1 --2- 172.21.15.38:0/3490115127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b8c004800 0x7f5b8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.390+0000 7f5ba3fff640 1 -- 172.21.15.38:0/3490115127 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8c003520 con 0x7f5b8c0051c0 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.391+0000 7f5ba3fff640 1 -- 172.21.15.38:0/3490115127 reap_dead start 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.391+0000 7f5ba3fff640 1 --2- 172.21.15.38:0/3490115127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b8c0051c0 0x7f5b8c00b9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009f90 tx=0x7f5b8c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d94991bb74e0680 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.391+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c035020 con 0x7f5b8c0051c0 2024-04-25T08:35:40.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.391+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9c003b90 con 0x7f5b8c0051c0 2024-04-25T08:35:40.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.391+0000 7f5ba27fc640 1 -- 172.21.15.38:0/3490115127 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c009c10 con 0x7f5b8c0051c0 2024-04-25T08:35:40.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.392+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3490115127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b8c0051c0 msgr2=0x7f5b8c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.392+0000 7f5ba9eb3640 1 --2- 172.21.15.38:0/3490115127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b8c0051c0 0x7f5b8c00b9c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c009f90 tx=0x7f5b8c005830 comp rx=0 tx=0).stop 2024-04-25T08:35:40.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.392+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3490115127 shutdown_connections 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.392+0000 7f5ba9eb3640 1 --2- 172.21.15.38:0/3490115127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b8c0051c0 0x7f5b8c00b9c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.392+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3490115127 >> 172.21.15.38:0/3490115127 conn(0x7f5ba40668f0 msgr2=0x7f5ba4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.392+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3490115127 shutdown_connections 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.392+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3490115127 wait complete. 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.393+0000 7f5ba9eb3640 1 Processor -- start 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.393+0000 7f5ba9eb3640 1 -- start start 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.394+0000 7f5ba9eb3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b8c0051c0 0x7f5ba4157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.394+0000 7f5ba9eb3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba4157100 0x7f5ba40cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.394+0000 7f5ba9eb3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ba40cccc0 0x7f5ba415ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.394+0000 7f5ba9eb3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba4161210 con 0x7f5ba4157100 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.394+0000 7f5ba9eb3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40d2eb0 con 0x7f5b8c0051c0 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.394+0000 7f5ba9eb3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba4161390 con 0x7f5ba40cccc0 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.394+0000 7f5ba2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba4157100 0x7f5ba40cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.394+0000 7f5ba2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba4157100 0x7f5ba40cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56988/0 (socket says 172.21.15.38:56988) 2024-04-25T08:35:40.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.394+0000 7f5ba2ffd640 1 -- 172.21.15.38:0/3651827040 learned_addr learned my addr 172.21.15.38:0/3651827040 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.394+0000 7f5ba37fe640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b8c0051c0 0x7f5ba4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.394+0000 7f5ba3fff640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ba40cccc0 0x7f5ba415ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.395+0000 7f5ba2ffd640 1 -- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ba40cccc0 msgr2=0x7f5ba415ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.395+0000 7f5ba2ffd640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ba40cccc0 0x7f5ba415ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.395+0000 7f5ba2ffd640 1 -- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b8c0051c0 msgr2=0x7f5ba4157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.395+0000 7f5ba2ffd640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b8c0051c0 0x7f5ba4157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.395+0000 7f5ba2ffd640 1 -- 172.21.15.38:0/3651827040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba4157580 con 0x7f5ba4157100 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.395+0000 7f5ba37fe640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b8c0051c0 0x7f5ba4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.395+0000 7f5ba2ffd640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba4157100 0x7f5ba40cb3f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800f5f0 tx=0x7f5b9800fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.395+0000 7f5b8bfff640 1 -- 172.21.15.38:0/3651827040 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b980052e0 con 0x7f5ba4157100 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.395+0000 7f5b8bfff640 1 -- 172.21.15.38:0/3651827040 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b98018070 con 0x7f5ba4157100 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.395+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3651827040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba40cb930 con 0x7f5ba4157100 2024-04-25T08:35:40.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.395+0000 7f5b8bfff640 1 -- 172.21.15.38:0/3651827040 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98003810 con 0x7f5ba4157100 2024-04-25T08:35:40.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.395+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3651827040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba40cbf00 con 0x7f5ba4157100 2024-04-25T08:35:40.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.396+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3651827040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ba40b7670 con 0x7f5ba4157100 2024-04-25T08:35:40.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.396+0000 7f5b8bfff640 1 -- 172.21.15.38:0/3651827040 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b980039b0 con 0x7f5ba4157100 2024-04-25T08:35:40.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.397+0000 7f5b8bfff640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b9006b4d0 0x7f5b9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:40.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.397+0000 7f5b8bfff640 1 -- 172.21.15.38:0/3651827040 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f5b98089f30 con 0x7f5ba4157100 2024-04-25T08:35:40.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.397+0000 7f5ba37fe640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b9006b4d0 0x7f5b9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:40.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.398+0000 7f5ba37fe640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b9006b4d0 0x7f5b9006d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40b3650 tx=0x7f5b94007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:40.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.402+0000 7f5b8bfff640 1 -- 172.21.15.38:0/3651827040 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ba40b7670 con 0x7f5ba4157100 2024-04-25T08:35:40.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:40.621+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3651827040 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5ba4068e80 con 0x7f5ba4157100 2024-04-25T08:35:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.252+0000 7f5b8bfff640 1 -- 172.21.15.38:0/3651827040 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v33) v1 ==== 173+0+0 (secure 0 0 0) 0x7f5b980586c0 con 0x7f5ba4157100 2024-04-25T08:35:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T08:35:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.256+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b9006b4d0 msgr2=0x7f5b9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.256+0000 7f5ba9eb3640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b9006b4d0 0x7f5b9006d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40b3650 tx=0x7f5b94007040 comp rx=0 tx=0).stop 2024-04-25T08:35:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.256+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3651827040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba4157100 msgr2=0x7f5ba40cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.256+0000 7f5ba9eb3640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba4157100 0x7f5ba40cb3f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800f5f0 tx=0x7f5b9800fab0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.256+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3651827040 shutdown_connections 2024-04-25T08:35:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.256+0000 7f5ba9eb3640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b9006b4d0 0x7f5b9006d990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.256+0000 7f5ba9eb3640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ba40cccc0 0x7f5ba415ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.256+0000 7f5ba9eb3640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ba4157100 0x7f5ba40cb3f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.256+0000 7f5ba9eb3640 1 --2- 172.21.15.38:0/3651827040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b8c0051c0 0x7f5ba4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.256+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3651827040 >> 172.21.15.38:0/3651827040 conn(0x7f5ba40668f0 msgr2=0x7f5ba40b6f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.256+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3651827040 shutdown_connections 2024-04-25T08:35:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.256+0000 7f5ba9eb3640 1 -- 172.21.15.38:0/3651827040 wait complete. 2024-04-25T08:35:41.278 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:41.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.391+0000 7fccffde0640 1 Processor -- start 2024-04-25T08:35:41.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.391+0000 7fccffde0640 1 -- start start 2024-04-25T08:35:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccffde0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fccf815df50 0x7fccf80ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccffde0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fccf80ce2e0 0x7fccf80d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccffde0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf80ceff0 con 0x7fccf806c110 2024-04-25T08:35:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccffde0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf80cf2d0 con 0x7fccf80ce2e0 2024-04-25T08:35:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccffde0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf80d1f70 con 0x7fccf815df50 2024-04-25T08:35:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccfdb7c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fccf806c110 0x7fccf806c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:39862/0 (socket says 172.21.15.38:39862) 2024-04-25T08:35:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccfdb7c640 1 -- 172.21.15.38:0/447602681 learned_addr learned my addr 172.21.15.38:0/447602681 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccfe37d640 1 --2- 172.21.15.38:0/447602681 >> v2:172.21.15.107:3300/0 conn(0x7fccf80ce2e0 0x7fccf80d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccfd37b640 1 --2- 172.21.15.38:0/447602681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fccf815df50 0x7fccf80ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccfcb7a640 1 -- 172.21.15.38:0/447602681 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149742851 0 0) 0x7fccf80ceff0 con 0x7fccf806c110 2024-04-25T08:35:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccfcb7a640 1 -- 172.21.15.38:0/447602681 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce4003520 con 0x7fccf806c110 2024-04-25T08:35:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccfe37d640 1 -- 172.21.15.38:0/447602681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fccf815df50 msgr2=0x7fccf80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccfe37d640 1 --2- 172.21.15.38:0/447602681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fccf815df50 0x7fccf80ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccfe37d640 1 -- 172.21.15.38:0/447602681 >> v1:172.21.15.38:6789/0 conn(0x7fccf806c110 legacy=0x7fccf806c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccfe37d640 1 -- 172.21.15.38:0/447602681 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf80d2250 con 0x7fccf80ce2e0 2024-04-25T08:35:41.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.392+0000 7fccfd37b640 1 --2- 172.21.15.38:0/447602681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fccf815df50 0x7fccf80ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.393+0000 7fccfe37d640 1 --2- 172.21.15.38:0/447602681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80ce2e0 0x7fccf80d2a50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009d90 tx=0x7fccf002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6fb806b979426b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:41.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.393+0000 7fccfcb7a640 1 -- 172.21.15.38:0/447602681 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0007740 con 0x7fccf80ce2e0 2024-04-25T08:35:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.393+0000 7fccfcb7a640 1 -- 172.21.15.38:0/447602681 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf00078e0 con 0x7fccf80ce2e0 2024-04-25T08:35:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.393+0000 7fccfcb7a640 1 -- 172.21.15.38:0/447602681 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf0007c90 con 0x7fccf80ce2e0 2024-04-25T08:35:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.394+0000 7fccffde0640 1 -- 172.21.15.38:0/447602681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80ce2e0 msgr2=0x7fccf80d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.394+0000 7fccffde0640 1 --2- 172.21.15.38:0/447602681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80ce2e0 0x7fccf80d2a50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fccf0009d90 tx=0x7fccf002a660 comp rx=0 tx=0).stop 2024-04-25T08:35:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.394+0000 7fccffde0640 1 -- 172.21.15.38:0/447602681 shutdown_connections 2024-04-25T08:35:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.394+0000 7fccffde0640 1 --2- 172.21.15.38:0/447602681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf80ce2e0 0x7fccf80d2a50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.394+0000 7fccffde0640 1 --2- 172.21.15.38:0/447602681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fccf815df50 0x7fccf80ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.394+0000 7fccffde0640 1 -- 172.21.15.38:0/447602681 >> 172.21.15.38:0/447602681 conn(0x7fccf80d3fb0 msgr2=0x7fccf80d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.394+0000 7fccffde0640 1 -- 172.21.15.38:0/447602681 shutdown_connections 2024-04-25T08:35:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.394+0000 7fccffde0640 1 -- 172.21.15.38:0/447602681 wait complete. 2024-04-25T08:35:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.395+0000 7fccffde0640 1 Processor -- start 2024-04-25T08:35:41.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.395+0000 7fccffde0640 1 -- start start 2024-04-25T08:35:41.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccffde0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccf806c110 0x7fccf815a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccffde0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fccf80ce2e0 0x7fccf81f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccffde0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf815df50 0x7fccf81f4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccffde0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf80d2f90 con 0x7fccf806c110 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccffde0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf80cece0 con 0x7fccf815df50 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccffde0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf80d3110 con 0x7fccf80ce2e0 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccfdb7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccf806c110 0x7fccf815a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccfdb7c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccf806c110 0x7fccf815a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56992/0 (socket says 172.21.15.38:56992) 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccfdb7c640 1 -- 172.21.15.38:0/1033272572 learned_addr learned my addr 172.21.15.38:0/1033272572 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccfe37d640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf815df50 0x7fccf81f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccfd37b640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fccf80ce2e0 0x7fccf81f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccfdb7c640 1 -- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fccf80ce2e0 msgr2=0x7fccf81f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccfdb7c640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fccf80ce2e0 0x7fccf81f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccfdb7c640 1 -- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf815df50 msgr2=0x7fccf81f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccfdb7c640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf815df50 0x7fccf81f4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.396+0000 7fccfdb7c640 1 -- 172.21.15.38:0/1033272572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf81f4e70 con 0x7fccf806c110 2024-04-25T08:35:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.397+0000 7fccfdb7c640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccf806c110 0x7fccf815a760 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c3c0 tx=0x7fccf400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.397+0000 7fccee7fc640 1 -- 172.21.15.38:0/1033272572 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4017070 con 0x7fccf806c110 2024-04-25T08:35:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.397+0000 7fccee7fc640 1 -- 172.21.15.38:0/1033272572 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf4005ce0 con 0x7fccf806c110 2024-04-25T08:35:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.397+0000 7fccee7fc640 1 -- 172.21.15.38:0/1033272572 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf40072f0 con 0x7fccf806c110 2024-04-25T08:35:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.397+0000 7fccffde0640 1 -- 172.21.15.38:0/1033272572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf81f5150 con 0x7fccf806c110 2024-04-25T08:35:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.397+0000 7fccffde0640 1 -- 172.21.15.38:0/1033272572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf81f5710 con 0x7fccf806c110 2024-04-25T08:35:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.398+0000 7fccffde0640 1 -- 172.21.15.38:0/1033272572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf8068e80 con 0x7fccf806c110 2024-04-25T08:35:41.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.398+0000 7fccee7fc640 1 -- 172.21.15.38:0/1033272572 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccf401a070 con 0x7fccf806c110 2024-04-25T08:35:41.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.399+0000 7fccee7fc640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fccdc06b4d0 0x7fccdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.399+0000 7fccee7fc640 1 -- 172.21.15.38:0/1033272572 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fccf4089760 con 0x7fccf806c110 2024-04-25T08:35:41.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.399+0000 7fccfd37b640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fccdc06b4d0 0x7fccdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.399+0000 7fccfd37b640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fccdc06b4d0 0x7fccdc06d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009560 tx=0x7fcce8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:41.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.403+0000 7fccee7fc640 1 -- 172.21.15.38:0/1033272572 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fccf4058450 con 0x7fccf806c110 2024-04-25T08:35:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.588+0000 7fccffde0640 1 -- 172.21.15.38:0/1033272572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fccf806d650 con 0x7fccf806c110 2024-04-25T08:35:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.589+0000 7fccee7fc640 1 -- 172.21.15.38:0/1033272572 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16885 (secure 0 0 0) 0x7fccf4057d50 con 0x7fccf806c110 2024-04-25T08:35:41.584 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:41.584 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":66,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:40.623039+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:38.884603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:39.590140+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:41.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.591+0000 7fccffde0640 1 -- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fccdc06b4d0 msgr2=0x7fccdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.591+0000 7fccffde0640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fccdc06b4d0 0x7fccdc06d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009560 tx=0x7fcce8008040 comp rx=0 tx=0).stop 2024-04-25T08:35:41.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.591+0000 7fccffde0640 1 -- 172.21.15.38:0/1033272572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccf806c110 msgr2=0x7fccf815a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.591+0000 7fccffde0640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccf806c110 0x7fccf815a760 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c3c0 tx=0x7fccf400c880 comp rx=0 tx=0).stop 2024-04-25T08:35:41.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.591+0000 7fccffde0640 1 -- 172.21.15.38:0/1033272572 shutdown_connections 2024-04-25T08:35:41.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.591+0000 7fccffde0640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fccf815df50 0x7fccf81f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.591+0000 7fccffde0640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fccdc06b4d0 0x7fccdc06d990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.591+0000 7fccffde0640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fccf80ce2e0 0x7fccf81f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.591+0000 7fccffde0640 1 --2- 172.21.15.38:0/1033272572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fccf806c110 0x7fccf815a760 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.591+0000 7fccffde0640 1 -- 172.21.15.38:0/1033272572 >> 172.21.15.38:0/1033272572 conn(0x7fccf80d3fb0 msgr2=0x7fccf815b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:41.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.591+0000 7fccffde0640 1 -- 172.21.15.38:0/1033272572 shutdown_connections 2024-04-25T08:35:41.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.591+0000 7fccffde0640 1 -- 172.21.15.38:0/1033272572 wait complete. 2024-04-25T08:35:41.597 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:41.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.703+0000 7f7418c38640 1 Processor -- start 2024-04-25T08:35:41.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.703+0000 7f7418c38640 1 -- start start 2024-04-25T08:35:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7418c38640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74140ce2e0 0x7f74140d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7418c38640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74140ceff0 con 0x7f741406c110 2024-04-25T08:35:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7418c38640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74140d1f70 con 0x7f741415df50 2024-04-25T08:35:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7418c38640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74140cf2d0 con 0x7f74140ce2e0 2024-04-25T08:35:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f741259c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f741406c110 0x7f741406c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:39866/0 (socket says 172.21.15.38:39866) 2024-04-25T08:35:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f741259c640 1 -- 172.21.15.38:0/2323740854 learned_addr learned my addr 172.21.15.38:0/2323740854 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715762580 0 0) 0x7f74140ceff0 con 0x7f741406c110 2024-04-25T08:35:41.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7400003520 con 0x7f741406c110 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7412d9d640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74140ce2e0 0x7f74140d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 869912666 0 0) 0x7f7400003520 con 0x7f741406c110 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74140ce2e0 msgr2=0x7f74140d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74140ce2e0 0x7f74140d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 >> v1:172.21.15.107:6789/0 conn(0x7f741415df50 legacy=0x7f74140ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74140d2250 con 0x7f741406c110 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f7404001670 con 0x7f741406c110 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 >> v1:172.21.15.38:6789/0 conn(0x7f741406c110 legacy=0x7f741406c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74000039e0 0x7f7400003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74140ce2e0 0x7f7400004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7400004d40 0x7f74000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74140cf2d0 con 0x7f74000039e0 2024-04-25T08:35:41.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740000bb50 con 0x7f7400004d40 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.704+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f740000bd20 con 0x7f74140ce2e0 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.705+0000 7f741259c640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74000039e0 0x7f7400003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.705+0000 7f740bfff640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74140ce2e0 0x7f7400004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.705+0000 7f7412d9d640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7400004d40 0x7f74000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.705+0000 7f741259c640 1 -- 172.21.15.38:0/2323740854 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74140ce2e0 msgr2=0x7f7400004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.705+0000 7f741259c640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74140ce2e0 0x7f7400004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.705+0000 7f741259c640 1 -- 172.21.15.38:0/2323740854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7400004d40 msgr2=0x7f74000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.705+0000 7f741259c640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7400004d40 0x7f74000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.705+0000 7f741259c640 1 -- 172.21.15.38:0/2323740854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7400003520 con 0x7f74000039e0 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.705+0000 7f7412d9d640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7400004d40 0x7f74000051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.705+0000 7f741259c640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74000039e0 0x7f7400003e00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f740400c240 tx=0x7f7400005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6fad5979fd2035b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.706+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404007d30 con 0x7f74000039e0 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.706+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7404005af0 con 0x7f74000039e0 2024-04-25T08:35:41.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.706+0000 7f7411d9b640 1 -- 172.21.15.38:0/2323740854 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7404007070 con 0x7f74000039e0 2024-04-25T08:35:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.706+0000 7f7418c38640 1 -- 172.21.15.38:0/2323740854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74000039e0 msgr2=0x7f7400003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.706+0000 7f7418c38640 1 --2- 172.21.15.38:0/2323740854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74000039e0 0x7f7400003e00 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f740400c240 tx=0x7f7400005830 comp rx=0 tx=0).stop 2024-04-25T08:35:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.706+0000 7f7412d9d640 1 -- 172.21.15.38:0/2323740854 reap_dead start 2024-04-25T08:35:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.706+0000 7f7418c38640 1 -- 172.21.15.38:0/2323740854 shutdown_connections 2024-04-25T08:35:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.706+0000 7f7418c38640 1 -- 172.21.15.38:0/2323740854 >> 172.21.15.38:0/2323740854 conn(0x7f74140d3fb0 msgr2=0x7f74140d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.706+0000 7f7418c38640 1 -- 172.21.15.38:0/2323740854 shutdown_connections 2024-04-25T08:35:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.706+0000 7f7418c38640 1 -- 172.21.15.38:0/2323740854 wait complete. 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.708+0000 7f7418c38640 1 Processor -- start 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.708+0000 7f7418c38640 1 -- start start 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.708+0000 7f7418c38640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74000039e0 0x7f74141565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.708+0000 7f7418c38640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140b8170 0x7f7414152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.708+0000 7f7418c38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74140c9430 0x7f74140c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.708+0000 7f7418c38640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74140d2bd0 con 0x7f74140c9430 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.708+0000 7f7418c38640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74140d2a50 con 0x7f74140b8170 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.708+0000 7f7418c38640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74140d2d50 con 0x7f74000039e0 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.708+0000 7f7412d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74140c9430 0x7f74140c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.708+0000 7f7412d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74140c9430 0x7f74140c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57004/0 (socket says 172.21.15.38:57004) 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.708+0000 7f7412d9d640 1 -- 172.21.15.38:0/2948007842 learned_addr learned my addr 172.21.15.38:0/2948007842 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f741259c640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74000039e0 0x7f74141565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f7412d9d640 1 -- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74000039e0 msgr2=0x7f74141565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f740bfff640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140b8170 0x7f7414152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f7412d9d640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74000039e0 0x7f74141565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f7412d9d640 1 -- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140b8170 msgr2=0x7f7414152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f7412d9d640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140b8170 0x7f7414152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f7412d9d640 1 -- 172.21.15.38:0/2948007842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74141533c0 con 0x7f74140c9430 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f740bfff640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140b8170 0x7f7414152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f741259c640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74000039e0 0x7f74141565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f7412d9d640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74140c9430 0x7f74140c98b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f74140ce750 tx=0x7f740c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f740affd640 1 -- 172.21.15.38:0/2948007842 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c007d80 con 0x7f74140c9430 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f740affd640 1 -- 172.21.15.38:0/2948007842 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740c019070 con 0x7f74140c9430 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f7418c38640 1 -- 172.21.15.38:0/2948007842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74140c72d0 con 0x7f74140c9430 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f740affd640 1 -- 172.21.15.38:0/2948007842 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740c014650 con 0x7f74140c9430 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.709+0000 7f7418c38640 1 -- 172.21.15.38:0/2948007842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74140c7890 con 0x7f74140c9430 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.710+0000 7f7418c38640 1 -- 172.21.15.38:0/2948007842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74140b6570 con 0x7f74140c9430 2024-04-25T08:35:41.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.711+0000 7f740affd640 1 -- 172.21.15.38:0/2948007842 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f740c0147f0 con 0x7f74140c9430 2024-04-25T08:35:41.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.711+0000 7f740affd640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f73f406b4d0 0x7f73f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:41.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.711+0000 7f740affd640 1 -- 172.21.15.38:0/2948007842 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f740c08a800 con 0x7f74140c9430 2024-04-25T08:35:41.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.711+0000 7f741259c640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f73f406b4d0 0x7f73f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:41.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.712+0000 7f741259c640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f73f406b4d0 0x7f73f406d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7404033000 tx=0x7f7404009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:41.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.715+0000 7f740affd640 1 -- 172.21.15.38:0/2948007842 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74140b6570 con 0x7f74140c9430 2024-04-25T08:35:41.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.896+0000 7f7418c38640 1 -- 172.21.15.38:0/2948007842 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7414068e80 con 0x7f74140c9430 2024-04-25T08:35:41.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.897+0000 7f740affd640 1 -- 172.21.15.38:0/2948007842 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f74141533c0 con 0x7f74140c9430 2024-04-25T08:35:41.892 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:41.892 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":66,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:40.623039+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:38.884603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:39.590140+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.899+0000 7f7418c38640 1 -- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f73f406b4d0 msgr2=0x7f73f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.899+0000 7f7418c38640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f73f406b4d0 0x7f73f406d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7404033000 tx=0x7f7404009030 comp rx=0 tx=0).stop 2024-04-25T08:35:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.899+0000 7f7418c38640 1 -- 172.21.15.38:0/2948007842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74140c9430 msgr2=0x7f74140c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.899+0000 7f7418c38640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74140c9430 0x7f74140c98b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f74140ce750 tx=0x7f740c004840 comp rx=0 tx=0).stop 2024-04-25T08:35:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.899+0000 7f7418c38640 1 -- 172.21.15.38:0/2948007842 shutdown_connections 2024-04-25T08:35:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.899+0000 7f7418c38640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f74140c9430 0x7f74140c98b0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.899+0000 7f7418c38640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f74140b8170 0x7f7414152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.899+0000 7f7418c38640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f73f406b4d0 0x7f73f406d990 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.899+0000 7f7418c38640 1 --2- 172.21.15.38:0/2948007842 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f74000039e0 0x7f74141565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.899+0000 7f7418c38640 1 -- 172.21.15.38:0/2948007842 >> 172.21.15.38:0/2948007842 conn(0x7f74140d3fb0 msgr2=0x7f7414161840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.899+0000 7f7418c38640 1 -- 172.21.15.38:0/2948007842 shutdown_connections 2024-04-25T08:35:41.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:41.899+0000 7f7418c38640 1 -- 172.21.15.38:0/2948007842 wait complete. 2024-04-25T08:35:41.905 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:35:42.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.014+0000 7ffb3db6a640 1 Processor -- start 2024-04-25T08:35:42.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.014+0000 7ffb3db6a640 1 -- start start 2024-04-25T08:35:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.015+0000 7ffb3db6a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ffb3815d990 0x7ffb380cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.015+0000 7ffb3db6a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb3806dfd0 0x7ffb380ca350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.015+0000 7ffb3db6a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3806e7b0 con 0x7ffb380ca890 2024-04-25T08:35:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.015+0000 7ffb3db6a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3806ea90 con 0x7ffb3815d990 2024-04-25T08:35:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.015+0000 7ffb3db6a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3806ed70 con 0x7ffb3806dfd0 2024-04-25T08:35:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.015+0000 7ffb37fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ffb380ca890 0x7ffb3806bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:39876/0 (socket says 172.21.15.38:39876) 2024-04-25T08:35:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.015+0000 7ffb37fff640 1 -- 172.21.15.38:0/1699843266 learned_addr learned my addr 172.21.15.38:0/1699843266 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.015+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547642879 0 0) 0x7ffb3806e7b0 con 0x7ffb380ca890 2024-04-25T08:35:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.015+0000 7ffb377fe640 1 --2- 172.21.15.38:0/1699843266 >> v2:172.21.15.107:3300/0 conn(0x7ffb3815d990 0x7ffb380cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.015+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb20003520 con 0x7ffb380ca890 2024-04-25T08:35:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.015+0000 7ffb36ffd640 1 --2- 172.21.15.38:0/1699843266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb3806dfd0 0x7ffb380ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3213369565 0 0) 0x7ffb20003520 con 0x7ffb380ca890 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb3806dfd0 msgr2=0x7ffb380ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 --2- 172.21.15.38:0/1699843266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb3806dfd0 0x7ffb380ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 >> v2:172.21.15.107:3300/0 conn(0x7ffb3815d990 msgr2=0x7ffb380cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 --2- 172.21.15.38:0/1699843266 >> v2:172.21.15.107:3300/0 conn(0x7ffb3815d990 0x7ffb380cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3806f050 con 0x7ffb380ca890 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7ffb300019f0 con 0x7ffb380ca890 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 >> v1:172.21.15.38:6789/0 conn(0x7ffb380ca890 legacy=0x7ffb3806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 --2- 172.21.15.38:0/1699843266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb200039b0 0x7ffb20003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 --2- 172.21.15.38:0/1699843266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb20004800 0x7ffb20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 --2- 172.21.15.38:0/1699843266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb3806dfd0 0x7ffb200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3806ea90 con 0x7ffb20004800 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3806ed70 con 0x7ffb200039b0 2024-04-25T08:35:42.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2000b9c0 con 0x7ffb3806dfd0 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb36ffd640 1 --2- 172.21.15.38:0/1699843266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb20004800 0x7ffb20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb377fe640 1 --2- 172.21.15.38:0/1699843266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb200039b0 0x7ffb20003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb37fff640 1 --2- 172.21.15.38:0/1699843266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb3806dfd0 0x7ffb200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb36ffd640 1 -- 172.21.15.38:0/1699843266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb3806dfd0 msgr2=0x7ffb200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb36ffd640 1 --2- 172.21.15.38:0/1699843266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb3806dfd0 0x7ffb200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb36ffd640 1 -- 172.21.15.38:0/1699843266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb200039b0 msgr2=0x7ffb20003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb36ffd640 1 --2- 172.21.15.38:0/1699843266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb200039b0 0x7ffb20003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.016+0000 7ffb36ffd640 1 -- 172.21.15.38:0/1699843266 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb20003520 con 0x7ffb20004800 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.017+0000 7ffb36ffd640 1 --2- 172.21.15.38:0/1699843266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb20004800 0x7ffb20004c80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009aa0 tx=0x7ffb20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3beffba7a1267758 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.017+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c036070 con 0x7ffb20004800 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.017+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c031400 con 0x7ffb20004800 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.017+0000 7ffb367fc640 1 -- 172.21.15.38:0/1699843266 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c009670 con 0x7ffb20004800 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.018+0000 7ffb3db6a640 1 -- 172.21.15.38:0/1699843266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb20004800 msgr2=0x7ffb20004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.018+0000 7ffb3db6a640 1 --2- 172.21.15.38:0/1699843266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb20004800 0x7ffb20004c80 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009aa0 tx=0x7ffb20005830 comp rx=0 tx=0).stop 2024-04-25T08:35:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.018+0000 7ffb37fff640 1 -- 172.21.15.38:0/1699843266 reap_dead start 2024-04-25T08:35:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.018+0000 7ffb3db6a640 1 -- 172.21.15.38:0/1699843266 shutdown_connections 2024-04-25T08:35:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.018+0000 7ffb3db6a640 1 -- 172.21.15.38:0/1699843266 >> 172.21.15.38:0/1699843266 conn(0x7ffb380717f0 msgr2=0x7ffb38071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.018+0000 7ffb3db6a640 1 -- 172.21.15.38:0/1699843266 shutdown_connections 2024-04-25T08:35:42.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.018+0000 7ffb3db6a640 1 -- 172.21.15.38:0/1699843266 wait complete. 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.019+0000 7ffb3db6a640 1 Processor -- start 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.019+0000 7ffb3db6a640 1 -- start start 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb3db6a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb20004800 0x7ffb38152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb3db6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb38160370 0x7ffb380b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb3db6a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb380b5380 0x7ffb380c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb3db6a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3806e590 con 0x7ffb38160370 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb3db6a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3806e410 con 0x7ffb380b5380 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb3db6a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3815dd70 con 0x7ffb20004800 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb36ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb38160370 0x7ffb380b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb36ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb38160370 0x7ffb380b4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57026/0 (socket says 172.21.15.38:57026) 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb36ffd640 1 -- 172.21.15.38:0/281596225 learned_addr learned my addr 172.21.15.38:0/281596225 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb377fe640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb20004800 0x7ffb38152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb37fff640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb380b5380 0x7ffb380c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb36ffd640 1 -- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb20004800 msgr2=0x7ffb38152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb36ffd640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb20004800 0x7ffb38152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb36ffd640 1 -- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb380b5380 msgr2=0x7ffb380c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb36ffd640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb380b5380 0x7ffb380c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb36ffd640 1 -- 172.21.15.38:0/281596225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb380c7a10 con 0x7ffb38160370 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.020+0000 7ffb36ffd640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb38160370 0x7ffb380b4e40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009bc0 tx=0x7ffb2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.021+0000 7ffb1ffff640 1 -- 172.21.15.38:0/281596225 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c041070 con 0x7ffb38160370 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.021+0000 7ffb1ffff640 1 -- 172.21.15.38:0/281596225 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c03d680 con 0x7ffb38160370 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.021+0000 7ffb3db6a640 1 -- 172.21.15.38:0/281596225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb380c7cf0 con 0x7ffb38160370 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.021+0000 7ffb1ffff640 1 -- 172.21.15.38:0/281596225 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c036040 con 0x7ffb38160370 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.021+0000 7ffb3db6a640 1 -- 172.21.15.38:0/281596225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3815fb70 con 0x7ffb38160370 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.022+0000 7ffb3db6a640 1 -- 172.21.15.38:0/281596225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb38068e80 con 0x7ffb38160370 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.022+0000 7ffb1ffff640 1 -- 172.21.15.38:0/281596225 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb2c044070 con 0x7ffb38160370 2024-04-25T08:35:42.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.023+0000 7ffb1ffff640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffb2406b500 0x7ffb2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.023+0000 7ffb1ffff640 1 -- 172.21.15.38:0/281596225 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb2c0aa680 con 0x7ffb38160370 2024-04-25T08:35:42.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.023+0000 7ffb377fe640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffb2406b500 0x7ffb2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.023+0000 7ffb377fe640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffb2406b500 0x7ffb2406d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009bc0 tx=0x7ffb28009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:42.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.027+0000 7ffb1ffff640 1 -- 172.21.15.38:0/281596225 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb2c078f00 con 0x7ffb38160370 2024-04-25T08:35:42.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.249+0000 7ffb3db6a640 1 -- 172.21.15.38:0/281596225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffb380b1500 con 0x7ffb38160370 2024-04-25T08:35:42.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.272+0000 7ffb1ffff640 1 -- 172.21.15.38:0/281596225 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffb2c07cfa0 con 0x7ffb38160370 2024-04-25T08:35:42.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.274+0000 7ffb3db6a640 1 -- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffb2406b500 msgr2=0x7ffb2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.275+0000 7ffb3db6a640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffb2406b500 0x7ffb2406d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009bc0 tx=0x7ffb28009190 comp rx=0 tx=0).stop 2024-04-25T08:35:42.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.275+0000 7ffb3db6a640 1 -- 172.21.15.38:0/281596225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb38160370 msgr2=0x7ffb380b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.275+0000 7ffb3db6a640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb38160370 0x7ffb380b4e40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009bc0 tx=0x7ffb2c004730 comp rx=0 tx=0).stop 2024-04-25T08:35:42.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.275+0000 7ffb3db6a640 1 -- 172.21.15.38:0/281596225 shutdown_connections 2024-04-25T08:35:42.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.275+0000 7ffb3db6a640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb380b5380 0x7ffb380c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.275+0000 7ffb3db6a640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb38160370 0x7ffb380b4e40 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.275+0000 7ffb3db6a640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffb2406b500 0x7ffb2406d9c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.275+0000 7ffb3db6a640 1 --2- 172.21.15.38:0/281596225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb20004800 0x7ffb38152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.275+0000 7ffb3db6a640 1 -- 172.21.15.38:0/281596225 >> 172.21.15.38:0/281596225 conn(0x7ffb380717f0 msgr2=0x7ffb3806c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:42.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.275+0000 7ffb3db6a640 1 -- 172.21.15.38:0/281596225 shutdown_connections 2024-04-25T08:35:42.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.275+0000 7ffb3db6a640 1 -- 172.21.15.38:0/281596225 wait complete. 2024-04-25T08:35:42.280 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:35:42.280 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:35:42.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.389+0000 7f460d1f5640 1 Processor -- start 2024-04-25T08:35:42.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.389+0000 7f460d1f5640 1 -- start start 2024-04-25T08:35:42.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f460d1f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46080d0320 0x7f46080d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f460d1f5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f46080b6980 0x7f46080b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f460d1f5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46080d2060 con 0x7f46080d0700 2024-04-25T08:35:42.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f460d1f5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46080d2330 con 0x7f46080b6980 2024-04-25T08:35:42.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f460d1f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46080d2610 con 0x7f46080d0320 2024-04-25T08:35:42.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f460759e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f46080b6980 0x7f46080b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f460759e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f46080b6980 0x7f46080b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57042/0 (socket says 172.21.15.38:57042) 2024-04-25T08:35:42.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f460759e640 1 -- 172.21.15.38:0/3776887401 learned_addr learned my addr 172.21.15.38:0/3776887401 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:42.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f4606d9d640 1 --2- 172.21.15.38:0/3776887401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46080d0320 0x7f46080d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f460759e640 1 -- 172.21.15.38:0/3776887401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46080d0320 msgr2=0x7f46080d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:42.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f460759e640 1 --2- 172.21.15.38:0/3776887401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46080d0320 0x7f46080d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f460759e640 1 -- 172.21.15.38:0/3776887401 >> v1:172.21.15.107:6789/0 conn(0x7f46080d0700 legacy=0x7f46080d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f460759e640 1 -- 172.21.15.38:0/3776887401 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46080d28f0 con 0x7f46080b6980 2024-04-25T08:35:42.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.390+0000 7f4606d9d640 1 --2- 172.21.15.38:0/3776887401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46080d0320 0x7f46080d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:42.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.391+0000 7f460759e640 1 --2- 172.21.15.38:0/3776887401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080b6980 0x7f46080b6e10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4600009870 tx=0x7f460002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50f7c61c322e13e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:42.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.391+0000 7f4605d9b640 1 -- 172.21.15.38:0/3776887401 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600037070 con 0x7f46080b6980 2024-04-25T08:35:42.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.391+0000 7f4605d9b640 1 -- 172.21.15.38:0/3776887401 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460002ae50 con 0x7f46080b6980 2024-04-25T08:35:42.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.391+0000 7f4605d9b640 1 -- 172.21.15.38:0/3776887401 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600032860 con 0x7f46080b6980 2024-04-25T08:35:42.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.392+0000 7f460d1f5640 1 -- 172.21.15.38:0/3776887401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080b6980 msgr2=0x7f46080b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.392+0000 7f460d1f5640 1 --2- 172.21.15.38:0/3776887401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080b6980 0x7f46080b6e10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f4600009870 tx=0x7f460002a300 comp rx=0 tx=0).stop 2024-04-25T08:35:42.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.392+0000 7f460d1f5640 1 -- 172.21.15.38:0/3776887401 shutdown_connections 2024-04-25T08:35:42.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.392+0000 7f460d1f5640 1 --2- 172.21.15.38:0/3776887401 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080b6980 0x7f46080b6e10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.392+0000 7f460d1f5640 1 --2- 172.21.15.38:0/3776887401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46080d0320 0x7f46080d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.392+0000 7f460d1f5640 1 -- 172.21.15.38:0/3776887401 >> 172.21.15.38:0/3776887401 conn(0x7f46080668f0 msgr2=0x7f4608066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:42.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.392+0000 7f460d1f5640 1 -- 172.21.15.38:0/3776887401 shutdown_connections 2024-04-25T08:35:42.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.392+0000 7f460d1f5640 1 -- 172.21.15.38:0/3776887401 wait complete. 2024-04-25T08:35:42.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.393+0000 7f460d1f5640 1 Processor -- start 2024-04-25T08:35:42.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.393+0000 7f460d1f5640 1 -- start start 2024-04-25T08:35:42.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.394+0000 7f460d1f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46080b6980 0x7f46081591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.394+0000 7f460d1f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46080d0320 0x7f46080c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.394+0000 7f460d1f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080d0700 0x7f4608155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.394+0000 7f460d1f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46080d0b40 con 0x7f46080d0700 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.394+0000 7f460d1f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46080d2eb0 con 0x7f46080d0320 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.394+0000 7f460d1f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46081567b0 con 0x7f46080b6980 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.394+0000 7f460759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080d0700 0x7f4608155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.394+0000 7f460759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080d0700 0x7f4608155d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57058/0 (socket says 172.21.15.38:57058) 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.394+0000 7f460759e640 1 -- 172.21.15.38:0/2658003663 learned_addr learned my addr 172.21.15.38:0/2658003663 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.394+0000 7f4606d9d640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46080b6980 0x7f46081591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.394+0000 7f460659c640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46080d0320 0x7f46080c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f460759e640 1 -- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46080b6980 msgr2=0x7f46081591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f460759e640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46080b6980 0x7f46081591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f460759e640 1 -- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46080d0320 msgr2=0x7f46080c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f460759e640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46080d0320 0x7f46080c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:42.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f460759e640 1 -- 172.21.15.38:0/2658003663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46080b44a0 con 0x7f46080d0700 2024-04-25T08:35:42.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f4606d9d640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46080b6980 0x7f46081591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:42.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f460659c640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46080d0320 0x7f46080c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:42.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f460759e640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080d0700 0x7f4608155d30 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4600009870 tx=0x7f46000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:42.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f45ef7fe640 1 -- 172.21.15.38:0/2658003663 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600041070 con 0x7f46080d0700 2024-04-25T08:35:42.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f45ef7fe640 1 -- 172.21.15.38:0/2658003663 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460003c680 con 0x7f46080d0700 2024-04-25T08:35:42.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f460d1f5640 1 -- 172.21.15.38:0/2658003663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46080b4780 con 0x7f46080d0700 2024-04-25T08:35:42.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f45ef7fe640 1 -- 172.21.15.38:0/2658003663 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4600037040 con 0x7f46080d0700 2024-04-25T08:35:42.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.395+0000 7f460d1f5640 1 -- 172.21.15.38:0/2658003663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46080c9c80 con 0x7f46080d0700 2024-04-25T08:35:42.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.396+0000 7f460d1f5640 1 -- 172.21.15.38:0/2658003663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46080b42c0 con 0x7f46080d0700 2024-04-25T08:35:42.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.396+0000 7f45ef7fe640 1 -- 172.21.15.38:0/2658003663 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4600044070 con 0x7f46080d0700 2024-04-25T08:35:42.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.397+0000 7f45ef7fe640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f45f406b4d0 0x7f45f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:42.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.397+0000 7f45ef7fe640 1 -- 172.21.15.38:0/2658003663 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f46000aacf0 con 0x7f46080d0700 2024-04-25T08:35:42.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.397+0000 7f4606d9d640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f45f406b4d0 0x7f45f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:42.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.398+0000 7f4606d9d640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f45f406b4d0 0x7f45f406d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4608154910 tx=0x7f45f8003a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:42.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.401+0000 7f45ef7fe640 1 -- 172.21.15.38:0/2658003663 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46080b42c0 con 0x7f46080d0700 2024-04-25T08:35:42.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:42.621+0000 7f460d1f5640 1 -- 172.21.15.38:0/2658003663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4608068e80 con 0x7f46080d0700 2024-04-25T08:35:43.217 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-04-25T08:35:43.222+0000 7f5bfa3d5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:35:43.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.284+0000 7f45ef7fe640 1 -- 172.21.15.38:0/2658003663 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4600079b00 con 0x7f46080d0700 2024-04-25T08:35:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.287+0000 7f460d1f5640 1 -- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f45f406b4d0 msgr2=0x7f45f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.287+0000 7f460d1f5640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f45f406b4d0 0x7f45f406d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4608154910 tx=0x7f45f8003a50 comp rx=0 tx=0).stop 2024-04-25T08:35:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.287+0000 7f460d1f5640 1 -- 172.21.15.38:0/2658003663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080d0700 msgr2=0x7f4608155d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.287+0000 7f460d1f5640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080d0700 0x7f4608155d30 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f4600009870 tx=0x7f46000046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.287+0000 7f460d1f5640 1 -- 172.21.15.38:0/2658003663 shutdown_connections 2024-04-25T08:35:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.287+0000 7f460d1f5640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46080d0700 0x7f4608155d30 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.287+0000 7f460d1f5640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46080d0320 0x7f46080c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.287+0000 7f460d1f5640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f45f406b4d0 0x7f45f406d990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.287+0000 7f460d1f5640 1 --2- 172.21.15.38:0/2658003663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f46080b6980 0x7f46081591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.287+0000 7f460d1f5640 1 -- 172.21.15.38:0/2658003663 >> 172.21.15.38:0/2658003663 conn(0x7f46080668f0 msgr2=0x7f46080b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:43.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.287+0000 7f460d1f5640 1 -- 172.21.15.38:0/2658003663 shutdown_connections 2024-04-25T08:35:43.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.287+0000 7f460d1f5640 1 -- 172.21.15.38:0/2658003663 wait complete. 2024-04-25T08:35:43.295 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:35:43.295 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:43.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.406+0000 7f44ef288640 1 Processor -- start 2024-04-25T08:35:43.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.406+0000 7f44ef288640 1 -- start start 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.407+0000 7f44ef288640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f44e815bb10 0x7f44e809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.407+0000 7f44ef288640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44e815fbf0 0x7f44e809a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.407+0000 7f44ef288640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e8160340 con 0x7f44e80d0460 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.407+0000 7f44ef288640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e80cffd0 con 0x7f44e815bb10 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.407+0000 7f44ef288640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80cf400 con 0x7f44e815fbf0 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.407+0000 7f44ed024640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f44e815bb10 0x7f44e809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.407+0000 7f44ec823640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f44e80d0460 0x7f44e815f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:39882/0 (socket says 172.21.15.38:39882) 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.407+0000 7f44ec823640 1 -- 172.21.15.38:0/1574533286 learned_addr learned my addr 172.21.15.38:0/1574533286 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:43.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.407+0000 7f44ed024640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f44e815bb10 0x7f44e809a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57060/0 (socket says 172.21.15.38:57060) 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.408+0000 7f44ed825640 1 --2- 172.21.15.38:0/1574533286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44e815fbf0 0x7f44e809a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.408+0000 7f44ed024640 1 -- 172.21.15.38:0/1574533286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44e815fbf0 msgr2=0x7f44e809a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.408+0000 7f44ed024640 1 --2- 172.21.15.38:0/1574533286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44e815fbf0 0x7f44e809a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.408+0000 7f44ed024640 1 -- 172.21.15.38:0/1574533286 >> v1:172.21.15.38:6789/0 conn(0x7f44e80d0460 legacy=0x7f44e815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.408+0000 7f44ed024640 1 -- 172.21.15.38:0/1574533286 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80cf6a0 con 0x7f44e815bb10 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.408+0000 7f44ed024640 1 --2- 172.21.15.38:0/1574533286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e815bb10 0x7f44e809a480 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009d90 tx=0x7f44d802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0597ee245cf570e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.409+0000 7f44e7fff640 1 -- 172.21.15.38:0/1574533286 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8031600 con 0x7f44e815bb10 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.409+0000 7f44e7fff640 1 -- 172.21.15.38:0/1574533286 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d80317a0 con 0x7f44e815bb10 2024-04-25T08:35:43.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.409+0000 7f44e7fff640 1 -- 172.21.15.38:0/1574533286 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8031b70 con 0x7f44e815bb10 2024-04-25T08:35:43.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.409+0000 7f44ef288640 1 -- 172.21.15.38:0/1574533286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e815bb10 msgr2=0x7f44e809a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.409+0000 7f44ef288640 1 --2- 172.21.15.38:0/1574533286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e815bb10 0x7f44e809a480 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009d90 tx=0x7f44d802a5d0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.409+0000 7f44ef288640 1 -- 172.21.15.38:0/1574533286 shutdown_connections 2024-04-25T08:35:43.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.409+0000 7f44ef288640 1 --2- 172.21.15.38:0/1574533286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44e815fbf0 0x7f44e809a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.409+0000 7f44ef288640 1 --2- 172.21.15.38:0/1574533286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e815bb10 0x7f44e809a480 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.409+0000 7f44ef288640 1 -- 172.21.15.38:0/1574533286 >> 172.21.15.38:0/1574533286 conn(0x7f44e80b31f0 msgr2=0x7f44e80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:43.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.409+0000 7f44ef288640 1 -- 172.21.15.38:0/1574533286 shutdown_connections 2024-04-25T08:35:43.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.409+0000 7f44ef288640 1 -- 172.21.15.38:0/1574533286 wait complete. 2024-04-25T08:35:43.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.410+0000 7f44ef288640 1 Processor -- start 2024-04-25T08:35:43.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.410+0000 7f44ef288640 1 -- start start 2024-04-25T08:35:43.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.411+0000 7f44ef288640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e80d0460 0x7f44e814efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.411+0000 7f44ef288640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44e815bb10 0x7f44e80ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.411+0000 7f44ef288640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e815fbf0 0x7f44e80cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.411+0000 7f44ef288640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80c5250 con 0x7f44e815fbf0 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.411+0000 7f44ef288640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80cfc60 con 0x7f44e80d0460 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.411+0000 7f44ef288640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e80c53d0 con 0x7f44e815bb10 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.411+0000 7f44ed825640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e815fbf0 0x7f44e80cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.411+0000 7f44ed825640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e815fbf0 0x7f44e80cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57062/0 (socket says 172.21.15.38:57062) 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.411+0000 7f44ed825640 1 -- 172.21.15.38:0/2259078103 learned_addr learned my addr 172.21.15.38:0/2259078103 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44ed024640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e80d0460 0x7f44e814efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44ed825640 1 -- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44e815bb10 msgr2=0x7f44e80ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44ec823640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44e815bb10 0x7f44e80ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44ed825640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44e815bb10 0x7f44e80ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44ed825640 1 -- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e80d0460 msgr2=0x7f44e814efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44ed825640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e80d0460 0x7f44e814efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44ed825640 1 -- 172.21.15.38:0/2259078103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80c94c0 con 0x7f44e815fbf0 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44ed024640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e80d0460 0x7f44e814efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44ed825640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e815fbf0 0x7f44e80cae10 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f44e000d5a0 tx=0x7f44e000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44e5ffb640 1 -- 172.21.15.38:0/2259078103 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e00125f0 con 0x7f44e815fbf0 2024-04-25T08:35:43.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44e5ffb640 1 -- 172.21.15.38:0/2259078103 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e0019070 con 0x7f44e815fbf0 2024-04-25T08:35:43.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44ef288640 1 -- 172.21.15.38:0/2259078103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e80c97a0 con 0x7f44e815fbf0 2024-04-25T08:35:43.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44e5ffb640 1 -- 172.21.15.38:0/2259078103 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e00129b0 con 0x7f44e815fbf0 2024-04-25T08:35:43.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.412+0000 7f44ef288640 1 -- 172.21.15.38:0/2259078103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e80c9d60 con 0x7f44e815fbf0 2024-04-25T08:35:43.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.413+0000 7f44ef288640 1 -- 172.21.15.38:0/2259078103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e8068e80 con 0x7f44e815fbf0 2024-04-25T08:35:43.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.413+0000 7f44e5ffb640 1 -- 172.21.15.38:0/2259078103 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44e0012b50 con 0x7f44e815fbf0 2024-04-25T08:35:43.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.414+0000 7f44e5ffb640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44d406b4d0 0x7f44d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.414+0000 7f44e5ffb640 1 -- 172.21.15.38:0/2259078103 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f44e008a770 con 0x7f44e815fbf0 2024-04-25T08:35:43.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.414+0000 7f44ed024640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44d406b4d0 0x7f44d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.415+0000 7f44ed024640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44d406b4d0 0x7f44d406d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f44d8034000 tx=0x7f44d802f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.418+0000 7f44e5ffb640 1 -- 172.21.15.38:0/2259078103 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44e000f300 con 0x7f44e815fbf0 2024-04-25T08:35:43.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.637+0000 7f44ef288640 1 -- 172.21.15.38:0/2259078103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44e80cdbf0 con 0x7f44e815fbf0 2024-04-25T08:35:43.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.638+0000 7f44e5ffb640 1 -- 172.21.15.38:0/2259078103 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f44e0059470 con 0x7f44e815fbf0 2024-04-25T08:35:43.633 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:43.633 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3966080134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3966080134},{"type":"v1","addr":"172.21.15.107:6836","nonce":3966080134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/632873469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":632873469},{"type":"v1","addr":"172.21.15.107:6837","nonce":632873469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4257593739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4257593739},{"type":"v1","addr":"172.21.15.38:6834","nonce":4257593739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:40.623355+0000","modified":"2024-04-25T08:35:42.623054+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6813},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6813":{"gid":6813,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/239365678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":239365678},{"type":"v1","addr":"172.21.15.38:6835","nonce":239365678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:35:43.633 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 35 2024-04-25T08:35:43.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.640+0000 7f44ef288640 1 -- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44d406b4d0 msgr2=0x7f44d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.640+0000 7f44ef288640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44d406b4d0 0x7f44d406d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f44d8034000 tx=0x7f44d802f000 comp rx=0 tx=0).stop 2024-04-25T08:35:43.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.640+0000 7f44ef288640 1 -- 172.21.15.38:0/2259078103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e815fbf0 msgr2=0x7f44e80cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.640+0000 7f44ef288640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e815fbf0 0x7f44e80cae10 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f44e000d5a0 tx=0x7f44e000da60 comp rx=0 tx=0).stop 2024-04-25T08:35:43.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.640+0000 7f44ef288640 1 -- 172.21.15.38:0/2259078103 shutdown_connections 2024-04-25T08:35:43.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.640+0000 7f44ef288640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44e815fbf0 0x7f44e80cae10 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.640+0000 7f44ef288640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44d406b4d0 0x7f44d406d990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.640+0000 7f44ef288640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44e815bb10 0x7f44e80ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.640+0000 7f44ef288640 1 --2- 172.21.15.38:0/2259078103 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44e80d0460 0x7f44e814efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.640+0000 7f44ef288640 1 -- 172.21.15.38:0/2259078103 >> 172.21.15.38:0/2259078103 conn(0x7f44e80b31f0 msgr2=0x7f44e8159b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.640+0000 7f44ef288640 1 -- 172.21.15.38:0/2259078103 shutdown_connections 2024-04-25T08:35:43.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.640+0000 7f44ef288640 1 -- 172.21.15.38:0/2259078103 wait complete. 2024-04-25T08:35:43.645 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.752+0000 7f8d5e08f640 1 Processor -- start 2024-04-25T08:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.752+0000 7f8d5e08f640 1 -- start start 2024-04-25T08:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.753+0000 7f8d5e08f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8d58144760 0x7f8d5815faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.753+0000 7f8d5e08f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d580b50c0 con 0x7f8d58160030 2024-04-25T08:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.753+0000 7f8d5e08f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d580b53a0 con 0x7f8d580cea70 2024-04-25T08:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.753+0000 7f8d5e08f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5815f380 con 0x7f8d58144760 2024-04-25T08:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.753+0000 7f8d577fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8d58144760 0x7f8d5815faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.753+0000 7f8d577fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8d58144760 0x7f8d5815faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57078/0 (socket says 172.21.15.38:57078) 2024-04-25T08:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.753+0000 7f8d56ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f8d58160030 0x7f8d580ce360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:39884/0 (socket says 172.21.15.38:39884) 2024-04-25T08:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.753+0000 7f8d577fe640 1 -- 172.21.15.38:0/2030185035 learned_addr learned my addr 172.21.15.38:0/2030185035 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.753+0000 7f8d577fe640 1 -- 172.21.15.38:0/2030185035 >> v1:172.21.15.107:6789/0 conn(0x7f8d580cea70 legacy=0x7f8d580b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.753+0000 7f8d577fe640 1 -- 172.21.15.38:0/2030185035 >> v1:172.21.15.38:6789/0 conn(0x7f8d58160030 legacy=0x7f8d580ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.753+0000 7f8d577fe640 1 -- 172.21.15.38:0/2030185035 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d580b5680 con 0x7f8d58144760 2024-04-25T08:35:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.754+0000 7f8d577fe640 1 --2- 172.21.15.38:0/2030185035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d58144760 0x7f8d5815faf0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8d5815f500 tx=0x7f8d4802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78c7b6391c2f270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.754+0000 7f8d567fc640 1 -- 172.21.15.38:0/2030185035 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48037070 con 0x7f8d58144760 2024-04-25T08:35:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.754+0000 7f8d567fc640 1 -- 172.21.15.38:0/2030185035 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d48032440 con 0x7f8d58144760 2024-04-25T08:35:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.754+0000 7f8d567fc640 1 -- 172.21.15.38:0/2030185035 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48032810 con 0x7f8d58144760 2024-04-25T08:35:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.755+0000 7f8d5e08f640 1 -- 172.21.15.38:0/2030185035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d58144760 msgr2=0x7f8d5815faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.755+0000 7f8d5e08f640 1 --2- 172.21.15.38:0/2030185035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d58144760 0x7f8d5815faf0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8d5815f500 tx=0x7f8d4802a210 comp rx=0 tx=0).stop 2024-04-25T08:35:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.755+0000 7f8d5e08f640 1 -- 172.21.15.38:0/2030185035 shutdown_connections 2024-04-25T08:35:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.755+0000 7f8d5e08f640 1 --2- 172.21.15.38:0/2030185035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d58144760 0x7f8d5815faf0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.755+0000 7f8d5e08f640 1 -- 172.21.15.38:0/2030185035 >> 172.21.15.38:0/2030185035 conn(0x7f8d5806b210 msgr2=0x7f8d5806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.755+0000 7f8d5e08f640 1 -- 172.21.15.38:0/2030185035 shutdown_connections 2024-04-25T08:35:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.755+0000 7f8d5e08f640 1 -- 172.21.15.38:0/2030185035 wait complete. 2024-04-25T08:35:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.756+0000 7f8d5e08f640 1 Processor -- start 2024-04-25T08:35:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.756+0000 7f8d5e08f640 1 -- start start 2024-04-25T08:35:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d5e08f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d580cea70 0x7f8d58138a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d5e08f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58160030 0x7f8d581397f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d5e08f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d58139d30 0x7f8d5815ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d5e08f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d580b3f20 con 0x7f8d580cea70 2024-04-25T08:35:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d5e08f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5815f660 con 0x7f8d58160030 2024-04-25T08:35:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d5e08f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d580b40a0 con 0x7f8d58139d30 2024-04-25T08:35:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d580cea70 0x7f8d58138a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d577fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d580cea70 0x7f8d58138a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57080/0 (socket says 172.21.15.38:57080) 2024-04-25T08:35:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d577fe640 1 -- 172.21.15.38:0/472218436 learned_addr learned my addr 172.21.15.38:0/472218436 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d56ffd640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58160030 0x7f8d581397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d57fff640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d58139d30 0x7f8d5815ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d577fe640 1 -- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d58139d30 msgr2=0x7f8d5815ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d577fe640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d58139d30 0x7f8d5815ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d577fe640 1 -- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58160030 msgr2=0x7f8d581397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d577fe640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58160030 0x7f8d581397f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d577fe640 1 -- 172.21.15.38:0/472218436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d58138fc0 con 0x7f8d580cea70 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d56ffd640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58160030 0x7f8d581397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.757+0000 7f8d57fff640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d58139d30 0x7f8d5815ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.758+0000 7f8d577fe640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d580cea70 0x7f8d58138a80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8d48000c00 tx=0x7f8d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.758+0000 7f8d3ffff640 1 -- 172.21.15.38:0/472218436 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48041070 con 0x7f8d580cea70 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.758+0000 7f8d3ffff640 1 -- 172.21.15.38:0/472218436 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4803c870 con 0x7f8d580cea70 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.758+0000 7f8d5e08f640 1 -- 172.21.15.38:0/472218436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5813a290 con 0x7f8d580cea70 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.758+0000 7f8d3ffff640 1 -- 172.21.15.38:0/472218436 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48037040 con 0x7f8d580cea70 2024-04-25T08:35:43.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.758+0000 7f8d5e08f640 1 -- 172.21.15.38:0/472218436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5813afd0 con 0x7f8d580cea70 2024-04-25T08:35:43.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.759+0000 7f8d5e08f640 1 -- 172.21.15.38:0/472218436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5813adf0 con 0x7f8d580cea70 2024-04-25T08:35:43.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.759+0000 7f8d3ffff640 1 -- 172.21.15.38:0/472218436 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d48044070 con 0x7f8d580cea70 2024-04-25T08:35:43.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.760+0000 7f8d3ffff640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8d4406b4d0 0x7f8d4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:43.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.760+0000 7f8d3ffff640 1 -- 172.21.15.38:0/472218436 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d480ab000 con 0x7f8d580cea70 2024-04-25T08:35:43.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.760+0000 7f8d56ffd640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8d4406b4d0 0x7f8d4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:43.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.760+0000 7f8d56ffd640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8d4406b4d0 0x7f8d4406d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8d581f4fd0 tx=0x7f8d4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.764+0000 7f8d3ffff640 1 -- 172.21.15.38:0/472218436 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d5813adf0 con 0x7f8d580cea70 2024-04-25T08:35:43.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.948+0000 7f8d5e08f640 1 -- 172.21.15.38:0/472218436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d58068e80 con 0x7f8d580cea70 2024-04-25T08:35:43.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.949+0000 7f8d3ffff640 1 -- 172.21.15.38:0/472218436 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f8d58138fc0 con 0x7f8d580cea70 2024-04-25T08:35:43.945 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:43.945 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":67,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:42.252762+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:38.884603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:39.590140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:43.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.951+0000 7f8d5e08f640 1 -- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8d4406b4d0 msgr2=0x7f8d4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.951+0000 7f8d5e08f640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8d4406b4d0 0x7f8d4406d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f8d581f4fd0 tx=0x7f8d4c009040 comp rx=0 tx=0).stop 2024-04-25T08:35:43.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.951+0000 7f8d5e08f640 1 -- 172.21.15.38:0/472218436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d580cea70 msgr2=0x7f8d58138a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:43.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.951+0000 7f8d5e08f640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d580cea70 0x7f8d58138a80 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8d48000c00 tx=0x7f8d480046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.951+0000 7f8d5e08f640 1 -- 172.21.15.38:0/472218436 shutdown_connections 2024-04-25T08:35:43.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.951+0000 7f8d5e08f640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8d4406b4d0 0x7f8d4406d990 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.951+0000 7f8d5e08f640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d58139d30 0x7f8d5815ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.951+0000 7f8d5e08f640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d58160030 0x7f8d581397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.951+0000 7f8d5e08f640 1 --2- 172.21.15.38:0/472218436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d580cea70 0x7f8d58138a80 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:43.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.951+0000 7f8d5e08f640 1 -- 172.21.15.38:0/472218436 >> 172.21.15.38:0/472218436 conn(0x7f8d5806b210 msgr2=0x7f8d580c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:43.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.951+0000 7f8d5e08f640 1 -- 172.21.15.38:0/472218436 shutdown_connections 2024-04-25T08:35:43.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:43.951+0000 7f8d5e08f640 1 -- 172.21.15.38:0/472218436 wait complete. 2024-04-25T08:35:43.957 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:35:43.957 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:35:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.063+0000 7fc60fa07640 1 Processor -- start 2024-04-25T08:35:44.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.064+0000 7fc60fa07640 1 -- start start 2024-04-25T08:35:44.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.064+0000 7fc60fa07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6080d3900 0x7fc6080d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.064+0000 7fc60fa07640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc6080d0d90 0x7fc6080b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.064+0000 7fc60fa07640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6080d3e70 con 0x7fc60815dc10 2024-04-25T08:35:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.064+0000 7fc60fa07640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60815dff0 con 0x7fc6080d0d90 2024-04-25T08:35:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.064+0000 7fc60fa07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6080b6990 con 0x7fc6080d3900 2024-04-25T08:35:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.065+0000 7fc60dfa4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc6080d0d90 0x7fc6080b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.065+0000 7fc60dfa4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc6080d0d90 0x7fc6080b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57088/0 (socket says 172.21.15.38:57088) 2024-04-25T08:35:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.065+0000 7fc60dfa4640 1 -- 172.21.15.38:0/1726603247 learned_addr learned my addr 172.21.15.38:0/1726603247 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.065+0000 7fc60cfa2640 1 --2- 172.21.15.38:0/1726603247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6080d3900 0x7fc6080d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.065+0000 7fc60dfa4640 1 -- 172.21.15.38:0/1726603247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6080d3900 msgr2=0x7fc6080d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.065+0000 7fc60dfa4640 1 --2- 172.21.15.38:0/1726603247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6080d3900 0x7fc6080d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.065+0000 7fc60dfa4640 1 -- 172.21.15.38:0/1726603247 >> v1:172.21.15.107:6789/0 conn(0x7fc60815dc10 legacy=0x7fc6080ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:44.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.065+0000 7fc60dfa4640 1 -- 172.21.15.38:0/1726603247 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6080d4150 con 0x7fc6080d0d90 2024-04-25T08:35:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.066+0000 7fc60dfa4640 1 --2- 172.21.15.38:0/1726603247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6080d0d90 0x7fc6080b2370 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc600009e50 tx=0x7fc60002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28088c71725759ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.066+0000 7fc5fffff640 1 -- 172.21.15.38:0/1726603247 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600036070 con 0x7fc6080d0d90 2024-04-25T08:35:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.066+0000 7fc5fffff640 1 -- 172.21.15.38:0/1726603247 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600031600 con 0x7fc6080d0d90 2024-04-25T08:35:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.066+0000 7fc5fffff640 1 -- 172.21.15.38:0/1726603247 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6000319d0 con 0x7fc6080d0d90 2024-04-25T08:35:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.066+0000 7fc60fa07640 1 -- 172.21.15.38:0/1726603247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6080d0d90 msgr2=0x7fc6080b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.066+0000 7fc60fa07640 1 --2- 172.21.15.38:0/1726603247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6080d0d90 0x7fc6080b2370 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fc600009e50 tx=0x7fc60002a630 comp rx=0 tx=0).stop 2024-04-25T08:35:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.066+0000 7fc60fa07640 1 -- 172.21.15.38:0/1726603247 shutdown_connections 2024-04-25T08:35:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.066+0000 7fc60fa07640 1 --2- 172.21.15.38:0/1726603247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6080d0d90 0x7fc6080b2370 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.066+0000 7fc60fa07640 1 --2- 172.21.15.38:0/1726603247 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6080d3900 0x7fc6080d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.066+0000 7fc60fa07640 1 -- 172.21.15.38:0/1726603247 >> 172.21.15.38:0/1726603247 conn(0x7fc608066d10 msgr2=0x7fc608067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.066+0000 7fc60fa07640 1 -- 172.21.15.38:0/1726603247 shutdown_connections 2024-04-25T08:35:44.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.067+0000 7fc60fa07640 1 -- 172.21.15.38:0/1726603247 wait complete. 2024-04-25T08:35:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.068+0000 7fc60fa07640 1 Processor -- start 2024-04-25T08:35:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.068+0000 7fc60fa07640 1 -- start start 2024-04-25T08:35:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.068+0000 7fc60fa07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6080d0d90 0x7fc6080ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.068+0000 7fc60fa07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6080d3900 0x7fc608159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.068+0000 7fc60fa07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60815dc10 0x7fc608145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.068+0000 7fc60fa07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6080b6bc0 con 0x7fc60815dc10 2024-04-25T08:35:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.068+0000 7fc60fa07640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6080d4710 con 0x7fc6080d3900 2024-04-25T08:35:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.068+0000 7fc60fa07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6080cef30 con 0x7fc6080d0d90 2024-04-25T08:35:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.068+0000 7fc60dfa4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60815dc10 0x7fc608145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60dfa4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60815dc10 0x7fc608145c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57090/0 (socket says 172.21.15.38:57090) 2024-04-25T08:35:44.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60dfa4640 1 -- 172.21.15.38:0/421706342 learned_addr learned my addr 172.21.15.38:0/421706342 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:44.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60cfa2640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6080d3900 0x7fc608159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60d7a3640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6080d0d90 0x7fc6080ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60dfa4640 1 -- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6080d0d90 msgr2=0x7fc6080ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60dfa4640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6080d0d90 0x7fc6080ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60dfa4640 1 -- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6080d3900 msgr2=0x7fc608159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60dfa4640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6080d3900 0x7fc608159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60dfa4640 1 -- 172.21.15.38:0/421706342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc608152f60 con 0x7fc60815dc10 2024-04-25T08:35:44.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60cfa2640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6080d3900 0x7fc608159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:44.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60dfa4640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60815dc10 0x7fc608145c50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc600009f70 tx=0x7fc600004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:44.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc5fdffb640 1 -- 172.21.15.38:0/421706342 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60003f020 con 0x7fc60815dc10 2024-04-25T08:35:44.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc5fdffb640 1 -- 172.21.15.38:0/421706342 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600037040 con 0x7fc60815dc10 2024-04-25T08:35:44.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60fa07640 1 -- 172.21.15.38:0/421706342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc608153d90 con 0x7fc60815dc10 2024-04-25T08:35:44.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc5fdffb640 1 -- 172.21.15.38:0/421706342 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600036070 con 0x7fc60815dc10 2024-04-25T08:35:44.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.069+0000 7fc60fa07640 1 -- 172.21.15.38:0/421706342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6080cc4d0 con 0x7fc60815dc10 2024-04-25T08:35:44.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.070+0000 7fc60fa07640 1 -- 172.21.15.38:0/421706342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc608068e80 con 0x7fc60815dc10 2024-04-25T08:35:44.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.071+0000 7fc5fdffb640 1 -- 172.21.15.38:0/421706342 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc600042020 con 0x7fc60815dc10 2024-04-25T08:35:44.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.071+0000 7fc5fdffb640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5ec06b4d0 0x7fc5ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.071+0000 7fc5fdffb640 1 -- 172.21.15.38:0/421706342 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc6000aa7a0 con 0x7fc60815dc10 2024-04-25T08:35:44.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.072+0000 7fc60d7a3640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5ec06b4d0 0x7fc5ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.072+0000 7fc60d7a3640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5ec06b4d0 0x7fc5ec06d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc604006fd0 tx=0x7fc604009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:44.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.075+0000 7fc5fdffb640 1 -- 172.21.15.38:0/421706342 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc600049030 con 0x7fc60815dc10 2024-04-25T08:35:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.251+0000 7fc60fa07640 1 -- 172.21.15.38:0/421706342 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6080d1c30 con 0x7fc5ec06b4d0 2024-04-25T08:35:44.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.258+0000 7fc5fdffb640 1 -- 172.21.15.38:0/421706342 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251949 (secure 0 0 0) 0x7fc6080d1c30 con 0x7fc5ec06b4d0 2024-04-25T08:35:44.256 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:44.257 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:35:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.261+0000 7fc60fa07640 1 -- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5ec06b4d0 msgr2=0x7fc5ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.261+0000 7fc60fa07640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5ec06b4d0 0x7fc5ec06d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc604006fd0 tx=0x7fc604009060 comp rx=0 tx=0).stop 2024-04-25T08:35:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.261+0000 7fc60fa07640 1 -- 172.21.15.38:0/421706342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60815dc10 msgr2=0x7fc608145c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.261+0000 7fc60fa07640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60815dc10 0x7fc608145c50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc600009f70 tx=0x7fc600004a40 comp rx=0 tx=0).stop 2024-04-25T08:35:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.261+0000 7fc60fa07640 1 -- 172.21.15.38:0/421706342 shutdown_connections 2024-04-25T08:35:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.261+0000 7fc60fa07640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc60815dc10 0x7fc608145c50 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.261+0000 7fc60fa07640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6080d3900 0x7fc608159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.261+0000 7fc60fa07640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5ec06b4d0 0x7fc5ec06d990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.261+0000 7fc60fa07640 1 --2- 172.21.15.38:0/421706342 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6080d0d90 0x7fc6080ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.261+0000 7fc60fa07640 1 -- 172.21.15.38:0/421706342 >> 172.21.15.38:0/421706342 conn(0x7fc608066d10 msgr2=0x7fc60806bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:44.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.261+0000 7fc60fa07640 1 -- 172.21.15.38:0/421706342 shutdown_connections 2024-04-25T08:35:44.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.261+0000 7fc60fa07640 1 -- 172.21.15.38:0/421706342 wait complete. 2024-04-25T08:35:44.267 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-04-25T08:35:43.262449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":336036,"kb_used_data":7464,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754638684,"statfs":{"total":773094113280,"available":772750012416,"internally_reserved":0,"allocated":7643136,"data_stored":4093220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":336318000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020705"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290846+0000","last_change":"2024-04-25T08:35:41.287246+0000","last_active":"2024-04-25T08:35:41.290846+0000","last_peered":"2024-04-25T08:35:41.290846+0000","last_clean":"2024-04-25T08:35:41.290846+0000","last_became_active":"2024-04-25T08:35:41.286039+0000","last_became_peered":"2024-04-25T08:35:41.286039+0000","last_unstale":"2024-04-25T08:35:41.290846+0000","last_undegraded":"2024-04-25T08:35:41.290846+0000","last_fullsized":"2024-04-25T08:35:41.290846+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:35.472816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290768+0000","last_change":"2024-04-25T08:35:41.285554+0000","last_active":"2024-04-25T08:35:41.290768+0000","last_peered":"2024-04-25T08:35:41.290768+0000","last_clean":"2024-04-25T08:35:41.290768+0000","last_became_active":"2024-04-25T08:35:41.285249+0000","last_became_peered":"2024-04-25T08:35:41.285249+0000","last_unstale":"2024-04-25T08:35:41.290768+0000","last_undegraded":"2024-04-25T08:35:41.290768+0000","last_fullsized":"2024-04-25T08:35:41.290768+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:01.626304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.277813+0000","last_change":"2024-04-25T08:35:41.272722+0000","last_active":"2024-04-25T08:35:41.277813+0000","last_peered":"2024-04-25T08:35:41.277813+0000","last_clean":"2024-04-25T08:35:41.277813+0000","last_became_active":"2024-04-25T08:35:41.272446+0000","last_became_peered":"2024-04-25T08:35:41.272446+0000","last_unstale":"2024-04-25T08:35:41.277813+0000","last_undegraded":"2024-04-25T08:35:41.277813+0000","last_fullsized":"2024-04-25T08:35:41.277813+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:19.507911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.282261+0000","last_change":"2024-04-25T08:35:41.279637+0000","last_active":"2024-04-25T08:35:41.282261+0000","last_peered":"2024-04-25T08:35:41.282261+0000","last_clean":"2024-04-25T08:35:41.282261+0000","last_became_active":"2024-04-25T08:35:41.279345+0000","last_became_peered":"2024-04-25T08:35:41.279345+0000","last_unstale":"2024-04-25T08:35:41.282261+0000","last_undegraded":"2024-04-25T08:35:41.282261+0000","last_fullsized":"2024-04-25T08:35:41.282261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.714069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.269400+0000","last_change":"2024-04-25T08:35:41.264763+0000","last_active":"2024-04-25T08:35:41.269400+0000","last_peered":"2024-04-25T08:35:41.269400+0000","last_clean":"2024-04-25T08:35:41.269400+0000","last_became_active":"2024-04-25T08:35:41.264468+0000","last_became_peered":"2024-04-25T08:35:41.264468+0000","last_unstale":"2024-04-25T08:35:41.269400+0000","last_undegraded":"2024-04-25T08:35:41.269400+0000","last_fullsized":"2024-04-25T08:35:41.269400+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.714069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.280665+0000","last_change":"2024-04-25T08:35:41.275921+0000","last_active":"2024-04-25T08:35:41.280665+0000","last_peered":"2024-04-25T08:35:41.280665+0000","last_clean":"2024-04-25T08:35:41.280665+0000","last_became_active":"2024-04-25T08:35:41.275516+0000","last_became_peered":"2024-04-25T08:35:41.275516+0000","last_unstale":"2024-04-25T08:35:41.280665+0000","last_undegraded":"2024-04-25T08:35:41.280665+0000","last_fullsized":"2024-04-25T08:35:41.280665+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:03.266564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.284011+0000","last_change":"2024-04-25T08:35:41.280722+0000","last_active":"2024-04-25T08:35:41.284011+0000","last_peered":"2024-04-25T08:35:41.284011+0000","last_clean":"2024-04-25T08:35:41.284011+0000","last_became_active":"2024-04-25T08:35:41.279759+0000","last_became_peered":"2024-04-25T08:35:41.279759+0000","last_unstale":"2024-04-25T08:35:41.284011+0000","last_undegraded":"2024-04-25T08:35:41.284011+0000","last_fullsized":"2024-04-25T08:35:41.284011+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:09.237958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.277805+0000","last_change":"2024-04-25T08:35:41.271914+0000","last_active":"2024-04-25T08:35:41.277805+0000","last_peered":"2024-04-25T08:35:41.277805+0000","last_clean":"2024-04-25T08:35:41.277805+0000","last_became_active":"2024-04-25T08:35:41.270062+0000","last_became_peered":"2024-04-25T08:35:41.270062+0000","last_unstale":"2024-04-25T08:35:41.277805+0000","last_undegraded":"2024-04-25T08:35:41.277805+0000","last_fullsized":"2024-04-25T08:35:41.277805+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:30.604184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.267157+0000","last_change":"2024-04-25T08:35:41.265682+0000","last_active":"2024-04-25T08:35:41.267157+0000","last_peered":"2024-04-25T08:35:41.267157+0000","last_clean":"2024-04-25T08:35:41.267157+0000","last_became_active":"2024-04-25T08:35:41.265394+0000","last_became_peered":"2024-04-25T08:35:41.265394+0000","last_unstale":"2024-04-25T08:35:41.267157+0000","last_undegraded":"2024-04-25T08:35:41.267157+0000","last_fullsized":"2024-04-25T08:35:41.267157+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:44.396919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287970+0000","last_change":"2024-04-25T08:35:41.286813+0000","last_active":"2024-04-25T08:35:41.287970+0000","last_peered":"2024-04-25T08:35:41.287970+0000","last_clean":"2024-04-25T08:35:41.287970+0000","last_became_active":"2024-04-25T08:35:41.286439+0000","last_became_peered":"2024-04-25T08:35:41.286439+0000","last_unstale":"2024-04-25T08:35:41.287970+0000","last_undegraded":"2024-04-25T08:35:41.287970+0000","last_fullsized":"2024-04-25T08:35:41.287970+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:41.362181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.288217+0000","last_change":"2024-04-25T08:35:41.287559+0000","last_active":"2024-04-25T08:35:41.288217+0000","last_peered":"2024-04-25T08:35:41.288217+0000","last_clean":"2024-04-25T08:35:41.288217+0000","last_became_active":"2024-04-25T08:35:41.287267+0000","last_became_peered":"2024-04-25T08:35:41.287267+0000","last_unstale":"2024-04-25T08:35:41.288217+0000","last_undegraded":"2024-04-25T08:35:41.288217+0000","last_fullsized":"2024-04-25T08:35:41.288217+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:23.681487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.272019+0000","last_change":"2024-04-25T08:35:41.270585+0000","last_active":"2024-04-25T08:35:41.272019+0000","last_peered":"2024-04-25T08:35:41.272019+0000","last_clean":"2024-04-25T08:35:41.272019+0000","last_became_active":"2024-04-25T08:35:41.270041+0000","last_became_peered":"2024-04-25T08:35:41.270041+0000","last_unstale":"2024-04-25T08:35:41.272019+0000","last_undegraded":"2024-04-25T08:35:41.272019+0000","last_fullsized":"2024-04-25T08:35:41.272019+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:25.514354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.277506+0000","last_change":"2024-04-25T08:35:41.275625+0000","last_active":"2024-04-25T08:35:41.277506+0000","last_peered":"2024-04-25T08:35:41.277506+0000","last_clean":"2024-04-25T08:35:41.277506+0000","last_became_active":"2024-04-25T08:35:41.275357+0000","last_became_peered":"2024-04-25T08:35:41.275357+0000","last_unstale":"2024-04-25T08:35:41.277506+0000","last_undegraded":"2024-04-25T08:35:41.277506+0000","last_fullsized":"2024-04-25T08:35:41.277506+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:40.578929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.282988+0000","last_change":"2024-04-25T08:35:41.282228+0000","last_active":"2024-04-25T08:35:41.282988+0000","last_peered":"2024-04-25T08:35:41.282988+0000","last_clean":"2024-04-25T08:35:41.282988+0000","last_became_active":"2024-04-25T08:35:41.281639+0000","last_became_peered":"2024-04-25T08:35:41.281639+0000","last_unstale":"2024-04-25T08:35:41.282988+0000","last_undegraded":"2024-04-25T08:35:41.282988+0000","last_fullsized":"2024-04-25T08:35:41.282988+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:16.032826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287972+0000","last_change":"2024-04-25T08:35:41.286523+0000","last_active":"2024-04-25T08:35:41.287972+0000","last_peered":"2024-04-25T08:35:41.287972+0000","last_clean":"2024-04-25T08:35:41.287972+0000","last_became_active":"2024-04-25T08:35:41.286261+0000","last_became_peered":"2024-04-25T08:35:41.286261+0000","last_unstale":"2024-04-25T08:35:41.287972+0000","last_undegraded":"2024-04-25T08:35:41.287972+0000","last_fullsized":"2024-04-25T08:35:41.287972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:36.353555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.284067+0000","last_change":"2024-04-25T08:35:41.277647+0000","last_active":"2024-04-25T08:35:41.284067+0000","last_peered":"2024-04-25T08:35:41.284067+0000","last_clean":"2024-04-25T08:35:41.284067+0000","last_became_active":"2024-04-25T08:35:41.277418+0000","last_became_peered":"2024-04-25T08:35:41.277418+0000","last_unstale":"2024-04-25T08:35:41.284067+0000","last_undegraded":"2024-04-25T08:35:41.284067+0000","last_fullsized":"2024-04-25T08:35:41.284067+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:52.868767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290796+0000","last_change":"2024-04-25T08:35:41.285419+0000","last_active":"2024-04-25T08:35:41.290796+0000","last_peered":"2024-04-25T08:35:41.290796+0000","last_clean":"2024-04-25T08:35:41.290796+0000","last_became_active":"2024-04-25T08:35:41.285111+0000","last_became_peered":"2024-04-25T08:35:41.285111+0000","last_unstale":"2024-04-25T08:35:41.290796+0000","last_undegraded":"2024-04-25T08:35:41.290796+0000","last_fullsized":"2024-04-25T08:35:41.290796+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:57.886488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.270603+0000","last_change":"2024-04-25T08:35:41.268261+0000","last_active":"2024-04-25T08:35:41.270603+0000","last_peered":"2024-04-25T08:35:41.270603+0000","last_clean":"2024-04-25T08:35:41.270603+0000","last_became_active":"2024-04-25T08:35:41.267919+0000","last_became_peered":"2024-04-25T08:35:41.267919+0000","last_unstale":"2024-04-25T08:35:41.270603+0000","last_undegraded":"2024-04-25T08:35:41.270603+0000","last_fullsized":"2024-04-25T08:35:41.270603+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:01.633527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.278635+0000","last_change":"2024-04-25T08:35:41.277311+0000","last_active":"2024-04-25T08:35:41.278635+0000","last_peered":"2024-04-25T08:35:41.278635+0000","last_clean":"2024-04-25T08:35:41.278635+0000","last_became_active":"2024-04-25T08:35:41.277016+0000","last_became_peered":"2024-04-25T08:35:41.277016+0000","last_unstale":"2024-04-25T08:35:41.278635+0000","last_undegraded":"2024-04-25T08:35:41.278635+0000","last_fullsized":"2024-04-25T08:35:41.278635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:57.082004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.275196+0000","last_change":"2024-04-25T08:35:41.274477+0000","last_active":"2024-04-25T08:35:41.275196+0000","last_peered":"2024-04-25T08:35:41.275196+0000","last_clean":"2024-04-25T08:35:41.275196+0000","last_became_active":"2024-04-25T08:35:41.274185+0000","last_became_peered":"2024-04-25T08:35:41.274185+0000","last_unstale":"2024-04-25T08:35:41.275196+0000","last_undegraded":"2024-04-25T08:35:41.275196+0000","last_fullsized":"2024-04-25T08:35:41.275196+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:18.949036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.288283+0000","last_change":"2024-04-25T08:35:41.287655+0000","last_active":"2024-04-25T08:35:41.288283+0000","last_peered":"2024-04-25T08:35:41.288283+0000","last_clean":"2024-04-25T08:35:41.288283+0000","last_became_active":"2024-04-25T08:35:41.287367+0000","last_became_peered":"2024-04-25T08:35:41.287367+0000","last_unstale":"2024-04-25T08:35:41.288283+0000","last_undegraded":"2024-04-25T08:35:41.288283+0000","last_fullsized":"2024-04-25T08:35:41.288283+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:14.998969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.282257+0000","last_change":"2024-04-25T08:35:41.281047+0000","last_active":"2024-04-25T08:35:41.282257+0000","last_peered":"2024-04-25T08:35:41.282257+0000","last_clean":"2024-04-25T08:35:41.282257+0000","last_became_active":"2024-04-25T08:35:41.280762+0000","last_became_peered":"2024-04-25T08:35:41.280762+0000","last_unstale":"2024-04-25T08:35:41.282257+0000","last_undegraded":"2024-04-25T08:35:41.282257+0000","last_fullsized":"2024-04-25T08:35:41.282257+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:03.266564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.282249+0000","last_change":"2024-04-25T08:35:41.280433+0000","last_active":"2024-04-25T08:35:41.282249+0000","last_peered":"2024-04-25T08:35:41.282249+0000","last_clean":"2024-04-25T08:35:41.282249+0000","last_became_active":"2024-04-25T08:35:41.280122+0000","last_became_peered":"2024-04-25T08:35:41.280122+0000","last_unstale":"2024-04-25T08:35:41.282249+0000","last_undegraded":"2024-04-25T08:35:41.282249+0000","last_fullsized":"2024-04-25T08:35:41.282249+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.153383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.278869+0000","last_change":"2024-04-25T08:35:41.275896+0000","last_active":"2024-04-25T08:35:41.278869+0000","last_peered":"2024-04-25T08:35:41.278869+0000","last_clean":"2024-04-25T08:35:41.278869+0000","last_became_active":"2024-04-25T08:35:41.275630+0000","last_became_peered":"2024-04-25T08:35:41.275630+0000","last_unstale":"2024-04-25T08:35:41.278869+0000","last_undegraded":"2024-04-25T08:35:41.278869+0000","last_fullsized":"2024-04-25T08:35:41.278869+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:00.186751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.285223+0000","last_change":"2024-04-25T08:35:41.284694+0000","last_active":"2024-04-25T08:35:41.285223+0000","last_peered":"2024-04-25T08:35:41.285223+0000","last_clean":"2024-04-25T08:35:41.285223+0000","last_became_active":"2024-04-25T08:35:41.284413+0000","last_became_peered":"2024-04-25T08:35:41.284413+0000","last_unstale":"2024-04-25T08:35:41.285223+0000","last_undegraded":"2024-04-25T08:35:41.285223+0000","last_fullsized":"2024-04-25T08:35:41.285223+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:44.396919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.285421+0000","last_change":"2024-04-25T08:35:41.284766+0000","last_active":"2024-04-25T08:35:41.285421+0000","last_peered":"2024-04-25T08:35:41.285421+0000","last_clean":"2024-04-25T08:35:41.285421+0000","last_became_active":"2024-04-25T08:35:41.284441+0000","last_became_peered":"2024-04-25T08:35:41.284441+0000","last_unstale":"2024-04-25T08:35:41.285421+0000","last_undegraded":"2024-04-25T08:35:41.285421+0000","last_fullsized":"2024-04-25T08:35:41.285421+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:16.741664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.283387+0000","last_change":"2024-04-25T08:35:41.282841+0000","last_active":"2024-04-25T08:35:41.283387+0000","last_peered":"2024-04-25T08:35:41.283387+0000","last_clean":"2024-04-25T08:35:41.283387+0000","last_became_active":"2024-04-25T08:35:41.282569+0000","last_became_peered":"2024-04-25T08:35:41.282569+0000","last_unstale":"2024-04-25T08:35:41.283387+0000","last_undegraded":"2024-04-25T08:35:41.283387+0000","last_fullsized":"2024-04-25T08:35:41.283387+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:15.313710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290896+0000","last_change":"2024-04-25T08:35:41.287424+0000","last_active":"2024-04-25T08:35:41.290896+0000","last_peered":"2024-04-25T08:35:41.290896+0000","last_clean":"2024-04-25T08:35:41.290896+0000","last_became_active":"2024-04-25T08:35:41.286068+0000","last_became_peered":"2024-04-25T08:35:41.286068+0000","last_unstale":"2024-04-25T08:35:41.290896+0000","last_undegraded":"2024-04-25T08:35:41.290896+0000","last_fullsized":"2024-04-25T08:35:41.290896+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:04.329215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.269651+0000","last_change":"2024-04-25T08:35:41.266298+0000","last_active":"2024-04-25T08:35:41.269651+0000","last_peered":"2024-04-25T08:35:41.269651+0000","last_clean":"2024-04-25T08:35:41.269651+0000","last_became_active":"2024-04-25T08:35:41.266026+0000","last_became_peered":"2024-04-25T08:35:41.266026+0000","last_unstale":"2024-04-25T08:35:41.269651+0000","last_undegraded":"2024-04-25T08:35:41.269651+0000","last_fullsized":"2024-04-25T08:35:41.269651+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:14.998969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.281394+0000","last_change":"2024-04-25T08:35:41.277529+0000","last_active":"2024-04-25T08:35:41.281394+0000","last_peered":"2024-04-25T08:35:41.281394+0000","last_clean":"2024-04-25T08:35:41.281394+0000","last_became_active":"2024-04-25T08:35:41.274806+0000","last_became_peered":"2024-04-25T08:35:41.274806+0000","last_unstale":"2024-04-25T08:35:41.281394+0000","last_undegraded":"2024-04-25T08:35:41.281394+0000","last_fullsized":"2024-04-25T08:35:41.281394+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:42.263995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.284768+0000","last_change":"2024-04-25T08:35:41.283352+0000","last_active":"2024-04-25T08:35:41.284768+0000","last_peered":"2024-04-25T08:35:41.284768+0000","last_clean":"2024-04-25T08:35:41.284768+0000","last_became_active":"2024-04-25T08:35:41.283063+0000","last_became_peered":"2024-04-25T08:35:41.283063+0000","last_unstale":"2024-04-25T08:35:41.284768+0000","last_undegraded":"2024-04-25T08:35:41.284768+0000","last_fullsized":"2024-04-25T08:35:41.284768+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:15.347371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.286740+0000","last_change":"2024-04-25T08:35:41.283499+0000","last_active":"2024-04-25T08:35:41.286740+0000","last_peered":"2024-04-25T08:35:41.286740+0000","last_clean":"2024-04-25T08:35:41.286740+0000","last_became_active":"2024-04-25T08:35:41.283197+0000","last_became_peered":"2024-04-25T08:35:41.283197+0000","last_unstale":"2024-04-25T08:35:41.286740+0000","last_undegraded":"2024-04-25T08:35:41.286740+0000","last_fullsized":"2024-04-25T08:35:41.286740+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.359399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290741+0000","last_change":"2024-04-25T08:35:41.290386+0000","last_active":"2024-04-25T08:35:41.290741+0000","last_peered":"2024-04-25T08:35:41.290741+0000","last_clean":"2024-04-25T08:35:41.290741+0000","last_became_active":"2024-04-25T08:35:41.290048+0000","last_became_peered":"2024-04-25T08:35:41.290048+0000","last_unstale":"2024-04-25T08:35:41.290741+0000","last_undegraded":"2024-04-25T08:35:41.290741+0000","last_fullsized":"2024-04-25T08:35:41.290741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:37.876835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287609+0000","last_change":"2024-04-25T08:35:40.265386+0000","last_active":"2024-04-25T08:35:41.287609+0000","last_peered":"2024-04-25T08:35:41.287609+0000","last_clean":"2024-04-25T08:35:41.287609+0000","last_became_active":"2024-04-25T08:35:40.264762+0000","last_became_peered":"2024-04-25T08:35:40.264762+0000","last_unstale":"2024-04-25T08:35:41.287609+0000","last_undegraded":"2024-04-25T08:35:41.287609+0000","last_fullsized":"2024-04-25T08:35:41.287609+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:36.874371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.264187+0000","last_change":"2024-04-25T08:35:41.263680+0000","last_active":"2024-04-25T08:35:41.264187+0000","last_peered":"2024-04-25T08:35:41.264187+0000","last_clean":"2024-04-25T08:35:41.264187+0000","last_became_active":"2024-04-25T08:35:41.263370+0000","last_became_peered":"2024-04-25T08:35:41.263370+0000","last_unstale":"2024-04-25T08:35:41.264187+0000","last_undegraded":"2024-04-25T08:35:41.264187+0000","last_fullsized":"2024-04-25T08:35:41.264187+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:55.300955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.280970+0000","last_change":"2024-04-25T08:35:40.260869+0000","last_active":"2024-04-25T08:35:41.280970+0000","last_peered":"2024-04-25T08:35:41.280970+0000","last_clean":"2024-04-25T08:35:41.280970+0000","last_became_active":"2024-04-25T08:35:40.260546+0000","last_became_peered":"2024-04-25T08:35:40.260546+0000","last_unstale":"2024-04-25T08:35:41.280970+0000","last_undegraded":"2024-04-25T08:35:41.280970+0000","last_fullsized":"2024-04-25T08:35:41.280970+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:40.874197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.286291+0000","last_change":"2024-04-25T08:35:41.284290+0000","last_active":"2024-04-25T08:35:41.286291+0000","last_peered":"2024-04-25T08:35:41.286291+0000","last_clean":"2024-04-25T08:35:41.286291+0000","last_became_active":"2024-04-25T08:35:41.283940+0000","last_became_peered":"2024-04-25T08:35:41.283940+0000","last_unstale":"2024-04-25T08:35:41.286291+0000","last_undegraded":"2024-04-25T08:35:41.286291+0000","last_fullsized":"2024-04-25T08:35:41.286291+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:40.595051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.276051+0000","last_change":"2024-04-25T08:35:40.267779+0000","last_active":"2024-04-25T08:35:41.276051+0000","last_peered":"2024-04-25T08:35:41.276051+0000","last_clean":"2024-04-25T08:35:41.276051+0000","last_became_active":"2024-04-25T08:35:40.267476+0000","last_became_peered":"2024-04-25T08:35:40.267476+0000","last_unstale":"2024-04-25T08:35:41.276051+0000","last_undegraded":"2024-04-25T08:35:41.276051+0000","last_fullsized":"2024-04-25T08:35:41.276051+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:00.296484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287977+0000","last_change":"2024-04-25T08:35:41.287151+0000","last_active":"2024-04-25T08:35:41.287977+0000","last_peered":"2024-04-25T08:35:41.287977+0000","last_clean":"2024-04-25T08:35:41.287977+0000","last_became_active":"2024-04-25T08:35:41.286838+0000","last_became_peered":"2024-04-25T08:35:41.286838+0000","last_unstale":"2024-04-25T08:35:41.287977+0000","last_undegraded":"2024-04-25T08:35:41.287977+0000","last_fullsized":"2024-04-25T08:35:41.287977+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:03.658493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.289494+0000","last_change":"2024-04-25T08:35:40.265561+0000","last_active":"2024-04-25T08:35:41.289494+0000","last_peered":"2024-04-25T08:35:41.289494+0000","last_clean":"2024-04-25T08:35:41.289494+0000","last_became_active":"2024-04-25T08:35:40.265285+0000","last_became_peered":"2024-04-25T08:35:40.265285+0000","last_unstale":"2024-04-25T08:35:41.289494+0000","last_undegraded":"2024-04-25T08:35:41.289494+0000","last_fullsized":"2024-04-25T08:35:41.289494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:07.150919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287952+0000","last_change":"2024-04-25T08:35:41.287012+0000","last_active":"2024-04-25T08:35:41.287952+0000","last_peered":"2024-04-25T08:35:41.287952+0000","last_clean":"2024-04-25T08:35:41.287952+0000","last_became_active":"2024-04-25T08:35:41.286710+0000","last_became_peered":"2024-04-25T08:35:41.286710+0000","last_unstale":"2024-04-25T08:35:41.287952+0000","last_undegraded":"2024-04-25T08:35:41.287952+0000","last_fullsized":"2024-04-25T08:35:41.287952+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:15.971000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.281452+0000","last_change":"2024-04-25T08:35:40.261660+0000","last_active":"2024-04-25T08:35:41.281452+0000","last_peered":"2024-04-25T08:35:41.281452+0000","last_clean":"2024-04-25T08:35:41.281452+0000","last_became_active":"2024-04-25T08:35:40.261355+0000","last_became_peered":"2024-04-25T08:35:40.261355+0000","last_unstale":"2024-04-25T08:35:41.281452+0000","last_undegraded":"2024-04-25T08:35:41.281452+0000","last_fullsized":"2024-04-25T08:35:41.281452+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:40.359717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.288773+0000","last_change":"2024-04-25T08:35:41.281657+0000","last_active":"2024-04-25T08:35:41.288773+0000","last_peered":"2024-04-25T08:35:41.288773+0000","last_clean":"2024-04-25T08:35:41.288773+0000","last_became_active":"2024-04-25T08:35:41.280363+0000","last_became_peered":"2024-04-25T08:35:41.280363+0000","last_unstale":"2024-04-25T08:35:41.288773+0000","last_undegraded":"2024-04-25T08:35:41.288773+0000","last_fullsized":"2024-04-25T08:35:41.288773+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:30.344111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.284232+0000","last_change":"2024-04-25T08:35:40.260460+0000","last_active":"2024-04-25T08:35:41.284232+0000","last_peered":"2024-04-25T08:35:41.284232+0000","last_clean":"2024-04-25T08:35:41.284232+0000","last_became_active":"2024-04-25T08:35:40.260157+0000","last_became_peered":"2024-04-25T08:35:40.260157+0000","last_unstale":"2024-04-25T08:35:41.284232+0000","last_undegraded":"2024-04-25T08:35:41.284232+0000","last_fullsized":"2024-04-25T08:35:41.284232+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:47.688555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.279024+0000","last_change":"2024-04-25T08:35:41.278483+0000","last_active":"2024-04-25T08:35:41.279024+0000","last_peered":"2024-04-25T08:35:41.279024+0000","last_clean":"2024-04-25T08:35:41.279024+0000","last_became_active":"2024-04-25T08:35:41.278210+0000","last_became_peered":"2024-04-25T08:35:41.278210+0000","last_unstale":"2024-04-25T08:35:41.279024+0000","last_undegraded":"2024-04-25T08:35:41.279024+0000","last_fullsized":"2024-04-25T08:35:41.279024+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:27.862067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.268154+0000","last_change":"2024-04-25T08:35:40.264492+0000","last_active":"2024-04-25T08:35:41.268154+0000","last_peered":"2024-04-25T08:35:41.268154+0000","last_clean":"2024-04-25T08:35:41.268154+0000","last_became_active":"2024-04-25T08:35:40.264093+0000","last_became_peered":"2024-04-25T08:35:40.264093+0000","last_unstale":"2024-04-25T08:35:41.268154+0000","last_undegraded":"2024-04-25T08:35:41.268154+0000","last_fullsized":"2024-04-25T08:35:41.268154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:02.656029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.274420+0000","last_change":"2024-04-25T08:35:41.268358+0000","last_active":"2024-04-25T08:35:41.274420+0000","last_peered":"2024-04-25T08:35:41.274420+0000","last_clean":"2024-04-25T08:35:41.274420+0000","last_became_active":"2024-04-25T08:35:41.267757+0000","last_became_peered":"2024-04-25T08:35:41.267757+0000","last_unstale":"2024-04-25T08:35:41.274420+0000","last_undegraded":"2024-04-25T08:35:41.274420+0000","last_fullsized":"2024-04-25T08:35:41.274420+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:24.182066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.258304+0000","last_change":"2024-04-25T08:35:40.260833+0000","last_active":"2024-04-25T08:35:41.258304+0000","last_peered":"2024-04-25T08:35:41.258304+0000","last_clean":"2024-04-25T08:35:41.258304+0000","last_became_active":"2024-04-25T08:35:40.260521+0000","last_became_peered":"2024-04-25T08:35:40.260521+0000","last_unstale":"2024-04-25T08:35:41.258304+0000","last_undegraded":"2024-04-25T08:35:41.258304+0000","last_fullsized":"2024-04-25T08:35:41.258304+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:38.576626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.289090+0000","last_change":"2024-04-25T08:35:41.284386+0000","last_active":"2024-04-25T08:35:41.289090+0000","last_peered":"2024-04-25T08:35:41.289090+0000","last_clean":"2024-04-25T08:35:41.289090+0000","last_became_active":"2024-04-25T08:35:41.284082+0000","last_became_peered":"2024-04-25T08:35:41.284082+0000","last_unstale":"2024-04-25T08:35:41.289090+0000","last_undegraded":"2024-04-25T08:35:41.289090+0000","last_fullsized":"2024-04-25T08:35:41.289090+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:57.930974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.260604+0000","last_change":"2024-04-25T08:35:40.267540+0000","last_active":"2024-04-25T08:35:41.260604+0000","last_peered":"2024-04-25T08:35:41.260604+0000","last_clean":"2024-04-25T08:35:41.260604+0000","last_became_active":"2024-04-25T08:35:40.267246+0000","last_became_peered":"2024-04-25T08:35:40.267246+0000","last_unstale":"2024-04-25T08:35:41.260604+0000","last_undegraded":"2024-04-25T08:35:41.260604+0000","last_fullsized":"2024-04-25T08:35:41.260604+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:41.261531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.274824+0000","last_change":"2024-04-25T08:35:41.272072+0000","last_active":"2024-04-25T08:35:41.274824+0000","last_peered":"2024-04-25T08:35:41.274824+0000","last_clean":"2024-04-25T08:35:41.274824+0000","last_became_active":"2024-04-25T08:35:41.271732+0000","last_became_peered":"2024-04-25T08:35:41.271732+0000","last_unstale":"2024-04-25T08:35:41.274824+0000","last_undegraded":"2024-04-25T08:35:41.274824+0000","last_fullsized":"2024-04-25T08:35:41.274824+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:41.876660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.258119+0000","last_change":"2024-04-25T08:35:40.267085+0000","last_active":"2024-04-25T08:35:41.258119+0000","last_peered":"2024-04-25T08:35:41.258119+0000","last_clean":"2024-04-25T08:35:41.258119+0000","last_became_active":"2024-04-25T08:35:40.266626+0000","last_became_peered":"2024-04-25T08:35:40.266626+0000","last_unstale":"2024-04-25T08:35:41.258119+0000","last_undegraded":"2024-04-25T08:35:41.258119+0000","last_fullsized":"2024-04-25T08:35:41.258119+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:28.384828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.274432+0000","last_change":"2024-04-25T08:35:41.273775+0000","last_active":"2024-04-25T08:35:41.274432+0000","last_peered":"2024-04-25T08:35:41.274432+0000","last_clean":"2024-04-25T08:35:41.274432+0000","last_became_active":"2024-04-25T08:35:41.273480+0000","last_became_peered":"2024-04-25T08:35:41.273480+0000","last_unstale":"2024-04-25T08:35:41.274432+0000","last_undegraded":"2024-04-25T08:35:41.274432+0000","last_fullsized":"2024-04-25T08:35:41.274432+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:23.681487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.289906+0000","last_change":"2024-04-25T08:35:40.265428+0000","last_active":"2024-04-25T08:35:41.289906+0000","last_peered":"2024-04-25T08:35:41.289906+0000","last_clean":"2024-04-25T08:35:41.289906+0000","last_became_active":"2024-04-25T08:35:40.265126+0000","last_became_peered":"2024-04-25T08:35:40.265126+0000","last_unstale":"2024-04-25T08:35:41.289906+0000","last_undegraded":"2024-04-25T08:35:41.289906+0000","last_fullsized":"2024-04-25T08:35:41.289906+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:39.576465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.267446+0000","last_change":"2024-04-25T08:35:41.266887+0000","last_active":"2024-04-25T08:35:41.267446+0000","last_peered":"2024-04-25T08:35:41.267446+0000","last_clean":"2024-04-25T08:35:41.267446+0000","last_became_active":"2024-04-25T08:35:41.266567+0000","last_became_peered":"2024-04-25T08:35:41.266567+0000","last_unstale":"2024-04-25T08:35:41.267446+0000","last_undegraded":"2024-04-25T08:35:41.267446+0000","last_fullsized":"2024-04-25T08:35:41.267446+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:39.579089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.291264+0000","last_change":"2024-04-25T08:35:40.263573+0000","last_active":"2024-04-25T08:35:41.291264+0000","last_peered":"2024-04-25T08:35:41.291264+0000","last_clean":"2024-04-25T08:35:41.291264+0000","last_became_active":"2024-04-25T08:35:40.263116+0000","last_became_peered":"2024-04-25T08:35:40.263116+0000","last_unstale":"2024-04-25T08:35:41.291264+0000","last_undegraded":"2024-04-25T08:35:41.291264+0000","last_fullsized":"2024-04-25T08:35:41.291264+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:01.463669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.266709+0000","last_change":"2024-04-25T08:35:41.266135+0000","last_active":"2024-04-25T08:35:41.266709+0000","last_peered":"2024-04-25T08:35:41.266709+0000","last_clean":"2024-04-25T08:35:41.266709+0000","last_became_active":"2024-04-25T08:35:41.264700+0000","last_became_peered":"2024-04-25T08:35:41.264700+0000","last_unstale":"2024-04-25T08:35:41.266709+0000","last_undegraded":"2024-04-25T08:35:41.266709+0000","last_fullsized":"2024-04-25T08:35:41.266709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:14.635814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.277885+0000","last_change":"2024-04-25T08:35:40.263814+0000","last_active":"2024-04-25T08:35:41.277885+0000","last_peered":"2024-04-25T08:35:41.277885+0000","last_clean":"2024-04-25T08:35:41.277885+0000","last_became_active":"2024-04-25T08:35:40.261759+0000","last_became_peered":"2024-04-25T08:35:40.261759+0000","last_unstale":"2024-04-25T08:35:41.277885+0000","last_undegraded":"2024-04-25T08:35:41.277885+0000","last_fullsized":"2024-04-25T08:35:41.277885+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:24.511890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.277477+0000","last_change":"2024-04-25T08:35:41.272440+0000","last_active":"2024-04-25T08:35:41.277477+0000","last_peered":"2024-04-25T08:35:41.277477+0000","last_clean":"2024-04-25T08:35:41.277477+0000","last_became_active":"2024-04-25T08:35:41.272175+0000","last_became_peered":"2024-04-25T08:35:41.272175+0000","last_unstale":"2024-04-25T08:35:41.277477+0000","last_undegraded":"2024-04-25T08:35:41.277477+0000","last_fullsized":"2024-04-25T08:35:41.277477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:07.864540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.269597+0000","last_change":"2024-04-25T08:35:40.268329+0000","last_active":"2024-04-25T08:35:41.269597+0000","last_peered":"2024-04-25T08:35:41.269597+0000","last_clean":"2024-04-25T08:35:41.269597+0000","last_became_active":"2024-04-25T08:35:40.267977+0000","last_became_peered":"2024-04-25T08:35:40.267977+0000","last_unstale":"2024-04-25T08:35:41.269597+0000","last_undegraded":"2024-04-25T08:35:41.269597+0000","last_fullsized":"2024-04-25T08:35:41.269597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:03.326751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.278994+0000","last_change":"2024-04-25T08:35:41.277480+0000","last_active":"2024-04-25T08:35:41.278994+0000","last_peered":"2024-04-25T08:35:41.278994+0000","last_clean":"2024-04-25T08:35:41.278994+0000","last_became_active":"2024-04-25T08:35:41.277195+0000","last_became_peered":"2024-04-25T08:35:41.277195+0000","last_unstale":"2024-04-25T08:35:41.278994+0000","last_undegraded":"2024-04-25T08:35:41.278994+0000","last_fullsized":"2024-04-25T08:35:41.278994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:21.411829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.260680+0000","last_change":"2024-04-25T08:35:40.260776+0000","last_active":"2024-04-25T08:35:41.260680+0000","last_peered":"2024-04-25T08:35:41.260680+0000","last_clean":"2024-04-25T08:35:41.260680+0000","last_became_active":"2024-04-25T08:35:40.260496+0000","last_became_peered":"2024-04-25T08:35:40.260496+0000","last_unstale":"2024-04-25T08:35:41.260680+0000","last_undegraded":"2024-04-25T08:35:41.260680+0000","last_fullsized":"2024-04-25T08:35:41.260680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:38.919175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290749+0000","last_change":"2024-04-25T08:35:41.285449+0000","last_active":"2024-04-25T08:35:41.290749+0000","last_peered":"2024-04-25T08:35:41.290749+0000","last_clean":"2024-04-25T08:35:41.290749+0000","last_became_active":"2024-04-25T08:35:41.285145+0000","last_became_peered":"2024-04-25T08:35:41.285145+0000","last_unstale":"2024-04-25T08:35:41.290749+0000","last_undegraded":"2024-04-25T08:35:41.290749+0000","last_fullsized":"2024-04-25T08:35:41.290749+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:59.698763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.285711+0000","last_change":"2024-04-25T08:35:40.258994+0000","last_active":"2024-04-25T08:35:41.285711+0000","last_peered":"2024-04-25T08:35:41.285711+0000","last_clean":"2024-04-25T08:35:41.285711+0000","last_became_active":"2024-04-25T08:35:40.258685+0000","last_became_peered":"2024-04-25T08:35:40.258685+0000","last_unstale":"2024-04-25T08:35:41.285711+0000","last_undegraded":"2024-04-25T08:35:41.285711+0000","last_fullsized":"2024-04-25T08:35:41.285711+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:32.349483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.267941+0000","last_change":"2024-04-25T08:35:41.266844+0000","last_active":"2024-04-25T08:35:41.267941+0000","last_peered":"2024-04-25T08:35:41.267941+0000","last_clean":"2024-04-25T08:35:41.267941+0000","last_became_active":"2024-04-25T08:35:41.266459+0000","last_became_peered":"2024-04-25T08:35:41.266459+0000","last_unstale":"2024-04-25T08:35:41.267941+0000","last_undegraded":"2024-04-25T08:35:41.267941+0000","last_fullsized":"2024-04-25T08:35:41.267941+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:16.741664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.289265+0000","last_change":"2024-04-25T08:35:40.270098+0000","last_active":"2024-04-25T08:35:41.289265+0000","last_peered":"2024-04-25T08:35:41.289265+0000","last_clean":"2024-04-25T08:35:41.289265+0000","last_became_active":"2024-04-25T08:35:40.269823+0000","last_became_peered":"2024-04-25T08:35:40.269823+0000","last_unstale":"2024-04-25T08:35:41.289265+0000","last_undegraded":"2024-04-25T08:35:41.289265+0000","last_fullsized":"2024-04-25T08:35:41.289265+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:40.450827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.268983+0000","last_change":"2024-04-25T08:35:41.261767+0000","last_active":"2024-04-25T08:35:41.268983+0000","last_peered":"2024-04-25T08:35:41.268983+0000","last_clean":"2024-04-25T08:35:41.268983+0000","last_became_active":"2024-04-25T08:35:41.261457+0000","last_became_peered":"2024-04-25T08:35:41.261457+0000","last_unstale":"2024-04-25T08:35:41.268983+0000","last_undegraded":"2024-04-25T08:35:41.268983+0000","last_fullsized":"2024-04-25T08:35:41.268983+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:01.633527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.281134+0000","last_change":"2024-04-25T08:35:40.260112+0000","last_active":"2024-04-25T08:35:41.281134+0000","last_peered":"2024-04-25T08:35:41.281134+0000","last_clean":"2024-04-25T08:35:41.281134+0000","last_became_active":"2024-04-25T08:35:40.258446+0000","last_became_peered":"2024-04-25T08:35:40.258446+0000","last_unstale":"2024-04-25T08:35:41.281134+0000","last_undegraded":"2024-04-25T08:35:41.281134+0000","last_fullsized":"2024-04-25T08:35:41.281134+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:06.862076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.263705+0000","last_change":"2024-04-25T08:35:41.262963+0000","last_active":"2024-04-25T08:35:41.263705+0000","last_peered":"2024-04-25T08:35:41.263705+0000","last_clean":"2024-04-25T08:35:41.263705+0000","last_became_active":"2024-04-25T08:35:41.262686+0000","last_became_peered":"2024-04-25T08:35:41.262686+0000","last_unstale":"2024-04-25T08:35:41.263705+0000","last_undegraded":"2024-04-25T08:35:41.263705+0000","last_fullsized":"2024-04-25T08:35:41.263705+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:02.003427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.289938+0000","last_change":"2024-04-25T08:35:40.266422+0000","last_active":"2024-04-25T08:35:41.289938+0000","last_peered":"2024-04-25T08:35:41.289938+0000","last_clean":"2024-04-25T08:35:41.289938+0000","last_became_active":"2024-04-25T08:35:40.266160+0000","last_became_peered":"2024-04-25T08:35:40.266160+0000","last_unstale":"2024-04-25T08:35:41.289938+0000","last_undegraded":"2024-04-25T08:35:41.289938+0000","last_fullsized":"2024-04-25T08:35:41.289938+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:21.997308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.270502+0000","last_change":"2024-04-25T08:35:41.268401+0000","last_active":"2024-04-25T08:35:41.270502+0000","last_peered":"2024-04-25T08:35:41.270502+0000","last_clean":"2024-04-25T08:35:41.270502+0000","last_became_active":"2024-04-25T08:35:41.267365+0000","last_became_peered":"2024-04-25T08:35:41.267365+0000","last_unstale":"2024-04-25T08:35:41.270502+0000","last_undegraded":"2024-04-25T08:35:41.270502+0000","last_fullsized":"2024-04-25T08:35:41.270502+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:15.347371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.263627+0000","last_change":"2024-04-25T08:35:40.262760+0000","last_active":"2024-04-25T08:35:41.263627+0000","last_peered":"2024-04-25T08:35:41.263627+0000","last_clean":"2024-04-25T08:35:41.263627+0000","last_became_active":"2024-04-25T08:35:40.262269+0000","last_became_peered":"2024-04-25T08:35:40.262269+0000","last_unstale":"2024-04-25T08:35:41.263627+0000","last_undegraded":"2024-04-25T08:35:41.263627+0000","last_fullsized":"2024-04-25T08:35:41.263627+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:35.351092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.284765+0000","last_change":"2024-04-25T08:35:41.281101+0000","last_active":"2024-04-25T08:35:41.284765+0000","last_peered":"2024-04-25T08:35:41.284765+0000","last_clean":"2024-04-25T08:35:41.284765+0000","last_became_active":"2024-04-25T08:35:41.280675+0000","last_became_peered":"2024-04-25T08:35:41.280675+0000","last_unstale":"2024-04-25T08:35:41.284765+0000","last_undegraded":"2024-04-25T08:35:41.284765+0000","last_fullsized":"2024-04-25T08:35:41.284765+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:03.389798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.275440+0000","last_change":"2024-04-25T08:35:40.262619+0000","last_active":"2024-04-25T08:35:41.275440+0000","last_peered":"2024-04-25T08:35:41.275440+0000","last_clean":"2024-04-25T08:35:41.275440+0000","last_became_active":"2024-04-25T08:35:40.261539+0000","last_became_peered":"2024-04-25T08:35:40.261539+0000","last_unstale":"2024-04-25T08:35:41.275440+0000","last_undegraded":"2024-04-25T08:35:41.275440+0000","last_fullsized":"2024-04-25T08:35:41.275440+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:17.946572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.278190+0000","last_change":"2024-04-25T08:35:41.276673+0000","last_active":"2024-04-25T08:35:41.278190+0000","last_peered":"2024-04-25T08:35:41.278190+0000","last_clean":"2024-04-25T08:35:41.278190+0000","last_became_active":"2024-04-25T08:35:41.276411+0000","last_became_peered":"2024-04-25T08:35:41.276411+0000","last_unstale":"2024-04-25T08:35:41.278190+0000","last_undegraded":"2024-04-25T08:35:41.278190+0000","last_fullsized":"2024-04-25T08:35:41.278190+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:22.999772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.269757+0000","last_change":"2024-04-25T08:35:40.267673+0000","last_active":"2024-04-25T08:35:41.269757+0000","last_peered":"2024-04-25T08:35:41.269757+0000","last_clean":"2024-04-25T08:35:41.269757+0000","last_became_active":"2024-04-25T08:35:40.267362+0000","last_became_peered":"2024-04-25T08:35:40.267362+0000","last_unstale":"2024-04-25T08:35:41.269757+0000","last_undegraded":"2024-04-25T08:35:41.269757+0000","last_fullsized":"2024-04-25T08:35:41.269757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:31.396343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.278608+0000","last_change":"2024-04-25T08:35:41.276828+0000","last_active":"2024-04-25T08:35:41.278608+0000","last_peered":"2024-04-25T08:35:41.278608+0000","last_clean":"2024-04-25T08:35:41.278608+0000","last_became_active":"2024-04-25T08:35:41.276544+0000","last_became_peered":"2024-04-25T08:35:41.276544+0000","last_unstale":"2024-04-25T08:35:41.278608+0000","last_undegraded":"2024-04-25T08:35:41.278608+0000","last_fullsized":"2024-04-25T08:35:41.278608+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:41.270237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.264077+0000","last_change":"2024-04-25T08:35:40.258315+0000","last_active":"2024-04-25T08:35:41.264077+0000","last_peered":"2024-04-25T08:35:41.264077+0000","last_clean":"2024-04-25T08:35:41.264077+0000","last_became_active":"2024-04-25T08:35:40.258006+0000","last_became_peered":"2024-04-25T08:35:40.258006+0000","last_unstale":"2024-04-25T08:35:41.264077+0000","last_undegraded":"2024-04-25T08:35:41.264077+0000","last_fullsized":"2024-04-25T08:35:41.264077+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.179602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.278587+0000","last_change":"2024-04-25T08:35:41.276010+0000","last_active":"2024-04-25T08:35:41.278587+0000","last_peered":"2024-04-25T08:35:41.278587+0000","last_clean":"2024-04-25T08:35:41.278587+0000","last_became_active":"2024-04-25T08:35:41.275634+0000","last_became_peered":"2024-04-25T08:35:41.275634+0000","last_unstale":"2024-04-25T08:35:41.278587+0000","last_undegraded":"2024-04-25T08:35:41.278587+0000","last_fullsized":"2024-04-25T08:35:41.278587+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:02.639501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.279810+0000","last_change":"2024-04-25T08:35:40.268768+0000","last_active":"2024-04-25T08:35:41.279810+0000","last_peered":"2024-04-25T08:35:41.279810+0000","last_clean":"2024-04-25T08:35:41.279810+0000","last_became_active":"2024-04-25T08:35:40.268480+0000","last_became_peered":"2024-04-25T08:35:40.268480+0000","last_unstale":"2024-04-25T08:35:41.279810+0000","last_undegraded":"2024-04-25T08:35:41.279810+0000","last_fullsized":"2024-04-25T08:35:41.279810+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:45.095049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.283992+0000","last_change":"2024-04-25T08:35:41.280199+0000","last_active":"2024-04-25T08:35:41.283992+0000","last_peered":"2024-04-25T08:35:41.283992+0000","last_clean":"2024-04-25T08:35:41.283992+0000","last_became_active":"2024-04-25T08:35:41.279959+0000","last_became_peered":"2024-04-25T08:35:41.279959+0000","last_unstale":"2024-04-25T08:35:41.283992+0000","last_undegraded":"2024-04-25T08:35:41.283992+0000","last_fullsized":"2024-04-25T08:35:41.283992+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:24.514514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.269620+0000","last_change":"2024-04-25T08:35:40.267806+0000","last_active":"2024-04-25T08:35:41.269620+0000","last_peered":"2024-04-25T08:35:41.269620+0000","last_clean":"2024-04-25T08:35:41.269620+0000","last_became_active":"2024-04-25T08:35:40.267428+0000","last_became_peered":"2024-04-25T08:35:40.267428+0000","last_unstale":"2024-04-25T08:35:41.269620+0000","last_undegraded":"2024-04-25T08:35:41.269620+0000","last_fullsized":"2024-04-25T08:35:41.269620+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:17.189446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.280468+0000","last_change":"2024-04-25T08:35:41.279565+0000","last_active":"2024-04-25T08:35:41.280468+0000","last_peered":"2024-04-25T08:35:41.280468+0000","last_clean":"2024-04-25T08:35:41.280468+0000","last_became_active":"2024-04-25T08:35:41.279273+0000","last_became_peered":"2024-04-25T08:35:41.279273+0000","last_unstale":"2024-04-25T08:35:41.280468+0000","last_undegraded":"2024-04-25T08:35:41.280468+0000","last_fullsized":"2024-04-25T08:35:41.280468+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:39.921639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.265510+0000","last_change":"2024-04-25T08:35:40.266497+0000","last_active":"2024-04-25T08:35:41.265510+0000","last_peered":"2024-04-25T08:35:41.265510+0000","last_clean":"2024-04-25T08:35:41.265510+0000","last_became_active":"2024-04-25T08:35:40.266220+0000","last_became_peered":"2024-04-25T08:35:40.266220+0000","last_unstale":"2024-04-25T08:35:41.265510+0000","last_undegraded":"2024-04-25T08:35:41.265510+0000","last_fullsized":"2024-04-25T08:35:41.265510+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:47.688555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.270481+0000","last_change":"2024-04-25T08:35:41.267541+0000","last_active":"2024-04-25T08:35:41.270481+0000","last_peered":"2024-04-25T08:35:41.270481+0000","last_clean":"2024-04-25T08:35:41.270481+0000","last_became_active":"2024-04-25T08:35:41.267260+0000","last_became_peered":"2024-04-25T08:35:41.267260+0000","last_unstale":"2024-04-25T08:35:41.270481+0000","last_undegraded":"2024-04-25T08:35:41.270481+0000","last_fullsized":"2024-04-25T08:35:41.270481+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:15.313710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.272254+0000","last_change":"2024-04-25T08:35:40.258676+0000","last_active":"2024-04-25T08:35:41.272254+0000","last_peered":"2024-04-25T08:35:41.272254+0000","last_clean":"2024-04-25T08:35:41.272254+0000","last_became_active":"2024-04-25T08:35:40.258389+0000","last_became_peered":"2024-04-25T08:35:40.258389+0000","last_unstale":"2024-04-25T08:35:41.272254+0000","last_undegraded":"2024-04-25T08:35:41.272254+0000","last_fullsized":"2024-04-25T08:35:41.272254+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:22.053802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287850+0000","last_change":"2024-04-25T08:35:41.286638+0000","last_active":"2024-04-25T08:35:41.287850+0000","last_peered":"2024-04-25T08:35:41.287850+0000","last_clean":"2024-04-25T08:35:41.287850+0000","last_became_active":"2024-04-25T08:35:41.286367+0000","last_became_peered":"2024-04-25T08:35:41.286367+0000","last_unstale":"2024-04-25T08:35:41.287850+0000","last_undegraded":"2024-04-25T08:35:41.287850+0000","last_fullsized":"2024-04-25T08:35:41.287850+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:23.056265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.259778+0000","last_change":"2024-04-25T08:35:40.261449+0000","last_active":"2024-04-25T08:35:41.259778+0000","last_peered":"2024-04-25T08:35:41.259778+0000","last_clean":"2024-04-25T08:35:41.259778+0000","last_became_active":"2024-04-25T08:35:40.261186+0000","last_became_peered":"2024-04-25T08:35:40.261186+0000","last_unstale":"2024-04-25T08:35:41.259778+0000","last_undegraded":"2024-04-25T08:35:41.259778+0000","last_fullsized":"2024-04-25T08:35:41.259778+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:29.601720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.266289+0000","last_change":"2024-04-25T08:35:41.264247+0000","last_active":"2024-04-25T08:35:41.266289+0000","last_peered":"2024-04-25T08:35:41.266289+0000","last_clean":"2024-04-25T08:35:41.266289+0000","last_became_active":"2024-04-25T08:35:41.263934+0000","last_became_peered":"2024-04-25T08:35:41.263934+0000","last_unstale":"2024-04-25T08:35:41.266289+0000","last_undegraded":"2024-04-25T08:35:41.266289+0000","last_fullsized":"2024-04-25T08:35:41.266289+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:32.631968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.268601+0000","last_change":"2024-04-25T08:35:40.265800+0000","last_active":"2024-04-25T08:35:41.268601+0000","last_peered":"2024-04-25T08:35:41.268601+0000","last_clean":"2024-04-25T08:35:41.268601+0000","last_became_active":"2024-04-25T08:35:40.265441+0000","last_became_peered":"2024-04-25T08:35:40.265441+0000","last_unstale":"2024-04-25T08:35:41.268601+0000","last_undegraded":"2024-04-25T08:35:41.268601+0000","last_fullsized":"2024-04-25T08:35:41.268601+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:00.296484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290843+0000","last_change":"2024-04-25T08:35:41.286509+0000","last_active":"2024-04-25T08:35:41.290843+0000","last_peered":"2024-04-25T08:35:41.290843+0000","last_clean":"2024-04-25T08:35:41.290843+0000","last_became_active":"2024-04-25T08:35:41.286192+0000","last_became_peered":"2024-04-25T08:35:41.286192+0000","last_unstale":"2024-04-25T08:35:41.290843+0000","last_undegraded":"2024-04-25T08:35:41.290843+0000","last_fullsized":"2024-04-25T08:35:41.290843+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:23.926844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.268324+0000","last_change":"2024-04-25T08:35:40.261296+0000","last_active":"2024-04-25T08:35:41.268324+0000","last_peered":"2024-04-25T08:35:41.268324+0000","last_clean":"2024-04-25T08:35:41.268324+0000","last_became_active":"2024-04-25T08:35:40.260938+0000","last_became_peered":"2024-04-25T08:35:40.260938+0000","last_unstale":"2024-04-25T08:35:41.268324+0000","last_undegraded":"2024-04-25T08:35:41.268324+0000","last_fullsized":"2024-04-25T08:35:41.268324+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:57.580825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.285801+0000","last_change":"2024-04-25T08:35:41.281046+0000","last_active":"2024-04-25T08:35:41.285801+0000","last_peered":"2024-04-25T08:35:41.285801+0000","last_clean":"2024-04-25T08:35:41.285801+0000","last_became_active":"2024-04-25T08:35:41.279338+0000","last_became_peered":"2024-04-25T08:35:41.279338+0000","last_unstale":"2024-04-25T08:35:41.285801+0000","last_undegraded":"2024-04-25T08:35:41.285801+0000","last_fullsized":"2024-04-25T08:35:41.285801+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:40.314635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.269694+0000","last_change":"2024-04-25T08:35:40.261801+0000","last_active":"2024-04-25T08:35:41.269694+0000","last_peered":"2024-04-25T08:35:41.269694+0000","last_clean":"2024-04-25T08:35:41.269694+0000","last_became_active":"2024-04-25T08:35:40.261540+0000","last_became_peered":"2024-04-25T08:35:40.261540+0000","last_unstale":"2024-04-25T08:35:41.269694+0000","last_undegraded":"2024-04-25T08:35:41.269694+0000","last_fullsized":"2024-04-25T08:35:41.269694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:26.859603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287126+0000","last_change":"2024-04-25T08:35:41.286523+0000","last_active":"2024-04-25T08:35:41.287126+0000","last_peered":"2024-04-25T08:35:41.287126+0000","last_clean":"2024-04-25T08:35:41.287126+0000","last_became_active":"2024-04-25T08:35:41.286236+0000","last_became_peered":"2024-04-25T08:35:41.286236+0000","last_unstale":"2024-04-25T08:35:41.287126+0000","last_undegraded":"2024-04-25T08:35:41.287126+0000","last_fullsized":"2024-04-25T08:35:41.287126+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:02.466133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.262340+0000","last_change":"2024-04-25T08:35:40.261725+0000","last_active":"2024-04-25T08:35:41.262340+0000","last_peered":"2024-04-25T08:35:41.262340+0000","last_clean":"2024-04-25T08:35:41.262340+0000","last_became_active":"2024-04-25T08:35:40.261441+0000","last_became_peered":"2024-04-25T08:35:40.261441+0000","last_unstale":"2024-04-25T08:35:41.262340+0000","last_undegraded":"2024-04-25T08:35:41.262340+0000","last_fullsized":"2024-04-25T08:35:41.262340+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:15.030362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":258,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.261874+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:35:41.261874+0000","last_peered":"2024-04-25T08:35:41.261874+0000","last_clean":"2024-04-25T08:35:41.261874+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:35:41.261874+0000","last_undegraded":"2024-04-25T08:35:41.261874+0000","last_fullsized":"2024-04-25T08:35:41.261874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46452,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325388,"statfs":{"total":96636764160,"available":96589197312,"internally_reserved":0,"allocated":1409024,"data_stored":954602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40288,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331552,"statfs":{"total":96636764160,"available":96595509248,"internally_reserved":0,"allocated":802816,"data_stored":363492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40284,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331556,"statfs":{"total":96636764160,"available":96595513344,"internally_reserved":0,"allocated":802816,"data_stored":363480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45860,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325980,"statfs":{"total":96636764160,"available":96589803520,"internally_reserved":0,"allocated":811008,"data_stored":365070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":363458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40288,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331552,"statfs":{"total":96636764160,"available":96595509248,"internally_reserved":0,"allocated":798720,"data_stored":363458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40308,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331532,"statfs":{"total":96636764160,"available":96595488768,"internally_reserved":0,"allocated":827392,"data_stored":365834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54756,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317084,"statfs":{"total":96636764160,"available":96580694016,"internally_reserved":0,"allocated":1392640,"data_stored":953826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:35:44.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:35:44.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.414+0000 7fc10e59e640 1 Processor -- start 2024-04-25T08:35:44.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.414+0000 7fc10e59e640 1 -- start start 2024-04-25T08:35:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.414+0000 7fc10e59e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc10815da90 0x7fc1080d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.414+0000 7fc10e59e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc1080d3fb0 0x7fc1080b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.414+0000 7fc10e59e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1080d1b60 con 0x7fc1080ceca0 2024-04-25T08:35:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.414+0000 7fc10e59e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1080d2bc0 con 0x7fc10815da90 2024-04-25T08:35:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.414+0000 7fc10e59e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1080d1e00 con 0x7fc1080d3fb0 2024-04-25T08:35:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.415+0000 7fc107fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc10815da90 0x7fc1080d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.415+0000 7fc107fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc10815da90 0x7fc1080d2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57106/0 (socket says 172.21.15.38:57106) 2024-04-25T08:35:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.415+0000 7fc107fff640 1 -- 172.21.15.38:0/1261721304 learned_addr learned my addr 172.21.15.38:0/1261721304 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.415+0000 7fc1077fe640 1 --1- 172.21.15.38:0/1261721304 >> v1:172.21.15.38:6789/0 conn(0x7fc1080ceca0 0x7fc1080d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:39890/0 (socket says 172.21.15.38:39890) 2024-04-25T08:35:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.415+0000 7fc10cb3b640 1 --2- 172.21.15.38:0/1261721304 >> v2:172.21.15.107:3300/0 conn(0x7fc1080d3fb0 0x7fc1080b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.415+0000 7fc107fff640 1 -- 172.21.15.38:0/1261721304 >> v2:172.21.15.107:3300/0 conn(0x7fc1080d3fb0 msgr2=0x7fc1080b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.415+0000 7fc107fff640 1 --2- 172.21.15.38:0/1261721304 >> v2:172.21.15.107:3300/0 conn(0x7fc1080d3fb0 0x7fc1080b69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.415+0000 7fc107fff640 1 -- 172.21.15.38:0/1261721304 >> v1:172.21.15.38:6789/0 conn(0x7fc1080ceca0 legacy=0x7fc1080d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.415+0000 7fc107fff640 1 -- 172.21.15.38:0/1261721304 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1080d20e0 con 0x7fc10815da90 2024-04-25T08:35:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.415+0000 7fc10cb3b640 1 --2- 172.21.15.38:0/1261721304 >> v2:172.21.15.107:3300/0 conn(0x7fc1080d3fb0 0x7fc1080b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.416+0000 7fc107fff640 1 --2- 172.21.15.38:0/1261721304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10815da90 0x7fc1080d2f70 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc1080c7410 tx=0x7fc0f802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777673cea582299f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.416+0000 7fc106ffd640 1 -- 172.21.15.38:0/1261721304 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8037070 con 0x7fc10815da90 2024-04-25T08:35:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.416+0000 7fc106ffd640 1 -- 172.21.15.38:0/1261721304 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f8032440 con 0x7fc10815da90 2024-04-25T08:35:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.416+0000 7fc106ffd640 1 -- 172.21.15.38:0/1261721304 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8032810 con 0x7fc10815da90 2024-04-25T08:35:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.416+0000 7fc10e59e640 1 -- 172.21.15.38:0/1261721304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10815da90 msgr2=0x7fc1080d2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.416+0000 7fc10e59e640 1 --2- 172.21.15.38:0/1261721304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10815da90 0x7fc1080d2f70 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fc1080c7410 tx=0x7fc0f802a270 comp rx=0 tx=0).stop 2024-04-25T08:35:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.416+0000 7fc10e59e640 1 -- 172.21.15.38:0/1261721304 shutdown_connections 2024-04-25T08:35:44.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.416+0000 7fc10e59e640 1 --2- 172.21.15.38:0/1261721304 >> v2:172.21.15.107:3300/0 conn(0x7fc1080d3fb0 0x7fc1080b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.416+0000 7fc10e59e640 1 --2- 172.21.15.38:0/1261721304 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc10815da90 0x7fc1080d2f70 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.416+0000 7fc10e59e640 1 -- 172.21.15.38:0/1261721304 >> 172.21.15.38:0/1261721304 conn(0x7fc10806bed0 msgr2=0x7fc10806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:44.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.416+0000 7fc10e59e640 1 -- 172.21.15.38:0/1261721304 shutdown_connections 2024-04-25T08:35:44.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.416+0000 7fc10e59e640 1 -- 172.21.15.38:0/1261721304 wait complete. 2024-04-25T08:35:44.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.417+0000 7fc10e59e640 1 Processor -- start 2024-04-25T08:35:44.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.418+0000 7fc10e59e640 1 -- start start 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.418+0000 7fc10e59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1080d3fb0 0x7fc1080cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.418+0000 7fc10e59e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc10815da90 0x7fc108146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.418+0000 7fc10e59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc108156cb0 0x7fc108147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.418+0000 7fc10e59e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10815dff0 con 0x7fc108156cb0 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.418+0000 7fc10e59e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10815de70 con 0x7fc1080d3fb0 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.418+0000 7fc10e59e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1080d34b0 con 0x7fc10815da90 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc10cb3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc108156cb0 0x7fc108147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc10cb3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc108156cb0 0x7fc108147040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57118/0 (socket says 172.21.15.38:57118) 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc10cb3b640 1 -- 172.21.15.38:0/2400025418 learned_addr learned my addr 172.21.15.38:0/2400025418 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc107fff640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1080d3fb0 0x7fc1080cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc1077fe640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc10815da90 0x7fc108146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc10cb3b640 1 -- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc10815da90 msgr2=0x7fc108146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc10cb3b640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc10815da90 0x7fc108146b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc10cb3b640 1 -- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1080d3fb0 msgr2=0x7fc1080cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc10cb3b640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1080d3fb0 0x7fc1080cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc10cb3b640 1 -- 172.21.15.38:0/2400025418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc108157150 con 0x7fc108156cb0 2024-04-25T08:35:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc10cb3b640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc108156cb0 0x7fc108147040 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc10000cef0 tx=0x7fc10000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc104ff9640 1 -- 172.21.15.38:0/2400025418 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10001a070 con 0x7fc108156cb0 2024-04-25T08:35:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc104ff9640 1 -- 172.21.15.38:0/2400025418 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10000d860 con 0x7fc108156cb0 2024-04-25T08:35:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc10e59e640 1 -- 172.21.15.38:0/2400025418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1080cba60 con 0x7fc108156cb0 2024-04-25T08:35:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc104ff9640 1 -- 172.21.15.38:0/2400025418 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10000dc50 con 0x7fc108156cb0 2024-04-25T08:35:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.419+0000 7fc10e59e640 1 -- 172.21.15.38:0/2400025418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc108157430 con 0x7fc108156cb0 2024-04-25T08:35:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.420+0000 7fc10e59e640 1 -- 172.21.15.38:0/2400025418 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc108068e80 con 0x7fc108156cb0 2024-04-25T08:35:44.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.421+0000 7fc104ff9640 1 -- 172.21.15.38:0/2400025418 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc10001d070 con 0x7fc108156cb0 2024-04-25T08:35:44.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.421+0000 7fc104ff9640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0f406b4d0 0x7fc0f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.421+0000 7fc104ff9640 1 -- 172.21.15.38:0/2400025418 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc10008ab90 con 0x7fc108156cb0 2024-04-25T08:35:44.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.421+0000 7fc107fff640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0f406b4d0 0x7fc0f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.422+0000 7fc107fff640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0f406b4d0 0x7fc0f406d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8034000 tx=0x7fc0f802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:44.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.425+0000 7fc104ff9640 1 -- 172.21.15.38:0/2400025418 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc100059300 con 0x7fc108156cb0 2024-04-25T08:35:44.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.602+0000 7fc10e59e640 1 -- 172.21.15.38:0/2400025418 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc1080b9130 con 0x7fc0f406b4d0 2024-04-25T08:35:44.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.608+0000 7fc104ff9640 1 -- 172.21.15.38:0/2400025418 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251949 (secure 0 0 0) 0x7fc1080b9130 con 0x7fc0f406b4d0 2024-04-25T08:35:44.605 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:35:44.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.611+0000 7fc10e59e640 1 -- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0f406b4d0 msgr2=0x7fc0f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.611+0000 7fc10e59e640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0f406b4d0 0x7fc0f406d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8034000 tx=0x7fc0f802e000 comp rx=0 tx=0).stop 2024-04-25T08:35:44.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.611+0000 7fc10e59e640 1 -- 172.21.15.38:0/2400025418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc108156cb0 msgr2=0x7fc108147040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.611+0000 7fc10e59e640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc108156cb0 0x7fc108147040 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc10000cef0 tx=0x7fc10000a710 comp rx=0 tx=0).stop 2024-04-25T08:35:44.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.611+0000 7fc10e59e640 1 -- 172.21.15.38:0/2400025418 shutdown_connections 2024-04-25T08:35:44.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.611+0000 7fc10e59e640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc108156cb0 0x7fc108147040 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.611+0000 7fc10e59e640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0f406b4d0 0x7fc0f406d990 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.611+0000 7fc10e59e640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc10815da90 0x7fc108146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.611+0000 7fc10e59e640 1 --2- 172.21.15.38:0/2400025418 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1080d3fb0 0x7fc1080cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.611+0000 7fc10e59e640 1 -- 172.21.15.38:0/2400025418 >> 172.21.15.38:0/2400025418 conn(0x7fc10806bed0 msgr2=0x7fc10806c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:44.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.611+0000 7fc10e59e640 1 -- 172.21.15.38:0/2400025418 shutdown_connections 2024-04-25T08:35:44.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.611+0000 7fc10e59e640 1 -- 172.21.15.38:0/2400025418 wait complete. 2024-04-25T08:35:44.617 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":191,"stamp":"2024-04-25T08:35:43.262449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":336036,"kb_used_data":7464,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754638684,"statfs":{"total":773094113280,"available":772750012416,"internally_reserved":0,"allocated":7643136,"data_stored":4093220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":336318000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020705"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290846+0000","last_change":"2024-04-25T08:35:41.287246+0000","last_active":"2024-04-25T08:35:41.290846+0000","last_peered":"2024-04-25T08:35:41.290846+0000","last_clean":"2024-04-25T08:35:41.290846+0000","last_became_active":"2024-04-25T08:35:41.286039+0000","last_became_peered":"2024-04-25T08:35:41.286039+0000","last_unstale":"2024-04-25T08:35:41.290846+0000","last_undegraded":"2024-04-25T08:35:41.290846+0000","last_fullsized":"2024-04-25T08:35:41.290846+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:35.472816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290768+0000","last_change":"2024-04-25T08:35:41.285554+0000","last_active":"2024-04-25T08:35:41.290768+0000","last_peered":"2024-04-25T08:35:41.290768+0000","last_clean":"2024-04-25T08:35:41.290768+0000","last_became_active":"2024-04-25T08:35:41.285249+0000","last_became_peered":"2024-04-25T08:35:41.285249+0000","last_unstale":"2024-04-25T08:35:41.290768+0000","last_undegraded":"2024-04-25T08:35:41.290768+0000","last_fullsized":"2024-04-25T08:35:41.290768+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:01.626304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.277813+0000","last_change":"2024-04-25T08:35:41.272722+0000","last_active":"2024-04-25T08:35:41.277813+0000","last_peered":"2024-04-25T08:35:41.277813+0000","last_clean":"2024-04-25T08:35:41.277813+0000","last_became_active":"2024-04-25T08:35:41.272446+0000","last_became_peered":"2024-04-25T08:35:41.272446+0000","last_unstale":"2024-04-25T08:35:41.277813+0000","last_undegraded":"2024-04-25T08:35:41.277813+0000","last_fullsized":"2024-04-25T08:35:41.277813+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:19.507911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.282261+0000","last_change":"2024-04-25T08:35:41.279637+0000","last_active":"2024-04-25T08:35:41.282261+0000","last_peered":"2024-04-25T08:35:41.282261+0000","last_clean":"2024-04-25T08:35:41.282261+0000","last_became_active":"2024-04-25T08:35:41.279345+0000","last_became_peered":"2024-04-25T08:35:41.279345+0000","last_unstale":"2024-04-25T08:35:41.282261+0000","last_undegraded":"2024-04-25T08:35:41.282261+0000","last_fullsized":"2024-04-25T08:35:41.282261+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.714069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.269400+0000","last_change":"2024-04-25T08:35:41.264763+0000","last_active":"2024-04-25T08:35:41.269400+0000","last_peered":"2024-04-25T08:35:41.269400+0000","last_clean":"2024-04-25T08:35:41.269400+0000","last_became_active":"2024-04-25T08:35:41.264468+0000","last_became_peered":"2024-04-25T08:35:41.264468+0000","last_unstale":"2024-04-25T08:35:41.269400+0000","last_undegraded":"2024-04-25T08:35:41.269400+0000","last_fullsized":"2024-04-25T08:35:41.269400+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.714069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.280665+0000","last_change":"2024-04-25T08:35:41.275921+0000","last_active":"2024-04-25T08:35:41.280665+0000","last_peered":"2024-04-25T08:35:41.280665+0000","last_clean":"2024-04-25T08:35:41.280665+0000","last_became_active":"2024-04-25T08:35:41.275516+0000","last_became_peered":"2024-04-25T08:35:41.275516+0000","last_unstale":"2024-04-25T08:35:41.280665+0000","last_undegraded":"2024-04-25T08:35:41.280665+0000","last_fullsized":"2024-04-25T08:35:41.280665+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:03.266564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.284011+0000","last_change":"2024-04-25T08:35:41.280722+0000","last_active":"2024-04-25T08:35:41.284011+0000","last_peered":"2024-04-25T08:35:41.284011+0000","last_clean":"2024-04-25T08:35:41.284011+0000","last_became_active":"2024-04-25T08:35:41.279759+0000","last_became_peered":"2024-04-25T08:35:41.279759+0000","last_unstale":"2024-04-25T08:35:41.284011+0000","last_undegraded":"2024-04-25T08:35:41.284011+0000","last_fullsized":"2024-04-25T08:35:41.284011+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:09.237958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.277805+0000","last_change":"2024-04-25T08:35:41.271914+0000","last_active":"2024-04-25T08:35:41.277805+0000","last_peered":"2024-04-25T08:35:41.277805+0000","last_clean":"2024-04-25T08:35:41.277805+0000","last_became_active":"2024-04-25T08:35:41.270062+0000","last_became_peered":"2024-04-25T08:35:41.270062+0000","last_unstale":"2024-04-25T08:35:41.277805+0000","last_undegraded":"2024-04-25T08:35:41.277805+0000","last_fullsized":"2024-04-25T08:35:41.277805+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:30.604184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.267157+0000","last_change":"2024-04-25T08:35:41.265682+0000","last_active":"2024-04-25T08:35:41.267157+0000","last_peered":"2024-04-25T08:35:41.267157+0000","last_clean":"2024-04-25T08:35:41.267157+0000","last_became_active":"2024-04-25T08:35:41.265394+0000","last_became_peered":"2024-04-25T08:35:41.265394+0000","last_unstale":"2024-04-25T08:35:41.267157+0000","last_undegraded":"2024-04-25T08:35:41.267157+0000","last_fullsized":"2024-04-25T08:35:41.267157+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:44.396919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287970+0000","last_change":"2024-04-25T08:35:41.286813+0000","last_active":"2024-04-25T08:35:41.287970+0000","last_peered":"2024-04-25T08:35:41.287970+0000","last_clean":"2024-04-25T08:35:41.287970+0000","last_became_active":"2024-04-25T08:35:41.286439+0000","last_became_peered":"2024-04-25T08:35:41.286439+0000","last_unstale":"2024-04-25T08:35:41.287970+0000","last_undegraded":"2024-04-25T08:35:41.287970+0000","last_fullsized":"2024-04-25T08:35:41.287970+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:41.362181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.288217+0000","last_change":"2024-04-25T08:35:41.287559+0000","last_active":"2024-04-25T08:35:41.288217+0000","last_peered":"2024-04-25T08:35:41.288217+0000","last_clean":"2024-04-25T08:35:41.288217+0000","last_became_active":"2024-04-25T08:35:41.287267+0000","last_became_peered":"2024-04-25T08:35:41.287267+0000","last_unstale":"2024-04-25T08:35:41.288217+0000","last_undegraded":"2024-04-25T08:35:41.288217+0000","last_fullsized":"2024-04-25T08:35:41.288217+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:23.681487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.272019+0000","last_change":"2024-04-25T08:35:41.270585+0000","last_active":"2024-04-25T08:35:41.272019+0000","last_peered":"2024-04-25T08:35:41.272019+0000","last_clean":"2024-04-25T08:35:41.272019+0000","last_became_active":"2024-04-25T08:35:41.270041+0000","last_became_peered":"2024-04-25T08:35:41.270041+0000","last_unstale":"2024-04-25T08:35:41.272019+0000","last_undegraded":"2024-04-25T08:35:41.272019+0000","last_fullsized":"2024-04-25T08:35:41.272019+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:25.514354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.277506+0000","last_change":"2024-04-25T08:35:41.275625+0000","last_active":"2024-04-25T08:35:41.277506+0000","last_peered":"2024-04-25T08:35:41.277506+0000","last_clean":"2024-04-25T08:35:41.277506+0000","last_became_active":"2024-04-25T08:35:41.275357+0000","last_became_peered":"2024-04-25T08:35:41.275357+0000","last_unstale":"2024-04-25T08:35:41.277506+0000","last_undegraded":"2024-04-25T08:35:41.277506+0000","last_fullsized":"2024-04-25T08:35:41.277506+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:40.578929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.282988+0000","last_change":"2024-04-25T08:35:41.282228+0000","last_active":"2024-04-25T08:35:41.282988+0000","last_peered":"2024-04-25T08:35:41.282988+0000","last_clean":"2024-04-25T08:35:41.282988+0000","last_became_active":"2024-04-25T08:35:41.281639+0000","last_became_peered":"2024-04-25T08:35:41.281639+0000","last_unstale":"2024-04-25T08:35:41.282988+0000","last_undegraded":"2024-04-25T08:35:41.282988+0000","last_fullsized":"2024-04-25T08:35:41.282988+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:16.032826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287972+0000","last_change":"2024-04-25T08:35:41.286523+0000","last_active":"2024-04-25T08:35:41.287972+0000","last_peered":"2024-04-25T08:35:41.287972+0000","last_clean":"2024-04-25T08:35:41.287972+0000","last_became_active":"2024-04-25T08:35:41.286261+0000","last_became_peered":"2024-04-25T08:35:41.286261+0000","last_unstale":"2024-04-25T08:35:41.287972+0000","last_undegraded":"2024-04-25T08:35:41.287972+0000","last_fullsized":"2024-04-25T08:35:41.287972+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:36.353555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.284067+0000","last_change":"2024-04-25T08:35:41.277647+0000","last_active":"2024-04-25T08:35:41.284067+0000","last_peered":"2024-04-25T08:35:41.284067+0000","last_clean":"2024-04-25T08:35:41.284067+0000","last_became_active":"2024-04-25T08:35:41.277418+0000","last_became_peered":"2024-04-25T08:35:41.277418+0000","last_unstale":"2024-04-25T08:35:41.284067+0000","last_undegraded":"2024-04-25T08:35:41.284067+0000","last_fullsized":"2024-04-25T08:35:41.284067+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:52.868767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290796+0000","last_change":"2024-04-25T08:35:41.285419+0000","last_active":"2024-04-25T08:35:41.290796+0000","last_peered":"2024-04-25T08:35:41.290796+0000","last_clean":"2024-04-25T08:35:41.290796+0000","last_became_active":"2024-04-25T08:35:41.285111+0000","last_became_peered":"2024-04-25T08:35:41.285111+0000","last_unstale":"2024-04-25T08:35:41.290796+0000","last_undegraded":"2024-04-25T08:35:41.290796+0000","last_fullsized":"2024-04-25T08:35:41.290796+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:57.886488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.270603+0000","last_change":"2024-04-25T08:35:41.268261+0000","last_active":"2024-04-25T08:35:41.270603+0000","last_peered":"2024-04-25T08:35:41.270603+0000","last_clean":"2024-04-25T08:35:41.270603+0000","last_became_active":"2024-04-25T08:35:41.267919+0000","last_became_peered":"2024-04-25T08:35:41.267919+0000","last_unstale":"2024-04-25T08:35:41.270603+0000","last_undegraded":"2024-04-25T08:35:41.270603+0000","last_fullsized":"2024-04-25T08:35:41.270603+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:01.633527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.278635+0000","last_change":"2024-04-25T08:35:41.277311+0000","last_active":"2024-04-25T08:35:41.278635+0000","last_peered":"2024-04-25T08:35:41.278635+0000","last_clean":"2024-04-25T08:35:41.278635+0000","last_became_active":"2024-04-25T08:35:41.277016+0000","last_became_peered":"2024-04-25T08:35:41.277016+0000","last_unstale":"2024-04-25T08:35:41.278635+0000","last_undegraded":"2024-04-25T08:35:41.278635+0000","last_fullsized":"2024-04-25T08:35:41.278635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:57.082004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.275196+0000","last_change":"2024-04-25T08:35:41.274477+0000","last_active":"2024-04-25T08:35:41.275196+0000","last_peered":"2024-04-25T08:35:41.275196+0000","last_clean":"2024-04-25T08:35:41.275196+0000","last_became_active":"2024-04-25T08:35:41.274185+0000","last_became_peered":"2024-04-25T08:35:41.274185+0000","last_unstale":"2024-04-25T08:35:41.275196+0000","last_undegraded":"2024-04-25T08:35:41.275196+0000","last_fullsized":"2024-04-25T08:35:41.275196+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:18.949036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.288283+0000","last_change":"2024-04-25T08:35:41.287655+0000","last_active":"2024-04-25T08:35:41.288283+0000","last_peered":"2024-04-25T08:35:41.288283+0000","last_clean":"2024-04-25T08:35:41.288283+0000","last_became_active":"2024-04-25T08:35:41.287367+0000","last_became_peered":"2024-04-25T08:35:41.287367+0000","last_unstale":"2024-04-25T08:35:41.288283+0000","last_undegraded":"2024-04-25T08:35:41.288283+0000","last_fullsized":"2024-04-25T08:35:41.288283+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:14.998969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.282257+0000","last_change":"2024-04-25T08:35:41.281047+0000","last_active":"2024-04-25T08:35:41.282257+0000","last_peered":"2024-04-25T08:35:41.282257+0000","last_clean":"2024-04-25T08:35:41.282257+0000","last_became_active":"2024-04-25T08:35:41.280762+0000","last_became_peered":"2024-04-25T08:35:41.280762+0000","last_unstale":"2024-04-25T08:35:41.282257+0000","last_undegraded":"2024-04-25T08:35:41.282257+0000","last_fullsized":"2024-04-25T08:35:41.282257+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:03.266564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.282249+0000","last_change":"2024-04-25T08:35:41.280433+0000","last_active":"2024-04-25T08:35:41.282249+0000","last_peered":"2024-04-25T08:35:41.282249+0000","last_clean":"2024-04-25T08:35:41.282249+0000","last_became_active":"2024-04-25T08:35:41.280122+0000","last_became_peered":"2024-04-25T08:35:41.280122+0000","last_unstale":"2024-04-25T08:35:41.282249+0000","last_undegraded":"2024-04-25T08:35:41.282249+0000","last_fullsized":"2024-04-25T08:35:41.282249+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:08.153383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.278869+0000","last_change":"2024-04-25T08:35:41.275896+0000","last_active":"2024-04-25T08:35:41.278869+0000","last_peered":"2024-04-25T08:35:41.278869+0000","last_clean":"2024-04-25T08:35:41.278869+0000","last_became_active":"2024-04-25T08:35:41.275630+0000","last_became_peered":"2024-04-25T08:35:41.275630+0000","last_unstale":"2024-04-25T08:35:41.278869+0000","last_undegraded":"2024-04-25T08:35:41.278869+0000","last_fullsized":"2024-04-25T08:35:41.278869+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:00.186751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.285223+0000","last_change":"2024-04-25T08:35:41.284694+0000","last_active":"2024-04-25T08:35:41.285223+0000","last_peered":"2024-04-25T08:35:41.285223+0000","last_clean":"2024-04-25T08:35:41.285223+0000","last_became_active":"2024-04-25T08:35:41.284413+0000","last_became_peered":"2024-04-25T08:35:41.284413+0000","last_unstale":"2024-04-25T08:35:41.285223+0000","last_undegraded":"2024-04-25T08:35:41.285223+0000","last_fullsized":"2024-04-25T08:35:41.285223+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:44.396919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.285421+0000","last_change":"2024-04-25T08:35:41.284766+0000","last_active":"2024-04-25T08:35:41.285421+0000","last_peered":"2024-04-25T08:35:41.285421+0000","last_clean":"2024-04-25T08:35:41.285421+0000","last_became_active":"2024-04-25T08:35:41.284441+0000","last_became_peered":"2024-04-25T08:35:41.284441+0000","last_unstale":"2024-04-25T08:35:41.285421+0000","last_undegraded":"2024-04-25T08:35:41.285421+0000","last_fullsized":"2024-04-25T08:35:41.285421+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:16.741664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.283387+0000","last_change":"2024-04-25T08:35:41.282841+0000","last_active":"2024-04-25T08:35:41.283387+0000","last_peered":"2024-04-25T08:35:41.283387+0000","last_clean":"2024-04-25T08:35:41.283387+0000","last_became_active":"2024-04-25T08:35:41.282569+0000","last_became_peered":"2024-04-25T08:35:41.282569+0000","last_unstale":"2024-04-25T08:35:41.283387+0000","last_undegraded":"2024-04-25T08:35:41.283387+0000","last_fullsized":"2024-04-25T08:35:41.283387+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:15.313710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290896+0000","last_change":"2024-04-25T08:35:41.287424+0000","last_active":"2024-04-25T08:35:41.290896+0000","last_peered":"2024-04-25T08:35:41.290896+0000","last_clean":"2024-04-25T08:35:41.290896+0000","last_became_active":"2024-04-25T08:35:41.286068+0000","last_became_peered":"2024-04-25T08:35:41.286068+0000","last_unstale":"2024-04-25T08:35:41.290896+0000","last_undegraded":"2024-04-25T08:35:41.290896+0000","last_fullsized":"2024-04-25T08:35:41.290896+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:04.329215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.269651+0000","last_change":"2024-04-25T08:35:41.266298+0000","last_active":"2024-04-25T08:35:41.269651+0000","last_peered":"2024-04-25T08:35:41.269651+0000","last_clean":"2024-04-25T08:35:41.269651+0000","last_became_active":"2024-04-25T08:35:41.266026+0000","last_became_peered":"2024-04-25T08:35:41.266026+0000","last_unstale":"2024-04-25T08:35:41.269651+0000","last_undegraded":"2024-04-25T08:35:41.269651+0000","last_fullsized":"2024-04-25T08:35:41.269651+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:14.998969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.281394+0000","last_change":"2024-04-25T08:35:41.277529+0000","last_active":"2024-04-25T08:35:41.281394+0000","last_peered":"2024-04-25T08:35:41.281394+0000","last_clean":"2024-04-25T08:35:41.281394+0000","last_became_active":"2024-04-25T08:35:41.274806+0000","last_became_peered":"2024-04-25T08:35:41.274806+0000","last_unstale":"2024-04-25T08:35:41.281394+0000","last_undegraded":"2024-04-25T08:35:41.281394+0000","last_fullsized":"2024-04-25T08:35:41.281394+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:42.263995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.284768+0000","last_change":"2024-04-25T08:35:41.283352+0000","last_active":"2024-04-25T08:35:41.284768+0000","last_peered":"2024-04-25T08:35:41.284768+0000","last_clean":"2024-04-25T08:35:41.284768+0000","last_became_active":"2024-04-25T08:35:41.283063+0000","last_became_peered":"2024-04-25T08:35:41.283063+0000","last_unstale":"2024-04-25T08:35:41.284768+0000","last_undegraded":"2024-04-25T08:35:41.284768+0000","last_fullsized":"2024-04-25T08:35:41.284768+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:15.347371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.286740+0000","last_change":"2024-04-25T08:35:41.283499+0000","last_active":"2024-04-25T08:35:41.286740+0000","last_peered":"2024-04-25T08:35:41.286740+0000","last_clean":"2024-04-25T08:35:41.286740+0000","last_became_active":"2024-04-25T08:35:41.283197+0000","last_became_peered":"2024-04-25T08:35:41.283197+0000","last_unstale":"2024-04-25T08:35:41.286740+0000","last_undegraded":"2024-04-25T08:35:41.286740+0000","last_fullsized":"2024-04-25T08:35:41.286740+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:27.359399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290741+0000","last_change":"2024-04-25T08:35:41.290386+0000","last_active":"2024-04-25T08:35:41.290741+0000","last_peered":"2024-04-25T08:35:41.290741+0000","last_clean":"2024-04-25T08:35:41.290741+0000","last_became_active":"2024-04-25T08:35:41.290048+0000","last_became_peered":"2024-04-25T08:35:41.290048+0000","last_unstale":"2024-04-25T08:35:41.290741+0000","last_undegraded":"2024-04-25T08:35:41.290741+0000","last_fullsized":"2024-04-25T08:35:41.290741+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:37.876835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":16,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287609+0000","last_change":"2024-04-25T08:35:40.265386+0000","last_active":"2024-04-25T08:35:41.287609+0000","last_peered":"2024-04-25T08:35:41.287609+0000","last_clean":"2024-04-25T08:35:41.287609+0000","last_became_active":"2024-04-25T08:35:40.264762+0000","last_became_peered":"2024-04-25T08:35:40.264762+0000","last_unstale":"2024-04-25T08:35:41.287609+0000","last_undegraded":"2024-04-25T08:35:41.287609+0000","last_fullsized":"2024-04-25T08:35:41.287609+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:36.874371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.264187+0000","last_change":"2024-04-25T08:35:41.263680+0000","last_active":"2024-04-25T08:35:41.264187+0000","last_peered":"2024-04-25T08:35:41.264187+0000","last_clean":"2024-04-25T08:35:41.264187+0000","last_became_active":"2024-04-25T08:35:41.263370+0000","last_became_peered":"2024-04-25T08:35:41.263370+0000","last_unstale":"2024-04-25T08:35:41.264187+0000","last_undegraded":"2024-04-25T08:35:41.264187+0000","last_fullsized":"2024-04-25T08:35:41.264187+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:55.300955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.280970+0000","last_change":"2024-04-25T08:35:40.260869+0000","last_active":"2024-04-25T08:35:41.280970+0000","last_peered":"2024-04-25T08:35:41.280970+0000","last_clean":"2024-04-25T08:35:41.280970+0000","last_became_active":"2024-04-25T08:35:40.260546+0000","last_became_peered":"2024-04-25T08:35:40.260546+0000","last_unstale":"2024-04-25T08:35:41.280970+0000","last_undegraded":"2024-04-25T08:35:41.280970+0000","last_fullsized":"2024-04-25T08:35:41.280970+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:40.874197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.286291+0000","last_change":"2024-04-25T08:35:41.284290+0000","last_active":"2024-04-25T08:35:41.286291+0000","last_peered":"2024-04-25T08:35:41.286291+0000","last_clean":"2024-04-25T08:35:41.286291+0000","last_became_active":"2024-04-25T08:35:41.283940+0000","last_became_peered":"2024-04-25T08:35:41.283940+0000","last_unstale":"2024-04-25T08:35:41.286291+0000","last_undegraded":"2024-04-25T08:35:41.286291+0000","last_fullsized":"2024-04-25T08:35:41.286291+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:40.595051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.276051+0000","last_change":"2024-04-25T08:35:40.267779+0000","last_active":"2024-04-25T08:35:41.276051+0000","last_peered":"2024-04-25T08:35:41.276051+0000","last_clean":"2024-04-25T08:35:41.276051+0000","last_became_active":"2024-04-25T08:35:40.267476+0000","last_became_peered":"2024-04-25T08:35:40.267476+0000","last_unstale":"2024-04-25T08:35:41.276051+0000","last_undegraded":"2024-04-25T08:35:41.276051+0000","last_fullsized":"2024-04-25T08:35:41.276051+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:00.296484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287977+0000","last_change":"2024-04-25T08:35:41.287151+0000","last_active":"2024-04-25T08:35:41.287977+0000","last_peered":"2024-04-25T08:35:41.287977+0000","last_clean":"2024-04-25T08:35:41.287977+0000","last_became_active":"2024-04-25T08:35:41.286838+0000","last_became_peered":"2024-04-25T08:35:41.286838+0000","last_unstale":"2024-04-25T08:35:41.287977+0000","last_undegraded":"2024-04-25T08:35:41.287977+0000","last_fullsized":"2024-04-25T08:35:41.287977+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:03.658493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.289494+0000","last_change":"2024-04-25T08:35:40.265561+0000","last_active":"2024-04-25T08:35:41.289494+0000","last_peered":"2024-04-25T08:35:41.289494+0000","last_clean":"2024-04-25T08:35:41.289494+0000","last_became_active":"2024-04-25T08:35:40.265285+0000","last_became_peered":"2024-04-25T08:35:40.265285+0000","last_unstale":"2024-04-25T08:35:41.289494+0000","last_undegraded":"2024-04-25T08:35:41.289494+0000","last_fullsized":"2024-04-25T08:35:41.289494+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:07.150919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287952+0000","last_change":"2024-04-25T08:35:41.287012+0000","last_active":"2024-04-25T08:35:41.287952+0000","last_peered":"2024-04-25T08:35:41.287952+0000","last_clean":"2024-04-25T08:35:41.287952+0000","last_became_active":"2024-04-25T08:35:41.286710+0000","last_became_peered":"2024-04-25T08:35:41.286710+0000","last_unstale":"2024-04-25T08:35:41.287952+0000","last_undegraded":"2024-04-25T08:35:41.287952+0000","last_fullsized":"2024-04-25T08:35:41.287952+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:15.971000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.281452+0000","last_change":"2024-04-25T08:35:40.261660+0000","last_active":"2024-04-25T08:35:41.281452+0000","last_peered":"2024-04-25T08:35:41.281452+0000","last_clean":"2024-04-25T08:35:41.281452+0000","last_became_active":"2024-04-25T08:35:40.261355+0000","last_became_peered":"2024-04-25T08:35:40.261355+0000","last_unstale":"2024-04-25T08:35:41.281452+0000","last_undegraded":"2024-04-25T08:35:41.281452+0000","last_fullsized":"2024-04-25T08:35:41.281452+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:40.359717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.288773+0000","last_change":"2024-04-25T08:35:41.281657+0000","last_active":"2024-04-25T08:35:41.288773+0000","last_peered":"2024-04-25T08:35:41.288773+0000","last_clean":"2024-04-25T08:35:41.288773+0000","last_became_active":"2024-04-25T08:35:41.280363+0000","last_became_peered":"2024-04-25T08:35:41.280363+0000","last_unstale":"2024-04-25T08:35:41.288773+0000","last_undegraded":"2024-04-25T08:35:41.288773+0000","last_fullsized":"2024-04-25T08:35:41.288773+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:30.344111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.284232+0000","last_change":"2024-04-25T08:35:40.260460+0000","last_active":"2024-04-25T08:35:41.284232+0000","last_peered":"2024-04-25T08:35:41.284232+0000","last_clean":"2024-04-25T08:35:41.284232+0000","last_became_active":"2024-04-25T08:35:40.260157+0000","last_became_peered":"2024-04-25T08:35:40.260157+0000","last_unstale":"2024-04-25T08:35:41.284232+0000","last_undegraded":"2024-04-25T08:35:41.284232+0000","last_fullsized":"2024-04-25T08:35:41.284232+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:47.688555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.279024+0000","last_change":"2024-04-25T08:35:41.278483+0000","last_active":"2024-04-25T08:35:41.279024+0000","last_peered":"2024-04-25T08:35:41.279024+0000","last_clean":"2024-04-25T08:35:41.279024+0000","last_became_active":"2024-04-25T08:35:41.278210+0000","last_became_peered":"2024-04-25T08:35:41.278210+0000","last_unstale":"2024-04-25T08:35:41.279024+0000","last_undegraded":"2024-04-25T08:35:41.279024+0000","last_fullsized":"2024-04-25T08:35:41.279024+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:27.862067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.268154+0000","last_change":"2024-04-25T08:35:40.264492+0000","last_active":"2024-04-25T08:35:41.268154+0000","last_peered":"2024-04-25T08:35:41.268154+0000","last_clean":"2024-04-25T08:35:41.268154+0000","last_became_active":"2024-04-25T08:35:40.264093+0000","last_became_peered":"2024-04-25T08:35:40.264093+0000","last_unstale":"2024-04-25T08:35:41.268154+0000","last_undegraded":"2024-04-25T08:35:41.268154+0000","last_fullsized":"2024-04-25T08:35:41.268154+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:02.656029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.274420+0000","last_change":"2024-04-25T08:35:41.268358+0000","last_active":"2024-04-25T08:35:41.274420+0000","last_peered":"2024-04-25T08:35:41.274420+0000","last_clean":"2024-04-25T08:35:41.274420+0000","last_became_active":"2024-04-25T08:35:41.267757+0000","last_became_peered":"2024-04-25T08:35:41.267757+0000","last_unstale":"2024-04-25T08:35:41.274420+0000","last_undegraded":"2024-04-25T08:35:41.274420+0000","last_fullsized":"2024-04-25T08:35:41.274420+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:24.182066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.258304+0000","last_change":"2024-04-25T08:35:40.260833+0000","last_active":"2024-04-25T08:35:41.258304+0000","last_peered":"2024-04-25T08:35:41.258304+0000","last_clean":"2024-04-25T08:35:41.258304+0000","last_became_active":"2024-04-25T08:35:40.260521+0000","last_became_peered":"2024-04-25T08:35:40.260521+0000","last_unstale":"2024-04-25T08:35:41.258304+0000","last_undegraded":"2024-04-25T08:35:41.258304+0000","last_fullsized":"2024-04-25T08:35:41.258304+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:38.576626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.289090+0000","last_change":"2024-04-25T08:35:41.284386+0000","last_active":"2024-04-25T08:35:41.289090+0000","last_peered":"2024-04-25T08:35:41.289090+0000","last_clean":"2024-04-25T08:35:41.289090+0000","last_became_active":"2024-04-25T08:35:41.284082+0000","last_became_peered":"2024-04-25T08:35:41.284082+0000","last_unstale":"2024-04-25T08:35:41.289090+0000","last_undegraded":"2024-04-25T08:35:41.289090+0000","last_fullsized":"2024-04-25T08:35:41.289090+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:57.930974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.260604+0000","last_change":"2024-04-25T08:35:40.267540+0000","last_active":"2024-04-25T08:35:41.260604+0000","last_peered":"2024-04-25T08:35:41.260604+0000","last_clean":"2024-04-25T08:35:41.260604+0000","last_became_active":"2024-04-25T08:35:40.267246+0000","last_became_peered":"2024-04-25T08:35:40.267246+0000","last_unstale":"2024-04-25T08:35:41.260604+0000","last_undegraded":"2024-04-25T08:35:41.260604+0000","last_fullsized":"2024-04-25T08:35:41.260604+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:41.261531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.274824+0000","last_change":"2024-04-25T08:35:41.272072+0000","last_active":"2024-04-25T08:35:41.274824+0000","last_peered":"2024-04-25T08:35:41.274824+0000","last_clean":"2024-04-25T08:35:41.274824+0000","last_became_active":"2024-04-25T08:35:41.271732+0000","last_became_peered":"2024-04-25T08:35:41.271732+0000","last_unstale":"2024-04-25T08:35:41.274824+0000","last_undegraded":"2024-04-25T08:35:41.274824+0000","last_fullsized":"2024-04-25T08:35:41.274824+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:41.876660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.258119+0000","last_change":"2024-04-25T08:35:40.267085+0000","last_active":"2024-04-25T08:35:41.258119+0000","last_peered":"2024-04-25T08:35:41.258119+0000","last_clean":"2024-04-25T08:35:41.258119+0000","last_became_active":"2024-04-25T08:35:40.266626+0000","last_became_peered":"2024-04-25T08:35:40.266626+0000","last_unstale":"2024-04-25T08:35:41.258119+0000","last_undegraded":"2024-04-25T08:35:41.258119+0000","last_fullsized":"2024-04-25T08:35:41.258119+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:28.384828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.274432+0000","last_change":"2024-04-25T08:35:41.273775+0000","last_active":"2024-04-25T08:35:41.274432+0000","last_peered":"2024-04-25T08:35:41.274432+0000","last_clean":"2024-04-25T08:35:41.274432+0000","last_became_active":"2024-04-25T08:35:41.273480+0000","last_became_peered":"2024-04-25T08:35:41.273480+0000","last_unstale":"2024-04-25T08:35:41.274432+0000","last_undegraded":"2024-04-25T08:35:41.274432+0000","last_fullsized":"2024-04-25T08:35:41.274432+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:23.681487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":15,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.289906+0000","last_change":"2024-04-25T08:35:40.265428+0000","last_active":"2024-04-25T08:35:41.289906+0000","last_peered":"2024-04-25T08:35:41.289906+0000","last_clean":"2024-04-25T08:35:41.289906+0000","last_became_active":"2024-04-25T08:35:40.265126+0000","last_became_peered":"2024-04-25T08:35:40.265126+0000","last_unstale":"2024-04-25T08:35:41.289906+0000","last_undegraded":"2024-04-25T08:35:41.289906+0000","last_fullsized":"2024-04-25T08:35:41.289906+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:39.576465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.267446+0000","last_change":"2024-04-25T08:35:41.266887+0000","last_active":"2024-04-25T08:35:41.267446+0000","last_peered":"2024-04-25T08:35:41.267446+0000","last_clean":"2024-04-25T08:35:41.267446+0000","last_became_active":"2024-04-25T08:35:41.266567+0000","last_became_peered":"2024-04-25T08:35:41.266567+0000","last_unstale":"2024-04-25T08:35:41.267446+0000","last_undegraded":"2024-04-25T08:35:41.267446+0000","last_fullsized":"2024-04-25T08:35:41.267446+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:39.579089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":18,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.291264+0000","last_change":"2024-04-25T08:35:40.263573+0000","last_active":"2024-04-25T08:35:41.291264+0000","last_peered":"2024-04-25T08:35:41.291264+0000","last_clean":"2024-04-25T08:35:41.291264+0000","last_became_active":"2024-04-25T08:35:40.263116+0000","last_became_peered":"2024-04-25T08:35:40.263116+0000","last_unstale":"2024-04-25T08:35:41.291264+0000","last_undegraded":"2024-04-25T08:35:41.291264+0000","last_fullsized":"2024-04-25T08:35:41.291264+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:01.463669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.266709+0000","last_change":"2024-04-25T08:35:41.266135+0000","last_active":"2024-04-25T08:35:41.266709+0000","last_peered":"2024-04-25T08:35:41.266709+0000","last_clean":"2024-04-25T08:35:41.266709+0000","last_became_active":"2024-04-25T08:35:41.264700+0000","last_became_peered":"2024-04-25T08:35:41.264700+0000","last_unstale":"2024-04-25T08:35:41.266709+0000","last_undegraded":"2024-04-25T08:35:41.266709+0000","last_fullsized":"2024-04-25T08:35:41.266709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:14.635814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.277885+0000","last_change":"2024-04-25T08:35:40.263814+0000","last_active":"2024-04-25T08:35:41.277885+0000","last_peered":"2024-04-25T08:35:41.277885+0000","last_clean":"2024-04-25T08:35:41.277885+0000","last_became_active":"2024-04-25T08:35:40.261759+0000","last_became_peered":"2024-04-25T08:35:40.261759+0000","last_unstale":"2024-04-25T08:35:41.277885+0000","last_undegraded":"2024-04-25T08:35:41.277885+0000","last_fullsized":"2024-04-25T08:35:41.277885+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:24.511890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.277477+0000","last_change":"2024-04-25T08:35:41.272440+0000","last_active":"2024-04-25T08:35:41.277477+0000","last_peered":"2024-04-25T08:35:41.277477+0000","last_clean":"2024-04-25T08:35:41.277477+0000","last_became_active":"2024-04-25T08:35:41.272175+0000","last_became_peered":"2024-04-25T08:35:41.272175+0000","last_unstale":"2024-04-25T08:35:41.277477+0000","last_undegraded":"2024-04-25T08:35:41.277477+0000","last_fullsized":"2024-04-25T08:35:41.277477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:07.864540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.269597+0000","last_change":"2024-04-25T08:35:40.268329+0000","last_active":"2024-04-25T08:35:41.269597+0000","last_peered":"2024-04-25T08:35:41.269597+0000","last_clean":"2024-04-25T08:35:41.269597+0000","last_became_active":"2024-04-25T08:35:40.267977+0000","last_became_peered":"2024-04-25T08:35:40.267977+0000","last_unstale":"2024-04-25T08:35:41.269597+0000","last_undegraded":"2024-04-25T08:35:41.269597+0000","last_fullsized":"2024-04-25T08:35:41.269597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:03.326751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.278994+0000","last_change":"2024-04-25T08:35:41.277480+0000","last_active":"2024-04-25T08:35:41.278994+0000","last_peered":"2024-04-25T08:35:41.278994+0000","last_clean":"2024-04-25T08:35:41.278994+0000","last_became_active":"2024-04-25T08:35:41.277195+0000","last_became_peered":"2024-04-25T08:35:41.277195+0000","last_unstale":"2024-04-25T08:35:41.278994+0000","last_undegraded":"2024-04-25T08:35:41.278994+0000","last_fullsized":"2024-04-25T08:35:41.278994+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:21.411829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.260680+0000","last_change":"2024-04-25T08:35:40.260776+0000","last_active":"2024-04-25T08:35:41.260680+0000","last_peered":"2024-04-25T08:35:41.260680+0000","last_clean":"2024-04-25T08:35:41.260680+0000","last_became_active":"2024-04-25T08:35:40.260496+0000","last_became_peered":"2024-04-25T08:35:40.260496+0000","last_unstale":"2024-04-25T08:35:41.260680+0000","last_undegraded":"2024-04-25T08:35:41.260680+0000","last_fullsized":"2024-04-25T08:35:41.260680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:38.919175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290749+0000","last_change":"2024-04-25T08:35:41.285449+0000","last_active":"2024-04-25T08:35:41.290749+0000","last_peered":"2024-04-25T08:35:41.290749+0000","last_clean":"2024-04-25T08:35:41.290749+0000","last_became_active":"2024-04-25T08:35:41.285145+0000","last_became_peered":"2024-04-25T08:35:41.285145+0000","last_unstale":"2024-04-25T08:35:41.290749+0000","last_undegraded":"2024-04-25T08:35:41.290749+0000","last_fullsized":"2024-04-25T08:35:41.290749+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:59.698763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":20,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.285711+0000","last_change":"2024-04-25T08:35:40.258994+0000","last_active":"2024-04-25T08:35:41.285711+0000","last_peered":"2024-04-25T08:35:41.285711+0000","last_clean":"2024-04-25T08:35:41.285711+0000","last_became_active":"2024-04-25T08:35:40.258685+0000","last_became_peered":"2024-04-25T08:35:40.258685+0000","last_unstale":"2024-04-25T08:35:41.285711+0000","last_undegraded":"2024-04-25T08:35:41.285711+0000","last_fullsized":"2024-04-25T08:35:41.285711+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:32.349483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.267941+0000","last_change":"2024-04-25T08:35:41.266844+0000","last_active":"2024-04-25T08:35:41.267941+0000","last_peered":"2024-04-25T08:35:41.267941+0000","last_clean":"2024-04-25T08:35:41.267941+0000","last_became_active":"2024-04-25T08:35:41.266459+0000","last_became_peered":"2024-04-25T08:35:41.266459+0000","last_unstale":"2024-04-25T08:35:41.267941+0000","last_undegraded":"2024-04-25T08:35:41.267941+0000","last_fullsized":"2024-04-25T08:35:41.267941+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:16.741664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.289265+0000","last_change":"2024-04-25T08:35:40.270098+0000","last_active":"2024-04-25T08:35:41.289265+0000","last_peered":"2024-04-25T08:35:41.289265+0000","last_clean":"2024-04-25T08:35:41.289265+0000","last_became_active":"2024-04-25T08:35:40.269823+0000","last_became_peered":"2024-04-25T08:35:40.269823+0000","last_unstale":"2024-04-25T08:35:41.289265+0000","last_undegraded":"2024-04-25T08:35:41.289265+0000","last_fullsized":"2024-04-25T08:35:41.289265+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:40.450827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.268983+0000","last_change":"2024-04-25T08:35:41.261767+0000","last_active":"2024-04-25T08:35:41.268983+0000","last_peered":"2024-04-25T08:35:41.268983+0000","last_clean":"2024-04-25T08:35:41.268983+0000","last_became_active":"2024-04-25T08:35:41.261457+0000","last_became_peered":"2024-04-25T08:35:41.261457+0000","last_unstale":"2024-04-25T08:35:41.268983+0000","last_undegraded":"2024-04-25T08:35:41.268983+0000","last_fullsized":"2024-04-25T08:35:41.268983+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:01.633527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.281134+0000","last_change":"2024-04-25T08:35:40.260112+0000","last_active":"2024-04-25T08:35:41.281134+0000","last_peered":"2024-04-25T08:35:41.281134+0000","last_clean":"2024-04-25T08:35:41.281134+0000","last_became_active":"2024-04-25T08:35:40.258446+0000","last_became_peered":"2024-04-25T08:35:40.258446+0000","last_unstale":"2024-04-25T08:35:41.281134+0000","last_undegraded":"2024-04-25T08:35:41.281134+0000","last_fullsized":"2024-04-25T08:35:41.281134+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:06.862076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.263705+0000","last_change":"2024-04-25T08:35:41.262963+0000","last_active":"2024-04-25T08:35:41.263705+0000","last_peered":"2024-04-25T08:35:41.263705+0000","last_clean":"2024-04-25T08:35:41.263705+0000","last_became_active":"2024-04-25T08:35:41.262686+0000","last_became_peered":"2024-04-25T08:35:41.262686+0000","last_unstale":"2024-04-25T08:35:41.263705+0000","last_undegraded":"2024-04-25T08:35:41.263705+0000","last_fullsized":"2024-04-25T08:35:41.263705+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:02.003427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":13,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.289938+0000","last_change":"2024-04-25T08:35:40.266422+0000","last_active":"2024-04-25T08:35:41.289938+0000","last_peered":"2024-04-25T08:35:41.289938+0000","last_clean":"2024-04-25T08:35:41.289938+0000","last_became_active":"2024-04-25T08:35:40.266160+0000","last_became_peered":"2024-04-25T08:35:40.266160+0000","last_unstale":"2024-04-25T08:35:41.289938+0000","last_undegraded":"2024-04-25T08:35:41.289938+0000","last_fullsized":"2024-04-25T08:35:41.289938+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:21.997308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.270502+0000","last_change":"2024-04-25T08:35:41.268401+0000","last_active":"2024-04-25T08:35:41.270502+0000","last_peered":"2024-04-25T08:35:41.270502+0000","last_clean":"2024-04-25T08:35:41.270502+0000","last_became_active":"2024-04-25T08:35:41.267365+0000","last_became_peered":"2024-04-25T08:35:41.267365+0000","last_unstale":"2024-04-25T08:35:41.270502+0000","last_undegraded":"2024-04-25T08:35:41.270502+0000","last_fullsized":"2024-04-25T08:35:41.270502+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:15.347371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.263627+0000","last_change":"2024-04-25T08:35:40.262760+0000","last_active":"2024-04-25T08:35:41.263627+0000","last_peered":"2024-04-25T08:35:41.263627+0000","last_clean":"2024-04-25T08:35:41.263627+0000","last_became_active":"2024-04-25T08:35:40.262269+0000","last_became_peered":"2024-04-25T08:35:40.262269+0000","last_unstale":"2024-04-25T08:35:41.263627+0000","last_undegraded":"2024-04-25T08:35:41.263627+0000","last_fullsized":"2024-04-25T08:35:41.263627+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:35.351092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.284765+0000","last_change":"2024-04-25T08:35:41.281101+0000","last_active":"2024-04-25T08:35:41.284765+0000","last_peered":"2024-04-25T08:35:41.284765+0000","last_clean":"2024-04-25T08:35:41.284765+0000","last_became_active":"2024-04-25T08:35:41.280675+0000","last_became_peered":"2024-04-25T08:35:41.280675+0000","last_unstale":"2024-04-25T08:35:41.284765+0000","last_undegraded":"2024-04-25T08:35:41.284765+0000","last_fullsized":"2024-04-25T08:35:41.284765+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:03.389798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.275440+0000","last_change":"2024-04-25T08:35:40.262619+0000","last_active":"2024-04-25T08:35:41.275440+0000","last_peered":"2024-04-25T08:35:41.275440+0000","last_clean":"2024-04-25T08:35:41.275440+0000","last_became_active":"2024-04-25T08:35:40.261539+0000","last_became_peered":"2024-04-25T08:35:40.261539+0000","last_unstale":"2024-04-25T08:35:41.275440+0000","last_undegraded":"2024-04-25T08:35:41.275440+0000","last_fullsized":"2024-04-25T08:35:41.275440+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:17.946572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.278190+0000","last_change":"2024-04-25T08:35:41.276673+0000","last_active":"2024-04-25T08:35:41.278190+0000","last_peered":"2024-04-25T08:35:41.278190+0000","last_clean":"2024-04-25T08:35:41.278190+0000","last_became_active":"2024-04-25T08:35:41.276411+0000","last_became_peered":"2024-04-25T08:35:41.276411+0000","last_unstale":"2024-04-25T08:35:41.278190+0000","last_undegraded":"2024-04-25T08:35:41.278190+0000","last_fullsized":"2024-04-25T08:35:41.278190+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:22.999772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.269757+0000","last_change":"2024-04-25T08:35:40.267673+0000","last_active":"2024-04-25T08:35:41.269757+0000","last_peered":"2024-04-25T08:35:41.269757+0000","last_clean":"2024-04-25T08:35:41.269757+0000","last_became_active":"2024-04-25T08:35:40.267362+0000","last_became_peered":"2024-04-25T08:35:40.267362+0000","last_unstale":"2024-04-25T08:35:41.269757+0000","last_undegraded":"2024-04-25T08:35:41.269757+0000","last_fullsized":"2024-04-25T08:35:41.269757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:31.396343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.278608+0000","last_change":"2024-04-25T08:35:41.276828+0000","last_active":"2024-04-25T08:35:41.278608+0000","last_peered":"2024-04-25T08:35:41.278608+0000","last_clean":"2024-04-25T08:35:41.278608+0000","last_became_active":"2024-04-25T08:35:41.276544+0000","last_became_peered":"2024-04-25T08:35:41.276544+0000","last_unstale":"2024-04-25T08:35:41.278608+0000","last_undegraded":"2024-04-25T08:35:41.278608+0000","last_fullsized":"2024-04-25T08:35:41.278608+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:41.270237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.264077+0000","last_change":"2024-04-25T08:35:40.258315+0000","last_active":"2024-04-25T08:35:41.264077+0000","last_peered":"2024-04-25T08:35:41.264077+0000","last_clean":"2024-04-25T08:35:41.264077+0000","last_became_active":"2024-04-25T08:35:40.258006+0000","last_became_peered":"2024-04-25T08:35:40.258006+0000","last_unstale":"2024-04-25T08:35:41.264077+0000","last_undegraded":"2024-04-25T08:35:41.264077+0000","last_fullsized":"2024-04-25T08:35:41.264077+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:23.179602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.278587+0000","last_change":"2024-04-25T08:35:41.276010+0000","last_active":"2024-04-25T08:35:41.278587+0000","last_peered":"2024-04-25T08:35:41.278587+0000","last_clean":"2024-04-25T08:35:41.278587+0000","last_became_active":"2024-04-25T08:35:41.275634+0000","last_became_peered":"2024-04-25T08:35:41.275634+0000","last_unstale":"2024-04-25T08:35:41.278587+0000","last_undegraded":"2024-04-25T08:35:41.278587+0000","last_fullsized":"2024-04-25T08:35:41.278587+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:02.639501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":14,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.279810+0000","last_change":"2024-04-25T08:35:40.268768+0000","last_active":"2024-04-25T08:35:41.279810+0000","last_peered":"2024-04-25T08:35:41.279810+0000","last_clean":"2024-04-25T08:35:41.279810+0000","last_became_active":"2024-04-25T08:35:40.268480+0000","last_became_peered":"2024-04-25T08:35:40.268480+0000","last_unstale":"2024-04-25T08:35:41.279810+0000","last_undegraded":"2024-04-25T08:35:41.279810+0000","last_fullsized":"2024-04-25T08:35:41.279810+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:45.095049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.283992+0000","last_change":"2024-04-25T08:35:41.280199+0000","last_active":"2024-04-25T08:35:41.283992+0000","last_peered":"2024-04-25T08:35:41.283992+0000","last_clean":"2024-04-25T08:35:41.283992+0000","last_became_active":"2024-04-25T08:35:41.279959+0000","last_became_peered":"2024-04-25T08:35:41.279959+0000","last_unstale":"2024-04-25T08:35:41.283992+0000","last_undegraded":"2024-04-25T08:35:41.283992+0000","last_fullsized":"2024-04-25T08:35:41.283992+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:24.514514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.269620+0000","last_change":"2024-04-25T08:35:40.267806+0000","last_active":"2024-04-25T08:35:41.269620+0000","last_peered":"2024-04-25T08:35:41.269620+0000","last_clean":"2024-04-25T08:35:41.269620+0000","last_became_active":"2024-04-25T08:35:40.267428+0000","last_became_peered":"2024-04-25T08:35:40.267428+0000","last_unstale":"2024-04-25T08:35:41.269620+0000","last_undegraded":"2024-04-25T08:35:41.269620+0000","last_fullsized":"2024-04-25T08:35:41.269620+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:17.189446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.280468+0000","last_change":"2024-04-25T08:35:41.279565+0000","last_active":"2024-04-25T08:35:41.280468+0000","last_peered":"2024-04-25T08:35:41.280468+0000","last_clean":"2024-04-25T08:35:41.280468+0000","last_became_active":"2024-04-25T08:35:41.279273+0000","last_became_peered":"2024-04-25T08:35:41.279273+0000","last_unstale":"2024-04-25T08:35:41.280468+0000","last_undegraded":"2024-04-25T08:35:41.280468+0000","last_fullsized":"2024-04-25T08:35:41.280468+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:39.921639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.265510+0000","last_change":"2024-04-25T08:35:40.266497+0000","last_active":"2024-04-25T08:35:41.265510+0000","last_peered":"2024-04-25T08:35:41.265510+0000","last_clean":"2024-04-25T08:35:41.265510+0000","last_became_active":"2024-04-25T08:35:40.266220+0000","last_became_peered":"2024-04-25T08:35:40.266220+0000","last_unstale":"2024-04-25T08:35:41.265510+0000","last_undegraded":"2024-04-25T08:35:41.265510+0000","last_fullsized":"2024-04-25T08:35:41.265510+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:47.688555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.270481+0000","last_change":"2024-04-25T08:35:41.267541+0000","last_active":"2024-04-25T08:35:41.270481+0000","last_peered":"2024-04-25T08:35:41.270481+0000","last_clean":"2024-04-25T08:35:41.270481+0000","last_became_active":"2024-04-25T08:35:41.267260+0000","last_became_peered":"2024-04-25T08:35:41.267260+0000","last_unstale":"2024-04-25T08:35:41.270481+0000","last_undegraded":"2024-04-25T08:35:41.270481+0000","last_fullsized":"2024-04-25T08:35:41.270481+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:15.313710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.272254+0000","last_change":"2024-04-25T08:35:40.258676+0000","last_active":"2024-04-25T08:35:41.272254+0000","last_peered":"2024-04-25T08:35:41.272254+0000","last_clean":"2024-04-25T08:35:41.272254+0000","last_became_active":"2024-04-25T08:35:40.258389+0000","last_became_peered":"2024-04-25T08:35:40.258389+0000","last_unstale":"2024-04-25T08:35:41.272254+0000","last_undegraded":"2024-04-25T08:35:41.272254+0000","last_fullsized":"2024-04-25T08:35:41.272254+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:22.053802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287850+0000","last_change":"2024-04-25T08:35:41.286638+0000","last_active":"2024-04-25T08:35:41.287850+0000","last_peered":"2024-04-25T08:35:41.287850+0000","last_clean":"2024-04-25T08:35:41.287850+0000","last_became_active":"2024-04-25T08:35:41.286367+0000","last_became_peered":"2024-04-25T08:35:41.286367+0000","last_unstale":"2024-04-25T08:35:41.287850+0000","last_undegraded":"2024-04-25T08:35:41.287850+0000","last_fullsized":"2024-04-25T08:35:41.287850+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:23.056265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.259778+0000","last_change":"2024-04-25T08:35:40.261449+0000","last_active":"2024-04-25T08:35:41.259778+0000","last_peered":"2024-04-25T08:35:41.259778+0000","last_clean":"2024-04-25T08:35:41.259778+0000","last_became_active":"2024-04-25T08:35:40.261186+0000","last_became_peered":"2024-04-25T08:35:40.261186+0000","last_unstale":"2024-04-25T08:35:41.259778+0000","last_undegraded":"2024-04-25T08:35:41.259778+0000","last_fullsized":"2024-04-25T08:35:41.259778+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:29.601720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.266289+0000","last_change":"2024-04-25T08:35:41.264247+0000","last_active":"2024-04-25T08:35:41.266289+0000","last_peered":"2024-04-25T08:35:41.266289+0000","last_clean":"2024-04-25T08:35:41.266289+0000","last_became_active":"2024-04-25T08:35:41.263934+0000","last_became_peered":"2024-04-25T08:35:41.263934+0000","last_unstale":"2024-04-25T08:35:41.266289+0000","last_undegraded":"2024-04-25T08:35:41.266289+0000","last_fullsized":"2024-04-25T08:35:41.266289+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:53:32.631968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.268601+0000","last_change":"2024-04-25T08:35:40.265800+0000","last_active":"2024-04-25T08:35:41.268601+0000","last_peered":"2024-04-25T08:35:41.268601+0000","last_clean":"2024-04-25T08:35:41.268601+0000","last_became_active":"2024-04-25T08:35:40.265441+0000","last_became_peered":"2024-04-25T08:35:40.265441+0000","last_unstale":"2024-04-25T08:35:41.268601+0000","last_undegraded":"2024-04-25T08:35:41.268601+0000","last_fullsized":"2024-04-25T08:35:41.268601+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:00.296484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.290843+0000","last_change":"2024-04-25T08:35:41.286509+0000","last_active":"2024-04-25T08:35:41.290843+0000","last_peered":"2024-04-25T08:35:41.290843+0000","last_clean":"2024-04-25T08:35:41.290843+0000","last_became_active":"2024-04-25T08:35:41.286192+0000","last_became_peered":"2024-04-25T08:35:41.286192+0000","last_unstale":"2024-04-25T08:35:41.290843+0000","last_undegraded":"2024-04-25T08:35:41.290843+0000","last_fullsized":"2024-04-25T08:35:41.290843+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:23.926844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.268324+0000","last_change":"2024-04-25T08:35:40.261296+0000","last_active":"2024-04-25T08:35:41.268324+0000","last_peered":"2024-04-25T08:35:41.268324+0000","last_clean":"2024-04-25T08:35:41.268324+0000","last_became_active":"2024-04-25T08:35:40.260938+0000","last_became_peered":"2024-04-25T08:35:40.260938+0000","last_unstale":"2024-04-25T08:35:41.268324+0000","last_undegraded":"2024-04-25T08:35:41.268324+0000","last_fullsized":"2024-04-25T08:35:41.268324+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:57.580825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.285801+0000","last_change":"2024-04-25T08:35:41.281046+0000","last_active":"2024-04-25T08:35:41.285801+0000","last_peered":"2024-04-25T08:35:41.285801+0000","last_clean":"2024-04-25T08:35:41.285801+0000","last_became_active":"2024-04-25T08:35:41.279338+0000","last_became_peered":"2024-04-25T08:35:41.279338+0000","last_unstale":"2024-04-25T08:35:41.285801+0000","last_undegraded":"2024-04-25T08:35:41.285801+0000","last_fullsized":"2024-04-25T08:35:41.285801+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:40.314635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.269694+0000","last_change":"2024-04-25T08:35:40.261801+0000","last_active":"2024-04-25T08:35:41.269694+0000","last_peered":"2024-04-25T08:35:41.269694+0000","last_clean":"2024-04-25T08:35:41.269694+0000","last_became_active":"2024-04-25T08:35:40.261540+0000","last_became_peered":"2024-04-25T08:35:40.261540+0000","last_unstale":"2024-04-25T08:35:41.269694+0000","last_undegraded":"2024-04-25T08:35:41.269694+0000","last_fullsized":"2024-04-25T08:35:41.269694+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:26.859603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.287126+0000","last_change":"2024-04-25T08:35:41.286523+0000","last_active":"2024-04-25T08:35:41.287126+0000","last_peered":"2024-04-25T08:35:41.287126+0000","last_clean":"2024-04-25T08:35:41.287126+0000","last_became_active":"2024-04-25T08:35:41.286236+0000","last_became_peered":"2024-04-25T08:35:41.286236+0000","last_unstale":"2024-04-25T08:35:41.287126+0000","last_undegraded":"2024-04-25T08:35:41.287126+0000","last_fullsized":"2024-04-25T08:35:41.287126+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:40.236380+0000","last_clean_scrub_stamp":"2024-04-25T08:35:40.236380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:02.466133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.262340+0000","last_change":"2024-04-25T08:35:40.261725+0000","last_active":"2024-04-25T08:35:41.262340+0000","last_peered":"2024-04-25T08:35:41.262340+0000","last_clean":"2024-04-25T08:35:41.262340+0000","last_became_active":"2024-04-25T08:35:40.261441+0000","last_became_peered":"2024-04-25T08:35:40.261441+0000","last_unstale":"2024-04-25T08:35:41.262340+0000","last_undegraded":"2024-04-25T08:35:41.262340+0000","last_fullsized":"2024-04-25T08:35:41.262340+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:35:39.233916+0000","last_clean_scrub_stamp":"2024-04-25T08:35:39.233916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:15.030362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":258,"reported_epoch":66,"state":"active+clean","last_fresh":"2024-04-25T08:35:41.261874+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:35:41.261874+0000","last_peered":"2024-04-25T08:35:41.261874+0000","last_clean":"2024-04-25T08:35:41.261874+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:35:41.261874+0000","last_undegraded":"2024-04-25T08:35:41.261874+0000","last_fullsized":"2024-04-25T08:35:41.261874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574904,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46452,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325388,"statfs":{"total":96636764160,"available":96589197312,"internally_reserved":0,"allocated":1409024,"data_stored":954602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574904,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40288,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331552,"statfs":{"total":96636764160,"available":96595509248,"internally_reserved":0,"allocated":802816,"data_stored":363492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40284,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331556,"statfs":{"total":96636764160,"available":96595513344,"internally_reserved":0,"allocated":802816,"data_stored":363480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45860,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325980,"statfs":{"total":96636764160,"available":96589803520,"internally_reserved":0,"allocated":811008,"data_stored":365070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":363458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40288,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331552,"statfs":{"total":96636764160,"available":96595509248,"internally_reserved":0,"allocated":798720,"data_stored":363458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40308,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331532,"statfs":{"total":96636764160,"available":96595488768,"internally_reserved":0,"allocated":827392,"data_stored":365834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574904,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54756,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317084,"statfs":{"total":96636764160,"available":96580694016,"internally_reserved":0,"allocated":1392640,"data_stored":953826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:35:44.621 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:35:44.621 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:35:44.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.771+0000 7fc6854bd640 1 Processor -- start 2024-04-25T08:35:44.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.771+0000 7fc6854bd640 1 -- start start 2024-04-25T08:35:44.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.772+0000 7fc6854bd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc68006abe0 0x7fc68006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.772+0000 7fc6854bd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6800b3400 0x7fc6800b0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.772+0000 7fc6854bd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc68006b500 con 0x7fc6800b3880 2024-04-25T08:35:44.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.772+0000 7fc6854bd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6800b1c30 con 0x7fc68006abe0 2024-04-25T08:35:44.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.772+0000 7fc6854bd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6800b5390 con 0x7fc6800b3400 2024-04-25T08:35:44.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.772+0000 7fc67effd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc68006abe0 0x7fc68006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.772+0000 7fc67effd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc68006abe0 0x7fc68006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57130/0 (socket says 172.21.15.38:57130) 2024-04-25T08:35:44.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.772+0000 7fc67effd640 1 -- 172.21.15.38:0/1915399606 learned_addr learned my addr 172.21.15.38:0/1915399606 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:44.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.773+0000 7fc67e7fc640 1 --2- 172.21.15.38:0/1915399606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6800b3400 0x7fc6800b0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.773+0000 7fc67effd640 1 -- 172.21.15.38:0/1915399606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6800b3400 msgr2=0x7fc6800b0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:44.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.773+0000 7fc67effd640 1 --2- 172.21.15.38:0/1915399606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6800b3400 0x7fc6800b0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.773+0000 7fc67effd640 1 -- 172.21.15.38:0/1915399606 >> v1:172.21.15.107:6789/0 conn(0x7fc6800b3880 legacy=0x7fc6800ad3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.773+0000 7fc67effd640 1 -- 172.21.15.38:0/1915399606 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6800affe0 con 0x7fc68006abe0 2024-04-25T08:35:44.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.773+0000 7fc67effd640 1 --2- 172.21.15.38:0/1915399606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc68006abe0 0x7fc68006afc0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc678009870 tx=0x7fc67802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae55755e9fc608d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:44.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.774+0000 7fc67dffb640 1 -- 172.21.15.38:0/1915399606 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678037070 con 0x7fc68006abe0 2024-04-25T08:35:44.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.774+0000 7fc67dffb640 1 -- 172.21.15.38:0/1915399606 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678032440 con 0x7fc68006abe0 2024-04-25T08:35:44.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.774+0000 7fc67dffb640 1 -- 172.21.15.38:0/1915399606 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678032810 con 0x7fc68006abe0 2024-04-25T08:35:44.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.774+0000 7fc6854bd640 1 -- 172.21.15.38:0/1915399606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc68006abe0 msgr2=0x7fc68006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.774+0000 7fc6854bd640 1 --2- 172.21.15.38:0/1915399606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc68006abe0 0x7fc68006afc0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc678009870 tx=0x7fc67802a270 comp rx=0 tx=0).stop 2024-04-25T08:35:44.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.774+0000 7fc6854bd640 1 -- 172.21.15.38:0/1915399606 shutdown_connections 2024-04-25T08:35:44.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.774+0000 7fc6854bd640 1 --2- 172.21.15.38:0/1915399606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6800b3400 0x7fc6800b0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.774+0000 7fc6854bd640 1 --2- 172.21.15.38:0/1915399606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc68006abe0 0x7fc68006afc0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.774+0000 7fc6854bd640 1 -- 172.21.15.38:0/1915399606 >> 172.21.15.38:0/1915399606 conn(0x7fc680099a70 msgr2=0x7fc6800b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:44.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.774+0000 7fc6854bd640 1 -- 172.21.15.38:0/1915399606 shutdown_connections 2024-04-25T08:35:44.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.774+0000 7fc6854bd640 1 -- 172.21.15.38:0/1915399606 wait complete. 2024-04-25T08:35:44.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.775+0000 7fc6854bd640 1 Processor -- start 2024-04-25T08:35:44.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.776+0000 7fc6854bd640 1 -- start start 2024-04-25T08:35:44.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.776+0000 7fc6854bd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc68006abe0 0x7fc68015a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.776+0000 7fc6854bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6800b3400 0x7fc68009d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.776+0000 7fc6854bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6800b3880 0x7fc68015f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.776+0000 7fc6854bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6800b5670 con 0x7fc6800b3400 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.776+0000 7fc6854bd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6800b05a0 con 0x7fc6800b3880 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.776+0000 7fc6854bd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68015ed20 con 0x7fc68006abe0 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.776+0000 7fc67e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6800b3400 0x7fc68009d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.776+0000 7fc67e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6800b3400 0x7fc68009d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57132/0 (socket says 172.21.15.38:57132) 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.776+0000 7fc67e7fc640 1 -- 172.21.15.38:0/3516437524 learned_addr learned my addr 172.21.15.38:0/3516437524 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc67effd640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc68006abe0 0x7fc68015a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.776+0000 7fc67f7fe640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6800b3880 0x7fc68015f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc67e7fc640 1 -- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc68006abe0 msgr2=0x7fc68015a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc67e7fc640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc68006abe0 0x7fc68015a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc67e7fc640 1 -- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6800b3880 msgr2=0x7fc68015f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc67e7fc640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6800b3880 0x7fc68015f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc67e7fc640 1 -- 172.21.15.38:0/3516437524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68009e510 con 0x7fc6800b3400 2024-04-25T08:35:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc67f7fe640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6800b3880 0x7fc68015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:44.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc67e7fc640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6800b3400 0x7fc68009d470 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc66800d800 tx=0x7fc66800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:44.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc6777fe640 1 -- 172.21.15.38:0/3516437524 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66800b6c0 con 0x7fc6800b3400 2024-04-25T08:35:44.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc6777fe640 1 -- 172.21.15.38:0/3516437524 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc668019070 con 0x7fc6800b3400 2024-04-25T08:35:44.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc6854bd640 1 -- 172.21.15.38:0/3516437524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68009df50 con 0x7fc6800b3400 2024-04-25T08:35:44.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc6777fe640 1 -- 172.21.15.38:0/3516437524 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66800ba80 con 0x7fc6800b3400 2024-04-25T08:35:44.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.777+0000 7fc6854bd640 1 -- 172.21.15.38:0/3516437524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc68009a600 con 0x7fc6800b3400 2024-04-25T08:35:44.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.778+0000 7fc6854bd640 1 -- 172.21.15.38:0/3516437524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6801e9a20 con 0x7fc6800b3400 2024-04-25T08:35:44.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.778+0000 7fc6777fe640 1 -- 172.21.15.38:0/3516437524 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc66800bc20 con 0x7fc6800b3400 2024-04-25T08:35:44.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.779+0000 7fc6777fe640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc66006b4d0 0x7fc66006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:44.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.779+0000 7fc6777fe640 1 -- 172.21.15.38:0/3516437524 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc66808a790 con 0x7fc6800b3400 2024-04-25T08:35:44.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.779+0000 7fc67effd640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc66006b4d0 0x7fc66006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:44.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.780+0000 7fc67effd640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc66006b4d0 0x7fc66006d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc678034000 tx=0x7fc67802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:44.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:44.783+0000 7fc6777fe640 1 -- 172.21.15.38:0/3516437524 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc668059470 con 0x7fc6800b3400 2024-04-25T08:35:45.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.030+0000 7fc6854bd640 1 -- 172.21.15.38:0/3516437524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc68009a9a0 con 0x7fc6800b3400 2024-04-25T08:35:45.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.036+0000 7fc6777fe640 1 -- 172.21.15.38:0/3516437524 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc668058d70 con 0x7fc6800b3400 2024-04-25T08:35:45.031 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-04-25T08:35:45.031 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw== 2024-04-25T08:35:45.032 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-04-25T08:35:45.032 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-04-25T08:35:45.032 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:35:45.032 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-04-25T08:35:45.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.038+0000 7fc6854bd640 1 -- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc66006b4d0 msgr2=0x7fc66006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.038+0000 7fc6854bd640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc66006b4d0 0x7fc66006d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc678034000 tx=0x7fc67802e000 comp rx=0 tx=0).stop 2024-04-25T08:35:45.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.038+0000 7fc6854bd640 1 -- 172.21.15.38:0/3516437524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6800b3400 msgr2=0x7fc68009d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.038+0000 7fc6854bd640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6800b3400 0x7fc68009d470 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fc66800d800 tx=0x7fc66800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.038+0000 7fc6854bd640 1 -- 172.21.15.38:0/3516437524 shutdown_connections 2024-04-25T08:35:45.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.038+0000 7fc6854bd640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6800b3880 0x7fc68015f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.038+0000 7fc6854bd640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6800b3400 0x7fc68009d470 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.038+0000 7fc6854bd640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc66006b4d0 0x7fc66006d990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.038+0000 7fc6854bd640 1 --2- 172.21.15.38:0/3516437524 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc68006abe0 0x7fc68015a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.038+0000 7fc6854bd640 1 -- 172.21.15.38:0/3516437524 >> 172.21.15.38:0/3516437524 conn(0x7fc680099a70 msgr2=0x7fc68015cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.038+0000 7fc6854bd640 1 -- 172.21.15.38:0/3516437524 shutdown_connections 2024-04-25T08:35:45.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.038+0000 7fc6854bd640 1 -- 172.21.15.38:0/3516437524 wait complete. 2024-04-25T08:35:45.045 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:45.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.150+0000 7f1845128640 1 Processor -- start 2024-04-25T08:35:45.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.150+0000 7f1845128640 1 -- start start 2024-04-25T08:35:45.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.151+0000 7f1845128640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f18400b5710 0x7f1840161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.151+0000 7f1845128640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f184015ddf0 0x7f18400b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.151+0000 7f1845128640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f18400b5af0 0x7f184006bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.151+0000 7f1845128640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18400b9930 con 0x7f18400b5710 2024-04-25T08:35:45.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.151+0000 7f1845128640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1840161e00 con 0x7f18400b5af0 2024-04-25T08:35:45.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.151+0000 7f1845128640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18401620e0 con 0x7f184015ddf0 2024-04-25T08:35:45.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.151+0000 7f183ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f18400b5710 0x7f1840161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.151+0000 7f183ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f18400b5710 0x7f1840161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57142/0 (socket says 172.21.15.38:57142) 2024-04-25T08:35:45.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.151+0000 7f183ed9d640 1 -- 172.21.15.38:0/2938731888 learned_addr learned my addr 172.21.15.38:0/2938731888 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:45.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.152+0000 7f183f59e640 1 --2- 172.21.15.38:0/2938731888 >> v2:172.21.15.107:3300/0 conn(0x7f18400b5af0 0x7f184006bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.152+0000 7f183e59c640 1 --2- 172.21.15.38:0/2938731888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f184015ddf0 0x7f18400b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.152+0000 7f183ed9d640 1 -- 172.21.15.38:0/2938731888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f184015ddf0 msgr2=0x7f18400b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.152+0000 7f183ed9d640 1 --2- 172.21.15.38:0/2938731888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f184015ddf0 0x7f18400b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.152+0000 7f183ed9d640 1 -- 172.21.15.38:0/2938731888 >> v2:172.21.15.107:3300/0 conn(0x7f18400b5af0 msgr2=0x7f184006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.152+0000 7f183ed9d640 1 --2- 172.21.15.38:0/2938731888 >> v2:172.21.15.107:3300/0 conn(0x7f18400b5af0 0x7f184006bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.152+0000 7f183ed9d640 1 -- 172.21.15.38:0/2938731888 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18401623c0 con 0x7f18400b5710 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.152+0000 7f183e59c640 1 --2- 172.21.15.38:0/2938731888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f184015ddf0 0x7f18400b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.152+0000 7f183ed9d640 1 --2- 172.21.15.38:0/2938731888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18400b5710 0x7f1840161700 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f18400d1bd0 tx=0x7f183002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b60500412f2824 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.153+0000 7f183dd9b640 1 -- 172.21.15.38:0/2938731888 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830037070 con 0x7f18400b5710 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.153+0000 7f183dd9b640 1 -- 172.21.15.38:0/2938731888 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1830032440 con 0x7f18400b5710 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.153+0000 7f183dd9b640 1 -- 172.21.15.38:0/2938731888 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1830032810 con 0x7f18400b5710 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.153+0000 7f1845128640 1 -- 172.21.15.38:0/2938731888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18400b5710 msgr2=0x7f1840161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.153+0000 7f1845128640 1 --2- 172.21.15.38:0/2938731888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18400b5710 0x7f1840161700 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f18400d1bd0 tx=0x7f183002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.153+0000 7f1845128640 1 -- 172.21.15.38:0/2938731888 shutdown_connections 2024-04-25T08:35:45.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.153+0000 7f1845128640 1 --2- 172.21.15.38:0/2938731888 >> v2:172.21.15.107:3300/0 conn(0x7f18400b5af0 0x7f184006bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.153+0000 7f1845128640 1 --2- 172.21.15.38:0/2938731888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f184015ddf0 0x7f18400b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.153+0000 7f1845128640 1 --2- 172.21.15.38:0/2938731888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18400b5710 0x7f1840161700 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.153+0000 7f1845128640 1 -- 172.21.15.38:0/2938731888 >> 172.21.15.38:0/2938731888 conn(0x7f18400d07c0 msgr2=0x7f18400d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.153+0000 7f1845128640 1 -- 172.21.15.38:0/2938731888 shutdown_connections 2024-04-25T08:35:45.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.153+0000 7f1845128640 1 -- 172.21.15.38:0/2938731888 wait complete. 2024-04-25T08:35:45.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.155+0000 7f1845128640 1 Processor -- start 2024-04-25T08:35:45.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.155+0000 7f1845128640 1 -- start start 2024-04-25T08:35:45.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.155+0000 7f1845128640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18400b5710 0x7f1840158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.155+0000 7f1845128640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18400b5af0 0x7f18401e73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.155+0000 7f1845128640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f184015ddf0 0x7f18401e8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.155+0000 7f1845128640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184006c6b0 con 0x7f18400b5af0 2024-04-25T08:35:45.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f1845128640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f184006c530 con 0x7f18400b5710 2024-04-25T08:35:45.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f1845128640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184006c830 con 0x7f184015ddf0 2024-04-25T08:35:45.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f183e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18400b5af0 0x7f18401e73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f183e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18400b5af0 0x7f18401e73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57148/0 (socket says 172.21.15.38:57148) 2024-04-25T08:35:45.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f183e59c640 1 -- 172.21.15.38:0/1215148589 learned_addr learned my addr 172.21.15.38:0/1215148589 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:45.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f183ed9d640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18400b5710 0x7f1840158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f183f59e640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f184015ddf0 0x7f18401e8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f183e59c640 1 -- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f184015ddf0 msgr2=0x7f18401e8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f183e59c640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f184015ddf0 0x7f18401e8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f183e59c640 1 -- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18400b5710 msgr2=0x7f1840158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f183e59c640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18400b5710 0x7f1840158a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f183e59c640 1 -- 172.21.15.38:0/1215148589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18401e7c20 con 0x7f18400b5af0 2024-04-25T08:35:45.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f183e59c640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18400b5af0 0x7f18401e73b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f183400c7c0 tx=0x7f183400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f18277fe640 1 -- 172.21.15.38:0/1215148589 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834015070 con 0x7f18400b5af0 2024-04-25T08:35:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f18277fe640 1 -- 172.21.15.38:0/1215148589 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1834007820 con 0x7f18400b5af0 2024-04-25T08:35:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f18277fe640 1 -- 172.21.15.38:0/1215148589 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1834007bf0 con 0x7f18400b5af0 2024-04-25T08:35:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f1845128640 1 -- 172.21.15.38:0/1215148589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18401e7f00 con 0x7f18400b5af0 2024-04-25T08:35:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.156+0000 7f1845128640 1 -- 172.21.15.38:0/1215148589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18401e84c0 con 0x7f18400b5af0 2024-04-25T08:35:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.157+0000 7f1845128640 1 -- 172.21.15.38:0/1215148589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1840068e80 con 0x7f18400b5af0 2024-04-25T08:35:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.158+0000 7f18277fe640 1 -- 172.21.15.38:0/1215148589 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1834018070 con 0x7f18400b5af0 2024-04-25T08:35:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.158+0000 7f18277fe640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f182c06b4d0 0x7f182c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.158+0000 7f18277fe640 1 -- 172.21.15.38:0/1215148589 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f183408a150 con 0x7f18400b5af0 2024-04-25T08:35:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.158+0000 7f183ed9d640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f182c06b4d0 0x7f182c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.159+0000 7f183ed9d640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f182c06b4d0 0x7f182c06d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1830034000 tx=0x7f183002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.162+0000 7f18277fe640 1 -- 172.21.15.38:0/1215148589 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1834058880 con 0x7f18400b5af0 2024-04-25T08:35:45.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.380+0000 7f1845128640 1 -- 172.21.15.38:0/1215148589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18400cfdd0 con 0x7f18400b5af0 2024-04-25T08:35:45.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.380+0000 7f18277fe640 1 -- 172.21.15.38:0/1215148589 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f183405c920 con 0x7f18400b5af0 2024-04-25T08:35:45.376 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:45.376 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3966080134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3966080134},{"type":"v1","addr":"172.21.15.107:6836","nonce":3966080134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/632873469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":632873469},{"type":"v1","addr":"172.21.15.107:6837","nonce":632873469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4257593739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4257593739},{"type":"v1","addr":"172.21.15.38:6834","nonce":4257593739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:40.623355+0000","modified":"2024-04-25T08:35:42.623054+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6813},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6813":{"gid":6813,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/239365678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":239365678},{"type":"v1","addr":"172.21.15.38:6835","nonce":239365678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:35:45.376 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 35 2024-04-25T08:35:45.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.382+0000 7f1845128640 1 -- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f182c06b4d0 msgr2=0x7f182c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.382+0000 7f1845128640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f182c06b4d0 0x7f182c06d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1830034000 tx=0x7f183002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:45.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.382+0000 7f1845128640 1 -- 172.21.15.38:0/1215148589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18400b5af0 msgr2=0x7f18401e73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.382+0000 7f1845128640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18400b5af0 0x7f18401e73b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f183400c7c0 tx=0x7f183400cc80 comp rx=0 tx=0).stop 2024-04-25T08:35:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.382+0000 7f1845128640 1 -- 172.21.15.38:0/1215148589 shutdown_connections 2024-04-25T08:35:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.382+0000 7f1845128640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f182c06b4d0 0x7f182c06d990 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.382+0000 7f1845128640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f184015ddf0 0x7f18401e8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.382+0000 7f1845128640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f18400b5af0 0x7f18401e73b0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.382+0000 7f1845128640 1 --2- 172.21.15.38:0/1215148589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f18400b5710 0x7f1840158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.382+0000 7f1845128640 1 -- 172.21.15.38:0/1215148589 >> 172.21.15.38:0/1215148589 conn(0x7f18400d07c0 msgr2=0x7f18400b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.382+0000 7f1845128640 1 -- 172.21.15.38:0/1215148589 shutdown_connections 2024-04-25T08:35:45.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.382+0000 7f1845128640 1 -- 172.21.15.38:0/1215148589 wait complete. 2024-04-25T08:35:45.792 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:35:40.623355+0000', 'modified': '2024-04-25T08:35:42.623054+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6813}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6813': {'gid': 6813, 'name': 'a', 'rank': 0, 'incarnation': 33, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6835/239365678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6833', 'nonce': 239365678}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 239365678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:35:45.792 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:35:45.792 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:35:45.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.903+0000 7ff4a20f7640 1 Processor -- start 2024-04-25T08:35:45.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.903+0000 7ff4a20f7640 1 -- start start 2024-04-25T08:35:45.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.904+0000 7ff4a20f7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff49c0cf7b0 0x7ff49c06c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.904+0000 7ff4a20f7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c06d9f0 con 0x7ff49c099a70 2024-04-25T08:35:45.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.904+0000 7ff4a20f7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c06ce70 con 0x7ff49c15f990 2024-04-25T08:35:45.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.904+0000 7ff4a20f7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c06d150 con 0x7ff49c0cf7b0 2024-04-25T08:35:45.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.904+0000 7ff49b7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff49c099a70 0x7ff49c099e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:39900/0 (socket says 172.21.15.38:39900) 2024-04-25T08:35:45.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.904+0000 7ff49b7fe640 1 -- 172.21.15.38:0/1922751335 learned_addr learned my addr 172.21.15.38:0/1922751335 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:45.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49affd640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff49c0cf7b0 0x7ff49c06c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083685031 0 0) 0x7ff49c06d9f0 con 0x7ff49c099a70 2024-04-25T08:35:45.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff484003520 con 0x7ff49c099a70 2024-04-25T08:35:45.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215788760 0 0) 0x7ff49c06ce70 con 0x7ff49c15f990 2024-04-25T08:35:45.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff49c06d9f0 con 0x7ff49c15f990 2024-04-25T08:35:45.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 274813267 0 0) 0x7ff484003520 con 0x7ff49c099a70 2024-04-25T08:35:45.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff49c0cf7b0 msgr2=0x7ff49c06c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff49c0cf7b0 0x7ff49c06c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 >> v1:172.21.15.107:6789/0 conn(0x7ff49c15f990 legacy=0x7ff49c15fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c06d430 con 0x7ff49c099a70 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7ff48c001670 con 0x7ff49c099a70 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 >> v1:172.21.15.38:6789/0 conn(0x7ff49c099a70 legacy=0x7ff49c099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff49c0cf7b0 0x7ff484003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4840040d0 0x7ff484004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff484004d40 0x7ff4840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c06d150 con 0x7ff4840040d0 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48400bb50 con 0x7ff484004d40 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48400bd20 con 0x7ff49c0cf7b0 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49affd640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4840040d0 0x7ff484004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff49c0cf7b0 0x7ff484003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49bfff640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff484004d40 0x7ff4840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49affd640 1 -- 172.21.15.38:0/1922751335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff49c0cf7b0 msgr2=0x7ff484003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49affd640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff49c0cf7b0 0x7ff484003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49affd640 1 -- 172.21.15.38:0/1922751335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff484004d40 msgr2=0x7ff4840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49affd640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff484004d40 0x7ff4840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49affd640 1 -- 172.21.15.38:0/1922751335 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff484003520 con 0x7ff4840040d0 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.905+0000 7ff49bfff640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff484004d40 0x7ff4840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.906+0000 7ff49affd640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4840040d0 0x7ff484004800 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff490009ac0 tx=0x7ff484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf9c8d3b1faf5d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.906+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490036070 con 0x7ff4840040d0 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.906+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff490031400 con 0x7ff4840040d0 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.906+0000 7ff49a7fc640 1 -- 172.21.15.38:0/1922751335 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490009690 con 0x7ff4840040d0 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.907+0000 7ff4a20f7640 1 -- 172.21.15.38:0/1922751335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4840040d0 msgr2=0x7ff484004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.907+0000 7ff4a20f7640 1 --2- 172.21.15.38:0/1922751335 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff4840040d0 0x7ff484004800 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7ff490009ac0 tx=0x7ff484005830 comp rx=0 tx=0).stop 2024-04-25T08:35:45.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.907+0000 7ff49bfff640 1 -- 172.21.15.38:0/1922751335 reap_dead start 2024-04-25T08:35:45.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.907+0000 7ff4a20f7640 1 -- 172.21.15.38:0/1922751335 shutdown_connections 2024-04-25T08:35:45.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.907+0000 7ff4a20f7640 1 -- 172.21.15.38:0/1922751335 >> 172.21.15.38:0/1922751335 conn(0x7ff49c0b3190 msgr2=0x7ff49c0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.907+0000 7ff4a20f7640 1 -- 172.21.15.38:0/1922751335 shutdown_connections 2024-04-25T08:35:45.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.907+0000 7ff4a20f7640 1 -- 172.21.15.38:0/1922751335 wait complete. 2024-04-25T08:35:45.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.908+0000 7ff4a20f7640 1 Processor -- start 2024-04-25T08:35:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.908+0000 7ff4a20f7640 1 -- start start 2024-04-25T08:35:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff4a20f7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4840040d0 0x7ff49c14abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff4a20f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff49c0b6b00 0x7ff49c0c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff4a20f7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0b5ed0 0x7ff49c0b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff4a20f7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c06c520 con 0x7ff49c0b6b00 2024-04-25T08:35:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff4a20f7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0d08a0 con 0x7ff49c0b5ed0 2024-04-25T08:35:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff4a20f7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c06c6a0 con 0x7ff4840040d0 2024-04-25T08:35:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff49affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff49c0b6b00 0x7ff49c0c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff49affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff49c0b6b00 0x7ff49c0c79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57174/0 (socket says 172.21.15.38:57174) 2024-04-25T08:35:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff49affd640 1 -- 172.21.15.38:0/3638627613 learned_addr learned my addr 172.21.15.38:0/3638627613 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:45.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4840040d0 0x7ff49c14abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff49bfff640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0b5ed0 0x7ff49c0b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff49affd640 1 -- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4840040d0 msgr2=0x7ff49c14abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff49affd640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4840040d0 0x7ff49c14abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff49affd640 1 -- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0b5ed0 msgr2=0x7ff49c0b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff49affd640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0b5ed0 0x7ff49c0b6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff49affd640 1 -- 172.21.15.38:0/3638627613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c0b4910 con 0x7ff49c0b6b00 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4840040d0 0x7ff49c14abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.909+0000 7ff49bfff640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0b5ed0 0x7ff49c0b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.910+0000 7ff49affd640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff49c0b6b00 0x7ff49c0c79d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff490000c00 tx=0x7ff490004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.910+0000 7ff483fff640 1 -- 172.21.15.38:0/3638627613 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490040020 con 0x7ff49c0b6b00 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.910+0000 7ff483fff640 1 -- 172.21.15.38:0/3638627613 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff490003c60 con 0x7ff49c0b6b00 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.910+0000 7ff483fff640 1 -- 172.21.15.38:0/3638627613 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490036040 con 0x7ff49c0b6b00 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.910+0000 7ff4a20f7640 1 -- 172.21.15.38:0/3638627613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c0b4bf0 con 0x7ff49c0b6b00 2024-04-25T08:35:45.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.910+0000 7ff4a20f7640 1 -- 172.21.15.38:0/3638627613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c0b51b0 con 0x7ff49c0b6b00 2024-04-25T08:35:45.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.911+0000 7ff483fff640 1 -- 172.21.15.38:0/3638627613 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff490043020 con 0x7ff49c0b6b00 2024-04-25T08:35:45.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.912+0000 7ff483fff640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff48806b500 0x7ff48806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.912+0000 7ff483fff640 1 -- 172.21.15.38:0/3638627613 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4900a9e90 con 0x7ff49c0b6b00 2024-04-25T08:35:45.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.912+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff48806b500 0x7ff48806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.913+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff48806b500 0x7ff48806d9c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0011e0 tx=0x7ff48c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.913+0000 7ff4a4df0740 1 Processor -- start 2024-04-25T08:35:45.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.913+0000 7ff4a4df0740 1 -- start start 2024-04-25T08:35:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff4a4df0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 0x55647bfd4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff4a4df0740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bff1ad0 0x55647bff2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff4a4df0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfa9f10 0x55647bff0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff4a4df0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55647bf0ca10 con 0x55647bfa9f10 2024-04-25T08:35:45.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff4a4df0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55647beb5b30 con 0x55647bfd4230 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff4a4df0740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55647bf08c50 con 0x55647bff1ad0 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff49bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfa9f10 0x55647bff0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff49bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfa9f10 0x55647bff0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57188/0 (socket says 172.21.15.38:57188) 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff49bfff640 1 -- 172.21.15.38:0/2141884486 learned_addr learned my addr 172.21.15.38:0/2141884486 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/2141884486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 0x55647bfd4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff49affd640 1 --2- 172.21.15.38:0/2141884486 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bff1ad0 0x55647bff2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff49bfff640 1 -- 172.21.15.38:0/2141884486 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bff1ad0 msgr2=0x55647bff2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff49bfff640 1 --2- 172.21.15.38:0/2141884486 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bff1ad0 0x55647bff2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff49bfff640 1 -- 172.21.15.38:0/2141884486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 msgr2=0x55647bfd4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff49bfff640 1 --2- 172.21.15.38:0/2141884486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 0x55647bfd4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff49bfff640 1 -- 172.21.15.38:0/2141884486 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55647becae10 con 0x55647bfa9f10 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff49bfff640 1 --2- 172.21.15.38:0/2141884486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfa9f10 0x55647bff0190 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff49c15fd70 tx=0x7ff49400f1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435fcac77e75ffe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff4817fa640 1 -- 172.21.15.38:0/2141884486 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494003b80 con 0x55647bfa9f10 2024-04-25T08:35:45.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff4817fa640 1 -- 172.21.15.38:0/2141884486 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff494018070 con 0x55647bfa9f10 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.914+0000 7ff4817fa640 1 -- 172.21.15.38:0/2141884486 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400f8f0 con 0x55647bfa9f10 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.915+0000 7ff4a4df0740 1 -- 172.21.15.38:0/2141884486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfa9f10 msgr2=0x55647bff0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.915+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/2141884486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfa9f10 0x55647bff0190 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff49c15fd70 tx=0x7ff49400f1d0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.915+0000 7ff4a4df0740 1 -- 172.21.15.38:0/2141884486 shutdown_connections 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.915+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/2141884486 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfa9f10 0x55647bff0190 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.915+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/2141884486 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bff1ad0 0x55647bff2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.915+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/2141884486 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 0x55647bfd4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.915+0000 7ff4a4df0740 1 -- 172.21.15.38:0/2141884486 >> 172.21.15.38:0/2141884486 conn(0x55647bfa6840 msgr2=0x55647bfef040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.915+0000 7ff4a4df0740 1 -- 172.21.15.38:0/2141884486 shutdown_connections 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.915+0000 7ff4a4df0740 1 -- 172.21.15.38:0/2141884486 wait complete. 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.916+0000 7ff4a4df0740 1 -- start start 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.916+0000 7ff4a4df0740 1 Processor -- start 2024-04-25T08:35:45.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.916+0000 7ff4a4df0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 0x55647bfc3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.916+0000 7ff4a4df0740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bff1ad0 0x55647bfd2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff4a4df0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 0x55647bfe6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff4a4df0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55647bfaacb0 con 0x55647bfd2f90 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff4a4df0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55647bec90c0 con 0x55647bfd4230 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff4a4df0740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55647bfa4770 con 0x55647bff1ad0 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff4a4df0740 10 client.0 _ll_register_callbacks cb 0x55647bebcec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff49bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 0x55647bfe6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff49bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 0x55647bfe6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57194/0 (socket says 172.21.15.38:57194) 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff49bfff640 1 -- 172.21.15.38:0/3371761136 learned_addr learned my addr 172.21.15.38:0/3371761136 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 0x55647bfc3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff49affd640 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bff1ad0 0x55647bfd2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff49bfff640 1 -- 172.21.15.38:0/3371761136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bff1ad0 msgr2=0x55647bfd2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff49bfff640 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bff1ad0 0x55647bfd2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff49bfff640 1 -- 172.21.15.38:0/3371761136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 msgr2=0x55647bfc3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff49bfff640 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 0x55647bfc3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff49bfff640 1 -- 172.21.15.38:0/3371761136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55647bc212d0 con 0x55647bfd2f90 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff49bfff640 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 0x55647bfe6970 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7ff494015040 tx=0x7ff494001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceff94679ed5413c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff47a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff47a7fc640 1 -- 172.21.15.38:0/3371761136 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494001a40 con 0x55647bfd2f90 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3371761136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff494018070 con 0x55647bfd2f90 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff47a7fc640 1 -- 172.21.15.38:0/3371761136 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff494010cb0 con 0x55647bfd2f90 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff47a7fc640 1 -- 172.21.15.38:0/3371761136 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff49402e050 con 0x55647bfd2f90 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff4a4df0740 10 client.6954 fetch_fsmap learned FSMap version 35 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.917+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3371761136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55647bceb6b0 con 0x55647bfd2f90 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.918+0000 7ff47a7fc640 1 -- 172.21.15.38:0/3371761136 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff494018070 con 0x55647bfd2f90 2024-04-25T08:35:45.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.918+0000 7ff47a7fc640 1 -- 172.21.15.38:0/3371761136 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff494031020 con 0x55647bfd2f90 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.918+0000 7ff4a4df0740 10 client.6954 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.918+0000 7ff4a4df0740 10 client.6954 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.918+0000 7ff4a4df0740 20 client.6954 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.918+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.38:6833/239365678,v1:172.21.15.38:6835/239365678] conn(0x55647bfd7020 0x55647bfd7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.918+0000 7ff4a4df0740 4 client.6954 mds_command: new command op to 6813 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.918+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3371761136 --> [v2:172.21.15.38:6833/239365678,v1:172.21.15.38:6835/239365678] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55647bf6b920 con 0x55647bfd7020 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.918+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.38:6833/239365678,v1:172.21.15.38:6835/239365678] conn(0x55647bfd7020 0x55647bfd7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.918+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.38:6833/239365678,v1:172.21.15.38:6835/239365678] conn(0x55647bfd7020 0x55647bfd7400 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=481f45dcd9cb57d0 server_cookie=a0ddd61b84b2a8a6 in_seq=0 out_seq=0 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.918+0000 7ff47a7fc640 10 client.6954 ms_handle_connect on v2:172.21.15.38:6833/239365678 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff47a7fc640 1 -- 172.21.15.38:0/3371761136 <== mds.0 v2:172.21.15.38:6833/239365678 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55647bf6b920 con 0x55647bfd7020 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff47a7fc640 10 client.6954 handle_command_reply: tid=0 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff4a4df0740 1 client.6954 shutdown 2024-04-25T08:35:45.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3371761136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 msgr2=0x55647bfe6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 0x55647bfe6970 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7ff494015040 tx=0x7ff494001650 comp rx=0 tx=0).stop 2024-04-25T08:35:45.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3371761136 shutdown_connections 2024-04-25T08:35:45.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.38:6833/239365678,v1:172.21.15.38:6835/239365678] conn(0x55647bfd7020 0x55647bfd7400 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff47a7fc640 0 client.6954 ms_handle_reset on v2:172.21.15.38:6833/239365678 2024-04-25T08:35:45.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 0x55647bfe6970 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bff1ad0 0x55647bfd2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/3371761136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 0x55647bfc3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3371761136 >> 172.21.15.38:0/3371761136 conn(0x55647bfa9f10 msgr2=0x55647bff25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3371761136 shutdown_connections 2024-04-25T08:35:45.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3371761136 wait complete. 2024-04-25T08:35:45.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.919+0000 7ff4a4df0740 20 client.6954 trim_cache size 0 max 16384 2024-04-25T08:35:45.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.935+0000 7ff4a4df0740 1 Processor -- start 2024-04-25T08:35:45.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.935+0000 7ff4a4df0740 1 -- start start 2024-04-25T08:35:45.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff4a4df0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 0x55647c051350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff4a4df0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 0x55647bfef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff4a4df0740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bfd7020 0x55647bfef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff4a4df0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c1ffd90 con 0x55647bfd2f90 2024-04-25T08:35:45.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff4a4df0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c1ff190 con 0x55647bfd4230 2024-04-25T08:35:45.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff4a4df0740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c1ff790 con 0x55647bfd7020 2024-04-25T08:35:45.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff49b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 0x55647c051350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff49b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 0x55647c051350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57196/0 (socket says 172.21.15.38:57196) 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff49b7fe640 1 -- 172.21.15.38:0/2572452238 learned_addr learned my addr 172.21.15.38:0/2572452238 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff49affd640 1 --2- 172.21.15.38:0/2572452238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 0x55647bfef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff49bfff640 1 --2- 172.21.15.38:0/2572452238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bfd7020 0x55647bfef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff49b7fe640 1 -- 172.21.15.38:0/2572452238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bfd7020 msgr2=0x55647bfef550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/2572452238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bfd7020 0x55647bfef550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff49b7fe640 1 -- 172.21.15.38:0/2572452238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 msgr2=0x55647bfef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/2572452238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 0x55647bfef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff49b7fe640 1 -- 172.21.15.38:0/2572452238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55647bfa5490 con 0x55647bfd2f90 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/2572452238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 0x55647c051350 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0071c0 tx=0x7ff48c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82efc2d58a03e411 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.936+0000 7ff480ff9640 1 -- 172.21.15.38:0/2572452238 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c02e070 con 0x55647bfd2f90 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.937+0000 7ff480ff9640 1 -- 172.21.15.38:0/2572452238 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c014400 con 0x55647bfd2f90 2024-04-25T08:35:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.937+0000 7ff480ff9640 1 -- 172.21.15.38:0/2572452238 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0147b0 con 0x55647bfd2f90 2024-04-25T08:35:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.937+0000 7ff4a4df0740 1 -- 172.21.15.38:0/2572452238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 msgr2=0x55647c051350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.937+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/2572452238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 0x55647c051350 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0071c0 tx=0x7ff48c013000 comp rx=0 tx=0).stop 2024-04-25T08:35:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.937+0000 7ff4a4df0740 1 -- 172.21.15.38:0/2572452238 shutdown_connections 2024-04-25T08:35:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.937+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/2572452238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bfd7020 0x55647bfef550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.937+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/2572452238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd4230 0x55647bfef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.937+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/2572452238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd2f90 0x55647c051350 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.937+0000 7ff4a4df0740 1 -- 172.21.15.38:0/2572452238 >> 172.21.15.38:0/2572452238 conn(0x55647bfa9f10 msgr2=0x55647bff13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.937+0000 7ff4a4df0740 1 -- 172.21.15.38:0/2572452238 shutdown_connections 2024-04-25T08:35:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.937+0000 7ff4a4df0740 1 -- 172.21.15.38:0/2572452238 wait complete. 2024-04-25T08:35:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.938+0000 7ff4a4df0740 1 -- start start 2024-04-25T08:35:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.938+0000 7ff4a4df0740 1 Processor -- start 2024-04-25T08:35:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff4a4df0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd2f90 0x55647c03ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff4a4df0740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bfd4230 0x55647bfd04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff4a4df0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd7020 0x55647bfd09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff4a4df0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55647bfd7d40 con 0x55647bfd7020 2024-04-25T08:35:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff4a4df0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55647bfd64c0 con 0x55647bfd2f90 2024-04-25T08:35:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff4a4df0740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55647c003090 con 0x55647bfd4230 2024-04-25T08:35:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff4a4df0740 10 client.0 _ll_register_callbacks cb 0x55647bfa4c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff49bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd7020 0x55647bfd09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff49bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd7020 0x55647bfd09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57198/0 (socket says 172.21.15.38:57198) 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff49bfff640 1 -- 172.21.15.38:0/3321834638 learned_addr learned my addr 172.21.15.38:0/3321834638 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd2f90 0x55647c03ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff49affd640 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bfd4230 0x55647bfd04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff49bfff640 1 -- 172.21.15.38:0/3321834638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bfd4230 msgr2=0x55647bfd04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff49bfff640 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bfd4230 0x55647bfd04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff49bfff640 1 -- 172.21.15.38:0/3321834638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd2f90 msgr2=0x55647c03ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff49bfff640 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd2f90 0x55647c03ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff49bfff640 1 -- 172.21.15.38:0/3321834638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55647c011e10 con 0x55647bfd7020 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff49bfff640 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd7020 0x55647bfd09f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55647bfeefe0 tx=0x7ff494009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e261b1426df5bd6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff47a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff47a7fc640 1 -- 172.21.15.38:0/3321834638 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494036070 con 0x55647bfd7020 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff47a7fc640 1 -- 172.21.15.38:0/3321834638 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4940052e0 con 0x55647bfd7020 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.939+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3321834638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff48c014400 con 0x55647bfd7020 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff47a7fc640 1 -- 172.21.15.38:0/3321834638 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff494042050 con 0x55647bfd7020 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff4a4df0740 10 client.6960 fetch_fsmap learned FSMap version 35 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3321834638 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55647bfe6aa0 con 0x55647bfd7020 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff47a7fc640 1 -- 172.21.15.38:0/3321834638 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c014400 con 0x55647bfd7020 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff47a7fc640 1 -- 172.21.15.38:0/3321834638 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff494045020 con 0x55647bfd7020 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff4a4df0740 10 client.6960 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff4a4df0740 10 client.6960 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff4a4df0740 20 client.6960 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.38:6833/239365678,v1:172.21.15.38:6835/239365678] conn(0x55647bff1ad0 0x55647bff0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff4a4df0740 4 client.6960 mds_command: new command op to 6813 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3321834638 --> [v2:172.21.15.38:6833/239365678,v1:172.21.15.38:6835/239365678] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55647bf19f50 con 0x55647bff1ad0 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.38:6833/239365678,v1:172.21.15.38:6835/239365678] conn(0x55647bff1ad0 0x55647bff0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:45.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff47a7fc640 10 client.6960 ms_handle_connect on v2:172.21.15.38:6833/239365678 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff49b7fe640 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.38:6833/239365678,v1:172.21.15.38:6835/239365678] conn(0x55647bff1ad0 0x55647bff0700 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60fe792ec1937ead server_cookie=b47315c78d6c2874 in_seq=0 out_seq=0 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff47a7fc640 1 -- 172.21.15.38:0/3321834638 <== mds.0 v2:172.21.15.38:6833/239365678 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55647bf19f50 con 0x55647bff1ad0 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff47a7fc640 10 client.6960 handle_command_reply: tid=0 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.940+0000 7ff4a4df0740 1 client.6960 shutdown 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3321834638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd7020 msgr2=0x55647bfd09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd7020 0x55647bfd09f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x55647bfeefe0 tx=0x7ff494009040 comp rx=0 tx=0).stop 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3321834638 shutdown_connections 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.38:6833/239365678,v1:172.21.15.38:6835/239365678] conn(0x55647bff1ad0 0x55647bff0700 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55647bfd7020 0x55647bfd09f0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55647bfd4230 0x55647bfd04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff47a7fc640 0 client.6960 ms_handle_reset on v2:172.21.15.38:6833/239365678 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a4df0740 1 --2- 172.21.15.38:0/3321834638 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55647bfd2f90 0x55647c03ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3321834638 >> 172.21.15.38:0/3321834638 conn(0x55647bfa9f10 msgr2=0x55647bfaa2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3321834638 shutdown_connections 2024-04-25T08:35:45.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a4df0740 1 -- 172.21.15.38:0/3321834638 wait complete. 2024-04-25T08:35:45.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a4df0740 20 client.6960 trim_cache size 0 max 16384 2024-04-25T08:35:45.943 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:45.943 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:35:45.943 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:35:45.943 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 6813, 2024-04-25T08:35:45.943 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:35:45.943 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:35:45.943 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:35:45.943 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 4.6891628040000004, 2024-04-25T08:35:45.944 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 35, 2024-04-25T08:35:45.944 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 67, 2024-04-25T08:35:45.944 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 67, 2024-04-25T08:35:45.944 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 9.7316544880000002 2024-04-25T08:35:45.944 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a20f7640 1 -- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff48806b500 msgr2=0x7ff48806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a20f7640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff48806b500 0x7ff48806d9c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0011e0 tx=0x7ff48c009040 comp rx=0 tx=0).stop 2024-04-25T08:35:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a20f7640 1 -- 172.21.15.38:0/3638627613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff49c0b6b00 msgr2=0x7ff49c0c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a20f7640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff49c0b6b00 0x7ff49c0c79d0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff490000c00 tx=0x7ff490004730 comp rx=0 tx=0).stop 2024-04-25T08:35:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a20f7640 1 -- 172.21.15.38:0/3638627613 shutdown_connections 2024-04-25T08:35:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a20f7640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff49c0b5ed0 0x7ff49c0b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a20f7640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff49c0b6b00 0x7ff49c0c79d0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a20f7640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff48806b500 0x7ff48806d9c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a20f7640 1 --2- 172.21.15.38:0/3638627613 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4840040d0 0x7ff49c14abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a20f7640 1 -- 172.21.15.38:0/3638627613 >> 172.21.15.38:0/3638627613 conn(0x7ff49c0b3190 msgr2=0x7ff49c099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a20f7640 1 -- 172.21.15.38:0/3638627613 shutdown_connections 2024-04-25T08:35:45.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:45.941+0000 7ff4a20f7640 1 -- 172.21.15.38:0/3638627613 wait complete. 2024-04-25T08:35:45.948 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:35:46.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.053+0000 7fd2e5916640 1 Processor -- start 2024-04-25T08:35:46.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.053+0000 7fd2e5916640 1 -- start start 2024-04-25T08:35:46.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.054+0000 7fd2e5916640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2e015dc10 0x7fd2e00ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.054+0000 7fd2e5916640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd2e00ced60 0x7fd2e00d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.054+0000 7fd2e5916640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e00d3e70 con 0x7fd2e00d3900 2024-04-25T08:35:46.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.054+0000 7fd2e5916640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e015dff0 con 0x7fd2e00ced60 2024-04-25T08:35:46.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.054+0000 7fd2e5916640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e00b6990 con 0x7fd2e015dc10 2024-04-25T08:35:46.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.054+0000 7fd2deffd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2e015dc10 0x7fd2e00ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.054+0000 7fd2de7fc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd2e00ced60 0x7fd2e00d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.054+0000 7fd2df7fe640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fd2e00d3900 0x7fd2e00b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:40548/0 (socket says 172.21.15.38:40548) 2024-04-25T08:35:46.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.054+0000 7fd2df7fe640 1 -- 172.21.15.38:0/60451556 learned_addr learned my addr 172.21.15.38:0/60451556 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:46.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.055+0000 7fd2deffd640 1 -- 172.21.15.38:0/60451556 >> v2:172.21.15.107:3300/0 conn(0x7fd2e00ced60 msgr2=0x7fd2e00d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:46.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.055+0000 7fd2deffd640 1 --2- 172.21.15.38:0/60451556 >> v2:172.21.15.107:3300/0 conn(0x7fd2e00ced60 0x7fd2e00d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.055+0000 7fd2deffd640 1 -- 172.21.15.38:0/60451556 >> v1:172.21.15.107:6789/0 conn(0x7fd2e00d3900 legacy=0x7fd2e00b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.055+0000 7fd2ddffb640 1 -- 172.21.15.38:0/60451556 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171402195 0 0) 0x7fd2e00d3e70 con 0x7fd2e00d3900 2024-04-25T08:35:46.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.055+0000 7fd2deffd640 1 -- 172.21.15.38:0/60451556 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e00d4150 con 0x7fd2e015dc10 2024-04-25T08:35:46.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.056+0000 7fd2deffd640 1 --2- 172.21.15.38:0/60451556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2e015dc10 0x7fd2e00ce820 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80097a0 tx=0x7fd2d802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94d547a0294ff875 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:46.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.056+0000 7fd2ddffb640 1 -- 172.21.15.38:0/60451556 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8037070 con 0x7fd2e015dc10 2024-04-25T08:35:46.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.056+0000 7fd2ddffb640 1 -- 172.21.15.38:0/60451556 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d802ae50 con 0x7fd2e015dc10 2024-04-25T08:35:46.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.056+0000 7fd2ddffb640 1 -- 172.21.15.38:0/60451556 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8032660 con 0x7fd2e015dc10 2024-04-25T08:35:46.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.056+0000 7fd2e5916640 1 -- 172.21.15.38:0/60451556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2e015dc10 msgr2=0x7fd2e00ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.056+0000 7fd2e5916640 1 --2- 172.21.15.38:0/60451556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2e015dc10 0x7fd2e00ce820 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80097a0 tx=0x7fd2d802a100 comp rx=0 tx=0).stop 2024-04-25T08:35:46.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.056+0000 7fd2e5916640 1 -- 172.21.15.38:0/60451556 shutdown_connections 2024-04-25T08:35:46.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.056+0000 7fd2e5916640 1 --2- 172.21.15.38:0/60451556 >> v2:172.21.15.107:3300/0 conn(0x7fd2e00ced60 0x7fd2e00d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.056+0000 7fd2e5916640 1 --2- 172.21.15.38:0/60451556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2e015dc10 0x7fd2e00ce820 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.056+0000 7fd2e5916640 1 -- 172.21.15.38:0/60451556 >> 172.21.15.38:0/60451556 conn(0x7fd2e0066d10 msgr2=0x7fd2e0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:46.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.056+0000 7fd2e5916640 1 -- 172.21.15.38:0/60451556 shutdown_connections 2024-04-25T08:35:46.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.056+0000 7fd2e5916640 1 -- 172.21.15.38:0/60451556 wait complete. 2024-04-25T08:35:46.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.058+0000 7fd2e5916640 1 Processor -- start 2024-04-25T08:35:46.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.058+0000 7fd2e5916640 1 -- start start 2024-04-25T08:35:46.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.058+0000 7fd2e5916640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2e00ced60 0x7fd2e015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.058+0000 7fd2e5916640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2e00d3900 0x7fd2e00c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.058+0000 7fd2e5916640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2e015dc10 0x7fd2e00acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.058+0000 7fd2e5916640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e00b6bc0 con 0x7fd2e00d3900 2024-04-25T08:35:46.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.058+0000 7fd2e5916640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e00d4710 con 0x7fd2e015dc10 2024-04-25T08:35:46.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2e5916640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e00d0d90 con 0x7fd2e00ced60 2024-04-25T08:35:46.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2de7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2e00d3900 0x7fd2e00c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2de7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2e00d3900 0x7fd2e00c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57208/0 (socket says 172.21.15.38:57208) 2024-04-25T08:35:46.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2de7fc640 1 -- 172.21.15.38:0/1343684617 learned_addr learned my addr 172.21.15.38:0/1343684617 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:46.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2deffd640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2e00ced60 0x7fd2e015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2df7fe640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2e015dc10 0x7fd2e00acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2de7fc640 1 -- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2e00ced60 msgr2=0x7fd2e015bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2de7fc640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2e00ced60 0x7fd2e015bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2de7fc640 1 -- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2e015dc10 msgr2=0x7fd2e00acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2de7fc640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2e015dc10 0x7fd2e00acaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2de7fc640 1 -- 172.21.15.38:0/1343684617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e00c8fd0 con 0x7fd2e00d3900 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2de7fc640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2e00d3900 0x7fd2e00c85c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800e930 tx=0x7fd2c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2d7fff640 1 -- 172.21.15.38:0/1343684617 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8017070 con 0x7fd2e00d3900 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2d7fff640 1 -- 172.21.15.38:0/1343684617 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c800ca00 con 0x7fd2e00d3900 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2e5916640 1 -- 172.21.15.38:0/1343684617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e00aa2f0 con 0x7fd2e00d3900 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2d7fff640 1 -- 172.21.15.38:0/1343684617 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c800cdf0 con 0x7fd2e00d3900 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.059+0000 7fd2e5916640 1 -- 172.21.15.38:0/1343684617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e00aa8b0 con 0x7fd2e00d3900 2024-04-25T08:35:46.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.060+0000 7fd2e5916640 1 -- 172.21.15.38:0/1343684617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e0068e80 con 0x7fd2e00d3900 2024-04-25T08:35:46.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.061+0000 7fd2d7fff640 1 -- 172.21.15.38:0/1343684617 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2c801a070 con 0x7fd2e00d3900 2024-04-25T08:35:46.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.061+0000 7fd2d7fff640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd2c006b500 0x7fd2c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:46.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.061+0000 7fd2d7fff640 1 -- 172.21.15.38:0/1343684617 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2c808a700 con 0x7fd2e00d3900 2024-04-25T08:35:46.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.061+0000 7fd2deffd640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd2c006b500 0x7fd2c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:46.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.062+0000 7fd2deffd640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd2c006b500 0x7fd2c006d9c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8034000 tx=0x7fd2d802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:46.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.065+0000 7fd2d7fff640 1 -- 172.21.15.38:0/1343684617 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2c8058e60 con 0x7fd2e00d3900 2024-04-25T08:35:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.285+0000 7fd2e5916640 1 -- 172.21.15.38:0/1343684617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd2e00d0b10 con 0x7fd2e00d3900 2024-04-25T08:35:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.286+0000 7fd2d7fff640 1 -- 172.21.15.38:0/1343684617 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd2c805de20 con 0x7fd2e00d3900 2024-04-25T08:35:46.281 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:35:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.287+0000 7fd2e5916640 1 -- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd2c006b500 msgr2=0x7fd2c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.287+0000 7fd2e5916640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd2c006b500 0x7fd2c006d9c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8034000 tx=0x7fd2d802e000 comp rx=0 tx=0).stop 2024-04-25T08:35:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.287+0000 7fd2e5916640 1 -- 172.21.15.38:0/1343684617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2e00d3900 msgr2=0x7fd2e00c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.287+0000 7fd2e5916640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2e00d3900 0x7fd2e00c85c0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800e930 tx=0x7fd2c800edf0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.288+0000 7fd2e5916640 1 -- 172.21.15.38:0/1343684617 shutdown_connections 2024-04-25T08:35:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.288+0000 7fd2e5916640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2e015dc10 0x7fd2e00acaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.288+0000 7fd2e5916640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2e00d3900 0x7fd2e00c85c0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.288+0000 7fd2e5916640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd2c006b500 0x7fd2c006d9c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.288+0000 7fd2e5916640 1 --2- 172.21.15.38:0/1343684617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2e00ced60 0x7fd2e015bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.288+0000 7fd2e5916640 1 -- 172.21.15.38:0/1343684617 >> 172.21.15.38:0/1343684617 conn(0x7fd2e0066d10 msgr2=0x7fd2e015e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.288+0000 7fd2e5916640 1 -- 172.21.15.38:0/1343684617 shutdown_connections 2024-04-25T08:35:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:46.288+0000 7fd2e5916640 1 -- 172.21.15.38:0/1343684617 wait complete. 2024-04-25T08:35:46.294 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:35:46.294 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:35:46.294 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:35:46.295 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:35:46.295 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:46.295 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:35:46.295 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:35:46.295 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:35:46.295 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:35:46.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:46.295 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:35:46.311 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42491sec preferred_lft 42491sec 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::440f:97ff:fe63:87b/64 scope link 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:46.312 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:35:46.313 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ea:23:76:ab:b1:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:35:46.313 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::e823:76ff:feab:b1e9/64 scope link 2024-04-25T08:35:46.313 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:46.313 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:35:46.313 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:35:46.313 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::38c7:38ff:fe2e:6c92/64 scope link 2024-04-25T08:35:46.313 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:46.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:46.313 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:35:46.374 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:35:46.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:46.376 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:35:46.432 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:35:46.432 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:35:46.432 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:35:46.432 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:35:46.432 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:35:46.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:46.433 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:35:46.433 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:35:46.433 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:35:46.433 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:35:46.433 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:35:46.433 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:35:46.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:46.518 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:35:46.533 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:35:46.533 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:35:46.533 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:35:46.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:46.533 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T08:35:46.598 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:35:46.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:35:46.598 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:46.653 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:35:46.654 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:35:46.654 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:46.709 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:35:46.774 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:35:46.830 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:35:46.831 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:35:46.832 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:35:46.833 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:35:46.833 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:35:46.833 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:35:46.833 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:35:46.833 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:35:46.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:46.833 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:35:46.886 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:35:46.886 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:35:46.929 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:35:46.953 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:35:47.009 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:35:47.010 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:35:47.011 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:35:47.011 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:35:47.011 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:35:47.011 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:35:47.011 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:35:47.011 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:35:47.011 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:35:47.011 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:35:47.011 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:35:47.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:47.012 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:35:47.013 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-04-25T08:35:47.015+0000 7fd9181b4480 -1 init, newargv = 0x55f345910310 newargc=15 2024-04-25T08:35:47.013 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[65507]: starting ceph client 2024-04-25T08:35:47.015 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[65507]: starting fuse 2024-04-25T08:35:47.029 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:35:47.030 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:35:47.030 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:35:47.030 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:35:47.030 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:35:47.030 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:35:47.030 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:35:47.030 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:47.030 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:35:47.030 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:35:47.030 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:35:47.030 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:35:47.030 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-04-25T08:35:47.031 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:35:47.149 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.65507.asok 2024-04-25T08:35:47.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:35:47.160 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.65507.asok 2024-04-25T08:35:47.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:47.161 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.65507.asok status 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:35:47.296 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "65507", 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 6969, 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 6969 2024-04-25T08:35:47.297 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1475366027 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1475366027 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.6969 192.168.0.1:0/1475366027", 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/1475366027", 2024-04-25T08:35:47.298 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:35:47.299 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 35, 2024-04-25T08:35:47.299 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 67, 2024-04-25T08:35:47.299 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:35:47.299 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:35:47.299 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:35:47.299 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:47.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:47.305 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:35:47.305 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:47.320 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:35:47.320 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:47.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:47.321 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:47.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:47.386 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-25T08:35:47.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:47.448 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-25T08:35:47.507 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-04-25T08:35:47.507 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:35:47.507 DEBUG:teuthology.orchestra.run.smithi038:> import stat 2024-04-25T08:35:47.507 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:47.507 DEBUG:teuthology.orchestra.run.smithi038:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-25T08:35:47.507 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:35:47.614 INFO:teuthology.orchestra.run.smithi038.stdout:1099511627776 2024-04-25T08:35:47.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:35:47.624 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:35:47.624 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:47.681 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.825+0000 7fc5e8a76640 1 Processor -- start 2024-04-25T08:35:47.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.825+0000 7fc5e8a76640 1 -- start start 2024-04-25T08:35:47.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e8a76640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5e40b5710 0x7fc5e4161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e8a76640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e415ddf0 0x7fc5e40b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e8a76640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e4161e00 con 0x7fc5e40b5af0 2024-04-25T08:35:47.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e8a76640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e40b9930 con 0x7fc5e40b5710 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e8a76640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e41620e0 con 0x7fc5e415ddf0 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e259c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5e40b5710 0x7fc5e4161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e2d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc5e40b5af0 0x7fc5e406bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:39904/0 (socket says 172.21.15.38:39904) 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e2d9d640 1 -- 172.21.15.38:0/1549104469 learned_addr learned my addr 172.21.15.38:0/1549104469 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e1d9b640 1 --2- 172.21.15.38:0/1549104469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e415ddf0 0x7fc5e40b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e259c640 1 -- 172.21.15.38:0/1549104469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e415ddf0 msgr2=0x7fc5e40b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e259c640 1 --2- 172.21.15.38:0/1549104469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e415ddf0 0x7fc5e40b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e259c640 1 -- 172.21.15.38:0/1549104469 >> v1:172.21.15.38:6789/0 conn(0x7fc5e40b5af0 legacy=0x7fc5e406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e259c640 1 -- 172.21.15.38:0/1549104469 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e41623c0 con 0x7fc5e40b5710 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.826+0000 7fc5e159a640 1 -- 172.21.15.38:0/1549104469 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517976764 0 0) 0x7fc5e4161e00 con 0x7fc5e40b5af0 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.827+0000 7fc5e259c640 1 --2- 172.21.15.38:0/1549104469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40b5710 0x7fc5e4161700 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0097a0 tx=0x7fc5dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aeda3f6439aa128 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.827+0000 7fc5e159a640 1 -- 172.21.15.38:0/1549104469 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc037070 con 0x7fc5e40b5710 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.827+0000 7fc5e159a640 1 -- 172.21.15.38:0/1549104469 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5dc032440 con 0x7fc5e40b5710 2024-04-25T08:35:47.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.827+0000 7fc5e159a640 1 -- 172.21.15.38:0/1549104469 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc032810 con 0x7fc5e40b5710 2024-04-25T08:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.827+0000 7fc5e8a76640 1 -- 172.21.15.38:0/1549104469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40b5710 msgr2=0x7fc5e4161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.827+0000 7fc5e8a76640 1 --2- 172.21.15.38:0/1549104469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40b5710 0x7fc5e4161700 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc0097a0 tx=0x7fc5dc02a180 comp rx=0 tx=0).stop 2024-04-25T08:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.828+0000 7fc5e8a76640 1 -- 172.21.15.38:0/1549104469 shutdown_connections 2024-04-25T08:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.828+0000 7fc5e8a76640 1 --2- 172.21.15.38:0/1549104469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e415ddf0 0x7fc5e40b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.828+0000 7fc5e8a76640 1 --2- 172.21.15.38:0/1549104469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40b5710 0x7fc5e4161700 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.828+0000 7fc5e8a76640 1 -- 172.21.15.38:0/1549104469 >> 172.21.15.38:0/1549104469 conn(0x7fc5e40d07c0 msgr2=0x7fc5e40d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.828+0000 7fc5e8a76640 1 -- 172.21.15.38:0/1549104469 shutdown_connections 2024-04-25T08:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.828+0000 7fc5e8a76640 1 -- 172.21.15.38:0/1549104469 wait complete. 2024-04-25T08:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.829+0000 7fc5e8a76640 1 Processor -- start 2024-04-25T08:35:47.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.829+0000 7fc5e8a76640 1 -- start start 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e8a76640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e40b5710 0x7fc5e40cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e8a76640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e40b5af0 0x7fc5e40ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e8a76640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e415ddf0 0x7fc5e41f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e8a76640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e40d34d0 con 0x7fc5e415ddf0 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e8a76640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e40d3350 con 0x7fc5e40b5710 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e8a76640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e40d3650 con 0x7fc5e40b5af0 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e415ddf0 0x7fc5e41f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e415ddf0 0x7fc5e41f6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57222/0 (socket says 172.21.15.38:57222) 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e2d9d640 1 -- 172.21.15.38:0/2891459743 learned_addr learned my addr 172.21.15.38:0/2891459743 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e259c640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e40b5710 0x7fc5e40cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e1d9b640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e40b5af0 0x7fc5e40ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e2d9d640 1 -- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e40b5af0 msgr2=0x7fc5e40ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e2d9d640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e40b5af0 0x7fc5e40ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e2d9d640 1 -- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e40b5710 msgr2=0x7fc5e40cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:47.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e2d9d640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e40b5710 0x7fc5e40cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e2d9d640 1 -- 172.21.15.38:0/2891459743 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e40cb650 con 0x7fc5e415ddf0 2024-04-25T08:35:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.830+0000 7fc5e2d9d640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e415ddf0 0x7fc5e41f6a00 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400c7a0 tx=0x7fc5d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.831+0000 7fc5daffd640 1 -- 172.21.15.38:0/2891459743 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4018070 con 0x7fc5e415ddf0 2024-04-25T08:35:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.831+0000 7fc5daffd640 1 -- 172.21.15.38:0/2891459743 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d400a640 con 0x7fc5e415ddf0 2024-04-25T08:35:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.831+0000 7fc5e8a76640 1 -- 172.21.15.38:0/2891459743 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e40cb930 con 0x7fc5e415ddf0 2024-04-25T08:35:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.831+0000 7fc5daffd640 1 -- 172.21.15.38:0/2891459743 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d400aa70 con 0x7fc5e415ddf0 2024-04-25T08:35:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.831+0000 7fc5e8a76640 1 -- 172.21.15.38:0/2891459743 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e40cbef0 con 0x7fc5e415ddf0 2024-04-25T08:35:47.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.832+0000 7fc5e8a76640 1 -- 172.21.15.38:0/2891459743 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e4068e80 con 0x7fc5e415ddf0 2024-04-25T08:35:47.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.832+0000 7fc5daffd640 1 -- 172.21.15.38:0/2891459743 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5d401b070 con 0x7fc5e415ddf0 2024-04-25T08:35:47.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.833+0000 7fc5daffd640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5c406b4d0 0x7fc5c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:47.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.833+0000 7fc5daffd640 1 -- 172.21.15.38:0/2891459743 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc5d4089820 con 0x7fc5e415ddf0 2024-04-25T08:35:47.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.833+0000 7fc5e259c640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5c406b4d0 0x7fc5c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:47.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.833+0000 7fc5e259c640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5c406b4d0 0x7fc5c406d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc034000 tx=0x7fc5dc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:47.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:47.837+0000 7fc5daffd640 1 -- 172.21.15.38:0/2891459743 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5d4057f30 con 0x7fc5e415ddf0 2024-04-25T08:35:48.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.019+0000 7fc5e8a76640 1 -- 172.21.15.38:0/2891459743 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5e40cc0c0 con 0x7fc5e415ddf0 2024-04-25T08:35:48.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.020+0000 7fc5daffd640 1 -- 172.21.15.38:0/2891459743 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16872 (secure 0 0 0) 0x7fc5d405bfd0 con 0x7fc5e415ddf0 2024-04-25T08:35:48.015 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:48.016 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":67,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:42.252762+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:38.884603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:39.590140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:48.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.022+0000 7fc5e8a76640 1 -- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5c406b4d0 msgr2=0x7fc5c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.022+0000 7fc5e8a76640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5c406b4d0 0x7fc5c406d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc034000 tx=0x7fc5dc02e000 comp rx=0 tx=0).stop 2024-04-25T08:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.022+0000 7fc5e8a76640 1 -- 172.21.15.38:0/2891459743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e415ddf0 msgr2=0x7fc5e41f6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.022+0000 7fc5e8a76640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e415ddf0 0x7fc5e41f6a00 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400c7a0 tx=0x7fc5d400cc60 comp rx=0 tx=0).stop 2024-04-25T08:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.022+0000 7fc5e8a76640 1 -- 172.21.15.38:0/2891459743 shutdown_connections 2024-04-25T08:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.022+0000 7fc5e8a76640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e415ddf0 0x7fc5e41f6a00 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.022+0000 7fc5e8a76640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5c406b4d0 0x7fc5c406d990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.022+0000 7fc5e8a76640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e40b5af0 0x7fc5e40ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.022+0000 7fc5e8a76640 1 --2- 172.21.15.38:0/2891459743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e40b5710 0x7fc5e40cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.022+0000 7fc5e8a76640 1 -- 172.21.15.38:0/2891459743 >> 172.21.15.38:0/2891459743 conn(0x7fc5e40d07c0 msgr2=0x7fc5e40b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.022+0000 7fc5e8a76640 1 -- 172.21.15.38:0/2891459743 shutdown_connections 2024-04-25T08:35:48.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.022+0000 7fc5e8a76640 1 -- 172.21.15.38:0/2891459743 wait complete. 2024-04-25T08:35:48.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:35:38.884603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:35:39.590140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:35:48.029 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:35:48.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.138+0000 7fa23abee640 1 Processor -- start 2024-04-25T08:35:48.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.138+0000 7fa23abee640 1 -- start start 2024-04-25T08:35:48.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.138+0000 7fa23abee640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa23415df30 0x7fa23409a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa23abee640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa234162820 con 0x7fa234162390 2024-04-25T08:35:48.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa23abee640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa23415f300 con 0x7fa23409a180 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa23abee640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23415e310 con 0x7fa23415df30 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa233fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa234162390 0x7fa234099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:39906/0 (socket says 172.21.15.38:39906) 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa233fff640 1 -- 172.21.15.38:0/1232339714 learned_addr learned my addr 172.21.15.38:0/1232339714 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa23898a640 1 --2- 172.21.15.38:0/1232339714 >> v2:172.21.15.107:3300/0 conn(0x7fa23415df30 0x7fa23409a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa2337fe640 1 -- 172.21.15.38:0/1232339714 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132394722 0 0) 0x7fa228001110 con 0x7fa234162390 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa2337fe640 1 -- 172.21.15.38:0/1232339714 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa21c003520 con 0x7fa234162390 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa2337fe640 1 -- 172.21.15.38:0/1232339714 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522038039 0 0) 0x7fa23415f300 con 0x7fa23409a180 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa2337fe640 1 -- 172.21.15.38:0/1232339714 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa228001110 con 0x7fa23409a180 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa23898a640 1 -- 172.21.15.38:0/1232339714 >> v1:172.21.15.107:6789/0 conn(0x7fa23409a180 legacy=0x7fa2340cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa23898a640 1 -- 172.21.15.38:0/1232339714 >> v1:172.21.15.38:6789/0 conn(0x7fa234162390 legacy=0x7fa234099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.139+0000 7fa23898a640 1 -- 172.21.15.38:0/1232339714 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23415e540 con 0x7fa23415df30 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.140+0000 7fa23898a640 1 --2- 172.21.15.38:0/1232339714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa23415df30 0x7fa23409a7b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa2240097d0 tx=0x7fa22402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b58dec9d9b1dc572 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.140+0000 7fa2337fe640 1 -- 172.21.15.38:0/1232339714 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224037070 con 0x7fa23415df30 2024-04-25T08:35:48.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.140+0000 7fa2337fe640 1 -- 172.21.15.38:0/1232339714 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa224032440 con 0x7fa23415df30 2024-04-25T08:35:48.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.140+0000 7fa2337fe640 1 -- 172.21.15.38:0/1232339714 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224032810 con 0x7fa23415df30 2024-04-25T08:35:48.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.141+0000 7fa23abee640 1 -- 172.21.15.38:0/1232339714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa23415df30 msgr2=0x7fa23409a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.141+0000 7fa23abee640 1 --2- 172.21.15.38:0/1232339714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa23415df30 0x7fa23409a7b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa2240097d0 tx=0x7fa22402a250 comp rx=0 tx=0).stop 2024-04-25T08:35:48.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.141+0000 7fa23abee640 1 -- 172.21.15.38:0/1232339714 shutdown_connections 2024-04-25T08:35:48.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.141+0000 7fa23abee640 1 --2- 172.21.15.38:0/1232339714 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa23415df30 0x7fa23409a7b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.141+0000 7fa23abee640 1 -- 172.21.15.38:0/1232339714 >> 172.21.15.38:0/1232339714 conn(0x7fa2340b3110 msgr2=0x7fa2340b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.141+0000 7fa23abee640 1 -- 172.21.15.38:0/1232339714 shutdown_connections 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.141+0000 7fa23abee640 1 -- 172.21.15.38:0/1232339714 wait complete. 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.142+0000 7fa23abee640 1 Processor -- start 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.142+0000 7fa23abee640 1 -- start start 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa23abee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa23409a180 0x7fa234153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa23abee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa23415df30 0x7fa2340cff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa23abee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa234162390 0x7fa2340d0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa23abee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23415ee70 con 0x7fa23415df30 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa23abee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23409aeb0 con 0x7fa234162390 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa23abee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23415eb00 con 0x7fa23409a180 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa233fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa23415df30 0x7fa2340cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa233fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa23415df30 0x7fa2340cff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57232/0 (socket says 172.21.15.38:57232) 2024-04-25T08:35:48.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa233fff640 1 -- 172.21.15.38:0/2923729946 learned_addr learned my addr 172.21.15.38:0/2923729946 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa23918b640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa234162390 0x7fa2340d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa23898a640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa23409a180 0x7fa234153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa233fff640 1 -- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa23409a180 msgr2=0x7fa234153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa233fff640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa23409a180 0x7fa234153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa233fff640 1 -- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa234162390 msgr2=0x7fa2340d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa233fff640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa234162390 0x7fa2340d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa233fff640 1 -- 172.21.15.38:0/2923729946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340c6140 con 0x7fa23415df30 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa233fff640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa23415df30 0x7fa2340cff20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fa22800c730 tx=0x7fa22800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa2317fa640 1 -- 172.21.15.38:0/2923729946 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228017070 con 0x7fa23415df30 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa2317fa640 1 -- 172.21.15.38:0/2923729946 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa228005ab0 con 0x7fa23415df30 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa2317fa640 1 -- 172.21.15.38:0/2923729946 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228007070 con 0x7fa23415df30 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.143+0000 7fa23abee640 1 -- 172.21.15.38:0/2923729946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2340c6420 con 0x7fa23415df30 2024-04-25T08:35:48.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.144+0000 7fa23abee640 1 -- 172.21.15.38:0/2923729946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2340c69e0 con 0x7fa23415df30 2024-04-25T08:35:48.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.145+0000 7fa23abee640 1 -- 172.21.15.38:0/2923729946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa234068e80 con 0x7fa23415df30 2024-04-25T08:35:48.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.145+0000 7fa2317fa640 1 -- 172.21.15.38:0/2923729946 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa22801a070 con 0x7fa23415df30 2024-04-25T08:35:48.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.146+0000 7fa2317fa640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa22006b4d0 0x7fa22006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.146+0000 7fa2317fa640 1 -- 172.21.15.38:0/2923729946 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa228089900 con 0x7fa23415df30 2024-04-25T08:35:48.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.146+0000 7fa23898a640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa22006b4d0 0x7fa22006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.146+0000 7fa23898a640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa22006b4d0 0x7fa22006d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa224034000 tx=0x7fa22402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:48.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.150+0000 7fa2317fa640 1 -- 172.21.15.38:0/2923729946 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa234162820 con 0x7fa23415df30 2024-04-25T08:35:48.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.332+0000 7fa23abee640 1 -- 172.21.15.38:0/2923729946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2340b4020 con 0x7fa23415df30 2024-04-25T08:35:48.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.332+0000 7fa2317fa640 1 -- 172.21.15.38:0/2923729946 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2280585e0 con 0x7fa23415df30 2024-04-25T08:35:48.328 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:35:48.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.334+0000 7fa23abee640 1 -- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa22006b4d0 msgr2=0x7fa22006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.334+0000 7fa23abee640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa22006b4d0 0x7fa22006d990 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa224034000 tx=0x7fa22402e000 comp rx=0 tx=0).stop 2024-04-25T08:35:48.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.334+0000 7fa23abee640 1 -- 172.21.15.38:0/2923729946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa23415df30 msgr2=0x7fa2340cff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.334+0000 7fa23abee640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa23415df30 0x7fa2340cff20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fa22800c730 tx=0x7fa22800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.334+0000 7fa23abee640 1 -- 172.21.15.38:0/2923729946 shutdown_connections 2024-04-25T08:35:48.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.334+0000 7fa23abee640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa234162390 0x7fa2340d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.334+0000 7fa23abee640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa23415df30 0x7fa2340cff20 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.334+0000 7fa23abee640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa22006b4d0 0x7fa22006d990 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.334+0000 7fa23abee640 1 --2- 172.21.15.38:0/2923729946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa23409a180 0x7fa234153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.334+0000 7fa23abee640 1 -- 172.21.15.38:0/2923729946 >> 172.21.15.38:0/2923729946 conn(0x7fa2340b3110 msgr2=0x7fa23406c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:48.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.334+0000 7fa23abee640 1 -- 172.21.15.38:0/2923729946 shutdown_connections 2024-04-25T08:35:48.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.334+0000 7fa23abee640 1 -- 172.21.15.38:0/2923729946 wait complete. 2024-04-25T08:35:48.341 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:35:48.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.448+0000 7fb54cce4640 1 Processor -- start 2024-04-25T08:35:48.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.448+0000 7fb54cce4640 1 -- start start 2024-04-25T08:35:48.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.449+0000 7fb54cce4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb5480d0320 0x7fb5480d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.449+0000 7fb54cce4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb5480d0700 0x7fb5480d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.449+0000 7fb54cce4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5480d2330 con 0x7fb5480d1e90 2024-04-25T08:35:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.449+0000 7fb54cce4640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480d2610 con 0x7fb5480d0700 2024-04-25T08:35:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.449+0000 7fb54cce4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480d28f0 con 0x7fb5480d0320 2024-04-25T08:35:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.449+0000 7fb545d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb5480d0700 0x7fb5480d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.449+0000 7fb545d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb5480d0700 0x7fb5480d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57234/0 (socket says 172.21.15.38:57234) 2024-04-25T08:35:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.449+0000 7fb545d9b640 1 -- 172.21.15.38:0/1073346282 learned_addr learned my addr 172.21.15.38:0/1073346282 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.450+0000 7fb54659c640 1 --2- 172.21.15.38:0/1073346282 >> v2:172.21.15.107:3300/0 conn(0x7fb5480d0320 0x7fb5480d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.450+0000 7fb545d9b640 1 -- 172.21.15.38:0/1073346282 >> v2:172.21.15.107:3300/0 conn(0x7fb5480d0320 msgr2=0x7fb5480d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.450+0000 7fb545d9b640 1 --2- 172.21.15.38:0/1073346282 >> v2:172.21.15.107:3300/0 conn(0x7fb5480d0320 0x7fb5480d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.450+0000 7fb545d9b640 1 -- 172.21.15.38:0/1073346282 >> v1:172.21.15.107:6789/0 conn(0x7fb5480d1e90 legacy=0x7fb5480b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.450+0000 7fb545d9b640 1 -- 172.21.15.38:0/1073346282 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5480d2bd0 con 0x7fb5480d0700 2024-04-25T08:35:48.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.450+0000 7fb545d9b640 1 --2- 172.21.15.38:0/1073346282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5480d0700 0x7fb5480d1950 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb548069100 tx=0x7fb53c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d4d57516c0edf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.451+0000 7fb54559a640 1 -- 172.21.15.38:0/1073346282 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c037070 con 0x7fb5480d0700 2024-04-25T08:35:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.451+0000 7fb54559a640 1 -- 172.21.15.38:0/1073346282 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c02ae50 con 0x7fb5480d0700 2024-04-25T08:35:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.451+0000 7fb54559a640 1 -- 172.21.15.38:0/1073346282 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c032860 con 0x7fb5480d0700 2024-04-25T08:35:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.451+0000 7fb54cce4640 1 -- 172.21.15.38:0/1073346282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5480d0700 msgr2=0x7fb5480d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.451+0000 7fb54cce4640 1 --2- 172.21.15.38:0/1073346282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5480d0700 0x7fb5480d1950 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb548069100 tx=0x7fb53c02a300 comp rx=0 tx=0).stop 2024-04-25T08:35:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.451+0000 7fb54cce4640 1 -- 172.21.15.38:0/1073346282 shutdown_connections 2024-04-25T08:35:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.451+0000 7fb54cce4640 1 --2- 172.21.15.38:0/1073346282 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5480d0700 0x7fb5480d1950 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.451+0000 7fb54cce4640 1 --2- 172.21.15.38:0/1073346282 >> v2:172.21.15.107:3300/0 conn(0x7fb5480d0320 0x7fb5480d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.451+0000 7fb54cce4640 1 -- 172.21.15.38:0/1073346282 >> 172.21.15.38:0/1073346282 conn(0x7fb5480668f0 msgr2=0x7fb548066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.451+0000 7fb54cce4640 1 -- 172.21.15.38:0/1073346282 shutdown_connections 2024-04-25T08:35:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.451+0000 7fb54cce4640 1 -- 172.21.15.38:0/1073346282 wait complete. 2024-04-25T08:35:48.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.453+0000 7fb54cce4640 1 Processor -- start 2024-04-25T08:35:48.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.453+0000 7fb54cce4640 1 -- start start 2024-04-25T08:35:48.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.453+0000 7fb54cce4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5480d0320 0x7fb5480aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.453+0000 7fb54cce4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5480d0700 0x7fb5480b5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.453+0000 7fb54cce4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5480d1e90 0x7fb5480abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.453+0000 7fb54cce4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480d0b40 con 0x7fb5480d0320 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.453+0000 7fb54cce4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548161210 con 0x7fb5480d0700 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.453+0000 7fb54cce4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480aa7e0 con 0x7fb5480d1e90 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb54659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5480d0320 0x7fb5480aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb54659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5480d0320 0x7fb5480aa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57236/0 (socket says 172.21.15.38:57236) 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb54659c640 1 -- 172.21.15.38:0/522506345 learned_addr learned my addr 172.21.15.38:0/522506345 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb545d9b640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5480d0700 0x7fb5480b5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb546d9d640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5480d1e90 0x7fb5480abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb54659c640 1 -- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5480d1e90 msgr2=0x7fb5480abce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb54659c640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5480d1e90 0x7fb5480abce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb54659c640 1 -- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5480d0700 msgr2=0x7fb5480b5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb54659c640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5480d0700 0x7fb5480b5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb54659c640 1 -- 172.21.15.38:0/522506345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5480aaa10 con 0x7fb5480d0320 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb545d9b640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5480d0700 0x7fb5480b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:48.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb546d9d640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5480d1e90 0x7fb5480abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:48.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb54659c640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5480d0320 0x7fb5480aa2a0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb54000e9b0 tx=0x7fb54000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:48.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb536ffd640 1 -- 172.21.15.38:0/522506345 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540017070 con 0x7fb5480d0320 2024-04-25T08:35:48.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb536ffd640 1 -- 172.21.15.38:0/522506345 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54000c9a0 con 0x7fb5480d0320 2024-04-25T08:35:48.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb536ffd640 1 -- 172.21.15.38:0/522506345 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54000cd90 con 0x7fb5480d0320 2024-04-25T08:35:48.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb54cce4640 1 -- 172.21.15.38:0/522506345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5480aacf0 con 0x7fb5480d0320 2024-04-25T08:35:48.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.454+0000 7fb54cce4640 1 -- 172.21.15.38:0/522506345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5480ab2b0 con 0x7fb5480d0320 2024-04-25T08:35:48.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.455+0000 7fb54cce4640 1 -- 172.21.15.38:0/522506345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb548068e80 con 0x7fb5480d0320 2024-04-25T08:35:48.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.456+0000 7fb536ffd640 1 -- 172.21.15.38:0/522506345 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb54001a070 con 0x7fb5480d0320 2024-04-25T08:35:48.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.456+0000 7fb536ffd640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb52806b4d0 0x7fb52806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.456+0000 7fb536ffd640 1 -- 172.21.15.38:0/522506345 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb54008a770 con 0x7fb5480d0320 2024-04-25T08:35:48.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.457+0000 7fb545d9b640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb52806b4d0 0x7fb52806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.457+0000 7fb545d9b640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb52806b4d0 0x7fb52806d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb53c034000 tx=0x7fb53c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:48.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.460+0000 7fb536ffd640 1 -- 172.21.15.38:0/522506345 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb540059460 con 0x7fb5480d0320 2024-04-25T08:35:48.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.640+0000 7fb54cce4640 1 -- 172.21.15.38:0/522506345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb548067dd0 con 0x7fb5480d0320 2024-04-25T08:35:48.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.640+0000 7fb536ffd640 1 -- 172.21.15.38:0/522506345 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb540058d60 con 0x7fb5480d0320 2024-04-25T08:35:48.636 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:35:48.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.642+0000 7fb54cce4640 1 -- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb52806b4d0 msgr2=0x7fb52806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.642+0000 7fb54cce4640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb52806b4d0 0x7fb52806d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb53c034000 tx=0x7fb53c02e000 comp rx=0 tx=0).stop 2024-04-25T08:35:48.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.642+0000 7fb54cce4640 1 -- 172.21.15.38:0/522506345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5480d0320 msgr2=0x7fb5480aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.642+0000 7fb54cce4640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5480d0320 0x7fb5480aa2a0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb54000e9b0 tx=0x7fb54000ee70 comp rx=0 tx=0).stop 2024-04-25T08:35:48.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.642+0000 7fb54cce4640 1 -- 172.21.15.38:0/522506345 shutdown_connections 2024-04-25T08:35:48.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.642+0000 7fb54cce4640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb52806b4d0 0x7fb52806d990 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.642+0000 7fb54cce4640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5480d1e90 0x7fb5480abce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.642+0000 7fb54cce4640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5480d0700 0x7fb5480b5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.642+0000 7fb54cce4640 1 --2- 172.21.15.38:0/522506345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5480d0320 0x7fb5480aa2a0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.642+0000 7fb54cce4640 1 -- 172.21.15.38:0/522506345 >> 172.21.15.38:0/522506345 conn(0x7fb5480668f0 msgr2=0x7fb5480cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:48.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.642+0000 7fb54cce4640 1 -- 172.21.15.38:0/522506345 shutdown_connections 2024-04-25T08:35:48.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.642+0000 7fb54cce4640 1 -- 172.21.15.38:0/522506345 wait complete. 2024-04-25T08:35:48.649 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:35:48.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.757+0000 7f2bcefe0640 1 Processor -- start 2024-04-25T08:35:48.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.757+0000 7f2bcefe0640 1 -- start start 2024-04-25T08:35:48.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.758+0000 7f2bcefe0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bc80b1d60 0x7f2bc80cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.758+0000 7f2bcefe0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2bc8161e20 0x7f2bc815d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.758+0000 7f2bcefe0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc815ef40 con 0x7f2bc80cff10 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.758+0000 7f2bcefe0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc806abe0 con 0x7f2bc8161e20 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.758+0000 7f2bcefe0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc806b480 con 0x7f2bc80b1d60 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.758+0000 7f2bccd7c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bc80b1d60 0x7f2bc80cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.758+0000 7f2bcd57d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2bc8161e20 0x7f2bc815d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.758+0000 7f2bc7fff640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f2bc80cff10 0x7f2bc8161710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:40580/0 (socket says 172.21.15.38:40580) 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.758+0000 7f2bcd57d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2bc8161e20 0x7f2bc815d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:49848/0 (socket says 172.21.15.38:49848) 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.759+0000 7f2bc7fff640 1 -- 172.21.15.38:0/853328519 learned_addr learned my addr 172.21.15.38:0/853328519 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.759+0000 7f2bc77fe640 1 -- 172.21.15.38:0/853328519 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373802442 0 0) 0x7f2bc815ef40 con 0x7f2bc80cff10 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.759+0000 7f2bc77fe640 1 -- 172.21.15.38:0/853328519 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb4003520 con 0x7f2bc80cff10 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.759+0000 7f2bccd7c640 1 -- 172.21.15.38:0/853328519 >> v2:172.21.15.107:3300/0 conn(0x7f2bc8161e20 msgr2=0x7f2bc815d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.759+0000 7f2bccd7c640 1 --2- 172.21.15.38:0/853328519 >> v2:172.21.15.107:3300/0 conn(0x7f2bc8161e20 0x7f2bc815d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.759+0000 7f2bccd7c640 1 -- 172.21.15.38:0/853328519 >> v1:172.21.15.107:6789/0 conn(0x7f2bc80cff10 legacy=0x7f2bc8161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.759+0000 7f2bccd7c640 1 -- 172.21.15.38:0/853328519 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc806ae30 con 0x7f2bc80b1d60 2024-04-25T08:35:48.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.759+0000 7f2bcd57d640 1 --2- 172.21.15.38:0/853328519 >> v2:172.21.15.107:3300/0 conn(0x7f2bc8161e20 0x7f2bc815d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:48.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.760+0000 7f2bccd7c640 1 --2- 172.21.15.38:0/853328519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bc80b1d60 0x7f2bc80cf9d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009d90 tx=0x7f2bc002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3b10fae8c379973 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:48.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.760+0000 7f2bc77fe640 1 -- 172.21.15.38:0/853328519 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0007740 con 0x7f2bc80b1d60 2024-04-25T08:35:48.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.760+0000 7f2bc77fe640 1 -- 172.21.15.38:0/853328519 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc00078e0 con 0x7f2bc80b1d60 2024-04-25T08:35:48.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.760+0000 7f2bc77fe640 1 -- 172.21.15.38:0/853328519 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0007cb0 con 0x7f2bc80b1d60 2024-04-25T08:35:48.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.760+0000 7f2bcefe0640 1 -- 172.21.15.38:0/853328519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bc80b1d60 msgr2=0x7f2bc80cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.760+0000 7f2bcefe0640 1 --2- 172.21.15.38:0/853328519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bc80b1d60 0x7f2bc80cf9d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0009d90 tx=0x7f2bc002a660 comp rx=0 tx=0).stop 2024-04-25T08:35:48.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.760+0000 7f2bcefe0640 1 -- 172.21.15.38:0/853328519 shutdown_connections 2024-04-25T08:35:48.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.760+0000 7f2bcefe0640 1 --2- 172.21.15.38:0/853328519 >> v2:172.21.15.107:3300/0 conn(0x7f2bc8161e20 0x7f2bc815d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.760+0000 7f2bcefe0640 1 --2- 172.21.15.38:0/853328519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bc80b1d60 0x7f2bc80cf9d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.760+0000 7f2bcefe0640 1 -- 172.21.15.38:0/853328519 >> 172.21.15.38:0/853328519 conn(0x7f2bc80d06b0 msgr2=0x7f2bc80d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.760+0000 7f2bcefe0640 1 -- 172.21.15.38:0/853328519 shutdown_connections 2024-04-25T08:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.760+0000 7f2bcefe0640 1 -- 172.21.15.38:0/853328519 wait complete. 2024-04-25T08:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.762+0000 7f2bcefe0640 1 Processor -- start 2024-04-25T08:35:48.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.762+0000 7f2bcefe0640 1 -- start start 2024-04-25T08:35:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.762+0000 7f2bcefe0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc80b1d60 0x7f2bc81539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.762+0000 7f2bcefe0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bc80cff10 0x7f2bc80c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.762+0000 7f2bcefe0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bc8161e20 0x7f2bc80c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.762+0000 7f2bcefe0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc806b910 con 0x7f2bc8161e20 2024-04-25T08:35:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.762+0000 7f2bcefe0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc80d0370 con 0x7f2bc80b1d60 2024-04-25T08:35:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.762+0000 7f2bcefe0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc806ba90 con 0x7f2bc80cff10 2024-04-25T08:35:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bcd57d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bc8161e20 0x7f2bc80c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bcd57d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bc8161e20 0x7f2bc80c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57244/0 (socket says 172.21.15.38:57244) 2024-04-25T08:35:48.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bcd57d640 1 -- 172.21.15.38:0/3681509273 learned_addr learned my addr 172.21.15.38:0/3681509273 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bccd7c640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc80b1d60 0x7f2bc81539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bc7fff640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bc80cff10 0x7f2bc80c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bcd57d640 1 -- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bc80cff10 msgr2=0x7f2bc80c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bcd57d640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bc80cff10 0x7f2bc80c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bcd57d640 1 -- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc80b1d60 msgr2=0x7f2bc81539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bcd57d640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc80b1d60 0x7f2bc81539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bcd57d640 1 -- 172.21.15.38:0/3681509273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc8153f20 con 0x7f2bc8161e20 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bcd57d640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bc8161e20 0x7f2bc80c89d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e930 tx=0x7f2bb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bc57fa640 1 -- 172.21.15.38:0/3681509273 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0017070 con 0x7f2bc8161e20 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bc57fa640 1 -- 172.21.15.38:0/3681509273 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb000ca00 con 0x7f2bc8161e20 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bcefe0640 1 -- 172.21.15.38:0/3681509273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc80c5ff0 con 0x7f2bc8161e20 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bc57fa640 1 -- 172.21.15.38:0/3681509273 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000cdd0 con 0x7f2bc8161e20 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.763+0000 7f2bcefe0640 1 -- 172.21.15.38:0/3681509273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc80c81d0 con 0x7f2bc8161e20 2024-04-25T08:35:48.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.764+0000 7f2bcefe0640 1 -- 172.21.15.38:0/3681509273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc80ce230 con 0x7f2bc8161e20 2024-04-25T08:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.765+0000 7f2bc57fa640 1 -- 172.21.15.38:0/3681509273 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bb0013470 con 0x7f2bc8161e20 2024-04-25T08:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.765+0000 7f2bc57fa640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ba806b4d0 0x7f2ba806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:48.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.765+0000 7f2bc57fa640 1 -- 172.21.15.38:0/3681509273 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2bb0089eb0 con 0x7f2bc8161e20 2024-04-25T08:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.766+0000 7f2bccd7c640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ba806b4d0 0x7f2ba806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:48.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.766+0000 7f2bccd7c640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ba806b4d0 0x7f2ba806d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0034000 tx=0x7f2bc002f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:48.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.770+0000 7f2bc57fa640 1 -- 172.21.15.38:0/3681509273 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2bc80ce230 con 0x7f2bc8161e20 2024-04-25T08:35:48.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.948+0000 7f2bcefe0640 1 -- 172.21.15.38:0/3681509273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2bc806c9e0 con 0x7f2bc8161e20 2024-04-25T08:35:48.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.948+0000 7f2bc57fa640 1 -- 172.21.15.38:0/3681509273 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bb0058590 con 0x7f2bc8161e20 2024-04-25T08:35:48.944 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:35:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.950+0000 7f2bcefe0640 1 -- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ba806b4d0 msgr2=0x7f2ba806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.950+0000 7f2bcefe0640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ba806b4d0 0x7f2ba806d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2bc0034000 tx=0x7f2bc002f000 comp rx=0 tx=0).stop 2024-04-25T08:35:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.950+0000 7f2bcefe0640 1 -- 172.21.15.38:0/3681509273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bc8161e20 msgr2=0x7f2bc80c89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.950+0000 7f2bcefe0640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bc8161e20 0x7f2bc80c89d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000e930 tx=0x7f2bb000edf0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.950+0000 7f2bcefe0640 1 -- 172.21.15.38:0/3681509273 shutdown_connections 2024-04-25T08:35:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.950+0000 7f2bcefe0640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2bc8161e20 0x7f2bc80c89d0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.950+0000 7f2bcefe0640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ba806b4d0 0x7f2ba806d990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.950+0000 7f2bcefe0640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2bc80cff10 0x7f2bc80c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.950+0000 7f2bcefe0640 1 --2- 172.21.15.38:0/3681509273 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2bc80b1d60 0x7f2bc81539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:48.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.950+0000 7f2bcefe0640 1 -- 172.21.15.38:0/3681509273 >> 172.21.15.38:0/3681509273 conn(0x7f2bc80d06b0 msgr2=0x7f2bc815fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:48.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.950+0000 7f2bcefe0640 1 -- 172.21.15.38:0/3681509273 shutdown_connections 2024-04-25T08:35:48.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:48.950+0000 7f2bcefe0640 1 -- 172.21.15.38:0/3681509273 wait complete. 2024-04-25T08:35:48.957 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:35:49.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.063+0000 7f654f1c8640 1 Processor -- start 2024-04-25T08:35:49.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.063+0000 7f654f1c8640 1 -- start start 2024-04-25T08:35:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.064+0000 7f654f1c8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f654815cf20 0x7f65480cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.064+0000 7f654f1c8640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f654815d380 con 0x7f65480cc350 2024-04-25T08:35:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.064+0000 7f654f1c8640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6548072240 con 0x7f6548145080 2024-04-25T08:35:49.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.064+0000 7f654f1c8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6548145460 con 0x7f654815cf20 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.064+0000 7f654d765640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f65480cc350 0x7f654815f630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32814/0 (socket says 172.21.15.38:32814) 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.064+0000 7f654d765640 1 -- 172.21.15.38:0/2193448492 learned_addr learned my addr 172.21.15.38:0/2193448492 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.064+0000 7f6547fff640 1 --2- 172.21.15.38:0/2193448492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f654815cf20 0x7f65480cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.064+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 962569051 0 0) 0x7f654815d380 con 0x7f65480cc350 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.064+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6530003520 con 0x7f65480cc350 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602798791 0 0) 0x7f6548072240 con 0x7f6548145080 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f654815d380 con 0x7f6548145080 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1314925420 0 0) 0x7f6530003520 con 0x7f65480cc350 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f654815cf20 msgr2=0x7f65480cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 --2- 172.21.15.38:0/2193448492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f654815cf20 0x7f65480cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 >> v1:172.21.15.107:6789/0 conn(0x7f6548145080 legacy=0x7f6548071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654815c130 con 0x7f65480cc350 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f6538001670 con 0x7f65480cc350 2024-04-25T08:35:49.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 >> v1:172.21.15.38:6789/0 conn(0x7f65480cc350 legacy=0x7f654815f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 --2- 172.21.15.38:0/2193448492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f654815cf20 0x7f6530003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 --2- 172.21.15.38:0/2193448492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6530004120 0x7f6530004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 --2- 172.21.15.38:0/2193448492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6530004d40 0x7f65300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653000bad0 con 0x7f6530004d40 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653000bc80 con 0x7f6530004120 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653000be80 con 0x7f654815cf20 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f654d765640 1 --2- 172.21.15.38:0/2193448492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6530004d40 0x7f65300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f654cf64640 1 --2- 172.21.15.38:0/2193448492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f654815cf20 0x7f6530003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f6547fff640 1 --2- 172.21.15.38:0/2193448492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6530004120 0x7f6530004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f654d765640 1 -- 172.21.15.38:0/2193448492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f654815cf20 msgr2=0x7f6530003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f654d765640 1 --2- 172.21.15.38:0/2193448492 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f654815cf20 0x7f6530003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f654d765640 1 -- 172.21.15.38:0/2193448492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6530004120 msgr2=0x7f6530004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f654d765640 1 --2- 172.21.15.38:0/2193448492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6530004120 0x7f6530004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.065+0000 7f654d765640 1 -- 172.21.15.38:0/2193448492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6530003520 con 0x7f6530004d40 2024-04-25T08:35:49.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.066+0000 7f654d765640 1 --2- 172.21.15.38:0/2193448492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6530004d40 0x7f65300051c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6538007fc0 tx=0x7f6530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae05855b21005ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:49.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.066+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538007be0 con 0x7f6530004d40 2024-04-25T08:35:49.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.066+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6538005af0 con 0x7f6530004d40 2024-04-25T08:35:49.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.066+0000 7f65477fe640 1 -- 172.21.15.38:0/2193448492 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538007070 con 0x7f6530004d40 2024-04-25T08:35:49.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.067+0000 7f654f1c8640 1 -- 172.21.15.38:0/2193448492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6530004d40 msgr2=0x7f65300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.067+0000 7f654f1c8640 1 --2- 172.21.15.38:0/2193448492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6530004d40 0x7f65300051c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6538007fc0 tx=0x7f6530005830 comp rx=0 tx=0).stop 2024-04-25T08:35:49.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.067+0000 7f654d765640 1 -- 172.21.15.38:0/2193448492 reap_dead start 2024-04-25T08:35:49.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.067+0000 7f654f1c8640 1 -- 172.21.15.38:0/2193448492 shutdown_connections 2024-04-25T08:35:49.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.067+0000 7f654f1c8640 1 -- 172.21.15.38:0/2193448492 >> 172.21.15.38:0/2193448492 conn(0x7f654806abe0 msgr2=0x7f654806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:49.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.067+0000 7f654f1c8640 1 -- 172.21.15.38:0/2193448492 shutdown_connections 2024-04-25T08:35:49.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.067+0000 7f654f1c8640 1 -- 172.21.15.38:0/2193448492 wait complete. 2024-04-25T08:35:49.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.068+0000 7f654f1c8640 1 Processor -- start 2024-04-25T08:35:49.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.068+0000 7f654f1c8640 1 -- start start 2024-04-25T08:35:49.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654f1c8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6548142df0 0x7f65481e5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654f1c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65481431d0 0x7f654815ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654f1c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f654815b040 0x7f654812e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654f1c8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654815c9d0 con 0x7f654815b040 2024-04-25T08:35:49.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654f1c8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f654815c6f0 con 0x7f65481431d0 2024-04-25T08:35:49.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654f1c8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654815ccb0 con 0x7f6548142df0 2024-04-25T08:35:49.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654d765640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f654815b040 0x7f654812e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654d765640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f654815b040 0x7f654812e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33940/0 (socket says 172.21.15.38:33940) 2024-04-25T08:35:49.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654d765640 1 -- 172.21.15.38:0/2682322603 learned_addr learned my addr 172.21.15.38:0/2682322603 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f6547fff640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65481431d0 0x7f654815ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654cf64640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6548142df0 0x7f65481e5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654d765640 1 -- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6548142df0 msgr2=0x7f65481e5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654d765640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6548142df0 0x7f65481e5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654d765640 1 -- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65481431d0 msgr2=0x7f654815ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654d765640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65481431d0 0x7f654815ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654d765640 1 -- 172.21.15.38:0/2682322603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654815b4c0 con 0x7f654815b040 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654cf64640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6548142df0 0x7f65481e5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.069+0000 7f654d765640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f654815b040 0x7f654812e960 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f653802acd0 tx=0x7f654815c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.070+0000 7f65457fa640 1 -- 172.21.15.38:0/2682322603 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538037040 con 0x7f654815b040 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.070+0000 7f65457fa640 1 -- 172.21.15.38:0/2682322603 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6538041070 con 0x7f654815b040 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.070+0000 7f65457fa640 1 -- 172.21.15.38:0/2682322603 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538007be0 con 0x7f654815b040 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.070+0000 7f654f1c8640 1 -- 172.21.15.38:0/2682322603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f654812c8d0 con 0x7f654815b040 2024-04-25T08:35:49.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.070+0000 7f654f1c8640 1 -- 172.21.15.38:0/2682322603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65480a8b90 con 0x7f654815b040 2024-04-25T08:35:49.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.071+0000 7f654f1c8640 1 -- 172.21.15.38:0/2682322603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f654804ef90 con 0x7f654815b040 2024-04-25T08:35:49.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.071+0000 7f65457fa640 1 -- 172.21.15.38:0/2682322603 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6538007290 con 0x7f654815b040 2024-04-25T08:35:49.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.071+0000 7f65457fa640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f653406b4d0 0x7f653406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.072+0000 7f65457fa640 1 -- 172.21.15.38:0/2682322603 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65380aa000 con 0x7f654815b040 2024-04-25T08:35:49.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.072+0000 7f654cf64640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f653406b4d0 0x7f653406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.072+0000 7f654cf64640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f653406b4d0 0x7f653406d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6540000f80 tx=0x7f6540009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:49.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.076+0000 7f65457fa640 1 -- 172.21.15.38:0/2682322603 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6538078db0 con 0x7f654815b040 2024-04-25T08:35:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.273+0000 7f654f1c8640 1 -- 172.21.15.38:0/2682322603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6548143bc0 con 0x7f654815b040 2024-04-25T08:35:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.273+0000 7f65457fa640 1 -- 172.21.15.38:0/2682322603 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) v1 ==== 80+0+0 (secure 0 0 0) 0x7f65380786b0 con 0x7f654815b040 2024-04-25T08:35:49.269 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:35:49.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.276+0000 7f654f1c8640 1 -- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f653406b4d0 msgr2=0x7f653406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.276+0000 7f654f1c8640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f653406b4d0 0x7f653406d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6540000f80 tx=0x7f6540009040 comp rx=0 tx=0).stop 2024-04-25T08:35:49.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.276+0000 7f654f1c8640 1 -- 172.21.15.38:0/2682322603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f654815b040 msgr2=0x7f654812e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.276+0000 7f654f1c8640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f654815b040 0x7f654812e960 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f653802acd0 tx=0x7f654815c300 comp rx=0 tx=0).stop 2024-04-25T08:35:49.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.276+0000 7f654f1c8640 1 -- 172.21.15.38:0/2682322603 shutdown_connections 2024-04-25T08:35:49.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.276+0000 7f654f1c8640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f654815b040 0x7f654812e960 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.276+0000 7f654f1c8640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f65481431d0 0x7f654815ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.276+0000 7f654f1c8640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f653406b4d0 0x7f653406d990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.276+0000 7f654f1c8640 1 --2- 172.21.15.38:0/2682322603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6548142df0 0x7f65481e5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.276+0000 7f654f1c8640 1 -- 172.21.15.38:0/2682322603 >> 172.21.15.38:0/2682322603 conn(0x7f654806abe0 msgr2=0x7f65480b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:49.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.276+0000 7f654f1c8640 1 -- 172.21.15.38:0/2682322603 shutdown_connections 2024-04-25T08:35:49.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.276+0000 7f654f1c8640 1 -- 172.21.15.38:0/2682322603 wait complete. 2024-04-25T08:35:49.283 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:35:49.283 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:35:49.300 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:35:49.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.445+0000 7f56896ab640 1 Processor -- start 2024-04-25T08:35:49.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.445+0000 7f56896ab640 1 -- start start 2024-04-25T08:35:49.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.446+0000 7f56896ab640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 0x7f56840b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.446+0000 7f56896ab640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f56840b4e60 0x7f56840cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.446+0000 7f56896ab640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56840ce360 con 0x7f56840cdb70 2024-04-25T08:35:49.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.446+0000 7f56896ab640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56840b52c0 con 0x7f56840b4e60 2024-04-25T08:35:49.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.446+0000 7f56896ab640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56840ce640 con 0x7f5684143f40 2024-04-25T08:35:49.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.447+0000 7f5682ffd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 0x7f56840b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.447+0000 7f56827fc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f56840b4e60 0x7f56840cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.447+0000 7f5682ffd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 0x7f56840b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:33216/0 (socket says 172.21.15.38:33216) 2024-04-25T08:35:49.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.447+0000 7f5682ffd640 1 -- 172.21.15.38:0/3369236110 learned_addr learned my addr 172.21.15.38:0/3369236110 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:49.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.447+0000 7f56827fc640 1 -- 172.21.15.38:0/3369236110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 msgr2=0x7f56840b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:49.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.447+0000 7f56827fc640 1 --2- 172.21.15.38:0/3369236110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 0x7f56840b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.447+0000 7f56827fc640 1 -- 172.21.15.38:0/3369236110 >> v1:172.21.15.107:6789/0 conn(0x7f56840cdb70 legacy=0x7f568415e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.447+0000 7f56827fc640 1 -- 172.21.15.38:0/3369236110 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56840ce920 con 0x7f56840b4e60 2024-04-25T08:35:49.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.447+0000 7f5682ffd640 1 --2- 172.21.15.38:0/3369236110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 0x7f56840b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:49.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.448+0000 7f56827fc640 1 --2- 172.21.15.38:0/3369236110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56840b4e60 0x7f56840cd630 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f566c0097a0 tx=0x7f566c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3065e6db2297813f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:49.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.448+0000 7f5681ffb640 1 -- 172.21.15.38:0/3369236110 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c037070 con 0x7f56840b4e60 2024-04-25T08:35:49.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.448+0000 7f5681ffb640 1 -- 172.21.15.38:0/3369236110 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c032440 con 0x7f56840b4e60 2024-04-25T08:35:49.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.448+0000 7f5681ffb640 1 -- 172.21.15.38:0/3369236110 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c032810 con 0x7f56840b4e60 2024-04-25T08:35:49.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.448+0000 7f56896ab640 1 -- 172.21.15.38:0/3369236110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56840b4e60 msgr2=0x7f56840cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.448+0000 7f56896ab640 1 --2- 172.21.15.38:0/3369236110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56840b4e60 0x7f56840cd630 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f566c0097a0 tx=0x7f566c02a180 comp rx=0 tx=0).stop 2024-04-25T08:35:49.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.448+0000 7f56896ab640 1 -- 172.21.15.38:0/3369236110 shutdown_connections 2024-04-25T08:35:49.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.448+0000 7f56896ab640 1 --2- 172.21.15.38:0/3369236110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56840b4e60 0x7f56840cd630 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.448+0000 7f56896ab640 1 --2- 172.21.15.38:0/3369236110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 0x7f56840b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.448+0000 7f56896ab640 1 -- 172.21.15.38:0/3369236110 >> 172.21.15.38:0/3369236110 conn(0x7f568406b210 msgr2=0x7f568406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:49.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.449+0000 7f56896ab640 1 -- 172.21.15.38:0/3369236110 shutdown_connections 2024-04-25T08:35:49.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.449+0000 7f56896ab640 1 -- 172.21.15.38:0/3369236110 wait complete. 2024-04-25T08:35:49.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.450+0000 7f56896ab640 1 Processor -- start 2024-04-25T08:35:49.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.450+0000 7f56896ab640 1 -- start start 2024-04-25T08:35:49.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f56896ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b4e60 0x7f56841f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f56896ab640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56840cdb70 0x7f56841ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f56896ab640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 0x7f5684156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f56896ab640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56840cdfc0 con 0x7f56840b4e60 2024-04-25T08:35:49.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f56896ab640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56840ceee0 con 0x7f56840cdb70 2024-04-25T08:35:49.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f56896ab640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56840ce140 con 0x7f5684143f40 2024-04-25T08:35:49.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f5682ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b4e60 0x7f56841f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f5682ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b4e60 0x7f56841f1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33954/0 (socket says 172.21.15.38:33954) 2024-04-25T08:35:49.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f5682ffd640 1 -- 172.21.15.38:0/388424915 learned_addr learned my addr 172.21.15.38:0/388424915 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f56827fc640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56840cdb70 0x7f56841ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f56837fe640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 0x7f5684156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f5682ffd640 1 -- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 msgr2=0x7f5684156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f5682ffd640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 0x7f5684156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f5682ffd640 1 -- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56840cdb70 msgr2=0x7f56841ee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f5682ffd640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56840cdb70 0x7f56841ee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f5682ffd640 1 -- 172.21.15.38:0/388424915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684155db0 con 0x7f56840b4e60 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f56837fe640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 0x7f5684156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f5682ffd640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b4e60 0x7f56841f1ef0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f567c00e9e0 tx=0x7f567c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f567b7fe640 1 -- 172.21.15.38:0/388424915 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c017070 con 0x7f56840b4e60 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f567b7fe640 1 -- 172.21.15.38:0/388424915 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c00c9d0 con 0x7f56840b4e60 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f56896ab640 1 -- 172.21.15.38:0/388424915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5684156090 con 0x7f56840b4e60 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f567b7fe640 1 -- 172.21.15.38:0/388424915 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c00cd80 con 0x7f56840b4e60 2024-04-25T08:35:49.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.451+0000 7f56896ab640 1 -- 172.21.15.38:0/388424915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5684156650 con 0x7f56840b4e60 2024-04-25T08:35:49.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.452+0000 7f56896ab640 1 -- 172.21.15.38:0/388424915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5684068e80 con 0x7f56840b4e60 2024-04-25T08:35:49.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.453+0000 7f567b7fe640 1 -- 172.21.15.38:0/388424915 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f567c01a070 con 0x7f56840b4e60 2024-04-25T08:35:49.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.453+0000 7f567b7fe640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f566406b4d0 0x7f566406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.453+0000 7f567b7fe640 1 -- 172.21.15.38:0/388424915 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f567c08a520 con 0x7f56840b4e60 2024-04-25T08:35:49.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.454+0000 7f56827fc640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f566406b4d0 0x7f566406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.454+0000 7f56827fc640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f566406b4d0 0x7f566406d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f566c034000 tx=0x7f566c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:49.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.457+0000 7f567b7fe640 1 -- 172.21.15.38:0/388424915 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f567c058bc0 con 0x7f56840b4e60 2024-04-25T08:35:49.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.636+0000 7f56896ab640 1 -- 172.21.15.38:0/388424915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f568413f620 con 0x7f56840b4e60 2024-04-25T08:35:49.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.638+0000 7f567b7fe640 1 -- 172.21.15.38:0/388424915 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f567c05cc60 con 0x7f56840b4e60 2024-04-25T08:35:49.633 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:49.634 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":67,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:42.252762+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:38.884603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:39.590140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:35:49.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.639+0000 7f56896ab640 1 -- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f566406b4d0 msgr2=0x7f566406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.639+0000 7f56896ab640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f566406b4d0 0x7f566406d990 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f566c034000 tx=0x7f566c02e000 comp rx=0 tx=0).stop 2024-04-25T08:35:49.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.639+0000 7f56896ab640 1 -- 172.21.15.38:0/388424915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b4e60 msgr2=0x7f56841f1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.639+0000 7f56896ab640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b4e60 0x7f56841f1ef0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f567c00e9e0 tx=0x7f567c00eea0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.639+0000 7f56896ab640 1 -- 172.21.15.38:0/388424915 shutdown_connections 2024-04-25T08:35:49.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.640+0000 7f56896ab640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f566406b4d0 0x7f566406d990 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.640+0000 7f56896ab640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5684143f40 0x7f5684156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.640+0000 7f56896ab640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56840cdb70 0x7f56841ee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.640+0000 7f56896ab640 1 --2- 172.21.15.38:0/388424915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56840b4e60 0x7f56841f1ef0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.640+0000 7f56896ab640 1 -- 172.21.15.38:0/388424915 >> 172.21.15.38:0/388424915 conn(0x7f568406b210 msgr2=0x7f568413dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:49.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.640+0000 7f56896ab640 1 -- 172.21.15.38:0/388424915 shutdown_connections 2024-04-25T08:35:49.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.640+0000 7f56896ab640 1 -- 172.21.15.38:0/388424915 wait complete. 2024-04-25T08:35:49.646 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:35:38.884603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:35:39.590140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:35:49.646 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.755+0000 7f87f1e5a640 1 Processor -- start 2024-04-25T08:35:49.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.755+0000 7f87f1e5a640 1 -- start start 2024-04-25T08:35:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87f1e5a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f87ec0d0320 0x7f87ec0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87f1e5a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f87ec0d0700 0x7f87ec0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87f1e5a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec0d2330 con 0x7f87ec0d1e90 2024-04-25T08:35:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87f1e5a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ec0d2610 con 0x7f87ec0d0320 2024-04-25T08:35:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87f1e5a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ec0d28f0 con 0x7f87ec0d0700 2024-04-25T08:35:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87eb7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f87ec0d0320 0x7f87ec0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87eb7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f87ec0d0320 0x7f87ec0d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33956/0 (socket says 172.21.15.38:33956) 2024-04-25T08:35:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87eb7fe640 1 -- 172.21.15.38:0/2791386783 learned_addr learned my addr 172.21.15.38:0/2791386783 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87eaffd640 1 --2- 172.21.15.38:0/2791386783 >> v2:172.21.15.107:3300/0 conn(0x7f87ec0d0700 0x7f87ec0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87eb7fe640 1 -- 172.21.15.38:0/2791386783 >> v2:172.21.15.107:3300/0 conn(0x7f87ec0d0700 msgr2=0x7f87ec0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87eb7fe640 1 --2- 172.21.15.38:0/2791386783 >> v2:172.21.15.107:3300/0 conn(0x7f87ec0d0700 0x7f87ec0d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87eb7fe640 1 -- 172.21.15.38:0/2791386783 >> v1:172.21.15.107:6789/0 conn(0x7f87ec0d1e90 legacy=0x7f87ec0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87eb7fe640 1 -- 172.21.15.38:0/2791386783 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec0d2bd0 con 0x7f87ec0d0320 2024-04-25T08:35:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.756+0000 7f87eaffd640 1 --2- 172.21.15.38:0/2791386783 >> v2:172.21.15.107:3300/0 conn(0x7f87ec0d0700 0x7f87ec0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.757+0000 7f87eb7fe640 1 --2- 172.21.15.38:0/2791386783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87ec0d0320 0x7f87ec0d33d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0b6410 tx=0x7f87dc02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abee14788766ae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.757+0000 7f87ea7fc640 1 -- 172.21.15.38:0/2791386783 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc037070 con 0x7f87ec0d0320 2024-04-25T08:35:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.757+0000 7f87ea7fc640 1 -- 172.21.15.38:0/2791386783 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc032440 con 0x7f87ec0d0320 2024-04-25T08:35:49.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.757+0000 7f87ea7fc640 1 -- 172.21.15.38:0/2791386783 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc032810 con 0x7f87ec0d0320 2024-04-25T08:35:49.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.758+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2791386783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87ec0d0320 msgr2=0x7f87ec0d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.758+0000 7f87f1e5a640 1 --2- 172.21.15.38:0/2791386783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87ec0d0320 0x7f87ec0d33d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0b6410 tx=0x7f87dc02a2c0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.758+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2791386783 shutdown_connections 2024-04-25T08:35:49.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.758+0000 7f87f1e5a640 1 --2- 172.21.15.38:0/2791386783 >> v2:172.21.15.107:3300/0 conn(0x7f87ec0d0700 0x7f87ec0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.758+0000 7f87f1e5a640 1 --2- 172.21.15.38:0/2791386783 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87ec0d0320 0x7f87ec0d33d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.758+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2791386783 >> 172.21.15.38:0/2791386783 conn(0x7f87ec0668f0 msgr2=0x7f87ec066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:49.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.758+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2791386783 shutdown_connections 2024-04-25T08:35:49.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.758+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2791386783 wait complete. 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.759+0000 7f87f1e5a640 1 Processor -- start 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.759+0000 7f87f1e5a640 1 -- start start 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87f1e5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87ec0d0320 0x7f87ec0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87f1e5a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87ec0d0700 0x7f87ec0cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87f1e5a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87ec0d1e90 0x7f87ec0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87f1e5a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0d0b40 con 0x7f87ec0d0320 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87f1e5a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec161210 con 0x7f87ec0d0700 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87f1e5a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0b62c0 con 0x7f87ec0d1e90 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87eb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87ec0d0320 0x7f87ec0c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87eb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87ec0d0320 0x7f87ec0c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33970/0 (socket says 172.21.15.38:33970) 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87eb7fe640 1 -- 172.21.15.38:0/2588990115 learned_addr learned my addr 172.21.15.38:0/2588990115 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87eaffd640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87ec0d0700 0x7f87ec0cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87ebfff640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87ec0d1e90 0x7f87ec0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87eb7fe640 1 -- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87ec0d1e90 msgr2=0x7f87ec0cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87eb7fe640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87ec0d1e90 0x7f87ec0cca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87eb7fe640 1 -- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87ec0d0700 msgr2=0x7f87ec0cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87eb7fe640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87ec0d0700 0x7f87ec0cc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87eb7fe640 1 -- 172.21.15.38:0/2588990115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec145480 con 0x7f87ec0d0320 2024-04-25T08:35:49.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.760+0000 7f87eaffd640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87ec0d0700 0x7f87ec0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:49.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.761+0000 7f87eb7fe640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87ec0d0320 0x7f87ec0c9420 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000c00 tx=0x7f87dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:49.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.761+0000 7f87d3fff640 1 -- 172.21.15.38:0/2588990115 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc041020 con 0x7f87ec0d0320 2024-04-25T08:35:49.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.761+0000 7f87d3fff640 1 -- 172.21.15.38:0/2588990115 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc030970 con 0x7f87ec0d0320 2024-04-25T08:35:49.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.761+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2588990115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec0b5fe0 con 0x7f87ec0d0320 2024-04-25T08:35:49.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.761+0000 7f87d3fff640 1 -- 172.21.15.38:0/2588990115 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc037040 con 0x7f87ec0d0320 2024-04-25T08:35:49.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.761+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2588990115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec145a00 con 0x7f87ec0d0320 2024-04-25T08:35:49.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.762+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2588990115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec0675e0 con 0x7f87ec0d0320 2024-04-25T08:35:49.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.762+0000 7f87d3fff640 1 -- 172.21.15.38:0/2588990115 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87dc044020 con 0x7f87ec0d0320 2024-04-25T08:35:49.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.763+0000 7f87d3fff640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f87d806b4d0 0x7f87d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:49.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.763+0000 7f87d3fff640 1 -- 172.21.15.38:0/2588990115 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87dc0aad90 con 0x7f87ec0d0320 2024-04-25T08:35:49.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.763+0000 7f87eaffd640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f87d806b4d0 0x7f87d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:49.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.764+0000 7f87eaffd640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f87d806b4d0 0x7f87d806d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f87e0006fd0 tx=0x7f87e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:49.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.767+0000 7f87d3fff640 1 -- 172.21.15.38:0/2588990115 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87ec0675e0 con 0x7f87ec0d0320 2024-04-25T08:35:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.951+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2588990115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87ec0d1000 con 0x7f87ec0d0320 2024-04-25T08:35:49.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.952+0000 7f87d3fff640 1 -- 172.21.15.38:0/2588990115 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f87ec0d1000 con 0x7f87ec0d0320 2024-04-25T08:35:49.947 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:35:49.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.954+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f87d806b4d0 msgr2=0x7f87d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.954+0000 7f87f1e5a640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f87d806b4d0 0x7f87d806d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f87e0006fd0 tx=0x7f87e0008040 comp rx=0 tx=0).stop 2024-04-25T08:35:49.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.954+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2588990115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87ec0d0320 msgr2=0x7f87ec0c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.954+0000 7f87f1e5a640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87ec0d0320 0x7f87ec0c9420 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000c00 tx=0x7f87dc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.954+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2588990115 shutdown_connections 2024-04-25T08:35:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.954+0000 7f87f1e5a640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f87d806b4d0 0x7f87d806d990 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.954+0000 7f87f1e5a640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87ec0d1e90 0x7f87ec0cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.954+0000 7f87f1e5a640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87ec0d0700 0x7f87ec0cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.954+0000 7f87f1e5a640 1 --2- 172.21.15.38:0/2588990115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87ec0d0320 0x7f87ec0c9420 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.954+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2588990115 >> 172.21.15.38:0/2588990115 conn(0x7f87ec0668f0 msgr2=0x7f87ec0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.954+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2588990115 shutdown_connections 2024-04-25T08:35:49.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:49.954+0000 7f87f1e5a640 1 -- 172.21.15.38:0/2588990115 wait complete. 2024-04-25T08:35:49.960 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:35:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.069+0000 7fafb62e1640 1 Processor -- start 2024-04-25T08:35:50.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.069+0000 7fafb62e1640 1 -- start start 2024-04-25T08:35:50.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.069+0000 7fafb62e1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fafb015b990 0x7fafb00cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.069+0000 7fafb62e1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb015fd00 con 0x7fafb00d1d20 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.069+0000 7fafb62e1640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb015fa30 con 0x7fafb015f5f0 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.069+0000 7fafb62e1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb0160580 con 0x7fafb015b990 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.070+0000 7fafb487e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fafb00d1d20 0x7fafb00b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32826/0 (socket says 172.21.15.38:32826) 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.070+0000 7fafb487e640 1 -- 172.21.15.38:0/1396312398 learned_addr learned my addr 172.21.15.38:0/1396312398 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.070+0000 7fafaffff640 1 --2- 172.21.15.38:0/1396312398 >> v2:172.21.15.107:3300/0 conn(0x7fafb015b990 0x7fafb00cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.070+0000 7fafaeffd640 1 -- 172.21.15.38:0/1396312398 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98449315 0 0) 0x7fafb015fd00 con 0x7fafb00d1d20 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.070+0000 7fafaeffd640 1 -- 172.21.15.38:0/1396312398 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf98003520 con 0x7fafb00d1d20 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.070+0000 7fafaffff640 1 -- 172.21.15.38:0/1396312398 >> v1:172.21.15.107:6789/0 conn(0x7fafb015f5f0 legacy=0x7fafb00d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.070+0000 7fafaffff640 1 -- 172.21.15.38:0/1396312398 >> v1:172.21.15.38:6789/0 conn(0x7fafb00d1d20 legacy=0x7fafb00b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.070+0000 7fafaeffd640 1 -- 172.21.15.38:0/1396312398 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 478730157 0 0) 0x7faf98003520 con 0x7fafb00d1d20 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.070+0000 7fafaffff640 1 -- 172.21.15.38:0/1396312398 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb015ffc0 con 0x7fafb015b990 2024-04-25T08:35:50.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.071+0000 7fafaffff640 1 --2- 172.21.15.38:0/1396312398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb015b990 0x7fafb00cf580 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fafa00097a0 tx=0x7fafa002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=609db711a2468ae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.071+0000 7fafaeffd640 1 -- 172.21.15.38:0/1396312398 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0037070 con 0x7fafb015b990 2024-04-25T08:35:50.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.071+0000 7fafaeffd640 1 -- 172.21.15.38:0/1396312398 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa002abc0 con 0x7fafb015b990 2024-04-25T08:35:50.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.071+0000 7fafaeffd640 1 -- 172.21.15.38:0/1396312398 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa80019f0 con 0x7fafb015b990 2024-04-25T08:35:50.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.072+0000 7fafb62e1640 1 -- 172.21.15.38:0/1396312398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb015b990 msgr2=0x7fafb00cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.072+0000 7fafb62e1640 1 --2- 172.21.15.38:0/1396312398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb015b990 0x7fafb00cf580 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fafa00097a0 tx=0x7fafa002a130 comp rx=0 tx=0).stop 2024-04-25T08:35:50.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.072+0000 7fafb62e1640 1 -- 172.21.15.38:0/1396312398 shutdown_connections 2024-04-25T08:35:50.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.072+0000 7fafb62e1640 1 --2- 172.21.15.38:0/1396312398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb015b990 0x7fafb00cf580 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.072+0000 7fafb62e1640 1 -- 172.21.15.38:0/1396312398 >> 172.21.15.38:0/1396312398 conn(0x7fafb008b000 msgr2=0x7fafb008b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:50.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.072+0000 7fafb62e1640 1 -- 172.21.15.38:0/1396312398 shutdown_connections 2024-04-25T08:35:50.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.072+0000 7fafb62e1640 1 -- 172.21.15.38:0/1396312398 wait complete. 2024-04-25T08:35:50.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.073+0000 7fafb62e1640 1 Processor -- start 2024-04-25T08:35:50.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.073+0000 7fafb62e1640 1 -- start start 2024-04-25T08:35:50.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafb62e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb00d1d20 0x7fafb006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafb62e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb015b990 0x7fafb015e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafb62e1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafb015f5f0 0x7fafb0159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafb62e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb015bd70 con 0x7fafb00d1d20 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafb62e1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb00b1ac0 con 0x7fafb015b990 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafb62e1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb015bef0 con 0x7fafb015f5f0 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafaffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb00d1d20 0x7fafb006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafaffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb00d1d20 0x7fafb006bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:33976/0 (socket says 172.21.15.38:33976) 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafaffff640 1 -- 172.21.15.38:0/3859532694 learned_addr learned my addr 172.21.15.38:0/3859532694 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafaf7fe640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb015b990 0x7fafb015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafb487e640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafb015f5f0 0x7fafb0159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafaffff640 1 -- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafb015f5f0 msgr2=0x7fafb0159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafaffff640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafb015f5f0 0x7fafb0159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafaffff640 1 -- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb015b990 msgr2=0x7fafb015e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafaffff640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb015b990 0x7fafb015e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafaffff640 1 -- 172.21.15.38:0/3859532694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb015eec0 con 0x7fafb00d1d20 2024-04-25T08:35:50.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.074+0000 7fafaffff640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb00d1d20 0x7fafb006bda0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fafa0035040 tx=0x7fafa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.075+0000 7fafacff9640 1 -- 172.21.15.38:0/3859532694 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0041070 con 0x7fafb00d1d20 2024-04-25T08:35:50.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.075+0000 7fafacff9640 1 -- 172.21.15.38:0/3859532694 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa003c660 con 0x7fafb00d1d20 2024-04-25T08:35:50.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.075+0000 7fafb62e1640 1 -- 172.21.15.38:0/3859532694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb006c410 con 0x7fafb00d1d20 2024-04-25T08:35:50.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.075+0000 7fafacff9640 1 -- 172.21.15.38:0/3859532694 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0037040 con 0x7fafb00d1d20 2024-04-25T08:35:50.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.075+0000 7fafb62e1640 1 -- 172.21.15.38:0/3859532694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb006c9d0 con 0x7fafb00d1d20 2024-04-25T08:35:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.076+0000 7fafb62e1640 1 -- 172.21.15.38:0/3859532694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb00cba70 con 0x7fafb00d1d20 2024-04-25T08:35:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.076+0000 7fafacff9640 1 -- 172.21.15.38:0/3859532694 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fafa0044070 con 0x7fafb00d1d20 2024-04-25T08:35:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.077+0000 7fafacff9640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faf9c06b4d0 0x7faf9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.077+0000 7fafacff9640 1 -- 172.21.15.38:0/3859532694 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fafa00aaa70 con 0x7fafb00d1d20 2024-04-25T08:35:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.077+0000 7fafaf7fe640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faf9c06b4d0 0x7faf9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.077+0000 7fafaf7fe640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faf9c06b4d0 0x7faf9c06d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fafb00cb7c0 tx=0x7fafa4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.081+0000 7fafacff9640 1 -- 172.21.15.38:0/3859532694 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fafb00cba70 con 0x7fafb00d1d20 2024-04-25T08:35:50.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.260+0000 7fafb62e1640 1 -- 172.21.15.38:0/3859532694 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fafb00b1220 con 0x7fafb00d1d20 2024-04-25T08:35:50.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.261+0000 7fafacff9640 1 -- 172.21.15.38:0/3859532694 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7fafa00791a0 con 0x7fafb00d1d20 2024-04-25T08:35:50.257 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:35:50.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.263+0000 7fafb62e1640 1 -- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faf9c06b4d0 msgr2=0x7faf9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.263+0000 7fafb62e1640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faf9c06b4d0 0x7faf9c06d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fafb00cb7c0 tx=0x7fafa4009040 comp rx=0 tx=0).stop 2024-04-25T08:35:50.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.263+0000 7fafb62e1640 1 -- 172.21.15.38:0/3859532694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb00d1d20 msgr2=0x7fafb006bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.263+0000 7fafb62e1640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb00d1d20 0x7fafb006bda0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fafa0035040 tx=0x7fafa00046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.263+0000 7fafb62e1640 1 -- 172.21.15.38:0/3859532694 shutdown_connections 2024-04-25T08:35:50.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.263+0000 7fafb62e1640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faf9c06b4d0 0x7faf9c06d990 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.263+0000 7fafb62e1640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fafb015f5f0 0x7fafb0159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.263+0000 7fafb62e1640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fafb015b990 0x7fafb015e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.263+0000 7fafb62e1640 1 --2- 172.21.15.38:0/3859532694 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fafb00d1d20 0x7fafb006bda0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.263+0000 7fafb62e1640 1 -- 172.21.15.38:0/3859532694 >> 172.21.15.38:0/3859532694 conn(0x7fafb008b000 msgr2=0x7fafb015aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:50.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.263+0000 7fafb62e1640 1 -- 172.21.15.38:0/3859532694 shutdown_connections 2024-04-25T08:35:50.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.263+0000 7fafb62e1640 1 -- 172.21.15.38:0/3859532694 wait complete. 2024-04-25T08:35:50.269 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:35:50.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.377+0000 7f32286b3640 1 Processor -- start 2024-04-25T08:35:50.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.378+0000 7f32286b3640 1 -- start start 2024-04-25T08:35:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.378+0000 7f32286b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f322006abe0 0x7f322006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.378+0000 7f32286b3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f32200cc2c0 0x7f32200cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.378+0000 7f32286b3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322006b500 con 0x7f322015f490 2024-04-25T08:35:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.378+0000 7f32286b3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32200ccc80 con 0x7f32200cc2c0 2024-04-25T08:35:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.378+0000 7f32286b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322015c1b0 con 0x7f322006abe0 2024-04-25T08:35:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.379+0000 7f3225c4e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f32200cc2c0 0x7f32200cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.379+0000 7f3226c50640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f322015f490 0x7f322015f920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32842/0 (socket says 172.21.15.38:32842) 2024-04-25T08:35:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.379+0000 7f3226c50640 1 -- 172.21.15.38:0/38434581 learned_addr learned my addr 172.21.15.38:0/38434581 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.379+0000 7f3225c4e640 1 -- 172.21.15.38:0/38434581 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f322006abe0 msgr2=0x7f322006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.379+0000 7f322644f640 1 --2- 172.21.15.38:0/38434581 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f322006abe0 0x7f322006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.379+0000 7f3225c4e640 1 --2- 172.21.15.38:0/38434581 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f322006abe0 0x7f322006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.379+0000 7f322544d640 1 -- 172.21.15.38:0/38434581 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933532656 0 0) 0x7f322006b500 con 0x7f322015f490 2024-04-25T08:35:50.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.379+0000 7f3225c4e640 1 -- 172.21.15.38:0/38434581 >> v1:172.21.15.38:6789/0 conn(0x7f322015f490 legacy=0x7f322015f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.379+0000 7f3225c4e640 1 -- 172.21.15.38:0/38434581 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32200ccf30 con 0x7f32200cc2c0 2024-04-25T08:35:50.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.379+0000 7f322644f640 1 --2- 172.21.15.38:0/38434581 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f322006abe0 0x7f322006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:50.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.380+0000 7f3225c4e640 1 --2- 172.21.15.38:0/38434581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32200cc2c0 0x7f32200cc740 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f32100097a0 tx=0x7f321002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d662ebf29779ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.380+0000 7f322544d640 1 -- 172.21.15.38:0/38434581 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210037070 con 0x7f32200cc2c0 2024-04-25T08:35:50.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.380+0000 7f322544d640 1 -- 172.21.15.38:0/38434581 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321002ae50 con 0x7f32200cc2c0 2024-04-25T08:35:50.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.380+0000 7f322544d640 1 -- 172.21.15.38:0/38434581 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3210032660 con 0x7f32200cc2c0 2024-04-25T08:35:50.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.380+0000 7f32286b3640 1 -- 172.21.15.38:0/38434581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32200cc2c0 msgr2=0x7f32200cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.380+0000 7f32286b3640 1 --2- 172.21.15.38:0/38434581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32200cc2c0 0x7f32200cc740 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f32100097a0 tx=0x7f321002a100 comp rx=0 tx=0).stop 2024-04-25T08:35:50.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.380+0000 7f32286b3640 1 -- 172.21.15.38:0/38434581 shutdown_connections 2024-04-25T08:35:50.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.380+0000 7f32286b3640 1 --2- 172.21.15.38:0/38434581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32200cc2c0 0x7f32200cc740 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.380+0000 7f32286b3640 1 --2- 172.21.15.38:0/38434581 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f322006abe0 0x7f322006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.380+0000 7f32286b3640 1 -- 172.21.15.38:0/38434581 >> 172.21.15.38:0/38434581 conn(0x7f32200d06b0 msgr2=0x7f32200d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:50.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.380+0000 7f32286b3640 1 -- 172.21.15.38:0/38434581 shutdown_connections 2024-04-25T08:35:50.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.380+0000 7f32286b3640 1 -- 172.21.15.38:0/38434581 wait complete. 2024-04-25T08:35:50.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.382+0000 7f32286b3640 1 Processor -- start 2024-04-25T08:35:50.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.382+0000 7f32286b3640 1 -- start start 2024-04-25T08:35:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.382+0000 7f32286b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f322006abe0 0x7f3220157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.382+0000 7f32286b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32200cc2c0 0x7f32200b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.382+0000 7f32286b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f322015f490 0x7f3220156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.382+0000 7f32286b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200cd7d0 con 0x7f322006abe0 2024-04-25T08:35:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.382+0000 7f32286b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32200cd4f0 con 0x7f322015f490 2024-04-25T08:35:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.382+0000 7f32286b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3220159790 con 0x7f32200cc2c0 2024-04-25T08:35:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.382+0000 7f322644f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f322006abe0 0x7f3220157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.382+0000 7f322644f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f322006abe0 0x7f3220157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34006/0 (socket says 172.21.15.38:34006) 2024-04-25T08:35:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.382+0000 7f322644f640 1 -- 172.21.15.38:0/2538131319 learned_addr learned my addr 172.21.15.38:0/2538131319 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f3225c4e640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32200cc2c0 0x7f32200b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f3226c50640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f322015f490 0x7f3220156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f322644f640 1 -- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32200cc2c0 msgr2=0x7f32200b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f322644f640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32200cc2c0 0x7f32200b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f322644f640 1 -- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f322015f490 msgr2=0x7f3220156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f322644f640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f322015f490 0x7f3220156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f322644f640 1 -- 172.21.15.38:0/2538131319 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32200c5ab0 con 0x7f322006abe0 2024-04-25T08:35:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f3225c4e640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32200cc2c0 0x7f32200b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f3226c50640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f322015f490 0x7f3220156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f322644f640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f322006abe0 0x7f3220157970 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f321c00c7f0 tx=0x7f321c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f3216ffd640 1 -- 172.21.15.38:0/2538131319 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c015070 con 0x7f322006abe0 2024-04-25T08:35:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f3216ffd640 1 -- 172.21.15.38:0/2538131319 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f321c007a30 con 0x7f322006abe0 2024-04-25T08:35:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f32286b3640 1 -- 172.21.15.38:0/2538131319 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32200c5d90 con 0x7f322006abe0 2024-04-25T08:35:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f3216ffd640 1 -- 172.21.15.38:0/2538131319 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c007e00 con 0x7f322006abe0 2024-04-25T08:35:50.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.383+0000 7f32286b3640 1 -- 172.21.15.38:0/2538131319 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32200c6350 con 0x7f322006abe0 2024-04-25T08:35:50.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.384+0000 7f32286b3640 1 -- 172.21.15.38:0/2538131319 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3220068e80 con 0x7f322006abe0 2024-04-25T08:35:50.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.385+0000 7f3216ffd640 1 -- 172.21.15.38:0/2538131319 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f321c018070 con 0x7f322006abe0 2024-04-25T08:35:50.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.385+0000 7f3216ffd640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f320406b4d0 0x7f320406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.385+0000 7f3216ffd640 1 -- 172.21.15.38:0/2538131319 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f321c08a270 con 0x7f322006abe0 2024-04-25T08:35:50.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.385+0000 7f3225c4e640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f320406b4d0 0x7f320406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.386+0000 7f3225c4e640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f320406b4d0 0x7f320406d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3210034000 tx=0x7f321002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.389+0000 7f3216ffd640 1 -- 172.21.15.38:0/2538131319 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f321c058ff0 con 0x7f322006abe0 2024-04-25T08:35:50.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.568+0000 7f32286b3640 1 -- 172.21.15.38:0/2538131319 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3220158480 con 0x7f322006abe0 2024-04-25T08:35:50.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.568+0000 7f3216ffd640 1 -- 172.21.15.38:0/2538131319 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7f321c0588f0 con 0x7f322006abe0 2024-04-25T08:35:50.564 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:35:50.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.570+0000 7f32286b3640 1 -- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f320406b4d0 msgr2=0x7f320406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.570+0000 7f32286b3640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f320406b4d0 0x7f320406d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3210034000 tx=0x7f321002e000 comp rx=0 tx=0).stop 2024-04-25T08:35:50.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.570+0000 7f32286b3640 1 -- 172.21.15.38:0/2538131319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f322006abe0 msgr2=0x7f3220157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.570+0000 7f32286b3640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f322006abe0 0x7f3220157970 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f321c00c7f0 tx=0x7f321c00cbf0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.570+0000 7f32286b3640 1 -- 172.21.15.38:0/2538131319 shutdown_connections 2024-04-25T08:35:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.570+0000 7f32286b3640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f322015f490 0x7f3220156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.570+0000 7f32286b3640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f320406b4d0 0x7f320406d990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.570+0000 7f32286b3640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f32200cc2c0 0x7f32200b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.570+0000 7f32286b3640 1 --2- 172.21.15.38:0/2538131319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f322006abe0 0x7f3220157970 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.570+0000 7f32286b3640 1 -- 172.21.15.38:0/2538131319 >> 172.21.15.38:0/2538131319 conn(0x7f32200d06b0 msgr2=0x7f3220159a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:50.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.570+0000 7f32286b3640 1 -- 172.21.15.38:0/2538131319 shutdown_connections 2024-04-25T08:35:50.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.570+0000 7f32286b3640 1 -- 172.21.15.38:0/2538131319 wait complete. 2024-04-25T08:35:50.577 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:35:50.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.684+0000 7f388ee5d640 1 Processor -- start 2024-04-25T08:35:50.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.684+0000 7f388ee5d640 1 -- start start 2024-04-25T08:35:50.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.685+0000 7f388ee5d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f388815d900 0x7f38880b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.685+0000 7f388ee5d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f38880ce500 0x7f38880ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.685+0000 7f388ee5d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38880d1ce0 con 0x7f38880d1870 2024-04-25T08:35:50.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.685+0000 7f388ee5d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38880b7b20 con 0x7f388815d900 2024-04-25T08:35:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.685+0000 7f388ee5d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388815dce0 con 0x7f38880ce500 2024-04-25T08:35:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.685+0000 7f388cbf9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f388815d900 0x7f38880b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.685+0000 7f388cbf9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f388815d900 0x7f38880b68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34016/0 (socket says 172.21.15.38:34016) 2024-04-25T08:35:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.685+0000 7f388cbf9640 1 -- 172.21.15.38:0/937127610 learned_addr learned my addr 172.21.15.38:0/937127610 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.685+0000 7f388cbf9640 1 -- 172.21.15.38:0/937127610 >> v2:172.21.15.107:3300/0 conn(0x7f38880ce500 msgr2=0x7f38880ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.685+0000 7f388cbf9640 1 --2- 172.21.15.38:0/937127610 >> v2:172.21.15.107:3300/0 conn(0x7f38880ce500 0x7f38880ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.685+0000 7f388cbf9640 1 -- 172.21.15.38:0/937127610 >> v1:172.21.15.107:6789/0 conn(0x7f38880d1870 legacy=0x7f38880b9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.685+0000 7f388cbf9640 1 -- 172.21.15.38:0/937127610 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38880b6f10 con 0x7f388815d900 2024-04-25T08:35:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.686+0000 7f388cbf9640 1 --2- 172.21.15.38:0/937127610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388815d900 0x7f38880b68a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f38880d1630 tx=0x7f387802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d891757f0949073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.686+0000 7f38877fe640 1 -- 172.21.15.38:0/937127610 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878037070 con 0x7f388815d900 2024-04-25T08:35:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.686+0000 7f38877fe640 1 -- 172.21.15.38:0/937127610 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3878032440 con 0x7f388815d900 2024-04-25T08:35:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.686+0000 7f38877fe640 1 -- 172.21.15.38:0/937127610 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878032810 con 0x7f388815d900 2024-04-25T08:35:50.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.687+0000 7f388ee5d640 1 -- 172.21.15.38:0/937127610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388815d900 msgr2=0x7f38880b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.687+0000 7f388ee5d640 1 --2- 172.21.15.38:0/937127610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388815d900 0x7f38880b68a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f38880d1630 tx=0x7f387802a230 comp rx=0 tx=0).stop 2024-04-25T08:35:50.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.687+0000 7f388ee5d640 1 -- 172.21.15.38:0/937127610 shutdown_connections 2024-04-25T08:35:50.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.687+0000 7f388ee5d640 1 --2- 172.21.15.38:0/937127610 >> v2:172.21.15.107:3300/0 conn(0x7f38880ce500 0x7f38880ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.687+0000 7f388ee5d640 1 --2- 172.21.15.38:0/937127610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388815d900 0x7f38880b68a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.687+0000 7f388ee5d640 1 -- 172.21.15.38:0/937127610 >> 172.21.15.38:0/937127610 conn(0x7f3888066d10 msgr2=0x7f3888067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.687+0000 7f388ee5d640 1 -- 172.21.15.38:0/937127610 shutdown_connections 2024-04-25T08:35:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.687+0000 7f388ee5d640 1 -- 172.21.15.38:0/937127610 wait complete. 2024-04-25T08:35:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.688+0000 7f388ee5d640 1 Processor -- start 2024-04-25T08:35:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.688+0000 7f388ee5d640 1 -- start start 2024-04-25T08:35:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.689+0000 7f388ee5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38880ce500 0x7f3888151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.689+0000 7f388ee5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38880d1870 0x7f38880ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.689+0000 7f388ee5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388815d900 0x7f38880cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.689+0000 7f388ee5d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38880b77b0 con 0x7f388815d900 2024-04-25T08:35:50.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.689+0000 7f388ee5d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38880b74d0 con 0x7f38880d1870 2024-04-25T08:35:50.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.689+0000 7f388ee5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38880cee90 con 0x7f38880ce500 2024-04-25T08:35:50.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.689+0000 7f388d3fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388815d900 0x7f38880cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.689+0000 7f388d3fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388815d900 0x7f38880cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34020/0 (socket says 172.21.15.38:34020) 2024-04-25T08:35:50.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.689+0000 7f388d3fa640 1 -- 172.21.15.38:0/2535535669 learned_addr learned my addr 172.21.15.38:0/2535535669 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:50.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.689+0000 7f3887fff640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38880d1870 0x7f38880ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f388cbf9640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38880ce500 0x7f3888151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f388d3fa640 1 -- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38880ce500 msgr2=0x7f3888151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f388d3fa640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38880ce500 0x7f3888151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f388d3fa640 1 -- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38880d1870 msgr2=0x7f38880ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f388d3fa640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38880d1870 0x7f38880ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f388d3fa640 1 -- 172.21.15.38:0/2535535669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3888151f50 con 0x7f388815d900 2024-04-25T08:35:50.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f388cbf9640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38880ce500 0x7f3888151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:50.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f388d3fa640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388815d900 0x7f38880cae40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f388000b9a0 tx=0x7f388000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f38857fa640 1 -- 172.21.15.38:0/2535535669 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880019070 con 0x7f388815d900 2024-04-25T08:35:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f38857fa640 1 -- 172.21.15.38:0/2535535669 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388000cbc0 con 0x7f388815d900 2024-04-25T08:35:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f38857fa640 1 -- 172.21.15.38:0/2535535669 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3880014440 con 0x7f388815d900 2024-04-25T08:35:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f388ee5d640 1 -- 172.21.15.38:0/2535535669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3888152230 con 0x7f388815d900 2024-04-25T08:35:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.690+0000 7f388ee5d640 1 -- 172.21.15.38:0/2535535669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38881527f0 con 0x7f388815d900 2024-04-25T08:35:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.691+0000 7f388ee5d640 1 -- 172.21.15.38:0/2535535669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38880cf380 con 0x7f388815d900 2024-04-25T08:35:50.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.691+0000 7f38857fa640 1 -- 172.21.15.38:0/2535535669 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f388001c070 con 0x7f388815d900 2024-04-25T08:35:50.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.692+0000 7f38857fa640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f387406b4d0 0x7f387406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:50.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.692+0000 7f38857fa640 1 -- 172.21.15.38:0/2535535669 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f388008a0c0 con 0x7f388815d900 2024-04-25T08:35:50.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.692+0000 7f388cbf9640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f387406b4d0 0x7f387406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:50.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.693+0000 7f388cbf9640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f387406b4d0 0x7f387406d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3878034000 tx=0x7f387802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.696+0000 7f38857fa640 1 -- 172.21.15.38:0/2535535669 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38880cf380 con 0x7f388815d900 2024-04-25T08:35:50.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.888+0000 7f388ee5d640 1 -- 172.21.15.38:0/2535535669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f388804ef90 con 0x7f388815d900 2024-04-25T08:35:50.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.889+0000 7f38857fa640 1 -- 172.21.15.38:0/2535535669 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3880058db0 con 0x7f388815d900 2024-04-25T08:35:50.885 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:35:50.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.891+0000 7f388ee5d640 1 -- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f387406b4d0 msgr2=0x7f387406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.891+0000 7f388ee5d640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f387406b4d0 0x7f387406d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3878034000 tx=0x7f387802e000 comp rx=0 tx=0).stop 2024-04-25T08:35:50.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.891+0000 7f388ee5d640 1 -- 172.21.15.38:0/2535535669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388815d900 msgr2=0x7f38880cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:50.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.891+0000 7f388ee5d640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388815d900 0x7f38880cae40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f388000b9a0 tx=0x7f388000be60 comp rx=0 tx=0).stop 2024-04-25T08:35:50.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.891+0000 7f388ee5d640 1 -- 172.21.15.38:0/2535535669 shutdown_connections 2024-04-25T08:35:50.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.891+0000 7f388ee5d640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f388815d900 0x7f38880cae40 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.891+0000 7f388ee5d640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38880d1870 0x7f38880ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.891+0000 7f388ee5d640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f387406b4d0 0x7f387406d990 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.891+0000 7f388ee5d640 1 --2- 172.21.15.38:0/2535535669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38880ce500 0x7f3888151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:50.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.891+0000 7f388ee5d640 1 -- 172.21.15.38:0/2535535669 >> 172.21.15.38:0/2535535669 conn(0x7f3888066d10 msgr2=0x7f3888160640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:50.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.891+0000 7f388ee5d640 1 -- 172.21.15.38:0/2535535669 shutdown_connections 2024-04-25T08:35:50.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:50.891+0000 7f388ee5d640 1 -- 172.21.15.38:0/2535535669 wait complete. 2024-04-25T08:35:50.897 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:35:50.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:50.898 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:50.921 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[65507]: fuse finished with error 0 and tester_r 0 2024-04-25T08:35:50.923 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:51.003 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:35:51.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:51.004 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:35:51.020 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:51.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.165+0000 7fa1cf19b640 1 Processor -- start 2024-04-25T08:35:51.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.165+0000 7fa1cf19b640 1 -- start start 2024-04-25T08:35:51.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.166+0000 7fa1cf19b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1c815dc40 0x7fa1c80d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.166+0000 7fa1cf19b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa1c80b9580 0x7fa1c80d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.166+0000 7fa1cf19b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa1c80d1e40 0x7fa1c80d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.166+0000 7fa1cf19b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c80d19b0 con 0x7fa1c80b9580 2024-04-25T08:35:51.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.166+0000 7fa1cf19b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c80b8400 con 0x7fa1c80d1e40 2024-04-25T08:35:51.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.166+0000 7fa1cf19b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c80b86e0 con 0x7fa1c815dc40 2024-04-25T08:35:51.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.166+0000 7fa1c7fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa1c80b9580 0x7fa1c80d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.166+0000 7fa1c7fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa1c80b9580 0x7fa1c80d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34030/0 (socket says 172.21.15.38:34030) 2024-04-25T08:35:51.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.166+0000 7fa1c7fff640 1 -- 172.21.15.38:0/196773534 learned_addr learned my addr 172.21.15.38:0/196773534 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:51.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.166+0000 7fa1cd738640 1 --2- 172.21.15.38:0/196773534 >> v2:172.21.15.107:3300/0 conn(0x7fa1c80d1e40 0x7fa1c80d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.166+0000 7fa1ccf37640 1 --2- 172.21.15.38:0/196773534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1c815dc40 0x7fa1c80d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.167+0000 7fa1c7fff640 1 -- 172.21.15.38:0/196773534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1c815dc40 msgr2=0x7fa1c80d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:51.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.167+0000 7fa1c7fff640 1 --2- 172.21.15.38:0/196773534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1c815dc40 0x7fa1c80d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.167+0000 7fa1c7fff640 1 -- 172.21.15.38:0/196773534 >> v2:172.21.15.107:3300/0 conn(0x7fa1c80d1e40 msgr2=0x7fa1c80d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:51.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.167+0000 7fa1c7fff640 1 --2- 172.21.15.38:0/196773534 >> v2:172.21.15.107:3300/0 conn(0x7fa1c80d1e40 0x7fa1c80d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.167+0000 7fa1c7fff640 1 -- 172.21.15.38:0/196773534 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c80b89c0 con 0x7fa1c80b9580 2024-04-25T08:35:51.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.167+0000 7fa1ccf37640 1 --2- 172.21.15.38:0/196773534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1c815dc40 0x7fa1c80d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:51.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.167+0000 7fa1c7fff640 1 --2- 172.21.15.38:0/196773534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1c80b9580 0x7fa1c80d0e10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80d4990 tx=0x7fa1b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2330564deb9f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.167+0000 7fa1c77fe640 1 -- 172.21.15.38:0/196773534 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0037070 con 0x7fa1c80b9580 2024-04-25T08:35:51.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.167+0000 7fa1c77fe640 1 -- 172.21.15.38:0/196773534 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b0032440 con 0x7fa1c80b9580 2024-04-25T08:35:51.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.168+0000 7fa1c77fe640 1 -- 172.21.15.38:0/196773534 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0032810 con 0x7fa1c80b9580 2024-04-25T08:35:51.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.168+0000 7fa1cf19b640 1 -- 172.21.15.38:0/196773534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1c80b9580 msgr2=0x7fa1c80d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.168+0000 7fa1cf19b640 1 --2- 172.21.15.38:0/196773534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1c80b9580 0x7fa1c80d0e10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80d4990 tx=0x7fa1b002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.168+0000 7fa1cf19b640 1 -- 172.21.15.38:0/196773534 shutdown_connections 2024-04-25T08:35:51.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.168+0000 7fa1cf19b640 1 --2- 172.21.15.38:0/196773534 >> v2:172.21.15.107:3300/0 conn(0x7fa1c80d1e40 0x7fa1c80d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.168+0000 7fa1cf19b640 1 --2- 172.21.15.38:0/196773534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1c80b9580 0x7fa1c80d0e10 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.168+0000 7fa1cf19b640 1 --2- 172.21.15.38:0/196773534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1c815dc40 0x7fa1c80d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.168+0000 7fa1cf19b640 1 -- 172.21.15.38:0/196773534 >> 172.21.15.38:0/196773534 conn(0x7fa1c8066ce0 msgr2=0x7fa1c80670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:51.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.168+0000 7fa1cf19b640 1 -- 172.21.15.38:0/196773534 shutdown_connections 2024-04-25T08:35:51.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.168+0000 7fa1cf19b640 1 -- 172.21.15.38:0/196773534 wait complete. 2024-04-25T08:35:51.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.169+0000 7fa1cf19b640 1 Processor -- start 2024-04-25T08:35:51.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.169+0000 7fa1cf19b640 1 -- start start 2024-04-25T08:35:51.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.170+0000 7fa1cf19b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1c80b9580 0x7fa1c81f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.170+0000 7fa1cf19b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1c80d1e40 0x7fa1c815ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.170+0000 7fa1cf19b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c815dc40 0x7fa1c8160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.170+0000 7fa1cf19b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c80d2950 con 0x7fa1c80d1e40 2024-04-25T08:35:51.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.170+0000 7fa1cf19b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c80d27d0 con 0x7fa1c815dc40 2024-04-25T08:35:51.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.170+0000 7fa1cf19b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c80d2ad0 con 0x7fa1c80b9580 2024-04-25T08:35:51.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.170+0000 7fa1c7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1c80d1e40 0x7fa1c815ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.170+0000 7fa1c7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1c80d1e40 0x7fa1c815ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34044/0 (socket says 172.21.15.38:34044) 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.170+0000 7fa1c7fff640 1 -- 172.21.15.38:0/2813427811 learned_addr learned my addr 172.21.15.38:0/2813427811 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1cd738640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c815dc40 0x7fa1c8160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1ccf37640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1c80b9580 0x7fa1c81f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1c7fff640 1 -- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1c80b9580 msgr2=0x7fa1c81f72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1c7fff640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1c80b9580 0x7fa1c81f72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1c7fff640 1 -- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c815dc40 msgr2=0x7fa1c8160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1c7fff640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c815dc40 0x7fa1c8160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1c7fff640 1 -- 172.21.15.38:0/2813427811 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c81f7810 con 0x7fa1c80d1e40 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1ccf37640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1c80b9580 0x7fa1c81f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1c7fff640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1c80d1e40 0x7fa1c815ff30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa1b002a220 tx=0x7fa1b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1c57fa640 1 -- 172.21.15.38:0/2813427811 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0041020 con 0x7fa1c80d1e40 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1c57fa640 1 -- 172.21.15.38:0/2813427811 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b00306c0 con 0x7fa1c80d1e40 2024-04-25T08:35:51.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1cf19b640 1 -- 172.21.15.38:0/2813427811 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c815e3c0 con 0x7fa1c80d1e40 2024-04-25T08:35:51.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1c57fa640 1 -- 172.21.15.38:0/2813427811 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b0037040 con 0x7fa1c80d1e40 2024-04-25T08:35:51.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.171+0000 7fa1cf19b640 1 -- 172.21.15.38:0/2813427811 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c815e980 con 0x7fa1c80d1e40 2024-04-25T08:35:51.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.172+0000 7fa1cf19b640 1 -- 172.21.15.38:0/2813427811 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c8068e80 con 0x7fa1c80d1e40 2024-04-25T08:35:51.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.173+0000 7fa1c57fa640 1 -- 172.21.15.38:0/2813427811 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1b0044020 con 0x7fa1c80d1e40 2024-04-25T08:35:51.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.173+0000 7fa1c57fa640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa1a806b4d0 0x7fa1a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.173+0000 7fa1c57fa640 1 -- 172.21.15.38:0/2813427811 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1b00aab10 con 0x7fa1c80d1e40 2024-04-25T08:35:51.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.173+0000 7fa1ccf37640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa1a806b4d0 0x7fa1a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.174+0000 7fa1ccf37640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa1a806b4d0 0x7fa1a806d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80c8e10 tx=0x7fa1c00074b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.178+0000 7fa1c57fa640 1 -- 172.21.15.38:0/2813427811 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1b004b030 con 0x7fa1c80d1e40 2024-04-25T08:35:51.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.399+0000 7fa1cf19b640 1 -- 172.21.15.38:0/2813427811 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1c80cdcf0 con 0x7fa1c80d1e40 2024-04-25T08:35:51.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.399+0000 7fa1c57fa640 1 -- 172.21.15.38:0/2813427811 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fa1c81f7810 con 0x7fa1c80d1e40 2024-04-25T08:35:51.395 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:51.395 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3966080134","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3966080134},{"type":"v1","addr":"172.21.15.107:6836","nonce":3966080134}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6145,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/632873469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":632873469},{"type":"v1","addr":"172.21.15.107:6837","nonce":632873469}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32},{"gid":6810,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4257593739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4257593739},{"type":"v1","addr":"172.21.15.38:6834","nonce":4257593739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:40.623355+0000","modified":"2024-04-25T08:35:42.623054+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6813},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6813":{"gid":6813,"name":"a","rank":0,"incarnation":33,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/239365678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":239365678},{"type":"v1","addr":"172.21.15.38:6835","nonce":239365678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:35:51.395 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 35 2024-04-25T08:35:51.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.401+0000 7fa1cf19b640 1 -- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa1a806b4d0 msgr2=0x7fa1a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.401+0000 7fa1cf19b640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa1a806b4d0 0x7fa1a806d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80c8e10 tx=0x7fa1c00074b0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.401+0000 7fa1cf19b640 1 -- 172.21.15.38:0/2813427811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1c80d1e40 msgr2=0x7fa1c815ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.401+0000 7fa1cf19b640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1c80d1e40 0x7fa1c815ff30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa1b002a220 tx=0x7fa1b00046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.401+0000 7fa1cf19b640 1 -- 172.21.15.38:0/2813427811 shutdown_connections 2024-04-25T08:35:51.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.401+0000 7fa1cf19b640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1c815dc40 0x7fa1c8160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.401+0000 7fa1cf19b640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1c80d1e40 0x7fa1c815ff30 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.401+0000 7fa1cf19b640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa1a806b4d0 0x7fa1a806d990 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.401+0000 7fa1cf19b640 1 --2- 172.21.15.38:0/2813427811 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1c80b9580 0x7fa1c81f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.401+0000 7fa1cf19b640 1 -- 172.21.15.38:0/2813427811 >> 172.21.15.38:0/2813427811 conn(0x7fa1c8066ce0 msgr2=0x7fa1c80b7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:51.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.402+0000 7fa1cf19b640 1 -- 172.21.15.38:0/2813427811 shutdown_connections 2024-04-25T08:35:51.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.402+0000 7fa1cf19b640 1 -- 172.21.15.38:0/2813427811 wait complete. 2024-04-25T08:35:51.408 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T08:35:51.537 INFO:teuthology.orchestra.run.smithi038.stdout:{"message":"","return_code":0} 2024-04-25T08:35:51.538 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:35:51.539 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:35:51.539 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:51.539 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:35:51.539 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:51.540 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:35:51.540 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:51.540 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:35:51.540 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:35:51.549 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:35:51.549 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:51.550 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:35:51.550 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:51.551 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:35:51.551 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:51.562 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:35:51.562 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:51.562 DEBUG:teuthology.orchestra.run.smithi038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-25T08:35:51.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.637+0000 7ff49251bf40 1 Processor -- start 2024-04-25T08:35:51.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.637+0000 7ff49251bf40 1 -- start start 2024-04-25T08:35:51.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff49251bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e77c4c0 0x55f45e8480c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff49251bf40 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f45e7c26a0 0x55f45e7b9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff49251bf40 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f45e84eb10 0x55f45e84c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff49251bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45e720df0 con 0x55f45e77c4c0 2024-04-25T08:35:51.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff49251bf40 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45e7c3010 con 0x55f45e84eb10 2024-04-25T08:35:51.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff49251bf40 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f45e848600 con 0x55f45e7c26a0 2024-04-25T08:35:51.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff48f56d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e77c4c0 0x55f45e8480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff48f56d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e77c4c0 0x55f45e8480c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34046/0 (socket says 172.21.15.38:34046) 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff48f56d640 1 -- 172.21.15.38:0/3098708091 learned_addr learned my addr 172.21.15.38:0/3098708091 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff48ed6c640 1 --2- 172.21.15.38:0/3098708091 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f45e7c26a0 0x55f45e7b9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff48fd6e640 1 --2- 172.21.15.38:0/3098708091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f45e84eb10 0x55f45e84c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff48f56d640 1 -- 172.21.15.38:0/3098708091 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f45e7c26a0 msgr2=0x55f45e7b9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff48f56d640 1 --2- 172.21.15.38:0/3098708091 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f45e7c26a0 0x55f45e7b9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff48f56d640 1 -- 172.21.15.38:0/3098708091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f45e84eb10 msgr2=0x55f45e84c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff48f56d640 1 --2- 172.21.15.38:0/3098708091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f45e84eb10 0x55f45e84c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff48f56d640 1 -- 172.21.15.38:0/3098708091 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f45e6a6c50 con 0x55f45e77c4c0 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff48f56d640 1 --2- 172.21.15.38:0/3098708091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e77c4c0 0x55f45e8480c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b240 tx=0x7ff48800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b188c72a4f771515 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.638+0000 7ff48dd6a640 1 -- 172.21.15.38:0/3098708091 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488008070 con 0x55f45e77c4c0 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.639+0000 7ff48dd6a640 1 -- 172.21.15.38:0/3098708091 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48800d500 con 0x55f45e77c4c0 2024-04-25T08:35:51.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.639+0000 7ff48dd6a640 1 -- 172.21.15.38:0/3098708091 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48800d8b0 con 0x55f45e77c4c0 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.639+0000 7ff49251bf40 1 -- 172.21.15.38:0/3098708091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e77c4c0 msgr2=0x55f45e8480c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.639+0000 7ff49251bf40 1 --2- 172.21.15.38:0/3098708091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e77c4c0 0x55f45e8480c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b240 tx=0x7ff48800b700 comp rx=0 tx=0).stop 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.639+0000 7ff49251bf40 1 -- 172.21.15.38:0/3098708091 shutdown_connections 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.639+0000 7ff49251bf40 1 --2- 172.21.15.38:0/3098708091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f45e84eb10 0x55f45e84c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.639+0000 7ff49251bf40 1 --2- 172.21.15.38:0/3098708091 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f45e7c26a0 0x55f45e7b9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.639+0000 7ff49251bf40 1 --2- 172.21.15.38:0/3098708091 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e77c4c0 0x55f45e8480c0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.639+0000 7ff49251bf40 1 -- 172.21.15.38:0/3098708091 >> 172.21.15.38:0/3098708091 conn(0x55f45e779900 msgr2=0x55f45e77b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.639+0000 7ff49251bf40 1 -- 172.21.15.38:0/3098708091 shutdown_connections 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.639+0000 7ff49251bf40 1 -- 172.21.15.38:0/3098708091 wait complete. 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.640+0000 7ff49251bf40 1 Processor -- start 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.640+0000 7ff49251bf40 1 -- start start 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.640+0000 7ff49251bf40 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f45e77c4c0 0x55f45e845c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.640+0000 7ff49251bf40 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e7c26a0 0x55f45e7a83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.640+0000 7ff49251bf40 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f45e84eb10 0x55f45e8476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.640+0000 7ff49251bf40 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45e7c1d70 con 0x55f45e7c26a0 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.640+0000 7ff49251bf40 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f45e7bafd0 con 0x55f45e84eb10 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.640+0000 7ff49251bf40 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f45e77bca0 con 0x55f45e77c4c0 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.640+0000 7ff48ed6c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e7c26a0 0x55f45e7a83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.640+0000 7ff48ed6c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e7c26a0 0x55f45e7a83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34052/0 (socket says 172.21.15.38:34052) 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.640+0000 7ff48ed6c640 1 -- 172.21.15.38:0/4149707472 learned_addr learned my addr 172.21.15.38:0/4149707472 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff48fd6e640 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f45e84eb10 0x55f45e8476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff48f56d640 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f45e77c4c0 0x55f45e845c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff48ed6c640 1 -- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f45e77c4c0 msgr2=0x55f45e845c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff48ed6c640 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f45e77c4c0 0x55f45e845c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff48ed6c640 1 -- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f45e84eb10 msgr2=0x55f45e8476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff48ed6c640 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f45e84eb10 0x55f45e8476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff48ed6c640 1 -- 172.21.15.38:0/4149707472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f45e8461d0 con 0x55f45e7c26a0 2024-04-25T08:35:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff48ed6c640 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e7c26a0 0x55f45e7a83f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff484025ba0 tx=0x7ff484002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff48cd68640 1 -- 172.21.15.38:0/4149707472 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484063030 con 0x55f45e7c26a0 2024-04-25T08:35:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff48cd68640 1 -- 172.21.15.38:0/4149707472 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48400a280 con 0x55f45e7c26a0 2024-04-25T08:35:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff49251bf40 1 -- 172.21.15.38:0/4149707472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f45e7a8c40 con 0x55f45e7c26a0 2024-04-25T08:35:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff48cd68640 1 -- 172.21.15.38:0/4149707472 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484059040 con 0x55f45e7c26a0 2024-04-25T08:35:51.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.641+0000 7ff49251bf40 1 -- 172.21.15.38:0/4149707472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f45e77b080 con 0x55f45e7c26a0 2024-04-25T08:35:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.642+0000 7ff48cd68640 1 -- 172.21.15.38:0/4149707472 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff484066030 con 0x55f45e7c26a0 2024-04-25T08:35:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.643+0000 7ff48cd68640 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff47c069ee0 0x7ff47c06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.643+0000 7ff48cd68640 1 -- 172.21.15.38:0/4149707472 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4840cc650 con 0x55f45e7c26a0 2024-04-25T08:35:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.643+0000 7ff49251bf40 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x55f45e77b880 0x55f45e7a7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.643+0000 7ff49251bf40 1 -- 172.21.15.38:0/4149707472 --> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] -- osd_op(unknown.0.0:1 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e67) v9 -- 0x55f45e7a7890 con 0x55f45e77b880 2024-04-25T08:35:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.643+0000 7ff48f56d640 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff47c069ee0 0x7ff47c06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.643+0000 7ff48fd6e640 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x55f45e77b880 0x55f45e7a7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.644+0000 7ff48f56d640 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff47c069ee0 0x7ff47c06c3a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b740 tx=0x7ff488008000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.644+0000 7ff48fd6e640 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x55f45e77b880 0x55f45e7a7350 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.645+0000 7ff48fd6e640 1 -- 172.21.15.38:0/4149707472 <== osd.4 v2:172.21.15.107:6800/592699329 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v67'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff48000aa80 con 0x55f45e77b880 2024-04-25T08:35:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.645+0000 7ff49251bf40 1 -- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x55f45e77b880 msgr2=0x55f45e7a7350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.645+0000 7ff49251bf40 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x55f45e77b880 0x55f45e7a7350 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.645+0000 7ff49251bf40 1 -- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff47c069ee0 msgr2=0x7ff47c06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.645+0000 7ff49251bf40 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff47c069ee0 0x7ff47c06c3a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b740 tx=0x7ff488008000 comp rx=0 tx=0).stop 2024-04-25T08:35:51.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.645+0000 7ff49251bf40 1 -- 172.21.15.38:0/4149707472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e7c26a0 msgr2=0x55f45e7a83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.645+0000 7ff49251bf40 1 --2- 172.21.15.38:0/4149707472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f45e7c26a0 0x55f45e7a83f0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff484025ba0 tx=0x7ff484002fb0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.645+0000 7ff48fd6e640 1 -- 172.21.15.38:0/4149707472 reap_dead start 2024-04-25T08:35:51.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.645+0000 7ff49251bf40 1 -- 172.21.15.38:0/4149707472 shutdown_connections 2024-04-25T08:35:51.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.645+0000 7ff49251bf40 1 -- 172.21.15.38:0/4149707472 >> 172.21.15.38:0/4149707472 conn(0x55f45e779900 msgr2=0x55f45e851750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:51.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.645+0000 7ff49251bf40 1 -- 172.21.15.38:0/4149707472 shutdown_connections 2024-04-25T08:35:51.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.645+0000 7ff49251bf40 1 -- 172.21.15.38:0/4149707472 wait complete. 2024-04-25T08:35:51.646 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-25T08:35:51.646 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T08:35:51.646 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-25T08:35:51.646 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T08:35:51.647 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-25T08:35:51.647 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T08:35:51.647 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-25T08:35:51.647 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T08:35:51.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.796+0000 7fd138b1d640 1 Processor -- start 2024-04-25T08:35:51.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.796+0000 7f331425d640 1 Processor -- start 2024-04-25T08:35:51.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.796+0000 7f5b260fc640 1 Processor -- start 2024-04-25T08:35:51.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.796+0000 7fd138b1d640 1 -- start start 2024-04-25T08:35:51.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.796+0000 7fb4d6e9c640 1 Processor -- start 2024-04-25T08:35:51.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.796+0000 7f5b260fc640 1 -- start start 2024-04-25T08:35:51.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.796+0000 7fb4d6e9c640 1 -- start start 2024-04-25T08:35:51.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.796+0000 7f331425d640 1 -- start start 2024-04-25T08:35:51.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f5b260fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5b2015a600 0x7f5b2015ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f5b260fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b200ceba0 0x7f5b20070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fb4d6e9c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb4d015dbb0 0x7fb4d00b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fd138b1d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340b3b80 0x7fd134162d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f331425d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c0cec30 0x7f330c15a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f331425d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330c0b46a0 con 0x7f330c0cbe20 2024-04-25T08:35:51.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f331425d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330c15aa00 con 0x7f330c0b2d20 2024-04-25T08:35:51.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f331425d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c15db40 con 0x7f330c0cec30 2024-04-25T08:35:51.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f5b260fc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b200b2430 con 0x7f5b200714c0 2024-04-25T08:35:51.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f5b260fc640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b20071d70 con 0x7f5b2015a600 2024-04-25T08:35:51.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f5b260fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b200b26d0 con 0x7f5b200ceba0 2024-04-25T08:35:51.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fd138b1d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1340b4240 con 0x7fd1340cbe10 2024-04-25T08:35:51.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fd138b1d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1340b4520 con 0x7fd134161e00 2024-04-25T08:35:51.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fd138b1d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd134165dd0 con 0x7fd1340b3b80 2024-04-25T08:35:51.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f33127fa640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f330c0cbe20 0x7f330c0cd1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32860/0 (socket says 172.21.15.38:32860) 2024-04-25T08:35:51.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f33127fa640 1 -- 172.21.15.38:0/3989004082 learned_addr learned my addr 172.21.15.38:0/3989004082 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:51.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fd132d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd1340cbe10 0x7fd1340ce5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32846/0 (socket says 172.21.15.38:32846) 2024-04-25T08:35:51.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fd132d9d640 1 -- 172.21.15.38:0/985682770 learned_addr learned my addr 172.21.15.38:0/985682770 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:51.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f5b1effd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b200ceba0 0x7f5b20070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fb4d6e9c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d00b1c60 con 0x7fb4d006cb60 2024-04-25T08:35:51.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fb4d4c38640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb4d015dbb0 0x7fb4d00b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fb4d4c38640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb4d015dbb0 0x7fb4d00b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:54122/0 (socket says 172.21.15.38:54122) 2024-04-25T08:35:51.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fb4d4c38640 1 -- 172.21.15.38:0/3402740285 learned_addr learned my addr 172.21.15.38:0/3402740285 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:51.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/3402740285 --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4d00b1f40 con 0x7fb4d00b1510 2024-04-25T08:35:51.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f5b1effd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b200ceba0 0x7f5b20070f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:33312/0 (socket says 172.21.15.38:33312) 2024-04-25T08:35:51.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f5b1effd640 1 -- 172.21.15.38:0/1148366875 learned_addr learned my addr 172.21.15.38:0/1148366875 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:51.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7f5b1f7fe640 1 --2- 172.21.15.38:0/1148366875 >> v2:172.21.15.38:3300/0 conn(0x7f5b2015a600 0x7f5b2015ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fd131d9b640 1 --2- 172.21.15.38:0/985682770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340b3b80 0x7fd134162d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.797+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/3402740285 --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4d00cfae0 con 0x7fb4d015dbb0 2024-04-25T08:35:51.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f33117f8640 1 --2- 172.21.15.38:0/3989004082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c0cec30 0x7f330c15a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971104246 0 0) 0x7f330c0b46a0 con 0x7f330c0cbe20 2024-04-25T08:35:51.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f8005730 con 0x7f330c0cbe20 2024-04-25T08:35:51.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7fd13159a640 1 -- 172.21.15.38:0/985682770 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175916355 0 0) 0x7fd1340b4240 con 0x7fd1340cbe10 2024-04-25T08:35:51.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7fd13159a640 1 -- 172.21.15.38:0/985682770 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd11c003520 con 0x7fd1340cbe10 2024-04-25T08:35:51.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f5b1effd640 1 -- 172.21.15.38:0/1148366875 >> v2:172.21.15.38:3300/0 conn(0x7f5b2015a600 msgr2=0x7f5b2015ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:51.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f5b1effd640 1 --2- 172.21.15.38:0/1148366875 >> v2:172.21.15.38:3300/0 conn(0x7f5b2015a600 0x7f5b2015ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f5b1effd640 1 -- 172.21.15.38:0/1148366875 >> v1:172.21.15.107:6789/0 conn(0x7f5b200714c0 legacy=0x7f5b200cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f5b1effd640 1 -- 172.21.15.38:0/1148366875 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b200b29b0 con 0x7f5b200ceba0 2024-04-25T08:35:51.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659603698 0 0) 0x7f330c15aa00 con 0x7f330c0b2d20 2024-04-25T08:35:51.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f330c0b46a0 con 0x7f330c0b2d20 2024-04-25T08:35:51.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7fb4d4c38640 1 -- 172.21.15.38:0/3402740285 >> v1:172.21.15.107:6789/0 conn(0x7fb4d00b1510 legacy=0x7fb4d006c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7fb4d4c38640 1 -- 172.21.15.38:0/3402740285 >> v1:172.21.15.38:6789/0 conn(0x7fb4d006cb60 legacy=0x7fb4d00cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7fb4d4c38640 1 -- 172.21.15.38:0/3402740285 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d00b2220 con 0x7fb4d015dbb0 2024-04-25T08:35:51.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 <== mon.1 v1:172.21.15.107:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3845904956 0 0) 0x7f330c0b46a0 con 0x7f330c0b2d20 2024-04-25T08:35:51.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c0cec30 msgr2=0x7f330c15a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:51.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 --2- 172.21.15.38:0/3989004082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c0cec30 0x7f330c15a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 >> v1:172.21.15.38:6789/0 conn(0x7f330c0cbe20 legacy=0x7f330c0cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 --> v1:172.21.15.107:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c15ace0 con 0x7f330c0b2d20 2024-04-25T08:35:51.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 <== mon.1 v1:172.21.15.107:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f33080019f0 con 0x7f330c0b2d20 2024-04-25T08:35:51.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f33117f8640 1 --2- 172.21.15.38:0/3989004082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c0cec30 0x7f330c15a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:51.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 >> v1:172.21.15.107:6789/0 conn(0x7f330c0b2d20 legacy=0x7f330c15dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 --2- 172.21.15.38:0/3989004082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32f8005d50 0x7f32f8006150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 --2- 172.21.15.38:0/3989004082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f8006a10 0x7f32f8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 --2- 172.21.15.38:0/3989004082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c0cec30 0x7f32f80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c15db40 con 0x7f32f8005d50 2024-04-25T08:35:51.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f800dbd0 con 0x7f32f8006a10 2024-04-25T08:35:51.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f800ddd0 con 0x7f330c0cec30 2024-04-25T08:35:51.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7f3311ff9640 1 --2- 172.21.15.38:0/3989004082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32f8005d50 0x7f32f8006150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7fd131d9b640 1 -- 172.21.15.38:0/985682770 >> v1:172.21.15.107:6789/0 conn(0x7fd134161e00 legacy=0x7fd134166260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7fd131d9b640 1 -- 172.21.15.38:0/985682770 >> v1:172.21.15.38:6789/0 conn(0x7fd1340cbe10 legacy=0x7fd1340ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.798+0000 7fd131d9b640 1 -- 172.21.15.38:0/985682770 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd134165a60 con 0x7fd1340b3b80 2024-04-25T08:35:51.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7f33117f8640 1 --2- 172.21.15.38:0/3989004082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f8006a10 0x7f32f8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7f3311ff9640 1 -- 172.21.15.38:0/3989004082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c0cec30 msgr2=0x7f32f80073d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:35:51.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7f3311ff9640 1 --2- 172.21.15.38:0/3989004082 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c0cec30 0x7f32f80073d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7f3311ff9640 1 -- 172.21.15.38:0/3989004082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f8006a10 msgr2=0x7f32f8006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:51.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7f3311ff9640 1 --2- 172.21.15.38:0/3989004082 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32f8006a10 0x7f32f8006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7f5b1effd640 1 --2- 172.21.15.38:0/1148366875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b200ceba0 0x7f5b20070f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009870 tx=0x7f5b1402a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34b581a11dcd94e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7f3311ff9640 1 -- 172.21.15.38:0/3989004082 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c15ace0 con 0x7f32f8005d50 2024-04-25T08:35:51.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7f5b1e7fc640 1 -- 172.21.15.38:0/1148366875 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14037070 con 0x7f5b200ceba0 2024-04-25T08:35:51.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7fb4d4c38640 1 --2- 172.21.15.38:0/3402740285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d015dbb0 0x7fb4d00b0cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40097f0 tx=0x7fb4c402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67a93a890fe0b92d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7f5b1e7fc640 1 -- 172.21.15.38:0/1148366875 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1402adf0 con 0x7f5b200ceba0 2024-04-25T08:35:51.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7f5b1e7fc640 1 -- 172.21.15.38:0/1148366875 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14032660 con 0x7f5b200ceba0 2024-04-25T08:35:51.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7fd131d9b640 1 --2- 172.21.15.38:0/985682770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340b3b80 0x7fd134162d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd1280097d0 tx=0x7fd12802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=806b309a78dd29c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7fd13159a640 1 -- 172.21.15.38:0/985682770 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd128037070 con 0x7fd1340b3b80 2024-04-25T08:35:51.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7fd13159a640 1 -- 172.21.15.38:0/985682770 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd128032440 con 0x7fd1340b3b80 2024-04-25T08:35:51.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7fd13159a640 1 -- 172.21.15.38:0/985682770 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd128032810 con 0x7fd1340b3b80 2024-04-25T08:35:51.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7fb4cf7fe640 1 -- 172.21.15.38:0/3402740285 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4037070 con 0x7fb4d015dbb0 2024-04-25T08:35:51.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7fb4cf7fe640 1 -- 172.21.15.38:0/3402740285 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c4032440 con 0x7fb4d015dbb0 2024-04-25T08:35:51.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.799+0000 7fb4cf7fe640 1 -- 172.21.15.38:0/3402740285 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4032810 con 0x7fb4d015dbb0 2024-04-25T08:35:51.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7f3311ff9640 1 --2- 172.21.15.38:0/3989004082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32f8005d50 0x7f32f8006150 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f330800c570 tx=0x7f32f8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a808ce89943ec9cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7f5b260fc640 1 -- 172.21.15.38:0/1148366875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b200ceba0 msgr2=0x7f5b20070f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7f5b260fc640 1 --2- 172.21.15.38:0/1148366875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b200ceba0 0x7f5b20070f80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009870 tx=0x7f5b1402a160 comp rx=0 tx=0).stop 2024-04-25T08:35:51.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330800c140 con 0x7f32f8005d50 2024-04-25T08:35:51.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7f5b260fc640 1 -- 172.21.15.38:0/1148366875 shutdown_connections 2024-04-25T08:35:51.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7f5b260fc640 1 --2- 172.21.15.38:0/1148366875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b200ceba0 0x7f5b20070f80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7f5b260fc640 1 --2- 172.21.15.38:0/1148366875 >> v2:172.21.15.38:3300/0 conn(0x7f5b2015a600 0x7f5b2015ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7f5b260fc640 1 -- 172.21.15.38:0/1148366875 >> 172.21.15.38:0/1148366875 conn(0x7f5b2006abe0 msgr2=0x7f5b2006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:51.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330802c6f0 con 0x7f32f8005d50 2024-04-25T08:35:51.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7f3310ff7640 1 -- 172.21.15.38:0/3989004082 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330802cae0 con 0x7f32f8005d50 2024-04-25T08:35:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7f5b260fc640 1 -- 172.21.15.38:0/1148366875 shutdown_connections 2024-04-25T08:35:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7f5b260fc640 1 -- 172.21.15.38:0/1148366875 wait complete. 2024-04-25T08:35:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7fd138b1d640 1 -- 172.21.15.38:0/985682770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340b3b80 msgr2=0x7fd134162d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7fd138b1d640 1 --2- 172.21.15.38:0/985682770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340b3b80 0x7fd134162d10 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd1280097d0 tx=0x7fd12802a210 comp rx=0 tx=0).stop 2024-04-25T08:35:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/3402740285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d015dbb0 msgr2=0x7fb4d00b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7fb4d6e9c640 1 --2- 172.21.15.38:0/3402740285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d015dbb0 0x7fb4d00b0cc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40097f0 tx=0x7fb4c402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7fd138b1d640 1 -- 172.21.15.38:0/985682770 shutdown_connections 2024-04-25T08:35:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/3402740285 shutdown_connections 2024-04-25T08:35:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7fb4d6e9c640 1 --2- 172.21.15.38:0/3402740285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d015dbb0 0x7fb4d00b0cc0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/3402740285 >> 172.21.15.38:0/3402740285 conn(0x7fb4d00d15e0 msgr2=0x7fb4d00d19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7fd138b1d640 1 --2- 172.21.15.38:0/985682770 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1340b3b80 0x7fd134162d10 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.800+0000 7fd138b1d640 1 -- 172.21.15.38:0/985682770 >> 172.21.15.38:0/985682770 conn(0x7fd13406abe0 msgr2=0x7fd13406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:51.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.801+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/3402740285 shutdown_connections 2024-04-25T08:35:51.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.801+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/3402740285 wait complete. 2024-04-25T08:35:51.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.801+0000 7fd138b1d640 1 -- 172.21.15.38:0/985682770 shutdown_connections 2024-04-25T08:35:51.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.801+0000 7fd138b1d640 1 -- 172.21.15.38:0/985682770 wait complete. 2024-04-25T08:35:51.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.801+0000 7f331425d640 1 -- 172.21.15.38:0/3989004082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32f8005d50 msgr2=0x7f32f8006150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.801+0000 7f331425d640 1 --2- 172.21.15.38:0/3989004082 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32f8005d50 0x7f32f8006150 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f330800c570 tx=0x7f32f8007a40 comp rx=0 tx=0).stop 2024-04-25T08:35:51.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.801+0000 7f33127fa640 1 -- 172.21.15.38:0/3989004082 reap_dead start 2024-04-25T08:35:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.801+0000 7f331425d640 1 -- 172.21.15.38:0/3989004082 shutdown_connections 2024-04-25T08:35:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.801+0000 7f331425d640 1 -- 172.21.15.38:0/3989004082 >> 172.21.15.38:0/3989004082 conn(0x7f330c06abe0 msgr2=0x7f330c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.801+0000 7f331425d640 1 -- 172.21.15.38:0/3989004082 shutdown_connections 2024-04-25T08:35:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.801+0000 7f331425d640 1 -- 172.21.15.38:0/3989004082 wait complete. 2024-04-25T08:35:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.801+0000 7f5b260fc640 1 Processor -- start 2024-04-25T08:35:51.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.802+0000 7f5b260fc640 1 -- start start 2024-04-25T08:35:51.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.802+0000 7fb4d6e9c640 1 Processor -- start 2024-04-25T08:35:51.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.802+0000 7fb4d6e9c640 1 -- start start 2024-04-25T08:35:51.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.802+0000 7fd138b1d640 1 Processor -- start 2024-04-25T08:35:51.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.802+0000 7fd138b1d640 1 -- start start 2024-04-25T08:35:51.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.802+0000 7f5b260fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b200714c0 0x7f5b2015d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.802+0000 7f5b260fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b200ceba0 0x7f5b20159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b260fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b2015a600 0x7f5b2015a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b1effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b200ceba0 0x7f5b20159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b260fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b200b3600 con 0x7f5b200ceba0 2024-04-25T08:35:51.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b1effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b200ceba0 0x7f5b20159630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34082/0 (socket says 172.21.15.38:34082) 2024-04-25T08:35:51.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b1effd640 1 -- 172.21.15.38:0/8598399 learned_addr learned my addr 172.21.15.38:0/8598399 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:51.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b260fc640 1 -- 172.21.15.38:0/8598399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2006e380 con 0x7f5b200714c0 2024-04-25T08:35:51.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b260fc640 1 -- 172.21.15.38:0/8598399 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2014a5f0 con 0x7f5b2015a600 2024-04-25T08:35:51.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f331425d640 1 Processor -- start 2024-04-25T08:35:51.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b1f7fe640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b200714c0 0x7f5b2015d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b1ffff640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b2015a600 0x7f5b2015a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fb4d6e9c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d006cb60 0x7fb4d00b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b1effd640 1 -- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b2015a600 msgr2=0x7f5b2015a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b1effd640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b2015a600 0x7f5b2015a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b1effd640 1 -- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b200714c0 msgr2=0x7f5b2015d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b1effd640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b200714c0 0x7f5b2015d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fb4d6e9c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d00b1510 0x7fb4d014f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b1effd640 1 -- 172.21.15.38:0/8598399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2014a8d0 con 0x7f5b200ceba0 2024-04-25T08:35:51.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fb4d6e9c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4d015dbb0 0x7fb4d014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b1ffff640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b2015a600 0x7f5b2015a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:51.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fb4d6e9c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00d0680 con 0x7fb4d006cb60 2024-04-25T08:35:51.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fb4d6e9c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00b1950 con 0x7fb4d00b1510 2024-04-25T08:35:51.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fb4d4c38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d006cb60 0x7fb4d00b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f331425d640 1 -- start start 2024-04-25T08:35:51.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fb4d6e9c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4d00d0800 con 0x7fb4d015dbb0 2024-04-25T08:35:51.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fd138b1d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1340b3b80 0x7fd1341ecfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fb4d4c38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d006cb60 0x7fb4d00b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34084/0 (socket says 172.21.15.38:34084) 2024-04-25T08:35:51.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fb4d4c38640 1 -- 172.21.15.38:0/2371266339 learned_addr learned my addr 172.21.15.38:0/2371266339 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:51.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fd138b1d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1340cbe10 0x7fd13414c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7f5b1effd640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b200ceba0 0x7f5b20159630 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009870 tx=0x7f5b140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fb4d5439640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4d015dbb0 0x7fb4d014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fb4cffff640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d00b1510 0x7fb4d014f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.803+0000 7fd138b1d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd134161e00 0x7fd1341eb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fd131d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1340cbe10 0x7fd13414c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fb4d4c38640 1 -- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4d015dbb0 msgr2=0x7fb4d014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fb4d4c38640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4d015dbb0 0x7fb4d014fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fd13259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1340b3b80 0x7fd1341ecfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fb4d4c38640 1 -- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d00b1510 msgr2=0x7fb4d014f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fb4d4c38640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d00b1510 0x7fb4d014f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fb4d4c38640 1 -- 172.21.15.38:0/2371266339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d015ed70 con 0x7fb4d006cb60 2024-04-25T08:35:51.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7f5b07fff640 1 -- 172.21.15.38:0/8598399 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14041020 con 0x7f5b200ceba0 2024-04-25T08:35:51.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7f5b07fff640 1 -- 172.21.15.38:0/8598399 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1403b610 con 0x7f5b200ceba0 2024-04-25T08:35:51.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7f5b07fff640 1 -- 172.21.15.38:0/8598399 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14037040 con 0x7f5b200ceba0 2024-04-25T08:35:51.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fd131d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1340cbe10 0x7fd13414c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34088/0 (socket says 172.21.15.38:34088) 2024-04-25T08:35:51.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fd131d9b640 1 -- 172.21.15.38:0/3117909209 learned_addr learned my addr 172.21.15.38:0/3117909209 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:51.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fb4cffff640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d00b1510 0x7fb4d014f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:51.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fd132d9d640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd134161e00 0x7fd1341eb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7f5b260fc640 1 -- 172.21.15.38:0/8598399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2014abb0 con 0x7f5b200ceba0 2024-04-25T08:35:51.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fd138b1d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340cf070 con 0x7fd1340cbe10 2024-04-25T08:35:51.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fd138b1d640 1 -- 172.21.15.38:0/3117909209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340ceef0 con 0x7fd1340b3b80 2024-04-25T08:35:51.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fd138b1d640 1 -- 172.21.15.38:0/3117909209 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340b3fe0 con 0x7fd134161e00 2024-04-25T08:35:51.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7f5b260fc640 1 -- 172.21.15.38:0/8598399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2015d6a0 con 0x7f5b200ceba0 2024-04-25T08:35:51.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7f331425d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32f8005d50 0x7f330c157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7f331425d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c155780 0x7f330c155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7f331425d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f330c0b1600 0x7f330c0b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7f331425d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0cd8d0 con 0x7f32f8005d50 2024-04-25T08:35:51.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7f331425d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c15e6e0 con 0x7f330c0b1600 2024-04-25T08:35:51.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7f331425d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c0cda50 con 0x7f330c155780 2024-04-25T08:35:51.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.804+0000 7fb4d4c38640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d006cb60 0x7fb4d00b5ad0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40097f0 tx=0x7fb4c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7f33117f8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c155780 0x7f330c155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7f33127fa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f330c0b1600 0x7f330c0b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7f33117f8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c155780 0x7f330c155c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:33354/0 (socket says 172.21.15.38:33354) 2024-04-25T08:35:51.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7f33117f8640 1 -- 172.21.15.38:0/2565608226 learned_addr learned my addr 172.21.15.38:0/2565608226 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:51.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7fd13259c640 1 -- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd134161e00 msgr2=0x7fd1341eb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7fd13259c640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd134161e00 0x7fd1341eb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7fd13259c640 1 -- 172.21.15.38:0/3117909209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1340cbe10 msgr2=0x7fd13414c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7fd13259c640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1340cbe10 0x7fd13414c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7fd13259c640 1 -- 172.21.15.38:0/3117909209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13414cc10 con 0x7fd1340b3b80 2024-04-25T08:35:51.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7fb4cd7fa640 1 -- 172.21.15.38:0/2371266339 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4041020 con 0x7fb4d006cb60 2024-04-25T08:35:51.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7fb4cd7fa640 1 -- 172.21.15.38:0/2371266339 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4c4030a30 con 0x7fb4d006cb60 2024-04-25T08:35:51.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7fb4cd7fa640 1 -- 172.21.15.38:0/2371266339 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4c4037040 con 0x7fb4d006cb60 2024-04-25T08:35:51.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/2371266339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4d015f050 con 0x7fb4d006cb60 2024-04-25T08:35:51.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7f3311ff9640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32f8005d50 0x7f330c157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7f5b260fc640 1 -- 172.21.15.38:0/8598399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2004ef90 con 0x7f5b200ceba0 2024-04-25T08:35:51.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7fd13259c640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1340b3b80 0x7fd1341ecfd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd12400c3c0 tx=0x7fd12400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.806+0000 7fd122ffd640 1 -- 172.21.15.38:0/3117909209 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1240052d0 con 0x7fd1340b3b80 2024-04-25T08:35:51.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/2371266339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4d015f610 con 0x7fb4d006cb60 2024-04-25T08:35:51.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7f33117f8640 1 -- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f330c0b1600 msgr2=0x7f330c0b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7f33117f8640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f330c0b1600 0x7f330c0b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7f33117f8640 1 -- 172.21.15.38:0/2565608226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32f8005d50 msgr2=0x7f330c157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:51.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7f33117f8640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32f8005d50 0x7f330c157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:51.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.805+0000 7f33117f8640 1 -- 172.21.15.38:0/2565608226 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c154440 con 0x7f330c155780 2024-04-25T08:35:51.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.806+0000 7fd122ffd640 1 -- 172.21.15.38:0/3117909209 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd124005490 con 0x7fd1340b3b80 2024-04-25T08:35:51.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.806+0000 7fd122ffd640 1 -- 172.21.15.38:0/3117909209 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124007260 con 0x7fd1340b3b80 2024-04-25T08:35:51.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.806+0000 7fd138b1d640 1 -- 172.21.15.38:0/3117909209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1341ebf30 con 0x7fd1340b3b80 2024-04-25T08:35:51.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.806+0000 7f33117f8640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c155780 0x7f330c155c00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f32fc011a00 tx=0x7f32fc011ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.806+0000 7f5b07fff640 1 -- 172.21.15.38:0/8598399 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b14044020 con 0x7f5b200ceba0 2024-04-25T08:35:51.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.806+0000 7fd138b1d640 1 -- 172.21.15.38:0/3117909209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1341ec4f0 con 0x7fd1340b3b80 2024-04-25T08:35:51.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.806+0000 7f33027fc640 1 -- 172.21.15.38:0/2565608226 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00faf0 con 0x7f330c155780 2024-04-25T08:35:51.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.806+0000 7f33027fc640 1 -- 172.21.15.38:0/2565608226 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32fc00fc90 con 0x7f330c155780 2024-04-25T08:35:51.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.806+0000 7f5b07fff640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b0c06b4d0 0x7f5b0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.806+0000 7f5b07fff640 1 -- 172.21.15.38:0/8598399 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b140aa650 con 0x7f5b200ceba0 2024-04-25T08:35:51.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.807+0000 7f5b1f7fe640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b0c06b4d0 0x7f5b0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.807+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/2371266339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4d004ef90 con 0x7fb4d006cb60 2024-04-25T08:35:51.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.807+0000 7f33027fc640 1 -- 172.21.15.38:0/2565608226 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc004030 con 0x7f330c155780 2024-04-25T08:35:51.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.807+0000 7f331425d640 1 -- 172.21.15.38:0/2565608226 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330c1546e0 con 0x7f330c155780 2024-04-25T08:35:51.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.807+0000 7f331425d640 1 -- 172.21.15.38:0/2565608226 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330c154ca0 con 0x7f330c155780 2024-04-25T08:35:51.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.807+0000 7fd138b1d640 1 -- 172.21.15.38:0/3117909209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13404ef90 con 0x7fd1340b3b80 2024-04-25T08:35:51.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.808+0000 7fd122ffd640 1 -- 172.21.15.38:0/3117909209 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd124007400 con 0x7fd1340b3b80 2024-04-25T08:35:51.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.808+0000 7f5b1f7fe640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b0c06b4d0 0x7f5b0c06d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5b10006fd0 tx=0x7f5b10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.808+0000 7fb4cd7fa640 1 -- 172.21.15.38:0/2371266339 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4c4044020 con 0x7fb4d006cb60 2024-04-25T08:35:51.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.808+0000 7fd122ffd640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd11406b4d0 0x7fd11406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.808+0000 7fd122ffd640 1 -- 172.21.15.38:0/3117909209 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd124089630 con 0x7fd1340b3b80 2024-04-25T08:35:51.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.808+0000 7f331425d640 1 -- 172.21.15.38:0/2565608226 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330c04ef90 con 0x7f330c155780 2024-04-25T08:35:51.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.809+0000 7fb4cd7fa640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb4b406b5a0 0x7fb4b406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.809+0000 7fd131d9b640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd11406b4d0 0x7fd11406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.809+0000 7fb4cd7fa640 1 -- 172.21.15.38:0/2371266339 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb4c40aae90 con 0x7fb4d006cb60 2024-04-25T08:35:51.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.809+0000 7fb4cffff640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb4b406b5a0 0x7fb4b406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.809+0000 7f33027fc640 1 -- 172.21.15.38:0/2565608226 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32fc012040 con 0x7f330c155780 2024-04-25T08:35:51.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.809+0000 7fd131d9b640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd11406b4d0 0x7fd11406d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd128000c00 tx=0x7fd12802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.809+0000 7fb4cffff640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb4b406b5a0 0x7fb4b406da60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc007fd0 tx=0x7fb4bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.809+0000 7f33027fc640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32f006b500 0x7f32f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:51.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.810+0000 7f33027fc640 1 -- 172.21.15.38:0/2565608226 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32fc08a6f0 con 0x7f330c155780 2024-04-25T08:35:51.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.810+0000 7f3311ff9640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32f006b500 0x7f32f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:51.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.810+0000 7f3311ff9640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32f006b500 0x7f32f006d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f330800c540 tx=0x7f3308009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:51.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.812+0000 7f5b07fff640 1 -- 172.21.15.38:0/8598399 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b1404b030 con 0x7f5b200ceba0 2024-04-25T08:35:51.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.813+0000 7fd122ffd640 1 -- 172.21.15.38:0/3117909209 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd124057cd0 con 0x7fd1340b3b80 2024-04-25T08:35:51.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.815+0000 7f33027fc640 1 -- 172.21.15.38:0/2565608226 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32fc019070 con 0x7f330c155780 2024-04-25T08:35:51.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:51.817+0000 7fb4cd7fa640 1 -- 172.21.15.38:0/2371266339 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4c404b030 con 0x7fb4d006cb60 2024-04-25T08:35:52.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.068+0000 7f331425d640 1 -- 172.21.15.38:0/2565608226 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f330c0b3420 con 0x7f330c155780 2024-04-25T08:35:52.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.070+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/2371266339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fb4d0068050 con 0x7fb4d006cb60 2024-04-25T08:35:52.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.085+0000 7f5b260fc640 1 -- 172.21.15.38:0/8598399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f5b2015e7c0 con 0x7f5b200ceba0 2024-04-25T08:35:52.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.086+0000 7fd138b1d640 1 -- 172.21.15.38:0/3117909209 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7fd1340aee60 con 0x7fd1340b3b80 2024-04-25T08:35:52.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.330+0000 7f33027fc640 1 -- 172.21.15.38:0/2565608226 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 6142 v36) v1 ==== 95+0+0 (secure 0 0 0) 0x7f32fc058d90 con 0x7f330c155780 2024-04-25T08:35:52.324 INFO:teuthology.orchestra.run.smithi038.stderr:failed mds gid 6142 2024-04-25T08:35:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.332+0000 7f331425d640 1 -- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32f006b500 msgr2=0x7f32f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.332+0000 7f331425d640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32f006b500 0x7f32f006d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f330800c540 tx=0x7f3308009030 comp rx=0 tx=0).stop 2024-04-25T08:35:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.332+0000 7f331425d640 1 -- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c155780 msgr2=0x7f330c155c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.332+0000 7f331425d640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c155780 0x7f330c155c00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f32fc011a00 tx=0x7f32fc011ec0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.332+0000 7f331425d640 1 -- 172.21.15.38:0/2565608226 shutdown_connections 2024-04-25T08:35:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.332+0000 7f331425d640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f330c0b1600 0x7f330c0b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.332+0000 7f331425d640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32f006b500 0x7f32f006d9c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.332+0000 7f331425d640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f330c155780 0x7f330c155c00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.332+0000 7f331425d640 1 --2- 172.21.15.38:0/2565608226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32f8005d50 0x7f330c157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.332+0000 7f331425d640 1 -- 172.21.15.38:0/2565608226 >> 172.21.15.38:0/2565608226 conn(0x7f330c06abe0 msgr2=0x7f330c15b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.332+0000 7f331425d640 1 -- 172.21.15.38:0/2565608226 shutdown_connections 2024-04-25T08:35:52.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.332+0000 7f331425d640 1 -- 172.21.15.38:0/2565608226 wait complete. 2024-04-25T08:35:52.339 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:35:52.339 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:35:52.339 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:35:52.344+0000 7ff790feb640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:35:52.341 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:35:52.341 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:52.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.349+0000 7fb4cd7fa640 1 -- 172.21.15.38:0/2371266339 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 6145 v37) v1 ==== 95+0+0 (secure 0 0 0) 0x7fb4c4079bf0 con 0x7fb4d006cb60 2024-04-25T08:35:52.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.349+0000 7f5b07fff640 1 -- 172.21.15.38:0/8598399 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6813 v37) v1 ==== 95+0+0 (secure 0 0 0) 0x7f5b14079330 con 0x7f5b200ceba0 2024-04-25T08:35:52.344 INFO:teuthology.orchestra.run.smithi038.stderr:failed mds gid 6145 2024-04-25T08:35:52.344 INFO:teuthology.orchestra.run.smithi038.stderr:failed mds gid 6813 2024-04-25T08:35:52.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.351+0000 7f5b260fc640 1 -- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b0c06b4d0 msgr2=0x7f5b0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.351+0000 7f5b260fc640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b0c06b4d0 0x7f5b0c06d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5b10006fd0 tx=0x7f5b10008040 comp rx=0 tx=0).stop 2024-04-25T08:35:52.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.351+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb4b406b5a0 msgr2=0x7fb4b406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.351+0000 7fb4d6e9c640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb4b406b5a0 0x7fb4b406da60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc007fd0 tx=0x7fb4bc009040 comp rx=0 tx=0).stop 2024-04-25T08:35:52.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.351+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/2371266339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d006cb60 msgr2=0x7fb4d00b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.351+0000 7fb4d6e9c640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d006cb60 0x7fb4d00b5ad0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb4c40097f0 tx=0x7fb4c40046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.351+0000 7f5b260fc640 1 -- 172.21.15.38:0/8598399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b200ceba0 msgr2=0x7f5b20159630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:52.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.351+0000 7f5b260fc640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b200ceba0 0x7f5b20159630 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009870 tx=0x7f5b140046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.351+0000 7f5b260fc640 1 -- 172.21.15.38:0/8598399 shutdown_connections 2024-04-25T08:35:52.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7f5b260fc640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b0c06b4d0 0x7f5b0c06d990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7f5b260fc640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b2015a600 0x7f5b2015a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7f5b260fc640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b200ceba0 0x7f5b20159630 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7f5b260fc640 1 --2- 172.21.15.38:0/8598399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b200714c0 0x7f5b2015d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7f5b260fc640 1 -- 172.21.15.38:0/8598399 >> 172.21.15.38:0/8598399 conn(0x7f5b2006abe0 msgr2=0x7f5b200cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/2371266339 shutdown_connections 2024-04-25T08:35:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7fb4d6e9c640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb4b406b5a0 0x7fb4b406da60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7fb4d6e9c640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4d015dbb0 0x7fb4d014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7fb4d6e9c640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4d00b1510 0x7fb4d014f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7fb4d6e9c640 1 --2- 172.21.15.38:0/2371266339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4d006cb60 0x7fb4d00b5ad0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:52.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/2371266339 >> 172.21.15.38:0/2371266339 conn(0x7fb4d00d15e0 msgr2=0x7fb4d015cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:52.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/2371266339 shutdown_connections 2024-04-25T08:35:52.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7fb4d6e9c640 1 -- 172.21.15.38:0/2371266339 wait complete. 2024-04-25T08:35:52.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7f5b260fc640 1 -- 172.21.15.38:0/8598399 shutdown_connections 2024-04-25T08:35:52.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:52.352+0000 7f5b260fc640 1 -- 172.21.15.38:0/8598399 wait complete. 2024-04-25T08:35:52.359 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:35:52.359 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:35:52.360 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:35:52.360 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:35:52.362 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:35:52.363 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:52.402 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:35:52.402 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:52.421 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:35:52.438 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-25T08:35:52.477 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:35:53.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.353+0000 7fd122ffd640 1 -- 172.21.15.38:0/3117909209 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6810 v38) v1 ==== 95+0+0 (secure 0 0 0) 0x7fd12405bd70 con 0x7fd1340b3b80 2024-04-25T08:35:53.348 INFO:teuthology.orchestra.run.smithi038.stderr:failed mds gid 6810 2024-04-25T08:35:53.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.356+0000 7fd138b1d640 1 -- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd11406b4d0 msgr2=0x7fd11406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:53.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.356+0000 7fd138b1d640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd11406b4d0 0x7fd11406d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd128000c00 tx=0x7fd12802e000 comp rx=0 tx=0).stop 2024-04-25T08:35:53.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.356+0000 7fd138b1d640 1 -- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1340b3b80 msgr2=0x7fd1341ecfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:53.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.356+0000 7fd138b1d640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1340b3b80 0x7fd1341ecfd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd12400c3c0 tx=0x7fd12400c880 comp rx=0 tx=0).stop 2024-04-25T08:35:53.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.356+0000 7fd138b1d640 1 -- 172.21.15.38:0/3117909209 shutdown_connections 2024-04-25T08:35:53.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.356+0000 7fd138b1d640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd11406b4d0 0x7fd11406d990 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.356+0000 7fd138b1d640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd134161e00 0x7fd1341eb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.356+0000 7fd138b1d640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1340cbe10 0x7fd13414c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.356+0000 7fd138b1d640 1 --2- 172.21.15.38:0/3117909209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1340b3b80 0x7fd1341ecfd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.356+0000 7fd138b1d640 1 -- 172.21.15.38:0/3117909209 >> 172.21.15.38:0/3117909209 conn(0x7fd13406abe0 msgr2=0x7fd134073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:53.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.357+0000 7fd138b1d640 1 -- 172.21.15.38:0/3117909209 shutdown_connections 2024-04-25T08:35:53.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.357+0000 7fd138b1d640 1 -- 172.21.15.38:0/3117909209 wait complete. 2024-04-25T08:35:53.368 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:35:53.368 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:35:53.411 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:35:53.411 DEBUG:teuthology.parallel:result is None 2024-04-25T08:35:53.411 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:53.494 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:35:53.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.527+0000 7f4be7fce640 1 Processor -- start 2024-04-25T08:35:53.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.527+0000 7f4be7fce640 1 -- start start 2024-04-25T08:35:53.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.528+0000 7f4be7fce640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4be015da90 0x7f4be00d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:53.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.528+0000 7f4be7fce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4be00d3fb0 0x7f4be00b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.528+0000 7f4be7fce640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be00d1b60 con 0x7f4be00ceca0 2024-04-25T08:35:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.528+0000 7f4be7fce640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be00d2bc0 con 0x7f4be015da90 2024-04-25T08:35:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.528+0000 7f4be7fce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00d1e00 con 0x7f4be00d3fb0 2024-04-25T08:35:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.529+0000 7f4be5d6a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4be015da90 0x7f4be00d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.529+0000 7f4be5569640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f4be00ceca0 0x7f4be00d38a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:60302/0 (socket says 172.21.15.38:60302) 2024-04-25T08:35:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.529+0000 7f4be5569640 1 -- 172.21.15.38:0/2065902915 learned_addr learned my addr 172.21.15.38:0/2065902915 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.529+0000 7f4be656b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4be00d3fb0 0x7f4be00b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.529+0000 7f4be5d6a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4be015da90 0x7f4be00d2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:54226/0 (socket says 172.21.15.38:54226) 2024-04-25T08:35:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.529+0000 7f4be4d68640 1 -- 172.21.15.38:0/2065902915 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1048558880 0 0) 0x7f4be00d1b60 con 0x7f4be00ceca0 2024-04-25T08:35:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.529+0000 7f4be4d68640 1 -- 172.21.15.38:0/2065902915 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bcc003520 con 0x7f4be00ceca0 2024-04-25T08:35:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.529+0000 7f4be5d6a640 1 -- 172.21.15.38:0/2065902915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4be00d3fb0 msgr2=0x7f4be00b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:53.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.529+0000 7f4be5d6a640 1 --2- 172.21.15.38:0/2065902915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4be00d3fb0 0x7f4be00b69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.529+0000 7f4be5d6a640 1 -- 172.21.15.38:0/2065902915 >> v1:172.21.15.107:6789/0 conn(0x7f4be00ceca0 legacy=0x7f4be00d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:53.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.529+0000 7f4be5d6a640 1 -- 172.21.15.38:0/2065902915 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00d20e0 con 0x7f4be015da90 2024-04-25T08:35:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.530+0000 7f4be5d6a640 1 --2- 172.21.15.38:0/2065902915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be015da90 0x7f4be00d2f70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0097a0 tx=0x7f4bdc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e508053f4f17b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.530+0000 7f4be4d68640 1 -- 172.21.15.38:0/2065902915 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc037070 con 0x7f4be015da90 2024-04-25T08:35:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.530+0000 7f4be4d68640 1 -- 172.21.15.38:0/2065902915 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc032440 con 0x7f4be015da90 2024-04-25T08:35:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.530+0000 7f4be4d68640 1 -- 172.21.15.38:0/2065902915 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc032810 con 0x7f4be015da90 2024-04-25T08:35:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.531+0000 7f4be7fce640 1 -- 172.21.15.38:0/2065902915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be015da90 msgr2=0x7f4be00d2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.531+0000 7f4be7fce640 1 --2- 172.21.15.38:0/2065902915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be015da90 0x7f4be00d2f70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc0097a0 tx=0x7f4bdc02a180 comp rx=0 tx=0).stop 2024-04-25T08:35:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.531+0000 7f4be7fce640 1 -- 172.21.15.38:0/2065902915 shutdown_connections 2024-04-25T08:35:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.531+0000 7f4be7fce640 1 --2- 172.21.15.38:0/2065902915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4be00d3fb0 0x7f4be00b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.531+0000 7f4be7fce640 1 --2- 172.21.15.38:0/2065902915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be015da90 0x7f4be00d2f70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.531+0000 7f4be7fce640 1 -- 172.21.15.38:0/2065902915 >> 172.21.15.38:0/2065902915 conn(0x7f4be006bed0 msgr2=0x7f4be006c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:53.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.531+0000 7f4be7fce640 1 -- 172.21.15.38:0/2065902915 shutdown_connections 2024-04-25T08:35:53.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.531+0000 7f4be7fce640 1 -- 172.21.15.38:0/2065902915 wait complete. 2024-04-25T08:35:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.532+0000 7f4be7fce640 1 Processor -- start 2024-04-25T08:35:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.532+0000 7f4be7fce640 1 -- start start 2024-04-25T08:35:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be7fce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be00d3fb0 0x7f4be01e8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be7fce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be015da90 0x7f4be00c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be7fce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4be00cd2f0 0x7f4be015f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be7fce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be015dff0 con 0x7f4be015da90 2024-04-25T08:35:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be7fce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be015de70 con 0x7f4be00d3fb0 2024-04-25T08:35:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be7fce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00d34b0 con 0x7f4be00cd2f0 2024-04-25T08:35:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be5569640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be015da90 0x7f4be00c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be5d6a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be00d3fb0 0x7f4be01e8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:53.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be5569640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be015da90 0x7f4be00c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34150/0 (socket says 172.21.15.38:34150) 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be5569640 1 -- 172.21.15.38:0/287244915 learned_addr learned my addr 172.21.15.38:0/287244915 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be656b640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4be00cd2f0 0x7f4be015f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be5d6a640 1 -- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4be00cd2f0 msgr2=0x7f4be015f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be5d6a640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4be00cd2f0 0x7f4be015f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be5d6a640 1 -- 172.21.15.38:0/287244915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be015da90 msgr2=0x7f4be00c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be5d6a640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be015da90 0x7f4be00c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be5d6a640 1 -- 172.21.15.38:0/287244915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00c7b50 con 0x7f4be00d3fb0 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be5569640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be015da90 0x7f4be00c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.533+0000 7f4be5d6a640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be00d3fb0 0x7f4be01e8cd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc031000 tx=0x7f4bdc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.534+0000 7f4bd67fc640 1 -- 172.21.15.38:0/287244915 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc041030 con 0x7f4be00d3fb0 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.534+0000 7f4bd67fc640 1 -- 172.21.15.38:0/287244915 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc03d4e0 con 0x7f4be00d3fb0 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.534+0000 7f4be7fce640 1 -- 172.21.15.38:0/287244915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be00c7e30 con 0x7f4be00d3fb0 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.534+0000 7f4bd67fc640 1 -- 172.21.15.38:0/287244915 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc037040 con 0x7f4be00d3fb0 2024-04-25T08:35:53.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.534+0000 7f4be7fce640 1 -- 172.21.15.38:0/287244915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be00cd7a0 con 0x7f4be00d3fb0 2024-04-25T08:35:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.535+0000 7f4be7fce640 1 -- 172.21.15.38:0/287244915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be00c8350 con 0x7f4be00d3fb0 2024-04-25T08:35:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.535+0000 7f4bd67fc640 1 -- 172.21.15.38:0/287244915 <== mon.1 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bdc03d8b0 con 0x7f4be00d3fb0 2024-04-25T08:35:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.536+0000 7f4bd67fc640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bc406b4d0 0x7f4bc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.536+0000 7f4bd67fc640 1 -- 172.21.15.38:0/287244915 <== mon.1 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4bdc0aaa00 con 0x7f4be00d3fb0 2024-04-25T08:35:53.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.536+0000 7f4be5569640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bc406b4d0 0x7f4bc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:53.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.536+0000 7f4be5569640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bc406b4d0 0x7f4bc406d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4be00cdc10 tx=0x7f4bd0007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:53.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.540+0000 7f4bd67fc640 1 -- 172.21.15.38:0/287244915 <== mon.1 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4be00c8350 con 0x7f4be00d3fb0 2024-04-25T08:35:53.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.757+0000 7f4be7fce640 1 -- 172.21.15.38:0/287244915 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4be0068e80 con 0x7f4be00d3fb0 2024-04-25T08:35:53.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.758+0000 7f4bd67fc640 1 -- 172.21.15.38:0/287244915 <== mon.1 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4152 (secure 0 0 0) 0x7f4bdc04b020 con 0x7f4be00d3fb0 2024-04-25T08:35:53.752 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:53.752 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/1366115592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1366115592},{"type":"v1","addr":"172.21.15.107:6837","nonce":1366115592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6379,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/397891417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":397891417},{"type":"v1","addr":"172.21.15.107:6835","nonce":397891417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:40.623355+0000","modified":"2024-04-25T08:35:53.353707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7080":{"gid":7080,"name":"a","rank":0,"incarnation":39,"state":"up:replay","state_seq":1,"addr":"172.21.15.38:6833/1952521723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1952521723},{"type":"v1","addr":"172.21.15.38:6833","nonce":1952521723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:35:53.752 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 39 2024-04-25T08:35:53.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.759+0000 7f4be7fce640 1 -- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bc406b4d0 msgr2=0x7f4bc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:53.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.759+0000 7f4be7fce640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bc406b4d0 0x7f4bc406d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4be00cdc10 tx=0x7f4bd0007d70 comp rx=0 tx=0).stop 2024-04-25T08:35:53.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.760+0000 7f4be7fce640 1 -- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be00d3fb0 msgr2=0x7f4be01e8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:53.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.760+0000 7f4be7fce640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be00d3fb0 0x7f4be01e8cd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc031000 tx=0x7f4bdc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.760+0000 7f4be7fce640 1 -- 172.21.15.38:0/287244915 shutdown_connections 2024-04-25T08:35:53.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.760+0000 7f4be7fce640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bc406b4d0 0x7f4bc406d990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.760+0000 7f4be7fce640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4be00cd2f0 0x7f4be015f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.760+0000 7f4be7fce640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4be015da90 0x7f4be00c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.760+0000 7f4be7fce640 1 --2- 172.21.15.38:0/287244915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4be00d3fb0 0x7f4be01e8cd0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:53.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.760+0000 7f4be7fce640 1 -- 172.21.15.38:0/287244915 >> 172.21.15.38:0/287244915 conn(0x7f4be006bed0 msgr2=0x7f4be0159840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:53.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.760+0000 7f4be7fce640 1 -- 172.21.15.38:0/287244915 shutdown_connections 2024-04-25T08:35:53.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:53.760+0000 7f4be7fce640 1 -- 172.21.15.38:0/287244915 wait complete. 2024-04-25T08:35:54.168 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:35:40.623355+0000', 'modified': '2024-04-25T08:35:53.353707+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7080}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7080': {'gid': 7080, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/1952521723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 1952521723}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 1952521723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:35:54.168 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7080:up:replay 2024-04-25T08:35:55.169 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:55.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.289+0000 7fa79d8d2640 1 Processor -- start 2024-04-25T08:35:55.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.289+0000 7fa79d8d2640 1 -- start start 2024-04-25T08:35:55.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa79d8d2640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa79815dc10 0x7fa7980ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:55.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa79d8d2640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa7980ced60 0x7fa7980d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:55.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa79d8d2640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7980d3e70 con 0x7fa7980d3900 2024-04-25T08:35:55.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa79d8d2640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa79815dff0 con 0x7fa79815dc10 2024-04-25T08:35:55.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa79d8d2640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7980b6990 con 0x7fa7980ced60 2024-04-25T08:35:55.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa796ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa79815dc10 0x7fa7980ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:55.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa7977fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa7980d3900 0x7fa7980b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32892/0 (socket says 172.21.15.38:32892) 2024-04-25T08:35:55.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa7977fe640 1 -- 172.21.15.38:0/453242334 learned_addr learned my addr 172.21.15.38:0/453242334 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:55.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa78ffff640 1 --2- 172.21.15.38:0/453242334 >> v2:172.21.15.107:3300/0 conn(0x7fa7980ced60 0x7fa7980d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:55.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa796ffd640 1 -- 172.21.15.38:0/453242334 >> v2:172.21.15.107:3300/0 conn(0x7fa7980ced60 msgr2=0x7fa7980d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:55.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa796ffd640 1 --2- 172.21.15.38:0/453242334 >> v2:172.21.15.107:3300/0 conn(0x7fa7980ced60 0x7fa7980d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:55.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa796ffd640 1 -- 172.21.15.38:0/453242334 >> v1:172.21.15.38:6789/0 conn(0x7fa7980d3900 legacy=0x7fa7980b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:55.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa7967fc640 1 -- 172.21.15.38:0/453242334 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648345505 0 0) 0x7fa7980d3e70 con 0x7fa7980d3900 2024-04-25T08:35:55.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.290+0000 7fa796ffd640 1 -- 172.21.15.38:0/453242334 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7980d4150 con 0x7fa79815dc10 2024-04-25T08:35:55.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.291+0000 7fa796ffd640 1 --2- 172.21.15.38:0/453242334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa79815dc10 0x7fa7980ce820 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa7980b9600 tx=0x7fa78802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c043dcd24db18d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:55.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.291+0000 7fa7967fc640 1 -- 172.21.15.38:0/453242334 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788037070 con 0x7fa79815dc10 2024-04-25T08:35:55.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.291+0000 7fa7967fc640 1 -- 172.21.15.38:0/453242334 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa788032440 con 0x7fa79815dc10 2024-04-25T08:35:55.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.291+0000 7fa7967fc640 1 -- 172.21.15.38:0/453242334 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78802ae60 con 0x7fa79815dc10 2024-04-25T08:35:55.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.292+0000 7fa79d8d2640 1 -- 172.21.15.38:0/453242334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa79815dc10 msgr2=0x7fa7980ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:55.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.292+0000 7fa79d8d2640 1 --2- 172.21.15.38:0/453242334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa79815dc10 0x7fa7980ce820 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fa7980b9600 tx=0x7fa78802a110 comp rx=0 tx=0).stop 2024-04-25T08:35:55.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.292+0000 7fa79d8d2640 1 -- 172.21.15.38:0/453242334 shutdown_connections 2024-04-25T08:35:55.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.292+0000 7fa79d8d2640 1 --2- 172.21.15.38:0/453242334 >> v2:172.21.15.107:3300/0 conn(0x7fa7980ced60 0x7fa7980d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:55.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.292+0000 7fa79d8d2640 1 --2- 172.21.15.38:0/453242334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa79815dc10 0x7fa7980ce820 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:55.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.292+0000 7fa79d8d2640 1 -- 172.21.15.38:0/453242334 >> 172.21.15.38:0/453242334 conn(0x7fa798066d10 msgr2=0x7fa798067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:55.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.292+0000 7fa79d8d2640 1 -- 172.21.15.38:0/453242334 shutdown_connections 2024-04-25T08:35:55.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.292+0000 7fa79d8d2640 1 -- 172.21.15.38:0/453242334 wait complete. 2024-04-25T08:35:55.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.293+0000 7fa79d8d2640 1 Processor -- start 2024-04-25T08:35:55.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.293+0000 7fa79d8d2640 1 -- start start 2024-04-25T08:35:55.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa79d8d2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7980ced60 0x7fa7980ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:55.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa79d8d2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7980d3900 0x7fa79814ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:55.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa79d8d2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa79815dc10 0x7fa7980c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:55.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa79d8d2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980b6bc0 con 0x7fa7980ced60 2024-04-25T08:35:55.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa79d8d2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980d4710 con 0x7fa7980d3900 2024-04-25T08:35:55.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa79d8d2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7980d0d90 con 0x7fa79815dc10 2024-04-25T08:35:55.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa796ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7980ced60 0x7fa7980ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:55.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa796ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7980ced60 0x7fa7980ca230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34170/0 (socket says 172.21.15.38:34170) 2024-04-25T08:35:55.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa796ffd640 1 -- 172.21.15.38:0/1079841372 learned_addr learned my addr 172.21.15.38:0/1079841372 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa7977fe640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa79815dc10 0x7fa7980c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa78ffff640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7980d3900 0x7fa79814ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa796ffd640 1 -- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa79815dc10 msgr2=0x7fa7980c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa796ffd640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa79815dc10 0x7fa7980c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa796ffd640 1 -- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7980d3900 msgr2=0x7fa79814ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa796ffd640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7980d3900 0x7fa79814ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa796ffd640 1 -- 172.21.15.38:0/1079841372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7980c9190 con 0x7fa7980ced60 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa78ffff640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7980d3900 0x7fa79814ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.294+0000 7fa796ffd640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7980ced60 0x7fa7980ca230 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa788000c00 tx=0x7fa7880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.295+0000 7fa794ff9640 1 -- 172.21.15.38:0/1079841372 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788037050 con 0x7fa7980ced60 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.295+0000 7fa794ff9640 1 -- 172.21.15.38:0/1079841372 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa788038040 con 0x7fa7980ced60 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.295+0000 7fa79d8d2640 1 -- 172.21.15.38:0/1079841372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7980c9470 con 0x7fa7980ced60 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.295+0000 7fa794ff9640 1 -- 172.21.15.38:0/1079841372 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa788004450 con 0x7fa7980ced60 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.295+0000 7fa79d8d2640 1 -- 172.21.15.38:0/1079841372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7980c9a30 con 0x7fa7980ced60 2024-04-25T08:35:55.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.296+0000 7fa79d8d2640 1 -- 172.21.15.38:0/1079841372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa798068e80 con 0x7fa7980ced60 2024-04-25T08:35:55.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.296+0000 7fa794ff9640 1 -- 172.21.15.38:0/1079841372 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa788030980 con 0x7fa7980ced60 2024-04-25T08:35:55.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.297+0000 7fa794ff9640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa77806b4d0 0x7fa77806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:55.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.297+0000 7fa794ff9640 1 -- 172.21.15.38:0/1079841372 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa7880ab280 con 0x7fa7980ced60 2024-04-25T08:35:55.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.297+0000 7fa78ffff640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa77806b4d0 0x7fa77806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:55.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.297+0000 7fa78ffff640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa77806b4d0 0x7fa77806d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa7980aaa10 tx=0x7fa780008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:55.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.301+0000 7fa794ff9640 1 -- 172.21.15.38:0/1079841372 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa788079fb0 con 0x7fa7980ced60 2024-04-25T08:35:55.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.523+0000 7fa79d8d2640 1 -- 172.21.15.38:0/1079841372 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa798067ed0 con 0x7fa7980ced60 2024-04-25T08:35:55.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.523+0000 7fa794ff9640 1 -- 172.21.15.38:0/1079841372 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa7880798b0 con 0x7fa7980ced60 2024-04-25T08:35:55.518 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:55.518 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":41,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/1366115592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1366115592},{"type":"v1","addr":"172.21.15.107:6837","nonce":1366115592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6379,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/397891417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":397891417},{"type":"v1","addr":"172.21.15.107:6835","nonce":397891417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1355647168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1355647168},{"type":"v1","addr":"172.21.15.38:6835","nonce":1355647168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:40.623355+0000","modified":"2024-04-25T08:35:54.370083+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7080":{"gid":7080,"name":"a","rank":0,"incarnation":39,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.38:6833/1952521723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1952521723},{"type":"v1","addr":"172.21.15.38:6833","nonce":1952521723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:35:55.519 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 41 2024-04-25T08:35:55.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.526+0000 7fa79d8d2640 1 -- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa77806b4d0 msgr2=0x7fa77806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.526+0000 7fa79d8d2640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa77806b4d0 0x7fa77806d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa7980aaa10 tx=0x7fa780008040 comp rx=0 tx=0).stop 2024-04-25T08:35:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.526+0000 7fa79d8d2640 1 -- 172.21.15.38:0/1079841372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7980ced60 msgr2=0x7fa7980ca230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.526+0000 7fa79d8d2640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7980ced60 0x7fa7980ca230 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa788000c00 tx=0x7fa7880046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.526+0000 7fa79d8d2640 1 -- 172.21.15.38:0/1079841372 shutdown_connections 2024-04-25T08:35:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.526+0000 7fa79d8d2640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa77806b4d0 0x7fa77806d990 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.526+0000 7fa79d8d2640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa79815dc10 0x7fa7980c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.526+0000 7fa79d8d2640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa7980d3900 0x7fa79814ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.526+0000 7fa79d8d2640 1 --2- 172.21.15.38:0/1079841372 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa7980ced60 0x7fa7980ca230 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.526+0000 7fa79d8d2640 1 -- 172.21.15.38:0/1079841372 >> 172.21.15.38:0/1079841372 conn(0x7fa798066d10 msgr2=0x7fa7980b6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.526+0000 7fa79d8d2640 1 -- 172.21.15.38:0/1079841372 shutdown_connections 2024-04-25T08:35:55.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:55.526+0000 7fa79d8d2640 1 -- 172.21.15.38:0/1079841372 wait complete. 2024-04-25T08:35:55.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:35:40.623355+0000', 'modified': '2024-04-25T08:35:54.370083+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7080}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7080': {'gid': 7080, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.38:6833/1952521723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 1952521723}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 1952521723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:35:55.934 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7080:up:rejoin 2024-04-25T08:35:56.935 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:35:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.055+0000 7f715341f640 1 Processor -- start 2024-04-25T08:35:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.055+0000 7f715341f640 1 -- start start 2024-04-25T08:35:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f715341f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f714c06c110 0x7f714c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f715341f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f714c0ce2e0 0x7f714c0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f715341f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f714c06ca30 con 0x7f714c15df50 2024-04-25T08:35:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f715341f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714c0cf2d0 con 0x7f714c06c110 2024-04-25T08:35:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f715341f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714c0ceff0 con 0x7f714c0ce2e0 2024-04-25T08:35:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f71511bb640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f714c06c110 0x7f714c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f71509ba640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f714c15df50 0x7f714c0ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32896/0 (socket says 172.21.15.38:32896) 2024-04-25T08:35:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f71511bb640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f714c06c110 0x7f714c06c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34178/0 (socket says 172.21.15.38:34178) 2024-04-25T08:35:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f71509ba640 1 -- 172.21.15.38:0/3668533823 learned_addr learned my addr 172.21.15.38:0/3668533823 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f71519bc640 1 --2- 172.21.15.38:0/3668533823 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f714c0ce2e0 0x7f714c0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f71511bb640 1 -- 172.21.15.38:0/3668533823 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f714c0ce2e0 msgr2=0x7f714c0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f71511bb640 1 --2- 172.21.15.38:0/3668533823 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f714c0ce2e0 0x7f714c0d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f71511bb640 1 -- 172.21.15.38:0/3668533823 >> v1:172.21.15.38:6789/0 conn(0x7f714c15df50 legacy=0x7f714c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f71511bb640 1 -- 172.21.15.38:0/3668533823 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c0d1f70 con 0x7f714c06c110 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.056+0000 7f7143fff640 1 -- 172.21.15.38:0/3668533823 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223501355 0 0) 0x7f714c06ca30 con 0x7f714c15df50 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.057+0000 7f71511bb640 1 --2- 172.21.15.38:0/3668533823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f714c06c110 0x7f714c06c4f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f71480097a0 tx=0x7f714802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f821321f36426c43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.057+0000 7f7143fff640 1 -- 172.21.15.38:0/3668533823 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148037070 con 0x7f714c06c110 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.057+0000 7f7143fff640 1 -- 172.21.15.38:0/3668533823 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7148032440 con 0x7f714c06c110 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.057+0000 7f7143fff640 1 -- 172.21.15.38:0/3668533823 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148032810 con 0x7f714c06c110 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.058+0000 7f715341f640 1 -- 172.21.15.38:0/3668533823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f714c06c110 msgr2=0x7f714c06c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.058+0000 7f715341f640 1 --2- 172.21.15.38:0/3668533823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f714c06c110 0x7f714c06c4f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f71480097a0 tx=0x7f714802a180 comp rx=0 tx=0).stop 2024-04-25T08:35:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.058+0000 7f715341f640 1 -- 172.21.15.38:0/3668533823 shutdown_connections 2024-04-25T08:35:57.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.058+0000 7f715341f640 1 --2- 172.21.15.38:0/3668533823 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f714c0ce2e0 0x7f714c0d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.058+0000 7f715341f640 1 --2- 172.21.15.38:0/3668533823 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f714c06c110 0x7f714c06c4f0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.058+0000 7f715341f640 1 -- 172.21.15.38:0/3668533823 >> 172.21.15.38:0/3668533823 conn(0x7f714c0d3fb0 msgr2=0x7f714c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:57.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.058+0000 7f715341f640 1 -- 172.21.15.38:0/3668533823 shutdown_connections 2024-04-25T08:35:57.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.058+0000 7f715341f640 1 -- 172.21.15.38:0/3668533823 wait complete. 2024-04-25T08:35:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.059+0000 7f715341f640 1 Processor -- start 2024-04-25T08:35:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.059+0000 7f715341f640 1 -- start start 2024-04-25T08:35:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f715341f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f714c06c110 0x7f714c0cae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f715341f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f714c0ce2e0 0x7f714c14dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f715341f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f714c15df50 0x7f714c15f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f715341f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c0d2f90 con 0x7f714c06c110 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f715341f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f714c0d2530 con 0x7f714c15df50 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f715341f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714c0d3110 con 0x7f714c0ce2e0 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f71511bb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f714c06c110 0x7f714c0cae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f71511bb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f714c06c110 0x7f714c0cae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34188/0 (socket says 172.21.15.38:34188) 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f71511bb640 1 -- 172.21.15.38:0/95511556 learned_addr learned my addr 172.21.15.38:0/95511556 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f71509ba640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f714c0ce2e0 0x7f714c14dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f71519bc640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f714c15df50 0x7f714c15f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f71511bb640 1 -- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f714c0ce2e0 msgr2=0x7f714c14dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f71511bb640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f714c0ce2e0 0x7f714c14dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f71511bb640 1 -- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f714c15df50 msgr2=0x7f714c15f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f71511bb640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f714c15df50 0x7f714c15f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f71511bb640 1 -- 172.21.15.38:0/95511556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714c14d700 con 0x7f714c06c110 2024-04-25T08:35:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.060+0000 7f71519bc640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f714c15df50 0x7f714c15f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.061+0000 7f71511bb640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f714c06c110 0x7f714c0cae40 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f71480098c0 tx=0x7f71480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.061+0000 7f7141ffb640 1 -- 172.21.15.38:0/95511556 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148037040 con 0x7f714c06c110 2024-04-25T08:35:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.061+0000 7f7141ffb640 1 -- 172.21.15.38:0/95511556 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7148042070 con 0x7f714c06c110 2024-04-25T08:35:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.061+0000 7f715341f640 1 -- 172.21.15.38:0/95511556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f714c15e8a0 con 0x7f714c06c110 2024-04-25T08:35:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.061+0000 7f7141ffb640 1 -- 172.21.15.38:0/95511556 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7148030bb0 con 0x7f714c06c110 2024-04-25T08:35:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.061+0000 7f715341f640 1 -- 172.21.15.38:0/95511556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f714c15ee60 con 0x7f714c06c110 2024-04-25T08:35:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.062+0000 7f715341f640 1 -- 172.21.15.38:0/95511556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f714c068e80 con 0x7f714c06c110 2024-04-25T08:35:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.063+0000 7f7141ffb640 1 -- 172.21.15.38:0/95511556 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7148047460 con 0x7f714c06c110 2024-04-25T08:35:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.063+0000 7f7141ffb640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f713006b4d0 0x7f713006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.063+0000 7f7141ffb640 1 -- 172.21.15.38:0/95511556 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f71480aae70 con 0x7f714c06c110 2024-04-25T08:35:57.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.063+0000 7f71509ba640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f713006b4d0 0x7f713006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.064+0000 7f71509ba640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f713006b4d0 0x7f713006d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f714c0ca2a0 tx=0x7f713c007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.067+0000 7f7141ffb640 1 -- 172.21.15.38:0/95511556 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7148032e20 con 0x7f714c06c110 2024-04-25T08:35:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.288+0000 7f715341f640 1 -- 172.21.15.38:0/95511556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f714c06c690 con 0x7f714c06c110 2024-04-25T08:35:57.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.289+0000 7f7141ffb640 1 -- 172.21.15.38:0/95511556 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f7148079b10 con 0x7f714c06c110 2024-04-25T08:35:57.284 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:35:57.284 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/1366115592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1366115592},{"type":"v1","addr":"172.21.15.107:6837","nonce":1366115592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6379,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/397891417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":397891417},{"type":"v1","addr":"172.21.15.107:6835","nonce":397891417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1355647168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1355647168},{"type":"v1","addr":"172.21.15.38:6835","nonce":1355647168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:40.623355+0000","modified":"2024-04-25T08:35:55.386447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7080":{"gid":7080,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/1952521723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1952521723},{"type":"v1","addr":"172.21.15.38:6833","nonce":1952521723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:35:57.284 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 42 2024-04-25T08:35:57.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.291+0000 7f715341f640 1 -- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f713006b4d0 msgr2=0x7f713006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.291+0000 7f715341f640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f713006b4d0 0x7f713006d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f714c0ca2a0 tx=0x7f713c007de0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.291+0000 7f715341f640 1 -- 172.21.15.38:0/95511556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f714c06c110 msgr2=0x7f714c0cae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.291+0000 7f715341f640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f714c06c110 0x7f714c0cae40 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f71480098c0 tx=0x7f71480046c0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.291+0000 7f715341f640 1 -- 172.21.15.38:0/95511556 shutdown_connections 2024-04-25T08:35:57.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.291+0000 7f715341f640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f714c15df50 0x7f714c15f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.291+0000 7f715341f640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f713006b4d0 0x7f713006d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.291+0000 7f715341f640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f714c0ce2e0 0x7f714c14dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.291+0000 7f715341f640 1 --2- 172.21.15.38:0/95511556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f714c06c110 0x7f714c0cae40 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.291+0000 7f715341f640 1 -- 172.21.15.38:0/95511556 >> 172.21.15.38:0/95511556 conn(0x7f714c0d3fb0 msgr2=0x7f714c0d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:57.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.291+0000 7f715341f640 1 -- 172.21.15.38:0/95511556 shutdown_connections 2024-04-25T08:35:57.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.291+0000 7f715341f640 1 -- 172.21.15.38:0/95511556 wait complete. 2024-04-25T08:35:57.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:35:40.623355+0000', 'modified': '2024-04-25T08:35:55.386447+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 70, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7080}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7080': {'gid': 7080, 'name': 'a', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.38:6833/1952521723', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 1952521723}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 1952521723}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:35:57.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:35:57.700 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:35:57.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.814+0000 7fdc721b7640 1 Processor -- start 2024-04-25T08:35:57.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.814+0000 7fdc721b7640 1 -- start start 2024-04-25T08:35:57.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.815+0000 7fdc721b7640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdc6c0b9180 0x7fdc6c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.815+0000 7fdc721b7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6c0b9610 con 0x7fdc6c0b2ab0 2024-04-25T08:35:57.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.815+0000 7fdc721b7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6c1616a0 con 0x7fdc6c160ef0 2024-04-25T08:35:57.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.815+0000 7fdc721b7640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6c161cf0 con 0x7fdc6c0b9180 2024-04-25T08:35:57.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.815+0000 7fdc6bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdc6c0b2ab0 0x7fdc6c0d1eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32910/0 (socket says 172.21.15.38:32910) 2024-04-25T08:35:57.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.815+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1851280478 learned_addr learned my addr 172.21.15.38:0/1851280478 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:57.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6affd640 1 --2- 172.21.15.38:0/1851280478 >> v2:172.21.15.107:3300/0 conn(0x7fdc6c0b9180 0x7fdc6c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406522329 0 0) 0x7fdc6c0b9610 con 0x7fdc6c0b2ab0 2024-04-25T08:35:57.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc54003520 con 0x7fdc6c0b2ab0 2024-04-25T08:35:57.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4221959637 0 0) 0x7fdc54003520 con 0x7fdc6c0b2ab0 2024-04-25T08:35:57.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 >> v2:172.21.15.107:3300/0 conn(0x7fdc6c0b9180 msgr2=0x7fdc6c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 --2- 172.21.15.38:0/1851280478 >> v2:172.21.15.107:3300/0 conn(0x7fdc6c0b9180 0x7fdc6c0d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 >> v1:172.21.15.107:6789/0 conn(0x7fdc6c160ef0 legacy=0x7fdc6c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c161980 con 0x7fdc6c0b2ab0 2024-04-25T08:35:57.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fdc640019f0 con 0x7fdc6c0b2ab0 2024-04-25T08:35:57.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 >> v1:172.21.15.38:6789/0 conn(0x7fdc6c0b2ab0 legacy=0x7fdc6c0d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 --2- 172.21.15.38:0/1851280478 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc540039b0 0x7fdc54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 --2- 172.21.15.38:0/1851280478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 --2- 172.21.15.38:0/1851280478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc540051c0 0x7fdc5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c161cf0 con 0x7fdc540051c0 2024-04-25T08:35:57.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5400c160 con 0x7fdc54004800 2024-04-25T08:35:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5400c360 con 0x7fdc540039b0 2024-04-25T08:35:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/1851280478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc540051c0 0x7fdc5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/1851280478 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc540039b0 0x7fdc54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.816+0000 7fdc6affd640 1 --2- 172.21.15.38:0/1851280478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.817+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1851280478 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc540039b0 msgr2=0x7fdc54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.817+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/1851280478 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc540039b0 0x7fdc54003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.817+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1851280478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc54004800 msgr2=0x7fdc54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.817+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/1851280478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc54004800 0x7fdc54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.817+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1851280478 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc54003520 con 0x7fdc540051c0 2024-04-25T08:35:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.817+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1851280478 reap_dead start 2024-04-25T08:35:57.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.817+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/1851280478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc540051c0 0x7fdc5400b9c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400a860 tx=0x7fdc54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d84c8fd9b3d8a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.817+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6400a480 con 0x7fdc540051c0 2024-04-25T08:35:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.817+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6402f070 con 0x7fdc540051c0 2024-04-25T08:35:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.817+0000 7fdc6a7fc640 1 -- 172.21.15.38:0/1851280478 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6402bcc0 con 0x7fdc540051c0 2024-04-25T08:35:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.818+0000 7fdc721b7640 1 -- 172.21.15.38:0/1851280478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc540051c0 msgr2=0x7fdc5400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.818+0000 7fdc721b7640 1 --2- 172.21.15.38:0/1851280478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc540051c0 0x7fdc5400b9c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400a860 tx=0x7fdc54005830 comp rx=0 tx=0).stop 2024-04-25T08:35:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.818+0000 7fdc721b7640 1 -- 172.21.15.38:0/1851280478 shutdown_connections 2024-04-25T08:35:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.818+0000 7fdc721b7640 1 --2- 172.21.15.38:0/1851280478 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc540051c0 0x7fdc5400b9c0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.818+0000 7fdc721b7640 1 -- 172.21.15.38:0/1851280478 >> 172.21.15.38:0/1851280478 conn(0x7fdc6c066d10 msgr2=0x7fdc6c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.818+0000 7fdc721b7640 1 -- 172.21.15.38:0/1851280478 shutdown_connections 2024-04-25T08:35:57.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.818+0000 7fdc721b7640 1 -- 172.21.15.38:0/1851280478 wait complete. 2024-04-25T08:35:57.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.819+0000 7fdc721b7640 1 Processor -- start 2024-04-25T08:35:57.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.820+0000 7fdc721b7640 1 -- start start 2024-04-25T08:35:57.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.820+0000 7fdc721b7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc540051c0 0x7fdc6c1f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.820+0000 7fdc721b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc6c1f7230 0x7fdc6c1f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.820+0000 7fdc721b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c14b5a0 0x7fdc6c14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.820+0000 7fdc721b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c1612d0 con 0x7fdc6c1f7230 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.820+0000 7fdc721b7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c162390 con 0x7fdc6c14b5a0 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.820+0000 7fdc721b7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0b2480 con 0x7fdc540051c0 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.820+0000 7fdc6affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc6c1f7230 0x7fdc6c1f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.820+0000 7fdc6affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc6c1f7230 0x7fdc6c1f76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34210/0 (socket says 172.21.15.38:34210) 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.820+0000 7fdc6affd640 1 -- 172.21.15.38:0/1639234118 learned_addr learned my addr 172.21.15.38:0/1639234118 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc540051c0 0x7fdc6c1f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c14b5a0 0x7fdc6c14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc6affd640 1 -- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc540051c0 msgr2=0x7fdc6c1f3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc6affd640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc540051c0 0x7fdc6c1f3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc6affd640 1 -- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c14b5a0 msgr2=0x7fdc6c14ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc6affd640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c14b5a0 0x7fdc6c14ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc6affd640 1 -- 172.21.15.38:0/1639234118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c1f3d40 con 0x7fdc6c1f7230 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc540051c0 0x7fdc6c1f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc6affd640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc6c1f7230 0x7fdc6c1f76b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fdc60011910 tx=0x7fdc60011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc53fff640 1 -- 172.21.15.38:0/1639234118 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6000faf0 con 0x7fdc6c1f7230 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc53fff640 1 -- 172.21.15.38:0/1639234118 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6000fc90 con 0x7fdc6c1f7230 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc53fff640 1 -- 172.21.15.38:0/1639234118 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc60014440 con 0x7fdc6c1f7230 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc721b7640 1 -- 172.21.15.38:0/1639234118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c1f4020 con 0x7fdc6c1f7230 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.821+0000 7fdc721b7640 1 -- 172.21.15.38:0/1639234118 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c1f6d10 con 0x7fdc6c1f7230 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.823+0000 7fdc53fff640 1 -- 172.21.15.38:0/1639234118 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc60012040 con 0x7fdc6c1f7230 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.823+0000 7fdc53fff640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdc5806b500 0x7fdc5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.823+0000 7fdc53fff640 1 -- 172.21.15.38:0/1639234118 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdc6008aa50 con 0x7fdc6c1f7230 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.823+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdc5806b500 0x7fdc5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.824+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdc5806b500 0x7fdc5806d9c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c000c00 tx=0x7fdc5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.824+0000 7fdc74f9a740 1 Processor -- start 2024-04-25T08:35:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.824+0000 7fdc74f9a740 1 -- start start 2024-04-25T08:35:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc74f9a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac8771230 0x556ac8771610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc74f9a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac878ead0 0x556ac878f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc74f9a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8746f10 0x556ac878d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc74f9a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac86a9a10 con 0x556ac8746f10 2024-04-25T08:35:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc74f9a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac8652b30 con 0x556ac8771230 2024-04-25T08:35:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc74f9a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ac86a5c50 con 0x556ac878ead0 2024-04-25T08:35:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc6bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8746f10 0x556ac878d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc6bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8746f10 0x556ac878d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34220/0 (socket says 172.21.15.38:34220) 2024-04-25T08:35:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1513658146 learned_addr learned my addr 172.21.15.38:0/1513658146 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:57.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/1513658146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac8771230 0x556ac8771610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc6affd640 1 --2- 172.21.15.38:0/1513658146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac878ead0 0x556ac878f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1513658146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac878ead0 msgr2=0x556ac878f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/1513658146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac878ead0 0x556ac878f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1513658146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac8771230 msgr2=0x556ac8771610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/1513658146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac8771230 0x556ac8771610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1513658146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ac8667e10 con 0x556ac8746f10 2024-04-25T08:35:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/1513658146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8746f10 0x556ac878d190 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fdc64000f80 tx=0x7fdc64009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c299d278f735484 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.825+0000 7fdc517fa640 1 -- 172.21.15.38:0/1513658146 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64044070 con 0x556ac8746f10 2024-04-25T08:35:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.826+0000 7fdc517fa640 1 -- 172.21.15.38:0/1513658146 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6402f040 con 0x556ac8746f10 2024-04-25T08:35:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.826+0000 7fdc517fa640 1 -- 172.21.15.38:0/1513658146 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc640019f0 con 0x556ac8746f10 2024-04-25T08:35:57.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.826+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1513658146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8746f10 msgr2=0x556ac878d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.826+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1513658146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8746f10 0x556ac878d190 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fdc64000f80 tx=0x7fdc64009030 comp rx=0 tx=0).stop 2024-04-25T08:35:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.826+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1513658146 shutdown_connections 2024-04-25T08:35:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.826+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1513658146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8746f10 0x556ac878d190 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.826+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1513658146 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac878ead0 0x556ac878f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.826+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1513658146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac8771230 0x556ac8771610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.826+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1513658146 >> 172.21.15.38:0/1513658146 conn(0x556ac8743840 msgr2=0x556ac878c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:57.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.826+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1513658146 shutdown_connections 2024-04-25T08:35:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.826+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1513658146 wait complete. 2024-04-25T08:35:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.827+0000 7fdc74f9a740 1 -- start start 2024-04-25T08:35:57.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.827+0000 7fdc74f9a740 1 Processor -- start 2024-04-25T08:35:57.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc74f9a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8771230 0x556ac8786270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc74f9a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac87830f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc74f9a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac879c2b0 0x556ac879c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc74f9a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac8747cb0 con 0x556ac8771230 2024-04-25T08:35:57.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc74f9a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac86660c0 con 0x556ac878ead0 2024-04-25T08:35:57.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc74f9a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ac8741770 con 0x556ac879c2b0 2024-04-25T08:35:57.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc74f9a740 10 client.0 _ll_register_callbacks cb 0x556ac8659ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:57.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc6b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8771230 0x556ac8786270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc6b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8771230 0x556ac8786270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34236/0 (socket says 172.21.15.38:34236) 2024-04-25T08:35:57.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc6b7fe640 1 -- 172.21.15.38:0/1802463217 learned_addr learned my addr 172.21.15.38:0/1802463217 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:57.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc6affd640 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac87830f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac879c2b0 0x556ac879c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc6b7fe640 1 -- 172.21.15.38:0/1802463217 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac879c2b0 msgr2=0x556ac879c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac879c2b0 0x556ac879c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc6b7fe640 1 -- 172.21.15.38:0/1802463217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 msgr2=0x556ac87830f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac87830f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc6b7fe640 1 -- 172.21.15.38:0/1802463217 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ac83be2d0 con 0x556ac8771230 2024-04-25T08:35:57.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8771230 0x556ac8786270 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c015750 tx=0x7fdc5c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76ca13d1beca290 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:57.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc4a7fc640 1 -- 172.21.15.38:0/1802463217 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c0145e0 con 0x556ac8771230 2024-04-25T08:35:57.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc4a7fc640 1 -- 172.21.15.38:0/1802463217 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdc5c014780 con 0x556ac8771230 2024-04-25T08:35:57.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.828+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1802463217 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc6402f040 con 0x556ac8771230 2024-04-25T08:35:57.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc4a7fc640 1 -- 172.21.15.38:0/1802463217 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc5c030050 con 0x556ac8771230 2024-04-25T08:35:57.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc74f9a740 10 client.7122 fetch_fsmap learned FSMap version 42 2024-04-25T08:35:57.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1802463217 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x556ac84886b0 con 0x556ac8771230 2024-04-25T08:35:57.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc4a7fc640 1 -- 172.21.15.38:0/1802463217 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6402f040 con 0x556ac8771230 2024-04-25T08:35:57.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc4a7fc640 1 -- 172.21.15.38:0/1802463217 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdc5c033020 con 0x556ac8771230 2024-04-25T08:35:57.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc74f9a740 10 client.7122 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T08:35:57.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc74f9a740 10 client.7122 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc74f9a740 20 client.7122 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x556ac87a8230 0x556ac87aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc74f9a740 4 client.7122 mds_command: new command op to 7080 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1802463217 --> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ac8708920 con 0x556ac87a8230 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc6affd640 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x556ac87a8230 0x556ac87aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc4a7fc640 10 client.7122 ms_handle_connect on v2:172.21.15.38:6832/1952521723 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.829+0000 7fdc6affd640 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x556ac87a8230 0x556ac87aa620 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a56b3f05b266e77 server_cookie=56ea2afef3b24a76 in_seq=0 out_seq=0 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc4a7fc640 1 -- 172.21.15.38:0/1802463217 <== mds.0 v2:172.21.15.38:6832/1952521723 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556ac8708920 con 0x556ac87a8230 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc4a7fc640 10 client.7122 handle_command_reply: tid=0 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc74f9a740 1 client.7122 shutdown 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1802463217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8771230 msgr2=0x556ac8786270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8771230 0x556ac8786270 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c015750 tx=0x7fdc5c001120 comp rx=0 tx=0).stop 2024-04-25T08:35:57.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1802463217 shutdown_connections 2024-04-25T08:35:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x556ac87a8230 0x556ac87aa620 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac879c2b0 0x556ac879c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac87830f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc4a7fc640 0 client.7122 ms_handle_reset on v2:172.21.15.38:6832/1952521723 2024-04-25T08:35:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1802463217 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac8771230 0x556ac8786270 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1802463217 >> 172.21.15.38:0/1802463217 conn(0x556ac8746f10 msgr2=0x556ac878f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1802463217 shutdown_connections 2024-04-25T08:35:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1802463217 wait complete. 2024-04-25T08:35:57.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.830+0000 7fdc74f9a740 20 client.7122 trim_cache size 0 max 16384 2024-04-25T08:35:57.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.846+0000 7fdc74f9a740 1 Processor -- start 2024-04-25T08:35:57.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.846+0000 7fdc74f9a740 1 -- start start 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.847+0000 7fdc74f9a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac8771230 0x556ac87461f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.847+0000 7fdc74f9a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac8746730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.847+0000 7fdc74f9a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 0x556ac87ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.847+0000 7fdc74f9a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c1ffbe0 con 0x556ac879c2b0 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.847+0000 7fdc74f9a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c1fefe0 con 0x556ac878ead0 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.847+0000 7fdc74f9a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c1ff5e0 con 0x556ac8771230 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.847+0000 7fdc6bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 0x556ac87ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.847+0000 7fdc6bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 0x556ac87ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34242/0 (socket says 172.21.15.38:34242) 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.847+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1152648618 learned_addr learned my addr 172.21.15.38:0/1152648618 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/1152648618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac8771230 0x556ac87461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc6affd640 1 --2- 172.21.15.38:0/1152648618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac8746730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1152648618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac8771230 msgr2=0x556ac87461f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/1152648618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac8771230 0x556ac87461f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1152648618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 msgr2=0x556ac8746730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/1152648618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac8746730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc6bfff640 1 -- 172.21.15.38:0/1152648618 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ac8775570 con 0x556ac879c2b0 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/1152648618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac8771230 0x556ac87461f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc6affd640 1 --2- 172.21.15.38:0/1152648618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac8746730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/1152648618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 0x556ac87ae640 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400a980 tx=0x7fdc64007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592fedfccea4a437 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc50ff9640 1 -- 172.21.15.38:0/1152648618 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64044040 con 0x556ac879c2b0 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc50ff9640 1 -- 172.21.15.38:0/1152648618 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6402f070 con 0x556ac879c2b0 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.848+0000 7fdc50ff9640 1 -- 172.21.15.38:0/1152648618 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6402b880 con 0x556ac879c2b0 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.849+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1152648618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 msgr2=0x556ac87ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.849+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1152648618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 0x556ac87ae640 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400a980 tx=0x7fdc64007000 comp rx=0 tx=0).stop 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.849+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1152648618 shutdown_connections 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.849+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1152648618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 0x556ac87ae640 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.849+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1152648618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac8746730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.849+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/1152648618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac8771230 0x556ac87461f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.849+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1152648618 >> 172.21.15.38:0/1152648618 conn(0x556ac8746f10 msgr2=0x556ac876cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.849+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1152648618 shutdown_connections 2024-04-25T08:35:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.849+0000 7fdc74f9a740 1 -- 172.21.15.38:0/1152648618 wait complete. 2024-04-25T08:35:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.849+0000 7fdc74f9a740 1 -- start start 2024-04-25T08:35:57.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.849+0000 7fdc74f9a740 1 Processor -- start 2024-04-25T08:35:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.850+0000 7fdc74f9a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac8771230 0x556ac87ae640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.850+0000 7fdc74f9a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac878f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.850+0000 7fdc74f9a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 0x556ac878bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.850+0000 7fdc74f9a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac8773a90 con 0x556ac879c2b0 2024-04-25T08:35:57.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.850+0000 7fdc74f9a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ac8773690 con 0x556ac878ead0 2024-04-25T08:35:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.850+0000 7fdc74f9a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ac87a6aa0 con 0x556ac8771230 2024-04-25T08:35:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.850+0000 7fdc74f9a740 10 client.0 _ll_register_callbacks cb 0x556ac8741c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.850+0000 7fdc6bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 0x556ac878bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.850+0000 7fdc6bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 0x556ac878bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34258/0 (socket says 172.21.15.38:34258) 2024-04-25T08:35:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.850+0000 7fdc6bfff640 1 -- 172.21.15.38:0/3968373358 learned_addr learned my addr 172.21.15.38:0/3968373358 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.850+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac8771230 0x556ac87ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.850+0000 7fdc6affd640 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac878f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc6bfff640 1 -- 172.21.15.38:0/3968373358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac8771230 msgr2=0x556ac87ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac8771230 0x556ac87ae640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc6bfff640 1 -- 172.21.15.38:0/3968373358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 msgr2=0x556ac878f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac878f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc6bfff640 1 -- 172.21.15.38:0/3968373358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ac87819d0 con 0x556ac879c2b0 2024-04-25T08:35:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc6bfff640 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 0x556ac878bf60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fdc6402b530 tx=0x7fdc6402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e57f7cec95f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:57.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc4a7fc640 1 -- 172.21.15.38:0/3968373358 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64044030 con 0x556ac879c2b0 2024-04-25T08:35:57.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc4a7fc640 1 -- 172.21.15.38:0/3968373358 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdc64001a00 con 0x556ac879c2b0 2024-04-25T08:35:57.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc74f9a740 1 -- 172.21.15.38:0/3968373358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdc6402f070 con 0x556ac879c2b0 2024-04-25T08:35:57.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc4a7fc640 1 -- 172.21.15.38:0/3968373358 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc64054050 con 0x556ac879c2b0 2024-04-25T08:35:57.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc74f9a740 10 client.7128 fetch_fsmap learned FSMap version 42 2024-04-25T08:35:57.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc74f9a740 1 -- 172.21.15.38:0/3968373358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x556ac87aebe0 con 0x556ac879c2b0 2024-04-25T08:35:57.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc4a7fc640 1 -- 172.21.15.38:0/3968373358 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6402f070 con 0x556ac879c2b0 2024-04-25T08:35:57.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc4a7fc640 1 -- 172.21.15.38:0/3968373358 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdc64057020 con 0x556ac879c2b0 2024-04-25T08:35:57.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc74f9a740 10 client.7128 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T08:35:57.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc74f9a740 10 client.7128 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:35:57.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc74f9a740 20 client.7128 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:57.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x556ac87f64c0 0x556ac878c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc74f9a740 4 client.7128 mds_command: new command op to 7080 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:35:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.851+0000 7fdc74f9a740 1 -- 172.21.15.38:0/3968373358 --> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556ac86b6f50 con 0x556ac87f64c0 2024-04-25T08:35:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x556ac87f64c0 0x556ac878c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc6b7fe640 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x556ac87f64c0 0x556ac878c4a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51eca0374464112d server_cookie=e0c6783a88e8691c in_seq=0 out_seq=0 2024-04-25T08:35:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc4a7fc640 10 client.7128 ms_handle_connect on v2:172.21.15.38:6832/1952521723 2024-04-25T08:35:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc4a7fc640 1 -- 172.21.15.38:0/3968373358 <== mds.0 v2:172.21.15.38:6832/1952521723 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x556ac86b6f50 con 0x556ac87f64c0 2024-04-25T08:35:57.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc4a7fc640 10 client.7128 handle_command_reply: tid=0 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc74f9a740 1 client.7128 shutdown 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc74f9a740 1 -- 172.21.15.38:0/3968373358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 msgr2=0x556ac878bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 0x556ac878bf60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fdc6402b530 tx=0x7fdc6402d070 comp rx=0 tx=0).stop 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc74f9a740 1 -- 172.21.15.38:0/3968373358 shutdown_connections 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x556ac87f64c0 0x556ac878c4a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556ac879c2b0 0x556ac878bf60 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556ac878ead0 0x556ac878f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc4a7fc640 0 client.7128 ms_handle_reset on v2:172.21.15.38:6832/1952521723 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc74f9a740 1 --2- 172.21.15.38:0/3968373358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556ac8771230 0x556ac87ae640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc74f9a740 1 -- 172.21.15.38:0/3968373358 >> 172.21.15.38:0/3968373358 conn(0x556ac8746f10 msgr2=0x556ac878df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:57.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc74f9a740 1 -- 172.21.15.38:0/3968373358 shutdown_connections 2024-04-25T08:35:57.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc74f9a740 1 -- 172.21.15.38:0/3968373358 wait complete. 2024-04-25T08:35:57.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.852+0000 7fdc74f9a740 20 client.7128 trim_cache size 0 max 16384 2024-04-25T08:35:57.859 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:57.859 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:35:57.859 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:35:57.859 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 7080, 2024-04-25T08:35:57.859 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:35:57.859 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:35:57.859 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:35:57.860 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 4.4951173190000002, 2024-04-25T08:35:57.860 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 42, 2024-04-25T08:35:57.860 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 71, 2024-04-25T08:35:57.860 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 71, 2024-04-25T08:35:57.860 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 5.3683331619999999 2024-04-25T08:35:57.860 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:57.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.853+0000 7fdc721b7640 1 -- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdc5806b500 msgr2=0x7fdc5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.853+0000 7fdc721b7640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdc5806b500 0x7fdc5806d9c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c000c00 tx=0x7fdc5c009040 comp rx=0 tx=0).stop 2024-04-25T08:35:57.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.853+0000 7fdc721b7640 1 -- 172.21.15.38:0/1639234118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc6c1f7230 msgr2=0x7fdc6c1f76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.853+0000 7fdc721b7640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc6c1f7230 0x7fdc6c1f76b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fdc60011910 tx=0x7fdc60011dd0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.853+0000 7fdc721b7640 1 -- 172.21.15.38:0/1639234118 shutdown_connections 2024-04-25T08:35:57.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.853+0000 7fdc721b7640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc6c14b5a0 0x7fdc6c14ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.853+0000 7fdc721b7640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdc6c1f7230 0x7fdc6c1f76b0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.853+0000 7fdc721b7640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdc5806b500 0x7fdc5806d9c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.853+0000 7fdc721b7640 1 --2- 172.21.15.38:0/1639234118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdc540051c0 0x7fdc6c1f3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.853+0000 7fdc721b7640 1 -- 172.21.15.38:0/1639234118 >> 172.21.15.38:0/1639234118 conn(0x7fdc6c066d10 msgr2=0x7fdc6c0d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:57.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.853+0000 7fdc721b7640 1 -- 172.21.15.38:0/1639234118 shutdown_connections 2024-04-25T08:35:57.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.853+0000 7fdc721b7640 1 -- 172.21.15.38:0/1639234118 wait complete. 2024-04-25T08:35:57.863 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:35:57.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.969+0000 7f2623f9e640 1 Processor -- start 2024-04-25T08:35:57.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.970+0000 7f2623f9e640 1 -- start start 2024-04-25T08:35:57.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.970+0000 7f2623f9e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f261c06cff0 0x7f261c099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.970+0000 7f2623f9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f261c099fb0 0x7f261c15d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.970+0000 7f2623f9e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261c0d05c0 con 0x7f261c06cc10 2024-04-25T08:35:57.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.970+0000 7f2623f9e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261c09a400 con 0x7f261c06cff0 2024-04-25T08:35:57.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.970+0000 7f2623f9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c0d08a0 con 0x7f261c099fb0 2024-04-25T08:35:57.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.971+0000 7f2621539640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f261c06cff0 0x7f261c099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.971+0000 7f262253b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f261c099fb0 0x7f261c15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.971+0000 7f2621539640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f261c06cff0 0x7f261c099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:54330/0 (socket says 172.21.15.38:54330) 2024-04-25T08:35:57.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.971+0000 7f2621539640 1 -- 172.21.15.38:0/1303071793 learned_addr learned my addr 172.21.15.38:0/1303071793 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:57.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.971+0000 7f2621539640 1 -- 172.21.15.38:0/1303071793 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f261c099fb0 msgr2=0x7f261c15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:57.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.971+0000 7f2621539640 1 --2- 172.21.15.38:0/1303071793 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f261c099fb0 0x7f261c15d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.971+0000 7f2621539640 1 -- 172.21.15.38:0/1303071793 >> v1:172.21.15.107:6789/0 conn(0x7f261c06cc10 legacy=0x7f261c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.971+0000 7f2621539640 1 -- 172.21.15.38:0/1303071793 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c0d0b80 con 0x7f261c06cff0 2024-04-25T08:35:57.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.971+0000 7f2620d38640 1 -- 172.21.15.38:0/1303071793 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698553557 0 0) 0x7f261c0d05c0 con 0x7f261c06cc10 2024-04-25T08:35:57.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.972+0000 7f2621539640 1 --2- 172.21.15.38:0/1303071793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c06cff0 0x7f261c099a70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f260c009870 tx=0x7f260c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dd76ac0fbe757b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.972+0000 7f2620d38640 1 -- 172.21.15.38:0/1303071793 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c037070 con 0x7f261c06cff0 2024-04-25T08:35:57.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.972+0000 7f2620d38640 1 -- 172.21.15.38:0/1303071793 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c032440 con 0x7f261c06cff0 2024-04-25T08:35:57.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.972+0000 7f2620d38640 1 -- 172.21.15.38:0/1303071793 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c032810 con 0x7f261c06cff0 2024-04-25T08:35:57.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.972+0000 7f2623f9e640 1 -- 172.21.15.38:0/1303071793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c06cff0 msgr2=0x7f261c099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.972+0000 7f2623f9e640 1 --2- 172.21.15.38:0/1303071793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c06cff0 0x7f261c099a70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f260c009870 tx=0x7f260c02a270 comp rx=0 tx=0).stop 2024-04-25T08:35:57.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.973+0000 7f2623f9e640 1 -- 172.21.15.38:0/1303071793 shutdown_connections 2024-04-25T08:35:57.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.973+0000 7f2623f9e640 1 --2- 172.21.15.38:0/1303071793 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f261c099fb0 0x7f261c15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.973+0000 7f2623f9e640 1 --2- 172.21.15.38:0/1303071793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c06cff0 0x7f261c099a70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.973+0000 7f2623f9e640 1 -- 172.21.15.38:0/1303071793 >> 172.21.15.38:0/1303071793 conn(0x7f261c0b31f0 msgr2=0x7f261c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:57.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.973+0000 7f2623f9e640 1 -- 172.21.15.38:0/1303071793 shutdown_connections 2024-04-25T08:35:57.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.973+0000 7f2623f9e640 1 -- 172.21.15.38:0/1303071793 wait complete. 2024-04-25T08:35:57.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.974+0000 7f2623f9e640 1 Processor -- start 2024-04-25T08:35:57.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.974+0000 7f2623f9e640 1 -- start start 2024-04-25T08:35:57.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2623f9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c06cc10 0x7f261c0c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2623f9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f261c06cff0 0x7f261c0c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2623f9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c099fb0 0x7f261c0c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2623f9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c06c150 con 0x7f261c06cc10 2024-04-25T08:35:57.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2623f9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c0d1140 con 0x7f261c099fb0 2024-04-25T08:35:57.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2623f9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c06c2d0 con 0x7f261c06cff0 2024-04-25T08:35:57.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2621d3a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c06cc10 0x7f261c0c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2621d3a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c06cc10 0x7f261c0c7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34272/0 (socket says 172.21.15.38:34272) 2024-04-25T08:35:57.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2621d3a640 1 -- 172.21.15.38:0/3658855567 learned_addr learned my addr 172.21.15.38:0/3658855567 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f262253b640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c099fb0 0x7f261c0c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2621539640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f261c06cff0 0x7f261c0c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2621d3a640 1 -- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f261c06cff0 msgr2=0x7f261c0c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2621d3a640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f261c06cff0 0x7f261c0c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2621d3a640 1 -- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c099fb0 msgr2=0x7f261c0c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2621d3a640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c099fb0 0x7f261c0c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2621d3a640 1 -- 172.21.15.38:0/3658855567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c0c8390 con 0x7f261c06cc10 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2621d3a640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c06cc10 0x7f261c0c7e50 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f261800c4b0 tx=0x7f261800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f26127fc640 1 -- 172.21.15.38:0/3658855567 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618017070 con 0x7f261c06cc10 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f26127fc640 1 -- 172.21.15.38:0/3658855567 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2618005480 con 0x7f261c06cc10 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f2623f9e640 1 -- 172.21.15.38:0/3658855567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f261c0c9ec0 con 0x7f261c06cc10 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.975+0000 7f26127fc640 1 -- 172.21.15.38:0/3658855567 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618003830 con 0x7f261c06cc10 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.976+0000 7f2623f9e640 1 -- 172.21.15.38:0/3658855567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261c0c9620 con 0x7f261c06cc10 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.976+0000 7f2623f9e640 1 -- 172.21.15.38:0/3658855567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261c068e80 con 0x7f261c06cc10 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.977+0000 7f26127fc640 1 -- 172.21.15.38:0/3658855567 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f261801a070 con 0x7f261c06cc10 2024-04-25T08:35:57.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.977+0000 7f26127fc640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f260006b4d0 0x7f260006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:57.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.978+0000 7f26127fc640 1 -- 172.21.15.38:0/3658855567 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2618089700 con 0x7f261c06cc10 2024-04-25T08:35:57.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.978+0000 7f2621539640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f260006b4d0 0x7f260006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:57.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.978+0000 7f2621539640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f260006b4d0 0x7f260006d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f260c034000 tx=0x7f260c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:57.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:57.982+0000 7f26127fc640 1 -- 172.21.15.38:0/3658855567 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2618058430 con 0x7f261c06cc10 2024-04-25T08:35:58.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.199+0000 7f2623f9e640 1 -- 172.21.15.38:0/3658855567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f261c0c4fe0 con 0x7f261c06cc10 2024-04-25T08:35:58.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.200+0000 7f26127fc640 1 -- 172.21.15.38:0/3658855567 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2618057d30 con 0x7f261c06cc10 2024-04-25T08:35:58.194 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:35:58.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.202+0000 7f2623f9e640 1 -- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f260006b4d0 msgr2=0x7f260006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.202+0000 7f2623f9e640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f260006b4d0 0x7f260006d990 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f260c034000 tx=0x7f260c02e000 comp rx=0 tx=0).stop 2024-04-25T08:35:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.202+0000 7f2623f9e640 1 -- 172.21.15.38:0/3658855567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c06cc10 msgr2=0x7f261c0c7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.202+0000 7f2623f9e640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c06cc10 0x7f261c0c7e50 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f261800c4b0 tx=0x7f261800c970 comp rx=0 tx=0).stop 2024-04-25T08:35:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.202+0000 7f2623f9e640 1 -- 172.21.15.38:0/3658855567 shutdown_connections 2024-04-25T08:35:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.202+0000 7f2623f9e640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f261c099fb0 0x7f261c0c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.202+0000 7f2623f9e640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f260006b4d0 0x7f260006d990 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.202+0000 7f2623f9e640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f261c06cff0 0x7f261c0c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.202+0000 7f2623f9e640 1 --2- 172.21.15.38:0/3658855567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f261c06cc10 0x7f261c0c7e50 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.202+0000 7f2623f9e640 1 -- 172.21.15.38:0/3658855567 >> 172.21.15.38:0/3658855567 conn(0x7f261c0b31f0 msgr2=0x7f261c0cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.202+0000 7f2623f9e640 1 -- 172.21.15.38:0/3658855567 shutdown_connections 2024-04-25T08:35:58.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:58.202+0000 7f2623f9e640 1 -- 172.21.15.38:0/3658855567 wait complete. 2024-04-25T08:35:58.207 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:35:58.207 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:35:58.207 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:35:58.207 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:35:58.208 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:58.208 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:35:58.208 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:35:58.208 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:35:58.208 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:35:58.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:58.208 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:35:58.225 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:35:58.225 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:35:58.225 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:35:58.226 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:58.226 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:35:58.226 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:58.226 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:35:58.226 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:58.226 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:35:58.227 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:35:58.227 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:58.227 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:35:58.227 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:35:58.227 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:58.228 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:35:58.228 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:35:58.228 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42479sec preferred_lft 42479sec 2024-04-25T08:35:58.228 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:35:58.228 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:58.228 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:35:58.228 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:58.229 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:35:58.229 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:35:58.229 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:35:58.229 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:35:58.229 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:58.229 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::440f:97ff:fe63:87b/64 scope link 2024-04-25T08:35:58.230 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:58.230 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:35:58.230 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ea:23:76:ab:b1:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:35:58.230 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::e823:76ff:feab:b1e9/64 scope link 2024-04-25T08:35:58.230 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:58.230 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:35:58.231 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:35:58.231 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::38c7:38ff:fe2e:6c92/64 scope link 2024-04-25T08:35:58.231 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:35:58.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:58.232 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:35:58.288 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:35:58.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:58.290 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:35:58.346 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:35:58.347 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:35:58.347 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:35:58.347 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:35:58.347 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:35:58.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:58.347 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:35:58.347 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:35:58.347 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:35:58.348 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:35:58.348 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:35:58.348 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:35:58.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:58.433 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:35:58.448 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:35:58.448 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:35:58.448 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:35:58.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:58.448 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T08:35:58.515 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:35:58.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:35:58.515 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:58.570 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:35:58.570 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:35:58.571 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:58.625 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:35:58.690 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:35:58.746 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:35:58.747 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:35:58.747 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:35:58.747 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:35:58.747 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:35:58.747 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:35:58.747 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:35:58.747 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:35:58.747 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:35:58.747 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:35:58.747 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:35:58.747 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:35:58.748 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:35:58.749 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:35:58.749 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:35:58.749 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:35:58.749 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:35:58.749 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:35:58.749 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:35:58.749 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:35:58.749 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:35:58.749 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:35:58.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:58.750 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:35:58.802 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:35:58.803 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:35:58.846 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:35:58.870 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:35:58.926 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:35:58.926 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:35:58.927 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:35:58.927 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:35:58.927 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:35:58.927 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:35:58.927 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:35:58.927 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:35:58.927 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:35:58.927 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:35:58.927 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:35:58.927 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:35:58.927 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:35:58.928 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:35:58.928 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:35:58.928 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:35:58.928 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:35:58.928 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:35:58.928 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:35:58.928 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:35:58.928 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:35:58.928 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:35:58.929 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:35:58.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:58.930 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:35:58.931 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-04-25T08:35:58.933+0000 7f94b0219480 -1 init, newargv = 0x55c93353bdf0 newargc=15 2024-04-25T08:35:58.931 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[67012]: starting ceph client 2024-04-25T08:35:58.931 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[67012]: starting fuse 2024-04-25T08:35:58.945 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:35:58.945 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:35:58.946 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:35:58.946 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:35:58.946 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:35:58.946 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:35:58.946 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:35:58.946 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:58.946 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:35:58.946 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:35:58.946 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:35:58.946 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:35:58.946 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-04-25T08:35:58.947 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:35:59.064 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.67012.asok 2024-04-25T08:35:59.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:35:59.075 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.67012.asok 2024-04-25T08:35:59.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:59.075 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.67012.asok status 2024-04-25T08:35:59.211 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:59.211 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:35:59.211 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:35:59.211 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:35:59.211 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-04-25T08:35:59.211 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "67012", 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 7140, 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 7140 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:35:59.212 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3904931322 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3904931322 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.7140 192.168.0.1:0/3904931322", 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/3904931322", 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 42, 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 71, 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:35:59.213 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:35:59.214 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:59.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:59.220 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:35:59.220 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:59.235 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:35:59.235 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:59.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:59.236 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:35:59.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:35:59.301 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T08:35:59.351 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-04-25T08:35:59.357+0000 7f4d72ed1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:35:59.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:35:59.359 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-25T08:35:59.359 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-25T08:35:59.359 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:35:59.359 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-25T08:35:59.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.510+0000 7fc20211d640 1 Processor -- start 2024-04-25T08:35:59.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.510+0000 7fc20211d640 1 -- start start 2024-04-25T08:35:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc20211d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc1fc0d0870 0x7fc1fc0d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc20211d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc0d11c0 con 0x7fc1fc1612c0 2024-04-25T08:35:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc20211d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1fc0b9880 con 0x7fc1fc0b93f0 2024-04-25T08:35:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc20211d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b7200 con 0x7fc1fc0d0870 2024-04-25T08:35:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fb7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc1fc1612c0 0x7fc1fc0b6a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59422/0 (socket says 172.21.15.38:59422) 2024-04-25T08:35:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fb7fe640 1 -- 172.21.15.38:0/4247035746 learned_addr learned my addr 172.21.15.38:0/4247035746 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:35:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/4247035746 >> v2:172.21.15.107:3300/0 conn(0x7fc1fc0d0870 0x7fc1fc0d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485619339 0 0) 0x7fc1ec001110 con 0x7fc1fc1612c0 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e4003520 con 0x7fc1fc1612c0 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1065473285 0 0) 0x7fc1e4003520 con 0x7fc1fc1612c0 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 >> v2:172.21.15.107:3300/0 conn(0x7fc1fc0d0870 msgr2=0x7fc1fc0d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 --2- 172.21.15.38:0/4247035746 >> v2:172.21.15.107:3300/0 conn(0x7fc1fc0d0870 0x7fc1fc0d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 >> v1:172.21.15.107:6789/0 conn(0x7fc1fc0b93f0 legacy=0x7fc1fc0d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc0b74e0 con 0x7fc1fc1612c0 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fc1ec001840 con 0x7fc1fc1612c0 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 >> v1:172.21.15.38:6789/0 conn(0x7fc1fc1612c0 legacy=0x7fc1fc0b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 --2- 172.21.15.38:0/4247035746 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e40039b0 0x7fc1e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 --2- 172.21.15.38:0/4247035746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e4004800 0x7fc1e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 --2- 172.21.15.38:0/4247035746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e40051c0 0x7fc1e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b7200 con 0x7fc1e40051c0 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e400c160 con 0x7fc1e4004800 2024-04-25T08:35:59.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.511+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e400c360 con 0x7fc1e40039b0 2024-04-25T08:35:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.512+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/4247035746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e40051c0 0x7fc1e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.512+0000 7fc1faffd640 1 --2- 172.21.15.38:0/4247035746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e4004800 0x7fc1e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.512+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/4247035746 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e40039b0 0x7fc1e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.512+0000 7fc1fbfff640 1 -- 172.21.15.38:0/4247035746 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e40039b0 msgr2=0x7fc1e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.512+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/4247035746 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e40039b0 0x7fc1e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.512+0000 7fc1fbfff640 1 -- 172.21.15.38:0/4247035746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e4004800 msgr2=0x7fc1e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.512+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/4247035746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1e4004800 0x7fc1e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.512+0000 7fc1fbfff640 1 -- 172.21.15.38:0/4247035746 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e4003520 con 0x7fc1e40051c0 2024-04-25T08:35:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.512+0000 7fc1fbfff640 1 -- 172.21.15.38:0/4247035746 reap_dead start 2024-04-25T08:35:59.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.513+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/4247035746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e40051c0 0x7fc1e400b9c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009fc0 tx=0x7fc1e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bdec4e8cd3daca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.513+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4035020 con 0x7fc1e40051c0 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.513+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f4003bd0 con 0x7fc1e40051c0 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.513+0000 7fc1fa7fc640 1 -- 172.21.15.38:0/4247035746 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4009c10 con 0x7fc1e40051c0 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.513+0000 7fc20211d640 1 -- 172.21.15.38:0/4247035746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e40051c0 msgr2=0x7fc1e400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.513+0000 7fc20211d640 1 --2- 172.21.15.38:0/4247035746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e40051c0 0x7fc1e400b9c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009fc0 tx=0x7fc1e4005830 comp rx=0 tx=0).stop 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.513+0000 7fc20211d640 1 -- 172.21.15.38:0/4247035746 shutdown_connections 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.513+0000 7fc20211d640 1 --2- 172.21.15.38:0/4247035746 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1e40051c0 0x7fc1e400b9c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.513+0000 7fc20211d640 1 -- 172.21.15.38:0/4247035746 >> 172.21.15.38:0/4247035746 conn(0x7fc1fc066d10 msgr2=0x7fc1fc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.513+0000 7fc20211d640 1 -- 172.21.15.38:0/4247035746 shutdown_connections 2024-04-25T08:35:59.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.513+0000 7fc20211d640 1 -- 172.21.15.38:0/4247035746 wait complete. 2024-04-25T08:35:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.514+0000 7fc20211d640 1 Processor -- start 2024-04-25T08:35:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.515+0000 7fc20211d640 1 -- start start 2024-04-25T08:35:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.515+0000 7fc20211d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e40051c0 0x7fc1fc1f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.515+0000 7fc20211d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1fc15da90 0x7fc1fc0cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.515+0000 7fc20211d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1fc0cc950 0x7fc1fc0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.515+0000 7fc20211d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b7d80 con 0x7fc1fc0cc950 2024-04-25T08:35:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.515+0000 7fc20211d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0b7aa0 con 0x7fc1fc15da90 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.515+0000 7fc20211d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc0d0c80 con 0x7fc1e40051c0 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1fc0cc950 0x7fc1fc0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1fc0cc950 0x7fc1fc0ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35900/0 (socket says 172.21.15.38:35900) 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1fbfff640 1 -- 172.21.15.38:0/446101106 learned_addr learned my addr 172.21.15.38:0/446101106 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e40051c0 0x7fc1fc1f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1faffd640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1fc15da90 0x7fc1fc0cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1fbfff640 1 -- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e40051c0 msgr2=0x7fc1fc1f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e40051c0 0x7fc1fc1f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1fbfff640 1 -- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1fc15da90 msgr2=0x7fc1fc0cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1fc15da90 0x7fc1fc0cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1fbfff640 1 -- 172.21.15.38:0/446101106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1fc0cc0d0 con 0x7fc1fc0cc950 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1faffd640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1fc15da90 0x7fc1fc0cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1fc0cc950 0x7fc1fc0ccdd0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc1f402af10 tx=0x7fc1f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:59.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1e3fff640 1 -- 172.21.15.38:0/446101106 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4035030 con 0x7fc1fc0cc950 2024-04-25T08:35:59.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1e3fff640 1 -- 172.21.15.38:0/446101106 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f4043070 con 0x7fc1fc0cc950 2024-04-25T08:35:59.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc1e3fff640 1 -- 172.21.15.38:0/446101106 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f402d070 con 0x7fc1fc0cc950 2024-04-25T08:35:59.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc20211d640 1 -- 172.21.15.38:0/446101106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1fc0cc3b0 con 0x7fc1fc0cc950 2024-04-25T08:35:59.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.516+0000 7fc20211d640 1 -- 172.21.15.38:0/446101106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1fc1f37e0 con 0x7fc1fc0cc950 2024-04-25T08:35:59.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.518+0000 7fc1e3fff640 1 -- 172.21.15.38:0/446101106 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1f4005d10 con 0x7fc1fc0cc950 2024-04-25T08:35:59.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.518+0000 7fc1e3fff640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1e806b500 0x7fc1e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.518+0000 7fc1e3fff640 1 -- 172.21.15.38:0/446101106 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc1f40aac60 con 0x7fc1fc0cc950 2024-04-25T08:35:59.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.518+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1e806b500 0x7fc1e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.519+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1e806b500 0x7fc1e806d9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec001050 tx=0x7fc1ec000f80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:59.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.519+0000 7fc204e19740 1 Processor -- start 2024-04-25T08:35:59.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.519+0000 7fc204e19740 1 -- start start 2024-04-25T08:35:59.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.520+0000 7fc204e19740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 0x55e0498046b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.520+0000 7fc204e19740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049834e40 0x55e0498341c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.520+0000 7fc204e19740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049834700 0x55e049809510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.520+0000 7fc204e19740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04976cf40 con 0x55e049851d20 2024-04-25T08:35:59.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.520+0000 7fc204e19740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04970c0d0 con 0x55e049834e40 2024-04-25T08:35:59.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.520+0000 7fc204e19740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e049768fe0 con 0x55e049834700 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.520+0000 7fc1fb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 0x55e0498046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.520+0000 7fc1fb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 0x55e0498046b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35914/0 (socket says 172.21.15.38:35914) 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.520+0000 7fc1fb7fe640 1 -- 172.21.15.38:0/490339213 learned_addr learned my addr 172.21.15.38:0/490339213 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.520+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/490339213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049834700 0x55e049809510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.520+0000 7fc1faffd640 1 --2- 172.21.15.38:0/490339213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049834e40 0x55e0498341c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.521+0000 7fc1fb7fe640 1 -- 172.21.15.38:0/490339213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049834700 msgr2=0x55e049809510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.521+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/490339213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049834700 0x55e049809510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.521+0000 7fc1fb7fe640 1 -- 172.21.15.38:0/490339213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049834e40 msgr2=0x55e0498341c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.521+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/490339213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049834e40 0x55e0498341c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.521+0000 7fc1fb7fe640 1 -- 172.21.15.38:0/490339213 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e04954daf0 con 0x55e049851d20 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.521+0000 7fc1faffd640 1 --2- 172.21.15.38:0/490339213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049834e40 0x55e0498341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.521+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/490339213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 0x55e0498046b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec004d20 tx=0x7fc1ec00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a88db0853c9816 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:59.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.521+0000 7fc1e17fa640 1 -- 172.21.15.38:0/490339213 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec028070 con 0x55e049851d20 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.521+0000 7fc1e17fa640 1 -- 172.21.15.38:0/490339213 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ec004e10 con 0x55e049851d20 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.521+0000 7fc1e17fa640 1 -- 172.21.15.38:0/490339213 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec015dd0 con 0x55e049851d20 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.522+0000 7fc204e19740 1 -- 172.21.15.38:0/490339213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 msgr2=0x55e0498046b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.522+0000 7fc204e19740 1 --2- 172.21.15.38:0/490339213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 0x55e0498046b0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec004d20 tx=0x7fc1ec00c000 comp rx=0 tx=0).stop 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.522+0000 7fc204e19740 1 -- 172.21.15.38:0/490339213 shutdown_connections 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.522+0000 7fc204e19740 1 --2- 172.21.15.38:0/490339213 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049834700 0x55e049809510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.522+0000 7fc204e19740 1 --2- 172.21.15.38:0/490339213 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049834e40 0x55e0498341c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.522+0000 7fc204e19740 1 --2- 172.21.15.38:0/490339213 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 0x55e0498046b0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.522+0000 7fc204e19740 1 -- 172.21.15.38:0/490339213 >> 172.21.15.38:0/490339213 conn(0x55e049806920 msgr2=0x55e04984f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.522+0000 7fc204e19740 1 -- 172.21.15.38:0/490339213 shutdown_connections 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.522+0000 7fc204e19740 1 -- 172.21.15.38:0/490339213 wait complete. 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.522+0000 7fc204e19740 1 -- start start 2024-04-25T08:35:59.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc204e19740 1 Processor -- start 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc204e19740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049834e40 0x55e049821ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc204e19740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049851d20 0x55e049836c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc204e19740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049809f20 0x55e04980a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc204e19740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e049808ad0 con 0x55e049809f20 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc204e19740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e049709ba0 con 0x55e049851d20 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc204e19740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e049807ec0 con 0x55e049834e40 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc204e19740 10 client.0 _ll_register_callbacks cb 0x55e049709570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc1fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049809f20 0x55e04980a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc1fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049809f20 0x55e04980a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35924/0 (socket says 172.21.15.38:35924) 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc1fbfff640 1 -- 172.21.15.38:0/575780540 learned_addr learned my addr 172.21.15.38:0/575780540 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049834e40 0x55e049821ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.523+0000 7fc1faffd640 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049851d20 0x55e049836c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc1fbfff640 1 -- 172.21.15.38:0/575780540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049834e40 msgr2=0x55e049821ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049834e40 0x55e049821ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc1fbfff640 1 -- 172.21.15.38:0/575780540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049851d20 msgr2=0x55e049836c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049851d20 0x55e049836c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc1fbfff640 1 -- 172.21.15.38:0/575780540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0494413e0 con 0x55e049809f20 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049809f20 0x55e04980a3a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4079180 tx=0x7fc1f40798f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d20ae96a4e54af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc1da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc1da7fc640 1 -- 172.21.15.38:0/575780540 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40a7060 con 0x55e049809f20 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc1da7fc640 1 -- 172.21.15.38:0/575780540 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc1f4009c10 con 0x55e049809f20 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc204e19740 1 -- 172.21.15.38:0/575780540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1ec004e10 con 0x55e049809f20 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc1da7fc640 1 -- 172.21.15.38:0/575780540 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1f4095050 con 0x55e049809f20 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc204e19740 10 client.7152 fetch_fsmap learned FSMap version 42 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc204e19740 1 -- 172.21.15.38:0/575780540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55e0495ae280 con 0x55e049809f20 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc1da7fc640 1 -- 172.21.15.38:0/575780540 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ec004e10 con 0x55e049809f20 2024-04-25T08:35:59.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc1da7fc640 1 -- 172.21.15.38:0/575780540 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc1f4090030 con 0x55e049809f20 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc204e19740 10 client.7152 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc204e19740 10 client.7152 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.524+0000 7fc204e19740 20 client.7152 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.525+0000 7fc204e19740 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x55e0498468b0 0x55e049846cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.525+0000 7fc204e19740 4 client.7152 mds_command: new command op to 7080 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.525+0000 7fc204e19740 1 -- 172.21.15.38:0/575780540 --> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e04977a480 con 0x55e0498468b0 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.525+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x55e0498468b0 0x55e049846cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.525+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x55e0498468b0 0x55e049846cc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd8aad71e0dadc47 server_cookie=86c7c60fb01b84c6 in_seq=0 out_seq=0 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.525+0000 7fc1da7fc640 10 client.7152 ms_handle_connect on v2:172.21.15.38:6832/1952521723 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.525+0000 7fc1da7fc640 1 -- 172.21.15.38:0/575780540 <== mds.0 v2:172.21.15.38:6832/1952521723 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e04977a480 con 0x55e0498468b0 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.525+0000 7fc1da7fc640 10 client.7152 handle_command_reply: tid=0 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.525+0000 7fc204e19740 1 client.7152 shutdown 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.526+0000 7fc204e19740 1 -- 172.21.15.38:0/575780540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049809f20 msgr2=0x55e04980a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.526+0000 7fc204e19740 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049809f20 0x55e04980a3a0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4079180 tx=0x7fc1f40798f0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.526+0000 7fc204e19740 1 -- 172.21.15.38:0/575780540 shutdown_connections 2024-04-25T08:35:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.526+0000 7fc204e19740 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x55e0498468b0 0x55e049846cc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.526+0000 7fc204e19740 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049809f20 0x55e04980a3a0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.526+0000 7fc204e19740 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049851d20 0x55e049836c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.526+0000 7fc204e19740 1 --2- 172.21.15.38:0/575780540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049834e40 0x55e049821ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.526+0000 7fc1da7fc640 0 client.7152 ms_handle_reset on v2:172.21.15.38:6832/1952521723 2024-04-25T08:35:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.526+0000 7fc204e19740 1 -- 172.21.15.38:0/575780540 >> 172.21.15.38:0/575780540 conn(0x55e049834700 msgr2=0x55e0498502f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.526+0000 7fc204e19740 1 -- 172.21.15.38:0/575780540 shutdown_connections 2024-04-25T08:35:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.526+0000 7fc204e19740 1 -- 172.21.15.38:0/575780540 wait complete. 2024-04-25T08:35:59.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.526+0000 7fc204e19740 20 client.7152 trim_cache size 0 max 16384 2024-04-25T08:35:59.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.541+0000 7fc204e19740 1 Processor -- start 2024-04-25T08:35:59.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.541+0000 7fc204e19740 1 -- start start 2024-04-25T08:35:59.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc204e19740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049809f20 0x55e049804bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc204e19740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049834e40 0x55e0498502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc204e19740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 0x55e049850f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc204e19740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc200c60 con 0x55e049851d20 2024-04-25T08:35:59.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc204e19740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc1ffdc0 con 0x55e049834e40 2024-04-25T08:35:59.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc204e19740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc1ff1c0 con 0x55e049809f20 2024-04-25T08:35:59.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc1fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 0x55e049850f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc1fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 0x55e049850f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35934/0 (socket says 172.21.15.38:35934) 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc1fbfff640 1 -- 172.21.15.38:0/695386615 learned_addr learned my addr 172.21.15.38:0/695386615 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/695386615 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049809f20 0x55e049804bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc1faffd640 1 --2- 172.21.15.38:0/695386615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049834e40 0x55e0498502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc1fbfff640 1 -- 172.21.15.38:0/695386615 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049809f20 msgr2=0x55e049804bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/695386615 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049809f20 0x55e049804bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc1fbfff640 1 -- 172.21.15.38:0/695386615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049834e40 msgr2=0x55e0498502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/695386615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049834e40 0x55e0498502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc1fbfff640 1 -- 172.21.15.38:0/695386615 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e04941a560 con 0x55e049851d20 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.542+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/695386615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 0x55e049850f80 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4077740 tx=0x7fc1f408e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5138e6a298792c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.543+0000 7fc1e0ff9640 1 -- 172.21.15.38:0/695386615 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f40a7070 con 0x55e049851d20 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.543+0000 7fc1e0ff9640 1 -- 172.21.15.38:0/695386615 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f40774f0 con 0x55e049851d20 2024-04-25T08:35:59.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.543+0000 7fc1e0ff9640 1 -- 172.21.15.38:0/695386615 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f4078640 con 0x55e049851d20 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.543+0000 7fc204e19740 1 -- 172.21.15.38:0/695386615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 msgr2=0x55e049850f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.543+0000 7fc204e19740 1 --2- 172.21.15.38:0/695386615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 0x55e049850f80 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4077740 tx=0x7fc1f408e000 comp rx=0 tx=0).stop 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.543+0000 7fc204e19740 1 -- 172.21.15.38:0/695386615 shutdown_connections 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.543+0000 7fc204e19740 1 --2- 172.21.15.38:0/695386615 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049851d20 0x55e049850f80 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.543+0000 7fc204e19740 1 --2- 172.21.15.38:0/695386615 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049834e40 0x55e0498502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.543+0000 7fc204e19740 1 --2- 172.21.15.38:0/695386615 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049809f20 0x55e049804bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.543+0000 7fc204e19740 1 -- 172.21.15.38:0/695386615 >> 172.21.15.38:0/695386615 conn(0x55e049834700 msgr2=0x55e049846c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.543+0000 7fc204e19740 1 -- 172.21.15.38:0/695386615 shutdown_connections 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.543+0000 7fc204e19740 1 -- 172.21.15.38:0/695386615 wait complete. 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.544+0000 7fc204e19740 1 -- start start 2024-04-25T08:35:59.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.544+0000 7fc204e19740 1 Processor -- start 2024-04-25T08:35:59.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc204e19740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049809f20 0x55e049845030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc204e19740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049834e40 0x55e049845570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc204e19740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049851d20 0x55e049835330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc204e19740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1fc1ff7c0 con 0x55e049834e40 2024-04-25T08:35:59.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc204e19740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e04984e6c0 con 0x55e049851d20 2024-04-25T08:35:59.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc204e19740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e049850950 con 0x55e049809f20 2024-04-25T08:35:59.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc204e19740 10 client.0 _ll_register_callbacks cb 0x7fc1f4078fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1faffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049834e40 0x55e049845570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1faffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049834e40 0x55e049845570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35948/0 (socket says 172.21.15.38:35948) 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1faffd640 1 -- 172.21.15.38:0/25865535 learned_addr learned my addr 172.21.15.38:0/25865535 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049809f20 0x55e049845030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1fbfff640 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049851d20 0x55e049835330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1faffd640 1 -- 172.21.15.38:0/25865535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049809f20 msgr2=0x55e049845030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1faffd640 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049809f20 0x55e049845030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1faffd640 1 -- 172.21.15.38:0/25865535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049851d20 msgr2=0x55e049835330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1faffd640 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049851d20 0x55e049835330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1faffd640 1 -- 172.21.15.38:0/25865535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e04985eee0 con 0x55e049834e40 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1faffd640 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049834e40 0x55e049845570 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0000c00 tx=0x7fc1f0017800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631c3c5d990ef96f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1da7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:35:59.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1da7fc640 1 -- 172.21.15.38:0/25865535 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0017e60 con 0x55e049834e40 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc1da7fc640 1 -- 172.21.15.38:0/25865535 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc1f00014a0 con 0x55e049834e40 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.545+0000 7fc204e19740 1 -- 172.21.15.38:0/25865535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1f40774f0 con 0x55e049834e40 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc1da7fc640 1 -- 172.21.15.38:0/25865535 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1f0028050 con 0x55e049834e40 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc204e19740 10 client.7158 fetch_fsmap learned FSMap version 42 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc204e19740 1 -- 172.21.15.38:0/25865535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x55e0498502f0 con 0x55e049834e40 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc1da7fc640 1 -- 172.21.15.38:0/25865535 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f40774f0 con 0x55e049834e40 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc1da7fc640 1 -- 172.21.15.38:0/25865535 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 42) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc1f002b020 con 0x55e049834e40 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc204e19740 10 client.7158 fetch_fsmap finished waiting for FSMap version 42 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc204e19740 10 client.7158 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc204e19740 20 client.7158 populate_metadata read hostname 'smithi038' 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc204e19740 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x55e049848900 0x55e049835870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc204e19740 4 client.7158 mds_command: new command op to 7080 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc204e19740 1 -- 172.21.15.38:0/25865535 --> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x55e0497747d0 con 0x55e049848900 2024-04-25T08:35:59.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x55e049848900 0x55e049835870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc1fb7fe640 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x55e049848900 0x55e049835870 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ad41c2dabcd2a9 server_cookie=7d7ba5910b791985 in_seq=0 out_seq=0 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.546+0000 7fc1da7fc640 10 client.7158 ms_handle_connect on v2:172.21.15.38:6832/1952521723 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc1da7fc640 1 -- 172.21.15.38:0/25865535 <== mds.0 v2:172.21.15.38:6832/1952521723 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e0497747d0 con 0x55e049848900 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc1da7fc640 10 client.7158 handle_command_reply: tid=0 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc204e19740 1 client.7158 shutdown 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc204e19740 1 -- 172.21.15.38:0/25865535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049834e40 msgr2=0x55e049845570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc204e19740 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049834e40 0x55e049845570 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0000c00 tx=0x7fc1f0017800 comp rx=0 tx=0).stop 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc204e19740 1 -- 172.21.15.38:0/25865535 shutdown_connections 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc204e19740 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.38:6832/1952521723,v1:172.21.15.38:6833/1952521723] conn(0x55e049848900 0x55e049835870 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc204e19740 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e049851d20 0x55e049835330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc204e19740 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e049834e40 0x55e049845570 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc1da7fc640 0 client.7158 ms_handle_reset on v2:172.21.15.38:6832/1952521723 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc204e19740 1 --2- 172.21.15.38:0/25865535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e049809f20 0x55e049845030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc204e19740 1 -- 172.21.15.38:0/25865535 >> 172.21.15.38:0/25865535 conn(0x55e049834700 msgr2=0x55e049836a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:59.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc204e19740 1 -- 172.21.15.38:0/25865535 shutdown_connections 2024-04-25T08:35:59.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc204e19740 1 -- 172.21.15.38:0/25865535 wait complete. 2024-04-25T08:35:59.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.547+0000 7fc204e19740 20 client.7158 trim_cache size 0 max 16384 2024-04-25T08:35:59.549 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:35:59.549 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:35:59.549 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "31d05f30-3028-4e91-b399-4ed52a756a80", 2024-04-25T08:35:59.549 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:35:59.549 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:35:59.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.548+0000 7fc20211d640 1 -- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1e806b500 msgr2=0x7fc1e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:59.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.548+0000 7fc20211d640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1e806b500 0x7fc1e806d9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec001050 tx=0x7fc1ec000f80 comp rx=0 tx=0).stop 2024-04-25T08:35:59.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.548+0000 7fc20211d640 1 -- 172.21.15.38:0/446101106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1fc0cc950 msgr2=0x7fc1fc0ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:35:59.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.548+0000 7fc20211d640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1fc0cc950 0x7fc1fc0ccdd0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc1f402af10 tx=0x7fc1f4009750 comp rx=0 tx=0).stop 2024-04-25T08:35:59.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.548+0000 7fc20211d640 1 -- 172.21.15.38:0/446101106 shutdown_connections 2024-04-25T08:35:59.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.548+0000 7fc20211d640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1fc0cc950 0x7fc1fc0ccdd0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.548+0000 7fc20211d640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc1fc15da90 0x7fc1fc0cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.548+0000 7fc20211d640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc1e806b500 0x7fc1e806d9c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.548+0000 7fc20211d640 1 --2- 172.21.15.38:0/446101106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc1e40051c0 0x7fc1fc1f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:35:59.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.548+0000 7fc20211d640 1 -- 172.21.15.38:0/446101106 >> 172.21.15.38:0/446101106 conn(0x7fc1fc066d10 msgr2=0x7fc1fc160260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:35:59.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.548+0000 7fc20211d640 1 -- 172.21.15.38:0/446101106 shutdown_connections 2024-04-25T08:35:59.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:35:59.548+0000 7fc20211d640 1 -- 172.21.15.38:0/446101106 wait complete. 2024-04-25T08:35:59.553 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': '31d05f30-3028-4e91-b399-4ed52a756a80', 'mode': 'asynchronous'}' 2024-04-25T08:36:03.217 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:36:03.220 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:36:09.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:09.554 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T08:36:09.577 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:36:09.577 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:36:09.658 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:36:09.700 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:36:09.700 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:36:09.700 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:09.718 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:09.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.873+0000 7fcd44a5d640 1 Processor -- start 2024-04-25T08:36:09.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.873+0000 7fcd44a5d640 1 -- start start 2024-04-25T08:36:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.874+0000 7fcd44a5d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcd4006cff0 0x7fcd40099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.874+0000 7fcd44a5d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcd40099fb0 0x7fcd4015d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:09.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.874+0000 7fcd44a5d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4009a3c0 con 0x7fcd4006cc10 2024-04-25T08:36:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.874+0000 7fcd44a5d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd400d05c0 con 0x7fcd4006cff0 2024-04-25T08:36:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.874+0000 7fcd44a5d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd400d08a0 con 0x7fcd40099fb0 2024-04-25T08:36:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.874+0000 7fcd3dd9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcd4006cff0 0x7fcd40099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.874+0000 7fcd3e59c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcd4006cc10 0x7fcd400cfa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34834/0 (socket says 172.21.15.38:34834) 2024-04-25T08:36:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.874+0000 7fcd3e59c640 1 -- 172.21.15.38:0/3591896168 learned_addr learned my addr 172.21.15.38:0/3591896168 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.875+0000 7fcd3dd9b640 1 -- 172.21.15.38:0/3591896168 >> v2:172.21.15.107:3300/0 conn(0x7fcd40099fb0 msgr2=0x7fcd4015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.875+0000 7fcd3ed9d640 1 --2- 172.21.15.38:0/3591896168 >> v2:172.21.15.107:3300/0 conn(0x7fcd40099fb0 0x7fcd4015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.875+0000 7fcd3dd9b640 1 --2- 172.21.15.38:0/3591896168 >> v2:172.21.15.107:3300/0 conn(0x7fcd40099fb0 0x7fcd4015d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.875+0000 7fcd3d59a640 1 -- 172.21.15.38:0/3591896168 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599678295 0 0) 0x7fcd38001110 con 0x7fcd4006cc10 2024-04-25T08:36:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.875+0000 7fcd3dd9b640 1 -- 172.21.15.38:0/3591896168 >> v1:172.21.15.38:6789/0 conn(0x7fcd4006cc10 legacy=0x7fcd400cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:09.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.875+0000 7fcd3dd9b640 1 -- 172.21.15.38:0/3591896168 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd400d0b80 con 0x7fcd4006cff0 2024-04-25T08:36:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.876+0000 7fcd3dd9b640 1 --2- 172.21.15.38:0/3591896168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4006cff0 0x7fcd40099a70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fcd40161700 tx=0x7fcd2802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82279bbe64278ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.876+0000 7fcd3d59a640 1 -- 172.21.15.38:0/3591896168 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28037070 con 0x7fcd4006cff0 2024-04-25T08:36:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.876+0000 7fcd3d59a640 1 -- 172.21.15.38:0/3591896168 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd28032440 con 0x7fcd4006cff0 2024-04-25T08:36:09.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.876+0000 7fcd3d59a640 1 -- 172.21.15.38:0/3591896168 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd28032810 con 0x7fcd4006cff0 2024-04-25T08:36:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.876+0000 7fcd44a5d640 1 -- 172.21.15.38:0/3591896168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4006cff0 msgr2=0x7fcd40099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.876+0000 7fcd44a5d640 1 --2- 172.21.15.38:0/3591896168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4006cff0 0x7fcd40099a70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fcd40161700 tx=0x7fcd2802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:36:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.876+0000 7fcd44a5d640 1 -- 172.21.15.38:0/3591896168 shutdown_connections 2024-04-25T08:36:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.876+0000 7fcd44a5d640 1 --2- 172.21.15.38:0/3591896168 >> v2:172.21.15.107:3300/0 conn(0x7fcd40099fb0 0x7fcd4015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.876+0000 7fcd44a5d640 1 --2- 172.21.15.38:0/3591896168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4006cff0 0x7fcd40099a70 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.876+0000 7fcd44a5d640 1 -- 172.21.15.38:0/3591896168 >> 172.21.15.38:0/3591896168 conn(0x7fcd400b31f0 msgr2=0x7fcd400b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.876+0000 7fcd44a5d640 1 -- 172.21.15.38:0/3591896168 shutdown_connections 2024-04-25T08:36:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.876+0000 7fcd44a5d640 1 -- 172.21.15.38:0/3591896168 wait complete. 2024-04-25T08:36:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.878+0000 7fcd44a5d640 1 Processor -- start 2024-04-25T08:36:09.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.878+0000 7fcd44a5d640 1 -- start start 2024-04-25T08:36:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.878+0000 7fcd44a5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4006cc10 0x7fcd400b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.878+0000 7fcd44a5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd4006cff0 0x7fcd40158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd44a5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd40099fb0 0x7fcd400c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd44a5d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4006c150 con 0x7fcd4006cc10 2024-04-25T08:36:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd44a5d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400d1140 con 0x7fcd4006cff0 2024-04-25T08:36:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd44a5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4006c2d0 con 0x7fcd40099fb0 2024-04-25T08:36:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd3e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4006cc10 0x7fcd400b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd3e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4006cc10 0x7fcd400b4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53778/0 (socket says 172.21.15.38:53778) 2024-04-25T08:36:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd3e59c640 1 -- 172.21.15.38:0/2101667868 learned_addr learned my addr 172.21.15.38:0/2101667868 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd3ed9d640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd40099fb0 0x7fcd400c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd3dd9b640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd4006cff0 0x7fcd40158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd3e59c640 1 -- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd40099fb0 msgr2=0x7fcd400c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd3e59c640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd40099fb0 0x7fcd400c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:09.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd3e59c640 1 -- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd4006cff0 msgr2=0x7fcd40158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd3e59c640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd4006cff0 0x7fcd40158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd3e59c640 1 -- 172.21.15.38:0/2101667868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd400b4370 con 0x7fcd4006cc10 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd3e59c640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4006cc10 0x7fcd400b4e10 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800c780 tx=0x7fcd3800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd36ffd640 1 -- 172.21.15.38:0/2101667868 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38018070 con 0x7fcd4006cc10 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd36ffd640 1 -- 172.21.15.38:0/2101667868 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd380072d0 con 0x7fcd4006cc10 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd44a5d640 1 -- 172.21.15.38:0/2101667868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd400c6440 con 0x7fcd4006cc10 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd36ffd640 1 -- 172.21.15.38:0/2101667868 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd380047e0 con 0x7fcd4006cc10 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.879+0000 7fcd44a5d640 1 -- 172.21.15.38:0/2101667868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd400b48f0 con 0x7fcd4006cc10 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.881+0000 7fcd44a5d640 1 -- 172.21.15.38:0/2101667868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd400c67e0 con 0x7fcd4006cc10 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.881+0000 7fcd36ffd640 1 -- 172.21.15.38:0/2101667868 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd3801b070 con 0x7fcd4006cc10 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.881+0000 7fcd36ffd640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd2006b4d0 0x7fcd2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.882+0000 7fcd36ffd640 1 -- 172.21.15.38:0/2101667868 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcd38089750 con 0x7fcd4006cc10 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.882+0000 7fcd3dd9b640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd2006b4d0 0x7fcd2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:09.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.882+0000 7fcd3dd9b640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd2006b4d0 0x7fcd2006d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcd28034000 tx=0x7fcd2802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:09.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:09.886+0000 7fcd36ffd640 1 -- 172.21.15.38:0/2101667868 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd4009a3c0 con 0x7fcd4006cc10 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.070+0000 7fcd44a5d640 1 -- 172.21.15.38:0/2101667868 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd400cefc0 con 0x7fcd4006cc10 2024-04-25T08:36:10.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.071+0000 7fcd36ffd640 1 -- 172.21.15.38:0/2101667868 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16999 (secure 0 0 0) 0x7fcd400cefc0 con 0x7fcd4006cc10 2024-04-25T08:36:10.066 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:10.066 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":71,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:52.358128+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:38.884603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:39.590140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/239365678":"2024-04-26T08:35:52.345208+0000","172.21.15.38:6833/239365678":"2024-04-26T08:35:52.345208+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:10.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.073+0000 7fcd44a5d640 1 -- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd2006b4d0 msgr2=0x7fcd2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.073+0000 7fcd44a5d640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd2006b4d0 0x7fcd2006d990 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcd28034000 tx=0x7fcd2802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:10.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.073+0000 7fcd44a5d640 1 -- 172.21.15.38:0/2101667868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4006cc10 msgr2=0x7fcd400b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.073+0000 7fcd44a5d640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4006cc10 0x7fcd400b4e10 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800c780 tx=0x7fcd3800cc40 comp rx=0 tx=0).stop 2024-04-25T08:36:10.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.074+0000 7fcd44a5d640 1 -- 172.21.15.38:0/2101667868 shutdown_connections 2024-04-25T08:36:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.074+0000 7fcd44a5d640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd2006b4d0 0x7fcd2006d990 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.074+0000 7fcd44a5d640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd40099fb0 0x7fcd400c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.074+0000 7fcd44a5d640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd4006cff0 0x7fcd40158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.074+0000 7fcd44a5d640 1 --2- 172.21.15.38:0/2101667868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd4006cc10 0x7fcd400b4e10 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.074+0000 7fcd44a5d640 1 -- 172.21.15.38:0/2101667868 >> 172.21.15.38:0/2101667868 conn(0x7fcd400b31f0 msgr2=0x7fcd400ae3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.074+0000 7fcd44a5d640 1 -- 172.21.15.38:0/2101667868 shutdown_connections 2024-04-25T08:36:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.074+0000 7fcd44a5d640 1 -- 172.21.15.38:0/2101667868 wait complete. 2024-04-25T08:36:10.079 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:35:38.884603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:35:39.590140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:36:10.079 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:36:10.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.190+0000 7f66af1c4640 1 Processor -- start 2024-04-25T08:36:10.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.190+0000 7f66af1c4640 1 -- start start 2024-04-25T08:36:10.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.190+0000 7f66af1c4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f66a80d0320 0x7f66a80d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.191+0000 7f66af1c4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a80d2060 con 0x7f66a80b6980 2024-04-25T08:36:10.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.191+0000 7f66af1c4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a80d2330 con 0x7f66a80d0700 2024-04-25T08:36:10.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.191+0000 7f66af1c4640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a80d2610 con 0x7f66a80d0320 2024-04-25T08:36:10.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.191+0000 7f66acf60640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f66a80d0320 0x7f66a80d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.191+0000 7f66ad761640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f66a80b6980 0x7f66a80b6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34836/0 (socket says 172.21.15.38:34836) 2024-04-25T08:36:10.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.191+0000 7f66ad761640 1 -- 172.21.15.38:0/1234494348 learned_addr learned my addr 172.21.15.38:0/1234494348 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.191+0000 7f66acf60640 1 -- 172.21.15.38:0/1234494348 >> v1:172.21.15.107:6789/0 conn(0x7f66a80d0700 legacy=0x7f66a80d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.191+0000 7f66acf60640 1 -- 172.21.15.38:0/1234494348 >> v1:172.21.15.38:6789/0 conn(0x7f66a80b6980 legacy=0x7f66a80b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.191+0000 7f66a77fe640 1 -- 172.21.15.38:0/1234494348 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531983675 0 0) 0x7f66a80d2060 con 0x7f66a80b6980 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.191+0000 7f66acf60640 1 -- 172.21.15.38:0/1234494348 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a80d28f0 con 0x7f66a80d0320 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.192+0000 7f66acf60640 1 --2- 172.21.15.38:0/1234494348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a80d0320 0x7f66a80d33d0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f669c0097a0 tx=0x7f669c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84162315825c77bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.192+0000 7f66a77fe640 1 -- 172.21.15.38:0/1234494348 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c037070 con 0x7f66a80d0320 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.192+0000 7f66a77fe640 1 -- 172.21.15.38:0/1234494348 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f669c032640 con 0x7f66a80d0320 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.192+0000 7f66a77fe640 1 -- 172.21.15.38:0/1234494348 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f669c032a10 con 0x7f66a80d0320 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.192+0000 7f66af1c4640 1 -- 172.21.15.38:0/1234494348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a80d0320 msgr2=0x7f66a80d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.192+0000 7f66af1c4640 1 --2- 172.21.15.38:0/1234494348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a80d0320 0x7f66a80d33d0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f669c0097a0 tx=0x7f669c02a2c0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.192+0000 7f66af1c4640 1 -- 172.21.15.38:0/1234494348 shutdown_connections 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.192+0000 7f66af1c4640 1 --2- 172.21.15.38:0/1234494348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a80d0320 0x7f66a80d33d0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.192+0000 7f66af1c4640 1 -- 172.21.15.38:0/1234494348 >> 172.21.15.38:0/1234494348 conn(0x7f66a80668f0 msgr2=0x7f66a8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:10.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.193+0000 7f66af1c4640 1 -- 172.21.15.38:0/1234494348 shutdown_connections 2024-04-25T08:36:10.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.193+0000 7f66af1c4640 1 -- 172.21.15.38:0/1234494348 wait complete. 2024-04-25T08:36:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.194+0000 7f66af1c4640 1 Processor -- start 2024-04-25T08:36:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.194+0000 7f66af1c4640 1 -- start start 2024-04-25T08:36:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.194+0000 7f66af1c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66a80b6980 0x7f66a81f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.194+0000 7f66af1c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f66a80d0320 0x7f66a81e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66af1c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a80d0700 0x7f66a81e81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66af1c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a80d0b40 con 0x7f66a80d0700 2024-04-25T08:36:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66af1c4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a80d2eb0 con 0x7f66a80b6980 2024-04-25T08:36:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66af1c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a81f6b40 con 0x7f66a80d0320 2024-04-25T08:36:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66ad761640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a80d0700 0x7f66a81e81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66ad761640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a80d0700 0x7f66a81e81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53800/0 (socket says 172.21.15.38:53800) 2024-04-25T08:36:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66ad761640 1 -- 172.21.15.38:0/3844985428 learned_addr learned my addr 172.21.15.38:0/3844985428 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66acf60640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66a80b6980 0x7f66a81f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66a7fff640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f66a80d0320 0x7f66a81e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66ad761640 1 -- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f66a80d0320 msgr2=0x7f66a81e7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66ad761640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f66a80d0320 0x7f66a81e7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66ad761640 1 -- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66a80b6980 msgr2=0x7f66a81f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66ad761640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66a80b6980 0x7f66a81f6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66ad761640 1 -- 172.21.15.38:0/3844985428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a80b3e00 con 0x7f66a80d0700 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66acf60640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66a80b6980 0x7f66a81f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66ad761640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a80d0700 0x7f66a81e81d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f66a000c7a0 tx=0x7f66a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66a57fa640 1 -- 172.21.15.38:0/3844985428 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0018070 con 0x7f66a80d0700 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66a57fa640 1 -- 172.21.15.38:0/3844985428 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a000a640 con 0x7f66a80d0700 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66a57fa640 1 -- 172.21.15.38:0/3844985428 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a000aab0 con 0x7f66a80d0700 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66af1c4640 1 -- 172.21.15.38:0/3844985428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a80b40e0 con 0x7f66a80d0700 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.195+0000 7f66af1c4640 1 -- 172.21.15.38:0/3844985428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a80b46a0 con 0x7f66a80d0700 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.197+0000 7f66af1c4640 1 -- 172.21.15.38:0/3844985428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a8068e80 con 0x7f66a80d0700 2024-04-25T08:36:10.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.197+0000 7f66a57fa640 1 -- 172.21.15.38:0/3844985428 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66a000ac50 con 0x7f66a80d0700 2024-04-25T08:36:10.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.197+0000 7f66a57fa640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f668c06b4d0 0x7f668c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.197+0000 7f66a57fa640 1 -- 172.21.15.38:0/3844985428 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f66a00898c0 con 0x7f66a80d0700 2024-04-25T08:36:10.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.198+0000 7f66acf60640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f668c06b4d0 0x7f668c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.198+0000 7f66acf60640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f668c06b4d0 0x7f668c06d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f669c034000 tx=0x7f669c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.201+0000 7f66a57fa640 1 -- 172.21.15.38:0/3844985428 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f66a0057fa0 con 0x7f66a80d0700 2024-04-25T08:36:10.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.384+0000 7f66af1c4640 1 -- 172.21.15.38:0/3844985428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66a80b3050 con 0x7f66a80d0700 2024-04-25T08:36:10.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.384+0000 7f66a57fa640 1 -- 172.21.15.38:0/3844985428 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66a005c040 con 0x7f66a80d0700 2024-04-25T08:36:10.379 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:36:10.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.386+0000 7f66af1c4640 1 -- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f668c06b4d0 msgr2=0x7f668c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.386+0000 7f66af1c4640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f668c06b4d0 0x7f668c06d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f669c034000 tx=0x7f669c02e000 comp rx=0 tx=0).stop 2024-04-25T08:36:10.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.386+0000 7f66af1c4640 1 -- 172.21.15.38:0/3844985428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a80d0700 msgr2=0x7f66a81e81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.386+0000 7f66af1c4640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a80d0700 0x7f66a81e81d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f66a000c7a0 tx=0x7f66a000cc60 comp rx=0 tx=0).stop 2024-04-25T08:36:10.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.386+0000 7f66af1c4640 1 -- 172.21.15.38:0/3844985428 shutdown_connections 2024-04-25T08:36:10.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.386+0000 7f66af1c4640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f66a80d0700 0x7f66a81e81d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.386+0000 7f66af1c4640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f668c06b4d0 0x7f668c06d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.386+0000 7f66af1c4640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f66a80d0320 0x7f66a81e7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.386+0000 7f66af1c4640 1 --2- 172.21.15.38:0/3844985428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f66a80b6980 0x7f66a81f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.386+0000 7f66af1c4640 1 -- 172.21.15.38:0/3844985428 >> 172.21.15.38:0/3844985428 conn(0x7f66a80668f0 msgr2=0x7f66a8160430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:10.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.386+0000 7f66af1c4640 1 -- 172.21.15.38:0/3844985428 shutdown_connections 2024-04-25T08:36:10.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.386+0000 7f66af1c4640 1 -- 172.21.15.38:0/3844985428 wait complete. 2024-04-25T08:36:10.392 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:36:10.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.502+0000 7f56483d5640 1 Processor -- start 2024-04-25T08:36:10.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.502+0000 7f56483d5640 1 -- start start 2024-04-25T08:36:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.503+0000 7f56483d5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f56400d0320 0x7f56400d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.503+0000 7f56483d5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56400d0700 0x7f56400d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.503+0000 7f56483d5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56400d2330 con 0x7f56400d1e90 2024-04-25T08:36:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.503+0000 7f56483d5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56400d2610 con 0x7f56400d0320 2024-04-25T08:36:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.503+0000 7f56483d5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56400d28f0 con 0x7f56400d0700 2024-04-25T08:36:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.504+0000 7f5646171640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f56400d0320 0x7f56400d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.504+0000 7f5646972640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f56400d1e90 0x7f56400b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34850/0 (socket says 172.21.15.38:34850) 2024-04-25T08:36:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.504+0000 7f5646972640 1 -- 172.21.15.38:0/1031257840 learned_addr learned my addr 172.21.15.38:0/1031257840 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.504+0000 7f5645970640 1 --2- 172.21.15.38:0/1031257840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56400d0700 0x7f56400d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.504+0000 7f564516f640 1 -- 172.21.15.38:0/1031257840 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998825841 0 0) 0x7f56400d2330 con 0x7f56400d1e90 2024-04-25T08:36:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.504+0000 7f564516f640 1 -- 172.21.15.38:0/1031257840 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f562c003520 con 0x7f56400d1e90 2024-04-25T08:36:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.504+0000 7f5646171640 1 -- 172.21.15.38:0/1031257840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56400d0700 msgr2=0x7f56400d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.504+0000 7f5646171640 1 --2- 172.21.15.38:0/1031257840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56400d0700 0x7f56400d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.504+0000 7f5646171640 1 -- 172.21.15.38:0/1031257840 >> v1:172.21.15.38:6789/0 conn(0x7f56400d1e90 legacy=0x7f56400b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.504+0000 7f5646171640 1 -- 172.21.15.38:0/1031257840 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56400d2bd0 con 0x7f56400d0320 2024-04-25T08:36:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.504+0000 7f5645970640 1 --2- 172.21.15.38:0/1031257840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56400d0700 0x7f56400d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.505+0000 7f5646171640 1 --2- 172.21.15.38:0/1031257840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56400d0320 0x7f56400d33d0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f563c0097a0 tx=0x7f563c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ba850990020e00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.505+0000 7f564516f640 1 -- 172.21.15.38:0/1031257840 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c037070 con 0x7f56400d0320 2024-04-25T08:36:10.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.505+0000 7f564516f640 1 -- 172.21.15.38:0/1031257840 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c032440 con 0x7f56400d0320 2024-04-25T08:36:10.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.505+0000 7f564516f640 1 -- 172.21.15.38:0/1031257840 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c032810 con 0x7f56400d0320 2024-04-25T08:36:10.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.505+0000 7f56483d5640 1 -- 172.21.15.38:0/1031257840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56400d0320 msgr2=0x7f56400d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.505+0000 7f56483d5640 1 --2- 172.21.15.38:0/1031257840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56400d0320 0x7f56400d33d0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f563c0097a0 tx=0x7f563c02a210 comp rx=0 tx=0).stop 2024-04-25T08:36:10.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.505+0000 7f56483d5640 1 -- 172.21.15.38:0/1031257840 shutdown_connections 2024-04-25T08:36:10.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.505+0000 7f56483d5640 1 --2- 172.21.15.38:0/1031257840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56400d0700 0x7f56400d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.505+0000 7f56483d5640 1 --2- 172.21.15.38:0/1031257840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56400d0320 0x7f56400d33d0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.505+0000 7f56483d5640 1 -- 172.21.15.38:0/1031257840 >> 172.21.15.38:0/1031257840 conn(0x7f56400668f0 msgr2=0x7f5640066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:10.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.505+0000 7f56483d5640 1 -- 172.21.15.38:0/1031257840 shutdown_connections 2024-04-25T08:36:10.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.505+0000 7f56483d5640 1 -- 172.21.15.38:0/1031257840 wait complete. 2024-04-25T08:36:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.507+0000 7f56483d5640 1 Processor -- start 2024-04-25T08:36:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.507+0000 7f56483d5640 1 -- start start 2024-04-25T08:36:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.507+0000 7f56483d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56400d0320 0x7f564014ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.507+0000 7f56483d5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56400d0700 0x7f56400cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.507+0000 7f56483d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56400d1e90 0x7f5640156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.507+0000 7f56483d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56400d0b40 con 0x7f56400d0320 2024-04-25T08:36:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.507+0000 7f56483d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5640161210 con 0x7f56400d1e90 2024-04-25T08:36:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.507+0000 7f56483d5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56400cd210 con 0x7f56400d0700 2024-04-25T08:36:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.507+0000 7f5646171640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56400d0320 0x7f564014ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.507+0000 7f5646171640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56400d0320 0x7f564014ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53822/0 (socket says 172.21.15.38:53822) 2024-04-25T08:36:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.507+0000 7f5646171640 1 -- 172.21.15.38:0/4135456711 learned_addr learned my addr 172.21.15.38:0/4135456711 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.508+0000 7f5646171640 1 -- 172.21.15.38:0/4135456711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56400d0700 msgr2=0x7f56400cbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.508+0000 7f5646171640 1 --2- 172.21.15.38:0/4135456711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56400d0700 0x7f56400cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.508+0000 7f5646171640 1 -- 172.21.15.38:0/4135456711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56400d1e90 msgr2=0x7f5640156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.508+0000 7f5646171640 1 --2- 172.21.15.38:0/4135456711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56400d1e90 0x7f5640156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.508+0000 7f5646171640 1 -- 172.21.15.38:0/4135456711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56400cc600 con 0x7f56400d0320 2024-04-25T08:36:10.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.508+0000 7f5646171640 1 --2- 172.21.15.38:0/4135456711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56400d0320 0x7f564014ae10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f563c035040 tx=0x7f563c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.508+0000 7f5636ffd640 1 -- 172.21.15.38:0/4135456711 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c037040 con 0x7f56400d0320 2024-04-25T08:36:10.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.508+0000 7f5636ffd640 1 -- 172.21.15.38:0/4135456711 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c030500 con 0x7f56400d0320 2024-04-25T08:36:10.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.508+0000 7f56483d5640 1 -- 172.21.15.38:0/4135456711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56400cc8e0 con 0x7f56400d0320 2024-04-25T08:36:10.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.508+0000 7f5636ffd640 1 -- 172.21.15.38:0/4135456711 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c0308d0 con 0x7f56400d0320 2024-04-25T08:36:10.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.508+0000 7f56483d5640 1 -- 172.21.15.38:0/4135456711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56400ccea0 con 0x7f56400d0320 2024-04-25T08:36:10.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.509+0000 7f56483d5640 1 -- 172.21.15.38:0/4135456711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5640068e80 con 0x7f56400d0320 2024-04-25T08:36:10.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.509+0000 7f5636ffd640 1 -- 172.21.15.38:0/4135456711 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f563c044020 con 0x7f56400d0320 2024-04-25T08:36:10.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.510+0000 7f5636ffd640 1 --2- 172.21.15.38:0/4135456711 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f562406b4d0 0x7f562406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.510+0000 7f5636ffd640 1 -- 172.21.15.38:0/4135456711 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f563c0aa610 con 0x7f56400d0320 2024-04-25T08:36:10.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.510+0000 7f5645970640 1 --2- 172.21.15.38:0/4135456711 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f562406b4d0 0x7f562406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.511+0000 7f5645970640 1 --2- 172.21.15.38:0/4135456711 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f562406b4d0 0x7f562406d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5630006fd0 tx=0x7f5630008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.514+0000 7f5636ffd640 1 -- 172.21.15.38:0/4135456711 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f563c0792b0 con 0x7f56400d0320 2024-04-25T08:36:10.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.693+0000 7f56483d5640 1 -- 172.21.15.38:0/4135456711 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f56400b3050 con 0x7f56400d0320 2024-04-25T08:36:10.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.693+0000 7f5636ffd640 1 -- 172.21.15.38:0/4135456711 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f563c078bb0 con 0x7f56400d0320 2024-04-25T08:36:10.687 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:36:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.695+0000 7f56483d5640 1 -- 172.21.15.38:0/4135456711 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f562406b4d0 msgr2=0x7f562406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.695+0000 7f56483d5640 1 --2- 172.21.15.38:0/4135456711 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f562406b4d0 0x7f562406d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5630006fd0 tx=0x7f5630008040 comp rx=0 tx=0).stop 2024-04-25T08:36:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.695+0000 7f56483d5640 1 -- 172.21.15.38:0/4135456711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56400d0320 msgr2=0x7f564014ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.695+0000 7f56483d5640 1 --2- 172.21.15.38:0/4135456711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56400d0320 0x7f564014ae10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f563c035040 tx=0x7f563c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.695+0000 7f56483d5640 1 -- 172.21.15.38:0/4135456711 shutdown_connections 2024-04-25T08:36:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.695+0000 7f56483d5640 1 --2- 172.21.15.38:0/4135456711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56400d1e90 0x7f5640156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.695+0000 7f56483d5640 1 --2- 172.21.15.38:0/4135456711 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f562406b4d0 0x7f562406d990 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.695+0000 7f56483d5640 1 --2- 172.21.15.38:0/4135456711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56400d0700 0x7f56400cbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.695+0000 7f56483d5640 1 --2- 172.21.15.38:0/4135456711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f56400d0320 0x7f564014ae10 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.695+0000 7f56483d5640 1 -- 172.21.15.38:0/4135456711 >> 172.21.15.38:0/4135456711 conn(0x7f56400668f0 msgr2=0x7f564015e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:10.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.695+0000 7f56483d5640 1 -- 172.21.15.38:0/4135456711 shutdown_connections 2024-04-25T08:36:10.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.695+0000 7f56483d5640 1 -- 172.21.15.38:0/4135456711 wait complete. 2024-04-25T08:36:10.700 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:36:10.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.809+0000 7f6ed715c640 1 Processor -- start 2024-04-25T08:36:10.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.809+0000 7f6ed715c640 1 -- start start 2024-04-25T08:36:10.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.810+0000 7f6ed715c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ed00d0320 0x7f6ed00d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.810+0000 7f6ed715c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed00d2060 con 0x7f6ed00d0700 2024-04-25T08:36:10.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.810+0000 7f6ed715c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed00d2330 con 0x7f6ed00b6980 2024-04-25T08:36:10.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.810+0000 7f6ed715c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed00d2610 con 0x7f6ed00d0320 2024-04-25T08:36:10.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.810+0000 7f6ecffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6ed00d0700 0x7f6ed00d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34854/0 (socket says 172.21.15.38:34854) 2024-04-25T08:36:10.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.810+0000 7f6ecffff640 1 -- 172.21.15.38:0/1545278042 learned_addr learned my addr 172.21.15.38:0/1545278042 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:10.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.810+0000 7f6ed4ef8640 1 --2- 172.21.15.38:0/1545278042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ed00d0320 0x7f6ed00d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.811+0000 7f6ecf7fe640 1 -- 172.21.15.38:0/1545278042 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723935998 0 0) 0x7f6ed00d2060 con 0x7f6ed00d0700 2024-04-25T08:36:10.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.811+0000 7f6ecf7fe640 1 -- 172.21.15.38:0/1545278042 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ebc003520 con 0x7f6ed00d0700 2024-04-25T08:36:10.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.811+0000 7f6ecf7fe640 1 -- 172.21.15.38:0/1545278042 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623716952 0 0) 0x7f6ed00d2330 con 0x7f6ed00b6980 2024-04-25T08:36:10.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.811+0000 7f6ecf7fe640 1 -- 172.21.15.38:0/1545278042 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ed00d2060 con 0x7f6ed00b6980 2024-04-25T08:36:10.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.811+0000 7f6ed4ef8640 1 -- 172.21.15.38:0/1545278042 >> v1:172.21.15.107:6789/0 conn(0x7f6ed00b6980 legacy=0x7f6ed00b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.811+0000 7f6ed4ef8640 1 -- 172.21.15.38:0/1545278042 >> v1:172.21.15.38:6789/0 conn(0x7f6ed00d0700 legacy=0x7f6ed00d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.811+0000 7f6ecf7fe640 1 -- 172.21.15.38:0/1545278042 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1145637830 0 0) 0x7f6ebc003520 con 0x7f6ed00d0700 2024-04-25T08:36:10.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.811+0000 7f6ed4ef8640 1 -- 172.21.15.38:0/1545278042 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed00d28f0 con 0x7f6ed00d0320 2024-04-25T08:36:10.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.812+0000 7f6ed4ef8640 1 --2- 172.21.15.38:0/1545278042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ed00d0320 0x7f6ed00d33d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009dc0 tx=0x7f6ec802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fa932295b9742e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.812+0000 7f6ecf7fe640 1 -- 172.21.15.38:0/1545278042 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0001670 con 0x7f6ed00d0320 2024-04-25T08:36:10.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.812+0000 7f6ecf7fe640 1 -- 172.21.15.38:0/1545278042 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec802ae50 con 0x7f6ed00d0320 2024-04-25T08:36:10.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.812+0000 7f6ecf7fe640 1 -- 172.21.15.38:0/1545278042 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8007740 con 0x7f6ed00d0320 2024-04-25T08:36:10.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.812+0000 7f6ed715c640 1 -- 172.21.15.38:0/1545278042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ed00d0320 msgr2=0x7f6ed00d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.812+0000 7f6ed715c640 1 --2- 172.21.15.38:0/1545278042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ed00d0320 0x7f6ed00d33d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009dc0 tx=0x7f6ec802a6f0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.812+0000 7f6ed715c640 1 -- 172.21.15.38:0/1545278042 shutdown_connections 2024-04-25T08:36:10.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.812+0000 7f6ed715c640 1 --2- 172.21.15.38:0/1545278042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ed00d0320 0x7f6ed00d33d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.812+0000 7f6ed715c640 1 -- 172.21.15.38:0/1545278042 >> 172.21.15.38:0/1545278042 conn(0x7f6ed00668f0 msgr2=0x7f6ed0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:10.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.812+0000 7f6ed715c640 1 -- 172.21.15.38:0/1545278042 shutdown_connections 2024-04-25T08:36:10.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.812+0000 7f6ed715c640 1 -- 172.21.15.38:0/1545278042 wait complete. 2024-04-25T08:36:10.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.813+0000 7f6ed715c640 1 Processor -- start 2024-04-25T08:36:10.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.814+0000 7f6ed715c640 1 -- start start 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.814+0000 7f6ed715c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ed00b6980 0x7f6ed00ab430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.814+0000 7f6ed715c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ed00d0320 0x7f6ed00abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.814+0000 7f6ed715c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ed00d0700 0x7f6ed00ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.814+0000 7f6ed715c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed00d0b40 con 0x7f6ed00d0320 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.814+0000 7f6ed715c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed00d2eb0 con 0x7f6ed00d0700 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.814+0000 7f6ed715c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed00b07d0 con 0x7f6ed00b6980 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.814+0000 7f6ecffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ed00d0320 0x7f6ed00abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ecffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ed00d0320 0x7f6ed00abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53830/0 (socket says 172.21.15.38:53830) 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ecffff640 1 -- 172.21.15.38:0/2781226366 learned_addr learned my addr 172.21.15.38:0/2781226366 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ed4ef8640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ed00b6980 0x7f6ed00ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ed56f9640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ed00d0700 0x7f6ed00ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ecffff640 1 -- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ed00b6980 msgr2=0x7f6ed00ab430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ecffff640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ed00b6980 0x7f6ed00ab430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ecffff640 1 -- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ed00d0700 msgr2=0x7f6ed00ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ecffff640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ed00d0700 0x7f6ed00ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ecffff640 1 -- 172.21.15.38:0/2781226366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed00aa3d0 con 0x7f6ed00d0320 2024-04-25T08:36:10.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ed4ef8640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ed00b6980 0x7f6ed00ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:10.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ecffff640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ed00d0320 0x7f6ed00abe90 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000c4f0 tx=0x7f6ec000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ecd7fa640 1 -- 172.21.15.38:0/2781226366 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0017070 con 0x7f6ed00d0320 2024-04-25T08:36:10.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ecd7fa640 1 -- 172.21.15.38:0/2781226366 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec0005ce0 con 0x7f6ed00d0320 2024-04-25T08:36:10.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ed715c640 1 -- 172.21.15.38:0/2781226366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed00aa670 con 0x7f6ed00d0320 2024-04-25T08:36:10.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ecd7fa640 1 -- 172.21.15.38:0/2781226366 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0007290 con 0x7f6ed00d0320 2024-04-25T08:36:10.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.815+0000 7f6ed715c640 1 -- 172.21.15.38:0/2781226366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed00aac30 con 0x7f6ed00d0320 2024-04-25T08:36:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.816+0000 7f6ed715c640 1 -- 172.21.15.38:0/2781226366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed0068e80 con 0x7f6ed00d0320 2024-04-25T08:36:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.817+0000 7f6ecd7fa640 1 -- 172.21.15.38:0/2781226366 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ec001a070 con 0x7f6ed00d0320 2024-04-25T08:36:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.817+0000 7f6ecd7fa640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6eb006b4d0 0x7f6eb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.817+0000 7f6ecd7fa640 1 -- 172.21.15.38:0/2781226366 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6ec00897f0 con 0x7f6ed00d0320 2024-04-25T08:36:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.817+0000 7f6ed4ef8640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6eb006b4d0 0x7f6eb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.818+0000 7f6ed4ef8640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6eb006b4d0 0x7f6eb006d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8034000 tx=0x7f6ec802f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:10.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.821+0000 7f6ecd7fa640 1 -- 172.21.15.38:0/2781226366 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ec0058520 con 0x7f6ed00d0320 2024-04-25T08:36:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.999+0000 7f6ed715c640 1 -- 172.21.15.38:0/2781226366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6ed00b8150 con 0x7f6ed00d0320 2024-04-25T08:36:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:10.999+0000 7f6ecd7fa640 1 -- 172.21.15.38:0/2781226366 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6ec0057e20 con 0x7f6ed00d0320 2024-04-25T08:36:10.994 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:36:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.001+0000 7f6ed715c640 1 -- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6eb006b4d0 msgr2=0x7f6eb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.001+0000 7f6ed715c640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6eb006b4d0 0x7f6eb006d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8034000 tx=0x7f6ec802f000 comp rx=0 tx=0).stop 2024-04-25T08:36:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.001+0000 7f6ed715c640 1 -- 172.21.15.38:0/2781226366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ed00d0320 msgr2=0x7f6ed00abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.001+0000 7f6ed715c640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ed00d0320 0x7f6ed00abe90 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6ec000c4f0 tx=0x7f6ec000c9b0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.001+0000 7f6ed715c640 1 -- 172.21.15.38:0/2781226366 shutdown_connections 2024-04-25T08:36:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.001+0000 7f6ed715c640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ed00d0700 0x7f6ed00ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.001+0000 7f6ed715c640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ed00d0320 0x7f6ed00abe90 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.001+0000 7f6ed715c640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6eb006b4d0 0x7f6eb006d990 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.001+0000 7f6ed715c640 1 --2- 172.21.15.38:0/2781226366 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ed00b6980 0x7f6ed00ab430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.001+0000 7f6ed715c640 1 -- 172.21.15.38:0/2781226366 >> 172.21.15.38:0/2781226366 conn(0x7f6ed00668f0 msgr2=0x7f6ed00d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.001+0000 7f6ed715c640 1 -- 172.21.15.38:0/2781226366 shutdown_connections 2024-04-25T08:36:10.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.001+0000 7f6ed715c640 1 -- 172.21.15.38:0/2781226366 wait complete. 2024-04-25T08:36:11.007 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:36:11.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.114+0000 7ff9719e0640 1 Processor -- start 2024-04-25T08:36:11.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.114+0000 7ff9719e0640 1 -- start start 2024-04-25T08:36:11.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff9719e0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff96c15dcc0 0x7ff96c0d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff9719e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff96c0d1eb0 0x7ff96c0d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff9719e0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96c0d24a0 con 0x7ff96c0d0720 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff9719e0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96c0d0bb0 con 0x7ff96c15dcc0 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff9719e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0d3540 con 0x7ff96c0d1eb0 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff96affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff96c15dcc0 0x7ff96c0d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff96b7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff96c0d0720 0x7ff96c0b5270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34868/0 (socket says 172.21.15.38:34868) 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff96b7fe640 1 -- 172.21.15.38:0/3729890006 learned_addr learned my addr 172.21.15.38:0/3729890006 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff96a7fc640 1 --2- 172.21.15.38:0/3729890006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff96c0d1eb0 0x7ff96c0d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff969ffb640 1 -- 172.21.15.38:0/3729890006 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3442168093 0 0) 0x7ff96c0d24a0 con 0x7ff96c0d0720 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff969ffb640 1 -- 172.21.15.38:0/3729890006 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff958003520 con 0x7ff96c0d0720 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff96affd640 1 -- 172.21.15.38:0/3729890006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff96c0d1eb0 msgr2=0x7ff96c0d01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.115+0000 7ff96affd640 1 --2- 172.21.15.38:0/3729890006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff96c0d1eb0 0x7ff96c0d01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.116+0000 7ff96affd640 1 -- 172.21.15.38:0/3729890006 >> v1:172.21.15.38:6789/0 conn(0x7ff96c0d0720 legacy=0x7ff96c0b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.116+0000 7ff96affd640 1 -- 172.21.15.38:0/3729890006 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c0d2780 con 0x7ff96c15dcc0 2024-04-25T08:36:11.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.116+0000 7ff96a7fc640 1 --2- 172.21.15.38:0/3729890006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff96c0d1eb0 0x7ff96c0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:11.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.116+0000 7ff96affd640 1 --2- 172.21.15.38:0/3729890006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff96c15dcc0 0x7ff96c0d1970 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff9640097a0 tx=0x7ff96402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6ad3f5355bbc92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:11.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.116+0000 7ff969ffb640 1 -- 172.21.15.38:0/3729890006 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964037070 con 0x7ff96c15dcc0 2024-04-25T08:36:11.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.116+0000 7ff969ffb640 1 -- 172.21.15.38:0/3729890006 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff964032440 con 0x7ff96c15dcc0 2024-04-25T08:36:11.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.116+0000 7ff969ffb640 1 -- 172.21.15.38:0/3729890006 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964032810 con 0x7ff96c15dcc0 2024-04-25T08:36:11.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.117+0000 7ff9719e0640 1 -- 172.21.15.38:0/3729890006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff96c15dcc0 msgr2=0x7ff96c0d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.117+0000 7ff9719e0640 1 --2- 172.21.15.38:0/3729890006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff96c15dcc0 0x7ff96c0d1970 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff9640097a0 tx=0x7ff96402a210 comp rx=0 tx=0).stop 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.117+0000 7ff9719e0640 1 -- 172.21.15.38:0/3729890006 shutdown_connections 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.117+0000 7ff9719e0640 1 --2- 172.21.15.38:0/3729890006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff96c0d1eb0 0x7ff96c0d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.117+0000 7ff9719e0640 1 --2- 172.21.15.38:0/3729890006 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff96c15dcc0 0x7ff96c0d1970 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.117+0000 7ff9719e0640 1 -- 172.21.15.38:0/3729890006 >> 172.21.15.38:0/3729890006 conn(0x7ff96c06bde0 msgr2=0x7ff96c06c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.117+0000 7ff9719e0640 1 -- 172.21.15.38:0/3729890006 shutdown_connections 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.117+0000 7ff9719e0640 1 -- 172.21.15.38:0/3729890006 wait complete. 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.118+0000 7ff9719e0640 1 Processor -- start 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.118+0000 7ff9719e0640 1 -- start start 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.119+0000 7ff9719e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff96c0d0720 0x7ff96c1f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.119+0000 7ff9719e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff96c0d1eb0 0x7ff96c1f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.119+0000 7ff9719e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff96c15dcc0 0x7ff96c1459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.119+0000 7ff9719e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0d3020 con 0x7ff96c0d0720 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.119+0000 7ff9719e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0d2d40 con 0x7ff96c15dcc0 2024-04-25T08:36:11.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.119+0000 7ff9719e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0b9720 con 0x7ff96c0d1eb0 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.119+0000 7ff96affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff96c0d0720 0x7ff96c1f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.119+0000 7ff96affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff96c0d0720 0x7ff96c1f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53842/0 (socket says 172.21.15.38:53842) 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.119+0000 7ff96affd640 1 -- 172.21.15.38:0/3154328042 learned_addr learned my addr 172.21.15.38:0/3154328042 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff96a7fc640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff96c0d1eb0 0x7ff96c1f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff96affd640 1 -- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff96c0d1eb0 msgr2=0x7ff96c1f6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff96affd640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff96c0d1eb0 0x7ff96c1f6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff96b7fe640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff96c15dcc0 0x7ff96c1459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff96affd640 1 -- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff96c15dcc0 msgr2=0x7ff96c1459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff96affd640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff96c15dcc0 0x7ff96c1459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff96affd640 1 -- 172.21.15.38:0/3154328042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c15ff90 con 0x7ff96c0d0720 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff96affd640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff96c0d0720 0x7ff96c1f5d40 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff9640097a0 tx=0x7ff9640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff963fff640 1 -- 172.21.15.38:0/3154328042 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964041070 con 0x7ff96c0d0720 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff963fff640 1 -- 172.21.15.38:0/3154328042 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96403c870 con 0x7ff96c0d0720 2024-04-25T08:36:11.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff963fff640 1 -- 172.21.15.38:0/3154328042 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964037040 con 0x7ff96c0d0720 2024-04-25T08:36:11.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff9719e0640 1 -- 172.21.15.38:0/3154328042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96c160270 con 0x7ff96c0d0720 2024-04-25T08:36:11.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.120+0000 7ff9719e0640 1 -- 172.21.15.38:0/3154328042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c160830 con 0x7ff96c0d0720 2024-04-25T08:36:11.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.121+0000 7ff9719e0640 1 -- 172.21.15.38:0/3154328042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96c04ef90 con 0x7ff96c0d0720 2024-04-25T08:36:11.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.121+0000 7ff963fff640 1 -- 172.21.15.38:0/3154328042 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff964044070 con 0x7ff96c0d0720 2024-04-25T08:36:11.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.122+0000 7ff963fff640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff94c06b4d0 0x7ff94c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.122+0000 7ff963fff640 1 -- 172.21.15.38:0/3154328042 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff9640ab1e0 con 0x7ff96c0d0720 2024-04-25T08:36:11.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.122+0000 7ff96a7fc640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff94c06b4d0 0x7ff94c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.123+0000 7ff96a7fc640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff94c06b4d0 0x7ff94c06d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0b93c0 tx=0x7ff954008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:11.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.126+0000 7ff963fff640 1 -- 172.21.15.38:0/3154328042 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff964079fa0 con 0x7ff96c0d0720 2024-04-25T08:36:11.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.325+0000 7ff9719e0640 1 -- 172.21.15.38:0/3154328042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff96c15cfe0 con 0x7ff96c0d0720 2024-04-25T08:36:11.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.326+0000 7ff963fff640 1 -- 172.21.15.38:0/3154328042 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v71) v1 ==== 80+0+120 (secure 0 0 0) 0x7ff9640798a0 con 0x7ff96c0d0720 2024-04-25T08:36:11.320 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/239365678 2024-04-26T08:35:52.345208+0000 2024-04-25T08:36:11.320 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6833/239365678 2024-04-26T08:35:52.345208+0000 2024-04-25T08:36:11.320 INFO:teuthology.orchestra.run.smithi038.stderr:listed 2 entries 2024-04-25T08:36:11.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.328+0000 7ff9719e0640 1 -- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff94c06b4d0 msgr2=0x7ff94c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.328+0000 7ff9719e0640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff94c06b4d0 0x7ff94c06d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0b93c0 tx=0x7ff954008040 comp rx=0 tx=0).stop 2024-04-25T08:36:11.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.328+0000 7ff9719e0640 1 -- 172.21.15.38:0/3154328042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff96c0d0720 msgr2=0x7ff96c1f5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.328+0000 7ff9719e0640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff96c0d0720 0x7ff96c1f5d40 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff9640097a0 tx=0x7ff9640046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.328+0000 7ff9719e0640 1 -- 172.21.15.38:0/3154328042 shutdown_connections 2024-04-25T08:36:11.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.328+0000 7ff9719e0640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff96c15dcc0 0x7ff96c1459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.328+0000 7ff9719e0640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff94c06b4d0 0x7ff94c06d990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.328+0000 7ff9719e0640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff96c0d1eb0 0x7ff96c1f6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.328+0000 7ff9719e0640 1 --2- 172.21.15.38:0/3154328042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff96c0d0720 0x7ff96c1f5d40 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.328+0000 7ff9719e0640 1 -- 172.21.15.38:0/3154328042 >> 172.21.15.38:0/3154328042 conn(0x7ff96c06bde0 msgr2=0x7ff96c15a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:11.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.328+0000 7ff9719e0640 1 -- 172.21.15.38:0/3154328042 shutdown_connections 2024-04-25T08:36:11.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.328+0000 7ff9719e0640 1 -- 172.21.15.38:0/3154328042 wait complete. 2024-04-25T08:36:11.333 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:36:11.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:11.333 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:11.356 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[67012]: fuse finished with error 0 and tester_r 0 2024-04-25T08:36:11.358 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:36:11.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:11.358 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:36:11.527 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:36:11.528 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:11.546 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:36:11.547 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:36:11.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:11.547 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:36:11.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:36:11.604 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:36:11.605 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:36:11.605 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:11.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:36:11.621 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:36:11.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:11.622 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:36:11.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:36:11.677 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:36:11.677 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:36:11.677 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:11.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:36:11.695 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:36:11.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:11.695 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:36:11.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:36:11.753 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:36:11.753 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:11.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.860+0000 7f034cabd640 1 Processor -- start 2024-04-25T08:36:11.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.860+0000 7f034cabd640 1 -- start start 2024-04-25T08:36:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.861+0000 7f034cabd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f03480b5710 0x7f0348161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.861+0000 7f034cabd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f034815ddf0 0x7f03480b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.861+0000 7f034cabd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0348161e00 con 0x7f03480b5af0 2024-04-25T08:36:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.861+0000 7f034cabd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03480b9930 con 0x7f03480b5710 2024-04-25T08:36:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.861+0000 7f034cabd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03481620e0 con 0x7f034815ddf0 2024-04-25T08:36:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.861+0000 7f034659c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f03480b5710 0x7f0348161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.861+0000 7f034659c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f03480b5710 0x7f0348161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53858/0 (socket says 172.21.15.38:53858) 2024-04-25T08:36:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.861+0000 7f034659c640 1 -- 172.21.15.38:0/426346850 learned_addr learned my addr 172.21.15.38:0/426346850 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.861+0000 7f0345d9b640 1 --2- 172.21.15.38:0/426346850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f034815ddf0 0x7f03480b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.862+0000 7f034659c640 1 -- 172.21.15.38:0/426346850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f034815ddf0 msgr2=0x7f03480b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.862+0000 7f034659c640 1 --2- 172.21.15.38:0/426346850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f034815ddf0 0x7f03480b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.862+0000 7f034659c640 1 -- 172.21.15.38:0/426346850 >> v1:172.21.15.107:6789/0 conn(0x7f03480b5af0 legacy=0x7f034806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.862+0000 7f034659c640 1 -- 172.21.15.38:0/426346850 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03481623c0 con 0x7f03480b5710 2024-04-25T08:36:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.862+0000 7f0345d9b640 1 --2- 172.21.15.38:0/426346850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f034815ddf0 0x7f03480b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.862+0000 7f034659c640 1 --2- 172.21.15.38:0/426346850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03480b5710 0x7f0348161700 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f0340009870 tx=0x7f034002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d97451a3650fe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.862+0000 7f034559a640 1 -- 172.21.15.38:0/426346850 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340037070 con 0x7f03480b5710 2024-04-25T08:36:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.863+0000 7f034559a640 1 -- 172.21.15.38:0/426346850 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0340032440 con 0x7f03480b5710 2024-04-25T08:36:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.863+0000 7f034559a640 1 -- 172.21.15.38:0/426346850 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0340032810 con 0x7f03480b5710 2024-04-25T08:36:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.863+0000 7f034cabd640 1 -- 172.21.15.38:0/426346850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03480b5710 msgr2=0x7f0348161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.863+0000 7f034cabd640 1 --2- 172.21.15.38:0/426346850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03480b5710 0x7f0348161700 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f0340009870 tx=0x7f034002a270 comp rx=0 tx=0).stop 2024-04-25T08:36:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.863+0000 7f034cabd640 1 -- 172.21.15.38:0/426346850 shutdown_connections 2024-04-25T08:36:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.863+0000 7f034cabd640 1 --2- 172.21.15.38:0/426346850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f034815ddf0 0x7f03480b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.863+0000 7f034cabd640 1 --2- 172.21.15.38:0/426346850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03480b5710 0x7f0348161700 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.863+0000 7f034cabd640 1 -- 172.21.15.38:0/426346850 >> 172.21.15.38:0/426346850 conn(0x7f03480d07c0 msgr2=0x7f03480d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.863+0000 7f034cabd640 1 -- 172.21.15.38:0/426346850 shutdown_connections 2024-04-25T08:36:11.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.863+0000 7f034cabd640 1 -- 172.21.15.38:0/426346850 wait complete. 2024-04-25T08:36:11.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.864+0000 7f034cabd640 1 Processor -- start 2024-04-25T08:36:11.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.864+0000 7f034cabd640 1 -- start start 2024-04-25T08:36:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.865+0000 7f034cabd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03480b5710 0x7f03481f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.865+0000 7f034cabd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03480b5af0 0x7f034814cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.865+0000 7f034cabd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f034815ddf0 0x7f03480b6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.865+0000 7f034cabd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03480d34d0 con 0x7f034815ddf0 2024-04-25T08:36:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.865+0000 7f034cabd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03480d3350 con 0x7f03480b5710 2024-04-25T08:36:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.865+0000 7f034cabd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03480d3650 con 0x7f03480b5af0 2024-04-25T08:36:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.865+0000 7f0346d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f034815ddf0 0x7f03480b6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.865+0000 7f0346d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f034815ddf0 0x7f03480b6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53872/0 (socket says 172.21.15.38:53872) 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.865+0000 7f0346d9d640 1 -- 172.21.15.38:0/441269626 learned_addr learned my addr 172.21.15.38:0/441269626 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.865+0000 7f0345d9b640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03480b5af0 0x7f034814cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.865+0000 7f034659c640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03480b5710 0x7f03481f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.866+0000 7f0346d9d640 1 -- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03480b5af0 msgr2=0x7f034814cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.866+0000 7f0346d9d640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03480b5af0 0x7f034814cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.866+0000 7f0346d9d640 1 -- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03480b5710 msgr2=0x7f03481f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.866+0000 7f0346d9d640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03480b5710 0x7f03481f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.866+0000 7f0346d9d640 1 -- 172.21.15.38:0/441269626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03481f7530 con 0x7f034815ddf0 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.866+0000 7f0346d9d640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f034815ddf0 0x7f03480b6df0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f033800c7d0 tx=0x7f033800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.866+0000 7f033effd640 1 -- 172.21.15.38:0/441269626 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338003b70 con 0x7f034815ddf0 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.866+0000 7f033effd640 1 -- 172.21.15.38:0/441269626 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0338003d30 con 0x7f034815ddf0 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.866+0000 7f034cabd640 1 -- 172.21.15.38:0/441269626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03481f77d0 con 0x7f034815ddf0 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.866+0000 7f033effd640 1 -- 172.21.15.38:0/441269626 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033800aa30 con 0x7f034815ddf0 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.866+0000 7f034cabd640 1 -- 172.21.15.38:0/441269626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03481f7d90 con 0x7f034815ddf0 2024-04-25T08:36:11.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.867+0000 7f034cabd640 1 -- 172.21.15.38:0/441269626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0348068e80 con 0x7f034815ddf0 2024-04-25T08:36:11.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.867+0000 7f033effd640 1 -- 172.21.15.38:0/441269626 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f033800abd0 con 0x7f034815ddf0 2024-04-25T08:36:11.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.868+0000 7f033effd640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f032806b500 0x7f032806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:11.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.868+0000 7f033effd640 1 -- 172.21.15.38:0/441269626 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f033808a2b0 con 0x7f034815ddf0 2024-04-25T08:36:11.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.868+0000 7f034659c640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f032806b500 0x7f032806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:11.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.869+0000 7f034659c640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f032806b500 0x7f032806d9c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0340034000 tx=0x7f034002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:11.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:11.872+0000 7f033effd640 1 -- 172.21.15.38:0/441269626 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0338058f50 con 0x7f034815ddf0 2024-04-25T08:36:12.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.092+0000 7f034cabd640 1 -- 172.21.15.38:0/441269626 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03481578a0 con 0x7f034815ddf0 2024-04-25T08:36:12.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.092+0000 7f033effd640 1 -- 172.21.15.38:0/441269626 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0338058850 con 0x7f034815ddf0 2024-04-25T08:36:12.087 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:12.087 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/1366115592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1366115592},{"type":"v1","addr":"172.21.15.107:6837","nonce":1366115592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6379,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/397891417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":397891417},{"type":"v1","addr":"172.21.15.107:6835","nonce":397891417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1355647168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1355647168},{"type":"v1","addr":"172.21.15.38:6835","nonce":1355647168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:40.623355+0000","modified":"2024-04-25T08:35:55.386447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7080":{"gid":7080,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/1952521723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1952521723},{"type":"v1","addr":"172.21.15.38:6833","nonce":1952521723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:36:12.087 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 42 2024-04-25T08:36:12.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.094+0000 7f034cabd640 1 -- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f032806b500 msgr2=0x7f032806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.094+0000 7f034cabd640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f032806b500 0x7f032806d9c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0340034000 tx=0x7f034002e000 comp rx=0 tx=0).stop 2024-04-25T08:36:12.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.094+0000 7f034cabd640 1 -- 172.21.15.38:0/441269626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f034815ddf0 msgr2=0x7f03480b6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.094+0000 7f034cabd640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f034815ddf0 0x7f03480b6df0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f033800c7d0 tx=0x7f033800cc90 comp rx=0 tx=0).stop 2024-04-25T08:36:12.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.094+0000 7f034cabd640 1 -- 172.21.15.38:0/441269626 shutdown_connections 2024-04-25T08:36:12.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.094+0000 7f034cabd640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f034815ddf0 0x7f03480b6df0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.094+0000 7f034cabd640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f032806b500 0x7f032806d9c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.094+0000 7f034cabd640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03480b5af0 0x7f034814cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.094+0000 7f034cabd640 1 --2- 172.21.15.38:0/441269626 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03480b5710 0x7f03481f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.094+0000 7f034cabd640 1 -- 172.21.15.38:0/441269626 >> 172.21.15.38:0/441269626 conn(0x7f03480d07c0 msgr2=0x7f03480d3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:12.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.095+0000 7f034cabd640 1 -- 172.21.15.38:0/441269626 shutdown_connections 2024-04-25T08:36:12.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.095+0000 7f034cabd640 1 -- 172.21.15.38:0/441269626 wait complete. 2024-04-25T08:36:12.100 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:12.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.207+0000 7f9c674f5640 1 Processor -- start 2024-04-25T08:36:12.202 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.207+0000 7f9c674f5640 1 -- start start 2024-04-25T08:36:12.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c674f5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9c6015ddf0 0x7f9c60161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c674f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9c600b93f0 0x7f9c600d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c674f5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c600b98a0 con 0x7f9c600b5710 2024-04-25T08:36:12.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c674f5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c60161e00 con 0x7f9c6015ddf0 2024-04-25T08:36:12.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c674f5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c601620e0 con 0x7f9c600b93f0 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c64a90640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9c6015ddf0 0x7f9c60161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c65291640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9c600b5710 0x7f9c600b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34878/0 (socket says 172.21.15.38:34878) 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c65291640 1 -- 172.21.15.38:0/1487117400 learned_addr learned my addr 172.21.15.38:0/1487117400 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c64a90640 1 --2- 172.21.15.38:0/1487117400 >> v2:172.21.15.38:3300/0 conn(0x7f9c6015ddf0 0x7f9c60161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53882/0 (socket says 172.21.15.38:53882) 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c64a90640 1 -- 172.21.15.38:0/1487117400 >> v2:172.21.15.107:3300/0 conn(0x7f9c600b93f0 msgr2=0x7f9c600d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c65a92640 1 --2- 172.21.15.38:0/1487117400 >> v2:172.21.15.107:3300/0 conn(0x7f9c600b93f0 0x7f9c600d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c64a90640 1 --2- 172.21.15.38:0/1487117400 >> v2:172.21.15.107:3300/0 conn(0x7f9c600b93f0 0x7f9c600d2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.208+0000 7f9c57fff640 1 -- 172.21.15.38:0/1487117400 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157216699 0 0) 0x7f9c600b98a0 con 0x7f9c600b5710 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.209+0000 7f9c64a90640 1 -- 172.21.15.38:0/1487117400 >> v1:172.21.15.38:6789/0 conn(0x7f9c600b5710 legacy=0x7f9c600b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.209+0000 7f9c64a90640 1 -- 172.21.15.38:0/1487117400 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c601623c0 con 0x7f9c6015ddf0 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.209+0000 7f9c64a90640 1 --2- 172.21.15.38:0/1487117400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c6015ddf0 0x7f9c60161700 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9c6008b240 tx=0x7f9c5002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc0420cb1601de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.209+0000 7f9c57fff640 1 -- 172.21.15.38:0/1487117400 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50037070 con 0x7f9c6015ddf0 2024-04-25T08:36:12.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.209+0000 7f9c57fff640 1 -- 172.21.15.38:0/1487117400 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c50032440 con 0x7f9c6015ddf0 2024-04-25T08:36:12.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.209+0000 7f9c57fff640 1 -- 172.21.15.38:0/1487117400 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50032810 con 0x7f9c6015ddf0 2024-04-25T08:36:12.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.210+0000 7f9c674f5640 1 -- 172.21.15.38:0/1487117400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c6015ddf0 msgr2=0x7f9c60161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.210+0000 7f9c674f5640 1 --2- 172.21.15.38:0/1487117400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c6015ddf0 0x7f9c60161700 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9c6008b240 tx=0x7f9c5002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.210+0000 7f9c674f5640 1 -- 172.21.15.38:0/1487117400 shutdown_connections 2024-04-25T08:36:12.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.210+0000 7f9c674f5640 1 --2- 172.21.15.38:0/1487117400 >> v2:172.21.15.107:3300/0 conn(0x7f9c600b93f0 0x7f9c600d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.210+0000 7f9c674f5640 1 --2- 172.21.15.38:0/1487117400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c6015ddf0 0x7f9c60161700 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.210+0000 7f9c674f5640 1 -- 172.21.15.38:0/1487117400 >> 172.21.15.38:0/1487117400 conn(0x7f9c600d07c0 msgr2=0x7f9c600d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:12.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.210+0000 7f9c674f5640 1 -- 172.21.15.38:0/1487117400 shutdown_connections 2024-04-25T08:36:12.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.210+0000 7f9c674f5640 1 -- 172.21.15.38:0/1487117400 wait complete. 2024-04-25T08:36:12.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.211+0000 7f9c674f5640 1 Processor -- start 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.211+0000 7f9c674f5640 1 -- start start 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c674f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c600b5710 0x7f9c601f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c674f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c600b93f0 0x7f9c600c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c674f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c6015ddf0 0x7f9c600ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c674f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c600d2c10 con 0x7f9c600b5710 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c674f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c600d2a90 con 0x7f9c6015ddf0 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c674f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c600d2d90 con 0x7f9c600b93f0 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c65291640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c600b5710 0x7f9c601f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c65291640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c600b5710 0x7f9c601f2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53884/0 (socket says 172.21.15.38:53884) 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c65291640 1 -- 172.21.15.38:0/1412248521 learned_addr learned my addr 172.21.15.38:0/1412248521 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c65291640 1 -- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c600b93f0 msgr2=0x7f9c600c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c64a90640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c600b93f0 0x7f9c600c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c65291640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c600b93f0 0x7f9c600c7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c65291640 1 -- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c6015ddf0 msgr2=0x7f9c600ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c65291640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c6015ddf0 0x7f9c600ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.212+0000 7f9c65291640 1 -- 172.21.15.38:0/1412248521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c600c8870 con 0x7f9c600b5710 2024-04-25T08:36:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.213+0000 7f9c65291640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c600b5710 0x7f9c601f2ef0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00c4d0 tx=0x7f9c5c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.213+0000 7f9c55ffb640 1 -- 172.21.15.38:0/1412248521 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c018070 con 0x7f9c600b5710 2024-04-25T08:36:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.213+0000 7f9c55ffb640 1 -- 172.21.15.38:0/1412248521 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c007500 con 0x7f9c600b5710 2024-04-25T08:36:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.213+0000 7f9c674f5640 1 -- 172.21.15.38:0/1412248521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c600c8b50 con 0x7f9c600b5710 2024-04-25T08:36:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.213+0000 7f9c55ffb640 1 -- 172.21.15.38:0/1412248521 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c004a20 con 0x7f9c600b5710 2024-04-25T08:36:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.213+0000 7f9c674f5640 1 -- 172.21.15.38:0/1412248521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c600cb320 con 0x7f9c600b5710 2024-04-25T08:36:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.214+0000 7f9c674f5640 1 -- 172.21.15.38:0/1412248521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c600cae60 con 0x7f9c600b5710 2024-04-25T08:36:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.214+0000 7f9c55ffb640 1 -- 172.21.15.38:0/1412248521 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c5c01b070 con 0x7f9c600b5710 2024-04-25T08:36:12.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.215+0000 7f9c55ffb640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9c4406b500 0x7f9c4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.215+0000 7f9c55ffb640 1 -- 172.21.15.38:0/1412248521 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9c5c089630 con 0x7f9c600b5710 2024-04-25T08:36:12.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.215+0000 7f9c64a90640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9c4406b500 0x7f9c4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.215+0000 7f9c64a90640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9c4406b500 0x7f9c4406d9c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9c50034000 tx=0x7f9c5002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:12.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.219+0000 7f9c55ffb640 1 -- 172.21.15.38:0/1412248521 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c600cae60 con 0x7f9c600b5710 2024-04-25T08:36:12.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.405+0000 7f9c674f5640 1 -- 172.21.15.38:0/1412248521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c60068e80 con 0x7f9c600b5710 2024-04-25T08:36:12.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.407+0000 7f9c55ffb640 1 -- 172.21.15.38:0/1412248521 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f9c5c058360 con 0x7f9c600b5710 2024-04-25T08:36:12.402 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:12.402 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":71,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:52.358128+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:38.884603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:39.590140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/239365678":"2024-04-26T08:35:52.345208+0000","172.21.15.38:6833/239365678":"2024-04-26T08:35:52.345208+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:12.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.409+0000 7f9c674f5640 1 -- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9c4406b500 msgr2=0x7f9c4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.409+0000 7f9c674f5640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9c4406b500 0x7f9c4406d9c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9c50034000 tx=0x7f9c5002e000 comp rx=0 tx=0).stop 2024-04-25T08:36:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.409+0000 7f9c674f5640 1 -- 172.21.15.38:0/1412248521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c600b5710 msgr2=0x7f9c601f2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.409+0000 7f9c674f5640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c600b5710 0x7f9c601f2ef0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c00c4d0 tx=0x7f9c5c00c990 comp rx=0 tx=0).stop 2024-04-25T08:36:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.409+0000 7f9c674f5640 1 -- 172.21.15.38:0/1412248521 shutdown_connections 2024-04-25T08:36:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.409+0000 7f9c674f5640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c6015ddf0 0x7f9c600ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.409+0000 7f9c674f5640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9c4406b500 0x7f9c4406d9c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.409+0000 7f9c674f5640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c600b93f0 0x7f9c600c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.409+0000 7f9c674f5640 1 --2- 172.21.15.38:0/1412248521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c600b5710 0x7f9c601f2ef0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.409+0000 7f9c674f5640 1 -- 172.21.15.38:0/1412248521 >> 172.21.15.38:0/1412248521 conn(0x7f9c600d07c0 msgr2=0x7f9c60161440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.409+0000 7f9c674f5640 1 -- 172.21.15.38:0/1412248521 shutdown_connections 2024-04-25T08:36:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.409+0000 7f9c674f5640 1 -- 172.21.15.38:0/1412248521 wait complete. 2024-04-25T08:36:12.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:35:38.884603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:35:39.590140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:36:12.415 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:36:12.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.524+0000 7f7fff3e0640 1 Processor -- start 2024-04-25T08:36:12.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.524+0000 7f7fff3e0640 1 -- start start 2024-04-25T08:36:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.525+0000 7f7fff3e0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7ff80d0320 0x7f7ff80d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.525+0000 7f7fff3e0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7ff80b6980 0x7f7ff80b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.525+0000 7f7fff3e0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff80d2060 con 0x7f7ff80d0700 2024-04-25T08:36:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.525+0000 7f7fff3e0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff80d2330 con 0x7f7ff80b6980 2024-04-25T08:36:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.525+0000 7f7fff3e0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff80d2610 con 0x7f7ff80d0320 2024-04-25T08:36:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.525+0000 7f7ffd97d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7ff80b6980 0x7f7ff80b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.525+0000 7f7ffc97b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7ff80d0700 0x7f7ff80d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34880/0 (socket says 172.21.15.38:34880) 2024-04-25T08:36:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.525+0000 7f7ffd97d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7ff80b6980 0x7f7ff80b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53886/0 (socket says 172.21.15.38:53886) 2024-04-25T08:36:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.525+0000 7f7ffc97b640 1 -- 172.21.15.38:0/1274325300 learned_addr learned my addr 172.21.15.38:0/1274325300 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.526+0000 7f7ffd17c640 1 --2- 172.21.15.38:0/1274325300 >> v2:172.21.15.107:3300/0 conn(0x7f7ff80d0320 0x7f7ff80d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.526+0000 7f7ffd97d640 1 -- 172.21.15.38:0/1274325300 >> v2:172.21.15.107:3300/0 conn(0x7f7ff80d0320 msgr2=0x7f7ff80d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.526+0000 7f7ffd97d640 1 --2- 172.21.15.38:0/1274325300 >> v2:172.21.15.107:3300/0 conn(0x7f7ff80d0320 0x7f7ff80d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.526+0000 7f7ffd97d640 1 -- 172.21.15.38:0/1274325300 >> v1:172.21.15.38:6789/0 conn(0x7f7ff80d0700 legacy=0x7f7ff80d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.526+0000 7f7ffd97d640 1 -- 172.21.15.38:0/1274325300 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80d28f0 con 0x7f7ff80b6980 2024-04-25T08:36:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.526+0000 7f7ff7fff640 1 -- 172.21.15.38:0/1274325300 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804882475 0 0) 0x7f7ff80d2060 con 0x7f7ff80d0700 2024-04-25T08:36:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.526+0000 7f7ffd17c640 1 --2- 172.21.15.38:0/1274325300 >> v2:172.21.15.107:3300/0 conn(0x7f7ff80d0320 0x7f7ff80d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.526+0000 7f7ffd97d640 1 --2- 172.21.15.38:0/1274325300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ff80b6980 0x7f7ff80b6dd0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8069100 tx=0x7f7fe802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b151c45e9921e1e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.527+0000 7f7ff7fff640 1 -- 172.21.15.38:0/1274325300 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8007740 con 0x7f7ff80b6980 2024-04-25T08:36:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.527+0000 7f7ff7fff640 1 -- 172.21.15.38:0/1274325300 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe80078e0 con 0x7f7ff80b6980 2024-04-25T08:36:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.527+0000 7f7ff7fff640 1 -- 172.21.15.38:0/1274325300 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8007cb0 con 0x7f7ff80b6980 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.527+0000 7f7fff3e0640 1 -- 172.21.15.38:0/1274325300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ff80b6980 msgr2=0x7f7ff80b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.527+0000 7f7fff3e0640 1 --2- 172.21.15.38:0/1274325300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ff80b6980 0x7f7ff80b6dd0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8069100 tx=0x7f7fe802a710 comp rx=0 tx=0).stop 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.527+0000 7f7fff3e0640 1 -- 172.21.15.38:0/1274325300 shutdown_connections 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.527+0000 7f7fff3e0640 1 --2- 172.21.15.38:0/1274325300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ff80b6980 0x7f7ff80b6dd0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.527+0000 7f7fff3e0640 1 --2- 172.21.15.38:0/1274325300 >> v2:172.21.15.107:3300/0 conn(0x7f7ff80d0320 0x7f7ff80d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.527+0000 7f7fff3e0640 1 -- 172.21.15.38:0/1274325300 >> 172.21.15.38:0/1274325300 conn(0x7f7ff80668f0 msgr2=0x7f7ff8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.527+0000 7f7fff3e0640 1 -- 172.21.15.38:0/1274325300 shutdown_connections 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.527+0000 7f7fff3e0640 1 -- 172.21.15.38:0/1274325300 wait complete. 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.528+0000 7f7fff3e0640 1 Processor -- start 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.528+0000 7f7fff3e0640 1 -- start start 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.529+0000 7f7fff3e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ff80b6980 0x7f7ff81e7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.529+0000 7f7fff3e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ff80d0320 0x7f7ff80b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.529+0000 7f7fff3e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ff80d0700 0x7f7ff80b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.529+0000 7f7fff3e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff80d0b40 con 0x7f7ff80d0700 2024-04-25T08:36:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.529+0000 7f7fff3e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff80d2eb0 con 0x7f7ff80b6980 2024-04-25T08:36:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.529+0000 7f7fff3e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff81e8590 con 0x7f7ff80d0320 2024-04-25T08:36:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.529+0000 7f7ffd97d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ff80d0700 0x7f7ff80b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.529+0000 7f7ffd97d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ff80d0700 0x7f7ff80b5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53892/0 (socket says 172.21.15.38:53892) 2024-04-25T08:36:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.529+0000 7f7ffd97d640 1 -- 172.21.15.38:0/2335033020 learned_addr learned my addr 172.21.15.38:0/2335033020 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7ffc97b640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ff80d0320 0x7f7ff80b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7ffd17c640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ff80b6980 0x7f7ff81e7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7ffd97d640 1 -- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ff80d0320 msgr2=0x7f7ff80b5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7ffd97d640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ff80d0320 0x7f7ff80b5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7ffd97d640 1 -- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ff80b6980 msgr2=0x7f7ff81e7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7ffd97d640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ff80b6980 0x7f7ff81e7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7ffd97d640 1 -- 172.21.15.38:0/2335033020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff81e8870 con 0x7f7ff80d0700 2024-04-25T08:36:12.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7ffd17c640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ff80b6980 0x7f7ff81e7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:12.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7ffd97d640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ff80d0700 0x7f7ff80b5980 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009e80 tx=0x7f7fe8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:12.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7ff5ffb640 1 -- 172.21.15.38:0/2335033020 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8040070 con 0x7f7ff80d0700 2024-04-25T08:36:12.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7ff5ffb640 1 -- 172.21.15.38:0/2335033020 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe8031c40 con 0x7f7ff80d0700 2024-04-25T08:36:12.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7fff3e0640 1 -- 172.21.15.38:0/2335033020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff81e8b50 con 0x7f7ff80d0700 2024-04-25T08:36:12.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7ff5ffb640 1 -- 172.21.15.38:0/2335033020 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8003b90 con 0x7f7ff80d0700 2024-04-25T08:36:12.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.530+0000 7f7fff3e0640 1 -- 172.21.15.38:0/2335033020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff81e81a0 con 0x7f7ff80d0700 2024-04-25T08:36:12.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.531+0000 7f7fff3e0640 1 -- 172.21.15.38:0/2335033020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff8068e80 con 0x7f7ff80d0700 2024-04-25T08:36:12.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.532+0000 7f7ff5ffb640 1 -- 172.21.15.38:0/2335033020 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fe8039040 con 0x7f7ff80d0700 2024-04-25T08:36:12.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.532+0000 7f7ff5ffb640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7fe406b4d0 0x7f7fe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.532+0000 7f7ff5ffb640 1 -- 172.21.15.38:0/2335033020 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7fe80ab6a0 con 0x7f7ff80d0700 2024-04-25T08:36:12.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.532+0000 7f7ffd17c640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7fe406b4d0 0x7f7fe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.533+0000 7f7ffd17c640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7fe406b4d0 0x7f7fe406d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0006fd0 tx=0x7f7ff0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:12.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.536+0000 7f7ff5ffb640 1 -- 172.21.15.38:0/2335033020 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7fe807a340 con 0x7f7ff80d0700 2024-04-25T08:36:12.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.716+0000 7f7fff3e0640 1 -- 172.21.15.38:0/2335033020 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ff80cd860 con 0x7f7ff80d0700 2024-04-25T08:36:12.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.716+0000 7f7ff5ffb640 1 -- 172.21.15.38:0/2335033020 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fe8079c40 con 0x7f7ff80d0700 2024-04-25T08:36:12.711 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:36:12.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.718+0000 7f7fff3e0640 1 -- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7fe406b4d0 msgr2=0x7f7fe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.718+0000 7f7fff3e0640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7fe406b4d0 0x7f7fe406d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0006fd0 tx=0x7f7ff0008040 comp rx=0 tx=0).stop 2024-04-25T08:36:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.718+0000 7f7fff3e0640 1 -- 172.21.15.38:0/2335033020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ff80d0700 msgr2=0x7f7ff80b5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.718+0000 7f7fff3e0640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ff80d0700 0x7f7ff80b5980 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8009e80 tx=0x7f7fe8004a40 comp rx=0 tx=0).stop 2024-04-25T08:36:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.718+0000 7f7fff3e0640 1 -- 172.21.15.38:0/2335033020 shutdown_connections 2024-04-25T08:36:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.718+0000 7f7fff3e0640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ff80d0700 0x7f7ff80b5980 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.718+0000 7f7fff3e0640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7fe406b4d0 0x7f7fe406d990 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.718+0000 7f7fff3e0640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ff80d0320 0x7f7ff80b5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.718+0000 7f7fff3e0640 1 --2- 172.21.15.38:0/2335033020 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ff80b6980 0x7f7ff81e7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.719+0000 7f7fff3e0640 1 -- 172.21.15.38:0/2335033020 >> 172.21.15.38:0/2335033020 conn(0x7f7ff80668f0 msgr2=0x7f7ff815f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.719+0000 7f7fff3e0640 1 -- 172.21.15.38:0/2335033020 shutdown_connections 2024-04-25T08:36:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.719+0000 7f7fff3e0640 1 -- 172.21.15.38:0/2335033020 wait complete. 2024-04-25T08:36:12.724 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:36:12.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.833+0000 7f7a00353640 1 Processor -- start 2024-04-25T08:36:12.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.833+0000 7f7a00353640 1 -- start start 2024-04-25T08:36:12.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.834+0000 7f7a00353640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f79f815db80 0x7f79f80cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.834+0000 7f7a00353640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f79f80cfb70 0x7f79f80ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.834+0000 7f7a00353640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f80ac430 con 0x7f79f80ccfc0 2024-04-25T08:36:12.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.834+0000 7f7a00353640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f80cffe0 con 0x7f79f80cfb70 2024-04-25T08:36:12.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.834+0000 7f7a00353640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f80ac1e0 con 0x7f79f815db80 2024-04-25T08:36:12.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.834+0000 7f79fe8f0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f79f80cfb70 0x7f79f80ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.834+0000 7f79fe8f0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f79f80cfb70 0x7f79f80ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53904/0 (socket says 172.21.15.38:53904) 2024-04-25T08:36:12.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.834+0000 7f79fe8f0640 1 -- 172.21.15.38:0/3052252667 learned_addr learned my addr 172.21.15.38:0/3052252667 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:12.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.834+0000 7f79fe0ef640 1 --2- 172.21.15.38:0/3052252667 >> v2:172.21.15.107:3300/0 conn(0x7f79f815db80 0x7f79f80cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.835+0000 7f79fe8f0640 1 -- 172.21.15.38:0/3052252667 >> v2:172.21.15.107:3300/0 conn(0x7f79f815db80 msgr2=0x7f79f80cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:12.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.835+0000 7f79fe8f0640 1 --2- 172.21.15.38:0/3052252667 >> v2:172.21.15.107:3300/0 conn(0x7f79f815db80 0x7f79f80cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.835+0000 7f79fe8f0640 1 -- 172.21.15.38:0/3052252667 >> v1:172.21.15.107:6789/0 conn(0x7f79f80ccfc0 legacy=0x7f79f80c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:12.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.835+0000 7f79fe8f0640 1 -- 172.21.15.38:0/3052252667 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f80ac710 con 0x7f79f80cfb70 2024-04-25T08:36:12.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.835+0000 7f79fe0ef640 1 --2- 172.21.15.38:0/3052252667 >> v2:172.21.15.107:3300/0 conn(0x7f79f815db80 0x7f79f80cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:12.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.835+0000 7f79fe8f0640 1 --2- 172.21.15.38:0/3052252667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79f80cfb70 0x7f79f80ca1e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f79f806bbc0 tx=0x7f79f002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d1f1b087609b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:12.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.835+0000 7f79fd0ed640 1 -- 172.21.15.38:0/3052252667 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0037070 con 0x7f79f80cfb70 2024-04-25T08:36:12.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.835+0000 7f79fd0ed640 1 -- 172.21.15.38:0/3052252667 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f0032440 con 0x7f79f80cfb70 2024-04-25T08:36:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.835+0000 7f79fd0ed640 1 -- 172.21.15.38:0/3052252667 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f0032810 con 0x7f79f80cfb70 2024-04-25T08:36:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.836+0000 7f7a00353640 1 -- 172.21.15.38:0/3052252667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79f80cfb70 msgr2=0x7f79f80ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.836+0000 7f7a00353640 1 --2- 172.21.15.38:0/3052252667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79f80cfb70 0x7f79f80ca1e0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f79f806bbc0 tx=0x7f79f002a230 comp rx=0 tx=0).stop 2024-04-25T08:36:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.836+0000 7f7a00353640 1 -- 172.21.15.38:0/3052252667 shutdown_connections 2024-04-25T08:36:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.836+0000 7f7a00353640 1 --2- 172.21.15.38:0/3052252667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79f80cfb70 0x7f79f80ca1e0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.836+0000 7f7a00353640 1 --2- 172.21.15.38:0/3052252667 >> v2:172.21.15.107:3300/0 conn(0x7f79f815db80 0x7f79f80cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.836+0000 7f7a00353640 1 -- 172.21.15.38:0/3052252667 >> 172.21.15.38:0/3052252667 conn(0x7f79f806adc0 msgr2=0x7f79f806b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.836+0000 7f7a00353640 1 -- 172.21.15.38:0/3052252667 shutdown_connections 2024-04-25T08:36:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.836+0000 7f7a00353640 1 -- 172.21.15.38:0/3052252667 wait complete. 2024-04-25T08:36:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.837+0000 7f7a00353640 1 Processor -- start 2024-04-25T08:36:12.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.837+0000 7f7a00353640 1 -- start start 2024-04-25T08:36:12.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f7a00353640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79f80ccfc0 0x7f79f8156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f7a00353640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79f80cfb70 0x7f79f8157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f7a00353640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f815db80 0x7f79f80cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f7a00353640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f815df60 con 0x7f79f80ccfc0 2024-04-25T08:36:12.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f7a00353640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f80accd0 con 0x7f79f815db80 2024-04-25T08:36:12.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f7a00353640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f815e2c0 con 0x7f79f80cfb70 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f79fe0ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79f80ccfc0 0x7f79f8156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f79fe0ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79f80ccfc0 0x7f79f8156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53910/0 (socket says 172.21.15.38:53910) 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f79fe0ef640 1 -- 172.21.15.38:0/1293576495 learned_addr learned my addr 172.21.15.38:0/1293576495 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f79fe8f0640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f815db80 0x7f79f80cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f79fd8ee640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79f80cfb70 0x7f79f8157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f79fe0ef640 1 -- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79f80cfb70 msgr2=0x7f79f8157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f79fe0ef640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79f80cfb70 0x7f79f8157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f79fe0ef640 1 -- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f815db80 msgr2=0x7f79f80cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f79fe0ef640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f815db80 0x7f79f80cbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.838+0000 7f79fe0ef640 1 -- 172.21.15.38:0/1293576495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f815aca0 con 0x7f79f80ccfc0 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.839+0000 7f79fe0ef640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79f80ccfc0 0x7f79f8156280 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f79e800ce00 tx=0x7f79e800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.839+0000 7f79f6ffd640 1 -- 172.21.15.38:0/1293576495 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e800f890 con 0x7f79f80ccfc0 2024-04-25T08:36:12.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.839+0000 7f79f6ffd640 1 -- 172.21.15.38:0/1293576495 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e800fa30 con 0x7f79f80ccfc0 2024-04-25T08:36:12.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.839+0000 7f79f6ffd640 1 -- 172.21.15.38:0/1293576495 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e800fe40 con 0x7f79f80ccfc0 2024-04-25T08:36:12.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.839+0000 7f7a00353640 1 -- 172.21.15.38:0/1293576495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f80c6710 con 0x7f79f80ccfc0 2024-04-25T08:36:12.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.839+0000 7f7a00353640 1 -- 172.21.15.38:0/1293576495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f80c6cd0 con 0x7f79f80ccfc0 2024-04-25T08:36:12.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.840+0000 7f7a00353640 1 -- 172.21.15.38:0/1293576495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f80cb570 con 0x7f79f80ccfc0 2024-04-25T08:36:12.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.840+0000 7f79f6ffd640 1 -- 172.21.15.38:0/1293576495 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79e8018400 con 0x7f79f80ccfc0 2024-04-25T08:36:12.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.841+0000 7f79f6ffd640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f79e406b4d0 0x7f79e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:12.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.841+0000 7f79f6ffd640 1 -- 172.21.15.38:0/1293576495 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f79e808b180 con 0x7f79f80ccfc0 2024-04-25T08:36:12.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.841+0000 7f79fd8ee640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f79e406b4d0 0x7f79e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:12.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.842+0000 7f79fd8ee640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f79e406b4d0 0x7f79e406d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f79f80c9530 tx=0x7f79ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:12.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:12.845+0000 7f79f6ffd640 1 -- 172.21.15.38:0/1293576495 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79f80cb570 con 0x7f79f80ccfc0 2024-04-25T08:36:13.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.026+0000 7f7a00353640 1 -- 172.21.15.38:0/1293576495 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f79f8068e80 con 0x7f79f80ccfc0 2024-04-25T08:36:13.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.026+0000 7f79f6ffd640 1 -- 172.21.15.38:0/1293576495 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f79f815aca0 con 0x7f79f80ccfc0 2024-04-25T08:36:13.021 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:36:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.028+0000 7f7a00353640 1 -- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f79e406b4d0 msgr2=0x7f79e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.028+0000 7f7a00353640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f79e406b4d0 0x7f79e406d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f79f80c9530 tx=0x7f79ec009040 comp rx=0 tx=0).stop 2024-04-25T08:36:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.028+0000 7f7a00353640 1 -- 172.21.15.38:0/1293576495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79f80ccfc0 msgr2=0x7f79f8156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.028+0000 7f7a00353640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79f80ccfc0 0x7f79f8156280 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f79e800ce00 tx=0x7f79e800a650 comp rx=0 tx=0).stop 2024-04-25T08:36:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.028+0000 7f7a00353640 1 -- 172.21.15.38:0/1293576495 shutdown_connections 2024-04-25T08:36:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.028+0000 7f7a00353640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f79f815db80 0x7f79f80cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.028+0000 7f7a00353640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f79e406b4d0 0x7f79e406d990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.028+0000 7f7a00353640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79f80cfb70 0x7f79f8157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.028+0000 7f7a00353640 1 --2- 172.21.15.38:0/1293576495 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79f80ccfc0 0x7f79f8156280 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.028+0000 7f7a00353640 1 -- 172.21.15.38:0/1293576495 >> 172.21.15.38:0/1293576495 conn(0x7f79f806adc0 msgr2=0x7f79f8157360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.028+0000 7f7a00353640 1 -- 172.21.15.38:0/1293576495 shutdown_connections 2024-04-25T08:36:13.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.029+0000 7f7a00353640 1 -- 172.21.15.38:0/1293576495 wait complete. 2024-04-25T08:36:13.034 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:36:13.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.143+0000 7fab1d309640 1 Processor -- start 2024-04-25T08:36:13.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.143+0000 7fab1d309640 1 -- start start 2024-04-25T08:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.144+0000 7fab1d309640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fab180d1e00 0x7fab180d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.144+0000 7fab1d309640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fab1815de60 0x7fab180d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.144+0000 7fab1d309640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab180d2720 con 0x7fab180cf7b0 2024-04-25T08:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.144+0000 7fab1d309640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab180ce4e0 con 0x7fab1815de60 2024-04-25T08:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.144+0000 7fab1d309640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab180cf320 con 0x7fab180d1e00 2024-04-25T08:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.144+0000 7fab167fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fab1815de60 0x7fab180d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.144+0000 7fab177fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fab180cf7b0 0x7fab18161770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34882/0 (socket says 172.21.15.38:34882) 2024-04-25T08:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.144+0000 7fab177fe640 1 -- 172.21.15.38:0/3788924994 learned_addr learned my addr 172.21.15.38:0/3788924994 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.144+0000 7fab16ffd640 1 --2- 172.21.15.38:0/3788924994 >> v2:172.21.15.107:3300/0 conn(0x7fab180d1e00 0x7fab180d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.145+0000 7fab167fc640 1 -- 172.21.15.38:0/3788924994 >> v2:172.21.15.107:3300/0 conn(0x7fab180d1e00 msgr2=0x7fab180d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.145+0000 7fab167fc640 1 --2- 172.21.15.38:0/3788924994 >> v2:172.21.15.107:3300/0 conn(0x7fab180d1e00 0x7fab180d42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.145+0000 7fab167fc640 1 -- 172.21.15.38:0/3788924994 >> v1:172.21.15.38:6789/0 conn(0x7fab180cf7b0 legacy=0x7fab18161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.145+0000 7fab15ffb640 1 -- 172.21.15.38:0/3788924994 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876035019 0 0) 0x7fab180d2720 con 0x7fab180cf7b0 2024-04-25T08:36:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.145+0000 7fab167fc640 1 -- 172.21.15.38:0/3788924994 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180ce710 con 0x7fab1815de60 2024-04-25T08:36:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.145+0000 7fab167fc640 1 --2- 172.21.15.38:0/3788924994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab1815de60 0x7fab180d21e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fab180d3780 tx=0x7fab0002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c8a2486f0a66c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.146+0000 7fab15ffb640 1 -- 172.21.15.38:0/3788924994 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00037070 con 0x7fab1815de60 2024-04-25T08:36:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.146+0000 7fab15ffb640 1 -- 172.21.15.38:0/3788924994 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab00032440 con 0x7fab1815de60 2024-04-25T08:36:13.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.146+0000 7fab15ffb640 1 -- 172.21.15.38:0/3788924994 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00032810 con 0x7fab1815de60 2024-04-25T08:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.146+0000 7fab1d309640 1 -- 172.21.15.38:0/3788924994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab1815de60 msgr2=0x7fab180d21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.146+0000 7fab1d309640 1 --2- 172.21.15.38:0/3788924994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab1815de60 0x7fab180d21e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fab180d3780 tx=0x7fab0002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.146+0000 7fab1d309640 1 -- 172.21.15.38:0/3788924994 shutdown_connections 2024-04-25T08:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.146+0000 7fab1d309640 1 --2- 172.21.15.38:0/3788924994 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab1815de60 0x7fab180d21e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.146+0000 7fab1d309640 1 --2- 172.21.15.38:0/3788924994 >> v2:172.21.15.107:3300/0 conn(0x7fab180d1e00 0x7fab180d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.146+0000 7fab1d309640 1 -- 172.21.15.38:0/3788924994 >> 172.21.15.38:0/3788924994 conn(0x7fab1808b000 msgr2=0x7fab180d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.146+0000 7fab1d309640 1 -- 172.21.15.38:0/3788924994 shutdown_connections 2024-04-25T08:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.146+0000 7fab1d309640 1 -- 172.21.15.38:0/3788924994 wait complete. 2024-04-25T08:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.148+0000 7fab1d309640 1 Processor -- start 2024-04-25T08:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.148+0000 7fab1d309640 1 -- start start 2024-04-25T08:36:13.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.148+0000 7fab1d309640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab180cf7b0 0x7fab18160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.148+0000 7fab1d309640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab180d1e00 0x7fab18152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.148+0000 7fab1d309640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1815de60 0x7fab1815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.148+0000 7fab1d309640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab180cefb0 con 0x7fab180d1e00 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.148+0000 7fab1d309640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab180cecd0 con 0x7fab1815de60 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.148+0000 7fab1d309640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab180d1600 con 0x7fab180cf7b0 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.148+0000 7fab167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab180d1e00 0x7fab18152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.148+0000 7fab167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab180d1e00 0x7fab18152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53936/0 (socket says 172.21.15.38:53936) 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.148+0000 7fab167fc640 1 -- 172.21.15.38:0/1663224339 learned_addr learned my addr 172.21.15.38:0/1663224339 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.149+0000 7fab167fc640 1 -- 172.21.15.38:0/1663224339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab180cf7b0 msgr2=0x7fab18160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.149+0000 7fab167fc640 1 --2- 172.21.15.38:0/1663224339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab180cf7b0 0x7fab18160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.149+0000 7fab167fc640 1 -- 172.21.15.38:0/1663224339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1815de60 msgr2=0x7fab1815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.149+0000 7fab167fc640 1 --2- 172.21.15.38:0/1663224339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1815de60 0x7fab1815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.149+0000 7fab167fc640 1 -- 172.21.15.38:0/1663224339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180b8fe0 con 0x7fab180d1e00 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.149+0000 7fab167fc640 1 --2- 172.21.15.38:0/1663224339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab180d1e00 0x7fab18152d40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fab00009890 tx=0x7fab000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.149+0000 7fab0f7fe640 1 -- 172.21.15.38:0/1663224339 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00037040 con 0x7fab180d1e00 2024-04-25T08:36:13.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.149+0000 7fab0f7fe640 1 -- 172.21.15.38:0/1663224339 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab00042070 con 0x7fab180d1e00 2024-04-25T08:36:13.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.149+0000 7fab0f7fe640 1 -- 172.21.15.38:0/1663224339 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00030920 con 0x7fab180d1e00 2024-04-25T08:36:13.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.149+0000 7fab1d309640 1 -- 172.21.15.38:0/1663224339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab180b92c0 con 0x7fab180d1e00 2024-04-25T08:36:13.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.149+0000 7fab1d309640 1 -- 172.21.15.38:0/1663224339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab18160020 con 0x7fab180d1e00 2024-04-25T08:36:13.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.150+0000 7fab1d309640 1 -- 172.21.15.38:0/1663224339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab18068e80 con 0x7fab180d1e00 2024-04-25T08:36:13.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.150+0000 7fab0f7fe640 1 -- 172.21.15.38:0/1663224339 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab00030ac0 con 0x7fab180d1e00 2024-04-25T08:36:13.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.151+0000 7fab0f7fe640 1 --2- 172.21.15.38:0/1663224339 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faaf806b500 0x7faaf806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.151+0000 7fab0f7fe640 1 -- 172.21.15.38:0/1663224339 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fab000aac70 con 0x7fab180d1e00 2024-04-25T08:36:13.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.151+0000 7fab16ffd640 1 --2- 172.21.15.38:0/1663224339 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faaf806b500 0x7faaf806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.152+0000 7fab16ffd640 1 --2- 172.21.15.38:0/1663224339 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faaf806b500 0x7faaf806d9c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fab10006a90 tx=0x7fab10006a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.155+0000 7fab0f7fe640 1 -- 172.21.15.38:0/1663224339 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab000793e0 con 0x7fab180d1e00 2024-04-25T08:36:13.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.338+0000 7fab1d309640 1 -- 172.21.15.38:0/1663224339 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab180d0f10 con 0x7fab180d1e00 2024-04-25T08:36:13.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.338+0000 7fab0f7fe640 1 -- 172.21.15.38:0/1663224339 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab0007d480 con 0x7fab180d1e00 2024-04-25T08:36:13.332 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:36:13.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.340+0000 7fab1d309640 1 -- 172.21.15.38:0/1663224339 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faaf806b500 msgr2=0x7faaf806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.340+0000 7fab1d309640 1 --2- 172.21.15.38:0/1663224339 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faaf806b500 0x7faaf806d9c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fab10006a90 tx=0x7fab10006a20 comp rx=0 tx=0).stop 2024-04-25T08:36:13.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.340+0000 7fab1d309640 1 -- 172.21.15.38:0/1663224339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab180d1e00 msgr2=0x7fab18152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.340+0000 7fab1d309640 1 --2- 172.21.15.38:0/1663224339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab180d1e00 0x7fab18152d40 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fab00009890 tx=0x7fab000046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.340+0000 7fab1d309640 1 -- 172.21.15.38:0/1663224339 shutdown_connections 2024-04-25T08:36:13.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.340+0000 7fab1d309640 1 --2- 172.21.15.38:0/1663224339 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab1815de60 0x7fab1815ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.340+0000 7fab1d309640 1 --2- 172.21.15.38:0/1663224339 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab180d1e00 0x7fab18152d40 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.340+0000 7fab1d309640 1 --2- 172.21.15.38:0/1663224339 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faaf806b500 0x7faaf806d9c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.340+0000 7fab1d309640 1 --2- 172.21.15.38:0/1663224339 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab180cf7b0 0x7fab18160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.340+0000 7fab1d309640 1 -- 172.21.15.38:0/1663224339 >> 172.21.15.38:0/1663224339 conn(0x7fab1808b000 msgr2=0x7fab180b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:13.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.340+0000 7fab1d309640 1 -- 172.21.15.38:0/1663224339 shutdown_connections 2024-04-25T08:36:13.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.340+0000 7fab1d309640 1 -- 172.21.15.38:0/1663224339 wait complete. 2024-04-25T08:36:13.345 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:13.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.452+0000 7f1089a1c640 1 Processor -- start 2024-04-25T08:36:13.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.453+0000 7f1089a1c640 1 -- start start 2024-04-25T08:36:13.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.453+0000 7f1089a1c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f10840d0f60 0x7f10840b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.453+0000 7f1089a1c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10840d3e70 con 0x7f10840d3900 2024-04-25T08:36:13.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.453+0000 7f1089a1c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10840d4150 con 0x7f108415dc10 2024-04-25T08:36:13.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.453+0000 7f1089a1c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f108415dff0 con 0x7f10840d0f60 2024-04-25T08:36:13.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.453+0000 7f10827fc640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f10840d3900 0x7f10840d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34886/0 (socket says 172.21.15.38:34886) 2024-04-25T08:36:13.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.453+0000 7f10827fc640 1 -- 172.21.15.38:0/2565992650 learned_addr learned my addr 172.21.15.38:0/2565992650 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:13.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.454+0000 7f10837fe640 1 --2- 172.21.15.38:0/2565992650 >> v2:172.21.15.107:3300/0 conn(0x7f10840d0f60 0x7f10840b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.454+0000 7f10837fe640 1 -- 172.21.15.38:0/2565992650 >> v1:172.21.15.107:6789/0 conn(0x7f108415dc10 legacy=0x7f10840ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.454+0000 7f10837fe640 1 -- 172.21.15.38:0/2565992650 >> v1:172.21.15.38:6789/0 conn(0x7f10840d3900 legacy=0x7f10840d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.454+0000 7f10837fe640 1 -- 172.21.15.38:0/2565992650 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10840d4430 con 0x7f10840d0f60 2024-04-25T08:36:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.455+0000 7f10837fe640 1 --2- 172.21.15.38:0/2565992650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10840d0f60 0x7f10840b2370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f107c0097a0 tx=0x7f107c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f1d241127e32c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.455+0000 7f1081ffb640 1 -- 172.21.15.38:0/2565992650 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c037070 con 0x7f10840d0f60 2024-04-25T08:36:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.455+0000 7f1081ffb640 1 -- 172.21.15.38:0/2565992650 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107c032440 con 0x7f10840d0f60 2024-04-25T08:36:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.455+0000 7f1081ffb640 1 -- 172.21.15.38:0/2565992650 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107c032810 con 0x7f10840d0f60 2024-04-25T08:36:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.456+0000 7f1089a1c640 1 -- 172.21.15.38:0/2565992650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10840d0f60 msgr2=0x7f10840b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.456+0000 7f1089a1c640 1 --2- 172.21.15.38:0/2565992650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10840d0f60 0x7f10840b2370 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f107c0097a0 tx=0x7f107c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.456+0000 7f1089a1c640 1 -- 172.21.15.38:0/2565992650 shutdown_connections 2024-04-25T08:36:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.456+0000 7f1089a1c640 1 --2- 172.21.15.38:0/2565992650 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10840d0f60 0x7f10840b2370 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.456+0000 7f1089a1c640 1 -- 172.21.15.38:0/2565992650 >> 172.21.15.38:0/2565992650 conn(0x7f1084066d10 msgr2=0x7f1084067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.456+0000 7f1089a1c640 1 -- 172.21.15.38:0/2565992650 shutdown_connections 2024-04-25T08:36:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.456+0000 7f1089a1c640 1 -- 172.21.15.38:0/2565992650 wait complete. 2024-04-25T08:36:13.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.457+0000 7f1089a1c640 1 Processor -- start 2024-04-25T08:36:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.457+0000 7f1089a1c640 1 -- start start 2024-04-25T08:36:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1089a1c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10840d0f60 0x7f108415b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1089a1c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10840d3900 0x7f10840c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1089a1c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f108415dc10 0x7f108415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1089a1c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10840d4710 con 0x7f10840d0f60 2024-04-25T08:36:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1089a1c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10840cef30 con 0x7f108415dc10 2024-04-25T08:36:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1089a1c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10840b80f0 con 0x7f10840d3900 2024-04-25T08:36:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1082ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10840d0f60 0x7f108415b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1082ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10840d0f60 0x7f108415b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53946/0 (socket says 172.21.15.38:53946) 2024-04-25T08:36:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1082ffd640 1 -- 172.21.15.38:0/70349135 learned_addr learned my addr 172.21.15.38:0/70349135 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f10827fc640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10840d3900 0x7f10840c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f10837fe640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f108415dc10 0x7f108415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1082ffd640 1 -- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10840d3900 msgr2=0x7f10840c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1082ffd640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10840d3900 0x7f10840c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1082ffd640 1 -- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f108415dc10 msgr2=0x7f108415b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1082ffd640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f108415dc10 0x7f108415b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1082ffd640 1 -- 172.21.15.38:0/70349135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10840c7dd0 con 0x7f10840d0f60 2024-04-25T08:36:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.458+0000 7f1082ffd640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10840d0f60 0x7f108415b930 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f107400c730 tx=0x7f107400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.459+0000 7f107bfff640 1 -- 172.21.15.38:0/70349135 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1074003b70 con 0x7f10840d0f60 2024-04-25T08:36:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.459+0000 7f107bfff640 1 -- 172.21.15.38:0/70349135 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1074003d30 con 0x7f10840d0f60 2024-04-25T08:36:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.459+0000 7f107bfff640 1 -- 172.21.15.38:0/70349135 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107400aa90 con 0x7f10840d0f60 2024-04-25T08:36:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.459+0000 7f1089a1c640 1 -- 172.21.15.38:0/70349135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10840c8050 con 0x7f10840d0f60 2024-04-25T08:36:13.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.459+0000 7f1089a1c640 1 -- 172.21.15.38:0/70349135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10840c8610 con 0x7f10840d0f60 2024-04-25T08:36:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.460+0000 7f1089a1c640 1 -- 172.21.15.38:0/70349135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1084068e80 con 0x7f10840d0f60 2024-04-25T08:36:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.460+0000 7f107bfff640 1 -- 172.21.15.38:0/70349135 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f107400ac30 con 0x7f10840d0f60 2024-04-25T08:36:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.461+0000 7f107bfff640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f106406b4d0 0x7f106406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.461+0000 7f107bfff640 1 -- 172.21.15.38:0/70349135 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f107408a250 con 0x7f10840d0f60 2024-04-25T08:36:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.461+0000 7f10827fc640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f106406b4d0 0x7f106406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.461+0000 7f10827fc640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f106406b4d0 0x7f106406d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f10840d3290 tx=0x7f106c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.465+0000 7f107bfff640 1 -- 172.21.15.38:0/70349135 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10840c7dd0 con 0x7f10840d0f60 2024-04-25T08:36:13.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.683+0000 7f1089a1c640 1 -- 172.21.15.38:0/70349135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1084160c30 con 0x7f10840d0f60 2024-04-25T08:36:13.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.683+0000 7f107bfff640 1 -- 172.21.15.38:0/70349135 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1074058800 con 0x7f10840d0f60 2024-04-25T08:36:13.678 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:13.678 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/1366115592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1366115592},{"type":"v1","addr":"172.21.15.107:6837","nonce":1366115592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6379,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/397891417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":397891417},{"type":"v1","addr":"172.21.15.107:6835","nonce":397891417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1355647168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1355647168},{"type":"v1","addr":"172.21.15.38:6835","nonce":1355647168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:40.623355+0000","modified":"2024-04-25T08:35:55.386447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7080":{"gid":7080,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/1952521723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1952521723},{"type":"v1","addr":"172.21.15.38:6833","nonce":1952521723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:36:13.678 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 42 2024-04-25T08:36:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.685+0000 7f1089a1c640 1 -- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f106406b4d0 msgr2=0x7f106406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.685+0000 7f1089a1c640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f106406b4d0 0x7f106406d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f10840d3290 tx=0x7f106c009040 comp rx=0 tx=0).stop 2024-04-25T08:36:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.685+0000 7f1089a1c640 1 -- 172.21.15.38:0/70349135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10840d0f60 msgr2=0x7f108415b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.685+0000 7f1089a1c640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10840d0f60 0x7f108415b930 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f107400c730 tx=0x7f107400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.685+0000 7f1089a1c640 1 -- 172.21.15.38:0/70349135 shutdown_connections 2024-04-25T08:36:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.685+0000 7f1089a1c640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f108415dc10 0x7f108415b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.685+0000 7f1089a1c640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f106406b4d0 0x7f106406d990 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.685+0000 7f1089a1c640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10840d3900 0x7f10840c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.685+0000 7f1089a1c640 1 --2- 172.21.15.38:0/70349135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10840d0f60 0x7f108415b930 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.685+0000 7f1089a1c640 1 -- 172.21.15.38:0/70349135 >> 172.21.15.38:0/70349135 conn(0x7f1084066d10 msgr2=0x7f10840ce6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:13.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.685+0000 7f1089a1c640 1 -- 172.21.15.38:0/70349135 shutdown_connections 2024-04-25T08:36:13.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.685+0000 7f1089a1c640 1 -- 172.21.15.38:0/70349135 wait complete. 2024-04-25T08:36:13.691 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:13.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.797+0000 7fc5aebce640 1 Processor -- start 2024-04-25T08:36:13.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.797+0000 7fc5aebce640 1 -- start start 2024-04-25T08:36:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.798+0000 7fc5aebce640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5a80b5710 0x7fc5a8161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.798+0000 7fc5aebce640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc5a815ddf0 0x7fc5a80b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.798+0000 7fc5aebce640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a8161e00 con 0x7fc5a80b5af0 2024-04-25T08:36:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.798+0000 7fc5aebce640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a80b9930 con 0x7fc5a80b5710 2024-04-25T08:36:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.798+0000 7fc5aebce640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a81620e0 con 0x7fc5a815ddf0 2024-04-25T08:36:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.798+0000 7fc5ac96a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5a80b5710 0x7fc5a8161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.798+0000 7fc5ad16b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc5a80b5af0 0x7fc5a806bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34890/0 (socket says 172.21.15.38:34890) 2024-04-25T08:36:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.798+0000 7fc5ac96a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5a80b5710 0x7fc5a8161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53962/0 (socket says 172.21.15.38:53962) 2024-04-25T08:36:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.798+0000 7fc5ad16b640 1 -- 172.21.15.38:0/3757008283 learned_addr learned my addr 172.21.15.38:0/3757008283 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:13.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.798+0000 7fc5a7fff640 1 --2- 172.21.15.38:0/3757008283 >> v2:172.21.15.107:3300/0 conn(0x7fc5a815ddf0 0x7fc5a80b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.799+0000 7fc5ac96a640 1 -- 172.21.15.38:0/3757008283 >> v2:172.21.15.107:3300/0 conn(0x7fc5a815ddf0 msgr2=0x7fc5a80b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:13.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.799+0000 7fc5ac96a640 1 --2- 172.21.15.38:0/3757008283 >> v2:172.21.15.107:3300/0 conn(0x7fc5a815ddf0 0x7fc5a80b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.799+0000 7fc5ac96a640 1 -- 172.21.15.38:0/3757008283 >> v1:172.21.15.38:6789/0 conn(0x7fc5a80b5af0 legacy=0x7fc5a806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.799+0000 7fc5ac96a640 1 -- 172.21.15.38:0/3757008283 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a81623c0 con 0x7fc5a80b5710 2024-04-25T08:36:13.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.799+0000 7fc5a77fe640 1 -- 172.21.15.38:0/3757008283 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451695373 0 0) 0x7fc5a8161e00 con 0x7fc5a80b5af0 2024-04-25T08:36:13.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.799+0000 7fc5ac96a640 1 --2- 172.21.15.38:0/3757008283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a80b5710 0x7fc5a8161700 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80d0300 tx=0x7fc5a002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b203816ac03fedb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.799+0000 7fc5a77fe640 1 -- 172.21.15.38:0/3757008283 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0037070 con 0x7fc5a80b5710 2024-04-25T08:36:13.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.799+0000 7fc5a77fe640 1 -- 172.21.15.38:0/3757008283 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0032440 con 0x7fc5a80b5710 2024-04-25T08:36:13.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.799+0000 7fc5a77fe640 1 -- 172.21.15.38:0/3757008283 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0032810 con 0x7fc5a80b5710 2024-04-25T08:36:13.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.800+0000 7fc5aebce640 1 -- 172.21.15.38:0/3757008283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a80b5710 msgr2=0x7fc5a8161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.800+0000 7fc5aebce640 1 --2- 172.21.15.38:0/3757008283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a80b5710 0x7fc5a8161700 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80d0300 tx=0x7fc5a002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.800+0000 7fc5aebce640 1 -- 172.21.15.38:0/3757008283 shutdown_connections 2024-04-25T08:36:13.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.800+0000 7fc5aebce640 1 --2- 172.21.15.38:0/3757008283 >> v2:172.21.15.107:3300/0 conn(0x7fc5a815ddf0 0x7fc5a80b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.800+0000 7fc5aebce640 1 --2- 172.21.15.38:0/3757008283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a80b5710 0x7fc5a8161700 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.800+0000 7fc5aebce640 1 -- 172.21.15.38:0/3757008283 >> 172.21.15.38:0/3757008283 conn(0x7fc5a80d07c0 msgr2=0x7fc5a80d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:13.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.800+0000 7fc5aebce640 1 -- 172.21.15.38:0/3757008283 shutdown_connections 2024-04-25T08:36:13.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.800+0000 7fc5aebce640 1 -- 172.21.15.38:0/3757008283 wait complete. 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.801+0000 7fc5aebce640 1 Processor -- start 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.801+0000 7fc5aebce640 1 -- start start 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5aebce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5a80b5710 0x7fc5a814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5aebce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a80b5af0 0x7fc5a815f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5aebce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5a815ddf0 0x7fc5a815f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5aebce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80d34d0 con 0x7fc5a80b5af0 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5aebce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80d3350 con 0x7fc5a815ddf0 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5aebce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80d3650 con 0x7fc5a80b5710 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a80b5af0 0x7fc5a815f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a80b5af0 0x7fc5a815f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53974/0 (socket says 172.21.15.38:53974) 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5a7fff640 1 -- 172.21.15.38:0/3933210249 learned_addr learned my addr 172.21.15.38:0/3933210249 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5a7fff640 1 -- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5a80b5710 msgr2=0x7fc5a814ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5ac96a640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5a80b5710 0x7fc5a814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5a7fff640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5a80b5710 0x7fc5a814ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5ad16b640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5a815ddf0 0x7fc5a815f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.802+0000 7fc5a7fff640 1 -- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5a815ddf0 msgr2=0x7fc5a815f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.803+0000 7fc5a7fff640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5a815ddf0 0x7fc5a815f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.803+0000 7fc5a7fff640 1 -- 172.21.15.38:0/3933210249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a815e2a0 con 0x7fc5a80b5af0 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.803+0000 7fc5ac96a640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5a80b5710 0x7fc5a814ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.803+0000 7fc5ad16b640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5a815ddf0 0x7fc5a815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.803+0000 7fc5a7fff640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a80b5af0 0x7fc5a815f340 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc598007e30 tx=0x7fc59800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.803+0000 7fc5a57fa640 1 -- 172.21.15.38:0/3933210249 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598014620 con 0x7fc5a80b5af0 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.803+0000 7fc5a57fa640 1 -- 172.21.15.38:0/3933210249 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc598019070 con 0x7fc5a80b5af0 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.803+0000 7fc5aebce640 1 -- 172.21.15.38:0/3933210249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a815e580 con 0x7fc5a80b5af0 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.803+0000 7fc5a57fa640 1 -- 172.21.15.38:0/3933210249 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598014a00 con 0x7fc5a80b5af0 2024-04-25T08:36:13.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.803+0000 7fc5aebce640 1 -- 172.21.15.38:0/3933210249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a815eb40 con 0x7fc5a80b5af0 2024-04-25T08:36:13.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.804+0000 7fc5aebce640 1 -- 172.21.15.38:0/3933210249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a8068e80 con 0x7fc5a80b5af0 2024-04-25T08:36:13.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.804+0000 7fc5a57fa640 1 -- 172.21.15.38:0/3933210249 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc598014ba0 con 0x7fc5a80b5af0 2024-04-25T08:36:13.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.805+0000 7fc5a57fa640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc58806b4d0 0x7fc58806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:13.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.805+0000 7fc5a57fa640 1 -- 172.21.15.38:0/3933210249 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc59808abf0 con 0x7fc5a80b5af0 2024-04-25T08:36:13.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.805+0000 7fc5ac96a640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc58806b4d0 0x7fc58806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:13.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.805+0000 7fc5ac96a640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc58806b4d0 0x7fc58806d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0034000 tx=0x7fc5a002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:13.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.809+0000 7fc5a57fa640 1 -- 172.21.15.38:0/3933210249 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc598059920 con 0x7fc5a80b5af0 2024-04-25T08:36:13.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.991+0000 7fc5aebce640 1 -- 172.21.15.38:0/3933210249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5a80b3fd0 con 0x7fc5a80b5af0 2024-04-25T08:36:13.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.992+0000 7fc5a57fa640 1 -- 172.21.15.38:0/3933210249 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16999 (secure 0 0 0) 0x7fc598059220 con 0x7fc5a80b5af0 2024-04-25T08:36:13.987 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:13.987 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":71,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:52.358128+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:38.884603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:39.590140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/239365678":"2024-04-26T08:35:52.345208+0000","172.21.15.38:6833/239365678":"2024-04-26T08:35:52.345208+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:13.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.994+0000 7fc5aebce640 1 -- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc58806b4d0 msgr2=0x7fc58806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.994+0000 7fc5aebce640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc58806b4d0 0x7fc58806d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc5a0034000 tx=0x7fc5a002e000 comp rx=0 tx=0).stop 2024-04-25T08:36:13.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.994+0000 7fc5aebce640 1 -- 172.21.15.38:0/3933210249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a80b5af0 msgr2=0x7fc5a815f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:13.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.994+0000 7fc5aebce640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a80b5af0 0x7fc5a815f340 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fc598007e30 tx=0x7fc59800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.994+0000 7fc5aebce640 1 -- 172.21.15.38:0/3933210249 shutdown_connections 2024-04-25T08:36:13.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.994+0000 7fc5aebce640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5a815ddf0 0x7fc5a815f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.994+0000 7fc5aebce640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5a80b5af0 0x7fc5a815f340 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.994+0000 7fc5aebce640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc58806b4d0 0x7fc58806d990 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.994+0000 7fc5aebce640 1 --2- 172.21.15.38:0/3933210249 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5a80b5710 0x7fc5a814ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:13.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.994+0000 7fc5aebce640 1 -- 172.21.15.38:0/3933210249 >> 172.21.15.38:0/3933210249 conn(0x7fc5a80d07c0 msgr2=0x7fc5a815d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:13.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.994+0000 7fc5aebce640 1 -- 172.21.15.38:0/3933210249 shutdown_connections 2024-04-25T08:36:13.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:13.994+0000 7fc5aebce640 1 -- 172.21.15.38:0/3933210249 wait complete. 2024-04-25T08:36:14.000 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:36:14.000 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:14.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.106+0000 7f82de01f640 1 Processor -- start 2024-04-25T08:36:14.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.106+0000 7f82de01f640 1 -- start start 2024-04-25T08:36:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82de01f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82d80aef90 0x7f82d80acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82de01f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82d80ad2f0 0x7f82d806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82de01f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d815be30 con 0x7f82d815f7b0 2024-04-25T08:36:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82de01f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d80af3f0 con 0x7f82d80ad2f0 2024-04-25T08:36:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82de01f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d80ad760 con 0x7f82d80aef90 2024-04-25T08:36:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82d7fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82d80ad2f0 0x7f82d806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82d7fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82d80ad2f0 0x7f82d806abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53990/0 (socket says 172.21.15.38:53990) 2024-04-25T08:36:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82d7fff640 1 -- 172.21.15.38:0/601950850 learned_addr learned my addr 172.21.15.38:0/601950850 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:14.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82d6ffd640 1 --2- 172.21.15.38:0/601950850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82d80aef90 0x7f82d80acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82d7fff640 1 -- 172.21.15.38:0/601950850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82d80aef90 msgr2=0x7f82d80acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82d7fff640 1 --2- 172.21.15.38:0/601950850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82d80aef90 0x7f82d80acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82d7fff640 1 -- 172.21.15.38:0/601950850 >> v1:172.21.15.107:6789/0 conn(0x7f82d815f7b0 legacy=0x7f82d815fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82d7fff640 1 -- 172.21.15.38:0/601950850 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d815c110 con 0x7f82d80ad2f0 2024-04-25T08:36:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.107+0000 7f82d6ffd640 1 --2- 172.21.15.38:0/601950850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82d80aef90 0x7f82d80acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:14.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.108+0000 7f82d7fff640 1 --2- 172.21.15.38:0/601950850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82d80ad2f0 0x7f82d806abe0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009e80 tx=0x7f82d002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8927bc1ffac3197b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.108+0000 7f82d67fc640 1 -- 172.21.15.38:0/601950850 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0036070 con 0x7f82d80ad2f0 2024-04-25T08:36:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.108+0000 7f82d67fc640 1 -- 172.21.15.38:0/601950850 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d0031600 con 0x7f82d80ad2f0 2024-04-25T08:36:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.108+0000 7f82d67fc640 1 -- 172.21.15.38:0/601950850 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d00319d0 con 0x7f82d80ad2f0 2024-04-25T08:36:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.108+0000 7f82de01f640 1 -- 172.21.15.38:0/601950850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82d80ad2f0 msgr2=0x7f82d806abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.108+0000 7f82de01f640 1 --2- 172.21.15.38:0/601950850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82d80ad2f0 0x7f82d806abe0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009e80 tx=0x7f82d002a750 comp rx=0 tx=0).stop 2024-04-25T08:36:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.109+0000 7f82de01f640 1 -- 172.21.15.38:0/601950850 shutdown_connections 2024-04-25T08:36:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.109+0000 7f82de01f640 1 --2- 172.21.15.38:0/601950850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82d80ad2f0 0x7f82d806abe0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.109+0000 7f82de01f640 1 --2- 172.21.15.38:0/601950850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82d80aef90 0x7f82d80acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.109+0000 7f82de01f640 1 -- 172.21.15.38:0/601950850 >> 172.21.15.38:0/601950850 conn(0x7f82d80cfd30 msgr2=0x7f82d80d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.109+0000 7f82de01f640 1 -- 172.21.15.38:0/601950850 shutdown_connections 2024-04-25T08:36:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.109+0000 7f82de01f640 1 -- 172.21.15.38:0/601950850 wait complete. 2024-04-25T08:36:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.110+0000 7f82de01f640 1 Processor -- start 2024-04-25T08:36:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.110+0000 7f82de01f640 1 -- start start 2024-04-25T08:36:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82de01f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d80ad2f0 0x7f82d81410e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82de01f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82d80aef90 0x7f82d815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82de01f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82d815f7b0 0x7f82d815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82de01f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d815c6d0 con 0x7f82d815f7b0 2024-04-25T08:36:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82de01f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d81602a0 con 0x7f82d80ad2f0 2024-04-25T08:36:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82de01f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d806b120 con 0x7f82d80aef90 2024-04-25T08:36:14.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82d815f7b0 0x7f82d815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82d815f7b0 0x7f82d815ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54006/0 (socket says 172.21.15.38:54006) 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82d7fff640 1 -- 172.21.15.38:0/3062724406 learned_addr learned my addr 172.21.15.38:0/3062724406 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82d6ffd640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82d80aef90 0x7f82d815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82d77fe640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d80ad2f0 0x7f82d81410e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82d7fff640 1 -- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82d80aef90 msgr2=0x7f82d815a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82d7fff640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82d80aef90 0x7f82d815a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82d7fff640 1 -- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d80ad2f0 msgr2=0x7f82d81410e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82d7fff640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d80ad2f0 0x7f82d81410e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82d7fff640 1 -- 172.21.15.38:0/3062724406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d80ce6d0 con 0x7f82d815f7b0 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82d7fff640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82d815f7b0 0x7f82d815ad30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f82d0034040 tx=0x7f82d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82bffff640 1 -- 172.21.15.38:0/3062724406 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0036050 con 0x7f82d815f7b0 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82bffff640 1 -- 172.21.15.38:0/3062724406 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d0042070 con 0x7f82d815f7b0 2024-04-25T08:36:14.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82bffff640 1 -- 172.21.15.38:0/3062724406 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0009510 con 0x7f82d815f7b0 2024-04-25T08:36:14.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.111+0000 7f82de01f640 1 -- 172.21.15.38:0/3062724406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d80ce9b0 con 0x7f82d815f7b0 2024-04-25T08:36:14.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.112+0000 7f82de01f640 1 -- 172.21.15.38:0/3062724406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d8152f40 con 0x7f82d815f7b0 2024-04-25T08:36:14.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.113+0000 7f82de01f640 1 -- 172.21.15.38:0/3062724406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d8068e80 con 0x7f82d815f7b0 2024-04-25T08:36:14.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.113+0000 7f82bffff640 1 -- 172.21.15.38:0/3062724406 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82d0007bd0 con 0x7f82d815f7b0 2024-04-25T08:36:14.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.113+0000 7f82bffff640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f82c406b4d0 0x7f82c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.114+0000 7f82bffff640 1 -- 172.21.15.38:0/3062724406 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f82d00ab110 con 0x7f82d815f7b0 2024-04-25T08:36:14.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.114+0000 7f82d77fe640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f82c406b4d0 0x7f82c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.114+0000 7f82d77fe640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f82c406b4d0 0x7f82c406d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f82c80016e0 tx=0x7f82c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:14.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.118+0000 7f82bffff640 1 -- 172.21.15.38:0/3062724406 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82d0079760 con 0x7f82d815f7b0 2024-04-25T08:36:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.334+0000 7f82de01f640 1 -- 172.21.15.38:0/3062724406 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82d8066eb0 con 0x7f82d815f7b0 2024-04-25T08:36:14.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.334+0000 7f82bffff640 1 -- 172.21.15.38:0/3062724406 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f82d007d800 con 0x7f82d815f7b0 2024-04-25T08:36:14.329 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:14.329 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/1366115592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1366115592},{"type":"v1","addr":"172.21.15.107:6837","nonce":1366115592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6379,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/397891417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":397891417},{"type":"v1","addr":"172.21.15.107:6835","nonce":397891417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1355647168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1355647168},{"type":"v1","addr":"172.21.15.38:6835","nonce":1355647168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:40.623355+0000","modified":"2024-04-25T08:35:55.386447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7080":{"gid":7080,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/1952521723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1952521723},{"type":"v1","addr":"172.21.15.38:6833","nonce":1952521723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:36:14.329 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 42 2024-04-25T08:36:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.336+0000 7f82de01f640 1 -- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f82c406b4d0 msgr2=0x7f82c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.336+0000 7f82de01f640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f82c406b4d0 0x7f82c406d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f82c80016e0 tx=0x7f82c8009040 comp rx=0 tx=0).stop 2024-04-25T08:36:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.336+0000 7f82de01f640 1 -- 172.21.15.38:0/3062724406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82d815f7b0 msgr2=0x7f82d815ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.336+0000 7f82de01f640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82d815f7b0 0x7f82d815ad30 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f82d0034040 tx=0x7f82d0004a40 comp rx=0 tx=0).stop 2024-04-25T08:36:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.336+0000 7f82de01f640 1 -- 172.21.15.38:0/3062724406 shutdown_connections 2024-04-25T08:36:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.336+0000 7f82de01f640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82d815f7b0 0x7f82d815ad30 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.336+0000 7f82de01f640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f82c406b4d0 0x7f82c406d990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.336+0000 7f82de01f640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82d80aef90 0x7f82d815a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.336+0000 7f82de01f640 1 --2- 172.21.15.38:0/3062724406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82d80ad2f0 0x7f82d81410e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.336+0000 7f82de01f640 1 -- 172.21.15.38:0/3062724406 >> 172.21.15.38:0/3062724406 conn(0x7f82d80cfd30 msgr2=0x7f82d815fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.336+0000 7f82de01f640 1 -- 172.21.15.38:0/3062724406 shutdown_connections 2024-04-25T08:36:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.336+0000 7f82de01f640 1 -- 172.21.15.38:0/3062724406 wait complete. 2024-04-25T08:36:14.341 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.448+0000 7f1a9c0bd640 1 Processor -- start 2024-04-25T08:36:14.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.448+0000 7f1a9c0bd640 1 -- start start 2024-04-25T08:36:14.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a9c0bd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1a94099a70 0x7f1a94099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a9c0bd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1a9406cd90 0x7f1a94167710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a9c0bd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a9409a390 con 0x7f1a940cf470 2024-04-25T08:36:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a9c0bd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a941614d0 con 0x7f1a9406cd90 2024-04-25T08:36:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a9c0bd640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a940cd970 con 0x7f1a94099a70 2024-04-25T08:36:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a9a65a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1a9406cd90 0x7f1a94167710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a9a65a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1a9406cd90 0x7f1a94167710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54010/0 (socket says 172.21.15.38:54010) 2024-04-25T08:36:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a9a65a640 1 -- 172.21.15.38:0/4156859743 learned_addr learned my addr 172.21.15.38:0/4156859743 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a99e59640 1 --2- 172.21.15.38:0/4156859743 >> v2:172.21.15.107:3300/0 conn(0x7f1a94099a70 0x7f1a94099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a9a65a640 1 -- 172.21.15.38:0/4156859743 >> v2:172.21.15.107:3300/0 conn(0x7f1a94099a70 msgr2=0x7f1a94099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:14.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a9a65a640 1 --2- 172.21.15.38:0/4156859743 >> v2:172.21.15.107:3300/0 conn(0x7f1a94099a70 0x7f1a94099e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a9a65a640 1 -- 172.21.15.38:0/4156859743 >> v1:172.21.15.107:6789/0 conn(0x7f1a940cf470 legacy=0x7f1a940cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:14.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a9a65a640 1 -- 172.21.15.38:0/4156859743 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a940ccd60 con 0x7f1a9406cd90 2024-04-25T08:36:14.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.449+0000 7f1a99e59640 1 --2- 172.21.15.38:0/4156859743 >> v2:172.21.15.107:3300/0 conn(0x7f1a94099a70 0x7f1a94099e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:14.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.450+0000 7f1a9a65a640 1 --2- 172.21.15.38:0/4156859743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9406cd90 0x7f1a94167710 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1a940729d0 tx=0x7f1a8c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ef0b8ff204b5f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:14.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.450+0000 7f1a98e57640 1 -- 172.21.15.38:0/4156859743 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c036070 con 0x7f1a9406cd90 2024-04-25T08:36:14.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.450+0000 7f1a98e57640 1 -- 172.21.15.38:0/4156859743 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8c031600 con 0x7f1a9406cd90 2024-04-25T08:36:14.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.450+0000 7f1a98e57640 1 -- 172.21.15.38:0/4156859743 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c0319d0 con 0x7f1a9406cd90 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.451+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/4156859743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9406cd90 msgr2=0x7f1a94167710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.451+0000 7f1a9c0bd640 1 --2- 172.21.15.38:0/4156859743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9406cd90 0x7f1a94167710 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f1a940729d0 tx=0x7f1a8c02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.451+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/4156859743 shutdown_connections 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.451+0000 7f1a9c0bd640 1 --2- 172.21.15.38:0/4156859743 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9406cd90 0x7f1a94167710 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.451+0000 7f1a9c0bd640 1 --2- 172.21.15.38:0/4156859743 >> v2:172.21.15.107:3300/0 conn(0x7f1a94099a70 0x7f1a94099e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.451+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/4156859743 >> 172.21.15.38:0/4156859743 conn(0x7f1a940b3180 msgr2=0x7f1a940b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.451+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/4156859743 shutdown_connections 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.451+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/4156859743 wait complete. 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.452+0000 7f1a9c0bd640 1 Processor -- start 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.452+0000 7f1a9c0bd640 1 -- start start 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a9c0bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9406cd90 0x7f1a940c5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a9c0bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94099a70 0x7f1a940cb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a9c0bd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a940cf470 0x7f1a940cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a9c0bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940cd600 con 0x7f1a9406cd90 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a9c0bd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940cd320 con 0x7f1a94099a70 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a9c0bd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a940d0780 con 0x7f1a940cf470 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a99e59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9406cd90 0x7f1a940c5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a99e59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9406cd90 0x7f1a940c5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54012/0 (socket says 172.21.15.38:54012) 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a99e59640 1 -- 172.21.15.38:0/2351490695 learned_addr learned my addr 172.21.15.38:0/2351490695 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a9a65a640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a940cf470 0x7f1a940cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a99658640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94099a70 0x7f1a940cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a99e59640 1 -- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a940cf470 msgr2=0x7f1a940cb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a99e59640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a940cf470 0x7f1a940cb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a99e59640 1 -- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94099a70 msgr2=0x7f1a940cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a99e59640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94099a70 0x7f1a940cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a99e59640 1 -- 172.21.15.38:0/2351490695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a940c5e70 con 0x7f1a9406cd90 2024-04-25T08:36:14.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a99658640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94099a70 0x7f1a940cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:14.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.453+0000 7f1a99e59640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9406cd90 0x7f1a940c5930 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000c820 tx=0x7f1a9000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:14.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.455+0000 7f1a8a7fc640 1 -- 172.21.15.38:0/2351490695 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90015070 con 0x7f1a9406cd90 2024-04-25T08:36:14.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.455+0000 7f1a8a7fc640 1 -- 172.21.15.38:0/2351490695 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a90007a30 con 0x7f1a9406cd90 2024-04-25T08:36:14.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.455+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/2351490695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a940ca570 con 0x7f1a9406cd90 2024-04-25T08:36:14.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.455+0000 7f1a8a7fc640 1 -- 172.21.15.38:0/2351490695 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90007e00 con 0x7f1a9406cd90 2024-04-25T08:36:14.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.455+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/2351490695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a940cab10 con 0x7f1a9406cd90 2024-04-25T08:36:14.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.456+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/2351490695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a94068e80 con 0x7f1a9406cd90 2024-04-25T08:36:14.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.457+0000 7f1a8a7fc640 1 -- 172.21.15.38:0/2351490695 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a90018070 con 0x7f1a9406cd90 2024-04-25T08:36:14.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.457+0000 7f1a8a7fc640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1a7806b500 0x7f1a7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.457+0000 7f1a8a7fc640 1 -- 172.21.15.38:0/2351490695 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1a9008a230 con 0x7f1a9406cd90 2024-04-25T08:36:14.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.458+0000 7f1a99658640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1a7806b500 0x7f1a7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.458+0000 7f1a99658640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1a7806b500 0x7f1a7806d9c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1a9415eac0 tx=0x7f1a84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:14.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.461+0000 7f1a8a7fc640 1 -- 172.21.15.38:0/2351490695 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a90058910 con 0x7f1a9406cd90 2024-04-25T08:36:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.679+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/2351490695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a940cebc0 con 0x7f1a9406cd90 2024-04-25T08:36:14.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.679+0000 7f1a8a7fc640 1 -- 172.21.15.38:0/2351490695 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1a9005c9b0 con 0x7f1a9406cd90 2024-04-25T08:36:14.674 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:14.674 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":42,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/1366115592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1366115592},{"type":"v1","addr":"172.21.15.107:6837","nonce":1366115592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6379,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/397891417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":397891417},{"type":"v1","addr":"172.21.15.107:6835","nonce":397891417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1355647168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1355647168},{"type":"v1","addr":"172.21.15.38:6835","nonce":1355647168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:35:40.623355+0000","modified":"2024-04-25T08:35:55.386447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":70,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7080},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7080":{"gid":7080,"name":"a","rank":0,"incarnation":39,"state":"up:active","state_seq":4,"addr":"172.21.15.38:6833/1952521723","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1952521723},{"type":"v1","addr":"172.21.15.38:6833","nonce":1952521723}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:36:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 42 2024-04-25T08:36:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.681+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1a7806b500 msgr2=0x7f1a7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.681+0000 7f1a9c0bd640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1a7806b500 0x7f1a7806d9c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1a9415eac0 tx=0x7f1a84009040 comp rx=0 tx=0).stop 2024-04-25T08:36:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.681+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/2351490695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9406cd90 msgr2=0x7f1a940c5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.681+0000 7f1a9c0bd640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9406cd90 0x7f1a940c5930 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f1a9000c820 tx=0x7f1a9000cc20 comp rx=0 tx=0).stop 2024-04-25T08:36:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.681+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/2351490695 shutdown_connections 2024-04-25T08:36:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.681+0000 7f1a9c0bd640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1a7806b500 0x7f1a7806d9c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.681+0000 7f1a9c0bd640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1a940cf470 0x7f1a940cb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.681+0000 7f1a9c0bd640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a94099a70 0x7f1a940cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.681+0000 7f1a9c0bd640 1 --2- 172.21.15.38:0/2351490695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a9406cd90 0x7f1a940c5930 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.681+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/2351490695 >> 172.21.15.38:0/2351490695 conn(0x7f1a940b3180 msgr2=0x7f1a940ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.681+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/2351490695 shutdown_connections 2024-04-25T08:36:14.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.681+0000 7f1a9c0bd640 1 -- 172.21.15.38:0/2351490695 wait complete. 2024-04-25T08:36:14.687 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:14.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.793+0000 7f0a91363640 1 Processor -- start 2024-04-25T08:36:14.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.793+0000 7f0a91363640 1 -- start start 2024-04-25T08:36:14.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.794+0000 7f0a91363640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c06c110 0x7f0a8c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.794+0000 7f0a91363640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a8c06ca30 con 0x7f0a8c0ce2e0 2024-04-25T08:36:14.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.794+0000 7f0a91363640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a8c0ceff0 con 0x7f0a8c15df50 2024-04-25T08:36:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.794+0000 7f0a91363640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c0cf2d0 con 0x7f0a8c06c110 2024-04-25T08:36:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.794+0000 7f0a8b7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0a8c0ce2e0 0x7f0a8c0d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34892/0 (socket says 172.21.15.38:34892) 2024-04-25T08:36:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.794+0000 7f0a8b7fe640 1 -- 172.21.15.38:0/3492986502 learned_addr learned my addr 172.21.15.38:0/3492986502 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.794+0000 7f0a8affd640 1 --2- 172.21.15.38:0/3492986502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c06c110 0x7f0a8c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.794+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951343227 0 0) 0x7f0a8c06ca30 con 0x7f0a8c0ce2e0 2024-04-25T08:36:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.794+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a74003520 con 0x7f0a8c0ce2e0 2024-04-25T08:36:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1873418152 0 0) 0x7f0a74003520 con 0x7f0a8c0ce2e0 2024-04-25T08:36:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c06c110 msgr2=0x7f0a8c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 --2- 172.21.15.38:0/3492986502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c06c110 0x7f0a8c06c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 >> v1:172.21.15.107:6789/0 conn(0x7f0a8c15df50 legacy=0x7f0a8c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c0d1f70 con 0x7f0a8c0ce2e0 2024-04-25T08:36:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f0a7c001670 con 0x7f0a8c0ce2e0 2024-04-25T08:36:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 >> v1:172.21.15.38:6789/0 conn(0x7f0a8c0ce2e0 legacy=0x7f0a8c0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 --2- 172.21.15.38:0/3492986502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c06c110 0x7f0a740039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 --2- 172.21.15.38:0/3492986502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a74003f20 0x7f0a74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 --2- 172.21.15.38:0/3492986502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a74004d40 0x7f0a740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c0cf2d0 con 0x7f0a74004d40 2024-04-25T08:36:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7400ba20 con 0x7f0a74003f20 2024-04-25T08:36:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7400bc20 con 0x7f0a8c06c110 2024-04-25T08:36:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a8b7fe640 1 --2- 172.21.15.38:0/3492986502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a74004d40 0x7f0a740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a8affd640 1 --2- 172.21.15.38:0/3492986502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c06c110 0x7f0a740039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a8a7fc640 1 --2- 172.21.15.38:0/3492986502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a74003f20 0x7f0a74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a8b7fe640 1 -- 172.21.15.38:0/3492986502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c06c110 msgr2=0x7f0a740039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a8b7fe640 1 --2- 172.21.15.38:0/3492986502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c06c110 0x7f0a740039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a8b7fe640 1 -- 172.21.15.38:0/3492986502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a74003f20 msgr2=0x7f0a74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a8b7fe640 1 --2- 172.21.15.38:0/3492986502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a74003f20 0x7f0a74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.795+0000 7f0a8b7fe640 1 -- 172.21.15.38:0/3492986502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a74003520 con 0x7f0a74004d40 2024-04-25T08:36:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.796+0000 7f0a8b7fe640 1 --2- 172.21.15.38:0/3492986502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a74004d40 0x7f0a740051c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c007fd0 tx=0x7f0a74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b980b9affe3d8858 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.796+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c007d50 con 0x7f0a74004d40 2024-04-25T08:36:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.796+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c0058d0 con 0x7f0a74004d40 2024-04-25T08:36:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.796+0000 7f0a89ffb640 1 -- 172.21.15.38:0/3492986502 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c005d20 con 0x7f0a74004d40 2024-04-25T08:36:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.796+0000 7f0a91363640 1 -- 172.21.15.38:0/3492986502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a74004d40 msgr2=0x7f0a740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.796+0000 7f0a91363640 1 --2- 172.21.15.38:0/3492986502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a74004d40 0x7f0a740051c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c007fd0 tx=0x7f0a74005830 comp rx=0 tx=0).stop 2024-04-25T08:36:14.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.796+0000 7f0a8b7fe640 1 -- 172.21.15.38:0/3492986502 reap_dead start 2024-04-25T08:36:14.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.797+0000 7f0a91363640 1 -- 172.21.15.38:0/3492986502 shutdown_connections 2024-04-25T08:36:14.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.797+0000 7f0a91363640 1 -- 172.21.15.38:0/3492986502 >> 172.21.15.38:0/3492986502 conn(0x7f0a8c0d3fb0 msgr2=0x7f0a8c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:14.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.797+0000 7f0a91363640 1 -- 172.21.15.38:0/3492986502 shutdown_connections 2024-04-25T08:36:14.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.797+0000 7f0a91363640 1 -- 172.21.15.38:0/3492986502 wait complete. 2024-04-25T08:36:14.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.798+0000 7f0a91363640 1 Processor -- start 2024-04-25T08:36:14.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.798+0000 7f0a91363640 1 -- start start 2024-04-25T08:36:14.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a91363640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a74004d40 0x7f0a8c160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a91363640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c15f8f0 0x7f0a8c0ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a91363640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a8c14cc60 0x7f0a8c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a91363640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c0d3160 con 0x7f0a8c14cc60 2024-04-25T08:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a91363640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c0d2530 con 0x7f0a74004d40 2024-04-25T08:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a91363640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c0d32e0 con 0x7f0a8c15f8f0 2024-04-25T08:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a8b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a8c14cc60 0x7f0a8c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a8b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a8c14cc60 0x7f0a8c0cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54026/0 (socket says 172.21.15.38:54026) 2024-04-25T08:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a8b7fe640 1 -- 172.21.15.38:0/4288584840 learned_addr learned my addr 172.21.15.38:0/4288584840 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a8affd640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a74004d40 0x7f0a8c160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a8a7fc640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c15f8f0 0x7f0a8c0ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a8b7fe640 1 -- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c15f8f0 msgr2=0x7f0a8c0ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a8b7fe640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c15f8f0 0x7f0a8c0ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a8b7fe640 1 -- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a74004d40 msgr2=0x7f0a8c160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a8b7fe640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a74004d40 0x7f0a8c160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a8b7fe640 1 -- 172.21.15.38:0/4288584840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c15fdb0 con 0x7f0a8c14cc60 2024-04-25T08:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a8a7fc640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c15f8f0 0x7f0a8c0ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a8b7fe640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a8c14cc60 0x7f0a8c0cb580 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00c350 tx=0x7f0a7c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.799+0000 7f0a737fe640 1 -- 172.21.15.38:0/4288584840 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c042070 con 0x7f0a8c14cc60 2024-04-25T08:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.800+0000 7f0a737fe640 1 -- 172.21.15.38:0/4288584840 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7c007550 con 0x7f0a8c14cc60 2024-04-25T08:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.800+0000 7f0a91363640 1 -- 172.21.15.38:0/4288584840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8c160090 con 0x7f0a8c14cc60 2024-04-25T08:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.800+0000 7f0a737fe640 1 -- 172.21.15.38:0/4288584840 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7c038040 con 0x7f0a8c14cc60 2024-04-25T08:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.800+0000 7f0a91363640 1 -- 172.21.15.38:0/4288584840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c160650 con 0x7f0a8c14cc60 2024-04-25T08:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.801+0000 7f0a91363640 1 -- 172.21.15.38:0/4288584840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a8c068e80 con 0x7f0a8c14cc60 2024-04-25T08:36:14.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.801+0000 7f0a737fe640 1 -- 172.21.15.38:0/4288584840 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a7c045070 con 0x7f0a8c14cc60 2024-04-25T08:36:14.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.801+0000 7f0a737fe640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0a7806b4d0 0x7f0a7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:14.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.802+0000 7f0a737fe640 1 -- 172.21.15.38:0/4288584840 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0a7c0a98e0 con 0x7f0a8c14cc60 2024-04-25T08:36:14.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.802+0000 7f0a8affd640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0a7806b4d0 0x7f0a7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:14.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.802+0000 7f0a8affd640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0a7806b4d0 0x7f0a7806d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0a84007520 tx=0x7f0a8400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:14.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.806+0000 7f0a737fe640 1 -- 172.21.15.38:0/4288584840 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a7c0784f0 con 0x7f0a8c14cc60 2024-04-25T08:36:14.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.987+0000 7f0a91363640 1 -- 172.21.15.38:0/4288584840 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a8c0cec00 con 0x7f0a8c14cc60 2024-04-25T08:36:14.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.989+0000 7f0a737fe640 1 -- 172.21.15.38:0/4288584840 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f0a7c077df0 con 0x7f0a8c14cc60 2024-04-25T08:36:14.983 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:14.983 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":71,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:35:52.358128+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:35:38.884603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:35:39.590140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/239365678":"2024-04-26T08:35:52.345208+0000","172.21.15.38:6833/239365678":"2024-04-26T08:35:52.345208+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:14.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.991+0000 7f0a91363640 1 -- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0a7806b4d0 msgr2=0x7f0a7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.991+0000 7f0a91363640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0a7806b4d0 0x7f0a7806d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0a84007520 tx=0x7f0a8400b040 comp rx=0 tx=0).stop 2024-04-25T08:36:14.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.991+0000 7f0a91363640 1 -- 172.21.15.38:0/4288584840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a8c14cc60 msgr2=0x7f0a8c0cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.991+0000 7f0a91363640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a8c14cc60 0x7f0a8c0cb580 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0a7c00c350 tx=0x7f0a7c001a40 comp rx=0 tx=0).stop 2024-04-25T08:36:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.991+0000 7f0a91363640 1 -- 172.21.15.38:0/4288584840 shutdown_connections 2024-04-25T08:36:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.991+0000 7f0a91363640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0a8c14cc60 0x7f0a8c0cb580 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.991+0000 7f0a91363640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0a7806b4d0 0x7f0a7806d990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.991+0000 7f0a91363640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0a8c15f8f0 0x7f0a8c0ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.991+0000 7f0a91363640 1 --2- 172.21.15.38:0/4288584840 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0a74004d40 0x7f0a8c160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.991+0000 7f0a91363640 1 -- 172.21.15.38:0/4288584840 >> 172.21.15.38:0/4288584840 conn(0x7f0a8c0d3fb0 msgr2=0x7f0a8c0b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.991+0000 7f0a91363640 1 -- 172.21.15.38:0/4288584840 shutdown_connections 2024-04-25T08:36:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:14.991+0000 7f0a91363640 1 -- 172.21.15.38:0/4288584840 wait complete. 2024-04-25T08:36:14.996 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:36:15.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.104+0000 7ff083efc640 1 Processor -- start 2024-04-25T08:36:15.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.104+0000 7ff083efc640 1 -- start start 2024-04-25T08:36:15.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff083efc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff07c0b9600 0x7ff07c0d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff083efc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff07c0d0660 0x7ff07c0d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff083efc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07c162030 con 0x7ff07c15db80 2024-04-25T08:36:15.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff083efc640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07c161df0 con 0x7ff07c0b9600 2024-04-25T08:36:15.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff083efc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c1625f0 con 0x7ff07c0d0660 2024-04-25T08:36:15.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff081c98640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff07c15db80 0x7ff07c1616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34906/0 (socket says 172.21.15.38:34906) 2024-04-25T08:36:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff081c98640 1 -- 172.21.15.38:0/27905610 learned_addr learned my addr 172.21.15.38:0/27905610 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749633731 0 0) 0x7ff07c162030 con 0x7ff07c15db80 2024-04-25T08:36:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff068003520 con 0x7ff07c15db80 2024-04-25T08:36:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff082499640 1 --2- 172.21.15.38:0/27905610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff07c0d0660 0x7ff07c0d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff081497640 1 --2- 172.21.15.38:0/27905610 >> v2:172.21.15.107:3300/0 conn(0x7ff07c0b9600 0x7ff07c0d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1325850068 0 0) 0x7ff068003520 con 0x7ff07c15db80 2024-04-25T08:36:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff07c0d0660 msgr2=0x7ff07c0d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 --2- 172.21.15.38:0/27905610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff07c0d0660 0x7ff07c0d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 >> v2:172.21.15.107:3300/0 conn(0x7ff07c0b9600 msgr2=0x7ff07c0d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 --2- 172.21.15.38:0/27905610 >> v2:172.21.15.107:3300/0 conn(0x7ff07c0b9600 0x7ff07c0d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c162310 con 0x7ff07c15db80 2024-04-25T08:36:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7ff078001670 con 0x7ff07c15db80 2024-04-25T08:36:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 >> v1:172.21.15.38:6789/0 conn(0x7ff07c15db80 legacy=0x7ff07c1616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 --2- 172.21.15.38:0/27905610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0680039b0 0x7ff068003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 --2- 172.21.15.38:0/27905610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff068004800 0x7ff068004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.105+0000 7ff080c96640 1 --2- 172.21.15.38:0/27905610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff07c0d0660 0x7ff0680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.106+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c161df0 con 0x7ff068004800 2024-04-25T08:36:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.106+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c1625f0 con 0x7ff0680039b0 2024-04-25T08:36:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.106+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06800b9c0 con 0x7ff07c0d0660 2024-04-25T08:36:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.106+0000 7ff081497640 1 --2- 172.21.15.38:0/27905610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff068004800 0x7ff068004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.106+0000 7ff081c98640 1 --2- 172.21.15.38:0/27905610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0680039b0 0x7ff068003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.106+0000 7ff082499640 1 --2- 172.21.15.38:0/27905610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff07c0d0660 0x7ff0680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.106+0000 7ff081497640 1 -- 172.21.15.38:0/27905610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff07c0d0660 msgr2=0x7ff0680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.106+0000 7ff081497640 1 --2- 172.21.15.38:0/27905610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff07c0d0660 0x7ff0680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.106+0000 7ff081497640 1 -- 172.21.15.38:0/27905610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0680039b0 msgr2=0x7ff068003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.106+0000 7ff081497640 1 --2- 172.21.15.38:0/27905610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0680039b0 0x7ff068003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.106+0000 7ff081497640 1 -- 172.21.15.38:0/27905610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff068003520 con 0x7ff068004800 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.107+0000 7ff081497640 1 --2- 172.21.15.38:0/27905610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff068004800 0x7ff068004c80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009ac0 tx=0x7ff068005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f153ace013aa626d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.107+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c036070 con 0x7ff068004800 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.107+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c031400 con 0x7ff068004800 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.107+0000 7ff080c96640 1 -- 172.21.15.38:0/27905610 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c009690 con 0x7ff068004800 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.107+0000 7ff083efc640 1 -- 172.21.15.38:0/27905610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff068004800 msgr2=0x7ff068004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.107+0000 7ff083efc640 1 --2- 172.21.15.38:0/27905610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff068004800 0x7ff068004c80 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009ac0 tx=0x7ff068005830 comp rx=0 tx=0).stop 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.107+0000 7ff082499640 1 -- 172.21.15.38:0/27905610 reap_dead start 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.107+0000 7ff083efc640 1 -- 172.21.15.38:0/27905610 shutdown_connections 2024-04-25T08:36:15.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.107+0000 7ff083efc640 1 -- 172.21.15.38:0/27905610 >> 172.21.15.38:0/27905610 conn(0x7ff07c0668f0 msgr2=0x7ff07c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:15.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.107+0000 7ff083efc640 1 -- 172.21.15.38:0/27905610 shutdown_connections 2024-04-25T08:36:15.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.107+0000 7ff083efc640 1 -- 172.21.15.38:0/27905610 wait complete. 2024-04-25T08:36:15.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.109+0000 7ff083efc640 1 Processor -- start 2024-04-25T08:36:15.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.109+0000 7ff083efc640 1 -- start start 2024-04-25T08:36:15.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.109+0000 7ff083efc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff068004800 0x7ff07c151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.109+0000 7ff083efc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff07c0cc990 0x7ff07c0ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.109+0000 7ff083efc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c0caa20 0x7ff07c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.109+0000 7ff083efc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c15df60 con 0x7ff07c0cc990 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.109+0000 7ff083efc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c0d0ab0 con 0x7ff07c0caa20 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.109+0000 7ff083efc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c0d22b0 con 0x7ff068004800 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.109+0000 7ff081497640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff07c0cc990 0x7ff07c0ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.109+0000 7ff081497640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff07c0cc990 0x7ff07c0ccdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54036/0 (socket says 172.21.15.38:54036) 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff081497640 1 -- 172.21.15.38:0/1314753392 learned_addr learned my addr 172.21.15.38:0/1314753392 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff082499640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c0caa20 0x7ff07c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff081c98640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff068004800 0x7ff07c151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff081497640 1 -- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff068004800 msgr2=0x7ff07c151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff081497640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff068004800 0x7ff07c151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff081497640 1 -- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c0caa20 msgr2=0x7ff07c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff081497640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c0caa20 0x7ff07c0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff081497640 1 -- 172.21.15.38:0/1314753392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c0cc2d0 con 0x7ff07c0cc990 2024-04-25T08:36:15.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff081497640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff07c0cc990 0x7ff07c0ccdf0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009ac0 tx=0x7ff06c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:15.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff0727fc640 1 -- 172.21.15.38:0/1314753392 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c041020 con 0x7ff07c0cc990 2024-04-25T08:36:15.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff0727fc640 1 -- 172.21.15.38:0/1314753392 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c03c470 con 0x7ff07c0cc990 2024-04-25T08:36:15.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff0727fc640 1 -- 172.21.15.38:0/1314753392 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c036040 con 0x7ff07c0cc990 2024-04-25T08:36:15.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff083efc640 1 -- 172.21.15.38:0/1314753392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07c0cb750 con 0x7ff07c0cc990 2024-04-25T08:36:15.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.110+0000 7ff083efc640 1 -- 172.21.15.38:0/1314753392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07c0cbd10 con 0x7ff07c0cc990 2024-04-25T08:36:15.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.111+0000 7ff083efc640 1 -- 172.21.15.38:0/1314753392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff07c04ef90 con 0x7ff07c0cc990 2024-04-25T08:36:15.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.112+0000 7ff0727fc640 1 -- 172.21.15.38:0/1314753392 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff06c044020 con 0x7ff07c0cc990 2024-04-25T08:36:15.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.112+0000 7ff0727fc640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff06006b500 0x7ff06006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.112+0000 7ff0727fc640 1 -- 172.21.15.38:0/1314753392 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff06c0aa170 con 0x7ff07c0cc990 2024-04-25T08:36:15.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.112+0000 7ff081c98640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff06006b500 0x7ff06006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.113+0000 7ff081c98640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff06006b500 0x7ff06006d9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff078001eb0 tx=0x7ff078007970 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:15.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.116+0000 7ff0727fc640 1 -- 172.21.15.38:0/1314753392 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff06c078850 con 0x7ff07c0cc990 2024-04-25T08:36:15.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.334+0000 7ff083efc640 1 -- 172.21.15.38:0/1314753392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff07c15e980 con 0x7ff07c0cc990 2024-04-25T08:36:15.328 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:36:15.334+0000 7ff7937f0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:36:15.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.336+0000 7ff0727fc640 1 -- 172.21.15.38:0/1314753392 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff06c07c8f0 con 0x7ff07c0cc990 2024-04-25T08:36:15.330 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:36:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.340+0000 7ff083efc640 1 -- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff06006b500 msgr2=0x7ff06006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.340+0000 7ff083efc640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff06006b500 0x7ff06006d9c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff078001eb0 tx=0x7ff078007970 comp rx=0 tx=0).stop 2024-04-25T08:36:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.340+0000 7ff083efc640 1 -- 172.21.15.38:0/1314753392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff07c0cc990 msgr2=0x7ff07c0ccdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.340+0000 7ff083efc640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff07c0cc990 0x7ff07c0ccdf0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009ac0 tx=0x7ff06c004730 comp rx=0 tx=0).stop 2024-04-25T08:36:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.340+0000 7ff083efc640 1 -- 172.21.15.38:0/1314753392 shutdown_connections 2024-04-25T08:36:15.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.340+0000 7ff083efc640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff07c0caa20 0x7ff07c0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.340+0000 7ff083efc640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff07c0cc990 0x7ff07c0ccdf0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.340+0000 7ff083efc640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff06006b500 0x7ff06006d9c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.340+0000 7ff083efc640 1 --2- 172.21.15.38:0/1314753392 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff068004800 0x7ff07c151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.340+0000 7ff083efc640 1 -- 172.21.15.38:0/1314753392 >> 172.21.15.38:0/1314753392 conn(0x7ff07c0668f0 msgr2=0x7ff07c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.340+0000 7ff083efc640 1 -- 172.21.15.38:0/1314753392 shutdown_connections 2024-04-25T08:36:15.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.340+0000 7ff083efc640 1 -- 172.21.15.38:0/1314753392 wait complete. 2024-04-25T08:36:15.336 INFO:tasks.ceph.mds.a.smithi038.stderr: -2127> 2024-04-25T08:35:59.357+0000 7f4d72ed1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:36:15.348 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:36:15.367 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:36:15.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.458+0000 7f829537e640 1 Processor -- start 2024-04-25T08:36:15.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.459+0000 7f829537e640 1 -- start start 2024-04-25T08:36:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.459+0000 7f829537e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f82900d33d0 0x7f82900d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.459+0000 7f829537e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82900d1e90 0x7f82900b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.459+0000 7f829537e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82900d2330 con 0x7f82900d0320 2024-04-25T08:36:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.459+0000 7f829537e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82900d2610 con 0x7f82900d1e90 2024-04-25T08:36:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.459+0000 7f829537e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82900d28f0 con 0x7f82900d33d0 2024-04-25T08:36:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.459+0000 7f828f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f82900d1e90 0x7f82900b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.460+0000 7f828effd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f82900d0320 0x7f82900d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34938/0 (socket says 172.21.15.38:34938) 2024-04-25T08:36:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.460+0000 7f828effd640 1 -- 172.21.15.38:0/2128304847 learned_addr learned my addr 172.21.15.38:0/2128304847 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.460+0000 7f828e7fc640 1 --2- 172.21.15.38:0/2128304847 >> v2:172.21.15.107:3300/0 conn(0x7f82900d33d0 0x7f82900d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.460+0000 7f828f7fe640 1 -- 172.21.15.38:0/2128304847 >> v2:172.21.15.107:3300/0 conn(0x7f82900d33d0 msgr2=0x7f82900d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.460+0000 7f828f7fe640 1 --2- 172.21.15.38:0/2128304847 >> v2:172.21.15.107:3300/0 conn(0x7f82900d33d0 0x7f82900d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.460+0000 7f828f7fe640 1 -- 172.21.15.38:0/2128304847 >> v1:172.21.15.38:6789/0 conn(0x7f82900d0320 legacy=0x7f82900d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.460+0000 7f828f7fe640 1 -- 172.21.15.38:0/2128304847 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82900d2bd0 con 0x7f82900d1e90 2024-04-25T08:36:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.460+0000 7f828dffb640 1 -- 172.21.15.38:0/2128304847 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071965423 0 0) 0x7f82900d2330 con 0x7f82900d0320 2024-04-25T08:36:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.461+0000 7f828f7fe640 1 --2- 172.21.15.38:0/2128304847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900d1e90 0x7f82900b6980 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f82900b6410 tx=0x7f828802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3c496778fdee7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.461+0000 7f828dffb640 1 -- 172.21.15.38:0/2128304847 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288007740 con 0x7f82900d1e90 2024-04-25T08:36:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.461+0000 7f828dffb640 1 -- 172.21.15.38:0/2128304847 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82880078e0 con 0x7f82900d1e90 2024-04-25T08:36:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.461+0000 7f828dffb640 1 -- 172.21.15.38:0/2128304847 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288007cb0 con 0x7f82900d1e90 2024-04-25T08:36:15.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.461+0000 7f829537e640 1 -- 172.21.15.38:0/2128304847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900d1e90 msgr2=0x7f82900b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.461+0000 7f829537e640 1 --2- 172.21.15.38:0/2128304847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900d1e90 0x7f82900b6980 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f82900b6410 tx=0x7f828802a710 comp rx=0 tx=0).stop 2024-04-25T08:36:15.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.461+0000 7f829537e640 1 -- 172.21.15.38:0/2128304847 shutdown_connections 2024-04-25T08:36:15.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.461+0000 7f829537e640 1 --2- 172.21.15.38:0/2128304847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900d1e90 0x7f82900b6980 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.461+0000 7f829537e640 1 --2- 172.21.15.38:0/2128304847 >> v2:172.21.15.107:3300/0 conn(0x7f82900d33d0 0x7f82900d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.461+0000 7f829537e640 1 -- 172.21.15.38:0/2128304847 >> 172.21.15.38:0/2128304847 conn(0x7f82900668f0 msgr2=0x7f8290066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:15.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.461+0000 7f829537e640 1 -- 172.21.15.38:0/2128304847 shutdown_connections 2024-04-25T08:36:15.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.461+0000 7f829537e640 1 -- 172.21.15.38:0/2128304847 wait complete. 2024-04-25T08:36:15.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.463+0000 7f829537e640 1 Processor -- start 2024-04-25T08:36:15.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.463+0000 7f829537e640 1 -- start start 2024-04-25T08:36:15.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.463+0000 7f829537e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82900d0320 0x7f82901573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.463+0000 7f829537e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900d1e90 0x7f82900b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.463+0000 7f829537e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900d33d0 0x7f82900b6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.463+0000 7f829537e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8290161390 con 0x7f82900d33d0 2024-04-25T08:36:15.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.463+0000 7f829537e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8290161210 con 0x7f82900d0320 2024-04-25T08:36:15.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.463+0000 7f829537e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82900b9720 con 0x7f82900d1e90 2024-04-25T08:36:15.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900d33d0 0x7f82900b6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f828f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900d33d0 0x7f82900b6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54072/0 (socket says 172.21.15.38:54072) 2024-04-25T08:36:15.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f828f7fe640 1 -- 172.21.15.38:0/2342115837 learned_addr learned my addr 172.21.15.38:0/2342115837 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:15.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f828effd640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82900d0320 0x7f82901573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f828e7fc640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900d1e90 0x7f82900b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f828f7fe640 1 -- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900d1e90 msgr2=0x7f82900b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f828f7fe640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900d1e90 0x7f82900b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f828f7fe640 1 -- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82900d0320 msgr2=0x7f82901573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:15.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f828f7fe640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82900d0320 0x7f82901573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:15.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f828f7fe640 1 -- 172.21.15.38:0/2342115837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82901578e0 con 0x7f82900d33d0 2024-04-25T08:36:15.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f828f7fe640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900d33d0 0x7f82900b6f90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8288009e50 tx=0x7f8288004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:15.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f82777fe640 1 -- 172.21.15.38:0/2342115837 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288040070 con 0x7f82900d33d0 2024-04-25T08:36:15.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f82777fe640 1 -- 172.21.15.38:0/2342115837 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8288031bf0 con 0x7f82900d33d0 2024-04-25T08:36:15.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f829537e640 1 -- 172.21.15.38:0/2342115837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8290157df0 con 0x7f82900d33d0 2024-04-25T08:36:15.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f82777fe640 1 -- 172.21.15.38:0/2342115837 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288009450 con 0x7f82900d33d0 2024-04-25T08:36:15.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.464+0000 7f829537e640 1 -- 172.21.15.38:0/2342115837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82901585a0 con 0x7f82900d33d0 2024-04-25T08:36:15.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.465+0000 7f829537e640 1 -- 172.21.15.38:0/2342115837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8290068e80 con 0x7f82900d33d0 2024-04-25T08:36:15.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.466+0000 7f82777fe640 1 -- 172.21.15.38:0/2342115837 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8288039040 con 0x7f82900d33d0 2024-04-25T08:36:15.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.466+0000 7f82777fe640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f827c06b4d0 0x7f827c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:15.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.466+0000 7f82777fe640 1 -- 172.21.15.38:0/2342115837 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f82880abca0 con 0x7f82900d33d0 2024-04-25T08:36:15.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.466+0000 7f828effd640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f827c06b4d0 0x7f827c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:15.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.467+0000 7f828effd640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f827c06b4d0 0x7f827c06d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8280000f60 tx=0x7f8280007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:15.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.470+0000 7f82777fe640 1 -- 172.21.15.38:0/2342115837 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f828807a8e0 con 0x7f82900d33d0 2024-04-25T08:36:15.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:15.697+0000 7f829537e640 1 -- 172.21.15.38:0/2342115837 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f82900c9e60 con 0x7f82900d33d0 2024-04-25T08:36:16.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.337+0000 7f82777fe640 1 -- 172.21.15.38:0/2342115837 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 104+0+0 (secure 0 0 0) 0x7f828807a1e0 con 0x7f82900d33d0 2024-04-25T08:36:16.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.341+0000 7f829537e640 1 -- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f827c06b4d0 msgr2=0x7f827c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:16.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.341+0000 7f829537e640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f827c06b4d0 0x7f827c06d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8280000f60 tx=0x7f8280007b50 comp rx=0 tx=0).stop 2024-04-25T08:36:16.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.341+0000 7f829537e640 1 -- 172.21.15.38:0/2342115837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900d33d0 msgr2=0x7f82900b6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:16.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.341+0000 7f829537e640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900d33d0 0x7f82900b6f90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8288009e50 tx=0x7f8288004a40 comp rx=0 tx=0).stop 2024-04-25T08:36:16.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.341+0000 7f829537e640 1 -- 172.21.15.38:0/2342115837 shutdown_connections 2024-04-25T08:36:16.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.341+0000 7f829537e640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900d33d0 0x7f82900b6f90 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:16.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.341+0000 7f829537e640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f827c06b4d0 0x7f827c06d990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:16.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.341+0000 7f829537e640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900d1e90 0x7f82900b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:16.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.341+0000 7f829537e640 1 --2- 172.21.15.38:0/2342115837 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82900d0320 0x7f82901573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:16.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.341+0000 7f829537e640 1 -- 172.21.15.38:0/2342115837 >> 172.21.15.38:0/2342115837 conn(0x7f82900668f0 msgr2=0x7f82900cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:16.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.341+0000 7f829537e640 1 -- 172.21.15.38:0/2342115837 shutdown_connections 2024-04-25T08:36:16.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.342+0000 7f829537e640 1 -- 172.21.15.38:0/2342115837 wait complete. 2024-04-25T08:36:16.350 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:36:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.509+0000 7f80c5b6a640 1 Processor -- start 2024-04-25T08:36:16.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.509+0000 7f80c5b6a640 1 -- start start 2024-04-25T08:36:16.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.510+0000 7f80c5b6a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f80c01612c0 0x7f80c00b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:16.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.510+0000 7f80c5b6a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f80c00b6a00 0x7f80c00d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:16.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.510+0000 7f80c5b6a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80c00d0870 0x7f80c00d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:16.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.510+0000 7f80c5b6a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c00d11b0 con 0x7f80c01612c0 2024-04-25T08:36:16.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.510+0000 7f80c5b6a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c00b7200 con 0x7f80c00b6a00 2024-04-25T08:36:16.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.510+0000 7f80c5b6a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c00b74e0 con 0x7f80c00d0870 2024-04-25T08:36:16.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.510+0000 7f80bf7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f80c01612c0 0x7f80c00b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.510+0000 7f80bf7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f80c01612c0 0x7f80c00b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54078/0 (socket says 172.21.15.38:54078) 2024-04-25T08:36:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.510+0000 7f80bf7fe640 1 -- 172.21.15.38:0/2177833327 learned_addr learned my addr 172.21.15.38:0/2177833327 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.510+0000 7f80bffff640 1 --2- 172.21.15.38:0/2177833327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80c00d0870 0x7f80c00d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.510+0000 7f80beffd640 1 --2- 172.21.15.38:0/2177833327 >> v2:172.21.15.107:3300/0 conn(0x7f80c00b6a00 0x7f80c00d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.511+0000 7f80bf7fe640 1 -- 172.21.15.38:0/2177833327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80c00d0870 msgr2=0x7f80c00d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.511+0000 7f80bf7fe640 1 --2- 172.21.15.38:0/2177833327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80c00d0870 0x7f80c00d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.511+0000 7f80bf7fe640 1 -- 172.21.15.38:0/2177833327 >> v2:172.21.15.107:3300/0 conn(0x7f80c00b6a00 msgr2=0x7f80c00d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.511+0000 7f80bf7fe640 1 --2- 172.21.15.38:0/2177833327 >> v2:172.21.15.107:3300/0 conn(0x7f80c00b6a00 0x7f80c00d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.511+0000 7f80bf7fe640 1 -- 172.21.15.38:0/2177833327 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c00b77c0 con 0x7f80c01612c0 2024-04-25T08:36:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.511+0000 7f80beffd640 1 --2- 172.21.15.38:0/2177833327 >> v2:172.21.15.107:3300/0 conn(0x7f80c00b6a00 0x7f80c00d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:16.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.511+0000 7f80bffff640 1 --2- 172.21.15.38:0/2177833327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80c00d0870 0x7f80c00d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.511+0000 7f80bf7fe640 1 --2- 172.21.15.38:0/2177833327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80c01612c0 0x7f80c00b93f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f80c00c71f0 tx=0x7f80b002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0b45081397cc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.511+0000 7f80be7fc640 1 -- 172.21.15.38:0/2177833327 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0037070 con 0x7f80c01612c0 2024-04-25T08:36:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.511+0000 7f80be7fc640 1 -- 172.21.15.38:0/2177833327 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b0032440 con 0x7f80c01612c0 2024-04-25T08:36:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.511+0000 7f80be7fc640 1 -- 172.21.15.38:0/2177833327 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0032810 con 0x7f80c01612c0 2024-04-25T08:36:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.512+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2177833327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80c01612c0 msgr2=0x7f80c00b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.512+0000 7f80c5b6a640 1 --2- 172.21.15.38:0/2177833327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80c01612c0 0x7f80c00b93f0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f80c00c71f0 tx=0x7f80b002a140 comp rx=0 tx=0).stop 2024-04-25T08:36:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.512+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2177833327 shutdown_connections 2024-04-25T08:36:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.512+0000 7f80c5b6a640 1 --2- 172.21.15.38:0/2177833327 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80c00d0870 0x7f80c00d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.512+0000 7f80c5b6a640 1 --2- 172.21.15.38:0/2177833327 >> v2:172.21.15.107:3300/0 conn(0x7f80c00b6a00 0x7f80c00d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.512+0000 7f80c5b6a640 1 --2- 172.21.15.38:0/2177833327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80c01612c0 0x7f80c00b93f0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.512+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2177833327 >> 172.21.15.38:0/2177833327 conn(0x7f80c0066d10 msgr2=0x7f80c0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.512+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2177833327 shutdown_connections 2024-04-25T08:36:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.512+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2177833327 wait complete. 2024-04-25T08:36:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.513+0000 7f80c5b6a640 1 Processor -- start 2024-04-25T08:36:16.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.513+0000 7f80c5b6a640 1 -- start start 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.514+0000 7f80c5b6a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80c00b6a00 0x7f80c01503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.514+0000 7f80c5b6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80c00d0870 0x7f80c014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.514+0000 7f80c5b6a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80c01612c0 0x7f80c015e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.514+0000 7f80c5b6a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c00d2170 con 0x7f80c00d0870 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.514+0000 7f80c5b6a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c00b7d80 con 0x7f80c00b6a00 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.514+0000 7f80c5b6a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c00b6e90 con 0x7f80c01612c0 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.514+0000 7f80beffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80c00d0870 0x7f80c014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.514+0000 7f80beffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80c00d0870 0x7f80c014ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54094/0 (socket says 172.21.15.38:54094) 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.514+0000 7f80beffd640 1 -- 172.21.15.38:0/2579738234 learned_addr learned my addr 172.21.15.38:0/2579738234 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.514+0000 7f80bffff640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80c01612c0 0x7f80c015e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.514+0000 7f80bf7fe640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80c00b6a00 0x7f80c01503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.515+0000 7f80beffd640 1 -- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80c01612c0 msgr2=0x7f80c015e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:16.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.515+0000 7f80beffd640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80c01612c0 0x7f80c015e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.515+0000 7f80beffd640 1 -- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80c00b6a00 msgr2=0x7f80c01503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.515+0000 7f80beffd640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80c00b6a00 0x7f80c01503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.515+0000 7f80beffd640 1 -- 172.21.15.38:0/2579738234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c015c730 con 0x7f80c00d0870 2024-04-25T08:36:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.515+0000 7f80bf7fe640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80c00b6a00 0x7f80c01503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.515+0000 7f80beffd640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80c00d0870 0x7f80c014ec00 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f80b400e810 tx=0x7f80b400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.515+0000 7f80a7fff640 1 -- 172.21.15.38:0/2579738234 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b400f040 con 0x7f80c00d0870 2024-04-25T08:36:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.515+0000 7f80a7fff640 1 -- 172.21.15.38:0/2579738234 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b4004030 con 0x7f80c00d0870 2024-04-25T08:36:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.515+0000 7f80a7fff640 1 -- 172.21.15.38:0/2579738234 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4004400 con 0x7f80c00d0870 2024-04-25T08:36:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.515+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2579738234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c015ca10 con 0x7f80c00d0870 2024-04-25T08:36:16.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.515+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2579738234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c015ef20 con 0x7f80c00d0870 2024-04-25T08:36:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.516+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2579738234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c0068e80 con 0x7f80c00d0870 2024-04-25T08:36:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.516+0000 7f80a7fff640 1 -- 172.21.15.38:0/2579738234 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80b400c6d0 con 0x7f80c00d0870 2024-04-25T08:36:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.517+0000 7f80a7fff640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f80ac06b4d0 0x7f80ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.517+0000 7f80a7fff640 1 -- 172.21.15.38:0/2579738234 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f80b408a090 con 0x7f80c00d0870 2024-04-25T08:36:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.517+0000 7f80bf7fe640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f80ac06b4d0 0x7f80ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:16.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.518+0000 7f80bf7fe640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f80ac06b4d0 0x7f80ac06d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f80b0034000 tx=0x7f80b002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:16.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.521+0000 7f80a7fff640 1 -- 172.21.15.38:0/2579738234 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80b4017020 con 0x7f80c00d0870 2024-04-25T08:36:16.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:16.702+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2579738234 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f80c015cbe0 con 0x7f80c00d0870 2024-04-25T08:36:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.467+0000 7f80a7fff640 1 -- 172.21.15.38:0/2579738234 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7f80b4058d60 con 0x7f80c00d0870 2024-04-25T08:36:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:36:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.470+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f80ac06b4d0 msgr2=0x7f80ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.470+0000 7f80c5b6a640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f80ac06b4d0 0x7f80ac06d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f80b0034000 tx=0x7f80b002e000 comp rx=0 tx=0).stop 2024-04-25T08:36:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.470+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2579738234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80c00d0870 msgr2=0x7f80c014ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.470+0000 7f80c5b6a640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80c00d0870 0x7f80c014ec00 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f80b400e810 tx=0x7f80b400ecd0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.470+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2579738234 shutdown_connections 2024-04-25T08:36:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.470+0000 7f80c5b6a640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f80ac06b4d0 0x7f80ac06d990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.470+0000 7f80c5b6a640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80c01612c0 0x7f80c015e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.470+0000 7f80c5b6a640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f80c00d0870 0x7f80c014ec00 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.470+0000 7f80c5b6a640 1 --2- 172.21.15.38:0/2579738234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80c00b6a00 0x7f80c01503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.470+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2579738234 >> 172.21.15.38:0/2579738234 conn(0x7f80c0066d10 msgr2=0x7f80c00d22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.470+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2579738234 shutdown_connections 2024-04-25T08:36:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.470+0000 7f80c5b6a640 1 -- 172.21.15.38:0/2579738234 wait complete. 2024-04-25T08:36:17.482 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:36:17.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.599+0000 7feedbe76640 1 Processor -- start 2024-04-25T08:36:17.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.599+0000 7feedbe76640 1 -- start start 2024-04-25T08:36:17.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feedbe76640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7feed41612c0 0x7feed40b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feedbe76640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feed40b6a00 0x7feed40d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feedbe76640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed40b7200 con 0x7feed40d0870 2024-04-25T08:36:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feedbe76640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed40b74e0 con 0x7feed41612c0 2024-04-25T08:36:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feedbe76640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed40b77c0 con 0x7feed40b6a00 2024-04-25T08:36:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feed9c12640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7feed41612c0 0x7feed40b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feeda413640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7feed40d0870 0x7feed40b82a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34946/0 (socket says 172.21.15.38:34946) 2024-04-25T08:36:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feeda413640 1 -- 172.21.15.38:0/1690442619 learned_addr learned my addr 172.21.15.38:0/1690442619 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feed9c12640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7feed41612c0 0x7feed40b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54106/0 (socket says 172.21.15.38:54106) 2024-04-25T08:36:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feed9411640 1 --2- 172.21.15.38:0/1690442619 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feed40b6a00 0x7feed40d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feed9c12640 1 -- 172.21.15.38:0/1690442619 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feed40b6a00 msgr2=0x7feed40d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feed9c12640 1 --2- 172.21.15.38:0/1690442619 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feed40b6a00 0x7feed40d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feed9c12640 1 -- 172.21.15.38:0/1690442619 >> v1:172.21.15.38:6789/0 conn(0x7feed40d0870 legacy=0x7feed40b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feed9c12640 1 -- 172.21.15.38:0/1690442619 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed40b7aa0 con 0x7feed41612c0 2024-04-25T08:36:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feed8c10640 1 -- 172.21.15.38:0/1690442619 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092179742 0 0) 0x7feed40b7200 con 0x7feed40d0870 2024-04-25T08:36:17.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.600+0000 7feed9411640 1 --2- 172.21.15.38:0/1690442619 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feed40b6a00 0x7feed40d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:17.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.601+0000 7feed9c12640 1 --2- 172.21.15.38:0/1690442619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feed41612c0 0x7feed40b93f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7feed00097a0 tx=0x7feed002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98485dd78cf2ce8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:17.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.601+0000 7feed8c10640 1 -- 172.21.15.38:0/1690442619 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0037070 con 0x7feed41612c0 2024-04-25T08:36:17.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.601+0000 7feed8c10640 1 -- 172.21.15.38:0/1690442619 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed002ae50 con 0x7feed41612c0 2024-04-25T08:36:17.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.601+0000 7feed8c10640 1 -- 172.21.15.38:0/1690442619 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0032660 con 0x7feed41612c0 2024-04-25T08:36:17.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.602+0000 7feedbe76640 1 -- 172.21.15.38:0/1690442619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feed41612c0 msgr2=0x7feed40b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.602+0000 7feedbe76640 1 --2- 172.21.15.38:0/1690442619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feed41612c0 0x7feed40b93f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7feed00097a0 tx=0x7feed002a100 comp rx=0 tx=0).stop 2024-04-25T08:36:17.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.602+0000 7feedbe76640 1 -- 172.21.15.38:0/1690442619 shutdown_connections 2024-04-25T08:36:17.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.602+0000 7feedbe76640 1 --2- 172.21.15.38:0/1690442619 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feed40b6a00 0x7feed40d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.602+0000 7feedbe76640 1 --2- 172.21.15.38:0/1690442619 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feed41612c0 0x7feed40b93f0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.602+0000 7feedbe76640 1 -- 172.21.15.38:0/1690442619 >> 172.21.15.38:0/1690442619 conn(0x7feed4066d10 msgr2=0x7feed4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:17.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.602+0000 7feedbe76640 1 -- 172.21.15.38:0/1690442619 shutdown_connections 2024-04-25T08:36:17.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.602+0000 7feedbe76640 1 -- 172.21.15.38:0/1690442619 wait complete. 2024-04-25T08:36:17.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.603+0000 7feedbe76640 1 Processor -- start 2024-04-25T08:36:17.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.603+0000 7feedbe76640 1 -- start start 2024-04-25T08:36:17.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feedbe76640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed40b6a00 0x7feed415e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feedbe76640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feed40d0870 0x7feed41f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feedbe76640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feed41612c0 0x7feed40cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feedbe76640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed40b6e60 con 0x7feed40d0870 2024-04-25T08:36:17.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feedbe76640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed40d2170 con 0x7feed40b6a00 2024-04-25T08:36:17.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feedbe76640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed40b6fe0 con 0x7feed41612c0 2024-04-25T08:36:17.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feed9411640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feed40d0870 0x7feed41f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feed9411640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feed40d0870 0x7feed41f7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54116/0 (socket says 172.21.15.38:54116) 2024-04-25T08:36:17.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feed9411640 1 -- 172.21.15.38:0/580894661 learned_addr learned my addr 172.21.15.38:0/580894661 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:17.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feeda413640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feed41612c0 0x7feed40cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feed9c12640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed40b6a00 0x7feed415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feed9411640 1 -- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feed41612c0 msgr2=0x7feed40cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feed9411640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feed41612c0 0x7feed40cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feed9411640 1 -- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed40b6a00 msgr2=0x7feed415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:17.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feed9411640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed40b6a00 0x7feed415e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:17.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feed9411640 1 -- 172.21.15.38:0/580894661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed415d690 con 0x7feed40d0870 2024-04-25T08:36:17.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feeda413640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feed41612c0 0x7feed40cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:17.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feed9411640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feed40d0870 0x7feed41f7710 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7feec400c7f0 tx=0x7feec400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:17.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.604+0000 7feeca7fc640 1 -- 172.21.15.38:0/580894661 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4015070 con 0x7feed40d0870 2024-04-25T08:36:17.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.605+0000 7feeca7fc640 1 -- 172.21.15.38:0/580894661 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec4007a30 con 0x7feed40d0870 2024-04-25T08:36:17.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.605+0000 7feedbe76640 1 -- 172.21.15.38:0/580894661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed415d970 con 0x7feed40d0870 2024-04-25T08:36:17.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.605+0000 7feeca7fc640 1 -- 172.21.15.38:0/580894661 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4007e00 con 0x7feed40d0870 2024-04-25T08:36:17.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.605+0000 7feedbe76640 1 -- 172.21.15.38:0/580894661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed415df30 con 0x7feed40d0870 2024-04-25T08:36:17.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.606+0000 7feedbe76640 1 -- 172.21.15.38:0/580894661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed40cdf70 con 0x7feed40d0870 2024-04-25T08:36:17.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.606+0000 7feeca7fc640 1 -- 172.21.15.38:0/580894661 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feec4018070 con 0x7feed40d0870 2024-04-25T08:36:17.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.606+0000 7feeca7fc640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feeb806b4d0 0x7feeb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:17.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.607+0000 7feeca7fc640 1 -- 172.21.15.38:0/580894661 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5922+0+0 (secure 0 0 0) 0x7feec408a200 con 0x7feed40d0870 2024-04-25T08:36:17.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.607+0000 7feed9c12640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feeb806b4d0 0x7feeb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:17.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.607+0000 7feed9c12640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feeb806b4d0 0x7feeb806d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feed0034000 tx=0x7feed002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:17.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.611+0000 7feeca7fc640 1 -- 172.21.15.38:0/580894661 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feed40cdf70 con 0x7feed40d0870 2024-04-25T08:36:17.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:17.794+0000 7feedbe76640 1 -- 172.21.15.38:0/580894661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feed4068e80 con 0x7feed40d0870 2024-04-25T08:36:18.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.474+0000 7feeca7fc640 1 -- 172.21.15.38:0/580894661 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7feec4059030 con 0x7feed40d0870 2024-04-25T08:36:18.469 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-04-25T08:36:18.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.477+0000 7feedbe76640 1 -- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feeb806b4d0 msgr2=0x7feeb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.477+0000 7feedbe76640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feeb806b4d0 0x7feeb806d990 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feed0034000 tx=0x7feed002e000 comp rx=0 tx=0).stop 2024-04-25T08:36:18.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.477+0000 7feedbe76640 1 -- 172.21.15.38:0/580894661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feed40d0870 msgr2=0x7feed41f7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.477+0000 7feedbe76640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feed40d0870 0x7feed41f7710 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7feec400c7f0 tx=0x7feec400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.477+0000 7feedbe76640 1 -- 172.21.15.38:0/580894661 shutdown_connections 2024-04-25T08:36:18.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.477+0000 7feedbe76640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feeb806b4d0 0x7feeb806d990 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.477+0000 7feedbe76640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feed41612c0 0x7feed40cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.477+0000 7feedbe76640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feed40d0870 0x7feed41f7710 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.477+0000 7feedbe76640 1 --2- 172.21.15.38:0/580894661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feed40b6a00 0x7feed415e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.477+0000 7feedbe76640 1 -- 172.21.15.38:0/580894661 >> 172.21.15.38:0/580894661 conn(0x7feed4066d10 msgr2=0x7feed40c93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:18.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.477+0000 7feedbe76640 1 -- 172.21.15.38:0/580894661 shutdown_connections 2024-04-25T08:36:18.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.477+0000 7feedbe76640 1 -- 172.21.15.38:0/580894661 wait complete. 2024-04-25T08:36:18.489 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:36:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.608+0000 7efe354e1640 1 Processor -- start 2024-04-25T08:36:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.608+0000 7efe354e1640 1 -- start start 2024-04-25T08:36:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.609+0000 7efe354e1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7efe300d33d0 0x7efe300d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.609+0000 7efe354e1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe300d2330 con 0x7efe300d0320 2024-04-25T08:36:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.609+0000 7efe354e1640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe300d2610 con 0x7efe300d1e90 2024-04-25T08:36:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.609+0000 7efe354e1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe300d28f0 con 0x7efe300d33d0 2024-04-25T08:36:18.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.609+0000 7efe2effd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7efe300d0320 0x7efe300d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:34950/0 (socket says 172.21.15.38:34950) 2024-04-25T08:36:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.609+0000 7efe2effd640 1 -- 172.21.15.38:0/2989717214 learned_addr learned my addr 172.21.15.38:0/2989717214 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.609+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029086669 0 0) 0x7efe300d2330 con 0x7efe300d0320 2024-04-25T08:36:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.609+0000 7efe2e7fc640 1 --2- 172.21.15.38:0/2989717214 >> v2:172.21.15.107:3300/0 conn(0x7efe300d33d0 0x7efe300d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.609+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe1c003520 con 0x7efe300d0320 2024-04-25T08:36:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4113985345 0 0) 0x7efe1c003520 con 0x7efe300d0320 2024-04-25T08:36:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 >> v2:172.21.15.107:3300/0 conn(0x7efe300d33d0 msgr2=0x7efe300d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 --2- 172.21.15.38:0/2989717214 >> v2:172.21.15.107:3300/0 conn(0x7efe300d33d0 0x7efe300d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 >> v1:172.21.15.107:6789/0 conn(0x7efe300d1e90 legacy=0x7efe300b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300d2bd0 con 0x7efe300d0320 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7efe28001670 con 0x7efe300d0320 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 >> v1:172.21.15.38:6789/0 conn(0x7efe300d0320 legacy=0x7efe300d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 --2- 172.21.15.38:0/2989717214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe1c0039b0 0x7efe1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 --2- 172.21.15.38:0/2989717214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 0x7efe1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 --2- 172.21.15.38:0/2989717214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0051c0 0x7efe1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe300d28f0 con 0x7efe1c004800 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c00c070 con 0x7efe1c0051c0 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c00c270 con 0x7efe1c0039b0 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2e7fc640 1 --2- 172.21.15.38:0/2989717214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 0x7efe1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2f7fe640 1 --2- 172.21.15.38:0/2989717214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0051c0 0x7efe1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2effd640 1 --2- 172.21.15.38:0/2989717214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe1c0039b0 0x7efe1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2e7fc640 1 -- 172.21.15.38:0/2989717214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe1c0039b0 msgr2=0x7efe1c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2e7fc640 1 --2- 172.21.15.38:0/2989717214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe1c0039b0 0x7efe1c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2e7fc640 1 -- 172.21.15.38:0/2989717214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0051c0 msgr2=0x7efe1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:18.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2e7fc640 1 --2- 172.21.15.38:0/2989717214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0051c0 0x7efe1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2e7fc640 1 -- 172.21.15.38:0/2989717214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c003520 con 0x7efe1c004800 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2f7fe640 1 --2- 172.21.15.38:0/2989717214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe1c0051c0 0x7efe1c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.610+0000 7efe2f7fe640 1 -- 172.21.15.38:0/2989717214 reap_dead start 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.611+0000 7efe2e7fc640 1 --2- 172.21.15.38:0/2989717214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 0x7efe1c004c80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7efe18009ac0 tx=0x7efe1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d21bf6d40b673d4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.611+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18036070 con 0x7efe1c004800 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.611+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe18031400 con 0x7efe1c004800 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.611+0000 7efe2dffb640 1 -- 172.21.15.38:0/2989717214 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe18009690 con 0x7efe1c004800 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.612+0000 7efe354e1640 1 -- 172.21.15.38:0/2989717214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 msgr2=0x7efe1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.612+0000 7efe354e1640 1 --2- 172.21.15.38:0/2989717214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 0x7efe1c004c80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7efe18009ac0 tx=0x7efe1c005830 comp rx=0 tx=0).stop 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.612+0000 7efe354e1640 1 -- 172.21.15.38:0/2989717214 shutdown_connections 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.612+0000 7efe354e1640 1 --2- 172.21.15.38:0/2989717214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 0x7efe1c004c80 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.612+0000 7efe354e1640 1 -- 172.21.15.38:0/2989717214 >> 172.21.15.38:0/2989717214 conn(0x7efe300668f0 msgr2=0x7efe30066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.612+0000 7efe354e1640 1 -- 172.21.15.38:0/2989717214 shutdown_connections 2024-04-25T08:36:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.612+0000 7efe354e1640 1 -- 172.21.15.38:0/2989717214 wait complete. 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.613+0000 7efe354e1640 1 Processor -- start 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.613+0000 7efe354e1640 1 -- start start 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe354e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 0x7efe30156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe354e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe300cc000 0x7efe301f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe354e1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe301f7510 0x7efe30148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe354e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe30161390 con 0x7efe1c004800 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe354e1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe30161210 con 0x7efe300cc000 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe354e1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe300b9720 con 0x7efe301f7510 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe2effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 0x7efe30156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe2effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 0x7efe30156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54132/0 (socket says 172.21.15.38:54132) 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe2effd640 1 -- 172.21.15.38:0/2577955097 learned_addr learned my addr 172.21.15.38:0/2577955097 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe2effd640 1 -- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe301f7510 msgr2=0x7efe30148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe2f7fe640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe301f7510 0x7efe30148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe2effd640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe301f7510 0x7efe30148d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe2effd640 1 -- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe300cc000 msgr2=0x7efe301f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe2effd640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe300cc000 0x7efe301f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe2effd640 1 -- 172.21.15.38:0/2577955097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300b5c70 con 0x7efe1c004800 2024-04-25T08:36:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe2effd640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 0x7efe30156780 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7efe28001b80 tx=0x7efe28007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe277fe640 1 -- 172.21.15.38:0/2577955097 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2800f960 con 0x7efe1c004800 2024-04-25T08:36:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.614+0000 7efe277fe640 1 -- 172.21.15.38:0/2577955097 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe28018070 con 0x7efe1c004800 2024-04-25T08:36:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.615+0000 7efe277fe640 1 -- 172.21.15.38:0/2577955097 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2800fd20 con 0x7efe1c004800 2024-04-25T08:36:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.615+0000 7efe354e1640 1 -- 172.21.15.38:0/2577955097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe300b5f20 con 0x7efe1c004800 2024-04-25T08:36:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.615+0000 7efe354e1640 1 -- 172.21.15.38:0/2577955097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe300cb800 con 0x7efe1c004800 2024-04-25T08:36:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.615+0000 7efe354e1640 1 -- 172.21.15.38:0/2577955097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe30068e80 con 0x7efe1c004800 2024-04-25T08:36:18.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.616+0000 7efe277fe640 1 -- 172.21.15.38:0/2577955097 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe2801d460 con 0x7efe1c004800 2024-04-25T08:36:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.616+0000 7efe277fe640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efe1006b4d0 0x7efe1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.616+0000 7efe277fe640 1 -- 172.21.15.38:0/2577955097 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7efe2808a190 con 0x7efe1c004800 2024-04-25T08:36:18.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.617+0000 7efe2e7fc640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efe1006b4d0 0x7efe1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.617+0000 7efe2e7fc640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efe1006b4d0 0x7efe1006d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efe18033000 tx=0x7efe1802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.621+0000 7efe277fe640 1 -- 172.21.15.38:0/2577955097 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe280591c0 con 0x7efe1c004800 2024-04-25T08:36:18.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.803+0000 7efe354e1640 1 -- 172.21.15.38:0/2577955097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7efe300b3050 con 0x7efe1c004800 2024-04-25T08:36:18.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.806+0000 7efe277fe640 1 -- 172.21.15.38:0/2577955097 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7efe28058ac0 con 0x7efe1c004800 2024-04-25T08:36:18.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.808+0000 7efe354e1640 1 -- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efe1006b4d0 msgr2=0x7efe1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.808+0000 7efe354e1640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efe1006b4d0 0x7efe1006d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efe18033000 tx=0x7efe1802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:18.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.808+0000 7efe354e1640 1 -- 172.21.15.38:0/2577955097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 msgr2=0x7efe30156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.808+0000 7efe354e1640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 0x7efe30156780 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7efe28001b80 tx=0x7efe28007720 comp rx=0 tx=0).stop 2024-04-25T08:36:18.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.808+0000 7efe354e1640 1 -- 172.21.15.38:0/2577955097 shutdown_connections 2024-04-25T08:36:18.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.808+0000 7efe354e1640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efe1006b4d0 0x7efe1006d990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.808+0000 7efe354e1640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efe301f7510 0x7efe30148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.808+0000 7efe354e1640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe300cc000 0x7efe301f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.808+0000 7efe354e1640 1 --2- 172.21.15.38:0/2577955097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe1c004800 0x7efe30156780 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.808+0000 7efe354e1640 1 -- 172.21.15.38:0/2577955097 >> 172.21.15.38:0/2577955097 conn(0x7efe300668f0 msgr2=0x7efe300d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:18.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.808+0000 7efe354e1640 1 -- 172.21.15.38:0/2577955097 shutdown_connections 2024-04-25T08:36:18.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.808+0000 7efe354e1640 1 -- 172.21.15.38:0/2577955097 wait complete. 2024-04-25T08:36:18.814 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T08:36:18.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.921+0000 7f10ce168640 1 Processor -- start 2024-04-25T08:36:18.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.921+0000 7f10ce168640 1 -- start start 2024-04-25T08:36:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.922+0000 7f10ce168640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f10c806cff0 0x7f10c8099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.922+0000 7f10ce168640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c809a3c0 con 0x7f10c806cc10 2024-04-25T08:36:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.922+0000 7f10ce168640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c80d05c0 con 0x7f10c8099fb0 2024-04-25T08:36:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.922+0000 7f10ce168640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10c80d08a0 con 0x7f10c806cff0 2024-04-25T08:36:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.922+0000 7f10bffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f10c806cff0 0x7f10c8099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.922+0000 7f10c77fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f10c806cc10 0x7f10c80cfa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37438/0 (socket says 172.21.15.38:37438) 2024-04-25T08:36:18.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.922+0000 7f10c77fe640 1 -- 172.21.15.38:0/2819265188 learned_addr learned my addr 172.21.15.38:0/2819265188 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:18.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.922+0000 7f10bffff640 1 -- 172.21.15.38:0/2819265188 >> v1:172.21.15.107:6789/0 conn(0x7f10c8099fb0 legacy=0x7f10c815d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:18.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.922+0000 7f10bffff640 1 -- 172.21.15.38:0/2819265188 >> v1:172.21.15.38:6789/0 conn(0x7f10c806cc10 legacy=0x7f10c80cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.922+0000 7f10c6ffd640 1 -- 172.21.15.38:0/2819265188 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363608883 0 0) 0x7f10b8001110 con 0x7f10c806cc10 2024-04-25T08:36:18.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.922+0000 7f10bffff640 1 -- 172.21.15.38:0/2819265188 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c80d0b80 con 0x7f10c806cff0 2024-04-25T08:36:18.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.923+0000 7f10bffff640 1 --2- 172.21.15.38:0/2819265188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10c806cff0 0x7f10c8099a70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f10b00097a0 tx=0x7f10b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18db1110490ebb6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.923+0000 7f10c6ffd640 1 -- 172.21.15.38:0/2819265188 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0037070 con 0x7f10c806cff0 2024-04-25T08:36:18.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.923+0000 7f10c6ffd640 1 -- 172.21.15.38:0/2819265188 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b0032440 con 0x7f10c806cff0 2024-04-25T08:36:18.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.923+0000 7f10c6ffd640 1 -- 172.21.15.38:0/2819265188 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0032810 con 0x7f10c806cff0 2024-04-25T08:36:18.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.923+0000 7f10ce168640 1 -- 172.21.15.38:0/2819265188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10c806cff0 msgr2=0x7f10c8099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.923+0000 7f10ce168640 1 --2- 172.21.15.38:0/2819265188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10c806cff0 0x7f10c8099a70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f10b00097a0 tx=0x7f10b002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.923+0000 7f10ce168640 1 -- 172.21.15.38:0/2819265188 shutdown_connections 2024-04-25T08:36:18.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.923+0000 7f10ce168640 1 --2- 172.21.15.38:0/2819265188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10c806cff0 0x7f10c8099a70 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.923+0000 7f10ce168640 1 -- 172.21.15.38:0/2819265188 >> 172.21.15.38:0/2819265188 conn(0x7f10c80b31f0 msgr2=0x7f10c80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:18.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.924+0000 7f10ce168640 1 -- 172.21.15.38:0/2819265188 shutdown_connections 2024-04-25T08:36:18.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.924+0000 7f10ce168640 1 -- 172.21.15.38:0/2819265188 wait complete. 2024-04-25T08:36:18.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.925+0000 7f10ce168640 1 Processor -- start 2024-04-25T08:36:18.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.925+0000 7f10ce168640 1 -- start start 2024-04-25T08:36:18.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.925+0000 7f10ce168640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10c806cc10 0x7f10c8159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.925+0000 7f10ce168640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10c806cff0 0x7f10c80c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.925+0000 7f10ce168640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10c8099fb0 0x7f10c80c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.925+0000 7f10ce168640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c806c150 con 0x7f10c806cff0 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.925+0000 7f10ce168640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c80d1140 con 0x7f10c8099fb0 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.925+0000 7f10ce168640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c806c2d0 con 0x7f10c806cc10 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.925+0000 7f10bffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10c806cff0 0x7f10c80c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10bffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10c806cff0 0x7f10c80c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55922/0 (socket says 172.21.15.38:55922) 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10bffff640 1 -- 172.21.15.38:0/950784802 learned_addr learned my addr 172.21.15.38:0/950784802 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10bffff640 1 -- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10c806cc10 msgr2=0x7f10c8159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10c7fff640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10c8099fb0 0x7f10c80c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10c77fe640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10c806cc10 0x7f10c8159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10bffff640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10c806cc10 0x7f10c8159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10bffff640 1 -- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10c8099fb0 msgr2=0x7f10c80c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10bffff640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10c8099fb0 0x7f10c80c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10bffff640 1 -- 172.21.15.38:0/950784802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c80c8080 con 0x7f10c806cff0 2024-04-25T08:36:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10c7fff640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10c8099fb0 0x7f10c80c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10c77fe640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10c806cc10 0x7f10c8159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10bffff640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10c806cff0 0x7f10c80c5230 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f10b00097a0 tx=0x7f10b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10c4ff9640 1 -- 172.21.15.38:0/950784802 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0041070 con 0x7f10c806cff0 2024-04-25T08:36:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10c4ff9640 1 -- 172.21.15.38:0/950784802 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b003c870 con 0x7f10c806cff0 2024-04-25T08:36:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10ce168640 1 -- 172.21.15.38:0/950784802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c80b5300 con 0x7f10c806cff0 2024-04-25T08:36:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10c4ff9640 1 -- 172.21.15.38:0/950784802 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b0037040 con 0x7f10c806cff0 2024-04-25T08:36:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.926+0000 7f10ce168640 1 -- 172.21.15.38:0/950784802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c80b58c0 con 0x7f10c806cff0 2024-04-25T08:36:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.927+0000 7f10ce168640 1 -- 172.21.15.38:0/950784802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c80b6660 con 0x7f10c806cff0 2024-04-25T08:36:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.928+0000 7f10c4ff9640 1 -- 172.21.15.38:0/950784802 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f10b0044070 con 0x7f10c806cff0 2024-04-25T08:36:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.928+0000 7f10c4ff9640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f10a806b4d0 0x7f10a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.928+0000 7f10c4ff9640 1 -- 172.21.15.38:0/950784802 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f10b00aac40 con 0x7f10c806cff0 2024-04-25T08:36:18.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.928+0000 7f10c77fe640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f10a806b4d0 0x7f10a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:18.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.929+0000 7f10c77fe640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f10a806b4d0 0x7f10a806d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f10b8000f60 tx=0x7f10b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:18.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:18.932+0000 7f10c4ff9640 1 -- 172.21.15.38:0/950784802 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10c80b6660 con 0x7f10c806cff0 2024-04-25T08:36:19.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.182+0000 7f10ce168640 1 -- 172.21.15.38:0/950784802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f10c8068e80 con 0x7f10c806cff0 2024-04-25T08:36:19.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.480+0000 7f10c4ff9640 1 -- 172.21.15.38:0/950784802 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v248) v1 ==== 143+0+0 (secure 0 0 0) 0x7f10b0079c70 con 0x7f10c806cff0 2024-04-25T08:36:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.482+0000 7f10ce168640 1 -- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f10a806b4d0 msgr2=0x7f10a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.482+0000 7f10ce168640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f10a806b4d0 0x7f10a806d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f10b8000f60 tx=0x7f10b8009040 comp rx=0 tx=0).stop 2024-04-25T08:36:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.482+0000 7f10ce168640 1 -- 172.21.15.38:0/950784802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10c806cff0 msgr2=0x7f10c80c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.482+0000 7f10ce168640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10c806cff0 0x7f10c80c5230 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f10b00097a0 tx=0x7f10b00046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.482+0000 7f10ce168640 1 -- 172.21.15.38:0/950784802 shutdown_connections 2024-04-25T08:36:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.482+0000 7f10ce168640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10c8099fb0 0x7f10c80c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.482+0000 7f10ce168640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10c806cff0 0x7f10c80c5230 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.482+0000 7f10ce168640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f10a806b4d0 0x7f10a806d990 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.482+0000 7f10ce168640 1 --2- 172.21.15.38:0/950784802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10c806cc10 0x7f10c8159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.482+0000 7f10ce168640 1 -- 172.21.15.38:0/950784802 >> 172.21.15.38:0/950784802 conn(0x7f10c80b31f0 msgr2=0x7f10c80af780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.482+0000 7f10ce168640 1 -- 172.21.15.38:0/950784802 shutdown_connections 2024-04-25T08:36:19.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.482+0000 7f10ce168640 1 -- 172.21.15.38:0/950784802 wait complete. 2024-04-25T08:36:19.489 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T08:36:19.489 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-25T08:36:19.490 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T08:36:19.490 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:19.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.598+0000 7f94e9bc8640 1 Processor -- start 2024-04-25T08:36:19.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.598+0000 7f94e9bc8640 1 -- start start 2024-04-25T08:36:19.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e9bc8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94e415dc10 0x7f94e40ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e9bc8640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e40d3e70 con 0x7f94e40d0f60 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e9bc8640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e40d4150 con 0x7f94e40ced60 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e9bc8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e415dff0 con 0x7f94e415dc10 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e3fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f94e40d0f60 0x7f94e40b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37440/0 (socket says 172.21.15.38:37440) 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e3fff640 1 -- 172.21.15.38:0/3024940665 learned_addr learned my addr 172.21.15.38:0/3024940665 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e37fe640 1 --2- 172.21.15.38:0/3024940665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94e415dc10 0x7f94e40ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e27fc640 1 -- 172.21.15.38:0/3024940665 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766630461 0 0) 0x7f94e40d3e70 con 0x7f94e40d0f60 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e27fc640 1 -- 172.21.15.38:0/3024940665 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94cc003520 con 0x7f94e40d0f60 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e27fc640 1 -- 172.21.15.38:0/3024940665 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 185409826 0 0) 0x7f94e40d4150 con 0x7f94e40ced60 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e27fc640 1 -- 172.21.15.38:0/3024940665 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94e40d3e70 con 0x7f94e40ced60 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e27fc640 1 -- 172.21.15.38:0/3024940665 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3370850266 0 0) 0x7f94cc003520 con 0x7f94e40d0f60 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e37fe640 1 -- 172.21.15.38:0/3024940665 >> v1:172.21.15.107:6789/0 conn(0x7f94e40ced60 legacy=0x7f94e40d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e37fe640 1 -- 172.21.15.38:0/3024940665 >> v1:172.21.15.38:6789/0 conn(0x7f94e40d0f60 legacy=0x7f94e40b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.599+0000 7f94e37fe640 1 -- 172.21.15.38:0/3024940665 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40d4430 con 0x7f94e415dc10 2024-04-25T08:36:19.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.600+0000 7f94e37fe640 1 --2- 172.21.15.38:0/3024940665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94e415dc10 0x7f94e40ce820 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f94d40097d0 tx=0x7f94d402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fc19d472177bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:19.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.600+0000 7f94e27fc640 1 -- 172.21.15.38:0/3024940665 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4037070 con 0x7f94e415dc10 2024-04-25T08:36:19.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.600+0000 7f94e27fc640 1 -- 172.21.15.38:0/3024940665 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d402ab90 con 0x7f94e415dc10 2024-04-25T08:36:19.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.600+0000 7f94e27fc640 1 -- 172.21.15.38:0/3024940665 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0019f0 con 0x7f94e415dc10 2024-04-25T08:36:19.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.601+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3024940665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94e415dc10 msgr2=0x7f94e40ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.601+0000 7f94e9bc8640 1 --2- 172.21.15.38:0/3024940665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94e415dc10 0x7f94e40ce820 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f94d40097d0 tx=0x7f94d402a100 comp rx=0 tx=0).stop 2024-04-25T08:36:19.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.601+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3024940665 shutdown_connections 2024-04-25T08:36:19.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.601+0000 7f94e9bc8640 1 --2- 172.21.15.38:0/3024940665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94e415dc10 0x7f94e40ce820 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.601+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3024940665 >> 172.21.15.38:0/3024940665 conn(0x7f94e4066d10 msgr2=0x7f94e4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:19.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.601+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3024940665 shutdown_connections 2024-04-25T08:36:19.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.601+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3024940665 wait complete. 2024-04-25T08:36:19.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.602+0000 7f94e9bc8640 1 Processor -- start 2024-04-25T08:36:19.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.602+0000 7f94e9bc8640 1 -- start start 2024-04-25T08:36:19.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.603+0000 7f94e9bc8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e40ced60 0x7f94e415b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.603+0000 7f94e9bc8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94e40d0f60 0x7f94e40d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.603+0000 7f94e9bc8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94e415dc10 0x7f94e40d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.603+0000 7f94e9bc8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40d4710 con 0x7f94e40ced60 2024-04-25T08:36:19.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.603+0000 7f94e9bc8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40b80f0 con 0x7f94e40d0f60 2024-04-25T08:36:19.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.603+0000 7f94e9bc8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40b8270 con 0x7f94e415dc10 2024-04-25T08:36:19.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.603+0000 7f94e37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e40ced60 0x7f94e415b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.603+0000 7f94e37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e40ced60 0x7f94e415b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55926/0 (socket says 172.21.15.38:55926) 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.603+0000 7f94e37fe640 1 -- 172.21.15.38:0/3365837957 learned_addr learned my addr 172.21.15.38:0/3365837957 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.603+0000 7f94e3fff640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94e415dc10 0x7f94e40d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.603+0000 7f94e2ffd640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94e40d0f60 0x7f94e40d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.604+0000 7f94e37fe640 1 -- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94e415dc10 msgr2=0x7f94e40d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.604+0000 7f94e37fe640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94e415dc10 0x7f94e40d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.604+0000 7f94e37fe640 1 -- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94e40d0f60 msgr2=0x7f94e40d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.604+0000 7f94e37fe640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94e40d0f60 0x7f94e40d1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.604+0000 7f94e37fe640 1 -- 172.21.15.38:0/3365837957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e41f3a00 con 0x7f94e40ced60 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.604+0000 7f94e37fe640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e40ced60 0x7f94e415b530 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f94d4035040 tx=0x7f94d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.604+0000 7f94cbfff640 1 -- 172.21.15.38:0/3365837957 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4041020 con 0x7f94e40ced60 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.604+0000 7f94cbfff640 1 -- 172.21.15.38:0/3365837957 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d403d640 con 0x7f94e40ced60 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.604+0000 7f94cbfff640 1 -- 172.21.15.38:0/3365837957 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4037040 con 0x7f94e40ced60 2024-04-25T08:36:19.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.604+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3365837957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e41f3ce0 con 0x7f94e40ced60 2024-04-25T08:36:19.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.604+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3365837957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e41f36d0 con 0x7f94e40ced60 2024-04-25T08:36:19.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.605+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3365837957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e4068e80 con 0x7f94e40ced60 2024-04-25T08:36:19.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.605+0000 7f94cbfff640 1 -- 172.21.15.38:0/3365837957 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94d4044020 con 0x7f94e40ced60 2024-04-25T08:36:19.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.606+0000 7f94cbfff640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94d006b4d0 0x7f94d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.606+0000 7f94cbfff640 1 -- 172.21.15.38:0/3365837957 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f94d40aa5e0 con 0x7f94e40ced60 2024-04-25T08:36:19.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.606+0000 7f94e2ffd640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94d006b4d0 0x7f94d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.606+0000 7f94e2ffd640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94d006b4d0 0x7f94d006d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f94e40b5180 tx=0x7f94d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:19.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.610+0000 7f94cbfff640 1 -- 172.21.15.38:0/3365837957 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94d404b030 con 0x7f94e40ced60 2024-04-25T08:36:19.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.789+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3365837957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94e40b4ed0 con 0x7f94e40ced60 2024-04-25T08:36:19.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.790+0000 7f94cbfff640 1 -- 172.21.15.38:0/3365837957 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13410 (secure 0 0 0) 0x7f94d4079590 con 0x7f94e40ced60 2024-04-25T08:36:19.785 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:19.785 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":74,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:36:18.468753+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6833/1952521723":"2024-04-26T08:36:15.335208+0000","172.21.15.38:6832/1952521723":"2024-04-26T08:36:15.335208+0000","172.21.15.38:6835/239365678":"2024-04-26T08:35:52.345208+0000","172.21.15.38:6833/239365678":"2024-04-26T08:35:52.345208+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.792+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94d006b4d0 msgr2=0x7f94d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.792+0000 7f94e9bc8640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94d006b4d0 0x7f94d006d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f94e40b5180 tx=0x7f94d8009040 comp rx=0 tx=0).stop 2024-04-25T08:36:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.792+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3365837957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e40ced60 msgr2=0x7f94e415b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.792+0000 7f94e9bc8640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e40ced60 0x7f94e415b530 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f94d4035040 tx=0x7f94d40046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.792+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3365837957 shutdown_connections 2024-04-25T08:36:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.792+0000 7f94e9bc8640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94d006b4d0 0x7f94d006d990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.792+0000 7f94e9bc8640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94e415dc10 0x7f94e40d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.792+0000 7f94e9bc8640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94e40d0f60 0x7f94e40d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.792+0000 7f94e9bc8640 1 --2- 172.21.15.38:0/3365837957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94e40ced60 0x7f94e415b530 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.792+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3365837957 >> 172.21.15.38:0/3365837957 conn(0x7f94e4066d10 msgr2=0x7f94e415cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:19.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.792+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3365837957 shutdown_connections 2024-04-25T08:36:19.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.792+0000 7f94e9bc8640 1 -- 172.21.15.38:0/3365837957 wait complete. 2024-04-25T08:36:19.798 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:36:19.798 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:36:19.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.908+0000 7f872c4e1640 1 Processor -- start 2024-04-25T08:36:19.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.908+0000 7f872c4e1640 1 -- start start 2024-04-25T08:36:19.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.909+0000 7f872c4e1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f87240d33d0 0x7f87240d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.909+0000 7f872c4e1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240d2330 con 0x7f87240d0320 2024-04-25T08:36:19.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.909+0000 7f872c4e1640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240d2610 con 0x7f87240d1e90 2024-04-25T08:36:19.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.909+0000 7f872c4e1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87240d28f0 con 0x7f87240d33d0 2024-04-25T08:36:19.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.909+0000 7f8729a7c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f87240d33d0 0x7f87240d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.909+0000 7f8729a7c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f87240d33d0 0x7f87240d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55942/0 (socket says 172.21.15.38:55942) 2024-04-25T08:36:19.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.909+0000 7f872a27d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f87240d0320 0x7f87240d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37452/0 (socket says 172.21.15.38:37452) 2024-04-25T08:36:19.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.909+0000 7f8729a7c640 1 -- 172.21.15.38:0/492850872 learned_addr learned my addr 172.21.15.38:0/492850872 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:19.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.909+0000 7f8729a7c640 1 -- 172.21.15.38:0/492850872 >> v1:172.21.15.107:6789/0 conn(0x7f87240d1e90 legacy=0x7f87240b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:19.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.909+0000 7f8729a7c640 1 -- 172.21.15.38:0/492850872 >> v1:172.21.15.38:6789/0 conn(0x7f87240d0320 legacy=0x7f87240d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.909+0000 7f8729a7c640 1 -- 172.21.15.38:0/492850872 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240d2bd0 con 0x7f87240d33d0 2024-04-25T08:36:19.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.909+0000 7f872927b640 1 -- 172.21.15.38:0/492850872 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373108827 0 0) 0x7f87240d2330 con 0x7f87240d0320 2024-04-25T08:36:19.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.910+0000 7f8729a7c640 1 --2- 172.21.15.38:0/492850872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87240d33d0 0x7f87240d1950 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8714009870 tx=0x7f871402a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315c4ebb80f96aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:19.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.910+0000 7f872927b640 1 -- 172.21.15.38:0/492850872 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714037070 con 0x7f87240d33d0 2024-04-25T08:36:19.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.910+0000 7f872927b640 1 -- 172.21.15.38:0/492850872 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8714032640 con 0x7f87240d33d0 2024-04-25T08:36:19.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.910+0000 7f872927b640 1 -- 172.21.15.38:0/492850872 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714032a10 con 0x7f87240d33d0 2024-04-25T08:36:19.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.911+0000 7f872c4e1640 1 -- 172.21.15.38:0/492850872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87240d33d0 msgr2=0x7f87240d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.911+0000 7f872c4e1640 1 --2- 172.21.15.38:0/492850872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87240d33d0 0x7f87240d1950 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f8714009870 tx=0x7f871402a330 comp rx=0 tx=0).stop 2024-04-25T08:36:19.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.911+0000 7f872c4e1640 1 -- 172.21.15.38:0/492850872 shutdown_connections 2024-04-25T08:36:19.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.911+0000 7f872c4e1640 1 --2- 172.21.15.38:0/492850872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87240d33d0 0x7f87240d1950 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.911+0000 7f872c4e1640 1 -- 172.21.15.38:0/492850872 >> 172.21.15.38:0/492850872 conn(0x7f87240668f0 msgr2=0x7f8724066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:19.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.911+0000 7f872c4e1640 1 -- 172.21.15.38:0/492850872 shutdown_connections 2024-04-25T08:36:19.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.911+0000 7f872c4e1640 1 -- 172.21.15.38:0/492850872 wait complete. 2024-04-25T08:36:19.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.912+0000 7f872c4e1640 1 Processor -- start 2024-04-25T08:36:19.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.912+0000 7f872c4e1640 1 -- start start 2024-04-25T08:36:19.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872c4e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87240d0320 0x7f8724158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872c4e1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87240d1e90 0x7f87240b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872c4e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87240d33d0 0x7f87240aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872c4e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8724161390 con 0x7f87240d0320 2024-04-25T08:36:19.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872c4e1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8724161210 con 0x7f87240d33d0 2024-04-25T08:36:19.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872c4e1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87240b9720 con 0x7f87240d1e90 2024-04-25T08:36:19.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872a27d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87240d0320 0x7f8724158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872a27d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87240d0320 0x7f8724158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55950/0 (socket says 172.21.15.38:55950) 2024-04-25T08:36:19.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872a27d640 1 -- 172.21.15.38:0/3050398380 learned_addr learned my addr 172.21.15.38:0/3050398380 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:19.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f8729a7c640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87240d1e90 0x7f87240b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872a27d640 1 -- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87240d1e90 msgr2=0x7f87240b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872a27d640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87240d1e90 0x7f87240b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872a27d640 1 -- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87240d33d0 msgr2=0x7f87240aa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:19.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872a27d640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87240d33d0 0x7f87240aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:19.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872a27d640 1 -- 172.21.15.38:0/3050398380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240b3930 con 0x7f87240d0320 2024-04-25T08:36:19.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872a27d640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87240d0320 0x7f8724158920 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f872000c4d0 tx=0x7f872000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:19.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f871affd640 1 -- 172.21.15.38:0/3050398380 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720018070 con 0x7f87240d0320 2024-04-25T08:36:19.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f871affd640 1 -- 172.21.15.38:0/3050398380 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8720007500 con 0x7f87240d0320 2024-04-25T08:36:19.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f872c4e1640 1 -- 172.21.15.38:0/3050398380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87240b3c10 con 0x7f87240d0320 2024-04-25T08:36:19.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.913+0000 7f871affd640 1 -- 172.21.15.38:0/3050398380 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720004a20 con 0x7f87240d0320 2024-04-25T08:36:19.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.914+0000 7f872c4e1640 1 -- 172.21.15.38:0/3050398380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87240b41d0 con 0x7f87240d0320 2024-04-25T08:36:19.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.914+0000 7f872c4e1640 1 -- 172.21.15.38:0/3050398380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8724068e80 con 0x7f87240d0320 2024-04-25T08:36:19.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.915+0000 7f871affd640 1 -- 172.21.15.38:0/3050398380 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f872001b070 con 0x7f87240d0320 2024-04-25T08:36:19.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.915+0000 7f871affd640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f870806b4d0 0x7f870806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:19.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.915+0000 7f871affd640 1 -- 172.21.15.38:0/3050398380 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8720089300 con 0x7f87240d0320 2024-04-25T08:36:19.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.915+0000 7f8729a7c640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f870806b4d0 0x7f870806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:19.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.916+0000 7f8729a7c640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f870806b4d0 0x7f870806d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8714034000 tx=0x7f871402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:19.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:19.919+0000 7f871affd640 1 -- 172.21.15.38:0/3050398380 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8720058330 con 0x7f87240d0320 2024-04-25T08:36:20.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.100+0000 7f872c4e1640 1 -- 172.21.15.38:0/3050398380 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f87240b3050 con 0x7f87240d0320 2024-04-25T08:36:20.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.101+0000 7f871affd640 1 -- 172.21.15.38:0/3050398380 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8720057c30 con 0x7f87240d0320 2024-04-25T08:36:20.095 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:36:20.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.103+0000 7f872c4e1640 1 -- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f870806b4d0 msgr2=0x7f870806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.103+0000 7f872c4e1640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f870806b4d0 0x7f870806d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8714034000 tx=0x7f871402e000 comp rx=0 tx=0).stop 2024-04-25T08:36:20.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.103+0000 7f872c4e1640 1 -- 172.21.15.38:0/3050398380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87240d0320 msgr2=0x7f8724158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.103+0000 7f872c4e1640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87240d0320 0x7f8724158920 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f872000c4d0 tx=0x7f872000c990 comp rx=0 tx=0).stop 2024-04-25T08:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.103+0000 7f872c4e1640 1 -- 172.21.15.38:0/3050398380 shutdown_connections 2024-04-25T08:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.103+0000 7f872c4e1640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87240d33d0 0x7f87240aa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.103+0000 7f872c4e1640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f870806b4d0 0x7f870806d990 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.103+0000 7f872c4e1640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f87240d1e90 0x7f87240b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.103+0000 7f872c4e1640 1 --2- 172.21.15.38:0/3050398380 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87240d0320 0x7f8724158920 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.103+0000 7f872c4e1640 1 -- 172.21.15.38:0/3050398380 >> 172.21.15.38:0/3050398380 conn(0x7f87240668f0 msgr2=0x7f87240d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.103+0000 7f872c4e1640 1 -- 172.21.15.38:0/3050398380 shutdown_connections 2024-04-25T08:36:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.103+0000 7f872c4e1640 1 -- 172.21.15.38:0/3050398380 wait complete. 2024-04-25T08:36:20.108 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T08:36:20.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.215+0000 7f74918a3640 1 Processor -- start 2024-04-25T08:36:20.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.215+0000 7f74918a3640 1 -- start start 2024-04-25T08:36:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.216+0000 7f74918a3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f748c06cc10 0x7f748c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.216+0000 7f74918a3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f748c09a180 0x7f748c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.216+0000 7f74918a3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c0d05c0 con 0x7f748c06cff0 2024-04-25T08:36:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.216+0000 7f74918a3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c0d08a0 con 0x7f748c09a180 2024-04-25T08:36:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.216+0000 7f74918a3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c0d0b80 con 0x7f748c06cc10 2024-04-25T08:36:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.216+0000 7f748b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f748c09a180 0x7f748c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.216+0000 7f748b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f748c09a180 0x7f748c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55960/0 (socket says 172.21.15.38:55960) 2024-04-25T08:36:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.216+0000 7f748b7fe640 1 -- 172.21.15.38:0/672573843 learned_addr learned my addr 172.21.15.38:0/672573843 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.217+0000 7f748affd640 1 --2- 172.21.15.38:0/672573843 >> v2:172.21.15.107:3300/0 conn(0x7f748c06cc10 0x7f748c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.217+0000 7f748b7fe640 1 -- 172.21.15.38:0/672573843 >> v2:172.21.15.107:3300/0 conn(0x7f748c06cc10 msgr2=0x7f748c0cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.217+0000 7f748b7fe640 1 --2- 172.21.15.38:0/672573843 >> v2:172.21.15.107:3300/0 conn(0x7f748c06cc10 0x7f748c0cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.217+0000 7f748b7fe640 1 -- 172.21.15.38:0/672573843 >> v1:172.21.15.38:6789/0 conn(0x7f748c06cff0 legacy=0x7f748c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.217+0000 7f748b7fe640 1 -- 172.21.15.38:0/672573843 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0d0e60 con 0x7f748c09a180 2024-04-25T08:36:20.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.217+0000 7f7489ffb640 1 -- 172.21.15.38:0/672573843 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340501576 0 0) 0x7f748c0d05c0 con 0x7f748c06cff0 2024-04-25T08:36:20.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.217+0000 7f748b7fe640 1 --2- 172.21.15.38:0/672573843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f748c09a180 0x7f748c0cffb0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f748c067970 tx=0x7f747c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e653a05c5f3f2cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:20.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.218+0000 7f7489ffb640 1 -- 172.21.15.38:0/672573843 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c037070 con 0x7f748c09a180 2024-04-25T08:36:20.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.218+0000 7f7489ffb640 1 -- 172.21.15.38:0/672573843 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c032440 con 0x7f748c09a180 2024-04-25T08:36:20.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.218+0000 7f7489ffb640 1 -- 172.21.15.38:0/672573843 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c032810 con 0x7f748c09a180 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.218+0000 7f74918a3640 1 -- 172.21.15.38:0/672573843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f748c09a180 msgr2=0x7f748c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.218+0000 7f74918a3640 1 --2- 172.21.15.38:0/672573843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f748c09a180 0x7f748c0cffb0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f748c067970 tx=0x7f747c02a270 comp rx=0 tx=0).stop 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.218+0000 7f74918a3640 1 -- 172.21.15.38:0/672573843 shutdown_connections 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.218+0000 7f74918a3640 1 --2- 172.21.15.38:0/672573843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f748c09a180 0x7f748c0cffb0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.218+0000 7f74918a3640 1 --2- 172.21.15.38:0/672573843 >> v2:172.21.15.107:3300/0 conn(0x7f748c06cc10 0x7f748c0cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.218+0000 7f74918a3640 1 -- 172.21.15.38:0/672573843 >> 172.21.15.38:0/672573843 conn(0x7f748c0b31f0 msgr2=0x7f748c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.218+0000 7f74918a3640 1 -- 172.21.15.38:0/672573843 shutdown_connections 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.218+0000 7f74918a3640 1 -- 172.21.15.38:0/672573843 wait complete. 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.219+0000 7f74918a3640 1 Processor -- start 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.220+0000 7f74918a3640 1 -- start start 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.220+0000 7f74918a3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f748c06cc10 0x7f748c0c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.220+0000 7f74918a3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f748c06cff0 0x7f748c0b5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.220+0000 7f74918a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f748c09a180 0x7f748c0b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.220+0000 7f74918a3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c06c2d0 con 0x7f748c09a180 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.220+0000 7f74918a3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c06c150 con 0x7f748c06cc10 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.220+0000 7f74918a3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c15d370 con 0x7f748c06cff0 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.220+0000 7f748b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f748c09a180 0x7f748c0b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.220+0000 7f748b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f748c09a180 0x7f748c0b5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55962/0 (socket says 172.21.15.38:55962) 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.220+0000 7f748b7fe640 1 -- 172.21.15.38:0/3257942084 learned_addr learned my addr 172.21.15.38:0/3257942084 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.221+0000 7f748b7fe640 1 -- 172.21.15.38:0/3257942084 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f748c06cff0 msgr2=0x7f748c0b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.221+0000 7f748b7fe640 1 --2- 172.21.15.38:0/3257942084 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f748c06cff0 0x7f748c0b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.221+0000 7f748b7fe640 1 -- 172.21.15.38:0/3257942084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f748c06cc10 msgr2=0x7f748c0c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.221+0000 7f748b7fe640 1 --2- 172.21.15.38:0/3257942084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f748c06cc10 0x7f748c0c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.221+0000 7f748b7fe640 1 -- 172.21.15.38:0/3257942084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0c5340 con 0x7f748c09a180 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.221+0000 7f748b7fe640 1 --2- 172.21.15.38:0/3257942084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f748c09a180 0x7f748c0b5670 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f747c02a2b0 tx=0x7f747c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:20.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.221+0000 7f7483fff640 1 -- 172.21.15.38:0/3257942084 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c037040 con 0x7f748c09a180 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.221+0000 7f7483fff640 1 -- 172.21.15.38:0/3257942084 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c004030 con 0x7f748c09a180 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.221+0000 7f74918a3640 1 -- 172.21.15.38:0/3257942084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c0c5620 con 0x7f748c09a180 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.221+0000 7f7483fff640 1 -- 172.21.15.38:0/3257942084 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c004400 con 0x7f748c09a180 2024-04-25T08:36:20.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.221+0000 7f74918a3640 1 -- 172.21.15.38:0/3257942084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c0c5be0 con 0x7f748c09a180 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.222+0000 7f74918a3640 1 -- 172.21.15.38:0/3257942084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c068e80 con 0x7f748c09a180 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.223+0000 7f7483fff640 1 -- 172.21.15.38:0/3257942084 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f747c03a070 con 0x7f748c09a180 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.223+0000 7f7483fff640 1 --2- 172.21.15.38:0/3257942084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f746c06b4d0 0x7f746c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.223+0000 7f7483fff640 1 -- 172.21.15.38:0/3257942084 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f747c0aa790 con 0x7f748c09a180 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.223+0000 7f748affd640 1 --2- 172.21.15.38:0/3257942084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f746c06b4d0 0x7f746c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:20.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.224+0000 7f748affd640 1 --2- 172.21.15.38:0/3257942084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f746c06b4d0 0x7f746c06d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f74840070b0 tx=0x7f7484007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.227+0000 7f7483fff640 1 -- 172.21.15.38:0/3257942084 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f747c041020 con 0x7f748c09a180 2024-04-25T08:36:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:20.479+0000 7f74918a3640 1 -- 172.21.15.38:0/3257942084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f748c06d450 con 0x7f748c09a180 2024-04-25T08:36:21.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.492+0000 7f7483fff640 1 -- 172.21.15.38:0/3257942084 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v250) v1 ==== 161+0+0 (secure 0 0 0) 0x7f747c0797c0 con 0x7f748c09a180 2024-04-25T08:36:21.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.495+0000 7f74918a3640 1 -- 172.21.15.38:0/3257942084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f746c06b4d0 msgr2=0x7f746c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.495+0000 7f74918a3640 1 --2- 172.21.15.38:0/3257942084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f746c06b4d0 0x7f746c06d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f74840070b0 tx=0x7f7484007040 comp rx=0 tx=0).stop 2024-04-25T08:36:21.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.495+0000 7f74918a3640 1 -- 172.21.15.38:0/3257942084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f748c09a180 msgr2=0x7f748c0b5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.495+0000 7f74918a3640 1 --2- 172.21.15.38:0/3257942084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f748c09a180 0x7f748c0b5670 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f747c02a2b0 tx=0x7f747c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.495+0000 7f74918a3640 1 -- 172.21.15.38:0/3257942084 shutdown_connections 2024-04-25T08:36:21.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.495+0000 7f74918a3640 1 --2- 172.21.15.38:0/3257942084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f748c09a180 0x7f748c0b5670 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.495+0000 7f74918a3640 1 --2- 172.21.15.38:0/3257942084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f746c06b4d0 0x7f746c06d990 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.495+0000 7f74918a3640 1 --2- 172.21.15.38:0/3257942084 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f748c06cff0 0x7f748c0b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.495+0000 7f74918a3640 1 --2- 172.21.15.38:0/3257942084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f748c06cc10 0x7f748c0c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.495+0000 7f74918a3640 1 -- 172.21.15.38:0/3257942084 >> 172.21.15.38:0/3257942084 conn(0x7f748c0b31f0 msgr2=0x7f748c0cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:21.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.496+0000 7f74918a3640 1 -- 172.21.15.38:0/3257942084 shutdown_connections 2024-04-25T08:36:21.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.496+0000 7f74918a3640 1 -- 172.21.15.38:0/3257942084 wait complete. 2024-04-25T08:36:21.504 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:36:21.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.620+0000 7fab6eecd640 1 Processor -- start 2024-04-25T08:36:21.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.620+0000 7fab6eecd640 1 -- start start 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.620+0000 7fab6eecd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab6815df30 0x7fab6809a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.620+0000 7fab6eecd640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6815f300 con 0x7fab6809a180 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.620+0000 7fab6eecd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab681627f0 con 0x7fab68162390 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.620+0000 7fab6eecd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6815e310 con 0x7fab6815df30 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab6d46a640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fab6809a180 0x7fab680cf8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37458/0 (socket says 172.21.15.38:37458) 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab6d46a640 1 -- 172.21.15.38:0/2664797383 learned_addr learned my addr 172.21.15.38:0/2664797383 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab6cc69640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab6815df30 0x7fab6809a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227131439 0 0) 0x7fab6815f300 con 0x7fab6809a180 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab58003520 con 0x7fab6809a180 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3417945382 0 0) 0x7fab58003520 con 0x7fab6809a180 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab6815df30 msgr2=0x7fab6809a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab6815df30 0x7fab6809a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 >> v1:172.21.15.107:6789/0 conn(0x7fab68162390 legacy=0x7fab68099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6815e540 con 0x7fab6809a180 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fab600019f0 con 0x7fab6809a180 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 >> v1:172.21.15.38:6789/0 conn(0x7fab6809a180 legacy=0x7fab680cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab580039e0 0x7fab58003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab6815df30 0x7fab58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab58004d40 0x7fab580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5800ba70 con 0x7fab58004d40 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5800bbf0 con 0x7fab580039e0 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5800bdf0 con 0x7fab6815df30 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab6d46a640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab58004d40 0x7fab580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab6cc69640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab580039e0 0x7fab58003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.621+0000 7fab67fff640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab6815df30 0x7fab58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.622+0000 7fab6d46a640 1 -- 172.21.15.38:0/2664797383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab6815df30 msgr2=0x7fab58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.622+0000 7fab6d46a640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab6815df30 0x7fab58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.622+0000 7fab6d46a640 1 -- 172.21.15.38:0/2664797383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab580039e0 msgr2=0x7fab58003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:21.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.622+0000 7fab6d46a640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab580039e0 0x7fab58003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.622+0000 7fab6d46a640 1 -- 172.21.15.38:0/2664797383 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab58003520 con 0x7fab58004d40 2024-04-25T08:36:21.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.622+0000 7fab67fff640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab6815df30 0x7fab58004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:21.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.622+0000 7fab6d46a640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab58004d40 0x7fab580051c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fab60002230 tx=0x7fab58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63b27b2526c4727 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:21.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.623+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6000a4a0 con 0x7fab58004d40 2024-04-25T08:36:21.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.623+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6002f070 con 0x7fab58004d40 2024-04-25T08:36:21.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.623+0000 7fab677fe640 1 -- 172.21.15.38:0/2664797383 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6002bc70 con 0x7fab58004d40 2024-04-25T08:36:21.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.623+0000 7fab6eecd640 1 -- 172.21.15.38:0/2664797383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab58004d40 msgr2=0x7fab580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.623+0000 7fab6eecd640 1 --2- 172.21.15.38:0/2664797383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab58004d40 0x7fab580051c0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fab60002230 tx=0x7fab58005830 comp rx=0 tx=0).stop 2024-04-25T08:36:21.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.623+0000 7fab6d46a640 1 -- 172.21.15.38:0/2664797383 reap_dead start 2024-04-25T08:36:21.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.623+0000 7fab6eecd640 1 -- 172.21.15.38:0/2664797383 shutdown_connections 2024-04-25T08:36:21.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.623+0000 7fab6eecd640 1 -- 172.21.15.38:0/2664797383 >> 172.21.15.38:0/2664797383 conn(0x7fab680b3110 msgr2=0x7fab680b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:21.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.623+0000 7fab6eecd640 1 -- 172.21.15.38:0/2664797383 shutdown_connections 2024-04-25T08:36:21.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.623+0000 7fab6eecd640 1 -- 172.21.15.38:0/2664797383 wait complete. 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.624+0000 7fab6eecd640 1 Processor -- start 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.624+0000 7fab6eecd640 1 -- start start 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.625+0000 7fab6eecd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab58004d40 0x7fab68157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.625+0000 7fab6eecd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab681592a0 0x7fab680cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.625+0000 7fab6eecd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab68153940 0x7fab680c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.625+0000 7fab6eecd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6815ee70 con 0x7fab58004d40 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.625+0000 7fab6eecd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6809aeb0 con 0x7fab681592a0 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.625+0000 7fab6eecd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6815eb00 con 0x7fab68153940 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.625+0000 7fab6cc69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab58004d40 0x7fab68157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.625+0000 7fab6cc69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab58004d40 0x7fab68157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55966/0 (socket says 172.21.15.38:55966) 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab6cc69640 1 -- 172.21.15.38:0/3409606720 learned_addr learned my addr 172.21.15.38:0/3409606720 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab6d46a640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab68153940 0x7fab680c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab6cc69640 1 -- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab68153940 msgr2=0x7fab680c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab6cc69640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab68153940 0x7fab680c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab6cc69640 1 -- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab681592a0 msgr2=0x7fab680cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab67fff640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab681592a0 0x7fab680cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab6cc69640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab681592a0 0x7fab680cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab6cc69640 1 -- 172.21.15.38:0/3409606720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab68158200 con 0x7fab58004d40 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab6cc69640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab58004d40 0x7fab68157b70 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0118e0 tx=0x7fab5c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab657fa640 1 -- 172.21.15.38:0/3409606720 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c00faf0 con 0x7fab58004d40 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab657fa640 1 -- 172.21.15.38:0/3409606720 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c00fc90 con 0x7fab58004d40 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab6eecd640 1 -- 172.21.15.38:0/3409606720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab681584e0 con 0x7fab58004d40 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab657fa640 1 -- 172.21.15.38:0/3409606720 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c014440 con 0x7fab58004d40 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.626+0000 7fab6eecd640 1 -- 172.21.15.38:0/3409606720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab68158aa0 con 0x7fab58004d40 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.627+0000 7fab6eecd640 1 -- 172.21.15.38:0/3409606720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab68068e80 con 0x7fab58004d40 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.628+0000 7fab657fa640 1 -- 172.21.15.38:0/3409606720 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab5c012040 con 0x7fab58004d40 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.628+0000 7fab657fa640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fab4c06b500 0x7fab4c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:21.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.628+0000 7fab657fa640 1 -- 172.21.15.38:0/3409606720 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fab5c08a7c0 con 0x7fab58004d40 2024-04-25T08:36:21.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.628+0000 7fab67fff640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fab4c06b500 0x7fab4c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:21.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.629+0000 7fab67fff640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fab4c06b500 0x7fab4c06d9c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fab680b6ed0 tx=0x7fab54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:21.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.632+0000 7fab657fa640 1 -- 172.21.15.38:0/3409606720 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab5c019030 con 0x7fab58004d40 2024-04-25T08:36:21.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.812+0000 7fab6eecd640 1 -- 172.21.15.38:0/3409606720 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fab680b3de0 con 0x7fab58004d40 2024-04-25T08:36:21.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.819+0000 7fab657fa640 1 -- 172.21.15.38:0/3409606720 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fab5c059110 con 0x7fab58004d40 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.821+0000 7fab6eecd640 1 -- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fab4c06b500 msgr2=0x7fab4c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.821+0000 7fab6eecd640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fab4c06b500 0x7fab4c06d9c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fab680b6ed0 tx=0x7fab54009040 comp rx=0 tx=0).stop 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.821+0000 7fab6eecd640 1 -- 172.21.15.38:0/3409606720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab58004d40 msgr2=0x7fab68157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.821+0000 7fab6eecd640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab58004d40 0x7fab68157b70 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fab5c0118e0 tx=0x7fab5c011da0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.821+0000 7fab6eecd640 1 -- 172.21.15.38:0/3409606720 shutdown_connections 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.821+0000 7fab6eecd640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fab4c06b500 0x7fab4c06d9c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.821+0000 7fab6eecd640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab68153940 0x7fab680c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.821+0000 7fab6eecd640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab681592a0 0x7fab680cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.821+0000 7fab6eecd640 1 --2- 172.21.15.38:0/3409606720 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab58004d40 0x7fab68157b70 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:21.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.821+0000 7fab6eecd640 1 -- 172.21.15.38:0/3409606720 >> 172.21.15.38:0/3409606720 conn(0x7fab680b3110 msgr2=0x7fab6806f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:21.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.821+0000 7fab6eecd640 1 -- 172.21.15.38:0/3409606720 shutdown_connections 2024-04-25T08:36:21.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:21.822+0000 7fab6eecd640 1 -- 172.21.15.38:0/3409606720 wait complete. 2024-04-25T08:36:21.827 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:36:21.827 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:36:21.871 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:36:21.914 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:36:21.914 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:36:21.914 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:21.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:36:21.930 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:36:21.930 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:21.972 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:36:21.973 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:36:21.973 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:21.988 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:36:21.988 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:36:21.988 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:22.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:36:22.031 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:36:22.031 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.137+0000 7fd7a742c640 1 Processor -- start 2024-04-25T08:36:22.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.137+0000 7fd7a742c640 1 -- start start 2024-04-25T08:36:22.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.138+0000 7fd7a742c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd7a006cc10 0x7fd7a00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.138+0000 7fd7a742c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a00d05c0 con 0x7fd7a006cff0 2024-04-25T08:36:22.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.138+0000 7fd7a742c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a00d08a0 con 0x7fd7a009a180 2024-04-25T08:36:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.138+0000 7fd7a742c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a00d0b80 con 0x7fd7a006cc10 2024-04-25T08:36:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.138+0000 7fd7a51c8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd7a006cc10 0x7fd7a00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.138+0000 7fd7a49c7640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd7a006cff0 0x7fd7a0099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37470/0 (socket says 172.21.15.38:37470) 2024-04-25T08:36:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.138+0000 7fd7a49c7640 1 -- 172.21.15.38:0/3964231032 learned_addr learned my addr 172.21.15.38:0/3964231032 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.139+0000 7fd7a51c8640 1 -- 172.21.15.38:0/3964231032 >> v1:172.21.15.107:6789/0 conn(0x7fd7a009a180 legacy=0x7fd7a015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.139+0000 7fd7a51c8640 1 -- 172.21.15.38:0/3964231032 >> v1:172.21.15.38:6789/0 conn(0x7fd7a006cff0 legacy=0x7fd7a0099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.139+0000 7fd7a51c8640 1 -- 172.21.15.38:0/3964231032 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a00d0e60 con 0x7fd7a006cc10 2024-04-25T08:36:22.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.139+0000 7fd7a51c8640 1 --2- 172.21.15.38:0/3964231032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a006cc10 0x7fd7a00cfa70 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd7980097a0 tx=0x7fd79802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebb99eac1ffdb3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.140+0000 7fd79ffff640 1 -- 172.21.15.38:0/3964231032 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798037070 con 0x7fd7a006cc10 2024-04-25T08:36:22.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.140+0000 7fd79ffff640 1 -- 172.21.15.38:0/3964231032 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd798032440 con 0x7fd7a006cc10 2024-04-25T08:36:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.140+0000 7fd79ffff640 1 -- 172.21.15.38:0/3964231032 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798032810 con 0x7fd7a006cc10 2024-04-25T08:36:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.140+0000 7fd7a742c640 1 -- 172.21.15.38:0/3964231032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a006cc10 msgr2=0x7fd7a00cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.140+0000 7fd7a742c640 1 --2- 172.21.15.38:0/3964231032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a006cc10 0x7fd7a00cfa70 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd7980097a0 tx=0x7fd79802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.140+0000 7fd7a742c640 1 -- 172.21.15.38:0/3964231032 shutdown_connections 2024-04-25T08:36:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.140+0000 7fd7a742c640 1 --2- 172.21.15.38:0/3964231032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a006cc10 0x7fd7a00cfa70 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.140+0000 7fd7a742c640 1 -- 172.21.15.38:0/3964231032 >> 172.21.15.38:0/3964231032 conn(0x7fd7a00b31f0 msgr2=0x7fd7a00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.140+0000 7fd7a742c640 1 -- 172.21.15.38:0/3964231032 shutdown_connections 2024-04-25T08:36:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.140+0000 7fd7a742c640 1 -- 172.21.15.38:0/3964231032 wait complete. 2024-04-25T08:36:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.141+0000 7fd7a742c640 1 Processor -- start 2024-04-25T08:36:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.141+0000 7fd7a742c640 1 -- start start 2024-04-25T08:36:22.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.142+0000 7fd7a742c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a006cc10 0x7fd7a00caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.142+0000 7fd7a742c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7a006cff0 0x7fd7a00b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.142+0000 7fd7a742c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a009a180 0x7fd7a00b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.142+0000 7fd7a742c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a006c2d0 con 0x7fd7a009a180 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.142+0000 7fd7a742c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a006c150 con 0x7fd7a006cc10 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.142+0000 7fd7a742c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a00ccd60 con 0x7fd7a006cff0 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.142+0000 7fd7a59c9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a009a180 0x7fd7a00b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.142+0000 7fd7a59c9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a009a180 0x7fd7a00b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55974/0 (socket says 172.21.15.38:55974) 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.142+0000 7fd7a59c9640 1 -- 172.21.15.38:0/3985362957 learned_addr learned my addr 172.21.15.38:0/3985362957 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd7a51c8640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a006cc10 0x7fd7a00caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd7a49c7640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7a006cff0 0x7fd7a00b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd7a59c9640 1 -- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7a006cff0 msgr2=0x7fd7a00b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd7a59c9640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7a006cff0 0x7fd7a00b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd7a59c9640 1 -- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a006cc10 msgr2=0x7fd7a00caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd7a59c9640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a006cc10 0x7fd7a00caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd7a59c9640 1 -- 172.21.15.38:0/3985362957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a00b4970 con 0x7fd7a009a180 2024-04-25T08:36:22.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd7a51c8640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a006cc10 0x7fd7a00caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:22.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd7a59c9640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a009a180 0x7fd7a00b4240 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd79000b970 tx=0x7fd79000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd79dffb640 1 -- 172.21.15.38:0/3985362957 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd790019070 con 0x7fd7a009a180 2024-04-25T08:36:22.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd79dffb640 1 -- 172.21.15.38:0/3985362957 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79000cae0 con 0x7fd7a009a180 2024-04-25T08:36:22.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd7a742c640 1 -- 172.21.15.38:0/3985362957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a00b4c50 con 0x7fd7a009a180 2024-04-25T08:36:22.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd79dffb640 1 -- 172.21.15.38:0/3985362957 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd790014440 con 0x7fd7a009a180 2024-04-25T08:36:22.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.143+0000 7fd7a742c640 1 -- 172.21.15.38:0/3985362957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a00b5210 con 0x7fd7a009a180 2024-04-25T08:36:22.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.144+0000 7fd7a742c640 1 -- 172.21.15.38:0/3985362957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a0068e80 con 0x7fd7a009a180 2024-04-25T08:36:22.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.144+0000 7fd79dffb640 1 -- 172.21.15.38:0/3985362957 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd79001c070 con 0x7fd7a009a180 2024-04-25T08:36:22.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.145+0000 7fd79dffb640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd78c06b4d0 0x7fd78c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.145+0000 7fd79dffb640 1 -- 172.21.15.38:0/3985362957 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd790089bf0 con 0x7fd7a009a180 2024-04-25T08:36:22.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.145+0000 7fd7a51c8640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd78c06b4d0 0x7fd78c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.145+0000 7fd7a51c8640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd78c06b4d0 0x7fd78c06d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd798034000 tx=0x7fd79802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.149+0000 7fd79dffb640 1 -- 172.21.15.38:0/3985362957 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd790058c20 con 0x7fd7a009a180 2024-04-25T08:36:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.366+0000 7fd7a742c640 1 -- 172.21.15.38:0/3985362957 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7a006d450 con 0x7fd7a009a180 2024-04-25T08:36:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.366+0000 7fd79dffb640 1 -- 172.21.15.38:0/3985362957 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fd790058520 con 0x7fd7a009a180 2024-04-25T08:36:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:22.361 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":44,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5927,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/1366115592","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":1366115592},{"type":"v1","addr":"172.21.15.107:6837","nonce":1366115592}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6379,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/397891417","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":397891417},{"type":"v1","addr":"172.21.15.107:6835","nonce":397891417}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":7089,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1355647168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1355647168},{"type":"v1","addr":"172.21.15.38:6835","nonce":1355647168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":40},{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3449876654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3449876654},{"type":"v1","addr":"172.21.15.38:6833","nonce":3449876654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":44}],"filesystems":[]} 2024-04-25T08:36:22.361 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 44 2024-04-25T08:36:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.368+0000 7fd7a742c640 1 -- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd78c06b4d0 msgr2=0x7fd78c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.368+0000 7fd7a742c640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd78c06b4d0 0x7fd78c06d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd798034000 tx=0x7fd79802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.368+0000 7fd7a742c640 1 -- 172.21.15.38:0/3985362957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a009a180 msgr2=0x7fd7a00b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.368+0000 7fd7a742c640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a009a180 0x7fd7a00b4240 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd79000b970 tx=0x7fd79000be30 comp rx=0 tx=0).stop 2024-04-25T08:36:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.368+0000 7fd7a742c640 1 -- 172.21.15.38:0/3985362957 shutdown_connections 2024-04-25T08:36:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.368+0000 7fd7a742c640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a009a180 0x7fd7a00b4240 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.368+0000 7fd7a742c640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd78c06b4d0 0x7fd78c06d990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.368+0000 7fd7a742c640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd7a006cff0 0x7fd7a00b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.368+0000 7fd7a742c640 1 --2- 172.21.15.38:0/3985362957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd7a006cc10 0x7fd7a00caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.368+0000 7fd7a742c640 1 -- 172.21.15.38:0/3985362957 >> 172.21.15.38:0/3985362957 conn(0x7fd7a00b31f0 msgr2=0x7fd7a0160f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.368+0000 7fd7a742c640 1 -- 172.21.15.38:0/3985362957 shutdown_connections 2024-04-25T08:36:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.368+0000 7fd7a742c640 1 -- 172.21.15.38:0/3985362957 wait complete. 2024-04-25T08:36:22.374 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:36:22.375 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:36:22.375 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:36:22.375 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:36:22.376 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:36:22.376 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:36:22.376 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:36:22.376 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:36:22.376 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:36:22.377 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:36:22.377 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:36:22.377 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:36:22.377 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:36:22.378 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:36:22.378 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:36:22.378 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:36:22.389 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:36:22.389 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:36:22.391 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:36:22.391 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:36:22.392 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:36:22.393 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:36:22.393 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:36:22.393 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:36:22.432 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:36:22.432 DEBUG:teuthology.parallel:result is None 2024-04-25T08:36:22.433 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:36:22.433 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:36:22.433 DEBUG:teuthology.parallel:result is None 2024-04-25T08:36:22.433 DEBUG:teuthology.parallel:result is None 2024-04-25T08:36:22.434 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:36:22.434 DEBUG:teuthology.parallel:result is None 2024-04-25T08:36:22.435 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:36:22.507 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:36:22.507 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:36:22.511 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-25T08:36:22.511 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:36:22.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.548+0000 7fc559fc1640 1 Processor -- start 2024-04-25T08:36:22.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.548+0000 7fc559fc1640 1 -- start start 2024-04-25T08:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.548+0000 7fc559fc1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc55415a780 0x7fc5540cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.548+0000 7fc559fc1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc5540cd670 0x7fc5540b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.548+0000 7fc559fc1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5540b3870 0x7fc5540ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.548+0000 7fc559fc1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5540cde90 con 0x7fc5540b3870 2024-04-25T08:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.548+0000 7fc559fc1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5540cb190 con 0x7fc5540cd670 2024-04-25T08:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.548+0000 7fc559fc1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540cb470 con 0x7fc55415a780 2024-04-25T08:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.549+0000 7fc553fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5540b3870 0x7fc5540ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.549+0000 7fc553fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5540b3870 0x7fc5540ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56018/0 (socket says 172.21.15.38:56018) 2024-04-25T08:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.549+0000 7fc553fff640 1 -- 172.21.15.38:0/2886290344 learned_addr learned my addr 172.21.15.38:0/2886290344 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.549+0000 7fc5537fe640 1 --2- 172.21.15.38:0/2886290344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc55415a780 0x7fc5540cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.549+0000 7fc552ffd640 1 --2- 172.21.15.38:0/2886290344 >> v2:172.21.15.107:3300/0 conn(0x7fc5540cd670 0x7fc5540b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.549+0000 7fc553fff640 1 -- 172.21.15.38:0/2886290344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc55415a780 msgr2=0x7fc5540cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.549+0000 7fc553fff640 1 --2- 172.21.15.38:0/2886290344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc55415a780 0x7fc5540cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.549+0000 7fc553fff640 1 -- 172.21.15.38:0/2886290344 >> v2:172.21.15.107:3300/0 conn(0x7fc5540cd670 msgr2=0x7fc5540b3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.549+0000 7fc553fff640 1 --2- 172.21.15.38:0/2886290344 >> v2:172.21.15.107:3300/0 conn(0x7fc5540cd670 0x7fc5540b3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.549+0000 7fc553fff640 1 -- 172.21.15.38:0/2886290344 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5540cb750 con 0x7fc5540b3870 2024-04-25T08:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc553fff640 1 --2- 172.21.15.38:0/2886290344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5540b3870 0x7fc5540ce560 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fc554070540 tx=0x7fc54c02c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59ed426467cd4b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc5527fc640 1 -- 172.21.15.38:0/2886290344 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c00b2e0 con 0x7fc5540b3870 2024-04-25T08:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc5527fc640 1 -- 172.21.15.38:0/2886290344 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c009600 con 0x7fc5540b3870 2024-04-25T08:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc5527fc640 1 -- 172.21.15.38:0/2886290344 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c00b480 con 0x7fc5540b3870 2024-04-25T08:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc559fc1640 1 -- 172.21.15.38:0/2886290344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5540b3870 msgr2=0x7fc5540ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc559fc1640 1 --2- 172.21.15.38:0/2886290344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5540b3870 0x7fc5540ce560 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fc554070540 tx=0x7fc54c02c8a0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc559fc1640 1 -- 172.21.15.38:0/2886290344 shutdown_connections 2024-04-25T08:36:22.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc559fc1640 1 --2- 172.21.15.38:0/2886290344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5540b3870 0x7fc5540ce560 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc559fc1640 1 --2- 172.21.15.38:0/2886290344 >> v2:172.21.15.107:3300/0 conn(0x7fc5540cd670 0x7fc5540b3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc559fc1640 1 --2- 172.21.15.38:0/2886290344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc55415a780 0x7fc5540cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc559fc1640 1 -- 172.21.15.38:0/2886290344 >> 172.21.15.38:0/2886290344 conn(0x7fc5540b1ff0 msgr2=0x7fc5540b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:22.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc559fc1640 1 -- 172.21.15.38:0/2886290344 shutdown_connections 2024-04-25T08:36:22.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.550+0000 7fc559fc1640 1 -- 172.21.15.38:0/2886290344 wait complete. 2024-04-25T08:36:22.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.552+0000 7fc559fc1640 1 Processor -- start 2024-04-25T08:36:22.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.552+0000 7fc559fc1640 1 -- start start 2024-04-25T08:36:22.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc559fc1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5540b3870 0x7fc55414db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc559fc1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5540cd670 0x7fc55414cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc559fc1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc55415a780 0x7fc5540c8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc559fc1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540cda90 con 0x7fc55415a780 2024-04-25T08:36:22.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc559fc1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55415ab60 con 0x7fc5540b3870 2024-04-25T08:36:22.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc559fc1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540cdc10 con 0x7fc5540cd670 2024-04-25T08:36:22.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc553fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc55415a780 0x7fc5540c8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc553fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc55415a780 0x7fc5540c8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56020/0 (socket says 172.21.15.38:56020) 2024-04-25T08:36:22.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc553fff640 1 -- 172.21.15.38:0/2800559200 learned_addr learned my addr 172.21.15.38:0/2800559200 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:22.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc5537fe640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5540b3870 0x7fc55414db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc552ffd640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5540cd670 0x7fc55414cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc553fff640 1 -- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5540cd670 msgr2=0x7fc55414cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc553fff640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5540cd670 0x7fc55414cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc553fff640 1 -- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5540b3870 msgr2=0x7fc55414db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:22.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc553fff640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5540b3870 0x7fc55414db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:22.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc553fff640 1 -- 172.21.15.38:0/2800559200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55415d970 con 0x7fc55415a780 2024-04-25T08:36:22.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc5537fe640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5540b3870 0x7fc55414db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:22.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc553fff640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc55415a780 0x7fc5540c8c90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fc54c02c8e0 tx=0x7fc54c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.553+0000 7fc53bfff640 1 -- 172.21.15.38:0/2800559200 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c041030 con 0x7fc55415a780 2024-04-25T08:36:22.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.554+0000 7fc53bfff640 1 -- 172.21.15.38:0/2800559200 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c03d650 con 0x7fc55415a780 2024-04-25T08:36:22.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.554+0000 7fc53bfff640 1 -- 172.21.15.38:0/2800559200 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c03da20 con 0x7fc55415a780 2024-04-25T08:36:22.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.554+0000 7fc559fc1640 1 -- 172.21.15.38:0/2800559200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5540c9a60 con 0x7fc55415a780 2024-04-25T08:36:22.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.554+0000 7fc559fc1640 1 -- 172.21.15.38:0/2800559200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5540ca020 con 0x7fc55415a780 2024-04-25T08:36:22.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.555+0000 7fc559fc1640 1 -- 172.21.15.38:0/2800559200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55404ef90 con 0x7fc55415a780 2024-04-25T08:36:22.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.555+0000 7fc53bfff640 1 -- 172.21.15.38:0/2800559200 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc54c044030 con 0x7fc55415a780 2024-04-25T08:36:22.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.555+0000 7fc53bfff640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc54006f8e0 0x7fc540071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:22.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.556+0000 7fc53bfff640 1 -- 172.21.15.38:0/2800559200 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fc54c030020 con 0x7fc55415a780 2024-04-25T08:36:22.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.556+0000 7fc5537fe640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc54006f8e0 0x7fc540071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:22.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.556+0000 7fc5537fe640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc54006f8e0 0x7fc540071da0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc544006fd0 tx=0x7fc544008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:22.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.560+0000 7fc53bfff640 1 -- 172.21.15.38:0/2800559200 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc54c079490 con 0x7fc55415a780 2024-04-25T08:36:22.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:22.752+0000 7fc559fc1640 1 -- 172.21.15.38:0/2800559200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc5540b6460 con 0x7fc55415a780 2024-04-25T08:36:23.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.514+0000 7fc53bfff640 1 -- 172.21.15.38:0/2800559200 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v76) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc54c07d530 con 0x7fc55415a780 2024-04-25T08:36:23.508 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-04-25T08:36:23.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.517+0000 7fc559fc1640 1 -- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc54006f8e0 msgr2=0x7fc540071da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.517+0000 7fc559fc1640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc54006f8e0 0x7fc540071da0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc544006fd0 tx=0x7fc544008040 comp rx=0 tx=0).stop 2024-04-25T08:36:23.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.517+0000 7fc559fc1640 1 -- 172.21.15.38:0/2800559200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc55415a780 msgr2=0x7fc5540c8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.517+0000 7fc559fc1640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc55415a780 0x7fc5540c8c90 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fc54c02c8e0 tx=0x7fc54c004a40 comp rx=0 tx=0).stop 2024-04-25T08:36:23.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.517+0000 7fc559fc1640 1 -- 172.21.15.38:0/2800559200 shutdown_connections 2024-04-25T08:36:23.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.517+0000 7fc559fc1640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc55415a780 0x7fc5540c8c90 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.517+0000 7fc559fc1640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc54006f8e0 0x7fc540071da0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.517+0000 7fc559fc1640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5540cd670 0x7fc55414cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.517+0000 7fc559fc1640 1 --2- 172.21.15.38:0/2800559200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5540b3870 0x7fc55414db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.517+0000 7fc559fc1640 1 -- 172.21.15.38:0/2800559200 >> 172.21.15.38:0/2800559200 conn(0x7fc5540b1ff0 msgr2=0x7fc5540cbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:23.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.517+0000 7fc559fc1640 1 -- 172.21.15.38:0/2800559200 shutdown_connections 2024-04-25T08:36:23.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.517+0000 7fc559fc1640 1 -- 172.21.15.38:0/2800559200 wait complete. 2024-04-25T08:36:23.526 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:36:23.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.639+0000 7f05d3e6b640 1 Processor -- start 2024-04-25T08:36:23.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.639+0000 7f05d3e6b640 1 -- start start 2024-04-25T08:36:23.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d3e6b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f05cc0b5710 0x7f05cc161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d3e6b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05cc15ddf0 0x7f05cc0b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d3e6b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f05cc0b5af0 0x7f05cc06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d3e6b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05cc0b9930 con 0x7f05cc0b5710 2024-04-25T08:36:23.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d3e6b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05cc161e00 con 0x7f05cc0b5af0 2024-04-25T08:36:23.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d3e6b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc1620e0 con 0x7f05cc15ddf0 2024-04-25T08:36:23.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d1c07640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f05cc0b5710 0x7f05cc161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d1c07640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f05cc0b5710 0x7f05cc161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56022/0 (socket says 172.21.15.38:56022) 2024-04-25T08:36:23.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d1c07640 1 -- 172.21.15.38:0/10284745 learned_addr learned my addr 172.21.15.38:0/10284745 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:23.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d1406640 1 --2- 172.21.15.38:0/10284745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05cc15ddf0 0x7f05cc0b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d2408640 1 --2- 172.21.15.38:0/10284745 >> v2:172.21.15.107:3300/0 conn(0x7f05cc0b5af0 0x7f05cc06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d1c07640 1 -- 172.21.15.38:0/10284745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05cc15ddf0 msgr2=0x7f05cc0b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:23.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d1c07640 1 --2- 172.21.15.38:0/10284745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05cc15ddf0 0x7f05cc0b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d1c07640 1 -- 172.21.15.38:0/10284745 >> v2:172.21.15.107:3300/0 conn(0x7f05cc0b5af0 msgr2=0x7f05cc06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:23.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d1c07640 1 --2- 172.21.15.38:0/10284745 >> v2:172.21.15.107:3300/0 conn(0x7f05cc0b5af0 0x7f05cc06bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d1c07640 1 -- 172.21.15.38:0/10284745 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc1623c0 con 0x7f05cc0b5710 2024-04-25T08:36:23.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d2408640 1 --2- 172.21.15.38:0/10284745 >> v2:172.21.15.107:3300/0 conn(0x7f05cc0b5af0 0x7f05cc06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:23.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.640+0000 7f05d1406640 1 --2- 172.21.15.38:0/10284745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05cc15ddf0 0x7f05cc0b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:23.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.641+0000 7f05d1c07640 1 --2- 172.21.15.38:0/10284745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05cc0b5710 0x7f05cc161700 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0d1bd0 tx=0x7f05c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b7674e0ce43034 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:23.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.641+0000 7f05d0c05640 1 -- 172.21.15.38:0/10284745 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8037070 con 0x7f05cc0b5710 2024-04-25T08:36:23.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.641+0000 7f05d0c05640 1 -- 172.21.15.38:0/10284745 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c8032440 con 0x7f05cc0b5710 2024-04-25T08:36:23.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.641+0000 7f05d0c05640 1 -- 172.21.15.38:0/10284745 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c8032810 con 0x7f05cc0b5710 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.642+0000 7f05d3e6b640 1 -- 172.21.15.38:0/10284745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05cc0b5710 msgr2=0x7f05cc161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.642+0000 7f05d3e6b640 1 --2- 172.21.15.38:0/10284745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05cc0b5710 0x7f05cc161700 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f05cc0d1bd0 tx=0x7f05c802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.642+0000 7f05d3e6b640 1 -- 172.21.15.38:0/10284745 shutdown_connections 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.642+0000 7f05d3e6b640 1 --2- 172.21.15.38:0/10284745 >> v2:172.21.15.107:3300/0 conn(0x7f05cc0b5af0 0x7f05cc06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.642+0000 7f05d3e6b640 1 --2- 172.21.15.38:0/10284745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05cc15ddf0 0x7f05cc0b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.642+0000 7f05d3e6b640 1 --2- 172.21.15.38:0/10284745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05cc0b5710 0x7f05cc161700 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.642+0000 7f05d3e6b640 1 -- 172.21.15.38:0/10284745 >> 172.21.15.38:0/10284745 conn(0x7f05cc0d07c0 msgr2=0x7f05cc0d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.642+0000 7f05d3e6b640 1 -- 172.21.15.38:0/10284745 shutdown_connections 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.642+0000 7f05d3e6b640 1 -- 172.21.15.38:0/10284745 wait complete. 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.643+0000 7f05d3e6b640 1 Processor -- start 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.643+0000 7f05d3e6b640 1 -- start start 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d3e6b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05cc0b5710 0x7f05cc0cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d3e6b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc0b5af0 0x7f05cc0b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d3e6b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05cc15ddf0 0x7f05cc14c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d3e6b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc06c6b0 con 0x7f05cc15ddf0 2024-04-25T08:36:23.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d3e6b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc06c530 con 0x7f05cc0b5af0 2024-04-25T08:36:23.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d3e6b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc06c830 con 0x7f05cc0b5710 2024-04-25T08:36:23.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d2408640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05cc15ddf0 0x7f05cc14c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d2408640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05cc15ddf0 0x7f05cc14c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56042/0 (socket says 172.21.15.38:56042) 2024-04-25T08:36:23.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d2408640 1 -- 172.21.15.38:0/3539850596 learned_addr learned my addr 172.21.15.38:0/3539850596 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:23.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d1c07640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05cc0b5710 0x7f05cc0cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d1406640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc0b5af0 0x7f05cc0b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d2408640 1 -- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05cc0b5710 msgr2=0x7f05cc0cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d2408640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05cc0b5710 0x7f05cc0cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d2408640 1 -- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc0b5af0 msgr2=0x7f05cc0b8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d2408640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc0b5af0 0x7f05cc0b8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.644+0000 7f05d2408640 1 -- 172.21.15.38:0/3539850596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc0cb640 con 0x7f05cc15ddf0 2024-04-25T08:36:23.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.645+0000 7f05d2408640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05cc15ddf0 0x7f05cc14c460 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f05c400cef0 tx=0x7f05c400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:23.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.645+0000 7f05c27fc640 1 -- 172.21.15.38:0/3539850596 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c401a070 con 0x7f05cc15ddf0 2024-04-25T08:36:23.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.645+0000 7f05c27fc640 1 -- 172.21.15.38:0/3539850596 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05c400d840 con 0x7f05cc15ddf0 2024-04-25T08:36:23.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.645+0000 7f05d3e6b640 1 -- 172.21.15.38:0/3539850596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc0cb920 con 0x7f05cc15ddf0 2024-04-25T08:36:23.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.645+0000 7f05c27fc640 1 -- 172.21.15.38:0/3539850596 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c400dc30 con 0x7f05cc15ddf0 2024-04-25T08:36:23.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.645+0000 7f05d3e6b640 1 -- 172.21.15.38:0/3539850596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc0cbee0 con 0x7f05cc15ddf0 2024-04-25T08:36:23.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.646+0000 7f05d3e6b640 1 -- 172.21.15.38:0/3539850596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc068e80 con 0x7f05cc15ddf0 2024-04-25T08:36:23.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.646+0000 7f05c27fc640 1 -- 172.21.15.38:0/3539850596 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05c401d070 con 0x7f05cc15ddf0 2024-04-25T08:36:23.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.647+0000 7f05c27fc640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f05b006b4d0 0x7f05b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:23.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.647+0000 7f05c27fc640 1 -- 172.21.15.38:0/3539850596 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f05c408a8e0 con 0x7f05cc15ddf0 2024-04-25T08:36:23.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.647+0000 7f05d1c07640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f05b006b4d0 0x7f05b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:23.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.647+0000 7f05d1c07640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f05b006b4d0 0x7f05b006d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f05c8034000 tx=0x7f05c802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:23.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.651+0000 7f05c27fc640 1 -- 172.21.15.38:0/3539850596 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05c40591c0 con 0x7f05cc15ddf0 2024-04-25T08:36:23.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.902+0000 7f05d3e6b640 1 -- 172.21.15.38:0/3539850596 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f05cc15d380 con 0x7f05cc15ddf0 2024-04-25T08:36:23.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.902+0000 7f05c27fc640 1 -- 172.21.15.38:0/3539850596 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f05c405d260 con 0x7f05cc15ddf0 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmwq8CDBAA+cjd+QXtFgwZ3kxdABRWlA==", 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.897 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmPRFbFxAAojn2r/62vvf1qTy9QVuyIw==", 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmGLjqEhAAyPRgyy5iMgtccapydv4uOQ==", 2024-04-25T08:36:23.898 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmt0AUHhAAoQyQ7wXBMGE+7Hi84tUfAg==", 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.899 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCxFCpmv+iTExAAEJihmsvBbp9xXxPnj49D6g==", 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC0FCpmdnQKFBAADLAsYZEUhoJx292jIYZroQ==", 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.900 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC3FCpm4f5BExAAMVSPkqsrnuLcO88AXKrGHQ==", 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.901 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC6FCpmHNN7EhAABPhmnxGo70ZOd8Oex62Mdw==", 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDAFCpmiThGFhAAG5Pyvn/JzL9NUoxHJkPMvg==", 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:36:23.902 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDDFCpmb/FHFBAA2cMhMqQQc4Fk6j8FpjhnaQ==", 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDGFCpmAZsSEhAAnGIqqcvuHMVSc8Zbk309kA==", 2024-04-25T08:36:23.903 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDJFCpmqdVUEBAAb9abblyUztrjwxxe4jsAwQ==", 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.904 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw==", 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmL7eKKhAASeQEdHDBInfozdLVzP4DQw==", 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:36:23.905 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:36:23.906 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:36:23.906 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:36:23.906 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.906 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.906 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.906 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-04-25T08:36:23.906 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmH22HJxAAWyrooyJn5hRs/bb0Rn1HHw==", 2024-04-25T08:36:23.906 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.906 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:36:23.906 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:36:23.906 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpm6Zr/LxAAxh9Y2gvvqj1jIaVVrQAvgw==", 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.907 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCpFCpmlj/fLBAAvxLluFQFR8uhppGlNpaygA==", 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcXHhCBAAvI5zmWidVTvOmEcVRWasEA==", 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:36:23.908 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcr96BRAAQwUPxILJukYERAiEXeur+g==", 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T08:36:23.909 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:36:23.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.904+0000 7f05d3e6b640 1 -- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f05b006b4d0 msgr2=0x7f05b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.904+0000 7f05d3e6b640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f05b006b4d0 0x7f05b006d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f05c8034000 tx=0x7f05c802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:23.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.904+0000 7f05d3e6b640 1 -- 172.21.15.38:0/3539850596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05cc15ddf0 msgr2=0x7f05cc14c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:23.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.904+0000 7f05d3e6b640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05cc15ddf0 0x7f05cc14c460 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f05c400cef0 tx=0x7f05c400a710 comp rx=0 tx=0).stop 2024-04-25T08:36:23.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.904+0000 7f05d3e6b640 1 -- 172.21.15.38:0/3539850596 shutdown_connections 2024-04-25T08:36:23.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.904+0000 7f05d3e6b640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05cc15ddf0 0x7f05cc14c460 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.904+0000 7f05d3e6b640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05cc0b5af0 0x7f05cc0b8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.904+0000 7f05d3e6b640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f05b006b4d0 0x7f05b006d990 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.904+0000 7f05d3e6b640 1 --2- 172.21.15.38:0/3539850596 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05cc0b5710 0x7f05cc0cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:23.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.904+0000 7f05d3e6b640 1 -- 172.21.15.38:0/3539850596 >> 172.21.15.38:0/3539850596 conn(0x7f05cc0d07c0 msgr2=0x7f05cc0b6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:23.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.904+0000 7f05d3e6b640 1 -- 172.21.15.38:0/3539850596 shutdown_connections 2024-04-25T08:36:23.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:23.904+0000 7f05d3e6b640 1 -- 172.21.15.38:0/3539850596 wait complete. 2024-04-25T08:36:23.912 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:24.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.019+0000 7efeb6858640 1 Processor -- start 2024-04-25T08:36:24.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.019+0000 7efeb6858640 1 -- start start 2024-04-25T08:36:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.020+0000 7efeb6858640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efeb015b730 0x7efeb015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.020+0000 7efeb6858640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efeb00d0cd0 0x7efeb00d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.020+0000 7efeb6858640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb0073220 con 0x7efeb00ccd00 2024-04-25T08:36:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.020+0000 7efeb6858640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb00cdd10 con 0x7efeb00d0cd0 2024-04-25T08:36:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.020+0000 7efeb6858640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00cf7f0 con 0x7efeb015b730 2024-04-25T08:36:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.020+0000 7efeaf7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efeb00d0cd0 0x7efeb00d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.020+0000 7efeaf7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efeb00d0cd0 0x7efeb00d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56052/0 (socket says 172.21.15.38:56052) 2024-04-25T08:36:24.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.020+0000 7efeaf7fe640 1 -- 172.21.15.38:0/1003129861 learned_addr learned my addr 172.21.15.38:0/1003129861 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.020+0000 7efeaffff640 1 --2- 172.21.15.38:0/1003129861 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efeb015b730 0x7efeb015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.021+0000 7efeaf7fe640 1 -- 172.21.15.38:0/1003129861 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efeb015b730 msgr2=0x7efeb015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.021+0000 7efeaf7fe640 1 --2- 172.21.15.38:0/1003129861 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efeb015b730 0x7efeb015bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.021+0000 7efeaf7fe640 1 -- 172.21.15.38:0/1003129861 >> v1:172.21.15.107:6789/0 conn(0x7efeb00ccd00 legacy=0x7efeb00b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:24.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.021+0000 7efeaf7fe640 1 -- 172.21.15.38:0/1003129861 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb00cfad0 con 0x7efeb00d0cd0 2024-04-25T08:36:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.021+0000 7efeaf7fe640 1 --2- 172.21.15.38:0/1003129861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efeb00d0cd0 0x7efeb00d1130 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7efea0009870 tx=0x7efea002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c53071cacc13e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.021+0000 7efeaeffd640 1 -- 172.21.15.38:0/1003129861 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0037070 con 0x7efeb00d0cd0 2024-04-25T08:36:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.022+0000 7efeaeffd640 1 -- 172.21.15.38:0/1003129861 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0032440 con 0x7efeb00d0cd0 2024-04-25T08:36:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.022+0000 7efeaeffd640 1 -- 172.21.15.38:0/1003129861 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0032810 con 0x7efeb00d0cd0 2024-04-25T08:36:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.022+0000 7efeb6858640 1 -- 172.21.15.38:0/1003129861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efeb00d0cd0 msgr2=0x7efeb00d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.022+0000 7efeb6858640 1 --2- 172.21.15.38:0/1003129861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efeb00d0cd0 0x7efeb00d1130 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7efea0009870 tx=0x7efea002a270 comp rx=0 tx=0).stop 2024-04-25T08:36:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.022+0000 7efeb6858640 1 -- 172.21.15.38:0/1003129861 shutdown_connections 2024-04-25T08:36:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.022+0000 7efeb6858640 1 --2- 172.21.15.38:0/1003129861 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efeb00d0cd0 0x7efeb00d1130 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.022+0000 7efeb6858640 1 --2- 172.21.15.38:0/1003129861 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efeb015b730 0x7efeb015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.022+0000 7efeb6858640 1 -- 172.21.15.38:0/1003129861 >> 172.21.15.38:0/1003129861 conn(0x7efeb00b5c90 msgr2=0x7efeb00b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:24.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.022+0000 7efeb6858640 1 -- 172.21.15.38:0/1003129861 shutdown_connections 2024-04-25T08:36:24.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.022+0000 7efeb6858640 1 -- 172.21.15.38:0/1003129861 wait complete. 2024-04-25T08:36:24.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.023+0000 7efeb6858640 1 Processor -- start 2024-04-25T08:36:24.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.023+0000 7efeb6858640 1 -- start start 2024-04-25T08:36:24.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.024+0000 7efeb6858640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efeb00ccd00 0x7efeb01561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.024+0000 7efeb6858640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efeb00d0cd0 0x7efeb01407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.024+0000 7efeb6858640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efeb015b730 0x7efeb013fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.024+0000 7efeb6858640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00d0370 con 0x7efeb00d0cd0 2024-04-25T08:36:24.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.024+0000 7efeb6858640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00d0090 con 0x7efeb015b730 2024-04-25T08:36:24.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.024+0000 7efeb6858640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00cf4e0 con 0x7efeb00ccd00 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.024+0000 7efeaf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efeb00d0cd0 0x7efeb01407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.024+0000 7efeaf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efeb00d0cd0 0x7efeb01407e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56062/0 (socket says 172.21.15.38:56062) 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.024+0000 7efeaf7fe640 1 -- 172.21.15.38:0/3184612898 learned_addr learned my addr 172.21.15.38:0/3184612898 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.024+0000 7efeaffff640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efeb00ccd00 0x7efeb01561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.024+0000 7efeb4df5640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efeb015b730 0x7efeb013fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.025+0000 7efeaf7fe640 1 -- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efeb00ccd00 msgr2=0x7efeb01561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.025+0000 7efeaf7fe640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efeb00ccd00 0x7efeb01561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.025+0000 7efeaf7fe640 1 -- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efeb015b730 msgr2=0x7efeb013fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.025+0000 7efeaf7fe640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efeb015b730 0x7efeb013fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.025+0000 7efeaf7fe640 1 -- 172.21.15.38:0/3184612898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0140d20 con 0x7efeb00d0cd0 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.025+0000 7efeaffff640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efeb00ccd00 0x7efeb01561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.025+0000 7efeaf7fe640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efeb00d0cd0 0x7efeb01407e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7efea0000c00 tx=0x7efea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.025+0000 7efeacff9640 1 -- 172.21.15.38:0/3184612898 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0041020 con 0x7efeb00d0cd0 2024-04-25T08:36:24.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.025+0000 7efeacff9640 1 -- 172.21.15.38:0/3184612898 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea00306c0 con 0x7efeb00d0cd0 2024-04-25T08:36:24.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.025+0000 7efeb6858640 1 -- 172.21.15.38:0/3184612898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb0140fa0 con 0x7efeb00d0cd0 2024-04-25T08:36:24.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.025+0000 7efeacff9640 1 -- 172.21.15.38:0/3184612898 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0037040 con 0x7efeb00d0cd0 2024-04-25T08:36:24.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.025+0000 7efeb6858640 1 -- 172.21.15.38:0/3184612898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb0141560 con 0x7efeb00d0cd0 2024-04-25T08:36:24.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.026+0000 7efeb6858640 1 -- 172.21.15.38:0/3184612898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb0068e80 con 0x7efeb00d0cd0 2024-04-25T08:36:24.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.027+0000 7efeacff9640 1 -- 172.21.15.38:0/3184612898 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efea0044020 con 0x7efeb00d0cd0 2024-04-25T08:36:24.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.027+0000 7efeacff9640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efe9006b500 0x7efe9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.027+0000 7efeacff9640 1 -- 172.21.15.38:0/3184612898 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efea00aa7b0 con 0x7efeb00d0cd0 2024-04-25T08:36:24.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.027+0000 7efeaffff640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efe9006b500 0x7efe9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.028+0000 7efeaffff640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efe9006b500 0x7efe9006d9c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efe98006fd0 tx=0x7efe98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.031+0000 7efeacff9640 1 -- 172.21.15.38:0/3184612898 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efea004b030 con 0x7efeb00d0cd0 2024-04-25T08:36:24.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.213+0000 7efeb6858640 1 -- 172.21.15.38:0/3184612898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efeb006d330 con 0x7efeb00d0cd0 2024-04-25T08:36:24.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.214+0000 7efeacff9640 1 -- 172.21.15.38:0/3184612898 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+13153 (secure 0 0 0) 0x7efeb0140d20 con 0x7efeb00d0cd0 2024-04-25T08:36:24.208 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:24.208 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":78,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:36:23.523023+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:24.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.215+0000 7efeb6858640 1 -- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efe9006b500 msgr2=0x7efe9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.215+0000 7efeb6858640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efe9006b500 0x7efe9006d9c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efe98006fd0 tx=0x7efe98008040 comp rx=0 tx=0).stop 2024-04-25T08:36:24.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.215+0000 7efeb6858640 1 -- 172.21.15.38:0/3184612898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efeb00d0cd0 msgr2=0x7efeb01407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.215+0000 7efeb6858640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efeb00d0cd0 0x7efeb01407e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7efea0000c00 tx=0x7efea00046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.216+0000 7efeb6858640 1 -- 172.21.15.38:0/3184612898 shutdown_connections 2024-04-25T08:36:24.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.216+0000 7efeb6858640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efeb015b730 0x7efeb013fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.216+0000 7efeb6858640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efeb00d0cd0 0x7efeb01407e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.216+0000 7efeb6858640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efe9006b500 0x7efe9006d9c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.216+0000 7efeb6858640 1 --2- 172.21.15.38:0/3184612898 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efeb00ccd00 0x7efeb01561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.216+0000 7efeb6858640 1 -- 172.21.15.38:0/3184612898 >> 172.21.15.38:0/3184612898 conn(0x7efeb00b5c90 msgr2=0x7efeb0156ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:24.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.216+0000 7efeb6858640 1 -- 172.21.15.38:0/3184612898 shutdown_connections 2024-04-25T08:36:24.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.216+0000 7efeb6858640 1 -- 172.21.15.38:0/3184612898 wait complete. 2024-04-25T08:36:24.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:36:24.222 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:36:24.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.331+0000 7f94dc1f9640 1 Processor -- start 2024-04-25T08:36:24.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.331+0000 7f94dc1f9640 1 -- start start 2024-04-25T08:36:24.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94dc1f9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f94d409a180 0x7f94d409a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94dc1f9640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f94d4162390 0x7f94d40cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94dc1f9640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94d415f300 con 0x7f94d415df30 2024-04-25T08:36:24.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94dc1f9640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d415e310 con 0x7f94d409a180 2024-04-25T08:36:24.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94dc1f9640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94d409aeb0 con 0x7f94d4162390 2024-04-25T08:36:24.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94d9794640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f94d409a180 0x7f94d409a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94d9794640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f94d409a180 0x7f94d409a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56078/0 (socket says 172.21.15.38:56078) 2024-04-25T08:36:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94d9f95640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f94d415df30 0x7f94d4099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37492/0 (socket says 172.21.15.38:37492) 2024-04-25T08:36:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94d9794640 1 -- 172.21.15.38:0/759321968 learned_addr learned my addr 172.21.15.38:0/759321968 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94da796640 1 --2- 172.21.15.38:0/759321968 >> v2:172.21.15.107:3300/0 conn(0x7f94d4162390 0x7f94d40cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94d9794640 1 -- 172.21.15.38:0/759321968 >> v2:172.21.15.107:3300/0 conn(0x7f94d4162390 msgr2=0x7f94d40cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94d9794640 1 --2- 172.21.15.38:0/759321968 >> v2:172.21.15.107:3300/0 conn(0x7f94d4162390 0x7f94d40cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94d9794640 1 -- 172.21.15.38:0/759321968 >> v1:172.21.15.38:6789/0 conn(0x7f94d415df30 legacy=0x7f94d4099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94d9794640 1 -- 172.21.15.38:0/759321968 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d415e540 con 0x7f94d409a180 2024-04-25T08:36:24.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.332+0000 7f94d8f93640 1 -- 172.21.15.38:0/759321968 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158822279 0 0) 0x7f94d415f300 con 0x7f94d415df30 2024-04-25T08:36:24.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.333+0000 7f94d9794640 1 --2- 172.21.15.38:0/759321968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94d409a180 0x7f94d409a7b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f94d40d2680 tx=0x7f94c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec22ad5a00f06c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.333+0000 7f94d8f93640 1 -- 172.21.15.38:0/759321968 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8037070 con 0x7f94d409a180 2024-04-25T08:36:24.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.333+0000 7f94d8f93640 1 -- 172.21.15.38:0/759321968 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c8032440 con 0x7f94d409a180 2024-04-25T08:36:24.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.333+0000 7f94d8f93640 1 -- 172.21.15.38:0/759321968 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8032810 con 0x7f94d409a180 2024-04-25T08:36:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.333+0000 7f94dc1f9640 1 -- 172.21.15.38:0/759321968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94d409a180 msgr2=0x7f94d409a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.333+0000 7f94dc1f9640 1 --2- 172.21.15.38:0/759321968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94d409a180 0x7f94d409a7b0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f94d40d2680 tx=0x7f94c802a270 comp rx=0 tx=0).stop 2024-04-25T08:36:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.334+0000 7f94dc1f9640 1 -- 172.21.15.38:0/759321968 shutdown_connections 2024-04-25T08:36:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.334+0000 7f94dc1f9640 1 --2- 172.21.15.38:0/759321968 >> v2:172.21.15.107:3300/0 conn(0x7f94d4162390 0x7f94d40cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.334+0000 7f94dc1f9640 1 --2- 172.21.15.38:0/759321968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94d409a180 0x7f94d409a7b0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.334+0000 7f94dc1f9640 1 -- 172.21.15.38:0/759321968 >> 172.21.15.38:0/759321968 conn(0x7f94d40b3110 msgr2=0x7f94d40b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.334+0000 7f94dc1f9640 1 -- 172.21.15.38:0/759321968 shutdown_connections 2024-04-25T08:36:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.334+0000 7f94dc1f9640 1 -- 172.21.15.38:0/759321968 wait complete. 2024-04-25T08:36:24.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.335+0000 7f94dc1f9640 1 Processor -- start 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.335+0000 7f94dc1f9640 1 -- start start 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.335+0000 7f94dc1f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94d409a180 0x7f94d4159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94dc1f9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94d415df30 0x7f94d40c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94dc1f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d4162390 0x7f94d40c64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94dc1f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d415eb00 con 0x7f94d409a180 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94dc1f9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d415ee70 con 0x7f94d4162390 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94dc1f9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d41627a0 con 0x7f94d415df30 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d9f95640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94d409a180 0x7f94d4159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d9f95640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94d409a180 0x7f94d4159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56096/0 (socket says 172.21.15.38:56096) 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d9f95640 1 -- 172.21.15.38:0/1287265802 learned_addr learned my addr 172.21.15.38:0/1287265802 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d9794640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94d415df30 0x7f94d40c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d9f95640 1 -- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94d415df30 msgr2=0x7f94d40c7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d9f95640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94d415df30 0x7f94d40c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94da796640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d4162390 0x7f94d40c64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d9f95640 1 -- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d4162390 msgr2=0x7f94d40c64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d9f95640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d4162390 0x7f94d40c64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d9f95640 1 -- 172.21.15.38:0/1287265802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d40c5150 con 0x7f94d409a180 2024-04-25T08:36:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d9f95640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94d409a180 0x7f94d4159bb0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00c4d0 tx=0x7f94cc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d27fc640 1 -- 172.21.15.38:0/1287265802 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc018070 con 0x7f94d409a180 2024-04-25T08:36:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d27fc640 1 -- 172.21.15.38:0/1287265802 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94cc007500 con 0x7f94d409a180 2024-04-25T08:36:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94dc1f9640 1 -- 172.21.15.38:0/1287265802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d40c5430 con 0x7f94d409a180 2024-04-25T08:36:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94d27fc640 1 -- 172.21.15.38:0/1287265802 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94cc004a20 con 0x7f94d409a180 2024-04-25T08:36:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.336+0000 7f94dc1f9640 1 -- 172.21.15.38:0/1287265802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d40c59f0 con 0x7f94d409a180 2024-04-25T08:36:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.337+0000 7f94dc1f9640 1 -- 172.21.15.38:0/1287265802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94d4068e80 con 0x7f94d409a180 2024-04-25T08:36:24.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.338+0000 7f94d27fc640 1 -- 172.21.15.38:0/1287265802 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94cc01b070 con 0x7f94d409a180 2024-04-25T08:36:24.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.338+0000 7f94d27fc640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94c006b4d0 0x7f94c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.338+0000 7f94d27fc640 1 -- 172.21.15.38:0/1287265802 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f94cc0893f0 con 0x7f94d409a180 2024-04-25T08:36:24.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.339+0000 7f94d9794640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94c006b4d0 0x7f94c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.339+0000 7f94d9794640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94c006b4d0 0x7f94c006d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f94c8034000 tx=0x7f94c802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.342+0000 7f94d27fc640 1 -- 172.21.15.38:0/1287265802 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94cc058390 con 0x7f94d409a180 2024-04-25T08:36:24.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.525+0000 7f94dc1f9640 1 -- 172.21.15.38:0/1287265802 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94d40b3de0 con 0x7f94d409a180 2024-04-25T08:36:24.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.525+0000 7f94d27fc640 1 -- 172.21.15.38:0/1287265802 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94cc057c90 con 0x7f94d409a180 2024-04-25T08:36:24.519 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:36:24.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.527+0000 7f94dc1f9640 1 -- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94c006b4d0 msgr2=0x7f94c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.527+0000 7f94dc1f9640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94c006b4d0 0x7f94c006d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f94c8034000 tx=0x7f94c802e000 comp rx=0 tx=0).stop 2024-04-25T08:36:24.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.527+0000 7f94dc1f9640 1 -- 172.21.15.38:0/1287265802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94d409a180 msgr2=0x7f94d4159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.527+0000 7f94dc1f9640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94d409a180 0x7f94d4159bb0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f94cc00c4d0 tx=0x7f94cc00c990 comp rx=0 tx=0).stop 2024-04-25T08:36:24.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.527+0000 7f94dc1f9640 1 -- 172.21.15.38:0/1287265802 shutdown_connections 2024-04-25T08:36:24.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.527+0000 7f94dc1f9640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94d4162390 0x7f94d40c64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.527+0000 7f94dc1f9640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94c006b4d0 0x7f94c006d990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.527+0000 7f94dc1f9640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94d415df30 0x7f94d40c7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.527+0000 7f94dc1f9640 1 --2- 172.21.15.38:0/1287265802 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94d409a180 0x7f94d4159bb0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.527+0000 7f94dc1f9640 1 -- 172.21.15.38:0/1287265802 >> 172.21.15.38:0/1287265802 conn(0x7f94d40b3110 msgr2=0x7f94d415c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:24.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.527+0000 7f94dc1f9640 1 -- 172.21.15.38:0/1287265802 shutdown_connections 2024-04-25T08:36:24.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.527+0000 7f94dc1f9640 1 -- 172.21.15.38:0/1287265802 wait complete. 2024-04-25T08:36:24.533 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:36:24.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.639+0000 7fbd83dc8640 1 Processor -- start 2024-04-25T08:36:24.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.639+0000 7fbd83dc8640 1 -- start start 2024-04-25T08:36:24.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.640+0000 7fbd83dc8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd7c0d3900 0x7fbd7c0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.640+0000 7fbd83dc8640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbd7c0d0d90 0x7fbd7c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.640+0000 7fbd83dc8640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7c0d3e70 con 0x7fbd7c15dc10 2024-04-25T08:36:24.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.640+0000 7fbd83dc8640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd7c15dff0 con 0x7fbd7c0d3900 2024-04-25T08:36:24.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.640+0000 7fbd83dc8640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd7c0b6990 con 0x7fbd7c0d0d90 2024-04-25T08:36:24.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.640+0000 7fbd81363640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd7c0d3900 0x7fbd7c0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.640+0000 7fbd81363640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbd7c0d3900 0x7fbd7c0d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56098/0 (socket says 172.21.15.38:56098) 2024-04-25T08:36:24.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.640+0000 7fbd81363640 1 -- 172.21.15.38:0/2269265533 learned_addr learned my addr 172.21.15.38:0/2269265533 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:24.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.640+0000 7fbd82365640 1 --2- 172.21.15.38:0/2269265533 >> v2:172.21.15.107:3300/0 conn(0x7fbd7c0d0d90 0x7fbd7c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.641+0000 7fbd81363640 1 -- 172.21.15.38:0/2269265533 >> v2:172.21.15.107:3300/0 conn(0x7fbd7c0d0d90 msgr2=0x7fbd7c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:24.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.641+0000 7fbd81363640 1 --2- 172.21.15.38:0/2269265533 >> v2:172.21.15.107:3300/0 conn(0x7fbd7c0d0d90 0x7fbd7c0b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.641+0000 7fbd81363640 1 -- 172.21.15.38:0/2269265533 >> v1:172.21.15.107:6789/0 conn(0x7fbd7c15dc10 legacy=0x7fbd7c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.641+0000 7fbd81363640 1 -- 172.21.15.38:0/2269265533 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0d4150 con 0x7fbd7c0d3900 2024-04-25T08:36:24.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.641+0000 7fbd81363640 1 --2- 172.21.15.38:0/2269265533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd7c0d3900 0x7fbd7c0d0850 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0d0330 tx=0x7fbd7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbc54ddcbe0f409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.642+0000 7fbd80b62640 1 -- 172.21.15.38:0/2269265533 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70037070 con 0x7fbd7c0d3900 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.642+0000 7fbd80b62640 1 -- 172.21.15.38:0/2269265533 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd70032440 con 0x7fbd7c0d3900 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.642+0000 7fbd80b62640 1 -- 172.21.15.38:0/2269265533 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70032810 con 0x7fbd7c0d3900 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.642+0000 7fbd83dc8640 1 -- 172.21.15.38:0/2269265533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd7c0d3900 msgr2=0x7fbd7c0d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.642+0000 7fbd83dc8640 1 --2- 172.21.15.38:0/2269265533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd7c0d3900 0x7fbd7c0d0850 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c0d0330 tx=0x7fbd7002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.642+0000 7fbd83dc8640 1 -- 172.21.15.38:0/2269265533 shutdown_connections 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.642+0000 7fbd83dc8640 1 --2- 172.21.15.38:0/2269265533 >> v2:172.21.15.107:3300/0 conn(0x7fbd7c0d0d90 0x7fbd7c0b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.642+0000 7fbd83dc8640 1 --2- 172.21.15.38:0/2269265533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd7c0d3900 0x7fbd7c0d0850 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.642+0000 7fbd83dc8640 1 -- 172.21.15.38:0/2269265533 >> 172.21.15.38:0/2269265533 conn(0x7fbd7c066d10 msgr2=0x7fbd7c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.642+0000 7fbd83dc8640 1 -- 172.21.15.38:0/2269265533 shutdown_connections 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.642+0000 7fbd83dc8640 1 -- 172.21.15.38:0/2269265533 wait complete. 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.643+0000 7fbd83dc8640 1 Processor -- start 2024-04-25T08:36:24.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.643+0000 7fbd83dc8640 1 -- start start 2024-04-25T08:36:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.644+0000 7fbd83dc8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c0d0d90 0x7fbd7c0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.644+0000 7fbd83dc8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd7c0d3900 0x7fbd7c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.644+0000 7fbd83dc8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd7c15dc10 0x7fbd7c0ab490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.644+0000 7fbd83dc8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0d4710 con 0x7fbd7c15dc10 2024-04-25T08:36:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.644+0000 7fbd83dc8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0d11e0 con 0x7fbd7c0d0d90 2024-04-25T08:36:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.644+0000 7fbd83dc8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0b6bc0 con 0x7fbd7c0d3900 2024-04-25T08:36:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.644+0000 7fbd82365640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd7c15dc10 0x7fbd7c0ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.644+0000 7fbd82365640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd7c15dc10 0x7fbd7c0ab490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56106/0 (socket says 172.21.15.38:56106) 2024-04-25T08:36:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.644+0000 7fbd82365640 1 -- 172.21.15.38:0/3596856611 learned_addr learned my addr 172.21.15.38:0/3596856611 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd82365640 1 -- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd7c0d3900 msgr2=0x7fbd7c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd81b64640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c0d0d90 0x7fbd7c0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd81363640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd7c0d3900 0x7fbd7c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd82365640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd7c0d3900 0x7fbd7c15be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd82365640 1 -- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c0d0d90 msgr2=0x7fbd7c0c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd82365640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c0d0d90 0x7fbd7c0c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd82365640 1 -- 172.21.15.38:0/3596856611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0c81e0 con 0x7fbd7c15dc10 2024-04-25T08:36:24.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd81b64640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c0d0d90 0x7fbd7c0c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:24.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd82365640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd7c15dc10 0x7fbd7c0ab490 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00efc0 tx=0x7fbd6c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd7a7fc640 1 -- 172.21.15.38:0/3596856611 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c018020 con 0x7fbd7c15dc10 2024-04-25T08:36:24.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd7a7fc640 1 -- 172.21.15.38:0/3596856611 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c014480 con 0x7fbd7c15dc10 2024-04-25T08:36:24.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd83dc8640 1 -- 172.21.15.38:0/3596856611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7c0c84c0 con 0x7fbd7c15dc10 2024-04-25T08:36:24.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd7a7fc640 1 -- 172.21.15.38:0/3596856611 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c014870 con 0x7fbd7c15dc10 2024-04-25T08:36:24.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.645+0000 7fbd83dc8640 1 -- 172.21.15.38:0/3596856611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7c0c8a80 con 0x7fbd7c15dc10 2024-04-25T08:36:24.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.646+0000 7fbd83dc8640 1 -- 172.21.15.38:0/3596856611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7c068e80 con 0x7fbd7c15dc10 2024-04-25T08:36:24.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.646+0000 7fbd7a7fc640 1 -- 172.21.15.38:0/3596856611 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd6c01b020 con 0x7fbd7c15dc10 2024-04-25T08:36:24.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.647+0000 7fbd7a7fc640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd6806b4d0 0x7fbd6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.647+0000 7fbd7a7fc640 1 -- 172.21.15.38:0/3596856611 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbd6c08ac20 con 0x7fbd7c15dc10 2024-04-25T08:36:24.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.647+0000 7fbd81b64640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd6806b4d0 0x7fbd6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.648+0000 7fbd81b64640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd6806b4d0 0x7fbd6806d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbd740015e0 tx=0x7fbd74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.651+0000 7fbd7a7fc640 1 -- 172.21.15.38:0/3596856611 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd6c022030 con 0x7fbd7c15dc10 2024-04-25T08:36:24.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.832+0000 7fbd83dc8640 1 -- 172.21.15.38:0/3596856611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbd7c0b8340 con 0x7fbd7c15dc10 2024-04-25T08:36:24.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.833+0000 7fbd7a7fc640 1 -- 172.21.15.38:0/3596856611 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbd6c010040 con 0x7fbd7c15dc10 2024-04-25T08:36:24.827 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:24.827 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T08:36:24.827 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:36:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-04-25T08:36:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-04-25T08:36:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:24.828 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T08:36:24.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.835+0000 7fbd83dc8640 1 -- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd6806b4d0 msgr2=0x7fbd6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.835+0000 7fbd83dc8640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd6806b4d0 0x7fbd6806d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbd740015e0 tx=0x7fbd74009040 comp rx=0 tx=0).stop 2024-04-25T08:36:24.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.835+0000 7fbd83dc8640 1 -- 172.21.15.38:0/3596856611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd7c15dc10 msgr2=0x7fbd7c0ab490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.835+0000 7fbd83dc8640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd7c15dc10 0x7fbd7c0ab490 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00efc0 tx=0x7fbd6c00c490 comp rx=0 tx=0).stop 2024-04-25T08:36:24.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.835+0000 7fbd83dc8640 1 -- 172.21.15.38:0/3596856611 shutdown_connections 2024-04-25T08:36:24.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.835+0000 7fbd83dc8640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbd7c15dc10 0x7fbd7c0ab490 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.835+0000 7fbd83dc8640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbd6806b4d0 0x7fbd6806d990 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.835+0000 7fbd83dc8640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbd7c0d3900 0x7fbd7c15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.835+0000 7fbd83dc8640 1 --2- 172.21.15.38:0/3596856611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbd7c0d0d90 0x7fbd7c0c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.835+0000 7fbd83dc8640 1 -- 172.21.15.38:0/3596856611 >> 172.21.15.38:0/3596856611 conn(0x7fbd7c066d10 msgr2=0x7fbd7c0b79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:24.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.835+0000 7fbd83dc8640 1 -- 172.21.15.38:0/3596856611 shutdown_connections 2024-04-25T08:36:24.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.835+0000 7fbd83dc8640 1 -- 172.21.15.38:0/3596856611 wait complete. 2024-04-25T08:36:24.840 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:36:24.841 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:36:24.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.951+0000 7f14f48dd640 1 Processor -- start 2024-04-25T08:36:24.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.951+0000 7f14f48dd640 1 -- start start 2024-04-25T08:36:24.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14f48dd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f14f01612c0 0x7f14f00b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14f48dd640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f00b7200 con 0x7f14f00b6a00 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14f48dd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14f00b74e0 con 0x7f14f00d0870 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14f48dd640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f00b77c0 con 0x7f14f01612c0 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14edd9b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f14f00b6a00 0x7f14f00d1c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37502/0 (socket says 172.21.15.38:37502) 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14edd9b640 1 -- 172.21.15.38:0/760555414 learned_addr learned my addr 172.21.15.38:0/760555414 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ee59c640 1 --2- 172.21.15.38:0/760555414 >> v2:172.21.15.107:3300/0 conn(0x7f14f01612c0 0x7f14f00b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870941607 0 0) 0x7f14f00b7200 con 0x7f14f00b6a00 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14d8003520 con 0x7f14f00b6a00 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3093655092 0 0) 0x7f14d8003520 con 0x7f14f00b6a00 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 >> v2:172.21.15.107:3300/0 conn(0x7f14f01612c0 msgr2=0x7f14f00b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 --2- 172.21.15.38:0/760555414 >> v2:172.21.15.107:3300/0 conn(0x7f14f01612c0 0x7f14f00b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 >> v1:172.21.15.107:6789/0 conn(0x7f14f00d0870 legacy=0x7f14f00b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f00b7aa0 con 0x7f14f00b6a00 2024-04-25T08:36:24.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f14e4001670 con 0x7f14f00b6a00 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 >> v1:172.21.15.38:6789/0 conn(0x7f14f00b6a00 legacy=0x7f14f00d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 --2- 172.21.15.38:0/760555414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14d80039b0 0x7f14d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 --2- 172.21.15.38:0/760555414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 0x7f14d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 --2- 172.21.15.38:0/760555414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14d80051c0 0x7f14d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00b77c0 con 0x7f14d8004800 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d800c160 con 0x7f14d80051c0 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.952+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d800c360 con 0x7f14d80039b0 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.953+0000 7f14edd9b640 1 --2- 172.21.15.38:0/760555414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 0x7f14d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.953+0000 7f14ee59c640 1 --2- 172.21.15.38:0/760555414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14d80039b0 0x7f14d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.953+0000 7f14eed9d640 1 --2- 172.21.15.38:0/760555414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14d80051c0 0x7f14d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.953+0000 7f14edd9b640 1 -- 172.21.15.38:0/760555414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14d80039b0 msgr2=0x7f14d8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.953+0000 7f14edd9b640 1 --2- 172.21.15.38:0/760555414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14d80039b0 0x7f14d8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.953+0000 7f14edd9b640 1 -- 172.21.15.38:0/760555414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14d80051c0 msgr2=0x7f14d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.953+0000 7f14edd9b640 1 --2- 172.21.15.38:0/760555414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14d80051c0 0x7f14d800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.953+0000 7f14edd9b640 1 -- 172.21.15.38:0/760555414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d8003520 con 0x7f14d8004800 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.953+0000 7f14eed9d640 1 -- 172.21.15.38:0/760555414 reap_dead start 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.954+0000 7f14edd9b640 1 --2- 172.21.15.38:0/760555414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 0x7f14d8004c80 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c240 tx=0x7f14d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5ba4d075d5be2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.954+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4007cf0 con 0x7f14d8004800 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.954+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e40058d0 con 0x7f14d8004800 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.954+0000 7f14ed59a640 1 -- 172.21.15.38:0/760555414 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4005ca0 con 0x7f14d8004800 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.954+0000 7f14f48dd640 1 -- 172.21.15.38:0/760555414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 msgr2=0x7f14d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.954+0000 7f14f48dd640 1 --2- 172.21.15.38:0/760555414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 0x7f14d8004c80 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c240 tx=0x7f14d8005830 comp rx=0 tx=0).stop 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.954+0000 7f14f48dd640 1 -- 172.21.15.38:0/760555414 shutdown_connections 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.954+0000 7f14f48dd640 1 --2- 172.21.15.38:0/760555414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 0x7f14d8004c80 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.954+0000 7f14f48dd640 1 -- 172.21.15.38:0/760555414 >> 172.21.15.38:0/760555414 conn(0x7f14f0066d10 msgr2=0x7f14f0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:24.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.954+0000 7f14f48dd640 1 -- 172.21.15.38:0/760555414 shutdown_connections 2024-04-25T08:36:24.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.954+0000 7f14f48dd640 1 -- 172.21.15.38:0/760555414 wait complete. 2024-04-25T08:36:24.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.956+0000 7f14f48dd640 1 Processor -- start 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.956+0000 7f14f48dd640 1 -- start start 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.956+0000 7f14f48dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 0x7f14f0158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.956+0000 7f14f48dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14f0156990 0x7f14f01f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.956+0000 7f14f48dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14f0157060 0x7f14f01574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.956+0000 7f14f48dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00b93f0 con 0x7f14d8004800 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.956+0000 7f14f48dd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00b6e90 con 0x7f14f0157060 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.956+0000 7f14f48dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f00b9570 con 0x7f14f0156990 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.956+0000 7f14ee59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 0x7f14f0158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.956+0000 7f14ee59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 0x7f14f0158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56122/0 (socket says 172.21.15.38:56122) 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.956+0000 7f14ee59c640 1 -- 172.21.15.38:0/2333442125 learned_addr learned my addr 172.21.15.38:0/2333442125 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14edd9b640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14f0156990 0x7f14f01f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14eed9d640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14f0157060 0x7f14f01574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14ee59c640 1 -- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14f0156990 msgr2=0x7f14f01f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14ee59c640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14f0156990 0x7f14f01f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14ee59c640 1 -- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14f0157060 msgr2=0x7f14f01574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14ee59c640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14f0157060 0x7f14f01574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14ee59c640 1 -- 172.21.15.38:0/2333442125 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f0156e10 con 0x7f14d8004800 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14ee59c640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 0x7f14f0158f30 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f14e0011e30 tx=0x7f14e000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14deffd640 1 -- 172.21.15.38:0/2333442125 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0015630 con 0x7f14d8004800 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14deffd640 1 -- 172.21.15.38:0/2333442125 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e00157d0 con 0x7f14d8004800 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14f48dd640 1 -- 172.21.15.38:0/2333442125 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14f01595f0 con 0x7f14d8004800 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14deffd640 1 -- 172.21.15.38:0/2333442125 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0015ba0 con 0x7f14d8004800 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.957+0000 7f14f48dd640 1 -- 172.21.15.38:0/2333442125 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f0159bb0 con 0x7f14d8004800 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.958+0000 7f14f48dd640 1 -- 172.21.15.38:0/2333442125 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f0068e80 con 0x7f14d8004800 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.959+0000 7f14deffd640 1 -- 172.21.15.38:0/2333442125 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14e001d400 con 0x7f14d8004800 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.959+0000 7f14deffd640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f14d006b4d0 0x7f14d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:24.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.959+0000 7f14deffd640 1 -- 172.21.15.38:0/2333442125 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f14e008b120 con 0x7f14d8004800 2024-04-25T08:36:24.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.959+0000 7f14edd9b640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f14d006b4d0 0x7f14d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:24.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.960+0000 7f14edd9b640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f14d006b4d0 0x7f14d006d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f14e4033000 tx=0x7f14e4009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:24.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:24.963+0000 7f14deffd640 1 -- 172.21.15.38:0/2333442125 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14e0019030 con 0x7f14d8004800 2024-04-25T08:36:25.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.144+0000 7f14f48dd640 1 -- 172.21.15.38:0/2333442125 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f14f00b8bd0 con 0x7f14d8004800 2024-04-25T08:36:25.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.525+0000 7f14deffd640 1 -- 172.21.15.38:0/2333442125 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7f14e0010040 con 0x7f14d8004800 2024-04-25T08:36:25.520 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-04-25T08:36:25.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.528+0000 7f14f48dd640 1 -- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f14d006b4d0 msgr2=0x7f14d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:25.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.528+0000 7f14f48dd640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f14d006b4d0 0x7f14d006d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f14e4033000 tx=0x7f14e4009030 comp rx=0 tx=0).stop 2024-04-25T08:36:25.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.528+0000 7f14f48dd640 1 -- 172.21.15.38:0/2333442125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 msgr2=0x7f14f0158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:25.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.528+0000 7f14f48dd640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 0x7f14f0158f30 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f14e0011e30 tx=0x7f14e000f580 comp rx=0 tx=0).stop 2024-04-25T08:36:25.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.528+0000 7f14f48dd640 1 -- 172.21.15.38:0/2333442125 shutdown_connections 2024-04-25T08:36:25.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.528+0000 7f14f48dd640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14f0157060 0x7f14f01574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.528+0000 7f14f48dd640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f14d006b4d0 0x7f14d006d990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.528+0000 7f14f48dd640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14f0156990 0x7f14f01f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.528+0000 7f14f48dd640 1 --2- 172.21.15.38:0/2333442125 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14d8004800 0x7f14f0158f30 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.528+0000 7f14f48dd640 1 -- 172.21.15.38:0/2333442125 >> 172.21.15.38:0/2333442125 conn(0x7f14f0066d10 msgr2=0x7f14f00d3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:25.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.528+0000 7f14f48dd640 1 -- 172.21.15.38:0/2333442125 shutdown_connections 2024-04-25T08:36:25.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.528+0000 7f14f48dd640 1 -- 172.21.15.38:0/2333442125 wait complete. 2024-04-25T08:36:25.536 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:36:25.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.651+0000 7f41afaee640 1 Processor -- start 2024-04-25T08:36:25.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.651+0000 7f41afaee640 1 -- start start 2024-04-25T08:36:25.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.652+0000 7f41afaee640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f41a80d0320 0x7f41a80d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:25.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.652+0000 7f41afaee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41a80d0700 0x7f41a80d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:25.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.652+0000 7f41afaee640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a80d2330 con 0x7f41a80d1e90 2024-04-25T08:36:25.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.652+0000 7f41afaee640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a80d2610 con 0x7f41a80d0320 2024-04-25T08:36:25.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.652+0000 7f41afaee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a80d28f0 con 0x7f41a80d0700 2024-04-25T08:36:25.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.652+0000 7f41ad88a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f41a80d0320 0x7f41a80d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:25.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.652+0000 7f41ae08b640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f41a80d1e90 0x7f41a80b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:33296/0 (socket says 172.21.15.38:33296) 2024-04-25T08:36:25.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.652+0000 7f41ad089640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41a80d0700 0x7f41a80d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:25.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.652+0000 7f41ae08b640 1 -- 172.21.15.38:0/2965883802 learned_addr learned my addr 172.21.15.38:0/2965883802 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:25.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.653+0000 7f41ad089640 1 -- 172.21.15.38:0/2965883802 >> v2:172.21.15.107:3300/0 conn(0x7f41a80d0320 msgr2=0x7f41a80d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:25.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.653+0000 7f41ad089640 1 --2- 172.21.15.38:0/2965883802 >> v2:172.21.15.107:3300/0 conn(0x7f41a80d0320 0x7f41a80d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.653+0000 7f41ad089640 1 -- 172.21.15.38:0/2965883802 >> v1:172.21.15.107:6789/0 conn(0x7f41a80d1e90 legacy=0x7f41a80b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:25.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.653+0000 7f41ad089640 1 -- 172.21.15.38:0/2965883802 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a80d2bd0 con 0x7f41a80d0700 2024-04-25T08:36:25.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.653+0000 7f41ad88a640 1 --2- 172.21.15.38:0/2965883802 >> v2:172.21.15.107:3300/0 conn(0x7f41a80d0320 0x7f41a80d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:25.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.653+0000 7f41ad089640 1 --2- 172.21.15.38:0/2965883802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41a80d0700 0x7f41a80d1950 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f419c0097a0 tx=0x7f419c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fecd4291b069cbb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:25.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.654+0000 7f41ac888640 1 -- 172.21.15.38:0/2965883802 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c037070 con 0x7f41a80d0700 2024-04-25T08:36:25.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.654+0000 7f41ac888640 1 -- 172.21.15.38:0/2965883802 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c032440 con 0x7f41a80d0700 2024-04-25T08:36:25.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.654+0000 7f41ac888640 1 -- 172.21.15.38:0/2965883802 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c032810 con 0x7f41a80d0700 2024-04-25T08:36:25.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.654+0000 7f41afaee640 1 -- 172.21.15.38:0/2965883802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41a80d0700 msgr2=0x7f41a80d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:25.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.654+0000 7f41afaee640 1 --2- 172.21.15.38:0/2965883802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41a80d0700 0x7f41a80d1950 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f419c0097a0 tx=0x7f419c02a210 comp rx=0 tx=0).stop 2024-04-25T08:36:25.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.654+0000 7f41afaee640 1 -- 172.21.15.38:0/2965883802 shutdown_connections 2024-04-25T08:36:25.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.654+0000 7f41afaee640 1 --2- 172.21.15.38:0/2965883802 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41a80d0700 0x7f41a80d1950 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.654+0000 7f41afaee640 1 --2- 172.21.15.38:0/2965883802 >> v2:172.21.15.107:3300/0 conn(0x7f41a80d0320 0x7f41a80d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.654+0000 7f41afaee640 1 -- 172.21.15.38:0/2965883802 >> 172.21.15.38:0/2965883802 conn(0x7f41a80668f0 msgr2=0x7f41a8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:25.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.654+0000 7f41afaee640 1 -- 172.21.15.38:0/2965883802 shutdown_connections 2024-04-25T08:36:25.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.654+0000 7f41afaee640 1 -- 172.21.15.38:0/2965883802 wait complete. 2024-04-25T08:36:25.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.655+0000 7f41afaee640 1 Processor -- start 2024-04-25T08:36:25.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.655+0000 7f41afaee640 1 -- start start 2024-04-25T08:36:25.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.656+0000 7f41afaee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a80d0320 0x7f41a81f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:25.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.656+0000 7f41afaee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41a80d0700 0x7f41a80b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:25.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.656+0000 7f41afaee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41a80d1e90 0x7f41a80b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:25.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.656+0000 7f41afaee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a80d0b40 con 0x7f41a80d0320 2024-04-25T08:36:25.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.656+0000 7f41afaee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a8161210 con 0x7f41a80d1e90 2024-04-25T08:36:25.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.656+0000 7f41afaee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a8155ed0 con 0x7f41a80d0700 2024-04-25T08:36:25.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.656+0000 7f41ad88a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a80d0320 0x7f41a81f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:25.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.656+0000 7f41ad88a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a80d0320 0x7f41a81f76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56132/0 (socket says 172.21.15.38:56132) 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.656+0000 7f41ad88a640 1 -- 172.21.15.38:0/3251296432 learned_addr learned my addr 172.21.15.38:0/3251296432 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.656+0000 7f41ad089640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41a80d0700 0x7f41a80b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.656+0000 7f41ae08b640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41a80d1e90 0x7f41a80b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.657+0000 7f41ad88a640 1 -- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41a80d0700 msgr2=0x7f41a80b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.657+0000 7f41ad88a640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41a80d0700 0x7f41a80b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.657+0000 7f41ad88a640 1 -- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41a80d1e90 msgr2=0x7f41a80b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.657+0000 7f41ad88a640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41a80d1e90 0x7f41a80b3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.657+0000 7f41ad88a640 1 -- 172.21.15.38:0/3251296432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a81f7cc0 con 0x7f41a80d0320 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.657+0000 7f41ad089640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41a80d0700 0x7f41a80b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.657+0000 7f41ad88a640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a80d0320 0x7f41a81f76e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4198009f60 tx=0x7f4198009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.657+0000 7f41a67fc640 1 -- 172.21.15.38:0/3251296432 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198018020 con 0x7f41a80d0320 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.657+0000 7f41a67fc640 1 -- 172.21.15.38:0/3251296432 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4198014680 con 0x7f41a80d0320 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.657+0000 7f41a67fc640 1 -- 172.21.15.38:0/3251296432 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198014a50 con 0x7f41a80d0320 2024-04-25T08:36:25.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.657+0000 7f41afaee640 1 -- 172.21.15.38:0/3251296432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a81f7fa0 con 0x7f41a80d0320 2024-04-25T08:36:25.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.657+0000 7f41afaee640 1 -- 172.21.15.38:0/3251296432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a814a800 con 0x7f41a80d0320 2024-04-25T08:36:25.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.658+0000 7f41afaee640 1 -- 172.21.15.38:0/3251296432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a8068e80 con 0x7f41a80d0320 2024-04-25T08:36:25.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.659+0000 7f41a67fc640 1 -- 172.21.15.38:0/3251296432 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4198005d10 con 0x7f41a80d0320 2024-04-25T08:36:25.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.659+0000 7f41a67fc640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f419406b4d0 0x7f419406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:25.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.659+0000 7f41a67fc640 1 -- 172.21.15.38:0/3251296432 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f419808b0d0 con 0x7f41a80d0320 2024-04-25T08:36:25.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.659+0000 7f41ad089640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f419406b4d0 0x7f419406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:25.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.660+0000 7f41ad089640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f419406b4d0 0x7f419406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f419c034000 tx=0x7f419c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:25.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.663+0000 7f41a67fc640 1 -- 172.21.15.38:0/3251296432 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4198022030 con 0x7f41a80d0320 2024-04-25T08:36:25.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:25.846+0000 7f41afaee640 1 -- 172.21.15.38:0/3251296432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f41a80b3050 con 0x7f41a80d0320 2024-04-25T08:36:26.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.529+0000 7f41a67fc640 1 -- 172.21.15.38:0/3251296432 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4198010040 con 0x7f41a80d0320 2024-04-25T08:36:26.523 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-04-25T08:36:26.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.532+0000 7f41afaee640 1 -- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f419406b4d0 msgr2=0x7f419406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:26.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.532+0000 7f41afaee640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f419406b4d0 0x7f419406d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f419c034000 tx=0x7f419c02e000 comp rx=0 tx=0).stop 2024-04-25T08:36:26.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.532+0000 7f41afaee640 1 -- 172.21.15.38:0/3251296432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a80d0320 msgr2=0x7f41a81f76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:26.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.532+0000 7f41afaee640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a80d0320 0x7f41a81f76e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f4198009f60 tx=0x7f4198009f90 comp rx=0 tx=0).stop 2024-04-25T08:36:26.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.533+0000 7f41afaee640 1 -- 172.21.15.38:0/3251296432 shutdown_connections 2024-04-25T08:36:26.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.533+0000 7f41afaee640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41a80d1e90 0x7f41a80b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.533+0000 7f41afaee640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f419406b4d0 0x7f419406d990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.534+0000 7f41afaee640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41a80d0700 0x7f41a80b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.534+0000 7f41afaee640 1 --2- 172.21.15.38:0/3251296432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a80d0320 0x7f41a81f76e0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.534+0000 7f41afaee640 1 -- 172.21.15.38:0/3251296432 >> 172.21.15.38:0/3251296432 conn(0x7f41a80668f0 msgr2=0x7f41a80d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:26.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.546+0000 7f41afaee640 1 -- 172.21.15.38:0/3251296432 shutdown_connections 2024-04-25T08:36:26.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.548+0000 7f41afaee640 1 -- 172.21.15.38:0/3251296432 wait complete. 2024-04-25T08:36:26.558 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:36:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.673+0000 7f33e6e07640 1 Processor -- start 2024-04-25T08:36:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.673+0000 7f33e6e07640 1 -- start start 2024-04-25T08:36:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.673+0000 7f33e6e07640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f33e009a180 0x7f33e009a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.673+0000 7f33e6e07640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33e0162390 0x7f33e00cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.673+0000 7f33e6e07640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e015f300 con 0x7f33e015df30 2024-04-25T08:36:26.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.673+0000 7f33e6e07640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e015e310 con 0x7f33e0162390 2024-04-25T08:36:26.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.673+0000 7f33e6e07640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e009aeb0 con 0x7f33e009a180 2024-04-25T08:36:26.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.674+0000 7f33e53a4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33e0162390 0x7f33e00cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:26.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.674+0000 7f33e53a4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33e0162390 0x7f33e00cf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56140/0 (socket says 172.21.15.38:56140) 2024-04-25T08:36:26.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.674+0000 7f33e53a4640 1 -- 172.21.15.38:0/1224210785 learned_addr learned my addr 172.21.15.38:0/1224210785 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:26.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.674+0000 7f33dffff640 1 --2- 172.21.15.38:0/1224210785 >> v2:172.21.15.107:3300/0 conn(0x7f33e009a180 0x7f33e009a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:26.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.674+0000 7f33e53a4640 1 -- 172.21.15.38:0/1224210785 >> v2:172.21.15.107:3300/0 conn(0x7f33e009a180 msgr2=0x7f33e009a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:26.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.674+0000 7f33e53a4640 1 --2- 172.21.15.38:0/1224210785 >> v2:172.21.15.107:3300/0 conn(0x7f33e009a180 0x7f33e009a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.674+0000 7f33e53a4640 1 -- 172.21.15.38:0/1224210785 >> v1:172.21.15.107:6789/0 conn(0x7f33e015df30 legacy=0x7f33e0099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:26.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.674+0000 7f33e53a4640 1 -- 172.21.15.38:0/1224210785 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e015e540 con 0x7f33e0162390 2024-04-25T08:36:26.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.674+0000 7f33dffff640 1 --2- 172.21.15.38:0/1224210785 >> v2:172.21.15.107:3300/0 conn(0x7f33e009a180 0x7f33e009a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.675+0000 7f33e53a4640 1 --2- 172.21.15.38:0/1224210785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33e0162390 0x7f33e00cf8f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f33e0072410 tx=0x7f33d802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83590f888aed1a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.675+0000 7f33df7fe640 1 -- 172.21.15.38:0/1224210785 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8036070 con 0x7f33e0162390 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.675+0000 7f33df7fe640 1 -- 172.21.15.38:0/1224210785 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d8031600 con 0x7f33e0162390 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.675+0000 7f33df7fe640 1 -- 172.21.15.38:0/1224210785 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d80319d0 con 0x7f33e0162390 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.675+0000 7f33e6e07640 1 -- 172.21.15.38:0/1224210785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33e0162390 msgr2=0x7f33e00cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.675+0000 7f33e6e07640 1 --2- 172.21.15.38:0/1224210785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33e0162390 0x7f33e00cf8f0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f33e0072410 tx=0x7f33d802a6f0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.675+0000 7f33e6e07640 1 -- 172.21.15.38:0/1224210785 shutdown_connections 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.675+0000 7f33e6e07640 1 --2- 172.21.15.38:0/1224210785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33e0162390 0x7f33e00cf8f0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.675+0000 7f33e6e07640 1 --2- 172.21.15.38:0/1224210785 >> v2:172.21.15.107:3300/0 conn(0x7f33e009a180 0x7f33e009a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.675+0000 7f33e6e07640 1 -- 172.21.15.38:0/1224210785 >> 172.21.15.38:0/1224210785 conn(0x7f33e00b3110 msgr2=0x7f33e00b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.675+0000 7f33e6e07640 1 -- 172.21.15.38:0/1224210785 shutdown_connections 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.675+0000 7f33e6e07640 1 -- 172.21.15.38:0/1224210785 wait complete. 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.676+0000 7f33e6e07640 1 Processor -- start 2024-04-25T08:36:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.676+0000 7f33e6e07640 1 -- start start 2024-04-25T08:36:26.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.677+0000 7f33e6e07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33e009a180 0x7f33e00c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:26.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.677+0000 7f33e6e07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33e015df30 0x7f33e00c7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:26.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.677+0000 7f33e6e07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33e0162390 0x7f33e01f2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:26.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.677+0000 7f33e6e07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e015eb00 con 0x7f33e015df30 2024-04-25T08:36:26.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.677+0000 7f33e6e07640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e015ee70 con 0x7f33e009a180 2024-04-25T08:36:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.677+0000 7f33e6e07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e01627a0 con 0x7f33e0162390 2024-04-25T08:36:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.677+0000 7f33dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33e015df30 0x7f33e00c7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.677+0000 7f33dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33e015df30 0x7f33e00c7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56152/0 (socket says 172.21.15.38:56152) 2024-04-25T08:36:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33dffff640 1 -- 172.21.15.38:0/2412373408 learned_addr learned my addr 172.21.15.38:0/2412373408 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33e4ba3640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33e009a180 0x7f33e00c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33dffff640 1 -- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33e0162390 msgr2=0x7f33e01f2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33dffff640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33e0162390 0x7f33e01f2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33dffff640 1 -- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33e009a180 msgr2=0x7f33e00c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33dffff640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33e009a180 0x7f33e00c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33dffff640 1 -- 172.21.15.38:0/2412373408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e00c7f80 con 0x7f33e015df30 2024-04-25T08:36:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33dffff640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33e015df30 0x7f33e00c7750 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f33cc00c7f0 tx=0x7f33cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33dd7fa640 1 -- 172.21.15.38:0/2412373408 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc015070 con 0x7f33e015df30 2024-04-25T08:36:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33dd7fa640 1 -- 172.21.15.38:0/2412373408 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc007820 con 0x7f33e015df30 2024-04-25T08:36:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33e6e07640 1 -- 172.21.15.38:0/2412373408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e00c8260 con 0x7f33e015df30 2024-04-25T08:36:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33dd7fa640 1 -- 172.21.15.38:0/2412373408 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc007bd0 con 0x7f33e015df30 2024-04-25T08:36:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.678+0000 7f33e6e07640 1 -- 172.21.15.38:0/2412373408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e00b58b0 con 0x7f33e015df30 2024-04-25T08:36:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.679+0000 7f33e6e07640 1 -- 172.21.15.38:0/2412373408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e0068e80 con 0x7f33e015df30 2024-04-25T08:36:26.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.680+0000 7f33dd7fa640 1 -- 172.21.15.38:0/2412373408 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33cc018070 con 0x7f33e015df30 2024-04-25T08:36:26.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.680+0000 7f33dd7fa640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33c406b4d0 0x7f33c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:26.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.680+0000 7f33dd7fa640 1 -- 172.21.15.38:0/2412373408 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f33cc08a450 con 0x7f33e015df30 2024-04-25T08:36:26.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.681+0000 7f33e4ba3640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33c406b4d0 0x7f33c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:26.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.681+0000 7f33e4ba3640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33c406b4d0 0x7f33c406d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f33d4003d90 tx=0x7f33d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:26.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.685+0000 7f33dd7fa640 1 -- 172.21.15.38:0/2412373408 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33cc0591a0 con 0x7f33e015df30 2024-04-25T08:36:26.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:26.902+0000 7f33e6e07640 1 -- 172.21.15.38:0/2412373408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f33e00b3de0 con 0x7f33e015df30 2024-04-25T08:36:27.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.548+0000 7f33dd7fa640 1 -- 172.21.15.38:0/2412373408 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v47) v1 ==== 174+0+0 (secure 0 0 0) 0x7f33cc058aa0 con 0x7f33e015df30 2024-04-25T08:36:27.543 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T08:36:27.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.558+0000 7f33e6e07640 1 -- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33c406b4d0 msgr2=0x7f33c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.558+0000 7f33e6e07640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33c406b4d0 0x7f33c406d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f33d4003d90 tx=0x7f33d4009040 comp rx=0 tx=0).stop 2024-04-25T08:36:27.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.558+0000 7f33e6e07640 1 -- 172.21.15.38:0/2412373408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33e015df30 msgr2=0x7f33e00c7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.558+0000 7f33e6e07640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33e015df30 0x7f33e00c7750 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f33cc00c7f0 tx=0x7f33cc00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.559+0000 7f33e6e07640 1 -- 172.21.15.38:0/2412373408 shutdown_connections 2024-04-25T08:36:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.559+0000 7f33e6e07640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33c406b4d0 0x7f33c406d990 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.559+0000 7f33e6e07640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33e0162390 0x7f33e01f2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.559+0000 7f33e6e07640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33e015df30 0x7f33e00c7750 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.559+0000 7f33e6e07640 1 --2- 172.21.15.38:0/2412373408 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33e009a180 0x7f33e00c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.559+0000 7f33e6e07640 1 -- 172.21.15.38:0/2412373408 >> 172.21.15.38:0/2412373408 conn(0x7f33e00b3110 msgr2=0x7f33e00d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.560+0000 7f33e6e07640 1 -- 172.21.15.38:0/2412373408 shutdown_connections 2024-04-25T08:36:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.562+0000 7f33e6e07640 1 -- 172.21.15.38:0/2412373408 wait complete. 2024-04-25T08:36:27.579 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.691+0000 7fa6b6c3a640 1 Processor -- start 2024-04-25T08:36:27.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.691+0000 7fa6b6c3a640 1 -- start start 2024-04-25T08:36:27.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6b6c3a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6b006c110 0x7fa6b006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6b6c3a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa6b015df50 0x7fa6b00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6b6c3a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6b006ca30 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6b6c3a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6b00cf2d0 con 0x7fa6b015df50 2024-04-25T08:36:27.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6b6c3a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00ceff0 con 0x7fa6b006c110 2024-04-25T08:36:27.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6affff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa6b015df50 0x7fa6b00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6affff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa6b015df50 0x7fa6b00ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56158/0 (socket says 172.21.15.38:56158) 2024-04-25T08:36:27.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6affff640 1 -- 172.21.15.38:0/3300027378 learned_addr learned my addr 172.21.15.38:0/3300027378 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:27.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6b49d6640 1 --2- 172.21.15.38:0/3300027378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6b006c110 0x7fa6b006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6affff640 1 -- 172.21.15.38:0/3300027378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6b006c110 msgr2=0x7fa6b006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:27.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6affff640 1 --2- 172.21.15.38:0/3300027378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6b006c110 0x7fa6b006c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6affff640 1 -- 172.21.15.38:0/3300027378 >> v1:172.21.15.107:6789/0 conn(0x7fa6b00ce2e0 legacy=0x7fa6b00d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.692+0000 7fa6affff640 1 -- 172.21.15.38:0/3300027378 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b00d1f70 con 0x7fa6b015df50 2024-04-25T08:36:27.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.693+0000 7fa6affff640 1 --2- 172.21.15.38:0/3300027378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6b015df50 0x7fa6b00ce7a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009870 tx=0x7fa69c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab592529a5cbb772 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:27.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.693+0000 7fa6af7fe640 1 -- 172.21.15.38:0/3300027378 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c037070 con 0x7fa6b015df50 2024-04-25T08:36:27.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.693+0000 7fa6af7fe640 1 -- 172.21.15.38:0/3300027378 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c032440 con 0x7fa6b015df50 2024-04-25T08:36:27.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.693+0000 7fa6af7fe640 1 -- 172.21.15.38:0/3300027378 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c032810 con 0x7fa6b015df50 2024-04-25T08:36:27.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.694+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/3300027378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6b015df50 msgr2=0x7fa6b00ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.694+0000 7fa6b6c3a640 1 --2- 172.21.15.38:0/3300027378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6b015df50 0x7fa6b00ce7a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fa69c009870 tx=0x7fa69c02a270 comp rx=0 tx=0).stop 2024-04-25T08:36:27.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.694+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/3300027378 shutdown_connections 2024-04-25T08:36:27.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.694+0000 7fa6b6c3a640 1 --2- 172.21.15.38:0/3300027378 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6b015df50 0x7fa6b00ce7a0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.694+0000 7fa6b6c3a640 1 --2- 172.21.15.38:0/3300027378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6b006c110 0x7fa6b006c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.694+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/3300027378 >> 172.21.15.38:0/3300027378 conn(0x7fa6b00d3fb0 msgr2=0x7fa6b00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.694+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/3300027378 shutdown_connections 2024-04-25T08:36:27.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.694+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/3300027378 wait complete. 2024-04-25T08:36:27.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.695+0000 7fa6b6c3a640 1 Processor -- start 2024-04-25T08:36:27.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.695+0000 7fa6b6c3a640 1 -- start start 2024-04-25T08:36:27.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6b6c3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b006c110 0x7fa6b014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6b6c3a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6b00ce2e0 0x7fa6b00c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6b6c3a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6b015df50 0x7fa6b0160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6b6c3a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00cece0 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6b6c3a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00d2530 con 0x7fa6b006c110 2024-04-25T08:36:27.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6b6c3a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b00d3160 con 0x7fa6b015df50 2024-04-25T08:36:27.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6b00ce2e0 0x7fa6b00c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6b00ce2e0 0x7fa6b00c71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56162/0 (socket says 172.21.15.38:56162) 2024-04-25T08:36:27.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6affff640 1 -- 172.21.15.38:0/1872693520 learned_addr learned my addr 172.21.15.38:0/1872693520 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:27.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6b49d6640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b006c110 0x7fa6b014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6b51d7640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6b015df50 0x7fa6b0160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6affff640 1 -- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6b015df50 msgr2=0x7fa6b0160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6affff640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6b015df50 0x7fa6b0160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6affff640 1 -- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b006c110 msgr2=0x7fa6b014ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6affff640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b006c110 0x7fa6b014ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6affff640 1 -- 172.21.15.38:0/1872693520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b00cacf0 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.696+0000 7fa6b51d7640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6b015df50 0x7fa6b0160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.697+0000 7fa6affff640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6b00ce2e0 0x7fa6b00c71a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0097c0 tx=0x7fa69c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.697+0000 7fa6ad7fa640 1 -- 172.21.15.38:0/1872693520 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c041020 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.697+0000 7fa6ad7fa640 1 -- 172.21.15.38:0/1872693520 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c03d620 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.697+0000 7fa6ad7fa640 1 -- 172.21.15.38:0/1872693520 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c037040 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.697+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/1872693520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b00cafd0 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.697+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/1872693520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b00cb590 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.698+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/1872693520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b015d790 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.698+0000 7fa6ad7fa640 1 -- 172.21.15.38:0/1872693520 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa69c044020 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.699+0000 7fa6ad7fa640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa69406b4d0 0x7fa69406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.699+0000 7fa6ad7fa640 1 -- 172.21.15.38:0/1872693520 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa69c0aaa70 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.699+0000 7fa6b49d6640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa69406b4d0 0x7fa69406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.699+0000 7fa6b49d6640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa69406b4d0 0x7fa69406d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009ad0 tx=0x7fa6a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:27.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.703+0000 7fa6ad7fa640 1 -- 172.21.15.38:0/1872693520 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6b015d790 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.884+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/1872693520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6b0068e80 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.885+0000 7fa6ad7fa640 1 -- 172.21.15.38:0/1872693520 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16946 (secure 0 0 0) 0x7fa69c04b030 con 0x7fa6b00ce2e0 2024-04-25T08:36:27.879 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:27.879 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":81,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:36:26.903752+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:25.176504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:25.877954+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:27.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.887+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa69406b4d0 msgr2=0x7fa69406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.887+0000 7fa6b6c3a640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa69406b4d0 0x7fa69406d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009ad0 tx=0x7fa6a4008040 comp rx=0 tx=0).stop 2024-04-25T08:36:27.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.887+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/1872693520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6b00ce2e0 msgr2=0x7fa6b00c71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.887+0000 7fa6b6c3a640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6b00ce2e0 0x7fa6b00c71a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0097c0 tx=0x7fa69c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.887+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/1872693520 shutdown_connections 2024-04-25T08:36:27.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.887+0000 7fa6b6c3a640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa69406b4d0 0x7fa69406d990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.887+0000 7fa6b6c3a640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa6b015df50 0x7fa6b0160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.887+0000 7fa6b6c3a640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6b00ce2e0 0x7fa6b00c71a0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.887+0000 7fa6b6c3a640 1 --2- 172.21.15.38:0/1872693520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b006c110 0x7fa6b014ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.887+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/1872693520 >> 172.21.15.38:0/1872693520 conn(0x7fa6b00d3fb0 msgr2=0x7fa6b00b3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:27.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.887+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/1872693520 shutdown_connections 2024-04-25T08:36:27.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:27.887+0000 7fa6b6c3a640 1 -- 172.21.15.38:0/1872693520 wait complete. 2024-04-25T08:36:27.893 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:27.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.000+0000 7fd297934640 1 Processor -- start 2024-04-25T08:36:27.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.001+0000 7fd297934640 1 -- start start 2024-04-25T08:36:27.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.001+0000 7fd297934640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd2900b4b50 0x7fd2900ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.001+0000 7fd297934640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd290144b70 0x7fd2900ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:27.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd297934640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2900b50c0 con 0x7fd290144760 2024-04-25T08:36:27.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd297934640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29015f380 con 0x7fd2900b4b50 2024-04-25T08:36:27.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd297934640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29015f660 con 0x7fd290144b70 2024-04-25T08:36:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd294ecf640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd2900b4b50 0x7fd2900ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd294ecf640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd2900b4b50 0x7fd2900ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56172/0 (socket says 172.21.15.38:56172) 2024-04-25T08:36:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd294ecf640 1 -- 172.21.15.38:0/1645974908 learned_addr learned my addr 172.21.15.38:0/1645974908 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd2956d0640 1 --1- 172.21.15.38:0/1645974908 >> v1:172.21.15.38:6789/0 conn(0x7fd290144760 0x7fd29015faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37510/0 (socket says 172.21.15.38:37510) 2024-04-25T08:36:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd295ed1640 1 --2- 172.21.15.38:0/1645974908 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd290144b70 0x7fd2900ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd294ecf640 1 -- 172.21.15.38:0/1645974908 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd290144b70 msgr2=0x7fd2900ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd294ecf640 1 --2- 172.21.15.38:0/1645974908 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd290144b70 0x7fd2900ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd294ecf640 1 -- 172.21.15.38:0/1645974908 >> v1:172.21.15.38:6789/0 conn(0x7fd290144760 legacy=0x7fd29015faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd294ecf640 1 -- 172.21.15.38:0/1645974908 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900b53a0 con 0x7fd2900b4b50 2024-04-25T08:36:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd28ffff640 1 -- 172.21.15.38:0/1645974908 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034749923 0 0) 0x7fd2900b50c0 con 0x7fd290144760 2024-04-25T08:36:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.002+0000 7fd295ed1640 1 --2- 172.21.15.38:0/1645974908 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd290144b70 0x7fd2900ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:27.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.003+0000 7fd294ecf640 1 --2- 172.21.15.38:0/1645974908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2900b4b50 0x7fd2900ce360 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd284009870 tx=0x7fd28402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b35c75ea291fe7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.003+0000 7fd28ffff640 1 -- 172.21.15.38:0/1645974908 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284037070 con 0x7fd2900b4b50 2024-04-25T08:36:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.003+0000 7fd28ffff640 1 -- 172.21.15.38:0/1645974908 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd284032440 con 0x7fd2900b4b50 2024-04-25T08:36:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.003+0000 7fd28ffff640 1 -- 172.21.15.38:0/1645974908 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284032810 con 0x7fd2900b4b50 2024-04-25T08:36:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.003+0000 7fd297934640 1 -- 172.21.15.38:0/1645974908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2900b4b50 msgr2=0x7fd2900ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.003+0000 7fd297934640 1 --2- 172.21.15.38:0/1645974908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2900b4b50 0x7fd2900ce360 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd284009870 tx=0x7fd28402a1c0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.003+0000 7fd297934640 1 -- 172.21.15.38:0/1645974908 shutdown_connections 2024-04-25T08:36:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.003+0000 7fd297934640 1 --2- 172.21.15.38:0/1645974908 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd290144b70 0x7fd2900ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.003+0000 7fd297934640 1 --2- 172.21.15.38:0/1645974908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2900b4b50 0x7fd2900ce360 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.003+0000 7fd297934640 1 -- 172.21.15.38:0/1645974908 >> 172.21.15.38:0/1645974908 conn(0x7fd29006b210 msgr2=0x7fd29006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.004+0000 7fd297934640 1 -- 172.21.15.38:0/1645974908 shutdown_connections 2024-04-25T08:36:28.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.004+0000 7fd297934640 1 -- 172.21.15.38:0/1645974908 wait complete. 2024-04-25T08:36:28.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.005+0000 7fd297934640 1 Processor -- start 2024-04-25T08:36:28.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.005+0000 7fd297934640 1 -- start start 2024-04-25T08:36:28.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.005+0000 7fd297934640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2900b4b50 0x7fd29015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.005+0000 7fd297934640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd290144b70 0x7fd29015d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.005+0000 7fd297934640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2900cd630 0x7fd2900cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.005+0000 7fd297934640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd290145280 con 0x7fd290144b70 2024-04-25T08:36:28.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.005+0000 7fd297934640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd290160200 con 0x7fd2900b4b50 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.005+0000 7fd297934640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd290145400 con 0x7fd2900cd630 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd294ecf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd290144b70 0x7fd29015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd294ecf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd290144b70 0x7fd29015d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56176/0 (socket says 172.21.15.38:56176) 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd294ecf640 1 -- 172.21.15.38:0/2899533159 learned_addr learned my addr 172.21.15.38:0/2899533159 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd295ed1640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2900cd630 0x7fd2900cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd2956d0640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2900b4b50 0x7fd29015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd294ecf640 1 -- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2900cd630 msgr2=0x7fd2900cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd294ecf640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2900cd630 0x7fd2900cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd294ecf640 1 -- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2900b4b50 msgr2=0x7fd29015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd294ecf640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2900b4b50 0x7fd29015d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd294ecf640 1 -- 172.21.15.38:0/2899533159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2900cc1f0 con 0x7fd290144b70 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd294ecf640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd290144b70 0x7fd29015d800 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd284035040 tx=0x7fd2840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd28dffb640 1 -- 172.21.15.38:0/2899533159 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284037040 con 0x7fd290144b70 2024-04-25T08:36:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd28dffb640 1 -- 172.21.15.38:0/2899533159 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd284004230 con 0x7fd290144b70 2024-04-25T08:36:28.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd297934640 1 -- 172.21.15.38:0/2899533159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2900cc4d0 con 0x7fd290144b70 2024-04-25T08:36:28.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd28dffb640 1 -- 172.21.15.38:0/2899533159 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284003610 con 0x7fd290144b70 2024-04-25T08:36:28.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.006+0000 7fd297934640 1 -- 172.21.15.38:0/2899533159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2900cca90 con 0x7fd290144b70 2024-04-25T08:36:28.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.007+0000 7fd297934640 1 -- 172.21.15.38:0/2899533159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd290068e80 con 0x7fd290144b70 2024-04-25T08:36:28.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.008+0000 7fd28dffb640 1 -- 172.21.15.38:0/2899533159 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd28402ad10 con 0x7fd290144b70 2024-04-25T08:36:28.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.008+0000 7fd28dffb640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd27c06b4d0 0x7fd27c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.008+0000 7fd28dffb640 1 -- 172.21.15.38:0/2899533159 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2840ab260 con 0x7fd290144b70 2024-04-25T08:36:28.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.008+0000 7fd2956d0640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd27c06b4d0 0x7fd27c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.009+0000 7fd2956d0640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd27c06b4d0 0x7fd27c06d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd2800012e0 tx=0x7fd280005eb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:28.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.012+0000 7fd28dffb640 1 -- 172.21.15.38:0/2899533159 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd284079fd0 con 0x7fd290144b70 2024-04-25T08:36:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.193+0000 7fd297934640 1 -- 172.21.15.38:0/2899533159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd29006df60 con 0x7fd290144b70 2024-04-25T08:36:28.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.194+0000 7fd28dffb640 1 -- 172.21.15.38:0/2899533159 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16946 (secure 0 0 0) 0x7fd2840798d0 con 0x7fd290144b70 2024-04-25T08:36:28.189 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:28.190 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":81,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:36:26.903752+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:25.176504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:25.877954+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:28.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.196+0000 7fd297934640 1 -- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd27c06b4d0 msgr2=0x7fd27c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.196+0000 7fd297934640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd27c06b4d0 0x7fd27c06d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd2800012e0 tx=0x7fd280005eb0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.196+0000 7fd297934640 1 -- 172.21.15.38:0/2899533159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd290144b70 msgr2=0x7fd29015d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.196+0000 7fd297934640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd290144b70 0x7fd29015d800 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd284035040 tx=0x7fd2840046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.196+0000 7fd297934640 1 -- 172.21.15.38:0/2899533159 shutdown_connections 2024-04-25T08:36:28.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.196+0000 7fd297934640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd27c06b4d0 0x7fd27c06d990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.196+0000 7fd297934640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2900cd630 0x7fd2900cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.196+0000 7fd297934640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd290144b70 0x7fd29015d800 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.196+0000 7fd297934640 1 --2- 172.21.15.38:0/2899533159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2900b4b50 0x7fd29015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.196+0000 7fd297934640 1 -- 172.21.15.38:0/2899533159 >> 172.21.15.38:0/2899533159 conn(0x7fd29006b210 msgr2=0x7fd2900ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:28.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.197+0000 7fd297934640 1 -- 172.21.15.38:0/2899533159 shutdown_connections 2024-04-25T08:36:28.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.197+0000 7fd297934640 1 -- 172.21.15.38:0/2899533159 wait complete. 2024-04-25T08:36:28.203 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:36:28.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.313+0000 7f60ee1ce640 1 Processor -- start 2024-04-25T08:36:28.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.313+0000 7f60ee1ce640 1 -- start start 2024-04-25T08:36:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.314+0000 7f60ee1ce640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f60e80d0320 0x7f60e80d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.314+0000 7f60ee1ce640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f60e80d0700 0x7f60e80d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.314+0000 7f60ee1ce640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e80d2330 con 0x7f60e80d1e90 2024-04-25T08:36:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.314+0000 7f60ee1ce640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e80d2610 con 0x7f60e80d0700 2024-04-25T08:36:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.314+0000 7f60ee1ce640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e80d28f0 con 0x7f60e80d0320 2024-04-25T08:36:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.314+0000 7f60e6ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f60e80d0700 0x7f60e80d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.314+0000 7f60e6ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f60e80d0700 0x7f60e80d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56180/0 (socket says 172.21.15.38:56180) 2024-04-25T08:36:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.314+0000 7f60e6ffd640 1 -- 172.21.15.38:0/2653523115 learned_addr learned my addr 172.21.15.38:0/2653523115 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.314+0000 7f60e77fe640 1 --2- 172.21.15.38:0/2653523115 >> v2:172.21.15.107:3300/0 conn(0x7f60e80d0320 0x7f60e80d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.314+0000 7f60e6ffd640 1 -- 172.21.15.38:0/2653523115 >> v2:172.21.15.107:3300/0 conn(0x7f60e80d0320 msgr2=0x7f60e80d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.315+0000 7f60e6ffd640 1 --2- 172.21.15.38:0/2653523115 >> v2:172.21.15.107:3300/0 conn(0x7f60e80d0320 0x7f60e80d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.315+0000 7f60e6ffd640 1 -- 172.21.15.38:0/2653523115 >> v1:172.21.15.38:6789/0 conn(0x7f60e80d1e90 legacy=0x7f60e80b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.315+0000 7f60e6ffd640 1 -- 172.21.15.38:0/2653523115 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e80d2bd0 con 0x7f60e80d0700 2024-04-25T08:36:28.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.315+0000 7f60e67fc640 1 -- 172.21.15.38:0/2653523115 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058858708 0 0) 0x7f60e80d2330 con 0x7f60e80d1e90 2024-04-25T08:36:28.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.315+0000 7f60e6ffd640 1 --2- 172.21.15.38:0/2653523115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e80d0700 0x7f60e80d1950 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f60e8069100 tx=0x7f60dc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9b021e82e6a272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.315+0000 7f60e67fc640 1 -- 172.21.15.38:0/2653523115 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc037070 con 0x7f60e80d0700 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.315+0000 7f60e67fc640 1 -- 172.21.15.38:0/2653523115 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc02ae50 con 0x7f60e80d0700 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.315+0000 7f60e67fc640 1 -- 172.21.15.38:0/2653523115 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc032860 con 0x7f60e80d0700 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.316+0000 7f60ee1ce640 1 -- 172.21.15.38:0/2653523115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e80d0700 msgr2=0x7f60e80d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.316+0000 7f60ee1ce640 1 --2- 172.21.15.38:0/2653523115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e80d0700 0x7f60e80d1950 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f60e8069100 tx=0x7f60dc02a300 comp rx=0 tx=0).stop 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.316+0000 7f60ee1ce640 1 -- 172.21.15.38:0/2653523115 shutdown_connections 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.316+0000 7f60ee1ce640 1 --2- 172.21.15.38:0/2653523115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e80d0700 0x7f60e80d1950 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.316+0000 7f60ee1ce640 1 --2- 172.21.15.38:0/2653523115 >> v2:172.21.15.107:3300/0 conn(0x7f60e80d0320 0x7f60e80d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.316+0000 7f60ee1ce640 1 -- 172.21.15.38:0/2653523115 >> 172.21.15.38:0/2653523115 conn(0x7f60e80668f0 msgr2=0x7f60e8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.316+0000 7f60ee1ce640 1 -- 172.21.15.38:0/2653523115 shutdown_connections 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.316+0000 7f60ee1ce640 1 -- 172.21.15.38:0/2653523115 wait complete. 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.317+0000 7f60ee1ce640 1 Processor -- start 2024-04-25T08:36:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.317+0000 7f60ee1ce640 1 -- start start 2024-04-25T08:36:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60ee1ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e80d0320 0x7f60e8158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60ee1ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e80d0700 0x7f60e80b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60ee1ce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e80d1e90 0x7f60e8155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60ee1ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e80d0b40 con 0x7f60e80d0320 2024-04-25T08:36:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60ee1ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e8161210 con 0x7f60e80d0700 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60ee1ce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e80b4360 con 0x7f60e80d1e90 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e80d0320 0x7f60e8158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60e77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e80d0320 0x7f60e8158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56182/0 (socket says 172.21.15.38:56182) 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60e77fe640 1 -- 172.21.15.38:0/4235494784 learned_addr learned my addr 172.21.15.38:0/4235494784 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60e77fe640 1 -- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e80d1e90 msgr2=0x7f60e8155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60e6ffd640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e80d0700 0x7f60e80b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60e7fff640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e80d1e90 0x7f60e8155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60e77fe640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e80d1e90 0x7f60e8155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.318+0000 7f60e77fe640 1 -- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e80d0700 msgr2=0x7f60e80b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.319+0000 7f60e77fe640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e80d0700 0x7f60e80b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.319+0000 7f60e77fe640 1 -- 172.21.15.38:0/4235494784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e8159130 con 0x7f60e80d0320 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.319+0000 7f60e77fe640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e80d0320 0x7f60e8158a80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f60d800e9e0 tx=0x7f60d800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.319+0000 7f60cffff640 1 -- 172.21.15.38:0/4235494784 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d8017070 con 0x7f60e80d0320 2024-04-25T08:36:28.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.319+0000 7f60cffff640 1 -- 172.21.15.38:0/4235494784 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d800c9d0 con 0x7f60e80d0320 2024-04-25T08:36:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.319+0000 7f60ee1ce640 1 -- 172.21.15.38:0/4235494784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e8159410 con 0x7f60e80d0320 2024-04-25T08:36:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.319+0000 7f60cffff640 1 -- 172.21.15.38:0/4235494784 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d800cdc0 con 0x7f60e80d0320 2024-04-25T08:36:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.319+0000 7f60ee1ce640 1 -- 172.21.15.38:0/4235494784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e81599d0 con 0x7f60e80d0320 2024-04-25T08:36:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.320+0000 7f60ee1ce640 1 -- 172.21.15.38:0/4235494784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e8068e80 con 0x7f60e80d0320 2024-04-25T08:36:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.320+0000 7f60cffff640 1 -- 172.21.15.38:0/4235494784 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60d801a070 con 0x7f60e80d0320 2024-04-25T08:36:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.321+0000 7f60cffff640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60d406b4d0 0x7f60d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.321+0000 7f60cffff640 1 -- 172.21.15.38:0/4235494784 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f60d808a6b0 con 0x7f60e80d0320 2024-04-25T08:36:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.321+0000 7f60e6ffd640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60d406b4d0 0x7f60d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:28.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.321+0000 7f60e6ffd640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60d406b4d0 0x7f60d406d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f60dc034000 tx=0x7f60dc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.325+0000 7f60cffff640 1 -- 172.21.15.38:0/4235494784 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60d8059430 con 0x7f60e80d0320 2024-04-25T08:36:28.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:28.546+0000 7f60ee1ce640 1 -- 172.21.15.38:0/4235494784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f60e80676e0 con 0x7f60e80d0320 2024-04-25T08:36:29.509 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-04-25T08:36:29.515+0000 7efdb7a0e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:36:29.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.560+0000 7f60cffff640 1 -- 172.21.15.38:0/4235494784 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v49) v1 ==== 118+0+0 (secure 0 0 0) 0x7f60d8058d30 con 0x7f60e80d0320 2024-04-25T08:36:29.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.563+0000 7f60ee1ce640 1 -- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60d406b4d0 msgr2=0x7f60d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.563+0000 7f60ee1ce640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60d406b4d0 0x7f60d406d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f60dc034000 tx=0x7f60dc02e000 comp rx=0 tx=0).stop 2024-04-25T08:36:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.563+0000 7f60ee1ce640 1 -- 172.21.15.38:0/4235494784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e80d0320 msgr2=0x7f60e8158a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.563+0000 7f60ee1ce640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e80d0320 0x7f60e8158a80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f60d800e9e0 tx=0x7f60d800eea0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.563+0000 7f60ee1ce640 1 -- 172.21.15.38:0/4235494784 shutdown_connections 2024-04-25T08:36:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.563+0000 7f60ee1ce640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60d406b4d0 0x7f60d406d990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.563+0000 7f60ee1ce640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e80d1e90 0x7f60e8155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.563+0000 7f60ee1ce640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e80d0700 0x7f60e80b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.563+0000 7f60ee1ce640 1 --2- 172.21.15.38:0/4235494784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e80d0320 0x7f60e8158a80 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.563+0000 7f60ee1ce640 1 -- 172.21.15.38:0/4235494784 >> 172.21.15.38:0/4235494784 conn(0x7f60e80668f0 msgr2=0x7f60e815e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.563+0000 7f60ee1ce640 1 -- 172.21.15.38:0/4235494784 shutdown_connections 2024-04-25T08:36:29.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.563+0000 7f60ee1ce640 1 -- 172.21.15.38:0/4235494784 wait complete. 2024-04-25T08:36:29.571 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:36:29.571 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:36:29.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.687+0000 7f94857c1640 1 Processor -- start 2024-04-25T08:36:29.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.687+0000 7f94857c1640 1 -- start start 2024-04-25T08:36:29.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.688+0000 7f94857c1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94800d33d0 0x7f94800d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:29.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.688+0000 7f94857c1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f94800d1e90 0x7f94800b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.688+0000 7f94857c1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94800d2330 con 0x7f94800d0320 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.688+0000 7f94857c1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94800d2610 con 0x7f94800d1e90 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.688+0000 7f94857c1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94800d28f0 con 0x7f94800d33d0 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.688+0000 7f947f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f94800d1e90 0x7f94800b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.688+0000 7f947f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f94800d1e90 0x7f94800b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47634/0 (socket says 172.21.15.38:47634) 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.688+0000 7f947effd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f94800d0320 0x7f94800d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49318/0 (socket says 172.21.15.38:49318) 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.688+0000 7f947f7fe640 1 -- 172.21.15.38:0/396861027 learned_addr learned my addr 172.21.15.38:0/396861027 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.688+0000 7f947e7fc640 1 --2- 172.21.15.38:0/396861027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94800d33d0 0x7f94800d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.689+0000 7f947f7fe640 1 -- 172.21.15.38:0/396861027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94800d33d0 msgr2=0x7f94800d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.689+0000 7f947f7fe640 1 --2- 172.21.15.38:0/396861027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94800d33d0 0x7f94800d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.689+0000 7f947f7fe640 1 -- 172.21.15.38:0/396861027 >> v1:172.21.15.38:6789/0 conn(0x7f94800d0320 legacy=0x7f94800d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.689+0000 7f947f7fe640 1 -- 172.21.15.38:0/396861027 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94800d2bd0 con 0x7f94800d1e90 2024-04-25T08:36:29.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.689+0000 7f947dffb640 1 -- 172.21.15.38:0/396861027 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3178048318 0 0) 0x7f94800d2330 con 0x7f94800d0320 2024-04-25T08:36:29.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.689+0000 7f947f7fe640 1 --2- 172.21.15.38:0/396861027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94800d1e90 0x7f94800b6980 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f9470009e50 tx=0x7f947002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5fb2daac3336d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:29.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.689+0000 7f947dffb640 1 -- 172.21.15.38:0/396861027 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470036070 con 0x7f94800d1e90 2024-04-25T08:36:29.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.689+0000 7f947dffb640 1 -- 172.21.15.38:0/396861027 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9470031600 con 0x7f94800d1e90 2024-04-25T08:36:29.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.689+0000 7f947dffb640 1 -- 172.21.15.38:0/396861027 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94700319d0 con 0x7f94800d1e90 2024-04-25T08:36:29.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.690+0000 7f94857c1640 1 -- 172.21.15.38:0/396861027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94800d1e90 msgr2=0x7f94800b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:29.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.690+0000 7f94857c1640 1 --2- 172.21.15.38:0/396861027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94800d1e90 0x7f94800b6980 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f9470009e50 tx=0x7f947002a7a0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.690+0000 7f94857c1640 1 -- 172.21.15.38:0/396861027 shutdown_connections 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.690+0000 7f94857c1640 1 --2- 172.21.15.38:0/396861027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94800d1e90 0x7f94800b6980 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.690+0000 7f94857c1640 1 --2- 172.21.15.38:0/396861027 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94800d33d0 0x7f94800d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.690+0000 7f94857c1640 1 -- 172.21.15.38:0/396861027 >> 172.21.15.38:0/396861027 conn(0x7f94800668f0 msgr2=0x7f9480066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.690+0000 7f94857c1640 1 -- 172.21.15.38:0/396861027 shutdown_connections 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.690+0000 7f94857c1640 1 -- 172.21.15.38:0/396861027 wait complete. 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.691+0000 7f94857c1640 1 Processor -- start 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.691+0000 7f94857c1640 1 -- start start 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.692+0000 7f94857c1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94800d0320 0x7f9480153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.692+0000 7f94857c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94800d1e90 0x7f94800aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.692+0000 7f94857c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94800d33d0 0x7f94800ab010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.692+0000 7f94857c1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9480161390 con 0x7f94800d1e90 2024-04-25T08:36:29.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.692+0000 7f94857c1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9480161210 con 0x7f94800d33d0 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.692+0000 7f94857c1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94800b9720 con 0x7f94800d0320 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.692+0000 7f947e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94800d1e90 0x7f94800aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.692+0000 7f947e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94800d1e90 0x7f94800aaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47646/0 (socket says 172.21.15.38:47646) 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.692+0000 7f947e7fc640 1 -- 172.21.15.38:0/3424189039 learned_addr learned my addr 172.21.15.38:0/3424189039 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.692+0000 7f947effd640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94800d0320 0x7f9480153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.692+0000 7f947f7fe640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94800d33d0 0x7f94800ab010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.693+0000 7f947e7fc640 1 -- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94800d0320 msgr2=0x7f9480153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.693+0000 7f947e7fc640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94800d0320 0x7f9480153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.693+0000 7f947e7fc640 1 -- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94800d33d0 msgr2=0x7f94800ab010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.693+0000 7f947e7fc640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94800d33d0 0x7f94800ab010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.693+0000 7f947e7fc640 1 -- 172.21.15.38:0/3424189039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94800ac6e0 con 0x7f94800d1e90 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.693+0000 7f947e7fc640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94800d1e90 0x7f94800aaad0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f946800d5f0 tx=0x7f946800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:29.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.693+0000 7f94777fe640 1 -- 172.21.15.38:0/3424189039 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680125f0 con 0x7f94800d1e90 2024-04-25T08:36:29.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.693+0000 7f94777fe640 1 -- 172.21.15.38:0/3424189039 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468019070 con 0x7f94800d1e90 2024-04-25T08:36:29.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.693+0000 7f94857c1640 1 -- 172.21.15.38:0/3424189039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94800ab640 con 0x7f94800d1e90 2024-04-25T08:36:29.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.693+0000 7f94777fe640 1 -- 172.21.15.38:0/3424189039 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680129d0 con 0x7f94800d1e90 2024-04-25T08:36:29.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.693+0000 7f94857c1640 1 -- 172.21.15.38:0/3424189039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94800abc00 con 0x7f94800d1e90 2024-04-25T08:36:29.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.694+0000 7f94857c1640 1 -- 172.21.15.38:0/3424189039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9480068e80 con 0x7f94800d1e90 2024-04-25T08:36:29.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.695+0000 7f94777fe640 1 -- 172.21.15.38:0/3424189039 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9468012b70 con 0x7f94800d1e90 2024-04-25T08:36:29.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.695+0000 7f94777fe640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f946006b4d0 0x7f946006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:29.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.695+0000 7f94777fe640 1 -- 172.21.15.38:0/3424189039 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f946808a610 con 0x7f94800d1e90 2024-04-25T08:36:29.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.695+0000 7f947effd640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f946006b4d0 0x7f946006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:29.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.696+0000 7f947effd640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f946006b4d0 0x7f946006d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9478000f60 tx=0x7f9478007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:29.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.699+0000 7f94777fe640 1 -- 172.21.15.38:0/3424189039 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9468058d40 con 0x7f94800d1e90 2024-04-25T08:36:29.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:29.919+0000 7f94857c1640 1 -- 172.21.15.38:0/3424189039 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f94800b3050 con 0x7f94800d1e90 2024-04-25T08:36:30.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.564+0000 7f94777fe640 1 -- 172.21.15.38:0/3424189039 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v50) v1 ==== 122+0+0 (secure 0 0 0) 0x7f946805cde0 con 0x7f94800d1e90 2024-04-25T08:36:30.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.568+0000 7f94857c1640 1 -- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f946006b4d0 msgr2=0x7f946006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.568+0000 7f94857c1640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f946006b4d0 0x7f946006d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9478000f60 tx=0x7f9478007d50 comp rx=0 tx=0).stop 2024-04-25T08:36:30.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.568+0000 7f94857c1640 1 -- 172.21.15.38:0/3424189039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94800d1e90 msgr2=0x7f94800aaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.568+0000 7f94857c1640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94800d1e90 0x7f94800aaad0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f946800d5f0 tx=0x7f946800dab0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.568+0000 7f94857c1640 1 -- 172.21.15.38:0/3424189039 shutdown_connections 2024-04-25T08:36:30.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.568+0000 7f94857c1640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94800d33d0 0x7f94800ab010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.568+0000 7f94857c1640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94800d1e90 0x7f94800aaad0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.568+0000 7f94857c1640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f946006b4d0 0x7f946006d990 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.568+0000 7f94857c1640 1 --2- 172.21.15.38:0/3424189039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94800d0320 0x7f9480153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.568+0000 7f94857c1640 1 -- 172.21.15.38:0/3424189039 >> 172.21.15.38:0/3424189039 conn(0x7f94800668f0 msgr2=0x7f94800d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:30.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.568+0000 7f94857c1640 1 -- 172.21.15.38:0/3424189039 shutdown_connections 2024-04-25T08:36:30.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.568+0000 7f94857c1640 1 -- 172.21.15.38:0/3424189039 wait complete. 2024-04-25T08:36:30.575 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:36:30.576 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:30.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.689+0000 7f7e47110640 1 Processor -- start 2024-04-25T08:36:30.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.689+0000 7f7e47110640 1 -- start start 2024-04-25T08:36:30.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.690+0000 7f7e47110640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7e400b5710 0x7f7e40161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.690+0000 7f7e47110640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e400b98a0 con 0x7f7e4015ddf0 2024-04-25T08:36:30.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.690+0000 7f7e47110640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e40161e00 con 0x7f7e400b93f0 2024-04-25T08:36:30.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.690+0000 7f7e47110640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e401620e0 con 0x7f7e400b5710 2024-04-25T08:36:30.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.690+0000 7f7e3ffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7e4015ddf0 0x7f7e400b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49332/0 (socket says 172.21.15.38:49332) 2024-04-25T08:36:30.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.690+0000 7f7e3ffff640 1 -- 172.21.15.38:0/1842378705 learned_addr learned my addr 172.21.15.38:0/1842378705 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:30.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.690+0000 7f7e44eac640 1 --2- 172.21.15.38:0/1842378705 >> v2:172.21.15.107:3300/0 conn(0x7f7e400b5710 0x7f7e40161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.690+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048895596 0 0) 0x7f7e400b98a0 con 0x7f7e4015ddf0 2024-04-25T08:36:30.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e30003520 con 0x7f7e4015ddf0 2024-04-25T08:36:30.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3640982183 0 0) 0x7f7e30003520 con 0x7f7e4015ddf0 2024-04-25T08:36:30.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 >> v2:172.21.15.107:3300/0 conn(0x7f7e400b5710 msgr2=0x7f7e40161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:30.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 --2- 172.21.15.38:0/1842378705 >> v2:172.21.15.107:3300/0 conn(0x7f7e400b5710 0x7f7e40161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 >> v1:172.21.15.107:6789/0 conn(0x7f7e400b93f0 legacy=0x7f7e400d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e401623c0 con 0x7f7e4015ddf0 2024-04-25T08:36:30.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f7e2c001670 con 0x7f7e4015ddf0 2024-04-25T08:36:30.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 >> v1:172.21.15.38:6789/0 conn(0x7f7e4015ddf0 legacy=0x7f7e400b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 --2- 172.21.15.38:0/1842378705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e300039b0 0x7f7e30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 --2- 172.21.15.38:0/1842378705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e300043b0 0x7f7e30004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 --2- 172.21.15.38:0/1842378705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e30005080 0x7f7e3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e401620e0 con 0x7f7e30005080 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3000bfe0 con 0x7f7e300043b0 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3000c1e0 con 0x7f7e300039b0 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e456ad640 1 --2- 172.21.15.38:0/1842378705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e30005080 0x7f7e3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e3ffff640 1 --2- 172.21.15.38:0/1842378705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e300043b0 0x7f7e30004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e44eac640 1 --2- 172.21.15.38:0/1842378705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e300039b0 0x7f7e30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e456ad640 1 -- 172.21.15.38:0/1842378705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e300039b0 msgr2=0x7f7e30003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e456ad640 1 --2- 172.21.15.38:0/1842378705 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e300039b0 0x7f7e30003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e456ad640 1 -- 172.21.15.38:0/1842378705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e300043b0 msgr2=0x7f7e30004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e456ad640 1 --2- 172.21.15.38:0/1842378705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e300043b0 0x7f7e30004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.691+0000 7f7e456ad640 1 -- 172.21.15.38:0/1842378705 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e30003520 con 0x7f7e30005080 2024-04-25T08:36:30.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.692+0000 7f7e456ad640 1 -- 172.21.15.38:0/1842378705 reap_dead start 2024-04-25T08:36:30.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.692+0000 7f7e456ad640 1 --2- 172.21.15.38:0/1842378705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e30005080 0x7f7e3000b9c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f7e38001f60 tx=0x7f7e30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619164551a602be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:30.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.692+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38001b30 con 0x7f7e30005080 2024-04-25T08:36:30.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.692+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e38003b90 con 0x7f7e30005080 2024-04-25T08:36:30.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.692+0000 7f7e3f7fe640 1 -- 172.21.15.38:0/1842378705 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38005110 con 0x7f7e30005080 2024-04-25T08:36:30.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.693+0000 7f7e47110640 1 -- 172.21.15.38:0/1842378705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e30005080 msgr2=0x7f7e3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.693+0000 7f7e47110640 1 --2- 172.21.15.38:0/1842378705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e30005080 0x7f7e3000b9c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f7e38001f60 tx=0x7f7e30005830 comp rx=0 tx=0).stop 2024-04-25T08:36:30.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.693+0000 7f7e47110640 1 -- 172.21.15.38:0/1842378705 shutdown_connections 2024-04-25T08:36:30.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.693+0000 7f7e47110640 1 --2- 172.21.15.38:0/1842378705 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e30005080 0x7f7e3000b9c0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.693+0000 7f7e47110640 1 -- 172.21.15.38:0/1842378705 >> 172.21.15.38:0/1842378705 conn(0x7f7e400d07c0 msgr2=0x7f7e400d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:30.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.693+0000 7f7e47110640 1 -- 172.21.15.38:0/1842378705 shutdown_connections 2024-04-25T08:36:30.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.693+0000 7f7e47110640 1 -- 172.21.15.38:0/1842378705 wait complete. 2024-04-25T08:36:30.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.694+0000 7f7e47110640 1 Processor -- start 2024-04-25T08:36:30.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.694+0000 7f7e47110640 1 -- start start 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e47110640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e30005080 0x7f7e40153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e47110640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e400c9e20 0x7f7e4015fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e47110640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e401602e0 0x7f7e400cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e47110640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e40161a90 con 0x7f7e401602e0 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e47110640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e40161910 con 0x7f7e30005080 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e47110640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400b5af0 con 0x7f7e400c9e20 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e456ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e401602e0 0x7f7e400cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e456ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e401602e0 0x7f7e400cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47660/0 (socket says 172.21.15.38:47660) 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e456ad640 1 -- 172.21.15.38:0/3799230664 learned_addr learned my addr 172.21.15.38:0/3799230664 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e44eac640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e30005080 0x7f7e40153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e3ffff640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e400c9e20 0x7f7e4015fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e456ad640 1 -- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e400c9e20 msgr2=0x7f7e4015fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e456ad640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e400c9e20 0x7f7e4015fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e456ad640 1 -- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e30005080 msgr2=0x7f7e40153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e456ad640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e30005080 0x7f7e40153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e456ad640 1 -- 172.21.15.38:0/3799230664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e400cadd0 con 0x7f7e401602e0 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e44eac640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e30005080 0x7f7e40153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.695+0000 7f7e3ffff640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e400c9e20 0x7f7e4015fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.696+0000 7f7e456ad640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e401602e0 0x7f7e400cc4e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f7e3802aa00 tx=0x7f7e380015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.696+0000 7f7e3d7fa640 1 -- 172.21.15.38:0/3799230664 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38006b50 con 0x7f7e401602e0 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.696+0000 7f7e3d7fa640 1 -- 172.21.15.38:0/3799230664 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e38006cf0 con 0x7f7e401602e0 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.696+0000 7f7e47110640 1 -- 172.21.15.38:0/3799230664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e40160a70 con 0x7f7e401602e0 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.696+0000 7f7e3d7fa640 1 -- 172.21.15.38:0/3799230664 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e38007250 con 0x7f7e401602e0 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.696+0000 7f7e47110640 1 -- 172.21.15.38:0/3799230664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e40160760 con 0x7f7e401602e0 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.697+0000 7f7e47110640 1 -- 172.21.15.38:0/3799230664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e40068e80 con 0x7f7e401602e0 2024-04-25T08:36:30.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.697+0000 7f7e3d7fa640 1 -- 172.21.15.38:0/3799230664 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e3803c070 con 0x7f7e401602e0 2024-04-25T08:36:30.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.698+0000 7f7e3d7fa640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e2406b500 0x7f7e2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:30.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.698+0000 7f7e3d7fa640 1 -- 172.21.15.38:0/3799230664 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7e380aa420 con 0x7f7e401602e0 2024-04-25T08:36:30.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.698+0000 7f7e44eac640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e2406b500 0x7f7e2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:30.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.698+0000 7f7e44eac640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e2406b500 0x7f7e2406d9c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7e340093d0 tx=0x7f7e34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:30.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.702+0000 7f7e3d7fa640 1 -- 172.21.15.38:0/3799230664 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e38044060 con 0x7f7e401602e0 2024-04-25T08:36:30.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.921+0000 7f7e47110640 1 -- 172.21.15.38:0/3799230664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e4015d3c0 con 0x7f7e401602e0 2024-04-25T08:36:30.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.921+0000 7f7e3d7fa640 1 -- 172.21.15.38:0/3799230664 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f7e38079180 con 0x7f7e401602e0 2024-04-25T08:36:30.916 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:30.916 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3781069672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3781069672},{"type":"v1","addr":"172.21.15.107:6837","nonce":3781069672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/624970524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":624970524},{"type":"v1","addr":"172.21.15.107:6836","nonce":624970524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7380,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1256978001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1256978001},{"type":"v1","addr":"172.21.15.38:6834","nonce":1256978001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:26.904065+0000","modified":"2024-04-25T08:36:29.920325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/380751658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":380751658},{"type":"v1","addr":"172.21.15.38:6835","nonce":380751658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:36:30.916 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 50 2024-04-25T08:36:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.923+0000 7f7e47110640 1 -- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e2406b500 msgr2=0x7f7e2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.923+0000 7f7e47110640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e2406b500 0x7f7e2406d9c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7e340093d0 tx=0x7f7e34008040 comp rx=0 tx=0).stop 2024-04-25T08:36:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.923+0000 7f7e47110640 1 -- 172.21.15.38:0/3799230664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e401602e0 msgr2=0x7f7e400cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.923+0000 7f7e47110640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e401602e0 0x7f7e400cc4e0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f7e3802aa00 tx=0x7f7e380015b0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.923+0000 7f7e47110640 1 -- 172.21.15.38:0/3799230664 shutdown_connections 2024-04-25T08:36:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.923+0000 7f7e47110640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e401602e0 0x7f7e400cc4e0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.923+0000 7f7e47110640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e2406b500 0x7f7e2406d9c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.923+0000 7f7e47110640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e400c9e20 0x7f7e4015fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.923+0000 7f7e47110640 1 --2- 172.21.15.38:0/3799230664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e30005080 0x7f7e40153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.923+0000 7f7e47110640 1 -- 172.21.15.38:0/3799230664 >> 172.21.15.38:0/3799230664 conn(0x7f7e400d07c0 msgr2=0x7f7e400b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.923+0000 7f7e47110640 1 -- 172.21.15.38:0/3799230664 shutdown_connections 2024-04-25T08:36:30.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:30.923+0000 7f7e47110640 1 -- 172.21.15.38:0/3799230664 wait complete. 2024-04-25T08:36:30.929 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:36:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.035+0000 7f03492fc640 1 Processor -- start 2024-04-25T08:36:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.035+0000 7f03492fc640 1 -- start start 2024-04-25T08:36:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03492fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f03440d3900 0x7f03440d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03492fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03440d0d90 0x7f03440b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03492fc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03440d3e70 con 0x7f034415dc10 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03492fc640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034415dff0 con 0x7f03440d3900 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03492fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03440b6990 con 0x7f03440d0d90 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03427fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f03440d3900 0x7f03440d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03427fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f03440d3900 0x7f03440d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47670/0 (socket says 172.21.15.38:47670) 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03427fc640 1 -- 172.21.15.38:0/2067205522 learned_addr learned my addr 172.21.15.38:0/2067205522 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03437fe640 1 --2- 172.21.15.38:0/2067205522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03440d0d90 0x7f03440b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03427fc640 1 -- 172.21.15.38:0/2067205522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03440d0d90 msgr2=0x7f03440b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03427fc640 1 --2- 172.21.15.38:0/2067205522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03440d0d90 0x7f03440b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03427fc640 1 -- 172.21.15.38:0/2067205522 >> v1:172.21.15.107:6789/0 conn(0x7f034415dc10 legacy=0x7f03440ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.036+0000 7f03427fc640 1 -- 172.21.15.38:0/2067205522 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03440d4150 con 0x7f03440d3900 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.037+0000 7f03427fc640 1 --2- 172.21.15.38:0/2067205522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03440d3900 0x7f03440d0850 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f032c009870 tx=0x7f032c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90b63d691650ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.037+0000 7f0341ffb640 1 -- 172.21.15.38:0/2067205522 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c037070 con 0x7f03440d3900 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.037+0000 7f0341ffb640 1 -- 172.21.15.38:0/2067205522 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c032440 con 0x7f03440d3900 2024-04-25T08:36:31.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.037+0000 7f0341ffb640 1 -- 172.21.15.38:0/2067205522 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c032810 con 0x7f03440d3900 2024-04-25T08:36:31.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.038+0000 7f03492fc640 1 -- 172.21.15.38:0/2067205522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03440d3900 msgr2=0x7f03440d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.038+0000 7f03492fc640 1 --2- 172.21.15.38:0/2067205522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03440d3900 0x7f03440d0850 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f032c009870 tx=0x7f032c02a1c0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.038+0000 7f03492fc640 1 -- 172.21.15.38:0/2067205522 shutdown_connections 2024-04-25T08:36:31.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.038+0000 7f03492fc640 1 --2- 172.21.15.38:0/2067205522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03440d0d90 0x7f03440b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.038+0000 7f03492fc640 1 --2- 172.21.15.38:0/2067205522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03440d3900 0x7f03440d0850 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.038+0000 7f03492fc640 1 -- 172.21.15.38:0/2067205522 >> 172.21.15.38:0/2067205522 conn(0x7f0344066d10 msgr2=0x7f0344067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:31.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.038+0000 7f03492fc640 1 -- 172.21.15.38:0/2067205522 shutdown_connections 2024-04-25T08:36:31.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.038+0000 7f03492fc640 1 -- 172.21.15.38:0/2067205522 wait complete. 2024-04-25T08:36:31.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.039+0000 7f03492fc640 1 Processor -- start 2024-04-25T08:36:31.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.039+0000 7f03492fc640 1 -- start start 2024-04-25T08:36:31.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.040+0000 7f03492fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03440d0d90 0x7f03440cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.040+0000 7f03492fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03440d3900 0x7f03440aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.040+0000 7f03492fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f034415dc10 0x7f034415ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.040+0000 7f03492fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03440b6bc0 con 0x7f03440d3900 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.040+0000 7f03492fc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03440d4710 con 0x7f034415dc10 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.040+0000 7f03492fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03440cef30 con 0x7f03440d0d90 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.040+0000 7f03427fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03440d3900 0x7f03440aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.040+0000 7f03427fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03440d3900 0x7f03440aacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47686/0 (socket says 172.21.15.38:47686) 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.040+0000 7f03427fc640 1 -- 172.21.15.38:0/4083366455 learned_addr learned my addr 172.21.15.38:0/4083366455 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.040+0000 7f03437fe640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f034415dc10 0x7f034415ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.040+0000 7f0342ffd640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03440d0d90 0x7f03440cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.041+0000 7f03427fc640 1 -- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03440d0d90 msgr2=0x7f03440cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.041+0000 7f03427fc640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03440d0d90 0x7f03440cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.041+0000 7f03427fc640 1 -- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f034415dc10 msgr2=0x7f034415ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.041+0000 7f03427fc640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f034415dc10 0x7f034415ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.041+0000 7f03427fc640 1 -- 172.21.15.38:0/4083366455 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03440ab7a0 con 0x7f03440d3900 2024-04-25T08:36:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.041+0000 7f03427fc640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03440d3900 0x7f03440aacd0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f032c000c00 tx=0x7f032c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.041+0000 7f033b7fe640 1 -- 172.21.15.38:0/4083366455 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c037040 con 0x7f03440d3900 2024-04-25T08:36:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.041+0000 7f033b7fe640 1 -- 172.21.15.38:0/4083366455 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c042070 con 0x7f03440d3900 2024-04-25T08:36:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.041+0000 7f03492fc640 1 -- 172.21.15.38:0/4083366455 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03440aba80 con 0x7f03440d3900 2024-04-25T08:36:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.041+0000 7f033b7fe640 1 -- 172.21.15.38:0/4083366455 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c030880 con 0x7f03440d3900 2024-04-25T08:36:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.041+0000 7f03492fc640 1 -- 172.21.15.38:0/4083366455 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03440ac040 con 0x7f03440d3900 2024-04-25T08:36:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.042+0000 7f03492fc640 1 -- 172.21.15.38:0/4083366455 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03440ac380 con 0x7f03440d3900 2024-04-25T08:36:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.042+0000 7f033b7fe640 1 -- 172.21.15.38:0/4083366455 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f032c02ad10 con 0x7f03440d3900 2024-04-25T08:36:31.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.043+0000 7f033b7fe640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f032406b4d0 0x7f032406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.043+0000 7f033b7fe640 1 -- 172.21.15.38:0/4083366455 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f032c0aa9d0 con 0x7f03440d3900 2024-04-25T08:36:31.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.043+0000 7f0342ffd640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f032406b4d0 0x7f032406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.044+0000 7f0342ffd640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f032406b4d0 0x7f032406d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f033c001640 tx=0x7f033c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:31.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.047+0000 7f033b7fe640 1 -- 172.21.15.38:0/4083366455 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03440ac380 con 0x7f03440d3900 2024-04-25T08:36:31.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.227+0000 7f03492fc640 1 -- 172.21.15.38:0/4083366455 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0344068e80 con 0x7f03440d3900 2024-04-25T08:36:31.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.228+0000 7f033b7fe640 1 -- 172.21.15.38:0/4083366455 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f032c079750 con 0x7f03440d3900 2024-04-25T08:36:31.223 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:31.223 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":82,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:36:28.545137+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:25.176504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:25.877954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:36:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.230+0000 7f03492fc640 1 -- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f032406b4d0 msgr2=0x7f032406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.230+0000 7f03492fc640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f032406b4d0 0x7f032406d990 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f033c001640 tx=0x7f033c009040 comp rx=0 tx=0).stop 2024-04-25T08:36:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.230+0000 7f03492fc640 1 -- 172.21.15.38:0/4083366455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03440d3900 msgr2=0x7f03440aacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.230+0000 7f03492fc640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03440d3900 0x7f03440aacd0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f032c000c00 tx=0x7f032c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.230+0000 7f03492fc640 1 -- 172.21.15.38:0/4083366455 shutdown_connections 2024-04-25T08:36:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.230+0000 7f03492fc640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f034415dc10 0x7f034415ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.230+0000 7f03492fc640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f03440d3900 0x7f03440aacd0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.230+0000 7f03492fc640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f032406b4d0 0x7f032406d990 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.230+0000 7f03492fc640 1 --2- 172.21.15.38:0/4083366455 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03440d0d90 0x7f03440cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.230+0000 7f03492fc640 1 -- 172.21.15.38:0/4083366455 >> 172.21.15.38:0/4083366455 conn(0x7f0344066d10 msgr2=0x7f034415e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:31.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.230+0000 7f03492fc640 1 -- 172.21.15.38:0/4083366455 shutdown_connections 2024-04-25T08:36:31.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.230+0000 7f03492fc640 1 -- 172.21.15.38:0/4083366455 wait complete. 2024-04-25T08:36:31.236 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:36:31.236 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:36:31.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.343+0000 7ff530e5d640 1 Processor -- start 2024-04-25T08:36:31.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.343+0000 7ff530e5d640 1 -- start start 2024-04-25T08:36:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff530e5d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff52c0cd8d0 0x7ff52c06c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff530e5d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff52c09acb0 0x7ff52c06cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff530e5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c0b2170 0x7ff52c0cf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff530e5d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c09a820 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff530e5d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c161540 con 0x7ff52c09acb0 2024-04-25T08:36:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff530e5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c09a520 con 0x7ff52c0b2170 2024-04-25T08:36:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff52a59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff52c0cd8d0 0x7ff52c06c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff52a59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff52c0cd8d0 0x7ff52c06c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47688/0 (socket says 172.21.15.38:47688) 2024-04-25T08:36:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff52a59c640 1 -- 172.21.15.38:0/1202106815 learned_addr learned my addr 172.21.15.38:0/1202106815 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff529d9b640 1 --2- 172.21.15.38:0/1202106815 >> v2:172.21.15.107:3300/0 conn(0x7ff52c09acb0 0x7ff52c06cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff52a59c640 1 -- 172.21.15.38:0/1202106815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c0b2170 msgr2=0x7ff52c0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff52ad9d640 1 --2- 172.21.15.38:0/1202106815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c0b2170 0x7ff52c0cf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff52a59c640 1 --2- 172.21.15.38:0/1202106815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c0b2170 0x7ff52c0cf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff52a59c640 1 -- 172.21.15.38:0/1202106815 >> v2:172.21.15.107:3300/0 conn(0x7ff52c09acb0 msgr2=0x7ff52c06cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff52a59c640 1 --2- 172.21.15.38:0/1202106815 >> v2:172.21.15.107:3300/0 conn(0x7ff52c09acb0 0x7ff52c06cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff52a59c640 1 -- 172.21.15.38:0/1202106815 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c06be90 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff529d9b640 1 --2- 172.21.15.38:0/1202106815 >> v2:172.21.15.107:3300/0 conn(0x7ff52c09acb0 0x7ff52c06cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.344+0000 7ff52ad9d640 1 --2- 172.21.15.38:0/1202106815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c0b2170 0x7ff52c0cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:31.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.345+0000 7ff52a59c640 1 --2- 172.21.15.38:0/1202106815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0cd8d0 0x7ff52c06c6e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0cdcb0 tx=0x7ff51c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804ed5156dd564a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:31.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.345+0000 7ff52959a640 1 -- 172.21.15.38:0/1202106815 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c037070 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.345+0000 7ff52959a640 1 -- 172.21.15.38:0/1202106815 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c032440 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.345+0000 7ff52959a640 1 -- 172.21.15.38:0/1202106815 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c032810 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.346+0000 7ff530e5d640 1 -- 172.21.15.38:0/1202106815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0cd8d0 msgr2=0x7ff52c06c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.346+0000 7ff530e5d640 1 --2- 172.21.15.38:0/1202106815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0cd8d0 0x7ff52c06c6e0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0cdcb0 tx=0x7ff51c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.346+0000 7ff530e5d640 1 -- 172.21.15.38:0/1202106815 shutdown_connections 2024-04-25T08:36:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.346+0000 7ff530e5d640 1 --2- 172.21.15.38:0/1202106815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c0b2170 0x7ff52c0cf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.346+0000 7ff530e5d640 1 --2- 172.21.15.38:0/1202106815 >> v2:172.21.15.107:3300/0 conn(0x7ff52c09acb0 0x7ff52c06cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.346+0000 7ff530e5d640 1 --2- 172.21.15.38:0/1202106815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0cd8d0 0x7ff52c06c6e0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.346+0000 7ff530e5d640 1 -- 172.21.15.38:0/1202106815 >> 172.21.15.38:0/1202106815 conn(0x7ff52c0b3160 msgr2=0x7ff52c0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.346+0000 7ff530e5d640 1 -- 172.21.15.38:0/1202106815 shutdown_connections 2024-04-25T08:36:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.346+0000 7ff530e5d640 1 -- 172.21.15.38:0/1202106815 wait complete. 2024-04-25T08:36:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.347+0000 7ff530e5d640 1 Processor -- start 2024-04-25T08:36:31.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.347+0000 7ff530e5d640 1 -- start start 2024-04-25T08:36:31.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.348+0000 7ff530e5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c09acb0 0x7ff52c0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.348+0000 7ff530e5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c0b2170 0x7ff52c0b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.348+0000 7ff530e5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0cd8d0 0x7ff52c1f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.348+0000 7ff530e5d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c06c410 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.348+0000 7ff530e5d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0d0ad0 con 0x7ff52c0b2170 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.348+0000 7ff530e5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0b26c0 con 0x7ff52c09acb0 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.348+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0cd8d0 0x7ff52c1f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.348+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0cd8d0 0x7ff52c1f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47696/0 (socket says 172.21.15.38:47696) 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.348+0000 7ff52ad9d640 1 -- 172.21.15.38:0/1513690322 learned_addr learned my addr 172.21.15.38:0/1513690322 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.348+0000 7ff529d9b640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c0b2170 0x7ff52c0b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.348+0000 7ff52a59c640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c09acb0 0x7ff52c0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.349+0000 7ff52ad9d640 1 -- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c09acb0 msgr2=0x7ff52c0c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.349+0000 7ff52ad9d640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c09acb0 0x7ff52c0c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.349+0000 7ff52ad9d640 1 -- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c0b2170 msgr2=0x7ff52c0b70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.349+0000 7ff52ad9d640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c0b2170 0x7ff52c0b70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.349+0000 7ff52ad9d640 1 -- 172.21.15.38:0/1513690322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c0c5830 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.349+0000 7ff52ad9d640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0cd8d0 0x7ff52c1f3f00 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff52400cef0 tx=0x7ff52400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.349+0000 7ff51affd640 1 -- 172.21.15.38:0/1513690322 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52401a070 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.349+0000 7ff51affd640 1 -- 172.21.15.38:0/1513690322 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52400d860 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.349+0000 7ff530e5d640 1 -- 172.21.15.38:0/1513690322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c0c5b10 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.349+0000 7ff51affd640 1 -- 172.21.15.38:0/1513690322 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52400dc50 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.349+0000 7ff530e5d640 1 -- 172.21.15.38:0/1513690322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c0c60d0 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.350+0000 7ff530e5d640 1 -- 172.21.15.38:0/1513690322 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c068e80 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.350+0000 7ff51affd640 1 -- 172.21.15.38:0/1513690322 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff52401d070 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.351+0000 7ff51affd640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50c06b4d0 0x7ff50c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.351+0000 7ff51affd640 1 -- 172.21.15.38:0/1513690322 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff52408ab10 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.351+0000 7ff52a59c640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50c06b4d0 0x7ff50c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.352+0000 7ff52a59c640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50c06b4d0 0x7ff50c06d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff51c034000 tx=0x7ff51c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:31.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.355+0000 7ff51affd640 1 -- 172.21.15.38:0/1513690322 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff524059270 con 0x7ff52c0cd8d0 2024-04-25T08:36:31.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.530+0000 7ff530e5d640 1 -- 172.21.15.38:0/1513690322 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff52c0cb770 con 0x7ff50c06b4d0 2024-04-25T08:36:31.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.537+0000 7ff51affd640 1 -- 172.21.15.38:0/1513690322 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246753 (secure 0 0 0) 0x7ff52c0cb770 con 0x7ff50c06b4d0 2024-04-25T08:36:31.534 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:31.536 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:36:31.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.539+0000 7ff530e5d640 1 -- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50c06b4d0 msgr2=0x7ff50c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.539+0000 7ff530e5d640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50c06b4d0 0x7ff50c06d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff51c034000 tx=0x7ff51c02e000 comp rx=0 tx=0).stop 2024-04-25T08:36:31.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.539+0000 7ff530e5d640 1 -- 172.21.15.38:0/1513690322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0cd8d0 msgr2=0x7ff52c1f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.539+0000 7ff530e5d640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0cd8d0 0x7ff52c1f3f00 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff52400cef0 tx=0x7ff52400a710 comp rx=0 tx=0).stop 2024-04-25T08:36:31.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.539+0000 7ff530e5d640 1 -- 172.21.15.38:0/1513690322 shutdown_connections 2024-04-25T08:36:31.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.539+0000 7ff530e5d640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0cd8d0 0x7ff52c1f3f00 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.539+0000 7ff530e5d640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c0b2170 0x7ff52c0b70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.539+0000 7ff530e5d640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50c06b4d0 0x7ff50c06d990 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.539+0000 7ff530e5d640 1 --2- 172.21.15.38:0/1513690322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c09acb0 0x7ff52c0c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.539+0000 7ff530e5d640 1 -- 172.21.15.38:0/1513690322 >> 172.21.15.38:0/1513690322 conn(0x7ff52c0b3160 msgr2=0x7ff52c099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:31.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.539+0000 7ff530e5d640 1 -- 172.21.15.38:0/1513690322 shutdown_connections 2024-04-25T08:36:31.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.539+0000 7ff530e5d640 1 -- 172.21.15.38:0/1513690322 wait complete. 2024-04-25T08:36:31.545 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-04-25T08:36:31.273377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":337352,"kb_used_data":8796,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754637368,"statfs":{"total":773094113280,"available":772748664832,"internally_reserved":0,"allocated":9007104,"data_stored":4773429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":336318000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.742948"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537773+0000","last_change":"2024-04-25T08:36:26.537773+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537773+0000","last_undegraded":"2024-04-25T08:36:26.537773+0000","last_fullsized":"2024-04-25T08:36:26.537773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.544249+0000","last_change":"2024-04-25T08:36:26.544249+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.544249+0000","last_undegraded":"2024-04-25T08:36:26.544249+0000","last_fullsized":"2024-04-25T08:36:26.544249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.545202+0000","last_change":"2024-04-25T08:36:26.545202+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.545202+0000","last_undegraded":"2024-04-25T08:36:26.545202+0000","last_fullsized":"2024-04-25T08:36:26.545202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538453+0000","last_change":"2024-04-25T08:36:26.538453+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538453+0000","last_undegraded":"2024-04-25T08:36:26.538453+0000","last_fullsized":"2024-04-25T08:36:26.538453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538575+0000","last_change":"2024-04-25T08:36:26.538575+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538575+0000","last_undegraded":"2024-04-25T08:36:26.538575+0000","last_fullsized":"2024-04-25T08:36:26.538575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538753+0000","last_change":"2024-04-25T08:36:26.538753+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538753+0000","last_undegraded":"2024-04-25T08:36:26.538753+0000","last_fullsized":"2024-04-25T08:36:26.538753+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534187+0000","last_change":"2024-04-25T08:36:26.534187+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534187+0000","last_undegraded":"2024-04-25T08:36:26.534187+0000","last_fullsized":"2024-04-25T08:36:26.534187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537353+0000","last_change":"2024-04-25T08:36:26.537353+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537353+0000","last_undegraded":"2024-04-25T08:36:26.537353+0000","last_fullsized":"2024-04-25T08:36:26.537353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534390+0000","last_change":"2024-04-25T08:36:26.534390+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534390+0000","last_undegraded":"2024-04-25T08:36:26.534390+0000","last_fullsized":"2024-04-25T08:36:26.534390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538815+0000","last_change":"2024-04-25T08:36:26.538815+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538815+0000","last_undegraded":"2024-04-25T08:36:26.538815+0000","last_fullsized":"2024-04-25T08:36:26.538815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.543800+0000","last_change":"2024-04-25T08:36:26.543800+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.543800+0000","last_undegraded":"2024-04-25T08:36:26.543800+0000","last_fullsized":"2024-04-25T08:36:26.543800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537131+0000","last_change":"2024-04-25T08:36:26.537131+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537131+0000","last_undegraded":"2024-04-25T08:36:26.537131+0000","last_fullsized":"2024-04-25T08:36:26.537131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536911+0000","last_change":"2024-04-25T08:36:26.536911+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536911+0000","last_undegraded":"2024-04-25T08:36:26.536911+0000","last_fullsized":"2024-04-25T08:36:26.536911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538336+0000","last_change":"2024-04-25T08:36:26.538336+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538336+0000","last_undegraded":"2024-04-25T08:36:26.538336+0000","last_fullsized":"2024-04-25T08:36:26.538336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.544447+0000","last_change":"2024-04-25T08:36:26.544447+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.544447+0000","last_undegraded":"2024-04-25T08:36:26.544447+0000","last_fullsized":"2024-04-25T08:36:26.544447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537142+0000","last_change":"2024-04-25T08:36:26.537142+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537142+0000","last_undegraded":"2024-04-25T08:36:26.537142+0000","last_fullsized":"2024-04-25T08:36:26.537142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537724+0000","last_change":"2024-04-25T08:36:26.537724+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537724+0000","last_undegraded":"2024-04-25T08:36:26.537724+0000","last_fullsized":"2024-04-25T08:36:26.537724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.543802+0000","last_change":"2024-04-25T08:36:26.543802+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.543802+0000","last_undegraded":"2024-04-25T08:36:26.543802+0000","last_fullsized":"2024-04-25T08:36:26.543802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537441+0000","last_change":"2024-04-25T08:36:26.537441+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537441+0000","last_undegraded":"2024-04-25T08:36:26.537441+0000","last_fullsized":"2024-04-25T08:36:26.537441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.539548+0000","last_change":"2024-04-25T08:36:26.539548+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.539548+0000","last_undegraded":"2024-04-25T08:36:26.539548+0000","last_fullsized":"2024-04-25T08:36:26.539548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536226+0000","last_change":"2024-04-25T08:36:26.536226+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536226+0000","last_undegraded":"2024-04-25T08:36:26.536226+0000","last_fullsized":"2024-04-25T08:36:26.536226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538328+0000","last_change":"2024-04-25T08:36:26.538328+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538328+0000","last_undegraded":"2024-04-25T08:36:26.538328+0000","last_fullsized":"2024-04-25T08:36:26.538328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534103+0000","last_change":"2024-04-25T08:36:26.534103+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534103+0000","last_undegraded":"2024-04-25T08:36:26.534103+0000","last_fullsized":"2024-04-25T08:36:26.534103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.541048+0000","last_change":"2024-04-25T08:36:26.541048+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.541048+0000","last_undegraded":"2024-04-25T08:36:26.541048+0000","last_fullsized":"2024-04-25T08:36:26.541048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540285+0000","last_change":"2024-04-25T08:36:26.540285+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540285+0000","last_undegraded":"2024-04-25T08:36:26.540285+0000","last_fullsized":"2024-04-25T08:36:26.540285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537767+0000","last_change":"2024-04-25T08:36:26.537767+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537767+0000","last_undegraded":"2024-04-25T08:36:26.537767+0000","last_fullsized":"2024-04-25T08:36:26.537767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537155+0000","last_change":"2024-04-25T08:36:26.537155+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537155+0000","last_undegraded":"2024-04-25T08:36:26.537155+0000","last_fullsized":"2024-04-25T08:36:26.537155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538315+0000","last_change":"2024-04-25T08:36:26.538315+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538315+0000","last_undegraded":"2024-04-25T08:36:26.538315+0000","last_fullsized":"2024-04-25T08:36:26.538315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538511+0000","last_change":"2024-04-25T08:36:26.538511+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538511+0000","last_undegraded":"2024-04-25T08:36:26.538511+0000","last_fullsized":"2024-04-25T08:36:26.538511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536776+0000","last_change":"2024-04-25T08:36:26.536776+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536776+0000","last_undegraded":"2024-04-25T08:36:26.536776+0000","last_fullsized":"2024-04-25T08:36:26.536776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540684+0000","last_change":"2024-04-25T08:36:26.540684+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540684+0000","last_undegraded":"2024-04-25T08:36:26.540684+0000","last_fullsized":"2024-04-25T08:36:26.540684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536655+0000","last_change":"2024-04-25T08:36:26.536655+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536655+0000","last_undegraded":"2024-04-25T08:36:26.536655+0000","last_fullsized":"2024-04-25T08:36:26.536655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.539609+0000","last_change":"2024-04-25T08:36:26.539609+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.539609+0000","last_undegraded":"2024-04-25T08:36:26.539609+0000","last_fullsized":"2024-04-25T08:36:26.539609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.553534+0000","last_change":"2024-04-25T08:36:26.552070+0000","last_active":"2024-04-25T08:36:26.553534+0000","last_peered":"2024-04-25T08:36:26.553534+0000","last_clean":"2024-04-25T08:36:26.553534+0000","last_became_active":"2024-04-25T08:36:26.550647+0000","last_became_peered":"2024-04-25T08:36:26.550647+0000","last_unstale":"2024-04-25T08:36:26.553534+0000","last_undegraded":"2024-04-25T08:36:26.553534+0000","last_fullsized":"2024-04-25T08:36:26.553534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:18.633914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537186+0000","last_change":"2024-04-25T08:36:26.537186+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537186+0000","last_undegraded":"2024-04-25T08:36:26.537186+0000","last_fullsized":"2024-04-25T08:36:26.537186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551229+0000","last_change":"2024-04-25T08:36:26.549707+0000","last_active":"2024-04-25T08:36:26.551229+0000","last_peered":"2024-04-25T08:36:26.551229+0000","last_clean":"2024-04-25T08:36:26.551229+0000","last_became_active":"2024-04-25T08:36:26.549227+0000","last_became_peered":"2024-04-25T08:36:26.549227+0000","last_unstale":"2024-04-25T08:36:26.551229+0000","last_undegraded":"2024-04-25T08:36:26.551229+0000","last_fullsized":"2024-04-25T08:36:26.551229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:30.421525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535476+0000","last_change":"2024-04-25T08:36:26.535476+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535476+0000","last_undegraded":"2024-04-25T08:36:26.535476+0000","last_fullsized":"2024-04-25T08:36:26.535476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.552110+0000","last_change":"2024-04-25T08:36:26.551664+0000","last_active":"2024-04-25T08:36:26.552110+0000","last_peered":"2024-04-25T08:36:26.552110+0000","last_clean":"2024-04-25T08:36:26.552110+0000","last_became_active":"2024-04-25T08:36:26.551388+0000","last_became_peered":"2024-04-25T08:36:26.551388+0000","last_unstale":"2024-04-25T08:36:26.552110+0000","last_undegraded":"2024-04-25T08:36:26.552110+0000","last_fullsized":"2024-04-25T08:36:26.552110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:58.975343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535807+0000","last_change":"2024-04-25T08:36:26.535807+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535807+0000","last_undegraded":"2024-04-25T08:36:26.535807+0000","last_fullsized":"2024-04-25T08:36:26.535807+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547750+0000","last_change":"2024-04-25T08:36:26.545919+0000","last_active":"2024-04-25T08:36:26.547750+0000","last_peered":"2024-04-25T08:36:26.547750+0000","last_clean":"2024-04-25T08:36:26.547750+0000","last_became_active":"2024-04-25T08:36:26.545633+0000","last_became_peered":"2024-04-25T08:36:26.545633+0000","last_unstale":"2024-04-25T08:36:26.547750+0000","last_undegraded":"2024-04-25T08:36:26.547750+0000","last_fullsized":"2024-04-25T08:36:26.547750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:27.158628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538946+0000","last_change":"2024-04-25T08:36:26.538946+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538946+0000","last_undegraded":"2024-04-25T08:36:26.538946+0000","last_fullsized":"2024-04-25T08:36:26.538946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550474+0000","last_change":"2024-04-25T08:36:26.549171+0000","last_active":"2024-04-25T08:36:26.550474+0000","last_peered":"2024-04-25T08:36:26.550474+0000","last_clean":"2024-04-25T08:36:26.550474+0000","last_became_active":"2024-04-25T08:36:26.548875+0000","last_became_peered":"2024-04-25T08:36:26.548875+0000","last_unstale":"2024-04-25T08:36:26.550474+0000","last_undegraded":"2024-04-25T08:36:26.550474+0000","last_fullsized":"2024-04-25T08:36:26.550474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:04.789878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536149+0000","last_change":"2024-04-25T08:36:26.536149+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536149+0000","last_undegraded":"2024-04-25T08:36:26.536149+0000","last_fullsized":"2024-04-25T08:36:26.536149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550104+0000","last_change":"2024-04-25T08:36:26.544834+0000","last_active":"2024-04-25T08:36:26.550104+0000","last_peered":"2024-04-25T08:36:26.550104+0000","last_clean":"2024-04-25T08:36:26.550104+0000","last_became_active":"2024-04-25T08:36:26.544549+0000","last_became_peered":"2024-04-25T08:36:26.544549+0000","last_unstale":"2024-04-25T08:36:26.550104+0000","last_undegraded":"2024-04-25T08:36:26.550104+0000","last_fullsized":"2024-04-25T08:36:26.550104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:59.366292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533938+0000","last_change":"2024-04-25T08:36:26.533938+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533938+0000","last_undegraded":"2024-04-25T08:36:26.533938+0000","last_fullsized":"2024-04-25T08:36:26.533938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550148+0000","last_change":"2024-04-25T08:36:26.546765+0000","last_active":"2024-04-25T08:36:26.550148+0000","last_peered":"2024-04-25T08:36:26.550148+0000","last_clean":"2024-04-25T08:36:26.550148+0000","last_became_active":"2024-04-25T08:36:26.546467+0000","last_became_peered":"2024-04-25T08:36:26.546467+0000","last_unstale":"2024-04-25T08:36:26.550148+0000","last_undegraded":"2024-04-25T08:36:26.550148+0000","last_fullsized":"2024-04-25T08:36:26.550148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:44.885589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535281+0000","last_change":"2024-04-25T08:36:26.535281+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535281+0000","last_undegraded":"2024-04-25T08:36:26.535281+0000","last_fullsized":"2024-04-25T08:36:26.535281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.548729+0000","last_change":"2024-04-25T08:36:26.544689+0000","last_active":"2024-04-25T08:36:26.548729+0000","last_peered":"2024-04-25T08:36:26.548729+0000","last_clean":"2024-04-25T08:36:26.548729+0000","last_became_active":"2024-04-25T08:36:26.544353+0000","last_became_peered":"2024-04-25T08:36:26.544353+0000","last_unstale":"2024-04-25T08:36:26.548729+0000","last_undegraded":"2024-04-25T08:36:26.548729+0000","last_fullsized":"2024-04-25T08:36:26.548729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:19.783413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536997+0000","last_change":"2024-04-25T08:36:26.536997+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536997+0000","last_undegraded":"2024-04-25T08:36:26.536997+0000","last_fullsized":"2024-04-25T08:36:26.536997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551564+0000","last_change":"2024-04-25T08:36:26.549714+0000","last_active":"2024-04-25T08:36:26.551564+0000","last_peered":"2024-04-25T08:36:26.551564+0000","last_clean":"2024-04-25T08:36:26.551564+0000","last_became_active":"2024-04-25T08:36:26.548992+0000","last_became_peered":"2024-04-25T08:36:26.548992+0000","last_unstale":"2024-04-25T08:36:26.551564+0000","last_undegraded":"2024-04-25T08:36:26.551564+0000","last_fullsized":"2024-04-25T08:36:26.551564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:57.849522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536670+0000","last_change":"2024-04-25T08:36:26.536670+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536670+0000","last_undegraded":"2024-04-25T08:36:26.536670+0000","last_fullsized":"2024-04-25T08:36:26.536670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550140+0000","last_change":"2024-04-25T08:36:26.546855+0000","last_active":"2024-04-25T08:36:26.550140+0000","last_peered":"2024-04-25T08:36:26.550140+0000","last_clean":"2024-04-25T08:36:26.550140+0000","last_became_active":"2024-04-25T08:36:26.546607+0000","last_became_peered":"2024-04-25T08:36:26.546607+0000","last_unstale":"2024-04-25T08:36:26.550140+0000","last_undegraded":"2024-04-25T08:36:26.550140+0000","last_fullsized":"2024-04-25T08:36:26.550140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:26.454697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537812+0000","last_change":"2024-04-25T08:36:26.537812+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537812+0000","last_undegraded":"2024-04-25T08:36:26.537812+0000","last_fullsized":"2024-04-25T08:36:26.537812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549618+0000","last_change":"2024-04-25T08:36:26.548301+0000","last_active":"2024-04-25T08:36:26.549618+0000","last_peered":"2024-04-25T08:36:26.549618+0000","last_clean":"2024-04-25T08:36:26.549618+0000","last_became_active":"2024-04-25T08:36:26.547849+0000","last_became_peered":"2024-04-25T08:36:26.547849+0000","last_unstale":"2024-04-25T08:36:26.549618+0000","last_undegraded":"2024-04-25T08:36:26.549618+0000","last_fullsized":"2024-04-25T08:36:26.549618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.444339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540620+0000","last_change":"2024-04-25T08:36:26.540620+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540620+0000","last_undegraded":"2024-04-25T08:36:26.540620+0000","last_fullsized":"2024-04-25T08:36:26.540620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.546872+0000","last_change":"2024-04-25T08:36:26.545189+0000","last_active":"2024-04-25T08:36:26.546872+0000","last_peered":"2024-04-25T08:36:26.546872+0000","last_clean":"2024-04-25T08:36:26.546872+0000","last_became_active":"2024-04-25T08:36:26.544888+0000","last_became_peered":"2024-04-25T08:36:26.544888+0000","last_unstale":"2024-04-25T08:36:26.546872+0000","last_undegraded":"2024-04-25T08:36:26.546872+0000","last_fullsized":"2024-04-25T08:36:26.546872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:33.972987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536242+0000","last_change":"2024-04-25T08:36:26.536242+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536242+0000","last_undegraded":"2024-04-25T08:36:26.536242+0000","last_fullsized":"2024-04-25T08:36:26.536242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.548075+0000","last_change":"2024-04-25T08:36:26.544584+0000","last_active":"2024-04-25T08:36:26.548075+0000","last_peered":"2024-04-25T08:36:26.548075+0000","last_clean":"2024-04-25T08:36:26.548075+0000","last_became_active":"2024-04-25T08:36:26.544307+0000","last_became_peered":"2024-04-25T08:36:26.544307+0000","last_unstale":"2024-04-25T08:36:26.548075+0000","last_undegraded":"2024-04-25T08:36:26.548075+0000","last_fullsized":"2024-04-25T08:36:26.548075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:04.226062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535774+0000","last_change":"2024-04-25T08:36:26.535774+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535774+0000","last_undegraded":"2024-04-25T08:36:26.535774+0000","last_fullsized":"2024-04-25T08:36:26.535774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551293+0000","last_change":"2024-04-25T08:36:26.550747+0000","last_active":"2024-04-25T08:36:26.551293+0000","last_peered":"2024-04-25T08:36:26.551293+0000","last_clean":"2024-04-25T08:36:26.551293+0000","last_became_active":"2024-04-25T08:36:26.550435+0000","last_became_peered":"2024-04-25T08:36:26.550435+0000","last_unstale":"2024-04-25T08:36:26.551293+0000","last_undegraded":"2024-04-25T08:36:26.551293+0000","last_fullsized":"2024-04-25T08:36:26.551293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:36.698952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534041+0000","last_change":"2024-04-25T08:36:26.534041+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534041+0000","last_undegraded":"2024-04-25T08:36:26.534041+0000","last_fullsized":"2024-04-25T08:36:26.534041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.553950+0000","last_change":"2024-04-25T08:36:26.552811+0000","last_active":"2024-04-25T08:36:26.553950+0000","last_peered":"2024-04-25T08:36:26.553950+0000","last_clean":"2024-04-25T08:36:26.553950+0000","last_became_active":"2024-04-25T08:36:26.552472+0000","last_became_peered":"2024-04-25T08:36:26.552472+0000","last_unstale":"2024-04-25T08:36:26.553950+0000","last_undegraded":"2024-04-25T08:36:26.553950+0000","last_fullsized":"2024-04-25T08:36:26.553950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:14.669260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534879+0000","last_change":"2024-04-25T08:36:26.534879+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534879+0000","last_undegraded":"2024-04-25T08:36:26.534879+0000","last_fullsized":"2024-04-25T08:36:26.534879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550100+0000","last_change":"2024-04-25T08:36:26.545891+0000","last_active":"2024-04-25T08:36:26.550100+0000","last_peered":"2024-04-25T08:36:26.550100+0000","last_clean":"2024-04-25T08:36:26.550100+0000","last_became_active":"2024-04-25T08:36:26.545452+0000","last_became_peered":"2024-04-25T08:36:26.545452+0000","last_unstale":"2024-04-25T08:36:26.550100+0000","last_undegraded":"2024-04-25T08:36:26.550100+0000","last_fullsized":"2024-04-25T08:36:26.550100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.121065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536761+0000","last_change":"2024-04-25T08:36:26.536761+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536761+0000","last_undegraded":"2024-04-25T08:36:26.536761+0000","last_fullsized":"2024-04-25T08:36:26.536761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549858+0000","last_change":"2024-04-25T08:36:26.547438+0000","last_active":"2024-04-25T08:36:26.549858+0000","last_peered":"2024-04-25T08:36:26.549858+0000","last_clean":"2024-04-25T08:36:26.549858+0000","last_became_active":"2024-04-25T08:36:26.547171+0000","last_became_peered":"2024-04-25T08:36:26.547171+0000","last_unstale":"2024-04-25T08:36:26.549858+0000","last_undegraded":"2024-04-25T08:36:26.549858+0000","last_fullsized":"2024-04-25T08:36:26.549858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:37.899171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536841+0000","last_change":"2024-04-25T08:36:26.536841+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536841+0000","last_undegraded":"2024-04-25T08:36:26.536841+0000","last_fullsized":"2024-04-25T08:36:26.536841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.546253+0000","last_change":"2024-04-25T08:36:26.543814+0000","last_active":"2024-04-25T08:36:26.546253+0000","last_peered":"2024-04-25T08:36:26.546253+0000","last_clean":"2024-04-25T08:36:26.546253+0000","last_became_active":"2024-04-25T08:36:26.543502+0000","last_became_peered":"2024-04-25T08:36:26.543502+0000","last_unstale":"2024-04-25T08:36:26.546253+0000","last_undegraded":"2024-04-25T08:36:26.546253+0000","last_fullsized":"2024-04-25T08:36:26.546253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:00.280937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540283+0000","last_change":"2024-04-25T08:36:26.540283+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540283+0000","last_undegraded":"2024-04-25T08:36:26.540283+0000","last_fullsized":"2024-04-25T08:36:26.540283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549698+0000","last_change":"2024-04-25T08:36:26.546563+0000","last_active":"2024-04-25T08:36:26.549698+0000","last_peered":"2024-04-25T08:36:26.549698+0000","last_clean":"2024-04-25T08:36:26.549698+0000","last_became_active":"2024-04-25T08:36:26.544474+0000","last_became_peered":"2024-04-25T08:36:26.544474+0000","last_unstale":"2024-04-25T08:36:26.549698+0000","last_undegraded":"2024-04-25T08:36:26.549698+0000","last_fullsized":"2024-04-25T08:36:26.549698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:02.642404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538228+0000","last_change":"2024-04-25T08:36:26.538228+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538228+0000","last_undegraded":"2024-04-25T08:36:26.538228+0000","last_fullsized":"2024-04-25T08:36:26.538228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547548+0000","last_change":"2024-04-25T08:36:26.541992+0000","last_active":"2024-04-25T08:36:26.547548+0000","last_peered":"2024-04-25T08:36:26.547548+0000","last_clean":"2024-04-25T08:36:26.547548+0000","last_became_active":"2024-04-25T08:36:26.541695+0000","last_became_peered":"2024-04-25T08:36:26.541695+0000","last_unstale":"2024-04-25T08:36:26.547548+0000","last_undegraded":"2024-04-25T08:36:26.547548+0000","last_fullsized":"2024-04-25T08:36:26.547548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:53.435350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537636+0000","last_change":"2024-04-25T08:36:26.537636+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537636+0000","last_undegraded":"2024-04-25T08:36:26.537636+0000","last_fullsized":"2024-04-25T08:36:26.537636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551034+0000","last_change":"2024-04-25T08:36:26.548435+0000","last_active":"2024-04-25T08:36:26.551034+0000","last_peered":"2024-04-25T08:36:26.551034+0000","last_clean":"2024-04-25T08:36:26.551034+0000","last_became_active":"2024-04-25T08:36:26.548141+0000","last_became_peered":"2024-04-25T08:36:26.548141+0000","last_unstale":"2024-04-25T08:36:26.551034+0000","last_undegraded":"2024-04-25T08:36:26.551034+0000","last_fullsized":"2024-04-25T08:36:26.551034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:42.921010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536476+0000","last_change":"2024-04-25T08:36:26.536476+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536476+0000","last_undegraded":"2024-04-25T08:36:26.536476+0000","last_fullsized":"2024-04-25T08:36:26.536476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.545484+0000","last_change":"2024-04-25T08:36:26.544037+0000","last_active":"2024-04-25T08:36:26.545484+0000","last_peered":"2024-04-25T08:36:26.545484+0000","last_clean":"2024-04-25T08:36:26.545484+0000","last_became_active":"2024-04-25T08:36:26.543747+0000","last_became_peered":"2024-04-25T08:36:26.543747+0000","last_unstale":"2024-04-25T08:36:26.545484+0000","last_undegraded":"2024-04-25T08:36:26.545484+0000","last_fullsized":"2024-04-25T08:36:26.545484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.121065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534472+0000","last_change":"2024-04-25T08:36:26.534472+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534472+0000","last_undegraded":"2024-04-25T08:36:26.534472+0000","last_fullsized":"2024-04-25T08:36:26.534472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.545018+0000","last_change":"2024-04-25T08:36:26.543672+0000","last_active":"2024-04-25T08:36:26.545018+0000","last_peered":"2024-04-25T08:36:26.545018+0000","last_clean":"2024-04-25T08:36:26.545018+0000","last_became_active":"2024-04-25T08:36:26.543395+0000","last_became_peered":"2024-04-25T08:36:26.543395+0000","last_unstale":"2024-04-25T08:36:26.545018+0000","last_undegraded":"2024-04-25T08:36:26.545018+0000","last_fullsized":"2024-04-25T08:36:26.545018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:58.975343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533707+0000","last_change":"2024-04-25T08:36:26.533707+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533707+0000","last_undegraded":"2024-04-25T08:36:26.533707+0000","last_fullsized":"2024-04-25T08:36:26.533707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.552206+0000","last_change":"2024-04-25T08:36:26.551656+0000","last_active":"2024-04-25T08:36:26.552206+0000","last_peered":"2024-04-25T08:36:26.552206+0000","last_clean":"2024-04-25T08:36:26.552206+0000","last_became_active":"2024-04-25T08:36:26.551356+0000","last_became_peered":"2024-04-25T08:36:26.551356+0000","last_unstale":"2024-04-25T08:36:26.552206+0000","last_undegraded":"2024-04-25T08:36:26.552206+0000","last_fullsized":"2024-04-25T08:36:26.552206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:46.989455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533723+0000","last_change":"2024-04-25T08:36:26.533723+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533723+0000","last_undegraded":"2024-04-25T08:36:26.533723+0000","last_fullsized":"2024-04-25T08:36:26.533723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550893+0000","last_change":"2024-04-25T08:36:26.549899+0000","last_active":"2024-04-25T08:36:26.550893+0000","last_peered":"2024-04-25T08:36:26.550893+0000","last_clean":"2024-04-25T08:36:26.550893+0000","last_became_active":"2024-04-25T08:36:26.549599+0000","last_became_peered":"2024-04-25T08:36:26.549599+0000","last_unstale":"2024-04-25T08:36:26.550893+0000","last_undegraded":"2024-04-25T08:36:26.550893+0000","last_fullsized":"2024-04-25T08:36:26.550893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:42.921010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534938+0000","last_change":"2024-04-25T08:36:26.534938+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534938+0000","last_undegraded":"2024-04-25T08:36:26.534938+0000","last_fullsized":"2024-04-25T08:36:26.534938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551852+0000","last_change":"2024-04-25T08:36:26.551038+0000","last_active":"2024-04-25T08:36:26.551852+0000","last_peered":"2024-04-25T08:36:26.551852+0000","last_clean":"2024-04-25T08:36:26.551852+0000","last_became_active":"2024-04-25T08:36:26.550729+0000","last_became_peered":"2024-04-25T08:36:26.550729+0000","last_unstale":"2024-04-25T08:36:26.551852+0000","last_undegraded":"2024-04-25T08:36:26.551852+0000","last_fullsized":"2024-04-25T08:36:26.551852+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:26.538681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536406+0000","last_change":"2024-04-25T08:36:26.536406+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536406+0000","last_undegraded":"2024-04-25T08:36:26.536406+0000","last_fullsized":"2024-04-25T08:36:26.536406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550248+0000","last_change":"2024-04-25T08:36:26.548899+0000","last_active":"2024-04-25T08:36:26.550248+0000","last_peered":"2024-04-25T08:36:26.550248+0000","last_clean":"2024-04-25T08:36:26.550248+0000","last_became_active":"2024-04-25T08:36:26.548616+0000","last_became_peered":"2024-04-25T08:36:26.548616+0000","last_unstale":"2024-04-25T08:36:26.550248+0000","last_undegraded":"2024-04-25T08:36:26.550248+0000","last_fullsized":"2024-04-25T08:36:26.550248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:00.902526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535097+0000","last_change":"2024-04-25T08:36:26.535097+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535097+0000","last_undegraded":"2024-04-25T08:36:26.535097+0000","last_fullsized":"2024-04-25T08:36:26.535097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547719+0000","last_change":"2024-04-25T08:36:26.546019+0000","last_active":"2024-04-25T08:36:26.547719+0000","last_peered":"2024-04-25T08:36:26.547719+0000","last_clean":"2024-04-25T08:36:26.547719+0000","last_became_active":"2024-04-25T08:36:26.545702+0000","last_became_peered":"2024-04-25T08:36:26.545702+0000","last_unstale":"2024-04-25T08:36:26.547719+0000","last_undegraded":"2024-04-25T08:36:26.547719+0000","last_fullsized":"2024-04-25T08:36:26.547719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:04.231004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533653+0000","last_change":"2024-04-25T08:36:26.533653+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533653+0000","last_undegraded":"2024-04-25T08:36:26.533653+0000","last_fullsized":"2024-04-25T08:36:26.533653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.553551+0000","last_change":"2024-04-25T08:36:26.551919+0000","last_active":"2024-04-25T08:36:26.553551+0000","last_peered":"2024-04-25T08:36:26.553551+0000","last_clean":"2024-04-25T08:36:26.553551+0000","last_became_active":"2024-04-25T08:36:26.550730+0000","last_became_peered":"2024-04-25T08:36:26.550730+0000","last_unstale":"2024-04-25T08:36:26.553551+0000","last_undegraded":"2024-04-25T08:36:26.553551+0000","last_fullsized":"2024-04-25T08:36:26.553551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.423931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538357+0000","last_change":"2024-04-25T08:36:26.538357+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538357+0000","last_undegraded":"2024-04-25T08:36:26.538357+0000","last_fullsized":"2024-04-25T08:36:26.538357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549068+0000","last_change":"2024-04-25T08:36:26.546564+0000","last_active":"2024-04-25T08:36:26.549068+0000","last_peered":"2024-04-25T08:36:26.549068+0000","last_clean":"2024-04-25T08:36:26.549068+0000","last_became_active":"2024-04-25T08:36:26.546293+0000","last_became_peered":"2024-04-25T08:36:26.546293+0000","last_unstale":"2024-04-25T08:36:26.549068+0000","last_undegraded":"2024-04-25T08:36:26.549068+0000","last_fullsized":"2024-04-25T08:36:26.549068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:53.435350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.539109+0000","last_change":"2024-04-25T08:36:26.539109+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.539109+0000","last_undegraded":"2024-04-25T08:36:26.539109+0000","last_fullsized":"2024-04-25T08:36:26.539109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547656+0000","last_change":"2024-04-25T08:36:26.546007+0000","last_active":"2024-04-25T08:36:26.547656+0000","last_peered":"2024-04-25T08:36:26.547656+0000","last_clean":"2024-04-25T08:36:26.547656+0000","last_became_active":"2024-04-25T08:36:26.545660+0000","last_became_peered":"2024-04-25T08:36:26.545660+0000","last_unstale":"2024-04-25T08:36:26.547656+0000","last_undegraded":"2024-04-25T08:36:26.547656+0000","last_fullsized":"2024-04-25T08:36:26.547656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:48.548532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536701+0000","last_change":"2024-04-25T08:36:26.536701+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536701+0000","last_undegraded":"2024-04-25T08:36:26.536701+0000","last_fullsized":"2024-04-25T08:36:26.536701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.545751+0000","last_change":"2024-04-25T08:36:26.544427+0000","last_active":"2024-04-25T08:36:26.545751+0000","last_peered":"2024-04-25T08:36:26.545751+0000","last_clean":"2024-04-25T08:36:26.545751+0000","last_became_active":"2024-04-25T08:36:26.544096+0000","last_became_peered":"2024-04-25T08:36:26.544096+0000","last_unstale":"2024-04-25T08:36:26.545751+0000","last_undegraded":"2024-04-25T08:36:26.545751+0000","last_fullsized":"2024-04-25T08:36:26.545751+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:44.885589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":283,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.537575+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:36:26.537575+0000","last_peered":"2024-04-25T08:36:26.537575+0000","last_clean":"2024-04-25T08:36:26.537575+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:36:26.537575+0000","last_undegraded":"2024-04-25T08:36:26.537575+0000","last_fullsized":"2024-04-25T08:36:26.537575+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46608,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325232,"statfs":{"total":96636764160,"available":96589037568,"internally_reserved":0,"allocated":1572864,"data_stored":1040288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40456,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331384,"statfs":{"total":96636764160,"available":96595337216,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46024,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325816,"statfs":{"total":96636764160,"available":96589635584,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40456,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331384,"statfs":{"total":96636764160,"available":96595337216,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54912,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316928,"statfs":{"total":96636764160,"available":96580534272,"internally_reserved":0,"allocated":1560576,"data_stored":1033621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:36:31.549 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:36:31.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.692+0000 7f4c7e17d640 1 Processor -- start 2024-04-25T08:36:31.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.692+0000 7f4c7e17d640 1 -- start start 2024-04-25T08:36:31.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.692+0000 7f4c7e17d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4c780d3900 0x7f4c780d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.692+0000 7f4c7e17d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c780d0d90 0x7f4c780b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.692+0000 7f4c7e17d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c780d3e70 con 0x7f4c7815dc10 2024-04-25T08:36:31.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.692+0000 7f4c7e17d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7815dff0 con 0x7f4c780d3900 2024-04-25T08:36:31.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.692+0000 7f4c7e17d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c780b6990 con 0x7f4c780d0d90 2024-04-25T08:36:31.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.693+0000 7f4c76ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4c780d3900 0x7f4c780d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.693+0000 7f4c76ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4c780d3900 0x7f4c780d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47710/0 (socket says 172.21.15.38:47710) 2024-04-25T08:36:31.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.693+0000 7f4c76ffd640 1 -- 172.21.15.38:0/1843418789 learned_addr learned my addr 172.21.15.38:0/1843418789 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:31.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.693+0000 7f4c77fff640 1 --2- 172.21.15.38:0/1843418789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c780d0d90 0x7f4c780b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.693+0000 7f4c76ffd640 1 -- 172.21.15.38:0/1843418789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c780d0d90 msgr2=0x7f4c780b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:31.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.693+0000 7f4c76ffd640 1 --2- 172.21.15.38:0/1843418789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c780d0d90 0x7f4c780b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.693+0000 7f4c76ffd640 1 -- 172.21.15.38:0/1843418789 >> v1:172.21.15.107:6789/0 conn(0x7f4c7815dc10 legacy=0x7f4c780ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.693+0000 7f4c76ffd640 1 -- 172.21.15.38:0/1843418789 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c780d4150 con 0x7f4c780d3900 2024-04-25T08:36:31.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.694+0000 7f4c76ffd640 1 --2- 172.21.15.38:0/1843418789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c780d3900 0x7f4c780d0850 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009870 tx=0x7f4c6c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56d4297daa90d816 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:31.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.694+0000 7f4c767fc640 1 -- 172.21.15.38:0/1843418789 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c037070 con 0x7f4c780d3900 2024-04-25T08:36:31.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.694+0000 7f4c767fc640 1 -- 172.21.15.38:0/1843418789 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c032440 con 0x7f4c780d3900 2024-04-25T08:36:31.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.694+0000 7f4c767fc640 1 -- 172.21.15.38:0/1843418789 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c032810 con 0x7f4c780d3900 2024-04-25T08:36:31.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.694+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1843418789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c780d3900 msgr2=0x7f4c780d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.694+0000 7f4c7e17d640 1 --2- 172.21.15.38:0/1843418789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c780d3900 0x7f4c780d0850 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009870 tx=0x7f4c6c02a1c0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.694+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1843418789 shutdown_connections 2024-04-25T08:36:31.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.694+0000 7f4c7e17d640 1 --2- 172.21.15.38:0/1843418789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c780d0d90 0x7f4c780b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.694+0000 7f4c7e17d640 1 --2- 172.21.15.38:0/1843418789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c780d3900 0x7f4c780d0850 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.694+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1843418789 >> 172.21.15.38:0/1843418789 conn(0x7f4c78066d10 msgr2=0x7f4c78067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:31.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.694+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1843418789 shutdown_connections 2024-04-25T08:36:31.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.694+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1843418789 wait complete. 2024-04-25T08:36:31.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.696+0000 7f4c7e17d640 1 Processor -- start 2024-04-25T08:36:31.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.696+0000 7f4c7e17d640 1 -- start start 2024-04-25T08:36:31.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.696+0000 7f4c7e17d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c780d0d90 0x7f4c780cca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.696+0000 7f4c7e17d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c780d3900 0x7f4c780aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.696+0000 7f4c7e17d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c7815dc10 0x7f4c7815ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.696+0000 7f4c7e17d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c780b6bc0 con 0x7f4c780d3900 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.696+0000 7f4c7e17d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c780d4710 con 0x7f4c7815dc10 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.696+0000 7f4c7e17d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c780cef30 con 0x7f4c780d0d90 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c76ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c780d3900 0x7f4c780aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c76ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c780d3900 0x7f4c780aacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47712/0 (socket says 172.21.15.38:47712) 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c76ffd640 1 -- 172.21.15.38:0/1580348276 learned_addr learned my addr 172.21.15.38:0/1580348276 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c77fff640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c7815dc10 0x7f4c7815ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c777fe640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c780d0d90 0x7f4c780cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c76ffd640 1 -- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c780d0d90 msgr2=0x7f4c780cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c76ffd640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c780d0d90 0x7f4c780cca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c76ffd640 1 -- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c7815dc10 msgr2=0x7f4c7815ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c76ffd640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c7815dc10 0x7f4c7815ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c76ffd640 1 -- 172.21.15.38:0/1580348276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c780ab7a0 con 0x7f4c780d3900 2024-04-25T08:36:31.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c77fff640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c7815dc10 0x7f4c7815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:31.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c777fe640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c780d0d90 0x7f4c780cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:31.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c76ffd640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c780d3900 0x7f4c780aacd0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c000c00 tx=0x7f4c6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:31.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c5ffff640 1 -- 172.21.15.38:0/1580348276 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c037040 con 0x7f4c780d3900 2024-04-25T08:36:31.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c5ffff640 1 -- 172.21.15.38:0/1580348276 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c042070 con 0x7f4c780d3900 2024-04-25T08:36:31.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1580348276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c780aba80 con 0x7f4c780d3900 2024-04-25T08:36:31.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c5ffff640 1 -- 172.21.15.38:0/1580348276 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c030880 con 0x7f4c780d3900 2024-04-25T08:36:31.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.697+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1580348276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c780ac040 con 0x7f4c780d3900 2024-04-25T08:36:31.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.698+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1580348276 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c780ac380 con 0x7f4c780d3900 2024-04-25T08:36:31.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.699+0000 7f4c5ffff640 1 -- 172.21.15.38:0/1580348276 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c6c02ad10 con 0x7f4c780d3900 2024-04-25T08:36:31.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.699+0000 7f4c5ffff640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4c6406b4d0 0x7f4c6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:31.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.699+0000 7f4c5ffff640 1 -- 172.21.15.38:0/1580348276 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c6c0aa9d0 con 0x7f4c780d3900 2024-04-25T08:36:31.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.699+0000 7f4c777fe640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4c6406b4d0 0x7f4c6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:31.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.700+0000 7f4c777fe640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4c6406b4d0 0x7f4c6406d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4c68001680 tx=0x7f4c68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:31.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.703+0000 7f4c5ffff640 1 -- 172.21.15.38:0/1580348276 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c780ac380 con 0x7f4c780d3900 2024-04-25T08:36:31.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.882+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1580348276 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c78068e80 con 0x7f4c6406b4d0 2024-04-25T08:36:31.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.888+0000 7f4c5ffff640 1 -- 172.21.15.38:0/1580348276 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246753 (secure 0 0 0) 0x7f4c78068e80 con 0x7f4c6406b4d0 2024-04-25T08:36:31.885 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:31.886 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:36:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.891+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4c6406b4d0 msgr2=0x7f4c6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.891+0000 7f4c7e17d640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4c6406b4d0 0x7f4c6406d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4c68001680 tx=0x7f4c68009040 comp rx=0 tx=0).stop 2024-04-25T08:36:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.891+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1580348276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c780d3900 msgr2=0x7f4c780aacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.891+0000 7f4c7e17d640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c780d3900 0x7f4c780aacd0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c000c00 tx=0x7f4c6c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.891+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1580348276 shutdown_connections 2024-04-25T08:36:31.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.891+0000 7f4c7e17d640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4c7815dc10 0x7f4c7815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.891+0000 7f4c7e17d640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c780d3900 0x7f4c780aacd0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.891+0000 7f4c7e17d640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4c6406b4d0 0x7f4c6406d990 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.891+0000 7f4c7e17d640 1 --2- 172.21.15.38:0/1580348276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4c780d0d90 0x7f4c780cca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:31.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.891+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1580348276 >> 172.21.15.38:0/1580348276 conn(0x7f4c78066d10 msgr2=0x7f4c7815e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:31.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.891+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1580348276 shutdown_connections 2024-04-25T08:36:31.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:31.891+0000 7f4c7e17d640 1 -- 172.21.15.38:0/1580348276 wait complete. 2024-04-25T08:36:31.896 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-04-25T08:36:31.273377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":337352,"kb_used_data":8796,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754637368,"statfs":{"total":773094113280,"available":772748664832,"internally_reserved":0,"allocated":9007104,"data_stored":4773429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":336318000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.742948"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537773+0000","last_change":"2024-04-25T08:36:26.537773+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537773+0000","last_undegraded":"2024-04-25T08:36:26.537773+0000","last_fullsized":"2024-04-25T08:36:26.537773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.544249+0000","last_change":"2024-04-25T08:36:26.544249+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.544249+0000","last_undegraded":"2024-04-25T08:36:26.544249+0000","last_fullsized":"2024-04-25T08:36:26.544249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.545202+0000","last_change":"2024-04-25T08:36:26.545202+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.545202+0000","last_undegraded":"2024-04-25T08:36:26.545202+0000","last_fullsized":"2024-04-25T08:36:26.545202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538453+0000","last_change":"2024-04-25T08:36:26.538453+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538453+0000","last_undegraded":"2024-04-25T08:36:26.538453+0000","last_fullsized":"2024-04-25T08:36:26.538453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538575+0000","last_change":"2024-04-25T08:36:26.538575+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538575+0000","last_undegraded":"2024-04-25T08:36:26.538575+0000","last_fullsized":"2024-04-25T08:36:26.538575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538753+0000","last_change":"2024-04-25T08:36:26.538753+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538753+0000","last_undegraded":"2024-04-25T08:36:26.538753+0000","last_fullsized":"2024-04-25T08:36:26.538753+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534187+0000","last_change":"2024-04-25T08:36:26.534187+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534187+0000","last_undegraded":"2024-04-25T08:36:26.534187+0000","last_fullsized":"2024-04-25T08:36:26.534187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537353+0000","last_change":"2024-04-25T08:36:26.537353+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537353+0000","last_undegraded":"2024-04-25T08:36:26.537353+0000","last_fullsized":"2024-04-25T08:36:26.537353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534390+0000","last_change":"2024-04-25T08:36:26.534390+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534390+0000","last_undegraded":"2024-04-25T08:36:26.534390+0000","last_fullsized":"2024-04-25T08:36:26.534390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538815+0000","last_change":"2024-04-25T08:36:26.538815+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538815+0000","last_undegraded":"2024-04-25T08:36:26.538815+0000","last_fullsized":"2024-04-25T08:36:26.538815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.543800+0000","last_change":"2024-04-25T08:36:26.543800+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.543800+0000","last_undegraded":"2024-04-25T08:36:26.543800+0000","last_fullsized":"2024-04-25T08:36:26.543800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537131+0000","last_change":"2024-04-25T08:36:26.537131+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537131+0000","last_undegraded":"2024-04-25T08:36:26.537131+0000","last_fullsized":"2024-04-25T08:36:26.537131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536911+0000","last_change":"2024-04-25T08:36:26.536911+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536911+0000","last_undegraded":"2024-04-25T08:36:26.536911+0000","last_fullsized":"2024-04-25T08:36:26.536911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538336+0000","last_change":"2024-04-25T08:36:26.538336+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538336+0000","last_undegraded":"2024-04-25T08:36:26.538336+0000","last_fullsized":"2024-04-25T08:36:26.538336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.544447+0000","last_change":"2024-04-25T08:36:26.544447+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.544447+0000","last_undegraded":"2024-04-25T08:36:26.544447+0000","last_fullsized":"2024-04-25T08:36:26.544447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537142+0000","last_change":"2024-04-25T08:36:26.537142+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537142+0000","last_undegraded":"2024-04-25T08:36:26.537142+0000","last_fullsized":"2024-04-25T08:36:26.537142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537724+0000","last_change":"2024-04-25T08:36:26.537724+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537724+0000","last_undegraded":"2024-04-25T08:36:26.537724+0000","last_fullsized":"2024-04-25T08:36:26.537724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.543802+0000","last_change":"2024-04-25T08:36:26.543802+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.543802+0000","last_undegraded":"2024-04-25T08:36:26.543802+0000","last_fullsized":"2024-04-25T08:36:26.543802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537441+0000","last_change":"2024-04-25T08:36:26.537441+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537441+0000","last_undegraded":"2024-04-25T08:36:26.537441+0000","last_fullsized":"2024-04-25T08:36:26.537441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.539548+0000","last_change":"2024-04-25T08:36:26.539548+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.539548+0000","last_undegraded":"2024-04-25T08:36:26.539548+0000","last_fullsized":"2024-04-25T08:36:26.539548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536226+0000","last_change":"2024-04-25T08:36:26.536226+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536226+0000","last_undegraded":"2024-04-25T08:36:26.536226+0000","last_fullsized":"2024-04-25T08:36:26.536226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538328+0000","last_change":"2024-04-25T08:36:26.538328+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538328+0000","last_undegraded":"2024-04-25T08:36:26.538328+0000","last_fullsized":"2024-04-25T08:36:26.538328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534103+0000","last_change":"2024-04-25T08:36:26.534103+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534103+0000","last_undegraded":"2024-04-25T08:36:26.534103+0000","last_fullsized":"2024-04-25T08:36:26.534103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.541048+0000","last_change":"2024-04-25T08:36:26.541048+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.541048+0000","last_undegraded":"2024-04-25T08:36:26.541048+0000","last_fullsized":"2024-04-25T08:36:26.541048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540285+0000","last_change":"2024-04-25T08:36:26.540285+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540285+0000","last_undegraded":"2024-04-25T08:36:26.540285+0000","last_fullsized":"2024-04-25T08:36:26.540285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537767+0000","last_change":"2024-04-25T08:36:26.537767+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537767+0000","last_undegraded":"2024-04-25T08:36:26.537767+0000","last_fullsized":"2024-04-25T08:36:26.537767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537155+0000","last_change":"2024-04-25T08:36:26.537155+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537155+0000","last_undegraded":"2024-04-25T08:36:26.537155+0000","last_fullsized":"2024-04-25T08:36:26.537155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538315+0000","last_change":"2024-04-25T08:36:26.538315+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538315+0000","last_undegraded":"2024-04-25T08:36:26.538315+0000","last_fullsized":"2024-04-25T08:36:26.538315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538511+0000","last_change":"2024-04-25T08:36:26.538511+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538511+0000","last_undegraded":"2024-04-25T08:36:26.538511+0000","last_fullsized":"2024-04-25T08:36:26.538511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536776+0000","last_change":"2024-04-25T08:36:26.536776+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536776+0000","last_undegraded":"2024-04-25T08:36:26.536776+0000","last_fullsized":"2024-04-25T08:36:26.536776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540684+0000","last_change":"2024-04-25T08:36:26.540684+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540684+0000","last_undegraded":"2024-04-25T08:36:26.540684+0000","last_fullsized":"2024-04-25T08:36:26.540684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536655+0000","last_change":"2024-04-25T08:36:26.536655+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536655+0000","last_undegraded":"2024-04-25T08:36:26.536655+0000","last_fullsized":"2024-04-25T08:36:26.536655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.539609+0000","last_change":"2024-04-25T08:36:26.539609+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.539609+0000","last_undegraded":"2024-04-25T08:36:26.539609+0000","last_fullsized":"2024-04-25T08:36:26.539609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.553534+0000","last_change":"2024-04-25T08:36:26.552070+0000","last_active":"2024-04-25T08:36:26.553534+0000","last_peered":"2024-04-25T08:36:26.553534+0000","last_clean":"2024-04-25T08:36:26.553534+0000","last_became_active":"2024-04-25T08:36:26.550647+0000","last_became_peered":"2024-04-25T08:36:26.550647+0000","last_unstale":"2024-04-25T08:36:26.553534+0000","last_undegraded":"2024-04-25T08:36:26.553534+0000","last_fullsized":"2024-04-25T08:36:26.553534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:18.633914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537186+0000","last_change":"2024-04-25T08:36:26.537186+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537186+0000","last_undegraded":"2024-04-25T08:36:26.537186+0000","last_fullsized":"2024-04-25T08:36:26.537186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551229+0000","last_change":"2024-04-25T08:36:26.549707+0000","last_active":"2024-04-25T08:36:26.551229+0000","last_peered":"2024-04-25T08:36:26.551229+0000","last_clean":"2024-04-25T08:36:26.551229+0000","last_became_active":"2024-04-25T08:36:26.549227+0000","last_became_peered":"2024-04-25T08:36:26.549227+0000","last_unstale":"2024-04-25T08:36:26.551229+0000","last_undegraded":"2024-04-25T08:36:26.551229+0000","last_fullsized":"2024-04-25T08:36:26.551229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:30.421525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535476+0000","last_change":"2024-04-25T08:36:26.535476+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535476+0000","last_undegraded":"2024-04-25T08:36:26.535476+0000","last_fullsized":"2024-04-25T08:36:26.535476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.552110+0000","last_change":"2024-04-25T08:36:26.551664+0000","last_active":"2024-04-25T08:36:26.552110+0000","last_peered":"2024-04-25T08:36:26.552110+0000","last_clean":"2024-04-25T08:36:26.552110+0000","last_became_active":"2024-04-25T08:36:26.551388+0000","last_became_peered":"2024-04-25T08:36:26.551388+0000","last_unstale":"2024-04-25T08:36:26.552110+0000","last_undegraded":"2024-04-25T08:36:26.552110+0000","last_fullsized":"2024-04-25T08:36:26.552110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:58.975343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535807+0000","last_change":"2024-04-25T08:36:26.535807+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535807+0000","last_undegraded":"2024-04-25T08:36:26.535807+0000","last_fullsized":"2024-04-25T08:36:26.535807+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547750+0000","last_change":"2024-04-25T08:36:26.545919+0000","last_active":"2024-04-25T08:36:26.547750+0000","last_peered":"2024-04-25T08:36:26.547750+0000","last_clean":"2024-04-25T08:36:26.547750+0000","last_became_active":"2024-04-25T08:36:26.545633+0000","last_became_peered":"2024-04-25T08:36:26.545633+0000","last_unstale":"2024-04-25T08:36:26.547750+0000","last_undegraded":"2024-04-25T08:36:26.547750+0000","last_fullsized":"2024-04-25T08:36:26.547750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:27.158628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538946+0000","last_change":"2024-04-25T08:36:26.538946+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538946+0000","last_undegraded":"2024-04-25T08:36:26.538946+0000","last_fullsized":"2024-04-25T08:36:26.538946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550474+0000","last_change":"2024-04-25T08:36:26.549171+0000","last_active":"2024-04-25T08:36:26.550474+0000","last_peered":"2024-04-25T08:36:26.550474+0000","last_clean":"2024-04-25T08:36:26.550474+0000","last_became_active":"2024-04-25T08:36:26.548875+0000","last_became_peered":"2024-04-25T08:36:26.548875+0000","last_unstale":"2024-04-25T08:36:26.550474+0000","last_undegraded":"2024-04-25T08:36:26.550474+0000","last_fullsized":"2024-04-25T08:36:26.550474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:04.789878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536149+0000","last_change":"2024-04-25T08:36:26.536149+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536149+0000","last_undegraded":"2024-04-25T08:36:26.536149+0000","last_fullsized":"2024-04-25T08:36:26.536149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550104+0000","last_change":"2024-04-25T08:36:26.544834+0000","last_active":"2024-04-25T08:36:26.550104+0000","last_peered":"2024-04-25T08:36:26.550104+0000","last_clean":"2024-04-25T08:36:26.550104+0000","last_became_active":"2024-04-25T08:36:26.544549+0000","last_became_peered":"2024-04-25T08:36:26.544549+0000","last_unstale":"2024-04-25T08:36:26.550104+0000","last_undegraded":"2024-04-25T08:36:26.550104+0000","last_fullsized":"2024-04-25T08:36:26.550104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:59.366292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533938+0000","last_change":"2024-04-25T08:36:26.533938+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533938+0000","last_undegraded":"2024-04-25T08:36:26.533938+0000","last_fullsized":"2024-04-25T08:36:26.533938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550148+0000","last_change":"2024-04-25T08:36:26.546765+0000","last_active":"2024-04-25T08:36:26.550148+0000","last_peered":"2024-04-25T08:36:26.550148+0000","last_clean":"2024-04-25T08:36:26.550148+0000","last_became_active":"2024-04-25T08:36:26.546467+0000","last_became_peered":"2024-04-25T08:36:26.546467+0000","last_unstale":"2024-04-25T08:36:26.550148+0000","last_undegraded":"2024-04-25T08:36:26.550148+0000","last_fullsized":"2024-04-25T08:36:26.550148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:44.885589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535281+0000","last_change":"2024-04-25T08:36:26.535281+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535281+0000","last_undegraded":"2024-04-25T08:36:26.535281+0000","last_fullsized":"2024-04-25T08:36:26.535281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.548729+0000","last_change":"2024-04-25T08:36:26.544689+0000","last_active":"2024-04-25T08:36:26.548729+0000","last_peered":"2024-04-25T08:36:26.548729+0000","last_clean":"2024-04-25T08:36:26.548729+0000","last_became_active":"2024-04-25T08:36:26.544353+0000","last_became_peered":"2024-04-25T08:36:26.544353+0000","last_unstale":"2024-04-25T08:36:26.548729+0000","last_undegraded":"2024-04-25T08:36:26.548729+0000","last_fullsized":"2024-04-25T08:36:26.548729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:19.783413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536997+0000","last_change":"2024-04-25T08:36:26.536997+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536997+0000","last_undegraded":"2024-04-25T08:36:26.536997+0000","last_fullsized":"2024-04-25T08:36:26.536997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551564+0000","last_change":"2024-04-25T08:36:26.549714+0000","last_active":"2024-04-25T08:36:26.551564+0000","last_peered":"2024-04-25T08:36:26.551564+0000","last_clean":"2024-04-25T08:36:26.551564+0000","last_became_active":"2024-04-25T08:36:26.548992+0000","last_became_peered":"2024-04-25T08:36:26.548992+0000","last_unstale":"2024-04-25T08:36:26.551564+0000","last_undegraded":"2024-04-25T08:36:26.551564+0000","last_fullsized":"2024-04-25T08:36:26.551564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:57.849522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536670+0000","last_change":"2024-04-25T08:36:26.536670+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536670+0000","last_undegraded":"2024-04-25T08:36:26.536670+0000","last_fullsized":"2024-04-25T08:36:26.536670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550140+0000","last_change":"2024-04-25T08:36:26.546855+0000","last_active":"2024-04-25T08:36:26.550140+0000","last_peered":"2024-04-25T08:36:26.550140+0000","last_clean":"2024-04-25T08:36:26.550140+0000","last_became_active":"2024-04-25T08:36:26.546607+0000","last_became_peered":"2024-04-25T08:36:26.546607+0000","last_unstale":"2024-04-25T08:36:26.550140+0000","last_undegraded":"2024-04-25T08:36:26.550140+0000","last_fullsized":"2024-04-25T08:36:26.550140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:26.454697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537812+0000","last_change":"2024-04-25T08:36:26.537812+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537812+0000","last_undegraded":"2024-04-25T08:36:26.537812+0000","last_fullsized":"2024-04-25T08:36:26.537812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549618+0000","last_change":"2024-04-25T08:36:26.548301+0000","last_active":"2024-04-25T08:36:26.549618+0000","last_peered":"2024-04-25T08:36:26.549618+0000","last_clean":"2024-04-25T08:36:26.549618+0000","last_became_active":"2024-04-25T08:36:26.547849+0000","last_became_peered":"2024-04-25T08:36:26.547849+0000","last_unstale":"2024-04-25T08:36:26.549618+0000","last_undegraded":"2024-04-25T08:36:26.549618+0000","last_fullsized":"2024-04-25T08:36:26.549618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.444339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540620+0000","last_change":"2024-04-25T08:36:26.540620+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540620+0000","last_undegraded":"2024-04-25T08:36:26.540620+0000","last_fullsized":"2024-04-25T08:36:26.540620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.546872+0000","last_change":"2024-04-25T08:36:26.545189+0000","last_active":"2024-04-25T08:36:26.546872+0000","last_peered":"2024-04-25T08:36:26.546872+0000","last_clean":"2024-04-25T08:36:26.546872+0000","last_became_active":"2024-04-25T08:36:26.544888+0000","last_became_peered":"2024-04-25T08:36:26.544888+0000","last_unstale":"2024-04-25T08:36:26.546872+0000","last_undegraded":"2024-04-25T08:36:26.546872+0000","last_fullsized":"2024-04-25T08:36:26.546872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:33.972987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536242+0000","last_change":"2024-04-25T08:36:26.536242+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536242+0000","last_undegraded":"2024-04-25T08:36:26.536242+0000","last_fullsized":"2024-04-25T08:36:26.536242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.548075+0000","last_change":"2024-04-25T08:36:26.544584+0000","last_active":"2024-04-25T08:36:26.548075+0000","last_peered":"2024-04-25T08:36:26.548075+0000","last_clean":"2024-04-25T08:36:26.548075+0000","last_became_active":"2024-04-25T08:36:26.544307+0000","last_became_peered":"2024-04-25T08:36:26.544307+0000","last_unstale":"2024-04-25T08:36:26.548075+0000","last_undegraded":"2024-04-25T08:36:26.548075+0000","last_fullsized":"2024-04-25T08:36:26.548075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:04.226062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535774+0000","last_change":"2024-04-25T08:36:26.535774+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535774+0000","last_undegraded":"2024-04-25T08:36:26.535774+0000","last_fullsized":"2024-04-25T08:36:26.535774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551293+0000","last_change":"2024-04-25T08:36:26.550747+0000","last_active":"2024-04-25T08:36:26.551293+0000","last_peered":"2024-04-25T08:36:26.551293+0000","last_clean":"2024-04-25T08:36:26.551293+0000","last_became_active":"2024-04-25T08:36:26.550435+0000","last_became_peered":"2024-04-25T08:36:26.550435+0000","last_unstale":"2024-04-25T08:36:26.551293+0000","last_undegraded":"2024-04-25T08:36:26.551293+0000","last_fullsized":"2024-04-25T08:36:26.551293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:36.698952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534041+0000","last_change":"2024-04-25T08:36:26.534041+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534041+0000","last_undegraded":"2024-04-25T08:36:26.534041+0000","last_fullsized":"2024-04-25T08:36:26.534041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.553950+0000","last_change":"2024-04-25T08:36:26.552811+0000","last_active":"2024-04-25T08:36:26.553950+0000","last_peered":"2024-04-25T08:36:26.553950+0000","last_clean":"2024-04-25T08:36:26.553950+0000","last_became_active":"2024-04-25T08:36:26.552472+0000","last_became_peered":"2024-04-25T08:36:26.552472+0000","last_unstale":"2024-04-25T08:36:26.553950+0000","last_undegraded":"2024-04-25T08:36:26.553950+0000","last_fullsized":"2024-04-25T08:36:26.553950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:14.669260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534879+0000","last_change":"2024-04-25T08:36:26.534879+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534879+0000","last_undegraded":"2024-04-25T08:36:26.534879+0000","last_fullsized":"2024-04-25T08:36:26.534879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550100+0000","last_change":"2024-04-25T08:36:26.545891+0000","last_active":"2024-04-25T08:36:26.550100+0000","last_peered":"2024-04-25T08:36:26.550100+0000","last_clean":"2024-04-25T08:36:26.550100+0000","last_became_active":"2024-04-25T08:36:26.545452+0000","last_became_peered":"2024-04-25T08:36:26.545452+0000","last_unstale":"2024-04-25T08:36:26.550100+0000","last_undegraded":"2024-04-25T08:36:26.550100+0000","last_fullsized":"2024-04-25T08:36:26.550100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.121065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536761+0000","last_change":"2024-04-25T08:36:26.536761+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536761+0000","last_undegraded":"2024-04-25T08:36:26.536761+0000","last_fullsized":"2024-04-25T08:36:26.536761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549858+0000","last_change":"2024-04-25T08:36:26.547438+0000","last_active":"2024-04-25T08:36:26.549858+0000","last_peered":"2024-04-25T08:36:26.549858+0000","last_clean":"2024-04-25T08:36:26.549858+0000","last_became_active":"2024-04-25T08:36:26.547171+0000","last_became_peered":"2024-04-25T08:36:26.547171+0000","last_unstale":"2024-04-25T08:36:26.549858+0000","last_undegraded":"2024-04-25T08:36:26.549858+0000","last_fullsized":"2024-04-25T08:36:26.549858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:37.899171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536841+0000","last_change":"2024-04-25T08:36:26.536841+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536841+0000","last_undegraded":"2024-04-25T08:36:26.536841+0000","last_fullsized":"2024-04-25T08:36:26.536841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.546253+0000","last_change":"2024-04-25T08:36:26.543814+0000","last_active":"2024-04-25T08:36:26.546253+0000","last_peered":"2024-04-25T08:36:26.546253+0000","last_clean":"2024-04-25T08:36:26.546253+0000","last_became_active":"2024-04-25T08:36:26.543502+0000","last_became_peered":"2024-04-25T08:36:26.543502+0000","last_unstale":"2024-04-25T08:36:26.546253+0000","last_undegraded":"2024-04-25T08:36:26.546253+0000","last_fullsized":"2024-04-25T08:36:26.546253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:00.280937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540283+0000","last_change":"2024-04-25T08:36:26.540283+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540283+0000","last_undegraded":"2024-04-25T08:36:26.540283+0000","last_fullsized":"2024-04-25T08:36:26.540283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549698+0000","last_change":"2024-04-25T08:36:26.546563+0000","last_active":"2024-04-25T08:36:26.549698+0000","last_peered":"2024-04-25T08:36:26.549698+0000","last_clean":"2024-04-25T08:36:26.549698+0000","last_became_active":"2024-04-25T08:36:26.544474+0000","last_became_peered":"2024-04-25T08:36:26.544474+0000","last_unstale":"2024-04-25T08:36:26.549698+0000","last_undegraded":"2024-04-25T08:36:26.549698+0000","last_fullsized":"2024-04-25T08:36:26.549698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:02.642404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538228+0000","last_change":"2024-04-25T08:36:26.538228+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538228+0000","last_undegraded":"2024-04-25T08:36:26.538228+0000","last_fullsized":"2024-04-25T08:36:26.538228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547548+0000","last_change":"2024-04-25T08:36:26.541992+0000","last_active":"2024-04-25T08:36:26.547548+0000","last_peered":"2024-04-25T08:36:26.547548+0000","last_clean":"2024-04-25T08:36:26.547548+0000","last_became_active":"2024-04-25T08:36:26.541695+0000","last_became_peered":"2024-04-25T08:36:26.541695+0000","last_unstale":"2024-04-25T08:36:26.547548+0000","last_undegraded":"2024-04-25T08:36:26.547548+0000","last_fullsized":"2024-04-25T08:36:26.547548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:53.435350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537636+0000","last_change":"2024-04-25T08:36:26.537636+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537636+0000","last_undegraded":"2024-04-25T08:36:26.537636+0000","last_fullsized":"2024-04-25T08:36:26.537636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551034+0000","last_change":"2024-04-25T08:36:26.548435+0000","last_active":"2024-04-25T08:36:26.551034+0000","last_peered":"2024-04-25T08:36:26.551034+0000","last_clean":"2024-04-25T08:36:26.551034+0000","last_became_active":"2024-04-25T08:36:26.548141+0000","last_became_peered":"2024-04-25T08:36:26.548141+0000","last_unstale":"2024-04-25T08:36:26.551034+0000","last_undegraded":"2024-04-25T08:36:26.551034+0000","last_fullsized":"2024-04-25T08:36:26.551034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:42.921010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536476+0000","last_change":"2024-04-25T08:36:26.536476+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536476+0000","last_undegraded":"2024-04-25T08:36:26.536476+0000","last_fullsized":"2024-04-25T08:36:26.536476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.545484+0000","last_change":"2024-04-25T08:36:26.544037+0000","last_active":"2024-04-25T08:36:26.545484+0000","last_peered":"2024-04-25T08:36:26.545484+0000","last_clean":"2024-04-25T08:36:26.545484+0000","last_became_active":"2024-04-25T08:36:26.543747+0000","last_became_peered":"2024-04-25T08:36:26.543747+0000","last_unstale":"2024-04-25T08:36:26.545484+0000","last_undegraded":"2024-04-25T08:36:26.545484+0000","last_fullsized":"2024-04-25T08:36:26.545484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.121065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534472+0000","last_change":"2024-04-25T08:36:26.534472+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534472+0000","last_undegraded":"2024-04-25T08:36:26.534472+0000","last_fullsized":"2024-04-25T08:36:26.534472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.545018+0000","last_change":"2024-04-25T08:36:26.543672+0000","last_active":"2024-04-25T08:36:26.545018+0000","last_peered":"2024-04-25T08:36:26.545018+0000","last_clean":"2024-04-25T08:36:26.545018+0000","last_became_active":"2024-04-25T08:36:26.543395+0000","last_became_peered":"2024-04-25T08:36:26.543395+0000","last_unstale":"2024-04-25T08:36:26.545018+0000","last_undegraded":"2024-04-25T08:36:26.545018+0000","last_fullsized":"2024-04-25T08:36:26.545018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:58.975343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533707+0000","last_change":"2024-04-25T08:36:26.533707+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533707+0000","last_undegraded":"2024-04-25T08:36:26.533707+0000","last_fullsized":"2024-04-25T08:36:26.533707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.552206+0000","last_change":"2024-04-25T08:36:26.551656+0000","last_active":"2024-04-25T08:36:26.552206+0000","last_peered":"2024-04-25T08:36:26.552206+0000","last_clean":"2024-04-25T08:36:26.552206+0000","last_became_active":"2024-04-25T08:36:26.551356+0000","last_became_peered":"2024-04-25T08:36:26.551356+0000","last_unstale":"2024-04-25T08:36:26.552206+0000","last_undegraded":"2024-04-25T08:36:26.552206+0000","last_fullsized":"2024-04-25T08:36:26.552206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:46.989455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533723+0000","last_change":"2024-04-25T08:36:26.533723+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533723+0000","last_undegraded":"2024-04-25T08:36:26.533723+0000","last_fullsized":"2024-04-25T08:36:26.533723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550893+0000","last_change":"2024-04-25T08:36:26.549899+0000","last_active":"2024-04-25T08:36:26.550893+0000","last_peered":"2024-04-25T08:36:26.550893+0000","last_clean":"2024-04-25T08:36:26.550893+0000","last_became_active":"2024-04-25T08:36:26.549599+0000","last_became_peered":"2024-04-25T08:36:26.549599+0000","last_unstale":"2024-04-25T08:36:26.550893+0000","last_undegraded":"2024-04-25T08:36:26.550893+0000","last_fullsized":"2024-04-25T08:36:26.550893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:42.921010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534938+0000","last_change":"2024-04-25T08:36:26.534938+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534938+0000","last_undegraded":"2024-04-25T08:36:26.534938+0000","last_fullsized":"2024-04-25T08:36:26.534938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551852+0000","last_change":"2024-04-25T08:36:26.551038+0000","last_active":"2024-04-25T08:36:26.551852+0000","last_peered":"2024-04-25T08:36:26.551852+0000","last_clean":"2024-04-25T08:36:26.551852+0000","last_became_active":"2024-04-25T08:36:26.550729+0000","last_became_peered":"2024-04-25T08:36:26.550729+0000","last_unstale":"2024-04-25T08:36:26.551852+0000","last_undegraded":"2024-04-25T08:36:26.551852+0000","last_fullsized":"2024-04-25T08:36:26.551852+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:26.538681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536406+0000","last_change":"2024-04-25T08:36:26.536406+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536406+0000","last_undegraded":"2024-04-25T08:36:26.536406+0000","last_fullsized":"2024-04-25T08:36:26.536406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550248+0000","last_change":"2024-04-25T08:36:26.548899+0000","last_active":"2024-04-25T08:36:26.550248+0000","last_peered":"2024-04-25T08:36:26.550248+0000","last_clean":"2024-04-25T08:36:26.550248+0000","last_became_active":"2024-04-25T08:36:26.548616+0000","last_became_peered":"2024-04-25T08:36:26.548616+0000","last_unstale":"2024-04-25T08:36:26.550248+0000","last_undegraded":"2024-04-25T08:36:26.550248+0000","last_fullsized":"2024-04-25T08:36:26.550248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:00.902526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535097+0000","last_change":"2024-04-25T08:36:26.535097+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535097+0000","last_undegraded":"2024-04-25T08:36:26.535097+0000","last_fullsized":"2024-04-25T08:36:26.535097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547719+0000","last_change":"2024-04-25T08:36:26.546019+0000","last_active":"2024-04-25T08:36:26.547719+0000","last_peered":"2024-04-25T08:36:26.547719+0000","last_clean":"2024-04-25T08:36:26.547719+0000","last_became_active":"2024-04-25T08:36:26.545702+0000","last_became_peered":"2024-04-25T08:36:26.545702+0000","last_unstale":"2024-04-25T08:36:26.547719+0000","last_undegraded":"2024-04-25T08:36:26.547719+0000","last_fullsized":"2024-04-25T08:36:26.547719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:04.231004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533653+0000","last_change":"2024-04-25T08:36:26.533653+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533653+0000","last_undegraded":"2024-04-25T08:36:26.533653+0000","last_fullsized":"2024-04-25T08:36:26.533653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.553551+0000","last_change":"2024-04-25T08:36:26.551919+0000","last_active":"2024-04-25T08:36:26.553551+0000","last_peered":"2024-04-25T08:36:26.553551+0000","last_clean":"2024-04-25T08:36:26.553551+0000","last_became_active":"2024-04-25T08:36:26.550730+0000","last_became_peered":"2024-04-25T08:36:26.550730+0000","last_unstale":"2024-04-25T08:36:26.553551+0000","last_undegraded":"2024-04-25T08:36:26.553551+0000","last_fullsized":"2024-04-25T08:36:26.553551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.423931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538357+0000","last_change":"2024-04-25T08:36:26.538357+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538357+0000","last_undegraded":"2024-04-25T08:36:26.538357+0000","last_fullsized":"2024-04-25T08:36:26.538357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549068+0000","last_change":"2024-04-25T08:36:26.546564+0000","last_active":"2024-04-25T08:36:26.549068+0000","last_peered":"2024-04-25T08:36:26.549068+0000","last_clean":"2024-04-25T08:36:26.549068+0000","last_became_active":"2024-04-25T08:36:26.546293+0000","last_became_peered":"2024-04-25T08:36:26.546293+0000","last_unstale":"2024-04-25T08:36:26.549068+0000","last_undegraded":"2024-04-25T08:36:26.549068+0000","last_fullsized":"2024-04-25T08:36:26.549068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:53.435350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.539109+0000","last_change":"2024-04-25T08:36:26.539109+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.539109+0000","last_undegraded":"2024-04-25T08:36:26.539109+0000","last_fullsized":"2024-04-25T08:36:26.539109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547656+0000","last_change":"2024-04-25T08:36:26.546007+0000","last_active":"2024-04-25T08:36:26.547656+0000","last_peered":"2024-04-25T08:36:26.547656+0000","last_clean":"2024-04-25T08:36:26.547656+0000","last_became_active":"2024-04-25T08:36:26.545660+0000","last_became_peered":"2024-04-25T08:36:26.545660+0000","last_unstale":"2024-04-25T08:36:26.547656+0000","last_undegraded":"2024-04-25T08:36:26.547656+0000","last_fullsized":"2024-04-25T08:36:26.547656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:48.548532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536701+0000","last_change":"2024-04-25T08:36:26.536701+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536701+0000","last_undegraded":"2024-04-25T08:36:26.536701+0000","last_fullsized":"2024-04-25T08:36:26.536701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.545751+0000","last_change":"2024-04-25T08:36:26.544427+0000","last_active":"2024-04-25T08:36:26.545751+0000","last_peered":"2024-04-25T08:36:26.545751+0000","last_clean":"2024-04-25T08:36:26.545751+0000","last_became_active":"2024-04-25T08:36:26.544096+0000","last_became_peered":"2024-04-25T08:36:26.544096+0000","last_unstale":"2024-04-25T08:36:26.545751+0000","last_undegraded":"2024-04-25T08:36:26.545751+0000","last_fullsized":"2024-04-25T08:36:26.545751+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:44.885589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":283,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.537575+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:36:26.537575+0000","last_peered":"2024-04-25T08:36:26.537575+0000","last_clean":"2024-04-25T08:36:26.537575+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:36:26.537575+0000","last_undegraded":"2024-04-25T08:36:26.537575+0000","last_fullsized":"2024-04-25T08:36:26.537575+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46608,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325232,"statfs":{"total":96636764160,"available":96589037568,"internally_reserved":0,"allocated":1572864,"data_stored":1040288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40456,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331384,"statfs":{"total":96636764160,"available":96595337216,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46024,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325816,"statfs":{"total":96636764160,"available":96589635584,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40456,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331384,"statfs":{"total":96636764160,"available":96595337216,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54912,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316928,"statfs":{"total":96636764160,"available":96580534272,"internally_reserved":0,"allocated":1560576,"data_stored":1033621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:36:31.901 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:36:32.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.042+0000 7f578745a640 1 Processor -- start 2024-04-25T08:36:32.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.042+0000 7f578745a640 1 -- start start 2024-04-25T08:36:32.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.043+0000 7f578745a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f578015ee20 0x7f57800cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.043+0000 7f578745a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5780099ba0 con 0x7f57800cd390 2024-04-25T08:36:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.043+0000 7f578745a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5780099e80 con 0x7f578009ac40 2024-04-25T08:36:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.043+0000 7f578745a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578009a160 con 0x7f578015ee20 2024-04-25T08:36:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.043+0000 7f57859f7640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f57800cd390 0x7f57800c51e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49344/0 (socket says 172.21.15.38:49344) 2024-04-25T08:36:32.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.043+0000 7f57859f7640 1 -- 172.21.15.38:0/1363570885 learned_addr learned my addr 172.21.15.38:0/1363570885 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:32.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.043+0000 7f57851f6640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f578015ee20 0x7f57800cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409952767 0 0) 0x7f5780099ba0 con 0x7f57800cd390 2024-04-25T08:36:32.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f576c003520 con 0x7f57800cd390 2024-04-25T08:36:32.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 330087058 0 0) 0x7f576c003520 con 0x7f57800cd390 2024-04-25T08:36:32.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f578015ee20 msgr2=0x7f57800cfc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:32.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f578015ee20 0x7f57800cfc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 >> v1:172.21.15.107:6789/0 conn(0x7f578009ac40 legacy=0x7f57800ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578009a440 con 0x7f57800cd390 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f57780019f0 con 0x7f57800cd390 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f57851f6640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f578015ee20 0x7f57800cfc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 >> v1:172.21.15.38:6789/0 conn(0x7f57800cd390 legacy=0x7f57800c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f576c0039e0 0x7f576c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f576c004390 0x7f576c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f578015ee20 0x7f576c005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578009a160 con 0x7f576c0039e0 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c00ba10 con 0x7f576c004390 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c00bc10 con 0x7f578015ee20 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f57851f6640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f576c0039e0 0x7f576c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f57859f7640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f578015ee20 0x7f576c005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f57849f5640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f576c004390 0x7f576c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f57851f6640 1 -- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f578015ee20 msgr2=0x7f576c005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f57851f6640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f578015ee20 0x7f576c005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f57851f6640 1 -- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f576c004390 msgr2=0x7f576c004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f57851f6640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f576c004390 0x7f576c004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f57851f6640 1 -- 172.21.15.38:0/1363570885 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c003520 con 0x7f576c0039e0 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.044+0000 7f57859f7640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f578015ee20 0x7f576c005120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.045+0000 7f57851f6640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f576c0039e0 0x7f576c003dc0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f577c009c90 tx=0x7f576c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99898f17c6339ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.045+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c036070 con 0x7f576c0039e0 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.045+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577c031400 con 0x7f576c0039e0 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.045+0000 7f5777fff640 1 -- 172.21.15.38:0/1363570885 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c009860 con 0x7f576c0039e0 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.046+0000 7f578745a640 1 -- 172.21.15.38:0/1363570885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f576c0039e0 msgr2=0x7f576c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.046+0000 7f578745a640 1 --2- 172.21.15.38:0/1363570885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f576c0039e0 0x7f576c003dc0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f577c009c90 tx=0x7f576c005830 comp rx=0 tx=0).stop 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.046+0000 7f57859f7640 1 -- 172.21.15.38:0/1363570885 reap_dead start 2024-04-25T08:36:32.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.046+0000 7f578745a640 1 -- 172.21.15.38:0/1363570885 shutdown_connections 2024-04-25T08:36:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.046+0000 7f578745a640 1 -- 172.21.15.38:0/1363570885 >> 172.21.15.38:0/1363570885 conn(0x7f57800b3180 msgr2=0x7f57800b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.046+0000 7f578745a640 1 -- 172.21.15.38:0/1363570885 shutdown_connections 2024-04-25T08:36:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.046+0000 7f578745a640 1 -- 172.21.15.38:0/1363570885 wait complete. 2024-04-25T08:36:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.047+0000 7f578745a640 1 Processor -- start 2024-04-25T08:36:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.047+0000 7f578745a640 1 -- start start 2024-04-25T08:36:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f578745a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f576c0039e0 0x7f57801ed860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f578745a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57801f4740 0x7f57800ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f578745a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57800cac60 0x7f578009eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f578745a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578015f380 con 0x7f57801f4740 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f578745a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f578015f200 con 0x7f57800cac60 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f578745a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57800c58f0 con 0x7f576c0039e0 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f57849f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57801f4740 0x7f57800ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f57849f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57801f4740 0x7f57800ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47736/0 (socket says 172.21.15.38:47736) 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f57849f5640 1 -- 172.21.15.38:0/4205579097 learned_addr learned my addr 172.21.15.38:0/4205579097 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f57851f6640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f576c0039e0 0x7f57801ed860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f57859f7640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57800cac60 0x7f578009eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f57849f5640 1 -- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f576c0039e0 msgr2=0x7f57801ed860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f57849f5640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f576c0039e0 0x7f57801ed860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f57849f5640 1 -- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57800cac60 msgr2=0x7f578009eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f57849f5640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57800cac60 0x7f578009eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f57849f5640 1 -- 172.21.15.38:0/4205579097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57801f3ca0 con 0x7f57801f4740 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f57849f5640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57801f4740 0x7f57800ca720 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f577000e830 tx=0x7f577000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.048+0000 7f5775ffb640 1 -- 172.21.15.38:0/4205579097 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57700050d0 con 0x7f57801f4740 2024-04-25T08:36:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.049+0000 7f5775ffb640 1 -- 172.21.15.38:0/4205579097 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5770018070 con 0x7f57801f4740 2024-04-25T08:36:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.049+0000 7f578745a640 1 -- 172.21.15.38:0/4205579097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57800cb480 con 0x7f57801f4740 2024-04-25T08:36:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.049+0000 7f5775ffb640 1 -- 172.21.15.38:0/4205579097 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577000c6c0 con 0x7f57801f4740 2024-04-25T08:36:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.049+0000 7f578745a640 1 -- 172.21.15.38:0/4205579097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57801f3f40 con 0x7f57801f4740 2024-04-25T08:36:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.050+0000 7f578745a640 1 -- 172.21.15.38:0/4205579097 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5780068e80 con 0x7f57801f4740 2024-04-25T08:36:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.050+0000 7f5775ffb640 1 -- 172.21.15.38:0/4205579097 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f577000ca70 con 0x7f57801f4740 2024-04-25T08:36:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.051+0000 7f5775ffb640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f576406b4d0 0x7f576406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.051+0000 7f5775ffb640 1 -- 172.21.15.38:0/4205579097 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f577008a360 con 0x7f57801f4740 2024-04-25T08:36:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.051+0000 7f57851f6640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f576406b4d0 0x7f576406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.051+0000 7f57851f6640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f576406b4d0 0x7f576406d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f577c033000 tx=0x7f577c008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.055+0000 7f5775ffb640 1 -- 172.21.15.38:0/4205579097 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5770058960 con 0x7f57801f4740 2024-04-25T08:36:32.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.232+0000 7f578745a640 1 -- 172.21.15.38:0/4205579097 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57801f3af0 con 0x7f576406b4d0 2024-04-25T08:36:32.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.238+0000 7f5775ffb640 1 -- 172.21.15.38:0/4205579097 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246753 (secure 0 0 0) 0x7f57801f3af0 con 0x7f576406b4d0 2024-04-25T08:36:32.235 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:32.236 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.241+0000 7f578745a640 1 -- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f576406b4d0 msgr2=0x7f576406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.241+0000 7f578745a640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f576406b4d0 0x7f576406d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f577c033000 tx=0x7f577c008030 comp rx=0 tx=0).stop 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.241+0000 7f578745a640 1 -- 172.21.15.38:0/4205579097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57801f4740 msgr2=0x7f57800ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.241+0000 7f578745a640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57801f4740 0x7f57800ca720 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f577000e830 tx=0x7f577000ecf0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.241+0000 7f578745a640 1 -- 172.21.15.38:0/4205579097 shutdown_connections 2024-04-25T08:36:32.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.241+0000 7f578745a640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f57800cac60 0x7f578009eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.241+0000 7f578745a640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57801f4740 0x7f57800ca720 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.241+0000 7f578745a640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f576406b4d0 0x7f576406d990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.241+0000 7f578745a640 1 --2- 172.21.15.38:0/4205579097 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f576c0039e0 0x7f57801ed860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.241+0000 7f578745a640 1 -- 172.21.15.38:0/4205579097 >> 172.21.15.38:0/4205579097 conn(0x7f57800b3180 msgr2=0x7f57800cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:32.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.241+0000 7f578745a640 1 -- 172.21.15.38:0/4205579097 shutdown_connections 2024-04-25T08:36:32.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.241+0000 7f578745a640 1 -- 172.21.15.38:0/4205579097 wait complete. 2024-04-25T08:36:32.246 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-04-25T08:36:31.273377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":337352,"kb_used_data":8796,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754637368,"statfs":{"total":773094113280,"available":772748664832,"internally_reserved":0,"allocated":9007104,"data_stored":4773429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":336318000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.742948"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537773+0000","last_change":"2024-04-25T08:36:26.537773+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537773+0000","last_undegraded":"2024-04-25T08:36:26.537773+0000","last_fullsized":"2024-04-25T08:36:26.537773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.544249+0000","last_change":"2024-04-25T08:36:26.544249+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.544249+0000","last_undegraded":"2024-04-25T08:36:26.544249+0000","last_fullsized":"2024-04-25T08:36:26.544249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.545202+0000","last_change":"2024-04-25T08:36:26.545202+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.545202+0000","last_undegraded":"2024-04-25T08:36:26.545202+0000","last_fullsized":"2024-04-25T08:36:26.545202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538453+0000","last_change":"2024-04-25T08:36:26.538453+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538453+0000","last_undegraded":"2024-04-25T08:36:26.538453+0000","last_fullsized":"2024-04-25T08:36:26.538453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538575+0000","last_change":"2024-04-25T08:36:26.538575+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538575+0000","last_undegraded":"2024-04-25T08:36:26.538575+0000","last_fullsized":"2024-04-25T08:36:26.538575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538753+0000","last_change":"2024-04-25T08:36:26.538753+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538753+0000","last_undegraded":"2024-04-25T08:36:26.538753+0000","last_fullsized":"2024-04-25T08:36:26.538753+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534187+0000","last_change":"2024-04-25T08:36:26.534187+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534187+0000","last_undegraded":"2024-04-25T08:36:26.534187+0000","last_fullsized":"2024-04-25T08:36:26.534187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537353+0000","last_change":"2024-04-25T08:36:26.537353+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537353+0000","last_undegraded":"2024-04-25T08:36:26.537353+0000","last_fullsized":"2024-04-25T08:36:26.537353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534390+0000","last_change":"2024-04-25T08:36:26.534390+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534390+0000","last_undegraded":"2024-04-25T08:36:26.534390+0000","last_fullsized":"2024-04-25T08:36:26.534390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538815+0000","last_change":"2024-04-25T08:36:26.538815+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538815+0000","last_undegraded":"2024-04-25T08:36:26.538815+0000","last_fullsized":"2024-04-25T08:36:26.538815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.543800+0000","last_change":"2024-04-25T08:36:26.543800+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.543800+0000","last_undegraded":"2024-04-25T08:36:26.543800+0000","last_fullsized":"2024-04-25T08:36:26.543800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537131+0000","last_change":"2024-04-25T08:36:26.537131+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537131+0000","last_undegraded":"2024-04-25T08:36:26.537131+0000","last_fullsized":"2024-04-25T08:36:26.537131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536911+0000","last_change":"2024-04-25T08:36:26.536911+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536911+0000","last_undegraded":"2024-04-25T08:36:26.536911+0000","last_fullsized":"2024-04-25T08:36:26.536911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538336+0000","last_change":"2024-04-25T08:36:26.538336+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538336+0000","last_undegraded":"2024-04-25T08:36:26.538336+0000","last_fullsized":"2024-04-25T08:36:26.538336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.544447+0000","last_change":"2024-04-25T08:36:26.544447+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.544447+0000","last_undegraded":"2024-04-25T08:36:26.544447+0000","last_fullsized":"2024-04-25T08:36:26.544447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537142+0000","last_change":"2024-04-25T08:36:26.537142+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537142+0000","last_undegraded":"2024-04-25T08:36:26.537142+0000","last_fullsized":"2024-04-25T08:36:26.537142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537724+0000","last_change":"2024-04-25T08:36:26.537724+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537724+0000","last_undegraded":"2024-04-25T08:36:26.537724+0000","last_fullsized":"2024-04-25T08:36:26.537724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.543802+0000","last_change":"2024-04-25T08:36:26.543802+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.543802+0000","last_undegraded":"2024-04-25T08:36:26.543802+0000","last_fullsized":"2024-04-25T08:36:26.543802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537441+0000","last_change":"2024-04-25T08:36:26.537441+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537441+0000","last_undegraded":"2024-04-25T08:36:26.537441+0000","last_fullsized":"2024-04-25T08:36:26.537441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.539548+0000","last_change":"2024-04-25T08:36:26.539548+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.539548+0000","last_undegraded":"2024-04-25T08:36:26.539548+0000","last_fullsized":"2024-04-25T08:36:26.539548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536226+0000","last_change":"2024-04-25T08:36:26.536226+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536226+0000","last_undegraded":"2024-04-25T08:36:26.536226+0000","last_fullsized":"2024-04-25T08:36:26.536226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538328+0000","last_change":"2024-04-25T08:36:26.538328+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538328+0000","last_undegraded":"2024-04-25T08:36:26.538328+0000","last_fullsized":"2024-04-25T08:36:26.538328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534103+0000","last_change":"2024-04-25T08:36:26.534103+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534103+0000","last_undegraded":"2024-04-25T08:36:26.534103+0000","last_fullsized":"2024-04-25T08:36:26.534103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.541048+0000","last_change":"2024-04-25T08:36:26.541048+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.541048+0000","last_undegraded":"2024-04-25T08:36:26.541048+0000","last_fullsized":"2024-04-25T08:36:26.541048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540285+0000","last_change":"2024-04-25T08:36:26.540285+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540285+0000","last_undegraded":"2024-04-25T08:36:26.540285+0000","last_fullsized":"2024-04-25T08:36:26.540285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537767+0000","last_change":"2024-04-25T08:36:26.537767+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537767+0000","last_undegraded":"2024-04-25T08:36:26.537767+0000","last_fullsized":"2024-04-25T08:36:26.537767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537155+0000","last_change":"2024-04-25T08:36:26.537155+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537155+0000","last_undegraded":"2024-04-25T08:36:26.537155+0000","last_fullsized":"2024-04-25T08:36:26.537155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538315+0000","last_change":"2024-04-25T08:36:26.538315+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538315+0000","last_undegraded":"2024-04-25T08:36:26.538315+0000","last_fullsized":"2024-04-25T08:36:26.538315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538511+0000","last_change":"2024-04-25T08:36:26.538511+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538511+0000","last_undegraded":"2024-04-25T08:36:26.538511+0000","last_fullsized":"2024-04-25T08:36:26.538511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536776+0000","last_change":"2024-04-25T08:36:26.536776+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536776+0000","last_undegraded":"2024-04-25T08:36:26.536776+0000","last_fullsized":"2024-04-25T08:36:26.536776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540684+0000","last_change":"2024-04-25T08:36:26.540684+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540684+0000","last_undegraded":"2024-04-25T08:36:26.540684+0000","last_fullsized":"2024-04-25T08:36:26.540684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536655+0000","last_change":"2024-04-25T08:36:26.536655+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536655+0000","last_undegraded":"2024-04-25T08:36:26.536655+0000","last_fullsized":"2024-04-25T08:36:26.536655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.539609+0000","last_change":"2024-04-25T08:36:26.539609+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.539609+0000","last_undegraded":"2024-04-25T08:36:26.539609+0000","last_fullsized":"2024-04-25T08:36:26.539609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.553534+0000","last_change":"2024-04-25T08:36:26.552070+0000","last_active":"2024-04-25T08:36:26.553534+0000","last_peered":"2024-04-25T08:36:26.553534+0000","last_clean":"2024-04-25T08:36:26.553534+0000","last_became_active":"2024-04-25T08:36:26.550647+0000","last_became_peered":"2024-04-25T08:36:26.550647+0000","last_unstale":"2024-04-25T08:36:26.553534+0000","last_undegraded":"2024-04-25T08:36:26.553534+0000","last_fullsized":"2024-04-25T08:36:26.553534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:18.633914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537186+0000","last_change":"2024-04-25T08:36:26.537186+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537186+0000","last_undegraded":"2024-04-25T08:36:26.537186+0000","last_fullsized":"2024-04-25T08:36:26.537186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551229+0000","last_change":"2024-04-25T08:36:26.549707+0000","last_active":"2024-04-25T08:36:26.551229+0000","last_peered":"2024-04-25T08:36:26.551229+0000","last_clean":"2024-04-25T08:36:26.551229+0000","last_became_active":"2024-04-25T08:36:26.549227+0000","last_became_peered":"2024-04-25T08:36:26.549227+0000","last_unstale":"2024-04-25T08:36:26.551229+0000","last_undegraded":"2024-04-25T08:36:26.551229+0000","last_fullsized":"2024-04-25T08:36:26.551229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:30.421525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535476+0000","last_change":"2024-04-25T08:36:26.535476+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535476+0000","last_undegraded":"2024-04-25T08:36:26.535476+0000","last_fullsized":"2024-04-25T08:36:26.535476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.552110+0000","last_change":"2024-04-25T08:36:26.551664+0000","last_active":"2024-04-25T08:36:26.552110+0000","last_peered":"2024-04-25T08:36:26.552110+0000","last_clean":"2024-04-25T08:36:26.552110+0000","last_became_active":"2024-04-25T08:36:26.551388+0000","last_became_peered":"2024-04-25T08:36:26.551388+0000","last_unstale":"2024-04-25T08:36:26.552110+0000","last_undegraded":"2024-04-25T08:36:26.552110+0000","last_fullsized":"2024-04-25T08:36:26.552110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:58.975343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535807+0000","last_change":"2024-04-25T08:36:26.535807+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535807+0000","last_undegraded":"2024-04-25T08:36:26.535807+0000","last_fullsized":"2024-04-25T08:36:26.535807+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547750+0000","last_change":"2024-04-25T08:36:26.545919+0000","last_active":"2024-04-25T08:36:26.547750+0000","last_peered":"2024-04-25T08:36:26.547750+0000","last_clean":"2024-04-25T08:36:26.547750+0000","last_became_active":"2024-04-25T08:36:26.545633+0000","last_became_peered":"2024-04-25T08:36:26.545633+0000","last_unstale":"2024-04-25T08:36:26.547750+0000","last_undegraded":"2024-04-25T08:36:26.547750+0000","last_fullsized":"2024-04-25T08:36:26.547750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:27.158628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538946+0000","last_change":"2024-04-25T08:36:26.538946+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538946+0000","last_undegraded":"2024-04-25T08:36:26.538946+0000","last_fullsized":"2024-04-25T08:36:26.538946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550474+0000","last_change":"2024-04-25T08:36:26.549171+0000","last_active":"2024-04-25T08:36:26.550474+0000","last_peered":"2024-04-25T08:36:26.550474+0000","last_clean":"2024-04-25T08:36:26.550474+0000","last_became_active":"2024-04-25T08:36:26.548875+0000","last_became_peered":"2024-04-25T08:36:26.548875+0000","last_unstale":"2024-04-25T08:36:26.550474+0000","last_undegraded":"2024-04-25T08:36:26.550474+0000","last_fullsized":"2024-04-25T08:36:26.550474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:04.789878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536149+0000","last_change":"2024-04-25T08:36:26.536149+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536149+0000","last_undegraded":"2024-04-25T08:36:26.536149+0000","last_fullsized":"2024-04-25T08:36:26.536149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550104+0000","last_change":"2024-04-25T08:36:26.544834+0000","last_active":"2024-04-25T08:36:26.550104+0000","last_peered":"2024-04-25T08:36:26.550104+0000","last_clean":"2024-04-25T08:36:26.550104+0000","last_became_active":"2024-04-25T08:36:26.544549+0000","last_became_peered":"2024-04-25T08:36:26.544549+0000","last_unstale":"2024-04-25T08:36:26.550104+0000","last_undegraded":"2024-04-25T08:36:26.550104+0000","last_fullsized":"2024-04-25T08:36:26.550104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:59.366292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533938+0000","last_change":"2024-04-25T08:36:26.533938+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533938+0000","last_undegraded":"2024-04-25T08:36:26.533938+0000","last_fullsized":"2024-04-25T08:36:26.533938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550148+0000","last_change":"2024-04-25T08:36:26.546765+0000","last_active":"2024-04-25T08:36:26.550148+0000","last_peered":"2024-04-25T08:36:26.550148+0000","last_clean":"2024-04-25T08:36:26.550148+0000","last_became_active":"2024-04-25T08:36:26.546467+0000","last_became_peered":"2024-04-25T08:36:26.546467+0000","last_unstale":"2024-04-25T08:36:26.550148+0000","last_undegraded":"2024-04-25T08:36:26.550148+0000","last_fullsized":"2024-04-25T08:36:26.550148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:44.885589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535281+0000","last_change":"2024-04-25T08:36:26.535281+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535281+0000","last_undegraded":"2024-04-25T08:36:26.535281+0000","last_fullsized":"2024-04-25T08:36:26.535281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.548729+0000","last_change":"2024-04-25T08:36:26.544689+0000","last_active":"2024-04-25T08:36:26.548729+0000","last_peered":"2024-04-25T08:36:26.548729+0000","last_clean":"2024-04-25T08:36:26.548729+0000","last_became_active":"2024-04-25T08:36:26.544353+0000","last_became_peered":"2024-04-25T08:36:26.544353+0000","last_unstale":"2024-04-25T08:36:26.548729+0000","last_undegraded":"2024-04-25T08:36:26.548729+0000","last_fullsized":"2024-04-25T08:36:26.548729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:19.783413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536997+0000","last_change":"2024-04-25T08:36:26.536997+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536997+0000","last_undegraded":"2024-04-25T08:36:26.536997+0000","last_fullsized":"2024-04-25T08:36:26.536997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551564+0000","last_change":"2024-04-25T08:36:26.549714+0000","last_active":"2024-04-25T08:36:26.551564+0000","last_peered":"2024-04-25T08:36:26.551564+0000","last_clean":"2024-04-25T08:36:26.551564+0000","last_became_active":"2024-04-25T08:36:26.548992+0000","last_became_peered":"2024-04-25T08:36:26.548992+0000","last_unstale":"2024-04-25T08:36:26.551564+0000","last_undegraded":"2024-04-25T08:36:26.551564+0000","last_fullsized":"2024-04-25T08:36:26.551564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:57.849522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536670+0000","last_change":"2024-04-25T08:36:26.536670+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536670+0000","last_undegraded":"2024-04-25T08:36:26.536670+0000","last_fullsized":"2024-04-25T08:36:26.536670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550140+0000","last_change":"2024-04-25T08:36:26.546855+0000","last_active":"2024-04-25T08:36:26.550140+0000","last_peered":"2024-04-25T08:36:26.550140+0000","last_clean":"2024-04-25T08:36:26.550140+0000","last_became_active":"2024-04-25T08:36:26.546607+0000","last_became_peered":"2024-04-25T08:36:26.546607+0000","last_unstale":"2024-04-25T08:36:26.550140+0000","last_undegraded":"2024-04-25T08:36:26.550140+0000","last_fullsized":"2024-04-25T08:36:26.550140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:26.454697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537812+0000","last_change":"2024-04-25T08:36:26.537812+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537812+0000","last_undegraded":"2024-04-25T08:36:26.537812+0000","last_fullsized":"2024-04-25T08:36:26.537812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549618+0000","last_change":"2024-04-25T08:36:26.548301+0000","last_active":"2024-04-25T08:36:26.549618+0000","last_peered":"2024-04-25T08:36:26.549618+0000","last_clean":"2024-04-25T08:36:26.549618+0000","last_became_active":"2024-04-25T08:36:26.547849+0000","last_became_peered":"2024-04-25T08:36:26.547849+0000","last_unstale":"2024-04-25T08:36:26.549618+0000","last_undegraded":"2024-04-25T08:36:26.549618+0000","last_fullsized":"2024-04-25T08:36:26.549618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.444339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540620+0000","last_change":"2024-04-25T08:36:26.540620+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540620+0000","last_undegraded":"2024-04-25T08:36:26.540620+0000","last_fullsized":"2024-04-25T08:36:26.540620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.546872+0000","last_change":"2024-04-25T08:36:26.545189+0000","last_active":"2024-04-25T08:36:26.546872+0000","last_peered":"2024-04-25T08:36:26.546872+0000","last_clean":"2024-04-25T08:36:26.546872+0000","last_became_active":"2024-04-25T08:36:26.544888+0000","last_became_peered":"2024-04-25T08:36:26.544888+0000","last_unstale":"2024-04-25T08:36:26.546872+0000","last_undegraded":"2024-04-25T08:36:26.546872+0000","last_fullsized":"2024-04-25T08:36:26.546872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:33.972987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536242+0000","last_change":"2024-04-25T08:36:26.536242+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536242+0000","last_undegraded":"2024-04-25T08:36:26.536242+0000","last_fullsized":"2024-04-25T08:36:26.536242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.548075+0000","last_change":"2024-04-25T08:36:26.544584+0000","last_active":"2024-04-25T08:36:26.548075+0000","last_peered":"2024-04-25T08:36:26.548075+0000","last_clean":"2024-04-25T08:36:26.548075+0000","last_became_active":"2024-04-25T08:36:26.544307+0000","last_became_peered":"2024-04-25T08:36:26.544307+0000","last_unstale":"2024-04-25T08:36:26.548075+0000","last_undegraded":"2024-04-25T08:36:26.548075+0000","last_fullsized":"2024-04-25T08:36:26.548075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:04.226062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535774+0000","last_change":"2024-04-25T08:36:26.535774+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535774+0000","last_undegraded":"2024-04-25T08:36:26.535774+0000","last_fullsized":"2024-04-25T08:36:26.535774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551293+0000","last_change":"2024-04-25T08:36:26.550747+0000","last_active":"2024-04-25T08:36:26.551293+0000","last_peered":"2024-04-25T08:36:26.551293+0000","last_clean":"2024-04-25T08:36:26.551293+0000","last_became_active":"2024-04-25T08:36:26.550435+0000","last_became_peered":"2024-04-25T08:36:26.550435+0000","last_unstale":"2024-04-25T08:36:26.551293+0000","last_undegraded":"2024-04-25T08:36:26.551293+0000","last_fullsized":"2024-04-25T08:36:26.551293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:36.698952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534041+0000","last_change":"2024-04-25T08:36:26.534041+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534041+0000","last_undegraded":"2024-04-25T08:36:26.534041+0000","last_fullsized":"2024-04-25T08:36:26.534041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.553950+0000","last_change":"2024-04-25T08:36:26.552811+0000","last_active":"2024-04-25T08:36:26.553950+0000","last_peered":"2024-04-25T08:36:26.553950+0000","last_clean":"2024-04-25T08:36:26.553950+0000","last_became_active":"2024-04-25T08:36:26.552472+0000","last_became_peered":"2024-04-25T08:36:26.552472+0000","last_unstale":"2024-04-25T08:36:26.553950+0000","last_undegraded":"2024-04-25T08:36:26.553950+0000","last_fullsized":"2024-04-25T08:36:26.553950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:14.669260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534879+0000","last_change":"2024-04-25T08:36:26.534879+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534879+0000","last_undegraded":"2024-04-25T08:36:26.534879+0000","last_fullsized":"2024-04-25T08:36:26.534879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550100+0000","last_change":"2024-04-25T08:36:26.545891+0000","last_active":"2024-04-25T08:36:26.550100+0000","last_peered":"2024-04-25T08:36:26.550100+0000","last_clean":"2024-04-25T08:36:26.550100+0000","last_became_active":"2024-04-25T08:36:26.545452+0000","last_became_peered":"2024-04-25T08:36:26.545452+0000","last_unstale":"2024-04-25T08:36:26.550100+0000","last_undegraded":"2024-04-25T08:36:26.550100+0000","last_fullsized":"2024-04-25T08:36:26.550100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.121065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536761+0000","last_change":"2024-04-25T08:36:26.536761+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536761+0000","last_undegraded":"2024-04-25T08:36:26.536761+0000","last_fullsized":"2024-04-25T08:36:26.536761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549858+0000","last_change":"2024-04-25T08:36:26.547438+0000","last_active":"2024-04-25T08:36:26.549858+0000","last_peered":"2024-04-25T08:36:26.549858+0000","last_clean":"2024-04-25T08:36:26.549858+0000","last_became_active":"2024-04-25T08:36:26.547171+0000","last_became_peered":"2024-04-25T08:36:26.547171+0000","last_unstale":"2024-04-25T08:36:26.549858+0000","last_undegraded":"2024-04-25T08:36:26.549858+0000","last_fullsized":"2024-04-25T08:36:26.549858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:37.899171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536841+0000","last_change":"2024-04-25T08:36:26.536841+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536841+0000","last_undegraded":"2024-04-25T08:36:26.536841+0000","last_fullsized":"2024-04-25T08:36:26.536841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.546253+0000","last_change":"2024-04-25T08:36:26.543814+0000","last_active":"2024-04-25T08:36:26.546253+0000","last_peered":"2024-04-25T08:36:26.546253+0000","last_clean":"2024-04-25T08:36:26.546253+0000","last_became_active":"2024-04-25T08:36:26.543502+0000","last_became_peered":"2024-04-25T08:36:26.543502+0000","last_unstale":"2024-04-25T08:36:26.546253+0000","last_undegraded":"2024-04-25T08:36:26.546253+0000","last_fullsized":"2024-04-25T08:36:26.546253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:00.280937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540283+0000","last_change":"2024-04-25T08:36:26.540283+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540283+0000","last_undegraded":"2024-04-25T08:36:26.540283+0000","last_fullsized":"2024-04-25T08:36:26.540283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549698+0000","last_change":"2024-04-25T08:36:26.546563+0000","last_active":"2024-04-25T08:36:26.549698+0000","last_peered":"2024-04-25T08:36:26.549698+0000","last_clean":"2024-04-25T08:36:26.549698+0000","last_became_active":"2024-04-25T08:36:26.544474+0000","last_became_peered":"2024-04-25T08:36:26.544474+0000","last_unstale":"2024-04-25T08:36:26.549698+0000","last_undegraded":"2024-04-25T08:36:26.549698+0000","last_fullsized":"2024-04-25T08:36:26.549698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:02.642404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538228+0000","last_change":"2024-04-25T08:36:26.538228+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538228+0000","last_undegraded":"2024-04-25T08:36:26.538228+0000","last_fullsized":"2024-04-25T08:36:26.538228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547548+0000","last_change":"2024-04-25T08:36:26.541992+0000","last_active":"2024-04-25T08:36:26.547548+0000","last_peered":"2024-04-25T08:36:26.547548+0000","last_clean":"2024-04-25T08:36:26.547548+0000","last_became_active":"2024-04-25T08:36:26.541695+0000","last_became_peered":"2024-04-25T08:36:26.541695+0000","last_unstale":"2024-04-25T08:36:26.547548+0000","last_undegraded":"2024-04-25T08:36:26.547548+0000","last_fullsized":"2024-04-25T08:36:26.547548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:53.435350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537636+0000","last_change":"2024-04-25T08:36:26.537636+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537636+0000","last_undegraded":"2024-04-25T08:36:26.537636+0000","last_fullsized":"2024-04-25T08:36:26.537636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551034+0000","last_change":"2024-04-25T08:36:26.548435+0000","last_active":"2024-04-25T08:36:26.551034+0000","last_peered":"2024-04-25T08:36:26.551034+0000","last_clean":"2024-04-25T08:36:26.551034+0000","last_became_active":"2024-04-25T08:36:26.548141+0000","last_became_peered":"2024-04-25T08:36:26.548141+0000","last_unstale":"2024-04-25T08:36:26.551034+0000","last_undegraded":"2024-04-25T08:36:26.551034+0000","last_fullsized":"2024-04-25T08:36:26.551034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:42.921010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536476+0000","last_change":"2024-04-25T08:36:26.536476+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536476+0000","last_undegraded":"2024-04-25T08:36:26.536476+0000","last_fullsized":"2024-04-25T08:36:26.536476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.545484+0000","last_change":"2024-04-25T08:36:26.544037+0000","last_active":"2024-04-25T08:36:26.545484+0000","last_peered":"2024-04-25T08:36:26.545484+0000","last_clean":"2024-04-25T08:36:26.545484+0000","last_became_active":"2024-04-25T08:36:26.543747+0000","last_became_peered":"2024-04-25T08:36:26.543747+0000","last_unstale":"2024-04-25T08:36:26.545484+0000","last_undegraded":"2024-04-25T08:36:26.545484+0000","last_fullsized":"2024-04-25T08:36:26.545484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.121065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534472+0000","last_change":"2024-04-25T08:36:26.534472+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534472+0000","last_undegraded":"2024-04-25T08:36:26.534472+0000","last_fullsized":"2024-04-25T08:36:26.534472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.545018+0000","last_change":"2024-04-25T08:36:26.543672+0000","last_active":"2024-04-25T08:36:26.545018+0000","last_peered":"2024-04-25T08:36:26.545018+0000","last_clean":"2024-04-25T08:36:26.545018+0000","last_became_active":"2024-04-25T08:36:26.543395+0000","last_became_peered":"2024-04-25T08:36:26.543395+0000","last_unstale":"2024-04-25T08:36:26.545018+0000","last_undegraded":"2024-04-25T08:36:26.545018+0000","last_fullsized":"2024-04-25T08:36:26.545018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:58.975343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533707+0000","last_change":"2024-04-25T08:36:26.533707+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533707+0000","last_undegraded":"2024-04-25T08:36:26.533707+0000","last_fullsized":"2024-04-25T08:36:26.533707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.552206+0000","last_change":"2024-04-25T08:36:26.551656+0000","last_active":"2024-04-25T08:36:26.552206+0000","last_peered":"2024-04-25T08:36:26.552206+0000","last_clean":"2024-04-25T08:36:26.552206+0000","last_became_active":"2024-04-25T08:36:26.551356+0000","last_became_peered":"2024-04-25T08:36:26.551356+0000","last_unstale":"2024-04-25T08:36:26.552206+0000","last_undegraded":"2024-04-25T08:36:26.552206+0000","last_fullsized":"2024-04-25T08:36:26.552206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:46.989455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533723+0000","last_change":"2024-04-25T08:36:26.533723+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533723+0000","last_undegraded":"2024-04-25T08:36:26.533723+0000","last_fullsized":"2024-04-25T08:36:26.533723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550893+0000","last_change":"2024-04-25T08:36:26.549899+0000","last_active":"2024-04-25T08:36:26.550893+0000","last_peered":"2024-04-25T08:36:26.550893+0000","last_clean":"2024-04-25T08:36:26.550893+0000","last_became_active":"2024-04-25T08:36:26.549599+0000","last_became_peered":"2024-04-25T08:36:26.549599+0000","last_unstale":"2024-04-25T08:36:26.550893+0000","last_undegraded":"2024-04-25T08:36:26.550893+0000","last_fullsized":"2024-04-25T08:36:26.550893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:42.921010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534938+0000","last_change":"2024-04-25T08:36:26.534938+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534938+0000","last_undegraded":"2024-04-25T08:36:26.534938+0000","last_fullsized":"2024-04-25T08:36:26.534938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551852+0000","last_change":"2024-04-25T08:36:26.551038+0000","last_active":"2024-04-25T08:36:26.551852+0000","last_peered":"2024-04-25T08:36:26.551852+0000","last_clean":"2024-04-25T08:36:26.551852+0000","last_became_active":"2024-04-25T08:36:26.550729+0000","last_became_peered":"2024-04-25T08:36:26.550729+0000","last_unstale":"2024-04-25T08:36:26.551852+0000","last_undegraded":"2024-04-25T08:36:26.551852+0000","last_fullsized":"2024-04-25T08:36:26.551852+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:26.538681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536406+0000","last_change":"2024-04-25T08:36:26.536406+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536406+0000","last_undegraded":"2024-04-25T08:36:26.536406+0000","last_fullsized":"2024-04-25T08:36:26.536406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550248+0000","last_change":"2024-04-25T08:36:26.548899+0000","last_active":"2024-04-25T08:36:26.550248+0000","last_peered":"2024-04-25T08:36:26.550248+0000","last_clean":"2024-04-25T08:36:26.550248+0000","last_became_active":"2024-04-25T08:36:26.548616+0000","last_became_peered":"2024-04-25T08:36:26.548616+0000","last_unstale":"2024-04-25T08:36:26.550248+0000","last_undegraded":"2024-04-25T08:36:26.550248+0000","last_fullsized":"2024-04-25T08:36:26.550248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:00.902526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535097+0000","last_change":"2024-04-25T08:36:26.535097+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535097+0000","last_undegraded":"2024-04-25T08:36:26.535097+0000","last_fullsized":"2024-04-25T08:36:26.535097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547719+0000","last_change":"2024-04-25T08:36:26.546019+0000","last_active":"2024-04-25T08:36:26.547719+0000","last_peered":"2024-04-25T08:36:26.547719+0000","last_clean":"2024-04-25T08:36:26.547719+0000","last_became_active":"2024-04-25T08:36:26.545702+0000","last_became_peered":"2024-04-25T08:36:26.545702+0000","last_unstale":"2024-04-25T08:36:26.547719+0000","last_undegraded":"2024-04-25T08:36:26.547719+0000","last_fullsized":"2024-04-25T08:36:26.547719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:04.231004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533653+0000","last_change":"2024-04-25T08:36:26.533653+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533653+0000","last_undegraded":"2024-04-25T08:36:26.533653+0000","last_fullsized":"2024-04-25T08:36:26.533653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.553551+0000","last_change":"2024-04-25T08:36:26.551919+0000","last_active":"2024-04-25T08:36:26.553551+0000","last_peered":"2024-04-25T08:36:26.553551+0000","last_clean":"2024-04-25T08:36:26.553551+0000","last_became_active":"2024-04-25T08:36:26.550730+0000","last_became_peered":"2024-04-25T08:36:26.550730+0000","last_unstale":"2024-04-25T08:36:26.553551+0000","last_undegraded":"2024-04-25T08:36:26.553551+0000","last_fullsized":"2024-04-25T08:36:26.553551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.423931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538357+0000","last_change":"2024-04-25T08:36:26.538357+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538357+0000","last_undegraded":"2024-04-25T08:36:26.538357+0000","last_fullsized":"2024-04-25T08:36:26.538357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549068+0000","last_change":"2024-04-25T08:36:26.546564+0000","last_active":"2024-04-25T08:36:26.549068+0000","last_peered":"2024-04-25T08:36:26.549068+0000","last_clean":"2024-04-25T08:36:26.549068+0000","last_became_active":"2024-04-25T08:36:26.546293+0000","last_became_peered":"2024-04-25T08:36:26.546293+0000","last_unstale":"2024-04-25T08:36:26.549068+0000","last_undegraded":"2024-04-25T08:36:26.549068+0000","last_fullsized":"2024-04-25T08:36:26.549068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:53.435350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.539109+0000","last_change":"2024-04-25T08:36:26.539109+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.539109+0000","last_undegraded":"2024-04-25T08:36:26.539109+0000","last_fullsized":"2024-04-25T08:36:26.539109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547656+0000","last_change":"2024-04-25T08:36:26.546007+0000","last_active":"2024-04-25T08:36:26.547656+0000","last_peered":"2024-04-25T08:36:26.547656+0000","last_clean":"2024-04-25T08:36:26.547656+0000","last_became_active":"2024-04-25T08:36:26.545660+0000","last_became_peered":"2024-04-25T08:36:26.545660+0000","last_unstale":"2024-04-25T08:36:26.547656+0000","last_undegraded":"2024-04-25T08:36:26.547656+0000","last_fullsized":"2024-04-25T08:36:26.547656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:48.548532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536701+0000","last_change":"2024-04-25T08:36:26.536701+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536701+0000","last_undegraded":"2024-04-25T08:36:26.536701+0000","last_fullsized":"2024-04-25T08:36:26.536701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.545751+0000","last_change":"2024-04-25T08:36:26.544427+0000","last_active":"2024-04-25T08:36:26.545751+0000","last_peered":"2024-04-25T08:36:26.545751+0000","last_clean":"2024-04-25T08:36:26.545751+0000","last_became_active":"2024-04-25T08:36:26.544096+0000","last_became_peered":"2024-04-25T08:36:26.544096+0000","last_unstale":"2024-04-25T08:36:26.545751+0000","last_undegraded":"2024-04-25T08:36:26.545751+0000","last_fullsized":"2024-04-25T08:36:26.545751+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:44.885589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":283,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.537575+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:36:26.537575+0000","last_peered":"2024-04-25T08:36:26.537575+0000","last_clean":"2024-04-25T08:36:26.537575+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:36:26.537575+0000","last_undegraded":"2024-04-25T08:36:26.537575+0000","last_fullsized":"2024-04-25T08:36:26.537575+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46608,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325232,"statfs":{"total":96636764160,"available":96589037568,"internally_reserved":0,"allocated":1572864,"data_stored":1040288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40456,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331384,"statfs":{"total":96636764160,"available":96595337216,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46024,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325816,"statfs":{"total":96636764160,"available":96589635584,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40456,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331384,"statfs":{"total":96636764160,"available":96595337216,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54912,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316928,"statfs":{"total":96636764160,"available":96580534272,"internally_reserved":0,"allocated":1560576,"data_stored":1033621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:36:32.250 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3f is not active+clean 2024-04-25T08:36:32.250 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.537773+0000', 'last_change': '2024-04-25T08:36:26.537773+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.537773+0000', 'last_undegraded': '2024-04-25T08:36:26.537773+0000', 'last_fullsized': '2024-04-25T08:36:26.537773+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:36:32.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3e is not active+clean 2024-04-25T08:36:32.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.544249+0000', 'last_change': '2024-04-25T08:36:26.544249+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.544249+0000', 'last_undegraded': '2024-04-25T08:36:26.544249+0000', 'last_fullsized': '2024-04-25T08:36:26.544249+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3d is not active+clean 2024-04-25T08:36:32.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.545202+0000', 'last_change': '2024-04-25T08:36:26.545202+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.545202+0000', 'last_undegraded': '2024-04-25T08:36:26.545202+0000', 'last_fullsized': '2024-04-25T08:36:26.545202+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:32.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3c is not active+clean 2024-04-25T08:36:32.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.538453+0000', 'last_change': '2024-04-25T08:36:26.538453+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.538453+0000', 'last_undegraded': '2024-04-25T08:36:26.538453+0000', 'last_fullsized': '2024-04-25T08:36:26.538453+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:36:32.251 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3b is not active+clean 2024-04-25T08:36:32.251 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.538575+0000', 'last_change': '2024-04-25T08:36:26.538575+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.538575+0000', 'last_undegraded': '2024-04-25T08:36:26.538575+0000', 'last_fullsized': '2024-04-25T08:36:26.538575+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:32.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3a is not active+clean 2024-04-25T08:36:32.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.538753+0000', 'last_change': '2024-04-25T08:36:26.538753+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.538753+0000', 'last_undegraded': '2024-04-25T08:36:26.538753+0000', 'last_fullsized': '2024-04-25T08:36:26.538753+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:36:32.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.39 is not active+clean 2024-04-25T08:36:32.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.534187+0000', 'last_change': '2024-04-25T08:36:26.534187+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.534187+0000', 'last_undegraded': '2024-04-25T08:36:26.534187+0000', 'last_fullsized': '2024-04-25T08:36:26.534187+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:36:32.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.38 is not active+clean 2024-04-25T08:36:32.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.537353+0000', 'last_change': '2024-04-25T08:36:26.537353+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.537353+0000', 'last_undegraded': '2024-04-25T08:36:26.537353+0000', 'last_fullsized': '2024-04-25T08:36:26.537353+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:32.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.37 is not active+clean 2024-04-25T08:36:32.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.534390+0000', 'last_change': '2024-04-25T08:36:26.534390+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.534390+0000', 'last_undegraded': '2024-04-25T08:36:26.534390+0000', 'last_fullsized': '2024-04-25T08:36:26.534390+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:32.252 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.36 is not active+clean 2024-04-25T08:36:32.252 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.538815+0000', 'last_change': '2024-04-25T08:36:26.538815+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.538815+0000', 'last_undegraded': '2024-04-25T08:36:26.538815+0000', 'last_fullsized': '2024-04-25T08:36:26.538815+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:36:32.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.35 is not active+clean 2024-04-25T08:36:32.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.543800+0000', 'last_change': '2024-04-25T08:36:26.543800+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.543800+0000', 'last_undegraded': '2024-04-25T08:36:26.543800+0000', 'last_fullsized': '2024-04-25T08:36:26.543800+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.34 is not active+clean 2024-04-25T08:36:32.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.537131+0000', 'last_change': '2024-04-25T08:36:26.537131+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.537131+0000', 'last_undegraded': '2024-04-25T08:36:26.537131+0000', 'last_fullsized': '2024-04-25T08:36:26.537131+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:36:32.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.33 is not active+clean 2024-04-25T08:36:32.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536911+0000', 'last_change': '2024-04-25T08:36:26.536911+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536911+0000', 'last_undegraded': '2024-04-25T08:36:26.536911+0000', 'last_fullsized': '2024-04-25T08:36:26.536911+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:32.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.32 is not active+clean 2024-04-25T08:36:32.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.538336+0000', 'last_change': '2024-04-25T08:36:26.538336+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.538336+0000', 'last_undegraded': '2024-04-25T08:36:26.538336+0000', 'last_fullsized': '2024-04-25T08:36:26.538336+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:36:32.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.31 is not active+clean 2024-04-25T08:36:32.253 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.544447+0000', 'last_change': '2024-04-25T08:36:26.544447+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.544447+0000', 'last_undegraded': '2024-04-25T08:36:26.544447+0000', 'last_fullsized': '2024-04-25T08:36:26.544447+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.30 is not active+clean 2024-04-25T08:36:32.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.537142+0000', 'last_change': '2024-04-25T08:36:26.537142+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.537142+0000', 'last_undegraded': '2024-04-25T08:36:26.537142+0000', 'last_fullsized': '2024-04-25T08:36:26.537142+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:36:32.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2f is not active+clean 2024-04-25T08:36:32.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.537724+0000', 'last_change': '2024-04-25T08:36:26.537724+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.537724+0000', 'last_undegraded': '2024-04-25T08:36:26.537724+0000', 'last_fullsized': '2024-04-25T08:36:26.537724+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:32.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2e is not active+clean 2024-04-25T08:36:32.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.543802+0000', 'last_change': '2024-04-25T08:36:26.543802+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.543802+0000', 'last_undegraded': '2024-04-25T08:36:26.543802+0000', 'last_fullsized': '2024-04-25T08:36:26.543802+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:32.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2d is not active+clean 2024-04-25T08:36:32.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.537441+0000', 'last_change': '2024-04-25T08:36:26.537441+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.537441+0000', 'last_undegraded': '2024-04-25T08:36:26.537441+0000', 'last_fullsized': '2024-04-25T08:36:26.537441+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:36:32.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2c is not active+clean 2024-04-25T08:36:32.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.539548+0000', 'last_change': '2024-04-25T08:36:26.539548+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.539548+0000', 'last_undegraded': '2024-04-25T08:36:26.539548+0000', 'last_fullsized': '2024-04-25T08:36:26.539548+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:32.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2b is not active+clean 2024-04-25T08:36:32.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536226+0000', 'last_change': '2024-04-25T08:36:26.536226+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536226+0000', 'last_undegraded': '2024-04-25T08:36:26.536226+0000', 'last_fullsized': '2024-04-25T08:36:26.536226+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2a is not active+clean 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.538328+0000', 'last_change': '2024-04-25T08:36:26.538328+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.538328+0000', 'last_undegraded': '2024-04-25T08:36:26.538328+0000', 'last_fullsized': '2024-04-25T08:36:26.538328+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.29 is not active+clean 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.534103+0000', 'last_change': '2024-04-25T08:36:26.534103+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.534103+0000', 'last_undegraded': '2024-04-25T08:36:26.534103+0000', 'last_fullsized': '2024-04-25T08:36:26.534103+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.28 is not active+clean 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.541048+0000', 'last_change': '2024-04-25T08:36:26.541048+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.541048+0000', 'last_undegraded': '2024-04-25T08:36:26.541048+0000', 'last_fullsized': '2024-04-25T08:36:26.541048+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.27 is not active+clean 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.540285+0000', 'last_change': '2024-04-25T08:36:26.540285+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.540285+0000', 'last_undegraded': '2024-04-25T08:36:26.540285+0000', 'last_fullsized': '2024-04-25T08:36:26.540285+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.26 is not active+clean 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.537767+0000', 'last_change': '2024-04-25T08:36:26.537767+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.537767+0000', 'last_undegraded': '2024-04-25T08:36:26.537767+0000', 'last_fullsized': '2024-04-25T08:36:26.537767+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.25 is not active+clean 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.537155+0000', 'last_change': '2024-04-25T08:36:26.537155+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.537155+0000', 'last_undegraded': '2024-04-25T08:36:26.537155+0000', 'last_fullsized': '2024-04-25T08:36:26.537155+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:36:32.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.24 is not active+clean 2024-04-25T08:36:32.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.538315+0000', 'last_change': '2024-04-25T08:36:26.538315+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.538315+0000', 'last_undegraded': '2024-04-25T08:36:26.538315+0000', 'last_fullsized': '2024-04-25T08:36:26.538315+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:32.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.23 is not active+clean 2024-04-25T08:36:32.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.538511+0000', 'last_change': '2024-04-25T08:36:26.538511+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.538511+0000', 'last_undegraded': '2024-04-25T08:36:26.538511+0000', 'last_fullsized': '2024-04-25T08:36:26.538511+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.22 is not active+clean 2024-04-25T08:36:32.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536776+0000', 'last_change': '2024-04-25T08:36:26.536776+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536776+0000', 'last_undegraded': '2024-04-25T08:36:26.536776+0000', 'last_fullsized': '2024-04-25T08:36:26.536776+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:36:32.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.21 is not active+clean 2024-04-25T08:36:32.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.540684+0000', 'last_change': '2024-04-25T08:36:26.540684+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.540684+0000', 'last_undegraded': '2024-04-25T08:36:26.540684+0000', 'last_fullsized': '2024-04-25T08:36:26.540684+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:32.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.20 is not active+clean 2024-04-25T08:36:32.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536655+0000', 'last_change': '2024-04-25T08:36:26.536655+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536655+0000', 'last_undegraded': '2024-04-25T08:36:26.536655+0000', 'last_fullsized': '2024-04-25T08:36:26.536655+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:36:32.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.18 is not active+clean 2024-04-25T08:36:32.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.539609+0000', 'last_change': '2024-04-25T08:36:26.539609+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.539609+0000', 'last_undegraded': '2024-04-25T08:36:26.539609+0000', 'last_fullsized': '2024-04-25T08:36:26.539609+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:32.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.19 is not active+clean 2024-04-25T08:36:32.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.537186+0000', 'last_change': '2024-04-25T08:36:26.537186+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.537186+0000', 'last_undegraded': '2024-04-25T08:36:26.537186+0000', 'last_fullsized': '2024-04-25T08:36:26.537186+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1a is not active+clean 2024-04-25T08:36:32.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.535476+0000', 'last_change': '2024-04-25T08:36:26.535476+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.535476+0000', 'last_undegraded': '2024-04-25T08:36:26.535476+0000', 'last_fullsized': '2024-04-25T08:36:26.535476+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:32.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1b is not active+clean 2024-04-25T08:36:32.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.535807+0000', 'last_change': '2024-04-25T08:36:26.535807+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.535807+0000', 'last_undegraded': '2024-04-25T08:36:26.535807+0000', 'last_fullsized': '2024-04-25T08:36:26.535807+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:32.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1c is not active+clean 2024-04-25T08:36:32.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.538946+0000', 'last_change': '2024-04-25T08:36:26.538946+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.538946+0000', 'last_undegraded': '2024-04-25T08:36:26.538946+0000', 'last_fullsized': '2024-04-25T08:36:26.538946+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1d is not active+clean 2024-04-25T08:36:32.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536149+0000', 'last_change': '2024-04-25T08:36:26.536149+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536149+0000', 'last_undegraded': '2024-04-25T08:36:26.536149+0000', 'last_fullsized': '2024-04-25T08:36:26.536149+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1e is not active+clean 2024-04-25T08:36:32.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.533938+0000', 'last_change': '2024-04-25T08:36:26.533938+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.533938+0000', 'last_undegraded': '2024-04-25T08:36:26.533938+0000', 'last_fullsized': '2024-04-25T08:36:26.533938+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:32.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1f is not active+clean 2024-04-25T08:36:32.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.535281+0000', 'last_change': '2024-04-25T08:36:26.535281+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.535281+0000', 'last_undegraded': '2024-04-25T08:36:26.535281+0000', 'last_fullsized': '2024-04-25T08:36:26.535281+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:36:32.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.10 is not active+clean 2024-04-25T08:36:32.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536997+0000', 'last_change': '2024-04-25T08:36:26.536997+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536997+0000', 'last_undegraded': '2024-04-25T08:36:26.536997+0000', 'last_fullsized': '2024-04-25T08:36:26.536997+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.11 is not active+clean 2024-04-25T08:36:32.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536670+0000', 'last_change': '2024-04-25T08:36:26.536670+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536670+0000', 'last_undegraded': '2024-04-25T08:36:26.536670+0000', 'last_fullsized': '2024-04-25T08:36:26.536670+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:36:32.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.12 is not active+clean 2024-04-25T08:36:32.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.537812+0000', 'last_change': '2024-04-25T08:36:26.537812+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.537812+0000', 'last_undegraded': '2024-04-25T08:36:26.537812+0000', 'last_fullsized': '2024-04-25T08:36:26.537812+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:36:32.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.13 is not active+clean 2024-04-25T08:36:32.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.540620+0000', 'last_change': '2024-04-25T08:36:26.540620+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.540620+0000', 'last_undegraded': '2024-04-25T08:36:26.540620+0000', 'last_fullsized': '2024-04-25T08:36:26.540620+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.14 is not active+clean 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536242+0000', 'last_change': '2024-04-25T08:36:26.536242+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536242+0000', 'last_undegraded': '2024-04-25T08:36:26.536242+0000', 'last_fullsized': '2024-04-25T08:36:26.536242+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.15 is not active+clean 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.535774+0000', 'last_change': '2024-04-25T08:36:26.535774+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.535774+0000', 'last_undegraded': '2024-04-25T08:36:26.535774+0000', 'last_fullsized': '2024-04-25T08:36:26.535774+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.16 is not active+clean 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.534041+0000', 'last_change': '2024-04-25T08:36:26.534041+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.534041+0000', 'last_undegraded': '2024-04-25T08:36:26.534041+0000', 'last_fullsized': '2024-04-25T08:36:26.534041+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.17 is not active+clean 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.534879+0000', 'last_change': '2024-04-25T08:36:26.534879+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.534879+0000', 'last_undegraded': '2024-04-25T08:36:26.534879+0000', 'last_fullsized': '2024-04-25T08:36:26.534879+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.8 is not active+clean 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536761+0000', 'last_change': '2024-04-25T08:36:26.536761+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536761+0000', 'last_undegraded': '2024-04-25T08:36:26.536761+0000', 'last_fullsized': '2024-04-25T08:36:26.536761+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.9 is not active+clean 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536841+0000', 'last_change': '2024-04-25T08:36:26.536841+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536841+0000', 'last_undegraded': '2024-04-25T08:36:26.536841+0000', 'last_fullsized': '2024-04-25T08:36:26.536841+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:32.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.a is not active+clean 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.540283+0000', 'last_change': '2024-04-25T08:36:26.540283+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.540283+0000', 'last_undegraded': '2024-04-25T08:36:26.540283+0000', 'last_fullsized': '2024-04-25T08:36:26.540283+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.b is not active+clean 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.538228+0000', 'last_change': '2024-04-25T08:36:26.538228+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.538228+0000', 'last_undegraded': '2024-04-25T08:36:26.538228+0000', 'last_fullsized': '2024-04-25T08:36:26.538228+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.c is not active+clean 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.537636+0000', 'last_change': '2024-04-25T08:36:26.537636+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.537636+0000', 'last_undegraded': '2024-04-25T08:36:26.537636+0000', 'last_fullsized': '2024-04-25T08:36:26.537636+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.e is not active+clean 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536476+0000', 'last_change': '2024-04-25T08:36:26.536476+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536476+0000', 'last_undegraded': '2024-04-25T08:36:26.536476+0000', 'last_fullsized': '2024-04-25T08:36:26.536476+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.f is not active+clean 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.534472+0000', 'last_change': '2024-04-25T08:36:26.534472+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.534472+0000', 'last_undegraded': '2024-04-25T08:36:26.534472+0000', 'last_fullsized': '2024-04-25T08:36:26.534472+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.0 is not active+clean 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.533707+0000', 'last_change': '2024-04-25T08:36:26.533707+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.533707+0000', 'last_undegraded': '2024-04-25T08:36:26.533707+0000', 'last_fullsized': '2024-04-25T08:36:26.533707+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.1 is not active+clean 2024-04-25T08:36:32.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.533723+0000', 'last_change': '2024-04-25T08:36:26.533723+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.533723+0000', 'last_undegraded': '2024-04-25T08:36:26.533723+0000', 'last_fullsized': '2024-04-25T08:36:26.533723+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:36:32.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.2 is not active+clean 2024-04-25T08:36:32.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.534938+0000', 'last_change': '2024-04-25T08:36:26.534938+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.534938+0000', 'last_undegraded': '2024-04-25T08:36:26.534938+0000', 'last_fullsized': '2024-04-25T08:36:26.534938+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:36:32.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.3 is not active+clean 2024-04-25T08:36:32.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536406+0000', 'last_change': '2024-04-25T08:36:26.536406+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536406+0000', 'last_undegraded': '2024-04-25T08:36:26.536406+0000', 'last_fullsized': '2024-04-25T08:36:26.536406+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:36:32.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.4 is not active+clean 2024-04-25T08:36:32.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.535097+0000', 'last_change': '2024-04-25T08:36:26.535097+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.535097+0000', 'last_undegraded': '2024-04-25T08:36:26.535097+0000', 'last_fullsized': '2024-04-25T08:36:26.535097+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:36:32.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.5 is not active+clean 2024-04-25T08:36:32.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.533653+0000', 'last_change': '2024-04-25T08:36:26.533653+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.533653+0000', 'last_undegraded': '2024-04-25T08:36:26.533653+0000', 'last_fullsized': '2024-04-25T08:36:26.533653+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:36:32.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.6 is not active+clean 2024-04-25T08:36:32.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.538357+0000', 'last_change': '2024-04-25T08:36:26.538357+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.538357+0000', 'last_undegraded': '2024-04-25T08:36:26.538357+0000', 'last_fullsized': '2024-04-25T08:36:26.538357+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:32.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.7 is not active+clean 2024-04-25T08:36:32.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.539109+0000', 'last_change': '2024-04-25T08:36:26.539109+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.539109+0000', 'last_undegraded': '2024-04-25T08:36:26.539109+0000', 'last_fullsized': '2024-04-25T08:36:26.539109+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:36:32.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 12.d is not active+clean 2024-04-25T08:36:32.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '12.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 80, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:36:26.536701+0000', 'last_change': '2024-04-25T08:36:26.536701+0000', 'last_active': '2024-04-25T08:36:26.526708+0000', 'last_peered': '2024-04-25T08:36:26.526708+0000', 'last_clean': '2024-04-25T08:36:26.526708+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:36:26.536701+0000', 'last_undegraded': '2024-04-25T08:36:26.536701+0000', 'last_fullsized': '2024-04-25T08:36:26.536701+0000', 'mapping_epoch': 80, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 80, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'last_clean_scrub_stamp': '2024-04-25T08:36:26.526708+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:36:32.262 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:36:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.393+0000 7f246ad16640 1 Processor -- start 2024-04-25T08:36:32.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.393+0000 7f246ad16640 1 -- start start 2024-04-25T08:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.393+0000 7f246ad16640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f246415ddf0 0x7f2464161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.393+0000 7f246ad16640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f24640b93f0 0x7f24640d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.393+0000 7f246ad16640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2464161e00 con 0x7f24640b5710 2024-04-25T08:36:32.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.393+0000 7f246ad16640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24641620e0 con 0x7f24640b93f0 2024-04-25T08:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.393+0000 7f246ad16640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24641623c0 con 0x7f246415ddf0 2024-04-25T08:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.394+0000 7f24692b3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f24640b93f0 0x7f24640d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.394+0000 7f245bfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f246415ddf0 0x7f2464161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.394+0000 7f2468ab2640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f24640b5710 0x7f24640b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:52796/0 (socket says 172.21.15.38:52796) 2024-04-25T08:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.394+0000 7f24692b3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f24640b93f0 0x7f24640d2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:54064/0 (socket says 172.21.15.38:54064) 2024-04-25T08:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.394+0000 7f2468ab2640 1 -- 172.21.15.38:0/2604626900 learned_addr learned my addr 172.21.15.38:0/2604626900 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.394+0000 7f245bfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f246415ddf0 0x7f2464161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:45852/0 (socket says 172.21.15.38:45852) 2024-04-25T08:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.394+0000 7f24692b3640 1 -- 172.21.15.38:0/2604626900 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f246415ddf0 msgr2=0x7f2464161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.394+0000 7f24692b3640 1 --2- 172.21.15.38:0/2604626900 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f246415ddf0 0x7f2464161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.394+0000 7f24692b3640 1 -- 172.21.15.38:0/2604626900 >> v1:172.21.15.107:6789/0 conn(0x7f24640b5710 legacy=0x7f24640b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.394+0000 7f24692b3640 1 -- 172.21.15.38:0/2604626900 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24640b9870 con 0x7f24640b93f0 2024-04-25T08:36:32.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.394+0000 7f245b7fe640 1 -- 172.21.15.38:0/2604626900 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2000577127 0 0) 0x7f2464161e00 con 0x7f24640b5710 2024-04-25T08:36:32.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.395+0000 7f24692b3640 1 --2- 172.21.15.38:0/2604626900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24640b93f0 0x7f24640d2340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f245c009d90 tx=0x7f245c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6213048514d10a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.395+0000 7f245b7fe640 1 -- 172.21.15.38:0/2604626900 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c007740 con 0x7f24640b93f0 2024-04-25T08:36:32.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.395+0000 7f245b7fe640 1 -- 172.21.15.38:0/2604626900 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245c0078e0 con 0x7f24640b93f0 2024-04-25T08:36:32.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.395+0000 7f245b7fe640 1 -- 172.21.15.38:0/2604626900 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c007cb0 con 0x7f24640b93f0 2024-04-25T08:36:32.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.396+0000 7f246ad16640 1 -- 172.21.15.38:0/2604626900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24640b93f0 msgr2=0x7f24640d2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.396+0000 7f246ad16640 1 --2- 172.21.15.38:0/2604626900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24640b93f0 0x7f24640d2340 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f245c009d90 tx=0x7f245c02a660 comp rx=0 tx=0).stop 2024-04-25T08:36:32.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.396+0000 7f246ad16640 1 -- 172.21.15.38:0/2604626900 shutdown_connections 2024-04-25T08:36:32.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.396+0000 7f246ad16640 1 --2- 172.21.15.38:0/2604626900 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24640b93f0 0x7f24640d2340 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.396+0000 7f246ad16640 1 --2- 172.21.15.38:0/2604626900 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f246415ddf0 0x7f2464161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.396+0000 7f246ad16640 1 -- 172.21.15.38:0/2604626900 >> 172.21.15.38:0/2604626900 conn(0x7f24640d07c0 msgr2=0x7f24640d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:32.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.396+0000 7f246ad16640 1 -- 172.21.15.38:0/2604626900 shutdown_connections 2024-04-25T08:36:32.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.396+0000 7f246ad16640 1 -- 172.21.15.38:0/2604626900 wait complete. 2024-04-25T08:36:32.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.397+0000 7f246ad16640 1 Processor -- start 2024-04-25T08:36:32.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.397+0000 7f246ad16640 1 -- start start 2024-04-25T08:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f246ad16640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24640b5710 0x7f24640c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f246ad16640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24640b93f0 0x7f246414cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f246ad16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f246415ddf0 0x7f24640cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f246ad16640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24640d2c90 con 0x7f246415ddf0 2024-04-25T08:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f246ad16640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24640d2b10 con 0x7f24640b93f0 2024-04-25T08:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f246ad16640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24640d2e10 con 0x7f24640b5710 2024-04-25T08:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f24692b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f246415ddf0 0x7f24640cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f24692b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f246415ddf0 0x7f24640cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47740/0 (socket says 172.21.15.38:47740) 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f24692b3640 1 -- 172.21.15.38:0/3662607721 learned_addr learned my addr 172.21.15.38:0/3662607721 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f245bfff640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24640b93f0 0x7f246414cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f2468ab2640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24640b5710 0x7f24640c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f24692b3640 1 -- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24640b5710 msgr2=0x7f24640c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f24692b3640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24640b5710 0x7f24640c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f24692b3640 1 -- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24640b93f0 msgr2=0x7f246414cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f24692b3640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24640b93f0 0x7f246414cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f24692b3640 1 -- 172.21.15.38:0/3662607721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24640cd390 con 0x7f246415ddf0 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.398+0000 7f24692b3640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f246415ddf0 0x7f24640cc3a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f245c02ac50 tx=0x7f245c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.399+0000 7f24597fa640 1 -- 172.21.15.38:0/3662607721 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c040070 con 0x7f246415ddf0 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.399+0000 7f24597fa640 1 -- 172.21.15.38:0/3662607721 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245c031c40 con 0x7f246415ddf0 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.399+0000 7f246ad16640 1 -- 172.21.15.38:0/3662607721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24640cd670 con 0x7f246415ddf0 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.399+0000 7f24597fa640 1 -- 172.21.15.38:0/3662607721 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c009450 con 0x7f246415ddf0 2024-04-25T08:36:32.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.399+0000 7f246ad16640 1 -- 172.21.15.38:0/3662607721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24640cdc30 con 0x7f246415ddf0 2024-04-25T08:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.400+0000 7f246ad16640 1 -- 172.21.15.38:0/3662607721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2464068e80 con 0x7f246415ddf0 2024-04-25T08:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.400+0000 7f24597fa640 1 -- 172.21.15.38:0/3662607721 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f245c039040 con 0x7f246415ddf0 2024-04-25T08:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.400+0000 7f24597fa640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f245006b4d0 0x7f245006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.400+0000 7f24597fa640 1 -- 172.21.15.38:0/3662607721 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f245c0ab1f0 con 0x7f246415ddf0 2024-04-25T08:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.401+0000 7f2468ab2640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f245006b4d0 0x7f245006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.401+0000 7f2468ab2640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f245006b4d0 0x7f245006d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2460000f60 tx=0x7f2460007b50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.405+0000 7f24597fa640 1 -- 172.21.15.38:0/3662607721 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f245c079ee0 con 0x7f246415ddf0 2024-04-25T08:36:32.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.654+0000 7f246ad16640 1 -- 172.21.15.38:0/3662607721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f24640b2e70 con 0x7f246415ddf0 2024-04-25T08:36:32.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.654+0000 7f24597fa640 1 -- 172.21.15.38:0/3662607721 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1583 (secure 0 0 0) 0x7f245c0797e0 con 0x7f246415ddf0 2024-04-25T08:36:32.648 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:32.648 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":321,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":82,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":345448448,"bytes_avail":772748664832,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":50,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":7383}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T08:36:15.270042+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:36:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.656+0000 7f246ad16640 1 -- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f245006b4d0 msgr2=0x7f245006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.656+0000 7f246ad16640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f245006b4d0 0x7f245006d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2460000f60 tx=0x7f2460007b50 comp rx=0 tx=0).stop 2024-04-25T08:36:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.656+0000 7f246ad16640 1 -- 172.21.15.38:0/3662607721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f246415ddf0 msgr2=0x7f24640cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.656+0000 7f246ad16640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f246415ddf0 0x7f24640cc3a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f245c02ac50 tx=0x7f245c004a40 comp rx=0 tx=0).stop 2024-04-25T08:36:32.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.656+0000 7f246ad16640 1 -- 172.21.15.38:0/3662607721 shutdown_connections 2024-04-25T08:36:32.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.656+0000 7f246ad16640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f246415ddf0 0x7f24640cc3a0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.656+0000 7f246ad16640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24640b93f0 0x7f246414cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.656+0000 7f246ad16640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f245006b4d0 0x7f245006d990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.656+0000 7f246ad16640 1 --2- 172.21.15.38:0/3662607721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f24640b5710 0x7f24640c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.656+0000 7f246ad16640 1 -- 172.21.15.38:0/3662607721 >> 172.21.15.38:0/3662607721 conn(0x7f24640d07c0 msgr2=0x7f24640b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:32.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.656+0000 7f246ad16640 1 -- 172.21.15.38:0/3662607721 shutdown_connections 2024-04-25T08:36:32.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.656+0000 7f246ad16640 1 -- 172.21.15.38:0/3662607721 wait complete. 2024-04-25T08:36:32.661 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:36:32.661 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:36:32.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.767+0000 7f9637e9a640 1 Processor -- start 2024-04-25T08:36:32.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.768+0000 7f9637e9a640 1 -- start start 2024-04-25T08:36:32.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.768+0000 7f9637e9a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f96300b5710 0x7f9630161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.768+0000 7f9637e9a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96300b98a0 con 0x7f963015ddf0 2024-04-25T08:36:32.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.768+0000 7f9637e9a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9630161e00 con 0x7f96300b93f0 2024-04-25T08:36:32.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.768+0000 7f9637e9a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96301620e0 con 0x7f96300b5710 2024-04-25T08:36:32.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.769+0000 7f9635c36640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f96300b5710 0x7f9630161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.769+0000 7f9635435640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f963015ddf0 0x7f96300b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49358/0 (socket says 172.21.15.38:49358) 2024-04-25T08:36:32.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.769+0000 7f9635c36640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f96300b5710 0x7f9630161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47752/0 (socket says 172.21.15.38:47752) 2024-04-25T08:36:32.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.769+0000 7f9635435640 1 -- 172.21.15.38:0/1480489920 learned_addr learned my addr 172.21.15.38:0/1480489920 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:32.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.769+0000 7f9635c36640 1 -- 172.21.15.38:0/1480489920 >> v1:172.21.15.107:6789/0 conn(0x7f96300b93f0 legacy=0x7f96300d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.769+0000 7f9635c36640 1 -- 172.21.15.38:0/1480489920 >> v1:172.21.15.38:6789/0 conn(0x7f963015ddf0 legacy=0x7f96300b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.769+0000 7f9635c36640 1 -- 172.21.15.38:0/1480489920 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96301623c0 con 0x7f96300b5710 2024-04-25T08:36:32.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.770+0000 7f9635c36640 1 --2- 172.21.15.38:0/1480489920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96300b5710 0x7f9630161700 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f962c0097a0 tx=0x7f962c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8e932340848264 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.770+0000 7f9634c34640 1 -- 172.21.15.38:0/1480489920 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c037070 con 0x7f96300b5710 2024-04-25T08:36:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.770+0000 7f9634c34640 1 -- 172.21.15.38:0/1480489920 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962c032440 con 0x7f96300b5710 2024-04-25T08:36:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.770+0000 7f9634c34640 1 -- 172.21.15.38:0/1480489920 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c032810 con 0x7f96300b5710 2024-04-25T08:36:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.770+0000 7f9637e9a640 1 -- 172.21.15.38:0/1480489920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96300b5710 msgr2=0x7f9630161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.770+0000 7f9637e9a640 1 --2- 172.21.15.38:0/1480489920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96300b5710 0x7f9630161700 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f962c0097a0 tx=0x7f962c02a230 comp rx=0 tx=0).stop 2024-04-25T08:36:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.770+0000 7f9637e9a640 1 -- 172.21.15.38:0/1480489920 shutdown_connections 2024-04-25T08:36:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.770+0000 7f9637e9a640 1 --2- 172.21.15.38:0/1480489920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96300b5710 0x7f9630161700 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.770+0000 7f9637e9a640 1 -- 172.21.15.38:0/1480489920 >> 172.21.15.38:0/1480489920 conn(0x7f96300d07c0 msgr2=0x7f96300d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.770+0000 7f9637e9a640 1 -- 172.21.15.38:0/1480489920 shutdown_connections 2024-04-25T08:36:32.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.770+0000 7f9637e9a640 1 -- 172.21.15.38:0/1480489920 wait complete. 2024-04-25T08:36:32.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.772+0000 7f9637e9a640 1 Processor -- start 2024-04-25T08:36:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.772+0000 7f9637e9a640 1 -- start start 2024-04-25T08:36:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.772+0000 7f9637e9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96300b5710 0x7f96300cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.772+0000 7f9637e9a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96300b93f0 0x7f96301e73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.772+0000 7f9637e9a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f963015ddf0 0x7f96301e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.772+0000 7f9637e9a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96300d2de0 con 0x7f96300b5710 2024-04-25T08:36:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.772+0000 7f9637e9a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96300d2c60 con 0x7f96300b93f0 2024-04-25T08:36:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.772+0000 7f9637e9a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96300d2f60 con 0x7f963015ddf0 2024-04-25T08:36:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.772+0000 7f9635c36640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96300b5710 0x7f96300cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.772+0000 7f9635c36640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96300b5710 0x7f96300cb050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47762/0 (socket says 172.21.15.38:47762) 2024-04-25T08:36:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f9635c36640 1 -- 172.21.15.38:0/1018910211 learned_addr learned my addr 172.21.15.38:0/1018910211 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:32.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f9635435640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96300b93f0 0x7f96301e73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f9636437640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f963015ddf0 0x7f96301e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f9635c36640 1 -- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f963015ddf0 msgr2=0x7f96301e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f9635c36640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f963015ddf0 0x7f96301e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f9635c36640 1 -- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96300b93f0 msgr2=0x7f96301e73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f9635c36640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96300b93f0 0x7f96301e73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f9635c36640 1 -- 172.21.15.38:0/1018910211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96300cb590 con 0x7f96300b5710 2024-04-25T08:36:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f9635c36640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96300b5710 0x7f96300cb050 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f962c035040 tx=0x7f962c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f96267fc640 1 -- 172.21.15.38:0/1018910211 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c037040 con 0x7f96300b5710 2024-04-25T08:36:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f96267fc640 1 -- 172.21.15.38:0/1018910211 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962c030770 con 0x7f96300b5710 2024-04-25T08:36:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f9637e9a640 1 -- 172.21.15.38:0/1018910211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9630149c90 con 0x7f96300b5710 2024-04-25T08:36:32.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f96267fc640 1 -- 172.21.15.38:0/1018910211 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962c030b80 con 0x7f96300b5710 2024-04-25T08:36:32.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.773+0000 7f9637e9a640 1 -- 172.21.15.38:0/1018910211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963014a230 con 0x7f96300b5710 2024-04-25T08:36:32.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.774+0000 7f9637e9a640 1 -- 172.21.15.38:0/1018910211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96300cfe00 con 0x7f96300b5710 2024-04-25T08:36:32.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.775+0000 7f96267fc640 1 -- 172.21.15.38:0/1018910211 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f962c044020 con 0x7f96300b5710 2024-04-25T08:36:32.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.775+0000 7f96267fc640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f961406b4d0 0x7f961406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:32.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.775+0000 7f96267fc640 1 -- 172.21.15.38:0/1018910211 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f962c0aa950 con 0x7f96300b5710 2024-04-25T08:36:32.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.775+0000 7f9635435640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f961406b4d0 0x7f961406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:32.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.776+0000 7f9635435640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f961406b4d0 0x7f961406d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f963014cc60 tx=0x7f9620009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:32.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.780+0000 7f96267fc640 1 -- 172.21.15.38:0/1018910211 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96300cfe00 con 0x7f96300b5710 2024-04-25T08:36:32.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.958+0000 7f9637e9a640 1 -- 172.21.15.38:0/1018910211 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9630068e80 con 0x7f961406b4d0 2024-04-25T08:36:32.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.965+0000 7f96267fc640 1 -- 172.21.15.38:0/1018910211 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246753 (secure 0 0 0) 0x7f9630068e80 con 0x7f961406b4d0 2024-04-25T08:36:32.962 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:32.963 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:36:32.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.967+0000 7f9637e9a640 1 -- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f961406b4d0 msgr2=0x7f961406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.967+0000 7f9637e9a640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f961406b4d0 0x7f961406d990 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f963014cc60 tx=0x7f9620009040 comp rx=0 tx=0).stop 2024-04-25T08:36:32.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.967+0000 7f9637e9a640 1 -- 172.21.15.38:0/1018910211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96300b5710 msgr2=0x7f96300cb050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:32.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.967+0000 7f9637e9a640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96300b5710 0x7f96300cb050 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f962c035040 tx=0x7f962c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.968+0000 7f9637e9a640 1 -- 172.21.15.38:0/1018910211 shutdown_connections 2024-04-25T08:36:32.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.968+0000 7f9637e9a640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f961406b4d0 0x7f961406d990 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.968+0000 7f9637e9a640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f963015ddf0 0x7f96301e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.968+0000 7f9637e9a640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96300b93f0 0x7f96301e73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.968+0000 7f9637e9a640 1 --2- 172.21.15.38:0/1018910211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96300b5710 0x7f96300cb050 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:32.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.968+0000 7f9637e9a640 1 -- 172.21.15.38:0/1018910211 >> 172.21.15.38:0/1018910211 conn(0x7f96300d07c0 msgr2=0x7f96300d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:32.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.968+0000 7f9637e9a640 1 -- 172.21.15.38:0/1018910211 shutdown_connections 2024-04-25T08:36:32.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:32.968+0000 7f9637e9a640 1 -- 172.21.15.38:0/1018910211 wait complete. 2024-04-25T08:36:32.973 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-04-25T08:36:31.273377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":337352,"kb_used_data":8796,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754637368,"statfs":{"total":773094113280,"available":772748664832,"internally_reserved":0,"allocated":9007104,"data_stored":4773429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":336318000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.742948"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537773+0000","last_change":"2024-04-25T08:36:26.537773+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537773+0000","last_undegraded":"2024-04-25T08:36:26.537773+0000","last_fullsized":"2024-04-25T08:36:26.537773+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.544249+0000","last_change":"2024-04-25T08:36:26.544249+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.544249+0000","last_undegraded":"2024-04-25T08:36:26.544249+0000","last_fullsized":"2024-04-25T08:36:26.544249+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.545202+0000","last_change":"2024-04-25T08:36:26.545202+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.545202+0000","last_undegraded":"2024-04-25T08:36:26.545202+0000","last_fullsized":"2024-04-25T08:36:26.545202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538453+0000","last_change":"2024-04-25T08:36:26.538453+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538453+0000","last_undegraded":"2024-04-25T08:36:26.538453+0000","last_fullsized":"2024-04-25T08:36:26.538453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538575+0000","last_change":"2024-04-25T08:36:26.538575+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538575+0000","last_undegraded":"2024-04-25T08:36:26.538575+0000","last_fullsized":"2024-04-25T08:36:26.538575+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538753+0000","last_change":"2024-04-25T08:36:26.538753+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538753+0000","last_undegraded":"2024-04-25T08:36:26.538753+0000","last_fullsized":"2024-04-25T08:36:26.538753+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534187+0000","last_change":"2024-04-25T08:36:26.534187+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534187+0000","last_undegraded":"2024-04-25T08:36:26.534187+0000","last_fullsized":"2024-04-25T08:36:26.534187+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537353+0000","last_change":"2024-04-25T08:36:26.537353+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537353+0000","last_undegraded":"2024-04-25T08:36:26.537353+0000","last_fullsized":"2024-04-25T08:36:26.537353+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534390+0000","last_change":"2024-04-25T08:36:26.534390+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534390+0000","last_undegraded":"2024-04-25T08:36:26.534390+0000","last_fullsized":"2024-04-25T08:36:26.534390+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538815+0000","last_change":"2024-04-25T08:36:26.538815+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538815+0000","last_undegraded":"2024-04-25T08:36:26.538815+0000","last_fullsized":"2024-04-25T08:36:26.538815+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.543800+0000","last_change":"2024-04-25T08:36:26.543800+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.543800+0000","last_undegraded":"2024-04-25T08:36:26.543800+0000","last_fullsized":"2024-04-25T08:36:26.543800+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537131+0000","last_change":"2024-04-25T08:36:26.537131+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537131+0000","last_undegraded":"2024-04-25T08:36:26.537131+0000","last_fullsized":"2024-04-25T08:36:26.537131+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536911+0000","last_change":"2024-04-25T08:36:26.536911+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536911+0000","last_undegraded":"2024-04-25T08:36:26.536911+0000","last_fullsized":"2024-04-25T08:36:26.536911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538336+0000","last_change":"2024-04-25T08:36:26.538336+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538336+0000","last_undegraded":"2024-04-25T08:36:26.538336+0000","last_fullsized":"2024-04-25T08:36:26.538336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.544447+0000","last_change":"2024-04-25T08:36:26.544447+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.544447+0000","last_undegraded":"2024-04-25T08:36:26.544447+0000","last_fullsized":"2024-04-25T08:36:26.544447+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537142+0000","last_change":"2024-04-25T08:36:26.537142+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537142+0000","last_undegraded":"2024-04-25T08:36:26.537142+0000","last_fullsized":"2024-04-25T08:36:26.537142+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537724+0000","last_change":"2024-04-25T08:36:26.537724+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537724+0000","last_undegraded":"2024-04-25T08:36:26.537724+0000","last_fullsized":"2024-04-25T08:36:26.537724+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.543802+0000","last_change":"2024-04-25T08:36:26.543802+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.543802+0000","last_undegraded":"2024-04-25T08:36:26.543802+0000","last_fullsized":"2024-04-25T08:36:26.543802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537441+0000","last_change":"2024-04-25T08:36:26.537441+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537441+0000","last_undegraded":"2024-04-25T08:36:26.537441+0000","last_fullsized":"2024-04-25T08:36:26.537441+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.539548+0000","last_change":"2024-04-25T08:36:26.539548+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.539548+0000","last_undegraded":"2024-04-25T08:36:26.539548+0000","last_fullsized":"2024-04-25T08:36:26.539548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536226+0000","last_change":"2024-04-25T08:36:26.536226+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536226+0000","last_undegraded":"2024-04-25T08:36:26.536226+0000","last_fullsized":"2024-04-25T08:36:26.536226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538328+0000","last_change":"2024-04-25T08:36:26.538328+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538328+0000","last_undegraded":"2024-04-25T08:36:26.538328+0000","last_fullsized":"2024-04-25T08:36:26.538328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534103+0000","last_change":"2024-04-25T08:36:26.534103+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534103+0000","last_undegraded":"2024-04-25T08:36:26.534103+0000","last_fullsized":"2024-04-25T08:36:26.534103+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.541048+0000","last_change":"2024-04-25T08:36:26.541048+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.541048+0000","last_undegraded":"2024-04-25T08:36:26.541048+0000","last_fullsized":"2024-04-25T08:36:26.541048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540285+0000","last_change":"2024-04-25T08:36:26.540285+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540285+0000","last_undegraded":"2024-04-25T08:36:26.540285+0000","last_fullsized":"2024-04-25T08:36:26.540285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537767+0000","last_change":"2024-04-25T08:36:26.537767+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537767+0000","last_undegraded":"2024-04-25T08:36:26.537767+0000","last_fullsized":"2024-04-25T08:36:26.537767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537155+0000","last_change":"2024-04-25T08:36:26.537155+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537155+0000","last_undegraded":"2024-04-25T08:36:26.537155+0000","last_fullsized":"2024-04-25T08:36:26.537155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538315+0000","last_change":"2024-04-25T08:36:26.538315+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538315+0000","last_undegraded":"2024-04-25T08:36:26.538315+0000","last_fullsized":"2024-04-25T08:36:26.538315+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538511+0000","last_change":"2024-04-25T08:36:26.538511+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538511+0000","last_undegraded":"2024-04-25T08:36:26.538511+0000","last_fullsized":"2024-04-25T08:36:26.538511+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536776+0000","last_change":"2024-04-25T08:36:26.536776+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536776+0000","last_undegraded":"2024-04-25T08:36:26.536776+0000","last_fullsized":"2024-04-25T08:36:26.536776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540684+0000","last_change":"2024-04-25T08:36:26.540684+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540684+0000","last_undegraded":"2024-04-25T08:36:26.540684+0000","last_fullsized":"2024-04-25T08:36:26.540684+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536655+0000","last_change":"2024-04-25T08:36:26.536655+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536655+0000","last_undegraded":"2024-04-25T08:36:26.536655+0000","last_fullsized":"2024-04-25T08:36:26.536655+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.539609+0000","last_change":"2024-04-25T08:36:26.539609+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.539609+0000","last_undegraded":"2024-04-25T08:36:26.539609+0000","last_fullsized":"2024-04-25T08:36:26.539609+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.553534+0000","last_change":"2024-04-25T08:36:26.552070+0000","last_active":"2024-04-25T08:36:26.553534+0000","last_peered":"2024-04-25T08:36:26.553534+0000","last_clean":"2024-04-25T08:36:26.553534+0000","last_became_active":"2024-04-25T08:36:26.550647+0000","last_became_peered":"2024-04-25T08:36:26.550647+0000","last_unstale":"2024-04-25T08:36:26.553534+0000","last_undegraded":"2024-04-25T08:36:26.553534+0000","last_fullsized":"2024-04-25T08:36:26.553534+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:18.633914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537186+0000","last_change":"2024-04-25T08:36:26.537186+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537186+0000","last_undegraded":"2024-04-25T08:36:26.537186+0000","last_fullsized":"2024-04-25T08:36:26.537186+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551229+0000","last_change":"2024-04-25T08:36:26.549707+0000","last_active":"2024-04-25T08:36:26.551229+0000","last_peered":"2024-04-25T08:36:26.551229+0000","last_clean":"2024-04-25T08:36:26.551229+0000","last_became_active":"2024-04-25T08:36:26.549227+0000","last_became_peered":"2024-04-25T08:36:26.549227+0000","last_unstale":"2024-04-25T08:36:26.551229+0000","last_undegraded":"2024-04-25T08:36:26.551229+0000","last_fullsized":"2024-04-25T08:36:26.551229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:30.421525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535476+0000","last_change":"2024-04-25T08:36:26.535476+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535476+0000","last_undegraded":"2024-04-25T08:36:26.535476+0000","last_fullsized":"2024-04-25T08:36:26.535476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.552110+0000","last_change":"2024-04-25T08:36:26.551664+0000","last_active":"2024-04-25T08:36:26.552110+0000","last_peered":"2024-04-25T08:36:26.552110+0000","last_clean":"2024-04-25T08:36:26.552110+0000","last_became_active":"2024-04-25T08:36:26.551388+0000","last_became_peered":"2024-04-25T08:36:26.551388+0000","last_unstale":"2024-04-25T08:36:26.552110+0000","last_undegraded":"2024-04-25T08:36:26.552110+0000","last_fullsized":"2024-04-25T08:36:26.552110+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:58.975343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535807+0000","last_change":"2024-04-25T08:36:26.535807+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535807+0000","last_undegraded":"2024-04-25T08:36:26.535807+0000","last_fullsized":"2024-04-25T08:36:26.535807+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547750+0000","last_change":"2024-04-25T08:36:26.545919+0000","last_active":"2024-04-25T08:36:26.547750+0000","last_peered":"2024-04-25T08:36:26.547750+0000","last_clean":"2024-04-25T08:36:26.547750+0000","last_became_active":"2024-04-25T08:36:26.545633+0000","last_became_peered":"2024-04-25T08:36:26.545633+0000","last_unstale":"2024-04-25T08:36:26.547750+0000","last_undegraded":"2024-04-25T08:36:26.547750+0000","last_fullsized":"2024-04-25T08:36:26.547750+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:27.158628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538946+0000","last_change":"2024-04-25T08:36:26.538946+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538946+0000","last_undegraded":"2024-04-25T08:36:26.538946+0000","last_fullsized":"2024-04-25T08:36:26.538946+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550474+0000","last_change":"2024-04-25T08:36:26.549171+0000","last_active":"2024-04-25T08:36:26.550474+0000","last_peered":"2024-04-25T08:36:26.550474+0000","last_clean":"2024-04-25T08:36:26.550474+0000","last_became_active":"2024-04-25T08:36:26.548875+0000","last_became_peered":"2024-04-25T08:36:26.548875+0000","last_unstale":"2024-04-25T08:36:26.550474+0000","last_undegraded":"2024-04-25T08:36:26.550474+0000","last_fullsized":"2024-04-25T08:36:26.550474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:04.789878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536149+0000","last_change":"2024-04-25T08:36:26.536149+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536149+0000","last_undegraded":"2024-04-25T08:36:26.536149+0000","last_fullsized":"2024-04-25T08:36:26.536149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550104+0000","last_change":"2024-04-25T08:36:26.544834+0000","last_active":"2024-04-25T08:36:26.550104+0000","last_peered":"2024-04-25T08:36:26.550104+0000","last_clean":"2024-04-25T08:36:26.550104+0000","last_became_active":"2024-04-25T08:36:26.544549+0000","last_became_peered":"2024-04-25T08:36:26.544549+0000","last_unstale":"2024-04-25T08:36:26.550104+0000","last_undegraded":"2024-04-25T08:36:26.550104+0000","last_fullsized":"2024-04-25T08:36:26.550104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:59.366292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533938+0000","last_change":"2024-04-25T08:36:26.533938+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533938+0000","last_undegraded":"2024-04-25T08:36:26.533938+0000","last_fullsized":"2024-04-25T08:36:26.533938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550148+0000","last_change":"2024-04-25T08:36:26.546765+0000","last_active":"2024-04-25T08:36:26.550148+0000","last_peered":"2024-04-25T08:36:26.550148+0000","last_clean":"2024-04-25T08:36:26.550148+0000","last_became_active":"2024-04-25T08:36:26.546467+0000","last_became_peered":"2024-04-25T08:36:26.546467+0000","last_unstale":"2024-04-25T08:36:26.550148+0000","last_undegraded":"2024-04-25T08:36:26.550148+0000","last_fullsized":"2024-04-25T08:36:26.550148+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:44.885589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535281+0000","last_change":"2024-04-25T08:36:26.535281+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535281+0000","last_undegraded":"2024-04-25T08:36:26.535281+0000","last_fullsized":"2024-04-25T08:36:26.535281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.548729+0000","last_change":"2024-04-25T08:36:26.544689+0000","last_active":"2024-04-25T08:36:26.548729+0000","last_peered":"2024-04-25T08:36:26.548729+0000","last_clean":"2024-04-25T08:36:26.548729+0000","last_became_active":"2024-04-25T08:36:26.544353+0000","last_became_peered":"2024-04-25T08:36:26.544353+0000","last_unstale":"2024-04-25T08:36:26.548729+0000","last_undegraded":"2024-04-25T08:36:26.548729+0000","last_fullsized":"2024-04-25T08:36:26.548729+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:19.783413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536997+0000","last_change":"2024-04-25T08:36:26.536997+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536997+0000","last_undegraded":"2024-04-25T08:36:26.536997+0000","last_fullsized":"2024-04-25T08:36:26.536997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551564+0000","last_change":"2024-04-25T08:36:26.549714+0000","last_active":"2024-04-25T08:36:26.551564+0000","last_peered":"2024-04-25T08:36:26.551564+0000","last_clean":"2024-04-25T08:36:26.551564+0000","last_became_active":"2024-04-25T08:36:26.548992+0000","last_became_peered":"2024-04-25T08:36:26.548992+0000","last_unstale":"2024-04-25T08:36:26.551564+0000","last_undegraded":"2024-04-25T08:36:26.551564+0000","last_fullsized":"2024-04-25T08:36:26.551564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:57.849522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536670+0000","last_change":"2024-04-25T08:36:26.536670+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536670+0000","last_undegraded":"2024-04-25T08:36:26.536670+0000","last_fullsized":"2024-04-25T08:36:26.536670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550140+0000","last_change":"2024-04-25T08:36:26.546855+0000","last_active":"2024-04-25T08:36:26.550140+0000","last_peered":"2024-04-25T08:36:26.550140+0000","last_clean":"2024-04-25T08:36:26.550140+0000","last_became_active":"2024-04-25T08:36:26.546607+0000","last_became_peered":"2024-04-25T08:36:26.546607+0000","last_unstale":"2024-04-25T08:36:26.550140+0000","last_undegraded":"2024-04-25T08:36:26.550140+0000","last_fullsized":"2024-04-25T08:36:26.550140+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:26.454697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537812+0000","last_change":"2024-04-25T08:36:26.537812+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537812+0000","last_undegraded":"2024-04-25T08:36:26.537812+0000","last_fullsized":"2024-04-25T08:36:26.537812+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549618+0000","last_change":"2024-04-25T08:36:26.548301+0000","last_active":"2024-04-25T08:36:26.549618+0000","last_peered":"2024-04-25T08:36:26.549618+0000","last_clean":"2024-04-25T08:36:26.549618+0000","last_became_active":"2024-04-25T08:36:26.547849+0000","last_became_peered":"2024-04-25T08:36:26.547849+0000","last_unstale":"2024-04-25T08:36:26.549618+0000","last_undegraded":"2024-04-25T08:36:26.549618+0000","last_fullsized":"2024-04-25T08:36:26.549618+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.444339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540620+0000","last_change":"2024-04-25T08:36:26.540620+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540620+0000","last_undegraded":"2024-04-25T08:36:26.540620+0000","last_fullsized":"2024-04-25T08:36:26.540620+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.546872+0000","last_change":"2024-04-25T08:36:26.545189+0000","last_active":"2024-04-25T08:36:26.546872+0000","last_peered":"2024-04-25T08:36:26.546872+0000","last_clean":"2024-04-25T08:36:26.546872+0000","last_became_active":"2024-04-25T08:36:26.544888+0000","last_became_peered":"2024-04-25T08:36:26.544888+0000","last_unstale":"2024-04-25T08:36:26.546872+0000","last_undegraded":"2024-04-25T08:36:26.546872+0000","last_fullsized":"2024-04-25T08:36:26.546872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:33.972987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536242+0000","last_change":"2024-04-25T08:36:26.536242+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536242+0000","last_undegraded":"2024-04-25T08:36:26.536242+0000","last_fullsized":"2024-04-25T08:36:26.536242+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.548075+0000","last_change":"2024-04-25T08:36:26.544584+0000","last_active":"2024-04-25T08:36:26.548075+0000","last_peered":"2024-04-25T08:36:26.548075+0000","last_clean":"2024-04-25T08:36:26.548075+0000","last_became_active":"2024-04-25T08:36:26.544307+0000","last_became_peered":"2024-04-25T08:36:26.544307+0000","last_unstale":"2024-04-25T08:36:26.548075+0000","last_undegraded":"2024-04-25T08:36:26.548075+0000","last_fullsized":"2024-04-25T08:36:26.548075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:04.226062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535774+0000","last_change":"2024-04-25T08:36:26.535774+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535774+0000","last_undegraded":"2024-04-25T08:36:26.535774+0000","last_fullsized":"2024-04-25T08:36:26.535774+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551293+0000","last_change":"2024-04-25T08:36:26.550747+0000","last_active":"2024-04-25T08:36:26.551293+0000","last_peered":"2024-04-25T08:36:26.551293+0000","last_clean":"2024-04-25T08:36:26.551293+0000","last_became_active":"2024-04-25T08:36:26.550435+0000","last_became_peered":"2024-04-25T08:36:26.550435+0000","last_unstale":"2024-04-25T08:36:26.551293+0000","last_undegraded":"2024-04-25T08:36:26.551293+0000","last_fullsized":"2024-04-25T08:36:26.551293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:36.698952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534041+0000","last_change":"2024-04-25T08:36:26.534041+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534041+0000","last_undegraded":"2024-04-25T08:36:26.534041+0000","last_fullsized":"2024-04-25T08:36:26.534041+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.553950+0000","last_change":"2024-04-25T08:36:26.552811+0000","last_active":"2024-04-25T08:36:26.553950+0000","last_peered":"2024-04-25T08:36:26.553950+0000","last_clean":"2024-04-25T08:36:26.553950+0000","last_became_active":"2024-04-25T08:36:26.552472+0000","last_became_peered":"2024-04-25T08:36:26.552472+0000","last_unstale":"2024-04-25T08:36:26.553950+0000","last_undegraded":"2024-04-25T08:36:26.553950+0000","last_fullsized":"2024-04-25T08:36:26.553950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:14.669260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534879+0000","last_change":"2024-04-25T08:36:26.534879+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534879+0000","last_undegraded":"2024-04-25T08:36:26.534879+0000","last_fullsized":"2024-04-25T08:36:26.534879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550100+0000","last_change":"2024-04-25T08:36:26.545891+0000","last_active":"2024-04-25T08:36:26.550100+0000","last_peered":"2024-04-25T08:36:26.550100+0000","last_clean":"2024-04-25T08:36:26.550100+0000","last_became_active":"2024-04-25T08:36:26.545452+0000","last_became_peered":"2024-04-25T08:36:26.545452+0000","last_unstale":"2024-04-25T08:36:26.550100+0000","last_undegraded":"2024-04-25T08:36:26.550100+0000","last_fullsized":"2024-04-25T08:36:26.550100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.121065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536761+0000","last_change":"2024-04-25T08:36:26.536761+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536761+0000","last_undegraded":"2024-04-25T08:36:26.536761+0000","last_fullsized":"2024-04-25T08:36:26.536761+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549858+0000","last_change":"2024-04-25T08:36:26.547438+0000","last_active":"2024-04-25T08:36:26.549858+0000","last_peered":"2024-04-25T08:36:26.549858+0000","last_clean":"2024-04-25T08:36:26.549858+0000","last_became_active":"2024-04-25T08:36:26.547171+0000","last_became_peered":"2024-04-25T08:36:26.547171+0000","last_unstale":"2024-04-25T08:36:26.549858+0000","last_undegraded":"2024-04-25T08:36:26.549858+0000","last_fullsized":"2024-04-25T08:36:26.549858+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:37.899171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536841+0000","last_change":"2024-04-25T08:36:26.536841+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536841+0000","last_undegraded":"2024-04-25T08:36:26.536841+0000","last_fullsized":"2024-04-25T08:36:26.536841+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.546253+0000","last_change":"2024-04-25T08:36:26.543814+0000","last_active":"2024-04-25T08:36:26.546253+0000","last_peered":"2024-04-25T08:36:26.546253+0000","last_clean":"2024-04-25T08:36:26.546253+0000","last_became_active":"2024-04-25T08:36:26.543502+0000","last_became_peered":"2024-04-25T08:36:26.543502+0000","last_unstale":"2024-04-25T08:36:26.546253+0000","last_undegraded":"2024-04-25T08:36:26.546253+0000","last_fullsized":"2024-04-25T08:36:26.546253+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:00.280937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.540283+0000","last_change":"2024-04-25T08:36:26.540283+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.540283+0000","last_undegraded":"2024-04-25T08:36:26.540283+0000","last_fullsized":"2024-04-25T08:36:26.540283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549698+0000","last_change":"2024-04-25T08:36:26.546563+0000","last_active":"2024-04-25T08:36:26.549698+0000","last_peered":"2024-04-25T08:36:26.549698+0000","last_clean":"2024-04-25T08:36:26.549698+0000","last_became_active":"2024-04-25T08:36:26.544474+0000","last_became_peered":"2024-04-25T08:36:26.544474+0000","last_unstale":"2024-04-25T08:36:26.549698+0000","last_undegraded":"2024-04-25T08:36:26.549698+0000","last_fullsized":"2024-04-25T08:36:26.549698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:02.642404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538228+0000","last_change":"2024-04-25T08:36:26.538228+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538228+0000","last_undegraded":"2024-04-25T08:36:26.538228+0000","last_fullsized":"2024-04-25T08:36:26.538228+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547548+0000","last_change":"2024-04-25T08:36:26.541992+0000","last_active":"2024-04-25T08:36:26.547548+0000","last_peered":"2024-04-25T08:36:26.547548+0000","last_clean":"2024-04-25T08:36:26.547548+0000","last_became_active":"2024-04-25T08:36:26.541695+0000","last_became_peered":"2024-04-25T08:36:26.541695+0000","last_unstale":"2024-04-25T08:36:26.547548+0000","last_undegraded":"2024-04-25T08:36:26.547548+0000","last_fullsized":"2024-04-25T08:36:26.547548+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:53.435350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.537636+0000","last_change":"2024-04-25T08:36:26.537636+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.537636+0000","last_undegraded":"2024-04-25T08:36:26.537636+0000","last_fullsized":"2024-04-25T08:36:26.537636+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551034+0000","last_change":"2024-04-25T08:36:26.548435+0000","last_active":"2024-04-25T08:36:26.551034+0000","last_peered":"2024-04-25T08:36:26.551034+0000","last_clean":"2024-04-25T08:36:26.551034+0000","last_became_active":"2024-04-25T08:36:26.548141+0000","last_became_peered":"2024-04-25T08:36:26.548141+0000","last_unstale":"2024-04-25T08:36:26.551034+0000","last_undegraded":"2024-04-25T08:36:26.551034+0000","last_fullsized":"2024-04-25T08:36:26.551034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:42.921010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536476+0000","last_change":"2024-04-25T08:36:26.536476+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536476+0000","last_undegraded":"2024-04-25T08:36:26.536476+0000","last_fullsized":"2024-04-25T08:36:26.536476+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.545484+0000","last_change":"2024-04-25T08:36:26.544037+0000","last_active":"2024-04-25T08:36:26.545484+0000","last_peered":"2024-04-25T08:36:26.545484+0000","last_clean":"2024-04-25T08:36:26.545484+0000","last_became_active":"2024-04-25T08:36:26.543747+0000","last_became_peered":"2024-04-25T08:36:26.543747+0000","last_unstale":"2024-04-25T08:36:26.545484+0000","last_undegraded":"2024-04-25T08:36:26.545484+0000","last_fullsized":"2024-04-25T08:36:26.545484+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.121065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534472+0000","last_change":"2024-04-25T08:36:26.534472+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534472+0000","last_undegraded":"2024-04-25T08:36:26.534472+0000","last_fullsized":"2024-04-25T08:36:26.534472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.545018+0000","last_change":"2024-04-25T08:36:26.543672+0000","last_active":"2024-04-25T08:36:26.545018+0000","last_peered":"2024-04-25T08:36:26.545018+0000","last_clean":"2024-04-25T08:36:26.545018+0000","last_became_active":"2024-04-25T08:36:26.543395+0000","last_became_peered":"2024-04-25T08:36:26.543395+0000","last_unstale":"2024-04-25T08:36:26.545018+0000","last_undegraded":"2024-04-25T08:36:26.545018+0000","last_fullsized":"2024-04-25T08:36:26.545018+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:58.975343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533707+0000","last_change":"2024-04-25T08:36:26.533707+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533707+0000","last_undegraded":"2024-04-25T08:36:26.533707+0000","last_fullsized":"2024-04-25T08:36:26.533707+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.552206+0000","last_change":"2024-04-25T08:36:26.551656+0000","last_active":"2024-04-25T08:36:26.552206+0000","last_peered":"2024-04-25T08:36:26.552206+0000","last_clean":"2024-04-25T08:36:26.552206+0000","last_became_active":"2024-04-25T08:36:26.551356+0000","last_became_peered":"2024-04-25T08:36:26.551356+0000","last_unstale":"2024-04-25T08:36:26.552206+0000","last_undegraded":"2024-04-25T08:36:26.552206+0000","last_fullsized":"2024-04-25T08:36:26.552206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:46.989455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533723+0000","last_change":"2024-04-25T08:36:26.533723+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533723+0000","last_undegraded":"2024-04-25T08:36:26.533723+0000","last_fullsized":"2024-04-25T08:36:26.533723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550893+0000","last_change":"2024-04-25T08:36:26.549899+0000","last_active":"2024-04-25T08:36:26.550893+0000","last_peered":"2024-04-25T08:36:26.550893+0000","last_clean":"2024-04-25T08:36:26.550893+0000","last_became_active":"2024-04-25T08:36:26.549599+0000","last_became_peered":"2024-04-25T08:36:26.549599+0000","last_unstale":"2024-04-25T08:36:26.550893+0000","last_undegraded":"2024-04-25T08:36:26.550893+0000","last_fullsized":"2024-04-25T08:36:26.550893+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:42.921010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.534938+0000","last_change":"2024-04-25T08:36:26.534938+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.534938+0000","last_undegraded":"2024-04-25T08:36:26.534938+0000","last_fullsized":"2024-04-25T08:36:26.534938+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.551852+0000","last_change":"2024-04-25T08:36:26.551038+0000","last_active":"2024-04-25T08:36:26.551852+0000","last_peered":"2024-04-25T08:36:26.551852+0000","last_clean":"2024-04-25T08:36:26.551852+0000","last_became_active":"2024-04-25T08:36:26.550729+0000","last_became_peered":"2024-04-25T08:36:26.550729+0000","last_unstale":"2024-04-25T08:36:26.551852+0000","last_undegraded":"2024-04-25T08:36:26.551852+0000","last_fullsized":"2024-04-25T08:36:26.551852+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:26.538681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536406+0000","last_change":"2024-04-25T08:36:26.536406+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536406+0000","last_undegraded":"2024-04-25T08:36:26.536406+0000","last_fullsized":"2024-04-25T08:36:26.536406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.550248+0000","last_change":"2024-04-25T08:36:26.548899+0000","last_active":"2024-04-25T08:36:26.550248+0000","last_peered":"2024-04-25T08:36:26.550248+0000","last_clean":"2024-04-25T08:36:26.550248+0000","last_became_active":"2024-04-25T08:36:26.548616+0000","last_became_peered":"2024-04-25T08:36:26.548616+0000","last_unstale":"2024-04-25T08:36:26.550248+0000","last_undegraded":"2024-04-25T08:36:26.550248+0000","last_fullsized":"2024-04-25T08:36:26.550248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:00.902526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.535097+0000","last_change":"2024-04-25T08:36:26.535097+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.535097+0000","last_undegraded":"2024-04-25T08:36:26.535097+0000","last_fullsized":"2024-04-25T08:36:26.535097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547719+0000","last_change":"2024-04-25T08:36:26.546019+0000","last_active":"2024-04-25T08:36:26.547719+0000","last_peered":"2024-04-25T08:36:26.547719+0000","last_clean":"2024-04-25T08:36:26.547719+0000","last_became_active":"2024-04-25T08:36:26.545702+0000","last_became_peered":"2024-04-25T08:36:26.545702+0000","last_unstale":"2024-04-25T08:36:26.547719+0000","last_undegraded":"2024-04-25T08:36:26.547719+0000","last_fullsized":"2024-04-25T08:36:26.547719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:04.231004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.533653+0000","last_change":"2024-04-25T08:36:26.533653+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.533653+0000","last_undegraded":"2024-04-25T08:36:26.533653+0000","last_fullsized":"2024-04-25T08:36:26.533653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.553551+0000","last_change":"2024-04-25T08:36:26.551919+0000","last_active":"2024-04-25T08:36:26.553551+0000","last_peered":"2024-04-25T08:36:26.553551+0000","last_clean":"2024-04-25T08:36:26.553551+0000","last_became_active":"2024-04-25T08:36:26.550730+0000","last_became_peered":"2024-04-25T08:36:26.550730+0000","last_unstale":"2024-04-25T08:36:26.553551+0000","last_undegraded":"2024-04-25T08:36:26.553551+0000","last_fullsized":"2024-04-25T08:36:26.553551+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.423931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.538357+0000","last_change":"2024-04-25T08:36:26.538357+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.538357+0000","last_undegraded":"2024-04-25T08:36:26.538357+0000","last_fullsized":"2024-04-25T08:36:26.538357+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.549068+0000","last_change":"2024-04-25T08:36:26.546564+0000","last_active":"2024-04-25T08:36:26.549068+0000","last_peered":"2024-04-25T08:36:26.549068+0000","last_clean":"2024-04-25T08:36:26.549068+0000","last_became_active":"2024-04-25T08:36:26.546293+0000","last_became_peered":"2024-04-25T08:36:26.546293+0000","last_unstale":"2024-04-25T08:36:26.549068+0000","last_undegraded":"2024-04-25T08:36:26.549068+0000","last_fullsized":"2024-04-25T08:36:26.549068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:53.435350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.539109+0000","last_change":"2024-04-25T08:36:26.539109+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.539109+0000","last_undegraded":"2024-04-25T08:36:26.539109+0000","last_fullsized":"2024-04-25T08:36:26.539109+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.547656+0000","last_change":"2024-04-25T08:36:26.546007+0000","last_active":"2024-04-25T08:36:26.547656+0000","last_peered":"2024-04-25T08:36:26.547656+0000","last_clean":"2024-04-25T08:36:26.547656+0000","last_became_active":"2024-04-25T08:36:26.545660+0000","last_became_peered":"2024-04-25T08:36:26.545660+0000","last_unstale":"2024-04-25T08:36:26.547656+0000","last_undegraded":"2024-04-25T08:36:26.547656+0000","last_fullsized":"2024-04-25T08:36:26.547656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:48.548532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":80,"state":"creating+peering","last_fresh":"2024-04-25T08:36:26.536701+0000","last_change":"2024-04-25T08:36:26.536701+0000","last_active":"2024-04-25T08:36:26.526708+0000","last_peered":"2024-04-25T08:36:26.526708+0000","last_clean":"2024-04-25T08:36:26.526708+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:36:26.536701+0000","last_undegraded":"2024-04-25T08:36:26.536701+0000","last_fullsized":"2024-04-25T08:36:26.536701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.545751+0000","last_change":"2024-04-25T08:36:26.544427+0000","last_active":"2024-04-25T08:36:26.545751+0000","last_peered":"2024-04-25T08:36:26.545751+0000","last_clean":"2024-04-25T08:36:26.545751+0000","last_became_active":"2024-04-25T08:36:26.544096+0000","last_became_peered":"2024-04-25T08:36:26.544096+0000","last_unstale":"2024-04-25T08:36:26.545751+0000","last_undegraded":"2024-04-25T08:36:26.545751+0000","last_fullsized":"2024-04-25T08:36:26.545751+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:44.885589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":283,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-25T08:36:26.537575+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:36:26.537575+0000","last_peered":"2024-04-25T08:36:26.537575+0000","last_clean":"2024-04-25T08:36:26.537575+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:36:26.537575+0000","last_undegraded":"2024-04-25T08:36:26.537575+0000","last_fullsized":"2024-04-25T08:36:26.537575+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46608,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325232,"statfs":{"total":96636764160,"available":96589037568,"internally_reserved":0,"allocated":1572864,"data_stored":1040288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40456,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331384,"statfs":{"total":96636764160,"available":96595337216,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574912,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46024,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325816,"statfs":{"total":96636764160,"available":96589635584,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40460,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331380,"statfs":{"total":96636764160,"available":96595333120,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40456,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331384,"statfs":{"total":96636764160,"available":96595337216,"internally_reserved":0,"allocated":978944,"data_stored":449920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54912,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316928,"statfs":{"total":96636764160,"available":96580534272,"internally_reserved":0,"allocated":1560576,"data_stored":1033621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:36:33.371 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:36:33.374 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:36:35.977 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:36:36.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.096+0000 7f93eb5d5640 1 Processor -- start 2024-04-25T08:36:36.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.096+0000 7f93eb5d5640 1 -- start start 2024-04-25T08:36:36.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.097+0000 7f93eb5d5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f93e4154cb0 0x7f93e415be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.097+0000 7f93eb5d5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e40c4ed0 0x7f93e40b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.097+0000 7f93eb5d5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e40cd690 con 0x7f93e40b4e60 2024-04-25T08:36:36.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.097+0000 7f93eb5d5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e40b52d0 con 0x7f93e4154cb0 2024-04-25T08:36:36.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.097+0000 7f93eb5d5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40cd440 con 0x7f93e40c4ed0 2024-04-25T08:36:36.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.097+0000 7f93e9b72640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f93e40b4e60 0x7f93e40ce730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49368/0 (socket says 172.21.15.38:49368) 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.097+0000 7f93e9b72640 1 -- 172.21.15.38:0/3846190825 learned_addr learned my addr 172.21.15.38:0/3846190825 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e9371640 1 --2- 172.21.15.38:0/3846190825 >> v2:172.21.15.107:3300/0 conn(0x7f93e4154cb0 0x7f93e415be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e8b70640 1 --2- 172.21.15.38:0/3846190825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e40c4ed0 0x7f93e40b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146563105 0 0) 0x7f93e40cd690 con 0x7f93e40b4e60 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93cc003520 con 0x7f93e40b4e60 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 229731461 0 0) 0x7f93cc003520 con 0x7f93e40b4e60 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e40c4ed0 msgr2=0x7f93e40b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 --2- 172.21.15.38:0/3846190825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e40c4ed0 0x7f93e40b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 >> v2:172.21.15.107:3300/0 conn(0x7f93e4154cb0 msgr2=0x7f93e415be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 --2- 172.21.15.38:0/3846190825 >> v2:172.21.15.107:3300/0 conn(0x7f93e4154cb0 0x7f93e415be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e40cd970 con 0x7f93e40b4e60 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f93d4001670 con 0x7f93e40b4e60 2024-04-25T08:36:36.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 >> v1:172.21.15.38:6789/0 conn(0x7f93e40b4e60 legacy=0x7f93e40ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 --2- 172.21.15.38:0/3846190825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc0039b0 0x7f93cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 --2- 172.21.15.38:0/3846190825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e40c4ed0 0x7f93cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 --2- 172.21.15.38:0/3846190825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93cc004d40 0x7f93cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40b52d0 con 0x7f93cc0039b0 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40cd440 con 0x7f93cc004d40 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc00b9c0 con 0x7f93e40c4ed0 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e9371640 1 --2- 172.21.15.38:0/3846190825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc0039b0 0x7f93cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e8b70640 1 --2- 172.21.15.38:0/3846190825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e40c4ed0 0x7f93cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e9b72640 1 --2- 172.21.15.38:0/3846190825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93cc004d40 0x7f93cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e9371640 1 -- 172.21.15.38:0/3846190825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e40c4ed0 msgr2=0x7f93cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e9371640 1 --2- 172.21.15.38:0/3846190825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e40c4ed0 0x7f93cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e9371640 1 -- 172.21.15.38:0/3846190825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93cc004d40 msgr2=0x7f93cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.098+0000 7f93e9371640 1 --2- 172.21.15.38:0/3846190825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93cc004d40 0x7f93cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.099+0000 7f93e9371640 1 -- 172.21.15.38:0/3846190825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93cc003520 con 0x7f93cc0039b0 2024-04-25T08:36:36.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.099+0000 7f93e9371640 1 --2- 172.21.15.38:0/3846190825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc0039b0 0x7f93cc003dd0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e170 tx=0x7f93cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f455f5dd1dacf5a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.099+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc034020 con 0x7f93cc0039b0 2024-04-25T08:36:36.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.099+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc030430 con 0x7f93cc0039b0 2024-04-25T08:36:36.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.099+0000 7f93e3fff640 1 -- 172.21.15.38:0/3846190825 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc009c10 con 0x7f93cc0039b0 2024-04-25T08:36:36.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.100+0000 7f93eb5d5640 1 -- 172.21.15.38:0/3846190825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc0039b0 msgr2=0x7f93cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.100+0000 7f93eb5d5640 1 --2- 172.21.15.38:0/3846190825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc0039b0 0x7f93cc003dd0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f93dc00e170 tx=0x7f93cc005830 comp rx=0 tx=0).stop 2024-04-25T08:36:36.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.100+0000 7f93e9b72640 1 -- 172.21.15.38:0/3846190825 reap_dead start 2024-04-25T08:36:36.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.100+0000 7f93eb5d5640 1 -- 172.21.15.38:0/3846190825 shutdown_connections 2024-04-25T08:36:36.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.100+0000 7f93eb5d5640 1 -- 172.21.15.38:0/3846190825 >> 172.21.15.38:0/3846190825 conn(0x7f93e406b210 msgr2=0x7f93e406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:36.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.100+0000 7f93eb5d5640 1 -- 172.21.15.38:0/3846190825 shutdown_connections 2024-04-25T08:36:36.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.100+0000 7f93eb5d5640 1 -- 172.21.15.38:0/3846190825 wait complete. 2024-04-25T08:36:36.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.101+0000 7f93eb5d5640 1 Processor -- start 2024-04-25T08:36:36.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.101+0000 7f93eb5d5640 1 -- start start 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.102+0000 7f93eb5d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc0039b0 0x7f93e4146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.102+0000 7f93eb5d5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e4071700 0x7f93e4071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.102+0000 7f93eb5d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93e4151170 0x7f93e41515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.102+0000 7f93eb5d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40ce210 con 0x7f93cc0039b0 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.102+0000 7f93eb5d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40cdf30 con 0x7f93e4151170 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.102+0000 7f93eb5d5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e4155090 con 0x7f93e4071700 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.102+0000 7f93e9371640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc0039b0 0x7f93e4146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.102+0000 7f93e9371640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc0039b0 0x7f93e4146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47784/0 (socket says 172.21.15.38:47784) 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.102+0000 7f93e9371640 1 -- 172.21.15.38:0/285924792 learned_addr learned my addr 172.21.15.38:0/285924792 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.102+0000 7f93e8b70640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e4071700 0x7f93e4071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.102+0000 7f93e9b72640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93e4151170 0x7f93e41515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.103+0000 7f93e9371640 1 -- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e4071700 msgr2=0x7f93e4071b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.103+0000 7f93e9371640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e4071700 0x7f93e4071b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.103+0000 7f93e9371640 1 -- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93e4151170 msgr2=0x7f93e41515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.103+0000 7f93e9371640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93e4151170 0x7f93e41515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.103+0000 7f93e9371640 1 -- 172.21.15.38:0/285924792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e414dd50 con 0x7f93cc0039b0 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.103+0000 7f93e9371640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc0039b0 0x7f93e4146d80 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f93dc033040 tx=0x7f93dc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.103+0000 7f93e1ffb640 1 -- 172.21.15.38:0/285924792 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc034040 con 0x7f93cc0039b0 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.103+0000 7f93e1ffb640 1 -- 172.21.15.38:0/285924792 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc003b90 con 0x7f93cc0039b0 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.103+0000 7f93eb5d5640 1 -- 172.21.15.38:0/285924792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e414bc20 con 0x7f93cc0039b0 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.103+0000 7f93e1ffb640 1 -- 172.21.15.38:0/285924792 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc009c10 con 0x7f93cc0039b0 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.103+0000 7f93eb5d5640 1 -- 172.21.15.38:0/285924792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e414c1e0 con 0x7f93cc0039b0 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.104+0000 7f93eb5d5640 1 -- 172.21.15.38:0/285924792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e40720c0 con 0x7f93cc0039b0 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.105+0000 7f93e1ffb640 1 -- 172.21.15.38:0/285924792 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93dc037020 con 0x7f93cc0039b0 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.105+0000 7f93e1ffb640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f93d006b500 0x7f93d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.105+0000 7f93e1ffb640 1 -- 172.21.15.38:0/285924792 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f93dc0aaf30 con 0x7f93cc0039b0 2024-04-25T08:36:36.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.106+0000 7f93e8b70640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f93d006b500 0x7f93d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.106+0000 7f93e8b70640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f93d006b500 0x7f93d006d9c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f93e40b32b0 tx=0x7f93d8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.110+0000 7f93e1ffb640 1 -- 172.21.15.38:0/285924792 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93dc079c20 con 0x7f93cc0039b0 2024-04-25T08:36:36.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.286+0000 7f93eb5d5640 1 -- 172.21.15.38:0/285924792 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f93e40cd010 con 0x7f93d006b500 2024-04-25T08:36:36.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.293+0000 7f93e1ffb640 1 -- 172.21.15.38:0/285924792 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252082 (secure 0 0 0) 0x7f93e40cd010 con 0x7f93d006b500 2024-04-25T08:36:36.289 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:36.291 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:36:36.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.296+0000 7f93eb5d5640 1 -- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f93d006b500 msgr2=0x7f93d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.296+0000 7f93eb5d5640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f93d006b500 0x7f93d006d9c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f93e40b32b0 tx=0x7f93d8009190 comp rx=0 tx=0).stop 2024-04-25T08:36:36.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.296+0000 7f93eb5d5640 1 -- 172.21.15.38:0/285924792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc0039b0 msgr2=0x7f93e4146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.296+0000 7f93eb5d5640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc0039b0 0x7f93e4146d80 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f93dc033040 tx=0x7f93dc009750 comp rx=0 tx=0).stop 2024-04-25T08:36:36.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.296+0000 7f93eb5d5640 1 -- 172.21.15.38:0/285924792 shutdown_connections 2024-04-25T08:36:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.296+0000 7f93eb5d5640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93e4151170 0x7f93e41515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.296+0000 7f93eb5d5640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f93d006b500 0x7f93d006d9c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.296+0000 7f93eb5d5640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f93e4071700 0x7f93e4071b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.296+0000 7f93eb5d5640 1 --2- 172.21.15.38:0/285924792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93cc0039b0 0x7f93e4146d80 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.296+0000 7f93eb5d5640 1 -- 172.21.15.38:0/285924792 >> 172.21.15.38:0/285924792 conn(0x7f93e406b210 msgr2=0x7f93e40b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.296+0000 7f93eb5d5640 1 -- 172.21.15.38:0/285924792 shutdown_connections 2024-04-25T08:36:36.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.296+0000 7f93eb5d5640 1 -- 172.21.15.38:0/285924792 wait complete. 2024-04-25T08:36:36.301 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-04-25T08:36:35.274627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":337652,"kb_used_data":9064,"kb_used_omap":12,"kb_used_meta":328435,"kb_avail":754637068,"statfs":{"total":773094113280,"available":772748357632,"internally_reserved":0,"allocated":9281536,"data_stored":4895788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":336318000},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002176"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551776+0000","last_change":"2024-04-25T08:36:27.554431+0000","last_active":"2024-04-25T08:36:28.551776+0000","last_peered":"2024-04-25T08:36:28.551776+0000","last_clean":"2024-04-25T08:36:28.551776+0000","last_became_active":"2024-04-25T08:36:27.554120+0000","last_became_peered":"2024-04-25T08:36:27.554120+0000","last_unstale":"2024-04-25T08:36:28.551776+0000","last_undegraded":"2024-04-25T08:36:28.551776+0000","last_fullsized":"2024-04-25T08:36:28.551776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.770460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771693+0000","last_change":"2024-04-25T08:36:27.568294+0000","last_active":"2024-04-25T08:36:28.771693+0000","last_peered":"2024-04-25T08:36:28.771693+0000","last_clean":"2024-04-25T08:36:28.771693+0000","last_became_active":"2024-04-25T08:36:27.567820+0000","last_became_peered":"2024-04-25T08:36:27.567820+0000","last_unstale":"2024-04-25T08:36:28.771693+0000","last_undegraded":"2024-04-25T08:36:28.771693+0000","last_fullsized":"2024-04-25T08:36:28.771693+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:57.778625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770548+0000","last_change":"2024-04-25T08:36:27.562772+0000","last_active":"2024-04-25T08:36:28.770548+0000","last_peered":"2024-04-25T08:36:28.770548+0000","last_clean":"2024-04-25T08:36:28.770548+0000","last_became_active":"2024-04-25T08:36:27.562453+0000","last_became_peered":"2024-04-25T08:36:27.562453+0000","last_unstale":"2024-04-25T08:36:28.770548+0000","last_undegraded":"2024-04-25T08:36:28.770548+0000","last_fullsized":"2024-04-25T08:36:28.770548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:02.402247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769755+0000","last_change":"2024-04-25T08:36:27.552542+0000","last_active":"2024-04-25T08:36:28.769755+0000","last_peered":"2024-04-25T08:36:28.769755+0000","last_clean":"2024-04-25T08:36:28.769755+0000","last_became_active":"2024-04-25T08:36:27.551811+0000","last_became_peered":"2024-04-25T08:36:27.551811+0000","last_unstale":"2024-04-25T08:36:28.769755+0000","last_undegraded":"2024-04-25T08:36:28.769755+0000","last_fullsized":"2024-04-25T08:36:28.769755+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:43.929370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770405+0000","last_change":"2024-04-25T08:36:27.557131+0000","last_active":"2024-04-25T08:36:28.770405+0000","last_peered":"2024-04-25T08:36:28.770405+0000","last_clean":"2024-04-25T08:36:28.770405+0000","last_became_active":"2024-04-25T08:36:27.556382+0000","last_became_peered":"2024-04-25T08:36:27.556382+0000","last_unstale":"2024-04-25T08:36:28.770405+0000","last_undegraded":"2024-04-25T08:36:28.770405+0000","last_fullsized":"2024-04-25T08:36:28.770405+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:45.989091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771858+0000","last_change":"2024-04-25T08:36:27.572598+0000","last_active":"2024-04-25T08:36:28.771858+0000","last_peered":"2024-04-25T08:36:28.771858+0000","last_clean":"2024-04-25T08:36:28.771858+0000","last_became_active":"2024-04-25T08:36:27.572307+0000","last_became_peered":"2024-04-25T08:36:27.572307+0000","last_unstale":"2024-04-25T08:36:28.771858+0000","last_undegraded":"2024-04-25T08:36:28.771858+0000","last_fullsized":"2024-04-25T08:36:28.771858+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:03.734110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552662+0000","last_change":"2024-04-25T08:36:27.564700+0000","last_active":"2024-04-25T08:36:28.552662+0000","last_peered":"2024-04-25T08:36:28.552662+0000","last_clean":"2024-04-25T08:36:28.552662+0000","last_became_active":"2024-04-25T08:36:27.564433+0000","last_became_peered":"2024-04-25T08:36:27.564433+0000","last_unstale":"2024-04-25T08:36:28.552662+0000","last_undegraded":"2024-04-25T08:36:28.552662+0000","last_fullsized":"2024-04-25T08:36:28.552662+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:02.402247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770016+0000","last_change":"2024-04-25T08:36:27.557878+0000","last_active":"2024-04-25T08:36:28.770016+0000","last_peered":"2024-04-25T08:36:28.770016+0000","last_clean":"2024-04-25T08:36:28.770016+0000","last_became_active":"2024-04-25T08:36:27.557582+0000","last_became_peered":"2024-04-25T08:36:27.557582+0000","last_unstale":"2024-04-25T08:36:28.770016+0000","last_undegraded":"2024-04-25T08:36:28.770016+0000","last_fullsized":"2024-04-25T08:36:28.770016+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:16.634438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552873+0000","last_change":"2024-04-25T08:36:27.564668+0000","last_active":"2024-04-25T08:36:28.552873+0000","last_peered":"2024-04-25T08:36:28.552873+0000","last_clean":"2024-04-25T08:36:28.552873+0000","last_became_active":"2024-04-25T08:36:27.564369+0000","last_became_peered":"2024-04-25T08:36:27.564369+0000","last_unstale":"2024-04-25T08:36:28.552873+0000","last_undegraded":"2024-04-25T08:36:28.552873+0000","last_fullsized":"2024-04-25T08:36:28.552873+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:13.649727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769885+0000","last_change":"2024-04-25T08:36:27.548080+0000","last_active":"2024-04-25T08:36:28.769885+0000","last_peered":"2024-04-25T08:36:28.769885+0000","last_clean":"2024-04-25T08:36:28.769885+0000","last_became_active":"2024-04-25T08:36:27.547793+0000","last_became_peered":"2024-04-25T08:36:27.547793+0000","last_unstale":"2024-04-25T08:36:28.769885+0000","last_undegraded":"2024-04-25T08:36:28.769885+0000","last_fullsized":"2024-04-25T08:36:28.769885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:22.191973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771047+0000","last_change":"2024-04-25T08:36:27.570758+0000","last_active":"2024-04-25T08:36:28.771047+0000","last_peered":"2024-04-25T08:36:28.771047+0000","last_clean":"2024-04-25T08:36:28.771047+0000","last_became_active":"2024-04-25T08:36:27.570450+0000","last_became_peered":"2024-04-25T08:36:27.570450+0000","last_unstale":"2024-04-25T08:36:28.771047+0000","last_undegraded":"2024-04-25T08:36:28.771047+0000","last_fullsized":"2024-04-25T08:36:28.771047+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:51.990474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769871+0000","last_change":"2024-04-25T08:36:27.553105+0000","last_active":"2024-04-25T08:36:28.769871+0000","last_peered":"2024-04-25T08:36:28.769871+0000","last_clean":"2024-04-25T08:36:28.769871+0000","last_became_active":"2024-04-25T08:36:27.552846+0000","last_became_peered":"2024-04-25T08:36:27.552846+0000","last_unstale":"2024-04-25T08:36:28.769871+0000","last_undegraded":"2024-04-25T08:36:28.769871+0000","last_fullsized":"2024-04-25T08:36:28.769871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:54.348569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770569+0000","last_change":"2024-04-25T08:36:27.565421+0000","last_active":"2024-04-25T08:36:28.770569+0000","last_peered":"2024-04-25T08:36:28.770569+0000","last_clean":"2024-04-25T08:36:28.770569+0000","last_became_active":"2024-04-25T08:36:27.565097+0000","last_became_peered":"2024-04-25T08:36:27.565097+0000","last_unstale":"2024-04-25T08:36:28.770569+0000","last_undegraded":"2024-04-25T08:36:28.770569+0000","last_fullsized":"2024-04-25T08:36:28.770569+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:21.763143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771463+0000","last_change":"2024-04-25T08:36:27.572498+0000","last_active":"2024-04-25T08:36:28.771463+0000","last_peered":"2024-04-25T08:36:28.771463+0000","last_clean":"2024-04-25T08:36:28.771463+0000","last_became_active":"2024-04-25T08:36:27.572059+0000","last_became_peered":"2024-04-25T08:36:27.572059+0000","last_unstale":"2024-04-25T08:36:28.771463+0000","last_undegraded":"2024-04-25T08:36:28.771463+0000","last_fullsized":"2024-04-25T08:36:28.771463+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:44.062765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769866+0000","last_change":"2024-04-25T08:36:27.562767+0000","last_active":"2024-04-25T08:36:28.769866+0000","last_peered":"2024-04-25T08:36:28.769866+0000","last_clean":"2024-04-25T08:36:28.769866+0000","last_became_active":"2024-04-25T08:36:27.560476+0000","last_became_peered":"2024-04-25T08:36:27.560476+0000","last_unstale":"2024-04-25T08:36:28.769866+0000","last_undegraded":"2024-04-25T08:36:28.769866+0000","last_fullsized":"2024-04-25T08:36:28.769866+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:44.241567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551272+0000","last_change":"2024-04-25T08:36:27.551848+0000","last_active":"2024-04-25T08:36:28.551272+0000","last_peered":"2024-04-25T08:36:28.551272+0000","last_clean":"2024-04-25T08:36:28.551272+0000","last_became_active":"2024-04-25T08:36:27.551576+0000","last_became_peered":"2024-04-25T08:36:27.551576+0000","last_unstale":"2024-04-25T08:36:28.551272+0000","last_undegraded":"2024-04-25T08:36:28.551272+0000","last_fullsized":"2024-04-25T08:36:28.551272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:00.734796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770496+0000","last_change":"2024-04-25T08:36:27.565046+0000","last_active":"2024-04-25T08:36:28.770496+0000","last_peered":"2024-04-25T08:36:28.770496+0000","last_clean":"2024-04-25T08:36:28.770496+0000","last_became_active":"2024-04-25T08:36:27.562211+0000","last_became_peered":"2024-04-25T08:36:27.562211+0000","last_unstale":"2024-04-25T08:36:28.770496+0000","last_undegraded":"2024-04-25T08:36:28.770496+0000","last_fullsized":"2024-04-25T08:36:28.770496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:10.217172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770564+0000","last_change":"2024-04-25T08:36:27.555601+0000","last_active":"2024-04-25T08:36:28.770564+0000","last_peered":"2024-04-25T08:36:28.770564+0000","last_clean":"2024-04-25T08:36:28.770564+0000","last_became_active":"2024-04-25T08:36:27.555278+0000","last_became_peered":"2024-04-25T08:36:27.555278+0000","last_unstale":"2024-04-25T08:36:28.770564+0000","last_undegraded":"2024-04-25T08:36:28.770564+0000","last_fullsized":"2024-04-25T08:36:28.770564+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:27.148316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771776+0000","last_change":"2024-04-25T08:36:27.570628+0000","last_active":"2024-04-25T08:36:28.771776+0000","last_peered":"2024-04-25T08:36:28.771776+0000","last_clean":"2024-04-25T08:36:28.771776+0000","last_became_active":"2024-04-25T08:36:27.569009+0000","last_became_peered":"2024-04-25T08:36:27.569009+0000","last_unstale":"2024-04-25T08:36:28.771776+0000","last_undegraded":"2024-04-25T08:36:28.771776+0000","last_fullsized":"2024-04-25T08:36:28.771776+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:01.328616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770411+0000","last_change":"2024-04-25T08:36:27.563083+0000","last_active":"2024-04-25T08:36:28.770411+0000","last_peered":"2024-04-25T08:36:28.770411+0000","last_clean":"2024-04-25T08:36:28.770411+0000","last_became_active":"2024-04-25T08:36:27.562760+0000","last_became_peered":"2024-04-25T08:36:27.562760+0000","last_unstale":"2024-04-25T08:36:28.770411+0000","last_undegraded":"2024-04-25T08:36:28.770411+0000","last_fullsized":"2024-04-25T08:36:28.770411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:47.916632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552406+0000","last_change":"2024-04-25T08:36:27.560674+0000","last_active":"2024-04-25T08:36:28.552406+0000","last_peered":"2024-04-25T08:36:28.552406+0000","last_clean":"2024-04-25T08:36:28.552406+0000","last_became_active":"2024-04-25T08:36:27.560330+0000","last_became_peered":"2024-04-25T08:36:27.560330+0000","last_unstale":"2024-04-25T08:36:28.552406+0000","last_undegraded":"2024-04-25T08:36:28.552406+0000","last_fullsized":"2024-04-25T08:36:28.552406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:54.443711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770337+0000","last_change":"2024-04-25T08:36:27.565513+0000","last_active":"2024-04-25T08:36:28.770337+0000","last_peered":"2024-04-25T08:36:28.770337+0000","last_clean":"2024-04-25T08:36:28.770337+0000","last_became_active":"2024-04-25T08:36:27.565183+0000","last_became_peered":"2024-04-25T08:36:27.565183+0000","last_unstale":"2024-04-25T08:36:28.770337+0000","last_undegraded":"2024-04-25T08:36:28.770337+0000","last_fullsized":"2024-04-25T08:36:28.770337+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:44.176816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552214+0000","last_change":"2024-04-25T08:36:27.556478+0000","last_active":"2024-04-25T08:36:28.552214+0000","last_peered":"2024-04-25T08:36:28.552214+0000","last_clean":"2024-04-25T08:36:28.552214+0000","last_became_active":"2024-04-25T08:36:27.556198+0000","last_became_peered":"2024-04-25T08:36:27.556198+0000","last_unstale":"2024-04-25T08:36:28.552214+0000","last_undegraded":"2024-04-25T08:36:28.552214+0000","last_fullsized":"2024-04-25T08:36:28.552214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:05.239364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770151+0000","last_change":"2024-04-25T08:36:27.558726+0000","last_active":"2024-04-25T08:36:28.770151+0000","last_peered":"2024-04-25T08:36:28.770151+0000","last_clean":"2024-04-25T08:36:28.770151+0000","last_became_active":"2024-04-25T08:36:27.558453+0000","last_became_peered":"2024-04-25T08:36:27.558453+0000","last_unstale":"2024-04-25T08:36:28.770151+0000","last_undegraded":"2024-04-25T08:36:28.770151+0000","last_fullsized":"2024-04-25T08:36:28.770151+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:09.873213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771672+0000","last_change":"2024-04-25T08:36:27.564917+0000","last_active":"2024-04-25T08:36:28.771672+0000","last_peered":"2024-04-25T08:36:28.771672+0000","last_clean":"2024-04-25T08:36:28.771672+0000","last_became_active":"2024-04-25T08:36:27.564612+0000","last_became_peered":"2024-04-25T08:36:27.564612+0000","last_unstale":"2024-04-25T08:36:28.771672+0000","last_undegraded":"2024-04-25T08:36:28.771672+0000","last_fullsized":"2024-04-25T08:36:28.771672+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:15.796724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769901+0000","last_change":"2024-04-25T08:36:27.561807+0000","last_active":"2024-04-25T08:36:28.769901+0000","last_peered":"2024-04-25T08:36:28.769901+0000","last_clean":"2024-04-25T08:36:28.769901+0000","last_became_active":"2024-04-25T08:36:27.561525+0000","last_became_peered":"2024-04-25T08:36:27.561525+0000","last_unstale":"2024-04-25T08:36:28.769901+0000","last_undegraded":"2024-04-25T08:36:28.769901+0000","last_fullsized":"2024-04-25T08:36:28.769901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:45.569630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551765+0000","last_change":"2024-04-25T08:36:27.554824+0000","last_active":"2024-04-25T08:36:28.551765+0000","last_peered":"2024-04-25T08:36:28.551765+0000","last_clean":"2024-04-25T08:36:28.551765+0000","last_became_active":"2024-04-25T08:36:27.552189+0000","last_became_peered":"2024-04-25T08:36:27.552189+0000","last_unstale":"2024-04-25T08:36:28.551765+0000","last_undegraded":"2024-04-25T08:36:28.551765+0000","last_fullsized":"2024-04-25T08:36:28.551765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:01.328616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770568+0000","last_change":"2024-04-25T08:36:27.559720+0000","last_active":"2024-04-25T08:36:28.770568+0000","last_peered":"2024-04-25T08:36:28.770568+0000","last_clean":"2024-04-25T08:36:28.770568+0000","last_became_active":"2024-04-25T08:36:27.559445+0000","last_became_peered":"2024-04-25T08:36:27.559445+0000","last_unstale":"2024-04-25T08:36:28.770568+0000","last_undegraded":"2024-04-25T08:36:28.770568+0000","last_fullsized":"2024-04-25T08:36:28.770568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:11:50.619543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771944+0000","last_change":"2024-04-25T08:36:27.560308+0000","last_active":"2024-04-25T08:36:28.771944+0000","last_peered":"2024-04-25T08:36:28.771944+0000","last_clean":"2024-04-25T08:36:28.771944+0000","last_became_active":"2024-04-25T08:36:27.560011+0000","last_became_peered":"2024-04-25T08:36:27.560011+0000","last_unstale":"2024-04-25T08:36:28.771944+0000","last_undegraded":"2024-04-25T08:36:28.771944+0000","last_fullsized":"2024-04-25T08:36:28.771944+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:35.163946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551504+0000","last_change":"2024-04-25T08:36:27.558736+0000","last_active":"2024-04-25T08:36:28.551504+0000","last_peered":"2024-04-25T08:36:28.551504+0000","last_clean":"2024-04-25T08:36:28.551504+0000","last_became_active":"2024-04-25T08:36:27.558452+0000","last_became_peered":"2024-04-25T08:36:27.558452+0000","last_unstale":"2024-04-25T08:36:28.551504+0000","last_undegraded":"2024-04-25T08:36:28.551504+0000","last_fullsized":"2024-04-25T08:36:28.551504+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:44.062765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770225+0000","last_change":"2024-04-25T08:36:27.546652+0000","last_active":"2024-04-25T08:36:28.770225+0000","last_peered":"2024-04-25T08:36:28.770225+0000","last_clean":"2024-04-25T08:36:28.770225+0000","last_became_active":"2024-04-25T08:36:27.546354+0000","last_became_peered":"2024-04-25T08:36:27.546354+0000","last_unstale":"2024-04-25T08:36:28.770225+0000","last_undegraded":"2024-04-25T08:36:28.770225+0000","last_fullsized":"2024-04-25T08:36:28.770225+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:47.997815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551648+0000","last_change":"2024-04-25T08:36:27.555837+0000","last_active":"2024-04-25T08:36:28.551648+0000","last_peered":"2024-04-25T08:36:28.551648+0000","last_clean":"2024-04-25T08:36:28.551648+0000","last_became_active":"2024-04-25T08:36:27.555545+0000","last_became_peered":"2024-04-25T08:36:27.555545+0000","last_unstale":"2024-04-25T08:36:28.551648+0000","last_undegraded":"2024-04-25T08:36:28.551648+0000","last_fullsized":"2024-04-25T08:36:28.551648+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:29.886186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769868+0000","last_change":"2024-04-25T08:36:27.556387+0000","last_active":"2024-04-25T08:36:28.769868+0000","last_peered":"2024-04-25T08:36:28.769868+0000","last_clean":"2024-04-25T08:36:28.769868+0000","last_became_active":"2024-04-25T08:36:27.556089+0000","last_became_peered":"2024-04-25T08:36:27.556089+0000","last_unstale":"2024-04-25T08:36:28.769868+0000","last_undegraded":"2024-04-25T08:36:28.769868+0000","last_fullsized":"2024-04-25T08:36:28.769868+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:00.374652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770680+0000","last_change":"2024-04-25T08:36:26.552070+0000","last_active":"2024-04-25T08:36:28.770680+0000","last_peered":"2024-04-25T08:36:28.770680+0000","last_clean":"2024-04-25T08:36:28.770680+0000","last_became_active":"2024-04-25T08:36:26.550647+0000","last_became_peered":"2024-04-25T08:36:26.550647+0000","last_unstale":"2024-04-25T08:36:28.770680+0000","last_undegraded":"2024-04-25T08:36:28.770680+0000","last_fullsized":"2024-04-25T08:36:28.770680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:40.582923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770032+0000","last_change":"2024-04-25T08:36:27.558222+0000","last_active":"2024-04-25T08:36:28.770032+0000","last_peered":"2024-04-25T08:36:28.770032+0000","last_clean":"2024-04-25T08:36:28.770032+0000","last_became_active":"2024-04-25T08:36:27.557968+0000","last_became_peered":"2024-04-25T08:36:27.557968+0000","last_unstale":"2024-04-25T08:36:28.770032+0000","last_undegraded":"2024-04-25T08:36:28.770032+0000","last_fullsized":"2024-04-25T08:36:28.770032+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:02.886555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551574+0000","last_change":"2024-04-25T08:36:26.549707+0000","last_active":"2024-04-25T08:36:28.551574+0000","last_peered":"2024-04-25T08:36:28.551574+0000","last_clean":"2024-04-25T08:36:28.551574+0000","last_became_active":"2024-04-25T08:36:26.549227+0000","last_became_peered":"2024-04-25T08:36:26.549227+0000","last_unstale":"2024-04-25T08:36:28.551574+0000","last_undegraded":"2024-04-25T08:36:28.551574+0000","last_fullsized":"2024-04-25T08:36:28.551574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:42.921010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552638+0000","last_change":"2024-04-25T08:36:27.562194+0000","last_active":"2024-04-25T08:36:28.552638+0000","last_peered":"2024-04-25T08:36:28.552638+0000","last_clean":"2024-04-25T08:36:28.552638+0000","last_became_active":"2024-04-25T08:36:27.561931+0000","last_became_peered":"2024-04-25T08:36:27.561931+0000","last_unstale":"2024-04-25T08:36:28.552638+0000","last_undegraded":"2024-04-25T08:36:28.552638+0000","last_fullsized":"2024-04-25T08:36:28.552638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:48.929829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551761+0000","last_change":"2024-04-25T08:36:26.551664+0000","last_active":"2024-04-25T08:36:28.551761+0000","last_peered":"2024-04-25T08:36:28.551761+0000","last_clean":"2024-04-25T08:36:28.551761+0000","last_became_active":"2024-04-25T08:36:26.551388+0000","last_became_peered":"2024-04-25T08:36:26.551388+0000","last_unstale":"2024-04-25T08:36:28.551761+0000","last_undegraded":"2024-04-25T08:36:28.551761+0000","last_fullsized":"2024-04-25T08:36:28.551761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:44.561270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552420+0000","last_change":"2024-04-25T08:36:27.559882+0000","last_active":"2024-04-25T08:36:28.552420+0000","last_peered":"2024-04-25T08:36:28.552420+0000","last_clean":"2024-04-25T08:36:28.552420+0000","last_became_active":"2024-04-25T08:36:27.559582+0000","last_became_peered":"2024-04-25T08:36:27.559582+0000","last_unstale":"2024-04-25T08:36:28.552420+0000","last_undegraded":"2024-04-25T08:36:28.552420+0000","last_fullsized":"2024-04-25T08:36:28.552420+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:49.556892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552887+0000","last_change":"2024-04-25T08:36:26.545919+0000","last_active":"2024-04-25T08:36:28.552887+0000","last_peered":"2024-04-25T08:36:28.552887+0000","last_clean":"2024-04-25T08:36:28.552887+0000","last_became_active":"2024-04-25T08:36:26.545633+0000","last_became_peered":"2024-04-25T08:36:26.545633+0000","last_unstale":"2024-04-25T08:36:28.552887+0000","last_undegraded":"2024-04-25T08:36:28.552887+0000","last_fullsized":"2024-04-25T08:36:28.552887+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:47.921469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771303+0000","last_change":"2024-04-25T08:36:27.567546+0000","last_active":"2024-04-25T08:36:28.771303+0000","last_peered":"2024-04-25T08:36:28.771303+0000","last_clean":"2024-04-25T08:36:28.771303+0000","last_became_active":"2024-04-25T08:36:27.567271+0000","last_became_peered":"2024-04-25T08:36:27.567271+0000","last_unstale":"2024-04-25T08:36:28.771303+0000","last_undegraded":"2024-04-25T08:36:28.771303+0000","last_fullsized":"2024-04-25T08:36:28.771303+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:14.776371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552524+0000","last_change":"2024-04-25T08:36:26.549171+0000","last_active":"2024-04-25T08:36:28.552524+0000","last_peered":"2024-04-25T08:36:28.552524+0000","last_clean":"2024-04-25T08:36:28.552524+0000","last_became_active":"2024-04-25T08:36:26.548875+0000","last_became_peered":"2024-04-25T08:36:26.548875+0000","last_unstale":"2024-04-25T08:36:28.552524+0000","last_undegraded":"2024-04-25T08:36:28.552524+0000","last_fullsized":"2024-04-25T08:36:28.552524+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:01.314794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771172+0000","last_change":"2024-04-25T08:36:27.570536+0000","last_active":"2024-04-25T08:36:28.771172+0000","last_peered":"2024-04-25T08:36:28.771172+0000","last_clean":"2024-04-25T08:36:28.771172+0000","last_became_active":"2024-04-25T08:36:27.570255+0000","last_became_peered":"2024-04-25T08:36:27.570255+0000","last_unstale":"2024-04-25T08:36:28.771172+0000","last_undegraded":"2024-04-25T08:36:28.771172+0000","last_fullsized":"2024-04-25T08:36:28.771172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:38.270713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769532+0000","last_change":"2024-04-25T08:36:26.544834+0000","last_active":"2024-04-25T08:36:28.769532+0000","last_peered":"2024-04-25T08:36:28.769532+0000","last_clean":"2024-04-25T08:36:28.769532+0000","last_became_active":"2024-04-25T08:36:26.544549+0000","last_became_peered":"2024-04-25T08:36:26.544549+0000","last_unstale":"2024-04-25T08:36:28.769532+0000","last_undegraded":"2024-04-25T08:36:28.769532+0000","last_fullsized":"2024-04-25T08:36:28.769532+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:26.139956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.553137+0000","last_change":"2024-04-25T08:36:27.564422+0000","last_active":"2024-04-25T08:36:28.553137+0000","last_peered":"2024-04-25T08:36:28.553137+0000","last_clean":"2024-04-25T08:36:28.553137+0000","last_became_active":"2024-04-25T08:36:27.563789+0000","last_became_peered":"2024-04-25T08:36:27.563789+0000","last_unstale":"2024-04-25T08:36:28.553137+0000","last_undegraded":"2024-04-25T08:36:28.553137+0000","last_fullsized":"2024-04-25T08:36:28.553137+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:26.885379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770889+0000","last_change":"2024-04-25T08:36:26.546765+0000","last_active":"2024-04-25T08:36:28.770889+0000","last_peered":"2024-04-25T08:36:28.770889+0000","last_clean":"2024-04-25T08:36:28.770889+0000","last_became_active":"2024-04-25T08:36:26.546467+0000","last_became_peered":"2024-04-25T08:36:26.546467+0000","last_unstale":"2024-04-25T08:36:28.770889+0000","last_undegraded":"2024-04-25T08:36:28.770889+0000","last_fullsized":"2024-04-25T08:36:28.770889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.183613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.553515+0000","last_change":"2024-04-25T08:36:27.563404+0000","last_active":"2024-04-25T08:36:28.553515+0000","last_peered":"2024-04-25T08:36:28.553515+0000","last_clean":"2024-04-25T08:36:28.553515+0000","last_became_active":"2024-04-25T08:36:27.563141+0000","last_became_peered":"2024-04-25T08:36:27.563141+0000","last_unstale":"2024-04-25T08:36:28.553515+0000","last_undegraded":"2024-04-25T08:36:28.553515+0000","last_fullsized":"2024-04-25T08:36:28.553515+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:31.429885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770106+0000","last_change":"2024-04-25T08:36:26.544689+0000","last_active":"2024-04-25T08:36:28.770106+0000","last_peered":"2024-04-25T08:36:28.770106+0000","last_clean":"2024-04-25T08:36:28.770106+0000","last_became_active":"2024-04-25T08:36:26.544353+0000","last_became_peered":"2024-04-25T08:36:26.544353+0000","last_unstale":"2024-04-25T08:36:28.770106+0000","last_undegraded":"2024-04-25T08:36:28.770106+0000","last_fullsized":"2024-04-25T08:36:28.770106+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:47.285395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769909+0000","last_change":"2024-04-25T08:36:27.557302+0000","last_active":"2024-04-25T08:36:28.769909+0000","last_peered":"2024-04-25T08:36:28.769909+0000","last_clean":"2024-04-25T08:36:28.769909+0000","last_became_active":"2024-04-25T08:36:27.556979+0000","last_became_peered":"2024-04-25T08:36:27.556979+0000","last_unstale":"2024-04-25T08:36:28.769909+0000","last_undegraded":"2024-04-25T08:36:28.769909+0000","last_fullsized":"2024-04-25T08:36:28.769909+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:39.609650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551881+0000","last_change":"2024-04-25T08:36:26.549714+0000","last_active":"2024-04-25T08:36:28.551881+0000","last_peered":"2024-04-25T08:36:28.551881+0000","last_clean":"2024-04-25T08:36:28.551881+0000","last_became_active":"2024-04-25T08:36:26.548992+0000","last_became_peered":"2024-04-25T08:36:26.548992+0000","last_unstale":"2024-04-25T08:36:28.551881+0000","last_undegraded":"2024-04-25T08:36:28.551881+0000","last_fullsized":"2024-04-25T08:36:28.551881+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.183613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769670+0000","last_change":"2024-04-25T08:36:27.551737+0000","last_active":"2024-04-25T08:36:28.769670+0000","last_peered":"2024-04-25T08:36:28.769670+0000","last_clean":"2024-04-25T08:36:28.769670+0000","last_became_active":"2024-04-25T08:36:27.551470+0000","last_became_peered":"2024-04-25T08:36:27.551470+0000","last_unstale":"2024-04-25T08:36:28.769670+0000","last_undegraded":"2024-04-25T08:36:28.769670+0000","last_fullsized":"2024-04-25T08:36:28.769670+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:45.893950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770000+0000","last_change":"2024-04-25T08:36:26.546855+0000","last_active":"2024-04-25T08:36:28.770000+0000","last_peered":"2024-04-25T08:36:28.770000+0000","last_clean":"2024-04-25T08:36:28.770000+0000","last_became_active":"2024-04-25T08:36:26.546607+0000","last_became_peered":"2024-04-25T08:36:26.546607+0000","last_unstale":"2024-04-25T08:36:28.770000+0000","last_undegraded":"2024-04-25T08:36:28.770000+0000","last_fullsized":"2024-04-25T08:36:28.770000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:37.899171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771810+0000","last_change":"2024-04-25T08:36:27.561946+0000","last_active":"2024-04-25T08:36:28.771810+0000","last_peered":"2024-04-25T08:36:28.771810+0000","last_clean":"2024-04-25T08:36:28.771810+0000","last_became_active":"2024-04-25T08:36:27.561644+0000","last_became_peered":"2024-04-25T08:36:27.561644+0000","last_unstale":"2024-04-25T08:36:28.771810+0000","last_undegraded":"2024-04-25T08:36:28.771810+0000","last_fullsized":"2024-04-25T08:36:28.771810+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:22.770460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770522+0000","last_change":"2024-04-25T08:36:26.548301+0000","last_active":"2024-04-25T08:36:28.770522+0000","last_peered":"2024-04-25T08:36:28.770522+0000","last_clean":"2024-04-25T08:36:28.770522+0000","last_became_active":"2024-04-25T08:36:26.547849+0000","last_became_peered":"2024-04-25T08:36:26.547849+0000","last_unstale":"2024-04-25T08:36:28.770522+0000","last_undegraded":"2024-04-25T08:36:28.770522+0000","last_fullsized":"2024-04-25T08:36:28.770522+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:46.915495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770148+0000","last_change":"2024-04-25T08:36:27.561791+0000","last_active":"2024-04-25T08:36:28.770148+0000","last_peered":"2024-04-25T08:36:28.770148+0000","last_clean":"2024-04-25T08:36:28.770148+0000","last_became_active":"2024-04-25T08:36:27.561260+0000","last_became_peered":"2024-04-25T08:36:27.561260+0000","last_unstale":"2024-04-25T08:36:28.770148+0000","last_undegraded":"2024-04-25T08:36:28.770148+0000","last_fullsized":"2024-04-25T08:36:28.770148+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:27.463057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.553103+0000","last_change":"2024-04-25T08:36:26.545189+0000","last_active":"2024-04-25T08:36:28.553103+0000","last_peered":"2024-04-25T08:36:28.553103+0000","last_clean":"2024-04-25T08:36:28.553103+0000","last_became_active":"2024-04-25T08:36:26.544888+0000","last_became_peered":"2024-04-25T08:36:26.544888+0000","last_unstale":"2024-04-25T08:36:28.553103+0000","last_undegraded":"2024-04-25T08:36:28.553103+0000","last_fullsized":"2024-04-25T08:36:28.553103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:48.671766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769937+0000","last_change":"2024-04-25T08:36:27.560547+0000","last_active":"2024-04-25T08:36:28.769937+0000","last_peered":"2024-04-25T08:36:28.769937+0000","last_clean":"2024-04-25T08:36:28.769937+0000","last_became_active":"2024-04-25T08:36:27.560251+0000","last_became_peered":"2024-04-25T08:36:27.560251+0000","last_unstale":"2024-04-25T08:36:28.769937+0000","last_undegraded":"2024-04-25T08:36:28.769937+0000","last_fullsized":"2024-04-25T08:36:28.769937+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:00.904609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769536+0000","last_change":"2024-04-25T08:36:26.544584+0000","last_active":"2024-04-25T08:36:28.769536+0000","last_peered":"2024-04-25T08:36:28.769536+0000","last_clean":"2024-04-25T08:36:28.769536+0000","last_became_active":"2024-04-25T08:36:26.544307+0000","last_became_peered":"2024-04-25T08:36:26.544307+0000","last_unstale":"2024-04-25T08:36:28.769536+0000","last_undegraded":"2024-04-25T08:36:28.769536+0000","last_fullsized":"2024-04-25T08:36:28.769536+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:01.393887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.553196+0000","last_change":"2024-04-25T08:36:27.555182+0000","last_active":"2024-04-25T08:36:28.553196+0000","last_peered":"2024-04-25T08:36:28.553196+0000","last_clean":"2024-04-25T08:36:28.553196+0000","last_became_active":"2024-04-25T08:36:27.554877+0000","last_became_peered":"2024-04-25T08:36:27.554877+0000","last_unstale":"2024-04-25T08:36:28.553196+0000","last_undegraded":"2024-04-25T08:36:28.553196+0000","last_fullsized":"2024-04-25T08:36:28.553196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:38.907532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551560+0000","last_change":"2024-04-25T08:36:26.550747+0000","last_active":"2024-04-25T08:36:28.551560+0000","last_peered":"2024-04-25T08:36:28.551560+0000","last_clean":"2024-04-25T08:36:28.551560+0000","last_became_active":"2024-04-25T08:36:26.550435+0000","last_became_peered":"2024-04-25T08:36:26.550435+0000","last_unstale":"2024-04-25T08:36:28.551560+0000","last_undegraded":"2024-04-25T08:36:28.551560+0000","last_fullsized":"2024-04-25T08:36:28.551560+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:44.885589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.553732+0000","last_change":"2024-04-25T08:36:27.553818+0000","last_active":"2024-04-25T08:36:28.553732+0000","last_peered":"2024-04-25T08:36:28.553732+0000","last_clean":"2024-04-25T08:36:28.553732+0000","last_became_active":"2024-04-25T08:36:27.553517+0000","last_became_peered":"2024-04-25T08:36:27.553517+0000","last_unstale":"2024-04-25T08:36:28.553732+0000","last_undegraded":"2024-04-25T08:36:28.553732+0000","last_fullsized":"2024-04-25T08:36:28.553732+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:27.547041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770070+0000","last_change":"2024-04-25T08:36:26.552811+0000","last_active":"2024-04-25T08:36:28.770070+0000","last_peered":"2024-04-25T08:36:28.770070+0000","last_clean":"2024-04-25T08:36:28.770070+0000","last_became_active":"2024-04-25T08:36:26.552472+0000","last_became_peered":"2024-04-25T08:36:26.552472+0000","last_unstale":"2024-04-25T08:36:28.770070+0000","last_undegraded":"2024-04-25T08:36:28.770070+0000","last_fullsized":"2024-04-25T08:36:28.770070+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:17.913936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.553650+0000","last_change":"2024-04-25T08:36:27.562759+0000","last_active":"2024-04-25T08:36:28.553650+0000","last_peered":"2024-04-25T08:36:28.553650+0000","last_clean":"2024-04-25T08:36:28.553650+0000","last_became_active":"2024-04-25T08:36:27.562438+0000","last_became_peered":"2024-04-25T08:36:27.562438+0000","last_unstale":"2024-04-25T08:36:28.553650+0000","last_undegraded":"2024-04-25T08:36:28.553650+0000","last_fullsized":"2024-04-25T08:36:28.553650+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:58.857883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770007+0000","last_change":"2024-04-25T08:36:26.545891+0000","last_active":"2024-04-25T08:36:28.770007+0000","last_peered":"2024-04-25T08:36:28.770007+0000","last_clean":"2024-04-25T08:36:28.770007+0000","last_became_active":"2024-04-25T08:36:26.545452+0000","last_became_peered":"2024-04-25T08:36:26.545452+0000","last_unstale":"2024-04-25T08:36:28.770007+0000","last_undegraded":"2024-04-25T08:36:28.770007+0000","last_fullsized":"2024-04-25T08:36:28.770007+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:53.340209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770979+0000","last_change":"2024-04-25T08:36:27.567409+0000","last_active":"2024-04-25T08:36:28.770979+0000","last_peered":"2024-04-25T08:36:28.770979+0000","last_clean":"2024-04-25T08:36:28.770979+0000","last_became_active":"2024-04-25T08:36:27.567125+0000","last_became_peered":"2024-04-25T08:36:27.567125+0000","last_unstale":"2024-04-25T08:36:28.770979+0000","last_undegraded":"2024-04-25T08:36:28.770979+0000","last_fullsized":"2024-04-25T08:36:28.770979+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:00.734796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551994+0000","last_change":"2024-04-25T08:36:26.547438+0000","last_active":"2024-04-25T08:36:28.551994+0000","last_peered":"2024-04-25T08:36:28.551994+0000","last_clean":"2024-04-25T08:36:28.551994+0000","last_became_active":"2024-04-25T08:36:26.547171+0000","last_became_peered":"2024-04-25T08:36:26.547171+0000","last_unstale":"2024-04-25T08:36:28.551994+0000","last_undegraded":"2024-04-25T08:36:28.551994+0000","last_fullsized":"2024-04-25T08:36:28.551994+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:53.340209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769913+0000","last_change":"2024-04-25T08:36:27.562772+0000","last_active":"2024-04-25T08:36:28.769913+0000","last_peered":"2024-04-25T08:36:28.769913+0000","last_clean":"2024-04-25T08:36:28.769913+0000","last_became_active":"2024-04-25T08:36:27.562477+0000","last_became_peered":"2024-04-25T08:36:27.562477+0000","last_unstale":"2024-04-25T08:36:28.769913+0000","last_undegraded":"2024-04-25T08:36:28.769913+0000","last_fullsized":"2024-04-25T08:36:28.769913+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:24.167163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.553079+0000","last_change":"2024-04-25T08:36:26.543814+0000","last_active":"2024-04-25T08:36:28.553079+0000","last_peered":"2024-04-25T08:36:28.553079+0000","last_clean":"2024-04-25T08:36:28.553079+0000","last_became_active":"2024-04-25T08:36:26.543502+0000","last_became_peered":"2024-04-25T08:36:26.543502+0000","last_unstale":"2024-04-25T08:36:28.553079+0000","last_undegraded":"2024-04-25T08:36:28.553079+0000","last_fullsized":"2024-04-25T08:36:28.553079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:45.468719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770290+0000","last_change":"2024-04-25T08:36:27.568750+0000","last_active":"2024-04-25T08:36:28.770290+0000","last_peered":"2024-04-25T08:36:28.770290+0000","last_clean":"2024-04-25T08:36:28.770290+0000","last_became_active":"2024-04-25T08:36:27.568483+0000","last_became_peered":"2024-04-25T08:36:27.568483+0000","last_unstale":"2024-04-25T08:36:28.770290+0000","last_undegraded":"2024-04-25T08:36:28.770290+0000","last_fullsized":"2024-04-25T08:36:28.770290+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:44.221302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770628+0000","last_change":"2024-04-25T08:36:26.546563+0000","last_active":"2024-04-25T08:36:28.770628+0000","last_peered":"2024-04-25T08:36:28.770628+0000","last_clean":"2024-04-25T08:36:28.770628+0000","last_became_active":"2024-04-25T08:36:26.544474+0000","last_became_peered":"2024-04-25T08:36:26.544474+0000","last_unstale":"2024-04-25T08:36:28.770628+0000","last_undegraded":"2024-04-25T08:36:28.770628+0000","last_fullsized":"2024-04-25T08:36:28.770628+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:24.861057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771509+0000","last_change":"2024-04-25T08:36:27.568061+0000","last_active":"2024-04-25T08:36:28.771509+0000","last_peered":"2024-04-25T08:36:28.771509+0000","last_clean":"2024-04-25T08:36:28.771509+0000","last_became_active":"2024-04-25T08:36:27.567781+0000","last_became_peered":"2024-04-25T08:36:27.567781+0000","last_unstale":"2024-04-25T08:36:28.771509+0000","last_undegraded":"2024-04-25T08:36:28.771509+0000","last_fullsized":"2024-04-25T08:36:28.771509+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:29.886186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770345+0000","last_change":"2024-04-25T08:36:26.541992+0000","last_active":"2024-04-25T08:36:28.770345+0000","last_peered":"2024-04-25T08:36:28.770345+0000","last_clean":"2024-04-25T08:36:28.770345+0000","last_became_active":"2024-04-25T08:36:26.541695+0000","last_became_peered":"2024-04-25T08:36:26.541695+0000","last_unstale":"2024-04-25T08:36:28.770345+0000","last_undegraded":"2024-04-25T08:36:28.770345+0000","last_fullsized":"2024-04-25T08:36:28.770345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:48.671766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771445+0000","last_change":"2024-04-25T08:36:27.556663+0000","last_active":"2024-04-25T08:36:28.771445+0000","last_peered":"2024-04-25T08:36:28.771445+0000","last_clean":"2024-04-25T08:36:28.771445+0000","last_became_active":"2024-04-25T08:36:27.556395+0000","last_became_peered":"2024-04-25T08:36:27.556395+0000","last_unstale":"2024-04-25T08:36:28.771445+0000","last_undegraded":"2024-04-25T08:36:28.771445+0000","last_fullsized":"2024-04-25T08:36:28.771445+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:49.451317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771599+0000","last_change":"2024-04-25T08:36:26.548435+0000","last_active":"2024-04-25T08:36:28.771599+0000","last_peered":"2024-04-25T08:36:28.771599+0000","last_clean":"2024-04-25T08:36:28.771599+0000","last_became_active":"2024-04-25T08:36:26.548141+0000","last_became_peered":"2024-04-25T08:36:26.548141+0000","last_unstale":"2024-04-25T08:36:28.771599+0000","last_undegraded":"2024-04-25T08:36:28.771599+0000","last_fullsized":"2024-04-25T08:36:28.771599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:21.183613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771831+0000","last_change":"2024-04-25T08:36:27.556348+0000","last_active":"2024-04-25T08:36:28.771831+0000","last_peered":"2024-04-25T08:36:28.771831+0000","last_clean":"2024-04-25T08:36:28.771831+0000","last_became_active":"2024-04-25T08:36:27.556007+0000","last_became_peered":"2024-04-25T08:36:27.556007+0000","last_unstale":"2024-04-25T08:36:28.771831+0000","last_undegraded":"2024-04-25T08:36:28.771831+0000","last_fullsized":"2024-04-25T08:36:28.771831+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:19:24.054014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771202+0000","last_change":"2024-04-25T08:36:26.544037+0000","last_active":"2024-04-25T08:36:28.771202+0000","last_peered":"2024-04-25T08:36:28.771202+0000","last_clean":"2024-04-25T08:36:28.771202+0000","last_became_active":"2024-04-25T08:36:26.543747+0000","last_became_peered":"2024-04-25T08:36:26.543747+0000","last_unstale":"2024-04-25T08:36:28.771202+0000","last_undegraded":"2024-04-25T08:36:28.771202+0000","last_fullsized":"2024-04-25T08:36:28.771202+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:59.896249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.553716+0000","last_change":"2024-04-25T08:36:27.546700+0000","last_active":"2024-04-25T08:36:28.553716+0000","last_peered":"2024-04-25T08:36:28.553716+0000","last_clean":"2024-04-25T08:36:28.553716+0000","last_became_active":"2024-04-25T08:36:27.546426+0000","last_became_peered":"2024-04-25T08:36:27.546426+0000","last_unstale":"2024-04-25T08:36:28.553716+0000","last_undegraded":"2024-04-25T08:36:28.553716+0000","last_fullsized":"2024-04-25T08:36:28.553716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:37.707312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771129+0000","last_change":"2024-04-25T08:36:26.543672+0000","last_active":"2024-04-25T08:36:28.771129+0000","last_peered":"2024-04-25T08:36:28.771129+0000","last_clean":"2024-04-25T08:36:28.771129+0000","last_became_active":"2024-04-25T08:36:26.543395+0000","last_became_peered":"2024-04-25T08:36:26.543395+0000","last_unstale":"2024-04-25T08:36:28.771129+0000","last_undegraded":"2024-04-25T08:36:28.771129+0000","last_fullsized":"2024-04-25T08:36:28.771129+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:53.340209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.553232+0000","last_change":"2024-04-25T08:36:27.553869+0000","last_active":"2024-04-25T08:36:28.553232+0000","last_peered":"2024-04-25T08:36:28.553232+0000","last_clean":"2024-04-25T08:36:28.553232+0000","last_became_active":"2024-04-25T08:36:27.553422+0000","last_became_peered":"2024-04-25T08:36:27.553422+0000","last_unstale":"2024-04-25T08:36:28.553232+0000","last_undegraded":"2024-04-25T08:36:28.553232+0000","last_fullsized":"2024-04-25T08:36:28.553232+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:41.842534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769830+0000","last_change":"2024-04-25T08:36:26.551656+0000","last_active":"2024-04-25T08:36:28.769830+0000","last_peered":"2024-04-25T08:36:28.769830+0000","last_clean":"2024-04-25T08:36:28.769830+0000","last_became_active":"2024-04-25T08:36:26.551356+0000","last_became_peered":"2024-04-25T08:36:26.551356+0000","last_unstale":"2024-04-25T08:36:28.769830+0000","last_undegraded":"2024-04-25T08:36:28.769830+0000","last_fullsized":"2024-04-25T08:36:28.769830+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:51.149994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552805+0000","last_change":"2024-04-25T08:36:27.556549+0000","last_active":"2024-04-25T08:36:28.552805+0000","last_peered":"2024-04-25T08:36:28.552805+0000","last_clean":"2024-04-25T08:36:28.552805+0000","last_became_active":"2024-04-25T08:36:27.556236+0000","last_became_peered":"2024-04-25T08:36:27.556236+0000","last_unstale":"2024-04-25T08:36:28.552805+0000","last_undegraded":"2024-04-25T08:36:28.552805+0000","last_fullsized":"2024-04-25T08:36:28.552805+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:27.148316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"82'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771823+0000","last_change":"2024-04-25T08:36:26.549899+0000","last_active":"2024-04-25T08:36:28.771823+0000","last_peered":"2024-04-25T08:36:28.771823+0000","last_clean":"2024-04-25T08:36:28.771823+0000","last_became_active":"2024-04-25T08:36:26.549599+0000","last_became_peered":"2024-04-25T08:36:26.549599+0000","last_unstale":"2024-04-25T08:36:28.771823+0000","last_undegraded":"2024-04-25T08:36:28.771823+0000","last_fullsized":"2024-04-25T08:36:28.771823+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:44.561270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552708+0000","last_change":"2024-04-25T08:36:27.562935+0000","last_active":"2024-04-25T08:36:28.552708+0000","last_peered":"2024-04-25T08:36:28.552708+0000","last_clean":"2024-04-25T08:36:28.552708+0000","last_became_active":"2024-04-25T08:36:27.562654+0000","last_became_peered":"2024-04-25T08:36:27.562654+0000","last_unstale":"2024-04-25T08:36:28.552708+0000","last_undegraded":"2024-04-25T08:36:28.552708+0000","last_fullsized":"2024-04-25T08:36:28.552708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:49.680126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551895+0000","last_change":"2024-04-25T08:36:26.551038+0000","last_active":"2024-04-25T08:36:28.551895+0000","last_peered":"2024-04-25T08:36:28.551895+0000","last_clean":"2024-04-25T08:36:28.551895+0000","last_became_active":"2024-04-25T08:36:26.550729+0000","last_became_peered":"2024-04-25T08:36:26.550729+0000","last_unstale":"2024-04-25T08:36:28.551895+0000","last_undegraded":"2024-04-25T08:36:28.551895+0000","last_fullsized":"2024-04-25T08:36:28.551895+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:59.896249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771422+0000","last_change":"2024-04-25T08:36:27.557375+0000","last_active":"2024-04-25T08:36:28.771422+0000","last_peered":"2024-04-25T08:36:28.771422+0000","last_clean":"2024-04-25T08:36:28.771422+0000","last_became_active":"2024-04-25T08:36:27.557093+0000","last_became_peered":"2024-04-25T08:36:27.557093+0000","last_unstale":"2024-04-25T08:36:28.771422+0000","last_undegraded":"2024-04-25T08:36:28.771422+0000","last_fullsized":"2024-04-25T08:36:28.771422+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:24.321649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551640+0000","last_change":"2024-04-25T08:36:26.548899+0000","last_active":"2024-04-25T08:36:28.551640+0000","last_peered":"2024-04-25T08:36:28.551640+0000","last_clean":"2024-04-25T08:36:28.551640+0000","last_became_active":"2024-04-25T08:36:26.548616+0000","last_became_peered":"2024-04-25T08:36:26.548616+0000","last_unstale":"2024-04-25T08:36:28.551640+0000","last_undegraded":"2024-04-25T08:36:28.551640+0000","last_fullsized":"2024-04-25T08:36:28.551640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:00.121065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552978+0000","last_change":"2024-04-25T08:36:27.556400+0000","last_active":"2024-04-25T08:36:28.552978+0000","last_peered":"2024-04-25T08:36:28.552978+0000","last_clean":"2024-04-25T08:36:28.552978+0000","last_became_active":"2024-04-25T08:36:27.556108+0000","last_became_peered":"2024-04-25T08:36:27.556108+0000","last_unstale":"2024-04-25T08:36:28.552978+0000","last_undegraded":"2024-04-25T08:36:28.552978+0000","last_fullsized":"2024-04-25T08:36:28.552978+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:52.158354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552932+0000","last_change":"2024-04-25T08:36:26.546019+0000","last_active":"2024-04-25T08:36:28.552932+0000","last_peered":"2024-04-25T08:36:28.552932+0000","last_clean":"2024-04-25T08:36:28.552932+0000","last_became_active":"2024-04-25T08:36:26.545702+0000","last_became_peered":"2024-04-25T08:36:26.545702+0000","last_unstale":"2024-04-25T08:36:28.552932+0000","last_undegraded":"2024-04-25T08:36:28.552932+0000","last_fullsized":"2024-04-25T08:36:28.552932+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:48.548532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.553366+0000","last_change":"2024-04-25T08:36:27.567217+0000","last_active":"2024-04-25T08:36:28.553366+0000","last_peered":"2024-04-25T08:36:28.553366+0000","last_clean":"2024-04-25T08:36:28.553366+0000","last_became_active":"2024-04-25T08:36:27.566962+0000","last_became_peered":"2024-04-25T08:36:27.566962+0000","last_unstale":"2024-04-25T08:36:28.553366+0000","last_undegraded":"2024-04-25T08:36:28.553366+0000","last_fullsized":"2024-04-25T08:36:28.553366+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:01.910886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770397+0000","last_change":"2024-04-25T08:36:26.551919+0000","last_active":"2024-04-25T08:36:28.770397+0000","last_peered":"2024-04-25T08:36:28.770397+0000","last_clean":"2024-04-25T08:36:28.770397+0000","last_became_active":"2024-04-25T08:36:26.550730+0000","last_became_peered":"2024-04-25T08:36:26.550730+0000","last_unstale":"2024-04-25T08:36:28.770397+0000","last_undegraded":"2024-04-25T08:36:28.770397+0000","last_fullsized":"2024-04-25T08:36:28.770397+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:59.917782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770703+0000","last_change":"2024-04-25T08:36:27.561685+0000","last_active":"2024-04-25T08:36:28.770703+0000","last_peered":"2024-04-25T08:36:28.770703+0000","last_clean":"2024-04-25T08:36:28.770703+0000","last_became_active":"2024-04-25T08:36:27.561373+0000","last_became_peered":"2024-04-25T08:36:27.561373+0000","last_unstale":"2024-04-25T08:36:28.770703+0000","last_undegraded":"2024-04-25T08:36:28.770703+0000","last_fullsized":"2024-04-25T08:36:28.770703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:52.158354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.552889+0000","last_change":"2024-04-25T08:36:26.546564+0000","last_active":"2024-04-25T08:36:28.552889+0000","last_peered":"2024-04-25T08:36:28.552889+0000","last_clean":"2024-04-25T08:36:28.552889+0000","last_became_active":"2024-04-25T08:36:26.546293+0000","last_became_peered":"2024-04-25T08:36:26.546293+0000","last_unstale":"2024-04-25T08:36:28.552889+0000","last_undegraded":"2024-04-25T08:36:28.552889+0000","last_fullsized":"2024-04-25T08:36:28.552889+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:12.641367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.770951+0000","last_change":"2024-04-25T08:36:27.554643+0000","last_active":"2024-04-25T08:36:28.770951+0000","last_peered":"2024-04-25T08:36:28.770951+0000","last_clean":"2024-04-25T08:36:28.770951+0000","last_became_active":"2024-04-25T08:36:27.554348+0000","last_became_peered":"2024-04-25T08:36:27.554348+0000","last_unstale":"2024-04-25T08:36:28.770951+0000","last_undegraded":"2024-04-25T08:36:28.770951+0000","last_fullsized":"2024-04-25T08:36:28.770951+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:38.907532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.769978+0000","last_change":"2024-04-25T08:36:26.546007+0000","last_active":"2024-04-25T08:36:28.769978+0000","last_peered":"2024-04-25T08:36:28.769978+0000","last_clean":"2024-04-25T08:36:28.769978+0000","last_became_active":"2024-04-25T08:36:26.545660+0000","last_became_peered":"2024-04-25T08:36:26.545660+0000","last_unstale":"2024-04-25T08:36:28.769978+0000","last_undegraded":"2024-04-25T08:36:28.769978+0000","last_fullsized":"2024-04-25T08:36:28.769978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:47.921469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551926+0000","last_change":"2024-04-25T08:36:27.560317+0000","last_active":"2024-04-25T08:36:28.551926+0000","last_peered":"2024-04-25T08:36:28.551926+0000","last_clean":"2024-04-25T08:36:28.551926+0000","last_became_active":"2024-04-25T08:36:27.560054+0000","last_became_peered":"2024-04-25T08:36:27.560054+0000","last_unstale":"2024-04-25T08:36:28.551926+0000","last_undegraded":"2024-04-25T08:36:28.551926+0000","last_fullsized":"2024-04-25T08:36:28.551926+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:26.526708+0000","last_clean_scrub_stamp":"2024-04-25T08:36:26.526708+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:03.734110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.771958+0000","last_change":"2024-04-25T08:36:26.544427+0000","last_active":"2024-04-25T08:36:28.771958+0000","last_peered":"2024-04-25T08:36:28.771958+0000","last_clean":"2024-04-25T08:36:28.771958+0000","last_became_active":"2024-04-25T08:36:26.544096+0000","last_became_peered":"2024-04-25T08:36:26.544096+0000","last_unstale":"2024-04-25T08:36:28.771958+0000","last_undegraded":"2024-04-25T08:36:28.771958+0000","last_fullsized":"2024-04-25T08:36:28.771958+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:36:25.518347+0000","last_clean_scrub_stamp":"2024-04-25T08:36:25.518347+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:44.561270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":287,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:36:28.551382+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:36:28.551382+0000","last_peered":"2024-04-25T08:36:28.551382+0000","last_clean":"2024-04-25T08:36:28.551382+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:36:28.551382+0000","last_undegraded":"2024-04-25T08:36:28.551382+0000","last_fullsized":"2024-04-25T08:36:28.551382+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46636,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325204,"statfs":{"total":96636764160,"available":96589008896,"internally_reserved":0,"allocated":1601536,"data_stored":1054169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40492,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331348,"statfs":{"total":96636764160,"available":96595300352,"internally_reserved":0,"allocated":1015808,"data_stored":465391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40492,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331348,"statfs":{"total":96636764160,"available":96595300352,"internally_reserved":0,"allocated":1007616,"data_stored":463813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46076,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325764,"statfs":{"total":96636764160,"available":96589582336,"internally_reserved":0,"allocated":1024000,"data_stored":465495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1007616,"data_stored":464361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40496,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331344,"statfs":{"total":96636764160,"available":96595296256,"internally_reserved":0,"allocated":1015808,"data_stored":464521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40496,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331344,"statfs":{"total":96636764160,"available":96595296256,"internally_reserved":0,"allocated":1011712,"data_stored":463891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54952,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316888,"statfs":{"total":96636764160,"available":96580493312,"internally_reserved":0,"allocated":1597440,"data_stored":1054147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:36:36.306 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:36:36.306 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:36:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.455+0000 7f4a8662a640 1 Processor -- start 2024-04-25T08:36:36.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.455+0000 7f4a8662a640 1 -- start start 2024-04-25T08:36:36.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.455+0000 7f4a8662a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a8006fc60 0x7f4a8015ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.455+0000 7f4a8662a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4a8006f460 0x7f4a80070040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.455+0000 7f4a8662a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a80070580 con 0x7f4a8009a730 2024-04-25T08:36:36.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.455+0000 7f4a8662a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a8015e3f0 con 0x7f4a8006f460 2024-04-25T08:36:36.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.455+0000 7f4a8662a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8015e6d0 con 0x7f4a8006fc60 2024-04-25T08:36:36.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.456+0000 7f4a7f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4a8006f460 0x7f4a80070040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.456+0000 7f4a7f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4a8006f460 0x7f4a80070040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47790/0 (socket says 172.21.15.38:47790) 2024-04-25T08:36:36.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.456+0000 7f4a7f7fe640 1 -- 172.21.15.38:0/2879841565 learned_addr learned my addr 172.21.15.38:0/2879841565 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:36.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.456+0000 7f4a7ffff640 1 --2- 172.21.15.38:0/2879841565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a8006fc60 0x7f4a8015ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.456+0000 7f4a7f7fe640 1 -- 172.21.15.38:0/2879841565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a8006fc60 msgr2=0x7f4a8015ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:36.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.456+0000 7f4a7f7fe640 1 --2- 172.21.15.38:0/2879841565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a8006fc60 0x7f4a8015ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.456+0000 7f4a7f7fe640 1 -- 172.21.15.38:0/2879841565 >> v1:172.21.15.107:6789/0 conn(0x7f4a8009a730 legacy=0x7f4a801616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.456+0000 7f4a7f7fe640 1 -- 172.21.15.38:0/2879841565 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8015e9b0 con 0x7f4a8006f460 2024-04-25T08:36:36.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.457+0000 7f4a7f7fe640 1 --2- 172.21.15.38:0/2879841565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a8006f460 0x7f4a80070040 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009870 tx=0x7f4a7402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b55fc3ce721032 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.457+0000 7f4a7effd640 1 -- 172.21.15.38:0/2879841565 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74037070 con 0x7f4a8006f460 2024-04-25T08:36:36.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.457+0000 7f4a7effd640 1 -- 172.21.15.38:0/2879841565 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a74032440 con 0x7f4a8006f460 2024-04-25T08:36:36.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.457+0000 7f4a7effd640 1 -- 172.21.15.38:0/2879841565 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74032810 con 0x7f4a8006f460 2024-04-25T08:36:36.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.457+0000 7f4a8662a640 1 -- 172.21.15.38:0/2879841565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a8006f460 msgr2=0x7f4a80070040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.457+0000 7f4a8662a640 1 --2- 172.21.15.38:0/2879841565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a8006f460 0x7f4a80070040 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f4a74009870 tx=0x7f4a7402a1c0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.457+0000 7f4a8662a640 1 -- 172.21.15.38:0/2879841565 shutdown_connections 2024-04-25T08:36:36.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.457+0000 7f4a8662a640 1 --2- 172.21.15.38:0/2879841565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a8006f460 0x7f4a80070040 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.457+0000 7f4a8662a640 1 --2- 172.21.15.38:0/2879841565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a8006fc60 0x7f4a8015ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.457+0000 7f4a8662a640 1 -- 172.21.15.38:0/2879841565 >> 172.21.15.38:0/2879841565 conn(0x7f4a800b3140 msgr2=0x7f4a800b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:36.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.458+0000 7f4a8662a640 1 -- 172.21.15.38:0/2879841565 shutdown_connections 2024-04-25T08:36:36.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.458+0000 7f4a8662a640 1 -- 172.21.15.38:0/2879841565 wait complete. 2024-04-25T08:36:36.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.459+0000 7f4a8662a640 1 Processor -- start 2024-04-25T08:36:36.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.459+0000 7f4a8662a640 1 -- start start 2024-04-25T08:36:36.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a8662a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a8006f460 0x7f4a80156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a8662a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a8006fc60 0x7f4a80150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a8662a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a8009a730 0x7f4a801f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a8662a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8009abc0 con 0x7f4a8006fc60 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a8662a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8015f1b0 con 0x7f4a8009a730 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a8662a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8009ad40 con 0x7f4a8006f460 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a8006fc60 0x7f4a80150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a8006fc60 0x7f4a80150460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47806/0 (socket says 172.21.15.38:47806) 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7f7fe640 1 -- 172.21.15.38:0/554896669 learned_addr learned my addr 172.21.15.38:0/554896669 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a84bc7640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a8009a730 0x7f4a801f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7ffff640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a8006f460 0x7f4a80156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7f7fe640 1 -- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a8006f460 msgr2=0x7f4a80156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7f7fe640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a8006f460 0x7f4a80156770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7f7fe640 1 -- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a8009a730 msgr2=0x7f4a801f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7f7fe640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a8009a730 0x7f4a801f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7f7fe640 1 -- 172.21.15.38:0/554896669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a801f7170 con 0x7f4a8006fc60 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7ffff640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a8006f460 0x7f4a80156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7f7fe640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a8006fc60 0x7f4a80150460 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f4a74000c00 tx=0x7f4a740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7cff9640 1 -- 172.21.15.38:0/554896669 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74041020 con 0x7f4a8006fc60 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7cff9640 1 -- 172.21.15.38:0/554896669 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a740306c0 con 0x7f4a8006fc60 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a7cff9640 1 -- 172.21.15.38:0/554896669 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a74037040 con 0x7f4a8006fc60 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.460+0000 7f4a8662a640 1 -- 172.21.15.38:0/554896669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a801f7450 con 0x7f4a8006fc60 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.461+0000 7f4a8662a640 1 -- 172.21.15.38:0/554896669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a801f7a10 con 0x7f4a8006fc60 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.461+0000 7f4a8662a640 1 -- 172.21.15.38:0/554896669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a80068e80 con 0x7f4a8006fc60 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.462+0000 7f4a7cff9640 1 -- 172.21.15.38:0/554896669 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a74044020 con 0x7f4a8006fc60 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.462+0000 7f4a7cff9640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4a6c06b4d0 0x7f4a6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.462+0000 7f4a7cff9640 1 -- 172.21.15.38:0/554896669 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a740aaa30 con 0x7f4a8006fc60 2024-04-25T08:36:36.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.463+0000 7f4a7ffff640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4a6c06b4d0 0x7f4a6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.463+0000 7f4a7ffff640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4a6c06b4d0 0x7f4a6c06d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000b9f0 tx=0x7f4a70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.467+0000 7f4a7cff9640 1 -- 172.21.15.38:0/554896669 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a7404b030 con 0x7f4a8006fc60 2024-04-25T08:36:36.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.715+0000 7f4a8662a640 1 -- 172.21.15.38:0/554896669 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4a8006fa40 con 0x7f4a8006fc60 2024-04-25T08:36:36.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.717+0000 7f4a7cff9640 1 -- 172.21.15.38:0/554896669 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4a740797b0 con 0x7f4a8006fc60 2024-04-25T08:36:36.711 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-04-25T08:36:36.712 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw== 2024-04-25T08:36:36.712 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-04-25T08:36:36.712 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-04-25T08:36:36.712 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:36:36.712 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-04-25T08:36:36.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.719+0000 7f4a8662a640 1 -- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4a6c06b4d0 msgr2=0x7f4a6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.719+0000 7f4a8662a640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4a6c06b4d0 0x7f4a6c06d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000b9f0 tx=0x7f4a70008040 comp rx=0 tx=0).stop 2024-04-25T08:36:36.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.719+0000 7f4a8662a640 1 -- 172.21.15.38:0/554896669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a8006fc60 msgr2=0x7f4a80150460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.719+0000 7f4a8662a640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a8006fc60 0x7f4a80150460 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f4a74000c00 tx=0x7f4a740046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.719+0000 7f4a8662a640 1 -- 172.21.15.38:0/554896669 shutdown_connections 2024-04-25T08:36:36.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.719+0000 7f4a8662a640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4a8009a730 0x7f4a801f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.719+0000 7f4a8662a640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a8006fc60 0x7f4a80150460 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.719+0000 7f4a8662a640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4a6c06b4d0 0x7f4a6c06d990 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.719+0000 7f4a8662a640 1 --2- 172.21.15.38:0/554896669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4a8006f460 0x7f4a80156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.719+0000 7f4a8662a640 1 -- 172.21.15.38:0/554896669 >> 172.21.15.38:0/554896669 conn(0x7f4a800b3140 msgr2=0x7f4a8006cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:36.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.719+0000 7f4a8662a640 1 -- 172.21.15.38:0/554896669 shutdown_connections 2024-04-25T08:36:36.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.719+0000 7f4a8662a640 1 -- 172.21.15.38:0/554896669 wait complete. 2024-04-25T08:36:36.725 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:36:36.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.831+0000 7fd90e838640 1 Processor -- start 2024-04-25T08:36:36.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.831+0000 7fd90e838640 1 -- start start 2024-04-25T08:36:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd90e838640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd90815ddf0 0x7fd908161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd90e838640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd9080b93f0 0x7fd9080d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd90e838640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9080b98a0 con 0x7fd9080b5710 2024-04-25T08:36:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd90e838640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd908161e00 con 0x7fd90815ddf0 2024-04-25T08:36:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd90e838640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9081620e0 con 0x7fd9080b93f0 2024-04-25T08:36:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd9077fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd90815ddf0 0x7fd908161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd9077fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd90815ddf0 0x7fd908161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47818/0 (socket says 172.21.15.38:47818) 2024-04-25T08:36:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd9077fe640 1 -- 172.21.15.38:0/1714339458 learned_addr learned my addr 172.21.15.38:0/1714339458 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd90cdd5640 1 --2- 172.21.15.38:0/1714339458 >> v2:172.21.15.107:3300/0 conn(0x7fd9080b93f0 0x7fd9080d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd9077fe640 1 -- 172.21.15.38:0/1714339458 >> v2:172.21.15.107:3300/0 conn(0x7fd9080b93f0 msgr2=0x7fd9080d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd9077fe640 1 --2- 172.21.15.38:0/1714339458 >> v2:172.21.15.107:3300/0 conn(0x7fd9080b93f0 0x7fd9080d2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd9077fe640 1 -- 172.21.15.38:0/1714339458 >> v1:172.21.15.107:6789/0 conn(0x7fd9080b5710 legacy=0x7fd9080b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.832+0000 7fd9077fe640 1 -- 172.21.15.38:0/1714339458 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9081623c0 con 0x7fd90815ddf0 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.833+0000 7fd9077fe640 1 --2- 172.21.15.38:0/1714339458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd90815ddf0 0x7fd908161700 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fd90808b240 tx=0x7fd8fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d704901f7d1806d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.833+0000 7fd906ffd640 1 -- 172.21.15.38:0/1714339458 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc037070 con 0x7fd90815ddf0 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.833+0000 7fd906ffd640 1 -- 172.21.15.38:0/1714339458 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8fc032440 con 0x7fd90815ddf0 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.833+0000 7fd906ffd640 1 -- 172.21.15.38:0/1714339458 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc032810 con 0x7fd90815ddf0 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.833+0000 7fd90e838640 1 -- 172.21.15.38:0/1714339458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd90815ddf0 msgr2=0x7fd908161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.833+0000 7fd90e838640 1 --2- 172.21.15.38:0/1714339458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd90815ddf0 0x7fd908161700 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fd90808b240 tx=0x7fd8fc02a270 comp rx=0 tx=0).stop 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.833+0000 7fd90e838640 1 -- 172.21.15.38:0/1714339458 shutdown_connections 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.833+0000 7fd90e838640 1 --2- 172.21.15.38:0/1714339458 >> v2:172.21.15.107:3300/0 conn(0x7fd9080b93f0 0x7fd9080d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.833+0000 7fd90e838640 1 --2- 172.21.15.38:0/1714339458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd90815ddf0 0x7fd908161700 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.833+0000 7fd90e838640 1 -- 172.21.15.38:0/1714339458 >> 172.21.15.38:0/1714339458 conn(0x7fd9080d07c0 msgr2=0x7fd9080d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.834+0000 7fd90e838640 1 -- 172.21.15.38:0/1714339458 shutdown_connections 2024-04-25T08:36:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.834+0000 7fd90e838640 1 -- 172.21.15.38:0/1714339458 wait complete. 2024-04-25T08:36:36.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.835+0000 7fd90e838640 1 Processor -- start 2024-04-25T08:36:36.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.835+0000 7fd90e838640 1 -- start start 2024-04-25T08:36:36.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90e838640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9080b5710 0x7fd908156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90e838640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9080b93f0 0x7fd9081f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90e838640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd90815ddf0 0x7fd90815f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90e838640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080d2c10 con 0x7fd90815ddf0 2024-04-25T08:36:36.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90e838640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080d2a90 con 0x7fd9080b5710 2024-04-25T08:36:36.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90e838640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080d2d90 con 0x7fd9080b93f0 2024-04-25T08:36:36.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90cdd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd90815ddf0 0x7fd90815f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90cdd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd90815ddf0 0x7fd90815f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47832/0 (socket says 172.21.15.38:47832) 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90cdd5640 1 -- 172.21.15.38:0/3798621395 learned_addr learned my addr 172.21.15.38:0/3798621395 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd9077fe640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9080b93f0 0x7fd9081f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd907fff640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9080b5710 0x7fd908156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90cdd5640 1 -- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9080b93f0 msgr2=0x7fd9081f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90cdd5640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9080b93f0 0x7fd9081f3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90cdd5640 1 -- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9080b5710 msgr2=0x7fd908156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90cdd5640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9080b5710 0x7fd908156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90cdd5640 1 -- 172.21.15.38:0/3798621395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9080cd0f0 con 0x7fd90815ddf0 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90cdd5640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd90815ddf0 0x7fd90815f730 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd900007e40 tx=0x7fd90000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd904ff9640 1 -- 172.21.15.38:0/3798621395 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900014620 con 0x7fd90815ddf0 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd904ff9640 1 -- 172.21.15.38:0/3798621395 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd900019070 con 0x7fd90815ddf0 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90e838640 1 -- 172.21.15.38:0/3798621395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9080cd3d0 con 0x7fd90815ddf0 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd904ff9640 1 -- 172.21.15.38:0/3798621395 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900014a20 con 0x7fd90815ddf0 2024-04-25T08:36:36.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.836+0000 7fd90e838640 1 -- 172.21.15.38:0/3798621395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9080c85a0 con 0x7fd90815ddf0 2024-04-25T08:36:36.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.838+0000 7fd90e838640 1 -- 172.21.15.38:0/3798621395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd908068e80 con 0x7fd90815ddf0 2024-04-25T08:36:36.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.838+0000 7fd904ff9640 1 -- 172.21.15.38:0/3798621395 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd900014bc0 con 0x7fd90815ddf0 2024-04-25T08:36:36.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.838+0000 7fd904ff9640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd8f406b4d0 0x7fd8f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:36.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.839+0000 7fd904ff9640 1 -- 172.21.15.38:0/3798621395 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd90008ac90 con 0x7fd90815ddf0 2024-04-25T08:36:36.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.839+0000 7fd907fff640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd8f406b4d0 0x7fd8f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:36.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.839+0000 7fd907fff640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd8f406b4d0 0x7fd8f406d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8001780 tx=0x7fd8f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:36.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:36.843+0000 7fd904ff9640 1 -- 172.21.15.38:0/3798621395 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd900059180 con 0x7fd90815ddf0 2024-04-25T08:36:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.061+0000 7fd90e838640 1 -- 172.21.15.38:0/3798621395 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9081614e0 con 0x7fd90815ddf0 2024-04-25T08:36:37.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.061+0000 7fd904ff9640 1 -- 172.21.15.38:0/3798621395 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fd90005d220 con 0x7fd90815ddf0 2024-04-25T08:36:37.056 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:36:37.056 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3781069672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3781069672},{"type":"v1","addr":"172.21.15.107:6837","nonce":3781069672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/624970524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":624970524},{"type":"v1","addr":"172.21.15.107:6836","nonce":624970524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7380,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1256978001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1256978001},{"type":"v1","addr":"172.21.15.38:6834","nonce":1256978001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:36:26.904065+0000","modified":"2024-04-25T08:36:29.920325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/380751658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":380751658},{"type":"v1","addr":"172.21.15.38:6835","nonce":380751658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:36:37.056 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 50 2024-04-25T08:36:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.063+0000 7fd90e838640 1 -- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd8f406b4d0 msgr2=0x7fd8f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.063+0000 7fd90e838640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd8f406b4d0 0x7fd8f406d990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8001780 tx=0x7fd8f8009040 comp rx=0 tx=0).stop 2024-04-25T08:36:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.063+0000 7fd90e838640 1 -- 172.21.15.38:0/3798621395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd90815ddf0 msgr2=0x7fd90815f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.063+0000 7fd90e838640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd90815ddf0 0x7fd90815f730 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd900007e40 tx=0x7fd90000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.063+0000 7fd90e838640 1 -- 172.21.15.38:0/3798621395 shutdown_connections 2024-04-25T08:36:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.063+0000 7fd90e838640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd90815ddf0 0x7fd90815f730 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.063+0000 7fd90e838640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd8f406b4d0 0x7fd8f406d990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.063+0000 7fd90e838640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9080b93f0 0x7fd9081f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.063+0000 7fd90e838640 1 --2- 172.21.15.38:0/3798621395 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9080b5710 0x7fd908156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.063+0000 7fd90e838640 1 -- 172.21.15.38:0/3798621395 >> 172.21.15.38:0/3798621395 conn(0x7fd9080d07c0 msgr2=0x7fd908073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.063+0000 7fd90e838640 1 -- 172.21.15.38:0/3798621395 shutdown_connections 2024-04-25T08:36:37.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.063+0000 7fd90e838640 1 -- 172.21.15.38:0/3798621395 wait complete. 2024-04-25T08:36:37.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:36:26.904065+0000', 'modified': '2024-04-25T08:36:29.920325+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7383}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7383': {'gid': 7383, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6835/380751658', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6833', 'nonce': 380751658}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 380751658}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:36:37.472 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:36:37.472 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:36:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.582+0000 7fab50f89640 1 Processor -- start 2024-04-25T08:36:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.583+0000 7fab50f89640 1 -- start start 2024-04-25T08:36:37.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.583+0000 7fab50f89640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab4c15db80 0x7fab4c1616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.583+0000 7fab50f89640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fab4c0b9600 0x7fab4c0d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab50f89640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fab4c0d0660 0x7fab4c0b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab50f89640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4c0d0ad0 con 0x7fab4c0d0660 2024-04-25T08:36:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab50f89640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4c1625f0 con 0x7fab4c0b9600 2024-04-25T08:36:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab50f89640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c162030 con 0x7fab4c15db80 2024-04-25T08:36:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab4ad9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fab4c0d0660 0x7fab4c0b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab4ad9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fab4c0d0660 0x7fab4c0b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47834/0 (socket says 172.21.15.38:47834) 2024-04-25T08:36:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab4ad9d640 1 -- 172.21.15.38:0/799061631 learned_addr learned my addr 172.21.15.38:0/799061631 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab4ad9d640 1 -- 172.21.15.38:0/799061631 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab4c15db80 msgr2=0x7fab4c1616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab4a59c640 1 --2- 172.21.15.38:0/799061631 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab4c15db80 0x7fab4c1616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab49d9b640 1 --2- 172.21.15.38:0/799061631 >> v2:172.21.15.107:3300/0 conn(0x7fab4c0b9600 0x7fab4c0d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/799061631 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab4c15db80 0x7fab4c1616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab4ad9d640 1 -- 172.21.15.38:0/799061631 >> v2:172.21.15.107:3300/0 conn(0x7fab4c0b9600 msgr2=0x7fab4c0d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/799061631 >> v2:172.21.15.107:3300/0 conn(0x7fab4c0b9600 0x7fab4c0d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab4ad9d640 1 -- 172.21.15.38:0/799061631 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c162310 con 0x7fab4c0d0660 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab4a59c640 1 --2- 172.21.15.38:0/799061631 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab4c15db80 0x7fab4c1616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.584+0000 7fab49d9b640 1 --2- 172.21.15.38:0/799061631 >> v2:172.21.15.107:3300/0 conn(0x7fab4c0b9600 0x7fab4c0d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/799061631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab4c0d0660 0x7fab4c0b52f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0b63c0 tx=0x7fab3c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20092414701b4df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab4959a640 1 -- 172.21.15.38:0/799061631 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c037070 con 0x7fab4c0d0660 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab4959a640 1 -- 172.21.15.38:0/799061631 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c032440 con 0x7fab4c0d0660 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab4959a640 1 -- 172.21.15.38:0/799061631 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c032810 con 0x7fab4c0d0660 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab50f89640 1 -- 172.21.15.38:0/799061631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab4c0d0660 msgr2=0x7fab4c0b52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab50f89640 1 --2- 172.21.15.38:0/799061631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab4c0d0660 0x7fab4c0b52f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fab4c0b63c0 tx=0x7fab3c02a140 comp rx=0 tx=0).stop 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab50f89640 1 -- 172.21.15.38:0/799061631 shutdown_connections 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab50f89640 1 --2- 172.21.15.38:0/799061631 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab4c0d0660 0x7fab4c0b52f0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab50f89640 1 --2- 172.21.15.38:0/799061631 >> v2:172.21.15.107:3300/0 conn(0x7fab4c0b9600 0x7fab4c0d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab50f89640 1 --2- 172.21.15.38:0/799061631 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab4c15db80 0x7fab4c1616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab50f89640 1 -- 172.21.15.38:0/799061631 >> 172.21.15.38:0/799061631 conn(0x7fab4c0668f0 msgr2=0x7fab4c066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:37.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab50f89640 1 -- 172.21.15.38:0/799061631 shutdown_connections 2024-04-25T08:36:37.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.585+0000 7fab50f89640 1 -- 172.21.15.38:0/799061631 wait complete. 2024-04-25T08:36:37.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.587+0000 7fab50f89640 1 Processor -- start 2024-04-25T08:36:37.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.587+0000 7fab50f89640 1 -- start start 2024-04-25T08:36:37.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.587+0000 7fab50f89640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab4c0b9600 0x7fab4c1f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.587+0000 7fab50f89640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab4c0d0660 0x7fab4c0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.587+0000 7fab50f89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab4c15db80 0x7fab4c0cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.587+0000 7fab50f89640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c161c20 con 0x7fab4c15db80 2024-04-25T08:36:37.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.587+0000 7fab50f89640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c15df60 con 0x7fab4c0d0660 2024-04-25T08:36:37.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.587+0000 7fab50f89640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c161da0 con 0x7fab4c0b9600 2024-04-25T08:36:37.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab4ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab4c15db80 0x7fab4c0cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab4ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab4c15db80 0x7fab4c0cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47842/0 (socket says 172.21.15.38:47842) 2024-04-25T08:36:37.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab4ad9d640 1 -- 172.21.15.38:0/789022310 learned_addr learned my addr 172.21.15.38:0/789022310 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:37.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab4a59c640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab4c0b9600 0x7fab4c1f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab49d9b640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab4c0d0660 0x7fab4c0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab4ad9d640 1 -- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab4c0b9600 msgr2=0x7fab4c1f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab4c0b9600 0x7fab4c1f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab4ad9d640 1 -- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab4c0d0660 msgr2=0x7fab4c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab4c0d0660 0x7fab4c0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab4ad9d640 1 -- 172.21.15.38:0/789022310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c15f330 con 0x7fab4c15db80 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab49d9b640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab4c0d0660 0x7fab4c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab4c15db80 0x7fab4c0cac50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0090b0 tx=0x7fab3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab42ffd640 1 -- 172.21.15.38:0/789022310 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c041070 con 0x7fab4c15db80 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab42ffd640 1 -- 172.21.15.38:0/789022310 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c03c890 con 0x7fab4c15db80 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab50f89640 1 -- 172.21.15.38:0/789022310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4c160a10 con 0x7fab4c15db80 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab42ffd640 1 -- 172.21.15.38:0/789022310 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c037040 con 0x7fab4c15db80 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.588+0000 7fab50f89640 1 -- 172.21.15.38:0/789022310 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4c160fd0 con 0x7fab4c15db80 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.590+0000 7fab42ffd640 1 -- 172.21.15.38:0/789022310 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab3c044070 con 0x7fab4c15db80 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.590+0000 7fab42ffd640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fab2c06b4d0 0x7fab2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.590+0000 7fab42ffd640 1 -- 172.21.15.38:0/789022310 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab3c0aaff0 con 0x7fab4c15db80 2024-04-25T08:36:37.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.591+0000 7fab4a59c640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fab2c06b4d0 0x7fab2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.591+0000 7fab4a59c640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fab2c06b4d0 0x7fab2c06d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fab440075b0 tx=0x7fab44007540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.591+0000 7fab53d6c740 1 Processor -- start 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.592+0000 7fab53d6c740 1 -- start start 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.592+0000 7fab53d6c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b59f230 0x56264b59f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.592+0000 7fab53d6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5bcad0 0x56264b5bd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.592+0000 7fab53d6c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b574f10 0x56264b5bb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.592+0000 7fab53d6c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264b4d7a10 con 0x56264b574f10 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.592+0000 7fab53d6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264b480b30 con 0x56264b5bcad0 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.592+0000 7fab53d6c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56264b4d3c50 con 0x56264b59f230 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.592+0000 7fab4ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b574f10 0x56264b5bb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.592+0000 7fab4ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b574f10 0x56264b5bb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47858/0 (socket says 172.21.15.38:47858) 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.592+0000 7fab4ad9d640 1 -- 172.21.15.38:0/1640319917 learned_addr learned my addr 172.21.15.38:0/1640319917 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.592+0000 7fab4a59c640 1 --2- 172.21.15.38:0/1640319917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b59f230 0x56264b59f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.592+0000 7fab49d9b640 1 --2- 172.21.15.38:0/1640319917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5bcad0 0x56264b5bd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.593+0000 7fab4ad9d640 1 -- 172.21.15.38:0/1640319917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b59f230 msgr2=0x56264b59f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.593+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/1640319917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b59f230 0x56264b59f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.593+0000 7fab4ad9d640 1 -- 172.21.15.38:0/1640319917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5bcad0 msgr2=0x56264b5bd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.593+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/1640319917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5bcad0 0x56264b5bd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.593+0000 7fab4ad9d640 1 -- 172.21.15.38:0/1640319917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56264b495e10 con 0x56264b574f10 2024-04-25T08:36:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.593+0000 7fab49d9b640 1 --2- 172.21.15.38:0/1640319917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5bcad0 0x56264b5bd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.593+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/1640319917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b574f10 0x56264b5bb190 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0795b0 tx=0x7fab3c078f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1df10c0a80d73ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.593+0000 7fab2bfff640 1 -- 172.21.15.38:0/1640319917 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c04b040 con 0x56264b574f10 2024-04-25T08:36:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.593+0000 7fab2bfff640 1 -- 172.21.15.38:0/1640319917 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c07d770 con 0x56264b574f10 2024-04-25T08:36:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.593+0000 7fab2bfff640 1 -- 172.21.15.38:0/1640319917 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c081d30 con 0x56264b574f10 2024-04-25T08:36:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.594+0000 7fab53d6c740 1 -- 172.21.15.38:0/1640319917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b574f10 msgr2=0x56264b5bb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.594+0000 7fab53d6c740 1 --2- 172.21.15.38:0/1640319917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b574f10 0x56264b5bb190 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0795b0 tx=0x7fab3c078f80 comp rx=0 tx=0).stop 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.594+0000 7fab53d6c740 1 -- 172.21.15.38:0/1640319917 shutdown_connections 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.594+0000 7fab53d6c740 1 --2- 172.21.15.38:0/1640319917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b574f10 0x56264b5bb190 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.594+0000 7fab53d6c740 1 --2- 172.21.15.38:0/1640319917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5bcad0 0x56264b5bd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.594+0000 7fab53d6c740 1 --2- 172.21.15.38:0/1640319917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b59f230 0x56264b59f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.594+0000 7fab53d6c740 1 -- 172.21.15.38:0/1640319917 >> 172.21.15.38:0/1640319917 conn(0x56264b571840 msgr2=0x56264b5ba040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.594+0000 7fab53d6c740 1 -- 172.21.15.38:0/1640319917 shutdown_connections 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.594+0000 7fab53d6c740 1 -- 172.21.15.38:0/1640319917 wait complete. 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.594+0000 7fab53d6c740 1 -- start start 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.594+0000 7fab53d6c740 1 Processor -- start 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.595+0000 7fab53d6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b59f230 0x56264b5b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.595+0000 7fab53d6c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b5bcad0 0x56264b5909b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.595+0000 7fab53d6c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5a1dd0 0x56264b5bb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.595+0000 7fab53d6c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264b480b30 con 0x56264b5bcad0 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.595+0000 7fab53d6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264b4940c0 con 0x56264b59f230 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.595+0000 7fab53d6c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56264b4d7a10 con 0x56264b5a1dd0 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.595+0000 7fab53d6c740 10 client.0 _ll_register_callbacks cb 0x56264b487ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.595+0000 7fab49d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b5bcad0 0x56264b5909b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.595+0000 7fab49d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b5bcad0 0x56264b5909b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47870/0 (socket says 172.21.15.38:47870) 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.595+0000 7fab49d9b640 1 -- 172.21.15.38:0/3981613427 learned_addr learned my addr 172.21.15.38:0/3981613427 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.596+0000 7fab4a59c640 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b59f230 0x56264b5b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.596+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5a1dd0 0x56264b5bb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.596+0000 7fab49d9b640 1 -- 172.21.15.38:0/3981613427 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5a1dd0 msgr2=0x56264b5bb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.596+0000 7fab49d9b640 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5a1dd0 0x56264b5bb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.596+0000 7fab49d9b640 1 -- 172.21.15.38:0/3981613427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b59f230 msgr2=0x56264b5b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.596+0000 7fab49d9b640 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b59f230 0x56264b5b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.596+0000 7fab49d9b640 1 -- 172.21.15.38:0/3981613427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56264b1ec2d0 con 0x56264b5bcad0 2024-04-25T08:36:37.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.596+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5a1dd0 0x56264b5bb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:37.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.596+0000 7fab4a59c640 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b59f230 0x56264b5b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:37.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.596+0000 7fab49d9b640 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b5bcad0 0x56264b5909b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fab3400f540 tx=0x7fab3400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9de1b264256c68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.596+0000 7fab297fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:36:37.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.602+0000 7fab297fa640 1 -- 172.21.15.38:0/3981613427 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34019020 con 0x56264b5bcad0 2024-04-25T08:36:37.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.602+0000 7fab297fa640 1 -- 172.21.15.38:0/3981613427 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab34015880 con 0x56264b5bcad0 2024-04-25T08:36:37.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.602+0000 7fab53d6c740 1 -- 172.21.15.38:0/3981613427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab3c07d770 con 0x56264b5bcad0 2024-04-25T08:36:37.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab297fa640 1 -- 172.21.15.38:0/3981613427 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab34023050 con 0x56264b5bcad0 2024-04-25T08:36:37.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab53d6c740 10 client.7542 fetch_fsmap learned FSMap version 50 2024-04-25T08:36:37.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab53d6c740 1 -- 172.21.15.38:0/3981613427 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x56264b2b66b0 con 0x56264b5bcad0 2024-04-25T08:36:37.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab297fa640 1 -- 172.21.15.38:0/3981613427 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c07d770 con 0x56264b5bcad0 2024-04-25T08:36:37.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab297fa640 1 -- 172.21.15.38:0/3981613427 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fab34026020 con 0x56264b5bcad0 2024-04-25T08:36:37.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab53d6c740 10 client.7542 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T08:36:37.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab53d6c740 10 client.7542 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:36:37.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab53d6c740 20 client.7542 populate_metadata read hostname 'smithi038' 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab53d6c740 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x56264b5b5370 0x56264b5b5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab53d6c740 4 client.7542 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab53d6c740 1 -- 172.21.15.38:0/3981613427 --> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56264b536920 con 0x56264b5b5370 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab4a59c640 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x56264b5b5370 0x56264b5b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab297fa640 10 client.7542 ms_handle_connect on v2:172.21.15.38:6833/380751658 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.603+0000 7fab4a59c640 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x56264b5b5370 0x56264b5b5750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8550855ebd8b50f server_cookie=dd4781838df56fa0 in_seq=0 out_seq=0 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab297fa640 1 -- 172.21.15.38:0/3981613427 <== mds.0 v2:172.21.15.38:6833/380751658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56264b536920 con 0x56264b5b5370 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab297fa640 10 client.7542 handle_command_reply: tid=0 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab53d6c740 1 client.7542 shutdown 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab53d6c740 1 -- 172.21.15.38:0/3981613427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b5bcad0 msgr2=0x56264b5909b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab53d6c740 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b5bcad0 0x56264b5909b0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fab3400f540 tx=0x7fab3400fa00 comp rx=0 tx=0).stop 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab53d6c740 1 -- 172.21.15.38:0/3981613427 shutdown_connections 2024-04-25T08:36:37.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab53d6c740 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x56264b5b5370 0x56264b5b5750 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab53d6c740 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5a1dd0 0x56264b5bb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab53d6c740 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b5bcad0 0x56264b5909b0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab53d6c740 1 --2- 172.21.15.38:0/3981613427 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b59f230 0x56264b5b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab297fa640 0 client.7542 ms_handle_reset on v2:172.21.15.38:6833/380751658 2024-04-25T08:36:37.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab53d6c740 1 -- 172.21.15.38:0/3981613427 >> 172.21.15.38:0/3981613427 conn(0x56264b574f10 msgr2=0x56264b5bde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:37.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab53d6c740 1 -- 172.21.15.38:0/3981613427 shutdown_connections 2024-04-25T08:36:37.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.604+0000 7fab53d6c740 1 -- 172.21.15.38:0/3981613427 wait complete. 2024-04-25T08:36:37.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.605+0000 7fab53d6c740 20 client.7542 trim_cache size 0 max 16384 2024-04-25T08:36:37.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.620+0000 7fab53d6c740 1 Processor -- start 2024-04-25T08:36:37.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.621+0000 7fab53d6c740 1 -- start start 2024-04-25T08:36:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.621+0000 7fab53d6c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 0x56264b5b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.621+0000 7fab53d6c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5a1dd0 0x56264b5d31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.621+0000 7fab53d6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5bcad0 0x56264b5d3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.621+0000 7fab53d6c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c1f3d20 con 0x56264b59f230 2024-04-25T08:36:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.621+0000 7fab53d6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c159480 con 0x56264b5bcad0 2024-04-25T08:36:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.621+0000 7fab53d6c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c159a50 con 0x56264b5a1dd0 2024-04-25T08:36:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.621+0000 7fab4a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 0x56264b5b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.621+0000 7fab4a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 0x56264b5b28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47872/0 (socket says 172.21.15.38:47872) 2024-04-25T08:36:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.621+0000 7fab4a59c640 1 -- 172.21.15.38:0/65123691 learned_addr learned my addr 172.21.15.38:0/65123691 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.622+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/65123691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5bcad0 0x56264b5d3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.622+0000 7fab49d9b640 1 --2- 172.21.15.38:0/65123691 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5a1dd0 0x56264b5d31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.622+0000 7fab4a59c640 1 -- 172.21.15.38:0/65123691 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5a1dd0 msgr2=0x56264b5d31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.622+0000 7fab4a59c640 1 --2- 172.21.15.38:0/65123691 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5a1dd0 0x56264b5d31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.622+0000 7fab4a59c640 1 -- 172.21.15.38:0/65123691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5bcad0 msgr2=0x56264b5d3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.622+0000 7fab4a59c640 1 --2- 172.21.15.38:0/65123691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5bcad0 0x56264b5d3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.622+0000 7fab4a59c640 1 -- 172.21.15.38:0/65123691 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56264b5bc580 con 0x56264b59f230 2024-04-25T08:36:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.622+0000 7fab4a59c640 1 --2- 172.21.15.38:0/65123691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 0x56264b5b28e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fab4400edb0 tx=0x7fab44015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb1daafe453a1b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.622+0000 7fab2b7fe640 1 -- 172.21.15.38:0/65123691 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4402c070 con 0x56264b59f230 2024-04-25T08:36:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.622+0000 7fab2b7fe640 1 -- 172.21.15.38:0/65123691 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab44027230 con 0x56264b59f230 2024-04-25T08:36:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.622+0000 7fab2b7fe640 1 -- 172.21.15.38:0/65123691 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab440275e0 con 0x56264b59f230 2024-04-25T08:36:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.623+0000 7fab53d6c740 1 -- 172.21.15.38:0/65123691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 msgr2=0x56264b5b28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.623+0000 7fab53d6c740 1 --2- 172.21.15.38:0/65123691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 0x56264b5b28e0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fab4400edb0 tx=0x7fab44015000 comp rx=0 tx=0).stop 2024-04-25T08:36:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.623+0000 7fab53d6c740 1 -- 172.21.15.38:0/65123691 shutdown_connections 2024-04-25T08:36:37.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.623+0000 7fab53d6c740 1 --2- 172.21.15.38:0/65123691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5bcad0 0x56264b5d3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.623+0000 7fab53d6c740 1 --2- 172.21.15.38:0/65123691 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5a1dd0 0x56264b5d31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.623+0000 7fab53d6c740 1 --2- 172.21.15.38:0/65123691 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 0x56264b5b28e0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.623+0000 7fab53d6c740 1 -- 172.21.15.38:0/65123691 >> 172.21.15.38:0/65123691 conn(0x56264b574f10 msgr2=0x56264b5dc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:37.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.623+0000 7fab53d6c740 1 -- 172.21.15.38:0/65123691 shutdown_connections 2024-04-25T08:36:37.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.623+0000 7fab53d6c740 1 -- 172.21.15.38:0/65123691 wait complete. 2024-04-25T08:36:37.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.623+0000 7fab53d6c740 1 -- start start 2024-04-25T08:36:37.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.623+0000 7fab53d6c740 1 Processor -- start 2024-04-25T08:36:37.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.624+0000 7fab53d6c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 0x56264b5b7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.624+0000 7fab53d6c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5a1dd0 0x56264b6245d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.624+0000 7fab53d6c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5bcad0 0x56264b5b9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.624+0000 7fab53d6c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264b5a03c0 con 0x56264b59f230 2024-04-25T08:36:37.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.624+0000 7fab53d6c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56264b5b2570 con 0x56264b5a1dd0 2024-04-25T08:36:37.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.624+0000 7fab53d6c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56264b5a0900 con 0x56264b5bcad0 2024-04-25T08:36:37.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.624+0000 7fab53d6c740 10 client.0 _ll_register_callbacks cb 0x56264b56fc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:36:37.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.624+0000 7fab4a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 0x56264b5b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.624+0000 7fab4a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 0x56264b5b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47886/0 (socket says 172.21.15.38:47886) 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.624+0000 7fab4a59c640 1 -- 172.21.15.38:0/2071315775 learned_addr learned my addr 172.21.15.38:0/2071315775 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.624+0000 7fab4ad9d640 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5bcad0 0x56264b5b9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.624+0000 7fab49d9b640 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5a1dd0 0x56264b6245d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab4a59c640 1 -- 172.21.15.38:0/2071315775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5bcad0 msgr2=0x56264b5b9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab4a59c640 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5bcad0 0x56264b5b9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab4a59c640 1 -- 172.21.15.38:0/2071315775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5a1dd0 msgr2=0x56264b6245d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab4a59c640 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5a1dd0 0x56264b6245d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab4a59c640 1 -- 172.21.15.38:0/2071315775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56264b58f910 con 0x56264b59f230 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab49d9b640 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5a1dd0 0x56264b6245d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab4a59c640 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 0x56264b5b7760 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fab4400c910 tx=0x7fab44004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fb1a272cfb966c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab297fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab297fa640 1 -- 172.21.15.38:0/2071315775 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44034070 con 0x56264b59f230 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab297fa640 1 -- 172.21.15.38:0/2071315775 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab440276a0 con 0x56264b59f230 2024-04-25T08:36:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab53d6c740 1 -- 172.21.15.38:0/2071315775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab44027230 con 0x56264b59f230 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab297fa640 1 -- 172.21.15.38:0/2071315775 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab4402c040 con 0x56264b59f230 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab53d6c740 10 client.7548 fetch_fsmap learned FSMap version 50 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab53d6c740 1 -- 172.21.15.38:0/2071315775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x56264b5a1ac0 con 0x56264b59f230 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab297fa640 1 -- 172.21.15.38:0/2071315775 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab44027230 con 0x56264b59f230 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab297fa640 1 -- 172.21.15.38:0/2071315775 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fab4402f050 con 0x56264b59f230 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab53d6c740 10 client.7548 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab53d6c740 10 client.7548 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab53d6c740 20 client.7548 populate_metadata read hostname 'smithi038' 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab53d6c740 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x56264b5af9d0 0x56264b5afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab53d6c740 4 client.7548 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.625+0000 7fab53d6c740 1 -- 172.21.15.38:0/2071315775 --> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56264b4e4f50 con 0x56264b5af9d0 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab49d9b640 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x56264b5af9d0 0x56264b5afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab49d9b640 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x56264b5af9d0 0x56264b5afdb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267bbcfc717bc99 server_cookie=e105a5f96f941ded in_seq=0 out_seq=0 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab297fa640 10 client.7548 ms_handle_connect on v2:172.21.15.38:6833/380751658 2024-04-25T08:36:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab297fa640 1 -- 172.21.15.38:0/2071315775 <== mds.0 v2:172.21.15.38:6833/380751658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56264b4e4f50 con 0x56264b5af9d0 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab297fa640 10 client.7548 handle_command_reply: tid=0 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab53d6c740 1 client.7548 shutdown 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab53d6c740 1 -- 172.21.15.38:0/2071315775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 msgr2=0x56264b5b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab53d6c740 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 0x56264b5b7760 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fab4400c910 tx=0x7fab44004ab0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab53d6c740 1 -- 172.21.15.38:0/2071315775 shutdown_connections 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab53d6c740 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x56264b5af9d0 0x56264b5afdb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab53d6c740 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56264b5bcad0 0x56264b5b9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab53d6c740 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56264b5a1dd0 0x56264b6245d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab53d6c740 1 --2- 172.21.15.38:0/2071315775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56264b59f230 0x56264b5b7760 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab297fa640 0 client.7548 ms_handle_reset on v2:172.21.15.38:6833/380751658 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab53d6c740 1 -- 172.21.15.38:0/2071315775 >> 172.21.15.38:0/2071315775 conn(0x56264b574f10 msgr2=0x56264b5cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab53d6c740 1 -- 172.21.15.38:0/2071315775 shutdown_connections 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab53d6c740 1 -- 172.21.15.38:0/2071315775 wait complete. 2024-04-25T08:36:37.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.626+0000 7fab53d6c740 20 client.7548 trim_cache size 0 max 16384 2024-04-25T08:36:37.629 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:36:37.629 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:36:37.629 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:36:37.629 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 7383, 2024-04-25T08:36:37.629 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:36:37.629 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:36:37.629 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:36:37.629 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 10.081232107, 2024-04-25T08:36:37.629 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 50, 2024-04-25T08:36:37.629 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 82, 2024-04-25T08:36:37.629 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 82, 2024-04-25T08:36:37.629 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 15.111136976999999 2024-04-25T08:36:37.630 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:36:37.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.627+0000 7fab50f89640 1 -- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fab2c06b4d0 msgr2=0x7fab2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.627+0000 7fab50f89640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fab2c06b4d0 0x7fab2c06d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fab440075b0 tx=0x7fab44007540 comp rx=0 tx=0).stop 2024-04-25T08:36:37.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.627+0000 7fab50f89640 1 -- 172.21.15.38:0/789022310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab4c15db80 msgr2=0x7fab4c0cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.627+0000 7fab50f89640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab4c15db80 0x7fab4c0cac50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0090b0 tx=0x7fab3c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.627+0000 7fab50f89640 1 -- 172.21.15.38:0/789022310 shutdown_connections 2024-04-25T08:36:37.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.627+0000 7fab50f89640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab4c15db80 0x7fab4c0cac50 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.627+0000 7fab50f89640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fab4c0d0660 0x7fab4c0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.627+0000 7fab50f89640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fab2c06b4d0 0x7fab2c06d990 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.627+0000 7fab50f89640 1 --2- 172.21.15.38:0/789022310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fab4c0b9600 0x7fab4c1f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.627+0000 7fab50f89640 1 -- 172.21.15.38:0/789022310 >> 172.21.15.38:0/789022310 conn(0x7fab4c0668f0 msgr2=0x7fab4c0b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:37.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.627+0000 7fab50f89640 1 -- 172.21.15.38:0/789022310 shutdown_connections 2024-04-25T08:36:37.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.627+0000 7fab50f89640 1 -- 172.21.15.38:0/789022310 wait complete. 2024-04-25T08:36:37.632 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:36:37.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.739+0000 7f9ed9846640 1 Processor -- start 2024-04-25T08:36:37.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.739+0000 7f9ed9846640 1 -- start start 2024-04-25T08:36:37.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.740+0000 7f9ed9846640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9ed40ceca0 0x7f9ed40d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.740+0000 7f9ed9846640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40d3de0 0x7f9ed40b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.740+0000 7f9ed9846640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed40d1b60 con 0x7f9ed415da90 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.740+0000 7f9ed9846640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed40d4250 con 0x7f9ed40ceca0 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.740+0000 7f9ed9846640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40d2bc0 con 0x7f9ed40d3de0 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.740+0000 7f9ed27fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9ed40ceca0 0x7f9ed40d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.740+0000 7f9ed27fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9ed40ceca0 0x7f9ed40d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47894/0 (socket says 172.21.15.38:47894) 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.740+0000 7f9ed27fc640 1 -- 172.21.15.38:0/1992872324 learned_addr learned my addr 172.21.15.38:0/1992872324 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.741+0000 7f9ed37fe640 1 --2- 172.21.15.38:0/1992872324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40d3de0 0x7f9ed40b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.741+0000 7f9ed27fc640 1 -- 172.21.15.38:0/1992872324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40d3de0 msgr2=0x7f9ed40b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.741+0000 7f9ed27fc640 1 --2- 172.21.15.38:0/1992872324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40d3de0 0x7f9ed40b69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.741+0000 7f9ed27fc640 1 -- 172.21.15.38:0/1992872324 >> v1:172.21.15.107:6789/0 conn(0x7f9ed415da90 legacy=0x7f9ed40d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.741+0000 7f9ed27fc640 1 -- 172.21.15.38:0/1992872324 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40d3680 con 0x7f9ed40ceca0 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.741+0000 7f9ed27fc640 1 --2- 172.21.15.38:0/1992872324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ed40ceca0 0x7f9ed40d38a0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4074800 tx=0x7f9ebc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0b320f01dcf15c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.741+0000 7f9ed1ffb640 1 -- 172.21.15.38:0/1992872324 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc037070 con 0x7f9ed40ceca0 2024-04-25T08:36:37.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.742+0000 7f9ed1ffb640 1 -- 172.21.15.38:0/1992872324 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc032440 con 0x7f9ed40ceca0 2024-04-25T08:36:37.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.742+0000 7f9ed1ffb640 1 -- 172.21.15.38:0/1992872324 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc032810 con 0x7f9ed40ceca0 2024-04-25T08:36:37.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.742+0000 7f9ed9846640 1 -- 172.21.15.38:0/1992872324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ed40ceca0 msgr2=0x7f9ed40d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.742+0000 7f9ed9846640 1 --2- 172.21.15.38:0/1992872324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ed40ceca0 0x7f9ed40d38a0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4074800 tx=0x7f9ebc02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.742+0000 7f9ed9846640 1 -- 172.21.15.38:0/1992872324 shutdown_connections 2024-04-25T08:36:37.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.742+0000 7f9ed9846640 1 --2- 172.21.15.38:0/1992872324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40d3de0 0x7f9ed40b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.742+0000 7f9ed9846640 1 --2- 172.21.15.38:0/1992872324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ed40ceca0 0x7f9ed40d38a0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.742+0000 7f9ed9846640 1 -- 172.21.15.38:0/1992872324 >> 172.21.15.38:0/1992872324 conn(0x7f9ed406bed0 msgr2=0x7f9ed406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:37.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.742+0000 7f9ed9846640 1 -- 172.21.15.38:0/1992872324 shutdown_connections 2024-04-25T08:36:37.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.742+0000 7f9ed9846640 1 -- 172.21.15.38:0/1992872324 wait complete. 2024-04-25T08:36:37.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.743+0000 7f9ed9846640 1 Processor -- start 2024-04-25T08:36:37.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.744+0000 7f9ed9846640 1 -- start start 2024-04-25T08:36:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.744+0000 7f9ed9846640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ed40d3de0 0x7f9ed40cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.744+0000 7f9ed9846640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed415da90 0x7f9ed40cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.744+0000 7f9ed9846640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40cbe70 0x7f9ed40c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.744+0000 7f9ed9846640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40d23c0 con 0x7f9ed40d3de0 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.744+0000 7f9ed9846640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40d20e0 con 0x7f9ed415da90 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.744+0000 7f9ed9846640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed415de70 con 0x7f9ed40cbe70 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.744+0000 7f9ed2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ed40d3de0 0x7f9ed40cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.744+0000 7f9ed2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ed40d3de0 0x7f9ed40cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47896/0 (socket says 172.21.15.38:47896) 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.744+0000 7f9ed2ffd640 1 -- 172.21.15.38:0/339143728 learned_addr learned my addr 172.21.15.38:0/339143728 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ed37fe640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40cbe70 0x7f9ed40c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ed27fc640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed415da90 0x7f9ed40cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ed2ffd640 1 -- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40cbe70 msgr2=0x7f9ed40c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ed2ffd640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40cbe70 0x7f9ed40c9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ed2ffd640 1 -- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed415da90 msgr2=0x7f9ed40cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ed2ffd640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed415da90 0x7f9ed40cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ed2ffd640 1 -- 172.21.15.38:0/339143728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40cc5e0 con 0x7f9ed40d3de0 2024-04-25T08:36:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ed2ffd640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ed40d3de0 0x7f9ed40cd680 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00b3f0 tx=0x7f9ecc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ecbfff640 1 -- 172.21.15.38:0/339143728 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc015020 con 0x7f9ed40d3de0 2024-04-25T08:36:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ecbfff640 1 -- 172.21.15.38:0/339143728 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc011830 con 0x7f9ed40d3de0 2024-04-25T08:36:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ed9846640 1 -- 172.21.15.38:0/339143728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed40cc8c0 con 0x7f9ed40d3de0 2024-04-25T08:36:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ecbfff640 1 -- 172.21.15.38:0/339143728 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc011c20 con 0x7f9ed40d3de0 2024-04-25T08:36:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.745+0000 7f9ed9846640 1 -- 172.21.15.38:0/339143728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed40cce80 con 0x7f9ed40d3de0 2024-04-25T08:36:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.746+0000 7f9ed9846640 1 -- 172.21.15.38:0/339143728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed4068e80 con 0x7f9ed40d3de0 2024-04-25T08:36:37.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.747+0000 7f9ecbfff640 1 -- 172.21.15.38:0/339143728 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ecc018020 con 0x7f9ed40d3de0 2024-04-25T08:36:37.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.747+0000 7f9ecbfff640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9eb406b4d0 0x7f9eb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:37.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.747+0000 7f9ecbfff640 1 -- 172.21.15.38:0/339143728 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9ecc089eb0 con 0x7f9ed40d3de0 2024-04-25T08:36:37.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.747+0000 7f9ed27fc640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9eb406b4d0 0x7f9eb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:37.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.748+0000 7f9ed27fc640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9eb406b4d0 0x7f9eb406d990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc034000 tx=0x7f9ebc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:37.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.751+0000 7f9ecbfff640 1 -- 172.21.15.38:0/339143728 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ecc01f030 con 0x7f9ed40d3de0 2024-04-25T08:36:37.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.973+0000 7f9ed9846640 1 -- 172.21.15.38:0/339143728 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ed40b3e40 con 0x7f9ed40d3de0 2024-04-25T08:36:37.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.974+0000 7f9ecbfff640 1 -- 172.21.15.38:0/339143728 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9ecc058550 con 0x7f9ed40d3de0 2024-04-25T08:36:37.968 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:36:37.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.976+0000 7f9ed9846640 1 -- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9eb406b4d0 msgr2=0x7f9eb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.976+0000 7f9ed9846640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9eb406b4d0 0x7f9eb406d990 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc034000 tx=0x7f9ebc02e000 comp rx=0 tx=0).stop 2024-04-25T08:36:37.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.976+0000 7f9ed9846640 1 -- 172.21.15.38:0/339143728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ed40d3de0 msgr2=0x7f9ed40cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:37.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.976+0000 7f9ed9846640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ed40d3de0 0x7f9ed40cd680 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00b3f0 tx=0x7f9ecc00b8b0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.976+0000 7f9ed9846640 1 -- 172.21.15.38:0/339143728 shutdown_connections 2024-04-25T08:36:37.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.976+0000 7f9ed9846640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9eb406b4d0 0x7f9eb406d990 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.976+0000 7f9ed9846640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ed40cbe70 0x7f9ed40c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.976+0000 7f9ed9846640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ed415da90 0x7f9ed40cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.976+0000 7f9ed9846640 1 --2- 172.21.15.38:0/339143728 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ed40d3de0 0x7f9ed40cd680 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:37.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.976+0000 7f9ed9846640 1 -- 172.21.15.38:0/339143728 >> 172.21.15.38:0/339143728 conn(0x7f9ed406bed0 msgr2=0x7f9ed406c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:37.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.976+0000 7f9ed9846640 1 -- 172.21.15.38:0/339143728 shutdown_connections 2024-04-25T08:36:37.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:37.976+0000 7f9ed9846640 1 -- 172.21.15.38:0/339143728 wait complete. 2024-04-25T08:36:37.981 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:36:37.981 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:36:37.982 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:36:37.982 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:36:37.982 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:37.982 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:36:37.982 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:36:37.982 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:36:37.982 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:36:37.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:37.985 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:36:38.001 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:36:38.001 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:36:38.001 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:36:38.001 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:38.001 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:36:38.001 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:38.002 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:36:38.002 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:36:38.002 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:36:38.002 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:36:38.002 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:36:38.002 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:36:38.002 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:36:38.002 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:36:38.002 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:36:38.002 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:36:38.002 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42439sec preferred_lft 42439sec 2024-04-25T08:36:38.003 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:36:38.003 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:38.003 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:36:38.003 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:36:38.003 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:36:38.003 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:36:38.003 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:36:38.003 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:36:38.003 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:38.003 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::440f:97ff:fe63:87b/64 scope link 2024-04-25T08:36:38.003 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:38.004 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:36:38.004 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ea:23:76:ab:b1:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:36:38.004 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::e823:76ff:feab:b1e9/64 scope link 2024-04-25T08:36:38.004 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:38.004 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:36:38.004 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:36:38.004 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::38c7:38ff:fe2e:6c92/64 scope link 2024-04-25T08:36:38.004 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:36:38.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:38.005 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:36:38.064 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:36:38.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:38.066 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:36:38.121 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:36:38.122 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:36:38.122 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:36:38.122 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:36:38.122 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:36:38.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:38.122 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:36:38.122 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:36:38.122 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:36:38.122 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:36:38.122 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:36:38.123 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:36:38.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:38.208 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:36:38.223 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:36:38.223 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:36:38.223 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:36:38.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:38.223 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T08:36:38.287 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:36:38.287 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:36:38.287 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:38.342 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:36:38.342 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:36:38.343 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:38.398 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:36:38.462 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:36:38.519 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:36:38.519 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:36:38.519 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:36:38.519 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:36:38.519 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:36:38.519 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:36:38.519 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:36:38.519 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:36:38.520 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:36:38.521 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:36:38.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:38.522 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:36:38.574 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:36:38.574 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:36:38.616 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:36:38.640 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:36:38.696 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:36:38.696 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:36:38.696 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:36:38.696 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:36:38.696 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:36:38.696 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:36:38.696 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:36:38.696 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:36:38.696 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:36:38.696 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:36:38.696 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:36:38.697 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:36:38.698 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:36:38.698 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:36:38.698 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:36:38.698 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:36:38.698 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:36:38.698 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:36:38.698 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:36:38.698 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:36:38.698 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:36:38.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:38.698 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:36:38.699 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-04-25T08:36:38.703+0000 7fd027b25480 -1 init, newargv = 0x5601ddc8b4a0 newargc=15 2024-04-25T08:36:38.699 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[69919]: starting ceph client 2024-04-25T08:36:38.702 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[69919]: starting fuse 2024-04-25T08:36:38.714 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:36:38.715 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:36:38.715 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-04-25T08:36:38.716 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:36:38.834 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.69919.asok 2024-04-25T08:36:38.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:36:38.845 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.69919.asok 2024-04-25T08:36:38.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:38.845 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.69919.asok status 2024-04-25T08:36:38.981 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:36:38.981 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:36:38.981 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:36:38.981 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:36:38.981 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-04-25T08:36:38.981 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:36:38.981 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:36:38.981 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "69919", 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 7563, 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 7563 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:36:38.982 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3184966310 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3184966310 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.7563 192.168.0.1:0/3184966310", 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/3184966310", 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 50, 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 82, 2024-04-25T08:36:38.983 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:36:38.984 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:36:38.984 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:36:38.984 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:36:38.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:38.989 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:36:38.989 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:39.005 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:36:39.005 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:39.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:39.005 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:36:39.070 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T08:36:39.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.221+0000 7f69026c4640 1 Processor -- start 2024-04-25T08:36:39.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.221+0000 7f69026c4640 1 -- start start 2024-04-25T08:36:39.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.221+0000 7f69026c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc1612c0 0x7f68fc0b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.221+0000 7f69026c4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f68fc0d0870 0x7f68fc0d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.221+0000 7f69026c4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68fc0b7200 con 0x7f68fc0b6a00 2024-04-25T08:36:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.221+0000 7f69026c4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68fc0d11e0 con 0x7f68fc0d0870 2024-04-25T08:36:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.221+0000 7f69026c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0b74e0 con 0x7f68fc1612c0 2024-04-25T08:36:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68fb7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f68fc0b6a00 0x7f68fc0d1c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:59818/0 (socket says 172.21.15.38:59818) 2024-04-25T08:36:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68fb7fe640 1 -- 172.21.15.38:0/2027075958 learned_addr learned my addr 172.21.15.38:0/2027075958 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:36:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68fbfff640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc1612c0 0x7f68fc0b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f6900c61640 1 --2- 172.21.15.38:0/2027075958 >> v2:172.21.15.107:3300/0 conn(0x7f68fc0d0870 0x7f68fc0d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850927352 0 0) 0x7f68fc0b7200 con 0x7f68fc0b6a00 2024-04-25T08:36:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e4003520 con 0x7f68fc0b6a00 2024-04-25T08:36:39.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3888637829 0 0) 0x7f68e4003520 con 0x7f68fc0b6a00 2024-04-25T08:36:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc1612c0 msgr2=0x7f68fc0b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc1612c0 0x7f68fc0b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 >> v2:172.21.15.107:3300/0 conn(0x7f68fc0d0870 msgr2=0x7f68fc0d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 --2- 172.21.15.38:0/2027075958 >> v2:172.21.15.107:3300/0 conn(0x7f68fc0d0870 0x7f68fc0d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc0b77c0 con 0x7f68fc0b6a00 2024-04-25T08:36:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f6900c61640 1 --2- 172.21.15.38:0/2027075958 >> v2:172.21.15.107:3300/0 conn(0x7f68fc0d0870 0x7f68fc0d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f68f0001670 con 0x7f68fc0b6a00 2024-04-25T08:36:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 >> v1:172.21.15.38:6789/0 conn(0x7f68fc0b6a00 legacy=0x7f68fc0d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e40039b0 0x7f68e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e4004800 0x7f68e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68e40051c0 0x7f68e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0d11e0 con 0x7f68e40051c0 2024-04-25T08:36:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0b74e0 con 0x7f68e40039b0 2024-04-25T08:36:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e400bf50 con 0x7f68e4004800 2024-04-25T08:36:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68fbfff640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc1612c0 0x7f68fc0b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:36:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f6900c61640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68e40051c0 0x7f68e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68fb7fe640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e4004800 0x7f68e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.222+0000 7f68fbfff640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e40039b0 0x7f68e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.223+0000 7f6900c61640 1 -- 172.21.15.38:0/2027075958 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e4004800 msgr2=0x7f68e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.223+0000 7f6900c61640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68e4004800 0x7f68e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.223+0000 7f6900c61640 1 -- 172.21.15.38:0/2027075958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e40039b0 msgr2=0x7f68e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:36:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.223+0000 7f6900c61640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68e40039b0 0x7f68e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.223+0000 7f6900c61640 1 -- 172.21.15.38:0/2027075958 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e4003520 con 0x7f68e40051c0 2024-04-25T08:36:39.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.223+0000 7f6900c61640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68e40051c0 0x7f68e400b9c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f68f400e380 tx=0x7f68e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e324ce6d7eaa5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:39.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.223+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4009de0 con 0x7f68e40051c0 2024-04-25T08:36:39.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.223+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f402c620 con 0x7f68e40051c0 2024-04-25T08:36:39.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.223+0000 7f68faffd640 1 -- 172.21.15.38:0/2027075958 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f402c9d0 con 0x7f68e40051c0 2024-04-25T08:36:39.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.224+0000 7f69026c4640 1 -- 172.21.15.38:0/2027075958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68e40051c0 msgr2=0x7f68e400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.224+0000 7f69026c4640 1 --2- 172.21.15.38:0/2027075958 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68e40051c0 0x7f68e400b9c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f68f400e380 tx=0x7f68e4005830 comp rx=0 tx=0).stop 2024-04-25T08:36:39.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.224+0000 7f6900c61640 1 -- 172.21.15.38:0/2027075958 reap_dead start 2024-04-25T08:36:39.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.224+0000 7f69026c4640 1 -- 172.21.15.38:0/2027075958 shutdown_connections 2024-04-25T08:36:39.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.224+0000 7f69026c4640 1 -- 172.21.15.38:0/2027075958 >> 172.21.15.38:0/2027075958 conn(0x7f68fc066d10 msgr2=0x7f68fc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:39.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.224+0000 7f69026c4640 1 -- 172.21.15.38:0/2027075958 shutdown_connections 2024-04-25T08:36:39.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.224+0000 7f69026c4640 1 -- 172.21.15.38:0/2027075958 wait complete. 2024-04-25T08:36:39.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.225+0000 7f69026c4640 1 Processor -- start 2024-04-25T08:36:39.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.225+0000 7f69026c4640 1 -- start start 2024-04-25T08:36:39.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.226+0000 7f69026c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68e40051c0 0x7f68fc144cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.226+0000 7f69026c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc159e80 0x7f68fc15a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.226+0000 7f69026c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68fc157080 0x7f68fc157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.226+0000 7f69026c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0b6e60 con 0x7f68e40051c0 2024-04-25T08:36:39.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.226+0000 7f69026c4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0b7d80 con 0x7f68fc157080 2024-04-25T08:36:39.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.226+0000 7f69026c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc0b6fe0 con 0x7f68fc159e80 2024-04-25T08:36:39.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.226+0000 7f68fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68e40051c0 0x7f68fc144cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.226+0000 7f68fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68e40051c0 0x7f68fc144cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56064/0 (socket says 172.21.15.38:56064) 2024-04-25T08:36:39.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.226+0000 7f68fbfff640 1 -- 172.21.15.38:0/1265551200 learned_addr learned my addr 172.21.15.38:0/1265551200 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:36:39.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f6900c61640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68fc157080 0x7f68fc157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f68fb7fe640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc159e80 0x7f68fc15a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f68fbfff640 1 -- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc159e80 msgr2=0x7f68fc15a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f68fbfff640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc159e80 0x7f68fc15a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f68fbfff640 1 -- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68fc157080 msgr2=0x7f68fc157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f68fbfff640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68fc157080 0x7f68fc157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f68fbfff640 1 -- 172.21.15.38:0/1265551200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc15a900 con 0x7f68e40051c0 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f6900c61640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68fc157080 0x7f68fc157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f68fbfff640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68e40051c0 0x7f68fc144cd0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f68ec011830 tx=0x7f68ec011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f68f8ff9640 1 -- 172.21.15.38:0/1265551200 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00f9c0 con 0x7f68e40051c0 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f68f8ff9640 1 -- 172.21.15.38:0/1265551200 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec00fb60 con 0x7f68e40051c0 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f69026c4640 1 -- 172.21.15.38:0/1265551200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68fc1455c0 con 0x7f68e40051c0 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f68f8ff9640 1 -- 172.21.15.38:0/1265551200 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec015460 con 0x7f68e40051c0 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.227+0000 7f69026c4640 1 -- 172.21.15.38:0/1265551200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68fc145b80 con 0x7f68e40051c0 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.228+0000 7f69026c4640 1 -- 172.21.15.38:0/1265551200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68fc068e80 con 0x7f68e40051c0 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.228+0000 7f68f8ff9640 1 -- 172.21.15.38:0/1265551200 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68ec015600 con 0x7f68e40051c0 2024-04-25T08:36:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.229+0000 7f68f8ff9640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f68e806b4d0 0x7f68e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:36:39.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.229+0000 7f68f8ff9640 1 -- 172.21.15.38:0/1265551200 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68ec08ae90 con 0x7f68e40051c0 2024-04-25T08:36:39.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.229+0000 7f68fb7fe640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f68e806b4d0 0x7f68e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:36:39.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.230+0000 7f68fb7fe640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f68e806b4d0 0x7f68e806d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f68f0000c00 tx=0x7f68f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:36:39.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.233+0000 7f68f8ff9640 1 -- 172.21.15.38:0/1265551200 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68ec019030 con 0x7f68e40051c0 2024-04-25T08:36:39.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.453+0000 7f69026c4640 1 -- 172.21.15.38:0/1265551200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f68fc0d3f60 con 0x7f68e40051c0 2024-04-25T08:36:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.729+0000 7f68f8ff9640 1 -- 172.21.15.38:0/1265551200 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v51) v1 ==== 167+0+0 (secure 0 0 0) 0x7f68ec059ae0 con 0x7f68e40051c0 2024-04-25T08:36:39.723 INFO:teuthology.orchestra.run.smithi038.stderr:enabled new snapshots 2024-04-25T08:36:39.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.731+0000 7f69026c4640 1 -- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f68e806b4d0 msgr2=0x7f68e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.731+0000 7f69026c4640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f68e806b4d0 0x7f68e806d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f68f0000c00 tx=0x7f68f0009040 comp rx=0 tx=0).stop 2024-04-25T08:36:39.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.731+0000 7f69026c4640 1 -- 172.21.15.38:0/1265551200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68e40051c0 msgr2=0x7f68fc144cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:36:39.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.731+0000 7f69026c4640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68e40051c0 0x7f68fc144cd0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f68ec011830 tx=0x7f68ec011cf0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.731+0000 7f69026c4640 1 -- 172.21.15.38:0/1265551200 shutdown_connections 2024-04-25T08:36:39.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.731+0000 7f69026c4640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f68fc157080 0x7f68fc157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.731+0000 7f69026c4640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f68e806b4d0 0x7f68e806d990 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.731+0000 7f69026c4640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f68fc159e80 0x7f68fc15a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.731+0000 7f69026c4640 1 --2- 172.21.15.38:0/1265551200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f68e40051c0 0x7f68fc144cd0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:36:39.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.731+0000 7f69026c4640 1 -- 172.21.15.38:0/1265551200 >> 172.21.15.38:0/1265551200 conn(0x7f68fc066d10 msgr2=0x7f68fc0cc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:36:39.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.731+0000 7f69026c4640 1 -- 172.21.15.38:0/1265551200 shutdown_connections 2024-04-25T08:36:39.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:36:39.731+0000 7f69026c4640 1 -- 172.21.15.38:0/1265551200 wait complete. 2024-04-25T08:36:39.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:36:39.737 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T08:36:39.758 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:36:39.758 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:36:39.814 DEBUG:teuthology.orchestra.run.smithi038:> stdin-killer -- python3 -c ' 2024-04-25T08:36:39.814 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:36:39.814 DEBUG:teuthology.orchestra.run.smithi038:> import uuid 2024-04-25T08:36:39.814 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> n = 2000 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> create_hard_links = False 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> if 3 > 0: 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> create_hard_links = True 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> dpath = os.path.dirname(path) 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> fnameprefix = os.path.basename(path) 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> os.makedirs(dpath, exist_ok=True) 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> try: 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> for i in range(n): 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T08:36:39.815 DEBUG:teuthology.orchestra.run.smithi038:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> f.write(f"{i}") 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> if False: 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> f.flush() 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> os.fsync(f.fileno()) 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> if False: 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> os.unlink(fpath) 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> if False: 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> os.fsync(dirfd) 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> if create_hard_links: 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> for j in range(3): 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> if False: 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> os.fsync(dirfd) 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> finally: 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> os.close(dirfd) 2024-04-25T08:36:39.816 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:37:03.432 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:37:03.436 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:37:07.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:37:07.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:07.996 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T08:37:08.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:08.045 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T08:37:33.496 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:37:33.499 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:37:37.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:37.593 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T08:37:37.618 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:37:37.618 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T08:37:37.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.766+0000 7f3dd07cf640 1 Processor -- start 2024-04-25T08:37:37.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.766+0000 7f3dd07cf640 1 -- start start 2024-04-25T08:37:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.767+0000 7f3dd07cf640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3dc80d0320 0x7f3dc80d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.767+0000 7f3dd07cf640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dc80b6980 0x7f3dc80b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.767+0000 7f3dd07cf640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc80d2060 con 0x7f3dc80d0700 2024-04-25T08:37:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.767+0000 7f3dd07cf640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc80d2330 con 0x7f3dc80d0320 2024-04-25T08:37:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.767+0000 7f3dd07cf640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80d2610 con 0x7f3dc80b6980 2024-04-25T08:37:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.767+0000 7f3dce56b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3dc80d0320 0x7f3dc80d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.767+0000 7f3dce56b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3dc80d0320 0x7f3dc80d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55162/0 (socket says 172.21.15.38:55162) 2024-04-25T08:37:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.767+0000 7f3dce56b640 1 -- 172.21.15.38:0/2559079736 learned_addr learned my addr 172.21.15.38:0/2559079736 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.768+0000 7f3dce56b640 1 -- 172.21.15.38:0/2559079736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dc80b6980 msgr2=0x7f3dc80b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.768+0000 7f3dce56b640 1 --2- 172.21.15.38:0/2559079736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dc80b6980 0x7f3dc80b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.768+0000 7f3dce56b640 1 -- 172.21.15.38:0/2559079736 >> v1:172.21.15.107:6789/0 conn(0x7f3dc80d0700 legacy=0x7f3dc80d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.768+0000 7f3dce56b640 1 -- 172.21.15.38:0/2559079736 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc80d28f0 con 0x7f3dc80d0320 2024-04-25T08:37:37.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.769+0000 7f3dce56b640 1 --2- 172.21.15.38:0/2559079736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc80d0320 0x7f3dc80d33d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009870 tx=0x7f3dc402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5118081ebd1f26e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:37.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.769+0000 7f3dcd569640 1 -- 172.21.15.38:0/2559079736 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4037070 con 0x7f3dc80d0320 2024-04-25T08:37:37.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.769+0000 7f3dcd569640 1 -- 172.21.15.38:0/2559079736 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc4032440 con 0x7f3dc80d0320 2024-04-25T08:37:37.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.769+0000 7f3dcd569640 1 -- 172.21.15.38:0/2559079736 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4032810 con 0x7f3dc80d0320 2024-04-25T08:37:37.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.770+0000 7f3dd07cf640 1 -- 172.21.15.38:0/2559079736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc80d0320 msgr2=0x7f3dc80d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:37.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.770+0000 7f3dd07cf640 1 --2- 172.21.15.38:0/2559079736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc80d0320 0x7f3dc80d33d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009870 tx=0x7f3dc402a2a0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.770+0000 7f3dd07cf640 1 -- 172.21.15.38:0/2559079736 shutdown_connections 2024-04-25T08:37:37.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.770+0000 7f3dd07cf640 1 --2- 172.21.15.38:0/2559079736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dc80b6980 0x7f3dc80b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.770+0000 7f3dd07cf640 1 --2- 172.21.15.38:0/2559079736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc80d0320 0x7f3dc80d33d0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.770+0000 7f3dd07cf640 1 -- 172.21.15.38:0/2559079736 >> 172.21.15.38:0/2559079736 conn(0x7f3dc80668f0 msgr2=0x7f3dc8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:37.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.770+0000 7f3dd07cf640 1 -- 172.21.15.38:0/2559079736 shutdown_connections 2024-04-25T08:37:37.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.770+0000 7f3dd07cf640 1 -- 172.21.15.38:0/2559079736 wait complete. 2024-04-25T08:37:37.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.772+0000 7f3dd07cf640 1 Processor -- start 2024-04-25T08:37:37.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.772+0000 7f3dd07cf640 1 -- start start 2024-04-25T08:37:37.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.773+0000 7f3dd07cf640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dc80b6980 0x7f3dc81f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.773+0000 7f3dd07cf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc80d0320 0x7f3dc80b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.774+0000 7f3dcdd6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc80d0320 0x7f3dc80b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.774+0000 7f3dcdd6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc80d0320 0x7f3dc80b3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55172/0 (socket says 172.21.15.38:55172) 2024-04-25T08:37:37.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.774+0000 7f3dce56b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dc80b6980 0x7f3dc81f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.774+0000 7f3dce56b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dc80b6980 0x7f3dc81f6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:54770/0 (socket says 172.21.15.38:54770) 2024-04-25T08:37:37.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.774+0000 7f3dd07cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dc80d0700 0x7f3dc80b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.774+0000 7f3dcdd6a640 1 -- 172.21.15.38:0/871328012 learned_addr learned my addr 172.21.15.38:0/871328012 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:37.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.774+0000 7f3dd07cf640 1 -- 172.21.15.38:0/871328012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80d0b40 con 0x7f3dc80d0320 2024-04-25T08:37:37.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.774+0000 7f3dd07cf640 1 -- 172.21.15.38:0/871328012 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80d2eb0 con 0x7f3dc80d0700 2024-04-25T08:37:37.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.775+0000 7f3dd07cf640 1 -- 172.21.15.38:0/871328012 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc81f7110 con 0x7f3dc80b6980 2024-04-25T08:37:37.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.776+0000 7f3dced6c640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dc80d0700 0x7f3dc80b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.777+0000 7f3dcdd6a640 1 -- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dc80b6980 msgr2=0x7f3dc81f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:37.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.777+0000 7f3dcdd6a640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dc80b6980 0x7f3dc81f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.777+0000 7f3dcdd6a640 1 -- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dc80d0700 msgr2=0x7f3dc80b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:37.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.777+0000 7f3dcdd6a640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dc80d0700 0x7f3dc80b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.777+0000 7f3dcdd6a640 1 -- 172.21.15.38:0/871328012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc80b3e80 con 0x7f3dc80d0320 2024-04-25T08:37:37.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.777+0000 7f3dcdd6a640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc80d0320 0x7f3dc80b3270 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3db800b3e0 tx=0x7f3db800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:37.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.780+0000 7f3dce56b640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dc80b6980 0x7f3dc81f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:37.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.781+0000 7f3dbeffd640 1 -- 172.21.15.38:0/871328012 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8015020 con 0x7f3dc80d0320 2024-04-25T08:37:37.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.782+0000 7f3dbeffd640 1 -- 172.21.15.38:0/871328012 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db8011850 con 0x7f3dc80d0320 2024-04-25T08:37:37.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.782+0000 7f3dd07cf640 1 -- 172.21.15.38:0/871328012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc80b4160 con 0x7f3dc80d0320 2024-04-25T08:37:37.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.782+0000 7f3dbeffd640 1 -- 172.21.15.38:0/871328012 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8011c40 con 0x7f3dc80d0320 2024-04-25T08:37:37.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.782+0000 7f3dd07cf640 1 -- 172.21.15.38:0/871328012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc815bc70 con 0x7f3dc80d0320 2024-04-25T08:37:37.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.785+0000 7f3dbeffd640 1 -- 172.21.15.38:0/871328012 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3db8018020 con 0x7f3dc80d0320 2024-04-25T08:37:37.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.788+0000 7f3dbeffd640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3dac06b5a0 0x7f3dac06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.789+0000 7f3dce56b640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3dac06b5a0 0x7f3dac06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.790+0000 7f3dbeffd640 1 -- 172.21.15.38:0/871328012 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3db808a2b0 con 0x7f3dc80d0320 2024-04-25T08:37:37.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.790+0000 7f3dd35a9740 1 Processor -- start 2024-04-25T08:37:37.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.791+0000 7f3dd35a9740 1 -- start start 2024-04-25T08:37:37.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.793+0000 7f3dd35a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb1c4b0 0x55814eb1e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.793+0000 7f3dd35a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 0x55814eb1c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.793+0000 7f3dd35a9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb46f70 0x55814eb473f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.793+0000 7f3dd35a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814ea76b90 con 0x55814eb1c4b0 2024-04-25T08:37:37.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.793+0000 7f3dd35a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814ea1fe40 con 0x55814eb58b80 2024-04-25T08:37:37.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.793+0000 7f3dd35a9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55814ea72c30 con 0x55814eb46f70 2024-04-25T08:37:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.795+0000 7f3dced6c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb46f70 0x55814eb473f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.795+0000 7f3dced6c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb46f70 0x55814eb473f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:54786/0 (socket says 172.21.15.38:54786) 2024-04-25T08:37:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.795+0000 7f3dce56b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb1c4b0 0x55814eb1e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.795+0000 7f3dced6c640 1 -- 172.21.15.38:0/823439192 learned_addr learned my addr 172.21.15.38:0/823439192 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.795+0000 7f3dcdd6a640 1 --2- 172.21.15.38:0/823439192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 0x55814eb1c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.795+0000 7f3dce56b640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3dac06b5a0 0x7f3dac06da60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4034000 tx=0x7f3dc402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.796+0000 7f3dce56b640 1 -- 172.21.15.38:0/823439192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb46f70 msgr2=0x55814eb473f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.796+0000 7f3dce56b640 1 --2- 172.21.15.38:0/823439192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb46f70 0x55814eb473f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.796+0000 7f3dce56b640 1 -- 172.21.15.38:0/823439192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 msgr2=0x55814eb1c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.796+0000 7f3dce56b640 1 --2- 172.21.15.38:0/823439192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 0x55814eb1c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.796+0000 7f3dce56b640 1 -- 172.21.15.38:0/823439192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55814e8557d0 con 0x55814eb1c4b0 2024-04-25T08:37:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.796+0000 7f3dce56b640 1 --2- 172.21.15.38:0/823439192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb1c4b0 0x55814eb1e610 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4031000 tx=0x7f3dc4008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5a61b7a622defb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.796+0000 7f3dabfff640 1 -- 172.21.15.38:0/823439192 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4037040 con 0x55814eb1c4b0 2024-04-25T08:37:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.797+0000 7f3dabfff640 1 -- 172.21.15.38:0/823439192 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc40476d0 con 0x55814eb1c4b0 2024-04-25T08:37:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.798+0000 7f3dd35a9740 1 -- 172.21.15.38:0/823439192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb1c4b0 msgr2=0x55814eb1e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:37.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.798+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/823439192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb1c4b0 0x55814eb1e610 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4031000 tx=0x7f3dc4008030 comp rx=0 tx=0).stop 2024-04-25T08:37:37.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.798+0000 7f3dd35a9740 1 -- 172.21.15.38:0/823439192 shutdown_connections 2024-04-25T08:37:37.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.798+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/823439192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb46f70 0x55814eb473f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.798+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/823439192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 0x55814eb1c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.798+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/823439192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb1c4b0 0x55814eb1e610 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.798+0000 7f3dd35a9740 1 -- 172.21.15.38:0/823439192 >> 172.21.15.38:0/823439192 conn(0x55814eadb870 msgr2=0x55814eb61f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:37.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.799+0000 7f3dd35a9740 1 -- 172.21.15.38:0/823439192 shutdown_connections 2024-04-25T08:37:37.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.799+0000 7f3dd35a9740 1 -- 172.21.15.38:0/823439192 wait complete. 2024-04-25T08:37:37.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.800+0000 7f3dd35a9740 1 -- start start 2024-04-25T08:37:37.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.800+0000 7f3dd35a9740 1 Processor -- start 2024-04-25T08:37:37.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.802+0000 7f3dd35a9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb46f70 0x55814eb41590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.802+0000 7f3dd35a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 0x55814eb5a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.802+0000 7f3dd35a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 0x55814eb3ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.802+0000 7f3dd35a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814ea333e0 con 0x55814eb442c0 2024-04-25T08:37:37.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.802+0000 7f3dd35a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814eb44ac0 con 0x55814eb58b80 2024-04-25T08:37:37.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.802+0000 7f3dd35a9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55814eb62ef0 con 0x55814eb46f70 2024-04-25T08:37:37.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.802+0000 7f3dd35a9740 10 client.0 _ll_register_callbacks cb 0x55814ea32db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:37.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.803+0000 7f3dced6c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 0x55814eb3ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.803+0000 7f3dced6c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 0x55814eb3ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55198/0 (socket says 172.21.15.38:55198) 2024-04-25T08:37:37.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.803+0000 7f3dced6c640 1 -- 172.21.15.38:0/1481636101 learned_addr learned my addr 172.21.15.38:0/1481636101 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:37.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.804+0000 7f3dce56b640 1 --2- 172.21.15.38:0/1481636101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb46f70 0x55814eb41590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.804+0000 7f3dced6c640 1 -- 172.21.15.38:0/1481636101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb46f70 msgr2=0x55814eb41590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.804+0000 7f3dced6c640 1 --2- 172.21.15.38:0/1481636101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb46f70 0x55814eb41590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.804+0000 7f3dced6c640 1 -- 172.21.15.38:0/1481636101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 msgr2=0x55814eb5a3b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:37:37.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.804+0000 7f3dced6c640 1 --2- 172.21.15.38:0/1481636101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 0x55814eb5a3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.804+0000 7f3dced6c640 1 -- 172.21.15.38:0/1481636101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55814e74b0f0 con 0x55814eb442c0 2024-04-25T08:37:37.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.807+0000 7f3dced6c640 1 --2- 172.21.15.38:0/1481636101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 0x55814eb3ff30 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000f8a0 tx=0x7f3dc000fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a9432b824d01de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:37.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.807+0000 7f3da97fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:37:37.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.808+0000 7f3da97fa640 1 -- 172.21.15.38:0/1481636101 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0017070 con 0x55814eb442c0 2024-04-25T08:37:37.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.808+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1481636101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55814ea840d0 con 0x55814eb442c0 2024-04-25T08:37:37.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.808+0000 7f3da97fa640 1 -- 172.21.15.38:0/1481636101 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3dc000dbf0 con 0x55814eb442c0 2024-04-25T08:37:37.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.808+0000 7f3da97fa640 1 -- 172.21.15.38:0/1481636101 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dc0022050 con 0x55814eb442c0 2024-04-25T08:37:37.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.808+0000 7f3dd35a9740 10 client.7581 fetch_fsmap learned FSMap version 51 2024-04-25T08:37:37.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.808+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1481636101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55814e8b5f60 con 0x55814eb442c0 2024-04-25T08:37:37.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.809+0000 7f3da97fa640 1 -- 172.21.15.38:0/1481636101 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55814ea840d0 con 0x55814eb442c0 2024-04-25T08:37:37.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.810+0000 7f3da97fa640 1 -- 172.21.15.38:0/1481636101 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3dc0025020 con 0x55814eb442c0 2024-04-25T08:37:37.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.810+0000 7f3dd35a9740 10 client.7581 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:37:37.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.810+0000 7f3dd35a9740 10 client.7581 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:37:37.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.810+0000 7f3dd35a9740 20 client.7581 populate_metadata read hostname 'smithi038' 2024-04-25T08:37:37.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.810+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/1481636101 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55814eb753a0 0x55814eb757b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.810+0000 7f3dd35a9740 4 client.7581 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:37.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.810+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1481636101 --> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55814ea7e420 con 0x55814eb753a0 2024-04-25T08:37:37.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.811+0000 7f3dce56b640 1 --2- 172.21.15.38:0/1481636101 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55814eb753a0 0x55814eb757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.811+0000 7f3da97fa640 10 client.7581 ms_handle_connect on v2:172.21.15.38:6833/380751658 2024-04-25T08:37:37.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.811+0000 7f3dce56b640 1 --2- 172.21.15.38:0/1481636101 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55814eb753a0 0x55814eb757b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d26772c5c1afa63 server_cookie=b3a8947e32ba6d8b in_seq=0 out_seq=0 2024-04-25T08:37:37.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.812+0000 7f3da97fa640 1 -- 172.21.15.38:0/1481636101 <== mds.0 v2:172.21.15.38:6833/380751658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55814ea7e420 con 0x55814eb753a0 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.812+0000 7f3da97fa640 10 client.7581 handle_command_reply: tid=0 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.812+0000 7f3dd35a9740 1 client.7581 shutdown 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.812+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1481636101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 msgr2=0x55814eb3ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.812+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/1481636101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 0x55814eb3ff30 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000f8a0 tx=0x7f3dc000fd60 comp rx=0 tx=0).stop 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.812+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1481636101 shutdown_connections 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.812+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/1481636101 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55814eb753a0 0x55814eb757b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.812+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/1481636101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 0x55814eb3ff30 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.812+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/1481636101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 0x55814eb5a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.812+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/1481636101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb46f70 0x55814eb41590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.812+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1481636101 >> 172.21.15.38:0/1481636101 conn(0x55814eadb870 msgr2=0x55814eb467d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.812+0000 7f3da97fa640 0 client.7581 ms_handle_reset on v2:172.21.15.38:6833/380751658 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.813+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1481636101 shutdown_connections 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.813+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1481636101 wait complete. 2024-04-25T08:37:37.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.813+0000 7f3dd35a9740 20 client.7581 trim_cache size 0 max 16384 2024-04-25T08:37:37.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.827+0000 7f3dd35a9740 1 Processor -- start 2024-04-25T08:37:37.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.827+0000 7f3dd35a9740 1 -- start start 2024-04-25T08:37:37.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dd35a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 0x55814eb1c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dd35a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb46f70 0x55814eb48630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dd35a9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb58b80 0x55814eb48b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dd35a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc806ff50 con 0x55814eb442c0 2024-04-25T08:37:37.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dd35a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc806f190 con 0x55814eb46f70 2024-04-25T08:37:37.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dd35a9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc81f4b30 con 0x55814eb58b80 2024-04-25T08:37:37.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dce56b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 0x55814eb1c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dce56b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 0x55814eb1c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55214/0 (socket says 172.21.15.38:55214) 2024-04-25T08:37:37.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dce56b640 1 -- 172.21.15.38:0/4162394251 learned_addr learned my addr 172.21.15.38:0/4162394251 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:37.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dced6c640 1 --2- 172.21.15.38:0/4162394251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb58b80 0x55814eb48b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dcdd6a640 1 --2- 172.21.15.38:0/4162394251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb46f70 0x55814eb48630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dce56b640 1 -- 172.21.15.38:0/4162394251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb58b80 msgr2=0x55814eb48b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dce56b640 1 --2- 172.21.15.38:0/4162394251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb58b80 0x55814eb48b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dce56b640 1 -- 172.21.15.38:0/4162394251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb46f70 msgr2=0x55814eb48630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dce56b640 1 --2- 172.21.15.38:0/4162394251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb46f70 0x55814eb48630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dce56b640 1 -- 172.21.15.38:0/4162394251 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55814e7241b0 con 0x55814eb442c0 2024-04-25T08:37:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dced6c640 1 --2- 172.21.15.38:0/4162394251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb58b80 0x55814eb48b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dcdd6a640 1 --2- 172.21.15.38:0/4162394251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb46f70 0x55814eb48630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:37:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.828+0000 7f3dce56b640 1 --2- 172.21.15.38:0/4162394251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 0x55814eb1c650 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009990 tx=0x7f3dc403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fe8aedc4ad55b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.829+0000 7f3dab7fe640 1 -- 172.21.15.38:0/4162394251 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4047690 con 0x55814eb442c0 2024-04-25T08:37:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.829+0000 7f3dab7fe640 1 -- 172.21.15.38:0/4162394251 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc4061070 con 0x55814eb442c0 2024-04-25T08:37:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.829+0000 7f3dab7fe640 1 -- 172.21.15.38:0/4162394251 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc405d230 con 0x55814eb442c0 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.829+0000 7f3dd35a9740 1 -- 172.21.15.38:0/4162394251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 msgr2=0x55814eb1c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.829+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/4162394251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 0x55814eb1c650 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009990 tx=0x7f3dc403d000 comp rx=0 tx=0).stop 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.829+0000 7f3dd35a9740 1 -- 172.21.15.38:0/4162394251 shutdown_connections 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.829+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/4162394251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb58b80 0x55814eb48b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.829+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/4162394251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb46f70 0x55814eb48630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.829+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/4162394251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb442c0 0x55814eb1c650 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.829+0000 7f3dd35a9740 1 -- 172.21.15.38:0/4162394251 >> 172.21.15.38:0/4162394251 conn(0x55814eadb870 msgr2=0x55814eb3fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.829+0000 7f3dd35a9740 1 -- 172.21.15.38:0/4162394251 shutdown_connections 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.829+0000 7f3dd35a9740 1 -- 172.21.15.38:0/4162394251 wait complete. 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.830+0000 7f3dd35a9740 1 -- start start 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.830+0000 7f3dd35a9740 1 Processor -- start 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dd35a9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb442c0 0x55814ebc8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dd35a9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb46f70 0x55814eb399c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dd35a9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 0x55814ebcec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dd35a9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55814eb84d40 con 0x55814eb46f70 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dd35a9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc815cf90 con 0x55814eb58b80 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dd35a9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55814eb5b3d0 con 0x55814eb442c0 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dd35a9740 10 client.0 _ll_register_callbacks cb 0x7f3dc00078e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dcdd6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb46f70 0x55814eb399c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dcdd6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb46f70 0x55814eb399c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55230/0 (socket says 172.21.15.38:55230) 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dcdd6a640 1 -- 172.21.15.38:0/1708385896 learned_addr learned my addr 172.21.15.38:0/1708385896 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dce56b640 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb442c0 0x55814ebc8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dced6c640 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 0x55814ebcec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dcdd6a640 1 -- 172.21.15.38:0/1708385896 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb442c0 msgr2=0x55814ebc8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dcdd6a640 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb442c0 0x55814ebc8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dcdd6a640 1 -- 172.21.15.38:0/1708385896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 msgr2=0x55814ebcec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dcdd6a640 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 0x55814ebcec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dcdd6a640 1 -- 172.21.15.38:0/1708385896 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55814ebcc560 con 0x55814eb46f70 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3dcdd6a640 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb46f70 0x55814eb399c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f3db8058580 tx=0x7f3db8058940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ef483db523f89e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.831+0000 7f3da97fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.832+0000 7f3da97fa640 1 -- 172.21.15.38:0/1708385896 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8070020 con 0x55814eb46f70 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.832+0000 7f3da97fa640 1 -- 172.21.15.38:0/1708385896 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3db80701c0 con 0x55814eb46f70 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.832+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1708385896 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3dc4061070 con 0x55814eb46f70 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.832+0000 7f3da97fa640 1 -- 172.21.15.38:0/1708385896 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3db80703c0 con 0x55814eb46f70 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.832+0000 7f3dd35a9740 10 client.7587 fetch_fsmap learned FSMap version 51 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.832+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1708385896 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55814eb42b80 con 0x55814eb46f70 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.832+0000 7f3da97fa640 1 -- 172.21.15.38:0/1708385896 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc4061070 con 0x55814eb46f70 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.833+0000 7f3da97fa640 1 -- 172.21.15.38:0/1708385896 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3db80880c0 con 0x55814eb46f70 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.833+0000 7f3dd35a9740 10 client.7587 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.833+0000 7f3dd35a9740 10 client.7587 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.833+0000 7f3dd35a9740 20 client.7587 populate_metadata read hostname 'smithi038' 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.833+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55814eb3f800 0x55814ebcf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.834+0000 7f3dd35a9740 4 client.7587 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T08:37:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.834+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1708385896 --> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55814e9fce60 con 0x55814eb3f800 2024-04-25T08:37:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.834+0000 7f3dce56b640 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55814eb3f800 0x55814ebcf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.834+0000 7f3dce56b640 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55814eb3f800 0x55814ebcf190 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63ad1112dfc51555 server_cookie=34768ac3d34721c5 in_seq=0 out_seq=0 2024-04-25T08:37:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.834+0000 7f3da97fa640 10 client.7587 ms_handle_connect on v2:172.21.15.38:6833/380751658 2024-04-25T08:37:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.834+0000 7f3da97fa640 1 -- 172.21.15.38:0/1708385896 <== mds.0 v2:172.21.15.38:6833/380751658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17930 (crc 0 0 0) 0x55814e9fce60 con 0x55814eb3f800 2024-04-25T08:37:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.834+0000 7f3da97fa640 10 client.7587 handle_command_reply: tid=0 2024-04-25T08:37:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3dd35a9740 1 client.7587 shutdown 2024-04-25T08:37:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1708385896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb46f70 msgr2=0x55814eb399c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb46f70 0x55814eb399c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f3db8058580 tx=0x7f3db8058940 comp rx=0 tx=0).stop 2024-04-25T08:37:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1708385896 shutdown_connections 2024-04-25T08:37:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55814eb3f800 0x55814ebcf190 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55814eb58b80 0x55814ebcec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3da97fa640 0 client.7587 ms_handle_reset on v2:172.21.15.38:6833/380751658 2024-04-25T08:37:37.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55814eb46f70 0x55814eb399c0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3dd35a9740 1 --2- 172.21.15.38:0/1708385896 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55814eb442c0 0x55814ebc8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1708385896 >> 172.21.15.38:0/1708385896 conn(0x55814eadb870 msgr2=0x55814eb64f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:37.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1708385896 shutdown_connections 2024-04-25T08:37:37.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3dd35a9740 1 -- 172.21.15.38:0/1708385896 wait complete. 2024-04-25T08:37:37.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.835+0000 7f3dd35a9740 20 client.7587 trim_cache size 0 max 16384 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 2476, 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 2476, 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 779915, 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 13571143, 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 9, 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 3, 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.254454901, 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.095979585, 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.039970223, 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.103310380, 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:37:37.836 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2475, 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.388225632, 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000156858 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2, 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000145, 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000072 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 34519, 2024-04-25T08:37:37.837 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 60443, 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 12092337, 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 50917554, 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 7, 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 2.959985466, 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 1.475055572, 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 1.742745902, 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.050613651, 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 60443, 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.190066169, 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000019689 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.838 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 88598, 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.013672323, 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000154 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 119440, 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 20624 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 3828, 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 3845, 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 946140, 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 46958666, 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 6, 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.358766061, 2024-04-25T08:37:37.839 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.175818705, 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.063676130, 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.097412842, 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3844, 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.278424866, 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000072431 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 111920 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-04-25T08:37:37.840 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 9363, 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 12.533037560, 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001338570 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1337, 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.546773379, 2024-04-25T08:37:37.841 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000408955 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 28303, 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 28303, 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 28303, 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 231.591748725, 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.008182586 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-04-25T08:37:37.842 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 12, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 19868, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 19868, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 19868, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 482, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 482, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 2, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 47568, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 47051, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-04-25T08:37:37.843 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 161, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T08:37:37.844 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-04-25T08:37:37.845 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T08:37:37.845 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-04-25T08:37:37.845 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 0, 2024-04-25T08:37:37.845 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T08:37:37.845 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T08:37:37.845 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 0, 2024-04-25T08:37:37.845 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-04-25T08:37:37.846 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 6203, 2024-04-25T08:37:37.846 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 6010, 2024-04-25T08:37:37.846 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T08:37:37.846 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T08:37:37.846 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T08:37:37.846 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 6203, 2024-04-25T08:37:37.846 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T08:37:37.846 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 24, 2024-04-25T08:37:37.846 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 17853 2024-04-25T08:37:37.846 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 1765, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 219, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 4127, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 1520, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 2236, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-04-25T08:37:37.847 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 21709, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-04-25T08:37:37.848 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 21709, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 28, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 28, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 3, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 80919723, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 0, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6567, 2024-04-25T08:37:37.849 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 67.619159418, 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.010296811 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 0 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 2014, 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 2014, 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 0, 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 13, 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 13, 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 0, 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 19868, 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 19868, 2024-04-25T08:37:37.850 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 0, 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 263, 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 2002, 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 1739, 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 103540, 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215396 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 38030, 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 28303, 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 5, 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-04-25T08:37:37.851 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2000, 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.353677167, 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000676838 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2008, 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.594442108, 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000296036 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-04-25T08:37:37.852 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6000, 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 9.429110689, 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001571518 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 8046, 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 7.163849316, 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000890361 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-04-25T08:37:37.853 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.854 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-04-25T08:37:37.855 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3, 2024-04-25T08:37:37.855 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.002449250, 2024-04-25T08:37:37.855 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000816416 2024-04-25T08:37:37.855 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.855 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-04-25T08:37:37.855 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.855 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.855 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.855 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.855 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-04-25T08:37:37.855 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.002671373, 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002671373 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.005265049, 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.005265049 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-04-25T08:37:37.856 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6, 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.092269220, 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.015378203 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2236, 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 197.798565641, 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.088460896 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-04-25T08:37:37.857 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.858 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.003176777, 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.003176777 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000366527, 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000366527 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.859 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-04-25T08:37:37.860 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 8000, 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 15.145905608, 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001893238 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-04-25T08:37:37.861 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 1, 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 0, 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 19606, 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 19606, 2024-04-25T08:37:37.862 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 58, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-04-25T08:37:37.863 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T08:37:37.864 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 17784110, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 4517, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-04-25T08:37:37.865 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-04-25T08:37:37.866 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-04-25T08:37:37.866 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-04-25T08:37:37.866 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 5424, 2024-04-25T08:37:37.866 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 76, 2024-04-25T08:37:37.866 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T08:37:37.866 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-04-25T08:37:37.866 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-04-25T08:37:37.866 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-04-25T08:37:37.866 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 21189357, 2024-04-25T08:37:37.866 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 819869, 2024-04-25T08:37:37.867 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-04-25T08:37:37.867 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-04-25T08:37:37.867 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-04-25T08:37:37.867 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-04-25T08:37:37.867 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.867 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-04-25T08:37:37.867 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-04-25T08:37:37.867 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-04-25T08:37:37.867 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 8327, 2024-04-25T08:37:37.867 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 76765958, 2024-04-25T08:37:37.867 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 36, 2024-04-25T08:37:37.867 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 8327, 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 8327, 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 19.912644686, 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002391334 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 8327, 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 8387 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 8327, 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 55, 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 8272, 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-04-25T08:37:37.868 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 0, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 11, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 55, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 6622, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 36, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 1285, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 265, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 0, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 22, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-04-25T08:37:37.869 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-04-25T08:37:37.870 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 82, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 1, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 8, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 8, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-04-25T08:37:37.871 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 77, 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 0, 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 14 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 1, 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-04-25T08:37:37.872 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 32, 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 1520, 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 1520, 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:37:37.873 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 40818, 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 10139650, 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 40818, 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 40818, 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 10139650, 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:37:37.874 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 8327, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 76929758, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 8327, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 76929758, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.875 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.876 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.876 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:37:37.876 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.876 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-04-25T08:37:37.876 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:37:37.876 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-04-25T08:37:37.876 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:37:37.876 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T08:37:37.876 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T08:37:37.876 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:37:37.876 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 8327, 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 8327, 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 8327, 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 8327, 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-25T08:37:37.877 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 1520, 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 163800, 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 1520, 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 55, 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 163800, 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:37:37.878 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55ba286926e0": { 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 140849, 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 21709, 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 76725419, 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 21709, 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 6567, 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 76584570, 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:37:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:37:37.880 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:37:37.880 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:37:37.880 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:37:37.880 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:37:37.880 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:37:37.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.836+0000 7f3dd07cf640 1 -- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3dac06b5a0 msgr2=0x7f3dac06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:37.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.836+0000 7f3dd07cf640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3dac06b5a0 0x7f3dac06da60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4034000 tx=0x7f3dc402e000 comp rx=0 tx=0).stop 2024-04-25T08:37:37.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.836+0000 7f3dd07cf640 1 -- 172.21.15.38:0/871328012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc80d0320 msgr2=0x7f3dc80b3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:37.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.836+0000 7f3dd07cf640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc80d0320 0x7f3dc80b3270 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3db800b3e0 tx=0x7f3db800b8a0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.836+0000 7f3dd07cf640 1 -- 172.21.15.38:0/871328012 shutdown_connections 2024-04-25T08:37:37.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.836+0000 7f3dd07cf640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3dc80d0700 0x7f3dc80b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.836+0000 7f3dd07cf640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3dc80d0320 0x7f3dc80b3270 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.836+0000 7f3dd07cf640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3dac06b5a0 0x7f3dac06da60 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.836+0000 7f3dd07cf640 1 --2- 172.21.15.38:0/871328012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3dc80b6980 0x7f3dc81f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.836+0000 7f3dd07cf640 1 -- 172.21.15.38:0/871328012 >> 172.21.15.38:0/871328012 conn(0x7f3dc80668f0 msgr2=0x7f3dc80b84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:37.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.836+0000 7f3dd07cf640 1 -- 172.21.15.38:0/871328012 shutdown_connections 2024-04-25T08:37:37.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.836+0000 7f3dd07cf640 1 -- 172.21.15.38:0/871328012 wait complete. 2024-04-25T08:37:37.886 INFO:tasks.cephfs.test_scrub_checks:num of strays: 1765 2024-04-25T08:37:37.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:37.887 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-25T08:37:37.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.997+0000 7f381d948640 1 Processor -- start 2024-04-25T08:37:37.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.997+0000 7f381d948640 1 -- start start 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f381d948640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180af800 0x7f38180ccd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f381d948640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3818162380 0x7f381806abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f381d948640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381806b430 con 0x7f38180cf8e0 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f381d948640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381806b710 con 0x7f3818162380 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f381d948640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3818161ef0 con 0x7f38180af800 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f38177fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f38180cf8e0 0x7f38180cfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56312/0 (socket says 172.21.15.38:56312) 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f38177fe640 1 -- 172.21.15.38:0/225864316 learned_addr learned my addr 172.21.15.38:0/225864316 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f3816ffd640 1 --2- 172.21.15.38:0/225864316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180af800 0x7f38180ccd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f380ffff640 1 --2- 172.21.15.38:0/225864316 >> v2:172.21.15.107:3300/0 conn(0x7f3818162380 0x7f381806abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903919173 0 0) 0x7f381806b430 con 0x7f38180cf8e0 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3804003520 con 0x7f38180cf8e0 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 543978425 0 0) 0x7f3804003520 con 0x7f38180cf8e0 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180af800 msgr2=0x7f38180ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f38167fc640 1 --2- 172.21.15.38:0/225864316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180af800 0x7f38180ccd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 >> v2:172.21.15.107:3300/0 conn(0x7f3818162380 msgr2=0x7f381806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f38167fc640 1 --2- 172.21.15.38:0/225864316 >> v2:172.21.15.107:3300/0 conn(0x7f3818162380 0x7f381806abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38181618a0 con 0x7f38180cf8e0 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.998+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f38100019f0 con 0x7f38180cf8e0 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 >> v1:172.21.15.38:6789/0 conn(0x7f38180cf8e0 legacy=0x7f38180cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38167fc640 1 --2- 172.21.15.38:0/225864316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180af800 0x7f38040039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38167fc640 1 --2- 172.21.15.38:0/225864316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3804003f30 0x7f3804004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38167fc640 1 --2- 172.21.15.38:0/225864316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3804004d40 0x7f38040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381806b710 con 0x7f3804004d40 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3818161ef0 con 0x7f3804003f30 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380400baa0 con 0x7f38180af800 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38177fe640 1 --2- 172.21.15.38:0/225864316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3804004d40 0x7f38040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f380ffff640 1 --2- 172.21.15.38:0/225864316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3804003f30 0x7f3804004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f3816ffd640 1 --2- 172.21.15.38:0/225864316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180af800 0x7f38040039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:37.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38177fe640 1 -- 172.21.15.38:0/225864316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180af800 msgr2=0x7f38040039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38177fe640 1 --2- 172.21.15.38:0/225864316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180af800 0x7f38040039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38177fe640 1 -- 172.21.15.38:0/225864316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3804003f30 msgr2=0x7f3804004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:37.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38177fe640 1 --2- 172.21.15.38:0/225864316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3804003f30 0x7f3804004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:37.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:37.999+0000 7f38177fe640 1 -- 172.21.15.38:0/225864316 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3804003520 con 0x7f3804004d40 2024-04-25T08:37:37.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.000+0000 7f38177fe640 1 --2- 172.21.15.38:0/225864316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3804004d40 0x7f38040051c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f381000a8a0 tx=0x7f3804005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a3986f2e4d82be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:37.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.000+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381000a4c0 con 0x7f3804004d40 2024-04-25T08:37:37.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.000+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381002f070 con 0x7f3804004d40 2024-04-25T08:37:37.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.000+0000 7f38167fc640 1 -- 172.21.15.38:0/225864316 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381002bd70 con 0x7f3804004d40 2024-04-25T08:37:38.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.000+0000 7f381d948640 1 -- 172.21.15.38:0/225864316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3804004d40 msgr2=0x7f38040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.000+0000 7f381d948640 1 --2- 172.21.15.38:0/225864316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3804004d40 0x7f38040051c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f381000a8a0 tx=0x7f3804005830 comp rx=0 tx=0).stop 2024-04-25T08:37:38.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.000+0000 7f38177fe640 1 -- 172.21.15.38:0/225864316 reap_dead start 2024-04-25T08:37:38.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.000+0000 7f381d948640 1 -- 172.21.15.38:0/225864316 shutdown_connections 2024-04-25T08:37:38.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.000+0000 7f381d948640 1 -- 172.21.15.38:0/225864316 >> 172.21.15.38:0/225864316 conn(0x7f3818072610 msgr2=0x7f3818099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:38.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.000+0000 7f381d948640 1 -- 172.21.15.38:0/225864316 shutdown_connections 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.001+0000 7f381d948640 1 -- 172.21.15.38:0/225864316 wait complete. 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.002+0000 7f381d948640 1 Processor -- start 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.002+0000 7f381d948640 1 -- start start 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.002+0000 7f381d948640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3804004d40 0x7f381814ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.002+0000 7f381d948640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180b5460 0x7f38181f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.002+0000 7f381d948640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38180b31c0 0x7f38180c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.002+0000 7f381d948640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38180afbe0 con 0x7f3804004d40 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.002+0000 7f381d948640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381806b120 con 0x7f38180b31c0 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.002+0000 7f381d948640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38180afd60 con 0x7f38180b5460 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f3816ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3804004d40 0x7f381814ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f3816ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3804004d40 0x7f381814ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55248/0 (socket says 172.21.15.38:55248) 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f3816ffd640 1 -- 172.21.15.38:0/1567467355 learned_addr learned my addr 172.21.15.38:0/1567467355 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f380ffff640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180b5460 0x7f38181f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f38177fe640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38180b31c0 0x7f38180c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f3816ffd640 1 -- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180b5460 msgr2=0x7f38181f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f3816ffd640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180b5460 0x7f38181f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f3816ffd640 1 -- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38180b31c0 msgr2=0x7f38180c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f3816ffd640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38180b31c0 0x7f38180c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f3816ffd640 1 -- 172.21.15.38:0/1567467355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38181f47d0 con 0x7f3804004d40 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f380ffff640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180b5460 0x7f38181f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f38177fe640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38180b31c0 0x7f38180c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f3816ffd640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3804004d40 0x7f381814ed90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f38080119a0 tx=0x7f3808011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f3814ff9640 1 -- 172.21.15.38:0/1567467355 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f380800faf0 con 0x7f3804004d40 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f3814ff9640 1 -- 172.21.15.38:0/1567467355 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f380800fc90 con 0x7f3804004d40 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f381d948640 1 -- 172.21.15.38:0/1567467355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38180b5990 con 0x7f3804004d40 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f3814ff9640 1 -- 172.21.15.38:0/1567467355 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808014440 con 0x7f3804004d40 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.003+0000 7f381d948640 1 -- 172.21.15.38:0/1567467355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38180c7c10 con 0x7f3804004d40 2024-04-25T08:37:38.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.005+0000 7f3814ff9640 1 -- 172.21.15.38:0/1567467355 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3808012040 con 0x7f3804004d40 2024-04-25T08:37:38.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.005+0000 7f3814ff9640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f37f806b4d0 0x7f37f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.005+0000 7f3814ff9640 1 -- 172.21.15.38:0/1567467355 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f380808aa30 con 0x7f3804004d40 2024-04-25T08:37:38.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.005+0000 7f380ffff640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f37f806b4d0 0x7f37f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.006+0000 7f380ffff640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f37f806b4d0 0x7f37f806d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3818066ec0 tx=0x7f3800008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.006+0000 7f3820645740 1 Processor -- start 2024-04-25T08:37:38.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.006+0000 7f3820645740 1 -- start start 2024-04-25T08:37:38.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.007+0000 7f3820645740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 0x55e5c98d3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.007+0000 7f3820645740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d3aa0 0x55e5c98a3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.007+0000 7f3820645740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98a3740 0x55e5c98cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.007+0000 7f3820645740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5c980c0a0 con 0x55e5c98a3740 2024-04-25T08:37:38.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.007+0000 7f3820645740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5c97b4d40 con 0x55e5c98d3080 2024-04-25T08:37:38.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.007+0000 7f3820645740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5c9808140 con 0x55e5c98d3aa0 2024-04-25T08:37:38.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.007+0000 7f38177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98a3740 0x55e5c98cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.007+0000 7f38177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98a3740 0x55e5c98cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55264/0 (socket says 172.21.15.38:55264) 2024-04-25T08:37:38.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.007+0000 7f38177fe640 1 -- 172.21.15.38:0/3971485929 learned_addr learned my addr 172.21.15.38:0/3971485929 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:38.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.007+0000 7f380ffff640 1 --2- 172.21.15.38:0/3971485929 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d3aa0 0x55e5c98a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.007+0000 7f3816ffd640 1 --2- 172.21.15.38:0/3971485929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 0x55e5c98d3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.008+0000 7f38177fe640 1 -- 172.21.15.38:0/3971485929 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d3aa0 msgr2=0x55e5c98a3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.008+0000 7f38177fe640 1 --2- 172.21.15.38:0/3971485929 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d3aa0 0x55e5c98a3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.008+0000 7f38177fe640 1 -- 172.21.15.38:0/3971485929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 msgr2=0x55e5c98d3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.008+0000 7f38177fe640 1 --2- 172.21.15.38:0/3971485929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 0x55e5c98d3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.008+0000 7f38177fe640 1 -- 172.21.15.38:0/3971485929 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5c95eca20 con 0x55e5c98a3740 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.008+0000 7f38177fe640 1 --2- 172.21.15.38:0/3971485929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98a3740 0x55e5c98cb500 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f381000a8a0 tx=0x7f381002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0742fcc560de6be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.008+0000 7f380d7fa640 1 -- 172.21.15.38:0/3971485929 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38100019b0 con 0x55e5c98a3740 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.008+0000 7f380d7fa640 1 -- 172.21.15.38:0/3971485929 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381002f070 con 0x55e5c98a3740 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.008+0000 7f380d7fa640 1 -- 172.21.15.38:0/3971485929 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810004da0 con 0x55e5c98a3740 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.009+0000 7f3820645740 1 -- 172.21.15.38:0/3971485929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98a3740 msgr2=0x55e5c98cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.009+0000 7f3820645740 1 --2- 172.21.15.38:0/3971485929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98a3740 0x55e5c98cb500 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f381000a8a0 tx=0x7f381002d000 comp rx=0 tx=0).stop 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.009+0000 7f3820645740 1 -- 172.21.15.38:0/3971485929 shutdown_connections 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.009+0000 7f3820645740 1 --2- 172.21.15.38:0/3971485929 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98a3740 0x55e5c98cb500 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.009+0000 7f3820645740 1 --2- 172.21.15.38:0/3971485929 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d3aa0 0x55e5c98a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.009+0000 7f3820645740 1 --2- 172.21.15.38:0/3971485929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 0x55e5c98d3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.009+0000 7f3820645740 1 -- 172.21.15.38:0/3971485929 >> 172.21.15.38:0/3971485929 conn(0x55e5c98a5470 msgr2=0x55e5c98ede90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.009+0000 7f3820645740 1 -- 172.21.15.38:0/3971485929 shutdown_connections 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.009+0000 7f3820645740 1 -- 172.21.15.38:0/3971485929 wait complete. 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.009+0000 7f3820645740 1 -- start start 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.009+0000 7f3820645740 1 Processor -- start 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3820645740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3080 0x55e5c9900ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3820645740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3aa0 0x55e5c98d2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3820645740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 0x55e5c98ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3820645740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5c98cf7d0 con 0x55e5c98d3080 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3820645740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5c97aa3d0 con 0x55e5c98d3aa0 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3820645740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5c98cfa70 con 0x55e5c98d2760 2024-04-25T08:37:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3820645740 10 client.0 _ll_register_callbacks cb 0x55e5c951fe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3816ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3080 0x55e5c9900ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3816ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3080 0x55e5c9900ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55272/0 (socket says 172.21.15.38:55272) 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3816ffd640 1 -- 172.21.15.38:0/2771019786 learned_addr learned my addr 172.21.15.38:0/2771019786 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f38177fe640 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 0x55e5c98ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f380ffff640 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3aa0 0x55e5c98d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3816ffd640 1 -- 172.21.15.38:0/2771019786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 msgr2=0x55e5c98ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3816ffd640 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 0x55e5c98ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3816ffd640 1 -- 172.21.15.38:0/2771019786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3aa0 msgr2=0x55e5c98d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.010+0000 7f3816ffd640 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3aa0 0x55e5c98d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f3816ffd640 1 -- 172.21.15.38:0/2771019786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5c94e0310 con 0x55e5c98d3080 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f3816ffd640 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3080 0x55e5c9900ea0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f3808058fb0 tx=0x7f3808058aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b534fe497f73e4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f37f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:37:38.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f37f67fc640 1 -- 172.21.15.38:0/2771019786 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3808086070 con 0x55e5c98d3080 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f37f67fc640 1 -- 172.21.15.38:0/2771019786 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f38080614a0 con 0x55e5c98d3080 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f3820645740 1 -- 172.21.15.38:0/2771019786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f381002f070 con 0x55e5c98d3080 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f37f67fc640 1 -- 172.21.15.38:0/2771019786 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3808058470 con 0x55e5c98d3080 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f3820645740 10 client.7599 fetch_fsmap learned FSMap version 51 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f3820645740 1 -- 172.21.15.38:0/2771019786 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55e5c964d1b0 con 0x55e5c98d3080 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f37f67fc640 1 -- 172.21.15.38:0/2771019786 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381002f070 con 0x55e5c98d3080 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f37f67fc640 1 -- 172.21.15.38:0/2771019786 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3808025030 con 0x55e5c98d3080 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f3820645740 10 client.7599 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f3820645740 10 client.7599 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f3820645740 20 client.7599 populate_metadata read hostname 'smithi038' 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f3820645740 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55e5c98cc410 0x55e5c98cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f3820645740 4 client.7599 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:38.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f3820645740 1 -- 172.21.15.38:0/2771019786 --> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e5c98195e0 con 0x55e5c98cc410 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.011+0000 7f380ffff640 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55e5c98cc410 0x55e5c98cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f380ffff640 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55e5c98cc410 0x55e5c98cc810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6ee55f596e4e66b server_cookie=582f2d60a1dd2218 in_seq=0 out_seq=0 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f37f67fc640 10 client.7599 ms_handle_connect on v2:172.21.15.38:6833/380751658 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f37f67fc640 1 -- 172.21.15.38:0/2771019786 <== mds.0 v2:172.21.15.38:6833/380751658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e5c98195e0 con 0x55e5c98cc410 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f37f67fc640 10 client.7599 handle_command_reply: tid=0 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f3820645740 1 client.7599 shutdown 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f3820645740 1 -- 172.21.15.38:0/2771019786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3080 msgr2=0x55e5c9900ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f3820645740 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3080 0x55e5c9900ea0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f3808058fb0 tx=0x7f3808058aa0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f3820645740 1 -- 172.21.15.38:0/2771019786 shutdown_connections 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f3820645740 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55e5c98cc410 0x55e5c98cc810 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f3820645740 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 0x55e5c98ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f3820645740 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3aa0 0x55e5c98d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f37f67fc640 0 client.7599 ms_handle_reset on v2:172.21.15.38:6833/380751658 2024-04-25T08:37:38.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f3820645740 1 --2- 172.21.15.38:0/2771019786 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3080 0x55e5c9900ea0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.012+0000 7f3820645740 1 -- 172.21.15.38:0/2771019786 >> 172.21.15.38:0/2771019786 conn(0x55e5c98a3740 msgr2=0x55e5c98a28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:38.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.013+0000 7f3820645740 1 -- 172.21.15.38:0/2771019786 shutdown_connections 2024-04-25T08:37:38.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.013+0000 7f3820645740 1 -- 172.21.15.38:0/2771019786 wait complete. 2024-04-25T08:37:38.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.013+0000 7f3820645740 20 client.7599 trim_cache size 0 max 16384 2024-04-25T08:37:38.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.028+0000 7f3820645740 1 Processor -- start 2024-04-25T08:37:38.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.028+0000 7f3820645740 1 -- start start 2024-04-25T08:37:38.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f3820645740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 0x55e5c98a9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f3820645740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 0x55e5c9907f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f3820645740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 0x55e5c98a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f3820645740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3818200860 con 0x55e5c98d3aa0 2024-04-25T08:37:38.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f3820645740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38181ffa30 con 0x55e5c98d3080 2024-04-25T08:37:38.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f3820645740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38181524e0 con 0x55e5c98d2760 2024-04-25T08:37:38.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f38177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 0x55e5c98a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f38177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 0x55e5c98a61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55278/0 (socket says 172.21.15.38:55278) 2024-04-25T08:37:38.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f38177fe640 1 -- 172.21.15.38:0/1879611256 learned_addr learned my addr 172.21.15.38:0/1879611256 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:38.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f38177fe640 1 -- 172.21.15.38:0/1879611256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 msgr2=0x55e5c98a9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f3816ffd640 1 --2- 172.21.15.38:0/1879611256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 0x55e5c98a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f380ffff640 1 --2- 172.21.15.38:0/1879611256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 0x55e5c9907f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f38177fe640 1 --2- 172.21.15.38:0/1879611256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 0x55e5c98a9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f38177fe640 1 -- 172.21.15.38:0/1879611256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 msgr2=0x55e5c9907f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f38177fe640 1 --2- 172.21.15.38:0/1879611256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 0x55e5c9907f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f38177fe640 1 -- 172.21.15.38:0/1879611256 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5c98ceb20 con 0x55e5c98d3aa0 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.029+0000 7f38177fe640 1 --2- 172.21.15.38:0/1879611256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 0x55e5c98a61b0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f381000a990 tx=0x7f3810009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829e9b5deb5c130d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.030+0000 7f380cff9640 1 -- 172.21.15.38:0/1879611256 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810004b60 con 0x55e5c98d3aa0 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.030+0000 7f380cff9640 1 -- 172.21.15.38:0/1879611256 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381002f040 con 0x55e5c98d3aa0 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.030+0000 7f380cff9640 1 -- 172.21.15.38:0/1879611256 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38100017b0 con 0x55e5c98d3aa0 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.030+0000 7f3820645740 1 -- 172.21.15.38:0/1879611256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 msgr2=0x55e5c98a61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.030+0000 7f3820645740 1 --2- 172.21.15.38:0/1879611256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 0x55e5c98a61b0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f381000a990 tx=0x7f3810009030 comp rx=0 tx=0).stop 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.030+0000 7f3820645740 1 -- 172.21.15.38:0/1879611256 shutdown_connections 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.030+0000 7f3820645740 1 --2- 172.21.15.38:0/1879611256 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 0x55e5c98a61b0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.030+0000 7f3820645740 1 --2- 172.21.15.38:0/1879611256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 0x55e5c9907f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.030+0000 7f3820645740 1 --2- 172.21.15.38:0/1879611256 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 0x55e5c98a9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.030+0000 7f3820645740 1 -- 172.21.15.38:0/1879611256 >> 172.21.15.38:0/1879611256 conn(0x55e5c98a3740 msgr2=0x55e5c98edc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.030+0000 7f3820645740 1 -- 172.21.15.38:0/1879611256 shutdown_connections 2024-04-25T08:37:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.030+0000 7f3820645740 1 -- 172.21.15.38:0/1879611256 wait complete. 2024-04-25T08:37:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.031+0000 7f3820645740 1 -- start start 2024-04-25T08:37:38.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.031+0000 7f3820645740 1 Processor -- start 2024-04-25T08:37:38.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f3820645740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 0x55e5c98edce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f3820645740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 0x55e5c9958ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f3820645740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 0x55e5c99593e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f3820645740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38181ff430 con 0x55e5c98d3aa0 2024-04-25T08:37:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f3820645740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5c98a6910 con 0x55e5c98d3080 2024-04-25T08:37:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f3820645740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e5c9959eb0 con 0x55e5c98d2760 2024-04-25T08:37:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f3820645740 10 client.0 _ll_register_callbacks cb 0x7f38040036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f38177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 0x55e5c99593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f38177fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 0x55e5c99593e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55288/0 (socket says 172.21.15.38:55288) 2024-04-25T08:37:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f38177fe640 1 -- 172.21.15.38:0/767105316 learned_addr learned my addr 172.21.15.38:0/767105316 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f3816ffd640 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 0x55e5c98edce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f380ffff640 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 0x55e5c9958ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f38177fe640 1 -- 172.21.15.38:0/767105316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 msgr2=0x55e5c98edce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f38177fe640 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 0x55e5c98edce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f38177fe640 1 -- 172.21.15.38:0/767105316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 msgr2=0x55e5c9958ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f38177fe640 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 0x55e5c9958ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f38177fe640 1 -- 172.21.15.38:0/767105316 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5c94b9490 con 0x55e5c98d3aa0 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f38177fe640 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 0x55e5c99593e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f3810043040 tx=0x7f3810001e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1945dd978e96efde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f37f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f37f67fc640 1 -- 172.21.15.38:0/767105316 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810045030 con 0x55e5c98d3aa0 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f37f67fc640 1 -- 172.21.15.38:0/767105316 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f38100019c0 con 0x55e5c98d3aa0 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f3820645740 1 -- 172.21.15.38:0/767105316 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f381002f040 con 0x55e5c98d3aa0 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f37f67fc640 1 -- 172.21.15.38:0/767105316 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f381005b050 con 0x55e5c98d3aa0 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f3820645740 10 client.7605 fetch_fsmap learned FSMap version 51 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.032+0000 7f3820645740 1 -- 172.21.15.38:0/767105316 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55e5c98ee220 con 0x55e5c98d3aa0 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.033+0000 7f37f67fc640 1 -- 172.21.15.38:0/767105316 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381002f040 con 0x55e5c98d3aa0 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.033+0000 7f37f67fc640 1 -- 172.21.15.38:0/767105316 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f381005e020 con 0x55e5c98d3aa0 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.033+0000 7f3820645740 10 client.7605 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.033+0000 7f3820645740 10 client.7605 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:37:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.033+0000 7f3820645740 20 client.7605 populate_metadata read hostname 'smithi038' 2024-04-25T08:37:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.033+0000 7f3820645740 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55e5c98fc2b0 0x55e5c98fc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.033+0000 7f3820645740 4 client.7605 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T08:37:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.033+0000 7f3820645740 1 -- 172.21.15.38:0/767105316 --> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x55e5c98ce320 con 0x55e5c98fc2b0 2024-04-25T08:37:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.033+0000 7f3816ffd640 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55e5c98fc2b0 0x55e5c98fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.033+0000 7f37f67fc640 10 client.7605 ms_handle_connect on v2:172.21.15.38:6833/380751658 2024-04-25T08:37:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.033+0000 7f3816ffd640 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55e5c98fc2b0 0x55e5c98fc690 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=972f7620fd2f3ac server_cookie=86e2e9791b12bde1 in_seq=0 out_seq=0 2024-04-25T08:37:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.040+0000 7f37f67fc640 1 -- 172.21.15.38:0/767105316 <== mds.0 v2:172.21.15.38:6833/380751658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e5c98ce320 con 0x55e5c98fc2b0 2024-04-25T08:37:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.040+0000 7f37f67fc640 10 client.7605 handle_command_reply: tid=0 2024-04-25T08:37:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.040+0000 7f3820645740 1 client.7605 shutdown 2024-04-25T08:37:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.040+0000 7f3820645740 1 -- 172.21.15.38:0/767105316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 msgr2=0x55e5c99593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.040+0000 7f3820645740 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 0x55e5c99593e0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f3810043040 tx=0x7f3810001e30 comp rx=0 tx=0).stop 2024-04-25T08:37:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f3820645740 1 -- 172.21.15.38:0/767105316 shutdown_connections 2024-04-25T08:37:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f3820645740 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55e5c98fc2b0 0x55e5c98fc690 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f3820645740 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e5c98d3aa0 0x55e5c99593e0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f3820645740 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e5c98d3080 0x55e5c9958ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f37f67fc640 0 client.7605 ms_handle_reset on v2:172.21.15.38:6833/380751658 2024-04-25T08:37:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f3820645740 1 --2- 172.21.15.38:0/767105316 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e5c98d2760 0x55e5c98edce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f3820645740 1 -- 172.21.15.38:0/767105316 >> 172.21.15.38:0/767105316 conn(0x55e5c98a3740 msgr2=0x55e5c98a8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f3820645740 1 -- 172.21.15.38:0/767105316 shutdown_connections 2024-04-25T08:37:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f3820645740 1 -- 172.21.15.38:0/767105316 wait complete. 2024-04-25T08:37:38.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f3820645740 20 client.7605 trim_cache size 0 max 16384 2024-04-25T08:37:38.037 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:37:38.037 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:37:38.037 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "f86291ca-6814-433f-a505-ae0ed12b213c", 2024-04-25T08:37:38.038 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:37:38.038 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:37:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f381d948640 1 -- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f37f806b4d0 msgr2=0x7f37f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f381d948640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f37f806b4d0 0x7f37f806d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f3818066ec0 tx=0x7f3800008040 comp rx=0 tx=0).stop 2024-04-25T08:37:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f381d948640 1 -- 172.21.15.38:0/1567467355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3804004d40 msgr2=0x7f381814ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f381d948640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3804004d40 0x7f381814ed90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f38080119a0 tx=0x7f3808011e60 comp rx=0 tx=0).stop 2024-04-25T08:37:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f381d948640 1 -- 172.21.15.38:0/1567467355 shutdown_connections 2024-04-25T08:37:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f381d948640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38180b31c0 0x7f38180c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.041+0000 7f381d948640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f37f806b4d0 0x7f37f806d990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.042+0000 7f381d948640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38180b5460 0x7f38181f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.042+0000 7f381d948640 1 --2- 172.21.15.38:0/1567467355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3804004d40 0x7f381814ed90 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.042+0000 7f381d948640 1 -- 172.21.15.38:0/1567467355 >> 172.21.15.38:0/1567467355 conn(0x7f3818072610 msgr2=0x7f3818160a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.042+0000 7f381d948640 1 -- 172.21.15.38:0/1567467355 shutdown_connections 2024-04-25T08:37:38.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.042+0000 7f381d948640 1 -- 172.21.15.38:0/1567467355 wait complete. 2024-04-25T08:37:38.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:37:38.047 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:37:38.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.209+0000 7f119c568640 1 Processor -- start 2024-04-25T08:37:38.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.210+0000 7f119c568640 1 -- start start 2024-04-25T08:37:38.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.211+0000 7f119c568640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11940d1e00 0x7f11940d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.211+0000 7f119c568640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f11940cf7b0 0x7f11940b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.211+0000 7f119c568640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11940ce710 con 0x7f119415de60 2024-04-25T08:37:38.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.211+0000 7f119c568640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11940ce4e0 con 0x7f11940cf7b0 2024-04-25T08:37:38.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.211+0000 7f119c568640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11940cf320 con 0x7f11940d1e00 2024-04-25T08:37:38.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.211+0000 7f119ab05640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f11940cf7b0 0x7f11940b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.211+0000 7f1199b03640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f119415de60 0x7f11940d21e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56328/0 (socket says 172.21.15.38:56328) 2024-04-25T08:37:38.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.211+0000 7f1199b03640 1 -- 172.21.15.38:0/736808588 learned_addr learned my addr 172.21.15.38:0/736808588 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:37:38.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.211+0000 7f119a304640 1 --2- 172.21.15.38:0/736808588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11940d1e00 0x7f11940d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.212+0000 7f119ab05640 1 -- 172.21.15.38:0/736808588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11940d1e00 msgr2=0x7f11940d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.212+0000 7f119ab05640 1 --2- 172.21.15.38:0/736808588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11940d1e00 0x7f11940d42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.212+0000 7f1199302640 1 -- 172.21.15.38:0/736808588 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187955650 0 0) 0x7f11940ce710 con 0x7f119415de60 2024-04-25T08:37:38.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.212+0000 7f119ab05640 1 -- 172.21.15.38:0/736808588 >> v1:172.21.15.38:6789/0 conn(0x7f119415de60 legacy=0x7f11940d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.212+0000 7f119ab05640 1 -- 172.21.15.38:0/736808588 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11940ce9f0 con 0x7f11940cf7b0 2024-04-25T08:37:38.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.213+0000 7f119ab05640 1 --2- 172.21.15.38:0/736808588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11940cf7b0 0x7f11940b3180 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f118c009d90 tx=0x7f118c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f8f3183c5f4e0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.213+0000 7f1199302640 1 -- 172.21.15.38:0/736808588 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c007740 con 0x7f11940cf7b0 2024-04-25T08:37:38.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.213+0000 7f1199302640 1 -- 172.21.15.38:0/736808588 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118c0078e0 con 0x7f11940cf7b0 2024-04-25T08:37:38.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.213+0000 7f1199302640 1 -- 172.21.15.38:0/736808588 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c007cb0 con 0x7f11940cf7b0 2024-04-25T08:37:38.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.214+0000 7f119c568640 1 -- 172.21.15.38:0/736808588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11940cf7b0 msgr2=0x7f11940b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.214+0000 7f119c568640 1 --2- 172.21.15.38:0/736808588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11940cf7b0 0x7f11940b3180 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f118c009d90 tx=0x7f118c02a660 comp rx=0 tx=0).stop 2024-04-25T08:37:38.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.214+0000 7f119c568640 1 -- 172.21.15.38:0/736808588 shutdown_connections 2024-04-25T08:37:38.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.214+0000 7f119c568640 1 --2- 172.21.15.38:0/736808588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11940cf7b0 0x7f11940b3180 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.214+0000 7f119c568640 1 --2- 172.21.15.38:0/736808588 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11940d1e00 0x7f11940d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.214+0000 7f119c568640 1 -- 172.21.15.38:0/736808588 >> 172.21.15.38:0/736808588 conn(0x7f119408b000 msgr2=0x7f11940d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:38.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.214+0000 7f119c568640 1 -- 172.21.15.38:0/736808588 shutdown_connections 2024-04-25T08:37:38.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.214+0000 7f119c568640 1 -- 172.21.15.38:0/736808588 wait complete. 2024-04-25T08:37:38.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.216+0000 7f119c568640 1 Processor -- start 2024-04-25T08:37:38.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.216+0000 7f119c568640 1 -- start start 2024-04-25T08:37:38.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f119c568640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11940cf7b0 0x7f11941533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f119c568640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11940d1e00 0x7f119415e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f119c568640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f119415de60 0x7f11941602f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f119c568640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11940d1600 con 0x7f11940d1e00 2024-04-25T08:37:38.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f119c568640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11940cefb0 con 0x7f119415de60 2024-04-25T08:37:38.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f119c568640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11940d1780 con 0x7f11940cf7b0 2024-04-25T08:37:38.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f1199b03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11940d1e00 0x7f119415e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f119a304640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11940cf7b0 0x7f11941533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f119a304640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11940cf7b0 0x7f11941533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:54882/0 (socket says 172.21.15.38:54882) 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f119a304640 1 -- 172.21.15.38:0/1666881479 learned_addr learned my addr 172.21.15.38:0/1666881479 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f1199b03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11940d1e00 0x7f119415e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55308/0 (socket says 172.21.15.38:55308) 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f119ab05640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f119415de60 0x7f11941602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.217+0000 7f119a304640 1 -- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f119415de60 msgr2=0x7f11941602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.218+0000 7f119a304640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f119415de60 0x7f11941602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.218+0000 7f119a304640 1 -- 172.21.15.38:0/1666881479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11940d1e00 msgr2=0x7f119415e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.218+0000 7f119a304640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11940d1e00 0x7f119415e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.218+0000 7f119a304640 1 -- 172.21.15.38:0/1666881479 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119415ed10 con 0x7f11940cf7b0 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.218+0000 7f1199b03640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11940d1e00 0x7f119415e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.218+0000 7f119a304640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11940cf7b0 0x7f11941533d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f119000e9b0 tx=0x7f119000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.218+0000 7f118affd640 1 -- 172.21.15.38:0/1666881479 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190017070 con 0x7f11940cf7b0 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.218+0000 7f118affd640 1 -- 172.21.15.38:0/1666881479 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119000c9a0 con 0x7f11940cf7b0 2024-04-25T08:37:38.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.218+0000 7f119c568640 1 -- 172.21.15.38:0/1666881479 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119415eff0 con 0x7f11940cf7b0 2024-04-25T08:37:38.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.218+0000 7f118affd640 1 -- 172.21.15.38:0/1666881479 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119000cd90 con 0x7f11940cf7b0 2024-04-25T08:37:38.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.218+0000 7f119c568640 1 -- 172.21.15.38:0/1666881479 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119415f5b0 con 0x7f11940cf7b0 2024-04-25T08:37:38.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.220+0000 7f118affd640 1 -- 172.21.15.38:0/1666881479 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f119001a070 con 0x7f11940cf7b0 2024-04-25T08:37:38.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.221+0000 7f118affd640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f117806b500 0x7f117806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.221+0000 7f118affd640 1 -- 172.21.15.38:0/1666881479 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f119008a610 con 0x7f11940cf7b0 2024-04-25T08:37:38.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.221+0000 7f1199b03640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f117806b500 0x7f117806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.222+0000 7f1199b03640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f117806b500 0x7f117806d9c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f11940b9870 tx=0x7f1184007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.223+0000 7f119f262740 1 Processor -- start 2024-04-25T08:37:38.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.223+0000 7f119f262740 1 -- start start 2024-04-25T08:37:38.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.224+0000 7f119f262740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c161b10 0x55778c164320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.224+0000 7f119f262740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161ef0 0x55778c164860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.224+0000 7f119f262740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c134770 0x55778c137f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.224+0000 7f119f262740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778c09dda0 con 0x55778c161ef0 2024-04-25T08:37:38.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.224+0000 7f119f262740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778c046b80 con 0x55778c161b10 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.224+0000 7f119f262740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55778c099e40 con 0x55778c134770 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.224+0000 7f119a304640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c161b10 0x55778c164320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.224+0000 7f119a304640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c161b10 0x55778c164320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:54094/0 (socket says 172.21.15.38:54094) 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.224+0000 7f119a304640 1 -- 172.21.15.38:0/3740705745 learned_addr learned my addr 172.21.15.38:0/3740705745 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.224+0000 7f1199b03640 1 --2- 172.21.15.38:0/3740705745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161ef0 0x55778c164860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.224+0000 7f119ab05640 1 --2- 172.21.15.38:0/3740705745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c134770 0x55778c137f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.225+0000 7f119a304640 1 -- 172.21.15.38:0/3740705745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c134770 msgr2=0x55778c137f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.225+0000 7f119a304640 1 --2- 172.21.15.38:0/3740705745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c134770 0x55778c137f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.225+0000 7f119a304640 1 -- 172.21.15.38:0/3740705745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161ef0 msgr2=0x55778c164860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.225+0000 7f119a304640 1 --2- 172.21.15.38:0/3740705745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161ef0 0x55778c164860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.225+0000 7f119a304640 1 -- 172.21.15.38:0/3740705745 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55778c050f10 con 0x55778c161b10 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.225+0000 7f119a304640 1 --2- 172.21.15.38:0/3740705745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c161b10 0x55778c164320 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1190055120 tx=0x7f1190058bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=297c3f5a12560bde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.225+0000 7f1177fff640 1 -- 172.21.15.38:0/3740705745 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190021070 con 0x55778c161b10 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.225+0000 7f1177fff640 1 -- 172.21.15.38:0/3740705745 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1190061320 con 0x55778c161b10 2024-04-25T08:37:38.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.225+0000 7f1177fff640 1 -- 172.21.15.38:0/3740705745 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190055330 con 0x55778c161b10 2024-04-25T08:37:38.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.226+0000 7f119f262740 1 -- 172.21.15.38:0/3740705745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c161b10 msgr2=0x55778c164320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.226+0000 7f119f262740 1 --2- 172.21.15.38:0/3740705745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c161b10 0x55778c164320 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1190055120 tx=0x7f1190058bc0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.226+0000 7f119f262740 1 -- 172.21.15.38:0/3740705745 shutdown_connections 2024-04-25T08:37:38.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.226+0000 7f119f262740 1 --2- 172.21.15.38:0/3740705745 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c134770 0x55778c137f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.226+0000 7f119f262740 1 --2- 172.21.15.38:0/3740705745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161ef0 0x55778c164860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.226+0000 7f119f262740 1 --2- 172.21.15.38:0/3740705745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c161b10 0x55778c164320 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.226+0000 7f119f262740 1 -- 172.21.15.38:0/3740705745 >> 172.21.15.38:0/3740705745 conn(0x55778c1369e0 msgr2=0x55778c17f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:38.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.227+0000 7f119f262740 1 -- 172.21.15.38:0/3740705745 shutdown_connections 2024-04-25T08:37:38.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.227+0000 7f119f262740 1 -- 172.21.15.38:0/3740705745 wait complete. 2024-04-25T08:37:38.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.228+0000 7f119f262740 1 -- start start 2024-04-25T08:37:38.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.228+0000 7f119f262740 1 Processor -- start 2024-04-25T08:37:38.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f119f262740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161b10 0x55778c17b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f119f262740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161ef0 0x55778c182fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f119f262740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c178620 0x55778c19b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f119f262740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778c139b90 con 0x55778c161ef0 2024-04-25T08:37:38.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f119f262740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778c03c4b0 con 0x55778c178620 2024-04-25T08:37:38.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f119f262740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55778c139e70 con 0x55778c161b10 2024-04-25T08:37:38.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f119f262740 10 client.0 _ll_register_callbacks cb 0x55778c07c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:38.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f119a304640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161b10 0x55778c17b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f119ab05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c178620 0x55778c19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f1199b03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161ef0 0x55778c182fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f1199b03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161ef0 0x55778c182fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55322/0 (socket says 172.21.15.38:55322) 2024-04-25T08:37:38.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f119a304640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161b10 0x55778c17b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:54898/0 (socket says 172.21.15.38:54898) 2024-04-25T08:37:38.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.229+0000 7f1199b03640 1 -- 172.21.15.38:0/1297963572 learned_addr learned my addr 172.21.15.38:0/1297963572 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:38.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.230+0000 7f119ab05640 1 -- 172.21.15.38:0/1297963572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161b10 msgr2=0x55778c17b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.230+0000 7f119ab05640 1 --2- 172.21.15.38:0/1297963572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161b10 0x55778c17b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.230+0000 7f119ab05640 1 -- 172.21.15.38:0/1297963572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161ef0 msgr2=0x55778c182fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.230+0000 7f119ab05640 1 --2- 172.21.15.38:0/1297963572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161ef0 0x55778c182fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.230+0000 7f119ab05640 1 -- 172.21.15.38:0/1297963572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55778be7c7d0 con 0x55778c178620 2024-04-25T08:37:38.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.230+0000 7f119ab05640 1 --2- 172.21.15.38:0/1297963572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c178620 0x55778c19b660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1194069220 tx=0x7f118c02c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f01f223f57b1ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.230+0000 7f11757fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-04-25T08:37:38.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.230+0000 7f11757fa640 1 -- 172.21.15.38:0/1297963572 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118c045410 con 0x55778c178620 2024-04-25T08:37:38.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.230+0000 7f11757fa640 1 -- 172.21.15.38:0/1297963572 <== mon.1 v2:172.21.15.107:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f118c0455b0 con 0x55778c178620 2024-04-25T08:37:38.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.230+0000 7f119f262740 1 -- 172.21.15.38:0/1297963572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1190061320 con 0x55778c178620 2024-04-25T08:37:38.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.230+0000 7f11757fa640 1 -- 172.21.15.38:0/1297963572 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f118c04f050 con 0x55778c178620 2024-04-25T08:37:38.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.231+0000 7f119f262740 10 client.6727 fetch_fsmap learned FSMap version 51 2024-04-25T08:37:38.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.231+0000 7f119f262740 1 -- 172.21.15.38:0/1297963572 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55778bdb23f0 con 0x55778c178620 2024-04-25T08:37:38.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.231+0000 7f11757fa640 1 -- 172.21.15.38:0/1297963572 <== mon.1 v2:172.21.15.107:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1190061320 con 0x55778c178620 2024-04-25T08:37:38.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.231+0000 7f11757fa640 1 -- 172.21.15.38:0/1297963572 <== mon.1 v2:172.21.15.107:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f118c052020 con 0x55778c178620 2024-04-25T08:37:38.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.231+0000 7f119f262740 10 client.6727 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:37:38.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.231+0000 7f119f262740 10 client.6727 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:37:38.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.231+0000 7f119f262740 20 client.6727 populate_metadata read hostname 'smithi038' 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.231+0000 7f119f262740 1 --2- 172.21.15.38:0/1297963572 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55778c163dc0 0x55778c180450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.231+0000 7f119f262740 4 client.6727 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.231+0000 7f119f262740 1 -- 172.21.15.38:0/1297963572 --> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55778c0ab2e0 con 0x55778c163dc0 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.231+0000 7f119a304640 1 --2- 172.21.15.38:0/1297963572 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55778c163dc0 0x55778c180450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.232+0000 7f11757fa640 10 client.6727 ms_handle_connect on v2:172.21.15.38:6833/380751658 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.232+0000 7f119a304640 1 --2- 172.21.15.38:0/1297963572 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55778c163dc0 0x55778c180450 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303c63add8778f13 server_cookie=c922ce2f73954709 in_seq=0 out_seq=0 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.233+0000 7f11757fa640 1 -- 172.21.15.38:0/1297963572 <== mds.0 v2:172.21.15.38:6833/380751658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55778c0ab2e0 con 0x55778c163dc0 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.233+0000 7f11757fa640 10 client.6727 handle_command_reply: tid=0 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.233+0000 7f119f262740 1 client.6727 shutdown 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.233+0000 7f119f262740 1 -- 172.21.15.38:0/1297963572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c178620 msgr2=0x55778c19b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.234+0000 7f119f262740 1 --2- 172.21.15.38:0/1297963572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c178620 0x55778c19b660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1194069220 tx=0x7f118c02c000 comp rx=0 tx=0).stop 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.234+0000 7f119f262740 1 -- 172.21.15.38:0/1297963572 shutdown_connections 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.234+0000 7f119f262740 1 --2- 172.21.15.38:0/1297963572 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55778c163dc0 0x55778c180450 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.234+0000 7f11757fa640 0 client.6727 ms_handle_reset on v2:172.21.15.38:6833/380751658 2024-04-25T08:37:38.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.234+0000 7f119f262740 1 --2- 172.21.15.38:0/1297963572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c178620 0x55778c19b660 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.234+0000 7f119f262740 1 --2- 172.21.15.38:0/1297963572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161ef0 0x55778c182fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.234+0000 7f119f262740 1 --2- 172.21.15.38:0/1297963572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161b10 0x55778c17b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.234+0000 7f119f262740 1 -- 172.21.15.38:0/1297963572 >> 172.21.15.38:0/1297963572 conn(0x55778c134770 msgr2=0x55778c139330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:38.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.234+0000 7f119f262740 1 -- 172.21.15.38:0/1297963572 shutdown_connections 2024-04-25T08:37:38.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.234+0000 7f119f262740 1 -- 172.21.15.38:0/1297963572 wait complete. 2024-04-25T08:37:38.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.234+0000 7f119f262740 20 client.6727 trim_cache size 0 max 16384 2024-04-25T08:37:38.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.258+0000 7f119f262740 1 Processor -- start 2024-04-25T08:37:38.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.258+0000 7f119f262740 1 -- start start 2024-04-25T08:37:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.259+0000 7f119f262740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 0x55778c134e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.259+0000 7f119f262740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 0x55778c138110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.259+0000 7f119f262740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c163dc0 0x55778c138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.259+0000 7f119f262740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11940704f0 con 0x55778c161b10 2024-04-25T08:37:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.259+0000 7f119f262740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119406f710 con 0x55778c163dc0 2024-04-25T08:37:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.259+0000 7f119f262740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11940b7830 con 0x55778c161ef0 2024-04-25T08:37:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.260+0000 7f119ab05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c163dc0 0x55778c138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.260+0000 7f1199b03640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 0x55778c138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.260+0000 7f1199b03640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 0x55778c138110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:54912/0 (socket says 172.21.15.38:54912) 2024-04-25T08:37:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.260+0000 7f1199b03640 1 -- 172.21.15.38:0/3761661736 learned_addr learned my addr 172.21.15.38:0/3761661736 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:38.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.260+0000 7f119a304640 1 --2- 172.21.15.38:0/3761661736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 0x55778c134e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.260+0000 7f1199b03640 1 -- 172.21.15.38:0/3761661736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c163dc0 msgr2=0x55778c138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.260+0000 7f1199b03640 1 --2- 172.21.15.38:0/3761661736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c163dc0 0x55778c138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.260+0000 7f1199b03640 1 -- 172.21.15.38:0/3761661736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 msgr2=0x55778c134e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.261+0000 7f1199b03640 1 --2- 172.21.15.38:0/3761661736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 0x55778c134e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.261+0000 7f1199b03640 1 -- 172.21.15.38:0/3761661736 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55778c175d20 con 0x55778c161ef0 2024-04-25T08:37:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.261+0000 7f119a304640 1 --2- 172.21.15.38:0/3761661736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 0x55778c134e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:37:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.261+0000 7f1199b03640 1 --2- 172.21.15.38:0/3761661736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 0x55778c138110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1184013550 tx=0x7f1184013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8bc2c326902b7788 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.261+0000 7f11777fe640 1 -- 172.21.15.38:0/3761661736 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11840226c0 con 0x55778c161ef0 2024-04-25T08:37:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.261+0000 7f11777fe640 1 -- 172.21.15.38:0/3761661736 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1184022860 con 0x55778c161ef0 2024-04-25T08:37:38.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.261+0000 7f11777fe640 1 -- 172.21.15.38:0/3761661736 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1184022c10 con 0x55778c161ef0 2024-04-25T08:37:38.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.262+0000 7f119f262740 1 -- 172.21.15.38:0/3761661736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 msgr2=0x55778c138110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.262+0000 7f119f262740 1 --2- 172.21.15.38:0/3761661736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 0x55778c138110 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1184013550 tx=0x7f1184013000 comp rx=0 tx=0).stop 2024-04-25T08:37:38.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.262+0000 7f119f262740 1 -- 172.21.15.38:0/3761661736 shutdown_connections 2024-04-25T08:37:38.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.262+0000 7f119f262740 1 --2- 172.21.15.38:0/3761661736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c163dc0 0x55778c138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.262+0000 7f119f262740 1 --2- 172.21.15.38:0/3761661736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 0x55778c138110 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.262+0000 7f119f262740 1 --2- 172.21.15.38:0/3761661736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 0x55778c134e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.262+0000 7f119f262740 1 -- 172.21.15.38:0/3761661736 >> 172.21.15.38:0/3761661736 conn(0x55778c134770 msgr2=0x55778c178e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.262+0000 7f119f262740 1 -- 172.21.15.38:0/3761661736 shutdown_connections 2024-04-25T08:37:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.262+0000 7f119f262740 1 -- 172.21.15.38:0/3761661736 wait complete. 2024-04-25T08:37:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.263+0000 7f119f262740 1 -- start start 2024-04-25T08:37:38.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.263+0000 7f119f262740 1 Processor -- start 2024-04-25T08:37:38.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.264+0000 7f119f262740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 0x55778c1806f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.264+0000 7f119f262740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 0x55778c19b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119a304640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 0x55778c1806f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119f262740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c163dc0 0x55778c1f2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119a304640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 0x55778c1806f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55338/0 (socket says 172.21.15.38:55338) 2024-04-25T08:37:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119a304640 1 -- 172.21.15.38:0/4191338656 learned_addr learned my addr 172.21.15.38:0/4191338656 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:37:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119f262740 1 -- 172.21.15.38:0/4191338656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55778c0ab2e0 con 0x55778c161b10 2024-04-25T08:37:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119f262740 1 -- 172.21.15.38:0/4191338656 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119406f110 con 0x55778c163dc0 2024-04-25T08:37:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119f262740 1 -- 172.21.15.38:0/4191338656 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55778c13b110 con 0x55778c161ef0 2024-04-25T08:37:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119ab05640 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c163dc0 0x55778c1f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119f262740 10 client.0 _ll_register_callbacks cb 0x7f1190009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:37:38.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f1199b03640 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 0x55778c19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119a304640 1 -- 172.21.15.38:0/4191338656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 msgr2=0x55778c19b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119a304640 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 0x55778c19b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119a304640 1 -- 172.21.15.38:0/4191338656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c163dc0 msgr2=0x55778c1f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:37:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119a304640 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c163dc0 0x55778c1f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.265+0000 7f119a304640 1 -- 172.21.15.38:0/4191338656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55778bd720f0 con 0x55778c161b10 2024-04-25T08:37:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.266+0000 7f119a304640 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 0x55778c1806f0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1190055120 tx=0x7f1190000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffc6f37abe399b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:37:38.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.266+0000 7f11757fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.266+0000 7f1199b03640 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 0x55778c19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.266+0000 7f11757fa640 1 -- 172.21.15.38:0/4191338656 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190061320 con 0x55778c161b10 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.266+0000 7f11757fa640 1 -- 172.21.15.38:0/4191338656 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1190055330 con 0x55778c161b10 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.266+0000 7f119f262740 1 -- 172.21.15.38:0/4191338656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1184022860 con 0x55778c161b10 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.266+0000 7f11757fa640 1 -- 172.21.15.38:0/4191338656 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f119006cbd0 con 0x55778c161b10 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.266+0000 7f119f262740 10 client.7626 fetch_fsmap learned FSMap version 51 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.266+0000 7f119f262740 1 -- 172.21.15.38:0/4191338656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55778c179110 con 0x55778c161b10 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.267+0000 7f11757fa640 1 -- 172.21.15.38:0/4191338656 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1184022860 con 0x55778c161b10 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.267+0000 7f11757fa640 1 -- 172.21.15.38:0/4191338656 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1190077b60 con 0x55778c161b10 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.267+0000 7f119f262740 10 client.7626 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.267+0000 7f119f262740 10 client.7626 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.267+0000 7f119f262740 20 client.7626 populate_metadata read hostname 'smithi038' 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.267+0000 7f119f262740 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55778c178620 0x55778c133600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.267+0000 7f119f262740 4 client.7626 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:37:38.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.267+0000 7f119f262740 1 -- 172.21.15.38:0/4191338656 --> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55778c0a5630 con 0x55778c178620 2024-04-25T08:37:38.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.267+0000 7f1199b03640 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55778c178620 0x55778c133600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:37:38.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.268+0000 7f11757fa640 10 client.7626 ms_handle_connect on v2:172.21.15.38:6833/380751658 2024-04-25T08:37:38.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.268+0000 7f1199b03640 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55778c178620 0x55778c133600 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd84626206abf6b server_cookie=da495b5da2f8f392 in_seq=0 out_seq=0 2024-04-25T08:37:38.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.268+0000 7f11757fa640 1 -- 172.21.15.38:0/4191338656 <== mds.0 v2:172.21.15.38:6833/380751658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x55778c0a5630 con 0x55778c178620 2024-04-25T08:37:38.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.268+0000 7f11757fa640 10 client.7626 handle_command_reply: tid=0 2024-04-25T08:37:38.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.268+0000 7f119f262740 1 client.7626 shutdown 2024-04-25T08:37:38.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.269+0000 7f119f262740 1 -- 172.21.15.38:0/4191338656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 msgr2=0x55778c1806f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.269+0000 7f119f262740 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 0x55778c1806f0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f1190055120 tx=0x7f1190000bc0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.269+0000 7f119f262740 1 -- 172.21.15.38:0/4191338656 shutdown_connections 2024-04-25T08:37:38.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.269+0000 7f119f262740 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55778c178620 0x55778c133600 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.269+0000 7f119f262740 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55778c163dc0 0x55778c1f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.269+0000 7f119f262740 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55778c161ef0 0x55778c19b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.269+0000 7f119f262740 1 --2- 172.21.15.38:0/4191338656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55778c161b10 0x55778c1806f0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.269+0000 7f119f262740 1 -- 172.21.15.38:0/4191338656 >> 172.21.15.38:0/4191338656 conn(0x55778c134770 msgr2=0x55778c178a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:38.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.269+0000 7f11757fa640 0 client.7626 ms_handle_reset on v2:172.21.15.38:6833/380751658 2024-04-25T08:37:38.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.269+0000 7f119f262740 1 -- 172.21.15.38:0/4191338656 shutdown_connections 2024-04-25T08:37:38.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.269+0000 7f119f262740 1 -- 172.21.15.38:0/4191338656 wait complete. 2024-04-25T08:37:38.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.269+0000 7f119f262740 20 client.7626 trim_cache size 0 max 16384 2024-04-25T08:37:38.268 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:37:38.269 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "scrub active (3 inodes in the stack)", 2024-04-25T08:37:38.269 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-04-25T08:37:38.269 INFO:teuthology.orchestra.run.smithi038.stdout: "f86291ca-6814-433f-a505-ae0ed12b213c": { 2024-04-25T08:37:38.269 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "~mds0", 2024-04-25T08:37:38.269 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "f86291ca-6814-433f-a505-ae0ed12b213c", 2024-04-25T08:37:38.269 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive" 2024-04-25T08:37:38.269 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:37:38.269 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:37:38.269 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:37:38.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.270+0000 7f119c568640 1 -- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f117806b500 msgr2=0x7f117806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.270+0000 7f119c568640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f117806b500 0x7f117806d9c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f11940b9870 tx=0x7f1184007d50 comp rx=0 tx=0).stop 2024-04-25T08:37:38.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.270+0000 7f119c568640 1 -- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11940cf7b0 msgr2=0x7f11941533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:37:38.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.270+0000 7f119c568640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11940cf7b0 0x7f11941533d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f119000e9b0 tx=0x7f119000ee70 comp rx=0 tx=0).stop 2024-04-25T08:37:38.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.270+0000 7f119c568640 1 -- 172.21.15.38:0/1666881479 shutdown_connections 2024-04-25T08:37:38.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.270+0000 7f119c568640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f119415de60 0x7f11941602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.270+0000 7f119c568640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f11940d1e00 0x7f119415e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.270+0000 7f119c568640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f117806b500 0x7f117806d9c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.270+0000 7f119c568640 1 --2- 172.21.15.38:0/1666881479 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f11940cf7b0 0x7f11941533d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:37:38.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.270+0000 7f119c568640 1 -- 172.21.15.38:0/1666881479 >> 172.21.15.38:0/1666881479 conn(0x7f119408b000 msgr2=0x7f119406bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:37:38.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.270+0000 7f119c568640 1 -- 172.21.15.38:0/1666881479 shutdown_connections 2024-04-25T08:37:38.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:37:38.270+0000 7f119c568640 1 -- 172.21.15.38:0/1666881479 wait complete. 2024-04-25T08:37:38.275 INFO:tasks.cephfs.filesystem:scrub status for tag:f86291ca-6814-433f-a505-ae0ed12b213c - {'path': '~mds0', 'tag': 'f86291ca-6814-433f-a505-ae0ed12b213c', 'options': 'recursive'} 2024-04-25T08:38:03.877 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:38:03.881 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:38:08.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:08.276 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:38:08.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.399+0000 7f6ec13e2640 1 Processor -- start 2024-04-25T08:38:08.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.399+0000 7f6ec13e2640 1 -- start start 2024-04-25T08:38:08.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.400+0000 7f6ec13e2640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6ebc0d0320 0x7f6ebc0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.400+0000 7f6ec13e2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ebc0d0700 0x7f6ebc0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.400+0000 7f6ec13e2640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc0d2330 con 0x7f6ebc0d1e90 2024-04-25T08:38:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.400+0000 7f6ec13e2640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc0d2610 con 0x7f6ebc0d0320 2024-04-25T08:38:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.400+0000 7f6ec13e2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0d28f0 con 0x7f6ebc0d0700 2024-04-25T08:38:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.400+0000 7f6ebb7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6ebc0d1e90 0x7f6ebc0b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45544/0 (socket says 172.21.15.38:45544) 2024-04-25T08:38:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.400+0000 7f6ebb7fe640 1 -- 172.21.15.38:0/1309396467 learned_addr learned my addr 172.21.15.38:0/1309396467 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.400+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/1309396467 >> v2:172.21.15.107:3300/0 conn(0x7f6ebc0d0320 0x7f6ebc0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.400+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/1309396467 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ebc0d0700 0x7f6ebc0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.400+0000 7f6eb9ffb640 1 -- 172.21.15.38:0/1309396467 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809451816 0 0) 0x7f6ebc0d2330 con 0x7f6ebc0d1e90 2024-04-25T08:38:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.401+0000 7f6eb9ffb640 1 -- 172.21.15.38:0/1309396467 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea4003520 con 0x7f6ebc0d1e90 2024-04-25T08:38:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.401+0000 7f6eba7fc640 1 -- 172.21.15.38:0/1309396467 >> v2:172.21.15.107:3300/0 conn(0x7f6ebc0d0320 msgr2=0x7f6ebc0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.401+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/1309396467 >> v2:172.21.15.107:3300/0 conn(0x7f6ebc0d0320 0x7f6ebc0d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.401+0000 7f6eba7fc640 1 -- 172.21.15.38:0/1309396467 >> v1:172.21.15.38:6789/0 conn(0x7f6ebc0d1e90 legacy=0x7f6ebc0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.401+0000 7f6eba7fc640 1 -- 172.21.15.38:0/1309396467 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc0d2bd0 con 0x7f6ebc0d0700 2024-04-25T08:38:08.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.401+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/1309396467 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ebc0d0700 0x7f6ebc0d1950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00097a0 tx=0x7f6eb002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1325d11ce2f09ead server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.402+0000 7f6eb9ffb640 1 -- 172.21.15.38:0/1309396467 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0037070 con 0x7f6ebc0d0700 2024-04-25T08:38:08.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.402+0000 7f6eb9ffb640 1 -- 172.21.15.38:0/1309396467 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb0032440 con 0x7f6ebc0d0700 2024-04-25T08:38:08.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.402+0000 7f6eb9ffb640 1 -- 172.21.15.38:0/1309396467 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00327f0 con 0x7f6ebc0d0700 2024-04-25T08:38:08.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.402+0000 7f6ec13e2640 1 -- 172.21.15.38:0/1309396467 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ebc0d0700 msgr2=0x7f6ebc0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.402+0000 7f6ec13e2640 1 --2- 172.21.15.38:0/1309396467 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ebc0d0700 0x7f6ebc0d1950 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00097a0 tx=0x7f6eb002a210 comp rx=0 tx=0).stop 2024-04-25T08:38:08.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.402+0000 7f6ec13e2640 1 -- 172.21.15.38:0/1309396467 shutdown_connections 2024-04-25T08:38:08.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.402+0000 7f6ec13e2640 1 --2- 172.21.15.38:0/1309396467 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ebc0d0700 0x7f6ebc0d1950 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.402+0000 7f6ec13e2640 1 --2- 172.21.15.38:0/1309396467 >> v2:172.21.15.107:3300/0 conn(0x7f6ebc0d0320 0x7f6ebc0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.402+0000 7f6ec13e2640 1 -- 172.21.15.38:0/1309396467 >> 172.21.15.38:0/1309396467 conn(0x7f6ebc0668f0 msgr2=0x7f6ebc066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.402+0000 7f6ec13e2640 1 -- 172.21.15.38:0/1309396467 shutdown_connections 2024-04-25T08:38:08.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.402+0000 7f6ec13e2640 1 -- 172.21.15.38:0/1309396467 wait complete. 2024-04-25T08:38:08.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.404+0000 7f6ec13e2640 1 Processor -- start 2024-04-25T08:38:08.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.404+0000 7f6ec13e2640 1 -- start start 2024-04-25T08:38:08.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.404+0000 7f6ec13e2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ebc0d0320 0x7f6ebc0abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.404+0000 7f6ec13e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ebc0d0700 0x7f6ebc1f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.404+0000 7f6ec13e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ebc0d1e90 0x7f6ebc1f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.404+0000 7f6ec13e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0d0b40 con 0x7f6ebc0d0700 2024-04-25T08:38:08.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.404+0000 7f6ec13e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc161210 con 0x7f6ebc0d1e90 2024-04-25T08:38:08.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.404+0000 7f6ec13e2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc0b57d0 con 0x7f6ebc0d0320 2024-04-25T08:38:08.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6eba7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ebc0d0700 0x7f6ebc1f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6eba7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ebc0d0700 0x7f6ebc1f3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49594/0 (socket says 172.21.15.38:49594) 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6eba7fc640 1 -- 172.21.15.38:0/4283141877 learned_addr learned my addr 172.21.15.38:0/4283141877 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ebc0d0320 0x7f6ebc0abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6ebb7fe640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ebc0d1e90 0x7f6ebc1f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6eba7fc640 1 -- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ebc0d0320 msgr2=0x7f6ebc0abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ebc0d0320 0x7f6ebc0abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6eba7fc640 1 -- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ebc0d1e90 msgr2=0x7f6ebc1f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ebc0d1e90 0x7f6ebc1f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6eba7fc640 1 -- 172.21.15.38:0/4283141877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc0aae80 con 0x7f6ebc0d0700 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ebc0d0700 0x7f6ebc1f3aa0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00098c0 tx=0x7f6eb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6ea37fe640 1 -- 172.21.15.38:0/4283141877 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0037040 con 0x7f6ebc0d0700 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6ea37fe640 1 -- 172.21.15.38:0/4283141877 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb00044c0 con 0x7f6ebc0d0700 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6ec13e2640 1 -- 172.21.15.38:0/4283141877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc0ab160 con 0x7f6ebc0d0700 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6ea37fe640 1 -- 172.21.15.38:0/4283141877 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb00325f0 con 0x7f6ebc0d0700 2024-04-25T08:38:08.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.405+0000 7f6ec13e2640 1 -- 172.21.15.38:0/4283141877 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc0aba00 con 0x7f6ebc0d0700 2024-04-25T08:38:08.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.407+0000 7f6ea37fe640 1 -- 172.21.15.38:0/4283141877 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6eb0044020 con 0x7f6ebc0d0700 2024-04-25T08:38:08.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.407+0000 7f6ea37fe640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ea806b4d0 0x7f6ea806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.407+0000 7f6ea37fe640 1 -- 172.21.15.38:0/4283141877 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f6eb00aa510 con 0x7f6ebc0d0700 2024-04-25T08:38:08.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.408+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ea806b4d0 0x7f6ea806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.408+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ea806b4d0 0x7f6ea806d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009aa0 tx=0x7f6eac009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.408+0000 7f6ec41b7740 1 Processor -- start 2024-04-25T08:38:08.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.409+0000 7f6ec41b7740 1 -- start start 2024-04-25T08:38:08.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.409+0000 7f6ec41b7740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564b10 0x562e6d567320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.409+0000 7f6ec41b7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564ef0 0x562e6d567860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.409+0000 7f6ec41b7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d537770 0x562e6d53af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.409+0000 7f6ec41b7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e6d4a0da0 con 0x562e6d537770 2024-04-25T08:38:08.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.409+0000 7f6ec41b7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e6d449b80 con 0x562e6d564ef0 2024-04-25T08:38:08.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.409+0000 7f6ec41b7740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e6d49ce40 con 0x562e6d564b10 2024-04-25T08:38:08.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.409+0000 7f6ebb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d537770 0x562e6d53af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6ebb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d537770 0x562e6d53af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49606/0 (socket says 172.21.15.38:49606) 2024-04-25T08:38:08.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6ebb7fe640 1 -- 172.21.15.38:0/4256999937 learned_addr learned my addr 172.21.15.38:0/4256999937 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:08.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6ebaffd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564b10 0x562e6d567320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/4256999937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564ef0 0x562e6d567860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6ebb7fe640 1 -- 172.21.15.38:0/4256999937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564b10 msgr2=0x562e6d567320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6ebb7fe640 1 --2- 172.21.15.38:0/4256999937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564b10 0x562e6d567320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6ebb7fe640 1 -- 172.21.15.38:0/4256999937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564ef0 msgr2=0x562e6d567860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6ebb7fe640 1 --2- 172.21.15.38:0/4256999937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564ef0 0x562e6d567860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6ebb7fe640 1 -- 172.21.15.38:0/4256999937 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e6d453f10 con 0x562e6d537770 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6ebb7fe640 1 --2- 172.21.15.38:0/4256999937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d537770 0x562e6d53af80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4002060 tx=0x7f6eb4007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc84ed0883baac85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6ea0ff9640 1 -- 172.21.15.38:0/4256999937 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4007ac0 con 0x562e6d537770 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6ea0ff9640 1 -- 172.21.15.38:0/4256999937 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4018070 con 0x562e6d537770 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.410+0000 7f6ea0ff9640 1 -- 172.21.15.38:0/4256999937 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4012780 con 0x562e6d537770 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.411+0000 7f6ec41b7740 1 -- 172.21.15.38:0/4256999937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d537770 msgr2=0x562e6d53af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.411+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/4256999937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d537770 0x562e6d53af80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4002060 tx=0x7f6eb4007410 comp rx=0 tx=0).stop 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.411+0000 7f6ec41b7740 1 -- 172.21.15.38:0/4256999937 shutdown_connections 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.411+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/4256999937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d537770 0x562e6d53af80 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.411+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/4256999937 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564ef0 0x562e6d567860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.411+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/4256999937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564b10 0x562e6d567320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.411+0000 7f6ec41b7740 1 -- 172.21.15.38:0/4256999937 >> 172.21.15.38:0/4256999937 conn(0x562e6d5399e0 msgr2=0x562e6d582320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.411+0000 7f6ec41b7740 1 -- 172.21.15.38:0/4256999937 shutdown_connections 2024-04-25T08:38:08.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.411+0000 7f6ec41b7740 1 -- 172.21.15.38:0/4256999937 wait complete. 2024-04-25T08:38:08.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.412+0000 7f6ec41b7740 1 -- start start 2024-04-25T08:38:08.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.412+0000 7f6ec41b7740 1 Processor -- start 2024-04-25T08:38:08.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.412+0000 7f6ec41b7740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564b10 0x562e6d5859b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.412+0000 7f6ec41b7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564ef0 0x562e6d566bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.412+0000 7f6ec41b7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d567130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.412+0000 7f6ec41b7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e6d53cb90 con 0x562e6d55bbe0 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.412+0000 7f6ec41b7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e6d43f4b0 con 0x562e6d564ef0 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.412+0000 7f6ec41b7740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e6d53ce70 con 0x562e6d564b10 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.412+0000 7f6ec41b7740 10 client.0 _ll_register_callbacks cb 0x562e6d47f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ebb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d567130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ebb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d567130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49620/0 (socket says 172.21.15.38:49620) 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ebb7fe640 1 -- 172.21.15.38:0/3536994306 learned_addr learned my addr 172.21.15.38:0/3536994306 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564b10 0x562e6d5859b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564ef0 0x562e6d566bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ebb7fe640 1 -- 172.21.15.38:0/3536994306 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564b10 msgr2=0x562e6d5859b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ebb7fe640 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564b10 0x562e6d5859b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ebb7fe640 1 -- 172.21.15.38:0/3536994306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564ef0 msgr2=0x562e6d566bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ebb7fe640 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564ef0 0x562e6d566bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ebb7fe640 1 -- 172.21.15.38:0/3536994306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e6d27f7d0 con 0x562e6d55bbe0 2024-04-25T08:38:08.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ebb7fe640 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d567130 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4001560 tx=0x7f6eb4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4803104463ac31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6e99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:08.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3536994306 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4027670 con 0x562e6d55bbe0 2024-04-25T08:38:08.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3536994306 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f6eb4027810 con 0x562e6d55bbe0 2024-04-25T08:38:08.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3536994306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eb4018070 con 0x562e6d55bbe0 2024-04-25T08:38:08.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3536994306 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eb402f050 con 0x562e6d55bbe0 2024-04-25T08:38:08.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ec41b7740 10 client.7638 fetch_fsmap learned FSMap version 51 2024-04-25T08:38:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.413+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3536994306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x562e6d1b53f0 con 0x562e6d55bbe0 2024-04-25T08:38:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.414+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3536994306 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4018070 con 0x562e6d55bbe0 2024-04-25T08:38:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.414+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3536994306 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6eb4032020 con 0x562e6d55bbe0 2024-04-25T08:38:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.414+0000 7f6ec41b7740 10 client.7638 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:38:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.414+0000 7f6ec41b7740 10 client.7638 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:38:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.414+0000 7f6ec41b7740 20 client.7638 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.414+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x562e6d55de50 0x562e6d55e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.414+0000 7f6ec41b7740 4 client.7638 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:38:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.414+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3536994306 --> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e6d4ae2e0 con 0x562e6d55de50 2024-04-25T08:38:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.414+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x562e6d55de50 0x562e6d55e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.414+0000 7f6e99ffb640 10 client.7638 ms_handle_connect on v2:172.21.15.38:6833/380751658 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.414+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x562e6d55de50 0x562e6d55e260 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=290ec309ec132c4e server_cookie=6a4a3b18b5b453da in_seq=0 out_seq=0 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3536994306 <== mds.0 v2:172.21.15.38:6833/380751658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e6d4ae2e0 con 0x562e6d55de50 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6e99ffb640 10 client.7638 handle_command_reply: tid=0 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6ec41b7740 1 client.7638 shutdown 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3536994306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 msgr2=0x562e6d567130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d567130 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4001560 tx=0x7f6eb4001250 comp rx=0 tx=0).stop 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3536994306 shutdown_connections 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x562e6d55de50 0x562e6d55e260 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d567130 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564ef0 0x562e6d566bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6e99ffb640 0 client.7638 ms_handle_reset on v2:172.21.15.38:6833/380751658 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3536994306 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564b10 0x562e6d5859b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3536994306 >> 172.21.15.38:0/3536994306 conn(0x562e6d537770 msgr2=0x562e6d538420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3536994306 shutdown_connections 2024-04-25T08:38:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3536994306 wait complete. 2024-04-25T08:38:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.415+0000 7f6ec41b7740 20 client.7638 trim_cache size 0 max 16384 2024-04-25T08:38:08.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.431+0000 7f6ec41b7740 1 Processor -- start 2024-04-25T08:38:08.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.431+0000 7f6ec41b7740 1 -- start start 2024-04-25T08:38:08.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.431+0000 7f6ec41b7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d5e2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.431+0000 7f6ec41b7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564b10 0x562e6d582320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.431+0000 7f6ec41b7740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564ef0 0x562e6d582860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.431+0000 7f6ec41b7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e6d59c3f0 con 0x562e6d55bbe0 2024-04-25T08:38:08.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.431+0000 7f6ec41b7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc200280 con 0x562e6d564b10 2024-04-25T08:38:08.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.431+0000 7f6ec41b7740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1ff3c0 con 0x562e6d564ef0 2024-04-25T08:38:08.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.431+0000 7f6ebaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d5e2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.431+0000 7f6ebaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d5e2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49636/0 (socket says 172.21.15.38:49636) 2024-04-25T08:38:08.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.431+0000 7f6ebaffd640 1 -- 172.21.15.38:0/3773098417 learned_addr learned my addr 172.21.15.38:0/3773098417 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:08.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.431+0000 7f6ebb7fe640 1 --2- 172.21.15.38:0/3773098417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564ef0 0x562e6d582860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/3773098417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564b10 0x562e6d582320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ebaffd640 1 -- 172.21.15.38:0/3773098417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564ef0 msgr2=0x562e6d582860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/3773098417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564ef0 0x562e6d582860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ebaffd640 1 -- 172.21.15.38:0/3773098417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564b10 msgr2=0x562e6d582320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/3773098417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564b10 0x562e6d582320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ebaffd640 1 -- 172.21.15.38:0/3773098417 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e6d1750f0 con 0x562e6d55bbe0 2024-04-25T08:38:08.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ebb7fe640 1 --2- 172.21.15.38:0/3773098417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564ef0 0x562e6d582860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/3773098417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d5e2990 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6eac02ceb0 tx=0x7f6eac018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8374a8cab24cff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6e9bfff640 1 -- 172.21.15.38:0/3773098417 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac032070 con 0x562e6d55bbe0 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6e9bfff640 1 -- 172.21.15.38:0/3773098417 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac02c3a0 con 0x562e6d55bbe0 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6e9bfff640 1 -- 172.21.15.38:0/3773098417 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac02c750 con 0x562e6d55bbe0 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3773098417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 msgr2=0x562e6d5e2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3773098417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d5e2990 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6eac02ceb0 tx=0x7f6eac018000 comp rx=0 tx=0).stop 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3773098417 shutdown_connections 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3773098417 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564ef0 0x562e6d582860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3773098417 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564b10 0x562e6d582320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3773098417 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d5e2990 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.432+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3773098417 >> 172.21.15.38:0/3773098417 conn(0x562e6d537770 msgr2=0x562e6d56aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.433+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3773098417 shutdown_connections 2024-04-25T08:38:08.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.433+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3773098417 wait complete. 2024-04-25T08:38:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.433+0000 7f6ec41b7740 1 -- start start 2024-04-25T08:38:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.433+0000 7f6ec41b7740 1 Processor -- start 2024-04-25T08:38:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ec41b7740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d585820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ec41b7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564b10 0x562e6d57f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ec41b7740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564ef0 0x562e6d5d62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ec41b7740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc1fedf0 con 0x562e6d55bbe0 2024-04-25T08:38:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ec41b7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc152070 con 0x562e6d564b10 2024-04-25T08:38:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ec41b7740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e6d53caf0 con 0x562e6d564ef0 2024-04-25T08:38:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ec41b7740 10 client.0 _ll_register_callbacks cb 0x7f6eb4006cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ebaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d585820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ebaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d585820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49640/0 (socket says 172.21.15.38:49640) 2024-04-25T08:38:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ebaffd640 1 -- 172.21.15.38:0/3403854076 learned_addr learned my addr 172.21.15.38:0/3403854076 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564b10 0x562e6d57f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ebb7fe640 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564ef0 0x562e6d5d62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ebaffd640 1 -- 172.21.15.38:0/3403854076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564ef0 msgr2=0x562e6d5d62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564ef0 0x562e6d5d62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ebaffd640 1 -- 172.21.15.38:0/3403854076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564b10 msgr2=0x562e6d57f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564b10 0x562e6d57f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ebaffd640 1 -- 172.21.15.38:0/3403854076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e6d53c150 con 0x562e6d55bbe0 2024-04-25T08:38:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564b10 0x562e6d57f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6ebaffd640 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d585820 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6eac030040 tx=0x7f6eac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f1f9c9aac4ea8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.434+0000 7f6e99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3403854076 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac032070 con 0x562e6d55bbe0 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3403854076 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f6eac005da0 con 0x562e6d55bbe0 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3403854076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6eac02c3a0 con 0x562e6d55bbe0 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3403854076 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6eac003610 con 0x562e6d55bbe0 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6ec41b7740 10 client.7644 fetch_fsmap learned FSMap version 51 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3403854076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x562e6d5e53d0 con 0x562e6d55bbe0 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3403854076 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac02c3a0 con 0x562e6d55bbe0 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3403854076 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6eac0037b0 con 0x562e6d55bbe0 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6ec41b7740 10 client.7644 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6ec41b7740 10 client.7644 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6ec41b7740 20 client.7644 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x562e6d5d67f0 0x562e6d5e4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6ec41b7740 4 client.7644 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3403854076 --> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562e6d4a8630 con 0x562e6d5d67f0 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.435+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x562e6d5d67f0 0x562e6d5e4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6e99ffb640 10 client.7644 ms_handle_connect on v2:172.21.15.38:6833/380751658 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6eba7fc640 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x562e6d5d67f0 0x562e6d5e4510 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0624fdb4b4b2a11 server_cookie=293f6fdee2962112 in_seq=0 out_seq=0 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6e99ffb640 1 -- 172.21.15.38:0/3403854076 <== mds.0 v2:172.21.15.38:6833/380751658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562e6d4a8630 con 0x562e6d5d67f0 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6e99ffb640 10 client.7644 handle_command_reply: tid=0 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6ec41b7740 1 client.7644 shutdown 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3403854076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 msgr2=0x562e6d585820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d585820 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6eac030040 tx=0x7f6eac004730 comp rx=0 tx=0).stop 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3403854076 shutdown_connections 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x562e6d5d67f0 0x562e6d5e4510 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562e6d564ef0 0x562e6d5d62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e6d564b10 0x562e6d57f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6e99ffb640 0 client.7644 ms_handle_reset on v2:172.21.15.38:6833/380751658 2024-04-25T08:38:08.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6ec41b7740 1 --2- 172.21.15.38:0/3403854076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562e6d55bbe0 0x562e6d585820 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3403854076 >> 172.21.15.38:0/3403854076 conn(0x562e6d537770 msgr2=0x562e6d55ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3403854076 shutdown_connections 2024-04-25T08:38:08.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6ec41b7740 1 -- 172.21.15.38:0/3403854076 wait complete. 2024-04-25T08:38:08.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.436+0000 7f6ec41b7740 20 client.7644 trim_cache size 0 max 16384 2024-04-25T08:38:08.459 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:38:08.459 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-04-25T08:38:08.459 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-04-25T08:38:08.459 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:38:08.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.437+0000 7f6ec13e2640 1 -- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ea806b4d0 msgr2=0x7f6ea806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.437+0000 7f6ec13e2640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ea806b4d0 0x7f6ea806d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009aa0 tx=0x7f6eac009190 comp rx=0 tx=0).stop 2024-04-25T08:38:08.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.437+0000 7f6ec13e2640 1 -- 172.21.15.38:0/4283141877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ebc0d0700 msgr2=0x7f6ebc1f3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.437+0000 7f6ec13e2640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ebc0d0700 0x7f6ebc1f3aa0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f6eb00098c0 tx=0x7f6eb00046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.437+0000 7f6ec13e2640 1 -- 172.21.15.38:0/4283141877 shutdown_connections 2024-04-25T08:38:08.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.437+0000 7f6ec13e2640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ebc0d1e90 0x7f6ebc1f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.437+0000 7f6ec13e2640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ebc0d0700 0x7f6ebc1f3aa0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.437+0000 7f6ec13e2640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ea806b4d0 0x7f6ea806d990 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.437+0000 7f6ec13e2640 1 --2- 172.21.15.38:0/4283141877 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6ebc0d0320 0x7f6ebc0abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.437+0000 7f6ec13e2640 1 -- 172.21.15.38:0/4283141877 >> 172.21.15.38:0/4283141877 conn(0x7f6ebc0668f0 msgr2=0x7f6ebc0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.437+0000 7f6ec13e2640 1 -- 172.21.15.38:0/4283141877 shutdown_connections 2024-04-25T08:38:08.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.437+0000 7f6ec13e2640 1 -- 172.21.15.38:0/4283141877 wait complete. 2024-04-25T08:38:08.462 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:38:08.462 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:38:08.462 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T08:38:08.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.572+0000 7f0ed4568640 1 Processor -- start 2024-04-25T08:38:08.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.572+0000 7f0ed4568640 1 -- start start 2024-04-25T08:38:08.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed4568640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0ecc15df30 0x7f0ecc09a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed4568640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0ecc162390 0x7f0ecc099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed4568640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc162820 con 0x7f0ecc099fb0 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed4568640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc15e310 con 0x7f0ecc15df30 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed4568640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc09a400 con 0x7f0ecc162390 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed2304640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0ecc15df30 0x7f0ecc09a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed2304640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0ecc15df30 0x7f0ecc09a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49652/0 (socket says 172.21.15.38:49652) 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed2304640 1 -- 172.21.15.38:0/2179289079 learned_addr learned my addr 172.21.15.38:0/2179289079 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/2179289079 >> v2:172.21.15.107:3300/0 conn(0x7f0ecc162390 0x7f0ecc099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed2304640 1 -- 172.21.15.38:0/2179289079 >> v2:172.21.15.107:3300/0 conn(0x7f0ecc162390 msgr2=0x7f0ecc099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed2304640 1 --2- 172.21.15.38:0/2179289079 >> v2:172.21.15.107:3300/0 conn(0x7f0ecc162390 0x7f0ecc099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed2304640 1 -- 172.21.15.38:0/2179289079 >> v1:172.21.15.107:6789/0 conn(0x7f0ecc099fb0 legacy=0x7f0ecc06bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed2304640 1 -- 172.21.15.38:0/2179289079 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc15f300 con 0x7f0ecc15df30 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.573+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/2179289079 >> v2:172.21.15.107:3300/0 conn(0x7f0ecc162390 0x7f0ecc099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.574+0000 7f0ed2304640 1 --2- 172.21.15.38:0/2179289079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ecc15df30 0x7f0ecc09a7b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009870 tx=0x7f0ec802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f4b533bb78115c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.574+0000 7f0ed1302640 1 -- 172.21.15.38:0/2179289079 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8037070 con 0x7f0ecc15df30 2024-04-25T08:38:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.574+0000 7f0ed1302640 1 -- 172.21.15.38:0/2179289079 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8032440 con 0x7f0ecc15df30 2024-04-25T08:38:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.574+0000 7f0ed1302640 1 -- 172.21.15.38:0/2179289079 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8032810 con 0x7f0ecc15df30 2024-04-25T08:38:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.575+0000 7f0ed4568640 1 -- 172.21.15.38:0/2179289079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ecc15df30 msgr2=0x7f0ecc09a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.575+0000 7f0ed4568640 1 --2- 172.21.15.38:0/2179289079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ecc15df30 0x7f0ecc09a7b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009870 tx=0x7f0ec802a270 comp rx=0 tx=0).stop 2024-04-25T08:38:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.575+0000 7f0ed4568640 1 -- 172.21.15.38:0/2179289079 shutdown_connections 2024-04-25T08:38:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.575+0000 7f0ed4568640 1 --2- 172.21.15.38:0/2179289079 >> v2:172.21.15.107:3300/0 conn(0x7f0ecc162390 0x7f0ecc099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.575+0000 7f0ed4568640 1 --2- 172.21.15.38:0/2179289079 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ecc15df30 0x7f0ecc09a7b0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.575+0000 7f0ed4568640 1 -- 172.21.15.38:0/2179289079 >> 172.21.15.38:0/2179289079 conn(0x7f0ecc0b3110 msgr2=0x7f0ecc0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.575+0000 7f0ed4568640 1 -- 172.21.15.38:0/2179289079 shutdown_connections 2024-04-25T08:38:08.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.575+0000 7f0ed4568640 1 -- 172.21.15.38:0/2179289079 wait complete. 2024-04-25T08:38:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.576+0000 7f0ed4568640 1 Processor -- start 2024-04-25T08:38:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.576+0000 7f0ed4568640 1 -- start start 2024-04-25T08:38:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed4568640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ecc099fb0 0x7f0ecc09eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed4568640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ecc15df30 0x7f0ecc0c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed4568640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ecc162390 0x7f0ecc0c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed4568640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc15ee70 con 0x7f0ecc099fb0 2024-04-25T08:38:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed4568640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc09aeb0 con 0x7f0ecc162390 2024-04-25T08:38:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed4568640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc15e820 con 0x7f0ecc15df30 2024-04-25T08:38:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed2304640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ecc099fb0 0x7f0ecc09eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed2304640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ecc099fb0 0x7f0ecc09eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49654/0 (socket says 172.21.15.38:49654) 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed2304640 1 -- 172.21.15.38:0/3613354442 learned_addr learned my addr 172.21.15.38:0/3613354442 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed2b05640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ecc162390 0x7f0ecc0c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed2304640 1 -- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ecc15df30 msgr2=0x7f0ecc0c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed2304640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ecc15df30 0x7f0ecc0c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed2304640 1 -- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ecc162390 msgr2=0x7f0ecc0c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed2304640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ecc162390 0x7f0ecc0c63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed2304640 1 -- 172.21.15.38:0/3613354442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0c6a90 con 0x7f0ecc099fb0 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed2304640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ecc099fb0 0x7f0ecc09eb80 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009870 tx=0x7f0ec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ec2ffd640 1 -- 172.21.15.38:0/3613354442 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8037040 con 0x7f0ecc099fb0 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ec2ffd640 1 -- 172.21.15.38:0/3613354442 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8004230 con 0x7f0ecc099fb0 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ed4568640 1 -- 172.21.15.38:0/3613354442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc09b4f0 con 0x7f0ecc099fb0 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.577+0000 7f0ec2ffd640 1 -- 172.21.15.38:0/3613354442 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8030620 con 0x7f0ecc099fb0 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.578+0000 7f0ed4568640 1 -- 172.21.15.38:0/3613354442 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc09bab0 con 0x7f0ecc099fb0 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.579+0000 7f0ec2ffd640 1 -- 172.21.15.38:0/3613354442 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ec80307c0 con 0x7f0ecc099fb0 2024-04-25T08:38:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.579+0000 7f0ec2ffd640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0eb006b4d0 0x7f0eb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.579+0000 7f0ec2ffd640 1 -- 172.21.15.38:0/3613354442 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f0ec80ab360 con 0x7f0ecc099fb0 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.580+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0eb006b4d0 0x7f0eb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.580+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0eb006b4d0 0x7f0eb006d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc006fd0 tx=0x7f0ebc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.580+0000 7f0ed7261740 1 Processor -- start 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.581+0000 7f0ed7261740 1 -- start start 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.581+0000 7f0ed7261740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc03e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.581+0000 7f0ed7261740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc3e370 0x55f2bfc02080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.581+0000 7f0ed7261740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc2c760 0x55f2bfc2cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.581+0000 7f0ed7261740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2bfb64b90 con 0x55f2bfc01ca0 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.581+0000 7f0ed7261740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2bfb0de40 con 0x55f2bfc2c760 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.581+0000 7f0ed7261740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2bfb60c30 con 0x55f2bfc3e370 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.581+0000 7f0ed2304640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc03e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.581+0000 7f0ed2304640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc03e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49668/0 (socket says 172.21.15.38:49668) 2024-04-25T08:38:08.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.581+0000 7f0ed2304640 1 -- 172.21.15.38:0/1150266814 learned_addr learned my addr 172.21.15.38:0/1150266814 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.582+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/1150266814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc3e370 0x55f2bfc02080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.582+0000 7f0ed2b05640 1 --2- 172.21.15.38:0/1150266814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc2c760 0x55f2bfc2cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.582+0000 7f0ed2304640 1 -- 172.21.15.38:0/1150266814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc3e370 msgr2=0x55f2bfc02080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.582+0000 7f0ed2304640 1 --2- 172.21.15.38:0/1150266814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc3e370 0x55f2bfc02080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.582+0000 7f0ed2304640 1 -- 172.21.15.38:0/1150266814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc2c760 msgr2=0x55f2bfc2cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.582+0000 7f0ed2304640 1 --2- 172.21.15.38:0/1150266814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc2c760 0x55f2bfc2cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.582+0000 7f0ed2304640 1 -- 172.21.15.38:0/1150266814 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2bf9437d0 con 0x55f2bfc01ca0 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.582+0000 7f0ed2304640 1 --2- 172.21.15.38:0/1150266814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc03e00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80a9650 tx=0x7f0ec80796c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db0bf7b6715e915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.582+0000 7f0eaffff640 1 -- 172.21.15.38:0/1150266814 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec807dba0 con 0x55f2bfc01ca0 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.582+0000 7f0eaffff640 1 -- 172.21.15.38:0/1150266814 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8082160 con 0x55f2bfc01ca0 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.582+0000 7f0eaffff640 1 -- 172.21.15.38:0/1150266814 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8077d50 con 0x55f2bfc01ca0 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.583+0000 7f0ed7261740 1 -- 172.21.15.38:0/1150266814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 msgr2=0x55f2bfc03e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.583+0000 7f0ed7261740 1 --2- 172.21.15.38:0/1150266814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc03e00 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0ec80a9650 tx=0x7f0ec80796c0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.583+0000 7f0ed7261740 1 -- 172.21.15.38:0/1150266814 shutdown_connections 2024-04-25T08:38:08.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.583+0000 7f0ed7261740 1 --2- 172.21.15.38:0/1150266814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc2c760 0x55f2bfc2cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.583+0000 7f0ed7261740 1 --2- 172.21.15.38:0/1150266814 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc3e370 0x55f2bfc02080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.583+0000 7f0ed7261740 1 --2- 172.21.15.38:0/1150266814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc03e00 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.583+0000 7f0ed7261740 1 -- 172.21.15.38:0/1150266814 >> 172.21.15.38:0/1150266814 conn(0x55f2bfbc9870 msgr2=0x55f2bfc47760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.583+0000 7f0ed7261740 1 -- 172.21.15.38:0/1150266814 shutdown_connections 2024-04-25T08:38:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.583+0000 7f0ed7261740 1 -- 172.21.15.38:0/1150266814 wait complete. 2024-04-25T08:38:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.583+0000 7f0ed7261740 1 -- start start 2024-04-25T08:38:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.584+0000 7f0ed7261740 1 Processor -- start 2024-04-25T08:38:08.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.584+0000 7f0ed7261740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc01ca0 0x55f2bfc1b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.584+0000 7f0ed7261740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 0x55f2bfc1bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.584+0000 7f0ed7261740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc3e370 0x55f2bfc41320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.584+0000 7f0ed7261740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2bfb64b90 con 0x55f2bfc2c760 2024-04-25T08:38:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.584+0000 7f0ed7261740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2bfb213e0 con 0x55f2bfc3e370 2024-04-25T08:38:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.584+0000 7f0ed7261740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2bfc1b870 con 0x55f2bfc01ca0 2024-04-25T08:38:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed7261740 10 client.0 _ll_register_callbacks cb 0x55f2bfb20db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed1b03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 0x55f2bfc1bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed1b03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 0x55f2bfc1bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49678/0 (socket says 172.21.15.38:49678) 2024-04-25T08:38:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed1b03640 1 -- 172.21.15.38:0/148374863 learned_addr learned my addr 172.21.15.38:0/148374863 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed2304640 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc01ca0 0x55f2bfc1b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed2b05640 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc3e370 0x55f2bfc41320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed1b03640 1 -- 172.21.15.38:0/148374863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc01ca0 msgr2=0x55f2bfc1b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc01ca0 0x55f2bfc1b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed1b03640 1 -- 172.21.15.38:0/148374863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc3e370 msgr2=0x55f2bfc41320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc3e370 0x55f2bfc41320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed1b03640 1 -- 172.21.15.38:0/148374863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2bf8390f0 con 0x55f2bfc2c760 2024-04-25T08:38:08.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 0x55f2bfc1bd20 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00cdd0 tx=0x7f0ebc006c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472b785b5128aae7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ead7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:08.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ead7fa640 1 -- 172.21.15.38:0/148374863 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc01f660 con 0x55f2bfc2c760 2024-04-25T08:38:08.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ead7fa640 1 -- 172.21.15.38:0/148374863 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f0ebc01f800 con 0x55f2bfc2c760 2024-04-25T08:38:08.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed7261740 1 -- 172.21.15.38:0/148374863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ec8082160 con 0x55f2bfc2c760 2024-04-25T08:38:08.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ead7fa640 1 -- 172.21.15.38:0/148374863 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ebc02c050 con 0x55f2bfc2c760 2024-04-25T08:38:08.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed7261740 10 client.7656 fetch_fsmap learned FSMap version 51 2024-04-25T08:38:08.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.585+0000 7f0ed7261740 1 -- 172.21.15.38:0/148374863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55f2bf9a3f60 con 0x55f2bfc2c760 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.586+0000 7f0ead7fa640 1 -- 172.21.15.38:0/148374863 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8082160 con 0x55f2bfc2c760 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.586+0000 7f0ead7fa640 1 -- 172.21.15.38:0/148374863 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0ebc02f020 con 0x55f2bfc2c760 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.586+0000 7f0ed7261740 10 client.7656 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.586+0000 7f0ed7261740 10 client.7656 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.586+0000 7f0ed7261740 20 client.7656 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.586+0000 7f0ed7261740 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55f2bfc3fa90 0x55f2bfc41c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.586+0000 7f0ed7261740 4 client.7656 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.586+0000 7f0ed7261740 1 -- 172.21.15.38:0/148374863 --> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2bfb720d0 con 0x55f2bfc3fa90 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.586+0000 7f0ed2304640 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55f2bfc3fa90 0x55f2bfc41c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.586+0000 7f0ead7fa640 10 client.7656 ms_handle_connect on v2:172.21.15.38:6833/380751658 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.586+0000 7f0ed2304640 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55f2bfc3fa90 0x55f2bfc41c70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62e4c0ef24dbbfc5 server_cookie=cb09a8942211d789 in_seq=0 out_seq=0 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ead7fa640 1 -- 172.21.15.38:0/148374863 <== mds.0 v2:172.21.15.38:6833/380751658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f2bfb720d0 con 0x55f2bfc3fa90 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ead7fa640 10 client.7656 handle_command_reply: tid=0 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ed7261740 1 client.7656 shutdown 2024-04-25T08:38:08.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ed7261740 1 -- 172.21.15.38:0/148374863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 msgr2=0x55f2bfc1bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ed7261740 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 0x55f2bfc1bd20 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00cdd0 tx=0x7f0ebc006c90 comp rx=0 tx=0).stop 2024-04-25T08:38:08.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ed7261740 1 -- 172.21.15.38:0/148374863 shutdown_connections 2024-04-25T08:38:08.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ed7261740 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55f2bfc3fa90 0x55f2bfc41c70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ed7261740 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc3e370 0x55f2bfc41320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ed7261740 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 0x55f2bfc1bd20 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ed7261740 1 --2- 172.21.15.38:0/148374863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc01ca0 0x55f2bfc1b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ead7fa640 0 client.7656 ms_handle_reset on v2:172.21.15.38:6833/380751658 2024-04-25T08:38:08.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ed7261740 1 -- 172.21.15.38:0/148374863 >> 172.21.15.38:0/148374863 conn(0x55f2bfbc9870 msgr2=0x55f2bfc4aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ed7261740 1 -- 172.21.15.38:0/148374863 shutdown_connections 2024-04-25T08:38:08.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ed7261740 1 -- 172.21.15.38:0/148374863 wait complete. 2024-04-25T08:38:08.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.587+0000 7f0ed7261740 20 client.7656 trim_cache size 0 max 16384 2024-04-25T08:38:08.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.601+0000 7f0ed7261740 1 Processor -- start 2024-04-25T08:38:08.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.601+0000 7f0ed7261740 1 -- start start 2024-04-25T08:38:08.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.602+0000 7f0ed7261740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc2bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.602+0000 7f0ed7261740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 0x55f2bfc3e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.602+0000 7f0ed7261740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc3e370 0x55f2bfc3ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.602+0000 7f0ed7261740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc2004b0 con 0x55f2bfc2c760 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.602+0000 7f0ed7261740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc1ff6c0 con 0x55f2bfc01ca0 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.602+0000 7f0ed7261740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc09cc10 con 0x55f2bfc3e370 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.602+0000 7f0ed1b03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 0x55f2bfc3e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.602+0000 7f0ed1b03640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 0x55f2bfc3e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49682/0 (socket says 172.21.15.38:49682) 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.602+0000 7f0ed1b03640 1 -- 172.21.15.38:0/2877391278 learned_addr learned my addr 172.21.15.38:0/2877391278 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.602+0000 7f0ed2304640 1 --2- 172.21.15.38:0/2877391278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc2bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.602+0000 7f0ed2b05640 1 --2- 172.21.15.38:0/2877391278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc3e370 0x55f2bfc3ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.603+0000 7f0ed1b03640 1 -- 172.21.15.38:0/2877391278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc3e370 msgr2=0x55f2bfc3ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.603+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/2877391278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc3e370 0x55f2bfc3ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.603+0000 7f0ed1b03640 1 -- 172.21.15.38:0/2877391278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc01ca0 msgr2=0x55f2bfc2bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.603+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/2877391278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc2bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.603+0000 7f0ed1b03640 1 -- 172.21.15.38:0/2877391278 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2bfbfd390 con 0x55f2bfc2c760 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.603+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/2877391278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 0x55f2bfc3e750 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00a770 tx=0x7f0ebc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac388ca6385457d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.603+0000 7f0eaf7fe640 1 -- 172.21.15.38:0/2877391278 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc01fbd0 con 0x55f2bfc2c760 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.603+0000 7f0eaf7fe640 1 -- 172.21.15.38:0/2877391278 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc024070 con 0x55f2bfc2c760 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.603+0000 7f0eaf7fe640 1 -- 172.21.15.38:0/2877391278 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc01f680 con 0x55f2bfc2c760 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.604+0000 7f0ed7261740 1 -- 172.21.15.38:0/2877391278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 msgr2=0x55f2bfc3e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.604+0000 7f0ed7261740 1 --2- 172.21.15.38:0/2877391278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 0x55f2bfc3e750 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00a770 tx=0x7f0ebc01c000 comp rx=0 tx=0).stop 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.604+0000 7f0ed7261740 1 -- 172.21.15.38:0/2877391278 shutdown_connections 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.604+0000 7f0ed7261740 1 --2- 172.21.15.38:0/2877391278 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc3e370 0x55f2bfc3ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.604+0000 7f0ed7261740 1 --2- 172.21.15.38:0/2877391278 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc2c760 0x55f2bfc3e750 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.604+0000 7f0ed7261740 1 --2- 172.21.15.38:0/2877391278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc2bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.604+0000 7f0ed7261740 1 -- 172.21.15.38:0/2877391278 >> 172.21.15.38:0/2877391278 conn(0x55f2bfbc9870 msgr2=0x55f2bfbf7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.604+0000 7f0ed7261740 1 -- 172.21.15.38:0/2877391278 shutdown_connections 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.604+0000 7f0ed7261740 1 -- 172.21.15.38:0/2877391278 wait complete. 2024-04-25T08:38:08.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.604+0000 7f0ed7261740 1 -- start start 2024-04-25T08:38:08.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.604+0000 7f0ed7261740 1 Processor -- start 2024-04-25T08:38:08.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed7261740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc3e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed7261740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc2c760 0x55f2bfc3ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed7261740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc3e370 0x55f2bfc1b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed7261740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2bfc3f2f0 con 0x55f2bfc01ca0 2024-04-25T08:38:08.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed7261740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc09d210 con 0x55f2bfc3e370 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed7261740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2bfc59fc0 con 0x55f2bfc2c760 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed7261740 10 client.0 _ll_register_callbacks cb 0x7f0ebc00a5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed2304640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc3e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed2304640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc3e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49696/0 (socket says 172.21.15.38:49696) 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed2304640 1 -- 172.21.15.38:0/3065645562 learned_addr learned my addr 172.21.15.38:0/3065645562 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed2b05640 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc3e370 0x55f2bfc1b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc2c760 0x55f2bfc3ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed2304640 1 -- 172.21.15.38:0/3065645562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc2c760 msgr2=0x55f2bfc3ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed2304640 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc2c760 0x55f2bfc3ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed2304640 1 -- 172.21.15.38:0/3065645562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc3e370 msgr2=0x55f2bfc1b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.605+0000 7f0ed2304640 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc3e370 0x55f2bfc1b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed2304640 1 -- 172.21.15.38:0/3065645562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2bf8121b0 con 0x55f2bfc01ca0 2024-04-25T08:38:08.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc2c760 0x55f2bfc3ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed2b05640 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc3e370 0x55f2bfc1b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed2304640 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc3e750 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8079750 tx=0x7f0ec8009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1454b9a13b7ba3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ead7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ead7fa640 1 -- 172.21.15.38:0/3065645562 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8077d50 con 0x55f2bfc01ca0 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ead7fa640 1 -- 172.21.15.38:0/3065645562 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f0ec80ab0c0 con 0x55f2bfc01ca0 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed7261740 1 -- 172.21.15.38:0/3065645562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ebc024070 con 0x55f2bfc01ca0 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ead7fa640 1 -- 172.21.15.38:0/3065645562 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec80760c0 con 0x55f2bfc01ca0 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed7261740 10 client.7662 fetch_fsmap learned FSMap version 51 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed7261740 1 -- 172.21.15.38:0/3065645562 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55f2bfcb2710 con 0x55f2bfc01ca0 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ead7fa640 1 -- 172.21.15.38:0/3065645562 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc024070 con 0x55f2bfc01ca0 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ead7fa640 1 -- 172.21.15.38:0/3065645562 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0ec804e060 con 0x55f2bfc01ca0 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed7261740 10 client.7662 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed7261740 10 client.7662 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:38:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed7261740 20 client.7662 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed7261740 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55f2bfc29e40 0x55f2bfc1b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed7261740 4 client.7662 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed7261740 1 -- 172.21.15.38:0/3065645562 --> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x55f2bfb6c420 con 0x55f2bfc29e40 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.606+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55f2bfc29e40 0x55f2bfc1b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ead7fa640 10 client.7662 ms_handle_connect on v2:172.21.15.38:6833/380751658 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ed1b03640 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55f2bfc29e40 0x55f2bfc1b680 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d18fe3963510f2ce server_cookie=d7404e1aa0919071 in_seq=0 out_seq=0 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ead7fa640 1 -- 172.21.15.38:0/3065645562 <== mds.0 v2:172.21.15.38:6833/380751658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17948 (crc 0 0 0) 0x55f2bfb6c420 con 0x55f2bfc29e40 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ead7fa640 10 client.7662 handle_command_reply: tid=0 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ed7261740 1 client.7662 shutdown 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ed7261740 1 -- 172.21.15.38:0/3065645562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 msgr2=0x55f2bfc3e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ed7261740 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc3e750 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8079750 tx=0x7f0ec8009240 comp rx=0 tx=0).stop 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ed7261740 1 -- 172.21.15.38:0/3065645562 shutdown_connections 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ed7261740 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.38:6833/380751658,v1:172.21.15.38:6835/380751658] conn(0x55f2bfc29e40 0x55f2bfc1b680 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ed7261740 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f2bfc3e370 0x55f2bfc1b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ed7261740 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55f2bfc2c760 0x55f2bfc3ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ead7fa640 0 client.7662 ms_handle_reset on v2:172.21.15.38:6833/380751658 2024-04-25T08:38:08.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ed7261740 1 --2- 172.21.15.38:0/3065645562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f2bfc01ca0 0x55f2bfc3e750 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.607+0000 7f0ed7261740 1 -- 172.21.15.38:0/3065645562 >> 172.21.15.38:0/3065645562 conn(0x55f2bfbc9870 msgr2=0x55f2bfc47f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed7261740 1 -- 172.21.15.38:0/3065645562 shutdown_connections 2024-04-25T08:38:08.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed7261740 1 -- 172.21.15.38:0/3065645562 wait complete. 2024-04-25T08:38:08.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed7261740 20 client.7662 trim_cache size 0 max 16384 2024-04-25T08:38:08.607 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:38:08.608 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T08:38:08.608 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 4036, 2024-04-25T08:38:08.608 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 4036, 2024-04-25T08:38:08.608 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1353890, 2024-04-25T08:38:08.608 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 14355620, 2024-04-25T08:38:08.608 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 17, 2024-04-25T08:38:08.608 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 6, 2024-04-25T08:38:08.608 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.360787200, 2024-04-25T08:38:08.608 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.139759698, 2024-04-25T08:38:08.608 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.063910932, 2024-04-25T08:38:08.608 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.135914300, 2024-04-25T08:38:08.608 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 4035, 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.526116984, 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000130388 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 10, 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000001026, 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000102 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T08:38:08.609 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 35344, 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 61237, 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 12313307, 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 52383202, 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 7, 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 3.024453500, 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 1.495500798, 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 1.756819401, 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.071421333, 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 61237, 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.244448599, 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000020321 2024-04-25T08:38:08.610 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 88633, 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.013679467, 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000154 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 123584, 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 27488 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 5064, 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 5087, 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 1260732, 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 47376273, 2024-04-25T08:38:08.611 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 6, 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 4, 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.449524399, 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.213621143, 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.082481026, 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.122593655, 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 5086, 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.389448064, 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000076572 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.612 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 122672 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 1, 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-MDSRank": { 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 10499, 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 12.856418266, 2024-04-25T08:38:08.613 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001224537 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: "finisher-PurgeQueue": { 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: "queue_len": 0, 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: "complete_latency": { 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2931, 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.767088107, 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000261715 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": { 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: "request": 28303, 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: "reply": 28303, 2024-04-25T08:38:08.614 INFO:teuthology.orchestra.run.smithi038.stdout: "reply_latency": { 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 28303, 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 231.591748725, 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.008182586 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_reply": 0, 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: "forward": 0, 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_complete": 0, 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_fetch_keys": 0, 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_commit": 12, 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_split": 0, 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_merge": 0, 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes": 17868, 2024-04-25T08:38:08.615 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_top": 0, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_bottom": 17868, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pin_tail": 0, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_pinned": 17868, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_expired": 0, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "inodes_with_caps": 2, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": 2, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "subtrees": 2, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse": 47569, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_hit": 47052, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_forward": 0, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_discover": 0, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_dir_fetch": 0, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_remote_ino": 0, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "traverse_lock": 161, 2024-04-25T08:38:08.616 INFO:teuthology.orchestra.run.smithi038.stdout: "load_cent": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "q": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "exported": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "exported_inodes": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "imported": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "imported_inodes": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_dir_fetch": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_backtrace_fetch": 1776, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "openino_peer_discover": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rfiles": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rbytes": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "root_rsnaps": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_fetch": 1776, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_set_tag": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T08:38:08.617 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_inotable_repaired": 0, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_inodes": 11, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dir_base_inodes": 1, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_dirfrag_rstats": 10, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_file_inodes": 1765, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_inode_file_caps": 0, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_revoke": 6203, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_grant": 6010, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps": 6203, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_cap_release": 25, 2024-04-25T08:38:08.618 INFO:teuthology.orchestra.run.smithi038.stdout: "process_request_cap_release": 17853 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_cache": { 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update": 0, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_update_receipt": 0, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_try_discover": 0, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_send_discover": 0, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "dir_handle_discover": 0, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays": 0, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_delayed": 0, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "num_strays_enqueuing": 0, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_created": 4127, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_enqueued": 3285, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_reintegrated": 2236, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "strays_migrated": 0, 2024-04-25T08:38:08.619 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_processing": 0, 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_enqueued": 0, 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: "num_recovering_prioritized": 0, 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_started": 0, 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: "recovery_completed": 0, 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_exportdir": 0, 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_flush": 0, 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragmentdir": 0, 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_fragstats": 0, 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: "ireq_inodestats": 0 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_log": { 2024-04-25T08:38:08.620 INFO:teuthology.orchestra.run.smithi038.stdout: "evlrg": 0, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "evadd": 23481, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "evex": 0, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "evtrm": 0, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "ev": 23481, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "evexg": 0, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "evexd": 0, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "segadd": 30, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "segex": 0, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "segtrm": 0, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "seg": 30, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "segmjr": 3, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "segexg": 0, 2024-04-25T08:38:08.621 INFO:teuthology.orchestra.run.smithi038.stdout: "segexd": 0, 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: "expos": 4194304, 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: "wrpos": 81981514, 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: "rdpos": 0, 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: "jlat": { 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6571, 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 67.644484738, 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.010294397 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: "replayed": 0 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_mem": { 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: "ino": 14, 2024-04-25T08:38:08.622 INFO:teuthology.orchestra.run.smithi038.stdout: "ino+": 2015, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "ino-": 2001, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "dir": 13, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "dir+": 13, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "dir-": 0, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "dn": 17868, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "dn+": 19868, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "dn-": 2000, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "cap": 2, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "cap+": 2002, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "cap-": 2000, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "rss": 97232, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "heap": 215396 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.623 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_server": { 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_client_request": 38030, 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "dispatch_server_request": 0, 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_request": 28303, 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_client_session": 7, 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "handle_peer_request": 0, 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "req_create_latency": { 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2000, 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 1.353677167, 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000676838 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getattr_latency": { 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2008, 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.594442108, 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000296036 2024-04-25T08:38:08.624 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getfilelock_latency": { 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "req_link_latency": { 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6000, 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 9.429110689, 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001571518 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookup_latency": { 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 8046, 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 7.163849316, 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000890361 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.625 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookuphash_latency": { 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupino_latency": { 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupname_latency": { 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.626 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupparent_latency": { 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lookupsnap_latency": { 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3, 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.002449250, 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000816416 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "req_lssnap_latency": { 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mkdir_latency": { 2024-04-25T08:38:08.627 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.002671373, 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002671373 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mknod_latency": { 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: "req_mksnap_latency": { 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.005265049, 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.005265049 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.628 INFO:teuthology.orchestra.run.smithi038.stdout: "req_open_latency": { 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: "req_readdir_latency": { 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 6, 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.092269220, 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.015378203 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rename_latency": { 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 2236, 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 197.798565641, 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.088460896 2024-04-25T08:38:08.629 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: "req_renamesnap_latency": { 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: "req_snapdiff_latency": { 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmdir_latency": { 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.630 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmsnap_latency": { 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.003176777, 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.003176777 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: "req_rmxattr_latency": { 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setattr_latency": { 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 1, 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000366527, 2024-04-25T08:38:08.631 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000366527 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setdirlayout_latency": { 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setfilelock_latency": { 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setlayout_latency": { 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.632 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: "req_setxattr_latency": { 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: "req_symlink_latency": { 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: "req_unlink_latency": { 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 8000, 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 15.145905608, 2024-04-25T08:38:08.633 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001893238 2024-04-25T08:38:08.634 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.634 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_revoke_eviction": 0, 2024-04-25T08:38:08.634 INFO:teuthology.orchestra.run.smithi038.stdout: "cap_acquisition_throttle": 0, 2024-04-25T08:38:08.634 INFO:teuthology.orchestra.run.smithi038.stdout: "req_getvxattr_latency": { 2024-04-25T08:38:08.634 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.634 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.634 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.634 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:08.634 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.634 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_sessions": { 2024-04-25T08:38:08.634 INFO:teuthology.orchestra.run.smithi038.stdout: "session_count": 1, 2024-04-25T08:38:08.634 INFO:teuthology.orchestra.run.smithi038.stdout: "session_add": 1, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "session_remove": 0, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_open": 1, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "sessions_stale": 0, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "total_load": 19606, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "average_load": 19606, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "avg_session_uptime": 88, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "mdthresh_evicted": 0 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T08:38:08.635 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T08:38:08.636 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-04-25T08:38:08.637 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 251336, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 86, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 5528, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 78, 2024-04-25T08:38:08.638 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 13454124, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 661179, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "objecter": { 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "op_active": 0, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "op_laggy": 0, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send": 11893, 2024-04-25T08:38:08.639 INFO:teuthology.orchestra.run.smithi038.stdout: "op_send_bytes": 78158074, 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: "op_resend": 36, 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: "op_reply": 11893, 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: "op_latency": { 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 11893, 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 28.909484294, 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.002430798 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: "op_inflight": 0, 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: "oplen_avg": { 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 11893, 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 11955 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.640 INFO:teuthology.orchestra.run.smithi038.stdout: "op": 11893, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "op_r": 1839, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "op_w": 10054, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "op_rmw": 0, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "op_pg": 0, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_stat": 0, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_create": 11, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_read": 62, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_write": 6632, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writefull": 40, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_writesame": 0, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_append": 0, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_zero": 1285, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_truncate": 0, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_delete": 2030, 2024-04-25T08:38:08.641 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_mapext": 0, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_sparse_read": 0, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_clonerange": 0, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_getxattr": 1777, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_setxattr": 22, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_cmpxattr": 0, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_rmxattr": 0, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_resetxattrs": 0, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_call": 0, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_watch": 0, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_notify": 0, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls": 0, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_pgls_filter": 0, 2024-04-25T08:38:08.642 INFO:teuthology.orchestra.run.smithi038.stdout: "osdop_other": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_active": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_send": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_resend": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "linger_ping": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_active": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_send": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "poolop_resend": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_active": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_send": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "poolstat_resend": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_active": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_send": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "statfs_resend": 0, 2024-04-25T08:38:08.643 INFO:teuthology.orchestra.run.smithi038.stdout: "command_active": 0, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "command_send": 0, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "command_resend": 0, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "map_epoch": 84, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "map_full": 0, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "map_inc": 3, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_sessions": 8, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_open": 8, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_session_close": 0, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_laggy": 0, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_wr": 80, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_rd": 0, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_del": 16 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.644 INFO:teuthology.orchestra.run.smithi038.stdout: "oft": { 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_objs": 1, 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_kv_pairs": 0, 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_updates": 0, 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: "omap_total_removes": 0 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: "osd-slow-ops": { 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: "slow_ops_count": 0 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: "purge_queue": { 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops": 0, 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing": 0, 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executing_high_water": 32, 2024-04-25T08:38:08.645 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed_ops": 3285, 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_executed": 3285, 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: "pq_item_in_journal": 0 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 44439, 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 10922624, 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 44439, 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T08:38:08.646 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 44439, 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 10922624, 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_bytes": { 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T08:38:08.647 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 11893, 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 78500139, 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 11893, 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 78500139, 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.648 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-objecter_ops": { 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 1024, 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 0, 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 0, 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 0, 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 11893, 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 11893, 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 11893, 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 11893, 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:38:08.649 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle": { 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 3285, 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 342065, 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 3285, 2024-04-25T08:38:08.650 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 61, 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 342065, 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-write_buf_throttle-0x55ba286926e0": { 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 3758096384, 2024-04-25T08:38:08.651 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 23481, 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 77787210, 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 23481, 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 6571, 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 77787210, 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-04-25T08:38:08.652 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:08.653 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:08.653 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:38:08.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed4568640 1 -- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0eb006b4d0 msgr2=0x7f0eb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed4568640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0eb006b4d0 0x7f0eb006d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc006fd0 tx=0x7f0ebc008040 comp rx=0 tx=0).stop 2024-04-25T08:38:08.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed4568640 1 -- 172.21.15.38:0/3613354442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ecc099fb0 msgr2=0x7f0ecc09eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed4568640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ecc099fb0 0x7f0ecc09eb80 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8009870 tx=0x7f0ec80046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed4568640 1 -- 172.21.15.38:0/3613354442 shutdown_connections 2024-04-25T08:38:08.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed4568640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ecc162390 0x7f0ecc0c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed4568640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0eb006b4d0 0x7f0eb006d990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed4568640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ecc15df30 0x7f0ecc0c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed4568640 1 --2- 172.21.15.38:0/3613354442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ecc099fb0 0x7f0ecc09eb80 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed4568640 1 -- 172.21.15.38:0/3613354442 >> 172.21.15.38:0/3613354442 conn(0x7f0ecc0b3110 msgr2=0x7f0ecc161870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed4568640 1 -- 172.21.15.38:0/3613354442 shutdown_connections 2024-04-25T08:38:08.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.608+0000 7f0ed4568640 1 -- 172.21.15.38:0/3613354442 wait complete. 2024-04-25T08:38:08.656 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:38:08.656 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:38:08.690 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:38:08.735 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:38:08.736 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:38:08.736 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:08.752 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:08.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.899+0000 7fea00f30640 1 Processor -- start 2024-04-25T08:38:08.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.899+0000 7fea00f30640 1 -- start start 2024-04-25T08:38:08.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.899+0000 7fea00f30640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0d3900 0x7fe9fc0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.899+0000 7fea00f30640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9fc0d3e70 con 0x7fe9fc0d0d90 2024-04-25T08:38:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.899+0000 7fea00f30640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9fc0d4150 con 0x7fe9fc15dc10 2024-04-25T08:38:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.899+0000 7fea00f30640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc15dff0 con 0x7fe9fc0d3900 2024-04-25T08:38:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9fad9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe9fc0d0d90 0x7fe9fc0b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35342/0 (socket says 172.21.15.38:35342) 2024-04-25T08:38:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9fad9d640 1 -- 172.21.15.38:0/1418526674 learned_addr learned my addr 172.21.15.38:0/1418526674 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814407071 0 0) 0x7fe9fc0d3e70 con 0x7fe9fc0d0d90 2024-04-25T08:38:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e8003520 con 0x7fe9fc0d0d90 2024-04-25T08:38:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f3fff640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0d3900 0x7fe9fc0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1498783979 0 0) 0x7fe9e8003520 con 0x7fe9fc0d0d90 2024-04-25T08:38:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0d3900 msgr2=0x7fe9fc0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0d3900 0x7fe9fc0d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 >> v1:172.21.15.107:6789/0 conn(0x7fe9fc15dc10 legacy=0x7fe9fc0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc0d4430 con 0x7fe9fc0d0d90 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fe9f40019f0 con 0x7fe9fc0d0d90 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 >> v1:172.21.15.38:6789/0 conn(0x7fe9fc0d0d90 legacy=0x7fe9fc0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9e80039e0 0x7fe9e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0d3900 0x7fe9e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e8004d40 0x7fe9e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc15dff0 con 0x7fe9e80039e0 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e800baf0 con 0x7fe9e8004d40 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e800bcf0 con 0x7fe9fc0d3900 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9fa59c640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9e80039e0 0x7fe9e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9f3fff640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0d3900 0x7fe9e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.900+0000 7fe9fad9d640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e8004d40 0x7fe9e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.901+0000 7fe9fa59c640 1 -- 172.21.15.38:0/1418526674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0d3900 msgr2=0x7fe9e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.901+0000 7fe9fa59c640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc0d3900 0x7fe9e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.901+0000 7fe9fa59c640 1 -- 172.21.15.38:0/1418526674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e8004d40 msgr2=0x7fe9e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.901+0000 7fe9fa59c640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e8004d40 0x7fe9e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.901+0000 7fe9fa59c640 1 -- 172.21.15.38:0/1418526674 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e8003520 con 0x7fe9e80039e0 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.901+0000 7fe9fad9d640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e8004d40 0x7fe9e80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.901+0000 7fe9fa59c640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9e80039e0 0x7fe9e8003e00 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec001750 tx=0x7fe9e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260b58334d6a4210 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.901+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec001370 con 0x7fe9e80039e0 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.901+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ec0052d0 con 0x7fe9e80039e0 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.901+0000 7fe9f9d9b640 1 -- 172.21.15.38:0/1418526674 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec02ab20 con 0x7fe9e80039e0 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.902+0000 7fea00f30640 1 -- 172.21.15.38:0/1418526674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9e80039e0 msgr2=0x7fe9e8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.902+0000 7fea00f30640 1 --2- 172.21.15.38:0/1418526674 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9e80039e0 0x7fe9e8003e00 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec001750 tx=0x7fe9e8005830 comp rx=0 tx=0).stop 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.902+0000 7fe9fad9d640 1 -- 172.21.15.38:0/1418526674 reap_dead start 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.902+0000 7fea00f30640 1 -- 172.21.15.38:0/1418526674 shutdown_connections 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.902+0000 7fea00f30640 1 -- 172.21.15.38:0/1418526674 >> 172.21.15.38:0/1418526674 conn(0x7fe9fc066d10 msgr2=0x7fe9fc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:08.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.902+0000 7fea00f30640 1 -- 172.21.15.38:0/1418526674 shutdown_connections 2024-04-25T08:38:08.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.902+0000 7fea00f30640 1 -- 172.21.15.38:0/1418526674 wait complete. 2024-04-25T08:38:08.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.903+0000 7fea00f30640 1 Processor -- start 2024-04-25T08:38:08.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.903+0000 7fea00f30640 1 -- start start 2024-04-25T08:38:08.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fea00f30640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e80039e0 0x7fe9fc0ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fea00f30640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc15c0b0 0x7fe9fc0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fea00f30640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b3ba0 0x7fe9fc1f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fea00f30640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0d4710 con 0x7fe9fc0b3ba0 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fea00f30640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0cef30 con 0x7fe9e80039e0 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fea00f30640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0b80f0 con 0x7fe9fc15c0b0 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fe9fad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b3ba0 0x7fe9fc1f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fe9fad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b3ba0 0x7fe9fc1f6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48860/0 (socket says 172.21.15.38:48860) 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fe9fad9d640 1 -- 172.21.15.38:0/608173595 learned_addr learned my addr 172.21.15.38:0/608173595 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fe9fa59c640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e80039e0 0x7fe9fc0ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fe9f3fff640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc15c0b0 0x7fe9fc0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fe9fad9d640 1 -- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc15c0b0 msgr2=0x7fe9fc0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fe9fad9d640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc15c0b0 0x7fe9fc0b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fe9fad9d640 1 -- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e80039e0 msgr2=0x7fe9fc0ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fe9fad9d640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e80039e0 0x7fe9fc0ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fe9fad9d640 1 -- 172.21.15.38:0/608173595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc0ab2e0 con 0x7fe9fc0b3ba0 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.904+0000 7fe9fa59c640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e80039e0 0x7fe9fc0ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:08.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.905+0000 7fe9fad9d640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b3ba0 0x7fe9fc1f6be0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0b5ae0 tx=0x7fe9f400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.905+0000 7fe9f2ffd640 1 -- 172.21.15.38:0/608173595 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f400ec10 con 0x7fe9fc0b3ba0 2024-04-25T08:38:08.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.905+0000 7fe9f2ffd640 1 -- 172.21.15.38:0/608173595 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f400edb0 con 0x7fe9fc0b3ba0 2024-04-25T08:38:08.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.905+0000 7fea00f30640 1 -- 172.21.15.38:0/608173595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc0aa760 con 0x7fe9fc0b3ba0 2024-04-25T08:38:08.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.905+0000 7fe9f2ffd640 1 -- 172.21.15.38:0/608173595 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4014650 con 0x7fe9fc0b3ba0 2024-04-25T08:38:08.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.905+0000 7fea00f30640 1 -- 172.21.15.38:0/608173595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc0aad20 con 0x7fe9fc0b3ba0 2024-04-25T08:38:08.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.906+0000 7fea00f30640 1 -- 172.21.15.38:0/608173595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9fc068e80 con 0x7fe9fc0b3ba0 2024-04-25T08:38:08.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.906+0000 7fe9f2ffd640 1 -- 172.21.15.38:0/608173595 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9f40147f0 con 0x7fe9fc0b3ba0 2024-04-25T08:38:08.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.907+0000 7fe9f2ffd640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9dc06b500 0x7fe9dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:08.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.907+0000 7fe9f2ffd640 1 -- 172.21.15.38:0/608173595 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fe9f408a0c0 con 0x7fe9fc0b3ba0 2024-04-25T08:38:08.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.907+0000 7fe9fa59c640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9dc06b500 0x7fe9dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:08.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.907+0000 7fe9fa59c640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9dc06b500 0x7fe9dc06d9c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec032000 tx=0x7fe9ec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:08.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:08.911+0000 7fe9f2ffd640 1 -- 172.21.15.38:0/608173595 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9f4058e20 con 0x7fe9fc0b3ba0 2024-04-25T08:38:09.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.093+0000 7fea00f30640 1 -- 172.21.15.38:0/608173595 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9fc0b4880 con 0x7fe9fc0b3ba0 2024-04-25T08:38:09.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.094+0000 7fe9f2ffd640 1 -- 172.21.15.38:0/608173595 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16999 (secure 0 0 0) 0x7fe9f4058720 con 0x7fe9fc0b3ba0 2024-04-25T08:38:09.089 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:09.089 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":84,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:37:50.914128+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:25.176504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:25.877954+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":83,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:09.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.096+0000 7fea00f30640 1 -- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9dc06b500 msgr2=0x7fe9dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.096+0000 7fea00f30640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9dc06b500 0x7fe9dc06d9c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec032000 tx=0x7fe9ec02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:09.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.096+0000 7fea00f30640 1 -- 172.21.15.38:0/608173595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b3ba0 msgr2=0x7fe9fc1f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.096+0000 7fea00f30640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b3ba0 0x7fe9fc1f6be0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0b5ae0 tx=0x7fe9f400e530 comp rx=0 tx=0).stop 2024-04-25T08:38:09.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.096+0000 7fea00f30640 1 -- 172.21.15.38:0/608173595 shutdown_connections 2024-04-25T08:38:09.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.096+0000 7fea00f30640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9fc0b3ba0 0x7fe9fc1f6be0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.096+0000 7fea00f30640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9dc06b500 0x7fe9dc06d9c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.096+0000 7fea00f30640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9fc15c0b0 0x7fe9fc0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.096+0000 7fea00f30640 1 --2- 172.21.15.38:0/608173595 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9e80039e0 0x7fe9fc0ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.096+0000 7fea00f30640 1 -- 172.21.15.38:0/608173595 >> 172.21.15.38:0/608173595 conn(0x7fe9fc066d10 msgr2=0x7fe9fc0d2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:09.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.096+0000 7fea00f30640 1 -- 172.21.15.38:0/608173595 shutdown_connections 2024-04-25T08:38:09.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.097+0000 7fea00f30640 1 -- 172.21.15.38:0/608173595 wait complete. 2024-04-25T08:38:09.102 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:36:25.176504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:36:25.877954+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 83, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:38:09.103 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:38:09.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.213+0000 7f6e645e0640 1 Processor -- start 2024-04-25T08:38:09.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.213+0000 7f6e645e0640 1 -- start start 2024-04-25T08:38:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e645e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cfd40 0x7f6e5c0c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e645e0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6e5c0ccfc0 0x7f6e5c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e645e0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e5c0ac430 con 0x7f6e5c15db80 2024-04-25T08:38:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e645e0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5c0ac1e0 con 0x7f6e5c0ccfc0 2024-04-25T08:38:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e645e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0ac710 con 0x7f6e5c0cfd40 2024-04-25T08:38:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e61b7b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cfd40 0x7f6e5c0c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e6237c640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f6e5c15db80 0x7f6e5c0cf630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:60246/0 (socket says 172.21.15.38:60246) 2024-04-25T08:38:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e61b7b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cfd40 0x7f6e5c0c9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:45826/0 (socket says 172.21.15.38:45826) 2024-04-25T08:38:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e6237c640 1 -- 172.21.15.38:0/2153620611 learned_addr learned my addr 172.21.15.38:0/2153620611 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e62b7d640 1 --2- 172.21.15.38:0/2153620611 >> v2:172.21.15.107:3300/0 conn(0x7f6e5c0ccfc0 0x7f6e5c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e61b7b640 1 -- 172.21.15.38:0/2153620611 >> v2:172.21.15.107:3300/0 conn(0x7f6e5c0ccfc0 msgr2=0x7f6e5c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e61b7b640 1 --2- 172.21.15.38:0/2153620611 >> v2:172.21.15.107:3300/0 conn(0x7f6e5c0ccfc0 0x7f6e5c0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e61b7b640 1 -- 172.21.15.38:0/2153620611 >> v1:172.21.15.107:6789/0 conn(0x7f6e5c15db80 legacy=0x7f6e5c0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e6137a640 1 -- 172.21.15.38:0/2153620611 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495421767 0 0) 0x7f6e5c0ac430 con 0x7f6e5c15db80 2024-04-25T08:38:09.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.214+0000 7f6e61b7b640 1 -- 172.21.15.38:0/2153620611 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c0ac9f0 con 0x7f6e5c0cfd40 2024-04-25T08:38:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.215+0000 7f6e61b7b640 1 --2- 172.21.15.38:0/2153620611 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cfd40 0x7f6e5c0c9ad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00ba40 tx=0x7f6e4c02c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fd4e91026d0392b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.215+0000 7f6e6137a640 1 -- 172.21.15.38:0/2153620611 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c032750 con 0x7f6e5c0cfd40 2024-04-25T08:38:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.215+0000 7f6e6137a640 1 -- 172.21.15.38:0/2153620611 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c030050 con 0x7f6e5c0cfd40 2024-04-25T08:38:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.215+0000 7f6e6137a640 1 -- 172.21.15.38:0/2153620611 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c032b10 con 0x7f6e5c0cfd40 2024-04-25T08:38:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.216+0000 7f6e645e0640 1 -- 172.21.15.38:0/2153620611 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cfd40 msgr2=0x7f6e5c0c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.216+0000 7f6e645e0640 1 --2- 172.21.15.38:0/2153620611 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cfd40 0x7f6e5c0c9ad0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00ba40 tx=0x7f6e4c02c850 comp rx=0 tx=0).stop 2024-04-25T08:38:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.216+0000 7f6e645e0640 1 -- 172.21.15.38:0/2153620611 shutdown_connections 2024-04-25T08:38:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.216+0000 7f6e645e0640 1 --2- 172.21.15.38:0/2153620611 >> v2:172.21.15.107:3300/0 conn(0x7f6e5c0ccfc0 0x7f6e5c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.216+0000 7f6e645e0640 1 --2- 172.21.15.38:0/2153620611 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cfd40 0x7f6e5c0c9ad0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.216+0000 7f6e645e0640 1 -- 172.21.15.38:0/2153620611 >> 172.21.15.38:0/2153620611 conn(0x7f6e5c06adc0 msgr2=0x7f6e5c06b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:09.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.216+0000 7f6e645e0640 1 -- 172.21.15.38:0/2153620611 shutdown_connections 2024-04-25T08:38:09.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.216+0000 7f6e645e0640 1 -- 172.21.15.38:0/2153620611 wait complete. 2024-04-25T08:38:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.217+0000 7f6e645e0640 1 Processor -- start 2024-04-25T08:38:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.217+0000 7f6e645e0640 1 -- start start 2024-04-25T08:38:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e645e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5c0ccfc0 0x7f6e5c1546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e645e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cfd40 0x7f6e5c15ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e645e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c15db80 0x7f6e5c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e645e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c15df60 con 0x7f6e5c15db80 2024-04-25T08:38:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e645e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0ca550 con 0x7f6e5c0ccfc0 2024-04-25T08:38:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e645e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c15e2c0 con 0x7f6e5c0cfd40 2024-04-25T08:38:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e62b7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c15db80 0x7f6e5c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e62b7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c15db80 0x7f6e5c15bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48864/0 (socket says 172.21.15.38:48864) 2024-04-25T08:38:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e62b7d640 1 -- 172.21.15.38:0/797322176 learned_addr learned my addr 172.21.15.38:0/797322176 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:09.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e6237c640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5c0ccfc0 0x7f6e5c1546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e61b7b640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cfd40 0x7f6e5c15ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e62b7d640 1 -- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cfd40 msgr2=0x7f6e5c15ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e62b7d640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cfd40 0x7f6e5c15ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e62b7d640 1 -- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5c0ccfc0 msgr2=0x7f6e5c1546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e62b7d640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5c0ccfc0 0x7f6e5c1546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e62b7d640 1 -- 172.21.15.38:0/797322176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c156100 con 0x7f6e5c15db80 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e62b7d640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c15db80 0x7f6e5c15bfc0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009e30 tx=0x7f6e5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.218+0000 7f6e52ffd640 1 -- 172.21.15.38:0/797322176 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54018020 con 0x7f6e5c15db80 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.219+0000 7f6e52ffd640 1 -- 172.21.15.38:0/797322176 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e54014480 con 0x7f6e5c15db80 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.219+0000 7f6e645e0640 1 -- 172.21.15.38:0/797322176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5c1563e0 con 0x7f6e5c15db80 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.219+0000 7f6e52ffd640 1 -- 172.21.15.38:0/797322176 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54014870 con 0x7f6e5c15db80 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.219+0000 7f6e645e0640 1 -- 172.21.15.38:0/797322176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5c070eb0 con 0x7f6e5c15db80 2024-04-25T08:38:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.219+0000 7f6e645e0640 1 -- 172.21.15.38:0/797322176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5c068e80 con 0x7f6e5c15db80 2024-04-25T08:38:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.220+0000 7f6e52ffd640 1 -- 172.21.15.38:0/797322176 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e54005d10 con 0x7f6e5c15db80 2024-04-25T08:38:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.221+0000 7f6e52ffd640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e4006b4d0 0x7f6e4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.221+0000 7f6e52ffd640 1 -- 172.21.15.38:0/797322176 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f6e5408b090 con 0x7f6e5c15db80 2024-04-25T08:38:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.221+0000 7f6e6237c640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e4006b4d0 0x7f6e4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.221+0000 7f6e6237c640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e4006b4d0 0x7f6e4006d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6e58007fd0 tx=0x7f6e58007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.225+0000 7f6e52ffd640 1 -- 172.21.15.38:0/797322176 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e54022030 con 0x7f6e5c15db80 2024-04-25T08:38:09.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.405+0000 7f6e645e0640 1 -- 172.21.15.38:0/797322176 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e5c0b4230 con 0x7f6e5c15db80 2024-04-25T08:38:09.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.405+0000 7f6e52ffd640 1 -- 172.21.15.38:0/797322176 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e54010040 con 0x7f6e5c15db80 2024-04-25T08:38:09.399 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:38:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.407+0000 7f6e645e0640 1 -- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e4006b4d0 msgr2=0x7f6e4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.407+0000 7f6e645e0640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e4006b4d0 0x7f6e4006d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6e58007fd0 tx=0x7f6e58007de0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.407+0000 7f6e645e0640 1 -- 172.21.15.38:0/797322176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c15db80 msgr2=0x7f6e5c15bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.407+0000 7f6e645e0640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c15db80 0x7f6e5c15bfc0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009e30 tx=0x7f6e5400c490 comp rx=0 tx=0).stop 2024-04-25T08:38:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.407+0000 7f6e645e0640 1 -- 172.21.15.38:0/797322176 shutdown_connections 2024-04-25T08:38:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.407+0000 7f6e645e0640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5c15db80 0x7f6e5c15bfc0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.407+0000 7f6e645e0640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e4006b4d0 0x7f6e4006d990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.407+0000 7f6e645e0640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5c0cfd40 0x7f6e5c15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.407+0000 7f6e645e0640 1 --2- 172.21.15.38:0/797322176 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5c0ccfc0 0x7f6e5c1546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.407+0000 7f6e645e0640 1 -- 172.21.15.38:0/797322176 >> 172.21.15.38:0/797322176 conn(0x7f6e5c06adc0 msgr2=0x7f6e5c0b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:09.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.407+0000 7f6e645e0640 1 -- 172.21.15.38:0/797322176 shutdown_connections 2024-04-25T08:38:09.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.407+0000 7f6e645e0640 1 -- 172.21.15.38:0/797322176 wait complete. 2024-04-25T08:38:09.412 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:38:09.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.522+0000 7f267d6b3640 1 Processor -- start 2024-04-25T08:38:09.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.522+0000 7f267d6b3640 1 -- start start 2024-04-25T08:38:09.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f267d6b3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f26780d33d0 0x7f26780d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f267d6b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26780d1e90 0x7f26780b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f267d6b3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26780d2330 con 0x7f26780d0320 2024-04-25T08:38:09.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f267d6b3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26780d2610 con 0x7f26780d33d0 2024-04-25T08:38:09.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f267d6b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26780d28f0 con 0x7f26780d1e90 2024-04-25T08:38:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f2676ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f26780d0320 0x7f26780d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35346/0 (socket says 172.21.15.38:35346) 2024-04-25T08:38:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f2676ffd640 1 -- 172.21.15.38:0/183614878 learned_addr learned my addr 172.21.15.38:0/183614878 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f26777fe640 1 --2- 172.21.15.38:0/183614878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26780d1e90 0x7f26780b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f26767fc640 1 --2- 172.21.15.38:0/183614878 >> v2:172.21.15.107:3300/0 conn(0x7f26780d33d0 0x7f26780d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f2675ffb640 1 -- 172.21.15.38:0/183614878 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938737966 0 0) 0x7f26780d2330 con 0x7f26780d0320 2024-04-25T08:38:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f2675ffb640 1 -- 172.21.15.38:0/183614878 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2660003520 con 0x7f26780d0320 2024-04-25T08:38:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f26777fe640 1 -- 172.21.15.38:0/183614878 >> v2:172.21.15.107:3300/0 conn(0x7f26780d33d0 msgr2=0x7f26780d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f26777fe640 1 --2- 172.21.15.38:0/183614878 >> v2:172.21.15.107:3300/0 conn(0x7f26780d33d0 0x7f26780d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f26777fe640 1 -- 172.21.15.38:0/183614878 >> v1:172.21.15.38:6789/0 conn(0x7f26780d0320 legacy=0x7f26780d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.523+0000 7f26777fe640 1 -- 172.21.15.38:0/183614878 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26780d2bd0 con 0x7f26780d1e90 2024-04-25T08:38:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.524+0000 7f26777fe640 1 --2- 172.21.15.38:0/183614878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26780d1e90 0x7f26780b6980 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2670009d90 tx=0x7f267002a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41d0e4a8eb659c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.524+0000 7f2675ffb640 1 -- 172.21.15.38:0/183614878 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670007740 con 0x7f26780d1e90 2024-04-25T08:38:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.524+0000 7f2675ffb640 1 -- 172.21.15.38:0/183614878 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26700078e0 con 0x7f26780d1e90 2024-04-25T08:38:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.524+0000 7f2675ffb640 1 -- 172.21.15.38:0/183614878 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670007cb0 con 0x7f26780d1e90 2024-04-25T08:38:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.525+0000 7f267d6b3640 1 -- 172.21.15.38:0/183614878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26780d1e90 msgr2=0x7f26780b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.525+0000 7f267d6b3640 1 --2- 172.21.15.38:0/183614878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26780d1e90 0x7f26780b6980 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2670009d90 tx=0x7f267002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.525+0000 7f267d6b3640 1 -- 172.21.15.38:0/183614878 shutdown_connections 2024-04-25T08:38:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.525+0000 7f267d6b3640 1 --2- 172.21.15.38:0/183614878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26780d1e90 0x7f26780b6980 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.525+0000 7f267d6b3640 1 --2- 172.21.15.38:0/183614878 >> v2:172.21.15.107:3300/0 conn(0x7f26780d33d0 0x7f26780d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.525+0000 7f267d6b3640 1 -- 172.21.15.38:0/183614878 >> 172.21.15.38:0/183614878 conn(0x7f26780668f0 msgr2=0x7f2678066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.525+0000 7f267d6b3640 1 -- 172.21.15.38:0/183614878 shutdown_connections 2024-04-25T08:38:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.525+0000 7f267d6b3640 1 -- 172.21.15.38:0/183614878 wait complete. 2024-04-25T08:38:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.526+0000 7f267d6b3640 1 Processor -- start 2024-04-25T08:38:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.526+0000 7f267d6b3640 1 -- start start 2024-04-25T08:38:09.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f267d6b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26780d0320 0x7f26780b1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f267d6b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26780d1e90 0x7f26780aa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f267d6b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26780d33d0 0x7f26781e7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f267d6b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678161390 con 0x7f26780d33d0 2024-04-25T08:38:09.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f267d6b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678161210 con 0x7f26780d1e90 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f267d6b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26780b9720 con 0x7f26780d0320 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f26777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26780d33d0 0x7f26781e7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f26777fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26780d33d0 0x7f26781e7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48876/0 (socket says 172.21.15.38:48876) 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f26777fe640 1 -- 172.21.15.38:0/3210355986 learned_addr learned my addr 172.21.15.38:0/3210355986 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f26767fc640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26780d1e90 0x7f26780aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f2676ffd640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26780d0320 0x7f26780b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f26777fe640 1 -- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26780d0320 msgr2=0x7f26780b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f26777fe640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26780d0320 0x7f26780b1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f26777fe640 1 -- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26780d1e90 msgr2=0x7f26780aa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f26777fe640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26780d1e90 0x7f26780aa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f26777fe640 1 -- 172.21.15.38:0/3210355986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26781e8400 con 0x7f26780d33d0 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.527+0000 7f26767fc640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26780d1e90 0x7f26780aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:09.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.528+0000 7f26777fe640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26780d33d0 0x7f26781e7ec0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f2670009eb0 tx=0x7f2670004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.528+0000 7f265f7fe640 1 -- 172.21.15.38:0/3210355986 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670040070 con 0x7f26780d33d0 2024-04-25T08:38:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.528+0000 7f267d6b3640 1 -- 172.21.15.38:0/3210355986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26781e86d0 con 0x7f26780d33d0 2024-04-25T08:38:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.528+0000 7f265f7fe640 1 -- 172.21.15.38:0/3210355986 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2670031bc0 con 0x7f26780d33d0 2024-04-25T08:38:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.528+0000 7f265f7fe640 1 -- 172.21.15.38:0/3210355986 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2670009450 con 0x7f26780d33d0 2024-04-25T08:38:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.528+0000 7f267d6b3640 1 -- 172.21.15.38:0/3210355986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26781e8c90 con 0x7f26780d33d0 2024-04-25T08:38:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.529+0000 7f267d6b3640 1 -- 172.21.15.38:0/3210355986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2678068e80 con 0x7f26780d33d0 2024-04-25T08:38:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.529+0000 7f265f7fe640 1 -- 172.21.15.38:0/3210355986 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2670039040 con 0x7f26780d33d0 2024-04-25T08:38:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.530+0000 7f265f7fe640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f266406b4d0 0x7f266406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.530+0000 7f265f7fe640 1 -- 172.21.15.38:0/3210355986 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f26700aba80 con 0x7f26780d33d0 2024-04-25T08:38:09.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.530+0000 7f2676ffd640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f266406b4d0 0x7f266406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.530+0000 7f2676ffd640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f266406b4d0 0x7f266406d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2668005880 tx=0x7f2668009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.534+0000 7f265f7fe640 1 -- 172.21.15.38:0/3210355986 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f267007a880 con 0x7f26780d33d0 2024-04-25T08:38:09.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.712+0000 7f267d6b3640 1 -- 172.21.15.38:0/3210355986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26780b3050 con 0x7f26780d33d0 2024-04-25T08:38:09.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.712+0000 7f265f7fe640 1 -- 172.21.15.38:0/3210355986 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f267007a180 con 0x7f26780d33d0 2024-04-25T08:38:09.706 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:38:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.714+0000 7f267d6b3640 1 -- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f266406b4d0 msgr2=0x7f266406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.714+0000 7f267d6b3640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f266406b4d0 0x7f266406d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2668005880 tx=0x7f2668009040 comp rx=0 tx=0).stop 2024-04-25T08:38:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.714+0000 7f267d6b3640 1 -- 172.21.15.38:0/3210355986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26780d33d0 msgr2=0x7f26781e7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.714+0000 7f267d6b3640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26780d33d0 0x7f26781e7ec0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f2670009eb0 tx=0x7f2670004a40 comp rx=0 tx=0).stop 2024-04-25T08:38:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.714+0000 7f267d6b3640 1 -- 172.21.15.38:0/3210355986 shutdown_connections 2024-04-25T08:38:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.714+0000 7f267d6b3640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26780d33d0 0x7f26781e7ec0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.714+0000 7f267d6b3640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26780d1e90 0x7f26780aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.714+0000 7f267d6b3640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f266406b4d0 0x7f266406d990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.714+0000 7f267d6b3640 1 --2- 172.21.15.38:0/3210355986 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26780d0320 0x7f26780b1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.714+0000 7f267d6b3640 1 -- 172.21.15.38:0/3210355986 >> 172.21.15.38:0/3210355986 conn(0x7f26780668f0 msgr2=0x7f26780cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:09.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.714+0000 7f267d6b3640 1 -- 172.21.15.38:0/3210355986 shutdown_connections 2024-04-25T08:38:09.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.714+0000 7f267d6b3640 1 -- 172.21.15.38:0/3210355986 wait complete. 2024-04-25T08:38:09.719 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:38:09.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.829+0000 7f543d2b3640 1 Processor -- start 2024-04-25T08:38:09.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.829+0000 7f543d2b3640 1 -- start start 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f543d2b3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f54380d0320 0x7f54380d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f543d2b3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f54380b6980 0x7f54380b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f543d2b3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54380d2060 con 0x7f54380d0700 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f543d2b3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54380d2330 con 0x7f54380b6980 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f543d2b3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54380d2610 con 0x7f54380d0320 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f54377fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f54380b6980 0x7f54380b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f54377fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f54380b6980 0x7f54380b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48880/0 (socket says 172.21.15.38:48880) 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f54377fe640 1 -- 172.21.15.38:0/1033107766 learned_addr learned my addr 172.21.15.38:0/1033107766 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f5436ffd640 1 --2- 172.21.15.38:0/1033107766 >> v2:172.21.15.107:3300/0 conn(0x7f54380d0320 0x7f54380d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f54377fe640 1 -- 172.21.15.38:0/1033107766 >> v2:172.21.15.107:3300/0 conn(0x7f54380d0320 msgr2=0x7f54380d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f54377fe640 1 --2- 172.21.15.38:0/1033107766 >> v2:172.21.15.107:3300/0 conn(0x7f54380d0320 0x7f54380d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f54377fe640 1 -- 172.21.15.38:0/1033107766 >> v1:172.21.15.107:6789/0 conn(0x7f54380d0700 legacy=0x7f54380d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:09.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.830+0000 7f54377fe640 1 -- 172.21.15.38:0/1033107766 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54380d28f0 con 0x7f54380b6980 2024-04-25T08:38:09.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.831+0000 7f54377fe640 1 --2- 172.21.15.38:0/1033107766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380b6980 0x7f54380b6dd0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f5438069100 tx=0x7f543002a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e2e23bab5b290f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.831+0000 7f54367fc640 1 -- 172.21.15.38:0/1033107766 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430036070 con 0x7f54380b6980 2024-04-25T08:38:09.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.831+0000 7f54367fc640 1 -- 172.21.15.38:0/1033107766 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5430031600 con 0x7f54380b6980 2024-04-25T08:38:09.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.831+0000 7f54367fc640 1 -- 172.21.15.38:0/1033107766 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54300319d0 con 0x7f54380b6980 2024-04-25T08:38:09.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.832+0000 7f543d2b3640 1 -- 172.21.15.38:0/1033107766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380b6980 msgr2=0x7f54380b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.832+0000 7f543d2b3640 1 --2- 172.21.15.38:0/1033107766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380b6980 0x7f54380b6dd0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f5438069100 tx=0x7f543002a780 comp rx=0 tx=0).stop 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.832+0000 7f543d2b3640 1 -- 172.21.15.38:0/1033107766 shutdown_connections 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.832+0000 7f543d2b3640 1 --2- 172.21.15.38:0/1033107766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380b6980 0x7f54380b6dd0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.832+0000 7f543d2b3640 1 --2- 172.21.15.38:0/1033107766 >> v2:172.21.15.107:3300/0 conn(0x7f54380d0320 0x7f54380d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.832+0000 7f543d2b3640 1 -- 172.21.15.38:0/1033107766 >> 172.21.15.38:0/1033107766 conn(0x7f54380668f0 msgr2=0x7f5438066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.832+0000 7f543d2b3640 1 -- 172.21.15.38:0/1033107766 shutdown_connections 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.832+0000 7f543d2b3640 1 -- 172.21.15.38:0/1033107766 wait complete. 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.833+0000 7f543d2b3640 1 Processor -- start 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.833+0000 7f543d2b3640 1 -- start start 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f543d2b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380b6980 0x7f5438149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f543d2b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54380d0320 0x7f5438145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f543d2b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54380d0700 0x7f54380b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f543d2b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54380d0b40 con 0x7f54380b6980 2024-04-25T08:38:09.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f543d2b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54380d2eb0 con 0x7f54380d0700 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f543d2b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54380b59a0 con 0x7f54380d0320 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f5436ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380b6980 0x7f5438149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f5436ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380b6980 0x7f5438149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48896/0 (socket says 172.21.15.38:48896) 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f5436ffd640 1 -- 172.21.15.38:0/1363685784 learned_addr learned my addr 172.21.15.38:0/1363685784 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f542ffff640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54380d0320 0x7f5438145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f54377fe640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54380d0700 0x7f54380b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f5436ffd640 1 -- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54380d0320 msgr2=0x7f5438145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f5436ffd640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54380d0320 0x7f5438145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f5436ffd640 1 -- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54380d0700 msgr2=0x7f54380b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f5436ffd640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54380d0700 0x7f54380b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f5436ffd640 1 -- 172.21.15.38:0/1363685784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54380b4d90 con 0x7f54380b6980 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.834+0000 7f5436ffd640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380b6980 0x7f5438149040 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f542800d800 tx=0x7f542800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.835+0000 7f542f7fe640 1 -- 172.21.15.38:0/1363685784 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800b6c0 con 0x7f54380b6980 2024-04-25T08:38:09.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.835+0000 7f542f7fe640 1 -- 172.21.15.38:0/1363685784 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5428019070 con 0x7f54380b6980 2024-04-25T08:38:09.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.835+0000 7f543d2b3640 1 -- 172.21.15.38:0/1363685784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54380b5070 con 0x7f54380b6980 2024-04-25T08:38:09.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.835+0000 7f542f7fe640 1 -- 172.21.15.38:0/1363685784 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542800ba80 con 0x7f54380b6980 2024-04-25T08:38:09.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.835+0000 7f543d2b3640 1 -- 172.21.15.38:0/1363685784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54380b5630 con 0x7f54380b6980 2024-04-25T08:38:09.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.836+0000 7f543d2b3640 1 -- 172.21.15.38:0/1363685784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5438161210 con 0x7f54380b6980 2024-04-25T08:38:09.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.836+0000 7f542f7fe640 1 -- 172.21.15.38:0/1363685784 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f542800bc20 con 0x7f54380b6980 2024-04-25T08:38:09.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.837+0000 7f542f7fe640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f541806b500 0x7f541806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:09.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.837+0000 7f542f7fe640 1 -- 172.21.15.38:0/1363685784 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f542808a6c0 con 0x7f54380b6980 2024-04-25T08:38:09.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.837+0000 7f542ffff640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f541806b500 0x7f541806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:09.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.837+0000 7f542ffff640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f541806b500 0x7f541806d9c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5438158f40 tx=0x7f5420009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:09.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:09.841+0000 7f542f7fe640 1 -- 172.21.15.38:0/1363685784 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5438161210 con 0x7f54380b6980 2024-04-25T08:38:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.021+0000 7f543d2b3640 1 -- 172.21.15.38:0/1363685784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5438068e80 con 0x7f54380b6980 2024-04-25T08:38:10.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.021+0000 7f542f7fe640 1 -- 172.21.15.38:0/1363685784 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5428058d40 con 0x7f54380b6980 2024-04-25T08:38:10.015 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:38:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.024+0000 7f543d2b3640 1 -- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f541806b500 msgr2=0x7f541806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.024+0000 7f543d2b3640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f541806b500 0x7f541806d9c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5438158f40 tx=0x7f5420009060 comp rx=0 tx=0).stop 2024-04-25T08:38:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.024+0000 7f543d2b3640 1 -- 172.21.15.38:0/1363685784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380b6980 msgr2=0x7f5438149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.024+0000 7f543d2b3640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380b6980 0x7f5438149040 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f542800d800 tx=0x7f542800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.024+0000 7f543d2b3640 1 -- 172.21.15.38:0/1363685784 shutdown_connections 2024-04-25T08:38:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.024+0000 7f543d2b3640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54380d0700 0x7f54380b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.024+0000 7f543d2b3640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f541806b500 0x7f541806d9c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.024+0000 7f543d2b3640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54380d0320 0x7f5438145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.024+0000 7f543d2b3640 1 --2- 172.21.15.38:0/1363685784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54380b6980 0x7f5438149040 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.024+0000 7f543d2b3640 1 -- 172.21.15.38:0/1363685784 >> 172.21.15.38:0/1363685784 conn(0x7f54380668f0 msgr2=0x7f54380ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.024+0000 7f543d2b3640 1 -- 172.21.15.38:0/1363685784 shutdown_connections 2024-04-25T08:38:10.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.024+0000 7f543d2b3640 1 -- 172.21.15.38:0/1363685784 wait complete. 2024-04-25T08:38:10.028 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:38:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.136+0000 7f1f7249e640 1 Processor -- start 2024-04-25T08:38:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.137+0000 7f1f7249e640 1 -- start start 2024-04-25T08:38:10.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.137+0000 7f1f7249e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f6c06d670 0x7f1f6c0cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.137+0000 7f1f7249e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1f6c0cd7e0 0x7f1f6c099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.137+0000 7f1f7249e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6c0cf7b0 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.137+0000 7f1f7249e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6c0cf560 con 0x7f1f6c0cd7e0 2024-04-25T08:38:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.137+0000 7f1f7249e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0cd070 con 0x7f1f6c06d670 2024-04-25T08:38:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.137+0000 7f1f6b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1f6c0cd7e0 0x7f1f6c099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.137+0000 7f1f6b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1f6c0cd7e0 0x7f1f6c099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48898/0 (socket says 172.21.15.38:48898) 2024-04-25T08:38:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.137+0000 7f1f6b7fe640 1 -- 172.21.15.38:0/1237804832 learned_addr learned my addr 172.21.15.38:0/1237804832 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.138+0000 7f1f6b7fe640 1 -- 172.21.15.38:0/1237804832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f6c06d670 msgr2=0x7f1f6c0cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.138+0000 7f1f6bfff640 1 --2- 172.21.15.38:0/1237804832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f6c06d670 0x7f1f6c0cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.138+0000 7f1f6b7fe640 1 --2- 172.21.15.38:0/1237804832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f6c06d670 0x7f1f6c0cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.138+0000 7f1f6b7fe640 1 -- 172.21.15.38:0/1237804832 >> v1:172.21.15.107:6789/0 conn(0x7f1f6c0b3c70 legacy=0x7f1f6c1619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:10.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.138+0000 7f1f6b7fe640 1 -- 172.21.15.38:0/1237804832 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c0cfa90 con 0x7f1f6c0cd7e0 2024-04-25T08:38:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.138+0000 7f1f6b7fe640 1 --2- 172.21.15.38:0/1237804832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f6c0cd7e0 0x7f1f6c099a70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009870 tx=0x7f1f5c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9921c2fe8a8d01af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.139+0000 7f1f6affd640 1 -- 172.21.15.38:0/1237804832 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c037070 con 0x7f1f6c0cd7e0 2024-04-25T08:38:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.139+0000 7f1f6affd640 1 -- 172.21.15.38:0/1237804832 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c032440 con 0x7f1f6c0cd7e0 2024-04-25T08:38:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.139+0000 7f1f6affd640 1 -- 172.21.15.38:0/1237804832 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c032810 con 0x7f1f6c0cd7e0 2024-04-25T08:38:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.139+0000 7f1f7249e640 1 -- 172.21.15.38:0/1237804832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f6c0cd7e0 msgr2=0x7f1f6c099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.139+0000 7f1f7249e640 1 --2- 172.21.15.38:0/1237804832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f6c0cd7e0 0x7f1f6c099a70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009870 tx=0x7f1f5c02a2a0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.139+0000 7f1f7249e640 1 -- 172.21.15.38:0/1237804832 shutdown_connections 2024-04-25T08:38:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.139+0000 7f1f7249e640 1 --2- 172.21.15.38:0/1237804832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f6c0cd7e0 0x7f1f6c099a70 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.139+0000 7f1f7249e640 1 --2- 172.21.15.38:0/1237804832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f6c06d670 0x7f1f6c0cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.139+0000 7f1f7249e640 1 -- 172.21.15.38:0/1237804832 >> 172.21.15.38:0/1237804832 conn(0x7f1f6c0b3150 msgr2=0x7f1f6c0b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:10.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.139+0000 7f1f7249e640 1 -- 172.21.15.38:0/1237804832 shutdown_connections 2024-04-25T08:38:10.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.139+0000 7f1f7249e640 1 -- 172.21.15.38:0/1237804832 wait complete. 2024-04-25T08:38:10.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.140+0000 7f1f7249e640 1 Processor -- start 2024-04-25T08:38:10.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.141+0000 7f1f7249e640 1 -- start start 2024-04-25T08:38:10.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.141+0000 7f1f7249e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c06d670 0x7f1f6c161fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.141+0000 7f1f7249e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f6c0b3c70 0x7f1f6c0c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.141+0000 7f1f7249e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f6c0cd7e0 0x7f1f6c157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.141+0000 7f1f7249e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c06da50 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.141+0000 7f1f7249e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0d04b0 con 0x7f1f6c06d670 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.141+0000 7f1f7249e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c099fb0 con 0x7f1f6c0cd7e0 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.141+0000 7f1f6b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f6c0b3c70 0x7f1f6c0c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.141+0000 7f1f6b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f6c0b3c70 0x7f1f6c0c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48910/0 (socket says 172.21.15.38:48910) 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.141+0000 7f1f6b7fe640 1 -- 172.21.15.38:0/161189678 learned_addr learned my addr 172.21.15.38:0/161189678 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.142+0000 7f1f6b7fe640 1 -- 172.21.15.38:0/161189678 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f6c0cd7e0 msgr2=0x7f1f6c157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.142+0000 7f1f6b7fe640 1 --2- 172.21.15.38:0/161189678 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f6c0cd7e0 0x7f1f6c157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.142+0000 7f1f6b7fe640 1 -- 172.21.15.38:0/161189678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c06d670 msgr2=0x7f1f6c161fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.142+0000 7f1f6b7fe640 1 --2- 172.21.15.38:0/161189678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c06d670 0x7f1f6c161fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.142+0000 7f1f6b7fe640 1 -- 172.21.15.38:0/161189678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c0c7390 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.142+0000 7f1f6b7fe640 1 --2- 172.21.15.38:0/161189678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f6c0b3c70 0x7f1f6c0c5b80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009870 tx=0x7f1f5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:10.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.142+0000 7f1f68ff9640 1 -- 172.21.15.38:0/161189678 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c041020 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.142+0000 7f1f7249e640 1 -- 172.21.15.38:0/161189678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c0c7610 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.142+0000 7f1f68ff9640 1 -- 172.21.15.38:0/161189678 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c030970 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.142+0000 7f1f68ff9640 1 -- 172.21.15.38:0/161189678 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c037040 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.142+0000 7f1f7249e640 1 -- 172.21.15.38:0/161189678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c0c7bd0 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.143+0000 7f1f7249e640 1 -- 172.21.15.38:0/161189678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c04ef90 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.143+0000 7f1f68ff9640 1 -- 172.21.15.38:0/161189678 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f5c044020 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.144+0000 7f1f68ff9640 1 --2- 172.21.15.38:0/161189678 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f4c06b4d0 0x7f1f4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.144+0000 7f1f68ff9640 1 -- 172.21.15.38:0/161189678 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f1f5c0aada0 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.144+0000 7f1f6bfff640 1 --2- 172.21.15.38:0/161189678 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f4c06b4d0 0x7f1f4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.145+0000 7f1f6bfff640 1 --2- 172.21.15.38:0/161189678 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f4c06b4d0 0x7f1f4c06d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1f54006fd0 tx=0x7f1f54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:10.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.148+0000 7f1f68ff9640 1 -- 172.21.15.38:0/161189678 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f5c04b030 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.340+0000 7f1f7249e640 1 -- 172.21.15.38:0/161189678 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1f6c0b75e0 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.340+0000 7f1f68ff9640 1 -- 172.21.15.38:0/161189678 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v84) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1f6c0c7390 con 0x7f1f6c0b3c70 2024-04-25T08:38:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:38:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.342+0000 7f1f7249e640 1 -- 172.21.15.38:0/161189678 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f4c06b4d0 msgr2=0x7f1f4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.342+0000 7f1f7249e640 1 --2- 172.21.15.38:0/161189678 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f4c06b4d0 0x7f1f4c06d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1f54006fd0 tx=0x7f1f54008040 comp rx=0 tx=0).stop 2024-04-25T08:38:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.342+0000 7f1f7249e640 1 -- 172.21.15.38:0/161189678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f6c0b3c70 msgr2=0x7f1f6c0c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.342+0000 7f1f7249e640 1 --2- 172.21.15.38:0/161189678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f6c0b3c70 0x7f1f6c0c5b80 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009870 tx=0x7f1f5c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.342+0000 7f1f7249e640 1 -- 172.21.15.38:0/161189678 shutdown_connections 2024-04-25T08:38:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.342+0000 7f1f7249e640 1 --2- 172.21.15.38:0/161189678 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f4c06b4d0 0x7f1f4c06d990 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.342+0000 7f1f7249e640 1 --2- 172.21.15.38:0/161189678 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f6c0cd7e0 0x7f1f6c157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.342+0000 7f1f7249e640 1 --2- 172.21.15.38:0/161189678 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f6c0b3c70 0x7f1f6c0c5b80 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.342+0000 7f1f7249e640 1 --2- 172.21.15.38:0/161189678 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f6c06d670 0x7f1f6c161fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.342+0000 7f1f7249e640 1 -- 172.21.15.38:0/161189678 >> 172.21.15.38:0/161189678 conn(0x7f1f6c0b3150 msgr2=0x7f1f6c070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.342+0000 7f1f7249e640 1 -- 172.21.15.38:0/161189678 shutdown_connections 2024-04-25T08:38:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.342+0000 7f1f7249e640 1 -- 172.21.15.38:0/161189678 wait complete. 2024-04-25T08:38:10.346 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:38:10.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:10.347 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:10.370 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[69919]: fuse finished with error 0 and tester_r 0 2024-04-25T08:38:10.374 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:38:10.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:10.375 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:38:10.543 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:38:10.544 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:10.564 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:38:10.564 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:38:10.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:10.564 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:38:10.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:38:10.621 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:38:10.622 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:38:10.622 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:10.637 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:38:10.638 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:38:10.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:10.638 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:38:10.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:38:10.692 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:38:10.693 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:38:10.693 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:10.709 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:38:10.710 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:38:10.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:10.710 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:38:10.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:38:10.766 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:38:10.767 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:10.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.875+0000 7f4d26a9e640 1 Processor -- start 2024-04-25T08:38:10.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.875+0000 7f4d26a9e640 1 -- start start 2024-04-25T08:38:10.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.875+0000 7f4d26a9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d2015ddf0 0x7f4d20161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.875+0000 7f4d26a9e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4d200b93f0 0x7f4d200d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.875+0000 7f4d26a9e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d20161e00 con 0x7f4d200b5710 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.875+0000 7f4d26a9e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d201620e0 con 0x7f4d200b93f0 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.875+0000 7f4d26a9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d201623c0 con 0x7f4d2015ddf0 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d2483a640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4d200b5710 0x7f4d200b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35354/0 (socket says 172.21.15.38:35354) 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d2483a640 1 -- 172.21.15.38:0/999067375 learned_addr learned my addr 172.21.15.38:0/999067375 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d2503b640 1 --2- 172.21.15.38:0/999067375 >> v2:172.21.15.107:3300/0 conn(0x7f4d200b93f0 0x7f4d200d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1ffff640 1 --2- 172.21.15.38:0/999067375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d2015ddf0 0x7f4d20161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475305472 0 0) 0x7f4d20161e00 con 0x7f4d200b5710 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d10003520 con 0x7f4d200b5710 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641792515 0 0) 0x7f4d10003520 con 0x7f4d200b5710 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d2015ddf0 msgr2=0x7f4d20161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 --2- 172.21.15.38:0/999067375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d2015ddf0 0x7f4d20161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 >> v2:172.21.15.107:3300/0 conn(0x7f4d200b93f0 msgr2=0x7f4d200d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 --2- 172.21.15.38:0/999067375 >> v2:172.21.15.107:3300/0 conn(0x7f4d200b93f0 0x7f4d200d2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d200b9870 con 0x7f4d200b5710 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f4d14001670 con 0x7f4d200b5710 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 >> v1:172.21.15.38:6789/0 conn(0x7f4d200b5710 legacy=0x7f4d200b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 --2- 172.21.15.38:0/999067375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d100039b0 0x7f4d10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 --2- 172.21.15.38:0/999067375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d2015ddf0 0x7f4d10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 --2- 172.21.15.38:0/999067375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d10004d40 0x7f4d100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d201620e0 con 0x7f4d100039b0 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d201623c0 con 0x7f4d10004d40 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1000b9c0 con 0x7f4d2015ddf0 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d2483a640 1 --2- 172.21.15.38:0/999067375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d100039b0 0x7f4d10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d1ffff640 1 --2- 172.21.15.38:0/999067375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d2015ddf0 0x7f4d10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.876+0000 7f4d2503b640 1 --2- 172.21.15.38:0/999067375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d10004d40 0x7f4d100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.877+0000 7f4d2483a640 1 -- 172.21.15.38:0/999067375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d2015ddf0 msgr2=0x7f4d10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.877+0000 7f4d2483a640 1 --2- 172.21.15.38:0/999067375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d2015ddf0 0x7f4d10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.877+0000 7f4d2483a640 1 -- 172.21.15.38:0/999067375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d10004d40 msgr2=0x7f4d100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.877+0000 7f4d2483a640 1 --2- 172.21.15.38:0/999067375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d10004d40 0x7f4d100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.877+0000 7f4d2483a640 1 -- 172.21.15.38:0/999067375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d10003520 con 0x7f4d100039b0 2024-04-25T08:38:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.877+0000 7f4d2483a640 1 --2- 172.21.15.38:0/999067375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d100039b0 0x7f4d10003dd0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400c210 tx=0x7f4d10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81ef82071f0e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.878+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14007d50 con 0x7f4d100039b0 2024-04-25T08:38:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.878+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d14005af0 con 0x7f4d100039b0 2024-04-25T08:38:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.878+0000 7f4d1f7fe640 1 -- 172.21.15.38:0/999067375 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14007070 con 0x7f4d100039b0 2024-04-25T08:38:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.878+0000 7f4d26a9e640 1 -- 172.21.15.38:0/999067375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d100039b0 msgr2=0x7f4d10003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.878+0000 7f4d26a9e640 1 --2- 172.21.15.38:0/999067375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d100039b0 0x7f4d10003dd0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f4d1400c210 tx=0x7f4d10005830 comp rx=0 tx=0).stop 2024-04-25T08:38:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.878+0000 7f4d2503b640 1 -- 172.21.15.38:0/999067375 reap_dead start 2024-04-25T08:38:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.878+0000 7f4d26a9e640 1 -- 172.21.15.38:0/999067375 shutdown_connections 2024-04-25T08:38:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.878+0000 7f4d26a9e640 1 -- 172.21.15.38:0/999067375 >> 172.21.15.38:0/999067375 conn(0x7f4d200d07c0 msgr2=0x7f4d200d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:10.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.878+0000 7f4d26a9e640 1 -- 172.21.15.38:0/999067375 shutdown_connections 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.878+0000 7f4d26a9e640 1 -- 172.21.15.38:0/999067375 wait complete. 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.879+0000 7f4d26a9e640 1 Processor -- start 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.879+0000 7f4d26a9e640 1 -- start start 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.880+0000 7f4d26a9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d100039b0 0x7f4d20148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.880+0000 7f4d26a9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d2015f250 0x7f4d201f4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.880+0000 7f4d26a9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d201f46c0 0x7f4d201575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.880+0000 7f4d26a9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d20161b10 con 0x7f4d201f46c0 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.880+0000 7f4d26a9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d20161990 con 0x7f4d2015f250 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.880+0000 7f4d26a9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d200d2880 con 0x7f4d100039b0 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.880+0000 7f4d2503b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d201f46c0 0x7f4d201575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.880+0000 7f4d2503b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d201f46c0 0x7f4d201575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48922/0 (socket says 172.21.15.38:48922) 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.880+0000 7f4d2503b640 1 -- 172.21.15.38:0/580272511 learned_addr learned my addr 172.21.15.38:0/580272511 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.880+0000 7f4d2483a640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d100039b0 0x7f4d20148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.880+0000 7f4d1ffff640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d2015f250 0x7f4d201f4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.881+0000 7f4d2503b640 1 -- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d100039b0 msgr2=0x7f4d20148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.881+0000 7f4d2503b640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d100039b0 0x7f4d20148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.881+0000 7f4d2503b640 1 -- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d2015f250 msgr2=0x7f4d201f4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.881+0000 7f4d2503b640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d2015f250 0x7f4d201f4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.881+0000 7f4d2503b640 1 -- 172.21.15.38:0/580272511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d20149580 con 0x7f4d201f46c0 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.881+0000 7f4d1ffff640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d2015f250 0x7f4d201f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.881+0000 7f4d2503b640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d201f46c0 0x7f4d201575a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4d200b6820 tx=0x7f4d1800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.881+0000 7f4d1d7fa640 1 -- 172.21.15.38:0/580272511 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1800fe10 con 0x7f4d201f46c0 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.881+0000 7f4d1d7fa640 1 -- 172.21.15.38:0/580272511 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d18015430 con 0x7f4d201f46c0 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.881+0000 7f4d1d7fa640 1 -- 172.21.15.38:0/580272511 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d18015820 con 0x7f4d201f46c0 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.881+0000 7f4d26a9e640 1 -- 172.21.15.38:0/580272511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d20149860 con 0x7f4d201f46c0 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.881+0000 7f4d26a9e640 1 -- 172.21.15.38:0/580272511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d201597c0 con 0x7f4d201f46c0 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.882+0000 7f4d26a9e640 1 -- 172.21.15.38:0/580272511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d20068e80 con 0x7f4d201f46c0 2024-04-25T08:38:10.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.883+0000 7f4d1d7fa640 1 -- 172.21.15.38:0/580272511 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d180159c0 con 0x7f4d201f46c0 2024-04-25T08:38:10.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.883+0000 7f4d1d7fa640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d0406b4d0 0x7f4d0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:10.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.883+0000 7f4d1d7fa640 1 -- 172.21.15.38:0/580272511 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f4d1808b100 con 0x7f4d201f46c0 2024-04-25T08:38:10.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.883+0000 7f4d2483a640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d0406b4d0 0x7f4d0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:10.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.884+0000 7f4d2483a640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d0406b4d0 0x7f4d0406d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4d14033000 tx=0x7f4d14009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:10.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:10.887+0000 7f4d1d7fa640 1 -- 172.21.15.38:0/580272511 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d18019030 con 0x7f4d201f46c0 2024-04-25T08:38:11.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.107+0000 7f4d26a9e640 1 -- 172.21.15.38:0/580272511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d200b8600 con 0x7f4d201f46c0 2024-04-25T08:38:11.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.107+0000 7f4d1d7fa640 1 -- 172.21.15.38:0/580272511 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f4d18010040 con 0x7f4d201f46c0 2024-04-25T08:38:11.101 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:11.101 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3781069672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3781069672},{"type":"v1","addr":"172.21.15.107:6837","nonce":3781069672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/624970524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":624970524},{"type":"v1","addr":"172.21.15.107:6836","nonce":624970524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7380,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1256978001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1256978001},{"type":"v1","addr":"172.21.15.38:6834","nonce":1256978001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:36:26.904065+0000","modified":"2024-04-25T08:36:39.455074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/380751658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":380751658},{"type":"v1","addr":"172.21.15.38:6835","nonce":380751658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:38:11.102 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 51 2024-04-25T08:38:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.109+0000 7f4d26a9e640 1 -- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d0406b4d0 msgr2=0x7f4d0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.109+0000 7f4d26a9e640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d0406b4d0 0x7f4d0406d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4d14033000 tx=0x7f4d14009030 comp rx=0 tx=0).stop 2024-04-25T08:38:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.109+0000 7f4d26a9e640 1 -- 172.21.15.38:0/580272511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d201f46c0 msgr2=0x7f4d201575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.109+0000 7f4d26a9e640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d201f46c0 0x7f4d201575a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f4d200b6820 tx=0x7f4d1800f580 comp rx=0 tx=0).stop 2024-04-25T08:38:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.110+0000 7f4d26a9e640 1 -- 172.21.15.38:0/580272511 shutdown_connections 2024-04-25T08:38:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.110+0000 7f4d26a9e640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4d201f46c0 0x7f4d201575a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.110+0000 7f4d26a9e640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d2015f250 0x7f4d201f4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.110+0000 7f4d26a9e640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d0406b4d0 0x7f4d0406d990 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.110+0000 7f4d26a9e640 1 --2- 172.21.15.38:0/580272511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4d100039b0 0x7f4d20148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.110+0000 7f4d26a9e640 1 -- 172.21.15.38:0/580272511 >> 172.21.15.38:0/580272511 conn(0x7f4d200d07c0 msgr2=0x7f4d200b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.110+0000 7f4d26a9e640 1 -- 172.21.15.38:0/580272511 shutdown_connections 2024-04-25T08:38:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.110+0000 7f4d26a9e640 1 -- 172.21.15.38:0/580272511 wait complete. 2024-04-25T08:38:11.115 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:11.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.223+0000 7fb165c6b640 1 Processor -- start 2024-04-25T08:38:11.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.223+0000 7fb165c6b640 1 -- start start 2024-04-25T08:38:11.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb165c6b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1600d0fe0 0x7fb1600b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb165c6b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb1600d2e50 0x7fb1600b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb165c6b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1600b8400 con 0x7fb16015dc40 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb165c6b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1600d19b0 con 0x7fb1600d2e50 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb165c6b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1600b86e0 con 0x7fb1600d0fe0 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb15f7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb16015dc40 0x7fb1600d08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35366/0 (socket says 172.21.15.38:35366) 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb15f7fe640 1 -- 172.21.15.38:0/3791421357 learned_addr learned my addr 172.21.15.38:0/3791421357 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb15effd640 1 --2- 172.21.15.38:0/3791421357 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1600d0fe0 0x7fb1600b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb15ffff640 1 --2- 172.21.15.38:0/3791421357 >> v2:172.21.15.107:3300/0 conn(0x7fb1600d2e50 0x7fb1600b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb15e7fc640 1 -- 172.21.15.38:0/3791421357 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016835247 0 0) 0x7fb1600b8400 con 0x7fb16015dc40 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb15e7fc640 1 -- 172.21.15.38:0/3791421357 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb148003520 con 0x7fb16015dc40 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb15effd640 1 -- 172.21.15.38:0/3791421357 >> v2:172.21.15.107:3300/0 conn(0x7fb1600d2e50 msgr2=0x7fb1600b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb15effd640 1 --2- 172.21.15.38:0/3791421357 >> v2:172.21.15.107:3300/0 conn(0x7fb1600d2e50 0x7fb1600b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb15effd640 1 -- 172.21.15.38:0/3791421357 >> v1:172.21.15.38:6789/0 conn(0x7fb16015dc40 legacy=0x7fb1600d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.224+0000 7fb15effd640 1 -- 172.21.15.38:0/3791421357 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1600b89c0 con 0x7fb1600d0fe0 2024-04-25T08:38:11.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.225+0000 7fb15effd640 1 --2- 172.21.15.38:0/3791421357 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1600d0fe0 0x7fb1600b6a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb1500097a0 tx=0x7fb15002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=248b6bfbe88f91fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:11.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.225+0000 7fb15e7fc640 1 -- 172.21.15.38:0/3791421357 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150037070 con 0x7fb1600d0fe0 2024-04-25T08:38:11.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.225+0000 7fb15e7fc640 1 -- 172.21.15.38:0/3791421357 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15002ae50 con 0x7fb1600d0fe0 2024-04-25T08:38:11.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.225+0000 7fb15e7fc640 1 -- 172.21.15.38:0/3791421357 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150032640 con 0x7fb1600d0fe0 2024-04-25T08:38:11.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.226+0000 7fb165c6b640 1 -- 172.21.15.38:0/3791421357 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1600d0fe0 msgr2=0x7fb1600b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.226+0000 7fb165c6b640 1 --2- 172.21.15.38:0/3791421357 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1600d0fe0 0x7fb1600b6a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb1500097a0 tx=0x7fb15002a100 comp rx=0 tx=0).stop 2024-04-25T08:38:11.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.226+0000 7fb165c6b640 1 -- 172.21.15.38:0/3791421357 shutdown_connections 2024-04-25T08:38:11.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.226+0000 7fb165c6b640 1 --2- 172.21.15.38:0/3791421357 >> v2:172.21.15.107:3300/0 conn(0x7fb1600d2e50 0x7fb1600b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.226+0000 7fb165c6b640 1 --2- 172.21.15.38:0/3791421357 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1600d0fe0 0x7fb1600b6a60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.226+0000 7fb165c6b640 1 -- 172.21.15.38:0/3791421357 >> 172.21.15.38:0/3791421357 conn(0x7fb160066ce0 msgr2=0x7fb1600670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:11.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.226+0000 7fb165c6b640 1 -- 172.21.15.38:0/3791421357 shutdown_connections 2024-04-25T08:38:11.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.226+0000 7fb165c6b640 1 -- 172.21.15.38:0/3791421357 wait complete. 2024-04-25T08:38:11.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.227+0000 7fb165c6b640 1 Processor -- start 2024-04-25T08:38:11.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.227+0000 7fb165c6b640 1 -- start start 2024-04-25T08:38:11.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb165c6b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1600d0fe0 0x7fb1600ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb165c6b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1600d2e50 0x7fb1600c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb165c6b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb16015dc40 0x7fb1600c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb165c6b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1600b9700 con 0x7fb1600d0fe0 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb165c6b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1600b9580 con 0x7fb16015dc40 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb165c6b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1600b9880 con 0x7fb1600d2e50 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb15f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1600d0fe0 0x7fb1600ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb15f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1600d0fe0 0x7fb1600ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48926/0 (socket says 172.21.15.38:48926) 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb15f7fe640 1 -- 172.21.15.38:0/1549316155 learned_addr learned my addr 172.21.15.38:0/1549316155 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb15effd640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1600d2e50 0x7fb1600c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb15ffff640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb16015dc40 0x7fb1600c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb15f7fe640 1 -- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1600d2e50 msgr2=0x7fb1600c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb15f7fe640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1600d2e50 0x7fb1600c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb15f7fe640 1 -- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb16015dc40 msgr2=0x7fb1600c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb15f7fe640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb16015dc40 0x7fb1600c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb15f7fe640 1 -- 172.21.15.38:0/1549316155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1600c7040 con 0x7fb1600d0fe0 2024-04-25T08:38:11.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb15effd640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1600d2e50 0x7fb1600c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:11.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.228+0000 7fb15f7fe640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1600d0fe0 0x7fb1600ccfb0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c390 tx=0x7fb15400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:11.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.229+0000 7fb147fff640 1 -- 172.21.15.38:0/1549316155 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb154017070 con 0x7fb1600d0fe0 2024-04-25T08:38:11.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.229+0000 7fb147fff640 1 -- 172.21.15.38:0/1549316155 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb154005ce0 con 0x7fb1600d0fe0 2024-04-25T08:38:11.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.229+0000 7fb147fff640 1 -- 172.21.15.38:0/1549316155 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1540072f0 con 0x7fb1600d0fe0 2024-04-25T08:38:11.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.229+0000 7fb165c6b640 1 -- 172.21.15.38:0/1549316155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1600cbf10 con 0x7fb1600d0fe0 2024-04-25T08:38:11.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.229+0000 7fb165c6b640 1 -- 172.21.15.38:0/1549316155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1600cc4d0 con 0x7fb1600d0fe0 2024-04-25T08:38:11.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.230+0000 7fb165c6b640 1 -- 172.21.15.38:0/1549316155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb160068e80 con 0x7fb1600d0fe0 2024-04-25T08:38:11.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.230+0000 7fb147fff640 1 -- 172.21.15.38:0/1549316155 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb15401a070 con 0x7fb1600d0fe0 2024-04-25T08:38:11.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.231+0000 7fb147fff640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb14c06b500 0x7fb14c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.231+0000 7fb147fff640 1 -- 172.21.15.38:0/1549316155 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fb1540896f0 con 0x7fb1600d0fe0 2024-04-25T08:38:11.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.231+0000 7fb15effd640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb14c06b500 0x7fb14c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.231+0000 7fb15effd640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb14c06b500 0x7fb14c06d9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb150034000 tx=0x7fb15002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:11.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.235+0000 7fb147fff640 1 -- 172.21.15.38:0/1549316155 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb154058450 con 0x7fb1600d0fe0 2024-04-25T08:38:11.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.417+0000 7fb165c6b640 1 -- 172.21.15.38:0/1549316155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb16015d100 con 0x7fb1600d0fe0 2024-04-25T08:38:11.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.418+0000 7fb147fff640 1 -- 172.21.15.38:0/1549316155 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16999 (secure 0 0 0) 0x7fb154057d50 con 0x7fb1600d0fe0 2024-04-25T08:38:11.412 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:11.413 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":84,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:37:50.914128+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:25.176504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:25.877954+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":83,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.420+0000 7fb165c6b640 1 -- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb14c06b500 msgr2=0x7fb14c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.420+0000 7fb165c6b640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb14c06b500 0x7fb14c06d9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb150034000 tx=0x7fb15002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.420+0000 7fb165c6b640 1 -- 172.21.15.38:0/1549316155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1600d0fe0 msgr2=0x7fb1600ccfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.420+0000 7fb165c6b640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1600d0fe0 0x7fb1600ccfb0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c390 tx=0x7fb15400c850 comp rx=0 tx=0).stop 2024-04-25T08:38:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.421+0000 7fb165c6b640 1 -- 172.21.15.38:0/1549316155 shutdown_connections 2024-04-25T08:38:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.421+0000 7fb165c6b640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb16015dc40 0x7fb1600c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.421+0000 7fb165c6b640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb14c06b500 0x7fb14c06d9c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.421+0000 7fb165c6b640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1600d2e50 0x7fb1600c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.421+0000 7fb165c6b640 1 --2- 172.21.15.38:0/1549316155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1600d0fe0 0x7fb1600ccfb0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.421+0000 7fb165c6b640 1 -- 172.21.15.38:0/1549316155 >> 172.21.15.38:0/1549316155 conn(0x7fb160066ce0 msgr2=0x7fb1600d3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.421+0000 7fb165c6b640 1 -- 172.21.15.38:0/1549316155 shutdown_connections 2024-04-25T08:38:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.421+0000 7fb165c6b640 1 -- 172.21.15.38:0/1549316155 wait complete. 2024-04-25T08:38:11.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:36:25.176504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:36:25.877954+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 83, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:38:11.427 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:38:11.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.537+0000 7fd0c321f640 1 Processor -- start 2024-04-25T08:38:11.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.537+0000 7fd0c321f640 1 -- start start 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.538+0000 7fd0c321f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d33d0 0x7fd0bc0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.538+0000 7fd0c321f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd0bc0d1e90 0x7fd0bc0b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.538+0000 7fd0c321f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0bc0d2330 con 0x7fd0bc0d0320 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.538+0000 7fd0c321f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0bc0d2610 con 0x7fd0bc0d1e90 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.538+0000 7fd0c321f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0d28f0 con 0x7fd0bc0d33d0 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.538+0000 7fd0c0fbb640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd0bc0d0320 0x7fd0bc0d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35374/0 (socket says 172.21.15.38:35374) 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.538+0000 7fd0c0fbb640 1 -- 172.21.15.38:0/474810510 learned_addr learned my addr 172.21.15.38:0/474810510 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0c17bc640 1 --2- 172.21.15.38:0/474810510 >> v2:172.21.15.107:3300/0 conn(0x7fd0bc0d1e90 0x7fd0bc0b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bbfff640 1 --2- 172.21.15.38:0/474810510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d33d0 0x7fd0bc0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179116929 0 0) 0x7fd0bc0d2330 con 0x7fd0bc0d0320 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0a4003520 con 0x7fd0bc0d0320 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1974859848 0 0) 0x7fd0a4003520 con 0x7fd0bc0d0320 2024-04-25T08:38:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d33d0 msgr2=0x7fd0bc0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 --2- 172.21.15.38:0/474810510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d33d0 0x7fd0bc0d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 >> v2:172.21.15.107:3300/0 conn(0x7fd0bc0d1e90 msgr2=0x7fd0bc0b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 --2- 172.21.15.38:0/474810510 >> v2:172.21.15.107:3300/0 conn(0x7fd0bc0d1e90 0x7fd0bc0b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc0d2bd0 con 0x7fd0bc0d0320 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fd0ac001670 con 0x7fd0bc0d0320 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 >> v1:172.21.15.38:6789/0 conn(0x7fd0bc0d0320 legacy=0x7fd0bc0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 --2- 172.21.15.38:0/474810510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d33d0 0x7fd0a40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 --2- 172.21.15.38:0/474810510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a4003f30 0x7fd0a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 --2- 172.21.15.38:0/474810510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a4004d40 0x7fd0a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0d2610 con 0x7fd0a4004d40 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0d28f0 con 0x7fd0a4003f30 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a400b9f0 con 0x7fd0bc0d33d0 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0c17bc640 1 --2- 172.21.15.38:0/474810510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a4004d40 0x7fd0a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0c0fbb640 1 --2- 172.21.15.38:0/474810510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d33d0 0x7fd0a40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0bbfff640 1 --2- 172.21.15.38:0/474810510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a4003f30 0x7fd0a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0c17bc640 1 -- 172.21.15.38:0/474810510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d33d0 msgr2=0x7fd0a40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0c17bc640 1 --2- 172.21.15.38:0/474810510 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc0d33d0 0x7fd0a40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0c17bc640 1 -- 172.21.15.38:0/474810510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a4003f30 msgr2=0x7fd0a4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0c17bc640 1 --2- 172.21.15.38:0/474810510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0a4003f30 0x7fd0a4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.539+0000 7fd0c17bc640 1 -- 172.21.15.38:0/474810510 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a4003520 con 0x7fd0a4004d40 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.540+0000 7fd0c17bc640 1 --2- 172.21.15.38:0/474810510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a4004d40 0x7fd0a40051c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009f90 tx=0x7fd0a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbde166873acc19a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.540+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4009c10 con 0x7fd0a4004d40 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.540+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4036070 con 0x7fd0a4004d40 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.540+0000 7fd0bb7fe640 1 -- 172.21.15.38:0/474810510 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4031840 con 0x7fd0a4004d40 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.541+0000 7fd0c321f640 1 -- 172.21.15.38:0/474810510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a4004d40 msgr2=0x7fd0a40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.541+0000 7fd0c321f640 1 --2- 172.21.15.38:0/474810510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0a4004d40 0x7fd0a40051c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009f90 tx=0x7fd0a4005830 comp rx=0 tx=0).stop 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.541+0000 7fd0c17bc640 1 -- 172.21.15.38:0/474810510 reap_dead start 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.541+0000 7fd0c321f640 1 -- 172.21.15.38:0/474810510 shutdown_connections 2024-04-25T08:38:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.541+0000 7fd0c321f640 1 -- 172.21.15.38:0/474810510 >> 172.21.15.38:0/474810510 conn(0x7fd0bc0668f0 msgr2=0x7fd0bc066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.541+0000 7fd0c321f640 1 -- 172.21.15.38:0/474810510 shutdown_connections 2024-04-25T08:38:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.541+0000 7fd0c321f640 1 -- 172.21.15.38:0/474810510 wait complete. 2024-04-25T08:38:11.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.542+0000 7fd0c321f640 1 Processor -- start 2024-04-25T08:38:11.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.542+0000 7fd0c321f640 1 -- start start 2024-04-25T08:38:11.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.543+0000 7fd0c321f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc152020 0x7fd0bc0b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.543+0000 7fd0c321f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc152400 0x7fd0bc0aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.543+0000 7fd0c321f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0bc15ff80 0x7fd0bc160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.543+0000 7fd0c321f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc161390 con 0x7fd0bc15ff80 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.543+0000 7fd0c321f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc161210 con 0x7fd0bc152020 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.543+0000 7fd0c321f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0bc0b9720 con 0x7fd0bc152400 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.543+0000 7fd0c17bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0bc15ff80 0x7fd0bc160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.543+0000 7fd0c17bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0bc15ff80 0x7fd0bc160430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48950/0 (socket says 172.21.15.38:48950) 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.543+0000 7fd0c17bc640 1 -- 172.21.15.38:0/3890517827 learned_addr learned my addr 172.21.15.38:0/3890517827 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.543+0000 7fd0c0fbb640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc152020 0x7fd0bc0b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.543+0000 7fd0bbfff640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc152400 0x7fd0bc0aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.543+0000 7fd0c17bc640 1 -- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc152400 msgr2=0x7fd0bc0aacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.544+0000 7fd0c17bc640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc152400 0x7fd0bc0aacb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.544+0000 7fd0c17bc640 1 -- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc152020 msgr2=0x7fd0bc0b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.544+0000 7fd0c17bc640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc152020 0x7fd0bc0b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.544+0000 7fd0c17bc640 1 -- 172.21.15.38:0/3890517827 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc0ab420 con 0x7fd0bc15ff80 2024-04-25T08:38:11.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.544+0000 7fd0bbfff640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc152400 0x7fd0bc0aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.544+0000 7fd0c17bc640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0bc15ff80 0x7fd0bc160430 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4033040 tx=0x7fd0b4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.544+0000 7fd0b97fa640 1 -- 172.21.15.38:0/3890517827 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4009c10 con 0x7fd0bc15ff80 2024-04-25T08:38:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.544+0000 7fd0b97fa640 1 -- 172.21.15.38:0/3890517827 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4031dd0 con 0x7fd0bc15ff80 2024-04-25T08:38:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.544+0000 7fd0c321f640 1 -- 172.21.15.38:0/3890517827 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0bc0ab700 con 0x7fd0bc15ff80 2024-04-25T08:38:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.544+0000 7fd0b97fa640 1 -- 172.21.15.38:0/3890517827 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4045670 con 0x7fd0bc15ff80 2024-04-25T08:38:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.544+0000 7fd0c321f640 1 -- 172.21.15.38:0/3890517827 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0bc0abcc0 con 0x7fd0bc15ff80 2024-04-25T08:38:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.545+0000 7fd0c321f640 1 -- 172.21.15.38:0/3890517827 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0bc068e80 con 0x7fd0bc15ff80 2024-04-25T08:38:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.545+0000 7fd0b97fa640 1 -- 172.21.15.38:0/3890517827 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0b4005d10 con 0x7fd0bc15ff80 2024-04-25T08:38:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.546+0000 7fd0b97fa640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd0a806b500 0x7fd0a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.546+0000 7fd0b97fa640 1 -- 172.21.15.38:0/3890517827 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fd0b40ab3d0 con 0x7fd0bc15ff80 2024-04-25T08:38:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.546+0000 7fd0c0fbb640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd0a806b500 0x7fd0a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.547+0000 7fd0c0fbb640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd0a806b500 0x7fd0a806d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac004840 tx=0x7fd0ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:11.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.550+0000 7fd0b97fa640 1 -- 172.21.15.38:0/3890517827 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0b407a250 con 0x7fd0bc15ff80 2024-04-25T08:38:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.733+0000 7fd0c321f640 1 -- 172.21.15.38:0/3890517827 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0bc15dbd0 con 0x7fd0bc15ff80 2024-04-25T08:38:11.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.733+0000 7fd0b97fa640 1 -- 172.21.15.38:0/3890517827 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd0b4079b50 con 0x7fd0bc15ff80 2024-04-25T08:38:11.727 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:38:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.735+0000 7fd0c321f640 1 -- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd0a806b500 msgr2=0x7fd0a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.735+0000 7fd0c321f640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd0a806b500 0x7fd0a806d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd0ac004840 tx=0x7fd0ac009040 comp rx=0 tx=0).stop 2024-04-25T08:38:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.735+0000 7fd0c321f640 1 -- 172.21.15.38:0/3890517827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0bc15ff80 msgr2=0x7fd0bc160430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.735+0000 7fd0c321f640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0bc15ff80 0x7fd0bc160430 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4033040 tx=0x7fd0b4009750 comp rx=0 tx=0).stop 2024-04-25T08:38:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.735+0000 7fd0c321f640 1 -- 172.21.15.38:0/3890517827 shutdown_connections 2024-04-25T08:38:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.735+0000 7fd0c321f640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0bc15ff80 0x7fd0bc160430 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.735+0000 7fd0c321f640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd0a806b500 0x7fd0a806d9c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.735+0000 7fd0c321f640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0bc152400 0x7fd0bc0aacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.735+0000 7fd0c321f640 1 --2- 172.21.15.38:0/3890517827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0bc152020 0x7fd0bc0b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.735+0000 7fd0c321f640 1 -- 172.21.15.38:0/3890517827 >> 172.21.15.38:0/3890517827 conn(0x7fd0bc0668f0 msgr2=0x7fd0bc073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.735+0000 7fd0c321f640 1 -- 172.21.15.38:0/3890517827 shutdown_connections 2024-04-25T08:38:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.735+0000 7fd0c321f640 1 -- 172.21.15.38:0/3890517827 wait complete. 2024-04-25T08:38:11.740 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:38:11.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.850+0000 7fea18c12640 1 Processor -- start 2024-04-25T08:38:11.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.850+0000 7fea18c12640 1 -- start start 2024-04-25T08:38:11.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.851+0000 7fea18c12640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fea140d33d0 0x7fea140d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.851+0000 7fea18c12640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea140d2330 con 0x7fea140d0320 2024-04-25T08:38:11.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.851+0000 7fea18c12640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea140d2610 con 0x7fea140d1e90 2024-04-25T08:38:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.851+0000 7fea18c12640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea140d28f0 con 0x7fea140d33d0 2024-04-25T08:38:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.851+0000 7fea11d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fea140d33d0 0x7fea140d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.851+0000 7fea11d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fea140d33d0 0x7fea140d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48960/0 (socket says 172.21.15.38:48960) 2024-04-25T08:38:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.851+0000 7fea11d9b640 1 -- 172.21.15.38:0/2425499359 learned_addr learned my addr 172.21.15.38:0/2425499359 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.851+0000 7fea1259c640 1 --1- 172.21.15.38:0/2425499359 >> v1:172.21.15.38:6789/0 conn(0x7fea140d0320 0x7fea140d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35376/0 (socket says 172.21.15.38:35376) 2024-04-25T08:38:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.852+0000 7fea11d9b640 1 -- 172.21.15.38:0/2425499359 >> v1:172.21.15.107:6789/0 conn(0x7fea140d1e90 legacy=0x7fea140b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.852+0000 7fea11d9b640 1 -- 172.21.15.38:0/2425499359 >> v1:172.21.15.38:6789/0 conn(0x7fea140d0320 legacy=0x7fea140d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.852+0000 7fea11d9b640 1 -- 172.21.15.38:0/2425499359 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea140d2bd0 con 0x7fea140d33d0 2024-04-25T08:38:11.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.852+0000 7fea1159a640 1 -- 172.21.15.38:0/2425499359 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331602609 0 0) 0x7fea140d2330 con 0x7fea140d0320 2024-04-25T08:38:11.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.852+0000 7fea11d9b640 1 --2- 172.21.15.38:0/2425499359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea140d33d0 0x7fea140d1950 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fea08009870 tx=0x7fea0802a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44c041b6416bd1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:11.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.852+0000 7fea1159a640 1 -- 172.21.15.38:0/2425499359 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08037070 con 0x7fea140d33d0 2024-04-25T08:38:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.853+0000 7fea1159a640 1 -- 172.21.15.38:0/2425499359 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea08032640 con 0x7fea140d33d0 2024-04-25T08:38:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.853+0000 7fea1159a640 1 -- 172.21.15.38:0/2425499359 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08032a10 con 0x7fea140d33d0 2024-04-25T08:38:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.853+0000 7fea18c12640 1 -- 172.21.15.38:0/2425499359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea140d33d0 msgr2=0x7fea140d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.853+0000 7fea18c12640 1 --2- 172.21.15.38:0/2425499359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea140d33d0 0x7fea140d1950 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fea08009870 tx=0x7fea0802a330 comp rx=0 tx=0).stop 2024-04-25T08:38:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.853+0000 7fea18c12640 1 -- 172.21.15.38:0/2425499359 shutdown_connections 2024-04-25T08:38:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.853+0000 7fea18c12640 1 --2- 172.21.15.38:0/2425499359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea140d33d0 0x7fea140d1950 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.853+0000 7fea18c12640 1 -- 172.21.15.38:0/2425499359 >> 172.21.15.38:0/2425499359 conn(0x7fea140668f0 msgr2=0x7fea14066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.853+0000 7fea18c12640 1 -- 172.21.15.38:0/2425499359 shutdown_connections 2024-04-25T08:38:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.853+0000 7fea18c12640 1 -- 172.21.15.38:0/2425499359 wait complete. 2024-04-25T08:38:11.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.854+0000 7fea18c12640 1 Processor -- start 2024-04-25T08:38:11.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.854+0000 7fea18c12640 1 -- start start 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.855+0000 7fea18c12640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea140d0320 0x7fea14155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.855+0000 7fea18c12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea140d1e90 0x7fea140b5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.855+0000 7fea18c12640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea140d33d0 0x7fea140cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.855+0000 7fea18c12640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea14161390 con 0x7fea140d1e90 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.855+0000 7fea18c12640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea14161210 con 0x7fea140d0320 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.855+0000 7fea18c12640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea140b9720 con 0x7fea140d33d0 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.855+0000 7fea11d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea140d1e90 0x7fea140b5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.855+0000 7fea11d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea140d1e90 0x7fea140b5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48968/0 (socket says 172.21.15.38:48968) 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.855+0000 7fea11d9b640 1 -- 172.21.15.38:0/278809793 learned_addr learned my addr 172.21.15.38:0/278809793 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea12d9d640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea140d33d0 0x7fea140cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea11d9b640 1 -- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea140d33d0 msgr2=0x7fea140cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea1259c640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea140d0320 0x7fea14155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea11d9b640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea140d33d0 0x7fea140cc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea11d9b640 1 -- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea140d0320 msgr2=0x7fea14155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea11d9b640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea140d0320 0x7fea14155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea11d9b640 1 -- 172.21.15.38:0/278809793 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea140b5770 con 0x7fea140d1e90 2024-04-25T08:38:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea11d9b640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea140d1e90 0x7fea140b5230 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fea080090b0 tx=0x7fea080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea02ffd640 1 -- 172.21.15.38:0/278809793 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08041070 con 0x7fea140d1e90 2024-04-25T08:38:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea02ffd640 1 -- 172.21.15.38:0/278809793 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0803c890 con 0x7fea140d1e90 2024-04-25T08:38:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea18c12640 1 -- 172.21.15.38:0/278809793 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea141f3570 con 0x7fea140d1e90 2024-04-25T08:38:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea02ffd640 1 -- 172.21.15.38:0/278809793 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08037040 con 0x7fea140d1e90 2024-04-25T08:38:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.856+0000 7fea18c12640 1 -- 172.21.15.38:0/278809793 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea141f3b30 con 0x7fea140d1e90 2024-04-25T08:38:11.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.857+0000 7fea18c12640 1 -- 172.21.15.38:0/278809793 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea14068e80 con 0x7fea140d1e90 2024-04-25T08:38:11.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.857+0000 7fea02ffd640 1 -- 172.21.15.38:0/278809793 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea08044070 con 0x7fea140d1e90 2024-04-25T08:38:11.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.858+0000 7fea02ffd640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9f406b4d0 0x7fe9f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:11.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.858+0000 7fea02ffd640 1 -- 172.21.15.38:0/278809793 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fea080ab1b0 con 0x7fea140d1e90 2024-04-25T08:38:11.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.858+0000 7fea1259c640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9f406b4d0 0x7fe9f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:11.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.859+0000 7fea1259c640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9f406b4d0 0x7fe9f406d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fea0c007fd0 tx=0x7fea0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:11.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:11.862+0000 7fea02ffd640 1 -- 172.21.15.38:0/278809793 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea08079f10 con 0x7fea140d1e90 2024-04-25T08:38:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.047+0000 7fea18c12640 1 -- 172.21.15.38:0/278809793 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea14067ff0 con 0x7fea140d1e90 2024-04-25T08:38:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.048+0000 7fea02ffd640 1 -- 172.21.15.38:0/278809793 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea08079810 con 0x7fea140d1e90 2024-04-25T08:38:12.042 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:38:12.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.050+0000 7fea18c12640 1 -- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9f406b4d0 msgr2=0x7fe9f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.050+0000 7fea18c12640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9f406b4d0 0x7fe9f406d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fea0c007fd0 tx=0x7fea0c009040 comp rx=0 tx=0).stop 2024-04-25T08:38:12.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.050+0000 7fea18c12640 1 -- 172.21.15.38:0/278809793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea140d1e90 msgr2=0x7fea140b5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.050+0000 7fea18c12640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea140d1e90 0x7fea140b5230 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fea080090b0 tx=0x7fea080046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.050+0000 7fea18c12640 1 -- 172.21.15.38:0/278809793 shutdown_connections 2024-04-25T08:38:12.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.050+0000 7fea18c12640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9f406b4d0 0x7fe9f406d990 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.050+0000 7fea18c12640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea140d33d0 0x7fea140cc9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.050+0000 7fea18c12640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea140d1e90 0x7fea140b5230 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.050+0000 7fea18c12640 1 --2- 172.21.15.38:0/278809793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea140d0320 0x7fea14155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.050+0000 7fea18c12640 1 -- 172.21.15.38:0/278809793 >> 172.21.15.38:0/278809793 conn(0x7fea140668f0 msgr2=0x7fea140b8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:12.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.050+0000 7fea18c12640 1 -- 172.21.15.38:0/278809793 shutdown_connections 2024-04-25T08:38:12.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.050+0000 7fea18c12640 1 -- 172.21.15.38:0/278809793 wait complete. 2024-04-25T08:38:12.055 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:38:12.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.165+0000 7fe7283f9640 1 Processor -- start 2024-04-25T08:38:12.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.165+0000 7fe7283f9640 1 -- start start 2024-04-25T08:38:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.166+0000 7fe7283f9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200d0320 0x7fe7200d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.166+0000 7fe7283f9640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7200d2060 con 0x7fe7200d0700 2024-04-25T08:38:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.166+0000 7fe7283f9640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7200d2330 con 0x7fe7200b6980 2024-04-25T08:38:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.166+0000 7fe7283f9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200d2610 con 0x7fe7200d0320 2024-04-25T08:38:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.166+0000 7fe725994640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe7200d0700 0x7fe7200d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35392/0 (socket says 172.21.15.38:35392) 2024-04-25T08:38:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.166+0000 7fe725994640 1 -- 172.21.15.38:0/2549472567 learned_addr learned my addr 172.21.15.38:0/2549472567 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.166+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740907556 0 0) 0x7fe7200d2060 con 0x7fe7200d0700 2024-04-25T08:38:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.166+0000 7fe726195640 1 --2- 172.21.15.38:0/2549472567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200d0320 0x7fe7200d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.166+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe70c003520 con 0x7fe7200d0700 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2609023006 0 0) 0x7fe70c003520 con 0x7fe7200d0700 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200d0320 msgr2=0x7fe7200d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 --2- 172.21.15.38:0/2549472567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200d0320 0x7fe7200d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 >> v1:172.21.15.107:6789/0 conn(0x7fe7200b6980 legacy=0x7fe7200b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7200d28f0 con 0x7fe7200d0700 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fe71c001670 con 0x7fe7200d0700 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 >> v1:172.21.15.38:6789/0 conn(0x7fe7200d0700 legacy=0x7fe7200d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 --2- 172.21.15.38:0/2549472567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe70c0039e0 0x7fe70c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 --2- 172.21.15.38:0/2549472567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200d0320 0x7fe70c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 --2- 172.21.15.38:0/2549472567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe70c004d40 0x7fe70c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200d2610 con 0x7fe70c004d40 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c00bb00 con 0x7fe70c0039e0 2024-04-25T08:38:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70c00bd00 con 0x7fe7200d0320 2024-04-25T08:38:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe726996640 1 --2- 172.21.15.38:0/2549472567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe70c004d40 0x7fe70c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe726195640 1 --2- 172.21.15.38:0/2549472567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe70c0039e0 0x7fe70c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe725994640 1 --2- 172.21.15.38:0/2549472567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200d0320 0x7fe70c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe726996640 1 -- 172.21.15.38:0/2549472567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200d0320 msgr2=0x7fe70c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe726996640 1 --2- 172.21.15.38:0/2549472567 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200d0320 0x7fe70c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe726996640 1 -- 172.21.15.38:0/2549472567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe70c0039e0 msgr2=0x7fe70c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe726996640 1 --2- 172.21.15.38:0/2549472567 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe70c0039e0 0x7fe70c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.167+0000 7fe726996640 1 -- 172.21.15.38:0/2549472567 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70c003520 con 0x7fe70c004d40 2024-04-25T08:38:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.168+0000 7fe726996640 1 --2- 172.21.15.38:0/2549472567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe70c004d40 0x7fe70c0051c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe718001e20 tx=0x7fe70c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7edf4e72f1b350a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.168+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718001a70 con 0x7fe70c004d40 2024-04-25T08:38:12.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.168+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71802ae40 con 0x7fe70c004d40 2024-04-25T08:38:12.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.168+0000 7fe725193640 1 -- 172.21.15.38:0/2549472567 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718003bb0 con 0x7fe70c004d40 2024-04-25T08:38:12.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.169+0000 7fe7283f9640 1 -- 172.21.15.38:0/2549472567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe70c004d40 msgr2=0x7fe70c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.169+0000 7fe7283f9640 1 --2- 172.21.15.38:0/2549472567 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe70c004d40 0x7fe70c0051c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe718001e20 tx=0x7fe70c005830 comp rx=0 tx=0).stop 2024-04-25T08:38:12.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.169+0000 7fe726996640 1 -- 172.21.15.38:0/2549472567 reap_dead start 2024-04-25T08:38:12.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.169+0000 7fe7283f9640 1 -- 172.21.15.38:0/2549472567 shutdown_connections 2024-04-25T08:38:12.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.169+0000 7fe7283f9640 1 -- 172.21.15.38:0/2549472567 >> 172.21.15.38:0/2549472567 conn(0x7fe7200668f0 msgr2=0x7fe720066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:12.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.169+0000 7fe7283f9640 1 -- 172.21.15.38:0/2549472567 shutdown_connections 2024-04-25T08:38:12.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.169+0000 7fe7283f9640 1 -- 172.21.15.38:0/2549472567 wait complete. 2024-04-25T08:38:12.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.170+0000 7fe7283f9640 1 Processor -- start 2024-04-25T08:38:12.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.170+0000 7fe7283f9640 1 -- start start 2024-04-25T08:38:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe7283f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe70c004d40 0x7fe7201e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe7283f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7200b5780 0x7fe72015aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe7283f9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200b4440 0x7fe7201e7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe7283f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200d0b40 con 0x7fe70c004d40 2024-04-25T08:38:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe7283f9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200d2eb0 con 0x7fe7200b5780 2024-04-25T08:38:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe7283f9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200b33a0 con 0x7fe7200b4440 2024-04-25T08:38:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe726195640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe70c004d40 0x7fe7201e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe726195640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe70c004d40 0x7fe7201e6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48996/0 (socket says 172.21.15.38:48996) 2024-04-25T08:38:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe726195640 1 -- 172.21.15.38:0/3775719768 learned_addr learned my addr 172.21.15.38:0/3775719768 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe726996640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200b4440 0x7fe7201e7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe725994640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7200b5780 0x7fe72015aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe726195640 1 -- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200b4440 msgr2=0x7fe7201e7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe726195640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200b4440 0x7fe7201e7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe726195640 1 -- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7200b5780 msgr2=0x7fe72015aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe726195640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7200b5780 0x7fe72015aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe726195640 1 -- 172.21.15.38:0/3775719768 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7200b3680 con 0x7fe70c004d40 2024-04-25T08:38:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe726195640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe70c004d40 0x7fe7201e6bd0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe7100119d0 tx=0x7fe710011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe716ffd640 1 -- 172.21.15.38:0/3775719768 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000fb50 con 0x7fe70c004d40 2024-04-25T08:38:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe716ffd640 1 -- 172.21.15.38:0/3775719768 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71000fcf0 con 0x7fe70c004d40 2024-04-25T08:38:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe716ffd640 1 -- 172.21.15.38:0/3775719768 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000fea0 con 0x7fe70c004d40 2024-04-25T08:38:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe7283f9640 1 -- 172.21.15.38:0/3775719768 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7200b3960 con 0x7fe70c004d40 2024-04-25T08:38:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.171+0000 7fe7283f9640 1 -- 172.21.15.38:0/3775719768 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7200b3f20 con 0x7fe70c004d40 2024-04-25T08:38:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.172+0000 7fe7283f9640 1 -- 172.21.15.38:0/3775719768 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe720068e80 con 0x7fe70c004d40 2024-04-25T08:38:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.173+0000 7fe716ffd640 1 -- 172.21.15.38:0/3775719768 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe710012040 con 0x7fe70c004d40 2024-04-25T08:38:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.173+0000 7fe716ffd640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe70406b500 0x7fe70406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.174+0000 7fe716ffd640 1 -- 172.21.15.38:0/3775719768 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fe71008a9a0 con 0x7fe70c004d40 2024-04-25T08:38:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.174+0000 7fe725994640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe70406b500 0x7fe70406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.174+0000 7fe725994640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe70406b500 0x7fe70406d9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe7200738e0 tx=0x7fe71c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.178+0000 7fe716ffd640 1 -- 172.21.15.38:0/3775719768 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe710019030 con 0x7fe70c004d40 2024-04-25T08:38:12.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.359+0000 7fe7283f9640 1 -- 172.21.15.38:0/3775719768 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe72015fda0 con 0x7fe70c004d40 2024-04-25T08:38:12.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.359+0000 7fe716ffd640 1 -- 172.21.15.38:0/3775719768 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe710059700 con 0x7fe70c004d40 2024-04-25T08:38:12.354 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:38:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.361+0000 7fe7283f9640 1 -- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe70406b500 msgr2=0x7fe70406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.361+0000 7fe7283f9640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe70406b500 0x7fe70406d9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe7200738e0 tx=0x7fe71c009040 comp rx=0 tx=0).stop 2024-04-25T08:38:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.361+0000 7fe7283f9640 1 -- 172.21.15.38:0/3775719768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe70c004d40 msgr2=0x7fe7201e6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.361+0000 7fe7283f9640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe70c004d40 0x7fe7201e6bd0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fe7100119d0 tx=0x7fe710011e90 comp rx=0 tx=0).stop 2024-04-25T08:38:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.361+0000 7fe7283f9640 1 -- 172.21.15.38:0/3775719768 shutdown_connections 2024-04-25T08:38:12.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.361+0000 7fe7283f9640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe70406b500 0x7fe70406d9c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.361+0000 7fe7283f9640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7200b4440 0x7fe7201e7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.361+0000 7fe7283f9640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7200b5780 0x7fe72015aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.361+0000 7fe7283f9640 1 --2- 172.21.15.38:0/3775719768 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe70c004d40 0x7fe7201e6bd0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.361+0000 7fe7283f9640 1 -- 172.21.15.38:0/3775719768 >> 172.21.15.38:0/3775719768 conn(0x7fe7200668f0 msgr2=0x7fe72015ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:12.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.361+0000 7fe7283f9640 1 -- 172.21.15.38:0/3775719768 shutdown_connections 2024-04-25T08:38:12.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.361+0000 7fe7283f9640 1 -- 172.21.15.38:0/3775719768 wait complete. 2024-04-25T08:38:12.366 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:12.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.473+0000 7f7136f42640 1 Processor -- start 2024-04-25T08:38:12.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.473+0000 7f7136f42640 1 -- start start 2024-04-25T08:38:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.474+0000 7f7136f42640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f713015dc10 0x7f71300ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.474+0000 7f7136f42640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f71300ced60 0x7f71300d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.474+0000 7f7136f42640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f71300d3900 0x7f71300d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.474+0000 7f7136f42640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f713015dff0 con 0x7f71300ced60 2024-04-25T08:38:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.474+0000 7f7136f42640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71300b6990 con 0x7f71300d3900 2024-04-25T08:38:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.474+0000 7f7136f42640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71300d4710 con 0x7f713015dc10 2024-04-25T08:38:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.474+0000 7f712ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f71300ced60 0x7f71300d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.474+0000 7f712ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f71300ced60 0x7f71300d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49006/0 (socket says 172.21.15.38:49006) 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.474+0000 7f712ffff640 1 -- 172.21.15.38:0/3713384245 learned_addr learned my addr 172.21.15.38:0/3713384245 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.475+0000 7f71354df640 1 --2- 172.21.15.38:0/3713384245 >> v2:172.21.15.107:3300/0 conn(0x7f71300d3900 0x7f71300d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.475+0000 7f7134cde640 1 --2- 172.21.15.38:0/3713384245 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f713015dc10 0x7f71300ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.475+0000 7f712ffff640 1 -- 172.21.15.38:0/3713384245 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f713015dc10 msgr2=0x7f71300ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.475+0000 7f712ffff640 1 --2- 172.21.15.38:0/3713384245 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f713015dc10 0x7f71300ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.475+0000 7f712ffff640 1 -- 172.21.15.38:0/3713384245 >> v2:172.21.15.107:3300/0 conn(0x7f71300d3900 msgr2=0x7f71300d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.475+0000 7f712ffff640 1 --2- 172.21.15.38:0/3713384245 >> v2:172.21.15.107:3300/0 conn(0x7f71300d3900 0x7f71300d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.475+0000 7f712ffff640 1 -- 172.21.15.38:0/3713384245 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71300d3e70 con 0x7f71300ced60 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.475+0000 7f712ffff640 1 --2- 172.21.15.38:0/3713384245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300ced60 0x7f71300d0850 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f71300b9380 tx=0x7f712002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8331dc8170421267 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.476+0000 7f712f7fe640 1 -- 172.21.15.38:0/3713384245 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120036070 con 0x7f71300ced60 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.476+0000 7f712f7fe640 1 -- 172.21.15.38:0/3713384245 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7120031600 con 0x7f71300ced60 2024-04-25T08:38:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.476+0000 7f712f7fe640 1 -- 172.21.15.38:0/3713384245 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200319d0 con 0x7f71300ced60 2024-04-25T08:38:12.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.476+0000 7f7136f42640 1 -- 172.21.15.38:0/3713384245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300ced60 msgr2=0x7f71300d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.476+0000 7f7136f42640 1 --2- 172.21.15.38:0/3713384245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300ced60 0x7f71300d0850 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f71300b9380 tx=0x7f712002a610 comp rx=0 tx=0).stop 2024-04-25T08:38:12.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.476+0000 7f7136f42640 1 -- 172.21.15.38:0/3713384245 shutdown_connections 2024-04-25T08:38:12.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.476+0000 7f7136f42640 1 --2- 172.21.15.38:0/3713384245 >> v2:172.21.15.107:3300/0 conn(0x7f71300d3900 0x7f71300d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.476+0000 7f7136f42640 1 --2- 172.21.15.38:0/3713384245 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300ced60 0x7f71300d0850 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.476+0000 7f7136f42640 1 --2- 172.21.15.38:0/3713384245 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f713015dc10 0x7f71300ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.476+0000 7f7136f42640 1 -- 172.21.15.38:0/3713384245 >> 172.21.15.38:0/3713384245 conn(0x7f7130066d10 msgr2=0x7f7130067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.476+0000 7f7136f42640 1 -- 172.21.15.38:0/3713384245 shutdown_connections 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.476+0000 7f7136f42640 1 -- 172.21.15.38:0/3713384245 wait complete. 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.477+0000 7f7136f42640 1 Processor -- start 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.477+0000 7f7136f42640 1 -- start start 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.478+0000 7f7136f42640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71300ced60 0x7f71300c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.478+0000 7f7136f42640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d3900 0x7f71300ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.478+0000 7f7136f42640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f713015dc10 0x7f713015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.478+0000 7f7136f42640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71300b6bc0 con 0x7f71300d3900 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.478+0000 7f7136f42640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71300d4430 con 0x7f713015dc10 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.478+0000 7f7136f42640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71300b2370 con 0x7f71300ced60 2024-04-25T08:38:12.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.478+0000 7f712ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d3900 0x7f71300ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.478+0000 7f712ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d3900 0x7f71300ccd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49010/0 (socket says 172.21.15.38:49010) 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.478+0000 7f712ffff640 1 -- 172.21.15.38:0/4142824727 learned_addr learned my addr 172.21.15.38:0/4142824727 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f7134cde640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71300ced60 0x7f71300c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f71354df640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f713015dc10 0x7f713015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f712ffff640 1 -- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71300ced60 msgr2=0x7f71300c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f712ffff640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71300ced60 0x7f71300c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f712ffff640 1 -- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f713015dc10 msgr2=0x7f713015fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f712ffff640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f713015dc10 0x7f713015fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f712ffff640 1 -- 172.21.15.38:0/4142824727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713015f460 con 0x7f71300d3900 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f712ffff640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d3900 0x7f71300ccd20 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f7120009f40 tx=0x7f7120004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f712d7fa640 1 -- 172.21.15.38:0/4142824727 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120041020 con 0x7f71300d3900 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f712d7fa640 1 -- 172.21.15.38:0/4142824727 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7120007800 con 0x7f71300d3900 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f7136f42640 1 -- 172.21.15.38:0/4142824727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71300cc240 con 0x7f71300d3900 2024-04-25T08:38:12.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f712d7fa640 1 -- 172.21.15.38:0/4142824727 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7120036040 con 0x7f71300d3900 2024-04-25T08:38:12.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.479+0000 7f7136f42640 1 -- 172.21.15.38:0/4142824727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71300cbea0 con 0x7f71300d3900 2024-04-25T08:38:12.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.480+0000 7f7136f42640 1 -- 172.21.15.38:0/4142824727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7130068e80 con 0x7f71300d3900 2024-04-25T08:38:12.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.480+0000 7f712d7fa640 1 -- 172.21.15.38:0/4142824727 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7120044020 con 0x7f71300d3900 2024-04-25T08:38:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.481+0000 7f712d7fa640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f711006b500 0x7f711006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.481+0000 7f712d7fa640 1 -- 172.21.15.38:0/4142824727 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f71200ab010 con 0x7f71300d3900 2024-04-25T08:38:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.481+0000 7f7134cde640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f711006b500 0x7f711006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.482+0000 7f7134cde640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f711006b500 0x7f711006d9c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f71280079a0 tx=0x7f7128008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.485+0000 7f712d7fa640 1 -- 172.21.15.38:0/4142824727 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f712004b030 con 0x7f71300d3900 2024-04-25T08:38:12.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.703+0000 7f7136f42640 1 -- 172.21.15.38:0/4142824727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71300c8d90 con 0x7f71300d3900 2024-04-25T08:38:12.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.704+0000 7f712d7fa640 1 -- 172.21.15.38:0/4142824727 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f71200796c0 con 0x7f71300d3900 2024-04-25T08:38:12.698 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:12.698 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3781069672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3781069672},{"type":"v1","addr":"172.21.15.107:6837","nonce":3781069672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/624970524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":624970524},{"type":"v1","addr":"172.21.15.107:6836","nonce":624970524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7380,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1256978001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1256978001},{"type":"v1","addr":"172.21.15.38:6834","nonce":1256978001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:36:26.904065+0000","modified":"2024-04-25T08:36:39.455074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/380751658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":380751658},{"type":"v1","addr":"172.21.15.38:6835","nonce":380751658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:38:12.698 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 51 2024-04-25T08:38:12.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.706+0000 7f7136f42640 1 -- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f711006b500 msgr2=0x7f711006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.706+0000 7f7136f42640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f711006b500 0x7f711006d9c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f71280079a0 tx=0x7f7128008040 comp rx=0 tx=0).stop 2024-04-25T08:38:12.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.706+0000 7f7136f42640 1 -- 172.21.15.38:0/4142824727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d3900 msgr2=0x7f71300ccd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.706+0000 7f7136f42640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d3900 0x7f71300ccd20 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f7120009f40 tx=0x7f7120004a40 comp rx=0 tx=0).stop 2024-04-25T08:38:12.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.706+0000 7f7136f42640 1 -- 172.21.15.38:0/4142824727 shutdown_connections 2024-04-25T08:38:12.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.706+0000 7f7136f42640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f713015dc10 0x7f713015fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.706+0000 7f7136f42640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f71300d3900 0x7f71300ccd20 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.706+0000 7f7136f42640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f711006b500 0x7f711006d9c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.706+0000 7f7136f42640 1 --2- 172.21.15.38:0/4142824727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71300ced60 0x7f71300c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.706+0000 7f7136f42640 1 -- 172.21.15.38:0/4142824727 >> 172.21.15.38:0/4142824727 conn(0x7f7130066d10 msgr2=0x7f71300b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:12.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.706+0000 7f7136f42640 1 -- 172.21.15.38:0/4142824727 shutdown_connections 2024-04-25T08:38:12.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.706+0000 7f7136f42640 1 -- 172.21.15.38:0/4142824727 wait complete. 2024-04-25T08:38:12.711 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:12.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.818+0000 7f408301f640 1 Processor -- start 2024-04-25T08:38:12.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.818+0000 7f408301f640 1 -- start start 2024-04-25T08:38:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.819+0000 7f408301f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f407c0d0f60 0x7f407c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.819+0000 7f408301f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c0d3e70 con 0x7f407c0d3900 2024-04-25T08:38:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.819+0000 7f408301f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c0d4150 con 0x7f407c15dc10 2024-04-25T08:38:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.819+0000 7f408301f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c15dff0 con 0x7f407c0d0f60 2024-04-25T08:38:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.819+0000 7f407bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f407c0d3900 0x7f407c0d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35396/0 (socket says 172.21.15.38:35396) 2024-04-25T08:38:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.819+0000 7f407bfff640 1 -- 172.21.15.38:0/2225118315 learned_addr learned my addr 172.21.15.38:0/2225118315 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.820+0000 7f40815bc640 1 --2- 172.21.15.38:0/2225118315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f407c0d0f60 0x7f407c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.820+0000 7f407b7fe640 1 -- 172.21.15.38:0/2225118315 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586241724 0 0) 0x7f407c0d3e70 con 0x7f407c0d3900 2024-04-25T08:38:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.820+0000 7f407b7fe640 1 -- 172.21.15.38:0/2225118315 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4064003520 con 0x7f407c0d3900 2024-04-25T08:38:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.820+0000 7f407b7fe640 1 -- 172.21.15.38:0/2225118315 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2855317005 0 0) 0x7f407c0d4150 con 0x7f407c15dc10 2024-04-25T08:38:12.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.820+0000 7f407b7fe640 1 -- 172.21.15.38:0/2225118315 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f407c0d3e70 con 0x7f407c15dc10 2024-04-25T08:38:12.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.820+0000 7f40815bc640 1 -- 172.21.15.38:0/2225118315 >> v1:172.21.15.107:6789/0 conn(0x7f407c15dc10 legacy=0x7f407c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.820+0000 7f40815bc640 1 -- 172.21.15.38:0/2225118315 >> v1:172.21.15.38:6789/0 conn(0x7f407c0d3900 legacy=0x7f407c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.820+0000 7f407b7fe640 1 -- 172.21.15.38:0/2225118315 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1884433660 0 0) 0x7f4064003520 con 0x7f407c0d3900 2024-04-25T08:38:12.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.820+0000 7f40815bc640 1 -- 172.21.15.38:0/2225118315 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0d4430 con 0x7f407c0d0f60 2024-04-25T08:38:12.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.821+0000 7f40815bc640 1 --2- 172.21.15.38:0/2225118315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f407c0d0f60 0x7f407c0b2370 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f406c0097d0 tx=0x7f406c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a38aaad13b94026 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.821+0000 7f407b7fe640 1 -- 172.21.15.38:0/2225118315 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c037070 con 0x7f407c0d0f60 2024-04-25T08:38:12.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.821+0000 7f407b7fe640 1 -- 172.21.15.38:0/2225118315 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406c02ab90 con 0x7f407c0d0f60 2024-04-25T08:38:12.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.821+0000 7f407b7fe640 1 -- 172.21.15.38:0/2225118315 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4070001690 con 0x7f407c0d0f60 2024-04-25T08:38:12.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.821+0000 7f408301f640 1 -- 172.21.15.38:0/2225118315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f407c0d0f60 msgr2=0x7f407c0b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.821+0000 7f408301f640 1 --2- 172.21.15.38:0/2225118315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f407c0d0f60 0x7f407c0b2370 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f406c0097d0 tx=0x7f406c02a100 comp rx=0 tx=0).stop 2024-04-25T08:38:12.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.821+0000 7f408301f640 1 -- 172.21.15.38:0/2225118315 shutdown_connections 2024-04-25T08:38:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.821+0000 7f408301f640 1 --2- 172.21.15.38:0/2225118315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f407c0d0f60 0x7f407c0b2370 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.821+0000 7f408301f640 1 -- 172.21.15.38:0/2225118315 >> 172.21.15.38:0/2225118315 conn(0x7f407c066d10 msgr2=0x7f407c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.822+0000 7f408301f640 1 -- 172.21.15.38:0/2225118315 shutdown_connections 2024-04-25T08:38:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.822+0000 7f408301f640 1 -- 172.21.15.38:0/2225118315 wait complete. 2024-04-25T08:38:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.823+0000 7f408301f640 1 Processor -- start 2024-04-25T08:38:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.823+0000 7f408301f640 1 -- start start 2024-04-25T08:38:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f408301f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f407c0d0f60 0x7f407c0cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f408301f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f407c0d3900 0x7f407c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f408301f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f407c15dc10 0x7f407c0ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f408301f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c0d4710 con 0x7f407c15dc10 2024-04-25T08:38:12.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f408301f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c0cef30 con 0x7f407c0d0f60 2024-04-25T08:38:12.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f408301f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c0b80f0 con 0x7f407c0d3900 2024-04-25T08:38:12.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f40815bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f407c15dc10 0x7f407c0ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f40815bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f407c15dc10 0x7f407c0ac3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49016/0 (socket says 172.21.15.38:49016) 2024-04-25T08:38:12.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f40815bc640 1 -- 172.21.15.38:0/34329447 learned_addr learned my addr 172.21.15.38:0/34329447 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:12.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f407bfff640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f407c0d3900 0x7f407c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f4080dbb640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f407c0d0f60 0x7f407c0cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f40815bc640 1 -- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f407c0d3900 msgr2=0x7f407c0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f40815bc640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f407c0d3900 0x7f407c0cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f40815bc640 1 -- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f407c0d0f60 msgr2=0x7f407c0cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:12.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f40815bc640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f407c0d0f60 0x7f407c0cbf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:12.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f40815bc640 1 -- 172.21.15.38:0/34329447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0b3f40 con 0x7f407c15dc10 2024-04-25T08:38:12.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f4080dbb640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f407c0d0f60 0x7f407c0cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:12.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f40815bc640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f407c15dc10 0x7f407c0ac3c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f406c009720 tx=0x7f406c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f40797fa640 1 -- 172.21.15.38:0/34329447 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c041070 con 0x7f407c15dc10 2024-04-25T08:38:12.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f40797fa640 1 -- 172.21.15.38:0/34329447 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406c03c660 con 0x7f407c15dc10 2024-04-25T08:38:12.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f408301f640 1 -- 172.21.15.38:0/34329447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407c0b4220 con 0x7f407c15dc10 2024-04-25T08:38:12.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f40797fa640 1 -- 172.21.15.38:0/34329447 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c037040 con 0x7f407c15dc10 2024-04-25T08:38:12.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.824+0000 7f408301f640 1 -- 172.21.15.38:0/34329447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407c0cb760 con 0x7f407c15dc10 2024-04-25T08:38:12.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.825+0000 7f408301f640 1 -- 172.21.15.38:0/34329447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407c0abd40 con 0x7f407c15dc10 2024-04-25T08:38:12.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.826+0000 7f40797fa640 1 -- 172.21.15.38:0/34329447 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f406c044070 con 0x7f407c15dc10 2024-04-25T08:38:12.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.826+0000 7f40797fa640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f406806b4d0 0x7f406806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:12.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.826+0000 7f40797fa640 1 -- 172.21.15.38:0/34329447 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f406c0aada0 con 0x7f407c15dc10 2024-04-25T08:38:12.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.827+0000 7f4080dbb640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f406806b4d0 0x7f406806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:12.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.827+0000 7f4080dbb640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f406806b4d0 0x7f406806d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4074007fd0 tx=0x7f4074009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:12.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:12.830+0000 7f40797fa640 1 -- 172.21.15.38:0/34329447 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f407c0abd40 con 0x7f407c15dc10 2024-04-25T08:38:13.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.011+0000 7f408301f640 1 -- 172.21.15.38:0/34329447 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f407c068e80 con 0x7f407c15dc10 2024-04-25T08:38:13.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.012+0000 7f40797fa640 1 -- 172.21.15.38:0/34329447 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16999 (secure 0 0 0) 0x7f406c079af0 con 0x7f407c15dc10 2024-04-25T08:38:13.005 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:13.006 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":84,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:37:50.914128+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:25.176504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:25.877954+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":83,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:13.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.013+0000 7f408301f640 1 -- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f406806b4d0 msgr2=0x7f406806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.013+0000 7f408301f640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f406806b4d0 0x7f406806d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4074007fd0 tx=0x7f4074009040 comp rx=0 tx=0).stop 2024-04-25T08:38:13.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.014+0000 7f408301f640 1 -- 172.21.15.38:0/34329447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f407c15dc10 msgr2=0x7f407c0ac3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.014+0000 7f408301f640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f407c15dc10 0x7f407c0ac3c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f406c009720 tx=0x7f406c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.014+0000 7f408301f640 1 -- 172.21.15.38:0/34329447 shutdown_connections 2024-04-25T08:38:13.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.014+0000 7f408301f640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f407c15dc10 0x7f407c0ac3c0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.014+0000 7f408301f640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f406806b4d0 0x7f406806d990 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.014+0000 7f408301f640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f407c0d3900 0x7f407c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.014+0000 7f408301f640 1 --2- 172.21.15.38:0/34329447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f407c0d0f60 0x7f407c0cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.014+0000 7f408301f640 1 -- 172.21.15.38:0/34329447 >> 172.21.15.38:0/34329447 conn(0x7f407c066d10 msgr2=0x7f407c15e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:13.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.014+0000 7f408301f640 1 -- 172.21.15.38:0/34329447 shutdown_connections 2024-04-25T08:38:13.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.014+0000 7f408301f640 1 -- 172.21.15.38:0/34329447 wait complete. 2024-04-25T08:38:13.020 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:38:13.020 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:13.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.127+0000 7f43e221f640 1 Processor -- start 2024-04-25T08:38:13.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.127+0000 7f43e221f640 1 -- start start 2024-04-25T08:38:13.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.128+0000 7f43e221f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f43dc06cc10 0x7f43dc0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.128+0000 7f43e221f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43dc0d05c0 con 0x7f43dc06cff0 2024-04-25T08:38:13.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.128+0000 7f43e221f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43dc0d08a0 con 0x7f43dc09a180 2024-04-25T08:38:13.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.128+0000 7f43e221f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43dc0d0b80 con 0x7f43dc06cc10 2024-04-25T08:38:13.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.128+0000 7f43db7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f43dc06cc10 0x7f43dc0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.128+0000 7f43db7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f43dc06cc10 0x7f43dc0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49020/0 (socket says 172.21.15.38:49020) 2024-04-25T08:38:13.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.128+0000 7f43daffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f43dc06cff0 0x7f43dc099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35408/0 (socket says 172.21.15.38:35408) 2024-04-25T08:38:13.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.128+0000 7f43db7fe640 1 -- 172.21.15.38:0/2857830764 learned_addr learned my addr 172.21.15.38:0/2857830764 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:13.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.129+0000 7f43db7fe640 1 -- 172.21.15.38:0/2857830764 >> v1:172.21.15.107:6789/0 conn(0x7f43dc09a180 legacy=0x7f43dc15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:13.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.129+0000 7f43db7fe640 1 -- 172.21.15.38:0/2857830764 >> v1:172.21.15.38:6789/0 conn(0x7f43dc06cff0 legacy=0x7f43dc099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.129+0000 7f43db7fe640 1 -- 172.21.15.38:0/2857830764 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc0d0e60 con 0x7f43dc06cc10 2024-04-25T08:38:13.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.129+0000 7f43db7fe640 1 --2- 172.21.15.38:0/2857830764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43dc06cc10 0x7f43dc0cfa70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009870 tx=0x7f43cc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfba00bf9a41e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:13.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.130+0000 7f43da7fc640 1 -- 172.21.15.38:0/2857830764 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc037070 con 0x7f43dc06cc10 2024-04-25T08:38:13.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.130+0000 7f43da7fc640 1 -- 172.21.15.38:0/2857830764 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc032440 con 0x7f43dc06cc10 2024-04-25T08:38:13.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.130+0000 7f43da7fc640 1 -- 172.21.15.38:0/2857830764 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc032810 con 0x7f43dc06cc10 2024-04-25T08:38:13.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.130+0000 7f43e221f640 1 -- 172.21.15.38:0/2857830764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43dc06cc10 msgr2=0x7f43dc0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.130+0000 7f43e221f640 1 --2- 172.21.15.38:0/2857830764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43dc06cc10 0x7f43dc0cfa70 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009870 tx=0x7f43cc02a2a0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.130+0000 7f43e221f640 1 -- 172.21.15.38:0/2857830764 shutdown_connections 2024-04-25T08:38:13.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.130+0000 7f43e221f640 1 --2- 172.21.15.38:0/2857830764 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43dc06cc10 0x7f43dc0cfa70 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.130+0000 7f43e221f640 1 -- 172.21.15.38:0/2857830764 >> 172.21.15.38:0/2857830764 conn(0x7f43dc0b31f0 msgr2=0x7f43dc0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:13.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.130+0000 7f43e221f640 1 -- 172.21.15.38:0/2857830764 shutdown_connections 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.130+0000 7f43e221f640 1 -- 172.21.15.38:0/2857830764 wait complete. 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.131+0000 7f43e221f640 1 Processor -- start 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.132+0000 7f43e221f640 1 -- start start 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.132+0000 7f43e221f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43dc06cc10 0x7f43dc0c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.132+0000 7f43e221f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc06cff0 0x7f43dc0ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.132+0000 7f43e221f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc09a180 0x7f43dc09ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.132+0000 7f43e221f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc06c2d0 con 0x7f43dc06cc10 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.132+0000 7f43e221f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc06c150 con 0x7f43dc09a180 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.132+0000 7f43e221f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43dc0ccd60 con 0x7f43dc06cff0 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.132+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43dc06cc10 0x7f43dc0c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.132+0000 7f43db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43dc06cc10 0x7f43dc0c87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49036/0 (socket says 172.21.15.38:49036) 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.132+0000 7f43db7fe640 1 -- 172.21.15.38:0/3650480238 learned_addr learned my addr 172.21.15.38:0/3650480238 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.133+0000 7f43db7fe640 1 -- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc06cff0 msgr2=0x7f43dc0ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.133+0000 7f43db7fe640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc06cff0 0x7f43dc0ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.133+0000 7f43db7fe640 1 -- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc09a180 msgr2=0x7f43dc09ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.133+0000 7f43dbfff640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc09a180 0x7f43dc09ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.133+0000 7f43db7fe640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc09a180 0x7f43dc09ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.133+0000 7f43db7fe640 1 -- 172.21.15.38:0/3650480238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc0ca8a0 con 0x7f43dc06cc10 2024-04-25T08:38:13.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.133+0000 7f43db7fe640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43dc06cc10 0x7f43dc0c87d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009870 tx=0x7f43cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:13.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.133+0000 7f43c3fff640 1 -- 172.21.15.38:0/3650480238 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc041070 con 0x7f43dc06cc10 2024-04-25T08:38:13.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.133+0000 7f43c3fff640 1 -- 172.21.15.38:0/3650480238 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc03c870 con 0x7f43dc06cc10 2024-04-25T08:38:13.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.133+0000 7f43c3fff640 1 -- 172.21.15.38:0/3650480238 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc037040 con 0x7f43dc06cc10 2024-04-25T08:38:13.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.133+0000 7f43e221f640 1 -- 172.21.15.38:0/3650480238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43dc0c8db0 con 0x7f43dc06cc10 2024-04-25T08:38:13.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.133+0000 7f43e221f640 1 -- 172.21.15.38:0/3650480238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43dc0c9370 con 0x7f43dc06cc10 2024-04-25T08:38:13.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.134+0000 7f43e221f640 1 -- 172.21.15.38:0/3650480238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43dc0b1fa0 con 0x7f43dc06cc10 2024-04-25T08:38:13.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.134+0000 7f43c3fff640 1 -- 172.21.15.38:0/3650480238 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43cc044070 con 0x7f43dc06cc10 2024-04-25T08:38:13.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.135+0000 7f43c3fff640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f43c806b4d0 0x7f43c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.135+0000 7f43c3fff640 1 -- 172.21.15.38:0/3650480238 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f43cc0aaf20 con 0x7f43dc06cc10 2024-04-25T08:38:13.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.135+0000 7f43daffd640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f43c806b4d0 0x7f43c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.136+0000 7f43daffd640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f43c806b4d0 0x7f43c806d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f43d0000f60 tx=0x7f43d0007a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:13.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.139+0000 7f43c3fff640 1 -- 172.21.15.38:0/3650480238 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43dc0b1fa0 con 0x7f43dc06cc10 2024-04-25T08:38:13.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.355+0000 7f43e221f640 1 -- 172.21.15.38:0/3650480238 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43dc0b14b0 con 0x7f43dc06cc10 2024-04-25T08:38:13.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.356+0000 7f43c3fff640 1 -- 172.21.15.38:0/3650480238 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f43dc0b14b0 con 0x7f43dc06cc10 2024-04-25T08:38:13.350 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:13.350 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3781069672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3781069672},{"type":"v1","addr":"172.21.15.107:6837","nonce":3781069672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/624970524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":624970524},{"type":"v1","addr":"172.21.15.107:6836","nonce":624970524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7380,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1256978001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1256978001},{"type":"v1","addr":"172.21.15.38:6834","nonce":1256978001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:36:26.904065+0000","modified":"2024-04-25T08:36:39.455074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/380751658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":380751658},{"type":"v1","addr":"172.21.15.38:6835","nonce":380751658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:38:13.350 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 51 2024-04-25T08:38:13.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.358+0000 7f43e221f640 1 -- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f43c806b4d0 msgr2=0x7f43c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.358+0000 7f43e221f640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f43c806b4d0 0x7f43c806d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f43d0000f60 tx=0x7f43d0007a00 comp rx=0 tx=0).stop 2024-04-25T08:38:13.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.358+0000 7f43e221f640 1 -- 172.21.15.38:0/3650480238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43dc06cc10 msgr2=0x7f43dc0c87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.358+0000 7f43e221f640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43dc06cc10 0x7f43dc0c87d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009870 tx=0x7f43cc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.358+0000 7f43e221f640 1 -- 172.21.15.38:0/3650480238 shutdown_connections 2024-04-25T08:38:13.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.358+0000 7f43e221f640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43dc09a180 0x7f43dc09ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.358+0000 7f43e221f640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f43c806b4d0 0x7f43c806d990 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.358+0000 7f43e221f640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43dc06cff0 0x7f43dc0ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.358+0000 7f43e221f640 1 --2- 172.21.15.38:0/3650480238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43dc06cc10 0x7f43dc0c87d0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.358+0000 7f43e221f640 1 -- 172.21.15.38:0/3650480238 >> 172.21.15.38:0/3650480238 conn(0x7f43dc0b31f0 msgr2=0x7f43dc0b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:13.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.358+0000 7f43e221f640 1 -- 172.21.15.38:0/3650480238 shutdown_connections 2024-04-25T08:38:13.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.358+0000 7f43e221f640 1 -- 172.21.15.38:0/3650480238 wait complete. 2024-04-25T08:38:13.364 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:13.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.472+0000 7f7e6db30640 1 Processor -- start 2024-04-25T08:38:13.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.472+0000 7f7e6db30640 1 -- start start 2024-04-25T08:38:13.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.473+0000 7f7e6db30640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e6815da90 0x7f7e68161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.473+0000 7f7e6db30640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e68161c50 0x7f7e680d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.473+0000 7f7e6db30640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e68162160 con 0x7f7e680d0680 2024-04-25T08:38:13.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.473+0000 7f7e6db30640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e680d19b0 con 0x7f7e68161c50 2024-04-25T08:38:13.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.473+0000 7f7e6db30640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680d1f70 con 0x7f7e6815da90 2024-04-25T08:38:13.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.473+0000 7f7e66ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e68161c50 0x7f7e680d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.473+0000 7f7e66ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7e68161c50 0x7f7e680d0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49046/0 (socket says 172.21.15.38:49046) 2024-04-25T08:38:13.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.473+0000 7f7e66ffd640 1 -- 172.21.15.38:0/3634285194 learned_addr learned my addr 172.21.15.38:0/3634285194 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:13.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.473+0000 7f7e677fe640 1 --2- 172.21.15.38:0/3634285194 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e6815da90 0x7f7e68161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.474+0000 7f7e66ffd640 1 -- 172.21.15.38:0/3634285194 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e6815da90 msgr2=0x7f7e68161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:13.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.474+0000 7f7e66ffd640 1 --2- 172.21.15.38:0/3634285194 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e6815da90 0x7f7e68161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.474+0000 7f7e66ffd640 1 -- 172.21.15.38:0/3634285194 >> v1:172.21.15.107:6789/0 conn(0x7f7e680d0680 legacy=0x7f7e680b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.474+0000 7f7e66ffd640 1 -- 172.21.15.38:0/3634285194 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e68162440 con 0x7f7e68161c50 2024-04-25T08:38:13.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.474+0000 7f7e66ffd640 1 --2- 172.21.15.38:0/3634285194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e68161c50 0x7f7e680d0140 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0098a0 tx=0x7f7e5c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f9fc1e5449e32a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:13.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.474+0000 7f7e667fc640 1 -- 172.21.15.38:0/3634285194 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c037070 con 0x7f7e68161c50 2024-04-25T08:38:13.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.475+0000 7f7e667fc640 1 -- 172.21.15.38:0/3634285194 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c032440 con 0x7f7e68161c50 2024-04-25T08:38:13.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.475+0000 7f7e667fc640 1 -- 172.21.15.38:0/3634285194 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c032810 con 0x7f7e68161c50 2024-04-25T08:38:13.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.475+0000 7f7e6db30640 1 -- 172.21.15.38:0/3634285194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e68161c50 msgr2=0x7f7e680d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.475+0000 7f7e6db30640 1 --2- 172.21.15.38:0/3634285194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e68161c50 0x7f7e680d0140 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0098a0 tx=0x7f7e5c02a290 comp rx=0 tx=0).stop 2024-04-25T08:38:13.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.475+0000 7f7e6db30640 1 -- 172.21.15.38:0/3634285194 shutdown_connections 2024-04-25T08:38:13.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.475+0000 7f7e6db30640 1 --2- 172.21.15.38:0/3634285194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e68161c50 0x7f7e680d0140 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.475+0000 7f7e6db30640 1 --2- 172.21.15.38:0/3634285194 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e6815da90 0x7f7e68161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.475+0000 7f7e6db30640 1 -- 172.21.15.38:0/3634285194 >> 172.21.15.38:0/3634285194 conn(0x7f7e6806c060 msgr2=0x7f7e6806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:13.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.475+0000 7f7e6db30640 1 -- 172.21.15.38:0/3634285194 shutdown_connections 2024-04-25T08:38:13.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.475+0000 7f7e6db30640 1 -- 172.21.15.38:0/3634285194 wait complete. 2024-04-25T08:38:13.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.476+0000 7f7e6db30640 1 Processor -- start 2024-04-25T08:38:13.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.476+0000 7f7e6db30640 1 -- start start 2024-04-25T08:38:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.477+0000 7f7e6db30640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e680d0680 0x7f7e6815b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.477+0000 7f7e6db30640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6815da90 0x7f7e6815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.477+0000 7f7e6db30640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e68161c50 0x7f7e68160be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.477+0000 7f7e6db30640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e680b9400 con 0x7f7e6815da90 2024-04-25T08:38:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.477+0000 7f7e6db30640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6815de70 con 0x7f7e680d0680 2024-04-25T08:38:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.477+0000 7f7e6db30640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e681590a0 con 0x7f7e68161c50 2024-04-25T08:38:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.477+0000 7f7e66ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6815da90 0x7f7e6815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.477+0000 7f7e66ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6815da90 0x7f7e6815b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49052/0 (socket says 172.21.15.38:49052) 2024-04-25T08:38:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.477+0000 7f7e66ffd640 1 -- 172.21.15.38:0/380164229 learned_addr learned my addr 172.21.15.38:0/380164229 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e677fe640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e680d0680 0x7f7e6815b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e67fff640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e68161c50 0x7f7e68160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e66ffd640 1 -- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e68161c50 msgr2=0x7f7e68160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e66ffd640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e68161c50 0x7f7e68160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e66ffd640 1 -- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e680d0680 msgr2=0x7f7e6815b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e66ffd640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e680d0680 0x7f7e6815b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e66ffd640 1 -- 172.21.15.38:0/380164229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6815bf60 con 0x7f7e6815da90 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e677fe640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e680d0680 0x7f7e6815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e67fff640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e68161c50 0x7f7e68160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e66ffd640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6815da90 0x7f7e6815b930 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c000c00 tx=0x7f7e5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e4ffff640 1 -- 172.21.15.38:0/380164229 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c041020 con 0x7f7e6815da90 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e4ffff640 1 -- 172.21.15.38:0/380164229 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c0306c0 con 0x7f7e6815da90 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e6db30640 1 -- 172.21.15.38:0/380164229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6815c240 con 0x7f7e6815da90 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e4ffff640 1 -- 172.21.15.38:0/380164229 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c037040 con 0x7f7e6815da90 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.478+0000 7f7e6db30640 1 -- 172.21.15.38:0/380164229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e68149150 con 0x7f7e6815da90 2024-04-25T08:38:13.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.479+0000 7f7e6db30640 1 -- 172.21.15.38:0/380164229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e68149350 con 0x7f7e6815da90 2024-04-25T08:38:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.480+0000 7f7e4ffff640 1 -- 172.21.15.38:0/380164229 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e5c044020 con 0x7f7e6815da90 2024-04-25T08:38:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.480+0000 7f7e4ffff640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e5406b4d0 0x7f7e5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.480+0000 7f7e4ffff640 1 -- 172.21.15.38:0/380164229 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f7e5c0aaaf0 con 0x7f7e6815da90 2024-04-25T08:38:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.480+0000 7f7e677fe640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e5406b4d0 0x7f7e5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.481+0000 7f7e677fe640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e5406b4d0 0x7f7e5406d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7e58006fd0 tx=0x7f7e58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:13.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.484+0000 7f7e4ffff640 1 -- 172.21.15.38:0/380164229 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e68149350 con 0x7f7e6815da90 2024-04-25T08:38:13.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.702+0000 7f7e6db30640 1 -- 172.21.15.38:0/380164229 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e68068e80 con 0x7f7e6815da90 2024-04-25T08:38:13.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.703+0000 7f7e4ffff640 1 -- 172.21.15.38:0/380164229 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f7e5c04b030 con 0x7f7e6815da90 2024-04-25T08:38:13.697 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:13.697 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3781069672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3781069672},{"type":"v1","addr":"172.21.15.107:6837","nonce":3781069672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/624970524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":624970524},{"type":"v1","addr":"172.21.15.107:6836","nonce":624970524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7380,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1256978001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1256978001},{"type":"v1","addr":"172.21.15.38:6834","nonce":1256978001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:36:26.904065+0000","modified":"2024-04-25T08:36:39.455074+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/380751658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":380751658},{"type":"v1","addr":"172.21.15.38:6835","nonce":380751658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:38:13.697 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 51 2024-04-25T08:38:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.705+0000 7f7e6db30640 1 -- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e5406b4d0 msgr2=0x7f7e5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.705+0000 7f7e6db30640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e5406b4d0 0x7f7e5406d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f7e58006fd0 tx=0x7f7e58008040 comp rx=0 tx=0).stop 2024-04-25T08:38:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.705+0000 7f7e6db30640 1 -- 172.21.15.38:0/380164229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6815da90 msgr2=0x7f7e6815b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.705+0000 7f7e6db30640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6815da90 0x7f7e6815b930 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c000c00 tx=0x7f7e5c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.705+0000 7f7e6db30640 1 -- 172.21.15.38:0/380164229 shutdown_connections 2024-04-25T08:38:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.705+0000 7f7e6db30640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e5406b4d0 0x7f7e5406d990 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.705+0000 7f7e6db30640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7e68161c50 0x7f7e68160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.705+0000 7f7e6db30640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e6815da90 0x7f7e6815b930 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.705+0000 7f7e6db30640 1 --2- 172.21.15.38:0/380164229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e680d0680 0x7f7e6815b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.705+0000 7f7e6db30640 1 -- 172.21.15.38:0/380164229 >> 172.21.15.38:0/380164229 conn(0x7f7e6806c060 msgr2=0x7f7e680c9470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.705+0000 7f7e6db30640 1 -- 172.21.15.38:0/380164229 shutdown_connections 2024-04-25T08:38:13.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.705+0000 7f7e6db30640 1 -- 172.21.15.38:0/380164229 wait complete. 2024-04-25T08:38:13.710 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:13.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.817+0000 7fcb961f4640 1 Processor -- start 2024-04-25T08:38:13.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.817+0000 7fcb961f4640 1 -- start start 2024-04-25T08:38:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb961f4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb900cd8d0 0x7fcb9006c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb961f4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcb900b2170 0x7fcb9006cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb961f4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb90161540 con 0x7fcb9009acb0 2024-04-25T08:38:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb961f4640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9009a820 con 0x7fcb900b2170 2024-04-25T08:38:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb961f4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9006be90 con 0x7fcb900cd8d0 2024-04-25T08:38:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb8ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcb900b2170 0x7fcb9006cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb8ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcb900b2170 0x7fcb9006cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49066/0 (socket says 172.21.15.38:49066) 2024-04-25T08:38:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb8ffff640 1 -- 172.21.15.38:0/834767534 learned_addr learned my addr 172.21.15.38:0/834767534 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb8effd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcb9009acb0 0x7fcb900cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35414/0 (socket says 172.21.15.38:35414) 2024-04-25T08:38:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb8f7fe640 1 --2- 172.21.15.38:0/834767534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb900cd8d0 0x7fcb9006c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb8ffff640 1 -- 172.21.15.38:0/834767534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb900cd8d0 msgr2=0x7fcb9006c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:13.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb8ffff640 1 --2- 172.21.15.38:0/834767534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb900cd8d0 0x7fcb9006c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb8ffff640 1 -- 172.21.15.38:0/834767534 >> v1:172.21.15.38:6789/0 conn(0x7fcb9009acb0 legacy=0x7fcb900cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb8ffff640 1 -- 172.21.15.38:0/834767534 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9009a520 con 0x7fcb900b2170 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.818+0000 7fcb8e7fc640 1 -- 172.21.15.38:0/834767534 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055221136 0 0) 0x7fcb90161540 con 0x7fcb9009acb0 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.819+0000 7fcb8ffff640 1 --2- 172.21.15.38:0/834767534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900b2170 0x7fcb9006cc20 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009e50 tx=0x7fcb8802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76b3a31c3c8dfba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.819+0000 7fcb8e7fc640 1 -- 172.21.15.38:0/834767534 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88036070 con 0x7fcb900b2170 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.819+0000 7fcb8e7fc640 1 -- 172.21.15.38:0/834767534 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb88031600 con 0x7fcb900b2170 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.819+0000 7fcb8e7fc640 1 -- 172.21.15.38:0/834767534 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb880319d0 con 0x7fcb900b2170 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.820+0000 7fcb961f4640 1 -- 172.21.15.38:0/834767534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900b2170 msgr2=0x7fcb9006cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.820+0000 7fcb961f4640 1 --2- 172.21.15.38:0/834767534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900b2170 0x7fcb9006cc20 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009e50 tx=0x7fcb8802a710 comp rx=0 tx=0).stop 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.820+0000 7fcb961f4640 1 -- 172.21.15.38:0/834767534 shutdown_connections 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.820+0000 7fcb961f4640 1 --2- 172.21.15.38:0/834767534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb900b2170 0x7fcb9006cc20 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.820+0000 7fcb961f4640 1 --2- 172.21.15.38:0/834767534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb900cd8d0 0x7fcb9006c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.820+0000 7fcb961f4640 1 -- 172.21.15.38:0/834767534 >> 172.21.15.38:0/834767534 conn(0x7fcb900b3160 msgr2=0x7fcb900b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:13.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.820+0000 7fcb961f4640 1 -- 172.21.15.38:0/834767534 shutdown_connections 2024-04-25T08:38:13.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.820+0000 7fcb961f4640 1 -- 172.21.15.38:0/834767534 wait complete. 2024-04-25T08:38:13.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.821+0000 7fcb961f4640 1 Processor -- start 2024-04-25T08:38:13.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.821+0000 7fcb961f4640 1 -- start start 2024-04-25T08:38:13.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb961f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb9009acb0 0x7fcb9015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb961f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb900b2170 0x7fcb90152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb961f4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb900cd8d0 0x7fcb900b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb961f4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9006c410 con 0x7fcb9009acb0 2024-04-25T08:38:13.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb961f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900d0ad0 con 0x7fcb900b2170 2024-04-25T08:38:13.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb961f4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900b26c0 con 0x7fcb900cd8d0 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb8f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb9009acb0 0x7fcb9015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb8f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb9009acb0 0x7fcb9015bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49072/0 (socket says 172.21.15.38:49072) 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb8f7fe640 1 -- 172.21.15.38:0/4192728657 learned_addr learned my addr 172.21.15.38:0/4192728657 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb8f7fe640 1 -- 172.21.15.38:0/4192728657 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb900cd8d0 msgr2=0x7fcb900b47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb8f7fe640 1 --2- 172.21.15.38:0/4192728657 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb900cd8d0 0x7fcb900b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb8f7fe640 1 -- 172.21.15.38:0/4192728657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb900b2170 msgr2=0x7fcb90152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb8f7fe640 1 --2- 172.21.15.38:0/4192728657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb900b2170 0x7fcb90152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb8f7fe640 1 -- 172.21.15.38:0/4192728657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb90153060 con 0x7fcb9009acb0 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.822+0000 7fcb8f7fe640 1 --2- 172.21.15.38:0/4192728657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb9009acb0 0x7fcb9015bbc0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400e870 tx=0x7fcb8400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.823+0000 7fcb77fff640 1 -- 172.21.15.38:0/4192728657 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84017070 con 0x7fcb9009acb0 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.823+0000 7fcb77fff640 1 -- 172.21.15.38:0/4192728657 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8400c940 con 0x7fcb9009acb0 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.823+0000 7fcb961f4640 1 -- 172.21.15.38:0/4192728657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb900b45e0 con 0x7fcb9009acb0 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.823+0000 7fcb77fff640 1 -- 172.21.15.38:0/4192728657 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8400cd30 con 0x7fcb9009acb0 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.823+0000 7fcb961f4640 1 -- 172.21.15.38:0/4192728657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb900c7e40 con 0x7fcb9009acb0 2024-04-25T08:38:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.824+0000 7fcb961f4640 1 -- 172.21.15.38:0/4192728657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb900ca7e0 con 0x7fcb9009acb0 2024-04-25T08:38:13.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.824+0000 7fcb77fff640 1 -- 172.21.15.38:0/4192728657 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb8401a070 con 0x7fcb9009acb0 2024-04-25T08:38:13.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.824+0000 7fcb77fff640 1 --2- 172.21.15.38:0/4192728657 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcb7c06b4d0 0x7fcb7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:13.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.825+0000 7fcb77fff640 1 -- 172.21.15.38:0/4192728657 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fcb8408a1e0 con 0x7fcb9009acb0 2024-04-25T08:38:13.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.825+0000 7fcb8effd640 1 --2- 172.21.15.38:0/4192728657 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcb7c06b4d0 0x7fcb7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:13.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.825+0000 7fcb8effd640 1 --2- 172.21.15.38:0/4192728657 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcb7c06b4d0 0x7fcb7c06d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcb80007fd0 tx=0x7fcb80007920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:13.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:13.829+0000 7fcb77fff640 1 -- 172.21.15.38:0/4192728657 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb900ca7e0 con 0x7fcb9009acb0 2024-04-25T08:38:14.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.012+0000 7fcb961f4640 1 -- 172.21.15.38:0/4192728657 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb900b6780 con 0x7fcb9009acb0 2024-04-25T08:38:14.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.013+0000 7fcb77fff640 1 -- 172.21.15.38:0/4192728657 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16999 (secure 0 0 0) 0x7fcb90153060 con 0x7fcb9009acb0 2024-04-25T08:38:14.007 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:14.007 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":84,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:37:50.914128+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:36:25.176504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:36:25.877954+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":83,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.015+0000 7fcb961f4640 1 -- 172.21.15.38:0/4192728657 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcb7c06b4d0 msgr2=0x7fcb7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.015+0000 7fcb961f4640 1 --2- 172.21.15.38:0/4192728657 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcb7c06b4d0 0x7fcb7c06d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcb80007fd0 tx=0x7fcb80007920 comp rx=0 tx=0).stop 2024-04-25T08:38:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.015+0000 7fcb961f4640 1 -- 172.21.15.38:0/4192728657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb9009acb0 msgr2=0x7fcb9015bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.015+0000 7fcb961f4640 1 --2- 172.21.15.38:0/4192728657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb9009acb0 0x7fcb9015bbc0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400e870 tx=0x7fcb8400ed30 comp rx=0 tx=0).stop 2024-04-25T08:38:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.015+0000 7fcb961f4640 1 -- 172.21.15.38:0/4192728657 shutdown_connections 2024-04-25T08:38:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.015+0000 7fcb961f4640 1 --2- 172.21.15.38:0/4192728657 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcb7c06b4d0 0x7fcb7c06d990 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.015+0000 7fcb961f4640 1 --2- 172.21.15.38:0/4192728657 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcb900cd8d0 0x7fcb900b47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.015+0000 7fcb961f4640 1 --2- 172.21.15.38:0/4192728657 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb900b2170 0x7fcb90152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.015+0000 7fcb961f4640 1 --2- 172.21.15.38:0/4192728657 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb9009acb0 0x7fcb9015bbc0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.015+0000 7fcb961f4640 1 -- 172.21.15.38:0/4192728657 >> 172.21.15.38:0/4192728657 conn(0x7fcb900b3160 msgr2=0x7fcb90070970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.015+0000 7fcb961f4640 1 -- 172.21.15.38:0/4192728657 shutdown_connections 2024-04-25T08:38:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.015+0000 7fcb961f4640 1 -- 172.21.15.38:0/4192728657 wait complete. 2024-04-25T08:38:14.020 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:38:14.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.128+0000 7f8e61f7e640 1 Processor -- start 2024-04-25T08:38:14.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.129+0000 7f8e61f7e640 1 -- start start 2024-04-25T08:38:14.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.129+0000 7f8e61f7e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8e5c15db70 0x7f8e5c0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.129+0000 7f8e61f7e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8e5c0cecf0 0x7f8e5c0d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.129+0000 7f8e61f7e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e5c15df50 con 0x7f8e5c0b92c0 2024-04-25T08:38:14.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.129+0000 7f8e61f7e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c0d1d10 con 0x7f8e5c15db70 2024-04-25T08:38:14.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.129+0000 7f8e61f7e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c0d0750 con 0x7f8e5c0cecf0 2024-04-25T08:38:14.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.129+0000 7f8e5b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8e5c15db70 0x7f8e5c0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.129+0000 7f8e5b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8e5c15db70 0x7f8e5c0d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49076/0 (socket says 172.21.15.38:49076) 2024-04-25T08:38:14.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.129+0000 7f8e5b7fe640 1 -- 172.21.15.38:0/130663116 learned_addr learned my addr 172.21.15.38:0/130663116 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:14.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.130+0000 7f8e5bfff640 1 --2- 172.21.15.38:0/130663116 >> v2:172.21.15.107:3300/0 conn(0x7f8e5c0cecf0 0x7f8e5c0d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.130+0000 7f8e5b7fe640 1 -- 172.21.15.38:0/130663116 >> v2:172.21.15.107:3300/0 conn(0x7f8e5c0cecf0 msgr2=0x7f8e5c0d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:14.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.130+0000 7f8e5b7fe640 1 --2- 172.21.15.38:0/130663116 >> v2:172.21.15.107:3300/0 conn(0x7f8e5c0cecf0 0x7f8e5c0d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.130+0000 7f8e5b7fe640 1 -- 172.21.15.38:0/130663116 >> v1:172.21.15.107:6789/0 conn(0x7f8e5c0b92c0 legacy=0x7f8e5c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:14.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.130+0000 7f8e5b7fe640 1 -- 172.21.15.38:0/130663116 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c0d0a30 con 0x7f8e5c15db70 2024-04-25T08:38:14.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.130+0000 7f8e5bfff640 1 --2- 172.21.15.38:0/130663116 >> v2:172.21.15.107:3300/0 conn(0x7f8e5c0cecf0 0x7f8e5c0d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:14.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.130+0000 7f8e5b7fe640 1 --2- 172.21.15.38:0/130663116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e5c15db70 0x7f8e5c0d00c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0d4910 tx=0x7f8e4c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a501a31389995d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.131+0000 7f8e5a7fc640 1 -- 172.21.15.38:0/130663116 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c037070 con 0x7f8e5c15db70 2024-04-25T08:38:14.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.131+0000 7f8e5a7fc640 1 -- 172.21.15.38:0/130663116 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c032440 con 0x7f8e5c15db70 2024-04-25T08:38:14.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.131+0000 7f8e5a7fc640 1 -- 172.21.15.38:0/130663116 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c032810 con 0x7f8e5c15db70 2024-04-25T08:38:14.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.131+0000 7f8e61f7e640 1 -- 172.21.15.38:0/130663116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e5c15db70 msgr2=0x7f8e5c0d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.131+0000 7f8e61f7e640 1 --2- 172.21.15.38:0/130663116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e5c15db70 0x7f8e5c0d00c0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0d4910 tx=0x7f8e4c02a230 comp rx=0 tx=0).stop 2024-04-25T08:38:14.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.131+0000 7f8e61f7e640 1 -- 172.21.15.38:0/130663116 shutdown_connections 2024-04-25T08:38:14.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.131+0000 7f8e61f7e640 1 --2- 172.21.15.38:0/130663116 >> v2:172.21.15.107:3300/0 conn(0x7f8e5c0cecf0 0x7f8e5c0d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.131+0000 7f8e61f7e640 1 --2- 172.21.15.38:0/130663116 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e5c15db70 0x7f8e5c0d00c0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.131+0000 7f8e61f7e640 1 -- 172.21.15.38:0/130663116 >> 172.21.15.38:0/130663116 conn(0x7f8e5c06bed0 msgr2=0x7f8e5c06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.131+0000 7f8e61f7e640 1 -- 172.21.15.38:0/130663116 shutdown_connections 2024-04-25T08:38:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.131+0000 7f8e61f7e640 1 -- 172.21.15.38:0/130663116 wait complete. 2024-04-25T08:38:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.132+0000 7f8e61f7e640 1 Processor -- start 2024-04-25T08:38:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.133+0000 7f8e61f7e640 1 -- start start 2024-04-25T08:38:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.133+0000 7f8e61f7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e5c0b92c0 0x7f8e5c0c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.133+0000 7f8e61f7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e5c0cecf0 0x7f8e5c0c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.133+0000 7f8e61f7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e5c15db70 0x7f8e5c0b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.133+0000 7f8e61f7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0d0ff0 con 0x7f8e5c15db70 2024-04-25T08:38:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.133+0000 7f8e61f7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b9740 con 0x7f8e5c0cecf0 2024-04-25T08:38:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.133+0000 7f8e61f7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0d26e0 con 0x7f8e5c0b92c0 2024-04-25T08:38:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.133+0000 7f8e5bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e5c15db70 0x7f8e5c0b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.133+0000 7f8e5bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e5c15db70 0x7f8e5c0b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49086/0 (socket says 172.21.15.38:49086) 2024-04-25T08:38:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.133+0000 7f8e5bfff640 1 -- 172.21.15.38:0/185028042 learned_addr learned my addr 172.21.15.38:0/185028042 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:14.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e5bfff640 1 -- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e5c0b92c0 msgr2=0x7f8e5c0c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e5bfff640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e5c0b92c0 0x7f8e5c0c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e5bfff640 1 -- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e5c0cecf0 msgr2=0x7f8e5c0c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e5affd640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e5c0cecf0 0x7f8e5c0c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e5bfff640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e5c0cecf0 0x7f8e5c0c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e5bfff640 1 -- 172.21.15.38:0/185028042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c0c68b0 con 0x7f8e5c15db70 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e5affd640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e5c0cecf0 0x7f8e5c0c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e5bfff640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e5c15db70 0x7f8e5c0b8370 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400ce80 tx=0x7f8e5400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e43fff640 1 -- 172.21.15.38:0/185028042 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5401a070 con 0x7f8e5c15db70 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e43fff640 1 -- 172.21.15.38:0/185028042 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5400d880 con 0x7f8e5c15db70 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e61f7e640 1 -- 172.21.15.38:0/185028042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c1485d0 con 0x7f8e5c15db70 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e43fff640 1 -- 172.21.15.38:0/185028042 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5400dc70 con 0x7f8e5c15db70 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.134+0000 7f8e61f7e640 1 -- 172.21.15.38:0/185028042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c148b90 con 0x7f8e5c15db70 2024-04-25T08:38:14.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.135+0000 7f8e61f7e640 1 -- 172.21.15.38:0/185028042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c0d3f40 con 0x7f8e5c15db70 2024-04-25T08:38:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.136+0000 7f8e43fff640 1 -- 172.21.15.38:0/185028042 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e5401d070 con 0x7f8e5c15db70 2024-04-25T08:38:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.136+0000 7f8e43fff640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8e4806b4d0 0x7f8e4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.136+0000 7f8e43fff640 1 -- 172.21.15.38:0/185028042 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f8e5408ab30 con 0x7f8e5c15db70 2024-04-25T08:38:14.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.137+0000 7f8e5b7fe640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8e4806b4d0 0x7f8e4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.137+0000 7f8e5b7fe640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8e4806b4d0 0x7f8e4806d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c034000 tx=0x7f8e4c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.141+0000 7f8e43fff640 1 -- 172.21.15.38:0/185028042 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e5c0d3f40 con 0x7f8e5c15db70 2024-04-25T08:38:14.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.357+0000 7f8e61f7e640 1 -- 172.21.15.38:0/185028042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8e5c04ef90 con 0x7f8e5c15db70 2024-04-25T08:38:14.351 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:38:14.357+0000 7ff7937f0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:38:14.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.359+0000 7f8e43fff640 1 -- 172.21.15.38:0/185028042 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v52) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8e5c0c68b0 con 0x7f8e5c15db70 2024-04-25T08:38:14.352 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:38:14.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.362+0000 7f8e61f7e640 1 -- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8e4806b4d0 msgr2=0x7f8e4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.362+0000 7f8e61f7e640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8e4806b4d0 0x7f8e4806d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c034000 tx=0x7f8e4c02c000 comp rx=0 tx=0).stop 2024-04-25T08:38:14.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.362+0000 7f8e61f7e640 1 -- 172.21.15.38:0/185028042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e5c15db70 msgr2=0x7f8e5c0b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.362+0000 7f8e61f7e640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e5c15db70 0x7f8e5c0b8370 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400ce80 tx=0x7f8e5400a710 comp rx=0 tx=0).stop 2024-04-25T08:38:14.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.362+0000 7f8e61f7e640 1 -- 172.21.15.38:0/185028042 shutdown_connections 2024-04-25T08:38:14.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.362+0000 7f8e61f7e640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e5c15db70 0x7f8e5c0b8370 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.362+0000 7f8e61f7e640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e5c0cecf0 0x7f8e5c0c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.362+0000 7f8e61f7e640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8e4806b4d0 0x7f8e4806d990 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.362+0000 7f8e61f7e640 1 --2- 172.21.15.38:0/185028042 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e5c0b92c0 0x7f8e5c0c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.362+0000 7f8e61f7e640 1 -- 172.21.15.38:0/185028042 >> 172.21.15.38:0/185028042 conn(0x7f8e5c06bed0 msgr2=0x7f8e5c0d2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:14.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.362+0000 7f8e61f7e640 1 -- 172.21.15.38:0/185028042 shutdown_connections 2024-04-25T08:38:14.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.362+0000 7f8e61f7e640 1 -- 172.21.15.38:0/185028042 wait complete. 2024-04-25T08:38:14.368 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:38:14.394 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:38:14.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.478+0000 7fd9ceb09640 1 Processor -- start 2024-04-25T08:38:14.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.478+0000 7fd9ceb09640 1 -- start start 2024-04-25T08:38:14.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9ceb09640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd9c8166610 0x7fd9c8168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9ceb09640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c80d2060 con 0x7fd9c80d0320 2024-04-25T08:38:14.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9ceb09640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c80d2330 con 0x7fd9c80d33d0 2024-04-25T08:38:14.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9ceb09640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c80d2610 con 0x7fd9c8166610 2024-04-25T08:38:14.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9cc8a5640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd9c80d0320 0x7fd9c80d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35428/0 (socket says 172.21.15.38:35428) 2024-04-25T08:38:14.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9cc8a5640 1 -- 172.21.15.38:0/3292748574 learned_addr learned my addr 172.21.15.38:0/3292748574 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:14.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9cd0a6640 1 --2- 172.21.15.38:0/3292748574 >> v2:172.21.15.107:3300/0 conn(0x7fd9c8166610 0x7fd9c8168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1902041488 0 0) 0x7fd9c80d2060 con 0x7fd9c80d0320 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9b0003520 con 0x7fd9c80d0320 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3044737319 0 0) 0x7fd9b0003520 con 0x7fd9c80d0320 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 >> v2:172.21.15.107:3300/0 conn(0x7fd9c8166610 msgr2=0x7fd9c8168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 --2- 172.21.15.38:0/3292748574 >> v2:172.21.15.107:3300/0 conn(0x7fd9c8166610 0x7fd9c8168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 >> v1:172.21.15.107:6789/0 conn(0x7fd9c80d33d0 legacy=0x7fd9c80d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c80d28f0 con 0x7fd9c80d0320 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fd9c4001670 con 0x7fd9c80d0320 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 >> v1:172.21.15.38:6789/0 conn(0x7fd9c80d0320 legacy=0x7fd9c80d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 --2- 172.21.15.38:0/3292748574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 0x7fd9b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 --2- 172.21.15.38:0/3292748574 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9b00043b0 0x7fd9b0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 --2- 172.21.15.38:0/3292748574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9b0005080 0x7fd9b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80d2610 con 0x7fd9b00039b0 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b000bf30 con 0x7fd9b0005080 2024-04-25T08:38:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.479+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b000c130 con 0x7fd9b00043b0 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.480+0000 7fd9cc8a5640 1 --2- 172.21.15.38:0/3292748574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 0x7fd9b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.480+0000 7fd9cd0a6640 1 --2- 172.21.15.38:0/3292748574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9b0005080 0x7fd9b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.480+0000 7fd9bffff640 1 --2- 172.21.15.38:0/3292748574 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9b00043b0 0x7fd9b0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.480+0000 7fd9cc8a5640 1 -- 172.21.15.38:0/3292748574 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9b00043b0 msgr2=0x7fd9b0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.480+0000 7fd9cc8a5640 1 --2- 172.21.15.38:0/3292748574 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9b00043b0 0x7fd9b0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.480+0000 7fd9cc8a5640 1 -- 172.21.15.38:0/3292748574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9b0005080 msgr2=0x7fd9b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.480+0000 7fd9cc8a5640 1 --2- 172.21.15.38:0/3292748574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9b0005080 0x7fd9b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.480+0000 7fd9cc8a5640 1 -- 172.21.15.38:0/3292748574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b0003520 con 0x7fd9b00039b0 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.480+0000 7fd9cd0a6640 1 --2- 172.21.15.38:0/3292748574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9b0005080 0x7fd9b000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.480+0000 7fd9cd0a6640 1 -- 172.21.15.38:0/3292748574 reap_dead start 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.480+0000 7fd9cc8a5640 1 --2- 172.21.15.38:0/3292748574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 0x7fd9b0003d90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c210 tx=0x7fd9b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05ef876549bdbfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.481+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4007ce0 con 0x7fd9b00039b0 2024-04-25T08:38:14.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.481+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c4005af0 con 0x7fd9b00039b0 2024-04-25T08:38:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.481+0000 7fd9bf7fe640 1 -- 172.21.15.38:0/3292748574 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4007070 con 0x7fd9b00039b0 2024-04-25T08:38:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.481+0000 7fd9ceb09640 1 -- 172.21.15.38:0/3292748574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 msgr2=0x7fd9b0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.481+0000 7fd9ceb09640 1 --2- 172.21.15.38:0/3292748574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 0x7fd9b0003d90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400c210 tx=0x7fd9b0005830 comp rx=0 tx=0).stop 2024-04-25T08:38:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.481+0000 7fd9ceb09640 1 -- 172.21.15.38:0/3292748574 shutdown_connections 2024-04-25T08:38:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.481+0000 7fd9ceb09640 1 --2- 172.21.15.38:0/3292748574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 0x7fd9b0003d90 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.481+0000 7fd9ceb09640 1 -- 172.21.15.38:0/3292748574 >> 172.21.15.38:0/3292748574 conn(0x7fd9c80668f0 msgr2=0x7fd9c8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.481+0000 7fd9ceb09640 1 -- 172.21.15.38:0/3292748574 shutdown_connections 2024-04-25T08:38:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.481+0000 7fd9ceb09640 1 -- 172.21.15.38:0/3292748574 wait complete. 2024-04-25T08:38:14.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.482+0000 7fd9ceb09640 1 Processor -- start 2024-04-25T08:38:14.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.482+0000 7fd9ceb09640 1 -- start start 2024-04-25T08:38:14.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.483+0000 7fd9ceb09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 0x7fd9c80b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.483+0000 7fd9ceb09640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9c81536c0 0x7fd9c8152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.483+0000 7fd9ceb09640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9c80ccfc0 0x7fd9c80cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.483+0000 7fd9ceb09640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c8161210 con 0x7fd9b00039b0 2024-04-25T08:38:14.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.483+0000 7fd9ceb09640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c80d2eb0 con 0x7fd9c80ccfc0 2024-04-25T08:38:14.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.483+0000 7fd9ceb09640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c8161390 con 0x7fd9c81536c0 2024-04-25T08:38:14.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.483+0000 7fd9cc8a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 0x7fd9c80b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.483+0000 7fd9cc8a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 0x7fd9c80b45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49098/0 (socket says 172.21.15.38:49098) 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.483+0000 7fd9cc8a5640 1 -- 172.21.15.38:0/282975436 learned_addr learned my addr 172.21.15.38:0/282975436 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.483+0000 7fd9cd0a6640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9c80ccfc0 0x7fd9c80cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.483+0000 7fd9bffff640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9c81536c0 0x7fd9c8152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.484+0000 7fd9cc8a5640 1 -- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9c81536c0 msgr2=0x7fd9c8152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.484+0000 7fd9cc8a5640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9c81536c0 0x7fd9c8152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.484+0000 7fd9cc8a5640 1 -- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9c80ccfc0 msgr2=0x7fd9c80cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.484+0000 7fd9cc8a5640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9c80ccfc0 0x7fd9c80cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.484+0000 7fd9cc8a5640 1 -- 172.21.15.38:0/282975436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c80cc110 con 0x7fd9b00039b0 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.484+0000 7fd9bffff640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9c81536c0 0x7fd9c8152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.484+0000 7fd9cc8a5640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 0x7fd9c80b45b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40018f0 tx=0x7fd9c80d2500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.484+0000 7fd9bd7fa640 1 -- 172.21.15.38:0/282975436 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4037040 con 0x7fd9b00039b0 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.484+0000 7fd9bd7fa640 1 -- 172.21.15.38:0/282975436 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c40047e0 con 0x7fd9b00039b0 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.484+0000 7fd9ceb09640 1 -- 172.21.15.38:0/282975436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c80cc3f0 con 0x7fd9b00039b0 2024-04-25T08:38:14.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.484+0000 7fd9bd7fa640 1 -- 172.21.15.38:0/282975436 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4007ce0 con 0x7fd9b00039b0 2024-04-25T08:38:14.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.484+0000 7fd9ceb09640 1 -- 172.21.15.38:0/282975436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c81533e0 con 0x7fd9b00039b0 2024-04-25T08:38:14.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.485+0000 7fd9ceb09640 1 -- 172.21.15.38:0/282975436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c8068e80 con 0x7fd9b00039b0 2024-04-25T08:38:14.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.485+0000 7fd9bd7fa640 1 -- 172.21.15.38:0/282975436 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9c4004bd0 con 0x7fd9b00039b0 2024-04-25T08:38:14.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.486+0000 7fd9bd7fa640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd9b406b4d0 0x7fd9b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:14.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.486+0000 7fd9bd7fa640 1 -- 172.21.15.38:0/282975436 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd9c40aa0f0 con 0x7fd9b00039b0 2024-04-25T08:38:14.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.486+0000 7fd9bffff640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd9b406b4d0 0x7fd9b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:14.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.487+0000 7fd9bffff640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd9b406b4d0 0x7fd9b406d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8007fd0 tx=0x7fd9b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:14.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.490+0000 7fd9bd7fa640 1 -- 172.21.15.38:0/282975436 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9c4078de0 con 0x7fd9b00039b0 2024-04-25T08:38:14.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:14.709+0000 7fd9ceb09640 1 -- 172.21.15.38:0/282975436 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd9c80b3050 con 0x7fd9b00039b0 2024-04-25T08:38:15.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.362+0000 7fd9bd7fa640 1 -- 172.21.15.38:0/282975436 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v53) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd9c40786e0 con 0x7fd9b00039b0 2024-04-25T08:38:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.365+0000 7fd9ceb09640 1 -- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd9b406b4d0 msgr2=0x7fd9b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.365+0000 7fd9ceb09640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd9b406b4d0 0x7fd9b406d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8007fd0 tx=0x7fd9b8009040 comp rx=0 tx=0).stop 2024-04-25T08:38:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.365+0000 7fd9ceb09640 1 -- 172.21.15.38:0/282975436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 msgr2=0x7fd9c80b45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.365+0000 7fd9ceb09640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 0x7fd9c80b45b0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40018f0 tx=0x7fd9c80d2500 comp rx=0 tx=0).stop 2024-04-25T08:38:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.365+0000 7fd9ceb09640 1 -- 172.21.15.38:0/282975436 shutdown_connections 2024-04-25T08:38:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.365+0000 7fd9ceb09640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9c80ccfc0 0x7fd9c80cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.365+0000 7fd9ceb09640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd9b406b4d0 0x7fd9b406d990 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.365+0000 7fd9ceb09640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9c81536c0 0x7fd9c8152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.365+0000 7fd9ceb09640 1 --2- 172.21.15.38:0/282975436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9b00039b0 0x7fd9c80b45b0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.365+0000 7fd9ceb09640 1 -- 172.21.15.38:0/282975436 >> 172.21.15.38:0/282975436 conn(0x7fd9c80668f0 msgr2=0x7fd9c80d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.365+0000 7fd9ceb09640 1 -- 172.21.15.38:0/282975436 shutdown_connections 2024-04-25T08:38:15.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.365+0000 7fd9ceb09640 1 -- 172.21.15.38:0/282975436 wait complete. 2024-04-25T08:38:15.373 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:38:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.534+0000 7f8bfe658640 1 Processor -- start 2024-04-25T08:38:15.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.534+0000 7f8bfe658640 1 -- start start 2024-04-25T08:38:15.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.535+0000 7f8bfe658640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf80d0870 0x7f8bf80d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.535+0000 7f8bfe658640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf80d11c0 con 0x7f8bf80b93f0 2024-04-25T08:38:15.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.535+0000 7f8bfe658640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf80b9880 con 0x7f8bf81612c0 2024-04-25T08:38:15.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.535+0000 7f8bfe658640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80b7200 con 0x7f8bf80d0870 2024-04-25T08:38:15.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.535+0000 7f8bf77fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f8bf80b93f0 0x7f8bf80d1c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35442/0 (socket says 172.21.15.38:35442) 2024-04-25T08:38:15.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.535+0000 7f8bf77fe640 1 -- 172.21.15.38:0/200364398 learned_addr learned my addr 172.21.15.38:0/200364398 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:15.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.535+0000 7f8bfcbf5640 1 --2- 172.21.15.38:0/200364398 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf80d0870 0x7f8bf80d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4191485338 0 0) 0x7f8bec001110 con 0x7f8bf80b93f0 2024-04-25T08:38:15.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8be0003520 con 0x7f8bf80b93f0 2024-04-25T08:38:15.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1815152511 0 0) 0x7f8bf80b9880 con 0x7f8bf81612c0 2024-04-25T08:38:15.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bec001110 con 0x7f8bf81612c0 2024-04-25T08:38:15.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2742022201 0 0) 0x7f8be0003520 con 0x7f8bf80b93f0 2024-04-25T08:38:15.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf80d0870 msgr2=0x7f8bf80d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:15.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 --2- 172.21.15.38:0/200364398 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf80d0870 0x7f8bf80d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 >> v1:172.21.15.107:6789/0 conn(0x7f8bf81612c0 legacy=0x7f8bf80b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf80b74e0 con 0x7f8bf80b93f0 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f8bec001840 con 0x7f8bf80b93f0 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 >> v1:172.21.15.38:6789/0 conn(0x7f8bf80b93f0 legacy=0x7f8bf80d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 --2- 172.21.15.38:0/200364398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8be0003b70 0x7f8be0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 --2- 172.21.15.38:0/200364398 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf80d0870 0x7f8be0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 --2- 172.21.15.38:0/200364398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8be0004d40 0x7f8be00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80b7200 con 0x7f8be0004d40 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be000bb50 con 0x7f8be0003b70 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be000bd50 con 0x7f8bf80d0870 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bfcbf5640 1 --2- 172.21.15.38:0/200364398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8be0004d40 0x7f8be00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf77fe640 1 --2- 172.21.15.38:0/200364398 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf80d0870 0x7f8be0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bf7fff640 1 --2- 172.21.15.38:0/200364398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8be0003b70 0x7f8be0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bfcbf5640 1 -- 172.21.15.38:0/200364398 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf80d0870 msgr2=0x7f8be0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bfcbf5640 1 --2- 172.21.15.38:0/200364398 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf80d0870 0x7f8be0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bfcbf5640 1 -- 172.21.15.38:0/200364398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8be0003b70 msgr2=0x7f8be0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.536+0000 7f8bfcbf5640 1 --2- 172.21.15.38:0/200364398 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8be0003b70 0x7f8be0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.537+0000 7f8bfcbf5640 1 -- 172.21.15.38:0/200364398 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be0003520 con 0x7f8be0004d40 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.537+0000 7f8bfcbf5640 1 --2- 172.21.15.38:0/200364398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8be0004d40 0x7f8be00051c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009fc0 tx=0x7f8be0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd56232aeee88e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.537+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0009c10 con 0x7f8be0004d40 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.537+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf0036070 con 0x7f8be0004d40 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.537+0000 7f8bf6ffd640 1 -- 172.21.15.38:0/200364398 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0031880 con 0x7f8be0004d40 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.538+0000 7f8bfe658640 1 -- 172.21.15.38:0/200364398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8be0004d40 msgr2=0x7f8be00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.538+0000 7f8bfe658640 1 --2- 172.21.15.38:0/200364398 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8be0004d40 0x7f8be00051c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0009fc0 tx=0x7f8be0005830 comp rx=0 tx=0).stop 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.538+0000 7f8bfcbf5640 1 -- 172.21.15.38:0/200364398 reap_dead start 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.538+0000 7f8bfe658640 1 -- 172.21.15.38:0/200364398 shutdown_connections 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.538+0000 7f8bfe658640 1 -- 172.21.15.38:0/200364398 >> 172.21.15.38:0/200364398 conn(0x7f8bf8066d10 msgr2=0x7f8bf8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.538+0000 7f8bfe658640 1 -- 172.21.15.38:0/200364398 shutdown_connections 2024-04-25T08:38:15.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.538+0000 7f8bfe658640 1 -- 172.21.15.38:0/200364398 wait complete. 2024-04-25T08:38:15.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.539+0000 7f8bfe658640 1 Processor -- start 2024-04-25T08:38:15.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.539+0000 7f8bfe658640 1 -- start start 2024-04-25T08:38:15.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.540+0000 7f8bfe658640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8be0004d40 0x7f8bf815beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.540+0000 7f8bfe658640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf815dc40 0x7f8bf80ac650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.540+0000 7f8bfe658640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf815fbd0 0x7f8bf8160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.540+0000 7f8bfe658640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80b7d80 con 0x7f8be0004d40 2024-04-25T08:38:15.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.540+0000 7f8bfe658640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80b7aa0 con 0x7f8bf815fbd0 2024-04-25T08:38:15.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.540+0000 7f8bfe658640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf80d0c80 con 0x7f8bf815dc40 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.540+0000 7f8bf7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8be0004d40 0x7f8bf815beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.540+0000 7f8bf7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8be0004d40 0x7f8bf815beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49116/0 (socket says 172.21.15.38:49116) 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.540+0000 7f8bf7fff640 1 -- 172.21.15.38:0/3713379904 learned_addr learned my addr 172.21.15.38:0/3713379904 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bf7fff640 1 -- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf815dc40 msgr2=0x7f8bf80ac650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bf77fe640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf815dc40 0x7f8bf80ac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bf7fff640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf815dc40 0x7f8bf80ac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bfcbf5640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf815fbd0 0x7f8bf8160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bf7fff640 1 -- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf815fbd0 msgr2=0x7f8bf8160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bf7fff640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf815fbd0 0x7f8bf8160050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bf7fff640 1 -- 172.21.15.38:0/3713379904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf815c3f0 con 0x7f8be0004d40 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bf7fff640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8be0004d40 0x7f8bf815beb0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f8be800f770 tx=0x7f8be800fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bf4ff9640 1 -- 172.21.15.38:0/3713379904 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be801a070 con 0x7f8be0004d40 2024-04-25T08:38:15.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bf4ff9640 1 -- 172.21.15.38:0/3713379904 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be800fe80 con 0x7f8be0004d40 2024-04-25T08:38:15.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bf4ff9640 1 -- 172.21.15.38:0/3713379904 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be8007050 con 0x7f8be0004d40 2024-04-25T08:38:15.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bfe658640 1 -- 172.21.15.38:0/3713379904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf815e200 con 0x7f8be0004d40 2024-04-25T08:38:15.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.541+0000 7f8bfe658640 1 -- 172.21.15.38:0/3713379904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf815e7c0 con 0x7f8be0004d40 2024-04-25T08:38:15.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.542+0000 7f8bfe658640 1 -- 172.21.15.38:0/3713379904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf8068e80 con 0x7f8be0004d40 2024-04-25T08:38:15.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.543+0000 7f8bf4ff9640 1 -- 172.21.15.38:0/3713379904 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8be801d070 con 0x7f8be0004d40 2024-04-25T08:38:15.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.543+0000 7f8bf4ff9640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8be406b4d0 0x7f8be406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:15.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.543+0000 7f8bf4ff9640 1 -- 172.21.15.38:0/3713379904 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8be8089c00 con 0x7f8be0004d40 2024-04-25T08:38:15.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.543+0000 7f8bf77fe640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8be406b4d0 0x7f8be406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:15.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.544+0000 7f8bf77fe640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8be406b4d0 0x7f8be406d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80ab460 tx=0x7f8bec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:15.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.547+0000 7f8bf4ff9640 1 -- 172.21.15.38:0/3713379904 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8be8090080 con 0x7f8be0004d40 2024-04-25T08:38:15.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:15.727+0000 7f8bfe658640 1 -- 172.21.15.38:0/3713379904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8bf80672c0 con 0x7f8be0004d40 2024-04-25T08:38:16.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.363+0000 7f8bf4ff9640 1 -- 172.21.15.38:0/3713379904 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8be8058930 con 0x7f8be0004d40 2024-04-25T08:38:16.357 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:38:16.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.366+0000 7f8bfe658640 1 -- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8be406b4d0 msgr2=0x7f8be406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.366+0000 7f8bfe658640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8be406b4d0 0x7f8be406d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80ab460 tx=0x7f8bec009040 comp rx=0 tx=0).stop 2024-04-25T08:38:16.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.366+0000 7f8bfe658640 1 -- 172.21.15.38:0/3713379904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8be0004d40 msgr2=0x7f8bf815beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.366+0000 7f8bfe658640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8be0004d40 0x7f8bf815beb0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f8be800f770 tx=0x7f8be800fc30 comp rx=0 tx=0).stop 2024-04-25T08:38:16.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.366+0000 7f8bfe658640 1 -- 172.21.15.38:0/3713379904 shutdown_connections 2024-04-25T08:38:16.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.366+0000 7f8bfe658640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf815fbd0 0x7f8bf8160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.366+0000 7f8bfe658640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8be406b4d0 0x7f8be406d990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.366+0000 7f8bfe658640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf815dc40 0x7f8bf80ac650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.366+0000 7f8bfe658640 1 --2- 172.21.15.38:0/3713379904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8be0004d40 0x7f8bf815beb0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.366+0000 7f8bfe658640 1 -- 172.21.15.38:0/3713379904 >> 172.21.15.38:0/3713379904 conn(0x7f8bf8066d10 msgr2=0x7f8bf80c9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:16.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.366+0000 7f8bfe658640 1 -- 172.21.15.38:0/3713379904 shutdown_connections 2024-04-25T08:38:16.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.366+0000 7f8bfe658640 1 -- 172.21.15.38:0/3713379904 wait complete. 2024-04-25T08:38:16.374 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:38:16.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.491+0000 7fa164996640 1 Processor -- start 2024-04-25T08:38:16.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.491+0000 7fa164996640 1 -- start start 2024-04-25T08:38:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.492+0000 7fa164996640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa16015de20 0x7fa16006c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:16.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.492+0000 7fa164996640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1600cfe10 con 0x7fa1600cf970 2024-04-25T08:38:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.492+0000 7fa164996640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1600d00f0 con 0x7fa160099a70 2024-04-25T08:38:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.492+0000 7fa164996640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16006c560 con 0x7fa16015de20 2024-04-25T08:38:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.493+0000 7fa15e59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa16015de20 0x7fa16006c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.493+0000 7fa15e59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa16015de20 0x7fa16006c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49132/0 (socket says 172.21.15.38:49132) 2024-04-25T08:38:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.493+0000 7fa15dd9b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa1600cf970 0x7fa1600d0eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35446/0 (socket says 172.21.15.38:35446) 2024-04-25T08:38:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.493+0000 7fa15e59c640 1 -- 172.21.15.38:0/677090461 learned_addr learned my addr 172.21.15.38:0/677090461 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.493+0000 7fa15e59c640 1 -- 172.21.15.38:0/677090461 >> v1:172.21.15.107:6789/0 conn(0x7fa160099a70 legacy=0x7fa1600cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.493+0000 7fa15e59c640 1 -- 172.21.15.38:0/677090461 >> v1:172.21.15.38:6789/0 conn(0x7fa1600cf970 legacy=0x7fa1600d0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.493+0000 7fa15e59c640 1 -- 172.21.15.38:0/677090461 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1600d03d0 con 0x7fa16015de20 2024-04-25T08:38:16.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.494+0000 7fa15e59c640 1 --2- 172.21.15.38:0/677090461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa16015de20 0x7fa16006c020 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa150009870 tx=0x7fa15002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31bf8a82653f878d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:16.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.494+0000 7fa15d59a640 1 -- 172.21.15.38:0/677090461 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150037070 con 0x7fa16015de20 2024-04-25T08:38:16.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.494+0000 7fa15d59a640 1 -- 172.21.15.38:0/677090461 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa150032440 con 0x7fa16015de20 2024-04-25T08:38:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.494+0000 7fa15d59a640 1 -- 172.21.15.38:0/677090461 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa150032810 con 0x7fa16015de20 2024-04-25T08:38:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.494+0000 7fa164996640 1 -- 172.21.15.38:0/677090461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa16015de20 msgr2=0x7fa16006c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.494+0000 7fa164996640 1 --2- 172.21.15.38:0/677090461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa16015de20 0x7fa16006c020 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa150009870 tx=0x7fa15002a270 comp rx=0 tx=0).stop 2024-04-25T08:38:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.494+0000 7fa164996640 1 -- 172.21.15.38:0/677090461 shutdown_connections 2024-04-25T08:38:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.494+0000 7fa164996640 1 --2- 172.21.15.38:0/677090461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa16015de20 0x7fa16006c020 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.494+0000 7fa164996640 1 -- 172.21.15.38:0/677090461 >> 172.21.15.38:0/677090461 conn(0x7fa1600b3190 msgr2=0x7fa1600b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.495+0000 7fa164996640 1 -- 172.21.15.38:0/677090461 shutdown_connections 2024-04-25T08:38:16.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.495+0000 7fa164996640 1 -- 172.21.15.38:0/677090461 wait complete. 2024-04-25T08:38:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.496+0000 7fa164996640 1 Processor -- start 2024-04-25T08:38:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.496+0000 7fa164996640 1 -- start start 2024-04-25T08:38:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa164996640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa160099a70 0x7fa160155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:16.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa164996640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1600cf970 0x7fa1600c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa164996640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa16015de20 0x7fa1600c5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa164996640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1600b0a60 con 0x7fa16015de20 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa164996640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1600d0a20 con 0x7fa160099a70 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa164996640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1600c5c50 con 0x7fa1600cf970 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa15ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa16015de20 0x7fa1600c5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa15ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa16015de20 0x7fa1600c5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49142/0 (socket says 172.21.15.38:49142) 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa15ed9d640 1 -- 172.21.15.38:0/985969918 learned_addr learned my addr 172.21.15.38:0/985969918 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa15e59c640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa160099a70 0x7fa160155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa15dd9b640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1600cf970 0x7fa1600c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa15ed9d640 1 -- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1600cf970 msgr2=0x7fa1600c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa15ed9d640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1600cf970 0x7fa1600c51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa15ed9d640 1 -- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa160099a70 msgr2=0x7fa160155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa15ed9d640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa160099a70 0x7fa160155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa15ed9d640 1 -- 172.21.15.38:0/985969918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1600c5ea0 con 0x7fa16015de20 2024-04-25T08:38:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa15ed9d640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa16015de20 0x7fa1600c5710 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c6b0 tx=0x7fa15800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa14effd640 1 -- 172.21.15.38:0/985969918 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158003b70 con 0x7fa16015de20 2024-04-25T08:38:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa14effd640 1 -- 172.21.15.38:0/985969918 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa158003d30 con 0x7fa16015de20 2024-04-25T08:38:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa164996640 1 -- 172.21.15.38:0/985969918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1600c6180 con 0x7fa16015de20 2024-04-25T08:38:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa14effd640 1 -- 172.21.15.38:0/985969918 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158005310 con 0x7fa16015de20 2024-04-25T08:38:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.497+0000 7fa164996640 1 -- 172.21.15.38:0/985969918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1600c7270 con 0x7fa16015de20 2024-04-25T08:38:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.498+0000 7fa164996640 1 -- 172.21.15.38:0/985969918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa160068e80 con 0x7fa16015de20 2024-04-25T08:38:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.499+0000 7fa14effd640 1 -- 172.21.15.38:0/985969918 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1580054b0 con 0x7fa16015de20 2024-04-25T08:38:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.499+0000 7fa14effd640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa14006b4d0 0x7fa14006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.499+0000 7fa14effd640 1 -- 172.21.15.38:0/985969918 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fa158089ef0 con 0x7fa16015de20 2024-04-25T08:38:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.500+0000 7fa15e59c640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa14006b4d0 0x7fa14006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:16.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.500+0000 7fa15e59c640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa14006b4d0 0x7fa14006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa150034000 tx=0x7fa15002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.504+0000 7fa14effd640 1 -- 172.21.15.38:0/985969918 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa158058780 con 0x7fa16015de20 2024-04-25T08:38:16.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:16.690+0000 7fa164996640 1 -- 172.21.15.38:0/985969918 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa1600b3f10 con 0x7fa16015de20 2024-04-25T08:38:17.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.365+0000 7fa14effd640 1 -- 172.21.15.38:0/985969918 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa15805c820 con 0x7fa16015de20 2024-04-25T08:38:17.359 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-04-25T08:38:17.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.368+0000 7fa164996640 1 -- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa14006b4d0 msgr2=0x7fa14006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.368+0000 7fa164996640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa14006b4d0 0x7fa14006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa150034000 tx=0x7fa15002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:17.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.368+0000 7fa164996640 1 -- 172.21.15.38:0/985969918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa16015de20 msgr2=0x7fa1600c5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.368+0000 7fa164996640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa16015de20 0x7fa1600c5710 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c6b0 tx=0x7fa15800cb70 comp rx=0 tx=0).stop 2024-04-25T08:38:17.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.368+0000 7fa164996640 1 -- 172.21.15.38:0/985969918 shutdown_connections 2024-04-25T08:38:17.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.368+0000 7fa164996640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa16015de20 0x7fa1600c5710 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.368+0000 7fa164996640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa14006b4d0 0x7fa14006d990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.368+0000 7fa164996640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1600cf970 0x7fa1600c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.368+0000 7fa164996640 1 --2- 172.21.15.38:0/985969918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa160099a70 0x7fa160155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.368+0000 7fa164996640 1 -- 172.21.15.38:0/985969918 >> 172.21.15.38:0/985969918 conn(0x7fa1600b3190 msgr2=0x7fa1600cdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:17.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.368+0000 7fa164996640 1 -- 172.21.15.38:0/985969918 shutdown_connections 2024-04-25T08:38:17.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.368+0000 7fa164996640 1 -- 172.21.15.38:0/985969918 wait complete. 2024-04-25T08:38:17.376 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:38:17.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.492+0000 7fef81e5d640 1 Processor -- start 2024-04-25T08:38:17.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.492+0000 7fef81e5d640 1 -- start start 2024-04-25T08:38:17.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef81e5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef7c0d0320 0x7fef7c0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:17.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef81e5d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fef7c0d0700 0x7fef7c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:17.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef81e5d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef7c0d2330 con 0x7fef7c0d1e90 2024-04-25T08:38:17.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef81e5d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef7c0d2610 con 0x7fef7c0d0700 2024-04-25T08:38:17.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef81e5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0d28f0 con 0x7fef7c0d0320 2024-04-25T08:38:17.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef7affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fef7c0d0700 0x7fef7c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:17.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef7affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fef7c0d0700 0x7fef7c0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49158/0 (socket says 172.21.15.38:49158) 2024-04-25T08:38:17.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef7affd640 1 -- 172.21.15.38:0/1193524118 learned_addr learned my addr 172.21.15.38:0/1193524118 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:17.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef7affd640 1 -- 172.21.15.38:0/1193524118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef7c0d0320 msgr2=0x7fef7c0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:17.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef7b7fe640 1 --2- 172.21.15.38:0/1193524118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef7c0d0320 0x7fef7c0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:17.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef7affd640 1 --2- 172.21.15.38:0/1193524118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef7c0d0320 0x7fef7c0d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef7affd640 1 -- 172.21.15.38:0/1193524118 >> v1:172.21.15.107:6789/0 conn(0x7fef7c0d1e90 legacy=0x7fef7c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:17.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.493+0000 7fef7affd640 1 -- 172.21.15.38:0/1193524118 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c0d2bd0 con 0x7fef7c0d0700 2024-04-25T08:38:17.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.494+0000 7fef7affd640 1 --2- 172.21.15.38:0/1193524118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef7c0d0700 0x7fef7c0d1950 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009870 tx=0x7fef6c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af313628d305573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:17.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.494+0000 7fef7a7fc640 1 -- 172.21.15.38:0/1193524118 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c037070 con 0x7fef7c0d0700 2024-04-25T08:38:17.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.494+0000 7fef7a7fc640 1 -- 172.21.15.38:0/1193524118 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c032440 con 0x7fef7c0d0700 2024-04-25T08:38:17.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.494+0000 7fef7a7fc640 1 -- 172.21.15.38:0/1193524118 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c032810 con 0x7fef7c0d0700 2024-04-25T08:38:17.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.495+0000 7fef81e5d640 1 -- 172.21.15.38:0/1193524118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef7c0d0700 msgr2=0x7fef7c0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.495+0000 7fef81e5d640 1 --2- 172.21.15.38:0/1193524118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef7c0d0700 0x7fef7c0d1950 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009870 tx=0x7fef6c02a2a0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.495+0000 7fef81e5d640 1 -- 172.21.15.38:0/1193524118 shutdown_connections 2024-04-25T08:38:17.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.495+0000 7fef81e5d640 1 --2- 172.21.15.38:0/1193524118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef7c0d0700 0x7fef7c0d1950 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.495+0000 7fef81e5d640 1 --2- 172.21.15.38:0/1193524118 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef7c0d0320 0x7fef7c0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.495+0000 7fef81e5d640 1 -- 172.21.15.38:0/1193524118 >> 172.21.15.38:0/1193524118 conn(0x7fef7c0668f0 msgr2=0x7fef7c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:17.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.495+0000 7fef81e5d640 1 -- 172.21.15.38:0/1193524118 shutdown_connections 2024-04-25T08:38:17.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.495+0000 7fef81e5d640 1 -- 172.21.15.38:0/1193524118 wait complete. 2024-04-25T08:38:17.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.496+0000 7fef81e5d640 1 Processor -- start 2024-04-25T08:38:17.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.496+0000 7fef81e5d640 1 -- start start 2024-04-25T08:38:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef81e5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef7c0d0320 0x7fef7c15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef81e5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef7c0d0700 0x7fef7c1f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef81e5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef7c0d1e90 0x7fef7c1f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef81e5d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0d0b40 con 0x7fef7c0d0320 2024-04-25T08:38:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef81e5d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c161210 con 0x7fef7c0d0700 2024-04-25T08:38:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef81e5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7c0b53f0 con 0x7fef7c0d1e90 2024-04-25T08:38:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef7b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef7c0d0320 0x7fef7c15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef7b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef7c0d0320 0x7fef7c15abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49164/0 (socket says 172.21.15.38:49164) 2024-04-25T08:38:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef7b7fe640 1 -- 172.21.15.38:0/3043573411 learned_addr learned my addr 172.21.15.38:0/3043573411 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef7bfff640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef7c0d1e90 0x7fef7c1f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef7affd640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef7c0d0700 0x7fef7c1f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:17.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef7b7fe640 1 -- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef7c0d1e90 msgr2=0x7fef7c1f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef7b7fe640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef7c0d1e90 0x7fef7c1f4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef7b7fe640 1 -- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef7c0d0700 msgr2=0x7fef7c1f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef7b7fe640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef7c0d0700 0x7fef7c1f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef7b7fe640 1 -- 172.21.15.38:0/3043573411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7c0b19b0 con 0x7fef7c0d0320 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.497+0000 7fef7bfff640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef7c0d1e90 0x7fef7c1f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.498+0000 7fef7b7fe640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef7c0d0320 0x7fef7c15abf0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fef7000c7f0 tx=0x7fef7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.498+0000 7fef63fff640 1 -- 172.21.15.38:0/3043573411 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70015070 con 0x7fef7c0d0320 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.498+0000 7fef63fff640 1 -- 172.21.15.38:0/3043573411 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef70004240 con 0x7fef7c0d0320 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.498+0000 7fef63fff640 1 -- 172.21.15.38:0/3043573411 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef70007620 con 0x7fef7c0d0320 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.498+0000 7fef81e5d640 1 -- 172.21.15.38:0/3043573411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7c1e63d0 con 0x7fef7c0d0320 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.498+0000 7fef81e5d640 1 -- 172.21.15.38:0/3043573411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7c1e6670 con 0x7fef7c0d0320 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.499+0000 7fef81e5d640 1 -- 172.21.15.38:0/3043573411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7c068e80 con 0x7fef7c0d0320 2024-04-25T08:38:17.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.499+0000 7fef63fff640 1 -- 172.21.15.38:0/3043573411 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef700079d0 con 0x7fef7c0d0320 2024-04-25T08:38:17.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.500+0000 7fef63fff640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fef6806b4d0 0x7fef6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:17.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.500+0000 7fef63fff640 1 -- 172.21.15.38:0/3043573411 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fef7008a4a0 con 0x7fef7c0d0320 2024-04-25T08:38:17.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.500+0000 7fef7affd640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fef6806b4d0 0x7fef6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:17.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.501+0000 7fef7affd640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fef6806b4d0 0x7fef6806d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fef6c034000 tx=0x7fef6c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:17.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.504+0000 7fef63fff640 1 -- 172.21.15.38:0/3043573411 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef70059490 con 0x7fef7c0d0320 2024-04-25T08:38:17.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.685+0000 7fef81e5d640 1 -- 172.21.15.38:0/3043573411 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fef7c0b3050 con 0x7fef7c0d0320 2024-04-25T08:38:17.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.693+0000 7fef63fff640 1 -- 172.21.15.38:0/3043573411 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fef70058d90 con 0x7fef7c0d0320 2024-04-25T08:38:17.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.695+0000 7fef81e5d640 1 -- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fef6806b4d0 msgr2=0x7fef6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.695+0000 7fef81e5d640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fef6806b4d0 0x7fef6806d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fef6c034000 tx=0x7fef6c02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:17.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.695+0000 7fef81e5d640 1 -- 172.21.15.38:0/3043573411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef7c0d0320 msgr2=0x7fef7c15abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.695+0000 7fef81e5d640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef7c0d0320 0x7fef7c15abf0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fef7000c7f0 tx=0x7fef7000ccb0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.695+0000 7fef81e5d640 1 -- 172.21.15.38:0/3043573411 shutdown_connections 2024-04-25T08:38:17.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.695+0000 7fef81e5d640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fef6806b4d0 0x7fef6806d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.695+0000 7fef81e5d640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fef7c0d1e90 0x7fef7c1f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.695+0000 7fef81e5d640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef7c0d0700 0x7fef7c1f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.695+0000 7fef81e5d640 1 --2- 172.21.15.38:0/3043573411 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fef7c0d0320 0x7fef7c15abf0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.695+0000 7fef81e5d640 1 -- 172.21.15.38:0/3043573411 >> 172.21.15.38:0/3043573411 conn(0x7fef7c0668f0 msgr2=0x7fef7c15ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:17.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.695+0000 7fef81e5d640 1 -- 172.21.15.38:0/3043573411 shutdown_connections 2024-04-25T08:38:17.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.695+0000 7fef81e5d640 1 -- 172.21.15.38:0/3043573411 wait complete. 2024-04-25T08:38:17.702 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T08:38:17.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.810+0000 7f0789dce640 1 Processor -- start 2024-04-25T08:38:17.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.810+0000 7f0789dce640 1 -- start start 2024-04-25T08:38:17.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f0789dce640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f078415df50 0x7f07840ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:17.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f0789dce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840ce2e0 0x7f07840d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:17.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f0789dce640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07840ceff0 con 0x7f078406c110 2024-04-25T08:38:17.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f0789dce640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07840cf2d0 con 0x7f078415df50 2024-04-25T08:38:17.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f0789dce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07840d1f70 con 0x7f07840ce2e0 2024-04-25T08:38:17.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f0782ffd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f078415df50 0x7f07840ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:17.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f0783fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840ce2e0 0x7f07840d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:17.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f07837fe640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f078406c110 0x7f078406c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:60356/0 (socket says 172.21.15.38:60356) 2024-04-25T08:38:17.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f07837fe640 1 -- 172.21.15.38:0/2660181018 learned_addr learned my addr 172.21.15.38:0/2660181018 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:17.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f0783fff640 1 -- 172.21.15.38:0/2660181018 >> v2:172.21.15.107:3300/0 conn(0x7f078415df50 msgr2=0x7f07840ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:17.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f0783fff640 1 --2- 172.21.15.38:0/2660181018 >> v2:172.21.15.107:3300/0 conn(0x7f078415df50 0x7f07840ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f0783fff640 1 -- 172.21.15.38:0/2660181018 >> v1:172.21.15.107:6789/0 conn(0x7f078406c110 legacy=0x7f078406c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f0783fff640 1 -- 172.21.15.38:0/2660181018 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07840d2250 con 0x7f07840ce2e0 2024-04-25T08:38:17.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.811+0000 7f0782ffd640 1 --2- 172.21.15.38:0/2660181018 >> v2:172.21.15.107:3300/0 conn(0x7f078415df50 0x7f07840ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:17.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.812+0000 7f0783fff640 1 --2- 172.21.15.38:0/2660181018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840ce2e0 0x7f07840d2a50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f077c009d90 tx=0x7f077c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=926534f89b5edb0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:17.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.812+0000 7f07827fc640 1 -- 172.21.15.38:0/2660181018 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c007740 con 0x7f07840ce2e0 2024-04-25T08:38:17.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.812+0000 7f07827fc640 1 -- 172.21.15.38:0/2660181018 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c0078e0 con 0x7f07840ce2e0 2024-04-25T08:38:17.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.812+0000 7f07827fc640 1 -- 172.21.15.38:0/2660181018 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c007cb0 con 0x7f07840ce2e0 2024-04-25T08:38:17.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.813+0000 7f0789dce640 1 -- 172.21.15.38:0/2660181018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840ce2e0 msgr2=0x7f07840d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.813+0000 7f0789dce640 1 --2- 172.21.15.38:0/2660181018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840ce2e0 0x7f07840d2a50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f077c009d90 tx=0x7f077c02a660 comp rx=0 tx=0).stop 2024-04-25T08:38:17.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.813+0000 7f0789dce640 1 -- 172.21.15.38:0/2660181018 shutdown_connections 2024-04-25T08:38:17.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.813+0000 7f0789dce640 1 --2- 172.21.15.38:0/2660181018 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07840ce2e0 0x7f07840d2a50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.813+0000 7f0789dce640 1 --2- 172.21.15.38:0/2660181018 >> v2:172.21.15.107:3300/0 conn(0x7f078415df50 0x7f07840ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.813+0000 7f0789dce640 1 -- 172.21.15.38:0/2660181018 >> 172.21.15.38:0/2660181018 conn(0x7f07840d3fb0 msgr2=0x7f07840d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:17.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.813+0000 7f0789dce640 1 -- 172.21.15.38:0/2660181018 shutdown_connections 2024-04-25T08:38:17.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.813+0000 7f0789dce640 1 -- 172.21.15.38:0/2660181018 wait complete. 2024-04-25T08:38:17.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.814+0000 7f0789dce640 1 Processor -- start 2024-04-25T08:38:17.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.814+0000 7f0789dce640 1 -- start start 2024-04-25T08:38:17.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0789dce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f078406c110 0x7f07840ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:17.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0789dce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840ce2e0 0x7f07841f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:17.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0789dce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f078415df50 0x7f07841f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:17.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0789dce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07840d2f90 con 0x7f07840ce2e0 2024-04-25T08:38:17.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0789dce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07840cece0 con 0x7f078415df50 2024-04-25T08:38:17.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0789dce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07840d3110 con 0x7f078406c110 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0782ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840ce2e0 0x7f07841f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0782ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840ce2e0 0x7f07841f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49176/0 (socket says 172.21.15.38:49176) 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0782ffd640 1 -- 172.21.15.38:0/2487127329 learned_addr learned my addr 172.21.15.38:0/2487127329 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f07837fe640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f078406c110 0x7f07840ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0783fff640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f078415df50 0x7f07841f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0782ffd640 1 -- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f078406c110 msgr2=0x7f07840ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0782ffd640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f078406c110 0x7f07840ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0782ffd640 1 -- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f078415df50 msgr2=0x7f07841f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0782ffd640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f078415df50 0x7f07841f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f0782ffd640 1 -- 172.21.15.38:0/2487127329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07840cd000 con 0x7f07840ce2e0 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.815+0000 7f07837fe640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f078406c110 0x7f07840ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.816+0000 7f0782ffd640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840ce2e0 0x7f07841f6c70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f077800ebe0 tx=0x7f077800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.816+0000 7f076bfff640 1 -- 172.21.15.38:0/2487127329 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778017070 con 0x7f07840ce2e0 2024-04-25T08:38:17.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.816+0000 7f0789dce640 1 -- 172.21.15.38:0/2487127329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07840cd2e0 con 0x7f07840ce2e0 2024-04-25T08:38:17.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.816+0000 7f076bfff640 1 -- 172.21.15.38:0/2487127329 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077800cc30 con 0x7f07840ce2e0 2024-04-25T08:38:17.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.816+0000 7f076bfff640 1 -- 172.21.15.38:0/2487127329 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778013470 con 0x7f07840ce2e0 2024-04-25T08:38:17.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.816+0000 7f0789dce640 1 -- 172.21.15.38:0/2487127329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07840cd8a0 con 0x7f07840ce2e0 2024-04-25T08:38:17.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.817+0000 7f0789dce640 1 -- 172.21.15.38:0/2487127329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0784068e80 con 0x7f07840ce2e0 2024-04-25T08:38:17.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.817+0000 7f076bfff640 1 -- 172.21.15.38:0/2487127329 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f077801a070 con 0x7f07840ce2e0 2024-04-25T08:38:17.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.818+0000 7f076bfff640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f077006b4d0 0x7f077006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:17.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.818+0000 7f076bfff640 1 -- 172.21.15.38:0/2487127329 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f077808a370 con 0x7f07840ce2e0 2024-04-25T08:38:17.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.818+0000 7f07837fe640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f077006b4d0 0x7f077006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:17.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.818+0000 7f07837fe640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f077006b4d0 0x7f077006d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0774000f60 tx=0x7f0774009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:17.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:17.822+0000 7f076bfff640 1 -- 172.21.15.38:0/2487127329 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0778059370 con 0x7f07840ce2e0 2024-04-25T08:38:18.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.080+0000 7f0789dce640 1 -- 172.21.15.38:0/2487127329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f07840cdc80 con 0x7f07840ce2e0 2024-04-25T08:38:18.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.367+0000 7f076bfff640 1 -- 172.21.15.38:0/2487127329 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v323) v1 ==== 158+0+0 (secure 0 0 0) 0x7f0778058c70 con 0x7f07840ce2e0 2024-04-25T08:38:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.369+0000 7f0789dce640 1 -- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f077006b4d0 msgr2=0x7f077006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.369+0000 7f0789dce640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f077006b4d0 0x7f077006d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f0774000f60 tx=0x7f0774009040 comp rx=0 tx=0).stop 2024-04-25T08:38:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.369+0000 7f0789dce640 1 -- 172.21.15.38:0/2487127329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840ce2e0 msgr2=0x7f07841f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.369+0000 7f0789dce640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840ce2e0 0x7f07841f6c70 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f077800ebe0 tx=0x7f077800c520 comp rx=0 tx=0).stop 2024-04-25T08:38:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.369+0000 7f0789dce640 1 -- 172.21.15.38:0/2487127329 shutdown_connections 2024-04-25T08:38:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.369+0000 7f0789dce640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f078415df50 0x7f07841f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.369+0000 7f0789dce640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07840ce2e0 0x7f07841f6c70 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.369+0000 7f0789dce640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f077006b4d0 0x7f077006d990 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.369+0000 7f0789dce640 1 --2- 172.21.15.38:0/2487127329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f078406c110 0x7f07840ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.369+0000 7f0789dce640 1 -- 172.21.15.38:0/2487127329 >> 172.21.15.38:0/2487127329 conn(0x7f07840d3fb0 msgr2=0x7f07840b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:18.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.369+0000 7f0789dce640 1 -- 172.21.15.38:0/2487127329 shutdown_connections 2024-04-25T08:38:18.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.369+0000 7f0789dce640 1 -- 172.21.15.38:0/2487127329 wait complete. 2024-04-25T08:38:18.375 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-25T08:38:18.375 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:38:18.375 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:18.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.484+0000 7f37a5db0640 1 Processor -- start 2024-04-25T08:38:18.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.484+0000 7f37a5db0640 1 -- start start 2024-04-25T08:38:18.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.485+0000 7f37a5db0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a006c110 0x7f37a006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.485+0000 7f37a5db0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f37a00ce2e0 0x7f37a00d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.485+0000 7f37a5db0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a006ca30 con 0x7f37a015df50 2024-04-25T08:38:18.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.485+0000 7f37a5db0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a00cf2d0 con 0x7f37a00ce2e0 2024-04-25T08:38:18.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.485+0000 7f37a5db0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00ceff0 con 0x7f37a006c110 2024-04-25T08:38:18.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.485+0000 7f379ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f37a00ce2e0 0x7f37a00d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.485+0000 7f379effd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f37a015df50 0x7f37a00ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35462/0 (socket says 172.21.15.38:35462) 2024-04-25T08:38:18.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.485+0000 7f379effd640 1 -- 172.21.15.38:0/263079246 learned_addr learned my addr 172.21.15.38:0/263079246 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:18.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.485+0000 7f379ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f37a00ce2e0 0x7f37a00d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49184/0 (socket says 172.21.15.38:49184) 2024-04-25T08:38:18.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.485+0000 7f379f7fe640 1 --2- 172.21.15.38:0/263079246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a006c110 0x7f37a006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.486+0000 7f379ffff640 1 -- 172.21.15.38:0/263079246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a006c110 msgr2=0x7f37a006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:18.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.486+0000 7f379ffff640 1 --2- 172.21.15.38:0/263079246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a006c110 0x7f37a006c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.486+0000 7f379ffff640 1 -- 172.21.15.38:0/263079246 >> v1:172.21.15.38:6789/0 conn(0x7f37a015df50 legacy=0x7f37a00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.486+0000 7f379ffff640 1 -- 172.21.15.38:0/263079246 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a00d1f70 con 0x7f37a00ce2e0 2024-04-25T08:38:18.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.486+0000 7f379ffff640 1 --2- 172.21.15.38:0/263079246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a00ce2e0 0x7f37a00d2a50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f3798009d90 tx=0x7f379802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa2aaca706c619a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.486+0000 7f379e7fc640 1 -- 172.21.15.38:0/263079246 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798007740 con 0x7f37a00ce2e0 2024-04-25T08:38:18.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.487+0000 7f379e7fc640 1 -- 172.21.15.38:0/263079246 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37980078e0 con 0x7f37a00ce2e0 2024-04-25T08:38:18.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.487+0000 7f379e7fc640 1 -- 172.21.15.38:0/263079246 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798007cb0 con 0x7f37a00ce2e0 2024-04-25T08:38:18.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.487+0000 7f37a5db0640 1 -- 172.21.15.38:0/263079246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a00ce2e0 msgr2=0x7f37a00d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.487+0000 7f37a5db0640 1 --2- 172.21.15.38:0/263079246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a00ce2e0 0x7f37a00d2a50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f3798009d90 tx=0x7f379802a660 comp rx=0 tx=0).stop 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.487+0000 7f37a5db0640 1 -- 172.21.15.38:0/263079246 shutdown_connections 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.487+0000 7f37a5db0640 1 --2- 172.21.15.38:0/263079246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a00ce2e0 0x7f37a00d2a50 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.487+0000 7f37a5db0640 1 --2- 172.21.15.38:0/263079246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a006c110 0x7f37a006c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.487+0000 7f37a5db0640 1 -- 172.21.15.38:0/263079246 >> 172.21.15.38:0/263079246 conn(0x7f37a00d3fb0 msgr2=0x7f37a00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.487+0000 7f37a5db0640 1 -- 172.21.15.38:0/263079246 shutdown_connections 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.487+0000 7f37a5db0640 1 -- 172.21.15.38:0/263079246 wait complete. 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.488+0000 7f37a5db0640 1 Processor -- start 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.488+0000 7f37a5db0640 1 -- start start 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f37a5db0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a006c110 0x7f37a0154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f37a5db0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a00ce2e0 0x7f37a0160060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f37a5db0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a015df50 0x7f37a015a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f37a5db0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00d2f90 con 0x7f37a006c110 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f37a5db0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00d2530 con 0x7f37a015df50 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f37a5db0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a00d3110 con 0x7f37a00ce2e0 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f379f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a006c110 0x7f37a0154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f379f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a006c110 0x7f37a0154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49186/0 (socket says 172.21.15.38:49186) 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f379f7fe640 1 -- 172.21.15.38:0/4134863289 learned_addr learned my addr 172.21.15.38:0/4134863289 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f379f7fe640 1 -- 172.21.15.38:0/4134863289 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a00ce2e0 msgr2=0x7f37a0160060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f379f7fe640 1 --2- 172.21.15.38:0/4134863289 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a00ce2e0 0x7f37a0160060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f379f7fe640 1 -- 172.21.15.38:0/4134863289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a015df50 msgr2=0x7f37a015a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.489+0000 7f379f7fe640 1 --2- 172.21.15.38:0/4134863289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a015df50 0x7f37a015a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.490+0000 7f379f7fe640 1 -- 172.21.15.38:0/4134863289 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a015e6a0 con 0x7f37a006c110 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.490+0000 7f379f7fe640 1 --2- 172.21.15.38:0/4134863289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a006c110 0x7f37a0154c70 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f379000d5f0 tx=0x7f379000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.490+0000 7f3787fff640 1 -- 172.21.15.38:0/4134863289 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37900125f0 con 0x7f37a006c110 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.490+0000 7f3787fff640 1 -- 172.21.15.38:0/4134863289 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3790019070 con 0x7f37a006c110 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.490+0000 7f37a5db0640 1 -- 172.21.15.38:0/4134863289 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a015e980 con 0x7f37a006c110 2024-04-25T08:38:18.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.490+0000 7f3787fff640 1 -- 172.21.15.38:0/4134863289 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37900129d0 con 0x7f37a006c110 2024-04-25T08:38:18.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.490+0000 7f37a5db0640 1 -- 172.21.15.38:0/4134863289 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a015ef40 con 0x7f37a006c110 2024-04-25T08:38:18.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.491+0000 7f37a5db0640 1 -- 172.21.15.38:0/4134863289 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a0068e80 con 0x7f37a006c110 2024-04-25T08:38:18.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.491+0000 7f3787fff640 1 -- 172.21.15.38:0/4134863289 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3790012b70 con 0x7f37a006c110 2024-04-25T08:38:18.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.492+0000 7f3787fff640 1 --2- 172.21.15.38:0/4134863289 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f378c06b500 0x7f378c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.492+0000 7f3787fff640 1 -- 172.21.15.38:0/4134863289 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f379008a460 con 0x7f37a006c110 2024-04-25T08:38:18.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.492+0000 7f379effd640 1 --2- 172.21.15.38:0/4134863289 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f378c06b500 0x7f378c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.492+0000 7f379effd640 1 --2- 172.21.15.38:0/4134863289 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f378c06b500 0x7f378c06d9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3794000f60 tx=0x7f3794007a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.496+0000 7f3787fff640 1 -- 172.21.15.38:0/4134863289 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3790059450 con 0x7f37a006c110 2024-04-25T08:38:18.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.676+0000 7f37a5db0640 1 -- 172.21.15.38:0/4134863289 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37a00c6bf0 con 0x7f37a006c110 2024-04-25T08:38:18.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.677+0000 7f3787fff640 1 -- 172.21.15.38:0/4134863289 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+13280 (secure 0 0 0) 0x7f3790058d50 con 0x7f37a006c110 2024-04-25T08:38:18.671 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:18.671 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":87,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:38:17.364593+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/380751658":"2024-04-26T08:38:14.358570+0000","172.21.15.38:6833/380751658":"2024-04-26T08:38:14.358570+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:18.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.679+0000 7f37a5db0640 1 -- 172.21.15.38:0/4134863289 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f378c06b500 msgr2=0x7f378c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.679+0000 7f37a5db0640 1 --2- 172.21.15.38:0/4134863289 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f378c06b500 0x7f378c06d9c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f3794000f60 tx=0x7f3794007a80 comp rx=0 tx=0).stop 2024-04-25T08:38:18.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.679+0000 7f37a5db0640 1 -- 172.21.15.38:0/4134863289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a006c110 msgr2=0x7f37a0154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.679+0000 7f37a5db0640 1 --2- 172.21.15.38:0/4134863289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a006c110 0x7f37a0154c70 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f379000d5f0 tx=0x7f379000dab0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.679+0000 7f37a5db0640 1 -- 172.21.15.38:0/4134863289 shutdown_connections 2024-04-25T08:38:18.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.679+0000 7f37a5db0640 1 --2- 172.21.15.38:0/4134863289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37a015df50 0x7f37a015a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.679+0000 7f37a5db0640 1 --2- 172.21.15.38:0/4134863289 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f378c06b500 0x7f378c06d9c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.679+0000 7f37a5db0640 1 --2- 172.21.15.38:0/4134863289 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37a00ce2e0 0x7f37a0160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.679+0000 7f37a5db0640 1 --2- 172.21.15.38:0/4134863289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f37a006c110 0x7f37a0154c70 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.679+0000 7f37a5db0640 1 -- 172.21.15.38:0/4134863289 >> 172.21.15.38:0/4134863289 conn(0x7f37a00d3fb0 msgr2=0x7f37a00b7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:18.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.679+0000 7f37a5db0640 1 -- 172.21.15.38:0/4134863289 shutdown_connections 2024-04-25T08:38:18.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.679+0000 7f37a5db0640 1 -- 172.21.15.38:0/4134863289 wait complete. 2024-04-25T08:38:18.684 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:38:18.685 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:38:18.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.794+0000 7f9f9acb3640 1 Processor -- start 2024-04-25T08:38:18.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.794+0000 7f9f9acb3640 1 -- start start 2024-04-25T08:38:18.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.795+0000 7f9f9acb3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9f940d00e0 0x7f9f940cce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.795+0000 7f9f9acb3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f940cd3b0 0x7f9f940cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.795+0000 7f9f9acb3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f940cdd80 con 0x7f9f940b1d60 2024-04-25T08:38:18.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.795+0000 7f9f9acb3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9406abe0 con 0x7f9f940d00e0 2024-04-25T08:38:18.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.795+0000 7f9f9acb3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9406b480 con 0x7f9f940cd3b0 2024-04-25T08:38:18.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.795+0000 7f9f93fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9f940d00e0 0x7f9f940cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.795+0000 7f9f93fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9f940d00e0 0x7f9f940cce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49194/0 (socket says 172.21.15.38:49194) 2024-04-25T08:38:18.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.795+0000 7f9f93fff640 1 -- 172.21.15.38:0/1668805641 learned_addr learned my addr 172.21.15.38:0/1668805641 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:18.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.796+0000 7f9f99250640 1 --2- 172.21.15.38:0/1668805641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f940cd3b0 0x7f9f940cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.796+0000 7f9f93fff640 1 -- 172.21.15.38:0/1668805641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f940cd3b0 msgr2=0x7f9f940cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:18.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.796+0000 7f9f93fff640 1 --2- 172.21.15.38:0/1668805641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f940cd3b0 0x7f9f940cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.796+0000 7f9f93fff640 1 -- 172.21.15.38:0/1668805641 >> v1:172.21.15.107:6789/0 conn(0x7f9f940b1d60 legacy=0x7f9f940cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:18.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.796+0000 7f9f93fff640 1 -- 172.21.15.38:0/1668805641 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9415ef40 con 0x7f9f940d00e0 2024-04-25T08:38:18.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.796+0000 7f9f93fff640 1 --2- 172.21.15.38:0/1668805641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f940d00e0 0x7f9f940cce70 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009870 tx=0x7f9f7c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c1bf7fed039be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.797+0000 7f9f937fe640 1 -- 172.21.15.38:0/1668805641 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c037070 con 0x7f9f940d00e0 2024-04-25T08:38:18.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.797+0000 7f9f937fe640 1 -- 172.21.15.38:0/1668805641 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c032440 con 0x7f9f940d00e0 2024-04-25T08:38:18.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.797+0000 7f9f937fe640 1 -- 172.21.15.38:0/1668805641 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c032810 con 0x7f9f940d00e0 2024-04-25T08:38:18.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.797+0000 7f9f9acb3640 1 -- 172.21.15.38:0/1668805641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f940d00e0 msgr2=0x7f9f940cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.797+0000 7f9f9acb3640 1 --2- 172.21.15.38:0/1668805641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f940d00e0 0x7f9f940cce70 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c009870 tx=0x7f9f7c02a1c0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.797+0000 7f9f9acb3640 1 -- 172.21.15.38:0/1668805641 shutdown_connections 2024-04-25T08:38:18.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.797+0000 7f9f9acb3640 1 --2- 172.21.15.38:0/1668805641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f940cd3b0 0x7f9f940cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.797+0000 7f9f9acb3640 1 --2- 172.21.15.38:0/1668805641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f940d00e0 0x7f9f940cce70 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.797+0000 7f9f9acb3640 1 -- 172.21.15.38:0/1668805641 >> 172.21.15.38:0/1668805641 conn(0x7f9f940d06b0 msgr2=0x7f9f940d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.797+0000 7f9f9acb3640 1 -- 172.21.15.38:0/1668805641 shutdown_connections 2024-04-25T08:38:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.797+0000 7f9f9acb3640 1 -- 172.21.15.38:0/1668805641 wait complete. 2024-04-25T08:38:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.798+0000 7f9f9acb3640 1 Processor -- start 2024-04-25T08:38:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.799+0000 7f9f9acb3640 1 -- start start 2024-04-25T08:38:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.799+0000 7f9f9acb3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f940b1d60 0x7f9f940b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.799+0000 7f9f9acb3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f940cd3b0 0x7f9f941530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.799+0000 7f9f9acb3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f940d00e0 0x7f9f941e7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.799+0000 7f9f9acb3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9406b910 con 0x7f9f940b1d60 2024-04-25T08:38:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.799+0000 7f9f9acb3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9406b110 con 0x7f9f940cd3b0 2024-04-25T08:38:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.799+0000 7f9f9acb3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9406ba90 con 0x7f9f940d00e0 2024-04-25T08:38:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f98a4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f940b1d60 0x7f9f940b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f98a4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f940b1d60 0x7f9f940b4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49206/0 (socket says 172.21.15.38:49206) 2024-04-25T08:38:18.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f98a4f640 1 -- 172.21.15.38:0/2776886697 learned_addr learned my addr 172.21.15.38:0/2776886697 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f98a4f640 1 -- 172.21.15.38:0/2776886697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f940d00e0 msgr2=0x7f9f941e7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f98a4f640 1 --2- 172.21.15.38:0/2776886697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f940d00e0 0x7f9f941e7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f98a4f640 1 -- 172.21.15.38:0/2776886697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f940cd3b0 msgr2=0x7f9f941530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f98a4f640 1 --2- 172.21.15.38:0/2776886697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f940cd3b0 0x7f9f941530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f98a4f640 1 -- 172.21.15.38:0/2776886697 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f940b5ea0 con 0x7f9f940b1d60 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f98a4f640 1 --2- 172.21.15.38:0/2776886697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f940b1d60 0x7f9f940b4800 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00c730 tx=0x7f9f8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f917fa640 1 -- 172.21.15.38:0/2776886697 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c016020 con 0x7f9f940b1d60 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f917fa640 1 -- 172.21.15.38:0/2776886697 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c006d50 con 0x7f9f940b1d60 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f9acb3640 1 -- 172.21.15.38:0/2776886697 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f940b3d20 con 0x7f9f940b1d60 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f917fa640 1 -- 172.21.15.38:0/2776886697 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c003bb0 con 0x7f9f940b1d60 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.800+0000 7f9f9acb3640 1 -- 172.21.15.38:0/2776886697 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f940b6180 con 0x7f9f940b1d60 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.801+0000 7f9f9acb3640 1 -- 172.21.15.38:0/2776886697 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f94068e80 con 0x7f9f940b1d60 2024-04-25T08:38:18.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.802+0000 7f9f917fa640 1 -- 172.21.15.38:0/2776886697 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f8c00a850 con 0x7f9f940b1d60 2024-04-25T08:38:18.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.802+0000 7f9f917fa640 1 --2- 172.21.15.38:0/2776886697 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f7406b4d0 0x7f9f7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:18.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.802+0000 7f9f917fa640 1 -- 172.21.15.38:0/2776886697 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9f8c089d20 con 0x7f9f940b1d60 2024-04-25T08:38:18.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.802+0000 7f9f93fff640 1 --2- 172.21.15.38:0/2776886697 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f7406b4d0 0x7f9f7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:18.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.803+0000 7f9f93fff640 1 --2- 172.21.15.38:0/2776886697 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f7406b4d0 0x7f9f7406d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c034000 tx=0x7f9f7c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:18.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.806+0000 7f9f917fa640 1 -- 172.21.15.38:0/2776886697 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f8c020020 con 0x7f9f940b1d60 2024-04-25T08:38:18.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:18.999+0000 7f9f9acb3640 1 -- 172.21.15.38:0/2776886697 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f94161430 con 0x7f9f940b1d60 2024-04-25T08:38:18.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.000+0000 7f9f917fa640 1 -- 172.21.15.38:0/2776886697 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f8c019310 con 0x7f9f940b1d60 2024-04-25T08:38:18.993 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:38:18.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.001+0000 7f9f9acb3640 1 -- 172.21.15.38:0/2776886697 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f7406b4d0 msgr2=0x7f9f7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.002+0000 7f9f9acb3640 1 --2- 172.21.15.38:0/2776886697 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f7406b4d0 0x7f9f7406d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c034000 tx=0x7f9f7c02c000 comp rx=0 tx=0).stop 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.002+0000 7f9f9acb3640 1 -- 172.21.15.38:0/2776886697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f940b1d60 msgr2=0x7f9f940b4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.002+0000 7f9f9acb3640 1 --2- 172.21.15.38:0/2776886697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f940b1d60 0x7f9f940b4800 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00c730 tx=0x7f9f8c00cbf0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.002+0000 7f9f9acb3640 1 -- 172.21.15.38:0/2776886697 shutdown_connections 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.002+0000 7f9f9acb3640 1 --2- 172.21.15.38:0/2776886697 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f7406b4d0 0x7f9f7406d990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.002+0000 7f9f9acb3640 1 --2- 172.21.15.38:0/2776886697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f940d00e0 0x7f9f941e7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.002+0000 7f9f9acb3640 1 --2- 172.21.15.38:0/2776886697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f940cd3b0 0x7f9f941530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.002+0000 7f9f9acb3640 1 --2- 172.21.15.38:0/2776886697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f940b1d60 0x7f9f940b4800 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.002+0000 7f9f9acb3640 1 -- 172.21.15.38:0/2776886697 >> 172.21.15.38:0/2776886697 conn(0x7f9f940d06b0 msgr2=0x7f9f940cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.002+0000 7f9f9acb3640 1 -- 172.21.15.38:0/2776886697 shutdown_connections 2024-04-25T08:38:18.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.002+0000 7f9f9acb3640 1 -- 172.21.15.38:0/2776886697 wait complete. 2024-04-25T08:38:19.007 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T08:38:19.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.114+0000 7f5efc212640 1 Processor -- start 2024-04-25T08:38:19.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.114+0000 7f5efc212640 1 -- start start 2024-04-25T08:38:19.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5efc212640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5ef4099a70 0x7f5ef4099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:19.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5efc212640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5ef406cd90 0x7f5ef4167710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:19.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5efc212640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef409a390 con 0x7f5ef40cf470 2024-04-25T08:38:19.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5efc212640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef41614d0 con 0x7f5ef4099a70 2024-04-25T08:38:19.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5efc212640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef40cd970 con 0x7f5ef406cd90 2024-04-25T08:38:19.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5ef9fae640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5ef4099a70 0x7f5ef4099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:19.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5ef9fae640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5ef4099a70 0x7f5ef4099e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51742/0 (socket says 172.21.15.38:51742) 2024-04-25T08:38:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5ef9fae640 1 -- 172.21.15.38:0/2447631425 learned_addr learned my addr 172.21.15.38:0/2447631425 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5efa7af640 1 --2- 172.21.15.38:0/2447631425 >> v2:172.21.15.107:3300/0 conn(0x7f5ef406cd90 0x7f5ef4167710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5ef9fae640 1 -- 172.21.15.38:0/2447631425 >> v2:172.21.15.107:3300/0 conn(0x7f5ef406cd90 msgr2=0x7f5ef4167710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5ef9fae640 1 --2- 172.21.15.38:0/2447631425 >> v2:172.21.15.107:3300/0 conn(0x7f5ef406cd90 0x7f5ef4167710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5ef9fae640 1 -- 172.21.15.38:0/2447631425 >> v1:172.21.15.107:6789/0 conn(0x7f5ef40cf470 legacy=0x7f5ef40cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.115+0000 7f5ef9fae640 1 -- 172.21.15.38:0/2447631425 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40ccd60 con 0x7f5ef4099a70 2024-04-25T08:38:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.116+0000 7f5ef9fae640 1 --2- 172.21.15.38:0/2447631425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef4099a70 0x7f5ef4099e50 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4072a30 tx=0x7f5eec02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17446b56bcb4e5ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:19.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.116+0000 7f5ef8fac640 1 -- 172.21.15.38:0/2447631425 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec037070 con 0x7f5ef4099a70 2024-04-25T08:38:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.116+0000 7f5ef8fac640 1 -- 172.21.15.38:0/2447631425 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eec032440 con 0x7f5ef4099a70 2024-04-25T08:38:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.116+0000 7f5ef8fac640 1 -- 172.21.15.38:0/2447631425 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec032810 con 0x7f5ef4099a70 2024-04-25T08:38:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.117+0000 7f5efc212640 1 -- 172.21.15.38:0/2447631425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef4099a70 msgr2=0x7f5ef4099e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.117+0000 7f5efc212640 1 --2- 172.21.15.38:0/2447631425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef4099a70 0x7f5ef4099e50 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4072a30 tx=0x7f5eec02a230 comp rx=0 tx=0).stop 2024-04-25T08:38:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.117+0000 7f5efc212640 1 -- 172.21.15.38:0/2447631425 shutdown_connections 2024-04-25T08:38:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.117+0000 7f5efc212640 1 --2- 172.21.15.38:0/2447631425 >> v2:172.21.15.107:3300/0 conn(0x7f5ef406cd90 0x7f5ef4167710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.117+0000 7f5efc212640 1 --2- 172.21.15.38:0/2447631425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef4099a70 0x7f5ef4099e50 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.117+0000 7f5efc212640 1 -- 172.21.15.38:0/2447631425 >> 172.21.15.38:0/2447631425 conn(0x7f5ef40b3180 msgr2=0x7f5ef40b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.117+0000 7f5efc212640 1 -- 172.21.15.38:0/2447631425 shutdown_connections 2024-04-25T08:38:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.117+0000 7f5efc212640 1 -- 172.21.15.38:0/2447631425 wait complete. 2024-04-25T08:38:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.118+0000 7f5efc212640 1 Processor -- start 2024-04-25T08:38:19.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.118+0000 7f5efc212640 1 -- start start 2024-04-25T08:38:19.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5efc212640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ef406cd90 0x7f5ef414fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5efc212640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef4099a70 0x7f5ef40c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5efc212640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ef40cf470 0x7f5ef40c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5efc212640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40cd600 con 0x7f5ef4099a70 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5efc212640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40cd320 con 0x7f5ef406cd90 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5efc212640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef40d0780 con 0x7f5ef40cf470 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5ef97ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef4099a70 0x7f5ef40c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5ef97ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef4099a70 0x7f5ef40c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51752/0 (socket says 172.21.15.38:51752) 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5ef97ad640 1 -- 172.21.15.38:0/456503606 learned_addr learned my addr 172.21.15.38:0/456503606 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5ef9fae640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ef406cd90 0x7f5ef414fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5efa7af640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ef40cf470 0x7f5ef40c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5ef97ad640 1 -- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ef40cf470 msgr2=0x7f5ef40c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5ef97ad640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ef40cf470 0x7f5ef40c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:19.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5ef97ad640 1 -- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ef406cd90 msgr2=0x7f5ef414fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5ef97ad640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ef406cd90 0x7f5ef414fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5ef97ad640 1 -- 172.21.15.38:0/456503606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef40c7880 con 0x7f5ef4099a70 2024-04-25T08:38:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5efa7af640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ef40cf470 0x7f5ef40c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.119+0000 7f5ef97ad640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef4099a70 0x7f5ef40c7340 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800c430 tx=0x7f5ee800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.120+0000 7f5ef27fc640 1 -- 172.21.15.38:0/456503606 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8011790 con 0x7f5ef4099a70 2024-04-25T08:38:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.120+0000 7f5ef27fc640 1 -- 172.21.15.38:0/456503606 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee8017070 con 0x7f5ef4099a70 2024-04-25T08:38:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.120+0000 7f5efc212640 1 -- 172.21.15.38:0/456503606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef40c7130 con 0x7f5ef4099a70 2024-04-25T08:38:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.120+0000 7f5ef27fc640 1 -- 172.21.15.38:0/456503606 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8011bb0 con 0x7f5ef4099a70 2024-04-25T08:38:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.120+0000 7f5efc212640 1 -- 172.21.15.38:0/456503606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef40c6b70 con 0x7f5ef4099a70 2024-04-25T08:38:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.121+0000 7f5efc212640 1 -- 172.21.15.38:0/456503606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef4068e80 con 0x7f5ef4099a70 2024-04-25T08:38:19.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.121+0000 7f5ef27fc640 1 -- 172.21.15.38:0/456503606 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ee801c460 con 0x7f5ef4099a70 2024-04-25T08:38:19.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.122+0000 7f5ef27fc640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5ee006b4d0 0x7f5ee006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:19.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.122+0000 7f5ef27fc640 1 -- 172.21.15.38:0/456503606 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5ee8089e30 con 0x7f5ef4099a70 2024-04-25T08:38:19.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.122+0000 7f5ef9fae640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5ee006b4d0 0x7f5ee006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:19.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.122+0000 7f5ef9fae640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5ee006b4d0 0x7f5ee006d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5eec034000 tx=0x7f5eec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:19.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.126+0000 7f5ef27fc640 1 -- 172.21.15.38:0/456503606 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ef40c7880 con 0x7f5ef4099a70 2024-04-25T08:38:19.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:19.375+0000 7f5efc212640 1 -- 172.21.15.38:0/456503606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f5ef409abb0 con 0x7f5ef4099a70 2024-04-25T08:38:20.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.373+0000 7f5ef27fc640 1 -- 172.21.15.38:0/456503606 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v325) v1 ==== 147+0+0 (secure 0 0 0) 0x7f5ee8058eb0 con 0x7f5ef4099a70 2024-04-25T08:38:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.376+0000 7f5efc212640 1 -- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5ee006b4d0 msgr2=0x7f5ee006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.376+0000 7f5efc212640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5ee006b4d0 0x7f5ee006d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5eec034000 tx=0x7f5eec02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.376+0000 7f5efc212640 1 -- 172.21.15.38:0/456503606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef4099a70 msgr2=0x7f5ef40c7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.376+0000 7f5efc212640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef4099a70 0x7f5ef40c7340 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800c430 tx=0x7f5ee800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.376+0000 7f5efc212640 1 -- 172.21.15.38:0/456503606 shutdown_connections 2024-04-25T08:38:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.376+0000 7f5efc212640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5ee006b4d0 0x7f5ee006d990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.376+0000 7f5efc212640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5ef40cf470 0x7f5ef40c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.376+0000 7f5efc212640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5ef4099a70 0x7f5ef40c7340 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.376+0000 7f5efc212640 1 --2- 172.21.15.38:0/456503606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ef406cd90 0x7f5ef414fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.376+0000 7f5efc212640 1 -- 172.21.15.38:0/456503606 >> 172.21.15.38:0/456503606 conn(0x7f5ef40b3180 msgr2=0x7f5ef40cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:20.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.376+0000 7f5efc212640 1 -- 172.21.15.38:0/456503606 shutdown_connections 2024-04-25T08:38:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.376+0000 7f5efc212640 1 -- 172.21.15.38:0/456503606 wait complete. 2024-04-25T08:38:20.384 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:38:20.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.500+0000 7f06dc837640 1 Processor -- start 2024-04-25T08:38:20.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.500+0000 7f06dc837640 1 -- start start 2024-04-25T08:38:20.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.501+0000 7f06dc837640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f06d8166610 0x7f06d8168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.501+0000 7f06dc837640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d80d2060 con 0x7f06d80d0320 2024-04-25T08:38:20.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.501+0000 7f06dc837640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d80d2330 con 0x7f06d80d33d0 2024-04-25T08:38:20.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.501+0000 7f06dc837640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d80d2610 con 0x7f06d8166610 2024-04-25T08:38:20.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d659c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f06d80d0320 0x7f06d80d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47272/0 (socket says 172.21.15.38:47272) 2024-04-25T08:38:20.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d659c640 1 -- 172.21.15.38:0/747484574 learned_addr learned my addr 172.21.15.38:0/747484574 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:20.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255410841 0 0) 0x7f06d80d2060 con 0x7f06d80d0320 2024-04-25T08:38:20.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d6d9d640 1 --2- 172.21.15.38:0/747484574 >> v2:172.21.15.107:3300/0 conn(0x7f06d8166610 0x7f06d8168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06c4003520 con 0x7f06d80d0320 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 234657084 0 0) 0x7f06c4003520 con 0x7f06d80d0320 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 >> v2:172.21.15.107:3300/0 conn(0x7f06d8166610 msgr2=0x7f06d8168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 --2- 172.21.15.38:0/747484574 >> v2:172.21.15.107:3300/0 conn(0x7f06d8166610 0x7f06d8168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 >> v1:172.21.15.107:6789/0 conn(0x7f06d80d33d0 legacy=0x7f06d80d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d80d28f0 con 0x7f06d80d0320 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f06c0001670 con 0x7f06d80d0320 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 >> v1:172.21.15.38:6789/0 conn(0x7f06d80d0320 legacy=0x7f06d80d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 --2- 172.21.15.38:0/747484574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c4003a00 0x7f06c4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 --2- 172.21.15.38:0/747484574 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06c4004800 0x7f06c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 --2- 172.21.15.38:0/747484574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06c40051c0 0x7f06c400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80d2610 con 0x7f06c4003a00 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c400c0a0 con 0x7f06c40051c0 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c400c2a0 con 0x7f06c4004800 2024-04-25T08:38:20.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d659c640 1 --2- 172.21.15.38:0/747484574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c4003a00 0x7f06c4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d5d9b640 1 --2- 172.21.15.38:0/747484574 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06c4004800 0x7f06c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.502+0000 7f06d6d9d640 1 --2- 172.21.15.38:0/747484574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06c40051c0 0x7f06c400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.503+0000 7f06d659c640 1 -- 172.21.15.38:0/747484574 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06c4004800 msgr2=0x7f06c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.503+0000 7f06d659c640 1 --2- 172.21.15.38:0/747484574 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06c4004800 0x7f06c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.503+0000 7f06d659c640 1 -- 172.21.15.38:0/747484574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06c40051c0 msgr2=0x7f06c400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.503+0000 7f06d659c640 1 --2- 172.21.15.38:0/747484574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06c40051c0 0x7f06c400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.503+0000 7f06d659c640 1 -- 172.21.15.38:0/747484574 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c4003520 con 0x7f06c4003a00 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.503+0000 7f06d6d9d640 1 --2- 172.21.15.38:0/747484574 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06c40051c0 0x7f06c400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.503+0000 7f06d6d9d640 1 -- 172.21.15.38:0/747484574 reap_dead start 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.503+0000 7f06d659c640 1 --2- 172.21.15.38:0/747484574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c4003a00 0x7f06c4003e20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f06c000c110 tx=0x7f06c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599892d4f93ed312 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.504+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0007c40 con 0x7f06c4003a00 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.504+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c0005af0 con 0x7f06c4003a00 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.504+0000 7f06d559a640 1 -- 172.21.15.38:0/747484574 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c0007070 con 0x7f06c4003a00 2024-04-25T08:38:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.504+0000 7f06dc837640 1 -- 172.21.15.38:0/747484574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c4003a00 msgr2=0x7f06c4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.504+0000 7f06dc837640 1 --2- 172.21.15.38:0/747484574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c4003a00 0x7f06c4003e20 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f06c000c110 tx=0x7f06c4005830 comp rx=0 tx=0).stop 2024-04-25T08:38:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.504+0000 7f06dc837640 1 -- 172.21.15.38:0/747484574 shutdown_connections 2024-04-25T08:38:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.504+0000 7f06dc837640 1 --2- 172.21.15.38:0/747484574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06c4003a00 0x7f06c4003e20 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.504+0000 7f06dc837640 1 -- 172.21.15.38:0/747484574 >> 172.21.15.38:0/747484574 conn(0x7f06d80668f0 msgr2=0x7f06d8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.504+0000 7f06dc837640 1 -- 172.21.15.38:0/747484574 shutdown_connections 2024-04-25T08:38:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.504+0000 7f06dc837640 1 -- 172.21.15.38:0/747484574 wait complete. 2024-04-25T08:38:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.506+0000 7f06dc837640 1 Processor -- start 2024-04-25T08:38:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.506+0000 7f06dc837640 1 -- start start 2024-04-25T08:38:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.506+0000 7f06dc837640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06c4003a00 0x7f06d815f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.506+0000 7f06dc837640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d8157e60 0x7f06d80b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.506+0000 7f06dc837640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06d80b3c80 0x7f06d81f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.506+0000 7f06dc837640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d8161210 con 0x7f06d8157e60 2024-04-25T08:38:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.506+0000 7f06dc837640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80d2eb0 con 0x7f06d80b3c80 2024-04-25T08:38:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.506+0000 7f06dc837640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d8161390 con 0x7f06c4003a00 2024-04-25T08:38:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.506+0000 7f06d5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d8157e60 0x7f06d80b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06d5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d8157e60 0x7f06d80b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51756/0 (socket says 172.21.15.38:51756) 2024-04-25T08:38:20.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06d5d9b640 1 -- 172.21.15.38:0/2375699305 learned_addr learned my addr 172.21.15.38:0/2375699305 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06d6d9d640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06d80b3c80 0x7f06d81f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06d659c640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06c4003a00 0x7f06d815f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06d5d9b640 1 -- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06c4003a00 msgr2=0x7f06d815f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06d5d9b640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06c4003a00 0x7f06d815f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06d5d9b640 1 -- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06d80b3c80 msgr2=0x7f06d81f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06d5d9b640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06d80b3c80 0x7f06d81f3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06d5d9b640 1 -- 172.21.15.38:0/2375699305 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d815dc50 con 0x7f06d8157e60 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06d6d9d640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06d80b3c80 0x7f06d81f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06d659c640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06c4003a00 0x7f06d815f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06d5d9b640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d8157e60 0x7f06d80b3740 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f06c800f820 tx=0x7f06c800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06ceffd640 1 -- 172.21.15.38:0/2375699305 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80050d0 con 0x7f06d8157e60 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06ceffd640 1 -- 172.21.15.38:0/2375699305 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c8018070 con 0x7f06d8157e60 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06dc837640 1 -- 172.21.15.38:0/2375699305 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d815df30 con 0x7f06d8157e60 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06ceffd640 1 -- 172.21.15.38:0/2375699305 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8005490 con 0x7f06d8157e60 2024-04-25T08:38:20.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.507+0000 7f06dc837640 1 -- 172.21.15.38:0/2375699305 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d81582e0 con 0x7f06d8157e60 2024-04-25T08:38:20.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.508+0000 7f06dc837640 1 -- 172.21.15.38:0/2375699305 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d8068e80 con 0x7f06d8157e60 2024-04-25T08:38:20.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.509+0000 7f06ceffd640 1 -- 172.21.15.38:0/2375699305 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06c800d730 con 0x7f06d8157e60 2024-04-25T08:38:20.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.509+0000 7f06ceffd640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06b806b4d0 0x7f06b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:20.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.509+0000 7f06ceffd640 1 -- 172.21.15.38:0/2375699305 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f06c8089e70 con 0x7f06d8157e60 2024-04-25T08:38:20.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.510+0000 7f06d659c640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06b806b4d0 0x7f06b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:20.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.510+0000 7f06d659c640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06b806b4d0 0x7f06b806d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f06c0032000 tx=0x7f06c002d000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:20.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.514+0000 7f06ceffd640 1 -- 172.21.15.38:0/2375699305 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06c8058ef0 con 0x7f06d8157e60 2024-04-25T08:38:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.693+0000 7f06dc837640 1 -- 172.21.15.38:0/2375699305 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f06d80b4060 con 0x7f06d8157e60 2024-04-25T08:38:20.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.696+0000 7f06ceffd640 1 -- 172.21.15.38:0/2375699305 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f06c80587f0 con 0x7f06d8157e60 2024-04-25T08:38:20.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.698+0000 7f06dc837640 1 -- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06b806b4d0 msgr2=0x7f06b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.698+0000 7f06dc837640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06b806b4d0 0x7f06b806d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f06c0032000 tx=0x7f06c002d000 comp rx=0 tx=0).stop 2024-04-25T08:38:20.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.698+0000 7f06dc837640 1 -- 172.21.15.38:0/2375699305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d8157e60 msgr2=0x7f06d80b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.698+0000 7f06dc837640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d8157e60 0x7f06d80b3740 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f06c800f820 tx=0x7f06c800fce0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.699+0000 7f06dc837640 1 -- 172.21.15.38:0/2375699305 shutdown_connections 2024-04-25T08:38:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.699+0000 7f06dc837640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06d80b3c80 0x7f06d81f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.699+0000 7f06dc837640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f06d8157e60 0x7f06d80b3740 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.699+0000 7f06dc837640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f06b806b4d0 0x7f06b806d990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.699+0000 7f06dc837640 1 --2- 172.21.15.38:0/2375699305 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06c4003a00 0x7f06d815f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.699+0000 7f06dc837640 1 -- 172.21.15.38:0/2375699305 >> 172.21.15.38:0/2375699305 conn(0x7f06d80668f0 msgr2=0x7f06d80ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.699+0000 7f06dc837640 1 -- 172.21.15.38:0/2375699305 shutdown_connections 2024-04-25T08:38:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:20.699+0000 7f06dc837640 1 -- 172.21.15.38:0/2375699305 wait complete. 2024-04-25T08:38:20.703 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:38:20.704 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:38:20.749 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:38:20.800 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:38:20.800 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:38:20.801 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:20.816 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:38:20.816 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:38:20.816 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:20.834 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:38:20.834 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:38:20.834 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:20.872 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:38:20.872 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:38:20.872 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:20.891 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:38:20.891 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:38:20.892 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:21.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.020+0000 7f2abe076640 1 Processor -- start 2024-04-25T08:38:21.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.020+0000 7f2abe076640 1 -- start start 2024-04-25T08:38:21.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.020+0000 7f2abe076640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2ab815be80 0x7f2ab80b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2abe076640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ab80b4e60 0x7f2ab80ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2abe076640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab80b5270 con 0x7f2ab8154cb0 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2abe076640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab80cd440 con 0x7f2ab815be80 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2abe076640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab815c330 con 0x7f2ab80b4e60 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab77fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2ab8154cb0 0x7f2ab80c4ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47286/0 (socket says 172.21.15.38:47286) 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab77fe640 1 -- 172.21.15.38:0/3450285130 learned_addr learned my addr 172.21.15.38:0/3450285130 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab7fff640 1 --2- 172.21.15.38:0/3450285130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ab80b4e60 0x7f2ab80ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab6ffd640 1 --2- 172.21.15.38:0/3450285130 >> v2:172.21.15.107:3300/0 conn(0x7f2ab815be80 0x7f2ab80b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672113342 0 0) 0x7f2aa8001110 con 0x7f2ab8154cb0 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2aa0003520 con 0x7f2ab8154cb0 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1530669799 0 0) 0x7f2aa0003520 con 0x7f2ab8154cb0 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ab80b4e60 msgr2=0x7f2ab80ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:21.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 --2- 172.21.15.38:0/3450285130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ab80b4e60 0x7f2ab80ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 >> v2:172.21.15.107:3300/0 conn(0x7f2ab815be80 msgr2=0x7f2ab80b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 --2- 172.21.15.38:0/3450285130 >> v2:172.21.15.107:3300/0 conn(0x7f2ab815be80 0x7f2ab80b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab80cd690 con 0x7f2ab8154cb0 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f2aa8001860 con 0x7f2ab8154cb0 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab6ffd640 1 --2- 172.21.15.38:0/3450285130 >> v2:172.21.15.107:3300/0 conn(0x7f2ab815be80 0x7f2ab80b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 >> v1:172.21.15.38:6789/0 conn(0x7f2ab8154cb0 legacy=0x7f2ab80c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 --2- 172.21.15.38:0/3450285130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ab80b4e60 0x7f2aa00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 --2- 172.21.15.38:0/3450285130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2aa0003f30 0x7f2aa0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 --2- 172.21.15.38:0/3450285130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa0004d40 0x7f2aa00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80cd440 con 0x7f2aa0004d40 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab815c330 con 0x7f2aa0003f30 2024-04-25T08:38:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa000b9c0 con 0x7f2ab80b4e60 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.021+0000 7f2ab7fff640 1 --2- 172.21.15.38:0/3450285130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa0004d40 0x7f2aa00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.022+0000 7f2ab6ffd640 1 --2- 172.21.15.38:0/3450285130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2aa0003f30 0x7f2aa0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.022+0000 7f2ab77fe640 1 --2- 172.21.15.38:0/3450285130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ab80b4e60 0x7f2aa00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.022+0000 7f2ab7fff640 1 -- 172.21.15.38:0/3450285130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ab80b4e60 msgr2=0x7f2aa00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.022+0000 7f2ab7fff640 1 --2- 172.21.15.38:0/3450285130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2ab80b4e60 0x7f2aa00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.022+0000 7f2ab7fff640 1 -- 172.21.15.38:0/3450285130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2aa0003f30 msgr2=0x7f2aa0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.022+0000 7f2ab7fff640 1 --2- 172.21.15.38:0/3450285130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2aa0003f30 0x7f2aa0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.022+0000 7f2ab7fff640 1 -- 172.21.15.38:0/3450285130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa0003520 con 0x7f2aa0004d40 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.022+0000 7f2ab7fff640 1 --2- 172.21.15.38:0/3450285130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa0004d40 0x7f2aa00051c0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009fc0 tx=0x7f2aa0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3123bec8fef6b6ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.023+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0009c10 con 0x7f2aa0004d40 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.023+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab0036070 con 0x7f2aa0004d40 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.023+0000 7f2ab67fc640 1 -- 172.21.15.38:0/3450285130 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0031670 con 0x7f2aa0004d40 2024-04-25T08:38:21.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.023+0000 7f2abe076640 1 -- 172.21.15.38:0/3450285130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa0004d40 msgr2=0x7f2aa00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.023+0000 7f2abe076640 1 --2- 172.21.15.38:0/3450285130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2aa0004d40 0x7f2aa00051c0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0009fc0 tx=0x7f2aa0005830 comp rx=0 tx=0).stop 2024-04-25T08:38:21.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.023+0000 7f2ab7fff640 1 -- 172.21.15.38:0/3450285130 reap_dead start 2024-04-25T08:38:21.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.023+0000 7f2abe076640 1 -- 172.21.15.38:0/3450285130 shutdown_connections 2024-04-25T08:38:21.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.023+0000 7f2abe076640 1 -- 172.21.15.38:0/3450285130 >> 172.21.15.38:0/3450285130 conn(0x7f2ab806b210 msgr2=0x7f2ab806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:21.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.023+0000 7f2abe076640 1 -- 172.21.15.38:0/3450285130 shutdown_connections 2024-04-25T08:38:21.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.023+0000 7f2abe076640 1 -- 172.21.15.38:0/3450285130 wait complete. 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.024+0000 7f2abe076640 1 Processor -- start 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.025+0000 7f2abe076640 1 -- start start 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.025+0000 7f2abe076640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2aa0004d40 0x7f2ab81f1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.025+0000 7f2abe076640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab8145a70 0x7f2ab8145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.025+0000 7f2abe076640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab8148610 0x7f2ab8148a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.025+0000 7f2abe076640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80cdf30 con 0x7f2ab8148610 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.025+0000 7f2abe076640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80cdc50 con 0x7f2ab8145a70 2024-04-25T08:38:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.025+0000 7f2abe076640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab80ce210 con 0x7f2aa0004d40 2024-04-25T08:38:21.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.025+0000 7f2ab7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab8148610 0x7f2ab8148a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.025+0000 7f2ab7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab8148610 0x7f2ab8148a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51774/0 (socket says 172.21.15.38:51774) 2024-04-25T08:38:21.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.025+0000 7f2ab7fff640 1 -- 172.21.15.38:0/3096856236 learned_addr learned my addr 172.21.15.38:0/3096856236 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:21.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2ab77fe640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2aa0004d40 0x7f2ab81f1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2ab6ffd640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab8145a70 0x7f2ab8145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2ab7fff640 1 -- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2aa0004d40 msgr2=0x7f2ab81f1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2ab7fff640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2aa0004d40 0x7f2ab81f1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2ab7fff640 1 -- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab8145a70 msgr2=0x7f2ab8145ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2ab7fff640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab8145a70 0x7f2ab8145ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2ab7fff640 1 -- 172.21.15.38:0/3096856236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab81f15a0 con 0x7f2ab8148610 2024-04-25T08:38:21.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2ab6ffd640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab8145a70 0x7f2ab8145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:21.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2ab7fff640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab8148610 0x7f2ab8148a90 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000e230 tx=0x7f2ab0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2a9ffff640 1 -- 172.21.15.38:0/3096856236 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0009c10 con 0x7f2ab8148610 2024-04-25T08:38:21.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2a9ffff640 1 -- 172.21.15.38:0/3096856236 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab0003990 con 0x7f2ab8148610 2024-04-25T08:38:21.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2a9ffff640 1 -- 172.21.15.38:0/3096856236 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab0003b60 con 0x7f2ab8148610 2024-04-25T08:38:21.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2abe076640 1 -- 172.21.15.38:0/3096856236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab8146530 con 0x7f2ab8148610 2024-04-25T08:38:21.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.026+0000 7f2abe076640 1 -- 172.21.15.38:0/3096856236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab8146af0 con 0x7f2ab8148610 2024-04-25T08:38:21.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.027+0000 7f2abe076640 1 -- 172.21.15.38:0/3096856236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab8068e80 con 0x7f2ab8148610 2024-04-25T08:38:21.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.028+0000 7f2a9ffff640 1 -- 172.21.15.38:0/3096856236 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ab004a020 con 0x7f2ab8148610 2024-04-25T08:38:21.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.028+0000 7f2a9ffff640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2aa406b500 0x7f2aa406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.028+0000 7f2a9ffff640 1 -- 172.21.15.38:0/3096856236 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2ab00ac7a0 con 0x7f2ab8148610 2024-04-25T08:38:21.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.028+0000 7f2ab77fe640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2aa406b500 0x7f2aa406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.029+0000 7f2ab77fe640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2aa406b500 0x7f2aa406d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80ccf70 tx=0x7f2aa8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.032+0000 7f2a9ffff640 1 -- 172.21.15.38:0/3096856236 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ab81f15a0 con 0x7f2ab8148610 2024-04-25T08:38:21.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.249+0000 7f2abe076640 1 -- 172.21.15.38:0/3096856236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ab8154230 con 0x7f2ab8148610 2024-04-25T08:38:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.249+0000 7f2a9ffff640 1 -- 172.21.15.38:0/3096856236 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f2ab007b780 con 0x7f2ab8148610 2024-04-25T08:38:21.243 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:21.243 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6080,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3781069672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3781069672},{"type":"v1","addr":"172.21.15.107:6837","nonce":3781069672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":6565,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/624970524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":624970524},{"type":"v1","addr":"172.21.15.107:6836","nonce":624970524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46},{"gid":7380,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1256978001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1256978001},{"type":"v1","addr":"172.21.15.38:6834","nonce":1256978001}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":7770,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1309967758","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":1309967758},{"type":"v1","addr":"172.21.15.38:6835","nonce":1309967758}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[]} 2024-04-25T08:38:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 53 2024-04-25T08:38:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.251+0000 7f2abe076640 1 -- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2aa406b500 msgr2=0x7f2aa406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.251+0000 7f2abe076640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2aa406b500 0x7f2aa406d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80ccf70 tx=0x7f2aa8009040 comp rx=0 tx=0).stop 2024-04-25T08:38:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.251+0000 7f2abe076640 1 -- 172.21.15.38:0/3096856236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab8148610 msgr2=0x7f2ab8148a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.251+0000 7f2abe076640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab8148610 0x7f2ab8148a90 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000e230 tx=0x7f2ab0009750 comp rx=0 tx=0).stop 2024-04-25T08:38:21.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.251+0000 7f2abe076640 1 -- 172.21.15.38:0/3096856236 shutdown_connections 2024-04-25T08:38:21.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.251+0000 7f2abe076640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ab8148610 0x7f2ab8148a90 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.251+0000 7f2abe076640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ab8145a70 0x7f2ab8145ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.251+0000 7f2abe076640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2aa406b500 0x7f2aa406d9c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.251+0000 7f2abe076640 1 --2- 172.21.15.38:0/3096856236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2aa0004d40 0x7f2ab81f1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.251+0000 7f2abe076640 1 -- 172.21.15.38:0/3096856236 >> 172.21.15.38:0/3096856236 conn(0x7f2ab806b210 msgr2=0x7f2ab8149bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:21.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.251+0000 7f2abe076640 1 -- 172.21.15.38:0/3096856236 shutdown_connections 2024-04-25T08:38:21.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.252+0000 7f2abe076640 1 -- 172.21.15.38:0/3096856236 wait complete. 2024-04-25T08:38:21.257 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:38:21.257 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:38:21.257 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:38:21.257 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:38:21.257 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:38:21.258 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:38:21.258 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:38:21.258 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:38:21.258 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:38:21.258 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:38:21.258 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:38:21.258 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:38:21.259 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:38:21.259 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:38:21.259 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:38:21.259 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:38:21.267 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:38:21.267 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:38:21.268 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:38:21.268 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:38:21.269 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:38:21.269 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:38:21.270 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:38:21.270 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:38:21.310 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:38:21.310 DEBUG:teuthology.parallel:result is None 2024-04-25T08:38:21.311 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:38:21.311 DEBUG:teuthology.parallel:result is None 2024-04-25T08:38:21.312 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:38:21.312 DEBUG:teuthology.parallel:result is None 2024-04-25T08:38:21.313 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:38:21.313 DEBUG:teuthology.parallel:result is None 2024-04-25T08:38:21.314 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:38:21.386 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:38:21.387 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:38:21.387 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-04-25T08:38:21.393+0000 7fde061ae640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T08:38:21.391 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:38:21.391 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-25T08:38:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.425+0000 7fc6ecbc8640 1 Processor -- start 2024-04-25T08:38:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.426+0000 7fc6ecbc8640 1 -- start start 2024-04-25T08:38:21.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6ecbc8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6e8160ef0 0x7fc6e80d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6ecbc8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc6e80b2ab0 0x7fc6e80d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6e6d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc6e80b2ab0 0x7fc6e80d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6ecbc8640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e80b9610 con 0x7fc6e80b9180 2024-04-25T08:38:21.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6ecbc8640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e8161cf0 con 0x7fc6e80b2ab0 2024-04-25T08:38:21.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6ecbc8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e81612d0 con 0x7fc6e8160ef0 2024-04-25T08:38:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6e6d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc6e80b2ab0 0x7fc6e80d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51814/0 (socket says 172.21.15.38:51814) 2024-04-25T08:38:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6e5d9b640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fc6e80b9180 0x7fc6e80d37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:51936/0 (socket says 172.21.15.38:51936) 2024-04-25T08:38:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6e6d9d640 1 -- 172.21.15.38:0/2632103263 learned_addr learned my addr 172.21.15.38:0/2632103263 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6e659c640 1 --2- 172.21.15.38:0/2632103263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6e8160ef0 0x7fc6e80d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6e6d9d640 1 -- 172.21.15.38:0/2632103263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6e8160ef0 msgr2=0x7fc6e80d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6e6d9d640 1 --2- 172.21.15.38:0/2632103263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6e8160ef0 0x7fc6e80d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6e6d9d640 1 -- 172.21.15.38:0/2632103263 >> v1:172.21.15.107:6789/0 conn(0x7fc6e80b9180 legacy=0x7fc6e80d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6e6d9d640 1 -- 172.21.15.38:0/2632103263 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e81616a0 con 0x7fc6e80b2ab0 2024-04-25T08:38:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.427+0000 7fc6e659c640 1 --2- 172.21.15.38:0/2632103263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6e8160ef0 0x7fc6e80d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.428+0000 7fc6e6d9d640 1 --2- 172.21.15.38:0/2632103263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6e80b2ab0 0x7fc6e80d0cf0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009e50 tx=0x7fc6e002a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8eb40af16408ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.428+0000 7fc6e559a640 1 -- 172.21.15.38:0/2632103263 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0036070 con 0x7fc6e80b2ab0 2024-04-25T08:38:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.428+0000 7fc6e559a640 1 -- 172.21.15.38:0/2632103263 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e0031600 con 0x7fc6e80b2ab0 2024-04-25T08:38:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.428+0000 7fc6e559a640 1 -- 172.21.15.38:0/2632103263 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e00319d0 con 0x7fc6e80b2ab0 2024-04-25T08:38:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.429+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/2632103263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6e80b2ab0 msgr2=0x7fc6e80d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.429+0000 7fc6ecbc8640 1 --2- 172.21.15.38:0/2632103263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6e80b2ab0 0x7fc6e80d0cf0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009e50 tx=0x7fc6e002a590 comp rx=0 tx=0).stop 2024-04-25T08:38:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.429+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/2632103263 shutdown_connections 2024-04-25T08:38:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.429+0000 7fc6ecbc8640 1 --2- 172.21.15.38:0/2632103263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6e80b2ab0 0x7fc6e80d0cf0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.429+0000 7fc6ecbc8640 1 --2- 172.21.15.38:0/2632103263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6e8160ef0 0x7fc6e80d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.429+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/2632103263 >> 172.21.15.38:0/2632103263 conn(0x7fc6e8066d10 msgr2=0x7fc6e8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:21.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.429+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/2632103263 shutdown_connections 2024-04-25T08:38:21.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.429+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/2632103263 wait complete. 2024-04-25T08:38:21.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.431+0000 7fc6ecbc8640 1 Processor -- start 2024-04-25T08:38:21.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.431+0000 7fc6ecbc8640 1 -- start start 2024-04-25T08:38:21.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.431+0000 7fc6ecbc8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e80b2ab0 0x7fc6e81f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.431+0000 7fc6ecbc8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6e8160ef0 0x7fc6e80ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.431+0000 7fc6ecbc8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6e80cc340 0x7fc6e80b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6e5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6e8160ef0 0x7fc6e80ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6ecbc8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e8162300 con 0x7fc6e8160ef0 2024-04-25T08:38:21.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6ecbc8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e8162180 con 0x7fc6e80b2ab0 2024-04-25T08:38:21.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6ecbc8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e8162480 con 0x7fc6e80cc340 2024-04-25T08:38:21.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6e5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6e8160ef0 0x7fc6e80ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51826/0 (socket says 172.21.15.38:51826) 2024-04-25T08:38:21.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6e5d9b640 1 -- 172.21.15.38:0/48209749 learned_addr learned my addr 172.21.15.38:0/48209749 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:21.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6e659c640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e80b2ab0 0x7fc6e81f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6e6d9d640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6e80cc340 0x7fc6e80b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6e5d9b640 1 -- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6e80cc340 msgr2=0x7fc6e80b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6e5d9b640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6e80cc340 0x7fc6e80b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6e5d9b640 1 -- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e80b2ab0 msgr2=0x7fc6e81f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6e5d9b640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e80b2ab0 0x7fc6e81f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6e5d9b640 1 -- 172.21.15.38:0/48209749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e81f32b0 con 0x7fc6e8160ef0 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6e659c640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e80b2ab0 0x7fc6e81f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6e5d9b640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6e8160ef0 0x7fc6e80ca930 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c7b0 tx=0x7fc6d800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6deffd640 1 -- 172.21.15.38:0/48209749 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8018070 con 0x7fc6e8160ef0 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6deffd640 1 -- 172.21.15.38:0/48209749 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d80072d0 con 0x7fc6e8160ef0 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6deffd640 1 -- 172.21.15.38:0/48209749 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d80047e0 con 0x7fc6e8160ef0 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/48209749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e80cbbb0 con 0x7fc6e8160ef0 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.432+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/48209749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e80cb5f0 con 0x7fc6e8160ef0 2024-04-25T08:38:21.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.433+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/48209749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e804ef90 con 0x7fc6e8160ef0 2024-04-25T08:38:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.434+0000 7fc6deffd640 1 -- 172.21.15.38:0/48209749 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6d8004980 con 0x7fc6e8160ef0 2024-04-25T08:38:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.435+0000 7fc6deffd640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc6c806b500 0x7fc6c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.435+0000 7fc6deffd640 1 -- 172.21.15.38:0/48209749 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc6d80898c0 con 0x7fc6e8160ef0 2024-04-25T08:38:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.435+0000 7fc6e659c640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc6c806b500 0x7fc6c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.435+0000 7fc6e659c640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc6c806b500 0x7fc6c806d9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009af0 tx=0x7fc6d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:21.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.438+0000 7fc6deffd640 1 -- 172.21.15.38:0/48209749 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6e81f32b0 con 0x7fc6e8160ef0 2024-04-25T08:38:21.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:21.631+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/48209749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc6e80cb7f0 con 0x7fc6e8160ef0 2024-04-25T08:38:22.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.712+0000 7fc6deffd640 1 -- 172.21.15.38:0/48209749 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc6d8004c70 con 0x7fc6e8160ef0 2024-04-25T08:38:22.706 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-04-25T08:38:22.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.715+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc6c806b500 msgr2=0x7fc6c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.715+0000 7fc6ecbc8640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc6c806b500 0x7fc6c806d9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009af0 tx=0x7fc6d0008040 comp rx=0 tx=0).stop 2024-04-25T08:38:22.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.715+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/48209749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6e8160ef0 msgr2=0x7fc6e80ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.715+0000 7fc6ecbc8640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6e8160ef0 0x7fc6e80ca930 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fc6d800c7b0 tx=0x7fc6d800cc70 comp rx=0 tx=0).stop 2024-04-25T08:38:22.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.715+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/48209749 shutdown_connections 2024-04-25T08:38:22.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.715+0000 7fc6ecbc8640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc6c806b500 0x7fc6c806d9c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.715+0000 7fc6ecbc8640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc6e80cc340 0x7fc6e80b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.715+0000 7fc6ecbc8640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6e8160ef0 0x7fc6e80ca930 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.715+0000 7fc6ecbc8640 1 --2- 172.21.15.38:0/48209749 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6e80b2ab0 0x7fc6e81f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.715+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/48209749 >> 172.21.15.38:0/48209749 conn(0x7fc6e8066d10 msgr2=0x7fc6e80d1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:22.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.715+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/48209749 shutdown_connections 2024-04-25T08:38:22.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.715+0000 7fc6ecbc8640 1 -- 172.21.15.38:0/48209749 wait complete. 2024-04-25T08:38:22.723 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:38:22.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.837+0000 7f2299258640 1 Processor -- start 2024-04-25T08:38:22.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.837+0000 7f2299258640 1 -- start start 2024-04-25T08:38:22.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.838+0000 7f2299258640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f229415bb10 0x7f229409a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:22.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.838+0000 7f2299258640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2294160340 con 0x7f229415fbf0 2024-04-25T08:38:22.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.838+0000 7f2299258640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22940cf400 con 0x7f22940d0460 2024-04-25T08:38:22.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.838+0000 7f2299258640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22940cffd0 con 0x7f229415bb10 2024-04-25T08:38:22.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.838+0000 7f229359e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f229415fbf0 0x7f22940c5250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47304/0 (socket says 172.21.15.38:47304) 2024-04-25T08:38:22.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.838+0000 7f229359e640 1 -- 172.21.15.38:0/3035276209 learned_addr learned my addr 172.21.15.38:0/3035276209 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:22.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.839+0000 7f2292d9d640 1 --2- 172.21.15.38:0/3035276209 >> v2:172.21.15.107:3300/0 conn(0x7f229415bb10 0x7f229409a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:22.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.839+0000 7f2291d9b640 1 -- 172.21.15.38:0/3035276209 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77217399 0 0) 0x7f2284001490 con 0x7f229415fbf0 2024-04-25T08:38:22.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.839+0000 7f2291d9b640 1 -- 172.21.15.38:0/3035276209 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2280003520 con 0x7f229415fbf0 2024-04-25T08:38:22.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.839+0000 7f2292d9d640 1 -- 172.21.15.38:0/3035276209 >> v1:172.21.15.107:6789/0 conn(0x7f22940d0460 legacy=0x7f229415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.839+0000 7f2291d9b640 1 -- 172.21.15.38:0/3035276209 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3643196992 0 0) 0x7f2280003520 con 0x7f229415fbf0 2024-04-25T08:38:22.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.839+0000 7f2292d9d640 1 -- 172.21.15.38:0/3035276209 >> v1:172.21.15.38:6789/0 conn(0x7f229415fbf0 legacy=0x7f22940c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.839+0000 7f2292d9d640 1 -- 172.21.15.38:0/3035276209 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22940cf6a0 con 0x7f229415bb10 2024-04-25T08:38:22.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.840+0000 7f2292d9d640 1 --2- 172.21.15.38:0/3035276209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f229415bb10 0x7f229409a480 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f228c0097a0 tx=0x7f228c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f77d622ab37e316e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:22.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.840+0000 7f2291d9b640 1 -- 172.21.15.38:0/3035276209 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c037070 con 0x7f229415bb10 2024-04-25T08:38:22.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.840+0000 7f2291d9b640 1 -- 172.21.15.38:0/3035276209 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c02abc0 con 0x7f229415bb10 2024-04-25T08:38:22.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.840+0000 7f2291d9b640 1 -- 172.21.15.38:0/3035276209 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284001bc0 con 0x7f229415bb10 2024-04-25T08:38:22.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.840+0000 7f2299258640 1 -- 172.21.15.38:0/3035276209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f229415bb10 msgr2=0x7f229409a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.840+0000 7f2299258640 1 --2- 172.21.15.38:0/3035276209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f229415bb10 0x7f229409a480 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f228c0097a0 tx=0x7f228c02a130 comp rx=0 tx=0).stop 2024-04-25T08:38:22.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.840+0000 7f2299258640 1 -- 172.21.15.38:0/3035276209 shutdown_connections 2024-04-25T08:38:22.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.840+0000 7f2299258640 1 --2- 172.21.15.38:0/3035276209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f229415bb10 0x7f229409a480 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.841+0000 7f2299258640 1 -- 172.21.15.38:0/3035276209 >> 172.21.15.38:0/3035276209 conn(0x7f22940b31f0 msgr2=0x7f22940b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:22.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.841+0000 7f2299258640 1 -- 172.21.15.38:0/3035276209 shutdown_connections 2024-04-25T08:38:22.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.841+0000 7f2299258640 1 -- 172.21.15.38:0/3035276209 wait complete. 2024-04-25T08:38:22.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.842+0000 7f2299258640 1 Processor -- start 2024-04-25T08:38:22.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.842+0000 7f2299258640 1 -- start start 2024-04-25T08:38:22.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f2299258640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22940d0460 0x7f2294151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:22.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f2299258640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229415bb10 0x7f229406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:22.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f2299258640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f229415fbf0 0x7f229406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:22.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f2299258640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229409a9c0 con 0x7f229415bb10 2024-04-25T08:38:22.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f2299258640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22940cfc60 con 0x7f22940d0460 2024-04-25T08:38:22.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f2299258640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229409ab40 con 0x7f229415fbf0 2024-04-25T08:38:22.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f229259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229415bb10 0x7f229406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:22.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f229259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229415bb10 0x7f229406ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51828/0 (socket says 172.21.15.38:51828) 2024-04-25T08:38:22.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f229259c640 1 -- 172.21.15.38:0/3308639801 learned_addr learned my addr 172.21.15.38:0/3308639801 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f2292d9d640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22940d0460 0x7f2294151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f229359e640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f229415fbf0 0x7f229406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f229259c640 1 -- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f229415fbf0 msgr2=0x7f229406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f229259c640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f229415fbf0 0x7f229406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f229259c640 1 -- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22940d0460 msgr2=0x7f2294151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f229259c640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22940d0460 0x7f2294151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f229259c640 1 -- 172.21.15.38:0/3308639801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22940cafd0 con 0x7f229415bb10 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.843+0000 7f229259c640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229415bb10 0x7f229406ca80 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f227c00c450 tx=0x7f227c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.844+0000 7f228b7fe640 1 -- 172.21.15.38:0/3308639801 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c0052d0 con 0x7f229415bb10 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.844+0000 7f228b7fe640 1 -- 172.21.15.38:0/3308639801 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c005470 con 0x7f229415bb10 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.844+0000 7f2299258640 1 -- 172.21.15.38:0/3308639801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f229415c100 con 0x7f229415bb10 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.844+0000 7f228b7fe640 1 -- 172.21.15.38:0/3308639801 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c012870 con 0x7f229415bb10 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.844+0000 7f2299258640 1 -- 172.21.15.38:0/3308639801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f229415c6c0 con 0x7f229415bb10 2024-04-25T08:38:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.845+0000 7f2299258640 1 -- 172.21.15.38:0/3308639801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f229415bef0 con 0x7f229415bb10 2024-04-25T08:38:22.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.845+0000 7f228b7fe640 1 -- 172.21.15.38:0/3308639801 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f227c012a10 con 0x7f229415bb10 2024-04-25T08:38:22.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.846+0000 7f228b7fe640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f227406b4d0 0x7f227406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:22.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.846+0000 7f228b7fe640 1 -- 172.21.15.38:0/3308639801 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f227c089c80 con 0x7f229415bb10 2024-04-25T08:38:22.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.846+0000 7f2292d9d640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f227406b4d0 0x7f227406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:22.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.846+0000 7f2292d9d640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f227406b4d0 0x7f227406d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f228c034000 tx=0x7f228c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:22.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:22.850+0000 7f228b7fe640 1 -- 172.21.15.38:0/3308639801 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f227c058cc0 con 0x7f229415bb10 2024-04-25T08:38:23.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.100+0000 7f2299258640 1 -- 172.21.15.38:0/3308639801 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2294068e80 con 0x7f229415bb10 2024-04-25T08:38:23.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.101+0000 7f228b7fe640 1 -- 172.21.15.38:0/3308639801 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f227c0585c0 con 0x7f229415bb10 2024-04-25T08:38:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:23.095 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:38:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-04-25T08:38:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-04-25T08:38:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmwq8CDBAA+cjd+QXtFgwZ3kxdABRWlA==", 2024-04-25T08:38:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:38:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:38:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:38:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmPRFbFxAAojn2r/62vvf1qTy9QVuyIw==", 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-04-25T08:38:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmGLjqEhAAyPRgyy5iMgtccapydv4uOQ==", 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmt0AUHhAAoQyQ7wXBMGE+7Hi84tUfAg==", 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:38:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCxFCpmv+iTExAAEJihmsvBbp9xXxPnj49D6g==", 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-04-25T08:38:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC0FCpmdnQKFBAADLAsYZEUhoJx292jIYZroQ==", 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC3FCpm4f5BExAAMVSPkqsrnuLcO88AXKrGHQ==", 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-04-25T08:38:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC6FCpmHNN7EhAABPhmnxGo70ZOd8Oex62Mdw==", 2024-04-25T08:38:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:38:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDAFCpmiThGFhAAG5Pyvn/JzL9NUoxHJkPMvg==", 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDDFCpmb/FHFBAA2cMhMqQQc4Fk6j8FpjhnaQ==", 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDGFCpmAZsSEhAAnGIqqcvuHMVSc8Zbk309kA==", 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDJFCpmqdVUEBAAb9abblyUztrjwxxe4jsAwQ==", 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw==", 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-04-25T08:38:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:38:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-04-25T08:38:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmL7eKKhAASeQEdHDBInfozdLVzP4DQw==", 2024-04-25T08:38:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:38:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:38:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:38:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:38:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmH22HJxAAWyrooyJn5hRs/bb0Rn1HHw==", 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpm6Zr/LxAAxh9Y2gvvqj1jIaVVrQAvgw==", 2024-04-25T08:38:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCpFCpmlj/fLBAAvxLluFQFR8uhppGlNpaygA==", 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:38:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcXHhCBAAvI5zmWidVTvOmEcVRWasEA==", 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-04-25T08:38:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcr96BRAAQwUPxILJukYERAiEXeur+g==", 2024-04-25T08:38:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:38:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:38:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:38:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:38:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T08:38:23.108 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:38:23.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.103+0000 7f2299258640 1 -- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f227406b4d0 msgr2=0x7f227406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.103+0000 7f2299258640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f227406b4d0 0x7f227406d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f228c034000 tx=0x7f228c02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:23.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.103+0000 7f2299258640 1 -- 172.21.15.38:0/3308639801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229415bb10 msgr2=0x7f229406ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.103+0000 7f2299258640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229415bb10 0x7f229406ca80 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f227c00c450 tx=0x7f227c00c910 comp rx=0 tx=0).stop 2024-04-25T08:38:23.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.103+0000 7f2299258640 1 -- 172.21.15.38:0/3308639801 shutdown_connections 2024-04-25T08:38:23.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.103+0000 7f2299258640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f227406b4d0 0x7f227406d990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.103+0000 7f2299258640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f229415fbf0 0x7f229406cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.103+0000 7f2299258640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f229415bb10 0x7f229406ca80 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.103+0000 7f2299258640 1 --2- 172.21.15.38:0/3308639801 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22940d0460 0x7f2294151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.103+0000 7f2299258640 1 -- 172.21.15.38:0/3308639801 >> 172.21.15.38:0/3308639801 conn(0x7f22940b31f0 msgr2=0x7f22940c5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:23.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.103+0000 7f2299258640 1 -- 172.21.15.38:0/3308639801 shutdown_connections 2024-04-25T08:38:23.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.103+0000 7f2299258640 1 -- 172.21.15.38:0/3308639801 wait complete. 2024-04-25T08:38:23.111 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:23.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.219+0000 7f568e9cf640 1 Processor -- start 2024-04-25T08:38:23.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.219+0000 7f568e9cf640 1 -- start start 2024-04-25T08:38:23.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f568e9cf640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f568806c110 0x7f568806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f568e9cf640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56880ce2e0 0x7f56880d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f568e9cf640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568806ca30 con 0x7f568815df50 2024-04-25T08:38:23.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f568e9cf640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56880cf2d0 con 0x7f568806c110 2024-04-25T08:38:23.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f568e9cf640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56880ceff0 con 0x7f56880ce2e0 2024-04-25T08:38:23.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f5687fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f568806c110 0x7f568806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f56877fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f568815df50 0x7f56880ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47314/0 (socket says 172.21.15.38:47314) 2024-04-25T08:38:23.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f5687fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f568806c110 0x7f568806c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51830/0 (socket says 172.21.15.38:51830) 2024-04-25T08:38:23.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f56877fe640 1 -- 172.21.15.38:0/3834100513 learned_addr learned my addr 172.21.15.38:0/3834100513 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:23.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f568cf6c640 1 --2- 172.21.15.38:0/3834100513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56880ce2e0 0x7f56880d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f5687fff640 1 -- 172.21.15.38:0/3834100513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56880ce2e0 msgr2=0x7f56880d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:23.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f5687fff640 1 --2- 172.21.15.38:0/3834100513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56880ce2e0 0x7f56880d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f5687fff640 1 -- 172.21.15.38:0/3834100513 >> v1:172.21.15.38:6789/0 conn(0x7f568815df50 legacy=0x7f56880ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f5686ffd640 1 -- 172.21.15.38:0/3834100513 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362248889 0 0) 0x7f568806ca30 con 0x7f568815df50 2024-04-25T08:38:23.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.220+0000 7f5687fff640 1 -- 172.21.15.38:0/3834100513 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880d1f70 con 0x7f568806c110 2024-04-25T08:38:23.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.221+0000 7f5687fff640 1 --2- 172.21.15.38:0/3834100513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568806c110 0x7f568806c4f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f56780097a0 tx=0x7f567802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d341bbb5b53ede server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:23.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.221+0000 7f5686ffd640 1 -- 172.21.15.38:0/3834100513 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678037070 con 0x7f568806c110 2024-04-25T08:38:23.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.221+0000 7f5686ffd640 1 -- 172.21.15.38:0/3834100513 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5678032440 con 0x7f568806c110 2024-04-25T08:38:23.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.221+0000 7f5686ffd640 1 -- 172.21.15.38:0/3834100513 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678032810 con 0x7f568806c110 2024-04-25T08:38:23.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.222+0000 7f568e9cf640 1 -- 172.21.15.38:0/3834100513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568806c110 msgr2=0x7f568806c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.222+0000 7f568e9cf640 1 --2- 172.21.15.38:0/3834100513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568806c110 0x7f568806c4f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f56780097a0 tx=0x7f567802a180 comp rx=0 tx=0).stop 2024-04-25T08:38:23.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.222+0000 7f568e9cf640 1 -- 172.21.15.38:0/3834100513 shutdown_connections 2024-04-25T08:38:23.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.222+0000 7f568e9cf640 1 --2- 172.21.15.38:0/3834100513 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f56880ce2e0 0x7f56880d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.222+0000 7f568e9cf640 1 --2- 172.21.15.38:0/3834100513 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568806c110 0x7f568806c4f0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.222+0000 7f568e9cf640 1 -- 172.21.15.38:0/3834100513 >> 172.21.15.38:0/3834100513 conn(0x7f56880d3fb0 msgr2=0x7f56880d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:23.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.222+0000 7f568e9cf640 1 -- 172.21.15.38:0/3834100513 shutdown_connections 2024-04-25T08:38:23.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.222+0000 7f568e9cf640 1 -- 172.21.15.38:0/3834100513 wait complete. 2024-04-25T08:38:23.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.223+0000 7f568e9cf640 1 Processor -- start 2024-04-25T08:38:23.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.223+0000 7f568e9cf640 1 -- start start 2024-04-25T08:38:23.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f568e9cf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568806c110 0x7f568815ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f568e9cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56880ce2e0 0x7f56880ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f568e9cf640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f568815df50 0x7f56880c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f568e9cf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880d2f90 con 0x7f568806c110 2024-04-25T08:38:23.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f568e9cf640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56880d2530 con 0x7f56880ce2e0 2024-04-25T08:38:23.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f568e9cf640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56880d3110 con 0x7f568815df50 2024-04-25T08:38:23.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f5687fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568806c110 0x7f568815ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f5687fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568806c110 0x7f568815ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51840/0 (socket says 172.21.15.38:51840) 2024-04-25T08:38:23.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f5687fff640 1 -- 172.21.15.38:0/4074751365 learned_addr learned my addr 172.21.15.38:0/4074751365 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f568cf6c640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f568815df50 0x7f56880c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f56877fe640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56880ce2e0 0x7f56880ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f5687fff640 1 -- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f568815df50 msgr2=0x7f56880c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f5687fff640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f568815df50 0x7f56880c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f5687fff640 1 -- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56880ce2e0 msgr2=0x7f56880ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f5687fff640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56880ce2e0 0x7f56880ccfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.224+0000 7f5687fff640 1 -- 172.21.15.38:0/4074751365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56880c8550 con 0x7f568806c110 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.225+0000 7f5687fff640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568806c110 0x7f568815ee90 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f56780096f0 tx=0x7f56780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.225+0000 7f5684ff9640 1 -- 172.21.15.38:0/4074751365 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678037040 con 0x7f568806c110 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.225+0000 7f5684ff9640 1 -- 172.21.15.38:0/4074751365 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56780307b0 con 0x7f568806c110 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.225+0000 7f568e9cf640 1 -- 172.21.15.38:0/4074751365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56880c9aa0 con 0x7f568806c110 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.225+0000 7f5684ff9640 1 -- 172.21.15.38:0/4074751365 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678030b80 con 0x7f568806c110 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.225+0000 7f568e9cf640 1 -- 172.21.15.38:0/4074751365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56880cad40 con 0x7f568806c110 2024-04-25T08:38:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.226+0000 7f568e9cf640 1 -- 172.21.15.38:0/4074751365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5688068e80 con 0x7f568806c110 2024-04-25T08:38:23.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.226+0000 7f5684ff9640 1 -- 172.21.15.38:0/4074751365 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5678044020 con 0x7f568806c110 2024-04-25T08:38:23.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.226+0000 7f5684ff9640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f566806b4d0 0x7f566806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.227+0000 7f5684ff9640 1 -- 172.21.15.38:0/4074751365 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f56780aa3f0 con 0x7f568806c110 2024-04-25T08:38:23.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.227+0000 7f56877fe640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f566806b4d0 0x7f566806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.227+0000 7f56877fe640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f566806b4d0 0x7f566806d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f56880cd9c0 tx=0x7f5670007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:23.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.231+0000 7f5684ff9640 1 -- 172.21.15.38:0/4074751365 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f567804b020 con 0x7f568806c110 2024-04-25T08:38:23.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.414+0000 7f568e9cf640 1 -- 172.21.15.38:0/4074751365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56880d0c00 con 0x7f568806c110 2024-04-25T08:38:23.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.414+0000 7f5684ff9640 1 -- 172.21.15.38:0/4074751365 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+13153 (secure 0 0 0) 0x7f5678032e20 con 0x7f568806c110 2024-04-25T08:38:23.408 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:23.408 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":92,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:38:22.711584+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:23.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.416+0000 7f568e9cf640 1 -- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f566806b4d0 msgr2=0x7f566806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.416+0000 7f568e9cf640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f566806b4d0 0x7f566806d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f56880cd9c0 tx=0x7f5670007c00 comp rx=0 tx=0).stop 2024-04-25T08:38:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.416+0000 7f568e9cf640 1 -- 172.21.15.38:0/4074751365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568806c110 msgr2=0x7f568815ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.416+0000 7f568e9cf640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568806c110 0x7f568815ee90 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f56780096f0 tx=0x7f56780046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.416+0000 7f568e9cf640 1 -- 172.21.15.38:0/4074751365 shutdown_connections 2024-04-25T08:38:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.416+0000 7f568e9cf640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f566806b4d0 0x7f566806d990 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.416+0000 7f568e9cf640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f568815df50 0x7f56880c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.417+0000 7f568e9cf640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56880ce2e0 0x7f56880ccfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.417+0000 7f568e9cf640 1 --2- 172.21.15.38:0/4074751365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568806c110 0x7f568815ee90 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.417+0000 7f568e9cf640 1 -- 172.21.15.38:0/4074751365 >> 172.21.15.38:0/4074751365 conn(0x7f56880d3fb0 msgr2=0x7f56880d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.417+0000 7f568e9cf640 1 -- 172.21.15.38:0/4074751365 shutdown_connections 2024-04-25T08:38:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.417+0000 7f568e9cf640 1 -- 172.21.15.38:0/4074751365 wait complete. 2024-04-25T08:38:23.422 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:38:23.423 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:38:23.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.533+0000 7fba50f22640 1 Processor -- start 2024-04-25T08:38:23.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.533+0000 7fba50f22640 1 -- start start 2024-04-25T08:38:23.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba50f22640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fba4c0d0320 0x7fba4c0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba50f22640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fba4c0d0700 0x7fba4c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba50f22640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba4c0d2330 con 0x7fba4c0d1e90 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba50f22640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4c0d2610 con 0x7fba4c0d0320 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba50f22640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba4c0d28f0 con 0x7fba4c0d0700 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba4a59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fba4c0d0320 0x7fba4c0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba4ad9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fba4c0d1e90 0x7fba4c0b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47326/0 (socket says 172.21.15.38:47326) 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba4ad9d640 1 -- 172.21.15.38:0/3178955287 learned_addr learned my addr 172.21.15.38:0/3178955287 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba4a59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fba4c0d0320 0x7fba4c0d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51852/0 (socket says 172.21.15.38:51852) 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba49d9b640 1 --2- 172.21.15.38:0/3178955287 >> v2:172.21.15.107:3300/0 conn(0x7fba4c0d0700 0x7fba4c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba4a59c640 1 -- 172.21.15.38:0/3178955287 >> v2:172.21.15.107:3300/0 conn(0x7fba4c0d0700 msgr2=0x7fba4c0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba4a59c640 1 --2- 172.21.15.38:0/3178955287 >> v2:172.21.15.107:3300/0 conn(0x7fba4c0d0700 0x7fba4c0d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba4a59c640 1 -- 172.21.15.38:0/3178955287 >> v1:172.21.15.38:6789/0 conn(0x7fba4c0d1e90 legacy=0x7fba4c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.534+0000 7fba4a59c640 1 -- 172.21.15.38:0/3178955287 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4c0d2bd0 con 0x7fba4c0d0320 2024-04-25T08:38:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.535+0000 7fba4a59c640 1 --2- 172.21.15.38:0/3178955287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba4c0d0320 0x7fba4c0d33d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0b6410 tx=0x7fba3c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8290045aefdde320 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:23.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.535+0000 7fba4959a640 1 -- 172.21.15.38:0/3178955287 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c037070 con 0x7fba4c0d0320 2024-04-25T08:38:23.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.535+0000 7fba4959a640 1 -- 172.21.15.38:0/3178955287 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3c032440 con 0x7fba4c0d0320 2024-04-25T08:38:23.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.535+0000 7fba4959a640 1 -- 172.21.15.38:0/3178955287 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c032810 con 0x7fba4c0d0320 2024-04-25T08:38:23.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.536+0000 7fba50f22640 1 -- 172.21.15.38:0/3178955287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba4c0d0320 msgr2=0x7fba4c0d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.536+0000 7fba50f22640 1 --2- 172.21.15.38:0/3178955287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba4c0d0320 0x7fba4c0d33d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fba4c0b6410 tx=0x7fba3c02a230 comp rx=0 tx=0).stop 2024-04-25T08:38:23.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.536+0000 7fba50f22640 1 -- 172.21.15.38:0/3178955287 shutdown_connections 2024-04-25T08:38:23.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.536+0000 7fba50f22640 1 --2- 172.21.15.38:0/3178955287 >> v2:172.21.15.107:3300/0 conn(0x7fba4c0d0700 0x7fba4c0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.536+0000 7fba50f22640 1 --2- 172.21.15.38:0/3178955287 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba4c0d0320 0x7fba4c0d33d0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.536+0000 7fba50f22640 1 -- 172.21.15.38:0/3178955287 >> 172.21.15.38:0/3178955287 conn(0x7fba4c0668f0 msgr2=0x7fba4c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:23.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.536+0000 7fba50f22640 1 -- 172.21.15.38:0/3178955287 shutdown_connections 2024-04-25T08:38:23.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.536+0000 7fba50f22640 1 -- 172.21.15.38:0/3178955287 wait complete. 2024-04-25T08:38:23.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.537+0000 7fba50f22640 1 Processor -- start 2024-04-25T08:38:23.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.537+0000 7fba50f22640 1 -- start start 2024-04-25T08:38:23.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba50f22640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba4c0d0320 0x7fba4c153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba50f22640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba4c0d0700 0x7fba4c1f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba50f22640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba4c0d1e90 0x7fba4c156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba50f22640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c0d0b40 con 0x7fba4c0d0700 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba50f22640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c161210 con 0x7fba4c0d0320 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba50f22640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba4c153e80 con 0x7fba4c0d1e90 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba49d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba4c0d0700 0x7fba4c1f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba49d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba4c0d0700 0x7fba4c1f34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51862/0 (socket says 172.21.15.38:51862) 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba49d9b640 1 -- 172.21.15.38:0/3362059126 learned_addr learned my addr 172.21.15.38:0/3362059126 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba4a59c640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba4c0d0320 0x7fba4c153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba4ad9d640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba4c0d1e90 0x7fba4c156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba49d9b640 1 -- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba4c0d1e90 msgr2=0x7fba4c156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba49d9b640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba4c0d1e90 0x7fba4c156720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba49d9b640 1 -- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba4c0d0320 msgr2=0x7fba4c153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba49d9b640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba4c0d0320 0x7fba4c153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba49d9b640 1 -- 172.21.15.38:0/3362059126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4c0b3270 con 0x7fba4c0d0700 2024-04-25T08:38:23.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.538+0000 7fba4ad9d640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba4c0d1e90 0x7fba4c156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:23.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.539+0000 7fba49d9b640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba4c0d0700 0x7fba4c1f34e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fba4000d800 tx=0x7fba4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:23.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.539+0000 7fba3affd640 1 -- 172.21.15.38:0/3362059126 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4000b6c0 con 0x7fba4c0d0700 2024-04-25T08:38:23.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.539+0000 7fba3affd640 1 -- 172.21.15.38:0/3362059126 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba40019070 con 0x7fba4c0d0700 2024-04-25T08:38:23.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.539+0000 7fba50f22640 1 -- 172.21.15.38:0/3362059126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba4c1f3dc0 con 0x7fba4c0d0700 2024-04-25T08:38:23.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.539+0000 7fba3affd640 1 -- 172.21.15.38:0/3362059126 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4000ba80 con 0x7fba4c0d0700 2024-04-25T08:38:23.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.539+0000 7fba50f22640 1 -- 172.21.15.38:0/3362059126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba4c1f4380 con 0x7fba4c0d0700 2024-04-25T08:38:23.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.540+0000 7fba50f22640 1 -- 172.21.15.38:0/3362059126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4c068e80 con 0x7fba4c0d0700 2024-04-25T08:38:23.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.540+0000 7fba3affd640 1 -- 172.21.15.38:0/3362059126 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba4000bc20 con 0x7fba4c0d0700 2024-04-25T08:38:23.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.541+0000 7fba3affd640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fba2c06b4d0 0x7fba2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.541+0000 7fba3affd640 1 -- 172.21.15.38:0/3362059126 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fba4008a3a0 con 0x7fba4c0d0700 2024-04-25T08:38:23.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.541+0000 7fba4a59c640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fba2c06b4d0 0x7fba2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.542+0000 7fba4a59c640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fba2c06b4d0 0x7fba2c06d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fba3c034000 tx=0x7fba3c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:23.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.545+0000 7fba3affd640 1 -- 172.21.15.38:0/3362059126 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba40059470 con 0x7fba4c0d0700 2024-04-25T08:38:23.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.723+0000 7fba50f22640 1 -- 172.21.15.38:0/3362059126 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fba4c0b3050 con 0x7fba4c0d0700 2024-04-25T08:38:23.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.723+0000 7fba3affd640 1 -- 172.21.15.38:0/3362059126 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fba40058d70 con 0x7fba4c0d0700 2024-04-25T08:38:23.717 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:38:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.725+0000 7fba50f22640 1 -- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fba2c06b4d0 msgr2=0x7fba2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.725+0000 7fba50f22640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fba2c06b4d0 0x7fba2c06d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fba3c034000 tx=0x7fba3c02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.725+0000 7fba50f22640 1 -- 172.21.15.38:0/3362059126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba4c0d0700 msgr2=0x7fba4c1f34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.725+0000 7fba50f22640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba4c0d0700 0x7fba4c1f34e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fba4000d800 tx=0x7fba4000dcc0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.725+0000 7fba50f22640 1 -- 172.21.15.38:0/3362059126 shutdown_connections 2024-04-25T08:38:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.725+0000 7fba50f22640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fba2c06b4d0 0x7fba2c06d990 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.725+0000 7fba50f22640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba4c0d1e90 0x7fba4c156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.725+0000 7fba50f22640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fba4c0d0700 0x7fba4c1f34e0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.725+0000 7fba50f22640 1 --2- 172.21.15.38:0/3362059126 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba4c0d0320 0x7fba4c153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.725+0000 7fba50f22640 1 -- 172.21.15.38:0/3362059126 >> 172.21.15.38:0/3362059126 conn(0x7fba4c0668f0 msgr2=0x7fba4c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:23.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.725+0000 7fba50f22640 1 -- 172.21.15.38:0/3362059126 shutdown_connections 2024-04-25T08:38:23.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.725+0000 7fba50f22640 1 -- 172.21.15.38:0/3362059126 wait complete. 2024-04-25T08:38:23.730 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:38:23.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.838+0000 7f460a122640 1 Processor -- start 2024-04-25T08:38:23.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.838+0000 7f460a122640 1 -- start start 2024-04-25T08:38:23.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.838+0000 7f460a122640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f460409acb0 0x7f46040cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.838+0000 7f460a122640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4604161540 con 0x7f46040cd8d0 2024-04-25T08:38:23.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.838+0000 7f460a122640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460409a520 con 0x7f460406cdf0 2024-04-25T08:38:23.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.838+0000 7f460a122640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460409a820 con 0x7f460409acb0 2024-04-25T08:38:23.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f46037fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f46040cd8d0 0x7f460406c6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47330/0 (socket says 172.21.15.38:47330) 2024-04-25T08:38:23.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f46037fe640 1 -- 172.21.15.38:0/1536779433 learned_addr learned my addr 172.21.15.38:0/1536779433 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f45faffd640 1 --2- 172.21.15.38:0/1536779433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f460409acb0 0x7f46040cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325164297 0 0) 0x7f4604161540 con 0x7f46040cd8d0 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f0003520 con 0x7f46040cd8d0 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1006736860 0 0) 0x7f45f0003520 con 0x7f46040cd8d0 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f460409acb0 msgr2=0x7f46040cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 --2- 172.21.15.38:0/1536779433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f460409acb0 0x7f46040cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 >> v1:172.21.15.107:6789/0 conn(0x7f460406cdf0 legacy=0x7f46040cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460406be90 con 0x7f46040cd8d0 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f45f4001670 con 0x7f46040cd8d0 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 >> v1:172.21.15.38:6789/0 conn(0x7f46040cd8d0 legacy=0x7f460406c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 --2- 172.21.15.38:0/1536779433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45f00039e0 0x7f45f0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 --2- 172.21.15.38:0/1536779433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f460409acb0 0x7f45f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 --2- 172.21.15.38:0/1536779433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45f0004d40 0x7f45f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460409a820 con 0x7f45f00039e0 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f000b9f0 con 0x7f45f0004d40 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f000bbf0 con 0x7f460409acb0 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f46037fe640 1 --2- 172.21.15.38:0/1536779433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45f00039e0 0x7f45f0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.840+0000 7f4603fff640 1 --2- 172.21.15.38:0/1536779433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45f0004d40 0x7f45f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.839+0000 7f45faffd640 1 --2- 172.21.15.38:0/1536779433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f460409acb0 0x7f45f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.840+0000 7f46037fe640 1 -- 172.21.15.38:0/1536779433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f460409acb0 msgr2=0x7f45f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.840+0000 7f46037fe640 1 --2- 172.21.15.38:0/1536779433 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f460409acb0 0x7f45f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.840+0000 7f46037fe640 1 -- 172.21.15.38:0/1536779433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45f0004d40 msgr2=0x7f45f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.840+0000 7f46037fe640 1 --2- 172.21.15.38:0/1536779433 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45f0004d40 0x7f45f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.840+0000 7f46037fe640 1 -- 172.21.15.38:0/1536779433 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f0003520 con 0x7f45f00039e0 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.840+0000 7f46037fe640 1 --2- 172.21.15.38:0/1536779433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45f00039e0 0x7f45f0003dc0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f45f400c290 tx=0x7f45f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed50b3aae5740fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.841+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4007d50 con 0x7f45f00039e0 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.841+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f4005af0 con 0x7f45f00039e0 2024-04-25T08:38:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.841+0000 7f4602ffd640 1 -- 172.21.15.38:0/1536779433 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4007070 con 0x7f45f00039e0 2024-04-25T08:38:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.841+0000 7f460a122640 1 -- 172.21.15.38:0/1536779433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45f00039e0 msgr2=0x7f45f0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.841+0000 7f460a122640 1 --2- 172.21.15.38:0/1536779433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45f00039e0 0x7f45f0003dc0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f45f400c290 tx=0x7f45f0005830 comp rx=0 tx=0).stop 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.841+0000 7f4603fff640 1 -- 172.21.15.38:0/1536779433 reap_dead start 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.841+0000 7f460a122640 1 -- 172.21.15.38:0/1536779433 shutdown_connections 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.841+0000 7f460a122640 1 -- 172.21.15.38:0/1536779433 >> 172.21.15.38:0/1536779433 conn(0x7f46040b3160 msgr2=0x7f46040b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.841+0000 7f460a122640 1 -- 172.21.15.38:0/1536779433 shutdown_connections 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.841+0000 7f460a122640 1 -- 172.21.15.38:0/1536779433 wait complete. 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.842+0000 7f460a122640 1 Processor -- start 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.843+0000 7f460a122640 1 -- start start 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.843+0000 7f460a122640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f00039e0 0x7f4604156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.843+0000 7f460a122640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46040b6a10 0x7f46041f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.843+0000 7f460a122640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46041f4070 0x7f46040ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.843+0000 7f460a122640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460406c410 con 0x7f46041f4070 2024-04-25T08:38:23.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.843+0000 7f460a122640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46040d0ad0 con 0x7f46040b6a10 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.843+0000 7f460a122640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46040cc910 con 0x7f45f00039e0 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.843+0000 7f4603fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46041f4070 0x7f46040ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.843+0000 7f4603fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46041f4070 0x7f46040ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51880/0 (socket says 172.21.15.38:51880) 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.843+0000 7f4603fff640 1 -- 172.21.15.38:0/2273352519 learned_addr learned my addr 172.21.15.38:0/2273352519 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.843+0000 7f46037fe640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f00039e0 0x7f4604156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.843+0000 7f45faffd640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46040b6a10 0x7f46041f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.844+0000 7f4603fff640 1 -- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f00039e0 msgr2=0x7f4604156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.844+0000 7f4603fff640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f00039e0 0x7f4604156cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.844+0000 7f4603fff640 1 -- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46040b6a10 msgr2=0x7f46041f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.844+0000 7f4603fff640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46040b6a10 0x7f46041f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.844+0000 7f4603fff640 1 -- 172.21.15.38:0/2273352519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46040ae800 con 0x7f46041f4070 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.844+0000 7f4603fff640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46041f4070 0x7f46040ca910 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f46040729d0 tx=0x7f45fc00feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.844+0000 7f4600ff9640 1 -- 172.21.15.38:0/2273352519 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc019070 con 0x7f46041f4070 2024-04-25T08:38:23.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.844+0000 7f4600ff9640 1 -- 172.21.15.38:0/2273352519 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc006d60 con 0x7f46041f4070 2024-04-25T08:38:23.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.844+0000 7f460a122640 1 -- 172.21.15.38:0/2273352519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46040aeae0 con 0x7f46041f4070 2024-04-25T08:38:23.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.844+0000 7f4600ff9640 1 -- 172.21.15.38:0/2273352519 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc003bb0 con 0x7f46041f4070 2024-04-25T08:38:23.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.844+0000 7f460a122640 1 -- 172.21.15.38:0/2273352519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46040af0a0 con 0x7f46041f4070 2024-04-25T08:38:23.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.845+0000 7f460a122640 1 -- 172.21.15.38:0/2273352519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4604068e80 con 0x7f46041f4070 2024-04-25T08:38:23.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.845+0000 7f4600ff9640 1 -- 172.21.15.38:0/2273352519 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45fc01c070 con 0x7f46041f4070 2024-04-25T08:38:23.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.846+0000 7f4600ff9640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f45e406b4d0 0x7f45e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:23.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.846+0000 7f4600ff9640 1 -- 172.21.15.38:0/2273352519 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f45fc0891c0 con 0x7f46041f4070 2024-04-25T08:38:23.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.846+0000 7f46037fe640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f45e406b4d0 0x7f45e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:23.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.847+0000 7f46037fe640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f45e406b4d0 0x7f45e406d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f45f4032000 tx=0x7f45f4034040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:23.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:23.850+0000 7f4600ff9640 1 -- 172.21.15.38:0/2273352519 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45fc057c40 con 0x7f46041f4070 2024-04-25T08:38:24.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.034+0000 7f460a122640 1 -- 172.21.15.38:0/2273352519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f46040711b0 con 0x7f46041f4070 2024-04-25T08:38:24.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.034+0000 7f4600ff9640 1 -- 172.21.15.38:0/2273352519 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) v1 ==== 84+0+65 (secure 0 0 0) 0x7f45fc05bce0 con 0x7f46041f4070 2024-04-25T08:38:24.029 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:24.029 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T08:38:24.029 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:38:24.030 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-04-25T08:38:24.030 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-04-25T08:38:24.031 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:24.031 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T08:38:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.036+0000 7f460a122640 1 -- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f45e406b4d0 msgr2=0x7f45e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.036+0000 7f460a122640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f45e406b4d0 0x7f45e406d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f45f4032000 tx=0x7f45f4034040 comp rx=0 tx=0).stop 2024-04-25T08:38:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.036+0000 7f460a122640 1 -- 172.21.15.38:0/2273352519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46041f4070 msgr2=0x7f46040ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.036+0000 7f460a122640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46041f4070 0x7f46040ca910 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f46040729d0 tx=0x7f45fc00feb0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.036+0000 7f460a122640 1 -- 172.21.15.38:0/2273352519 shutdown_connections 2024-04-25T08:38:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.036+0000 7f460a122640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46041f4070 0x7f46040ca910 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.036+0000 7f460a122640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f46040b6a10 0x7f46041f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.036+0000 7f460a122640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f45e406b4d0 0x7f45e406d990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.036+0000 7f460a122640 1 --2- 172.21.15.38:0/2273352519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f45f00039e0 0x7f4604156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.036+0000 7f460a122640 1 -- 172.21.15.38:0/2273352519 >> 172.21.15.38:0/2273352519 conn(0x7f46040b3160 msgr2=0x7f46040c7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.036+0000 7f460a122640 1 -- 172.21.15.38:0/2273352519 shutdown_connections 2024-04-25T08:38:24.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.036+0000 7f460a122640 1 -- 172.21.15.38:0/2273352519 wait complete. 2024-04-25T08:38:24.041 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:38:24.041 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:38:24.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.152+0000 7faad38d8640 1 Processor -- start 2024-04-25T08:38:24.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.153+0000 7faad38d8640 1 -- start start 2024-04-25T08:38:24.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad38d8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7faacc15b940 0x7faacc09a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:24.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad38d8640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7faacc09a180 0x7faacc0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:24.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad38d8640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faacc0cc590 con 0x7faacc0cd350 2024-04-25T08:38:24.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad38d8640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faacc0cc870 con 0x7faacc15b940 2024-04-25T08:38:24.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad38d8640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faacc0ccb50 con 0x7faacc09a180 2024-04-25T08:38:24.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad1674640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7faacc15b940 0x7faacc09a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:24.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad0e73640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7faacc0cd350 0x7faacc099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47332/0 (socket says 172.21.15.38:47332) 2024-04-25T08:38:24.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad1674640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7faacc15b940 0x7faacc09a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51894/0 (socket says 172.21.15.38:51894) 2024-04-25T08:38:24.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad0e73640 1 -- 172.21.15.38:0/2468946344 learned_addr learned my addr 172.21.15.38:0/2468946344 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:24.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad1e75640 1 --2- 172.21.15.38:0/2468946344 >> v2:172.21.15.107:3300/0 conn(0x7faacc09a180 0x7faacc0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:24.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad1674640 1 -- 172.21.15.38:0/2468946344 >> v2:172.21.15.107:3300/0 conn(0x7faacc09a180 msgr2=0x7faacc0b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:24.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad1674640 1 --2- 172.21.15.38:0/2468946344 >> v2:172.21.15.107:3300/0 conn(0x7faacc09a180 0x7faacc0b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad1674640 1 -- 172.21.15.38:0/2468946344 >> v1:172.21.15.38:6789/0 conn(0x7faacc0cd350 legacy=0x7faacc099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faac3fff640 1 -- 172.21.15.38:0/2468946344 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1119839711 0 0) 0x7faacc0cc590 con 0x7faacc0cd350 2024-04-25T08:38:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.154+0000 7faad1674640 1 -- 172.21.15.38:0/2468946344 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faacc0cce30 con 0x7faacc15b940 2024-04-25T08:38:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.155+0000 7faad1674640 1 --2- 172.21.15.38:0/2468946344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faacc15b940 0x7faacc09a810 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7faacc15f530 tx=0x7faac802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54b171e448943706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.155+0000 7faac3fff640 1 -- 172.21.15.38:0/2468946344 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8037070 con 0x7faacc15b940 2024-04-25T08:38:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.155+0000 7faac3fff640 1 -- 172.21.15.38:0/2468946344 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac802add0 con 0x7faacc15b940 2024-04-25T08:38:24.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.155+0000 7faac3fff640 1 -- 172.21.15.38:0/2468946344 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8032660 con 0x7faacc15b940 2024-04-25T08:38:24.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.156+0000 7faad38d8640 1 -- 172.21.15.38:0/2468946344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faacc15b940 msgr2=0x7faacc09a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.156+0000 7faad38d8640 1 --2- 172.21.15.38:0/2468946344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faacc15b940 0x7faacc09a810 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7faacc15f530 tx=0x7faac802a080 comp rx=0 tx=0).stop 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.156+0000 7faad38d8640 1 -- 172.21.15.38:0/2468946344 shutdown_connections 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.156+0000 7faad38d8640 1 --2- 172.21.15.38:0/2468946344 >> v2:172.21.15.107:3300/0 conn(0x7faacc09a180 0x7faacc0b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.156+0000 7faad38d8640 1 --2- 172.21.15.38:0/2468946344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faacc15b940 0x7faacc09a810 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.156+0000 7faad38d8640 1 -- 172.21.15.38:0/2468946344 >> 172.21.15.38:0/2468946344 conn(0x7faacc0b3190 msgr2=0x7faacc0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.156+0000 7faad38d8640 1 -- 172.21.15.38:0/2468946344 shutdown_connections 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.156+0000 7faad38d8640 1 -- 172.21.15.38:0/2468946344 wait complete. 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.157+0000 7faad38d8640 1 Processor -- start 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.157+0000 7faad38d8640 1 -- start start 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad38d8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faacc09a180 0x7faacc0b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad38d8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faacc0cd350 0x7faacc1f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad38d8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faacc15b940 0x7faacc15a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad38d8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc15bd20 con 0x7faacc0cd350 2024-04-25T08:38:24.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad38d8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc0cc240 con 0x7faacc15b940 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad38d8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faacc15bea0 con 0x7faacc09a180 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad0e73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faacc0cd350 0x7faacc1f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad0e73640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faacc0cd350 0x7faacc1f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51898/0 (socket says 172.21.15.38:51898) 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad0e73640 1 -- 172.21.15.38:0/3068133084 learned_addr learned my addr 172.21.15.38:0/3068133084 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad1674640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faacc09a180 0x7faacc0b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad1e75640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faacc15b940 0x7faacc15a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad0e73640 1 -- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faacc09a180 msgr2=0x7faacc0b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad0e73640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faacc09a180 0x7faacc0b4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad0e73640 1 -- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faacc15b940 msgr2=0x7faacc15a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad0e73640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faacc15b940 0x7faacc15a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad0e73640 1 -- 172.21.15.38:0/3068133084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faacc1f5270 con 0x7faacc0cd350 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad1e75640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faacc15b940 0x7faacc15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:24.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faad0e73640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faacc0cd350 0x7faacc1f4900 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7faabc00c460 tx=0x7faabc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faac1ffb640 1 -- 172.21.15.38:0/3068133084 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc018070 con 0x7faacc0cd350 2024-04-25T08:38:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faac1ffb640 1 -- 172.21.15.38:0/3068133084 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faabc007500 con 0x7faacc0cd350 2024-04-25T08:38:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.158+0000 7faac1ffb640 1 -- 172.21.15.38:0/3068133084 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc004a20 con 0x7faacc0cd350 2024-04-25T08:38:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.159+0000 7faad38d8640 1 -- 172.21.15.38:0/3068133084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faacc1f5550 con 0x7faacc0cd350 2024-04-25T08:38:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.159+0000 7faad38d8640 1 -- 172.21.15.38:0/3068133084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faacc0b5010 con 0x7faacc0cd350 2024-04-25T08:38:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.160+0000 7faad38d8640 1 -- 172.21.15.38:0/3068133084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faacc068e80 con 0x7faacc0cd350 2024-04-25T08:38:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.160+0000 7faac1ffb640 1 -- 172.21.15.38:0/3068133084 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faabc01b070 con 0x7faacc0cd350 2024-04-25T08:38:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.160+0000 7faac1ffb640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faab006b4d0 0x7faab006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.161+0000 7faac1ffb640 1 -- 172.21.15.38:0/3068133084 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5249+0+0 (secure 0 0 0) 0x7faabc089310 con 0x7faacc0cd350 2024-04-25T08:38:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.161+0000 7faad1674640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faab006b4d0 0x7faab006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.161+0000 7faad1674640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faab006b4d0 0x7faab006d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faac8034000 tx=0x7faac802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:24.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.165+0000 7faac1ffb640 1 -- 172.21.15.38:0/3068133084 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faabc058350 con 0x7faacc0cd350 2024-04-25T08:38:24.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.345+0000 7faad38d8640 1 -- 172.21.15.38:0/3068133084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7faacc0cf4d0 con 0x7faacc0cd350 2024-04-25T08:38:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.715+0000 7faac1ffb640 1 -- 172.21.15.38:0/3068133084 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v93) v1 ==== 138+0+0 (secure 0 0 0) 0x7faabc087e20 con 0x7faacc0cd350 2024-04-25T08:38:24.709 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-04-25T08:38:24.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.718+0000 7faad38d8640 1 -- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faab006b4d0 msgr2=0x7faab006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.718+0000 7faad38d8640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faab006b4d0 0x7faab006d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7faac8034000 tx=0x7faac802e000 comp rx=0 tx=0).stop 2024-04-25T08:38:24.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.718+0000 7faad38d8640 1 -- 172.21.15.38:0/3068133084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faacc0cd350 msgr2=0x7faacc1f4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.718+0000 7faad38d8640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faacc0cd350 0x7faacc1f4900 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7faabc00c460 tx=0x7faabc00c920 comp rx=0 tx=0).stop 2024-04-25T08:38:24.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.718+0000 7faad38d8640 1 -- 172.21.15.38:0/3068133084 shutdown_connections 2024-04-25T08:38:24.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.718+0000 7faad38d8640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faacc15b940 0x7faacc15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.718+0000 7faad38d8640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faacc0cd350 0x7faacc1f4900 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.718+0000 7faad38d8640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faab006b4d0 0x7faab006d990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.718+0000 7faad38d8640 1 --2- 172.21.15.38:0/3068133084 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faacc09a180 0x7faacc0b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.718+0000 7faad38d8640 1 -- 172.21.15.38:0/3068133084 >> 172.21.15.38:0/3068133084 conn(0x7faacc0b3190 msgr2=0x7faacc073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:24.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.718+0000 7faad38d8640 1 -- 172.21.15.38:0/3068133084 shutdown_connections 2024-04-25T08:38:24.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.718+0000 7faad38d8640 1 -- 172.21.15.38:0/3068133084 wait complete. 2024-04-25T08:38:24.727 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:38:24.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.843+0000 7ff5f5ded640 1 Processor -- start 2024-04-25T08:38:24.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.843+0000 7ff5f5ded640 1 -- start start 2024-04-25T08:38:24.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.844+0000 7ff5f5ded640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff5f009a180 0x7ff5f009a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:24.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.844+0000 7ff5f5ded640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f015f300 con 0x7ff5f0162390 2024-04-25T08:38:24.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.844+0000 7ff5f5ded640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f015e540 con 0x7ff5f015df30 2024-04-25T08:38:24.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.844+0000 7ff5f5ded640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f015e310 con 0x7ff5f009a180 2024-04-25T08:38:24.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.844+0000 7ff5eeffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff5f009a180 0x7ff5f009a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:24.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.844+0000 7ff5eeffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff5f009a180 0x7ff5f009a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51902/0 (socket says 172.21.15.38:51902) 2024-04-25T08:38:24.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.844+0000 7ff5effff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff5f0162390 0x7ff5f00cf8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47336/0 (socket says 172.21.15.38:47336) 2024-04-25T08:38:24.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.844+0000 7ff5eeffd640 1 -- 172.21.15.38:0/1314066885 learned_addr learned my addr 172.21.15.38:0/1314066885 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:24.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.844+0000 7ff5eeffd640 1 -- 172.21.15.38:0/1314066885 >> v1:172.21.15.107:6789/0 conn(0x7ff5f015df30 legacy=0x7ff5f0099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.844+0000 7ff5eeffd640 1 -- 172.21.15.38:0/1314066885 >> v1:172.21.15.38:6789/0 conn(0x7ff5f0162390 legacy=0x7ff5f00cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.844+0000 7ff5eeffd640 1 -- 172.21.15.38:0/1314066885 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f015e820 con 0x7ff5f009a180 2024-04-25T08:38:24.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.845+0000 7ff5eeffd640 1 --2- 172.21.15.38:0/1314066885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f009a180 0x7ff5f009a7b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009870 tx=0x7ff5e402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6c9e468a66328a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:24.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.845+0000 7ff5ee7fc640 1 -- 172.21.15.38:0/1314066885 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4037070 con 0x7ff5f009a180 2024-04-25T08:38:24.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.845+0000 7ff5ee7fc640 1 -- 172.21.15.38:0/1314066885 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e402ae50 con 0x7ff5f009a180 2024-04-25T08:38:24.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.845+0000 7ff5ee7fc640 1 -- 172.21.15.38:0/1314066885 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4032860 con 0x7ff5f009a180 2024-04-25T08:38:24.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.846+0000 7ff5f5ded640 1 -- 172.21.15.38:0/1314066885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f009a180 msgr2=0x7ff5f009a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.846+0000 7ff5f5ded640 1 --2- 172.21.15.38:0/1314066885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f009a180 0x7ff5f009a7b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4009870 tx=0x7ff5e402a300 comp rx=0 tx=0).stop 2024-04-25T08:38:24.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.846+0000 7ff5f5ded640 1 -- 172.21.15.38:0/1314066885 shutdown_connections 2024-04-25T08:38:24.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.846+0000 7ff5f5ded640 1 --2- 172.21.15.38:0/1314066885 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f009a180 0x7ff5f009a7b0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.846+0000 7ff5f5ded640 1 -- 172.21.15.38:0/1314066885 >> 172.21.15.38:0/1314066885 conn(0x7ff5f00b3110 msgr2=0x7ff5f00b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:24.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.846+0000 7ff5f5ded640 1 -- 172.21.15.38:0/1314066885 shutdown_connections 2024-04-25T08:38:24.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.846+0000 7ff5f5ded640 1 -- 172.21.15.38:0/1314066885 wait complete. 2024-04-25T08:38:24.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.847+0000 7ff5f5ded640 1 Processor -- start 2024-04-25T08:38:24.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.847+0000 7ff5f5ded640 1 -- start start 2024-04-25T08:38:24.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5f5ded640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5f009a180 0x7ff5f00cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5f5ded640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5f015df30 0x7ff5f01596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5f5ded640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f0162390 0x7ff5f00cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5f5ded640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f015ee70 con 0x7ff5f0162390 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5f5ded640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f009aeb0 con 0x7ff5f009a180 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5f5ded640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f01627a0 con 0x7ff5f015df30 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f0162390 0x7ff5f00cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5effff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f0162390 0x7ff5f00cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51912/0 (socket says 172.21.15.38:51912) 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5effff640 1 -- 172.21.15.38:0/961587701 learned_addr learned my addr 172.21.15.38:0/961587701 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5ef7fe640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5f009a180 0x7ff5f00cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5eeffd640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5f015df30 0x7ff5f01596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5effff640 1 -- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5f015df30 msgr2=0x7ff5f01596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5effff640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5f015df30 0x7ff5f01596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5effff640 1 -- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5f009a180 msgr2=0x7ff5f00cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:24.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5effff640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5f009a180 0x7ff5f00cbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:24.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.848+0000 7ff5effff640 1 -- 172.21.15.38:0/961587701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f0159c30 con 0x7ff5f0162390 2024-04-25T08:38:24.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.849+0000 7ff5effff640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f0162390 0x7ff5f00cc940 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800c7b0 tx=0x7ff5e800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:24.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.849+0000 7ff5d7fff640 1 -- 172.21.15.38:0/961587701 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e8017070 con 0x7ff5f0162390 2024-04-25T08:38:24.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.849+0000 7ff5d7fff640 1 -- 172.21.15.38:0/961587701 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e800a640 con 0x7ff5f0162390 2024-04-25T08:38:24.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.849+0000 7ff5f5ded640 1 -- 172.21.15.38:0/961587701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f00c6540 con 0x7ff5f0162390 2024-04-25T08:38:24.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.849+0000 7ff5d7fff640 1 -- 172.21.15.38:0/961587701 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e800a9f0 con 0x7ff5f0162390 2024-04-25T08:38:24.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.849+0000 7ff5f5ded640 1 -- 172.21.15.38:0/961587701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f00c6b00 con 0x7ff5f0162390 2024-04-25T08:38:24.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.850+0000 7ff5f5ded640 1 -- 172.21.15.38:0/961587701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f0068e80 con 0x7ff5f0162390 2024-04-25T08:38:24.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.850+0000 7ff5d7fff640 1 -- 172.21.15.38:0/961587701 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5e801a070 con 0x7ff5f0162390 2024-04-25T08:38:24.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.851+0000 7ff5d7fff640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff5dc06b500 0x7ff5dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:24.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.851+0000 7ff5d7fff640 1 -- 172.21.15.38:0/961587701 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5619+0+0 (secure 0 0 0) 0x7ff5e8089700 con 0x7ff5f0162390 2024-04-25T08:38:24.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.851+0000 7ff5ef7fe640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff5dc06b500 0x7ff5dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:24.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.852+0000 7ff5ef7fe640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff5dc06b500 0x7ff5dc06d9c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0001780 tx=0x7ff5e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:24.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:24.855+0000 7ff5d7fff640 1 -- 172.21.15.38:0/961587701 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5e8057f00 con 0x7ff5f0162390 2024-04-25T08:38:25.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.037+0000 7ff5f5ded640 1 -- 172.21.15.38:0/961587701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff5f00b4020 con 0x7ff5f0162390 2024-04-25T08:38:25.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.723+0000 7ff5d7fff640 1 -- 172.21.15.38:0/961587701 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v94) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff5e805bfa0 con 0x7ff5f0162390 2024-04-25T08:38:25.716 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-04-25T08:38:25.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.725+0000 7ff5f5ded640 1 -- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff5dc06b500 msgr2=0x7ff5dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.725+0000 7ff5f5ded640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff5dc06b500 0x7ff5dc06d9c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0001780 tx=0x7ff5e0009040 comp rx=0 tx=0).stop 2024-04-25T08:38:25.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.725+0000 7ff5f5ded640 1 -- 172.21.15.38:0/961587701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f0162390 msgr2=0x7ff5f00cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.725+0000 7ff5f5ded640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f0162390 0x7ff5f00cc940 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800c7b0 tx=0x7ff5e800cc70 comp rx=0 tx=0).stop 2024-04-25T08:38:25.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.725+0000 7ff5f5ded640 1 -- 172.21.15.38:0/961587701 shutdown_connections 2024-04-25T08:38:25.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.725+0000 7ff5f5ded640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff5f0162390 0x7ff5f00cc940 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.725+0000 7ff5f5ded640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff5dc06b500 0x7ff5dc06d9c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.725+0000 7ff5f5ded640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff5f015df30 0x7ff5f01596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.725+0000 7ff5f5ded640 1 --2- 172.21.15.38:0/961587701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5f009a180 0x7ff5f00cbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.725+0000 7ff5f5ded640 1 -- 172.21.15.38:0/961587701 >> 172.21.15.38:0/961587701 conn(0x7ff5f00b3110 msgr2=0x7ff5f00cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:25.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.726+0000 7ff5f5ded640 1 -- 172.21.15.38:0/961587701 shutdown_connections 2024-04-25T08:38:25.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.726+0000 7ff5f5ded640 1 -- 172.21.15.38:0/961587701 wait complete. 2024-04-25T08:38:25.742 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:38:25.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.858+0000 7f7c96989640 1 Processor -- start 2024-04-25T08:38:25.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.858+0000 7f7c96989640 1 -- start start 2024-04-25T08:38:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c96989640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c900d0320 0x7f7c900d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c96989640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c900d2060 con 0x7f7c900b6980 2024-04-25T08:38:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c96989640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c900d2330 con 0x7f7c900d0700 2024-04-25T08:38:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c96989640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c900d2610 con 0x7f7c900d0320 2024-04-25T08:38:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c94f26640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7c900b6980 0x7f7c900b6e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47346/0 (socket says 172.21.15.38:47346) 2024-04-25T08:38:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c94f26640 1 -- 172.21.15.38:0/2513422221 learned_addr learned my addr 172.21.15.38:0/2513422221 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:25.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c8ffff640 1 --2- 172.21.15.38:0/2513422221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c900d0320 0x7f7c900d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c8effd640 1 -- 172.21.15.38:0/2513422221 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633673573 0 0) 0x7f7c900d2060 con 0x7f7c900b6980 2024-04-25T08:38:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c8effd640 1 -- 172.21.15.38:0/2513422221 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c78003520 con 0x7f7c900b6980 2024-04-25T08:38:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c8effd640 1 -- 172.21.15.38:0/2513422221 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805045547 0 0) 0x7f7c900d2330 con 0x7f7c900d0700 2024-04-25T08:38:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c8effd640 1 -- 172.21.15.38:0/2513422221 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c900d2060 con 0x7f7c900d0700 2024-04-25T08:38:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c8ffff640 1 -- 172.21.15.38:0/2513422221 >> v1:172.21.15.107:6789/0 conn(0x7f7c900d0700 legacy=0x7f7c900d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c8ffff640 1 -- 172.21.15.38:0/2513422221 >> v1:172.21.15.38:6789/0 conn(0x7f7c900b6980 legacy=0x7f7c900b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.859+0000 7f7c8ffff640 1 -- 172.21.15.38:0/2513422221 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c900d28f0 con 0x7f7c900d0320 2024-04-25T08:38:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.860+0000 7f7c8ffff640 1 --2- 172.21.15.38:0/2513422221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c900d0320 0x7f7c900d33d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7c800097d0 tx=0x7f7c8002a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c2c902692a60f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.860+0000 7f7c8effd640 1 -- 172.21.15.38:0/2513422221 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80037070 con 0x7f7c900d0320 2024-04-25T08:38:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.860+0000 7f7c8effd640 1 -- 172.21.15.38:0/2513422221 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c80032440 con 0x7f7c900d0320 2024-04-25T08:38:25.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.860+0000 7f7c8effd640 1 -- 172.21.15.38:0/2513422221 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c800327f0 con 0x7f7c900d0320 2024-04-25T08:38:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.861+0000 7f7c96989640 1 -- 172.21.15.38:0/2513422221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c900d0320 msgr2=0x7f7c900d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.861+0000 7f7c96989640 1 --2- 172.21.15.38:0/2513422221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c900d0320 0x7f7c900d33d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7c800097d0 tx=0x7f7c8002a290 comp rx=0 tx=0).stop 2024-04-25T08:38:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.861+0000 7f7c96989640 1 -- 172.21.15.38:0/2513422221 shutdown_connections 2024-04-25T08:38:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.861+0000 7f7c96989640 1 --2- 172.21.15.38:0/2513422221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c900d0320 0x7f7c900d33d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.861+0000 7f7c96989640 1 -- 172.21.15.38:0/2513422221 >> 172.21.15.38:0/2513422221 conn(0x7f7c900668f0 msgr2=0x7f7c90066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.861+0000 7f7c96989640 1 -- 172.21.15.38:0/2513422221 shutdown_connections 2024-04-25T08:38:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.861+0000 7f7c96989640 1 -- 172.21.15.38:0/2513422221 wait complete. 2024-04-25T08:38:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.862+0000 7f7c96989640 1 Processor -- start 2024-04-25T08:38:25.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.862+0000 7f7c96989640 1 -- start start 2024-04-25T08:38:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c96989640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c900b6980 0x7f7c9014fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c96989640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c900d0320 0x7f7c9014c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c96989640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c900d0700 0x7f7c900aa550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c96989640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c900d0b40 con 0x7f7c900d0700 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c96989640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c900d2eb0 con 0x7f7c900b6980 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c96989640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9014c9f0 con 0x7f7c900d0320 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c94f26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c900d0700 0x7f7c900aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c94f26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c900d0700 0x7f7c900aa550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51922/0 (socket says 172.21.15.38:51922) 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c94f26640 1 -- 172.21.15.38:0/3338924300 learned_addr learned my addr 172.21.15.38:0/3338924300 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c8ffff640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c900b6980 0x7f7c9014fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c8f7fe640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c900d0320 0x7f7c9014c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c94f26640 1 -- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c900d0320 msgr2=0x7f7c9014c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c94f26640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c900d0320 0x7f7c9014c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c94f26640 1 -- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c900b6980 msgr2=0x7f7c9014fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c94f26640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c900b6980 0x7f7c9014fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c94f26640 1 -- 172.21.15.38:0/3338924300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9014cc90 con 0x7f7c900d0700 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.863+0000 7f7c8ffff640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c900b6980 0x7f7c9014fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:25.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.864+0000 7f7c94f26640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c900d0700 0x7f7c900aa550 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800cb40 tx=0x7f7c8800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.864+0000 7f7c8cff9640 1 -- 172.21.15.38:0/3338924300 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c88017070 con 0x7f7c900d0700 2024-04-25T08:38:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.864+0000 7f7c8cff9640 1 -- 172.21.15.38:0/3338924300 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8800aca0 con 0x7f7c900d0700 2024-04-25T08:38:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.864+0000 7f7c96989640 1 -- 172.21.15.38:0/3338924300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9014cf70 con 0x7f7c900d0700 2024-04-25T08:38:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.864+0000 7f7c8cff9640 1 -- 172.21.15.38:0/3338924300 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c88012600 con 0x7f7c900d0700 2024-04-25T08:38:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.864+0000 7f7c96989640 1 -- 172.21.15.38:0/3338924300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9014c5e0 con 0x7f7c900d0700 2024-04-25T08:38:25.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.865+0000 7f7c96989640 1 -- 172.21.15.38:0/3338924300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c90068e80 con 0x7f7c900d0700 2024-04-25T08:38:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.865+0000 7f7c8cff9640 1 -- 172.21.15.38:0/3338924300 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c8801a070 con 0x7f7c900d0700 2024-04-25T08:38:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.866+0000 7f7c8cff9640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c7c06b4d0 0x7f7c7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.866+0000 7f7c8cff9640 1 -- 172.21.15.38:0/3338924300 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f7c8808a0d0 con 0x7f7c900d0700 2024-04-25T08:38:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.866+0000 7f7c8ffff640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c7c06b4d0 0x7f7c7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:25.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.867+0000 7f7c8ffff640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c7c06b4d0 0x7f7c7c06d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7c80034000 tx=0x7f7c8002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:25.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:25.870+0000 7f7c8cff9640 1 -- 172.21.15.38:0/3338924300 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c88058740 con 0x7f7c900d0700 2024-04-25T08:38:26.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.093+0000 7f7c96989640 1 -- 172.21.15.38:0/3338924300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7c900ceea0 con 0x7f7c900d0700 2024-04-25T08:38:26.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.724+0000 7f7c8cff9640 1 -- 172.21.15.38:0/3338924300 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v55) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7c8805c7e0 con 0x7f7c900d0700 2024-04-25T08:38:26.718 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T08:38:26.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.728+0000 7f7c96989640 1 -- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c7c06b4d0 msgr2=0x7f7c7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.728+0000 7f7c96989640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c7c06b4d0 0x7f7c7c06d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f7c80034000 tx=0x7f7c8002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.728+0000 7f7c96989640 1 -- 172.21.15.38:0/3338924300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c900d0700 msgr2=0x7f7c900aa550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.728+0000 7f7c96989640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c900d0700 0x7f7c900aa550 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800cb40 tx=0x7f7c8800a060 comp rx=0 tx=0).stop 2024-04-25T08:38:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.728+0000 7f7c96989640 1 -- 172.21.15.38:0/3338924300 shutdown_connections 2024-04-25T08:38:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.728+0000 7f7c96989640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c900d0700 0x7f7c900aa550 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.728+0000 7f7c96989640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c7c06b4d0 0x7f7c7c06d990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.728+0000 7f7c96989640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c900d0320 0x7f7c9014c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.728+0000 7f7c96989640 1 --2- 172.21.15.38:0/3338924300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c900b6980 0x7f7c9014fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.728+0000 7f7c96989640 1 -- 172.21.15.38:0/3338924300 >> 172.21.15.38:0/3338924300 conn(0x7f7c900668f0 msgr2=0x7f7c900c9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.728+0000 7f7c96989640 1 -- 172.21.15.38:0/3338924300 shutdown_connections 2024-04-25T08:38:26.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.728+0000 7f7c96989640 1 -- 172.21.15.38:0/3338924300 wait complete. 2024-04-25T08:38:26.747 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.864+0000 7fd313d13640 1 Processor -- start 2024-04-25T08:38:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.864+0000 7fd313d13640 1 -- start start 2024-04-25T08:38:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.865+0000 7fd313d13640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd30c0cb890 0x7fd30c0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.865+0000 7fd313d13640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd30c0d0780 0x7fd30c0c4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.865+0000 7fd313d13640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30c0cbd20 con 0x7fd30c15a8f0 2024-04-25T08:38:26.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.865+0000 7fd313d13640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd30c0cc000 con 0x7fd30c0cb890 2024-04-25T08:38:26.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.865+0000 7fd313d13640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0cc2e0 con 0x7fd30c0d0780 2024-04-25T08:38:26.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.865+0000 7fd3112ae640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd30c0cb890 0x7fd30c0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.865+0000 7fd3112ae640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd30c0cb890 0x7fd30c0d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51926/0 (socket says 172.21.15.38:51926) 2024-04-25T08:38:26.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.865+0000 7fd3112ae640 1 -- 172.21.15.38:0/1754769 learned_addr learned my addr 172.21.15.38:0/1754769 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:26.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.866+0000 7fd3122b0640 1 --2- 172.21.15.38:0/1754769 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd30c0d0780 0x7fd30c0c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.866+0000 7fd3112ae640 1 -- 172.21.15.38:0/1754769 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd30c0d0780 msgr2=0x7fd30c0c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:26.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.866+0000 7fd3112ae640 1 --2- 172.21.15.38:0/1754769 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd30c0d0780 0x7fd30c0c4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.866+0000 7fd3112ae640 1 -- 172.21.15.38:0/1754769 >> v1:172.21.15.107:6789/0 conn(0x7fd30c15a8f0 legacy=0x7fd30c0cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:26.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.866+0000 7fd3112ae640 1 -- 172.21.15.38:0/1754769 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c0cc5c0 con 0x7fd30c0cb890 2024-04-25T08:38:26.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.866+0000 7fd3122b0640 1 --2- 172.21.15.38:0/1754769 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd30c0d0780 0x7fd30c0c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:26.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.866+0000 7fd3112ae640 1 --2- 172.21.15.38:0/1754769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd30c0cb890 0x7fd30c0d0240 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fd300009870 tx=0x7fd30002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c577e049ec700e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.867+0000 7fd310aad640 1 -- 172.21.15.38:0/1754769 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300037070 con 0x7fd30c0cb890 2024-04-25T08:38:26.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.867+0000 7fd310aad640 1 -- 172.21.15.38:0/1754769 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300032440 con 0x7fd30c0cb890 2024-04-25T08:38:26.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.867+0000 7fd310aad640 1 -- 172.21.15.38:0/1754769 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300032810 con 0x7fd30c0cb890 2024-04-25T08:38:26.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.867+0000 7fd313d13640 1 -- 172.21.15.38:0/1754769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd30c0cb890 msgr2=0x7fd30c0d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.867+0000 7fd313d13640 1 --2- 172.21.15.38:0/1754769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd30c0cb890 0x7fd30c0d0240 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fd300009870 tx=0x7fd30002a200 comp rx=0 tx=0).stop 2024-04-25T08:38:26.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.867+0000 7fd313d13640 1 -- 172.21.15.38:0/1754769 shutdown_connections 2024-04-25T08:38:26.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.867+0000 7fd313d13640 1 --2- 172.21.15.38:0/1754769 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd30c0d0780 0x7fd30c0c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.867+0000 7fd313d13640 1 --2- 172.21.15.38:0/1754769 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd30c0cb890 0x7fd30c0d0240 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.867+0000 7fd313d13640 1 -- 172.21.15.38:0/1754769 >> 172.21.15.38:0/1754769 conn(0x7fd30c072610 msgr2=0x7fd30c099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:26.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.867+0000 7fd313d13640 1 -- 172.21.15.38:0/1754769 shutdown_connections 2024-04-25T08:38:26.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.867+0000 7fd313d13640 1 -- 172.21.15.38:0/1754769 wait complete. 2024-04-25T08:38:26.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.868+0000 7fd313d13640 1 Processor -- start 2024-04-25T08:38:26.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.868+0000 7fd313d13640 1 -- start start 2024-04-25T08:38:26.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.869+0000 7fd313d13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd30c0d0780 0x7fd30c1f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.869+0000 7fd313d13640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd30c15a8f0 0x7fd30c0c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.869+0000 7fd313d13640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd30c0c3580 0x7fd30c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.869+0000 7fd313d13640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0ce470 con 0x7fd30c0d0780 2024-04-25T08:38:26.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.869+0000 7fd313d13640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c0d0c10 con 0x7fd30c15a8f0 2024-04-25T08:38:26.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.869+0000 7fd313d13640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c15acd0 con 0x7fd30c0c3580 2024-04-25T08:38:26.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.869+0000 7fd311aaf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd30c0d0780 0x7fd30c1f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.869+0000 7fd311aaf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd30c0d0780 0x7fd30c1f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51932/0 (socket says 172.21.15.38:51932) 2024-04-25T08:38:26.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.869+0000 7fd311aaf640 1 -- 172.21.15.38:0/2137121955 learned_addr learned my addr 172.21.15.38:0/2137121955 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:26.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd3122b0640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd30c0c3580 0x7fd30c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd311aaf640 1 -- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd30c0c3580 msgr2=0x7fd30c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd3112ae640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd30c15a8f0 0x7fd30c0c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd311aaf640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd30c0c3580 0x7fd30c06abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd311aaf640 1 -- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd30c15a8f0 msgr2=0x7fd30c0c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd311aaf640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd30c15a8f0 0x7fd30c0c3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd311aaf640 1 -- 172.21.15.38:0/2137121955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c0c88c0 con 0x7fd30c0d0780 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd311aaf640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd30c0d0780 0x7fd30c1f4330 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00ba50 tx=0x7fd2fc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd30a7fc640 1 -- 172.21.15.38:0/2137121955 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc019070 con 0x7fd30c0d0780 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd30a7fc640 1 -- 172.21.15.38:0/2137121955 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2fc00cbc0 con 0x7fd30c0d0780 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd313d13640 1 -- 172.21.15.38:0/2137121955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c0c8ba0 con 0x7fd30c0d0780 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd30a7fc640 1 -- 172.21.15.38:0/2137121955 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc014440 con 0x7fd30c0d0780 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.870+0000 7fd313d13640 1 -- 172.21.15.38:0/2137121955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c0c9160 con 0x7fd30c0d0780 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.871+0000 7fd313d13640 1 -- 172.21.15.38:0/2137121955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c0c6b20 con 0x7fd30c0d0780 2024-04-25T08:38:26.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.871+0000 7fd30a7fc640 1 -- 172.21.15.38:0/2137121955 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd2fc01c070 con 0x7fd30c0d0780 2024-04-25T08:38:26.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.872+0000 7fd30a7fc640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd2f806b4d0 0x7fd2f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:26.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.872+0000 7fd30a7fc640 1 -- 172.21.15.38:0/2137121955 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2fc089f50 con 0x7fd30c0d0780 2024-04-25T08:38:26.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.872+0000 7fd3112ae640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd2f806b4d0 0x7fd2f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:26.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.873+0000 7fd3112ae640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd2f806b4d0 0x7fd2f806d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd300034000 tx=0x7fd30002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:26.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:26.877+0000 7fd30a7fc640 1 -- 172.21.15.38:0/2137121955 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd2fc058680 con 0x7fd30c0d0780 2024-04-25T08:38:27.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.057+0000 7fd313d13640 1 -- 172.21.15.38:0/2137121955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd30c159d40 con 0x7fd30c0d0780 2024-04-25T08:38:27.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.058+0000 7fd30a7fc640 1 -- 172.21.15.38:0/2137121955 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16886 (secure 0 0 0) 0x7fd2fc05c720 con 0x7fd30c0d0780 2024-04-25T08:38:27.052 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:27.052 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":95,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:38:26.095079+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:24.377232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:25.069513+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:27.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.060+0000 7fd313d13640 1 -- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd2f806b4d0 msgr2=0x7fd2f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.060+0000 7fd313d13640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd2f806b4d0 0x7fd2f806d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd300034000 tx=0x7fd30002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:27.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.060+0000 7fd313d13640 1 -- 172.21.15.38:0/2137121955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd30c0d0780 msgr2=0x7fd30c1f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.060+0000 7fd313d13640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd30c0d0780 0x7fd30c1f4330 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00ba50 tx=0x7fd2fc00bf10 comp rx=0 tx=0).stop 2024-04-25T08:38:27.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.060+0000 7fd313d13640 1 -- 172.21.15.38:0/2137121955 shutdown_connections 2024-04-25T08:38:27.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.060+0000 7fd313d13640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd2f806b4d0 0x7fd2f806d990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.060+0000 7fd313d13640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd30c0c3580 0x7fd30c06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.060+0000 7fd313d13640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd30c15a8f0 0x7fd30c0c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.060+0000 7fd313d13640 1 --2- 172.21.15.38:0/2137121955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd30c0d0780 0x7fd30c1f4330 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.060+0000 7fd313d13640 1 -- 172.21.15.38:0/2137121955 >> 172.21.15.38:0/2137121955 conn(0x7fd30c072610 msgr2=0x7fd30c0c4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:27.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.060+0000 7fd313d13640 1 -- 172.21.15.38:0/2137121955 shutdown_connections 2024-04-25T08:38:27.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.060+0000 7fd313d13640 1 -- 172.21.15.38:0/2137121955 wait complete. 2024-04-25T08:38:27.066 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:27.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.174+0000 7f0ff57ce640 1 Processor -- start 2024-04-25T08:38:27.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.174+0000 7f0ff57ce640 1 -- start start 2024-04-25T08:38:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0ff57ce640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0ff015df50 0x7f0ff00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0ff57ce640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0ff00ce2e0 0x7f0ff00d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0ff57ce640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff00ceff0 con 0x7f0ff006c110 2024-04-25T08:38:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0ff57ce640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff00cf2d0 con 0x7f0ff015df50 2024-04-25T08:38:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0ff57ce640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff00d1f70 con 0x7f0ff00ce2e0 2024-04-25T08:38:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0fe7fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0ff015df50 0x7f0ff00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0fe7fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0ff015df50 0x7f0ff00ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51948/0 (socket says 172.21.15.38:51948) 2024-04-25T08:38:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0fe7fff640 1 -- 172.21.15.38:0/1214627608 learned_addr learned my addr 172.21.15.38:0/1214627608 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:27.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0fef7fe640 1 --2- 172.21.15.38:0/1214627608 >> v2:172.21.15.107:3300/0 conn(0x7f0ff00ce2e0 0x7f0ff00d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0fe7fff640 1 -- 172.21.15.38:0/1214627608 >> v2:172.21.15.107:3300/0 conn(0x7f0ff00ce2e0 msgr2=0x7f0ff00d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0fe7fff640 1 --2- 172.21.15.38:0/1214627608 >> v2:172.21.15.107:3300/0 conn(0x7f0ff00ce2e0 0x7f0ff00d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0fe7fff640 1 -- 172.21.15.38:0/1214627608 >> v1:172.21.15.107:6789/0 conn(0x7f0ff006c110 legacy=0x7f0ff006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0fe7fff640 1 -- 172.21.15.38:0/1214627608 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff00d2250 con 0x7f0ff015df50 2024-04-25T08:38:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.175+0000 7f0fef7fe640 1 --2- 172.21.15.38:0/1214627608 >> v2:172.21.15.107:3300/0 conn(0x7f0ff00ce2e0 0x7f0ff00d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.176+0000 7f0fe7fff640 1 --2- 172.21.15.38:0/1214627608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff015df50 0x7f0ff00ce7a0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00b6520 tx=0x7f0fd802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f882be24c2d03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.176+0000 7f0fee7fc640 1 -- 172.21.15.38:0/1214627608 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8037070 con 0x7f0ff015df50 2024-04-25T08:38:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.176+0000 7f0fee7fc640 1 -- 172.21.15.38:0/1214627608 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd8032440 con 0x7f0ff015df50 2024-04-25T08:38:27.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.176+0000 7f0fee7fc640 1 -- 172.21.15.38:0/1214627608 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd8032810 con 0x7f0ff015df50 2024-04-25T08:38:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.177+0000 7f0ff57ce640 1 -- 172.21.15.38:0/1214627608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff015df50 msgr2=0x7f0ff00ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.177+0000 7f0ff57ce640 1 --2- 172.21.15.38:0/1214627608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff015df50 0x7f0ff00ce7a0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00b6520 tx=0x7f0fd802a270 comp rx=0 tx=0).stop 2024-04-25T08:38:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.177+0000 7f0ff57ce640 1 -- 172.21.15.38:0/1214627608 shutdown_connections 2024-04-25T08:38:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.177+0000 7f0ff57ce640 1 --2- 172.21.15.38:0/1214627608 >> v2:172.21.15.107:3300/0 conn(0x7f0ff00ce2e0 0x7f0ff00d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.177+0000 7f0ff57ce640 1 --2- 172.21.15.38:0/1214627608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff015df50 0x7f0ff00ce7a0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.177+0000 7f0ff57ce640 1 -- 172.21.15.38:0/1214627608 >> 172.21.15.38:0/1214627608 conn(0x7f0ff00d3fb0 msgr2=0x7f0ff00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.177+0000 7f0ff57ce640 1 -- 172.21.15.38:0/1214627608 shutdown_connections 2024-04-25T08:38:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.177+0000 7f0ff57ce640 1 -- 172.21.15.38:0/1214627608 wait complete. 2024-04-25T08:38:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.178+0000 7f0ff57ce640 1 Processor -- start 2024-04-25T08:38:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.178+0000 7f0ff57ce640 1 -- start start 2024-04-25T08:38:27.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0ff57ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ff006c110 0x7f0ff00ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0ff57ce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ff00ce2e0 0x7f0ff01450b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0ff57ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff015df50 0x7f0ff01f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0ff57ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00d2f90 con 0x7f0ff015df50 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0ff57ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00cece0 con 0x7f0ff006c110 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0ff57ce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00d3110 con 0x7f0ff00ce2e0 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0fef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff015df50 0x7f0ff01f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0fef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff015df50 0x7f0ff01f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51950/0 (socket says 172.21.15.38:51950) 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0fef7fe640 1 -- 172.21.15.38:0/3818578070 learned_addr learned my addr 172.21.15.38:0/3818578070 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0fe7fff640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ff00ce2e0 0x7f0ff01450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0feeffd640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ff006c110 0x7f0ff00ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0fef7fe640 1 -- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ff00ce2e0 msgr2=0x7f0ff01450b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0fef7fe640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ff00ce2e0 0x7f0ff01450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0fef7fe640 1 -- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ff006c110 msgr2=0x7f0ff00ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0fef7fe640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ff006c110 0x7f0ff00ca000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0fef7fe640 1 -- 172.21.15.38:0/3818578070 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff01f4020 con 0x7f0ff015df50 2024-04-25T08:38:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.179+0000 7f0fef7fe640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff015df50 0x7f0ff01f3530 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8007ef0 tx=0x7f0fe800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.180+0000 7f0fe77fe640 1 -- 172.21.15.38:0/3818578070 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe801a070 con 0x7f0ff015df50 2024-04-25T08:38:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.180+0000 7f0fe77fe640 1 -- 172.21.15.38:0/3818578070 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe8003b90 con 0x7f0ff015df50 2024-04-25T08:38:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.180+0000 7f0ff57ce640 1 -- 172.21.15.38:0/3818578070 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff01f4300 con 0x7f0ff015df50 2024-04-25T08:38:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.180+0000 7f0fe77fe640 1 -- 172.21.15.38:0/3818578070 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8004030 con 0x7f0ff015df50 2024-04-25T08:38:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.180+0000 7f0ff57ce640 1 -- 172.21.15.38:0/3818578070 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff01448b0 con 0x7f0ff015df50 2024-04-25T08:38:27.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.181+0000 7f0ff57ce640 1 -- 172.21.15.38:0/3818578070 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff0068e80 con 0x7f0ff015df50 2024-04-25T08:38:27.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.181+0000 7f0fe77fe640 1 -- 172.21.15.38:0/3818578070 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fe801d070 con 0x7f0ff015df50 2024-04-25T08:38:27.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.181+0000 7f0fe77fe640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0fd006b4d0 0x7f0fd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.182+0000 7f0fe77fe640 1 -- 172.21.15.38:0/3818578070 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0fe808a5d0 con 0x7f0ff015df50 2024-04-25T08:38:27.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.182+0000 7f0feeffd640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0fd006b4d0 0x7f0fd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.182+0000 7f0feeffd640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0fd006b4d0 0x7f0fd006d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00015a0 tx=0x7f0fe0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:27.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.186+0000 7f0fe77fe640 1 -- 172.21.15.38:0/3818578070 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0fe8010070 con 0x7f0ff015df50 2024-04-25T08:38:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.368+0000 7f0ff57ce640 1 -- 172.21.15.38:0/3818578070 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ff00d1710 con 0x7f0ff015df50 2024-04-25T08:38:27.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.370+0000 7f0fe77fe640 1 -- 172.21.15.38:0/3818578070 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16886 (secure 0 0 0) 0x7f0fe8058c70 con 0x7f0ff015df50 2024-04-25T08:38:27.363 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:27.364 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":95,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:38:26.095079+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:24.377232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:25.069513+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.372+0000 7f0ff57ce640 1 -- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0fd006b4d0 msgr2=0x7f0fd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.372+0000 7f0ff57ce640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0fd006b4d0 0x7f0fd006d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00015a0 tx=0x7f0fe0009040 comp rx=0 tx=0).stop 2024-04-25T08:38:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.372+0000 7f0ff57ce640 1 -- 172.21.15.38:0/3818578070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff015df50 msgr2=0x7f0ff01f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.372+0000 7f0ff57ce640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff015df50 0x7f0ff01f3530 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8007ef0 tx=0x7f0fe800d4a0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.372+0000 7f0ff57ce640 1 -- 172.21.15.38:0/3818578070 shutdown_connections 2024-04-25T08:38:27.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.372+0000 7f0ff57ce640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff015df50 0x7f0ff01f3530 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.372+0000 7f0ff57ce640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0fd006b4d0 0x7f0fd006d990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.372+0000 7f0ff57ce640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0ff00ce2e0 0x7f0ff01450b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.372+0000 7f0ff57ce640 1 --2- 172.21.15.38:0/3818578070 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0ff006c110 0x7f0ff00ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.372+0000 7f0ff57ce640 1 -- 172.21.15.38:0/3818578070 >> 172.21.15.38:0/3818578070 conn(0x7f0ff00d3fb0 msgr2=0x7f0ff0073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.372+0000 7f0ff57ce640 1 -- 172.21.15.38:0/3818578070 shutdown_connections 2024-04-25T08:38:27.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.372+0000 7f0ff57ce640 1 -- 172.21.15.38:0/3818578070 wait complete. 2024-04-25T08:38:27.378 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:38:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.487+0000 7f6ad4753640 1 Processor -- start 2024-04-25T08:38:27.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.488+0000 7f6ad4753640 1 -- start start 2024-04-25T08:38:27.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.488+0000 7f6ad4753640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0d33d0 0x7f6acc0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.488+0000 7f6ad4753640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6acc0d2330 con 0x7f6acc0d1e90 2024-04-25T08:38:27.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.488+0000 7f6ad4753640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6acc0d2610 con 0x7f6acc0d0320 2024-04-25T08:38:27.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.488+0000 7f6ad4753640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0d28f0 con 0x7f6acc0d33d0 2024-04-25T08:38:27.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad2cf0640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6acc0d1e90 0x7f6acc0b6980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47348/0 (socket says 172.21.15.38:47348) 2024-04-25T08:38:27.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad2cf0640 1 -- 172.21.15.38:0/3083025102 learned_addr learned my addr 172.21.15.38:0/3083025102 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:27.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad1cee640 1 --2- 172.21.15.38:0/3083025102 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0d33d0 0x7f6acc0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217191241 0 0) 0x7f6acc0d2330 con 0x7f6acc0d1e90 2024-04-25T08:38:27.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ab8003520 con 0x7f6acc0d1e90 2024-04-25T08:38:27.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873887873 0 0) 0x7f6acc0d2610 con 0x7f6acc0d0320 2024-04-25T08:38:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6acc0d2330 con 0x7f6acc0d0320 2024-04-25T08:38:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3714878529 0 0) 0x7f6ab8003520 con 0x7f6acc0d1e90 2024-04-25T08:38:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0d33d0 msgr2=0x7f6acc0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 --2- 172.21.15.38:0/3083025102 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0d33d0 0x7f6acc0d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 >> v1:172.21.15.107:6789/0 conn(0x7f6acc0d0320 legacy=0x7f6acc0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc0d2bd0 con 0x7f6acc0d1e90 2024-04-25T08:38:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f6ac40019f0 con 0x7f6acc0d1e90 2024-04-25T08:38:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 >> v1:172.21.15.38:6789/0 conn(0x7f6acc0d1e90 legacy=0x7f6acc0b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 --2- 172.21.15.38:0/3083025102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab8003b70 0x7f6ab8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 --2- 172.21.15.38:0/3083025102 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0d33d0 0x7f6ab8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 --2- 172.21.15.38:0/3083025102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8004d40 0x7f6ab80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0d28f0 con 0x7f6ab8004d40 2024-04-25T08:38:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab800ba70 con 0x7f6ab8003b70 2024-04-25T08:38:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab800bc20 con 0x7f6acc0d33d0 2024-04-25T08:38:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad2cf0640 1 --2- 172.21.15.38:0/3083025102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8004d40 0x7f6ab80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad1cee640 1 --2- 172.21.15.38:0/3083025102 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0d33d0 0x7f6ab8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.489+0000 7f6ad24ef640 1 --2- 172.21.15.38:0/3083025102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab8003b70 0x7f6ab8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.490+0000 7f6ad2cf0640 1 -- 172.21.15.38:0/3083025102 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0d33d0 msgr2=0x7f6ab8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.490+0000 7f6ad2cf0640 1 --2- 172.21.15.38:0/3083025102 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0d33d0 0x7f6ab8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.490+0000 7f6ad2cf0640 1 -- 172.21.15.38:0/3083025102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab8003b70 msgr2=0x7f6ab8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.490+0000 7f6ad2cf0640 1 --2- 172.21.15.38:0/3083025102 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6ab8003b70 0x7f6ab8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.490+0000 7f6ad2cf0640 1 -- 172.21.15.38:0/3083025102 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab8003520 con 0x7f6ab8004d40 2024-04-25T08:38:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.490+0000 7f6ad2cf0640 1 --2- 172.21.15.38:0/3083025102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8004d40 0x7f6ab80051c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400a7c0 tx=0x7f6ab8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7ca160789b404f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.490+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac400a410 con 0x7f6ab8004d40 2024-04-25T08:38:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.491+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac402f070 con 0x7f6ab8004d40 2024-04-25T08:38:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.491+0000 7f6ad14ed640 1 -- 172.21.15.38:0/3083025102 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac402bc00 con 0x7f6ab8004d40 2024-04-25T08:38:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.491+0000 7f6ad4753640 1 -- 172.21.15.38:0/3083025102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8004d40 msgr2=0x7f6ab80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.491+0000 7f6ad4753640 1 --2- 172.21.15.38:0/3083025102 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8004d40 0x7f6ab80051c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400a7c0 tx=0x7f6ab8005830 comp rx=0 tx=0).stop 2024-04-25T08:38:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.491+0000 7f6ad2cf0640 1 -- 172.21.15.38:0/3083025102 reap_dead start 2024-04-25T08:38:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.491+0000 7f6ad4753640 1 -- 172.21.15.38:0/3083025102 shutdown_connections 2024-04-25T08:38:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.491+0000 7f6ad4753640 1 -- 172.21.15.38:0/3083025102 >> 172.21.15.38:0/3083025102 conn(0x7f6acc0668f0 msgr2=0x7f6acc066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.491+0000 7f6ad4753640 1 -- 172.21.15.38:0/3083025102 shutdown_connections 2024-04-25T08:38:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.491+0000 7f6ad4753640 1 -- 172.21.15.38:0/3083025102 wait complete. 2024-04-25T08:38:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.492+0000 7f6ad4753640 1 Processor -- start 2024-04-25T08:38:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.492+0000 7f6ad4753640 1 -- start start 2024-04-25T08:38:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad4753640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8004d40 0x7f6acc0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad4753640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6acc0b42c0 0x7f6acc0b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad4753640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0c9140 0x7f6acc0c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad4753640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc161390 con 0x7f6ab8004d40 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad4753640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc161210 con 0x7f6acc0b42c0 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad4753640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0b9720 con 0x7f6acc0c9140 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad24ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8004d40 0x7f6acc0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad24ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8004d40 0x7f6acc0ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51970/0 (socket says 172.21.15.38:51970) 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad24ef640 1 -- 172.21.15.38:0/2668995606 learned_addr learned my addr 172.21.15.38:0/2668995606 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad1cee640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6acc0b42c0 0x7f6acc0b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad2cf0640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0c9140 0x7f6acc0c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad24ef640 1 -- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0c9140 msgr2=0x7f6acc0c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.493+0000 7f6ad24ef640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0c9140 0x7f6acc0c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.494+0000 7f6ad24ef640 1 -- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6acc0b42c0 msgr2=0x7f6acc0b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.494+0000 7f6ad24ef640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6acc0b42c0 0x7f6acc0b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.494+0000 7f6ad24ef640 1 -- 172.21.15.38:0/2668995606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc0c9b00 con 0x7f6ab8004d40 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.494+0000 7f6ad2cf0640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0c9140 0x7f6acc0c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.494+0000 7f6ad24ef640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8004d40 0x7f6acc0ccc60 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800f7a0 tx=0x7f6ac800fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.494+0000 7f6ac2ffd640 1 -- 172.21.15.38:0/2668995606 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac80050d0 con 0x7f6ab8004d40 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.494+0000 7f6ac2ffd640 1 -- 172.21.15.38:0/2668995606 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac8018070 con 0x7f6ab8004d40 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.494+0000 7f6ad4753640 1 -- 172.21.15.38:0/2668995606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6acc0b4d80 con 0x7f6ab8004d40 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.494+0000 7f6ac2ffd640 1 -- 172.21.15.38:0/2668995606 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac800d730 con 0x7f6ab8004d40 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.494+0000 7f6ad4753640 1 -- 172.21.15.38:0/2668995606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc159270 con 0x7f6ab8004d40 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.495+0000 7f6ad4753640 1 -- 172.21.15.38:0/2668995606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc068e80 con 0x7f6ab8004d40 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.495+0000 7f6ac2ffd640 1 -- 172.21.15.38:0/2668995606 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ac800d8d0 con 0x7f6ab8004d40 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.496+0000 7f6ac2ffd640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ab006b500 0x7f6ab006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.496+0000 7f6ac2ffd640 1 -- 172.21.15.38:0/2668995606 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ac808a170 con 0x7f6ab8004d40 2024-04-25T08:38:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.496+0000 7f6ad1cee640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ab006b500 0x7f6ab006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.497+0000 7f6ad1cee640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ab006b500 0x7f6ab006d9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0b6490 tx=0x7f6abc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.500+0000 7f6ac2ffd640 1 -- 172.21.15.38:0/2668995606 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ac801a310 con 0x7f6ab8004d40 2024-04-25T08:38:27.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.718+0000 7f6ad4753640 1 -- 172.21.15.38:0/2668995606 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6acc0b3050 con 0x7f6ab8004d40 2024-04-25T08:38:27.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.747+0000 7f6ac2ffd640 1 -- 172.21.15.38:0/2668995606 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6ac8058ef0 con 0x7f6ab8004d40 2024-04-25T08:38:27.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.750+0000 7f6ad4753640 1 -- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ab006b500 msgr2=0x7f6ab006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.750+0000 7f6ad4753640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ab006b500 0x7f6ab006d9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6acc0b6490 tx=0x7f6abc009190 comp rx=0 tx=0).stop 2024-04-25T08:38:27.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.750+0000 7f6ad4753640 1 -- 172.21.15.38:0/2668995606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8004d40 msgr2=0x7f6acc0ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.750+0000 7f6ad4753640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8004d40 0x7f6acc0ccc60 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800f7a0 tx=0x7f6ac800fba0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.750+0000 7f6ad4753640 1 -- 172.21.15.38:0/2668995606 shutdown_connections 2024-04-25T08:38:27.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.750+0000 7f6ad4753640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6ab006b500 0x7f6ab006d9c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.750+0000 7f6ad4753640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6acc0c9140 0x7f6acc0c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.750+0000 7f6ad4753640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6acc0b42c0 0x7f6acc0b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.750+0000 7f6ad4753640 1 --2- 172.21.15.38:0/2668995606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6ab8004d40 0x7f6acc0ccc60 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.750+0000 7f6ad4753640 1 -- 172.21.15.38:0/2668995606 >> 172.21.15.38:0/2668995606 conn(0x7f6acc0668f0 msgr2=0x7f6acc0b8470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:27.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.750+0000 7f6ad4753640 1 -- 172.21.15.38:0/2668995606 shutdown_connections 2024-04-25T08:38:27.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.750+0000 7f6ad4753640 1 -- 172.21.15.38:0/2668995606 wait complete. 2024-04-25T08:38:27.755 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:38:27.755 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:38:27.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.865+0000 7f3a5b483640 1 Processor -- start 2024-04-25T08:38:27.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.865+0000 7f3a5b483640 1 -- start start 2024-04-25T08:38:27.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.866+0000 7f3a5b483640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3a54162110 0x7f3a5415e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.866+0000 7f3a5b483640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a5415e580 0x7f3a540cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.866+0000 7f3a5b483640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5415ec20 con 0x7f3a5406c700 2024-04-25T08:38:27.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.866+0000 7f3a5b483640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a54161c80 con 0x7f3a54162110 2024-04-25T08:38:27.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.866+0000 7f3a5b483640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a541625c0 con 0x7f3a5415e580 2024-04-25T08:38:27.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.866+0000 7f3a58a1e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3a54162110 0x7f3a5415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.866+0000 7f3a58a1e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3a54162110 0x7f3a5415e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51978/0 (socket says 172.21.15.38:51978) 2024-04-25T08:38:27.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.866+0000 7f3a58a1e640 1 -- 172.21.15.38:0/2987273000 learned_addr learned my addr 172.21.15.38:0/2987273000 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:27.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.867+0000 7f3a59a20640 1 --2- 172.21.15.38:0/2987273000 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a5415e580 0x7f3a540cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.867+0000 7f3a58a1e640 1 -- 172.21.15.38:0/2987273000 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a5415e580 msgr2=0x7f3a540cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:27.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.867+0000 7f3a58a1e640 1 --2- 172.21.15.38:0/2987273000 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a5415e580 0x7f3a540cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.867+0000 7f3a58a1e640 1 -- 172.21.15.38:0/2987273000 >> v1:172.21.15.107:6789/0 conn(0x7f3a5406c700 legacy=0x7f3a5406cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.867+0000 7f3a58a1e640 1 -- 172.21.15.38:0/2987273000 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5415ef00 con 0x7f3a54162110 2024-04-25T08:38:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.867+0000 7f3a58a1e640 1 --2- 172.21.15.38:0/2987273000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a54162110 0x7f3a5415e040 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009870 tx=0x7f3a4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ed166360ff726e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.868+0000 7f3a53fff640 1 -- 172.21.15.38:0/2987273000 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48037070 con 0x7f3a54162110 2024-04-25T08:38:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.868+0000 7f3a53fff640 1 -- 172.21.15.38:0/2987273000 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a48032440 con 0x7f3a54162110 2024-04-25T08:38:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.868+0000 7f3a53fff640 1 -- 172.21.15.38:0/2987273000 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48032810 con 0x7f3a54162110 2024-04-25T08:38:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.868+0000 7f3a5b483640 1 -- 172.21.15.38:0/2987273000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a54162110 msgr2=0x7f3a5415e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.868+0000 7f3a5b483640 1 --2- 172.21.15.38:0/2987273000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a54162110 0x7f3a5415e040 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3a48009870 tx=0x7f3a4802a270 comp rx=0 tx=0).stop 2024-04-25T08:38:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.868+0000 7f3a5b483640 1 -- 172.21.15.38:0/2987273000 shutdown_connections 2024-04-25T08:38:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.868+0000 7f3a5b483640 1 --2- 172.21.15.38:0/2987273000 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a5415e580 0x7f3a540cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.868+0000 7f3a5b483640 1 --2- 172.21.15.38:0/2987273000 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a54162110 0x7f3a5415e040 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.868+0000 7f3a5b483640 1 -- 172.21.15.38:0/2987273000 >> 172.21.15.38:0/2987273000 conn(0x7f3a5408b000 msgr2=0x7f3a5408b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.868+0000 7f3a5b483640 1 -- 172.21.15.38:0/2987273000 shutdown_connections 2024-04-25T08:38:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.868+0000 7f3a5b483640 1 -- 172.21.15.38:0/2987273000 wait complete. 2024-04-25T08:38:27.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.869+0000 7f3a5b483640 1 Processor -- start 2024-04-25T08:38:27.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.869+0000 7f3a5b483640 1 -- start start 2024-04-25T08:38:27.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a5b483640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a5406c700 0x7f3a5414fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a5b483640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a5415e580 0x7f3a540d1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a5b483640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a54162110 0x7f3a540d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a5b483640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5406c190 con 0x7f3a5415e580 2024-04-25T08:38:27.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a5b483640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a54161910 con 0x7f3a5406c700 2024-04-25T08:38:27.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a5b483640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5415b920 con 0x7f3a54162110 2024-04-25T08:38:27.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a58a1e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a5415e580 0x7f3a540d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a58a1e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a5415e580 0x7f3a540d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51982/0 (socket says 172.21.15.38:51982) 2024-04-25T08:38:27.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a58a1e640 1 -- 172.21.15.38:0/725559456 learned_addr learned my addr 172.21.15.38:0/725559456 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:27.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a59a20640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a54162110 0x7f3a540d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a5921f640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a5406c700 0x7f3a5414fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a58a1e640 1 -- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a54162110 msgr2=0x7f3a540d18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a58a1e640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a54162110 0x7f3a540d18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a58a1e640 1 -- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a5406c700 msgr2=0x7f3a5414fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a58a1e640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a5406c700 0x7f3a5414fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.870+0000 7f3a58a1e640 1 -- 172.21.15.38:0/725559456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a54157c70 con 0x7f3a5415e580 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.871+0000 7f3a5921f640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a5406c700 0x7f3a5414fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.871+0000 7f3a58a1e640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a5415e580 0x7f3a540d1380 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3a48000c00 tx=0x7f3a480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.871+0000 7f3a51ffb640 1 -- 172.21.15.38:0/725559456 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48037040 con 0x7f3a5415e580 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.871+0000 7f3a51ffb640 1 -- 172.21.15.38:0/725559456 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a48030440 con 0x7f3a5415e580 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.871+0000 7f3a5b483640 1 -- 172.21.15.38:0/725559456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a54157f50 con 0x7f3a5415e580 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.871+0000 7f3a51ffb640 1 -- 172.21.15.38:0/725559456 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a48030810 con 0x7f3a5415e580 2024-04-25T08:38:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.871+0000 7f3a5b483640 1 -- 172.21.15.38:0/725559456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a54156fd0 con 0x7f3a5415e580 2024-04-25T08:38:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.872+0000 7f3a5b483640 1 -- 172.21.15.38:0/725559456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a54068e80 con 0x7f3a5415e580 2024-04-25T08:38:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.873+0000 7f3a51ffb640 1 -- 172.21.15.38:0/725559456 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a48044030 con 0x7f3a5415e580 2024-04-25T08:38:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.873+0000 7f3a51ffb640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a4006b4d0 0x7f3a4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.873+0000 7f3a51ffb640 1 -- 172.21.15.38:0/725559456 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a480aa020 con 0x7f3a5415e580 2024-04-25T08:38:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.873+0000 7f3a5921f640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a4006b4d0 0x7f3a4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.874+0000 7f3a5921f640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a4006b4d0 0x7f3a4006d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0015e0 tx=0x7f3a4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:27.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:27.877+0000 7f3a51ffb640 1 -- 172.21.15.38:0/725559456 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a48078da0 con 0x7f3a5415e580 2024-04-25T08:38:28.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.096+0000 7f3a5b483640 1 -- 172.21.15.38:0/725559456 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3a540c4ed0 con 0x7f3a5415e580 2024-04-25T08:38:28.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.760+0000 7f3a51ffb640 1 -- 172.21.15.38:0/725559456 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3a480786a0 con 0x7f3a5415e580 2024-04-25T08:38:28.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.762+0000 7f3a5b483640 1 -- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a4006b4d0 msgr2=0x7f3a4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.762+0000 7f3a5b483640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a4006b4d0 0x7f3a4006d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0015e0 tx=0x7f3a4c009040 comp rx=0 tx=0).stop 2024-04-25T08:38:28.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.762+0000 7f3a5b483640 1 -- 172.21.15.38:0/725559456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a5415e580 msgr2=0x7f3a540d1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.762+0000 7f3a5b483640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a5415e580 0x7f3a540d1380 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f3a48000c00 tx=0x7f3a480046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.763+0000 7f3a5b483640 1 -- 172.21.15.38:0/725559456 shutdown_connections 2024-04-25T08:38:28.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.763+0000 7f3a5b483640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a4006b4d0 0x7f3a4006d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.763+0000 7f3a5b483640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a54162110 0x7f3a540d18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.763+0000 7f3a5b483640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a5415e580 0x7f3a540d1380 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.763+0000 7f3a5b483640 1 --2- 172.21.15.38:0/725559456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a5406c700 0x7f3a5414fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.763+0000 7f3a5b483640 1 -- 172.21.15.38:0/725559456 >> 172.21.15.38:0/725559456 conn(0x7f3a5408b000 msgr2=0x7f3a540cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:28.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.763+0000 7f3a5b483640 1 -- 172.21.15.38:0/725559456 shutdown_connections 2024-04-25T08:38:28.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.763+0000 7f3a5b483640 1 -- 172.21.15.38:0/725559456 wait complete. 2024-04-25T08:38:28.770 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:38:28.770 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:28.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.884+0000 7fc0bebe2640 1 Processor -- start 2024-04-25T08:38:28.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.884+0000 7fc0bebe2640 1 -- start start 2024-04-25T08:38:28.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.885+0000 7fc0bebe2640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc0b815dc10 0x7fc0b80ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.885+0000 7fc0bebe2640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc0b80d0f60 0x7fc0b80b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.885+0000 7fc0bebe2640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b80d3e70 con 0x7fc0b80ced60 2024-04-25T08:38:28.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.885+0000 7fc0bebe2640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b815dff0 con 0x7fc0b815dc10 2024-04-25T08:38:28.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.885+0000 7fc0bebe2640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b80b6990 con 0x7fc0b80d0f60 2024-04-25T08:38:28.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.885+0000 7fc0bc97e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc0b815dc10 0x7fc0b80ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.885+0000 7fc0bc97e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc0b815dc10 0x7fc0b80ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54058/0 (socket says 172.21.15.38:54058) 2024-04-25T08:38:28.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.885+0000 7fc0bc97e640 1 -- 172.21.15.38:0/3336297066 learned_addr learned my addr 172.21.15.38:0/3336297066 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:28.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.886+0000 7fc0bd17f640 1 --2- 172.21.15.38:0/3336297066 >> v2:172.21.15.107:3300/0 conn(0x7fc0b80d0f60 0x7fc0b80b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.886+0000 7fc0bc97e640 1 -- 172.21.15.38:0/3336297066 >> v2:172.21.15.107:3300/0 conn(0x7fc0b80d0f60 msgr2=0x7fc0b80b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:28.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.886+0000 7fc0bc97e640 1 --2- 172.21.15.38:0/3336297066 >> v2:172.21.15.107:3300/0 conn(0x7fc0b80d0f60 0x7fc0b80b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.886+0000 7fc0bc97e640 1 -- 172.21.15.38:0/3336297066 >> v1:172.21.15.107:6789/0 conn(0x7fc0b80ced60 legacy=0x7fc0b80d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:28.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.886+0000 7fc0bc97e640 1 -- 172.21.15.38:0/3336297066 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b80d4150 con 0x7fc0b815dc10 2024-04-25T08:38:28.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.886+0000 7fc0bc97e640 1 --2- 172.21.15.38:0/3336297066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b815dc10 0x7fc0b80ce820 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80cdee0 tx=0x7fc0b002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae697482611dfa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:28.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.887+0000 7fc0b77fe640 1 -- 172.21.15.38:0/3336297066 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0036070 con 0x7fc0b815dc10 2024-04-25T08:38:28.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.887+0000 7fc0b77fe640 1 -- 172.21.15.38:0/3336297066 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b0031600 con 0x7fc0b815dc10 2024-04-25T08:38:28.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.887+0000 7fc0b77fe640 1 -- 172.21.15.38:0/3336297066 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b00319d0 con 0x7fc0b815dc10 2024-04-25T08:38:28.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.887+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3336297066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b815dc10 msgr2=0x7fc0b80ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.887+0000 7fc0bebe2640 1 --2- 172.21.15.38:0/3336297066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b815dc10 0x7fc0b80ce820 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc0b80cdee0 tx=0x7fc0b002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.887+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3336297066 shutdown_connections 2024-04-25T08:38:28.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.887+0000 7fc0bebe2640 1 --2- 172.21.15.38:0/3336297066 >> v2:172.21.15.107:3300/0 conn(0x7fc0b80d0f60 0x7fc0b80b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.887+0000 7fc0bebe2640 1 --2- 172.21.15.38:0/3336297066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b815dc10 0x7fc0b80ce820 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.887+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3336297066 >> 172.21.15.38:0/3336297066 conn(0x7fc0b8066d10 msgr2=0x7fc0b8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:28.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.887+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3336297066 shutdown_connections 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.887+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3336297066 wait complete. 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.888+0000 7fc0bebe2640 1 Processor -- start 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.888+0000 7fc0bebe2640 1 -- start start 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.889+0000 7fc0bebe2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0b80ced60 0x7fc0b81f86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.889+0000 7fc0bebe2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b80d0f60 0x7fc0b815a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.889+0000 7fc0bebe2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0b815dc10 0x7fc0b80d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.889+0000 7fc0bebe2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80d3900 con 0x7fc0b80d0f60 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.889+0000 7fc0bebe2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80b6bc0 con 0x7fc0b815dc10 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.889+0000 7fc0bebe2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b80d3a80 con 0x7fc0b80ced60 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.889+0000 7fc0b7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b80d0f60 0x7fc0b815a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.889+0000 7fc0b7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b80d0f60 0x7fc0b815a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54070/0 (socket says 172.21.15.38:54070) 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.889+0000 7fc0b7fff640 1 -- 172.21.15.38:0/3110482886 learned_addr learned my addr 172.21.15.38:0/3110482886 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0b7fff640 1 -- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0b80ced60 msgr2=0x7fc0b81f86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0b7fff640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0b80ced60 0x7fc0b81f86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0bd17f640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0b815dc10 0x7fc0b80d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0b7fff640 1 -- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0b815dc10 msgr2=0x7fc0b80d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:28.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0b7fff640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0b815dc10 0x7fc0b80d1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:28.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0b7fff640 1 -- 172.21.15.38:0/3110482886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b81f8be0 con 0x7fc0b80d0f60 2024-04-25T08:38:28.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0bd17f640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0b815dc10 0x7fc0b80d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:28.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0b7fff640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b80d0f60 0x7fc0b815a240 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800c3c0 tx=0x7fc0a800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:28.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0b57fa640 1 -- 172.21.15.38:0/3110482886 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a80052d0 con 0x7fc0b80d0f60 2024-04-25T08:38:28.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0b57fa640 1 -- 172.21.15.38:0/3110482886 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a8005490 con 0x7fc0b80d0f60 2024-04-25T08:38:28.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3110482886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b80b3070 con 0x7fc0b80d0f60 2024-04-25T08:38:28.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0b57fa640 1 -- 172.21.15.38:0/3110482886 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8007260 con 0x7fc0b80d0f60 2024-04-25T08:38:28.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.890+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3110482886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b80b3630 con 0x7fc0b80d0f60 2024-04-25T08:38:28.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.891+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3110482886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b8068e80 con 0x7fc0b80d0f60 2024-04-25T08:38:28.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.892+0000 7fc0b57fa640 1 -- 172.21.15.38:0/3110482886 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0a8007400 con 0x7fc0b80d0f60 2024-04-25T08:38:28.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.892+0000 7fc0b57fa640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc09806b4d0 0x7fc09806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:28.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.892+0000 7fc0b57fa640 1 -- 172.21.15.38:0/3110482886 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc0a8089830 con 0x7fc0b80d0f60 2024-04-25T08:38:28.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.892+0000 7fc0bc97e640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc09806b4d0 0x7fc09806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:28.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.893+0000 7fc0bc97e640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc09806b4d0 0x7fc09806d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0033000 tx=0x7fc0b002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:28.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:28.896+0000 7fc0b57fa640 1 -- 172.21.15.38:0/3110482886 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0b81f8be0 con 0x7fc0b80d0f60 2024-04-25T08:38:29.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.117+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3110482886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0b8067ed0 con 0x7fc0b80d0f60 2024-04-25T08:38:29.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.117+0000 7fc0b57fa640 1 -- 172.21.15.38:0/3110482886 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc0a8058520 con 0x7fc0b80d0f60 2024-04-25T08:38:29.111 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:29.111 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3680979257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3680979257},{"type":"v1","addr":"172.21.15.107:6836","nonce":3680979257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/697909489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":697909489},{"type":"v1","addr":"172.21.15.107:6837","nonce":697909489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3083871141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3083871141},{"type":"v1","addr":"172.21.15.38:6834","nonce":3083871141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:26.095391+0000","modified":"2024-04-25T08:38:28.097807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/2922734840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":2922734840},{"type":"v1","addr":"172.21.15.38:6835","nonce":2922734840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:38:29.111 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 57 2024-04-25T08:38:29.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.119+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc09806b4d0 msgr2=0x7fc09806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.119+0000 7fc0bebe2640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc09806b4d0 0x7fc09806d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0033000 tx=0x7fc0b002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:29.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.119+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3110482886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b80d0f60 msgr2=0x7fc0b815a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.119+0000 7fc0bebe2640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b80d0f60 0x7fc0b815a240 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fc0a800c3c0 tx=0x7fc0a800c880 comp rx=0 tx=0).stop 2024-04-25T08:38:29.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.119+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3110482886 shutdown_connections 2024-04-25T08:38:29.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.119+0000 7fc0bebe2640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0b815dc10 0x7fc0b80d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.119+0000 7fc0bebe2640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0b80d0f60 0x7fc0b815a240 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.119+0000 7fc0bebe2640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc09806b4d0 0x7fc09806d990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.119+0000 7fc0bebe2640 1 --2- 172.21.15.38:0/3110482886 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0b80ced60 0x7fc0b81f86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.119+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3110482886 >> 172.21.15.38:0/3110482886 conn(0x7fc0b8066d10 msgr2=0x7fc0b80ce160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:29.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.119+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3110482886 shutdown_connections 2024-04-25T08:38:29.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.120+0000 7fc0bebe2640 1 -- 172.21.15.38:0/3110482886 wait complete. 2024-04-25T08:38:29.124 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:29.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.233+0000 7fec75091640 1 Processor -- start 2024-04-25T08:38:29.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.233+0000 7fec75091640 1 -- start start 2024-04-25T08:38:29.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.234+0000 7fec75091640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fec7015bb10 0x7fec7009a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.234+0000 7fec75091640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fec7015fbf0 0x7fec7009a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.234+0000 7fec75091640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec70160340 con 0x7fec700d0460 2024-04-25T08:38:29.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.234+0000 7fec75091640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec700cffd0 con 0x7fec7015fbf0 2024-04-25T08:38:29.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.234+0000 7fec75091640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec700cf400 con 0x7fec7015bb10 2024-04-25T08:38:29.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.234+0000 7fec6f59e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fec7015fbf0 0x7fec7009a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.234+0000 7fec6f59e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fec7015fbf0 0x7fec7009a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54074/0 (socket says 172.21.15.38:54074) 2024-04-25T08:38:29.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.234+0000 7fec6f59e640 1 -- 172.21.15.38:0/16870973 learned_addr learned my addr 172.21.15.38:0/16870973 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:29.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.235+0000 7fec6ed9d640 1 --2- 172.21.15.38:0/16870973 >> v2:172.21.15.107:3300/0 conn(0x7fec7015bb10 0x7fec7009a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.235+0000 7fec6f59e640 1 -- 172.21.15.38:0/16870973 >> v2:172.21.15.107:3300/0 conn(0x7fec7015bb10 msgr2=0x7fec7009a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:29.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.235+0000 7fec6f59e640 1 --2- 172.21.15.38:0/16870973 >> v2:172.21.15.107:3300/0 conn(0x7fec7015bb10 0x7fec7009a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.235+0000 7fec6f59e640 1 -- 172.21.15.38:0/16870973 >> v1:172.21.15.107:6789/0 conn(0x7fec700d0460 legacy=0x7fec7015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.235+0000 7fec6f59e640 1 -- 172.21.15.38:0/16870973 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec700cf6a0 con 0x7fec7015fbf0 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.235+0000 7fec6ed9d640 1 --2- 172.21.15.38:0/16870973 >> v2:172.21.15.107:3300/0 conn(0x7fec7015bb10 0x7fec7009a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.235+0000 7fec6f59e640 1 --2- 172.21.15.38:0/16870973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7015fbf0 0x7fec7009a9c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fec700cf280 tx=0x7fec6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5029d39c93625e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.236+0000 7fec6dd9b640 1 -- 172.21.15.38:0/16870973 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68037070 con 0x7fec7015fbf0 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.236+0000 7fec6dd9b640 1 -- 172.21.15.38:0/16870973 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec68032440 con 0x7fec7015fbf0 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.236+0000 7fec6dd9b640 1 -- 172.21.15.38:0/16870973 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec68032810 con 0x7fec7015fbf0 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.236+0000 7fec75091640 1 -- 172.21.15.38:0/16870973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7015fbf0 msgr2=0x7fec7009a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.236+0000 7fec75091640 1 --2- 172.21.15.38:0/16870973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7015fbf0 0x7fec7009a9c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fec700cf280 tx=0x7fec6802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.236+0000 7fec75091640 1 -- 172.21.15.38:0/16870973 shutdown_connections 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.236+0000 7fec75091640 1 --2- 172.21.15.38:0/16870973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7015fbf0 0x7fec7009a9c0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.236+0000 7fec75091640 1 --2- 172.21.15.38:0/16870973 >> v2:172.21.15.107:3300/0 conn(0x7fec7015bb10 0x7fec7009a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.236+0000 7fec75091640 1 -- 172.21.15.38:0/16870973 >> 172.21.15.38:0/16870973 conn(0x7fec700b31f0 msgr2=0x7fec700b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:29.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.236+0000 7fec75091640 1 -- 172.21.15.38:0/16870973 shutdown_connections 2024-04-25T08:38:29.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.236+0000 7fec75091640 1 -- 172.21.15.38:0/16870973 wait complete. 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.238+0000 7fec75091640 1 Processor -- start 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.238+0000 7fec75091640 1 -- start start 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.238+0000 7fec75091640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec700d0460 0x7fec70149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.238+0000 7fec75091640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7015bb10 0x7fec70070440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.238+0000 7fec75091640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec7015fbf0 0x7fec70070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.238+0000 7fec75091640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec700c5250 con 0x7fec7015bb10 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.238+0000 7fec75091640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec700cfc60 con 0x7fec7015fbf0 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.238+0000 7fec75091640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec700c53d0 con 0x7fec700d0460 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.238+0000 7fec6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7015bb10 0x7fec70070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.238+0000 7fec6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7015bb10 0x7fec70070440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54076/0 (socket says 172.21.15.38:54076) 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.238+0000 7fec6e59c640 1 -- 172.21.15.38:0/882474475 learned_addr learned my addr 172.21.15.38:0/882474475 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec6f59e640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec7015fbf0 0x7fec70070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec6ed9d640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec700d0460 0x7fec70149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec6e59c640 1 -- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec700d0460 msgr2=0x7fec70149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec6e59c640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec700d0460 0x7fec70149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec6e59c640 1 -- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec7015fbf0 msgr2=0x7fec70070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec6e59c640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec7015fbf0 0x7fec70070980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec6e59c640 1 -- 172.21.15.38:0/882474475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec700c9770 con 0x7fec7015bb10 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec6ed9d640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec700d0460 0x7fec70149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec6e59c640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7015bb10 0x7fec70070440 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c430 tx=0x7fec5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec677fe640 1 -- 172.21.15.38:0/882474475 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58011790 con 0x7fec7015bb10 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec677fe640 1 -- 172.21.15.38:0/882474475 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec58017070 con 0x7fec7015bb10 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec75091640 1 -- 172.21.15.38:0/882474475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec700c9a50 con 0x7fec7015bb10 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec677fe640 1 -- 172.21.15.38:0/882474475 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec58011b70 con 0x7fec7015bb10 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.239+0000 7fec75091640 1 -- 172.21.15.38:0/882474475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec700ca010 con 0x7fec7015bb10 2024-04-25T08:38:29.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.240+0000 7fec75091640 1 -- 172.21.15.38:0/882474475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec70068e80 con 0x7fec7015bb10 2024-04-25T08:38:29.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.241+0000 7fec677fe640 1 -- 172.21.15.38:0/882474475 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec58011d10 con 0x7fec7015bb10 2024-04-25T08:38:29.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.241+0000 7fec677fe640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec5006b4d0 0x7fec5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.241+0000 7fec677fe640 1 -- 172.21.15.38:0/882474475 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fec5808a250 con 0x7fec7015bb10 2024-04-25T08:38:29.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.241+0000 7fec6ed9d640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec5006b4d0 0x7fec5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.242+0000 7fec6ed9d640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec5006b4d0 0x7fec5006d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fec701f4fa0 tx=0x7fec60007550 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:29.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.245+0000 7fec677fe640 1 -- 172.21.15.38:0/882474475 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec58058f40 con 0x7fec7015bb10 2024-04-25T08:38:29.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.428+0000 7fec75091640 1 -- 172.21.15.38:0/882474475 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec700cdbf0 con 0x7fec7015bb10 2024-04-25T08:38:29.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.429+0000 7fec677fe640 1 -- 172.21.15.38:0/882474475 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16873 (secure 0 0 0) 0x7fec58058840 con 0x7fec7015bb10 2024-04-25T08:38:29.426 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:29.427 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":96,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:38:27.725413+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:24.377232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:25.069513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:29.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.431+0000 7fec75091640 1 -- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec5006b4d0 msgr2=0x7fec5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.431+0000 7fec75091640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec5006b4d0 0x7fec5006d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fec701f4fa0 tx=0x7fec60007550 comp rx=0 tx=0).stop 2024-04-25T08:38:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.431+0000 7fec75091640 1 -- 172.21.15.38:0/882474475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7015bb10 msgr2=0x7fec70070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.431+0000 7fec75091640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7015bb10 0x7fec70070440 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c430 tx=0x7fec5800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.431+0000 7fec75091640 1 -- 172.21.15.38:0/882474475 shutdown_connections 2024-04-25T08:38:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.431+0000 7fec75091640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec7015fbf0 0x7fec70070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.431+0000 7fec75091640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec7015bb10 0x7fec70070440 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.431+0000 7fec75091640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec5006b4d0 0x7fec5006d990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.431+0000 7fec75091640 1 --2- 172.21.15.38:0/882474475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec700d0460 0x7fec70149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.431+0000 7fec75091640 1 -- 172.21.15.38:0/882474475 >> 172.21.15.38:0/882474475 conn(0x7fec700b31f0 msgr2=0x7fec700c7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.431+0000 7fec75091640 1 -- 172.21.15.38:0/882474475 shutdown_connections 2024-04-25T08:38:29.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.431+0000 7fec75091640 1 -- 172.21.15.38:0/882474475 wait complete. 2024-04-25T08:38:29.436 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:38:29.436 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:38:29.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.544+0000 7f83825d5640 1 Processor -- start 2024-04-25T08:38:29.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.544+0000 7f83825d5640 1 -- start start 2024-04-25T08:38:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f83825d5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f837c0d0fe0 0x7f837c0b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f83825d5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f837c0d2e50 0x7f837c0b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f83825d5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f837c0b8400 con 0x7f837c15dc40 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f83825d5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f837c0d19b0 con 0x7f837c0d0fe0 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f83825d5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f837c0b86e0 con 0x7f837c0d2e50 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f837b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f837c0d0fe0 0x7f837c0b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f837bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f837c15dc40 0x7f837c0d08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:48310/0 (socket says 172.21.15.38:48310) 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f837bfff640 1 -- 172.21.15.38:0/2283855043 learned_addr learned my addr 172.21.15.38:0/2283855043 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f8380b72640 1 --2- 172.21.15.38:0/2283855043 >> v2:172.21.15.107:3300/0 conn(0x7f837c0d2e50 0x7f837c0b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f837b7fe640 1 -- 172.21.15.38:0/2283855043 >> v2:172.21.15.107:3300/0 conn(0x7f837c0d2e50 msgr2=0x7f837c0b6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f837b7fe640 1 --2- 172.21.15.38:0/2283855043 >> v2:172.21.15.107:3300/0 conn(0x7f837c0d2e50 0x7f837c0b6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f837b7fe640 1 -- 172.21.15.38:0/2283855043 >> v1:172.21.15.38:6789/0 conn(0x7f837c15dc40 legacy=0x7f837c0d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f837b7fe640 1 -- 172.21.15.38:0/2283855043 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c0b89c0 con 0x7f837c0d0fe0 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f837affd640 1 -- 172.21.15.38:0/2283855043 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212217887 0 0) 0x7f837c0b8400 con 0x7f837c15dc40 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.545+0000 7f8380b72640 1 --2- 172.21.15.38:0/2283855043 >> v2:172.21.15.107:3300/0 conn(0x7f837c0d2e50 0x7f837c0b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.546+0000 7f837b7fe640 1 --2- 172.21.15.38:0/2283855043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f837c0d0fe0 0x7f837c0b6a60 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f837c0c7250 tx=0x7f837002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1ae60e03836e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.546+0000 7f837affd640 1 -- 172.21.15.38:0/2283855043 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370037070 con 0x7f837c0d0fe0 2024-04-25T08:38:29.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.546+0000 7f837affd640 1 -- 172.21.15.38:0/2283855043 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8370032440 con 0x7f837c0d0fe0 2024-04-25T08:38:29.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.546+0000 7f837affd640 1 -- 172.21.15.38:0/2283855043 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370032810 con 0x7f837c0d0fe0 2024-04-25T08:38:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.547+0000 7f83825d5640 1 -- 172.21.15.38:0/2283855043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f837c0d0fe0 msgr2=0x7f837c0b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.547+0000 7f83825d5640 1 --2- 172.21.15.38:0/2283855043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f837c0d0fe0 0x7f837c0b6a60 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f837c0c7250 tx=0x7f837002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.547+0000 7f83825d5640 1 -- 172.21.15.38:0/2283855043 shutdown_connections 2024-04-25T08:38:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.547+0000 7f83825d5640 1 --2- 172.21.15.38:0/2283855043 >> v2:172.21.15.107:3300/0 conn(0x7f837c0d2e50 0x7f837c0b6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.547+0000 7f83825d5640 1 --2- 172.21.15.38:0/2283855043 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f837c0d0fe0 0x7f837c0b6a60 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.547+0000 7f83825d5640 1 -- 172.21.15.38:0/2283855043 >> 172.21.15.38:0/2283855043 conn(0x7f837c066ce0 msgr2=0x7f837c0670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.547+0000 7f83825d5640 1 -- 172.21.15.38:0/2283855043 shutdown_connections 2024-04-25T08:38:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.547+0000 7f83825d5640 1 -- 172.21.15.38:0/2283855043 wait complete. 2024-04-25T08:38:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.548+0000 7f83825d5640 1 Processor -- start 2024-04-25T08:38:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.548+0000 7f83825d5640 1 -- start start 2024-04-25T08:38:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f83825d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f837c0d0fe0 0x7f837c159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f83825d5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f837c0d2e50 0x7f837c159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f83825d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f837c15dc40 0x7f837c0ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f83825d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0b9700 con 0x7f837c15dc40 2024-04-25T08:38:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f83825d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0b9580 con 0x7f837c0d0fe0 2024-04-25T08:38:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f83825d5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837c0b9880 con 0x7f837c0d2e50 2024-04-25T08:38:29.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f8380b72640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f837c15dc40 0x7f837c0ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f8380b72640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f837c15dc40 0x7f837c0ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54092/0 (socket says 172.21.15.38:54092) 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f8380b72640 1 -- 172.21.15.38:0/121790772 learned_addr learned my addr 172.21.15.38:0/121790772 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f837bfff640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f837c0d0fe0 0x7f837c159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f837b7fe640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f837c0d2e50 0x7f837c159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f8380b72640 1 -- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f837c0d2e50 msgr2=0x7f837c159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f8380b72640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f837c0d2e50 0x7f837c159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f8380b72640 1 -- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f837c0d0fe0 msgr2=0x7f837c159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f8380b72640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f837c0d0fe0 0x7f837c159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f8380b72640 1 -- 172.21.15.38:0/121790772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c161010 con 0x7f837c15dc40 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f837bfff640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f837c0d0fe0 0x7f837c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f837b7fe640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f837c0d2e50 0x7f837c159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.549+0000 7f8380b72640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f837c15dc40 0x7f837c0ac2d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f837400cef0 tx=0x7f837400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:29.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.550+0000 7f8378ff9640 1 -- 172.21.15.38:0/121790772 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837401a070 con 0x7f837c15dc40 2024-04-25T08:38:29.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.550+0000 7f8378ff9640 1 -- 172.21.15.38:0/121790772 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837400d860 con 0x7f837c15dc40 2024-04-25T08:38:29.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.550+0000 7f83825d5640 1 -- 172.21.15.38:0/121790772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f837c15a1d0 con 0x7f837c15dc40 2024-04-25T08:38:29.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.550+0000 7f8378ff9640 1 -- 172.21.15.38:0/121790772 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837400dc50 con 0x7f837c15dc40 2024-04-25T08:38:29.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.550+0000 7f83825d5640 1 -- 172.21.15.38:0/121790772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f837c0ad120 con 0x7f837c15dc40 2024-04-25T08:38:29.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.551+0000 7f83825d5640 1 -- 172.21.15.38:0/121790772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837c068e80 con 0x7f837c15dc40 2024-04-25T08:38:29.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.551+0000 7f8378ff9640 1 -- 172.21.15.38:0/121790772 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f837401d070 con 0x7f837c15dc40 2024-04-25T08:38:29.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.551+0000 7f8378ff9640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f836806b4d0 0x7f836806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.552+0000 7f8378ff9640 1 -- 172.21.15.38:0/121790772 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f837408ac70 con 0x7f837c15dc40 2024-04-25T08:38:29.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.552+0000 7f837bfff640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f836806b4d0 0x7f836806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.552+0000 7f837bfff640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f836806b4d0 0x7f836806d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f836c000f60 tx=0x7f836c007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:29.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.556+0000 7f8378ff9640 1 -- 172.21.15.38:0/121790772 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8374059a10 con 0x7f837c15dc40 2024-04-25T08:38:29.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.732+0000 7f83825d5640 1 -- 172.21.15.38:0/121790772 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f837c15c3e0 con 0x7f836806b4d0 2024-04-25T08:38:29.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.739+0000 7f8378ff9640 1 -- 172.21.15.38:0/121790772 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251992 (secure 0 0 0) 0x7f837c15c3e0 con 0x7f836806b4d0 2024-04-25T08:38:29.736 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:29.737 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:38:29.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.742+0000 7f83825d5640 1 -- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f836806b4d0 msgr2=0x7f836806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.742+0000 7f83825d5640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f836806b4d0 0x7f836806d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f836c000f60 tx=0x7f836c007ca0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.742+0000 7f83825d5640 1 -- 172.21.15.38:0/121790772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f837c15dc40 msgr2=0x7f837c0ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.742+0000 7f83825d5640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f837c15dc40 0x7f837c0ac2d0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f837400cef0 tx=0x7f837400a710 comp rx=0 tx=0).stop 2024-04-25T08:38:29.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.742+0000 7f83825d5640 1 -- 172.21.15.38:0/121790772 shutdown_connections 2024-04-25T08:38:29.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.742+0000 7f83825d5640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f837c15dc40 0x7f837c0ac2d0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.742+0000 7f83825d5640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f836806b4d0 0x7f836806d990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.742+0000 7f83825d5640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f837c0d2e50 0x7f837c159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.742+0000 7f83825d5640 1 --2- 172.21.15.38:0/121790772 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f837c0d0fe0 0x7f837c159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.742+0000 7f83825d5640 1 -- 172.21.15.38:0/121790772 >> 172.21.15.38:0/121790772 conn(0x7f837c066ce0 msgr2=0x7f837c0d40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:29.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.742+0000 7f83825d5640 1 -- 172.21.15.38:0/121790772 shutdown_connections 2024-04-25T08:38:29.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.742+0000 7f83825d5640 1 -- 172.21.15.38:0/121790772 wait complete. 2024-04-25T08:38:29.747 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-04-25T08:38:29.301993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":456888,"kb_used_data":10312,"kb_used_omap":12,"kb_used_meta":446451,"kb_avail":754517832,"statfs":{"total":773094113280,"available":772626259968,"internally_reserved":0,"allocated":10559488,"data_stored":5509179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":457166384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.579221"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.752588+0000","last_change":"2024-04-25T08:38:26.746055+0000","last_active":"2024-04-25T08:38:26.752588+0000","last_peered":"2024-04-25T08:38:26.752588+0000","last_clean":"2024-04-25T08:38:26.752588+0000","last_became_active":"2024-04-25T08:38:26.745737+0000","last_became_peered":"2024-04-25T08:38:26.745737+0000","last_unstale":"2024-04-25T08:38:26.752588+0000","last_undegraded":"2024-04-25T08:38:26.752588+0000","last_fullsized":"2024-04-25T08:38:26.752588+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:31.720896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.747207+0000","last_change":"2024-04-25T08:38:26.745987+0000","last_active":"2024-04-25T08:38:26.747207+0000","last_peered":"2024-04-25T08:38:26.747207+0000","last_clean":"2024-04-25T08:38:26.747207+0000","last_became_active":"2024-04-25T08:38:26.745685+0000","last_became_peered":"2024-04-25T08:38:26.745685+0000","last_unstale":"2024-04-25T08:38:26.747207+0000","last_undegraded":"2024-04-25T08:38:26.747207+0000","last_fullsized":"2024-04-25T08:38:26.747207+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:10.111406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.752412+0000","last_change":"2024-04-25T08:38:26.747498+0000","last_active":"2024-04-25T08:38:26.752412+0000","last_peered":"2024-04-25T08:38:26.752412+0000","last_clean":"2024-04-25T08:38:26.752412+0000","last_became_active":"2024-04-25T08:38:26.747216+0000","last_became_peered":"2024-04-25T08:38:26.747216+0000","last_unstale":"2024-04-25T08:38:26.752412+0000","last_undegraded":"2024-04-25T08:38:26.752412+0000","last_fullsized":"2024-04-25T08:38:26.752412+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:30.874392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759707+0000","last_change":"2024-04-25T08:38:26.753460+0000","last_active":"2024-04-25T08:38:26.759707+0000","last_peered":"2024-04-25T08:38:26.759707+0000","last_clean":"2024-04-25T08:38:26.759707+0000","last_became_active":"2024-04-25T08:38:26.753198+0000","last_became_peered":"2024-04-25T08:38:26.753198+0000","last_unstale":"2024-04-25T08:38:26.759707+0000","last_undegraded":"2024-04-25T08:38:26.759707+0000","last_fullsized":"2024-04-25T08:38:26.759707+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:50.567885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.733588+0000","last_change":"2024-04-25T08:38:26.732586+0000","last_active":"2024-04-25T08:38:26.733588+0000","last_peered":"2024-04-25T08:38:26.733588+0000","last_clean":"2024-04-25T08:38:26.733588+0000","last_became_active":"2024-04-25T08:38:26.732256+0000","last_became_peered":"2024-04-25T08:38:26.732256+0000","last_unstale":"2024-04-25T08:38:26.733588+0000","last_undegraded":"2024-04-25T08:38:26.733588+0000","last_fullsized":"2024-04-25T08:38:26.733588+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:47.208427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.764134+0000","last_change":"2024-04-25T08:38:26.763440+0000","last_active":"2024-04-25T08:38:26.764134+0000","last_peered":"2024-04-25T08:38:26.764134+0000","last_clean":"2024-04-25T08:38:26.764134+0000","last_became_active":"2024-04-25T08:38:26.763148+0000","last_became_peered":"2024-04-25T08:38:26.763148+0000","last_unstale":"2024-04-25T08:38:26.764134+0000","last_undegraded":"2024-04-25T08:38:26.764134+0000","last_fullsized":"2024-04-25T08:38:26.764134+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:31.732590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.741593+0000","last_change":"2024-04-25T08:38:26.740732+0000","last_active":"2024-04-25T08:38:26.741593+0000","last_peered":"2024-04-25T08:38:26.741593+0000","last_clean":"2024-04-25T08:38:26.741593+0000","last_became_active":"2024-04-25T08:38:26.740460+0000","last_became_peered":"2024-04-25T08:38:26.740460+0000","last_unstale":"2024-04-25T08:38:26.741593+0000","last_undegraded":"2024-04-25T08:38:26.741593+0000","last_fullsized":"2024-04-25T08:38:26.741593+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:41.567708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.762505+0000","last_change":"2024-04-25T08:38:26.752809+0000","last_active":"2024-04-25T08:38:26.762505+0000","last_peered":"2024-04-25T08:38:26.762505+0000","last_clean":"2024-04-25T08:38:26.762505+0000","last_became_active":"2024-04-25T08:38:26.744924+0000","last_became_peered":"2024-04-25T08:38:26.744924+0000","last_unstale":"2024-04-25T08:38:26.762505+0000","last_undegraded":"2024-04-25T08:38:26.762505+0000","last_fullsized":"2024-04-25T08:38:26.762505+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:02.220030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.746960+0000","last_change":"2024-04-25T08:38:26.743745+0000","last_active":"2024-04-25T08:38:26.746960+0000","last_peered":"2024-04-25T08:38:26.746960+0000","last_clean":"2024-04-25T08:38:26.746960+0000","last_became_active":"2024-04-25T08:38:26.743392+0000","last_became_peered":"2024-04-25T08:38:26.743392+0000","last_unstale":"2024-04-25T08:38:26.746960+0000","last_undegraded":"2024-04-25T08:38:26.746960+0000","last_fullsized":"2024-04-25T08:38:26.746960+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:58.463803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.762541+0000","last_change":"2024-04-25T08:38:26.761760+0000","last_active":"2024-04-25T08:38:26.762541+0000","last_peered":"2024-04-25T08:38:26.762541+0000","last_clean":"2024-04-25T08:38:26.762541+0000","last_became_active":"2024-04-25T08:38:26.761461+0000","last_became_peered":"2024-04-25T08:38:26.761461+0000","last_unstale":"2024-04-25T08:38:26.762541+0000","last_undegraded":"2024-04-25T08:38:26.762541+0000","last_fullsized":"2024-04-25T08:38:26.762541+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:22.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.755860+0000","last_change":"2024-04-25T08:38:26.754942+0000","last_active":"2024-04-25T08:38:26.755860+0000","last_peered":"2024-04-25T08:38:26.755860+0000","last_clean":"2024-04-25T08:38:26.755860+0000","last_became_active":"2024-04-25T08:38:26.754585+0000","last_became_peered":"2024-04-25T08:38:26.754585+0000","last_unstale":"2024-04-25T08:38:26.755860+0000","last_undegraded":"2024-04-25T08:38:26.755860+0000","last_fullsized":"2024-04-25T08:38:26.755860+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:27.775288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.745527+0000","last_change":"2024-04-25T08:38:26.743668+0000","last_active":"2024-04-25T08:38:26.745527+0000","last_peered":"2024-04-25T08:38:26.745527+0000","last_clean":"2024-04-25T08:38:26.745527+0000","last_became_active":"2024-04-25T08:38:26.743405+0000","last_became_peered":"2024-04-25T08:38:26.743405+0000","last_unstale":"2024-04-25T08:38:26.745527+0000","last_undegraded":"2024-04-25T08:38:26.745527+0000","last_fullsized":"2024-04-25T08:38:26.745527+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:21.456081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759808+0000","last_change":"2024-04-25T08:38:26.744820+0000","last_active":"2024-04-25T08:38:26.759808+0000","last_peered":"2024-04-25T08:38:26.759808+0000","last_clean":"2024-04-25T08:38:26.759808+0000","last_became_active":"2024-04-25T08:38:26.743666+0000","last_became_peered":"2024-04-25T08:38:26.743666+0000","last_unstale":"2024-04-25T08:38:26.759808+0000","last_undegraded":"2024-04-25T08:38:26.759808+0000","last_fullsized":"2024-04-25T08:38:26.759808+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:51.015551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.763134+0000","last_change":"2024-04-25T08:38:26.762672+0000","last_active":"2024-04-25T08:38:26.763134+0000","last_peered":"2024-04-25T08:38:26.763134+0000","last_clean":"2024-04-25T08:38:26.763134+0000","last_became_active":"2024-04-25T08:38:26.762385+0000","last_became_peered":"2024-04-25T08:38:26.762385+0000","last_unstale":"2024-04-25T08:38:26.763134+0000","last_undegraded":"2024-04-25T08:38:26.763134+0000","last_fullsized":"2024-04-25T08:38:26.763134+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:40.858231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.765383+0000","last_change":"2024-04-25T08:38:26.760115+0000","last_active":"2024-04-25T08:38:26.765383+0000","last_peered":"2024-04-25T08:38:26.765383+0000","last_clean":"2024-04-25T08:38:26.765383+0000","last_became_active":"2024-04-25T08:38:26.759806+0000","last_became_peered":"2024-04-25T08:38:26.759806+0000","last_unstale":"2024-04-25T08:38:26.765383+0000","last_undegraded":"2024-04-25T08:38:26.765383+0000","last_fullsized":"2024-04-25T08:38:26.765383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:37.915525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759191+0000","last_change":"2024-04-25T08:38:26.756715+0000","last_active":"2024-04-25T08:38:26.759191+0000","last_peered":"2024-04-25T08:38:26.759191+0000","last_clean":"2024-04-25T08:38:26.759191+0000","last_became_active":"2024-04-25T08:38:26.756254+0000","last_became_peered":"2024-04-25T08:38:26.756254+0000","last_unstale":"2024-04-25T08:38:26.759191+0000","last_undegraded":"2024-04-25T08:38:26.759191+0000","last_fullsized":"2024-04-25T08:38:26.759191+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:07.326931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.749410+0000","last_change":"2024-04-25T08:38:26.745910+0000","last_active":"2024-04-25T08:38:26.749410+0000","last_peered":"2024-04-25T08:38:26.749410+0000","last_clean":"2024-04-25T08:38:26.749410+0000","last_became_active":"2024-04-25T08:38:26.745654+0000","last_became_peered":"2024-04-25T08:38:26.745654+0000","last_unstale":"2024-04-25T08:38:26.749410+0000","last_undegraded":"2024-04-25T08:38:26.749410+0000","last_fullsized":"2024-04-25T08:38:26.749410+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:58.864522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.744497+0000","last_change":"2024-04-25T08:38:26.742550+0000","last_active":"2024-04-25T08:38:26.744497+0000","last_peered":"2024-04-25T08:38:26.744497+0000","last_clean":"2024-04-25T08:38:26.744497+0000","last_became_active":"2024-04-25T08:38:26.742236+0000","last_became_peered":"2024-04-25T08:38:26.742236+0000","last_unstale":"2024-04-25T08:38:26.744497+0000","last_undegraded":"2024-04-25T08:38:26.744497+0000","last_fullsized":"2024-04-25T08:38:26.744497+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:03.303938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.756227+0000","last_change":"2024-04-25T08:38:26.755691+0000","last_active":"2024-04-25T08:38:26.756227+0000","last_peered":"2024-04-25T08:38:26.756227+0000","last_clean":"2024-04-25T08:38:26.756227+0000","last_became_active":"2024-04-25T08:38:26.755373+0000","last_became_peered":"2024-04-25T08:38:26.755373+0000","last_unstale":"2024-04-25T08:38:26.756227+0000","last_undegraded":"2024-04-25T08:38:26.756227+0000","last_fullsized":"2024-04-25T08:38:26.756227+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:01.555678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.765337+0000","last_change":"2024-04-25T08:38:26.760098+0000","last_active":"2024-04-25T08:38:26.765337+0000","last_peered":"2024-04-25T08:38:26.765337+0000","last_clean":"2024-04-25T08:38:26.765337+0000","last_became_active":"2024-04-25T08:38:26.757944+0000","last_became_peered":"2024-04-25T08:38:26.757944+0000","last_unstale":"2024-04-25T08:38:26.765337+0000","last_undegraded":"2024-04-25T08:38:26.765337+0000","last_fullsized":"2024-04-25T08:38:26.765337+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:40.858231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.741734+0000","last_change":"2024-04-25T08:38:26.737429+0000","last_active":"2024-04-25T08:38:26.741734+0000","last_peered":"2024-04-25T08:38:26.741734+0000","last_clean":"2024-04-25T08:38:26.741734+0000","last_became_active":"2024-04-25T08:38:26.737020+0000","last_became_peered":"2024-04-25T08:38:26.737020+0000","last_unstale":"2024-04-25T08:38:26.741734+0000","last_undegraded":"2024-04-25T08:38:26.741734+0000","last_fullsized":"2024-04-25T08:38:26.741734+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:02.220030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.756252+0000","last_change":"2024-04-25T08:38:26.755720+0000","last_active":"2024-04-25T08:38:26.756252+0000","last_peered":"2024-04-25T08:38:26.756252+0000","last_clean":"2024-04-25T08:38:26.756252+0000","last_became_active":"2024-04-25T08:38:26.755444+0000","last_became_peered":"2024-04-25T08:38:26.755444+0000","last_unstale":"2024-04-25T08:38:26.756252+0000","last_undegraded":"2024-04-25T08:38:26.756252+0000","last_fullsized":"2024-04-25T08:38:26.756252+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:16.178662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.760850+0000","last_change":"2024-04-25T08:38:26.760387+0000","last_active":"2024-04-25T08:38:26.760850+0000","last_peered":"2024-04-25T08:38:26.760850+0000","last_clean":"2024-04-25T08:38:26.760850+0000","last_became_active":"2024-04-25T08:38:26.760081+0000","last_became_peered":"2024-04-25T08:38:26.760081+0000","last_unstale":"2024-04-25T08:38:26.760850+0000","last_undegraded":"2024-04-25T08:38:26.760850+0000","last_fullsized":"2024-04-25T08:38:26.760850+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:47.774811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.751065+0000","last_change":"2024-04-25T08:38:26.743924+0000","last_active":"2024-04-25T08:38:26.751065+0000","last_peered":"2024-04-25T08:38:26.751065+0000","last_clean":"2024-04-25T08:38:26.751065+0000","last_became_active":"2024-04-25T08:38:26.743591+0000","last_became_peered":"2024-04-25T08:38:26.743591+0000","last_unstale":"2024-04-25T08:38:26.751065+0000","last_undegraded":"2024-04-25T08:38:26.751065+0000","last_fullsized":"2024-04-25T08:38:26.751065+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:22.141165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.741615+0000","last_change":"2024-04-25T08:38:26.740310+0000","last_active":"2024-04-25T08:38:26.741615+0000","last_peered":"2024-04-25T08:38:26.741615+0000","last_clean":"2024-04-25T08:38:26.741615+0000","last_became_active":"2024-04-25T08:38:26.740043+0000","last_became_peered":"2024-04-25T08:38:26.740043+0000","last_unstale":"2024-04-25T08:38:26.741615+0000","last_undegraded":"2024-04-25T08:38:26.741615+0000","last_fullsized":"2024-04-25T08:38:26.741615+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:37.044827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.763318+0000","last_change":"2024-04-25T08:38:26.762866+0000","last_active":"2024-04-25T08:38:26.763318+0000","last_peered":"2024-04-25T08:38:26.763318+0000","last_clean":"2024-04-25T08:38:26.763318+0000","last_became_active":"2024-04-25T08:38:26.762344+0000","last_became_peered":"2024-04-25T08:38:26.762344+0000","last_unstale":"2024-04-25T08:38:26.763318+0000","last_undegraded":"2024-04-25T08:38:26.763318+0000","last_fullsized":"2024-04-25T08:38:26.763318+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:37.915525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.736789+0000","last_change":"2024-04-25T08:38:26.735107+0000","last_active":"2024-04-25T08:38:26.736789+0000","last_peered":"2024-04-25T08:38:26.736789+0000","last_clean":"2024-04-25T08:38:26.736789+0000","last_became_active":"2024-04-25T08:38:26.734798+0000","last_became_peered":"2024-04-25T08:38:26.734798+0000","last_unstale":"2024-04-25T08:38:26.736789+0000","last_undegraded":"2024-04-25T08:38:26.736789+0000","last_fullsized":"2024-04-25T08:38:26.736789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:44.760159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.765223+0000","last_change":"2024-04-25T08:38:26.757441+0000","last_active":"2024-04-25T08:38:26.765223+0000","last_peered":"2024-04-25T08:38:26.765223+0000","last_clean":"2024-04-25T08:38:26.765223+0000","last_became_active":"2024-04-25T08:38:26.756675+0000","last_became_peered":"2024-04-25T08:38:26.756675+0000","last_unstale":"2024-04-25T08:38:26.765223+0000","last_undegraded":"2024-04-25T08:38:26.765223+0000","last_fullsized":"2024-04-25T08:38:26.765223+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:16.178662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.739525+0000","last_change":"2024-04-25T08:38:26.738797+0000","last_active":"2024-04-25T08:38:26.739525+0000","last_peered":"2024-04-25T08:38:26.739525+0000","last_clean":"2024-04-25T08:38:26.739525+0000","last_became_active":"2024-04-25T08:38:26.738475+0000","last_became_peered":"2024-04-25T08:38:26.738475+0000","last_unstale":"2024-04-25T08:38:26.739525+0000","last_undegraded":"2024-04-25T08:38:26.739525+0000","last_fullsized":"2024-04-25T08:38:26.739525+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:10.727609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.743834+0000","last_change":"2024-04-25T08:38:26.742359+0000","last_active":"2024-04-25T08:38:26.743834+0000","last_peered":"2024-04-25T08:38:26.743834+0000","last_clean":"2024-04-25T08:38:26.743834+0000","last_became_active":"2024-04-25T08:38:26.742078+0000","last_became_peered":"2024-04-25T08:38:26.742078+0000","last_unstale":"2024-04-25T08:38:26.743834+0000","last_undegraded":"2024-04-25T08:38:26.743834+0000","last_fullsized":"2024-04-25T08:38:26.743834+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:50.567885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.763313+0000","last_change":"2024-04-25T08:38:26.762604+0000","last_active":"2024-04-25T08:38:26.763313+0000","last_peered":"2024-04-25T08:38:26.763313+0000","last_clean":"2024-04-25T08:38:26.763313+0000","last_became_active":"2024-04-25T08:38:26.762294+0000","last_became_peered":"2024-04-25T08:38:26.762294+0000","last_unstale":"2024-04-25T08:38:26.763313+0000","last_undegraded":"2024-04-25T08:38:26.763313+0000","last_fullsized":"2024-04-25T08:38:26.763313+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:03.303938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759464+0000","last_change":"2024-04-25T08:38:26.756706+0000","last_active":"2024-04-25T08:38:26.759464+0000","last_peered":"2024-04-25T08:38:26.759464+0000","last_clean":"2024-04-25T08:38:26.759464+0000","last_became_active":"2024-04-25T08:38:26.756206+0000","last_became_peered":"2024-04-25T08:38:26.756206+0000","last_unstale":"2024-04-25T08:38:26.759464+0000","last_undegraded":"2024-04-25T08:38:26.759464+0000","last_fullsized":"2024-04-25T08:38:26.759464+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:25.728911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759101+0000","last_change":"2024-04-25T08:38:26.749527+0000","last_active":"2024-04-25T08:38:26.759101+0000","last_peered":"2024-04-25T08:38:26.759101+0000","last_clean":"2024-04-25T08:38:26.759101+0000","last_became_active":"2024-04-25T08:38:26.749119+0000","last_became_peered":"2024-04-25T08:38:26.749119+0000","last_unstale":"2024-04-25T08:38:26.759101+0000","last_undegraded":"2024-04-25T08:38:26.759101+0000","last_fullsized":"2024-04-25T08:38:26.759101+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:36.034531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"95'4","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.758774+0000","last_change":"2024-04-25T08:38:25.741437+0000","last_active":"2024-04-25T08:38:26.758774+0000","last_peered":"2024-04-25T08:38:26.758774+0000","last_clean":"2024-04-25T08:38:26.758774+0000","last_became_active":"2024-04-25T08:38:25.741162+0000","last_became_peered":"2024-04-25T08:38:25.741162+0000","last_unstale":"2024-04-25T08:38:26.758774+0000","last_undegraded":"2024-04-25T08:38:26.758774+0000","last_fullsized":"2024-04-25T08:38:26.758774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:26.772853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759498+0000","last_change":"2024-04-25T08:38:26.758679+0000","last_active":"2024-04-25T08:38:26.759498+0000","last_peered":"2024-04-25T08:38:26.759498+0000","last_clean":"2024-04-25T08:38:26.759498+0000","last_became_active":"2024-04-25T08:38:26.758412+0000","last_became_peered":"2024-04-25T08:38:26.758412+0000","last_unstale":"2024-04-25T08:38:26.759498+0000","last_undegraded":"2024-04-25T08:38:26.759498+0000","last_fullsized":"2024-04-25T08:38:26.759498+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:31.654530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.749821+0000","last_change":"2024-04-25T08:38:25.740510+0000","last_active":"2024-04-25T08:38:26.749821+0000","last_peered":"2024-04-25T08:38:26.749821+0000","last_clean":"2024-04-25T08:38:26.749821+0000","last_became_active":"2024-04-25T08:38:25.739259+0000","last_became_peered":"2024-04-25T08:38:25.739259+0000","last_unstale":"2024-04-25T08:38:26.749821+0000","last_undegraded":"2024-04-25T08:38:26.749821+0000","last_fullsized":"2024-04-25T08:38:26.749821+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:52.285256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.761344+0000","last_change":"2024-04-25T08:38:26.760958+0000","last_active":"2024-04-25T08:38:26.761344+0000","last_peered":"2024-04-25T08:38:26.761344+0000","last_clean":"2024-04-25T08:38:26.761344+0000","last_became_active":"2024-04-25T08:38:26.760636+0000","last_became_peered":"2024-04-25T08:38:26.760636+0000","last_unstale":"2024-04-25T08:38:26.761344+0000","last_undegraded":"2024-04-25T08:38:26.761344+0000","last_fullsized":"2024-04-25T08:38:26.761344+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:49.707641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.752584+0000","last_change":"2024-04-25T08:38:25.744831+0000","last_active":"2024-04-25T08:38:26.752584+0000","last_peered":"2024-04-25T08:38:26.752584+0000","last_clean":"2024-04-25T08:38:26.752584+0000","last_became_active":"2024-04-25T08:38:25.744621+0000","last_became_peered":"2024-04-25T08:38:25.744621+0000","last_unstale":"2024-04-25T08:38:26.752584+0000","last_undegraded":"2024-04-25T08:38:26.752584+0000","last_fullsized":"2024-04-25T08:38:26.752584+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:44.426879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.757851+0000","last_change":"2024-04-25T08:38:26.751681+0000","last_active":"2024-04-25T08:38:26.757851+0000","last_peered":"2024-04-25T08:38:26.757851+0000","last_clean":"2024-04-25T08:38:26.757851+0000","last_became_active":"2024-04-25T08:38:26.749721+0000","last_became_peered":"2024-04-25T08:38:26.749721+0000","last_unstale":"2024-04-25T08:38:26.757851+0000","last_undegraded":"2024-04-25T08:38:26.757851+0000","last_fullsized":"2024-04-25T08:38:26.757851+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:17.771155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.760077+0000","last_change":"2024-04-25T08:38:25.737213+0000","last_active":"2024-04-25T08:38:26.760077+0000","last_peered":"2024-04-25T08:38:26.760077+0000","last_clean":"2024-04-25T08:38:26.760077+0000","last_became_active":"2024-04-25T08:38:25.736448+0000","last_became_peered":"2024-04-25T08:38:25.736448+0000","last_unstale":"2024-04-25T08:38:26.760077+0000","last_undegraded":"2024-04-25T08:38:26.760077+0000","last_fullsized":"2024-04-25T08:38:26.760077+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:00.553243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.768100+0000","last_change":"2024-04-25T08:38:26.762500+0000","last_active":"2024-04-25T08:38:26.768100+0000","last_peered":"2024-04-25T08:38:26.768100+0000","last_clean":"2024-04-25T08:38:26.768100+0000","last_became_active":"2024-04-25T08:38:26.759785+0000","last_became_peered":"2024-04-25T08:38:26.759785+0000","last_unstale":"2024-04-25T08:38:26.768100+0000","last_undegraded":"2024-04-25T08:38:26.768100+0000","last_fullsized":"2024-04-25T08:38:26.768100+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:49.659530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.749529+0000","last_change":"2024-04-25T08:38:25.747659+0000","last_active":"2024-04-25T08:38:26.749529+0000","last_peered":"2024-04-25T08:38:26.749529+0000","last_clean":"2024-04-25T08:38:26.749529+0000","last_became_active":"2024-04-25T08:38:25.746496+0000","last_became_peered":"2024-04-25T08:38:25.746496+0000","last_unstale":"2024-04-25T08:38:26.749529+0000","last_undegraded":"2024-04-25T08:38:26.749529+0000","last_fullsized":"2024-04-25T08:38:26.749529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:56.912745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.753063+0000","last_change":"2024-04-25T08:38:26.752692+0000","last_active":"2024-04-25T08:38:26.753063+0000","last_peered":"2024-04-25T08:38:26.753063+0000","last_clean":"2024-04-25T08:38:26.753063+0000","last_became_active":"2024-04-25T08:38:26.752399+0000","last_became_peered":"2024-04-25T08:38:26.752399+0000","last_unstale":"2024-04-25T08:38:26.753063+0000","last_undegraded":"2024-04-25T08:38:26.753063+0000","last_fullsized":"2024-04-25T08:38:26.753063+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:49.659530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.766617+0000","last_change":"2024-04-25T08:38:25.747074+0000","last_active":"2024-04-25T08:38:26.766617+0000","last_peered":"2024-04-25T08:38:26.766617+0000","last_clean":"2024-04-25T08:38:26.766617+0000","last_became_active":"2024-04-25T08:38:25.746763+0000","last_became_peered":"2024-04-25T08:38:25.746763+0000","last_unstale":"2024-04-25T08:38:26.766617+0000","last_undegraded":"2024-04-25T08:38:26.766617+0000","last_fullsized":"2024-04-25T08:38:26.766617+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:24.815005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.764120+0000","last_change":"2024-04-25T08:38:26.763175+0000","last_active":"2024-04-25T08:38:26.764120+0000","last_peered":"2024-04-25T08:38:26.764120+0000","last_clean":"2024-04-25T08:38:26.764120+0000","last_became_active":"2024-04-25T08:38:26.762759+0000","last_became_peered":"2024-04-25T08:38:26.762759+0000","last_unstale":"2024-04-25T08:38:26.764120+0000","last_undegraded":"2024-04-25T08:38:26.764120+0000","last_fullsized":"2024-04-25T08:38:26.764120+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:03.750859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.735272+0000","last_change":"2024-04-25T08:38:25.740909+0000","last_active":"2024-04-25T08:38:26.735272+0000","last_peered":"2024-04-25T08:38:26.735272+0000","last_clean":"2024-04-25T08:38:26.735272+0000","last_became_active":"2024-04-25T08:38:25.740576+0000","last_became_peered":"2024-04-25T08:38:25.740576+0000","last_unstale":"2024-04-25T08:38:26.735272+0000","last_undegraded":"2024-04-25T08:38:26.735272+0000","last_fullsized":"2024-04-25T08:38:26.735272+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:44.679631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.760826+0000","last_change":"2024-04-25T08:38:26.750794+0000","last_active":"2024-04-25T08:38:26.760826+0000","last_peered":"2024-04-25T08:38:26.760826+0000","last_clean":"2024-04-25T08:38:26.760826+0000","last_became_active":"2024-04-25T08:38:26.744556+0000","last_became_peered":"2024-04-25T08:38:26.744556+0000","last_unstale":"2024-04-25T08:38:26.760826+0000","last_undegraded":"2024-04-25T08:38:26.760826+0000","last_fullsized":"2024-04-25T08:38:26.760826+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:27.775288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.735051+0000","last_change":"2024-04-25T08:38:25.742614+0000","last_active":"2024-04-25T08:38:26.735051+0000","last_peered":"2024-04-25T08:38:26.735051+0000","last_clean":"2024-04-25T08:38:26.735051+0000","last_became_active":"2024-04-25T08:38:25.741874+0000","last_became_peered":"2024-04-25T08:38:25.741874+0000","last_unstale":"2024-04-25T08:38:26.735051+0000","last_undegraded":"2024-04-25T08:38:26.735051+0000","last_fullsized":"2024-04-25T08:38:26.735051+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:53.336653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.746434+0000","last_change":"2024-04-25T08:38:26.742447+0000","last_active":"2024-04-25T08:38:26.746434+0000","last_peered":"2024-04-25T08:38:26.746434+0000","last_clean":"2024-04-25T08:38:26.746434+0000","last_became_active":"2024-04-25T08:38:26.742139+0000","last_became_peered":"2024-04-25T08:38:26.742139+0000","last_unstale":"2024-04-25T08:38:26.746434+0000","last_undegraded":"2024-04-25T08:38:26.746434+0000","last_fullsized":"2024-04-25T08:38:26.746434+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:25.817440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.739351+0000","last_change":"2024-04-25T08:38:25.746816+0000","last_active":"2024-04-25T08:38:26.739351+0000","last_peered":"2024-04-25T08:38:26.739351+0000","last_clean":"2024-04-25T08:38:26.739351+0000","last_became_active":"2024-04-25T08:38:25.746493+0000","last_became_peered":"2024-04-25T08:38:25.746493+0000","last_unstale":"2024-04-25T08:38:26.739351+0000","last_undegraded":"2024-04-25T08:38:26.739351+0000","last_fullsized":"2024-04-25T08:38:26.739351+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:11.581685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.744888+0000","last_change":"2024-04-25T08:38:26.744445+0000","last_active":"2024-04-25T08:38:26.744888+0000","last_peered":"2024-04-25T08:38:26.744888+0000","last_clean":"2024-04-25T08:38:26.744888+0000","last_became_active":"2024-04-25T08:38:26.744141+0000","last_became_peered":"2024-04-25T08:38:26.744141+0000","last_unstale":"2024-04-25T08:38:26.744888+0000","last_undegraded":"2024-04-25T08:38:26.744888+0000","last_fullsized":"2024-04-25T08:38:26.744888+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:10.827812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.736004+0000","last_change":"2024-04-25T08:38:25.750043+0000","last_active":"2024-04-25T08:38:26.736004+0000","last_peered":"2024-04-25T08:38:26.736004+0000","last_clean":"2024-04-25T08:38:26.736004+0000","last_became_active":"2024-04-25T08:38:25.749731+0000","last_became_peered":"2024-04-25T08:38:25.749731+0000","last_unstale":"2024-04-25T08:38:26.736004+0000","last_undegraded":"2024-04-25T08:38:26.736004+0000","last_fullsized":"2024-04-25T08:38:26.736004+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:23.020979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.752840+0000","last_change":"2024-04-25T08:38:26.750870+0000","last_active":"2024-04-25T08:38:26.752840+0000","last_peered":"2024-04-25T08:38:26.752840+0000","last_clean":"2024-04-25T08:38:26.752840+0000","last_became_active":"2024-04-25T08:38:26.750572+0000","last_became_peered":"2024-04-25T08:38:26.750572+0000","last_unstale":"2024-04-25T08:38:26.752840+0000","last_undegraded":"2024-04-25T08:38:26.752840+0000","last_fullsized":"2024-04-25T08:38:26.752840+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:54.339088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"95'3","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759294+0000","last_change":"2024-04-25T08:38:25.746533+0000","last_active":"2024-04-25T08:38:26.759294+0000","last_peered":"2024-04-25T08:38:26.759294+0000","last_clean":"2024-04-25T08:38:26.759294+0000","last_became_active":"2024-04-25T08:38:25.745660+0000","last_became_peered":"2024-04-25T08:38:25.745660+0000","last_unstale":"2024-04-25T08:38:26.759294+0000","last_undegraded":"2024-04-25T08:38:26.759294+0000","last_fullsized":"2024-04-25T08:38:26.759294+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:49.066298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.765341+0000","last_change":"2024-04-25T08:38:26.760593+0000","last_active":"2024-04-25T08:38:26.765341+0000","last_peered":"2024-04-25T08:38:26.765341+0000","last_clean":"2024-04-25T08:38:26.765341+0000","last_became_active":"2024-04-25T08:38:26.760307+0000","last_became_peered":"2024-04-25T08:38:26.760307+0000","last_unstale":"2024-04-25T08:38:26.765341+0000","last_undegraded":"2024-04-25T08:38:26.765341+0000","last_fullsized":"2024-04-25T08:38:26.765341+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:59.525148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"95'6","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.764041+0000","last_change":"2024-04-25T08:38:25.741819+0000","last_active":"2024-04-25T08:38:26.764041+0000","last_peered":"2024-04-25T08:38:26.764041+0000","last_clean":"2024-04-25T08:38:26.764041+0000","last_became_active":"2024-04-25T08:38:25.740763+0000","last_became_peered":"2024-04-25T08:38:25.740763+0000","last_unstale":"2024-04-25T08:38:26.764041+0000","last_undegraded":"2024-04-25T08:38:26.764041+0000","last_fullsized":"2024-04-25T08:38:26.764041+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:21.804361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.762480+0000","last_change":"2024-04-25T08:38:26.744634+0000","last_active":"2024-04-25T08:38:26.762480+0000","last_peered":"2024-04-25T08:38:26.762480+0000","last_clean":"2024-04-25T08:38:26.762480+0000","last_became_active":"2024-04-25T08:38:26.744269+0000","last_became_peered":"2024-04-25T08:38:26.744269+0000","last_unstale":"2024-04-25T08:38:26.762480+0000","last_undegraded":"2024-04-25T08:38:26.762480+0000","last_fullsized":"2024-04-25T08:38:26.762480+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:47.208427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.758590+0000","last_change":"2024-04-25T08:38:25.741677+0000","last_active":"2024-04-25T08:38:26.758590+0000","last_peered":"2024-04-25T08:38:26.758590+0000","last_clean":"2024-04-25T08:38:26.758590+0000","last_became_active":"2024-04-25T08:38:25.740828+0000","last_became_peered":"2024-04-25T08:38:25.740828+0000","last_unstale":"2024-04-25T08:38:26.758590+0000","last_undegraded":"2024-04-25T08:38:26.758590+0000","last_fullsized":"2024-04-25T08:38:26.758590+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:39.455847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.732494+0000","last_change":"2024-04-25T08:38:26.732124+0000","last_active":"2024-04-25T08:38:26.732494+0000","last_peered":"2024-04-25T08:38:26.732494+0000","last_clean":"2024-04-25T08:38:26.732494+0000","last_became_active":"2024-04-25T08:38:26.731787+0000","last_became_peered":"2024-04-25T08:38:26.731787+0000","last_unstale":"2024-04-25T08:38:26.732494+0000","last_undegraded":"2024-04-25T08:38:26.732494+0000","last_fullsized":"2024-04-25T08:38:26.732494+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:44.683993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.747785+0000","last_change":"2024-04-25T08:38:25.744898+0000","last_active":"2024-04-25T08:38:26.747785+0000","last_peered":"2024-04-25T08:38:26.747785+0000","last_clean":"2024-04-25T08:38:26.747785+0000","last_became_active":"2024-04-25T08:38:25.744625+0000","last_became_peered":"2024-04-25T08:38:25.744625+0000","last_unstale":"2024-04-25T08:38:26.747785+0000","last_undegraded":"2024-04-25T08:38:26.747785+0000","last_fullsized":"2024-04-25T08:38:26.747785+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:12.974522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.763658+0000","last_change":"2024-04-25T08:38:26.761802+0000","last_active":"2024-04-25T08:38:26.763658+0000","last_peered":"2024-04-25T08:38:26.763658+0000","last_clean":"2024-04-25T08:38:26.763658+0000","last_became_active":"2024-04-25T08:38:26.761512+0000","last_became_peered":"2024-04-25T08:38:26.761512+0000","last_unstale":"2024-04-25T08:38:26.763658+0000","last_undegraded":"2024-04-25T08:38:26.763658+0000","last_fullsized":"2024-04-25T08:38:26.763658+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:44.609519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.756189+0000","last_change":"2024-04-25T08:38:25.748596+0000","last_active":"2024-04-25T08:38:26.756189+0000","last_peered":"2024-04-25T08:38:26.756189+0000","last_clean":"2024-04-25T08:38:26.756189+0000","last_became_active":"2024-04-25T08:38:25.748252+0000","last_became_peered":"2024-04-25T08:38:25.748252+0000","last_unstale":"2024-04-25T08:38:26.756189+0000","last_undegraded":"2024-04-25T08:38:26.756189+0000","last_fullsized":"2024-04-25T08:38:26.756189+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:57.461368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.736749+0000","last_change":"2024-04-25T08:38:26.735743+0000","last_active":"2024-04-25T08:38:26.736749+0000","last_peered":"2024-04-25T08:38:26.736749+0000","last_clean":"2024-04-25T08:38:26.736749+0000","last_became_active":"2024-04-25T08:38:26.735473+0000","last_became_peered":"2024-04-25T08:38:26.735473+0000","last_unstale":"2024-04-25T08:38:26.736749+0000","last_undegraded":"2024-04-25T08:38:26.736749+0000","last_fullsized":"2024-04-25T08:38:26.736749+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:00.095138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"95'8","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.768977+0000","last_change":"2024-04-25T08:38:25.747426+0000","last_active":"2024-04-25T08:38:26.768977+0000","last_peered":"2024-04-25T08:38:26.768977+0000","last_clean":"2024-04-25T08:38:26.768977+0000","last_became_active":"2024-04-25T08:38:25.746951+0000","last_became_peered":"2024-04-25T08:38:25.746951+0000","last_unstale":"2024-04-25T08:38:26.768977+0000","last_undegraded":"2024-04-25T08:38:26.768977+0000","last_fullsized":"2024-04-25T08:38:26.768977+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:05.319562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759126+0000","last_change":"2024-04-25T08:38:26.754438+0000","last_active":"2024-04-25T08:38:26.759126+0000","last_peered":"2024-04-25T08:38:26.759126+0000","last_clean":"2024-04-25T08:38:26.759126+0000","last_became_active":"2024-04-25T08:38:26.754123+0000","last_became_peered":"2024-04-25T08:38:26.754123+0000","last_unstale":"2024-04-25T08:38:26.759126+0000","last_undegraded":"2024-04-25T08:38:26.759126+0000","last_fullsized":"2024-04-25T08:38:26.759126+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:27.422753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.751455+0000","last_change":"2024-04-25T08:38:25.733704+0000","last_active":"2024-04-25T08:38:26.751455+0000","last_peered":"2024-04-25T08:38:26.751455+0000","last_clean":"2024-04-25T08:38:26.751455+0000","last_became_active":"2024-04-25T08:38:25.733298+0000","last_became_peered":"2024-04-25T08:38:25.733298+0000","last_unstale":"2024-04-25T08:38:26.751455+0000","last_undegraded":"2024-04-25T08:38:26.751455+0000","last_fullsized":"2024-04-25T08:38:26.751455+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:15.176227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.740505+0000","last_change":"2024-04-25T08:38:26.739347+0000","last_active":"2024-04-25T08:38:26.740505+0000","last_peered":"2024-04-25T08:38:26.740505+0000","last_clean":"2024-04-25T08:38:26.740505+0000","last_became_active":"2024-04-25T08:38:26.739070+0000","last_became_peered":"2024-04-25T08:38:26.739070+0000","last_unstale":"2024-04-25T08:38:26.740505+0000","last_undegraded":"2024-04-25T08:38:26.740505+0000","last_fullsized":"2024-04-25T08:38:26.740505+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:06.321997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.751836+0000","last_change":"2024-04-25T08:38:25.741392+0000","last_active":"2024-04-25T08:38:26.751836+0000","last_peered":"2024-04-25T08:38:26.751836+0000","last_clean":"2024-04-25T08:38:26.751836+0000","last_became_active":"2024-04-25T08:38:25.741053+0000","last_became_peered":"2024-04-25T08:38:25.741053+0000","last_unstale":"2024-04-25T08:38:26.751836+0000","last_undegraded":"2024-04-25T08:38:26.751836+0000","last_fullsized":"2024-04-25T08:38:26.751836+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.223454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.737895+0000","last_change":"2024-04-25T08:38:26.734577+0000","last_active":"2024-04-25T08:38:26.737895+0000","last_peered":"2024-04-25T08:38:26.737895+0000","last_clean":"2024-04-25T08:38:26.737895+0000","last_became_active":"2024-04-25T08:38:26.734101+0000","last_became_peered":"2024-04-25T08:38:26.734101+0000","last_unstale":"2024-04-25T08:38:26.737895+0000","last_undegraded":"2024-04-25T08:38:26.737895+0000","last_fullsized":"2024-04-25T08:38:26.737895+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:51.015551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.762618+0000","last_change":"2024-04-25T08:38:25.748674+0000","last_active":"2024-04-25T08:38:26.762618+0000","last_peered":"2024-04-25T08:38:26.762618+0000","last_clean":"2024-04-25T08:38:26.762618+0000","last_became_active":"2024-04-25T08:38:25.748380+0000","last_became_peered":"2024-04-25T08:38:25.748380+0000","last_unstale":"2024-04-25T08:38:26.762618+0000","last_undegraded":"2024-04-25T08:38:26.762618+0000","last_fullsized":"2024-04-25T08:38:26.762618+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:14.485431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.769490+0000","last_change":"2024-04-25T08:38:26.768902+0000","last_active":"2024-04-25T08:38:26.769490+0000","last_peered":"2024-04-25T08:38:26.769490+0000","last_clean":"2024-04-25T08:38:26.769490+0000","last_became_active":"2024-04-25T08:38:26.768587+0000","last_became_peered":"2024-04-25T08:38:26.768587+0000","last_unstale":"2024-04-25T08:38:26.769490+0000","last_undegraded":"2024-04-25T08:38:26.769490+0000","last_fullsized":"2024-04-25T08:38:26.769490+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:36.811665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.752983+0000","last_change":"2024-04-25T08:38:25.743224+0000","last_active":"2024-04-25T08:38:26.752983+0000","last_peered":"2024-04-25T08:38:26.752983+0000","last_clean":"2024-04-25T08:38:26.752983+0000","last_became_active":"2024-04-25T08:38:25.742773+0000","last_became_peered":"2024-04-25T08:38:25.742773+0000","last_unstale":"2024-04-25T08:38:26.752983+0000","last_undegraded":"2024-04-25T08:38:26.752983+0000","last_fullsized":"2024-04-25T08:38:26.752983+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:46.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.747634+0000","last_change":"2024-04-25T08:38:26.746584+0000","last_active":"2024-04-25T08:38:26.747634+0000","last_peered":"2024-04-25T08:38:26.747634+0000","last_clean":"2024-04-25T08:38:26.747634+0000","last_became_active":"2024-04-25T08:38:26.746315+0000","last_became_peered":"2024-04-25T08:38:26.746315+0000","last_unstale":"2024-04-25T08:38:26.747634+0000","last_undegraded":"2024-04-25T08:38:26.747634+0000","last_fullsized":"2024-04-25T08:38:26.747634+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:45.682066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.732344+0000","last_change":"2024-04-25T08:38:25.749965+0000","last_active":"2024-04-25T08:38:26.732344+0000","last_peered":"2024-04-25T08:38:26.732344+0000","last_clean":"2024-04-25T08:38:26.732344+0000","last_became_active":"2024-04-25T08:38:25.749575+0000","last_became_peered":"2024-04-25T08:38:25.749575+0000","last_unstale":"2024-04-25T08:38:26.732344+0000","last_undegraded":"2024-04-25T08:38:26.732344+0000","last_fullsized":"2024-04-25T08:38:26.732344+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:52.536663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.763292+0000","last_change":"2024-04-25T08:38:26.762578+0000","last_active":"2024-04-25T08:38:26.763292+0000","last_peered":"2024-04-25T08:38:26.763292+0000","last_clean":"2024-04-25T08:38:26.763292+0000","last_became_active":"2024-04-25T08:38:26.762304+0000","last_became_peered":"2024-04-25T08:38:26.762304+0000","last_unstale":"2024-04-25T08:38:26.763292+0000","last_undegraded":"2024-04-25T08:38:26.763292+0000","last_fullsized":"2024-04-25T08:38:26.763292+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:24.023414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.737761+0000","last_change":"2024-04-25T08:38:25.746037+0000","last_active":"2024-04-25T08:38:26.737761+0000","last_peered":"2024-04-25T08:38:26.737761+0000","last_clean":"2024-04-25T08:38:26.737761+0000","last_became_active":"2024-04-25T08:38:25.745723+0000","last_became_peered":"2024-04-25T08:38:25.745723+0000","last_unstale":"2024-04-25T08:38:26.737761+0000","last_undegraded":"2024-04-25T08:38:26.737761+0000","last_fullsized":"2024-04-25T08:38:26.737761+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:48.657096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.757690+0000","last_change":"2024-04-25T08:38:26.751764+0000","last_active":"2024-04-25T08:38:26.757690+0000","last_peered":"2024-04-25T08:38:26.757690+0000","last_clean":"2024-04-25T08:38:26.757690+0000","last_became_active":"2024-04-25T08:38:26.749663+0000","last_became_peered":"2024-04-25T08:38:26.749663+0000","last_unstale":"2024-04-25T08:38:26.757690+0000","last_undegraded":"2024-04-25T08:38:26.757690+0000","last_fullsized":"2024-04-25T08:38:26.757690+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:09.673659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.732074+0000","last_change":"2024-04-25T08:38:25.744522+0000","last_active":"2024-04-25T08:38:26.732074+0000","last_peered":"2024-04-25T08:38:26.732074+0000","last_clean":"2024-04-25T08:38:26.732074+0000","last_became_active":"2024-04-25T08:38:25.744250+0000","last_became_peered":"2024-04-25T08:38:25.744250+0000","last_unstale":"2024-04-25T08:38:26.732074+0000","last_undegraded":"2024-04-25T08:38:26.732074+0000","last_fullsized":"2024-04-25T08:38:26.732074+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.380067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.736765+0000","last_change":"2024-04-25T08:38:26.736277+0000","last_active":"2024-04-25T08:38:26.736765+0000","last_peered":"2024-04-25T08:38:26.736765+0000","last_clean":"2024-04-25T08:38:26.736765+0000","last_became_active":"2024-04-25T08:38:26.735904+0000","last_became_peered":"2024-04-25T08:38:26.735904+0000","last_unstale":"2024-04-25T08:38:26.736765+0000","last_undegraded":"2024-04-25T08:38:26.736765+0000","last_fullsized":"2024-04-25T08:38:26.736765+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:59.525148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.752317+0000","last_change":"2024-04-25T08:38:25.732510+0000","last_active":"2024-04-25T08:38:26.752317+0000","last_peered":"2024-04-25T08:38:26.752317+0000","last_clean":"2024-04-25T08:38:26.752317+0000","last_became_active":"2024-04-25T08:38:25.732228+0000","last_became_peered":"2024-04-25T08:38:25.732228+0000","last_unstale":"2024-04-25T08:38:26.752317+0000","last_undegraded":"2024-04-25T08:38:26.752317+0000","last_fullsized":"2024-04-25T08:38:26.752317+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:15.714624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.758379+0000","last_change":"2024-04-25T08:38:26.755389+0000","last_active":"2024-04-25T08:38:26.758379+0000","last_peered":"2024-04-25T08:38:26.758379+0000","last_clean":"2024-04-25T08:38:26.758379+0000","last_became_active":"2024-04-25T08:38:26.753204+0000","last_became_peered":"2024-04-25T08:38:26.753204+0000","last_unstale":"2024-04-25T08:38:26.758379+0000","last_undegraded":"2024-04-25T08:38:26.758379+0000","last_fullsized":"2024-04-25T08:38:26.758379+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:54.067829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.730771+0000","last_change":"2024-04-25T08:38:25.741760+0000","last_active":"2024-04-25T08:38:26.730771+0000","last_peered":"2024-04-25T08:38:26.730771+0000","last_clean":"2024-04-25T08:38:26.730771+0000","last_became_active":"2024-04-25T08:38:25.740807+0000","last_became_peered":"2024-04-25T08:38:25.740807+0000","last_unstale":"2024-04-25T08:38:26.730771+0000","last_undegraded":"2024-04-25T08:38:26.730771+0000","last_fullsized":"2024-04-25T08:38:26.730771+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:32.412304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.749172+0000","last_change":"2024-04-25T08:38:26.745102+0000","last_active":"2024-04-25T08:38:26.749172+0000","last_peered":"2024-04-25T08:38:26.749172+0000","last_clean":"2024-04-25T08:38:26.749172+0000","last_became_active":"2024-04-25T08:38:26.744821+0000","last_became_peered":"2024-04-25T08:38:26.744821+0000","last_unstale":"2024-04-25T08:38:26.749172+0000","last_undegraded":"2024-04-25T08:38:26.749172+0000","last_fullsized":"2024-04-25T08:38:26.749172+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:58.766753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.731406+0000","last_change":"2024-04-25T08:38:25.745864+0000","last_active":"2024-04-25T08:38:26.731406+0000","last_peered":"2024-04-25T08:38:26.731406+0000","last_clean":"2024-04-25T08:38:26.731406+0000","last_became_active":"2024-04-25T08:38:25.745524+0000","last_became_peered":"2024-04-25T08:38:25.745524+0000","last_unstale":"2024-04-25T08:38:26.731406+0000","last_undegraded":"2024-04-25T08:38:26.731406+0000","last_fullsized":"2024-04-25T08:38:26.731406+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:15.805008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.747133+0000","last_change":"2024-04-25T08:38:26.745323+0000","last_active":"2024-04-25T08:38:26.747133+0000","last_peered":"2024-04-25T08:38:26.747133+0000","last_clean":"2024-04-25T08:38:26.747133+0000","last_became_active":"2024-04-25T08:38:26.744709+0000","last_became_peered":"2024-04-25T08:38:26.744709+0000","last_unstale":"2024-04-25T08:38:26.747133+0000","last_undegraded":"2024-04-25T08:38:26.747133+0000","last_fullsized":"2024-04-25T08:38:26.747133+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:35.368835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.729169+0000","last_change":"2024-04-25T08:38:25.746710+0000","last_active":"2024-04-25T08:38:26.729169+0000","last_peered":"2024-04-25T08:38:26.729169+0000","last_clean":"2024-04-25T08:38:26.729169+0000","last_became_active":"2024-04-25T08:38:25.746399+0000","last_became_peered":"2024-04-25T08:38:25.746399+0000","last_unstale":"2024-04-25T08:38:26.729169+0000","last_undegraded":"2024-04-25T08:38:26.729169+0000","last_fullsized":"2024-04-25T08:38:26.729169+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:39.855796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.755493+0000","last_change":"2024-04-25T08:38:26.754753+0000","last_active":"2024-04-25T08:38:26.755493+0000","last_peered":"2024-04-25T08:38:26.755493+0000","last_clean":"2024-04-25T08:38:26.755493+0000","last_became_active":"2024-04-25T08:38:26.754457+0000","last_became_peered":"2024-04-25T08:38:26.754457+0000","last_unstale":"2024-04-25T08:38:26.755493+0000","last_undegraded":"2024-04-25T08:38:26.755493+0000","last_fullsized":"2024-04-25T08:38:26.755493+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:07.790364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.748179+0000","last_change":"2024-04-25T08:38:25.745574+0000","last_active":"2024-04-25T08:38:26.748179+0000","last_peered":"2024-04-25T08:38:26.748179+0000","last_clean":"2024-04-25T08:38:26.748179+0000","last_became_active":"2024-04-25T08:38:25.745298+0000","last_became_peered":"2024-04-25T08:38:25.745298+0000","last_unstale":"2024-04-25T08:38:26.748179+0000","last_undegraded":"2024-04-25T08:38:26.748179+0000","last_fullsized":"2024-04-25T08:38:26.748179+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:38.369059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.763967+0000","last_change":"2024-04-25T08:38:26.762078+0000","last_active":"2024-04-25T08:38:26.763967+0000","last_peered":"2024-04-25T08:38:26.763967+0000","last_clean":"2024-04-25T08:38:26.763967+0000","last_became_active":"2024-04-25T08:38:26.761678+0000","last_became_peered":"2024-04-25T08:38:26.761678+0000","last_unstale":"2024-04-25T08:38:26.763967+0000","last_undegraded":"2024-04-25T08:38:26.763967+0000","last_fullsized":"2024-04-25T08:38:26.763967+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:21.491526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.738464+0000","last_change":"2024-04-25T08:38:25.743465+0000","last_active":"2024-04-25T08:38:26.738464+0000","last_peered":"2024-04-25T08:38:26.738464+0000","last_clean":"2024-04-25T08:38:26.738464+0000","last_became_active":"2024-04-25T08:38:25.742760+0000","last_became_peered":"2024-04-25T08:38:25.742760+0000","last_unstale":"2024-04-25T08:38:26.738464+0000","last_undegraded":"2024-04-25T08:38:26.738464+0000","last_fullsized":"2024-04-25T08:38:26.738464+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:03.114483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.761290+0000","last_change":"2024-04-25T08:38:26.760909+0000","last_active":"2024-04-25T08:38:26.761290+0000","last_peered":"2024-04-25T08:38:26.761290+0000","last_clean":"2024-04-25T08:38:26.761290+0000","last_became_active":"2024-04-25T08:38:26.760621+0000","last_became_peered":"2024-04-25T08:38:26.760621+0000","last_unstale":"2024-04-25T08:38:26.761290+0000","last_undegraded":"2024-04-25T08:38:26.761290+0000","last_fullsized":"2024-04-25T08:38:26.761290+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:04.116918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.729438+0000","last_change":"2024-04-25T08:38:25.747783+0000","last_active":"2024-04-25T08:38:26.729438+0000","last_peered":"2024-04-25T08:38:26.729438+0000","last_clean":"2024-04-25T08:38:26.729438+0000","last_became_active":"2024-04-25T08:38:25.747165+0000","last_became_peered":"2024-04-25T08:38:25.747165+0000","last_unstale":"2024-04-25T08:38:26.729438+0000","last_undegraded":"2024-04-25T08:38:26.729438+0000","last_fullsized":"2024-04-25T08:38:26.729438+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:36.913090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.757714+0000","last_change":"2024-04-25T08:38:26.752251+0000","last_active":"2024-04-25T08:38:26.757714+0000","last_peered":"2024-04-25T08:38:26.757714+0000","last_clean":"2024-04-25T08:38:26.757714+0000","last_became_active":"2024-04-25T08:38:26.751975+0000","last_became_peered":"2024-04-25T08:38:26.751975+0000","last_unstale":"2024-04-25T08:38:26.757714+0000","last_undegraded":"2024-04-25T08:38:26.757714+0000","last_fullsized":"2024-04-25T08:38:26.757714+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:47.362122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.735176+0000","last_change":"2024-04-25T08:38:25.749831+0000","last_active":"2024-04-25T08:38:26.735176+0000","last_peered":"2024-04-25T08:38:26.735176+0000","last_clean":"2024-04-25T08:38:26.735176+0000","last_became_active":"2024-04-25T08:38:25.749601+0000","last_became_peered":"2024-04-25T08:38:25.749601+0000","last_unstale":"2024-04-25T08:38:26.735176+0000","last_undegraded":"2024-04-25T08:38:26.735176+0000","last_fullsized":"2024-04-25T08:38:26.735176+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:20.489091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.740672+0000","last_change":"2024-04-25T08:38:26.740103+0000","last_active":"2024-04-25T08:38:26.740672+0000","last_peered":"2024-04-25T08:38:26.740672+0000","last_clean":"2024-04-25T08:38:26.740672+0000","last_became_active":"2024-04-25T08:38:26.739782+0000","last_became_peered":"2024-04-25T08:38:26.739782+0000","last_unstale":"2024-04-25T08:38:26.740672+0000","last_undegraded":"2024-04-25T08:38:26.740672+0000","last_fullsized":"2024-04-25T08:38:26.740672+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:16.961096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.756477+0000","last_change":"2024-04-25T08:38:25.745798+0000","last_active":"2024-04-25T08:38:26.756477+0000","last_peered":"2024-04-25T08:38:26.756477+0000","last_clean":"2024-04-25T08:38:26.756477+0000","last_became_active":"2024-04-25T08:38:25.745487+0000","last_became_peered":"2024-04-25T08:38:25.745487+0000","last_unstale":"2024-04-25T08:38:26.756477+0000","last_undegraded":"2024-04-25T08:38:26.756477+0000","last_fullsized":"2024-04-25T08:38:26.756477+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:00.553243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":311,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.735110+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:38:26.735110+0000","last_peered":"2024-04-25T08:38:26.735110+0000","last_clean":"2024-04-25T08:38:26.735110+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:38:26.735110+0000","last_undegraded":"2024-04-25T08:38:26.735110+0000","last_fullsized":"2024-04-25T08:38:26.735110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574937,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59280,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312560,"statfs":{"total":96636764160,"available":96576061440,"internally_reserved":0,"allocated":1773568,"data_stored":1133863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54552,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317288,"statfs":{"total":96636764160,"available":96580902912,"internally_reserved":0,"allocated":1191936,"data_stored":545655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50376,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321464,"statfs":{"total":96636764160,"available":96585179136,"internally_reserved":0,"allocated":1171456,"data_stored":543439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574936,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64268,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307572,"statfs":{"total":96636764160,"available":96570953728,"internally_reserved":0,"allocated":1179648,"data_stored":543599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574938,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68420,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303420,"statfs":{"total":96636764160,"available":96566702080,"internally_reserved":0,"allocated":1171456,"data_stored":543439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574937,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58680,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313160,"statfs":{"total":96636764160,"available":96576675840,"internally_reserved":0,"allocated":1159168,"data_stored":536272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54524,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317316,"statfs":{"total":96636764160,"available":96580931584,"internally_reserved":0,"allocated":1159168,"data_stored":536272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574937,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46788,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325052,"statfs":{"total":96636764160,"available":96588853248,"internally_reserved":0,"allocated":1753088,"data_stored":1126640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:38:29.752 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:38:29.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.895+0000 7ffa8e607640 1 Processor -- start 2024-04-25T08:38:29.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.895+0000 7ffa8e607640 1 -- start start 2024-04-25T08:38:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa8e607640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffa880b5710 0x7ffa88161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa8e607640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8815ddf0 0x7ffa880b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa8e607640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ffa880b5af0 0x7ffa8806bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa8e607640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa880b9930 con 0x7ffa880b5710 2024-04-25T08:38:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa8e607640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa88161e00 con 0x7ffa880b5af0 2024-04-25T08:38:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa8e607640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa881620e0 con 0x7ffa8815ddf0 2024-04-25T08:38:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa87fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffa880b5710 0x7ffa88161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa87fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffa880b5710 0x7ffa88161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54094/0 (socket says 172.21.15.38:54094) 2024-04-25T08:38:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa87fff640 1 -- 172.21.15.38:0/971055072 learned_addr learned my addr 172.21.15.38:0/971055072 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa87fff640 1 -- 172.21.15.38:0/971055072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8815ddf0 msgr2=0x7ffa880b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa877fe640 1 --2- 172.21.15.38:0/971055072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8815ddf0 0x7ffa880b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa8cba4640 1 --2- 172.21.15.38:0/971055072 >> v2:172.21.15.107:3300/0 conn(0x7ffa880b5af0 0x7ffa8806bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa87fff640 1 --2- 172.21.15.38:0/971055072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8815ddf0 0x7ffa880b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa87fff640 1 -- 172.21.15.38:0/971055072 >> v2:172.21.15.107:3300/0 conn(0x7ffa880b5af0 msgr2=0x7ffa8806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa87fff640 1 --2- 172.21.15.38:0/971055072 >> v2:172.21.15.107:3300/0 conn(0x7ffa880b5af0 0x7ffa8806bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa87fff640 1 -- 172.21.15.38:0/971055072 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa881623c0 con 0x7ffa880b5710 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa877fe640 1 --2- 172.21.15.38:0/971055072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8815ddf0 0x7ffa880b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.896+0000 7ffa8cba4640 1 --2- 172.21.15.38:0/971055072 >> v2:172.21.15.107:3300/0 conn(0x7ffa880b5af0 0x7ffa8806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.897+0000 7ffa87fff640 1 --2- 172.21.15.38:0/971055072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa880b5710 0x7ffa88161700 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ffa880d1bd0 tx=0x7ffa7c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475213601da50981 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.897+0000 7ffa86ffd640 1 -- 172.21.15.38:0/971055072 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c037070 con 0x7ffa880b5710 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.897+0000 7ffa86ffd640 1 -- 172.21.15.38:0/971055072 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7c032440 con 0x7ffa880b5710 2024-04-25T08:38:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.897+0000 7ffa86ffd640 1 -- 172.21.15.38:0/971055072 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c032810 con 0x7ffa880b5710 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.898+0000 7ffa8e607640 1 -- 172.21.15.38:0/971055072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa880b5710 msgr2=0x7ffa88161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.898+0000 7ffa8e607640 1 --2- 172.21.15.38:0/971055072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa880b5710 0x7ffa88161700 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ffa880d1bd0 tx=0x7ffa7c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.898+0000 7ffa8e607640 1 -- 172.21.15.38:0/971055072 shutdown_connections 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.898+0000 7ffa8e607640 1 --2- 172.21.15.38:0/971055072 >> v2:172.21.15.107:3300/0 conn(0x7ffa880b5af0 0x7ffa8806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.898+0000 7ffa8e607640 1 --2- 172.21.15.38:0/971055072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa8815ddf0 0x7ffa880b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.898+0000 7ffa8e607640 1 --2- 172.21.15.38:0/971055072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa880b5710 0x7ffa88161700 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.898+0000 7ffa8e607640 1 -- 172.21.15.38:0/971055072 >> 172.21.15.38:0/971055072 conn(0x7ffa880d07c0 msgr2=0x7ffa880d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.898+0000 7ffa8e607640 1 -- 172.21.15.38:0/971055072 shutdown_connections 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.898+0000 7ffa8e607640 1 -- 172.21.15.38:0/971055072 wait complete. 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.899+0000 7ffa8e607640 1 Processor -- start 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.899+0000 7ffa8e607640 1 -- start start 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa8e607640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa880b5710 0x7ffa880cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa8e607640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa880b5af0 0x7ffa88160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa8e607640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa8815ddf0 0x7ffa88160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa8e607640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8806c6b0 con 0x7ffa880b5af0 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa8e607640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8806c530 con 0x7ffa8815ddf0 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa8e607640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa8806c830 con 0x7ffa880b5710 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa880b5af0 0x7ffa88160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa877fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa880b5af0 0x7ffa88160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54108/0 (socket says 172.21.15.38:54108) 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa877fe640 1 -- 172.21.15.38:0/2129504849 learned_addr learned my addr 172.21.15.38:0/2129504849 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa87fff640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa880b5710 0x7ffa880cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa8cba4640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa8815ddf0 0x7ffa88160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa877fe640 1 -- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa880b5710 msgr2=0x7ffa880cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa877fe640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa880b5710 0x7ffa880cc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa877fe640 1 -- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa8815ddf0 msgr2=0x7ffa88160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa877fe640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa8815ddf0 0x7ffa88160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:29.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.900+0000 7ffa877fe640 1 -- 172.21.15.38:0/2129504849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8815a5b0 con 0x7ffa880b5af0 2024-04-25T08:38:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.901+0000 7ffa877fe640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa880b5af0 0x7ffa88160210 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800c7c0 tx=0x7ffa7800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.901+0000 7ffa84ff9640 1 -- 172.21.15.38:0/2129504849 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78015070 con 0x7ffa880b5af0 2024-04-25T08:38:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.901+0000 7ffa84ff9640 1 -- 172.21.15.38:0/2129504849 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa78007820 con 0x7ffa880b5af0 2024-04-25T08:38:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.901+0000 7ffa84ff9640 1 -- 172.21.15.38:0/2129504849 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa78007bf0 con 0x7ffa880b5af0 2024-04-25T08:38:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.901+0000 7ffa8e607640 1 -- 172.21.15.38:0/2129504849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa8815a890 con 0x7ffa880b5af0 2024-04-25T08:38:29.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.901+0000 7ffa8e607640 1 -- 172.21.15.38:0/2129504849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa8815ae50 con 0x7ffa880b5af0 2024-04-25T08:38:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.902+0000 7ffa8e607640 1 -- 172.21.15.38:0/2129504849 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa880b3090 con 0x7ffa880b5af0 2024-04-25T08:38:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.902+0000 7ffa84ff9640 1 -- 172.21.15.38:0/2129504849 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffa78018070 con 0x7ffa880b5af0 2024-04-25T08:38:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.903+0000 7ffa84ff9640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffa7406b4d0 0x7ffa7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.903+0000 7ffa84ff9640 1 -- 172.21.15.38:0/2129504849 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffa7808a200 con 0x7ffa880b5af0 2024-04-25T08:38:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.903+0000 7ffa87fff640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffa7406b4d0 0x7ffa7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:29.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.903+0000 7ffa87fff640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffa7406b4d0 0x7ffa7406d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c034000 tx=0x7ffa7c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:29.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:29.907+0000 7ffa84ff9640 1 -- 172.21.15.38:0/2129504849 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa78058f80 con 0x7ffa880b5af0 2024-04-25T08:38:30.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.086+0000 7ffa8e607640 1 -- 172.21.15.38:0/2129504849 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffa88068e80 con 0x7ffa7406b4d0 2024-04-25T08:38:30.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.092+0000 7ffa84ff9640 1 -- 172.21.15.38:0/2129504849 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251992 (secure 0 0 0) 0x7ffa88068e80 con 0x7ffa7406b4d0 2024-04-25T08:38:30.088 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:30.089 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:38:30.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.095+0000 7ffa8e607640 1 -- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffa7406b4d0 msgr2=0x7ffa7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.095+0000 7ffa8e607640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffa7406b4d0 0x7ffa7406d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c034000 tx=0x7ffa7c02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:30.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.095+0000 7ffa8e607640 1 -- 172.21.15.38:0/2129504849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa880b5af0 msgr2=0x7ffa88160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.095+0000 7ffa8e607640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa880b5af0 0x7ffa88160210 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ffa7800c7c0 tx=0x7ffa7800cc80 comp rx=0 tx=0).stop 2024-04-25T08:38:30.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.095+0000 7ffa8e607640 1 -- 172.21.15.38:0/2129504849 shutdown_connections 2024-04-25T08:38:30.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.095+0000 7ffa8e607640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa8815ddf0 0x7ffa88160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.095+0000 7ffa8e607640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa880b5af0 0x7ffa88160210 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.095+0000 7ffa8e607640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffa7406b4d0 0x7ffa7406d990 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.095+0000 7ffa8e607640 1 --2- 172.21.15.38:0/2129504849 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa880b5710 0x7ffa880cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.095+0000 7ffa8e607640 1 -- 172.21.15.38:0/2129504849 >> 172.21.15.38:0/2129504849 conn(0x7ffa880d07c0 msgr2=0x7ffa880b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:30.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.095+0000 7ffa8e607640 1 -- 172.21.15.38:0/2129504849 shutdown_connections 2024-04-25T08:38:30.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.095+0000 7ffa8e607640 1 -- 172.21.15.38:0/2129504849 wait complete. 2024-04-25T08:38:30.099 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-04-25T08:38:29.301993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":456888,"kb_used_data":10312,"kb_used_omap":12,"kb_used_meta":446451,"kb_avail":754517832,"statfs":{"total":773094113280,"available":772626259968,"internally_reserved":0,"allocated":10559488,"data_stored":5509179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":457166384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.579221"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.752588+0000","last_change":"2024-04-25T08:38:26.746055+0000","last_active":"2024-04-25T08:38:26.752588+0000","last_peered":"2024-04-25T08:38:26.752588+0000","last_clean":"2024-04-25T08:38:26.752588+0000","last_became_active":"2024-04-25T08:38:26.745737+0000","last_became_peered":"2024-04-25T08:38:26.745737+0000","last_unstale":"2024-04-25T08:38:26.752588+0000","last_undegraded":"2024-04-25T08:38:26.752588+0000","last_fullsized":"2024-04-25T08:38:26.752588+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:31.720896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.747207+0000","last_change":"2024-04-25T08:38:26.745987+0000","last_active":"2024-04-25T08:38:26.747207+0000","last_peered":"2024-04-25T08:38:26.747207+0000","last_clean":"2024-04-25T08:38:26.747207+0000","last_became_active":"2024-04-25T08:38:26.745685+0000","last_became_peered":"2024-04-25T08:38:26.745685+0000","last_unstale":"2024-04-25T08:38:26.747207+0000","last_undegraded":"2024-04-25T08:38:26.747207+0000","last_fullsized":"2024-04-25T08:38:26.747207+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:10.111406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.752412+0000","last_change":"2024-04-25T08:38:26.747498+0000","last_active":"2024-04-25T08:38:26.752412+0000","last_peered":"2024-04-25T08:38:26.752412+0000","last_clean":"2024-04-25T08:38:26.752412+0000","last_became_active":"2024-04-25T08:38:26.747216+0000","last_became_peered":"2024-04-25T08:38:26.747216+0000","last_unstale":"2024-04-25T08:38:26.752412+0000","last_undegraded":"2024-04-25T08:38:26.752412+0000","last_fullsized":"2024-04-25T08:38:26.752412+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:30.874392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759707+0000","last_change":"2024-04-25T08:38:26.753460+0000","last_active":"2024-04-25T08:38:26.759707+0000","last_peered":"2024-04-25T08:38:26.759707+0000","last_clean":"2024-04-25T08:38:26.759707+0000","last_became_active":"2024-04-25T08:38:26.753198+0000","last_became_peered":"2024-04-25T08:38:26.753198+0000","last_unstale":"2024-04-25T08:38:26.759707+0000","last_undegraded":"2024-04-25T08:38:26.759707+0000","last_fullsized":"2024-04-25T08:38:26.759707+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:50.567885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.733588+0000","last_change":"2024-04-25T08:38:26.732586+0000","last_active":"2024-04-25T08:38:26.733588+0000","last_peered":"2024-04-25T08:38:26.733588+0000","last_clean":"2024-04-25T08:38:26.733588+0000","last_became_active":"2024-04-25T08:38:26.732256+0000","last_became_peered":"2024-04-25T08:38:26.732256+0000","last_unstale":"2024-04-25T08:38:26.733588+0000","last_undegraded":"2024-04-25T08:38:26.733588+0000","last_fullsized":"2024-04-25T08:38:26.733588+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:47.208427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.764134+0000","last_change":"2024-04-25T08:38:26.763440+0000","last_active":"2024-04-25T08:38:26.764134+0000","last_peered":"2024-04-25T08:38:26.764134+0000","last_clean":"2024-04-25T08:38:26.764134+0000","last_became_active":"2024-04-25T08:38:26.763148+0000","last_became_peered":"2024-04-25T08:38:26.763148+0000","last_unstale":"2024-04-25T08:38:26.764134+0000","last_undegraded":"2024-04-25T08:38:26.764134+0000","last_fullsized":"2024-04-25T08:38:26.764134+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:31.732590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.741593+0000","last_change":"2024-04-25T08:38:26.740732+0000","last_active":"2024-04-25T08:38:26.741593+0000","last_peered":"2024-04-25T08:38:26.741593+0000","last_clean":"2024-04-25T08:38:26.741593+0000","last_became_active":"2024-04-25T08:38:26.740460+0000","last_became_peered":"2024-04-25T08:38:26.740460+0000","last_unstale":"2024-04-25T08:38:26.741593+0000","last_undegraded":"2024-04-25T08:38:26.741593+0000","last_fullsized":"2024-04-25T08:38:26.741593+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:41.567708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.762505+0000","last_change":"2024-04-25T08:38:26.752809+0000","last_active":"2024-04-25T08:38:26.762505+0000","last_peered":"2024-04-25T08:38:26.762505+0000","last_clean":"2024-04-25T08:38:26.762505+0000","last_became_active":"2024-04-25T08:38:26.744924+0000","last_became_peered":"2024-04-25T08:38:26.744924+0000","last_unstale":"2024-04-25T08:38:26.762505+0000","last_undegraded":"2024-04-25T08:38:26.762505+0000","last_fullsized":"2024-04-25T08:38:26.762505+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:02.220030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.746960+0000","last_change":"2024-04-25T08:38:26.743745+0000","last_active":"2024-04-25T08:38:26.746960+0000","last_peered":"2024-04-25T08:38:26.746960+0000","last_clean":"2024-04-25T08:38:26.746960+0000","last_became_active":"2024-04-25T08:38:26.743392+0000","last_became_peered":"2024-04-25T08:38:26.743392+0000","last_unstale":"2024-04-25T08:38:26.746960+0000","last_undegraded":"2024-04-25T08:38:26.746960+0000","last_fullsized":"2024-04-25T08:38:26.746960+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:58.463803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.762541+0000","last_change":"2024-04-25T08:38:26.761760+0000","last_active":"2024-04-25T08:38:26.762541+0000","last_peered":"2024-04-25T08:38:26.762541+0000","last_clean":"2024-04-25T08:38:26.762541+0000","last_became_active":"2024-04-25T08:38:26.761461+0000","last_became_peered":"2024-04-25T08:38:26.761461+0000","last_unstale":"2024-04-25T08:38:26.762541+0000","last_undegraded":"2024-04-25T08:38:26.762541+0000","last_fullsized":"2024-04-25T08:38:26.762541+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:22.498280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.755860+0000","last_change":"2024-04-25T08:38:26.754942+0000","last_active":"2024-04-25T08:38:26.755860+0000","last_peered":"2024-04-25T08:38:26.755860+0000","last_clean":"2024-04-25T08:38:26.755860+0000","last_became_active":"2024-04-25T08:38:26.754585+0000","last_became_peered":"2024-04-25T08:38:26.754585+0000","last_unstale":"2024-04-25T08:38:26.755860+0000","last_undegraded":"2024-04-25T08:38:26.755860+0000","last_fullsized":"2024-04-25T08:38:26.755860+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:27.775288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.745527+0000","last_change":"2024-04-25T08:38:26.743668+0000","last_active":"2024-04-25T08:38:26.745527+0000","last_peered":"2024-04-25T08:38:26.745527+0000","last_clean":"2024-04-25T08:38:26.745527+0000","last_became_active":"2024-04-25T08:38:26.743405+0000","last_became_peered":"2024-04-25T08:38:26.743405+0000","last_unstale":"2024-04-25T08:38:26.745527+0000","last_undegraded":"2024-04-25T08:38:26.745527+0000","last_fullsized":"2024-04-25T08:38:26.745527+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:21.456081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759808+0000","last_change":"2024-04-25T08:38:26.744820+0000","last_active":"2024-04-25T08:38:26.759808+0000","last_peered":"2024-04-25T08:38:26.759808+0000","last_clean":"2024-04-25T08:38:26.759808+0000","last_became_active":"2024-04-25T08:38:26.743666+0000","last_became_peered":"2024-04-25T08:38:26.743666+0000","last_unstale":"2024-04-25T08:38:26.759808+0000","last_undegraded":"2024-04-25T08:38:26.759808+0000","last_fullsized":"2024-04-25T08:38:26.759808+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:51.015551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.763134+0000","last_change":"2024-04-25T08:38:26.762672+0000","last_active":"2024-04-25T08:38:26.763134+0000","last_peered":"2024-04-25T08:38:26.763134+0000","last_clean":"2024-04-25T08:38:26.763134+0000","last_became_active":"2024-04-25T08:38:26.762385+0000","last_became_peered":"2024-04-25T08:38:26.762385+0000","last_unstale":"2024-04-25T08:38:26.763134+0000","last_undegraded":"2024-04-25T08:38:26.763134+0000","last_fullsized":"2024-04-25T08:38:26.763134+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:40.858231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.765383+0000","last_change":"2024-04-25T08:38:26.760115+0000","last_active":"2024-04-25T08:38:26.765383+0000","last_peered":"2024-04-25T08:38:26.765383+0000","last_clean":"2024-04-25T08:38:26.765383+0000","last_became_active":"2024-04-25T08:38:26.759806+0000","last_became_peered":"2024-04-25T08:38:26.759806+0000","last_unstale":"2024-04-25T08:38:26.765383+0000","last_undegraded":"2024-04-25T08:38:26.765383+0000","last_fullsized":"2024-04-25T08:38:26.765383+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:37.915525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759191+0000","last_change":"2024-04-25T08:38:26.756715+0000","last_active":"2024-04-25T08:38:26.759191+0000","last_peered":"2024-04-25T08:38:26.759191+0000","last_clean":"2024-04-25T08:38:26.759191+0000","last_became_active":"2024-04-25T08:38:26.756254+0000","last_became_peered":"2024-04-25T08:38:26.756254+0000","last_unstale":"2024-04-25T08:38:26.759191+0000","last_undegraded":"2024-04-25T08:38:26.759191+0000","last_fullsized":"2024-04-25T08:38:26.759191+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:52:07.326931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.749410+0000","last_change":"2024-04-25T08:38:26.745910+0000","last_active":"2024-04-25T08:38:26.749410+0000","last_peered":"2024-04-25T08:38:26.749410+0000","last_clean":"2024-04-25T08:38:26.749410+0000","last_became_active":"2024-04-25T08:38:26.745654+0000","last_became_peered":"2024-04-25T08:38:26.745654+0000","last_unstale":"2024-04-25T08:38:26.749410+0000","last_undegraded":"2024-04-25T08:38:26.749410+0000","last_fullsized":"2024-04-25T08:38:26.749410+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:58.864522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.744497+0000","last_change":"2024-04-25T08:38:26.742550+0000","last_active":"2024-04-25T08:38:26.744497+0000","last_peered":"2024-04-25T08:38:26.744497+0000","last_clean":"2024-04-25T08:38:26.744497+0000","last_became_active":"2024-04-25T08:38:26.742236+0000","last_became_peered":"2024-04-25T08:38:26.742236+0000","last_unstale":"2024-04-25T08:38:26.744497+0000","last_undegraded":"2024-04-25T08:38:26.744497+0000","last_fullsized":"2024-04-25T08:38:26.744497+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:03.303938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.756227+0000","last_change":"2024-04-25T08:38:26.755691+0000","last_active":"2024-04-25T08:38:26.756227+0000","last_peered":"2024-04-25T08:38:26.756227+0000","last_clean":"2024-04-25T08:38:26.756227+0000","last_became_active":"2024-04-25T08:38:26.755373+0000","last_became_peered":"2024-04-25T08:38:26.755373+0000","last_unstale":"2024-04-25T08:38:26.756227+0000","last_undegraded":"2024-04-25T08:38:26.756227+0000","last_fullsized":"2024-04-25T08:38:26.756227+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:01.555678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.765337+0000","last_change":"2024-04-25T08:38:26.760098+0000","last_active":"2024-04-25T08:38:26.765337+0000","last_peered":"2024-04-25T08:38:26.765337+0000","last_clean":"2024-04-25T08:38:26.765337+0000","last_became_active":"2024-04-25T08:38:26.757944+0000","last_became_peered":"2024-04-25T08:38:26.757944+0000","last_unstale":"2024-04-25T08:38:26.765337+0000","last_undegraded":"2024-04-25T08:38:26.765337+0000","last_fullsized":"2024-04-25T08:38:26.765337+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:40.858231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.741734+0000","last_change":"2024-04-25T08:38:26.737429+0000","last_active":"2024-04-25T08:38:26.741734+0000","last_peered":"2024-04-25T08:38:26.741734+0000","last_clean":"2024-04-25T08:38:26.741734+0000","last_became_active":"2024-04-25T08:38:26.737020+0000","last_became_peered":"2024-04-25T08:38:26.737020+0000","last_unstale":"2024-04-25T08:38:26.741734+0000","last_undegraded":"2024-04-25T08:38:26.741734+0000","last_fullsized":"2024-04-25T08:38:26.741734+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:02.220030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.756252+0000","last_change":"2024-04-25T08:38:26.755720+0000","last_active":"2024-04-25T08:38:26.756252+0000","last_peered":"2024-04-25T08:38:26.756252+0000","last_clean":"2024-04-25T08:38:26.756252+0000","last_became_active":"2024-04-25T08:38:26.755444+0000","last_became_peered":"2024-04-25T08:38:26.755444+0000","last_unstale":"2024-04-25T08:38:26.756252+0000","last_undegraded":"2024-04-25T08:38:26.756252+0000","last_fullsized":"2024-04-25T08:38:26.756252+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:16.178662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.760850+0000","last_change":"2024-04-25T08:38:26.760387+0000","last_active":"2024-04-25T08:38:26.760850+0000","last_peered":"2024-04-25T08:38:26.760850+0000","last_clean":"2024-04-25T08:38:26.760850+0000","last_became_active":"2024-04-25T08:38:26.760081+0000","last_became_peered":"2024-04-25T08:38:26.760081+0000","last_unstale":"2024-04-25T08:38:26.760850+0000","last_undegraded":"2024-04-25T08:38:26.760850+0000","last_fullsized":"2024-04-25T08:38:26.760850+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:47.774811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.751065+0000","last_change":"2024-04-25T08:38:26.743924+0000","last_active":"2024-04-25T08:38:26.751065+0000","last_peered":"2024-04-25T08:38:26.751065+0000","last_clean":"2024-04-25T08:38:26.751065+0000","last_became_active":"2024-04-25T08:38:26.743591+0000","last_became_peered":"2024-04-25T08:38:26.743591+0000","last_unstale":"2024-04-25T08:38:26.751065+0000","last_undegraded":"2024-04-25T08:38:26.751065+0000","last_fullsized":"2024-04-25T08:38:26.751065+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:22.141165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.741615+0000","last_change":"2024-04-25T08:38:26.740310+0000","last_active":"2024-04-25T08:38:26.741615+0000","last_peered":"2024-04-25T08:38:26.741615+0000","last_clean":"2024-04-25T08:38:26.741615+0000","last_became_active":"2024-04-25T08:38:26.740043+0000","last_became_peered":"2024-04-25T08:38:26.740043+0000","last_unstale":"2024-04-25T08:38:26.741615+0000","last_undegraded":"2024-04-25T08:38:26.741615+0000","last_fullsized":"2024-04-25T08:38:26.741615+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:37.044827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.763318+0000","last_change":"2024-04-25T08:38:26.762866+0000","last_active":"2024-04-25T08:38:26.763318+0000","last_peered":"2024-04-25T08:38:26.763318+0000","last_clean":"2024-04-25T08:38:26.763318+0000","last_became_active":"2024-04-25T08:38:26.762344+0000","last_became_peered":"2024-04-25T08:38:26.762344+0000","last_unstale":"2024-04-25T08:38:26.763318+0000","last_undegraded":"2024-04-25T08:38:26.763318+0000","last_fullsized":"2024-04-25T08:38:26.763318+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:37.915525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.736789+0000","last_change":"2024-04-25T08:38:26.735107+0000","last_active":"2024-04-25T08:38:26.736789+0000","last_peered":"2024-04-25T08:38:26.736789+0000","last_clean":"2024-04-25T08:38:26.736789+0000","last_became_active":"2024-04-25T08:38:26.734798+0000","last_became_peered":"2024-04-25T08:38:26.734798+0000","last_unstale":"2024-04-25T08:38:26.736789+0000","last_undegraded":"2024-04-25T08:38:26.736789+0000","last_fullsized":"2024-04-25T08:38:26.736789+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:44.760159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.765223+0000","last_change":"2024-04-25T08:38:26.757441+0000","last_active":"2024-04-25T08:38:26.765223+0000","last_peered":"2024-04-25T08:38:26.765223+0000","last_clean":"2024-04-25T08:38:26.765223+0000","last_became_active":"2024-04-25T08:38:26.756675+0000","last_became_peered":"2024-04-25T08:38:26.756675+0000","last_unstale":"2024-04-25T08:38:26.765223+0000","last_undegraded":"2024-04-25T08:38:26.765223+0000","last_fullsized":"2024-04-25T08:38:26.765223+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:16.178662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.739525+0000","last_change":"2024-04-25T08:38:26.738797+0000","last_active":"2024-04-25T08:38:26.739525+0000","last_peered":"2024-04-25T08:38:26.739525+0000","last_clean":"2024-04-25T08:38:26.739525+0000","last_became_active":"2024-04-25T08:38:26.738475+0000","last_became_peered":"2024-04-25T08:38:26.738475+0000","last_unstale":"2024-04-25T08:38:26.739525+0000","last_undegraded":"2024-04-25T08:38:26.739525+0000","last_fullsized":"2024-04-25T08:38:26.739525+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:10.727609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.743834+0000","last_change":"2024-04-25T08:38:26.742359+0000","last_active":"2024-04-25T08:38:26.743834+0000","last_peered":"2024-04-25T08:38:26.743834+0000","last_clean":"2024-04-25T08:38:26.743834+0000","last_became_active":"2024-04-25T08:38:26.742078+0000","last_became_peered":"2024-04-25T08:38:26.742078+0000","last_unstale":"2024-04-25T08:38:26.743834+0000","last_undegraded":"2024-04-25T08:38:26.743834+0000","last_fullsized":"2024-04-25T08:38:26.743834+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:50.567885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.763313+0000","last_change":"2024-04-25T08:38:26.762604+0000","last_active":"2024-04-25T08:38:26.763313+0000","last_peered":"2024-04-25T08:38:26.763313+0000","last_clean":"2024-04-25T08:38:26.763313+0000","last_became_active":"2024-04-25T08:38:26.762294+0000","last_became_peered":"2024-04-25T08:38:26.762294+0000","last_unstale":"2024-04-25T08:38:26.763313+0000","last_undegraded":"2024-04-25T08:38:26.763313+0000","last_fullsized":"2024-04-25T08:38:26.763313+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:03.303938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759464+0000","last_change":"2024-04-25T08:38:26.756706+0000","last_active":"2024-04-25T08:38:26.759464+0000","last_peered":"2024-04-25T08:38:26.759464+0000","last_clean":"2024-04-25T08:38:26.759464+0000","last_became_active":"2024-04-25T08:38:26.756206+0000","last_became_peered":"2024-04-25T08:38:26.756206+0000","last_unstale":"2024-04-25T08:38:26.759464+0000","last_undegraded":"2024-04-25T08:38:26.759464+0000","last_fullsized":"2024-04-25T08:38:26.759464+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:25.728911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759101+0000","last_change":"2024-04-25T08:38:26.749527+0000","last_active":"2024-04-25T08:38:26.759101+0000","last_peered":"2024-04-25T08:38:26.759101+0000","last_clean":"2024-04-25T08:38:26.759101+0000","last_became_active":"2024-04-25T08:38:26.749119+0000","last_became_peered":"2024-04-25T08:38:26.749119+0000","last_unstale":"2024-04-25T08:38:26.759101+0000","last_undegraded":"2024-04-25T08:38:26.759101+0000","last_fullsized":"2024-04-25T08:38:26.759101+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:36.034531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"95'4","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.758774+0000","last_change":"2024-04-25T08:38:25.741437+0000","last_active":"2024-04-25T08:38:26.758774+0000","last_peered":"2024-04-25T08:38:26.758774+0000","last_clean":"2024-04-25T08:38:26.758774+0000","last_became_active":"2024-04-25T08:38:25.741162+0000","last_became_peered":"2024-04-25T08:38:25.741162+0000","last_unstale":"2024-04-25T08:38:26.758774+0000","last_undegraded":"2024-04-25T08:38:26.758774+0000","last_fullsized":"2024-04-25T08:38:26.758774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:26.772853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759498+0000","last_change":"2024-04-25T08:38:26.758679+0000","last_active":"2024-04-25T08:38:26.759498+0000","last_peered":"2024-04-25T08:38:26.759498+0000","last_clean":"2024-04-25T08:38:26.759498+0000","last_became_active":"2024-04-25T08:38:26.758412+0000","last_became_peered":"2024-04-25T08:38:26.758412+0000","last_unstale":"2024-04-25T08:38:26.759498+0000","last_undegraded":"2024-04-25T08:38:26.759498+0000","last_fullsized":"2024-04-25T08:38:26.759498+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:31.654530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.749821+0000","last_change":"2024-04-25T08:38:25.740510+0000","last_active":"2024-04-25T08:38:26.749821+0000","last_peered":"2024-04-25T08:38:26.749821+0000","last_clean":"2024-04-25T08:38:26.749821+0000","last_became_active":"2024-04-25T08:38:25.739259+0000","last_became_peered":"2024-04-25T08:38:25.739259+0000","last_unstale":"2024-04-25T08:38:26.749821+0000","last_undegraded":"2024-04-25T08:38:26.749821+0000","last_fullsized":"2024-04-25T08:38:26.749821+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:52.285256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.761344+0000","last_change":"2024-04-25T08:38:26.760958+0000","last_active":"2024-04-25T08:38:26.761344+0000","last_peered":"2024-04-25T08:38:26.761344+0000","last_clean":"2024-04-25T08:38:26.761344+0000","last_became_active":"2024-04-25T08:38:26.760636+0000","last_became_peered":"2024-04-25T08:38:26.760636+0000","last_unstale":"2024-04-25T08:38:26.761344+0000","last_undegraded":"2024-04-25T08:38:26.761344+0000","last_fullsized":"2024-04-25T08:38:26.761344+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:49.707641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.752584+0000","last_change":"2024-04-25T08:38:25.744831+0000","last_active":"2024-04-25T08:38:26.752584+0000","last_peered":"2024-04-25T08:38:26.752584+0000","last_clean":"2024-04-25T08:38:26.752584+0000","last_became_active":"2024-04-25T08:38:25.744621+0000","last_became_peered":"2024-04-25T08:38:25.744621+0000","last_unstale":"2024-04-25T08:38:26.752584+0000","last_undegraded":"2024-04-25T08:38:26.752584+0000","last_fullsized":"2024-04-25T08:38:26.752584+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:44.426879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.757851+0000","last_change":"2024-04-25T08:38:26.751681+0000","last_active":"2024-04-25T08:38:26.757851+0000","last_peered":"2024-04-25T08:38:26.757851+0000","last_clean":"2024-04-25T08:38:26.757851+0000","last_became_active":"2024-04-25T08:38:26.749721+0000","last_became_peered":"2024-04-25T08:38:26.749721+0000","last_unstale":"2024-04-25T08:38:26.757851+0000","last_undegraded":"2024-04-25T08:38:26.757851+0000","last_fullsized":"2024-04-25T08:38:26.757851+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:17.771155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.760077+0000","last_change":"2024-04-25T08:38:25.737213+0000","last_active":"2024-04-25T08:38:26.760077+0000","last_peered":"2024-04-25T08:38:26.760077+0000","last_clean":"2024-04-25T08:38:26.760077+0000","last_became_active":"2024-04-25T08:38:25.736448+0000","last_became_peered":"2024-04-25T08:38:25.736448+0000","last_unstale":"2024-04-25T08:38:26.760077+0000","last_undegraded":"2024-04-25T08:38:26.760077+0000","last_fullsized":"2024-04-25T08:38:26.760077+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:00.553243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.768100+0000","last_change":"2024-04-25T08:38:26.762500+0000","last_active":"2024-04-25T08:38:26.768100+0000","last_peered":"2024-04-25T08:38:26.768100+0000","last_clean":"2024-04-25T08:38:26.768100+0000","last_became_active":"2024-04-25T08:38:26.759785+0000","last_became_peered":"2024-04-25T08:38:26.759785+0000","last_unstale":"2024-04-25T08:38:26.768100+0000","last_undegraded":"2024-04-25T08:38:26.768100+0000","last_fullsized":"2024-04-25T08:38:26.768100+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:49.659530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.749529+0000","last_change":"2024-04-25T08:38:25.747659+0000","last_active":"2024-04-25T08:38:26.749529+0000","last_peered":"2024-04-25T08:38:26.749529+0000","last_clean":"2024-04-25T08:38:26.749529+0000","last_became_active":"2024-04-25T08:38:25.746496+0000","last_became_peered":"2024-04-25T08:38:25.746496+0000","last_unstale":"2024-04-25T08:38:26.749529+0000","last_undegraded":"2024-04-25T08:38:26.749529+0000","last_fullsized":"2024-04-25T08:38:26.749529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:56.912745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.753063+0000","last_change":"2024-04-25T08:38:26.752692+0000","last_active":"2024-04-25T08:38:26.753063+0000","last_peered":"2024-04-25T08:38:26.753063+0000","last_clean":"2024-04-25T08:38:26.753063+0000","last_became_active":"2024-04-25T08:38:26.752399+0000","last_became_peered":"2024-04-25T08:38:26.752399+0000","last_unstale":"2024-04-25T08:38:26.753063+0000","last_undegraded":"2024-04-25T08:38:26.753063+0000","last_fullsized":"2024-04-25T08:38:26.753063+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:49.659530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.766617+0000","last_change":"2024-04-25T08:38:25.747074+0000","last_active":"2024-04-25T08:38:26.766617+0000","last_peered":"2024-04-25T08:38:26.766617+0000","last_clean":"2024-04-25T08:38:26.766617+0000","last_became_active":"2024-04-25T08:38:25.746763+0000","last_became_peered":"2024-04-25T08:38:25.746763+0000","last_unstale":"2024-04-25T08:38:26.766617+0000","last_undegraded":"2024-04-25T08:38:26.766617+0000","last_fullsized":"2024-04-25T08:38:26.766617+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:24.815005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.764120+0000","last_change":"2024-04-25T08:38:26.763175+0000","last_active":"2024-04-25T08:38:26.764120+0000","last_peered":"2024-04-25T08:38:26.764120+0000","last_clean":"2024-04-25T08:38:26.764120+0000","last_became_active":"2024-04-25T08:38:26.762759+0000","last_became_peered":"2024-04-25T08:38:26.762759+0000","last_unstale":"2024-04-25T08:38:26.764120+0000","last_undegraded":"2024-04-25T08:38:26.764120+0000","last_fullsized":"2024-04-25T08:38:26.764120+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:03.750859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.735272+0000","last_change":"2024-04-25T08:38:25.740909+0000","last_active":"2024-04-25T08:38:26.735272+0000","last_peered":"2024-04-25T08:38:26.735272+0000","last_clean":"2024-04-25T08:38:26.735272+0000","last_became_active":"2024-04-25T08:38:25.740576+0000","last_became_peered":"2024-04-25T08:38:25.740576+0000","last_unstale":"2024-04-25T08:38:26.735272+0000","last_undegraded":"2024-04-25T08:38:26.735272+0000","last_fullsized":"2024-04-25T08:38:26.735272+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:44.679631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.760826+0000","last_change":"2024-04-25T08:38:26.750794+0000","last_active":"2024-04-25T08:38:26.760826+0000","last_peered":"2024-04-25T08:38:26.760826+0000","last_clean":"2024-04-25T08:38:26.760826+0000","last_became_active":"2024-04-25T08:38:26.744556+0000","last_became_peered":"2024-04-25T08:38:26.744556+0000","last_unstale":"2024-04-25T08:38:26.760826+0000","last_undegraded":"2024-04-25T08:38:26.760826+0000","last_fullsized":"2024-04-25T08:38:26.760826+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:27.775288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.735051+0000","last_change":"2024-04-25T08:38:25.742614+0000","last_active":"2024-04-25T08:38:26.735051+0000","last_peered":"2024-04-25T08:38:26.735051+0000","last_clean":"2024-04-25T08:38:26.735051+0000","last_became_active":"2024-04-25T08:38:25.741874+0000","last_became_peered":"2024-04-25T08:38:25.741874+0000","last_unstale":"2024-04-25T08:38:26.735051+0000","last_undegraded":"2024-04-25T08:38:26.735051+0000","last_fullsized":"2024-04-25T08:38:26.735051+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:53.336653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.746434+0000","last_change":"2024-04-25T08:38:26.742447+0000","last_active":"2024-04-25T08:38:26.746434+0000","last_peered":"2024-04-25T08:38:26.746434+0000","last_clean":"2024-04-25T08:38:26.746434+0000","last_became_active":"2024-04-25T08:38:26.742139+0000","last_became_peered":"2024-04-25T08:38:26.742139+0000","last_unstale":"2024-04-25T08:38:26.746434+0000","last_undegraded":"2024-04-25T08:38:26.746434+0000","last_fullsized":"2024-04-25T08:38:26.746434+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:25.817440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.739351+0000","last_change":"2024-04-25T08:38:25.746816+0000","last_active":"2024-04-25T08:38:26.739351+0000","last_peered":"2024-04-25T08:38:26.739351+0000","last_clean":"2024-04-25T08:38:26.739351+0000","last_became_active":"2024-04-25T08:38:25.746493+0000","last_became_peered":"2024-04-25T08:38:25.746493+0000","last_unstale":"2024-04-25T08:38:26.739351+0000","last_undegraded":"2024-04-25T08:38:26.739351+0000","last_fullsized":"2024-04-25T08:38:26.739351+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:11.581685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.744888+0000","last_change":"2024-04-25T08:38:26.744445+0000","last_active":"2024-04-25T08:38:26.744888+0000","last_peered":"2024-04-25T08:38:26.744888+0000","last_clean":"2024-04-25T08:38:26.744888+0000","last_became_active":"2024-04-25T08:38:26.744141+0000","last_became_peered":"2024-04-25T08:38:26.744141+0000","last_unstale":"2024-04-25T08:38:26.744888+0000","last_undegraded":"2024-04-25T08:38:26.744888+0000","last_fullsized":"2024-04-25T08:38:26.744888+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:10.827812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.736004+0000","last_change":"2024-04-25T08:38:25.750043+0000","last_active":"2024-04-25T08:38:26.736004+0000","last_peered":"2024-04-25T08:38:26.736004+0000","last_clean":"2024-04-25T08:38:26.736004+0000","last_became_active":"2024-04-25T08:38:25.749731+0000","last_became_peered":"2024-04-25T08:38:25.749731+0000","last_unstale":"2024-04-25T08:38:26.736004+0000","last_undegraded":"2024-04-25T08:38:26.736004+0000","last_fullsized":"2024-04-25T08:38:26.736004+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:23.020979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.752840+0000","last_change":"2024-04-25T08:38:26.750870+0000","last_active":"2024-04-25T08:38:26.752840+0000","last_peered":"2024-04-25T08:38:26.752840+0000","last_clean":"2024-04-25T08:38:26.752840+0000","last_became_active":"2024-04-25T08:38:26.750572+0000","last_became_peered":"2024-04-25T08:38:26.750572+0000","last_unstale":"2024-04-25T08:38:26.752840+0000","last_undegraded":"2024-04-25T08:38:26.752840+0000","last_fullsized":"2024-04-25T08:38:26.752840+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:54.339088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"95'3","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759294+0000","last_change":"2024-04-25T08:38:25.746533+0000","last_active":"2024-04-25T08:38:26.759294+0000","last_peered":"2024-04-25T08:38:26.759294+0000","last_clean":"2024-04-25T08:38:26.759294+0000","last_became_active":"2024-04-25T08:38:25.745660+0000","last_became_peered":"2024-04-25T08:38:25.745660+0000","last_unstale":"2024-04-25T08:38:26.759294+0000","last_undegraded":"2024-04-25T08:38:26.759294+0000","last_fullsized":"2024-04-25T08:38:26.759294+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:49.066298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.765341+0000","last_change":"2024-04-25T08:38:26.760593+0000","last_active":"2024-04-25T08:38:26.765341+0000","last_peered":"2024-04-25T08:38:26.765341+0000","last_clean":"2024-04-25T08:38:26.765341+0000","last_became_active":"2024-04-25T08:38:26.760307+0000","last_became_peered":"2024-04-25T08:38:26.760307+0000","last_unstale":"2024-04-25T08:38:26.765341+0000","last_undegraded":"2024-04-25T08:38:26.765341+0000","last_fullsized":"2024-04-25T08:38:26.765341+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:59.525148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"95'6","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.764041+0000","last_change":"2024-04-25T08:38:25.741819+0000","last_active":"2024-04-25T08:38:26.764041+0000","last_peered":"2024-04-25T08:38:26.764041+0000","last_clean":"2024-04-25T08:38:26.764041+0000","last_became_active":"2024-04-25T08:38:25.740763+0000","last_became_peered":"2024-04-25T08:38:25.740763+0000","last_unstale":"2024-04-25T08:38:26.764041+0000","last_undegraded":"2024-04-25T08:38:26.764041+0000","last_fullsized":"2024-04-25T08:38:26.764041+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:21.804361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.762480+0000","last_change":"2024-04-25T08:38:26.744634+0000","last_active":"2024-04-25T08:38:26.762480+0000","last_peered":"2024-04-25T08:38:26.762480+0000","last_clean":"2024-04-25T08:38:26.762480+0000","last_became_active":"2024-04-25T08:38:26.744269+0000","last_became_peered":"2024-04-25T08:38:26.744269+0000","last_unstale":"2024-04-25T08:38:26.762480+0000","last_undegraded":"2024-04-25T08:38:26.762480+0000","last_fullsized":"2024-04-25T08:38:26.762480+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:47.208427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.758590+0000","last_change":"2024-04-25T08:38:25.741677+0000","last_active":"2024-04-25T08:38:26.758590+0000","last_peered":"2024-04-25T08:38:26.758590+0000","last_clean":"2024-04-25T08:38:26.758590+0000","last_became_active":"2024-04-25T08:38:25.740828+0000","last_became_peered":"2024-04-25T08:38:25.740828+0000","last_unstale":"2024-04-25T08:38:26.758590+0000","last_undegraded":"2024-04-25T08:38:26.758590+0000","last_fullsized":"2024-04-25T08:38:26.758590+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:39.455847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.732494+0000","last_change":"2024-04-25T08:38:26.732124+0000","last_active":"2024-04-25T08:38:26.732494+0000","last_peered":"2024-04-25T08:38:26.732494+0000","last_clean":"2024-04-25T08:38:26.732494+0000","last_became_active":"2024-04-25T08:38:26.731787+0000","last_became_peered":"2024-04-25T08:38:26.731787+0000","last_unstale":"2024-04-25T08:38:26.732494+0000","last_undegraded":"2024-04-25T08:38:26.732494+0000","last_fullsized":"2024-04-25T08:38:26.732494+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:44.683993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.747785+0000","last_change":"2024-04-25T08:38:25.744898+0000","last_active":"2024-04-25T08:38:26.747785+0000","last_peered":"2024-04-25T08:38:26.747785+0000","last_clean":"2024-04-25T08:38:26.747785+0000","last_became_active":"2024-04-25T08:38:25.744625+0000","last_became_peered":"2024-04-25T08:38:25.744625+0000","last_unstale":"2024-04-25T08:38:26.747785+0000","last_undegraded":"2024-04-25T08:38:26.747785+0000","last_fullsized":"2024-04-25T08:38:26.747785+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:12.974522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.763658+0000","last_change":"2024-04-25T08:38:26.761802+0000","last_active":"2024-04-25T08:38:26.763658+0000","last_peered":"2024-04-25T08:38:26.763658+0000","last_clean":"2024-04-25T08:38:26.763658+0000","last_became_active":"2024-04-25T08:38:26.761512+0000","last_became_peered":"2024-04-25T08:38:26.761512+0000","last_unstale":"2024-04-25T08:38:26.763658+0000","last_undegraded":"2024-04-25T08:38:26.763658+0000","last_fullsized":"2024-04-25T08:38:26.763658+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:44.609519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.756189+0000","last_change":"2024-04-25T08:38:25.748596+0000","last_active":"2024-04-25T08:38:26.756189+0000","last_peered":"2024-04-25T08:38:26.756189+0000","last_clean":"2024-04-25T08:38:26.756189+0000","last_became_active":"2024-04-25T08:38:25.748252+0000","last_became_peered":"2024-04-25T08:38:25.748252+0000","last_unstale":"2024-04-25T08:38:26.756189+0000","last_undegraded":"2024-04-25T08:38:26.756189+0000","last_fullsized":"2024-04-25T08:38:26.756189+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:57.461368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.736749+0000","last_change":"2024-04-25T08:38:26.735743+0000","last_active":"2024-04-25T08:38:26.736749+0000","last_peered":"2024-04-25T08:38:26.736749+0000","last_clean":"2024-04-25T08:38:26.736749+0000","last_became_active":"2024-04-25T08:38:26.735473+0000","last_became_peered":"2024-04-25T08:38:26.735473+0000","last_unstale":"2024-04-25T08:38:26.736749+0000","last_undegraded":"2024-04-25T08:38:26.736749+0000","last_fullsized":"2024-04-25T08:38:26.736749+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:00.095138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"95'8","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.768977+0000","last_change":"2024-04-25T08:38:25.747426+0000","last_active":"2024-04-25T08:38:26.768977+0000","last_peered":"2024-04-25T08:38:26.768977+0000","last_clean":"2024-04-25T08:38:26.768977+0000","last_became_active":"2024-04-25T08:38:25.746951+0000","last_became_peered":"2024-04-25T08:38:25.746951+0000","last_unstale":"2024-04-25T08:38:26.768977+0000","last_undegraded":"2024-04-25T08:38:26.768977+0000","last_fullsized":"2024-04-25T08:38:26.768977+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:05.319562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.759126+0000","last_change":"2024-04-25T08:38:26.754438+0000","last_active":"2024-04-25T08:38:26.759126+0000","last_peered":"2024-04-25T08:38:26.759126+0000","last_clean":"2024-04-25T08:38:26.759126+0000","last_became_active":"2024-04-25T08:38:26.754123+0000","last_became_peered":"2024-04-25T08:38:26.754123+0000","last_unstale":"2024-04-25T08:38:26.759126+0000","last_undegraded":"2024-04-25T08:38:26.759126+0000","last_fullsized":"2024-04-25T08:38:26.759126+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:27.422753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.751455+0000","last_change":"2024-04-25T08:38:25.733704+0000","last_active":"2024-04-25T08:38:26.751455+0000","last_peered":"2024-04-25T08:38:26.751455+0000","last_clean":"2024-04-25T08:38:26.751455+0000","last_became_active":"2024-04-25T08:38:25.733298+0000","last_became_peered":"2024-04-25T08:38:25.733298+0000","last_unstale":"2024-04-25T08:38:26.751455+0000","last_undegraded":"2024-04-25T08:38:26.751455+0000","last_fullsized":"2024-04-25T08:38:26.751455+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:15.176227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.740505+0000","last_change":"2024-04-25T08:38:26.739347+0000","last_active":"2024-04-25T08:38:26.740505+0000","last_peered":"2024-04-25T08:38:26.740505+0000","last_clean":"2024-04-25T08:38:26.740505+0000","last_became_active":"2024-04-25T08:38:26.739070+0000","last_became_peered":"2024-04-25T08:38:26.739070+0000","last_unstale":"2024-04-25T08:38:26.740505+0000","last_undegraded":"2024-04-25T08:38:26.740505+0000","last_fullsized":"2024-04-25T08:38:26.740505+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:06.321997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.751836+0000","last_change":"2024-04-25T08:38:25.741392+0000","last_active":"2024-04-25T08:38:26.751836+0000","last_peered":"2024-04-25T08:38:26.751836+0000","last_clean":"2024-04-25T08:38:26.751836+0000","last_became_active":"2024-04-25T08:38:25.741053+0000","last_became_peered":"2024-04-25T08:38:25.741053+0000","last_unstale":"2024-04-25T08:38:26.751836+0000","last_undegraded":"2024-04-25T08:38:26.751836+0000","last_fullsized":"2024-04-25T08:38:26.751836+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:54.223454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.737895+0000","last_change":"2024-04-25T08:38:26.734577+0000","last_active":"2024-04-25T08:38:26.737895+0000","last_peered":"2024-04-25T08:38:26.737895+0000","last_clean":"2024-04-25T08:38:26.737895+0000","last_became_active":"2024-04-25T08:38:26.734101+0000","last_became_peered":"2024-04-25T08:38:26.734101+0000","last_unstale":"2024-04-25T08:38:26.737895+0000","last_undegraded":"2024-04-25T08:38:26.737895+0000","last_fullsized":"2024-04-25T08:38:26.737895+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:51.015551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.762618+0000","last_change":"2024-04-25T08:38:25.748674+0000","last_active":"2024-04-25T08:38:26.762618+0000","last_peered":"2024-04-25T08:38:26.762618+0000","last_clean":"2024-04-25T08:38:26.762618+0000","last_became_active":"2024-04-25T08:38:25.748380+0000","last_became_peered":"2024-04-25T08:38:25.748380+0000","last_unstale":"2024-04-25T08:38:26.762618+0000","last_undegraded":"2024-04-25T08:38:26.762618+0000","last_fullsized":"2024-04-25T08:38:26.762618+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:14.485431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.769490+0000","last_change":"2024-04-25T08:38:26.768902+0000","last_active":"2024-04-25T08:38:26.769490+0000","last_peered":"2024-04-25T08:38:26.769490+0000","last_clean":"2024-04-25T08:38:26.769490+0000","last_became_active":"2024-04-25T08:38:26.768587+0000","last_became_peered":"2024-04-25T08:38:26.768587+0000","last_unstale":"2024-04-25T08:38:26.769490+0000","last_undegraded":"2024-04-25T08:38:26.769490+0000","last_fullsized":"2024-04-25T08:38:26.769490+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:36.811665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.752983+0000","last_change":"2024-04-25T08:38:25.743224+0000","last_active":"2024-04-25T08:38:26.752983+0000","last_peered":"2024-04-25T08:38:26.752983+0000","last_clean":"2024-04-25T08:38:26.752983+0000","last_became_active":"2024-04-25T08:38:25.742773+0000","last_became_peered":"2024-04-25T08:38:25.742773+0000","last_unstale":"2024-04-25T08:38:26.752983+0000","last_undegraded":"2024-04-25T08:38:26.752983+0000","last_fullsized":"2024-04-25T08:38:26.752983+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:46.772376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.747634+0000","last_change":"2024-04-25T08:38:26.746584+0000","last_active":"2024-04-25T08:38:26.747634+0000","last_peered":"2024-04-25T08:38:26.747634+0000","last_clean":"2024-04-25T08:38:26.747634+0000","last_became_active":"2024-04-25T08:38:26.746315+0000","last_became_peered":"2024-04-25T08:38:26.746315+0000","last_unstale":"2024-04-25T08:38:26.747634+0000","last_undegraded":"2024-04-25T08:38:26.747634+0000","last_fullsized":"2024-04-25T08:38:26.747634+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:45.682066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.732344+0000","last_change":"2024-04-25T08:38:25.749965+0000","last_active":"2024-04-25T08:38:26.732344+0000","last_peered":"2024-04-25T08:38:26.732344+0000","last_clean":"2024-04-25T08:38:26.732344+0000","last_became_active":"2024-04-25T08:38:25.749575+0000","last_became_peered":"2024-04-25T08:38:25.749575+0000","last_unstale":"2024-04-25T08:38:26.732344+0000","last_undegraded":"2024-04-25T08:38:26.732344+0000","last_fullsized":"2024-04-25T08:38:26.732344+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:52.536663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.763292+0000","last_change":"2024-04-25T08:38:26.762578+0000","last_active":"2024-04-25T08:38:26.763292+0000","last_peered":"2024-04-25T08:38:26.763292+0000","last_clean":"2024-04-25T08:38:26.763292+0000","last_became_active":"2024-04-25T08:38:26.762304+0000","last_became_peered":"2024-04-25T08:38:26.762304+0000","last_unstale":"2024-04-25T08:38:26.763292+0000","last_undegraded":"2024-04-25T08:38:26.763292+0000","last_fullsized":"2024-04-25T08:38:26.763292+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:24.023414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.737761+0000","last_change":"2024-04-25T08:38:25.746037+0000","last_active":"2024-04-25T08:38:26.737761+0000","last_peered":"2024-04-25T08:38:26.737761+0000","last_clean":"2024-04-25T08:38:26.737761+0000","last_became_active":"2024-04-25T08:38:25.745723+0000","last_became_peered":"2024-04-25T08:38:25.745723+0000","last_unstale":"2024-04-25T08:38:26.737761+0000","last_undegraded":"2024-04-25T08:38:26.737761+0000","last_fullsized":"2024-04-25T08:38:26.737761+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:48.657096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.757690+0000","last_change":"2024-04-25T08:38:26.751764+0000","last_active":"2024-04-25T08:38:26.757690+0000","last_peered":"2024-04-25T08:38:26.757690+0000","last_clean":"2024-04-25T08:38:26.757690+0000","last_became_active":"2024-04-25T08:38:26.749663+0000","last_became_peered":"2024-04-25T08:38:26.749663+0000","last_unstale":"2024-04-25T08:38:26.757690+0000","last_undegraded":"2024-04-25T08:38:26.757690+0000","last_fullsized":"2024-04-25T08:38:26.757690+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:09.673659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.732074+0000","last_change":"2024-04-25T08:38:25.744522+0000","last_active":"2024-04-25T08:38:26.732074+0000","last_peered":"2024-04-25T08:38:26.732074+0000","last_clean":"2024-04-25T08:38:26.732074+0000","last_became_active":"2024-04-25T08:38:25.744250+0000","last_became_peered":"2024-04-25T08:38:25.744250+0000","last_unstale":"2024-04-25T08:38:26.732074+0000","last_undegraded":"2024-04-25T08:38:26.732074+0000","last_fullsized":"2024-04-25T08:38:26.732074+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:20.380067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.736765+0000","last_change":"2024-04-25T08:38:26.736277+0000","last_active":"2024-04-25T08:38:26.736765+0000","last_peered":"2024-04-25T08:38:26.736765+0000","last_clean":"2024-04-25T08:38:26.736765+0000","last_became_active":"2024-04-25T08:38:26.735904+0000","last_became_peered":"2024-04-25T08:38:26.735904+0000","last_unstale":"2024-04-25T08:38:26.736765+0000","last_undegraded":"2024-04-25T08:38:26.736765+0000","last_fullsized":"2024-04-25T08:38:26.736765+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:59.525148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.752317+0000","last_change":"2024-04-25T08:38:25.732510+0000","last_active":"2024-04-25T08:38:26.752317+0000","last_peered":"2024-04-25T08:38:26.752317+0000","last_clean":"2024-04-25T08:38:26.752317+0000","last_became_active":"2024-04-25T08:38:25.732228+0000","last_became_peered":"2024-04-25T08:38:25.732228+0000","last_unstale":"2024-04-25T08:38:26.752317+0000","last_undegraded":"2024-04-25T08:38:26.752317+0000","last_fullsized":"2024-04-25T08:38:26.752317+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:15.714624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.758379+0000","last_change":"2024-04-25T08:38:26.755389+0000","last_active":"2024-04-25T08:38:26.758379+0000","last_peered":"2024-04-25T08:38:26.758379+0000","last_clean":"2024-04-25T08:38:26.758379+0000","last_became_active":"2024-04-25T08:38:26.753204+0000","last_became_peered":"2024-04-25T08:38:26.753204+0000","last_unstale":"2024-04-25T08:38:26.758379+0000","last_undegraded":"2024-04-25T08:38:26.758379+0000","last_fullsized":"2024-04-25T08:38:26.758379+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:54.067829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.730771+0000","last_change":"2024-04-25T08:38:25.741760+0000","last_active":"2024-04-25T08:38:26.730771+0000","last_peered":"2024-04-25T08:38:26.730771+0000","last_clean":"2024-04-25T08:38:26.730771+0000","last_became_active":"2024-04-25T08:38:25.740807+0000","last_became_peered":"2024-04-25T08:38:25.740807+0000","last_unstale":"2024-04-25T08:38:26.730771+0000","last_undegraded":"2024-04-25T08:38:26.730771+0000","last_fullsized":"2024-04-25T08:38:26.730771+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:32.412304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.749172+0000","last_change":"2024-04-25T08:38:26.745102+0000","last_active":"2024-04-25T08:38:26.749172+0000","last_peered":"2024-04-25T08:38:26.749172+0000","last_clean":"2024-04-25T08:38:26.749172+0000","last_became_active":"2024-04-25T08:38:26.744821+0000","last_became_peered":"2024-04-25T08:38:26.744821+0000","last_unstale":"2024-04-25T08:38:26.749172+0000","last_undegraded":"2024-04-25T08:38:26.749172+0000","last_fullsized":"2024-04-25T08:38:26.749172+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:58.766753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.731406+0000","last_change":"2024-04-25T08:38:25.745864+0000","last_active":"2024-04-25T08:38:26.731406+0000","last_peered":"2024-04-25T08:38:26.731406+0000","last_clean":"2024-04-25T08:38:26.731406+0000","last_became_active":"2024-04-25T08:38:25.745524+0000","last_became_peered":"2024-04-25T08:38:25.745524+0000","last_unstale":"2024-04-25T08:38:26.731406+0000","last_undegraded":"2024-04-25T08:38:26.731406+0000","last_fullsized":"2024-04-25T08:38:26.731406+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:15.805008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.747133+0000","last_change":"2024-04-25T08:38:26.745323+0000","last_active":"2024-04-25T08:38:26.747133+0000","last_peered":"2024-04-25T08:38:26.747133+0000","last_clean":"2024-04-25T08:38:26.747133+0000","last_became_active":"2024-04-25T08:38:26.744709+0000","last_became_peered":"2024-04-25T08:38:26.744709+0000","last_unstale":"2024-04-25T08:38:26.747133+0000","last_undegraded":"2024-04-25T08:38:26.747133+0000","last_fullsized":"2024-04-25T08:38:26.747133+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:35.368835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.729169+0000","last_change":"2024-04-25T08:38:25.746710+0000","last_active":"2024-04-25T08:38:26.729169+0000","last_peered":"2024-04-25T08:38:26.729169+0000","last_clean":"2024-04-25T08:38:26.729169+0000","last_became_active":"2024-04-25T08:38:25.746399+0000","last_became_peered":"2024-04-25T08:38:25.746399+0000","last_unstale":"2024-04-25T08:38:26.729169+0000","last_undegraded":"2024-04-25T08:38:26.729169+0000","last_fullsized":"2024-04-25T08:38:26.729169+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:39.855796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.755493+0000","last_change":"2024-04-25T08:38:26.754753+0000","last_active":"2024-04-25T08:38:26.755493+0000","last_peered":"2024-04-25T08:38:26.755493+0000","last_clean":"2024-04-25T08:38:26.755493+0000","last_became_active":"2024-04-25T08:38:26.754457+0000","last_became_peered":"2024-04-25T08:38:26.754457+0000","last_unstale":"2024-04-25T08:38:26.755493+0000","last_undegraded":"2024-04-25T08:38:26.755493+0000","last_fullsized":"2024-04-25T08:38:26.755493+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:07.790364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.748179+0000","last_change":"2024-04-25T08:38:25.745574+0000","last_active":"2024-04-25T08:38:26.748179+0000","last_peered":"2024-04-25T08:38:26.748179+0000","last_clean":"2024-04-25T08:38:26.748179+0000","last_became_active":"2024-04-25T08:38:25.745298+0000","last_became_peered":"2024-04-25T08:38:25.745298+0000","last_unstale":"2024-04-25T08:38:26.748179+0000","last_undegraded":"2024-04-25T08:38:26.748179+0000","last_fullsized":"2024-04-25T08:38:26.748179+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:38.369059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.763967+0000","last_change":"2024-04-25T08:38:26.762078+0000","last_active":"2024-04-25T08:38:26.763967+0000","last_peered":"2024-04-25T08:38:26.763967+0000","last_clean":"2024-04-25T08:38:26.763967+0000","last_became_active":"2024-04-25T08:38:26.761678+0000","last_became_peered":"2024-04-25T08:38:26.761678+0000","last_unstale":"2024-04-25T08:38:26.763967+0000","last_undegraded":"2024-04-25T08:38:26.763967+0000","last_fullsized":"2024-04-25T08:38:26.763967+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:21.491526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.738464+0000","last_change":"2024-04-25T08:38:25.743465+0000","last_active":"2024-04-25T08:38:26.738464+0000","last_peered":"2024-04-25T08:38:26.738464+0000","last_clean":"2024-04-25T08:38:26.738464+0000","last_became_active":"2024-04-25T08:38:25.742760+0000","last_became_peered":"2024-04-25T08:38:25.742760+0000","last_unstale":"2024-04-25T08:38:26.738464+0000","last_undegraded":"2024-04-25T08:38:26.738464+0000","last_fullsized":"2024-04-25T08:38:26.738464+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:03.114483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.761290+0000","last_change":"2024-04-25T08:38:26.760909+0000","last_active":"2024-04-25T08:38:26.761290+0000","last_peered":"2024-04-25T08:38:26.761290+0000","last_clean":"2024-04-25T08:38:26.761290+0000","last_became_active":"2024-04-25T08:38:26.760621+0000","last_became_peered":"2024-04-25T08:38:26.760621+0000","last_unstale":"2024-04-25T08:38:26.761290+0000","last_undegraded":"2024-04-25T08:38:26.761290+0000","last_fullsized":"2024-04-25T08:38:26.761290+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:04.116918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.729438+0000","last_change":"2024-04-25T08:38:25.747783+0000","last_active":"2024-04-25T08:38:26.729438+0000","last_peered":"2024-04-25T08:38:26.729438+0000","last_clean":"2024-04-25T08:38:26.729438+0000","last_became_active":"2024-04-25T08:38:25.747165+0000","last_became_peered":"2024-04-25T08:38:25.747165+0000","last_unstale":"2024-04-25T08:38:26.729438+0000","last_undegraded":"2024-04-25T08:38:26.729438+0000","last_fullsized":"2024-04-25T08:38:26.729438+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:36.913090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.757714+0000","last_change":"2024-04-25T08:38:26.752251+0000","last_active":"2024-04-25T08:38:26.757714+0000","last_peered":"2024-04-25T08:38:26.757714+0000","last_clean":"2024-04-25T08:38:26.757714+0000","last_became_active":"2024-04-25T08:38:26.751975+0000","last_became_peered":"2024-04-25T08:38:26.751975+0000","last_unstale":"2024-04-25T08:38:26.757714+0000","last_undegraded":"2024-04-25T08:38:26.757714+0000","last_fullsized":"2024-04-25T08:38:26.757714+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:47.362122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.735176+0000","last_change":"2024-04-25T08:38:25.749831+0000","last_active":"2024-04-25T08:38:26.735176+0000","last_peered":"2024-04-25T08:38:26.735176+0000","last_clean":"2024-04-25T08:38:26.735176+0000","last_became_active":"2024-04-25T08:38:25.749601+0000","last_became_peered":"2024-04-25T08:38:25.749601+0000","last_unstale":"2024-04-25T08:38:26.735176+0000","last_undegraded":"2024-04-25T08:38:26.735176+0000","last_fullsized":"2024-04-25T08:38:26.735176+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:20.489091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.740672+0000","last_change":"2024-04-25T08:38:26.740103+0000","last_active":"2024-04-25T08:38:26.740672+0000","last_peered":"2024-04-25T08:38:26.740672+0000","last_clean":"2024-04-25T08:38:26.740672+0000","last_became_active":"2024-04-25T08:38:26.739782+0000","last_became_peered":"2024-04-25T08:38:26.739782+0000","last_unstale":"2024-04-25T08:38:26.740672+0000","last_undegraded":"2024-04-25T08:38:26.740672+0000","last_fullsized":"2024-04-25T08:38:26.740672+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:25.717237+0000","last_clean_scrub_stamp":"2024-04-25T08:38:25.717237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:16.961096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.756477+0000","last_change":"2024-04-25T08:38:25.745798+0000","last_active":"2024-04-25T08:38:26.756477+0000","last_peered":"2024-04-25T08:38:26.756477+0000","last_clean":"2024-04-25T08:38:26.756477+0000","last_became_active":"2024-04-25T08:38:25.745487+0000","last_became_peered":"2024-04-25T08:38:25.745487+0000","last_unstale":"2024-04-25T08:38:26.756477+0000","last_undegraded":"2024-04-25T08:38:26.756477+0000","last_fullsized":"2024-04-25T08:38:26.756477+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:38:24.714802+0000","last_clean_scrub_stamp":"2024-04-25T08:38:24.714802+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:00.553243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":311,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:38:26.735110+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:38:26.735110+0000","last_peered":"2024-04-25T08:38:26.735110+0000","last_clean":"2024-04-25T08:38:26.735110+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:38:26.735110+0000","last_undegraded":"2024-04-25T08:38:26.735110+0000","last_fullsized":"2024-04-25T08:38:26.735110+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574937,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59280,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312560,"statfs":{"total":96636764160,"available":96576061440,"internally_reserved":0,"allocated":1773568,"data_stored":1133863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574937,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54552,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317288,"statfs":{"total":96636764160,"available":96580902912,"internally_reserved":0,"allocated":1191936,"data_stored":545655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50376,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321464,"statfs":{"total":96636764160,"available":96585179136,"internally_reserved":0,"allocated":1171456,"data_stored":543439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574936,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64268,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307572,"statfs":{"total":96636764160,"available":96570953728,"internally_reserved":0,"allocated":1179648,"data_stored":543599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64616901},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574938,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68420,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303420,"statfs":{"total":96636764160,"available":96566702080,"internally_reserved":0,"allocated":1171456,"data_stored":543439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574937,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58680,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313160,"statfs":{"total":96636764160,"available":96576675840,"internally_reserved":0,"allocated":1159168,"data_stored":536272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54524,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317316,"statfs":{"total":96636764160,"available":96580931584,"internally_reserved":0,"allocated":1159168,"data_stored":536272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574937,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46788,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325052,"statfs":{"total":96636764160,"available":96588853248,"internally_reserved":0,"allocated":1753088,"data_stored":1126640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:38:30.103 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:38:30.103 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:38:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.255+0000 7f8fc8370640 1 Processor -- start 2024-04-25T08:38:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.255+0000 7f8fc8370640 1 -- start start 2024-04-25T08:38:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.256+0000 7f8fc8370640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8fc006fc60 0x7f8fc015ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.256+0000 7f8fc8370640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc015e3f0 con 0x7f8fc006f460 2024-04-25T08:38:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.256+0000 7f8fc8370640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc015e6d0 con 0x7f8fc009a730 2024-04-25T08:38:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.256+0000 7f8fc8370640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc015e9b0 con 0x7f8fc006fc60 2024-04-25T08:38:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.256+0000 7f8fc610c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8fc006fc60 0x7f8fc015ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.256+0000 7f8fc610c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8fc006fc60 0x7f8fc015ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54116/0 (socket says 172.21.15.38:54116) 2024-04-25T08:38:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.256+0000 7f8fc590b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f8fc006f460 0x7f8fc0070040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:48314/0 (socket says 172.21.15.38:48314) 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.256+0000 7f8fc610c640 1 -- 172.21.15.38:0/2211092160 learned_addr learned my addr 172.21.15.38:0/2211092160 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.256+0000 7f8fc610c640 1 -- 172.21.15.38:0/2211092160 >> v1:172.21.15.107:6789/0 conn(0x7f8fc009a730 legacy=0x7f8fc00cf7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.256+0000 7f8fc610c640 1 -- 172.21.15.38:0/2211092160 >> v1:172.21.15.38:6789/0 conn(0x7f8fc006f460 legacy=0x7f8fc0070040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.256+0000 7f8fc610c640 1 -- 172.21.15.38:0/2211092160 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc015ec90 con 0x7f8fc006fc60 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.257+0000 7f8fc610c640 1 --2- 172.21.15.38:0/2211092160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc006fc60 0x7f8fc015ddd0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009e50 tx=0x7f8fb802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4880a114479d1cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.257+0000 7f8fc510a640 1 -- 172.21.15.38:0/2211092160 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8036070 con 0x7f8fc006fc60 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.257+0000 7f8fc510a640 1 -- 172.21.15.38:0/2211092160 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb8031600 con 0x7f8fc006fc60 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.257+0000 7f8fc510a640 1 -- 172.21.15.38:0/2211092160 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80319d0 con 0x7f8fc006fc60 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.258+0000 7f8fc8370640 1 -- 172.21.15.38:0/2211092160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc006fc60 msgr2=0x7f8fc015ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.258+0000 7f8fc8370640 1 --2- 172.21.15.38:0/2211092160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc006fc60 0x7f8fc015ddd0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009e50 tx=0x7f8fb802a6d0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.258+0000 7f8fc8370640 1 -- 172.21.15.38:0/2211092160 shutdown_connections 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.258+0000 7f8fc8370640 1 --2- 172.21.15.38:0/2211092160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc006fc60 0x7f8fc015ddd0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.258+0000 7f8fc8370640 1 -- 172.21.15.38:0/2211092160 >> 172.21.15.38:0/2211092160 conn(0x7f8fc00b3140 msgr2=0x7f8fc00b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.258+0000 7f8fc8370640 1 -- 172.21.15.38:0/2211092160 shutdown_connections 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.258+0000 7f8fc8370640 1 -- 172.21.15.38:0/2211092160 wait complete. 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.259+0000 7f8fc8370640 1 Processor -- start 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.259+0000 7f8fc8370640 1 -- start start 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc8370640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc006f460 0x7f8fc00cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc8370640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8fc006fc60 0x7f8fc00b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc8370640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc009a730 0x7f8fc00b6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc8370640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc009ad40 con 0x7f8fc009a730 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc8370640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc009abc0 con 0x7f8fc006f460 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc8370640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00cfef0 con 0x7f8fc006fc60 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc690d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc009a730 0x7f8fc00b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc690d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc009a730 0x7f8fc00b6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54126/0 (socket says 172.21.15.38:54126) 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc690d640 1 -- 172.21.15.38:0/1439833472 learned_addr learned my addr 172.21.15.38:0/1439833472 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc610c640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc006f460 0x7f8fc00cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc590b640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8fc006fc60 0x7f8fc00b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc690d640 1 -- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8fc006fc60 msgr2=0x7f8fc00b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc690d640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8fc006fc60 0x7f8fc00b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc690d640 1 -- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc006f460 msgr2=0x7f8fc00cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc690d640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc006f460 0x7f8fc00cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc690d640 1 -- 172.21.15.38:0/1439833472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00b6ba0 con 0x7f8fc009a730 2024-04-25T08:38:30.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc590b640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8fc006fc60 0x7f8fc00b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:30.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.260+0000 7f8fc690d640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc009a730 0x7f8fc00b6040 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000b410 tx=0x7f8fb000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:30.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.261+0000 7f8fbeffd640 1 -- 172.21.15.38:0/1439833472 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0015030 con 0x7f8fc009a730 2024-04-25T08:38:30.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.261+0000 7f8fbeffd640 1 -- 172.21.15.38:0/1439833472 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0004020 con 0x7f8fc009a730 2024-04-25T08:38:30.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.261+0000 7f8fc8370640 1 -- 172.21.15.38:0/1439833472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc00b6e80 con 0x7f8fc009a730 2024-04-25T08:38:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.261+0000 7f8fbeffd640 1 -- 172.21.15.38:0/1439833472 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0004410 con 0x7f8fc009a730 2024-04-25T08:38:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.261+0000 7f8fc8370640 1 -- 172.21.15.38:0/1439833472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc00b6580 con 0x7f8fc009a730 2024-04-25T08:38:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.262+0000 7f8fc8370640 1 -- 172.21.15.38:0/1439833472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc0068e80 con 0x7f8fc009a730 2024-04-25T08:38:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.262+0000 7f8fbeffd640 1 -- 172.21.15.38:0/1439833472 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fb0018070 con 0x7f8fc009a730 2024-04-25T08:38:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.263+0000 7f8fbeffd640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8fac06b4d0 0x7f8fac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.263+0000 7f8fbeffd640 1 -- 172.21.15.38:0/1439833472 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8fb0089d40 con 0x7f8fc009a730 2024-04-25T08:38:30.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.263+0000 7f8fc610c640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8fac06b4d0 0x7f8fac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:30.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.263+0000 7f8fc610c640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8fac06b4d0 0x7f8fac06d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8033000 tx=0x7f8fb8009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:30.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.267+0000 7f8fbeffd640 1 -- 172.21.15.38:0/1439833472 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8fb0058ac0 con 0x7f8fc009a730 2024-04-25T08:38:30.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.516+0000 7f8fc8370640 1 -- 172.21.15.38:0/1439833472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8fc006fa40 con 0x7f8fc009a730 2024-04-25T08:38:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.519+0000 7f8fbeffd640 1 -- 172.21.15.38:0/1439833472 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8fb00583c0 con 0x7f8fc009a730 2024-04-25T08:38:30.512 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-04-25T08:38:30.512 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw== 2024-04-25T08:38:30.512 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-04-25T08:38:30.512 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-04-25T08:38:30.512 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:38:30.513 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-04-25T08:38:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.520+0000 7f8fc8370640 1 -- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8fac06b4d0 msgr2=0x7f8fac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.520+0000 7f8fc8370640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8fac06b4d0 0x7f8fac06d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8033000 tx=0x7f8fb8009000 comp rx=0 tx=0).stop 2024-04-25T08:38:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.520+0000 7f8fc8370640 1 -- 172.21.15.38:0/1439833472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc009a730 msgr2=0x7f8fc00b6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.520+0000 7f8fc8370640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc009a730 0x7f8fc00b6040 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000b410 tx=0x7f8fb000b8d0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.521+0000 7f8fc8370640 1 -- 172.21.15.38:0/1439833472 shutdown_connections 2024-04-25T08:38:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.521+0000 7f8fc8370640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8fc009a730 0x7f8fc00b6040 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.521+0000 7f8fc8370640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8fac06b4d0 0x7f8fac06d990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.521+0000 7f8fc8370640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8fc006fc60 0x7f8fc00b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.521+0000 7f8fc8370640 1 --2- 172.21.15.38:0/1439833472 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8fc006f460 0x7f8fc00cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.521+0000 7f8fc8370640 1 -- 172.21.15.38:0/1439833472 >> 172.21.15.38:0/1439833472 conn(0x7f8fc00b3140 msgr2=0x7f8fc006bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.521+0000 7f8fc8370640 1 -- 172.21.15.38:0/1439833472 shutdown_connections 2024-04-25T08:38:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.521+0000 7f8fc8370640 1 -- 172.21.15.38:0/1439833472 wait complete. 2024-04-25T08:38:30.525 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:30.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.632+0000 7f99e681f640 1 Processor -- start 2024-04-25T08:38:30.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.632+0000 7f99e681f640 1 -- start start 2024-04-25T08:38:30.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.633+0000 7f99e681f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f99e0143f40 0x7f99e00b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:30.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.633+0000 7f99e681f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99e00cdd40 0x7f99e015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.633+0000 7f99e681f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e00ce360 con 0x7f99e00b4e60 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.633+0000 7f99e681f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e00ce640 con 0x7f99e0143f40 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.633+0000 7f99e681f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e00ce920 con 0x7f99e00cdd40 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.633+0000 7f99dffff640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f99e0143f40 0x7f99e00b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.633+0000 7f99e4dbc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99e00cdd40 0x7f99e015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.634+0000 7f99dffff640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f99e0143f40 0x7f99e00b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:59750/0 (socket says 172.21.15.38:59750) 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.634+0000 7f99dffff640 1 -- 172.21.15.38:0/1541922982 learned_addr learned my addr 172.21.15.38:0/1541922982 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.634+0000 7f99df7fe640 1 --1- 172.21.15.38:0/1541922982 >> v1:172.21.15.107:6789/0 conn(0x7f99e00b4e60 0x7f99e00cd630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:52200/0 (socket says 172.21.15.38:52200) 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.634+0000 7f99e4dbc640 1 -- 172.21.15.38:0/1541922982 >> v2:172.21.15.107:3300/0 conn(0x7f99e0143f40 msgr2=0x7f99e00b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.634+0000 7f99e4dbc640 1 --2- 172.21.15.38:0/1541922982 >> v2:172.21.15.107:3300/0 conn(0x7f99e0143f40 0x7f99e00b4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.634+0000 7f99e4dbc640 1 -- 172.21.15.38:0/1541922982 >> v1:172.21.15.107:6789/0 conn(0x7f99e00b4e60 legacy=0x7f99e00cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.634+0000 7f99deffd640 1 -- 172.21.15.38:0/1541922982 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893170920 0 0) 0x7f99e00ce360 con 0x7f99e00b4e60 2024-04-25T08:38:30.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.634+0000 7f99e4dbc640 1 -- 172.21.15.38:0/1541922982 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e00cec00 con 0x7f99e00cdd40 2024-04-25T08:38:30.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.634+0000 7f99dffff640 1 --2- 172.21.15.38:0/1541922982 >> v2:172.21.15.107:3300/0 conn(0x7f99e0143f40 0x7f99e00b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:30.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.635+0000 7f99e4dbc640 1 --2- 172.21.15.38:0/1541922982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99e00cdd40 0x7f99e015abf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99d800f370 tx=0x7f99d802ea60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d4852aeb93835bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:30.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.635+0000 7f99deffd640 1 -- 172.21.15.38:0/1541922982 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8036070 con 0x7f99e00cdd40 2024-04-25T08:38:30.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.635+0000 7f99deffd640 1 -- 172.21.15.38:0/1541922982 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d800d4e0 con 0x7f99e00cdd40 2024-04-25T08:38:30.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.635+0000 7f99deffd640 1 -- 172.21.15.38:0/1541922982 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d800b960 con 0x7f99e00cdd40 2024-04-25T08:38:30.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.635+0000 7f99e681f640 1 -- 172.21.15.38:0/1541922982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99e00cdd40 msgr2=0x7f99e015abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.635+0000 7f99e681f640 1 --2- 172.21.15.38:0/1541922982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99e00cdd40 0x7f99e015abf0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99d800f370 tx=0x7f99d802ea60 comp rx=0 tx=0).stop 2024-04-25T08:38:30.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.635+0000 7f99e681f640 1 -- 172.21.15.38:0/1541922982 shutdown_connections 2024-04-25T08:38:30.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.635+0000 7f99e681f640 1 --2- 172.21.15.38:0/1541922982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99e00cdd40 0x7f99e015abf0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.635+0000 7f99e681f640 1 --2- 172.21.15.38:0/1541922982 >> v2:172.21.15.107:3300/0 conn(0x7f99e0143f40 0x7f99e00b4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.635+0000 7f99e681f640 1 -- 172.21.15.38:0/1541922982 >> 172.21.15.38:0/1541922982 conn(0x7f99e006b210 msgr2=0x7f99e006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:30.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.635+0000 7f99e681f640 1 -- 172.21.15.38:0/1541922982 shutdown_connections 2024-04-25T08:38:30.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.635+0000 7f99e681f640 1 -- 172.21.15.38:0/1541922982 wait complete. 2024-04-25T08:38:30.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.637+0000 7f99e681f640 1 Processor -- start 2024-04-25T08:38:30.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.637+0000 7f99e681f640 1 -- start start 2024-04-25T08:38:30.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.637+0000 7f99e681f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99e00b4e60 0x7f99e013aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:30.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.637+0000 7f99e681f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99e00cdd40 0x7f99e00cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:30.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.637+0000 7f99e681f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e0143f40 0x7f99e00b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:30.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.637+0000 7f99e681f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e01444a0 con 0x7f99e0143f40 2024-04-25T08:38:30.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.637+0000 7f99e681f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e0144320 con 0x7f99e00b4e60 2024-04-25T08:38:30.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.637+0000 7f99e681f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e015b130 con 0x7f99e00cdd40 2024-04-25T08:38:30.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99e4dbc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e0143f40 0x7f99e00b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:30.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99e4dbc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e0143f40 0x7f99e00b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54142/0 (socket says 172.21.15.38:54142) 2024-04-25T08:38:30.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99e4dbc640 1 -- 172.21.15.38:0/2796835204 learned_addr learned my addr 172.21.15.38:0/2796835204 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:30.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99df7fe640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99e00cdd40 0x7f99e00cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99dffff640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99e00b4e60 0x7f99e013aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99e4dbc640 1 -- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99e00cdd40 msgr2=0x7f99e00cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99e4dbc640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99e00cdd40 0x7f99e00cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99e4dbc640 1 -- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99e00b4e60 msgr2=0x7f99e013aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99e4dbc640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99e00b4e60 0x7f99e013aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99e4dbc640 1 -- 172.21.15.38:0/2796835204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e013b010 con 0x7f99e0143f40 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99e4dbc640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e0143f40 0x7f99e00b3aa0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f99d802f050 tx=0x7f99d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99dcff9640 1 -- 172.21.15.38:0/2796835204 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d804a070 con 0x7f99e0143f40 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99dcff9640 1 -- 172.21.15.38:0/2796835204 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d800d2e0 con 0x7f99e0143f40 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99dcff9640 1 -- 172.21.15.38:0/2796835204 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8036050 con 0x7f99e0143f40 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99e681f640 1 -- 172.21.15.38:0/2796835204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e013b2f0 con 0x7f99e0143f40 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.638+0000 7f99e681f640 1 -- 172.21.15.38:0/2796835204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e015b790 con 0x7f99e0143f40 2024-04-25T08:38:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.639+0000 7f99e681f640 1 -- 172.21.15.38:0/2796835204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e015b990 con 0x7f99e0143f40 2024-04-25T08:38:30.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.640+0000 7f99dcff9640 1 -- 172.21.15.38:0/2796835204 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99d804d070 con 0x7f99e0143f40 2024-04-25T08:38:30.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.640+0000 7f99dcff9640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f99c006b4d0 0x7f99c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:30.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.640+0000 7f99dcff9640 1 -- 172.21.15.38:0/2796835204 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99d80af0c0 con 0x7f99e0143f40 2024-04-25T08:38:30.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.641+0000 7f99dffff640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f99c006b4d0 0x7f99c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:30.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.641+0000 7f99dffff640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f99c006b4d0 0x7f99c006d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f99d0006ca0 tx=0x7f99d0009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:30.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.644+0000 7f99dcff9640 1 -- 172.21.15.38:0/2796835204 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99e015b990 con 0x7f99e0143f40 2024-04-25T08:38:30.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.865+0000 7f99e681f640 1 -- 172.21.15.38:0/2796835204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99e013b810 con 0x7f99e0143f40 2024-04-25T08:38:30.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.865+0000 7f99dcff9640 1 -- 172.21.15.38:0/2796835204 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f99d800d4b0 con 0x7f99e0143f40 2024-04-25T08:38:30.859 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:30.860 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3680979257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3680979257},{"type":"v1","addr":"172.21.15.107:6836","nonce":3680979257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/697909489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":697909489},{"type":"v1","addr":"172.21.15.107:6837","nonce":697909489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3083871141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3083871141},{"type":"v1","addr":"172.21.15.38:6834","nonce":3083871141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:26.095391+0000","modified":"2024-04-25T08:38:28.097807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/2922734840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":2922734840},{"type":"v1","addr":"172.21.15.38:6835","nonce":2922734840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:38:30.860 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 57 2024-04-25T08:38:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.868+0000 7f99e681f640 1 -- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f99c006b4d0 msgr2=0x7f99c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.868+0000 7f99e681f640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f99c006b4d0 0x7f99c006d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f99d0006ca0 tx=0x7f99d0009240 comp rx=0 tx=0).stop 2024-04-25T08:38:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.868+0000 7f99e681f640 1 -- 172.21.15.38:0/2796835204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e0143f40 msgr2=0x7f99e00b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.868+0000 7f99e681f640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e0143f40 0x7f99e00b3aa0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f99d802f050 tx=0x7f99d8004a40 comp rx=0 tx=0).stop 2024-04-25T08:38:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.868+0000 7f99e681f640 1 -- 172.21.15.38:0/2796835204 shutdown_connections 2024-04-25T08:38:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.868+0000 7f99e681f640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e0143f40 0x7f99e00b3aa0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.868+0000 7f99e681f640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f99c006b4d0 0x7f99c006d990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.868+0000 7f99e681f640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99e00cdd40 0x7f99e00cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.868+0000 7f99e681f640 1 --2- 172.21.15.38:0/2796835204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99e00b4e60 0x7f99e013aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.868+0000 7f99e681f640 1 -- 172.21.15.38:0/2796835204 >> 172.21.15.38:0/2796835204 conn(0x7f99e006b210 msgr2=0x7f99e0142940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.868+0000 7f99e681f640 1 -- 172.21.15.38:0/2796835204 shutdown_connections 2024-04-25T08:38:30.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:30.868+0000 7f99e681f640 1 -- 172.21.15.38:0/2796835204 wait complete. 2024-04-25T08:38:31.276 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:38:26.095391+0000', 'modified': '2024-04-25T08:38:28.097807+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7854}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7854': {'gid': 7854, 'name': 'c', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6835/2922734840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6833', 'nonce': 2922734840}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 2922734840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:38:31.276 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:38:31.276 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:38:31.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.388+0000 7f02dc8ac640 1 Processor -- start 2024-04-25T08:38:31.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.388+0000 7f02dc8ac640 1 -- start start 2024-04-25T08:38:31.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.389+0000 7f02dc8ac640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f02d80b68a0 0x7f02d80d1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.389+0000 7f02dc8ac640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f02d80ce500 0x7f02d80ce950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.389+0000 7f02dc8ac640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02d815dce0 con 0x7f02d815d900 2024-04-25T08:38:31.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.389+0000 7f02dc8ac640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d80b7b20 con 0x7f02d80b68a0 2024-04-25T08:38:31.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.389+0000 7f02dc8ac640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d80b6f10 con 0x7f02d80ce500 2024-04-25T08:38:31.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.389+0000 7f02d5d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f02d80b68a0 0x7f02d80d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.389+0000 7f02d5d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f02d80b68a0 0x7f02d80d1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54156/0 (socket says 172.21.15.38:54156) 2024-04-25T08:38:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.389+0000 7f02d5d9b640 1 -- 172.21.15.38:0/595013054 learned_addr learned my addr 172.21.15.38:0/595013054 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.389+0000 7f02d6d9d640 1 --2- 172.21.15.38:0/595013054 >> v2:172.21.15.107:3300/0 conn(0x7f02d80ce500 0x7f02d80ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.389+0000 7f02d5d9b640 1 -- 172.21.15.38:0/595013054 >> v2:172.21.15.107:3300/0 conn(0x7f02d80ce500 msgr2=0x7f02d80ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.389+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/595013054 >> v2:172.21.15.107:3300/0 conn(0x7f02d80ce500 0x7f02d80ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.389+0000 7f02d5d9b640 1 -- 172.21.15.38:0/595013054 >> v1:172.21.15.107:6789/0 conn(0x7f02d815d900 legacy=0x7f02d80b9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.390+0000 7f02d5d9b640 1 -- 172.21.15.38:0/595013054 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d80b71f0 con 0x7f02d80b68a0 2024-04-25T08:38:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.390+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/595013054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02d80b68a0 0x7f02d80d1870 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f02d80b7ca0 tx=0x7f02cc02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e10fbb5c1eeda7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.390+0000 7f02d559a640 1 -- 172.21.15.38:0/595013054 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc037070 con 0x7f02d80b68a0 2024-04-25T08:38:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.390+0000 7f02d559a640 1 -- 172.21.15.38:0/595013054 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc032440 con 0x7f02d80b68a0 2024-04-25T08:38:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.390+0000 7f02d559a640 1 -- 172.21.15.38:0/595013054 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc032810 con 0x7f02d80b68a0 2024-04-25T08:38:31.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.391+0000 7f02dc8ac640 1 -- 172.21.15.38:0/595013054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02d80b68a0 msgr2=0x7f02d80d1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.391+0000 7f02dc8ac640 1 --2- 172.21.15.38:0/595013054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02d80b68a0 0x7f02d80d1870 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f02d80b7ca0 tx=0x7f02cc02a2c0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.391+0000 7f02dc8ac640 1 -- 172.21.15.38:0/595013054 shutdown_connections 2024-04-25T08:38:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.391+0000 7f02dc8ac640 1 --2- 172.21.15.38:0/595013054 >> v2:172.21.15.107:3300/0 conn(0x7f02d80ce500 0x7f02d80ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.391+0000 7f02dc8ac640 1 --2- 172.21.15.38:0/595013054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02d80b68a0 0x7f02d80d1870 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.391+0000 7f02dc8ac640 1 -- 172.21.15.38:0/595013054 >> 172.21.15.38:0/595013054 conn(0x7f02d8066d10 msgr2=0x7f02d8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:31.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.391+0000 7f02dc8ac640 1 -- 172.21.15.38:0/595013054 shutdown_connections 2024-04-25T08:38:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.391+0000 7f02dc8ac640 1 -- 172.21.15.38:0/595013054 wait complete. 2024-04-25T08:38:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.392+0000 7f02dc8ac640 1 Processor -- start 2024-04-25T08:38:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.392+0000 7f02dc8ac640 1 -- start start 2024-04-25T08:38:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.393+0000 7f02dc8ac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02d80b68a0 0x7f02d80c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.393+0000 7f02dc8ac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d80ce500 0x7f02d80d1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.393+0000 7f02dc8ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02d815d900 0x7f02d80d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.393+0000 7f02dc8ac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80cee90 con 0x7f02d80b68a0 2024-04-25T08:38:31.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.393+0000 7f02dc8ac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80b77b0 con 0x7f02d80ce500 2024-04-25T08:38:31.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.393+0000 7f02dc8ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d80b7fb0 con 0x7f02d815d900 2024-04-25T08:38:31.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.393+0000 7f02d659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02d80b68a0 0x7f02d80c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.393+0000 7f02d659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02d80b68a0 0x7f02d80c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54172/0 (socket says 172.21.15.38:54172) 2024-04-25T08:38:31.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.393+0000 7f02d659c640 1 -- 172.21.15.38:0/3960579984 learned_addr learned my addr 172.21.15.38:0/3960579984 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:31.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.393+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d80ce500 0x7f02d80d1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.393+0000 7f02d6d9d640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02d815d900 0x7f02d80d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.394+0000 7f02d659c640 1 -- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02d815d900 msgr2=0x7f02d80d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.394+0000 7f02d659c640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02d815d900 0x7f02d80d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.394+0000 7f02d659c640 1 -- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d80ce500 msgr2=0x7f02d80d1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.394+0000 7f02d659c640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d80ce500 0x7f02d80d1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.394+0000 7f02d659c640 1 -- 172.21.15.38:0/3960579984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d80c9330 con 0x7f02d80b68a0 2024-04-25T08:38:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.394+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d80ce500 0x7f02d80d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.394+0000 7f02d659c640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02d80b68a0 0x7f02d80c5cc0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f02d000b3f0 tx=0x7f02d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:31.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.394+0000 7f02c6ffd640 1 -- 172.21.15.38:0/3960579984 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0015020 con 0x7f02d80b68a0 2024-04-25T08:38:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.394+0000 7f02c6ffd640 1 -- 172.21.15.38:0/3960579984 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d0011830 con 0x7f02d80b68a0 2024-04-25T08:38:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.394+0000 7f02dc8ac640 1 -- 172.21.15.38:0/3960579984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d80c9610 con 0x7f02d80b68a0 2024-04-25T08:38:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.394+0000 7f02c6ffd640 1 -- 172.21.15.38:0/3960579984 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0011c20 con 0x7f02d80b68a0 2024-04-25T08:38:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.394+0000 7f02dc8ac640 1 -- 172.21.15.38:0/3960579984 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d80c9bd0 con 0x7f02d80b68a0 2024-04-25T08:38:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.395+0000 7f02c6ffd640 1 -- 172.21.15.38:0/3960579984 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02d0018020 con 0x7f02d80b68a0 2024-04-25T08:38:31.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.396+0000 7f02c6ffd640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f02b806b4d0 0x7f02b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.396+0000 7f02c6ffd640 1 -- 172.21.15.38:0/3960579984 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f02d008a3b0 con 0x7f02d80b68a0 2024-04-25T08:38:31.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.396+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f02b806b4d0 0x7f02b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.397+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f02b806b4d0 0x7f02b806d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f02cc034000 tx=0x7f02cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:31.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.397+0000 7f02df5aa740 1 Processor -- start 2024-04-25T08:38:31.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.397+0000 7f02df5aa740 1 -- start start 2024-04-25T08:38:31.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02df5aa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de840230 0x55b5de840610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02df5aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de85dad0 0x55b5de85e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02df5aa740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de815f10 0x55b5de85c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02df5aa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5de778a10 con 0x55b5de840230 2024-04-25T08:38:31.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02df5aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5de721b30 con 0x55b5de85dad0 2024-04-25T08:38:31.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02df5aa740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5de774c50 con 0x55b5de815f10 2024-04-25T08:38:31.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02d659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de840230 0x55b5de840610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02d659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de840230 0x55b5de840610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54176/0 (socket says 172.21.15.38:54176) 2024-04-25T08:38:31.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02d659c640 1 -- 172.21.15.38:0/3362670192 learned_addr learned my addr 172.21.15.38:0/3362670192 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:31.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02d6d9d640 1 --2- 172.21.15.38:0/3362670192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de815f10 0x55b5de85c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/3362670192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de85dad0 0x55b5de85e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02d659c640 1 -- 172.21.15.38:0/3362670192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de815f10 msgr2=0x55b5de85c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:31.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02d659c640 1 --2- 172.21.15.38:0/3362670192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de815f10 0x55b5de85c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02d659c640 1 -- 172.21.15.38:0/3362670192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de85dad0 msgr2=0x55b5de85e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:31.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02d659c640 1 --2- 172.21.15.38:0/3362670192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de85dad0 0x55b5de85e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02d659c640 1 -- 172.21.15.38:0/3362670192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5de736e10 con 0x55b5de840230 2024-04-25T08:38:31.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02d659c640 1 --2- 172.21.15.38:0/3362670192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de840230 0x55b5de840610 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f02d0060b70 tx=0x7f02d00591c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c68a07fa060f854 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:31.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.398+0000 7f02b7fff640 1 -- 172.21.15.38:0/3362670192 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d0070020 con 0x55b5de840230 2024-04-25T08:38:31.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.399+0000 7f02b7fff640 1 -- 172.21.15.38:0/3362670192 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d0011dc0 con 0x55b5de840230 2024-04-25T08:38:31.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.399+0000 7f02b7fff640 1 -- 172.21.15.38:0/3362670192 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d008a070 con 0x55b5de840230 2024-04-25T08:38:31.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.399+0000 7f02df5aa740 1 -- 172.21.15.38:0/3362670192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de840230 msgr2=0x55b5de840610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.399+0000 7f02df5aa740 1 --2- 172.21.15.38:0/3362670192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de840230 0x55b5de840610 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f02d0060b70 tx=0x7f02d00591c0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.399+0000 7f02df5aa740 1 -- 172.21.15.38:0/3362670192 shutdown_connections 2024-04-25T08:38:31.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.399+0000 7f02df5aa740 1 --2- 172.21.15.38:0/3362670192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de815f10 0x55b5de85c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.399+0000 7f02df5aa740 1 --2- 172.21.15.38:0/3362670192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de85dad0 0x55b5de85e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.399+0000 7f02df5aa740 1 --2- 172.21.15.38:0/3362670192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de840230 0x55b5de840610 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.399+0000 7f02df5aa740 1 -- 172.21.15.38:0/3362670192 >> 172.21.15.38:0/3362670192 conn(0x55b5de812840 msgr2=0x55b5de85b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.399+0000 7f02df5aa740 1 -- 172.21.15.38:0/3362670192 shutdown_connections 2024-04-25T08:38:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.399+0000 7f02df5aa740 1 -- 172.21.15.38:0/3362670192 wait complete. 2024-04-25T08:38:31.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.400+0000 7f02df5aa740 1 -- start start 2024-04-25T08:38:31.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.400+0000 7f02df5aa740 1 Processor -- start 2024-04-25T08:38:31.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02df5aa740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de840230 0x55b5de8430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02df5aa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de843630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02df5aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de853ca0 0x55b5de83ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02df5aa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5de816cb0 con 0x55b5de85dad0 2024-04-25T08:38:31.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02df5aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5de7350c0 con 0x55b5de853ca0 2024-04-25T08:38:31.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02df5aa740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5de810770 con 0x55b5de840230 2024-04-25T08:38:31.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02df5aa740 10 client.0 _ll_register_callbacks cb 0x55b5de728ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:31.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02d5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de843630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02d5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de843630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54190/0 (socket says 172.21.15.38:54190) 2024-04-25T08:38:31.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02d5d9b640 1 -- 172.21.15.38:0/1229556776 learned_addr learned my addr 172.21.15.38:0/1229556776 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:31.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02d6d9d640 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de853ca0 0x55b5de83ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02d659c640 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de840230 0x55b5de8430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02d5d9b640 1 -- 172.21.15.38:0/1229556776 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de840230 msgr2=0x55b5de8430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:31.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de840230 0x55b5de8430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02d5d9b640 1 -- 172.21.15.38:0/1229556776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de853ca0 msgr2=0x55b5de83ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:31.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de853ca0 0x55b5de83ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.401+0000 7f02d5d9b640 1 -- 172.21.15.38:0/1229556776 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5de48d2d0 con 0x55b5de85dad0 2024-04-25T08:38:31.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de843630 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009930 tx=0x7f02cc02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad33178ab5d36c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:31.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:31.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02b57fa640 1 -- 172.21.15.38:0/1229556776 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc04f060 con 0x55b5de85dad0 2024-04-25T08:38:31.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02b57fa640 1 -- 172.21.15.38:0/1229556776 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f02cc03edc0 con 0x55b5de85dad0 2024-04-25T08:38:31.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02df5aa740 1 -- 172.21.15.38:0/1229556776 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02d0011dc0 con 0x55b5de85dad0 2024-04-25T08:38:31.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02b57fa640 1 -- 172.21.15.38:0/1229556776 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02cc055050 con 0x55b5de85dad0 2024-04-25T08:38:31.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02df5aa740 10 client.7995 fetch_fsmap learned FSMap version 57 2024-04-25T08:38:31.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02df5aa740 1 -- 172.21.15.38:0/1229556776 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55b5de5576b0 con 0x55b5de85dad0 2024-04-25T08:38:31.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02b57fa640 1 -- 172.21.15.38:0/1229556776 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d0011dc0 con 0x55b5de85dad0 2024-04-25T08:38:31.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02b57fa640 1 -- 172.21.15.38:0/1229556776 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f02cc058020 con 0x55b5de85dad0 2024-04-25T08:38:31.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02df5aa740 10 client.7995 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T08:38:31.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02df5aa740 10 client.7995 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:38:31.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02df5aa740 20 client.7995 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:31.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55b5de841540 0x55b5de86bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02df5aa740 4 client.7995 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:38:31.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02df5aa740 1 -- 172.21.15.38:0/1229556776 --> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5de7d7920 con 0x55b5de841540 2024-04-25T08:38:31.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.402+0000 7f02d659c640 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55b5de841540 0x55b5de86bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02b57fa640 10 client.7995 ms_handle_connect on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:31.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02d659c640 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55b5de841540 0x55b5de86bfd0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6d5ace3f14012c3 server_cookie=5d8f395a55c14a3a in_seq=0 out_seq=0 2024-04-25T08:38:31.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02b57fa640 1 -- 172.21.15.38:0/1229556776 <== mds.0 v2:172.21.15.38:6833/2922734840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b5de7d7920 con 0x55b5de841540 2024-04-25T08:38:31.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02b57fa640 10 client.7995 handle_command_reply: tid=0 2024-04-25T08:38:31.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02df5aa740 1 client.7995 shutdown 2024-04-25T08:38:31.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02df5aa740 1 -- 172.21.15.38:0/1229556776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 msgr2=0x55b5de843630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de843630 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009930 tx=0x7f02cc02c070 comp rx=0 tx=0).stop 2024-04-25T08:38:31.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02df5aa740 1 -- 172.21.15.38:0/1229556776 shutdown_connections 2024-04-25T08:38:31.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de853ca0 0x55b5de83ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55b5de841540 0x55b5de86bfd0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de843630 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1229556776 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de840230 0x55b5de8430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02df5aa740 1 -- 172.21.15.38:0/1229556776 >> 172.21.15.38:0/1229556776 conn(0x55b5de815f10 msgr2=0x55b5de85e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:31.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.403+0000 7f02b57fa640 0 client.7995 ms_handle_reset on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:31.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.404+0000 7f02df5aa740 1 -- 172.21.15.38:0/1229556776 shutdown_connections 2024-04-25T08:38:31.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.404+0000 7f02df5aa740 1 -- 172.21.15.38:0/1229556776 wait complete. 2024-04-25T08:38:31.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.404+0000 7f02df5aa740 20 client.7995 trim_cache size 0 max 16384 2024-04-25T08:38:31.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.420+0000 7f02df5aa740 1 Processor -- start 2024-04-25T08:38:31.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.420+0000 7f02df5aa740 1 -- start start 2024-04-25T08:38:31.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.420+0000 7f02df5aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de840230 0x55b5de859cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.420+0000 7f02df5aa740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de853ca0 0x55b5de8162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.420+0000 7f02df5aa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de816830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.420+0000 7f02df5aa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d81ff4d0 con 0x55b5de85dad0 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.420+0000 7f02df5aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d81492f0 con 0x55b5de840230 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.420+0000 7f02df5aa740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d81fef70 con 0x55b5de853ca0 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02d6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de816830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02d6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de816830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54206/0 (socket says 172.21.15.38:54206) 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02d6d9d640 1 -- 172.21.15.38:0/1282754365 learned_addr learned my addr 172.21.15.38:0/1282754365 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02d659c640 1 --2- 172.21.15.38:0/1282754365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de840230 0x55b5de859cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/1282754365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de853ca0 0x55b5de8162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02d6d9d640 1 -- 172.21.15.38:0/1282754365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de853ca0 msgr2=0x55b5de8162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02d6d9d640 1 --2- 172.21.15.38:0/1282754365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de853ca0 0x55b5de8162f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02d6d9d640 1 -- 172.21.15.38:0/1282754365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de840230 msgr2=0x55b5de859cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02d6d9d640 1 --2- 172.21.15.38:0/1282754365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de840230 0x55b5de859cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02d6d9d640 1 -- 172.21.15.38:0/1282754365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5de841e70 con 0x55b5de85dad0 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02d6d9d640 1 --2- 172.21.15.38:0/1282754365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de816830 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f02c80073c0 tx=0x7f02c8007880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3997de0b5f3f0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02b77fe640 1 -- 172.21.15.38:0/1282754365 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c801b030 con 0x55b5de85dad0 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02b77fe640 1 -- 172.21.15.38:0/1282754365 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c8017650 con 0x55b5de85dad0 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.421+0000 7f02b77fe640 1 -- 172.21.15.38:0/1282754365 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8017a40 con 0x55b5de85dad0 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.422+0000 7f02df5aa740 1 -- 172.21.15.38:0/1282754365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 msgr2=0x55b5de816830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.422+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1282754365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de816830 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f02c80073c0 tx=0x7f02c8007880 comp rx=0 tx=0).stop 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.422+0000 7f02df5aa740 1 -- 172.21.15.38:0/1282754365 shutdown_connections 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.422+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1282754365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de816830 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.422+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1282754365 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de853ca0 0x55b5de8162f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.422+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1282754365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de840230 0x55b5de859cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.422+0000 7f02df5aa740 1 -- 172.21.15.38:0/1282754365 >> 172.21.15.38:0/1282754365 conn(0x55b5de815f10 msgr2=0x55b5de83e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.422+0000 7f02df5aa740 1 -- 172.21.15.38:0/1282754365 shutdown_connections 2024-04-25T08:38:31.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.422+0000 7f02df5aa740 1 -- 172.21.15.38:0/1282754365 wait complete. 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.422+0000 7f02df5aa740 1 -- start start 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.422+0000 7f02df5aa740 1 Processor -- start 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.423+0000 7f02df5aa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de840230 0x55b5de8c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.423+0000 7f02df5aa740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de853ca0 0x55b5de842be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.423+0000 7f02df5aa740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de813de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.423+0000 7f02df5aa740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5de8413c0 con 0x55b5de85dad0 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.423+0000 7f02df5aa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5de852c30 con 0x55b5de840230 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.423+0000 7f02df5aa740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5de841990 con 0x55b5de853ca0 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.423+0000 7f02df5aa740 10 client.0 _ll_register_callbacks cb 0x55b5de810c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.423+0000 7f02d6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de813de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02d6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de813de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54218/0 (socket says 172.21.15.38:54218) 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02d6d9d640 1 -- 172.21.15.38:0/1469814576 learned_addr learned my addr 172.21.15.38:0/1469814576 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de853ca0 0x55b5de842be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02d659c640 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de840230 0x55b5de8c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02d6d9d640 1 -- 172.21.15.38:0/1469814576 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de853ca0 msgr2=0x55b5de842be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02d6d9d640 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de853ca0 0x55b5de842be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02d6d9d640 1 -- 172.21.15.38:0/1469814576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de840230 msgr2=0x55b5de8c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02d6d9d640 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de840230 0x55b5de8c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02d6d9d640 1 -- 172.21.15.38:0/1469814576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5de87dfa0 con 0x55b5de85dad0 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02d5d9b640 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de853ca0 0x55b5de842be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02d659c640 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de840230 0x55b5de8c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02d6d9d640 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de813de0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f02c800b010 tx=0x7f02c8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8efc8f1693847d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02b57fa640 1 -- 172.21.15.38:0/1469814576 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c8020820 con 0x55b5de85dad0 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02b57fa640 1 -- 172.21.15.38:0/1469814576 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f02c80209c0 con 0x55b5de85dad0 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02df5aa740 1 -- 172.21.15.38:0/1469814576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02c8017650 con 0x55b5de85dad0 2024-04-25T08:38:31.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02b57fa640 1 -- 172.21.15.38:0/1469814576 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02c8032050 con 0x55b5de85dad0 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02df5aa740 10 client.8001 fetch_fsmap learned FSMap version 57 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.424+0000 7f02df5aa740 1 -- 172.21.15.38:0/1469814576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55b5de85ebe0 con 0x55b5de85dad0 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02b57fa640 1 -- 172.21.15.38:0/1469814576 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c8017650 con 0x55b5de85dad0 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02b57fa640 1 -- 172.21.15.38:0/1469814576 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f02c8035020 con 0x55b5de85dad0 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 10 client.8001 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 10 client.8001 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 20 client.8001 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55b5de8145d0 0x55b5de86ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 4 client.8001 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 1 -- 172.21.15.38:0/1469814576 --> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b5de785f50 con 0x55b5de8145d0 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02d659c640 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55b5de8145d0 0x55b5de86ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02d659c640 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55b5de8145d0 0x55b5de86ce00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882fa6c5401f5b2e server_cookie=40c2f18779f32f0c in_seq=0 out_seq=0 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02b57fa640 10 client.8001 ms_handle_connect on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:31.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02b57fa640 1 -- 172.21.15.38:0/1469814576 <== mds.0 v2:172.21.15.38:6833/2922734840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55b5de785f50 con 0x55b5de8145d0 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02b57fa640 10 client.8001 handle_command_reply: tid=0 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 1 client.8001 shutdown 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 1 -- 172.21.15.38:0/1469814576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 msgr2=0x55b5de813de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de813de0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f02c800b010 tx=0x7f02c8004ab0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 1 -- 172.21.15.38:0/1469814576 shutdown_connections 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55b5de8145d0 0x55b5de86ce00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55b5de85dad0 0x55b5de813de0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02b57fa640 0 client.8001 ms_handle_reset on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55b5de853ca0 0x55b5de842be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 1 --2- 172.21.15.38:0/1469814576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b5de840230 0x55b5de8c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.425+0000 7f02df5aa740 1 -- 172.21.15.38:0/1469814576 >> 172.21.15.38:0/1469814576 conn(0x55b5de815f10 msgr2=0x55b5de80bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02df5aa740 1 -- 172.21.15.38:0/1469814576 shutdown_connections 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02df5aa740 1 -- 172.21.15.38:0/1469814576 wait complete. 2024-04-25T08:38:31.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02df5aa740 20 client.8001 trim_cache size 0 max 16384 2024-04-25T08:38:31.441 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:38:31.441 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:38:31.441 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:38:31.441 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 7854, 2024-04-25T08:38:31.441 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:38:31.441 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:38:31.441 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:38:31.441 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 4.7005130929999996, 2024-04-25T08:38:31.441 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 57, 2024-04-25T08:38:31.441 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 96, 2024-04-25T08:38:31.442 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 96, 2024-04-25T08:38:31.442 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 10.030444845 2024-04-25T08:38:31.442 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:38:31.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02dc8ac640 1 -- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f02b806b4d0 msgr2=0x7f02b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02dc8ac640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f02b806b4d0 0x7f02b806d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f02cc034000 tx=0x7f02cc02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:31.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02dc8ac640 1 -- 172.21.15.38:0/3960579984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02d80b68a0 msgr2=0x7f02d80c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02dc8ac640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02d80b68a0 0x7f02d80c5cc0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f02d000b3f0 tx=0x7f02d000b8b0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02dc8ac640 1 -- 172.21.15.38:0/3960579984 shutdown_connections 2024-04-25T08:38:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02dc8ac640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f02b806b4d0 0x7f02b806d990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02dc8ac640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02d815d900 0x7f02d80d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02dc8ac640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f02d80ce500 0x7f02d80d1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02dc8ac640 1 --2- 172.21.15.38:0/3960579984 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02d80b68a0 0x7f02d80c5cc0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02dc8ac640 1 -- 172.21.15.38:0/3960579984 >> 172.21.15.38:0/3960579984 conn(0x7f02d8066d10 msgr2=0x7f02d80670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02dc8ac640 1 -- 172.21.15.38:0/3960579984 shutdown_connections 2024-04-25T08:38:31.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.426+0000 7f02dc8ac640 1 -- 172.21.15.38:0/3960579984 wait complete. 2024-04-25T08:38:31.444 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:38:31.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.551+0000 7f8a87110640 1 Processor -- start 2024-04-25T08:38:31.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.551+0000 7f8a87110640 1 -- start start 2024-04-25T08:38:31.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.552+0000 7f8a87110640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8a8006cc10 0x7f8a800cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.552+0000 7f8a87110640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8a8006cff0 0x7f8a800cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.552+0000 7f8a87110640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a800d05c0 con 0x7f8a80099a70 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.552+0000 7f8a87110640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a800d08a0 con 0x7f8a8006cff0 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.552+0000 7f8a87110640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a800d0b80 con 0x7f8a8006cc10 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.552+0000 7f8a7ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8a8006cff0 0x7f8a800cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.552+0000 7f8a7ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8a8006cff0 0x7f8a800cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54226/0 (socket says 172.21.15.38:54226) 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.552+0000 7f8a7ffff640 1 -- 172.21.15.38:0/1551965202 learned_addr learned my addr 172.21.15.38:0/1551965202 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.553+0000 7f8a84eac640 1 --2- 172.21.15.38:0/1551965202 >> v2:172.21.15.107:3300/0 conn(0x7f8a8006cc10 0x7f8a800cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.553+0000 7f8a7ffff640 1 -- 172.21.15.38:0/1551965202 >> v2:172.21.15.107:3300/0 conn(0x7f8a8006cc10 msgr2=0x7f8a800cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.553+0000 7f8a7ffff640 1 --2- 172.21.15.38:0/1551965202 >> v2:172.21.15.107:3300/0 conn(0x7f8a8006cc10 0x7f8a800cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.553+0000 7f8a7ffff640 1 -- 172.21.15.38:0/1551965202 >> v1:172.21.15.107:6789/0 conn(0x7f8a80099a70 legacy=0x7f8a80099e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.553+0000 7f8a7ffff640 1 -- 172.21.15.38:0/1551965202 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a800d0e60 con 0x7f8a8006cff0 2024-04-25T08:38:31.547 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.553+0000 7f8a7ffff640 1 --2- 172.21.15.38:0/1551965202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a8006cff0 0x7f8a800cffb0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f8a80067970 tx=0x7f8a7402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c59b0615730a87b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:31.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.554+0000 7f8a7f7fe640 1 -- 172.21.15.38:0/1551965202 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74037070 con 0x7f8a8006cff0 2024-04-25T08:38:31.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.554+0000 7f8a7f7fe640 1 -- 172.21.15.38:0/1551965202 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a74032440 con 0x7f8a8006cff0 2024-04-25T08:38:31.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.554+0000 7f8a7f7fe640 1 -- 172.21.15.38:0/1551965202 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a74032810 con 0x7f8a8006cff0 2024-04-25T08:38:31.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.554+0000 7f8a87110640 1 -- 172.21.15.38:0/1551965202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a8006cff0 msgr2=0x7f8a800cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.554+0000 7f8a87110640 1 --2- 172.21.15.38:0/1551965202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a8006cff0 0x7f8a800cffb0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f8a80067970 tx=0x7f8a7402a230 comp rx=0 tx=0).stop 2024-04-25T08:38:31.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.554+0000 7f8a87110640 1 -- 172.21.15.38:0/1551965202 shutdown_connections 2024-04-25T08:38:31.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.554+0000 7f8a87110640 1 --2- 172.21.15.38:0/1551965202 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a8006cff0 0x7f8a800cffb0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.554+0000 7f8a87110640 1 --2- 172.21.15.38:0/1551965202 >> v2:172.21.15.107:3300/0 conn(0x7f8a8006cc10 0x7f8a800cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.554+0000 7f8a87110640 1 -- 172.21.15.38:0/1551965202 >> 172.21.15.38:0/1551965202 conn(0x7f8a800b31f0 msgr2=0x7f8a800b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:31.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.554+0000 7f8a87110640 1 -- 172.21.15.38:0/1551965202 shutdown_connections 2024-04-25T08:38:31.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.554+0000 7f8a87110640 1 -- 172.21.15.38:0/1551965202 wait complete. 2024-04-25T08:38:31.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.555+0000 7f8a87110640 1 Processor -- start 2024-04-25T08:38:31.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.555+0000 7f8a87110640 1 -- start start 2024-04-25T08:38:31.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.556+0000 7f8a87110640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a8006cc10 0x7f8a8014d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.556+0000 7f8a87110640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a8006cff0 0x7f8a800b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.556+0000 7f8a87110640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a80099a70 0x7f8a800c7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.556+0000 7f8a87110640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8006c2d0 con 0x7f8a80099a70 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.556+0000 7f8a87110640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8006c150 con 0x7f8a8006cff0 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.556+0000 7f8a87110640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8015d370 con 0x7f8a8006cc10 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.556+0000 7f8a856ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a80099a70 0x7f8a800c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.556+0000 7f8a856ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a80099a70 0x7f8a800c7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54232/0 (socket says 172.21.15.38:54232) 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.556+0000 7f8a856ad640 1 -- 172.21.15.38:0/4207044791 learned_addr learned my addr 172.21.15.38:0/4207044791 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a7ffff640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a8006cff0 0x7f8a800b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a84eac640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a8006cc10 0x7f8a8014d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a856ad640 1 -- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a8006cc10 msgr2=0x7f8a8014d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a856ad640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a8006cc10 0x7f8a8014d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a856ad640 1 -- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a8006cff0 msgr2=0x7f8a800b66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a856ad640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a8006cff0 0x7f8a800b66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a856ad640 1 -- 172.21.15.38:0/4207044791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8014da20 con 0x7f8a80099a70 2024-04-25T08:38:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a7ffff640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a8006cff0 0x7f8a800b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a856ad640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a80099a70 0x7f8a800c7400 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000b490 tx=0x7f8a7000b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a7d7fa640 1 -- 172.21.15.38:0/4207044791 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70015020 con 0x7f8a80099a70 2024-04-25T08:38:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a7d7fa640 1 -- 172.21.15.38:0/4207044791 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a70011850 con 0x7f8a80099a70 2024-04-25T08:38:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a87110640 1 -- 172.21.15.38:0/4207044791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a800b70f0 con 0x7f8a80099a70 2024-04-25T08:38:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a7d7fa640 1 -- 172.21.15.38:0/4207044791 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a70011c40 con 0x7f8a80099a70 2024-04-25T08:38:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.557+0000 7f8a87110640 1 -- 172.21.15.38:0/4207044791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a800c8d90 con 0x7f8a80099a70 2024-04-25T08:38:31.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.558+0000 7f8a87110640 1 -- 172.21.15.38:0/4207044791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a800c88e0 con 0x7f8a80099a70 2024-04-25T08:38:31.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.559+0000 7f8a7d7fa640 1 -- 172.21.15.38:0/4207044791 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a70018020 con 0x7f8a80099a70 2024-04-25T08:38:31.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.559+0000 7f8a7d7fa640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8a6c06b4d0 0x7f8a6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:31.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.559+0000 7f8a7d7fa640 1 -- 172.21.15.38:0/4207044791 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8a7008a270 con 0x7f8a80099a70 2024-04-25T08:38:31.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.560+0000 7f8a84eac640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8a6c06b4d0 0x7f8a6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:31.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.560+0000 7f8a84eac640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8a6c06b4d0 0x7f8a6c06d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8a800cd520 tx=0x7f8a78008460 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:31.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.564+0000 7f8a7d7fa640 1 -- 172.21.15.38:0/4207044791 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a7001f020 con 0x7f8a80099a70 2024-04-25T08:38:31.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.787+0000 7f8a87110640 1 -- 172.21.15.38:0/4207044791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8a80068e80 con 0x7f8a80099a70 2024-04-25T08:38:31.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.788+0000 7f8a7d7fa640 1 -- 172.21.15.38:0/4207044791 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8a700172c0 con 0x7f8a80099a70 2024-04-25T08:38:31.781 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:38:31.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.790+0000 7f8a87110640 1 -- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8a6c06b4d0 msgr2=0x7f8a6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.790+0000 7f8a87110640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8a6c06b4d0 0x7f8a6c06d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8a800cd520 tx=0x7f8a78008460 comp rx=0 tx=0).stop 2024-04-25T08:38:31.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.790+0000 7f8a87110640 1 -- 172.21.15.38:0/4207044791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a80099a70 msgr2=0x7f8a800c7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:31.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.790+0000 7f8a87110640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a80099a70 0x7f8a800c7400 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f8a7000b490 tx=0x7f8a7000b950 comp rx=0 tx=0).stop 2024-04-25T08:38:31.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.790+0000 7f8a87110640 1 -- 172.21.15.38:0/4207044791 shutdown_connections 2024-04-25T08:38:31.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.790+0000 7f8a87110640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8a80099a70 0x7f8a800c7400 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.790+0000 7f8a87110640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a8006cff0 0x7f8a800b66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.790+0000 7f8a87110640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8a6c06b4d0 0x7f8a6c06d990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.790+0000 7f8a87110640 1 --2- 172.21.15.38:0/4207044791 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a8006cc10 0x7f8a8014d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:31.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.790+0000 7f8a87110640 1 -- 172.21.15.38:0/4207044791 >> 172.21.15.38:0/4207044791 conn(0x7f8a800b31f0 msgr2=0x7f8a800b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:31.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.790+0000 7f8a87110640 1 -- 172.21.15.38:0/4207044791 shutdown_connections 2024-04-25T08:38:31.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:31.790+0000 7f8a87110640 1 -- 172.21.15.38:0/4207044791 wait complete. 2024-04-25T08:38:31.795 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:38:31.795 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:38:31.795 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:38:31.795 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:38:31.795 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:31.795 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:38:31.795 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:38:31.795 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:38:31.795 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:38:31.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:31.796 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:38:31.812 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:38:31.812 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:38:31.812 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:38:31.812 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42325sec preferred_lft 42325sec 2024-04-25T08:38:31.813 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::440f:97ff:fe63:87b/64 scope link 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ea:23:76:ab:b1:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:38:31.814 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::e823:76ff:feab:b1e9/64 scope link 2024-04-25T08:38:31.815 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:31.815 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:38:31.815 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:38:31.815 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::38c7:38ff:fe2e:6c92/64 scope link 2024-04-25T08:38:31.815 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:38:31.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:31.815 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:38:31.875 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:38:31.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:31.877 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:38:31.933 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:38:31.933 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:38:31.933 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:38:31.933 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:38:31.933 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:38:31.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:31.934 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:38:31.934 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:31.934 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:38:31.934 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:38:31.934 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:38:31.934 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:32.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:32.020 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:38:32.035 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:38:32.035 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:38:32.035 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:38:32.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:32.036 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T08:38:32.100 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:38:32.100 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:38:32.100 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:32.156 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:38:32.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:38:32.156 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:32.211 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:38:32.276 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:38:32.332 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:38:32.332 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:38:32.332 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:38:32.332 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:38:32.332 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:38:32.333 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:38:32.334 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:38:32.335 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:38:32.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:32.335 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:38:32.378 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-04-25T08:38:32.385+0000 7fde001a2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:38:32.387 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:38:32.388 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:38:32.431 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:38:32.455 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:38:32.512 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:38:32.512 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:38:32.512 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:38:32.512 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:38:32.512 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:38:32.512 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:38:32.512 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:38:32.512 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:38:32.512 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:38:32.512 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:38:32.513 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:38:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:38:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:38:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:38:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:38:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:38:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:38:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:38:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:38:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:38:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:38:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:38:32.514 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:38:32.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:32.515 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:38:32.516 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-04-25T08:38:32.519+0000 7f100cdb6480 -1 init, newargv = 0x55eae0353440 newargc=15 2024-04-25T08:38:32.516 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[79049]: starting ceph client 2024-04-25T08:38:32.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[79049]: starting fuse 2024-04-25T08:38:32.531 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:38:32.531 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:38:32.531 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:38:32.531 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:38:32.531 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:38:32.531 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:38:32.532 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:38:32.532 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:38:32.532 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:38:32.532 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:38:32.532 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:38:32.532 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:38:32.532 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:38:32.532 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:38:32.532 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:38:32.532 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:38:32.532 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:38:32.532 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:38:32.533 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:38:32.533 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:38:32.533 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:38:32.533 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:38:32.533 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:38:32.533 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:38:32.533 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:38:32.533 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:38:32.533 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:38:32.533 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-04-25T08:38:32.533 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:38:32.651 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.79049.asok 2024-04-25T08:38:32.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:38:32.661 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.79049.asok 2024-04-25T08:38:32.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:32.662 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.79049.asok status 2024-04-25T08:38:32.799 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:38:32.800 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:38:32.800 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:38:32.800 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:38:32.800 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-04-25T08:38:32.800 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:38:32.800 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:38:32.800 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "79049", 2024-04-25T08:38:32.800 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:38:32.800 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:32.800 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:38:32.801 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:38:32.801 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 8013, 2024-04-25T08:38:32.801 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:38:32.801 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:38:32.801 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:38:32.801 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 8013 2024-04-25T08:38:32.801 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:32.801 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:38:32.801 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:38:32.801 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:38:32.801 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2843655381 2024-04-25T08:38:32.802 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:32.802 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:32.802 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:38:32.802 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:38:32.802 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:38:32.802 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2843655381 2024-04-25T08:38:32.802 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:38:32.802 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.8013 192.168.0.1:0/2843655381", 2024-04-25T08:38:32.802 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/2843655381", 2024-04-25T08:38:32.802 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:38:32.803 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 57, 2024-04-25T08:38:32.803 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 96, 2024-04-25T08:38:32.803 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:38:32.803 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:38:32.803 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:38:32.803 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:38:32.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:32.808 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:38:32.808 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:32.824 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:38:32.824 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:32.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:32.824 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:32.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:32.889 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:32.890 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:32.890 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T08:38:32.890 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:32.890 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:32.890 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:32.890 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:32.966 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:32.966 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:32.966 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00947141 s, 111 MB/s 2024-04-25T08:38:32.975 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:32.975 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:32.976 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725437 s, 145 MB/s 2024-04-25T08:38:32.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:32.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:32.988 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101166 s, 104 MB/s 2024-04-25T08:38:32.997 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:32.997 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:32.997 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684486 s, 153 MB/s 2024-04-25T08:38:33.007 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.007 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.007 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793702 s, 132 MB/s 2024-04-25T08:38:33.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.018 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087361 s, 120 MB/s 2024-04-25T08:38:33.029 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.029 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.029 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896969 s, 117 MB/s 2024-04-25T08:38:33.040 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.040 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.040 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889735 s, 118 MB/s 2024-04-25T08:38:33.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.052 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984688 s, 106 MB/s 2024-04-25T08:38:33.063 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.063 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.063 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862787 s, 122 MB/s 2024-04-25T08:38:33.073 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.073 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.073 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085506 s, 123 MB/s 2024-04-25T08:38:33.083 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.084 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.084 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797683 s, 131 MB/s 2024-04-25T08:38:33.093 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.093 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.093 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007584 s, 138 MB/s 2024-04-25T08:38:33.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.104 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821755 s, 128 MB/s 2024-04-25T08:38:33.114 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.114 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.114 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791911 s, 132 MB/s 2024-04-25T08:38:33.123 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.123 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.123 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752256 s, 139 MB/s 2024-04-25T08:38:33.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.133 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714264 s, 147 MB/s 2024-04-25T08:38:33.144 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.144 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.144 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00945338 s, 111 MB/s 2024-04-25T08:38:33.155 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.155 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.155 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815616 s, 129 MB/s 2024-04-25T08:38:33.164 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.164 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.164 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731812 s, 143 MB/s 2024-04-25T08:38:33.174 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.175 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.175 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848564 s, 124 MB/s 2024-04-25T08:38:33.184 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.184 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.184 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073034 s, 144 MB/s 2024-04-25T08:38:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.198 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.011367 s, 92.2 MB/s 2024-04-25T08:38:33.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.209 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839476 s, 125 MB/s 2024-04-25T08:38:33.219 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.219 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861161 s, 122 MB/s 2024-04-25T08:38:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713699 s, 147 MB/s 2024-04-25T08:38:33.239 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.239 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.239 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833355 s, 126 MB/s 2024-04-25T08:38:33.250 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.250 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.250 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858786 s, 122 MB/s 2024-04-25T08:38:33.259 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.260 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.260 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076927 s, 136 MB/s 2024-04-25T08:38:33.271 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.271 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.271 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089668 s, 117 MB/s 2024-04-25T08:38:33.281 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.281 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.281 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798923 s, 131 MB/s 2024-04-25T08:38:33.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.292 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885146 s, 118 MB/s 2024-04-25T08:38:33.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:33.294 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:33.294 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:33.294 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T08:38:33.294 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:33.294 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:33.294 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:33.294 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:33.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.362 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713093 s, 147 MB/s 2024-04-25T08:38:33.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.373 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871747 s, 120 MB/s 2024-04-25T08:38:33.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.383 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.383 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762423 s, 138 MB/s 2024-04-25T08:38:33.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.393 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819395 s, 128 MB/s 2024-04-25T08:38:33.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.403 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811565 s, 129 MB/s 2024-04-25T08:38:33.413 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.414 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793819 s, 132 MB/s 2024-04-25T08:38:33.423 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.423 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.423 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742039 s, 141 MB/s 2024-04-25T08:38:33.433 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.433 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.433 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754178 s, 139 MB/s 2024-04-25T08:38:33.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.443 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804216 s, 130 MB/s 2024-04-25T08:38:33.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.454 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812062 s, 129 MB/s 2024-04-25T08:38:33.464 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.464 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.464 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852564 s, 123 MB/s 2024-04-25T08:38:33.475 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.475 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.475 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872837 s, 120 MB/s 2024-04-25T08:38:33.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.485 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731776 s, 143 MB/s 2024-04-25T08:38:33.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.494 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691068 s, 152 MB/s 2024-04-25T08:38:33.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.504 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.504 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738058 s, 142 MB/s 2024-04-25T08:38:33.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.513 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073306 s, 143 MB/s 2024-04-25T08:38:33.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.522 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069889 s, 150 MB/s 2024-04-25T08:38:33.532 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.532 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.532 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713381 s, 147 MB/s 2024-04-25T08:38:33.543 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.543 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.543 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853953 s, 123 MB/s 2024-04-25T08:38:33.553 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.553 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.553 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809328 s, 130 MB/s 2024-04-25T08:38:33.564 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.564 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.564 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922873 s, 114 MB/s 2024-04-25T08:38:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713819 s, 147 MB/s 2024-04-25T08:38:33.584 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.584 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.584 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786609 s, 133 MB/s 2024-04-25T08:38:33.594 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.595 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.595 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842372 s, 124 MB/s 2024-04-25T08:38:33.605 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.605 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.606 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867184 s, 121 MB/s 2024-04-25T08:38:33.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.616 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852307 s, 123 MB/s 2024-04-25T08:38:33.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.627 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789618 s, 133 MB/s 2024-04-25T08:38:33.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.636 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702366 s, 149 MB/s 2024-04-25T08:38:33.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.646 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792598 s, 132 MB/s 2024-04-25T08:38:33.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.656 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763551 s, 137 MB/s 2024-04-25T08:38:33.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.666 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826154 s, 127 MB/s 2024-04-25T08:38:33.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.676 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787742 s, 133 MB/s 2024-04-25T08:38:33.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:33.677 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:33.677 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:33.677 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T08:38:33.677 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:33.678 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:33.678 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:33.678 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:33.746 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.746 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.746 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802817 s, 131 MB/s 2024-04-25T08:38:33.756 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.756 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.756 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760837 s, 138 MB/s 2024-04-25T08:38:33.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.765 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698973 s, 150 MB/s 2024-04-25T08:38:33.776 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.776 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.776 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087743 s, 120 MB/s 2024-04-25T08:38:33.786 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.786 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.786 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769718 s, 136 MB/s 2024-04-25T08:38:33.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.796 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780756 s, 134 MB/s 2024-04-25T08:38:33.805 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.806 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.806 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787459 s, 133 MB/s 2024-04-25T08:38:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.816 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.817 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854331 s, 123 MB/s 2024-04-25T08:38:33.827 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.827 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.827 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883579 s, 119 MB/s 2024-04-25T08:38:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766416 s, 137 MB/s 2024-04-25T08:38:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751861 s, 139 MB/s 2024-04-25T08:38:33.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.857 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798089 s, 131 MB/s 2024-04-25T08:38:33.866 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.867 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.867 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070669 s, 148 MB/s 2024-04-25T08:38:33.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.877 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850629 s, 123 MB/s 2024-04-25T08:38:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.886 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683545 s, 153 MB/s 2024-04-25T08:38:33.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.897 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861803 s, 122 MB/s 2024-04-25T08:38:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.907 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757627 s, 138 MB/s 2024-04-25T08:38:33.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.917 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789086 s, 133 MB/s 2024-04-25T08:38:33.927 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.927 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.927 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833724 s, 126 MB/s 2024-04-25T08:38:33.937 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.937 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.937 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771965 s, 136 MB/s 2024-04-25T08:38:33.938 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:38:33.940 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:38:33.947 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.947 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.947 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804444 s, 130 MB/s 2024-04-25T08:38:33.958 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.959 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.959 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865511 s, 121 MB/s 2024-04-25T08:38:33.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.970 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872718 s, 120 MB/s 2024-04-25T08:38:33.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.980 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801167 s, 131 MB/s 2024-04-25T08:38:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724198 s, 145 MB/s 2024-04-25T08:38:33.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:33.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:33.999 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715425 s, 147 MB/s 2024-04-25T08:38:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850051 s, 123 MB/s 2024-04-25T08:38:34.020 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.020 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.020 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799009 s, 131 MB/s 2024-04-25T08:38:34.030 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.030 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.030 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081123 s, 129 MB/s 2024-04-25T08:38:34.039 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.039 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.040 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007149 s, 147 MB/s 2024-04-25T08:38:34.050 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.050 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.050 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852531 s, 123 MB/s 2024-04-25T08:38:34.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.060 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759003 s, 138 MB/s 2024-04-25T08:38:34.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:34.061 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:34.061 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:34.061 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T08:38:34.061 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:34.061 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:34.061 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:34.062 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:34.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.129 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.129 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694688 s, 151 MB/s 2024-04-25T08:38:34.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.139 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776753 s, 135 MB/s 2024-04-25T08:38:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701617 s, 149 MB/s 2024-04-25T08:38:34.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.158 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794534 s, 132 MB/s 2024-04-25T08:38:34.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.169 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.169 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008563 s, 122 MB/s 2024-04-25T08:38:34.179 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.179 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.179 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838384 s, 125 MB/s 2024-04-25T08:38:34.189 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.189 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.189 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755009 s, 139 MB/s 2024-04-25T08:38:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.199 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812178 s, 129 MB/s 2024-04-25T08:38:34.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.208 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692741 s, 151 MB/s 2024-04-25T08:38:34.219 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.219 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.219 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864072 s, 121 MB/s 2024-04-25T08:38:34.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.229 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737074 s, 142 MB/s 2024-04-25T08:38:34.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.239 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007826 s, 134 MB/s 2024-04-25T08:38:34.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.248 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709395 s, 148 MB/s 2024-04-25T08:38:34.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.257 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730995 s, 143 MB/s 2024-04-25T08:38:34.268 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.268 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.268 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869772 s, 121 MB/s 2024-04-25T08:38:34.279 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.279 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.279 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857578 s, 122 MB/s 2024-04-25T08:38:34.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.289 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757395 s, 138 MB/s 2024-04-25T08:38:34.298 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.298 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.299 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779493 s, 135 MB/s 2024-04-25T08:38:34.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.308 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.308 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695071 s, 151 MB/s 2024-04-25T08:38:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.317 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691991 s, 152 MB/s 2024-04-25T08:38:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.326 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721194 s, 145 MB/s 2024-04-25T08:38:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839951 s, 125 MB/s 2024-04-25T08:38:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077458 s, 135 MB/s 2024-04-25T08:38:34.357 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.357 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.357 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801632 s, 131 MB/s 2024-04-25T08:38:34.367 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.368 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.368 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861348 s, 122 MB/s 2024-04-25T08:38:34.377 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.377 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.377 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682879 s, 154 MB/s 2024-04-25T08:38:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.386 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710535 s, 148 MB/s 2024-04-25T08:38:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.396 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780964 s, 134 MB/s 2024-04-25T08:38:34.407 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.407 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.407 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851929 s, 123 MB/s 2024-04-25T08:38:34.418 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.418 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.418 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889145 s, 118 MB/s 2024-04-25T08:38:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.427 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704918 s, 149 MB/s 2024-04-25T08:38:34.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.437 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783724 s, 134 MB/s 2024-04-25T08:38:34.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:34.438 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:34.438 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:34.438 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T08:38:34.438 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:34.438 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:34.439 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:34.439 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:34.506 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.506 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.506 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774018 s, 135 MB/s 2024-04-25T08:38:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.516 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755903 s, 139 MB/s 2024-04-25T08:38:34.526 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.526 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.526 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831114 s, 126 MB/s 2024-04-25T08:38:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.536 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765846 s, 137 MB/s 2024-04-25T08:38:34.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.547 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842627 s, 124 MB/s 2024-04-25T08:38:34.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.557 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803885 s, 130 MB/s 2024-04-25T08:38:34.568 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.568 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.568 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883649 s, 119 MB/s 2024-04-25T08:38:34.578 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.579 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.579 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835607 s, 125 MB/s 2024-04-25T08:38:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706082 s, 149 MB/s 2024-04-25T08:38:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.597 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695348 s, 151 MB/s 2024-04-25T08:38:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769393 s, 136 MB/s 2024-04-25T08:38:34.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.618 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846675 s, 124 MB/s 2024-04-25T08:38:34.627 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.627 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.627 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076086 s, 138 MB/s 2024-04-25T08:38:34.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.636 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693289 s, 151 MB/s 2024-04-25T08:38:34.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.647 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080791 s, 130 MB/s 2024-04-25T08:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.656 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693906 s, 151 MB/s 2024-04-25T08:38:34.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.665 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071835 s, 146 MB/s 2024-04-25T08:38:34.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.674 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684839 s, 153 MB/s 2024-04-25T08:38:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.684 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070288 s, 149 MB/s 2024-04-25T08:38:34.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.694 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857517 s, 122 MB/s 2024-04-25T08:38:34.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.704 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786816 s, 133 MB/s 2024-04-25T08:38:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.715 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852506 s, 123 MB/s 2024-04-25T08:38:34.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.725 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816002 s, 129 MB/s 2024-04-25T08:38:34.735 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.735 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.735 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766417 s, 137 MB/s 2024-04-25T08:38:34.745 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.745 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.745 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773503 s, 136 MB/s 2024-04-25T08:38:34.756 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.756 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.757 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877791 s, 119 MB/s 2024-04-25T08:38:34.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.766 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703732 s, 149 MB/s 2024-04-25T08:38:34.775 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.775 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.775 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750785 s, 140 MB/s 2024-04-25T08:38:34.786 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.786 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.786 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855764 s, 123 MB/s 2024-04-25T08:38:34.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.796 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778959 s, 135 MB/s 2024-04-25T08:38:34.806 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.806 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.806 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790557 s, 133 MB/s 2024-04-25T08:38:34.817 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.817 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.817 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868678 s, 121 MB/s 2024-04-25T08:38:34.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:34.818 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:34.818 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:34.818 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T08:38:34.818 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:34.818 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:34.818 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:34.818 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:34.886 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.887 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729805 s, 144 MB/s 2024-04-25T08:38:34.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.896 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783401 s, 134 MB/s 2024-04-25T08:38:34.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.905 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741228 s, 141 MB/s 2024-04-25T08:38:34.916 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.916 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.916 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858617 s, 122 MB/s 2024-04-25T08:38:34.925 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.926 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776475 s, 135 MB/s 2024-04-25T08:38:34.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.936 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859846 s, 122 MB/s 2024-04-25T08:38:34.946 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.946 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.946 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714428 s, 147 MB/s 2024-04-25T08:38:34.956 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.956 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.956 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077051 s, 136 MB/s 2024-04-25T08:38:34.964 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.965 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695639 s, 151 MB/s 2024-04-25T08:38:34.974 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.974 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.974 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763252 s, 137 MB/s 2024-04-25T08:38:34.985 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.985 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.986 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878252 s, 119 MB/s 2024-04-25T08:38:34.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:34.995 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:34.995 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681943 s, 154 MB/s 2024-04-25T08:38:35.004 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.004 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.004 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760014 s, 138 MB/s 2024-04-25T08:38:35.014 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.014 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.014 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749598 s, 140 MB/s 2024-04-25T08:38:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.024 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764549 s, 137 MB/s 2024-04-25T08:38:35.033 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.033 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.033 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719478 s, 146 MB/s 2024-04-25T08:38:35.044 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.044 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.044 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843236 s, 124 MB/s 2024-04-25T08:38:35.055 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.055 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.055 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084864 s, 124 MB/s 2024-04-25T08:38:35.064 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.064 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.064 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696785 s, 150 MB/s 2024-04-25T08:38:35.073 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.073 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.073 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070974 s, 148 MB/s 2024-04-25T08:38:35.082 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.083 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.083 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750542 s, 140 MB/s 2024-04-25T08:38:35.092 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.092 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.092 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704449 s, 149 MB/s 2024-04-25T08:38:35.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.102 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.102 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752955 s, 139 MB/s 2024-04-25T08:38:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753427 s, 139 MB/s 2024-04-25T08:38:35.121 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.121 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.121 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007955 s, 132 MB/s 2024-04-25T08:38:35.131 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.132 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.132 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819506 s, 128 MB/s 2024-04-25T08:38:35.142 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.142 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.142 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803052 s, 131 MB/s 2024-04-25T08:38:35.151 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.152 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768778 s, 136 MB/s 2024-04-25T08:38:35.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.162 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850581 s, 123 MB/s 2024-04-25T08:38:35.172 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.173 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.173 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793394 s, 132 MB/s 2024-04-25T08:38:35.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.182 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071727 s, 146 MB/s 2024-04-25T08:38:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689773 s, 152 MB/s 2024-04-25T08:38:35.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:35.192 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:35.192 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:35.192 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T08:38:35.192 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:35.192 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:35.192 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:35.192 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:35.260 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.260 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.260 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758261 s, 138 MB/s 2024-04-25T08:38:35.269 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.269 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.269 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728498 s, 144 MB/s 2024-04-25T08:38:35.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.278 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682638 s, 154 MB/s 2024-04-25T08:38:35.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.288 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687847 s, 152 MB/s 2024-04-25T08:38:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.298 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784845 s, 134 MB/s 2024-04-25T08:38:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770105 s, 136 MB/s 2024-04-25T08:38:35.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.319 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886816 s, 118 MB/s 2024-04-25T08:38:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.330 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857172 s, 122 MB/s 2024-04-25T08:38:35.339 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.339 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.339 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762494 s, 138 MB/s 2024-04-25T08:38:35.348 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680503 s, 154 MB/s 2024-04-25T08:38:35.358 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.358 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.358 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729452 s, 144 MB/s 2024-04-25T08:38:35.368 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.368 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.369 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804465 s, 130 MB/s 2024-04-25T08:38:35.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.378 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723723 s, 145 MB/s 2024-04-25T08:38:35.387 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.387 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.387 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714239 s, 147 MB/s 2024-04-25T08:38:35.397 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.397 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.397 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073115 s, 143 MB/s 2024-04-25T08:38:35.406 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.406 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.406 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732005 s, 143 MB/s 2024-04-25T08:38:35.416 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.417 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.417 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808739 s, 130 MB/s 2024-04-25T08:38:35.425 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.426 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.426 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672965 s, 156 MB/s 2024-04-25T08:38:35.435 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.435 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.435 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723371 s, 145 MB/s 2024-04-25T08:38:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.444 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661736 s, 158 MB/s 2024-04-25T08:38:35.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.453 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073495 s, 143 MB/s 2024-04-25T08:38:35.462 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.463 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711082 s, 147 MB/s 2024-04-25T08:38:35.472 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.472 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.472 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727795 s, 144 MB/s 2024-04-25T08:38:35.481 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.482 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.482 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069903 s, 150 MB/s 2024-04-25T08:38:35.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.491 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689361 s, 152 MB/s 2024-04-25T08:38:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.501 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748202 s, 140 MB/s 2024-04-25T08:38:35.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.511 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815093 s, 129 MB/s 2024-04-25T08:38:35.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.521 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706025 s, 149 MB/s 2024-04-25T08:38:35.529 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.530 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.530 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727866 s, 144 MB/s 2024-04-25T08:38:35.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.539 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.539 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719473 s, 146 MB/s 2024-04-25T08:38:35.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.548 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.548 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676028 s, 155 MB/s 2024-04-25T08:38:35.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.558 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761113 s, 138 MB/s 2024-04-25T08:38:35.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:35.559 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:35.560 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:35.560 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T08:38:35.560 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:35.560 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:35.560 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:35.560 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:35.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.628 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834207 s, 126 MB/s 2024-04-25T08:38:35.637 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.638 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.638 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739821 s, 142 MB/s 2024-04-25T08:38:35.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.647 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666695 s, 157 MB/s 2024-04-25T08:38:35.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.656 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683697 s, 153 MB/s 2024-04-25T08:38:35.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.665 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735076 s, 143 MB/s 2024-04-25T08:38:35.675 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.675 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.675 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729784 s, 144 MB/s 2024-04-25T08:38:35.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.685 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804257 s, 130 MB/s 2024-04-25T08:38:35.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.694 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685379 s, 153 MB/s 2024-04-25T08:38:35.703 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.703 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.703 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688261 s, 152 MB/s 2024-04-25T08:38:35.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.715 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073564 s, 143 MB/s 2024-04-25T08:38:35.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.722 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699922 s, 150 MB/s 2024-04-25T08:38:35.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.731 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684064 s, 153 MB/s 2024-04-25T08:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.741 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815186 s, 129 MB/s 2024-04-25T08:38:35.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.751 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725891 s, 144 MB/s 2024-04-25T08:38:35.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.761 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008015 s, 131 MB/s 2024-04-25T08:38:35.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.771 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729957 s, 144 MB/s 2024-04-25T08:38:35.780 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.780 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.780 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740527 s, 142 MB/s 2024-04-25T08:38:35.789 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.789 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.789 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679643 s, 154 MB/s 2024-04-25T08:38:35.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.800 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802654 s, 131 MB/s 2024-04-25T08:38:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.809 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736324 s, 142 MB/s 2024-04-25T08:38:35.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738701 s, 142 MB/s 2024-04-25T08:38:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.828 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715304 s, 147 MB/s 2024-04-25T08:38:35.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.839 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799033 s, 131 MB/s 2024-04-25T08:38:35.849 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.849 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.849 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802476 s, 131 MB/s 2024-04-25T08:38:35.859 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.859 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.859 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776559 s, 135 MB/s 2024-04-25T08:38:35.868 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.868 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.868 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067082 s, 156 MB/s 2024-04-25T08:38:35.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.877 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700188 s, 150 MB/s 2024-04-25T08:38:35.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.887 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789685 s, 133 MB/s 2024-04-25T08:38:35.896 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.896 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.896 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662463 s, 158 MB/s 2024-04-25T08:38:35.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.905 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724583 s, 145 MB/s 2024-04-25T08:38:35.914 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.914 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.914 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678027 s, 155 MB/s 2024-04-25T08:38:35.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.924 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.924 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730742 s, 143 MB/s 2024-04-25T08:38:35.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:35.925 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:35.925 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:35.925 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T08:38:35.925 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:35.925 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:35.925 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:35.925 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:35.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:35.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:35.994 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080516 s, 130 MB/s 2024-04-25T08:38:36.004 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.004 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.004 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861788 s, 122 MB/s 2024-04-25T08:38:36.015 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.015 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.015 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810358 s, 129 MB/s 2024-04-25T08:38:36.024 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.024 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.024 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710245 s, 148 MB/s 2024-04-25T08:38:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646356 s, 162 MB/s 2024-04-25T08:38:36.043 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.043 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.043 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789811 s, 133 MB/s 2024-04-25T08:38:36.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.052 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067077 s, 156 MB/s 2024-04-25T08:38:36.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.062 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792707 s, 132 MB/s 2024-04-25T08:38:36.071 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.071 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.071 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733449 s, 143 MB/s 2024-04-25T08:38:36.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.080 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643819 s, 163 MB/s 2024-04-25T08:38:36.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.090 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821479 s, 128 MB/s 2024-04-25T08:38:36.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.100 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700852 s, 150 MB/s 2024-04-25T08:38:36.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.109 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706839 s, 148 MB/s 2024-04-25T08:38:36.118 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.118 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.118 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731538 s, 143 MB/s 2024-04-25T08:38:36.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.128 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738223 s, 142 MB/s 2024-04-25T08:38:36.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.138 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815066 s, 129 MB/s 2024-04-25T08:38:36.147 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.147 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.147 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638705 s, 164 MB/s 2024-04-25T08:38:36.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.156 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729865 s, 144 MB/s 2024-04-25T08:38:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.167 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802231 s, 131 MB/s 2024-04-25T08:38:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.176 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712953 s, 147 MB/s 2024-04-25T08:38:36.186 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.186 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.186 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752309 s, 139 MB/s 2024-04-25T08:38:36.195 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.195 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.195 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737607 s, 142 MB/s 2024-04-25T08:38:36.204 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.204 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.204 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707673 s, 148 MB/s 2024-04-25T08:38:36.214 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.214 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.215 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794534 s, 132 MB/s 2024-04-25T08:38:36.224 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.224 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.224 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745223 s, 141 MB/s 2024-04-25T08:38:36.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.234 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807806 s, 130 MB/s 2024-04-25T08:38:36.244 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.245 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008195 s, 128 MB/s 2024-04-25T08:38:36.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.255 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696392 s, 151 MB/s 2024-04-25T08:38:36.262 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.262 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.263 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669303 s, 157 MB/s 2024-04-25T08:38:36.272 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.273 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790671 s, 133 MB/s 2024-04-25T08:38:36.281 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.281 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.281 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666154 s, 157 MB/s 2024-04-25T08:38:36.290 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.290 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.290 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659337 s, 159 MB/s 2024-04-25T08:38:36.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:36.291 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:36.291 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:36.291 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T08:38:36.291 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:36.291 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:36.292 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:36.292 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:36.359 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.359 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.359 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732929 s, 143 MB/s 2024-04-25T08:38:36.368 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.368 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.368 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070033 s, 150 MB/s 2024-04-25T08:38:36.377 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.377 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.377 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654873 s, 160 MB/s 2024-04-25T08:38:36.385 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.386 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.386 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650661 s, 161 MB/s 2024-04-25T08:38:36.395 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.395 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.395 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727699 s, 144 MB/s 2024-04-25T08:38:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.407 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0094019 s, 112 MB/s 2024-04-25T08:38:36.420 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.420 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.420 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106785 s, 98.2 MB/s 2024-04-25T08:38:36.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.432 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835997 s, 125 MB/s 2024-04-25T08:38:36.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.444 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916772 s, 114 MB/s 2024-04-25T08:38:36.454 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.455 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.455 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868842 s, 121 MB/s 2024-04-25T08:38:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.465 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809925 s, 129 MB/s 2024-04-25T08:38:36.475 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.476 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779911 s, 134 MB/s 2024-04-25T08:38:36.486 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.486 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.486 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813459 s, 129 MB/s 2024-04-25T08:38:36.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.496 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752571 s, 139 MB/s 2024-04-25T08:38:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.505 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.506 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702584 s, 149 MB/s 2024-04-25T08:38:36.515 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.515 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.515 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729965 s, 144 MB/s 2024-04-25T08:38:36.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.528 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810454 s, 129 MB/s 2024-04-25T08:38:36.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.538 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742206 s, 141 MB/s 2024-04-25T08:38:36.548 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.548 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.548 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814643 s, 129 MB/s 2024-04-25T08:38:36.558 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.558 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.558 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761311 s, 138 MB/s 2024-04-25T08:38:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.568 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819054 s, 128 MB/s 2024-04-25T08:38:36.578 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.579 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.579 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805247 s, 130 MB/s 2024-04-25T08:38:36.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.588 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.588 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692781 s, 151 MB/s 2024-04-25T08:38:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752102 s, 139 MB/s 2024-04-25T08:38:36.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.606 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650377 s, 161 MB/s 2024-04-25T08:38:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.615 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681165 s, 154 MB/s 2024-04-25T08:38:36.624 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.624 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.625 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692662 s, 151 MB/s 2024-04-25T08:38:36.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.635 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792704 s, 132 MB/s 2024-04-25T08:38:36.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.645 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773059 s, 136 MB/s 2024-04-25T08:38:36.654 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.654 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.654 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670905 s, 156 MB/s 2024-04-25T08:38:36.663 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.663 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.663 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732221 s, 143 MB/s 2024-04-25T08:38:36.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.674 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823933 s, 127 MB/s 2024-04-25T08:38:36.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:36.675 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:36.675 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:36.675 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T08:38:36.675 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:36.675 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:36.676 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:36.676 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:36.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.743 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776174 s, 135 MB/s 2024-04-25T08:38:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.753 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701131 s, 150 MB/s 2024-04-25T08:38:36.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.762 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738358 s, 142 MB/s 2024-04-25T08:38:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.772 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823124 s, 127 MB/s 2024-04-25T08:38:36.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.781 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685619 s, 153 MB/s 2024-04-25T08:38:36.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.792 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773572 s, 136 MB/s 2024-04-25T08:38:36.801 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.801 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.801 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759262 s, 138 MB/s 2024-04-25T08:38:36.812 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.812 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.812 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853985 s, 123 MB/s 2024-04-25T08:38:36.821 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.821 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.822 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747737 s, 140 MB/s 2024-04-25T08:38:36.830 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.830 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.830 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00640475 s, 164 MB/s 2024-04-25T08:38:36.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.839 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677331 s, 155 MB/s 2024-04-25T08:38:36.850 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.850 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.850 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826138 s, 127 MB/s 2024-04-25T08:38:36.859 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.859 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.859 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691921 s, 152 MB/s 2024-04-25T08:38:36.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.869 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751884 s, 139 MB/s 2024-04-25T08:38:36.878 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.878 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.878 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724214 s, 145 MB/s 2024-04-25T08:38:36.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.888 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695831 s, 151 MB/s 2024-04-25T08:38:36.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.897 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685888 s, 153 MB/s 2024-04-25T08:38:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.906 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671388 s, 156 MB/s 2024-04-25T08:38:36.915 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.915 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.915 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740977 s, 142 MB/s 2024-04-25T08:38:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654629 s, 160 MB/s 2024-04-25T08:38:36.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.933 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655416 s, 160 MB/s 2024-04-25T08:38:36.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.943 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799335 s, 131 MB/s 2024-04-25T08:38:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720149 s, 146 MB/s 2024-04-25T08:38:36.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.961 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645685 s, 162 MB/s 2024-04-25T08:38:36.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.971 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751002 s, 140 MB/s 2024-04-25T08:38:36.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.981 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.981 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755835 s, 139 MB/s 2024-04-25T08:38:36.989 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.990 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648221 s, 162 MB/s 2024-04-25T08:38:36.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:36.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:36.998 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682449 s, 154 MB/s 2024-04-25T08:38:37.009 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.009 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.009 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845227 s, 124 MB/s 2024-04-25T08:38:37.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.018 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715632 s, 147 MB/s 2024-04-25T08:38:37.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.027 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679903 s, 154 MB/s 2024-04-25T08:38:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.037 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717581 s, 146 MB/s 2024-04-25T08:38:37.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:37.038 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:37.038 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:37.038 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T08:38:37.038 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:37.038 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:37.038 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:37.038 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:37.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.107 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854981 s, 123 MB/s 2024-04-25T08:38:37.115 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.115 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.115 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636756 s, 165 MB/s 2024-04-25T08:38:37.124 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.124 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.124 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706986 s, 148 MB/s 2024-04-25T08:38:37.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.134 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.134 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695053 s, 151 MB/s 2024-04-25T08:38:37.144 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.144 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.144 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839564 s, 125 MB/s 2024-04-25T08:38:37.154 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.154 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.154 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724618 s, 145 MB/s 2024-04-25T08:38:37.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.163 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.163 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679745 s, 154 MB/s 2024-04-25T08:38:37.172 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.172 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.172 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684929 s, 153 MB/s 2024-04-25T08:38:37.180 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.180 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.181 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064944 s, 161 MB/s 2024-04-25T08:38:37.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.191 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831147 s, 126 MB/s 2024-04-25T08:38:37.200 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.200 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.200 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659337 s, 159 MB/s 2024-04-25T08:38:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722028 s, 145 MB/s 2024-04-25T08:38:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.219 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715361 s, 147 MB/s 2024-04-25T08:38:37.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.228 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673669 s, 156 MB/s 2024-04-25T08:38:37.236 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.237 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006776 s, 155 MB/s 2024-04-25T08:38:37.246 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.246 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.246 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687018 s, 153 MB/s 2024-04-25T08:38:37.255 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.255 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.256 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743357 s, 141 MB/s 2024-04-25T08:38:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.265 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728761 s, 144 MB/s 2024-04-25T08:38:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730142 s, 144 MB/s 2024-04-25T08:38:37.285 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.285 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.285 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806829 s, 130 MB/s 2024-04-25T08:38:37.294 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.294 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.294 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727216 s, 144 MB/s 2024-04-25T08:38:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.303 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665549 s, 158 MB/s 2024-04-25T08:38:37.312 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.313 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698576 s, 150 MB/s 2024-04-25T08:38:37.322 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.322 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.322 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736704 s, 142 MB/s 2024-04-25T08:38:37.331 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.331 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.331 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068824 s, 152 MB/s 2024-04-25T08:38:37.341 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.341 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.341 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724996 s, 145 MB/s 2024-04-25T08:38:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.350 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680772 s, 154 MB/s 2024-04-25T08:38:37.361 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.361 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.361 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855612 s, 123 MB/s 2024-04-25T08:38:37.370 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.371 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728432 s, 144 MB/s 2024-04-25T08:38:37.380 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.381 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.381 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737798 s, 142 MB/s 2024-04-25T08:38:37.390 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.390 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.390 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720909 s, 145 MB/s 2024-04-25T08:38:37.399 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.401 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724304 s, 145 MB/s 2024-04-25T08:38:37.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:37.402 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:37.402 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:37.403 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T08:38:37.403 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:37.403 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:37.403 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:37.403 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:37.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.470 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750961 s, 140 MB/s 2024-04-25T08:38:37.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.479 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679852 s, 154 MB/s 2024-04-25T08:38:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007347 s, 143 MB/s 2024-04-25T08:38:37.498 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.498 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.498 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683815 s, 153 MB/s 2024-04-25T08:38:37.507 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.507 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.507 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689306 s, 152 MB/s 2024-04-25T08:38:37.517 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.517 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.517 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808463 s, 130 MB/s 2024-04-25T08:38:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850949 s, 123 MB/s 2024-04-25T08:38:37.537 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.537 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.537 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680346 s, 154 MB/s 2024-04-25T08:38:37.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.547 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080209 s, 131 MB/s 2024-04-25T08:38:37.556 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.557 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688799 s, 152 MB/s 2024-04-25T08:38:37.567 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.567 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.567 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817848 s, 128 MB/s 2024-04-25T08:38:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.578 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810286 s, 129 MB/s 2024-04-25T08:38:37.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.587 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747147 s, 140 MB/s 2024-04-25T08:38:37.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.598 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844396 s, 124 MB/s 2024-04-25T08:38:37.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.608 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744907 s, 141 MB/s 2024-04-25T08:38:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.618 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801593 s, 131 MB/s 2024-04-25T08:38:37.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.627 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653781 s, 160 MB/s 2024-04-25T08:38:37.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.636 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715478 s, 147 MB/s 2024-04-25T08:38:37.644 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.644 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.645 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648252 s, 162 MB/s 2024-04-25T08:38:37.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.655 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805108 s, 130 MB/s 2024-04-25T08:38:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.664 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750959 s, 140 MB/s 2024-04-25T08:38:37.673 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.673 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.673 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682749 s, 154 MB/s 2024-04-25T08:38:37.684 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.684 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.684 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808426 s, 130 MB/s 2024-04-25T08:38:37.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.695 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.695 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836213 s, 125 MB/s 2024-04-25T08:38:37.703 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.704 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667711 s, 157 MB/s 2024-04-25T08:38:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748984 s, 140 MB/s 2024-04-25T08:38:37.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.724 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814033 s, 129 MB/s 2024-04-25T08:38:37.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.733 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716576 s, 146 MB/s 2024-04-25T08:38:37.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.743 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655112 s, 160 MB/s 2024-04-25T08:38:37.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.752 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747764 s, 140 MB/s 2024-04-25T08:38:37.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.761 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740041 s, 142 MB/s 2024-04-25T08:38:37.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.772 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719828 s, 146 MB/s 2024-04-25T08:38:37.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:37.772 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:37.772 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:37.772 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T08:38:37.772 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:37.772 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:37.772 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:37.772 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:37.840 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.840 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.841 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738472 s, 142 MB/s 2024-04-25T08:38:37.850 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.850 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.850 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815619 s, 129 MB/s 2024-04-25T08:38:37.861 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.861 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.861 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814344 s, 129 MB/s 2024-04-25T08:38:37.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.869 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00641124 s, 164 MB/s 2024-04-25T08:38:37.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.879 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721256 s, 145 MB/s 2024-04-25T08:38:37.888 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.888 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.888 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696034 s, 151 MB/s 2024-04-25T08:38:37.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.898 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.898 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737751 s, 142 MB/s 2024-04-25T08:38:37.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.907 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678504 s, 155 MB/s 2024-04-25T08:38:37.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.917 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803114 s, 131 MB/s 2024-04-25T08:38:37.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.926 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654316 s, 160 MB/s 2024-04-25T08:38:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.936 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782523 s, 134 MB/s 2024-04-25T08:38:37.945 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.946 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.946 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757743 s, 138 MB/s 2024-04-25T08:38:37.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.955 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716567 s, 146 MB/s 2024-04-25T08:38:37.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.966 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.966 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839386 s, 125 MB/s 2024-04-25T08:38:37.974 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.974 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.975 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006575 s, 159 MB/s 2024-04-25T08:38:37.985 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.985 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.985 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826614 s, 127 MB/s 2024-04-25T08:38:37.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:37.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:37.994 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661375 s, 159 MB/s 2024-04-25T08:38:38.003 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.003 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.003 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069031 s, 152 MB/s 2024-04-25T08:38:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.012 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733091 s, 143 MB/s 2024-04-25T08:38:38.021 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.021 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.021 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662195 s, 158 MB/s 2024-04-25T08:38:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.031 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751696 s, 139 MB/s 2024-04-25T08:38:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.041 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761986 s, 138 MB/s 2024-04-25T08:38:38.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.051 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739148 s, 142 MB/s 2024-04-25T08:38:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862 s, 122 MB/s 2024-04-25T08:38:38.072 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.072 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.072 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814626 s, 129 MB/s 2024-04-25T08:38:38.081 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.081 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.081 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066314 s, 158 MB/s 2024-04-25T08:38:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.090 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670019 s, 156 MB/s 2024-04-25T08:38:38.100 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.100 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.100 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777274 s, 135 MB/s 2024-04-25T08:38:38.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.110 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806511 s, 130 MB/s 2024-04-25T08:38:38.119 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.119 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.119 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672943 s, 156 MB/s 2024-04-25T08:38:38.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.128 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675317 s, 155 MB/s 2024-04-25T08:38:38.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.138 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783187 s, 134 MB/s 2024-04-25T08:38:38.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:38.139 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:38.140 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:38.140 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T08:38:38.140 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:38.140 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:38.140 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:38.140 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:38.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.206 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647445 s, 162 MB/s 2024-04-25T08:38:38.217 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.217 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.217 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862712 s, 122 MB/s 2024-04-25T08:38:38.226 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.226 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.226 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688456 s, 152 MB/s 2024-04-25T08:38:38.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.237 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821897 s, 128 MB/s 2024-04-25T08:38:38.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.248 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863567 s, 121 MB/s 2024-04-25T08:38:38.258 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.258 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.258 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823456 s, 127 MB/s 2024-04-25T08:38:38.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.267 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682899 s, 154 MB/s 2024-04-25T08:38:38.277 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.278 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781418 s, 134 MB/s 2024-04-25T08:38:38.287 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.287 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.287 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072945 s, 144 MB/s 2024-04-25T08:38:38.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.297 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755481 s, 139 MB/s 2024-04-25T08:38:38.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.307 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809412 s, 130 MB/s 2024-04-25T08:38:38.316 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.316 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.316 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678982 s, 154 MB/s 2024-04-25T08:38:38.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.326 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731628 s, 143 MB/s 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665634 s, 158 MB/s 2024-04-25T08:38:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.344 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745435 s, 141 MB/s 2024-04-25T08:38:38.355 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.356 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.356 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910676 s, 115 MB/s 2024-04-25T08:38:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.366 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080622 s, 130 MB/s 2024-04-25T08:38:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.375 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661334 s, 159 MB/s 2024-04-25T08:38:38.383 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.384 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.384 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657782 s, 159 MB/s 2024-04-25T08:38:38.392 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.393 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671262 s, 156 MB/s 2024-04-25T08:38:38.402 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.402 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.402 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725286 s, 145 MB/s 2024-04-25T08:38:38.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.412 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080888 s, 130 MB/s 2024-04-25T08:38:38.420 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.421 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.421 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00633368 s, 166 MB/s 2024-04-25T08:38:38.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.431 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810949 s, 129 MB/s 2024-04-25T08:38:38.439 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.440 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.440 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661975 s, 158 MB/s 2024-04-25T08:38:38.450 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.450 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.450 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078691 s, 133 MB/s 2024-04-25T08:38:38.459 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.460 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.460 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748743 s, 140 MB/s 2024-04-25T08:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835749 s, 125 MB/s 2024-04-25T08:38:38.480 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.480 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.480 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734835 s, 143 MB/s 2024-04-25T08:38:38.490 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.490 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.490 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806101 s, 130 MB/s 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.500 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737766 s, 142 MB/s 2024-04-25T08:38:38.509 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.510 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761354 s, 138 MB/s 2024-04-25T08:38:38.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:38.511 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:38.511 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:38.511 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T08:38:38.511 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:38.511 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:38.511 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:38.511 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:38.579 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747584 s, 140 MB/s 2024-04-25T08:38:38.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.589 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074217 s, 141 MB/s 2024-04-25T08:38:38.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.599 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668594 s, 157 MB/s 2024-04-25T08:38:38.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.609 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737271 s, 142 MB/s 2024-04-25T08:38:38.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.618 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669565 s, 157 MB/s 2024-04-25T08:38:38.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.628 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742001 s, 141 MB/s 2024-04-25T08:38:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662895 s, 158 MB/s 2024-04-25T08:38:38.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.647 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067116 s, 156 MB/s 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678355 s, 155 MB/s 2024-04-25T08:38:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645263 s, 163 MB/s 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.676 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827135 s, 127 MB/s 2024-04-25T08:38:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.686 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735694 s, 143 MB/s 2024-04-25T08:38:38.695 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.695 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.695 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664529 s, 158 MB/s 2024-04-25T08:38:38.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.705 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727034 s, 144 MB/s 2024-04-25T08:38:38.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.715 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695442 s, 151 MB/s 2024-04-25T08:38:38.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.725 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746561 s, 140 MB/s 2024-04-25T08:38:38.736 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.736 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.736 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706531 s, 148 MB/s 2024-04-25T08:38:38.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.744 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695271 s, 151 MB/s 2024-04-25T08:38:38.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.754 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679414 s, 154 MB/s 2024-04-25T08:38:38.763 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.764 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.764 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722245 s, 145 MB/s 2024-04-25T08:38:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652968 s, 161 MB/s 2024-04-25T08:38:38.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.782 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672783 s, 156 MB/s 2024-04-25T08:38:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.792 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746017 s, 141 MB/s 2024-04-25T08:38:38.802 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.802 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.802 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732049 s, 143 MB/s 2024-04-25T08:38:38.811 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.811 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.812 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727482 s, 144 MB/s 2024-04-25T08:38:38.822 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.822 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.822 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823852 s, 127 MB/s 2024-04-25T08:38:38.832 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.832 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.832 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700461 s, 150 MB/s 2024-04-25T08:38:38.842 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.842 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.842 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712796 s, 147 MB/s 2024-04-25T08:38:38.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.853 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845213 s, 124 MB/s 2024-04-25T08:38:38.861 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675975 s, 155 MB/s 2024-04-25T08:38:38.871 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.871 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.871 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676922 s, 155 MB/s 2024-04-25T08:38:38.881 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.882 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797663 s, 131 MB/s 2024-04-25T08:38:38.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:38.883 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:38.883 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:38.883 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T08:38:38.883 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:38.883 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:38.883 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:38.883 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:38.951 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.952 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.952 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659754 s, 159 MB/s 2024-04-25T08:38:38.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.962 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.962 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745002 s, 141 MB/s 2024-04-25T08:38:38.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.971 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675432 s, 155 MB/s 2024-04-25T08:38:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725634 s, 145 MB/s 2024-04-25T08:38:38.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:38.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:38.991 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807883 s, 130 MB/s 2024-04-25T08:38:39.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.001 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675038 s, 155 MB/s 2024-04-25T08:38:39.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.011 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746413 s, 140 MB/s 2024-04-25T08:38:39.021 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.022 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833859 s, 126 MB/s 2024-04-25T08:38:39.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.032 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804126 s, 130 MB/s 2024-04-25T08:38:39.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.042 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.042 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663432 s, 158 MB/s 2024-04-25T08:38:39.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.051 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671715 s, 156 MB/s 2024-04-25T08:38:39.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.061 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073671 s, 142 MB/s 2024-04-25T08:38:39.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.070 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068605 s, 153 MB/s 2024-04-25T08:38:39.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.080 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711712 s, 147 MB/s 2024-04-25T08:38:39.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.091 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804785 s, 130 MB/s 2024-04-25T08:38:39.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.101 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079984 s, 131 MB/s 2024-04-25T08:38:39.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.110 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646515 s, 162 MB/s 2024-04-25T08:38:39.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.120 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706674 s, 148 MB/s 2024-04-25T08:38:39.129 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.129 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.129 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681389 s, 154 MB/s 2024-04-25T08:38:39.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.139 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742117 s, 141 MB/s 2024-04-25T08:38:39.150 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.150 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.150 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800213 s, 131 MB/s 2024-04-25T08:38:39.160 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.160 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.160 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738021 s, 142 MB/s 2024-04-25T08:38:39.169 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.169 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.169 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069116 s, 152 MB/s 2024-04-25T08:38:39.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.179 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681139 s, 154 MB/s 2024-04-25T08:38:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.188 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737529 s, 142 MB/s 2024-04-25T08:38:39.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.198 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713375 s, 147 MB/s 2024-04-25T08:38:39.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.208 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705667 s, 149 MB/s 2024-04-25T08:38:39.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.219 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809082 s, 130 MB/s 2024-04-25T08:38:39.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.229 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.229 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754083 s, 139 MB/s 2024-04-25T08:38:39.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.238 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680899 s, 154 MB/s 2024-04-25T08:38:39.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.247 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651302 s, 161 MB/s 2024-04-25T08:38:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.257 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666028 s, 157 MB/s 2024-04-25T08:38:39.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:39.258 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:39.258 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:39.258 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T08:38:39.258 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:39.258 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:39.258 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:39.258 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.326 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00644245 s, 163 MB/s 2024-04-25T08:38:39.334 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.334 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.334 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650656 s, 161 MB/s 2024-04-25T08:38:39.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.344 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730018 s, 144 MB/s 2024-04-25T08:38:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.353 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706579 s, 148 MB/s 2024-04-25T08:38:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.363 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721701 s, 145 MB/s 2024-04-25T08:38:39.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.374 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.374 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841949 s, 125 MB/s 2024-04-25T08:38:39.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.383 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669109 s, 157 MB/s 2024-04-25T08:38:39.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.391 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650632 s, 161 MB/s 2024-04-25T08:38:39.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.402 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.402 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794177 s, 132 MB/s 2024-04-25T08:38:39.410 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.411 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655644 s, 160 MB/s 2024-04-25T08:38:39.420 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.420 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.420 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077193 s, 136 MB/s 2024-04-25T08:38:39.429 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.429 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.429 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652599 s, 161 MB/s 2024-04-25T08:38:39.438 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.438 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.439 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728678 s, 144 MB/s 2024-04-25T08:38:39.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.449 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.449 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682402 s, 154 MB/s 2024-04-25T08:38:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.457 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690583 s, 152 MB/s 2024-04-25T08:38:39.466 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.466 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.466 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682017 s, 154 MB/s 2024-04-25T08:38:39.475 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.475 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.475 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687499 s, 153 MB/s 2024-04-25T08:38:39.484 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.484 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.484 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068097 s, 154 MB/s 2024-04-25T08:38:39.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.494 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777748 s, 135 MB/s 2024-04-25T08:38:39.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.503 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686665 s, 153 MB/s 2024-04-25T08:38:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.512 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681867 s, 154 MB/s 2024-04-25T08:38:39.521 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.521 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.522 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677322 s, 155 MB/s 2024-04-25T08:38:39.560 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.560 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.560 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777693 s, 135 MB/s 2024-04-25T08:38:39.569 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.569 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.569 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723 s, 145 MB/s 2024-04-25T08:38:39.578 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.579 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.579 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719426 s, 146 MB/s 2024-04-25T08:38:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.588 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748814 s, 140 MB/s 2024-04-25T08:38:39.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.597 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667119 s, 157 MB/s 2024-04-25T08:38:39.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.607 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736462 s, 142 MB/s 2024-04-25T08:38:39.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.616 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726503 s, 144 MB/s 2024-04-25T08:38:39.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.626 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760831 s, 138 MB/s 2024-04-25T08:38:39.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.635 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695749 s, 151 MB/s 2024-04-25T08:38:39.644 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.644 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.644 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684792 s, 153 MB/s 2024-04-25T08:38:39.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:39.645 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:39.645 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:39.645 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T08:38:39.645 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:39.645 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:39.645 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:39.645 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:39.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.714 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845721 s, 124 MB/s 2024-04-25T08:38:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.724 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753789 s, 139 MB/s 2024-04-25T08:38:39.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.733 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679546 s, 154 MB/s 2024-04-25T08:38:39.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.742 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066989 s, 157 MB/s 2024-04-25T08:38:39.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.751 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726074 s, 144 MB/s 2024-04-25T08:38:39.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.760 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669497 s, 157 MB/s 2024-04-25T08:38:39.770 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.770 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.770 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765612 s, 137 MB/s 2024-04-25T08:38:39.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.781 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882521 s, 119 MB/s 2024-04-25T08:38:39.790 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.790 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.790 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685159 s, 153 MB/s 2024-04-25T08:38:39.800 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.800 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.800 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716769 s, 146 MB/s 2024-04-25T08:38:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748311 s, 140 MB/s 2024-04-25T08:38:39.818 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.818 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.818 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065658 s, 160 MB/s 2024-04-25T08:38:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.828 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802839 s, 131 MB/s 2024-04-25T08:38:39.837 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.837 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.837 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670406 s, 156 MB/s 2024-04-25T08:38:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.846 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651961 s, 161 MB/s 2024-04-25T08:38:39.856 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.856 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.857 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805962 s, 130 MB/s 2024-04-25T08:38:39.866 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.866 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.866 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072826 s, 144 MB/s 2024-04-25T08:38:39.876 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.876 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.876 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818727 s, 128 MB/s 2024-04-25T08:38:39.885 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.885 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.885 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665503 s, 158 MB/s 2024-04-25T08:38:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.896 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816043 s, 128 MB/s 2024-04-25T08:38:39.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074791 s, 140 MB/s 2024-04-25T08:38:39.916 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.916 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.916 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803838 s, 130 MB/s 2024-04-25T08:38:39.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.926 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801935 s, 131 MB/s 2024-04-25T08:38:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.935 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00636654 s, 165 MB/s 2024-04-25T08:38:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006846 s, 153 MB/s 2024-04-25T08:38:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.954 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746474 s, 140 MB/s 2024-04-25T08:38:39.962 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.963 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.963 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671565 s, 156 MB/s 2024-04-25T08:38:39.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.973 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.973 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766109 s, 137 MB/s 2024-04-25T08:38:39.981 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.982 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681339 s, 154 MB/s 2024-04-25T08:38:39.992 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:39.993 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:39.993 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862565 s, 122 MB/s 2024-04-25T08:38:40.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.002 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716003 s, 146 MB/s 2024-04-25T08:38:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.013 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815983 s, 129 MB/s 2024-04-25T08:38:40.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:40.014 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:40.014 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:40.014 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T08:38:40.014 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:40.014 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:40.014 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:40.014 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:40.081 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.081 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.082 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717147 s, 146 MB/s 2024-04-25T08:38:40.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.090 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662819 s, 158 MB/s 2024-04-25T08:38:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080301 s, 131 MB/s 2024-04-25T08:38:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741265 s, 141 MB/s 2024-04-25T08:38:40.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.120 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081129 s, 129 MB/s 2024-04-25T08:38:40.131 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.131 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.131 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827513 s, 127 MB/s 2024-04-25T08:38:40.141 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.141 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.141 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816592 s, 128 MB/s 2024-04-25T08:38:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.152 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830738 s, 126 MB/s 2024-04-25T08:38:40.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.162 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788224 s, 133 MB/s 2024-04-25T08:38:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.171 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713881 s, 147 MB/s 2024-04-25T08:38:40.181 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.181 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.181 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708859 s, 148 MB/s 2024-04-25T08:38:40.190 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.190 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.191 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745496 s, 141 MB/s 2024-04-25T08:38:40.201 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.201 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.201 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836358 s, 125 MB/s 2024-04-25T08:38:40.210 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.210 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.210 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650621 s, 161 MB/s 2024-04-25T08:38:40.220 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.220 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.220 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743526 s, 141 MB/s 2024-04-25T08:38:40.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.229 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.229 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066226 s, 158 MB/s 2024-04-25T08:38:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807141 s, 130 MB/s 2024-04-25T08:38:40.249 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.249 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.249 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818628 s, 128 MB/s 2024-04-25T08:38:40.259 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.259 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.259 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746782 s, 140 MB/s 2024-04-25T08:38:40.269 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.270 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.270 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849212 s, 123 MB/s 2024-04-25T08:38:40.279 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.279 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.279 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728843 s, 144 MB/s 2024-04-25T08:38:40.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.289 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718951 s, 146 MB/s 2024-04-25T08:38:40.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.298 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702885 s, 149 MB/s 2024-04-25T08:38:40.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.308 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783185 s, 134 MB/s 2024-04-25T08:38:40.318 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.318 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.318 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797833 s, 131 MB/s 2024-04-25T08:38:40.327 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.327 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.327 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718743 s, 146 MB/s 2024-04-25T08:38:40.336 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.336 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.336 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073556 s, 143 MB/s 2024-04-25T08:38:40.346 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.347 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.347 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796041 s, 132 MB/s 2024-04-25T08:38:40.357 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.357 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.357 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796593 s, 132 MB/s 2024-04-25T08:38:40.366 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.366 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.366 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705026 s, 149 MB/s 2024-04-25T08:38:40.375 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.376 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.376 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741972 s, 141 MB/s 2024-04-25T08:38:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.386 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811124 s, 129 MB/s 2024-04-25T08:38:40.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:40.387 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:40.387 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:40.387 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T08:38:40.387 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:40.387 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:40.387 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:40.388 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.454 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696348 s, 151 MB/s 2024-04-25T08:38:40.464 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.464 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.464 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715943 s, 146 MB/s 2024-04-25T08:38:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.473 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710691 s, 148 MB/s 2024-04-25T08:38:40.482 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.482 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.482 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662115 s, 158 MB/s 2024-04-25T08:38:40.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.491 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729864 s, 144 MB/s 2024-04-25T08:38:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.500 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00638709 s, 164 MB/s 2024-04-25T08:38:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.510 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804093 s, 130 MB/s 2024-04-25T08:38:40.519 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.520 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684538 s, 153 MB/s 2024-04-25T08:38:40.532 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.532 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.532 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103778 s, 101 MB/s 2024-04-25T08:38:40.565 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.565 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.565 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733383 s, 143 MB/s 2024-04-25T08:38:40.573 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.574 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.574 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678293 s, 155 MB/s 2024-04-25T08:38:40.584 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.584 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.584 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817259 s, 128 MB/s 2024-04-25T08:38:40.592 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.592 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.593 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639828 s, 164 MB/s 2024-04-25T08:38:40.602 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.602 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.602 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709544 s, 148 MB/s 2024-04-25T08:38:40.611 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.611 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.611 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739925 s, 142 MB/s 2024-04-25T08:38:40.621 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.622 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.622 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811951 s, 129 MB/s 2024-04-25T08:38:40.631 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.631 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.631 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073723 s, 142 MB/s 2024-04-25T08:38:40.640 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.641 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.641 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711637 s, 147 MB/s 2024-04-25T08:38:40.650 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.650 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.650 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729365 s, 144 MB/s 2024-04-25T08:38:40.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.659 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668104 s, 157 MB/s 2024-04-25T08:38:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.668 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740498 s, 142 MB/s 2024-04-25T08:38:40.677 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.677 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.677 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649365 s, 161 MB/s 2024-04-25T08:38:40.686 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.686 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.686 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707784 s, 148 MB/s 2024-04-25T08:38:40.696 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.696 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.696 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070292 s, 149 MB/s 2024-04-25T08:38:40.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.705 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674552 s, 155 MB/s 2024-04-25T08:38:40.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.714 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690207 s, 152 MB/s 2024-04-25T08:38:40.723 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.723 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.723 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714847 s, 147 MB/s 2024-04-25T08:38:40.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.733 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705281 s, 149 MB/s 2024-04-25T08:38:40.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.743 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736371 s, 142 MB/s 2024-04-25T08:38:40.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.752 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667184 s, 157 MB/s 2024-04-25T08:38:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694313 s, 151 MB/s 2024-04-25T08:38:40.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.771 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071844 s, 146 MB/s 2024-04-25T08:38:40.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:40.772 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:40.772 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:40.772 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T08:38:40.772 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:40.772 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:40.772 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:40.772 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:40.841 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.841 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.841 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747436 s, 140 MB/s 2024-04-25T08:38:40.850 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.851 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.851 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712244 s, 147 MB/s 2024-04-25T08:38:40.859 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.860 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.860 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650022 s, 161 MB/s 2024-04-25T08:38:40.870 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.870 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.871 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820319 s, 128 MB/s 2024-04-25T08:38:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.880 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743124 s, 141 MB/s 2024-04-25T08:38:40.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.891 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759196 s, 138 MB/s 2024-04-25T08:38:40.900 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.901 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722045 s, 145 MB/s 2024-04-25T08:38:40.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.910 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708457 s, 148 MB/s 2024-04-25T08:38:40.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.921 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793746 s, 132 MB/s 2024-04-25T08:38:40.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.931 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725556 s, 145 MB/s 2024-04-25T08:38:40.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.941 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811072 s, 129 MB/s 2024-04-25T08:38:40.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.950 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700938 s, 150 MB/s 2024-04-25T08:38:40.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.960 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696345 s, 151 MB/s 2024-04-25T08:38:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.970 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007498 s, 140 MB/s 2024-04-25T08:38:40.979 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.979 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.979 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064939 s, 161 MB/s 2024-04-25T08:38:40.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.990 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842685 s, 124 MB/s 2024-04-25T08:38:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710694 s, 148 MB/s 2024-04-25T08:38:41.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.009 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654054 s, 160 MB/s 2024-04-25T08:38:41.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.018 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705707 s, 149 MB/s 2024-04-25T08:38:41.029 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.029 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.029 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842307 s, 124 MB/s 2024-04-25T08:38:41.039 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.039 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.039 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710861 s, 148 MB/s 2024-04-25T08:38:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.049 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717184 s, 146 MB/s 2024-04-25T08:38:41.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.058 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714812 s, 147 MB/s 2024-04-25T08:38:41.067 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.067 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.067 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708396 s, 148 MB/s 2024-04-25T08:38:41.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.077 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675067 s, 155 MB/s 2024-04-25T08:38:41.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.087 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765087 s, 137 MB/s 2024-04-25T08:38:41.097 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.097 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.097 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725068 s, 145 MB/s 2024-04-25T08:38:41.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.108 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.108 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795358 s, 132 MB/s 2024-04-25T08:38:41.117 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.117 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.117 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673108 s, 156 MB/s 2024-04-25T08:38:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00647231 s, 162 MB/s 2024-04-25T08:38:41.135 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.135 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.135 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657287 s, 160 MB/s 2024-04-25T08:38:41.145 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.145 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.145 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660643 s, 159 MB/s 2024-04-25T08:38:41.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:41.146 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:41.146 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:41.146 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T08:38:41.146 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:41.146 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:41.146 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:41.146 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:41.214 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.215 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736885 s, 142 MB/s 2024-04-25T08:38:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.224 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.224 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064627 s, 162 MB/s 2024-04-25T08:38:41.232 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.232 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.233 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675817 s, 155 MB/s 2024-04-25T08:38:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705019 s, 149 MB/s 2024-04-25T08:38:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673974 s, 156 MB/s 2024-04-25T08:38:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819069 s, 128 MB/s 2024-04-25T08:38:41.271 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658906 s, 159 MB/s 2024-04-25T08:38:41.282 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.282 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.282 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775808 s, 135 MB/s 2024-04-25T08:38:41.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.292 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798401 s, 131 MB/s 2024-04-25T08:38:41.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.303 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818363 s, 128 MB/s 2024-04-25T08:38:41.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.314 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829426 s, 126 MB/s 2024-04-25T08:38:41.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.323 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705819 s, 149 MB/s 2024-04-25T08:38:41.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.332 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656466 s, 160 MB/s 2024-04-25T08:38:41.343 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.343 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.343 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807928 s, 130 MB/s 2024-04-25T08:38:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.353 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688901 s, 152 MB/s 2024-04-25T08:38:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.362 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648899 s, 162 MB/s 2024-04-25T08:38:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.372 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707197 s, 148 MB/s 2024-04-25T08:38:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.381 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725734 s, 144 MB/s 2024-04-25T08:38:41.392 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.392 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.392 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746582 s, 140 MB/s 2024-04-25T08:38:41.402 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.402 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.402 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753256 s, 139 MB/s 2024-04-25T08:38:41.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.413 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.413 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719163 s, 146 MB/s 2024-04-25T08:38:41.423 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.423 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.423 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776267 s, 135 MB/s 2024-04-25T08:38:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.435 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101325 s, 103 MB/s 2024-04-25T08:38:41.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.447 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868461 s, 121 MB/s 2024-04-25T08:38:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.457 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765026 s, 137 MB/s 2024-04-25T08:38:41.469 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.469 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.469 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895174 s, 117 MB/s 2024-04-25T08:38:41.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.480 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.480 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807384 s, 130 MB/s 2024-04-25T08:38:41.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.489 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748347 s, 140 MB/s 2024-04-25T08:38:41.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.499 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738243 s, 142 MB/s 2024-04-25T08:38:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.511 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089356 s, 117 MB/s 2024-04-25T08:38:41.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.522 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843834 s, 124 MB/s 2024-04-25T08:38:41.541 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.542 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169547 s, 61.8 MB/s 2024-04-25T08:38:41.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:41.543 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:41.543 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:41.543 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T08:38:41.543 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:41.543 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:41.543 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:41.543 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:41.624 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.625 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.625 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739895 s, 142 MB/s 2024-04-25T08:38:41.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.635 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813375 s, 129 MB/s 2024-04-25T08:38:41.644 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.644 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.644 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746649 s, 140 MB/s 2024-04-25T08:38:41.653 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.653 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.653 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066542 s, 158 MB/s 2024-04-25T08:38:41.662 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.663 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726454 s, 144 MB/s 2024-04-25T08:38:41.673 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.673 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.673 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082688 s, 127 MB/s 2024-04-25T08:38:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.683 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808572 s, 130 MB/s 2024-04-25T08:38:41.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.694 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839535 s, 125 MB/s 2024-04-25T08:38:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.704 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791797 s, 132 MB/s 2024-04-25T08:38:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.715 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846999 s, 124 MB/s 2024-04-25T08:38:41.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.724 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718752 s, 146 MB/s 2024-04-25T08:38:41.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.734 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688592 s, 152 MB/s 2024-04-25T08:38:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.743 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759784 s, 138 MB/s 2024-04-25T08:38:41.753 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.753 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.753 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743086 s, 141 MB/s 2024-04-25T08:38:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724743 s, 145 MB/s 2024-04-25T08:38:41.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.772 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773985 s, 135 MB/s 2024-04-25T08:38:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.781 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665649 s, 158 MB/s 2024-04-25T08:38:41.790 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.790 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.790 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649912 s, 161 MB/s 2024-04-25T08:38:41.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.799 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652055 s, 161 MB/s 2024-04-25T08:38:41.808 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.808 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.809 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727259 s, 144 MB/s 2024-04-25T08:38:41.817 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.817 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.817 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701363 s, 150 MB/s 2024-04-25T08:38:41.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.828 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810833 s, 129 MB/s 2024-04-25T08:38:41.837 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.837 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.837 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687667 s, 152 MB/s 2024-04-25T08:38:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.847 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808542 s, 130 MB/s 2024-04-25T08:38:41.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.857 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742523 s, 141 MB/s 2024-04-25T08:38:41.867 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.867 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.867 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695759 s, 151 MB/s 2024-04-25T08:38:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840594 s, 125 MB/s 2024-04-25T08:38:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665622 s, 158 MB/s 2024-04-25T08:38:41.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.895 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717776 s, 146 MB/s 2024-04-25T08:38:41.904 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.904 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.904 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672883 s, 156 MB/s 2024-04-25T08:38:41.914 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.914 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.914 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765934 s, 137 MB/s 2024-04-25T08:38:41.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.923 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662974 s, 158 MB/s 2024-04-25T08:38:41.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:41.924 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:41.924 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:41.924 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T08:38:41.924 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:41.924 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:41.924 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:41.924 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:41.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:41.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:41.991 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676013 s, 155 MB/s 2024-04-25T08:38:42.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.002 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887357 s, 118 MB/s 2024-04-25T08:38:42.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.013 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802945 s, 131 MB/s 2024-04-25T08:38:42.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.022 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687031 s, 153 MB/s 2024-04-25T08:38:42.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.031 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684536 s, 153 MB/s 2024-04-25T08:38:42.039 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.040 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.040 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655705 s, 160 MB/s 2024-04-25T08:38:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.050 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850172 s, 123 MB/s 2024-04-25T08:38:42.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.060 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704467 s, 149 MB/s 2024-04-25T08:38:42.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.070 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796158 s, 132 MB/s 2024-04-25T08:38:42.079 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.079 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.079 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710118 s, 148 MB/s 2024-04-25T08:38:42.089 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.089 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.089 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724139 s, 145 MB/s 2024-04-25T08:38:42.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.100 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.100 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867616 s, 121 MB/s 2024-04-25T08:38:42.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.109 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724108 s, 145 MB/s 2024-04-25T08:38:42.119 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.119 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.119 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748991 s, 140 MB/s 2024-04-25T08:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.129 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791803 s, 132 MB/s 2024-04-25T08:38:42.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.138 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722432 s, 145 MB/s 2024-04-25T08:38:42.147 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701574 s, 149 MB/s 2024-04-25T08:38:42.157 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.157 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.157 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752566 s, 139 MB/s 2024-04-25T08:38:42.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.168 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817509 s, 128 MB/s 2024-04-25T08:38:42.176 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.176 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.177 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663716 s, 158 MB/s 2024-04-25T08:38:42.186 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.186 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.186 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742717 s, 141 MB/s 2024-04-25T08:38:42.195 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.196 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721005 s, 145 MB/s 2024-04-25T08:38:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.205 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714874 s, 147 MB/s 2024-04-25T08:38:42.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.215 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801415 s, 131 MB/s 2024-04-25T08:38:42.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.226 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816303 s, 128 MB/s 2024-04-25T08:38:42.235 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.235 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.236 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776236 s, 135 MB/s 2024-04-25T08:38:42.244 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.245 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692244 s, 151 MB/s 2024-04-25T08:38:42.253 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.254 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673587 s, 156 MB/s 2024-04-25T08:38:42.262 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.262 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.262 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669143 s, 157 MB/s 2024-04-25T08:38:42.271 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.271 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.271 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661053 s, 159 MB/s 2024-04-25T08:38:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.281 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734779 s, 143 MB/s 2024-04-25T08:38:42.291 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.291 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.291 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806043 s, 130 MB/s 2024-04-25T08:38:42.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:42.293 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:42.293 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:42.293 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T08:38:42.293 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:42.293 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:42.293 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:42.293 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:42.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.360 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651651 s, 161 MB/s 2024-04-25T08:38:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.370 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.370 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730111 s, 144 MB/s 2024-04-25T08:38:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.379 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751723 s, 139 MB/s 2024-04-25T08:38:42.388 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.388 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.388 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709217 s, 148 MB/s 2024-04-25T08:38:42.397 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.397 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.397 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664441 s, 158 MB/s 2024-04-25T08:38:42.407 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.407 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.407 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755402 s, 139 MB/s 2024-04-25T08:38:42.416 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.416 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.417 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726715 s, 144 MB/s 2024-04-25T08:38:42.426 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.426 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.426 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754636 s, 139 MB/s 2024-04-25T08:38:42.435 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.435 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.435 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710964 s, 147 MB/s 2024-04-25T08:38:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.444 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679797 s, 154 MB/s 2024-04-25T08:38:42.455 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.455 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.455 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826857 s, 127 MB/s 2024-04-25T08:38:42.464 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.465 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.465 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074116 s, 141 MB/s 2024-04-25T08:38:42.474 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.474 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.474 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726001 s, 144 MB/s 2024-04-25T08:38:42.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.485 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823757 s, 127 MB/s 2024-04-25T08:38:42.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.494 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656103 s, 160 MB/s 2024-04-25T08:38:42.504 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.504 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.504 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841001 s, 125 MB/s 2024-04-25T08:38:42.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.514 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783923 s, 134 MB/s 2024-04-25T08:38:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853298 s, 123 MB/s 2024-04-25T08:38:42.535 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.535 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.535 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705668 s, 149 MB/s 2024-04-25T08:38:42.553 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.553 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.553 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766461 s, 137 MB/s 2024-04-25T08:38:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.576 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745948 s, 141 MB/s 2024-04-25T08:38:42.584 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.584 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.584 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654092 s, 160 MB/s 2024-04-25T08:38:42.594 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.594 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.594 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745278 s, 141 MB/s 2024-04-25T08:38:42.604 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.604 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.604 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828974 s, 126 MB/s 2024-04-25T08:38:42.614 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.615 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.615 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808209 s, 130 MB/s 2024-04-25T08:38:42.625 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.625 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.625 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817988 s, 128 MB/s 2024-04-25T08:38:42.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.635 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815352 s, 129 MB/s 2024-04-25T08:38:42.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.646 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783792 s, 134 MB/s 2024-04-25T08:38:42.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.656 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803011 s, 131 MB/s 2024-04-25T08:38:42.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.665 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680191 s, 154 MB/s 2024-04-25T08:38:42.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.674 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692593 s, 151 MB/s 2024-04-25T08:38:42.684 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.684 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.684 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775467 s, 135 MB/s 2024-04-25T08:38:42.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:42.685 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:42.685 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:42.686 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T08:38:42.686 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:42.686 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:42.686 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:42.686 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:42.756 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.756 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.756 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854595 s, 123 MB/s 2024-04-25T08:38:42.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.766 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754351 s, 139 MB/s 2024-04-25T08:38:42.774 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.774 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.774 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672107 s, 156 MB/s 2024-04-25T08:38:42.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.784 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732855 s, 143 MB/s 2024-04-25T08:38:42.794 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.794 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.794 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753169 s, 139 MB/s 2024-04-25T08:38:42.804 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.804 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.804 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797863 s, 131 MB/s 2024-04-25T08:38:42.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.814 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803787 s, 130 MB/s 2024-04-25T08:38:42.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.823 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677773 s, 155 MB/s 2024-04-25T08:38:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810841 s, 129 MB/s 2024-04-25T08:38:42.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.843 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073086 s, 143 MB/s 2024-04-25T08:38:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.852 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065213 s, 161 MB/s 2024-04-25T08:38:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802669 s, 131 MB/s 2024-04-25T08:38:42.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.872 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797007 s, 132 MB/s 2024-04-25T08:38:42.881 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.882 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721189 s, 145 MB/s 2024-04-25T08:38:42.890 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.891 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675568 s, 155 MB/s 2024-04-25T08:38:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.901 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810399 s, 129 MB/s 2024-04-25T08:38:42.909 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.910 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653214 s, 161 MB/s 2024-04-25T08:38:42.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.920 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799382 s, 131 MB/s 2024-04-25T08:38:42.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.930 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802897 s, 131 MB/s 2024-04-25T08:38:42.939 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.940 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717393 s, 146 MB/s 2024-04-25T08:38:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082814 s, 127 MB/s 2024-04-25T08:38:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682894 s, 154 MB/s 2024-04-25T08:38:42.968 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.968 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.968 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678465 s, 155 MB/s 2024-04-25T08:38:42.979 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.979 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.979 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084873 s, 124 MB/s 2024-04-25T08:38:42.989 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.989 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.989 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804467 s, 130 MB/s 2024-04-25T08:38:42.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:42.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:42.998 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694295 s, 151 MB/s 2024-04-25T08:38:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.007 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068637 s, 153 MB/s 2024-04-25T08:38:43.016 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.017 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.017 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694283 s, 151 MB/s 2024-04-25T08:38:43.025 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.025 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.025 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657287 s, 160 MB/s 2024-04-25T08:38:43.035 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.035 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.035 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072805 s, 144 MB/s 2024-04-25T08:38:43.045 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.046 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.046 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817573 s, 128 MB/s 2024-04-25T08:38:43.055 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.055 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.055 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071596 s, 146 MB/s 2024-04-25T08:38:43.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:43.056 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:43.056 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:43.056 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T08:38:43.056 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:43.056 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:43.056 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:43.056 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:43.122 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.122 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.122 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713767 s, 147 MB/s 2024-04-25T08:38:43.132 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.132 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.132 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732229 s, 143 MB/s 2024-04-25T08:38:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721675 s, 145 MB/s 2024-04-25T08:38:43.151 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.151 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.151 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743771 s, 141 MB/s 2024-04-25T08:38:43.161 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.161 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.161 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816132 s, 128 MB/s 2024-04-25T08:38:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671353 s, 156 MB/s 2024-04-25T08:38:43.180 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.180 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.180 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800668 s, 131 MB/s 2024-04-25T08:38:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635154 s, 165 MB/s 2024-04-25T08:38:43.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.198 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685978 s, 153 MB/s 2024-04-25T08:38:43.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.207 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.207 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671226 s, 156 MB/s 2024-04-25T08:38:43.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.216 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694548 s, 151 MB/s 2024-04-25T08:38:43.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.226 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741231 s, 141 MB/s 2024-04-25T08:38:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720582 s, 146 MB/s 2024-04-25T08:38:43.246 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.246 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.246 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860404 s, 122 MB/s 2024-04-25T08:38:43.255 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.255 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.255 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654743 s, 160 MB/s 2024-04-25T08:38:43.264 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.264 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.264 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722914 s, 145 MB/s 2024-04-25T08:38:43.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.273 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672621 s, 156 MB/s 2024-04-25T08:38:43.282 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.283 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758328 s, 138 MB/s 2024-04-25T08:38:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.293 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785525 s, 133 MB/s 2024-04-25T08:38:43.301 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.301 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.301 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661431 s, 159 MB/s 2024-04-25T08:38:43.310 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.310 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.311 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680916 s, 154 MB/s 2024-04-25T08:38:43.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.319 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682795 s, 154 MB/s 2024-04-25T08:38:43.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.330 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820363 s, 128 MB/s 2024-04-25T08:38:43.338 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.339 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.339 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659409 s, 159 MB/s 2024-04-25T08:38:43.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.349 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782472 s, 134 MB/s 2024-04-25T08:38:43.358 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.358 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.358 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684326 s, 153 MB/s 2024-04-25T08:38:43.367 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.367 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.367 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722644 s, 145 MB/s 2024-04-25T08:38:43.376 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.376 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.376 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667515 s, 157 MB/s 2024-04-25T08:38:43.385 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.385 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.385 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698653 s, 150 MB/s 2024-04-25T08:38:43.395 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.395 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.395 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739704 s, 142 MB/s 2024-04-25T08:38:43.404 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.404 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.404 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704145 s, 149 MB/s 2024-04-25T08:38:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.414 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082451 s, 127 MB/s 2024-04-25T08:38:43.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:43.415 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:43.415 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:43.416 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T08:38:43.416 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:43.416 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:43.416 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:43.416 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.484 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832983 s, 126 MB/s 2024-04-25T08:38:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.494 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760442 s, 138 MB/s 2024-04-25T08:38:43.504 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.504 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.504 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082894 s, 126 MB/s 2024-04-25T08:38:43.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.514 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724459 s, 145 MB/s 2024-04-25T08:38:43.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.523 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680798 s, 154 MB/s 2024-04-25T08:38:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.532 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733258 s, 143 MB/s 2024-04-25T08:38:43.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.542 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814765 s, 129 MB/s 2024-04-25T08:38:43.556 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.556 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.556 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116432 s, 90.1 MB/s 2024-04-25T08:38:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.587 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848547 s, 124 MB/s 2024-04-25T08:38:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715833 s, 146 MB/s 2024-04-25T08:38:43.605 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.606 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709884 s, 148 MB/s 2024-04-25T08:38:43.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.616 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790809 s, 133 MB/s 2024-04-25T08:38:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.626 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831707 s, 126 MB/s 2024-04-25T08:38:43.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.635 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660121 s, 159 MB/s 2024-04-25T08:38:43.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.645 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814807 s, 129 MB/s 2024-04-25T08:38:43.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.655 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712762 s, 147 MB/s 2024-04-25T08:38:43.664 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.664 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.664 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669624 s, 157 MB/s 2024-04-25T08:38:43.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083631 s, 125 MB/s 2024-04-25T08:38:43.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.683 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006661 s, 157 MB/s 2024-04-25T08:38:43.693 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.694 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802585 s, 131 MB/s 2024-04-25T08:38:43.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.704 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080642 s, 130 MB/s 2024-04-25T08:38:43.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.713 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683682 s, 153 MB/s 2024-04-25T08:38:43.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.724 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837342 s, 125 MB/s 2024-04-25T08:38:43.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.734 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073777 s, 142 MB/s 2024-04-25T08:38:43.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.743 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749665 s, 140 MB/s 2024-04-25T08:38:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075783 s, 138 MB/s 2024-04-25T08:38:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.761 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066462 s, 158 MB/s 2024-04-25T08:38:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.772 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808775 s, 130 MB/s 2024-04-25T08:38:43.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.781 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664453 s, 158 MB/s 2024-04-25T08:38:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.791 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800594 s, 131 MB/s 2024-04-25T08:38:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802614 s, 131 MB/s 2024-04-25T08:38:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.811 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733577 s, 143 MB/s 2024-04-25T08:38:43.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:43.812 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:43.812 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:43.812 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T08:38:43.812 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:43.813 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:43.813 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:43.813 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:43.880 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764383 s, 137 MB/s 2024-04-25T08:38:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737851 s, 142 MB/s 2024-04-25T08:38:43.900 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.900 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.900 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728707 s, 144 MB/s 2024-04-25T08:38:43.909 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.909 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.909 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732735 s, 143 MB/s 2024-04-25T08:38:43.918 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.918 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.918 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686164 s, 153 MB/s 2024-04-25T08:38:43.927 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.927 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.927 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068051 s, 154 MB/s 2024-04-25T08:38:43.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.937 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067886 s, 154 MB/s 2024-04-25T08:38:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.945 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.946 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675132 s, 155 MB/s 2024-04-25T08:38:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.955 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073912 s, 142 MB/s 2024-04-25T08:38:43.964 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.965 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007163 s, 146 MB/s 2024-04-25T08:38:43.974 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.974 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.974 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732358 s, 143 MB/s 2024-04-25T08:38:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.985 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833494 s, 126 MB/s 2024-04-25T08:38:43.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:43.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:43.994 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666381 s, 157 MB/s 2024-04-25T08:38:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.003 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663292 s, 158 MB/s 2024-04-25T08:38:44.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.013 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818326 s, 128 MB/s 2024-04-25T08:38:44.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.022 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0064212 s, 163 MB/s 2024-04-25T08:38:44.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.031 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072004 s, 146 MB/s 2024-04-25T08:38:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.040 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650287 s, 161 MB/s 2024-04-25T08:38:44.050 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.050 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.050 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756508 s, 139 MB/s 2024-04-25T08:38:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.061 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801292 s, 131 MB/s 2024-04-25T08:38:44.069 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.069 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.070 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659137 s, 159 MB/s 2024-04-25T08:38:44.079 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.079 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.079 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742458 s, 141 MB/s 2024-04-25T08:38:44.089 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.090 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835724 s, 125 MB/s 2024-04-25T08:38:44.098 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.098 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.098 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663274 s, 158 MB/s 2024-04-25T08:38:44.108 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.108 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.108 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739145 s, 142 MB/s 2024-04-25T08:38:44.117 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.117 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.117 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656917 s, 160 MB/s 2024-04-25T08:38:44.127 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.127 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.127 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828671 s, 127 MB/s 2024-04-25T08:38:44.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.138 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828253 s, 127 MB/s 2024-04-25T08:38:44.147 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073551 s, 143 MB/s 2024-04-25T08:38:44.157 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.157 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.157 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070017 s, 150 MB/s 2024-04-25T08:38:44.165 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.166 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.166 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652877 s, 161 MB/s 2024-04-25T08:38:44.175 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.176 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.176 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786393 s, 133 MB/s 2024-04-25T08:38:44.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:44.177 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:44.177 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:44.177 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T08:38:44.177 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:44.177 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:44.177 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:44.177 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:44.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.246 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808751 s, 130 MB/s 2024-04-25T08:38:44.256 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.302 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.303 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826093 s, 127 MB/s 2024-04-25T08:38:44.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.303 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731267 s, 143 MB/s 2024-04-25T08:38:44.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.304 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659463 s, 159 MB/s 2024-04-25T08:38:44.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.304 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655922 s, 160 MB/s 2024-04-25T08:38:44.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.304 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793457 s, 132 MB/s 2024-04-25T08:38:44.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.305 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078835 s, 133 MB/s 2024-04-25T08:38:44.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.314 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789358 s, 133 MB/s 2024-04-25T08:38:44.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.323 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00637641 s, 164 MB/s 2024-04-25T08:38:44.331 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.331 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.332 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065175 s, 161 MB/s 2024-04-25T08:38:44.341 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.341 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.341 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737589 s, 142 MB/s 2024-04-25T08:38:44.350 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.350 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.350 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650697 s, 161 MB/s 2024-04-25T08:38:44.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.360 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800124 s, 131 MB/s 2024-04-25T08:38:44.369 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.369 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.369 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649424 s, 161 MB/s 2024-04-25T08:38:44.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.379 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719024 s, 146 MB/s 2024-04-25T08:38:44.387 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.387 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.387 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662677 s, 158 MB/s 2024-04-25T08:38:44.397 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.397 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.397 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763237 s, 137 MB/s 2024-04-25T08:38:44.406 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.406 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.406 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682398 s, 154 MB/s 2024-04-25T08:38:44.417 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.417 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.417 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817601 s, 128 MB/s 2024-04-25T08:38:44.426 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.426 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.426 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676678 s, 155 MB/s 2024-04-25T08:38:44.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.437 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799739 s, 131 MB/s 2024-04-25T08:38:44.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.447 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744187 s, 141 MB/s 2024-04-25T08:38:44.456 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.456 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.456 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643023 s, 163 MB/s 2024-04-25T08:38:44.465 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.465 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.466 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668166 s, 157 MB/s 2024-04-25T08:38:44.475 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.475 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.475 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733651 s, 143 MB/s 2024-04-25T08:38:44.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.485 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702174 s, 149 MB/s 2024-04-25T08:38:44.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.496 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851196 s, 123 MB/s 2024-04-25T08:38:44.507 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.507 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.507 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818876 s, 128 MB/s 2024-04-25T08:38:44.517 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.517 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.517 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757064 s, 139 MB/s 2024-04-25T08:38:44.526 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.526 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.527 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678635 s, 155 MB/s 2024-04-25T08:38:44.536 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.536 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.536 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690248 s, 152 MB/s 2024-04-25T08:38:44.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.547 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751698 s, 139 MB/s 2024-04-25T08:38:44.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:44.547 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:38:44.547 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:38:44.547 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T08:38:44.548 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:38:44.548 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:38:44.548 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:38:44.548 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:38:44.614 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.614 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.614 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693172 s, 151 MB/s 2024-04-25T08:38:44.625 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.625 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.625 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843451 s, 124 MB/s 2024-04-25T08:38:44.634 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.634 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.634 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717796 s, 146 MB/s 2024-04-25T08:38:44.643 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.643 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.644 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681405 s, 154 MB/s 2024-04-25T08:38:44.652 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.653 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.653 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069459 s, 151 MB/s 2024-04-25T08:38:44.663 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.663 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.663 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820691 s, 128 MB/s 2024-04-25T08:38:44.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.674 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875812 s, 120 MB/s 2024-04-25T08:38:44.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.685 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086416 s, 121 MB/s 2024-04-25T08:38:44.695 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.695 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.695 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786616 s, 133 MB/s 2024-04-25T08:38:44.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.705 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732253 s, 143 MB/s 2024-04-25T08:38:44.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.714 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727609 s, 144 MB/s 2024-04-25T08:38:44.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.724 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736949 s, 142 MB/s 2024-04-25T08:38:44.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.733 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733882 s, 143 MB/s 2024-04-25T08:38:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.744 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839293 s, 125 MB/s 2024-04-25T08:38:44.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.754 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727306 s, 144 MB/s 2024-04-25T08:38:44.764 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.764 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.764 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817379 s, 128 MB/s 2024-04-25T08:38:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.774 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743832 s, 141 MB/s 2024-04-25T08:38:44.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.784 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747401 s, 140 MB/s 2024-04-25T08:38:44.794 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.795 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.795 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843133 s, 124 MB/s 2024-04-25T08:38:44.804 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.804 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.805 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768249 s, 136 MB/s 2024-04-25T08:38:44.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.813 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006674 s, 157 MB/s 2024-04-25T08:38:44.822 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.822 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.823 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689811 s, 152 MB/s 2024-04-25T08:38:44.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.834 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084377 s, 124 MB/s 2024-04-25T08:38:44.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.844 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826867 s, 127 MB/s 2024-04-25T08:38:44.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.854 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743422 s, 141 MB/s 2024-04-25T08:38:44.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.863 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698704 s, 150 MB/s 2024-04-25T08:38:44.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.872 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693445 s, 151 MB/s 2024-04-25T08:38:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741201 s, 141 MB/s 2024-04-25T08:38:44.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.891 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074987 s, 140 MB/s 2024-04-25T08:38:44.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.901 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712162 s, 147 MB/s 2024-04-25T08:38:44.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.910 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702287 s, 149 MB/s 2024-04-25T08:38:44.919 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:38:44.919 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:38:44.919 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701755 s, 149 MB/s 2024-04-25T08:38:44.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:44.921 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-25T08:38:45.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.084+0000 7facd064d640 1 Processor -- start 2024-04-25T08:38:45.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.084+0000 7facd064d640 1 -- start start 2024-04-25T08:38:45.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.085+0000 7facd064d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7facc80cf970 0x7facc8165530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.085+0000 7facd064d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc815abb0 con 0x7facc80b47b0 2024-04-25T08:38:45.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.085+0000 7facd064d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facc815ae70 con 0x7facc806eab0 2024-04-25T08:38:45.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.085+0000 7facd064d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc815b7a0 con 0x7facc80cf970 2024-04-25T08:38:45.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.085+0000 7facce3e9640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7facc80b47b0 0x7facc806e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33582/0 (socket says 172.21.15.38:33582) 2024-04-25T08:38:45.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.085+0000 7facce3e9640 1 -- 172.21.15.38:0/2984055393 learned_addr learned my addr 172.21.15.38:0/2984055393 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:45.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.085+0000 7faccebea640 1 --2- 172.21.15.38:0/2984055393 >> v2:172.21.15.107:3300/0 conn(0x7facc80cf970 0x7facc8165530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.085+0000 7faccd3e7640 1 -- 172.21.15.38:0/2984055393 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744688327 0 0) 0x7facc815abb0 con 0x7facc80b47b0 2024-04-25T08:38:45.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.085+0000 7faccd3e7640 1 -- 172.21.15.38:0/2984055393 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facb4003520 con 0x7facc80b47b0 2024-04-25T08:38:45.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.085+0000 7faccebea640 1 -- 172.21.15.38:0/2984055393 >> v1:172.21.15.107:6789/0 conn(0x7facc806eab0 legacy=0x7facc80cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.085+0000 7faccebea640 1 -- 172.21.15.38:0/2984055393 >> v1:172.21.15.38:6789/0 conn(0x7facc80b47b0 legacy=0x7facc806e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.085+0000 7faccebea640 1 -- 172.21.15.38:0/2984055393 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc815b150 con 0x7facc80cf970 2024-04-25T08:38:45.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.086+0000 7faccebea640 1 --2- 172.21.15.38:0/2984055393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc80cf970 0x7facc8165530 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7facc0009d90 tx=0x7facc002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=966f5f9cfa21603c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.086+0000 7faccd3e7640 1 -- 172.21.15.38:0/2984055393 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0007740 con 0x7facc80cf970 2024-04-25T08:38:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.086+0000 7faccd3e7640 1 -- 172.21.15.38:0/2984055393 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc00078e0 con 0x7facc80cf970 2024-04-25T08:38:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.086+0000 7faccd3e7640 1 -- 172.21.15.38:0/2984055393 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0007c90 con 0x7facc80cf970 2024-04-25T08:38:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.087+0000 7facd064d640 1 -- 172.21.15.38:0/2984055393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc80cf970 msgr2=0x7facc8165530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.087+0000 7facd064d640 1 --2- 172.21.15.38:0/2984055393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc80cf970 0x7facc8165530 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7facc0009d90 tx=0x7facc002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.087+0000 7facd064d640 1 -- 172.21.15.38:0/2984055393 shutdown_connections 2024-04-25T08:38:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.087+0000 7facd064d640 1 --2- 172.21.15.38:0/2984055393 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc80cf970 0x7facc8165530 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.087+0000 7facd064d640 1 -- 172.21.15.38:0/2984055393 >> 172.21.15.38:0/2984055393 conn(0x7facc8066f20 msgr2=0x7facc806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.087+0000 7facd064d640 1 -- 172.21.15.38:0/2984055393 shutdown_connections 2024-04-25T08:38:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.087+0000 7facd064d640 1 -- 172.21.15.38:0/2984055393 wait complete. 2024-04-25T08:38:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.088+0000 7facd064d640 1 Processor -- start 2024-04-25T08:38:45.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.088+0000 7facd064d640 1 -- start start 2024-04-25T08:38:45.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facd064d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facc806eab0 0x7facc8148070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facd064d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc80b47b0 0x7facc8148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facd064d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facc80cf970 0x7facc80cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facd064d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc8070fa0 con 0x7facc806eab0 2024-04-25T08:38:45.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facd064d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc806d000 con 0x7facc80b47b0 2024-04-25T08:38:45.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facd064d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc8071120 con 0x7facc80cf970 2024-04-25T08:38:45.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facce3e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facc806eab0 0x7facc8148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facce3e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facc806eab0 0x7facc8148070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43376/0 (socket says 172.21.15.38:43376) 2024-04-25T08:38:45.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facce3e9640 1 -- 172.21.15.38:0/733390721 learned_addr learned my addr 172.21.15.38:0/733390721 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7faccdbe8640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc80b47b0 0x7facc8148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7faccebea640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facc80cf970 0x7facc80cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facce3e9640 1 -- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facc80cf970 msgr2=0x7facc80cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facce3e9640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facc80cf970 0x7facc80cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facce3e9640 1 -- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc80b47b0 msgr2=0x7facc8148ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facce3e9640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc80b47b0 0x7facc8148ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facce3e9640 1 -- 172.21.15.38:0/733390721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8149420 con 0x7facc806eab0 2024-04-25T08:38:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.089+0000 7facce3e9640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facc806eab0 0x7facc8148070 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7facc400c530 tx=0x7facc400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.090+0000 7facbaffd640 1 -- 172.21.15.38:0/733390721 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4017070 con 0x7facc806eab0 2024-04-25T08:38:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.090+0000 7facbaffd640 1 -- 172.21.15.38:0/733390721 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc4005ce0 con 0x7facc806eab0 2024-04-25T08:38:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.090+0000 7facd064d640 1 -- 172.21.15.38:0/733390721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc815bcb0 con 0x7facc806eab0 2024-04-25T08:38:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.090+0000 7facbaffd640 1 -- 172.21.15.38:0/733390721 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc40072b0 con 0x7facc806eab0 2024-04-25T08:38:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.090+0000 7facd064d640 1 -- 172.21.15.38:0/733390721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc81485b0 con 0x7facc806eab0 2024-04-25T08:38:45.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.091+0000 7facbaffd640 1 -- 172.21.15.38:0/733390721 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7facc401a070 con 0x7facc806eab0 2024-04-25T08:38:45.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.092+0000 7facbaffd640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7facac06b4d0 0x7facac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.092+0000 7facbaffd640 1 -- 172.21.15.38:0/733390721 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7facc4089770 con 0x7facc806eab0 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.092+0000 7faccdbe8640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7facac06b4d0 0x7facac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.092+0000 7faccdbe8640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7facac06b4d0 0x7facac06d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7facc806df10 tx=0x7facbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.093+0000 7facd342e740 1 Processor -- start 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.093+0000 7facd342e740 1 -- start start 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.093+0000 7facd342e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c17bff0 0x56023c17c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.093+0000 7facd342e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c1563a0 0x56023c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.093+0000 7facd342e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c19b350 0x56023c153260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7facd342e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023c0b8000 con 0x56023c19b350 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7facd342e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023c060f20 con 0x56023c17bff0 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7facd342e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56023c0b40a0 con 0x56023c1563a0 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7faccebea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c19b350 0x56023c153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7faccebea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c19b350 0x56023c153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43390/0 (socket says 172.21.15.38:43390) 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7faccebea640 1 -- 172.21.15.38:0/941845754 learned_addr learned my addr 172.21.15.38:0/941845754 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7facce3e9640 1 --2- 172.21.15.38:0/941845754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c17bff0 0x56023c17c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7faccdbe8640 1 --2- 172.21.15.38:0/941845754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c1563a0 0x56023c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7faccebea640 1 -- 172.21.15.38:0/941845754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c1563a0 msgr2=0x56023c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7faccebea640 1 --2- 172.21.15.38:0/941845754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c1563a0 0x56023c19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7faccebea640 1 -- 172.21.15.38:0/941845754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c17bff0 msgr2=0x56023c17c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7faccebea640 1 --2- 172.21.15.38:0/941845754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c17bff0 0x56023c17c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7faccebea640 1 -- 172.21.15.38:0/941845754 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56023be98ba0 con 0x56023c19b350 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7faccebea640 1 --2- 172.21.15.38:0/941845754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c19b350 0x56023c153260 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7facc0009d90 tx=0x7facc002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283ea1bc2edea0ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7facabfff640 1 -- 172.21.15.38:0/941845754 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0003d90 con 0x56023c19b350 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7facabfff640 1 -- 172.21.15.38:0/941845754 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc0004040 con 0x56023c19b350 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.094+0000 7facabfff640 1 -- 172.21.15.38:0/941845754 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc00043f0 con 0x56023c19b350 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.095+0000 7facd342e740 1 -- 172.21.15.38:0/941845754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c19b350 msgr2=0x56023c153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.095+0000 7facd342e740 1 --2- 172.21.15.38:0/941845754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c19b350 0x56023c153260 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7facc0009d90 tx=0x7facc002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:45.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.095+0000 7facd342e740 1 -- 172.21.15.38:0/941845754 shutdown_connections 2024-04-25T08:38:45.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.095+0000 7facd342e740 1 --2- 172.21.15.38:0/941845754 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c19b350 0x56023c153260 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.095+0000 7facd342e740 1 --2- 172.21.15.38:0/941845754 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c1563a0 0x56023c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.095+0000 7facd342e740 1 --2- 172.21.15.38:0/941845754 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c17bff0 0x56023c17c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.095+0000 7facd342e740 1 -- 172.21.15.38:0/941845754 >> 172.21.15.38:0/941845754 conn(0x56023c11c050 msgr2=0x56023c199cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.095+0000 7facd342e740 1 -- 172.21.15.38:0/941845754 shutdown_connections 2024-04-25T08:38:45.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.095+0000 7facd342e740 1 -- 172.21.15.38:0/941845754 wait complete. 2024-04-25T08:38:45.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.096+0000 7facd342e740 1 -- start start 2024-04-25T08:38:45.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.096+0000 7facd342e740 1 Processor -- start 2024-04-25T08:38:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facd342e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 0x56023c176610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facd342e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c17bff0 0x56023c181740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facd342e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c176b50 0x56023c181c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facd342e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023c150800 con 0x56023c1563a0 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facd342e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023c056610 con 0x56023c176b50 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facd342e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56023c155330 con 0x56023c17bff0 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facd342e740 10 client.0 _ll_register_callbacks cb 0x56023c06ef20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facce3e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 0x56023c176610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facce3e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 0x56023c176610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43392/0 (socket says 172.21.15.38:43392) 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facce3e9640 1 -- 172.21.15.38:0/1783956535 learned_addr learned my addr 172.21.15.38:0/1783956535 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7faccdbe8640 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c17bff0 0x56023c181740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7faccebea640 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c176b50 0x56023c181c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facce3e9640 1 -- 172.21.15.38:0/1783956535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c17bff0 msgr2=0x56023c181740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facce3e9640 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c17bff0 0x56023c181740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facce3e9640 1 -- 172.21.15.38:0/1783956535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c176b50 msgr2=0x56023c181c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facce3e9640 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c176b50 0x56023c181c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facce3e9640 1 -- 172.21.15.38:0/1783956535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56023bd8c490 con 0x56023c1563a0 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facce3e9640 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 0x56023c176610 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7facc40011e0 tx=0x7facc405bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce7fa29d0128fea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7faca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7faca97fa640 1 -- 172.21.15.38:0/1783956535 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4060000 con 0x56023c1563a0 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7faca97fa640 1 -- 172.21.15.38:0/1783956535 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7facc40601a0 con 0x56023c1563a0 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.097+0000 7facd342e740 1 -- 172.21.15.38:0/1783956535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facc0004040 con 0x56023c1563a0 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7faca97fa640 1 -- 172.21.15.38:0/1783956535 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7facc4007450 con 0x56023c1563a0 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7facd342e740 10 client.8025 fetch_fsmap learned FSMap version 57 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7facd342e740 1 -- 172.21.15.38:0/1783956535 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x56023bef9330 con 0x56023c1563a0 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7faca97fa640 1 -- 172.21.15.38:0/1783956535 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc0004040 con 0x56023c1563a0 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7faca97fa640 1 -- 172.21.15.38:0/1783956535 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7facc40075f0 con 0x56023c1563a0 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7facd342e740 10 client.8025 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7facd342e740 10 client.8025 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7facd342e740 20 client.8025 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:45.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7facd342e740 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56023c1b8650 0x56023c1baa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7facd342e740 4 client.8025 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7facd342e740 1 -- 172.21.15.38:0/1783956535 --> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56023c0c5540 con 0x56023c1b8650 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7faccdbe8640 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56023c1b8650 0x56023c1baa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7faca97fa640 10 client.8025 ms_handle_connect on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.098+0000 7faccdbe8640 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56023c1b8650 0x56023c1baa40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a69fc291eac1ac4f server_cookie=e70ace7550c94bf2 in_seq=0 out_seq=0 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7faca97fa640 1 -- 172.21.15.38:0/1783956535 <== mds.0 v2:172.21.15.38:6833/2922734840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56023c0c5540 con 0x56023c1b8650 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7faca97fa640 10 client.8025 handle_command_reply: tid=0 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7facd342e740 1 client.8025 shutdown 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7facd342e740 1 -- 172.21.15.38:0/1783956535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 msgr2=0x56023c176610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7facd342e740 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 0x56023c176610 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7facc40011e0 tx=0x7facc405bf50 comp rx=0 tx=0).stop 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7facd342e740 1 -- 172.21.15.38:0/1783956535 shutdown_connections 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7facd342e740 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c176b50 0x56023c181c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7facd342e740 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c17bff0 0x56023c181740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7facd342e740 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56023c1b8650 0x56023c1baa40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7facd342e740 1 --2- 172.21.15.38:0/1783956535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 0x56023c176610 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7facd342e740 1 -- 172.21.15.38:0/1783956535 >> 172.21.15.38:0/1783956535 conn(0x56023c11c050 msgr2=0x56023c179540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7faca97fa640 0 client.8025 ms_handle_reset on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:45.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7facd342e740 1 -- 172.21.15.38:0/1783956535 shutdown_connections 2024-04-25T08:38:45.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7facd342e740 1 -- 172.21.15.38:0/1783956535 wait complete. 2024-04-25T08:38:45.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.099+0000 7facd342e740 20 client.8025 trim_cache size 0 max 16384 2024-04-25T08:38:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.114+0000 7facd342e740 1 Processor -- start 2024-04-25T08:38:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.115+0000 7facd342e740 1 -- start start 2024-04-25T08:38:45.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.115+0000 7facd342e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 0x56023c1adf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.115+0000 7facd342e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c176b50 0x56023c14e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.115+0000 7facd342e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c17bff0 0x56023c176500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.115+0000 7facd342e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc8167880 con 0x56023c1563a0 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.115+0000 7facd342e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc815da30 con 0x56023c17bff0 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.115+0000 7facd342e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc8151210 con 0x56023c176b50 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.115+0000 7facce3e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 0x56023c1adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.115+0000 7facce3e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 0x56023c1adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43402/0 (socket says 172.21.15.38:43402) 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.115+0000 7facce3e9640 1 -- 172.21.15.38:0/1883296219 learned_addr learned my addr 172.21.15.38:0/1883296219 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7faccebea640 1 --2- 172.21.15.38:0/1883296219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c17bff0 0x56023c176500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7faccdbe8640 1 --2- 172.21.15.38:0/1883296219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c176b50 0x56023c14e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7facce3e9640 1 -- 172.21.15.38:0/1883296219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c176b50 msgr2=0x56023c14e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7facce3e9640 1 --2- 172.21.15.38:0/1883296219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c176b50 0x56023c14e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7facce3e9640 1 -- 172.21.15.38:0/1883296219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c17bff0 msgr2=0x56023c176500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7facce3e9640 1 --2- 172.21.15.38:0/1883296219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c17bff0 0x56023c176500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7facce3e9640 1 -- 172.21.15.38:0/1883296219 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56023c152560 con 0x56023c1563a0 2024-04-25T08:38:45.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7faccebea640 1 --2- 172.21.15.38:0/1883296219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c17bff0 0x56023c176500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7faccdbe8640 1 --2- 172.21.15.38:0/1883296219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c176b50 0x56023c14e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7facce3e9640 1 --2- 172.21.15.38:0/1883296219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 0x56023c1adf20 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7facc4000fb0 tx=0x7facc405a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab28a0c2c404bc7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7facab7fe640 1 -- 172.21.15.38:0/1883296219 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4060000 con 0x56023c1563a0 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7facab7fe640 1 -- 172.21.15.38:0/1883296219 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc4087070 con 0x56023c1563a0 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7facab7fe640 1 -- 172.21.15.38:0/1883296219 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc4076b90 con 0x56023c1563a0 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7facd342e740 1 -- 172.21.15.38:0/1883296219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 msgr2=0x56023c1adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.116+0000 7facd342e740 1 --2- 172.21.15.38:0/1883296219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 0x56023c1adf20 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7facc4000fb0 tx=0x7facc405a000 comp rx=0 tx=0).stop 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.117+0000 7facd342e740 1 -- 172.21.15.38:0/1883296219 shutdown_connections 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.117+0000 7facd342e740 1 --2- 172.21.15.38:0/1883296219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c17bff0 0x56023c176500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.117+0000 7facd342e740 1 --2- 172.21.15.38:0/1883296219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c176b50 0x56023c14e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.117+0000 7facd342e740 1 --2- 172.21.15.38:0/1883296219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c1563a0 0x56023c1adf20 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.117+0000 7facd342e740 1 -- 172.21.15.38:0/1883296219 >> 172.21.15.38:0/1883296219 conn(0x56023c11c050 msgr2=0x56023c154f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.117+0000 7facd342e740 1 -- 172.21.15.38:0/1883296219 shutdown_connections 2024-04-25T08:38:45.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.117+0000 7facd342e740 1 -- 172.21.15.38:0/1883296219 wait complete. 2024-04-25T08:38:45.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.117+0000 7facd342e740 1 -- start start 2024-04-25T08:38:45.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.117+0000 7facd342e740 1 Processor -- start 2024-04-25T08:38:45.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.118+0000 7facd342e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c1563a0 0x56023c1ad280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.118+0000 7facd342e740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c176b50 0x56023c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.118+0000 7facd342e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c17bff0 0x56023c208620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.118+0000 7facd342e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56023c14eba0 con 0x56023c17bff0 2024-04-25T08:38:45.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.118+0000 7facd342e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc815d430 con 0x56023c1563a0 2024-04-25T08:38:45.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.118+0000 7facd342e740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56023c17c740 con 0x56023c176b50 2024-04-25T08:38:45.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.118+0000 7facd342e740 10 client.0 _ll_register_callbacks cb 0x7facc0009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:45.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.118+0000 7faccebea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c17bff0 0x56023c208620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.118+0000 7faccebea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c17bff0 0x56023c208620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43414/0 (socket says 172.21.15.38:43414) 2024-04-25T08:38:45.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.118+0000 7facce3e9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c1563a0 0x56023c1ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.118+0000 7faccebea640 1 -- 172.21.15.38:0/2221624051 learned_addr learned my addr 172.21.15.38:0/2221624051 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.118+0000 7faccdbe8640 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c176b50 0x56023c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7faccebea640 1 -- 172.21.15.38:0/2221624051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c176b50 msgr2=0x56023c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7faccebea640 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c176b50 0x56023c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7faccebea640 1 -- 172.21.15.38:0/2221624051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c1563a0 msgr2=0x56023c1ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7faccebea640 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c1563a0 0x56023c1ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7faccebea640 1 -- 172.21.15.38:0/2221624051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56023bd65610 con 0x56023c17bff0 2024-04-25T08:38:45.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7facce3e9640 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c1563a0 0x56023c1ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:45.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7faccebea640 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c17bff0 0x56023c208620 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7facc0009eb0 tx=0x7facc0030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e554d902b96224 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7faca97fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7faca97fa640 1 -- 172.21.15.38:0/2221624051 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facc0004780 con 0x56023c17bff0 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7faca97fa640 1 -- 172.21.15.38:0/2221624051 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7facc0007540 con 0x56023c17bff0 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7facd342e740 1 -- 172.21.15.38:0/2221624051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facc4087070 con 0x56023c17bff0 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7faca97fa640 1 -- 172.21.15.38:0/2221624051 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7facc0007740 con 0x56023c17bff0 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7facd342e740 10 client.8031 fetch_fsmap learned FSMap version 57 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7facd342e740 1 -- 172.21.15.38:0/2221624051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x56023c19d000 con 0x56023c17bff0 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7faca97fa640 1 -- 172.21.15.38:0/2221624051 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facc4087070 con 0x56023c17bff0 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7faca97fa640 1 -- 172.21.15.38:0/2221624051 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7facc0052070 con 0x56023c17bff0 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7facd342e740 10 client.8031 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7facd342e740 10 client.8031 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7facd342e740 20 client.8031 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7facd342e740 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56023c208b60 0x56023c1794d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7facd342e740 4 client.8031 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T08:38:45.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.119+0000 7facd342e740 1 -- 172.21.15.38:0/2221624051 --> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x56023c19e9a0 con 0x56023c208b60 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.120+0000 7facce3e9640 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56023c208b60 0x56023c1794d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.120+0000 7faca97fa640 10 client.8031 ms_handle_connect on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.120+0000 7facce3e9640 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56023c208b60 0x56023c1794d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34b88f83aecac189 server_cookie=5308782255edbd3e in_seq=0 out_seq=0 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.122+0000 7faca97fa640 1 -- 172.21.15.38:0/2221624051 <== mds.0 v2:172.21.15.38:6833/2922734840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56023c19e9a0 con 0x56023c208b60 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.122+0000 7faca97fa640 10 client.8031 handle_command_reply: tid=0 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.122+0000 7facd342e740 1 client.8031 shutdown 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.122+0000 7facd342e740 1 -- 172.21.15.38:0/2221624051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c17bff0 msgr2=0x56023c208620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.122+0000 7facd342e740 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c17bff0 0x56023c208620 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7facc0009eb0 tx=0x7facc0030000 comp rx=0 tx=0).stop 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.122+0000 7facd342e740 1 -- 172.21.15.38:0/2221624051 shutdown_connections 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.122+0000 7facd342e740 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56023c208b60 0x56023c1794d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.122+0000 7facd342e740 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56023c17bff0 0x56023c208620 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.122+0000 7facd342e740 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56023c176b50 0x56023c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.122+0000 7faca97fa640 0 client.8031 ms_handle_reset on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.122+0000 7facd342e740 1 --2- 172.21.15.38:0/2221624051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56023c1563a0 0x56023c1ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.122+0000 7facd342e740 1 -- 172.21.15.38:0/2221624051 >> 172.21.15.38:0/2221624051 conn(0x56023c11c050 msgr2=0x56023c14e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd342e740 1 -- 172.21.15.38:0/2221624051 shutdown_connections 2024-04-25T08:38:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd342e740 1 -- 172.21.15.38:0/2221624051 wait complete. 2024-04-25T08:38:45.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd342e740 20 client.8031 trim_cache size 0 max 16384 2024-04-25T08:38:45.121 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:38:45.121 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:38:45.121 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "8a9873f7-d2fe-407c-a005-933a922070ef", 2024-04-25T08:38:45.121 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:38:45.121 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:38:45.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd064d640 1 -- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7facac06b4d0 msgr2=0x7facac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd064d640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7facac06b4d0 0x7facac06d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7facc806df10 tx=0x7facbc009040 comp rx=0 tx=0).stop 2024-04-25T08:38:45.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd064d640 1 -- 172.21.15.38:0/733390721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facc806eab0 msgr2=0x7facc8148070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd064d640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facc806eab0 0x7facc8148070 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7facc400c530 tx=0x7facc400c9f0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd064d640 1 -- 172.21.15.38:0/733390721 shutdown_connections 2024-04-25T08:38:45.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd064d640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7facac06b4d0 0x7facac06d990 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd064d640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7facc80cf970 0x7facc80cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd064d640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7facc80b47b0 0x7facc8148ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd064d640 1 --2- 172.21.15.38:0/733390721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7facc806eab0 0x7facc8148070 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.123+0000 7facd064d640 1 -- 172.21.15.38:0/733390721 >> 172.21.15.38:0/733390721 conn(0x7facc8066f20 msgr2=0x7facc80aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.124+0000 7facd064d640 1 -- 172.21.15.38:0/733390721 shutdown_connections 2024-04-25T08:38:45.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.124+0000 7facd064d640 1 -- 172.21.15.38:0/733390721 wait complete. 2024-04-25T08:38:45.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:45.131 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-25T08:38:45.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.266+0000 7ff7794c4640 1 Processor -- start 2024-04-25T08:38:45.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.266+0000 7ff7794c4640 1 -- start start 2024-04-25T08:38:45.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff7794c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7740cc130 0x7ff7740b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff7794c4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7740c3e40 con 0x7ff77415bb90 2024-04-25T08:38:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff7794c4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77406b030 con 0x7ff7740b13f0 2024-04-25T08:38:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff7794c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77406b8d0 con 0x7ff7740cc130 2024-04-25T08:38:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff76bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff77415bb90 0x7ff7740b17d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33584/0 (socket says 172.21.15.38:33584) 2024-04-25T08:38:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff76bfff640 1 -- 172.21.15.38:0/3960117666 learned_addr learned my addr 172.21.15.38:0/3960117666 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff7737fe640 1 --2- 172.21.15.38:0/3960117666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7740cc130 0x7ff7740b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff7727fc640 1 -- 172.21.15.38:0/3960117666 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313177093 0 0) 0x7ff7740c3e40 con 0x7ff77415bb90 2024-04-25T08:38:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff7727fc640 1 -- 172.21.15.38:0/3960117666 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff760003520 con 0x7ff77415bb90 2024-04-25T08:38:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff7727fc640 1 -- 172.21.15.38:0/3960117666 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039090960 0 0) 0x7ff77406b030 con 0x7ff7740b13f0 2024-04-25T08:38:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff7727fc640 1 -- 172.21.15.38:0/3960117666 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7740c3e40 con 0x7ff7740b13f0 2024-04-25T08:38:45.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff7737fe640 1 -- 172.21.15.38:0/3960117666 >> v1:172.21.15.107:6789/0 conn(0x7ff7740b13f0 legacy=0x7ff77415c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff7737fe640 1 -- 172.21.15.38:0/3960117666 >> v1:172.21.15.38:6789/0 conn(0x7ff77415bb90 legacy=0x7ff7740b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.267+0000 7ff7727fc640 1 -- 172.21.15.38:0/3960117666 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3312346235 0 0) 0x7ff760003520 con 0x7ff77415bb90 2024-04-25T08:38:45.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.268+0000 7ff7737fe640 1 -- 172.21.15.38:0/3960117666 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77406b310 con 0x7ff7740cc130 2024-04-25T08:38:45.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.268+0000 7ff7737fe640 1 --2- 172.21.15.38:0/3960117666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7740cc130 0x7ff7740b0540 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009dc0 tx=0x7ff76c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8535b4b4fd012f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.268+0000 7ff7727fc640 1 -- 172.21.15.38:0/3960117666 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c001690 con 0x7ff7740cc130 2024-04-25T08:38:45.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.268+0000 7ff7727fc640 1 -- 172.21.15.38:0/3960117666 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c007540 con 0x7ff7740cc130 2024-04-25T08:38:45.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.268+0000 7ff7727fc640 1 -- 172.21.15.38:0/3960117666 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c007910 con 0x7ff7740cc130 2024-04-25T08:38:45.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.269+0000 7ff7794c4640 1 -- 172.21.15.38:0/3960117666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7740cc130 msgr2=0x7ff7740b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.269+0000 7ff7794c4640 1 --2- 172.21.15.38:0/3960117666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7740cc130 0x7ff7740b0540 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009dc0 tx=0x7ff76c02a660 comp rx=0 tx=0).stop 2024-04-25T08:38:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.269+0000 7ff7794c4640 1 -- 172.21.15.38:0/3960117666 shutdown_connections 2024-04-25T08:38:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.269+0000 7ff7794c4640 1 --2- 172.21.15.38:0/3960117666 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7740cc130 0x7ff7740b0540 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.269+0000 7ff7794c4640 1 -- 172.21.15.38:0/3960117666 >> 172.21.15.38:0/3960117666 conn(0x7ff7740d0520 msgr2=0x7ff7740d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.269+0000 7ff7794c4640 1 -- 172.21.15.38:0/3960117666 shutdown_connections 2024-04-25T08:38:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.269+0000 7ff7794c4640 1 -- 172.21.15.38:0/3960117666 wait complete. 2024-04-25T08:38:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.270+0000 7ff7794c4640 1 Processor -- start 2024-04-25T08:38:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.270+0000 7ff7794c4640 1 -- start start 2024-04-25T08:38:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff7794c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7740b13f0 0x7ff7740c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff7794c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740cc130 0x7ff7740c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff7794c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff77415bb90 0x7ff77415cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff7794c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77406abe0 con 0x7ff7740cc130 2024-04-25T08:38:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff7794c4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740b00b0 con 0x7ff77415bb90 2024-04-25T08:38:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff7794c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77406ad60 con 0x7ff7740b13f0 2024-04-25T08:38:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff76bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740cc130 0x7ff7740c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff76bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740cc130 0x7ff7740c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43430/0 (socket says 172.21.15.38:43430) 2024-04-25T08:38:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff76bfff640 1 -- 172.21.15.38:0/748914748 learned_addr learned my addr 172.21.15.38:0/748914748 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff7737fe640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff77415bb90 0x7ff77415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff772ffd640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7740b13f0 0x7ff7740c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff76bfff640 1 -- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7740b13f0 msgr2=0x7ff7740c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff76bfff640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7740b13f0 0x7ff7740c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff76bfff640 1 -- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff77415bb90 msgr2=0x7ff77415cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff76bfff640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff77415bb90 0x7ff77415cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.271+0000 7ff76bfff640 1 -- 172.21.15.38:0/748914748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77415d900 con 0x7ff7740cc130 2024-04-25T08:38:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.272+0000 7ff76bfff640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740cc130 0x7ff7740c6820 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c4b0 tx=0x7ff75c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.272+0000 7ff76b7fe640 1 -- 172.21.15.38:0/748914748 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c017070 con 0x7ff7740cc130 2024-04-25T08:38:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.272+0000 7ff76b7fe640 1 -- 172.21.15.38:0/748914748 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c005ce0 con 0x7ff7740cc130 2024-04-25T08:38:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.272+0000 7ff76b7fe640 1 -- 172.21.15.38:0/748914748 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c007290 con 0x7ff7740cc130 2024-04-25T08:38:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.272+0000 7ff7794c4640 1 -- 172.21.15.38:0/748914748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77415dbe0 con 0x7ff7740cc130 2024-04-25T08:38:45.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.272+0000 7ff7794c4640 1 -- 172.21.15.38:0/748914748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77415e1a0 con 0x7ff7740cc130 2024-04-25T08:38:45.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.273+0000 7ff76b7fe640 1 -- 172.21.15.38:0/748914748 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff75c01a070 con 0x7ff7740cc130 2024-04-25T08:38:45.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.274+0000 7ff76b7fe640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff75406b4d0 0x7ff75406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.274+0000 7ff76b7fe640 1 -- 172.21.15.38:0/748914748 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff75c089590 con 0x7ff7740cc130 2024-04-25T08:38:45.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.274+0000 7ff772ffd640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff75406b4d0 0x7ff75406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.274+0000 7ff772ffd640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff75406b4d0 0x7ff75406d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff7640011e0 tx=0x7ff764009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.275+0000 7ff77c1c6740 1 Processor -- start 2024-04-25T08:38:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.275+0000 7ff77c1c6740 1 -- start start 2024-04-25T08:38:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.275+0000 7ff77c1c6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x557907390e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.275+0000 7ff77c1c6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 0x55790738f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.275+0000 7ff77c1c6740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 0x5579073b9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.275+0000 7ff77c1c6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579072f1b90 con 0x5579073cb370 2024-04-25T08:38:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.275+0000 7ff77c1c6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55790729ae40 con 0x55790738eca0 2024-04-25T08:38:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.275+0000 7ff77c1c6740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579072edc30 con 0x5579073b9760 2024-04-25T08:38:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff76bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 0x55790738f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff76bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 0x55790738f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43444/0 (socket says 172.21.15.38:43444) 2024-04-25T08:38:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff76bfff640 1 -- 172.21.15.38:0/2812732231 learned_addr learned my addr 172.21.15.38:0/2812732231 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff7737fe640 1 --2- 172.21.15.38:0/2812732231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 0x5579073b9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff772ffd640 1 --2- 172.21.15.38:0/2812732231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x557907390e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff76bfff640 1 -- 172.21.15.38:0/2812732231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 msgr2=0x5579073b9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff76bfff640 1 --2- 172.21.15.38:0/2812732231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 0x5579073b9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff76bfff640 1 -- 172.21.15.38:0/2812732231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 msgr2=0x557907390e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff76bfff640 1 --2- 172.21.15.38:0/2812732231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x557907390e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff76bfff640 1 -- 172.21.15.38:0/2812732231 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579070d07d0 con 0x5579073cb370 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff76bfff640 1 --2- 172.21.15.38:0/2812732231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 0x55790738f080 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7ff75c05fdf0 tx=0x7ff75c057d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3128833bf10bfe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff768ff9640 1 -- 172.21.15.38:0/2812732231 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c055fc0 con 0x5579073cb370 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff768ff9640 1 -- 172.21.15.38:0/2812732231 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c054330 con 0x5579073cb370 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.276+0000 7ff768ff9640 1 -- 172.21.15.38:0/2812732231 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c06be10 con 0x5579073cb370 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.277+0000 7ff77c1c6740 1 -- 172.21.15.38:0/2812732231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 msgr2=0x55790738f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.277+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/2812732231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 0x55790738f080 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7ff75c05fdf0 tx=0x7ff75c057d80 comp rx=0 tx=0).stop 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.277+0000 7ff77c1c6740 1 -- 172.21.15.38:0/2812732231 shutdown_connections 2024-04-25T08:38:45.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.277+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/2812732231 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 0x5579073b9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.277+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/2812732231 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 0x55790738f080 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.277+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/2812732231 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x557907390e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.277+0000 7ff77c1c6740 1 -- 172.21.15.38:0/2812732231 >> 172.21.15.38:0/2812732231 conn(0x557907356870 msgr2=0x5579073d4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.277+0000 7ff77c1c6740 1 -- 172.21.15.38:0/2812732231 shutdown_connections 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.277+0000 7ff77c1c6740 1 -- 172.21.15.38:0/2812732231 wait complete. 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.277+0000 7ff77c1c6740 1 -- start start 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.278+0000 7ff77c1c6740 1 Processor -- start 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.278+0000 7ff77c1c6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55790738eca0 0x5579073b7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.278+0000 7ff77c1c6740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 0x5579073b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.278+0000 7ff77c1c6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579073cb370 0x5579073bc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.278+0000 7ff77c1c6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579072ae3e0 con 0x55790738eca0 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.278+0000 7ff77c1c6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579073b72b0 con 0x5579073cb370 2024-04-25T08:38:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.278+0000 7ff77c1c6740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579073e4b90 con 0x5579073b9760 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.278+0000 7ff77c1c6740 10 client.0 _ll_register_callbacks cb 0x5579072addb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.278+0000 7ff772ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55790738eca0 0x5579073b7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.278+0000 7ff772ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55790738eca0 0x5579073b7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43456/0 (socket says 172.21.15.38:43456) 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.278+0000 7ff772ffd640 1 -- 172.21.15.38:0/1726618860 learned_addr learned my addr 172.21.15.38:0/1726618860 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff76bfff640 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 0x5579073b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff7737fe640 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579073cb370 0x5579073bc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff772ffd640 1 -- 172.21.15.38:0/1726618860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 msgr2=0x5579073b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff772ffd640 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 0x5579073b83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff772ffd640 1 -- 172.21.15.38:0/1726618860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579073cb370 msgr2=0x5579073bc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff772ffd640 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579073cb370 0x5579073bc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff772ffd640 1 -- 172.21.15.38:0/1726618860 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557906fc60f0 con 0x55790738eca0 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff772ffd640 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55790738eca0 0x5579073b7eb0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7ff7640147b0 tx=0x7ff764001150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5cbc4559261ed01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff751ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff751ffb640 1 -- 172.21.15.38:0/1726618860 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff764025020 con 0x55790738eca0 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff751ffb640 1 -- 172.21.15.38:0/1726618860 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff7640138b0 con 0x55790738eca0 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1726618860 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff75c054330 con 0x55790738eca0 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff751ffb640 1 -- 172.21.15.38:0/1726618860 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff76402f050 con 0x55790738eca0 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff77c1c6740 10 client.8043 fetch_fsmap learned FSMap version 57 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1726618860 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x557907130f60 con 0x55790738eca0 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.279+0000 7ff751ffb640 1 -- 172.21.15.38:0/1726618860 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75c054330 con 0x55790738eca0 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff751ffb640 1 -- 172.21.15.38:0/1726618860 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff764032020 con 0x55790738eca0 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff77c1c6740 10 client.8043 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff77c1c6740 10 client.8043 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff77c1c6740 20 client.8043 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x5579073cb840 0x5579073bc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff77c1c6740 4 client.8043 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:38:45.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1726618860 --> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5579072ff0d0 con 0x5579073cb840 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff76bfff640 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x5579073cb840 0x5579073bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff751ffb640 10 client.8043 ms_handle_connect on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff76bfff640 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x5579073cb840 0x5579073bc680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6d600335bf9321f server_cookie=2355ae670f03c651 in_seq=0 out_seq=0 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff751ffb640 1 -- 172.21.15.38:0/1726618860 <== mds.0 v2:172.21.15.38:6833/2922734840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5579072ff0d0 con 0x5579073cb840 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff751ffb640 10 client.8043 handle_command_reply: tid=0 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.280+0000 7ff77c1c6740 1 client.8043 shutdown 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.281+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1726618860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55790738eca0 msgr2=0x5579073b7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.281+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55790738eca0 0x5579073b7eb0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7ff7640147b0 tx=0x7ff764001150 comp rx=0 tx=0).stop 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.281+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1726618860 shutdown_connections 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.281+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5579073cb370 0x5579073bc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.281+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 0x5579073b83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.281+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x5579073cb840 0x5579073bc680 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.281+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/1726618860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55790738eca0 0x5579073b7eb0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.281+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1726618860 >> 172.21.15.38:0/1726618860 conn(0x557907356870 msgr2=0x5579073d7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.281+0000 7ff751ffb640 0 client.8043 ms_handle_reset on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:45.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.281+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1726618860 shutdown_connections 2024-04-25T08:38:45.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.281+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1726618860 wait complete. 2024-04-25T08:38:45.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.281+0000 7ff77c1c6740 20 client.8043 trim_cache size 0 max 16384 2024-04-25T08:38:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.296+0000 7ff77c1c6740 1 Processor -- start 2024-04-25T08:38:45.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.296+0000 7ff77c1c6740 1 -- start start 2024-04-25T08:38:45.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.296+0000 7ff77c1c6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x5579073d8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.296+0000 7ff77c1c6740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 0x5579073ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.296+0000 7ff77c1c6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 0x5579073cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.296+0000 7ff77c1c6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741fe0d0 con 0x5579073cb370 2024-04-25T08:38:45.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.296+0000 7ff77c1c6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7741fd2d0 con 0x55790738eca0 2024-04-25T08:38:45.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.296+0000 7ff77c1c6740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740b4ad0 con 0x5579073b9760 2024-04-25T08:38:45.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff7737fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 0x5579073cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff7737fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 0x5579073cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43466/0 (socket says 172.21.15.38:43466) 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff7737fe640 1 -- 172.21.15.38:0/2615362502 learned_addr learned my addr 172.21.15.38:0/2615362502 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff772ffd640 1 --2- 172.21.15.38:0/2615362502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x5579073d8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff76bfff640 1 --2- 172.21.15.38:0/2615362502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 0x5579073ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff7737fe640 1 -- 172.21.15.38:0/2615362502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 msgr2=0x5579073ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff7737fe640 1 --2- 172.21.15.38:0/2615362502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 0x5579073ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff7737fe640 1 -- 172.21.15.38:0/2615362502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 msgr2=0x5579073d8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff7737fe640 1 --2- 172.21.15.38:0/2615362502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x5579073d8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff7737fe640 1 -- 172.21.15.38:0/2615362502 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557906f9f1b0 con 0x5579073cb370 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff772ffd640 1 --2- 172.21.15.38:0/2615362502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x5579073d8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff7737fe640 1 --2- 172.21.15.38:0/2615362502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 0x5579073cceb0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009eb0 tx=0x7ff76c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102ba7d783821d08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff753fff640 1 -- 172.21.15.38:0/2615362502 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c007680 con 0x5579073cb370 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff753fff640 1 -- 172.21.15.38:0/2615362502 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c007820 con 0x5579073cb370 2024-04-25T08:38:45.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.297+0000 7ff753fff640 1 -- 172.21.15.38:0/2615362502 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c007bd0 con 0x5579073cb370 2024-04-25T08:38:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.298+0000 7ff77c1c6740 1 -- 172.21.15.38:0/2615362502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 msgr2=0x5579073cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.298+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/2615362502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 0x5579073cceb0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009eb0 tx=0x7ff76c02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.298+0000 7ff77c1c6740 1 -- 172.21.15.38:0/2615362502 shutdown_connections 2024-04-25T08:38:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.298+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/2615362502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073cb370 0x5579073cceb0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.298+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/2615362502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073b9760 0x5579073ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.298+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/2615362502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x5579073d8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.298+0000 7ff77c1c6740 1 -- 172.21.15.38:0/2615362502 >> 172.21.15.38:0/2615362502 conn(0x557907356870 msgr2=0x557907390c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.298+0000 7ff77c1c6740 1 -- 172.21.15.38:0/2615362502 shutdown_connections 2024-04-25T08:38:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.298+0000 7ff77c1c6740 1 -- 172.21.15.38:0/2615362502 wait complete. 2024-04-25T08:38:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.298+0000 7ff77c1c6740 1 -- start start 2024-04-25T08:38:45.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.298+0000 7ff77c1c6740 1 Processor -- start 2024-04-25T08:38:45.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff77c1c6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x55790738d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff77c1c6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073b9760 0x557907442be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff77c1c6740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073cb370 0x5579073edf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff77c1c6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579073d8ba0 con 0x5579073b9760 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff77c1c6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7740b50d0 con 0x55790738eca0 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff77c1c6740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5579073cb1a0 con 0x5579073cb370 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff77c1c6740 10 client.0 _ll_register_callbacks cb 0x7ff76400fe10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff76bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073b9760 0x557907442be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff76bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073b9760 0x557907442be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43476/0 (socket says 172.21.15.38:43476) 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff76bfff640 1 -- 172.21.15.38:0/1532663953 learned_addr learned my addr 172.21.15.38:0/1532663953 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff7737fe640 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073cb370 0x5579073edf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff76bfff640 1 -- 172.21.15.38:0/1532663953 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073cb370 msgr2=0x5579073edf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff772ffd640 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x55790738d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff76bfff640 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073cb370 0x5579073edf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff76bfff640 1 -- 172.21.15.38:0/1532663953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 msgr2=0x55790738d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.299+0000 7ff76bfff640 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x55790738d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff76bfff640 1 -- 172.21.15.38:0/1532663953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557907443120 con 0x5579073b9760 2024-04-25T08:38:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff76bfff640 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073b9760 0x557907442be0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0898b0 tx=0x7ff75c01e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411a9bd76f715a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff751ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff751ffb640 1 -- 172.21.15.38:0/1532663953 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75c054330 con 0x5579073b9760 2024-04-25T08:38:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff751ffb640 1 -- 172.21.15.38:0/1532663953 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff75c05fe90 con 0x5579073b9760 2024-04-25T08:38:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1532663953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff76c007820 con 0x5579073b9760 2024-04-25T08:38:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff751ffb640 1 -- 172.21.15.38:0/1532663953 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff75c0544d0 con 0x5579073b9760 2024-04-25T08:38:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff77c1c6740 10 client.8049 fetch_fsmap learned FSMap version 57 2024-04-25T08:38:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1532663953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5579073e39f0 con 0x5579073b9760 2024-04-25T08:38:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff751ffb640 1 -- 172.21.15.38:0/1532663953 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c007820 con 0x5579073b9760 2024-04-25T08:38:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff751ffb640 1 -- 172.21.15.38:0/1532663953 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff75c06bbd0 con 0x5579073b9760 2024-04-25T08:38:45.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff77c1c6740 10 client.8049 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T08:38:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff77c1c6740 10 client.8049 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:38:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff77c1c6740 20 client.8049 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x5579073cd450 0x5579073b4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff77c1c6740 4 client.8049 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T08:38:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1532663953 --> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x5579072f9420 con 0x5579073cd450 2024-04-25T08:38:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.300+0000 7ff772ffd640 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x5579073cd450 0x5579073b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff751ffb640 10 client.8049 ms_handle_connect on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff772ffd640 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x5579073cd450 0x5579073b4390 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fe35bffefcdf08b server_cookie=70ba64b0f7696a0a in_seq=0 out_seq=0 2024-04-25T08:38:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff751ffb640 1 -- 172.21.15.38:0/1532663953 <== mds.0 v2:172.21.15.38:6833/2922734840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5579072f9420 con 0x5579073cd450 2024-04-25T08:38:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff751ffb640 10 client.8049 handle_command_reply: tid=0 2024-04-25T08:38:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff77c1c6740 1 client.8049 shutdown 2024-04-25T08:38:45.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1532663953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073b9760 msgr2=0x557907442be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073b9760 0x557907442be0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0898b0 tx=0x7ff75c01e070 comp rx=0 tx=0).stop 2024-04-25T08:38:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1532663953 shutdown_connections 2024-04-25T08:38:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5579073cb370 0x5579073edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x5579073cd450 0x5579073b4390 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5579073b9760 0x557907442be0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff77c1c6740 1 --2- 172.21.15.38:0/1532663953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55790738eca0 0x55790738d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff751ffb640 0 client.8049 ms_handle_reset on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1532663953 >> 172.21.15.38:0/1532663953 conn(0x557907356870 msgr2=0x5579073b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1532663953 shutdown_connections 2024-04-25T08:38:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff77c1c6740 1 -- 172.21.15.38:0/1532663953 wait complete. 2024-04-25T08:38:45.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.301+0000 7ff77c1c6740 20 client.8049 trim_cache size 0 max 16384 2024-04-25T08:38:45.303 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:38:45.303 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:38:45.303 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:38:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.302+0000 7ff7794c4640 1 -- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff75406b4d0 msgr2=0x7ff75406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.302+0000 7ff7794c4640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff75406b4d0 0x7ff75406d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff7640011e0 tx=0x7ff764009040 comp rx=0 tx=0).stop 2024-04-25T08:38:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.302+0000 7ff7794c4640 1 -- 172.21.15.38:0/748914748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740cc130 msgr2=0x7ff7740c6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.302+0000 7ff7794c4640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740cc130 0x7ff7740c6820 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c4b0 tx=0x7ff75c00c970 comp rx=0 tx=0).stop 2024-04-25T08:38:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.302+0000 7ff7794c4640 1 -- 172.21.15.38:0/748914748 shutdown_connections 2024-04-25T08:38:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.302+0000 7ff7794c4640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff77415bb90 0x7ff77415cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.302+0000 7ff7794c4640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7740cc130 0x7ff7740c6820 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.302+0000 7ff7794c4640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff75406b4d0 0x7ff75406d990 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.302+0000 7ff7794c4640 1 --2- 172.21.15.38:0/748914748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7740b13f0 0x7ff7740c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.302+0000 7ff7794c4640 1 -- 172.21.15.38:0/748914748 >> 172.21.15.38:0/748914748 conn(0x7ff7740d0520 msgr2=0x7ff77406f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.302+0000 7ff7794c4640 1 -- 172.21.15.38:0/748914748 shutdown_connections 2024-04-25T08:38:45.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.302+0000 7ff7794c4640 1 -- 172.21.15.38:0/748914748 wait complete. 2024-04-25T08:38:45.306 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:38:45.306 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:38:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.414+0000 7ff2481ce640 1 Processor -- start 2024-04-25T08:38:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.415+0000 7ff2481ce640 1 -- start start 2024-04-25T08:38:45.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.415+0000 7ff2481ce640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff24006c700 0x7ff240162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.415+0000 7ff2481ce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff24006cae0 0x7ff24015e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.415+0000 7ff2481ce640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff24015ec20 con 0x7ff24015e580 2024-04-25T08:38:45.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.415+0000 7ff2481ce640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff240161c80 con 0x7ff24006c700 2024-04-25T08:38:45.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.415+0000 7ff2481ce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24015ef00 con 0x7ff24006cae0 2024-04-25T08:38:45.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.416+0000 7ff245769640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff24006cae0 0x7ff24015e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.416+0000 7ff245f6a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff24006c700 0x7ff240162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.416+0000 7ff24676b640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7ff24015e580 0x7ff2400cdf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:39562/0 (socket says 172.21.15.38:39562) 2024-04-25T08:38:45.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.416+0000 7ff24676b640 1 -- 172.21.15.38:0/3225901297 learned_addr learned my addr 172.21.15.38:0/3225901297 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:45.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.416+0000 7ff245f6a640 1 -- 172.21.15.38:0/3225901297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff24006cae0 msgr2=0x7ff24015e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.416+0000 7ff245f6a640 1 --2- 172.21.15.38:0/3225901297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff24006cae0 0x7ff24015e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.416+0000 7ff245f6a640 1 -- 172.21.15.38:0/3225901297 >> v1:172.21.15.107:6789/0 conn(0x7ff24015e580 legacy=0x7ff2400cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.416+0000 7ff245f6a640 1 -- 172.21.15.38:0/3225901297 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff240161630 con 0x7ff24006c700 2024-04-25T08:38:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.416+0000 7ff244f68640 1 -- 172.21.15.38:0/3225901297 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953137290 0 0) 0x7ff24015ec20 con 0x7ff24015e580 2024-04-25T08:38:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.416+0000 7ff245769640 1 --2- 172.21.15.38:0/3225901297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff24006cae0 0x7ff24015e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.417+0000 7ff245f6a640 1 --2- 172.21.15.38:0/3225901297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff24006c700 0x7ff240162110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff2380097a0 tx=0x7ff23802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=466c8ba466d6d9e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.417+0000 7ff244f68640 1 -- 172.21.15.38:0/3225901297 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238037070 con 0x7ff24006c700 2024-04-25T08:38:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.417+0000 7ff244f68640 1 -- 172.21.15.38:0/3225901297 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff238032440 con 0x7ff24006c700 2024-04-25T08:38:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.417+0000 7ff244f68640 1 -- 172.21.15.38:0/3225901297 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238032810 con 0x7ff24006c700 2024-04-25T08:38:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.417+0000 7ff2481ce640 1 -- 172.21.15.38:0/3225901297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff24006c700 msgr2=0x7ff240162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.417+0000 7ff2481ce640 1 --2- 172.21.15.38:0/3225901297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff24006c700 0x7ff240162110 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff2380097a0 tx=0x7ff23802a180 comp rx=0 tx=0).stop 2024-04-25T08:38:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.418+0000 7ff2481ce640 1 -- 172.21.15.38:0/3225901297 shutdown_connections 2024-04-25T08:38:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.418+0000 7ff2481ce640 1 --2- 172.21.15.38:0/3225901297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff24006cae0 0x7ff24015e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.418+0000 7ff2481ce640 1 --2- 172.21.15.38:0/3225901297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff24006c700 0x7ff240162110 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.418+0000 7ff2481ce640 1 -- 172.21.15.38:0/3225901297 >> 172.21.15.38:0/3225901297 conn(0x7ff24008b000 msgr2=0x7ff24008b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.418+0000 7ff2481ce640 1 -- 172.21.15.38:0/3225901297 shutdown_connections 2024-04-25T08:38:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.418+0000 7ff2481ce640 1 -- 172.21.15.38:0/3225901297 wait complete. 2024-04-25T08:38:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.419+0000 7ff2481ce640 1 Processor -- start 2024-04-25T08:38:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.419+0000 7ff2481ce640 1 -- start start 2024-04-25T08:38:45.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff2481ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff24006c700 0x7ff24014dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff2481ce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff24006cae0 0x7ff2400b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff2481ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff24015e580 0x7ff2401587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff2481ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24006cf60 con 0x7ff24015e580 2024-04-25T08:38:45.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff2481ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24006c190 con 0x7ff24006c700 2024-04-25T08:38:45.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff2481ce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24015b920 con 0x7ff24006cae0 2024-04-25T08:38:45.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff24676b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff24015e580 0x7ff2401587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff24676b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff24015e580 0x7ff2401587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43482/0 (socket says 172.21.15.38:43482) 2024-04-25T08:38:45.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff24676b640 1 -- 172.21.15.38:0/1812944353 learned_addr learned my addr 172.21.15.38:0/1812944353 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff245f6a640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff24006c700 0x7ff24014dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff245769640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff24006cae0 0x7ff2400b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff24676b640 1 -- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff24006cae0 msgr2=0x7ff2400b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff24676b640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff24006cae0 0x7ff2400b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff24676b640 1 -- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff24006c700 msgr2=0x7ff24014dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff24676b640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff24006c700 0x7ff24014dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff24676b640 1 -- 172.21.15.38:0/1812944353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24015d640 con 0x7ff24015e580 2024-04-25T08:38:45.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff24676b640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff24015e580 0x7ff2401587c0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c700 tx=0x7ff23000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff23e7fc640 1 -- 172.21.15.38:0/1812944353 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230018070 con 0x7ff24015e580 2024-04-25T08:38:45.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff23e7fc640 1 -- 172.21.15.38:0/1812944353 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23000ce10 con 0x7ff24015e580 2024-04-25T08:38:45.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff23e7fc640 1 -- 172.21.15.38:0/1812944353 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000a860 con 0x7ff24015e580 2024-04-25T08:38:45.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.420+0000 7ff2481ce640 1 -- 172.21.15.38:0/1812944353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24015d920 con 0x7ff24015e580 2024-04-25T08:38:45.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.421+0000 7ff2481ce640 1 -- 172.21.15.38:0/1812944353 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2400b0f80 con 0x7ff24015e580 2024-04-25T08:38:45.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.422+0000 7ff23e7fc640 1 -- 172.21.15.38:0/1812944353 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff23001b070 con 0x7ff24015e580 2024-04-25T08:38:45.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.422+0000 7ff23e7fc640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff22c06b500 0x7ff22c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.423+0000 7ff23e7fc640 1 -- 172.21.15.38:0/1812944353 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff2300898b0 con 0x7ff24015e580 2024-04-25T08:38:45.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.423+0000 7ff245f6a640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff22c06b500 0x7ff22c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.423+0000 7ff245f6a640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff22c06b500 0x7ff22c06d9c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff238034000 tx=0x7ff23802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.424+0000 7ff24afac740 1 Processor -- start 2024-04-25T08:38:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.424+0000 7ff24afac740 1 -- start start 2024-04-25T08:38:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.424+0000 7ff24afac740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 0x56206c8e3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.424+0000 7ff24afac740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 0x56206c8e3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.424+0000 7ff24afac740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8b3770 0x56206c8b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.424+0000 7ff24afac740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206c81cda0 con 0x56206c8e0b10 2024-04-25T08:38:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.424+0000 7ff24afac740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206c7c5b80 con 0x56206c8b3770 2024-04-25T08:38:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.424+0000 7ff24afac740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56206c818e40 con 0x56206c8e0ef0 2024-04-25T08:38:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.424+0000 7ff245f6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 0x56206c8e3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.424+0000 7ff245f6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 0x56206c8e3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43498/0 (socket says 172.21.15.38:43498) 2024-04-25T08:38:45.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.424+0000 7ff245f6a640 1 -- 172.21.15.38:0/312493904 learned_addr learned my addr 172.21.15.38:0/312493904 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.424+0000 7ff24676b640 1 --2- 172.21.15.38:0/312493904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8b3770 0x56206c8b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.425+0000 7ff245769640 1 --2- 172.21.15.38:0/312493904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 0x56206c8e3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.425+0000 7ff245f6a640 1 -- 172.21.15.38:0/312493904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 msgr2=0x56206c8e3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.425+0000 7ff245f6a640 1 --2- 172.21.15.38:0/312493904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 0x56206c8e3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.425+0000 7ff245f6a640 1 -- 172.21.15.38:0/312493904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8b3770 msgr2=0x56206c8b6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.425+0000 7ff245f6a640 1 --2- 172.21.15.38:0/312493904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8b3770 0x56206c8b6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.425+0000 7ff245f6a640 1 -- 172.21.15.38:0/312493904 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56206c7cff10 con 0x56206c8e0b10 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.425+0000 7ff245f6a640 1 --2- 172.21.15.38:0/312493904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 0x56206c8e3320 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff238000c00 tx=0x7ff23802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5adb36a6714315 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.425+0000 7ff2277fe640 1 -- 172.21.15.38:0/312493904 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23804d070 con 0x56206c8e0b10 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.425+0000 7ff2277fe640 1 -- 172.21.15.38:0/312493904 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff238032c40 con 0x56206c8e0b10 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.425+0000 7ff2277fe640 1 -- 172.21.15.38:0/312493904 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238037040 con 0x56206c8e0b10 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.426+0000 7ff24afac740 1 -- 172.21.15.38:0/312493904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 msgr2=0x56206c8e3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.426+0000 7ff24afac740 1 --2- 172.21.15.38:0/312493904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 0x56206c8e3320 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff238000c00 tx=0x7ff23802c000 comp rx=0 tx=0).stop 2024-04-25T08:38:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.426+0000 7ff24afac740 1 -- 172.21.15.38:0/312493904 shutdown_connections 2024-04-25T08:38:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.426+0000 7ff24afac740 1 --2- 172.21.15.38:0/312493904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8b3770 0x56206c8b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.426+0000 7ff24afac740 1 --2- 172.21.15.38:0/312493904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 0x56206c8e3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.426+0000 7ff24afac740 1 --2- 172.21.15.38:0/312493904 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 0x56206c8e3320 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.426+0000 7ff24afac740 1 -- 172.21.15.38:0/312493904 >> 172.21.15.38:0/312493904 conn(0x56206c8b59e0 msgr2=0x56206c8fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.426+0000 7ff24afac740 1 -- 172.21.15.38:0/312493904 shutdown_connections 2024-04-25T08:38:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.426+0000 7ff24afac740 1 -- 172.21.15.38:0/312493904 wait complete. 2024-04-25T08:38:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.426+0000 7ff24afac740 1 -- start start 2024-04-25T08:38:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.426+0000 7ff24afac740 1 Processor -- start 2024-04-25T08:38:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.427+0000 7ff24afac740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 0x56206c8e7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.427+0000 7ff24afac740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 0x56206c90e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.427+0000 7ff24afac740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8d64d0 0x56206c90e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.427+0000 7ff24afac740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206c8b8b90 con 0x56206c8e0b10 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.427+0000 7ff24afac740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206c7bb4b0 con 0x56206c8d64d0 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.427+0000 7ff24afac740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56206c8b8e70 con 0x56206c8e0ef0 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.427+0000 7ff24afac740 10 client.0 _ll_register_callbacks cb 0x56206c7fb180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.427+0000 7ff245f6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 0x56206c8e7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.427+0000 7ff245f6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 0x56206c8e7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43514/0 (socket says 172.21.15.38:43514) 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.427+0000 7ff245f6a640 1 -- 172.21.15.38:0/1035670919 learned_addr learned my addr 172.21.15.38:0/1035670919 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.427+0000 7ff245769640 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 0x56206c90e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.427+0000 7ff24676b640 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8d64d0 0x56206c90e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff245f6a640 1 -- 172.21.15.38:0/1035670919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 msgr2=0x56206c90e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff245f6a640 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 0x56206c90e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff245f6a640 1 -- 172.21.15.38:0/1035670919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8d64d0 msgr2=0x56206c90e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff245f6a640 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8d64d0 0x56206c90e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff245f6a640 1 -- 172.21.15.38:0/1035670919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56206c5fb7d0 con 0x56206c8e0b10 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff245f6a640 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 0x56206c8e7170 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff23802a770 tx=0x7ff238004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673dba04ee0a0697 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff224ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff224ff9640 1 -- 172.21.15.38:0/1035670919 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23804d070 con 0x56206c8e0b10 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff224ff9640 1 -- 172.21.15.38:0/1035670919 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff238003610 con 0x56206c8e0b10 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff24afac740 1 -- 172.21.15.38:0/1035670919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff238032c40 con 0x56206c8e0b10 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff224ff9640 1 -- 172.21.15.38:0/1035670919 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff238003810 con 0x56206c8e0b10 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff24afac740 10 client.8058 fetch_fsmap learned FSMap version 57 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff24afac740 1 -- 172.21.15.38:0/1035670919 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x56206c5313f0 con 0x56206c8e0b10 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff224ff9640 1 -- 172.21.15.38:0/1035670919 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff238032c40 con 0x56206c8e0b10 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff224ff9640 1 -- 172.21.15.38:0/1035670919 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff23805f080 con 0x56206c8e0b10 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff24afac740 10 client.8058 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T08:38:45.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff24afac740 10 client.8058 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:38:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff24afac740 20 client.8058 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff24afac740 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56206c901130 0x56206c901510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff24afac740 4 client.8058 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:38:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.428+0000 7ff24afac740 1 -- 172.21.15.38:0/1035670919 --> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56206c82a2e0 con 0x56206c901130 2024-04-25T08:38:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.429+0000 7ff245769640 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56206c901130 0x56206c901510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.429+0000 7ff224ff9640 10 client.8058 ms_handle_connect on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.429+0000 7ff245769640 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56206c901130 0x56206c901510 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69ada2cae680a453 server_cookie=6b260460ed3c9d31 in_seq=0 out_seq=0 2024-04-25T08:38:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.429+0000 7ff224ff9640 1 -- 172.21.15.38:0/1035670919 <== mds.0 v2:172.21.15.38:6833/2922734840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56206c82a2e0 con 0x56206c901130 2024-04-25T08:38:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.429+0000 7ff224ff9640 10 client.8058 handle_command_reply: tid=0 2024-04-25T08:38:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.429+0000 7ff24afac740 1 client.8058 shutdown 2024-04-25T08:38:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.430+0000 7ff24afac740 1 -- 172.21.15.38:0/1035670919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 msgr2=0x56206c8e7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.430+0000 7ff24afac740 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 0x56206c8e7170 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff23802a770 tx=0x7ff238004730 comp rx=0 tx=0).stop 2024-04-25T08:38:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.430+0000 7ff24afac740 1 -- 172.21.15.38:0/1035670919 shutdown_connections 2024-04-25T08:38:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.430+0000 7ff24afac740 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8d64d0 0x56206c90e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.430+0000 7ff24afac740 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 0x56206c90e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.430+0000 7ff24afac740 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56206c901130 0x56206c901510 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.430+0000 7ff224ff9640 0 client.8058 ms_handle_reset on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.430+0000 7ff24afac740 1 --2- 172.21.15.38:0/1035670919 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c8e0b10 0x56206c8e7170 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.430+0000 7ff24afac740 1 -- 172.21.15.38:0/1035670919 >> 172.21.15.38:0/1035670919 conn(0x56206c8b3770 msgr2=0x56206c8b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.430+0000 7ff24afac740 1 -- 172.21.15.38:0/1035670919 shutdown_connections 2024-04-25T08:38:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.430+0000 7ff24afac740 1 -- 172.21.15.38:0/1035670919 wait complete. 2024-04-25T08:38:45.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.430+0000 7ff24afac740 20 client.8058 trim_cache size 0 max 16384 2024-04-25T08:38:45.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.445+0000 7ff24afac740 1 Processor -- start 2024-04-25T08:38:45.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.445+0000 7ff24afac740 1 -- start start 2024-04-25T08:38:45.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24afac740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8d64d0 0x56206c8dfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24afac740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8e0ef0 0x56206c903340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24afac740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 0x56206c8b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24afac740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2402002c0 con 0x56206c91fa80 2024-04-25T08:38:45.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24afac740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2401ff4a0 con 0x56206c8e0ef0 2024-04-25T08:38:45.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24afac740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400cd700 con 0x56206c8d64d0 2024-04-25T08:38:45.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24676b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 0x56206c8b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24676b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 0x56206c8b72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43522/0 (socket says 172.21.15.38:43522) 2024-04-25T08:38:45.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24676b640 1 -- 172.21.15.38:0/3518809284 learned_addr learned my addr 172.21.15.38:0/3518809284 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff245769640 1 --2- 172.21.15.38:0/3518809284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8e0ef0 0x56206c903340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff245f6a640 1 --2- 172.21.15.38:0/3518809284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8d64d0 0x56206c8dfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24676b640 1 -- 172.21.15.38:0/3518809284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8d64d0 msgr2=0x56206c8dfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24676b640 1 --2- 172.21.15.38:0/3518809284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8d64d0 0x56206c8dfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24676b640 1 -- 172.21.15.38:0/3518809284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8e0ef0 msgr2=0x56206c903340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24676b640 1 --2- 172.21.15.38:0/3518809284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8e0ef0 0x56206c903340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24676b640 1 -- 172.21.15.38:0/3518809284 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56206c4f10f0 con 0x56206c91fa80 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff24676b640 1 --2- 172.21.15.38:0/3518809284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 0x56206c8b72d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c6d0 tx=0x7ff230009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67987026f3a1696 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff226ffd640 1 -- 172.21.15.38:0/3518809284 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230086070 con 0x56206c91fa80 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff226ffd640 1 -- 172.21.15.38:0/3518809284 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230056230 con 0x56206c91fa80 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.446+0000 7ff226ffd640 1 -- 172.21.15.38:0/3518809284 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23005e0c0 con 0x56206c91fa80 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.447+0000 7ff24afac740 1 -- 172.21.15.38:0/3518809284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 msgr2=0x56206c8b72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.447+0000 7ff24afac740 1 --2- 172.21.15.38:0/3518809284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 0x56206c8b72d0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c6d0 tx=0x7ff230009040 comp rx=0 tx=0).stop 2024-04-25T08:38:45.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.447+0000 7ff24afac740 1 -- 172.21.15.38:0/3518809284 shutdown_connections 2024-04-25T08:38:45.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.447+0000 7ff24afac740 1 --2- 172.21.15.38:0/3518809284 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 0x56206c8b72d0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.447+0000 7ff24afac740 1 --2- 172.21.15.38:0/3518809284 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8e0ef0 0x56206c903340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.447+0000 7ff24afac740 1 --2- 172.21.15.38:0/3518809284 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8d64d0 0x56206c8dfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.447+0000 7ff24afac740 1 -- 172.21.15.38:0/3518809284 >> 172.21.15.38:0/3518809284 conn(0x56206c8b3770 msgr2=0x56206c901a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.447+0000 7ff24afac740 1 -- 172.21.15.38:0/3518809284 shutdown_connections 2024-04-25T08:38:45.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.447+0000 7ff24afac740 1 -- 172.21.15.38:0/3518809284 wait complete. 2024-04-25T08:38:45.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.447+0000 7ff24afac740 1 -- start start 2024-04-25T08:38:45.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.448+0000 7ff24afac740 1 Processor -- start 2024-04-25T08:38:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.448+0000 7ff24afac740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8d64d0 0x56206c951cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.448+0000 7ff24afac740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 0x56206c9032b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.448+0000 7ff24afac740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 0x56206c9037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.448+0000 7ff24afac740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2401feed0 con 0x56206c91fa80 2024-04-25T08:38:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.448+0000 7ff24afac740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206c8fb4f0 con 0x56206c8d64d0 2024-04-25T08:38:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.448+0000 7ff24afac740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56206c90e450 con 0x56206c8e0ef0 2024-04-25T08:38:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24afac740 10 client.0 _ll_register_callbacks cb 0x7ff238035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24676b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 0x56206c9037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24676b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 0x56206c9037f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43534/0 (socket says 172.21.15.38:43534) 2024-04-25T08:38:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24676b640 1 -- 172.21.15.38:0/184477949 learned_addr learned my addr 172.21.15.38:0/184477949 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff245769640 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 0x56206c9032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff245f6a640 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8d64d0 0x56206c951cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24676b640 1 -- 172.21.15.38:0/184477949 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 msgr2=0x56206c9032b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24676b640 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 0x56206c9032b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24676b640 1 -- 172.21.15.38:0/184477949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8d64d0 msgr2=0x56206c951cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24676b640 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8d64d0 0x56206c951cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24676b640 1 -- 172.21.15.38:0/184477949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56206c90e230 con 0x56206c91fa80 2024-04-25T08:38:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24676b640 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 0x56206c9037f0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7ff230087d30 tx=0x7ff23006d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c603dd50376144ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff224ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff224ff9640 1 -- 172.21.15.38:0/184477949 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230086020 con 0x56206c91fa80 2024-04-25T08:38:45.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff224ff9640 1 -- 172.21.15.38:0/184477949 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff230057580 con 0x56206c91fa80 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24afac740 1 -- 172.21.15.38:0/184477949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff230056230 con 0x56206c91fa80 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff224ff9640 1 -- 172.21.15.38:0/184477949 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff230053c80 con 0x56206c91fa80 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24afac740 10 client.8064 fetch_fsmap learned FSMap version 57 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff24afac740 1 -- 172.21.15.38:0/184477949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x56206c8e75f0 con 0x56206c91fa80 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.449+0000 7ff224ff9640 1 -- 172.21.15.38:0/184477949 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230056230 con 0x56206c91fa80 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff224ff9640 1 -- 172.21.15.38:0/184477949 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff230053e20 con 0x56206c91fa80 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 10 client.8064 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 10 client.8064 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 20 client.8064 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56206c901510 0x56206c90cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 4 client.8064 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 1 -- 172.21.15.38:0/184477949 --> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56206c8b89e0 con 0x56206c901510 2024-04-25T08:38:45.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff245f6a640 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56206c901510 0x56206c90cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:45.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff224ff9640 10 client.8064 ms_handle_connect on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:45.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff245f6a640 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56206c901510 0x56206c90cc70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f5a8edda749aa75 server_cookie=e24356a6dcd1f9da in_seq=0 out_seq=0 2024-04-25T08:38:45.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff224ff9640 1 -- 172.21.15.38:0/184477949 <== mds.0 v2:172.21.15.38:6833/2922734840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x56206c8b89e0 con 0x56206c901510 2024-04-25T08:38:45.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff224ff9640 10 client.8064 handle_command_reply: tid=0 2024-04-25T08:38:45.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 1 client.8064 shutdown 2024-04-25T08:38:45.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 1 -- 172.21.15.38:0/184477949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 msgr2=0x56206c9037f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 0x56206c9037f0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7ff230087d30 tx=0x7ff23006d040 comp rx=0 tx=0).stop 2024-04-25T08:38:45.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 1 -- 172.21.15.38:0/184477949 shutdown_connections 2024-04-25T08:38:45.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x56206c901510 0x56206c90cc70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56206c91fa80 0x56206c9037f0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56206c8e0ef0 0x56206c9032b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 1 --2- 172.21.15.38:0/184477949 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56206c8d64d0 0x56206c951cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff224ff9640 0 client.8064 ms_handle_reset on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:45.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.450+0000 7ff24afac740 1 -- 172.21.15.38:0/184477949 >> 172.21.15.38:0/184477949 conn(0x56206c8b3770 msgr2=0x56206c901a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff24afac740 1 -- 172.21.15.38:0/184477949 shutdown_connections 2024-04-25T08:38:45.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff24afac740 1 -- 172.21.15.38:0/184477949 wait complete. 2024-04-25T08:38:45.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff24afac740 20 client.8064 trim_cache size 0 max 16384 2024-04-25T08:38:45.451 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:38:45.451 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:38:45.451 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-04-25T08:38:45.451 INFO:teuthology.orchestra.run.smithi038.stdout: "8a9873f7-d2fe-407c-a005-933a922070ef": { 2024-04-25T08:38:45.451 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/scrub_control_test_path", 2024-04-25T08:38:45.452 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "8a9873f7-d2fe-407c-a005-933a922070ef", 2024-04-25T08:38:45.452 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive" 2024-04-25T08:38:45.452 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:45.452 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:38:45.452 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:38:45.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff2481ce640 1 -- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff22c06b500 msgr2=0x7ff22c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff2481ce640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff22c06b500 0x7ff22c06d9c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff238034000 tx=0x7ff23802e000 comp rx=0 tx=0).stop 2024-04-25T08:38:45.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff2481ce640 1 -- 172.21.15.38:0/1812944353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff24015e580 msgr2=0x7ff2401587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:45.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff2481ce640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff24015e580 0x7ff2401587c0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c700 tx=0x7ff23000cbc0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff2481ce640 1 -- 172.21.15.38:0/1812944353 shutdown_connections 2024-04-25T08:38:45.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff2481ce640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff24015e580 0x7ff2401587c0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff2481ce640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff22c06b500 0x7ff22c06d9c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff2481ce640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff24006cae0 0x7ff2400b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff2481ce640 1 --2- 172.21.15.38:0/1812944353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff24006c700 0x7ff24014dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:45.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff2481ce640 1 -- 172.21.15.38:0/1812944353 >> 172.21.15.38:0/1812944353 conn(0x7ff24008b000 msgr2=0x7ff24015f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:45.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff2481ce640 1 -- 172.21.15.38:0/1812944353 shutdown_connections 2024-04-25T08:38:45.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:45.451+0000 7ff2481ce640 1 -- 172.21.15.38:0/1812944353 wait complete. 2024-04-25T08:38:50.456 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:38:50.456 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:38:50.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.579+0000 7f9f418f5640 1 Processor -- start 2024-04-25T08:38:50.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.579+0000 7f9f418f5640 1 -- start start 2024-04-25T08:38:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.579+0000 7f9f418f5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9f3c15df30 0x7f9f3c09a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.579+0000 7f9f418f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f3c162390 0x7f9f3c099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.579+0000 7f9f418f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9f3c099fb0 0x7f9f3c06bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.579+0000 7f9f418f5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3c15e310 con 0x7f9f3c15df30 2024-04-25T08:38:50.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.579+0000 7f9f418f5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3c09aeb0 con 0x7f9f3c099fb0 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.579+0000 7f9f418f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c15ee70 con 0x7f9f3c162390 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.580+0000 7f9f3affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9f3c15df30 0x7f9f3c09a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.580+0000 7f9f3affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9f3c15df30 0x7f9f3c09a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50620/0 (socket says 172.21.15.38:50620) 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.580+0000 7f9f3affd640 1 -- 172.21.15.38:0/1779241541 learned_addr learned my addr 172.21.15.38:0/1779241541 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.580+0000 7f9f3b7fe640 1 --2- 172.21.15.38:0/1779241541 >> v2:172.21.15.107:3300/0 conn(0x7f9f3c099fb0 0x7f9f3c06bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.580+0000 7f9f3a7fc640 1 --2- 172.21.15.38:0/1779241541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f3c162390 0x7f9f3c099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.580+0000 7f9f3affd640 1 -- 172.21.15.38:0/1779241541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f3c162390 msgr2=0x7f9f3c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.580+0000 7f9f3affd640 1 --2- 172.21.15.38:0/1779241541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f3c162390 0x7f9f3c099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.580+0000 7f9f3affd640 1 -- 172.21.15.38:0/1779241541 >> v2:172.21.15.107:3300/0 conn(0x7f9f3c099fb0 msgr2=0x7f9f3c06bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.580+0000 7f9f3affd640 1 --2- 172.21.15.38:0/1779241541 >> v2:172.21.15.107:3300/0 conn(0x7f9f3c099fb0 0x7f9f3c06bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.580+0000 7f9f3affd640 1 -- 172.21.15.38:0/1779241541 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c15f300 con 0x7f9f3c15df30 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.580+0000 7f9f3b7fe640 1 --2- 172.21.15.38:0/1779241541 >> v2:172.21.15.107:3300/0 conn(0x7f9f3c099fb0 0x7f9f3c06bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.581+0000 7f9f3affd640 1 --2- 172.21.15.38:0/1779241541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f3c15df30 0x7f9f3c09a7b0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0d2680 tx=0x7f9f2c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70273ae47e7c323 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:50.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.581+0000 7f9f39ffb640 1 -- 172.21.15.38:0/1779241541 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c037070 con 0x7f9f3c15df30 2024-04-25T08:38:50.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.581+0000 7f9f39ffb640 1 -- 172.21.15.38:0/1779241541 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c032440 con 0x7f9f3c15df30 2024-04-25T08:38:50.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.581+0000 7f9f39ffb640 1 -- 172.21.15.38:0/1779241541 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c032810 con 0x7f9f3c15df30 2024-04-25T08:38:50.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.581+0000 7f9f418f5640 1 -- 172.21.15.38:0/1779241541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f3c15df30 msgr2=0x7f9f3c09a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.581+0000 7f9f418f5640 1 --2- 172.21.15.38:0/1779241541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f3c15df30 0x7f9f3c09a7b0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0d2680 tx=0x7f9f2c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.582+0000 7f9f418f5640 1 -- 172.21.15.38:0/1779241541 shutdown_connections 2024-04-25T08:38:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.582+0000 7f9f418f5640 1 --2- 172.21.15.38:0/1779241541 >> v2:172.21.15.107:3300/0 conn(0x7f9f3c099fb0 0x7f9f3c06bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.582+0000 7f9f418f5640 1 --2- 172.21.15.38:0/1779241541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f3c162390 0x7f9f3c099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.582+0000 7f9f418f5640 1 --2- 172.21.15.38:0/1779241541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f3c15df30 0x7f9f3c09a7b0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.582+0000 7f9f418f5640 1 -- 172.21.15.38:0/1779241541 >> 172.21.15.38:0/1779241541 conn(0x7f9f3c0b3110 msgr2=0x7f9f3c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.582+0000 7f9f418f5640 1 -- 172.21.15.38:0/1779241541 shutdown_connections 2024-04-25T08:38:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.582+0000 7f9f418f5640 1 -- 172.21.15.38:0/1779241541 wait complete. 2024-04-25T08:38:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.583+0000 7f9f418f5640 1 Processor -- start 2024-04-25T08:38:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.583+0000 7f9f418f5640 1 -- start start 2024-04-25T08:38:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f418f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f3c099fb0 0x7f9f3c0c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f418f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f3c15df30 0x7f9f3c1e7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f418f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f3c162390 0x7f9f3c1607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f418f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c15e820 con 0x7f9f3c099fb0 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f418f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c15e540 con 0x7f9f3c162390 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f418f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c15eb00 con 0x7f9f3c15df30 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f3c099fb0 0x7f9f3c0c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f3c099fb0 0x7f9f3c0c90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50628/0 (socket says 172.21.15.38:50628) 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f3affd640 1 -- 172.21.15.38:0/906251722 learned_addr learned my addr 172.21.15.38:0/906251722 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f3a7fc640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f3c15df30 0x7f9f3c1e7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f3b7fe640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f3c162390 0x7f9f3c1607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f3affd640 1 -- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f3c15df30 msgr2=0x7f9f3c1e7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f3affd640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f3c15df30 0x7f9f3c1e7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f3affd640 1 -- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f3c162390 msgr2=0x7f9f3c1607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f3affd640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f3c162390 0x7f9f3c1607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f3affd640 1 -- 172.21.15.38:0/906251722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c1e7f60 con 0x7f9f3c099fb0 2024-04-25T08:38:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.584+0000 7f9f3b7fe640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f3c162390 0x7f9f3c1607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.585+0000 7f9f3affd640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f3c099fb0 0x7f9f3c0c90c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c000c00 tx=0x7f9f2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.585+0000 7f9f23fff640 1 -- 172.21.15.38:0/906251722 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c041020 con 0x7f9f3c099fb0 2024-04-25T08:38:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.585+0000 7f9f23fff640 1 -- 172.21.15.38:0/906251722 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c0306c0 con 0x7f9f3c099fb0 2024-04-25T08:38:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.585+0000 7f9f418f5640 1 -- 172.21.15.38:0/906251722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3c1e8240 con 0x7f9f3c099fb0 2024-04-25T08:38:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.585+0000 7f9f23fff640 1 -- 172.21.15.38:0/906251722 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c037040 con 0x7f9f3c099fb0 2024-04-25T08:38:50.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.585+0000 7f9f418f5640 1 -- 172.21.15.38:0/906251722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3c1e7c50 con 0x7f9f3c099fb0 2024-04-25T08:38:50.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.587+0000 7f9f23fff640 1 -- 172.21.15.38:0/906251722 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f2c044020 con 0x7f9f3c099fb0 2024-04-25T08:38:50.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.587+0000 7f9f23fff640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f2806b4d0 0x7f9f2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.588+0000 7f9f23fff640 1 -- 172.21.15.38:0/906251722 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9f2c0aaad0 con 0x7f9f3c099fb0 2024-04-25T08:38:50.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.588+0000 7f9f3a7fc640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f2806b4d0 0x7f9f2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.588+0000 7f9f3a7fc640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f2806b4d0 0x7f9f2806d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0c9600 tx=0x7f9f30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:50.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.590+0000 7f9f445f8740 1 Processor -- start 2024-04-25T08:38:50.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.590+0000 7f9f445f8740 1 -- start start 2024-04-25T08:38:50.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f445f8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44b10 0x55d582f47320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f445f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44ef0 0x55d582f47860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f445f8740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f17770 0x55d582f1af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f445f8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582e80da0 con 0x55d582f44b10 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f445f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582e29b80 con 0x55d582f44ef0 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f445f8740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d582e7ce40 con 0x55d582f17770 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44b10 0x55d582f47320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44b10 0x55d582f47320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50642/0 (socket says 172.21.15.38:50642) 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f3affd640 1 -- 172.21.15.38:0/1394758291 learned_addr learned my addr 172.21.15.38:0/1394758291 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f3a7fc640 1 --2- 172.21.15.38:0/1394758291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44ef0 0x55d582f47860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f3b7fe640 1 --2- 172.21.15.38:0/1394758291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f17770 0x55d582f1af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f3affd640 1 -- 172.21.15.38:0/1394758291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f17770 msgr2=0x55d582f1af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f3affd640 1 --2- 172.21.15.38:0/1394758291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f17770 0x55d582f1af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f3affd640 1 -- 172.21.15.38:0/1394758291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44ef0 msgr2=0x55d582f47860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.591+0000 7f9f3affd640 1 --2- 172.21.15.38:0/1394758291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44ef0 0x55d582f47860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.592+0000 7f9f3affd640 1 -- 172.21.15.38:0/1394758291 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d582e33f10 con 0x55d582f44b10 2024-04-25T08:38:50.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.592+0000 7f9f3affd640 1 --2- 172.21.15.38:0/1394758291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44b10 0x55d582f47320 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c035040 tx=0x7f9f2c078b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb47c71b5da67456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.592+0000 7f9f217fa640 1 -- 172.21.15.38:0/1394758291 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c07d2a0 con 0x55d582f44b10 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.592+0000 7f9f217fa640 1 -- 172.21.15.38:0/1394758291 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c081860 con 0x55d582f44b10 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.592+0000 7f9f217fa640 1 -- 172.21.15.38:0/1394758291 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0757c0 con 0x55d582f44b10 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.593+0000 7f9f445f8740 1 -- 172.21.15.38:0/1394758291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44b10 msgr2=0x55d582f47320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.593+0000 7f9f445f8740 1 --2- 172.21.15.38:0/1394758291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44b10 0x55d582f47320 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c035040 tx=0x7f9f2c078b70 comp rx=0 tx=0).stop 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.593+0000 7f9f445f8740 1 -- 172.21.15.38:0/1394758291 shutdown_connections 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.593+0000 7f9f445f8740 1 --2- 172.21.15.38:0/1394758291 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f17770 0x55d582f1af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.593+0000 7f9f445f8740 1 --2- 172.21.15.38:0/1394758291 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44ef0 0x55d582f47860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.593+0000 7f9f445f8740 1 --2- 172.21.15.38:0/1394758291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44b10 0x55d582f47320 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.593+0000 7f9f445f8740 1 -- 172.21.15.38:0/1394758291 >> 172.21.15.38:0/1394758291 conn(0x55d582f199e0 msgr2=0x55d582f62320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.593+0000 7f9f445f8740 1 -- 172.21.15.38:0/1394758291 shutdown_connections 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.593+0000 7f9f445f8740 1 -- 172.21.15.38:0/1394758291 wait complete. 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.593+0000 7f9f445f8740 1 -- start start 2024-04-25T08:38:50.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.593+0000 7f9f445f8740 1 Processor -- start 2024-04-25T08:38:50.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.594+0000 7f9f445f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 0x55d582f1c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.594+0000 7f9f445f8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44ef0 0x55d582f5e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.594+0000 7f9f445f8740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f16a00 0x55d582f487b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.594+0000 7f9f445f8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582e80da0 con 0x55d582f44ef0 2024-04-25T08:38:50.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.594+0000 7f9f445f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582e1f4b0 con 0x55d582f44b10 2024-04-25T08:38:50.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.594+0000 7f9f445f8740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d582f1cb90 con 0x55d582f16a00 2024-04-25T08:38:50.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.594+0000 7f9f445f8740 10 client.0 _ll_register_callbacks cb 0x55d582e5f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:50.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.594+0000 7f9f3a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44ef0 0x55d582f5e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.594+0000 7f9f3a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44ef0 0x55d582f5e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50658/0 (socket says 172.21.15.38:50658) 2024-04-25T08:38:50.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.594+0000 7f9f3a7fc640 1 -- 172.21.15.38:0/204112062 learned_addr learned my addr 172.21.15.38:0/204112062 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:50.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.594+0000 7f9f3affd640 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 0x55d582f1c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.594+0000 7f9f3b7fe640 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f16a00 0x55d582f487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f3a7fc640 1 -- 172.21.15.38:0/204112062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f16a00 msgr2=0x55d582f487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f3a7fc640 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f16a00 0x55d582f487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f3a7fc640 1 -- 172.21.15.38:0/204112062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 msgr2=0x55d582f1c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f3a7fc640 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 0x55d582f1c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f3a7fc640 1 -- 172.21.15.38:0/204112062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d582c5f7d0 con 0x55d582f44ef0 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f3a7fc640 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44ef0 0x55d582f5e370 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9f300167d0 tx=0x7f9f30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcad844894e4392 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f1a7fc640 1 -- 172.21.15.38:0/204112062 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30026020 con 0x55d582f44ef0 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f1a7fc640 1 -- 172.21.15.38:0/204112062 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9f300149b0 con 0x55d582f44ef0 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f445f8740 1 -- 172.21.15.38:0/204112062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f2c081860 con 0x55d582f44ef0 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f1a7fc640 1 -- 172.21.15.38:0/204112062 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f30030050 con 0x55d582f44ef0 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f445f8740 10 client.8076 fetch_fsmap learned FSMap version 57 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f445f8740 1 -- 172.21.15.38:0/204112062 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55d582b953f0 con 0x55d582f44ef0 2024-04-25T08:38:50.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.595+0000 7f9f1a7fc640 1 -- 172.21.15.38:0/204112062 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c081860 con 0x55d582f44ef0 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.596+0000 7f9f1a7fc640 1 -- 172.21.15.38:0/204112062 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9f30033020 con 0x55d582f44ef0 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.596+0000 7f9f445f8740 10 client.8076 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.596+0000 7f9f445f8740 10 client.8076 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.596+0000 7f9f445f8740 20 client.8076 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.596+0000 7f9f445f8740 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55d582f3b090 0x55d582f3b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.596+0000 7f9f445f8740 4 client.8076 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.596+0000 7f9f445f8740 1 -- 172.21.15.38:0/204112062 --> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d582e8e2e0 con 0x55d582f3b090 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.596+0000 7f9f3affd640 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55d582f3b090 0x55d582f3b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.596+0000 7f9f1a7fc640 10 client.8076 ms_handle_connect on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.596+0000 7f9f3affd640 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55d582f3b090 0x55d582f3b4c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=385297900b6f3cb2 server_cookie=693b369b3810cdfd in_seq=0 out_seq=0 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.597+0000 7f9f1a7fc640 1 -- 172.21.15.38:0/204112062 <== mds.0 v2:172.21.15.38:6833/2922734840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d582e8e2e0 con 0x55d582f3b090 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.597+0000 7f9f1a7fc640 10 client.8076 handle_command_reply: tid=0 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.597+0000 7f9f445f8740 1 client.8076 shutdown 2024-04-25T08:38:50.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.597+0000 7f9f445f8740 1 -- 172.21.15.38:0/204112062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44ef0 msgr2=0x55d582f5e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.597+0000 7f9f445f8740 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44ef0 0x55d582f5e370 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9f300167d0 tx=0x7f9f30004730 comp rx=0 tx=0).stop 2024-04-25T08:38:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.597+0000 7f9f445f8740 1 -- 172.21.15.38:0/204112062 shutdown_connections 2024-04-25T08:38:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.597+0000 7f9f445f8740 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f16a00 0x55d582f487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.597+0000 7f9f445f8740 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55d582f3b090 0x55d582f3b4c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.597+0000 7f9f445f8740 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f44ef0 0x55d582f5e370 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.597+0000 7f9f445f8740 1 --2- 172.21.15.38:0/204112062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 0x55d582f1c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.597+0000 7f9f445f8740 1 -- 172.21.15.38:0/204112062 >> 172.21.15.38:0/204112062 conn(0x55d582f17770 msgr2=0x55d582f47310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.597+0000 7f9f1a7fc640 0 client.8076 ms_handle_reset on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.598+0000 7f9f445f8740 1 -- 172.21.15.38:0/204112062 shutdown_connections 2024-04-25T08:38:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.598+0000 7f9f445f8740 1 -- 172.21.15.38:0/204112062 wait complete. 2024-04-25T08:38:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.598+0000 7f9f445f8740 20 client.8076 trim_cache size 0 max 16384 2024-04-25T08:38:50.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.613+0000 7f9f445f8740 1 Processor -- start 2024-04-25T08:38:50.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.613+0000 7f9f445f8740 1 -- start start 2024-04-25T08:38:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f445f8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 0x55d582f18730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f445f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 0x55d582f1cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f445f8740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f44ef0 0x55d582f3b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f445f8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c200300 con 0x55d582f16a00 2024-04-25T08:38:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f445f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c0c6740 con 0x55d582f44b10 2024-04-25T08:38:50.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f445f8740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c1f8300 con 0x55d582f44ef0 2024-04-25T08:38:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 0x55d582f18730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 0x55d582f18730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50660/0 (socket says 172.21.15.38:50660) 2024-04-25T08:38:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f3affd640 1 -- 172.21.15.38:0/1750949458 learned_addr learned my addr 172.21.15.38:0/1750949458 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f3affd640 1 -- 172.21.15.38:0/1750949458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f44ef0 msgr2=0x55d582f3b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f3affd640 1 --2- 172.21.15.38:0/1750949458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f44ef0 0x55d582f3b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f3affd640 1 -- 172.21.15.38:0/1750949458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 msgr2=0x55d582f1cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f3affd640 1 --2- 172.21.15.38:0/1750949458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 0x55d582f1cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.614+0000 7f9f3affd640 1 -- 172.21.15.38:0/1750949458 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d582b550f0 con 0x55d582f16a00 2024-04-25T08:38:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.615+0000 7f9f3affd640 1 --2- 172.21.15.38:0/1750949458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 0x55d582f18730 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0790e0 tx=0x7f9f2c097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52f8933b2f60c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.615+0000 7f9f20ff9640 1 -- 172.21.15.38:0/1750949458 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c09c070 con 0x55d582f16a00 2024-04-25T08:38:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.615+0000 7f9f20ff9640 1 -- 172.21.15.38:0/1750949458 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c077450 con 0x55d582f16a00 2024-04-25T08:38:50.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.615+0000 7f9f20ff9640 1 -- 172.21.15.38:0/1750949458 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c094070 con 0x55d582f16a00 2024-04-25T08:38:50.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.615+0000 7f9f445f8740 1 -- 172.21.15.38:0/1750949458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 msgr2=0x55d582f18730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.615+0000 7f9f445f8740 1 --2- 172.21.15.38:0/1750949458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 0x55d582f18730 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0790e0 tx=0x7f9f2c097040 comp rx=0 tx=0).stop 2024-04-25T08:38:50.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.615+0000 7f9f445f8740 1 -- 172.21.15.38:0/1750949458 shutdown_connections 2024-04-25T08:38:50.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.615+0000 7f9f445f8740 1 --2- 172.21.15.38:0/1750949458 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f44ef0 0x55d582f3b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.615+0000 7f9f445f8740 1 --2- 172.21.15.38:0/1750949458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 0x55d582f1cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.615+0000 7f9f445f8740 1 --2- 172.21.15.38:0/1750949458 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 0x55d582f18730 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.615+0000 7f9f445f8740 1 -- 172.21.15.38:0/1750949458 >> 172.21.15.38:0/1750949458 conn(0x55d582f17770 msgr2=0x55d582f1e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.616+0000 7f9f445f8740 1 -- 172.21.15.38:0/1750949458 shutdown_connections 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.616+0000 7f9f445f8740 1 -- 172.21.15.38:0/1750949458 wait complete. 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.616+0000 7f9f445f8740 1 -- start start 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.616+0000 7f9f445f8740 1 Processor -- start 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f445f8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 0x55d582f39070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f445f8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 0x55d582fcc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f445f8740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f44ef0 0x55d582f430a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f445f8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d582f1b5e0 con 0x55d582f16a00 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f445f8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c0c6140 con 0x55d582f44b10 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f445f8740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d582f5e3f0 con 0x55d582f44ef0 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f445f8740 10 client.0 _ll_register_callbacks cb 0x7f9f2c009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:38:50.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 0x55d582f39070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 0x55d582f39070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50672/0 (socket says 172.21.15.38:50672) 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3affd640 1 -- 172.21.15.38:0/3920611227 learned_addr learned my addr 172.21.15.38:0/3920611227 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3a7fc640 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 0x55d582fcc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3b7fe640 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f44ef0 0x55d582f430a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3affd640 1 -- 172.21.15.38:0/3920611227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f44ef0 msgr2=0x55d582f430a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3affd640 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f44ef0 0x55d582f430a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3affd640 1 -- 172.21.15.38:0/3920611227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 msgr2=0x55d582fcc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3affd640 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 0x55d582fcc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3affd640 1 -- 172.21.15.38:0/3920611227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d582fcc5f0 con 0x55d582f16a00 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3a7fc640 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 0x55d582fcc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3b7fe640 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f44ef0 0x55d582f430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.617+0000 7f9f3affd640 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 0x55d582f39070 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c078e50 tx=0x7f9f2c008070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b1542a97567aed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f1a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f1a7fc640 1 -- 172.21.15.38:0/3920611227 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c09c030 con 0x55d582f16a00 2024-04-25T08:38:50.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f1a7fc640 1 -- 172.21.15.38:0/3920611227 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9f2c0757c0 con 0x55d582f16a00 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f445f8740 1 -- 172.21.15.38:0/3920611227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9f2c077450 con 0x55d582f16a00 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f1a7fc640 1 -- 172.21.15.38:0/3920611227 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f2c094040 con 0x55d582f16a00 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f445f8740 10 client.8082 fetch_fsmap learned FSMap version 57 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f445f8740 1 -- 172.21.15.38:0/3920611227 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55d582fcc5f0 con 0x55d582f16a00 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f1a7fc640 1 -- 172.21.15.38:0/3920611227 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c077450 con 0x55d582f16a00 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f1a7fc640 1 -- 172.21.15.38:0/3920611227 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9f2c03a030 con 0x55d582f16a00 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f445f8740 10 client.8082 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f445f8740 10 client.8082 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f445f8740 20 client.8082 populate_metadata read hostname 'smithi038' 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f445f8740 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55d582f12f10 0x55d582f38390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f445f8740 4 client.8082 mds_command: new command op to 7854 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f445f8740 1 -- 172.21.15.38:0/3920611227 --> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d582e88630 con 0x55d582f12f10 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.618+0000 7f9f3a7fc640 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55d582f12f10 0x55d582f38390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f1a7fc640 10 client.8082 ms_handle_connect on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:50.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f3a7fc640 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55d582f12f10 0x55d582f38390 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=354e8dfc1b5db8d2 server_cookie=25fd0738f5d6ebbe in_seq=0 out_seq=0 2024-04-25T08:38:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f1a7fc640 1 -- 172.21.15.38:0/3920611227 <== mds.0 v2:172.21.15.38:6833/2922734840 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d582e88630 con 0x55d582f12f10 2024-04-25T08:38:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f1a7fc640 10 client.8082 handle_command_reply: tid=0 2024-04-25T08:38:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f445f8740 1 client.8082 shutdown 2024-04-25T08:38:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f445f8740 1 -- 172.21.15.38:0/3920611227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 msgr2=0x55d582f39070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f445f8740 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 0x55d582f39070 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c078e50 tx=0x7f9f2c008070 comp rx=0 tx=0).stop 2024-04-25T08:38:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f445f8740 1 -- 172.21.15.38:0/3920611227 shutdown_connections 2024-04-25T08:38:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f445f8740 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d582f44ef0 0x55d582f430a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f445f8740 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d582f44b10 0x55d582fcc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f445f8740 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.38:6833/2922734840,v1:172.21.15.38:6835/2922734840] conn(0x55d582f12f10 0x55d582f38390 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f445f8740 1 --2- 172.21.15.38:0/3920611227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d582f16a00 0x55d582f39070 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f1a7fc640 0 client.8082 ms_handle_reset on v2:172.21.15.38:6833/2922734840 2024-04-25T08:38:50.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f445f8740 1 -- 172.21.15.38:0/3920611227 >> 172.21.15.38:0/3920611227 conn(0x55d582f17770 msgr2=0x55d582f17c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:50.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f445f8740 1 -- 172.21.15.38:0/3920611227 shutdown_connections 2024-04-25T08:38:50.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f445f8740 1 -- 172.21.15.38:0/3920611227 wait complete. 2024-04-25T08:38:50.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f445f8740 20 client.8082 trim_cache size 0 max 16384 2024-04-25T08:38:50.619 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:38:50.619 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-04-25T08:38:50.619 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-04-25T08:38:50.619 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:38:50.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f418f5640 1 -- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f2806b4d0 msgr2=0x7f9f2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.619+0000 7f9f418f5640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f2806b4d0 0x7f9f2806d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0c9600 tx=0x7f9f30008040 comp rx=0 tx=0).stop 2024-04-25T08:38:50.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.620+0000 7f9f418f5640 1 -- 172.21.15.38:0/906251722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f3c099fb0 msgr2=0x7f9f3c0c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.620+0000 7f9f418f5640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f3c099fb0 0x7f9f3c0c90c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c000c00 tx=0x7f9f2c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.620+0000 7f9f418f5640 1 -- 172.21.15.38:0/906251722 shutdown_connections 2024-04-25T08:38:50.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.620+0000 7f9f418f5640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f3c162390 0x7f9f3c1607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.620+0000 7f9f418f5640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9f2806b4d0 0x7f9f2806d990 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.620+0000 7f9f418f5640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9f3c15df30 0x7f9f3c1e7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.620+0000 7f9f418f5640 1 --2- 172.21.15.38:0/906251722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9f3c099fb0 0x7f9f3c0c90c0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.620+0000 7f9f418f5640 1 -- 172.21.15.38:0/906251722 >> 172.21.15.38:0/906251722 conn(0x7f9f3c0b3110 msgr2=0x7f9f3c06f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:50.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.620+0000 7f9f418f5640 1 -- 172.21.15.38:0/906251722 shutdown_connections 2024-04-25T08:38:50.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.620+0000 7f9f418f5640 1 -- 172.21.15.38:0/906251722 wait complete. 2024-04-25T08:38:50.624 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:38:50.624 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:50.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.731+0000 7fbc48d10640 1 Processor -- start 2024-04-25T08:38:50.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.731+0000 7fbc48d10640 1 -- start start 2024-04-25T08:38:50.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc48d10640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc4406c110 0x7fbc4406c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc48d10640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbc4415df50 0x7fbc440ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc48d10640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbc440ce2e0 0x7fbc440d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc48d10640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc440cf2d0 con 0x7fbc4415df50 2024-04-25T08:38:50.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc48d10640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4406ca30 con 0x7fbc440ce2e0 2024-04-25T08:38:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc48d10640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc440ceff0 con 0x7fbc4406c110 2024-04-25T08:38:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc3bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbc4415df50 0x7fbc440ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc3bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbc4415df50 0x7fbc440ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50688/0 (socket says 172.21.15.38:50688) 2024-04-25T08:38:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc3bfff640 1 -- 172.21.15.38:0/2091944531 learned_addr learned my addr 172.21.15.38:0/2091944531 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc4259c640 1 --2- 172.21.15.38:0/2091944531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc4406c110 0x7fbc4406c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc42d9d640 1 --2- 172.21.15.38:0/2091944531 >> v2:172.21.15.107:3300/0 conn(0x7fbc440ce2e0 0x7fbc440d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc3bfff640 1 -- 172.21.15.38:0/2091944531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc4406c110 msgr2=0x7fbc4406c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc3bfff640 1 --2- 172.21.15.38:0/2091944531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc4406c110 0x7fbc4406c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc3bfff640 1 -- 172.21.15.38:0/2091944531 >> v2:172.21.15.107:3300/0 conn(0x7fbc440ce2e0 msgr2=0x7fbc440d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc3bfff640 1 --2- 172.21.15.38:0/2091944531 >> v2:172.21.15.107:3300/0 conn(0x7fbc440ce2e0 0x7fbc440d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.732+0000 7fbc3bfff640 1 -- 172.21.15.38:0/2091944531 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc440d1f70 con 0x7fbc4415df50 2024-04-25T08:38:50.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.733+0000 7fbc3bfff640 1 --2- 172.21.15.38:0/2091944531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc4415df50 0x7fbc440ce7a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fbc440b6480 tx=0x7fbc2c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8515e5a4ad00ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.733+0000 7fbc41d9b640 1 -- 172.21.15.38:0/2091944531 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c037070 con 0x7fbc4415df50 2024-04-25T08:38:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.733+0000 7fbc41d9b640 1 -- 172.21.15.38:0/2091944531 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c032440 con 0x7fbc4415df50 2024-04-25T08:38:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.733+0000 7fbc41d9b640 1 -- 172.21.15.38:0/2091944531 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c032810 con 0x7fbc4415df50 2024-04-25T08:38:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.733+0000 7fbc48d10640 1 -- 172.21.15.38:0/2091944531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc4415df50 msgr2=0x7fbc440ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.733+0000 7fbc48d10640 1 --2- 172.21.15.38:0/2091944531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc4415df50 0x7fbc440ce7a0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fbc440b6480 tx=0x7fbc2c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.734+0000 7fbc48d10640 1 -- 172.21.15.38:0/2091944531 shutdown_connections 2024-04-25T08:38:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.734+0000 7fbc48d10640 1 --2- 172.21.15.38:0/2091944531 >> v2:172.21.15.107:3300/0 conn(0x7fbc440ce2e0 0x7fbc440d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.734+0000 7fbc48d10640 1 --2- 172.21.15.38:0/2091944531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc4415df50 0x7fbc440ce7a0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.734+0000 7fbc48d10640 1 --2- 172.21.15.38:0/2091944531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc4406c110 0x7fbc4406c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.734+0000 7fbc48d10640 1 -- 172.21.15.38:0/2091944531 >> 172.21.15.38:0/2091944531 conn(0x7fbc440d3fb0 msgr2=0x7fbc440d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.734+0000 7fbc48d10640 1 -- 172.21.15.38:0/2091944531 shutdown_connections 2024-04-25T08:38:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.734+0000 7fbc48d10640 1 -- 172.21.15.38:0/2091944531 wait complete. 2024-04-25T08:38:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.735+0000 7fbc48d10640 1 Processor -- start 2024-04-25T08:38:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.735+0000 7fbc48d10640 1 -- start start 2024-04-25T08:38:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc48d10640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc4406c110 0x7fbc440cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc48d10640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc440ce2e0 0x7fbc440b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc48d10640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc4415df50 0x7fbc440c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc48d10640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc440cece0 con 0x7fbc4415df50 2024-04-25T08:38:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc48d10640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc440d2530 con 0x7fbc440ce2e0 2024-04-25T08:38:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc48d10640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc440d2f90 con 0x7fbc4406c110 2024-04-25T08:38:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc42d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc4415df50 0x7fbc440c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc42d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc4415df50 0x7fbc440c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50702/0 (socket says 172.21.15.38:50702) 2024-04-25T08:38:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc42d9d640 1 -- 172.21.15.38:0/2348438100 learned_addr learned my addr 172.21.15.38:0/2348438100 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc4259c640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc4406c110 0x7fbc440cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc3bfff640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc440ce2e0 0x7fbc440b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc42d9d640 1 -- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc4406c110 msgr2=0x7fbc440cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc42d9d640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc4406c110 0x7fbc440cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc42d9d640 1 -- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc440ce2e0 msgr2=0x7fbc440b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc42d9d640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc440ce2e0 0x7fbc440b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc42d9d640 1 -- 172.21.15.38:0/2348438100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc440cbdf0 con 0x7fbc4415df50 2024-04-25T08:38:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc42d9d640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc4415df50 0x7fbc440c8fe0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00efc0 tx=0x7fbc3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.736+0000 7fbc3affd640 1 -- 172.21.15.38:0/2348438100 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c018020 con 0x7fbc4415df50 2024-04-25T08:38:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.737+0000 7fbc3affd640 1 -- 172.21.15.38:0/2348438100 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c014480 con 0x7fbc4415df50 2024-04-25T08:38:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.737+0000 7fbc48d10640 1 -- 172.21.15.38:0/2348438100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc440cc0d0 con 0x7fbc4415df50 2024-04-25T08:38:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.737+0000 7fbc3affd640 1 -- 172.21.15.38:0/2348438100 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c014870 con 0x7fbc4415df50 2024-04-25T08:38:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.737+0000 7fbc48d10640 1 -- 172.21.15.38:0/2348438100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc440cc690 con 0x7fbc4415df50 2024-04-25T08:38:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.738+0000 7fbc48d10640 1 -- 172.21.15.38:0/2348438100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc441f4720 con 0x7fbc4415df50 2024-04-25T08:38:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.738+0000 7fbc3affd640 1 -- 172.21.15.38:0/2348438100 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc3c005d10 con 0x7fbc4415df50 2024-04-25T08:38:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.738+0000 7fbc3affd640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbc2406b500 0x7fbc2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.739+0000 7fbc3affd640 1 -- 172.21.15.38:0/2348438100 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc3c08b1d0 con 0x7fbc4415df50 2024-04-25T08:38:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.739+0000 7fbc4259c640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbc2406b500 0x7fbc2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.739+0000 7fbc4259c640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbc2406b500 0x7fbc2406d9c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009ac0 tx=0x7fbc34009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.743+0000 7fbc3affd640 1 -- 172.21.15.38:0/2348438100 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc441f4720 con 0x7fbc4415df50 2024-04-25T08:38:50.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.964+0000 7fbc48d10640 1 -- 172.21.15.38:0/2348438100 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc44068e80 con 0x7fbc4415df50 2024-04-25T08:38:50.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.965+0000 7fbc3affd640 1 -- 172.21.15.38:0/2348438100 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fbc3c022030 con 0x7fbc4415df50 2024-04-25T08:38:50.958 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:50.959 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3680979257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3680979257},{"type":"v1","addr":"172.21.15.107:6836","nonce":3680979257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/697909489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":697909489},{"type":"v1","addr":"172.21.15.107:6837","nonce":697909489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3083871141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3083871141},{"type":"v1","addr":"172.21.15.38:6834","nonce":3083871141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:26.095391+0000","modified":"2024-04-25T08:38:28.097807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/2922734840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":2922734840},{"type":"v1","addr":"172.21.15.38:6835","nonce":2922734840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:38:50.959 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 57 2024-04-25T08:38:50.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.966+0000 7fbc48d10640 1 -- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbc2406b500 msgr2=0x7fbc2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.966+0000 7fbc48d10640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbc2406b500 0x7fbc2406d9c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009ac0 tx=0x7fbc34009190 comp rx=0 tx=0).stop 2024-04-25T08:38:50.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.966+0000 7fbc48d10640 1 -- 172.21.15.38:0/2348438100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc4415df50 msgr2=0x7fbc440c8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:50.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.966+0000 7fbc48d10640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc4415df50 0x7fbc440c8fe0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00efc0 tx=0x7fbc3c00c490 comp rx=0 tx=0).stop 2024-04-25T08:38:50.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.967+0000 7fbc48d10640 1 -- 172.21.15.38:0/2348438100 shutdown_connections 2024-04-25T08:38:50.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.967+0000 7fbc48d10640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbc4415df50 0x7fbc440c8fe0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.967+0000 7fbc48d10640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc440ce2e0 0x7fbc440b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.967+0000 7fbc48d10640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbc2406b500 0x7fbc2406d9c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.967+0000 7fbc48d10640 1 --2- 172.21.15.38:0/2348438100 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc4406c110 0x7fbc440cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:50.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.967+0000 7fbc48d10640 1 -- 172.21.15.38:0/2348438100 >> 172.21.15.38:0/2348438100 conn(0x7fbc440d3fb0 msgr2=0x7fbc440b6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:50.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.967+0000 7fbc48d10640 1 -- 172.21.15.38:0/2348438100 shutdown_connections 2024-04-25T08:38:50.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:50.967+0000 7fbc48d10640 1 -- 172.21.15.38:0/2348438100 wait complete. 2024-04-25T08:38:50.971 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T08:38:50.971 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:38:51.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.077+0000 7fb1866ac640 1 Processor -- start 2024-04-25T08:38:51.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.077+0000 7fb1866ac640 1 -- start start 2024-04-25T08:38:51.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb1866ac640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb1800b5710 0x7fb180161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb1866ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb18015ddf0 0x7fb1800b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb1866ac640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb1800b5af0 0x7fb18006bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb1866ac640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1800b9930 con 0x7fb1800b5710 2024-04-25T08:38:51.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb1866ac640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb180161e00 con 0x7fb1800b5af0 2024-04-25T08:38:51.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb1866ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1801620e0 con 0x7fb18015ddf0 2024-04-25T08:38:51.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb17ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb1800b5710 0x7fb180161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb17ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb1800b5710 0x7fb180161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50718/0 (socket says 172.21.15.38:50718) 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb17ffff640 1 -- 172.21.15.38:0/1580851872 learned_addr learned my addr 172.21.15.38:0/1580851872 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb17ffff640 1 -- 172.21.15.38:0/1580851872 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb18015ddf0 msgr2=0x7fb1800b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb17f7fe640 1 --2- 172.21.15.38:0/1580851872 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb18015ddf0 0x7fb1800b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb184c49640 1 --2- 172.21.15.38:0/1580851872 >> v2:172.21.15.107:3300/0 conn(0x7fb1800b5af0 0x7fb18006bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb17ffff640 1 --2- 172.21.15.38:0/1580851872 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb18015ddf0 0x7fb1800b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb17ffff640 1 -- 172.21.15.38:0/1580851872 >> v2:172.21.15.107:3300/0 conn(0x7fb1800b5af0 msgr2=0x7fb18006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb17ffff640 1 --2- 172.21.15.38:0/1580851872 >> v2:172.21.15.107:3300/0 conn(0x7fb1800b5af0 0x7fb18006bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.078+0000 7fb17ffff640 1 -- 172.21.15.38:0/1580851872 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1801623c0 con 0x7fb1800b5710 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.079+0000 7fb17ffff640 1 --2- 172.21.15.38:0/1580851872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1800b5710 0x7fb180161700 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fb1800d1bd0 tx=0x7fb17402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1447c93a18b92f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.079+0000 7fb17effd640 1 -- 172.21.15.38:0/1580851872 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174037070 con 0x7fb1800b5710 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.079+0000 7fb17effd640 1 -- 172.21.15.38:0/1580851872 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb174032440 con 0x7fb1800b5710 2024-04-25T08:38:51.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.079+0000 7fb17effd640 1 -- 172.21.15.38:0/1580851872 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174032810 con 0x7fb1800b5710 2024-04-25T08:38:51.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.079+0000 7fb1866ac640 1 -- 172.21.15.38:0/1580851872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1800b5710 msgr2=0x7fb180161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.079+0000 7fb1866ac640 1 --2- 172.21.15.38:0/1580851872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1800b5710 0x7fb180161700 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fb1800d1bd0 tx=0x7fb17402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.080+0000 7fb1866ac640 1 -- 172.21.15.38:0/1580851872 shutdown_connections 2024-04-25T08:38:51.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.080+0000 7fb1866ac640 1 --2- 172.21.15.38:0/1580851872 >> v2:172.21.15.107:3300/0 conn(0x7fb1800b5af0 0x7fb18006bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.080+0000 7fb1866ac640 1 --2- 172.21.15.38:0/1580851872 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb18015ddf0 0x7fb1800b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.080+0000 7fb1866ac640 1 --2- 172.21.15.38:0/1580851872 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1800b5710 0x7fb180161700 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.080+0000 7fb1866ac640 1 -- 172.21.15.38:0/1580851872 >> 172.21.15.38:0/1580851872 conn(0x7fb1800d07c0 msgr2=0x7fb1800d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:51.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.080+0000 7fb1866ac640 1 -- 172.21.15.38:0/1580851872 shutdown_connections 2024-04-25T08:38:51.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.080+0000 7fb1866ac640 1 -- 172.21.15.38:0/1580851872 wait complete. 2024-04-25T08:38:51.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.081+0000 7fb1866ac640 1 Processor -- start 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.081+0000 7fb1866ac640 1 -- start start 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb1866ac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1800b5710 0x7fb1801e9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb1866ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1800b5af0 0x7fb1801e73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb1866ac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb18015ddf0 0x7fb18015b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb1866ac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18006c6b0 con 0x7fb1800b5710 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb1866ac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18006c530 con 0x7fb18015ddf0 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb1866ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18006c830 con 0x7fb1800b5af0 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1800b5710 0x7fb1801e9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1800b5710 0x7fb1801e9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50720/0 (socket says 172.21.15.38:50720) 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17ffff640 1 -- 172.21.15.38:0/3856760752 learned_addr learned my addr 172.21.15.38:0/3856760752 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17f7fe640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1800b5af0 0x7fb1801e73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb184c49640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb18015ddf0 0x7fb18015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17ffff640 1 -- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1800b5af0 msgr2=0x7fb1801e73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17ffff640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1800b5af0 0x7fb1801e73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17ffff640 1 -- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb18015ddf0 msgr2=0x7fb18015b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17ffff640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb18015ddf0 0x7fb18015b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17ffff640 1 -- 172.21.15.38:0/3856760752 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1801e78f0 con 0x7fb1800b5710 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb184c49640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb18015ddf0 0x7fb18015b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17f7fe640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1800b5af0 0x7fb1801e73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17ffff640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1800b5710 0x7fb1801e9e90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fb174000c00 tx=0x7fb1740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17cff9640 1 -- 172.21.15.38:0/3856760752 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174037040 con 0x7fb1800b5710 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17cff9640 1 -- 172.21.15.38:0/3856760752 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb174042070 con 0x7fb1800b5710 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb1866ac640 1 -- 172.21.15.38:0/3856760752 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1801e7b90 con 0x7fb1800b5710 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.082+0000 7fb17cff9640 1 -- 172.21.15.38:0/3856760752 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb174030bb0 con 0x7fb1800b5710 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.083+0000 7fb1866ac640 1 -- 172.21.15.38:0/3856760752 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1801e8150 con 0x7fb1800b5710 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.084+0000 7fb1866ac640 1 -- 172.21.15.38:0/3856760752 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb180068e80 con 0x7fb1800b5710 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.084+0000 7fb17cff9640 1 -- 172.21.15.38:0/3856760752 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb174047460 con 0x7fb1800b5710 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.084+0000 7fb17cff9640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb16c06b500 0x7fb16c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.085+0000 7fb17cff9640 1 -- 172.21.15.38:0/3856760752 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb1740aaf90 con 0x7fb1800b5710 2024-04-25T08:38:51.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.085+0000 7fb17f7fe640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb16c06b500 0x7fb16c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.085+0000 7fb17f7fe640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb16c06b500 0x7fb16c06d9c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb1800ca450 tx=0x7fb170008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:51.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.089+0000 7fb17cff9640 1 -- 172.21.15.38:0/3856760752 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb174045360 con 0x7fb1800b5710 2024-04-25T08:38:51.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.347+0000 7fb1866ac640 1 -- 172.21.15.38:0/3856760752 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb1801e8810 con 0x7fb1800b5710 2024-04-25T08:38:51.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.347+0000 7fb17cff9640 1 -- 172.21.15.38:0/3856760752 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1026 (secure 0 0 0) 0x7fb174079c80 con 0x7fb1800b5710 2024-04-25T08:38:51.341 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:51.341 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":460,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077689817,"bytes_used":2620907520,"bytes_avail":770473205760,"bytes_total":773094113280,"write_bytes_sec":65845544,"read_op_per_sec":0,"write_op_per_sec":65},"fsmap":{"epoch":57,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7854}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T08:38:15.298964+0000","services":{}},"progress_events":{}} 2024-04-25T08:38:51.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.350+0000 7fb1866ac640 1 -- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb16c06b500 msgr2=0x7fb16c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.350+0000 7fb1866ac640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb16c06b500 0x7fb16c06d9c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb1800ca450 tx=0x7fb170008040 comp rx=0 tx=0).stop 2024-04-25T08:38:51.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.350+0000 7fb1866ac640 1 -- 172.21.15.38:0/3856760752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1800b5710 msgr2=0x7fb1801e9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.350+0000 7fb1866ac640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1800b5710 0x7fb1801e9e90 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fb174000c00 tx=0x7fb1740046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.350+0000 7fb1866ac640 1 -- 172.21.15.38:0/3856760752 shutdown_connections 2024-04-25T08:38:51.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.350+0000 7fb1866ac640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb18015ddf0 0x7fb18015b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.350+0000 7fb1866ac640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb16c06b500 0x7fb16c06d9c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.350+0000 7fb1866ac640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1800b5af0 0x7fb1801e73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.350+0000 7fb1866ac640 1 --2- 172.21.15.38:0/3856760752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1800b5710 0x7fb1801e9e90 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.350+0000 7fb1866ac640 1 -- 172.21.15.38:0/3856760752 >> 172.21.15.38:0/3856760752 conn(0x7fb1800d07c0 msgr2=0x7fb1800b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:51.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.350+0000 7fb1866ac640 1 -- 172.21.15.38:0/3856760752 shutdown_connections 2024-04-25T08:38:51.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.350+0000 7fb1866ac640 1 -- 172.21.15.38:0/3856760752 wait complete. 2024-04-25T08:38:51.354 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:38:51.354 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:38:51.386 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:38:51.434 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:38:51.434 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:38:51.434 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:51.452 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:51.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.599+0000 7fda6c953640 1 Processor -- start 2024-04-25T08:38:51.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.599+0000 7fda6c953640 1 -- start start 2024-04-25T08:38:51.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.600+0000 7fda6c953640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fda6815dc10 0x7fda680ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.600+0000 7fda6c953640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda680ced60 0x7fda680d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.600+0000 7fda6c953640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda680d3e70 con 0x7fda680d3900 2024-04-25T08:38:51.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.600+0000 7fda6c953640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda6815dff0 con 0x7fda6815dc10 2024-04-25T08:38:51.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.600+0000 7fda6c953640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda680b6990 con 0x7fda680ced60 2024-04-25T08:38:51.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.601+0000 7fda65d9b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda680ced60 0x7fda680d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.601+0000 7fda6659c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fda6815dc10 0x7fda680ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.601+0000 7fda65d9b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda680ced60 0x7fda680d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:35596/0 (socket says 172.21.15.38:35596) 2024-04-25T08:38:51.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.601+0000 7fda65d9b640 1 -- 172.21.15.38:0/2695497656 learned_addr learned my addr 172.21.15.38:0/2695497656 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:51.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.601+0000 7fda65d9b640 1 -- 172.21.15.38:0/2695497656 >> v2:172.21.15.107:3300/0 conn(0x7fda6815dc10 msgr2=0x7fda680ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:51.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.601+0000 7fda65d9b640 1 --2- 172.21.15.38:0/2695497656 >> v2:172.21.15.107:3300/0 conn(0x7fda6815dc10 0x7fda680ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.601+0000 7fda65d9b640 1 -- 172.21.15.38:0/2695497656 >> v1:172.21.15.107:6789/0 conn(0x7fda680d3900 legacy=0x7fda680b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.601+0000 7fda65d9b640 1 -- 172.21.15.38:0/2695497656 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda680d4150 con 0x7fda680ced60 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.601+0000 7fda6659c640 1 --2- 172.21.15.38:0/2695497656 >> v2:172.21.15.107:3300/0 conn(0x7fda6815dc10 0x7fda680ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.602+0000 7fda65d9b640 1 --2- 172.21.15.38:0/2695497656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda680ced60 0x7fda680d0850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fda50009870 tx=0x7fda5002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43f2339f946ed6e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.602+0000 7fda6559a640 1 -- 172.21.15.38:0/2695497656 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50037070 con 0x7fda680ced60 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.602+0000 7fda6559a640 1 -- 172.21.15.38:0/2695497656 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda50032440 con 0x7fda680ced60 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.602+0000 7fda6559a640 1 -- 172.21.15.38:0/2695497656 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50032810 con 0x7fda680ced60 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.602+0000 7fda6c953640 1 -- 172.21.15.38:0/2695497656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda680ced60 msgr2=0x7fda680d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.602+0000 7fda6c953640 1 --2- 172.21.15.38:0/2695497656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda680ced60 0x7fda680d0850 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fda50009870 tx=0x7fda5002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.603+0000 7fda6c953640 1 -- 172.21.15.38:0/2695497656 shutdown_connections 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.603+0000 7fda6c953640 1 --2- 172.21.15.38:0/2695497656 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda680ced60 0x7fda680d0850 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.603+0000 7fda6c953640 1 --2- 172.21.15.38:0/2695497656 >> v2:172.21.15.107:3300/0 conn(0x7fda6815dc10 0x7fda680ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.603+0000 7fda6c953640 1 -- 172.21.15.38:0/2695497656 >> 172.21.15.38:0/2695497656 conn(0x7fda68066d10 msgr2=0x7fda68067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.603+0000 7fda6c953640 1 -- 172.21.15.38:0/2695497656 shutdown_connections 2024-04-25T08:38:51.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.603+0000 7fda6c953640 1 -- 172.21.15.38:0/2695497656 wait complete. 2024-04-25T08:38:51.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.604+0000 7fda6c953640 1 Processor -- start 2024-04-25T08:38:51.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.604+0000 7fda6c953640 1 -- start start 2024-04-25T08:38:51.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda6c953640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda680ced60 0x7fda680cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda6c953640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda680d3900 0x7fda680aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda6c953640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda6815dc10 0x7fda680ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda6c953640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda680b6bc0 con 0x7fda680d3900 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda6c953640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda680d4710 con 0x7fda680ced60 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda6c953640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda680d0d90 con 0x7fda6815dc10 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda65d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda680d3900 0x7fda680aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda65d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda680d3900 0x7fda680aa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50728/0 (socket says 172.21.15.38:50728) 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda65d9b640 1 -- 172.21.15.38:0/1606643509 learned_addr learned my addr 172.21.15.38:0/1606643509 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda66d9d640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda6815dc10 0x7fda680ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda6659c640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda680ced60 0x7fda680cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda65d9b640 1 -- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda6815dc10 msgr2=0x7fda680ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda65d9b640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda6815dc10 0x7fda680ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda65d9b640 1 -- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda680ced60 msgr2=0x7fda680cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda65d9b640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda680ced60 0x7fda680cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda65d9b640 1 -- 172.21.15.38:0/1606643509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda680cac20 con 0x7fda680d3900 2024-04-25T08:38:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.605+0000 7fda65d9b640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda680d3900 0x7fda680aa2f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fda50000c00 tx=0x7fda500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.606+0000 7fda5effd640 1 -- 172.21.15.38:0/1606643509 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50037040 con 0x7fda680d3900 2024-04-25T08:38:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.606+0000 7fda5effd640 1 -- 172.21.15.38:0/1606643509 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda50042070 con 0x7fda680d3900 2024-04-25T08:38:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.606+0000 7fda6c953640 1 -- 172.21.15.38:0/1606643509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda680caf00 con 0x7fda680d3900 2024-04-25T08:38:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.606+0000 7fda5effd640 1 -- 172.21.15.38:0/1606643509 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50030880 con 0x7fda680d3900 2024-04-25T08:38:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.606+0000 7fda6c953640 1 -- 172.21.15.38:0/1606643509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda680cb4c0 con 0x7fda680d3900 2024-04-25T08:38:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.607+0000 7fda6c953640 1 -- 172.21.15.38:0/1606643509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda68068e80 con 0x7fda680d3900 2024-04-25T08:38:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.607+0000 7fda5effd640 1 -- 172.21.15.38:0/1606643509 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda5002ad10 con 0x7fda680d3900 2024-04-25T08:38:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.608+0000 7fda5effd640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda4806b4d0 0x7fda4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.608+0000 7fda5effd640 1 -- 172.21.15.38:0/1606643509 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda500aa7a0 con 0x7fda680d3900 2024-04-25T08:38:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.608+0000 7fda6659c640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda4806b4d0 0x7fda4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.609+0000 7fda6659c640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda4806b4d0 0x7fda4806d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fda680b7f60 tx=0x7fda60009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.612+0000 7fda5effd640 1 -- 172.21.15.38:0/1606643509 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda50078e30 con 0x7fda680d3900 2024-04-25T08:38:51.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.793+0000 7fda6c953640 1 -- 172.21.15.38:0/1606643509 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda680b7440 con 0x7fda680d3900 2024-04-25T08:38:51.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.795+0000 7fda5effd640 1 -- 172.21.15.38:0/1606643509 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16873 (secure 0 0 0) 0x7fda5007ced0 con 0x7fda680d3900 2024-04-25T08:38:51.788 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:51.788 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":96,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:38:27.725413+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:24.377232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:25.069513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:51.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.796+0000 7fda6c953640 1 -- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda4806b4d0 msgr2=0x7fda4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.796+0000 7fda6c953640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda4806b4d0 0x7fda4806d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fda680b7f60 tx=0x7fda60009190 comp rx=0 tx=0).stop 2024-04-25T08:38:51.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.796+0000 7fda6c953640 1 -- 172.21.15.38:0/1606643509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda680d3900 msgr2=0x7fda680aa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.796+0000 7fda6c953640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda680d3900 0x7fda680aa2f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fda50000c00 tx=0x7fda500046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.796+0000 7fda6c953640 1 -- 172.21.15.38:0/1606643509 shutdown_connections 2024-04-25T08:38:51.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.796+0000 7fda6c953640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda4806b4d0 0x7fda4806d990 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.796+0000 7fda6c953640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fda6815dc10 0x7fda680ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.796+0000 7fda6c953640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda680d3900 0x7fda680aa2f0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.796+0000 7fda6c953640 1 --2- 172.21.15.38:0/1606643509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fda680ced60 0x7fda680cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.797+0000 7fda6c953640 1 -- 172.21.15.38:0/1606643509 >> 172.21.15.38:0/1606643509 conn(0x7fda68066d10 msgr2=0x7fda681605b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:51.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.797+0000 7fda6c953640 1 -- 172.21.15.38:0/1606643509 shutdown_connections 2024-04-25T08:38:51.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.797+0000 7fda6c953640 1 -- 172.21.15.38:0/1606643509 wait complete. 2024-04-25T08:38:51.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:38:24.377232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:38:25.069513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:38:51.801 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:38:51.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.911+0000 7f2d01570640 1 Processor -- start 2024-04-25T08:38:51.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.911+0000 7f2d01570640 1 -- start start 2024-04-25T08:38:51.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.912+0000 7f2d01570640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2cfc06c700 0x7f2cfc162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.912+0000 7f2d01570640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2cfc06cae0 0x7f2cfc15e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.912+0000 7f2d01570640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cfc15e580 0x7f2cfc0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.912+0000 7f2d01570640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cfc161c80 con 0x7f2cfc06c700 2024-04-25T08:38:51.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.912+0000 7f2d01570640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cfc15ec20 con 0x7f2cfc06cae0 2024-04-25T08:38:51.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.912+0000 7f2d01570640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc15ef00 con 0x7f2cfc15e580 2024-04-25T08:38:51.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.912+0000 7f2cfaffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2cfc06c700 0x7f2cfc162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.912+0000 7f2cfaffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2cfc06c700 0x7f2cfc162110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50742/0 (socket says 172.21.15.38:50742) 2024-04-25T08:38:51.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.912+0000 7f2cfaffd640 1 -- 172.21.15.38:0/826128416 learned_addr learned my addr 172.21.15.38:0/826128416 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:51.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.913+0000 7f2cfb7fe640 1 --2- 172.21.15.38:0/826128416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cfc15e580 0x7f2cfc0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.913+0000 7f2cfa7fc640 1 --2- 172.21.15.38:0/826128416 >> v2:172.21.15.107:3300/0 conn(0x7f2cfc06cae0 0x7f2cfc15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.913+0000 7f2cfaffd640 1 -- 172.21.15.38:0/826128416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cfc15e580 msgr2=0x7f2cfc0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:51.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.913+0000 7f2cfaffd640 1 --2- 172.21.15.38:0/826128416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cfc15e580 0x7f2cfc0cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.913+0000 7f2cfaffd640 1 -- 172.21.15.38:0/826128416 >> v2:172.21.15.107:3300/0 conn(0x7f2cfc06cae0 msgr2=0x7f2cfc15e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:51.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.913+0000 7f2cfaffd640 1 --2- 172.21.15.38:0/826128416 >> v2:172.21.15.107:3300/0 conn(0x7f2cfc06cae0 0x7f2cfc15e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.913+0000 7f2cfaffd640 1 -- 172.21.15.38:0/826128416 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc161630 con 0x7f2cfc06c700 2024-04-25T08:38:51.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.913+0000 7f2cfaffd640 1 --2- 172.21.15.38:0/826128416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cfc06c700 0x7f2cfc162110 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0b3050 tx=0x7f2cf402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626c3d76d6bf14f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:51.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.914+0000 7f2cf9ffb640 1 -- 172.21.15.38:0/826128416 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4037070 con 0x7f2cfc06c700 2024-04-25T08:38:51.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.914+0000 7f2cf9ffb640 1 -- 172.21.15.38:0/826128416 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf4032440 con 0x7f2cfc06c700 2024-04-25T08:38:51.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.914+0000 7f2cf9ffb640 1 -- 172.21.15.38:0/826128416 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4032810 con 0x7f2cfc06c700 2024-04-25T08:38:51.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.914+0000 7f2d01570640 1 -- 172.21.15.38:0/826128416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cfc06c700 msgr2=0x7f2cfc162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.914+0000 7f2d01570640 1 --2- 172.21.15.38:0/826128416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cfc06c700 0x7f2cfc162110 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc0b3050 tx=0x7f2cf402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.914+0000 7f2d01570640 1 -- 172.21.15.38:0/826128416 shutdown_connections 2024-04-25T08:38:51.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.914+0000 7f2d01570640 1 --2- 172.21.15.38:0/826128416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cfc15e580 0x7f2cfc0cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.914+0000 7f2d01570640 1 --2- 172.21.15.38:0/826128416 >> v2:172.21.15.107:3300/0 conn(0x7f2cfc06cae0 0x7f2cfc15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.914+0000 7f2d01570640 1 --2- 172.21.15.38:0/826128416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cfc06c700 0x7f2cfc162110 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.914+0000 7f2d01570640 1 -- 172.21.15.38:0/826128416 >> 172.21.15.38:0/826128416 conn(0x7f2cfc08b000 msgr2=0x7f2cfc08b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:51.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.914+0000 7f2d01570640 1 -- 172.21.15.38:0/826128416 shutdown_connections 2024-04-25T08:38:51.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.914+0000 7f2d01570640 1 -- 172.21.15.38:0/826128416 wait complete. 2024-04-25T08:38:51.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.916+0000 7f2d01570640 1 Processor -- start 2024-04-25T08:38:51.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.916+0000 7f2d01570640 1 -- start start 2024-04-25T08:38:51.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2d01570640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cfc06c700 0x7f2cfc1f67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2d01570640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cfc06cae0 0x7f2cfc0cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2d01570640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cfc15e580 0x7f2cfc15cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2d01570640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc06cf90 con 0x7f2cfc06c700 2024-04-25T08:38:51.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2d01570640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc06c190 con 0x7f2cfc15e580 2024-04-25T08:38:51.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2d01570640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc15b920 con 0x7f2cfc06cae0 2024-04-25T08:38:51.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2cfaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cfc06c700 0x7f2cfc1f67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2cfaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cfc06c700 0x7f2cfc1f67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50748/0 (socket says 172.21.15.38:50748) 2024-04-25T08:38:51.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2cfaffd640 1 -- 172.21.15.38:0/387646805 learned_addr learned my addr 172.21.15.38:0/387646805 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2cfa7fc640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cfc06cae0 0x7f2cfc0cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2cfb7fe640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cfc15e580 0x7f2cfc15cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2cfaffd640 1 -- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cfc06cae0 msgr2=0x7f2cfc0cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2cfaffd640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cfc06cae0 0x7f2cfc0cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2cfaffd640 1 -- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cfc15e580 msgr2=0x7f2cfc15cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2cfaffd640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cfc15e580 0x7f2cfc15cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2cfaffd640 1 -- 172.21.15.38:0/387646805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc1f6cf0 con 0x7f2cfc06c700 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2cfaffd640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cfc06c700 0x7f2cfc1f67b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f2cf402a220 tx=0x7f2cf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2ce37fe640 1 -- 172.21.15.38:0/387646805 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4041020 con 0x7f2cfc06c700 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2ce37fe640 1 -- 172.21.15.38:0/387646805 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf40306c0 con 0x7f2cfc06c700 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2d01570640 1 -- 172.21.15.38:0/387646805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cfc0cc7d0 con 0x7f2cfc06c700 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2ce37fe640 1 -- 172.21.15.38:0/387646805 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4037040 con 0x7f2cfc06c700 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.917+0000 7f2d01570640 1 -- 172.21.15.38:0/387646805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cfc0ccd90 con 0x7f2cfc06c700 2024-04-25T08:38:51.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.919+0000 7f2d01570640 1 -- 172.21.15.38:0/387646805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cfc068e80 con 0x7f2cfc06c700 2024-04-25T08:38:51.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.919+0000 7f2ce37fe640 1 -- 172.21.15.38:0/387646805 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2cf4044020 con 0x7f2cfc06c700 2024-04-25T08:38:51.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.919+0000 7f2ce37fe640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ce806b4d0 0x7f2ce806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:51.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.919+0000 7f2ce37fe640 1 -- 172.21.15.38:0/387646805 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2cf40aab00 con 0x7f2cfc06c700 2024-04-25T08:38:51.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.920+0000 7f2cfa7fc640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ce806b4d0 0x7f2ce806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:51.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.920+0000 7f2cfa7fc640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ce806b4d0 0x7f2ce806d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0006fd0 tx=0x7f2cf0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:51.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:51.924+0000 7f2ce37fe640 1 -- 172.21.15.38:0/387646805 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2cf404b030 con 0x7f2cfc06c700 2024-04-25T08:38:52.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.103+0000 7f2d01570640 1 -- 172.21.15.38:0/387646805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cfc0b2440 con 0x7f2cfc06c700 2024-04-25T08:38:52.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.103+0000 7f2ce37fe640 1 -- 172.21.15.38:0/387646805 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cfc1f6cf0 con 0x7f2cfc06c700 2024-04-25T08:38:52.096 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:38:52.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.105+0000 7f2d01570640 1 -- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ce806b4d0 msgr2=0x7f2ce806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.105+0000 7f2d01570640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ce806b4d0 0x7f2ce806d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0006fd0 tx=0x7f2cf0008040 comp rx=0 tx=0).stop 2024-04-25T08:38:52.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.105+0000 7f2d01570640 1 -- 172.21.15.38:0/387646805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cfc06c700 msgr2=0x7f2cfc1f67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.105+0000 7f2d01570640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cfc06c700 0x7f2cfc1f67b0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f2cf402a220 tx=0x7f2cf40046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.105+0000 7f2d01570640 1 -- 172.21.15.38:0/387646805 shutdown_connections 2024-04-25T08:38:52.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.105+0000 7f2d01570640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2cfc15e580 0x7f2cfc15cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.105+0000 7f2d01570640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ce806b4d0 0x7f2ce806d990 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.105+0000 7f2d01570640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2cfc06cae0 0x7f2cfc0cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.105+0000 7f2d01570640 1 --2- 172.21.15.38:0/387646805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2cfc06c700 0x7f2cfc1f67b0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.105+0000 7f2d01570640 1 -- 172.21.15.38:0/387646805 >> 172.21.15.38:0/387646805 conn(0x7f2cfc08b000 msgr2=0x7f2cfc0b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:52.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.105+0000 7f2d01570640 1 -- 172.21.15.38:0/387646805 shutdown_connections 2024-04-25T08:38:52.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.105+0000 7f2d01570640 1 -- 172.21.15.38:0/387646805 wait complete. 2024-04-25T08:38:52.109 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:38:52.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.219+0000 7fd947d3b640 1 Processor -- start 2024-04-25T08:38:52.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.220+0000 7fd947d3b640 1 -- start start 2024-04-25T08:38:52.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.220+0000 7fd947d3b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd9400d0320 0x7fd9400d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.220+0000 7fd947d3b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd9400d0700 0x7fd9400d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.220+0000 7fd947d3b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9400d2330 con 0x7fd9400d1e90 2024-04-25T08:38:52.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.220+0000 7fd947d3b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9400d2610 con 0x7fd9400d0700 2024-04-25T08:38:52.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.220+0000 7fd947d3b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9400d28f0 con 0x7fd9400d0320 2024-04-25T08:38:52.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.221+0000 7fd9452d6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd9400d0700 0x7fd9400d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.221+0000 7fd9462d8640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd9400d1e90 0x7fd9400b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50332/0 (socket says 172.21.15.38:50332) 2024-04-25T08:38:52.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.221+0000 7fd9452d6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd9400d0700 0x7fd9400d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50758/0 (socket says 172.21.15.38:50758) 2024-04-25T08:38:52.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.221+0000 7fd9462d8640 1 -- 172.21.15.38:0/206210106 learned_addr learned my addr 172.21.15.38:0/206210106 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:52.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.221+0000 7fd945ad7640 1 --2- 172.21.15.38:0/206210106 >> v2:172.21.15.107:3300/0 conn(0x7fd9400d0320 0x7fd9400d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.221+0000 7fd9452d6640 1 -- 172.21.15.38:0/206210106 >> v2:172.21.15.107:3300/0 conn(0x7fd9400d0320 msgr2=0x7fd9400d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:52.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.221+0000 7fd9452d6640 1 --2- 172.21.15.38:0/206210106 >> v2:172.21.15.107:3300/0 conn(0x7fd9400d0320 0x7fd9400d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.221+0000 7fd9452d6640 1 -- 172.21.15.38:0/206210106 >> v1:172.21.15.38:6789/0 conn(0x7fd9400d1e90 legacy=0x7fd9400b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.221+0000 7fd944ad5640 1 -- 172.21.15.38:0/206210106 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201087897 0 0) 0x7fd9400d2330 con 0x7fd9400d1e90 2024-04-25T08:38:52.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.221+0000 7fd9452d6640 1 -- 172.21.15.38:0/206210106 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9400d2bd0 con 0x7fd9400d0700 2024-04-25T08:38:52.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.222+0000 7fd9452d6640 1 --2- 172.21.15.38:0/206210106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9400d0700 0x7fd9400d1950 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd940069100 tx=0x7fd93402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5816c98b9fba476d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:52.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.222+0000 7fd944ad5640 1 -- 172.21.15.38:0/206210106 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934037070 con 0x7fd9400d0700 2024-04-25T08:38:52.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.222+0000 7fd944ad5640 1 -- 172.21.15.38:0/206210106 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd934032440 con 0x7fd9400d0700 2024-04-25T08:38:52.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.222+0000 7fd944ad5640 1 -- 172.21.15.38:0/206210106 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934032810 con 0x7fd9400d0700 2024-04-25T08:38:52.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.222+0000 7fd947d3b640 1 -- 172.21.15.38:0/206210106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9400d0700 msgr2=0x7fd9400d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.222+0000 7fd947d3b640 1 --2- 172.21.15.38:0/206210106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9400d0700 0x7fd9400d1950 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd940069100 tx=0x7fd93402a230 comp rx=0 tx=0).stop 2024-04-25T08:38:52.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.222+0000 7fd947d3b640 1 -- 172.21.15.38:0/206210106 shutdown_connections 2024-04-25T08:38:52.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.222+0000 7fd947d3b640 1 --2- 172.21.15.38:0/206210106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9400d0700 0x7fd9400d1950 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.222+0000 7fd947d3b640 1 --2- 172.21.15.38:0/206210106 >> v2:172.21.15.107:3300/0 conn(0x7fd9400d0320 0x7fd9400d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.222+0000 7fd947d3b640 1 -- 172.21.15.38:0/206210106 >> 172.21.15.38:0/206210106 conn(0x7fd9400668f0 msgr2=0x7fd940066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:52.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.222+0000 7fd947d3b640 1 -- 172.21.15.38:0/206210106 shutdown_connections 2024-04-25T08:38:52.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.222+0000 7fd947d3b640 1 -- 172.21.15.38:0/206210106 wait complete. 2024-04-25T08:38:52.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.224+0000 7fd947d3b640 1 Processor -- start 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.224+0000 7fd947d3b640 1 -- start start 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.224+0000 7fd947d3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9400d0320 0x7fd9400c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.224+0000 7fd947d3b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9400d0700 0x7fd940157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.224+0000 7fd947d3b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9400d1e90 0x7fd940157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.224+0000 7fd947d3b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9400d0b40 con 0x7fd9400d0320 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.224+0000 7fd947d3b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd940161210 con 0x7fd9400d0700 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.224+0000 7fd947d3b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9401f3dd0 con 0x7fd9400d1e90 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.224+0000 7fd945ad7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9400d0320 0x7fd9400c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.224+0000 7fd945ad7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9400d0320 0x7fd9400c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50760/0 (socket says 172.21.15.38:50760) 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.224+0000 7fd945ad7640 1 -- 172.21.15.38:0/1778968230 learned_addr learned my addr 172.21.15.38:0/1778968230 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.225+0000 7fd945ad7640 1 -- 172.21.15.38:0/1778968230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9400d1e90 msgr2=0x7fd940157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.225+0000 7fd945ad7640 1 --2- 172.21.15.38:0/1778968230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9400d1e90 0x7fd940157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.225+0000 7fd945ad7640 1 -- 172.21.15.38:0/1778968230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9400d0700 msgr2=0x7fd940157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.225+0000 7fd945ad7640 1 --2- 172.21.15.38:0/1778968230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9400d0700 0x7fd940157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.225+0000 7fd945ad7640 1 -- 172.21.15.38:0/1778968230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9400ca200 con 0x7fd9400d0320 2024-04-25T08:38:52.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.225+0000 7fd945ad7640 1 --2- 172.21.15.38:0/1778968230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9400d0320 0x7fd9400c9cc0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd93800d800 tx=0x7fd93800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:52.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.225+0000 7fd93e7fc640 1 -- 172.21.15.38:0/1778968230 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93800b6c0 con 0x7fd9400d0320 2024-04-25T08:38:52.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.225+0000 7fd93e7fc640 1 -- 172.21.15.38:0/1778968230 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd938019070 con 0x7fd9400d0320 2024-04-25T08:38:52.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.225+0000 7fd93e7fc640 1 -- 172.21.15.38:0/1778968230 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93800ba80 con 0x7fd9400d0320 2024-04-25T08:38:52.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.225+0000 7fd947d3b640 1 -- 172.21.15.38:0/1778968230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd940158680 con 0x7fd9400d0320 2024-04-25T08:38:52.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.225+0000 7fd947d3b640 1 -- 172.21.15.38:0/1778968230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd940159080 con 0x7fd9400d0320 2024-04-25T08:38:52.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.226+0000 7fd947d3b640 1 -- 172.21.15.38:0/1778968230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9400b7590 con 0x7fd9400d0320 2024-04-25T08:38:52.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.227+0000 7fd93e7fc640 1 -- 172.21.15.38:0/1778968230 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd93800bc20 con 0x7fd9400d0320 2024-04-25T08:38:52.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.227+0000 7fd93e7fc640 1 --2- 172.21.15.38:0/1778968230 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd92c06b4d0 0x7fd92c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.227+0000 7fd93e7fc640 1 -- 172.21.15.38:0/1778968230 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd93808a870 con 0x7fd9400d0320 2024-04-25T08:38:52.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.227+0000 7fd9452d6640 1 --2- 172.21.15.38:0/1778968230 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd92c06b4d0 0x7fd92c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.228+0000 7fd9452d6640 1 --2- 172.21.15.38:0/1778968230 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd92c06b4d0 0x7fd92c06d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd934034000 tx=0x7fd93402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:52.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.231+0000 7fd93e7fc640 1 -- 172.21.15.38:0/1778968230 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9400b7590 con 0x7fd9400d0320 2024-04-25T08:38:52.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.411+0000 7fd947d3b640 1 -- 172.21.15.38:0/1778968230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd940068e80 con 0x7fd9400d0320 2024-04-25T08:38:52.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.411+0000 7fd93e7fc640 1 -- 172.21.15.38:0/1778968230 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9400ca200 con 0x7fd9400d0320 2024-04-25T08:38:52.405 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:38:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.413+0000 7fd947d3b640 1 -- 172.21.15.38:0/1778968230 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd92c06b4d0 msgr2=0x7fd92c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.413+0000 7fd947d3b640 1 --2- 172.21.15.38:0/1778968230 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd92c06b4d0 0x7fd92c06d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd934034000 tx=0x7fd93402c000 comp rx=0 tx=0).stop 2024-04-25T08:38:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.413+0000 7fd947d3b640 1 -- 172.21.15.38:0/1778968230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9400d0320 msgr2=0x7fd9400c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.413+0000 7fd947d3b640 1 --2- 172.21.15.38:0/1778968230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9400d0320 0x7fd9400c9cc0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd93800d800 tx=0x7fd93800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.413+0000 7fd947d3b640 1 -- 172.21.15.38:0/1778968230 shutdown_connections 2024-04-25T08:38:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.413+0000 7fd947d3b640 1 --2- 172.21.15.38:0/1778968230 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd92c06b4d0 0x7fd92c06d990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.413+0000 7fd947d3b640 1 --2- 172.21.15.38:0/1778968230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9400d1e90 0x7fd940157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.413+0000 7fd947d3b640 1 --2- 172.21.15.38:0/1778968230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd9400d0700 0x7fd940157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.413+0000 7fd947d3b640 1 --2- 172.21.15.38:0/1778968230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9400d0320 0x7fd9400c9cc0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.413+0000 7fd947d3b640 1 -- 172.21.15.38:0/1778968230 >> 172.21.15.38:0/1778968230 conn(0x7fd9400668f0 msgr2=0x7fd9400d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:52.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.413+0000 7fd947d3b640 1 -- 172.21.15.38:0/1778968230 shutdown_connections 2024-04-25T08:38:52.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.413+0000 7fd947d3b640 1 -- 172.21.15.38:0/1778968230 wait complete. 2024-04-25T08:38:52.417 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:38:52.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.527+0000 7f6e0a770640 1 Processor -- start 2024-04-25T08:38:52.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.527+0000 7f6e0a770640 1 -- start start 2024-04-25T08:38:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.528+0000 7f6e0a770640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e040d0320 0x7f6e040d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.528+0000 7f6e0a770640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6e040b6980 0x7f6e040b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.528+0000 7f6e0a770640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e040d2060 con 0x7f6e040d0700 2024-04-25T08:38:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.528+0000 7f6e0a770640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e040d2330 con 0x7f6e040b6980 2024-04-25T08:38:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.528+0000 7f6e0a770640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040d2610 con 0x7f6e040d0320 2024-04-25T08:38:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.528+0000 7f6e08d0d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6e040b6980 0x7f6e040b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.528+0000 7f6e03fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e040d0320 0x7f6e040d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.528+0000 7f6e08d0d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6e040b6980 0x7f6e040b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:41838/0 (socket says 172.21.15.38:41838) 2024-04-25T08:38:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.528+0000 7f6e03fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e040d0320 0x7f6e040d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:35650/0 (socket says 172.21.15.38:35650) 2024-04-25T08:38:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.528+0000 7f6e08d0d640 1 -- 172.21.15.38:0/1821002432 learned_addr learned my addr 172.21.15.38:0/1821002432 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.528+0000 7f6e037fe640 1 --1- 172.21.15.38:0/1821002432 >> v1:172.21.15.107:6789/0 conn(0x7f6e040d0700 0x7f6e040d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:59454/0 (socket says 172.21.15.38:59454) 2024-04-25T08:38:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.529+0000 7f6e03fff640 1 -- 172.21.15.38:0/1821002432 >> v2:172.21.15.107:3300/0 conn(0x7f6e040b6980 msgr2=0x7f6e040b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.529+0000 7f6e03fff640 1 --2- 172.21.15.38:0/1821002432 >> v2:172.21.15.107:3300/0 conn(0x7f6e040b6980 0x7f6e040b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.529+0000 7f6e03fff640 1 -- 172.21.15.38:0/1821002432 >> v1:172.21.15.107:6789/0 conn(0x7f6e040d0700 legacy=0x7f6e040d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.529+0000 7f6e03fff640 1 -- 172.21.15.38:0/1821002432 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e040d28f0 con 0x7f6e040d0320 2024-04-25T08:38:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.529+0000 7f6e08d0d640 1 --2- 172.21.15.38:0/1821002432 >> v2:172.21.15.107:3300/0 conn(0x7f6e040b6980 0x7f6e040b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:38:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.530+0000 7f6e03fff640 1 --2- 172.21.15.38:0/1821002432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e040d0320 0x7f6e040d33d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0097a0 tx=0x7f6dec02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95cbde551fb628c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.530+0000 7f6e02ffd640 1 -- 172.21.15.38:0/1821002432 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec037070 con 0x7f6e040d0320 2024-04-25T08:38:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.530+0000 7f6e02ffd640 1 -- 172.21.15.38:0/1821002432 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec032440 con 0x7f6e040d0320 2024-04-25T08:38:52.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.530+0000 7f6e02ffd640 1 -- 172.21.15.38:0/1821002432 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec032810 con 0x7f6e040d0320 2024-04-25T08:38:52.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.530+0000 7f6e0a770640 1 -- 172.21.15.38:0/1821002432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e040d0320 msgr2=0x7f6e040d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.530+0000 7f6e0a770640 1 --2- 172.21.15.38:0/1821002432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e040d0320 0x7f6e040d33d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0097a0 tx=0x7f6dec02a210 comp rx=0 tx=0).stop 2024-04-25T08:38:52.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.530+0000 7f6e0a770640 1 -- 172.21.15.38:0/1821002432 shutdown_connections 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.530+0000 7f6e0a770640 1 --2- 172.21.15.38:0/1821002432 >> v2:172.21.15.107:3300/0 conn(0x7f6e040b6980 0x7f6e040b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.530+0000 7f6e0a770640 1 --2- 172.21.15.38:0/1821002432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e040d0320 0x7f6e040d33d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.530+0000 7f6e0a770640 1 -- 172.21.15.38:0/1821002432 >> 172.21.15.38:0/1821002432 conn(0x7f6e040668f0 msgr2=0x7f6e04066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.530+0000 7f6e0a770640 1 -- 172.21.15.38:0/1821002432 shutdown_connections 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.530+0000 7f6e0a770640 1 -- 172.21.15.38:0/1821002432 wait complete. 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.532+0000 7f6e0a770640 1 Processor -- start 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.532+0000 7f6e0a770640 1 -- start start 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.532+0000 7f6e0a770640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e040b6980 0x7f6e04158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.532+0000 7f6e0a770640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e040d0320 0x7f6e040b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.532+0000 7f6e0a770640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e040d0700 0x7f6e04155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.532+0000 7f6e0a770640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040d0b40 con 0x7f6e040d0700 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.532+0000 7f6e0a770640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040d2eb0 con 0x7f6e040d0320 2024-04-25T08:38:52.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.532+0000 7f6e0a770640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040b5c60 con 0x7f6e040b6980 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e08d0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e040d0700 0x7f6e04155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e08d0d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e040d0700 0x7f6e04155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50764/0 (socket says 172.21.15.38:50764) 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e08d0d640 1 -- 172.21.15.38:0/1703482530 learned_addr learned my addr 172.21.15.38:0/1703482530 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e037fe640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e040d0320 0x7f6e040b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e03fff640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e040b6980 0x7f6e04158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e08d0d640 1 -- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e040b6980 msgr2=0x7f6e04158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e08d0d640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e040b6980 0x7f6e04158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e08d0d640 1 -- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e040d0320 msgr2=0x7f6e040b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e08d0d640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e040d0320 0x7f6e040b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e08d0d640 1 -- 172.21.15.38:0/1703482530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e04158a90 con 0x7f6e040d0700 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e08d0d640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e040d0700 0x7f6e04155700 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00efc0 tx=0x7f6dfc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e00ff9640 1 -- 172.21.15.38:0/1703482530 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc00cb80 con 0x7f6e040d0700 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e00ff9640 1 -- 172.21.15.38:0/1703482530 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc00cd20 con 0x7f6e040d0700 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e0a770640 1 -- 172.21.15.38:0/1703482530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e040b4bc0 con 0x7f6e040d0700 2024-04-25T08:38:52.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e00ff9640 1 -- 172.21.15.38:0/1703482530 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc003bb0 con 0x7f6e040d0700 2024-04-25T08:38:52.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.533+0000 7f6e0a770640 1 -- 172.21.15.38:0/1703482530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e040b5180 con 0x7f6e040d0700 2024-04-25T08:38:52.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.534+0000 7f6e0a770640 1 -- 172.21.15.38:0/1703482530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e04068e80 con 0x7f6e040d0700 2024-04-25T08:38:52.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.535+0000 7f6e00ff9640 1 -- 172.21.15.38:0/1703482530 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6dfc01b030 con 0x7f6e040d0700 2024-04-25T08:38:52.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.535+0000 7f6e00ff9640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6de406b4d0 0x7f6de406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.536+0000 7f6e00ff9640 1 -- 172.21.15.38:0/1703482530 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6dfc08a5a0 con 0x7f6e040d0700 2024-04-25T08:38:52.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.536+0000 7f6e03fff640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6de406b4d0 0x7f6de406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.536+0000 7f6e03fff640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6de406b4d0 0x7f6de406d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6dec034000 tx=0x7f6dec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:52.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.540+0000 7f6e00ff9640 1 -- 172.21.15.38:0/1703482530 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6dfc022020 con 0x7f6e040d0700 2024-04-25T08:38:52.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.717+0000 7f6e0a770640 1 -- 172.21.15.38:0/1703482530 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e040b3050 con 0x7f6e040d0700 2024-04-25T08:38:52.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.717+0000 7f6e00ff9640 1 -- 172.21.15.38:0/1703482530 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6dfc010040 con 0x7f6e040d0700 2024-04-25T08:38:52.711 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:38:52.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.719+0000 7f6e0a770640 1 -- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6de406b4d0 msgr2=0x7f6de406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.719+0000 7f6e0a770640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6de406b4d0 0x7f6de406d990 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6dec034000 tx=0x7f6dec02e000 comp rx=0 tx=0).stop 2024-04-25T08:38:52.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.720+0000 7f6e0a770640 1 -- 172.21.15.38:0/1703482530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e040d0700 msgr2=0x7f6e04155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.720+0000 7f6e0a770640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e040d0700 0x7f6e04155700 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00efc0 tx=0x7f6dfc00c490 comp rx=0 tx=0).stop 2024-04-25T08:38:52.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.720+0000 7f6e0a770640 1 -- 172.21.15.38:0/1703482530 shutdown_connections 2024-04-25T08:38:52.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.720+0000 7f6e0a770640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e040d0700 0x7f6e04155700 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.720+0000 7f6e0a770640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e040d0320 0x7f6e040b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.720+0000 7f6e0a770640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6de406b4d0 0x7f6de406d990 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.720+0000 7f6e0a770640 1 --2- 172.21.15.38:0/1703482530 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e040b6980 0x7f6e04158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.720+0000 7f6e0a770640 1 -- 172.21.15.38:0/1703482530 >> 172.21.15.38:0/1703482530 conn(0x7f6e040668f0 msgr2=0x7f6e040671e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:52.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.720+0000 7f6e0a770640 1 -- 172.21.15.38:0/1703482530 shutdown_connections 2024-04-25T08:38:52.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.720+0000 7f6e0a770640 1 -- 172.21.15.38:0/1703482530 wait complete. 2024-04-25T08:38:52.724 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:38:52.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.832+0000 7f8603a07640 1 Processor -- start 2024-04-25T08:38:52.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.832+0000 7f8603a07640 1 -- start start 2024-04-25T08:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f8603a07640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f85fc160ef0 0x7f85fc0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f8603a07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85fc0b2ab0 0x7f85fc0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f8603a07640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85fc0b9610 con 0x7f85fc0b9180 2024-04-25T08:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f8603a07640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc161cf0 con 0x7f85fc160ef0 2024-04-25T08:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f8603a07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc1612d0 con 0x7f85fc0b2ab0 2024-04-25T08:38:52.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f86017a3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f85fc160ef0 0x7f85fc0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f86017a3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f85fc160ef0 0x7f85fc0d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50780/0 (socket says 172.21.15.38:50780) 2024-04-25T08:38:52.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f86017a3640 1 -- 172.21.15.38:0/1115340246 learned_addr learned my addr 172.21.15.38:0/1115340246 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:52.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f8601fa4640 1 --2- 172.21.15.38:0/1115340246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85fc0b2ab0 0x7f85fc0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f86017a3640 1 -- 172.21.15.38:0/1115340246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85fc0b2ab0 msgr2=0x7f85fc0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:52.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f86017a3640 1 --2- 172.21.15.38:0/1115340246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85fc0b2ab0 0x7f85fc0d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f86017a3640 1 -- 172.21.15.38:0/1115340246 >> v1:172.21.15.107:6789/0 conn(0x7f85fc0b9180 legacy=0x7f85fc0d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.833+0000 7f86017a3640 1 -- 172.21.15.38:0/1115340246 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc1616a0 con 0x7f85fc160ef0 2024-04-25T08:38:52.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.834+0000 7f86017a3640 1 --2- 172.21.15.38:0/1115340246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85fc160ef0 0x7f85fc0d07b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009870 tx=0x7f85f802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add2b2e69076699e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:52.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.834+0000 7f85f3fff640 1 -- 172.21.15.38:0/1115340246 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8037070 con 0x7f85fc160ef0 2024-04-25T08:38:52.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.834+0000 7f85f3fff640 1 -- 172.21.15.38:0/1115340246 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f8032440 con 0x7f85fc160ef0 2024-04-25T08:38:52.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.834+0000 7f85f3fff640 1 -- 172.21.15.38:0/1115340246 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8032810 con 0x7f85fc160ef0 2024-04-25T08:38:52.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.834+0000 7f8603a07640 1 -- 172.21.15.38:0/1115340246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85fc160ef0 msgr2=0x7f85fc0d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.834+0000 7f8603a07640 1 --2- 172.21.15.38:0/1115340246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85fc160ef0 0x7f85fc0d07b0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009870 tx=0x7f85f802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.834+0000 7f8603a07640 1 -- 172.21.15.38:0/1115340246 shutdown_connections 2024-04-25T08:38:52.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.835+0000 7f8603a07640 1 --2- 172.21.15.38:0/1115340246 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85fc0b2ab0 0x7f85fc0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.835+0000 7f8603a07640 1 --2- 172.21.15.38:0/1115340246 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85fc160ef0 0x7f85fc0d07b0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.835+0000 7f8603a07640 1 -- 172.21.15.38:0/1115340246 >> 172.21.15.38:0/1115340246 conn(0x7f85fc066d10 msgr2=0x7f85fc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:52.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.835+0000 7f8603a07640 1 -- 172.21.15.38:0/1115340246 shutdown_connections 2024-04-25T08:38:52.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.835+0000 7f8603a07640 1 -- 172.21.15.38:0/1115340246 wait complete. 2024-04-25T08:38:52.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.836+0000 7f8603a07640 1 Processor -- start 2024-04-25T08:38:52.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.836+0000 7f8603a07640 1 -- start start 2024-04-25T08:38:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.836+0000 7f8603a07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85fc0b2ab0 0x7f85fc1f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.836+0000 7f8603a07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85fc160ef0 0x7f85fc0c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f8603a07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85fc0ca930 0x7f85fc0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f8603a07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc162300 con 0x7f85fc0b2ab0 2024-04-25T08:38:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f8603a07640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc162180 con 0x7f85fc0ca930 2024-04-25T08:38:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f8603a07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc162480 con 0x7f85fc160ef0 2024-04-25T08:38:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f86017a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85fc0b2ab0 0x7f85fc1f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f86017a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85fc0b2ab0 0x7f85fc1f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50786/0 (socket says 172.21.15.38:50786) 2024-04-25T08:38:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f86017a3640 1 -- 172.21.15.38:0/767603242 learned_addr learned my addr 172.21.15.38:0/767603242 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f8601fa4640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85fc0ca930 0x7f85fc0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f8600fa2640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85fc160ef0 0x7f85fc0c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f86017a3640 1 -- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85fc160ef0 msgr2=0x7f85fc0c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f86017a3640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85fc160ef0 0x7f85fc0c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f86017a3640 1 -- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85fc0ca930 msgr2=0x7f85fc0cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f86017a3640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85fc0ca930 0x7f85fc0cadc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f86017a3640 1 -- 172.21.15.38:0/767603242 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc15bff0 con 0x7f85fc0b2ab0 2024-04-25T08:38:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f86017a3640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85fc0b2ab0 0x7f85fc1f7640 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000c00 tx=0x7f85f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f85f1ffb640 1 -- 172.21.15.38:0/767603242 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8041020 con 0x7f85fc0b2ab0 2024-04-25T08:38:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f85f1ffb640 1 -- 172.21.15.38:0/767603242 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f80306c0 con 0x7f85fc0b2ab0 2024-04-25T08:38:52.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f8603a07640 1 -- 172.21.15.38:0/767603242 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc0cb4a0 con 0x7f85fc0b2ab0 2024-04-25T08:38:52.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.837+0000 7f85f1ffb640 1 -- 172.21.15.38:0/767603242 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8037040 con 0x7f85fc0b2ab0 2024-04-25T08:38:52.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.838+0000 7f8603a07640 1 -- 172.21.15.38:0/767603242 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc1f6b60 con 0x7f85fc0b2ab0 2024-04-25T08:38:52.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.838+0000 7f8603a07640 1 -- 172.21.15.38:0/767603242 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85fc04ef90 con 0x7f85fc0b2ab0 2024-04-25T08:38:52.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.839+0000 7f85f1ffb640 1 -- 172.21.15.38:0/767603242 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85f8044020 con 0x7f85fc0b2ab0 2024-04-25T08:38:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.839+0000 7f85f1ffb640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f85e006b4d0 0x7f85e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.839+0000 7f85f1ffb640 1 -- 172.21.15.38:0/767603242 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85f80aab20 con 0x7f85fc0b2ab0 2024-04-25T08:38:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.840+0000 7f8600fa2640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f85e006b4d0 0x7f85e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.840+0000 7f8600fa2640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f85e006b4d0 0x7f85e006d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f85fc06cd30 tx=0x7f85ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:52.844+0000 7f85f1ffb640 1 -- 172.21.15.38:0/767603242 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85f804b030 con 0x7f85fc0b2ab0 2024-04-25T08:38:53.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.035+0000 7f8603a07640 1 -- 172.21.15.38:0/767603242 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f85fc0d32f0 con 0x7f85fc0b2ab0 2024-04-25T08:38:53.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.036+0000 7f85f1ffb640 1 -- 172.21.15.38:0/767603242 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v96) v1 ==== 80+0+0 (secure 0 0 0) 0x7f85f80798a0 con 0x7f85fc0b2ab0 2024-04-25T08:38:53.029 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:38:53.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.038+0000 7f8603a07640 1 -- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f85e006b4d0 msgr2=0x7f85e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:53.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.038+0000 7f8603a07640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f85e006b4d0 0x7f85e006d990 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f85fc06cd30 tx=0x7f85ec009040 comp rx=0 tx=0).stop 2024-04-25T08:38:53.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.038+0000 7f8603a07640 1 -- 172.21.15.38:0/767603242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85fc0b2ab0 msgr2=0x7f85fc1f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:53.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.038+0000 7f8603a07640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85fc0b2ab0 0x7f85fc1f7640 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f85f8000c00 tx=0x7f85f80046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.038+0000 7f8603a07640 1 -- 172.21.15.38:0/767603242 shutdown_connections 2024-04-25T08:38:53.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.038+0000 7f8603a07640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85fc0ca930 0x7f85fc0cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.038+0000 7f8603a07640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f85e006b4d0 0x7f85e006d990 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.038+0000 7f8603a07640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f85fc160ef0 0x7f85fc0c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.038+0000 7f8603a07640 1 --2- 172.21.15.38:0/767603242 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85fc0b2ab0 0x7f85fc1f7640 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.038+0000 7f8603a07640 1 -- 172.21.15.38:0/767603242 >> 172.21.15.38:0/767603242 conn(0x7f85fc066d10 msgr2=0x7f85fc0b75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:53.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.038+0000 7f8603a07640 1 -- 172.21.15.38:0/767603242 shutdown_connections 2024-04-25T08:38:53.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.038+0000 7f8603a07640 1 -- 172.21.15.38:0/767603242 wait complete. 2024-04-25T08:38:53.042 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:38:53.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:53.042 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:38:53.065 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[79049]: fuse finished with error 0 and tester_r 0 2024-04-25T08:38:53.151 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:38:53.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:53.151 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:38:53.317 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:38:53.317 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:53.335 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:38:53.335 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:38:53.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:53.335 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:38:53.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:38:53.391 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:38:53.392 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:38:53.392 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:53.408 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:38:53.409 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:38:53.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:53.409 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:38:53.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:38:53.463 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:38:53.463 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:38:53.464 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:38:53.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:38:53.481 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:38:53.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:38:53.481 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:38:53.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:38:53.537 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:38:53.538 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:53.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.646+0000 7f4f43f7e640 1 Processor -- start 2024-04-25T08:38:53.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.646+0000 7f4f43f7e640 1 -- start start 2024-04-25T08:38:53.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f43f7e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4f3c0cf8f0 0x7f4f3c06fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:53.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f43f7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f3c06c300 0x7f4f3c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:53.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f43f7e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f3c15ce50 con 0x7f4f3c0cf510 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f43f7e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f3c06e170 con 0x7f4f3c0cf8f0 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f43f7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c15d100 con 0x7f4f3c06c300 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f41519640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4f3c0cf8f0 0x7f4f3c06fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f41519640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4f3c0cf8f0 0x7f4f3c06fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50792/0 (socket says 172.21.15.38:50792) 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f41519640 1 -- 172.21.15.38:0/240583597 learned_addr learned my addr 172.21.15.38:0/240583597 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f4251b640 1 --2- 172.21.15.38:0/240583597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f3c06c300 0x7f4f3c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f41519640 1 -- 172.21.15.38:0/240583597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f3c06c300 msgr2=0x7f4f3c15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f41519640 1 --2- 172.21.15.38:0/240583597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f3c06c300 0x7f4f3c15d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f41519640 1 -- 172.21.15.38:0/240583597 >> v1:172.21.15.107:6789/0 conn(0x7f4f3c0cf510 legacy=0x7f4f3c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f41519640 1 -- 172.21.15.38:0/240583597 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c06e410 con 0x7f4f3c0cf8f0 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.647+0000 7f4f4251b640 1 --2- 172.21.15.38:0/240583597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f3c06c300 0x7f4f3c15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.648+0000 7f4f41519640 1 --2- 172.21.15.38:0/240583597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f3c0cf8f0 0x7f4f3c06fc90 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009870 tx=0x7f4f3802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20338e2eb0a98945 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:53.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.648+0000 7f4f40d18640 1 -- 172.21.15.38:0/240583597 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38037070 con 0x7f4f3c0cf8f0 2024-04-25T08:38:53.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.648+0000 7f4f40d18640 1 -- 172.21.15.38:0/240583597 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f38032440 con 0x7f4f3c0cf8f0 2024-04-25T08:38:53.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.648+0000 7f4f40d18640 1 -- 172.21.15.38:0/240583597 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38032810 con 0x7f4f3c0cf8f0 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.649+0000 7f4f43f7e640 1 -- 172.21.15.38:0/240583597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f3c0cf8f0 msgr2=0x7f4f3c06fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.649+0000 7f4f43f7e640 1 --2- 172.21.15.38:0/240583597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f3c0cf8f0 0x7f4f3c06fc90 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009870 tx=0x7f4f3802a180 comp rx=0 tx=0).stop 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.649+0000 7f4f43f7e640 1 -- 172.21.15.38:0/240583597 shutdown_connections 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.649+0000 7f4f43f7e640 1 --2- 172.21.15.38:0/240583597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f3c06c300 0x7f4f3c15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.649+0000 7f4f43f7e640 1 --2- 172.21.15.38:0/240583597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f3c0cf8f0 0x7f4f3c06fc90 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.649+0000 7f4f43f7e640 1 -- 172.21.15.38:0/240583597 >> 172.21.15.38:0/240583597 conn(0x7f4f3c071920 msgr2=0x7f4f3c071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.649+0000 7f4f43f7e640 1 -- 172.21.15.38:0/240583597 shutdown_connections 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.649+0000 7f4f43f7e640 1 -- 172.21.15.38:0/240583597 wait complete. 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.650+0000 7f4f43f7e640 1 Processor -- start 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.650+0000 7f4f43f7e640 1 -- start start 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f43f7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c06c300 0x7f4f3c13c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f43f7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f3c0cf510 0x7f4f3c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:53.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f43f7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f3c15da00 0x7f4f3c14dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f43f7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c06d9e0 con 0x7f4f3c0cf510 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f43f7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c06d860 con 0x7f4f3c06c300 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f43f7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3c15c920 con 0x7f4f3c15da00 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f41519640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f3c0cf510 0x7f4f3c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f41519640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f3c0cf510 0x7f4f3c15d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50802/0 (socket says 172.21.15.38:50802) 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f41519640 1 -- 172.21.15.38:0/1605352517 learned_addr learned my addr 172.21.15.38:0/1605352517 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f4251b640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f3c15da00 0x7f4f3c14dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f41d1a640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c06c300 0x7f4f3c13c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f41519640 1 -- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f3c15da00 msgr2=0x7f4f3c14dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f41519640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f3c15da00 0x7f4f3c14dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f41519640 1 -- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c06c300 msgr2=0x7f4f3c13c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f41519640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c06c300 0x7f4f3c13c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f41519640 1 -- 172.21.15.38:0/1605352517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3c1563b0 con 0x7f4f3c0cf510 2024-04-25T08:38:53.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f41d1a640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c06c300 0x7f4f3c13c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.651+0000 7f4f41519640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f3c0cf510 0x7f4f3c15d4c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4f380090b0 tx=0x7f4f380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.652+0000 7f4f327fc640 1 -- 172.21.15.38:0/1605352517 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38041020 con 0x7f4f3c0cf510 2024-04-25T08:38:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.652+0000 7f4f327fc640 1 -- 172.21.15.38:0/1605352517 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f38030990 con 0x7f4f3c0cf510 2024-04-25T08:38:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.652+0000 7f4f327fc640 1 -- 172.21.15.38:0/1605352517 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38037040 con 0x7f4f3c0cf510 2024-04-25T08:38:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.652+0000 7f4f43f7e640 1 -- 172.21.15.38:0/1605352517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f3c156690 con 0x7f4f3c0cf510 2024-04-25T08:38:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.652+0000 7f4f43f7e640 1 -- 172.21.15.38:0/1605352517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f3c156c50 con 0x7f4f3c0cf510 2024-04-25T08:38:53.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.653+0000 7f4f43f7e640 1 -- 172.21.15.38:0/1605352517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3c15de10 con 0x7f4f3c0cf510 2024-04-25T08:38:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.653+0000 7f4f327fc640 1 -- 172.21.15.38:0/1605352517 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f38044020 con 0x7f4f3c0cf510 2024-04-25T08:38:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.654+0000 7f4f327fc640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f2006b4d0 0x7f4f2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.654+0000 7f4f327fc640 1 -- 172.21.15.38:0/1605352517 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4f380aad40 con 0x7f4f3c0cf510 2024-04-25T08:38:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.654+0000 7f4f41d1a640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f2006b4d0 0x7f4f2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:53.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.654+0000 7f4f41d1a640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f2006b4d0 0x7f4f2006d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c003eb0 tx=0x7f4f2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:53.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.658+0000 7f4f327fc640 1 -- 172.21.15.38:0/1605352517 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f3c15de10 con 0x7f4f3c0cf510 2024-04-25T08:38:53.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.878+0000 7f4f43f7e640 1 -- 172.21.15.38:0/1605352517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f3c068600 con 0x7f4f3c0cf510 2024-04-25T08:38:53.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.879+0000 7f4f327fc640 1 -- 172.21.15.38:0/1605352517 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4f3804b030 con 0x7f4f3c0cf510 2024-04-25T08:38:53.873 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:53.873 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3680979257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3680979257},{"type":"v1","addr":"172.21.15.107:6836","nonce":3680979257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/697909489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":697909489},{"type":"v1","addr":"172.21.15.107:6837","nonce":697909489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3083871141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3083871141},{"type":"v1","addr":"172.21.15.38:6834","nonce":3083871141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:26.095391+0000","modified":"2024-04-25T08:38:28.097807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/2922734840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":2922734840},{"type":"v1","addr":"172.21.15.38:6835","nonce":2922734840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:38:53.873 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 57 2024-04-25T08:38:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.881+0000 7f4f43f7e640 1 -- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f2006b4d0 msgr2=0x7f4f2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.881+0000 7f4f43f7e640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f2006b4d0 0x7f4f2006d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c003eb0 tx=0x7f4f2c009040 comp rx=0 tx=0).stop 2024-04-25T08:38:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.881+0000 7f4f43f7e640 1 -- 172.21.15.38:0/1605352517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f3c0cf510 msgr2=0x7f4f3c15d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.881+0000 7f4f43f7e640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f3c0cf510 0x7f4f3c15d4c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f4f380090b0 tx=0x7f4f380046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.881+0000 7f4f43f7e640 1 -- 172.21.15.38:0/1605352517 shutdown_connections 2024-04-25T08:38:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.881+0000 7f4f43f7e640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f2006b4d0 0x7f4f2006d990 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.881+0000 7f4f43f7e640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f3c15da00 0x7f4f3c14dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.881+0000 7f4f43f7e640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f3c0cf510 0x7f4f3c15d4c0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.881+0000 7f4f43f7e640 1 --2- 172.21.15.38:0/1605352517 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f3c06c300 0x7f4f3c13c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.881+0000 7f4f43f7e640 1 -- 172.21.15.38:0/1605352517 >> 172.21.15.38:0/1605352517 conn(0x7f4f3c071920 msgr2=0x7f4f3c0b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.881+0000 7f4f43f7e640 1 -- 172.21.15.38:0/1605352517 shutdown_connections 2024-04-25T08:38:53.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.881+0000 7f4f43f7e640 1 -- 172.21.15.38:0/1605352517 wait complete. 2024-04-25T08:38:53.886 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.993+0000 7f2270c4c640 1 Processor -- start 2024-04-25T08:38:53.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.993+0000 7f2270c4c640 1 -- start start 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f2270c4c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f226c15b730 0x7f226c15bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f2270c4c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f226c0d0cd0 0x7f226c0d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f2270c4c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226c073220 con 0x7f226c0ccd00 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f2270c4c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f226c0cdd10 con 0x7f226c0d0cd0 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f2270c4c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0cf7f0 con 0x7f226c15b730 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f2263fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f226c0d0cd0 0x7f226c0d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f226ad9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f226c0ccd00 0x7f226c0b2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50344/0 (socket says 172.21.15.38:50344) 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f226ad9d640 1 -- 172.21.15.38:0/3173646069 learned_addr learned my addr 172.21.15.38:0/3173646069 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f226a59c640 1 --2- 172.21.15.38:0/3173646069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f226c15b730 0x7f226c15bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f2263fff640 1 -- 172.21.15.38:0/3173646069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f226c15b730 msgr2=0x7f226c15bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f2263fff640 1 --2- 172.21.15.38:0/3173646069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f226c15b730 0x7f226c15bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f2269d9b640 1 -- 172.21.15.38:0/3173646069 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550150090 0 0) 0x7f226c073220 con 0x7f226c0ccd00 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f2263fff640 1 -- 172.21.15.38:0/3173646069 >> v1:172.21.15.38:6789/0 conn(0x7f226c0ccd00 legacy=0x7f226c0b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:53.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.994+0000 7f2263fff640 1 -- 172.21.15.38:0/3173646069 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c0cfad0 con 0x7f226c0d0cd0 2024-04-25T08:38:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.995+0000 7f2263fff640 1 --2- 172.21.15.38:0/3173646069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f226c0d0cd0 0x7f226c0d1130 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f22540097a0 tx=0x7f225402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72df8c7f99b0ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.995+0000 7f2269d9b640 1 -- 172.21.15.38:0/3173646069 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2254037070 con 0x7f226c0d0cd0 2024-04-25T08:38:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.995+0000 7f2269d9b640 1 -- 172.21.15.38:0/3173646069 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2254032440 con 0x7f226c0d0cd0 2024-04-25T08:38:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.995+0000 7f2269d9b640 1 -- 172.21.15.38:0/3173646069 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2254032810 con 0x7f226c0d0cd0 2024-04-25T08:38:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.996+0000 7f2270c4c640 1 -- 172.21.15.38:0/3173646069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f226c0d0cd0 msgr2=0x7f226c0d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.996+0000 7f2270c4c640 1 --2- 172.21.15.38:0/3173646069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f226c0d0cd0 0x7f226c0d1130 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f22540097a0 tx=0x7f225402a180 comp rx=0 tx=0).stop 2024-04-25T08:38:53.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.996+0000 7f2270c4c640 1 -- 172.21.15.38:0/3173646069 shutdown_connections 2024-04-25T08:38:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.996+0000 7f2270c4c640 1 --2- 172.21.15.38:0/3173646069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f226c0d0cd0 0x7f226c0d1130 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.996+0000 7f2270c4c640 1 --2- 172.21.15.38:0/3173646069 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f226c15b730 0x7f226c15bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.996+0000 7f2270c4c640 1 -- 172.21.15.38:0/3173646069 >> 172.21.15.38:0/3173646069 conn(0x7f226c0b5c90 msgr2=0x7f226c0b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.996+0000 7f2270c4c640 1 -- 172.21.15.38:0/3173646069 shutdown_connections 2024-04-25T08:38:53.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.996+0000 7f2270c4c640 1 -- 172.21.15.38:0/3173646069 wait complete. 2024-04-25T08:38:53.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.997+0000 7f2270c4c640 1 Processor -- start 2024-04-25T08:38:53.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.997+0000 7f2270c4c640 1 -- start start 2024-04-25T08:38:53.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f2270c4c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f226c0ccd00 0x7f226c13bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:53.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f2270c4c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f226c0d0cd0 0x7f226c0b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:53.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f2270c4c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f226c15b730 0x7f226c158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:53.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f2270c4c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0d0370 con 0x7f226c15b730 2024-04-25T08:38:53.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f2270c4c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0d0090 con 0x7f226c0ccd00 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f2270c4c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0cf4e0 con 0x7f226c0d0cd0 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f226ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f226c15b730 0x7f226c158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f226ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f226c15b730 0x7f226c158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50818/0 (socket says 172.21.15.38:50818) 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f226ad9d640 1 -- 172.21.15.38:0/2599546991 learned_addr learned my addr 172.21.15.38:0/2599546991 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f2263fff640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f226c0d0cd0 0x7f226c0b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f226a59c640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f226c0ccd00 0x7f226c13bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f226ad9d640 1 -- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f226c0d0cd0 msgr2=0x7f226c0b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f226ad9d640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f226c0d0cd0 0x7f226c0b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f226ad9d640 1 -- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f226c0ccd00 msgr2=0x7f226c13bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f226ad9d640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f226c0ccd00 0x7f226c13bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.998+0000 7f226ad9d640 1 -- 172.21.15.38:0/2599546991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c13c4c0 con 0x7f226c15b730 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.999+0000 7f226ad9d640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f226c15b730 0x7f226c158250 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f226400c780 tx=0x7f226400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:53.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.999+0000 7f2262ffd640 1 -- 172.21.15.38:0/2599546991 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264018070 con 0x7f226c15b730 2024-04-25T08:38:53.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.999+0000 7f2262ffd640 1 -- 172.21.15.38:0/2599546991 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400ce30 con 0x7f226c15b730 2024-04-25T08:38:53.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.999+0000 7f2270c4c640 1 -- 172.21.15.38:0/2599546991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226c0b3a00 con 0x7f226c15b730 2024-04-25T08:38:53.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.999+0000 7f2262ffd640 1 -- 172.21.15.38:0/2599546991 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400a840 con 0x7f226c15b730 2024-04-25T08:38:53.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:53.999+0000 7f2270c4c640 1 -- 172.21.15.38:0/2599546991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226c0b3fc0 con 0x7f226c15b730 2024-04-25T08:38:53.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.000+0000 7f2270c4c640 1 -- 172.21.15.38:0/2599546991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226c068e80 con 0x7f226c15b730 2024-04-25T08:38:53.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.000+0000 7f2262ffd640 1 -- 172.21.15.38:0/2599546991 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f226401b070 con 0x7f226c15b730 2024-04-25T08:38:53.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.000+0000 7f2262ffd640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f224c06b520 0x7f224c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:53.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.001+0000 7f2262ffd640 1 -- 172.21.15.38:0/2599546991 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f22640899e0 con 0x7f226c15b730 2024-04-25T08:38:53.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.001+0000 7f226a59c640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f224c06b520 0x7f224c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:53.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.001+0000 7f226a59c640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f224c06b520 0x7f224c06d9e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f225c009b40 tx=0x7f225c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:53.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.005+0000 7f2262ffd640 1 -- 172.21.15.38:0/2599546991 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f226c13c4c0 con 0x7f226c15b730 2024-04-25T08:38:54.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.190+0000 7f2270c4c640 1 -- 172.21.15.38:0/2599546991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f226c06c8b0 con 0x7f226c15b730 2024-04-25T08:38:54.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.191+0000 7f2262ffd640 1 -- 172.21.15.38:0/2599546991 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f22640580d0 con 0x7f226c15b730 2024-04-25T08:38:54.185 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:54.185 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":96,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:38:27.725413+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:24.377232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:25.069513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.193+0000 7f2270c4c640 1 -- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f224c06b520 msgr2=0x7f224c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.193+0000 7f2270c4c640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f224c06b520 0x7f224c06d9e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f225c009b40 tx=0x7f225c009190 comp rx=0 tx=0).stop 2024-04-25T08:38:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.193+0000 7f2270c4c640 1 -- 172.21.15.38:0/2599546991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f226c15b730 msgr2=0x7f226c158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.193+0000 7f2270c4c640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f226c15b730 0x7f226c158250 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f226400c780 tx=0x7f226400cc40 comp rx=0 tx=0).stop 2024-04-25T08:38:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.193+0000 7f2270c4c640 1 -- 172.21.15.38:0/2599546991 shutdown_connections 2024-04-25T08:38:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.193+0000 7f2270c4c640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f226c15b730 0x7f226c158250 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.193+0000 7f2270c4c640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f224c06b520 0x7f224c06d9e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.193+0000 7f2270c4c640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f226c0d0cd0 0x7f226c0b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.193+0000 7f2270c4c640 1 --2- 172.21.15.38:0/2599546991 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f226c0ccd00 0x7f226c13bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.193+0000 7f2270c4c640 1 -- 172.21.15.38:0/2599546991 >> 172.21.15.38:0/2599546991 conn(0x7f226c0b5c90 msgr2=0x7f226c156b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:54.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.193+0000 7f2270c4c640 1 -- 172.21.15.38:0/2599546991 shutdown_connections 2024-04-25T08:38:54.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.193+0000 7f2270c4c640 1 -- 172.21.15.38:0/2599546991 wait complete. 2024-04-25T08:38:54.198 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:38:24.377232+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:38:25.069513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:38:54.199 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:38:54.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.309+0000 7fe28cefc640 1 Processor -- start 2024-04-25T08:38:54.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.309+0000 7fe28cefc640 1 -- start start 2024-04-25T08:38:54.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.310+0000 7fe28cefc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe288166610 0x7fe288168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.310+0000 7fe28cefc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2880d2060 con 0x7fe2880d33d0 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.310+0000 7fe28cefc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2880d2330 con 0x7fe2880d0320 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.310+0000 7fe28cefc640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2880d2610 con 0x7fe288166610 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.310+0000 7fe285d9b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe2880d33d0 0x7fe2880d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50360/0 (socket says 172.21.15.38:50360) 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.310+0000 7fe285d9b640 1 -- 172.21.15.38:0/1143624570 learned_addr learned my addr 172.21.15.38:0/1143624570 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.310+0000 7fe286d9d640 1 --2- 172.21.15.38:0/1143624570 >> v2:172.21.15.107:3300/0 conn(0x7fe288166610 0x7fe288168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016676349 0 0) 0x7fe2880d2060 con 0x7fe2880d33d0 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe274003520 con 0x7fe2880d33d0 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2495474373 0 0) 0x7fe274003520 con 0x7fe2880d33d0 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 >> v2:172.21.15.107:3300/0 conn(0x7fe288166610 msgr2=0x7fe288168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 --2- 172.21.15.38:0/1143624570 >> v2:172.21.15.107:3300/0 conn(0x7fe288166610 0x7fe288168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 >> v1:172.21.15.107:6789/0 conn(0x7fe2880d0320 legacy=0x7fe2880d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880d28f0 con 0x7fe2880d33d0 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fe270001670 con 0x7fe2880d33d0 2024-04-25T08:38:54.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 >> v1:172.21.15.38:6789/0 conn(0x7fe2880d33d0 legacy=0x7fe2880d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 --2- 172.21.15.38:0/1143624570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2740039b0 0x7fe274003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 --2- 172.21.15.38:0/1143624570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe274004800 0x7fe274004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 --2- 172.21.15.38:0/1143624570 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2740051c0 0x7fe27400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2880d2610 con 0x7fe2740039b0 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27400c070 con 0x7fe274004800 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27400c270 con 0x7fe2740051c0 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28659c640 1 --2- 172.21.15.38:0/1143624570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2740039b0 0x7fe274003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe285d9b640 1 --2- 172.21.15.38:0/1143624570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe274004800 0x7fe274004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe286d9d640 1 --2- 172.21.15.38:0/1143624570 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2740051c0 0x7fe27400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28659c640 1 -- 172.21.15.38:0/1143624570 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2740051c0 msgr2=0x7fe27400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28659c640 1 --2- 172.21.15.38:0/1143624570 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2740051c0 0x7fe27400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28659c640 1 -- 172.21.15.38:0/1143624570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe274004800 msgr2=0x7fe274004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28659c640 1 --2- 172.21.15.38:0/1143624570 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe274004800 0x7fe274004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe28659c640 1 -- 172.21.15.38:0/1143624570 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe274003520 con 0x7fe2740039b0 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.311+0000 7fe286d9d640 1 -- 172.21.15.38:0/1143624570 reap_dead start 2024-04-25T08:38:54.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.312+0000 7fe28659c640 1 --2- 172.21.15.38:0/1143624570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2740039b0 0x7fe274003dd0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe278001e50 tx=0x7fe274005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133f4012ee1293e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:54.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.312+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278001a70 con 0x7fe2740039b0 2024-04-25T08:38:54.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.312+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27802ae40 con 0x7fe2740039b0 2024-04-25T08:38:54.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.312+0000 7fe28559a640 1 -- 172.21.15.38:0/1143624570 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe278003bb0 con 0x7fe2740039b0 2024-04-25T08:38:54.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.313+0000 7fe28cefc640 1 -- 172.21.15.38:0/1143624570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2740039b0 msgr2=0x7fe274003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.313+0000 7fe28cefc640 1 --2- 172.21.15.38:0/1143624570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2740039b0 0x7fe274003dd0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fe278001e50 tx=0x7fe274005830 comp rx=0 tx=0).stop 2024-04-25T08:38:54.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.313+0000 7fe28cefc640 1 -- 172.21.15.38:0/1143624570 shutdown_connections 2024-04-25T08:38:54.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.313+0000 7fe28cefc640 1 --2- 172.21.15.38:0/1143624570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2740039b0 0x7fe274003dd0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.313+0000 7fe28cefc640 1 -- 172.21.15.38:0/1143624570 >> 172.21.15.38:0/1143624570 conn(0x7fe2880668f0 msgr2=0x7fe288066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:54.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.313+0000 7fe28cefc640 1 -- 172.21.15.38:0/1143624570 shutdown_connections 2024-04-25T08:38:54.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.313+0000 7fe28cefc640 1 -- 172.21.15.38:0/1143624570 wait complete. 2024-04-25T08:38:54.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.314+0000 7fe28cefc640 1 Processor -- start 2024-04-25T08:38:54.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.314+0000 7fe28cefc640 1 -- start start 2024-04-25T08:38:54.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe28cefc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2740039b0 0x7fe28814eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe28cefc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2880cbbb0 0x7fe28815de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe28cefc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe28815e3d0 0x7fe288150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe28cefc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe288161210 con 0x7fe2880cbbb0 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe28cefc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2880d2eb0 con 0x7fe2740039b0 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe28cefc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe288161390 con 0x7fe28815e3d0 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe285d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2880cbbb0 0x7fe28815de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe285d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2880cbbb0 0x7fe28815de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50824/0 (socket says 172.21.15.38:50824) 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe285d9b640 1 -- 172.21.15.38:0/2949241925 learned_addr learned my addr 172.21.15.38:0/2949241925 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe28659c640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2740039b0 0x7fe28814eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe286d9d640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe28815e3d0 0x7fe288150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe285d9b640 1 -- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe28815e3d0 msgr2=0x7fe288150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe285d9b640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe28815e3d0 0x7fe288150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe285d9b640 1 -- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2740039b0 msgr2=0x7fe28814eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe285d9b640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2740039b0 0x7fe28814eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe285d9b640 1 -- 172.21.15.38:0/2949241925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2880cc8b0 con 0x7fe2880cbbb0 2024-04-25T08:38:54.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.315+0000 7fe285d9b640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2880cbbb0 0x7fe28815de90 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe270007320 tx=0x7fe2700077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:54.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.316+0000 7fe27effd640 1 -- 172.21.15.38:0/2949241925 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27001a070 con 0x7fe2880cbbb0 2024-04-25T08:38:54.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.316+0000 7fe27effd640 1 -- 172.21.15.38:0/2949241925 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27000f8e0 con 0x7fe2880cbbb0 2024-04-25T08:38:54.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.316+0000 7fe28cefc640 1 -- 172.21.15.38:0/2949241925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2880ccb90 con 0x7fe2880cbbb0 2024-04-25T08:38:54.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.316+0000 7fe27effd640 1 -- 172.21.15.38:0/2949241925 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27000fcd0 con 0x7fe2880cbbb0 2024-04-25T08:38:54.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.316+0000 7fe28cefc640 1 -- 172.21.15.38:0/2949241925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2880cd150 con 0x7fe2880cbbb0 2024-04-25T08:38:54.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.317+0000 7fe28cefc640 1 -- 172.21.15.38:0/2949241925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe288068e80 con 0x7fe2880cbbb0 2024-04-25T08:38:54.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.317+0000 7fe27effd640 1 -- 172.21.15.38:0/2949241925 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe27001d070 con 0x7fe2880cbbb0 2024-04-25T08:38:54.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.317+0000 7fe27effd640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe26806b500 0x7fe26806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.318+0000 7fe27effd640 1 -- 172.21.15.38:0/2949241925 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe270089c30 con 0x7fe2880cbbb0 2024-04-25T08:38:54.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.318+0000 7fe28659c640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe26806b500 0x7fe26806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.318+0000 7fe28659c640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe26806b500 0x7fe26806d9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe278032000 tx=0x7fe27802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:54.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.322+0000 7fe27effd640 1 -- 172.21.15.38:0/2949241925 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe270058920 con 0x7fe2880cbbb0 2024-04-25T08:38:54.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.500+0000 7fe28cefc640 1 -- 172.21.15.38:0/2949241925 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2880b3050 con 0x7fe2880cbbb0 2024-04-25T08:38:54.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.501+0000 7fe27effd640 1 -- 172.21.15.38:0/2949241925 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe270058220 con 0x7fe2880cbbb0 2024-04-25T08:38:54.494 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:38:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.503+0000 7fe28cefc640 1 -- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe26806b500 msgr2=0x7fe26806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.503+0000 7fe28cefc640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe26806b500 0x7fe26806d9c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe278032000 tx=0x7fe27802e000 comp rx=0 tx=0).stop 2024-04-25T08:38:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.503+0000 7fe28cefc640 1 -- 172.21.15.38:0/2949241925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2880cbbb0 msgr2=0x7fe28815de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.503+0000 7fe28cefc640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2880cbbb0 0x7fe28815de90 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe270007320 tx=0x7fe2700077e0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.503+0000 7fe28cefc640 1 -- 172.21.15.38:0/2949241925 shutdown_connections 2024-04-25T08:38:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.503+0000 7fe28cefc640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe26806b500 0x7fe26806d9c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.503+0000 7fe28cefc640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe28815e3d0 0x7fe288150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.503+0000 7fe28cefc640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe2880cbbb0 0x7fe28815de90 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.503+0000 7fe28cefc640 1 --2- 172.21.15.38:0/2949241925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2740039b0 0x7fe28814eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.503+0000 7fe28cefc640 1 -- 172.21.15.38:0/2949241925 >> 172.21.15.38:0/2949241925 conn(0x7fe2880668f0 msgr2=0x7fe2880d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.503+0000 7fe28cefc640 1 -- 172.21.15.38:0/2949241925 shutdown_connections 2024-04-25T08:38:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.503+0000 7fe28cefc640 1 -- 172.21.15.38:0/2949241925 wait complete. 2024-04-25T08:38:54.508 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:38:54.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.618+0000 7f26d90db640 1 Processor -- start 2024-04-25T08:38:54.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.618+0000 7f26d90db640 1 -- start start 2024-04-25T08:38:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.619+0000 7f26d90db640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f26d40d1e00 0x7f26d40d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.619+0000 7f26d90db640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f26d40cf7b0 0x7f26d40b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.619+0000 7f26d90db640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d40ce710 con 0x7f26d415de60 2024-04-25T08:38:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.619+0000 7f26d90db640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d40ce4e0 con 0x7f26d40d1e00 2024-04-25T08:38:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.619+0000 7f26d90db640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d40cf320 con 0x7f26d40cf7b0 2024-04-25T08:38:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.619+0000 7f26d2d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f26d40d1e00 0x7f26d40d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.619+0000 7f26d2d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f26d40d1e00 0x7f26d40d42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50836/0 (socket says 172.21.15.38:50836) 2024-04-25T08:38:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.619+0000 7f26d2d9d640 1 -- 172.21.15.38:0/3609678367 learned_addr learned my addr 172.21.15.38:0/3609678367 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.619+0000 7f26d359e640 1 --2- 172.21.15.38:0/3609678367 >> v2:172.21.15.107:3300/0 conn(0x7f26d40cf7b0 0x7f26d40b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.620+0000 7f26d2d9d640 1 -- 172.21.15.38:0/3609678367 >> v2:172.21.15.107:3300/0 conn(0x7f26d40cf7b0 msgr2=0x7f26d40b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.620+0000 7f26d2d9d640 1 --2- 172.21.15.38:0/3609678367 >> v2:172.21.15.107:3300/0 conn(0x7f26d40cf7b0 0x7f26d40b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.620+0000 7f26d2d9d640 1 -- 172.21.15.38:0/3609678367 >> v1:172.21.15.107:6789/0 conn(0x7f26d415de60 legacy=0x7f26d40d21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.620+0000 7f26d2d9d640 1 -- 172.21.15.38:0/3609678367 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d40ce9f0 con 0x7f26d40d1e00 2024-04-25T08:38:54.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.620+0000 7f26d2d9d640 1 --2- 172.21.15.38:0/3609678367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d40d1e00 0x7f26d40d42a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f26d40cf4a0 tx=0x7f26cc02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda1fe695606ea6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:54.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.620+0000 7f26d1d9b640 1 -- 172.21.15.38:0/3609678367 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc037070 con 0x7f26d40d1e00 2024-04-25T08:38:54.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.620+0000 7f26d1d9b640 1 -- 172.21.15.38:0/3609678367 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26cc032440 con 0x7f26d40d1e00 2024-04-25T08:38:54.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.620+0000 7f26d1d9b640 1 -- 172.21.15.38:0/3609678367 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc032810 con 0x7f26d40d1e00 2024-04-25T08:38:54.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.621+0000 7f26d90db640 1 -- 172.21.15.38:0/3609678367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d40d1e00 msgr2=0x7f26d40d42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.621+0000 7f26d90db640 1 --2- 172.21.15.38:0/3609678367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d40d1e00 0x7f26d40d42a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f26d40cf4a0 tx=0x7f26cc02a290 comp rx=0 tx=0).stop 2024-04-25T08:38:54.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.621+0000 7f26d90db640 1 -- 172.21.15.38:0/3609678367 shutdown_connections 2024-04-25T08:38:54.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.621+0000 7f26d90db640 1 --2- 172.21.15.38:0/3609678367 >> v2:172.21.15.107:3300/0 conn(0x7f26d40cf7b0 0x7f26d40b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.621+0000 7f26d90db640 1 --2- 172.21.15.38:0/3609678367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d40d1e00 0x7f26d40d42a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.621+0000 7f26d90db640 1 -- 172.21.15.38:0/3609678367 >> 172.21.15.38:0/3609678367 conn(0x7f26d408b000 msgr2=0x7f26d40d3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:54.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.621+0000 7f26d90db640 1 -- 172.21.15.38:0/3609678367 shutdown_connections 2024-04-25T08:38:54.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.621+0000 7f26d90db640 1 -- 172.21.15.38:0/3609678367 wait complete. 2024-04-25T08:38:54.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.622+0000 7f26d90db640 1 Processor -- start 2024-04-25T08:38:54.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.623+0000 7f26d90db640 1 -- start start 2024-04-25T08:38:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.623+0000 7f26d90db640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d40cf7b0 0x7f26d41528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.623+0000 7f26d90db640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26d40d1e00 0x7f26d40c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.623+0000 7f26d90db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d415de60 0x7f26d4160110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.623+0000 7f26d90db640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40d1600 con 0x7f26d40cf7b0 2024-04-25T08:38:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.623+0000 7f26d90db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40cefb0 con 0x7f26d415de60 2024-04-25T08:38:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.623+0000 7f26d90db640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d40d1780 con 0x7f26d40d1e00 2024-04-25T08:38:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.623+0000 7f26d2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d40cf7b0 0x7f26d41528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.623+0000 7f26d2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d40cf7b0 0x7f26d41528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50844/0 (socket says 172.21.15.38:50844) 2024-04-25T08:38:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.623+0000 7f26d2d9d640 1 -- 172.21.15.38:0/1063122759 learned_addr learned my addr 172.21.15.38:0/1063122759 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26d259c640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26d40d1e00 0x7f26d40c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26d2d9d640 1 -- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26d40d1e00 msgr2=0x7f26d40c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26d2d9d640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26d40d1e00 0x7f26d40c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26d359e640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d415de60 0x7f26d4160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26d2d9d640 1 -- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d415de60 msgr2=0x7f26d4160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26d2d9d640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d415de60 0x7f26d4160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26d2d9d640 1 -- 172.21.15.38:0/1063122759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d40c8500 con 0x7f26d40cf7b0 2024-04-25T08:38:54.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26d2d9d640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d40cf7b0 0x7f26d41528f0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02a2d0 tx=0x7f26cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:54.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26cb7fe640 1 -- 172.21.15.38:0/1063122759 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc041020 con 0x7f26d40cf7b0 2024-04-25T08:38:54.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26cb7fe640 1 -- 172.21.15.38:0/1063122759 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26cc0306c0 con 0x7f26d40cf7b0 2024-04-25T08:38:54.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26d90db640 1 -- 172.21.15.38:0/1063122759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d40c87e0 con 0x7f26d40cf7b0 2024-04-25T08:38:54.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26cb7fe640 1 -- 172.21.15.38:0/1063122759 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26cc037040 con 0x7f26d40cf7b0 2024-04-25T08:38:54.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.624+0000 7f26d90db640 1 -- 172.21.15.38:0/1063122759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d40c8da0 con 0x7f26d40cf7b0 2024-04-25T08:38:54.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.625+0000 7f26d90db640 1 -- 172.21.15.38:0/1063122759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d4068e80 con 0x7f26d40cf7b0 2024-04-25T08:38:54.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.626+0000 7f26cb7fe640 1 -- 172.21.15.38:0/1063122759 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26cc044020 con 0x7f26d40cf7b0 2024-04-25T08:38:54.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.626+0000 7f26cb7fe640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f26b406b500 0x7f26b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.626+0000 7f26cb7fe640 1 -- 172.21.15.38:0/1063122759 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f26cc0aaab0 con 0x7f26d40cf7b0 2024-04-25T08:38:54.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.626+0000 7f26d259c640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f26b406b500 0x7f26b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.627+0000 7f26d259c640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f26b406b500 0x7f26b406d9c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f26d41f7820 tx=0x7f26bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:54.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.630+0000 7f26cb7fe640 1 -- 172.21.15.38:0/1063122759 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26cc04b030 con 0x7f26d40cf7b0 2024-04-25T08:38:54.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.808+0000 7f26d90db640 1 -- 172.21.15.38:0/1063122759 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26d40d1be0 con 0x7f26d40cf7b0 2024-04-25T08:38:54.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.809+0000 7f26cb7fe640 1 -- 172.21.15.38:0/1063122759 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26cc079830 con 0x7f26d40cf7b0 2024-04-25T08:38:54.802 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:38:54.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.811+0000 7f26d90db640 1 -- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f26b406b500 msgr2=0x7f26b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.811+0000 7f26d90db640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f26b406b500 0x7f26b406d9c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f26d41f7820 tx=0x7f26bc009040 comp rx=0 tx=0).stop 2024-04-25T08:38:54.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.811+0000 7f26d90db640 1 -- 172.21.15.38:0/1063122759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d40cf7b0 msgr2=0x7f26d41528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.811+0000 7f26d90db640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d40cf7b0 0x7f26d41528f0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f26cc02a2d0 tx=0x7f26cc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.811+0000 7f26d90db640 1 -- 172.21.15.38:0/1063122759 shutdown_connections 2024-04-25T08:38:54.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.811+0000 7f26d90db640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d415de60 0x7f26d4160110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.811+0000 7f26d90db640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f26b406b500 0x7f26b406d9c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.811+0000 7f26d90db640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26d40d1e00 0x7f26d40c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.811+0000 7f26d90db640 1 --2- 172.21.15.38:0/1063122759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f26d40cf7b0 0x7f26d41528f0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.811+0000 7f26d90db640 1 -- 172.21.15.38:0/1063122759 >> 172.21.15.38:0/1063122759 conn(0x7f26d408b000 msgr2=0x7f26d415c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:54.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.811+0000 7f26d90db640 1 -- 172.21.15.38:0/1063122759 shutdown_connections 2024-04-25T08:38:54.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.811+0000 7f26d90db640 1 -- 172.21.15.38:0/1063122759 wait complete. 2024-04-25T08:38:54.816 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:38:54.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.927+0000 7fdae966f640 1 Processor -- start 2024-04-25T08:38:54.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.927+0000 7fdae966f640 1 -- start start 2024-04-25T08:38:54.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.928+0000 7fdae966f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdae4166610 0x7fdae4168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.928+0000 7fdae966f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae40d2060 con 0x7fdae40d33d0 2024-04-25T08:38:54.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.928+0000 7fdae966f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae40d2330 con 0x7fdae40d0320 2024-04-25T08:38:54.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.928+0000 7fdae966f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae40d2610 con 0x7fdae4166610 2024-04-25T08:38:54.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.928+0000 7fdae37fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdae4166610 0x7fdae4168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.928+0000 7fdadbfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdae40d33d0 0x7fdae40d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50364/0 (socket says 172.21.15.38:50364) 2024-04-25T08:38:54.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.928+0000 7fdadbfff640 1 -- 172.21.15.38:0/3054882037 learned_addr learned my addr 172.21.15.38:0/3054882037 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:54.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.928+0000 7fdae37fe640 1 -- 172.21.15.38:0/3054882037 >> v1:172.21.15.107:6789/0 conn(0x7fdae40d0320 legacy=0x7fdae40d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:54.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.928+0000 7fdae37fe640 1 -- 172.21.15.38:0/3054882037 >> v1:172.21.15.38:6789/0 conn(0x7fdae40d33d0 legacy=0x7fdae40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.928+0000 7fdae37fe640 1 -- 172.21.15.38:0/3054882037 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae40d28f0 con 0x7fdae4166610 2024-04-25T08:38:54.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.928+0000 7fdae27fc640 1 -- 172.21.15.38:0/3054882037 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876933589 0 0) 0x7fdae40d2060 con 0x7fdae40d33d0 2024-04-25T08:38:54.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.929+0000 7fdae37fe640 1 --2- 172.21.15.38:0/3054882037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4166610 0x7fdae4168a00 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009d90 tx=0x7fdadc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2b602934c1774f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:54.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.929+0000 7fdae27fc640 1 -- 172.21.15.38:0/3054882037 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc007740 con 0x7fdae4166610 2024-04-25T08:38:54.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.929+0000 7fdae27fc640 1 -- 172.21.15.38:0/3054882037 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc0078e0 con 0x7fdae4166610 2024-04-25T08:38:54.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.929+0000 7fdae27fc640 1 -- 172.21.15.38:0/3054882037 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc007cb0 con 0x7fdae4166610 2024-04-25T08:38:54.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.929+0000 7fdae966f640 1 -- 172.21.15.38:0/3054882037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4166610 msgr2=0x7fdae4168a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.929+0000 7fdae966f640 1 --2- 172.21.15.38:0/3054882037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4166610 0x7fdae4168a00 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009d90 tx=0x7fdadc02a730 comp rx=0 tx=0).stop 2024-04-25T08:38:54.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.929+0000 7fdae966f640 1 -- 172.21.15.38:0/3054882037 shutdown_connections 2024-04-25T08:38:54.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.929+0000 7fdae966f640 1 --2- 172.21.15.38:0/3054882037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4166610 0x7fdae4168a00 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.929+0000 7fdae966f640 1 -- 172.21.15.38:0/3054882037 >> 172.21.15.38:0/3054882037 conn(0x7fdae40668f0 msgr2=0x7fdae4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:54.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.930+0000 7fdae966f640 1 -- 172.21.15.38:0/3054882037 shutdown_connections 2024-04-25T08:38:54.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.930+0000 7fdae966f640 1 -- 172.21.15.38:0/3054882037 wait complete. 2024-04-25T08:38:54.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.931+0000 7fdae966f640 1 Processor -- start 2024-04-25T08:38:54.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.931+0000 7fdae966f640 1 -- start start 2024-04-25T08:38:54.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.931+0000 7fdae966f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdae40d0320 0x7fdae414ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.931+0000 7fdae966f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdae40d33d0 0x7fdae40cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.931+0000 7fdae966f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4166610 0x7fdae40cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.931+0000 7fdae966f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae4161210 con 0x7fdae4166610 2024-04-25T08:38:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.931+0000 7fdae966f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae40d2eb0 con 0x7fdae40d0320 2024-04-25T08:38:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.931+0000 7fdae966f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae4161390 con 0x7fdae40d33d0 2024-04-25T08:38:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4166610 0x7fdae40cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4166610 0x7fdae40cbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50862/0 (socket says 172.21.15.38:50862) 2024-04-25T08:38:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae37fe640 1 -- 172.21.15.38:0/2795105634 learned_addr learned my addr 172.21.15.38:0/2795105634 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdadbfff640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdae40d33d0 0x7fdae40cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdae40d0320 0x7fdae414ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae37fe640 1 -- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdae40d33d0 msgr2=0x7fdae40cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae37fe640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdae40d33d0 0x7fdae40cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae37fe640 1 -- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdae40d0320 msgr2=0x7fdae414ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae37fe640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdae40d0320 0x7fdae414ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:54.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae37fe640 1 -- 172.21.15.38:0/2795105634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae4157c70 con 0x7fdae4166610 2024-04-25T08:38:54.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdae40d0320 0x7fdae414ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:54.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdadbfff640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdae40d33d0 0x7fdae40cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:54.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae37fe640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4166610 0x7fdae40cbf10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fdadc02ad20 tx=0x7fdadc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:54.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdadb7fe640 1 -- 172.21.15.38:0/2795105634 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc040070 con 0x7fdae4166610 2024-04-25T08:38:54.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdadb7fe640 1 -- 172.21.15.38:0/2795105634 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdadc031c90 con 0x7fdae4166610 2024-04-25T08:38:54.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae966f640 1 -- 172.21.15.38:0/2795105634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae40b3310 con 0x7fdae4166610 2024-04-25T08:38:54.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdadb7fe640 1 -- 172.21.15.38:0/2795105634 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc045680 con 0x7fdae4166610 2024-04-25T08:38:54.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.932+0000 7fdae966f640 1 -- 172.21.15.38:0/2795105634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae40b38d0 con 0x7fdae4166610 2024-04-25T08:38:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.933+0000 7fdae966f640 1 -- 172.21.15.38:0/2795105634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae4068e80 con 0x7fdae4166610 2024-04-25T08:38:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.934+0000 7fdadb7fe640 1 -- 172.21.15.38:0/2795105634 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdadc039040 con 0x7fdae4166610 2024-04-25T08:38:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.934+0000 7fdadb7fe640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdac406b4d0 0x7fdac406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.934+0000 7fdadb7fe640 1 -- 172.21.15.38:0/2795105634 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdadc0abc80 con 0x7fdae4166610 2024-04-25T08:38:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.934+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdac406b4d0 0x7fdac406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:54.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.935+0000 7fdae2ffd640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdac406b4d0 0x7fdac406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdad4007fd0 tx=0x7fdad4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:54.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:54.938+0000 7fdadb7fe640 1 -- 172.21.15.38:0/2795105634 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdadc07aa30 con 0x7fdae4166610 2024-04-25T08:38:55.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.119+0000 7fdae966f640 1 -- 172.21.15.38:0/2795105634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdae415bef0 con 0x7fdae4166610 2024-04-25T08:38:55.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.120+0000 7fdadb7fe640 1 -- 172.21.15.38:0/2795105634 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdadc07a330 con 0x7fdae4166610 2024-04-25T08:38:55.113 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:38:55.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.122+0000 7fdae966f640 1 -- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdac406b4d0 msgr2=0x7fdac406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.122+0000 7fdae966f640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdac406b4d0 0x7fdac406d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fdad4007fd0 tx=0x7fdad4009040 comp rx=0 tx=0).stop 2024-04-25T08:38:55.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.122+0000 7fdae966f640 1 -- 172.21.15.38:0/2795105634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4166610 msgr2=0x7fdae40cbf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.122+0000 7fdae966f640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4166610 0x7fdae40cbf10 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fdadc02ad20 tx=0x7fdadc004a40 comp rx=0 tx=0).stop 2024-04-25T08:38:55.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.122+0000 7fdae966f640 1 -- 172.21.15.38:0/2795105634 shutdown_connections 2024-04-25T08:38:55.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.122+0000 7fdae966f640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdae4166610 0x7fdae40cbf10 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.122+0000 7fdae966f640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdac406b4d0 0x7fdac406d990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.122+0000 7fdae966f640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdae40d33d0 0x7fdae40cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.122+0000 7fdae966f640 1 --2- 172.21.15.38:0/2795105634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdae40d0320 0x7fdae414ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.122+0000 7fdae966f640 1 -- 172.21.15.38:0/2795105634 >> 172.21.15.38:0/2795105634 conn(0x7fdae40668f0 msgr2=0x7fdae40b8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:55.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.122+0000 7fdae966f640 1 -- 172.21.15.38:0/2795105634 shutdown_connections 2024-04-25T08:38:55.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.122+0000 7fdae966f640 1 -- 172.21.15.38:0/2795105634 wait complete. 2024-04-25T08:38:55.127 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.234+0000 7f0f4dfb7640 1 Processor -- start 2024-04-25T08:38:55.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.234+0000 7f0f4dfb7640 1 -- start start 2024-04-25T08:38:55.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.235+0000 7f0f4dfb7640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0f4806cff0 0x7f0f48099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.235+0000 7f0f4dfb7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0f48099fb0 0x7f0f4815d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.235+0000 7f0f4dfb7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4809a3c0 con 0x7f0f4806cc10 2024-04-25T08:38:55.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.235+0000 7f0f4dfb7640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f480d05c0 con 0x7f0f48099fb0 2024-04-25T08:38:55.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.235+0000 7f0f4dfb7640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f480d08a0 con 0x7f0f4806cff0 2024-04-25T08:38:55.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.235+0000 7f0f47fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0f48099fb0 0x7f0f4815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.235+0000 7f0f47fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0f48099fb0 0x7f0f4815d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50866/0 (socket says 172.21.15.38:50866) 2024-04-25T08:38:55.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.235+0000 7f0f47fff640 1 -- 172.21.15.38:0/154594460 learned_addr learned my addr 172.21.15.38:0/154594460 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:55.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.236+0000 7f0f46ffd640 1 --2- 172.21.15.38:0/154594460 >> v2:172.21.15.107:3300/0 conn(0x7f0f4806cff0 0x7f0f48099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.236+0000 7f0f47fff640 1 -- 172.21.15.38:0/154594460 >> v2:172.21.15.107:3300/0 conn(0x7f0f4806cff0 msgr2=0x7f0f48099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:55.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.236+0000 7f0f47fff640 1 --2- 172.21.15.38:0/154594460 >> v2:172.21.15.107:3300/0 conn(0x7f0f4806cff0 0x7f0f48099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.236+0000 7f0f47fff640 1 -- 172.21.15.38:0/154594460 >> v1:172.21.15.107:6789/0 conn(0x7f0f4806cc10 legacy=0x7f0f480cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.236+0000 7f0f47fff640 1 -- 172.21.15.38:0/154594460 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f480d0b80 con 0x7f0f48099fb0 2024-04-25T08:38:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.236+0000 7f0f47fff640 1 --2- 172.21.15.38:0/154594460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f48099fb0 0x7f0f4815d370 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f0f4806db60 tx=0x7f0f4002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a6d2578ddde34d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.237+0000 7f0f467fc640 1 -- 172.21.15.38:0/154594460 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40036070 con 0x7f0f48099fb0 2024-04-25T08:38:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.237+0000 7f0f467fc640 1 -- 172.21.15.38:0/154594460 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f40031600 con 0x7f0f48099fb0 2024-04-25T08:38:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.237+0000 7f0f467fc640 1 -- 172.21.15.38:0/154594460 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f400319d0 con 0x7f0f48099fb0 2024-04-25T08:38:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.237+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/154594460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f48099fb0 msgr2=0x7f0f4815d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.237+0000 7f0f4dfb7640 1 --2- 172.21.15.38:0/154594460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f48099fb0 0x7f0f4815d370 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f0f4806db60 tx=0x7f0f4002a730 comp rx=0 tx=0).stop 2024-04-25T08:38:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.237+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/154594460 shutdown_connections 2024-04-25T08:38:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.237+0000 7f0f4dfb7640 1 --2- 172.21.15.38:0/154594460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f48099fb0 0x7f0f4815d370 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.237+0000 7f0f4dfb7640 1 --2- 172.21.15.38:0/154594460 >> v2:172.21.15.107:3300/0 conn(0x7f0f4806cff0 0x7f0f48099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.237+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/154594460 >> 172.21.15.38:0/154594460 conn(0x7f0f480b31f0 msgr2=0x7f0f480b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.237+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/154594460 shutdown_connections 2024-04-25T08:38:55.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.237+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/154594460 wait complete. 2024-04-25T08:38:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.238+0000 7f0f4dfb7640 1 Processor -- start 2024-04-25T08:38:55.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.239+0000 7f0f4dfb7640 1 -- start start 2024-04-25T08:38:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.239+0000 7f0f4dfb7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f4806cc10 0x7f0f481e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.239+0000 7f0f4dfb7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4806cff0 0x7f0f481e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.239+0000 7f0f4dfb7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f48099fb0 0x7f0f480ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.239+0000 7f0f4dfb7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4806c150 con 0x7f0f48099fb0 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.239+0000 7f0f4dfb7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f480d1140 con 0x7f0f4806cff0 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.239+0000 7f0f4dfb7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4806c2d0 con 0x7f0f4806cc10 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.239+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f48099fb0 0x7f0f480ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f48099fb0 0x7f0f480ae360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50870/0 (socket says 172.21.15.38:50870) 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f47fff640 1 -- 172.21.15.38:0/4203525903 learned_addr learned my addr 172.21.15.38:0/4203525903 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f46ffd640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4806cff0 0x7f0f481e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f477fe640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f4806cc10 0x7f0f481e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f47fff640 1 -- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f4806cc10 msgr2=0x7f0f481e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f47fff640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f4806cc10 0x7f0f481e6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f47fff640 1 -- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4806cff0 msgr2=0x7f0f481e3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f47fff640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4806cff0 0x7f0f481e3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f47fff640 1 -- 172.21.15.38:0/4203525903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f481e7180 con 0x7f0f48099fb0 2024-04-25T08:38:55.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f477fe640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f4806cc10 0x7f0f481e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f46ffd640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4806cff0 0x7f0f481e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f47fff640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f48099fb0 0x7f0f480ae360 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0f40034040 tx=0x7f0f40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f2ffff640 1 -- 172.21.15.38:0/4203525903 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40036050 con 0x7f0f48099fb0 2024-04-25T08:38:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f2ffff640 1 -- 172.21.15.38:0/4203525903 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f40042070 con 0x7f0f48099fb0 2024-04-25T08:38:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/4203525903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f481e7790 con 0x7f0f48099fb0 2024-04-25T08:38:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f2ffff640 1 -- 172.21.15.38:0/4203525903 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40009510 con 0x7f0f48099fb0 2024-04-25T08:38:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.240+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/4203525903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f481e7d50 con 0x7f0f48099fb0 2024-04-25T08:38:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.241+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/4203525903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f48067bc0 con 0x7f0f48099fb0 2024-04-25T08:38:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.242+0000 7f0f2ffff640 1 -- 172.21.15.38:0/4203525903 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f40007bf0 con 0x7f0f48099fb0 2024-04-25T08:38:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.242+0000 7f0f2ffff640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f3406b4d0 0x7f0f3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.242+0000 7f0f2ffff640 1 -- 172.21.15.38:0/4203525903 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f400ab1e0 con 0x7f0f48099fb0 2024-04-25T08:38:55.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.243+0000 7f0f477fe640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f3406b4d0 0x7f0f3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.243+0000 7f0f477fe640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f3406b4d0 0x7f0f3406d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0f38001680 tx=0x7f0f38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:55.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.246+0000 7f0f2ffff640 1 -- 172.21.15.38:0/4203525903 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f48067bc0 con 0x7f0f48099fb0 2024-04-25T08:38:55.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.462+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/4203525903 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f48068e80 con 0x7f0f48099fb0 2024-04-25T08:38:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.462+0000 7f0f2ffff640 1 -- 172.21.15.38:0/4203525903 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0f40079ff0 con 0x7f0f48099fb0 2024-04-25T08:38:55.456 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:55.456 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3680979257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3680979257},{"type":"v1","addr":"172.21.15.107:6836","nonce":3680979257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/697909489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":697909489},{"type":"v1","addr":"172.21.15.107:6837","nonce":697909489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3083871141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3083871141},{"type":"v1","addr":"172.21.15.38:6834","nonce":3083871141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:26.095391+0000","modified":"2024-04-25T08:38:28.097807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/2922734840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":2922734840},{"type":"v1","addr":"172.21.15.38:6835","nonce":2922734840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:38:55.456 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 57 2024-04-25T08:38:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.464+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f3406b4d0 msgr2=0x7f0f3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.464+0000 7f0f4dfb7640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f3406b4d0 0x7f0f3406d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0f38001680 tx=0x7f0f38009040 comp rx=0 tx=0).stop 2024-04-25T08:38:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.464+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/4203525903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f48099fb0 msgr2=0x7f0f480ae360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.464+0000 7f0f4dfb7640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f48099fb0 0x7f0f480ae360 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f0f40034040 tx=0x7f0f40004a40 comp rx=0 tx=0).stop 2024-04-25T08:38:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.464+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/4203525903 shutdown_connections 2024-04-25T08:38:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.464+0000 7f0f4dfb7640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f48099fb0 0x7f0f480ae360 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.464+0000 7f0f4dfb7640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f4806cff0 0x7f0f481e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.464+0000 7f0f4dfb7640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f3406b4d0 0x7f0f3406d990 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.464+0000 7f0f4dfb7640 1 --2- 172.21.15.38:0/4203525903 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f4806cc10 0x7f0f481e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.464+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/4203525903 >> 172.21.15.38:0/4203525903 conn(0x7f0f480b31f0 msgr2=0x7f0f481e5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.464+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/4203525903 shutdown_connections 2024-04-25T08:38:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.464+0000 7f0f4dfb7640 1 -- 172.21.15.38:0/4203525903 wait complete. 2024-04-25T08:38:55.469 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:55.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.576+0000 7f927bef5640 1 Processor -- start 2024-04-25T08:38:55.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.576+0000 7f927bef5640 1 -- start start 2024-04-25T08:38:55.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.577+0000 7f927bef5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f927415dc10 0x7f92740ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.577+0000 7f927bef5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f92740ced60 0x7f92740d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.577+0000 7f927bef5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92740d3e70 con 0x7f92740d3900 2024-04-25T08:38:55.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.577+0000 7f927bef5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927415dff0 con 0x7f92740ced60 2024-04-25T08:38:55.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.577+0000 7f927bef5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92740b6990 con 0x7f927415dc10 2024-04-25T08:38:55.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.577+0000 7f9279490640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f92740ced60 0x7f92740d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.577+0000 7f9279490640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f92740ced60 0x7f92740d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50874/0 (socket says 172.21.15.38:50874) 2024-04-25T08:38:55.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.577+0000 7f9279490640 1 -- 172.21.15.38:0/1663598626 learned_addr learned my addr 172.21.15.38:0/1663598626 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:55.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.578+0000 7f9279c91640 1 --2- 172.21.15.38:0/1663598626 >> v2:172.21.15.107:3300/0 conn(0x7f927415dc10 0x7f92740ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.578+0000 7f9279490640 1 -- 172.21.15.38:0/1663598626 >> v2:172.21.15.107:3300/0 conn(0x7f927415dc10 msgr2=0x7f92740ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:55.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.578+0000 7f9279490640 1 --2- 172.21.15.38:0/1663598626 >> v2:172.21.15.107:3300/0 conn(0x7f927415dc10 0x7f92740ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.578+0000 7f9279490640 1 -- 172.21.15.38:0/1663598626 >> v1:172.21.15.107:6789/0 conn(0x7f92740d3900 legacy=0x7f92740b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.578+0000 7f9279490640 1 -- 172.21.15.38:0/1663598626 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92740d4150 con 0x7f92740ced60 2024-04-25T08:38:55.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.578+0000 7f9279490640 1 --2- 172.21.15.38:0/1663598626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92740ced60 0x7f92740d0850 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f92740b5d50 tx=0x7f926402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a683f10fa72ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:55.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.579+0000 7f9278c8f640 1 -- 172.21.15.38:0/1663598626 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264037070 con 0x7f92740ced60 2024-04-25T08:38:55.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.579+0000 7f9278c8f640 1 -- 172.21.15.38:0/1663598626 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9264032440 con 0x7f92740ced60 2024-04-25T08:38:55.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.579+0000 7f9278c8f640 1 -- 172.21.15.38:0/1663598626 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264032810 con 0x7f92740ced60 2024-04-25T08:38:55.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.579+0000 7f927bef5640 1 -- 172.21.15.38:0/1663598626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92740ced60 msgr2=0x7f92740d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.579+0000 7f927bef5640 1 --2- 172.21.15.38:0/1663598626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92740ced60 0x7f92740d0850 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f92740b5d50 tx=0x7f926402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.579+0000 7f927bef5640 1 -- 172.21.15.38:0/1663598626 shutdown_connections 2024-04-25T08:38:55.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.579+0000 7f927bef5640 1 --2- 172.21.15.38:0/1663598626 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92740ced60 0x7f92740d0850 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.579+0000 7f927bef5640 1 --2- 172.21.15.38:0/1663598626 >> v2:172.21.15.107:3300/0 conn(0x7f927415dc10 0x7f92740ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.579+0000 7f927bef5640 1 -- 172.21.15.38:0/1663598626 >> 172.21.15.38:0/1663598626 conn(0x7f9274066d10 msgr2=0x7f9274067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:55.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.579+0000 7f927bef5640 1 -- 172.21.15.38:0/1663598626 shutdown_connections 2024-04-25T08:38:55.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.579+0000 7f927bef5640 1 -- 172.21.15.38:0/1663598626 wait complete. 2024-04-25T08:38:55.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.580+0000 7f927bef5640 1 Processor -- start 2024-04-25T08:38:55.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.581+0000 7f927bef5640 1 -- start start 2024-04-25T08:38:55.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.581+0000 7f927bef5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92740ced60 0x7f9274159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.581+0000 7f927bef5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92740d3900 0x7f92740b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.581+0000 7f927bef5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f927415dc10 0x7f92740b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.581+0000 7f927bef5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92740b6bc0 con 0x7f92740ced60 2024-04-25T08:38:55.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.581+0000 7f927bef5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92740d4710 con 0x7f927415dc10 2024-04-25T08:38:55.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.581+0000 7f927bef5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92740d0d90 con 0x7f92740d3900 2024-04-25T08:38:55.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.581+0000 7f9279c91640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92740ced60 0x7f9274159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.581+0000 7f9279c91640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92740ced60 0x7f9274159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50888/0 (socket says 172.21.15.38:50888) 2024-04-25T08:38:55.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.581+0000 7f9279c91640 1 -- 172.21.15.38:0/3296663603 learned_addr learned my addr 172.21.15.38:0/3296663603 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:55.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.582+0000 7f927a492640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f927415dc10 0x7f92740b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.582+0000 7f9279c91640 1 -- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92740d3900 msgr2=0x7f92740b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.582+0000 7f9279c91640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92740d3900 0x7f92740b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.582+0000 7f9279c91640 1 -- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f927415dc10 msgr2=0x7f92740b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.582+0000 7f9279c91640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f927415dc10 0x7f92740b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.582+0000 7f9279c91640 1 -- 172.21.15.38:0/3296663603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9274158e00 con 0x7f92740ced60 2024-04-25T08:38:55.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.582+0000 7f9279c91640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92740ced60 0x7f9274159b80 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f927000d600 tx=0x7f927000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:55.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.582+0000 7f926a7fc640 1 -- 172.21.15.38:0/3296663603 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92700125f0 con 0x7f92740ced60 2024-04-25T08:38:55.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.582+0000 7f926a7fc640 1 -- 172.21.15.38:0/3296663603 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9270019070 con 0x7f92740ced60 2024-04-25T08:38:55.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.582+0000 7f927bef5640 1 -- 172.21.15.38:0/3296663603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9274146c50 con 0x7f92740ced60 2024-04-25T08:38:55.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.582+0000 7f926a7fc640 1 -- 172.21.15.38:0/3296663603 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92700129f0 con 0x7f92740ced60 2024-04-25T08:38:55.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.582+0000 7f927bef5640 1 -- 172.21.15.38:0/3296663603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9274159380 con 0x7f92740ced60 2024-04-25T08:38:55.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.583+0000 7f927bef5640 1 -- 172.21.15.38:0/3296663603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9274068e80 con 0x7f92740ced60 2024-04-25T08:38:55.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.584+0000 7f926a7fc640 1 -- 172.21.15.38:0/3296663603 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9270012b90 con 0x7f92740ced60 2024-04-25T08:38:55.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.584+0000 7f926a7fc640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f925806b500 0x7f925806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.584+0000 7f926a7fc640 1 -- 172.21.15.38:0/3296663603 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f927008a780 con 0x7f92740ced60 2024-04-25T08:38:55.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.584+0000 7f9279490640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f925806b500 0x7f925806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.585+0000 7f9279490640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f925806b500 0x7f925806d9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9264034000 tx=0x7f926402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:55.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.588+0000 7f926a7fc640 1 -- 172.21.15.38:0/3296663603 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9270059470 con 0x7f92740ced60 2024-04-25T08:38:55.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.768+0000 7f927bef5640 1 -- 172.21.15.38:0/3296663603 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f927415e310 con 0x7f92740ced60 2024-04-25T08:38:55.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.769+0000 7f926a7fc640 1 -- 172.21.15.38:0/3296663603 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f9270058d70 con 0x7f92740ced60 2024-04-25T08:38:55.763 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:55.763 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":96,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:38:27.725413+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:24.377232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:25.069513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.771+0000 7f927bef5640 1 -- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f925806b500 msgr2=0x7f925806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.771+0000 7f927bef5640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f925806b500 0x7f925806d9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f9264034000 tx=0x7f926402c000 comp rx=0 tx=0).stop 2024-04-25T08:38:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.771+0000 7f927bef5640 1 -- 172.21.15.38:0/3296663603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92740ced60 msgr2=0x7f9274159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.771+0000 7f927bef5640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92740ced60 0x7f9274159b80 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f927000d600 tx=0x7f927000dac0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.771+0000 7f927bef5640 1 -- 172.21.15.38:0/3296663603 shutdown_connections 2024-04-25T08:38:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.771+0000 7f927bef5640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f927415dc10 0x7f92740b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.771+0000 7f927bef5640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f925806b500 0x7f925806d9c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.771+0000 7f927bef5640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92740d3900 0x7f92740b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.771+0000 7f927bef5640 1 --2- 172.21.15.38:0/3296663603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92740ced60 0x7f9274159b80 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.771+0000 7f927bef5640 1 -- 172.21.15.38:0/3296663603 >> 172.21.15.38:0/3296663603 conn(0x7f9274066d10 msgr2=0x7f92740d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.771+0000 7f927bef5640 1 -- 172.21.15.38:0/3296663603 shutdown_connections 2024-04-25T08:38:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.771+0000 7f927bef5640 1 -- 172.21.15.38:0/3296663603 wait complete. 2024-04-25T08:38:55.776 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:38:55.776 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:55.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.886+0000 7ffa15007640 1 Processor -- start 2024-04-25T08:38:55.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.886+0000 7ffa15007640 1 -- start start 2024-04-25T08:38:55.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa15007640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffa10144760 0x7ffa1015faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa15007640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ffa100cea70 0x7ffa100b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa15007640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa100b50c0 con 0x7ffa10160030 2024-04-25T08:38:55.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa15007640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1015f380 con 0x7ffa10144760 2024-04-25T08:38:55.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa15007640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1015f660 con 0x7ffa100cea70 2024-04-25T08:38:55.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa0ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffa10144760 0x7ffa1015faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa0ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffa10144760 0x7ffa1015faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50890/0 (socket says 172.21.15.38:50890) 2024-04-25T08:38:55.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa0ed9d640 1 -- 172.21.15.38:0/529980395 learned_addr learned my addr 172.21.15.38:0/529980395 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:55.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa0f59e640 1 --2- 172.21.15.38:0/529980395 >> v2:172.21.15.107:3300/0 conn(0x7ffa100cea70 0x7ffa100b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa0ed9d640 1 -- 172.21.15.38:0/529980395 >> v2:172.21.15.107:3300/0 conn(0x7ffa100cea70 msgr2=0x7ffa100b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:55.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa0ed9d640 1 --2- 172.21.15.38:0/529980395 >> v2:172.21.15.107:3300/0 conn(0x7ffa100cea70 0x7ffa100b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa0ed9d640 1 -- 172.21.15.38:0/529980395 >> v1:172.21.15.107:6789/0 conn(0x7ffa10160030 legacy=0x7ffa100ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.887+0000 7ffa0ed9d640 1 -- 172.21.15.38:0/529980395 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa100b53a0 con 0x7ffa10144760 2024-04-25T08:38:55.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.888+0000 7ffa0ed9d640 1 --2- 172.21.15.38:0/529980395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10144760 0x7ffa1015faf0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ffa10073310 tx=0x7ffa0802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a013efe27ba23c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:55.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.888+0000 7ffa0dd9b640 1 -- 172.21.15.38:0/529980395 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08037070 con 0x7ffa10144760 2024-04-25T08:38:55.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.888+0000 7ffa0dd9b640 1 -- 172.21.15.38:0/529980395 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa08032440 con 0x7ffa10144760 2024-04-25T08:38:55.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.888+0000 7ffa0dd9b640 1 -- 172.21.15.38:0/529980395 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa08032810 con 0x7ffa10144760 2024-04-25T08:38:55.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.889+0000 7ffa15007640 1 -- 172.21.15.38:0/529980395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10144760 msgr2=0x7ffa1015faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.889+0000 7ffa15007640 1 --2- 172.21.15.38:0/529980395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10144760 0x7ffa1015faf0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ffa10073310 tx=0x7ffa0802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.889+0000 7ffa15007640 1 -- 172.21.15.38:0/529980395 shutdown_connections 2024-04-25T08:38:55.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.889+0000 7ffa15007640 1 --2- 172.21.15.38:0/529980395 >> v2:172.21.15.107:3300/0 conn(0x7ffa100cea70 0x7ffa100b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.889+0000 7ffa15007640 1 --2- 172.21.15.38:0/529980395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10144760 0x7ffa1015faf0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.889+0000 7ffa15007640 1 -- 172.21.15.38:0/529980395 >> 172.21.15.38:0/529980395 conn(0x7ffa1006b210 msgr2=0x7ffa1006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:55.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.889+0000 7ffa15007640 1 -- 172.21.15.38:0/529980395 shutdown_connections 2024-04-25T08:38:55.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.889+0000 7ffa15007640 1 -- 172.21.15.38:0/529980395 wait complete. 2024-04-25T08:38:55.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.890+0000 7ffa15007640 1 Processor -- start 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.890+0000 7ffa15007640 1 -- start start 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa15007640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa100cea70 0x7ffa10143770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa15007640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10160030 0x7ffa10151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa15007640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa101408a0 0x7ffa101f1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa15007640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa10145400 con 0x7ffa10160030 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa15007640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa10145280 con 0x7ffa100cea70 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa15007640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa100b4b50 con 0x7ffa101408a0 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa0e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10160030 0x7ffa10151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa0e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10160030 0x7ffa10151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50896/0 (socket says 172.21.15.38:50896) 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa0e59c640 1 -- 172.21.15.38:0/2932218541 learned_addr learned my addr 172.21.15.38:0/2932218541 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa0ed9d640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa100cea70 0x7ffa10143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa0f59e640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa101408a0 0x7ffa101f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa0e59c640 1 -- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa101408a0 msgr2=0x7ffa101f1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa0e59c640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa101408a0 0x7ffa101f1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa0e59c640 1 -- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa100cea70 msgr2=0x7ffa10143770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa0e59c640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa100cea70 0x7ffa10143770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.891+0000 7ffa0e59c640 1 -- 172.21.15.38:0/2932218541 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa10143de0 con 0x7ffa10160030 2024-04-25T08:38:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.892+0000 7ffa0e59c640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10160030 0x7ffa10151090 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800c430 tx=0x7ff9f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.892+0000 7ffa077fe640 1 -- 172.21.15.38:0/2932218541 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8017070 con 0x7ffa10160030 2024-04-25T08:38:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.892+0000 7ffa077fe640 1 -- 172.21.15.38:0/2932218541 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f80052d0 con 0x7ffa10160030 2024-04-25T08:38:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.892+0000 7ffa15007640 1 -- 172.21.15.38:0/2932218541 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1013b160 con 0x7ffa10160030 2024-04-25T08:38:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.892+0000 7ffa077fe640 1 -- 172.21.15.38:0/2932218541 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8003830 con 0x7ffa10160030 2024-04-25T08:38:55.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.892+0000 7ffa15007640 1 -- 172.21.15.38:0/2932218541 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1013b720 con 0x7ffa10160030 2024-04-25T08:38:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.893+0000 7ffa15007640 1 -- 172.21.15.38:0/2932218541 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1013b920 con 0x7ffa10160030 2024-04-25T08:38:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.893+0000 7ffa077fe640 1 -- 172.21.15.38:0/2932218541 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9f801a070 con 0x7ffa10160030 2024-04-25T08:38:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.894+0000 7ffa077fe640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff9f006b4d0 0x7ff9f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.894+0000 7ffa077fe640 1 -- 172.21.15.38:0/2932218541 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff9f80895d0 con 0x7ffa10160030 2024-04-25T08:38:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.894+0000 7ffa0ed9d640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff9f006b4d0 0x7ff9f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:55.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.895+0000 7ffa0ed9d640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff9f006b4d0 0x7ff9f006d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ffa08034000 tx=0x7ffa0802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:55.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:55.898+0000 7ffa077fe640 1 -- 172.21.15.38:0/2932218541 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa1013b920 con 0x7ffa10160030 2024-04-25T08:38:56.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.119+0000 7ffa15007640 1 -- 172.21.15.38:0/2932218541 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa10068e80 con 0x7ffa10160030 2024-04-25T08:38:56.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.120+0000 7ffa077fe640 1 -- 172.21.15.38:0/2932218541 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4943 (secure 0 0 0) 0x7ff9f8057d00 con 0x7ffa10160030 2024-04-25T08:38:56.114 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:56.114 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3680979257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3680979257},{"type":"v1","addr":"172.21.15.107:6836","nonce":3680979257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/697909489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":697909489},{"type":"v1","addr":"172.21.15.107:6837","nonce":697909489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3083871141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3083871141},{"type":"v1","addr":"172.21.15.38:6834","nonce":3083871141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:26.095391+0000","modified":"2024-04-25T08:38:28.097807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/2922734840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":2922734840},{"type":"v1","addr":"172.21.15.38:6835","nonce":2922734840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:38:56.114 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 57 2024-04-25T08:38:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.122+0000 7ffa15007640 1 -- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff9f006b4d0 msgr2=0x7ff9f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.122+0000 7ffa15007640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff9f006b4d0 0x7ff9f006d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ffa08034000 tx=0x7ffa0802e000 comp rx=0 tx=0).stop 2024-04-25T08:38:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.122+0000 7ffa15007640 1 -- 172.21.15.38:0/2932218541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10160030 msgr2=0x7ffa10151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.122+0000 7ffa15007640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10160030 0x7ffa10151090 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800c430 tx=0x7ff9f800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.122+0000 7ffa15007640 1 -- 172.21.15.38:0/2932218541 shutdown_connections 2024-04-25T08:38:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.122+0000 7ffa15007640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff9f006b4d0 0x7ff9f006d990 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.122+0000 7ffa15007640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffa101408a0 0x7ffa101f1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.122+0000 7ffa15007640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10160030 0x7ffa10151090 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.122+0000 7ffa15007640 1 --2- 172.21.15.38:0/2932218541 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa100cea70 0x7ffa10143770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.122+0000 7ffa15007640 1 -- 172.21.15.38:0/2932218541 >> 172.21.15.38:0/2932218541 conn(0x7ffa1006b210 msgr2=0x7ffa100b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.122+0000 7ffa15007640 1 -- 172.21.15.38:0/2932218541 shutdown_connections 2024-04-25T08:38:56.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.122+0000 7ffa15007640 1 -- 172.21.15.38:0/2932218541 wait complete. 2024-04-25T08:38:56.127 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:38:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.235+0000 7fa92a15c640 1 Processor -- start 2024-04-25T08:38:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.235+0000 7fa92a15c640 1 -- start start 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa92a15c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9240d3900 0x7fa9240d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa92a15c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa9240d0d90 0x7fa9240b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa92a15c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9240d3e70 con 0x7fa92415dc10 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa92a15c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa92415dff0 con 0x7fa9240d0d90 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa92a15c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240b6990 con 0x7fa9240d3900 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa923fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa9240d0d90 0x7fa9240b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa9237fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa92415dc10 0x7fa9240ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50376/0 (socket says 172.21.15.38:50376) 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa9237fe640 1 -- 172.21.15.38:0/3703452015 learned_addr learned my addr 172.21.15.38:0/3703452015 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa923fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa9240d0d90 0x7fa9240b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50910/0 (socket says 172.21.15.38:50910) 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa922ffd640 1 --2- 172.21.15.38:0/3703452015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9240d3900 0x7fa9240d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa923fff640 1 -- 172.21.15.38:0/3703452015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9240d3900 msgr2=0x7fa9240d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa923fff640 1 --2- 172.21.15.38:0/3703452015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9240d3900 0x7fa9240d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa923fff640 1 -- 172.21.15.38:0/3703452015 >> v1:172.21.15.38:6789/0 conn(0x7fa92415dc10 legacy=0x7fa9240ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.236+0000 7fa923fff640 1 -- 172.21.15.38:0/3703452015 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9240d4150 con 0x7fa9240d0d90 2024-04-25T08:38:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.237+0000 7fa923fff640 1 --2- 172.21.15.38:0/3703452015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9240d0d90 0x7fa9240b2370 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009d90 tx=0x7fa91c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b10d7fcb16bcda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.237+0000 7fa9227fc640 1 -- 172.21.15.38:0/3703452015 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c031600 con 0x7fa9240d0d90 2024-04-25T08:38:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.237+0000 7fa9227fc640 1 -- 172.21.15.38:0/3703452015 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91c0317a0 con 0x7fa9240d0d90 2024-04-25T08:38:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.237+0000 7fa9227fc640 1 -- 172.21.15.38:0/3703452015 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91c031b70 con 0x7fa9240d0d90 2024-04-25T08:38:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.237+0000 7fa92a15c640 1 -- 172.21.15.38:0/3703452015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9240d0d90 msgr2=0x7fa9240b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.237+0000 7fa92a15c640 1 --2- 172.21.15.38:0/3703452015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9240d0d90 0x7fa9240b2370 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fa91c009d90 tx=0x7fa91c02a5d0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.237+0000 7fa92a15c640 1 -- 172.21.15.38:0/3703452015 shutdown_connections 2024-04-25T08:38:56.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.237+0000 7fa92a15c640 1 --2- 172.21.15.38:0/3703452015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9240d0d90 0x7fa9240b2370 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.237+0000 7fa92a15c640 1 --2- 172.21.15.38:0/3703452015 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9240d3900 0x7fa9240d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.238+0000 7fa92a15c640 1 -- 172.21.15.38:0/3703452015 >> 172.21.15.38:0/3703452015 conn(0x7fa924066d10 msgr2=0x7fa924067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.238+0000 7fa92a15c640 1 -- 172.21.15.38:0/3703452015 shutdown_connections 2024-04-25T08:38:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.238+0000 7fa92a15c640 1 -- 172.21.15.38:0/3703452015 wait complete. 2024-04-25T08:38:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.239+0000 7fa92a15c640 1 Processor -- start 2024-04-25T08:38:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.239+0000 7fa92a15c640 1 -- start start 2024-04-25T08:38:56.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa92a15c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9240d0d90 0x7fa9240cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa92a15c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9240d3900 0x7fa9240c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa92a15c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92415dc10 0x7fa924155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa92a15c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240b6bc0 con 0x7fa9240d3900 2024-04-25T08:38:56.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa92a15c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240d4710 con 0x7fa92415dc10 2024-04-25T08:38:56.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa92a15c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9240cef30 con 0x7fa9240d0d90 2024-04-25T08:38:56.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa922ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9240d3900 0x7fa9240c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa922ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9240d3900 0x7fa9240c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50914/0 (socket says 172.21.15.38:50914) 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa922ffd640 1 -- 172.21.15.38:0/1538159831 learned_addr learned my addr 172.21.15.38:0/1538159831 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa923fff640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92415dc10 0x7fa924155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa9237fe640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9240d0d90 0x7fa9240cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa922ffd640 1 -- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9240d0d90 msgr2=0x7fa9240cb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa922ffd640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9240d0d90 0x7fa9240cb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa922ffd640 1 -- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92415dc10 msgr2=0x7fa924155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa922ffd640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92415dc10 0x7fa924155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa922ffd640 1 -- 172.21.15.38:0/1538159831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa924153de0 con 0x7fa9240d3900 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa9237fe640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9240d0d90 0x7fa9240cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa922ffd640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9240d3900 0x7fa9240c8a00 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa91400e9b0 tx=0x7fa91400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa90bfff640 1 -- 172.21.15.38:0/1538159831 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa914017070 con 0x7fa9240d3900 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa90bfff640 1 -- 172.21.15.38:0/1538159831 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa91400c9a0 con 0x7fa9240d3900 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa92a15c640 1 -- 172.21.15.38:0/1538159831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9240cbe60 con 0x7fa9240d3900 2024-04-25T08:38:56.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa90bfff640 1 -- 172.21.15.38:0/1538159831 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa91400cd90 con 0x7fa9240d3900 2024-04-25T08:38:56.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.240+0000 7fa92a15c640 1 -- 172.21.15.38:0/1538159831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa924154520 con 0x7fa9240d3900 2024-04-25T08:38:56.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.241+0000 7fa92a15c640 1 -- 172.21.15.38:0/1538159831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa924068e80 con 0x7fa9240d3900 2024-04-25T08:38:56.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.242+0000 7fa90bfff640 1 -- 172.21.15.38:0/1538159831 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa91401a070 con 0x7fa9240d3900 2024-04-25T08:38:56.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.242+0000 7fa90bfff640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa91006b4d0 0x7fa91006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.243+0000 7fa90bfff640 1 -- 172.21.15.38:0/1538159831 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa91408a3d0 con 0x7fa9240d3900 2024-04-25T08:38:56.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.243+0000 7fa9237fe640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa91006b4d0 0x7fa91006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.243+0000 7fa9237fe640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa91006b4d0 0x7fa91006d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa918000f60 tx=0x7fa918009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:56.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.247+0000 7fa90bfff640 1 -- 172.21.15.38:0/1538159831 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9140590c0 con 0x7fa9240d3900 2024-04-25T08:38:56.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.466+0000 7fa92a15c640 1 -- 172.21.15.38:0/1538159831 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9240cc030 con 0x7fa9240d3900 2024-04-25T08:38:56.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.467+0000 7fa90bfff640 1 -- 172.21.15.38:0/1538159831 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa9140589c0 con 0x7fa9240d3900 2024-04-25T08:38:56.460 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:56.460 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3680979257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3680979257},{"type":"v1","addr":"172.21.15.107:6836","nonce":3680979257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/697909489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":697909489},{"type":"v1","addr":"172.21.15.107:6837","nonce":697909489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3083871141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3083871141},{"type":"v1","addr":"172.21.15.38:6834","nonce":3083871141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:38:26.095391+0000","modified":"2024-04-25T08:38:28.097807+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7854":{"gid":7854,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/2922734840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":2922734840},{"type":"v1","addr":"172.21.15.38:6835","nonce":2922734840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:38:56.460 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 57 2024-04-25T08:38:56.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.468+0000 7fa92a15c640 1 -- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa91006b4d0 msgr2=0x7fa91006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.468+0000 7fa92a15c640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa91006b4d0 0x7fa91006d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa918000f60 tx=0x7fa918009040 comp rx=0 tx=0).stop 2024-04-25T08:38:56.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.468+0000 7fa92a15c640 1 -- 172.21.15.38:0/1538159831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9240d3900 msgr2=0x7fa9240c8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.468+0000 7fa92a15c640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9240d3900 0x7fa9240c8a00 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fa91400e9b0 tx=0x7fa91400ee70 comp rx=0 tx=0).stop 2024-04-25T08:38:56.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.468+0000 7fa92a15c640 1 -- 172.21.15.38:0/1538159831 shutdown_connections 2024-04-25T08:38:56.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.468+0000 7fa92a15c640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92415dc10 0x7fa924155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.468+0000 7fa92a15c640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9240d3900 0x7fa9240c8a00 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.468+0000 7fa92a15c640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa91006b4d0 0x7fa91006d990 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.468+0000 7fa92a15c640 1 --2- 172.21.15.38:0/1538159831 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9240d0d90 0x7fa9240cb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.468+0000 7fa92a15c640 1 -- 172.21.15.38:0/1538159831 >> 172.21.15.38:0/1538159831 conn(0x7fa924066d10 msgr2=0x7fa9240b6fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:56.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.469+0000 7fa92a15c640 1 -- 172.21.15.38:0/1538159831 shutdown_connections 2024-04-25T08:38:56.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.469+0000 7fa92a15c640 1 -- 172.21.15.38:0/1538159831 wait complete. 2024-04-25T08:38:56.473 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:38:56.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.581+0000 7f5d5b3e0640 1 Processor -- start 2024-04-25T08:38:56.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.581+0000 7f5d5b3e0640 1 -- start start 2024-04-25T08:38:56.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5b3e0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5d5406c110 0x7f5d5406c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5b3e0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5d540ce2e0 0x7f5d540d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5b3e0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d5406ca30 con 0x7f5d5415df50 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5b3e0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540cf2d0 con 0x7f5d5406c110 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5b3e0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540ceff0 con 0x7f5d540ce2e0 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5917c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5d5406c110 0x7f5d5406c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5917c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5d5406c110 0x7f5d5406c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50926/0 (socket says 172.21.15.38:50926) 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5917c640 1 -- 172.21.15.38:0/3597055180 learned_addr learned my addr 172.21.15.38:0/3597055180 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5997d640 1 --2- 172.21.15.38:0/3597055180 >> v2:172.21.15.107:3300/0 conn(0x7f5d540ce2e0 0x7f5d540d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5917c640 1 -- 172.21.15.38:0/3597055180 >> v2:172.21.15.107:3300/0 conn(0x7f5d540ce2e0 msgr2=0x7f5d540d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5917c640 1 --2- 172.21.15.38:0/3597055180 >> v2:172.21.15.107:3300/0 conn(0x7f5d540ce2e0 0x7f5d540d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5917c640 1 -- 172.21.15.38:0/3597055180 >> v1:172.21.15.107:6789/0 conn(0x7f5d5415df50 legacy=0x7f5d540ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.582+0000 7f5d5917c640 1 -- 172.21.15.38:0/3597055180 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540d1f70 con 0x7f5d5406c110 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.583+0000 7f5d5917c640 1 --2- 172.21.15.38:0/3597055180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d5406c110 0x7f5d5406c4f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5d540b5400 tx=0x7f5d5002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c429f1ba2ad5ead server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.583+0000 7f5d4bfff640 1 -- 172.21.15.38:0/3597055180 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50037070 con 0x7f5d5406c110 2024-04-25T08:38:56.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.583+0000 7f5d4bfff640 1 -- 172.21.15.38:0/3597055180 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d50032440 con 0x7f5d5406c110 2024-04-25T08:38:56.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.583+0000 7f5d4bfff640 1 -- 172.21.15.38:0/3597055180 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50032810 con 0x7f5d5406c110 2024-04-25T08:38:56.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.583+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/3597055180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d5406c110 msgr2=0x7f5d5406c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.583+0000 7f5d5b3e0640 1 --2- 172.21.15.38:0/3597055180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d5406c110 0x7f5d5406c4f0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f5d540b5400 tx=0x7f5d5002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.584+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/3597055180 shutdown_connections 2024-04-25T08:38:56.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.584+0000 7f5d5b3e0640 1 --2- 172.21.15.38:0/3597055180 >> v2:172.21.15.107:3300/0 conn(0x7f5d540ce2e0 0x7f5d540d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.584+0000 7f5d5b3e0640 1 --2- 172.21.15.38:0/3597055180 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d5406c110 0x7f5d5406c4f0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.584+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/3597055180 >> 172.21.15.38:0/3597055180 conn(0x7f5d540d3fb0 msgr2=0x7f5d540d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:56.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.584+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/3597055180 shutdown_connections 2024-04-25T08:38:56.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.584+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/3597055180 wait complete. 2024-04-25T08:38:56.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.585+0000 7f5d5b3e0640 1 Processor -- start 2024-04-25T08:38:56.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.585+0000 7f5d5b3e0640 1 -- start start 2024-04-25T08:38:56.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.585+0000 7f5d5b3e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d5406c110 0x7f5d540c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.585+0000 7f5d5b3e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d540ce2e0 0x7f5d540c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.585+0000 7f5d5b3e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d5415df50 0x7f5d5414b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5b3e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540d2f90 con 0x7f5d540ce2e0 2024-04-25T08:38:56.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5b3e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540d2530 con 0x7f5d5406c110 2024-04-25T08:38:56.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5b3e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540d3110 con 0x7f5d5415df50 2024-04-25T08:38:56.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5897b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d540ce2e0 0x7f5d540c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5897b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d540ce2e0 0x7f5d540c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50934/0 (socket says 172.21.15.38:50934) 2024-04-25T08:38:56.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5897b640 1 -- 172.21.15.38:0/4135097233 learned_addr learned my addr 172.21.15.38:0/4135097233 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:56.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5917c640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d5406c110 0x7f5d540c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5997d640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d5415df50 0x7f5d5414b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5897b640 1 -- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d5415df50 msgr2=0x7f5d5414b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5897b640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d5415df50 0x7f5d5414b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5897b640 1 -- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d5406c110 msgr2=0x7f5d540c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5897b640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d5406c110 0x7f5d540c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5897b640 1 -- 172.21.15.38:0/4135097233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540c8e10 con 0x7f5d540ce2e0 2024-04-25T08:38:56.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5917c640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d5406c110 0x7f5d540c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:56.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5997d640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d5415df50 0x7f5d5414b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:56.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5897b640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d540ce2e0 0x7f5d540c8790 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400c430 tx=0x7f5d4400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:56.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d49ffb640 1 -- 172.21.15.38:0/4135097233 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44011790 con 0x7f5d540ce2e0 2024-04-25T08:38:56.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d49ffb640 1 -- 172.21.15.38:0/4135097233 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d44017070 con 0x7f5d540ce2e0 2024-04-25T08:38:56.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d49ffb640 1 -- 172.21.15.38:0/4135097233 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44011bb0 con 0x7f5d540ce2e0 2024-04-25T08:38:56.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/4135097233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d540c90f0 con 0x7f5d540ce2e0 2024-04-25T08:38:56.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.586+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/4135097233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d540c96b0 con 0x7f5d540ce2e0 2024-04-25T08:38:56.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.588+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/4135097233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5406cd90 con 0x7f5d540ce2e0 2024-04-25T08:38:56.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.588+0000 7f5d49ffb640 1 -- 172.21.15.38:0/4135097233 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d4401c460 con 0x7f5d540ce2e0 2024-04-25T08:38:56.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.588+0000 7f5d49ffb640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5d3806b4d0 0x7f5d3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.588+0000 7f5d49ffb640 1 -- 172.21.15.38:0/4135097233 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5d4408a1a0 con 0x7f5d540ce2e0 2024-04-25T08:38:56.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.589+0000 7f5d5917c640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5d3806b4d0 0x7f5d3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.589+0000 7f5d5917c640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5d3806b4d0 0x7f5d3806d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5d50034000 tx=0x7f5d5002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:56.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.593+0000 7f5d49ffb640 1 -- 172.21.15.38:0/4135097233 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d44058f20 con 0x7f5d540ce2e0 2024-04-25T08:38:56.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.773+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/4135097233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d540d15f0 con 0x7f5d540ce2e0 2024-04-25T08:38:56.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.774+0000 7f5d49ffb640 1 -- 172.21.15.38:0/4135097233 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16873 (secure 0 0 0) 0x7f5d44058820 con 0x7f5d540ce2e0 2024-04-25T08:38:56.768 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:38:56.768 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":96,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:38:27.725413+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:38:24.377232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:38:25.069513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:38:56.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.776+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5d3806b4d0 msgr2=0x7f5d3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.776+0000 7f5d5b3e0640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5d3806b4d0 0x7f5d3806d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5d50034000 tx=0x7f5d5002e000 comp rx=0 tx=0).stop 2024-04-25T08:38:56.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.776+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/4135097233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d540ce2e0 msgr2=0x7f5d540c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.776+0000 7f5d5b3e0640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d540ce2e0 0x7f5d540c8790 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400c430 tx=0x7f5d4400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.776+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/4135097233 shutdown_connections 2024-04-25T08:38:56.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.776+0000 7f5d5b3e0640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5d3806b4d0 0x7f5d3806d990 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.776+0000 7f5d5b3e0640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5d5415df50 0x7f5d5414b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.776+0000 7f5d5b3e0640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d540ce2e0 0x7f5d540c8790 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.776+0000 7f5d5b3e0640 1 --2- 172.21.15.38:0/4135097233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d5406c110 0x7f5d540c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.776+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/4135097233 >> 172.21.15.38:0/4135097233 conn(0x7f5d540d3fb0 msgr2=0x7f5d5415d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:56.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.776+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/4135097233 shutdown_connections 2024-04-25T08:38:56.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.776+0000 7f5d5b3e0640 1 -- 172.21.15.38:0/4135097233 wait complete. 2024-04-25T08:38:56.782 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:38:56.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.890+0000 7f9917f29640 1 Processor -- start 2024-04-25T08:38:56.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.890+0000 7f9917f29640 1 -- start start 2024-04-25T08:38:56.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.891+0000 7f9917f29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9910160ef0 0x7f99100d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.891+0000 7f9917f29640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f99100b9180 0x7f99100d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.891+0000 7f9917f29640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f99100b2ab0 0x7f99100d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.891+0000 7f9917f29640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9910161cf0 con 0x7f99100b2ab0 2024-04-25T08:38:56.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.891+0000 7f9917f29640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99101612d0 con 0x7f99100b9180 2024-04-25T08:38:56.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.891+0000 7f9917f29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99100b9610 con 0x7f9910160ef0 2024-04-25T08:38:56.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.891+0000 7f99164c6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f99100b2ab0 0x7f99100d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.891+0000 7f99164c6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f99100b2ab0 0x7f99100d37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50946/0 (socket says 172.21.15.38:50946) 2024-04-25T08:38:56.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.891+0000 7f99164c6640 1 -- 172.21.15.38:0/1354638396 learned_addr learned my addr 172.21.15.38:0/1354638396 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:56.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.891+0000 7f99154c4640 1 --2- 172.21.15.38:0/1354638396 >> v2:172.21.15.107:3300/0 conn(0x7f99100b9180 0x7f99100d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.891+0000 7f9915cc5640 1 --2- 172.21.15.38:0/1354638396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9910160ef0 0x7f99100d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.891+0000 7f99164c6640 1 -- 172.21.15.38:0/1354638396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9910160ef0 msgr2=0x7f99100d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:56.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.892+0000 7f99164c6640 1 --2- 172.21.15.38:0/1354638396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9910160ef0 0x7f99100d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.892+0000 7f99164c6640 1 -- 172.21.15.38:0/1354638396 >> v2:172.21.15.107:3300/0 conn(0x7f99100b9180 msgr2=0x7f99100d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:56.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.892+0000 7f99164c6640 1 --2- 172.21.15.38:0/1354638396 >> v2:172.21.15.107:3300/0 conn(0x7f99100b9180 0x7f99100d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.892+0000 7f99164c6640 1 -- 172.21.15.38:0/1354638396 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99101616a0 con 0x7f99100b2ab0 2024-04-25T08:38:56.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.892+0000 7f99154c4640 1 --2- 172.21.15.38:0/1354638396 >> v2:172.21.15.107:3300/0 conn(0x7f99100b9180 0x7f99100d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.892+0000 7f99164c6640 1 --2- 172.21.15.38:0/1354638396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99100b2ab0 0x7f99100d37f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f99100d48c0 tx=0x7f990002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988b7c82f90b6ee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.892+0000 7f9914cc3640 1 -- 172.21.15.38:0/1354638396 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900036070 con 0x7f99100b2ab0 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.892+0000 7f9914cc3640 1 -- 172.21.15.38:0/1354638396 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9900031600 con 0x7f99100b2ab0 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.892+0000 7f9914cc3640 1 -- 172.21.15.38:0/1354638396 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99000319d0 con 0x7f99100b2ab0 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.893+0000 7f9917f29640 1 -- 172.21.15.38:0/1354638396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99100b2ab0 msgr2=0x7f99100d37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.893+0000 7f9917f29640 1 --2- 172.21.15.38:0/1354638396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99100b2ab0 0x7f99100d37f0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f99100d48c0 tx=0x7f990002a610 comp rx=0 tx=0).stop 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.893+0000 7f9917f29640 1 -- 172.21.15.38:0/1354638396 shutdown_connections 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.893+0000 7f9917f29640 1 --2- 172.21.15.38:0/1354638396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99100b2ab0 0x7f99100d37f0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.893+0000 7f9917f29640 1 --2- 172.21.15.38:0/1354638396 >> v2:172.21.15.107:3300/0 conn(0x7f99100b9180 0x7f99100d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.893+0000 7f9917f29640 1 --2- 172.21.15.38:0/1354638396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9910160ef0 0x7f99100d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.893+0000 7f9917f29640 1 -- 172.21.15.38:0/1354638396 >> 172.21.15.38:0/1354638396 conn(0x7f9910066d10 msgr2=0x7f9910067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.893+0000 7f9917f29640 1 -- 172.21.15.38:0/1354638396 shutdown_connections 2024-04-25T08:38:56.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.893+0000 7f9917f29640 1 -- 172.21.15.38:0/1354638396 wait complete. 2024-04-25T08:38:56.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.894+0000 7f9917f29640 1 Processor -- start 2024-04-25T08:38:56.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.894+0000 7f9917f29640 1 -- start start 2024-04-25T08:38:56.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f9917f29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99100b2ab0 0x7f991014ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f9917f29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9910160ef0 0x7f99100b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f9917f29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99100b79d0 0x7f9910155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f9917f29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9910162300 con 0x7f99100b79d0 2024-04-25T08:38:56.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f9917f29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9910162180 con 0x7f99100b2ab0 2024-04-25T08:38:56.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f9917f29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9910162480 con 0x7f9910160ef0 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f99164c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99100b79d0 0x7f9910155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f99164c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99100b79d0 0x7f9910155c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50962/0 (socket says 172.21.15.38:50962) 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f99164c6640 1 -- 172.21.15.38:0/762652717 learned_addr learned my addr 172.21.15.38:0/762652717 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f99154c4640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9910160ef0 0x7f99100b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f9915cc5640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99100b2ab0 0x7f991014ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f99164c6640 1 -- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9910160ef0 msgr2=0x7f99100b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f99164c6640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9910160ef0 0x7f99100b7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f99164c6640 1 -- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99100b2ab0 msgr2=0x7f991014ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f99164c6640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99100b2ab0 0x7f991014ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.895+0000 7f99164c6640 1 -- 172.21.15.38:0/762652717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99101569b0 con 0x7f99100b79d0 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.896+0000 7f99164c6640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99100b79d0 0x7f9910155c60 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f990002ac00 tx=0x7f9900004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.896+0000 7f990e7fc640 1 -- 172.21.15.38:0/762652717 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990003f020 con 0x7f99100b79d0 2024-04-25T08:38:56.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.896+0000 7f990e7fc640 1 -- 172.21.15.38:0/762652717 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9900037040 con 0x7f99100b79d0 2024-04-25T08:38:56.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.896+0000 7f9917f29640 1 -- 172.21.15.38:0/762652717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9910156c90 con 0x7f99100b79d0 2024-04-25T08:38:56.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.896+0000 7f990e7fc640 1 -- 172.21.15.38:0/762652717 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900036070 con 0x7f99100b79d0 2024-04-25T08:38:56.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.896+0000 7f9917f29640 1 -- 172.21.15.38:0/762652717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9910157250 con 0x7f99100b79d0 2024-04-25T08:38:56.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.897+0000 7f9917f29640 1 -- 172.21.15.38:0/762652717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991004ef90 con 0x7f99100b79d0 2024-04-25T08:38:56.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.897+0000 7f990e7fc640 1 -- 172.21.15.38:0/762652717 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9900042020 con 0x7f99100b79d0 2024-04-25T08:38:56.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.898+0000 7f990e7fc640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98fc06b500 0x7f98fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:56.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.898+0000 7f990e7fc640 1 -- 172.21.15.38:0/762652717 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99000aaca0 con 0x7f99100b79d0 2024-04-25T08:38:56.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.898+0000 7f9915cc5640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98fc06b500 0x7f98fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:56.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.898+0000 7f9915cc5640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98fc06b500 0x7f98fc06d9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f990800b7c0 tx=0x7f9908008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:56.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:56.902+0000 7f990e7fc640 1 -- 172.21.15.38:0/762652717 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9900049020 con 0x7f99100b79d0 2024-04-25T08:38:57.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.118+0000 7f9917f29640 1 -- 172.21.15.38:0/762652717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f99100d4360 con 0x7f99100b79d0 2024-04-25T08:38:57.112 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:38:57.119+0000 7ff7937f0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:38:57.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.124+0000 7f990e7fc640 1 -- 172.21.15.38:0/762652717 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7f99000792a0 con 0x7f99100b79d0 2024-04-25T08:38:57.118 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:38:57.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.126+0000 7f9917f29640 1 -- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98fc06b500 msgr2=0x7f98fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.126+0000 7f9917f29640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98fc06b500 0x7f98fc06d9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f990800b7c0 tx=0x7f9908008040 comp rx=0 tx=0).stop 2024-04-25T08:38:57.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.126+0000 7f9917f29640 1 -- 172.21.15.38:0/762652717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99100b79d0 msgr2=0x7f9910155c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.126+0000 7f9917f29640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99100b79d0 0x7f9910155c60 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f990002ac00 tx=0x7f9900004a40 comp rx=0 tx=0).stop 2024-04-25T08:38:57.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.126+0000 7f9917f29640 1 -- 172.21.15.38:0/762652717 shutdown_connections 2024-04-25T08:38:57.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.126+0000 7f9917f29640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99100b79d0 0x7f9910155c60 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.126+0000 7f9917f29640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98fc06b500 0x7f98fc06d9c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.126+0000 7f9917f29640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9910160ef0 0x7f99100b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.126+0000 7f9917f29640 1 --2- 172.21.15.38:0/762652717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99100b2ab0 0x7f991014ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.127+0000 7f9917f29640 1 -- 172.21.15.38:0/762652717 >> 172.21.15.38:0/762652717 conn(0x7f9910066d10 msgr2=0x7f99100d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:57.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.127+0000 7f9917f29640 1 -- 172.21.15.38:0/762652717 shutdown_connections 2024-04-25T08:38:57.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.127+0000 7f9917f29640 1 -- 172.21.15.38:0/762652717 wait complete. 2024-04-25T08:38:57.133 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:38:57.159 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:38:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.246+0000 7fe9c9c9e640 1 Processor -- start 2024-04-25T08:38:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.247+0000 7fe9c9c9e640 1 -- start start 2024-04-25T08:38:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.247+0000 7fe9c9c9e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe9c40d33d0 0x7fe9c40d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.247+0000 7fe9c9c9e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c40d2330 con 0x7fe9c40d1e90 2024-04-25T08:38:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.247+0000 7fe9c9c9e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c40d2610 con 0x7fe9c40d0320 2024-04-25T08:38:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.247+0000 7fe9c9c9e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c40d28f0 con 0x7fe9c40d33d0 2024-04-25T08:38:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.247+0000 7fe9c3fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe9c40d1e90 0x7fe9c40b6980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:50402/0 (socket says 172.21.15.38:50402) 2024-04-25T08:38:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.247+0000 7fe9c3fff640 1 -- 172.21.15.38:0/2443359939 learned_addr learned my addr 172.21.15.38:0/2443359939 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.248+0000 7fe9c2ffd640 1 --2- 172.21.15.38:0/2443359939 >> v2:172.21.15.107:3300/0 conn(0x7fe9c40d33d0 0x7fe9c40d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.248+0000 7fe9c27fc640 1 -- 172.21.15.38:0/2443359939 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303423317 0 0) 0x7fe9c40d2330 con 0x7fe9c40d1e90 2024-04-25T08:38:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.248+0000 7fe9c27fc640 1 -- 172.21.15.38:0/2443359939 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9ac003520 con 0x7fe9c40d1e90 2024-04-25T08:38:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.248+0000 7fe9c2ffd640 1 -- 172.21.15.38:0/2443359939 >> v1:172.21.15.107:6789/0 conn(0x7fe9c40d0320 legacy=0x7fe9c40d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.248+0000 7fe9c2ffd640 1 -- 172.21.15.38:0/2443359939 >> v1:172.21.15.38:6789/0 conn(0x7fe9c40d1e90 legacy=0x7fe9c40b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.248+0000 7fe9c27fc640 1 -- 172.21.15.38:0/2443359939 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1318205859 0 0) 0x7fe9ac003520 con 0x7fe9c40d1e90 2024-04-25T08:38:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.248+0000 7fe9c2ffd640 1 -- 172.21.15.38:0/2443359939 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c40d2bd0 con 0x7fe9c40d33d0 2024-04-25T08:38:57.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.249+0000 7fe9c2ffd640 1 --2- 172.21.15.38:0/2443359939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c40d33d0 0x7fe9c40d1950 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40097a0 tx=0x7fe9b402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbceffc236b9438f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:57.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.249+0000 7fe9c27fc640 1 -- 172.21.15.38:0/2443359939 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4037070 con 0x7fe9c40d33d0 2024-04-25T08:38:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.249+0000 7fe9c27fc640 1 -- 172.21.15.38:0/2443359939 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b402ace0 con 0x7fe9c40d33d0 2024-04-25T08:38:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.249+0000 7fe9c27fc640 1 -- 172.21.15.38:0/2443359939 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0019f0 con 0x7fe9c40d33d0 2024-04-25T08:38:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.249+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/2443359939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c40d33d0 msgr2=0x7fe9c40d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.249+0000 7fe9c9c9e640 1 --2- 172.21.15.38:0/2443359939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c40d33d0 0x7fe9c40d1950 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40097a0 tx=0x7fe9b402a250 comp rx=0 tx=0).stop 2024-04-25T08:38:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.249+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/2443359939 shutdown_connections 2024-04-25T08:38:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.249+0000 7fe9c9c9e640 1 --2- 172.21.15.38:0/2443359939 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c40d33d0 0x7fe9c40d1950 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.249+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/2443359939 >> 172.21.15.38:0/2443359939 conn(0x7fe9c40668f0 msgr2=0x7fe9c4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.250+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/2443359939 shutdown_connections 2024-04-25T08:38:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.250+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/2443359939 wait complete. 2024-04-25T08:38:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.251+0000 7fe9c9c9e640 1 Processor -- start 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.251+0000 7fe9c9c9e640 1 -- start start 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c9c9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c40d0320 0x7fe9c40b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c9c9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9c40d1e90 0x7fe9c414dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c9c9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c40d33d0 0x7fe9c40b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c9c9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c4161390 con 0x7fe9c40d0320 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c9c9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c4161210 con 0x7fe9c40d33d0 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c9c9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40b9720 con 0x7fe9c40d1e90 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c40d0320 0x7fe9c40b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c40d0320 0x7fe9c40b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50974/0 (socket says 172.21.15.38:50974) 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c37fe640 1 -- 172.21.15.38:0/1812812701 learned_addr learned my addr 172.21.15.38:0/1812812701 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c2ffd640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9c40d1e90 0x7fe9c414dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c3fff640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c40d33d0 0x7fe9c40b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c37fe640 1 -- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9c40d1e90 msgr2=0x7fe9c414dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c37fe640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9c40d1e90 0x7fe9c414dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c37fe640 1 -- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c40d33d0 msgr2=0x7fe9c40b44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:57.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c37fe640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c40d33d0 0x7fe9c40b44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:57.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c37fe640 1 -- 172.21.15.38:0/1812812701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c414e290 con 0x7fe9c40d0320 2024-04-25T08:38:57.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c37fe640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c40d0320 0x7fe9c40b5ec0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c480 tx=0x7fe9b800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:57.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9abfff640 1 -- 172.21.15.38:0/1812812701 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80052d0 con 0x7fe9c40d0320 2024-04-25T08:38:57.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9abfff640 1 -- 172.21.15.38:0/1812812701 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8005470 con 0x7fe9c40d0320 2024-04-25T08:38:57.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/1812812701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c40b4e20 con 0x7fe9c40d0320 2024-04-25T08:38:57.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.252+0000 7fe9abfff640 1 -- 172.21.15.38:0/1812812701 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8012870 con 0x7fe9c40d0320 2024-04-25T08:38:57.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.253+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/1812812701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c40b53e0 con 0x7fe9c40d0320 2024-04-25T08:38:57.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.253+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/1812812701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c4068e80 con 0x7fe9c40d0320 2024-04-25T08:38:57.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.254+0000 7fe9abfff640 1 -- 172.21.15.38:0/1812812701 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9b8012a10 con 0x7fe9c40d0320 2024-04-25T08:38:57.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.254+0000 7fe9abfff640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9b006b4d0 0x7fe9b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:57.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.254+0000 7fe9abfff640 1 -- 172.21.15.38:0/1812812701 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe9b8089e90 con 0x7fe9c40d0320 2024-04-25T08:38:57.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.254+0000 7fe9c2ffd640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9b006b4d0 0x7fe9b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:57.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.255+0000 7fe9c2ffd640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9b006b4d0 0x7fe9b006d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4034000 tx=0x7fe9b402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:57.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.259+0000 7fe9abfff640 1 -- 172.21.15.38:0/1812812701 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9c414e290 con 0x7fe9c40d0320 2024-04-25T08:38:57.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:57.485+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/1812812701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe9c415b4b0 con 0x7fe9c40d0320 2024-04-25T08:38:58.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.136+0000 7fe9abfff640 1 -- 172.21.15.38:0/1812812701 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe9b8058570 con 0x7fe9c40d0320 2024-04-25T08:38:58.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.139+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9b006b4d0 msgr2=0x7fe9b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:58.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.139+0000 7fe9c9c9e640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9b006b4d0 0x7fe9b006d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4034000 tx=0x7fe9b402e000 comp rx=0 tx=0).stop 2024-04-25T08:38:58.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.139+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/1812812701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c40d0320 msgr2=0x7fe9c40b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:58.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.139+0000 7fe9c9c9e640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c40d0320 0x7fe9c40b5ec0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800c480 tx=0x7fe9b800c940 comp rx=0 tx=0).stop 2024-04-25T08:38:58.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.139+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/1812812701 shutdown_connections 2024-04-25T08:38:58.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.139+0000 7fe9c9c9e640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9c40d33d0 0x7fe9c40b44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:58.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.139+0000 7fe9c9c9e640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9b006b4d0 0x7fe9b006d990 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:58.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.139+0000 7fe9c9c9e640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9c40d1e90 0x7fe9c414dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:58.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.139+0000 7fe9c9c9e640 1 --2- 172.21.15.38:0/1812812701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9c40d0320 0x7fe9c40b5ec0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:58.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.139+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/1812812701 >> 172.21.15.38:0/1812812701 conn(0x7fe9c40668f0 msgr2=0x7fe9c40b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:58.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.140+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/1812812701 shutdown_connections 2024-04-25T08:38:58.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.140+0000 7fe9c9c9e640 1 -- 172.21.15.38:0/1812812701 wait complete. 2024-04-25T08:38:58.147 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:38:58.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.303+0000 7f7c8ac41640 1 Processor -- start 2024-04-25T08:38:58.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.303+0000 7f7c8ac41640 1 -- start start 2024-04-25T08:38:58.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c8ac41640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7c841612c0 0x7f7c840b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:58.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c8ac41640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c840b6a00 0x7f7c840d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:58.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c8ac41640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c840b7200 con 0x7f7c840d0870 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c8ac41640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c840b74e0 con 0x7f7c841612c0 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c8ac41640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840b77c0 con 0x7f7c840b6a00 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c889dd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7c841612c0 0x7f7c840b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c889dd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7c841612c0 0x7f7c840b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50976/0 (socket says 172.21.15.38:50976) 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c889dd640 1 -- 172.21.15.38:0/2745780487 learned_addr learned my addr 172.21.15.38:0/2745780487 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c83fff640 1 --2- 172.21.15.38:0/2745780487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c840b6a00 0x7f7c840d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c889dd640 1 -- 172.21.15.38:0/2745780487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c840b6a00 msgr2=0x7f7c840d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c889dd640 1 --2- 172.21.15.38:0/2745780487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c840b6a00 0x7f7c840d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c889dd640 1 -- 172.21.15.38:0/2745780487 >> v1:172.21.15.107:6789/0 conn(0x7f7c840d0870 legacy=0x7f7c840b82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.304+0000 7f7c889dd640 1 -- 172.21.15.38:0/2745780487 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c840b7aa0 con 0x7f7c841612c0 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.305+0000 7f7c889dd640 1 --2- 172.21.15.38:0/2745780487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c841612c0 0x7f7c840b93f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009870 tx=0x7f7c7802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7150f77f306de5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.305+0000 7f7c837fe640 1 -- 172.21.15.38:0/2745780487 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78037070 con 0x7f7c841612c0 2024-04-25T08:38:58.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.305+0000 7f7c837fe640 1 -- 172.21.15.38:0/2745780487 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c78032440 con 0x7f7c841612c0 2024-04-25T08:38:58.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.305+0000 7f7c837fe640 1 -- 172.21.15.38:0/2745780487 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78032810 con 0x7f7c841612c0 2024-04-25T08:38:58.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.306+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2745780487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c841612c0 msgr2=0x7f7c840b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:58.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.306+0000 7f7c8ac41640 1 --2- 172.21.15.38:0/2745780487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c841612c0 0x7f7c840b93f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009870 tx=0x7f7c7802a180 comp rx=0 tx=0).stop 2024-04-25T08:38:58.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.306+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2745780487 shutdown_connections 2024-04-25T08:38:58.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.306+0000 7f7c8ac41640 1 --2- 172.21.15.38:0/2745780487 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c840b6a00 0x7f7c840d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:58.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.306+0000 7f7c8ac41640 1 --2- 172.21.15.38:0/2745780487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c841612c0 0x7f7c840b93f0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:58.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.306+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2745780487 >> 172.21.15.38:0/2745780487 conn(0x7f7c84066d10 msgr2=0x7f7c84067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:58.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.306+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2745780487 shutdown_connections 2024-04-25T08:38:58.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.306+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2745780487 wait complete. 2024-04-25T08:38:58.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.307+0000 7f7c8ac41640 1 Processor -- start 2024-04-25T08:38:58.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.307+0000 7f7c8ac41640 1 -- start start 2024-04-25T08:38:58.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c8ac41640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c840b6a00 0x7f7c84150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:58.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c8ac41640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c840d0870 0x7f7c841521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:58.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c8ac41640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c841612c0 0x7f7c840ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:58.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c8ac41640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840b6e60 con 0x7f7c840d0870 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c8ac41640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840d2170 con 0x7f7c840b6a00 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c8ac41640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840b6fe0 con 0x7f7c841612c0 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c83fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c840d0870 0x7f7c841521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c83fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c840d0870 0x7f7c841521a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50978/0 (socket says 172.21.15.38:50978) 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c83fff640 1 -- 172.21.15.38:0/2625063432 learned_addr learned my addr 172.21.15.38:0/2625063432 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c889dd640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c840b6a00 0x7f7c84150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c891de640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c841612c0 0x7f7c840ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c83fff640 1 -- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c841612c0 msgr2=0x7f7c840ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c83fff640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c841612c0 0x7f7c840ccb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c83fff640 1 -- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c840b6a00 msgr2=0x7f7c84150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c83fff640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c840b6a00 0x7f7c84150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c83fff640 1 -- 172.21.15.38:0/2625063432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c841529f0 con 0x7f7c840d0870 2024-04-25T08:38:58.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c891de640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c841612c0 0x7f7c840ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:38:58.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.308+0000 7f7c83fff640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c840d0870 0x7f7c841521a0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c7f0 tx=0x7f7c7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:58.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.309+0000 7f7c817fa640 1 -- 172.21.15.38:0/2625063432 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70015070 con 0x7f7c840d0870 2024-04-25T08:38:58.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.309+0000 7f7c817fa640 1 -- 172.21.15.38:0/2625063432 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c70004240 con 0x7f7c840d0870 2024-04-25T08:38:58.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.309+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2625063432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c84152cd0 con 0x7f7c840d0870 2024-04-25T08:38:58.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.309+0000 7f7c817fa640 1 -- 172.21.15.38:0/2625063432 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70007620 con 0x7f7c840d0870 2024-04-25T08:38:58.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.309+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2625063432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c84153290 con 0x7f7c840d0870 2024-04-25T08:38:58.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.310+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2625063432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c84068e80 con 0x7f7c840d0870 2024-04-25T08:38:58.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.310+0000 7f7c817fa640 1 -- 172.21.15.38:0/2625063432 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c700079d0 con 0x7f7c840d0870 2024-04-25T08:38:58.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.311+0000 7f7c817fa640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c6806b4d0 0x7f7c6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:58.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.311+0000 7f7c817fa640 1 -- 172.21.15.38:0/2625063432 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7c7008a840 con 0x7f7c840d0870 2024-04-25T08:38:58.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.311+0000 7f7c889dd640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c6806b4d0 0x7f7c6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:58.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.312+0000 7f7c889dd640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c6806b4d0 0x7f7c6806d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7c78034000 tx=0x7f7c7802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:58.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.315+0000 7f7c817fa640 1 -- 172.21.15.38:0/2625063432 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c70058e00 con 0x7f7c840d0870 2024-04-25T08:38:58.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:58.495+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2625063432 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7c840cdc90 con 0x7f7c840d0870 2024-04-25T08:38:59.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.138+0000 7f7c817fa640 1 -- 172.21.15.38:0/2625063432 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7c7005cea0 con 0x7f7c840d0870 2024-04-25T08:38:59.132 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:38:59.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.141+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c6806b4d0 msgr2=0x7f7c6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:59.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.141+0000 7f7c8ac41640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c6806b4d0 0x7f7c6806d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7c78034000 tx=0x7f7c7802e000 comp rx=0 tx=0).stop 2024-04-25T08:38:59.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.141+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2625063432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c840d0870 msgr2=0x7f7c841521a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:59.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.141+0000 7f7c8ac41640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c840d0870 0x7f7c841521a0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c7f0 tx=0x7f7c7000ccb0 comp rx=0 tx=0).stop 2024-04-25T08:38:59.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.141+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2625063432 shutdown_connections 2024-04-25T08:38:59.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.141+0000 7f7c8ac41640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c6806b4d0 0x7f7c6806d990 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:59.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.141+0000 7f7c8ac41640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c841612c0 0x7f7c840ccb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:59.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.141+0000 7f7c8ac41640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c840d0870 0x7f7c841521a0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:59.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.141+0000 7f7c8ac41640 1 --2- 172.21.15.38:0/2625063432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c840b6a00 0x7f7c84150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:59.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.141+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2625063432 >> 172.21.15.38:0/2625063432 conn(0x7f7c84066d10 msgr2=0x7f7c840d3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:59.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.141+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2625063432 shutdown_connections 2024-04-25T08:38:59.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.141+0000 7f7c8ac41640 1 -- 172.21.15.38:0/2625063432 wait complete. 2024-04-25T08:38:59.150 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:38:59.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.268+0000 7fd180ab3640 1 Processor -- start 2024-04-25T08:38:59.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.268+0000 7fd180ab3640 1 -- start start 2024-04-25T08:38:59.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.269+0000 7fd180ab3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c0d0870 0x7fd17c0d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:59.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.269+0000 7fd180ab3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17c0d11c0 con 0x7fd17c1612c0 2024-04-25T08:38:59.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.269+0000 7fd180ab3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17c0b9880 con 0x7fd17c0b93f0 2024-04-25T08:38:59.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.269+0000 7fd180ab3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0b7200 con 0x7fd17c0d0870 2024-04-25T08:38:59.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.269+0000 7fd17a59c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd17c1612c0 0x7fd17c0b6a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37026/0 (socket says 172.21.15.38:37026) 2024-04-25T08:38:59.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.269+0000 7fd17a59c640 1 -- 172.21.15.38:0/1985790860 learned_addr learned my addr 172.21.15.38:0/1985790860 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:38:59.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.269+0000 7fd17ad9d640 1 --2- 172.21.15.38:0/1985790860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c0d0870 0x7fd17c0d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:59.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.269+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618998090 0 0) 0x7fd174001110 con 0x7fd17c1612c0 2024-04-25T08:38:59.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd168003520 con 0x7fd17c1612c0 2024-04-25T08:38:59.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204535158 0 0) 0x7fd17c0b9880 con 0x7fd17c0b93f0 2024-04-25T08:38:59.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd174001110 con 0x7fd17c0b93f0 2024-04-25T08:38:59.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3903695150 0 0) 0x7fd168003520 con 0x7fd17c1612c0 2024-04-25T08:38:59.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c0d0870 msgr2=0x7fd17c0d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:59.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 --2- 172.21.15.38:0/1985790860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c0d0870 0x7fd17c0d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:59.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 >> v1:172.21.15.107:6789/0 conn(0x7fd17c0b93f0 legacy=0x7fd17c0d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:59.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c0b74e0 con 0x7fd17c1612c0 2024-04-25T08:38:59.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fd174001840 con 0x7fd17c1612c0 2024-04-25T08:38:59.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 >> v1:172.21.15.38:6789/0 conn(0x7fd17c1612c0 legacy=0x7fd17c0b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:59.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 --2- 172.21.15.38:0/1985790860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd168003b70 0x7fd168003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:59.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 --2- 172.21.15.38:0/1985790860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c0d0870 0x7fd168004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:59.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 --2- 172.21.15.38:0/1985790860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd168004d40 0x7fd1680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:59.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0b7200 con 0x7fd168003b70 2024-04-25T08:38:59.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16800baf0 con 0x7fd168004d40 2024-04-25T08:38:59.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16800bcf0 con 0x7fd17c0d0870 2024-04-25T08:38:59.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17a59c640 1 --2- 172.21.15.38:0/1985790860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd168003b70 0x7fd168003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:59.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17ad9d640 1 --2- 172.21.15.38:0/1985790860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd168004d40 0x7fd1680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:59.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd179d9b640 1 --2- 172.21.15.38:0/1985790860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c0d0870 0x7fd168004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:59.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17a59c640 1 -- 172.21.15.38:0/1985790860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c0d0870 msgr2=0x7fd168004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:59.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17a59c640 1 --2- 172.21.15.38:0/1985790860 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c0d0870 0x7fd168004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17a59c640 1 -- 172.21.15.38:0/1985790860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd168004d40 msgr2=0x7fd1680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17a59c640 1 --2- 172.21.15.38:0/1985790860 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd168004d40 0x7fd1680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.270+0000 7fd17a59c640 1 -- 172.21.15.38:0/1985790860 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd168003520 con 0x7fd168003b70 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.271+0000 7fd17a59c640 1 --2- 172.21.15.38:0/1985790860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd168003b70 0x7fd168003f70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fd17400c4c0 tx=0x7fd168005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5479225e6a0903 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.271+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17400c0e0 con 0x7fd168003b70 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.271+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd174034070 con 0x7fd168003b70 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.271+0000 7fd17959a640 1 -- 172.21.15.38:0/1985790860 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174005d30 con 0x7fd168003b70 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.272+0000 7fd180ab3640 1 -- 172.21.15.38:0/1985790860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd168003b70 msgr2=0x7fd168003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.272+0000 7fd180ab3640 1 --2- 172.21.15.38:0/1985790860 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd168003b70 0x7fd168003f70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fd17400c4c0 tx=0x7fd168005830 comp rx=0 tx=0).stop 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.272+0000 7fd17ad9d640 1 -- 172.21.15.38:0/1985790860 reap_dead start 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.272+0000 7fd180ab3640 1 -- 172.21.15.38:0/1985790860 shutdown_connections 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.272+0000 7fd180ab3640 1 -- 172.21.15.38:0/1985790860 >> 172.21.15.38:0/1985790860 conn(0x7fd17c066d10 msgr2=0x7fd17c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.272+0000 7fd180ab3640 1 -- 172.21.15.38:0/1985790860 shutdown_connections 2024-04-25T08:38:59.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.272+0000 7fd180ab3640 1 -- 172.21.15.38:0/1985790860 wait complete. 2024-04-25T08:38:59.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.273+0000 7fd180ab3640 1 Processor -- start 2024-04-25T08:38:59.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.273+0000 7fd180ab3640 1 -- start start 2024-04-25T08:38:59.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.274+0000 7fd180ab3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd168003b70 0x7fd17c0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:59.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.274+0000 7fd180ab3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c141730 0x7fd17c160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:59.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.274+0000 7fd180ab3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd17c140a70 0x7fd17c140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:59.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.274+0000 7fd180ab3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0b7d80 con 0x7fd168003b70 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.274+0000 7fd180ab3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0b7aa0 con 0x7fd17c140a70 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.274+0000 7fd180ab3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0d0c80 con 0x7fd17c141730 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.274+0000 7fd17a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd168003b70 0x7fd17c0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.274+0000 7fd17a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd168003b70 0x7fd17c0cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53570/0 (socket says 172.21.15.38:53570) 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.274+0000 7fd17a59c640 1 -- 172.21.15.38:0/152560414 learned_addr learned my addr 172.21.15.38:0/152560414 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.274+0000 7fd17ad9d640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd17c140a70 0x7fd17c140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.274+0000 7fd179d9b640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c141730 0x7fd17c160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.275+0000 7fd17a59c640 1 -- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c141730 msgr2=0x7fd17c160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.275+0000 7fd17a59c640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c141730 0x7fd17c160460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.275+0000 7fd17a59c640 1 -- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd17c140a70 msgr2=0x7fd17c140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.275+0000 7fd17a59c640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd17c140a70 0x7fd17c140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.275+0000 7fd17a59c640 1 -- 172.21.15.38:0/152560414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c153990 con 0x7fd168003b70 2024-04-25T08:38:59.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.275+0000 7fd17a59c640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd168003b70 0x7fd17c0cae20 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fd17400c4c0 tx=0x7fd17c0b76b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.275+0000 7fd172ffd640 1 -- 172.21.15.38:0/152560414 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174007480 con 0x7fd168003b70 2024-04-25T08:38:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.275+0000 7fd172ffd640 1 -- 172.21.15.38:0/152560414 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd174007620 con 0x7fd168003b70 2024-04-25T08:38:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.275+0000 7fd180ab3640 1 -- 172.21.15.38:0/152560414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c153c70 con 0x7fd168003b70 2024-04-25T08:38:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.275+0000 7fd172ffd640 1 -- 172.21.15.38:0/152560414 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174039480 con 0x7fd168003b70 2024-04-25T08:38:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.275+0000 7fd180ab3640 1 -- 172.21.15.38:0/152560414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c1527e0 con 0x7fd168003b70 2024-04-25T08:38:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.276+0000 7fd180ab3640 1 -- 172.21.15.38:0/152560414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17c068e80 con 0x7fd168003b70 2024-04-25T08:38:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.276+0000 7fd172ffd640 1 -- 172.21.15.38:0/152560414 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd174039620 con 0x7fd168003b70 2024-04-25T08:38:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.277+0000 7fd172ffd640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd15c06b500 0x7fd15c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:38:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.277+0000 7fd172ffd640 1 -- 172.21.15.38:0/152560414 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fd1740aaf10 con 0x7fd168003b70 2024-04-25T08:38:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.277+0000 7fd179d9b640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd15c06b500 0x7fd15c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:38:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.278+0000 7fd179d9b640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd15c06b500 0x7fd15c06d9c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd17c154670 tx=0x7fd164009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:38:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.281+0000 7fd172ffd640 1 -- 172.21.15.38:0/152560414 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd174049060 con 0x7fd168003b70 2024-04-25T08:38:59.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:38:59.463+0000 7fd180ab3640 1 -- 172.21.15.38:0/152560414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd17c156d20 con 0x7fd168003b70 2024-04-25T08:39:00.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.140+0000 7fd172ffd640 1 -- 172.21.15.38:0/152560414 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd17c0d11c0 con 0x7fd168003b70 2024-04-25T08:39:00.133 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-04-25T08:39:00.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.145+0000 7fd180ab3640 1 -- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd15c06b500 msgr2=0x7fd15c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:00.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.145+0000 7fd180ab3640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd15c06b500 0x7fd15c06d9c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd17c154670 tx=0x7fd164009040 comp rx=0 tx=0).stop 2024-04-25T08:39:00.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.145+0000 7fd180ab3640 1 -- 172.21.15.38:0/152560414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd168003b70 msgr2=0x7fd17c0cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:00.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.145+0000 7fd180ab3640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd168003b70 0x7fd17c0cae20 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fd17400c4c0 tx=0x7fd17c0b76b0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.145+0000 7fd180ab3640 1 -- 172.21.15.38:0/152560414 shutdown_connections 2024-04-25T08:39:00.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.145+0000 7fd180ab3640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd17c140a70 0x7fd17c140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.145+0000 7fd180ab3640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd15c06b500 0x7fd15c06d9c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.145+0000 7fd180ab3640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd17c141730 0x7fd17c160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.145+0000 7fd180ab3640 1 --2- 172.21.15.38:0/152560414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd168003b70 0x7fd17c0cae20 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.145+0000 7fd180ab3640 1 -- 172.21.15.38:0/152560414 >> 172.21.15.38:0/152560414 conn(0x7fd17c066d10 msgr2=0x7fd17c0c7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:00.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.145+0000 7fd180ab3640 1 -- 172.21.15.38:0/152560414 shutdown_connections 2024-04-25T08:39:00.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.145+0000 7fd180ab3640 1 -- 172.21.15.38:0/152560414 wait complete. 2024-04-25T08:39:00.158 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:39:00.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.279+0000 7fb133e4d640 1 Processor -- start 2024-04-25T08:39:00.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.279+0000 7fb133e4d640 1 -- start start 2024-04-25T08:39:00.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb133e4d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb12c0d0320 0x7fb12c0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:00.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb133e4d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb12c0d0700 0x7fb12c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb133e4d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12c0d2330 con 0x7fb12c0d1e90 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb133e4d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb12c0d2610 con 0x7fb12c0d0700 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb133e4d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c0d28f0 con 0x7fb12c0d0320 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb1313e8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb12c0d0700 0x7fb12c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb1313e8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb12c0d0700 0x7fb12c0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53578/0 (socket says 172.21.15.38:53578) 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb1313e8640 1 -- 172.21.15.38:0/4244943604 learned_addr learned my addr 172.21.15.38:0/4244943604 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb131be9640 1 --2- 172.21.15.38:0/4244943604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb12c0d0320 0x7fb12c0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb1313e8640 1 -- 172.21.15.38:0/4244943604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb12c0d0320 msgr2=0x7fb12c0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb1313e8640 1 --2- 172.21.15.38:0/4244943604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb12c0d0320 0x7fb12c0d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb1313e8640 1 -- 172.21.15.38:0/4244943604 >> v1:172.21.15.107:6789/0 conn(0x7fb12c0d1e90 legacy=0x7fb12c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb1313e8640 1 -- 172.21.15.38:0/4244943604 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12c0d2bd0 con 0x7fb12c0d0700 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.280+0000 7fb131be9640 1 --2- 172.21.15.38:0/4244943604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb12c0d0320 0x7fb12c0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:00.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.281+0000 7fb1313e8640 1 --2- 172.21.15.38:0/4244943604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb12c0d0700 0x7fb12c0d1950 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009870 tx=0x7fb11c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536b3650f9ca8294 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.281+0000 7fb130be7640 1 -- 172.21.15.38:0/4244943604 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c037070 con 0x7fb12c0d0700 2024-04-25T08:39:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.281+0000 7fb130be7640 1 -- 172.21.15.38:0/4244943604 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c02ae50 con 0x7fb12c0d0700 2024-04-25T08:39:00.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.281+0000 7fb130be7640 1 -- 172.21.15.38:0/4244943604 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c032860 con 0x7fb12c0d0700 2024-04-25T08:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.282+0000 7fb133e4d640 1 -- 172.21.15.38:0/4244943604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb12c0d0700 msgr2=0x7fb12c0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.282+0000 7fb133e4d640 1 --2- 172.21.15.38:0/4244943604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb12c0d0700 0x7fb12c0d1950 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fb11c009870 tx=0x7fb11c02a300 comp rx=0 tx=0).stop 2024-04-25T08:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.282+0000 7fb133e4d640 1 -- 172.21.15.38:0/4244943604 shutdown_connections 2024-04-25T08:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.282+0000 7fb133e4d640 1 --2- 172.21.15.38:0/4244943604 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb12c0d0700 0x7fb12c0d1950 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.282+0000 7fb133e4d640 1 --2- 172.21.15.38:0/4244943604 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb12c0d0320 0x7fb12c0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.282+0000 7fb133e4d640 1 -- 172.21.15.38:0/4244943604 >> 172.21.15.38:0/4244943604 conn(0x7fb12c0668f0 msgr2=0x7fb12c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.282+0000 7fb133e4d640 1 -- 172.21.15.38:0/4244943604 shutdown_connections 2024-04-25T08:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.282+0000 7fb133e4d640 1 -- 172.21.15.38:0/4244943604 wait complete. 2024-04-25T08:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.283+0000 7fb133e4d640 1 Processor -- start 2024-04-25T08:39:00.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.283+0000 7fb133e4d640 1 -- start start 2024-04-25T08:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb133e4d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb12c0d0320 0x7fb12c0b5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb133e4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb12c0d0700 0x7fb12c153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb133e4d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb12c0d1e90 0x7fb12c1f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb133e4d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c0d0b40 con 0x7fb12c0d0700 2024-04-25T08:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb133e4d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c161210 con 0x7fb12c0d0320 2024-04-25T08:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb133e4d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12c0b5ed0 con 0x7fb12c0d1e90 2024-04-25T08:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb1313e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb12c0d0700 0x7fb12c153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb1313e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb12c0d0700 0x7fb12c153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53582/0 (socket says 172.21.15.38:53582) 2024-04-25T08:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb1313e8640 1 -- 172.21.15.38:0/3388179990 learned_addr learned my addr 172.21.15.38:0/3388179990 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb131be9640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb12c0d0320 0x7fb12c0b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb1323ea640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb12c0d1e90 0x7fb12c1f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:00.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb1313e8640 1 -- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb12c0d1e90 msgr2=0x7fb12c1f7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb1313e8640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb12c0d1e90 0x7fb12c1f7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb1313e8640 1 -- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb12c0d0320 msgr2=0x7fb12c0b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb1313e8640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb12c0d0320 0x7fb12c0b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb1313e8640 1 -- 172.21.15.38:0/3388179990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12c1f7cf0 con 0x7fb12c0d0700 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.284+0000 7fb1323ea640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb12c0d1e90 0x7fb12c1f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.285+0000 7fb1313e8640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb12c0d0700 0x7fb12c153b10 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fb11c000c00 tx=0x7fb11c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.285+0000 7fb1227fc640 1 -- 172.21.15.38:0/3388179990 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c041070 con 0x7fb12c0d0700 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.285+0000 7fb1227fc640 1 -- 172.21.15.38:0/3388179990 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c03c670 con 0x7fb12c0d0700 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.285+0000 7fb1227fc640 1 -- 172.21.15.38:0/3388179990 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c037040 con 0x7fb12c0d0700 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.285+0000 7fb133e4d640 1 -- 172.21.15.38:0/3388179990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12c1f7fd0 con 0x7fb12c0d0700 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.285+0000 7fb133e4d640 1 -- 172.21.15.38:0/3388179990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12c0b5470 con 0x7fb12c0d0700 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.286+0000 7fb133e4d640 1 -- 172.21.15.38:0/3388179990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12c068e80 con 0x7fb12c0d0700 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.286+0000 7fb1227fc640 1 -- 172.21.15.38:0/3388179990 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb11c044070 con 0x7fb12c0d0700 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.287+0000 7fb1227fc640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb11006b4d0 0x7fb11006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:00.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.287+0000 7fb1227fc640 1 -- 172.21.15.38:0/3388179990 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb11c0aaad0 con 0x7fb12c0d0700 2024-04-25T08:39:00.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.287+0000 7fb131be9640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb11006b4d0 0x7fb11006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:00.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.288+0000 7fb131be9640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb11006b4d0 0x7fb11006d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb128006fd0 tx=0x7fb128008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:00.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.291+0000 7fb1227fc640 1 -- 172.21.15.38:0/3388179990 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb11c079be0 con 0x7fb12c0d0700 2024-04-25T08:39:00.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.515+0000 7fb133e4d640 1 -- 172.21.15.38:0/3388179990 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb12c0b88c0 con 0x7fb12c0d0700 2024-04-25T08:39:00.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.518+0000 7fb1227fc640 1 -- 172.21.15.38:0/3388179990 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb11c0794e0 con 0x7fb12c0d0700 2024-04-25T08:39:00.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.520+0000 7fb133e4d640 1 -- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb11006b4d0 msgr2=0x7fb11006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:00.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.520+0000 7fb133e4d640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb11006b4d0 0x7fb11006d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb128006fd0 tx=0x7fb128008040 comp rx=0 tx=0).stop 2024-04-25T08:39:00.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.520+0000 7fb133e4d640 1 -- 172.21.15.38:0/3388179990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb12c0d0700 msgr2=0x7fb12c153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:00.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.520+0000 7fb133e4d640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb12c0d0700 0x7fb12c153b10 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fb11c000c00 tx=0x7fb11c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.520+0000 7fb133e4d640 1 -- 172.21.15.38:0/3388179990 shutdown_connections 2024-04-25T08:39:00.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.520+0000 7fb133e4d640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb11006b4d0 0x7fb11006d990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.520+0000 7fb133e4d640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb12c0d1e90 0x7fb12c1f7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.520+0000 7fb133e4d640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb12c0d0700 0x7fb12c153b10 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.520+0000 7fb133e4d640 1 --2- 172.21.15.38:0/3388179990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb12c0d0320 0x7fb12c0b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.520+0000 7fb133e4d640 1 -- 172.21.15.38:0/3388179990 >> 172.21.15.38:0/3388179990 conn(0x7fb12c0668f0 msgr2=0x7fb12c0d34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:00.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.520+0000 7fb133e4d640 1 -- 172.21.15.38:0/3388179990 shutdown_connections 2024-04-25T08:39:00.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.520+0000 7fb133e4d640 1 -- 172.21.15.38:0/3388179990 wait complete. 2024-04-25T08:39:00.525 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T08:39:00.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.632+0000 7f4cf756a640 1 Processor -- start 2024-04-25T08:39:00.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.633+0000 7f4cf756a640 1 -- start start 2024-04-25T08:39:00.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.633+0000 7f4cf756a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cf006c110 0x7f4cf006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:00.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.633+0000 7f4cf756a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4cf015df50 0x7f4cf00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:00.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.633+0000 7f4cf756a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf006ca30 con 0x7f4cf00ce2e0 2024-04-25T08:39:00.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.633+0000 7f4cf756a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf00cf2d0 con 0x7f4cf015df50 2024-04-25T08:39:00.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.633+0000 7f4cf756a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00ceff0 con 0x7f4cf006c110 2024-04-25T08:39:00.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.633+0000 7f4cf4b05640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4cf015df50 0x7f4cf00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:00.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.634+0000 7f4cf4b05640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4cf015df50 0x7f4cf00ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53592/0 (socket says 172.21.15.38:53592) 2024-04-25T08:39:00.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.634+0000 7f4cf4b05640 1 -- 172.21.15.38:0/4197248088 learned_addr learned my addr 172.21.15.38:0/4197248088 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:00.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.634+0000 7f4cf5306640 1 --2- 172.21.15.38:0/4197248088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cf006c110 0x7f4cf006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:00.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.634+0000 7f4cf4b05640 1 -- 172.21.15.38:0/4197248088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cf006c110 msgr2=0x7f4cf006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:00.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.634+0000 7f4cf4b05640 1 --2- 172.21.15.38:0/4197248088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cf006c110 0x7f4cf006c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.634+0000 7f4cf4b05640 1 -- 172.21.15.38:0/4197248088 >> v1:172.21.15.107:6789/0 conn(0x7f4cf00ce2e0 legacy=0x7f4cf00d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:00.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.634+0000 7f4cf4b05640 1 -- 172.21.15.38:0/4197248088 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf00d1f70 con 0x7f4cf015df50 2024-04-25T08:39:00.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.635+0000 7f4cf4b05640 1 --2- 172.21.15.38:0/4197248088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf015df50 0x7f4cf00ce7a0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009870 tx=0x7f4ce402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a519574931004ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:00.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.635+0000 7f4ceffff640 1 -- 172.21.15.38:0/4197248088 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4037070 con 0x7f4cf015df50 2024-04-25T08:39:00.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.635+0000 7f4ceffff640 1 -- 172.21.15.38:0/4197248088 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce4032440 con 0x7f4cf015df50 2024-04-25T08:39:00.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.635+0000 7f4ceffff640 1 -- 172.21.15.38:0/4197248088 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4032810 con 0x7f4cf015df50 2024-04-25T08:39:00.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.635+0000 7f4cf756a640 1 -- 172.21.15.38:0/4197248088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf015df50 msgr2=0x7f4cf00ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:00.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.635+0000 7f4cf756a640 1 --2- 172.21.15.38:0/4197248088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf015df50 0x7f4cf00ce7a0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4009870 tx=0x7f4ce402a270 comp rx=0 tx=0).stop 2024-04-25T08:39:00.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.635+0000 7f4cf756a640 1 -- 172.21.15.38:0/4197248088 shutdown_connections 2024-04-25T08:39:00.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.635+0000 7f4cf756a640 1 --2- 172.21.15.38:0/4197248088 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf015df50 0x7f4cf00ce7a0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.635+0000 7f4cf756a640 1 --2- 172.21.15.38:0/4197248088 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cf006c110 0x7f4cf006c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.635+0000 7f4cf756a640 1 -- 172.21.15.38:0/4197248088 >> 172.21.15.38:0/4197248088 conn(0x7f4cf00d3fb0 msgr2=0x7f4cf00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:00.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.635+0000 7f4cf756a640 1 -- 172.21.15.38:0/4197248088 shutdown_connections 2024-04-25T08:39:00.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.635+0000 7f4cf756a640 1 -- 172.21.15.38:0/4197248088 wait complete. 2024-04-25T08:39:00.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.636+0000 7f4cf756a640 1 Processor -- start 2024-04-25T08:39:00.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.637+0000 7f4cf756a640 1 -- start start 2024-04-25T08:39:00.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.637+0000 7f4cf756a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cf006c110 0x7f4cf015fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:00.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.637+0000 7f4cf756a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf00ce2e0 0x7f4cf014cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:00.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.637+0000 7f4cf756a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf015df50 0x7f4cf00c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:00.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.637+0000 7f4cf756a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00cece0 con 0x7f4cf015df50 2024-04-25T08:39:00.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.637+0000 7f4cf756a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00d2530 con 0x7f4cf00ce2e0 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.637+0000 7f4cf756a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf00d3160 con 0x7f4cf006c110 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.637+0000 7f4cf5b07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf015df50 0x7f4cf00c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.637+0000 7f4cf5b07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf015df50 0x7f4cf00c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53604/0 (socket says 172.21.15.38:53604) 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.637+0000 7f4cf5b07640 1 -- 172.21.15.38:0/2542754816 learned_addr learned my addr 172.21.15.38:0/2542754816 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cf5306640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cf006c110 0x7f4cf015fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cf4b05640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf00ce2e0 0x7f4cf014cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cf5b07640 1 -- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cf006c110 msgr2=0x7f4cf015fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cf5b07640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cf006c110 0x7f4cf015fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cf5b07640 1 -- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf00ce2e0 msgr2=0x7f4cf014cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cf5b07640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf00ce2e0 0x7f4cf014cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cf5b07640 1 -- 172.21.15.38:0/2542754816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf01f6990 con 0x7f4cf015df50 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cf5b07640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf015df50 0x7f4cf00c8390 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800c3d0 tx=0x7f4ce800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:00.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cedffb640 1 -- 172.21.15.38:0/2542754816 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8017070 con 0x7f4cf015df50 2024-04-25T08:39:00.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cedffb640 1 -- 172.21.15.38:0/2542754816 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce80052d0 con 0x7f4cf015df50 2024-04-25T08:39:00.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cf756a640 1 -- 172.21.15.38:0/2542754816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf01f6c70 con 0x7f4cf015df50 2024-04-25T08:39:00.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cedffb640 1 -- 172.21.15.38:0/2542754816 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8003830 con 0x7f4cf015df50 2024-04-25T08:39:00.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.638+0000 7f4cf756a640 1 -- 172.21.15.38:0/2542754816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf01f7230 con 0x7f4cf015df50 2024-04-25T08:39:00.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.639+0000 7f4cf756a640 1 -- 172.21.15.38:0/2542754816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf0068e80 con 0x7f4cf015df50 2024-04-25T08:39:00.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.640+0000 7f4cedffb640 1 -- 172.21.15.38:0/2542754816 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ce801a070 con 0x7f4cf015df50 2024-04-25T08:39:00.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.640+0000 7f4cedffb640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4cdc06b4d0 0x7f4cdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:00.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.640+0000 7f4cedffb640 1 -- 172.21.15.38:0/2542754816 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4ce8089360 con 0x7f4cf015df50 2024-04-25T08:39:00.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.640+0000 7f4cf5306640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4cdc06b4d0 0x7f4cdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:00.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.641+0000 7f4cf5306640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4cdc06b4d0 0x7f4cdc06d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00113e0 tx=0x7f4ce000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:00.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.644+0000 7f4cedffb640 1 -- 172.21.15.38:0/2542754816 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ce8057dc0 con 0x7f4cf015df50 2024-04-25T08:39:00.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:00.895+0000 7f4cf756a640 1 -- 172.21.15.38:0/2542754816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f4cf006b050 con 0x7f4cf015df50 2024-04-25T08:39:01.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.149+0000 7f4cedffb640 1 -- 172.21.15.38:0/2542754816 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v358) v1 ==== 144+0+0 (secure 0 0 0) 0x7f4ce8087e20 con 0x7f4cf015df50 2024-04-25T08:39:01.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.151+0000 7f4cf756a640 1 -- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4cdc06b4d0 msgr2=0x7f4cdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.151+0000 7f4cf756a640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4cdc06b4d0 0x7f4cdc06d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00113e0 tx=0x7f4ce000b040 comp rx=0 tx=0).stop 2024-04-25T08:39:01.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.151+0000 7f4cf756a640 1 -- 172.21.15.38:0/2542754816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf015df50 msgr2=0x7f4cf00c8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.151+0000 7f4cf756a640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf015df50 0x7f4cf00c8390 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f4ce800c3d0 tx=0x7f4ce800c890 comp rx=0 tx=0).stop 2024-04-25T08:39:01.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.151+0000 7f4cf756a640 1 -- 172.21.15.38:0/2542754816 shutdown_connections 2024-04-25T08:39:01.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.151+0000 7f4cf756a640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4cf015df50 0x7f4cf00c8390 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.151+0000 7f4cf756a640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4cf00ce2e0 0x7f4cf014cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.151+0000 7f4cf756a640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4cdc06b4d0 0x7f4cdc06d990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.151+0000 7f4cf756a640 1 --2- 172.21.15.38:0/2542754816 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4cf006c110 0x7f4cf015fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.151+0000 7f4cf756a640 1 -- 172.21.15.38:0/2542754816 >> 172.21.15.38:0/2542754816 conn(0x7f4cf00d3fb0 msgr2=0x7f4cf0073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:01.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.151+0000 7f4cf756a640 1 -- 172.21.15.38:0/2542754816 shutdown_connections 2024-04-25T08:39:01.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.151+0000 7f4cf756a640 1 -- 172.21.15.38:0/2542754816 wait complete. 2024-04-25T08:39:01.156 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T08:39:01.156 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:39:01.157 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:01.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.265+0000 7ff740589640 1 Processor -- start 2024-04-25T08:39:01.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.265+0000 7ff740589640 1 -- start start 2024-04-25T08:39:01.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff740589640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff73806c110 0x7ff73806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff740589640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff73815df50 0x7ff7380ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff740589640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff7380ce2e0 0x7ff7380d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff740589640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7380cf2d0 con 0x7ff73815df50 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff740589640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73806ca30 con 0x7ff7380ce2e0 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff740589640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380ceff0 con 0x7ff73806c110 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff73db24640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff73815df50 0x7ff7380ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff73db24640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff73815df50 0x7ff7380ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53606/0 (socket says 172.21.15.38:53606) 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff73db24640 1 -- 172.21.15.38:0/901642888 learned_addr learned my addr 172.21.15.38:0/901642888 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff73e325640 1 --2- 172.21.15.38:0/901642888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff73806c110 0x7ff73806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff73eb26640 1 --2- 172.21.15.38:0/901642888 >> v2:172.21.15.107:3300/0 conn(0x7ff7380ce2e0 0x7ff7380d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff73db24640 1 -- 172.21.15.38:0/901642888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff73806c110 msgr2=0x7ff73806c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff73db24640 1 --2- 172.21.15.38:0/901642888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff73806c110 0x7ff73806c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff73db24640 1 -- 172.21.15.38:0/901642888 >> v2:172.21.15.107:3300/0 conn(0x7ff7380ce2e0 msgr2=0x7ff7380d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff73db24640 1 --2- 172.21.15.38:0/901642888 >> v2:172.21.15.107:3300/0 conn(0x7ff7380ce2e0 0x7ff7380d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff73db24640 1 -- 172.21.15.38:0/901642888 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380d1f70 con 0x7ff73815df50 2024-04-25T08:39:01.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff73e325640 1 --2- 172.21.15.38:0/901642888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff73806c110 0x7ff73806c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:01.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.266+0000 7ff73eb26640 1 --2- 172.21.15.38:0/901642888 >> v2:172.21.15.107:3300/0 conn(0x7ff7380ce2e0 0x7ff7380d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:01.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.267+0000 7ff73db24640 1 --2- 172.21.15.38:0/901642888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff73815df50 0x7ff7380ce7a0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7ff7380b6480 tx=0x7ff72802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16abd2b47ab68c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:01.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.267+0000 7ff73d323640 1 -- 172.21.15.38:0/901642888 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728037070 con 0x7ff73815df50 2024-04-25T08:39:01.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.267+0000 7ff73d323640 1 -- 172.21.15.38:0/901642888 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff728032440 con 0x7ff73815df50 2024-04-25T08:39:01.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.267+0000 7ff73d323640 1 -- 172.21.15.38:0/901642888 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff728032810 con 0x7ff73815df50 2024-04-25T08:39:01.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.268+0000 7ff740589640 1 -- 172.21.15.38:0/901642888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff73815df50 msgr2=0x7ff7380ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.268+0000 7ff740589640 1 --2- 172.21.15.38:0/901642888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff73815df50 0x7ff7380ce7a0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7ff7380b6480 tx=0x7ff72802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.268+0000 7ff740589640 1 -- 172.21.15.38:0/901642888 shutdown_connections 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.268+0000 7ff740589640 1 --2- 172.21.15.38:0/901642888 >> v2:172.21.15.107:3300/0 conn(0x7ff7380ce2e0 0x7ff7380d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.268+0000 7ff740589640 1 --2- 172.21.15.38:0/901642888 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff73815df50 0x7ff7380ce7a0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.268+0000 7ff740589640 1 --2- 172.21.15.38:0/901642888 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff73806c110 0x7ff73806c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.268+0000 7ff740589640 1 -- 172.21.15.38:0/901642888 >> 172.21.15.38:0/901642888 conn(0x7ff7380d3fb0 msgr2=0x7ff7380d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.268+0000 7ff740589640 1 -- 172.21.15.38:0/901642888 shutdown_connections 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.268+0000 7ff740589640 1 -- 172.21.15.38:0/901642888 wait complete. 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.269+0000 7ff740589640 1 Processor -- start 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.269+0000 7ff740589640 1 -- start start 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff740589640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff73806c110 0x7ff7380c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff740589640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7380ce2e0 0x7ff7380b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff740589640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff73815df50 0x7ff738154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff740589640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380cece0 con 0x7ff73815df50 2024-04-25T08:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff740589640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380d2530 con 0x7ff73806c110 2024-04-25T08:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff740589640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380d2f90 con 0x7ff7380ce2e0 2024-04-25T08:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff73eb26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff73815df50 0x7ff738154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff73eb26640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff73815df50 0x7ff738154de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53608/0 (socket says 172.21.15.38:53608) 2024-04-25T08:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff73eb26640 1 -- 172.21.15.38:0/2869139177 learned_addr learned my addr 172.21.15.38:0/2869139177 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff73e325640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff73806c110 0x7ff7380c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff73db24640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7380ce2e0 0x7ff7380b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff73eb26640 1 -- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7380ce2e0 msgr2=0x7ff7380b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff73eb26640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7380ce2e0 0x7ff7380b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff73eb26640 1 -- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff73806c110 msgr2=0x7ff7380c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff73eb26640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff73806c110 0x7ff7380c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff73eb26640 1 -- 172.21.15.38:0/2869139177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73815ed70 con 0x7ff73815df50 2024-04-25T08:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff73eb26640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff73815df50 0x7ff738154de0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7ff73000cea0 tx=0x7ff73000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.270+0000 7ff72effd640 1 -- 172.21.15.38:0/2869139177 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730003b90 con 0x7ff73815df50 2024-04-25T08:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.271+0000 7ff72effd640 1 -- 172.21.15.38:0/2869139177 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff730003d50 con 0x7ff73815df50 2024-04-25T08:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.271+0000 7ff72effd640 1 -- 172.21.15.38:0/2869139177 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730004260 con 0x7ff73815df50 2024-04-25T08:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.271+0000 7ff740589640 1 -- 172.21.15.38:0/2869139177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7380c7c50 con 0x7ff73815df50 2024-04-25T08:39:01.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.271+0000 7ff740589640 1 -- 172.21.15.38:0/2869139177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7380c8210 con 0x7ff73815df50 2024-04-25T08:39:01.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.272+0000 7ff740589640 1 -- 172.21.15.38:0/2869139177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff738068e80 con 0x7ff73815df50 2024-04-25T08:39:01.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.272+0000 7ff72effd640 1 -- 172.21.15.38:0/2869139177 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff730004400 con 0x7ff73815df50 2024-04-25T08:39:01.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.272+0000 7ff72effd640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff71c06b4d0 0x7ff71c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.273+0000 7ff72effd640 1 -- 172.21.15.38:0/2869139177 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff73008a9f0 con 0x7ff73815df50 2024-04-25T08:39:01.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.273+0000 7ff73e325640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff71c06b4d0 0x7ff71c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.273+0000 7ff73e325640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff71c06b4d0 0x7ff71c06d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff734006b10 tx=0x7ff734008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:01.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.277+0000 7ff72effd640 1 -- 172.21.15.38:0/2869139177 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff730059390 con 0x7ff73815df50 2024-04-25T08:39:01.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.457+0000 7ff740589640 1 -- 172.21.15.38:0/2869139177 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7380c7960 con 0x7ff73815df50 2024-04-25T08:39:01.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.458+0000 7ff72effd640 1 -- 172.21.15.38:0/2869139177 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+13282 (secure 0 0 0) 0x7ff73005d430 con 0x7ff73815df50 2024-04-25T08:39:01.452 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:01.452 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":99,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:00.139246+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/2922734840":"2024-04-26T08:38:57.119778+0000","172.21.15.38:6833/2922734840":"2024-04-26T08:38:57.119778+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:01.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.460+0000 7ff740589640 1 -- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff71c06b4d0 msgr2=0x7ff71c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.460+0000 7ff740589640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff71c06b4d0 0x7ff71c06d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff734006b10 tx=0x7ff734008040 comp rx=0 tx=0).stop 2024-04-25T08:39:01.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.460+0000 7ff740589640 1 -- 172.21.15.38:0/2869139177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff73815df50 msgr2=0x7ff738154de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.460+0000 7ff740589640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff73815df50 0x7ff738154de0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7ff73000cea0 tx=0x7ff73000cf90 comp rx=0 tx=0).stop 2024-04-25T08:39:01.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.460+0000 7ff740589640 1 -- 172.21.15.38:0/2869139177 shutdown_connections 2024-04-25T08:39:01.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.460+0000 7ff740589640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff73815df50 0x7ff738154de0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.460+0000 7ff740589640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff71c06b4d0 0x7ff71c06d990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.460+0000 7ff740589640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7380ce2e0 0x7ff7380b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.460+0000 7ff740589640 1 --2- 172.21.15.38:0/2869139177 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff73806c110 0x7ff7380c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.460+0000 7ff740589640 1 -- 172.21.15.38:0/2869139177 >> 172.21.15.38:0/2869139177 conn(0x7ff7380d3fb0 msgr2=0x7ff7380b7b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:01.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.460+0000 7ff740589640 1 -- 172.21.15.38:0/2869139177 shutdown_connections 2024-04-25T08:39:01.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.460+0000 7ff740589640 1 -- 172.21.15.38:0/2869139177 wait complete. 2024-04-25T08:39:01.466 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:39:01.467 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:39:01.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.577+0000 7ff52517d640 1 Processor -- start 2024-04-25T08:39:01.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.577+0000 7ff52517d640 1 -- start start 2024-04-25T08:39:01.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.578+0000 7ff52517d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52006c700 0x7ff520162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.578+0000 7ff52517d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff52015e750 0x7ff5200cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.578+0000 7ff52517d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52015ec20 con 0x7ff52006cae0 2024-04-25T08:39:01.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.578+0000 7ff52517d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff520161c80 con 0x7ff52015e750 2024-04-25T08:39:01.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.578+0000 7ff52517d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52015ef00 con 0x7ff52006c700 2024-04-25T08:39:01.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.578+0000 7ff51f59e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff52015e750 0x7ff5200cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.578+0000 7ff51f59e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff52015e750 0x7ff5200cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53616/0 (socket says 172.21.15.38:53616) 2024-04-25T08:39:01.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.578+0000 7ff51f59e640 1 -- 172.21.15.38:0/309097087 learned_addr learned my addr 172.21.15.38:0/309097087 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:01.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.579+0000 7ff51ed9d640 1 --2- 172.21.15.38:0/309097087 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52006c700 0x7ff520162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.579+0000 7ff51f59e640 1 -- 172.21.15.38:0/309097087 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52006c700 msgr2=0x7ff520162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:01.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.579+0000 7ff51f59e640 1 --2- 172.21.15.38:0/309097087 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52006c700 0x7ff520162110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.579+0000 7ff51f59e640 1 -- 172.21.15.38:0/309097087 >> v1:172.21.15.107:6789/0 conn(0x7ff52006cae0 legacy=0x7ff52015e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:01.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.579+0000 7ff51f59e640 1 -- 172.21.15.38:0/309097087 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff520161630 con 0x7ff52015e750 2024-04-25T08:39:01.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.579+0000 7ff51f59e640 1 --2- 172.21.15.38:0/309097087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52015e750 0x7ff5200cdf90 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7ff510009e50 tx=0x7ff51002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8740d515c5c9eefe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:01.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.579+0000 7ff51dd9b640 1 -- 172.21.15.38:0/309097087 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510036070 con 0x7ff52015e750 2024-04-25T08:39:01.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.580+0000 7ff51dd9b640 1 -- 172.21.15.38:0/309097087 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff510031600 con 0x7ff52015e750 2024-04-25T08:39:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.580+0000 7ff51dd9b640 1 -- 172.21.15.38:0/309097087 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100319d0 con 0x7ff52015e750 2024-04-25T08:39:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.580+0000 7ff52517d640 1 -- 172.21.15.38:0/309097087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52015e750 msgr2=0x7ff5200cdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.580+0000 7ff52517d640 1 --2- 172.21.15.38:0/309097087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52015e750 0x7ff5200cdf90 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7ff510009e50 tx=0x7ff51002a6c0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.580+0000 7ff52517d640 1 -- 172.21.15.38:0/309097087 shutdown_connections 2024-04-25T08:39:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.580+0000 7ff52517d640 1 --2- 172.21.15.38:0/309097087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52015e750 0x7ff5200cdf90 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.580+0000 7ff52517d640 1 --2- 172.21.15.38:0/309097087 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52006c700 0x7ff520162110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.580+0000 7ff52517d640 1 -- 172.21.15.38:0/309097087 >> 172.21.15.38:0/309097087 conn(0x7ff52008b000 msgr2=0x7ff52008b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.580+0000 7ff52517d640 1 -- 172.21.15.38:0/309097087 shutdown_connections 2024-04-25T08:39:01.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.580+0000 7ff52517d640 1 -- 172.21.15.38:0/309097087 wait complete. 2024-04-25T08:39:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.582+0000 7ff52517d640 1 Processor -- start 2024-04-25T08:39:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.582+0000 7ff52517d640 1 -- start start 2024-04-25T08:39:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.582+0000 7ff52517d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52006c700 0x7ff5200b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.582+0000 7ff52517d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52006cae0 0x7ff520160740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.582+0000 7ff52517d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52015e750 0x7ff520160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.582+0000 7ff52517d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52006cf60 con 0x7ff52015e750 2024-04-25T08:39:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.582+0000 7ff52517d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52006c190 con 0x7ff52006cae0 2024-04-25T08:39:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.582+0000 7ff52517d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52015b920 con 0x7ff52006c700 2024-04-25T08:39:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.582+0000 7ff51f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52015e750 0x7ff520160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff51f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52015e750 0x7ff520160c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53624/0 (socket says 172.21.15.38:53624) 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff51f59e640 1 -- 172.21.15.38:0/3223309773 learned_addr learned my addr 172.21.15.38:0/3223309773 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff51ed9d640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52006c700 0x7ff5200b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff51e59c640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52006cae0 0x7ff520160740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff51f59e640 1 -- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52006c700 msgr2=0x7ff5200b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff51f59e640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52006c700 0x7ff5200b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff51f59e640 1 -- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52006cae0 msgr2=0x7ff520160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff51f59e640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52006cae0 0x7ff520160740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff51f59e640 1 -- 172.21.15.38:0/3223309773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff520157270 con 0x7ff52015e750 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff51e59c640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52006cae0 0x7ff520160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff51f59e640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52015e750 0x7ff520160c80 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ff510034040 tx=0x7ff510004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff5177fe640 1 -- 172.21.15.38:0/3223309773 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510041020 con 0x7ff52015e750 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff5177fe640 1 -- 172.21.15.38:0/3223309773 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff510007780 con 0x7ff52015e750 2024-04-25T08:39:01.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff52517d640 1 -- 172.21.15.38:0/3223309773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff520148930 con 0x7ff52015e750 2024-04-25T08:39:01.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff5177fe640 1 -- 172.21.15.38:0/3223309773 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510036040 con 0x7ff52015e750 2024-04-25T08:39:01.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.583+0000 7ff52517d640 1 -- 172.21.15.38:0/3223309773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff520148ea0 con 0x7ff52015e750 2024-04-25T08:39:01.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.584+0000 7ff52517d640 1 -- 172.21.15.38:0/3223309773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff520068e80 con 0x7ff52015e750 2024-04-25T08:39:01.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.585+0000 7ff5177fe640 1 -- 172.21.15.38:0/3223309773 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff510044020 con 0x7ff52015e750 2024-04-25T08:39:01.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.585+0000 7ff5177fe640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50006b4d0 0x7ff50006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.585+0000 7ff5177fe640 1 -- 172.21.15.38:0/3223309773 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff5100aac80 con 0x7ff52015e750 2024-04-25T08:39:01.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.585+0000 7ff51ed9d640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50006b4d0 0x7ff50006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.586+0000 7ff51ed9d640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50006b4d0 0x7ff50006d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff518006fd0 tx=0x7ff518008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:01.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.589+0000 7ff5177fe640 1 -- 172.21.15.38:0/3223309773 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff51004b030 con 0x7ff52015e750 2024-04-25T08:39:01.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.767+0000 7ff52517d640 1 -- 172.21.15.38:0/3223309773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5200c7f00 con 0x7ff52015e750 2024-04-25T08:39:01.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.768+0000 7ff5177fe640 1 -- 172.21.15.38:0/3223309773 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff520157270 con 0x7ff52015e750 2024-04-25T08:39:01.761 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:39:01.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.770+0000 7ff52517d640 1 -- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50006b4d0 msgr2=0x7ff50006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.770+0000 7ff52517d640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50006b4d0 0x7ff50006d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff518006fd0 tx=0x7ff518008040 comp rx=0 tx=0).stop 2024-04-25T08:39:01.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.770+0000 7ff52517d640 1 -- 172.21.15.38:0/3223309773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52015e750 msgr2=0x7ff520160c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.770+0000 7ff52517d640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52015e750 0x7ff520160c80 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ff510034040 tx=0x7ff510004a40 comp rx=0 tx=0).stop 2024-04-25T08:39:01.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.770+0000 7ff52517d640 1 -- 172.21.15.38:0/3223309773 shutdown_connections 2024-04-25T08:39:01.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.770+0000 7ff52517d640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52015e750 0x7ff520160c80 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.770+0000 7ff52517d640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52006cae0 0x7ff520160740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.770+0000 7ff52517d640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50006b4d0 0x7ff50006d990 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.770+0000 7ff52517d640 1 --2- 172.21.15.38:0/3223309773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52006c700 0x7ff5200b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.770+0000 7ff52517d640 1 -- 172.21.15.38:0/3223309773 >> 172.21.15.38:0/3223309773 conn(0x7ff52008b000 msgr2=0x7ff5200d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:01.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.770+0000 7ff52517d640 1 -- 172.21.15.38:0/3223309773 shutdown_connections 2024-04-25T08:39:01.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.770+0000 7ff52517d640 1 -- 172.21.15.38:0/3223309773 wait complete. 2024-04-25T08:39:01.775 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T08:39:01.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.882+0000 7fa1fc23a640 1 Processor -- start 2024-04-25T08:39:01.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.882+0000 7fa1fc23a640 1 -- start start 2024-04-25T08:39:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.883+0000 7fa1fc23a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa1f40d3900 0x7fa1f40d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.883+0000 7fa1fc23a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f40d3e70 con 0x7fa1f415dc10 2024-04-25T08:39:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.883+0000 7fa1fc23a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f40d4150 con 0x7fa1f40d0d90 2024-04-25T08:39:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.883+0000 7fa1fc23a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f415dff0 con 0x7fa1f40d3900 2024-04-25T08:39:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.883+0000 7fa1f9fd6640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa1f415dc10 0x7fa1f40ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37032/0 (socket says 172.21.15.38:37032) 2024-04-25T08:39:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.883+0000 7fa1f9fd6640 1 -- 172.21.15.38:0/126552479 learned_addr learned my addr 172.21.15.38:0/126552479 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.883+0000 7fa1f97d5640 1 --2- 172.21.15.38:0/126552479 >> v2:172.21.15.107:3300/0 conn(0x7fa1f40d3900 0x7fa1f40d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.883+0000 7fa1f8fd4640 1 -- 172.21.15.38:0/126552479 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365678789 0 0) 0x7fa1f40d3e70 con 0x7fa1f415dc10 2024-04-25T08:39:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.884+0000 7fa1f8fd4640 1 -- 172.21.15.38:0/126552479 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1e0003520 con 0x7fa1f415dc10 2024-04-25T08:39:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.884+0000 7fa1f97d5640 1 -- 172.21.15.38:0/126552479 >> v1:172.21.15.107:6789/0 conn(0x7fa1f40d0d90 legacy=0x7fa1f40b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.884+0000 7fa1f97d5640 1 -- 172.21.15.38:0/126552479 >> v1:172.21.15.38:6789/0 conn(0x7fa1f415dc10 legacy=0x7fa1f40ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.884+0000 7fa1f97d5640 1 -- 172.21.15.38:0/126552479 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f40d4430 con 0x7fa1f40d3900 2024-04-25T08:39:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.884+0000 7fa1f97d5640 1 --2- 172.21.15.38:0/126552479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1f40d3900 0x7fa1f40d0850 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40097a0 tx=0x7fa1e402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c080afc17f26ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.885+0000 7fa1f8fd4640 1 -- 172.21.15.38:0/126552479 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4037070 con 0x7fa1f40d3900 2024-04-25T08:39:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.885+0000 7fa1f8fd4640 1 -- 172.21.15.38:0/126552479 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e4032440 con 0x7fa1f40d3900 2024-04-25T08:39:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.885+0000 7fa1f8fd4640 1 -- 172.21.15.38:0/126552479 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40327f0 con 0x7fa1f40d3900 2024-04-25T08:39:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.885+0000 7fa1fc23a640 1 -- 172.21.15.38:0/126552479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1f40d3900 msgr2=0x7fa1f40d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.885+0000 7fa1fc23a640 1 --2- 172.21.15.38:0/126552479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1f40d3900 0x7fa1f40d0850 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40097a0 tx=0x7fa1e402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.885+0000 7fa1fc23a640 1 -- 172.21.15.38:0/126552479 shutdown_connections 2024-04-25T08:39:01.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.885+0000 7fa1fc23a640 1 --2- 172.21.15.38:0/126552479 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1f40d3900 0x7fa1f40d0850 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.885+0000 7fa1fc23a640 1 -- 172.21.15.38:0/126552479 >> 172.21.15.38:0/126552479 conn(0x7fa1f4066d10 msgr2=0x7fa1f4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:01.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.885+0000 7fa1fc23a640 1 -- 172.21.15.38:0/126552479 shutdown_connections 2024-04-25T08:39:01.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.885+0000 7fa1fc23a640 1 -- 172.21.15.38:0/126552479 wait complete. 2024-04-25T08:39:01.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.886+0000 7fa1fc23a640 1 Processor -- start 2024-04-25T08:39:01.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.886+0000 7fa1fc23a640 1 -- start start 2024-04-25T08:39:01.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.887+0000 7fa1fc23a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1f40d0d90 0x7fa1f40c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.887+0000 7fa1fc23a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1f40d3900 0x7fa1f41f68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.887+0000 7fa1fc23a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1f415dc10 0x7fa1f40aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.887+0000 7fa1fc23a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40d11e0 con 0x7fa1f40d3900 2024-04-25T08:39:01.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.887+0000 7fa1fc23a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40d4710 con 0x7fa1f40d0d90 2024-04-25T08:39:01.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.887+0000 7fa1fc23a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40cef30 con 0x7fa1f415dc10 2024-04-25T08:39:01.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.887+0000 7fa1f97d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1f40d3900 0x7fa1f41f68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.887+0000 7fa1f97d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1f40d3900 0x7fa1f41f68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53628/0 (socket says 172.21.15.38:53628) 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.887+0000 7fa1f97d5640 1 -- 172.21.15.38:0/355945926 learned_addr learned my addr 172.21.15.38:0/355945926 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.888+0000 7fa1f9fd6640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1f40d0d90 0x7fa1f40c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.887+0000 7fa1fa7d7640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1f415dc10 0x7fa1f40aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.888+0000 7fa1f97d5640 1 -- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1f415dc10 msgr2=0x7fa1f40aa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.888+0000 7fa1f97d5640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1f415dc10 0x7fa1f40aa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.888+0000 7fa1f97d5640 1 -- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1f40d0d90 msgr2=0x7fa1f40c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.888+0000 7fa1f97d5640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1f40d0d90 0x7fa1f40c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.888+0000 7fa1f97d5640 1 -- 172.21.15.38:0/355945926 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f41f6de0 con 0x7fa1f40d3900 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.888+0000 7fa1f97d5640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1f40d3900 0x7fa1f41f68a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40097a0 tx=0x7fa1e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.888+0000 7fa1ea7fc640 1 -- 172.21.15.38:0/355945926 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4037050 con 0x7fa1f40d3900 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.888+0000 7fa1ea7fc640 1 -- 172.21.15.38:0/355945926 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e4038040 con 0x7fa1f40d3900 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.888+0000 7fa1ea7fc640 1 -- 172.21.15.38:0/355945926 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e4004490 con 0x7fa1f40d3900 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.888+0000 7fa1fc23a640 1 -- 172.21.15.38:0/355945926 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f40aa830 con 0x7fa1f40d3900 2024-04-25T08:39:01.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.888+0000 7fa1fc23a640 1 -- 172.21.15.38:0/355945926 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f40aade0 con 0x7fa1f40d3900 2024-04-25T08:39:01.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.889+0000 7fa1fc23a640 1 -- 172.21.15.38:0/355945926 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f4068e80 con 0x7fa1f40d3900 2024-04-25T08:39:01.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.890+0000 7fa1ea7fc640 1 -- 172.21.15.38:0/355945926 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1e403c6d0 con 0x7fa1f40d3900 2024-04-25T08:39:01.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.890+0000 7fa1ea7fc640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa1d806b4d0 0x7fa1d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:01.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.890+0000 7fa1ea7fc640 1 -- 172.21.15.38:0/355945926 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa1e40aab20 con 0x7fa1f40d3900 2024-04-25T08:39:01.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.890+0000 7fa1f9fd6640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa1d806b4d0 0x7fa1d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:01.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.891+0000 7fa1f9fd6640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa1d806b4d0 0x7fa1d806d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00011e0 tx=0x7fa1f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:01.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:01.895+0000 7fa1ea7fc640 1 -- 172.21.15.38:0/355945926 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1e4079b10 con 0x7fa1f40d3900 2024-04-25T08:39:02.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.143+0000 7fa1fc23a640 1 -- 172.21.15.38:0/355945926 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7fa1f40d0b20 con 0x7fa1f40d3900 2024-04-25T08:39:02.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.155+0000 7fa1ea7fc640 1 -- 172.21.15.38:0/355945926 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v359) v1 ==== 158+0+0 (secure 0 0 0) 0x7fa1e4079410 con 0x7fa1f40d3900 2024-04-25T08:39:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.157+0000 7fa1fc23a640 1 -- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa1d806b4d0 msgr2=0x7fa1d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.157+0000 7fa1fc23a640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa1d806b4d0 0x7fa1d806d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00011e0 tx=0x7fa1f0009040 comp rx=0 tx=0).stop 2024-04-25T08:39:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.157+0000 7fa1fc23a640 1 -- 172.21.15.38:0/355945926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1f40d3900 msgr2=0x7fa1f41f68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.157+0000 7fa1fc23a640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1f40d3900 0x7fa1f41f68a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40097a0 tx=0x7fa1e40046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.157+0000 7fa1fc23a640 1 -- 172.21.15.38:0/355945926 shutdown_connections 2024-04-25T08:39:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.157+0000 7fa1fc23a640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa1d806b4d0 0x7fa1d806d990 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.157+0000 7fa1fc23a640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1f415dc10 0x7fa1f40aa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.157+0000 7fa1fc23a640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1f40d3900 0x7fa1f41f68a0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.157+0000 7fa1fc23a640 1 --2- 172.21.15.38:0/355945926 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1f40d0d90 0x7fa1f40c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.157+0000 7fa1fc23a640 1 -- 172.21.15.38:0/355945926 >> 172.21.15.38:0/355945926 conn(0x7fa1f4066d10 msgr2=0x7fa1f40d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.157+0000 7fa1fc23a640 1 -- 172.21.15.38:0/355945926 shutdown_connections 2024-04-25T08:39:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.157+0000 7fa1fc23a640 1 -- 172.21.15.38:0/355945926 wait complete. 2024-04-25T08:39:02.161 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:39:02.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.272+0000 7f961a638640 1 Processor -- start 2024-04-25T08:39:02.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.272+0000 7f961a638640 1 -- start start 2024-04-25T08:39:02.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.272+0000 7f961a638640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f96140cf580 0x7f961415f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.272+0000 7f961a638640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96140cf9f0 con 0x7f96140d1d20 2024-04-25T08:39:02.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.272+0000 7f961a638640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961415fd00 con 0x7f961415b990 2024-04-25T08:39:02.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.272+0000 7f961a638640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9614160580 con 0x7f96140cf580 2024-04-25T08:39:02.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.272+0000 7f9618bd5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f96140cf580 0x7f961415f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.273+0000 7f9618bd5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f96140cf580 0x7f961415f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53644/0 (socket says 172.21.15.38:53644) 2024-04-25T08:39:02.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.273+0000 7f96137fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f96140d1d20 0x7f96140b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37048/0 (socket says 172.21.15.38:37048) 2024-04-25T08:39:02.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.273+0000 7f9618bd5640 1 -- 172.21.15.38:0/114814666 learned_addr learned my addr 172.21.15.38:0/114814666 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:02.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.273+0000 7f9618bd5640 1 -- 172.21.15.38:0/114814666 >> v1:172.21.15.107:6789/0 conn(0x7f961415b990 legacy=0x7f96140d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:02.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.273+0000 7f9612ffd640 1 -- 172.21.15.38:0/114814666 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446360224 0 0) 0x7f96140cf9f0 con 0x7f96140d1d20 2024-04-25T08:39:02.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.273+0000 7f9618bd5640 1 -- 172.21.15.38:0/114814666 >> v1:172.21.15.38:6789/0 conn(0x7f96140d1d20 legacy=0x7f96140b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.273+0000 7f9618bd5640 1 -- 172.21.15.38:0/114814666 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961415ffc0 con 0x7f96140cf580 2024-04-25T08:39:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.274+0000 7f9618bd5640 1 --2- 172.21.15.38:0/114814666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96140cf580 0x7f961415f5f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f960c009e50 tx=0x7f960c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c82467607e9729 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.274+0000 7f9612ffd640 1 -- 172.21.15.38:0/114814666 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c036070 con 0x7f96140cf580 2024-04-25T08:39:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.274+0000 7f9612ffd640 1 -- 172.21.15.38:0/114814666 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c031600 con 0x7f96140cf580 2024-04-25T08:39:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.274+0000 7f9612ffd640 1 -- 172.21.15.38:0/114814666 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c0319d0 con 0x7f96140cf580 2024-04-25T08:39:02.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.274+0000 7f961a638640 1 -- 172.21.15.38:0/114814666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96140cf580 msgr2=0x7f961415f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.274+0000 7f961a638640 1 --2- 172.21.15.38:0/114814666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96140cf580 0x7f961415f5f0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f960c009e50 tx=0x7f960c02a730 comp rx=0 tx=0).stop 2024-04-25T08:39:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.274+0000 7f961a638640 1 -- 172.21.15.38:0/114814666 shutdown_connections 2024-04-25T08:39:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.274+0000 7f961a638640 1 --2- 172.21.15.38:0/114814666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96140cf580 0x7f961415f5f0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.274+0000 7f961a638640 1 -- 172.21.15.38:0/114814666 >> 172.21.15.38:0/114814666 conn(0x7f961408b000 msgr2=0x7f961408b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.274+0000 7f961a638640 1 -- 172.21.15.38:0/114814666 shutdown_connections 2024-04-25T08:39:02.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.274+0000 7f961a638640 1 -- 172.21.15.38:0/114814666 wait complete. 2024-04-25T08:39:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.275+0000 7f961a638640 1 Processor -- start 2024-04-25T08:39:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.276+0000 7f961a638640 1 -- start start 2024-04-25T08:39:02.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.276+0000 7f961a638640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140cf580 0x7f9614158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.276+0000 7f961a638640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96140d1d20 0x7f961415c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.276+0000 7f961a638640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f961415b990 0x7f96141f0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.276+0000 7f961a638640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961415bd70 con 0x7f961415b990 2024-04-25T08:39:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.276+0000 7f961a638640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96140b1ac0 con 0x7f96140cf580 2024-04-25T08:39:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.276+0000 7f961a638640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961415bef0 con 0x7f96140d1d20 2024-04-25T08:39:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9618bd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f961415b990 0x7f96141f0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9618bd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f961415b990 0x7f96141f0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53652/0 (socket says 172.21.15.38:53652) 2024-04-25T08:39:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9618bd5640 1 -- 172.21.15.38:0/1970570760 learned_addr learned my addr 172.21.15.38:0/1970570760 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f96137fe640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96140d1d20 0x7f961415c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9613fff640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140cf580 0x7f9614158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9618bd5640 1 -- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96140d1d20 msgr2=0x7f961415c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9618bd5640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96140d1d20 0x7f961415c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9618bd5640 1 -- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140cf580 msgr2=0x7f9614158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9618bd5640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140cf580 0x7f9614158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9618bd5640 1 -- 172.21.15.38:0/1970570760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9614158cc0 con 0x7f961415b990 2024-04-25T08:39:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f96137fe640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96140d1d20 0x7f961415c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9613fff640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140cf580 0x7f9614158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9618bd5640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f961415b990 0x7f96141f0aa0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f960c009f70 tx=0x7f960c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9610ff9640 1 -- 172.21.15.38:0/1970570760 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c036050 con 0x7f961415b990 2024-04-25T08:39:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9610ff9640 1 -- 172.21.15.38:0/1970570760 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960c042070 con 0x7f961415b990 2024-04-25T08:39:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f9610ff9640 1 -- 172.21.15.38:0/1970570760 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960c009510 con 0x7f961415b990 2024-04-25T08:39:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f961a638640 1 -- 172.21.15.38:0/1970570760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961415e120 con 0x7f961415b990 2024-04-25T08:39:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.277+0000 7f961a638640 1 -- 172.21.15.38:0/1970570760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961415e6e0 con 0x7f961415b990 2024-04-25T08:39:02.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.278+0000 7f961a638640 1 -- 172.21.15.38:0/1970570760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9614068e80 con 0x7f961415b990 2024-04-25T08:39:02.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.279+0000 7f9610ff9640 1 -- 172.21.15.38:0/1970570760 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f960c007bf0 con 0x7f961415b990 2024-04-25T08:39:02.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.279+0000 7f9610ff9640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f406b4d0 0x7f95f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.279+0000 7f9610ff9640 1 -- 172.21.15.38:0/1970570760 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f960c0aafb0 con 0x7f961415b990 2024-04-25T08:39:02.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.279+0000 7f9613fff640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f406b4d0 0x7f95f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.280+0000 7f9613fff640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f406b4d0 0x7f95f406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0015e0 tx=0x7f95fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:02.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.283+0000 7f9610ff9640 1 -- 172.21.15.38:0/1970570760 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f960c079fa0 con 0x7f961415b990 2024-04-25T08:39:02.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.462+0000 7f961a638640 1 -- 172.21.15.38:0/1970570760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f96140b1220 con 0x7f961415b990 2024-04-25T08:39:02.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.465+0000 7f9610ff9640 1 -- 172.21.15.38:0/1970570760 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f960c0798a0 con 0x7f961415b990 2024-04-25T08:39:02.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.467+0000 7f961a638640 1 -- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f406b4d0 msgr2=0x7f95f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.467+0000 7f961a638640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f406b4d0 0x7f95f406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0015e0 tx=0x7f95fc009040 comp rx=0 tx=0).stop 2024-04-25T08:39:02.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.467+0000 7f961a638640 1 -- 172.21.15.38:0/1970570760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f961415b990 msgr2=0x7f96141f0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.467+0000 7f961a638640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f961415b990 0x7f96141f0aa0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f960c009f70 tx=0x7f960c004a40 comp rx=0 tx=0).stop 2024-04-25T08:39:02.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.467+0000 7f961a638640 1 -- 172.21.15.38:0/1970570760 shutdown_connections 2024-04-25T08:39:02.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.467+0000 7f961a638640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f961415b990 0x7f96141f0aa0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.467+0000 7f961a638640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f406b4d0 0x7f95f406d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.467+0000 7f961a638640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f96140d1d20 0x7f961415c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.467+0000 7f961a638640 1 --2- 172.21.15.38:0/1970570760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f96140cf580 0x7f9614158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.467+0000 7f961a638640 1 -- 172.21.15.38:0/1970570760 >> 172.21.15.38:0/1970570760 conn(0x7f961408b000 msgr2=0x7f961408b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:02.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.467+0000 7f961a638640 1 -- 172.21.15.38:0/1970570760 shutdown_connections 2024-04-25T08:39:02.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.467+0000 7f961a638640 1 -- 172.21.15.38:0/1970570760 wait complete. 2024-04-25T08:39:02.472 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:39:02.473 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:39:02.508 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:39:02.551 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:39:02.551 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:39:02.551 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:02.567 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:39:02.567 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:39:02.567 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:02.609 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:02.609 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:39:02.609 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:02.624 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:39:02.624 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:39:02.624 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:02.667 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:02.667 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:39:02.667 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:02.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.775+0000 7fad5db96640 1 Processor -- start 2024-04-25T08:39:02.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.775+0000 7fad5db96640 1 -- start start 2024-04-25T08:39:02.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.776+0000 7fad5db96640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fad5809acb0 0x7fad580cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.776+0000 7fad5db96640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad5806cdf0 0x7fad580cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.776+0000 7fad5db96640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad58161540 con 0x7fad580cd8d0 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.776+0000 7fad5db96640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5809a820 con 0x7fad5809acb0 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.776+0000 7fad5db96640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5806be90 con 0x7fad5806cdf0 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.776+0000 7fad57fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad5806cdf0 0x7fad580cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.776+0000 7fad56ffd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fad5809acb0 0x7fad580cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.776+0000 7fad57fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad5806cdf0 0x7fad580cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:52174/0 (socket says 172.21.15.38:52174) 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.776+0000 7fad56ffd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fad5809acb0 0x7fad580cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:54058/0 (socket says 172.21.15.38:54058) 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.776+0000 7fad57fff640 1 -- 172.21.15.38:0/298659463 learned_addr learned my addr 172.21.15.38:0/298659463 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.777+0000 7fad567fc640 1 -- 172.21.15.38:0/298659463 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183369622 0 0) 0x7fad58161540 con 0x7fad580cd8d0 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.777+0000 7fad567fc640 1 -- 172.21.15.38:0/298659463 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad40003520 con 0x7fad580cd8d0 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.777+0000 7fad56ffd640 1 -- 172.21.15.38:0/298659463 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad5806cdf0 msgr2=0x7fad580cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.777+0000 7fad56ffd640 1 --2- 172.21.15.38:0/298659463 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad5806cdf0 0x7fad580cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.777+0000 7fad56ffd640 1 -- 172.21.15.38:0/298659463 >> v1:172.21.15.107:6789/0 conn(0x7fad580cd8d0 legacy=0x7fad5806c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.777+0000 7fad56ffd640 1 -- 172.21.15.38:0/298659463 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5809a520 con 0x7fad5809acb0 2024-04-25T08:39:02.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.777+0000 7fad57fff640 1 --2- 172.21.15.38:0/298659463 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad5806cdf0 0x7fad580cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:02.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.778+0000 7fad56ffd640 1 --2- 172.21.15.38:0/298659463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad5809acb0 0x7fad580cbcc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0097a0 tx=0x7fad4c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f72c46611af041b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:02.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.778+0000 7fad567fc640 1 -- 172.21.15.38:0/298659463 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c037070 con 0x7fad5809acb0 2024-04-25T08:39:02.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.778+0000 7fad567fc640 1 -- 172.21.15.38:0/298659463 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4c032440 con 0x7fad5809acb0 2024-04-25T08:39:02.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.778+0000 7fad567fc640 1 -- 172.21.15.38:0/298659463 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c032810 con 0x7fad5809acb0 2024-04-25T08:39:02.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.778+0000 7fad5db96640 1 -- 172.21.15.38:0/298659463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad5809acb0 msgr2=0x7fad580cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.778+0000 7fad5db96640 1 --2- 172.21.15.38:0/298659463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad5809acb0 0x7fad580cbcc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fad4c0097a0 tx=0x7fad4c02a180 comp rx=0 tx=0).stop 2024-04-25T08:39:02.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.778+0000 7fad5db96640 1 -- 172.21.15.38:0/298659463 shutdown_connections 2024-04-25T08:39:02.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.778+0000 7fad5db96640 1 --2- 172.21.15.38:0/298659463 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad5806cdf0 0x7fad580cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.778+0000 7fad5db96640 1 --2- 172.21.15.38:0/298659463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad5809acb0 0x7fad580cbcc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.778+0000 7fad5db96640 1 -- 172.21.15.38:0/298659463 >> 172.21.15.38:0/298659463 conn(0x7fad580b3160 msgr2=0x7fad580b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:02.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.778+0000 7fad5db96640 1 -- 172.21.15.38:0/298659463 shutdown_connections 2024-04-25T08:39:02.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.778+0000 7fad5db96640 1 -- 172.21.15.38:0/298659463 wait complete. 2024-04-25T08:39:02.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.780+0000 7fad5db96640 1 Processor -- start 2024-04-25T08:39:02.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.780+0000 7fad5db96640 1 -- start start 2024-04-25T08:39:02.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.780+0000 7fad5db96640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad5806cdf0 0x7fad5815fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.780+0000 7fad5db96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad5809acb0 0x7fad580aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.780+0000 7fad5db96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad580cd8d0 0x7fad58156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad5db96640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5806c410 con 0x7fad580cd8d0 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad5db96640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580d0ad0 con 0x7fad5809acb0 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad5db96640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad580b2170 con 0x7fad5806cdf0 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad57fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad580cd8d0 0x7fad58156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad57fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad580cd8d0 0x7fad58156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53664/0 (socket says 172.21.15.38:53664) 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad57fff640 1 -- 172.21.15.38:0/2026884010 learned_addr learned my addr 172.21.15.38:0/2026884010 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad577fe640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad5806cdf0 0x7fad5815fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad56ffd640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad5809acb0 0x7fad580aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad57fff640 1 -- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad5806cdf0 msgr2=0x7fad5815fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad57fff640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad5806cdf0 0x7fad5815fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad57fff640 1 -- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad5809acb0 msgr2=0x7fad580aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad57fff640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad5809acb0 0x7fad580aeee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:02.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad57fff640 1 -- 172.21.15.38:0/2026884010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad580b5900 con 0x7fad580cd8d0 2024-04-25T08:39:02.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad57fff640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad580cd8d0 0x7fad58156b40 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fad5000ef90 tx=0x7fad5000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:02.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad3ffff640 1 -- 172.21.15.38:0/2026884010 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5000cd80 con 0x7fad580cd8d0 2024-04-25T08:39:02.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad3ffff640 1 -- 172.21.15.38:0/2026884010 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad50003990 con 0x7fad580cd8d0 2024-04-25T08:39:02.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad5db96640 1 -- 172.21.15.38:0/2026884010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad58157080 con 0x7fad580cd8d0 2024-04-25T08:39:02.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad3ffff640 1 -- 172.21.15.38:0/2026884010 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad50003d80 con 0x7fad580cd8d0 2024-04-25T08:39:02.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.781+0000 7fad5db96640 1 -- 172.21.15.38:0/2026884010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad580af850 con 0x7fad580cd8d0 2024-04-25T08:39:02.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.782+0000 7fad5db96640 1 -- 172.21.15.38:0/2026884010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad58068e80 con 0x7fad580cd8d0 2024-04-25T08:39:02.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.783+0000 7fad3ffff640 1 -- 172.21.15.38:0/2026884010 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad50005d10 con 0x7fad580cd8d0 2024-04-25T08:39:02.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.783+0000 7fad3ffff640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad4406b4d0 0x7fad4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:02.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.783+0000 7fad3ffff640 1 -- 172.21.15.38:0/2026884010 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fad5008a360 con 0x7fad580cd8d0 2024-04-25T08:39:02.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.783+0000 7fad577fe640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad4406b4d0 0x7fad4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:02.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.784+0000 7fad577fe640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad4406b4d0 0x7fad4406d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fad48007fd0 tx=0x7fad48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:02.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:02.788+0000 7fad3ffff640 1 -- 172.21.15.38:0/2026884010 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad50022020 con 0x7fad580cd8d0 2024-04-25T08:39:02.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.004+0000 7fad5db96640 1 -- 172.21.15.38:0/2026884010 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad58067ef0 con 0x7fad580cd8d0 2024-04-25T08:39:02.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.004+0000 7fad3ffff640 1 -- 172.21.15.38:0/2026884010 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fad50088e20 con 0x7fad580cd8d0 2024-04-25T08:39:02.998 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:02.998 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":59,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3680979257","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3680979257},{"type":"v1","addr":"172.21.15.107:6836","nonce":3680979257}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":6359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/697909489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":697909489},{"type":"v1","addr":"172.21.15.107:6837","nonce":697909489}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":7851,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/3083871141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3083871141},{"type":"v1","addr":"172.21.15.38:6834","nonce":3083871141}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54},{"gid":8205,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/2108823839","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":2108823839},{"type":"v1","addr":"172.21.15.38:6835","nonce":2108823839}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59}],"filesystems":[]} 2024-04-25T08:39:02.998 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 59 2024-04-25T08:39:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.006+0000 7fad5db96640 1 -- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad4406b4d0 msgr2=0x7fad4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.006+0000 7fad5db96640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad4406b4d0 0x7fad4406d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fad48007fd0 tx=0x7fad48009040 comp rx=0 tx=0).stop 2024-04-25T08:39:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.006+0000 7fad5db96640 1 -- 172.21.15.38:0/2026884010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad580cd8d0 msgr2=0x7fad58156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.006+0000 7fad5db96640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad580cd8d0 0x7fad58156b40 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fad5000ef90 tx=0x7fad5000c490 comp rx=0 tx=0).stop 2024-04-25T08:39:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.006+0000 7fad5db96640 1 -- 172.21.15.38:0/2026884010 shutdown_connections 2024-04-25T08:39:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.006+0000 7fad5db96640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad580cd8d0 0x7fad58156b40 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.006+0000 7fad5db96640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad5809acb0 0x7fad580aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.006+0000 7fad5db96640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad4406b4d0 0x7fad4406d990 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.006+0000 7fad5db96640 1 --2- 172.21.15.38:0/2026884010 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad5806cdf0 0x7fad5815fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:03.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.006+0000 7fad5db96640 1 -- 172.21.15.38:0/2026884010 >> 172.21.15.38:0/2026884010 conn(0x7fad580b3160 msgr2=0x7fad58070310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.006+0000 7fad5db96640 1 -- 172.21.15.38:0/2026884010 shutdown_connections 2024-04-25T08:39:03.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.006+0000 7fad5db96640 1 -- 172.21.15.38:0/2026884010 wait complete. 2024-04-25T08:39:03.011 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:39:03.011 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:39:03.012 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:39:03.012 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:39:03.012 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:39:03.012 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:39:03.012 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:39:03.012 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:39:03.012 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:39:03.012 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:39:03.013 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:39:03.013 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:39:03.013 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:39:03.013 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:39:03.013 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:39:03.013 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:39:03.021 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:39:03.021 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:39:03.022 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:39:03.022 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:39:03.024 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:39:03.024 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:39:03.025 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:39:03.025 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:39:03.063 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:39:03.064 DEBUG:teuthology.parallel:result is None 2024-04-25T08:39:03.064 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:39:03.064 DEBUG:teuthology.parallel:result is None 2024-04-25T08:39:03.066 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:39:03.067 DEBUG:teuthology.parallel:result is None 2024-04-25T08:39:03.067 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:39:03.067 DEBUG:teuthology.parallel:result is None 2024-04-25T08:39:03.067 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:39:03.139 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:39:03.139 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:39:03.145 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-25T08:39:03.146 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:39:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.181+0000 7fc41fbd5640 1 Processor -- start 2024-04-25T08:39:03.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.181+0000 7fc41fbd5640 1 -- start start 2024-04-25T08:39:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.182+0000 7fc41fbd5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc4180b2cd0 0x7fc4180b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.182+0000 7fc41fbd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41806abe0 0x7fc41806b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.182+0000 7fc41fbd5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4180ad650 con 0x7fc418157a40 2024-04-25T08:39:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.182+0000 7fc41fbd5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc41815b220 con 0x7fc4180b2cd0 2024-04-25T08:39:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.182+0000 7fc41fbd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180ad8f0 con 0x7fc41806abe0 2024-04-25T08:39:03.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.183+0000 7fc41e172640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41806abe0 0x7fc41806b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:03.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.183+0000 7fc41d971640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fc418157a40 0x7fc4180ae3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:39778/0 (socket says 172.21.15.38:39778) 2024-04-25T08:39:03.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.183+0000 7fc41d971640 1 -- 172.21.15.38:0/2330179117 learned_addr learned my addr 172.21.15.38:0/2330179117 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:03.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.183+0000 7fc41d170640 1 --2- 172.21.15.38:0/2330179117 >> v2:172.21.15.107:3300/0 conn(0x7fc4180b2cd0 0x7fc4180b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:03.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.183+0000 7fc41d170640 1 -- 172.21.15.38:0/2330179117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41806abe0 msgr2=0x7fc41806b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:03.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.183+0000 7fc41d170640 1 --2- 172.21.15.38:0/2330179117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41806abe0 0x7fc41806b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:03.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.183+0000 7fc41d170640 1 -- 172.21.15.38:0/2330179117 >> v1:172.21.15.107:6789/0 conn(0x7fc418157a40 legacy=0x7fc4180ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:03.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.183+0000 7fc41d170640 1 -- 172.21.15.38:0/2330179117 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4180adbd0 con 0x7fc4180b2cd0 2024-04-25T08:39:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.184+0000 7fc41d170640 1 --2- 172.21.15.38:0/2330179117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4180b2cd0 0x7fc4180b3160 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc4080097a0 tx=0x7fc40802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4250a3dc6f2a3c36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.184+0000 7fc41c96f640 1 -- 172.21.15.38:0/2330179117 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408037070 con 0x7fc4180b2cd0 2024-04-25T08:39:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.184+0000 7fc41c96f640 1 -- 172.21.15.38:0/2330179117 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc408032440 con 0x7fc4180b2cd0 2024-04-25T08:39:03.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.184+0000 7fc41c96f640 1 -- 172.21.15.38:0/2330179117 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408032810 con 0x7fc4180b2cd0 2024-04-25T08:39:03.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.185+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2330179117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4180b2cd0 msgr2=0x7fc4180b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:03.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.185+0000 7fc41fbd5640 1 --2- 172.21.15.38:0/2330179117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4180b2cd0 0x7fc4180b3160 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc4080097a0 tx=0x7fc40802a180 comp rx=0 tx=0).stop 2024-04-25T08:39:03.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.185+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2330179117 shutdown_connections 2024-04-25T08:39:03.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.185+0000 7fc41fbd5640 1 --2- 172.21.15.38:0/2330179117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc41806abe0 0x7fc41806b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:03.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.185+0000 7fc41fbd5640 1 --2- 172.21.15.38:0/2330179117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4180b2cd0 0x7fc4180b3160 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:03.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.185+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2330179117 >> 172.21.15.38:0/2330179117 conn(0x7fc4180cfd20 msgr2=0x7fc4180d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:03.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.185+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2330179117 shutdown_connections 2024-04-25T08:39:03.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.185+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2330179117 wait complete. 2024-04-25T08:39:03.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.186+0000 7fc41fbd5640 1 Processor -- start 2024-04-25T08:39:03.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.186+0000 7fc41fbd5640 1 -- start start 2024-04-25T08:39:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41fbd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc41806abe0 0x7fc418150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41fbd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4180b2cd0 0x7fc41815cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41fbd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc418157a40 0x7fc41815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41fbd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41806b570 con 0x7fc41806abe0 2024-04-25T08:39:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41fbd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc418157e20 con 0x7fc4180b2cd0 2024-04-25T08:39:03.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41fbd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4180668f0 con 0x7fc418157a40 2024-04-25T08:39:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41d971640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc41806abe0 0x7fc418150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41d971640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc41806abe0 0x7fc418150100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53690/0 (socket says 172.21.15.38:53690) 2024-04-25T08:39:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41d971640 1 -- 172.21.15.38:0/2065261617 learned_addr learned my addr 172.21.15.38:0/2065261617 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41d170640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4180b2cd0 0x7fc41815cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41e172640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc418157a40 0x7fc41815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41d971640 1 -- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc418157a40 msgr2=0x7fc41815d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41d971640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc418157a40 0x7fc41815d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41d971640 1 -- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4180b2cd0 msgr2=0x7fc41815cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41d971640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4180b2cd0 0x7fc41815cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41d971640 1 -- 172.21.15.38:0/2065261617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41815b6b0 con 0x7fc41806abe0 2024-04-25T08:39:03.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.187+0000 7fc41d971640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc41806abe0 0x7fc418150100 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c3f0 tx=0x7fc41400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:03.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.192+0000 7fc40e7fc640 1 -- 172.21.15.38:0/2065261617 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414017070 con 0x7fc41806abe0 2024-04-25T08:39:03.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.192+0000 7fc40e7fc640 1 -- 172.21.15.38:0/2065261617 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc414005ce0 con 0x7fc41806abe0 2024-04-25T08:39:03.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.192+0000 7fc40e7fc640 1 -- 172.21.15.38:0/2065261617 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4140072f0 con 0x7fc41806abe0 2024-04-25T08:39:03.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.192+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2065261617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41815b990 con 0x7fc41806abe0 2024-04-25T08:39:03.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.192+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2065261617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41815bf50 con 0x7fc41806abe0 2024-04-25T08:39:03.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.193+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2065261617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc418067d30 con 0x7fc41806abe0 2024-04-25T08:39:03.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.193+0000 7fc40e7fc640 1 -- 172.21.15.38:0/2065261617 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc41401a070 con 0x7fc41806abe0 2024-04-25T08:39:03.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.194+0000 7fc40e7fc640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc3fc06b4d0 0x7fc3fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:03.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.194+0000 7fc40e7fc640 1 -- 172.21.15.38:0/2065261617 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc4140893f0 con 0x7fc41806abe0 2024-04-25T08:39:03.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.194+0000 7fc41d170640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc3fc06b4d0 0x7fc3fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:03.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.194+0000 7fc41d170640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc3fc06b4d0 0x7fc3fc06d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc408034000 tx=0x7fc40802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:03.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.198+0000 7fc40e7fc640 1 -- 172.21.15.38:0/2065261617 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc418067d30 con 0x7fc41806abe0 2024-04-25T08:39:03.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:03.390+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2065261617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc41804ef90 con 0x7fc41806abe0 2024-04-25T08:39:04.070 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:39:04.073 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:39:04.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.189+0000 7fc40e7fc640 1 -- 172.21.15.38:0/2065261617 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v104) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc414057d10 con 0x7fc41806abe0 2024-04-25T08:39:04.182 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-04-25T08:39:04.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.192+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc3fc06b4d0 msgr2=0x7fc3fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.192+0000 7fc41fbd5640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc3fc06b4d0 0x7fc3fc06d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc408034000 tx=0x7fc40802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:04.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.192+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2065261617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc41806abe0 msgr2=0x7fc418150100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.192+0000 7fc41fbd5640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc41806abe0 0x7fc418150100 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fc41400c3f0 tx=0x7fc41400c8b0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.192+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2065261617 shutdown_connections 2024-04-25T08:39:04.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.192+0000 7fc41fbd5640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc3fc06b4d0 0x7fc3fc06d990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.192+0000 7fc41fbd5640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc418157a40 0x7fc41815d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.192+0000 7fc41fbd5640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc4180b2cd0 0x7fc41815cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.192+0000 7fc41fbd5640 1 --2- 172.21.15.38:0/2065261617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc41806abe0 0x7fc418150100 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.192+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2065261617 >> 172.21.15.38:0/2065261617 conn(0x7fc4180cfd20 msgr2=0x7fc4180ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:04.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.192+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2065261617 shutdown_connections 2024-04-25T08:39:04.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.192+0000 7fc41fbd5640 1 -- 172.21.15.38:0/2065261617 wait complete. 2024-04-25T08:39:04.199 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:39:04.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.313+0000 7febe814f640 1 Processor -- start 2024-04-25T08:39:04.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.313+0000 7febe814f640 1 -- start start 2024-04-25T08:39:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.314+0000 7febe814f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7febe0099a70 0x7febe0099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.314+0000 7febe814f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe009a390 con 0x7febe00cf470 2024-04-25T08:39:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.314+0000 7febe814f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe00ccd60 con 0x7febe006cd90 2024-04-25T08:39:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.314+0000 7febe814f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe01614d0 con 0x7febe0099a70 2024-04-25T08:39:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.314+0000 7febe5eeb640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7febe0099a70 0x7febe0099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.314+0000 7febe56ea640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7febe00cf470 0x7febe00cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37070/0 (socket says 172.21.15.38:37070) 2024-04-25T08:39:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.314+0000 7febe56ea640 1 -- 172.21.15.38:0/1568141991 learned_addr learned my addr 172.21.15.38:0/1568141991 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.314+0000 7febe5eeb640 1 -- 172.21.15.38:0/1568141991 >> v1:172.21.15.107:6789/0 conn(0x7febe006cd90 legacy=0x7febe0167710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.314+0000 7febe5eeb640 1 -- 172.21.15.38:0/1568141991 >> v1:172.21.15.38:6789/0 conn(0x7febe00cf470 legacy=0x7febe00cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.314+0000 7febe5eeb640 1 -- 172.21.15.38:0/1568141991 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe00cd040 con 0x7febe0099a70 2024-04-25T08:39:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.314+0000 7febe4ee9640 1 -- 172.21.15.38:0/1568141991 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081281991 0 0) 0x7febe009a390 con 0x7febe00cf470 2024-04-25T08:39:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.315+0000 7febe5eeb640 1 --2- 172.21.15.38:0/1568141991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febe0099a70 0x7febe0099e50 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7febd0009d90 tx=0x7febd002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d201033d3b870b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.315+0000 7febe4ee9640 1 -- 172.21.15.38:0/1568141991 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0007740 con 0x7febe0099a70 2024-04-25T08:39:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.315+0000 7febe4ee9640 1 -- 172.21.15.38:0/1568141991 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd00078e0 con 0x7febe0099a70 2024-04-25T08:39:04.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.315+0000 7febe4ee9640 1 -- 172.21.15.38:0/1568141991 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0007cb0 con 0x7febe0099a70 2024-04-25T08:39:04.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.315+0000 7febe814f640 1 -- 172.21.15.38:0/1568141991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febe0099a70 msgr2=0x7febe0099e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.315+0000 7febe814f640 1 --2- 172.21.15.38:0/1568141991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febe0099a70 0x7febe0099e50 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7febd0009d90 tx=0x7febd002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.316+0000 7febe814f640 1 -- 172.21.15.38:0/1568141991 shutdown_connections 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.316+0000 7febe814f640 1 --2- 172.21.15.38:0/1568141991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febe0099a70 0x7febe0099e50 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.316+0000 7febe814f640 1 -- 172.21.15.38:0/1568141991 >> 172.21.15.38:0/1568141991 conn(0x7febe00b3180 msgr2=0x7febe00b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.316+0000 7febe814f640 1 -- 172.21.15.38:0/1568141991 shutdown_connections 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.316+0000 7febe814f640 1 -- 172.21.15.38:0/1568141991 wait complete. 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.317+0000 7febe814f640 1 Processor -- start 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.317+0000 7febe814f640 1 -- start start 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe814f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febe006cd90 0x7febe015fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe814f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febe0099a70 0x7febe00c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe814f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febe00cf470 0x7febe00c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe814f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe00cd600 con 0x7febe006cd90 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe814f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe00cd970 con 0x7febe0099a70 2024-04-25T08:39:04.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe814f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe00d0780 con 0x7febe00cf470 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe5eeb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febe006cd90 0x7febe015fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe5eeb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febe006cd90 0x7febe015fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53706/0 (socket says 172.21.15.38:53706) 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe5eeb640 1 -- 172.21.15.38:0/3915902005 learned_addr learned my addr 172.21.15.38:0/3915902005 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe56ea640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febe0099a70 0x7febe00c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe66ec640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febe00cf470 0x7febe00c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe5eeb640 1 -- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febe00cf470 msgr2=0x7febe00c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe5eeb640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febe00cf470 0x7febe00c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe5eeb640 1 -- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febe0099a70 msgr2=0x7febe00c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe5eeb640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febe0099a70 0x7febe00c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe5eeb640 1 -- 172.21.15.38:0/3915902005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe009b600 con 0x7febe006cd90 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe5eeb640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febe006cd90 0x7febe015fcf0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7febd0009d90 tx=0x7febd0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febde7fc640 1 -- 172.21.15.38:0/3915902005 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0040070 con 0x7febe006cd90 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febde7fc640 1 -- 172.21.15.38:0/3915902005 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd0031d00 con 0x7febe006cd90 2024-04-25T08:39:04.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.318+0000 7febe814f640 1 -- 172.21.15.38:0/3915902005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe009b870 con 0x7febe006cd90 2024-04-25T08:39:04.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.319+0000 7febde7fc640 1 -- 172.21.15.38:0/3915902005 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0045660 con 0x7febe006cd90 2024-04-25T08:39:04.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.319+0000 7febe814f640 1 -- 172.21.15.38:0/3915902005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe009be30 con 0x7febe006cd90 2024-04-25T08:39:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.320+0000 7febe814f640 1 -- 172.21.15.38:0/3915902005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe0068e80 con 0x7febe006cd90 2024-04-25T08:39:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.320+0000 7febde7fc640 1 -- 172.21.15.38:0/3915902005 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7febd0039040 con 0x7febe006cd90 2024-04-25T08:39:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.320+0000 7febde7fc640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7febcc06b4d0 0x7febcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.321+0000 7febde7fc640 1 -- 172.21.15.38:0/3915902005 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7febd00ab9d0 con 0x7febe006cd90 2024-04-25T08:39:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.321+0000 7febe56ea640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7febcc06b4d0 0x7febcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:04.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.321+0000 7febe56ea640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7febcc06b4d0 0x7febcc06d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7febe00c6ec0 tx=0x7febd4007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:04.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.325+0000 7febde7fc640 1 -- 172.21.15.38:0/3915902005 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7febd007a3b0 con 0x7febe006cd90 2024-04-25T08:39:04.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.574+0000 7febe814f640 1 -- 172.21.15.38:0/3915902005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7febe00b38e0 con 0x7febe006cd90 2024-04-25T08:39:04.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.575+0000 7febde7fc640 1 -- 172.21.15.38:0/3915902005 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7febd007e450 con 0x7febe006cd90 2024-04-25T08:39:04.568 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmwq8CDBAA+cjd+QXtFgwZ3kxdABRWlA==", 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-04-25T08:39:04.569 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmPRFbFxAAojn2r/62vvf1qTy9QVuyIw==", 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmGLjqEhAAyPRgyy5iMgtccapydv4uOQ==", 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:39:04.570 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmt0AUHhAAoQyQ7wXBMGE+7Hi84tUfAg==", 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-04-25T08:39:04.571 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCxFCpmv+iTExAAEJihmsvBbp9xXxPnj49D6g==", 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC0FCpmdnQKFBAADLAsYZEUhoJx292jIYZroQ==", 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:04.572 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC3FCpm4f5BExAAMVSPkqsrnuLcO88AXKrGHQ==", 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC6FCpmHNN7EhAABPhmnxGo70ZOd8Oex62Mdw==", 2024-04-25T08:39:04.573 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDAFCpmiThGFhAAG5Pyvn/JzL9NUoxHJkPMvg==", 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.574 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDDFCpmb/FHFBAA2cMhMqQQc4Fk6j8FpjhnaQ==", 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDGFCpmAZsSEhAAnGIqqcvuHMVSc8Zbk309kA==", 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.575 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:04.576 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:04.576 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.576 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.576 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.576 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.576 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-04-25T08:39:04.576 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDJFCpmqdVUEBAAb9abblyUztrjwxxe4jsAwQ==", 2024-04-25T08:39:04.576 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.576 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:04.576 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:04.576 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.576 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw==", 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmL7eKKhAASeQEdHDBInfozdLVzP4DQw==", 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.577 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmH22HJxAAWyrooyJn5hRs/bb0Rn1HHw==", 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:39:04.578 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpm6Zr/LxAAxh9Y2gvvqj1jIaVVrQAvgw==", 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.579 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCpFCpmlj/fLBAAvxLluFQFR8uhppGlNpaygA==", 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcXHhCBAAvI5zmWidVTvOmEcVRWasEA==", 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.580 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.581 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:04.581 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:04.581 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-04-25T08:39:04.581 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcr96BRAAQwUPxILJukYERAiEXeur+g==", 2024-04-25T08:39:04.581 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:04.581 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:39:04.581 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:39:04.581 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:04.581 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.581 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:04.581 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T08:39:04.581 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:39:04.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.577+0000 7febe814f640 1 -- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7febcc06b4d0 msgr2=0x7febcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.577+0000 7febe814f640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7febcc06b4d0 0x7febcc06d990 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7febe00c6ec0 tx=0x7febd4007e40 comp rx=0 tx=0).stop 2024-04-25T08:39:04.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.577+0000 7febe814f640 1 -- 172.21.15.38:0/3915902005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febe006cd90 msgr2=0x7febe015fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.577+0000 7febe814f640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febe006cd90 0x7febe015fcf0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7febd0009d90 tx=0x7febd0004a40 comp rx=0 tx=0).stop 2024-04-25T08:39:04.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.577+0000 7febe814f640 1 -- 172.21.15.38:0/3915902005 shutdown_connections 2024-04-25T08:39:04.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.577+0000 7febe814f640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7febcc06b4d0 0x7febcc06d990 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.577+0000 7febe814f640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7febe00cf470 0x7febe00c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.577+0000 7febe814f640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7febe0099a70 0x7febe00c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.577+0000 7febe814f640 1 --2- 172.21.15.38:0/3915902005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7febe006cd90 0x7febe015fcf0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.577+0000 7febe814f640 1 -- 172.21.15.38:0/3915902005 >> 172.21.15.38:0/3915902005 conn(0x7febe00b3180 msgr2=0x7febe00ca7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:04.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.577+0000 7febe814f640 1 -- 172.21.15.38:0/3915902005 shutdown_connections 2024-04-25T08:39:04.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.577+0000 7febe814f640 1 -- 172.21.15.38:0/3915902005 wait complete. 2024-04-25T08:39:04.584 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:04.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.692+0000 7f47835a9640 1 Processor -- start 2024-04-25T08:39:04.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.692+0000 7f47835a9640 1 -- start start 2024-04-25T08:39:04.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f47835a9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477c0b5030 0x7f477c0ce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:04.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f47835a9640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c0cd690 con 0x7f477c15be80 2024-04-25T08:39:04.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f47835a9640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c0cd970 con 0x7f477c154cb0 2024-04-25T08:39:04.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f47835a9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0cd440 con 0x7f477c0b5030 2024-04-25T08:39:04.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f4780b44640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f477c15be80 0x7f477c0b4920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37074/0 (socket says 172.21.15.38:37074) 2024-04-25T08:39:04.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f4780b44640 1 -- 172.21.15.38:0/2209364367 learned_addr learned my addr 172.21.15.38:0/2209364367 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:04.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f4781b46640 1 --2- 172.21.15.38:0/2209364367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477c0b5030 0x7f477c0ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:04.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f476ffff640 1 -- 172.21.15.38:0/2209364367 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597627480 0 0) 0x7f477c0cd690 con 0x7f477c15be80 2024-04-25T08:39:04.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f476ffff640 1 -- 172.21.15.38:0/2209364367 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4768003520 con 0x7f477c15be80 2024-04-25T08:39:04.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f476ffff640 1 -- 172.21.15.38:0/2209364367 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2325141055 0 0) 0x7f477c0cd970 con 0x7f477c154cb0 2024-04-25T08:39:04.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f476ffff640 1 -- 172.21.15.38:0/2209364367 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f477c0cd690 con 0x7f477c154cb0 2024-04-25T08:39:04.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f4781b46640 1 -- 172.21.15.38:0/2209364367 >> v1:172.21.15.107:6789/0 conn(0x7f477c154cb0 legacy=0x7f477c0c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f4781b46640 1 -- 172.21.15.38:0/2209364367 >> v1:172.21.15.38:6789/0 conn(0x7f477c15be80 legacy=0x7f477c0b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.693+0000 7f4781b46640 1 -- 172.21.15.38:0/2209364367 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0cdc50 con 0x7f477c0b5030 2024-04-25T08:39:04.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.694+0000 7f4781b46640 1 --2- 172.21.15.38:0/2209364367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477c0b5030 0x7f477c0ce730 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4774009dc0 tx=0x7f477402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43da1c9cb9b1a6ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:04.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.694+0000 7f476ffff640 1 -- 172.21.15.38:0/2209364367 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774007740 con 0x7f477c0b5030 2024-04-25T08:39:04.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.694+0000 7f476ffff640 1 -- 172.21.15.38:0/2209364367 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47740078e0 con 0x7f477c0b5030 2024-04-25T08:39:04.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.694+0000 7f476ffff640 1 -- 172.21.15.38:0/2209364367 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774007cb0 con 0x7f477c0b5030 2024-04-25T08:39:04.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.695+0000 7f47835a9640 1 -- 172.21.15.38:0/2209364367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477c0b5030 msgr2=0x7f477c0ce730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.695+0000 7f47835a9640 1 --2- 172.21.15.38:0/2209364367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477c0b5030 0x7f477c0ce730 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4774009dc0 tx=0x7f477402a6d0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.695+0000 7f47835a9640 1 -- 172.21.15.38:0/2209364367 shutdown_connections 2024-04-25T08:39:04.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.695+0000 7f47835a9640 1 --2- 172.21.15.38:0/2209364367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477c0b5030 0x7f477c0ce730 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.695+0000 7f47835a9640 1 -- 172.21.15.38:0/2209364367 >> 172.21.15.38:0/2209364367 conn(0x7f477c06b210 msgr2=0x7f477c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:04.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.695+0000 7f47835a9640 1 -- 172.21.15.38:0/2209364367 shutdown_connections 2024-04-25T08:39:04.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.695+0000 7f47835a9640 1 -- 172.21.15.38:0/2209364367 wait complete. 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.696+0000 7f47835a9640 1 Processor -- start 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.696+0000 7f47835a9640 1 -- start start 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f47835a9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477c0b5030 0x7f477c1452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f47835a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f477c154cb0 0x7f477c145b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f47835a9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477c15be80 0x7f477c159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f47835a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0ce210 con 0x7f477c154cb0 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f47835a9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c15c330 con 0x7f477c15be80 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f47835a9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c155090 con 0x7f477c0b5030 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f4780b44640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f477c154cb0 0x7f477c145b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f4780b44640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f477c154cb0 0x7f477c145b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53720/0 (socket says 172.21.15.38:53720) 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f4780b44640 1 -- 172.21.15.38:0/132289491 learned_addr learned my addr 172.21.15.38:0/132289491 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f4781345640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477c0b5030 0x7f477c1452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f4781b46640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477c15be80 0x7f477c159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:04.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f4780b44640 1 -- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477c0b5030 msgr2=0x7f477c1452b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f4780b44640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477c0b5030 0x7f477c1452b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f4780b44640 1 -- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477c15be80 msgr2=0x7f477c159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f4780b44640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477c15be80 0x7f477c159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f4780b44640 1 -- 172.21.15.38:0/132289491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c151180 con 0x7f477c154cb0 2024-04-25T08:39:04.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.697+0000 7f4780b44640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f477c154cb0 0x7f477c145b90 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f477000c430 tx=0x7f477000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:04.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.698+0000 7f476dffb640 1 -- 172.21.15.38:0/132289491 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770017070 con 0x7f477c154cb0 2024-04-25T08:39:04.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.698+0000 7f476dffb640 1 -- 172.21.15.38:0/132289491 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4770005d00 con 0x7f477c154cb0 2024-04-25T08:39:04.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.698+0000 7f47835a9640 1 -- 172.21.15.38:0/132289491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477c0b2740 con 0x7f477c154cb0 2024-04-25T08:39:04.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.698+0000 7f476dffb640 1 -- 172.21.15.38:0/132289491 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4770007310 con 0x7f477c154cb0 2024-04-25T08:39:04.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.698+0000 7f47835a9640 1 -- 172.21.15.38:0/132289491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c151dd0 con 0x7f477c154cb0 2024-04-25T08:39:04.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.699+0000 7f47835a9640 1 -- 172.21.15.38:0/132289491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477c068e80 con 0x7f477c154cb0 2024-04-25T08:39:04.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.699+0000 7f476dffb640 1 -- 172.21.15.38:0/132289491 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f477001a070 con 0x7f477c154cb0 2024-04-25T08:39:04.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.700+0000 7f476dffb640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f476006b4d0 0x7f476006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:04.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.700+0000 7f476dffb640 1 -- 172.21.15.38:0/132289491 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f47700893d0 con 0x7f477c154cb0 2024-04-25T08:39:04.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.700+0000 7f4781345640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f476006b4d0 0x7f476006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:04.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.700+0000 7f4781345640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f476006b4d0 0x7f476006d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4778007fd0 tx=0x7f4778009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:04.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.704+0000 7f476dffb640 1 -- 172.21.15.38:0/132289491 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4770057dc0 con 0x7f477c154cb0 2024-04-25T08:39:04.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.889+0000 7f47835a9640 1 -- 172.21.15.38:0/132289491 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f477c0cd010 con 0x7f477c154cb0 2024-04-25T08:39:04.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.890+0000 7f476dffb640 1 -- 172.21.15.38:0/132289491 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+13154 (secure 0 0 0) 0x7f477005be60 con 0x7f477c154cb0 2024-04-25T08:39:04.883 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:04.883 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":104,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:04.186858+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:04.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.892+0000 7f47835a9640 1 -- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f476006b4d0 msgr2=0x7f476006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.892+0000 7f47835a9640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f476006b4d0 0x7f476006d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f4778007fd0 tx=0x7f4778009040 comp rx=0 tx=0).stop 2024-04-25T08:39:04.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.892+0000 7f47835a9640 1 -- 172.21.15.38:0/132289491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f477c154cb0 msgr2=0x7f477c145b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:04.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.892+0000 7f47835a9640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f477c154cb0 0x7f477c145b90 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f477000c430 tx=0x7f477000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.892+0000 7f47835a9640 1 -- 172.21.15.38:0/132289491 shutdown_connections 2024-04-25T08:39:04.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.892+0000 7f47835a9640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477c15be80 0x7f477c159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.892+0000 7f47835a9640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f477c154cb0 0x7f477c145b90 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.892+0000 7f47835a9640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f476006b4d0 0x7f476006d990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.892+0000 7f47835a9640 1 --2- 172.21.15.38:0/132289491 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477c0b5030 0x7f477c1452b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:04.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.892+0000 7f47835a9640 1 -- 172.21.15.38:0/132289491 >> 172.21.15.38:0/132289491 conn(0x7f477c06b210 msgr2=0x7f477c15b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:04.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.892+0000 7f47835a9640 1 -- 172.21.15.38:0/132289491 shutdown_connections 2024-04-25T08:39:04.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:04.892+0000 7f47835a9640 1 -- 172.21.15.38:0/132289491 wait complete. 2024-04-25T08:39:04.897 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:39:04.897 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:39:05.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.008+0000 7f9743fc1640 1 Processor -- start 2024-04-25T08:39:05.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.008+0000 7f9743fc1640 1 -- start start 2024-04-25T08:39:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f9743fc1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f973c0d0320 0x7f973c0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f9743fc1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f973c0d0700 0x7f973c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f9743fc1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f973c0d2330 con 0x7f973c0d1e90 2024-04-25T08:39:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f9743fc1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c0d2610 con 0x7f973c0d0700 2024-04-25T08:39:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f9743fc1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f973c0d28f0 con 0x7f973c0d0320 2024-04-25T08:39:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f974155c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f973c0d0700 0x7f973c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f974255e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f973c0d1e90 0x7f973c0b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37086/0 (socket says 172.21.15.38:37086) 2024-04-25T08:39:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f974255e640 1 -- 172.21.15.38:0/1157472015 learned_addr learned my addr 172.21.15.38:0/1157472015 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f974155c640 1 -- 172.21.15.38:0/1157472015 >> v2:172.21.15.107:3300/0 conn(0x7f973c0d0320 msgr2=0x7f973c0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f974155c640 1 --2- 172.21.15.38:0/1157472015 >> v2:172.21.15.107:3300/0 conn(0x7f973c0d0320 0x7f973c0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f974155c640 1 -- 172.21.15.38:0/1157472015 >> v1:172.21.15.38:6789/0 conn(0x7f973c0d1e90 legacy=0x7f973c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f974155c640 1 -- 172.21.15.38:0/1157472015 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c0d2bd0 con 0x7f973c0d0700 2024-04-25T08:39:05.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.009+0000 7f9740d5b640 1 -- 172.21.15.38:0/1157472015 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009244750 0 0) 0x7f973c0d2330 con 0x7f973c0d1e90 2024-04-25T08:39:05.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.010+0000 7f974155c640 1 --2- 172.21.15.38:0/1157472015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f973c0d0700 0x7f973c0d1950 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f973c069100 tx=0x7f972c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cf2f375f1086b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:05.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.010+0000 7f9740d5b640 1 -- 172.21.15.38:0/1157472015 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c037070 con 0x7f973c0d0700 2024-04-25T08:39:05.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.010+0000 7f9740d5b640 1 -- 172.21.15.38:0/1157472015 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c032440 con 0x7f973c0d0700 2024-04-25T08:39:05.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.010+0000 7f9740d5b640 1 -- 172.21.15.38:0/1157472015 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c032810 con 0x7f973c0d0700 2024-04-25T08:39:05.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.011+0000 7f9743fc1640 1 -- 172.21.15.38:0/1157472015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f973c0d0700 msgr2=0x7f973c0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.011+0000 7f9743fc1640 1 --2- 172.21.15.38:0/1157472015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f973c0d0700 0x7f973c0d1950 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f973c069100 tx=0x7f972c02a230 comp rx=0 tx=0).stop 2024-04-25T08:39:05.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.011+0000 7f9743fc1640 1 -- 172.21.15.38:0/1157472015 shutdown_connections 2024-04-25T08:39:05.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.011+0000 7f9743fc1640 1 --2- 172.21.15.38:0/1157472015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f973c0d0700 0x7f973c0d1950 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.011+0000 7f9743fc1640 1 --2- 172.21.15.38:0/1157472015 >> v2:172.21.15.107:3300/0 conn(0x7f973c0d0320 0x7f973c0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.011+0000 7f9743fc1640 1 -- 172.21.15.38:0/1157472015 >> 172.21.15.38:0/1157472015 conn(0x7f973c0668f0 msgr2=0x7f973c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:05.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.011+0000 7f9743fc1640 1 -- 172.21.15.38:0/1157472015 shutdown_connections 2024-04-25T08:39:05.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.011+0000 7f9743fc1640 1 -- 172.21.15.38:0/1157472015 wait complete. 2024-04-25T08:39:05.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.012+0000 7f9743fc1640 1 Processor -- start 2024-04-25T08:39:05.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.012+0000 7f9743fc1640 1 -- start start 2024-04-25T08:39:05.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f9743fc1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f973c0d0320 0x7f973c153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f9743fc1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f973c0d0700 0x7f973c0cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f9743fc1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f973c0d1e90 0x7f973c0ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f9743fc1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0d0b40 con 0x7f973c0d0700 2024-04-25T08:39:05.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f9743fc1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c161210 con 0x7f973c0d0320 2024-04-25T08:39:05.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f9743fc1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973c0b4e90 con 0x7f973c0d1e90 2024-04-25T08:39:05.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f974155c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f973c0d0700 0x7f973c0cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f974155c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f973c0d0700 0x7f973c0cc8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53734/0 (socket says 172.21.15.38:53734) 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f974155c640 1 -- 172.21.15.38:0/2774929844 learned_addr learned my addr 172.21.15.38:0/2774929844 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f9741d5d640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f973c0d0320 0x7f973c153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f974255e640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f973c0d1e90 0x7f973c0ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f974155c640 1 -- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f973c0d1e90 msgr2=0x7f973c0ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f974155c640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f973c0d1e90 0x7f973c0ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f974155c640 1 -- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f973c0d0320 msgr2=0x7f973c153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f974155c640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f973c0d0320 0x7f973c153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.013+0000 7f974155c640 1 -- 172.21.15.38:0/2774929844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f973c1542f0 con 0x7f973c0d0700 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.014+0000 7f974155c640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f973c0d0700 0x7f973c0cc8a0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f972c000c00 tx=0x7f972c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.014+0000 7f97327fc640 1 -- 172.21.15.38:0/2774929844 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c041070 con 0x7f973c0d0700 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.014+0000 7f97327fc640 1 -- 172.21.15.38:0/2774929844 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f972c03c890 con 0x7f973c0d0700 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.014+0000 7f97327fc640 1 -- 172.21.15.38:0/2774929844 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f972c037040 con 0x7f973c0d0700 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.014+0000 7f9743fc1640 1 -- 172.21.15.38:0/2774929844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c0b4280 con 0x7f973c0d0700 2024-04-25T08:39:05.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.014+0000 7f9743fc1640 1 -- 172.21.15.38:0/2774929844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c0b4840 con 0x7f973c0d0700 2024-04-25T08:39:05.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.015+0000 7f9743fc1640 1 -- 172.21.15.38:0/2774929844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c0b2e40 con 0x7f973c0d0700 2024-04-25T08:39:05.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.015+0000 7f97327fc640 1 -- 172.21.15.38:0/2774929844 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f972c044070 con 0x7f973c0d0700 2024-04-25T08:39:05.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.016+0000 7f97327fc640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f972006b4d0 0x7f972006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.016+0000 7f97327fc640 1 -- 172.21.15.38:0/2774929844 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f972c0aacd0 con 0x7f973c0d0700 2024-04-25T08:39:05.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.016+0000 7f9741d5d640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f972006b4d0 0x7f972006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.016+0000 7f9741d5d640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f972006b4d0 0x7f972006d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9738009990 tx=0x7f9738009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:05.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.020+0000 7f97327fc640 1 -- 172.21.15.38:0/2774929844 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f973c0b2e40 con 0x7f973c0d0700 2024-04-25T08:39:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.202+0000 7f9743fc1640 1 -- 172.21.15.38:0/2774929844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f973c068e80 con 0x7f973c0d0700 2024-04-25T08:39:05.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.202+0000 7f97327fc640 1 -- 172.21.15.38:0/2774929844 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v104) v1 ==== 93+0+10 (secure 0 0 0) 0x7f972c0797e0 con 0x7f973c0d0700 2024-04-25T08:39:05.196 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:39:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.204+0000 7f9743fc1640 1 -- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f972006b4d0 msgr2=0x7f972006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.204+0000 7f9743fc1640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f972006b4d0 0x7f972006d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9738009990 tx=0x7f9738009110 comp rx=0 tx=0).stop 2024-04-25T08:39:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.204+0000 7f9743fc1640 1 -- 172.21.15.38:0/2774929844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f973c0d0700 msgr2=0x7f973c0cc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.204+0000 7f9743fc1640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f973c0d0700 0x7f973c0cc8a0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f972c000c00 tx=0x7f972c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.204+0000 7f9743fc1640 1 -- 172.21.15.38:0/2774929844 shutdown_connections 2024-04-25T08:39:05.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.204+0000 7f9743fc1640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f972006b4d0 0x7f972006d990 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.204+0000 7f9743fc1640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f973c0d1e90 0x7f973c0ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.204+0000 7f9743fc1640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f973c0d0700 0x7f973c0cc8a0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.204+0000 7f9743fc1640 1 --2- 172.21.15.38:0/2774929844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f973c0d0320 0x7f973c153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.204+0000 7f9743fc1640 1 -- 172.21.15.38:0/2774929844 >> 172.21.15.38:0/2774929844 conn(0x7f973c0668f0 msgr2=0x7f973c0d3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.204+0000 7f9743fc1640 1 -- 172.21.15.38:0/2774929844 shutdown_connections 2024-04-25T08:39:05.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.205+0000 7f9743fc1640 1 -- 172.21.15.38:0/2774929844 wait complete. 2024-04-25T08:39:05.209 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:39:05.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.316+0000 7fc7dc94f640 1 Processor -- start 2024-04-25T08:39:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.316+0000 7fc7dc94f640 1 -- start start 2024-04-25T08:39:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.317+0000 7fc7dc94f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7d80b2220 0x7fc7d80b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.317+0000 7fc7dc94f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc7d80c9910 0x7fc7d80c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.317+0000 7fc7dc94f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d815dfa0 con 0x7fc7d815dbc0 2024-04-25T08:39:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.317+0000 7fc7dc94f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d8073230 con 0x7fc7d80c9910 2024-04-25T08:39:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.317+0000 7fc7dc94f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80b2bc0 con 0x7fc7d80b2220 2024-04-25T08:39:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.317+0000 7fc7d659c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc7d815dbc0 0x7fc7d8072b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37094/0 (socket says 172.21.15.38:37094) 2024-04-25T08:39:05.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.317+0000 7fc7d659c640 1 -- 172.21.15.38:0/1167230409 learned_addr learned my addr 172.21.15.38:0/1167230409 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:05.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.317+0000 7fc7d5d9b640 1 --2- 172.21.15.38:0/1167230409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7d80b2220 0x7fc7d80b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.317+0000 7fc7d6d9d640 1 --2- 172.21.15.38:0/1167230409 >> v2:172.21.15.107:3300/0 conn(0x7fc7d80c9910 0x7fc7d80c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.317+0000 7fc7d559a640 1 -- 172.21.15.38:0/1167230409 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1519594633 0 0) 0x7fc7d815dfa0 con 0x7fc7d815dbc0 2024-04-25T08:39:05.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.317+0000 7fc7d559a640 1 -- 172.21.15.38:0/1167230409 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7c0003520 con 0x7fc7d815dbc0 2024-04-25T08:39:05.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.318+0000 7fc7d5d9b640 1 -- 172.21.15.38:0/1167230409 >> v2:172.21.15.107:3300/0 conn(0x7fc7d80c9910 msgr2=0x7fc7d80c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:05.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.318+0000 7fc7d5d9b640 1 --2- 172.21.15.38:0/1167230409 >> v2:172.21.15.107:3300/0 conn(0x7fc7d80c9910 0x7fc7d80c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.318+0000 7fc7d5d9b640 1 -- 172.21.15.38:0/1167230409 >> v1:172.21.15.38:6789/0 conn(0x7fc7d815dbc0 legacy=0x7fc7d8072b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.318+0000 7fc7d5d9b640 1 -- 172.21.15.38:0/1167230409 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d80b3970 con 0x7fc7d80b2220 2024-04-25T08:39:05.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.318+0000 7fc7d6d9d640 1 --2- 172.21.15.38:0/1167230409 >> v2:172.21.15.107:3300/0 conn(0x7fc7d80c9910 0x7fc7d80c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:05.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.319+0000 7fc7d5d9b640 1 --2- 172.21.15.38:0/1167230409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7d80b2220 0x7fc7d80b2680 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0097a0 tx=0x7fc7cc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcae8b399bb65f9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:05.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.319+0000 7fc7d559a640 1 -- 172.21.15.38:0/1167230409 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc037070 con 0x7fc7d80b2220 2024-04-25T08:39:05.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.319+0000 7fc7d559a640 1 -- 172.21.15.38:0/1167230409 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc032440 con 0x7fc7d80b2220 2024-04-25T08:39:05.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.319+0000 7fc7d559a640 1 -- 172.21.15.38:0/1167230409 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc032810 con 0x7fc7d80b2220 2024-04-25T08:39:05.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.319+0000 7fc7dc94f640 1 -- 172.21.15.38:0/1167230409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7d80b2220 msgr2=0x7fc7d80b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.319+0000 7fc7dc94f640 1 --2- 172.21.15.38:0/1167230409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7d80b2220 0x7fc7d80b2680 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc0097a0 tx=0x7fc7cc02a180 comp rx=0 tx=0).stop 2024-04-25T08:39:05.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.320+0000 7fc7dc94f640 1 -- 172.21.15.38:0/1167230409 shutdown_connections 2024-04-25T08:39:05.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.320+0000 7fc7dc94f640 1 --2- 172.21.15.38:0/1167230409 >> v2:172.21.15.107:3300/0 conn(0x7fc7d80c9910 0x7fc7d80c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.320+0000 7fc7dc94f640 1 --2- 172.21.15.38:0/1167230409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7d80b2220 0x7fc7d80b2680 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.320+0000 7fc7dc94f640 1 -- 172.21.15.38:0/1167230409 >> 172.21.15.38:0/1167230409 conn(0x7fc7d80b5c90 msgr2=0x7fc7d80b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:05.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.320+0000 7fc7dc94f640 1 -- 172.21.15.38:0/1167230409 shutdown_connections 2024-04-25T08:39:05.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.320+0000 7fc7dc94f640 1 -- 172.21.15.38:0/1167230409 wait complete. 2024-04-25T08:39:05.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.321+0000 7fc7dc94f640 1 Processor -- start 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.321+0000 7fc7dc94f640 1 -- start start 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7dc94f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d80b2220 0x7fc7d8145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7dc94f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d80c9910 0x7fc7d81597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7dc94f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7d815dbc0 0x7fc7d8159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7dc94f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80b4210 con 0x7fc7d80c9910 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7dc94f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80b3f30 con 0x7fc7d80b2220 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7dc94f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80b44f0 con 0x7fc7d815dbc0 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7d5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d80c9910 0x7fc7d81597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7d5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d80c9910 0x7fc7d81597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53748/0 (socket says 172.21.15.38:53748) 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7d5d9b640 1 -- 172.21.15.38:0/493498093 learned_addr learned my addr 172.21.15.38:0/493498093 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7d659c640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d80b2220 0x7fc7d8145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7d6d9d640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7d815dbc0 0x7fc7d8159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7d5d9b640 1 -- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7d815dbc0 msgr2=0x7fc7d8159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7d5d9b640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7d815dbc0 0x7fc7d8159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7d5d9b640 1 -- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d80b2220 msgr2=0x7fc7d8145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7d5d9b640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d80b2220 0x7fc7d8145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7d5d9b640 1 -- 172.21.15.38:0/493498093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d815f730 con 0x7fc7d80c9910 2024-04-25T08:39:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7d6d9d640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7d815dbc0 0x7fc7d8159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.322+0000 7fc7d5d9b640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d80c9910 0x7fc7d81597c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc035040 tx=0x7fc7cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.323+0000 7fc7c6ffd640 1 -- 172.21.15.38:0/493498093 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc041030 con 0x7fc7d80c9910 2024-04-25T08:39:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.323+0000 7fc7c6ffd640 1 -- 172.21.15.38:0/493498093 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc03d4e0 con 0x7fc7d80c9910 2024-04-25T08:39:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.323+0000 7fc7dc94f640 1 -- 172.21.15.38:0/493498093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d815fa10 con 0x7fc7d80c9910 2024-04-25T08:39:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.323+0000 7fc7c6ffd640 1 -- 172.21.15.38:0/493498093 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc037040 con 0x7fc7d80c9910 2024-04-25T08:39:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.323+0000 7fc7dc94f640 1 -- 172.21.15.38:0/493498093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d815ffd0 con 0x7fc7d80c9910 2024-04-25T08:39:05.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.324+0000 7fc7dc94f640 1 -- 172.21.15.38:0/493498093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d80d0a20 con 0x7fc7d80c9910 2024-04-25T08:39:05.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.324+0000 7fc7c6ffd640 1 -- 172.21.15.38:0/493498093 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7cc044070 con 0x7fc7d80c9910 2024-04-25T08:39:05.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.325+0000 7fc7c6ffd640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc7b806b500 0x7fc7b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.325+0000 7fc7c6ffd640 1 -- 172.21.15.38:0/493498093 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc7cc0aa4b0 con 0x7fc7d80c9910 2024-04-25T08:39:05.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.325+0000 7fc7d659c640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc7b806b500 0x7fc7b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.325+0000 7fc7d659c640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc7b806b500 0x7fc7b806d9c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8005880 tx=0x7fc7c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:05.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.329+0000 7fc7c6ffd640 1 -- 172.21.15.38:0/493498093 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7d80d0a20 con 0x7fc7d80c9910 2024-04-25T08:39:05.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.511+0000 7fc7dc94f640 1 -- 172.21.15.38:0/493498093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc7d8068e80 con 0x7fc7d80c9910 2024-04-25T08:39:05.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.511+0000 7fc7c6ffd640 1 -- 172.21.15.38:0/493498093 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v104) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc7cc078fd0 con 0x7fc7d80c9910 2024-04-25T08:39:05.505 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:05.505 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T08:39:05.505 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:05.505 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-04-25T08:39:05.505 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-04-25T08:39:05.505 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:05.505 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T08:39:05.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.513+0000 7fc7dc94f640 1 -- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc7b806b500 msgr2=0x7fc7b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.513+0000 7fc7dc94f640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc7b806b500 0x7fc7b806d9c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8005880 tx=0x7fc7c8009040 comp rx=0 tx=0).stop 2024-04-25T08:39:05.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.513+0000 7fc7dc94f640 1 -- 172.21.15.38:0/493498093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d80c9910 msgr2=0x7fc7d81597c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.513+0000 7fc7dc94f640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d80c9910 0x7fc7d81597c0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc035040 tx=0x7fc7cc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.513+0000 7fc7dc94f640 1 -- 172.21.15.38:0/493498093 shutdown_connections 2024-04-25T08:39:05.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.513+0000 7fc7dc94f640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc7b806b500 0x7fc7b806d9c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.513+0000 7fc7dc94f640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7d815dbc0 0x7fc7d8159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.513+0000 7fc7dc94f640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7d80c9910 0x7fc7d81597c0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.513+0000 7fc7dc94f640 1 --2- 172.21.15.38:0/493498093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7d80b2220 0x7fc7d8145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.513+0000 7fc7dc94f640 1 -- 172.21.15.38:0/493498093 >> 172.21.15.38:0/493498093 conn(0x7fc7d80b5c90 msgr2=0x7fc7d8158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:05.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.513+0000 7fc7dc94f640 1 -- 172.21.15.38:0/493498093 shutdown_connections 2024-04-25T08:39:05.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.513+0000 7fc7dc94f640 1 -- 172.21.15.38:0/493498093 wait complete. 2024-04-25T08:39:05.518 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:39:05.518 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:39:05.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.629+0000 7ffb0ac22640 1 Processor -- start 2024-04-25T08:39:05.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.630+0000 7ffb0ac22640 1 -- start start 2024-04-25T08:39:05.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.630+0000 7ffb0ac22640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb041612c0 0x7ffb040b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.630+0000 7ffb0ac22640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffb040d0870 0x7ffb040d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.630+0000 7ffb0ac22640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb040b7200 con 0x7ffb040b6a00 2024-04-25T08:39:05.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.630+0000 7ffb0ac22640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb040d11e0 con 0x7ffb040d0870 2024-04-25T08:39:05.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.630+0000 7ffb0ac22640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040b74e0 con 0x7ffb041612c0 2024-04-25T08:39:05.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.630+0000 7ffb091bf640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffb040d0870 0x7ffb040d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.631+0000 7ffb091bf640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffb040d0870 0x7ffb040d0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53752/0 (socket says 172.21.15.38:53752) 2024-04-25T08:39:05.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.631+0000 7ffb091bf640 1 -- 172.21.15.38:0/1257252729 learned_addr learned my addr 172.21.15.38:0/1257252729 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:05.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.631+0000 7ffb089be640 1 --2- 172.21.15.38:0/1257252729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb041612c0 0x7ffb040b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.631+0000 7ffb091bf640 1 -- 172.21.15.38:0/1257252729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb041612c0 msgr2=0x7ffb040b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:05.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.631+0000 7ffb091bf640 1 --2- 172.21.15.38:0/1257252729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb041612c0 0x7ffb040b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.631+0000 7ffb091bf640 1 -- 172.21.15.38:0/1257252729 >> v1:172.21.15.107:6789/0 conn(0x7ffb040b6a00 legacy=0x7ffb040d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.631+0000 7ffb091bf640 1 -- 172.21.15.38:0/1257252729 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb040b77c0 con 0x7ffb040d0870 2024-04-25T08:39:05.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.631+0000 7ffb037fe640 1 -- 172.21.15.38:0/1257252729 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983031127 0 0) 0x7ffb040b7200 con 0x7ffb040b6a00 2024-04-25T08:39:05.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.632+0000 7ffb091bf640 1 --2- 172.21.15.38:0/1257252729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb040d0870 0x7ffb040d0ca0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009e50 tx=0x7ffafc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd34356a0870119e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:05.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.632+0000 7ffb037fe640 1 -- 172.21.15.38:0/1257252729 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc036070 con 0x7ffb040d0870 2024-04-25T08:39:05.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.632+0000 7ffb037fe640 1 -- 172.21.15.38:0/1257252729 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffafc031600 con 0x7ffb040d0870 2024-04-25T08:39:05.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.632+0000 7ffb037fe640 1 -- 172.21.15.38:0/1257252729 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0319d0 con 0x7ffb040d0870 2024-04-25T08:39:05.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.632+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1257252729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb040d0870 msgr2=0x7ffb040d0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.632+0000 7ffb0ac22640 1 --2- 172.21.15.38:0/1257252729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb040d0870 0x7ffb040d0ca0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009e50 tx=0x7ffafc02a680 comp rx=0 tx=0).stop 2024-04-25T08:39:05.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.632+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1257252729 shutdown_connections 2024-04-25T08:39:05.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.632+0000 7ffb0ac22640 1 --2- 172.21.15.38:0/1257252729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb040d0870 0x7ffb040d0ca0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.632+0000 7ffb0ac22640 1 --2- 172.21.15.38:0/1257252729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb041612c0 0x7ffb040b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.632+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1257252729 >> 172.21.15.38:0/1257252729 conn(0x7ffb04066d10 msgr2=0x7ffb04067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.632+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1257252729 shutdown_connections 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.632+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1257252729 wait complete. 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.633+0000 7ffb0ac22640 1 Processor -- start 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.634+0000 7ffb0ac22640 1 -- start start 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.634+0000 7ffb0ac22640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb040b6a00 0x7ffb040c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.634+0000 7ffb0ac22640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb040d0870 0x7ffb0415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.634+0000 7ffb0ac22640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb041612c0 0x7ffb041499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.634+0000 7ffb0ac22640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040b6e60 con 0x7ffb040b6a00 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.634+0000 7ffb0ac22640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040b7d80 con 0x7ffb041612c0 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.634+0000 7ffb0ac22640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb040b6fe0 con 0x7ffb040d0870 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.634+0000 7ffb089be640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb040b6a00 0x7ffb040c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb089be640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb040b6a00 0x7ffb040c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53754/0 (socket says 172.21.15.38:53754) 2024-04-25T08:39:05.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb089be640 1 -- 172.21.15.38:0/1189191854 learned_addr learned my addr 172.21.15.38:0/1189191854 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:05.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb091bf640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb041612c0 0x7ffb041499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb03fff640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb040d0870 0x7ffb0415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb089be640 1 -- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb040d0870 msgr2=0x7ffb0415c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb089be640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb040d0870 0x7ffb0415c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb089be640 1 -- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb041612c0 msgr2=0x7ffb041499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:05.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb089be640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb041612c0 0x7ffb041499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:05.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb089be640 1 -- 172.21.15.38:0/1189191854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb041f73c0 con 0x7ffb040b6a00 2024-04-25T08:39:05.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb091bf640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb041612c0 0x7ffb041499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:05.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb03fff640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb040d0870 0x7ffb0415c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:05.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb089be640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb040b6a00 0x7ffb040c9d60 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800e870 tx=0x7ffaf800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:05.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb017fa640 1 -- 172.21.15.38:0/1189191854 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8017070 con 0x7ffb040b6a00 2024-04-25T08:39:05.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb017fa640 1 -- 172.21.15.38:0/1189191854 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf800c940 con 0x7ffb040b6a00 2024-04-25T08:39:05.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1189191854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb041f76a0 con 0x7ffb040b6a00 2024-04-25T08:39:05.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb017fa640 1 -- 172.21.15.38:0/1189191854 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800cd30 con 0x7ffb040b6a00 2024-04-25T08:39:05.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.635+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1189191854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb041f7c60 con 0x7ffb040b6a00 2024-04-25T08:39:05.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.636+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1189191854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb040b6590 con 0x7ffb040b6a00 2024-04-25T08:39:05.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.637+0000 7ffb017fa640 1 -- 172.21.15.38:0/1189191854 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffaf801a070 con 0x7ffb040b6a00 2024-04-25T08:39:05.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.637+0000 7ffb017fa640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffae806b4d0 0x7ffae806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:05.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.637+0000 7ffb017fa640 1 -- 172.21.15.38:0/1189191854 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ffaf808a2c0 con 0x7ffb040b6a00 2024-04-25T08:39:05.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.637+0000 7ffb03fff640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffae806b4d0 0x7ffae806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:05.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.638+0000 7ffb03fff640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffae806b4d0 0x7ffae806d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffb040c6f70 tx=0x7ffaf0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:05.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.641+0000 7ffb017fa640 1 -- 172.21.15.38:0/1189191854 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb040b6590 con 0x7ffb040b6a00 2024-04-25T08:39:05.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:05.820+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1189191854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ffb04068e80 con 0x7ffb040b6a00 2024-04-25T08:39:06.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.173+0000 7ffb017fa640 1 -- 172.21.15.38:0/1189191854 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v105) v1 ==== 138+0+0 (secure 0 0 0) 0x7ffaf8059300 con 0x7ffb040b6a00 2024-04-25T08:39:06.167 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-04-25T08:39:06.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.176+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffae806b4d0 msgr2=0x7ffae806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:06.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.176+0000 7ffb0ac22640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffae806b4d0 0x7ffae806d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ffb040c6f70 tx=0x7ffaf0009040 comp rx=0 tx=0).stop 2024-04-25T08:39:06.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.176+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1189191854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb040b6a00 msgr2=0x7ffb040c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:06.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.176+0000 7ffb0ac22640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb040b6a00 0x7ffb040c9d60 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7ffaf800e870 tx=0x7ffaf800ed30 comp rx=0 tx=0).stop 2024-04-25T08:39:06.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.176+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1189191854 shutdown_connections 2024-04-25T08:39:06.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.176+0000 7ffb0ac22640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffb041612c0 0x7ffb041499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:06.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.176+0000 7ffb0ac22640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffae806b4d0 0x7ffae806d990 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:06.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.176+0000 7ffb0ac22640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffb040d0870 0x7ffb0415c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:06.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.176+0000 7ffb0ac22640 1 --2- 172.21.15.38:0/1189191854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb040b6a00 0x7ffb040c9d60 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:06.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.176+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1189191854 >> 172.21.15.38:0/1189191854 conn(0x7ffb04066d10 msgr2=0x7ffb041601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:06.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.176+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1189191854 shutdown_connections 2024-04-25T08:39:06.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.176+0000 7ffb0ac22640 1 -- 172.21.15.38:0/1189191854 wait complete. 2024-04-25T08:39:06.185 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:39:06.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.302+0000 7eff54522640 1 Processor -- start 2024-04-25T08:39:06.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.302+0000 7eff54522640 1 -- start start 2024-04-25T08:39:06.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff54522640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4c0d0320 0x7eff4c0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:06.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff54522640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7eff4c0d0700 0x7eff4c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:06.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff54522640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7eff4c0d1e90 0x7eff4c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:06.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff54522640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff4c0d2330 con 0x7eff4c0d0700 2024-04-25T08:39:06.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff54522640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff4c0d2610 con 0x7eff4c0d1e90 2024-04-25T08:39:06.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff54522640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c0d28f0 con 0x7eff4c0d0320 2024-04-25T08:39:06.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff51abd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7eff4c0d0700 0x7eff4c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:06.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff51abd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7eff4c0d0700 0x7eff4c0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53762/0 (socket says 172.21.15.38:53762) 2024-04-25T08:39:06.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff51abd640 1 -- 172.21.15.38:0/3978862687 learned_addr learned my addr 172.21.15.38:0/3978862687 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:06.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff522be640 1 --2- 172.21.15.38:0/3978862687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4c0d0320 0x7eff4c0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:06.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff52abf640 1 --2- 172.21.15.38:0/3978862687 >> v2:172.21.15.107:3300/0 conn(0x7eff4c0d1e90 0x7eff4c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:06.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff51abd640 1 -- 172.21.15.38:0/3978862687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4c0d0320 msgr2=0x7eff4c0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:06.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff51abd640 1 --2- 172.21.15.38:0/3978862687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4c0d0320 0x7eff4c0d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:06.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff51abd640 1 -- 172.21.15.38:0/3978862687 >> v2:172.21.15.107:3300/0 conn(0x7eff4c0d1e90 msgr2=0x7eff4c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:06.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff51abd640 1 --2- 172.21.15.38:0/3978862687 >> v2:172.21.15.107:3300/0 conn(0x7eff4c0d1e90 0x7eff4c0b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:06.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff51abd640 1 -- 172.21.15.38:0/3978862687 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c0d2bd0 con 0x7eff4c0d0700 2024-04-25T08:39:06.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff522be640 1 --2- 172.21.15.38:0/3978862687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4c0d0320 0x7eff4c0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:06.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.303+0000 7eff52abf640 1 --2- 172.21.15.38:0/3978862687 >> v2:172.21.15.107:3300/0 conn(0x7eff4c0d1e90 0x7eff4c0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:06.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.304+0000 7eff51abd640 1 --2- 172.21.15.38:0/3978862687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4c0d0700 0x7eff4c0d1950 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0b68d0 tx=0x7eff3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896407bee6e352af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:06.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.304+0000 7eff512bc640 1 -- 172.21.15.38:0/3978862687 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c037070 con 0x7eff4c0d0700 2024-04-25T08:39:06.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.304+0000 7eff512bc640 1 -- 172.21.15.38:0/3978862687 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c032440 con 0x7eff4c0d0700 2024-04-25T08:39:06.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.304+0000 7eff512bc640 1 -- 172.21.15.38:0/3978862687 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c032810 con 0x7eff4c0d0700 2024-04-25T08:39:06.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.305+0000 7eff54522640 1 -- 172.21.15.38:0/3978862687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4c0d0700 msgr2=0x7eff4c0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:06.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.305+0000 7eff54522640 1 --2- 172.21.15.38:0/3978862687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4c0d0700 0x7eff4c0d1950 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7eff4c0b68d0 tx=0x7eff3c02a270 comp rx=0 tx=0).stop 2024-04-25T08:39:06.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.305+0000 7eff54522640 1 -- 172.21.15.38:0/3978862687 shutdown_connections 2024-04-25T08:39:06.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.305+0000 7eff54522640 1 --2- 172.21.15.38:0/3978862687 >> v2:172.21.15.107:3300/0 conn(0x7eff4c0d1e90 0x7eff4c0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:06.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.305+0000 7eff54522640 1 --2- 172.21.15.38:0/3978862687 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4c0d0700 0x7eff4c0d1950 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:06.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.305+0000 7eff54522640 1 --2- 172.21.15.38:0/3978862687 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4c0d0320 0x7eff4c0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:06.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.305+0000 7eff54522640 1 -- 172.21.15.38:0/3978862687 >> 172.21.15.38:0/3978862687 conn(0x7eff4c0668f0 msgr2=0x7eff4c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:06.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.305+0000 7eff54522640 1 -- 172.21.15.38:0/3978862687 shutdown_connections 2024-04-25T08:39:06.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.305+0000 7eff54522640 1 -- 172.21.15.38:0/3978862687 wait complete. 2024-04-25T08:39:06.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.306+0000 7eff54522640 1 Processor -- start 2024-04-25T08:39:06.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.306+0000 7eff54522640 1 -- start start 2024-04-25T08:39:06.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff54522640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4c0d0320 0x7eff4c151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff54522640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4c0d0700 0x7eff4c0b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff54522640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4c0d1e90 0x7eff4c0b7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff54522640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c0d0b40 con 0x7eff4c0d0320 2024-04-25T08:39:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff54522640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c161210 con 0x7eff4c0d1e90 2024-04-25T08:39:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff54522640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c1522b0 con 0x7eff4c0d0700 2024-04-25T08:39:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff522be640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4c0d0320 0x7eff4c151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff522be640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4c0d0320 0x7eff4c151d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53764/0 (socket says 172.21.15.38:53764) 2024-04-25T08:39:06.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff522be640 1 -- 172.21.15.38:0/2676517996 learned_addr learned my addr 172.21.15.38:0/2676517996 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:06.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff51abd640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4c0d0700 0x7eff4c0b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:06.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff52abf640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4c0d1e90 0x7eff4c0b7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:06.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff522be640 1 -- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4c0d0700 msgr2=0x7eff4c0b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:06.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff522be640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4c0d0700 0x7eff4c0b7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:06.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff522be640 1 -- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4c0d1e90 msgr2=0x7eff4c0b7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff522be640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4c0d1e90 0x7eff4c0b7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff522be640 1 -- 172.21.15.38:0/2676517996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c0b52d0 con 0x7eff4c0d0320 2024-04-25T08:39:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff522be640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4c0d0320 0x7eff4c151d70 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7eff4800c7c0 tx=0x7eff4800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff42ffd640 1 -- 172.21.15.38:0/2676517996 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48015070 con 0x7eff4c0d0320 2024-04-25T08:39:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff42ffd640 1 -- 172.21.15.38:0/2676517996 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff48007a30 con 0x7eff4c0d0320 2024-04-25T08:39:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.307+0000 7eff54522640 1 -- 172.21.15.38:0/2676517996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4c0b55b0 con 0x7eff4c0d0320 2024-04-25T08:39:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.308+0000 7eff42ffd640 1 -- 172.21.15.38:0/2676517996 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff48007e00 con 0x7eff4c0d0320 2024-04-25T08:39:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.308+0000 7eff54522640 1 -- 172.21.15.38:0/2676517996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff4c0b5b70 con 0x7eff4c0d0320 2024-04-25T08:39:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.308+0000 7eff54522640 1 -- 172.21.15.38:0/2676517996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4c1f3990 con 0x7eff4c0d0320 2024-04-25T08:39:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.309+0000 7eff42ffd640 1 -- 172.21.15.38:0/2676517996 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff48018070 con 0x7eff4c0d0320 2024-04-25T08:39:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.309+0000 7eff42ffd640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7eff3006b4d0 0x7eff3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.310+0000 7eff42ffd640 1 -- 172.21.15.38:0/2676517996 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5619+0+0 (secure 0 0 0) 0x7eff48089fe0 con 0x7eff4c0d0320 2024-04-25T08:39:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.310+0000 7eff51abd640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7eff3006b4d0 0x7eff3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.310+0000 7eff51abd640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7eff3006b4d0 0x7eff3006d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7eff3c034000 tx=0x7eff3c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:06.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.314+0000 7eff42ffd640 1 -- 172.21.15.38:0/2676517996 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff480588f0 con 0x7eff4c0d0320 2024-04-25T08:39:06.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:06.496+0000 7eff54522640 1 -- 172.21.15.38:0/2676517996 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7eff4c068e80 con 0x7eff4c0d0320 2024-04-25T08:39:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.176+0000 7eff42ffd640 1 -- 172.21.15.38:0/2676517996 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v106) v1 ==== 170+0+0 (secure 0 0 0) 0x7eff4805c990 con 0x7eff4c0d0320 2024-04-25T08:39:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-04-25T08:39:07.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.179+0000 7eff54522640 1 -- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7eff3006b4d0 msgr2=0x7eff3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:07.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.179+0000 7eff54522640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7eff3006b4d0 0x7eff3006d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7eff3c034000 tx=0x7eff3c02e000 comp rx=0 tx=0).stop 2024-04-25T08:39:07.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.179+0000 7eff54522640 1 -- 172.21.15.38:0/2676517996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4c0d0320 msgr2=0x7eff4c151d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:07.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.179+0000 7eff54522640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4c0d0320 0x7eff4c151d70 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7eff4800c7c0 tx=0x7eff4800cc80 comp rx=0 tx=0).stop 2024-04-25T08:39:07.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.180+0000 7eff54522640 1 -- 172.21.15.38:0/2676517996 shutdown_connections 2024-04-25T08:39:07.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.180+0000 7eff54522640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff4c0d1e90 0x7eff4c0b7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:07.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.180+0000 7eff54522640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7eff3006b4d0 0x7eff3006d990 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:07.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.180+0000 7eff54522640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff4c0d0700 0x7eff4c0b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:07.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.180+0000 7eff54522640 1 --2- 172.21.15.38:0/2676517996 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff4c0d0320 0x7eff4c151d70 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:07.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.180+0000 7eff54522640 1 -- 172.21.15.38:0/2676517996 >> 172.21.15.38:0/2676517996 conn(0x7eff4c0668f0 msgr2=0x7eff4c15f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:07.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.180+0000 7eff54522640 1 -- 172.21.15.38:0/2676517996 shutdown_connections 2024-04-25T08:39:07.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.180+0000 7eff54522640 1 -- 172.21.15.38:0/2676517996 wait complete. 2024-04-25T08:39:07.204 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:39:07.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.320+0000 7fdd8c81c640 1 Processor -- start 2024-04-25T08:39:07.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.320+0000 7fdd8c81c640 1 -- start start 2024-04-25T08:39:07.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd8c81c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdd880d0320 0x7fdd880d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd8c81c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fdd880b6980 0x7fdd880b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd8c81c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd880d2060 con 0x7fdd880d0700 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd8c81c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd880d2330 con 0x7fdd880d0320 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd8c81c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd880d2610 con 0x7fdd880b6980 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd8659c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdd880d0320 0x7fdd880d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd7ffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fdd880d0700 0x7fdd880d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:37108/0 (socket says 172.21.15.38:37108) 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd7ffff640 1 -- 172.21.15.38:0/4053199903 learned_addr learned my addr 172.21.15.38:0/4053199903 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd86d9d640 1 --2- 172.21.15.38:0/4053199903 >> v2:172.21.15.107:3300/0 conn(0x7fdd880b6980 0x7fdd880b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd8659c640 1 -- 172.21.15.38:0/4053199903 >> v2:172.21.15.107:3300/0 conn(0x7fdd880b6980 msgr2=0x7fdd880b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd8659c640 1 --2- 172.21.15.38:0/4053199903 >> v2:172.21.15.107:3300/0 conn(0x7fdd880b6980 0x7fdd880b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd8659c640 1 -- 172.21.15.38:0/4053199903 >> v1:172.21.15.38:6789/0 conn(0x7fdd880d0700 legacy=0x7fdd880d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd8659c640 1 -- 172.21.15.38:0/4053199903 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd880d28f0 con 0x7fdd880d0320 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.321+0000 7fdd85d9b640 1 -- 172.21.15.38:0/4053199903 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250833551 0 0) 0x7fdd880d2060 con 0x7fdd880d0700 2024-04-25T08:39:07.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.322+0000 7fdd8659c640 1 --2- 172.21.15.38:0/4053199903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd880d0320 0x7fdd880d33d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fdd88067590 tx=0x7fdd7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8594026f5ac02a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.322+0000 7fdd85d9b640 1 -- 172.21.15.38:0/4053199903 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78037070 con 0x7fdd880d0320 2024-04-25T08:39:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.322+0000 7fdd85d9b640 1 -- 172.21.15.38:0/4053199903 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd78032440 con 0x7fdd880d0320 2024-04-25T08:39:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.322+0000 7fdd85d9b640 1 -- 172.21.15.38:0/4053199903 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78032810 con 0x7fdd880d0320 2024-04-25T08:39:07.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.323+0000 7fdd8c81c640 1 -- 172.21.15.38:0/4053199903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd880d0320 msgr2=0x7fdd880d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:07.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.323+0000 7fdd8c81c640 1 --2- 172.21.15.38:0/4053199903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd880d0320 0x7fdd880d33d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fdd88067590 tx=0x7fdd7802a230 comp rx=0 tx=0).stop 2024-04-25T08:39:07.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.323+0000 7fdd8c81c640 1 -- 172.21.15.38:0/4053199903 shutdown_connections 2024-04-25T08:39:07.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.323+0000 7fdd8c81c640 1 --2- 172.21.15.38:0/4053199903 >> v2:172.21.15.107:3300/0 conn(0x7fdd880b6980 0x7fdd880b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:07.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.323+0000 7fdd8c81c640 1 --2- 172.21.15.38:0/4053199903 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd880d0320 0x7fdd880d33d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:07.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.323+0000 7fdd8c81c640 1 -- 172.21.15.38:0/4053199903 >> 172.21.15.38:0/4053199903 conn(0x7fdd880668f0 msgr2=0x7fdd88066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:07.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.323+0000 7fdd8c81c640 1 -- 172.21.15.38:0/4053199903 shutdown_connections 2024-04-25T08:39:07.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.323+0000 7fdd8c81c640 1 -- 172.21.15.38:0/4053199903 wait complete. 2024-04-25T08:39:07.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.324+0000 7fdd8c81c640 1 Processor -- start 2024-04-25T08:39:07.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.324+0000 7fdd8c81c640 1 -- start start 2024-04-25T08:39:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8c81c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd880b6980 0x7fdd8815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8c81c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd880d0320 0x7fdd880aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8c81c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd880d0700 0x7fdd880aa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8c81c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880d0b40 con 0x7fdd880b6980 2024-04-25T08:39:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8c81c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd880d2eb0 con 0x7fdd880d0320 2024-04-25T08:39:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8c81c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8814d8b0 con 0x7fdd880d0700 2024-04-25T08:39:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd880b6980 0x7fdd8815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd880b6980 0x7fdd8815ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53780/0 (socket says 172.21.15.38:53780) 2024-04-25T08:39:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8659c640 1 -- 172.21.15.38:0/677603847 learned_addr learned my addr 172.21.15.38:0/677603847 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8659c640 1 -- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd880d0700 msgr2=0x7fdd880aa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8659c640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd880d0700 0x7fdd880aa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:07.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8659c640 1 -- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd880d0320 msgr2=0x7fdd880aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:07.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd7ffff640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd880d0320 0x7fdd880aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:07.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8659c640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd880d0320 0x7fdd880aa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:07.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.325+0000 7fdd8659c640 1 -- 172.21.15.38:0/677603847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8814db40 con 0x7fdd880b6980 2024-04-25T08:39:07.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.326+0000 7fdd8659c640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd880b6980 0x7fdd8815ec80 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fdd78000c00 tx=0x7fdd780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:07.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.326+0000 7fdd7effd640 1 -- 172.21.15.38:0/677603847 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78037040 con 0x7fdd880b6980 2024-04-25T08:39:07.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.326+0000 7fdd7effd640 1 -- 172.21.15.38:0/677603847 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd78042070 con 0x7fdd880b6980 2024-04-25T08:39:07.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.326+0000 7fdd8c81c640 1 -- 172.21.15.38:0/677603847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd880aad20 con 0x7fdd880b6980 2024-04-25T08:39:07.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.326+0000 7fdd7effd640 1 -- 172.21.15.38:0/677603847 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78030bd0 con 0x7fdd880b6980 2024-04-25T08:39:07.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.326+0000 7fdd8c81c640 1 -- 172.21.15.38:0/677603847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd880aafb0 con 0x7fdd880b6980 2024-04-25T08:39:07.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.327+0000 7fdd8c81c640 1 -- 172.21.15.38:0/677603847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd88068e80 con 0x7fdd880b6980 2024-04-25T08:39:07.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.327+0000 7fdd7effd640 1 -- 172.21.15.38:0/677603847 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd78047460 con 0x7fdd880b6980 2024-04-25T08:39:07.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.328+0000 7fdd7effd640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6806b4d0 0x7fdd6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:07.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.328+0000 7fdd7effd640 1 -- 172.21.15.38:0/677603847 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fdd780aafc0 con 0x7fdd880b6980 2024-04-25T08:39:07.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.328+0000 7fdd7ffff640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6806b4d0 0x7fdd6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:07.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.329+0000 7fdd7ffff640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6806b4d0 0x7fdd6806d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdd880ca2b0 tx=0x7fdd70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:07.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.333+0000 7fdd7effd640 1 -- 172.21.15.38:0/677603847 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd8814db40 con 0x7fdd880b6980 2024-04-25T08:39:07.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:07.550+0000 7fdd8c81c640 1 -- 172.21.15.38:0/677603847 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdd8806ae90 con 0x7fdd880b6980 2024-04-25T08:39:08.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.201+0000 7fdd7effd640 1 -- 172.21.15.38:0/677603847 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v61) v1 ==== 174+0+0 (secure 0 0 0) 0x7fdd78032e20 con 0x7fdd880b6980 2024-04-25T08:39:08.200 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T08:39:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.213+0000 7fdd8c81c640 1 -- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6806b4d0 msgr2=0x7fdd6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.213+0000 7fdd8c81c640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6806b4d0 0x7fdd6806d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fdd880ca2b0 tx=0x7fdd70009040 comp rx=0 tx=0).stop 2024-04-25T08:39:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.213+0000 7fdd8c81c640 1 -- 172.21.15.38:0/677603847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd880b6980 msgr2=0x7fdd8815ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.213+0000 7fdd8c81c640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd880b6980 0x7fdd8815ec80 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fdd78000c00 tx=0x7fdd780046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.213+0000 7fdd8c81c640 1 -- 172.21.15.38:0/677603847 shutdown_connections 2024-04-25T08:39:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.213+0000 7fdd8c81c640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdd6806b4d0 0x7fdd6806d990 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.213+0000 7fdd8c81c640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdd880d0700 0x7fdd880aa7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.213+0000 7fdd8c81c640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd880d0320 0x7fdd880aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.213+0000 7fdd8c81c640 1 --2- 172.21.15.38:0/677603847 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd880b6980 0x7fdd8815ec80 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.213+0000 7fdd8c81c640 1 -- 172.21.15.38:0/677603847 >> 172.21.15.38:0/677603847 conn(0x7fdd880668f0 msgr2=0x7fdd8815fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.217+0000 7fdd8c81c640 1 -- 172.21.15.38:0/677603847 shutdown_connections 2024-04-25T08:39:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.217+0000 7fdd8c81c640 1 -- 172.21.15.38:0/677603847 wait complete. 2024-04-25T08:39:08.234 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:08.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.346+0000 7f08ecdf4640 1 Processor -- start 2024-04-25T08:39:08.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.346+0000 7f08ecdf4640 1 -- start start 2024-04-25T08:39:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.347+0000 7f08ecdf4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f08e80ceca0 0x7f08e80d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.347+0000 7f08ecdf4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f08e80d3de0 0x7f08e80b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.347+0000 7f08ecdf4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e80d1b60 con 0x7f08e815da90 2024-04-25T08:39:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.347+0000 7f08ecdf4640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e80d4250 con 0x7f08e80ceca0 2024-04-25T08:39:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.347+0000 7f08ecdf4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e80d2bc0 con 0x7f08e80d3de0 2024-04-25T08:39:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.348+0000 7f08e5d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f08e80ceca0 0x7f08e80d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.348+0000 7f08e5d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f08e80ceca0 0x7f08e80d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53784/0 (socket says 172.21.15.38:53784) 2024-04-25T08:39:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.348+0000 7f08e5d9b640 1 -- 172.21.15.38:0/46071488 learned_addr learned my addr 172.21.15.38:0/46071488 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.348+0000 7f08e6d9d640 1 --2- 172.21.15.38:0/46071488 >> v2:172.21.15.107:3300/0 conn(0x7f08e80d3de0 0x7f08e80b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.348+0000 7f08e5d9b640 1 -- 172.21.15.38:0/46071488 >> v2:172.21.15.107:3300/0 conn(0x7f08e80d3de0 msgr2=0x7f08e80b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.348+0000 7f08e5d9b640 1 --2- 172.21.15.38:0/46071488 >> v2:172.21.15.107:3300/0 conn(0x7f08e80d3de0 0x7f08e80b69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.348+0000 7f08e5d9b640 1 -- 172.21.15.38:0/46071488 >> v1:172.21.15.107:6789/0 conn(0x7f08e815da90 legacy=0x7f08e80d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.348+0000 7f08e5d9b640 1 -- 172.21.15.38:0/46071488 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e80d3680 con 0x7f08e80ceca0 2024-04-25T08:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.349+0000 7f08e5d9b640 1 --2- 172.21.15.38:0/46071488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08e80ceca0 0x7f08e80d38a0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f08e80c7410 tx=0x7f08dc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65faf946681cd2bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.349+0000 7f08e559a640 1 -- 172.21.15.38:0/46071488 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc037070 con 0x7f08e80ceca0 2024-04-25T08:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.349+0000 7f08e559a640 1 -- 172.21.15.38:0/46071488 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc032440 con 0x7f08e80ceca0 2024-04-25T08:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.349+0000 7f08e559a640 1 -- 172.21.15.38:0/46071488 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc032810 con 0x7f08e80ceca0 2024-04-25T08:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.349+0000 7f08ecdf4640 1 -- 172.21.15.38:0/46071488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08e80ceca0 msgr2=0x7f08e80d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.349+0000 7f08ecdf4640 1 --2- 172.21.15.38:0/46071488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08e80ceca0 0x7f08e80d38a0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f08e80c7410 tx=0x7f08dc02a270 comp rx=0 tx=0).stop 2024-04-25T08:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.349+0000 7f08ecdf4640 1 -- 172.21.15.38:0/46071488 shutdown_connections 2024-04-25T08:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.349+0000 7f08ecdf4640 1 --2- 172.21.15.38:0/46071488 >> v2:172.21.15.107:3300/0 conn(0x7f08e80d3de0 0x7f08e80b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.349+0000 7f08ecdf4640 1 --2- 172.21.15.38:0/46071488 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08e80ceca0 0x7f08e80d38a0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.349+0000 7f08ecdf4640 1 -- 172.21.15.38:0/46071488 >> 172.21.15.38:0/46071488 conn(0x7f08e806bed0 msgr2=0x7f08e806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.349+0000 7f08ecdf4640 1 -- 172.21.15.38:0/46071488 shutdown_connections 2024-04-25T08:39:08.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.349+0000 7f08ecdf4640 1 -- 172.21.15.38:0/46071488 wait complete. 2024-04-25T08:39:08.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.351+0000 7f08ecdf4640 1 Processor -- start 2024-04-25T08:39:08.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.351+0000 7f08ecdf4640 1 -- start start 2024-04-25T08:39:08.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.351+0000 7f08ecdf4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08e80d3de0 0x7f08e80ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.351+0000 7f08ecdf4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08e815da90 0x7f08e8156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.351+0000 7f08ecdf4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08e8145c60 0x7f08e8146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.351+0000 7f08ecdf4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e80d23c0 con 0x7f08e80d3de0 2024-04-25T08:39:08.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.351+0000 7f08ecdf4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e80d20e0 con 0x7f08e8145c60 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.351+0000 7f08ecdf4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e815de70 con 0x7f08e815da90 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08e80d3de0 0x7f08e80ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08e80d3de0 0x7f08e80ccf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53798/0 (socket says 172.21.15.38:53798) 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e659c640 1 -- 172.21.15.38:0/3696660577 learned_addr learned my addr 172.21.15.38:0/3696660577 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e5d9b640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08e815da90 0x7f08e8156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e6d9d640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08e8145c60 0x7f08e8146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e659c640 1 -- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08e815da90 msgr2=0x7f08e8156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e659c640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08e815da90 0x7f08e8156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e659c640 1 -- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08e8145c60 msgr2=0x7f08e8146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e659c640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08e8145c60 0x7f08e8146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e659c640 1 -- 172.21.15.38:0/3696660577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e80cbea0 con 0x7f08e80d3de0 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e5d9b640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08e815da90 0x7f08e8156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:08.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e6d9d640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08e8145c60 0x7f08e8146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08e659c640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08e80d3de0 0x7f08e80ccf40 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f08d800c340 tx=0x7f08d800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08d6ffd640 1 -- 172.21.15.38:0/3696660577 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8017070 con 0x7f08e80d3de0 2024-04-25T08:39:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08d6ffd640 1 -- 172.21.15.38:0/3696660577 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d80052d0 con 0x7f08e80d3de0 2024-04-25T08:39:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08ecdf4640 1 -- 172.21.15.38:0/3696660577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e80cc180 con 0x7f08e80d3de0 2024-04-25T08:39:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08d6ffd640 1 -- 172.21.15.38:0/3696660577 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d8003850 con 0x7f08e80d3de0 2024-04-25T08:39:08.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.352+0000 7f08ecdf4640 1 -- 172.21.15.38:0/3696660577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e80cc740 con 0x7f08e80d3de0 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.354+0000 7f08ecdf4640 1 -- 172.21.15.38:0/3696660577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e8068e80 con 0x7f08e80d3de0 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.354+0000 7f08d6ffd640 1 -- 172.21.15.38:0/3696660577 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08d801a070 con 0x7f08e80d3de0 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.355+0000 7f08d6ffd640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f08c806b4d0 0x7f08c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.355+0000 7f08d6ffd640 1 -- 172.21.15.38:0/3696660577 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f08d80895a0 con 0x7f08e80d3de0 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.355+0000 7f08e5d9b640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f08c806b4d0 0x7f08c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.355+0000 7f08e5d9b640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f08c806b4d0 0x7f08c806d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f08dc034000 tx=0x7f08dc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.359+0000 7f08d6ffd640 1 -- 172.21.15.38:0/3696660577 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08d8057c40 con 0x7f08e80d3de0 2024-04-25T08:39:08.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.539+0000 7f08ecdf4640 1 -- 172.21.15.38:0/3696660577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08e80d0ef0 con 0x7f08e80d3de0 2024-04-25T08:39:08.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.541+0000 7f08d6ffd640 1 -- 172.21.15.38:0/3696660577 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f08d805bce0 con 0x7f08e80d3de0 2024-04-25T08:39:08.534 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:08.534 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":107,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:07.551343+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:05.852531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:06.527775+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.543+0000 7f08ecdf4640 1 -- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f08c806b4d0 msgr2=0x7f08c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.543+0000 7f08ecdf4640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f08c806b4d0 0x7f08c806d990 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f08dc034000 tx=0x7f08dc02e000 comp rx=0 tx=0).stop 2024-04-25T08:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.543+0000 7f08ecdf4640 1 -- 172.21.15.38:0/3696660577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08e80d3de0 msgr2=0x7f08e80ccf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.543+0000 7f08ecdf4640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08e80d3de0 0x7f08e80ccf40 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f08d800c340 tx=0x7f08d800c800 comp rx=0 tx=0).stop 2024-04-25T08:39:08.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.543+0000 7f08ecdf4640 1 -- 172.21.15.38:0/3696660577 shutdown_connections 2024-04-25T08:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.543+0000 7f08ecdf4640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08e8145c60 0x7f08e8146110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.543+0000 7f08ecdf4640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f08c806b4d0 0x7f08c806d990 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.543+0000 7f08ecdf4640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f08e815da90 0x7f08e8156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.543+0000 7f08ecdf4640 1 --2- 172.21.15.38:0/3696660577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f08e80d3de0 0x7f08e80ccf40 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.543+0000 7f08ecdf4640 1 -- 172.21.15.38:0/3696660577 >> 172.21.15.38:0/3696660577 conn(0x7f08e806bed0 msgr2=0x7f08e815cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.543+0000 7f08ecdf4640 1 -- 172.21.15.38:0/3696660577 shutdown_connections 2024-04-25T08:39:08.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.543+0000 7f08ecdf4640 1 -- 172.21.15.38:0/3696660577 wait complete. 2024-04-25T08:39:08.548 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:08.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.655+0000 7f325f26b640 1 Processor -- start 2024-04-25T08:39:08.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.655+0000 7f325f26b640 1 -- start start 2024-04-25T08:39:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325f26b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f325809acb0 0x7f32580cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325f26b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f325806cdf0 0x7f32580cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325f26b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3258161540 con 0x7f32580cd8d0 2024-04-25T08:39:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325f26b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f325809a820 con 0x7f325809acb0 2024-04-25T08:39:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325f26b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f325806be90 con 0x7f325806cdf0 2024-04-25T08:39:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325c806640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f325809acb0 0x7f32580cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325c806640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f325809acb0 0x7f32580cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53810/0 (socket says 172.21.15.38:53810) 2024-04-25T08:39:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325c806640 1 -- 172.21.15.38:0/767466403 learned_addr learned my addr 172.21.15.38:0/767466403 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325d808640 1 --2- 172.21.15.38:0/767466403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f325806cdf0 0x7f32580cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325c806640 1 -- 172.21.15.38:0/767466403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f325806cdf0 msgr2=0x7f32580cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:08.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325c806640 1 --2- 172.21.15.38:0/767466403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f325806cdf0 0x7f32580cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325c806640 1 -- 172.21.15.38:0/767466403 >> v1:172.21.15.107:6789/0 conn(0x7f32580cd8d0 legacy=0x7f325806c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:08.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.656+0000 7f325c806640 1 -- 172.21.15.38:0/767466403 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325809a520 con 0x7f325809acb0 2024-04-25T08:39:08.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.657+0000 7f325c806640 1 --2- 172.21.15.38:0/767466403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f325809acb0 0x7f32580cbcc0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f324c009870 tx=0x7f324c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7b9ed9d4093cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.657+0000 7f324bfff640 1 -- 172.21.15.38:0/767466403 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c037070 con 0x7f325809acb0 2024-04-25T08:39:08.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.657+0000 7f324bfff640 1 -- 172.21.15.38:0/767466403 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f324c032440 con 0x7f325809acb0 2024-04-25T08:39:08.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.657+0000 7f324bfff640 1 -- 172.21.15.38:0/767466403 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f324c032810 con 0x7f325809acb0 2024-04-25T08:39:08.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.658+0000 7f325f26b640 1 -- 172.21.15.38:0/767466403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f325809acb0 msgr2=0x7f32580cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.658+0000 7f325f26b640 1 --2- 172.21.15.38:0/767466403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f325809acb0 0x7f32580cbcc0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f324c009870 tx=0x7f324c02a210 comp rx=0 tx=0).stop 2024-04-25T08:39:08.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.658+0000 7f325f26b640 1 -- 172.21.15.38:0/767466403 shutdown_connections 2024-04-25T08:39:08.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.658+0000 7f325f26b640 1 --2- 172.21.15.38:0/767466403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f325806cdf0 0x7f32580cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.658+0000 7f325f26b640 1 --2- 172.21.15.38:0/767466403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f325809acb0 0x7f32580cbcc0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.658+0000 7f325f26b640 1 -- 172.21.15.38:0/767466403 >> 172.21.15.38:0/767466403 conn(0x7f32580b3160 msgr2=0x7f32580b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:08.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.658+0000 7f325f26b640 1 -- 172.21.15.38:0/767466403 shutdown_connections 2024-04-25T08:39:08.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.658+0000 7f325f26b640 1 -- 172.21.15.38:0/767466403 wait complete. 2024-04-25T08:39:08.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.659+0000 7f325f26b640 1 Processor -- start 2024-04-25T08:39:08.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.659+0000 7f325f26b640 1 -- start start 2024-04-25T08:39:08.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325f26b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f325806cdf0 0x7f32580aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325f26b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325809acb0 0x7f32580cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325f26b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32580cd8d0 0x7f325814f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325f26b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325806c410 con 0x7f32580cd8d0 2024-04-25T08:39:08.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325f26b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32580d0ad0 con 0x7f325809acb0 2024-04-25T08:39:08.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325f26b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32580b2170 con 0x7f325806cdf0 2024-04-25T08:39:08.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325d808640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32580cd8d0 0x7f325814f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325d808640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32580cd8d0 0x7f325814f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53814/0 (socket says 172.21.15.38:53814) 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325d808640 1 -- 172.21.15.38:0/288200267 learned_addr learned my addr 172.21.15.38:0/288200267 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325c806640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325809acb0 0x7f32580cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325d007640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f325806cdf0 0x7f32580aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325d808640 1 -- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f325806cdf0 msgr2=0x7f32580aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325d808640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f325806cdf0 0x7f32580aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325d808640 1 -- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325809acb0 msgr2=0x7f32580cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325d808640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325809acb0 0x7f32580cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325d808640 1 -- 172.21.15.38:0/288200267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32580aff10 con 0x7f32580cd8d0 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325d007640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f325806cdf0 0x7f32580aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.660+0000 7f325d808640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32580cd8d0 0x7f325814f630 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f325000cdc0 tx=0x7f325000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.661+0000 7f3249ffb640 1 -- 172.21.15.38:0/288200267 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3250003b90 con 0x7f32580cd8d0 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.661+0000 7f3249ffb640 1 -- 172.21.15.38:0/288200267 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3250003d50 con 0x7f32580cd8d0 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.661+0000 7f325f26b640 1 -- 172.21.15.38:0/288200267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32580b0210 con 0x7f32580cd8d0 2024-04-25T08:39:08.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.661+0000 7f3249ffb640 1 -- 172.21.15.38:0/288200267 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3250004260 con 0x7f32580cd8d0 2024-04-25T08:39:08.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.661+0000 7f325f26b640 1 -- 172.21.15.38:0/288200267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32580b0a50 con 0x7f32580cd8d0 2024-04-25T08:39:08.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.662+0000 7f325f26b640 1 -- 172.21.15.38:0/288200267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3258068e80 con 0x7f32580cd8d0 2024-04-25T08:39:08.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.662+0000 7f3249ffb640 1 -- 172.21.15.38:0/288200267 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3250004400 con 0x7f32580cd8d0 2024-04-25T08:39:08.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.663+0000 7f3249ffb640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f323c06b4d0 0x7f323c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.663+0000 7f3249ffb640 1 -- 172.21.15.38:0/288200267 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f325008ae80 con 0x7f32580cd8d0 2024-04-25T08:39:08.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.663+0000 7f325d007640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f323c06b4d0 0x7f323c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.663+0000 7f325d007640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f323c06b4d0 0x7f323c06d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3254006fd0 tx=0x7f3254008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.667+0000 7f3249ffb640 1 -- 172.21.15.38:0/288200267 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32580b0210 con 0x7f32580cd8d0 2024-04-25T08:39:08.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.849+0000 7f325f26b640 1 -- 172.21.15.38:0/288200267 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32580b7210 con 0x7f32580cd8d0 2024-04-25T08:39:08.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.850+0000 7f3249ffb640 1 -- 172.21.15.38:0/288200267 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16885 (secure 0 0 0) 0x7f3250059ad0 con 0x7f32580cd8d0 2024-04-25T08:39:08.844 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:08.844 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":107,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:07.551343+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:05.852531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:06.527775+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:08.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.852+0000 7f325f26b640 1 -- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f323c06b4d0 msgr2=0x7f323c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.852+0000 7f325f26b640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f323c06b4d0 0x7f323c06d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3254006fd0 tx=0x7f3254008040 comp rx=0 tx=0).stop 2024-04-25T08:39:08.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.852+0000 7f325f26b640 1 -- 172.21.15.38:0/288200267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32580cd8d0 msgr2=0x7f325814f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.852+0000 7f325f26b640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32580cd8d0 0x7f325814f630 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f325000cdc0 tx=0x7f325000a660 comp rx=0 tx=0).stop 2024-04-25T08:39:08.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.852+0000 7f325f26b640 1 -- 172.21.15.38:0/288200267 shutdown_connections 2024-04-25T08:39:08.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.852+0000 7f325f26b640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32580cd8d0 0x7f325814f630 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.852+0000 7f325f26b640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f325809acb0 0x7f32580cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.852+0000 7f325f26b640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f323c06b4d0 0x7f323c06d990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.852+0000 7f325f26b640 1 --2- 172.21.15.38:0/288200267 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f325806cdf0 0x7f32580aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.852+0000 7f325f26b640 1 -- 172.21.15.38:0/288200267 >> 172.21.15.38:0/288200267 conn(0x7f32580b3160 msgr2=0x7f32580cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:08.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.852+0000 7f325f26b640 1 -- 172.21.15.38:0/288200267 shutdown_connections 2024-04-25T08:39:08.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.852+0000 7f325f26b640 1 -- 172.21.15.38:0/288200267 wait complete. 2024-04-25T08:39:08.857 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:39:08.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.967+0000 7f5e43bf5640 1 Processor -- start 2024-04-25T08:39:08.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.968+0000 7f5e43bf5640 1 -- start start 2024-04-25T08:39:08.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.968+0000 7f5e43bf5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5e3c15e750 0x7f5e3c0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.968+0000 7f5e43bf5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c15ec20 con 0x7f5e3c06c700 2024-04-25T08:39:08.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.968+0000 7f5e43bf5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c15ef00 con 0x7f5e3c162110 2024-04-25T08:39:08.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.968+0000 7f5e43bf5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c161c80 con 0x7f5e3c15e750 2024-04-25T08:39:08.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.969+0000 7f5e42192640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5e3c15e750 0x7f5e3c0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.969+0000 7f5e42192640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5e3c15e750 0x7f5e3c0cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44952/0 (socket says 172.21.15.38:44952) 2024-04-25T08:39:08.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.969+0000 7f5e42192640 1 -- 172.21.15.38:0/3238199991 learned_addr learned my addr 172.21.15.38:0/3238199991 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:08.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.969+0000 7f5e41991640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5e3c06c700 0x7f5e3c06cae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43548/0 (socket says 172.21.15.38:43548) 2024-04-25T08:39:08.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.969+0000 7f5e42192640 1 -- 172.21.15.38:0/3238199991 >> v1:172.21.15.107:6789/0 conn(0x7f5e3c162110 legacy=0x7f5e3c15e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:08.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.969+0000 7f5e42192640 1 -- 172.21.15.38:0/3238199991 >> v1:172.21.15.38:6789/0 conn(0x7f5e3c06c700 legacy=0x7f5e3c06cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.969+0000 7f5e42192640 1 -- 172.21.15.38:0/3238199991 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c161630 con 0x7f5e3c15e750 2024-04-25T08:39:08.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.969+0000 7f5e4098f640 1 -- 172.21.15.38:0/3238199991 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715122113 0 0) 0x7f5e3c15ec20 con 0x7f5e3c06c700 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.970+0000 7f5e42192640 1 --2- 172.21.15.38:0/3238199991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c15e750 0x7f5e3c0cdf90 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009870 tx=0x7f5e2c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34be360294da572 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.970+0000 7f5e4098f640 1 -- 172.21.15.38:0/3238199991 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c037070 con 0x7f5e3c15e750 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.970+0000 7f5e4098f640 1 -- 172.21.15.38:0/3238199991 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c032440 con 0x7f5e3c15e750 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.970+0000 7f5e4098f640 1 -- 172.21.15.38:0/3238199991 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c032810 con 0x7f5e3c15e750 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.970+0000 7f5e43bf5640 1 -- 172.21.15.38:0/3238199991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c15e750 msgr2=0x7f5e3c0cdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.970+0000 7f5e43bf5640 1 --2- 172.21.15.38:0/3238199991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c15e750 0x7f5e3c0cdf90 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009870 tx=0x7f5e2c02a2a0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.970+0000 7f5e43bf5640 1 -- 172.21.15.38:0/3238199991 shutdown_connections 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.970+0000 7f5e43bf5640 1 --2- 172.21.15.38:0/3238199991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c15e750 0x7f5e3c0cdf90 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.970+0000 7f5e43bf5640 1 -- 172.21.15.38:0/3238199991 >> 172.21.15.38:0/3238199991 conn(0x7f5e3c08b000 msgr2=0x7f5e3c08b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.970+0000 7f5e43bf5640 1 -- 172.21.15.38:0/3238199991 shutdown_connections 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.970+0000 7f5e43bf5640 1 -- 172.21.15.38:0/3238199991 wait complete. 2024-04-25T08:39:08.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.972+0000 7f5e43bf5640 1 Processor -- start 2024-04-25T08:39:08.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.972+0000 7f5e43bf5640 1 -- start start 2024-04-25T08:39:08.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.972+0000 7f5e43bf5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e3c06c700 0x7f5e3c14fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.972+0000 7f5e43bf5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c15e750 0x7f5e3c1588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.972+0000 7f5e43bf5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c162110 0x7f5e3c0affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.972+0000 7f5e43bf5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c06c190 con 0x7f5e3c15e750 2024-04-25T08:39:08.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.972+0000 7f5e43bf5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c1625c0 con 0x7f5e3c162110 2024-04-25T08:39:08.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.972+0000 7f5e43bf5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c15b920 con 0x7f5e3c06c700 2024-04-25T08:39:08.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.972+0000 7f5e41190640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c15e750 0x7f5e3c1588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.972+0000 7f5e41190640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c15e750 0x7f5e3c1588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44960/0 (socket says 172.21.15.38:44960) 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.972+0000 7f5e41190640 1 -- 172.21.15.38:0/423975261 learned_addr learned my addr 172.21.15.38:0/423975261 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.973+0000 7f5e41190640 1 -- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e3c06c700 msgr2=0x7f5e3c14fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.973+0000 7f5e41190640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e3c06c700 0x7f5e3c14fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.973+0000 7f5e41190640 1 -- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c162110 msgr2=0x7f5e3c0affa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.973+0000 7f5e42192640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c162110 0x7f5e3c0affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.973+0000 7f5e41190640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c162110 0x7f5e3c0affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.973+0000 7f5e41190640 1 -- 172.21.15.38:0/423975261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c1501b0 con 0x7f5e3c15e750 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.973+0000 7f5e41190640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c15e750 0x7f5e3c1588a0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b3f0 tx=0x7f5e3000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.973+0000 7f5e3a7fc640 1 -- 172.21.15.38:0/423975261 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30015020 con 0x7f5e3c15e750 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.973+0000 7f5e3a7fc640 1 -- 172.21.15.38:0/423975261 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e30011830 con 0x7f5e3c15e750 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.973+0000 7f5e43bf5640 1 -- 172.21.15.38:0/423975261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c0c6140 con 0x7f5e3c15e750 2024-04-25T08:39:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.973+0000 7f5e3a7fc640 1 -- 172.21.15.38:0/423975261 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30011c20 con 0x7f5e3c15e750 2024-04-25T08:39:08.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.973+0000 7f5e43bf5640 1 -- 172.21.15.38:0/423975261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c0c6700 con 0x7f5e3c15e750 2024-04-25T08:39:08.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.974+0000 7f5e43bf5640 1 -- 172.21.15.38:0/423975261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c068e80 con 0x7f5e3c15e750 2024-04-25T08:39:08.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.974+0000 7f5e3a7fc640 1 -- 172.21.15.38:0/423975261 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e30018020 con 0x7f5e3c15e750 2024-04-25T08:39:08.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.975+0000 7f5e3a7fc640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e2806b4d0 0x7f5e2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:08.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.975+0000 7f5e3a7fc640 1 -- 172.21.15.38:0/423975261 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e30089f90 con 0x7f5e3c15e750 2024-04-25T08:39:08.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.975+0000 7f5e41991640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e2806b4d0 0x7f5e2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:08.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.976+0000 7f5e41991640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e2806b4d0 0x7f5e2806d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5e34001560 tx=0x7f5e34007ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:08.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:08.979+0000 7f5e3a7fc640 1 -- 172.21.15.38:0/423975261 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e3001f030 con 0x7f5e3c15e750 2024-04-25T08:39:09.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.199+0000 7f5e43bf5640 1 -- 172.21.15.38:0/423975261 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5e3c08b6f0 con 0x7f5e3c15e750 2024-04-25T08:39:09.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.204+0000 7f5e3a7fc640 1 -- 172.21.15.38:0/423975261 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v62) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5e300172c0 con 0x7f5e3c15e750 2024-04-25T08:39:09.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.206+0000 7f5e43bf5640 1 -- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e2806b4d0 msgr2=0x7f5e2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.206+0000 7f5e43bf5640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e2806b4d0 0x7f5e2806d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5e34001560 tx=0x7f5e34007ca0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.206+0000 7f5e43bf5640 1 -- 172.21.15.38:0/423975261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c15e750 msgr2=0x7f5e3c1588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.206+0000 7f5e43bf5640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c15e750 0x7f5e3c1588a0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b3f0 tx=0x7f5e3000b8b0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.206+0000 7f5e43bf5640 1 -- 172.21.15.38:0/423975261 shutdown_connections 2024-04-25T08:39:09.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.206+0000 7f5e43bf5640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e3c162110 0x7f5e3c0affa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.206+0000 7f5e43bf5640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e3c15e750 0x7f5e3c1588a0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.206+0000 7f5e43bf5640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e2806b4d0 0x7f5e2806d990 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.206+0000 7f5e43bf5640 1 --2- 172.21.15.38:0/423975261 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e3c06c700 0x7f5e3c14fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.206+0000 7f5e43bf5640 1 -- 172.21.15.38:0/423975261 >> 172.21.15.38:0/423975261 conn(0x7f5e3c08b000 msgr2=0x7f5e3c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:09.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.206+0000 7f5e43bf5640 1 -- 172.21.15.38:0/423975261 shutdown_connections 2024-04-25T08:39:09.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.206+0000 7f5e43bf5640 1 -- 172.21.15.38:0/423975261 wait complete. 2024-04-25T08:39:09.211 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:39:09.211 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:39:09.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.323+0000 7f779179e640 1 Processor -- start 2024-04-25T08:39:09.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.323+0000 7f779179e640 1 -- start start 2024-04-25T08:39:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.324+0000 7f779179e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f778c0d39e0 0x7f778c15dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.324+0000 7f779179e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778c0ce130 con 0x7f778c15d8a0 2024-04-25T08:39:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.324+0000 7f779179e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778c0d1660 con 0x7f778c0d11f0 2024-04-25T08:39:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.324+0000 7f779179e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f778c0d0f30 con 0x7f778c0d39e0 2024-04-25T08:39:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.324+0000 7f7790f9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f778c0d39e0 0x7f778c15dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.324+0000 7f778bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f778c15d8a0 0x7f778c0cfe60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43564/0 (socket says 172.21.15.38:43564) 2024-04-25T08:39:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.324+0000 7f778bfff640 1 -- 172.21.15.38:0/2476627193 learned_addr learned my addr 172.21.15.38:0/2476627193 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.324+0000 7f7790f9d640 1 -- 172.21.15.38:0/2476627193 >> v1:172.21.15.107:6789/0 conn(0x7f778c0d11f0 legacy=0x7f778c0d32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.324+0000 7f7790f9d640 1 -- 172.21.15.38:0/2476627193 >> v1:172.21.15.38:6789/0 conn(0x7f778c15d8a0 legacy=0x7f778c0cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.324+0000 7f7790f9d640 1 -- 172.21.15.38:0/2476627193 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c0ce3b0 con 0x7f778c0d39e0 2024-04-25T08:39:09.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.325+0000 7f7790f9d640 1 --2- 172.21.15.38:0/2476627193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f778c0d39e0 0x7f778c15dc80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7784009d90 tx=0x7f778402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f68da319bfa0dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.325+0000 7f778affd640 1 -- 172.21.15.38:0/2476627193 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784007740 con 0x7f778c0d39e0 2024-04-25T08:39:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.325+0000 7f778affd640 1 -- 172.21.15.38:0/2476627193 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77840078e0 con 0x7f778c0d39e0 2024-04-25T08:39:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.325+0000 7f778affd640 1 -- 172.21.15.38:0/2476627193 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784007cb0 con 0x7f778c0d39e0 2024-04-25T08:39:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.326+0000 7f779179e640 1 -- 172.21.15.38:0/2476627193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f778c0d39e0 msgr2=0x7f778c15dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.326+0000 7f779179e640 1 --2- 172.21.15.38:0/2476627193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f778c0d39e0 0x7f778c15dc80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f7784009d90 tx=0x7f778402a6a0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.326+0000 7f779179e640 1 -- 172.21.15.38:0/2476627193 shutdown_connections 2024-04-25T08:39:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.326+0000 7f779179e640 1 --2- 172.21.15.38:0/2476627193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f778c0d39e0 0x7f778c15dc80 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.326+0000 7f779179e640 1 -- 172.21.15.38:0/2476627193 >> 172.21.15.38:0/2476627193 conn(0x7f778c09c980 msgr2=0x7f778c08b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.326+0000 7f779179e640 1 -- 172.21.15.38:0/2476627193 shutdown_connections 2024-04-25T08:39:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.326+0000 7f779179e640 1 -- 172.21.15.38:0/2476627193 wait complete. 2024-04-25T08:39:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.327+0000 7f779179e640 1 Processor -- start 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.327+0000 7f779179e640 1 -- start start 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f779179e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f778c0d11f0 0x7f778c15c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f779179e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f778c0d39e0 0x7f778c0c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f779179e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f778c15d8a0 0x7f778c0ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f779179e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c0d0970 con 0x7f778c0d11f0 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f779179e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c15e1c0 con 0x7f778c0d39e0 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f779179e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778c0d0c50 con 0x7f778c15d8a0 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f778bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f778c0d11f0 0x7f778c15c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f778bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f778c0d11f0 0x7f778c15c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44974/0 (socket says 172.21.15.38:44974) 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f778bfff640 1 -- 172.21.15.38:0/3939576982 learned_addr learned my addr 172.21.15.38:0/3939576982 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f778b7fe640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f778c0d39e0 0x7f778c0c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f7790f9d640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f778c15d8a0 0x7f778c0ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f778bfff640 1 -- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f778c15d8a0 msgr2=0x7f778c0ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f778bfff640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f778c15d8a0 0x7f778c0ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f778bfff640 1 -- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f778c0d39e0 msgr2=0x7f778c0c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f778bfff640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f778c0d39e0 0x7f778c0c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f778bfff640 1 -- 172.21.15.38:0/3939576982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c15cd60 con 0x7f778c0d11f0 2024-04-25T08:39:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.328+0000 7f7790f9d640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f778c15d8a0 0x7f778c0ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.329+0000 7f778bfff640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f778c0d11f0 0x7f778c15c820 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f777400c3c0 tx=0x7f777400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.329+0000 7f7788ff9640 1 -- 172.21.15.38:0/3939576982 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774017070 con 0x7f778c0d11f0 2024-04-25T08:39:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.329+0000 7f7788ff9640 1 -- 172.21.15.38:0/3939576982 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7774005ce0 con 0x7f778c0d11f0 2024-04-25T08:39:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.329+0000 7f779179e640 1 -- 172.21.15.38:0/3939576982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f778c0c8970 con 0x7f778c0d11f0 2024-04-25T08:39:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.329+0000 7f7788ff9640 1 -- 172.21.15.38:0/3939576982 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7774007310 con 0x7f778c0d11f0 2024-04-25T08:39:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.329+0000 7f779179e640 1 -- 172.21.15.38:0/3939576982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778c0c8f30 con 0x7f778c0d11f0 2024-04-25T08:39:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.330+0000 7f779179e640 1 -- 172.21.15.38:0/3939576982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778c04ef20 con 0x7f778c0d11f0 2024-04-25T08:39:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.330+0000 7f7788ff9640 1 -- 172.21.15.38:0/3939576982 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f777401a070 con 0x7f778c0d11f0 2024-04-25T08:39:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.331+0000 7f7788ff9640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f776c06b4d0 0x7f776c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.331+0000 7f7788ff9640 1 -- 172.21.15.38:0/3939576982 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f77740895f0 con 0x7f778c0d11f0 2024-04-25T08:39:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.331+0000 7f778b7fe640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f776c06b4d0 0x7f776c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.332+0000 7f778b7fe640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f776c06b4d0 0x7f776c06d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f778c0cfd20 tx=0x7f777c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:09.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.335+0000 7f7788ff9640 1 -- 172.21.15.38:0/3939576982 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f778c0ce130 con 0x7f778c0d11f0 2024-04-25T08:39:09.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:09.554+0000 7f779179e640 1 -- 172.21.15.38:0/3939576982 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f778c06be70 con 0x7f778c0d11f0 2024-04-25T08:39:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.216+0000 7f7788ff9640 1 -- 172.21.15.38:0/3939576982 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v63) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7774057d20 con 0x7f778c0d11f0 2024-04-25T08:39:10.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.218+0000 7f779179e640 1 -- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f776c06b4d0 msgr2=0x7f776c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.218+0000 7f779179e640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f776c06b4d0 0x7f776c06d990 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f778c0cfd20 tx=0x7f777c009040 comp rx=0 tx=0).stop 2024-04-25T08:39:10.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.218+0000 7f779179e640 1 -- 172.21.15.38:0/3939576982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f778c0d11f0 msgr2=0x7f778c15c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.218+0000 7f779179e640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f778c0d11f0 0x7f778c15c820 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f777400c3c0 tx=0x7f777400c880 comp rx=0 tx=0).stop 2024-04-25T08:39:10.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.219+0000 7f779179e640 1 -- 172.21.15.38:0/3939576982 shutdown_connections 2024-04-25T08:39:10.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.219+0000 7f779179e640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f776c06b4d0 0x7f776c06d990 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.219+0000 7f779179e640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f778c15d8a0 0x7f778c0ca6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.219+0000 7f779179e640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f778c0d39e0 0x7f778c0c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.219+0000 7f779179e640 1 --2- 172.21.15.38:0/3939576982 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f778c0d11f0 0x7f778c15c820 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.219+0000 7f779179e640 1 -- 172.21.15.38:0/3939576982 >> 172.21.15.38:0/3939576982 conn(0x7f778c09c980 msgr2=0x7f778c06c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:10.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.219+0000 7f779179e640 1 -- 172.21.15.38:0/3939576982 shutdown_connections 2024-04-25T08:39:10.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.219+0000 7f779179e640 1 -- 172.21.15.38:0/3939576982 wait complete. 2024-04-25T08:39:10.226 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:39:10.226 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.340+0000 7eff3ace1640 1 Processor -- start 2024-04-25T08:39:10.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.340+0000 7eff3ace1640 1 -- start start 2024-04-25T08:39:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.341+0000 7eff3ace1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7eff34144760 0x7eff3415faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.341+0000 7eff3ace1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7eff340cea70 0x7eff340b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.341+0000 7eff3ace1640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff340b50c0 con 0x7eff34160030 2024-04-25T08:39:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.341+0000 7eff3ace1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3415f380 con 0x7eff34144760 2024-04-25T08:39:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.341+0000 7eff3ace1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3415f660 con 0x7eff340cea70 2024-04-25T08:39:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.341+0000 7eff38a7d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7eff34144760 0x7eff3415faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.341+0000 7eff38a7d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7eff34144760 0x7eff3415faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44978/0 (socket says 172.21.15.38:44978) 2024-04-25T08:39:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.341+0000 7eff38a7d640 1 -- 172.21.15.38:0/1163000327 learned_addr learned my addr 172.21.15.38:0/1163000327 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.342+0000 7eff3927e640 1 --2- 172.21.15.38:0/1163000327 >> v2:172.21.15.107:3300/0 conn(0x7eff340cea70 0x7eff340b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.342+0000 7eff38a7d640 1 -- 172.21.15.38:0/1163000327 >> v2:172.21.15.107:3300/0 conn(0x7eff340cea70 msgr2=0x7eff340b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:10.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.342+0000 7eff38a7d640 1 --2- 172.21.15.38:0/1163000327 >> v2:172.21.15.107:3300/0 conn(0x7eff340cea70 0x7eff340b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.342+0000 7eff38a7d640 1 -- 172.21.15.38:0/1163000327 >> v1:172.21.15.107:6789/0 conn(0x7eff34160030 legacy=0x7eff340ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:10.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.342+0000 7eff38a7d640 1 -- 172.21.15.38:0/1163000327 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff340b53a0 con 0x7eff34144760 2024-04-25T08:39:10.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.342+0000 7eff38a7d640 1 --2- 172.21.15.38:0/1163000327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff34144760 0x7eff3415faf0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7eff34073310 tx=0x7eff2c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ab60a9ca01ab8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.343+0000 7eff337fe640 1 -- 172.21.15.38:0/1163000327 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c036070 con 0x7eff34144760 2024-04-25T08:39:10.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.343+0000 7eff337fe640 1 -- 172.21.15.38:0/1163000327 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c031600 con 0x7eff34144760 2024-04-25T08:39:10.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.343+0000 7eff337fe640 1 -- 172.21.15.38:0/1163000327 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c0319d0 con 0x7eff34144760 2024-04-25T08:39:10.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.343+0000 7eff3ace1640 1 -- 172.21.15.38:0/1163000327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff34144760 msgr2=0x7eff3415faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.343+0000 7eff3ace1640 1 --2- 172.21.15.38:0/1163000327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff34144760 0x7eff3415faf0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7eff34073310 tx=0x7eff2c02a660 comp rx=0 tx=0).stop 2024-04-25T08:39:10.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.343+0000 7eff3ace1640 1 -- 172.21.15.38:0/1163000327 shutdown_connections 2024-04-25T08:39:10.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.343+0000 7eff3ace1640 1 --2- 172.21.15.38:0/1163000327 >> v2:172.21.15.107:3300/0 conn(0x7eff340cea70 0x7eff340b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.343+0000 7eff3ace1640 1 --2- 172.21.15.38:0/1163000327 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff34144760 0x7eff3415faf0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.343+0000 7eff3ace1640 1 -- 172.21.15.38:0/1163000327 >> 172.21.15.38:0/1163000327 conn(0x7eff3406b210 msgr2=0x7eff3406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:10.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.343+0000 7eff3ace1640 1 -- 172.21.15.38:0/1163000327 shutdown_connections 2024-04-25T08:39:10.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.343+0000 7eff3ace1640 1 -- 172.21.15.38:0/1163000327 wait complete. 2024-04-25T08:39:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.344+0000 7eff3ace1640 1 Processor -- start 2024-04-25T08:39:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.345+0000 7eff3ace1640 1 -- start start 2024-04-25T08:39:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.345+0000 7eff3ace1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff340cea70 0x7eff3413c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.345+0000 7eff3ace1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff34160030 0x7eff34138e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.345+0000 7eff3ace1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3413a4f0 0x7eff3413a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.345+0000 7eff3ace1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff34145400 con 0x7eff3413a4f0 2024-04-25T08:39:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.345+0000 7eff3ace1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff34145280 con 0x7eff34160030 2024-04-25T08:39:10.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.345+0000 7eff3ace1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340b4b50 con 0x7eff340cea70 2024-04-25T08:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff3927e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3413a4f0 0x7eff3413a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff3927e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3413a4f0 0x7eff3413a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44992/0 (socket says 172.21.15.38:44992) 2024-04-25T08:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff3927e640 1 -- 172.21.15.38:0/2142258966 learned_addr learned my addr 172.21.15.38:0/2142258966 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff33fff640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff34160030 0x7eff34138e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff38a7d640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff340cea70 0x7eff3413c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff3927e640 1 -- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff340cea70 msgr2=0x7eff3413c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff3927e640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff340cea70 0x7eff3413c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff3927e640 1 -- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff34160030 msgr2=0x7eff34138e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff3927e640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff34160030 0x7eff34138e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff3927e640 1 -- 172.21.15.38:0/2142258966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3415e9e0 con 0x7eff3413a4f0 2024-04-25T08:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff3927e640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3413a4f0 0x7eff3413a980 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7eff2400d600 tx=0x7eff2400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff317fa640 1 -- 172.21.15.38:0/2142258966 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240125f0 con 0x7eff3413a4f0 2024-04-25T08:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff317fa640 1 -- 172.21.15.38:0/2142258966 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff24019070 con 0x7eff3413a4f0 2024-04-25T08:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff3ace1640 1 -- 172.21.15.38:0/2142258966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3413b060 con 0x7eff3413a4f0 2024-04-25T08:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff317fa640 1 -- 172.21.15.38:0/2142258966 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240129d0 con 0x7eff3413a4f0 2024-04-25T08:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.346+0000 7eff3ace1640 1 -- 172.21.15.38:0/2142258966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3413b620 con 0x7eff3413a4f0 2024-04-25T08:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.347+0000 7eff3ace1640 1 -- 172.21.15.38:0/2142258966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff34068e80 con 0x7eff3413a4f0 2024-04-25T08:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.348+0000 7eff317fa640 1 -- 172.21.15.38:0/2142258966 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff24012b70 con 0x7eff3413a4f0 2024-04-25T08:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.348+0000 7eff317fa640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7eff2006b4d0 0x7eff2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.348+0000 7eff317fa640 1 -- 172.21.15.38:0/2142258966 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7eff2408a6a0 con 0x7eff3413a4f0 2024-04-25T08:39:10.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.349+0000 7eff38a7d640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7eff2006b4d0 0x7eff2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.349+0000 7eff38a7d640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7eff2006b4d0 0x7eff2006d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7eff2c033000 tx=0x7eff2c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.353+0000 7eff317fa640 1 -- 172.21.15.38:0/2142258966 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff24058ca0 con 0x7eff3413a4f0 2024-04-25T08:39:10.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.569+0000 7eff3ace1640 1 -- 172.21.15.38:0/2142258966 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff34140180 con 0x7eff3413a4f0 2024-04-25T08:39:10.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.570+0000 7eff317fa640 1 -- 172.21.15.38:0/2142258966 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4946 (secure 0 0 0) 0x7eff2405cd40 con 0x7eff3413a4f0 2024-04-25T08:39:10.564 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:10.564 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:07.551641+0000","modified":"2024-04-25T08:39:09.555958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8292":{"gid":8292,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1263010516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1263010516},{"type":"v1","addr":"172.21.15.107:6837","nonce":1263010516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:39:10.564 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 63 2024-04-25T08:39:10.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.572+0000 7eff3ace1640 1 -- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7eff2006b4d0 msgr2=0x7eff2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.572+0000 7eff3ace1640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7eff2006b4d0 0x7eff2006d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7eff2c033000 tx=0x7eff2c009000 comp rx=0 tx=0).stop 2024-04-25T08:39:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.572+0000 7eff3ace1640 1 -- 172.21.15.38:0/2142258966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3413a4f0 msgr2=0x7eff3413a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.572+0000 7eff3ace1640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3413a4f0 0x7eff3413a980 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7eff2400d600 tx=0x7eff2400dac0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.572+0000 7eff3ace1640 1 -- 172.21.15.38:0/2142258966 shutdown_connections 2024-04-25T08:39:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.572+0000 7eff3ace1640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff3413a4f0 0x7eff3413a980 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.572+0000 7eff3ace1640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7eff34160030 0x7eff34138e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.572+0000 7eff3ace1640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7eff2006b4d0 0x7eff2006d990 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.572+0000 7eff3ace1640 1 --2- 172.21.15.38:0/2142258966 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7eff340cea70 0x7eff3413c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.572+0000 7eff3ace1640 1 -- 172.21.15.38:0/2142258966 >> 172.21.15.38:0/2142258966 conn(0x7eff3406b210 msgr2=0x7eff340727b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.572+0000 7eff3ace1640 1 -- 172.21.15.38:0/2142258966 shutdown_connections 2024-04-25T08:39:10.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.572+0000 7eff3ace1640 1 -- 172.21.15.38:0/2142258966 wait complete. 2024-04-25T08:39:10.577 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:10.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.685+0000 7f08a3a12640 1 Processor -- start 2024-04-25T08:39:10.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.685+0000 7f08a3a12640 1 -- start start 2024-04-25T08:39:10.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a3a12640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f089c09ab90 0x7f089c15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a3a12640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f089c0d0460 0x7f089c15fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a3a12640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f089c160340 con 0x7f089c15bb10 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a3a12640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089c0cffd0 con 0x7f089c09ab90 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a3a12640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0cf400 con 0x7f089c0d0460 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a0fad640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f089c09ab90 0x7f089c15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a17ae640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f089c15bb10 0x7f089c09a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43574/0 (socket says 172.21.15.38:43574) 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a17ae640 1 -- 172.21.15.38:0/939240673 learned_addr learned my addr 172.21.15.38:0/939240673 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a1faf640 1 --2- 172.21.15.38:0/939240673 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f089c0d0460 0x7f089c15fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f0893fff640 1 -- 172.21.15.38:0/939240673 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839067310 0 0) 0x7f0898001110 con 0x7f089c15bb10 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f0893fff640 1 -- 172.21.15.38:0/939240673 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0888003520 con 0x7f089c15bb10 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a0fad640 1 -- 172.21.15.38:0/939240673 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f089c0d0460 msgr2=0x7f089c15fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a0fad640 1 --2- 172.21.15.38:0/939240673 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f089c0d0460 0x7f089c15fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a0fad640 1 -- 172.21.15.38:0/939240673 >> v1:172.21.15.38:6789/0 conn(0x7f089c15bb10 legacy=0x7f089c09a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a0fad640 1 -- 172.21.15.38:0/939240673 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c0cf6a0 con 0x7f089c09ab90 2024-04-25T08:39:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.686+0000 7f08a1faf640 1 --2- 172.21.15.38:0/939240673 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f089c0d0460 0x7f089c15fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:10.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.687+0000 7f08a0fad640 1 --2- 172.21.15.38:0/939240673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f089c09ab90 0x7f089c15f4e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f088c0097a0 tx=0x7f088c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676c8df07204cb5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.687+0000 7f0893fff640 1 -- 172.21.15.38:0/939240673 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c037070 con 0x7f089c09ab90 2024-04-25T08:39:10.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.687+0000 7f0893fff640 1 -- 172.21.15.38:0/939240673 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088c02ae50 con 0x7f089c09ab90 2024-04-25T08:39:10.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.687+0000 7f0893fff640 1 -- 172.21.15.38:0/939240673 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088c032660 con 0x7f089c09ab90 2024-04-25T08:39:10.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.688+0000 7f08a3a12640 1 -- 172.21.15.38:0/939240673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f089c09ab90 msgr2=0x7f089c15f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.688+0000 7f08a3a12640 1 --2- 172.21.15.38:0/939240673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f089c09ab90 0x7f089c15f4e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f088c0097a0 tx=0x7f088c02a100 comp rx=0 tx=0).stop 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.688+0000 7f08a3a12640 1 -- 172.21.15.38:0/939240673 shutdown_connections 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.688+0000 7f08a3a12640 1 --2- 172.21.15.38:0/939240673 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f089c0d0460 0x7f089c15fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.688+0000 7f08a3a12640 1 --2- 172.21.15.38:0/939240673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f089c09ab90 0x7f089c15f4e0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.688+0000 7f08a3a12640 1 -- 172.21.15.38:0/939240673 >> 172.21.15.38:0/939240673 conn(0x7f089c0b31f0 msgr2=0x7f089c0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.688+0000 7f08a3a12640 1 -- 172.21.15.38:0/939240673 shutdown_connections 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.688+0000 7f08a3a12640 1 -- 172.21.15.38:0/939240673 wait complete. 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.689+0000 7f08a3a12640 1 Processor -- start 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.689+0000 7f08a3a12640 1 -- start start 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a3a12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f089c09ab90 0x7f089c0c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a3a12640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f089c0d0460 0x7f089c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a3a12640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c15bb10 0x7f089c0c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a3a12640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0c5250 con 0x7f089c09ab90 2024-04-25T08:39:10.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a3a12640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0cfc60 con 0x7f089c15bb10 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a3a12640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c0c53d0 con 0x7f089c0d0460 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a17ae640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f089c09ab90 0x7f089c0c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a17ae640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f089c09ab90 0x7f089c0c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45000/0 (socket says 172.21.15.38:45000) 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a17ae640 1 -- 172.21.15.38:0/2258310106 learned_addr learned my addr 172.21.15.38:0/2258310106 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a0fad640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f089c0d0460 0x7f089c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a1faf640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c15bb10 0x7f089c0c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a17ae640 1 -- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f089c0d0460 msgr2=0x7f089c15ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a17ae640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f089c0d0460 0x7f089c15ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a17ae640 1 -- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c15bb10 msgr2=0x7f089c0c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a17ae640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c15bb10 0x7f089c0c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a17ae640 1 -- 172.21.15.38:0/2258310106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c15ef70 con 0x7f089c09ab90 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f08a17ae640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f089c09ab90 0x7f089c0c4260 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f089800c3d0 tx=0x7f089800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.690+0000 7f0891ffb640 1 -- 172.21.15.38:0/2258310106 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0898017070 con 0x7f089c09ab90 2024-04-25T08:39:10.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.691+0000 7f0891ffb640 1 -- 172.21.15.38:0/2258310106 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0898005ce0 con 0x7f089c09ab90 2024-04-25T08:39:10.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.691+0000 7f08a3a12640 1 -- 172.21.15.38:0/2258310106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089c1e6820 con 0x7f089c09ab90 2024-04-25T08:39:10.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.691+0000 7f0891ffb640 1 -- 172.21.15.38:0/2258310106 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0898007310 con 0x7f089c09ab90 2024-04-25T08:39:10.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.691+0000 7f08a3a12640 1 -- 172.21.15.38:0/2258310106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c1e6de0 con 0x7f089c09ab90 2024-04-25T08:39:10.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.692+0000 7f08a3a12640 1 -- 172.21.15.38:0/2258310106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089c068e80 con 0x7f089c09ab90 2024-04-25T08:39:10.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.692+0000 7f0891ffb640 1 -- 172.21.15.38:0/2258310106 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f089801a070 con 0x7f089c09ab90 2024-04-25T08:39:10.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.692+0000 7f0891ffb640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f088006b4d0 0x7f088006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.693+0000 7f0891ffb640 1 -- 172.21.15.38:0/2258310106 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0898089790 con 0x7f089c09ab90 2024-04-25T08:39:10.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.693+0000 7f08a0fad640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f088006b4d0 0x7f088006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.693+0000 7f08a0fad640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f088006b4d0 0x7f088006d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f088c034000 tx=0x7f088c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.697+0000 7f0891ffb640 1 -- 172.21.15.38:0/2258310106 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f089c15ef70 con 0x7f089c09ab90 2024-04-25T08:39:10.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.877+0000 7f08a3a12640 1 -- 172.21.15.38:0/2258310106 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f089c0cdbf0 con 0x7f089c09ab90 2024-04-25T08:39:10.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.878+0000 7f0891ffb640 1 -- 172.21.15.38:0/2258310106 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16872 (secure 0 0 0) 0x7f089c160340 con 0x7f089c09ab90 2024-04-25T08:39:10.871 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:10.871 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":108,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:09.190717+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:05.852531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:06.527775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.880+0000 7f08a3a12640 1 -- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f088006b4d0 msgr2=0x7f088006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.880+0000 7f08a3a12640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f088006b4d0 0x7f088006d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f088c034000 tx=0x7f088c02e000 comp rx=0 tx=0).stop 2024-04-25T08:39:10.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.880+0000 7f08a3a12640 1 -- 172.21.15.38:0/2258310106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f089c09ab90 msgr2=0x7f089c0c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.880+0000 7f08a3a12640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f089c09ab90 0x7f089c0c4260 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f089800c3d0 tx=0x7f089800c890 comp rx=0 tx=0).stop 2024-04-25T08:39:10.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.880+0000 7f08a3a12640 1 -- 172.21.15.38:0/2258310106 shutdown_connections 2024-04-25T08:39:10.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.880+0000 7f08a3a12640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f089c15bb10 0x7f089c0c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.880+0000 7f08a3a12640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f088006b4d0 0x7f088006d990 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.880+0000 7f08a3a12640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f089c0d0460 0x7f089c15ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.880+0000 7f08a3a12640 1 --2- 172.21.15.38:0/2258310106 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f089c09ab90 0x7f089c0c4260 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.880+0000 7f08a3a12640 1 -- 172.21.15.38:0/2258310106 >> 172.21.15.38:0/2258310106 conn(0x7f089c0b31f0 msgr2=0x7f089c0b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:10.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.880+0000 7f08a3a12640 1 -- 172.21.15.38:0/2258310106 shutdown_connections 2024-04-25T08:39:10.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.880+0000 7f08a3a12640 1 -- 172.21.15.38:0/2258310106 wait complete. 2024-04-25T08:39:10.885 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:39:10.885 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:39:10.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.992+0000 7ff25c04d640 1 Processor -- start 2024-04-25T08:39:10.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.992+0000 7ff25c04d640 1 -- start start 2024-04-25T08:39:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff25c04d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff25415dc10 0x7ff2540ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff25c04d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2540ced60 0x7ff2540d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff25c04d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2540d3e70 con 0x7ff2540d3900 2024-04-25T08:39:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff25c04d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25415dff0 con 0x7ff25415dc10 2024-04-25T08:39:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff25c04d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540b6990 con 0x7ff2540ced60 2024-04-25T08:39:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff259de9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff25415dc10 0x7ff2540ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff25a5ea640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff2540d3900 0x7ff2540b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43580/0 (socket says 172.21.15.38:43580) 2024-04-25T08:39:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff25a5ea640 1 -- 172.21.15.38:0/1233873110 learned_addr learned my addr 172.21.15.38:0/1233873110 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:10.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff259de9640 1 -- 172.21.15.38:0/1233873110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2540ced60 msgr2=0x7ff2540d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:10.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff2595e8640 1 --2- 172.21.15.38:0/1233873110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2540ced60 0x7ff2540d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff259de9640 1 --2- 172.21.15.38:0/1233873110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2540ced60 0x7ff2540d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff259de9640 1 -- 172.21.15.38:0/1233873110 >> v1:172.21.15.38:6789/0 conn(0x7ff2540d3900 legacy=0x7ff2540b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff258de7640 1 -- 172.21.15.38:0/1233873110 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322665208 0 0) 0x7ff2540d3e70 con 0x7ff2540d3900 2024-04-25T08:39:10.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.993+0000 7ff259de9640 1 -- 172.21.15.38:0/1233873110 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2540d4150 con 0x7ff25415dc10 2024-04-25T08:39:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.994+0000 7ff259de9640 1 --2- 172.21.15.38:0/1233873110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff25415dc10 0x7ff2540ce820 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7ff2500097a0 tx=0x7ff25002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1b35e96a778756 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.994+0000 7ff258de7640 1 -- 172.21.15.38:0/1233873110 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250037070 con 0x7ff25415dc10 2024-04-25T08:39:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.994+0000 7ff258de7640 1 -- 172.21.15.38:0/1233873110 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25002ae50 con 0x7ff25415dc10 2024-04-25T08:39:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.994+0000 7ff258de7640 1 -- 172.21.15.38:0/1233873110 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250032660 con 0x7ff25415dc10 2024-04-25T08:39:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.995+0000 7ff25c04d640 1 -- 172.21.15.38:0/1233873110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff25415dc10 msgr2=0x7ff2540ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.995+0000 7ff25c04d640 1 --2- 172.21.15.38:0/1233873110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff25415dc10 0x7ff2540ce820 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7ff2500097a0 tx=0x7ff25002a100 comp rx=0 tx=0).stop 2024-04-25T08:39:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.995+0000 7ff25c04d640 1 -- 172.21.15.38:0/1233873110 shutdown_connections 2024-04-25T08:39:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.995+0000 7ff25c04d640 1 --2- 172.21.15.38:0/1233873110 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2540ced60 0x7ff2540d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.995+0000 7ff25c04d640 1 --2- 172.21.15.38:0/1233873110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff25415dc10 0x7ff2540ce820 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.995+0000 7ff25c04d640 1 -- 172.21.15.38:0/1233873110 >> 172.21.15.38:0/1233873110 conn(0x7ff254066d10 msgr2=0x7ff254067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.995+0000 7ff25c04d640 1 -- 172.21.15.38:0/1233873110 shutdown_connections 2024-04-25T08:39:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.995+0000 7ff25c04d640 1 -- 172.21.15.38:0/1233873110 wait complete. 2024-04-25T08:39:10.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.996+0000 7ff25c04d640 1 Processor -- start 2024-04-25T08:39:10.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.996+0000 7ff25c04d640 1 -- start start 2024-04-25T08:39:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff25c04d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2540ced60 0x7ff254159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff25c04d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2540d3900 0x7ff2540cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff25c04d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25415dc10 0x7ff2540cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff25c04d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540b6bc0 con 0x7ff2540ced60 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff25c04d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540d4710 con 0x7ff2540d3900 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff25c04d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540d0d90 con 0x7ff25415dc10 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff259de9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2540ced60 0x7ff254159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff259de9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2540ced60 0x7ff254159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45016/0 (socket says 172.21.15.38:45016) 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff259de9640 1 -- 172.21.15.38:0/2782506905 learned_addr learned my addr 172.21.15.38:0/2782506905 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff259de9640 1 -- 172.21.15.38:0/2782506905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25415dc10 msgr2=0x7ff2540cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff259de9640 1 --2- 172.21.15.38:0/2782506905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25415dc10 0x7ff2540cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff259de9640 1 -- 172.21.15.38:0/2782506905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2540d3900 msgr2=0x7ff2540cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff259de9640 1 --2- 172.21.15.38:0/2782506905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2540d3900 0x7ff2540cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff259de9640 1 -- 172.21.15.38:0/2782506905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2540c7680 con 0x7ff2540ced60 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.997+0000 7ff259de9640 1 --2- 172.21.15.38:0/2782506905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2540ced60 0x7ff254159c20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7ff2500098c0 tx=0x7ff2500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.998+0000 7ff24a7fc640 1 -- 172.21.15.38:0/2782506905 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250041030 con 0x7ff2540ced60 2024-04-25T08:39:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.998+0000 7ff24a7fc640 1 -- 172.21.15.38:0/2782506905 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25002ac50 con 0x7ff2540ced60 2024-04-25T08:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.998+0000 7ff25c04d640 1 -- 172.21.15.38:0/2782506905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2540c7960 con 0x7ff2540ced60 2024-04-25T08:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.998+0000 7ff24a7fc640 1 -- 172.21.15.38:0/2782506905 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff250037040 con 0x7ff2540ced60 2024-04-25T08:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.998+0000 7ff25c04d640 1 -- 172.21.15.38:0/2782506905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2540c7f20 con 0x7ff2540ced60 2024-04-25T08:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.999+0000 7ff25c04d640 1 -- 172.21.15.38:0/2782506905 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2540c8120 con 0x7ff2540ced60 2024-04-25T08:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:10.999+0000 7ff24a7fc640 1 -- 172.21.15.38:0/2782506905 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff250044070 con 0x7ff2540ced60 2024-04-25T08:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.000+0000 7ff24a7fc640 1 --2- 172.21.15.38:0/2782506905 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff23806b4d0 0x7ff23806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.000+0000 7ff24a7fc640 1 -- 172.21.15.38:0/2782506905 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff2500aa980 con 0x7ff2540ced60 2024-04-25T08:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.000+0000 7ff2595e8640 1 --2- 172.21.15.38:0/2782506905 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff23806b4d0 0x7ff23806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:10.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.000+0000 7ff2595e8640 1 --2- 172.21.15.38:0/2782506905 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff23806b4d0 0x7ff23806d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff244006fd0 tx=0x7ff244008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:10.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.004+0000 7ff24a7fc640 1 -- 172.21.15.38:0/2782506905 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2540c8120 con 0x7ff2540ced60 2024-04-25T08:39:11.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.182+0000 7ff25c04d640 1 -- 172.21.15.38:0/2782506905 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff254068e80 con 0x7ff23806b4d0 2024-04-25T08:39:11.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.188+0000 7ff24a7fc640 1 -- 172.21.15.38:0/2782506905 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247075 (secure 0 0 0) 0x7ff254068e80 con 0x7ff23806b4d0 2024-04-25T08:39:11.184 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:11.186 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:39:11.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.191+0000 7ff25c04d640 1 -- 172.21.15.38:0/2782506905 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff23806b4d0 msgr2=0x7ff23806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.191+0000 7ff25c04d640 1 --2- 172.21.15.38:0/2782506905 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff23806b4d0 0x7ff23806d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff244006fd0 tx=0x7ff244008040 comp rx=0 tx=0).stop 2024-04-25T08:39:11.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.191+0000 7ff25c04d640 1 -- 172.21.15.38:0/2782506905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2540ced60 msgr2=0x7ff254159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.191+0000 7ff25c04d640 1 --2- 172.21.15.38:0/2782506905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2540ced60 0x7ff254159c20 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7ff2500098c0 tx=0x7ff2500046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.191+0000 7ff25c04d640 1 -- 172.21.15.38:0/2782506905 shutdown_connections 2024-04-25T08:39:11.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.191+0000 7ff25c04d640 1 --2- 172.21.15.38:0/2782506905 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff23806b4d0 0x7ff23806d990 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.191+0000 7ff25c04d640 1 --2- 172.21.15.38:0/2782506905 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff25415dc10 0x7ff2540cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.191+0000 7ff25c04d640 1 --2- 172.21.15.38:0/2782506905 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2540d3900 0x7ff2540cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.191+0000 7ff25c04d640 1 --2- 172.21.15.38:0/2782506905 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2540ced60 0x7ff254159c20 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.191+0000 7ff25c04d640 1 -- 172.21.15.38:0/2782506905 >> 172.21.15.38:0/2782506905 conn(0x7ff254066d10 msgr2=0x7ff25415eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:11.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.191+0000 7ff25c04d640 1 -- 172.21.15.38:0/2782506905 shutdown_connections 2024-04-25T08:39:11.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.191+0000 7ff25c04d640 1 -- 172.21.15.38:0/2782506905 wait complete. 2024-04-25T08:39:11.195 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-04-25T08:39:09.311220+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466240,"kb_used_data":11348,"kb_used_omap":12,"kb_used_meta":454771,"kb_avail":754508480,"statfs":{"total":773094113280,"available":772616683520,"internally_reserved":0,"allocated":11620352,"data_stored":6009995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":465686064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000861"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185002+0000","last_change":"2024-04-25T08:39:07.185002+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185002+0000","last_undegraded":"2024-04-25T08:39:07.185002+0000","last_fullsized":"2024-04-25T08:39:07.185002+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182883+0000","last_change":"2024-04-25T08:39:07.182883+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182883+0000","last_undegraded":"2024-04-25T08:39:07.182883+0000","last_fullsized":"2024-04-25T08:39:07.182883+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186081+0000","last_change":"2024-04-25T08:39:07.186081+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186081+0000","last_undegraded":"2024-04-25T08:39:07.186081+0000","last_fullsized":"2024-04-25T08:39:07.186081+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183803+0000","last_change":"2024-04-25T08:39:07.183803+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183803+0000","last_undegraded":"2024-04-25T08:39:07.183803+0000","last_fullsized":"2024-04-25T08:39:07.183803+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.190401+0000","last_change":"2024-04-25T08:39:07.190401+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.190401+0000","last_undegraded":"2024-04-25T08:39:07.190401+0000","last_fullsized":"2024-04-25T08:39:07.190401+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185128+0000","last_change":"2024-04-25T08:39:07.185128+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185128+0000","last_undegraded":"2024-04-25T08:39:07.185128+0000","last_fullsized":"2024-04-25T08:39:07.185128+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182530+0000","last_change":"2024-04-25T08:39:07.182530+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182530+0000","last_undegraded":"2024-04-25T08:39:07.182530+0000","last_fullsized":"2024-04-25T08:39:07.182530+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185168+0000","last_change":"2024-04-25T08:39:07.185168+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185168+0000","last_undegraded":"2024-04-25T08:39:07.185168+0000","last_fullsized":"2024-04-25T08:39:07.185168+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181457+0000","last_change":"2024-04-25T08:39:07.181457+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181457+0000","last_undegraded":"2024-04-25T08:39:07.181457+0000","last_fullsized":"2024-04-25T08:39:07.181457+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187122+0000","last_change":"2024-04-25T08:39:07.187122+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187122+0000","last_undegraded":"2024-04-25T08:39:07.187122+0000","last_fullsized":"2024-04-25T08:39:07.187122+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187732+0000","last_change":"2024-04-25T08:39:07.187732+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187732+0000","last_undegraded":"2024-04-25T08:39:07.187732+0000","last_fullsized":"2024-04-25T08:39:07.187732+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187078+0000","last_change":"2024-04-25T08:39:07.187078+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187078+0000","last_undegraded":"2024-04-25T08:39:07.187078+0000","last_fullsized":"2024-04-25T08:39:07.187078+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182812+0000","last_change":"2024-04-25T08:39:07.182812+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182812+0000","last_undegraded":"2024-04-25T08:39:07.182812+0000","last_fullsized":"2024-04-25T08:39:07.182812+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185729+0000","last_change":"2024-04-25T08:39:07.185729+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185729+0000","last_undegraded":"2024-04-25T08:39:07.185729+0000","last_fullsized":"2024-04-25T08:39:07.185729+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185317+0000","last_change":"2024-04-25T08:39:07.185317+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185317+0000","last_undegraded":"2024-04-25T08:39:07.185317+0000","last_fullsized":"2024-04-25T08:39:07.185317+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185201+0000","last_change":"2024-04-25T08:39:07.185201+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185201+0000","last_undegraded":"2024-04-25T08:39:07.185201+0000","last_fullsized":"2024-04-25T08:39:07.185201+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184442+0000","last_change":"2024-04-25T08:39:07.184442+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184442+0000","last_undegraded":"2024-04-25T08:39:07.184442+0000","last_fullsized":"2024-04-25T08:39:07.184442+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182220+0000","last_change":"2024-04-25T08:39:07.182220+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182220+0000","last_undegraded":"2024-04-25T08:39:07.182220+0000","last_fullsized":"2024-04-25T08:39:07.182220+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187629+0000","last_change":"2024-04-25T08:39:07.187629+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187629+0000","last_undegraded":"2024-04-25T08:39:07.187629+0000","last_fullsized":"2024-04-25T08:39:07.187629+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183719+0000","last_change":"2024-04-25T08:39:07.183719+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183719+0000","last_undegraded":"2024-04-25T08:39:07.183719+0000","last_fullsized":"2024-04-25T08:39:07.183719+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.193271+0000","last_change":"2024-04-25T08:39:07.193271+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.193271+0000","last_undegraded":"2024-04-25T08:39:07.193271+0000","last_fullsized":"2024-04-25T08:39:07.193271+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.189389+0000","last_change":"2024-04-25T08:39:07.189389+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.189389+0000","last_undegraded":"2024-04-25T08:39:07.189389+0000","last_fullsized":"2024-04-25T08:39:07.189389+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183901+0000","last_change":"2024-04-25T08:39:07.183901+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183901+0000","last_undegraded":"2024-04-25T08:39:07.183901+0000","last_fullsized":"2024-04-25T08:39:07.183901+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188396+0000","last_change":"2024-04-25T08:39:07.188396+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188396+0000","last_undegraded":"2024-04-25T08:39:07.188396+0000","last_fullsized":"2024-04-25T08:39:07.188396+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183544+0000","last_change":"2024-04-25T08:39:07.183544+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183544+0000","last_undegraded":"2024-04-25T08:39:07.183544+0000","last_fullsized":"2024-04-25T08:39:07.183544+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183684+0000","last_change":"2024-04-25T08:39:07.183684+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183684+0000","last_undegraded":"2024-04-25T08:39:07.183684+0000","last_fullsized":"2024-04-25T08:39:07.183684+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187277+0000","last_change":"2024-04-25T08:39:07.187277+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187277+0000","last_undegraded":"2024-04-25T08:39:07.187277+0000","last_fullsized":"2024-04-25T08:39:07.187277+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181602+0000","last_change":"2024-04-25T08:39:07.181602+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181602+0000","last_undegraded":"2024-04-25T08:39:07.181602+0000","last_fullsized":"2024-04-25T08:39:07.181602+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.192641+0000","last_change":"2024-04-25T08:39:07.192641+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.192641+0000","last_undegraded":"2024-04-25T08:39:07.192641+0000","last_fullsized":"2024-04-25T08:39:07.192641+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184579+0000","last_change":"2024-04-25T08:39:07.184579+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184579+0000","last_undegraded":"2024-04-25T08:39:07.184579+0000","last_fullsized":"2024-04-25T08:39:07.184579+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.180795+0000","last_change":"2024-04-25T08:39:07.180795+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.180795+0000","last_undegraded":"2024-04-25T08:39:07.180795+0000","last_fullsized":"2024-04-25T08:39:07.180795+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186769+0000","last_change":"2024-04-25T08:39:07.186769+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186769+0000","last_undegraded":"2024-04-25T08:39:07.186769+0000","last_fullsized":"2024-04-25T08:39:07.186769+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184486+0000","last_change":"2024-04-25T08:39:07.184486+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184486+0000","last_undegraded":"2024-04-25T08:39:07.184486+0000","last_fullsized":"2024-04-25T08:39:07.184486+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192158+0000","last_change":"2024-04-25T08:39:07.190276+0000","last_active":"2024-04-25T08:39:07.192158+0000","last_peered":"2024-04-25T08:39:07.192158+0000","last_clean":"2024-04-25T08:39:07.192158+0000","last_became_active":"2024-04-25T08:39:07.189857+0000","last_became_peered":"2024-04-25T08:39:07.189857+0000","last_unstale":"2024-04-25T08:39:07.192158+0000","last_undegraded":"2024-04-25T08:39:07.192158+0000","last_fullsized":"2024-04-25T08:39:07.192158+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:03.259504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187736+0000","last_change":"2024-04-25T08:39:07.187736+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187736+0000","last_undegraded":"2024-04-25T08:39:07.187736+0000","last_fullsized":"2024-04-25T08:39:07.187736+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202189+0000","last_change":"2024-04-25T08:39:07.200692+0000","last_active":"2024-04-25T08:39:07.202189+0000","last_peered":"2024-04-25T08:39:07.202189+0000","last_clean":"2024-04-25T08:39:07.202189+0000","last_became_active":"2024-04-25T08:39:07.200405+0000","last_became_peered":"2024-04-25T08:39:07.200405+0000","last_unstale":"2024-04-25T08:39:07.202189+0000","last_undegraded":"2024-04-25T08:39:07.202189+0000","last_fullsized":"2024-04-25T08:39:07.202189+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.824524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184135+0000","last_change":"2024-04-25T08:39:07.184135+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184135+0000","last_undegraded":"2024-04-25T08:39:07.184135+0000","last_fullsized":"2024-04-25T08:39:07.184135+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.197048+0000","last_change":"2024-04-25T08:39:07.195456+0000","last_active":"2024-04-25T08:39:07.197048+0000","last_peered":"2024-04-25T08:39:07.197048+0000","last_clean":"2024-04-25T08:39:07.197048+0000","last_became_active":"2024-04-25T08:39:07.194725+0000","last_became_peered":"2024-04-25T08:39:07.194725+0000","last_unstale":"2024-04-25T08:39:07.197048+0000","last_undegraded":"2024-04-25T08:39:07.197048+0000","last_fullsized":"2024-04-25T08:39:07.197048+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:20.621755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185569+0000","last_change":"2024-04-25T08:39:07.185569+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185569+0000","last_undegraded":"2024-04-25T08:39:07.185569+0000","last_fullsized":"2024-04-25T08:39:07.185569+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199508+0000","last_change":"2024-04-25T08:39:07.197963+0000","last_active":"2024-04-25T08:39:07.199508+0000","last_peered":"2024-04-25T08:39:07.199508+0000","last_clean":"2024-04-25T08:39:07.199508+0000","last_became_active":"2024-04-25T08:39:07.197658+0000","last_became_peered":"2024-04-25T08:39:07.197658+0000","last_unstale":"2024-04-25T08:39:07.199508+0000","last_undegraded":"2024-04-25T08:39:07.199508+0000","last_fullsized":"2024-04-25T08:39:07.199508+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:39.219462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183293+0000","last_change":"2024-04-25T08:39:07.183293+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183293+0000","last_undegraded":"2024-04-25T08:39:07.183293+0000","last_fullsized":"2024-04-25T08:39:07.183293+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192072+0000","last_change":"2024-04-25T08:39:07.191029+0000","last_active":"2024-04-25T08:39:07.192072+0000","last_peered":"2024-04-25T08:39:07.192072+0000","last_clean":"2024-04-25T08:39:07.192072+0000","last_became_active":"2024-04-25T08:39:07.190722+0000","last_became_peered":"2024-04-25T08:39:07.190722+0000","last_unstale":"2024-04-25T08:39:07.192072+0000","last_undegraded":"2024-04-25T08:39:07.192072+0000","last_fullsized":"2024-04-25T08:39:07.192072+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:46.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186738+0000","last_change":"2024-04-25T08:39:07.186738+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186738+0000","last_undegraded":"2024-04-25T08:39:07.186738+0000","last_fullsized":"2024-04-25T08:39:07.186738+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199195+0000","last_change":"2024-04-25T08:39:07.198565+0000","last_active":"2024-04-25T08:39:07.199195+0000","last_peered":"2024-04-25T08:39:07.199195+0000","last_clean":"2024-04-25T08:39:07.199195+0000","last_became_active":"2024-04-25T08:39:07.197861+0000","last_became_peered":"2024-04-25T08:39:07.197861+0000","last_unstale":"2024-04-25T08:39:07.199195+0000","last_undegraded":"2024-04-25T08:39:07.199195+0000","last_fullsized":"2024-04-25T08:39:07.199195+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:54.429666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181730+0000","last_change":"2024-04-25T08:39:07.181730+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181730+0000","last_undegraded":"2024-04-25T08:39:07.181730+0000","last_fullsized":"2024-04-25T08:39:07.181730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.195665+0000","last_change":"2024-04-25T08:39:07.191219+0000","last_active":"2024-04-25T08:39:07.195665+0000","last_peered":"2024-04-25T08:39:07.195665+0000","last_clean":"2024-04-25T08:39:07.195665+0000","last_became_active":"2024-04-25T08:39:07.190902+0000","last_became_peered":"2024-04-25T08:39:07.190902+0000","last_unstale":"2024-04-25T08:39:07.195665+0000","last_undegraded":"2024-04-25T08:39:07.195665+0000","last_fullsized":"2024-04-25T08:39:07.195665+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.078468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182971+0000","last_change":"2024-04-25T08:39:07.182971+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182971+0000","last_undegraded":"2024-04-25T08:39:07.182971+0000","last_fullsized":"2024-04-25T08:39:07.182971+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192534+0000","last_change":"2024-04-25T08:39:07.191760+0000","last_active":"2024-04-25T08:39:07.192534+0000","last_peered":"2024-04-25T08:39:07.192534+0000","last_clean":"2024-04-25T08:39:07.192534+0000","last_became_active":"2024-04-25T08:39:07.191248+0000","last_became_peered":"2024-04-25T08:39:07.191248+0000","last_unstale":"2024-04-25T08:39:07.192534+0000","last_undegraded":"2024-04-25T08:39:07.192534+0000","last_fullsized":"2024-04-25T08:39:07.192534+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:16.487240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185073+0000","last_change":"2024-04-25T08:39:07.185073+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185073+0000","last_undegraded":"2024-04-25T08:39:07.185073+0000","last_fullsized":"2024-04-25T08:39:07.185073+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199328+0000","last_change":"2024-04-25T08:39:07.198061+0000","last_active":"2024-04-25T08:39:07.199328+0000","last_peered":"2024-04-25T08:39:07.199328+0000","last_clean":"2024-04-25T08:39:07.199328+0000","last_became_active":"2024-04-25T08:39:07.197146+0000","last_became_peered":"2024-04-25T08:39:07.197146+0000","last_unstale":"2024-04-25T08:39:07.199328+0000","last_undegraded":"2024-04-25T08:39:07.199328+0000","last_fullsized":"2024-04-25T08:39:07.199328+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:39.219462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188107+0000","last_change":"2024-04-25T08:39:07.188107+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188107+0000","last_undegraded":"2024-04-25T08:39:07.188107+0000","last_fullsized":"2024-04-25T08:39:07.188107+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.197020+0000","last_change":"2024-04-25T08:39:07.195425+0000","last_active":"2024-04-25T08:39:07.197020+0000","last_peered":"2024-04-25T08:39:07.197020+0000","last_clean":"2024-04-25T08:39:07.197020+0000","last_became_active":"2024-04-25T08:39:07.195087+0000","last_became_peered":"2024-04-25T08:39:07.195087+0000","last_unstale":"2024-04-25T08:39:07.197020+0000","last_undegraded":"2024-04-25T08:39:07.197020+0000","last_fullsized":"2024-04-25T08:39:07.197020+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:52.006665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187086+0000","last_change":"2024-04-25T08:39:07.187086+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187086+0000","last_undegraded":"2024-04-25T08:39:07.187086+0000","last_fullsized":"2024-04-25T08:39:07.187086+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192581+0000","last_change":"2024-04-25T08:39:07.192065+0000","last_active":"2024-04-25T08:39:07.192581+0000","last_peered":"2024-04-25T08:39:07.192581+0000","last_clean":"2024-04-25T08:39:07.192581+0000","last_became_active":"2024-04-25T08:39:07.191757+0000","last_became_peered":"2024-04-25T08:39:07.191757+0000","last_unstale":"2024-04-25T08:39:07.192581+0000","last_undegraded":"2024-04-25T08:39:07.192581+0000","last_fullsized":"2024-04-25T08:39:07.192581+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:32.537457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.189381+0000","last_change":"2024-04-25T08:39:07.189381+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.189381+0000","last_undegraded":"2024-04-25T08:39:07.189381+0000","last_fullsized":"2024-04-25T08:39:07.189381+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198326+0000","last_change":"2024-04-25T08:39:07.196396+0000","last_active":"2024-04-25T08:39:07.198326+0000","last_peered":"2024-04-25T08:39:07.198326+0000","last_clean":"2024-04-25T08:39:07.198326+0000","last_became_active":"2024-04-25T08:39:07.195109+0000","last_became_peered":"2024-04-25T08:39:07.195109+0000","last_unstale":"2024-04-25T08:39:07.198326+0000","last_undegraded":"2024-04-25T08:39:07.198326+0000","last_fullsized":"2024-04-25T08:39:07.198326+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.078468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186932+0000","last_change":"2024-04-25T08:39:07.186932+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186932+0000","last_undegraded":"2024-04-25T08:39:07.186932+0000","last_fullsized":"2024-04-25T08:39:07.186932+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198619+0000","last_change":"2024-04-25T08:39:07.197589+0000","last_active":"2024-04-25T08:39:07.198619+0000","last_peered":"2024-04-25T08:39:07.198619+0000","last_clean":"2024-04-25T08:39:07.198619+0000","last_became_active":"2024-04-25T08:39:07.197301+0000","last_became_peered":"2024-04-25T08:39:07.197301+0000","last_unstale":"2024-04-25T08:39:07.198619+0000","last_undegraded":"2024-04-25T08:39:07.198619+0000","last_fullsized":"2024-04-25T08:39:07.198619+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:20.621755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181101+0000","last_change":"2024-04-25T08:39:07.181101+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181101+0000","last_undegraded":"2024-04-25T08:39:07.181101+0000","last_fullsized":"2024-04-25T08:39:07.181101+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.189989+0000","last_change":"2024-04-25T08:39:07.189044+0000","last_active":"2024-04-25T08:39:07.189989+0000","last_peered":"2024-04-25T08:39:07.189989+0000","last_clean":"2024-04-25T08:39:07.189989+0000","last_became_active":"2024-04-25T08:39:07.188847+0000","last_became_peered":"2024-04-25T08:39:07.188847+0000","last_unstale":"2024-04-25T08:39:07.189989+0000","last_undegraded":"2024-04-25T08:39:07.189989+0000","last_fullsized":"2024-04-25T08:39:07.189989+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:19.824202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188221+0000","last_change":"2024-04-25T08:39:07.188221+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188221+0000","last_undegraded":"2024-04-25T08:39:07.188221+0000","last_fullsized":"2024-04-25T08:39:07.188221+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.200155+0000","last_change":"2024-04-25T08:39:07.198202+0000","last_active":"2024-04-25T08:39:07.200155+0000","last_peered":"2024-04-25T08:39:07.200155+0000","last_clean":"2024-04-25T08:39:07.200155+0000","last_became_active":"2024-04-25T08:39:07.197778+0000","last_became_peered":"2024-04-25T08:39:07.197778+0000","last_unstale":"2024-04-25T08:39:07.200155+0000","last_undegraded":"2024-04-25T08:39:07.200155+0000","last_fullsized":"2024-04-25T08:39:07.200155+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:03.697252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183954+0000","last_change":"2024-04-25T08:39:07.183954+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183954+0000","last_undegraded":"2024-04-25T08:39:07.183954+0000","last_fullsized":"2024-04-25T08:39:07.183954+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.195707+0000","last_change":"2024-04-25T08:39:07.191103+0000","last_active":"2024-04-25T08:39:07.195707+0000","last_peered":"2024-04-25T08:39:07.195707+0000","last_clean":"2024-04-25T08:39:07.195707+0000","last_became_active":"2024-04-25T08:39:07.190789+0000","last_became_peered":"2024-04-25T08:39:07.190789+0000","last_unstale":"2024-04-25T08:39:07.195707+0000","last_undegraded":"2024-04-25T08:39:07.195707+0000","last_fullsized":"2024-04-25T08:39:07.195707+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:30.160350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185617+0000","last_change":"2024-04-25T08:39:07.185617+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185617+0000","last_undegraded":"2024-04-25T08:39:07.185617+0000","last_fullsized":"2024-04-25T08:39:07.185617+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.196625+0000","last_change":"2024-04-25T08:39:07.194224+0000","last_active":"2024-04-25T08:39:07.196625+0000","last_peered":"2024-04-25T08:39:07.196625+0000","last_clean":"2024-04-25T08:39:07.196625+0000","last_became_active":"2024-04-25T08:39:07.193936+0000","last_became_peered":"2024-04-25T08:39:07.193936+0000","last_unstale":"2024-04-25T08:39:07.196625+0000","last_undegraded":"2024-04-25T08:39:07.196625+0000","last_fullsized":"2024-04-25T08:39:07.196625+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:30.160350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181477+0000","last_change":"2024-04-25T08:39:07.181477+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181477+0000","last_undegraded":"2024-04-25T08:39:07.181477+0000","last_fullsized":"2024-04-25T08:39:07.181477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199346+0000","last_change":"2024-04-25T08:39:07.197977+0000","last_active":"2024-04-25T08:39:07.199346+0000","last_peered":"2024-04-25T08:39:07.199346+0000","last_clean":"2024-04-25T08:39:07.199346+0000","last_became_active":"2024-04-25T08:39:07.197678+0000","last_became_peered":"2024-04-25T08:39:07.197678+0000","last_unstale":"2024-04-25T08:39:07.199346+0000","last_undegraded":"2024-04-25T08:39:07.199346+0000","last_fullsized":"2024-04-25T08:39:07.199346+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:22.700613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.189910+0000","last_change":"2024-04-25T08:39:07.189910+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.189910+0000","last_undegraded":"2024-04-25T08:39:07.189910+0000","last_fullsized":"2024-04-25T08:39:07.189910+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198861+0000","last_change":"2024-04-25T08:39:07.197738+0000","last_active":"2024-04-25T08:39:07.198861+0000","last_peered":"2024-04-25T08:39:07.198861+0000","last_clean":"2024-04-25T08:39:07.198861+0000","last_became_active":"2024-04-25T08:39:07.197436+0000","last_became_peered":"2024-04-25T08:39:07.197436+0000","last_unstale":"2024-04-25T08:39:07.198861+0000","last_undegraded":"2024-04-25T08:39:07.198861+0000","last_fullsized":"2024-04-25T08:39:07.198861+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:39.317231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185375+0000","last_change":"2024-04-25T08:39:07.185375+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185375+0000","last_undegraded":"2024-04-25T08:39:07.185375+0000","last_fullsized":"2024-04-25T08:39:07.185375+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202185+0000","last_change":"2024-04-25T08:39:07.200590+0000","last_active":"2024-04-25T08:39:07.202185+0000","last_peered":"2024-04-25T08:39:07.202185+0000","last_clean":"2024-04-25T08:39:07.202185+0000","last_became_active":"2024-04-25T08:39:07.200312+0000","last_became_peered":"2024-04-25T08:39:07.200312+0000","last_unstale":"2024-04-25T08:39:07.202185+0000","last_undegraded":"2024-04-25T08:39:07.202185+0000","last_fullsized":"2024-04-25T08:39:07.202185+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:14.807183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187174+0000","last_change":"2024-04-25T08:39:07.187174+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187174+0000","last_undegraded":"2024-04-25T08:39:07.187174+0000","last_fullsized":"2024-04-25T08:39:07.187174+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202389+0000","last_change":"2024-04-25T08:39:07.201896+0000","last_active":"2024-04-25T08:39:07.202389+0000","last_peered":"2024-04-25T08:39:07.202389+0000","last_clean":"2024-04-25T08:39:07.202389+0000","last_became_active":"2024-04-25T08:39:07.201576+0000","last_became_peered":"2024-04-25T08:39:07.201576+0000","last_unstale":"2024-04-25T08:39:07.202389+0000","last_undegraded":"2024-04-25T08:39:07.202389+0000","last_fullsized":"2024-04-25T08:39:07.202389+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:42.629045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184184+0000","last_change":"2024-04-25T08:39:07.184184+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184184+0000","last_undegraded":"2024-04-25T08:39:07.184184+0000","last_fullsized":"2024-04-25T08:39:07.184184+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.195198+0000","last_change":"2024-04-25T08:39:07.193296+0000","last_active":"2024-04-25T08:39:07.195198+0000","last_peered":"2024-04-25T08:39:07.195198+0000","last_clean":"2024-04-25T08:39:07.195198+0000","last_became_active":"2024-04-25T08:39:07.193017+0000","last_became_peered":"2024-04-25T08:39:07.193017+0000","last_unstale":"2024-04-25T08:39:07.195198+0000","last_undegraded":"2024-04-25T08:39:07.195198+0000","last_fullsized":"2024-04-25T08:39:07.195198+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:51.280521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185656+0000","last_change":"2024-04-25T08:39:07.185656+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185656+0000","last_undegraded":"2024-04-25T08:39:07.185656+0000","last_fullsized":"2024-04-25T08:39:07.185656+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202406+0000","last_change":"2024-04-25T08:39:07.201924+0000","last_active":"2024-04-25T08:39:07.202406+0000","last_peered":"2024-04-25T08:39:07.202406+0000","last_clean":"2024-04-25T08:39:07.202406+0000","last_became_active":"2024-04-25T08:39:07.201634+0000","last_became_peered":"2024-04-25T08:39:07.201634+0000","last_unstale":"2024-04-25T08:39:07.202406+0000","last_undegraded":"2024-04-25T08:39:07.202406+0000","last_fullsized":"2024-04-25T08:39:07.202406+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:13.867447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185786+0000","last_change":"2024-04-25T08:39:07.185786+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185786+0000","last_undegraded":"2024-04-25T08:39:07.185786+0000","last_fullsized":"2024-04-25T08:39:07.185786+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.196826+0000","last_change":"2024-04-25T08:39:07.196017+0000","last_active":"2024-04-25T08:39:07.196826+0000","last_peered":"2024-04-25T08:39:07.196826+0000","last_clean":"2024-04-25T08:39:07.196826+0000","last_became_active":"2024-04-25T08:39:07.195702+0000","last_became_peered":"2024-04-25T08:39:07.195702+0000","last_unstale":"2024-04-25T08:39:07.196826+0000","last_undegraded":"2024-04-25T08:39:07.196826+0000","last_fullsized":"2024-04-25T08:39:07.196826+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.078468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182412+0000","last_change":"2024-04-25T08:39:07.182412+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182412+0000","last_undegraded":"2024-04-25T08:39:07.182412+0000","last_fullsized":"2024-04-25T08:39:07.182412+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.200126+0000","last_change":"2024-04-25T08:39:07.198773+0000","last_active":"2024-04-25T08:39:07.200126+0000","last_peered":"2024-04-25T08:39:07.200126+0000","last_clean":"2024-04-25T08:39:07.200126+0000","last_became_active":"2024-04-25T08:39:07.197463+0000","last_became_peered":"2024-04-25T08:39:07.197463+0000","last_unstale":"2024-04-25T08:39:07.200126+0000","last_undegraded":"2024-04-25T08:39:07.200126+0000","last_fullsized":"2024-04-25T08:39:07.200126+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:12.088493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183713+0000","last_change":"2024-04-25T08:39:07.183713+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183713+0000","last_undegraded":"2024-04-25T08:39:07.183713+0000","last_fullsized":"2024-04-25T08:39:07.183713+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.190171+0000","last_change":"2024-04-25T08:39:07.188334+0000","last_active":"2024-04-25T08:39:07.190171+0000","last_peered":"2024-04-25T08:39:07.190171+0000","last_clean":"2024-04-25T08:39:07.190171+0000","last_became_active":"2024-04-25T08:39:07.187795+0000","last_became_peered":"2024-04-25T08:39:07.187795+0000","last_unstale":"2024-04-25T08:39:07.190171+0000","last_undegraded":"2024-04-25T08:39:07.190171+0000","last_fullsized":"2024-04-25T08:39:07.190171+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:57.413804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186076+0000","last_change":"2024-04-25T08:39:07.186076+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186076+0000","last_undegraded":"2024-04-25T08:39:07.186076+0000","last_fullsized":"2024-04-25T08:39:07.186076+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198255+0000","last_change":"2024-04-25T08:39:07.197165+0000","last_active":"2024-04-25T08:39:07.198255+0000","last_peered":"2024-04-25T08:39:07.198255+0000","last_clean":"2024-04-25T08:39:07.198255+0000","last_became_active":"2024-04-25T08:39:07.196785+0000","last_became_peered":"2024-04-25T08:39:07.196785+0000","last_unstale":"2024-04-25T08:39:07.198255+0000","last_undegraded":"2024-04-25T08:39:07.198255+0000","last_fullsized":"2024-04-25T08:39:07.198255+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:27.118962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184647+0000","last_change":"2024-04-25T08:39:07.184647+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184647+0000","last_undegraded":"2024-04-25T08:39:07.184647+0000","last_fullsized":"2024-04-25T08:39:07.184647+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.196795+0000","last_change":"2024-04-25T08:39:07.193944+0000","last_active":"2024-04-25T08:39:07.196795+0000","last_peered":"2024-04-25T08:39:07.196795+0000","last_clean":"2024-04-25T08:39:07.196795+0000","last_became_active":"2024-04-25T08:39:07.193625+0000","last_became_peered":"2024-04-25T08:39:07.193625+0000","last_unstale":"2024-04-25T08:39:07.196795+0000","last_undegraded":"2024-04-25T08:39:07.196795+0000","last_fullsized":"2024-04-25T08:39:07.196795+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:27.118962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185152+0000","last_change":"2024-04-25T08:39:07.185152+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185152+0000","last_undegraded":"2024-04-25T08:39:07.185152+0000","last_fullsized":"2024-04-25T08:39:07.185152+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.200071+0000","last_change":"2024-04-25T08:39:07.198072+0000","last_active":"2024-04-25T08:39:07.200071+0000","last_peered":"2024-04-25T08:39:07.200071+0000","last_clean":"2024-04-25T08:39:07.200071+0000","last_became_active":"2024-04-25T08:39:07.197770+0000","last_became_peered":"2024-04-25T08:39:07.197770+0000","last_unstale":"2024-04-25T08:39:07.200071+0000","last_undegraded":"2024-04-25T08:39:07.200071+0000","last_fullsized":"2024-04-25T08:39:07.200071+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.421862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186576+0000","last_change":"2024-04-25T08:39:07.186576+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186576+0000","last_undegraded":"2024-04-25T08:39:07.186576+0000","last_fullsized":"2024-04-25T08:39:07.186576+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.190198+0000","last_change":"2024-04-25T08:39:07.188807+0000","last_active":"2024-04-25T08:39:07.190198+0000","last_peered":"2024-04-25T08:39:07.190198+0000","last_clean":"2024-04-25T08:39:07.190198+0000","last_became_active":"2024-04-25T08:39:07.188447+0000","last_became_peered":"2024-04-25T08:39:07.188447+0000","last_unstale":"2024-04-25T08:39:07.190198+0000","last_undegraded":"2024-04-25T08:39:07.190198+0000","last_fullsized":"2024-04-25T08:39:07.190198+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:22.020416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.190873+0000","last_change":"2024-04-25T08:39:07.190873+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.190873+0000","last_undegraded":"2024-04-25T08:39:07.190873+0000","last_fullsized":"2024-04-25T08:39:07.190873+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.197208+0000","last_change":"2024-04-25T08:39:07.196369+0000","last_active":"2024-04-25T08:39:07.197208+0000","last_peered":"2024-04-25T08:39:07.197208+0000","last_clean":"2024-04-25T08:39:07.197208+0000","last_became_active":"2024-04-25T08:39:07.196073+0000","last_became_peered":"2024-04-25T08:39:07.196073+0000","last_unstale":"2024-04-25T08:39:07.197208+0000","last_undegraded":"2024-04-25T08:39:07.197208+0000","last_fullsized":"2024-04-25T08:39:07.197208+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:55.439962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188049+0000","last_change":"2024-04-25T08:39:07.188049+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188049+0000","last_undegraded":"2024-04-25T08:39:07.188049+0000","last_fullsized":"2024-04-25T08:39:07.188049+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202209+0000","last_change":"2024-04-25T08:39:07.200791+0000","last_active":"2024-04-25T08:39:07.202209+0000","last_peered":"2024-04-25T08:39:07.202209+0000","last_clean":"2024-04-25T08:39:07.202209+0000","last_became_active":"2024-04-25T08:39:07.200495+0000","last_became_peered":"2024-04-25T08:39:07.200495+0000","last_unstale":"2024-04-25T08:39:07.202209+0000","last_undegraded":"2024-04-25T08:39:07.202209+0000","last_fullsized":"2024-04-25T08:39:07.202209+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:59.362497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":332,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.186152+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:39:07.186152+0000","last_peered":"2024-04-25T08:39:07.186152+0000","last_clean":"2024-04-25T08:39:07.186152+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:39:07.186152+0000","last_undegraded":"2024-04-25T08:39:07.186152+0000","last_fullsized":"2024-04-25T08:39:07.186152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59400,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312440,"statfs":{"total":96636764160,"available":96575938560,"internally_reserved":0,"allocated":1896448,"data_stored":1193192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54660,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317180,"statfs":{"total":96636764160,"available":96580792320,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54664,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317176,"statfs":{"total":96636764160,"available":96580788224,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68548,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303292,"statfs":{"total":96636764160,"available":96566571008,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68548,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303292,"statfs":{"total":96636764160,"available":96566571008,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58824,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313016,"statfs":{"total":96636764160,"available":96576528384,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54660,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317180,"statfs":{"total":96636764160,"available":96580792320,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46936,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324904,"statfs":{"total":96636764160,"available":96588701696,"internally_reserved":0,"allocated":1908736,"data_stored":1199859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:39:11.200 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:39:11.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.342+0000 7fa0d3f96640 1 Processor -- start 2024-04-25T08:39:11.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.343+0000 7fa0d3f96640 1 -- start start 2024-04-25T08:39:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.343+0000 7fa0d3f96640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa0cc0d0f60 0x7fa0cc0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.343+0000 7fa0d3f96640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc0d3e70 con 0x7fa0cc15dc10 2024-04-25T08:39:11.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.343+0000 7fa0d3f96640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0cc0d4150 con 0x7fa0cc0d3900 2024-04-25T08:39:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.343+0000 7fa0d3f96640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0cc15dff0 con 0x7fa0cc0d0f60 2024-04-25T08:39:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.344+0000 7fa0d2533640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa0cc0d0f60 0x7fa0cc0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.344+0000 7fa0d1d32640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa0cc15dc10 0x7fa0cc0ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43596/0 (socket says 172.21.15.38:43596) 2024-04-25T08:39:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.344+0000 7fa0d1d32640 1 -- 172.21.15.38:0/1447444908 learned_addr learned my addr 172.21.15.38:0/1447444908 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.344+0000 7fa0d2533640 1 -- 172.21.15.38:0/1447444908 >> v1:172.21.15.107:6789/0 conn(0x7fa0cc0d3900 legacy=0x7fa0cc0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.344+0000 7fa0d2533640 1 -- 172.21.15.38:0/1447444908 >> v1:172.21.15.38:6789/0 conn(0x7fa0cc15dc10 legacy=0x7fa0cc0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.344+0000 7fa0d0d30640 1 -- 172.21.15.38:0/1447444908 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709335836 0 0) 0x7fa0cc0d3e70 con 0x7fa0cc15dc10 2024-04-25T08:39:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.344+0000 7fa0d2533640 1 -- 172.21.15.38:0/1447444908 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0d4430 con 0x7fa0cc0d0f60 2024-04-25T08:39:11.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.345+0000 7fa0d2533640 1 --2- 172.21.15.38:0/1447444908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0cc0d0f60 0x7fa0cc0b2370 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009d90 tx=0x7fa0bc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9756d4252daaea1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.345+0000 7fa0d0d30640 1 -- 172.21.15.38:0/1447444908 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc007740 con 0x7fa0cc0d0f60 2024-04-25T08:39:11.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.345+0000 7fa0d0d30640 1 -- 172.21.15.38:0/1447444908 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc0078e0 con 0x7fa0cc0d0f60 2024-04-25T08:39:11.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.345+0000 7fa0d0d30640 1 -- 172.21.15.38:0/1447444908 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc007cb0 con 0x7fa0cc0d0f60 2024-04-25T08:39:11.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.345+0000 7fa0d3f96640 1 -- 172.21.15.38:0/1447444908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0cc0d0f60 msgr2=0x7fa0cc0b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.345+0000 7fa0d3f96640 1 --2- 172.21.15.38:0/1447444908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0cc0d0f60 0x7fa0cc0b2370 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009d90 tx=0x7fa0bc02a660 comp rx=0 tx=0).stop 2024-04-25T08:39:11.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.345+0000 7fa0d3f96640 1 -- 172.21.15.38:0/1447444908 shutdown_connections 2024-04-25T08:39:11.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.345+0000 7fa0d3f96640 1 --2- 172.21.15.38:0/1447444908 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0cc0d0f60 0x7fa0cc0b2370 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.345+0000 7fa0d3f96640 1 -- 172.21.15.38:0/1447444908 >> 172.21.15.38:0/1447444908 conn(0x7fa0cc066d10 msgr2=0x7fa0cc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:11.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.345+0000 7fa0d3f96640 1 -- 172.21.15.38:0/1447444908 shutdown_connections 2024-04-25T08:39:11.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.345+0000 7fa0d3f96640 1 -- 172.21.15.38:0/1447444908 wait complete. 2024-04-25T08:39:11.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.347+0000 7fa0d3f96640 1 Processor -- start 2024-04-25T08:39:11.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.347+0000 7fa0d3f96640 1 -- start start 2024-04-25T08:39:11.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.347+0000 7fa0d3f96640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0cc0d0f60 0x7fa0cc0c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.347+0000 7fa0d3f96640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0cc0d3900 0x7fa0cc15ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.347+0000 7fa0d3f96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0cc15dc10 0x7fa0cc15bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.347+0000 7fa0d3f96640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0d4710 con 0x7fa0cc15dc10 2024-04-25T08:39:11.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.347+0000 7fa0d3f96640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0cef30 con 0x7fa0cc0d3900 2024-04-25T08:39:11.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.347+0000 7fa0d3f96640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0cc0b80f0 con 0x7fa0cc0d0f60 2024-04-25T08:39:11.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.347+0000 7fa0d2533640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0cc15dc10 0x7fa0cc15bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.347+0000 7fa0d2533640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0cc15dc10 0x7fa0cc15bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45024/0 (socket says 172.21.15.38:45024) 2024-04-25T08:39:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.347+0000 7fa0d2533640 1 -- 172.21.15.38:0/3025169232 learned_addr learned my addr 172.21.15.38:0/3025169232 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0d1531640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0cc0d3900 0x7fa0cc15ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0d1d32640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0cc0d0f60 0x7fa0cc0c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0d2533640 1 -- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0cc0d0f60 msgr2=0x7fa0cc0c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0d2533640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0cc0d0f60 0x7fa0cc0c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0d2533640 1 -- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0cc0d3900 msgr2=0x7fa0cc15ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0d2533640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0cc0d3900 0x7fa0cc15ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0d2533640 1 -- 172.21.15.38:0/3025169232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0cc0acb70 con 0x7fa0cc15dc10 2024-04-25T08:39:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0d2533640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0cc15dc10 0x7fa0cc15bf80 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009d90 tx=0x7fa0bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0ca7fc640 1 -- 172.21.15.38:0/3025169232 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc040070 con 0x7fa0cc15dc10 2024-04-25T08:39:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0ca7fc640 1 -- 172.21.15.38:0/3025169232 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc031c40 con 0x7fa0cc15dc10 2024-04-25T08:39:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0d3f96640 1 -- 172.21.15.38:0/3025169232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0cc0aa470 con 0x7fa0cc15dc10 2024-04-25T08:39:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0ca7fc640 1 -- 172.21.15.38:0/3025169232 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc003990 con 0x7fa0cc15dc10 2024-04-25T08:39:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.348+0000 7fa0d3f96640 1 -- 172.21.15.38:0/3025169232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0cc0aaa30 con 0x7fa0cc15dc10 2024-04-25T08:39:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.349+0000 7fa0d3f96640 1 -- 172.21.15.38:0/3025169232 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0cc068e80 con 0x7fa0cc15dc10 2024-04-25T08:39:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.350+0000 7fa0ca7fc640 1 -- 172.21.15.38:0/3025169232 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0bc039040 con 0x7fa0cc15dc10 2024-04-25T08:39:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.350+0000 7fa0ca7fc640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa0b806b4d0 0x7fa0b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.350+0000 7fa0ca7fc640 1 -- 172.21.15.38:0/3025169232 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0bc0ab6c0 con 0x7fa0cc15dc10 2024-04-25T08:39:11.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.350+0000 7fa0d1d32640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa0b806b4d0 0x7fa0b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.351+0000 7fa0d1d32640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa0b806b4d0 0x7fa0b806d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4000f60 tx=0x7fa0c4007d20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.354+0000 7fa0ca7fc640 1 -- 172.21.15.38:0/3025169232 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0bc07a3b0 con 0x7fa0cc15dc10 2024-04-25T08:39:11.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.532+0000 7fa0d3f96640 1 -- 172.21.15.38:0/3025169232 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0cc15e450 con 0x7fa0b806b4d0 2024-04-25T08:39:11.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.538+0000 7fa0ca7fc640 1 -- 172.21.15.38:0/3025169232 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247075 (secure 0 0 0) 0x7fa0cc15e450 con 0x7fa0b806b4d0 2024-04-25T08:39:11.534 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:11.535 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:39:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.541+0000 7fa0d3f96640 1 -- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa0b806b4d0 msgr2=0x7fa0b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.541+0000 7fa0d3f96640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa0b806b4d0 0x7fa0b806d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4000f60 tx=0x7fa0c4007d20 comp rx=0 tx=0).stop 2024-04-25T08:39:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.541+0000 7fa0d3f96640 1 -- 172.21.15.38:0/3025169232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0cc15dc10 msgr2=0x7fa0cc15bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.541+0000 7fa0d3f96640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0cc15dc10 0x7fa0cc15bf80 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009d90 tx=0x7fa0bc004a40 comp rx=0 tx=0).stop 2024-04-25T08:39:11.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.541+0000 7fa0d3f96640 1 -- 172.21.15.38:0/3025169232 shutdown_connections 2024-04-25T08:39:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.541+0000 7fa0d3f96640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0cc15dc10 0x7fa0cc15bf80 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.541+0000 7fa0d3f96640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0cc0d3900 0x7fa0cc15ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.541+0000 7fa0d3f96640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa0b806b4d0 0x7fa0b806d990 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.541+0000 7fa0d3f96640 1 --2- 172.21.15.38:0/3025169232 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0cc0d0f60 0x7fa0cc0c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.541+0000 7fa0d3f96640 1 -- 172.21.15.38:0/3025169232 >> 172.21.15.38:0/3025169232 conn(0x7fa0cc066d10 msgr2=0x7fa0cc0cde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.541+0000 7fa0d3f96640 1 -- 172.21.15.38:0/3025169232 shutdown_connections 2024-04-25T08:39:11.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.541+0000 7fa0d3f96640 1 -- 172.21.15.38:0/3025169232 wait complete. 2024-04-25T08:39:11.546 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-04-25T08:39:11.311618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466240,"kb_used_data":11348,"kb_used_omap":12,"kb_used_meta":454771,"kb_avail":754508480,"statfs":{"total":773094113280,"available":772616683520,"internally_reserved":0,"allocated":11620352,"data_stored":6009995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":465686064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.136186"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185002+0000","last_change":"2024-04-25T08:39:07.185002+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185002+0000","last_undegraded":"2024-04-25T08:39:07.185002+0000","last_fullsized":"2024-04-25T08:39:07.185002+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182883+0000","last_change":"2024-04-25T08:39:07.182883+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182883+0000","last_undegraded":"2024-04-25T08:39:07.182883+0000","last_fullsized":"2024-04-25T08:39:07.182883+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186081+0000","last_change":"2024-04-25T08:39:07.186081+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186081+0000","last_undegraded":"2024-04-25T08:39:07.186081+0000","last_fullsized":"2024-04-25T08:39:07.186081+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183803+0000","last_change":"2024-04-25T08:39:07.183803+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183803+0000","last_undegraded":"2024-04-25T08:39:07.183803+0000","last_fullsized":"2024-04-25T08:39:07.183803+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.190401+0000","last_change":"2024-04-25T08:39:07.190401+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.190401+0000","last_undegraded":"2024-04-25T08:39:07.190401+0000","last_fullsized":"2024-04-25T08:39:07.190401+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185128+0000","last_change":"2024-04-25T08:39:07.185128+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185128+0000","last_undegraded":"2024-04-25T08:39:07.185128+0000","last_fullsized":"2024-04-25T08:39:07.185128+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182530+0000","last_change":"2024-04-25T08:39:07.182530+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182530+0000","last_undegraded":"2024-04-25T08:39:07.182530+0000","last_fullsized":"2024-04-25T08:39:07.182530+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185168+0000","last_change":"2024-04-25T08:39:07.185168+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185168+0000","last_undegraded":"2024-04-25T08:39:07.185168+0000","last_fullsized":"2024-04-25T08:39:07.185168+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181457+0000","last_change":"2024-04-25T08:39:07.181457+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181457+0000","last_undegraded":"2024-04-25T08:39:07.181457+0000","last_fullsized":"2024-04-25T08:39:07.181457+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187122+0000","last_change":"2024-04-25T08:39:07.187122+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187122+0000","last_undegraded":"2024-04-25T08:39:07.187122+0000","last_fullsized":"2024-04-25T08:39:07.187122+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187732+0000","last_change":"2024-04-25T08:39:07.187732+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187732+0000","last_undegraded":"2024-04-25T08:39:07.187732+0000","last_fullsized":"2024-04-25T08:39:07.187732+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187078+0000","last_change":"2024-04-25T08:39:07.187078+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187078+0000","last_undegraded":"2024-04-25T08:39:07.187078+0000","last_fullsized":"2024-04-25T08:39:07.187078+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182812+0000","last_change":"2024-04-25T08:39:07.182812+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182812+0000","last_undegraded":"2024-04-25T08:39:07.182812+0000","last_fullsized":"2024-04-25T08:39:07.182812+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185729+0000","last_change":"2024-04-25T08:39:07.185729+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185729+0000","last_undegraded":"2024-04-25T08:39:07.185729+0000","last_fullsized":"2024-04-25T08:39:07.185729+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185317+0000","last_change":"2024-04-25T08:39:07.185317+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185317+0000","last_undegraded":"2024-04-25T08:39:07.185317+0000","last_fullsized":"2024-04-25T08:39:07.185317+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185201+0000","last_change":"2024-04-25T08:39:07.185201+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185201+0000","last_undegraded":"2024-04-25T08:39:07.185201+0000","last_fullsized":"2024-04-25T08:39:07.185201+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184442+0000","last_change":"2024-04-25T08:39:07.184442+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184442+0000","last_undegraded":"2024-04-25T08:39:07.184442+0000","last_fullsized":"2024-04-25T08:39:07.184442+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182220+0000","last_change":"2024-04-25T08:39:07.182220+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182220+0000","last_undegraded":"2024-04-25T08:39:07.182220+0000","last_fullsized":"2024-04-25T08:39:07.182220+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187629+0000","last_change":"2024-04-25T08:39:07.187629+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187629+0000","last_undegraded":"2024-04-25T08:39:07.187629+0000","last_fullsized":"2024-04-25T08:39:07.187629+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183719+0000","last_change":"2024-04-25T08:39:07.183719+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183719+0000","last_undegraded":"2024-04-25T08:39:07.183719+0000","last_fullsized":"2024-04-25T08:39:07.183719+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.193271+0000","last_change":"2024-04-25T08:39:07.193271+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.193271+0000","last_undegraded":"2024-04-25T08:39:07.193271+0000","last_fullsized":"2024-04-25T08:39:07.193271+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.189389+0000","last_change":"2024-04-25T08:39:07.189389+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.189389+0000","last_undegraded":"2024-04-25T08:39:07.189389+0000","last_fullsized":"2024-04-25T08:39:07.189389+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183901+0000","last_change":"2024-04-25T08:39:07.183901+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183901+0000","last_undegraded":"2024-04-25T08:39:07.183901+0000","last_fullsized":"2024-04-25T08:39:07.183901+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188396+0000","last_change":"2024-04-25T08:39:07.188396+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188396+0000","last_undegraded":"2024-04-25T08:39:07.188396+0000","last_fullsized":"2024-04-25T08:39:07.188396+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183544+0000","last_change":"2024-04-25T08:39:07.183544+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183544+0000","last_undegraded":"2024-04-25T08:39:07.183544+0000","last_fullsized":"2024-04-25T08:39:07.183544+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183684+0000","last_change":"2024-04-25T08:39:07.183684+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183684+0000","last_undegraded":"2024-04-25T08:39:07.183684+0000","last_fullsized":"2024-04-25T08:39:07.183684+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187277+0000","last_change":"2024-04-25T08:39:07.187277+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187277+0000","last_undegraded":"2024-04-25T08:39:07.187277+0000","last_fullsized":"2024-04-25T08:39:07.187277+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181602+0000","last_change":"2024-04-25T08:39:07.181602+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181602+0000","last_undegraded":"2024-04-25T08:39:07.181602+0000","last_fullsized":"2024-04-25T08:39:07.181602+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.192641+0000","last_change":"2024-04-25T08:39:07.192641+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.192641+0000","last_undegraded":"2024-04-25T08:39:07.192641+0000","last_fullsized":"2024-04-25T08:39:07.192641+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184579+0000","last_change":"2024-04-25T08:39:07.184579+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184579+0000","last_undegraded":"2024-04-25T08:39:07.184579+0000","last_fullsized":"2024-04-25T08:39:07.184579+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.180795+0000","last_change":"2024-04-25T08:39:07.180795+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.180795+0000","last_undegraded":"2024-04-25T08:39:07.180795+0000","last_fullsized":"2024-04-25T08:39:07.180795+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186769+0000","last_change":"2024-04-25T08:39:07.186769+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186769+0000","last_undegraded":"2024-04-25T08:39:07.186769+0000","last_fullsized":"2024-04-25T08:39:07.186769+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184486+0000","last_change":"2024-04-25T08:39:07.184486+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184486+0000","last_undegraded":"2024-04-25T08:39:07.184486+0000","last_fullsized":"2024-04-25T08:39:07.184486+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192158+0000","last_change":"2024-04-25T08:39:07.190276+0000","last_active":"2024-04-25T08:39:07.192158+0000","last_peered":"2024-04-25T08:39:07.192158+0000","last_clean":"2024-04-25T08:39:07.192158+0000","last_became_active":"2024-04-25T08:39:07.189857+0000","last_became_peered":"2024-04-25T08:39:07.189857+0000","last_unstale":"2024-04-25T08:39:07.192158+0000","last_undegraded":"2024-04-25T08:39:07.192158+0000","last_fullsized":"2024-04-25T08:39:07.192158+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:03.259504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187736+0000","last_change":"2024-04-25T08:39:07.187736+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187736+0000","last_undegraded":"2024-04-25T08:39:07.187736+0000","last_fullsized":"2024-04-25T08:39:07.187736+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202189+0000","last_change":"2024-04-25T08:39:07.200692+0000","last_active":"2024-04-25T08:39:07.202189+0000","last_peered":"2024-04-25T08:39:07.202189+0000","last_clean":"2024-04-25T08:39:07.202189+0000","last_became_active":"2024-04-25T08:39:07.200405+0000","last_became_peered":"2024-04-25T08:39:07.200405+0000","last_unstale":"2024-04-25T08:39:07.202189+0000","last_undegraded":"2024-04-25T08:39:07.202189+0000","last_fullsized":"2024-04-25T08:39:07.202189+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.824524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184135+0000","last_change":"2024-04-25T08:39:07.184135+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184135+0000","last_undegraded":"2024-04-25T08:39:07.184135+0000","last_fullsized":"2024-04-25T08:39:07.184135+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.197048+0000","last_change":"2024-04-25T08:39:07.195456+0000","last_active":"2024-04-25T08:39:07.197048+0000","last_peered":"2024-04-25T08:39:07.197048+0000","last_clean":"2024-04-25T08:39:07.197048+0000","last_became_active":"2024-04-25T08:39:07.194725+0000","last_became_peered":"2024-04-25T08:39:07.194725+0000","last_unstale":"2024-04-25T08:39:07.197048+0000","last_undegraded":"2024-04-25T08:39:07.197048+0000","last_fullsized":"2024-04-25T08:39:07.197048+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:20.621755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185569+0000","last_change":"2024-04-25T08:39:07.185569+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185569+0000","last_undegraded":"2024-04-25T08:39:07.185569+0000","last_fullsized":"2024-04-25T08:39:07.185569+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199508+0000","last_change":"2024-04-25T08:39:07.197963+0000","last_active":"2024-04-25T08:39:07.199508+0000","last_peered":"2024-04-25T08:39:07.199508+0000","last_clean":"2024-04-25T08:39:07.199508+0000","last_became_active":"2024-04-25T08:39:07.197658+0000","last_became_peered":"2024-04-25T08:39:07.197658+0000","last_unstale":"2024-04-25T08:39:07.199508+0000","last_undegraded":"2024-04-25T08:39:07.199508+0000","last_fullsized":"2024-04-25T08:39:07.199508+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:39.219462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183293+0000","last_change":"2024-04-25T08:39:07.183293+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183293+0000","last_undegraded":"2024-04-25T08:39:07.183293+0000","last_fullsized":"2024-04-25T08:39:07.183293+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192072+0000","last_change":"2024-04-25T08:39:07.191029+0000","last_active":"2024-04-25T08:39:07.192072+0000","last_peered":"2024-04-25T08:39:07.192072+0000","last_clean":"2024-04-25T08:39:07.192072+0000","last_became_active":"2024-04-25T08:39:07.190722+0000","last_became_peered":"2024-04-25T08:39:07.190722+0000","last_unstale":"2024-04-25T08:39:07.192072+0000","last_undegraded":"2024-04-25T08:39:07.192072+0000","last_fullsized":"2024-04-25T08:39:07.192072+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:46.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186738+0000","last_change":"2024-04-25T08:39:07.186738+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186738+0000","last_undegraded":"2024-04-25T08:39:07.186738+0000","last_fullsized":"2024-04-25T08:39:07.186738+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199195+0000","last_change":"2024-04-25T08:39:07.198565+0000","last_active":"2024-04-25T08:39:07.199195+0000","last_peered":"2024-04-25T08:39:07.199195+0000","last_clean":"2024-04-25T08:39:07.199195+0000","last_became_active":"2024-04-25T08:39:07.197861+0000","last_became_peered":"2024-04-25T08:39:07.197861+0000","last_unstale":"2024-04-25T08:39:07.199195+0000","last_undegraded":"2024-04-25T08:39:07.199195+0000","last_fullsized":"2024-04-25T08:39:07.199195+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:54.429666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181730+0000","last_change":"2024-04-25T08:39:07.181730+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181730+0000","last_undegraded":"2024-04-25T08:39:07.181730+0000","last_fullsized":"2024-04-25T08:39:07.181730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.195665+0000","last_change":"2024-04-25T08:39:07.191219+0000","last_active":"2024-04-25T08:39:07.195665+0000","last_peered":"2024-04-25T08:39:07.195665+0000","last_clean":"2024-04-25T08:39:07.195665+0000","last_became_active":"2024-04-25T08:39:07.190902+0000","last_became_peered":"2024-04-25T08:39:07.190902+0000","last_unstale":"2024-04-25T08:39:07.195665+0000","last_undegraded":"2024-04-25T08:39:07.195665+0000","last_fullsized":"2024-04-25T08:39:07.195665+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.078468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182971+0000","last_change":"2024-04-25T08:39:07.182971+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182971+0000","last_undegraded":"2024-04-25T08:39:07.182971+0000","last_fullsized":"2024-04-25T08:39:07.182971+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192534+0000","last_change":"2024-04-25T08:39:07.191760+0000","last_active":"2024-04-25T08:39:07.192534+0000","last_peered":"2024-04-25T08:39:07.192534+0000","last_clean":"2024-04-25T08:39:07.192534+0000","last_became_active":"2024-04-25T08:39:07.191248+0000","last_became_peered":"2024-04-25T08:39:07.191248+0000","last_unstale":"2024-04-25T08:39:07.192534+0000","last_undegraded":"2024-04-25T08:39:07.192534+0000","last_fullsized":"2024-04-25T08:39:07.192534+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:16.487240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185073+0000","last_change":"2024-04-25T08:39:07.185073+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185073+0000","last_undegraded":"2024-04-25T08:39:07.185073+0000","last_fullsized":"2024-04-25T08:39:07.185073+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199328+0000","last_change":"2024-04-25T08:39:07.198061+0000","last_active":"2024-04-25T08:39:07.199328+0000","last_peered":"2024-04-25T08:39:07.199328+0000","last_clean":"2024-04-25T08:39:07.199328+0000","last_became_active":"2024-04-25T08:39:07.197146+0000","last_became_peered":"2024-04-25T08:39:07.197146+0000","last_unstale":"2024-04-25T08:39:07.199328+0000","last_undegraded":"2024-04-25T08:39:07.199328+0000","last_fullsized":"2024-04-25T08:39:07.199328+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:39.219462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188107+0000","last_change":"2024-04-25T08:39:07.188107+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188107+0000","last_undegraded":"2024-04-25T08:39:07.188107+0000","last_fullsized":"2024-04-25T08:39:07.188107+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.197020+0000","last_change":"2024-04-25T08:39:07.195425+0000","last_active":"2024-04-25T08:39:07.197020+0000","last_peered":"2024-04-25T08:39:07.197020+0000","last_clean":"2024-04-25T08:39:07.197020+0000","last_became_active":"2024-04-25T08:39:07.195087+0000","last_became_peered":"2024-04-25T08:39:07.195087+0000","last_unstale":"2024-04-25T08:39:07.197020+0000","last_undegraded":"2024-04-25T08:39:07.197020+0000","last_fullsized":"2024-04-25T08:39:07.197020+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:52.006665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187086+0000","last_change":"2024-04-25T08:39:07.187086+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187086+0000","last_undegraded":"2024-04-25T08:39:07.187086+0000","last_fullsized":"2024-04-25T08:39:07.187086+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192581+0000","last_change":"2024-04-25T08:39:07.192065+0000","last_active":"2024-04-25T08:39:07.192581+0000","last_peered":"2024-04-25T08:39:07.192581+0000","last_clean":"2024-04-25T08:39:07.192581+0000","last_became_active":"2024-04-25T08:39:07.191757+0000","last_became_peered":"2024-04-25T08:39:07.191757+0000","last_unstale":"2024-04-25T08:39:07.192581+0000","last_undegraded":"2024-04-25T08:39:07.192581+0000","last_fullsized":"2024-04-25T08:39:07.192581+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:32.537457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.189381+0000","last_change":"2024-04-25T08:39:07.189381+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.189381+0000","last_undegraded":"2024-04-25T08:39:07.189381+0000","last_fullsized":"2024-04-25T08:39:07.189381+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198326+0000","last_change":"2024-04-25T08:39:07.196396+0000","last_active":"2024-04-25T08:39:07.198326+0000","last_peered":"2024-04-25T08:39:07.198326+0000","last_clean":"2024-04-25T08:39:07.198326+0000","last_became_active":"2024-04-25T08:39:07.195109+0000","last_became_peered":"2024-04-25T08:39:07.195109+0000","last_unstale":"2024-04-25T08:39:07.198326+0000","last_undegraded":"2024-04-25T08:39:07.198326+0000","last_fullsized":"2024-04-25T08:39:07.198326+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.078468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186932+0000","last_change":"2024-04-25T08:39:07.186932+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186932+0000","last_undegraded":"2024-04-25T08:39:07.186932+0000","last_fullsized":"2024-04-25T08:39:07.186932+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198619+0000","last_change":"2024-04-25T08:39:07.197589+0000","last_active":"2024-04-25T08:39:07.198619+0000","last_peered":"2024-04-25T08:39:07.198619+0000","last_clean":"2024-04-25T08:39:07.198619+0000","last_became_active":"2024-04-25T08:39:07.197301+0000","last_became_peered":"2024-04-25T08:39:07.197301+0000","last_unstale":"2024-04-25T08:39:07.198619+0000","last_undegraded":"2024-04-25T08:39:07.198619+0000","last_fullsized":"2024-04-25T08:39:07.198619+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:20.621755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181101+0000","last_change":"2024-04-25T08:39:07.181101+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181101+0000","last_undegraded":"2024-04-25T08:39:07.181101+0000","last_fullsized":"2024-04-25T08:39:07.181101+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.189989+0000","last_change":"2024-04-25T08:39:07.189044+0000","last_active":"2024-04-25T08:39:07.189989+0000","last_peered":"2024-04-25T08:39:07.189989+0000","last_clean":"2024-04-25T08:39:07.189989+0000","last_became_active":"2024-04-25T08:39:07.188847+0000","last_became_peered":"2024-04-25T08:39:07.188847+0000","last_unstale":"2024-04-25T08:39:07.189989+0000","last_undegraded":"2024-04-25T08:39:07.189989+0000","last_fullsized":"2024-04-25T08:39:07.189989+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:19.824202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188221+0000","last_change":"2024-04-25T08:39:07.188221+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188221+0000","last_undegraded":"2024-04-25T08:39:07.188221+0000","last_fullsized":"2024-04-25T08:39:07.188221+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.200155+0000","last_change":"2024-04-25T08:39:07.198202+0000","last_active":"2024-04-25T08:39:07.200155+0000","last_peered":"2024-04-25T08:39:07.200155+0000","last_clean":"2024-04-25T08:39:07.200155+0000","last_became_active":"2024-04-25T08:39:07.197778+0000","last_became_peered":"2024-04-25T08:39:07.197778+0000","last_unstale":"2024-04-25T08:39:07.200155+0000","last_undegraded":"2024-04-25T08:39:07.200155+0000","last_fullsized":"2024-04-25T08:39:07.200155+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:03.697252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183954+0000","last_change":"2024-04-25T08:39:07.183954+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183954+0000","last_undegraded":"2024-04-25T08:39:07.183954+0000","last_fullsized":"2024-04-25T08:39:07.183954+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.195707+0000","last_change":"2024-04-25T08:39:07.191103+0000","last_active":"2024-04-25T08:39:07.195707+0000","last_peered":"2024-04-25T08:39:07.195707+0000","last_clean":"2024-04-25T08:39:07.195707+0000","last_became_active":"2024-04-25T08:39:07.190789+0000","last_became_peered":"2024-04-25T08:39:07.190789+0000","last_unstale":"2024-04-25T08:39:07.195707+0000","last_undegraded":"2024-04-25T08:39:07.195707+0000","last_fullsized":"2024-04-25T08:39:07.195707+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:30.160350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185617+0000","last_change":"2024-04-25T08:39:07.185617+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185617+0000","last_undegraded":"2024-04-25T08:39:07.185617+0000","last_fullsized":"2024-04-25T08:39:07.185617+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.196625+0000","last_change":"2024-04-25T08:39:07.194224+0000","last_active":"2024-04-25T08:39:07.196625+0000","last_peered":"2024-04-25T08:39:07.196625+0000","last_clean":"2024-04-25T08:39:07.196625+0000","last_became_active":"2024-04-25T08:39:07.193936+0000","last_became_peered":"2024-04-25T08:39:07.193936+0000","last_unstale":"2024-04-25T08:39:07.196625+0000","last_undegraded":"2024-04-25T08:39:07.196625+0000","last_fullsized":"2024-04-25T08:39:07.196625+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:30.160350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181477+0000","last_change":"2024-04-25T08:39:07.181477+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181477+0000","last_undegraded":"2024-04-25T08:39:07.181477+0000","last_fullsized":"2024-04-25T08:39:07.181477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199346+0000","last_change":"2024-04-25T08:39:07.197977+0000","last_active":"2024-04-25T08:39:07.199346+0000","last_peered":"2024-04-25T08:39:07.199346+0000","last_clean":"2024-04-25T08:39:07.199346+0000","last_became_active":"2024-04-25T08:39:07.197678+0000","last_became_peered":"2024-04-25T08:39:07.197678+0000","last_unstale":"2024-04-25T08:39:07.199346+0000","last_undegraded":"2024-04-25T08:39:07.199346+0000","last_fullsized":"2024-04-25T08:39:07.199346+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:22.700613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.189910+0000","last_change":"2024-04-25T08:39:07.189910+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.189910+0000","last_undegraded":"2024-04-25T08:39:07.189910+0000","last_fullsized":"2024-04-25T08:39:07.189910+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198861+0000","last_change":"2024-04-25T08:39:07.197738+0000","last_active":"2024-04-25T08:39:07.198861+0000","last_peered":"2024-04-25T08:39:07.198861+0000","last_clean":"2024-04-25T08:39:07.198861+0000","last_became_active":"2024-04-25T08:39:07.197436+0000","last_became_peered":"2024-04-25T08:39:07.197436+0000","last_unstale":"2024-04-25T08:39:07.198861+0000","last_undegraded":"2024-04-25T08:39:07.198861+0000","last_fullsized":"2024-04-25T08:39:07.198861+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:39.317231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185375+0000","last_change":"2024-04-25T08:39:07.185375+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185375+0000","last_undegraded":"2024-04-25T08:39:07.185375+0000","last_fullsized":"2024-04-25T08:39:07.185375+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202185+0000","last_change":"2024-04-25T08:39:07.200590+0000","last_active":"2024-04-25T08:39:07.202185+0000","last_peered":"2024-04-25T08:39:07.202185+0000","last_clean":"2024-04-25T08:39:07.202185+0000","last_became_active":"2024-04-25T08:39:07.200312+0000","last_became_peered":"2024-04-25T08:39:07.200312+0000","last_unstale":"2024-04-25T08:39:07.202185+0000","last_undegraded":"2024-04-25T08:39:07.202185+0000","last_fullsized":"2024-04-25T08:39:07.202185+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:14.807183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187174+0000","last_change":"2024-04-25T08:39:07.187174+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187174+0000","last_undegraded":"2024-04-25T08:39:07.187174+0000","last_fullsized":"2024-04-25T08:39:07.187174+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202389+0000","last_change":"2024-04-25T08:39:07.201896+0000","last_active":"2024-04-25T08:39:07.202389+0000","last_peered":"2024-04-25T08:39:07.202389+0000","last_clean":"2024-04-25T08:39:07.202389+0000","last_became_active":"2024-04-25T08:39:07.201576+0000","last_became_peered":"2024-04-25T08:39:07.201576+0000","last_unstale":"2024-04-25T08:39:07.202389+0000","last_undegraded":"2024-04-25T08:39:07.202389+0000","last_fullsized":"2024-04-25T08:39:07.202389+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:42.629045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184184+0000","last_change":"2024-04-25T08:39:07.184184+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184184+0000","last_undegraded":"2024-04-25T08:39:07.184184+0000","last_fullsized":"2024-04-25T08:39:07.184184+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.195198+0000","last_change":"2024-04-25T08:39:07.193296+0000","last_active":"2024-04-25T08:39:07.195198+0000","last_peered":"2024-04-25T08:39:07.195198+0000","last_clean":"2024-04-25T08:39:07.195198+0000","last_became_active":"2024-04-25T08:39:07.193017+0000","last_became_peered":"2024-04-25T08:39:07.193017+0000","last_unstale":"2024-04-25T08:39:07.195198+0000","last_undegraded":"2024-04-25T08:39:07.195198+0000","last_fullsized":"2024-04-25T08:39:07.195198+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:51.280521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185656+0000","last_change":"2024-04-25T08:39:07.185656+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185656+0000","last_undegraded":"2024-04-25T08:39:07.185656+0000","last_fullsized":"2024-04-25T08:39:07.185656+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202406+0000","last_change":"2024-04-25T08:39:07.201924+0000","last_active":"2024-04-25T08:39:07.202406+0000","last_peered":"2024-04-25T08:39:07.202406+0000","last_clean":"2024-04-25T08:39:07.202406+0000","last_became_active":"2024-04-25T08:39:07.201634+0000","last_became_peered":"2024-04-25T08:39:07.201634+0000","last_unstale":"2024-04-25T08:39:07.202406+0000","last_undegraded":"2024-04-25T08:39:07.202406+0000","last_fullsized":"2024-04-25T08:39:07.202406+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:13.867447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185786+0000","last_change":"2024-04-25T08:39:07.185786+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185786+0000","last_undegraded":"2024-04-25T08:39:07.185786+0000","last_fullsized":"2024-04-25T08:39:07.185786+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.196826+0000","last_change":"2024-04-25T08:39:07.196017+0000","last_active":"2024-04-25T08:39:07.196826+0000","last_peered":"2024-04-25T08:39:07.196826+0000","last_clean":"2024-04-25T08:39:07.196826+0000","last_became_active":"2024-04-25T08:39:07.195702+0000","last_became_peered":"2024-04-25T08:39:07.195702+0000","last_unstale":"2024-04-25T08:39:07.196826+0000","last_undegraded":"2024-04-25T08:39:07.196826+0000","last_fullsized":"2024-04-25T08:39:07.196826+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.078468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182412+0000","last_change":"2024-04-25T08:39:07.182412+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182412+0000","last_undegraded":"2024-04-25T08:39:07.182412+0000","last_fullsized":"2024-04-25T08:39:07.182412+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.200126+0000","last_change":"2024-04-25T08:39:07.198773+0000","last_active":"2024-04-25T08:39:07.200126+0000","last_peered":"2024-04-25T08:39:07.200126+0000","last_clean":"2024-04-25T08:39:07.200126+0000","last_became_active":"2024-04-25T08:39:07.197463+0000","last_became_peered":"2024-04-25T08:39:07.197463+0000","last_unstale":"2024-04-25T08:39:07.200126+0000","last_undegraded":"2024-04-25T08:39:07.200126+0000","last_fullsized":"2024-04-25T08:39:07.200126+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:12.088493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183713+0000","last_change":"2024-04-25T08:39:07.183713+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183713+0000","last_undegraded":"2024-04-25T08:39:07.183713+0000","last_fullsized":"2024-04-25T08:39:07.183713+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.190171+0000","last_change":"2024-04-25T08:39:07.188334+0000","last_active":"2024-04-25T08:39:07.190171+0000","last_peered":"2024-04-25T08:39:07.190171+0000","last_clean":"2024-04-25T08:39:07.190171+0000","last_became_active":"2024-04-25T08:39:07.187795+0000","last_became_peered":"2024-04-25T08:39:07.187795+0000","last_unstale":"2024-04-25T08:39:07.190171+0000","last_undegraded":"2024-04-25T08:39:07.190171+0000","last_fullsized":"2024-04-25T08:39:07.190171+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:57.413804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186076+0000","last_change":"2024-04-25T08:39:07.186076+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186076+0000","last_undegraded":"2024-04-25T08:39:07.186076+0000","last_fullsized":"2024-04-25T08:39:07.186076+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198255+0000","last_change":"2024-04-25T08:39:07.197165+0000","last_active":"2024-04-25T08:39:07.198255+0000","last_peered":"2024-04-25T08:39:07.198255+0000","last_clean":"2024-04-25T08:39:07.198255+0000","last_became_active":"2024-04-25T08:39:07.196785+0000","last_became_peered":"2024-04-25T08:39:07.196785+0000","last_unstale":"2024-04-25T08:39:07.198255+0000","last_undegraded":"2024-04-25T08:39:07.198255+0000","last_fullsized":"2024-04-25T08:39:07.198255+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:27.118962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184647+0000","last_change":"2024-04-25T08:39:07.184647+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184647+0000","last_undegraded":"2024-04-25T08:39:07.184647+0000","last_fullsized":"2024-04-25T08:39:07.184647+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.196795+0000","last_change":"2024-04-25T08:39:07.193944+0000","last_active":"2024-04-25T08:39:07.196795+0000","last_peered":"2024-04-25T08:39:07.196795+0000","last_clean":"2024-04-25T08:39:07.196795+0000","last_became_active":"2024-04-25T08:39:07.193625+0000","last_became_peered":"2024-04-25T08:39:07.193625+0000","last_unstale":"2024-04-25T08:39:07.196795+0000","last_undegraded":"2024-04-25T08:39:07.196795+0000","last_fullsized":"2024-04-25T08:39:07.196795+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:27.118962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185152+0000","last_change":"2024-04-25T08:39:07.185152+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185152+0000","last_undegraded":"2024-04-25T08:39:07.185152+0000","last_fullsized":"2024-04-25T08:39:07.185152+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.200071+0000","last_change":"2024-04-25T08:39:07.198072+0000","last_active":"2024-04-25T08:39:07.200071+0000","last_peered":"2024-04-25T08:39:07.200071+0000","last_clean":"2024-04-25T08:39:07.200071+0000","last_became_active":"2024-04-25T08:39:07.197770+0000","last_became_peered":"2024-04-25T08:39:07.197770+0000","last_unstale":"2024-04-25T08:39:07.200071+0000","last_undegraded":"2024-04-25T08:39:07.200071+0000","last_fullsized":"2024-04-25T08:39:07.200071+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.421862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186576+0000","last_change":"2024-04-25T08:39:07.186576+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186576+0000","last_undegraded":"2024-04-25T08:39:07.186576+0000","last_fullsized":"2024-04-25T08:39:07.186576+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.190198+0000","last_change":"2024-04-25T08:39:07.188807+0000","last_active":"2024-04-25T08:39:07.190198+0000","last_peered":"2024-04-25T08:39:07.190198+0000","last_clean":"2024-04-25T08:39:07.190198+0000","last_became_active":"2024-04-25T08:39:07.188447+0000","last_became_peered":"2024-04-25T08:39:07.188447+0000","last_unstale":"2024-04-25T08:39:07.190198+0000","last_undegraded":"2024-04-25T08:39:07.190198+0000","last_fullsized":"2024-04-25T08:39:07.190198+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:22.020416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.190873+0000","last_change":"2024-04-25T08:39:07.190873+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.190873+0000","last_undegraded":"2024-04-25T08:39:07.190873+0000","last_fullsized":"2024-04-25T08:39:07.190873+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.197208+0000","last_change":"2024-04-25T08:39:07.196369+0000","last_active":"2024-04-25T08:39:07.197208+0000","last_peered":"2024-04-25T08:39:07.197208+0000","last_clean":"2024-04-25T08:39:07.197208+0000","last_became_active":"2024-04-25T08:39:07.196073+0000","last_became_peered":"2024-04-25T08:39:07.196073+0000","last_unstale":"2024-04-25T08:39:07.197208+0000","last_undegraded":"2024-04-25T08:39:07.197208+0000","last_fullsized":"2024-04-25T08:39:07.197208+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:55.439962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188049+0000","last_change":"2024-04-25T08:39:07.188049+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188049+0000","last_undegraded":"2024-04-25T08:39:07.188049+0000","last_fullsized":"2024-04-25T08:39:07.188049+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202209+0000","last_change":"2024-04-25T08:39:07.200791+0000","last_active":"2024-04-25T08:39:07.202209+0000","last_peered":"2024-04-25T08:39:07.202209+0000","last_clean":"2024-04-25T08:39:07.202209+0000","last_became_active":"2024-04-25T08:39:07.200495+0000","last_became_peered":"2024-04-25T08:39:07.200495+0000","last_unstale":"2024-04-25T08:39:07.202209+0000","last_undegraded":"2024-04-25T08:39:07.202209+0000","last_fullsized":"2024-04-25T08:39:07.202209+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:59.362497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":332,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.186152+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:39:07.186152+0000","last_peered":"2024-04-25T08:39:07.186152+0000","last_clean":"2024-04-25T08:39:07.186152+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:39:07.186152+0000","last_undegraded":"2024-04-25T08:39:07.186152+0000","last_fullsized":"2024-04-25T08:39:07.186152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59400,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312440,"statfs":{"total":96636764160,"available":96575938560,"internally_reserved":0,"allocated":1896448,"data_stored":1193192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54660,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317180,"statfs":{"total":96636764160,"available":96580792320,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54664,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317176,"statfs":{"total":96636764160,"available":96580788224,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68548,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303292,"statfs":{"total":96636764160,"available":96566571008,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68548,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303292,"statfs":{"total":96636764160,"available":96566571008,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58824,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313016,"statfs":{"total":96636764160,"available":96576528384,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54660,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317180,"statfs":{"total":96636764160,"available":96580792320,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46936,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324904,"statfs":{"total":96636764160,"available":96588701696,"internally_reserved":0,"allocated":1908736,"data_stored":1199859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:39:11.549 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:39:11.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.693+0000 7fc097ab5640 1 Processor -- start 2024-04-25T08:39:11.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.694+0000 7fc097ab5640 1 -- start start 2024-04-25T08:39:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.694+0000 7fc097ab5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc09015bb10 0x7fc09009a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.694+0000 7fc097ab5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc090160340 con 0x7fc09015fbf0 2024-04-25T08:39:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.694+0000 7fc097ab5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0900cf400 con 0x7fc0900d0460 2024-04-25T08:39:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.694+0000 7fc097ab5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0900cffd0 con 0x7fc09015bb10 2024-04-25T08:39:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.694+0000 7fc095851640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc09015bb10 0x7fc09009a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.694+0000 7fc096052640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc09015fbf0 0x7fc0900c5250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43598/0 (socket says 172.21.15.38:43598) 2024-04-25T08:39:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.695+0000 7fc096052640 1 -- 172.21.15.38:0/1318565165 learned_addr learned my addr 172.21.15.38:0/1318565165 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.695+0000 7fc095851640 1 -- 172.21.15.38:0/1318565165 >> v1:172.21.15.107:6789/0 conn(0x7fc0900d0460 legacy=0x7fc09015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.695+0000 7fc095851640 1 -- 172.21.15.38:0/1318565165 >> v1:172.21.15.38:6789/0 conn(0x7fc09015fbf0 legacy=0x7fc0900c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.695+0000 7fc095851640 1 -- 172.21.15.38:0/1318565165 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0900cf6a0 con 0x7fc09015bb10 2024-04-25T08:39:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.695+0000 7fc095851640 1 --2- 172.21.15.38:0/1318565165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc09015bb10 0x7fc09009a480 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc0880097a0 tx=0x7fc08802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafe231a79707857 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.696+0000 7fc09484f640 1 -- 172.21.15.38:0/1318565165 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088037070 con 0x7fc09015bb10 2024-04-25T08:39:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.696+0000 7fc09484f640 1 -- 172.21.15.38:0/1318565165 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc088032440 con 0x7fc09015bb10 2024-04-25T08:39:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.696+0000 7fc09484f640 1 -- 172.21.15.38:0/1318565165 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc088032810 con 0x7fc09015bb10 2024-04-25T08:39:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.696+0000 7fc097ab5640 1 -- 172.21.15.38:0/1318565165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc09015bb10 msgr2=0x7fc09009a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.696+0000 7fc097ab5640 1 --2- 172.21.15.38:0/1318565165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc09015bb10 0x7fc09009a480 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc0880097a0 tx=0x7fc08802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.696+0000 7fc097ab5640 1 -- 172.21.15.38:0/1318565165 shutdown_connections 2024-04-25T08:39:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.696+0000 7fc097ab5640 1 --2- 172.21.15.38:0/1318565165 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc09015bb10 0x7fc09009a480 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.696+0000 7fc097ab5640 1 -- 172.21.15.38:0/1318565165 >> 172.21.15.38:0/1318565165 conn(0x7fc0900b31f0 msgr2=0x7fc0900b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:11.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.696+0000 7fc097ab5640 1 -- 172.21.15.38:0/1318565165 shutdown_connections 2024-04-25T08:39:11.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.696+0000 7fc097ab5640 1 -- 172.21.15.38:0/1318565165 wait complete. 2024-04-25T08:39:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.698+0000 7fc097ab5640 1 Processor -- start 2024-04-25T08:39:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.698+0000 7fc097ab5640 1 -- start start 2024-04-25T08:39:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.698+0000 7fc097ab5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0900d0460 0x7fc0900c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.698+0000 7fc097ab5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc09015bb10 0x7fc09015eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.698+0000 7fc097ab5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc09015fbf0 0x7fc09006c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.698+0000 7fc097ab5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09009a9c0 con 0x7fc09015fbf0 2024-04-25T08:39:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.698+0000 7fc097ab5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0900cfc60 con 0x7fc0900d0460 2024-04-25T08:39:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.698+0000 7fc097ab5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09009ab40 con 0x7fc09015bb10 2024-04-25T08:39:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.698+0000 7fc096052640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc09015fbf0 0x7fc09006c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.698+0000 7fc096052640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc09015fbf0 0x7fc09006c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45052/0 (socket says 172.21.15.38:45052) 2024-04-25T08:39:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.698+0000 7fc096052640 1 -- 172.21.15.38:0/2245698599 learned_addr learned my addr 172.21.15.38:0/2245698599 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:11.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc095851640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0900d0460 0x7fc0900c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc095050640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc09015bb10 0x7fc09015eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc096052640 1 -- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc09015bb10 msgr2=0x7fc09015eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc096052640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc09015bb10 0x7fc09015eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc096052640 1 -- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0900d0460 msgr2=0x7fc0900c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc096052640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0900d0460 0x7fc0900c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc096052640 1 -- 172.21.15.38:0/2245698599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09015ca70 con 0x7fc09015fbf0 2024-04-25T08:39:11.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc095851640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0900d0460 0x7fc0900c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:11.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc096052640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc09015fbf0 0x7fc09006c700 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc08000ca20 tx=0x7fc08000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc08e7fc640 1 -- 172.21.15.38:0/2245698599 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080017070 con 0x7fc09015fbf0 2024-04-25T08:39:11.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc08e7fc640 1 -- 172.21.15.38:0/2245698599 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08000abd0 con 0x7fc09015fbf0 2024-04-25T08:39:11.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc08e7fc640 1 -- 172.21.15.38:0/2245698599 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080012400 con 0x7fc09015fbf0 2024-04-25T08:39:11.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc097ab5640 1 -- 172.21.15.38:0/2245698599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0900cb130 con 0x7fc09015fbf0 2024-04-25T08:39:11.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.699+0000 7fc097ab5640 1 -- 172.21.15.38:0/2245698599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0900ca5b0 con 0x7fc09015fbf0 2024-04-25T08:39:11.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.700+0000 7fc097ab5640 1 -- 172.21.15.38:0/2245698599 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc090068e80 con 0x7fc09015fbf0 2024-04-25T08:39:11.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.701+0000 7fc08e7fc640 1 -- 172.21.15.38:0/2245698599 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc08001a070 con 0x7fc09015fbf0 2024-04-25T08:39:11.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.701+0000 7fc08e7fc640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc07c06b500 0x7fc07c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:11.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.701+0000 7fc08e7fc640 1 -- 172.21.15.38:0/2245698599 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc080089e30 con 0x7fc09015fbf0 2024-04-25T08:39:11.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.701+0000 7fc095851640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc07c06b500 0x7fc07c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:11.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.702+0000 7fc095851640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc07c06b500 0x7fc07c06d9c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc088034000 tx=0x7fc08802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:11.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.705+0000 7fc08e7fc640 1 -- 172.21.15.38:0/2245698599 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc09015ca70 con 0x7fc09015fbf0 2024-04-25T08:39:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.884+0000 7fc097ab5640 1 -- 172.21.15.38:0/2245698599 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0900cdbf0 con 0x7fc07c06b500 2024-04-25T08:39:11.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.891+0000 7fc08e7fc640 1 -- 172.21.15.38:0/2245698599 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247075 (secure 0 0 0) 0x7fc0900cdbf0 con 0x7fc07c06b500 2024-04-25T08:39:11.887 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:39:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.894+0000 7fc097ab5640 1 -- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc07c06b500 msgr2=0x7fc07c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.894+0000 7fc097ab5640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc07c06b500 0x7fc07c06d9c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc088034000 tx=0x7fc08802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.894+0000 7fc097ab5640 1 -- 172.21.15.38:0/2245698599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc09015fbf0 msgr2=0x7fc09006c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.894+0000 7fc097ab5640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc09015fbf0 0x7fc09006c700 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc08000ca20 tx=0x7fc08000cee0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.894+0000 7fc097ab5640 1 -- 172.21.15.38:0/2245698599 shutdown_connections 2024-04-25T08:39:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.894+0000 7fc097ab5640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc09015fbf0 0x7fc09006c700 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.894+0000 7fc097ab5640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc07c06b500 0x7fc07c06d9c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.894+0000 7fc097ab5640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc09015bb10 0x7fc09015eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.894+0000 7fc097ab5640 1 --2- 172.21.15.38:0/2245698599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0900d0460 0x7fc0900c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.894+0000 7fc097ab5640 1 -- 172.21.15.38:0/2245698599 >> 172.21.15.38:0/2245698599 conn(0x7fc0900b31f0 msgr2=0x7fc09006f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.894+0000 7fc097ab5640 1 -- 172.21.15.38:0/2245698599 shutdown_connections 2024-04-25T08:39:11.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:11.894+0000 7fc097ab5640 1 -- 172.21.15.38:0/2245698599 wait complete. 2024-04-25T08:39:11.898 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-04-25T08:39:11.311618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466240,"kb_used_data":11348,"kb_used_omap":12,"kb_used_meta":454771,"kb_avail":754508480,"statfs":{"total":773094113280,"available":772616683520,"internally_reserved":0,"allocated":11620352,"data_stored":6009995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":465686064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.136186"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185002+0000","last_change":"2024-04-25T08:39:07.185002+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185002+0000","last_undegraded":"2024-04-25T08:39:07.185002+0000","last_fullsized":"2024-04-25T08:39:07.185002+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182883+0000","last_change":"2024-04-25T08:39:07.182883+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182883+0000","last_undegraded":"2024-04-25T08:39:07.182883+0000","last_fullsized":"2024-04-25T08:39:07.182883+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186081+0000","last_change":"2024-04-25T08:39:07.186081+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186081+0000","last_undegraded":"2024-04-25T08:39:07.186081+0000","last_fullsized":"2024-04-25T08:39:07.186081+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183803+0000","last_change":"2024-04-25T08:39:07.183803+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183803+0000","last_undegraded":"2024-04-25T08:39:07.183803+0000","last_fullsized":"2024-04-25T08:39:07.183803+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.190401+0000","last_change":"2024-04-25T08:39:07.190401+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.190401+0000","last_undegraded":"2024-04-25T08:39:07.190401+0000","last_fullsized":"2024-04-25T08:39:07.190401+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185128+0000","last_change":"2024-04-25T08:39:07.185128+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185128+0000","last_undegraded":"2024-04-25T08:39:07.185128+0000","last_fullsized":"2024-04-25T08:39:07.185128+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182530+0000","last_change":"2024-04-25T08:39:07.182530+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182530+0000","last_undegraded":"2024-04-25T08:39:07.182530+0000","last_fullsized":"2024-04-25T08:39:07.182530+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185168+0000","last_change":"2024-04-25T08:39:07.185168+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185168+0000","last_undegraded":"2024-04-25T08:39:07.185168+0000","last_fullsized":"2024-04-25T08:39:07.185168+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181457+0000","last_change":"2024-04-25T08:39:07.181457+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181457+0000","last_undegraded":"2024-04-25T08:39:07.181457+0000","last_fullsized":"2024-04-25T08:39:07.181457+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187122+0000","last_change":"2024-04-25T08:39:07.187122+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187122+0000","last_undegraded":"2024-04-25T08:39:07.187122+0000","last_fullsized":"2024-04-25T08:39:07.187122+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187732+0000","last_change":"2024-04-25T08:39:07.187732+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187732+0000","last_undegraded":"2024-04-25T08:39:07.187732+0000","last_fullsized":"2024-04-25T08:39:07.187732+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187078+0000","last_change":"2024-04-25T08:39:07.187078+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187078+0000","last_undegraded":"2024-04-25T08:39:07.187078+0000","last_fullsized":"2024-04-25T08:39:07.187078+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182812+0000","last_change":"2024-04-25T08:39:07.182812+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182812+0000","last_undegraded":"2024-04-25T08:39:07.182812+0000","last_fullsized":"2024-04-25T08:39:07.182812+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185729+0000","last_change":"2024-04-25T08:39:07.185729+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185729+0000","last_undegraded":"2024-04-25T08:39:07.185729+0000","last_fullsized":"2024-04-25T08:39:07.185729+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185317+0000","last_change":"2024-04-25T08:39:07.185317+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185317+0000","last_undegraded":"2024-04-25T08:39:07.185317+0000","last_fullsized":"2024-04-25T08:39:07.185317+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185201+0000","last_change":"2024-04-25T08:39:07.185201+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185201+0000","last_undegraded":"2024-04-25T08:39:07.185201+0000","last_fullsized":"2024-04-25T08:39:07.185201+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184442+0000","last_change":"2024-04-25T08:39:07.184442+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184442+0000","last_undegraded":"2024-04-25T08:39:07.184442+0000","last_fullsized":"2024-04-25T08:39:07.184442+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182220+0000","last_change":"2024-04-25T08:39:07.182220+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182220+0000","last_undegraded":"2024-04-25T08:39:07.182220+0000","last_fullsized":"2024-04-25T08:39:07.182220+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187629+0000","last_change":"2024-04-25T08:39:07.187629+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187629+0000","last_undegraded":"2024-04-25T08:39:07.187629+0000","last_fullsized":"2024-04-25T08:39:07.187629+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183719+0000","last_change":"2024-04-25T08:39:07.183719+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183719+0000","last_undegraded":"2024-04-25T08:39:07.183719+0000","last_fullsized":"2024-04-25T08:39:07.183719+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.193271+0000","last_change":"2024-04-25T08:39:07.193271+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.193271+0000","last_undegraded":"2024-04-25T08:39:07.193271+0000","last_fullsized":"2024-04-25T08:39:07.193271+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.189389+0000","last_change":"2024-04-25T08:39:07.189389+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.189389+0000","last_undegraded":"2024-04-25T08:39:07.189389+0000","last_fullsized":"2024-04-25T08:39:07.189389+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183901+0000","last_change":"2024-04-25T08:39:07.183901+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183901+0000","last_undegraded":"2024-04-25T08:39:07.183901+0000","last_fullsized":"2024-04-25T08:39:07.183901+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188396+0000","last_change":"2024-04-25T08:39:07.188396+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188396+0000","last_undegraded":"2024-04-25T08:39:07.188396+0000","last_fullsized":"2024-04-25T08:39:07.188396+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183544+0000","last_change":"2024-04-25T08:39:07.183544+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183544+0000","last_undegraded":"2024-04-25T08:39:07.183544+0000","last_fullsized":"2024-04-25T08:39:07.183544+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183684+0000","last_change":"2024-04-25T08:39:07.183684+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183684+0000","last_undegraded":"2024-04-25T08:39:07.183684+0000","last_fullsized":"2024-04-25T08:39:07.183684+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187277+0000","last_change":"2024-04-25T08:39:07.187277+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187277+0000","last_undegraded":"2024-04-25T08:39:07.187277+0000","last_fullsized":"2024-04-25T08:39:07.187277+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181602+0000","last_change":"2024-04-25T08:39:07.181602+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181602+0000","last_undegraded":"2024-04-25T08:39:07.181602+0000","last_fullsized":"2024-04-25T08:39:07.181602+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.192641+0000","last_change":"2024-04-25T08:39:07.192641+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.192641+0000","last_undegraded":"2024-04-25T08:39:07.192641+0000","last_fullsized":"2024-04-25T08:39:07.192641+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184579+0000","last_change":"2024-04-25T08:39:07.184579+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184579+0000","last_undegraded":"2024-04-25T08:39:07.184579+0000","last_fullsized":"2024-04-25T08:39:07.184579+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.180795+0000","last_change":"2024-04-25T08:39:07.180795+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.180795+0000","last_undegraded":"2024-04-25T08:39:07.180795+0000","last_fullsized":"2024-04-25T08:39:07.180795+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186769+0000","last_change":"2024-04-25T08:39:07.186769+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186769+0000","last_undegraded":"2024-04-25T08:39:07.186769+0000","last_fullsized":"2024-04-25T08:39:07.186769+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184486+0000","last_change":"2024-04-25T08:39:07.184486+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184486+0000","last_undegraded":"2024-04-25T08:39:07.184486+0000","last_fullsized":"2024-04-25T08:39:07.184486+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192158+0000","last_change":"2024-04-25T08:39:07.190276+0000","last_active":"2024-04-25T08:39:07.192158+0000","last_peered":"2024-04-25T08:39:07.192158+0000","last_clean":"2024-04-25T08:39:07.192158+0000","last_became_active":"2024-04-25T08:39:07.189857+0000","last_became_peered":"2024-04-25T08:39:07.189857+0000","last_unstale":"2024-04-25T08:39:07.192158+0000","last_undegraded":"2024-04-25T08:39:07.192158+0000","last_fullsized":"2024-04-25T08:39:07.192158+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:03.259504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187736+0000","last_change":"2024-04-25T08:39:07.187736+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187736+0000","last_undegraded":"2024-04-25T08:39:07.187736+0000","last_fullsized":"2024-04-25T08:39:07.187736+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202189+0000","last_change":"2024-04-25T08:39:07.200692+0000","last_active":"2024-04-25T08:39:07.202189+0000","last_peered":"2024-04-25T08:39:07.202189+0000","last_clean":"2024-04-25T08:39:07.202189+0000","last_became_active":"2024-04-25T08:39:07.200405+0000","last_became_peered":"2024-04-25T08:39:07.200405+0000","last_unstale":"2024-04-25T08:39:07.202189+0000","last_undegraded":"2024-04-25T08:39:07.202189+0000","last_fullsized":"2024-04-25T08:39:07.202189+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.824524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184135+0000","last_change":"2024-04-25T08:39:07.184135+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184135+0000","last_undegraded":"2024-04-25T08:39:07.184135+0000","last_fullsized":"2024-04-25T08:39:07.184135+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.197048+0000","last_change":"2024-04-25T08:39:07.195456+0000","last_active":"2024-04-25T08:39:07.197048+0000","last_peered":"2024-04-25T08:39:07.197048+0000","last_clean":"2024-04-25T08:39:07.197048+0000","last_became_active":"2024-04-25T08:39:07.194725+0000","last_became_peered":"2024-04-25T08:39:07.194725+0000","last_unstale":"2024-04-25T08:39:07.197048+0000","last_undegraded":"2024-04-25T08:39:07.197048+0000","last_fullsized":"2024-04-25T08:39:07.197048+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:20.621755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185569+0000","last_change":"2024-04-25T08:39:07.185569+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185569+0000","last_undegraded":"2024-04-25T08:39:07.185569+0000","last_fullsized":"2024-04-25T08:39:07.185569+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199508+0000","last_change":"2024-04-25T08:39:07.197963+0000","last_active":"2024-04-25T08:39:07.199508+0000","last_peered":"2024-04-25T08:39:07.199508+0000","last_clean":"2024-04-25T08:39:07.199508+0000","last_became_active":"2024-04-25T08:39:07.197658+0000","last_became_peered":"2024-04-25T08:39:07.197658+0000","last_unstale":"2024-04-25T08:39:07.199508+0000","last_undegraded":"2024-04-25T08:39:07.199508+0000","last_fullsized":"2024-04-25T08:39:07.199508+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:39.219462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183293+0000","last_change":"2024-04-25T08:39:07.183293+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183293+0000","last_undegraded":"2024-04-25T08:39:07.183293+0000","last_fullsized":"2024-04-25T08:39:07.183293+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192072+0000","last_change":"2024-04-25T08:39:07.191029+0000","last_active":"2024-04-25T08:39:07.192072+0000","last_peered":"2024-04-25T08:39:07.192072+0000","last_clean":"2024-04-25T08:39:07.192072+0000","last_became_active":"2024-04-25T08:39:07.190722+0000","last_became_peered":"2024-04-25T08:39:07.190722+0000","last_unstale":"2024-04-25T08:39:07.192072+0000","last_undegraded":"2024-04-25T08:39:07.192072+0000","last_fullsized":"2024-04-25T08:39:07.192072+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:46.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186738+0000","last_change":"2024-04-25T08:39:07.186738+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186738+0000","last_undegraded":"2024-04-25T08:39:07.186738+0000","last_fullsized":"2024-04-25T08:39:07.186738+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199195+0000","last_change":"2024-04-25T08:39:07.198565+0000","last_active":"2024-04-25T08:39:07.199195+0000","last_peered":"2024-04-25T08:39:07.199195+0000","last_clean":"2024-04-25T08:39:07.199195+0000","last_became_active":"2024-04-25T08:39:07.197861+0000","last_became_peered":"2024-04-25T08:39:07.197861+0000","last_unstale":"2024-04-25T08:39:07.199195+0000","last_undegraded":"2024-04-25T08:39:07.199195+0000","last_fullsized":"2024-04-25T08:39:07.199195+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:54.429666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181730+0000","last_change":"2024-04-25T08:39:07.181730+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181730+0000","last_undegraded":"2024-04-25T08:39:07.181730+0000","last_fullsized":"2024-04-25T08:39:07.181730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.195665+0000","last_change":"2024-04-25T08:39:07.191219+0000","last_active":"2024-04-25T08:39:07.195665+0000","last_peered":"2024-04-25T08:39:07.195665+0000","last_clean":"2024-04-25T08:39:07.195665+0000","last_became_active":"2024-04-25T08:39:07.190902+0000","last_became_peered":"2024-04-25T08:39:07.190902+0000","last_unstale":"2024-04-25T08:39:07.195665+0000","last_undegraded":"2024-04-25T08:39:07.195665+0000","last_fullsized":"2024-04-25T08:39:07.195665+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.078468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182971+0000","last_change":"2024-04-25T08:39:07.182971+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182971+0000","last_undegraded":"2024-04-25T08:39:07.182971+0000","last_fullsized":"2024-04-25T08:39:07.182971+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192534+0000","last_change":"2024-04-25T08:39:07.191760+0000","last_active":"2024-04-25T08:39:07.192534+0000","last_peered":"2024-04-25T08:39:07.192534+0000","last_clean":"2024-04-25T08:39:07.192534+0000","last_became_active":"2024-04-25T08:39:07.191248+0000","last_became_peered":"2024-04-25T08:39:07.191248+0000","last_unstale":"2024-04-25T08:39:07.192534+0000","last_undegraded":"2024-04-25T08:39:07.192534+0000","last_fullsized":"2024-04-25T08:39:07.192534+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:16.487240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185073+0000","last_change":"2024-04-25T08:39:07.185073+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185073+0000","last_undegraded":"2024-04-25T08:39:07.185073+0000","last_fullsized":"2024-04-25T08:39:07.185073+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199328+0000","last_change":"2024-04-25T08:39:07.198061+0000","last_active":"2024-04-25T08:39:07.199328+0000","last_peered":"2024-04-25T08:39:07.199328+0000","last_clean":"2024-04-25T08:39:07.199328+0000","last_became_active":"2024-04-25T08:39:07.197146+0000","last_became_peered":"2024-04-25T08:39:07.197146+0000","last_unstale":"2024-04-25T08:39:07.199328+0000","last_undegraded":"2024-04-25T08:39:07.199328+0000","last_fullsized":"2024-04-25T08:39:07.199328+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:39.219462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188107+0000","last_change":"2024-04-25T08:39:07.188107+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188107+0000","last_undegraded":"2024-04-25T08:39:07.188107+0000","last_fullsized":"2024-04-25T08:39:07.188107+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.197020+0000","last_change":"2024-04-25T08:39:07.195425+0000","last_active":"2024-04-25T08:39:07.197020+0000","last_peered":"2024-04-25T08:39:07.197020+0000","last_clean":"2024-04-25T08:39:07.197020+0000","last_became_active":"2024-04-25T08:39:07.195087+0000","last_became_peered":"2024-04-25T08:39:07.195087+0000","last_unstale":"2024-04-25T08:39:07.197020+0000","last_undegraded":"2024-04-25T08:39:07.197020+0000","last_fullsized":"2024-04-25T08:39:07.197020+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:52.006665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187086+0000","last_change":"2024-04-25T08:39:07.187086+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187086+0000","last_undegraded":"2024-04-25T08:39:07.187086+0000","last_fullsized":"2024-04-25T08:39:07.187086+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192581+0000","last_change":"2024-04-25T08:39:07.192065+0000","last_active":"2024-04-25T08:39:07.192581+0000","last_peered":"2024-04-25T08:39:07.192581+0000","last_clean":"2024-04-25T08:39:07.192581+0000","last_became_active":"2024-04-25T08:39:07.191757+0000","last_became_peered":"2024-04-25T08:39:07.191757+0000","last_unstale":"2024-04-25T08:39:07.192581+0000","last_undegraded":"2024-04-25T08:39:07.192581+0000","last_fullsized":"2024-04-25T08:39:07.192581+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:32.537457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.189381+0000","last_change":"2024-04-25T08:39:07.189381+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.189381+0000","last_undegraded":"2024-04-25T08:39:07.189381+0000","last_fullsized":"2024-04-25T08:39:07.189381+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198326+0000","last_change":"2024-04-25T08:39:07.196396+0000","last_active":"2024-04-25T08:39:07.198326+0000","last_peered":"2024-04-25T08:39:07.198326+0000","last_clean":"2024-04-25T08:39:07.198326+0000","last_became_active":"2024-04-25T08:39:07.195109+0000","last_became_peered":"2024-04-25T08:39:07.195109+0000","last_unstale":"2024-04-25T08:39:07.198326+0000","last_undegraded":"2024-04-25T08:39:07.198326+0000","last_fullsized":"2024-04-25T08:39:07.198326+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.078468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186932+0000","last_change":"2024-04-25T08:39:07.186932+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186932+0000","last_undegraded":"2024-04-25T08:39:07.186932+0000","last_fullsized":"2024-04-25T08:39:07.186932+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198619+0000","last_change":"2024-04-25T08:39:07.197589+0000","last_active":"2024-04-25T08:39:07.198619+0000","last_peered":"2024-04-25T08:39:07.198619+0000","last_clean":"2024-04-25T08:39:07.198619+0000","last_became_active":"2024-04-25T08:39:07.197301+0000","last_became_peered":"2024-04-25T08:39:07.197301+0000","last_unstale":"2024-04-25T08:39:07.198619+0000","last_undegraded":"2024-04-25T08:39:07.198619+0000","last_fullsized":"2024-04-25T08:39:07.198619+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:20.621755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181101+0000","last_change":"2024-04-25T08:39:07.181101+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181101+0000","last_undegraded":"2024-04-25T08:39:07.181101+0000","last_fullsized":"2024-04-25T08:39:07.181101+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.189989+0000","last_change":"2024-04-25T08:39:07.189044+0000","last_active":"2024-04-25T08:39:07.189989+0000","last_peered":"2024-04-25T08:39:07.189989+0000","last_clean":"2024-04-25T08:39:07.189989+0000","last_became_active":"2024-04-25T08:39:07.188847+0000","last_became_peered":"2024-04-25T08:39:07.188847+0000","last_unstale":"2024-04-25T08:39:07.189989+0000","last_undegraded":"2024-04-25T08:39:07.189989+0000","last_fullsized":"2024-04-25T08:39:07.189989+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:19.824202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188221+0000","last_change":"2024-04-25T08:39:07.188221+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188221+0000","last_undegraded":"2024-04-25T08:39:07.188221+0000","last_fullsized":"2024-04-25T08:39:07.188221+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.200155+0000","last_change":"2024-04-25T08:39:07.198202+0000","last_active":"2024-04-25T08:39:07.200155+0000","last_peered":"2024-04-25T08:39:07.200155+0000","last_clean":"2024-04-25T08:39:07.200155+0000","last_became_active":"2024-04-25T08:39:07.197778+0000","last_became_peered":"2024-04-25T08:39:07.197778+0000","last_unstale":"2024-04-25T08:39:07.200155+0000","last_undegraded":"2024-04-25T08:39:07.200155+0000","last_fullsized":"2024-04-25T08:39:07.200155+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:03.697252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183954+0000","last_change":"2024-04-25T08:39:07.183954+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183954+0000","last_undegraded":"2024-04-25T08:39:07.183954+0000","last_fullsized":"2024-04-25T08:39:07.183954+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.195707+0000","last_change":"2024-04-25T08:39:07.191103+0000","last_active":"2024-04-25T08:39:07.195707+0000","last_peered":"2024-04-25T08:39:07.195707+0000","last_clean":"2024-04-25T08:39:07.195707+0000","last_became_active":"2024-04-25T08:39:07.190789+0000","last_became_peered":"2024-04-25T08:39:07.190789+0000","last_unstale":"2024-04-25T08:39:07.195707+0000","last_undegraded":"2024-04-25T08:39:07.195707+0000","last_fullsized":"2024-04-25T08:39:07.195707+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:30.160350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185617+0000","last_change":"2024-04-25T08:39:07.185617+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185617+0000","last_undegraded":"2024-04-25T08:39:07.185617+0000","last_fullsized":"2024-04-25T08:39:07.185617+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.196625+0000","last_change":"2024-04-25T08:39:07.194224+0000","last_active":"2024-04-25T08:39:07.196625+0000","last_peered":"2024-04-25T08:39:07.196625+0000","last_clean":"2024-04-25T08:39:07.196625+0000","last_became_active":"2024-04-25T08:39:07.193936+0000","last_became_peered":"2024-04-25T08:39:07.193936+0000","last_unstale":"2024-04-25T08:39:07.196625+0000","last_undegraded":"2024-04-25T08:39:07.196625+0000","last_fullsized":"2024-04-25T08:39:07.196625+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:30.160350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181477+0000","last_change":"2024-04-25T08:39:07.181477+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181477+0000","last_undegraded":"2024-04-25T08:39:07.181477+0000","last_fullsized":"2024-04-25T08:39:07.181477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199346+0000","last_change":"2024-04-25T08:39:07.197977+0000","last_active":"2024-04-25T08:39:07.199346+0000","last_peered":"2024-04-25T08:39:07.199346+0000","last_clean":"2024-04-25T08:39:07.199346+0000","last_became_active":"2024-04-25T08:39:07.197678+0000","last_became_peered":"2024-04-25T08:39:07.197678+0000","last_unstale":"2024-04-25T08:39:07.199346+0000","last_undegraded":"2024-04-25T08:39:07.199346+0000","last_fullsized":"2024-04-25T08:39:07.199346+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:22.700613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.189910+0000","last_change":"2024-04-25T08:39:07.189910+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.189910+0000","last_undegraded":"2024-04-25T08:39:07.189910+0000","last_fullsized":"2024-04-25T08:39:07.189910+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198861+0000","last_change":"2024-04-25T08:39:07.197738+0000","last_active":"2024-04-25T08:39:07.198861+0000","last_peered":"2024-04-25T08:39:07.198861+0000","last_clean":"2024-04-25T08:39:07.198861+0000","last_became_active":"2024-04-25T08:39:07.197436+0000","last_became_peered":"2024-04-25T08:39:07.197436+0000","last_unstale":"2024-04-25T08:39:07.198861+0000","last_undegraded":"2024-04-25T08:39:07.198861+0000","last_fullsized":"2024-04-25T08:39:07.198861+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:39.317231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185375+0000","last_change":"2024-04-25T08:39:07.185375+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185375+0000","last_undegraded":"2024-04-25T08:39:07.185375+0000","last_fullsized":"2024-04-25T08:39:07.185375+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202185+0000","last_change":"2024-04-25T08:39:07.200590+0000","last_active":"2024-04-25T08:39:07.202185+0000","last_peered":"2024-04-25T08:39:07.202185+0000","last_clean":"2024-04-25T08:39:07.202185+0000","last_became_active":"2024-04-25T08:39:07.200312+0000","last_became_peered":"2024-04-25T08:39:07.200312+0000","last_unstale":"2024-04-25T08:39:07.202185+0000","last_undegraded":"2024-04-25T08:39:07.202185+0000","last_fullsized":"2024-04-25T08:39:07.202185+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:14.807183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187174+0000","last_change":"2024-04-25T08:39:07.187174+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187174+0000","last_undegraded":"2024-04-25T08:39:07.187174+0000","last_fullsized":"2024-04-25T08:39:07.187174+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202389+0000","last_change":"2024-04-25T08:39:07.201896+0000","last_active":"2024-04-25T08:39:07.202389+0000","last_peered":"2024-04-25T08:39:07.202389+0000","last_clean":"2024-04-25T08:39:07.202389+0000","last_became_active":"2024-04-25T08:39:07.201576+0000","last_became_peered":"2024-04-25T08:39:07.201576+0000","last_unstale":"2024-04-25T08:39:07.202389+0000","last_undegraded":"2024-04-25T08:39:07.202389+0000","last_fullsized":"2024-04-25T08:39:07.202389+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:42.629045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184184+0000","last_change":"2024-04-25T08:39:07.184184+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184184+0000","last_undegraded":"2024-04-25T08:39:07.184184+0000","last_fullsized":"2024-04-25T08:39:07.184184+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.195198+0000","last_change":"2024-04-25T08:39:07.193296+0000","last_active":"2024-04-25T08:39:07.195198+0000","last_peered":"2024-04-25T08:39:07.195198+0000","last_clean":"2024-04-25T08:39:07.195198+0000","last_became_active":"2024-04-25T08:39:07.193017+0000","last_became_peered":"2024-04-25T08:39:07.193017+0000","last_unstale":"2024-04-25T08:39:07.195198+0000","last_undegraded":"2024-04-25T08:39:07.195198+0000","last_fullsized":"2024-04-25T08:39:07.195198+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:51.280521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185656+0000","last_change":"2024-04-25T08:39:07.185656+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185656+0000","last_undegraded":"2024-04-25T08:39:07.185656+0000","last_fullsized":"2024-04-25T08:39:07.185656+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202406+0000","last_change":"2024-04-25T08:39:07.201924+0000","last_active":"2024-04-25T08:39:07.202406+0000","last_peered":"2024-04-25T08:39:07.202406+0000","last_clean":"2024-04-25T08:39:07.202406+0000","last_became_active":"2024-04-25T08:39:07.201634+0000","last_became_peered":"2024-04-25T08:39:07.201634+0000","last_unstale":"2024-04-25T08:39:07.202406+0000","last_undegraded":"2024-04-25T08:39:07.202406+0000","last_fullsized":"2024-04-25T08:39:07.202406+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:13.867447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185786+0000","last_change":"2024-04-25T08:39:07.185786+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185786+0000","last_undegraded":"2024-04-25T08:39:07.185786+0000","last_fullsized":"2024-04-25T08:39:07.185786+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.196826+0000","last_change":"2024-04-25T08:39:07.196017+0000","last_active":"2024-04-25T08:39:07.196826+0000","last_peered":"2024-04-25T08:39:07.196826+0000","last_clean":"2024-04-25T08:39:07.196826+0000","last_became_active":"2024-04-25T08:39:07.195702+0000","last_became_peered":"2024-04-25T08:39:07.195702+0000","last_unstale":"2024-04-25T08:39:07.196826+0000","last_undegraded":"2024-04-25T08:39:07.196826+0000","last_fullsized":"2024-04-25T08:39:07.196826+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.078468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182412+0000","last_change":"2024-04-25T08:39:07.182412+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182412+0000","last_undegraded":"2024-04-25T08:39:07.182412+0000","last_fullsized":"2024-04-25T08:39:07.182412+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.200126+0000","last_change":"2024-04-25T08:39:07.198773+0000","last_active":"2024-04-25T08:39:07.200126+0000","last_peered":"2024-04-25T08:39:07.200126+0000","last_clean":"2024-04-25T08:39:07.200126+0000","last_became_active":"2024-04-25T08:39:07.197463+0000","last_became_peered":"2024-04-25T08:39:07.197463+0000","last_unstale":"2024-04-25T08:39:07.200126+0000","last_undegraded":"2024-04-25T08:39:07.200126+0000","last_fullsized":"2024-04-25T08:39:07.200126+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:12.088493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183713+0000","last_change":"2024-04-25T08:39:07.183713+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183713+0000","last_undegraded":"2024-04-25T08:39:07.183713+0000","last_fullsized":"2024-04-25T08:39:07.183713+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.190171+0000","last_change":"2024-04-25T08:39:07.188334+0000","last_active":"2024-04-25T08:39:07.190171+0000","last_peered":"2024-04-25T08:39:07.190171+0000","last_clean":"2024-04-25T08:39:07.190171+0000","last_became_active":"2024-04-25T08:39:07.187795+0000","last_became_peered":"2024-04-25T08:39:07.187795+0000","last_unstale":"2024-04-25T08:39:07.190171+0000","last_undegraded":"2024-04-25T08:39:07.190171+0000","last_fullsized":"2024-04-25T08:39:07.190171+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:57.413804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186076+0000","last_change":"2024-04-25T08:39:07.186076+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186076+0000","last_undegraded":"2024-04-25T08:39:07.186076+0000","last_fullsized":"2024-04-25T08:39:07.186076+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198255+0000","last_change":"2024-04-25T08:39:07.197165+0000","last_active":"2024-04-25T08:39:07.198255+0000","last_peered":"2024-04-25T08:39:07.198255+0000","last_clean":"2024-04-25T08:39:07.198255+0000","last_became_active":"2024-04-25T08:39:07.196785+0000","last_became_peered":"2024-04-25T08:39:07.196785+0000","last_unstale":"2024-04-25T08:39:07.198255+0000","last_undegraded":"2024-04-25T08:39:07.198255+0000","last_fullsized":"2024-04-25T08:39:07.198255+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:27.118962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184647+0000","last_change":"2024-04-25T08:39:07.184647+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184647+0000","last_undegraded":"2024-04-25T08:39:07.184647+0000","last_fullsized":"2024-04-25T08:39:07.184647+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.196795+0000","last_change":"2024-04-25T08:39:07.193944+0000","last_active":"2024-04-25T08:39:07.196795+0000","last_peered":"2024-04-25T08:39:07.196795+0000","last_clean":"2024-04-25T08:39:07.196795+0000","last_became_active":"2024-04-25T08:39:07.193625+0000","last_became_peered":"2024-04-25T08:39:07.193625+0000","last_unstale":"2024-04-25T08:39:07.196795+0000","last_undegraded":"2024-04-25T08:39:07.196795+0000","last_fullsized":"2024-04-25T08:39:07.196795+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:27.118962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185152+0000","last_change":"2024-04-25T08:39:07.185152+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185152+0000","last_undegraded":"2024-04-25T08:39:07.185152+0000","last_fullsized":"2024-04-25T08:39:07.185152+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.200071+0000","last_change":"2024-04-25T08:39:07.198072+0000","last_active":"2024-04-25T08:39:07.200071+0000","last_peered":"2024-04-25T08:39:07.200071+0000","last_clean":"2024-04-25T08:39:07.200071+0000","last_became_active":"2024-04-25T08:39:07.197770+0000","last_became_peered":"2024-04-25T08:39:07.197770+0000","last_unstale":"2024-04-25T08:39:07.200071+0000","last_undegraded":"2024-04-25T08:39:07.200071+0000","last_fullsized":"2024-04-25T08:39:07.200071+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.421862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186576+0000","last_change":"2024-04-25T08:39:07.186576+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186576+0000","last_undegraded":"2024-04-25T08:39:07.186576+0000","last_fullsized":"2024-04-25T08:39:07.186576+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.190198+0000","last_change":"2024-04-25T08:39:07.188807+0000","last_active":"2024-04-25T08:39:07.190198+0000","last_peered":"2024-04-25T08:39:07.190198+0000","last_clean":"2024-04-25T08:39:07.190198+0000","last_became_active":"2024-04-25T08:39:07.188447+0000","last_became_peered":"2024-04-25T08:39:07.188447+0000","last_unstale":"2024-04-25T08:39:07.190198+0000","last_undegraded":"2024-04-25T08:39:07.190198+0000","last_fullsized":"2024-04-25T08:39:07.190198+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:22.020416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.190873+0000","last_change":"2024-04-25T08:39:07.190873+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.190873+0000","last_undegraded":"2024-04-25T08:39:07.190873+0000","last_fullsized":"2024-04-25T08:39:07.190873+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.197208+0000","last_change":"2024-04-25T08:39:07.196369+0000","last_active":"2024-04-25T08:39:07.197208+0000","last_peered":"2024-04-25T08:39:07.197208+0000","last_clean":"2024-04-25T08:39:07.197208+0000","last_became_active":"2024-04-25T08:39:07.196073+0000","last_became_peered":"2024-04-25T08:39:07.196073+0000","last_unstale":"2024-04-25T08:39:07.197208+0000","last_undegraded":"2024-04-25T08:39:07.197208+0000","last_fullsized":"2024-04-25T08:39:07.197208+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:55.439962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188049+0000","last_change":"2024-04-25T08:39:07.188049+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188049+0000","last_undegraded":"2024-04-25T08:39:07.188049+0000","last_fullsized":"2024-04-25T08:39:07.188049+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202209+0000","last_change":"2024-04-25T08:39:07.200791+0000","last_active":"2024-04-25T08:39:07.202209+0000","last_peered":"2024-04-25T08:39:07.202209+0000","last_clean":"2024-04-25T08:39:07.202209+0000","last_became_active":"2024-04-25T08:39:07.200495+0000","last_became_peered":"2024-04-25T08:39:07.200495+0000","last_unstale":"2024-04-25T08:39:07.202209+0000","last_undegraded":"2024-04-25T08:39:07.202209+0000","last_fullsized":"2024-04-25T08:39:07.202209+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:59.362497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":332,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.186152+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:39:07.186152+0000","last_peered":"2024-04-25T08:39:07.186152+0000","last_clean":"2024-04-25T08:39:07.186152+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:39:07.186152+0000","last_undegraded":"2024-04-25T08:39:07.186152+0000","last_fullsized":"2024-04-25T08:39:07.186152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59400,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312440,"statfs":{"total":96636764160,"available":96575938560,"internally_reserved":0,"allocated":1896448,"data_stored":1193192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54660,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317180,"statfs":{"total":96636764160,"available":96580792320,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54664,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317176,"statfs":{"total":96636764160,"available":96580788224,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68548,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303292,"statfs":{"total":96636764160,"available":96566571008,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68548,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303292,"statfs":{"total":96636764160,"available":96566571008,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58824,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313016,"statfs":{"total":96636764160,"available":96576528384,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54660,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317180,"statfs":{"total":96636764160,"available":96580792320,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46936,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324904,"statfs":{"total":96636764160,"available":96588701696,"internally_reserved":0,"allocated":1908736,"data_stored":1199859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:39:11.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3f is not active+clean 2024-04-25T08:39:11.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185002+0000', 'last_change': '2024-04-25T08:39:07.185002+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185002+0000', 'last_undegraded': '2024-04-25T08:39:07.185002+0000', 'last_fullsized': '2024-04-25T08:39:07.185002+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:11.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3e is not active+clean 2024-04-25T08:39:11.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.182883+0000', 'last_change': '2024-04-25T08:39:07.182883+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.182883+0000', 'last_undegraded': '2024-04-25T08:39:07.182883+0000', 'last_fullsized': '2024-04-25T08:39:07.182883+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:11.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3d is not active+clean 2024-04-25T08:39:11.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.186081+0000', 'last_change': '2024-04-25T08:39:07.186081+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.186081+0000', 'last_undegraded': '2024-04-25T08:39:07.186081+0000', 'last_fullsized': '2024-04-25T08:39:07.186081+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:11.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3c is not active+clean 2024-04-25T08:39:11.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.183803+0000', 'last_change': '2024-04-25T08:39:07.183803+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.183803+0000', 'last_undegraded': '2024-04-25T08:39:07.183803+0000', 'last_fullsized': '2024-04-25T08:39:07.183803+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:39:11.902 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3b is not active+clean 2024-04-25T08:39:11.902 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.190401+0000', 'last_change': '2024-04-25T08:39:07.190401+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.190401+0000', 'last_undegraded': '2024-04-25T08:39:07.190401+0000', 'last_fullsized': '2024-04-25T08:39:07.190401+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:39:11.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3a is not active+clean 2024-04-25T08:39:11.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185128+0000', 'last_change': '2024-04-25T08:39:07.185128+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185128+0000', 'last_undegraded': '2024-04-25T08:39:07.185128+0000', 'last_fullsized': '2024-04-25T08:39:07.185128+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:11.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.39 is not active+clean 2024-04-25T08:39:11.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.182530+0000', 'last_change': '2024-04-25T08:39:07.182530+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.182530+0000', 'last_undegraded': '2024-04-25T08:39:07.182530+0000', 'last_fullsized': '2024-04-25T08:39:07.182530+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:11.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.38 is not active+clean 2024-04-25T08:39:11.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185168+0000', 'last_change': '2024-04-25T08:39:07.185168+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185168+0000', 'last_undegraded': '2024-04-25T08:39:07.185168+0000', 'last_fullsized': '2024-04-25T08:39:07.185168+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:11.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.37 is not active+clean 2024-04-25T08:39:11.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.181457+0000', 'last_change': '2024-04-25T08:39:07.181457+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.181457+0000', 'last_undegraded': '2024-04-25T08:39:07.181457+0000', 'last_fullsized': '2024-04-25T08:39:07.181457+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:11.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.36 is not active+clean 2024-04-25T08:39:11.903 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.187122+0000', 'last_change': '2024-04-25T08:39:07.187122+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.187122+0000', 'last_undegraded': '2024-04-25T08:39:07.187122+0000', 'last_fullsized': '2024-04-25T08:39:07.187122+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:39:11.903 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.35 is not active+clean 2024-04-25T08:39:11.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.187732+0000', 'last_change': '2024-04-25T08:39:07.187732+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.187732+0000', 'last_undegraded': '2024-04-25T08:39:07.187732+0000', 'last_fullsized': '2024-04-25T08:39:07.187732+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:11.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2024-04-25T08:39:11.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.187078+0000', 'last_change': '2024-04-25T08:39:07.187078+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.187078+0000', 'last_undegraded': '2024-04-25T08:39:07.187078+0000', 'last_fullsized': '2024-04-25T08:39:07.187078+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:39:11.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.33 is not active+clean 2024-04-25T08:39:11.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.182812+0000', 'last_change': '2024-04-25T08:39:07.182812+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.182812+0000', 'last_undegraded': '2024-04-25T08:39:07.182812+0000', 'last_fullsized': '2024-04-25T08:39:07.182812+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:39:11.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.32 is not active+clean 2024-04-25T08:39:11.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185729+0000', 'last_change': '2024-04-25T08:39:07.185729+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185729+0000', 'last_undegraded': '2024-04-25T08:39:07.185729+0000', 'last_fullsized': '2024-04-25T08:39:07.185729+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:39:11.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.31 is not active+clean 2024-04-25T08:39:11.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185317+0000', 'last_change': '2024-04-25T08:39:07.185317+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185317+0000', 'last_undegraded': '2024-04-25T08:39:07.185317+0000', 'last_fullsized': '2024-04-25T08:39:07.185317+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:39:11.904 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.30 is not active+clean 2024-04-25T08:39:11.904 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185201+0000', 'last_change': '2024-04-25T08:39:07.185201+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185201+0000', 'last_undegraded': '2024-04-25T08:39:07.185201+0000', 'last_fullsized': '2024-04-25T08:39:07.185201+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2f is not active+clean 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.184442+0000', 'last_change': '2024-04-25T08:39:07.184442+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.184442+0000', 'last_undegraded': '2024-04-25T08:39:07.184442+0000', 'last_fullsized': '2024-04-25T08:39:07.184442+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2e is not active+clean 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.182220+0000', 'last_change': '2024-04-25T08:39:07.182220+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.182220+0000', 'last_undegraded': '2024-04-25T08:39:07.182220+0000', 'last_fullsized': '2024-04-25T08:39:07.182220+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2d is not active+clean 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.187629+0000', 'last_change': '2024-04-25T08:39:07.187629+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.187629+0000', 'last_undegraded': '2024-04-25T08:39:07.187629+0000', 'last_fullsized': '2024-04-25T08:39:07.187629+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.183719+0000', 'last_change': '2024-04-25T08:39:07.183719+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.183719+0000', 'last_undegraded': '2024-04-25T08:39:07.183719+0000', 'last_fullsized': '2024-04-25T08:39:07.183719+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2b is not active+clean 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.193271+0000', 'last_change': '2024-04-25T08:39:07.193271+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.193271+0000', 'last_undegraded': '2024-04-25T08:39:07.193271+0000', 'last_fullsized': '2024-04-25T08:39:07.193271+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2a is not active+clean 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.189389+0000', 'last_change': '2024-04-25T08:39:07.189389+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.189389+0000', 'last_undegraded': '2024-04-25T08:39:07.189389+0000', 'last_fullsized': '2024-04-25T08:39:07.189389+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:39:11.905 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.29 is not active+clean 2024-04-25T08:39:11.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.183901+0000', 'last_change': '2024-04-25T08:39:07.183901+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.183901+0000', 'last_undegraded': '2024-04-25T08:39:07.183901+0000', 'last_fullsized': '2024-04-25T08:39:07.183901+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:11.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.28 is not active+clean 2024-04-25T08:39:11.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.188396+0000', 'last_change': '2024-04-25T08:39:07.188396+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.188396+0000', 'last_undegraded': '2024-04-25T08:39:07.188396+0000', 'last_fullsized': '2024-04-25T08:39:07.188396+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:39:11.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2024-04-25T08:39:11.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.183544+0000', 'last_change': '2024-04-25T08:39:07.183544+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.183544+0000', 'last_undegraded': '2024-04-25T08:39:07.183544+0000', 'last_fullsized': '2024-04-25T08:39:07.183544+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:39:11.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2024-04-25T08:39:11.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.183684+0000', 'last_change': '2024-04-25T08:39:07.183684+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.183684+0000', 'last_undegraded': '2024-04-25T08:39:07.183684+0000', 'last_fullsized': '2024-04-25T08:39:07.183684+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:39:11.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.25 is not active+clean 2024-04-25T08:39:11.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.187277+0000', 'last_change': '2024-04-25T08:39:07.187277+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.187277+0000', 'last_undegraded': '2024-04-25T08:39:07.187277+0000', 'last_fullsized': '2024-04-25T08:39:07.187277+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:11.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.24 is not active+clean 2024-04-25T08:39:11.906 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.181602+0000', 'last_change': '2024-04-25T08:39:07.181602+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.181602+0000', 'last_undegraded': '2024-04-25T08:39:07.181602+0000', 'last_fullsized': '2024-04-25T08:39:07.181602+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:11.906 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.23 is not active+clean 2024-04-25T08:39:11.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.192641+0000', 'last_change': '2024-04-25T08:39:07.192641+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.192641+0000', 'last_undegraded': '2024-04-25T08:39:07.192641+0000', 'last_fullsized': '2024-04-25T08:39:07.192641+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:11.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2024-04-25T08:39:11.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.184579+0000', 'last_change': '2024-04-25T08:39:07.184579+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.184579+0000', 'last_undegraded': '2024-04-25T08:39:07.184579+0000', 'last_fullsized': '2024-04-25T08:39:07.184579+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:39:11.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.21 is not active+clean 2024-04-25T08:39:11.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.180795+0000', 'last_change': '2024-04-25T08:39:07.180795+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.180795+0000', 'last_undegraded': '2024-04-25T08:39:07.180795+0000', 'last_fullsized': '2024-04-25T08:39:07.180795+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:11.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.20 is not active+clean 2024-04-25T08:39:11.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.186769+0000', 'last_change': '2024-04-25T08:39:07.186769+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.186769+0000', 'last_undegraded': '2024-04-25T08:39:07.186769+0000', 'last_fullsized': '2024-04-25T08:39:07.186769+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:11.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2024-04-25T08:39:11.907 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.184486+0000', 'last_change': '2024-04-25T08:39:07.184486+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.184486+0000', 'last_undegraded': '2024-04-25T08:39:07.184486+0000', 'last_fullsized': '2024-04-25T08:39:07.184486+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:11.907 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2024-04-25T08:39:11.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.187736+0000', 'last_change': '2024-04-25T08:39:07.187736+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.187736+0000', 'last_undegraded': '2024-04-25T08:39:07.187736+0000', 'last_fullsized': '2024-04-25T08:39:07.187736+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:11.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2024-04-25T08:39:11.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.184135+0000', 'last_change': '2024-04-25T08:39:07.184135+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.184135+0000', 'last_undegraded': '2024-04-25T08:39:07.184135+0000', 'last_fullsized': '2024-04-25T08:39:07.184135+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:39:11.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2024-04-25T08:39:11.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185569+0000', 'last_change': '2024-04-25T08:39:07.185569+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185569+0000', 'last_undegraded': '2024-04-25T08:39:07.185569+0000', 'last_fullsized': '2024-04-25T08:39:07.185569+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:11.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2024-04-25T08:39:11.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.183293+0000', 'last_change': '2024-04-25T08:39:07.183293+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.183293+0000', 'last_undegraded': '2024-04-25T08:39:07.183293+0000', 'last_fullsized': '2024-04-25T08:39:07.183293+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:39:11.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2024-04-25T08:39:11.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.186738+0000', 'last_change': '2024-04-25T08:39:07.186738+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.186738+0000', 'last_undegraded': '2024-04-25T08:39:07.186738+0000', 'last_fullsized': '2024-04-25T08:39:07.186738+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:11.908 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2024-04-25T08:39:11.908 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.181730+0000', 'last_change': '2024-04-25T08:39:07.181730+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.181730+0000', 'last_undegraded': '2024-04-25T08:39:07.181730+0000', 'last_fullsized': '2024-04-25T08:39:07.181730+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.182971+0000', 'last_change': '2024-04-25T08:39:07.182971+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.182971+0000', 'last_undegraded': '2024-04-25T08:39:07.182971+0000', 'last_fullsized': '2024-04-25T08:39:07.182971+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185073+0000', 'last_change': '2024-04-25T08:39:07.185073+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185073+0000', 'last_undegraded': '2024-04-25T08:39:07.185073+0000', 'last_fullsized': '2024-04-25T08:39:07.185073+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.188107+0000', 'last_change': '2024-04-25T08:39:07.188107+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.188107+0000', 'last_undegraded': '2024-04-25T08:39:07.188107+0000', 'last_fullsized': '2024-04-25T08:39:07.188107+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.187086+0000', 'last_change': '2024-04-25T08:39:07.187086+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.187086+0000', 'last_undegraded': '2024-04-25T08:39:07.187086+0000', 'last_fullsized': '2024-04-25T08:39:07.187086+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.189381+0000', 'last_change': '2024-04-25T08:39:07.189381+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.189381+0000', 'last_undegraded': '2024-04-25T08:39:07.189381+0000', 'last_fullsized': '2024-04-25T08:39:07.189381+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.186932+0000', 'last_change': '2024-04-25T08:39:07.186932+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.186932+0000', 'last_undegraded': '2024-04-25T08:39:07.186932+0000', 'last_fullsized': '2024-04-25T08:39:07.186932+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:11.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2024-04-25T08:39:11.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.181101+0000', 'last_change': '2024-04-25T08:39:07.181101+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.181101+0000', 'last_undegraded': '2024-04-25T08:39:07.181101+0000', 'last_fullsized': '2024-04-25T08:39:07.181101+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:11.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2024-04-25T08:39:11.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.188221+0000', 'last_change': '2024-04-25T08:39:07.188221+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.188221+0000', 'last_undegraded': '2024-04-25T08:39:07.188221+0000', 'last_fullsized': '2024-04-25T08:39:07.188221+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:11.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2024-04-25T08:39:11.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.183954+0000', 'last_change': '2024-04-25T08:39:07.183954+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.183954+0000', 'last_undegraded': '2024-04-25T08:39:07.183954+0000', 'last_fullsized': '2024-04-25T08:39:07.183954+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:11.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2024-04-25T08:39:11.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185617+0000', 'last_change': '2024-04-25T08:39:07.185617+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185617+0000', 'last_undegraded': '2024-04-25T08:39:07.185617+0000', 'last_fullsized': '2024-04-25T08:39:07.185617+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:11.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2024-04-25T08:39:11.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.181477+0000', 'last_change': '2024-04-25T08:39:07.181477+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.181477+0000', 'last_undegraded': '2024-04-25T08:39:07.181477+0000', 'last_fullsized': '2024-04-25T08:39:07.181477+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:39:11.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2024-04-25T08:39:11.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.189910+0000', 'last_change': '2024-04-25T08:39:07.189910+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.189910+0000', 'last_undegraded': '2024-04-25T08:39:07.189910+0000', 'last_fullsized': '2024-04-25T08:39:07.189910+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:39:11.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2024-04-25T08:39:11.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185375+0000', 'last_change': '2024-04-25T08:39:07.185375+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185375+0000', 'last_undegraded': '2024-04-25T08:39:07.185375+0000', 'last_fullsized': '2024-04-25T08:39:07.185375+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:11.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2024-04-25T08:39:11.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.187174+0000', 'last_change': '2024-04-25T08:39:07.187174+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.187174+0000', 'last_undegraded': '2024-04-25T08:39:07.187174+0000', 'last_fullsized': '2024-04-25T08:39:07.187174+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:11.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2024-04-25T08:39:11.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.184184+0000', 'last_change': '2024-04-25T08:39:07.184184+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.184184+0000', 'last_undegraded': '2024-04-25T08:39:07.184184+0000', 'last_fullsized': '2024-04-25T08:39:07.184184+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:39:11.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2024-04-25T08:39:11.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185656+0000', 'last_change': '2024-04-25T08:39:07.185656+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185656+0000', 'last_undegraded': '2024-04-25T08:39:07.185656+0000', 'last_fullsized': '2024-04-25T08:39:07.185656+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:11.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2024-04-25T08:39:11.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185786+0000', 'last_change': '2024-04-25T08:39:07.185786+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185786+0000', 'last_undegraded': '2024-04-25T08:39:07.185786+0000', 'last_fullsized': '2024-04-25T08:39:07.185786+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:39:11.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2024-04-25T08:39:11.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.182412+0000', 'last_change': '2024-04-25T08:39:07.182412+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.182412+0000', 'last_undegraded': '2024-04-25T08:39:07.182412+0000', 'last_fullsized': '2024-04-25T08:39:07.182412+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:39:11.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2024-04-25T08:39:11.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.183713+0000', 'last_change': '2024-04-25T08:39:07.183713+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.183713+0000', 'last_undegraded': '2024-04-25T08:39:07.183713+0000', 'last_fullsized': '2024-04-25T08:39:07.183713+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:39:11.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2024-04-25T08:39:11.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.186076+0000', 'last_change': '2024-04-25T08:39:07.186076+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.186076+0000', 'last_undegraded': '2024-04-25T08:39:07.186076+0000', 'last_fullsized': '2024-04-25T08:39:07.186076+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:11.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2024-04-25T08:39:11.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.184647+0000', 'last_change': '2024-04-25T08:39:07.184647+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.184647+0000', 'last_undegraded': '2024-04-25T08:39:07.184647+0000', 'last_fullsized': '2024-04-25T08:39:07.184647+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:11.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2024-04-25T08:39:11.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.185152+0000', 'last_change': '2024-04-25T08:39:07.185152+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.185152+0000', 'last_undegraded': '2024-04-25T08:39:07.185152+0000', 'last_fullsized': '2024-04-25T08:39:07.185152+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:39:11.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2024-04-25T08:39:11.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.186576+0000', 'last_change': '2024-04-25T08:39:07.186576+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.186576+0000', 'last_undegraded': '2024-04-25T08:39:07.186576+0000', 'last_fullsized': '2024-04-25T08:39:07.186576+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:39:11.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2024-04-25T08:39:11.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.190873+0000', 'last_change': '2024-04-25T08:39:07.190873+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.190873+0000', 'last_undegraded': '2024-04-25T08:39:07.190873+0000', 'last_fullsized': '2024-04-25T08:39:07.190873+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:11.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2024-04-25T08:39:11.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 106, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:07.188049+0000', 'last_change': '2024-04-25T08:39:07.188049+0000', 'last_active': '2024-04-25T08:39:07.174694+0000', 'last_peered': '2024-04-25T08:39:07.174694+0000', 'last_clean': '2024-04-25T08:39:07.174694+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:07.188049+0000', 'last_undegraded': '2024-04-25T08:39:07.188049+0000', 'last_fullsized': '2024-04-25T08:39:07.188049+0000', 'mapping_epoch': 106, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 106, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:07.174694+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:39:11.913 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:39:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.045+0000 7f2a90930640 1 Processor -- start 2024-04-25T08:39:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.045+0000 7f2a90930640 1 -- start start 2024-04-25T08:39:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.046+0000 7f2a90930640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2a8c0b2d60 0x7f2a8c06c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.046+0000 7f2a90930640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c15e5c0 0x7f2a8c15ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.046+0000 7f2a90930640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a8c06b0a0 con 0x7f2a8c06abe0 2024-04-25T08:39:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.046+0000 7f2a90930640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8c06b380 con 0x7f2a8c0b2d60 2024-04-25T08:39:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.046+0000 7f2a90930640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c06b660 con 0x7f2a8c15e5c0 2024-04-25T08:39:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.046+0000 7f2a8a59c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2a8c0b2d60 0x7f2a8c06c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.046+0000 7f2a8ad9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c15e5c0 0x7f2a8c15ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.046+0000 7f2a89d9b640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f2a8c06abe0 0x7f2a8c0b3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:57780/0 (socket says 172.21.15.38:57780) 2024-04-25T08:39:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.047+0000 7f2a89d9b640 1 -- 172.21.15.38:0/1730972322 learned_addr learned my addr 172.21.15.38:0/1730972322 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.047+0000 7f2a8ad9d640 1 -- 172.21.15.38:0/1730972322 >> v2:172.21.15.107:3300/0 conn(0x7f2a8c0b2d60 msgr2=0x7f2a8c06c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.047+0000 7f2a8ad9d640 1 --2- 172.21.15.38:0/1730972322 >> v2:172.21.15.107:3300/0 conn(0x7f2a8c0b2d60 0x7f2a8c06c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.047+0000 7f2a8ad9d640 1 -- 172.21.15.38:0/1730972322 >> v1:172.21.15.107:6789/0 conn(0x7f2a8c06abe0 legacy=0x7f2a8c0b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.047+0000 7f2a8ad9d640 1 -- 172.21.15.38:0/1730972322 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c06b940 con 0x7f2a8c15e5c0 2024-04-25T08:39:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.047+0000 7f2a8959a640 1 -- 172.21.15.38:0/1730972322 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789739606 0 0) 0x7f2a8c06b0a0 con 0x7f2a8c06abe0 2024-04-25T08:39:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.047+0000 7f2a8a59c640 1 --2- 172.21.15.38:0/1730972322 >> v2:172.21.15.107:3300/0 conn(0x7f2a8c0b2d60 0x7f2a8c06c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:12.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.048+0000 7f2a8ad9d640 1 --2- 172.21.15.38:0/1730972322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c15e5c0 0x7f2a8c15ea30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0097a0 tx=0x7f2a7c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ddd7d487924bdee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.048+0000 7f2a8959a640 1 -- 172.21.15.38:0/1730972322 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c037070 con 0x7f2a8c15e5c0 2024-04-25T08:39:12.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.048+0000 7f2a8959a640 1 -- 172.21.15.38:0/1730972322 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c032440 con 0x7f2a8c15e5c0 2024-04-25T08:39:12.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.048+0000 7f2a8959a640 1 -- 172.21.15.38:0/1730972322 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c032810 con 0x7f2a8c15e5c0 2024-04-25T08:39:12.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.048+0000 7f2a90930640 1 -- 172.21.15.38:0/1730972322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c15e5c0 msgr2=0x7f2a8c15ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.048+0000 7f2a90930640 1 --2- 172.21.15.38:0/1730972322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c15e5c0 0x7f2a8c15ea30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0097a0 tx=0x7f2a7c02a180 comp rx=0 tx=0).stop 2024-04-25T08:39:12.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.049+0000 7f2a90930640 1 -- 172.21.15.38:0/1730972322 shutdown_connections 2024-04-25T08:39:12.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.049+0000 7f2a90930640 1 --2- 172.21.15.38:0/1730972322 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c15e5c0 0x7f2a8c15ea30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.049+0000 7f2a90930640 1 --2- 172.21.15.38:0/1730972322 >> v2:172.21.15.107:3300/0 conn(0x7f2a8c0b2d60 0x7f2a8c06c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.049+0000 7f2a90930640 1 -- 172.21.15.38:0/1730972322 >> 172.21.15.38:0/1730972322 conn(0x7f2a8c0cfd30 msgr2=0x7f2a8c0d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:12.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.049+0000 7f2a90930640 1 -- 172.21.15.38:0/1730972322 shutdown_connections 2024-04-25T08:39:12.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.049+0000 7f2a90930640 1 -- 172.21.15.38:0/1730972322 wait complete. 2024-04-25T08:39:12.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.050+0000 7f2a90930640 1 Processor -- start 2024-04-25T08:39:12.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.050+0000 7f2a90930640 1 -- start start 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a90930640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a8c06abe0 0x7f2a8c14cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a90930640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a8c0b2d60 0x7f2a8c0c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a90930640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c15e5c0 0x7f2a8c0c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a90930640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c15f1b0 con 0x7f2a8c06abe0 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a90930640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c15f030 con 0x7f2a8c0b2d60 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a90930640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c15f330 con 0x7f2a8c15e5c0 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a8a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a8c06abe0 0x7f2a8c14cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a8a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a8c06abe0 0x7f2a8c14cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45060/0 (socket says 172.21.15.38:45060) 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a8a59c640 1 -- 172.21.15.38:0/1163641663 learned_addr learned my addr 172.21.15.38:0/1163641663 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a8ad9d640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c15e5c0 0x7f2a8c0c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a89d9b640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a8c0b2d60 0x7f2a8c0c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a8a59c640 1 -- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c15e5c0 msgr2=0x7f2a8c0c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a8a59c640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c15e5c0 0x7f2a8c0c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a8a59c640 1 -- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a8c0b2d60 msgr2=0x7f2a8c0c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a8a59c640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a8c0b2d60 0x7f2a8c0c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a8a59c640 1 -- 172.21.15.38:0/1163641663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c0c4f70 con 0x7f2a8c06abe0 2024-04-25T08:39:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a89d9b640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a8c0b2d60 0x7f2a8c0c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a8a59c640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a8c06abe0 0x7f2a8c14cb80 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400efc0 tx=0x7f2a8400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a82ffd640 1 -- 172.21.15.38:0/1163641663 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84018020 con 0x7f2a8c06abe0 2024-04-25T08:39:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a82ffd640 1 -- 172.21.15.38:0/1163641663 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a84014680 con 0x7f2a8c06abe0 2024-04-25T08:39:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a90930640 1 -- 172.21.15.38:0/1163641663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c15cc30 con 0x7f2a8c06abe0 2024-04-25T08:39:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.051+0000 7f2a82ffd640 1 -- 172.21.15.38:0/1163641663 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84014a50 con 0x7f2a8c06abe0 2024-04-25T08:39:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.052+0000 7f2a90930640 1 -- 172.21.15.38:0/1163641663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c15d1f0 con 0x7f2a8c06abe0 2024-04-25T08:39:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.052+0000 7f2a90930640 1 -- 172.21.15.38:0/1163641663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8c068e80 con 0x7f2a8c06abe0 2024-04-25T08:39:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.053+0000 7f2a82ffd640 1 -- 172.21.15.38:0/1163641663 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a84005d10 con 0x7f2a8c06abe0 2024-04-25T08:39:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.053+0000 7f2a82ffd640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2a6c06b4d0 0x7f2a6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.053+0000 7f2a82ffd640 1 -- 172.21.15.38:0/1163641663 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2a8408b130 con 0x7f2a8c06abe0 2024-04-25T08:39:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.053+0000 7f2a89d9b640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2a6c06b4d0 0x7f2a6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.054+0000 7f2a89d9b640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2a6c06b4d0 0x7f2a6c06d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a74007fd0 tx=0x7f2a74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.058+0000 7f2a82ffd640 1 -- 172.21.15.38:0/1163641663 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a84022030 con 0x7f2a8c06abe0 2024-04-25T08:39:12.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.307+0000 7f2a90930640 1 -- 172.21.15.38:0/1163641663 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2a8c15dce0 con 0x7f2a8c06abe0 2024-04-25T08:39:12.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.308+0000 7f2a82ffd640 1 -- 172.21.15.38:0/1163641663 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1455 (secure 0 0 0) 0x7f2a84010040 con 0x7f2a8c06abe0 2024-04-25T08:39:12.301 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:12.301 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":481,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":108,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":477429760,"bytes_avail":772616683520,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":63,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8292}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T08:38:57.309082+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:39:12.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.310+0000 7f2a90930640 1 -- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2a6c06b4d0 msgr2=0x7f2a6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.310+0000 7f2a90930640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2a6c06b4d0 0x7f2a6c06d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a74007fd0 tx=0x7f2a74009040 comp rx=0 tx=0).stop 2024-04-25T08:39:12.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.310+0000 7f2a90930640 1 -- 172.21.15.38:0/1163641663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a8c06abe0 msgr2=0x7f2a8c14cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.310+0000 7f2a90930640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a8c06abe0 0x7f2a8c14cb80 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400efc0 tx=0x7f2a8400c490 comp rx=0 tx=0).stop 2024-04-25T08:39:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.310+0000 7f2a90930640 1 -- 172.21.15.38:0/1163641663 shutdown_connections 2024-04-25T08:39:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.310+0000 7f2a90930640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2a6c06b4d0 0x7f2a6c06d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.310+0000 7f2a90930640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2a8c15e5c0 0x7f2a8c0c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.310+0000 7f2a90930640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2a8c0b2d60 0x7f2a8c0c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.310+0000 7f2a90930640 1 --2- 172.21.15.38:0/1163641663 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2a8c06abe0 0x7f2a8c14cb80 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.310+0000 7f2a90930640 1 -- 172.21.15.38:0/1163641663 >> 172.21.15.38:0/1163641663 conn(0x7f2a8c0cfd30 msgr2=0x7f2a8c0d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.310+0000 7f2a90930640 1 -- 172.21.15.38:0/1163641663 shutdown_connections 2024-04-25T08:39:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.310+0000 7f2a90930640 1 -- 172.21.15.38:0/1163641663 wait complete. 2024-04-25T08:39:12.314 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:39:12.315 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:39:12.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.422+0000 7fa4fe8b3640 1 Processor -- start 2024-04-25T08:39:12.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.423+0000 7fa4fe8b3640 1 -- start start 2024-04-25T08:39:12.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.423+0000 7fa4fe8b3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa4f815dc10 0x7fa4f80ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.423+0000 7fa4fe8b3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa4f80ced60 0x7fa4f80d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.423+0000 7fa4fe8b3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f80d3e70 con 0x7fa4f80d3900 2024-04-25T08:39:12.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.423+0000 7fa4fe8b3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f815dff0 con 0x7fa4f80ced60 2024-04-25T08:39:12.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.423+0000 7fa4fe8b3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f80b6990 con 0x7fa4f815dc10 2024-04-25T08:39:12.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.423+0000 7fa4f77fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa4f80ced60 0x7fa4f80d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.424+0000 7fa4f77fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa4f80ced60 0x7fa4f80d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45062/0 (socket says 172.21.15.38:45062) 2024-04-25T08:39:12.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.424+0000 7fa4f77fe640 1 -- 172.21.15.38:0/3574736418 learned_addr learned my addr 172.21.15.38:0/3574736418 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:12.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.424+0000 7fa4f7fff640 1 --2- 172.21.15.38:0/3574736418 >> v2:172.21.15.107:3300/0 conn(0x7fa4f815dc10 0x7fa4f80ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.424+0000 7fa4f77fe640 1 -- 172.21.15.38:0/3574736418 >> v2:172.21.15.107:3300/0 conn(0x7fa4f815dc10 msgr2=0x7fa4f80ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:12.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.424+0000 7fa4f77fe640 1 --2- 172.21.15.38:0/3574736418 >> v2:172.21.15.107:3300/0 conn(0x7fa4f815dc10 0x7fa4f80ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.424+0000 7fa4f77fe640 1 -- 172.21.15.38:0/3574736418 >> v1:172.21.15.107:6789/0 conn(0x7fa4f80d3900 legacy=0x7fa4f80b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:12.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.424+0000 7fa4f77fe640 1 -- 172.21.15.38:0/3574736418 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f80d4150 con 0x7fa4f80ced60 2024-04-25T08:39:12.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.424+0000 7fa4f7fff640 1 --2- 172.21.15.38:0/3574736418 >> v2:172.21.15.107:3300/0 conn(0x7fa4f815dc10 0x7fa4f80ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:12.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.425+0000 7fa4f77fe640 1 --2- 172.21.15.38:0/3574736418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4f80ced60 0x7fa4f80d0850 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80b5d50 tx=0x7fa4ec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8034eff9261c1807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.425+0000 7fa4f6ffd640 1 -- 172.21.15.38:0/3574736418 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec037070 con 0x7fa4f80ced60 2024-04-25T08:39:12.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.425+0000 7fa4f6ffd640 1 -- 172.21.15.38:0/3574736418 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4ec032440 con 0x7fa4f80ced60 2024-04-25T08:39:12.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.425+0000 7fa4f6ffd640 1 -- 172.21.15.38:0/3574736418 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4ec032810 con 0x7fa4f80ced60 2024-04-25T08:39:12.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.425+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3574736418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4f80ced60 msgr2=0x7fa4f80d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.425+0000 7fa4fe8b3640 1 --2- 172.21.15.38:0/3574736418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4f80ced60 0x7fa4f80d0850 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fa4f80b5d50 tx=0x7fa4ec02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.425+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3574736418 shutdown_connections 2024-04-25T08:39:12.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.425+0000 7fa4fe8b3640 1 --2- 172.21.15.38:0/3574736418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4f80ced60 0x7fa4f80d0850 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.425+0000 7fa4fe8b3640 1 --2- 172.21.15.38:0/3574736418 >> v2:172.21.15.107:3300/0 conn(0x7fa4f815dc10 0x7fa4f80ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.425+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3574736418 >> 172.21.15.38:0/3574736418 conn(0x7fa4f8066d10 msgr2=0x7fa4f8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:12.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.425+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3574736418 shutdown_connections 2024-04-25T08:39:12.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.425+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3574736418 wait complete. 2024-04-25T08:39:12.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.426+0000 7fa4fe8b3640 1 Processor -- start 2024-04-25T08:39:12.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.427+0000 7fa4fe8b3640 1 -- start start 2024-04-25T08:39:12.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.427+0000 7fa4fe8b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4f80ced60 0x7fa4f81f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.427+0000 7fa4fe8b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4f80d3900 0x7fa4f80aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.427+0000 7fa4fe8b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4f815dc10 0x7fa4f81f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.427+0000 7fa4fe8b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f80b6bc0 con 0x7fa4f80ced60 2024-04-25T08:39:12.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.427+0000 7fa4fe8b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f80d4710 con 0x7fa4f815dc10 2024-04-25T08:39:12.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.427+0000 7fa4fe8b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f80d0d90 con 0x7fa4f80d3900 2024-04-25T08:39:12.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.427+0000 7fa4f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4f80ced60 0x7fa4f81f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4f7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4f80ced60 0x7fa4f81f6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45066/0 (socket says 172.21.15.38:45066) 2024-04-25T08:39:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4f7fff640 1 -- 172.21.15.38:0/3379173836 learned_addr learned my addr 172.21.15.38:0/3379173836 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4fce50640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4f815dc10 0x7fa4f81f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4f77fe640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4f80d3900 0x7fa4f80aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4f7fff640 1 -- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4f80d3900 msgr2=0x7fa4f80aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4f7fff640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4f80d3900 0x7fa4f80aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4f7fff640 1 -- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4f815dc10 msgr2=0x7fa4f81f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4f7fff640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4f815dc10 0x7fa4f81f39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4f7fff640 1 -- 172.21.15.38:0/3379173836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f81f6cc0 con 0x7fa4f80ced60 2024-04-25T08:39:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4f7fff640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4f80ced60 0x7fa4f81f6780 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800c7f0 tx=0x7fa4e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4f4ff9640 1 -- 172.21.15.38:0/3379173836 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8015070 con 0x7fa4f80ced60 2024-04-25T08:39:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4f4ff9640 1 -- 172.21.15.38:0/3379173836 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e8007820 con 0x7fa4f80ced60 2024-04-25T08:39:12.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3379173836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f80ab550 con 0x7fa4f80ced60 2024-04-25T08:39:12.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4f4ff9640 1 -- 172.21.15.38:0/3379173836 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e8007bd0 con 0x7fa4f80ced60 2024-04-25T08:39:12.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.428+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3379173836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f80abb10 con 0x7fa4f80ced60 2024-04-25T08:39:12.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.429+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3379173836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f8068e80 con 0x7fa4f80ced60 2024-04-25T08:39:12.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.430+0000 7fa4f4ff9640 1 -- 172.21.15.38:0/3379173836 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4e8018070 con 0x7fa4f80ced60 2024-04-25T08:39:12.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.430+0000 7fa4f4ff9640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa4e406b4d0 0x7fa4e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:12.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.430+0000 7fa4f4ff9640 1 -- 172.21.15.38:0/3379173836 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa4e808a4c0 con 0x7fa4f80ced60 2024-04-25T08:39:12.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.430+0000 7fa4f77fe640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa4e406b4d0 0x7fa4e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:12.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.431+0000 7fa4f77fe640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa4e406b4d0 0x7fa4e406d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec034000 tx=0x7fa4ec02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:12.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.434+0000 7fa4f4ff9640 1 -- 172.21.15.38:0/3379173836 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4e80591b0 con 0x7fa4f80ced60 2024-04-25T08:39:12.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.612+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3379173836 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4f80cda00 con 0x7fa4e406b4d0 2024-04-25T08:39:12.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.618+0000 7fa4f4ff9640 1 -- 172.21.15.38:0/3379173836 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247075 (secure 0 0 0) 0x7fa4ec07e540 con 0x7fa4e406b4d0 2024-04-25T08:39:12.614 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:12.615 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:39:12.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.621+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa4e406b4d0 msgr2=0x7fa4e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.621+0000 7fa4fe8b3640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa4e406b4d0 0x7fa4e406d990 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa4ec034000 tx=0x7fa4ec02e000 comp rx=0 tx=0).stop 2024-04-25T08:39:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.621+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3379173836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4f80ced60 msgr2=0x7fa4f81f6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.621+0000 7fa4fe8b3640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4f80ced60 0x7fa4f81f6780 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fa4e800c7f0 tx=0x7fa4e800ccb0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.621+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3379173836 shutdown_connections 2024-04-25T08:39:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.621+0000 7fa4fe8b3640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4f815dc10 0x7fa4f81f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.621+0000 7fa4fe8b3640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa4e406b4d0 0x7fa4e406d990 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.621+0000 7fa4fe8b3640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa4f80d3900 0x7fa4f80aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.621+0000 7fa4fe8b3640 1 --2- 172.21.15.38:0/3379173836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4f80ced60 0x7fa4f81f6780 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.621+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3379173836 >> 172.21.15.38:0/3379173836 conn(0x7fa4f8066d10 msgr2=0x7fa4f80d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.621+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3379173836 shutdown_connections 2024-04-25T08:39:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:12.621+0000 7fa4fe8b3640 1 -- 172.21.15.38:0/3379173836 wait complete. 2024-04-25T08:39:12.625 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-04-25T08:39:11.311618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":466240,"kb_used_data":11348,"kb_used_omap":12,"kb_used_meta":454771,"kb_avail":754508480,"statfs":{"total":773094113280,"available":772616683520,"internally_reserved":0,"allocated":11620352,"data_stored":6009995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":465686064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.136186"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185002+0000","last_change":"2024-04-25T08:39:07.185002+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185002+0000","last_undegraded":"2024-04-25T08:39:07.185002+0000","last_fullsized":"2024-04-25T08:39:07.185002+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182883+0000","last_change":"2024-04-25T08:39:07.182883+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182883+0000","last_undegraded":"2024-04-25T08:39:07.182883+0000","last_fullsized":"2024-04-25T08:39:07.182883+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186081+0000","last_change":"2024-04-25T08:39:07.186081+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186081+0000","last_undegraded":"2024-04-25T08:39:07.186081+0000","last_fullsized":"2024-04-25T08:39:07.186081+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183803+0000","last_change":"2024-04-25T08:39:07.183803+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183803+0000","last_undegraded":"2024-04-25T08:39:07.183803+0000","last_fullsized":"2024-04-25T08:39:07.183803+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.190401+0000","last_change":"2024-04-25T08:39:07.190401+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.190401+0000","last_undegraded":"2024-04-25T08:39:07.190401+0000","last_fullsized":"2024-04-25T08:39:07.190401+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185128+0000","last_change":"2024-04-25T08:39:07.185128+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185128+0000","last_undegraded":"2024-04-25T08:39:07.185128+0000","last_fullsized":"2024-04-25T08:39:07.185128+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182530+0000","last_change":"2024-04-25T08:39:07.182530+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182530+0000","last_undegraded":"2024-04-25T08:39:07.182530+0000","last_fullsized":"2024-04-25T08:39:07.182530+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185168+0000","last_change":"2024-04-25T08:39:07.185168+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185168+0000","last_undegraded":"2024-04-25T08:39:07.185168+0000","last_fullsized":"2024-04-25T08:39:07.185168+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181457+0000","last_change":"2024-04-25T08:39:07.181457+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181457+0000","last_undegraded":"2024-04-25T08:39:07.181457+0000","last_fullsized":"2024-04-25T08:39:07.181457+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187122+0000","last_change":"2024-04-25T08:39:07.187122+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187122+0000","last_undegraded":"2024-04-25T08:39:07.187122+0000","last_fullsized":"2024-04-25T08:39:07.187122+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187732+0000","last_change":"2024-04-25T08:39:07.187732+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187732+0000","last_undegraded":"2024-04-25T08:39:07.187732+0000","last_fullsized":"2024-04-25T08:39:07.187732+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187078+0000","last_change":"2024-04-25T08:39:07.187078+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187078+0000","last_undegraded":"2024-04-25T08:39:07.187078+0000","last_fullsized":"2024-04-25T08:39:07.187078+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182812+0000","last_change":"2024-04-25T08:39:07.182812+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182812+0000","last_undegraded":"2024-04-25T08:39:07.182812+0000","last_fullsized":"2024-04-25T08:39:07.182812+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185729+0000","last_change":"2024-04-25T08:39:07.185729+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185729+0000","last_undegraded":"2024-04-25T08:39:07.185729+0000","last_fullsized":"2024-04-25T08:39:07.185729+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185317+0000","last_change":"2024-04-25T08:39:07.185317+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185317+0000","last_undegraded":"2024-04-25T08:39:07.185317+0000","last_fullsized":"2024-04-25T08:39:07.185317+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185201+0000","last_change":"2024-04-25T08:39:07.185201+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185201+0000","last_undegraded":"2024-04-25T08:39:07.185201+0000","last_fullsized":"2024-04-25T08:39:07.185201+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184442+0000","last_change":"2024-04-25T08:39:07.184442+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184442+0000","last_undegraded":"2024-04-25T08:39:07.184442+0000","last_fullsized":"2024-04-25T08:39:07.184442+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182220+0000","last_change":"2024-04-25T08:39:07.182220+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182220+0000","last_undegraded":"2024-04-25T08:39:07.182220+0000","last_fullsized":"2024-04-25T08:39:07.182220+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187629+0000","last_change":"2024-04-25T08:39:07.187629+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187629+0000","last_undegraded":"2024-04-25T08:39:07.187629+0000","last_fullsized":"2024-04-25T08:39:07.187629+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183719+0000","last_change":"2024-04-25T08:39:07.183719+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183719+0000","last_undegraded":"2024-04-25T08:39:07.183719+0000","last_fullsized":"2024-04-25T08:39:07.183719+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.193271+0000","last_change":"2024-04-25T08:39:07.193271+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.193271+0000","last_undegraded":"2024-04-25T08:39:07.193271+0000","last_fullsized":"2024-04-25T08:39:07.193271+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.189389+0000","last_change":"2024-04-25T08:39:07.189389+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.189389+0000","last_undegraded":"2024-04-25T08:39:07.189389+0000","last_fullsized":"2024-04-25T08:39:07.189389+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183901+0000","last_change":"2024-04-25T08:39:07.183901+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183901+0000","last_undegraded":"2024-04-25T08:39:07.183901+0000","last_fullsized":"2024-04-25T08:39:07.183901+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188396+0000","last_change":"2024-04-25T08:39:07.188396+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188396+0000","last_undegraded":"2024-04-25T08:39:07.188396+0000","last_fullsized":"2024-04-25T08:39:07.188396+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183544+0000","last_change":"2024-04-25T08:39:07.183544+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183544+0000","last_undegraded":"2024-04-25T08:39:07.183544+0000","last_fullsized":"2024-04-25T08:39:07.183544+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183684+0000","last_change":"2024-04-25T08:39:07.183684+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183684+0000","last_undegraded":"2024-04-25T08:39:07.183684+0000","last_fullsized":"2024-04-25T08:39:07.183684+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187277+0000","last_change":"2024-04-25T08:39:07.187277+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187277+0000","last_undegraded":"2024-04-25T08:39:07.187277+0000","last_fullsized":"2024-04-25T08:39:07.187277+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181602+0000","last_change":"2024-04-25T08:39:07.181602+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181602+0000","last_undegraded":"2024-04-25T08:39:07.181602+0000","last_fullsized":"2024-04-25T08:39:07.181602+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.192641+0000","last_change":"2024-04-25T08:39:07.192641+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.192641+0000","last_undegraded":"2024-04-25T08:39:07.192641+0000","last_fullsized":"2024-04-25T08:39:07.192641+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184579+0000","last_change":"2024-04-25T08:39:07.184579+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184579+0000","last_undegraded":"2024-04-25T08:39:07.184579+0000","last_fullsized":"2024-04-25T08:39:07.184579+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.180795+0000","last_change":"2024-04-25T08:39:07.180795+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.180795+0000","last_undegraded":"2024-04-25T08:39:07.180795+0000","last_fullsized":"2024-04-25T08:39:07.180795+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186769+0000","last_change":"2024-04-25T08:39:07.186769+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186769+0000","last_undegraded":"2024-04-25T08:39:07.186769+0000","last_fullsized":"2024-04-25T08:39:07.186769+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184486+0000","last_change":"2024-04-25T08:39:07.184486+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184486+0000","last_undegraded":"2024-04-25T08:39:07.184486+0000","last_fullsized":"2024-04-25T08:39:07.184486+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192158+0000","last_change":"2024-04-25T08:39:07.190276+0000","last_active":"2024-04-25T08:39:07.192158+0000","last_peered":"2024-04-25T08:39:07.192158+0000","last_clean":"2024-04-25T08:39:07.192158+0000","last_became_active":"2024-04-25T08:39:07.189857+0000","last_became_peered":"2024-04-25T08:39:07.189857+0000","last_unstale":"2024-04-25T08:39:07.192158+0000","last_undegraded":"2024-04-25T08:39:07.192158+0000","last_fullsized":"2024-04-25T08:39:07.192158+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:03.259504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187736+0000","last_change":"2024-04-25T08:39:07.187736+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187736+0000","last_undegraded":"2024-04-25T08:39:07.187736+0000","last_fullsized":"2024-04-25T08:39:07.187736+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202189+0000","last_change":"2024-04-25T08:39:07.200692+0000","last_active":"2024-04-25T08:39:07.202189+0000","last_peered":"2024-04-25T08:39:07.202189+0000","last_clean":"2024-04-25T08:39:07.202189+0000","last_became_active":"2024-04-25T08:39:07.200405+0000","last_became_peered":"2024-04-25T08:39:07.200405+0000","last_unstale":"2024-04-25T08:39:07.202189+0000","last_undegraded":"2024-04-25T08:39:07.202189+0000","last_fullsized":"2024-04-25T08:39:07.202189+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.824524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184135+0000","last_change":"2024-04-25T08:39:07.184135+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184135+0000","last_undegraded":"2024-04-25T08:39:07.184135+0000","last_fullsized":"2024-04-25T08:39:07.184135+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.197048+0000","last_change":"2024-04-25T08:39:07.195456+0000","last_active":"2024-04-25T08:39:07.197048+0000","last_peered":"2024-04-25T08:39:07.197048+0000","last_clean":"2024-04-25T08:39:07.197048+0000","last_became_active":"2024-04-25T08:39:07.194725+0000","last_became_peered":"2024-04-25T08:39:07.194725+0000","last_unstale":"2024-04-25T08:39:07.197048+0000","last_undegraded":"2024-04-25T08:39:07.197048+0000","last_fullsized":"2024-04-25T08:39:07.197048+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:20.621755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185569+0000","last_change":"2024-04-25T08:39:07.185569+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185569+0000","last_undegraded":"2024-04-25T08:39:07.185569+0000","last_fullsized":"2024-04-25T08:39:07.185569+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199508+0000","last_change":"2024-04-25T08:39:07.197963+0000","last_active":"2024-04-25T08:39:07.199508+0000","last_peered":"2024-04-25T08:39:07.199508+0000","last_clean":"2024-04-25T08:39:07.199508+0000","last_became_active":"2024-04-25T08:39:07.197658+0000","last_became_peered":"2024-04-25T08:39:07.197658+0000","last_unstale":"2024-04-25T08:39:07.199508+0000","last_undegraded":"2024-04-25T08:39:07.199508+0000","last_fullsized":"2024-04-25T08:39:07.199508+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:39.219462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183293+0000","last_change":"2024-04-25T08:39:07.183293+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183293+0000","last_undegraded":"2024-04-25T08:39:07.183293+0000","last_fullsized":"2024-04-25T08:39:07.183293+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192072+0000","last_change":"2024-04-25T08:39:07.191029+0000","last_active":"2024-04-25T08:39:07.192072+0000","last_peered":"2024-04-25T08:39:07.192072+0000","last_clean":"2024-04-25T08:39:07.192072+0000","last_became_active":"2024-04-25T08:39:07.190722+0000","last_became_peered":"2024-04-25T08:39:07.190722+0000","last_unstale":"2024-04-25T08:39:07.192072+0000","last_undegraded":"2024-04-25T08:39:07.192072+0000","last_fullsized":"2024-04-25T08:39:07.192072+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:46.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186738+0000","last_change":"2024-04-25T08:39:07.186738+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186738+0000","last_undegraded":"2024-04-25T08:39:07.186738+0000","last_fullsized":"2024-04-25T08:39:07.186738+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199195+0000","last_change":"2024-04-25T08:39:07.198565+0000","last_active":"2024-04-25T08:39:07.199195+0000","last_peered":"2024-04-25T08:39:07.199195+0000","last_clean":"2024-04-25T08:39:07.199195+0000","last_became_active":"2024-04-25T08:39:07.197861+0000","last_became_peered":"2024-04-25T08:39:07.197861+0000","last_unstale":"2024-04-25T08:39:07.199195+0000","last_undegraded":"2024-04-25T08:39:07.199195+0000","last_fullsized":"2024-04-25T08:39:07.199195+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:54.429666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181730+0000","last_change":"2024-04-25T08:39:07.181730+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181730+0000","last_undegraded":"2024-04-25T08:39:07.181730+0000","last_fullsized":"2024-04-25T08:39:07.181730+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.195665+0000","last_change":"2024-04-25T08:39:07.191219+0000","last_active":"2024-04-25T08:39:07.195665+0000","last_peered":"2024-04-25T08:39:07.195665+0000","last_clean":"2024-04-25T08:39:07.195665+0000","last_became_active":"2024-04-25T08:39:07.190902+0000","last_became_peered":"2024-04-25T08:39:07.190902+0000","last_unstale":"2024-04-25T08:39:07.195665+0000","last_undegraded":"2024-04-25T08:39:07.195665+0000","last_fullsized":"2024-04-25T08:39:07.195665+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.078468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182971+0000","last_change":"2024-04-25T08:39:07.182971+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182971+0000","last_undegraded":"2024-04-25T08:39:07.182971+0000","last_fullsized":"2024-04-25T08:39:07.182971+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192534+0000","last_change":"2024-04-25T08:39:07.191760+0000","last_active":"2024-04-25T08:39:07.192534+0000","last_peered":"2024-04-25T08:39:07.192534+0000","last_clean":"2024-04-25T08:39:07.192534+0000","last_became_active":"2024-04-25T08:39:07.191248+0000","last_became_peered":"2024-04-25T08:39:07.191248+0000","last_unstale":"2024-04-25T08:39:07.192534+0000","last_undegraded":"2024-04-25T08:39:07.192534+0000","last_fullsized":"2024-04-25T08:39:07.192534+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:16.487240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185073+0000","last_change":"2024-04-25T08:39:07.185073+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185073+0000","last_undegraded":"2024-04-25T08:39:07.185073+0000","last_fullsized":"2024-04-25T08:39:07.185073+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199328+0000","last_change":"2024-04-25T08:39:07.198061+0000","last_active":"2024-04-25T08:39:07.199328+0000","last_peered":"2024-04-25T08:39:07.199328+0000","last_clean":"2024-04-25T08:39:07.199328+0000","last_became_active":"2024-04-25T08:39:07.197146+0000","last_became_peered":"2024-04-25T08:39:07.197146+0000","last_unstale":"2024-04-25T08:39:07.199328+0000","last_undegraded":"2024-04-25T08:39:07.199328+0000","last_fullsized":"2024-04-25T08:39:07.199328+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:39.219462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188107+0000","last_change":"2024-04-25T08:39:07.188107+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188107+0000","last_undegraded":"2024-04-25T08:39:07.188107+0000","last_fullsized":"2024-04-25T08:39:07.188107+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.197020+0000","last_change":"2024-04-25T08:39:07.195425+0000","last_active":"2024-04-25T08:39:07.197020+0000","last_peered":"2024-04-25T08:39:07.197020+0000","last_clean":"2024-04-25T08:39:07.197020+0000","last_became_active":"2024-04-25T08:39:07.195087+0000","last_became_peered":"2024-04-25T08:39:07.195087+0000","last_unstale":"2024-04-25T08:39:07.197020+0000","last_undegraded":"2024-04-25T08:39:07.197020+0000","last_fullsized":"2024-04-25T08:39:07.197020+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:52.006665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187086+0000","last_change":"2024-04-25T08:39:07.187086+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187086+0000","last_undegraded":"2024-04-25T08:39:07.187086+0000","last_fullsized":"2024-04-25T08:39:07.187086+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.192581+0000","last_change":"2024-04-25T08:39:07.192065+0000","last_active":"2024-04-25T08:39:07.192581+0000","last_peered":"2024-04-25T08:39:07.192581+0000","last_clean":"2024-04-25T08:39:07.192581+0000","last_became_active":"2024-04-25T08:39:07.191757+0000","last_became_peered":"2024-04-25T08:39:07.191757+0000","last_unstale":"2024-04-25T08:39:07.192581+0000","last_undegraded":"2024-04-25T08:39:07.192581+0000","last_fullsized":"2024-04-25T08:39:07.192581+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:32.537457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.189381+0000","last_change":"2024-04-25T08:39:07.189381+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.189381+0000","last_undegraded":"2024-04-25T08:39:07.189381+0000","last_fullsized":"2024-04-25T08:39:07.189381+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198326+0000","last_change":"2024-04-25T08:39:07.196396+0000","last_active":"2024-04-25T08:39:07.198326+0000","last_peered":"2024-04-25T08:39:07.198326+0000","last_clean":"2024-04-25T08:39:07.198326+0000","last_became_active":"2024-04-25T08:39:07.195109+0000","last_became_peered":"2024-04-25T08:39:07.195109+0000","last_unstale":"2024-04-25T08:39:07.198326+0000","last_undegraded":"2024-04-25T08:39:07.198326+0000","last_fullsized":"2024-04-25T08:39:07.198326+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.078468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186932+0000","last_change":"2024-04-25T08:39:07.186932+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186932+0000","last_undegraded":"2024-04-25T08:39:07.186932+0000","last_fullsized":"2024-04-25T08:39:07.186932+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198619+0000","last_change":"2024-04-25T08:39:07.197589+0000","last_active":"2024-04-25T08:39:07.198619+0000","last_peered":"2024-04-25T08:39:07.198619+0000","last_clean":"2024-04-25T08:39:07.198619+0000","last_became_active":"2024-04-25T08:39:07.197301+0000","last_became_peered":"2024-04-25T08:39:07.197301+0000","last_unstale":"2024-04-25T08:39:07.198619+0000","last_undegraded":"2024-04-25T08:39:07.198619+0000","last_fullsized":"2024-04-25T08:39:07.198619+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:20.621755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181101+0000","last_change":"2024-04-25T08:39:07.181101+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181101+0000","last_undegraded":"2024-04-25T08:39:07.181101+0000","last_fullsized":"2024-04-25T08:39:07.181101+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.189989+0000","last_change":"2024-04-25T08:39:07.189044+0000","last_active":"2024-04-25T08:39:07.189989+0000","last_peered":"2024-04-25T08:39:07.189989+0000","last_clean":"2024-04-25T08:39:07.189989+0000","last_became_active":"2024-04-25T08:39:07.188847+0000","last_became_peered":"2024-04-25T08:39:07.188847+0000","last_unstale":"2024-04-25T08:39:07.189989+0000","last_undegraded":"2024-04-25T08:39:07.189989+0000","last_fullsized":"2024-04-25T08:39:07.189989+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:19.824202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188221+0000","last_change":"2024-04-25T08:39:07.188221+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188221+0000","last_undegraded":"2024-04-25T08:39:07.188221+0000","last_fullsized":"2024-04-25T08:39:07.188221+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.200155+0000","last_change":"2024-04-25T08:39:07.198202+0000","last_active":"2024-04-25T08:39:07.200155+0000","last_peered":"2024-04-25T08:39:07.200155+0000","last_clean":"2024-04-25T08:39:07.200155+0000","last_became_active":"2024-04-25T08:39:07.197778+0000","last_became_peered":"2024-04-25T08:39:07.197778+0000","last_unstale":"2024-04-25T08:39:07.200155+0000","last_undegraded":"2024-04-25T08:39:07.200155+0000","last_fullsized":"2024-04-25T08:39:07.200155+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:03.697252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183954+0000","last_change":"2024-04-25T08:39:07.183954+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183954+0000","last_undegraded":"2024-04-25T08:39:07.183954+0000","last_fullsized":"2024-04-25T08:39:07.183954+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.195707+0000","last_change":"2024-04-25T08:39:07.191103+0000","last_active":"2024-04-25T08:39:07.195707+0000","last_peered":"2024-04-25T08:39:07.195707+0000","last_clean":"2024-04-25T08:39:07.195707+0000","last_became_active":"2024-04-25T08:39:07.190789+0000","last_became_peered":"2024-04-25T08:39:07.190789+0000","last_unstale":"2024-04-25T08:39:07.195707+0000","last_undegraded":"2024-04-25T08:39:07.195707+0000","last_fullsized":"2024-04-25T08:39:07.195707+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:30.160350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185617+0000","last_change":"2024-04-25T08:39:07.185617+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185617+0000","last_undegraded":"2024-04-25T08:39:07.185617+0000","last_fullsized":"2024-04-25T08:39:07.185617+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.196625+0000","last_change":"2024-04-25T08:39:07.194224+0000","last_active":"2024-04-25T08:39:07.196625+0000","last_peered":"2024-04-25T08:39:07.196625+0000","last_clean":"2024-04-25T08:39:07.196625+0000","last_became_active":"2024-04-25T08:39:07.193936+0000","last_became_peered":"2024-04-25T08:39:07.193936+0000","last_unstale":"2024-04-25T08:39:07.196625+0000","last_undegraded":"2024-04-25T08:39:07.196625+0000","last_fullsized":"2024-04-25T08:39:07.196625+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:30.160350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.181477+0000","last_change":"2024-04-25T08:39:07.181477+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.181477+0000","last_undegraded":"2024-04-25T08:39:07.181477+0000","last_fullsized":"2024-04-25T08:39:07.181477+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.199346+0000","last_change":"2024-04-25T08:39:07.197977+0000","last_active":"2024-04-25T08:39:07.199346+0000","last_peered":"2024-04-25T08:39:07.199346+0000","last_clean":"2024-04-25T08:39:07.199346+0000","last_became_active":"2024-04-25T08:39:07.197678+0000","last_became_peered":"2024-04-25T08:39:07.197678+0000","last_unstale":"2024-04-25T08:39:07.199346+0000","last_undegraded":"2024-04-25T08:39:07.199346+0000","last_fullsized":"2024-04-25T08:39:07.199346+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:22.700613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.189910+0000","last_change":"2024-04-25T08:39:07.189910+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.189910+0000","last_undegraded":"2024-04-25T08:39:07.189910+0000","last_fullsized":"2024-04-25T08:39:07.189910+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198861+0000","last_change":"2024-04-25T08:39:07.197738+0000","last_active":"2024-04-25T08:39:07.198861+0000","last_peered":"2024-04-25T08:39:07.198861+0000","last_clean":"2024-04-25T08:39:07.198861+0000","last_became_active":"2024-04-25T08:39:07.197436+0000","last_became_peered":"2024-04-25T08:39:07.197436+0000","last_unstale":"2024-04-25T08:39:07.198861+0000","last_undegraded":"2024-04-25T08:39:07.198861+0000","last_fullsized":"2024-04-25T08:39:07.198861+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:39.317231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185375+0000","last_change":"2024-04-25T08:39:07.185375+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185375+0000","last_undegraded":"2024-04-25T08:39:07.185375+0000","last_fullsized":"2024-04-25T08:39:07.185375+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202185+0000","last_change":"2024-04-25T08:39:07.200590+0000","last_active":"2024-04-25T08:39:07.202185+0000","last_peered":"2024-04-25T08:39:07.202185+0000","last_clean":"2024-04-25T08:39:07.202185+0000","last_became_active":"2024-04-25T08:39:07.200312+0000","last_became_peered":"2024-04-25T08:39:07.200312+0000","last_unstale":"2024-04-25T08:39:07.202185+0000","last_undegraded":"2024-04-25T08:39:07.202185+0000","last_fullsized":"2024-04-25T08:39:07.202185+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:14.807183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.187174+0000","last_change":"2024-04-25T08:39:07.187174+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.187174+0000","last_undegraded":"2024-04-25T08:39:07.187174+0000","last_fullsized":"2024-04-25T08:39:07.187174+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202389+0000","last_change":"2024-04-25T08:39:07.201896+0000","last_active":"2024-04-25T08:39:07.202389+0000","last_peered":"2024-04-25T08:39:07.202389+0000","last_clean":"2024-04-25T08:39:07.202389+0000","last_became_active":"2024-04-25T08:39:07.201576+0000","last_became_peered":"2024-04-25T08:39:07.201576+0000","last_unstale":"2024-04-25T08:39:07.202389+0000","last_undegraded":"2024-04-25T08:39:07.202389+0000","last_fullsized":"2024-04-25T08:39:07.202389+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:48:42.629045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184184+0000","last_change":"2024-04-25T08:39:07.184184+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184184+0000","last_undegraded":"2024-04-25T08:39:07.184184+0000","last_fullsized":"2024-04-25T08:39:07.184184+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.195198+0000","last_change":"2024-04-25T08:39:07.193296+0000","last_active":"2024-04-25T08:39:07.195198+0000","last_peered":"2024-04-25T08:39:07.195198+0000","last_clean":"2024-04-25T08:39:07.195198+0000","last_became_active":"2024-04-25T08:39:07.193017+0000","last_became_peered":"2024-04-25T08:39:07.193017+0000","last_unstale":"2024-04-25T08:39:07.195198+0000","last_undegraded":"2024-04-25T08:39:07.195198+0000","last_fullsized":"2024-04-25T08:39:07.195198+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:51.280521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185656+0000","last_change":"2024-04-25T08:39:07.185656+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185656+0000","last_undegraded":"2024-04-25T08:39:07.185656+0000","last_fullsized":"2024-04-25T08:39:07.185656+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202406+0000","last_change":"2024-04-25T08:39:07.201924+0000","last_active":"2024-04-25T08:39:07.202406+0000","last_peered":"2024-04-25T08:39:07.202406+0000","last_clean":"2024-04-25T08:39:07.202406+0000","last_became_active":"2024-04-25T08:39:07.201634+0000","last_became_peered":"2024-04-25T08:39:07.201634+0000","last_unstale":"2024-04-25T08:39:07.202406+0000","last_undegraded":"2024-04-25T08:39:07.202406+0000","last_fullsized":"2024-04-25T08:39:07.202406+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:13.867447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185786+0000","last_change":"2024-04-25T08:39:07.185786+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185786+0000","last_undegraded":"2024-04-25T08:39:07.185786+0000","last_fullsized":"2024-04-25T08:39:07.185786+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.196826+0000","last_change":"2024-04-25T08:39:07.196017+0000","last_active":"2024-04-25T08:39:07.196826+0000","last_peered":"2024-04-25T08:39:07.196826+0000","last_clean":"2024-04-25T08:39:07.196826+0000","last_became_active":"2024-04-25T08:39:07.195702+0000","last_became_peered":"2024-04-25T08:39:07.195702+0000","last_unstale":"2024-04-25T08:39:07.196826+0000","last_undegraded":"2024-04-25T08:39:07.196826+0000","last_fullsized":"2024-04-25T08:39:07.196826+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:24.078468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.182412+0000","last_change":"2024-04-25T08:39:07.182412+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.182412+0000","last_undegraded":"2024-04-25T08:39:07.182412+0000","last_fullsized":"2024-04-25T08:39:07.182412+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.200126+0000","last_change":"2024-04-25T08:39:07.198773+0000","last_active":"2024-04-25T08:39:07.200126+0000","last_peered":"2024-04-25T08:39:07.200126+0000","last_clean":"2024-04-25T08:39:07.200126+0000","last_became_active":"2024-04-25T08:39:07.197463+0000","last_became_peered":"2024-04-25T08:39:07.197463+0000","last_unstale":"2024-04-25T08:39:07.200126+0000","last_undegraded":"2024-04-25T08:39:07.200126+0000","last_fullsized":"2024-04-25T08:39:07.200126+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:12.088493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.183713+0000","last_change":"2024-04-25T08:39:07.183713+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.183713+0000","last_undegraded":"2024-04-25T08:39:07.183713+0000","last_fullsized":"2024-04-25T08:39:07.183713+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.190171+0000","last_change":"2024-04-25T08:39:07.188334+0000","last_active":"2024-04-25T08:39:07.190171+0000","last_peered":"2024-04-25T08:39:07.190171+0000","last_clean":"2024-04-25T08:39:07.190171+0000","last_became_active":"2024-04-25T08:39:07.187795+0000","last_became_peered":"2024-04-25T08:39:07.187795+0000","last_unstale":"2024-04-25T08:39:07.190171+0000","last_undegraded":"2024-04-25T08:39:07.190171+0000","last_fullsized":"2024-04-25T08:39:07.190171+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:57.413804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186076+0000","last_change":"2024-04-25T08:39:07.186076+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186076+0000","last_undegraded":"2024-04-25T08:39:07.186076+0000","last_fullsized":"2024-04-25T08:39:07.186076+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.198255+0000","last_change":"2024-04-25T08:39:07.197165+0000","last_active":"2024-04-25T08:39:07.198255+0000","last_peered":"2024-04-25T08:39:07.198255+0000","last_clean":"2024-04-25T08:39:07.198255+0000","last_became_active":"2024-04-25T08:39:07.196785+0000","last_became_peered":"2024-04-25T08:39:07.196785+0000","last_unstale":"2024-04-25T08:39:07.198255+0000","last_undegraded":"2024-04-25T08:39:07.198255+0000","last_fullsized":"2024-04-25T08:39:07.198255+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:27.118962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.184647+0000","last_change":"2024-04-25T08:39:07.184647+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.184647+0000","last_undegraded":"2024-04-25T08:39:07.184647+0000","last_fullsized":"2024-04-25T08:39:07.184647+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.196795+0000","last_change":"2024-04-25T08:39:07.193944+0000","last_active":"2024-04-25T08:39:07.196795+0000","last_peered":"2024-04-25T08:39:07.196795+0000","last_clean":"2024-04-25T08:39:07.196795+0000","last_became_active":"2024-04-25T08:39:07.193625+0000","last_became_peered":"2024-04-25T08:39:07.193625+0000","last_unstale":"2024-04-25T08:39:07.196795+0000","last_undegraded":"2024-04-25T08:39:07.196795+0000","last_fullsized":"2024-04-25T08:39:07.196795+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:27.118962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.185152+0000","last_change":"2024-04-25T08:39:07.185152+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.185152+0000","last_undegraded":"2024-04-25T08:39:07.185152+0000","last_fullsized":"2024-04-25T08:39:07.185152+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.200071+0000","last_change":"2024-04-25T08:39:07.198072+0000","last_active":"2024-04-25T08:39:07.200071+0000","last_peered":"2024-04-25T08:39:07.200071+0000","last_clean":"2024-04-25T08:39:07.200071+0000","last_became_active":"2024-04-25T08:39:07.197770+0000","last_became_peered":"2024-04-25T08:39:07.197770+0000","last_unstale":"2024-04-25T08:39:07.200071+0000","last_undegraded":"2024-04-25T08:39:07.200071+0000","last_fullsized":"2024-04-25T08:39:07.200071+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:37.421862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.186576+0000","last_change":"2024-04-25T08:39:07.186576+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.186576+0000","last_undegraded":"2024-04-25T08:39:07.186576+0000","last_fullsized":"2024-04-25T08:39:07.186576+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.190198+0000","last_change":"2024-04-25T08:39:07.188807+0000","last_active":"2024-04-25T08:39:07.190198+0000","last_peered":"2024-04-25T08:39:07.190198+0000","last_clean":"2024-04-25T08:39:07.190198+0000","last_became_active":"2024-04-25T08:39:07.188447+0000","last_became_peered":"2024-04-25T08:39:07.188447+0000","last_unstale":"2024-04-25T08:39:07.190198+0000","last_undegraded":"2024-04-25T08:39:07.190198+0000","last_fullsized":"2024-04-25T08:39:07.190198+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:22.020416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.190873+0000","last_change":"2024-04-25T08:39:07.190873+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.190873+0000","last_undegraded":"2024-04-25T08:39:07.190873+0000","last_fullsized":"2024-04-25T08:39:07.190873+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.197208+0000","last_change":"2024-04-25T08:39:07.196369+0000","last_active":"2024-04-25T08:39:07.197208+0000","last_peered":"2024-04-25T08:39:07.197208+0000","last_clean":"2024-04-25T08:39:07.197208+0000","last_became_active":"2024-04-25T08:39:07.196073+0000","last_became_peered":"2024-04-25T08:39:07.196073+0000","last_unstale":"2024-04-25T08:39:07.197208+0000","last_undegraded":"2024-04-25T08:39:07.197208+0000","last_fullsized":"2024-04-25T08:39:07.197208+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:55.439962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":106,"state":"creating+peering","last_fresh":"2024-04-25T08:39:07.188049+0000","last_change":"2024-04-25T08:39:07.188049+0000","last_active":"2024-04-25T08:39:07.174694+0000","last_peered":"2024-04-25T08:39:07.174694+0000","last_clean":"2024-04-25T08:39:07.174694+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:07.188049+0000","last_undegraded":"2024-04-25T08:39:07.188049+0000","last_fullsized":"2024-04-25T08:39:07.188049+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.202209+0000","last_change":"2024-04-25T08:39:07.200791+0000","last_active":"2024-04-25T08:39:07.202209+0000","last_peered":"2024-04-25T08:39:07.202209+0000","last_clean":"2024-04-25T08:39:07.202209+0000","last_became_active":"2024-04-25T08:39:07.200495+0000","last_became_peered":"2024-04-25T08:39:07.200495+0000","last_unstale":"2024-04-25T08:39:07.202209+0000","last_undegraded":"2024-04-25T08:39:07.202209+0000","last_fullsized":"2024-04-25T08:39:07.202209+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:59.362497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":332,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T08:39:07.186152+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:39:07.186152+0000","last_peered":"2024-04-25T08:39:07.186152+0000","last_clean":"2024-04-25T08:39:07.186152+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:39:07.186152+0000","last_undegraded":"2024-04-25T08:39:07.186152+0000","last_fullsized":"2024-04-25T08:39:07.186152+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59400,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312440,"statfs":{"total":96636764160,"available":96575938560,"internally_reserved":0,"allocated":1896448,"data_stored":1193192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54660,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317180,"statfs":{"total":96636764160,"available":96580792320,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54664,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317176,"statfs":{"total":96636764160,"available":96580788224,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68548,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303292,"statfs":{"total":96636764160,"available":96566571008,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574946,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68548,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303292,"statfs":{"total":96636764160,"available":96566571008,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58824,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313016,"statfs":{"total":96636764160,"available":96576528384,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54660,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317180,"statfs":{"total":96636764160,"available":96580792320,"internally_reserved":0,"allocated":1302528,"data_stored":602824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46936,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324904,"statfs":{"total":96636764160,"available":96588701696,"internally_reserved":0,"allocated":1908736,"data_stored":1199859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:39:13.465 INFO:tasks.ceph.mds.d.smithi107.stderr:2024-04-25T08:39:13.471+0000 7f5de6dfe640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:39:15.629 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:39:15.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.750+0000 7f41a8161640 1 Processor -- start 2024-04-25T08:39:15.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.750+0000 7f41a8161640 1 -- start start 2024-04-25T08:39:15.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.751+0000 7f41a8161640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f41a00b4b50 0x7f41a00ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.751+0000 7f41a8161640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a00b50c0 con 0x7f41a0144b70 2024-04-25T08:39:15.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.751+0000 7f41a8161640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a00b53a0 con 0x7f41a0144760 2024-04-25T08:39:15.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.751+0000 7f41a8161640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a015f380 con 0x7f41a00b4b50 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.751+0000 7f41a66fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f41a0144b70 0x7f41a00b3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43604/0 (socket says 172.21.15.38:43604) 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.751+0000 7f41a66fe640 1 -- 172.21.15.38:0/2391060369 learned_addr learned my addr 172.21.15.38:0/2391060369 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.751+0000 7f41a56fc640 1 --2- 172.21.15.38:0/2391060369 >> v2:172.21.15.107:3300/0 conn(0x7f41a00b4b50 0x7f41a00ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.751+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233252102 0 0) 0x7f41a00b50c0 con 0x7f41a0144b70 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.751+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f418c003520 con 0x7f41a0144b70 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2286226372 0 0) 0x7f418c003520 con 0x7f41a0144b70 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 >> v2:172.21.15.107:3300/0 conn(0x7f41a00b4b50 msgr2=0x7f41a00ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 --2- 172.21.15.38:0/2391060369 >> v2:172.21.15.107:3300/0 conn(0x7f41a00b4b50 0x7f41a00ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 >> v1:172.21.15.107:6789/0 conn(0x7f41a0144760 legacy=0x7f41a015faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a00b5680 con 0x7f41a0144b70 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f41980019f0 con 0x7f41a0144b70 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 >> v1:172.21.15.38:6789/0 conn(0x7f41a0144b70 legacy=0x7f41a00b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 --2- 172.21.15.38:0/2391060369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f418c0039b0 0x7f418c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 --2- 172.21.15.38:0/2391060369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f418c004800 0x7f418c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 --2- 172.21.15.38:0/2391060369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f418c0051c0 0x7f418c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a015f380 con 0x7f418c0039b0 2024-04-25T08:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f418c00c160 con 0x7f418c004800 2024-04-25T08:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f418c00c360 con 0x7f418c0051c0 2024-04-25T08:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a5efd640 1 --2- 172.21.15.38:0/2391060369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f418c0039b0 0x7f418c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a66fe640 1 --2- 172.21.15.38:0/2391060369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f418c0051c0 0x7f418c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a56fc640 1 --2- 172.21.15.38:0/2391060369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f418c004800 0x7f418c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a5efd640 1 -- 172.21.15.38:0/2391060369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f418c0051c0 msgr2=0x7f418c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a5efd640 1 --2- 172.21.15.38:0/2391060369 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f418c0051c0 0x7f418c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a5efd640 1 -- 172.21.15.38:0/2391060369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f418c004800 msgr2=0x7f418c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:15.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a5efd640 1 --2- 172.21.15.38:0/2391060369 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f418c004800 0x7f418c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a66fe640 1 -- 172.21.15.38:0/2391060369 reap_dead start 2024-04-25T08:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.752+0000 7f41a5efd640 1 -- 172.21.15.38:0/2391060369 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c003520 con 0x7f418c0039b0 2024-04-25T08:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.753+0000 7f41a5efd640 1 --2- 172.21.15.38:0/2391060369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f418c0039b0 0x7f418c003dd0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f419c001750 tx=0x7f418c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b436bf2c1bb9f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.753+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c001370 con 0x7f418c0039b0 2024-04-25T08:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.753+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c0052d0 con 0x7f418c0039b0 2024-04-25T08:39:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.753+0000 7f41a4efb640 1 -- 172.21.15.38:0/2391060369 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c02abc0 con 0x7f418c0039b0 2024-04-25T08:39:15.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.754+0000 7f41a8161640 1 -- 172.21.15.38:0/2391060369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f418c0039b0 msgr2=0x7f418c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.754+0000 7f41a8161640 1 --2- 172.21.15.38:0/2391060369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f418c0039b0 0x7f418c003dd0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f419c001750 tx=0x7f418c005830 comp rx=0 tx=0).stop 2024-04-25T08:39:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.754+0000 7f41a8161640 1 -- 172.21.15.38:0/2391060369 shutdown_connections 2024-04-25T08:39:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.754+0000 7f41a8161640 1 --2- 172.21.15.38:0/2391060369 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f418c0039b0 0x7f418c003dd0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.754+0000 7f41a8161640 1 -- 172.21.15.38:0/2391060369 >> 172.21.15.38:0/2391060369 conn(0x7f41a006b210 msgr2=0x7f41a006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.754+0000 7f41a8161640 1 -- 172.21.15.38:0/2391060369 shutdown_connections 2024-04-25T08:39:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.754+0000 7f41a8161640 1 -- 172.21.15.38:0/2391060369 wait complete. 2024-04-25T08:39:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.755+0000 7f41a8161640 1 Processor -- start 2024-04-25T08:39:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.755+0000 7f41a8161640 1 -- start start 2024-04-25T08:39:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a8161640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f418c0039b0 0x7f41a01f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a8161640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a00cc150 0x7f41a0138740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a8161640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41a01355b0 0x7f41a0135a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a8161640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a015faf0 con 0x7f41a00cc150 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a8161640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a015f660 con 0x7f41a01355b0 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a8161640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a015fc70 con 0x7f418c0039b0 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a56fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a00cc150 0x7f41a0138740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a56fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a00cc150 0x7f41a0138740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45084/0 (socket says 172.21.15.38:45084) 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a56fc640 1 -- 172.21.15.38:0/2040038205 learned_addr learned my addr 172.21.15.38:0/2040038205 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a66fe640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41a01355b0 0x7f41a0135a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a5efd640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f418c0039b0 0x7f41a01f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a56fc640 1 -- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f418c0039b0 msgr2=0x7f41a01f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a56fc640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f418c0039b0 0x7f41a01f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a56fc640 1 -- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41a01355b0 msgr2=0x7f41a0135a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a56fc640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41a01355b0 0x7f41a0135a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.756+0000 7f41a56fc640 1 -- 172.21.15.38:0/2040038205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a01f6c90 con 0x7f41a00cc150 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.757+0000 7f41a56fc640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a00cc150 0x7f41a0138740 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f4190011960 tx=0x7f4190011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:15.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.757+0000 7f41967fc640 1 -- 172.21.15.38:0/2040038205 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419000fa30 con 0x7f41a00cc150 2024-04-25T08:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.757+0000 7f41967fc640 1 -- 172.21.15.38:0/2040038205 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419000fbd0 con 0x7f41a00cc150 2024-04-25T08:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.757+0000 7f41967fc640 1 -- 172.21.15.38:0/2040038205 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4190014440 con 0x7f41a00cc150 2024-04-25T08:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.757+0000 7f41a8161640 1 -- 172.21.15.38:0/2040038205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a01f6f70 con 0x7f41a00cc150 2024-04-25T08:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.757+0000 7f41a8161640 1 -- 172.21.15.38:0/2040038205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a01f7530 con 0x7f41a00cc150 2024-04-25T08:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.758+0000 7f41a8161640 1 -- 172.21.15.38:0/2040038205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a0068e80 con 0x7f41a00cc150 2024-04-25T08:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.758+0000 7f41967fc640 1 -- 172.21.15.38:0/2040038205 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4190012040 con 0x7f41a00cc150 2024-04-25T08:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.759+0000 7f41967fc640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f418406b4d0 0x7f418406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.759+0000 7f41967fc640 1 -- 172.21.15.38:0/2040038205 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f419008a890 con 0x7f41a00cc150 2024-04-25T08:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.759+0000 7f41a5efd640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f418406b4d0 0x7f418406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:15.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.760+0000 7f41a5efd640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f418406b4d0 0x7f418406d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f419c032000 tx=0x7f419c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:15.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.763+0000 7f41967fc640 1 -- 172.21.15.38:0/2040038205 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4190019030 con 0x7f41a00cc150 2024-04-25T08:39:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.944+0000 7f41a8161640 1 -- 172.21.15.38:0/2040038205 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41a006df60 con 0x7f418406b4d0 2024-04-25T08:39:15.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.950+0000 7f41967fc640 1 -- 172.21.15.38:0/2040038205 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7f419c07fa60 con 0x7f418406b4d0 2024-04-25T08:39:15.947 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:39:15.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.953+0000 7f41a8161640 1 -- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f418406b4d0 msgr2=0x7f418406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.953+0000 7f41a8161640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f418406b4d0 0x7f418406d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f419c032000 tx=0x7f419c02e000 comp rx=0 tx=0).stop 2024-04-25T08:39:15.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.953+0000 7f41a8161640 1 -- 172.21.15.38:0/2040038205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a00cc150 msgr2=0x7f41a0138740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:15.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.953+0000 7f41a8161640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a00cc150 0x7f41a0138740 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f4190011960 tx=0x7f4190011d60 comp rx=0 tx=0).stop 2024-04-25T08:39:15.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.953+0000 7f41a8161640 1 -- 172.21.15.38:0/2040038205 shutdown_connections 2024-04-25T08:39:15.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.953+0000 7f41a8161640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41a01355b0 0x7f41a0135a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.953+0000 7f41a8161640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41a00cc150 0x7f41a0138740 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.953+0000 7f41a8161640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f418406b4d0 0x7f418406d990 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.953+0000 7f41a8161640 1 --2- 172.21.15.38:0/2040038205 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f418c0039b0 0x7f41a01f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:15.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.953+0000 7f41a8161640 1 -- 172.21.15.38:0/2040038205 >> 172.21.15.38:0/2040038205 conn(0x7f41a006b210 msgr2=0x7f41a00726a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:15.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.953+0000 7f41a8161640 1 -- 172.21.15.38:0/2040038205 shutdown_connections 2024-04-25T08:39:15.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:15.953+0000 7f41a8161640 1 -- 172.21.15.38:0/2040038205 wait complete. 2024-04-25T08:39:15.958 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-04-25T08:39:15.312785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":488784,"kb_used_data":11688,"kb_used_omap":12,"kb_used_meta":476979,"kb_avail":754485936,"statfs":{"total":773094113280,"available":772593598464,"internally_reserved":0,"allocated":11968512,"data_stored":6172356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":488427056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002075"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.209904+0000","last_change":"2024-04-25T08:39:08.220092+0000","last_active":"2024-04-25T08:39:09.209904+0000","last_peered":"2024-04-25T08:39:09.209904+0000","last_clean":"2024-04-25T08:39:09.209904+0000","last_became_active":"2024-04-25T08:39:08.219465+0000","last_became_peered":"2024-04-25T08:39:08.219465+0000","last_unstale":"2024-04-25T08:39:09.209904+0000","last_undegraded":"2024-04-25T08:39:09.209904+0000","last_fullsized":"2024-04-25T08:39:09.209904+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:09.842341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200543+0000","last_change":"2024-04-25T08:39:08.199386+0000","last_active":"2024-04-25T08:39:09.200543+0000","last_peered":"2024-04-25T08:39:09.200543+0000","last_clean":"2024-04-25T08:39:09.200543+0000","last_became_active":"2024-04-25T08:39:08.199128+0000","last_became_peered":"2024-04-25T08:39:08.199128+0000","last_unstale":"2024-04-25T08:39:09.200543+0000","last_undegraded":"2024-04-25T08:39:09.200543+0000","last_fullsized":"2024-04-25T08:39:09.200543+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:13.830148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.210092+0000","last_change":"2024-04-25T08:39:08.207947+0000","last_active":"2024-04-25T08:39:09.210092+0000","last_peered":"2024-04-25T08:39:09.210092+0000","last_clean":"2024-04-25T08:39:09.210092+0000","last_became_active":"2024-04-25T08:39:08.207673+0000","last_became_peered":"2024-04-25T08:39:08.207673+0000","last_unstale":"2024-04-25T08:39:09.210092+0000","last_undegraded":"2024-04-25T08:39:09.210092+0000","last_fullsized":"2024-04-25T08:39:09.210092+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:07:29.630325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270513+0000","last_change":"2024-04-25T08:39:08.213145+0000","last_active":"2024-04-25T08:39:09.270513+0000","last_peered":"2024-04-25T08:39:09.270513+0000","last_clean":"2024-04-25T08:39:09.270513+0000","last_became_active":"2024-04-25T08:39:08.212797+0000","last_became_peered":"2024-04-25T08:39:08.212797+0000","last_unstale":"2024-04-25T08:39:09.270513+0000","last_undegraded":"2024-04-25T08:39:09.270513+0000","last_fullsized":"2024-04-25T08:39:09.270513+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.777754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270618+0000","last_change":"2024-04-25T08:39:08.212881+0000","last_active":"2024-04-25T08:39:09.270618+0000","last_peered":"2024-04-25T08:39:09.270618+0000","last_clean":"2024-04-25T08:39:09.270618+0000","last_became_active":"2024-04-25T08:39:08.212537+0000","last_became_peered":"2024-04-25T08:39:08.212537+0000","last_unstale":"2024-04-25T08:39:09.270618+0000","last_undegraded":"2024-04-25T08:39:09.270618+0000","last_fullsized":"2024-04-25T08:39:09.270618+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:13.111987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.199980+0000","last_change":"2024-04-25T08:39:08.220575+0000","last_active":"2024-04-25T08:39:09.199980+0000","last_peered":"2024-04-25T08:39:09.199980+0000","last_clean":"2024-04-25T08:39:09.199980+0000","last_became_active":"2024-04-25T08:39:08.220296+0000","last_became_peered":"2024-04-25T08:39:08.220296+0000","last_unstale":"2024-04-25T08:39:09.199980+0000","last_undegraded":"2024-04-25T08:39:09.199980+0000","last_fullsized":"2024-04-25T08:39:09.199980+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:13.111987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200210+0000","last_change":"2024-04-25T08:39:08.220638+0000","last_active":"2024-04-25T08:39:09.200210+0000","last_peered":"2024-04-25T08:39:09.200210+0000","last_clean":"2024-04-25T08:39:09.200210+0000","last_became_active":"2024-04-25T08:39:08.219974+0000","last_became_peered":"2024-04-25T08:39:08.219974+0000","last_unstale":"2024-04-25T08:39:09.200210+0000","last_undegraded":"2024-04-25T08:39:09.200210+0000","last_fullsized":"2024-04-25T08:39:09.200210+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:13.190048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197312+0000","last_change":"2024-04-25T08:39:08.217573+0000","last_active":"2024-04-25T08:39:09.197312+0000","last_peered":"2024-04-25T08:39:09.197312+0000","last_clean":"2024-04-25T08:39:09.197312+0000","last_became_active":"2024-04-25T08:39:08.217289+0000","last_became_peered":"2024-04-25T08:39:08.217289+0000","last_unstale":"2024-04-25T08:39:09.197312+0000","last_undegraded":"2024-04-25T08:39:09.197312+0000","last_fullsized":"2024-04-25T08:39:09.197312+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:27.139523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200450+0000","last_change":"2024-04-25T08:39:08.212721+0000","last_active":"2024-04-25T08:39:09.200450+0000","last_peered":"2024-04-25T08:39:09.200450+0000","last_clean":"2024-04-25T08:39:09.200450+0000","last_became_active":"2024-04-25T08:39:08.212428+0000","last_became_peered":"2024-04-25T08:39:08.212428+0000","last_unstale":"2024-04-25T08:39:09.200450+0000","last_undegraded":"2024-04-25T08:39:09.200450+0000","last_fullsized":"2024-04-25T08:39:09.200450+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:18.269122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270894+0000","last_change":"2024-04-25T08:39:08.221779+0000","last_active":"2024-04-25T08:39:09.270894+0000","last_peered":"2024-04-25T08:39:09.270894+0000","last_clean":"2024-04-25T08:39:09.270894+0000","last_became_active":"2024-04-25T08:39:08.221500+0000","last_became_peered":"2024-04-25T08:39:08.221500+0000","last_unstale":"2024-04-25T08:39:09.270894+0000","last_undegraded":"2024-04-25T08:39:09.270894+0000","last_fullsized":"2024-04-25T08:39:09.270894+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:13.830148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.212578+0000","last_change":"2024-04-25T08:39:08.206513+0000","last_active":"2024-04-25T08:39:09.212578+0000","last_peered":"2024-04-25T08:39:09.212578+0000","last_clean":"2024-04-25T08:39:09.212578+0000","last_became_active":"2024-04-25T08:39:08.206250+0000","last_became_peered":"2024-04-25T08:39:08.206250+0000","last_unstale":"2024-04-25T08:39:09.212578+0000","last_undegraded":"2024-04-25T08:39:09.212578+0000","last_fullsized":"2024-04-25T08:39:09.212578+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:18.269122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200510+0000","last_change":"2024-04-25T08:39:08.210664+0000","last_active":"2024-04-25T08:39:09.200510+0000","last_peered":"2024-04-25T08:39:09.200510+0000","last_clean":"2024-04-25T08:39:09.200510+0000","last_became_active":"2024-04-25T08:39:08.210381+0000","last_became_peered":"2024-04-25T08:39:08.210381+0000","last_unstale":"2024-04-25T08:39:09.200510+0000","last_undegraded":"2024-04-25T08:39:09.200510+0000","last_fullsized":"2024-04-25T08:39:09.200510+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:13.830148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270060+0000","last_change":"2024-04-25T08:39:08.217018+0000","last_active":"2024-04-25T08:39:09.270060+0000","last_peered":"2024-04-25T08:39:09.270060+0000","last_clean":"2024-04-25T08:39:09.270060+0000","last_became_active":"2024-04-25T08:39:08.216719+0000","last_became_peered":"2024-04-25T08:39:08.216719+0000","last_unstale":"2024-04-25T08:39:09.270060+0000","last_undegraded":"2024-04-25T08:39:09.270060+0000","last_fullsized":"2024-04-25T08:39:09.270060+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:18.269122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270814+0000","last_change":"2024-04-25T08:39:08.202327+0000","last_active":"2024-04-25T08:39:09.270814+0000","last_peered":"2024-04-25T08:39:09.270814+0000","last_clean":"2024-04-25T08:39:09.270814+0000","last_became_active":"2024-04-25T08:39:08.201981+0000","last_became_peered":"2024-04-25T08:39:08.201981+0000","last_unstale":"2024-04-25T08:39:09.270814+0000","last_undegraded":"2024-04-25T08:39:09.270814+0000","last_fullsized":"2024-04-25T08:39:09.270814+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:45.586068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270816+0000","last_change":"2024-04-25T08:39:08.205994+0000","last_active":"2024-04-25T08:39:09.270816+0000","last_peered":"2024-04-25T08:39:09.270816+0000","last_clean":"2024-04-25T08:39:09.270816+0000","last_became_active":"2024-04-25T08:39:08.205690+0000","last_became_peered":"2024-04-25T08:39:08.205690+0000","last_unstale":"2024-04-25T08:39:09.270816+0000","last_undegraded":"2024-04-25T08:39:09.270816+0000","last_fullsized":"2024-04-25T08:39:09.270816+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:53.391415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270914+0000","last_change":"2024-04-25T08:39:08.208515+0000","last_active":"2024-04-25T08:39:09.270914+0000","last_peered":"2024-04-25T08:39:09.270914+0000","last_clean":"2024-04-25T08:39:09.270914+0000","last_became_active":"2024-04-25T08:39:08.208254+0000","last_became_peered":"2024-04-25T08:39:08.208254+0000","last_unstale":"2024-04-25T08:39:09.270914+0000","last_undegraded":"2024-04-25T08:39:09.270914+0000","last_fullsized":"2024-04-25T08:39:09.270914+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:57.759415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.209941+0000","last_change":"2024-04-25T08:39:08.220621+0000","last_active":"2024-04-25T08:39:09.209941+0000","last_peered":"2024-04-25T08:39:09.209941+0000","last_clean":"2024-04-25T08:39:09.209941+0000","last_became_active":"2024-04-25T08:39:08.220217+0000","last_became_peered":"2024-04-25T08:39:08.220217+0000","last_unstale":"2024-04-25T08:39:09.209941+0000","last_undegraded":"2024-04-25T08:39:09.209941+0000","last_fullsized":"2024-04-25T08:39:09.209941+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:02.373357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200715+0000","last_change":"2024-04-25T08:39:08.220100+0000","last_active":"2024-04-25T08:39:09.200715+0000","last_peered":"2024-04-25T08:39:09.200715+0000","last_clean":"2024-04-25T08:39:09.200715+0000","last_became_active":"2024-04-25T08:39:08.219791+0000","last_became_peered":"2024-04-25T08:39:08.219791+0000","last_unstale":"2024-04-25T08:39:09.200715+0000","last_undegraded":"2024-04-25T08:39:09.200715+0000","last_fullsized":"2024-04-25T08:39:09.200715+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.777754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.209862+0000","last_change":"2024-04-25T08:39:08.205234+0000","last_active":"2024-04-25T08:39:09.209862+0000","last_peered":"2024-04-25T08:39:09.209862+0000","last_clean":"2024-04-25T08:39:09.209862+0000","last_became_active":"2024-04-25T08:39:08.204836+0000","last_became_peered":"2024-04-25T08:39:08.204836+0000","last_unstale":"2024-04-25T08:39:09.209862+0000","last_undegraded":"2024-04-25T08:39:09.209862+0000","last_fullsized":"2024-04-25T08:39:09.209862+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:59.685750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200668+0000","last_change":"2024-04-25T08:39:08.199131+0000","last_active":"2024-04-25T08:39:09.200668+0000","last_peered":"2024-04-25T08:39:09.200668+0000","last_clean":"2024-04-25T08:39:09.200668+0000","last_became_active":"2024-04-25T08:39:08.198798+0000","last_became_peered":"2024-04-25T08:39:08.198798+0000","last_unstale":"2024-04-25T08:39:09.200668+0000","last_undegraded":"2024-04-25T08:39:09.200668+0000","last_fullsized":"2024-04-25T08:39:09.200668+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:08.964337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197357+0000","last_change":"2024-04-25T08:39:08.227451+0000","last_active":"2024-04-25T08:39:09.197357+0000","last_peered":"2024-04-25T08:39:09.197357+0000","last_clean":"2024-04-25T08:39:09.197357+0000","last_became_active":"2024-04-25T08:39:08.227176+0000","last_became_peered":"2024-04-25T08:39:08.227176+0000","last_unstale":"2024-04-25T08:39:09.197357+0000","last_undegraded":"2024-04-25T08:39:09.197357+0000","last_fullsized":"2024-04-25T08:39:09.197357+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:05.480871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270648+0000","last_change":"2024-04-25T08:39:08.218774+0000","last_active":"2024-04-25T08:39:09.270648+0000","last_peered":"2024-04-25T08:39:09.270648+0000","last_clean":"2024-04-25T08:39:09.270648+0000","last_became_active":"2024-04-25T08:39:08.218508+0000","last_became_peered":"2024-04-25T08:39:08.218508+0000","last_unstale":"2024-04-25T08:39:09.270648+0000","last_undegraded":"2024-04-25T08:39:09.270648+0000","last_fullsized":"2024-04-25T08:39:09.270648+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:13.190048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.209625+0000","last_change":"2024-04-25T08:39:08.207380+0000","last_active":"2024-04-25T08:39:09.209625+0000","last_peered":"2024-04-25T08:39:09.209625+0000","last_clean":"2024-04-25T08:39:09.209625+0000","last_became_active":"2024-04-25T08:39:08.207059+0000","last_became_peered":"2024-04-25T08:39:08.207059+0000","last_unstale":"2024-04-25T08:39:09.209625+0000","last_undegraded":"2024-04-25T08:39:09.209625+0000","last_fullsized":"2024-04-25T08:39:09.209625+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:07.620160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270946+0000","last_change":"2024-04-25T08:39:08.216454+0000","last_active":"2024-04-25T08:39:09.270946+0000","last_peered":"2024-04-25T08:39:09.270946+0000","last_clean":"2024-04-25T08:39:09.270946+0000","last_became_active":"2024-04-25T08:39:08.216108+0000","last_became_peered":"2024-04-25T08:39:08.216108+0000","last_unstale":"2024-04-25T08:39:09.270946+0000","last_undegraded":"2024-04-25T08:39:09.270946+0000","last_fullsized":"2024-04-25T08:39:09.270946+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:08.964337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200086+0000","last_change":"2024-04-25T08:39:08.218837+0000","last_active":"2024-04-25T08:39:09.200086+0000","last_peered":"2024-04-25T08:39:09.200086+0000","last_clean":"2024-04-25T08:39:09.200086+0000","last_became_active":"2024-04-25T08:39:08.218461+0000","last_became_peered":"2024-04-25T08:39:08.218461+0000","last_unstale":"2024-04-25T08:39:09.200086+0000","last_undegraded":"2024-04-25T08:39:09.200086+0000","last_fullsized":"2024-04-25T08:39:09.200086+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:13.190048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200204+0000","last_change":"2024-04-25T08:39:08.197730+0000","last_active":"2024-04-25T08:39:09.200204+0000","last_peered":"2024-04-25T08:39:09.200204+0000","last_clean":"2024-04-25T08:39:09.200204+0000","last_became_active":"2024-04-25T08:39:08.197426+0000","last_became_peered":"2024-04-25T08:39:08.197426+0000","last_unstale":"2024-04-25T08:39:09.200204+0000","last_undegraded":"2024-04-25T08:39:09.200204+0000","last_fullsized":"2024-04-25T08:39:09.200204+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:18.269122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.212584+0000","last_change":"2024-04-25T08:39:08.205710+0000","last_active":"2024-04-25T08:39:09.212584+0000","last_peered":"2024-04-25T08:39:09.212584+0000","last_clean":"2024-04-25T08:39:09.212584+0000","last_became_active":"2024-04-25T08:39:08.205430+0000","last_became_peered":"2024-04-25T08:39:08.205430+0000","last_unstale":"2024-04-25T08:39:09.212584+0000","last_undegraded":"2024-04-25T08:39:09.212584+0000","last_fullsized":"2024-04-25T08:39:09.212584+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:13.830148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200761+0000","last_change":"2024-04-25T08:39:08.210422+0000","last_active":"2024-04-25T08:39:09.200761+0000","last_peered":"2024-04-25T08:39:09.200761+0000","last_clean":"2024-04-25T08:39:09.200761+0000","last_became_active":"2024-04-25T08:39:08.210135+0000","last_became_peered":"2024-04-25T08:39:08.210135+0000","last_unstale":"2024-04-25T08:39:09.200761+0000","last_undegraded":"2024-04-25T08:39:09.200761+0000","last_fullsized":"2024-04-25T08:39:09.200761+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:23.705362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197509+0000","last_change":"2024-04-25T08:39:08.231778+0000","last_active":"2024-04-25T08:39:09.197509+0000","last_peered":"2024-04-25T08:39:09.197509+0000","last_clean":"2024-04-25T08:39:09.197509+0000","last_became_active":"2024-04-25T08:39:08.231005+0000","last_became_peered":"2024-04-25T08:39:08.231005+0000","last_unstale":"2024-04-25T08:39:09.197509+0000","last_undegraded":"2024-04-25T08:39:09.197509+0000","last_fullsized":"2024-04-25T08:39:09.197509+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:03.598622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200944+0000","last_change":"2024-04-25T08:39:08.207761+0000","last_active":"2024-04-25T08:39:09.200944+0000","last_peered":"2024-04-25T08:39:09.200944+0000","last_clean":"2024-04-25T08:39:09.200944+0000","last_became_active":"2024-04-25T08:39:08.207478+0000","last_became_peered":"2024-04-25T08:39:08.207478+0000","last_unstale":"2024-04-25T08:39:09.200944+0000","last_undegraded":"2024-04-25T08:39:09.200944+0000","last_fullsized":"2024-04-25T08:39:09.200944+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.777754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.201180+0000","last_change":"2024-04-25T08:39:08.209026+0000","last_active":"2024-04-25T08:39:09.201180+0000","last_peered":"2024-04-25T08:39:09.201180+0000","last_clean":"2024-04-25T08:39:09.201180+0000","last_became_active":"2024-04-25T08:39:08.205253+0000","last_became_peered":"2024-04-25T08:39:08.205253+0000","last_unstale":"2024-04-25T08:39:09.201180+0000","last_undegraded":"2024-04-25T08:39:09.201180+0000","last_fullsized":"2024-04-25T08:39:09.201180+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:38.777577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.196998+0000","last_change":"2024-04-25T08:39:08.227699+0000","last_active":"2024-04-25T08:39:09.196998+0000","last_peered":"2024-04-25T08:39:09.196998+0000","last_clean":"2024-04-25T08:39:09.196998+0000","last_became_active":"2024-04-25T08:39:08.227269+0000","last_became_peered":"2024-04-25T08:39:08.227269+0000","last_unstale":"2024-04-25T08:39:09.196998+0000","last_undegraded":"2024-04-25T08:39:09.196998+0000","last_fullsized":"2024-04-25T08:39:09.196998+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:54.041577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.209707+0000","last_change":"2024-04-25T08:39:08.221269+0000","last_active":"2024-04-25T08:39:09.209707+0000","last_peered":"2024-04-25T08:39:09.209707+0000","last_clean":"2024-04-25T08:39:09.209707+0000","last_became_active":"2024-04-25T08:39:08.220278+0000","last_became_peered":"2024-04-25T08:39:08.220278+0000","last_unstale":"2024-04-25T08:39:09.209707+0000","last_undegraded":"2024-04-25T08:39:09.209707+0000","last_fullsized":"2024-04-25T08:39:09.209707+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:22.168080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"107'4","reported_seq":18,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270431+0000","last_change":"2024-04-25T08:39:07.190276+0000","last_active":"2024-04-25T08:39:09.270431+0000","last_peered":"2024-04-25T08:39:09.270431+0000","last_clean":"2024-04-25T08:39:09.270431+0000","last_became_active":"2024-04-25T08:39:07.189857+0000","last_became_peered":"2024-04-25T08:39:07.189857+0000","last_unstale":"2024-04-25T08:39:09.270431+0000","last_undegraded":"2024-04-25T08:39:09.270431+0000","last_fullsized":"2024-04-25T08:39:09.270431+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:27.814831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197039+0000","last_change":"2024-04-25T08:39:08.217452+0000","last_active":"2024-04-25T08:39:09.197039+0000","last_peered":"2024-04-25T08:39:09.197039+0000","last_clean":"2024-04-25T08:39:09.197039+0000","last_became_active":"2024-04-25T08:39:08.217157+0000","last_became_peered":"2024-04-25T08:39:08.217157+0000","last_unstale":"2024-04-25T08:39:09.197039+0000","last_undegraded":"2024-04-25T08:39:09.197039+0000","last_fullsized":"2024-04-25T08:39:09.197039+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:29.232268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197371+0000","last_change":"2024-04-25T08:39:07.200692+0000","last_active":"2024-04-25T08:39:09.197371+0000","last_peered":"2024-04-25T08:39:09.197371+0000","last_clean":"2024-04-25T08:39:09.197371+0000","last_became_active":"2024-04-25T08:39:07.200405+0000","last_became_peered":"2024-04-25T08:39:07.200405+0000","last_unstale":"2024-04-25T08:39:09.197371+0000","last_undegraded":"2024-04-25T08:39:09.197371+0000","last_fullsized":"2024-04-25T08:39:09.197371+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:30.521442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200989+0000","last_change":"2024-04-25T08:39:08.219031+0000","last_active":"2024-04-25T08:39:09.200989+0000","last_peered":"2024-04-25T08:39:09.200989+0000","last_clean":"2024-04-25T08:39:09.200989+0000","last_became_active":"2024-04-25T08:39:08.218735+0000","last_became_peered":"2024-04-25T08:39:08.218735+0000","last_unstale":"2024-04-25T08:39:09.200989+0000","last_undegraded":"2024-04-25T08:39:09.200989+0000","last_fullsized":"2024-04-25T08:39:09.200989+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:33.542206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.199659+0000","last_change":"2024-04-25T08:39:07.195456+0000","last_active":"2024-04-25T08:39:09.199659+0000","last_peered":"2024-04-25T08:39:09.199659+0000","last_clean":"2024-04-25T08:39:09.199659+0000","last_became_active":"2024-04-25T08:39:07.194725+0000","last_became_peered":"2024-04-25T08:39:07.194725+0000","last_unstale":"2024-04-25T08:39:09.199659+0000","last_undegraded":"2024-04-25T08:39:09.199659+0000","last_fullsized":"2024-04-25T08:39:09.199659+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:03.259504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.212546+0000","last_change":"2024-04-25T08:39:08.208202+0000","last_active":"2024-04-25T08:39:09.212546+0000","last_peered":"2024-04-25T08:39:09.212546+0000","last_clean":"2024-04-25T08:39:09.212546+0000","last_became_active":"2024-04-25T08:39:08.203481+0000","last_became_peered":"2024-04-25T08:39:08.203481+0000","last_unstale":"2024-04-25T08:39:09.212546+0000","last_undegraded":"2024-04-25T08:39:09.212546+0000","last_fullsized":"2024-04-25T08:39:09.212546+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:45.208316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270640+0000","last_change":"2024-04-25T08:39:07.197963+0000","last_active":"2024-04-25T08:39:09.270640+0000","last_peered":"2024-04-25T08:39:09.270640+0000","last_clean":"2024-04-25T08:39:09.270640+0000","last_became_active":"2024-04-25T08:39:07.197658+0000","last_became_peered":"2024-04-25T08:39:07.197658+0000","last_unstale":"2024-04-25T08:39:09.270640+0000","last_undegraded":"2024-04-25T08:39:09.270640+0000","last_fullsized":"2024-04-25T08:39:09.270640+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:39.317231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270663+0000","last_change":"2024-04-25T08:39:08.217416+0000","last_active":"2024-04-25T08:39:09.270663+0000","last_peered":"2024-04-25T08:39:09.270663+0000","last_clean":"2024-04-25T08:39:09.270663+0000","last_became_active":"2024-04-25T08:39:08.217118+0000","last_became_peered":"2024-04-25T08:39:08.217118+0000","last_unstale":"2024-04-25T08:39:09.270663+0000","last_undegraded":"2024-04-25T08:39:09.270663+0000","last_fullsized":"2024-04-25T08:39:09.270663+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:33.542206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.271064+0000","last_change":"2024-04-25T08:39:07.191029+0000","last_active":"2024-04-25T08:39:09.271064+0000","last_peered":"2024-04-25T08:39:09.271064+0000","last_clean":"2024-04-25T08:39:09.271064+0000","last_became_active":"2024-04-25T08:39:07.190722+0000","last_became_peered":"2024-04-25T08:39:07.190722+0000","last_unstale":"2024-04-25T08:39:09.271064+0000","last_undegraded":"2024-04-25T08:39:09.271064+0000","last_fullsized":"2024-04-25T08:39:09.271064+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:13.944452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.210038+0000","last_change":"2024-04-25T08:39:08.205057+0000","last_active":"2024-04-25T08:39:09.210038+0000","last_peered":"2024-04-25T08:39:09.210038+0000","last_clean":"2024-04-25T08:39:09.210038+0000","last_became_active":"2024-04-25T08:39:08.204653+0000","last_became_peered":"2024-04-25T08:39:08.204653+0000","last_unstale":"2024-04-25T08:39:09.210038+0000","last_undegraded":"2024-04-25T08:39:09.210038+0000","last_fullsized":"2024-04-25T08:39:09.210038+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:49:41.473024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.212304+0000","last_change":"2024-04-25T08:39:07.198565+0000","last_active":"2024-04-25T08:39:09.212304+0000","last_peered":"2024-04-25T08:39:09.212304+0000","last_clean":"2024-04-25T08:39:09.212304+0000","last_became_active":"2024-04-25T08:39:07.197861+0000","last_became_peered":"2024-04-25T08:39:07.197861+0000","last_unstale":"2024-04-25T08:39:09.212304+0000","last_undegraded":"2024-04-25T08:39:09.212304+0000","last_fullsized":"2024-04-25T08:39:09.212304+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:03.259504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200894+0000","last_change":"2024-04-25T08:39:08.210035+0000","last_active":"2024-04-25T08:39:09.200894+0000","last_peered":"2024-04-25T08:39:09.200894+0000","last_clean":"2024-04-25T08:39:09.200894+0000","last_became_active":"2024-04-25T08:39:08.209740+0000","last_became_peered":"2024-04-25T08:39:08.209740+0000","last_unstale":"2024-04-25T08:39:09.200894+0000","last_undegraded":"2024-04-25T08:39:09.200894+0000","last_fullsized":"2024-04-25T08:39:09.200894+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:33.542206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.199874+0000","last_change":"2024-04-25T08:39:07.191219+0000","last_active":"2024-04-25T08:39:09.199874+0000","last_peered":"2024-04-25T08:39:09.199874+0000","last_clean":"2024-04-25T08:39:09.199874+0000","last_became_active":"2024-04-25T08:39:07.190902+0000","last_became_peered":"2024-04-25T08:39:07.190902+0000","last_unstale":"2024-04-25T08:39:09.199874+0000","last_undegraded":"2024-04-25T08:39:09.199874+0000","last_fullsized":"2024-04-25T08:39:09.199874+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:19.824202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270295+0000","last_change":"2024-04-25T08:39:08.222649+0000","last_active":"2024-04-25T08:39:09.270295+0000","last_peered":"2024-04-25T08:39:09.270295+0000","last_clean":"2024-04-25T08:39:09.270295+0000","last_became_active":"2024-04-25T08:39:08.222382+0000","last_became_peered":"2024-04-25T08:39:08.222382+0000","last_unstale":"2024-04-25T08:39:09.270295+0000","last_undegraded":"2024-04-25T08:39:09.270295+0000","last_fullsized":"2024-04-25T08:39:09.270295+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:08.964337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.269957+0000","last_change":"2024-04-25T08:39:07.191760+0000","last_active":"2024-04-25T08:39:09.269957+0000","last_peered":"2024-04-25T08:39:09.269957+0000","last_clean":"2024-04-25T08:39:09.269957+0000","last_became_active":"2024-04-25T08:39:07.191248+0000","last_became_peered":"2024-04-25T08:39:07.191248+0000","last_unstale":"2024-04-25T08:39:09.269957+0000","last_undegraded":"2024-04-25T08:39:09.269957+0000","last_fullsized":"2024-04-25T08:39:09.269957+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:58.223863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.209542+0000","last_change":"2024-04-25T08:39:08.221879+0000","last_active":"2024-04-25T08:39:09.209542+0000","last_peered":"2024-04-25T08:39:09.209542+0000","last_clean":"2024-04-25T08:39:09.209542+0000","last_became_active":"2024-04-25T08:39:08.220977+0000","last_became_peered":"2024-04-25T08:39:08.220977+0000","last_unstale":"2024-04-25T08:39:09.209542+0000","last_undegraded":"2024-04-25T08:39:09.209542+0000","last_fullsized":"2024-04-25T08:39:09.209542+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:43.103198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200182+0000","last_change":"2024-04-25T08:39:07.198061+0000","last_active":"2024-04-25T08:39:09.200182+0000","last_peered":"2024-04-25T08:39:09.200182+0000","last_clean":"2024-04-25T08:39:09.200182+0000","last_became_active":"2024-04-25T08:39:07.197146+0000","last_became_peered":"2024-04-25T08:39:07.197146+0000","last_unstale":"2024-04-25T08:39:09.200182+0000","last_undegraded":"2024-04-25T08:39:09.200182+0000","last_fullsized":"2024-04-25T08:39:09.200182+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:16.487240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270127+0000","last_change":"2024-04-25T08:39:08.226018+0000","last_active":"2024-04-25T08:39:09.270127+0000","last_peered":"2024-04-25T08:39:09.270127+0000","last_clean":"2024-04-25T08:39:09.270127+0000","last_became_active":"2024-04-25T08:39:08.225732+0000","last_became_peered":"2024-04-25T08:39:08.225732+0000","last_unstale":"2024-04-25T08:39:09.270127+0000","last_undegraded":"2024-04-25T08:39:09.270127+0000","last_fullsized":"2024-04-25T08:39:09.270127+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:07.186368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.209793+0000","last_change":"2024-04-25T08:39:07.195425+0000","last_active":"2024-04-25T08:39:09.209793+0000","last_peered":"2024-04-25T08:39:09.209793+0000","last_clean":"2024-04-25T08:39:09.209793+0000","last_became_active":"2024-04-25T08:39:07.195087+0000","last_became_peered":"2024-04-25T08:39:07.195087+0000","last_unstale":"2024-04-25T08:39:09.209793+0000","last_undegraded":"2024-04-25T08:39:09.209793+0000","last_fullsized":"2024-04-25T08:39:09.209793+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:32.251844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197185+0000","last_change":"2024-04-25T08:39:08.218098+0000","last_active":"2024-04-25T08:39:09.197185+0000","last_peered":"2024-04-25T08:39:09.197185+0000","last_clean":"2024-04-25T08:39:09.197185+0000","last_became_active":"2024-04-25T08:39:08.217805+0000","last_became_peered":"2024-04-25T08:39:08.217805+0000","last_unstale":"2024-04-25T08:39:09.197185+0000","last_undegraded":"2024-04-25T08:39:09.197185+0000","last_fullsized":"2024-04-25T08:39:09.197185+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:18.502284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"107'3","reported_seq":17,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270930+0000","last_change":"2024-04-25T08:39:07.192065+0000","last_active":"2024-04-25T08:39:09.270930+0000","last_peered":"2024-04-25T08:39:09.270930+0000","last_clean":"2024-04-25T08:39:09.270930+0000","last_became_active":"2024-04-25T08:39:07.191757+0000","last_became_peered":"2024-04-25T08:39:07.191757+0000","last_unstale":"2024-04-25T08:39:09.270930+0000","last_undegraded":"2024-04-25T08:39:09.270930+0000","last_fullsized":"2024-04-25T08:39:09.270930+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:17.547829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270783+0000","last_change":"2024-04-25T08:39:08.227282+0000","last_active":"2024-04-25T08:39:09.270783+0000","last_peered":"2024-04-25T08:39:09.270783+0000","last_clean":"2024-04-25T08:39:09.270783+0000","last_became_active":"2024-04-25T08:39:08.226984+0000","last_became_peered":"2024-04-25T08:39:08.226984+0000","last_unstale":"2024-04-25T08:39:09.270783+0000","last_undegraded":"2024-04-25T08:39:09.270783+0000","last_fullsized":"2024-04-25T08:39:09.270783+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:18.269122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"107'6","reported_seq":20,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200815+0000","last_change":"2024-04-25T08:39:07.196396+0000","last_active":"2024-04-25T08:39:09.200815+0000","last_peered":"2024-04-25T08:39:09.200815+0000","last_clean":"2024-04-25T08:39:09.200815+0000","last_became_active":"2024-04-25T08:39:07.195109+0000","last_became_peered":"2024-04-25T08:39:07.195109+0000","last_unstale":"2024-04-25T08:39:09.200815+0000","last_undegraded":"2024-04-25T08:39:09.200815+0000","last_fullsized":"2024-04-25T08:39:09.200815+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:27.118962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.212437+0000","last_change":"2024-04-25T08:39:08.217018+0000","last_active":"2024-04-25T08:39:09.212437+0000","last_peered":"2024-04-25T08:39:09.212437+0000","last_clean":"2024-04-25T08:39:09.212437+0000","last_became_active":"2024-04-25T08:39:08.216735+0000","last_became_peered":"2024-04-25T08:39:08.216735+0000","last_unstale":"2024-04-25T08:39:09.212437+0000","last_undegraded":"2024-04-25T08:39:09.212437+0000","last_fullsized":"2024-04-25T08:39:09.212437+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:13.111987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270737+0000","last_change":"2024-04-25T08:39:07.197589+0000","last_active":"2024-04-25T08:39:09.270737+0000","last_peered":"2024-04-25T08:39:09.270737+0000","last_clean":"2024-04-25T08:39:09.270737+0000","last_became_active":"2024-04-25T08:39:07.197301+0000","last_became_peered":"2024-04-25T08:39:07.197301+0000","last_unstale":"2024-04-25T08:39:09.270737+0000","last_undegraded":"2024-04-25T08:39:09.270737+0000","last_fullsized":"2024-04-25T08:39:09.270737+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:03.259504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200586+0000","last_change":"2024-04-25T08:39:08.214283+0000","last_active":"2024-04-25T08:39:09.200586+0000","last_peered":"2024-04-25T08:39:09.200586+0000","last_clean":"2024-04-25T08:39:09.200586+0000","last_became_active":"2024-04-25T08:39:08.213847+0000","last_became_peered":"2024-04-25T08:39:08.213847+0000","last_unstale":"2024-04-25T08:39:09.200586+0000","last_undegraded":"2024-04-25T08:39:09.200586+0000","last_fullsized":"2024-04-25T08:39:09.200586+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:08.964337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270040+0000","last_change":"2024-04-25T08:39:07.189044+0000","last_active":"2024-04-25T08:39:09.270040+0000","last_peered":"2024-04-25T08:39:09.270040+0000","last_clean":"2024-04-25T08:39:09.270040+0000","last_became_active":"2024-04-25T08:39:07.188847+0000","last_became_peered":"2024-04-25T08:39:07.188847+0000","last_unstale":"2024-04-25T08:39:09.270040+0000","last_undegraded":"2024-04-25T08:39:09.270040+0000","last_fullsized":"2024-04-25T08:39:09.270040+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:50.126367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197603+0000","last_change":"2024-04-25T08:39:08.221665+0000","last_active":"2024-04-25T08:39:09.197603+0000","last_peered":"2024-04-25T08:39:09.197603+0000","last_clean":"2024-04-25T08:39:09.197603+0000","last_became_active":"2024-04-25T08:39:08.221389+0000","last_became_peered":"2024-04-25T08:39:08.221389+0000","last_unstale":"2024-04-25T08:39:09.197603+0000","last_undegraded":"2024-04-25T08:39:09.197603+0000","last_fullsized":"2024-04-25T08:39:09.197603+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:13.178353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197032+0000","last_change":"2024-04-25T08:39:07.198202+0000","last_active":"2024-04-25T08:39:09.197032+0000","last_peered":"2024-04-25T08:39:09.197032+0000","last_clean":"2024-04-25T08:39:09.197032+0000","last_became_active":"2024-04-25T08:39:07.197778+0000","last_became_peered":"2024-04-25T08:39:07.197778+0000","last_unstale":"2024-04-25T08:39:09.197032+0000","last_undegraded":"2024-04-25T08:39:09.197032+0000","last_fullsized":"2024-04-25T08:39:09.197032+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:31.468260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.210049+0000","last_change":"2024-04-25T08:39:08.220341+0000","last_active":"2024-04-25T08:39:09.210049+0000","last_peered":"2024-04-25T08:39:09.210049+0000","last_clean":"2024-04-25T08:39:09.210049+0000","last_became_active":"2024-04-25T08:39:08.219762+0000","last_became_peered":"2024-04-25T08:39:08.219762+0000","last_unstale":"2024-04-25T08:39:09.210049+0000","last_undegraded":"2024-04-25T08:39:09.210049+0000","last_fullsized":"2024-04-25T08:39:09.210049+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:08.811959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"107'8","reported_seq":22,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.199542+0000","last_change":"2024-04-25T08:39:07.191103+0000","last_active":"2024-04-25T08:39:09.199542+0000","last_peered":"2024-04-25T08:39:09.199542+0000","last_clean":"2024-04-25T08:39:09.199542+0000","last_became_active":"2024-04-25T08:39:07.190789+0000","last_became_peered":"2024-04-25T08:39:07.190789+0000","last_unstale":"2024-04-25T08:39:09.199542+0000","last_undegraded":"2024-04-25T08:39:09.199542+0000","last_fullsized":"2024-04-25T08:39:09.199542+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:39.317231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197186+0000","last_change":"2024-04-25T08:39:08.212453+0000","last_active":"2024-04-25T08:39:09.197186+0000","last_peered":"2024-04-25T08:39:09.197186+0000","last_clean":"2024-04-25T08:39:09.197186+0000","last_became_active":"2024-04-25T08:39:08.212148+0000","last_became_peered":"2024-04-25T08:39:08.212148+0000","last_unstale":"2024-04-25T08:39:09.197186+0000","last_undegraded":"2024-04-25T08:39:09.197186+0000","last_fullsized":"2024-04-25T08:39:09.197186+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:16.826292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200550+0000","last_change":"2024-04-25T08:39:07.194224+0000","last_active":"2024-04-25T08:39:09.200550+0000","last_peered":"2024-04-25T08:39:09.200550+0000","last_clean":"2024-04-25T08:39:09.200550+0000","last_became_active":"2024-04-25T08:39:07.193936+0000","last_became_peered":"2024-04-25T08:39:07.193936+0000","last_unstale":"2024-04-25T08:39:09.200550+0000","last_undegraded":"2024-04-25T08:39:09.200550+0000","last_fullsized":"2024-04-25T08:39:09.200550+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:20.621755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200041+0000","last_change":"2024-04-25T08:39:08.202514+0000","last_active":"2024-04-25T08:39:09.200041+0000","last_peered":"2024-04-25T08:39:09.200041+0000","last_clean":"2024-04-25T08:39:09.200041+0000","last_became_active":"2024-04-25T08:39:08.202181+0000","last_became_peered":"2024-04-25T08:39:08.202181+0000","last_unstale":"2024-04-25T08:39:09.200041+0000","last_undegraded":"2024-04-25T08:39:09.200041+0000","last_fullsized":"2024-04-25T08:39:09.200041+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:45.208316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"107'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270991+0000","last_change":"2024-04-25T08:39:07.197977+0000","last_active":"2024-04-25T08:39:09.270991+0000","last_peered":"2024-04-25T08:39:09.270991+0000","last_clean":"2024-04-25T08:39:09.270991+0000","last_became_active":"2024-04-25T08:39:07.197678+0000","last_became_peered":"2024-04-25T08:39:07.197678+0000","last_unstale":"2024-04-25T08:39:09.270991+0000","last_undegraded":"2024-04-25T08:39:09.270991+0000","last_fullsized":"2024-04-25T08:39:09.270991+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:53.623300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270627+0000","last_change":"2024-04-25T08:39:08.213062+0000","last_active":"2024-04-25T08:39:09.270627+0000","last_peered":"2024-04-25T08:39:09.270627+0000","last_clean":"2024-04-25T08:39:09.270627+0000","last_became_active":"2024-04-25T08:39:08.212210+0000","last_became_peered":"2024-04-25T08:39:08.212210+0000","last_unstale":"2024-04-25T08:39:09.270627+0000","last_undegraded":"2024-04-25T08:39:09.270627+0000","last_fullsized":"2024-04-25T08:39:09.270627+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:45.208316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.212490+0000","last_change":"2024-04-25T08:39:07.197738+0000","last_active":"2024-04-25T08:39:09.212490+0000","last_peered":"2024-04-25T08:39:09.212490+0000","last_clean":"2024-04-25T08:39:09.212490+0000","last_became_active":"2024-04-25T08:39:07.197436+0000","last_became_peered":"2024-04-25T08:39:07.197436+0000","last_unstale":"2024-04-25T08:39:09.212490+0000","last_undegraded":"2024-04-25T08:39:09.212490+0000","last_fullsized":"2024-04-25T08:39:09.212490+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:19.824202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.209755+0000","last_change":"2024-04-25T08:39:08.218562+0000","last_active":"2024-04-25T08:39:09.209755+0000","last_peered":"2024-04-25T08:39:09.209755+0000","last_clean":"2024-04-25T08:39:09.209755+0000","last_became_active":"2024-04-25T08:39:08.218245+0000","last_became_peered":"2024-04-25T08:39:08.218245+0000","last_unstale":"2024-04-25T08:39:09.209755+0000","last_undegraded":"2024-04-25T08:39:09.209755+0000","last_fullsized":"2024-04-25T08:39:09.209755+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:17.579640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197559+0000","last_change":"2024-04-25T08:39:07.200590+0000","last_active":"2024-04-25T08:39:09.197559+0000","last_peered":"2024-04-25T08:39:09.197559+0000","last_clean":"2024-04-25T08:39:09.197559+0000","last_became_active":"2024-04-25T08:39:07.200312+0000","last_became_peered":"2024-04-25T08:39:07.200312+0000","last_unstale":"2024-04-25T08:39:09.197559+0000","last_undegraded":"2024-04-25T08:39:09.197559+0000","last_fullsized":"2024-04-25T08:39:09.197559+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:31.020594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.209953+0000","last_change":"2024-04-25T08:39:08.213244+0000","last_active":"2024-04-25T08:39:09.209953+0000","last_peered":"2024-04-25T08:39:09.209953+0000","last_clean":"2024-04-25T08:39:09.209953+0000","last_became_active":"2024-04-25T08:39:08.212965+0000","last_became_peered":"2024-04-25T08:39:08.212965+0000","last_unstale":"2024-04-25T08:39:09.209953+0000","last_undegraded":"2024-04-25T08:39:09.209953+0000","last_fullsized":"2024-04-25T08:39:09.209953+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:43.805299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197062+0000","last_change":"2024-04-25T08:39:07.201896+0000","last_active":"2024-04-25T08:39:09.197062+0000","last_peered":"2024-04-25T08:39:09.197062+0000","last_clean":"2024-04-25T08:39:09.197062+0000","last_became_active":"2024-04-25T08:39:07.201576+0000","last_became_peered":"2024-04-25T08:39:07.201576+0000","last_unstale":"2024-04-25T08:39:09.197062+0000","last_undegraded":"2024-04-25T08:39:09.197062+0000","last_fullsized":"2024-04-25T08:39:09.197062+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:55.940575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270217+0000","last_change":"2024-04-25T08:39:08.224283+0000","last_active":"2024-04-25T08:39:09.270217+0000","last_peered":"2024-04-25T08:39:09.270217+0000","last_clean":"2024-04-25T08:39:09.270217+0000","last_became_active":"2024-04-25T08:39:08.223957+0000","last_became_peered":"2024-04-25T08:39:08.223957+0000","last_unstale":"2024-04-25T08:39:09.270217+0000","last_undegraded":"2024-04-25T08:39:09.270217+0000","last_fullsized":"2024-04-25T08:39:09.270217+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:13.830148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.201105+0000","last_change":"2024-04-25T08:39:07.193296+0000","last_active":"2024-04-25T08:39:09.201105+0000","last_peered":"2024-04-25T08:39:09.201105+0000","last_clean":"2024-04-25T08:39:09.201105+0000","last_became_active":"2024-04-25T08:39:07.193017+0000","last_became_peered":"2024-04-25T08:39:07.193017+0000","last_unstale":"2024-04-25T08:39:09.201105+0000","last_undegraded":"2024-04-25T08:39:09.201105+0000","last_fullsized":"2024-04-25T08:39:09.201105+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:39.317231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197500+0000","last_change":"2024-04-25T08:39:08.219478+0000","last_active":"2024-04-25T08:39:09.197500+0000","last_peered":"2024-04-25T08:39:09.197500+0000","last_clean":"2024-04-25T08:39:09.197500+0000","last_became_active":"2024-04-25T08:39:08.219202+0000","last_became_peered":"2024-04-25T08:39:08.219202+0000","last_unstale":"2024-04-25T08:39:09.197500+0000","last_undegraded":"2024-04-25T08:39:09.197500+0000","last_fullsized":"2024-04-25T08:39:09.197500+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:44.761395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197284+0000","last_change":"2024-04-25T08:39:07.201924+0000","last_active":"2024-04-25T08:39:09.197284+0000","last_peered":"2024-04-25T08:39:09.197284+0000","last_clean":"2024-04-25T08:39:09.197284+0000","last_became_active":"2024-04-25T08:39:07.201634+0000","last_became_peered":"2024-04-25T08:39:07.201634+0000","last_unstale":"2024-04-25T08:39:09.197284+0000","last_undegraded":"2024-04-25T08:39:09.197284+0000","last_fullsized":"2024-04-25T08:39:09.197284+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:25.136701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.209541+0000","last_change":"2024-04-25T08:39:08.221564+0000","last_active":"2024-04-25T08:39:09.209541+0000","last_peered":"2024-04-25T08:39:09.209541+0000","last_clean":"2024-04-25T08:39:09.209541+0000","last_became_active":"2024-04-25T08:39:08.220899+0000","last_became_peered":"2024-04-25T08:39:08.220899+0000","last_unstale":"2024-04-25T08:39:09.209541+0000","last_undegraded":"2024-04-25T08:39:09.209541+0000","last_fullsized":"2024-04-25T08:39:09.209541+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:31.583339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270940+0000","last_change":"2024-04-25T08:39:07.196017+0000","last_active":"2024-04-25T08:39:09.270940+0000","last_peered":"2024-04-25T08:39:09.270940+0000","last_clean":"2024-04-25T08:39:09.270940+0000","last_became_active":"2024-04-25T08:39:07.195702+0000","last_became_peered":"2024-04-25T08:39:07.195702+0000","last_unstale":"2024-04-25T08:39:09.270940+0000","last_undegraded":"2024-04-25T08:39:09.270940+0000","last_fullsized":"2024-04-25T08:39:09.270940+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:19.824202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.199767+0000","last_change":"2024-04-25T08:39:08.206707+0000","last_active":"2024-04-25T08:39:09.199767+0000","last_peered":"2024-04-25T08:39:09.199767+0000","last_clean":"2024-04-25T08:39:09.199767+0000","last_became_active":"2024-04-25T08:39:08.206404+0000","last_became_peered":"2024-04-25T08:39:08.206404+0000","last_unstale":"2024-04-25T08:39:09.199767+0000","last_undegraded":"2024-04-25T08:39:09.199767+0000","last_fullsized":"2024-04-25T08:39:09.199767+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:45.208316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"108'1","reported_seq":15,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.209859+0000","last_change":"2024-04-25T08:39:07.198773+0000","last_active":"2024-04-25T08:39:09.209859+0000","last_peered":"2024-04-25T08:39:09.209859+0000","last_clean":"2024-04-25T08:39:09.209859+0000","last_became_active":"2024-04-25T08:39:07.197463+0000","last_became_peered":"2024-04-25T08:39:07.197463+0000","last_unstale":"2024-04-25T08:39:09.209859+0000","last_undegraded":"2024-04-25T08:39:09.209859+0000","last_fullsized":"2024-04-25T08:39:09.209859+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:58.021998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270035+0000","last_change":"2024-04-25T08:39:08.223707+0000","last_active":"2024-04-25T08:39:09.270035+0000","last_peered":"2024-04-25T08:39:09.270035+0000","last_clean":"2024-04-25T08:39:09.270035+0000","last_became_active":"2024-04-25T08:39:08.223431+0000","last_became_peered":"2024-04-25T08:39:08.223431+0000","last_unstale":"2024-04-25T08:39:09.270035+0000","last_undegraded":"2024-04-25T08:39:09.270035+0000","last_fullsized":"2024-04-25T08:39:09.270035+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:13.190048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.200690+0000","last_change":"2024-04-25T08:39:07.188334+0000","last_active":"2024-04-25T08:39:09.200690+0000","last_peered":"2024-04-25T08:39:09.200690+0000","last_clean":"2024-04-25T08:39:09.200690+0000","last_became_active":"2024-04-25T08:39:07.187795+0000","last_became_peered":"2024-04-25T08:39:07.187795+0000","last_unstale":"2024-04-25T08:39:09.200690+0000","last_undegraded":"2024-04-25T08:39:09.200690+0000","last_fullsized":"2024-04-25T08:39:09.200690+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:39.219462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.212555+0000","last_change":"2024-04-25T08:39:08.214665+0000","last_active":"2024-04-25T08:39:09.212555+0000","last_peered":"2024-04-25T08:39:09.212555+0000","last_clean":"2024-04-25T08:39:09.212555+0000","last_became_active":"2024-04-25T08:39:08.214400+0000","last_became_peered":"2024-04-25T08:39:08.214400+0000","last_unstale":"2024-04-25T08:39:09.212555+0000","last_undegraded":"2024-04-25T08:39:09.212555+0000","last_fullsized":"2024-04-25T08:39:09.212555+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:13.190048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270762+0000","last_change":"2024-04-25T08:39:07.197165+0000","last_active":"2024-04-25T08:39:09.270762+0000","last_peered":"2024-04-25T08:39:09.270762+0000","last_clean":"2024-04-25T08:39:09.270762+0000","last_became_active":"2024-04-25T08:39:07.196785+0000","last_became_peered":"2024-04-25T08:39:07.196785+0000","last_unstale":"2024-04-25T08:39:09.270762+0000","last_undegraded":"2024-04-25T08:39:09.270762+0000","last_fullsized":"2024-04-25T08:39:09.270762+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:54.429666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197099+0000","last_change":"2024-04-25T08:39:08.219932+0000","last_active":"2024-04-25T08:39:09.197099+0000","last_peered":"2024-04-25T08:39:09.197099+0000","last_clean":"2024-04-25T08:39:09.197099+0000","last_became_active":"2024-04-25T08:39:08.218615+0000","last_became_peered":"2024-04-25T08:39:08.218615+0000","last_unstale":"2024-04-25T08:39:09.197099+0000","last_undegraded":"2024-04-25T08:39:09.197099+0000","last_fullsized":"2024-04-25T08:39:09.197099+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:45.574375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.199865+0000","last_change":"2024-04-25T08:39:07.193944+0000","last_active":"2024-04-25T08:39:09.199865+0000","last_peered":"2024-04-25T08:39:09.199865+0000","last_clean":"2024-04-25T08:39:09.199865+0000","last_became_active":"2024-04-25T08:39:07.193625+0000","last_became_peered":"2024-04-25T08:39:07.193625+0000","last_unstale":"2024-04-25T08:39:09.199865+0000","last_undegraded":"2024-04-25T08:39:09.199865+0000","last_fullsized":"2024-04-25T08:39:09.199865+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:54.429666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197258+0000","last_change":"2024-04-25T08:39:08.218384+0000","last_active":"2024-04-25T08:39:09.197258+0000","last_peered":"2024-04-25T08:39:09.197258+0000","last_clean":"2024-04-25T08:39:09.197258+0000","last_became_active":"2024-04-25T08:39:08.216943+0000","last_became_peered":"2024-04-25T08:39:08.216943+0000","last_unstale":"2024-04-25T08:39:09.197258+0000","last_undegraded":"2024-04-25T08:39:09.197258+0000","last_fullsized":"2024-04-25T08:39:09.197258+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:51.568863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197178+0000","last_change":"2024-04-25T08:39:07.198072+0000","last_active":"2024-04-25T08:39:09.197178+0000","last_peered":"2024-04-25T08:39:09.197178+0000","last_clean":"2024-04-25T08:39:09.197178+0000","last_became_active":"2024-04-25T08:39:07.197770+0000","last_became_peered":"2024-04-25T08:39:07.197770+0000","last_unstale":"2024-04-25T08:39:09.197178+0000","last_undegraded":"2024-04-25T08:39:09.197178+0000","last_fullsized":"2024-04-25T08:39:09.197178+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:38.367889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270978+0000","last_change":"2024-04-25T08:39:08.204548+0000","last_active":"2024-04-25T08:39:09.270978+0000","last_peered":"2024-04-25T08:39:09.270978+0000","last_clean":"2024-04-25T08:39:09.270978+0000","last_became_active":"2024-04-25T08:39:08.204225+0000","last_became_peered":"2024-04-25T08:39:08.204225+0000","last_unstale":"2024-04-25T08:39:09.270978+0000","last_undegraded":"2024-04-25T08:39:09.270978+0000","last_fullsized":"2024-04-25T08:39:09.270978+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:41.651136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.201206+0000","last_change":"2024-04-25T08:39:07.188807+0000","last_active":"2024-04-25T08:39:09.201206+0000","last_peered":"2024-04-25T08:39:09.201206+0000","last_clean":"2024-04-25T08:39:09.201206+0000","last_became_active":"2024-04-25T08:39:07.188447+0000","last_became_peered":"2024-04-25T08:39:07.188447+0000","last_unstale":"2024-04-25T08:39:09.201206+0000","last_undegraded":"2024-04-25T08:39:09.201206+0000","last_fullsized":"2024-04-25T08:39:09.201206+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:54.429666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.212688+0000","last_change":"2024-04-25T08:39:08.215462+0000","last_active":"2024-04-25T08:39:09.212688+0000","last_peered":"2024-04-25T08:39:09.212688+0000","last_clean":"2024-04-25T08:39:09.212688+0000","last_became_active":"2024-04-25T08:39:08.214764+0000","last_became_peered":"2024-04-25T08:39:08.214764+0000","last_unstale":"2024-04-25T08:39:09.212688+0000","last_undegraded":"2024-04-25T08:39:09.212688+0000","last_fullsized":"2024-04-25T08:39:09.212688+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:08.964337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197380+0000","last_change":"2024-04-25T08:39:07.196369+0000","last_active":"2024-04-25T08:39:09.197380+0000","last_peered":"2024-04-25T08:39:09.197380+0000","last_clean":"2024-04-25T08:39:09.197380+0000","last_became_active":"2024-04-25T08:39:07.196073+0000","last_became_peered":"2024-04-25T08:39:07.196073+0000","last_unstale":"2024-04-25T08:39:09.197380+0000","last_undegraded":"2024-04-25T08:39:09.197380+0000","last_fullsized":"2024-04-25T08:39:09.197380+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:27.661136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.212348+0000","last_change":"2024-04-25T08:39:08.208526+0000","last_active":"2024-04-25T08:39:09.212348+0000","last_peered":"2024-04-25T08:39:09.212348+0000","last_clean":"2024-04-25T08:39:09.212348+0000","last_became_active":"2024-04-25T08:39:08.208239+0000","last_became_peered":"2024-04-25T08:39:08.208239+0000","last_unstale":"2024-04-25T08:39:09.212348+0000","last_undegraded":"2024-04-25T08:39:09.212348+0000","last_fullsized":"2024-04-25T08:39:09.212348+0000","mapping_epoch":106,"log_start":"0'0","ondisk_log_start":"0'0","created":106,"last_epoch_clean":107,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:07.174694+0000","last_clean_scrub_stamp":"2024-04-25T08:39:07.174694+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:07.186368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"107'2","reported_seq":16,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.197473+0000","last_change":"2024-04-25T08:39:07.200791+0000","last_active":"2024-04-25T08:39:09.197473+0000","last_peered":"2024-04-25T08:39:09.197473+0000","last_clean":"2024-04-25T08:39:09.197473+0000","last_became_active":"2024-04-25T08:39:07.200495+0000","last_became_peered":"2024-04-25T08:39:07.200495+0000","last_unstale":"2024-04-25T08:39:09.197473+0000","last_undegraded":"2024-04-25T08:39:09.197473+0000","last_fullsized":"2024-04-25T08:39:09.197473+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:06.169945+0000","last_clean_scrub_stamp":"2024-04-25T08:39:06.169945+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:57.260151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":336,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-04-25T08:39:09.270713+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:39:09.270713+0000","last_peered":"2024-04-25T08:39:09.270713+0000","last_clean":"2024-04-25T08:39:09.270713+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:39:09.270713+0000","last_undegraded":"2024-04-25T08:39:09.270713+0000","last_fullsized":"2024-04-25T08:39:09.270713+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574946,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77492,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294348,"statfs":{"total":96636764160,"available":96557412352,"internally_reserved":0,"allocated":1941504,"data_stored":1213878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574946,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58876,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312964,"statfs":{"total":96636764160,"available":96576475136,"internally_reserved":0,"allocated":1359872,"data_stored":625566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54700,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317140,"statfs":{"total":96636764160,"available":96580751360,"internally_reserved":0,"allocated":1339392,"data_stored":623350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68592,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303248,"statfs":{"total":96636764160,"available":96566525952,"internally_reserved":0,"allocated":1347584,"data_stored":623442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68588,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303252,"statfs":{"total":96636764160,"available":96566530048,"internally_reserved":0,"allocated":1343488,"data_stored":623932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574946,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58864,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312976,"statfs":{"total":96636764160,"available":96576487424,"internally_reserved":0,"allocated":1343488,"data_stored":623384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54700,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317140,"statfs":{"total":96636764160,"available":96580751360,"internally_reserved":0,"allocated":1343488,"data_stored":623384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574946,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46972,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324868,"statfs":{"total":96636764160,"available":96588664832,"internally_reserved":0,"allocated":1949696,"data_stored":1215420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:39:15.962 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:39:15.962 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:39:16.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.112+0000 7fc8e3eb3640 1 Processor -- start 2024-04-25T08:39:16.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.112+0000 7fc8e3eb3640 1 -- start start 2024-04-25T08:39:16.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.113+0000 7fc8e3eb3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8dc160f20 0x7fc8dc0b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.113+0000 7fc8e3eb3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc8dc0ce5c0 0x7fc8dc0cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.113+0000 7fc8e3eb3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8dc0d1d70 con 0x7fc8dc0d1930 2024-04-25T08:39:16.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.113+0000 7fc8e3eb3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8dc161300 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.113+0000 7fc8e3eb3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0cef90 con 0x7fc8dc160f20 2024-04-25T08:39:16.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.113+0000 7fc8e2450640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc8dc0ce5c0 0x7fc8dc0cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.113+0000 7fc8e144e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc8dc0d1930 0x7fc8dc0b9250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43618/0 (socket says 172.21.15.38:43618) 2024-04-25T08:39:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.113+0000 7fc8e2450640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc8dc0ce5c0 0x7fc8dc0cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45086/0 (socket says 172.21.15.38:45086) 2024-04-25T08:39:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.113+0000 7fc8e144e640 1 -- 172.21.15.38:0/342668667 learned_addr learned my addr 172.21.15.38:0/342668667 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.114+0000 7fc8e1c4f640 1 --2- 172.21.15.38:0/342668667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8dc160f20 0x7fc8dc0b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.114+0000 7fc8e2450640 1 -- 172.21.15.38:0/342668667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8dc160f20 msgr2=0x7fc8dc0b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.114+0000 7fc8e2450640 1 --2- 172.21.15.38:0/342668667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8dc160f20 0x7fc8dc0b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.114+0000 7fc8e2450640 1 -- 172.21.15.38:0/342668667 >> v1:172.21.15.38:6789/0 conn(0x7fc8dc0d1930 legacy=0x7fc8dc0b9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.114+0000 7fc8e2450640 1 -- 172.21.15.38:0/342668667 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc0b7040 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.114+0000 7fc8e0c4d640 1 -- 172.21.15.38:0/342668667 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828101961 0 0) 0x7fc8dc0d1d70 con 0x7fc8dc0d1930 2024-04-25T08:39:16.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.114+0000 7fc8e1c4f640 1 --2- 172.21.15.38:0/342668667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8dc160f20 0x7fc8dc0b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:16.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.114+0000 7fc8e2450640 1 --2- 172.21.15.38:0/342668667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8dc0ce5c0 0x7fc8dc0cea50 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009d90 tx=0x7fc8cc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c42973fbea7e454 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:16.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.114+0000 7fc8e0c4d640 1 -- 172.21.15.38:0/342668667 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc007740 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.114+0000 7fc8e0c4d640 1 -- 172.21.15.38:0/342668667 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8cc0078e0 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.114+0000 7fc8e0c4d640 1 -- 172.21.15.38:0/342668667 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc007cb0 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.115+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/342668667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8dc0ce5c0 msgr2=0x7fc8dc0cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.115+0000 7fc8e3eb3640 1 --2- 172.21.15.38:0/342668667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8dc0ce5c0 0x7fc8dc0cea50 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009d90 tx=0x7fc8cc02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.115+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/342668667 shutdown_connections 2024-04-25T08:39:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.115+0000 7fc8e3eb3640 1 --2- 172.21.15.38:0/342668667 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8dc0ce5c0 0x7fc8dc0cea50 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.115+0000 7fc8e3eb3640 1 --2- 172.21.15.38:0/342668667 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8dc160f20 0x7fc8dc0b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.115+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/342668667 >> 172.21.15.38:0/342668667 conn(0x7fc8dc066ce0 msgr2=0x7fc8dc0670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.115+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/342668667 shutdown_connections 2024-04-25T08:39:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.115+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/342668667 wait complete. 2024-04-25T08:39:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.116+0000 7fc8e3eb3640 1 Processor -- start 2024-04-25T08:39:16.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.116+0000 7fc8e3eb3640 1 -- start start 2024-04-25T08:39:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e3eb3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8dc0ce5c0 0x7fc8dc147d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e3eb3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8dc0d1930 0x7fc8dc0d31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e3eb3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8dc160f20 0x7fc8dc0d3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e3eb3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0b78e0 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e3eb3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0b7600 con 0x7fc8dc160f20 2024-04-25T08:39:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e3eb3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc0b7bc0 con 0x7fc8dc0d1930 2024-04-25T08:39:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e1c4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8dc0ce5c0 0x7fc8dc147d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e1c4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8dc0ce5c0 0x7fc8dc147d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45102/0 (socket says 172.21.15.38:45102) 2024-04-25T08:39:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e1c4f640 1 -- 172.21.15.38:0/2197878453 learned_addr learned my addr 172.21.15.38:0/2197878453 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e144e640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8dc0d1930 0x7fc8dc0d31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e2450640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8dc160f20 0x7fc8dc0d3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e1c4f640 1 -- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8dc0d1930 msgr2=0x7fc8dc0d31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e1c4f640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8dc0d1930 0x7fc8dc0d31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e1c4f640 1 -- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8dc160f20 msgr2=0x7fc8dc0d3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e1c4f640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8dc160f20 0x7fc8dc0d3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e1c4f640 1 -- 172.21.15.38:0/2197878453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc15cc10 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e144e640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8dc0d1930 0x7fc8dc0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.117+0000 7fc8e2450640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8dc160f20 0x7fc8dc0d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.118+0000 7fc8e1c4f640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8dc0ce5c0 0x7fc8dc147d20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c7f0 tx=0x7fc8d400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.118+0000 7fc8da7fc640 1 -- 172.21.15.38:0/2197878453 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4015070 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.118+0000 7fc8da7fc640 1 -- 172.21.15.38:0/2197878453 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4007820 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.118+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/2197878453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8dc15cef0 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.118+0000 7fc8da7fc640 1 -- 172.21.15.38:0/2197878453 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4007bd0 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.118+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/2197878453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8dc0cb0e0 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.119+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/2197878453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8dc068e80 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.119+0000 7fc8da7fc640 1 -- 172.21.15.38:0/2197878453 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8d4018070 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.120+0000 7fc8da7fc640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8c806b500 0x7fc8c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.120+0000 7fc8da7fc640 1 -- 172.21.15.38:0/2197878453 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc8d408a210 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.120+0000 7fc8e144e640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8c806b500 0x7fc8c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.120+0000 7fc8e144e640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8c806b500 0x7fc8c806d9c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0c8fa0 tx=0x7fc8d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:16.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.124+0000 7fc8da7fc640 1 -- 172.21.15.38:0/2197878453 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8d4058f90 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.382+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/2197878453 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc8dc161f40 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.384+0000 7fc8da7fc640 1 -- 172.21.15.38:0/2197878453 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc8d4058890 con 0x7fc8dc0ce5c0 2024-04-25T08:39:16.377 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-04-25T08:39:16.377 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw== 2024-04-25T08:39:16.377 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-04-25T08:39:16.378 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-04-25T08:39:16.378 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:39:16.378 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-04-25T08:39:16.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.386+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8c806b500 msgr2=0x7fc8c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.386+0000 7fc8e3eb3640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8c806b500 0x7fc8c806d9c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0c8fa0 tx=0x7fc8d0009040 comp rx=0 tx=0).stop 2024-04-25T08:39:16.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.386+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/2197878453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8dc0ce5c0 msgr2=0x7fc8dc147d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.386+0000 7fc8e3eb3640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8dc0ce5c0 0x7fc8dc147d20 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c7f0 tx=0x7fc8d400ccb0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.386+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/2197878453 shutdown_connections 2024-04-25T08:39:16.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.386+0000 7fc8e3eb3640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8dc160f20 0x7fc8dc0d3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.386+0000 7fc8e3eb3640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8c806b500 0x7fc8c806d9c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.386+0000 7fc8e3eb3640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8dc0d1930 0x7fc8dc0d31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.386+0000 7fc8e3eb3640 1 --2- 172.21.15.38:0/2197878453 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8dc0ce5c0 0x7fc8dc147d20 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.386+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/2197878453 >> 172.21.15.38:0/2197878453 conn(0x7fc8dc066ce0 msgr2=0x7fc8dc161490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:16.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.386+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/2197878453 shutdown_connections 2024-04-25T08:39:16.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.386+0000 7fc8e3eb3640 1 -- 172.21.15.38:0/2197878453 wait complete. 2024-04-25T08:39:16.391 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.498+0000 7ff425a74640 1 Processor -- start 2024-04-25T08:39:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.498+0000 7ff425a74640 1 -- start start 2024-04-25T08:39:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.499+0000 7ff425a74640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4200d0f60 0x7ff4200b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.499+0000 7ff425a74640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4200d3e70 con 0x7ff42015dc10 2024-04-25T08:39:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.499+0000 7ff425a74640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4200d4150 con 0x7ff4200d3900 2024-04-25T08:39:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.499+0000 7ff425a74640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42015dff0 con 0x7ff4200d0f60 2024-04-25T08:39:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.499+0000 7ff41f7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff42015dc10 0x7ff4200ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43624/0 (socket says 172.21.15.38:43624) 2024-04-25T08:39:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.499+0000 7ff41f7fe640 1 -- 172.21.15.38:0/1330413727 learned_addr learned my addr 172.21.15.38:0/1330413727 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41ffff640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4200d0f60 0x7ff4200b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826471523 0 0) 0x7ff4200d3e70 con 0x7ff42015dc10 2024-04-25T08:39:16.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff408003520 con 0x7ff42015dc10 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 911150835 0 0) 0x7ff4200d4150 con 0x7ff4200d3900 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4200d3e70 con 0x7ff4200d3900 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 922810266 0 0) 0x7ff408003520 con 0x7ff42015dc10 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4200d0f60 msgr2=0x7ff4200b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4200d0f60 0x7ff4200b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 >> v1:172.21.15.107:6789/0 conn(0x7ff4200d3900 legacy=0x7ff4200d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200d4430 con 0x7ff42015dc10 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7ff410001690 con 0x7ff42015dc10 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41ffff640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4200d0f60 0x7ff4200b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 >> v1:172.21.15.38:6789/0 conn(0x7ff42015dc10 legacy=0x7ff4200ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff408003b70 0x7ff408003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4200d0f60 0x7ff408004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff408004d40 0x7ff4080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42015dff0 con 0x7ff408004d40 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40800bad0 con 0x7ff408003b70 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff40800bcd0 con 0x7ff4200d0f60 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41ffff640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff408004d40 0x7ff4080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41f7fe640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff408003b70 0x7ff408003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.500+0000 7ff41effd640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4200d0f60 0x7ff408004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.501+0000 7ff41ffff640 1 -- 172.21.15.38:0/1330413727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4200d0f60 msgr2=0x7ff408004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.501+0000 7ff41ffff640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff4200d0f60 0x7ff408004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.501+0000 7ff41ffff640 1 -- 172.21.15.38:0/1330413727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff408003b70 msgr2=0x7ff408003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.501+0000 7ff41ffff640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff408003b70 0x7ff408003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.501+0000 7ff41ffff640 1 -- 172.21.15.38:0/1330413727 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff408003520 con 0x7ff408004d40 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.501+0000 7ff41ffff640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff408004d40 0x7ff4080051c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7ff42008cc80 tx=0x7ff408005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d342143de253fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:16.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.501+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418033070 con 0x7ff408004d40 2024-04-25T08:39:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.501+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41802f620 con 0x7ff408004d40 2024-04-25T08:39:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.501+0000 7ff41e7fc640 1 -- 172.21.15.38:0/1330413727 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418009de0 con 0x7ff408004d40 2024-04-25T08:39:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.502+0000 7ff425a74640 1 -- 172.21.15.38:0/1330413727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff408004d40 msgr2=0x7ff4080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.502+0000 7ff425a74640 1 --2- 172.21.15.38:0/1330413727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff408004d40 0x7ff4080051c0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7ff42008cc80 tx=0x7ff408005830 comp rx=0 tx=0).stop 2024-04-25T08:39:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.502+0000 7ff41ffff640 1 -- 172.21.15.38:0/1330413727 reap_dead start 2024-04-25T08:39:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.502+0000 7ff425a74640 1 -- 172.21.15.38:0/1330413727 shutdown_connections 2024-04-25T08:39:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.502+0000 7ff425a74640 1 -- 172.21.15.38:0/1330413727 >> 172.21.15.38:0/1330413727 conn(0x7ff420066d10 msgr2=0x7ff420067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.502+0000 7ff425a74640 1 -- 172.21.15.38:0/1330413727 shutdown_connections 2024-04-25T08:39:16.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.502+0000 7ff425a74640 1 -- 172.21.15.38:0/1330413727 wait complete. 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.503+0000 7ff425a74640 1 Processor -- start 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.503+0000 7ff425a74640 1 -- start start 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff425a74640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420159780 0x7ff4201e9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff425a74640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff420159c10 0x7ff4201e9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff425a74640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff42015a140 0x7ff42015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff425a74640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200d4710 con 0x7ff42015a140 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff425a74640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200cef30 con 0x7ff420159780 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff425a74640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200b80f0 con 0x7ff420159c10 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff41ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff42015a140 0x7ff42015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff41ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff42015a140 0x7ff42015b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45116/0 (socket says 172.21.15.38:45116) 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff41ffff640 1 -- 172.21.15.38:0/3239372023 learned_addr learned my addr 172.21.15.38:0/3239372023 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff41ffff640 1 -- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff420159c10 msgr2=0x7ff4201e9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff41ffff640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff420159c10 0x7ff4201e9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff41ffff640 1 -- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420159780 msgr2=0x7ff4201e9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.504+0000 7ff41f7fe640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420159780 0x7ff4201e9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.505+0000 7ff41ffff640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420159780 0x7ff4201e9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.505+0000 7ff41ffff640 1 -- 172.21.15.38:0/3239372023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4201f3c90 con 0x7ff42015a140 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.505+0000 7ff41ffff640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff42015a140 0x7ff42015b690 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff41802b000 tx=0x7ff418009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.505+0000 7ff407fff640 1 -- 172.21.15.38:0/3239372023 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418033040 con 0x7ff42015a140 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.505+0000 7ff407fff640 1 -- 172.21.15.38:0/3239372023 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff418009a50 con 0x7ff42015a140 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.505+0000 7ff425a74640 1 -- 172.21.15.38:0/3239372023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4201f3f70 con 0x7ff42015a140 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.505+0000 7ff407fff640 1 -- 172.21.15.38:0/3239372023 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418009de0 con 0x7ff42015a140 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.505+0000 7ff425a74640 1 -- 172.21.15.38:0/3239372023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4201f4530 con 0x7ff42015a140 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.506+0000 7ff425a74640 1 -- 172.21.15.38:0/3239372023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff420068e80 con 0x7ff42015a140 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.506+0000 7ff407fff640 1 -- 172.21.15.38:0/3239372023 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff418036070 con 0x7ff42015a140 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.507+0000 7ff407fff640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff40c06b500 0x7ff40c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.507+0000 7ff407fff640 1 -- 172.21.15.38:0/3239372023 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4180aa670 con 0x7ff42015a140 2024-04-25T08:39:16.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.507+0000 7ff41f7fe640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff40c06b500 0x7ff40c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:16.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.507+0000 7ff41f7fe640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff40c06b500 0x7ff40c06d9c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff4201ea230 tx=0x7ff410009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:16.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.511+0000 7ff407fff640 1 -- 172.21.15.38:0/3239372023 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff418048060 con 0x7ff42015a140 2024-04-25T08:39:16.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.728+0000 7ff425a74640 1 -- 172.21.15.38:0/3239372023 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4200c76e0 con 0x7ff42015a140 2024-04-25T08:39:16.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.729+0000 7ff407fff640 1 -- 172.21.15.38:0/3239372023 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4946 (secure 0 0 0) 0x7ff4180792d0 con 0x7ff42015a140 2024-04-25T08:39:16.722 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:16.723 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:07.551641+0000","modified":"2024-04-25T08:39:09.555958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8292":{"gid":8292,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1263010516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1263010516},{"type":"v1","addr":"172.21.15.107:6837","nonce":1263010516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:39:16.723 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 63 2024-04-25T08:39:16.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.730+0000 7ff425a74640 1 -- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff40c06b500 msgr2=0x7ff40c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.730+0000 7ff425a74640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff40c06b500 0x7ff40c06d9c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff4201ea230 tx=0x7ff410009040 comp rx=0 tx=0).stop 2024-04-25T08:39:16.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.730+0000 7ff425a74640 1 -- 172.21.15.38:0/3239372023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff42015a140 msgr2=0x7ff42015b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:16.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.730+0000 7ff425a74640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff42015a140 0x7ff42015b690 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff41802b000 tx=0x7ff418009750 comp rx=0 tx=0).stop 2024-04-25T08:39:16.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.730+0000 7ff425a74640 1 -- 172.21.15.38:0/3239372023 shutdown_connections 2024-04-25T08:39:16.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.730+0000 7ff425a74640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff42015a140 0x7ff42015b690 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.730+0000 7ff425a74640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff40c06b500 0x7ff40c06d9c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.730+0000 7ff425a74640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff420159c10 0x7ff4201e9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.730+0000 7ff425a74640 1 --2- 172.21.15.38:0/3239372023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff420159780 0x7ff4201e9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:16.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.731+0000 7ff425a74640 1 -- 172.21.15.38:0/3239372023 >> 172.21.15.38:0/3239372023 conn(0x7ff420066d10 msgr2=0x7ff42006bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:16.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.731+0000 7ff425a74640 1 -- 172.21.15.38:0/3239372023 shutdown_connections 2024-04-25T08:39:16.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:16.731+0000 7ff425a74640 1 -- 172.21.15.38:0/3239372023 wait complete. 2024-04-25T08:39:17.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:39:07.551641+0000', 'modified': '2024-04-25T08:39:09.555958+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8292}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8292': {'gid': 8292, 'name': 'd', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.107:6837/1263010516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6835', 'nonce': 1263010516}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 1263010516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:39:17.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:39:17.138 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T08:39:17.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.250+0000 7f0989df4640 1 Processor -- start 2024-04-25T08:39:17.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.250+0000 7f0989df4640 1 -- start start 2024-04-25T08:39:17.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.251+0000 7f0989df4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0984071690 0x7f098415ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.251+0000 7f0989df4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09840b2430 con 0x7f09840ceba0 2024-04-25T08:39:17.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.251+0000 7f0989df4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0984071d70 con 0x7f098415a600 2024-04-25T08:39:17.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.251+0000 7f0989df4640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09840b26d0 con 0x7f0984071690 2024-04-25T08:39:17.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.251+0000 7f0982ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0984071690 0x7f098415ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.251+0000 7f0983fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f09840ceba0 0x7f098406d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:43634/0 (socket says 172.21.15.38:43634) 2024-04-25T08:39:17.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.251+0000 7f0983fff640 1 -- 172.21.15.38:0/1833418351 learned_addr learned my addr 172.21.15.38:0/1833418351 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:17.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.252+0000 7f0982ffd640 1 -- 172.21.15.38:0/1833418351 >> v1:172.21.15.107:6789/0 conn(0x7f098415a600 legacy=0x7f0984070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.252+0000 7f0982ffd640 1 -- 172.21.15.38:0/1833418351 >> v1:172.21.15.38:6789/0 conn(0x7f09840ceba0 legacy=0x7f098406d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.252+0000 7f09827fc640 1 -- 172.21.15.38:0/1833418351 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834739059 0 0) 0x7f09840b2430 con 0x7f09840ceba0 2024-04-25T08:39:17.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.252+0000 7f0982ffd640 1 -- 172.21.15.38:0/1833418351 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09840b29b0 con 0x7f0984071690 2024-04-25T08:39:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.252+0000 7f0982ffd640 1 --2- 172.21.15.38:0/1833418351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0984071690 0x7f098415ebd0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f09780097a0 tx=0x7f097802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc905a4fa0284ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.252+0000 7f09827fc640 1 -- 172.21.15.38:0/1833418351 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978037070 con 0x7f0984071690 2024-04-25T08:39:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.253+0000 7f09827fc640 1 -- 172.21.15.38:0/1833418351 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0978032440 con 0x7f0984071690 2024-04-25T08:39:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.253+0000 7f09827fc640 1 -- 172.21.15.38:0/1833418351 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978032810 con 0x7f0984071690 2024-04-25T08:39:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.253+0000 7f0989df4640 1 -- 172.21.15.38:0/1833418351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0984071690 msgr2=0x7f098415ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.253+0000 7f0989df4640 1 --2- 172.21.15.38:0/1833418351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0984071690 0x7f098415ebd0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f09780097a0 tx=0x7f097802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.253+0000 7f0989df4640 1 -- 172.21.15.38:0/1833418351 shutdown_connections 2024-04-25T08:39:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.253+0000 7f0989df4640 1 --2- 172.21.15.38:0/1833418351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0984071690 0x7f098415ebd0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.253+0000 7f0989df4640 1 -- 172.21.15.38:0/1833418351 >> 172.21.15.38:0/1833418351 conn(0x7f098406abe0 msgr2=0x7f098406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.253+0000 7f0989df4640 1 -- 172.21.15.38:0/1833418351 shutdown_connections 2024-04-25T08:39:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.253+0000 7f0989df4640 1 -- 172.21.15.38:0/1833418351 wait complete. 2024-04-25T08:39:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.254+0000 7f0989df4640 1 Processor -- start 2024-04-25T08:39:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.254+0000 7f0989df4640 1 -- start start 2024-04-25T08:39:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.255+0000 7f0989df4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0984071690 0x7f0984151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.255+0000 7f0989df4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09840ceba0 0x7f0984152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.255+0000 7f0989df4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f098415a600 0x7f0984157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.255+0000 7f0989df4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09840b3600 con 0x7f09840ceba0 2024-04-25T08:39:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.255+0000 7f0989df4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098415a9e0 con 0x7f098415a600 2024-04-25T08:39:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.255+0000 7f0989df4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0984153410 con 0x7f0984071690 2024-04-25T08:39:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.255+0000 7f0982ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09840ceba0 0x7f0984152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.255+0000 7f0982ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09840ceba0 0x7f0984152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45146/0 (socket says 172.21.15.38:45146) 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.255+0000 7f0982ffd640 1 -- 172.21.15.38:0/1151919399 learned_addr learned my addr 172.21.15.38:0/1151919399 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f0983fff640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f098415a600 0x7f0984157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f0982ffd640 1 -- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0984071690 msgr2=0x7f0984151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f09837fe640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0984071690 0x7f0984151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f0982ffd640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0984071690 0x7f0984151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f0982ffd640 1 -- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f098415a600 msgr2=0x7f0984157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f0982ffd640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f098415a600 0x7f0984157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f0982ffd640 1 -- 172.21.15.38:0/1151919399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09840aca80 con 0x7f09840ceba0 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f0983fff640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f098415a600 0x7f0984157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f09837fe640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0984071690 0x7f0984151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f0982ffd640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09840ceba0 0x7f0984152ed0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f09780098c0 tx=0x7f09780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f096bfff640 1 -- 172.21.15.38:0/1151919399 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978041070 con 0x7f09840ceba0 2024-04-25T08:39:17.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f096bfff640 1 -- 172.21.15.38:0/1151919399 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097803c870 con 0x7f09840ceba0 2024-04-25T08:39:17.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f0989df4640 1 -- 172.21.15.38:0/1151919399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09840acd60 con 0x7f09840ceba0 2024-04-25T08:39:17.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f096bfff640 1 -- 172.21.15.38:0/1151919399 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978037040 con 0x7f09840ceba0 2024-04-25T08:39:17.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.256+0000 7f0989df4640 1 -- 172.21.15.38:0/1151919399 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0984150680 con 0x7f09840ceba0 2024-04-25T08:39:17.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.257+0000 7f096bfff640 1 -- 172.21.15.38:0/1151919399 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0978044070 con 0x7f09840ceba0 2024-04-25T08:39:17.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.258+0000 7f096bfff640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f097006b4d0 0x7f097006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.258+0000 7f096bfff640 1 -- 172.21.15.38:0/1151919399 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09780aafe0 con 0x7f09840ceba0 2024-04-25T08:39:17.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.258+0000 7f09837fe640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f097006b4d0 0x7f097006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.259+0000 7f09837fe640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f097006b4d0 0x7f097006d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0974003eb0 tx=0x7f0974009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.259+0000 7f098cbcc740 1 Processor -- start 2024-04-25T08:39:17.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.259+0000 7f098cbcc740 1 -- start start 2024-04-25T08:39:17.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f098cbcc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 0x5584651b2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f098cbcc740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651cfad0 0x5584651d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f098cbcc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558465187f10 0x5584651ce190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f098cbcc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584650eaa10 con 0x5584651b2230 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f098cbcc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x558465093b30 con 0x558465187f10 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f098cbcc740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584650e6c50 con 0x5584651cfad0 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f09837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 0x5584651b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f09837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 0x5584651b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45150/0 (socket says 172.21.15.38:45150) 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f09837fe640 1 -- 172.21.15.38:0/550311933 learned_addr learned my addr 172.21.15.38:0/550311933 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f0983fff640 1 --2- 172.21.15.38:0/550311933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558465187f10 0x5584651ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f0982ffd640 1 --2- 172.21.15.38:0/550311933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651cfad0 0x5584651d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f09837fe640 1 -- 172.21.15.38:0/550311933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651cfad0 msgr2=0x5584651d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f09837fe640 1 --2- 172.21.15.38:0/550311933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651cfad0 0x5584651d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f09837fe640 1 -- 172.21.15.38:0/550311933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558465187f10 msgr2=0x5584651ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f09837fe640 1 --2- 172.21.15.38:0/550311933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558465187f10 0x5584651ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.260+0000 7f09837fe640 1 -- 172.21.15.38:0/550311933 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584650a8e10 con 0x5584651b2230 2024-04-25T08:39:17.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.261+0000 7f09837fe640 1 --2- 172.21.15.38:0/550311933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 0x5584651b2610 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f0974003c90 tx=0x7f0974010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755fc877f1471b54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.261+0000 7f09697fa640 1 -- 172.21.15.38:0/550311933 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974024070 con 0x5584651b2230 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.261+0000 7f09697fa640 1 -- 172.21.15.38:0/550311933 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0974014db0 con 0x5584651b2230 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.261+0000 7f09697fa640 1 -- 172.21.15.38:0/550311933 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097401f640 con 0x5584651b2230 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.261+0000 7f098cbcc740 1 -- 172.21.15.38:0/550311933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 msgr2=0x5584651b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.261+0000 7f098cbcc740 1 --2- 172.21.15.38:0/550311933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 0x5584651b2610 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f0974003c90 tx=0x7f0974010000 comp rx=0 tx=0).stop 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.262+0000 7f098cbcc740 1 -- 172.21.15.38:0/550311933 shutdown_connections 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.262+0000 7f098cbcc740 1 --2- 172.21.15.38:0/550311933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558465187f10 0x5584651ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.262+0000 7f098cbcc740 1 --2- 172.21.15.38:0/550311933 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651cfad0 0x5584651d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.262+0000 7f098cbcc740 1 --2- 172.21.15.38:0/550311933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 0x5584651b2610 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.262+0000 7f098cbcc740 1 -- 172.21.15.38:0/550311933 >> 172.21.15.38:0/550311933 conn(0x558465184840 msgr2=0x5584651cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.262+0000 7f098cbcc740 1 -- 172.21.15.38:0/550311933 shutdown_connections 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.262+0000 7f098cbcc740 1 -- 172.21.15.38:0/550311933 wait complete. 2024-04-25T08:39:17.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.262+0000 7f098cbcc740 1 -- start start 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.262+0000 7f098cbcc740 1 Processor -- start 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f098cbcc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651b2230 0x5584651af180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f098cbcc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651cfad0 0x5584651b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f098cbcc740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651de0f0 0x5584651cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f098cbcc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x558465188cb0 con 0x5584651cfad0 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f098cbcc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584650a70c0 con 0x5584651b2230 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f098cbcc740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x558465182770 con 0x5584651de0f0 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f098cbcc740 10 client.0 _ll_register_callbacks cb 0x55846509aec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f0982ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651cfad0 0x5584651b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f0982ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651cfad0 0x5584651b5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45158/0 (socket says 172.21.15.38:45158) 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f0982ffd640 1 -- 172.21.15.38:0/1854914832 learned_addr learned my addr 172.21.15.38:0/1854914832 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f0983fff640 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651de0f0 0x5584651cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f09837fe640 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651b2230 0x5584651af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f0982ffd640 1 -- 172.21.15.38:0/1854914832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651de0f0 msgr2=0x5584651cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f0982ffd640 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651de0f0 0x5584651cc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f0982ffd640 1 -- 172.21.15.38:0/1854914832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651b2230 msgr2=0x5584651af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f0982ffd640 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651b2230 0x5584651af180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.263+0000 7f0982ffd640 1 -- 172.21.15.38:0/1854914832 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558464dff2d0 con 0x5584651cfad0 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f0982ffd640 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651cfad0 0x5584651b5280 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f09780a93b0 tx=0x7f0978079f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ad1ecd84d06597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f09627fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f09627fc640 1 -- 172.21.15.38:0/1854914832 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097804b040 con 0x5584651cfad0 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f09627fc640 1 -- 172.21.15.38:0/1854914832 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0978075df0 con 0x5584651cfad0 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f098cbcc740 1 -- 172.21.15.38:0/1854914832 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0974014db0 con 0x5584651cfad0 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f09627fc640 1 -- 172.21.15.38:0/1854914832 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09780a9050 con 0x5584651cfad0 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f098cbcc740 10 client.8463 fetch_fsmap learned FSMap version 63 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f098cbcc740 1 -- 172.21.15.38:0/1854914832 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x558464ec96b0 con 0x5584651cfad0 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f09627fc640 1 -- 172.21.15.38:0/1854914832 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0974014db0 con 0x5584651cfad0 2024-04-25T08:39:17.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f09627fc640 1 -- 172.21.15.38:0/1854914832 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f097803cc60 con 0x5584651cfad0 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f098cbcc740 10 client.8463 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f098cbcc740 10 client.8463 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f098cbcc740 20 client.8463 populate_metadata read hostname 'smithi038' 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f098cbcc740 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x5584651e0ee0 0x5584651e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f098cbcc740 4 client.8463 mds_command: new command op to 8292 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f098cbcc740 1 -- 172.21.15.38:0/1854914832 --> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558465149920 con 0x5584651e0ee0 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.264+0000 7f09837fe640 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x5584651e0ee0 0x5584651e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.265+0000 7f09627fc640 10 client.8463 ms_handle_connect on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.265+0000 7f09837fe640 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x5584651e0ee0 0x5584651e12c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aa0a9e2e961a82d server_cookie=394b109679308d12 in_seq=0 out_seq=0 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f09627fc640 1 -- 172.21.15.38:0/1854914832 <== mds.0 v2:172.21.15.107:6835/1263010516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558465149920 con 0x5584651e0ee0 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f09627fc640 10 client.8463 handle_command_reply: tid=0 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f098cbcc740 1 client.8463 shutdown 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f098cbcc740 1 -- 172.21.15.38:0/1854914832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651cfad0 msgr2=0x5584651b5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f098cbcc740 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651cfad0 0x5584651b5280 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f09780a93b0 tx=0x7f0978079f40 comp rx=0 tx=0).stop 2024-04-25T08:39:17.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f098cbcc740 1 -- 172.21.15.38:0/1854914832 shutdown_connections 2024-04-25T08:39:17.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f098cbcc740 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x5584651e0ee0 0x5584651e12c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f098cbcc740 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651de0f0 0x5584651cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f09627fc640 0 client.8463 ms_handle_reset on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:17.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f098cbcc740 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651cfad0 0x5584651b5280 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f098cbcc740 1 --2- 172.21.15.38:0/1854914832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651b2230 0x5584651af180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f098cbcc740 1 -- 172.21.15.38:0/1854914832 >> 172.21.15.38:0/1854914832 conn(0x558465187f10 msgr2=0x5584651d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:17.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f098cbcc740 1 -- 172.21.15.38:0/1854914832 shutdown_connections 2024-04-25T08:39:17.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f098cbcc740 1 -- 172.21.15.38:0/1854914832 wait complete. 2024-04-25T08:39:17.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.266+0000 7f098cbcc740 20 client.8463 trim_cache size 0 max 16384 2024-04-25T08:39:17.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.282+0000 7f098cbcc740 1 Processor -- start 2024-04-25T08:39:17.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.282+0000 7f098cbcc740 1 -- start start 2024-04-25T08:39:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f098cbcc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 0x5584651889e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f098cbcc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651cfad0 0x5584651af160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f098cbcc740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651de0f0 0x5584651dd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f098cbcc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09842001b0 con 0x5584651b2230 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f098cbcc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09841ff3b0 con 0x5584651cfad0 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f098cbcc740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09840b1b50 con 0x5584651de0f0 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f09837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 0x5584651889e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f09837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 0x5584651889e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45168/0 (socket says 172.21.15.38:45168) 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f09837fe640 1 -- 172.21.15.38:0/2170869679 learned_addr learned my addr 172.21.15.38:0/2170869679 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f0983fff640 1 --2- 172.21.15.38:0/2170869679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651de0f0 0x5584651dd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f0982ffd640 1 --2- 172.21.15.38:0/2170869679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651cfad0 0x5584651af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f09837fe640 1 -- 172.21.15.38:0/2170869679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651de0f0 msgr2=0x5584651dd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f09837fe640 1 --2- 172.21.15.38:0/2170869679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651de0f0 0x5584651dd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f09837fe640 1 -- 172.21.15.38:0/2170869679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651cfad0 msgr2=0x5584651af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f09837fe640 1 --2- 172.21.15.38:0/2170869679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651cfad0 0x5584651af160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f09837fe640 1 -- 172.21.15.38:0/2170869679 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584651b3780 con 0x5584651b2230 2024-04-25T08:39:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.283+0000 7f09837fe640 1 --2- 172.21.15.38:0/2170869679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 0x5584651889e0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f097400d520 tx=0x7f0974012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f967bc79ad72bb3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.284+0000 7f0968ff9640 1 -- 172.21.15.38:0/2170869679 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974024040 con 0x5584651b2230 2024-04-25T08:39:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.284+0000 7f0968ff9640 1 -- 172.21.15.38:0/2170869679 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0974007040 con 0x5584651b2230 2024-04-25T08:39:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.284+0000 7f0968ff9640 1 -- 172.21.15.38:0/2170869679 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974007430 con 0x5584651b2230 2024-04-25T08:39:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.284+0000 7f098cbcc740 1 -- 172.21.15.38:0/2170869679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 msgr2=0x5584651889e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.284+0000 7f098cbcc740 1 --2- 172.21.15.38:0/2170869679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 0x5584651889e0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f097400d520 tx=0x7f0974012000 comp rx=0 tx=0).stop 2024-04-25T08:39:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.284+0000 7f098cbcc740 1 -- 172.21.15.38:0/2170869679 shutdown_connections 2024-04-25T08:39:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.284+0000 7f098cbcc740 1 --2- 172.21.15.38:0/2170869679 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651de0f0 0x5584651dd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.284+0000 7f098cbcc740 1 --2- 172.21.15.38:0/2170869679 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651cfad0 0x5584651af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.284+0000 7f098cbcc740 1 --2- 172.21.15.38:0/2170869679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651b2230 0x5584651889e0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.284+0000 7f098cbcc740 1 -- 172.21.15.38:0/2170869679 >> 172.21.15.38:0/2170869679 conn(0x558465187f10 msgr2=0x55846517ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:17.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.284+0000 7f098cbcc740 1 -- 172.21.15.38:0/2170869679 shutdown_connections 2024-04-25T08:39:17.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.284+0000 7f098cbcc740 1 -- 172.21.15.38:0/2170869679 wait complete. 2024-04-25T08:39:17.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.285+0000 7f098cbcc740 1 -- start start 2024-04-25T08:39:17.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.285+0000 7f098cbcc740 1 Processor -- start 2024-04-25T08:39:17.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f098cbcc740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651b2230 0x5584651818a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f098cbcc740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651cfad0 0x5584651dab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f098cbcc740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651de0f0 0x5584651db0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f098cbcc740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584651aecf0 con 0x5584651de0f0 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f098cbcc740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09841fede0 con 0x5584651cfad0 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f098cbcc740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584651ae6a0 con 0x5584651b2230 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f098cbcc740 10 client.0 _ll_register_callbacks cb 0x558465182c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f0983fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651de0f0 0x5584651db0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f0983fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651de0f0 0x5584651db0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45184/0 (socket says 172.21.15.38:45184) 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f0983fff640 1 -- 172.21.15.38:0/1429684775 learned_addr learned my addr 172.21.15.38:0/1429684775 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f0982ffd640 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651cfad0 0x5584651dab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f09837fe640 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651b2230 0x5584651818a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f0983fff640 1 -- 172.21.15.38:0/1429684775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651b2230 msgr2=0x5584651818a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f0983fff640 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651b2230 0x5584651818a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f0983fff640 1 -- 172.21.15.38:0/1429684775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651cfad0 msgr2=0x5584651dab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f0983fff640 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651cfad0 0x5584651dab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f0983fff640 1 -- 172.21.15.38:0/1429684775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584651cc590 con 0x5584651de0f0 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f0983fff640 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651de0f0 0x5584651db0c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f097c004f00 tx=0x7f097c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6210b44ce518f072 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f09627fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.286+0000 7f09627fc640 1 -- 172.21.15.38:0/1429684775 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c019070 con 0x5584651de0f0 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f09627fc640 1 -- 172.21.15.38:0/1429684775 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f097c01aa50 con 0x5584651de0f0 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f098cbcc740 1 -- 172.21.15.38:0/1429684775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0974007040 con 0x5584651de0f0 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f09627fc640 1 -- 172.21.15.38:0/1429684775 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f097c025050 con 0x5584651de0f0 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f098cbcc740 10 client.8469 fetch_fsmap learned FSMap version 63 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f098cbcc740 1 -- 172.21.15.38:0/1429684775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x5584651c40f0 con 0x5584651de0f0 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f09627fc640 1 -- 172.21.15.38:0/1429684775 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0974007040 con 0x5584651de0f0 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f09627fc640 1 -- 172.21.15.38:0/1429684775 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f097c028020 con 0x5584651de0f0 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f098cbcc740 10 client.8469 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f098cbcc740 10 client.8469 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f098cbcc740 20 client.8469 populate_metadata read hostname 'smithi038' 2024-04-25T08:39:17.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f098cbcc740 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x5584651d05c0 0x5584651d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f098cbcc740 4 client.8469 mds_command: new command op to 8292 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f098cbcc740 1 -- 172.21.15.38:0/1429684775 --> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5584650f7f50 con 0x5584651d05c0 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.287+0000 7f09837fe640 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x5584651d05c0 0x5584651d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f09837fe640 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x5584651d05c0 0x5584651d09a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6eafbb01ebba049 server_cookie=575ddd8a498149c0 in_seq=0 out_seq=0 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f09627fc640 10 client.8469 ms_handle_connect on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f09627fc640 1 -- 172.21.15.38:0/1429684775 <== mds.0 v2:172.21.15.107:6835/1263010516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5584650f7f50 con 0x5584651d05c0 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f09627fc640 10 client.8469 handle_command_reply: tid=0 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f098cbcc740 1 client.8469 shutdown 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f098cbcc740 1 -- 172.21.15.38:0/1429684775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651de0f0 msgr2=0x5584651db0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f098cbcc740 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651de0f0 0x5584651db0c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f097c004f00 tx=0x7f097c007430 comp rx=0 tx=0).stop 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f098cbcc740 1 -- 172.21.15.38:0/1429684775 shutdown_connections 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f098cbcc740 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x5584651d05c0 0x5584651d09a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f098cbcc740 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5584651de0f0 0x5584651db0c0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f098cbcc740 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5584651cfad0 0x5584651dab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f09627fc640 0 client.8469 ms_handle_reset on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:17.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f098cbcc740 1 --2- 172.21.15.38:0/1429684775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5584651b2230 0x5584651818a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f098cbcc740 1 -- 172.21.15.38:0/1429684775 >> 172.21.15.38:0/1429684775 conn(0x558465187f10 msgr2=0x5584651dd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:17.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f098cbcc740 1 -- 172.21.15.38:0/1429684775 shutdown_connections 2024-04-25T08:39:17.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f098cbcc740 1 -- 172.21.15.38:0/1429684775 wait complete. 2024-04-25T08:39:17.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.288+0000 7f098cbcc740 20 client.8469 trim_cache size 0 max 16384 2024-04-25T08:39:17.290 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:39:17.290 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:39:17.290 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:39:17.290 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 8292, 2024-04-25T08:39:17.290 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:39:17.290 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:39:17.290 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:39:17.291 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 9.0852218419999993, 2024-04-25T08:39:17.291 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 63, 2024-04-25T08:39:17.291 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 108, 2024-04-25T08:39:17.291 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 108, 2024-04-25T08:39:17.291 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 14.133993909000001 2024-04-25T08:39:17.291 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:39:17.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.289+0000 7f0989df4640 1 -- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f097006b4d0 msgr2=0x7f097006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.289+0000 7f0989df4640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f097006b4d0 0x7f097006d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f0974003eb0 tx=0x7f0974009040 comp rx=0 tx=0).stop 2024-04-25T08:39:17.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.289+0000 7f0989df4640 1 -- 172.21.15.38:0/1151919399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09840ceba0 msgr2=0x7f0984152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.289+0000 7f0989df4640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09840ceba0 0x7f0984152ed0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f09780098c0 tx=0x7f09780046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.289+0000 7f0989df4640 1 -- 172.21.15.38:0/1151919399 shutdown_connections 2024-04-25T08:39:17.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.289+0000 7f0989df4640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f098415a600 0x7f0984157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.289+0000 7f0989df4640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f09840ceba0 0x7f0984152ed0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.289+0000 7f0989df4640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f097006b4d0 0x7f097006d990 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.289+0000 7f0989df4640 1 --2- 172.21.15.38:0/1151919399 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0984071690 0x7f0984151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.289+0000 7f0989df4640 1 -- 172.21.15.38:0/1151919399 >> 172.21.15.38:0/1151919399 conn(0x7f098406abe0 msgr2=0x7f098415d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:17.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.289+0000 7f0989df4640 1 -- 172.21.15.38:0/1151919399 shutdown_connections 2024-04-25T08:39:17.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.289+0000 7f0989df4640 1 -- 172.21.15.38:0/1151919399 wait complete. 2024-04-25T08:39:17.294 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:39:17.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.401+0000 7fd024b29640 1 Processor -- start 2024-04-25T08:39:17.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.402+0000 7fd024b29640 1 -- start start 2024-04-25T08:39:17.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.402+0000 7fd024b29640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd02015bb10 0x7fd02009a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.402+0000 7fd024b29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0200d0460 0x7fd02009a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.402+0000 7fd024b29640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd020160340 con 0x7fd02015f4e0 2024-04-25T08:39:17.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.402+0000 7fd024b29640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0200cffd0 con 0x7fd02015bb10 2024-04-25T08:39:17.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.402+0000 7fd024b29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0200cf400 con 0x7fd0200d0460 2024-04-25T08:39:17.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.402+0000 7fd01e59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd02015bb10 0x7fd02009a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.403+0000 7fd01e59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd02015bb10 0x7fd02009a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45198/0 (socket says 172.21.15.38:45198) 2024-04-25T08:39:17.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.403+0000 7fd01e59c640 1 -- 172.21.15.38:0/369850219 learned_addr learned my addr 172.21.15.38:0/369850219 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:17.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.403+0000 7fd015d9b640 1 --2- 172.21.15.38:0/369850219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0200d0460 0x7fd02009a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.403+0000 7fd01e59c640 1 -- 172.21.15.38:0/369850219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0200d0460 msgr2=0x7fd02009a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.403+0000 7fd01e59c640 1 --2- 172.21.15.38:0/369850219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0200d0460 0x7fd02009a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.403+0000 7fd01e59c640 1 -- 172.21.15.38:0/369850219 >> v1:172.21.15.107:6789/0 conn(0x7fd02015f4e0 legacy=0x7fd0200c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:17.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.403+0000 7fd01e59c640 1 -- 172.21.15.38:0/369850219 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0200cf6a0 con 0x7fd02015bb10 2024-04-25T08:39:17.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.403+0000 7fd01e59c640 1 --2- 172.21.15.38:0/369850219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02015bb10 0x7fd02009a480 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fd010009870 tx=0x7fd01002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52460f1e21fe13a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.404+0000 7fd01dd9b640 1 -- 172.21.15.38:0/369850219 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010037070 con 0x7fd02015bb10 2024-04-25T08:39:17.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.404+0000 7fd01dd9b640 1 -- 172.21.15.38:0/369850219 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd010032440 con 0x7fd02015bb10 2024-04-25T08:39:17.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.404+0000 7fd01dd9b640 1 -- 172.21.15.38:0/369850219 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010032810 con 0x7fd02015bb10 2024-04-25T08:39:17.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.404+0000 7fd024b29640 1 -- 172.21.15.38:0/369850219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02015bb10 msgr2=0x7fd02009a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.404+0000 7fd024b29640 1 --2- 172.21.15.38:0/369850219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02015bb10 0x7fd02009a480 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fd010009870 tx=0x7fd01002a180 comp rx=0 tx=0).stop 2024-04-25T08:39:17.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.404+0000 7fd024b29640 1 -- 172.21.15.38:0/369850219 shutdown_connections 2024-04-25T08:39:17.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.404+0000 7fd024b29640 1 --2- 172.21.15.38:0/369850219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd0200d0460 0x7fd02009a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.404+0000 7fd024b29640 1 --2- 172.21.15.38:0/369850219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02015bb10 0x7fd02009a480 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.404+0000 7fd024b29640 1 -- 172.21.15.38:0/369850219 >> 172.21.15.38:0/369850219 conn(0x7fd0200b31f0 msgr2=0x7fd0200b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:17.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.404+0000 7fd024b29640 1 -- 172.21.15.38:0/369850219 shutdown_connections 2024-04-25T08:39:17.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.404+0000 7fd024b29640 1 -- 172.21.15.38:0/369850219 wait complete. 2024-04-25T08:39:17.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.406+0000 7fd024b29640 1 Processor -- start 2024-04-25T08:39:17.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.406+0000 7fd024b29640 1 -- start start 2024-04-25T08:39:17.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.406+0000 7fd024b29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0200d0460 0x7fd0200c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.406+0000 7fd024b29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02015bb10 0x7fd02015aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.406+0000 7fd024b29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02015f4e0 0x7fd0201e5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.406+0000 7fd024b29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02015f970 con 0x7fd02015bb10 2024-04-25T08:39:17.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.406+0000 7fd024b29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0200cfc60 con 0x7fd0200d0460 2024-04-25T08:39:17.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.406+0000 7fd024b29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02015faf0 con 0x7fd02015f4e0 2024-04-25T08:39:17.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd015d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02015bb10 0x7fd02015aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd015d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02015bb10 0x7fd02015aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45202/0 (socket says 172.21.15.38:45202) 2024-04-25T08:39:17.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd015d9b640 1 -- 172.21.15.38:0/4080537924 learned_addr learned my addr 172.21.15.38:0/4080537924 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:17.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd01e59c640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0200d0460 0x7fd0200c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd01ed9d640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02015f4e0 0x7fd0201e5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd015d9b640 1 -- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02015f4e0 msgr2=0x7fd0201e5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd015d9b640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02015f4e0 0x7fd0201e5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd015d9b640 1 -- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0200d0460 msgr2=0x7fd0200c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd015d9b640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0200d0460 0x7fd0200c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd015d9b640 1 -- 172.21.15.38:0/4080537924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0200b5040 con 0x7fd02015bb10 2024-04-25T08:39:17.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd01e59c640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0200d0460 0x7fd0200c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:17.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd01ed9d640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02015f4e0 0x7fd0201e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:17.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd015d9b640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02015bb10 0x7fd02015aba0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd00800c820 tx=0x7fd00800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd0177fe640 1 -- 172.21.15.38:0/4080537924 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd008015070 con 0x7fd02015bb10 2024-04-25T08:39:17.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd0177fe640 1 -- 172.21.15.38:0/4080537924 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd008004240 con 0x7fd02015bb10 2024-04-25T08:39:17.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd024b29640 1 -- 172.21.15.38:0/4080537924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02006c830 con 0x7fd02015bb10 2024-04-25T08:39:17.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd0177fe640 1 -- 172.21.15.38:0/4080537924 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00800ce10 con 0x7fd02015bb10 2024-04-25T08:39:17.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.407+0000 7fd024b29640 1 -- 172.21.15.38:0/4080537924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02006cdf0 con 0x7fd02015bb10 2024-04-25T08:39:17.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.408+0000 7fd024b29640 1 -- 172.21.15.38:0/4080537924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd020068e80 con 0x7fd02015bb10 2024-04-25T08:39:17.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.409+0000 7fd0177fe640 1 -- 172.21.15.38:0/4080537924 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd008007830 con 0x7fd02015bb10 2024-04-25T08:39:17.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.409+0000 7fd0177fe640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd00006b500 0x7fd00006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:17.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.409+0000 7fd0177fe640 1 -- 172.21.15.38:0/4080537924 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd00808a1c0 con 0x7fd02015bb10 2024-04-25T08:39:17.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.409+0000 7fd01e59c640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd00006b500 0x7fd00006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:17.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.410+0000 7fd01e59c640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd00006b500 0x7fd00006d9c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd010034000 tx=0x7fd01002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:17.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.413+0000 7fd0177fe640 1 -- 172.21.15.38:0/4080537924 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd008058f40 con 0x7fd02015bb10 2024-04-25T08:39:17.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.634+0000 7fd024b29640 1 -- 172.21.15.38:0/4080537924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd0200cdbf0 con 0x7fd02015bb10 2024-04-25T08:39:17.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.634+0000 7fd0177fe640 1 -- 172.21.15.38:0/4080537924 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd008058840 con 0x7fd02015bb10 2024-04-25T08:39:17.627 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:39:17.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.636+0000 7fd024b29640 1 -- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd00006b500 msgr2=0x7fd00006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.636+0000 7fd024b29640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd00006b500 0x7fd00006d9c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fd010034000 tx=0x7fd01002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:17.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.636+0000 7fd024b29640 1 -- 172.21.15.38:0/4080537924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02015bb10 msgr2=0x7fd02015aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:17.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.636+0000 7fd024b29640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02015bb10 0x7fd02015aba0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fd00800c820 tx=0x7fd00800cce0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.636+0000 7fd024b29640 1 -- 172.21.15.38:0/4080537924 shutdown_connections 2024-04-25T08:39:17.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.636+0000 7fd024b29640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd00006b500 0x7fd00006d9c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.636+0000 7fd024b29640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd02015f4e0 0x7fd0201e5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.636+0000 7fd024b29640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd02015bb10 0x7fd02015aba0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.636+0000 7fd024b29640 1 --2- 172.21.15.38:0/4080537924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0200d0460 0x7fd0200c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:17.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.636+0000 7fd024b29640 1 -- 172.21.15.38:0/4080537924 >> 172.21.15.38:0/4080537924 conn(0x7fd0200b31f0 msgr2=0x7fd0200b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:17.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.636+0000 7fd024b29640 1 -- 172.21.15.38:0/4080537924 shutdown_connections 2024-04-25T08:39:17.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:17.636+0000 7fd024b29640 1 -- 172.21.15.38:0/4080537924 wait complete. 2024-04-25T08:39:17.641 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:39:17.641 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:39:17.641 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:39:17.641 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:39:17.641 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:17.641 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:39:17.641 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:39:17.641 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:39:17.641 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:39:17.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:17.641 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:39:17.657 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:39:17.657 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:39:17.658 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42280sec preferred_lft 42280sec 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:17.659 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::440f:97ff:fe63:87b/64 scope link 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ea:23:76:ab:b1:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::e823:76ff:feab:b1e9/64 scope link 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::38c7:38ff:fe2e:6c92/64 scope link 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:39:17.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:17.661 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:39:17.720 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:39:17.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:17.722 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:39:17.778 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:39:17.778 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:39:17.778 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:39:17.778 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:39:17.778 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:39:17.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:17.779 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:39:17.779 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:17.779 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:39:17.779 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:39:17.779 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:39:17.779 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:17.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:17.864 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:39:17.879 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:39:17.879 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:39:17.879 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:39:17.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:17.879 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T08:39:17.945 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:39:17.945 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:39:17.945 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:18.000 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:39:18.000 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:39:18.001 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:18.056 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:39:18.122 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:39:18.179 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:39:18.179 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:39:18.179 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:39:18.179 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:39:18.179 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:39:18.179 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:39:18.180 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:39:18.181 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:39:18.182 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:39:18.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:18.182 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:39:18.234 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:39:18.234 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:39:18.277 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:39:18.301 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:39:18.358 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:39:18.358 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:39:18.358 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:39:18.358 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:39:18.358 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:39:18.358 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:39:18.358 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:39:18.358 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:39:18.359 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:39:18.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:18.361 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:39:18.361 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-04-25T08:39:18.366+0000 7f22a271e480 -1 init, newargv = 0x559cafee26a0 newargc=15 2024-04-25T08:39:18.362 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[83796]: starting ceph client 2024-04-25T08:39:18.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[83796]: starting fuse 2024-04-25T08:39:18.376 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:39:18.376 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:39:18.376 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:39:18.376 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:39:18.376 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:39:18.376 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:39:18.377 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:39:18.378 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:39:18.378 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:39:18.378 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:39:18.378 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:39:18.378 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-04-25T08:39:18.378 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:39:18.495 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.83796.asok 2024-04-25T08:39:18.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:39:18.505 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.83796.asok 2024-04-25T08:39:18.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:18.506 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.83796.asok status 2024-04-25T08:39:18.642 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:39:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:39:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:39:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:39:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-04-25T08:39:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:39:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:39:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "83796", 2024-04-25T08:39:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:39:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:18.643 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 8478, 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 8478 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2844180847 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:18.644 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2844180847 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.8478 192.168.0.1:0/2844180847", 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/2844180847", 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 63, 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 109, 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:39:18.645 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:39:18.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:18.651 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:39:18.651 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:18.666 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:39:18.667 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:18.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:18.667 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:18.732 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:18.732 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-25T08:39:18.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:18.733 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:18.733 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:18.733 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T08:39:18.733 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:18.733 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:18.733 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:18.733 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:18.815 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.815 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.815 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0120423 s, 87.1 MB/s 2024-04-25T08:39:18.826 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.826 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.826 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862973 s, 122 MB/s 2024-04-25T08:39:18.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.838 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00965604 s, 109 MB/s 2024-04-25T08:39:18.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.848 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755561 s, 139 MB/s 2024-04-25T08:39:18.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.858 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778604 s, 135 MB/s 2024-04-25T08:39:18.868 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.869 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800001 s, 131 MB/s 2024-04-25T08:39:18.881 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.881 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.881 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101266 s, 104 MB/s 2024-04-25T08:39:18.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.892 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.892 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791278 s, 133 MB/s 2024-04-25T08:39:18.902 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.902 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.902 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799581 s, 131 MB/s 2024-04-25T08:39:18.912 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.913 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841538 s, 125 MB/s 2024-04-25T08:39:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.923 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841403 s, 125 MB/s 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.933 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726583 s, 144 MB/s 2024-04-25T08:39:18.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.944 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833884 s, 126 MB/s 2024-04-25T08:39:18.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.954 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815037 s, 129 MB/s 2024-04-25T08:39:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00940677 s, 111 MB/s 2024-04-25T08:39:18.975 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769129 s, 136 MB/s 2024-04-25T08:39:18.986 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.986 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.986 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751032 s, 140 MB/s 2024-04-25T08:39:18.997 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:18.997 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:18.997 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907402 s, 116 MB/s 2024-04-25T08:39:19.007 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.008 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806363 s, 130 MB/s 2024-04-25T08:39:19.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.018 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780013 s, 134 MB/s 2024-04-25T08:39:19.030 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.030 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.030 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00989803 s, 106 MB/s 2024-04-25T08:39:19.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.041 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829491 s, 126 MB/s 2024-04-25T08:39:19.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.052 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890378 s, 118 MB/s 2024-04-25T08:39:19.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.062 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762991 s, 137 MB/s 2024-04-25T08:39:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701134 s, 150 MB/s 2024-04-25T08:39:19.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.081 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704805 s, 149 MB/s 2024-04-25T08:39:19.091 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.091 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.091 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843238 s, 124 MB/s 2024-04-25T08:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.100 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700055 s, 150 MB/s 2024-04-25T08:39:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766752 s, 137 MB/s 2024-04-25T08:39:19.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.120 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754036 s, 139 MB/s 2024-04-25T08:39:19.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.130 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790113 s, 133 MB/s 2024-04-25T08:39:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733198 s, 143 MB/s 2024-04-25T08:39:19.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:19.141 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:19.141 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:19.141 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T08:39:19.141 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:19.141 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:19.141 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:19.141 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:19.209 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.209 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.209 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724259 s, 145 MB/s 2024-04-25T08:39:19.219 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.219 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.219 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797987 s, 131 MB/s 2024-04-25T08:39:19.229 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.229 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.229 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741142 s, 141 MB/s 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.239 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795121 s, 132 MB/s 2024-04-25T08:39:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.249 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792078 s, 132 MB/s 2024-04-25T08:39:19.260 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.260 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.260 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903872 s, 116 MB/s 2024-04-25T08:39:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.271 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.272 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859026 s, 122 MB/s 2024-04-25T08:39:19.282 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.283 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867054 s, 121 MB/s 2024-04-25T08:39:19.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.292 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737052 s, 142 MB/s 2024-04-25T08:39:19.302 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.302 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.302 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732542 s, 143 MB/s 2024-04-25T08:39:19.311 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.311 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.311 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702405 s, 149 MB/s 2024-04-25T08:39:19.322 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.322 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.323 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886585 s, 118 MB/s 2024-04-25T08:39:19.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.333 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779983 s, 134 MB/s 2024-04-25T08:39:19.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.342 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738736 s, 142 MB/s 2024-04-25T08:39:19.351 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.351 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.351 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704376 s, 149 MB/s 2024-04-25T08:39:19.361 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.362 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765902 s, 137 MB/s 2024-04-25T08:39:19.372 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.372 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.372 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836172 s, 125 MB/s 2024-04-25T08:39:19.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.382 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800804 s, 131 MB/s 2024-04-25T08:39:19.392 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.392 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.392 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754375 s, 139 MB/s 2024-04-25T08:39:19.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.401 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691001 s, 152 MB/s 2024-04-25T08:39:19.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.411 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755176 s, 139 MB/s 2024-04-25T08:39:19.421 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.421 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.421 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754915 s, 139 MB/s 2024-04-25T08:39:19.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.431 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078511 s, 134 MB/s 2024-04-25T08:39:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.441 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765531 s, 137 MB/s 2024-04-25T08:39:19.450 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.450 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.450 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754486 s, 139 MB/s 2024-04-25T08:39:19.460 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.460 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.460 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728681 s, 144 MB/s 2024-04-25T08:39:19.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.470 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761133 s, 138 MB/s 2024-04-25T08:39:19.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.480 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.480 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728805 s, 144 MB/s 2024-04-25T08:39:19.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.489 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710843 s, 148 MB/s 2024-04-25T08:39:19.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.499 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792749 s, 132 MB/s 2024-04-25T08:39:19.509 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.509 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.509 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778903 s, 135 MB/s 2024-04-25T08:39:19.519 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.519 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.519 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734353 s, 143 MB/s 2024-04-25T08:39:19.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:19.520 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:19.520 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:19.520 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T08:39:19.520 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:19.520 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:19.520 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:19.520 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:19.588 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.588 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.588 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764465 s, 137 MB/s 2024-04-25T08:39:19.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.598 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794345 s, 132 MB/s 2024-04-25T08:39:19.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.608 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766396 s, 137 MB/s 2024-04-25T08:39:19.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.618 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725788 s, 144 MB/s 2024-04-25T08:39:19.627 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.628 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751472 s, 140 MB/s 2024-04-25T08:39:19.637 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.637 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.638 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776488 s, 135 MB/s 2024-04-25T08:39:19.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.647 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732768 s, 143 MB/s 2024-04-25T08:39:19.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.657 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723953 s, 145 MB/s 2024-04-25T08:39:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.667 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813875 s, 129 MB/s 2024-04-25T08:39:19.677 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.677 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.677 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764187 s, 137 MB/s 2024-04-25T08:39:19.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.688 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874376 s, 120 MB/s 2024-04-25T08:39:19.698 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.698 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.698 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782064 s, 134 MB/s 2024-04-25T08:39:19.709 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.709 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.709 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849571 s, 123 MB/s 2024-04-25T08:39:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.719 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076861 s, 136 MB/s 2024-04-25T08:39:19.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.729 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774417 s, 135 MB/s 2024-04-25T08:39:19.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.740 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.740 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776871 s, 135 MB/s 2024-04-25T08:39:19.750 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.750 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.750 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797061 s, 132 MB/s 2024-04-25T08:39:19.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.760 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769366 s, 136 MB/s 2024-04-25T08:39:19.769 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.770 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.770 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753218 s, 139 MB/s 2024-04-25T08:39:19.780 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.780 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.780 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075896 s, 138 MB/s 2024-04-25T08:39:19.789 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.790 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768057 s, 137 MB/s 2024-04-25T08:39:19.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.799 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699087 s, 150 MB/s 2024-04-25T08:39:19.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.809 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076287 s, 137 MB/s 2024-04-25T08:39:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787975 s, 133 MB/s 2024-04-25T08:39:19.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.829 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847359 s, 124 MB/s 2024-04-25T08:39:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.840 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799089 s, 131 MB/s 2024-04-25T08:39:19.851 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.851 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.851 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891826 s, 118 MB/s 2024-04-25T08:39:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.861 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733688 s, 143 MB/s 2024-04-25T08:39:19.871 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.871 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.871 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776659 s, 135 MB/s 2024-04-25T08:39:19.881 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.881 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.881 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759781 s, 138 MB/s 2024-04-25T08:39:19.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.891 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759519 s, 138 MB/s 2024-04-25T08:39:19.900 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.900 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.900 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724018 s, 145 MB/s 2024-04-25T08:39:19.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:19.901 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:19.901 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:19.901 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T08:39:19.902 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:19.902 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:19.902 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:19.902 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:19.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732083 s, 143 MB/s 2024-04-25T08:39:19.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.980 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786523 s, 133 MB/s 2024-04-25T08:39:19.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:19.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:19.990 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808675 s, 130 MB/s 2024-04-25T08:39:20.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.001 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882681 s, 119 MB/s 2024-04-25T08:39:20.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.011 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746517 s, 140 MB/s 2024-04-25T08:39:20.021 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.021 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.021 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789831 s, 133 MB/s 2024-04-25T08:39:20.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.032 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894035 s, 117 MB/s 2024-04-25T08:39:20.042 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.042 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.042 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716149 s, 146 MB/s 2024-04-25T08:39:20.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.052 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773426 s, 136 MB/s 2024-04-25T08:39:20.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.062 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762878 s, 137 MB/s 2024-04-25T08:39:20.072 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.072 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.072 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851012 s, 123 MB/s 2024-04-25T08:39:20.082 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.082 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.083 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749512 s, 140 MB/s 2024-04-25T08:39:20.092 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.092 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.092 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785908 s, 133 MB/s 2024-04-25T08:39:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779942 s, 134 MB/s 2024-04-25T08:39:20.113 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.113 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.113 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849565 s, 123 MB/s 2024-04-25T08:39:20.123 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.123 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.123 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765516 s, 137 MB/s 2024-04-25T08:39:20.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.133 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843705 s, 124 MB/s 2024-04-25T08:39:20.144 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.144 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.144 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086673 s, 121 MB/s 2024-04-25T08:39:20.154 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.154 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.154 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755191 s, 139 MB/s 2024-04-25T08:39:20.164 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.164 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.164 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073663 s, 142 MB/s 2024-04-25T08:39:20.173 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.173 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.173 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072788 s, 144 MB/s 2024-04-25T08:39:20.183 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.183 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.184 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787987 s, 133 MB/s 2024-04-25T08:39:20.192 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.192 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.193 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699576 s, 150 MB/s 2024-04-25T08:39:20.202 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.202 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.202 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729249 s, 144 MB/s 2024-04-25T08:39:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754226 s, 139 MB/s 2024-04-25T08:39:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733998 s, 143 MB/s 2024-04-25T08:39:20.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.231 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712742 s, 147 MB/s 2024-04-25T08:39:20.240 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.241 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.241 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727867 s, 144 MB/s 2024-04-25T08:39:20.251 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.251 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.251 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777124 s, 135 MB/s 2024-04-25T08:39:20.261 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.262 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.262 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840547 s, 125 MB/s 2024-04-25T08:39:20.271 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.271 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.271 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788495 s, 133 MB/s 2024-04-25T08:39:20.282 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.283 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876968 s, 120 MB/s 2024-04-25T08:39:20.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:20.284 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:20.285 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:20.285 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T08:39:20.285 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:20.285 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:20.285 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:20.285 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:20.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.353 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756549 s, 139 MB/s 2024-04-25T08:39:20.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.363 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814707 s, 129 MB/s 2024-04-25T08:39:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.372 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749653 s, 140 MB/s 2024-04-25T08:39:20.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.382 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736851 s, 142 MB/s 2024-04-25T08:39:20.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.391 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006864 s, 153 MB/s 2024-04-25T08:39:20.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.402 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788781 s, 133 MB/s 2024-04-25T08:39:20.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.411 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791326 s, 133 MB/s 2024-04-25T08:39:20.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.422 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833138 s, 126 MB/s 2024-04-25T08:39:20.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.432 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764332 s, 137 MB/s 2024-04-25T08:39:20.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.442 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746801 s, 140 MB/s 2024-04-25T08:39:20.451 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.451 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.451 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685932 s, 153 MB/s 2024-04-25T08:39:20.461 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.461 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.461 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750904 s, 140 MB/s 2024-04-25T08:39:20.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.470 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744171 s, 141 MB/s 2024-04-25T08:39:20.480 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.480 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.480 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073385 s, 143 MB/s 2024-04-25T08:39:20.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.491 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846903 s, 124 MB/s 2024-04-25T08:39:20.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.501 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764047 s, 137 MB/s 2024-04-25T08:39:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.510 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766549 s, 137 MB/s 2024-04-25T08:39:20.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.520 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760959 s, 138 MB/s 2024-04-25T08:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.530 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.531 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739814 s, 142 MB/s 2024-04-25T08:39:20.540 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.540 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.540 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779907 s, 134 MB/s 2024-04-25T08:39:20.549 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.549 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.549 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737546 s, 142 MB/s 2024-04-25T08:39:20.559 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.559 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.559 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730925 s, 143 MB/s 2024-04-25T08:39:20.568 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.568 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.568 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071155 s, 147 MB/s 2024-04-25T08:39:20.578 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.578 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.578 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765845 s, 137 MB/s 2024-04-25T08:39:20.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.589 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873756 s, 120 MB/s 2024-04-25T08:39:20.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.599 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.599 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701995 s, 149 MB/s 2024-04-25T08:39:20.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.608 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727109 s, 144 MB/s 2024-04-25T08:39:20.619 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.619 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.619 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842445 s, 124 MB/s 2024-04-25T08:39:20.629 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.630 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.630 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838553 s, 125 MB/s 2024-04-25T08:39:20.639 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.640 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.640 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760114 s, 138 MB/s 2024-04-25T08:39:20.649 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.649 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.650 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780623 s, 134 MB/s 2024-04-25T08:39:20.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.659 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744321 s, 141 MB/s 2024-04-25T08:39:20.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:20.661 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:20.661 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:20.661 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T08:39:20.661 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:20.661 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:20.661 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:20.661 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:20.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.729 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762372 s, 138 MB/s 2024-04-25T08:39:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.739 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777743 s, 135 MB/s 2024-04-25T08:39:20.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.748 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729007 s, 144 MB/s 2024-04-25T08:39:20.758 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.758 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.758 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716923 s, 146 MB/s 2024-04-25T08:39:20.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.768 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832419 s, 126 MB/s 2024-04-25T08:39:20.778 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.778 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.778 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767785 s, 137 MB/s 2024-04-25T08:39:20.788 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.788 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.788 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747197 s, 140 MB/s 2024-04-25T08:39:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.797 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713359 s, 147 MB/s 2024-04-25T08:39:20.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.809 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877289 s, 120 MB/s 2024-04-25T08:39:20.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082336 s, 127 MB/s 2024-04-25T08:39:20.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.829 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694179 s, 151 MB/s 2024-04-25T08:39:20.837 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.837 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.837 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696392 s, 151 MB/s 2024-04-25T08:39:20.847 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.847 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.848 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760443 s, 138 MB/s 2024-04-25T08:39:20.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.858 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086064 s, 122 MB/s 2024-04-25T08:39:20.868 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.869 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806675 s, 130 MB/s 2024-04-25T08:39:20.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.879 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787497 s, 133 MB/s 2024-04-25T08:39:20.889 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.889 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.889 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847963 s, 124 MB/s 2024-04-25T08:39:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.900 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861448 s, 122 MB/s 2024-04-25T08:39:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.910 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757309 s, 138 MB/s 2024-04-25T08:39:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.920 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755373 s, 139 MB/s 2024-04-25T08:39:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.931 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862199 s, 122 MB/s 2024-04-25T08:39:20.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.940 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681382 s, 154 MB/s 2024-04-25T08:39:20.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.950 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761653 s, 138 MB/s 2024-04-25T08:39:20.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.960 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735191 s, 143 MB/s 2024-04-25T08:39:20.969 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.969 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.969 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736536 s, 142 MB/s 2024-04-25T08:39:20.978 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.978 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.978 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704182 s, 149 MB/s 2024-04-25T08:39:20.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.988 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782084 s, 134 MB/s 2024-04-25T08:39:20.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:20.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:20.999 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851358 s, 123 MB/s 2024-04-25T08:39:21.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.008 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695102 s, 151 MB/s 2024-04-25T08:39:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.019 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849779 s, 123 MB/s 2024-04-25T08:39:21.029 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.029 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.029 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751115 s, 140 MB/s 2024-04-25T08:39:21.039 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.040 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.040 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835981 s, 125 MB/s 2024-04-25T08:39:21.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:21.041 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:21.041 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:21.041 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T08:39:21.041 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:21.041 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:21.041 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:21.041 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:21.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.111 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.111 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830423 s, 126 MB/s 2024-04-25T08:39:21.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.120 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743482 s, 141 MB/s 2024-04-25T08:39:21.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.131 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.131 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830871 s, 126 MB/s 2024-04-25T08:39:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819704 s, 128 MB/s 2024-04-25T08:39:21.151 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.151 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.151 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745766 s, 141 MB/s 2024-04-25T08:39:21.161 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.162 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084728 s, 124 MB/s 2024-04-25T08:39:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.172 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.172 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762225 s, 138 MB/s 2024-04-25T08:39:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.181 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073708 s, 142 MB/s 2024-04-25T08:39:21.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.191 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830403 s, 126 MB/s 2024-04-25T08:39:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.200 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645379 s, 162 MB/s 2024-04-25T08:39:21.210 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.211 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826532 s, 127 MB/s 2024-04-25T08:39:21.221 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.221 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.221 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794168 s, 132 MB/s 2024-04-25T08:39:21.230 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.230 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.230 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708214 s, 148 MB/s 2024-04-25T08:39:21.239 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.239 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.239 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670217 s, 156 MB/s 2024-04-25T08:39:21.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.248 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667488 s, 157 MB/s 2024-04-25T08:39:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718085 s, 146 MB/s 2024-04-25T08:39:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851942 s, 123 MB/s 2024-04-25T08:39:21.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.278 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073038 s, 144 MB/s 2024-04-25T08:39:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.288 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784942 s, 134 MB/s 2024-04-25T08:39:21.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.297 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00635616 s, 165 MB/s 2024-04-25T08:39:21.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.306 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726215 s, 144 MB/s 2024-04-25T08:39:21.315 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.315 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.315 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670233 s, 156 MB/s 2024-04-25T08:39:21.325 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.325 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.325 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798068 s, 131 MB/s 2024-04-25T08:39:21.336 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.336 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.336 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802483 s, 131 MB/s 2024-04-25T08:39:21.346 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.346 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.346 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826064 s, 127 MB/s 2024-04-25T08:39:21.355 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.355 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.356 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070886 s, 148 MB/s 2024-04-25T08:39:21.366 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.366 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.366 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817404 s, 128 MB/s 2024-04-25T08:39:21.376 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831495 s, 126 MB/s 2024-04-25T08:39:21.386 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.387 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.387 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778654 s, 135 MB/s 2024-04-25T08:39:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725114 s, 145 MB/s 2024-04-25T08:39:21.405 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.405 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.406 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070676 s, 148 MB/s 2024-04-25T08:39:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753484 s, 139 MB/s 2024-04-25T08:39:21.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:21.416 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:21.417 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:21.417 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T08:39:21.417 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:21.417 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:21.417 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:21.417 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:21.484 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.484 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.485 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666839 s, 157 MB/s 2024-04-25T08:39:21.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.494 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713542 s, 147 MB/s 2024-04-25T08:39:21.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.503 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745711 s, 141 MB/s 2024-04-25T08:39:21.512 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.513 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711761 s, 147 MB/s 2024-04-25T08:39:21.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.522 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731617 s, 143 MB/s 2024-04-25T08:39:21.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.531 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666688 s, 157 MB/s 2024-04-25T08:39:21.541 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.542 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815149 s, 129 MB/s 2024-04-25T08:39:21.551 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.552 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.552 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764478 s, 137 MB/s 2024-04-25T08:39:21.561 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.561 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.561 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710645 s, 148 MB/s 2024-04-25T08:39:21.570 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.570 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.570 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733755 s, 143 MB/s 2024-04-25T08:39:21.580 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.580 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.581 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814894 s, 129 MB/s 2024-04-25T08:39:21.590 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.590 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.590 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745537 s, 141 MB/s 2024-04-25T08:39:21.600 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.600 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.600 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728058 s, 144 MB/s 2024-04-25T08:39:21.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.609 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007066 s, 148 MB/s 2024-04-25T08:39:21.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.619 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726415 s, 144 MB/s 2024-04-25T08:39:21.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.628 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710323 s, 148 MB/s 2024-04-25T08:39:21.638 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.638 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.638 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007523 s, 139 MB/s 2024-04-25T08:39:21.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.647 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728213 s, 144 MB/s 2024-04-25T08:39:21.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.657 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708302 s, 148 MB/s 2024-04-25T08:39:21.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.666 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746081 s, 141 MB/s 2024-04-25T08:39:21.677 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.677 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.677 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850154 s, 123 MB/s 2024-04-25T08:39:21.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.688 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820176 s, 128 MB/s 2024-04-25T08:39:21.697 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.698 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.698 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766343 s, 137 MB/s 2024-04-25T08:39:21.708 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.708 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.708 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813617 s, 129 MB/s 2024-04-25T08:39:21.717 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.717 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.717 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006659 s, 157 MB/s 2024-04-25T08:39:21.726 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.726 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.726 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674957 s, 155 MB/s 2024-04-25T08:39:21.735 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.735 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.735 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654225 s, 160 MB/s 2024-04-25T08:39:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.745 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778703 s, 135 MB/s 2024-04-25T08:39:21.755 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.755 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.755 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714386 s, 147 MB/s 2024-04-25T08:39:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.764 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672454 s, 156 MB/s 2024-04-25T08:39:21.773 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.773 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.773 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732065 s, 143 MB/s 2024-04-25T08:39:21.783 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.783 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.783 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075931 s, 138 MB/s 2024-04-25T08:39:21.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:21.785 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:21.785 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:21.785 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T08:39:21.785 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:21.785 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:21.785 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:21.785 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:21.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.854 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839039 s, 125 MB/s 2024-04-25T08:39:21.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.864 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.864 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724355 s, 145 MB/s 2024-04-25T08:39:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754873 s, 139 MB/s 2024-04-25T08:39:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764969 s, 137 MB/s 2024-04-25T08:39:21.893 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.893 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.893 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753481 s, 139 MB/s 2024-04-25T08:39:21.903 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.903 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.903 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750605 s, 140 MB/s 2024-04-25T08:39:21.912 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.912 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.912 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735665 s, 143 MB/s 2024-04-25T08:39:21.921 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.921 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.921 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668531 s, 157 MB/s 2024-04-25T08:39:21.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.931 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713498 s, 147 MB/s 2024-04-25T08:39:21.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.941 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751134 s, 140 MB/s 2024-04-25T08:39:21.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.950 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073311 s, 143 MB/s 2024-04-25T08:39:21.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.961 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812117 s, 129 MB/s 2024-04-25T08:39:21.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.970 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068804 s, 152 MB/s 2024-04-25T08:39:21.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.980 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717479 s, 146 MB/s 2024-04-25T08:39:21.989 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.989 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.989 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678448 s, 155 MB/s 2024-04-25T08:39:21.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:21.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:21.998 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733149 s, 143 MB/s 2024-04-25T08:39:22.007 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.008 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734373 s, 143 MB/s 2024-04-25T08:39:22.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.019 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084637 s, 124 MB/s 2024-04-25T08:39:22.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.028 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071021 s, 148 MB/s 2024-04-25T08:39:22.038 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.038 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.038 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008166 s, 128 MB/s 2024-04-25T08:39:22.049 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.049 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.049 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837062 s, 125 MB/s 2024-04-25T08:39:22.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.059 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.059 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690129 s, 152 MB/s 2024-04-25T08:39:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.069 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759733 s, 138 MB/s 2024-04-25T08:39:22.078 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.078 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.078 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727526 s, 144 MB/s 2024-04-25T08:39:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.088 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722575 s, 145 MB/s 2024-04-25T08:39:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817326 s, 128 MB/s 2024-04-25T08:39:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.108 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075579 s, 139 MB/s 2024-04-25T08:39:22.118 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.118 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.118 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770151 s, 136 MB/s 2024-04-25T08:39:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701567 s, 149 MB/s 2024-04-25T08:39:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694399 s, 151 MB/s 2024-04-25T08:39:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.147 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827078 s, 127 MB/s 2024-04-25T08:39:22.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.157 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745264 s, 141 MB/s 2024-04-25T08:39:22.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:22.158 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:22.158 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:22.158 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T08:39:22.158 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:22.158 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:22.158 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:22.158 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.227 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828843 s, 127 MB/s 2024-04-25T08:39:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.237 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713013 s, 147 MB/s 2024-04-25T08:39:22.246 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.246 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.246 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738533 s, 142 MB/s 2024-04-25T08:39:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.256 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708839 s, 148 MB/s 2024-04-25T08:39:22.266 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.266 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.266 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838944 s, 125 MB/s 2024-04-25T08:39:22.276 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.276 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.276 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721481 s, 145 MB/s 2024-04-25T08:39:22.285 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.285 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.286 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728942 s, 144 MB/s 2024-04-25T08:39:22.295 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.295 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.295 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720805 s, 145 MB/s 2024-04-25T08:39:22.305 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.306 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820608 s, 128 MB/s 2024-04-25T08:39:22.315 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.315 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.315 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714163 s, 147 MB/s 2024-04-25T08:39:22.325 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.326 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807077 s, 130 MB/s 2024-04-25T08:39:22.335 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.335 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.335 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673648 s, 156 MB/s 2024-04-25T08:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.344 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073486 s, 143 MB/s 2024-04-25T08:39:22.354 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.354 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.354 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708278 s, 148 MB/s 2024-04-25T08:39:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.364 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756556 s, 139 MB/s 2024-04-25T08:39:22.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.373 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718482 s, 146 MB/s 2024-04-25T08:39:22.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.383 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074402 s, 141 MB/s 2024-04-25T08:39:22.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.393 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826327 s, 127 MB/s 2024-04-25T08:39:22.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.403 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753492 s, 139 MB/s 2024-04-25T08:39:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.412 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693745 s, 151 MB/s 2024-04-25T08:39:22.421 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.422 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742196 s, 141 MB/s 2024-04-25T08:39:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.431 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752499 s, 139 MB/s 2024-04-25T08:39:22.440 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.441 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.441 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681005 s, 154 MB/s 2024-04-25T08:39:22.450 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.450 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.450 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701771 s, 149 MB/s 2024-04-25T08:39:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.460 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821912 s, 128 MB/s 2024-04-25T08:39:22.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.470 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074102 s, 142 MB/s 2024-04-25T08:39:22.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.479 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714668 s, 147 MB/s 2024-04-25T08:39:22.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.489 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748543 s, 140 MB/s 2024-04-25T08:39:22.498 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.498 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.498 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699611 s, 150 MB/s 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.508 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722723 s, 145 MB/s 2024-04-25T08:39:22.517 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.517 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.517 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734009 s, 143 MB/s 2024-04-25T08:39:22.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.528 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851833 s, 123 MB/s 2024-04-25T08:39:22.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:22.529 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:22.529 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:22.529 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T08:39:22.529 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:22.529 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:22.529 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:22.529 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:22.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.598 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724055 s, 145 MB/s 2024-04-25T08:39:22.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.607 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717143 s, 146 MB/s 2024-04-25T08:39:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757792 s, 138 MB/s 2024-04-25T08:39:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.626 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733303 s, 143 MB/s 2024-04-25T08:39:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.636 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786257 s, 133 MB/s 2024-04-25T08:39:22.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.646 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738072 s, 142 MB/s 2024-04-25T08:39:22.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.655 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689048 s, 152 MB/s 2024-04-25T08:39:22.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.665 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741397 s, 141 MB/s 2024-04-25T08:39:22.675 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.675 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.675 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792881 s, 132 MB/s 2024-04-25T08:39:22.684 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.684 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.685 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730433 s, 144 MB/s 2024-04-25T08:39:22.693 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.694 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703334 s, 149 MB/s 2024-04-25T08:39:22.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.703 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.703 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658531 s, 159 MB/s 2024-04-25T08:39:22.712 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746664 s, 140 MB/s 2024-04-25T08:39:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734074 s, 143 MB/s 2024-04-25T08:39:22.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.731 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700253 s, 150 MB/s 2024-04-25T08:39:22.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.742 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851076 s, 123 MB/s 2024-04-25T08:39:22.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.752 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706899 s, 148 MB/s 2024-04-25T08:39:22.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.761 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725947 s, 144 MB/s 2024-04-25T08:39:22.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.771 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812007 s, 129 MB/s 2024-04-25T08:39:22.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.781 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705179 s, 149 MB/s 2024-04-25T08:39:22.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.791 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784569 s, 134 MB/s 2024-04-25T08:39:22.801 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.801 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.801 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073174 s, 143 MB/s 2024-04-25T08:39:22.810 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.810 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.810 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694613 s, 151 MB/s 2024-04-25T08:39:22.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701591 s, 149 MB/s 2024-04-25T08:39:22.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.830 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.830 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815392 s, 129 MB/s 2024-04-25T08:39:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.839 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737527 s, 142 MB/s 2024-04-25T08:39:22.849 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.849 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.849 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727057 s, 144 MB/s 2024-04-25T08:39:22.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.858 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660021 s, 159 MB/s 2024-04-25T08:39:22.867 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.867 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.867 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711137 s, 147 MB/s 2024-04-25T08:39:22.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.878 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.878 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812891 s, 129 MB/s 2024-04-25T08:39:22.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.887 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717326 s, 146 MB/s 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.897 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816763 s, 128 MB/s 2024-04-25T08:39:22.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:22.899 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:22.899 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:22.899 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T08:39:22.899 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:22.899 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:22.899 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:22.899 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:22.967 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.967 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.967 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709856 s, 148 MB/s 2024-04-25T08:39:22.976 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.976 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.976 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730086 s, 144 MB/s 2024-04-25T08:39:22.985 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.985 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.985 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674557 s, 155 MB/s 2024-04-25T08:39:22.996 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:22.996 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:22.996 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812724 s, 129 MB/s 2024-04-25T08:39:23.006 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.006 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.006 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854004 s, 123 MB/s 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.017 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808771 s, 130 MB/s 2024-04-25T08:39:23.026 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.026 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.026 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706213 s, 148 MB/s 2024-04-25T08:39:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.036 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732051 s, 143 MB/s 2024-04-25T08:39:23.045 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.045 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.045 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717092 s, 146 MB/s 2024-04-25T08:39:23.056 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.056 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.056 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859339 s, 122 MB/s 2024-04-25T08:39:23.066 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.067 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.067 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831565 s, 126 MB/s 2024-04-25T08:39:23.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.077 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805777 s, 130 MB/s 2024-04-25T08:39:23.086 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.086 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.086 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661753 s, 158 MB/s 2024-04-25T08:39:23.095 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.095 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.095 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672328 s, 156 MB/s 2024-04-25T08:39:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726548 s, 144 MB/s 2024-04-25T08:39:23.114 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.114 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.114 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720604 s, 146 MB/s 2024-04-25T08:39:23.124 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.124 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.124 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726856 s, 144 MB/s 2024-04-25T08:39:23.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.133 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700409 s, 150 MB/s 2024-04-25T08:39:23.142 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.142 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.142 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712589 s, 147 MB/s 2024-04-25T08:39:23.152 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.152 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.152 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749703 s, 140 MB/s 2024-04-25T08:39:23.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.162 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074619 s, 141 MB/s 2024-04-25T08:39:23.172 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.172 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.172 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782663 s, 134 MB/s 2024-04-25T08:39:23.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.182 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767999 s, 137 MB/s 2024-04-25T08:39:23.194 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.194 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.195 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934826 s, 112 MB/s 2024-04-25T08:39:23.205 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.205 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.205 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849651 s, 123 MB/s 2024-04-25T08:39:23.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.215 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752892 s, 139 MB/s 2024-04-25T08:39:23.226 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.226 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.226 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082729 s, 127 MB/s 2024-04-25T08:39:23.236 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.236 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.236 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794814 s, 132 MB/s 2024-04-25T08:39:23.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.247 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826602 s, 127 MB/s 2024-04-25T08:39:23.258 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.258 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.258 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801587 s, 131 MB/s 2024-04-25T08:39:23.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.267 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678216 s, 155 MB/s 2024-04-25T08:39:23.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.278 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813693 s, 129 MB/s 2024-04-25T08:39:23.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:23.280 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:23.280 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:23.280 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T08:39:23.280 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:23.280 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:23.280 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:23.280 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:23.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.350 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768087 s, 137 MB/s 2024-04-25T08:39:23.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.360 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797175 s, 132 MB/s 2024-04-25T08:39:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.371 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811815 s, 129 MB/s 2024-04-25T08:39:23.381 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.381 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.381 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800223 s, 131 MB/s 2024-04-25T08:39:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.391 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715395 s, 147 MB/s 2024-04-25T08:39:23.402 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.402 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.402 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814062 s, 129 MB/s 2024-04-25T08:39:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.411 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674786 s, 155 MB/s 2024-04-25T08:39:23.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.423 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724924 s, 145 MB/s 2024-04-25T08:39:23.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.431 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752551 s, 139 MB/s 2024-04-25T08:39:23.441 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.442 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806059 s, 130 MB/s 2024-04-25T08:39:23.452 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.452 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.453 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823237 s, 127 MB/s 2024-04-25T08:39:23.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.463 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802143 s, 131 MB/s 2024-04-25T08:39:23.471 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.472 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.472 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662137 s, 158 MB/s 2024-04-25T08:39:23.482 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.482 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.482 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754512 s, 139 MB/s 2024-04-25T08:39:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.492 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797376 s, 132 MB/s 2024-04-25T08:39:23.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.500 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648157 s, 162 MB/s 2024-04-25T08:39:23.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.511 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791451 s, 132 MB/s 2024-04-25T08:39:23.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.520 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700826 s, 150 MB/s 2024-04-25T08:39:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.529 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708541 s, 148 MB/s 2024-04-25T08:39:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.539 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718557 s, 146 MB/s 2024-04-25T08:39:23.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.548 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673114 s, 156 MB/s 2024-04-25T08:39:23.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.557 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072 s, 146 MB/s 2024-04-25T08:39:23.567 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.567 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.567 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830126 s, 126 MB/s 2024-04-25T08:39:23.578 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.578 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.578 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822005 s, 128 MB/s 2024-04-25T08:39:23.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.588 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.588 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774459 s, 135 MB/s 2024-04-25T08:39:23.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.598 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838655 s, 125 MB/s 2024-04-25T08:39:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654505 s, 160 MB/s 2024-04-25T08:39:23.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.618 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802524 s, 131 MB/s 2024-04-25T08:39:23.630 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.630 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.631 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710163 s, 148 MB/s 2024-04-25T08:39:23.637 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.637 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.637 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083412 s, 126 MB/s 2024-04-25T08:39:23.648 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.648 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.648 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840425 s, 125 MB/s 2024-04-25T08:39:23.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.658 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695807 s, 151 MB/s 2024-04-25T08:39:23.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:23.659 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:23.659 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:23.659 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T08:39:23.660 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:23.660 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:23.660 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:23.660 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:23.728 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.729 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853808 s, 123 MB/s 2024-04-25T08:39:23.738 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.738 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.738 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724118 s, 145 MB/s 2024-04-25T08:39:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.748 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763888 s, 137 MB/s 2024-04-25T08:39:23.757 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.757 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.757 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735022 s, 143 MB/s 2024-04-25T08:39:23.766 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.766 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.767 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685409 s, 153 MB/s 2024-04-25T08:39:23.777 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.777 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.777 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810212 s, 129 MB/s 2024-04-25T08:39:23.787 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.787 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.787 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760973 s, 138 MB/s 2024-04-25T08:39:23.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.797 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746969 s, 140 MB/s 2024-04-25T08:39:23.806 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.806 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.806 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070931 s, 148 MB/s 2024-04-25T08:39:23.815 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.815 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.815 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696736 s, 150 MB/s 2024-04-25T08:39:23.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.825 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.825 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709614 s, 148 MB/s 2024-04-25T08:39:23.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.834 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691585 s, 152 MB/s 2024-04-25T08:39:23.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.844 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755375 s, 139 MB/s 2024-04-25T08:39:23.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.854 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082232 s, 128 MB/s 2024-04-25T08:39:23.864 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.864 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.864 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725401 s, 145 MB/s 2024-04-25T08:39:23.874 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.874 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.874 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844823 s, 124 MB/s 2024-04-25T08:39:23.884 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.884 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.884 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737819 s, 142 MB/s 2024-04-25T08:39:23.893 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.894 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.894 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732179 s, 143 MB/s 2024-04-25T08:39:23.903 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.903 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.903 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762861 s, 137 MB/s 2024-04-25T08:39:23.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.914 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007675 s, 137 MB/s 2024-04-25T08:39:23.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.924 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.924 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080873 s, 130 MB/s 2024-04-25T08:39:23.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.933 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710132 s, 148 MB/s 2024-04-25T08:39:23.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.943 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831651 s, 126 MB/s 2024-04-25T08:39:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.953 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720235 s, 146 MB/s 2024-04-25T08:39:23.963 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.964 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839983 s, 125 MB/s 2024-04-25T08:39:23.973 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.973 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.973 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716642 s, 146 MB/s 2024-04-25T08:39:23.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.982 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700833 s, 150 MB/s 2024-04-25T08:39:23.992 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:23.992 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:23.992 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819315 s, 128 MB/s 2024-04-25T08:39:24.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.002 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703686 s, 149 MB/s 2024-04-25T08:39:24.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.011 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680183 s, 154 MB/s 2024-04-25T08:39:24.020 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.020 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.020 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067775 s, 155 MB/s 2024-04-25T08:39:24.029 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.030 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.030 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714918 s, 147 MB/s 2024-04-25T08:39:24.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:24.031 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:24.031 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:24.031 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T08:39:24.031 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:24.031 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:24.031 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:24.031 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.099 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732242 s, 143 MB/s 2024-04-25T08:39:24.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.108 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665508 s, 158 MB/s 2024-04-25T08:39:24.117 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.117 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.117 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704305 s, 149 MB/s 2024-04-25T08:39:24.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.128 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870856 s, 120 MB/s 2024-04-25T08:39:24.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.138 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079307 s, 132 MB/s 2024-04-25T08:39:24.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071348 s, 147 MB/s 2024-04-25T08:39:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.158 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741433 s, 141 MB/s 2024-04-25T08:39:24.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.167 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072644 s, 144 MB/s 2024-04-25T08:39:24.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.177 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735182 s, 143 MB/s 2024-04-25T08:39:24.186 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.187 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.187 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742602 s, 141 MB/s 2024-04-25T08:39:24.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.197 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810659 s, 129 MB/s 2024-04-25T08:39:24.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.207 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741491 s, 141 MB/s 2024-04-25T08:39:24.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.217 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783946 s, 134 MB/s 2024-04-25T08:39:24.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.230 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757738 s, 138 MB/s 2024-04-25T08:39:24.236 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.236 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.237 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767332 s, 137 MB/s 2024-04-25T08:39:24.246 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.246 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.246 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072872 s, 144 MB/s 2024-04-25T08:39:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.256 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802562 s, 131 MB/s 2024-04-25T08:39:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.266 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821851 s, 128 MB/s 2024-04-25T08:39:24.276 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.276 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.276 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718296 s, 146 MB/s 2024-04-25T08:39:24.285 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.285 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.285 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736858 s, 142 MB/s 2024-04-25T08:39:24.294 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.295 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.295 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714708 s, 147 MB/s 2024-04-25T08:39:24.305 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.305 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.305 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832643 s, 126 MB/s 2024-04-25T08:39:24.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.314 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657926 s, 159 MB/s 2024-04-25T08:39:24.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.324 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708728 s, 148 MB/s 2024-04-25T08:39:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.333 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736459 s, 142 MB/s 2024-04-25T08:39:24.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.343 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717824 s, 146 MB/s 2024-04-25T08:39:24.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.352 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721723 s, 145 MB/s 2024-04-25T08:39:24.361 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.361 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.361 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705718 s, 149 MB/s 2024-04-25T08:39:24.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.373 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921292 s, 114 MB/s 2024-04-25T08:39:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.392 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702482 s, 149 MB/s 2024-04-25T08:39:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.401 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690725 s, 152 MB/s 2024-04-25T08:39:24.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.411 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813171 s, 129 MB/s 2024-04-25T08:39:24.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:24.413 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:24.413 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:24.413 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T08:39:24.413 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:24.413 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:24.413 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:24.413 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:24.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.480 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.480 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649277 s, 161 MB/s 2024-04-25T08:39:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803846 s, 130 MB/s 2024-04-25T08:39:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.499 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725272 s, 145 MB/s 2024-04-25T08:39:24.509 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.509 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.509 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735492 s, 143 MB/s 2024-04-25T08:39:24.518 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.518 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.518 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695179 s, 151 MB/s 2024-04-25T08:39:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.528 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703931 s, 149 MB/s 2024-04-25T08:39:24.537 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.537 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.537 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745775 s, 141 MB/s 2024-04-25T08:39:24.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.547 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754469 s, 139 MB/s 2024-04-25T08:39:24.556 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.557 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715928 s, 146 MB/s 2024-04-25T08:39:24.566 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.566 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.567 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746792 s, 140 MB/s 2024-04-25T08:39:24.576 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.576 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.576 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758147 s, 138 MB/s 2024-04-25T08:39:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.586 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803766 s, 130 MB/s 2024-04-25T08:39:24.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.598 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087936 s, 119 MB/s 2024-04-25T08:39:24.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.607 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759928 s, 138 MB/s 2024-04-25T08:39:24.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.617 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722362 s, 145 MB/s 2024-04-25T08:39:24.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.626 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696567 s, 151 MB/s 2024-04-25T08:39:24.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.636 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729756 s, 144 MB/s 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.645 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700631 s, 150 MB/s 2024-04-25T08:39:24.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.655 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728141 s, 144 MB/s 2024-04-25T08:39:24.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.666 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848306 s, 124 MB/s 2024-04-25T08:39:24.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.677 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860144 s, 122 MB/s 2024-04-25T08:39:24.686 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.686 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.686 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701444 s, 149 MB/s 2024-04-25T08:39:24.695 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.696 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.696 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761637 s, 138 MB/s 2024-04-25T08:39:24.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.705 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068274 s, 154 MB/s 2024-04-25T08:39:24.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.715 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744835 s, 141 MB/s 2024-04-25T08:39:24.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.724 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718788 s, 146 MB/s 2024-04-25T08:39:24.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.734 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722368 s, 145 MB/s 2024-04-25T08:39:24.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.743 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723152 s, 145 MB/s 2024-04-25T08:39:24.753 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.753 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.753 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809596 s, 130 MB/s 2024-04-25T08:39:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.763 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722033 s, 145 MB/s 2024-04-25T08:39:24.773 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.773 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.774 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825829 s, 127 MB/s 2024-04-25T08:39:24.783 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.783 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.783 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722858 s, 145 MB/s 2024-04-25T08:39:24.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:24.784 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:24.784 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:24.785 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T08:39:24.785 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:24.785 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:24.785 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:24.785 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:24.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.853 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751888 s, 139 MB/s 2024-04-25T08:39:24.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.863 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763308 s, 137 MB/s 2024-04-25T08:39:24.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.873 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722392 s, 145 MB/s 2024-04-25T08:39:24.883 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.883 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.883 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814717 s, 129 MB/s 2024-04-25T08:39:24.892 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.893 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.893 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734405 s, 143 MB/s 2024-04-25T08:39:24.902 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.902 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.902 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729957 s, 144 MB/s 2024-04-25T08:39:24.912 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.912 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.912 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792537 s, 132 MB/s 2024-04-25T08:39:24.922 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.922 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.922 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708394 s, 148 MB/s 2024-04-25T08:39:24.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.931 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727452 s, 144 MB/s 2024-04-25T08:39:24.942 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.942 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.942 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866035 s, 121 MB/s 2024-04-25T08:39:24.952 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.952 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.952 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748724 s, 140 MB/s 2024-04-25T08:39:24.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.961 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665205 s, 158 MB/s 2024-04-25T08:39:24.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.972 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082066 s, 128 MB/s 2024-04-25T08:39:24.983 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.983 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.983 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081885 s, 128 MB/s 2024-04-25T08:39:24.993 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:24.993 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:24.993 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816778 s, 128 MB/s 2024-04-25T08:39:25.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.002 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733152 s, 143 MB/s 2024-04-25T08:39:25.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.012 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766139 s, 137 MB/s 2024-04-25T08:39:25.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.022 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758306 s, 138 MB/s 2024-04-25T08:39:25.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.032 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729558 s, 144 MB/s 2024-04-25T08:39:25.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.041 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069581 s, 151 MB/s 2024-04-25T08:39:25.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.051 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727091 s, 144 MB/s 2024-04-25T08:39:25.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.062 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817554 s, 128 MB/s 2024-04-25T08:39:25.071 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.071 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.071 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721228 s, 145 MB/s 2024-04-25T08:39:25.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.080 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718121 s, 146 MB/s 2024-04-25T08:39:25.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.090 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715289 s, 147 MB/s 2024-04-25T08:39:25.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.099 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751428 s, 140 MB/s 2024-04-25T08:39:25.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.109 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759168 s, 138 MB/s 2024-04-25T08:39:25.119 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.119 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.119 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759749 s, 138 MB/s 2024-04-25T08:39:25.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.128 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068861 s, 152 MB/s 2024-04-25T08:39:25.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.138 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740996 s, 142 MB/s 2024-04-25T08:39:25.147 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.147 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.147 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681539 s, 154 MB/s 2024-04-25T08:39:25.157 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.157 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.157 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731057 s, 143 MB/s 2024-04-25T08:39:25.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:25.158 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:25.158 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:25.158 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T08:39:25.158 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:25.159 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:25.159 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:25.159 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:25.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.225 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681414 s, 154 MB/s 2024-04-25T08:39:25.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.234 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680949 s, 154 MB/s 2024-04-25T08:39:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817689 s, 128 MB/s 2024-04-25T08:39:25.255 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.255 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.255 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753294 s, 139 MB/s 2024-04-25T08:39:25.264 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.264 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.264 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662409 s, 158 MB/s 2024-04-25T08:39:25.272 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.272 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.273 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654446 s, 160 MB/s 2024-04-25T08:39:25.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.283 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820435 s, 128 MB/s 2024-04-25T08:39:25.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.293 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731422 s, 143 MB/s 2024-04-25T08:39:25.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.303 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825878 s, 127 MB/s 2024-04-25T08:39:25.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.313 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074228 s, 141 MB/s 2024-04-25T08:39:25.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.323 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758159 s, 138 MB/s 2024-04-25T08:39:25.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.333 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756514 s, 139 MB/s 2024-04-25T08:39:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806323 s, 130 MB/s 2024-04-25T08:39:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.352 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737933 s, 142 MB/s 2024-04-25T08:39:25.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.362 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713485 s, 147 MB/s 2024-04-25T08:39:25.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.373 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890566 s, 118 MB/s 2024-04-25T08:39:25.406 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.406 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.406 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668508 s, 157 MB/s 2024-04-25T08:39:25.416 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.417 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833958 s, 126 MB/s 2024-04-25T08:39:25.426 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.426 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.426 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724565 s, 145 MB/s 2024-04-25T08:39:25.436 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.436 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.436 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718539 s, 146 MB/s 2024-04-25T08:39:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.445 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693512 s, 151 MB/s 2024-04-25T08:39:25.455 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.455 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.455 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752783 s, 139 MB/s 2024-04-25T08:39:25.464 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.464 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.464 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692228 s, 151 MB/s 2024-04-25T08:39:25.473 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.474 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.474 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672762 s, 156 MB/s 2024-04-25T08:39:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.483 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067751 s, 155 MB/s 2024-04-25T08:39:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736812 s, 142 MB/s 2024-04-25T08:39:25.501 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.502 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.502 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690067 s, 152 MB/s 2024-04-25T08:39:25.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.511 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709865 s, 148 MB/s 2024-04-25T08:39:25.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.520 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726692 s, 144 MB/s 2024-04-25T08:39:25.530 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.530 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.531 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745463 s, 141 MB/s 2024-04-25T08:39:25.540 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.540 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.540 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071291 s, 147 MB/s 2024-04-25T08:39:25.549 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.549 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.549 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741689 s, 141 MB/s 2024-04-25T08:39:25.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:25.551 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:25.551 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:25.551 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T08:39:25.551 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:25.551 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:25.551 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:25.552 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:25.619 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.620 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755311 s, 139 MB/s 2024-04-25T08:39:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.629 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736764 s, 142 MB/s 2024-04-25T08:39:25.639 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.639 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.640 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830967 s, 126 MB/s 2024-04-25T08:39:25.650 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.650 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.650 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818515 s, 128 MB/s 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.659 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719522 s, 146 MB/s 2024-04-25T08:39:25.670 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.670 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.670 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824795 s, 127 MB/s 2024-04-25T08:39:25.679 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.679 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.679 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695051 s, 151 MB/s 2024-04-25T08:39:25.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.688 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066055 s, 159 MB/s 2024-04-25T08:39:25.699 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.699 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.699 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846591 s, 124 MB/s 2024-04-25T08:39:25.709 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.709 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.709 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077552 s, 135 MB/s 2024-04-25T08:39:25.718 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.718 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.718 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681248 s, 154 MB/s 2024-04-25T08:39:25.730 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.730 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.730 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901399 s, 116 MB/s 2024-04-25T08:39:25.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.740 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739404 s, 142 MB/s 2024-04-25T08:39:25.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.749 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074639 s, 140 MB/s 2024-04-25T08:39:25.759 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.759 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.759 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750264 s, 140 MB/s 2024-04-25T08:39:25.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.769 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726331 s, 144 MB/s 2024-04-25T08:39:25.777 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.777 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.777 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658906 s, 159 MB/s 2024-04-25T08:39:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.788 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845908 s, 124 MB/s 2024-04-25T08:39:25.798 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.798 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.799 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817808 s, 128 MB/s 2024-04-25T08:39:25.808 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.808 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.809 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751839 s, 139 MB/s 2024-04-25T08:39:25.817 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.818 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.818 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691061 s, 152 MB/s 2024-04-25T08:39:25.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.829 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866153 s, 121 MB/s 2024-04-25T08:39:25.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.839 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755521 s, 139 MB/s 2024-04-25T08:39:25.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.858 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160219 s, 65.4 MB/s 2024-04-25T08:39:25.868 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.868 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.868 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823081 s, 127 MB/s 2024-04-25T08:39:25.878 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.878 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.878 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765679 s, 137 MB/s 2024-04-25T08:39:25.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.887 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689441 s, 152 MB/s 2024-04-25T08:39:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762043 s, 138 MB/s 2024-04-25T08:39:25.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.907 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.907 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732132 s, 143 MB/s 2024-04-25T08:39:25.916 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.917 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748202 s, 140 MB/s 2024-04-25T08:39:25.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.927 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759665 s, 138 MB/s 2024-04-25T08:39:25.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:25.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:25.936 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752747 s, 139 MB/s 2024-04-25T08:39:25.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:25.937 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:25.937 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:25.937 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T08:39:25.937 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:25.937 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:25.937 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:25.937 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:26.006 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.006 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.006 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781517 s, 134 MB/s 2024-04-25T08:39:26.014 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.014 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.015 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667364 s, 157 MB/s 2024-04-25T08:39:26.024 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.024 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.024 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711256 s, 147 MB/s 2024-04-25T08:39:26.035 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.035 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.035 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830823 s, 126 MB/s 2024-04-25T08:39:26.044 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.044 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.045 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736774 s, 142 MB/s 2024-04-25T08:39:26.054 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.054 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.054 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732502 s, 143 MB/s 2024-04-25T08:39:26.064 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.064 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.064 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810888 s, 129 MB/s 2024-04-25T08:39:26.074 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.075 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.075 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801102 s, 131 MB/s 2024-04-25T08:39:26.084 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.084 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.084 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725375 s, 145 MB/s 2024-04-25T08:39:26.093 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.094 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.094 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724087 s, 145 MB/s 2024-04-25T08:39:26.103 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.103 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.103 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073698 s, 142 MB/s 2024-04-25T08:39:26.112 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.113 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.113 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716415 s, 146 MB/s 2024-04-25T08:39:26.121 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.121 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.121 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00642484 s, 163 MB/s 2024-04-25T08:39:26.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.131 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.131 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699272 s, 150 MB/s 2024-04-25T08:39:26.140 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.140 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.140 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696195 s, 151 MB/s 2024-04-25T08:39:26.151 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.151 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.151 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856025 s, 122 MB/s 2024-04-25T08:39:26.160 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.160 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.160 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712058 s, 147 MB/s 2024-04-25T08:39:26.170 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.171 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.171 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814752 s, 129 MB/s 2024-04-25T08:39:26.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.182 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872566 s, 120 MB/s 2024-04-25T08:39:26.192 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.192 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.192 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815492 s, 129 MB/s 2024-04-25T08:39:26.201 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.202 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.202 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695193 s, 151 MB/s 2024-04-25T08:39:26.211 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.211 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.211 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069637 s, 151 MB/s 2024-04-25T08:39:26.222 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.222 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.222 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828077 s, 127 MB/s 2024-04-25T08:39:26.233 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.233 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.233 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085273 s, 123 MB/s 2024-04-25T08:39:26.243 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.244 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.244 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869277 s, 121 MB/s 2024-04-25T08:39:26.253 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.253 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.253 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725722 s, 144 MB/s 2024-04-25T08:39:26.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.264 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808386 s, 130 MB/s 2024-04-25T08:39:26.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.273 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739328 s, 142 MB/s 2024-04-25T08:39:26.284 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.284 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.284 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834457 s, 126 MB/s 2024-04-25T08:39:26.294 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.295 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.295 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807096 s, 130 MB/s 2024-04-25T08:39:26.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.304 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759861 s, 138 MB/s 2024-04-25T08:39:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.314 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751427 s, 140 MB/s 2024-04-25T08:39:26.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:26.315 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:26.315 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:26.315 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T08:39:26.315 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:26.315 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:26.315 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:26.315 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:26.413 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.413 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.413 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662649 s, 158 MB/s 2024-04-25T08:39:26.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.422 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670628 s, 156 MB/s 2024-04-25T08:39:26.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.432 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754815 s, 139 MB/s 2024-04-25T08:39:26.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.442 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076179 s, 138 MB/s 2024-04-25T08:39:26.452 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.452 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.452 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007875 s, 133 MB/s 2024-04-25T08:39:26.462 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.462 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.462 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757926 s, 138 MB/s 2024-04-25T08:39:26.472 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.472 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.472 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740569 s, 142 MB/s 2024-04-25T08:39:26.481 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.481 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.481 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718125 s, 146 MB/s 2024-04-25T08:39:26.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.491 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739513 s, 142 MB/s 2024-04-25T08:39:26.501 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.501 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.501 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795848 s, 132 MB/s 2024-04-25T08:39:26.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.510 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071987 s, 146 MB/s 2024-04-25T08:39:26.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.520 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732719 s, 143 MB/s 2024-04-25T08:39:26.530 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.530 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.530 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807179 s, 130 MB/s 2024-04-25T08:39:26.539 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.539 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.539 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066223 s, 158 MB/s 2024-04-25T08:39:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.549 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744298 s, 141 MB/s 2024-04-25T08:39:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826178 s, 127 MB/s 2024-04-25T08:39:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.569 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658416 s, 159 MB/s 2024-04-25T08:39:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.580 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840668 s, 125 MB/s 2024-04-25T08:39:26.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.589 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693168 s, 151 MB/s 2024-04-25T08:39:26.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.598 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700177 s, 150 MB/s 2024-04-25T08:39:26.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.609 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802996 s, 131 MB/s 2024-04-25T08:39:26.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.618 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745118 s, 141 MB/s 2024-04-25T08:39:26.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.629 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809126 s, 130 MB/s 2024-04-25T08:39:26.638 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.639 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724287 s, 145 MB/s 2024-04-25T08:39:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698113 s, 150 MB/s 2024-04-25T08:39:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742748 s, 141 MB/s 2024-04-25T08:39:26.667 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.667 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.667 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726154 s, 144 MB/s 2024-04-25T08:39:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.677 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778216 s, 135 MB/s 2024-04-25T08:39:26.687 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.687 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.687 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736323 s, 142 MB/s 2024-04-25T08:39:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.697 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794865 s, 132 MB/s 2024-04-25T08:39:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.707 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747129 s, 140 MB/s 2024-04-25T08:39:26.716 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.716 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.716 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687776 s, 152 MB/s 2024-04-25T08:39:26.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:26.717 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:26.717 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:26.717 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T08:39:26.717 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:26.717 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:26.717 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:26.717 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:26.746 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.746 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.746 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766489 s, 137 MB/s 2024-04-25T08:39:26.756 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.757 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.757 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818935 s, 128 MB/s 2024-04-25T08:39:26.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.765 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067598 s, 155 MB/s 2024-04-25T08:39:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.775 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720147 s, 146 MB/s 2024-04-25T08:39:26.783 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.784 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664668 s, 158 MB/s 2024-04-25T08:39:26.794 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.794 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.794 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807203 s, 130 MB/s 2024-04-25T08:39:26.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.804 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.804 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726831 s, 144 MB/s 2024-04-25T08:39:26.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.814 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777967 s, 135 MB/s 2024-04-25T08:39:26.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.824 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838985 s, 125 MB/s 2024-04-25T08:39:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.835 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.835 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748484 s, 140 MB/s 2024-04-25T08:39:26.845 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.845 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.845 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816764 s, 128 MB/s 2024-04-25T08:39:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762341 s, 138 MB/s 2024-04-25T08:39:26.864 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.865 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.865 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727152 s, 144 MB/s 2024-04-25T08:39:26.874 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.874 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.874 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749075 s, 140 MB/s 2024-04-25T08:39:26.884 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.884 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.884 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072587 s, 144 MB/s 2024-04-25T08:39:26.894 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.895 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855695 s, 123 MB/s 2024-04-25T08:39:26.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.906 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840891 s, 125 MB/s 2024-04-25T08:39:26.915 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.916 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.916 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814996 s, 129 MB/s 2024-04-25T08:39:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756255 s, 139 MB/s 2024-04-25T08:39:26.935 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.935 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.935 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725427 s, 145 MB/s 2024-04-25T08:39:26.945 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.945 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.945 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725839 s, 144 MB/s 2024-04-25T08:39:26.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.956 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844291 s, 124 MB/s 2024-04-25T08:39:26.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.965 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735545 s, 143 MB/s 2024-04-25T08:39:26.975 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.975 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.975 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728618 s, 144 MB/s 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.984 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722725 s, 145 MB/s 2024-04-25T08:39:26.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:26.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:26.994 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727125 s, 144 MB/s 2024-04-25T08:39:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.002 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00650973 s, 161 MB/s 2024-04-25T08:39:27.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.012 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716857 s, 146 MB/s 2024-04-25T08:39:27.023 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.023 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.023 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817774 s, 128 MB/s 2024-04-25T08:39:27.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.032 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069938 s, 150 MB/s 2024-04-25T08:39:27.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.041 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698235 s, 150 MB/s 2024-04-25T08:39:27.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.051 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731676 s, 143 MB/s 2024-04-25T08:39:27.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:27.052 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:27.053 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:27.053 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T08:39:27.053 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:27.053 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:27.053 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:27.053 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800997 s, 131 MB/s 2024-04-25T08:39:27.131 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.131 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.131 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713899 s, 147 MB/s 2024-04-25T08:39:27.140 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.140 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.140 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070997 s, 148 MB/s 2024-04-25T08:39:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.150 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747072 s, 140 MB/s 2024-04-25T08:39:27.159 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.159 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.159 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070038 s, 150 MB/s 2024-04-25T08:39:27.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.168 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660788 s, 159 MB/s 2024-04-25T08:39:27.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.178 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737494 s, 142 MB/s 2024-04-25T08:39:27.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.188 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774839 s, 135 MB/s 2024-04-25T08:39:27.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.199 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828037 s, 127 MB/s 2024-04-25T08:39:27.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.208 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696064 s, 151 MB/s 2024-04-25T08:39:27.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.219 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.219 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823775 s, 127 MB/s 2024-04-25T08:39:27.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.228 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735893 s, 142 MB/s 2024-04-25T08:39:27.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.238 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755292 s, 139 MB/s 2024-04-25T08:39:27.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.248 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745058 s, 141 MB/s 2024-04-25T08:39:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.257 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718447 s, 146 MB/s 2024-04-25T08:39:27.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.267 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786067 s, 133 MB/s 2024-04-25T08:39:27.276 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.276 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.276 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704207 s, 149 MB/s 2024-04-25T08:39:27.286 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.286 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.286 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731198 s, 143 MB/s 2024-04-25T08:39:27.296 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.296 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.296 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742171 s, 141 MB/s 2024-04-25T08:39:27.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.314 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0149614 s, 70.1 MB/s 2024-04-25T08:39:27.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.323 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744829 s, 141 MB/s 2024-04-25T08:39:27.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.333 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735881 s, 142 MB/s 2024-04-25T08:39:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.343 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738957 s, 142 MB/s 2024-04-25T08:39:27.351 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.352 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066292 s, 158 MB/s 2024-04-25T08:39:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.362 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816898 s, 128 MB/s 2024-04-25T08:39:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.372 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708818 s, 148 MB/s 2024-04-25T08:39:27.383 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.383 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.383 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934973 s, 112 MB/s 2024-04-25T08:39:27.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.414 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710331 s, 148 MB/s 2024-04-25T08:39:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.423 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075654 s, 139 MB/s 2024-04-25T08:39:27.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.433 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695838 s, 151 MB/s 2024-04-25T08:39:27.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.443 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810945 s, 129 MB/s 2024-04-25T08:39:27.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.453 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744684 s, 141 MB/s 2024-04-25T08:39:27.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:27.454 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:27.454 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:27.454 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T08:39:27.454 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:27.454 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:27.455 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:27.455 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:27.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.523 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822948 s, 127 MB/s 2024-04-25T08:39:27.532 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.532 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.532 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729835 s, 144 MB/s 2024-04-25T08:39:27.541 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.541 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.541 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666688 s, 157 MB/s 2024-04-25T08:39:27.551 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.551 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.551 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745797 s, 141 MB/s 2024-04-25T08:39:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072646 s, 144 MB/s 2024-04-25T08:39:27.570 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.571 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.571 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817485 s, 128 MB/s 2024-04-25T08:39:27.580 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.580 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.580 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750369 s, 140 MB/s 2024-04-25T08:39:27.590 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.590 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.590 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738104 s, 142 MB/s 2024-04-25T08:39:27.599 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.599 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.599 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074116 s, 141 MB/s 2024-04-25T08:39:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813543 s, 129 MB/s 2024-04-25T08:39:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074679 s, 140 MB/s 2024-04-25T08:39:27.629 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.629 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.629 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744553 s, 141 MB/s 2024-04-25T08:39:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795678 s, 132 MB/s 2024-04-25T08:39:27.648 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.648 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.648 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658484 s, 159 MB/s 2024-04-25T08:39:27.658 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.658 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.658 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763264 s, 137 MB/s 2024-04-25T08:39:27.668 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.668 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.668 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810802 s, 129 MB/s 2024-04-25T08:39:27.678 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.678 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.678 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727002 s, 144 MB/s 2024-04-25T08:39:27.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.688 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755587 s, 139 MB/s 2024-04-25T08:39:27.697 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.697 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.698 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748026 s, 140 MB/s 2024-04-25T08:39:27.707 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.707 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.707 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738164 s, 142 MB/s 2024-04-25T08:39:27.717 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.717 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.717 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796063 s, 132 MB/s 2024-04-25T08:39:27.726 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.726 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.727 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714392 s, 147 MB/s 2024-04-25T08:39:27.736 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.736 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.736 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739155 s, 142 MB/s 2024-04-25T08:39:27.746 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.746 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.746 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809783 s, 129 MB/s 2024-04-25T08:39:27.756 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.756 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.756 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071998 s, 146 MB/s 2024-04-25T08:39:27.766 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.766 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.766 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783244 s, 134 MB/s 2024-04-25T08:39:27.776 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.776 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.776 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814162 s, 129 MB/s 2024-04-25T08:39:27.786 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.786 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.786 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072991 s, 144 MB/s 2024-04-25T08:39:27.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.796 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762745 s, 137 MB/s 2024-04-25T08:39:27.805 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.805 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.805 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741018 s, 142 MB/s 2024-04-25T08:39:27.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.814 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677363 s, 155 MB/s 2024-04-25T08:39:27.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.823 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684028 s, 153 MB/s 2024-04-25T08:39:27.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:27.824 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:27.824 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:27.825 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T08:39:27.825 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:27.825 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:27.825 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:27.825 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:27.893 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.894 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.894 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812886 s, 129 MB/s 2024-04-25T08:39:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.903 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748945 s, 140 MB/s 2024-04-25T08:39:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.914 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008026 s, 131 MB/s 2024-04-25T08:39:27.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.923 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731152 s, 143 MB/s 2024-04-25T08:39:27.932 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.933 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718012 s, 146 MB/s 2024-04-25T08:39:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.942 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734984 s, 143 MB/s 2024-04-25T08:39:27.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.954 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00952437 s, 110 MB/s 2024-04-25T08:39:27.964 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.964 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.964 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747887 s, 140 MB/s 2024-04-25T08:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.973 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699895 s, 150 MB/s 2024-04-25T08:39:27.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.982 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066147 s, 159 MB/s 2024-04-25T08:39:27.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:27.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:27.991 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690155 s, 152 MB/s 2024-04-25T08:39:28.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.001 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793581 s, 132 MB/s 2024-04-25T08:39:28.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.011 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753398 s, 139 MB/s 2024-04-25T08:39:28.020 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.021 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.021 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740054 s, 142 MB/s 2024-04-25T08:39:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.031 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808719 s, 130 MB/s 2024-04-25T08:39:28.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.041 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080507 s, 130 MB/s 2024-04-25T08:39:28.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.051 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714123 s, 147 MB/s 2024-04-25T08:39:28.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.060 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733643 s, 143 MB/s 2024-04-25T08:39:28.069 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.069 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.070 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703222 s, 149 MB/s 2024-04-25T08:39:28.079 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.079 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.079 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707068 s, 148 MB/s 2024-04-25T08:39:28.088 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.089 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.089 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750086 s, 140 MB/s 2024-04-25T08:39:28.098 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797885 s, 131 MB/s 2024-04-25T08:39:28.108 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.108 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.108 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680965 s, 154 MB/s 2024-04-25T08:39:28.116 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.117 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.117 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655208 s, 160 MB/s 2024-04-25T08:39:28.125 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.126 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.126 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667218 s, 157 MB/s 2024-04-25T08:39:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839206 s, 125 MB/s 2024-04-25T08:39:28.146 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.146 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.146 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701302 s, 150 MB/s 2024-04-25T08:39:28.157 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.157 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.157 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936383 s, 112 MB/s 2024-04-25T08:39:28.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.168 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827597 s, 127 MB/s 2024-04-25T08:39:28.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.178 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780468 s, 134 MB/s 2024-04-25T08:39:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.188 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758433 s, 138 MB/s 2024-04-25T08:39:28.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.197 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667008 s, 157 MB/s 2024-04-25T08:39:28.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:28.199 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:28.199 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:28.199 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T08:39:28.199 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:28.199 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:28.199 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:28.199 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:28.308 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.309 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682138 s, 154 MB/s 2024-04-25T08:39:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699621 s, 150 MB/s 2024-04-25T08:39:28.328 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.328 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.328 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743365 s, 141 MB/s 2024-04-25T08:39:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080535 s, 130 MB/s 2024-04-25T08:39:28.348 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.348 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.348 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729437 s, 144 MB/s 2024-04-25T08:39:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.358 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814412 s, 129 MB/s 2024-04-25T08:39:28.367 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.368 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.368 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679798 s, 154 MB/s 2024-04-25T08:39:28.377 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.378 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740767 s, 142 MB/s 2024-04-25T08:39:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.393 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753939 s, 139 MB/s 2024-04-25T08:39:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.420 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762877 s, 137 MB/s 2024-04-25T08:39:28.430 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.430 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.430 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764232 s, 137 MB/s 2024-04-25T08:39:28.439 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.440 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.440 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721521 s, 145 MB/s 2024-04-25T08:39:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.449 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668039 s, 157 MB/s 2024-04-25T08:39:28.458 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.458 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.458 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682337 s, 154 MB/s 2024-04-25T08:39:28.468 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.468 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.468 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707319 s, 148 MB/s 2024-04-25T08:39:28.478 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.478 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.478 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730779 s, 143 MB/s 2024-04-25T08:39:28.487 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.488 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.488 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730492 s, 144 MB/s 2024-04-25T08:39:28.497 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.497 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.497 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702309 s, 149 MB/s 2024-04-25T08:39:28.508 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.508 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.508 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797512 s, 131 MB/s 2024-04-25T08:39:28.518 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.519 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.519 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827831 s, 127 MB/s 2024-04-25T08:39:28.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.528 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672974 s, 156 MB/s 2024-04-25T08:39:28.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.538 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735321 s, 143 MB/s 2024-04-25T08:39:28.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.547 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669528 s, 157 MB/s 2024-04-25T08:39:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008074 s, 130 MB/s 2024-04-25T08:39:28.567 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.567 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.567 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645724 s, 162 MB/s 2024-04-25T08:39:28.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.577 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732891 s, 143 MB/s 2024-04-25T08:39:28.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.587 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810041 s, 129 MB/s 2024-04-25T08:39:28.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.599 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.599 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859304 s, 122 MB/s 2024-04-25T08:39:28.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.609 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753171 s, 139 MB/s 2024-04-25T08:39:28.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.620 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834936 s, 126 MB/s 2024-04-25T08:39:28.629 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.629 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.629 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676094 s, 155 MB/s 2024-04-25T08:39:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785132 s, 134 MB/s 2024-04-25T08:39:28.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:28.641 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:28.641 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:28.641 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T08:39:28.641 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:28.641 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:28.641 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:28.641 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:28.670 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.670 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.670 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769377 s, 136 MB/s 2024-04-25T08:39:28.680 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.680 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.680 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763163 s, 137 MB/s 2024-04-25T08:39:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.691 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836988 s, 125 MB/s 2024-04-25T08:39:28.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.702 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840076 s, 125 MB/s 2024-04-25T08:39:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.712 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753247 s, 139 MB/s 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.723 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082689 s, 127 MB/s 2024-04-25T08:39:28.734 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.734 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.734 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766224 s, 137 MB/s 2024-04-25T08:39:28.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.744 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822011 s, 128 MB/s 2024-04-25T08:39:28.753 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.754 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656142 s, 160 MB/s 2024-04-25T08:39:28.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.763 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.763 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656212 s, 160 MB/s 2024-04-25T08:39:28.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.772 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696017 s, 151 MB/s 2024-04-25T08:39:28.783 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.783 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.783 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825353 s, 127 MB/s 2024-04-25T08:39:28.793 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.793 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.793 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738517 s, 142 MB/s 2024-04-25T08:39:28.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.803 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772359 s, 136 MB/s 2024-04-25T08:39:28.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.814 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854089 s, 123 MB/s 2024-04-25T08:39:28.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.824 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745324 s, 141 MB/s 2024-04-25T08:39:28.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.834 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748051 s, 140 MB/s 2024-04-25T08:39:28.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.843 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771046 s, 136 MB/s 2024-04-25T08:39:28.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.854 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821524 s, 128 MB/s 2024-04-25T08:39:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672551 s, 156 MB/s 2024-04-25T08:39:28.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.873 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776467 s, 135 MB/s 2024-04-25T08:39:28.883 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.883 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.883 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802752 s, 131 MB/s 2024-04-25T08:39:28.894 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.894 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.894 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791397 s, 132 MB/s 2024-04-25T08:39:28.904 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.904 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.904 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008268 s, 127 MB/s 2024-04-25T08:39:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.914 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676507 s, 155 MB/s 2024-04-25T08:39:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.923 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739884 s, 142 MB/s 2024-04-25T08:39:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.933 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729565 s, 144 MB/s 2024-04-25T08:39:28.942 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.942 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.942 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708787 s, 148 MB/s 2024-04-25T08:39:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.952 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749937 s, 140 MB/s 2024-04-25T08:39:28.962 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.962 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.962 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815335 s, 129 MB/s 2024-04-25T08:39:28.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.972 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755189 s, 139 MB/s 2024-04-25T08:39:28.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:28.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:28.982 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730398 s, 144 MB/s 2024-04-25T08:39:28.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:28.983 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:28.983 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:28.983 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T08:39:28.983 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:28.983 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:28.983 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:28.984 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:29.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.052 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795631 s, 132 MB/s 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.061 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769714 s, 136 MB/s 2024-04-25T08:39:29.072 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.072 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.072 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832523 s, 126 MB/s 2024-04-25T08:39:29.082 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.082 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.082 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745771 s, 141 MB/s 2024-04-25T08:39:29.091 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.092 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.092 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739852 s, 142 MB/s 2024-04-25T08:39:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681007 s, 154 MB/s 2024-04-25T08:39:29.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.110 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072697 s, 144 MB/s 2024-04-25T08:39:29.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.120 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730487 s, 144 MB/s 2024-04-25T08:39:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813731 s, 129 MB/s 2024-04-25T08:39:29.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.139 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675106 s, 155 MB/s 2024-04-25T08:39:29.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682956 s, 154 MB/s 2024-04-25T08:39:29.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.158 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714132 s, 147 MB/s 2024-04-25T08:39:29.166 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.166 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.166 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639188 s, 164 MB/s 2024-04-25T08:39:29.176 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.177 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744681 s, 141 MB/s 2024-04-25T08:39:29.185 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.185 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.185 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707763 s, 148 MB/s 2024-04-25T08:39:29.196 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.196 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.196 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080042 s, 131 MB/s 2024-04-25T08:39:29.205 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.206 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744668 s, 141 MB/s 2024-04-25T08:39:29.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.215 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724017 s, 145 MB/s 2024-04-25T08:39:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667734 s, 157 MB/s 2024-04-25T08:39:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.234 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007296 s, 144 MB/s 2024-04-25T08:39:29.243 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.244 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.244 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071805 s, 146 MB/s 2024-04-25T08:39:29.253 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.253 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.253 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734754 s, 143 MB/s 2024-04-25T08:39:29.262 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.263 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727299 s, 144 MB/s 2024-04-25T08:39:29.272 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.273 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076397 s, 137 MB/s 2024-04-25T08:39:29.282 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.282 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.282 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716491 s, 146 MB/s 2024-04-25T08:39:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.292 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807141 s, 130 MB/s 2024-04-25T08:39:29.301 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.301 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.301 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686929 s, 153 MB/s 2024-04-25T08:39:29.310 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.311 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.311 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676572 s, 155 MB/s 2024-04-25T08:39:29.320 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.320 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.321 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748157 s, 140 MB/s 2024-04-25T08:39:29.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.330 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731872 s, 143 MB/s 2024-04-25T08:39:29.340 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.340 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.340 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716688 s, 146 MB/s 2024-04-25T08:39:29.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.349 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702643 s, 149 MB/s 2024-04-25T08:39:29.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:29.350 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:29.350 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:29.350 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T08:39:29.350 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:29.350 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:29.350 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:29.350 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:29.427 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.427 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.427 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718597 s, 146 MB/s 2024-04-25T08:39:29.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.437 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800882 s, 131 MB/s 2024-04-25T08:39:29.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.448 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826128 s, 127 MB/s 2024-04-25T08:39:29.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.457 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724889 s, 145 MB/s 2024-04-25T08:39:29.467 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.467 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.467 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729392 s, 144 MB/s 2024-04-25T08:39:29.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.476 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716153 s, 146 MB/s 2024-04-25T08:39:29.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.485 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712949 s, 147 MB/s 2024-04-25T08:39:29.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.494 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658901 s, 159 MB/s 2024-04-25T08:39:29.504 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.504 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.504 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070775 s, 148 MB/s 2024-04-25T08:39:29.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.513 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731311 s, 143 MB/s 2024-04-25T08:39:29.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.523 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744908 s, 141 MB/s 2024-04-25T08:39:29.532 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.533 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738495 s, 142 MB/s 2024-04-25T08:39:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.542 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737139 s, 142 MB/s 2024-04-25T08:39:29.551 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.551 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.551 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727349 s, 144 MB/s 2024-04-25T08:39:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.562 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709201 s, 148 MB/s 2024-04-25T08:39:29.569 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.570 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.570 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685619 s, 153 MB/s 2024-04-25T08:39:29.579 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.580 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747196 s, 140 MB/s 2024-04-25T08:39:29.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.589 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737651 s, 142 MB/s 2024-04-25T08:39:29.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.599 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.599 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720361 s, 146 MB/s 2024-04-25T08:39:29.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.608 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075604 s, 139 MB/s 2024-04-25T08:39:29.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.619 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806574 s, 130 MB/s 2024-04-25T08:39:29.627 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.628 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700148 s, 150 MB/s 2024-04-25T08:39:29.637 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.637 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.637 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745914 s, 141 MB/s 2024-04-25T08:39:29.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.648 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800206 s, 131 MB/s 2024-04-25T08:39:29.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.657 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702485 s, 149 MB/s 2024-04-25T08:39:29.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.667 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700824 s, 150 MB/s 2024-04-25T08:39:29.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.677 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771918 s, 136 MB/s 2024-04-25T08:39:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.686 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719308 s, 146 MB/s 2024-04-25T08:39:29.695 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.695 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.696 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729473 s, 144 MB/s 2024-04-25T08:39:29.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.705 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713958 s, 147 MB/s 2024-04-25T08:39:29.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.714 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666012 s, 157 MB/s 2024-04-25T08:39:29.723 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.723 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.724 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681414 s, 154 MB/s 2024-04-25T08:39:29.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:29.725 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:29.725 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:29.725 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T08:39:29.725 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:29.725 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:29.725 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:29.725 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.792 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721046 s, 145 MB/s 2024-04-25T08:39:29.802 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.802 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.803 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075292 s, 139 MB/s 2024-04-25T08:39:29.812 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.813 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083707 s, 125 MB/s 2024-04-25T08:39:29.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.824 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082003 s, 128 MB/s 2024-04-25T08:39:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.834 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083585 s, 125 MB/s 2024-04-25T08:39:29.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.844 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768387 s, 136 MB/s 2024-04-25T08:39:29.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.854 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743623 s, 141 MB/s 2024-04-25T08:39:29.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.863 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071911 s, 146 MB/s 2024-04-25T08:39:29.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.873 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767013 s, 137 MB/s 2024-04-25T08:39:29.883 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.883 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.883 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750759 s, 140 MB/s 2024-04-25T08:39:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.892 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667217 s, 157 MB/s 2024-04-25T08:39:29.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.901 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710095 s, 148 MB/s 2024-04-25T08:39:29.911 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.911 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.911 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801308 s, 131 MB/s 2024-04-25T08:39:29.921 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.921 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.922 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796196 s, 132 MB/s 2024-04-25T08:39:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.932 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820763 s, 128 MB/s 2024-04-25T08:39:29.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.941 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068669 s, 153 MB/s 2024-04-25T08:39:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.951 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727832 s, 144 MB/s 2024-04-25T08:39:29.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.961 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752616 s, 139 MB/s 2024-04-25T08:39:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773345 s, 136 MB/s 2024-04-25T08:39:29.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.980 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721167 s, 145 MB/s 2024-04-25T08:39:29.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.990 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736858 s, 142 MB/s 2024-04-25T08:39:29.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:29.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:29.999 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693345 s, 151 MB/s 2024-04-25T08:39:30.009 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.009 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.009 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772425 s, 136 MB/s 2024-04-25T08:39:30.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.018 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655925 s, 160 MB/s 2024-04-25T08:39:30.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.027 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00651762 s, 161 MB/s 2024-04-25T08:39:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.037 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082241 s, 128 MB/s 2024-04-25T08:39:30.047 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.047 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.047 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721711 s, 145 MB/s 2024-04-25T08:39:30.057 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.057 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.057 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760413 s, 138 MB/s 2024-04-25T08:39:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.067 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.068 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835712 s, 125 MB/s 2024-04-25T08:39:30.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.077 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728907 s, 144 MB/s 2024-04-25T08:39:30.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.087 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723584 s, 145 MB/s 2024-04-25T08:39:30.096 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.096 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.096 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672283 s, 156 MB/s 2024-04-25T08:39:30.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:30.097 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:30.097 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:30.097 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T08:39:30.097 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:30.097 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:30.097 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:30.097 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:30.166 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.166 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.166 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826456 s, 127 MB/s 2024-04-25T08:39:30.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.177 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083004 s, 126 MB/s 2024-04-25T08:39:30.186 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.186 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.186 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695237 s, 151 MB/s 2024-04-25T08:39:30.195 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.195 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.195 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660602 s, 159 MB/s 2024-04-25T08:39:30.204 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.205 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.205 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702603 s, 149 MB/s 2024-04-25T08:39:30.214 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.214 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.214 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716731 s, 146 MB/s 2024-04-25T08:39:30.223 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.223 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.223 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682874 s, 154 MB/s 2024-04-25T08:39:30.233 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.234 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823238 s, 127 MB/s 2024-04-25T08:39:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.243 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736805 s, 142 MB/s 2024-04-25T08:39:30.253 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.253 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.253 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777856 s, 135 MB/s 2024-04-25T08:39:30.262 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.263 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717983 s, 146 MB/s 2024-04-25T08:39:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730135 s, 144 MB/s 2024-04-25T08:39:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071539 s, 147 MB/s 2024-04-25T08:39:30.291 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.291 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.291 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730926 s, 143 MB/s 2024-04-25T08:39:30.300 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.300 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.300 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669785 s, 157 MB/s 2024-04-25T08:39:30.310 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.310 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.310 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082641 s, 127 MB/s 2024-04-25T08:39:30.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.320 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693199 s, 151 MB/s 2024-04-25T08:39:30.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.330 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742349 s, 141 MB/s 2024-04-25T08:39:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.339 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731273 s, 143 MB/s 2024-04-25T08:39:30.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.350 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.350 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826902 s, 127 MB/s 2024-04-25T08:39:30.359 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.359 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.359 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715972 s, 146 MB/s 2024-04-25T08:39:30.368 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.368 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.368 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724489 s, 145 MB/s 2024-04-25T08:39:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.378 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729518 s, 144 MB/s 2024-04-25T08:39:30.387 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.387 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.387 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712573 s, 147 MB/s 2024-04-25T08:39:30.400 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.400 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.400 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795668 s, 132 MB/s 2024-04-25T08:39:30.426 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.426 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.426 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073569 s, 143 MB/s 2024-04-25T08:39:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.437 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877537 s, 119 MB/s 2024-04-25T08:39:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.447 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761349 s, 138 MB/s 2024-04-25T08:39:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.457 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725097 s, 145 MB/s 2024-04-25T08:39:30.467 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.467 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.467 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796137 s, 132 MB/s 2024-04-25T08:39:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688654 s, 152 MB/s 2024-04-25T08:39:30.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.485 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674424 s, 155 MB/s 2024-04-25T08:39:30.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:30.486 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:39:30.486 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:39:30.486 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T08:39:30.486 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:39:30.487 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:39:30.487 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:39:30.487 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:39:30.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.555 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.555 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793531 s, 132 MB/s 2024-04-25T08:39:30.565 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.565 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.566 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868827 s, 121 MB/s 2024-04-25T08:39:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716202 s, 146 MB/s 2024-04-25T08:39:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073615 s, 142 MB/s 2024-04-25T08:39:30.594 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.594 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.594 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724817 s, 145 MB/s 2024-04-25T08:39:30.603 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.603 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.604 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692589 s, 151 MB/s 2024-04-25T08:39:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.613 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074146 s, 141 MB/s 2024-04-25T08:39:30.624 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.624 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.624 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084362 s, 124 MB/s 2024-04-25T08:39:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.634 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755101 s, 139 MB/s 2024-04-25T08:39:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.644 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766919 s, 137 MB/s 2024-04-25T08:39:30.652 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.652 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.652 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658623 s, 159 MB/s 2024-04-25T08:39:30.662 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.662 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.662 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705097 s, 149 MB/s 2024-04-25T08:39:30.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.672 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779726 s, 134 MB/s 2024-04-25T08:39:30.681 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.681 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.681 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700574 s, 150 MB/s 2024-04-25T08:39:30.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.691 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738949 s, 142 MB/s 2024-04-25T08:39:30.701 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.701 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.701 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756069 s, 139 MB/s 2024-04-25T08:39:30.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.710 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697224 s, 150 MB/s 2024-04-25T08:39:30.719 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.719 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.719 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661359 s, 159 MB/s 2024-04-25T08:39:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747824 s, 140 MB/s 2024-04-25T08:39:30.738 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.738 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.738 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684428 s, 153 MB/s 2024-04-25T08:39:30.747 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.748 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718809 s, 146 MB/s 2024-04-25T08:39:30.757 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.757 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.757 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073843 s, 142 MB/s 2024-04-25T08:39:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.768 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879764 s, 119 MB/s 2024-04-25T08:39:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.778 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068727 s, 153 MB/s 2024-04-25T08:39:30.790 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.790 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.790 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850345 s, 123 MB/s 2024-04-25T08:39:30.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.800 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838168 s, 125 MB/s 2024-04-25T08:39:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738371 s, 142 MB/s 2024-04-25T08:39:30.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710939 s, 147 MB/s 2024-04-25T08:39:30.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.830 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.830 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831848 s, 126 MB/s 2024-04-25T08:39:30.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.839 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688842 s, 152 MB/s 2024-04-25T08:39:30.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.849 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730737 s, 143 MB/s 2024-04-25T08:39:30.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:39:30.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:39:30.857 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066704 s, 157 MB/s 2024-04-25T08:39:30.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:30.859 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-25T08:39:31.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.010+0000 7fe4d662a640 1 Processor -- start 2024-04-25T08:39:31.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.010+0000 7fe4d662a640 1 -- start start 2024-04-25T08:39:31.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4d662a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe4d01612c0 0x7fe4d00b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4d662a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe4d00b6a00 0x7fe4d00d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4d662a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4d00d0870 0x7fe4d00d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4d662a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d00d11b0 con 0x7fe4d00b6a00 2024-04-25T08:39:31.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4d662a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d00b7200 con 0x7fe4d01612c0 2024-04-25T08:39:31.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4d662a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d00b74e0 con 0x7fe4d00d0870 2024-04-25T08:39:31.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4cf7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe4d00b6a00 0x7fe4d00d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4cf7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe4d00b6a00 0x7fe4d00d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42380/0 (socket says 172.21.15.38:42380) 2024-04-25T08:39:31.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/864185407 learned_addr learned my addr 172.21.15.38:0/864185407 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4d4bc7640 1 --2- 172.21.15.38:0/864185407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4d00d0870 0x7fe4d00d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4cffff640 1 --2- 172.21.15.38:0/864185407 >> v2:172.21.15.107:3300/0 conn(0x7fe4d01612c0 0x7fe4d00b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/864185407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4d00d0870 msgr2=0x7fe4d00d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/864185407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4d00d0870 0x7fe4d00d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/864185407 >> v2:172.21.15.107:3300/0 conn(0x7fe4d01612c0 msgr2=0x7fe4d00b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/864185407 >> v2:172.21.15.107:3300/0 conn(0x7fe4d01612c0 0x7fe4d00b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/864185407 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d00b77c0 con 0x7fe4d00b6a00 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4cffff640 1 --2- 172.21.15.38:0/864185407 >> v2:172.21.15.107:3300/0 conn(0x7fe4d01612c0 0x7fe4d00b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.011+0000 7fe4d4bc7640 1 --2- 172.21.15.38:0/864185407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4d00d0870 0x7fe4d00d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.012+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/864185407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00b6a00 0x7fe4d00d1c30 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00d49a0 tx=0x7fe4c002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22be18215180d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.012+0000 7fe4ceffd640 1 -- 172.21.15.38:0/864185407 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0037070 con 0x7fe4d00b6a00 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.012+0000 7fe4ceffd640 1 -- 172.21.15.38:0/864185407 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c0032440 con 0x7fe4d00b6a00 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.012+0000 7fe4ceffd640 1 -- 172.21.15.38:0/864185407 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0032810 con 0x7fe4d00b6a00 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.013+0000 7fe4d662a640 1 -- 172.21.15.38:0/864185407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00b6a00 msgr2=0x7fe4d00d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.013+0000 7fe4d662a640 1 --2- 172.21.15.38:0/864185407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00b6a00 0x7fe4d00d1c30 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00d49a0 tx=0x7fe4c002a140 comp rx=0 tx=0).stop 2024-04-25T08:39:31.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.013+0000 7fe4d662a640 1 -- 172.21.15.38:0/864185407 shutdown_connections 2024-04-25T08:39:31.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.013+0000 7fe4d662a640 1 --2- 172.21.15.38:0/864185407 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4d00d0870 0x7fe4d00d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.013+0000 7fe4d662a640 1 --2- 172.21.15.38:0/864185407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00b6a00 0x7fe4d00d1c30 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.013+0000 7fe4d662a640 1 --2- 172.21.15.38:0/864185407 >> v2:172.21.15.107:3300/0 conn(0x7fe4d01612c0 0x7fe4d00b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.013+0000 7fe4d662a640 1 -- 172.21.15.38:0/864185407 >> 172.21.15.38:0/864185407 conn(0x7fe4d0066d10 msgr2=0x7fe4d0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.013+0000 7fe4d662a640 1 -- 172.21.15.38:0/864185407 shutdown_connections 2024-04-25T08:39:31.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.013+0000 7fe4d662a640 1 -- 172.21.15.38:0/864185407 wait complete. 2024-04-25T08:39:31.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.014+0000 7fe4d662a640 1 Processor -- start 2024-04-25T08:39:31.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.014+0000 7fe4d662a640 1 -- start start 2024-04-25T08:39:31.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4d662a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00b6a00 0x7fe4d00cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4d662a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4d00d0870 0x7fe4d00cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4d662a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d01612c0 0x7fe4d00c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4d662a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d00d2170 con 0x7fe4d00b6a00 2024-04-25T08:39:31.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4d662a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d00b7d80 con 0x7fe4d01612c0 2024-04-25T08:39:31.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4d662a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d00b6e90 con 0x7fe4d00d0870 2024-04-25T08:39:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00b6a00 0x7fe4d00cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00b6a00 0x7fe4d00cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42382/0 (socket says 172.21.15.38:42382) 2024-04-25T08:39:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4cffff640 1 -- 172.21.15.38:0/2966255341 learned_addr learned my addr 172.21.15.38:0/2966255341 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4d00d0870 0x7fe4d00cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4d4bc7640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d01612c0 0x7fe4d00c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4cffff640 1 -- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4d00d0870 msgr2=0x7fe4d00cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4cffff640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4d00d0870 0x7fe4d00cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4cffff640 1 -- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d01612c0 msgr2=0x7fe4d00c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4cffff640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d01612c0 0x7fe4d00c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4cffff640 1 -- 172.21.15.38:0/2966255341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d00cd980 con 0x7fe4d00b6a00 2024-04-25T08:39:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4d4bc7640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d01612c0 0x7fe4d00c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.015+0000 7fe4cffff640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00b6a00 0x7fe4d00cbf30 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800d800 tx=0x7fe4b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.016+0000 7fe4ccff9640 1 -- 172.21.15.38:0/2966255341 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8004030 con 0x7fe4d00b6a00 2024-04-25T08:39:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.016+0000 7fe4ccff9640 1 -- 172.21.15.38:0/2966255341 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b80041d0 con 0x7fe4d00b6a00 2024-04-25T08:39:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.016+0000 7fe4ccff9640 1 -- 172.21.15.38:0/2966255341 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b80096a0 con 0x7fe4d00b6a00 2024-04-25T08:39:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.016+0000 7fe4d662a640 1 -- 172.21.15.38:0/2966255341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d00c60a0 con 0x7fe4d00b6a00 2024-04-25T08:39:31.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.016+0000 7fe4d662a640 1 -- 172.21.15.38:0/2966255341 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d00cb450 con 0x7fe4d00b6a00 2024-04-25T08:39:31.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.017+0000 7fe4ccff9640 1 -- 172.21.15.38:0/2966255341 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4b8009840 con 0x7fe4d00b6a00 2024-04-25T08:39:31.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.017+0000 7fe4ccff9640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe4b006b4d0 0x7fe4b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.018+0000 7fe4ccff9640 1 -- 172.21.15.38:0/2966255341 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe4b808a510 con 0x7fe4d00b6a00 2024-04-25T08:39:31.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.018+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe4b006b4d0 0x7fe4b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.018+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe4b006b4d0 0x7fe4b006d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0034000 tx=0x7fe4c002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.018+0000 7fe4d940a740 1 Processor -- start 2024-04-25T08:39:31.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.018+0000 7fe4d940a740 1 -- start start 2024-04-25T08:39:31.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.019+0000 7fe4d940a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe23200 0x55bc8fe25360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.019+0000 7fe4d940a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.019+0000 7fe4d940a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.019+0000 7fe4d940a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8fd87000 con 0x55bc8fe23200 2024-04-25T08:39:31.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.019+0000 7fe4d940a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8fd2ff20 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.019+0000 7fe4d940a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc8fd830a0 con 0x55bc8fe5f8d0 2024-04-25T08:39:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.019+0000 7fe4cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe23200 0x55bc8fe25360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.019+0000 7fe4cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe23200 0x55bc8fe25360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42394/0 (socket says 172.21.15.38:42394) 2024-04-25T08:39:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.019+0000 7fe4cffff640 1 -- 172.21.15.38:0/1483632236 learned_addr learned my addr 172.21.15.38:0/1483632236 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.019+0000 7fe4d4bc7640 1 --2- 172.21.15.38:0/1483632236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.019+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/1483632236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4cffff640 1 -- 172.21.15.38:0/1483632236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 msgr2=0x55bc8fe235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4cffff640 1 --2- 172.21.15.38:0/1483632236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4cffff640 1 -- 172.21.15.38:0/1483632236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe4dcc0 msgr2=0x55bc8fe4e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4cffff640 1 --2- 172.21.15.38:0/1483632236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4cffff640 1 -- 172.21.15.38:0/1483632236 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc8fb67ba0 con 0x55bc8fe23200 2024-04-25T08:39:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/1483632236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4cffff640 1 --2- 172.21.15.38:0/1483632236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe23200 0x55bc8fe25360 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8060cb0 tx=0x7fe4b8058cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555c45bd645d7f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4c5ffb640 1 -- 172.21.15.38:0/1483632236 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8056c20 con 0x55bc8fe23200 2024-04-25T08:39:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4c5ffb640 1 -- 172.21.15.38:0/1483632236 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b8056e80 con 0x55bc8fe23200 2024-04-25T08:39:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4c5ffb640 1 -- 172.21.15.38:0/1483632236 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b808a0c0 con 0x55bc8fe23200 2024-04-25T08:39:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4d940a740 1 -- 172.21.15.38:0/1483632236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe23200 msgr2=0x55bc8fe25360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4d940a740 1 --2- 172.21.15.38:0/1483632236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe23200 0x55bc8fe25360 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8060cb0 tx=0x7fe4b8058cc0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4d940a740 1 -- 172.21.15.38:0/1483632236 shutdown_connections 2024-04-25T08:39:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4d940a740 1 --2- 172.21.15.38:0/1483632236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4d940a740 1 --2- 172.21.15.38:0/1483632236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4d940a740 1 --2- 172.21.15.38:0/1483632236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe23200 0x55bc8fe25360 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4d940a740 1 -- 172.21.15.38:0/1483632236 >> 172.21.15.38:0/1483632236 conn(0x55bc8fdeb050 msgr2=0x55bc8fe68cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4d940a740 1 -- 172.21.15.38:0/1483632236 shutdown_connections 2024-04-25T08:39:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.020+0000 7fe4d940a740 1 -- 172.21.15.38:0/1483632236 wait complete. 2024-04-25T08:39:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.021+0000 7fe4d940a740 1 -- start start 2024-04-25T08:39:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.021+0000 7fe4d940a740 1 Processor -- start 2024-04-25T08:39:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4d940a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe497b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4d940a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4d940a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe50180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4d940a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8fd25610 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4d940a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8fe4b810 con 0x55bc8fe23200 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4d940a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc8fe6cc10 con 0x55bc8fe5f8d0 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4d940a740 10 client.0 _ll_register_callbacks cb 0x55bc8fd3df20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42398/0 (socket says 172.21.15.38:42398) 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4d4bc7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe50180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/819666815 learned_addr learned my addr 172.21.15.38:0/819666815 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4cffff640 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/819666815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 msgr2=0x55bc8fe50180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe50180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/819666815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 msgr2=0x55bc8fe497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/819666815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc8fa5b490 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.022+0000 7fe4cffff640 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4e140 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00090b0 tx=0x7fe4c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ef31d9cba850a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4aeffd640 1 -- 172.21.15.38:0/819666815 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c004d020 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4aeffd640 1 -- 172.21.15.38:0/819666815 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe4c0049860 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4d940a740 1 -- 172.21.15.38:0/819666815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4b8056e80 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4aeffd640 1 -- 172.21.15.38:0/819666815 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4c0057050 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4d940a740 10 client.8490 fetch_fsmap learned FSMap version 63 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4d940a740 1 -- 172.21.15.38:0/819666815 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55bc8fbc8330 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4aeffd640 1 -- 172.21.15.38:0/819666815 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b8056e80 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4aeffd640 1 -- 172.21.15.38:0/819666815 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe4c005a020 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4d940a740 10 client.8490 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4d940a740 10 client.8490 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T08:39:31.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4d940a740 20 client.8490 populate_metadata read hostname 'smithi038' 2024-04-25T08:39:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4d940a740 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55bc8fe69df0 0x55bc8fe6a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4d940a740 4 client.8490 mds_command: new command op to 8292 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:39:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.023+0000 7fe4d940a740 1 -- 172.21.15.38:0/819666815 --> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc8fd94540 con 0x55bc8fe69df0 2024-04-25T08:39:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.024+0000 7fe4cffff640 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55bc8fe69df0 0x55bc8fe6a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.024+0000 7fe4aeffd640 10 client.8490 ms_handle_connect on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.024+0000 7fe4cffff640 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55bc8fe69df0 0x55bc8fe6a220 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63412c198a6979ed server_cookie=9088bb5dbf69a2f9 in_seq=0 out_seq=0 2024-04-25T08:39:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.024+0000 7fe4aeffd640 1 -- 172.21.15.38:0/819666815 <== mds.0 v2:172.21.15.107:6835/1263010516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bc8fd94540 con 0x55bc8fe69df0 2024-04-25T08:39:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.024+0000 7fe4aeffd640 10 client.8490 handle_command_reply: tid=0 2024-04-25T08:39:31.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4d940a740 1 client.8490 shutdown 2024-04-25T08:39:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4d940a740 1 -- 172.21.15.38:0/819666815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 msgr2=0x55bc8fe4e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4d940a740 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4e140 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00090b0 tx=0x7fe4c0004730 comp rx=0 tx=0).stop 2024-04-25T08:39:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4d940a740 1 -- 172.21.15.38:0/819666815 shutdown_connections 2024-04-25T08:39:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4d940a740 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55bc8fe69df0 0x55bc8fe6a220 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4d940a740 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe50180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4aeffd640 0 client.8490 ms_handle_reset on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4d940a740 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4e140 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4d940a740 1 --2- 172.21.15.38:0/819666815 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4d940a740 1 -- 172.21.15.38:0/819666815 >> 172.21.15.38:0/819666815 conn(0x55bc8fdeb050 msgr2=0x55bc8fe21aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4d940a740 1 -- 172.21.15.38:0/819666815 shutdown_connections 2024-04-25T08:39:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4d940a740 1 -- 172.21.15.38:0/819666815 wait complete. 2024-04-25T08:39:31.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.025+0000 7fe4d940a740 20 client.8490 trim_cache size 0 max 16384 2024-04-25T08:39:31.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.040+0000 7fe4d940a740 1 Processor -- start 2024-04-25T08:39:31.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.040+0000 7fe4d940a740 1 -- start start 2024-04-25T08:39:31.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4d940a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe6a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4d940a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe62c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4d940a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe4fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4d940a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d0200380 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4d940a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d01ff520 con 0x55bc8fe23200 2024-04-25T08:39:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4d940a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d015d470 con 0x55bc8fe5f8d0 2024-04-25T08:39:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe62c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe62c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42402/0 (socket says 172.21.15.38:42402) 2024-04-25T08:39:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/2233820649 learned_addr learned my addr 172.21.15.38:0/2233820649 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4d4bc7640 1 --2- 172.21.15.38:0/2233820649 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe4fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4cffff640 1 --2- 172.21.15.38:0/2233820649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe6a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/2233820649 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 msgr2=0x55bc8fe4fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/2233820649 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe4fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/2233820649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 msgr2=0x55bc8fe6a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/2233820649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe6a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/2233820649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc8fa34610 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/2233820649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe62c70 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0000c00 tx=0x7fe4c0045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed1bc035bd8e406 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4c57fa640 1 -- 172.21.15.38:0/2233820649 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0060070 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4c57fa640 1 -- 172.21.15.38:0/2233820649 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c0049860 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.041+0000 7fe4c57fa640 1 -- 172.21.15.38:0/2233820649 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c004d040 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.042+0000 7fe4d940a740 1 -- 172.21.15.38:0/2233820649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 msgr2=0x55bc8fe62c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.042+0000 7fe4d940a740 1 --2- 172.21.15.38:0/2233820649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe62c70 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0000c00 tx=0x7fe4c0045000 comp rx=0 tx=0).stop 2024-04-25T08:39:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.042+0000 7fe4d940a740 1 -- 172.21.15.38:0/2233820649 shutdown_connections 2024-04-25T08:39:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.042+0000 7fe4d940a740 1 --2- 172.21.15.38:0/2233820649 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe4fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.042+0000 7fe4d940a740 1 --2- 172.21.15.38:0/2233820649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe62c70 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.042+0000 7fe4d940a740 1 --2- 172.21.15.38:0/2233820649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe6a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.042+0000 7fe4d940a740 1 -- 172.21.15.38:0/2233820649 >> 172.21.15.38:0/2233820649 conn(0x55bc8fdeb050 msgr2=0x55bc8fe19520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.042+0000 7fe4d940a740 1 -- 172.21.15.38:0/2233820649 shutdown_connections 2024-04-25T08:39:31.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.042+0000 7fe4d940a740 1 -- 172.21.15.38:0/2233820649 wait complete. 2024-04-25T08:39:31.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.043+0000 7fe4d940a740 1 -- start start 2024-04-25T08:39:31.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.043+0000 7fe4d940a740 1 Processor -- start 2024-04-25T08:39:31.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.043+0000 7fe4d940a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe62a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4d940a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4d940a740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe4ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4d940a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d015da70 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4d940a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8fd94540 con 0x55bc8fe23200 2024-04-25T08:39:31.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4d940a740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc8fe4a390 con 0x55bc8fe5f8d0 2024-04-25T08:39:31.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4d940a740 10 client.0 _ll_register_callbacks cb 0x7fe4b8058950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:31.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42404/0 (socket says 172.21.15.38:42404) 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/4084309222 learned_addr learned my addr 172.21.15.38:0/4084309222 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4cffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe62a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4d4bc7640 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe4ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/4084309222 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 msgr2=0x55bc8fe4ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe4ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/4084309222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 msgr2=0x55bc8fe62a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe62a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4cf7fe640 1 -- 172.21.15.38:0/4084309222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc8fe3d1b0 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4cffff640 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe62a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4cf7fe640 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4fa70 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0032fd0 tx=0x7fe4c0009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d86a0b78d5c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4aeffd640 1 -- 172.21.15.38:0/4084309222 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c006b060 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4aeffd640 1 -- 172.21.15.38:0/4084309222 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe4c0049a10 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4d940a740 1 -- 172.21.15.38:0/4084309222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4c0049860 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.044+0000 7fe4aeffd640 1 -- 172.21.15.38:0/4084309222 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4c0071050 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4d940a740 10 client.8496 fetch_fsmap learned FSMap version 63 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4d940a740 1 -- 172.21.15.38:0/4084309222 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55bc8fe4a130 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4aeffd640 1 -- 172.21.15.38:0/4084309222 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c0049860 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4aeffd640 1 -- 172.21.15.38:0/4084309222 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe4c0074020 con 0x55bc8fe4dcc0 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4d940a740 10 client.8496 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4d940a740 10 client.8496 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4d940a740 20 client.8496 populate_metadata read hostname 'smithi038' 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4d940a740 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55bc8febc870 0x55bc8fe3d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4d940a740 4 client.8496 mds_command: new command op to 8292 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4d940a740 1 -- 172.21.15.38:0/4084309222 --> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55bc8fd8e890 con 0x55bc8febc870 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4cffff640 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55bc8febc870 0x55bc8fe3d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4cffff640 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55bc8febc870 0x55bc8fe3d690 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb280b511760c73a server_cookie=91b4cad3a20f3ca3 in_seq=0 out_seq=0 2024-04-25T08:39:31.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.045+0000 7fe4aeffd640 10 client.8496 ms_handle_connect on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4aeffd640 1 -- 172.21.15.38:0/4084309222 <== mds.0 v2:172.21.15.107:6835/1263010516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bc8fd8e890 con 0x55bc8febc870 2024-04-25T08:39:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4aeffd640 10 client.8496 handle_command_reply: tid=0 2024-04-25T08:39:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4d940a740 1 client.8496 shutdown 2024-04-25T08:39:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4d940a740 1 -- 172.21.15.38:0/4084309222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 msgr2=0x55bc8fe4fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4d940a740 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4fa70 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0032fd0 tx=0x7fe4c0009240 comp rx=0 tx=0).stop 2024-04-25T08:39:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4d940a740 1 -- 172.21.15.38:0/4084309222 shutdown_connections 2024-04-25T08:39:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4d940a740 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55bc8febc870 0x55bc8fe3d690 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4d940a740 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55bc8fe5f8d0 0x55bc8fe4ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4d940a740 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bc8fe4dcc0 0x55bc8fe4fa70 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4d940a740 1 --2- 172.21.15.38:0/4084309222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc8fe23200 0x55bc8fe62a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4d940a740 1 -- 172.21.15.38:0/4084309222 >> 172.21.15.38:0/4084309222 conn(0x55bc8fdeb050 msgr2=0x55bc8fe62230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4aeffd640 0 client.8496 ms_handle_reset on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:31.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4d940a740 1 -- 172.21.15.38:0/4084309222 shutdown_connections 2024-04-25T08:39:31.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.049+0000 7fe4d940a740 1 -- 172.21.15.38:0/4084309222 wait complete. 2024-04-25T08:39:31.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d940a740 20 client.8496 trim_cache size 0 max 16384 2024-04-25T08:39:31.047 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:39:31.047 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:39:31.047 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "03dfe5b5-1ef8-4664-a393-033b435baafc", 2024-04-25T08:39:31.047 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:39:31.048 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:39:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d662a640 1 -- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe4b006b4d0 msgr2=0x7fe4b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d662a640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe4b006b4d0 0x7fe4b006d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0034000 tx=0x7fe4c002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d662a640 1 -- 172.21.15.38:0/2966255341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00b6a00 msgr2=0x7fe4d00cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d662a640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00b6a00 0x7fe4d00cbf30 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800d800 tx=0x7fe4b800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d662a640 1 -- 172.21.15.38:0/2966255341 shutdown_connections 2024-04-25T08:39:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d662a640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4d01612c0 0x7fe4d00c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d662a640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe4b006b4d0 0x7fe4b006d990 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d662a640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe4d00d0870 0x7fe4d00cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d662a640 1 --2- 172.21.15.38:0/2966255341 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe4d00b6a00 0x7fe4d00cbf30 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d662a640 1 -- 172.21.15.38:0/2966255341 >> 172.21.15.38:0/2966255341 conn(0x7fe4d0066d10 msgr2=0x7fe4d00c9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d662a640 1 -- 172.21.15.38:0/2966255341 shutdown_connections 2024-04-25T08:39:31.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.050+0000 7fe4d662a640 1 -- 172.21.15.38:0/2966255341 wait complete. 2024-04-25T08:39:31.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:31.055 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-25T08:39:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.178+0000 7fed319c4640 1 Processor -- start 2024-04-25T08:39:31.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.178+0000 7fed319c4640 1 -- start start 2024-04-25T08:39:31.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.178+0000 7fed319c4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fed2c0d0320 0x7fed2c0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.178+0000 7fed319c4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fed2c0b6980 0x7fed2c0b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.178+0000 7fed319c4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2c0d2060 con 0x7fed2c0d0700 2024-04-25T08:39:31.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.178+0000 7fed319c4640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2c0d2330 con 0x7fed2c0b6980 2024-04-25T08:39:31.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.178+0000 7fed319c4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2c0d2610 con 0x7fed2c0d0320 2024-04-25T08:39:31.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.179+0000 7fed2b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fed2c0b6980 0x7fed2c0b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.179+0000 7fed2b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fed2c0b6980 0x7fed2c0b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42410/0 (socket says 172.21.15.38:42410) 2024-04-25T08:39:31.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.179+0000 7fed2b7fe640 1 -- 172.21.15.38:0/1414172787 learned_addr learned my addr 172.21.15.38:0/1414172787 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.179+0000 7fed2affd640 1 --2- 172.21.15.38:0/1414172787 >> v2:172.21.15.107:3300/0 conn(0x7fed2c0d0320 0x7fed2c0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.179+0000 7fed2b7fe640 1 -- 172.21.15.38:0/1414172787 >> v2:172.21.15.107:3300/0 conn(0x7fed2c0d0320 msgr2=0x7fed2c0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.179+0000 7fed2b7fe640 1 --2- 172.21.15.38:0/1414172787 >> v2:172.21.15.107:3300/0 conn(0x7fed2c0d0320 0x7fed2c0d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.179+0000 7fed2b7fe640 1 -- 172.21.15.38:0/1414172787 >> v1:172.21.15.107:6789/0 conn(0x7fed2c0d0700 legacy=0x7fed2c0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.179+0000 7fed2b7fe640 1 -- 172.21.15.38:0/1414172787 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c0d28f0 con 0x7fed2c0b6980 2024-04-25T08:39:31.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.180+0000 7fed2b7fe640 1 --2- 172.21.15.38:0/1414172787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c0b6980 0x7fed2c0b6dd0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fed2c069100 tx=0x7fed2402a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b1e4487b3e803d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.180+0000 7fed2a7fc640 1 -- 172.21.15.38:0/1414172787 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24036070 con 0x7fed2c0b6980 2024-04-25T08:39:31.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.180+0000 7fed2a7fc640 1 -- 172.21.15.38:0/1414172787 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed24031600 con 0x7fed2c0b6980 2024-04-25T08:39:31.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.180+0000 7fed2a7fc640 1 -- 172.21.15.38:0/1414172787 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed240319f0 con 0x7fed2c0b6980 2024-04-25T08:39:31.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.180+0000 7fed319c4640 1 -- 172.21.15.38:0/1414172787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c0b6980 msgr2=0x7fed2c0b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.180+0000 7fed319c4640 1 --2- 172.21.15.38:0/1414172787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c0b6980 0x7fed2c0b6dd0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fed2c069100 tx=0x7fed2402a7c0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.180+0000 7fed319c4640 1 -- 172.21.15.38:0/1414172787 shutdown_connections 2024-04-25T08:39:31.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.180+0000 7fed319c4640 1 --2- 172.21.15.38:0/1414172787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c0b6980 0x7fed2c0b6dd0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.180+0000 7fed319c4640 1 --2- 172.21.15.38:0/1414172787 >> v2:172.21.15.107:3300/0 conn(0x7fed2c0d0320 0x7fed2c0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.180+0000 7fed319c4640 1 -- 172.21.15.38:0/1414172787 >> 172.21.15.38:0/1414172787 conn(0x7fed2c0668f0 msgr2=0x7fed2c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.181+0000 7fed319c4640 1 -- 172.21.15.38:0/1414172787 shutdown_connections 2024-04-25T08:39:31.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.181+0000 7fed319c4640 1 -- 172.21.15.38:0/1414172787 wait complete. 2024-04-25T08:39:31.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.182+0000 7fed319c4640 1 Processor -- start 2024-04-25T08:39:31.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.182+0000 7fed319c4640 1 -- start start 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.182+0000 7fed319c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed2c0b6980 0x7fed2c1529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.182+0000 7fed319c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed2c0d0320 0x7fed2c1e7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.182+0000 7fed319c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c0d0700 0x7fed2c0cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.182+0000 7fed319c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c0d0b40 con 0x7fed2c0d0700 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed319c4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c0d2eb0 con 0x7fed2c0d0320 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed319c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1580a0 con 0x7fed2c0b6980 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c0d0700 0x7fed2c0cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c0d0700 0x7fed2c0cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42412/0 (socket says 172.21.15.38:42412) 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed2b7fe640 1 -- 172.21.15.38:0/3971497101 learned_addr learned my addr 172.21.15.38:0/3971497101 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed23fff640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed2c0d0320 0x7fed2c1e7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed2affd640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed2c0b6980 0x7fed2c1529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed2b7fe640 1 -- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed2c0b6980 msgr2=0x7fed2c1529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed2b7fe640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed2c0b6980 0x7fed2c1529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed2b7fe640 1 -- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed2c0d0320 msgr2=0x7fed2c1e7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed2b7fe640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed2c0d0320 0x7fed2c1e7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed2b7fe640 1 -- 172.21.15.38:0/3971497101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c152f00 con 0x7fed2c0d0700 2024-04-25T08:39:31.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed2b7fe640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c0d0700 0x7fed2c0cb560 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fed2402a800 tx=0x7fed24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed28ff9640 1 -- 172.21.15.38:0/3971497101 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24036050 con 0x7fed2c0d0700 2024-04-25T08:39:31.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed28ff9640 1 -- 172.21.15.38:0/3971497101 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed24042070 con 0x7fed2c0d0700 2024-04-25T08:39:31.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed28ff9640 1 -- 172.21.15.38:0/3971497101 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed24009510 con 0x7fed2c0d0700 2024-04-25T08:39:31.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed319c4640 1 -- 172.21.15.38:0/3971497101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2c157ae0 con 0x7fed2c0d0700 2024-04-25T08:39:31.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.183+0000 7fed319c4640 1 -- 172.21.15.38:0/3971497101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2c0cc710 con 0x7fed2c0d0700 2024-04-25T08:39:31.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.185+0000 7fed28ff9640 1 -- 172.21.15.38:0/3971497101 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed24004450 con 0x7fed2c0d0700 2024-04-25T08:39:31.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.186+0000 7fed28ff9640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fed0c06b4d0 0x7fed0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.186+0000 7fed28ff9640 1 -- 172.21.15.38:0/3971497101 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fed240aafd0 con 0x7fed2c0d0700 2024-04-25T08:39:31.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.186+0000 7fed2affd640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fed0c06b4d0 0x7fed0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.186+0000 7fed2affd640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fed0c06b4d0 0x7fed0c06d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009b40 tx=0x7fed1c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.186+0000 7fed3479b740 1 Processor -- start 2024-04-25T08:39:31.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.186+0000 7fed3479b740 1 -- start start 2024-04-25T08:39:31.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.187+0000 7fed3479b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 0x56096d63be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.187+0000 7fed3479b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d676370 0x56096d63a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.187+0000 7fed3479b740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 0x56096d664be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.187+0000 7fed3479b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096d59cb90 con 0x56096d639ca0 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.187+0000 7fed3479b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096d545e40 con 0x56096d676370 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.187+0000 7fed3479b740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56096d598c30 con 0x56096d664760 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed2affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 0x56096d63be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed2affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 0x56096d63be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42416/0 (socket says 172.21.15.38:42416) 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed2affd640 1 -- 172.21.15.38:0/3783760224 learned_addr learned my addr 172.21.15.38:0/3783760224 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed23fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d676370 0x56096d63a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed2b7fe640 1 --2- 172.21.15.38:0/3783760224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 0x56096d664be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed2affd640 1 -- 172.21.15.38:0/3783760224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 msgr2=0x56096d664be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed2affd640 1 --2- 172.21.15.38:0/3783760224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 0x56096d664be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed2affd640 1 -- 172.21.15.38:0/3783760224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d676370 msgr2=0x56096d63a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed2affd640 1 --2- 172.21.15.38:0/3783760224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d676370 0x56096d63a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed2affd640 1 -- 172.21.15.38:0/3783760224 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56096d37b7d0 con 0x56096d639ca0 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed2affd640 1 --2- 172.21.15.38:0/3783760224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 0x56096d63be00 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fed1c011ee0 tx=0x7fed1c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e1d63205e5e2a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed217fa640 1 -- 172.21.15.38:0/3783760224 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c01b070 con 0x56096d639ca0 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed217fa640 1 -- 172.21.15.38:0/3783760224 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c027070 con 0x56096d639ca0 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.188+0000 7fed217fa640 1 -- 172.21.15.38:0/3783760224 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c003a20 con 0x56096d639ca0 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.189+0000 7fed3479b740 1 -- 172.21.15.38:0/3783760224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 msgr2=0x56096d63be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.189+0000 7fed3479b740 1 --2- 172.21.15.38:0/3783760224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 0x56096d63be00 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fed1c011ee0 tx=0x7fed1c014000 comp rx=0 tx=0).stop 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.189+0000 7fed3479b740 1 -- 172.21.15.38:0/3783760224 shutdown_connections 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.189+0000 7fed3479b740 1 --2- 172.21.15.38:0/3783760224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 0x56096d664be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.189+0000 7fed3479b740 1 --2- 172.21.15.38:0/3783760224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d676370 0x56096d63a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.189+0000 7fed3479b740 1 --2- 172.21.15.38:0/3783760224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 0x56096d63be00 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.189+0000 7fed3479b740 1 -- 172.21.15.38:0/3783760224 >> 172.21.15.38:0/3783760224 conn(0x56096d601870 msgr2=0x56096d67f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.189+0000 7fed3479b740 1 -- 172.21.15.38:0/3783760224 shutdown_connections 2024-04-25T08:39:31.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.189+0000 7fed3479b740 1 -- 172.21.15.38:0/3783760224 wait complete. 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.190+0000 7fed3479b740 1 -- start start 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.190+0000 7fed3479b740 1 Processor -- start 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.190+0000 7fed3479b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d639ca0 0x56096d660070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.190+0000 7fed3479b740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 0x56096d67b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.190+0000 7fed3479b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d676370 0x56096d67bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.190+0000 7fed3479b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096d5593e0 con 0x56096d676370 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.190+0000 7fed3479b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096d6622b0 con 0x56096d639ca0 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.190+0000 7fed3479b740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56096d678140 con 0x56096d664760 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.190+0000 7fed3479b740 10 client.0 _ll_register_callbacks cb 0x56096d558db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed2affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d639ca0 0x56096d660070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d676370 0x56096d67bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed2affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d639ca0 0x56096d660070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:46234/0 (socket says 172.21.15.38:46234) 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed2affd640 1 -- 172.21.15.38:0/1808164263 learned_addr learned my addr 172.21.15.38:0/1808164263 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed2b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d676370 0x56096d67bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42428/0 (socket says 172.21.15.38:42428) 2024-04-25T08:39:31.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed23fff640 1 --2- 172.21.15.38:0/1808164263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 0x56096d67b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed2b7fe640 1 -- 172.21.15.38:0/1808164263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 msgr2=0x56096d67b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed2b7fe640 1 --2- 172.21.15.38:0/1808164263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 0x56096d67b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed2b7fe640 1 -- 172.21.15.38:0/1808164263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d639ca0 msgr2=0x56096d660070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed2b7fe640 1 --2- 172.21.15.38:0/1808164263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d639ca0 0x56096d660070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed2b7fe640 1 -- 172.21.15.38:0/1808164263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56096d2710f0 con 0x56096d676370 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed2b7fe640 1 --2- 172.21.15.38:0/1808164263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d676370 0x56096d67bd00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fed240a9420 tx=0x7fed24079420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc2f64c2a677777 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1808164263 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed240a7060 con 0x56096d676370 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1808164263 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fed24004740 con 0x56096d676370 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed3479b740 1 -- 172.21.15.38:0/1808164263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed1c027070 con 0x56096d676370 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1808164263 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed24091070 con 0x56096d676370 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed3479b740 10 client.8508 fetch_fsmap learned FSMap version 63 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed3479b740 1 -- 172.21.15.38:0/1808164263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x56096d3dbf60 con 0x56096d676370 2024-04-25T08:39:31.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.191+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1808164263 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1c027070 con 0x56096d676370 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.192+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1808164263 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fed24078c40 con 0x56096d676370 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.192+0000 7fed3479b740 10 client.8508 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.192+0000 7fed3479b740 10 client.8508 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.192+0000 7fed3479b740 20 client.8508 populate_metadata read hostname 'smithi038' 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.192+0000 7fed3479b740 1 --2- 172.21.15.38:0/1808164263 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x56096d699b00 0x56096d69bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.192+0000 7fed3479b740 4 client.8508 mds_command: new command op to 8292 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.192+0000 7fed3479b740 1 -- 172.21.15.38:0/1808164263 --> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56096d5aa0d0 con 0x56096d699b00 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.192+0000 7fed2affd640 1 --2- 172.21.15.38:0/1808164263 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x56096d699b00 0x56096d69bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.192+0000 7fed0a7fc640 10 client.8508 ms_handle_connect on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.192+0000 7fed2affd640 1 --2- 172.21.15.38:0/1808164263 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x56096d699b00 0x56096d69bef0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10432d4b93990ca8 server_cookie=379656097f7c7790 in_seq=0 out_seq=0 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1808164263 <== mds.0 v2:172.21.15.107:6835/1263010516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56096d5aa0d0 con 0x56096d699b00 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed0a7fc640 10 client.8508 handle_command_reply: tid=0 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed3479b740 1 client.8508 shutdown 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed3479b740 1 -- 172.21.15.38:0/1808164263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d676370 msgr2=0x56096d67bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed3479b740 1 --2- 172.21.15.38:0/1808164263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d676370 0x56096d67bd00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fed240a9420 tx=0x7fed24079420 comp rx=0 tx=0).stop 2024-04-25T08:39:31.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed3479b740 1 -- 172.21.15.38:0/1808164263 shutdown_connections 2024-04-25T08:39:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed3479b740 1 --2- 172.21.15.38:0/1808164263 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x56096d699b00 0x56096d69bef0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed3479b740 1 --2- 172.21.15.38:0/1808164263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d676370 0x56096d67bd00 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed3479b740 1 --2- 172.21.15.38:0/1808164263 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 0x56096d67b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed3479b740 1 --2- 172.21.15.38:0/1808164263 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d639ca0 0x56096d660070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed3479b740 1 -- 172.21.15.38:0/1808164263 >> 172.21.15.38:0/1808164263 conn(0x56096d601870 msgr2=0x56096d662460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed0a7fc640 0 client.8508 ms_handle_reset on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed3479b740 1 -- 172.21.15.38:0/1808164263 shutdown_connections 2024-04-25T08:39:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed3479b740 1 -- 172.21.15.38:0/1808164263 wait complete. 2024-04-25T08:39:31.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.193+0000 7fed3479b740 20 client.8508 trim_cache size 0 max 16384 2024-04-25T08:39:31.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.209+0000 7fed3479b740 1 Processor -- start 2024-04-25T08:39:31.203 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.209+0000 7fed3479b740 1 -- start start 2024-04-25T08:39:31.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed3479b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d639ca0 0x56096d676980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed3479b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d664760 0x56096d667040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed3479b740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d676370 0x56096d667580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed3479b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c200480 con 0x56096d664760 2024-04-25T08:39:31.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed3479b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1ff5a0 con 0x56096d639ca0 2024-04-25T08:39:31.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed3479b740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c0b4de0 con 0x56096d676370 2024-04-25T08:39:31.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed23fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d664760 0x56096d667040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed23fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d664760 0x56096d667040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42432/0 (socket says 172.21.15.38:42432) 2024-04-25T08:39:31.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed23fff640 1 -- 172.21.15.38:0/31029329 learned_addr learned my addr 172.21.15.38:0/31029329 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed2affd640 1 --2- 172.21.15.38:0/31029329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d639ca0 0x56096d676980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed23fff640 1 -- 172.21.15.38:0/31029329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d676370 msgr2=0x56096d667580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed23fff640 1 --2- 172.21.15.38:0/31029329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d676370 0x56096d667580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed23fff640 1 -- 172.21.15.38:0/31029329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d639ca0 msgr2=0x56096d676980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed23fff640 1 --2- 172.21.15.38:0/31029329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d639ca0 0x56096d676980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.210+0000 7fed23fff640 1 -- 172.21.15.38:0/31029329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56096d65fc50 con 0x56096d664760 2024-04-25T08:39:31.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.211+0000 7fed23fff640 1 --2- 172.21.15.38:0/31029329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d664760 0x56096d667040 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fed2c1f71e0 tx=0x7fed14007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c0c159d4acc456 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.211+0000 7fed20ff9640 1 -- 172.21.15.38:0/31029329 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed140052e0 con 0x56096d664760 2024-04-25T08:39:31.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.211+0000 7fed20ff9640 1 -- 172.21.15.38:0/31029329 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1401a030 con 0x56096d664760 2024-04-25T08:39:31.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.211+0000 7fed20ff9640 1 -- 172.21.15.38:0/31029329 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14015ae0 con 0x56096d664760 2024-04-25T08:39:31.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.211+0000 7fed3479b740 1 -- 172.21.15.38:0/31029329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d664760 msgr2=0x56096d667040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.211+0000 7fed3479b740 1 --2- 172.21.15.38:0/31029329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d664760 0x56096d667040 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fed2c1f71e0 tx=0x7fed14007e50 comp rx=0 tx=0).stop 2024-04-25T08:39:31.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.211+0000 7fed3479b740 1 -- 172.21.15.38:0/31029329 shutdown_connections 2024-04-25T08:39:31.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.211+0000 7fed3479b740 1 --2- 172.21.15.38:0/31029329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d676370 0x56096d667580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.211+0000 7fed3479b740 1 --2- 172.21.15.38:0/31029329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d664760 0x56096d667040 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.211+0000 7fed3479b740 1 --2- 172.21.15.38:0/31029329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d639ca0 0x56096d676980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.211+0000 7fed3479b740 1 -- 172.21.15.38:0/31029329 >> 172.21.15.38:0/31029329 conn(0x56096d601870 msgr2=0x56096d635300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.212+0000 7fed3479b740 1 -- 172.21.15.38:0/31029329 shutdown_connections 2024-04-25T08:39:31.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.212+0000 7fed3479b740 1 -- 172.21.15.38:0/31029329 wait complete. 2024-04-25T08:39:31.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.212+0000 7fed3479b740 1 -- start start 2024-04-25T08:39:31.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.212+0000 7fed3479b740 1 Processor -- start 2024-04-25T08:39:31.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.213+0000 7fed3479b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 0x56096d638330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.213+0000 7fed3479b740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 0x56096d638870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.213+0000 7fed3479b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d676370 0x56096d662e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.213+0000 7fed3479b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096d630b40 con 0x56096d639ca0 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.213+0000 7fed3479b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c0b53e0 con 0x56096d676370 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.213+0000 7fed3479b740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x56096d636760 con 0x56096d664760 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.213+0000 7fed3479b740 10 client.0 _ll_register_callbacks cb 0x7fed240796b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.213+0000 7fed2affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 0x56096d638330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.213+0000 7fed23fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 0x56096d638870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.213+0000 7fed2affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 0x56096d638330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42436/0 (socket says 172.21.15.38:42436) 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.213+0000 7fed2affd640 1 -- 172.21.15.38:0/721753776 learned_addr learned my addr 172.21.15.38:0/721753776 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.213+0000 7fed2b7fe640 1 --2- 172.21.15.38:0/721753776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d676370 0x56096d662e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed2affd640 1 -- 172.21.15.38:0/721753776 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 msgr2=0x56096d638870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed2affd640 1 --2- 172.21.15.38:0/721753776 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 0x56096d638870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed2affd640 1 -- 172.21.15.38:0/721753776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d676370 msgr2=0x56096d662e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed2affd640 1 --2- 172.21.15.38:0/721753776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d676370 0x56096d662e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed2affd640 1 -- 172.21.15.38:0/721753776 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56096d24a1b0 con 0x56096d639ca0 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed2affd640 1 --2- 172.21.15.38:0/721753776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 0x56096d638330 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fed1c024040 tx=0x7fed1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc65ac341c85c52e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed0a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed0a7fc640 1 -- 172.21.15.38:0/721753776 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1c03d060 con 0x56096d639ca0 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed0a7fc640 1 -- 172.21.15.38:0/721753776 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fed1c003cb0 con 0x56096d639ca0 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed3479b740 1 -- 172.21.15.38:0/721753776 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed1401a030 con 0x56096d639ca0 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed0a7fc640 1 -- 172.21.15.38:0/721753776 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed1c043050 con 0x56096d639ca0 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed3479b740 10 client.8514 fetch_fsmap learned FSMap version 63 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed3479b740 1 -- 172.21.15.38:0/721753776 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x56096d638f90 con 0x56096d639ca0 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed0a7fc640 1 -- 172.21.15.38:0/721753776 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1401a030 con 0x56096d639ca0 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.214+0000 7fed0a7fc640 1 -- 172.21.15.38:0/721753776 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fed1c046020 con 0x56096d639ca0 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 10 client.8514 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 10 client.8514 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 20 client.8514 populate_metadata read hostname 'smithi038' 2024-04-25T08:39:31.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 --2- 172.21.15.38:0/721753776 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x56096d6e8bb0 0x56096d6633b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 4 client.8514 mds_command: new command op to 8292 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T08:39:31.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 -- 172.21.15.38:0/721753776 --> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x56096d5a4420 con 0x56096d6e8bb0 2024-04-25T08:39:31.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed23fff640 1 --2- 172.21.15.38:0/721753776 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x56096d6e8bb0 0x56096d6633b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed23fff640 1 --2- 172.21.15.38:0/721753776 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x56096d6e8bb0 0x56096d6633b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=342eefd39a4b4676 server_cookie=3bc0016c863baf73 in_seq=0 out_seq=0 2024-04-25T08:39:31.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed0a7fc640 10 client.8514 ms_handle_connect on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:31.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed0a7fc640 1 -- 172.21.15.38:0/721753776 <== mds.0 v2:172.21.15.107:6835/1263010516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x56096d5a4420 con 0x56096d6e8bb0 2024-04-25T08:39:31.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed0a7fc640 10 client.8514 handle_command_reply: tid=0 2024-04-25T08:39:31.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 client.8514 shutdown 2024-04-25T08:39:31.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 -- 172.21.15.38:0/721753776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 msgr2=0x56096d638330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 --2- 172.21.15.38:0/721753776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 0x56096d638330 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fed1c024040 tx=0x7fed1c004730 comp rx=0 tx=0).stop 2024-04-25T08:39:31.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 -- 172.21.15.38:0/721753776 shutdown_connections 2024-04-25T08:39:31.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 --2- 172.21.15.38:0/721753776 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x56096d6e8bb0 0x56096d6633b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 --2- 172.21.15.38:0/721753776 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56096d676370 0x56096d662e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 --2- 172.21.15.38:0/721753776 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x56096d664760 0x56096d638870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed0a7fc640 0 client.8514 ms_handle_reset on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:31.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 --2- 172.21.15.38:0/721753776 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56096d639ca0 0x56096d638330 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 -- 172.21.15.38:0/721753776 >> 172.21.15.38:0/721753776 conn(0x56096d601870 msgr2=0x56096d6a06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 -- 172.21.15.38:0/721753776 shutdown_connections 2024-04-25T08:39:31.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.215+0000 7fed3479b740 1 -- 172.21.15.38:0/721753776 wait complete. 2024-04-25T08:39:31.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed3479b740 20 client.8514 trim_cache size 0 max 16384 2024-04-25T08:39:31.216 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:39:31.216 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:39:31.217 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:39:31.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed319c4640 1 -- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fed0c06b4d0 msgr2=0x7fed0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed319c4640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fed0c06b4d0 0x7fed0c06d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009b40 tx=0x7fed1c009190 comp rx=0 tx=0).stop 2024-04-25T08:39:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed319c4640 1 -- 172.21.15.38:0/3971497101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c0d0700 msgr2=0x7fed2c0cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed319c4640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c0d0700 0x7fed2c0cb560 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fed2402a800 tx=0x7fed24004a40 comp rx=0 tx=0).stop 2024-04-25T08:39:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed319c4640 1 -- 172.21.15.38:0/3971497101 shutdown_connections 2024-04-25T08:39:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed319c4640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c0d0700 0x7fed2c0cb560 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed319c4640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed2c0d0320 0x7fed2c1e7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed319c4640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fed0c06b4d0 0x7fed0c06d990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed319c4640 1 --2- 172.21.15.38:0/3971497101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed2c0b6980 0x7fed2c1529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed319c4640 1 -- 172.21.15.38:0/3971497101 >> 172.21.15.38:0/3971497101 conn(0x7fed2c0668f0 msgr2=0x7fed2c0b7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed319c4640 1 -- 172.21.15.38:0/3971497101 shutdown_connections 2024-04-25T08:39:31.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.216+0000 7fed319c4640 1 -- 172.21.15.38:0/3971497101 wait complete. 2024-04-25T08:39:31.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:31.221 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T08:39:31.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.331+0000 7f7b0c615640 1 Processor -- start 2024-04-25T08:39:31.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.331+0000 7f7b0c615640 1 -- start start 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0c615640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7b0415df30 0x7f7b0409a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0c615640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b04162390 0x7f7b04099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0c615640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7b04099fb0 0x7f7b0406bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0c615640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0415e310 con 0x7f7b0415df30 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0c615640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0409aeb0 con 0x7f7b04099fb0 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0c615640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0415ee70 con 0x7f7b04162390 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0a3b1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7b0415df30 0x7f7b0409a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0a3b1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7b0415df30 0x7f7b0409a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42444/0 (socket says 172.21.15.38:42444) 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/3441587281 learned_addr learned my addr 172.21.15.38:0/3441587281 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/3441587281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b04162390 0x7f7b04099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0abb2640 1 --2- 172.21.15.38:0/3441587281 >> v2:172.21.15.107:3300/0 conn(0x7f7b04099fb0 0x7f7b0406bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/3441587281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b04162390 msgr2=0x7f7b04099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/3441587281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b04162390 0x7f7b04099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/3441587281 >> v2:172.21.15.107:3300/0 conn(0x7f7b04099fb0 msgr2=0x7f7b0406bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/3441587281 >> v2:172.21.15.107:3300/0 conn(0x7f7b04099fb0 0x7f7b0406bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/3441587281 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0415f300 con 0x7f7b0415df30 2024-04-25T08:39:31.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.332+0000 7f7b0abb2640 1 --2- 172.21.15.38:0/3441587281 >> v2:172.21.15.107:3300/0 conn(0x7f7b04099fb0 0x7f7b0406bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:31.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.333+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/3441587281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b0415df30 0x7f7b0409a7b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f7b040d2680 tx=0x7f7b0002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f921d62b5463fcc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.334+0000 7f7b093af640 1 -- 172.21.15.38:0/3441587281 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00037070 con 0x7f7b0415df30 2024-04-25T08:39:31.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.334+0000 7f7b093af640 1 -- 172.21.15.38:0/3441587281 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b00032440 con 0x7f7b0415df30 2024-04-25T08:39:31.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.334+0000 7f7b093af640 1 -- 172.21.15.38:0/3441587281 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00032810 con 0x7f7b0415df30 2024-04-25T08:39:31.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.334+0000 7f7b0c615640 1 -- 172.21.15.38:0/3441587281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b0415df30 msgr2=0x7f7b0409a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.334+0000 7f7b0c615640 1 --2- 172.21.15.38:0/3441587281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b0415df30 0x7f7b0409a7b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f7b040d2680 tx=0x7f7b0002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.334+0000 7f7b0c615640 1 -- 172.21.15.38:0/3441587281 shutdown_connections 2024-04-25T08:39:31.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.334+0000 7f7b0c615640 1 --2- 172.21.15.38:0/3441587281 >> v2:172.21.15.107:3300/0 conn(0x7f7b04099fb0 0x7f7b0406bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.334+0000 7f7b0c615640 1 --2- 172.21.15.38:0/3441587281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b04162390 0x7f7b04099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.334+0000 7f7b0c615640 1 --2- 172.21.15.38:0/3441587281 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b0415df30 0x7f7b0409a7b0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.334+0000 7f7b0c615640 1 -- 172.21.15.38:0/3441587281 >> 172.21.15.38:0/3441587281 conn(0x7f7b040b3110 msgr2=0x7f7b040b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.334+0000 7f7b0c615640 1 -- 172.21.15.38:0/3441587281 shutdown_connections 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.334+0000 7f7b0c615640 1 -- 172.21.15.38:0/3441587281 wait complete. 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.336+0000 7f7b0c615640 1 Processor -- start 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.336+0000 7f7b0c615640 1 -- start start 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.336+0000 7f7b0c615640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b04099fb0 0x7f7b040b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.336+0000 7f7b0c615640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0415df30 0x7f7b041607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.336+0000 7f7b0c615640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b04162390 0x7f7b0414e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.336+0000 7f7b0c615640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0415e820 con 0x7f7b04099fb0 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.336+0000 7f7b0c615640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0415e540 con 0x7f7b04162390 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.336+0000 7f7b0c615640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0415eb00 con 0x7f7b0415df30 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.336+0000 7f7b0a3b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b04099fb0 0x7f7b040b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.336+0000 7f7b0a3b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b04099fb0 0x7f7b040b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42448/0 (socket says 172.21.15.38:42448) 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.336+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/1608660780 learned_addr learned my addr 172.21.15.38:0/1608660780 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0415df30 0x7f7b041607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7b0abb2640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b04162390 0x7f7b0414e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0415df30 msgr2=0x7f7b041607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0415df30 0x7f7b041607c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b04162390 msgr2=0x7f7b0414e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b04162390 0x7f7b0414e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/1608660780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b040c6ca0 con 0x7f7b04099fb0 2024-04-25T08:39:31.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b04099fb0 0x7f7b040b4240 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7b00000c00 tx=0x7f7b000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7afaffd640 1 -- 172.21.15.38:0/1608660780 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00041020 con 0x7f7b04099fb0 2024-04-25T08:39:31.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7afaffd640 1 -- 172.21.15.38:0/1608660780 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b00030640 con 0x7f7b04099fb0 2024-04-25T08:39:31.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7afaffd640 1 -- 172.21.15.38:0/1608660780 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00037040 con 0x7f7b04099fb0 2024-04-25T08:39:31.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7b0c615640 1 -- 172.21.15.38:0/1608660780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b040c6f80 con 0x7f7b04099fb0 2024-04-25T08:39:31.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.337+0000 7f7b0c615640 1 -- 172.21.15.38:0/1608660780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b040c7540 con 0x7f7b04099fb0 2024-04-25T08:39:31.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.339+0000 7f7afaffd640 1 -- 172.21.15.38:0/1608660780 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b00044020 con 0x7f7b04099fb0 2024-04-25T08:39:31.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.339+0000 7f7afaffd640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ae806b4d0 0x7f7ae806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.339+0000 7f7afaffd640 1 -- 172.21.15.38:0/1608660780 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7b000aaa20 con 0x7f7b04099fb0 2024-04-25T08:39:31.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.339+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ae806b4d0 0x7f7ae806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.340+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ae806b4d0 0x7f7ae806d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7b0415d510 tx=0x7f7af4009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.340+0000 7f7b0f3f9740 1 Processor -- start 2024-04-25T08:39:31.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.340+0000 7f7b0f3f9740 1 -- start start 2024-04-25T08:39:31.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.341+0000 7f7b0f3f9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8b10 0x55e1262eb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.341+0000 7f7b0f3f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 0x55e1262eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.341+0000 7f7b0f3f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1262bb770 0x55e1262bef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.341+0000 7f7b0f3f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e126224da0 con 0x55e1262e8ef0 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.341+0000 7f7b0f3f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1261cdb80 con 0x55e1262bb770 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.341+0000 7f7b0f3f9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e126220e40 con 0x55e1262e8b10 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.341+0000 7f7b09bb0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 0x55e1262eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.341+0000 7f7b09bb0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 0x55e1262eb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42456/0 (socket says 172.21.15.38:42456) 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.341+0000 7f7b09bb0640 1 -- 172.21.15.38:0/1735233652 learned_addr learned my addr 172.21.15.38:0/1735233652 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.341+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/1735233652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8b10 0x55e1262eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.341+0000 7f7b0abb2640 1 --2- 172.21.15.38:0/1735233652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1262bb770 0x55e1262bef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.342+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/1735233652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1262bb770 msgr2=0x55e1262bef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.342+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/1735233652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1262bb770 0x55e1262bef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.342+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/1735233652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 msgr2=0x55e1262eb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.342+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/1735233652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 0x55e1262eb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.342+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/1735233652 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1261d7f10 con 0x55e1262e8b10 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.342+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/1735233652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 0x55e1262eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.342+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/1735233652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8b10 0x55e1262eb320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7b000a8de0 tx=0x7f7b00078ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac0533e2e310e642 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.342+0000 7f7ae7fff640 1 -- 172.21.15.38:0/1735233652 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00090020 con 0x55e1262e8b10 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.342+0000 7f7ae7fff640 1 -- 172.21.15.38:0/1735233652 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b000816f0 con 0x55e1262e8b10 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.342+0000 7f7ae7fff640 1 -- 172.21.15.38:0/1735233652 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00075700 con 0x55e1262e8b10 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.343+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/1735233652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8b10 msgr2=0x55e1262eb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.343+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/1735233652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8b10 0x55e1262eb320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7b000a8de0 tx=0x7f7b00078ca0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.343+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/1735233652 shutdown_connections 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.343+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/1735233652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1262bb770 0x55e1262bef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.343+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/1735233652 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 0x55e1262eb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.343+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/1735233652 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8b10 0x55e1262eb320 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.343+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/1735233652 >> 172.21.15.38:0/1735233652 conn(0x55e1262bd9e0 msgr2=0x55e126306320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.343+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/1735233652 shutdown_connections 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.343+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/1735233652 wait complete. 2024-04-25T08:39:31.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.343+0000 7f7b0f3f9740 1 -- start start 2024-04-25T08:39:31.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.343+0000 7f7b0f3f9740 1 Processor -- start 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.344+0000 7f7b0f3f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 0x55e126308d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.344+0000 7f7b0f3f9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8ef0 0x55e126304f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.344+0000 7f7b0f3f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e126305460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.344+0000 7f7b0f3f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e126220e40 con 0x55e1262e8b10 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.344+0000 7f7b0f3f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1261c34b0 con 0x55e126307450 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.344+0000 7f7b0f3f9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1262c0b90 con 0x55e1262e8ef0 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.344+0000 7f7b0f3f9740 10 client.0 _ll_register_callbacks cb 0x55e126203180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.344+0000 7f7b0a3b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 0x55e126308d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.344+0000 7f7b0abb2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e126305460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.344+0000 7f7b0a3b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 0x55e126308d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42470/0 (socket says 172.21.15.38:42470) 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.344+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8ef0 0x55e126304f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.344+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/3397164737 learned_addr learned my addr 172.21.15.38:0/3397164737 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8ef0 msgr2=0x55e126304f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8ef0 0x55e126304f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 msgr2=0x55e126305460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e126305460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/3397164737 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1260037d0 con 0x55e1262e8b10 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7b0abb2640 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e126305460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8ef0 0x55e126304f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 0x55e126308d20 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7b00079020 tx=0x7f7b00032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54257b2c80debd46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7ae57fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7ae57fa640 1 -- 172.21.15.38:0/3397164737 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00090070 con 0x55e1262e8b10 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7ae57fa640 1 -- 172.21.15.38:0/3397164737 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7b00081460 con 0x55e1262e8b10 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/3397164737 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b000816f0 con 0x55e1262e8b10 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7ae57fa640 1 -- 172.21.15.38:0/3397164737 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b0009f050 con 0x55e1262e8b10 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7b0f3f9740 10 client.8526 fetch_fsmap learned FSMap version 63 2024-04-25T08:39:31.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/3397164737 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55e125f393f0 con 0x55e1262e8b10 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.345+0000 7f7ae57fa640 1 -- 172.21.15.38:0/3397164737 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b000816f0 con 0x55e1262e8b10 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.346+0000 7f7ae57fa640 1 -- 172.21.15.38:0/3397164737 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7b000a2020 con 0x55e1262e8b10 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.346+0000 7f7b0f3f9740 10 client.8526 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.346+0000 7f7b0f3f9740 10 client.8526 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.346+0000 7f7b0f3f9740 20 client.8526 populate_metadata read hostname 'smithi038' 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.346+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55e126300e20 0x55e126301230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.346+0000 7f7b0f3f9740 4 client.8526 mds_command: new command op to 8292 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.346+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/3397164737 --> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1262322e0 con 0x55e126300e20 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.346+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55e126300e20 0x55e126301230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.346+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55e126300e20 0x55e126301230 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5463d03c786a8426 server_cookie=69230612503cab4a in_seq=0 out_seq=0 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.346+0000 7f7ae57fa640 10 client.8526 ms_handle_connect on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7ae57fa640 1 -- 172.21.15.38:0/3397164737 <== mds.0 v2:172.21.15.107:6835/1263010516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e1262322e0 con 0x55e126300e20 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7ae57fa640 10 client.8526 handle_command_reply: tid=0 2024-04-25T08:39:31.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7b0f3f9740 1 client.8526 shutdown 2024-04-25T08:39:31.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/3397164737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 msgr2=0x55e126308d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 0x55e126308d20 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7b00079020 tx=0x7f7b00032f90 comp rx=0 tx=0).stop 2024-04-25T08:39:31.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/3397164737 shutdown_connections 2024-04-25T08:39:31.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55e126300e20 0x55e126301230 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e126305460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8ef0 0x55e126304f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/3397164737 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 0x55e126308d20 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7ae57fa640 0 client.8526 ms_handle_reset on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:31.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/3397164737 >> 172.21.15.38:0/3397164737 conn(0x55e1262bb770 msgr2=0x55e1262e68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/3397164737 shutdown_connections 2024-04-25T08:39:31.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/3397164737 wait complete. 2024-04-25T08:39:31.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.347+0000 7f7b0f3f9740 20 client.8526 trim_cache size 0 max 16384 2024-04-25T08:39:31.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.362+0000 7f7b0f3f9740 1 Processor -- start 2024-04-25T08:39:31.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.362+0000 7f7b0f3f9740 1 -- start start 2024-04-25T08:39:31.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b0f3f9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8b10 0x55e1262ef150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b0f3f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 0x55e1262bef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b0f3f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e1262bf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b0f3f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04200430 con 0x55e1262e8ef0 2024-04-25T08:39:31.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b0f3f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b041f4f20 con 0x55e126307450 2024-04-25T08:39:31.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b0f3f9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b041f4320 con 0x55e1262e8b10 2024-04-25T08:39:31.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b09bb0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 0x55e1262bef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b09bb0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 0x55e1262bef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42474/0 (socket says 172.21.15.38:42474) 2024-04-25T08:39:31.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b09bb0640 1 -- 172.21.15.38:0/266084028 learned_addr learned my addr 172.21.15.38:0/266084028 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b09bb0640 1 -- 172.21.15.38:0/266084028 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8b10 msgr2=0x55e1262ef150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/266084028 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8b10 0x55e1262ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b09bb0640 1 -- 172.21.15.38:0/266084028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 msgr2=0x55e1262bf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/266084028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e1262bf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b09bb0640 1 -- 172.21.15.38:0/266084028 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e125ef90f0 con 0x55e1262e8ef0 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/266084028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 0x55e1262bef80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x55e1262ec1c0 tx=0x7f7af4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77414449ded15df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.363+0000 7f7ae77fe640 1 -- 172.21.15.38:0/266084028 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40148d0 con 0x55e1262e8ef0 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.364+0000 7f7ae77fe640 1 -- 172.21.15.38:0/266084028 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4031070 con 0x55e1262e8ef0 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.364+0000 7f7ae77fe640 1 -- 172.21.15.38:0/266084028 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4014c70 con 0x55e1262e8ef0 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.364+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/266084028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 msgr2=0x55e1262bef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.364+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/266084028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 0x55e1262bef80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x55e1262ec1c0 tx=0x7f7af4019000 comp rx=0 tx=0).stop 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.364+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/266084028 shutdown_connections 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.364+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/266084028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e1262bf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.364+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/266084028 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8ef0 0x55e1262bef80 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.364+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/266084028 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8b10 0x55e1262ef150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.364+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/266084028 >> 172.21.15.38:0/266084028 conn(0x55e1262bb770 msgr2=0x55e1262c2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.364+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/266084028 shutdown_connections 2024-04-25T08:39:31.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.364+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/266084028 wait complete. 2024-04-25T08:39:31.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.365+0000 7f7b0f3f9740 1 -- start start 2024-04-25T08:39:31.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.365+0000 7f7b0f3f9740 1 Processor -- start 2024-04-25T08:39:31.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0f3f9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 0x55e126309b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0f3f9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8ef0 0x55e126372790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0f3f9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e126372cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0f3f9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1262c1840 con 0x55e1262e8b10 2024-04-25T08:39:31.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0f3f9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b041f4920 con 0x55e126307450 2024-04-25T08:39:31.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0f3f9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1262e66c0 con 0x55e1262e8ef0 2024-04-25T08:39:31.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0f3f9740 10 client.0 _ll_register_callbacks cb 0x7f7b000091c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:31.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0a3b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 0x55e126309b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0a3b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 0x55e126309b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42482/0 (socket says 172.21.15.38:42482) 2024-04-25T08:39:31.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/802756047 learned_addr learned my addr 172.21.15.38:0/802756047 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0abb2640 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e126372cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8ef0 0x55e126372790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8ef0 msgr2=0x55e126372790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8ef0 0x55e126372790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 msgr2=0x55e126372cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e126372cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0a3b1640 1 -- 172.21.15.38:0/802756047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e12630a0b0 con 0x55e1262e8b10 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0abb2640 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e126372cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8ef0 0x55e126372790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0a3b1640 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 0x55e126309b70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f7b000754f0 tx=0x7f7b000a8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b189d5cadd1f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7ae57fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7ae57fa640 1 -- 172.21.15.38:0/802756047 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b000816f0 con 0x55e1262e8b10 2024-04-25T08:39:31.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7ae57fa640 1 -- 172.21.15.38:0/802756047 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7b0007f0c0 con 0x55e1262e8b10 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/802756047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7af4031070 con 0x55e1262e8b10 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7ae57fa640 1 -- 172.21.15.38:0/802756047 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b000a23e0 con 0x55e1262e8b10 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0f3f9740 10 client.8532 fetch_fsmap learned FSMap version 63 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.366+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/802756047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55e12630a0b0 con 0x55e1262e8b10 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7ae57fa640 1 -- 172.21.15.38:0/802756047 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4031070 con 0x55e1262e8b10 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7ae57fa640 1 -- 172.21.15.38:0/802756047 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7b000a2020 con 0x55e1262e8b10 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7b0f3f9740 10 client.8532 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7b0f3f9740 10 client.8532 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7b0f3f9740 20 client.8532 populate_metadata read hostname 'smithi038' 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55e126301300 0x55e1262fefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7b0f3f9740 4 client.8532 mds_command: new command op to 8292 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/802756047 --> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e12622c630 con 0x55e126301300 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55e126301300 0x55e1262fefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7ae57fa640 10 client.8532 ms_handle_connect on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7b09bb0640 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55e126301300 0x55e1262fefc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=790ecc5e90dd333c server_cookie=7ede8ff89003d35c in_seq=0 out_seq=0 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7ae57fa640 1 -- 172.21.15.38:0/802756047 <== mds.0 v2:172.21.15.107:6835/1263010516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55e12622c630 con 0x55e126301300 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7ae57fa640 10 client.8532 handle_command_reply: tid=0 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7b0f3f9740 1 client.8532 shutdown 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/802756047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 msgr2=0x55e126309b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.367+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 0x55e126309b70 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f7b000754f0 tx=0x7f7b000a8c60 comp rx=0 tx=0).stop 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/802756047 shutdown_connections 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55e126301300 0x55e1262fefc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e126307450 0x55e126372cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e1262e8ef0 0x55e126372790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0f3f9740 1 --2- 172.21.15.38:0/802756047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e1262e8b10 0x55e126309b70 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7ae57fa640 0 client.8532 ms_handle_reset on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/802756047 >> 172.21.15.38:0/802756047 conn(0x55e1262bb770 msgr2=0x55e1262c0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/802756047 shutdown_connections 2024-04-25T08:39:31.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0f3f9740 1 -- 172.21.15.38:0/802756047 wait complete. 2024-04-25T08:39:31.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0f3f9740 20 client.8532 trim_cache size 0 max 16384 2024-04-25T08:39:31.369 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:39:31.369 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T08:39:31.369 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-04-25T08:39:31.369 INFO:teuthology.orchestra.run.smithi038.stdout: "03dfe5b5-1ef8-4664-a393-033b435baafc": { 2024-04-25T08:39:31.369 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/scrub_control_test_path", 2024-04-25T08:39:31.369 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "03dfe5b5-1ef8-4664-a393-033b435baafc", 2024-04-25T08:39:31.369 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive" 2024-04-25T08:39:31.369 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:31.369 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:31.370 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:39:31.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0c615640 1 -- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ae806b4d0 msgr2=0x7f7ae806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0c615640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ae806b4d0 0x7f7ae806d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f7b0415d510 tx=0x7f7af4009190 comp rx=0 tx=0).stop 2024-04-25T08:39:31.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0c615640 1 -- 172.21.15.38:0/1608660780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b04099fb0 msgr2=0x7f7b040b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0c615640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b04099fb0 0x7f7b040b4240 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7b00000c00 tx=0x7f7b000046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0c615640 1 -- 172.21.15.38:0/1608660780 shutdown_connections 2024-04-25T08:39:31.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0c615640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b04162390 0x7f7b0414e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0c615640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ae806b4d0 0x7f7ae806d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0c615640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7b0415df30 0x7f7b041607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0c615640 1 --2- 172.21.15.38:0/1608660780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7b04099fb0 0x7f7b040b4240 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0c615640 1 -- 172.21.15.38:0/1608660780 >> 172.21.15.38:0/1608660780 conn(0x7f7b040b3110 msgr2=0x7f7b0406bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0c615640 1 -- 172.21.15.38:0/1608660780 shutdown_connections 2024-04-25T08:39:31.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.368+0000 7f7b0c615640 1 -- 172.21.15.38:0/1608660780 wait complete. 2024-04-25T08:39:31.373 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:31.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.481+0000 7f36a79e7640 1 Processor -- start 2024-04-25T08:39:31.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.481+0000 7f36a79e7640 1 -- start start 2024-04-25T08:39:31.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a79e7640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f36a006c110 0x7f36a006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a79e7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a015df50 0x7f36a00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a79e7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a006ca30 con 0x7f36a00ce2e0 2024-04-25T08:39:31.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a79e7640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a00cf2d0 con 0x7f36a006c110 2024-04-25T08:39:31.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a79e7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a00ceff0 con 0x7f36a015df50 2024-04-25T08:39:31.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a5783640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f36a006c110 0x7f36a006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a4f82640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a015df50 0x7f36a00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a5f84640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f36a00ce2e0 0x7f36a00d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:40904/0 (socket says 172.21.15.38:40904) 2024-04-25T08:39:31.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a5f84640 1 -- 172.21.15.38:0/1718065590 learned_addr learned my addr 172.21.15.38:0/1718065590 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:31.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a4f82640 1 -- 172.21.15.38:0/1718065590 >> v2:172.21.15.107:3300/0 conn(0x7f36a006c110 msgr2=0x7f36a006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a4f82640 1 --2- 172.21.15.38:0/1718065590 >> v2:172.21.15.107:3300/0 conn(0x7f36a006c110 0x7f36a006c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a4f82640 1 -- 172.21.15.38:0/1718065590 >> v1:172.21.15.107:6789/0 conn(0x7f36a00ce2e0 legacy=0x7f36a00d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a4f82640 1 -- 172.21.15.38:0/1718065590 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a00d1f70 con 0x7f36a015df50 2024-04-25T08:39:31.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f3697fff640 1 -- 172.21.15.38:0/1718065590 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256025392 0 0) 0x7f36a006ca30 con 0x7f36a00ce2e0 2024-04-25T08:39:31.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.482+0000 7f36a5783640 1 --2- 172.21.15.38:0/1718065590 >> v2:172.21.15.107:3300/0 conn(0x7f36a006c110 0x7f36a006c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:31.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.483+0000 7f36a4f82640 1 --2- 172.21.15.38:0/1718065590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a015df50 0x7f36a00ce7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f36900097a0 tx=0x7f369002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f427da16f8453ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.483+0000 7f3697fff640 1 -- 172.21.15.38:0/1718065590 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690037070 con 0x7f36a015df50 2024-04-25T08:39:31.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.483+0000 7f3697fff640 1 -- 172.21.15.38:0/1718065590 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3690032440 con 0x7f36a015df50 2024-04-25T08:39:31.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.483+0000 7f3697fff640 1 -- 172.21.15.38:0/1718065590 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690032810 con 0x7f36a015df50 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.484+0000 7f36a79e7640 1 -- 172.21.15.38:0/1718065590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a015df50 msgr2=0x7f36a00ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.484+0000 7f36a79e7640 1 --2- 172.21.15.38:0/1718065590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a015df50 0x7f36a00ce7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f36900097a0 tx=0x7f369002a180 comp rx=0 tx=0).stop 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.484+0000 7f36a79e7640 1 -- 172.21.15.38:0/1718065590 shutdown_connections 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.484+0000 7f36a79e7640 1 --2- 172.21.15.38:0/1718065590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a015df50 0x7f36a00ce7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.484+0000 7f36a79e7640 1 --2- 172.21.15.38:0/1718065590 >> v2:172.21.15.107:3300/0 conn(0x7f36a006c110 0x7f36a006c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.484+0000 7f36a79e7640 1 -- 172.21.15.38:0/1718065590 >> 172.21.15.38:0/1718065590 conn(0x7f36a00d3fb0 msgr2=0x7f36a00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.484+0000 7f36a79e7640 1 -- 172.21.15.38:0/1718065590 shutdown_connections 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.484+0000 7f36a79e7640 1 -- 172.21.15.38:0/1718065590 wait complete. 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.485+0000 7f36a79e7640 1 Processor -- start 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.485+0000 7f36a79e7640 1 -- start start 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.486+0000 7f36a79e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a006c110 0x7f36a01f3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.486+0000 7f36a79e7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a00ce2e0 0x7f36a00c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.486+0000 7f36a79e7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a015df50 0x7f36a00ca460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.486+0000 7f36a79e7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a00cece0 con 0x7f36a006c110 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.486+0000 7f36a79e7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a00d2530 con 0x7f36a00ce2e0 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.486+0000 7f36a79e7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a00d3160 con 0x7f36a015df50 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.486+0000 7f36a5783640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a006c110 0x7f36a01f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.486+0000 7f36a5783640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a006c110 0x7f36a01f3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42494/0 (socket says 172.21.15.38:42494) 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.486+0000 7f36a5783640 1 -- 172.21.15.38:0/848068363 learned_addr learned my addr 172.21.15.38:0/848068363 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.486+0000 7f36a4f82640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a00ce2e0 0x7f36a00c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.486+0000 7f36a5f84640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a015df50 0x7f36a00ca460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.487+0000 7f36a5783640 1 -- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a015df50 msgr2=0x7f36a00ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.487+0000 7f36a5783640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a015df50 0x7f36a00ca460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.487+0000 7f36a5783640 1 -- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a00ce2e0 msgr2=0x7f36a00c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.487+0000 7f36a5783640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a00ce2e0 0x7f36a00c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.487+0000 7f36a5783640 1 -- 172.21.15.38:0/848068363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a014f8b0 con 0x7f36a006c110 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.487+0000 7f36a5783640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a006c110 0x7f36a01f3190 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f369c00ebe0 tx=0x7f369c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.487+0000 7f3695ffb640 1 -- 172.21.15.38:0/848068363 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c017070 con 0x7f36a006c110 2024-04-25T08:39:31.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.487+0000 7f3695ffb640 1 -- 172.21.15.38:0/848068363 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c00cc30 con 0x7f36a006c110 2024-04-25T08:39:31.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.487+0000 7f36a79e7640 1 -- 172.21.15.38:0/848068363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a015e3d0 con 0x7f36a006c110 2024-04-25T08:39:31.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.487+0000 7f3695ffb640 1 -- 172.21.15.38:0/848068363 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c013470 con 0x7f36a006c110 2024-04-25T08:39:31.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.487+0000 7f36a79e7640 1 -- 172.21.15.38:0/848068363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a015ec30 con 0x7f36a006c110 2024-04-25T08:39:31.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.488+0000 7f36a79e7640 1 -- 172.21.15.38:0/848068363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a0068e80 con 0x7f36a006c110 2024-04-25T08:39:31.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.489+0000 7f3695ffb640 1 -- 172.21.15.38:0/848068363 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f369c01a070 con 0x7f36a006c110 2024-04-25T08:39:31.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.489+0000 7f3695ffb640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f368406b4d0 0x7f368406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.489+0000 7f3695ffb640 1 -- 172.21.15.38:0/848068363 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f369c08a210 con 0x7f36a006c110 2024-04-25T08:39:31.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.489+0000 7f36a4f82640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f368406b4d0 0x7f368406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.490+0000 7f36a4f82640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f368406b4d0 0x7f368406d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3690034000 tx=0x7f369002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.494+0000 7f3695ffb640 1 -- 172.21.15.38:0/848068363 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f369c058910 con 0x7f36a006c110 2024-04-25T08:39:31.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.709+0000 7f36a79e7640 1 -- 172.21.15.38:0/848068363 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36a006c590 con 0x7f36a006c110 2024-04-25T08:39:31.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.710+0000 7f3695ffb640 1 -- 172.21.15.38:0/848068363 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f369c05c9b0 con 0x7f36a006c110 2024-04-25T08:39:31.704 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:31.704 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:07.551641+0000","modified":"2024-04-25T08:39:09.555958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8292":{"gid":8292,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1263010516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1263010516},{"type":"v1","addr":"172.21.15.107:6837","nonce":1263010516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:39:31.705 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 63 2024-04-25T08:39:31.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.712+0000 7f36a79e7640 1 -- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f368406b4d0 msgr2=0x7f368406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.712+0000 7f36a79e7640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f368406b4d0 0x7f368406d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3690034000 tx=0x7f369002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:31.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.712+0000 7f36a79e7640 1 -- 172.21.15.38:0/848068363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a006c110 msgr2=0x7f36a01f3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.712+0000 7f36a79e7640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a006c110 0x7f36a01f3190 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f369c00ebe0 tx=0x7f369c00c520 comp rx=0 tx=0).stop 2024-04-25T08:39:31.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.712+0000 7f36a79e7640 1 -- 172.21.15.38:0/848068363 shutdown_connections 2024-04-25T08:39:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.712+0000 7f36a79e7640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f368406b4d0 0x7f368406d990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.712+0000 7f36a79e7640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f36a015df50 0x7f36a00ca460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.712+0000 7f36a79e7640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f36a00ce2e0 0x7f36a00c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.712+0000 7f36a79e7640 1 --2- 172.21.15.38:0/848068363 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a006c110 0x7f36a01f3190 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.712+0000 7f36a79e7640 1 -- 172.21.15.38:0/848068363 >> 172.21.15.38:0/848068363 conn(0x7f36a00d3fb0 msgr2=0x7f36a00d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.712+0000 7f36a79e7640 1 -- 172.21.15.38:0/848068363 shutdown_connections 2024-04-25T08:39:31.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.712+0000 7f36a79e7640 1 -- 172.21.15.38:0/848068363 wait complete. 2024-04-25T08:39:31.717 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:39:31.717 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:39:31.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.822+0000 7fa0168ac640 1 Processor -- start 2024-04-25T08:39:31.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.822+0000 7fa0168ac640 1 -- start start 2024-04-25T08:39:31.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa0168ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa01015b730 0x7fa01015bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa0168ac640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa0100d0cd0 0x7fa0100d1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa0168ac640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa010073220 con 0x7fa0100ccd00 2024-04-25T08:39:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa0168ac640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0100cdd10 con 0x7fa0100d0cd0 2024-04-25T08:39:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa0168ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0100cf7f0 con 0x7fa01015b730 2024-04-25T08:39:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa00f7fe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa0100d0cd0 0x7fa0100d1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa00ffff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa01015b730 0x7fa01015bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa00f7fe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa0100d0cd0 0x7fa0100d1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:46338/0 (socket says 172.21.15.38:46338) 2024-04-25T08:39:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa014e49640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fa0100ccd00 0x7fa0100b2220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:40912/0 (socket says 172.21.15.38:40912) 2024-04-25T08:39:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa00f7fe640 1 -- 172.21.15.38:0/3057004281 learned_addr learned my addr 172.21.15.38:0/3057004281 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa00f7fe640 1 -- 172.21.15.38:0/3057004281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa01015b730 msgr2=0x7fa01015bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa00f7fe640 1 --2- 172.21.15.38:0/3057004281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa01015b730 0x7fa01015bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa00f7fe640 1 -- 172.21.15.38:0/3057004281 >> v1:172.21.15.107:6789/0 conn(0x7fa0100ccd00 legacy=0x7fa0100b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa00f7fe640 1 -- 172.21.15.38:0/3057004281 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0100cfad0 con 0x7fa0100d0cd0 2024-04-25T08:39:31.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa00ffff640 1 --2- 172.21.15.38:0/3057004281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa01015b730 0x7fa01015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:31.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.823+0000 7fa00effd640 1 -- 172.21.15.38:0/3057004281 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786489513 0 0) 0x7fa010073220 con 0x7fa0100ccd00 2024-04-25T08:39:31.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.824+0000 7fa00f7fe640 1 --2- 172.21.15.38:0/3057004281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0100d0cd0 0x7fa0100d1130 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa004009870 tx=0x7fa00402a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c717d53d3f053757 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.824+0000 7fa00effd640 1 -- 172.21.15.38:0/3057004281 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004037070 con 0x7fa0100d0cd0 2024-04-25T08:39:31.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.824+0000 7fa00effd640 1 -- 172.21.15.38:0/3057004281 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa004032440 con 0x7fa0100d0cd0 2024-04-25T08:39:31.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.824+0000 7fa00effd640 1 -- 172.21.15.38:0/3057004281 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa004032810 con 0x7fa0100d0cd0 2024-04-25T08:39:31.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.825+0000 7fa0168ac640 1 -- 172.21.15.38:0/3057004281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0100d0cd0 msgr2=0x7fa0100d1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.825+0000 7fa0168ac640 1 --2- 172.21.15.38:0/3057004281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0100d0cd0 0x7fa0100d1130 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa004009870 tx=0x7fa00402a250 comp rx=0 tx=0).stop 2024-04-25T08:39:31.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.825+0000 7fa0168ac640 1 -- 172.21.15.38:0/3057004281 shutdown_connections 2024-04-25T08:39:31.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.825+0000 7fa0168ac640 1 --2- 172.21.15.38:0/3057004281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0100d0cd0 0x7fa0100d1130 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.825+0000 7fa0168ac640 1 --2- 172.21.15.38:0/3057004281 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa01015b730 0x7fa01015bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.825+0000 7fa0168ac640 1 -- 172.21.15.38:0/3057004281 >> 172.21.15.38:0/3057004281 conn(0x7fa0100b5c90 msgr2=0x7fa0100b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:31.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.825+0000 7fa0168ac640 1 -- 172.21.15.38:0/3057004281 shutdown_connections 2024-04-25T08:39:31.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.825+0000 7fa0168ac640 1 -- 172.21.15.38:0/3057004281 wait complete. 2024-04-25T08:39:31.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.826+0000 7fa0168ac640 1 Processor -- start 2024-04-25T08:39:31.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.826+0000 7fa0168ac640 1 -- start start 2024-04-25T08:39:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa0168ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0100ccd00 0x7fa010152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa0168ac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0cd0 0x7fa0100cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa0168ac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa01015b730 0x7fa01014eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa0168ac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0100d0370 con 0x7fa0100d0cd0 2024-04-25T08:39:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa0168ac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0100d0090 con 0x7fa01015b730 2024-04-25T08:39:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa0168ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0100cf4e0 con 0x7fa0100ccd00 2024-04-25T08:39:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa00f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0cd0 0x7fa0100cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa00f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0cd0 0x7fa0100cbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42500/0 (socket says 172.21.15.38:42500) 2024-04-25T08:39:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa00f7fe640 1 -- 172.21.15.38:0/301823825 learned_addr learned my addr 172.21.15.38:0/301823825 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa00ffff640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0100ccd00 0x7fa010152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa014e49640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa01015b730 0x7fa01014eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa00ffff640 1 -- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa01015b730 msgr2=0x7fa01014eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa00ffff640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa01015b730 0x7fa01014eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa00ffff640 1 -- 172.21.15.38:0/301823825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0cd0 msgr2=0x7fa0100cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa00ffff640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0cd0 0x7fa0100cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa00ffff640 1 -- 172.21.15.38:0/301823825 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa010152b00 con 0x7fa0100ccd00 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.827+0000 7fa00f7fe640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0cd0 0x7fa0100cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.828+0000 7fa00ffff640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0100ccd00 0x7fa010152420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa00000e930 tx=0x7fa00000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.828+0000 7fa00cff9640 1 -- 172.21.15.38:0/301823825 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000017070 con 0x7fa0100ccd00 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.828+0000 7fa00cff9640 1 -- 172.21.15.38:0/301823825 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00000ca00 con 0x7fa0100ccd00 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.828+0000 7fa0168ac640 1 -- 172.21.15.38:0/301823825 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0101550e0 con 0x7fa0100ccd00 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.828+0000 7fa00cff9640 1 -- 172.21.15.38:0/301823825 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00000cdd0 con 0x7fa0100ccd00 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.828+0000 7fa0168ac640 1 -- 172.21.15.38:0/301823825 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0101556a0 con 0x7fa0100ccd00 2024-04-25T08:39:31.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.829+0000 7fa0168ac640 1 -- 172.21.15.38:0/301823825 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa010068e80 con 0x7fa0100ccd00 2024-04-25T08:39:31.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.829+0000 7fa00cff9640 1 -- 172.21.15.38:0/301823825 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa00001a070 con 0x7fa0100ccd00 2024-04-25T08:39:31.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.830+0000 7fa00cff9640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ffc06b4d0 0x7f9ffc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:31.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.830+0000 7fa00cff9640 1 -- 172.21.15.38:0/301823825 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa00008a4d0 con 0x7fa0100ccd00 2024-04-25T08:39:31.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.830+0000 7fa00f7fe640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ffc06b4d0 0x7f9ffc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:31.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.830+0000 7fa00f7fe640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ffc06b4d0 0x7f9ffc06d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa0040090b0 tx=0x7fa00402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:31.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:31.834+0000 7fa00cff9640 1 -- 172.21.15.38:0/301823825 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa000058bd0 con 0x7fa0100ccd00 2024-04-25T08:39:32.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.084+0000 7fa0168ac640 1 -- 172.21.15.38:0/301823825 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa01015a3c0 con 0x7fa0100ccd00 2024-04-25T08:39:32.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.086+0000 7fa00cff9640 1 -- 172.21.15.38:0/301823825 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7fa00005cc70 con 0x7fa0100ccd00 2024-04-25T08:39:32.079 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:32.079 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":501,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":746,"data_bytes":758427586,"bytes_used":1895817216,"bytes_avail":771198296064,"bytes_total":773094113280,"write_bytes_sec":63146314,"read_op_per_sec":0,"write_op_per_sec":62},"fsmap":{"epoch":63,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8292}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T08:38:57.309082+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:39:32.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.087+0000 7fa0168ac640 1 -- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ffc06b4d0 msgr2=0x7f9ffc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:32.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.087+0000 7fa0168ac640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ffc06b4d0 0x7f9ffc06d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa0040090b0 tx=0x7fa00402e000 comp rx=0 tx=0).stop 2024-04-25T08:39:32.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.087+0000 7fa0168ac640 1 -- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0100ccd00 msgr2=0x7fa010152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:32.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.087+0000 7fa0168ac640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0100ccd00 0x7fa010152420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa00000e930 tx=0x7fa00000edf0 comp rx=0 tx=0).stop 2024-04-25T08:39:32.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.087+0000 7fa0168ac640 1 -- 172.21.15.38:0/301823825 shutdown_connections 2024-04-25T08:39:32.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.087+0000 7fa0168ac640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa01015b730 0x7fa01014eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:32.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.087+0000 7fa0168ac640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0cd0 0x7fa0100cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:32.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.087+0000 7fa0168ac640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ffc06b4d0 0x7f9ffc06d990 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:32.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.087+0000 7fa0168ac640 1 --2- 172.21.15.38:0/301823825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0100ccd00 0x7fa010152420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:32.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.087+0000 7fa0168ac640 1 -- 172.21.15.38:0/301823825 >> 172.21.15.38:0/301823825 conn(0x7fa0100b5c90 msgr2=0x7fa0101577c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:32.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.087+0000 7fa0168ac640 1 -- 172.21.15.38:0/301823825 shutdown_connections 2024-04-25T08:39:32.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:32.087+0000 7fa0168ac640 1 -- 172.21.15.38:0/301823825 wait complete. 2024-04-25T08:39:32.092 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:39:33.093 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:33.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.223+0000 7f6e5c6c4640 1 Processor -- start 2024-04-25T08:39:33.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.224+0000 7f6e5c6c4640 1 -- start start 2024-04-25T08:39:33.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e5c6c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5415d9d0 0x7f6e540cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e5c6c4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6e5406bf00 0x7f6e5406c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e5c6c4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e540d0240 con 0x7f6e5406cc40 2024-04-25T08:39:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e5c6c4640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5406d090 con 0x7f6e5406bf00 2024-04-25T08:39:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e5c6c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540d12e0 con 0x7f6e5415d9d0 2024-04-25T08:39:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e59c5f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6e5406bf00 0x7f6e5406c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e59c5f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6e5406bf00 0x7f6e5406c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42510/0 (socket says 172.21.15.38:42510) 2024-04-25T08:39:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e59c5f640 1 -- 172.21.15.38:0/2805717452 learned_addr learned my addr 172.21.15.38:0/2805717452 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e5a460640 1 --2- 172.21.15.38:0/2805717452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5415d9d0 0x7f6e540cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e59c5f640 1 -- 172.21.15.38:0/2805717452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5415d9d0 msgr2=0x7f6e540cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e59c5f640 1 --2- 172.21.15.38:0/2805717452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5415d9d0 0x7f6e540cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e59c5f640 1 -- 172.21.15.38:0/2805717452 >> v1:172.21.15.107:6789/0 conn(0x7f6e5406cc40 legacy=0x7f6e540b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:33.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.225+0000 7f6e59c5f640 1 -- 172.21.15.38:0/2805717452 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e540d0520 con 0x7f6e5406bf00 2024-04-25T08:39:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.226+0000 7f6e59c5f640 1 --2- 172.21.15.38:0/2805717452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5406bf00 0x7f6e5406c700 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6e440098a0 tx=0x7f6e4402a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c928db720f7f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.226+0000 7f6e5945e640 1 -- 172.21.15.38:0/2805717452 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44037070 con 0x7f6e5406bf00 2024-04-25T08:39:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.226+0000 7f6e5945e640 1 -- 172.21.15.38:0/2805717452 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e44032440 con 0x7f6e5406bf00 2024-04-25T08:39:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.226+0000 7f6e5945e640 1 -- 172.21.15.38:0/2805717452 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44032810 con 0x7f6e5406bf00 2024-04-25T08:39:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.227+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2805717452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5406bf00 msgr2=0x7f6e5406c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.227+0000 7f6e5c6c4640 1 --2- 172.21.15.38:0/2805717452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5406bf00 0x7f6e5406c700 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6e440098a0 tx=0x7f6e4402a290 comp rx=0 tx=0).stop 2024-04-25T08:39:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.227+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2805717452 shutdown_connections 2024-04-25T08:39:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.227+0000 7f6e5c6c4640 1 --2- 172.21.15.38:0/2805717452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5406bf00 0x7f6e5406c700 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.227+0000 7f6e5c6c4640 1 --2- 172.21.15.38:0/2805717452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5415d9d0 0x7f6e540cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.227+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2805717452 >> 172.21.15.38:0/2805717452 conn(0x7f6e540b3200 msgr2=0x7f6e540b3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.227+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2805717452 shutdown_connections 2024-04-25T08:39:33.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.227+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2805717452 wait complete. 2024-04-25T08:39:33.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.228+0000 7f6e5c6c4640 1 Processor -- start 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.228+0000 7f6e5c6c4640 1 -- start start 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.229+0000 7f6e5c6c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5406bf00 0x7f6e541f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.229+0000 7f6e5c6c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5406cc40 0x7f6e540c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.229+0000 7f6e5c6c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5415d9d0 0x7f6e540c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.229+0000 7f6e5c6c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540d0dc0 con 0x7f6e5406bf00 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.229+0000 7f6e5c6c4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540d0ae0 con 0x7f6e5406cc40 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.229+0000 7f6e5c6c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5409a680 con 0x7f6e5415d9d0 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.229+0000 7f6e5a460640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5406bf00 0x7f6e541f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.229+0000 7f6e5a460640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5406bf00 0x7f6e541f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42516/0 (socket says 172.21.15.38:42516) 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.229+0000 7f6e5a460640 1 -- 172.21.15.38:0/2184958136 learned_addr learned my addr 172.21.15.38:0/2184958136 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.229+0000 7f6e5ac61640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5415d9d0 0x7f6e540c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.230+0000 7f6e59c5f640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5406cc40 0x7f6e540c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.230+0000 7f6e5a460640 1 -- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5415d9d0 msgr2=0x7f6e540c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:33.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.230+0000 7f6e5a460640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5415d9d0 0x7f6e540c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.230+0000 7f6e5a460640 1 -- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5406cc40 msgr2=0x7f6e540c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.230+0000 7f6e5a460640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5406cc40 0x7f6e540c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.230+0000 7f6e5a460640 1 -- 172.21.15.38:0/2184958136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e54150150 con 0x7f6e5406bf00 2024-04-25T08:39:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.230+0000 7f6e5a460640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5406bf00 0x7f6e541f3580 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e8a0 tx=0x7f6e5000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.230+0000 7f6e4affd640 1 -- 172.21.15.38:0/2184958136 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50017070 con 0x7f6e5406bf00 2024-04-25T08:39:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.230+0000 7f6e4affd640 1 -- 172.21.15.38:0/2184958136 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5000c940 con 0x7f6e5406bf00 2024-04-25T08:39:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.230+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2184958136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e540c77f0 con 0x7f6e5406bf00 2024-04-25T08:39:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.230+0000 7f6e4affd640 1 -- 172.21.15.38:0/2184958136 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000cd30 con 0x7f6e5406bf00 2024-04-25T08:39:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.230+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2184958136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e541e6520 con 0x7f6e5406bf00 2024-04-25T08:39:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.231+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2184958136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e54068e80 con 0x7f6e5406bf00 2024-04-25T08:39:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.232+0000 7f6e4affd640 1 -- 172.21.15.38:0/2184958136 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e5001a070 con 0x7f6e5406bf00 2024-04-25T08:39:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.232+0000 7f6e4affd640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e3806b4d0 0x7f6e3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.232+0000 7f6e4affd640 1 -- 172.21.15.38:0/2184958136 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f6e5008a1a0 con 0x7f6e5406bf00 2024-04-25T08:39:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.233+0000 7f6e59c5f640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e3806b4d0 0x7f6e3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.233+0000 7f6e59c5f640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e3806b4d0 0x7f6e3806d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6e44034000 tx=0x7f6e4402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.237+0000 7f6e4affd640 1 -- 172.21.15.38:0/2184958136 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e50058810 con 0x7f6e5406bf00 2024-04-25T08:39:33.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.464+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2184958136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e540b40a0 con 0x7f6e5406bf00 2024-04-25T08:39:33.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.464+0000 7f6e4affd640 1 -- 172.21.15.38:0/2184958136 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f6e5005c8b0 con 0x7f6e5406bf00 2024-04-25T08:39:33.459 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:33.460 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:07.551641+0000","modified":"2024-04-25T08:39:09.555958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8292":{"gid":8292,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1263010516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1263010516},{"type":"v1","addr":"172.21.15.107:6837","nonce":1263010516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:39:33.460 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 63 2024-04-25T08:39:33.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.466+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e3806b4d0 msgr2=0x7f6e3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:33.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.466+0000 7f6e5c6c4640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e3806b4d0 0x7f6e3806d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6e44034000 tx=0x7f6e4402e000 comp rx=0 tx=0).stop 2024-04-25T08:39:33.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.466+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2184958136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5406bf00 msgr2=0x7f6e541f3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:33.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.466+0000 7f6e5c6c4640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5406bf00 0x7f6e541f3580 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000e8a0 tx=0x7f6e5000ed60 comp rx=0 tx=0).stop 2024-04-25T08:39:33.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.466+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2184958136 shutdown_connections 2024-04-25T08:39:33.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.466+0000 7f6e5c6c4640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e3806b4d0 0x7f6e3806d990 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.466+0000 7f6e5c6c4640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e5415d9d0 0x7f6e540c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.466+0000 7f6e5c6c4640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e5406cc40 0x7f6e540c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.466+0000 7f6e5c6c4640 1 --2- 172.21.15.38:0/2184958136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e5406bf00 0x7f6e541f3580 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.466+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2184958136 >> 172.21.15.38:0/2184958136 conn(0x7f6e540b3200 msgr2=0x7f6e540b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:33.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.466+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2184958136 shutdown_connections 2024-04-25T08:39:33.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.466+0000 7f6e5c6c4640 1 -- 172.21.15.38:0/2184958136 wait complete. 2024-04-25T08:39:33.471 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:39:33.472 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:39:33.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.577+0000 7f45629f5640 1 Processor -- start 2024-04-25T08:39:33.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.577+0000 7f45629f5640 1 -- start start 2024-04-25T08:39:33.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.578+0000 7f45629f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c15dc40 0x7f455c0d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:33.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.578+0000 7f45629f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f455c0d0e10 0x7f455c0b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.578+0000 7f45629f5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455c0b8400 con 0x7f455c0b9580 2024-04-25T08:39:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.578+0000 7f45629f5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455c0d1260 con 0x7f455c0d0e10 2024-04-25T08:39:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.578+0000 7f45629f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0d19b0 con 0x7f455c15dc40 2024-04-25T08:39:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.578+0000 7f455bfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c15dc40 0x7f455c0d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.578+0000 7f4560f92640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f455c0d0e10 0x7f455c0b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.578+0000 7f455b7fe640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f455c0b9580 0x7f455c0d1e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:40934/0 (socket says 172.21.15.38:40934) 2024-04-25T08:39:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.578+0000 7f455bfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c15dc40 0x7f455c0d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:36354/0 (socket says 172.21.15.38:36354) 2024-04-25T08:39:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.578+0000 7f455b7fe640 1 -- 172.21.15.38:0/2428394130 learned_addr learned my addr 172.21.15.38:0/2428394130 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.579+0000 7f455bfff640 1 -- 172.21.15.38:0/2428394130 >> v2:172.21.15.107:3300/0 conn(0x7f455c0d0e10 msgr2=0x7f455c0b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:33.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.579+0000 7f455bfff640 1 --2- 172.21.15.38:0/2428394130 >> v2:172.21.15.107:3300/0 conn(0x7f455c0d0e10 0x7f455c0b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.579+0000 7f455bfff640 1 -- 172.21.15.38:0/2428394130 >> v1:172.21.15.107:6789/0 conn(0x7f455c0b9580 legacy=0x7f455c0d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:33.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.579+0000 7f455affd640 1 -- 172.21.15.38:0/2428394130 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450032741 0 0) 0x7f455c0b8400 con 0x7f455c0b9580 2024-04-25T08:39:33.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.579+0000 7f455bfff640 1 -- 172.21.15.38:0/2428394130 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c0b86e0 con 0x7f455c15dc40 2024-04-25T08:39:33.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.579+0000 7f455bfff640 1 --2- 172.21.15.38:0/2428394130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c15dc40 0x7f455c0d08d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f45500097a0 tx=0x7f455002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b999f4b2eebb2050 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:33.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.579+0000 7f455affd640 1 -- 172.21.15.38:0/2428394130 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550037070 con 0x7f455c15dc40 2024-04-25T08:39:33.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.580+0000 7f455affd640 1 -- 172.21.15.38:0/2428394130 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4550032440 con 0x7f455c15dc40 2024-04-25T08:39:33.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.580+0000 7f455affd640 1 -- 172.21.15.38:0/2428394130 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550032810 con 0x7f455c15dc40 2024-04-25T08:39:33.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.580+0000 7f45629f5640 1 -- 172.21.15.38:0/2428394130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c15dc40 msgr2=0x7f455c0d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:33.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.580+0000 7f45629f5640 1 --2- 172.21.15.38:0/2428394130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c15dc40 0x7f455c0d08d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f45500097a0 tx=0x7f455002a180 comp rx=0 tx=0).stop 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.580+0000 7f45629f5640 1 -- 172.21.15.38:0/2428394130 shutdown_connections 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.580+0000 7f45629f5640 1 --2- 172.21.15.38:0/2428394130 >> v2:172.21.15.107:3300/0 conn(0x7f455c0d0e10 0x7f455c0b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.580+0000 7f45629f5640 1 --2- 172.21.15.38:0/2428394130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c15dc40 0x7f455c0d08d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.580+0000 7f45629f5640 1 -- 172.21.15.38:0/2428394130 >> 172.21.15.38:0/2428394130 conn(0x7f455c066ce0 msgr2=0x7f455c0670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.580+0000 7f45629f5640 1 -- 172.21.15.38:0/2428394130 shutdown_connections 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.580+0000 7f45629f5640 1 -- 172.21.15.38:0/2428394130 wait complete. 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.581+0000 7f45629f5640 1 Processor -- start 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.581+0000 7f45629f5640 1 -- start start 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.582+0000 7f45629f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c0b9580 0x7f455c0cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.582+0000 7f45629f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f455c0d0e10 0x7f455c1439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.582+0000 7f45629f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c15dc40 0x7f455c0cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.582+0000 7f45629f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0b7fb0 con 0x7f455c0b9580 2024-04-25T08:39:33.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.582+0000 7f45629f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0b8ca0 con 0x7f455c0d0e10 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.582+0000 7f45629f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0b8130 con 0x7f455c15dc40 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.582+0000 7f455bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c0b9580 0x7f455c0cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.582+0000 7f455bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c0b9580 0x7f455c0cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42528/0 (socket says 172.21.15.38:42528) 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.582+0000 7f455bfff640 1 -- 172.21.15.38:0/3610129555 learned_addr learned my addr 172.21.15.38:0/3610129555 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.582+0000 7f4560f92640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c15dc40 0x7f455c0cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.582+0000 7f455b7fe640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f455c0d0e10 0x7f455c1439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.583+0000 7f455bfff640 1 -- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c15dc40 msgr2=0x7f455c0cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.583+0000 7f455bfff640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c15dc40 0x7f455c0cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.583+0000 7f455bfff640 1 -- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f455c0d0e10 msgr2=0x7f455c1439c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.583+0000 7f455bfff640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f455c0d0e10 0x7f455c1439c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.583+0000 7f455bfff640 1 -- 172.21.15.38:0/3610129555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c160f90 con 0x7f455c0b9580 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.583+0000 7f455bfff640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c0b9580 0x7f455c0cbcc0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f45500098c0 tx=0x7f45500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.583+0000 7f4558ff9640 1 -- 172.21.15.38:0/3610129555 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550037040 con 0x7f455c0b9580 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.583+0000 7f4558ff9640 1 -- 172.21.15.38:0/3610129555 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4550042070 con 0x7f455c0b9580 2024-04-25T08:39:33.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.583+0000 7f45629f5640 1 -- 172.21.15.38:0/3610129555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455c15e270 con 0x7f455c0b9580 2024-04-25T08:39:33.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.583+0000 7f4558ff9640 1 -- 172.21.15.38:0/3610129555 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550030920 con 0x7f455c0b9580 2024-04-25T08:39:33.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.583+0000 7f45629f5640 1 -- 172.21.15.38:0/3610129555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c0cc200 con 0x7f455c0b9580 2024-04-25T08:39:33.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.584+0000 7f45629f5640 1 -- 172.21.15.38:0/3610129555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455c068e80 con 0x7f455c0b9580 2024-04-25T08:39:33.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.585+0000 7f4558ff9640 1 -- 172.21.15.38:0/3610129555 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4550030ac0 con 0x7f455c0b9580 2024-04-25T08:39:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.585+0000 7f4558ff9640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f454806b4d0 0x7f454806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.585+0000 7f4558ff9640 1 -- 172.21.15.38:0/3610129555 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f45500aabd0 con 0x7f455c0b9580 2024-04-25T08:39:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.585+0000 7f455b7fe640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f454806b4d0 0x7f454806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:33.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.586+0000 7f455b7fe640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f454806b4d0 0x7f454806d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f455c1f6b90 tx=0x7f454c007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:33.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.589+0000 7f4558ff9640 1 -- 172.21.15.38:0/3610129555 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4550079240 con 0x7f455c0b9580 2024-04-25T08:39:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.838+0000 7f45629f5640 1 -- 172.21.15.38:0/3610129555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f455c0b31d0 con 0x7f455c0b9580 2024-04-25T08:39:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.838+0000 7f4558ff9640 1 -- 172.21.15.38:0/3610129555 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7f455007d2e0 con 0x7f455c0b9580 2024-04-25T08:39:33.832 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:33.833 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":502,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":746,"data_bytes":758427586,"bytes_used":1895817216,"bytes_avail":771198296064,"bytes_total":773094113280,"write_bytes_sec":63146314,"read_op_per_sec":0,"write_op_per_sec":62},"fsmap":{"epoch":63,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8292}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T08:38:57.309082+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:39:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.840+0000 7f45629f5640 1 -- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f454806b4d0 msgr2=0x7f454806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.840+0000 7f45629f5640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f454806b4d0 0x7f454806d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f455c1f6b90 tx=0x7f454c007d50 comp rx=0 tx=0).stop 2024-04-25T08:39:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.840+0000 7f45629f5640 1 -- 172.21.15.38:0/3610129555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c0b9580 msgr2=0x7f455c0cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.840+0000 7f45629f5640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c0b9580 0x7f455c0cbcc0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f45500098c0 tx=0x7f45500046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.840+0000 7f45629f5640 1 -- 172.21.15.38:0/3610129555 shutdown_connections 2024-04-25T08:39:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.840+0000 7f45629f5640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f454806b4d0 0x7f454806d990 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.840+0000 7f45629f5640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f455c15dc40 0x7f455c0cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.840+0000 7f45629f5640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f455c0d0e10 0x7f455c1439c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.840+0000 7f45629f5640 1 --2- 172.21.15.38:0/3610129555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f455c0b9580 0x7f455c0cbcc0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.840+0000 7f45629f5640 1 -- 172.21.15.38:0/3610129555 >> 172.21.15.38:0/3610129555 conn(0x7f455c066ce0 msgr2=0x7f455c15ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.840+0000 7f45629f5640 1 -- 172.21.15.38:0/3610129555 shutdown_connections 2024-04-25T08:39:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:33.840+0000 7f45629f5640 1 -- 172.21.15.38:0/3610129555 wait complete. 2024-04-25T08:39:33.845 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:39:34.224 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:39:34.227 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:39:34.845 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:34.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.965+0000 7f5e65031640 1 Processor -- start 2024-04-25T08:39:34.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.965+0000 7f5e65031640 1 -- start start 2024-04-25T08:39:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.965+0000 7f5e65031640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5e6006c110 0x7f5e6006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.965+0000 7f5e65031640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5e6015df50 0x7f5e600ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.965+0000 7f5e65031640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6006ca30 con 0x7f5e600ce2e0 2024-04-25T08:39:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.965+0000 7f5e65031640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e600cf2d0 con 0x7f5e6006c110 2024-04-25T08:39:34.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.965+0000 7f5e65031640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e600ceff0 con 0x7f5e6015df50 2024-04-25T08:39:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.966+0000 7f5e5ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5e6006c110 0x7f5e6006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.966+0000 7f5e5ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5e6006c110 0x7f5e6006c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42536/0 (socket says 172.21.15.38:42536) 2024-04-25T08:39:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.966+0000 7f5e5ed9d640 1 -- 172.21.15.38:0/4290256149 learned_addr learned my addr 172.21.15.38:0/4290256149 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.966+0000 7f5e5e59c640 1 --2- 172.21.15.38:0/4290256149 >> v2:172.21.15.107:3300/0 conn(0x7f5e6015df50 0x7f5e600ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.966+0000 7f5e5ed9d640 1 -- 172.21.15.38:0/4290256149 >> v2:172.21.15.107:3300/0 conn(0x7f5e6015df50 msgr2=0x7f5e600ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.966+0000 7f5e5ed9d640 1 --2- 172.21.15.38:0/4290256149 >> v2:172.21.15.107:3300/0 conn(0x7f5e6015df50 0x7f5e600ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.966+0000 7f5e5ed9d640 1 -- 172.21.15.38:0/4290256149 >> v1:172.21.15.107:6789/0 conn(0x7f5e600ce2e0 legacy=0x7f5e600d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.966+0000 7f5e5ed9d640 1 -- 172.21.15.38:0/4290256149 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e600d1f70 con 0x7f5e6006c110 2024-04-25T08:39:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.967+0000 7f5e5ed9d640 1 --2- 172.21.15.38:0/4290256149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e6006c110 0x7f5e6006c4f0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f5e600d4960 tx=0x7f5e5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d0c4e8231f0edf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.967+0000 7f5e5dd9b640 1 -- 172.21.15.38:0/4290256149 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50037070 con 0x7f5e6006c110 2024-04-25T08:39:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.967+0000 7f5e5dd9b640 1 -- 172.21.15.38:0/4290256149 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e50032440 con 0x7f5e6006c110 2024-04-25T08:39:34.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.967+0000 7f5e5dd9b640 1 -- 172.21.15.38:0/4290256149 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50032810 con 0x7f5e6006c110 2024-04-25T08:39:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.967+0000 7f5e65031640 1 -- 172.21.15.38:0/4290256149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e6006c110 msgr2=0x7f5e6006c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.967+0000 7f5e65031640 1 --2- 172.21.15.38:0/4290256149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e6006c110 0x7f5e6006c4f0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f5e600d4960 tx=0x7f5e5002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.968+0000 7f5e65031640 1 -- 172.21.15.38:0/4290256149 shutdown_connections 2024-04-25T08:39:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.968+0000 7f5e65031640 1 --2- 172.21.15.38:0/4290256149 >> v2:172.21.15.107:3300/0 conn(0x7f5e6015df50 0x7f5e600ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.968+0000 7f5e65031640 1 --2- 172.21.15.38:0/4290256149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e6006c110 0x7f5e6006c4f0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.968+0000 7f5e65031640 1 -- 172.21.15.38:0/4290256149 >> 172.21.15.38:0/4290256149 conn(0x7f5e600d3fb0 msgr2=0x7f5e600d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.968+0000 7f5e65031640 1 -- 172.21.15.38:0/4290256149 shutdown_connections 2024-04-25T08:39:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.968+0000 7f5e65031640 1 -- 172.21.15.38:0/4290256149 wait complete. 2024-04-25T08:39:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.969+0000 7f5e65031640 1 Processor -- start 2024-04-25T08:39:34.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.969+0000 7f5e65031640 1 -- start start 2024-04-25T08:39:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e65031640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e6006c110 0x7f5e601f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e65031640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e600ce2e0 0x7f5e6015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e65031640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e6015df50 0x7f5e600c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e65031640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600cece0 con 0x7f5e6015df50 2024-04-25T08:39:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e65031640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600d2530 con 0x7f5e600ce2e0 2024-04-25T08:39:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e65031640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e600d3160 con 0x7f5e6006c110 2024-04-25T08:39:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e5f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e6015df50 0x7f5e600c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e5f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e6015df50 0x7f5e600c9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42540/0 (socket says 172.21.15.38:42540) 2024-04-25T08:39:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e5f59e640 1 -- 172.21.15.38:0/1497018617 learned_addr learned my addr 172.21.15.38:0/1497018617 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e5e59c640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e600ce2e0 0x7f5e6015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e5ed9d640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e6006c110 0x7f5e601f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e5f59e640 1 -- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e6006c110 msgr2=0x7f5e601f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e5f59e640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e6006c110 0x7f5e601f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e5f59e640 1 -- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e600ce2e0 msgr2=0x7f5e6015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e5f59e640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e600ce2e0 0x7f5e6015afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e5f59e640 1 -- 172.21.15.38:0/1497018617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e600c8680 con 0x7f5e6015df50 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e5e59c640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e600ce2e0 0x7f5e6015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.970+0000 7f5e5ed9d640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e6006c110 0x7f5e601f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.971+0000 7f5e5f59e640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e6015df50 0x7f5e600c9ae0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800c7d0 tx=0x7f5e5800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.971+0000 7f5e477fe640 1 -- 172.21.15.38:0/1497018617 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e58003b70 con 0x7f5e6015df50 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.971+0000 7f5e477fe640 1 -- 172.21.15.38:0/1497018617 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e58003d30 con 0x7f5e6015df50 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.971+0000 7f5e65031640 1 -- 172.21.15.38:0/1497018617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e600c8960 con 0x7f5e6015df50 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.971+0000 7f5e477fe640 1 -- 172.21.15.38:0/1497018617 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800aa90 con 0x7f5e6015df50 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.971+0000 7f5e65031640 1 -- 172.21.15.38:0/1497018617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e600c8f20 con 0x7f5e6015df50 2024-04-25T08:39:34.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.972+0000 7f5e65031640 1 -- 172.21.15.38:0/1497018617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e60068e80 con 0x7f5e6015df50 2024-04-25T08:39:34.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.972+0000 7f5e477fe640 1 -- 172.21.15.38:0/1497018617 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e5800ac30 con 0x7f5e6015df50 2024-04-25T08:39:34.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.973+0000 7f5e477fe640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e4c06b4d0 0x7f5e4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:34.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.973+0000 7f5e477fe640 1 -- 172.21.15.38:0/1497018617 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5e5808a3a0 con 0x7f5e6015df50 2024-04-25T08:39:34.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.973+0000 7f5e5ed9d640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e4c06b4d0 0x7f5e4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:34.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.973+0000 7f5e5ed9d640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e4c06b4d0 0x7f5e4c06d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5e50034000 tx=0x7f5e5002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:34.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:34.977+0000 7f5e477fe640 1 -- 172.21.15.38:0/1497018617 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e58058fc0 con 0x7f5e6015df50 2024-04-25T08:39:35.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.194+0000 7f5e65031640 1 -- 172.21.15.38:0/1497018617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e600738a0 con 0x7f5e6015df50 2024-04-25T08:39:35.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.195+0000 7f5e477fe640 1 -- 172.21.15.38:0/1497018617 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f5e580588c0 con 0x7f5e6015df50 2024-04-25T08:39:35.189 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:35.189 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:07.551641+0000","modified":"2024-04-25T08:39:09.555958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8292":{"gid":8292,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1263010516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1263010516},{"type":"v1","addr":"172.21.15.107:6837","nonce":1263010516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:39:35.189 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 63 2024-04-25T08:39:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.196+0000 7f5e65031640 1 -- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e4c06b4d0 msgr2=0x7f5e4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.196+0000 7f5e65031640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e4c06b4d0 0x7f5e4c06d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5e50034000 tx=0x7f5e5002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.196+0000 7f5e65031640 1 -- 172.21.15.38:0/1497018617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e6015df50 msgr2=0x7f5e600c9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.196+0000 7f5e65031640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e6015df50 0x7f5e600c9ae0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800c7d0 tx=0x7f5e5800cc90 comp rx=0 tx=0).stop 2024-04-25T08:39:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.196+0000 7f5e65031640 1 -- 172.21.15.38:0/1497018617 shutdown_connections 2024-04-25T08:39:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.196+0000 7f5e65031640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e6015df50 0x7f5e600c9ae0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.196+0000 7f5e65031640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e600ce2e0 0x7f5e6015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.196+0000 7f5e65031640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e4c06b4d0 0x7f5e4c06d990 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.196+0000 7f5e65031640 1 --2- 172.21.15.38:0/1497018617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e6006c110 0x7f5e601f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.196+0000 7f5e65031640 1 -- 172.21.15.38:0/1497018617 >> 172.21.15.38:0/1497018617 conn(0x7f5e600d3fb0 msgr2=0x7f5e600b8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:35.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.197+0000 7f5e65031640 1 -- 172.21.15.38:0/1497018617 shutdown_connections 2024-04-25T08:39:35.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.197+0000 7f5e65031640 1 -- 172.21.15.38:0/1497018617 wait complete. 2024-04-25T08:39:35.202 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:39:35.202 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:39:35.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.307+0000 7f3f43f14640 1 Processor -- start 2024-04-25T08:39:35.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.308+0000 7f3f43f14640 1 -- start start 2024-04-25T08:39:35.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.308+0000 7f3f43f14640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3f3c06abe0 0x7f3f3c06b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.308+0000 7f3f43f14640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3f3c0d01d0 0x7f3f3c0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.308+0000 7f3f43f14640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3c0cfa40 con 0x7f3f3c0aeeb0 2024-04-25T08:39:35.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.308+0000 7f3f43f14640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f3c0cd9b0 con 0x7f3f3c06abe0 2024-04-25T08:39:35.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.308+0000 7f3f43f14640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f3c0cdc90 con 0x7f3f3c0d01d0 2024-04-25T08:39:35.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.309+0000 7f3f414af640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3f3c06abe0 0x7f3f3c06b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.309+0000 7f3f414af640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3f3c06abe0 0x7f3f3c06b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42546/0 (socket says 172.21.15.38:42546) 2024-04-25T08:39:35.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.309+0000 7f3f424b1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3f3c0d01d0 0x7f3f3c0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.309+0000 7f3f414af640 1 -- 172.21.15.38:0/3409852870 learned_addr learned my addr 172.21.15.38:0/3409852870 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:35.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.309+0000 7f3f414af640 1 -- 172.21.15.38:0/3409852870 >> v2:172.21.15.107:3300/0 conn(0x7f3f3c0d01d0 msgr2=0x7f3f3c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:35.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.309+0000 7f3f414af640 1 --2- 172.21.15.38:0/3409852870 >> v2:172.21.15.107:3300/0 conn(0x7f3f3c0d01d0 0x7f3f3c0d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.309+0000 7f3f414af640 1 -- 172.21.15.38:0/3409852870 >> v1:172.21.15.107:6789/0 conn(0x7f3f3c0aeeb0 legacy=0x7f3f3c0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.309+0000 7f3f414af640 1 -- 172.21.15.38:0/3409852870 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c15e390 con 0x7f3f3c06abe0 2024-04-25T08:39:35.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.310+0000 7f3f414af640 1 --2- 172.21.15.38:0/3409852870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f3c06abe0 0x7f3f3c06b090 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0b66f0 tx=0x7f3f3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca957db70ea1d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.310+0000 7f3f40cae640 1 -- 172.21.15.38:0/3409852870 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30037070 con 0x7f3f3c06abe0 2024-04-25T08:39:35.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.310+0000 7f3f40cae640 1 -- 172.21.15.38:0/3409852870 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f30032440 con 0x7f3f3c06abe0 2024-04-25T08:39:35.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.310+0000 7f3f40cae640 1 -- 172.21.15.38:0/3409852870 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30032810 con 0x7f3f3c06abe0 2024-04-25T08:39:35.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.310+0000 7f3f43f14640 1 -- 172.21.15.38:0/3409852870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f3c06abe0 msgr2=0x7f3f3c06b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.310+0000 7f3f43f14640 1 --2- 172.21.15.38:0/3409852870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f3c06abe0 0x7f3f3c06b090 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0b66f0 tx=0x7f3f3002a270 comp rx=0 tx=0).stop 2024-04-25T08:39:35.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.310+0000 7f3f43f14640 1 -- 172.21.15.38:0/3409852870 shutdown_connections 2024-04-25T08:39:35.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.310+0000 7f3f43f14640 1 --2- 172.21.15.38:0/3409852870 >> v2:172.21.15.107:3300/0 conn(0x7f3f3c0d01d0 0x7f3f3c0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.310+0000 7f3f43f14640 1 --2- 172.21.15.38:0/3409852870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f3c06abe0 0x7f3f3c06b090 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.310+0000 7f3f43f14640 1 -- 172.21.15.38:0/3409852870 >> 172.21.15.38:0/3409852870 conn(0x7f3f3c072610 msgr2=0x7f3f3c099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:35.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.310+0000 7f3f43f14640 1 -- 172.21.15.38:0/3409852870 shutdown_connections 2024-04-25T08:39:35.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.310+0000 7f3f43f14640 1 -- 172.21.15.38:0/3409852870 wait complete. 2024-04-25T08:39:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.312+0000 7f3f43f14640 1 Processor -- start 2024-04-25T08:39:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.312+0000 7f3f43f14640 1 -- start start 2024-04-25T08:39:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.312+0000 7f3f43f14640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f3c06abe0 0x7f3f3c14cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.312+0000 7f3f43f14640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f3c0aeeb0 0x7f3f3c0b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.312+0000 7f3f43f14640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f3c0d01d0 0x7f3f3c0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.312+0000 7f3f43f14640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0d0af0 con 0x7f3f3c06abe0 2024-04-25T08:39:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.312+0000 7f3f43f14640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0cd620 con 0x7f3f3c0d01d0 2024-04-25T08:39:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.312+0000 7f3f43f14640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c0d0c70 con 0x7f3f3c0aeeb0 2024-04-25T08:39:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.312+0000 7f3f41cb0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f3c06abe0 0x7f3f3c14cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.312+0000 7f3f41cb0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f3c06abe0 0x7f3f3c14cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42562/0 (socket says 172.21.15.38:42562) 2024-04-25T08:39:35.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.312+0000 7f3f41cb0640 1 -- 172.21.15.38:0/921627343 learned_addr learned my addr 172.21.15.38:0/921627343 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f414af640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f3c0aeeb0 0x7f3f3c0b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f424b1640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f3c0d01d0 0x7f3f3c0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f41cb0640 1 -- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f3c0aeeb0 msgr2=0x7f3f3c0b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f41cb0640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f3c0aeeb0 0x7f3f3c0b4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f41cb0640 1 -- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f3c0d01d0 msgr2=0x7f3f3c0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f41cb0640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f3c0d01d0 0x7f3f3c0b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f41cb0640 1 -- 172.21.15.38:0/921627343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c1f4740 con 0x7f3f3c06abe0 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f41cb0640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f3c06abe0 0x7f3f3c14cb40 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400c400 tx=0x7f3f3400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f3a7fc640 1 -- 172.21.15.38:0/921627343 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f340052d0 con 0x7f3f3c06abe0 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f3a7fc640 1 -- 172.21.15.38:0/921627343 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f34005490 con 0x7f3f3c06abe0 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f43f14640 1 -- 172.21.15.38:0/921627343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3c1f3980 con 0x7f3f3c06abe0 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f3a7fc640 1 -- 172.21.15.38:0/921627343 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f34007260 con 0x7f3f3c06abe0 2024-04-25T08:39:35.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.313+0000 7f3f43f14640 1 -- 172.21.15.38:0/921627343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c1f3f40 con 0x7f3f3c06abe0 2024-04-25T08:39:35.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.314+0000 7f3f43f14640 1 -- 172.21.15.38:0/921627343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3c1562f0 con 0x7f3f3c06abe0 2024-04-25T08:39:35.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.315+0000 7f3f3a7fc640 1 -- 172.21.15.38:0/921627343 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f34007400 con 0x7f3f3c06abe0 2024-04-25T08:39:35.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.315+0000 7f3f3a7fc640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3f2806b500 0x7f3f2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:35.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.315+0000 7f3f3a7fc640 1 -- 172.21.15.38:0/921627343 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3f34089860 con 0x7f3f3c06abe0 2024-04-25T08:39:35.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.315+0000 7f3f414af640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3f2806b500 0x7f3f2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:35.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.316+0000 7f3f414af640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3f2806b500 0x7f3f2806d9c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3f30034000 tx=0x7f3f3002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:35.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.320+0000 7f3f3a7fc640 1 -- 172.21.15.38:0/921627343 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f340585b0 con 0x7f3f3c06abe0 2024-04-25T08:39:35.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.573+0000 7f3f43f14640 1 -- 172.21.15.38:0/921627343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3f3c0cc740 con 0x7f3f3c06abe0 2024-04-25T08:39:35.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.574+0000 7f3f3a7fc640 1 -- 172.21.15.38:0/921627343 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1446 (secure 0 0 0) 0x7f3f34057eb0 con 0x7f3f3c06abe0 2024-04-25T08:39:35.568 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:35.568 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":504,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076564125,"bytes_used":2656870400,"bytes_avail":770437242880,"bytes_total":773094113280,"write_bytes_sec":89656585,"read_op_per_sec":0,"write_op_per_sec":89},"fsmap":{"epoch":63,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8292}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T08:38:57.309082+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:39:35.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.575+0000 7f3f43f14640 1 -- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3f2806b500 msgr2=0x7f3f2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.575+0000 7f3f43f14640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3f2806b500 0x7f3f2806d9c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3f30034000 tx=0x7f3f3002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:35.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.575+0000 7f3f43f14640 1 -- 172.21.15.38:0/921627343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f3c06abe0 msgr2=0x7f3f3c14cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:35.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.575+0000 7f3f43f14640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f3c06abe0 0x7f3f3c14cb40 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400c400 tx=0x7f3f3400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.575+0000 7f3f43f14640 1 -- 172.21.15.38:0/921627343 shutdown_connections 2024-04-25T08:39:35.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.576+0000 7f3f43f14640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f3c0d01d0 0x7f3f3c0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.576+0000 7f3f43f14640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3f2806b500 0x7f3f2806d9c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.576+0000 7f3f43f14640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3f3c0aeeb0 0x7f3f3c0b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.576+0000 7f3f43f14640 1 --2- 172.21.15.38:0/921627343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f3c06abe0 0x7f3f3c14cb40 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:35.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.576+0000 7f3f43f14640 1 -- 172.21.15.38:0/921627343 >> 172.21.15.38:0/921627343 conn(0x7f3f3c072610 msgr2=0x7f3f3c156af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:35.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.576+0000 7f3f43f14640 1 -- 172.21.15.38:0/921627343 shutdown_connections 2024-04-25T08:39:35.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:35.576+0000 7f3f43f14640 1 -- 172.21.15.38:0/921627343 wait complete. 2024-04-25T08:39:35.580 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:39:36.582 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:36.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.702+0000 7f9fe0fa9640 1 Processor -- start 2024-04-25T08:39:36.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.702+0000 7f9fe0fa9640 1 -- start start 2024-04-25T08:39:36.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.703+0000 7f9fe0fa9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9fdc099a70 0x7f9fdc099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:36.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.703+0000 7f9fe0fa9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fdc0cf470 0x7f9fdc0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:36.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.703+0000 7f9fe0fa9640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc09a390 con 0x7f9fdc06bd80 2024-04-25T08:39:36.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.703+0000 7f9fe0fa9640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fdc1614d0 con 0x7f9fdc099a70 2024-04-25T08:39:36.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.703+0000 7f9fe0fa9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0cd970 con 0x7f9fdc0cf470 2024-04-25T08:39:36.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.703+0000 7f9fda59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9fdc099a70 0x7f9fdc099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:36.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.703+0000 7f9fda59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9fdc099a70 0x7f9fdc099e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42576/0 (socket says 172.21.15.38:42576) 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.703+0000 7f9fda59c640 1 -- 172.21.15.38:0/2837255009 learned_addr learned my addr 172.21.15.38:0/2837255009 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.703+0000 7f9fda59c640 1 -- 172.21.15.38:0/2837255009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fdc0cf470 msgr2=0x7f9fdc0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.703+0000 7f9fda59c640 1 --2- 172.21.15.38:0/2837255009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fdc0cf470 0x7f9fdc0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.703+0000 7f9fda59c640 1 -- 172.21.15.38:0/2837255009 >> v1:172.21.15.107:6789/0 conn(0x7f9fdc06bd80 legacy=0x7f9fdc06c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.703+0000 7f9fda59c640 1 -- 172.21.15.38:0/2837255009 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0ccd60 con 0x7f9fdc099a70 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.704+0000 7f9fda59c640 1 --2- 172.21.15.38:0/2837255009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fdc099a70 0x7f9fdc099e50 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009870 tx=0x7f9fcc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f2923e06f66025 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.704+0000 7f9fd9d9b640 1 -- 172.21.15.38:0/2837255009 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc037070 con 0x7f9fdc099a70 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.704+0000 7f9fd9d9b640 1 -- 172.21.15.38:0/2837255009 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc032440 con 0x7f9fdc099a70 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.704+0000 7f9fd9d9b640 1 -- 172.21.15.38:0/2837255009 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc032810 con 0x7f9fdc099a70 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.705+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2837255009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fdc099a70 msgr2=0x7f9fdc099e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.705+0000 7f9fe0fa9640 1 --2- 172.21.15.38:0/2837255009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fdc099a70 0x7f9fdc099e50 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009870 tx=0x7f9fcc02a210 comp rx=0 tx=0).stop 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.705+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2837255009 shutdown_connections 2024-04-25T08:39:36.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.705+0000 7f9fe0fa9640 1 --2- 172.21.15.38:0/2837255009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fdc0cf470 0x7f9fdc0cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.705+0000 7f9fe0fa9640 1 --2- 172.21.15.38:0/2837255009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fdc099a70 0x7f9fdc099e50 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.705+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2837255009 >> 172.21.15.38:0/2837255009 conn(0x7f9fdc0b3180 msgr2=0x7f9fdc0b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:36.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.705+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2837255009 shutdown_connections 2024-04-25T08:39:36.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.705+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2837255009 wait complete. 2024-04-25T08:39:36.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.706+0000 7f9fe0fa9640 1 Processor -- start 2024-04-25T08:39:36.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.706+0000 7f9fe0fa9640 1 -- start start 2024-04-25T08:39:36.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fe0fa9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fdc06bd80 0x7f9fdc1f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:36.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fe0fa9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fdc099a70 0x7f9fdc1ed600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:36.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fe0fa9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fdc0cf470 0x7f9fdc0ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:36.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fe0fa9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0cd600 con 0x7f9fdc099a70 2024-04-25T08:39:36.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fe0fa9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0cd320 con 0x7f9fdc06bd80 2024-04-25T08:39:36.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fe0fa9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0cc9c0 con 0x7f9fdc0cf470 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fd3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fdc099a70 0x7f9fdc1ed600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fd3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fdc099a70 0x7f9fdc1ed600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42590/0 (socket says 172.21.15.38:42590) 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fd3fff640 1 -- 172.21.15.38:0/2057506090 learned_addr learned my addr 172.21.15.38:0/2057506090 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fdad9d640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fdc0cf470 0x7f9fdc0ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fda59c640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fdc06bd80 0x7f9fdc1f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fd3fff640 1 -- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fdc0cf470 msgr2=0x7f9fdc0ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fd3fff640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fdc0cf470 0x7f9fdc0ca220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fd3fff640 1 -- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fdc06bd80 msgr2=0x7f9fdc1f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fd3fff640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fdc06bd80 0x7f9fdc1f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.707+0000 7f9fd3fff640 1 -- 172.21.15.38:0/2057506090 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0b56a0 con 0x7f9fdc099a70 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.708+0000 7f9fd3fff640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fdc099a70 0x7f9fdc1ed600 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400b410 tx=0x7f9fc400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.708+0000 7f9fd2ffd640 1 -- 172.21.15.38:0/2057506090 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4015020 con 0x7f9fdc099a70 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.708+0000 7f9fd2ffd640 1 -- 172.21.15.38:0/2057506090 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc4011850 con 0x7f9fdc099a70 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.708+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2057506090 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc0c8a70 con 0x7f9fdc099a70 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.708+0000 7f9fd2ffd640 1 -- 172.21.15.38:0/2057506090 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4011c40 con 0x7f9fdc099a70 2024-04-25T08:39:36.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.708+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2057506090 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc0c9030 con 0x7f9fdc099a70 2024-04-25T08:39:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.709+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2057506090 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc068e80 con 0x7f9fdc099a70 2024-04-25T08:39:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.709+0000 7f9fd2ffd640 1 -- 172.21.15.38:0/2057506090 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9fc4018020 con 0x7f9fdc099a70 2024-04-25T08:39:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.710+0000 7f9fd2ffd640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9fbc06b4d0 0x7f9fbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.710+0000 7f9fd2ffd640 1 -- 172.21.15.38:0/2057506090 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f9fc408a380 con 0x7f9fdc099a70 2024-04-25T08:39:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.710+0000 7f9fda59c640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9fbc06b4d0 0x7f9fbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:36.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.711+0000 7f9fda59c640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9fbc06b4d0 0x7f9fbc06d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc034000 tx=0x7f9fcc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:36.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.714+0000 7f9fd2ffd640 1 -- 172.21.15.38:0/2057506090 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9fc401f020 con 0x7f9fdc099a70 2024-04-25T08:39:36.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.934+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2057506090 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fdc0b0ff0 con 0x7f9fdc099a70 2024-04-25T08:39:36.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.935+0000 7f9fd2ffd640 1 -- 172.21.15.38:0/2057506090 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f9fc40172c0 con 0x7f9fdc099a70 2024-04-25T08:39:36.928 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:36.929 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:07.551641+0000","modified":"2024-04-25T08:39:09.555958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8292":{"gid":8292,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1263010516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1263010516},{"type":"v1","addr":"172.21.15.107:6837","nonce":1263010516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:39:36.929 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 63 2024-04-25T08:39:36.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.937+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9fbc06b4d0 msgr2=0x7f9fbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.937+0000 7f9fe0fa9640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9fbc06b4d0 0x7f9fbc06d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc034000 tx=0x7f9fcc02e000 comp rx=0 tx=0).stop 2024-04-25T08:39:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.937+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2057506090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fdc099a70 msgr2=0x7f9fdc1ed600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.937+0000 7f9fe0fa9640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fdc099a70 0x7f9fdc1ed600 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400b410 tx=0x7f9fc400b8d0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.937+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2057506090 shutdown_connections 2024-04-25T08:39:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.937+0000 7f9fe0fa9640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9fbc06b4d0 0x7f9fbc06d990 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.937+0000 7f9fe0fa9640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9fdc0cf470 0x7f9fdc0ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.937+0000 7f9fe0fa9640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9fdc099a70 0x7f9fdc1ed600 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.937+0000 7f9fe0fa9640 1 --2- 172.21.15.38:0/2057506090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9fdc06bd80 0x7f9fdc1f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.937+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2057506090 >> 172.21.15.38:0/2057506090 conn(0x7f9fdc0b3180 msgr2=0x7f9fdc0b1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.937+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2057506090 shutdown_connections 2024-04-25T08:39:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:36.937+0000 7f9fe0fa9640 1 -- 172.21.15.38:0/2057506090 wait complete. 2024-04-25T08:39:36.941 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:39:36.941 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:39:37.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.048+0000 7f4777f02640 1 Processor -- start 2024-04-25T08:39:37.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.048+0000 7f4777f02640 1 -- start start 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f4777f02640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f477009acb0 0x7f47700cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f4777f02640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477006cdf0 0x7f47700cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f4777f02640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4770161540 con 0x7f47700cd8d0 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f4777f02640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477009a820 con 0x7f477009acb0 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f4777f02640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477006be90 con 0x7f477006cdf0 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f477549d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f477009acb0 0x7f47700cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f477549d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f477009acb0 0x7f47700cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42606/0 (socket says 172.21.15.38:42606) 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f477549d640 1 -- 172.21.15.38:0/2929559914 learned_addr learned my addr 172.21.15.38:0/2929559914 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f477649f640 1 --2- 172.21.15.38:0/2929559914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477006cdf0 0x7f47700cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f477549d640 1 -- 172.21.15.38:0/2929559914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477006cdf0 msgr2=0x7f47700cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f477549d640 1 --2- 172.21.15.38:0/2929559914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477006cdf0 0x7f47700cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f477549d640 1 -- 172.21.15.38:0/2929559914 >> v1:172.21.15.107:6789/0 conn(0x7f47700cd8d0 legacy=0x7f477006c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.049+0000 7f477549d640 1 -- 172.21.15.38:0/2929559914 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477009a520 con 0x7f477009acb0 2024-04-25T08:39:37.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.050+0000 7f477549d640 1 --2- 172.21.15.38:0/2929559914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f477009acb0 0x7f47700cbcc0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f4760009870 tx=0x7f476002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf19691cbe250a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.050+0000 7f4774c9c640 1 -- 172.21.15.38:0/2929559914 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760037070 con 0x7f477009acb0 2024-04-25T08:39:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.050+0000 7f4774c9c640 1 -- 172.21.15.38:0/2929559914 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4760032440 con 0x7f477009acb0 2024-04-25T08:39:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.050+0000 7f4774c9c640 1 -- 172.21.15.38:0/2929559914 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760032810 con 0x7f477009acb0 2024-04-25T08:39:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.051+0000 7f4777f02640 1 -- 172.21.15.38:0/2929559914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f477009acb0 msgr2=0x7f47700cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.051+0000 7f4777f02640 1 --2- 172.21.15.38:0/2929559914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f477009acb0 0x7f47700cbcc0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f4760009870 tx=0x7f476002a270 comp rx=0 tx=0).stop 2024-04-25T08:39:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.051+0000 7f4777f02640 1 -- 172.21.15.38:0/2929559914 shutdown_connections 2024-04-25T08:39:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.051+0000 7f4777f02640 1 --2- 172.21.15.38:0/2929559914 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477006cdf0 0x7f47700cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.051+0000 7f4777f02640 1 --2- 172.21.15.38:0/2929559914 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f477009acb0 0x7f47700cbcc0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.051+0000 7f4777f02640 1 -- 172.21.15.38:0/2929559914 >> 172.21.15.38:0/2929559914 conn(0x7f47700b3160 msgr2=0x7f47700b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.051+0000 7f4777f02640 1 -- 172.21.15.38:0/2929559914 shutdown_connections 2024-04-25T08:39:37.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.051+0000 7f4777f02640 1 -- 172.21.15.38:0/2929559914 wait complete. 2024-04-25T08:39:37.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.052+0000 7f4777f02640 1 Processor -- start 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.052+0000 7f4777f02640 1 -- start start 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f4777f02640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477006cdf0 0x7f477014eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f4777f02640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477009acb0 0x7f477014efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f4777f02640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47700cd8d0 0x7f477014ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f4777f02640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477006c410 con 0x7f47700cd8d0 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f4777f02640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47700d0ad0 con 0x7f477009acb0 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f4777f02640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47700b2170 con 0x7f477006cdf0 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f477649f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47700cd8d0 0x7f477014ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f477649f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47700cd8d0 0x7f477014ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42612/0 (socket says 172.21.15.38:42612) 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f477649f640 1 -- 172.21.15.38:0/802179635 learned_addr learned my addr 172.21.15.38:0/802179635 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f477549d640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477009acb0 0x7f477014efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f4775c9e640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477006cdf0 0x7f477014eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f477649f640 1 -- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477006cdf0 msgr2=0x7f477014eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f477649f640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477006cdf0 0x7f477014eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f477649f640 1 -- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477009acb0 msgr2=0x7f477014efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f477649f640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477009acb0 0x7f477014efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f477649f640 1 -- 172.21.15.38:0/802179635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477015be50 con 0x7f47700cd8d0 2024-04-25T08:39:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.053+0000 7f477649f640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47700cd8d0 0x7f477014ca70 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f4768007e10 tx=0x7f476800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.054+0000 7f47667fc640 1 -- 172.21.15.38:0/802179635 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768014620 con 0x7f47700cd8d0 2024-04-25T08:39:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.054+0000 7f47667fc640 1 -- 172.21.15.38:0/802179635 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4768019070 con 0x7f47700cd8d0 2024-04-25T08:39:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.054+0000 7f4777f02640 1 -- 172.21.15.38:0/802179635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47700b47e0 con 0x7f47700cd8d0 2024-04-25T08:39:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.054+0000 7f47667fc640 1 -- 172.21.15.38:0/802179635 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768014a20 con 0x7f47700cd8d0 2024-04-25T08:39:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.054+0000 7f4777f02640 1 -- 172.21.15.38:0/802179635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47700b4da0 con 0x7f47700cd8d0 2024-04-25T08:39:37.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.054+0000 7f4777f02640 1 -- 172.21.15.38:0/802179635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4770066de0 con 0x7f47700cd8d0 2024-04-25T08:39:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.055+0000 7f47667fc640 1 -- 172.21.15.38:0/802179635 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4768014bc0 con 0x7f47700cd8d0 2024-04-25T08:39:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.055+0000 7f47667fc640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f475406b4d0 0x7f475406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.056+0000 7f47667fc640 1 -- 172.21.15.38:0/802179635 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f476808aaf0 con 0x7f47700cd8d0 2024-04-25T08:39:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.056+0000 7f4775c9e640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f475406b4d0 0x7f475406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.056+0000 7f4775c9e640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f475406b4d0 0x7f475406d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f476c0016e0 tx=0x7f476c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.060+0000 7f47667fc640 1 -- 172.21.15.38:0/802179635 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4770066de0 con 0x7f47700cd8d0 2024-04-25T08:39:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.311+0000 7f4777f02640 1 -- 172.21.15.38:0/802179635 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4770068e80 con 0x7f47700cd8d0 2024-04-25T08:39:37.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.312+0000 7f47667fc640 1 -- 172.21.15.38:0/802179635 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f4768059160 con 0x7f47700cd8d0 2024-04-25T08:39:37.305 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:37.305 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":506,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076564125,"bytes_used":2656870400,"bytes_avail":770437242880,"bytes_total":773094113280,"write_bytes_sec":64056275,"read_op_per_sec":0,"write_op_per_sec":63},"fsmap":{"epoch":63,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8292}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T08:39:35.317071+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T08:39:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.314+0000 7f4777f02640 1 -- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f475406b4d0 msgr2=0x7f475406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.314+0000 7f4777f02640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f475406b4d0 0x7f475406d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f476c0016e0 tx=0x7f476c009040 comp rx=0 tx=0).stop 2024-04-25T08:39:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.314+0000 7f4777f02640 1 -- 172.21.15.38:0/802179635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47700cd8d0 msgr2=0x7f477014ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.314+0000 7f4777f02640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47700cd8d0 0x7f477014ca70 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f4768007e10 tx=0x7f476800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.314+0000 7f4777f02640 1 -- 172.21.15.38:0/802179635 shutdown_connections 2024-04-25T08:39:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.314+0000 7f4777f02640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47700cd8d0 0x7f477014ca70 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.314+0000 7f4777f02640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f477009acb0 0x7f477014efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.314+0000 7f4777f02640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f475406b4d0 0x7f475406d990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.314+0000 7f4777f02640 1 --2- 172.21.15.38:0/802179635 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f477006cdf0 0x7f477014eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.314+0000 7f4777f02640 1 -- 172.21.15.38:0/802179635 >> 172.21.15.38:0/802179635 conn(0x7f47700b3160 msgr2=0x7f4770099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.314+0000 7f4777f02640 1 -- 172.21.15.38:0/802179635 shutdown_connections 2024-04-25T08:39:37.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.314+0000 7f4777f02640 1 -- 172.21.15.38:0/802179635 wait complete. 2024-04-25T08:39:37.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T08:39:37.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:37.318 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-25T08:39:37.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.428+0000 7fcc9ae7d640 1 Processor -- start 2024-04-25T08:39:37.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.428+0000 7fcc9ae7d640 1 -- start start 2024-04-25T08:39:37.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.428+0000 7fcc9ae7d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcc940d1d20 0x7fcc940cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.428+0000 7fcc9ae7d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9415f5f0 0x7fcc940b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.428+0000 7fcc9ae7d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc9415fd00 con 0x7fcc9415b990 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.428+0000 7fcc9ae7d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc94160580 con 0x7fcc940d1d20 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.428+0000 7fcc9ae7d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9415faa0 con 0x7fcc9415f5f0 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc98c19640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcc9415b990 0x7fcc940d1610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:52490/0 (socket says 172.21.15.38:52490) 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc98c19640 1 -- 172.21.15.38:0/1691322045 learned_addr learned my addr 172.21.15.38:0/1691322045 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459446090 0 0) 0x7fcc9415fd00 con 0x7fcc9415b990 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc9941a640 1 --2- 172.21.15.38:0/1691322045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9415f5f0 0x7fcc940b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc80003520 con 0x7fcc9415b990 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc93fff640 1 --2- 172.21.15.38:0/1691322045 >> v2:172.21.15.107:3300/0 conn(0x7fcc940d1d20 0x7fcc940cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2204486395 0 0) 0x7fcc80003520 con 0x7fcc9415b990 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9415f5f0 msgr2=0x7fcc940b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 --2- 172.21.15.38:0/1691322045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9415f5f0 0x7fcc940b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 >> v2:172.21.15.107:3300/0 conn(0x7fcc940d1d20 msgr2=0x7fcc940cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 --2- 172.21.15.38:0/1691322045 >> v2:172.21.15.107:3300/0 conn(0x7fcc940d1d20 0x7fcc940cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9415ffc0 con 0x7fcc9415b990 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fcc8c0019f0 con 0x7fcc9415b990 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 >> v1:172.21.15.38:6789/0 conn(0x7fcc9415b990 legacy=0x7fcc940d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 --2- 172.21.15.38:0/1691322045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc800039b0 0x7fcc80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 --2- 172.21.15.38:0/1691322045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9415f5f0 0x7fcc80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 --2- 172.21.15.38:0/1691322045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc80004d40 0x7fcc800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94160580 con 0x7fcc800039b0 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9415faa0 con 0x7fcc80004d40 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8000b9c0 con 0x7fcc9415f5f0 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc98c19640 1 --2- 172.21.15.38:0/1691322045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc800039b0 0x7fcc80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc93fff640 1 --2- 172.21.15.38:0/1691322045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9415f5f0 0x7fcc80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.429+0000 7fcc9941a640 1 --2- 172.21.15.38:0/1691322045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc80004d40 0x7fcc800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.430+0000 7fcc98c19640 1 -- 172.21.15.38:0/1691322045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9415f5f0 msgr2=0x7fcc80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.430+0000 7fcc98c19640 1 --2- 172.21.15.38:0/1691322045 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9415f5f0 0x7fcc80004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.430+0000 7fcc98c19640 1 -- 172.21.15.38:0/1691322045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc80004d40 msgr2=0x7fcc800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.430+0000 7fcc98c19640 1 --2- 172.21.15.38:0/1691322045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc80004d40 0x7fcc800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.430+0000 7fcc98c19640 1 -- 172.21.15.38:0/1691322045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc80003520 con 0x7fcc800039b0 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.430+0000 7fcc98c19640 1 --2- 172.21.15.38:0/1691322045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc800039b0 0x7fcc80003dd0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0013d0 tx=0x7fcc80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b49c193016287e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.430+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c00a4e0 con 0x7fcc800039b0 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.430+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8c02f070 con 0x7fcc800039b0 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.430+0000 7fcc937fe640 1 -- 172.21.15.38:0/1691322045 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c02bce0 con 0x7fcc800039b0 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.431+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/1691322045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc800039b0 msgr2=0x7fcc80003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.431+0000 7fcc9ae7d640 1 --2- 172.21.15.38:0/1691322045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc800039b0 0x7fcc80003dd0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c0013d0 tx=0x7fcc80005830 comp rx=0 tx=0).stop 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.431+0000 7fcc9941a640 1 -- 172.21.15.38:0/1691322045 reap_dead start 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.431+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/1691322045 shutdown_connections 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.431+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/1691322045 >> 172.21.15.38:0/1691322045 conn(0x7fcc9408b000 msgr2=0x7fcc9408b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.431+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/1691322045 shutdown_connections 2024-04-25T08:39:37.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.431+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/1691322045 wait complete. 2024-04-25T08:39:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.432+0000 7fcc9ae7d640 1 Processor -- start 2024-04-25T08:39:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.432+0000 7fcc9ae7d640 1 -- start start 2024-04-25T08:39:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.433+0000 7fcc9ae7d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc800039b0 0x7fcc941f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.433+0000 7fcc9ae7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc940c8750 0x7fcc940c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.433+0000 7fcc9ae7d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9406bcf0 0x7fcc9406c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.433+0000 7fcc9ae7d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9415bd70 con 0x7fcc940c8750 2024-04-25T08:39:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.433+0000 7fcc9ae7d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc940b1ac0 con 0x7fcc800039b0 2024-04-25T08:39:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.433+0000 7fcc9ae7d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9415bef0 con 0x7fcc9406bcf0 2024-04-25T08:39:37.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.433+0000 7fcc93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc940c8750 0x7fcc940c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.433+0000 7fcc93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc940c8750 0x7fcc940c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42640/0 (socket says 172.21.15.38:42640) 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.433+0000 7fcc93fff640 1 -- 172.21.15.38:0/508044910 learned_addr learned my addr 172.21.15.38:0/508044910 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc98c19640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc800039b0 0x7fcc941f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc9941a640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9406bcf0 0x7fcc9406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc93fff640 1 -- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9406bcf0 msgr2=0x7fcc9406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc93fff640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9406bcf0 0x7fcc9406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc93fff640 1 -- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc800039b0 msgr2=0x7fcc941f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc93fff640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc800039b0 0x7fcc941f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc93fff640 1 -- 172.21.15.38:0/508044910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9406ca20 con 0x7fcc940c8750 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc9941a640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9406bcf0 0x7fcc9406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc98c19640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc800039b0 0x7fcc941f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc93fff640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc940c8750 0x7fcc940c8bd0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fcc84011a70 tx=0x7fcc84011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc917fa640 1 -- 172.21.15.38:0/508044910 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8400fb50 con 0x7fcc940c8750 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc917fa640 1 -- 172.21.15.38:0/508044910 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8400fcf0 con 0x7fcc940c8750 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc917fa640 1 -- 172.21.15.38:0/508044910 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8400fea0 con 0x7fcc940c8750 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/508044910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9406cd00 con 0x7fcc940c8750 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.434+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/508044910 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9406d2c0 con 0x7fcc940c8750 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.436+0000 7fcc917fa640 1 -- 172.21.15.38:0/508044910 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc84012040 con 0x7fcc940c8750 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.436+0000 7fcc917fa640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcc7406b500 0x7fcc7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.436+0000 7fcc917fa640 1 -- 172.21.15.38:0/508044910 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcc8408aac0 con 0x7fcc940c8750 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.436+0000 7fcc98c19640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcc7406b500 0x7fcc7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.437+0000 7fcc98c19640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcc7406b500 0x7fcc7406d9c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c033000 tx=0x7fcc8c038040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.437+0000 7fcc9db83740 1 Processor -- start 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.437+0000 7fcc9db83740 1 -- start start 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc9db83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 0x55828f2a8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc9db83740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 0x55828f2a8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc9db83740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f278770 0x55828f27bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc9db83740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828f1e1da0 con 0x55828f2a5ef0 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc9db83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828f18ab80 con 0x55828f2a5b10 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc9db83740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55828f1dde40 con 0x55828f278770 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 0x55828f2a8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 0x55828f2a8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42644/0 (socket says 172.21.15.38:42644) 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc93fff640 1 -- 172.21.15.38:0/3013523850 learned_addr learned my addr 172.21.15.38:0/3013523850 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc98c19640 1 --2- 172.21.15.38:0/3013523850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 0x55828f2a8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc9941a640 1 --2- 172.21.15.38:0/3013523850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f278770 0x55828f27bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc93fff640 1 -- 172.21.15.38:0/3013523850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f278770 msgr2=0x55828f27bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc93fff640 1 --2- 172.21.15.38:0/3013523850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f278770 0x55828f27bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc93fff640 1 -- 172.21.15.38:0/3013523850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 msgr2=0x55828f2a8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc93fff640 1 --2- 172.21.15.38:0/3013523850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 0x55828f2a8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc93fff640 1 -- 172.21.15.38:0/3013523850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55828f194f10 con 0x55828f2a5ef0 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc93fff640 1 --2- 172.21.15.38:0/3013523850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 0x55828f2a8860 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fcc84006dc0 tx=0x7fcc84059760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0337f81296b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc8a7fc640 1 -- 172.21.15.38:0/3013523850 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84070030 con 0x55828f2a5ef0 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc8a7fc640 1 -- 172.21.15.38:0/3013523850 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc840616f0 con 0x55828f2a5ef0 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.438+0000 7fcc8a7fc640 1 -- 172.21.15.38:0/3013523850 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84055650 con 0x55828f2a5ef0 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.439+0000 7fcc9db83740 1 -- 172.21.15.38:0/3013523850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 msgr2=0x55828f2a8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.439+0000 7fcc9db83740 1 --2- 172.21.15.38:0/3013523850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 0x55828f2a8860 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fcc84006dc0 tx=0x7fcc84059760 comp rx=0 tx=0).stop 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.439+0000 7fcc9db83740 1 -- 172.21.15.38:0/3013523850 shutdown_connections 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.439+0000 7fcc9db83740 1 --2- 172.21.15.38:0/3013523850 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f278770 0x55828f27bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.439+0000 7fcc9db83740 1 --2- 172.21.15.38:0/3013523850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 0x55828f2a8860 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.439+0000 7fcc9db83740 1 --2- 172.21.15.38:0/3013523850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 0x55828f2a8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.439+0000 7fcc9db83740 1 -- 172.21.15.38:0/3013523850 >> 172.21.15.38:0/3013523850 conn(0x55828f27a9e0 msgr2=0x55828f2c3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.439+0000 7fcc9db83740 1 -- 172.21.15.38:0/3013523850 shutdown_connections 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.439+0000 7fcc9db83740 1 -- 172.21.15.38:0/3013523850 wait complete. 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.440+0000 7fcc9db83740 1 -- start start 2024-04-25T08:39:37.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.440+0000 7fcc9db83740 1 Processor -- start 2024-04-25T08:39:37.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9db83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 0x55828f29b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9db83740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2a5ef0 0x55828f2c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9db83740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2c6920 0x55828f2c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9db83740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828f27db90 con 0x55828f2c6920 2024-04-25T08:39:37.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9db83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828f1804b0 con 0x55828f2a5b10 2024-04-25T08:39:37.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9db83740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55828f27de70 con 0x55828f2a5ef0 2024-04-25T08:39:37.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9db83740 10 client.0 _ll_register_callbacks cb 0x55828f1c0180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:37.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9941a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2c6920 0x55828f2c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9941a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2c6920 0x55828f2c6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42648/0 (socket says 172.21.15.38:42648) 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9941a640 1 -- 172.21.15.38:0/2909520414 learned_addr learned my addr 172.21.15.38:0/2909520414 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc98c19640 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 0x55828f29b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc93fff640 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2a5ef0 0x55828f2c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9941a640 1 -- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2a5ef0 msgr2=0x55828f2c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9941a640 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2a5ef0 0x55828f2c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9941a640 1 -- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 msgr2=0x55828f29b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9941a640 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 0x55828f29b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9941a640 1 -- 172.21.15.38:0/2909520414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55828efc07d0 con 0x55828f2c6920 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc93fff640 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2a5ef0 0x55828f2c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc98c19640 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 0x55828f29b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9941a640 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2c6920 0x55828f2c6da0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009eb0 tx=0x7fcc7c0167a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0b1c4754b19245 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc737fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc737fe640 1 -- 172.21.15.38:0/2909520414 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c004240 con 0x55828f2c6920 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc737fe640 1 -- 172.21.15.38:0/2909520414 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcc7c0043e0 con 0x55828f2c6920 2024-04-25T08:39:37.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.441+0000 7fcc9db83740 1 -- 172.21.15.38:0/2909520414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc840616f0 con 0x55828f2c6920 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc737fe640 1 -- 172.21.15.38:0/2909520414 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc7c026050 con 0x55828f2c6920 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc9db83740 10 client.8583 fetch_fsmap learned FSMap version 63 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc9db83740 1 -- 172.21.15.38:0/2909520414 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55828eef63f0 con 0x55828f2c6920 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc737fe640 1 -- 172.21.15.38:0/2909520414 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc840616f0 con 0x55828f2c6920 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc737fe640 1 -- 172.21.15.38:0/2909520414 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcc7c029020 con 0x55828f2c6920 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc9db83740 10 client.8583 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc9db83740 10 client.8583 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc9db83740 20 client.8583 populate_metadata read hostname 'smithi038' 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc9db83740 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55828f29f2c0 0x55828f29f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc9db83740 4 client.8583 mds_command: new command op to 8292 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc9db83740 1 -- 172.21.15.38:0/2909520414 --> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55828f1ef2e0 con 0x55828f29f2c0 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc98c19640 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55828f29f2c0 0x55828f29f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc737fe640 10 client.8583 ms_handle_connect on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.442+0000 7fcc98c19640 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55828f29f2c0 0x55828f29f6d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e7ffe739035acf1 server_cookie=300a7faf42606bcf in_seq=0 out_seq=0 2024-04-25T08:39:37.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc737fe640 1 -- 172.21.15.38:0/2909520414 <== mds.0 v2:172.21.15.107:6835/1263010516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55828f1ef2e0 con 0x55828f29f2c0 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc737fe640 10 client.8583 handle_command_reply: tid=0 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc9db83740 1 client.8583 shutdown 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc9db83740 1 -- 172.21.15.38:0/2909520414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2c6920 msgr2=0x55828f2c6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc9db83740 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2c6920 0x55828f2c6da0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009eb0 tx=0x7fcc7c0167a0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc9db83740 1 -- 172.21.15.38:0/2909520414 shutdown_connections 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc9db83740 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55828f29f2c0 0x55828f29f6d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc9db83740 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2c6920 0x55828f2c6da0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc9db83740 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2a5ef0 0x55828f2c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc9db83740 1 --2- 172.21.15.38:0/2909520414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 0x55828f29b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc9db83740 1 -- 172.21.15.38:0/2909520414 >> 172.21.15.38:0/2909520414 conn(0x55828f278770 msgr2=0x55828f2a4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc737fe640 0 client.8583 ms_handle_reset on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc9db83740 1 -- 172.21.15.38:0/2909520414 shutdown_connections 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.443+0000 7fcc9db83740 1 -- 172.21.15.38:0/2909520414 wait complete. 2024-04-25T08:39:37.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.444+0000 7fcc9db83740 20 client.8583 trim_cache size 0 max 16384 2024-04-25T08:39:37.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.458+0000 7fcc9db83740 1 Processor -- start 2024-04-25T08:39:37.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.458+0000 7fcc9db83740 1 -- start start 2024-04-25T08:39:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc9db83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 0x55828f27f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc9db83740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 0x55828f2e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc9db83740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2c6920 0x55828f2a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc9db83740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94200c10 con 0x55828f2a5ef0 2024-04-25T08:39:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc9db83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc941ffe50 con 0x55828f2a5b10 2024-04-25T08:39:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc9db83740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc941f5400 con 0x55828f2c6920 2024-04-25T08:39:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 0x55828f2e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc93fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 0x55828f2e5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42654/0 (socket says 172.21.15.38:42654) 2024-04-25T08:39:37.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc93fff640 1 -- 172.21.15.38:0/3262922375 learned_addr learned my addr 172.21.15.38:0/3262922375 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc93fff640 1 -- 172.21.15.38:0/3262922375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2c6920 msgr2=0x55828f2a82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc93fff640 1 --2- 172.21.15.38:0/3262922375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2c6920 0x55828f2a82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc93fff640 1 -- 172.21.15.38:0/3262922375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 msgr2=0x55828f27f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc93fff640 1 --2- 172.21.15.38:0/3262922375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 0x55828f27f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.459+0000 7fcc93fff640 1 -- 172.21.15.38:0/3262922375 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55828eeb60f0 con 0x55828f2a5ef0 2024-04-25T08:39:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc93fff640 1 --2- 172.21.15.38:0/3262922375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 0x55828f2e5ce0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fcc8401c000 tx=0x7fcc8405c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bcb337e043cf04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc89ffb640 1 -- 172.21.15.38:0/3262922375 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84070040 con 0x55828f2a5ef0 2024-04-25T08:39:37.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc89ffb640 1 -- 172.21.15.38:0/3262922375 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc840572e0 con 0x55828f2a5ef0 2024-04-25T08:39:37.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc89ffb640 1 -- 172.21.15.38:0/3262922375 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84061260 con 0x55828f2a5ef0 2024-04-25T08:39:37.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc9db83740 1 -- 172.21.15.38:0/3262922375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 msgr2=0x55828f2e5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc9db83740 1 --2- 172.21.15.38:0/3262922375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 0x55828f2e5ce0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fcc8401c000 tx=0x7fcc8405c000 comp rx=0 tx=0).stop 2024-04-25T08:39:37.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc9db83740 1 -- 172.21.15.38:0/3262922375 shutdown_connections 2024-04-25T08:39:37.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc9db83740 1 --2- 172.21.15.38:0/3262922375 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2c6920 0x55828f2a82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc9db83740 1 --2- 172.21.15.38:0/3262922375 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5ef0 0x55828f2e5ce0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc9db83740 1 --2- 172.21.15.38:0/3262922375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5b10 0x55828f27f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc9db83740 1 -- 172.21.15.38:0/3262922375 >> 172.21.15.38:0/3262922375 conn(0x55828f278770 msgr2=0x55828f2c3320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc9db83740 1 -- 172.21.15.38:0/3262922375 shutdown_connections 2024-04-25T08:39:37.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.460+0000 7fcc9db83740 1 -- 172.21.15.38:0/3262922375 wait complete. 2024-04-25T08:39:37.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.461+0000 7fcc9db83740 1 -- start start 2024-04-25T08:39:37.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.461+0000 7fcc9db83740 1 Processor -- start 2024-04-25T08:39:37.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc9db83740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5b10 0x55828f277a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc9db83740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5ef0 0x55828f32ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc9db83740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2c6920 0x55828f32d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc9db83740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55828f2bfc20 con 0x55828f2a5b10 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc9db83740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc941f5a00 con 0x55828f2a5ef0 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc9db83740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55828f29cd90 con 0x55828f2c6920 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc9db83740 10 client.0 _ll_register_callbacks cb 0x7fcc84009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc98c19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5b10 0x55828f277a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc98c19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5b10 0x55828f277a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42656/0 (socket says 172.21.15.38:42656) 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc98c19640 1 -- 172.21.15.38:0/2512600489 learned_addr learned my addr 172.21.15.38:0/2512600489 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc93fff640 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5ef0 0x55828f32ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc9941a640 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2c6920 0x55828f32d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc98c19640 1 -- 172.21.15.38:0/2512600489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2c6920 msgr2=0x55828f32d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc98c19640 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2c6920 0x55828f32d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc98c19640 1 -- 172.21.15.38:0/2512600489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5ef0 msgr2=0x55828f32ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc98c19640 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5ef0 0x55828f32ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc98c19640 1 -- 172.21.15.38:0/2512600489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55828f27f8e0 con 0x55828f2a5b10 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc98c19640 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5b10 0x55828f277a00 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c02b530 tx=0x7fcc8c001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31bc5e05c4b873f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.462+0000 7fcc737fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc737fe640 1 -- 172.21.15.38:0/2512600489 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8c034dd0 con 0x55828f2a5b10 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc737fe640 1 -- 172.21.15.38:0/2512600489 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcc8c04f190 con 0x55828f2a5b10 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc9db83740 1 -- 172.21.15.38:0/2512600489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc840572e0 con 0x55828f2a5b10 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc737fe640 1 -- 172.21.15.38:0/2512600489 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc8c05b050 con 0x55828f2a5b10 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc9db83740 10 client.8589 fetch_fsmap learned FSMap version 63 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc9db83740 1 -- 172.21.15.38:0/2512600489 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55828f277f40 con 0x55828f2a5b10 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc737fe640 1 -- 172.21.15.38:0/2512600489 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc840572e0 con 0x55828f2a5b10 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc737fe640 1 -- 172.21.15.38:0/2512600489 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcc8c05e020 con 0x55828f2a5b10 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc9db83740 10 client.8589 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T08:39:37.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc9db83740 10 client.8589 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc9db83740 20 client.8589 populate_metadata read hostname 'smithi038' 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc9db83740 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55828f2b8cb0 0x55828f2b9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc9db83740 4 client.8589 mds_command: new command op to 8292 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc9db83740 1 -- 172.21.15.38:0/2512600489 --> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55828f1e9630 con 0x55828f2b8cb0 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.463+0000 7fcc93fff640 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55828f2b8cb0 0x55828f2b9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc737fe640 10 client.8589 ms_handle_connect on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc93fff640 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55828f2b8cb0 0x55828f2b9090 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a41ab4b71a4ebf9 server_cookie=e229223c6c6c5376 in_seq=0 out_seq=0 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc737fe640 1 -- 172.21.15.38:0/2512600489 <== mds.0 v2:172.21.15.107:6835/1263010516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55828f1e9630 con 0x55828f2b8cb0 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc737fe640 10 client.8589 handle_command_reply: tid=0 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc9db83740 1 client.8589 shutdown 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc9db83740 1 -- 172.21.15.38:0/2512600489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5b10 msgr2=0x55828f277a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc9db83740 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5b10 0x55828f277a00 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c02b530 tx=0x7fcc8c001360 comp rx=0 tx=0).stop 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc9db83740 1 -- 172.21.15.38:0/2512600489 shutdown_connections 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc9db83740 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x55828f2b8cb0 0x55828f2b9090 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc9db83740 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55828f2c6920 0x55828f32d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc9db83740 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55828f2a5ef0 0x55828f32ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc737fe640 0 client.8589 ms_handle_reset on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:37.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc9db83740 1 --2- 172.21.15.38:0/2512600489 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55828f2a5b10 0x55828f277a00 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc9db83740 1 -- 172.21.15.38:0/2512600489 >> 172.21.15.38:0/2512600489 conn(0x55828f278770 msgr2=0x55828f2e5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc9db83740 1 -- 172.21.15.38:0/2512600489 shutdown_connections 2024-04-25T08:39:37.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc9db83740 1 -- 172.21.15.38:0/2512600489 wait complete. 2024-04-25T08:39:37.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.464+0000 7fcc9db83740 20 client.8589 trim_cache size 0 max 16384 2024-04-25T08:39:37.464 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:39:37.464 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:39:37.465 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:39:37.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.465+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcc7406b500 msgr2=0x7fcc7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.465+0000 7fcc9ae7d640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcc7406b500 0x7fcc7406d9c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c033000 tx=0x7fcc8c038040 comp rx=0 tx=0).stop 2024-04-25T08:39:37.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.465+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/508044910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc940c8750 msgr2=0x7fcc940c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.465+0000 7fcc9ae7d640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc940c8750 0x7fcc940c8bd0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fcc84011a70 tx=0x7fcc84011f30 comp rx=0 tx=0).stop 2024-04-25T08:39:37.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.465+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/508044910 shutdown_connections 2024-04-25T08:39:37.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.465+0000 7fcc9ae7d640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcc7406b500 0x7fcc7406d9c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.465+0000 7fcc9ae7d640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc9406bcf0 0x7fcc9406c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.465+0000 7fcc9ae7d640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc940c8750 0x7fcc940c8bd0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.465+0000 7fcc9ae7d640 1 --2- 172.21.15.38:0/508044910 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc800039b0 0x7fcc941f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.465+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/508044910 >> 172.21.15.38:0/508044910 conn(0x7fcc9408b000 msgr2=0x7fcc9415aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.465+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/508044910 shutdown_connections 2024-04-25T08:39:37.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.465+0000 7fcc9ae7d640 1 -- 172.21.15.38:0/508044910 wait complete. 2024-04-25T08:39:37.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:37.469 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T08:39:37.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.579+0000 7fd5762ac640 1 Processor -- start 2024-04-25T08:39:37.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.579+0000 7fd5762ac640 1 -- start start 2024-04-25T08:39:37.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.579+0000 7fd5762ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd57015b990 0x7fd5700cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.579+0000 7fd5762ac640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd57015f5f0 0x7fd5700d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.579+0000 7fd5762ac640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57015fd00 con 0x7fd5700d1b50 2024-04-25T08:39:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.579+0000 7fd5762ac640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd570160580 con 0x7fd57015f5f0 2024-04-25T08:39:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.579+0000 7fd5762ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57015fa30 con 0x7fd57015b990 2024-04-25T08:39:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.580+0000 7fd56f7fe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd57015f5f0 0x7fd5700d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.580+0000 7fd574849640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fd5700d1b50 0x7fd5700b1fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:40972/0 (socket says 172.21.15.38:40972) 2024-04-25T08:39:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.580+0000 7fd56ffff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd57015b990 0x7fd5700cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.580+0000 7fd574849640 1 -- 172.21.15.38:0/1587702522 learned_addr learned my addr 172.21.15.38:0/1587702522 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.580+0000 7fd56f7fe640 1 -- 172.21.15.38:0/1587702522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd57015b990 msgr2=0x7fd5700cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.580+0000 7fd56f7fe640 1 --2- 172.21.15.38:0/1587702522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd57015b990 0x7fd5700cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.580+0000 7fd56f7fe640 1 -- 172.21.15.38:0/1587702522 >> v1:172.21.15.107:6789/0 conn(0x7fd5700d1b50 legacy=0x7fd5700b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.580+0000 7fd56f7fe640 1 -- 172.21.15.38:0/1587702522 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57015ffc0 con 0x7fd57015f5f0 2024-04-25T08:39:37.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.580+0000 7fd56effd640 1 -- 172.21.15.38:0/1587702522 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600766270 0 0) 0x7fd57015fd00 con 0x7fd5700d1b50 2024-04-25T08:39:37.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.580+0000 7fd56ffff640 1 --2- 172.21.15.38:0/1587702522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd57015b990 0x7fd5700cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:37.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.581+0000 7fd56f7fe640 1 --2- 172.21.15.38:0/1587702522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57015f5f0 0x7fd5700d1610 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd5640097a0 tx=0x7fd56402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80f4988cf0c64dee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.581+0000 7fd56effd640 1 -- 172.21.15.38:0/1587702522 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564037070 con 0x7fd57015f5f0 2024-04-25T08:39:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.581+0000 7fd56effd640 1 -- 172.21.15.38:0/1587702522 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56402ae50 con 0x7fd57015f5f0 2024-04-25T08:39:37.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.581+0000 7fd56effd640 1 -- 172.21.15.38:0/1587702522 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564032660 con 0x7fd57015f5f0 2024-04-25T08:39:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.581+0000 7fd5762ac640 1 -- 172.21.15.38:0/1587702522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57015f5f0 msgr2=0x7fd5700d1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.581+0000 7fd5762ac640 1 --2- 172.21.15.38:0/1587702522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57015f5f0 0x7fd5700d1610 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fd5640097a0 tx=0x7fd56402a100 comp rx=0 tx=0).stop 2024-04-25T08:39:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.581+0000 7fd5762ac640 1 -- 172.21.15.38:0/1587702522 shutdown_connections 2024-04-25T08:39:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.582+0000 7fd5762ac640 1 --2- 172.21.15.38:0/1587702522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57015f5f0 0x7fd5700d1610 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.582+0000 7fd5762ac640 1 --2- 172.21.15.38:0/1587702522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd57015b990 0x7fd5700cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.582+0000 7fd5762ac640 1 -- 172.21.15.38:0/1587702522 >> 172.21.15.38:0/1587702522 conn(0x7fd57008b000 msgr2=0x7fd57008b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.582+0000 7fd5762ac640 1 -- 172.21.15.38:0/1587702522 shutdown_connections 2024-04-25T08:39:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.582+0000 7fd5762ac640 1 -- 172.21.15.38:0/1587702522 wait complete. 2024-04-25T08:39:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.583+0000 7fd5762ac640 1 Processor -- start 2024-04-25T08:39:37.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.583+0000 7fd5762ac640 1 -- start start 2024-04-25T08:39:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd5762ac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5700d1b50 0x7fd570159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd5762ac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57015b990 0x7fd5700b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd5762ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd57015f5f0 0x7fd57015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd5762ac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57015bd70 con 0x7fd5700d1b50 2024-04-25T08:39:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd5762ac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700b1ac0 con 0x7fd57015b990 2024-04-25T08:39:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd5762ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57015bef0 con 0x7fd57015f5f0 2024-04-25T08:39:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5700d1b50 0x7fd570159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5700d1b50 0x7fd570159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42658/0 (socket says 172.21.15.38:42658) 2024-04-25T08:39:37.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56ffff640 1 -- 172.21.15.38:0/851347511 learned_addr learned my addr 172.21.15.38:0/851347511 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56f7fe640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57015b990 0x7fd5700b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd574849640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd57015f5f0 0x7fd57015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56ffff640 1 -- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd57015f5f0 msgr2=0x7fd57015d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56ffff640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd57015f5f0 0x7fd57015d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56ffff640 1 -- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57015b990 msgr2=0x7fd5700b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56ffff640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57015b990 0x7fd5700b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56ffff640 1 -- 172.21.15.38:0/851347511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5700b43e0 con 0x7fd5700d1b50 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56ffff640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5700d1b50 0x7fd570159cf0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd56000ebe0 tx=0x7fd56000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56cff9640 1 -- 172.21.15.38:0/851347511 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560017070 con 0x7fd5700d1b50 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56cff9640 1 -- 172.21.15.38:0/851347511 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56000cc30 con 0x7fd5700d1b50 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd5762ac640 1 -- 172.21.15.38:0/851347511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5700b46c0 con 0x7fd5700d1b50 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.584+0000 7fd56cff9640 1 -- 172.21.15.38:0/851347511 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560013470 con 0x7fd5700d1b50 2024-04-25T08:39:37.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.585+0000 7fd5762ac640 1 -- 172.21.15.38:0/851347511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5700b4c80 con 0x7fd5700d1b50 2024-04-25T08:39:37.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.586+0000 7fd56cff9640 1 -- 172.21.15.38:0/851347511 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd56001a070 con 0x7fd5700d1b50 2024-04-25T08:39:37.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.587+0000 7fd56cff9640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd55c06b4d0 0x7fd55c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.587+0000 7fd56cff9640 1 -- 172.21.15.38:0/851347511 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd56008a760 con 0x7fd5700d1b50 2024-04-25T08:39:37.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.587+0000 7fd56f7fe640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd55c06b4d0 0x7fd55c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.587+0000 7fd56f7fe640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd55c06b4d0 0x7fd55c06d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd564034000 tx=0x7fd56402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.587+0000 7fd578fa9740 1 Processor -- start 2024-04-25T08:39:37.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.587+0000 7fd578fa9740 1 -- start start 2024-04-25T08:39:37.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.588+0000 7fd578fa9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6b10 0x559b184c9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.588+0000 7fd578fa9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6ef0 0x559b184c9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.588+0000 7fd578fa9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b18499770 0x559b1849cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.588+0000 7fd578fa9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b18402da0 con 0x559b184c6ef0 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.588+0000 7fd578fa9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b183abb80 con 0x559b18499770 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.588+0000 7fd578fa9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b183fee40 con 0x559b184c6b10 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.588+0000 7fd56f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6ef0 0x559b184c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.588+0000 7fd56f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6ef0 0x559b184c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42674/0 (socket says 172.21.15.38:42674) 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.588+0000 7fd56f7fe640 1 -- 172.21.15.38:0/511034081 learned_addr learned my addr 172.21.15.38:0/511034081 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.588+0000 7fd56ffff640 1 --2- 172.21.15.38:0/511034081 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6b10 0x559b184c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.588+0000 7fd574849640 1 --2- 172.21.15.38:0/511034081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b18499770 0x559b1849cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.589+0000 7fd56f7fe640 1 -- 172.21.15.38:0/511034081 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6b10 msgr2=0x559b184c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.589+0000 7fd56f7fe640 1 --2- 172.21.15.38:0/511034081 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6b10 0x559b184c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.589+0000 7fd56f7fe640 1 -- 172.21.15.38:0/511034081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b18499770 msgr2=0x559b1849cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.589+0000 7fd56f7fe640 1 --2- 172.21.15.38:0/511034081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b18499770 0x559b1849cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.589+0000 7fd56f7fe640 1 -- 172.21.15.38:0/511034081 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b183b5f10 con 0x559b184c6ef0 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.589+0000 7fd56f7fe640 1 --2- 172.21.15.38:0/511034081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6ef0 0x559b184c9860 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fd56402a140 tx=0x7fd56402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789e39bd554a6d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.589+0000 7fd555ffb640 1 -- 172.21.15.38:0/511034081 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564037040 con 0x559b184c6ef0 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.589+0000 7fd555ffb640 1 -- 172.21.15.38:0/511034081 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56404d070 con 0x559b184c6ef0 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.589+0000 7fd555ffb640 1 -- 172.21.15.38:0/511034081 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd564003a10 con 0x559b184c6ef0 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.589+0000 7fd578fa9740 1 -- 172.21.15.38:0/511034081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6ef0 msgr2=0x559b184c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.589+0000 7fd578fa9740 1 --2- 172.21.15.38:0/511034081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6ef0 0x559b184c9860 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fd56402a140 tx=0x7fd56402c000 comp rx=0 tx=0).stop 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.590+0000 7fd578fa9740 1 -- 172.21.15.38:0/511034081 shutdown_connections 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.590+0000 7fd578fa9740 1 --2- 172.21.15.38:0/511034081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b18499770 0x559b1849cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.590+0000 7fd578fa9740 1 --2- 172.21.15.38:0/511034081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6ef0 0x559b184c9860 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.590+0000 7fd578fa9740 1 --2- 172.21.15.38:0/511034081 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6b10 0x559b184c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.590+0000 7fd578fa9740 1 -- 172.21.15.38:0/511034081 >> 172.21.15.38:0/511034081 conn(0x559b1849b9e0 msgr2=0x559b184e4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.590+0000 7fd578fa9740 1 -- 172.21.15.38:0/511034081 shutdown_connections 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.590+0000 7fd578fa9740 1 -- 172.21.15.38:0/511034081 wait complete. 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.590+0000 7fd578fa9740 1 -- start start 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.590+0000 7fd578fa9740 1 Processor -- start 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.591+0000 7fd578fa9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184c6b10 0x559b184e6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.591+0000 7fd578fa9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b1849a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.591+0000 7fd578fa9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 0x559b1849a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.591+0000 7fd578fa9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b1849eb90 con 0x559b184e7100 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.591+0000 7fd578fa9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b183a14b0 con 0x559b184c6b10 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.591+0000 7fd578fa9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b1849ee70 con 0x559b184c6ef0 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.591+0000 7fd578fa9740 10 client.0 _ll_register_callbacks cb 0x559b183e1180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.591+0000 7fd574849640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 0x559b1849a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.591+0000 7fd574849640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 0x559b1849a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42676/0 (socket says 172.21.15.38:42676) 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.591+0000 7fd574849640 1 -- 172.21.15.38:0/3204694671 learned_addr learned my addr 172.21.15.38:0/3204694671 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.591+0000 7fd56f7fe640 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b1849a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.591+0000 7fd56ffff640 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184c6b10 0x559b184e6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd574849640 1 -- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 msgr2=0x559b1849a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd574849640 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b1849a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd574849640 1 -- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184c6b10 msgr2=0x559b184e6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd574849640 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184c6b10 0x559b184e6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd574849640 1 -- 172.21.15.38:0/3204694671 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b181e17d0 con 0x559b184e7100 2024-04-25T08:39:37.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd56f7fe640 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b1849a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd574849640 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 0x559b1849a960 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd5680016b0 tx=0x7fd568001b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9abc52d1190d83e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd54effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd54effd640 1 -- 172.21.15.38:0/3204694671 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568004b60 con 0x559b184e7100 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd54effd640 1 -- 172.21.15.38:0/3204694671 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd568010040 con 0x559b184e7100 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd578fa9740 1 -- 172.21.15.38:0/3204694671 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd56404d070 con 0x559b184e7100 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd54effd640 1 -- 172.21.15.38:0/3204694671 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd568021050 con 0x559b184e7100 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd578fa9740 10 client.8598 fetch_fsmap learned FSMap version 63 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd578fa9740 1 -- 172.21.15.38:0/3204694671 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x559b181173f0 con 0x559b184e7100 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd54effd640 1 -- 172.21.15.38:0/3204694671 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56404d070 con 0x559b184e7100 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd54effd640 1 -- 172.21.15.38:0/3204694671 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd568024020 con 0x559b184e7100 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd578fa9740 10 client.8598 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd578fa9740 10 client.8598 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd578fa9740 20 client.8598 populate_metadata read hostname 'smithi038' 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.592+0000 7fd578fa9740 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x559b184bec60 0x559b184bf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.593+0000 7fd578fa9740 4 client.8598 mds_command: new command op to 8292 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.593+0000 7fd578fa9740 1 -- 172.21.15.38:0/3204694671 --> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b184102e0 con 0x559b184bec60 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.593+0000 7fd56ffff640 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x559b184bec60 0x559b184bf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.593+0000 7fd54effd640 10 client.8598 ms_handle_connect on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.593+0000 7fd56ffff640 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x559b184bec60 0x559b184bf040 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c102fcc9d23b8537 server_cookie=4f97ed6b4069ba2b in_seq=0 out_seq=0 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd54effd640 1 -- 172.21.15.38:0/3204694671 <== mds.0 v2:172.21.15.107:6835/1263010516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b184102e0 con 0x559b184bec60 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd54effd640 10 client.8598 handle_command_reply: tid=0 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd578fa9740 1 client.8598 shutdown 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd578fa9740 1 -- 172.21.15.38:0/3204694671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 msgr2=0x559b1849a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd578fa9740 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 0x559b1849a960 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fd5680016b0 tx=0x7fd568001b70 comp rx=0 tx=0).stop 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd578fa9740 1 -- 172.21.15.38:0/3204694671 shutdown_connections 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd578fa9740 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x559b184bec60 0x559b184bf040 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd578fa9740 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 0x559b1849a960 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd578fa9740 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b1849a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd54effd640 0 client.8598 ms_handle_reset on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd578fa9740 1 --2- 172.21.15.38:0/3204694671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184c6b10 0x559b184e6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd578fa9740 1 -- 172.21.15.38:0/3204694671 >> 172.21.15.38:0/3204694671 conn(0x559b18499770 msgr2=0x559b184c5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd578fa9740 1 -- 172.21.15.38:0/3204694671 shutdown_connections 2024-04-25T08:39:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd578fa9740 1 -- 172.21.15.38:0/3204694671 wait complete. 2024-04-25T08:39:37.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.594+0000 7fd578fa9740 20 client.8598 trim_cache size 0 max 16384 2024-04-25T08:39:37.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.609+0000 7fd578fa9740 1 Processor -- start 2024-04-25T08:39:37.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.609+0000 7fd578fa9740 1 -- start start 2024-04-25T08:39:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd578fa9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6b10 0x559b184cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd578fa9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b184e4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd578fa9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184e7100 0x559b184e4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd578fa9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5702001e0 con 0x559b184c6b10 2024-04-25T08:39:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd578fa9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701ff3a0 con 0x559b184e7100 2024-04-25T08:39:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd578fa9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5700c9a20 con 0x559b184c6ef0 2024-04-25T08:39:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd56ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6b10 0x559b184cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd56ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6b10 0x559b184cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42682/0 (socket says 172.21.15.38:42682) 2024-04-25T08:39:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd56ffff640 1 -- 172.21.15.38:0/2899955789 learned_addr learned my addr 172.21.15.38:0/2899955789 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd56f7fe640 1 --2- 172.21.15.38:0/2899955789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b184e4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd56ffff640 1 -- 172.21.15.38:0/2899955789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 msgr2=0x559b184e4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd56ffff640 1 --2- 172.21.15.38:0/2899955789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b184e4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd56ffff640 1 -- 172.21.15.38:0/2899955789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184e7100 msgr2=0x559b184e4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd574849640 1 --2- 172.21.15.38:0/2899955789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184e7100 0x559b184e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd56ffff640 1 --2- 172.21.15.38:0/2899955789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184e7100 0x559b184e4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd56ffff640 1 -- 172.21.15.38:0/2899955789 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b180d70f0 con 0x559b184c6b10 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.610+0000 7fd56ffff640 1 --2- 172.21.15.38:0/2899955789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6b10 0x559b184cb7b0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd56000ca20 tx=0x7fd560000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6299f7a54a3cb214 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.611+0000 7fd5557fa640 1 -- 172.21.15.38:0/2899955789 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560021040 con 0x559b184c6b10 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.611+0000 7fd5557fa640 1 -- 172.21.15.38:0/2899955789 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd560061390 con 0x559b184c6b10 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.611+0000 7fd5557fa640 1 -- 172.21.15.38:0/2899955789 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd560056f80 con 0x559b184c6b10 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.611+0000 7fd578fa9740 1 -- 172.21.15.38:0/2899955789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6b10 msgr2=0x559b184cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.611+0000 7fd578fa9740 1 --2- 172.21.15.38:0/2899955789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6b10 0x559b184cb7b0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fd56000ca20 tx=0x7fd560000bc0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.611+0000 7fd578fa9740 1 -- 172.21.15.38:0/2899955789 shutdown_connections 2024-04-25T08:39:37.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.611+0000 7fd578fa9740 1 --2- 172.21.15.38:0/2899955789 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184e7100 0x559b184e4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.611+0000 7fd578fa9740 1 --2- 172.21.15.38:0/2899955789 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b184e4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.611+0000 7fd578fa9740 1 --2- 172.21.15.38:0/2899955789 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184c6b10 0x559b184cb7b0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.611+0000 7fd578fa9740 1 -- 172.21.15.38:0/2899955789 >> 172.21.15.38:0/2899955789 conn(0x559b18499770 msgr2=0x559b184bf040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.611+0000 7fd578fa9740 1 -- 172.21.15.38:0/2899955789 shutdown_connections 2024-04-25T08:39:37.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.611+0000 7fd578fa9740 1 -- 172.21.15.38:0/2899955789 wait complete. 2024-04-25T08:39:37.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.612+0000 7fd578fa9740 1 -- start start 2024-04-25T08:39:37.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.612+0000 7fd578fa9740 1 Processor -- start 2024-04-25T08:39:37.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd578fa9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184c6b10 0x559b1854e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd578fa9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b184c4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd578fa9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 0x559b184c5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd578fa9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b184c94f0 con 0x559b184e7100 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd578fa9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5701feea0 con 0x559b184c6b10 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd578fa9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b184987c0 con 0x559b184c6ef0 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd578fa9740 10 client.0 _ll_register_callbacks cb 0x7fd564035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd574849640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 0x559b184c5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd574849640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 0x559b184c5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42698/0 (socket says 172.21.15.38:42698) 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd574849640 1 -- 172.21.15.38:0/4218982805 learned_addr learned my addr 172.21.15.38:0/4218982805 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd56ffff640 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184c6b10 0x559b1854e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd56f7fe640 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b184c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd574849640 1 -- 172.21.15.38:0/4218982805 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 msgr2=0x559b184c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd574849640 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b184c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd574849640 1 -- 172.21.15.38:0/4218982805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184c6b10 msgr2=0x559b1854e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd574849640 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184c6b10 0x559b1854e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd574849640 1 -- 172.21.15.38:0/4218982805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b18494d30 con 0x559b184e7100 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd574849640 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 0x559b184c5d20 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd568013e10 tx=0x7fd568001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff90261ca338ad6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd54effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd54effd640 1 -- 172.21.15.38:0/4218982805 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568029070 con 0x559b184e7100 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd54effd640 1 -- 172.21.15.38:0/4218982805 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd56801a040 con 0x559b184e7100 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.613+0000 7fd578fa9740 1 -- 172.21.15.38:0/4218982805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd560061390 con 0x559b184e7100 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd54effd640 1 -- 172.21.15.38:0/4218982805 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd568002070 con 0x559b184e7100 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd578fa9740 10 client.8604 fetch_fsmap learned FSMap version 63 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd578fa9740 1 -- 172.21.15.38:0/4218982805 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x559b184e1360 con 0x559b184e7100 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd54effd640 1 -- 172.21.15.38:0/4218982805 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd560061390 con 0x559b184e7100 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd54effd640 1 -- 172.21.15.38:0/4218982805 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd568024070 con 0x559b184e7100 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd578fa9740 10 client.8604 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd578fa9740 10 client.8604 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd578fa9740 20 client.8604 populate_metadata read hostname 'smithi038' 2024-04-25T08:39:37.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd578fa9740 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x559b184c62b0 0x559b184fd200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd578fa9740 4 client.8604 mds_command: new command op to 8292 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd578fa9740 1 -- 172.21.15.38:0/4218982805 --> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559b184cc720 con 0x559b184c62b0 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd56ffff640 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x559b184c62b0 0x559b184fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd54effd640 10 client.8604 ms_handle_connect on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.614+0000 7fd56ffff640 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x559b184c62b0 0x559b184fd200 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a24edfe9476b598 server_cookie=e05ca620c3ce5f2b in_seq=0 out_seq=0 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd54effd640 1 -- 172.21.15.38:0/4218982805 <== mds.0 v2:172.21.15.107:6835/1263010516 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559b184cc720 con 0x559b184c62b0 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd54effd640 10 client.8604 handle_command_reply: tid=0 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd578fa9740 1 client.8604 shutdown 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd578fa9740 1 -- 172.21.15.38:0/4218982805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 msgr2=0x559b184c5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd578fa9740 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 0x559b184c5d20 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fd568013e10 tx=0x7fd568001250 comp rx=0 tx=0).stop 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd578fa9740 1 -- 172.21.15.38:0/4218982805 shutdown_connections 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd578fa9740 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.107:6835/1263010516,v1:172.21.15.107:6837/1263010516] conn(0x559b184c62b0 0x559b184fd200 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd578fa9740 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x559b184e7100 0x559b184c5d20 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd578fa9740 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x559b184c6ef0 0x559b184c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd54effd640 0 client.8604 ms_handle_reset on v2:172.21.15.107:6835/1263010516 2024-04-25T08:39:37.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd578fa9740 1 --2- 172.21.15.38:0/4218982805 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559b184c6b10 0x559b1854e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd578fa9740 1 -- 172.21.15.38:0/4218982805 >> 172.21.15.38:0/4218982805 conn(0x559b18499770 msgr2=0x559b184e4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd578fa9740 1 -- 172.21.15.38:0/4218982805 shutdown_connections 2024-04-25T08:39:37.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd578fa9740 1 -- 172.21.15.38:0/4218982805 wait complete. 2024-04-25T08:39:37.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd578fa9740 20 client.8604 trim_cache size 0 max 16384 2024-04-25T08:39:37.614 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:39:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-04-25T08:39:37.614 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-04-25T08:39:37.614 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:39:37.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd5762ac640 1 -- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd55c06b4d0 msgr2=0x7fd55c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd5762ac640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd55c06b4d0 0x7fd55c06d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd564034000 tx=0x7fd56402e000 comp rx=0 tx=0).stop 2024-04-25T08:39:37.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd5762ac640 1 -- 172.21.15.38:0/851347511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5700d1b50 msgr2=0x7fd570159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.615+0000 7fd5762ac640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5700d1b50 0x7fd570159cf0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd56000ebe0 tx=0x7fd56000c520 comp rx=0 tx=0).stop 2024-04-25T08:39:37.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.616+0000 7fd5762ac640 1 -- 172.21.15.38:0/851347511 shutdown_connections 2024-04-25T08:39:37.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.616+0000 7fd5762ac640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd55c06b4d0 0x7fd55c06d990 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.616+0000 7fd5762ac640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd57015f5f0 0x7fd57015d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.616+0000 7fd5762ac640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd57015b990 0x7fd5700b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.616+0000 7fd5762ac640 1 --2- 172.21.15.38:0/851347511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5700d1b50 0x7fd570159cf0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.616+0000 7fd5762ac640 1 -- 172.21.15.38:0/851347511 >> 172.21.15.38:0/851347511 conn(0x7fd57008b000 msgr2=0x7fd57006fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.616+0000 7fd5762ac640 1 -- 172.21.15.38:0/851347511 shutdown_connections 2024-04-25T08:39:37.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.616+0000 7fd5762ac640 1 -- 172.21.15.38:0/851347511 wait complete. 2024-04-25T08:39:37.619 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:37.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.726+0000 7fc5f34cd640 1 Processor -- start 2024-04-25T08:39:37.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.726+0000 7fc5f34cd640 1 -- start start 2024-04-25T08:39:37.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f34cd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc5ec15df50 0x7fc5ec0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f34cd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5ec0ce2e0 0x7fc5ec0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f34cd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ec0ceff0 con 0x7fc5ec06c110 2024-04-25T08:39:37.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f34cd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec0cf2d0 con 0x7fc5ec0ce2e0 2024-04-25T08:39:37.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f34cd640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec0d1f70 con 0x7fc5ec15df50 2024-04-25T08:39:37.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f1a6a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5ec0ce2e0 0x7fc5ec0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f1a6a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5ec0ce2e0 0x7fc5ec0d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42702/0 (socket says 172.21.15.38:42702) 2024-04-25T08:39:37.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f1a6a640 1 -- 172.21.15.38:0/1798974964 learned_addr learned my addr 172.21.15.38:0/1798974964 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f0a68640 1 --2- 172.21.15.38:0/1798974964 >> v2:172.21.15.107:3300/0 conn(0x7fc5ec15df50 0x7fc5ec0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f1a6a640 1 -- 172.21.15.38:0/1798974964 >> v2:172.21.15.107:3300/0 conn(0x7fc5ec15df50 msgr2=0x7fc5ec0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:37.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f1a6a640 1 --2- 172.21.15.38:0/1798974964 >> v2:172.21.15.107:3300/0 conn(0x7fc5ec15df50 0x7fc5ec0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f1a6a640 1 -- 172.21.15.38:0/1798974964 >> v1:172.21.15.107:6789/0 conn(0x7fc5ec06c110 legacy=0x7fc5ec06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f1a6a640 1 -- 172.21.15.38:0/1798974964 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec0d2250 con 0x7fc5ec0ce2e0 2024-04-25T08:39:37.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.727+0000 7fc5f0a68640 1 --2- 172.21.15.38:0/1798974964 >> v2:172.21.15.107:3300/0 conn(0x7fc5ec15df50 0x7fc5ec0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.728+0000 7fc5f1a6a640 1 --2- 172.21.15.38:0/1798974964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5ec0ce2e0 0x7fc5ec0d2a50 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0d4960 tx=0x7fc5e802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c237517272aebc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.728+0000 7fc5e3fff640 1 -- 172.21.15.38:0/1798974964 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8036070 con 0x7fc5ec0ce2e0 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.728+0000 7fc5e3fff640 1 -- 172.21.15.38:0/1798974964 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e8031600 con 0x7fc5ec0ce2e0 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.728+0000 7fc5e3fff640 1 -- 172.21.15.38:0/1798974964 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e80319d0 con 0x7fc5ec0ce2e0 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.729+0000 7fc5f34cd640 1 -- 172.21.15.38:0/1798974964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5ec0ce2e0 msgr2=0x7fc5ec0d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.729+0000 7fc5f34cd640 1 --2- 172.21.15.38:0/1798974964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5ec0ce2e0 0x7fc5ec0d2a50 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec0d4960 tx=0x7fc5e802a730 comp rx=0 tx=0).stop 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.729+0000 7fc5f34cd640 1 -- 172.21.15.38:0/1798974964 shutdown_connections 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.729+0000 7fc5f34cd640 1 --2- 172.21.15.38:0/1798974964 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5ec0ce2e0 0x7fc5ec0d2a50 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.729+0000 7fc5f34cd640 1 --2- 172.21.15.38:0/1798974964 >> v2:172.21.15.107:3300/0 conn(0x7fc5ec15df50 0x7fc5ec0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.729+0000 7fc5f34cd640 1 -- 172.21.15.38:0/1798974964 >> 172.21.15.38:0/1798974964 conn(0x7fc5ec0d3fb0 msgr2=0x7fc5ec0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.729+0000 7fc5f34cd640 1 -- 172.21.15.38:0/1798974964 shutdown_connections 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.729+0000 7fc5f34cd640 1 -- 172.21.15.38:0/1798974964 wait complete. 2024-04-25T08:39:37.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.730+0000 7fc5f34cd640 1 Processor -- start 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.730+0000 7fc5f34cd640 1 -- start start 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f34cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5ec06c110 0x7fc5ec148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f34cd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec0ce2e0 0x7fc5ec157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f34cd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5ec15df50 0x7fc5ec158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f34cd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec0d2f90 con 0x7fc5ec06c110 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f34cd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec0cece0 con 0x7fc5ec0ce2e0 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f34cd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec0d3110 con 0x7fc5ec15df50 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f1269640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5ec06c110 0x7fc5ec148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f1269640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5ec06c110 0x7fc5ec148de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42714/0 (socket says 172.21.15.38:42714) 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f1269640 1 -- 172.21.15.38:0/2205900230 learned_addr learned my addr 172.21.15.38:0/2205900230 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f0a68640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec0ce2e0 0x7fc5ec157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f1a6a640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5ec15df50 0x7fc5ec158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f1269640 1 -- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5ec15df50 msgr2=0x7fc5ec158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f1269640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5ec15df50 0x7fc5ec158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f1269640 1 -- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec0ce2e0 msgr2=0x7fc5ec157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f1269640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec0ce2e0 0x7fc5ec157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f1269640 1 -- 172.21.15.38:0/2205900230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec0c7790 con 0x7fc5ec06c110 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f0a68640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec0ce2e0 0x7fc5ec157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f1a6a640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5ec15df50 0x7fc5ec158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f1269640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5ec06c110 0x7fc5ec148de0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400c400 tx=0x7fc5e400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5e1ffb640 1 -- 172.21.15.38:0/2205900230 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e40052d0 con 0x7fc5ec06c110 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5e1ffb640 1 -- 172.21.15.38:0/2205900230 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e4005490 con 0x7fc5ec06c110 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5f34cd640 1 -- 172.21.15.38:0/2205900230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec0c7a70 con 0x7fc5ec06c110 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.731+0000 7fc5e1ffb640 1 -- 172.21.15.38:0/2205900230 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4007260 con 0x7fc5ec06c110 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.732+0000 7fc5f34cd640 1 -- 172.21.15.38:0/2205900230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec0c8030 con 0x7fc5ec06c110 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.733+0000 7fc5f34cd640 1 -- 172.21.15.38:0/2205900230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec068e80 con 0x7fc5ec06c110 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.733+0000 7fc5e1ffb640 1 -- 172.21.15.38:0/2205900230 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5e4007400 con 0x7fc5ec06c110 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.733+0000 7fc5e1ffb640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5d006b4d0 0x7fc5d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.733+0000 7fc5e1ffb640 1 -- 172.21.15.38:0/2205900230 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc5e4089880 con 0x7fc5ec06c110 2024-04-25T08:39:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.734+0000 7fc5f0a68640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5d006b4d0 0x7fc5d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:37.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.734+0000 7fc5f0a68640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5d006b4d0 0x7fc5d006d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc006fd0 tx=0x7fc5dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:37.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.738+0000 7fc5e1ffb640 1 -- 172.21.15.38:0/2205900230 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5e40585d0 con 0x7fc5ec06c110 2024-04-25T08:39:37.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.955+0000 7fc5f34cd640 1 -- 172.21.15.38:0/2205900230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5ec0b9110 con 0x7fc5ec06c110 2024-04-25T08:39:37.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.955+0000 7fc5e1ffb640 1 -- 172.21.15.38:0/2205900230 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fc5e4057ed0 con 0x7fc5ec06c110 2024-04-25T08:39:37.948 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:37.949 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:07.551641+0000","modified":"2024-04-25T08:39:09.555958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8292":{"gid":8292,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1263010516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1263010516},{"type":"v1","addr":"172.21.15.107:6837","nonce":1263010516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:39:37.949 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 63 2024-04-25T08:39:37.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.957+0000 7fc5f34cd640 1 -- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5d006b4d0 msgr2=0x7fc5d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.957+0000 7fc5f34cd640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5d006b4d0 0x7fc5d006d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc006fd0 tx=0x7fc5dc008040 comp rx=0 tx=0).stop 2024-04-25T08:39:37.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.957+0000 7fc5f34cd640 1 -- 172.21.15.38:0/2205900230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5ec06c110 msgr2=0x7fc5ec148de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:37.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.957+0000 7fc5f34cd640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5ec06c110 0x7fc5ec148de0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400c400 tx=0x7fc5e400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.957+0000 7fc5f34cd640 1 -- 172.21.15.38:0/2205900230 shutdown_connections 2024-04-25T08:39:37.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.957+0000 7fc5f34cd640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5d006b4d0 0x7fc5d006d990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.957+0000 7fc5f34cd640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5ec15df50 0x7fc5ec158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.957+0000 7fc5f34cd640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5ec0ce2e0 0x7fc5ec157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.957+0000 7fc5f34cd640 1 --2- 172.21.15.38:0/2205900230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5ec06c110 0x7fc5ec148de0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:37.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.957+0000 7fc5f34cd640 1 -- 172.21.15.38:0/2205900230 >> 172.21.15.38:0/2205900230 conn(0x7fc5ec0d3fb0 msgr2=0x7fc5ec0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:37.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.957+0000 7fc5f34cd640 1 -- 172.21.15.38:0/2205900230 shutdown_connections 2024-04-25T08:39:37.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:37.957+0000 7fc5f34cd640 1 -- 172.21.15.38:0/2205900230 wait complete. 2024-04-25T08:39:37.961 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:39:37.961 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:39:38.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.068+0000 7f8511910640 1 Processor -- start 2024-04-25T08:39:38.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.068+0000 7f8511910640 1 -- start start 2024-04-25T08:39:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.069+0000 7f8511910640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f850c06abe0 0x7f850c06b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.069+0000 7f8511910640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f850c0d01d0 0x7f850c0d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.069+0000 7f8511910640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850c0cfa40 con 0x7f850c0aeeb0 2024-04-25T08:39:38.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.069+0000 7f8511910640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850c0cd9b0 con 0x7f850c06abe0 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.069+0000 7f8511910640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850c0cdc90 con 0x7f850c0d01d0 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.069+0000 7f850a7fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f850c06abe0 0x7f850c06b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.069+0000 7f850affd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f850c0aeeb0 0x7f850c0ccf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:52504/0 (socket says 172.21.15.38:52504) 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.069+0000 7f850affd640 1 -- 172.21.15.38:0/3520218344 learned_addr learned my addr 172.21.15.38:0/3520218344 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.069+0000 7f850b7fe640 1 --2- 172.21.15.38:0/3520218344 >> v2:172.21.15.107:3300/0 conn(0x7f850c0d01d0 0x7f850c0d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.070+0000 7f850a7fc640 1 -- 172.21.15.38:0/3520218344 >> v2:172.21.15.107:3300/0 conn(0x7f850c0d01d0 msgr2=0x7f850c0d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.070+0000 7f850a7fc640 1 --2- 172.21.15.38:0/3520218344 >> v2:172.21.15.107:3300/0 conn(0x7f850c0d01d0 0x7f850c0d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.070+0000 7f850a7fc640 1 -- 172.21.15.38:0/3520218344 >> v1:172.21.15.38:6789/0 conn(0x7f850c0aeeb0 legacy=0x7f850c0ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.070+0000 7f850a7fc640 1 -- 172.21.15.38:0/3520218344 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c15e390 con 0x7f850c06abe0 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.070+0000 7f8509ffb640 1 -- 172.21.15.38:0/3520218344 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634063427 0 0) 0x7f850c0cfa40 con 0x7f850c0aeeb0 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.070+0000 7f850b7fe640 1 --2- 172.21.15.38:0/3520218344 >> v2:172.21.15.107:3300/0 conn(0x7f850c0d01d0 0x7f850c0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.070+0000 7f850a7fc640 1 --2- 172.21.15.38:0/3520218344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850c06abe0 0x7f850c06b090 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f850c0b66f0 tx=0x7f84f402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c77ce6ee946d4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.070+0000 7f8509ffb640 1 -- 172.21.15.38:0/3520218344 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4037070 con 0x7f850c06abe0 2024-04-25T08:39:38.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.070+0000 7f8509ffb640 1 -- 172.21.15.38:0/3520218344 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f4032440 con 0x7f850c06abe0 2024-04-25T08:39:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.070+0000 7f8509ffb640 1 -- 172.21.15.38:0/3520218344 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4032810 con 0x7f850c06abe0 2024-04-25T08:39:38.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.071+0000 7f8511910640 1 -- 172.21.15.38:0/3520218344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850c06abe0 msgr2=0x7f850c06b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.071+0000 7f8511910640 1 --2- 172.21.15.38:0/3520218344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850c06abe0 0x7f850c06b090 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f850c0b66f0 tx=0x7f84f402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:39:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.071+0000 7f8511910640 1 -- 172.21.15.38:0/3520218344 shutdown_connections 2024-04-25T08:39:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.071+0000 7f8511910640 1 --2- 172.21.15.38:0/3520218344 >> v2:172.21.15.107:3300/0 conn(0x7f850c0d01d0 0x7f850c0d05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.071+0000 7f8511910640 1 --2- 172.21.15.38:0/3520218344 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850c06abe0 0x7f850c06b090 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.071+0000 7f8511910640 1 -- 172.21.15.38:0/3520218344 >> 172.21.15.38:0/3520218344 conn(0x7f850c072610 msgr2=0x7f850c099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.071+0000 7f8511910640 1 -- 172.21.15.38:0/3520218344 shutdown_connections 2024-04-25T08:39:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.071+0000 7f8511910640 1 -- 172.21.15.38:0/3520218344 wait complete. 2024-04-25T08:39:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.072+0000 7f8511910640 1 Processor -- start 2024-04-25T08:39:38.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.072+0000 7f8511910640 1 -- start start 2024-04-25T08:39:38.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.073+0000 7f8511910640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850c06abe0 0x7f850c14ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:38.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.073+0000 7f8511910640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f850c0aeeb0 0x7f850c0c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:38.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.073+0000 7f8511910640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f850c0d01d0 0x7f850c0b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:38.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.073+0000 7f8511910640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c0d0af0 con 0x7f850c06abe0 2024-04-25T08:39:38.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.073+0000 7f8511910640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c0cd620 con 0x7f850c0aeeb0 2024-04-25T08:39:38.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.073+0000 7f8511910640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c0d0c70 con 0x7f850c0d01d0 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.073+0000 7f850affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850c06abe0 0x7f850c14ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.073+0000 7f850affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850c06abe0 0x7f850c14ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42728/0 (socket says 172.21.15.38:42728) 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.073+0000 7f850affd640 1 -- 172.21.15.38:0/1336206145 learned_addr learned my addr 172.21.15.38:0/1336206145 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.073+0000 7f850b7fe640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f850c0d01d0 0x7f850c0b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.073+0000 7f850a7fc640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f850c0aeeb0 0x7f850c0c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.074+0000 7f850affd640 1 -- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f850c0d01d0 msgr2=0x7f850c0b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.074+0000 7f850affd640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f850c0d01d0 0x7f850c0b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.074+0000 7f850affd640 1 -- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f850c0aeeb0 msgr2=0x7f850c0c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.074+0000 7f850affd640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f850c0aeeb0 0x7f850c0c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.074+0000 7f850affd640 1 -- 172.21.15.38:0/1336206145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c1f6a70 con 0x7f850c06abe0 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.074+0000 7f850affd640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850c06abe0 0x7f850c14ca60 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f850400c4d0 tx=0x7f850400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.074+0000 7f8503fff640 1 -- 172.21.15.38:0/1336206145 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504018070 con 0x7f850c06abe0 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.074+0000 7f8503fff640 1 -- 172.21.15.38:0/1336206145 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8504007500 con 0x7f850c06abe0 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.074+0000 7f8511910640 1 -- 172.21.15.38:0/1336206145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850c1f6d50 con 0x7f850c06abe0 2024-04-25T08:39:38.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.074+0000 7f8503fff640 1 -- 172.21.15.38:0/1336206145 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8504004a20 con 0x7f850c06abe0 2024-04-25T08:39:38.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.074+0000 7f8511910640 1 -- 172.21.15.38:0/1336206145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c1f7310 con 0x7f850c06abe0 2024-04-25T08:39:38.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.075+0000 7f8511910640 1 -- 172.21.15.38:0/1336206145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850c156730 con 0x7f850c06abe0 2024-04-25T08:39:38.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.075+0000 7f8503fff640 1 -- 172.21.15.38:0/1336206145 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f850401b070 con 0x7f850c06abe0 2024-04-25T08:39:38.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.076+0000 7f8503fff640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f84ec06b500 0x7f84ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:38.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.076+0000 7f8503fff640 1 -- 172.21.15.38:0/1336206145 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f85040895f0 con 0x7f850c06abe0 2024-04-25T08:39:38.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.076+0000 7f850a7fc640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f84ec06b500 0x7f84ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:38.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.077+0000 7f850a7fc640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f84ec06b500 0x7f84ec06d9c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f84f4034000 tx=0x7f84f402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:38.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.080+0000 7f8503fff640 1 -- 172.21.15.38:0/1336206145 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8504057c60 con 0x7f850c06abe0 2024-04-25T08:39:38.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.341+0000 7f8511910640 1 -- 172.21.15.38:0/1336206145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f850c14cfa0 con 0x7f850c06abe0 2024-04-25T08:39:38.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.341+0000 7f8503fff640 1 -- 172.21.15.38:0/1336206145 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f850405bd00 con 0x7f850c06abe0 2024-04-25T08:39:38.335 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:38.335 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":507,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077847623,"bytes_used":2659434496,"bytes_avail":770434678784,"bytes_total":773094113280,"write_bytes_sec":64165684,"read_op_per_sec":0,"write_op_per_sec":64},"fsmap":{"epoch":63,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8292}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T08:39:35.317071+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T08:39:38.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.343+0000 7f8511910640 1 -- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f84ec06b500 msgr2=0x7f84ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:38.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.343+0000 7f8511910640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f84ec06b500 0x7f84ec06d9c0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f84f4034000 tx=0x7f84f402e000 comp rx=0 tx=0).stop 2024-04-25T08:39:38.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.343+0000 7f8511910640 1 -- 172.21.15.38:0/1336206145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850c06abe0 msgr2=0x7f850c14ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:38.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.343+0000 7f8511910640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850c06abe0 0x7f850c14ca60 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f850400c4d0 tx=0x7f850400c990 comp rx=0 tx=0).stop 2024-04-25T08:39:38.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.343+0000 7f8511910640 1 -- 172.21.15.38:0/1336206145 shutdown_connections 2024-04-25T08:39:38.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.343+0000 7f8511910640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f84ec06b500 0x7f84ec06d9c0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:38.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.343+0000 7f8511910640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f850c0d01d0 0x7f850c0b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:38.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.343+0000 7f8511910640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f850c0aeeb0 0x7f850c0c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:38.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.343+0000 7f8511910640 1 --2- 172.21.15.38:0/1336206145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f850c06abe0 0x7f850c14ca60 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:38.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.343+0000 7f8511910640 1 -- 172.21.15.38:0/1336206145 >> 172.21.15.38:0/1336206145 conn(0x7f850c072610 msgr2=0x7f850c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:38.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.343+0000 7f8511910640 1 -- 172.21.15.38:0/1336206145 shutdown_connections 2024-04-25T08:39:38.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:38.343+0000 7f8511910640 1 -- 172.21.15.38:0/1336206145 wait complete. 2024-04-25T08:39:38.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T08:39:39.348 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:39.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.469+0000 7fb23d05d640 1 Processor -- start 2024-04-25T08:39:39.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.469+0000 7fb23d05d640 1 -- start start 2024-04-25T08:39:39.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.470+0000 7fb23d05d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb23806c110 0x7fb23806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:39.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.470+0000 7fb23d05d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb23815df50 0x7fb2380ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:39.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.470+0000 7fb23d05d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23806ca30 con 0x7fb2380ce2e0 2024-04-25T08:39:39.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.470+0000 7fb23d05d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2380cf2d0 con 0x7fb23815df50 2024-04-25T08:39:39.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.470+0000 7fb23d05d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380ceff0 con 0x7fb23806c110 2024-04-25T08:39:39.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.470+0000 7fb22ffff640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb23815df50 0x7fb2380ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:39.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.470+0000 7fb236d9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb23806c110 0x7fb23806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:39.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.470+0000 7fb236d9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb23806c110 0x7fb23806c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:46892/0 (socket says 172.21.15.38:46892) 2024-04-25T08:39:39.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.470+0000 7fb22ffff640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb23815df50 0x7fb2380ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:59516/0 (socket says 172.21.15.38:59516) 2024-04-25T08:39:39.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.470+0000 7fb236d9d640 1 -- 172.21.15.38:0/3529210109 learned_addr learned my addr 172.21.15.38:0/3529210109 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:39.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.470+0000 7fb23759e640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fb2380ce2e0 0x7fb2380d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:33190/0 (socket says 172.21.15.38:33190) 2024-04-25T08:39:39.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.471+0000 7fb23659c640 1 -- 172.21.15.38:0/3529210109 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195788837 0 0) 0x7fb23806ca30 con 0x7fb2380ce2e0 2024-04-25T08:39:39.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.471+0000 7fb23659c640 1 -- 172.21.15.38:0/3529210109 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb224003520 con 0x7fb2380ce2e0 2024-04-25T08:39:39.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.471+0000 7fb236d9d640 1 -- 172.21.15.38:0/3529210109 >> v2:172.21.15.107:3300/0 conn(0x7fb23815df50 msgr2=0x7fb2380ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:39.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.471+0000 7fb236d9d640 1 --2- 172.21.15.38:0/3529210109 >> v2:172.21.15.107:3300/0 conn(0x7fb23815df50 0x7fb2380ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:39.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.471+0000 7fb236d9d640 1 -- 172.21.15.38:0/3529210109 >> v1:172.21.15.107:6789/0 conn(0x7fb2380ce2e0 legacy=0x7fb2380d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:39.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.471+0000 7fb236d9d640 1 -- 172.21.15.38:0/3529210109 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2380d1f70 con 0x7fb23806c110 2024-04-25T08:39:39.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.471+0000 7fb22ffff640 1 --2- 172.21.15.38:0/3529210109 >> v2:172.21.15.107:3300/0 conn(0x7fb23815df50 0x7fb2380ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:39.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.471+0000 7fb236d9d640 1 --2- 172.21.15.38:0/3529210109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb23806c110 0x7fb23806c4f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb228009870 tx=0x7fb22802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd2dc6a85be71992 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:39.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.472+0000 7fb23659c640 1 -- 172.21.15.38:0/3529210109 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228037070 con 0x7fb23806c110 2024-04-25T08:39:39.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.472+0000 7fb23659c640 1 -- 172.21.15.38:0/3529210109 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb228032440 con 0x7fb23806c110 2024-04-25T08:39:39.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.472+0000 7fb23659c640 1 -- 172.21.15.38:0/3529210109 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2280327f0 con 0x7fb23806c110 2024-04-25T08:39:39.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.472+0000 7fb23d05d640 1 -- 172.21.15.38:0/3529210109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb23806c110 msgr2=0x7fb23806c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:39.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.472+0000 7fb23d05d640 1 --2- 172.21.15.38:0/3529210109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb23806c110 0x7fb23806c4f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb228009870 tx=0x7fb22802a250 comp rx=0 tx=0).stop 2024-04-25T08:39:39.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.472+0000 7fb23d05d640 1 -- 172.21.15.38:0/3529210109 shutdown_connections 2024-04-25T08:39:39.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.472+0000 7fb23d05d640 1 --2- 172.21.15.38:0/3529210109 >> v2:172.21.15.107:3300/0 conn(0x7fb23815df50 0x7fb2380ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:39.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.472+0000 7fb23d05d640 1 --2- 172.21.15.38:0/3529210109 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb23806c110 0x7fb23806c4f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:39.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.472+0000 7fb23d05d640 1 -- 172.21.15.38:0/3529210109 >> 172.21.15.38:0/3529210109 conn(0x7fb2380d3fb0 msgr2=0x7fb2380d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:39.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.472+0000 7fb23d05d640 1 -- 172.21.15.38:0/3529210109 shutdown_connections 2024-04-25T08:39:39.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.472+0000 7fb23d05d640 1 -- 172.21.15.38:0/3529210109 wait complete. 2024-04-25T08:39:39.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.474+0000 7fb23d05d640 1 Processor -- start 2024-04-25T08:39:39.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.474+0000 7fb23d05d640 1 -- start start 2024-04-25T08:39:39.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.474+0000 7fb23d05d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb23806c110 0x7fb23814a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:39.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.474+0000 7fb23d05d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2380ce2e0 0x7fb2380b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:39.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.474+0000 7fb23d05d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb23815df50 0x7fb2380b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:39.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.474+0000 7fb23d05d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380cece0 con 0x7fb2380ce2e0 2024-04-25T08:39:39.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.474+0000 7fb23d05d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380d2530 con 0x7fb23815df50 2024-04-25T08:39:39.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.474+0000 7fb23d05d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380d3160 con 0x7fb23806c110 2024-04-25T08:39:39.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.474+0000 7fb22ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2380ce2e0 0x7fb2380b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:39.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.474+0000 7fb22ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2380ce2e0 0x7fb2380b75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38792/0 (socket says 172.21.15.38:38792) 2024-04-25T08:39:39.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb22ffff640 1 -- 172.21.15.38:0/463160856 learned_addr learned my addr 172.21.15.38:0/463160856 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:39.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb236d9d640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb23806c110 0x7fb23814a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:39.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb23759e640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb23815df50 0x7fb2380b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:39.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb22ffff640 1 -- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb23806c110 msgr2=0x7fb23814a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:39.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb22ffff640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb23806c110 0x7fb23814a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:39.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb22ffff640 1 -- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb23815df50 msgr2=0x7fb2380b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:39.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb22ffff640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb23815df50 0x7fb2380b7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:39.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb22ffff640 1 -- 172.21.15.38:0/463160856 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23814c860 con 0x7fb2380ce2e0 2024-04-25T08:39:39.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb22ffff640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2380ce2e0 0x7fb2380b75a0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fb22000e990 tx=0x7fb22000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:39.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb22f7fe640 1 -- 172.21.15.38:0/463160856 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220017070 con 0x7fb2380ce2e0 2024-04-25T08:39:39.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb22f7fe640 1 -- 172.21.15.38:0/463160856 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22000c9e0 con 0x7fb2380ce2e0 2024-04-25T08:39:39.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb23d05d640 1 -- 172.21.15.38:0/463160856 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2380cb6d0 con 0x7fb2380ce2e0 2024-04-25T08:39:39.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb22f7fe640 1 -- 172.21.15.38:0/463160856 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22000cdb0 con 0x7fb2380ce2e0 2024-04-25T08:39:39.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.475+0000 7fb23d05d640 1 -- 172.21.15.38:0/463160856 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23814ce00 con 0x7fb2380ce2e0 2024-04-25T08:39:39.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.476+0000 7fb23d05d640 1 -- 172.21.15.38:0/463160856 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb238068e80 con 0x7fb2380ce2e0 2024-04-25T08:39:39.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.477+0000 7fb22f7fe640 1 -- 172.21.15.38:0/463160856 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb22001a070 con 0x7fb2380ce2e0 2024-04-25T08:39:39.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.477+0000 7fb22f7fe640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb21806b4d0 0x7fb21806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:39.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.478+0000 7fb22f7fe640 1 -- 172.21.15.38:0/463160856 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb22008a540 con 0x7fb2380ce2e0 2024-04-25T08:39:39.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.478+0000 7fb236d9d640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb21806b4d0 0x7fb21806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:39.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.478+0000 7fb236d9d640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb21806b4d0 0x7fb21806d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb228034000 tx=0x7fb22802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:39.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.482+0000 7fb22f7fe640 1 -- 172.21.15.38:0/463160856 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb220058bb0 con 0x7fb2380ce2e0 2024-04-25T08:39:39.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.699+0000 7fb23d05d640 1 -- 172.21.15.38:0/463160856 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb23815d370 con 0x7fb2380ce2e0 2024-04-25T08:39:39.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.699+0000 7fb22f7fe640 1 -- 172.21.15.38:0/463160856 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb22005cc50 con 0x7fb2380ce2e0 2024-04-25T08:39:39.693 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:39.693 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:07.551641+0000","modified":"2024-04-25T08:39:09.555958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8292":{"gid":8292,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1263010516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1263010516},{"type":"v1","addr":"172.21.15.107:6837","nonce":1263010516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:39:39.693 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 63 2024-04-25T08:39:39.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.701+0000 7fb23d05d640 1 -- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb21806b4d0 msgr2=0x7fb21806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:39.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.701+0000 7fb23d05d640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb21806b4d0 0x7fb21806d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fb228034000 tx=0x7fb22802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:39.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.701+0000 7fb23d05d640 1 -- 172.21.15.38:0/463160856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2380ce2e0 msgr2=0x7fb2380b75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:39.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.701+0000 7fb23d05d640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2380ce2e0 0x7fb2380b75a0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fb22000e990 tx=0x7fb22000ee50 comp rx=0 tx=0).stop 2024-04-25T08:39:39.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.701+0000 7fb23d05d640 1 -- 172.21.15.38:0/463160856 shutdown_connections 2024-04-25T08:39:39.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.701+0000 7fb23d05d640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb23815df50 0x7fb2380b7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:39.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.701+0000 7fb23d05d640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2380ce2e0 0x7fb2380b75a0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:39.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.701+0000 7fb23d05d640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb21806b4d0 0x7fb21806d990 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:39.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.701+0000 7fb23d05d640 1 --2- 172.21.15.38:0/463160856 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb23806c110 0x7fb23814a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:39.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.701+0000 7fb23d05d640 1 -- 172.21.15.38:0/463160856 >> 172.21.15.38:0/463160856 conn(0x7fb2380d3fb0 msgr2=0x7fb2380d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:39.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.701+0000 7fb23d05d640 1 -- 172.21.15.38:0/463160856 shutdown_connections 2024-04-25T08:39:39.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.701+0000 7fb23d05d640 1 -- 172.21.15.38:0/463160856 wait complete. 2024-04-25T08:39:39.705 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:39:39.706 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:39:39.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.812+0000 7f10b77b7640 1 Processor -- start 2024-04-25T08:39:39.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.812+0000 7f10b77b7640 1 -- start start 2024-04-25T08:39:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10b77b7640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f10b015fbf0 0x7f10b00c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:39.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10b77b7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b0160340 con 0x7f10b015bb10 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10b77b7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b00cf400 con 0x7f10b009ab90 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10b77b7640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b00cffd0 con 0x7f10b015fbf0 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10b5553640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f10b015bb10 0x7f10b009a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55654/0 (socket says 172.21.15.38:55654) 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10b5553640 1 -- 172.21.15.38:0/271735844 learned_addr learned my addr 172.21.15.38:0/271735844 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10b5d54640 1 --2- 172.21.15.38:0/271735844 >> v2:172.21.15.107:3300/0 conn(0x7f10b015fbf0 0x7f10b00c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10a7fff640 1 -- 172.21.15.38:0/271735844 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964712215 0 0) 0x7f10ac001110 con 0x7f10b015bb10 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10a7fff640 1 -- 172.21.15.38:0/271735844 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f109c003520 con 0x7f10b015bb10 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10a7fff640 1 -- 172.21.15.38:0/271735844 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1025260812 0 0) 0x7f109c003520 con 0x7f10b015bb10 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10b5d54640 1 -- 172.21.15.38:0/271735844 >> v1:172.21.15.107:6789/0 conn(0x7f10b009ab90 legacy=0x7f10b015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10b5d54640 1 -- 172.21.15.38:0/271735844 >> v1:172.21.15.38:6789/0 conn(0x7f10b015bb10 legacy=0x7f10b009a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.813+0000 7f10b5d54640 1 -- 172.21.15.38:0/271735844 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b00cf6a0 con 0x7f10b015fbf0 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.814+0000 7f10b5d54640 1 --2- 172.21.15.38:0/271735844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10b015fbf0 0x7f10b00c5250 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009d90 tx=0x7f10a802a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9e2637ad45c228a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.814+0000 7f10a7fff640 1 -- 172.21.15.38:0/271735844 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8035020 con 0x7f10b015fbf0 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.814+0000 7f10a7fff640 1 -- 172.21.15.38:0/271735844 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a8007540 con 0x7f10b015fbf0 2024-04-25T08:39:39.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.814+0000 7f10a7fff640 1 -- 172.21.15.38:0/271735844 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac001840 con 0x7f10b015fbf0 2024-04-25T08:39:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.815+0000 7f10b77b7640 1 -- 172.21.15.38:0/271735844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10b015fbf0 msgr2=0x7f10b00c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:39.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.815+0000 7f10b77b7640 1 --2- 172.21.15.38:0/271735844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10b015fbf0 0x7f10b00c5250 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009d90 tx=0x7f10a802a510 comp rx=0 tx=0).stop 2024-04-25T08:39:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.815+0000 7f10b77b7640 1 -- 172.21.15.38:0/271735844 shutdown_connections 2024-04-25T08:39:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.815+0000 7f10b77b7640 1 --2- 172.21.15.38:0/271735844 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10b015fbf0 0x7f10b00c5250 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.815+0000 7f10b77b7640 1 -- 172.21.15.38:0/271735844 >> 172.21.15.38:0/271735844 conn(0x7f10b00b31f0 msgr2=0x7f10b00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.815+0000 7f10b77b7640 1 -- 172.21.15.38:0/271735844 shutdown_connections 2024-04-25T08:39:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.815+0000 7f10b77b7640 1 -- 172.21.15.38:0/271735844 wait complete. 2024-04-25T08:39:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.816+0000 7f10b77b7640 1 Processor -- start 2024-04-25T08:39:39.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.816+0000 7f10b77b7640 1 -- start start 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b77b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10b009ab90 0x7f10b00c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b77b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10b015bb10 0x7f10b00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b77b7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10b015fbf0 0x7f10b00b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b77b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b00d0460 con 0x7f10b009ab90 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b77b7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b00cfc60 con 0x7f10b015bb10 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b77b7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b00d05e0 con 0x7f10b015fbf0 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b5553640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10b009ab90 0x7f10b00c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b5553640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10b009ab90 0x7f10b00c4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38796/0 (socket says 172.21.15.38:38796) 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b5553640 1 -- 172.21.15.38:0/958526413 learned_addr learned my addr 172.21.15.38:0/958526413 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b5553640 1 -- 172.21.15.38:0/958526413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10b015fbf0 msgr2=0x7f10b00b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b5553640 1 --2- 172.21.15.38:0/958526413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10b015fbf0 0x7f10b00b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b5553640 1 -- 172.21.15.38:0/958526413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10b015bb10 msgr2=0x7f10b00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b5553640 1 --2- 172.21.15.38:0/958526413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10b015bb10 0x7f10b00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:39.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b5553640 1 -- 172.21.15.38:0/958526413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b01f1d70 con 0x7f10b009ab90 2024-04-25T08:39:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.817+0000 7f10b5553640 1 --2- 172.21.15.38:0/958526413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10b009ab90 0x7f10b00c4be0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0019b0 tx=0x7f10ac00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.818+0000 7f10a5ffb640 1 -- 172.21.15.38:0/958526413 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac018070 con 0x7f10b009ab90 2024-04-25T08:39:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.818+0000 7f10a5ffb640 1 -- 172.21.15.38:0/958526413 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ac0074e0 con 0x7f10b009ab90 2024-04-25T08:39:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.818+0000 7f10b77b7640 1 -- 172.21.15.38:0/958526413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b01e6f60 con 0x7f10b009ab90 2024-04-25T08:39:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.818+0000 7f10a5ffb640 1 -- 172.21.15.38:0/958526413 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac004a00 con 0x7f10b009ab90 2024-04-25T08:39:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.818+0000 7f10b77b7640 1 -- 172.21.15.38:0/958526413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b01e7520 con 0x7f10b009ab90 2024-04-25T08:39:39.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.819+0000 7f10b77b7640 1 -- 172.21.15.38:0/958526413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b0068e80 con 0x7f10b009ab90 2024-04-25T08:39:39.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.819+0000 7f10a5ffb640 1 -- 172.21.15.38:0/958526413 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f10ac01b070 con 0x7f10b009ab90 2024-04-25T08:39:39.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.819+0000 7f10a5ffb640 1 --2- 172.21.15.38:0/958526413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f109406b4d0 0x7f109406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:39.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.820+0000 7f10a5ffb640 1 -- 172.21.15.38:0/958526413 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f10ac089720 con 0x7f10b009ab90 2024-04-25T08:39:39.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.820+0000 7f10b4d52640 1 --2- 172.21.15.38:0/958526413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f109406b4d0 0x7f109406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:39.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.820+0000 7f10b4d52640 1 --2- 172.21.15.38:0/958526413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f109406b4d0 0x7f109406d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f10b01575c0 tx=0x7f10a0007950 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:39.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:39.824+0000 7f10a5ffb640 1 -- 172.21.15.38:0/958526413 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10b0160340 con 0x7f10b009ab90 2024-04-25T08:39:40.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.075+0000 7f10b77b7640 1 -- 172.21.15.38:0/958526413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f10b00cdbf0 con 0x7f10b009ab90 2024-04-25T08:39:40.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.076+0000 7f10a5ffb640 1 -- 172.21.15.38:0/958526413 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f10ac0583d0 con 0x7f10b009ab90 2024-04-25T08:39:40.069 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:40.069 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":508,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":110,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077847623,"bytes_used":2659434496,"bytes_avail":770434678784,"bytes_total":773094113280,"write_bytes_sec":38673815,"read_op_per_sec":0,"write_op_per_sec":39},"fsmap":{"epoch":63,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8292}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T08:39:39.318198+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:39:40.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.078+0000 7f10b77b7640 1 -- 172.21.15.38:0/958526413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f109406b4d0 msgr2=0x7f109406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.078+0000 7f10b77b7640 1 --2- 172.21.15.38:0/958526413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f109406b4d0 0x7f109406d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f10b01575c0 tx=0x7f10a0007950 comp rx=0 tx=0).stop 2024-04-25T08:39:40.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.078+0000 7f10b77b7640 1 -- 172.21.15.38:0/958526413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10b009ab90 msgr2=0x7f10b00c4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.078+0000 7f10b77b7640 1 --2- 172.21.15.38:0/958526413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10b009ab90 0x7f10b00c4be0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0019b0 tx=0x7f10ac00c9b0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.078+0000 7f10b77b7640 1 -- 172.21.15.38:0/958526413 shutdown_connections 2024-04-25T08:39:40.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.078+0000 7f10b77b7640 1 --2- 172.21.15.38:0/958526413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f109406b4d0 0x7f109406d990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.078+0000 7f10b77b7640 1 --2- 172.21.15.38:0/958526413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f10b015fbf0 0x7f10b00b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.078+0000 7f10b77b7640 1 --2- 172.21.15.38:0/958526413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f10b015bb10 0x7f10b00b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.078+0000 7f10b77b7640 1 --2- 172.21.15.38:0/958526413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10b009ab90 0x7f10b00c4be0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.078+0000 7f10b77b7640 1 -- 172.21.15.38:0/958526413 >> 172.21.15.38:0/958526413 conn(0x7f10b00b31f0 msgr2=0x7f10b00c7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:40.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.078+0000 7f10b77b7640 1 -- 172.21.15.38:0/958526413 shutdown_connections 2024-04-25T08:39:40.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.078+0000 7f10b77b7640 1 -- 172.21.15.38:0/958526413 wait complete. 2024-04-25T08:39:40.081 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:39:40.082 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:39:40.082 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:39:40.120 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:39:40.159 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:39:40.159 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:39:40.159 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:40.176 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:40.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.325+0000 7ff339e9a640 1 Processor -- start 2024-04-25T08:39:40.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.325+0000 7ff339e9a640 1 -- start start 2024-04-25T08:39:40.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.326+0000 7ff339e9a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff33415ddf0 0x7ff334161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.326+0000 7ff339e9a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3340b93f0 0x7ff3340d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.326+0000 7ff339e9a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3340b98a0 con 0x7ff3340b5710 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.326+0000 7ff339e9a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff334161e00 con 0x7ff33415ddf0 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.326+0000 7ff339e9a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3341620e0 con 0x7ff3340b93f0 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.326+0000 7ff3337fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff3340b5710 0x7ff3340b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55662/0 (socket says 172.21.15.38:55662) 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.326+0000 7ff3337fe640 1 -- 172.21.15.38:0/219932589 learned_addr learned my addr 172.21.15.38:0/219932589 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.326+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4988855 0 0) 0x7ff3340b98a0 con 0x7ff3340b5710 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff31c003520 con 0x7ff3340b5710 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff333fff640 1 --2- 172.21.15.38:0/219932589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3340b93f0 0x7ff3340d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff332ffd640 1 --2- 172.21.15.38:0/219932589 >> v2:172.21.15.107:3300/0 conn(0x7ff33415ddf0 0x7ff334161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 794312011 0 0) 0x7ff31c003520 con 0x7ff3340b5710 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3340b93f0 msgr2=0x7ff3340d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 --2- 172.21.15.38:0/219932589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3340b93f0 0x7ff3340d2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 >> v2:172.21.15.107:3300/0 conn(0x7ff33415ddf0 msgr2=0x7ff334161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 --2- 172.21.15.38:0/219932589 >> v2:172.21.15.107:3300/0 conn(0x7ff33415ddf0 0x7ff334161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3341623c0 con 0x7ff3340b5710 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7ff324001670 con 0x7ff3340b5710 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 >> v1:172.21.15.38:6789/0 conn(0x7ff3340b5710 legacy=0x7ff3340b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 --2- 172.21.15.38:0/219932589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3340b93f0 0x7ff31c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 --2- 172.21.15.38:0/219932589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff31c003f30 0x7ff31c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 --2- 172.21.15.38:0/219932589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c004d40 0x7ff31c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff334161e00 con 0x7ff31c004d40 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3341620e0 con 0x7ff31c003f30 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31c00b9c0 con 0x7ff3340b93f0 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff333fff640 1 --2- 172.21.15.38:0/219932589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c004d40 0x7ff31c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff3337fe640 1 --2- 172.21.15.38:0/219932589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3340b93f0 0x7ff31c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff332ffd640 1 --2- 172.21.15.38:0/219932589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff31c003f30 0x7ff31c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff333fff640 1 -- 172.21.15.38:0/219932589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3340b93f0 msgr2=0x7ff31c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff333fff640 1 --2- 172.21.15.38:0/219932589 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3340b93f0 0x7ff31c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff333fff640 1 -- 172.21.15.38:0/219932589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff31c003f30 msgr2=0x7ff31c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff333fff640 1 --2- 172.21.15.38:0/219932589 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff31c003f30 0x7ff31c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.327+0000 7ff333fff640 1 -- 172.21.15.38:0/219932589 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31c003520 con 0x7ff31c004d40 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.328+0000 7ff333fff640 1 --2- 172.21.15.38:0/219932589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c004d40 0x7ff31c0051c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009fc0 tx=0x7ff31c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95451c51037a0712 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.328+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c009c10 con 0x7ff31c004d40 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.328+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c036070 con 0x7ff31c004d40 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.328+0000 7ff3327fc640 1 -- 172.21.15.38:0/219932589 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c031670 con 0x7ff31c004d40 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.329+0000 7ff339e9a640 1 -- 172.21.15.38:0/219932589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c004d40 msgr2=0x7ff31c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.329+0000 7ff339e9a640 1 --2- 172.21.15.38:0/219932589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff31c004d40 0x7ff31c0051c0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009fc0 tx=0x7ff31c005830 comp rx=0 tx=0).stop 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.329+0000 7ff333fff640 1 -- 172.21.15.38:0/219932589 reap_dead start 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.329+0000 7ff339e9a640 1 -- 172.21.15.38:0/219932589 shutdown_connections 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.329+0000 7ff339e9a640 1 -- 172.21.15.38:0/219932589 >> 172.21.15.38:0/219932589 conn(0x7ff3340d07c0 msgr2=0x7ff3340d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.329+0000 7ff339e9a640 1 -- 172.21.15.38:0/219932589 shutdown_connections 2024-04-25T08:39:40.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.329+0000 7ff339e9a640 1 -- 172.21.15.38:0/219932589 wait complete. 2024-04-25T08:39:40.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.330+0000 7ff339e9a640 1 Processor -- start 2024-04-25T08:39:40.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.330+0000 7ff339e9a640 1 -- start start 2024-04-25T08:39:40.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff339e9a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff31c004d40 0x7ff334152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff339e9a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3340cbbb0 0x7ff33415e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff339e9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3340cc130 0x7ff3340cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff339e9a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340d26d0 con 0x7ff3340cc130 2024-04-25T08:39:40.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff339e9a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340d2550 con 0x7ff3340cbbb0 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff339e9a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340d2850 con 0x7ff31c004d40 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff333fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3340cc130 0x7ff3340cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff333fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3340cc130 0x7ff3340cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38824/0 (socket says 172.21.15.38:38824) 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff333fff640 1 -- 172.21.15.38:0/2039387135 learned_addr learned my addr 172.21.15.38:0/2039387135 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff3337fe640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff31c004d40 0x7ff334152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff332ffd640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3340cbbb0 0x7ff33415e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff333fff640 1 -- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff31c004d40 msgr2=0x7ff334152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff333fff640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff31c004d40 0x7ff334152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff333fff640 1 -- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3340cbbb0 msgr2=0x7ff33415e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff333fff640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3340cbbb0 0x7ff33415e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff333fff640 1 -- 172.21.15.38:0/2039387135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33415eb00 con 0x7ff3340cc130 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff3337fe640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff31c004d40 0x7ff334152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:40.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.331+0000 7ff333fff640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3340cc130 0x7ff3340cc5e0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00e230 tx=0x7ff32c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:40.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.332+0000 7ff31bfff640 1 -- 172.21.15.38:0/2039387135 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c009c10 con 0x7ff3340cc130 2024-04-25T08:39:40.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.332+0000 7ff31bfff640 1 -- 172.21.15.38:0/2039387135 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c031db0 con 0x7ff3340cc130 2024-04-25T08:39:40.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.332+0000 7ff339e9a640 1 -- 172.21.15.38:0/2039387135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33415ede0 con 0x7ff3340cc130 2024-04-25T08:39:40.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.332+0000 7ff31bfff640 1 -- 172.21.15.38:0/2039387135 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c003bd0 con 0x7ff3340cc130 2024-04-25T08:39:40.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.332+0000 7ff339e9a640 1 -- 172.21.15.38:0/2039387135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33415f3a0 con 0x7ff3340cc130 2024-04-25T08:39:40.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.333+0000 7ff339e9a640 1 -- 172.21.15.38:0/2039387135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff334068e80 con 0x7ff3340cc130 2024-04-25T08:39:40.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.333+0000 7ff31bfff640 1 -- 172.21.15.38:0/2039387135 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff32c005d10 con 0x7ff3340cc130 2024-04-25T08:39:40.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.334+0000 7ff31bfff640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff32006b500 0x7ff32006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.334+0000 7ff31bfff640 1 -- 172.21.15.38:0/2039387135 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff32c0ab6a0 con 0x7ff3340cc130 2024-04-25T08:39:40.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.334+0000 7ff3337fe640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff32006b500 0x7ff32006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.334+0000 7ff3337fe640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff32006b500 0x7ff32006d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff324001050 tx=0x7ff324009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:40.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.338+0000 7ff31bfff640 1 -- 172.21.15.38:0/2039387135 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff32c049060 con 0x7ff3340cc130 2024-04-25T08:39:40.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.519+0000 7ff339e9a640 1 -- 172.21.15.38:0/2039387135 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3340b98a0 con 0x7ff3340cc130 2024-04-25T08:39:40.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.520+0000 7ff31bfff640 1 -- 172.21.15.38:0/2039387135 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16970 (secure 0 0 0) 0x7ff32c07a3f0 con 0x7ff3340cc130 2024-04-25T08:39:40.514 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:40.514 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":110,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:18.940299+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:05.852531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:06.527775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":1,"to":7}]},{"pgid":"16.1e","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.522+0000 7ff339e9a640 1 -- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff32006b500 msgr2=0x7ff32006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.522+0000 7ff339e9a640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff32006b500 0x7ff32006d9c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff324001050 tx=0x7ff324009040 comp rx=0 tx=0).stop 2024-04-25T08:39:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.522+0000 7ff339e9a640 1 -- 172.21.15.38:0/2039387135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3340cc130 msgr2=0x7ff3340cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.522+0000 7ff339e9a640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3340cc130 0x7ff3340cc5e0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff32c00e230 tx=0x7ff32c009750 comp rx=0 tx=0).stop 2024-04-25T08:39:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.522+0000 7ff339e9a640 1 -- 172.21.15.38:0/2039387135 shutdown_connections 2024-04-25T08:39:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.522+0000 7ff339e9a640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff3340cc130 0x7ff3340cc5e0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.522+0000 7ff339e9a640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3340cbbb0 0x7ff33415e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.522+0000 7ff339e9a640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff32006b500 0x7ff32006d9c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.522+0000 7ff339e9a640 1 --2- 172.21.15.38:0/2039387135 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff31c004d40 0x7ff334152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.522+0000 7ff339e9a640 1 -- 172.21.15.38:0/2039387135 >> 172.21.15.38:0/2039387135 conn(0x7ff3340d07c0 msgr2=0x7ff3340d2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.522+0000 7ff339e9a640 1 -- 172.21.15.38:0/2039387135 shutdown_connections 2024-04-25T08:39:40.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.522+0000 7ff339e9a640 1 -- 172.21.15.38:0/2039387135 wait complete. 2024-04-25T08:39:40.526 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:39:05.852531+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:39:06.527775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:39:40.527 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:39:40.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.637+0000 7f5e38afc640 1 Processor -- start 2024-04-25T08:39:40.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.637+0000 7f5e38afc640 1 -- start start 2024-04-25T08:39:40.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.638+0000 7f5e38afc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5e340d0320 0x7f5e340d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.638+0000 7f5e38afc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e340d2060 con 0x7f5e340b6980 2024-04-25T08:39:40.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.638+0000 7f5e38afc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e340d2330 con 0x7f5e340d0700 2024-04-25T08:39:40.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.638+0000 7f5e38afc640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e340d2610 con 0x7f5e340d0320 2024-04-25T08:39:40.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.638+0000 7f5e32d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5e340b6980 0x7f5e340b6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55664/0 (socket says 172.21.15.38:55664) 2024-04-25T08:39:40.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.638+0000 7f5e32d9d640 1 -- 172.21.15.38:0/1538688282 learned_addr learned my addr 172.21.15.38:0/1538688282 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:40.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.638+0000 7f5e3259c640 1 --2- 172.21.15.38:0/1538688282 >> v2:172.21.15.107:3300/0 conn(0x7f5e340d0320 0x7f5e340d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.638+0000 7f5e3159a640 1 -- 172.21.15.38:0/1538688282 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439473692 0 0) 0x7f5e340d2060 con 0x7f5e340b6980 2024-04-25T08:39:40.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.638+0000 7f5e3159a640 1 -- 172.21.15.38:0/1538688282 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e20003520 con 0x7f5e340b6980 2024-04-25T08:39:40.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.639+0000 7f5e3159a640 1 -- 172.21.15.38:0/1538688282 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457173423 0 0) 0x7f5e340d2330 con 0x7f5e340d0700 2024-04-25T08:39:40.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.639+0000 7f5e3159a640 1 -- 172.21.15.38:0/1538688282 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e340d2060 con 0x7f5e340d0700 2024-04-25T08:39:40.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.639+0000 7f5e3159a640 1 -- 172.21.15.38:0/1538688282 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1110549571 0 0) 0x7f5e20003520 con 0x7f5e340b6980 2024-04-25T08:39:40.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.639+0000 7f5e3259c640 1 -- 172.21.15.38:0/1538688282 >> v1:172.21.15.107:6789/0 conn(0x7f5e340d0700 legacy=0x7f5e340d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.639+0000 7f5e3259c640 1 -- 172.21.15.38:0/1538688282 >> v1:172.21.15.38:6789/0 conn(0x7f5e340b6980 legacy=0x7f5e340b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.639+0000 7f5e3259c640 1 -- 172.21.15.38:0/1538688282 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e340d28f0 con 0x7f5e340d0320 2024-04-25T08:39:40.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.639+0000 7f5e3259c640 1 --2- 172.21.15.38:0/1538688282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e340d0320 0x7f5e340d33d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0097a0 tx=0x7f5e2c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c36c1432cbbd3cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:40.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.640+0000 7f5e3159a640 1 -- 172.21.15.38:0/1538688282 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c037070 con 0x7f5e340d0320 2024-04-25T08:39:40.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.640+0000 7f5e3159a640 1 -- 172.21.15.38:0/1538688282 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c02ace0 con 0x7f5e340d0320 2024-04-25T08:39:40.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.640+0000 7f5e3159a640 1 -- 172.21.15.38:0/1538688282 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e240019f0 con 0x7f5e340d0320 2024-04-25T08:39:40.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.640+0000 7f5e38afc640 1 -- 172.21.15.38:0/1538688282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e340d0320 msgr2=0x7f5e340d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.640+0000 7f5e38afc640 1 --2- 172.21.15.38:0/1538688282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e340d0320 0x7f5e340d33d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0097a0 tx=0x7f5e2c02a250 comp rx=0 tx=0).stop 2024-04-25T08:39:40.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.640+0000 7f5e38afc640 1 -- 172.21.15.38:0/1538688282 shutdown_connections 2024-04-25T08:39:40.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.640+0000 7f5e38afc640 1 --2- 172.21.15.38:0/1538688282 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e340d0320 0x7f5e340d33d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.640+0000 7f5e38afc640 1 -- 172.21.15.38:0/1538688282 >> 172.21.15.38:0/1538688282 conn(0x7f5e340668f0 msgr2=0x7f5e34066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:40.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.640+0000 7f5e38afc640 1 -- 172.21.15.38:0/1538688282 shutdown_connections 2024-04-25T08:39:40.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.640+0000 7f5e38afc640 1 -- 172.21.15.38:0/1538688282 wait complete. 2024-04-25T08:39:40.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.642+0000 7f5e38afc640 1 Processor -- start 2024-04-25T08:39:40.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.642+0000 7f5e38afc640 1 -- start start 2024-04-25T08:39:40.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.642+0000 7f5e38afc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e340b6980 0x7f5e3415da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.642+0000 7f5e38afc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e340d0320 0x7f5e340af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.642+0000 7f5e38afc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e340d0700 0x7f5e341f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.642+0000 7f5e38afc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340d0b40 con 0x7f5e340d0700 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.642+0000 7f5e38afc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340d2eb0 con 0x7f5e340b6980 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.642+0000 7f5e38afc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340acc60 con 0x7f5e340d0320 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.642+0000 7f5e32d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e340d0700 0x7f5e341f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.642+0000 7f5e32d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e340d0700 0x7f5e341f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38836/0 (socket says 172.21.15.38:38836) 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.642+0000 7f5e32d9d640 1 -- 172.21.15.38:0/1217884163 learned_addr learned my addr 172.21.15.38:0/1217884163 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e3259c640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e340b6980 0x7f5e3415da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e31d9b640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e340d0320 0x7f5e340af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e32d9d640 1 -- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e340d0320 msgr2=0x7f5e340af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e32d9d640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e340d0320 0x7f5e340af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e32d9d640 1 -- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e340b6980 msgr2=0x7f5e3415da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e32d9d640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e340b6980 0x7f5e3415da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e32d9d640 1 -- 172.21.15.38:0/1217884163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e341f74b0 con 0x7f5e340d0700 2024-04-25T08:39:40.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e32d9d640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e340d0700 0x7f5e341f6f70 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400cc80 tx=0x7f5e2400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:40.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e2affd640 1 -- 172.21.15.38:0/1217884163 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24017070 con 0x7f5e340d0700 2024-04-25T08:39:40.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e2affd640 1 -- 172.21.15.38:0/1217884163 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24012400 con 0x7f5e340d0700 2024-04-25T08:39:40.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e2affd640 1 -- 172.21.15.38:0/1217884163 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e240127f0 con 0x7f5e340d0700 2024-04-25T08:39:40.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e38afc640 1 -- 172.21.15.38:0/1217884163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e340ac930 con 0x7f5e340d0700 2024-04-25T08:39:40.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.643+0000 7f5e38afc640 1 -- 172.21.15.38:0/1217884163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e341f7a10 con 0x7f5e340d0700 2024-04-25T08:39:40.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.644+0000 7f5e38afc640 1 -- 172.21.15.38:0/1217884163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e34068e80 con 0x7f5e340d0700 2024-04-25T08:39:40.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.645+0000 7f5e2affd640 1 -- 172.21.15.38:0/1217884163 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e2401a070 con 0x7f5e340d0700 2024-04-25T08:39:40.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.645+0000 7f5e2affd640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e1406b4d0 0x7f5e1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.646+0000 7f5e2affd640 1 -- 172.21.15.38:0/1217884163 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5e2408a360 con 0x7f5e340d0700 2024-04-25T08:39:40.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.646+0000 7f5e3259c640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e1406b4d0 0x7f5e1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.646+0000 7f5e3259c640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e1406b4d0 0x7f5e1406d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c034000 tx=0x7f5e2c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:40.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.650+0000 7f5e2affd640 1 -- 172.21.15.38:0/1217884163 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e341f74b0 con 0x7f5e340d0700 2024-04-25T08:39:40.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.830+0000 7f5e38afc640 1 -- 172.21.15.38:0/1217884163 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e340b41d0 con 0x7f5e340d0700 2024-04-25T08:39:40.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.830+0000 7f5e2affd640 1 -- 172.21.15.38:0/1217884163 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e24059010 con 0x7f5e340d0700 2024-04-25T08:39:40.824 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:39:40.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.832+0000 7f5e38afc640 1 -- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e1406b4d0 msgr2=0x7f5e1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.832+0000 7f5e38afc640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e1406b4d0 0x7f5e1406d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c034000 tx=0x7f5e2c02e000 comp rx=0 tx=0).stop 2024-04-25T08:39:40.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.832+0000 7f5e38afc640 1 -- 172.21.15.38:0/1217884163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e340d0700 msgr2=0x7f5e341f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.832+0000 7f5e38afc640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e340d0700 0x7f5e341f6f70 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f5e2400cc80 tx=0x7f5e2400a430 comp rx=0 tx=0).stop 2024-04-25T08:39:40.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.832+0000 7f5e38afc640 1 -- 172.21.15.38:0/1217884163 shutdown_connections 2024-04-25T08:39:40.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.832+0000 7f5e38afc640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e340d0700 0x7f5e341f6f70 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.832+0000 7f5e38afc640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5e1406b4d0 0x7f5e1406d990 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.832+0000 7f5e38afc640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5e340d0320 0x7f5e340af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.832+0000 7f5e38afc640 1 --2- 172.21.15.38:0/1217884163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e340b6980 0x7f5e3415da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.832+0000 7f5e38afc640 1 -- 172.21.15.38:0/1217884163 >> 172.21.15.38:0/1217884163 conn(0x7f5e340668f0 msgr2=0x7f5e340cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:40.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.832+0000 7f5e38afc640 1 -- 172.21.15.38:0/1217884163 shutdown_connections 2024-04-25T08:39:40.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.832+0000 7f5e38afc640 1 -- 172.21.15.38:0/1217884163 wait complete. 2024-04-25T08:39:40.836 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:39:40.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.947+0000 7fd391dc1640 1 Processor -- start 2024-04-25T08:39:40.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.947+0000 7fd391dc1640 1 -- start start 2024-04-25T08:39:40.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd391dc1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd38c0d33d0 0x7fd38c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd391dc1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd38c0d1e90 0x7fd38c0b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd391dc1640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c0d2330 con 0x7fd38c0d0320 2024-04-25T08:39:40.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd391dc1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c0d2610 con 0x7fd38c0d1e90 2024-04-25T08:39:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd391dc1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0d28f0 con 0x7fd38c0d33d0 2024-04-25T08:39:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd38bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd38c0d1e90 0x7fd38c0b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd38bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd38c0d1e90 0x7fd38c0b6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38848/0 (socket says 172.21.15.38:38848) 2024-04-25T08:39:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd38bfff640 1 -- 172.21.15.38:0/813860285 learned_addr learned my addr 172.21.15.38:0/813860285 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd38affd640 1 --2- 172.21.15.38:0/813860285 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd38c0d33d0 0x7fd38c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd38bfff640 1 -- 172.21.15.38:0/813860285 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd38c0d33d0 msgr2=0x7fd38c0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd38bfff640 1 --2- 172.21.15.38:0/813860285 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd38c0d33d0 0x7fd38c0d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd38bfff640 1 -- 172.21.15.38:0/813860285 >> v1:172.21.15.107:6789/0 conn(0x7fd38c0d0320 legacy=0x7fd38c0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:40.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.948+0000 7fd38bfff640 1 -- 172.21.15.38:0/813860285 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0d2bd0 con 0x7fd38c0d1e90 2024-04-25T08:39:40.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.949+0000 7fd38bfff640 1 --2- 172.21.15.38:0/813860285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38c0d1e90 0x7fd38c0b6980 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fd384009e50 tx=0x7fd38402a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96ec3a3682496df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.949+0000 7fd38a7fc640 1 -- 172.21.15.38:0/813860285 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384036070 con 0x7fd38c0d1e90 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.949+0000 7fd38a7fc640 1 -- 172.21.15.38:0/813860285 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd384031600 con 0x7fd38c0d1e90 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.949+0000 7fd38a7fc640 1 -- 172.21.15.38:0/813860285 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3840319d0 con 0x7fd38c0d1e90 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.950+0000 7fd391dc1640 1 -- 172.21.15.38:0/813860285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38c0d1e90 msgr2=0x7fd38c0b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.950+0000 7fd391dc1640 1 --2- 172.21.15.38:0/813860285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38c0d1e90 0x7fd38c0b6980 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fd384009e50 tx=0x7fd38402a750 comp rx=0 tx=0).stop 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.950+0000 7fd391dc1640 1 -- 172.21.15.38:0/813860285 shutdown_connections 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.950+0000 7fd391dc1640 1 --2- 172.21.15.38:0/813860285 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38c0d1e90 0x7fd38c0b6980 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.950+0000 7fd391dc1640 1 --2- 172.21.15.38:0/813860285 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd38c0d33d0 0x7fd38c0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.950+0000 7fd391dc1640 1 -- 172.21.15.38:0/813860285 >> 172.21.15.38:0/813860285 conn(0x7fd38c0668f0 msgr2=0x7fd38c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.950+0000 7fd391dc1640 1 -- 172.21.15.38:0/813860285 shutdown_connections 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.950+0000 7fd391dc1640 1 -- 172.21.15.38:0/813860285 wait complete. 2024-04-25T08:39:40.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.951+0000 7fd391dc1640 1 Processor -- start 2024-04-25T08:39:40.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.951+0000 7fd391dc1640 1 -- start start 2024-04-25T08:39:40.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd391dc1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd38c0d0320 0x7fd38c1e7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd391dc1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38c0d1e90 0x7fd38c0b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd391dc1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd38c0d33d0 0x7fd38c0cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd391dc1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c161390 con 0x7fd38c0d1e90 2024-04-25T08:39:40.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd391dc1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c161210 con 0x7fd38c0d0320 2024-04-25T08:39:40.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd391dc1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0b9720 con 0x7fd38c0d33d0 2024-04-25T08:39:40.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd38affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38c0d1e90 0x7fd38c0b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd38affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38c0d1e90 0x7fd38c0b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38860/0 (socket says 172.21.15.38:38860) 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd38affd640 1 -- 172.21.15.38:0/672405804 learned_addr learned my addr 172.21.15.38:0/672405804 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd38bfff640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd38c0d33d0 0x7fd38c0cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd38b7fe640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd38c0d0320 0x7fd38c1e7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd38affd640 1 -- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd38c0d33d0 msgr2=0x7fd38c0cce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd38affd640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd38c0d33d0 0x7fd38c0cce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd38affd640 1 -- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd38c0d0320 msgr2=0x7fd38c1e7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd38affd640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd38c0d0320 0x7fd38c1e7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd38affd640 1 -- 172.21.15.38:0/672405804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c1f46f0 con 0x7fd38c0d1e90 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd38b7fe640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd38c0d0320 0x7fd38c1e7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd38affd640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38c0d1e90 0x7fd38c0b4a00 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd38000d5f0 tx=0x7fd38000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd373fff640 1 -- 172.21.15.38:0/672405804 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800125f0 con 0x7fd38c0d1e90 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd373fff640 1 -- 172.21.15.38:0/672405804 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd380019070 con 0x7fd38c0d1e90 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd391dc1640 1 -- 172.21.15.38:0/672405804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c1f3b70 con 0x7fd38c0d1e90 2024-04-25T08:39:40.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.952+0000 7fd373fff640 1 -- 172.21.15.38:0/672405804 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3800129d0 con 0x7fd38c0d1e90 2024-04-25T08:39:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.953+0000 7fd391dc1640 1 -- 172.21.15.38:0/672405804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c1f4410 con 0x7fd38c0d1e90 2024-04-25T08:39:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.953+0000 7fd391dc1640 1 -- 172.21.15.38:0/672405804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c1f3f60 con 0x7fd38c0d1e90 2024-04-25T08:39:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.954+0000 7fd373fff640 1 -- 172.21.15.38:0/672405804 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd380012b70 con 0x7fd38c0d1e90 2024-04-25T08:39:40.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.954+0000 7fd373fff640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd37806b4d0 0x7fd37806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:40.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.954+0000 7fd373fff640 1 -- 172.21.15.38:0/672405804 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd38008a720 con 0x7fd38c0d1e90 2024-04-25T08:39:40.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.955+0000 7fd38b7fe640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd37806b4d0 0x7fd37806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:40.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.955+0000 7fd38b7fe640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd37806b4d0 0x7fd37806d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd37c003eb0 tx=0x7fd37c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:40.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:40.959+0000 7fd373fff640 1 -- 172.21.15.38:0/672405804 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd380059470 con 0x7fd38c0d1e90 2024-04-25T08:39:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.138+0000 7fd391dc1640 1 -- 172.21.15.38:0/672405804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd38c068e80 con 0x7fd38c0d1e90 2024-04-25T08:39:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.138+0000 7fd373fff640 1 -- 172.21.15.38:0/672405804 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd380058d70 con 0x7fd38c0d1e90 2024-04-25T08:39:41.132 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:39:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.140+0000 7fd391dc1640 1 -- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd37806b4d0 msgr2=0x7fd37806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.140+0000 7fd391dc1640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd37806b4d0 0x7fd37806d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd37c003eb0 tx=0x7fd37c009040 comp rx=0 tx=0).stop 2024-04-25T08:39:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.140+0000 7fd391dc1640 1 -- 172.21.15.38:0/672405804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38c0d1e90 msgr2=0x7fd38c0b4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.140+0000 7fd391dc1640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38c0d1e90 0x7fd38c0b4a00 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd38000d5f0 tx=0x7fd38000dab0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.140+0000 7fd391dc1640 1 -- 172.21.15.38:0/672405804 shutdown_connections 2024-04-25T08:39:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.140+0000 7fd391dc1640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd37806b4d0 0x7fd37806d990 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.140+0000 7fd391dc1640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd38c0d33d0 0x7fd38c0cce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.140+0000 7fd391dc1640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd38c0d1e90 0x7fd38c0b4a00 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.140+0000 7fd391dc1640 1 --2- 172.21.15.38:0/672405804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd38c0d0320 0x7fd38c1e7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.140+0000 7fd391dc1640 1 -- 172.21.15.38:0/672405804 >> 172.21.15.38:0/672405804 conn(0x7fd38c0668f0 msgr2=0x7fd38c160350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.140+0000 7fd391dc1640 1 -- 172.21.15.38:0/672405804 shutdown_connections 2024-04-25T08:39:41.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.140+0000 7fd391dc1640 1 -- 172.21.15.38:0/672405804 wait complete. 2024-04-25T08:39:41.144 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:39:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.254+0000 7fa44aeb3640 1 Processor -- start 2024-04-25T08:39:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.254+0000 7fa44aeb3640 1 -- start start 2024-04-25T08:39:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.255+0000 7fa44aeb3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa44415b990 0x7fa4440cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.255+0000 7fa44aeb3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa4440d1d20 0x7fa4440b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.255+0000 7fa44aeb3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44415fd00 con 0x7fa44415f5f0 2024-04-25T08:39:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.255+0000 7fa44aeb3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa444160580 con 0x7fa4440d1d20 2024-04-25T08:39:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.255+0000 7fa44aeb3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44415fa30 con 0x7fa44415b990 2024-04-25T08:39:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.255+0000 7fa449450640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa4440d1d20 0x7fa4440b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.255+0000 7fa449450640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa4440d1d20 0x7fa4440b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38866/0 (socket says 172.21.15.38:38866) 2024-04-25T08:39:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.255+0000 7fa449450640 1 -- 172.21.15.38:0/1665275711 learned_addr learned my addr 172.21.15.38:0/1665275711 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.256+0000 7fa448c4f640 1 --2- 172.21.15.38:0/1665275711 >> v2:172.21.15.107:3300/0 conn(0x7fa44415b990 0x7fa4440cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.256+0000 7fa449450640 1 -- 172.21.15.38:0/1665275711 >> v2:172.21.15.107:3300/0 conn(0x7fa44415b990 msgr2=0x7fa4440cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.256+0000 7fa449450640 1 --2- 172.21.15.38:0/1665275711 >> v2:172.21.15.107:3300/0 conn(0x7fa44415b990 0x7fa4440cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.256+0000 7fa449450640 1 -- 172.21.15.38:0/1665275711 >> v1:172.21.15.107:6789/0 conn(0x7fa44415f5f0 legacy=0x7fa4440d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.256+0000 7fa449450640 1 -- 172.21.15.38:0/1665275711 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44415ffc0 con 0x7fa4440d1d20 2024-04-25T08:39:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.256+0000 7fa449450640 1 --2- 172.21.15.38:0/1665275711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4440d1d20 0x7fa4440b1fe0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa4440d15e0 tx=0x7fa43c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2beae74a6642ab62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.257+0000 7fa4437fe640 1 -- 172.21.15.38:0/1665275711 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c036070 con 0x7fa4440d1d20 2024-04-25T08:39:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.257+0000 7fa4437fe640 1 -- 172.21.15.38:0/1665275711 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43c031600 con 0x7fa4440d1d20 2024-04-25T08:39:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.257+0000 7fa4437fe640 1 -- 172.21.15.38:0/1665275711 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43c0319d0 con 0x7fa4440d1d20 2024-04-25T08:39:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.257+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1665275711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4440d1d20 msgr2=0x7fa4440b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.257+0000 7fa44aeb3640 1 --2- 172.21.15.38:0/1665275711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4440d1d20 0x7fa4440b1fe0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fa4440d15e0 tx=0x7fa43c02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.257+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1665275711 shutdown_connections 2024-04-25T08:39:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.257+0000 7fa44aeb3640 1 --2- 172.21.15.38:0/1665275711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4440d1d20 0x7fa4440b1fe0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.257+0000 7fa44aeb3640 1 --2- 172.21.15.38:0/1665275711 >> v2:172.21.15.107:3300/0 conn(0x7fa44415b990 0x7fa4440cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.257+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1665275711 >> 172.21.15.38:0/1665275711 conn(0x7fa44408b000 msgr2=0x7fa44408b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.257+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1665275711 shutdown_connections 2024-04-25T08:39:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.257+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1665275711 wait complete. 2024-04-25T08:39:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.258+0000 7fa44aeb3640 1 Processor -- start 2024-04-25T08:39:41.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.258+0000 7fa44aeb3640 1 -- start start 2024-04-25T08:39:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.259+0000 7fa44aeb3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4440d1d20 0x7fa444157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.259+0000 7fa44aeb3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa44415b990 0x7fa4441566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.259+0000 7fa44aeb3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa44415f5f0 0x7fa444156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.259+0000 7fa44aeb3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44415bd70 con 0x7fa4440d1d20 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.259+0000 7fa44aeb3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4440b1ac0 con 0x7fa44415f5f0 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.259+0000 7fa44aeb3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44415bef0 con 0x7fa44415b990 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.259+0000 7fa448c4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4440d1d20 0x7fa444157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.259+0000 7fa448c4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4440d1d20 0x7fa444157d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38878/0 (socket says 172.21.15.38:38878) 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.259+0000 7fa448c4f640 1 -- 172.21.15.38:0/1331624773 learned_addr learned my addr 172.21.15.38:0/1331624773 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa449450640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa44415f5f0 0x7fa444156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa448c4f640 1 -- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa44415b990 msgr2=0x7fa4441566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa443fff640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa44415b990 0x7fa4441566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa448c4f640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa44415b990 0x7fa4441566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa448c4f640 1 -- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa44415f5f0 msgr2=0x7fa444156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa448c4f640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa44415f5f0 0x7fa444156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa448c4f640 1 -- 172.21.15.38:0/1331624773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44415c3f0 con 0x7fa4440d1d20 2024-04-25T08:39:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa449450640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa44415f5f0 0x7fa444156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa448c4f640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4440d1d20 0x7fa444157d90 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fa43400d830 tx=0x7fa43400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa4417fa640 1 -- 172.21.15.38:0/1331624773 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43400b6c0 con 0x7fa4440d1d20 2024-04-25T08:39:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa4417fa640 1 -- 172.21.15.38:0/1331624773 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa434019070 con 0x7fa4440d1d20 2024-04-25T08:39:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa4417fa640 1 -- 172.21.15.38:0/1331624773 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43400bac0 con 0x7fa4440d1d20 2024-04-25T08:39:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1331624773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4440b43d0 con 0x7fa4440d1d20 2024-04-25T08:39:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.260+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1331624773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44415c690 con 0x7fa4440d1d20 2024-04-25T08:39:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.261+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1331624773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4440cefd0 con 0x7fa4440d1d20 2024-04-25T08:39:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.262+0000 7fa4417fa640 1 -- 172.21.15.38:0/1331624773 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa43401e460 con 0x7fa4440d1d20 2024-04-25T08:39:41.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.262+0000 7fa4417fa640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa43006b4d0 0x7fa43006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.262+0000 7fa4417fa640 1 -- 172.21.15.38:0/1331624773 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa43408aa50 con 0x7fa4440d1d20 2024-04-25T08:39:41.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.262+0000 7fa443fff640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa43006b4d0 0x7fa43006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.263+0000 7fa443fff640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa43006b4d0 0x7fa43006d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa444158fd0 tx=0x7fa438009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:41.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.266+0000 7fa4417fa640 1 -- 172.21.15.38:0/1331624773 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4440cefd0 con 0x7fa4440d1d20 2024-04-25T08:39:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.445+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1331624773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa444068e80 con 0x7fa4440d1d20 2024-04-25T08:39:41.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.445+0000 7fa4417fa640 1 -- 172.21.15.38:0/1331624773 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4340597a0 con 0x7fa4440d1d20 2024-04-25T08:39:41.439 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:39:41.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.447+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa43006b4d0 msgr2=0x7fa43006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.447+0000 7fa44aeb3640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa43006b4d0 0x7fa43006d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa444158fd0 tx=0x7fa438009040 comp rx=0 tx=0).stop 2024-04-25T08:39:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.447+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1331624773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4440d1d20 msgr2=0x7fa444157d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.447+0000 7fa44aeb3640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4440d1d20 0x7fa444157d90 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fa43400d830 tx=0x7fa43400dcf0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.447+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1331624773 shutdown_connections 2024-04-25T08:39:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.447+0000 7fa44aeb3640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa44415f5f0 0x7fa444156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.447+0000 7fa44aeb3640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa43006b4d0 0x7fa43006d990 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.447+0000 7fa44aeb3640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa44415b990 0x7fa4441566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.447+0000 7fa44aeb3640 1 --2- 172.21.15.38:0/1331624773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa4440d1d20 0x7fa444157d90 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.447+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1331624773 >> 172.21.15.38:0/1331624773 conn(0x7fa44408b000 msgr2=0x7fa4440ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.447+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1331624773 shutdown_connections 2024-04-25T08:39:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.447+0000 7fa44aeb3640 1 -- 172.21.15.38:0/1331624773 wait complete. 2024-04-25T08:39:41.451 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:39:41.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.560+0000 7f116ef42640 1 Processor -- start 2024-04-25T08:39:41.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.561+0000 7f116ef42640 1 -- start start 2024-04-25T08:39:41.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.561+0000 7f116ef42640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f116815def0 0x7f11680d0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.561+0000 7f116ef42640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f116806c8c0 0x7f1168099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.561+0000 7f116ef42640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1168099fb0 0x7f11680cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.561+0000 7f116ef42640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116815e2d0 con 0x7f1168099fb0 2024-04-25T08:39:41.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.561+0000 7f116ef42640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11680d1250 con 0x7f116815def0 2024-04-25T08:39:41.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.561+0000 7f116ef42640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11680cfc70 con 0x7f116806c8c0 2024-04-25T08:39:41.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.561+0000 7f116d4df640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1168099fb0 0x7f11680cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.561+0000 7f116d4df640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1168099fb0 0x7f11680cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38890/0 (socket says 172.21.15.38:38890) 2024-04-25T08:39:41.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.561+0000 7f116d4df640 1 -- 172.21.15.38:0/1916392681 learned_addr learned my addr 172.21.15.38:0/1916392681 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:41.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.562+0000 7f116d4df640 1 -- 172.21.15.38:0/1916392681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f116806c8c0 msgr2=0x7f1168099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:41.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.562+0000 7f116ccde640 1 --2- 172.21.15.38:0/1916392681 >> v2:172.21.15.107:3300/0 conn(0x7f116815def0 0x7f11680d0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.562+0000 7f1167fff640 1 --2- 172.21.15.38:0/1916392681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f116806c8c0 0x7f1168099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.562+0000 7f116d4df640 1 --2- 172.21.15.38:0/1916392681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f116806c8c0 0x7f1168099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.562+0000 7f116d4df640 1 -- 172.21.15.38:0/1916392681 >> v2:172.21.15.107:3300/0 conn(0x7f116815def0 msgr2=0x7f11680d0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:41.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.562+0000 7f116d4df640 1 --2- 172.21.15.38:0/1916392681 >> v2:172.21.15.107:3300/0 conn(0x7f116815def0 0x7f11680d0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.562+0000 7f116d4df640 1 -- 172.21.15.38:0/1916392681 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11680cff50 con 0x7f1168099fb0 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.562+0000 7f1167fff640 1 --2- 172.21.15.38:0/1916392681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f116806c8c0 0x7f1168099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f116d4df640 1 --2- 172.21.15.38:0/1916392681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1168099fb0 0x7f11680cbc50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f1168072ae0 tx=0x7f115802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8ada08f9b7678b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f11677fe640 1 -- 172.21.15.38:0/1916392681 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158037070 con 0x7f1168099fb0 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f11677fe640 1 -- 172.21.15.38:0/1916392681 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1158032440 con 0x7f1168099fb0 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f11677fe640 1 -- 172.21.15.38:0/1916392681 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158032810 con 0x7f1168099fb0 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f116ef42640 1 -- 172.21.15.38:0/1916392681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1168099fb0 msgr2=0x7f11680cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f116ef42640 1 --2- 172.21.15.38:0/1916392681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1168099fb0 0x7f11680cbc50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f1168072ae0 tx=0x7f115802a200 comp rx=0 tx=0).stop 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f116ef42640 1 -- 172.21.15.38:0/1916392681 shutdown_connections 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f116ef42640 1 --2- 172.21.15.38:0/1916392681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1168099fb0 0x7f11680cbc50 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f116ef42640 1 --2- 172.21.15.38:0/1916392681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f116806c8c0 0x7f1168099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f116ef42640 1 --2- 172.21.15.38:0/1916392681 >> v2:172.21.15.107:3300/0 conn(0x7f116815def0 0x7f11680d0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f116ef42640 1 -- 172.21.15.38:0/1916392681 >> 172.21.15.38:0/1916392681 conn(0x7f11680b30f0 msgr2=0x7f11680b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:41.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f116ef42640 1 -- 172.21.15.38:0/1916392681 shutdown_connections 2024-04-25T08:39:41.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.563+0000 7f116ef42640 1 -- 172.21.15.38:0/1916392681 wait complete. 2024-04-25T08:39:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.565+0000 7f116ef42640 1 Processor -- start 2024-04-25T08:39:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.565+0000 7f116ef42640 1 -- start start 2024-04-25T08:39:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.565+0000 7f116ef42640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f116806c8c0 0x7f1168158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f116ef42640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1168099fb0 0x7f11680ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f116ef42640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f116815def0 0x7f11680cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f116ef42640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11680d07f0 con 0x7f1168099fb0 2024-04-25T08:39:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f116ef42640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11680d0510 con 0x7f116806c8c0 2024-04-25T08:39:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f116ef42640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11680cf8d0 con 0x7f116815def0 2024-04-25T08:39:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f1167fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1168099fb0 0x7f11680ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f1167fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1168099fb0 0x7f11680ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38894/0 (socket says 172.21.15.38:38894) 2024-04-25T08:39:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f1167fff640 1 -- 172.21.15.38:0/3581998188 learned_addr learned my addr 172.21.15.38:0/3581998188 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f116d4df640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f116815def0 0x7f11680cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f116ccde640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f116806c8c0 0x7f1168158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f1167fff640 1 -- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f116815def0 msgr2=0x7f11680cade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f1167fff640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f116815def0 0x7f11680cade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f1167fff640 1 -- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f116806c8c0 msgr2=0x7f1168158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f1167fff640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f116806c8c0 0x7f1168158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f1167fff640 1 -- 172.21.15.38:0/3581998188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11681f7600 con 0x7f1168099fb0 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f116ccde640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f116806c8c0 0x7f1168158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f1167fff640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1168099fb0 0x7f11680ca8a0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f115c00c7f0 tx=0x7f115c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f11657fa640 1 -- 172.21.15.38:0/3581998188 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c015070 con 0x7f1168099fb0 2024-04-25T08:39:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f11657fa640 1 -- 172.21.15.38:0/3581998188 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c007a30 con 0x7f1168099fb0 2024-04-25T08:39:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f11657fa640 1 -- 172.21.15.38:0/3581998188 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c007e00 con 0x7f1168099fb0 2024-04-25T08:39:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f116ef42640 1 -- 172.21.15.38:0/3581998188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11681f78e0 con 0x7f1168099fb0 2024-04-25T08:39:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.566+0000 7f116ef42640 1 -- 172.21.15.38:0/3581998188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11681f7ea0 con 0x7f1168099fb0 2024-04-25T08:39:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.567+0000 7f116ef42640 1 -- 172.21.15.38:0/3581998188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116806d930 con 0x7f1168099fb0 2024-04-25T08:39:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.568+0000 7f11657fa640 1 -- 172.21.15.38:0/3581998188 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f115c018070 con 0x7f1168099fb0 2024-04-25T08:39:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.568+0000 7f11657fa640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f115406b500 0x7f115406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.568+0000 7f11657fa640 1 -- 172.21.15.38:0/3581998188 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f115c08a160 con 0x7f1168099fb0 2024-04-25T08:39:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.568+0000 7f116ccde640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f115406b500 0x7f115406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.569+0000 7f116ccde640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f115406b500 0x7f115406d9c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1160007fc0 tx=0x7f11600075b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.572+0000 7f11657fa640 1 -- 172.21.15.38:0/3581998188 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f116806d930 con 0x7f1168099fb0 2024-04-25T08:39:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.768+0000 7f116ef42640 1 -- 172.21.15.38:0/3581998188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f116804ef90 con 0x7f1168099fb0 2024-04-25T08:39:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.769+0000 7f11657fa640 1 -- 172.21.15.38:0/3581998188 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7f115c058860 con 0x7f1168099fb0 2024-04-25T08:39:41.762 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:39:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.771+0000 7f116ef42640 1 -- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f115406b500 msgr2=0x7f115406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.771+0000 7f116ef42640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f115406b500 0x7f115406d9c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1160007fc0 tx=0x7f11600075b0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.771+0000 7f116ef42640 1 -- 172.21.15.38:0/3581998188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1168099fb0 msgr2=0x7f11680ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.771+0000 7f116ef42640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1168099fb0 0x7f11680ca8a0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f115c00c7f0 tx=0x7f115c00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.771+0000 7f116ef42640 1 -- 172.21.15.38:0/3581998188 shutdown_connections 2024-04-25T08:39:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.771+0000 7f116ef42640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f115406b500 0x7f115406d9c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.771+0000 7f116ef42640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f116815def0 0x7f11680cade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.771+0000 7f116ef42640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1168099fb0 0x7f11680ca8a0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.771+0000 7f116ef42640 1 --2- 172.21.15.38:0/3581998188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f116806c8c0 0x7f1168158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.771+0000 7f116ef42640 1 -- 172.21.15.38:0/3581998188 >> 172.21.15.38:0/3581998188 conn(0x7f11680b30f0 msgr2=0x7f11680c72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.771+0000 7f116ef42640 1 -- 172.21.15.38:0/3581998188 shutdown_connections 2024-04-25T08:39:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:41.771+0000 7f116ef42640 1 -- 172.21.15.38:0/3581998188 wait complete. 2024-04-25T08:39:41.775 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:39:41.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:41.775 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:39:41.798 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[83796]: fuse finished with error 0 and tester_r 0 2024-04-25T08:39:41.870 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:39:41.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:41.871 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:39:41.974 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:39:41.974 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:41.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:41.991 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:39:41.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:41.992 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:39:42.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:39:42.048 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:39:42.049 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:39:42.049 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:42.065 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:39:42.065 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:39:42.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:42.065 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:39:42.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:39:42.120 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:39:42.120 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:39:42.120 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:42.136 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:42.136 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:39:42.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:39:42.136 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:39:42.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:39:42.192 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:39:42.192 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:42.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.300+0000 7f52662ac640 1 Processor -- start 2024-04-25T08:39:42.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.300+0000 7f52662ac640 1 -- start start 2024-04-25T08:39:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.301+0000 7f52662ac640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f52600cf470 0x7f5260168720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.301+0000 7f52662ac640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52600ccd60 con 0x7f5260099a70 2024-04-25T08:39:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.301+0000 7f52662ac640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52600cd040 con 0x7f526006cd90 2024-04-25T08:39:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.301+0000 7f52662ac640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52601614d0 con 0x7f52600cf470 2024-04-25T08:39:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.301+0000 7f5264849640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f52600cf470 0x7f5260168720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.301+0000 7f525ffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f5260099a70 0x7f5260099e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55670/0 (socket says 172.21.15.38:55670) 2024-04-25T08:39:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.301+0000 7f525ffff640 1 -- 172.21.15.38:0/771244492 learned_addr learned my addr 172.21.15.38:0/771244492 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.302+0000 7f5264849640 1 -- 172.21.15.38:0/771244492 >> v1:172.21.15.107:6789/0 conn(0x7f526006cd90 legacy=0x7f52600cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:42.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.302+0000 7f5264849640 1 -- 172.21.15.38:0/771244492 >> v1:172.21.15.38:6789/0 conn(0x7f5260099a70 legacy=0x7f5260099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.302+0000 7f525effd640 1 -- 172.21.15.38:0/771244492 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223177098 0 0) 0x7f52600ccd60 con 0x7f5260099a70 2024-04-25T08:39:42.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.302+0000 7f5264849640 1 -- 172.21.15.38:0/771244492 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52600cd320 con 0x7f52600cf470 2024-04-25T08:39:42.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.302+0000 7f5264849640 1 --2- 172.21.15.38:0/771244492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52600cf470 0x7f5260168720 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5258009d90 tx=0x7f525802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a1742e4920d0d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:42.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.302+0000 7f525effd640 1 -- 172.21.15.38:0/771244492 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5258007740 con 0x7f52600cf470 2024-04-25T08:39:42.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.302+0000 7f525effd640 1 -- 172.21.15.38:0/771244492 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52580078e0 con 0x7f52600cf470 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.303+0000 7f525effd640 1 -- 172.21.15.38:0/771244492 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5258007cb0 con 0x7f52600cf470 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.303+0000 7f52662ac640 1 -- 172.21.15.38:0/771244492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52600cf470 msgr2=0x7f5260168720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.303+0000 7f52662ac640 1 --2- 172.21.15.38:0/771244492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52600cf470 0x7f5260168720 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5258009d90 tx=0x7f525802a6f0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.303+0000 7f52662ac640 1 -- 172.21.15.38:0/771244492 shutdown_connections 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.303+0000 7f52662ac640 1 --2- 172.21.15.38:0/771244492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52600cf470 0x7f5260168720 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.303+0000 7f52662ac640 1 -- 172.21.15.38:0/771244492 >> 172.21.15.38:0/771244492 conn(0x7f52600b3180 msgr2=0x7f52600b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.303+0000 7f52662ac640 1 -- 172.21.15.38:0/771244492 shutdown_connections 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.303+0000 7f52662ac640 1 -- 172.21.15.38:0/771244492 wait complete. 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.304+0000 7f52662ac640 1 Processor -- start 2024-04-25T08:39:42.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.304+0000 7f52662ac640 1 -- start start 2024-04-25T08:39:42.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.305+0000 7f52662ac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f526006cd90 0x7f52601f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.305+0000 7f52662ac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5260099a70 0x7f52600ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.305+0000 7f52662ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52600cf470 0x7f52600ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.305+0000 7f52662ac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52600d0780 con 0x7f526006cd90 2024-04-25T08:39:42.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.305+0000 7f52662ac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52600cd970 con 0x7f5260099a70 2024-04-25T08:39:42.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.305+0000 7f52662ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52600d0900 con 0x7f52600cf470 2024-04-25T08:39:42.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.305+0000 7f525ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f526006cd90 0x7f52601f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.305+0000 7f525ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f526006cd90 0x7f52601f2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38922/0 (socket says 172.21.15.38:38922) 2024-04-25T08:39:42.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.305+0000 7f525ffff640 1 -- 172.21.15.38:0/2294278829 learned_addr learned my addr 172.21.15.38:0/2294278829 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:42.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.305+0000 7f5264849640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52600cf470 0x7f52600ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f525f7fe640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5260099a70 0x7f52600ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f525ffff640 1 -- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52600cf470 msgr2=0x7f52600ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f525ffff640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52600cf470 0x7f52600ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f525ffff640 1 -- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5260099a70 msgr2=0x7f52600ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f525ffff640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5260099a70 0x7f52600ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f525ffff640 1 -- 172.21.15.38:0/2294278829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52600c5c40 con 0x7f526006cd90 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f525f7fe640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5260099a70 0x7f52600ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f5264849640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52600cf470 0x7f52600ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f525ffff640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f526006cd90 0x7f52601f2c10 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f525000c4a0 tx=0x7f525000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f525cff9640 1 -- 172.21.15.38:0/2294278829 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250018070 con 0x7f526006cd90 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f525cff9640 1 -- 172.21.15.38:0/2294278829 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52500074e0 con 0x7f526006cd90 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f52662ac640 1 -- 172.21.15.38:0/2294278829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f526015e860 con 0x7f526006cd90 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f525cff9640 1 -- 172.21.15.38:0/2294278829 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5250004a20 con 0x7f526006cd90 2024-04-25T08:39:42.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.306+0000 7f52662ac640 1 -- 172.21.15.38:0/2294278829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52600c5960 con 0x7f526006cd90 2024-04-25T08:39:42.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.307+0000 7f52662ac640 1 -- 172.21.15.38:0/2294278829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5260068e80 con 0x7f526006cd90 2024-04-25T08:39:42.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.308+0000 7f525cff9640 1 -- 172.21.15.38:0/2294278829 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f525001b070 con 0x7f526006cd90 2024-04-25T08:39:42.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.308+0000 7f525cff9640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f524006b4d0 0x7f524006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.308+0000 7f525cff9640 1 -- 172.21.15.38:0/2294278829 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f52500896f0 con 0x7f526006cd90 2024-04-25T08:39:42.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.308+0000 7f525f7fe640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f524006b4d0 0x7f524006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.309+0000 7f525f7fe640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f524006b4d0 0x7f524006d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5248007fd0 tx=0x7f5248009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:42.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.312+0000 7f525cff9640 1 -- 172.21.15.38:0/2294278829 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f52500583b0 con 0x7f526006cd90 2024-04-25T08:39:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.529+0000 7f52662ac640 1 -- 172.21.15.38:0/2294278829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5260151860 con 0x7f526006cd90 2024-04-25T08:39:42.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.529+0000 7f525cff9640 1 -- 172.21.15.38:0/2294278829 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f5250057cb0 con 0x7f526006cd90 2024-04-25T08:39:42.523 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:42.523 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:07.551641+0000","modified":"2024-04-25T08:39:09.555958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8292":{"gid":8292,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1263010516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1263010516},{"type":"v1","addr":"172.21.15.107:6837","nonce":1263010516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:39:42.523 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 63 2024-04-25T08:39:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.531+0000 7f52662ac640 1 -- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f524006b4d0 msgr2=0x7f524006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.531+0000 7f52662ac640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f524006b4d0 0x7f524006d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5248007fd0 tx=0x7f5248009040 comp rx=0 tx=0).stop 2024-04-25T08:39:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.531+0000 7f52662ac640 1 -- 172.21.15.38:0/2294278829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f526006cd90 msgr2=0x7f52601f2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.531+0000 7f52662ac640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f526006cd90 0x7f52601f2c10 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f525000c4a0 tx=0x7f525000c960 comp rx=0 tx=0).stop 2024-04-25T08:39:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.531+0000 7f52662ac640 1 -- 172.21.15.38:0/2294278829 shutdown_connections 2024-04-25T08:39:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.531+0000 7f52662ac640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f524006b4d0 0x7f524006d990 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.531+0000 7f52662ac640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52600cf470 0x7f52600ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.531+0000 7f52662ac640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5260099a70 0x7f52600ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.531+0000 7f52662ac640 1 --2- 172.21.15.38:0/2294278829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f526006cd90 0x7f52601f2c10 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.531+0000 7f52662ac640 1 -- 172.21.15.38:0/2294278829 >> 172.21.15.38:0/2294278829 conn(0x7f52600b3180 msgr2=0x7f5260160ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:42.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.531+0000 7f52662ac640 1 -- 172.21.15.38:0/2294278829 shutdown_connections 2024-04-25T08:39:42.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.531+0000 7f52662ac640 1 -- 172.21.15.38:0/2294278829 wait complete. 2024-04-25T08:39:42.536 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.644+0000 7f8d64134640 1 Processor -- start 2024-04-25T08:39:42.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.644+0000 7f8d64134640 1 -- start start 2024-04-25T08:39:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.645+0000 7f8d64134640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8d5c06c110 0x7f8d5c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.645+0000 7f8d64134640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5c06ca30 con 0x7f8d5c0ce2e0 2024-04-25T08:39:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.645+0000 7f8d64134640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d5c0ceff0 con 0x7f8d5c15df50 2024-04-25T08:39:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.645+0000 7f8d64134640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d5c0cf2d0 con 0x7f8d5c06c110 2024-04-25T08:39:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.645+0000 7f8d61ed0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8d5c06c110 0x7f8d5c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.645+0000 7f8d626d1640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f8d5c0ce2e0 0x7f8d5c0d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55686/0 (socket says 172.21.15.38:55686) 2024-04-25T08:39:42.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.645+0000 7f8d626d1640 1 -- 172.21.15.38:0/2390025004 learned_addr learned my addr 172.21.15.38:0/2390025004 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:42.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.645+0000 7f8d61ed0640 1 -- 172.21.15.38:0/2390025004 >> v1:172.21.15.107:6789/0 conn(0x7f8d5c15df50 legacy=0x7f8d5c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:42.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.645+0000 7f8d61ed0640 1 -- 172.21.15.38:0/2390025004 >> v1:172.21.15.38:6789/0 conn(0x7f8d5c0ce2e0 legacy=0x7f8d5c0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.645+0000 7f8d61ed0640 1 -- 172.21.15.38:0/2390025004 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c0d1f70 con 0x7f8d5c06c110 2024-04-25T08:39:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.646+0000 7f8d61ed0640 1 --2- 172.21.15.38:0/2390025004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d5c06c110 0x7f8d5c06c4f0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f8d580097a0 tx=0x7f8d5802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e098528e5e4a3f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.646+0000 7f8d60ece640 1 -- 172.21.15.38:0/2390025004 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58037070 con 0x7f8d5c06c110 2024-04-25T08:39:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.646+0000 7f8d60ece640 1 -- 172.21.15.38:0/2390025004 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5802ae50 con 0x7f8d5c06c110 2024-04-25T08:39:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.646+0000 7f8d60ece640 1 -- 172.21.15.38:0/2390025004 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58032660 con 0x7f8d5c06c110 2024-04-25T08:39:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.646+0000 7f8d64134640 1 -- 172.21.15.38:0/2390025004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d5c06c110 msgr2=0x7f8d5c06c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.646+0000 7f8d64134640 1 --2- 172.21.15.38:0/2390025004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d5c06c110 0x7f8d5c06c4f0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f8d580097a0 tx=0x7f8d5802a140 comp rx=0 tx=0).stop 2024-04-25T08:39:42.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.647+0000 7f8d64134640 1 -- 172.21.15.38:0/2390025004 shutdown_connections 2024-04-25T08:39:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.647+0000 7f8d64134640 1 --2- 172.21.15.38:0/2390025004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d5c06c110 0x7f8d5c06c4f0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.647+0000 7f8d64134640 1 -- 172.21.15.38:0/2390025004 >> 172.21.15.38:0/2390025004 conn(0x7f8d5c0d3fb0 msgr2=0x7f8d5c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.647+0000 7f8d64134640 1 -- 172.21.15.38:0/2390025004 shutdown_connections 2024-04-25T08:39:42.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.647+0000 7f8d64134640 1 -- 172.21.15.38:0/2390025004 wait complete. 2024-04-25T08:39:42.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.648+0000 7f8d64134640 1 Processor -- start 2024-04-25T08:39:42.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.648+0000 7f8d64134640 1 -- start start 2024-04-25T08:39:42.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d64134640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d5c06c110 0x7f8d5c0c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d64134640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d5c0ce2e0 0x7f8d5c14ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d64134640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d5c15df50 0x7f8d5c14f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d64134640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0d3160 con 0x7f8d5c15df50 2024-04-25T08:39:42.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d64134640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0d2530 con 0x7f8d5c06c110 2024-04-25T08:39:42.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d64134640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5c0d32e0 con 0x7f8d5c0ce2e0 2024-04-25T08:39:42.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d626d1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d5c15df50 0x7f8d5c14f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d626d1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d5c15df50 0x7f8d5c14f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38936/0 (socket says 172.21.15.38:38936) 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d626d1640 1 -- 172.21.15.38:0/4154929066 learned_addr learned my addr 172.21.15.38:0/4154929066 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d61ed0640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d5c06c110 0x7f8d5c0c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d616cf640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d5c0ce2e0 0x7f8d5c14ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d626d1640 1 -- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d5c0ce2e0 msgr2=0x7f8d5c14ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d626d1640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d5c0ce2e0 0x7f8d5c14ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d626d1640 1 -- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d5c06c110 msgr2=0x7f8d5c0c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d626d1640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d5c06c110 0x7f8d5c0c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d626d1640 1 -- 172.21.15.38:0/4154929066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c0c7630 con 0x7f8d5c15df50 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d616cf640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d5c0ce2e0 0x7f8d5c14ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d626d1640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d5c15df50 0x7f8d5c14f530 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400cbc0 tx=0x7f8d5400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d527fc640 1 -- 172.21.15.38:0/4154929066 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54017070 con 0x7f8d5c15df50 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d527fc640 1 -- 172.21.15.38:0/4154929066 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5400ad40 con 0x7f8d5c15df50 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d64134640 1 -- 172.21.15.38:0/4154929066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d5c0c7910 con 0x7f8d5c15df50 2024-04-25T08:39:42.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.649+0000 7f8d527fc640 1 -- 172.21.15.38:0/4154929066 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d54012600 con 0x7f8d5c15df50 2024-04-25T08:39:42.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.650+0000 7f8d64134640 1 -- 172.21.15.38:0/4154929066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d5c0c8930 con 0x7f8d5c15df50 2024-04-25T08:39:42.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.650+0000 7f8d64134640 1 -- 172.21.15.38:0/4154929066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d5c068e80 con 0x7f8d5c15df50 2024-04-25T08:39:42.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.651+0000 7f8d527fc640 1 -- 172.21.15.38:0/4154929066 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d5401a070 con 0x7f8d5c15df50 2024-04-25T08:39:42.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.651+0000 7f8d527fc640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8d4006b4d0 0x7f8d4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.651+0000 7f8d527fc640 1 -- 172.21.15.38:0/4154929066 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f8d54089ff0 con 0x7f8d5c15df50 2024-04-25T08:39:42.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.652+0000 7f8d61ed0640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8d4006b4d0 0x7f8d4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.652+0000 7f8d61ed0640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8d4006b4d0 0x7f8d4006d990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8d58034000 tx=0x7f8d5802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:42.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.656+0000 7f8d527fc640 1 -- 172.21.15.38:0/4154929066 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d540586f0 con 0x7f8d5c15df50 2024-04-25T08:39:42.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.836+0000 7f8d64134640 1 -- 172.21.15.38:0/4154929066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d5c15c940 con 0x7f8d5c15df50 2024-04-25T08:39:42.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.837+0000 7f8d527fc640 1 -- 172.21.15.38:0/4154929066 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16970 (secure 0 0 0) 0x7f8d5405c790 con 0x7f8d5c15df50 2024-04-25T08:39:42.830 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:42.831 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":110,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:18.940299+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:05.852531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:06.527775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":1,"to":7}]},{"pgid":"16.1e","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:42.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.839+0000 7f8d64134640 1 -- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8d4006b4d0 msgr2=0x7f8d4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.839+0000 7f8d64134640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8d4006b4d0 0x7f8d4006d990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f8d58034000 tx=0x7f8d5802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:42.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.839+0000 7f8d64134640 1 -- 172.21.15.38:0/4154929066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d5c15df50 msgr2=0x7f8d5c14f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.839+0000 7f8d64134640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d5c15df50 0x7f8d5c14f530 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f8d5400cbc0 tx=0x7f8d5400a430 comp rx=0 tx=0).stop 2024-04-25T08:39:42.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.839+0000 7f8d64134640 1 -- 172.21.15.38:0/4154929066 shutdown_connections 2024-04-25T08:39:42.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.839+0000 7f8d64134640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8d5c15df50 0x7f8d5c14f530 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.839+0000 7f8d64134640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8d4006b4d0 0x7f8d4006d990 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.839+0000 7f8d64134640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8d5c0ce2e0 0x7f8d5c14ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.839+0000 7f8d64134640 1 --2- 172.21.15.38:0/4154929066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d5c06c110 0x7f8d5c0c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.839+0000 7f8d64134640 1 -- 172.21.15.38:0/4154929066 >> 172.21.15.38:0/4154929066 conn(0x7f8d5c0d3fb0 msgr2=0x7f8d5c0d2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.839+0000 7f8d64134640 1 -- 172.21.15.38:0/4154929066 shutdown_connections 2024-04-25T08:39:42.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.839+0000 7f8d64134640 1 -- 172.21.15.38:0/4154929066 wait complete. 2024-04-25T08:39:42.844 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:39:05.852531+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:39:06.527775+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:39:42.844 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:39:42.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.955+0000 7f47aa25a640 1 Processor -- start 2024-04-25T08:39:42.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.955+0000 7f47aa25a640 1 -- start start 2024-04-25T08:39:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47aa25a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f47a40d0320 0x7f47a40d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47aa25a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47a40b6980 0x7f47a40b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47aa25a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a40d2060 con 0x7f47a40d0700 2024-04-25T08:39:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47aa25a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a40d2330 con 0x7f47a40b6980 2024-04-25T08:39:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47aa25a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a40d2610 con 0x7f47a40d0320 2024-04-25T08:39:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47a3fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47a40b6980 0x7f47a40b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47a3fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47a40b6980 0x7f47a40b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38942/0 (socket says 172.21.15.38:38942) 2024-04-25T08:39:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47a3fff640 1 -- 172.21.15.38:0/26424959 learned_addr learned my addr 172.21.15.38:0/26424959 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47a37fe640 1 --2- 172.21.15.38:0/26424959 >> v2:172.21.15.107:3300/0 conn(0x7f47a40d0320 0x7f47a40d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47a3fff640 1 -- 172.21.15.38:0/26424959 >> v2:172.21.15.107:3300/0 conn(0x7f47a40d0320 msgr2=0x7f47a40d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47a3fff640 1 --2- 172.21.15.38:0/26424959 >> v2:172.21.15.107:3300/0 conn(0x7f47a40d0320 0x7f47a40d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47a3fff640 1 -- 172.21.15.38:0/26424959 >> v1:172.21.15.107:6789/0 conn(0x7f47a40d0700 legacy=0x7f47a40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:42.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.956+0000 7f47a3fff640 1 -- 172.21.15.38:0/26424959 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a40d28f0 con 0x7f47a40b6980 2024-04-25T08:39:42.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.957+0000 7f47a3fff640 1 --2- 172.21.15.38:0/26424959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a40b6980 0x7f47a40b6dd0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f47a4069100 tx=0x7f479c02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b47796b75cc7fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:42.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.957+0000 7f47a27fc640 1 -- 172.21.15.38:0/26424959 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c036070 con 0x7f47a40b6980 2024-04-25T08:39:42.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.957+0000 7f47a27fc640 1 -- 172.21.15.38:0/26424959 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c031600 con 0x7f47a40b6980 2024-04-25T08:39:42.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.957+0000 7f47a27fc640 1 -- 172.21.15.38:0/26424959 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c0319d0 con 0x7f47a40b6980 2024-04-25T08:39:42.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.958+0000 7f47aa25a640 1 -- 172.21.15.38:0/26424959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a40b6980 msgr2=0x7f47a40b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.958+0000 7f47aa25a640 1 --2- 172.21.15.38:0/26424959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a40b6980 0x7f47a40b6dd0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f47a4069100 tx=0x7f479c02a780 comp rx=0 tx=0).stop 2024-04-25T08:39:42.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.958+0000 7f47aa25a640 1 -- 172.21.15.38:0/26424959 shutdown_connections 2024-04-25T08:39:42.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.958+0000 7f47aa25a640 1 --2- 172.21.15.38:0/26424959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a40b6980 0x7f47a40b6dd0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.958+0000 7f47aa25a640 1 --2- 172.21.15.38:0/26424959 >> v2:172.21.15.107:3300/0 conn(0x7f47a40d0320 0x7f47a40d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.958+0000 7f47aa25a640 1 -- 172.21.15.38:0/26424959 >> 172.21.15.38:0/26424959 conn(0x7f47a40668f0 msgr2=0x7f47a4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:42.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.958+0000 7f47aa25a640 1 -- 172.21.15.38:0/26424959 shutdown_connections 2024-04-25T08:39:42.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.958+0000 7f47aa25a640 1 -- 172.21.15.38:0/26424959 wait complete. 2024-04-25T08:39:42.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.959+0000 7f47aa25a640 1 Processor -- start 2024-04-25T08:39:42.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.959+0000 7f47aa25a640 1 -- start start 2024-04-25T08:39:42.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47aa25a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47a40b6980 0x7f47a4149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47aa25a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47a40d0320 0x7f47a4145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47aa25a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a40d0700 0x7f47a40b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47aa25a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a40d0b40 con 0x7f47a40d0700 2024-04-25T08:39:42.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47aa25a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a40d2eb0 con 0x7f47a40d0320 2024-04-25T08:39:42.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47aa25a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a40b59a0 con 0x7f47a40b6980 2024-04-25T08:39:42.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47a3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a40d0700 0x7f47a40b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47a3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a40d0700 0x7f47a40b4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38954/0 (socket says 172.21.15.38:38954) 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47a3fff640 1 -- 172.21.15.38:0/2320809271 learned_addr learned my addr 172.21.15.38:0/2320809271 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47a2ffd640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47a40d0320 0x7f47a4145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47a37fe640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47a40b6980 0x7f47a4149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47a3fff640 1 -- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47a40b6980 msgr2=0x7f47a4149040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47a3fff640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47a40b6980 0x7f47a4149040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47a3fff640 1 -- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47a40d0320 msgr2=0x7f47a4145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47a3fff640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47a40d0320 0x7f47a4145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47a3fff640 1 -- 172.21.15.38:0/2320809271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a40b4d90 con 0x7f47a40d0700 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47a2ffd640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47a40d0320 0x7f47a4145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.960+0000 7f47a3fff640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a40d0700 0x7f47a40b4540 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f479c009f40 tx=0x7f479c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.961+0000 7f478bfff640 1 -- 172.21.15.38:0/2320809271 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c036050 con 0x7f47a40d0700 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.961+0000 7f478bfff640 1 -- 172.21.15.38:0/2320809271 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c042070 con 0x7f47a40d0700 2024-04-25T08:39:42.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.961+0000 7f47aa25a640 1 -- 172.21.15.38:0/2320809271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a40b5070 con 0x7f47a40d0700 2024-04-25T08:39:42.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.961+0000 7f478bfff640 1 -- 172.21.15.38:0/2320809271 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c009510 con 0x7f47a40d0700 2024-04-25T08:39:42.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.961+0000 7f47aa25a640 1 -- 172.21.15.38:0/2320809271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a40b5630 con 0x7f47a40d0700 2024-04-25T08:39:42.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.962+0000 7f47aa25a640 1 -- 172.21.15.38:0/2320809271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a4161210 con 0x7f47a40d0700 2024-04-25T08:39:42.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.962+0000 7f478bfff640 1 -- 172.21.15.38:0/2320809271 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f479c007bd0 con 0x7f47a40d0700 2024-04-25T08:39:42.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.962+0000 7f478bfff640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f479006b500 0x7f479006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:42.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.963+0000 7f478bfff640 1 -- 172.21.15.38:0/2320809271 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f479c0aabe0 con 0x7f47a40d0700 2024-04-25T08:39:42.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.963+0000 7f47a37fe640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f479006b500 0x7f479006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:42.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.963+0000 7f47a37fe640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f479006b500 0x7f479006d9c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4794006fd0 tx=0x7f4794008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:42.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:42.967+0000 7f478bfff640 1 -- 172.21.15.38:0/2320809271 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47a4161210 con 0x7f47a40d0700 2024-04-25T08:39:43.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.145+0000 7f47aa25a640 1 -- 172.21.15.38:0/2320809271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47a4068e80 con 0x7f47a40d0700 2024-04-25T08:39:43.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.146+0000 7f478bfff640 1 -- 172.21.15.38:0/2320809271 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f479c04b020 con 0x7f47a40d0700 2024-04-25T08:39:43.139 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.147+0000 7f47aa25a640 1 -- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f479006b500 msgr2=0x7f479006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.147+0000 7f47aa25a640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f479006b500 0x7f479006d9c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4794006fd0 tx=0x7f4794008040 comp rx=0 tx=0).stop 2024-04-25T08:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.147+0000 7f47aa25a640 1 -- 172.21.15.38:0/2320809271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a40d0700 msgr2=0x7f47a40b4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.147+0000 7f47aa25a640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a40d0700 0x7f47a40b4540 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f479c009f40 tx=0x7f479c004a40 comp rx=0 tx=0).stop 2024-04-25T08:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.147+0000 7f47aa25a640 1 -- 172.21.15.38:0/2320809271 shutdown_connections 2024-04-25T08:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.147+0000 7f47aa25a640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47a40d0700 0x7f47a40b4540 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.147+0000 7f47aa25a640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47a40d0320 0x7f47a4145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.147+0000 7f47aa25a640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f479006b500 0x7f479006d9c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.147+0000 7f47aa25a640 1 --2- 172.21.15.38:0/2320809271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47a40b6980 0x7f47a4149040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.147+0000 7f47aa25a640 1 -- 172.21.15.38:0/2320809271 >> 172.21.15.38:0/2320809271 conn(0x7f47a40668f0 msgr2=0x7f47a40ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:43.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.148+0000 7f47aa25a640 1 -- 172.21.15.38:0/2320809271 shutdown_connections 2024-04-25T08:39:43.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.148+0000 7f47aa25a640 1 -- 172.21.15.38:0/2320809271 wait complete. 2024-04-25T08:39:43.152 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:39:43.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.263+0000 7f73331f5640 1 Processor -- start 2024-04-25T08:39:43.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.263+0000 7f73331f5640 1 -- start start 2024-04-25T08:39:43.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.263+0000 7f73331f5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f732c0d0320 0x7f732c0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.263+0000 7f73331f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f732c0d0700 0x7f732c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.263+0000 7f73331f5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732c0d2330 con 0x7f732c0d1e90 2024-04-25T08:39:43.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.263+0000 7f73331f5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732c0d2610 con 0x7f732c0d0320 2024-04-25T08:39:43.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.263+0000 7f73331f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0d28f0 con 0x7f732c0d0700 2024-04-25T08:39:43.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.264+0000 7f7330f91640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f732c0d0320 0x7f732c0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.264+0000 7f7331792640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f732c0d1e90 0x7f732c0b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55696/0 (socket says 172.21.15.38:55696) 2024-04-25T08:39:43.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.264+0000 7f7331792640 1 -- 172.21.15.38:0/1876736711 learned_addr learned my addr 172.21.15.38:0/1876736711 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:43.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.264+0000 7f732bfff640 1 --2- 172.21.15.38:0/1876736711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f732c0d0700 0x7f732c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.264+0000 7f7330f91640 1 -- 172.21.15.38:0/1876736711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f732c0d0700 msgr2=0x7f732c0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:43.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.264+0000 7f7330f91640 1 --2- 172.21.15.38:0/1876736711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f732c0d0700 0x7f732c0d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.264+0000 7f7330f91640 1 -- 172.21.15.38:0/1876736711 >> v1:172.21.15.38:6789/0 conn(0x7f732c0d1e90 legacy=0x7f732c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.264+0000 7f7330f91640 1 -- 172.21.15.38:0/1876736711 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c0d2bd0 con 0x7f732c0d0320 2024-04-25T08:39:43.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.264+0000 7f732b7fe640 1 -- 172.21.15.38:0/1876736711 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660179692 0 0) 0x7f732c0d2330 con 0x7f732c0d1e90 2024-04-25T08:39:43.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.264+0000 7f732bfff640 1 --2- 172.21.15.38:0/1876736711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f732c0d0700 0x7f732c0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.265+0000 7f7330f91640 1 --2- 172.21.15.38:0/1876736711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732c0d0320 0x7f732c0d33d0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f73200097a0 tx=0x7f732002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c19fecbece876b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.265+0000 7f732b7fe640 1 -- 172.21.15.38:0/1876736711 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320037070 con 0x7f732c0d0320 2024-04-25T08:39:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.265+0000 7f732b7fe640 1 -- 172.21.15.38:0/1876736711 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7320032440 con 0x7f732c0d0320 2024-04-25T08:39:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.265+0000 7f732b7fe640 1 -- 172.21.15.38:0/1876736711 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320032810 con 0x7f732c0d0320 2024-04-25T08:39:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.265+0000 7f73331f5640 1 -- 172.21.15.38:0/1876736711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732c0d0320 msgr2=0x7f732c0d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.265+0000 7f73331f5640 1 --2- 172.21.15.38:0/1876736711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732c0d0320 0x7f732c0d33d0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f73200097a0 tx=0x7f732002a210 comp rx=0 tx=0).stop 2024-04-25T08:39:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.265+0000 7f73331f5640 1 -- 172.21.15.38:0/1876736711 shutdown_connections 2024-04-25T08:39:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.265+0000 7f73331f5640 1 --2- 172.21.15.38:0/1876736711 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f732c0d0700 0x7f732c0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.265+0000 7f73331f5640 1 --2- 172.21.15.38:0/1876736711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732c0d0320 0x7f732c0d33d0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.265+0000 7f73331f5640 1 -- 172.21.15.38:0/1876736711 >> 172.21.15.38:0/1876736711 conn(0x7f732c0668f0 msgr2=0x7f732c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:43.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.265+0000 7f73331f5640 1 -- 172.21.15.38:0/1876736711 shutdown_connections 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.265+0000 7f73331f5640 1 -- 172.21.15.38:0/1876736711 wait complete. 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.266+0000 7f73331f5640 1 Processor -- start 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.267+0000 7f73331f5640 1 -- start start 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.267+0000 7f73331f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f732c0d0320 0x7f732c153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.267+0000 7f73331f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732c0d0700 0x7f732c15e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.267+0000 7f73331f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f732c0d1e90 0x7f732c15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.267+0000 7f73331f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c0d0b40 con 0x7f732c0d0700 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.267+0000 7f73331f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732c161210 con 0x7f732c0d0320 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.267+0000 7f73331f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732c157350 con 0x7f732c0d1e90 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.267+0000 7f732bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732c0d0700 0x7f732c15e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f732bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732c0d0700 0x7f732c15e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38960/0 (socket says 172.21.15.38:38960) 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f732bfff640 1 -- 172.21.15.38:0/3986421421 learned_addr learned my addr 172.21.15.38:0/3986421421 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f7330f91640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f732c0d0320 0x7f732c153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f7331792640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f732c0d1e90 0x7f732c15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f732bfff640 1 -- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f732c0d1e90 msgr2=0x7f732c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f732bfff640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f732c0d1e90 0x7f732c15ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f732bfff640 1 -- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f732c0d0320 msgr2=0x7f732c153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f732bfff640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f732c0d0320 0x7f732c153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f732bfff640 1 -- 172.21.15.38:0/3986421421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732c1f7130 con 0x7f732c0d0700 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f7331792640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f732c0d1e90 0x7f732c15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f7330f91640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f732c0d0320 0x7f732c153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f732bfff640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732c0d0700 0x7f732c15e7f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f731800c7f0 tx=0x7f731800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f73297fa640 1 -- 172.21.15.38:0/3986421421 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318015070 con 0x7f732c0d0700 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f73297fa640 1 -- 172.21.15.38:0/3986421421 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7318007a30 con 0x7f732c0d0700 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f73331f5640 1 -- 172.21.15.38:0/3986421421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732c1f7410 con 0x7f732c0d0700 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f73297fa640 1 -- 172.21.15.38:0/3986421421 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318007e00 con 0x7f732c0d0700 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.268+0000 7f73331f5640 1 -- 172.21.15.38:0/3986421421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732c1f79d0 con 0x7f732c0d0700 2024-04-25T08:39:43.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.269+0000 7f73331f5640 1 -- 172.21.15.38:0/3986421421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732c068e80 con 0x7f732c0d0700 2024-04-25T08:39:43.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.270+0000 7f73297fa640 1 -- 172.21.15.38:0/3986421421 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7318018070 con 0x7f732c0d0700 2024-04-25T08:39:43.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.270+0000 7f73297fa640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f731006b4d0 0x7f731006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.270+0000 7f73297fa640 1 -- 172.21.15.38:0/3986421421 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f731808a2a0 con 0x7f732c0d0700 2024-04-25T08:39:43.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.270+0000 7f7330f91640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f731006b4d0 0x7f731006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.271+0000 7f7330f91640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f731006b4d0 0x7f731006d990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7320034000 tx=0x7f732002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:43.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.274+0000 7f73297fa640 1 -- 172.21.15.38:0/3986421421 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7318058ff0 con 0x7f732c0d0700 2024-04-25T08:39:43.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.456+0000 7f73331f5640 1 -- 172.21.15.38:0/3986421421 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f732c15b6a0 con 0x7f732c0d0700 2024-04-25T08:39:43.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.456+0000 7f73297fa640 1 -- 172.21.15.38:0/3986421421 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73180588f0 con 0x7f732c0d0700 2024-04-25T08:39:43.449 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:39:43.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.458+0000 7f73331f5640 1 -- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f731006b4d0 msgr2=0x7f731006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.458+0000 7f73331f5640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f731006b4d0 0x7f731006d990 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7320034000 tx=0x7f732002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.458+0000 7f73331f5640 1 -- 172.21.15.38:0/3986421421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732c0d0700 msgr2=0x7f732c15e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.458+0000 7f73331f5640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732c0d0700 0x7f732c15e7f0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f731800c7f0 tx=0x7f731800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.458+0000 7f73331f5640 1 -- 172.21.15.38:0/3986421421 shutdown_connections 2024-04-25T08:39:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.458+0000 7f73331f5640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f731006b4d0 0x7f731006d990 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.458+0000 7f73331f5640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f732c0d1e90 0x7f732c15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.458+0000 7f73331f5640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f732c0d0700 0x7f732c15e7f0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.458+0000 7f73331f5640 1 --2- 172.21.15.38:0/3986421421 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f732c0d0320 0x7f732c153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.458+0000 7f73331f5640 1 -- 172.21.15.38:0/3986421421 >> 172.21.15.38:0/3986421421 conn(0x7f732c0668f0 msgr2=0x7f732c073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.458+0000 7f73331f5640 1 -- 172.21.15.38:0/3986421421 shutdown_connections 2024-04-25T08:39:43.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.458+0000 7f73331f5640 1 -- 172.21.15.38:0/3986421421 wait complete. 2024-04-25T08:39:43.466 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:39:43.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.577+0000 7f5114c12640 1 Processor -- start 2024-04-25T08:39:43.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.577+0000 7f5114c12640 1 -- start start 2024-04-25T08:39:43.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f5114c12640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51100d0320 0x7f51100d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f5114c12640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f51100d0700 0x7f51100d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f5114c12640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51100d2330 con 0x7f51100d1e90 2024-04-25T08:39:43.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f5114c12640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51100d2610 con 0x7f51100d0700 2024-04-25T08:39:43.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f5114c12640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51100d28f0 con 0x7f51100d0320 2024-04-25T08:39:43.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f5107fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f51100d0700 0x7f51100d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f510ed9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f51100d1e90 0x7f51100b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55710/0 (socket says 172.21.15.38:55710) 2024-04-25T08:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f510ed9d640 1 -- 172.21.15.38:0/1391787614 learned_addr learned my addr 172.21.15.38:0/1391787614 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f510e59c640 1 --2- 172.21.15.38:0/1391787614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51100d0320 0x7f51100d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f5107fff640 1 -- 172.21.15.38:0/1391787614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51100d0320 msgr2=0x7f51100d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f5107fff640 1 --2- 172.21.15.38:0/1391787614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51100d0320 0x7f51100d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f5107fff640 1 -- 172.21.15.38:0/1391787614 >> v1:172.21.15.38:6789/0 conn(0x7f51100d1e90 legacy=0x7f51100b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f510dd9b640 1 -- 172.21.15.38:0/1391787614 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488110476 0 0) 0x7f51100d2330 con 0x7f51100d1e90 2024-04-25T08:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.578+0000 7f5107fff640 1 -- 172.21.15.38:0/1391787614 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51100d2bd0 con 0x7f51100d0700 2024-04-25T08:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.579+0000 7f5107fff640 1 --2- 172.21.15.38:0/1391787614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51100d0700 0x7f51100d1950 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f50f80097a0 tx=0x7f50f802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da56449ac20e9641 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.579+0000 7f510dd9b640 1 -- 172.21.15.38:0/1391787614 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8037070 con 0x7f51100d0700 2024-04-25T08:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.579+0000 7f510dd9b640 1 -- 172.21.15.38:0/1391787614 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f8032440 con 0x7f51100d0700 2024-04-25T08:39:43.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.579+0000 7f510dd9b640 1 -- 172.21.15.38:0/1391787614 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8032810 con 0x7f51100d0700 2024-04-25T08:39:43.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.580+0000 7f5114c12640 1 -- 172.21.15.38:0/1391787614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51100d0700 msgr2=0x7f51100d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.580+0000 7f5114c12640 1 --2- 172.21.15.38:0/1391787614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51100d0700 0x7f51100d1950 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f50f80097a0 tx=0x7f50f802a210 comp rx=0 tx=0).stop 2024-04-25T08:39:43.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.580+0000 7f5114c12640 1 -- 172.21.15.38:0/1391787614 shutdown_connections 2024-04-25T08:39:43.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.580+0000 7f5114c12640 1 --2- 172.21.15.38:0/1391787614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51100d0700 0x7f51100d1950 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.580+0000 7f5114c12640 1 --2- 172.21.15.38:0/1391787614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51100d0320 0x7f51100d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.580+0000 7f5114c12640 1 -- 172.21.15.38:0/1391787614 >> 172.21.15.38:0/1391787614 conn(0x7f51100668f0 msgr2=0x7f5110066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.580+0000 7f5114c12640 1 -- 172.21.15.38:0/1391787614 shutdown_connections 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.580+0000 7f5114c12640 1 -- 172.21.15.38:0/1391787614 wait complete. 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.581+0000 7f5114c12640 1 Processor -- start 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.581+0000 7f5114c12640 1 -- start start 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f5114c12640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51100d0320 0x7f51100b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f5114c12640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51100d0700 0x7f511015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f5114c12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51100d1e90 0x7f51100b3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f5114c12640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51100d0b40 con 0x7f51100d1e90 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f5114c12640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5110161210 con 0x7f51100d0700 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f5114c12640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511015c310 con 0x7f51100d0320 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f510ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51100d1e90 0x7f51100b3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f510ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51100d1e90 0x7f51100b3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38968/0 (socket says 172.21.15.38:38968) 2024-04-25T08:39:43.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f510ed9d640 1 -- 172.21.15.38:0/2130982160 learned_addr learned my addr 172.21.15.38:0/2130982160 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f510e59c640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51100d0320 0x7f51100b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f5107fff640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51100d0700 0x7f511015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f510ed9d640 1 -- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51100d0320 msgr2=0x7f51100b5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f510ed9d640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51100d0320 0x7f51100b5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f510ed9d640 1 -- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51100d0700 msgr2=0x7f511015c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f510ed9d640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51100d0700 0x7f511015c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f510ed9d640 1 -- 172.21.15.38:0/2130982160 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5110156f30 con 0x7f51100d1e90 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f510e59c640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51100d0320 0x7f51100b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f5107fff640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51100d0700 0x7f511015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.582+0000 7f510ed9d640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51100d1e90 0x7f51100b3e70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f510800c780 tx=0x7f510800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.583+0000 7f5106ffd640 1 -- 172.21.15.38:0/2130982160 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108018070 con 0x7f51100d1e90 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.583+0000 7f5106ffd640 1 -- 172.21.15.38:0/2130982160 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510800ce30 con 0x7f51100d1e90 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.583+0000 7f5114c12640 1 -- 172.21.15.38:0/2130982160 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511015bfe0 con 0x7f51100d1e90 2024-04-25T08:39:43.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.583+0000 7f5106ffd640 1 -- 172.21.15.38:0/2130982160 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510800a840 con 0x7f51100d1e90 2024-04-25T08:39:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.583+0000 7f5114c12640 1 -- 172.21.15.38:0/2130982160 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51101574b0 con 0x7f51100d1e90 2024-04-25T08:39:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.584+0000 7f5114c12640 1 -- 172.21.15.38:0/2130982160 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511015dde0 con 0x7f51100d1e90 2024-04-25T08:39:43.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.584+0000 7f5106ffd640 1 -- 172.21.15.38:0/2130982160 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f510801b070 con 0x7f51100d1e90 2024-04-25T08:39:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.584+0000 7f5106ffd640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f50f006b500 0x7f50f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.584+0000 7f5106ffd640 1 -- 172.21.15.38:0/2130982160 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5108089810 con 0x7f51100d1e90 2024-04-25T08:39:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.585+0000 7f510e59c640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f50f006b500 0x7f50f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.585+0000 7f510e59c640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f50f006b500 0x7f50f006d9c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5100006fd0 tx=0x7f5100008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:43.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.588+0000 7f5106ffd640 1 -- 172.21.15.38:0/2130982160 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f511015dde0 con 0x7f51100d1e90 2024-04-25T08:39:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.771+0000 7f5114c12640 1 -- 172.21.15.38:0/2130982160 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5110068e80 con 0x7f51100d1e90 2024-04-25T08:39:43.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.771+0000 7f5106ffd640 1 -- 172.21.15.38:0/2130982160 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f51080584d0 con 0x7f51100d1e90 2024-04-25T08:39:43.765 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:39:43.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.773+0000 7f5114c12640 1 -- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f50f006b500 msgr2=0x7f50f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.773+0000 7f5114c12640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f50f006b500 0x7f50f006d9c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5100006fd0 tx=0x7f5100008040 comp rx=0 tx=0).stop 2024-04-25T08:39:43.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.773+0000 7f5114c12640 1 -- 172.21.15.38:0/2130982160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51100d1e90 msgr2=0x7f51100b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.773+0000 7f5114c12640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51100d1e90 0x7f51100b3e70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f510800c780 tx=0x7f510800cc40 comp rx=0 tx=0).stop 2024-04-25T08:39:43.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.773+0000 7f5114c12640 1 -- 172.21.15.38:0/2130982160 shutdown_connections 2024-04-25T08:39:43.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.773+0000 7f5114c12640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51100d1e90 0x7f51100b3e70 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.773+0000 7f5114c12640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51100d0700 0x7f511015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.773+0000 7f5114c12640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f50f006b500 0x7f50f006d9c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.773+0000 7f5114c12640 1 --2- 172.21.15.38:0/2130982160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f51100d0320 0x7f51100b5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.773+0000 7f5114c12640 1 -- 172.21.15.38:0/2130982160 >> 172.21.15.38:0/2130982160 conn(0x7f51100668f0 msgr2=0x7f51100b8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:43.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.773+0000 7f5114c12640 1 -- 172.21.15.38:0/2130982160 shutdown_connections 2024-04-25T08:39:43.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.773+0000 7f5114c12640 1 -- 172.21.15.38:0/2130982160 wait complete. 2024-04-25T08:39:43.778 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:43.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.886+0000 7f479d8a3640 1 Processor -- start 2024-04-25T08:39:43.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.886+0000 7f479d8a3640 1 -- start start 2024-04-25T08:39:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f479d8a3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47980b5710 0x7f4798161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f479d8a3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f479815ddf0 0x7f47980b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f479d8a3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47980b5af0 0x7f479806bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f479d8a3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47980b9930 con 0x7f47980b5710 2024-04-25T08:39:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f479d8a3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4798161e00 con 0x7f479815ddf0 2024-04-25T08:39:43.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f479d8a3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47981620e0 con 0x7f47980b5af0 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f4796ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47980b5710 0x7f4798161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f4796ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f47980b5710 0x7f4798161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38972/0 (socket says 172.21.15.38:38972) 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f4796ffd640 1 -- 172.21.15.38:0/35361614 learned_addr learned my addr 172.21.15.38:0/35361614 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f478ffff640 1 --2- 172.21.15.38:0/35361614 >> v2:172.21.15.107:3300/0 conn(0x7f479815ddf0 0x7f47980b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f47977fe640 1 --2- 172.21.15.38:0/35361614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47980b5af0 0x7f479806bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f4796ffd640 1 -- 172.21.15.38:0/35361614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47980b5af0 msgr2=0x7f479806bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f4796ffd640 1 --2- 172.21.15.38:0/35361614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47980b5af0 0x7f479806bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f4796ffd640 1 -- 172.21.15.38:0/35361614 >> v2:172.21.15.107:3300/0 conn(0x7f479815ddf0 msgr2=0x7f47980b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f4796ffd640 1 --2- 172.21.15.38:0/35361614 >> v2:172.21.15.107:3300/0 conn(0x7f479815ddf0 0x7f47980b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.887+0000 7f4796ffd640 1 -- 172.21.15.38:0/35361614 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47981623c0 con 0x7f47980b5710 2024-04-25T08:39:43.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.888+0000 7f4796ffd640 1 --2- 172.21.15.38:0/35361614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47980b5710 0x7f4798161700 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f47980d0300 tx=0x7f478802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7725b32ebb989b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:43.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.888+0000 7f47967fc640 1 -- 172.21.15.38:0/35361614 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788037070 con 0x7f47980b5710 2024-04-25T08:39:43.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.888+0000 7f47967fc640 1 -- 172.21.15.38:0/35361614 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4788032440 con 0x7f47980b5710 2024-04-25T08:39:43.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.888+0000 7f47967fc640 1 -- 172.21.15.38:0/35361614 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788032810 con 0x7f47980b5710 2024-04-25T08:39:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.889+0000 7f479d8a3640 1 -- 172.21.15.38:0/35361614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47980b5710 msgr2=0x7f4798161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.889+0000 7f479d8a3640 1 --2- 172.21.15.38:0/35361614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47980b5710 0x7f4798161700 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f47980d0300 tx=0x7f478802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.889+0000 7f479d8a3640 1 -- 172.21.15.38:0/35361614 shutdown_connections 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.889+0000 7f479d8a3640 1 --2- 172.21.15.38:0/35361614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f47980b5af0 0x7f479806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.889+0000 7f479d8a3640 1 --2- 172.21.15.38:0/35361614 >> v2:172.21.15.107:3300/0 conn(0x7f479815ddf0 0x7f47980b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.889+0000 7f479d8a3640 1 --2- 172.21.15.38:0/35361614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47980b5710 0x7f4798161700 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.889+0000 7f479d8a3640 1 -- 172.21.15.38:0/35361614 >> 172.21.15.38:0/35361614 conn(0x7f47980d07c0 msgr2=0x7f47980d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.889+0000 7f479d8a3640 1 -- 172.21.15.38:0/35361614 shutdown_connections 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.889+0000 7f479d8a3640 1 -- 172.21.15.38:0/35361614 wait complete. 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.890+0000 7f479d8a3640 1 Processor -- start 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.890+0000 7f479d8a3640 1 -- start start 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f479d8a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47980b5710 0x7f47981e96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f479d8a3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47980b5af0 0x7f4798159130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f479d8a3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f479815ddf0 0x7f47980c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f479d8a3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479806c6b0 con 0x7f47980b5710 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f479d8a3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479806c530 con 0x7f47980b5af0 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f479d8a3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479806c830 con 0x7f479815ddf0 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f4796ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47980b5710 0x7f47981e96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f4796ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47980b5710 0x7f47981e96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38978/0 (socket says 172.21.15.38:38978) 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f4796ffd640 1 -- 172.21.15.38:0/1969107664 learned_addr learned my addr 172.21.15.38:0/1969107664 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f478ffff640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47980b5af0 0x7f4798159130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f47977fe640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f479815ddf0 0x7f47980c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f4796ffd640 1 -- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f479815ddf0 msgr2=0x7f47980c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f4796ffd640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f479815ddf0 0x7f47980c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f4796ffd640 1 -- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47980b5af0 msgr2=0x7f4798159130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f4796ffd640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47980b5af0 0x7f4798159130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f4796ffd640 1 -- 172.21.15.38:0/1969107664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980c87a0 con 0x7f47980b5710 2024-04-25T08:39:43.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f47977fe640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f479815ddf0 0x7f47980c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f4796ffd640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47980b5710 0x7f47981e96e0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f47880097c0 tx=0x7f47880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f4794ff9640 1 -- 172.21.15.38:0/1969107664 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788041020 con 0x7f47980b5710 2024-04-25T08:39:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f4794ff9640 1 -- 172.21.15.38:0/1969107664 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47880306c0 con 0x7f47980b5710 2024-04-25T08:39:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f479d8a3640 1 -- 172.21.15.38:0/1969107664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47980c8a80 con 0x7f47980b5710 2024-04-25T08:39:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.891+0000 7f4794ff9640 1 -- 172.21.15.38:0/1969107664 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788037040 con 0x7f47980b5710 2024-04-25T08:39:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.892+0000 7f479d8a3640 1 -- 172.21.15.38:0/1969107664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47980c9040 con 0x7f47980b5710 2024-04-25T08:39:43.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.892+0000 7f479d8a3640 1 -- 172.21.15.38:0/1969107664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4798068e80 con 0x7f47980b5710 2024-04-25T08:39:43.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.893+0000 7f4794ff9640 1 -- 172.21.15.38:0/1969107664 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4788044020 con 0x7f47980b5710 2024-04-25T08:39:43.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.893+0000 7f4794ff9640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f477806b4d0 0x7f477806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:43.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.893+0000 7f4794ff9640 1 -- 172.21.15.38:0/1969107664 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f47880aabc0 con 0x7f47980b5710 2024-04-25T08:39:43.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.894+0000 7f478ffff640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f477806b4d0 0x7f477806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:43.889 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.894+0000 7f478ffff640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f477806b4d0 0x7f477806d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f47980cdbf0 tx=0x7f4780008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:43.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:43.897+0000 7f4794ff9640 1 -- 172.21.15.38:0/1969107664 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f478804b030 con 0x7f47980b5710 2024-04-25T08:39:44.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.115+0000 7f479d8a3640 1 -- 172.21.15.38:0/1969107664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47980b4050 con 0x7f47980b5710 2024-04-25T08:39:44.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.115+0000 7f4794ff9640 1 -- 172.21.15.38:0/1969107664 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f47880797e0 con 0x7f47980b5710 2024-04-25T08:39:44.109 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:44.109 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:07.551641+0000","modified":"2024-04-25T08:39:09.555958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8292":{"gid":8292,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1263010516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1263010516},{"type":"v1","addr":"172.21.15.107:6837","nonce":1263010516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:39:44.109 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 63 2024-04-25T08:39:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.117+0000 7f479d8a3640 1 -- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f477806b4d0 msgr2=0x7f477806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.117+0000 7f479d8a3640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f477806b4d0 0x7f477806d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f47980cdbf0 tx=0x7f4780008040 comp rx=0 tx=0).stop 2024-04-25T08:39:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.117+0000 7f479d8a3640 1 -- 172.21.15.38:0/1969107664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47980b5710 msgr2=0x7f47981e96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.117+0000 7f479d8a3640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47980b5710 0x7f47981e96e0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f47880097c0 tx=0x7f47880046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.117+0000 7f479d8a3640 1 -- 172.21.15.38:0/1969107664 shutdown_connections 2024-04-25T08:39:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.117+0000 7f479d8a3640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f477806b4d0 0x7f477806d990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.117+0000 7f479d8a3640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f479815ddf0 0x7f47980c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.117+0000 7f479d8a3640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47980b5af0 0x7f4798159130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.117+0000 7f479d8a3640 1 --2- 172.21.15.38:0/1969107664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47980b5710 0x7f47981e96e0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.117+0000 7f479d8a3640 1 -- 172.21.15.38:0/1969107664 >> 172.21.15.38:0/1969107664 conn(0x7f47980d07c0 msgr2=0x7f47980b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:44.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.118+0000 7f479d8a3640 1 -- 172.21.15.38:0/1969107664 shutdown_connections 2024-04-25T08:39:44.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.118+0000 7f479d8a3640 1 -- 172.21.15.38:0/1969107664 wait complete. 2024-04-25T08:39:44.122 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:44.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.229+0000 7fed248d8640 1 Processor -- start 2024-04-25T08:39:44.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.229+0000 7fed248d8640 1 -- start start 2024-04-25T08:39:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.230+0000 7fed248d8640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fed2015dc10 0x7fed200ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.230+0000 7fed248d8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fed200ced60 0x7fed200d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.230+0000 7fed248d8640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed200d3e70 con 0x7fed200d3900 2024-04-25T08:39:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.230+0000 7fed248d8640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2015dff0 con 0x7fed200ced60 2024-04-25T08:39:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.230+0000 7fed248d8640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed200b6990 con 0x7fed2015dc10 2024-04-25T08:39:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.230+0000 7fed1dd9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fed200ced60 0x7fed200d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.230+0000 7fed1ed9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fed200d3900 0x7fed200b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55716/0 (socket says 172.21.15.38:55716) 2024-04-25T08:39:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.230+0000 7fed1ed9d640 1 -- 172.21.15.38:0/1546738171 learned_addr learned my addr 172.21.15.38:0/1546738171 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.231+0000 7fed1e59c640 1 --2- 172.21.15.38:0/1546738171 >> v2:172.21.15.107:3300/0 conn(0x7fed2015dc10 0x7fed200ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.231+0000 7fed1dd9b640 1 -- 172.21.15.38:0/1546738171 >> v2:172.21.15.107:3300/0 conn(0x7fed2015dc10 msgr2=0x7fed200ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.231+0000 7fed1dd9b640 1 --2- 172.21.15.38:0/1546738171 >> v2:172.21.15.107:3300/0 conn(0x7fed2015dc10 0x7fed200ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.231+0000 7fed1dd9b640 1 -- 172.21.15.38:0/1546738171 >> v1:172.21.15.38:6789/0 conn(0x7fed200d3900 legacy=0x7fed200b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.231+0000 7fed1dd9b640 1 -- 172.21.15.38:0/1546738171 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed200d4150 con 0x7fed200ced60 2024-04-25T08:39:44.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.231+0000 7fed1d59a640 1 -- 172.21.15.38:0/1546738171 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1548804355 0 0) 0x7fed200d3e70 con 0x7fed200d3900 2024-04-25T08:39:44.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.231+0000 7fed1dd9b640 1 --2- 172.21.15.38:0/1546738171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed200ced60 0x7fed200d0850 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fed200b5d50 tx=0x7fed1402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a06b023921ed74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:44.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.232+0000 7fed1d59a640 1 -- 172.21.15.38:0/1546738171 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14037070 con 0x7fed200ced60 2024-04-25T08:39:44.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.232+0000 7fed1d59a640 1 -- 172.21.15.38:0/1546738171 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed14032440 con 0x7fed200ced60 2024-04-25T08:39:44.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.232+0000 7fed1d59a640 1 -- 172.21.15.38:0/1546738171 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1402ae60 con 0x7fed200ced60 2024-04-25T08:39:44.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.232+0000 7fed248d8640 1 -- 172.21.15.38:0/1546738171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed200ced60 msgr2=0x7fed200d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.232+0000 7fed248d8640 1 --2- 172.21.15.38:0/1546738171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed200ced60 0x7fed200d0850 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fed200b5d50 tx=0x7fed1402a110 comp rx=0 tx=0).stop 2024-04-25T08:39:44.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.232+0000 7fed248d8640 1 -- 172.21.15.38:0/1546738171 shutdown_connections 2024-04-25T08:39:44.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.232+0000 7fed248d8640 1 --2- 172.21.15.38:0/1546738171 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed200ced60 0x7fed200d0850 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.232+0000 7fed248d8640 1 --2- 172.21.15.38:0/1546738171 >> v2:172.21.15.107:3300/0 conn(0x7fed2015dc10 0x7fed200ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.232+0000 7fed248d8640 1 -- 172.21.15.38:0/1546738171 >> 172.21.15.38:0/1546738171 conn(0x7fed20066d10 msgr2=0x7fed20067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.232+0000 7fed248d8640 1 -- 172.21.15.38:0/1546738171 shutdown_connections 2024-04-25T08:39:44.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.232+0000 7fed248d8640 1 -- 172.21.15.38:0/1546738171 wait complete. 2024-04-25T08:39:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.234+0000 7fed248d8640 1 Processor -- start 2024-04-25T08:39:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.234+0000 7fed248d8640 1 -- start start 2024-04-25T08:39:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.234+0000 7fed248d8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed200ced60 0x7fed2015a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.234+0000 7fed248d8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed200d3900 0x7fed200c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.234+0000 7fed248d8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed2015dc10 0x7fed2015ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.234+0000 7fed248d8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed200b6bc0 con 0x7fed200d3900 2024-04-25T08:39:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.234+0000 7fed248d8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed200d4710 con 0x7fed200ced60 2024-04-25T08:39:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.234+0000 7fed248d8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed200d0d90 con 0x7fed2015dc10 2024-04-25T08:39:44.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.234+0000 7fed1dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed200d3900 0x7fed200c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.234+0000 7fed1dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed200d3900 0x7fed200c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38998/0 (socket says 172.21.15.38:38998) 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed1dd9b640 1 -- 172.21.15.38:0/1681469890 learned_addr learned my addr 172.21.15.38:0/1681469890 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed1ed9d640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed2015dc10 0x7fed2015ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed1e59c640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed200ced60 0x7fed2015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed1dd9b640 1 -- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed2015dc10 msgr2=0x7fed2015ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed1dd9b640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed2015dc10 0x7fed2015ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed1dd9b640 1 -- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed200ced60 msgr2=0x7fed2015a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed1dd9b640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed200ced60 0x7fed2015a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed1dd9b640 1 -- 172.21.15.38:0/1681469890 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed201f4520 con 0x7fed200d3900 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed1ed9d640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed2015dc10 0x7fed2015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed1dd9b640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed200d3900 0x7fed200c8830 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fed14000c00 tx=0x7fed140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed0effd640 1 -- 172.21.15.38:0/1681469890 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14037040 con 0x7fed200d3900 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed0effd640 1 -- 172.21.15.38:0/1681469890 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed14042070 con 0x7fed200d3900 2024-04-25T08:39:44.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed248d8640 1 -- 172.21.15.38:0/1681469890 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed200cab70 con 0x7fed200d3900 2024-04-25T08:39:44.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed0effd640 1 -- 172.21.15.38:0/1681469890 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14030a50 con 0x7fed200d3900 2024-04-25T08:39:44.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.235+0000 7fed248d8640 1 -- 172.21.15.38:0/1681469890 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed200c8d70 con 0x7fed200d3900 2024-04-25T08:39:44.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.236+0000 7fed248d8640 1 -- 172.21.15.38:0/1681469890 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed200c9000 con 0x7fed200d3900 2024-04-25T08:39:44.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.237+0000 7fed0effd640 1 -- 172.21.15.38:0/1681469890 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed14030bf0 con 0x7fed200d3900 2024-04-25T08:39:44.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.237+0000 7fed0effd640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fed0006b4d0 0x7fed0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.237+0000 7fed0effd640 1 -- 172.21.15.38:0/1681469890 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fed140aaed0 con 0x7fed200d3900 2024-04-25T08:39:44.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.237+0000 7fed1e59c640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fed0006b4d0 0x7fed0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.238+0000 7fed1e59c640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fed0006b4d0 0x7fed0006d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fed1800b7c0 tx=0x7fed18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:44.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.242+0000 7fed0effd640 1 -- 172.21.15.38:0/1681469890 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed14045360 con 0x7fed200d3900 2024-04-25T08:39:44.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.422+0000 7fed248d8640 1 -- 172.21.15.38:0/1681469890 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed20068e80 con 0x7fed200d3900 2024-04-25T08:39:44.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.423+0000 7fed0effd640 1 -- 172.21.15.38:0/1681469890 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16970 (secure 0 0 0) 0x7fed14079b90 con 0x7fed200d3900 2024-04-25T08:39:44.417 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:44.417 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":110,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:18.940299+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:05.852531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:06.527775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":1,"to":7}]},{"pgid":"16.1e","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:44.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.425+0000 7fed248d8640 1 -- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fed0006b4d0 msgr2=0x7fed0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.425+0000 7fed248d8640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fed0006b4d0 0x7fed0006d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fed1800b7c0 tx=0x7fed18008040 comp rx=0 tx=0).stop 2024-04-25T08:39:44.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.425+0000 7fed248d8640 1 -- 172.21.15.38:0/1681469890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed200d3900 msgr2=0x7fed200c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.425+0000 7fed248d8640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed200d3900 0x7fed200c8830 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fed14000c00 tx=0x7fed140046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.425+0000 7fed248d8640 1 -- 172.21.15.38:0/1681469890 shutdown_connections 2024-04-25T08:39:44.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.425+0000 7fed248d8640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fed0006b4d0 0x7fed0006d990 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.425+0000 7fed248d8640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed2015dc10 0x7fed2015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.425+0000 7fed248d8640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed200d3900 0x7fed200c8830 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.425+0000 7fed248d8640 1 --2- 172.21.15.38:0/1681469890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed200ced60 0x7fed2015a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.425+0000 7fed248d8640 1 -- 172.21.15.38:0/1681469890 >> 172.21.15.38:0/1681469890 conn(0x7fed20066d10 msgr2=0x7fed200b83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:44.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.425+0000 7fed248d8640 1 -- 172.21.15.38:0/1681469890 shutdown_connections 2024-04-25T08:39:44.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.425+0000 7fed248d8640 1 -- 172.21.15.38:0/1681469890 wait complete. 2024-04-25T08:39:44.430 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:39:44.431 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:44.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.538+0000 7fa92f73b640 1 Processor -- start 2024-04-25T08:39:44.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.538+0000 7fa92f73b640 1 -- start start 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.539+0000 7fa92f73b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa928071bd0 0x7fa92815bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.539+0000 7fa92f73b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9280cd3f0 con 0x7fa92815b950 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.539+0000 7fa92f73b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9280ccb50 con 0x7fa92806f0f0 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.539+0000 7fa92f73b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9280cce30 con 0x7fa928071bd0 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.539+0000 7fa92dcd8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa928071bd0 0x7fa92815bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.539+0000 7fa92d4d7640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa92815b950 0x7fa9280cf020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55720/0 (socket says 172.21.15.38:55720) 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.539+0000 7fa92d4d7640 1 -- 172.21.15.38:0/4183824983 learned_addr learned my addr 172.21.15.38:0/4183824983 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.539+0000 7fa92dcd8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa928071bd0 0x7fa92815bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39012/0 (socket says 172.21.15.38:39012) 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.539+0000 7fa92dcd8640 1 -- 172.21.15.38:0/4183824983 >> v1:172.21.15.107:6789/0 conn(0x7fa92806f0f0 legacy=0x7fa9280714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.539+0000 7fa92dcd8640 1 -- 172.21.15.38:0/4183824983 >> v1:172.21.15.38:6789/0 conn(0x7fa92815b950 legacy=0x7fa9280cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.539+0000 7fa92dcd8640 1 -- 172.21.15.38:0/4183824983 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9280cd110 con 0x7fa928071bd0 2024-04-25T08:39:44.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.540+0000 7fa92dcd8640 1 --2- 172.21.15.38:0/4183824983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa928071bd0 0x7fa92815bd30 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fa9200097a0 tx=0x7fa92002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a6f56e11db484 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:44.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.540+0000 7fa927fff640 1 -- 172.21.15.38:0/4183824983 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920037070 con 0x7fa928071bd0 2024-04-25T08:39:44.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.540+0000 7fa927fff640 1 -- 172.21.15.38:0/4183824983 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa920032440 con 0x7fa928071bd0 2024-04-25T08:39:44.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.540+0000 7fa927fff640 1 -- 172.21.15.38:0/4183824983 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920032810 con 0x7fa928071bd0 2024-04-25T08:39:44.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.541+0000 7fa92f73b640 1 -- 172.21.15.38:0/4183824983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa928071bd0 msgr2=0x7fa92815bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.541+0000 7fa92f73b640 1 --2- 172.21.15.38:0/4183824983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa928071bd0 0x7fa92815bd30 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fa9200097a0 tx=0x7fa92002a230 comp rx=0 tx=0).stop 2024-04-25T08:39:44.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.541+0000 7fa92f73b640 1 -- 172.21.15.38:0/4183824983 shutdown_connections 2024-04-25T08:39:44.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.541+0000 7fa92f73b640 1 --2- 172.21.15.38:0/4183824983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa928071bd0 0x7fa92815bd30 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.541+0000 7fa92f73b640 1 -- 172.21.15.38:0/4183824983 >> 172.21.15.38:0/4183824983 conn(0x7fa9280cf970 msgr2=0x7fa9280cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:44.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.541+0000 7fa92f73b640 1 -- 172.21.15.38:0/4183824983 shutdown_connections 2024-04-25T08:39:44.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.541+0000 7fa92f73b640 1 -- 172.21.15.38:0/4183824983 wait complete. 2024-04-25T08:39:44.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.542+0000 7fa92f73b640 1 Processor -- start 2024-04-25T08:39:44.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.542+0000 7fa92f73b640 1 -- start start 2024-04-25T08:39:44.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92f73b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92806f0f0 0x7fa928147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92f73b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa928071bd0 0x7fa92815c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92f73b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92815b950 0x7fa92815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92f73b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa928072200 con 0x7fa92815b950 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92f73b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280cc7d0 con 0x7fa92806f0f0 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92f73b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa928072380 con 0x7fa928071bd0 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92dcd8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92815b950 0x7fa92815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92dcd8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92815b950 0x7fa92815c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39026/0 (socket says 172.21.15.38:39026) 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92dcd8640 1 -- 172.21.15.38:0/568918085 learned_addr learned my addr 172.21.15.38:0/568918085 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92d4d7640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92806f0f0 0x7fa928147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92dcd8640 1 -- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa928071bd0 msgr2=0x7fa92815c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92dcd8640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa928071bd0 0x7fa92815c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92dcd8640 1 -- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92806f0f0 msgr2=0x7fa928147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92dcd8640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92806f0f0 0x7fa928147340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.543+0000 7fa92dcd8640 1 -- 172.21.15.38:0/568918085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9281462a0 con 0x7fa92815b950 2024-04-25T08:39:44.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.544+0000 7fa92dcd8640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92815b950 0x7fa92815c6b0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa9200098c0 tx=0x7fa9200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:44.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.544+0000 7fa925ffb640 1 -- 172.21.15.38:0/568918085 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920037040 con 0x7fa92815b950 2024-04-25T08:39:44.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.544+0000 7fa925ffb640 1 -- 172.21.15.38:0/568918085 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa920042070 con 0x7fa92815b950 2024-04-25T08:39:44.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.544+0000 7fa92f73b640 1 -- 172.21.15.38:0/568918085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa928146580 con 0x7fa92815b950 2024-04-25T08:39:44.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.544+0000 7fa925ffb640 1 -- 172.21.15.38:0/568918085 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920030b50 con 0x7fa92815b950 2024-04-25T08:39:44.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.544+0000 7fa92f73b640 1 -- 172.21.15.38:0/568918085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa928146b40 con 0x7fa92815b950 2024-04-25T08:39:44.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.545+0000 7fa92f73b640 1 -- 172.21.15.38:0/568918085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa928068e80 con 0x7fa92815b950 2024-04-25T08:39:44.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.546+0000 7fa925ffb640 1 -- 172.21.15.38:0/568918085 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa920030cf0 con 0x7fa92815b950 2024-04-25T08:39:44.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.546+0000 7fa925ffb640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa91406b4d0 0x7fa91406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.546+0000 7fa925ffb640 1 -- 172.21.15.38:0/568918085 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa9200aaf60 con 0x7fa92815b950 2024-04-25T08:39:44.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.546+0000 7fa92d4d7640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa91406b4d0 0x7fa91406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.547+0000 7fa92d4d7640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa91406b4d0 0x7fa91406d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa918005c00 tx=0x7fa918009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:44.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.550+0000 7fa925ffb640 1 -- 172.21.15.38:0/568918085 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa920032e20 con 0x7fa92815b950 2024-04-25T08:39:44.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.765+0000 7fa92f73b640 1 -- 172.21.15.38:0/568918085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa92815ac20 con 0x7fa92815b950 2024-04-25T08:39:44.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.766+0000 7fa925ffb640 1 -- 172.21.15.38:0/568918085 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fa920045360 con 0x7fa92815b950 2024-04-25T08:39:44.759 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:44.759 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:07.551641+0000","modified":"2024-04-25T08:39:09.555958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8292":{"gid":8292,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1263010516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1263010516},{"type":"v1","addr":"172.21.15.107:6837","nonce":1263010516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:39:44.760 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 63 2024-04-25T08:39:44.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.768+0000 7fa92f73b640 1 -- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa91406b4d0 msgr2=0x7fa91406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.768+0000 7fa92f73b640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa91406b4d0 0x7fa91406d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fa918005c00 tx=0x7fa918009040 comp rx=0 tx=0).stop 2024-04-25T08:39:44.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.768+0000 7fa92f73b640 1 -- 172.21.15.38:0/568918085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92815b950 msgr2=0x7fa92815c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.768+0000 7fa92f73b640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92815b950 0x7fa92815c6b0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7fa9200098c0 tx=0x7fa9200046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.768+0000 7fa92f73b640 1 -- 172.21.15.38:0/568918085 shutdown_connections 2024-04-25T08:39:44.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.768+0000 7fa92f73b640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa92815b950 0x7fa92815c6b0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.768+0000 7fa92f73b640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa91406b4d0 0x7fa91406d990 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.768+0000 7fa92f73b640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa928071bd0 0x7fa92815c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.768+0000 7fa92f73b640 1 --2- 172.21.15.38:0/568918085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92806f0f0 0x7fa928147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.768+0000 7fa92f73b640 1 -- 172.21.15.38:0/568918085 >> 172.21.15.38:0/568918085 conn(0x7fa9280cf970 msgr2=0x7fa92806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:44.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.768+0000 7fa92f73b640 1 -- 172.21.15.38:0/568918085 shutdown_connections 2024-04-25T08:39:44.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.768+0000 7fa92f73b640 1 -- 172.21.15.38:0/568918085 wait complete. 2024-04-25T08:39:44.773 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:44.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.880+0000 7f6b67f90640 1 Processor -- start 2024-04-25T08:39:44.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.880+0000 7f6b67f90640 1 -- start start 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.881+0000 7f6b67f90640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6b600b5710 0x7f6b60161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.881+0000 7f6b67f90640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6b6015ddf0 0x7f6b600b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.881+0000 7f6b67f90640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b600b5af0 0x7f6b6006bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.881+0000 7f6b67f90640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b600b9930 con 0x7f6b6015ddf0 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.881+0000 7f6b67f90640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b60161e00 con 0x7f6b600b5710 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.881+0000 7f6b67f90640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b601620e0 con 0x7f6b600b5af0 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.881+0000 7f6b6552b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6b6015ddf0 0x7f6b600b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.881+0000 7f6b6552b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6b6015ddf0 0x7f6b600b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39038/0 (socket says 172.21.15.38:39038) 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.881+0000 7f6b6552b640 1 -- 172.21.15.38:0/2416783524 learned_addr learned my addr 172.21.15.38:0/2416783524 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:44.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.881+0000 7f6b6652d640 1 --2- 172.21.15.38:0/2416783524 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b600b5af0 0x7f6b6006bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.881+0000 7f6b65d2c640 1 --2- 172.21.15.38:0/2416783524 >> v2:172.21.15.107:3300/0 conn(0x7f6b600b5710 0x7f6b60161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.882+0000 7f6b6552b640 1 -- 172.21.15.38:0/2416783524 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b600b5af0 msgr2=0x7f6b6006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.882+0000 7f6b6552b640 1 --2- 172.21.15.38:0/2416783524 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b600b5af0 0x7f6b6006bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.882+0000 7f6b6552b640 1 -- 172.21.15.38:0/2416783524 >> v2:172.21.15.107:3300/0 conn(0x7f6b600b5710 msgr2=0x7f6b60161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.882+0000 7f6b6552b640 1 --2- 172.21.15.38:0/2416783524 >> v2:172.21.15.107:3300/0 conn(0x7f6b600b5710 0x7f6b60161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.882+0000 7f6b6552b640 1 -- 172.21.15.38:0/2416783524 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b601623c0 con 0x7f6b6015ddf0 2024-04-25T08:39:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.882+0000 7f6b6552b640 1 --2- 172.21.15.38:0/2416783524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b6015ddf0 0x7f6b600b93f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f6b6008be40 tx=0x7f6b5c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d90b834ba19d54e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:44.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.882+0000 7f6b64d2a640 1 -- 172.21.15.38:0/2416783524 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c037070 con 0x7f6b6015ddf0 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.883+0000 7f6b64d2a640 1 -- 172.21.15.38:0/2416783524 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c032440 con 0x7f6b6015ddf0 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.883+0000 7f6b64d2a640 1 -- 172.21.15.38:0/2416783524 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c032810 con 0x7f6b6015ddf0 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.883+0000 7f6b67f90640 1 -- 172.21.15.38:0/2416783524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b6015ddf0 msgr2=0x7f6b600b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.883+0000 7f6b67f90640 1 --2- 172.21.15.38:0/2416783524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b6015ddf0 0x7f6b600b93f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f6b6008be40 tx=0x7f6b5c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.883+0000 7f6b67f90640 1 -- 172.21.15.38:0/2416783524 shutdown_connections 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.883+0000 7f6b67f90640 1 --2- 172.21.15.38:0/2416783524 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b600b5af0 0x7f6b6006bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.883+0000 7f6b67f90640 1 --2- 172.21.15.38:0/2416783524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b6015ddf0 0x7f6b600b93f0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.883+0000 7f6b67f90640 1 --2- 172.21.15.38:0/2416783524 >> v2:172.21.15.107:3300/0 conn(0x7f6b600b5710 0x7f6b60161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.883+0000 7f6b67f90640 1 -- 172.21.15.38:0/2416783524 >> 172.21.15.38:0/2416783524 conn(0x7f6b600d07c0 msgr2=0x7f6b600d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.883+0000 7f6b67f90640 1 -- 172.21.15.38:0/2416783524 shutdown_connections 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.883+0000 7f6b67f90640 1 -- 172.21.15.38:0/2416783524 wait complete. 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.884+0000 7f6b67f90640 1 Processor -- start 2024-04-25T08:39:44.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.884+0000 7f6b67f90640 1 -- start start 2024-04-25T08:39:44.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b67f90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b600b5710 0x7f6b601593f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b67f90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600b5af0 0x7f6b600c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b67f90640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b6015ddf0 0x7f6b600ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b67f90640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6006c6b0 con 0x7f6b600b5710 2024-04-25T08:39:44.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b67f90640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6006c530 con 0x7f6b600b5af0 2024-04-25T08:39:44.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b67f90640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6006c830 con 0x7f6b6015ddf0 2024-04-25T08:39:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b65d2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b600b5710 0x7f6b601593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b65d2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b600b5710 0x7f6b601593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39044/0 (socket says 172.21.15.38:39044) 2024-04-25T08:39:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b65d2c640 1 -- 172.21.15.38:0/2527705501 learned_addr learned my addr 172.21.15.38:0/2527705501 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b6652d640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b6015ddf0 0x7f6b600ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b6552b640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600b5af0 0x7f6b600c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b65d2c640 1 -- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b6015ddf0 msgr2=0x7f6b600ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b65d2c640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b6015ddf0 0x7f6b600ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b65d2c640 1 -- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600b5af0 msgr2=0x7f6b600c8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:44.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b65d2c640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600b5af0 0x7f6b600c8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b65d2c640 1 -- 172.21.15.38:0/2527705501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b600c9c70 con 0x7f6b600b5710 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b6652d640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b6015ddf0 0x7f6b600ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.885+0000 7f6b6552b640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600b5af0 0x7f6b600c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.886+0000 7f6b65d2c640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b600b5710 0x7f6b601593f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000e8c0 tx=0x7f6b5000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.886+0000 7f6b567fc640 1 -- 172.21.15.38:0/2527705501 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b50017070 con 0x7f6b600b5710 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.886+0000 7f6b567fc640 1 -- 172.21.15.38:0/2527705501 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5000c940 con 0x7f6b600b5710 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.886+0000 7f6b67f90640 1 -- 172.21.15.38:0/2527705501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b600c93f0 con 0x7f6b600b5710 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.886+0000 7f6b567fc640 1 -- 172.21.15.38:0/2527705501 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5000cd10 con 0x7f6b600b5710 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.886+0000 7f6b67f90640 1 -- 172.21.15.38:0/2527705501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b600cc0c0 con 0x7f6b600b5710 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.887+0000 7f6b67f90640 1 -- 172.21.15.38:0/2527705501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b60068e80 con 0x7f6b600b5710 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.887+0000 7f6b567fc640 1 -- 172.21.15.38:0/2527705501 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b5001a070 con 0x7f6b600b5710 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.888+0000 7f6b567fc640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b4406b4d0 0x7f6b4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.888+0000 7f6b567fc640 1 -- 172.21.15.38:0/2527705501 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f6b5008a350 con 0x7f6b600b5710 2024-04-25T08:39:44.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.888+0000 7f6b6552b640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b4406b4d0 0x7f6b4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:44.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.888+0000 7f6b6552b640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b4406b4d0 0x7f6b4406d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c034000 tx=0x7f6b5c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:44.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:44.892+0000 7f6b567fc640 1 -- 172.21.15.38:0/2527705501 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b50059010 con 0x7f6b600b5710 2024-04-25T08:39:45.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.111+0000 7f6b67f90640 1 -- 172.21.15.38:0/2527705501 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b600ccaa0 con 0x7f6b600b5710 2024-04-25T08:39:45.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.112+0000 7f6b567fc640 1 -- 172.21.15.38:0/2527705501 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f6b50058910 con 0x7f6b600b5710 2024-04-25T08:39:45.105 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:45.105 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":63,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:07.551641+0000","modified":"2024-04-25T08:39:09.555958+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8292},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8292":{"gid":8292,"name":"d","rank":0,"incarnation":61,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/1263010516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1263010516},{"type":"v1","addr":"172.21.15.107:6837","nonce":1263010516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T08:39:45.106 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 63 2024-04-25T08:39:45.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.114+0000 7f6b67f90640 1 -- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b4406b4d0 msgr2=0x7f6b4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.114+0000 7f6b67f90640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b4406b4d0 0x7f6b4406d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c034000 tx=0x7f6b5c02e000 comp rx=0 tx=0).stop 2024-04-25T08:39:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.114+0000 7f6b67f90640 1 -- 172.21.15.38:0/2527705501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b600b5710 msgr2=0x7f6b601593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.114+0000 7f6b67f90640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b600b5710 0x7f6b601593f0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000e8c0 tx=0x7f6b5000ed80 comp rx=0 tx=0).stop 2024-04-25T08:39:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.114+0000 7f6b67f90640 1 -- 172.21.15.38:0/2527705501 shutdown_connections 2024-04-25T08:39:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.114+0000 7f6b67f90640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b4406b4d0 0x7f6b4406d990 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.114+0000 7f6b67f90640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b6015ddf0 0x7f6b600ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.114+0000 7f6b67f90640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b600b5af0 0x7f6b600c8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.114+0000 7f6b67f90640 1 --2- 172.21.15.38:0/2527705501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b600b5710 0x7f6b601593f0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.114+0000 7f6b67f90640 1 -- 172.21.15.38:0/2527705501 >> 172.21.15.38:0/2527705501 conn(0x7f6b600d07c0 msgr2=0x7f6b600d2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.114+0000 7f6b67f90640 1 -- 172.21.15.38:0/2527705501 shutdown_connections 2024-04-25T08:39:45.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.114+0000 7f6b67f90640 1 -- 172.21.15.38:0/2527705501 wait complete. 2024-04-25T08:39:45.118 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:45.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.226+0000 7fe1bd6b3640 1 Processor -- start 2024-04-25T08:39:45.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.226+0000 7fe1bd6b3640 1 -- start start 2024-04-25T08:39:45.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.227+0000 7fe1bd6b3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe1b806c110 0x7fe1b806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.227+0000 7fe1bd6b3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b806ca30 con 0x7fe1b815df50 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.227+0000 7fe1bd6b3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b80ceff0 con 0x7fe1b80ce2e0 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.227+0000 7fe1bd6b3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b80cf2d0 con 0x7fe1b806c110 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.227+0000 7fe1b6ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe1b806c110 0x7fe1b806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.227+0000 7fe1b67fc640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe1b815df50 0x7fe1b80ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:55726/0 (socket says 172.21.15.38:55726) 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.227+0000 7fe1b67fc640 1 -- 172.21.15.38:0/4209349931 learned_addr learned my addr 172.21.15.38:0/4209349931 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.227+0000 7fe1b6ffd640 1 -- 172.21.15.38:0/4209349931 >> v1:172.21.15.107:6789/0 conn(0x7fe1b80ce2e0 legacy=0x7fe1b80d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.227+0000 7fe1b6ffd640 1 -- 172.21.15.38:0/4209349931 >> v1:172.21.15.38:6789/0 conn(0x7fe1b815df50 legacy=0x7fe1b80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.227+0000 7fe1b5ffb640 1 -- 172.21.15.38:0/4209349931 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714401470 0 0) 0x7fe1b806ca30 con 0x7fe1b815df50 2024-04-25T08:39:45.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.227+0000 7fe1b6ffd640 1 -- 172.21.15.38:0/4209349931 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b80d1f70 con 0x7fe1b806c110 2024-04-25T08:39:45.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.228+0000 7fe1b6ffd640 1 --2- 172.21.15.38:0/4209349931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1b806c110 0x7fe1b806c4f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80097a0 tx=0x7fe1a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46d446b519c33f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:45.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.228+0000 7fe1b5ffb640 1 -- 172.21.15.38:0/4209349931 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8037070 con 0x7fe1b806c110 2024-04-25T08:39:45.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.228+0000 7fe1b5ffb640 1 -- 172.21.15.38:0/4209349931 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a8032440 con 0x7fe1b806c110 2024-04-25T08:39:45.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.228+0000 7fe1b5ffb640 1 -- 172.21.15.38:0/4209349931 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8032810 con 0x7fe1b806c110 2024-04-25T08:39:45.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.229+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/4209349931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1b806c110 msgr2=0x7fe1b806c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.229+0000 7fe1bd6b3640 1 --2- 172.21.15.38:0/4209349931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1b806c110 0x7fe1b806c4f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fe1a80097a0 tx=0x7fe1a802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.229+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/4209349931 shutdown_connections 2024-04-25T08:39:45.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.229+0000 7fe1bd6b3640 1 --2- 172.21.15.38:0/4209349931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1b806c110 0x7fe1b806c4f0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.229+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/4209349931 >> 172.21.15.38:0/4209349931 conn(0x7fe1b80d3fb0 msgr2=0x7fe1b80d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:45.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.229+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/4209349931 shutdown_connections 2024-04-25T08:39:45.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.229+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/4209349931 wait complete. 2024-04-25T08:39:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.230+0000 7fe1bd6b3640 1 Processor -- start 2024-04-25T08:39:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.230+0000 7fe1bd6b3640 1 -- start start 2024-04-25T08:39:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1bd6b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1b806c110 0x7fe1b81f0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1bd6b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1b80ce2e0 0x7fe1b81f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1bd6b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1b815df50 0x7fe1b80c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1bd6b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80d3160 con 0x7fe1b815df50 2024-04-25T08:39:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1bd6b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80d2530 con 0x7fe1b80ce2e0 2024-04-25T08:39:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1bd6b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b80d32e0 con 0x7fe1b806c110 2024-04-25T08:39:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1b77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1b815df50 0x7fe1b80c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1b77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1b815df50 0x7fe1b80c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39062/0 (socket says 172.21.15.38:39062) 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1b77fe640 1 -- 172.21.15.38:0/1621802704 learned_addr learned my addr 172.21.15.38:0/1621802704 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1b6ffd640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1b806c110 0x7fe1b81f0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1b67fc640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1b80ce2e0 0x7fe1b81f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1b77fe640 1 -- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1b806c110 msgr2=0x7fe1b81f0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1b77fe640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1b806c110 0x7fe1b81f0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1b77fe640 1 -- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1b80ce2e0 msgr2=0x7fe1b81f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1b77fe640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1b80ce2e0 0x7fe1b81f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1b77fe640 1 -- 172.21.15.38:0/1621802704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b80c8510 con 0x7fe1b815df50 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1b6ffd640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1b806c110 0x7fe1b81f0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.231+0000 7fe1b67fc640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1b80ce2e0 0x7fe1b81f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.232+0000 7fe1b77fe640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1b815df50 0x7fe1b80c8cd0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c7a0 tx=0x7fe1b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.232+0000 7fe19f7fe640 1 -- 172.21.15.38:0/1621802704 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0003b70 con 0x7fe1b815df50 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.232+0000 7fe19f7fe640 1 -- 172.21.15.38:0/1621802704 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b0003d30 con 0x7fe1b815df50 2024-04-25T08:39:45.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.232+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/1621802704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b81f4040 con 0x7fe1b815df50 2024-04-25T08:39:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.232+0000 7fe19f7fe640 1 -- 172.21.15.38:0/1621802704 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000aa90 con 0x7fe1b815df50 2024-04-25T08:39:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.232+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/1621802704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b80c87b0 con 0x7fe1b815df50 2024-04-25T08:39:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.233+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/1621802704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b80c89b0 con 0x7fe1b815df50 2024-04-25T08:39:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.233+0000 7fe19f7fe640 1 -- 172.21.15.38:0/1621802704 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1b000ac30 con 0x7fe1b815df50 2024-04-25T08:39:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.234+0000 7fe19f7fe640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe1a406b4d0 0x7fe1a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.234+0000 7fe19f7fe640 1 -- 172.21.15.38:0/1621802704 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe1b008a310 con 0x7fe1b815df50 2024-04-25T08:39:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.234+0000 7fe1b6ffd640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe1a406b4d0 0x7fe1a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.235+0000 7fe1b6ffd640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe1a406b4d0 0x7fe1a406d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8034000 tx=0x7fe1a802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:45.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.238+0000 7fe19f7fe640 1 -- 172.21.15.38:0/1621802704 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1b80c89b0 con 0x7fe1b815df50 2024-04-25T08:39:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.418+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/1621802704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1b8068e80 con 0x7fe1b815df50 2024-04-25T08:39:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.420+0000 7fe19f7fe640 1 -- 172.21.15.38:0/1621802704 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16970 (secure 0 0 0) 0x7fe1b0058fd0 con 0x7fe1b815df50 2024-04-25T08:39:45.413 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:45.413 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":110,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:18.940299+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:05.852531+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:06.527775+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1d","mappings":[{"from":1,"to":7}]},{"pgid":"16.1e","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:45.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.421+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe1a406b4d0 msgr2=0x7fe1a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.421+0000 7fe1bd6b3640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe1a406b4d0 0x7fe1a406d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe1a8034000 tx=0x7fe1a802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:45.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.421+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/1621802704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1b815df50 msgr2=0x7fe1b80c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.421+0000 7fe1bd6b3640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1b815df50 0x7fe1b80c8cd0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000c7a0 tx=0x7fe1b000cc60 comp rx=0 tx=0).stop 2024-04-25T08:39:45.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.421+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/1621802704 shutdown_connections 2024-04-25T08:39:45.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.421+0000 7fe1bd6b3640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1b815df50 0x7fe1b80c8cd0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.421+0000 7fe1bd6b3640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1b80ce2e0 0x7fe1b81f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.421+0000 7fe1bd6b3640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe1a406b4d0 0x7fe1a406d990 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.421+0000 7fe1bd6b3640 1 --2- 172.21.15.38:0/1621802704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1b806c110 0x7fe1b81f0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.421+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/1621802704 >> 172.21.15.38:0/1621802704 conn(0x7fe1b80d3fb0 msgr2=0x7fe1b806c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:45.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.422+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/1621802704 shutdown_connections 2024-04-25T08:39:45.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.422+0000 7fe1bd6b3640 1 -- 172.21.15.38:0/1621802704 wait complete. 2024-04-25T08:39:45.427 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:39:45.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.536+0000 7f5522141640 1 Processor -- start 2024-04-25T08:39:45.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.536+0000 7f5522141640 1 -- start start 2024-04-25T08:39:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f5522141640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f551c15a600 0x7f551c15ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f5522141640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f551c0ceba0 0x7f551c070f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f5522141640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551c0b2430 con 0x7f551c0714c0 2024-04-25T08:39:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f5522141640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f551c071d70 con 0x7f551c15a600 2024-04-25T08:39:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f5522141640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f551c0b26d0 con 0x7f551c0ceba0 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f551b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f551c15a600 0x7f551c15ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f551b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f551c15a600 0x7f551c15ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39064/0 (socket says 172.21.15.38:39064) 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f551b7fe640 1 -- 172.21.15.38:0/1054731711 learned_addr learned my addr 172.21.15.38:0/1054731711 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f5513fff640 1 --2- 172.21.15.38:0/1054731711 >> v2:172.21.15.107:3300/0 conn(0x7f551c0ceba0 0x7f551c070f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f551b7fe640 1 -- 172.21.15.38:0/1054731711 >> v2:172.21.15.107:3300/0 conn(0x7f551c0ceba0 msgr2=0x7f551c070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f551b7fe640 1 --2- 172.21.15.38:0/1054731711 >> v2:172.21.15.107:3300/0 conn(0x7f551c0ceba0 0x7f551c070f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f551b7fe640 1 -- 172.21.15.38:0/1054731711 >> v1:172.21.15.107:6789/0 conn(0x7f551c0714c0 legacy=0x7f551c0cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f551b7fe640 1 -- 172.21.15.38:0/1054731711 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c0b29b0 con 0x7f551c15a600 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.537+0000 7f5513fff640 1 --2- 172.21.15.38:0/1054731711 >> v2:172.21.15.107:3300/0 conn(0x7f551c0ceba0 0x7f551c070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.538+0000 7f551b7fe640 1 --2- 172.21.15.38:0/1054731711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f551c15a600 0x7f551c15ebd0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f551c0cd0d0 tx=0x7f550c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7da39e9b0f1dac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.538+0000 7f551affd640 1 -- 172.21.15.38:0/1054731711 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c037070 con 0x7f551c15a600 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.538+0000 7f551affd640 1 -- 172.21.15.38:0/1054731711 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550c032440 con 0x7f551c15a600 2024-04-25T08:39:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.538+0000 7f551affd640 1 -- 172.21.15.38:0/1054731711 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c032810 con 0x7f551c15a600 2024-04-25T08:39:45.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.539+0000 7f5522141640 1 -- 172.21.15.38:0/1054731711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f551c15a600 msgr2=0x7f551c15ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.539+0000 7f5522141640 1 --2- 172.21.15.38:0/1054731711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f551c15a600 0x7f551c15ebd0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f551c0cd0d0 tx=0x7f550c02a270 comp rx=0 tx=0).stop 2024-04-25T08:39:45.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.539+0000 7f5522141640 1 -- 172.21.15.38:0/1054731711 shutdown_connections 2024-04-25T08:39:45.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.539+0000 7f5522141640 1 --2- 172.21.15.38:0/1054731711 >> v2:172.21.15.107:3300/0 conn(0x7f551c0ceba0 0x7f551c070f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.539+0000 7f5522141640 1 --2- 172.21.15.38:0/1054731711 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f551c15a600 0x7f551c15ebd0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.539+0000 7f5522141640 1 -- 172.21.15.38:0/1054731711 >> 172.21.15.38:0/1054731711 conn(0x7f551c06abe0 msgr2=0x7f551c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:45.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.539+0000 7f5522141640 1 -- 172.21.15.38:0/1054731711 shutdown_connections 2024-04-25T08:39:45.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.539+0000 7f5522141640 1 -- 172.21.15.38:0/1054731711 wait complete. 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.540+0000 7f5522141640 1 Processor -- start 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.540+0000 7f5522141640 1 -- start start 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f5522141640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f551c0714c0 0x7f551c1e55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f5522141640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f551c0ceba0 0x7f551c1557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f5522141640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f551c15a600 0x7f551c0af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f5522141640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c0b3600 con 0x7f551c0714c0 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f5522141640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c06e380 con 0x7f551c0ceba0 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f5522141640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551c1e5b30 con 0x7f551c15a600 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f551b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f551c0714c0 0x7f551c1e55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f551b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f551c0714c0 0x7f551c1e55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39078/0 (socket says 172.21.15.38:39078) 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f551b7fe640 1 -- 172.21.15.38:0/1114599371 learned_addr learned my addr 172.21.15.38:0/1114599371 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f5513fff640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f551c0ceba0 0x7f551c1557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f551bfff640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f551c15a600 0x7f551c0af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f551b7fe640 1 -- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f551c15a600 msgr2=0x7f551c0af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f551b7fe640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f551c15a600 0x7f551c0af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f551b7fe640 1 -- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f551c0ceba0 msgr2=0x7f551c1557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f551b7fe640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f551c0ceba0 0x7f551c1557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f551b7fe640 1 -- 172.21.15.38:0/1114599371 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c0ad8b0 con 0x7f551c0714c0 2024-04-25T08:39:45.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f551b7fe640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f551c0714c0 0x7f551c1e55f0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f550c02a2b0 tx=0x7f550c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:45.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.541+0000 7f5518ff9640 1 -- 172.21.15.38:0/1114599371 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c041020 con 0x7f551c0714c0 2024-04-25T08:39:45.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.542+0000 7f5518ff9640 1 -- 172.21.15.38:0/1114599371 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550c0306c0 con 0x7f551c0714c0 2024-04-25T08:39:45.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.542+0000 7f5522141640 1 -- 172.21.15.38:0/1114599371 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551c0adb90 con 0x7f551c0714c0 2024-04-25T08:39:45.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.542+0000 7f5518ff9640 1 -- 172.21.15.38:0/1114599371 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c037040 con 0x7f551c0714c0 2024-04-25T08:39:45.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.542+0000 7f5522141640 1 -- 172.21.15.38:0/1114599371 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c0ae150 con 0x7f551c0714c0 2024-04-25T08:39:45.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.542+0000 7f5522141640 1 -- 172.21.15.38:0/1114599371 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551c15db40 con 0x7f551c0714c0 2024-04-25T08:39:45.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.543+0000 7f5518ff9640 1 -- 172.21.15.38:0/1114599371 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f550c044020 con 0x7f551c0714c0 2024-04-25T08:39:45.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.543+0000 7f5518ff9640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f54fc06b4d0 0x7f54fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.544+0000 7f5518ff9640 1 -- 172.21.15.38:0/1114599371 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f550c0aaa00 con 0x7f551c0714c0 2024-04-25T08:39:45.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.544+0000 7f5513fff640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f54fc06b4d0 0x7f54fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.544+0000 7f5513fff640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f54fc06b4d0 0x7f54fc06d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f551c159340 tx=0x7f5504008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:45.540 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.548+0000 7f5518ff9640 1 -- 172.21.15.38:0/1114599371 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f551c15db40 con 0x7f551c0714c0 2024-04-25T08:39:45.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.763+0000 7f5522141640 1 -- 172.21.15.38:0/1114599371 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f551c04ef90 con 0x7f551c0714c0 2024-04-25T08:39:45.757 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:39:45.764+0000 7ff7937f0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:39:45.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.765+0000 7f5518ff9640 1 -- 172.21.15.38:0/1114599371 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7f550c04b030 con 0x7f551c0714c0 2024-04-25T08:39:45.758 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.767+0000 7f5522141640 1 -- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f54fc06b4d0 msgr2=0x7f54fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.767+0000 7f5522141640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f54fc06b4d0 0x7f54fc06d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f551c159340 tx=0x7f5504008040 comp rx=0 tx=0).stop 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.767+0000 7f5522141640 1 -- 172.21.15.38:0/1114599371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f551c0714c0 msgr2=0x7f551c1e55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.767+0000 7f5522141640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f551c0714c0 0x7f551c1e55f0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f550c02a2b0 tx=0x7f550c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.767+0000 7f5522141640 1 -- 172.21.15.38:0/1114599371 shutdown_connections 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.767+0000 7f5522141640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f54fc06b4d0 0x7f54fc06d990 secure :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f551c159340 tx=0x7f5504008040 comp rx=0 tx=0).stop 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.767+0000 7f5522141640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f551c15a600 0x7f551c0af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.767+0000 7f5522141640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f551c0ceba0 0x7f551c1557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.767+0000 7f5522141640 1 --2- 172.21.15.38:0/1114599371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f551c0714c0 0x7f551c1e55f0 secure :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f550c02a2b0 tx=0x7f550c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.767+0000 7f5522141640 1 -- 172.21.15.38:0/1114599371 >> 172.21.15.38:0/1114599371 conn(0x7f551c06abe0 msgr2=0x7f551c0b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.768+0000 7f5522141640 1 -- 172.21.15.38:0/1114599371 shutdown_connections 2024-04-25T08:39:45.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.768+0000 7f5522141640 1 -- 172.21.15.38:0/1114599371 wait complete. 2024-04-25T08:39:45.772 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:39:45.802 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:39:45.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.883+0000 7fee0c007640 1 Processor -- start 2024-04-25T08:39:45.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.883+0000 7fee0c007640 1 -- start start 2024-04-25T08:39:45.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.884+0000 7fee0c007640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee040d0320 0x7fee040d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.884+0000 7fee0c007640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fee040b6980 0x7fee040b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.884+0000 7fee0c007640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee040d2060 con 0x7fee040d0700 2024-04-25T08:39:45.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.884+0000 7fee0c007640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee040d2330 con 0x7fee040b6980 2024-04-25T08:39:45.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.884+0000 7fee0c007640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee040d2610 con 0x7fee040d0320 2024-04-25T08:39:45.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.884+0000 7fee09da3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee040d0320 0x7fee040d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.884+0000 7fee0a5a4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fee040b6980 0x7fee040b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.884+0000 7fee095a2640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fee040d0700 0x7fee040d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:33322/0 (socket says 172.21.15.38:33322) 2024-04-25T08:39:45.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.884+0000 7fee095a2640 1 -- 172.21.15.38:0/1686016906 learned_addr learned my addr 172.21.15.38:0/1686016906 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:45.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.885+0000 7fee09da3640 1 -- 172.21.15.38:0/1686016906 >> v2:172.21.15.107:3300/0 conn(0x7fee040b6980 msgr2=0x7fee040b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:45.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.885+0000 7fee09da3640 1 --2- 172.21.15.38:0/1686016906 >> v2:172.21.15.107:3300/0 conn(0x7fee040b6980 0x7fee040b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.885+0000 7fee09da3640 1 -- 172.21.15.38:0/1686016906 >> v1:172.21.15.107:6789/0 conn(0x7fee040d0700 legacy=0x7fee040d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.885+0000 7fee09da3640 1 -- 172.21.15.38:0/1686016906 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee040d28f0 con 0x7fee040d0320 2024-04-25T08:39:45.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.885+0000 7fee09da3640 1 --2- 172.21.15.38:0/1686016906 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee040d0320 0x7fee040d33d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0097a0 tx=0x7fedfc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff6036b60dd30c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:45.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.885+0000 7fee08da1640 1 -- 172.21.15.38:0/1686016906 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc037070 con 0x7fee040d0320 2024-04-25T08:39:45.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.886+0000 7fee08da1640 1 -- 172.21.15.38:0/1686016906 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedfc032440 con 0x7fee040d0320 2024-04-25T08:39:45.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.886+0000 7fee08da1640 1 -- 172.21.15.38:0/1686016906 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedfc032810 con 0x7fee040d0320 2024-04-25T08:39:45.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.886+0000 7fee0c007640 1 -- 172.21.15.38:0/1686016906 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee040d0320 msgr2=0x7fee040d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.886+0000 7fee0c007640 1 --2- 172.21.15.38:0/1686016906 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee040d0320 0x7fee040d33d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fedfc0097a0 tx=0x7fedfc02a210 comp rx=0 tx=0).stop 2024-04-25T08:39:45.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.886+0000 7fee0c007640 1 -- 172.21.15.38:0/1686016906 shutdown_connections 2024-04-25T08:39:45.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.886+0000 7fee0c007640 1 --2- 172.21.15.38:0/1686016906 >> v2:172.21.15.107:3300/0 conn(0x7fee040b6980 0x7fee040b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.886+0000 7fee0c007640 1 --2- 172.21.15.38:0/1686016906 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee040d0320 0x7fee040d33d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.886+0000 7fee0c007640 1 -- 172.21.15.38:0/1686016906 >> 172.21.15.38:0/1686016906 conn(0x7fee040668f0 msgr2=0x7fee04066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:45.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.886+0000 7fee0c007640 1 -- 172.21.15.38:0/1686016906 shutdown_connections 2024-04-25T08:39:45.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.886+0000 7fee0c007640 1 -- 172.21.15.38:0/1686016906 wait complete. 2024-04-25T08:39:45.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.887+0000 7fee0c007640 1 Processor -- start 2024-04-25T08:39:45.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.887+0000 7fee0c007640 1 -- start start 2024-04-25T08:39:45.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.888+0000 7fee0c007640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee040b6980 0x7fee0415a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.888+0000 7fee0c007640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee040d0320 0x7fee041f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.888+0000 7fee0c007640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee040d0700 0x7fee040b58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.888+0000 7fee0c007640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee040d0b40 con 0x7fee040d0320 2024-04-25T08:39:45.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.888+0000 7fee0c007640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee040d2eb0 con 0x7fee040b6980 2024-04-25T08:39:45.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.888+0000 7fee0c007640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee040b3410 con 0x7fee040d0700 2024-04-25T08:39:45.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.888+0000 7fee095a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee040d0320 0x7fee041f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.888+0000 7fee095a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee040d0320 0x7fee041f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39090/0 (socket says 172.21.15.38:39090) 2024-04-25T08:39:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.888+0000 7fee095a2640 1 -- 172.21.15.38:0/2028498072 learned_addr learned my addr 172.21.15.38:0/2028498072 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.888+0000 7fee0a5a4640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee040d0700 0x7fee040b58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.888+0000 7fee09da3640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee040b6980 0x7fee0415a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.889+0000 7fee095a2640 1 -- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee040d0700 msgr2=0x7fee040b58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.889+0000 7fee095a2640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee040d0700 0x7fee040b58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.889+0000 7fee095a2640 1 -- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee040b6980 msgr2=0x7fee0415a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.889+0000 7fee095a2640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee040b6980 0x7fee0415a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.889+0000 7fee095a2640 1 -- 172.21.15.38:0/2028498072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee040b36b0 con 0x7fee040d0320 2024-04-25T08:39:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.889+0000 7fee095a2640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee040d0320 0x7fee041f36a0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c3c0 tx=0x7fedf800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.889+0000 7fee027fc640 1 -- 172.21.15.38:0/2028498072 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8017070 con 0x7fee040d0320 2024-04-25T08:39:45.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.889+0000 7fee027fc640 1 -- 172.21.15.38:0/2028498072 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf8005ce0 con 0x7fee040d0320 2024-04-25T08:39:45.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.889+0000 7fee0c007640 1 -- 172.21.15.38:0/2028498072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee040b3990 con 0x7fee040d0320 2024-04-25T08:39:45.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.889+0000 7fee027fc640 1 -- 172.21.15.38:0/2028498072 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf80072f0 con 0x7fee040d0320 2024-04-25T08:39:45.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.889+0000 7fee0c007640 1 -- 172.21.15.38:0/2028498072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee040b3f50 con 0x7fee040d0320 2024-04-25T08:39:45.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.890+0000 7fee0c007640 1 -- 172.21.15.38:0/2028498072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee04068e80 con 0x7fee040d0320 2024-04-25T08:39:45.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.890+0000 7fee027fc640 1 -- 172.21.15.38:0/2028498072 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fedf801a070 con 0x7fee040d0320 2024-04-25T08:39:45.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.891+0000 7fee027fc640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fedf006b4d0 0x7fedf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:45.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.891+0000 7fee027fc640 1 -- 172.21.15.38:0/2028498072 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fedf80896d0 con 0x7fee040d0320 2024-04-25T08:39:45.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.891+0000 7fee09da3640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fedf006b4d0 0x7fedf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:45.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.892+0000 7fee09da3640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fedf006b4d0 0x7fedf006d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fedfc034000 tx=0x7fedfc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:45.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:45.895+0000 7fee027fc640 1 -- 172.21.15.38:0/2028498072 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fedf8057d70 con 0x7fee040d0320 2024-04-25T08:39:46.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.115+0000 7fee0c007640 1 -- 172.21.15.38:0/2028498072 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fee040b4120 con 0x7fee040d0320 2024-04-25T08:39:46.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.771+0000 7fee027fc640 1 -- 172.21.15.38:0/2028498072 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7fedf805be10 con 0x7fee040d0320 2024-04-25T08:39:46.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.774+0000 7fee0c007640 1 -- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fedf006b4d0 msgr2=0x7fedf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.774+0000 7fee0c007640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fedf006b4d0 0x7fedf006d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fedfc034000 tx=0x7fedfc02e000 comp rx=0 tx=0).stop 2024-04-25T08:39:46.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.774+0000 7fee0c007640 1 -- 172.21.15.38:0/2028498072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee040d0320 msgr2=0x7fee041f36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.774+0000 7fee0c007640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee040d0320 0x7fee041f36a0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c3c0 tx=0x7fedf800c880 comp rx=0 tx=0).stop 2024-04-25T08:39:46.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.774+0000 7fee0c007640 1 -- 172.21.15.38:0/2028498072 shutdown_connections 2024-04-25T08:39:46.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.774+0000 7fee0c007640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fedf006b4d0 0x7fedf006d990 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.774+0000 7fee0c007640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fee040d0700 0x7fee040b58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.774+0000 7fee0c007640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fee040d0320 0x7fee041f36a0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.774+0000 7fee0c007640 1 --2- 172.21.15.38:0/2028498072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee040b6980 0x7fee0415a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.774+0000 7fee0c007640 1 -- 172.21.15.38:0/2028498072 >> 172.21.15.38:0/2028498072 conn(0x7fee040668f0 msgr2=0x7fee0415fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:46.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.774+0000 7fee0c007640 1 -- 172.21.15.38:0/2028498072 shutdown_connections 2024-04-25T08:39:46.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.774+0000 7fee0c007640 1 -- 172.21.15.38:0/2028498072 wait complete. 2024-04-25T08:39:46.781 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:39:46.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.900+0000 7fce87a41640 1 Processor -- start 2024-04-25T08:39:46.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.900+0000 7fce87a41640 1 -- start start 2024-04-25T08:39:46.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce87a41640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fce8006c6e0 0x7fce8006cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce87a41640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fce8015d330 0x7fce8015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce87a41640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce800d0c70 con 0x7fce800cd810 2024-04-25T08:39:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce87a41640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8009aaf0 con 0x7fce8015d330 2024-04-25T08:39:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce87a41640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8006d060 con 0x7fce8006c6e0 2024-04-25T08:39:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce85fde640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fce8015d330 0x7fce8015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce85fde640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fce8015d330 0x7fce8015d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39096/0 (socket says 172.21.15.38:39096) 2024-04-25T08:39:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce85fde640 1 -- 172.21.15.38:0/2651973219 learned_addr learned my addr 172.21.15.38:0/2651973219 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce84fdc640 1 --2- 172.21.15.38:0/2651973219 >> v2:172.21.15.107:3300/0 conn(0x7fce8006c6e0 0x7fce8006cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce85fde640 1 -- 172.21.15.38:0/2651973219 >> v2:172.21.15.107:3300/0 conn(0x7fce8006c6e0 msgr2=0x7fce8006cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce85fde640 1 --2- 172.21.15.38:0/2651973219 >> v2:172.21.15.107:3300/0 conn(0x7fce8006c6e0 0x7fce8006cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce85fde640 1 -- 172.21.15.38:0/2651973219 >> v1:172.21.15.107:6789/0 conn(0x7fce800cd810 legacy=0x7fce800d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.901+0000 7fce85fde640 1 -- 172.21.15.38:0/2651973219 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce80099ee0 con 0x7fce8015d330 2024-04-25T08:39:46.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.902+0000 7fce85fde640 1 --2- 172.21.15.38:0/2651973219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8015d330 0x7fce8015d7c0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fce800cbc40 tx=0x7fce7802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d724da600c8d107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:46.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.902+0000 7fce77fff640 1 -- 172.21.15.38:0/2651973219 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78036070 con 0x7fce8015d330 2024-04-25T08:39:46.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.902+0000 7fce77fff640 1 -- 172.21.15.38:0/2651973219 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce78031600 con 0x7fce8015d330 2024-04-25T08:39:46.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.902+0000 7fce77fff640 1 -- 172.21.15.38:0/2651973219 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce780319f0 con 0x7fce8015d330 2024-04-25T08:39:46.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.903+0000 7fce87a41640 1 -- 172.21.15.38:0/2651973219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8015d330 msgr2=0x7fce8015d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.903+0000 7fce87a41640 1 --2- 172.21.15.38:0/2651973219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8015d330 0x7fce8015d7c0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fce800cbc40 tx=0x7fce7802a7c0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.903+0000 7fce87a41640 1 -- 172.21.15.38:0/2651973219 shutdown_connections 2024-04-25T08:39:46.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.903+0000 7fce87a41640 1 --2- 172.21.15.38:0/2651973219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8015d330 0x7fce8015d7c0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.903+0000 7fce87a41640 1 --2- 172.21.15.38:0/2651973219 >> v2:172.21.15.107:3300/0 conn(0x7fce8006c6e0 0x7fce8006cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.903+0000 7fce87a41640 1 -- 172.21.15.38:0/2651973219 >> 172.21.15.38:0/2651973219 conn(0x7fce800b31b0 msgr2=0x7fce800b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:46.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.903+0000 7fce87a41640 1 -- 172.21.15.38:0/2651973219 shutdown_connections 2024-04-25T08:39:46.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.903+0000 7fce87a41640 1 -- 172.21.15.38:0/2651973219 wait complete. 2024-04-25T08:39:46.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.904+0000 7fce87a41640 1 Processor -- start 2024-04-25T08:39:46.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.904+0000 7fce87a41640 1 -- start start 2024-04-25T08:39:46.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.905+0000 7fce87a41640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8006c6e0 0x7fce800707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.905+0000 7fce87a41640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce800cd810 0x7fce800c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.905+0000 7fce87a41640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce8015d330 0x7fce801f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.905+0000 7fce87a41640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8009a780 con 0x7fce8006c6e0 2024-04-25T08:39:46.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.905+0000 7fce87a41640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8009a4a0 con 0x7fce8015d330 2024-04-25T08:39:46.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.905+0000 7fce87a41640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8009af80 con 0x7fce800cd810 2024-04-25T08:39:46.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.905+0000 7fce857dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8006c6e0 0x7fce800707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.905+0000 7fce857dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8006c6e0 0x7fce800707e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39104/0 (socket says 172.21.15.38:39104) 2024-04-25T08:39:46.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.905+0000 7fce857dd640 1 -- 172.21.15.38:0/2756157556 learned_addr learned my addr 172.21.15.38:0/2756157556 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:46.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.905+0000 7fce85fde640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce8015d330 0x7fce801f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.905+0000 7fce84fdc640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce800cd810 0x7fce800c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.906+0000 7fce857dd640 1 -- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce800cd810 msgr2=0x7fce800c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.906+0000 7fce857dd640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce800cd810 0x7fce800c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.906+0000 7fce857dd640 1 -- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce8015d330 msgr2=0x7fce801f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:46.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.906+0000 7fce857dd640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce8015d330 0x7fce801f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:46.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.906+0000 7fce857dd640 1 -- 172.21.15.38:0/2756157556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce801f4010 con 0x7fce8006c6e0 2024-04-25T08:39:46.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.906+0000 7fce84fdc640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce800cd810 0x7fce800c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:46.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.906+0000 7fce857dd640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8006c6e0 0x7fce800707e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00c400 tx=0x7fce7c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:46.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.906+0000 7fce75ffb640 1 -- 172.21.15.38:0/2756157556 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c0052d0 con 0x7fce8006c6e0 2024-04-25T08:39:46.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.906+0000 7fce75ffb640 1 -- 172.21.15.38:0/2756157556 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c005490 con 0x7fce8006c6e0 2024-04-25T08:39:46.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.906+0000 7fce87a41640 1 -- 172.21.15.38:0/2756157556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce801f42f0 con 0x7fce8006c6e0 2024-04-25T08:39:46.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.906+0000 7fce75ffb640 1 -- 172.21.15.38:0/2756157556 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c007260 con 0x7fce8006c6e0 2024-04-25T08:39:46.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.906+0000 7fce87a41640 1 -- 172.21.15.38:0/2756157556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce800c8c10 con 0x7fce8006c6e0 2024-04-25T08:39:46.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.907+0000 7fce87a41640 1 -- 172.21.15.38:0/2756157556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce80068e80 con 0x7fce8006c6e0 2024-04-25T08:39:46.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.908+0000 7fce75ffb640 1 -- 172.21.15.38:0/2756157556 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce7c007400 con 0x7fce8006c6e0 2024-04-25T08:39:46.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.908+0000 7fce75ffb640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce6406b500 0x7fce6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:46.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.908+0000 7fce75ffb640 1 -- 172.21.15.38:0/2756157556 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fce7c0897c0 con 0x7fce8006c6e0 2024-04-25T08:39:46.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.908+0000 7fce84fdc640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce6406b500 0x7fce6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:46.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.909+0000 7fce84fdc640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce6406b500 0x7fce6406d9c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fce801f1160 tx=0x7fce70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:46.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:46.912+0000 7fce75ffb640 1 -- 172.21.15.38:0/2756157556 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce7c057e60 con 0x7fce8006c6e0 2024-04-25T08:39:47.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:47.095+0000 7fce87a41640 1 -- 172.21.15.38:0/2756157556 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fce800ccc10 con 0x7fce8006c6e0 2024-04-25T08:39:48.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.042+0000 7fce75ffb640 1 -- 172.21.15.38:0/2756157556 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) v1 ==== 181+0+0 (secure 0 0 0) 0x7fce7c05ce20 con 0x7fce8006c6e0 2024-04-25T08:39:48.035 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:39:48.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.044+0000 7fce87a41640 1 -- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce6406b500 msgr2=0x7fce6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.044+0000 7fce87a41640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce6406b500 0x7fce6406d9c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fce801f1160 tx=0x7fce70008040 comp rx=0 tx=0).stop 2024-04-25T08:39:48.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.045+0000 7fce87a41640 1 -- 172.21.15.38:0/2756157556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8006c6e0 msgr2=0x7fce800707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.045+0000 7fce87a41640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8006c6e0 0x7fce800707e0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00c400 tx=0x7fce7c00c8c0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.045+0000 7fce87a41640 1 -- 172.21.15.38:0/2756157556 shutdown_connections 2024-04-25T08:39:48.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.045+0000 7fce87a41640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce8015d330 0x7fce801f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.045+0000 7fce87a41640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce6406b500 0x7fce6406d9c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.045+0000 7fce87a41640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce800cd810 0x7fce800c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.045+0000 7fce87a41640 1 --2- 172.21.15.38:0/2756157556 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce8006c6e0 0x7fce800707e0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.045+0000 7fce87a41640 1 -- 172.21.15.38:0/2756157556 >> 172.21.15.38:0/2756157556 conn(0x7fce800b31b0 msgr2=0x7fce800d01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:48.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.045+0000 7fce87a41640 1 -- 172.21.15.38:0/2756157556 shutdown_connections 2024-04-25T08:39:48.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.045+0000 7fce87a41640 1 -- 172.21.15.38:0/2756157556 wait complete. 2024-04-25T08:39:48.054 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:39:48.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.172+0000 7fe39c529640 1 Processor -- start 2024-04-25T08:39:48.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.172+0000 7fe39c529640 1 -- start start 2024-04-25T08:39:48.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe39c529640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe3940b93f0 0x7fe3940d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe39c529640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3940d0870 0x7fe3940d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe39c529640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3940d11c0 con 0x7fe3941612c0 2024-04-25T08:39:48.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe39c529640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3940b9880 con 0x7fe3940b93f0 2024-04-25T08:39:48.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe39c529640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3940b7200 con 0x7fe3940d0870 2024-04-25T08:39:48.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe39aac6640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3940d0870 0x7fe3940d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe399ac4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe3940b93f0 0x7fe3940d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe39a2c5640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fe3941612c0 0x7fe3940b6a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:33336/0 (socket says 172.21.15.38:33336) 2024-04-25T08:39:48.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe399ac4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe3940b93f0 0x7fe3940d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:59860/0 (socket says 172.21.15.38:59860) 2024-04-25T08:39:48.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe39a2c5640 1 -- 172.21.15.38:0/689823699 learned_addr learned my addr 172.21.15.38:0/689823699 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:48.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe399ac4640 1 -- 172.21.15.38:0/689823699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3940d0870 msgr2=0x7fe3940d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:48.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe399ac4640 1 --2- 172.21.15.38:0/689823699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3940d0870 0x7fe3940d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.173+0000 7fe399ac4640 1 -- 172.21.15.38:0/689823699 >> v1:172.21.15.107:6789/0 conn(0x7fe3941612c0 legacy=0x7fe3940b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.174+0000 7fe3992c3640 1 -- 172.21.15.38:0/689823699 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028370794 0 0) 0x7fe390001110 con 0x7fe3941612c0 2024-04-25T08:39:48.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.174+0000 7fe399ac4640 1 -- 172.21.15.38:0/689823699 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3940b74e0 con 0x7fe3940b93f0 2024-04-25T08:39:48.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.174+0000 7fe399ac4640 1 --2- 172.21.15.38:0/689823699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3940b93f0 0x7fe3940d1c30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe3840097a0 tx=0x7fe38402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=764bf20e8ed32547 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:48.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.174+0000 7fe3992c3640 1 -- 172.21.15.38:0/689823699 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384037070 con 0x7fe3940b93f0 2024-04-25T08:39:48.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.174+0000 7fe3992c3640 1 -- 172.21.15.38:0/689823699 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38402ae50 con 0x7fe3940b93f0 2024-04-25T08:39:48.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.175+0000 7fe3992c3640 1 -- 172.21.15.38:0/689823699 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384032660 con 0x7fe3940b93f0 2024-04-25T08:39:48.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.175+0000 7fe39c529640 1 -- 172.21.15.38:0/689823699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3940b93f0 msgr2=0x7fe3940d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.175+0000 7fe39c529640 1 --2- 172.21.15.38:0/689823699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3940b93f0 0x7fe3940d1c30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe3840097a0 tx=0x7fe38402a100 comp rx=0 tx=0).stop 2024-04-25T08:39:48.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.175+0000 7fe39c529640 1 -- 172.21.15.38:0/689823699 shutdown_connections 2024-04-25T08:39:48.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.175+0000 7fe39c529640 1 --2- 172.21.15.38:0/689823699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3940d0870 0x7fe3940d0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.175+0000 7fe39c529640 1 --2- 172.21.15.38:0/689823699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3940b93f0 0x7fe3940d1c30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.175+0000 7fe39c529640 1 -- 172.21.15.38:0/689823699 >> 172.21.15.38:0/689823699 conn(0x7fe394066d10 msgr2=0x7fe394067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:48.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.175+0000 7fe39c529640 1 -- 172.21.15.38:0/689823699 shutdown_connections 2024-04-25T08:39:48.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.175+0000 7fe39c529640 1 -- 172.21.15.38:0/689823699 wait complete. 2024-04-25T08:39:48.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.176+0000 7fe39c529640 1 Processor -- start 2024-04-25T08:39:48.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.176+0000 7fe39c529640 1 -- start start 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.177+0000 7fe39c529640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3940b93f0 0x7fe3940c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.177+0000 7fe39c529640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3940d0870 0x7fe394145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.177+0000 7fe39c529640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3941612c0 0x7fe394146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.177+0000 7fe39c529640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3940b7d80 con 0x7fe3940d0870 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.177+0000 7fe39c529640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3940b7aa0 con 0x7fe3941612c0 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.177+0000 7fe39c529640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3940d2170 con 0x7fe3940b93f0 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.177+0000 7fe399ac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3940d0870 0x7fe394145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.177+0000 7fe399ac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3940d0870 0x7fe394145d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39106/0 (socket says 172.21.15.38:39106) 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.177+0000 7fe399ac4640 1 -- 172.21.15.38:0/1636790810 learned_addr learned my addr 172.21.15.38:0/1636790810 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.177+0000 7fe39aac6640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3941612c0 0x7fe394146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.177+0000 7fe39a2c5640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3940b93f0 0x7fe3940c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.178+0000 7fe399ac4640 1 -- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3940b93f0 msgr2=0x7fe3940c9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.178+0000 7fe399ac4640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3940b93f0 0x7fe3940c9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.178+0000 7fe399ac4640 1 -- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3941612c0 msgr2=0x7fe394146290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:48.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.178+0000 7fe399ac4640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3941612c0 0x7fe394146290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:48.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.178+0000 7fe399ac4640 1 -- 172.21.15.38:0/1636790810 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39415b6d0 con 0x7fe3940d0870 2024-04-25T08:39:48.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.178+0000 7fe39a2c5640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3940b93f0 0x7fe3940c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:48.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.178+0000 7fe399ac4640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3940d0870 0x7fe394145d50 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fe384035040 tx=0x7fe3840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:48.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.178+0000 7fe38affd640 1 -- 172.21.15.38:0/1636790810 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384041070 con 0x7fe3940d0870 2024-04-25T08:39:48.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.178+0000 7fe39c529640 1 -- 172.21.15.38:0/1636790810 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe39415dad0 con 0x7fe3940d0870 2024-04-25T08:39:48.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.178+0000 7fe38affd640 1 -- 172.21.15.38:0/1636790810 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38403c460 con 0x7fe3940d0870 2024-04-25T08:39:48.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.178+0000 7fe38affd640 1 -- 172.21.15.38:0/1636790810 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe384037040 con 0x7fe3940d0870 2024-04-25T08:39:48.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.178+0000 7fe39c529640 1 -- 172.21.15.38:0/1636790810 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe39415e090 con 0x7fe3940d0870 2024-04-25T08:39:48.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.179+0000 7fe39c529640 1 -- 172.21.15.38:0/1636790810 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe394068e80 con 0x7fe3940d0870 2024-04-25T08:39:48.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.179+0000 7fe38affd640 1 -- 172.21.15.38:0/1636790810 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe38403a070 con 0x7fe3940d0870 2024-04-25T08:39:48.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.180+0000 7fe38affd640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe37806b500 0x7fe37806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:48.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.180+0000 7fe38affd640 1 -- 172.21.15.38:0/1636790810 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5807+0+0 (secure 0 0 0) 0x7fe3840aa350 con 0x7fe3940d0870 2024-04-25T08:39:48.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.180+0000 7fe39a2c5640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe37806b500 0x7fe37806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:48.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.181+0000 7fe39a2c5640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe37806b500 0x7fe37806d9c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe390000f60 tx=0x7fe390007a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:48.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.184+0000 7fe38affd640 1 -- 172.21.15.38:0/1636790810 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe384078a90 con 0x7fe3940d0870 2024-04-25T08:39:48.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:48.370+0000 7fe39c529640 1 -- 172.21.15.38:0/1636790810 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe3940b6d00 con 0x7fe3940d0870 2024-04-25T08:39:49.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.044+0000 7fe38affd640 1 -- 172.21.15.38:0/1636790810 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe384097070 con 0x7fe3940d0870 2024-04-25T08:39:49.037 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-04-25T08:39:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.047+0000 7fe39c529640 1 -- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe37806b500 msgr2=0x7fe37806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.047+0000 7fe39c529640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe37806b500 0x7fe37806d9c0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe390000f60 tx=0x7fe390007a10 comp rx=0 tx=0).stop 2024-04-25T08:39:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.047+0000 7fe39c529640 1 -- 172.21.15.38:0/1636790810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3940d0870 msgr2=0x7fe394145d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.047+0000 7fe39c529640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3940d0870 0x7fe394145d50 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fe384035040 tx=0x7fe3840046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.048+0000 7fe39c529640 1 -- 172.21.15.38:0/1636790810 shutdown_connections 2024-04-25T08:39:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.048+0000 7fe39c529640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe3941612c0 0x7fe394146290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.048+0000 7fe39c529640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3940d0870 0x7fe394145d50 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.048+0000 7fe39c529640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe37806b500 0x7fe37806d9c0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.048+0000 7fe39c529640 1 --2- 172.21.15.38:0/1636790810 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3940b93f0 0x7fe3940c9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.048+0000 7fe39c529640 1 -- 172.21.15.38:0/1636790810 >> 172.21.15.38:0/1636790810 conn(0x7fe394066d10 msgr2=0x7fe394159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.048+0000 7fe39c529640 1 -- 172.21.15.38:0/1636790810 shutdown_connections 2024-04-25T08:39:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.048+0000 7fe39c529640 1 -- 172.21.15.38:0/1636790810 wait complete. 2024-04-25T08:39:49.060 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:39:49.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.178+0000 7f6a1bf7e640 1 Processor -- start 2024-04-25T08:39:49.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.178+0000 7f6a1bf7e640 1 -- start start 2024-04-25T08:39:49.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.179+0000 7f6a1bf7e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6a14166610 0x7f6a14168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.179+0000 7f6a1bf7e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a140d2060 con 0x7f6a140d33d0 2024-04-25T08:39:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.179+0000 7f6a1bf7e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a140d2330 con 0x7f6a140d0320 2024-04-25T08:39:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.179+0000 7f6a1bf7e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a140d2610 con 0x7f6a14166610 2024-04-25T08:39:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.179+0000 7f6a1a51b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6a14166610 0x7f6a14168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.179+0000 7f6a1a51b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6a14166610 0x7f6a14168a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42530/0 (socket says 172.21.15.38:42530) 2024-04-25T08:39:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.179+0000 7f6a19519640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6a140d33d0 0x7f6a140d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56386/0 (socket says 172.21.15.38:56386) 2024-04-25T08:39:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.179+0000 7f6a1a51b640 1 -- 172.21.15.38:0/3976945673 learned_addr learned my addr 172.21.15.38:0/3976945673 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.180+0000 7f6a1a51b640 1 -- 172.21.15.38:0/3976945673 >> v1:172.21.15.107:6789/0 conn(0x7f6a140d0320 legacy=0x7f6a140d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.180+0000 7f6a1a51b640 1 -- 172.21.15.38:0/3976945673 >> v1:172.21.15.38:6789/0 conn(0x7f6a140d33d0 legacy=0x7f6a140d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.180+0000 7f6a1a51b640 1 -- 172.21.15.38:0/3976945673 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140d28f0 con 0x7f6a14166610 2024-04-25T08:39:49.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.180+0000 7f6a18d18640 1 -- 172.21.15.38:0/3976945673 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259697656 0 0) 0x7f6a140d2060 con 0x7f6a140d33d0 2024-04-25T08:39:49.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.180+0000 7f6a1a51b640 1 --2- 172.21.15.38:0/3976945673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a14166610 0x7f6a14168a00 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009870 tx=0x7f6a0402a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5cdaa02cb75758b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.181+0000 7f6a18d18640 1 -- 172.21.15.38:0/3976945673 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04037070 con 0x7f6a14166610 2024-04-25T08:39:49.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.181+0000 7f6a18d18640 1 -- 172.21.15.38:0/3976945673 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a04032640 con 0x7f6a14166610 2024-04-25T08:39:49.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.181+0000 7f6a18d18640 1 -- 172.21.15.38:0/3976945673 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a04032a10 con 0x7f6a14166610 2024-04-25T08:39:49.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.181+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/3976945673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a14166610 msgr2=0x7f6a14168a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.181+0000 7f6a1bf7e640 1 --2- 172.21.15.38:0/3976945673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a14166610 0x7f6a14168a00 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6a04009870 tx=0x7f6a0402a3c0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.181+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/3976945673 shutdown_connections 2024-04-25T08:39:49.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.181+0000 7f6a1bf7e640 1 --2- 172.21.15.38:0/3976945673 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a14166610 0x7f6a14168a00 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.181+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/3976945673 >> 172.21.15.38:0/3976945673 conn(0x7f6a140668f0 msgr2=0x7f6a14066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:49.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.181+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/3976945673 shutdown_connections 2024-04-25T08:39:49.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.181+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/3976945673 wait complete. 2024-04-25T08:39:49.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.182+0000 7f6a1bf7e640 1 Processor -- start 2024-04-25T08:39:49.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.182+0000 7f6a1bf7e640 1 -- start start 2024-04-25T08:39:49.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.183+0000 7f6a1bf7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a140d0320 0x7f6a140ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.183+0000 7f6a1bf7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a140d33d0 0x7f6a1415eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.183+0000 7f6a1bf7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a14166610 0x7f6a1414e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.183+0000 7f6a1bf7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14161210 con 0x7f6a140d33d0 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.183+0000 7f6a1bf7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140d2eb0 con 0x7f6a140d0320 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.183+0000 7f6a1bf7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14161390 con 0x7f6a14166610 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.183+0000 7f6a19519640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a140d33d0 0x7f6a1415eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.183+0000 7f6a19519640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a140d33d0 0x7f6a1415eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42542/0 (socket says 172.21.15.38:42542) 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.183+0000 7f6a19519640 1 -- 172.21.15.38:0/2717057349 learned_addr learned my addr 172.21.15.38:0/2717057349 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.183+0000 7f6a19d1a640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a140d0320 0x7f6a140ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.183+0000 7f6a1a51b640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a14166610 0x7f6a1414e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.184+0000 7f6a19519640 1 -- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a14166610 msgr2=0x7f6a1414e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.184+0000 7f6a19519640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a14166610 0x7f6a1414e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.184+0000 7f6a19519640 1 -- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a140d0320 msgr2=0x7f6a140ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.184+0000 7f6a19519640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a140d0320 0x7f6a140ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.184+0000 7f6a19519640 1 -- 172.21.15.38:0/2717057349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a1415cf70 con 0x7f6a140d33d0 2024-04-25T08:39:49.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.184+0000 7f6a19519640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a140d33d0 0x7f6a1415eb40 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800c4a0 tx=0x7f6a0800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.184+0000 7f6a127fc640 1 -- 172.21.15.38:0/2717057349 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a08018070 con 0x7f6a140d33d0 2024-04-25T08:39:49.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.184+0000 7f6a127fc640 1 -- 172.21.15.38:0/2717057349 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a080074e0 con 0x7f6a140d33d0 2024-04-25T08:39:49.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.184+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/2717057349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1415d250 con 0x7f6a140d33d0 2024-04-25T08:39:49.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.184+0000 7f6a127fc640 1 -- 172.21.15.38:0/2717057349 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a08004a20 con 0x7f6a140d33d0 2024-04-25T08:39:49.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.184+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/2717057349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1415d810 con 0x7f6a140d33d0 2024-04-25T08:39:49.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.185+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/2717057349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a14068e80 con 0x7f6a140d33d0 2024-04-25T08:39:49.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.185+0000 7f6a127fc640 1 -- 172.21.15.38:0/2717057349 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a08004bc0 con 0x7f6a140d33d0 2024-04-25T08:39:49.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.186+0000 7f6a127fc640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6a0006b4d0 0x7f6a0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.186+0000 7f6a127fc640 1 -- 172.21.15.38:0/2717057349 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6a08089440 con 0x7f6a140d33d0 2024-04-25T08:39:49.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.186+0000 7f6a19d1a640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6a0006b4d0 0x7f6a0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.187+0000 7f6a19d1a640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6a0006b4d0 0x7f6a0006d990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6a140b8540 tx=0x7f6a0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.190+0000 7f6a127fc640 1 -- 172.21.15.38:0/2717057349 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a08058440 con 0x7f6a140d33d0 2024-04-25T08:39:49.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.411+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/2717057349 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6a140b3050 con 0x7f6a140d33d0 2024-04-25T08:39:49.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.414+0000 7f6a127fc640 1 -- 172.21.15.38:0/2717057349 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6a08057d40 con 0x7f6a140d33d0 2024-04-25T08:39:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.416+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6a0006b4d0 msgr2=0x7f6a0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.416+0000 7f6a1bf7e640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6a0006b4d0 0x7f6a0006d990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6a140b8540 tx=0x7f6a0c009040 comp rx=0 tx=0).stop 2024-04-25T08:39:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.416+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/2717057349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a140d33d0 msgr2=0x7f6a1415eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.416+0000 7f6a1bf7e640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a140d33d0 0x7f6a1415eb40 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6a0800c4a0 tx=0x7f6a0800c960 comp rx=0 tx=0).stop 2024-04-25T08:39:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.416+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/2717057349 shutdown_connections 2024-04-25T08:39:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.416+0000 7f6a1bf7e640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6a0006b4d0 0x7f6a0006d990 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.416+0000 7f6a1bf7e640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6a14166610 0x7f6a1414e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.416+0000 7f6a1bf7e640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6a140d33d0 0x7f6a1415eb40 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.416+0000 7f6a1bf7e640 1 --2- 172.21.15.38:0/2717057349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a140d0320 0x7f6a140ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.416+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/2717057349 >> 172.21.15.38:0/2717057349 conn(0x7f6a140668f0 msgr2=0x7f6a1415fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.416+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/2717057349 shutdown_connections 2024-04-25T08:39:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.416+0000 7f6a1bf7e640 1 -- 172.21.15.38:0/2717057349 wait complete. 2024-04-25T08:39:49.421 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T08:39:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.529+0000 7fb5d5276640 1 Processor -- start 2024-04-25T08:39:49.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.529+0000 7fb5d5276640 1 -- start start 2024-04-25T08:39:49.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.529+0000 7fb5d5276640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d006cc10 0x7fb5d00cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.529+0000 7fb5d5276640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb5d006cff0 0x7fb5d00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.529+0000 7fb5d5276640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d00d05c0 con 0x7fb5d0099a70 2024-04-25T08:39:49.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.529+0000 7fb5d5276640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d00d08a0 con 0x7fb5d006cff0 2024-04-25T08:39:49.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.529+0000 7fb5d5276640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d00d0b80 con 0x7fb5d006cc10 2024-04-25T08:39:49.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cf7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb5d0099a70 0x7fb5d015d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56394/0 (socket says 172.21.15.38:56394) 2024-04-25T08:39:49.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cf7fe640 1 -- 172.21.15.38:0/1408211528 learned_addr learned my addr 172.21.15.38:0/1408211528 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:49.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5ceffd640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d006cc10 0x7fb5d00cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028365841 0 0) 0x7fb5d00d05c0 con 0x7fb5d0099a70 2024-04-25T08:39:49.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5ce7fc640 1 --2- 172.21.15.38:0/1408211528 >> v2:172.21.15.107:3300/0 conn(0x7fb5d006cff0 0x7fb5d00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b8003520 con 0x7fb5d0099a70 2024-04-25T08:39:49.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1340558361 0 0) 0x7fb5b8003520 con 0x7fb5d0099a70 2024-04-25T08:39:49.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d006cc10 msgr2=0x7fb5d00cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:49.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d006cc10 0x7fb5d00cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 >> v2:172.21.15.107:3300/0 conn(0x7fb5d006cff0 msgr2=0x7fb5d00cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:49.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 --2- 172.21.15.38:0/1408211528 >> v2:172.21.15.107:3300/0 conn(0x7fb5d006cff0 0x7fb5d00cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d00d0e60 con 0x7fb5d0099a70 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fb5c80019f0 con 0x7fb5d0099a70 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5ceffd640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d006cc10 0x7fb5d00cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5ce7fc640 1 --2- 172.21.15.38:0/1408211528 >> v2:172.21.15.107:3300/0 conn(0x7fb5d006cff0 0x7fb5d00cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 >> v1:172.21.15.38:6789/0 conn(0x7fb5d0099a70 legacy=0x7fb5d015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5b80039b0 0x7fb5b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d006cc10 0x7fb5b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5b8004d40 0x7fb5b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d00d08a0 con 0x7fb5b80039b0 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d00d0b80 con 0x7fb5b8004d40 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b800ba40 con 0x7fb5d006cc10 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5ceffd640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5b80039b0 0x7fb5b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5ce7fc640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d006cc10 0x7fb5b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.530+0000 7fb5cf7fe640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5b8004d40 0x7fb5b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.531+0000 7fb5ceffd640 1 -- 172.21.15.38:0/1408211528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d006cc10 msgr2=0x7fb5b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.531+0000 7fb5ceffd640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d006cc10 0x7fb5b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.531+0000 7fb5ceffd640 1 -- 172.21.15.38:0/1408211528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5b8004d40 msgr2=0x7fb5b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.531+0000 7fb5ceffd640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5b8004d40 0x7fb5b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.531+0000 7fb5ceffd640 1 -- 172.21.15.38:0/1408211528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b8003520 con 0x7fb5b80039b0 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.531+0000 7fb5ceffd640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5b80039b0 0x7fb5b8003dd0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00098a0 tx=0x7fb5b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61652bf2ecc31d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.531+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000f040 con 0x7fb5b80039b0 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.531+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c002ace0 con 0x7fb5b80039b0 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.531+0000 7fb5cdffb640 1 -- 172.21.15.38:0/1408211528 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0004250 con 0x7fb5b80039b0 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.532+0000 7fb5d5276640 1 -- 172.21.15.38:0/1408211528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5b80039b0 msgr2=0x7fb5b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.532+0000 7fb5d5276640 1 --2- 172.21.15.38:0/1408211528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5b80039b0 0x7fb5b8003dd0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00098a0 tx=0x7fb5b8005830 comp rx=0 tx=0).stop 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.532+0000 7fb5cf7fe640 1 -- 172.21.15.38:0/1408211528 reap_dead start 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.532+0000 7fb5d5276640 1 -- 172.21.15.38:0/1408211528 shutdown_connections 2024-04-25T08:39:49.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.532+0000 7fb5d5276640 1 -- 172.21.15.38:0/1408211528 >> 172.21.15.38:0/1408211528 conn(0x7fb5d00b31f0 msgr2=0x7fb5d00b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:49.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.532+0000 7fb5d5276640 1 -- 172.21.15.38:0/1408211528 shutdown_connections 2024-04-25T08:39:49.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.532+0000 7fb5d5276640 1 -- 172.21.15.38:0/1408211528 wait complete. 2024-04-25T08:39:49.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.533+0000 7fb5d5276640 1 Processor -- start 2024-04-25T08:39:49.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.533+0000 7fb5d5276640 1 -- start start 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.534+0000 7fb5d5276640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5b80039b0 0x7fb5d015a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.534+0000 7fb5d5276640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d0158ba0 0x7fb5d00cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.534+0000 7fb5d5276640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d0159020 0x7fb5d00cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.534+0000 7fb5d5276640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d006c2d0 con 0x7fb5d0159020 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.534+0000 7fb5d5276640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d006c150 con 0x7fb5b80039b0 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.534+0000 7fb5d5276640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d00cfa70 con 0x7fb5d0158ba0 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.534+0000 7fb5cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d0159020 0x7fb5d00cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.534+0000 7fb5cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d0159020 0x7fb5d00cb200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42556/0 (socket says 172.21.15.38:42556) 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.534+0000 7fb5cf7fe640 1 -- 172.21.15.38:0/94409403 learned_addr learned my addr 172.21.15.38:0/94409403 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5ce7fc640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d0158ba0 0x7fb5d00cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5ceffd640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5b80039b0 0x7fb5d015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5cf7fe640 1 -- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d0158ba0 msgr2=0x7fb5d00cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5cf7fe640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d0158ba0 0x7fb5d00cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5cf7fe640 1 -- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5b80039b0 msgr2=0x7fb5d015a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5cf7fe640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5b80039b0 0x7fb5d015a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5cf7fe640 1 -- 172.21.15.38:0/94409403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d00c7a60 con 0x7fb5d0159020 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5cf7fe640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d0159020 0x7fb5d00cb200 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00c51e0 tx=0x7fb5c8007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5b77fe640 1 -- 172.21.15.38:0/94409403 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c800abe0 con 0x7fb5d0159020 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5b77fe640 1 -- 172.21.15.38:0/94409403 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c8018070 con 0x7fb5d0159020 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5d5276640 1 -- 172.21.15.38:0/94409403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d01595d0 con 0x7fb5d0159020 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5b77fe640 1 -- 172.21.15.38:0/94409403 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c80133f0 con 0x7fb5d0159020 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.535+0000 7fb5d5276640 1 -- 172.21.15.38:0/94409403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d0159b90 con 0x7fb5d0159020 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.536+0000 7fb5d5276640 1 -- 172.21.15.38:0/94409403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d0068e80 con 0x7fb5d0159020 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.536+0000 7fb5b77fe640 1 -- 172.21.15.38:0/94409403 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5c8013590 con 0x7fb5d0159020 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.537+0000 7fb5b77fe640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb5bc06b4d0 0x7fb5bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.537+0000 7fb5b77fe640 1 -- 172.21.15.38:0/94409403 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb5c808a740 con 0x7fb5d0159020 2024-04-25T08:39:49.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.537+0000 7fb5ceffd640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb5bc06b4d0 0x7fb5bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.537+0000 7fb5ceffd640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb5bc06b4d0 0x7fb5bc06d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0033000 tx=0x7fb5c002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:49.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.541+0000 7fb5b77fe640 1 -- 172.21.15.38:0/94409403 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5c80596b0 con 0x7fb5d0159020 2024-04-25T08:39:49.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:49.792+0000 7fb5d5276640 1 -- 172.21.15.38:0/94409403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7fb5d006d450 con 0x7fb5d0159020 2024-04-25T08:39:50.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.060+0000 7fb5b77fe640 1 -- 172.21.15.38:0/94409403 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v399) v1 ==== 155+0+0 (secure 0 0 0) 0x7fb5c8058fb0 con 0x7fb5d0159020 2024-04-25T08:39:50.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.062+0000 7fb5d5276640 1 -- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb5bc06b4d0 msgr2=0x7fb5bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.062+0000 7fb5d5276640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb5bc06b4d0 0x7fb5bc06d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0033000 tx=0x7fb5c002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:50.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.062+0000 7fb5d5276640 1 -- 172.21.15.38:0/94409403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d0159020 msgr2=0x7fb5d00cb200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.062+0000 7fb5d5276640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d0159020 0x7fb5d00cb200 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00c51e0 tx=0x7fb5c8007a90 comp rx=0 tx=0).stop 2024-04-25T08:39:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.062+0000 7fb5d5276640 1 -- 172.21.15.38:0/94409403 shutdown_connections 2024-04-25T08:39:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.062+0000 7fb5d5276640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb5d0159020 0x7fb5d00cb200 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.062+0000 7fb5d5276640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb5bc06b4d0 0x7fb5bc06d990 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.062+0000 7fb5d5276640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb5d0158ba0 0x7fb5d00cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.062+0000 7fb5d5276640 1 --2- 172.21.15.38:0/94409403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb5b80039b0 0x7fb5d015a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.062+0000 7fb5d5276640 1 -- 172.21.15.38:0/94409403 >> 172.21.15.38:0/94409403 conn(0x7fb5d00b31f0 msgr2=0x7fb5d00b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.062+0000 7fb5d5276640 1 -- 172.21.15.38:0/94409403 shutdown_connections 2024-04-25T08:39:50.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.062+0000 7fb5d5276640 1 -- 172.21.15.38:0/94409403 wait complete. 2024-04-25T08:39:50.067 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T08:39:50.067 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:39:50.068 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:50.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.176+0000 7f29a513b640 1 Processor -- start 2024-04-25T08:39:50.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.176+0000 7f29a513b640 1 -- start start 2024-04-25T08:39:50.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f29a513b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f29a006c110 0x7f29a006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f29a513b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f29a015df50 0x7f29a00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f29a513b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a006ca30 con 0x7f29a00ce2e0 2024-04-25T08:39:50.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f29a513b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a00cf2d0 con 0x7f29a006c110 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f29a513b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a00ceff0 con 0x7f29a015df50 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f299ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f29a006c110 0x7f29a006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f299ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f29a006c110 0x7f29a006c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42570/0 (socket says 172.21.15.38:42570) 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f299f59e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f29a00ce2e0 0x7f29a00d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56404/0 (socket says 172.21.15.38:56404) 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f299ed9d640 1 -- 172.21.15.38:0/1058803779 learned_addr learned my addr 172.21.15.38:0/1058803779 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f299e59c640 1 --2- 172.21.15.38:0/1058803779 >> v2:172.21.15.107:3300/0 conn(0x7f29a015df50 0x7f29a00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f299ed9d640 1 -- 172.21.15.38:0/1058803779 >> v2:172.21.15.107:3300/0 conn(0x7f29a015df50 msgr2=0x7f29a00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f299ed9d640 1 --2- 172.21.15.38:0/1058803779 >> v2:172.21.15.107:3300/0 conn(0x7f29a015df50 0x7f29a00ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f299ed9d640 1 -- 172.21.15.38:0/1058803779 >> v1:172.21.15.38:6789/0 conn(0x7f29a00ce2e0 legacy=0x7f29a00d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f299ed9d640 1 -- 172.21.15.38:0/1058803779 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a00d1f70 con 0x7f29a006c110 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f299dd9b640 1 -- 172.21.15.38:0/1058803779 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407530784 0 0) 0x7f29a006ca30 con 0x7f29a00ce2e0 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.177+0000 7f299e59c640 1 --2- 172.21.15.38:0/1058803779 >> v2:172.21.15.107:3300/0 conn(0x7f29a015df50 0x7f29a00ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.178+0000 7f299ed9d640 1 --2- 172.21.15.38:0/1058803779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a006c110 0x7f29a006c4f0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f29a00d4960 tx=0x7f299002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5af2c31c875779 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:50.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.178+0000 7f299dd9b640 1 -- 172.21.15.38:0/1058803779 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990037070 con 0x7f29a006c110 2024-04-25T08:39:50.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.178+0000 7f299dd9b640 1 -- 172.21.15.38:0/1058803779 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2990032440 con 0x7f29a006c110 2024-04-25T08:39:50.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.178+0000 7f299dd9b640 1 -- 172.21.15.38:0/1058803779 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990032810 con 0x7f29a006c110 2024-04-25T08:39:50.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.179+0000 7f29a513b640 1 -- 172.21.15.38:0/1058803779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a006c110 msgr2=0x7f29a006c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.179+0000 7f29a513b640 1 --2- 172.21.15.38:0/1058803779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a006c110 0x7f29a006c4f0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f29a00d4960 tx=0x7f299002a200 comp rx=0 tx=0).stop 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.179+0000 7f29a513b640 1 -- 172.21.15.38:0/1058803779 shutdown_connections 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.179+0000 7f29a513b640 1 --2- 172.21.15.38:0/1058803779 >> v2:172.21.15.107:3300/0 conn(0x7f29a015df50 0x7f29a00ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.179+0000 7f29a513b640 1 --2- 172.21.15.38:0/1058803779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a006c110 0x7f29a006c4f0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.179+0000 7f29a513b640 1 -- 172.21.15.38:0/1058803779 >> 172.21.15.38:0/1058803779 conn(0x7f29a00d3fb0 msgr2=0x7f29a00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.179+0000 7f29a513b640 1 -- 172.21.15.38:0/1058803779 shutdown_connections 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.179+0000 7f29a513b640 1 -- 172.21.15.38:0/1058803779 wait complete. 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.180+0000 7f29a513b640 1 Processor -- start 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.180+0000 7f29a513b640 1 -- start start 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f29a513b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a006c110 0x7f29a00b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f29a513b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a00ce2e0 0x7f29a0149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f29a513b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29a015df50 0x7f29a015ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f29a513b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a00cece0 con 0x7f29a006c110 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f29a513b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a00d2530 con 0x7f29a00ce2e0 2024-04-25T08:39:50.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f29a513b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a00d3160 con 0x7f29a015df50 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f299ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a006c110 0x7f29a00b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f299ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a006c110 0x7f29a00b83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42578/0 (socket says 172.21.15.38:42578) 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f299ed9d640 1 -- 172.21.15.38:0/2231196484 learned_addr learned my addr 172.21.15.38:0/2231196484 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f299e59c640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a00ce2e0 0x7f29a0149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f299f59e640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29a015df50 0x7f29a015ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f299ed9d640 1 -- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29a015df50 msgr2=0x7f29a015ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f299ed9d640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29a015df50 0x7f29a015ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f299ed9d640 1 -- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a00ce2e0 msgr2=0x7f29a0149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.181+0000 7f299ed9d640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a00ce2e0 0x7f29a0149bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.182+0000 7f299ed9d640 1 -- 172.21.15.38:0/2231196484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a00b4c50 con 0x7f29a006c110 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.182+0000 7f299e59c640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a00ce2e0 0x7f29a0149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.182+0000 7f299ed9d640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a006c110 0x7f29a00b83c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f299002a240 tx=0x7f29900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.182+0000 7f29877fe640 1 -- 172.21.15.38:0/2231196484 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990037040 con 0x7f29a006c110 2024-04-25T08:39:50.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.182+0000 7f29877fe640 1 -- 172.21.15.38:0/2231196484 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2990042070 con 0x7f29a006c110 2024-04-25T08:39:50.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.182+0000 7f29a513b640 1 -- 172.21.15.38:0/2231196484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a00c8ee0 con 0x7f29a006c110 2024-04-25T08:39:50.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.182+0000 7f29877fe640 1 -- 172.21.15.38:0/2231196484 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2990030bb0 con 0x7f29a006c110 2024-04-25T08:39:50.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.182+0000 7f29a513b640 1 -- 172.21.15.38:0/2231196484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a00c94a0 con 0x7f29a006c110 2024-04-25T08:39:50.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.183+0000 7f29a513b640 1 -- 172.21.15.38:0/2231196484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a0068e80 con 0x7f29a006c110 2024-04-25T08:39:50.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.183+0000 7f29877fe640 1 -- 172.21.15.38:0/2231196484 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2990047460 con 0x7f29a006c110 2024-04-25T08:39:50.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.184+0000 7f29877fe640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f298c06b4d0 0x7f298c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.184+0000 7f29877fe640 1 -- 172.21.15.38:0/2231196484 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f29900aac10 con 0x7f29a006c110 2024-04-25T08:39:50.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.184+0000 7f299e59c640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f298c06b4d0 0x7f298c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.185+0000 7f299e59c640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f298c06b4d0 0x7f298c06d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f29a00b8900 tx=0x7f2994008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:50.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.188+0000 7f29877fe640 1 -- 172.21.15.38:0/2231196484 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2990045360 con 0x7f29a006c110 2024-04-25T08:39:50.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.372+0000 7f29a513b640 1 -- 172.21.15.38:0/2231196484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29a006c690 con 0x7f29a006c110 2024-04-25T08:39:50.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.373+0000 7f29877fe640 1 -- 172.21.15.38:0/2231196484 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+13293 (secure 0 0 0) 0x7f2990079c90 con 0x7f29a006c110 2024-04-25T08:39:50.366 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:50.366 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":113,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:49.042888+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:6837/1263010516":"2024-04-26T08:39:45.764759+0000","172.21.15.107:6835/1263010516":"2024-04-26T08:39:45.764759+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:50.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.375+0000 7f29a513b640 1 -- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f298c06b4d0 msgr2=0x7f298c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.375+0000 7f29a513b640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f298c06b4d0 0x7f298c06d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f29a00b8900 tx=0x7f2994008040 comp rx=0 tx=0).stop 2024-04-25T08:39:50.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.375+0000 7f29a513b640 1 -- 172.21.15.38:0/2231196484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a006c110 msgr2=0x7f29a00b83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.375+0000 7f29a513b640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a006c110 0x7f29a00b83c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f299002a240 tx=0x7f29900046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.375+0000 7f29a513b640 1 -- 172.21.15.38:0/2231196484 shutdown_connections 2024-04-25T08:39:50.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.375+0000 7f29a513b640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f298c06b4d0 0x7f298c06d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.375+0000 7f29a513b640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29a015df50 0x7f29a015ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.375+0000 7f29a513b640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a00ce2e0 0x7f29a0149bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.375+0000 7f29a513b640 1 --2- 172.21.15.38:0/2231196484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29a006c110 0x7f29a00b83c0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.375+0000 7f29a513b640 1 -- 172.21.15.38:0/2231196484 >> 172.21.15.38:0/2231196484 conn(0x7f29a00d3fb0 msgr2=0x7f29a015cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:50.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.375+0000 7f29a513b640 1 -- 172.21.15.38:0/2231196484 shutdown_connections 2024-04-25T08:39:50.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.375+0000 7f29a513b640 1 -- 172.21.15.38:0/2231196484 wait complete. 2024-04-25T08:39:50.380 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:39:50.380 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:39:50.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.490+0000 7f5957da9640 1 Processor -- start 2024-04-25T08:39:50.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.490+0000 7f5957da9640 1 -- start start 2024-04-25T08:39:50.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.491+0000 7f5957da9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f59500d0320 0x7f59500d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.491+0000 7f5957da9640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f59500b6980 0x7f59500b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.491+0000 7f5957da9640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59500d2060 con 0x7f59500d0700 2024-04-25T08:39:50.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.491+0000 7f5957da9640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59500d2330 con 0x7f59500d0320 2024-04-25T08:39:50.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.491+0000 7f5957da9640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59500d2610 con 0x7f59500b6980 2024-04-25T08:39:50.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.491+0000 7f5955b45640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f59500d0320 0x7f59500d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.491+0000 7f5955344640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f59500d0700 0x7f59500d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56420/0 (socket says 172.21.15.38:56420) 2024-04-25T08:39:50.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.491+0000 7f5955344640 1 -- 172.21.15.38:0/2688337856 learned_addr learned my addr 172.21.15.38:0/2688337856 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:50.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.492+0000 7f5956346640 1 --2- 172.21.15.38:0/2688337856 >> v2:172.21.15.107:3300/0 conn(0x7f59500b6980 0x7f59500b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.492+0000 7f5955b45640 1 -- 172.21.15.38:0/2688337856 >> v2:172.21.15.107:3300/0 conn(0x7f59500b6980 msgr2=0x7f59500b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:50.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.492+0000 7f5955b45640 1 --2- 172.21.15.38:0/2688337856 >> v2:172.21.15.107:3300/0 conn(0x7f59500b6980 0x7f59500b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.492+0000 7f5955b45640 1 -- 172.21.15.38:0/2688337856 >> v1:172.21.15.38:6789/0 conn(0x7f59500d0700 legacy=0x7f59500d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.492+0000 7f5954b43640 1 -- 172.21.15.38:0/2688337856 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363557403 0 0) 0x7f59500d2060 con 0x7f59500d0700 2024-04-25T08:39:50.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.492+0000 7f5955b45640 1 -- 172.21.15.38:0/2688337856 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500d28f0 con 0x7f59500d0320 2024-04-25T08:39:50.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.492+0000 7f5956346640 1 --2- 172.21.15.38:0/2688337856 >> v2:172.21.15.107:3300/0 conn(0x7f59500b6980 0x7f59500b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:50.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.492+0000 7f5955b45640 1 --2- 172.21.15.38:0/2688337856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59500d0320 0x7f59500d33d0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f5950067590 tx=0x7f594802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817436e9e3b3ac53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:50.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.492+0000 7f5954b43640 1 -- 172.21.15.38:0/2688337856 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948037070 con 0x7f59500d0320 2024-04-25T08:39:50.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.492+0000 7f5954b43640 1 -- 172.21.15.38:0/2688337856 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5948032440 con 0x7f59500d0320 2024-04-25T08:39:50.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.492+0000 7f5954b43640 1 -- 172.21.15.38:0/2688337856 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948032810 con 0x7f59500d0320 2024-04-25T08:39:50.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.493+0000 7f5957da9640 1 -- 172.21.15.38:0/2688337856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59500d0320 msgr2=0x7f59500d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.493+0000 7f5957da9640 1 --2- 172.21.15.38:0/2688337856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59500d0320 0x7f59500d33d0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f5950067590 tx=0x7f594802a230 comp rx=0 tx=0).stop 2024-04-25T08:39:50.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.493+0000 7f5957da9640 1 -- 172.21.15.38:0/2688337856 shutdown_connections 2024-04-25T08:39:50.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.493+0000 7f5957da9640 1 --2- 172.21.15.38:0/2688337856 >> v2:172.21.15.107:3300/0 conn(0x7f59500b6980 0x7f59500b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.493+0000 7f5957da9640 1 --2- 172.21.15.38:0/2688337856 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59500d0320 0x7f59500d33d0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.493+0000 7f5957da9640 1 -- 172.21.15.38:0/2688337856 >> 172.21.15.38:0/2688337856 conn(0x7f59500668f0 msgr2=0x7f5950066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:50.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.493+0000 7f5957da9640 1 -- 172.21.15.38:0/2688337856 shutdown_connections 2024-04-25T08:39:50.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.493+0000 7f5957da9640 1 -- 172.21.15.38:0/2688337856 wait complete. 2024-04-25T08:39:50.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.494+0000 7f5957da9640 1 Processor -- start 2024-04-25T08:39:50.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.494+0000 7f5957da9640 1 -- start start 2024-04-25T08:39:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5957da9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59500b6980 0x7f59500cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5957da9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59500d0320 0x7f5950159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5957da9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59500d0700 0x7f59500b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5957da9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59500d0b40 con 0x7f59500d0320 2024-04-25T08:39:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5957da9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59500d2eb0 con 0x7f59500b6980 2024-04-25T08:39:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5957da9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59500b4a20 con 0x7f59500d0700 2024-04-25T08:39:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5955344640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59500d0320 0x7f5950159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5955344640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59500d0320 0x7f5950159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42604/0 (socket says 172.21.15.38:42604) 2024-04-25T08:39:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5955344640 1 -- 172.21.15.38:0/3307825534 learned_addr learned my addr 172.21.15.38:0/3307825534 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5955b45640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59500b6980 0x7f59500cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5956346640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59500d0700 0x7f59500b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5955344640 1 -- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59500d0700 msgr2=0x7f59500b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.495+0000 7f5955344640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59500d0700 0x7f59500b5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.496+0000 7f5955344640 1 -- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59500b6980 msgr2=0x7f59500cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.496+0000 7f5955344640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59500b6980 0x7f59500cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.496+0000 7f5955344640 1 -- 172.21.15.38:0/3307825534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59500b4d00 con 0x7f59500d0320 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.496+0000 7f5955b45640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59500b6980 0x7f59500cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.496+0000 7f5955344640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59500d0320 0x7f5950159290 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f594400c430 tx=0x7f594400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.496+0000 7f594e7fc640 1 -- 172.21.15.38:0/3307825534 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440074e0 con 0x7f59500d0320 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.496+0000 7f594e7fc640 1 -- 172.21.15.38:0/3307825534 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59440047e0 con 0x7f59500d0320 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.496+0000 7f5957da9640 1 -- 172.21.15.38:0/3307825534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59500b4fe0 con 0x7f59500d0320 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.496+0000 7f594e7fc640 1 -- 172.21.15.38:0/3307825534 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944004bd0 con 0x7f59500d0320 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.496+0000 7f5957da9640 1 -- 172.21.15.38:0/3307825534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59500b55a0 con 0x7f59500d0320 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.497+0000 7f5957da9640 1 -- 172.21.15.38:0/3307825534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595015f1b0 con 0x7f59500d0320 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.497+0000 7f594e7fc640 1 -- 172.21.15.38:0/3307825534 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5944011070 con 0x7f59500d0320 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.498+0000 7f594e7fc640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f593c06b4d0 0x7f593c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.498+0000 7f594e7fc640 1 -- 172.21.15.38:0/3307825534 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5944089cc0 con 0x7f59500d0320 2024-04-25T08:39:50.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.498+0000 7f5955b45640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f593c06b4d0 0x7f593c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.498+0000 7f5955b45640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f593c06b4d0 0x7f593c06d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5948034000 tx=0x7f594802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:50.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.502+0000 7f594e7fc640 1 -- 172.21.15.38:0/3307825534 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f595015f1b0 con 0x7f59500d0320 2024-04-25T08:39:50.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.685+0000 7f5957da9640 1 -- 172.21.15.38:0/3307825534 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5950068e80 con 0x7f59500d0320 2024-04-25T08:39:50.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.685+0000 7f594e7fc640 1 -- 172.21.15.38:0/3307825534 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5944058d40 con 0x7f59500d0320 2024-04-25T08:39:50.679 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:39:50.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.687+0000 7f5957da9640 1 -- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f593c06b4d0 msgr2=0x7f593c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.687+0000 7f5957da9640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f593c06b4d0 0x7f593c06d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5948034000 tx=0x7f594802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.687+0000 7f5957da9640 1 -- 172.21.15.38:0/3307825534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59500d0320 msgr2=0x7f5950159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.687+0000 7f5957da9640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59500d0320 0x7f5950159290 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f594400c430 tx=0x7f594400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.687+0000 7f5957da9640 1 -- 172.21.15.38:0/3307825534 shutdown_connections 2024-04-25T08:39:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.687+0000 7f5957da9640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f593c06b4d0 0x7f593c06d990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.687+0000 7f5957da9640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59500d0700 0x7f59500b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.687+0000 7f5957da9640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59500d0320 0x7f5950159290 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.687+0000 7f5957da9640 1 --2- 172.21.15.38:0/3307825534 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59500b6980 0x7f59500cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.687+0000 7f5957da9640 1 -- 172.21.15.38:0/3307825534 >> 172.21.15.38:0/3307825534 conn(0x7f59500668f0 msgr2=0x7f595015eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.687+0000 7f5957da9640 1 -- 172.21.15.38:0/3307825534 shutdown_connections 2024-04-25T08:39:50.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.687+0000 7f5957da9640 1 -- 172.21.15.38:0/3307825534 wait complete. 2024-04-25T08:39:50.691 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T08:39:50.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.799+0000 7fbe67e07640 1 Processor -- start 2024-04-25T08:39:50.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.799+0000 7fbe67e07640 1 -- start start 2024-04-25T08:39:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.800+0000 7fbe67e07640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbe600b5710 0x7fbe60161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.800+0000 7fbe67e07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6015ddf0 0x7fbe600b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.800+0000 7fbe67e07640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe60161e00 con 0x7fbe600b5af0 2024-04-25T08:39:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.800+0000 7fbe67e07640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe600b9930 con 0x7fbe600b5710 2024-04-25T08:39:50.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.800+0000 7fbe67e07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe601620e0 con 0x7fbe6015ddf0 2024-04-25T08:39:50.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.800+0000 7fbe653a2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6015ddf0 0x7fbe600b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.800+0000 7fbe65ba3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbe600b5710 0x7fbe60161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.800+0000 7fbe653a2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6015ddf0 0x7fbe600b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:33048/0 (socket says 172.21.15.38:33048) 2024-04-25T08:39:50.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.800+0000 7fbe653a2640 1 -- 172.21.15.38:0/518315486 learned_addr learned my addr 172.21.15.38:0/518315486 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:50.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.801+0000 7fbe653a2640 1 -- 172.21.15.38:0/518315486 >> v2:172.21.15.107:3300/0 conn(0x7fbe600b5710 msgr2=0x7fbe60161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:50.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.801+0000 7fbe653a2640 1 --2- 172.21.15.38:0/518315486 >> v2:172.21.15.107:3300/0 conn(0x7fbe600b5710 0x7fbe60161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.801+0000 7fbe653a2640 1 -- 172.21.15.38:0/518315486 >> v1:172.21.15.107:6789/0 conn(0x7fbe600b5af0 legacy=0x7fbe6006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.801+0000 7fbe653a2640 1 -- 172.21.15.38:0/518315486 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe601623c0 con 0x7fbe6015ddf0 2024-04-25T08:39:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.801+0000 7fbe653a2640 1 --2- 172.21.15.38:0/518315486 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6015ddf0 0x7fbe600b93f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009870 tx=0x7fbe5002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51632f7cc7fa0b37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.802+0000 7fbe64ba1640 1 -- 172.21.15.38:0/518315486 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50037070 con 0x7fbe6015ddf0 2024-04-25T08:39:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.802+0000 7fbe64ba1640 1 -- 172.21.15.38:0/518315486 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe50032440 con 0x7fbe6015ddf0 2024-04-25T08:39:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.802+0000 7fbe64ba1640 1 -- 172.21.15.38:0/518315486 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50032810 con 0x7fbe6015ddf0 2024-04-25T08:39:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.802+0000 7fbe67e07640 1 -- 172.21.15.38:0/518315486 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6015ddf0 msgr2=0x7fbe600b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.802+0000 7fbe67e07640 1 --2- 172.21.15.38:0/518315486 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6015ddf0 0x7fbe600b93f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009870 tx=0x7fbe5002a270 comp rx=0 tx=0).stop 2024-04-25T08:39:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.802+0000 7fbe67e07640 1 -- 172.21.15.38:0/518315486 shutdown_connections 2024-04-25T08:39:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.802+0000 7fbe67e07640 1 --2- 172.21.15.38:0/518315486 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe6015ddf0 0x7fbe600b93f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.802+0000 7fbe67e07640 1 --2- 172.21.15.38:0/518315486 >> v2:172.21.15.107:3300/0 conn(0x7fbe600b5710 0x7fbe60161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.802+0000 7fbe67e07640 1 -- 172.21.15.38:0/518315486 >> 172.21.15.38:0/518315486 conn(0x7fbe600d07c0 msgr2=0x7fbe600d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:50.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.802+0000 7fbe67e07640 1 -- 172.21.15.38:0/518315486 shutdown_connections 2024-04-25T08:39:50.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.802+0000 7fbe67e07640 1 -- 172.21.15.38:0/518315486 wait complete. 2024-04-25T08:39:50.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.803+0000 7fbe67e07640 1 Processor -- start 2024-04-25T08:39:50.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.803+0000 7fbe67e07640 1 -- start start 2024-04-25T08:39:50.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.804+0000 7fbe67e07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe600b5710 0x7fbe600c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.804+0000 7fbe67e07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe600b5af0 0x7fbe601604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.804+0000 7fbe67e07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6015ddf0 0x7fbe600caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.804+0000 7fbe67e07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe600d34d0 con 0x7fbe600b5710 2024-04-25T08:39:50.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.804+0000 7fbe67e07640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe600d3350 con 0x7fbe6015ddf0 2024-04-25T08:39:50.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.804+0000 7fbe67e07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe600d3650 con 0x7fbe600b5af0 2024-04-25T08:39:50.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.804+0000 7fbe65ba3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe600b5710 0x7fbe600c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.804+0000 7fbe65ba3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe600b5710 0x7fbe600c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42616/0 (socket says 172.21.15.38:42616) 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.804+0000 7fbe65ba3640 1 -- 172.21.15.38:0/615242452 learned_addr learned my addr 172.21.15.38:0/615242452 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.804+0000 7fbe663a4640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6015ddf0 0x7fbe600caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.804+0000 7fbe653a2640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe600b5af0 0x7fbe601604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe65ba3640 1 -- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe600b5af0 msgr2=0x7fbe601604e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe65ba3640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe600b5af0 0x7fbe601604e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe65ba3640 1 -- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6015ddf0 msgr2=0x7fbe600caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe65ba3640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6015ddf0 0x7fbe600caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe65ba3640 1 -- 172.21.15.38:0/615242452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe60157d30 con 0x7fbe600b5710 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe653a2640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe600b5af0 0x7fbe601604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe663a4640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6015ddf0 0x7fbe600caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe65ba3640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe600b5710 0x7fbe600c9e30 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00e9b0 tx=0x7fbe5c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe567fc640 1 -- 172.21.15.38:0/615242452 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c017070 con 0x7fbe600b5710 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe567fc640 1 -- 172.21.15.38:0/615242452 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c00c9a0 con 0x7fbe600b5710 2024-04-25T08:39:50.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe67e07640 1 -- 172.21.15.38:0/615242452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe60158010 con 0x7fbe600b5710 2024-04-25T08:39:50.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe567fc640 1 -- 172.21.15.38:0/615242452 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c00cd90 con 0x7fbe600b5710 2024-04-25T08:39:50.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.805+0000 7fbe67e07640 1 -- 172.21.15.38:0/615242452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe601585d0 con 0x7fbe600b5710 2024-04-25T08:39:50.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.806+0000 7fbe67e07640 1 -- 172.21.15.38:0/615242452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe60068e80 con 0x7fbe600b5710 2024-04-25T08:39:50.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.806+0000 7fbe567fc640 1 -- 172.21.15.38:0/615242452 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe5c01a070 con 0x7fbe600b5710 2024-04-25T08:39:50.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.807+0000 7fbe567fc640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbe4406b4d0 0x7fbe4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:50.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.807+0000 7fbe567fc640 1 -- 172.21.15.38:0/615242452 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbe5c08a3e0 con 0x7fbe600b5710 2024-04-25T08:39:50.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.807+0000 7fbe653a2640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbe4406b4d0 0x7fbe4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:50.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.808+0000 7fbe653a2640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbe4406b4d0 0x7fbe4406d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbe50034000 tx=0x7fbe5002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:50.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:50.811+0000 7fbe567fc640 1 -- 172.21.15.38:0/615242452 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe5c059460 con 0x7fbe600b5710 2024-04-25T08:39:51.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:51.069+0000 7fbe67e07640 1 -- 172.21.15.38:0/615242452 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7fbe600b3790 con 0x7fbe600b5710 2024-04-25T08:39:52.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.067+0000 7fbe567fc640 1 -- 172.21.15.38:0/615242452 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v401) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbe5c058d60 con 0x7fbe600b5710 2024-04-25T08:39:52.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.070+0000 7fbe67e07640 1 -- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbe4406b4d0 msgr2=0x7fbe4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.070+0000 7fbe67e07640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbe4406b4d0 0x7fbe4406d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbe50034000 tx=0x7fbe5002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.070+0000 7fbe67e07640 1 -- 172.21.15.38:0/615242452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe600b5710 msgr2=0x7fbe600c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.070+0000 7fbe67e07640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe600b5710 0x7fbe600c9e30 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c00e9b0 tx=0x7fbe5c00ee70 comp rx=0 tx=0).stop 2024-04-25T08:39:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.070+0000 7fbe67e07640 1 -- 172.21.15.38:0/615242452 shutdown_connections 2024-04-25T08:39:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.070+0000 7fbe67e07640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6015ddf0 0x7fbe600caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.070+0000 7fbe67e07640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbe4406b4d0 0x7fbe4406d990 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.070+0000 7fbe67e07640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbe600b5af0 0x7fbe601604e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.070+0000 7fbe67e07640 1 --2- 172.21.15.38:0/615242452 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe600b5710 0x7fbe600c9e30 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.070+0000 7fbe67e07640 1 -- 172.21.15.38:0/615242452 >> 172.21.15.38:0/615242452 conn(0x7fbe600d07c0 msgr2=0x7fbe600d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.070+0000 7fbe67e07640 1 -- 172.21.15.38:0/615242452 shutdown_connections 2024-04-25T08:39:52.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.070+0000 7fbe67e07640 1 -- 172.21.15.38:0/615242452 wait complete. 2024-04-25T08:39:52.077 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:39:52.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.195+0000 7fac39846640 1 Processor -- start 2024-04-25T08:39:52.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.195+0000 7fac39846640 1 -- start start 2024-04-25T08:39:52.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.196+0000 7fac39846640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fac3415baf0 0x7fac340cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.196+0000 7fac39846640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac340b3250 con 0x7fac340cfb70 2024-04-25T08:39:52.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.196+0000 7fac39846640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac340731d0 con 0x7fac34072660 2024-04-25T08:39:52.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.196+0000 7fac39846640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac34072c10 con 0x7fac3415baf0 2024-04-25T08:39:52.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.196+0000 7fac32ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fac3415baf0 0x7fac340cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.196+0000 7fac327fc640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fac340cfb70 0x7fac34071f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56432/0 (socket says 172.21.15.38:56432) 2024-04-25T08:39:52.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.196+0000 7fac327fc640 1 -- 172.21.15.38:0/54893566 learned_addr learned my addr 172.21.15.38:0/54893566 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:52.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.196+0000 7fac32ffd640 1 -- 172.21.15.38:0/54893566 >> v1:172.21.15.107:6789/0 conn(0x7fac34072660 legacy=0x7fac340acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:52.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.196+0000 7fac32ffd640 1 -- 172.21.15.38:0/54893566 >> v1:172.21.15.38:6789/0 conn(0x7fac340cfb70 legacy=0x7fac34071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.196+0000 7fac32ffd640 1 -- 172.21.15.38:0/54893566 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac34072ef0 con 0x7fac3415baf0 2024-04-25T08:39:52.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.197+0000 7fac32ffd640 1 --2- 172.21.15.38:0/54893566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac3415baf0 0x7fac340cf630 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fac240097a0 tx=0x7fac2402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5f0db2617f1c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.197+0000 7fac31ffb640 1 -- 172.21.15.38:0/54893566 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac24037070 con 0x7fac3415baf0 2024-04-25T08:39:52.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.197+0000 7fac31ffb640 1 -- 172.21.15.38:0/54893566 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2402adc0 con 0x7fac3415baf0 2024-04-25T08:39:52.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.197+0000 7fac31ffb640 1 -- 172.21.15.38:0/54893566 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac24032660 con 0x7fac3415baf0 2024-04-25T08:39:52.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.198+0000 7fac39846640 1 -- 172.21.15.38:0/54893566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac3415baf0 msgr2=0x7fac340cf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.198+0000 7fac39846640 1 --2- 172.21.15.38:0/54893566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac3415baf0 0x7fac340cf630 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fac240097a0 tx=0x7fac2402a0b0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.198+0000 7fac39846640 1 -- 172.21.15.38:0/54893566 shutdown_connections 2024-04-25T08:39:52.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.198+0000 7fac39846640 1 --2- 172.21.15.38:0/54893566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac3415baf0 0x7fac340cf630 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.198+0000 7fac39846640 1 -- 172.21.15.38:0/54893566 >> 172.21.15.38:0/54893566 conn(0x7fac3406adc0 msgr2=0x7fac3406b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:52.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.198+0000 7fac39846640 1 -- 172.21.15.38:0/54893566 shutdown_connections 2024-04-25T08:39:52.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.198+0000 7fac39846640 1 -- 172.21.15.38:0/54893566 wait complete. 2024-04-25T08:39:52.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.199+0000 7fac39846640 1 Processor -- start 2024-04-25T08:39:52.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.199+0000 7fac39846640 1 -- start start 2024-04-25T08:39:52.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac39846640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac34072660 0x7fac340b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac39846640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac340cfb70 0x7fac34154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac39846640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac3415baf0 0x7fac341554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac39846640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac340ad3a0 con 0x7fac34072660 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac39846640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac340ad220 con 0x7fac340cfb70 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac39846640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac340ad520 con 0x7fac3415baf0 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac32ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac34072660 0x7fac340b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac32ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac34072660 0x7fac340b9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42634/0 (socket says 172.21.15.38:42634) 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac32ffd640 1 -- 172.21.15.38:0/2293106881 learned_addr learned my addr 172.21.15.38:0/2293106881 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac337fe640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac3415baf0 0x7fac341554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac327fc640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac340cfb70 0x7fac34154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac32ffd640 1 -- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac3415baf0 msgr2=0x7fac341554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac32ffd640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac3415baf0 0x7fac341554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac32ffd640 1 -- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac340cfb70 msgr2=0x7fac34154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac32ffd640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac340cfb70 0x7fac34154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac32ffd640 1 -- 172.21.15.38:0/2293106881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac341574f0 con 0x7fac34072660 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.200+0000 7fac32ffd640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac34072660 0x7fac340b9c50 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fac240098c0 tx=0x7fac240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.201+0000 7fac1bfff640 1 -- 172.21.15.38:0/2293106881 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac24041020 con 0x7fac34072660 2024-04-25T08:39:52.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.201+0000 7fac1bfff640 1 -- 172.21.15.38:0/2293106881 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2403d620 con 0x7fac34072660 2024-04-25T08:39:52.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.201+0000 7fac39846640 1 -- 172.21.15.38:0/2293106881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac341577d0 con 0x7fac34072660 2024-04-25T08:39:52.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.201+0000 7fac1bfff640 1 -- 172.21.15.38:0/2293106881 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac24037040 con 0x7fac34072660 2024-04-25T08:39:52.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.201+0000 7fac39846640 1 -- 172.21.15.38:0/2293106881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac34157d90 con 0x7fac34072660 2024-04-25T08:39:52.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.202+0000 7fac39846640 1 -- 172.21.15.38:0/2293106881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac34073750 con 0x7fac34072660 2024-04-25T08:39:52.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.202+0000 7fac1bfff640 1 -- 172.21.15.38:0/2293106881 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac24044020 con 0x7fac34072660 2024-04-25T08:39:52.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.203+0000 7fac1bfff640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fac2006b4d0 0x7fac2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.203+0000 7fac1bfff640 1 -- 172.21.15.38:0/2293106881 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fac240aa650 con 0x7fac34072660 2024-04-25T08:39:52.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.203+0000 7fac327fc640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fac2006b4d0 0x7fac2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.203+0000 7fac327fc640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fac2006b4d0 0x7fac2006d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fac28000f60 tx=0x7fac28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.207+0000 7fac1bfff640 1 -- 172.21.15.38:0/2293106881 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac34073750 con 0x7fac34072660 2024-04-25T08:39:52.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.391+0000 7fac39846640 1 -- 172.21.15.38:0/2293106881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fac3406e1e0 con 0x7fac34072660 2024-04-25T08:39:52.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.395+0000 7fac1bfff640 1 -- 172.21.15.38:0/2293106881 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fac2404b030 con 0x7fac34072660 2024-04-25T08:39:52.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.397+0000 7fac39846640 1 -- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fac2006b4d0 msgr2=0x7fac2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.398+0000 7fac39846640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fac2006b4d0 0x7fac2006d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fac28000f60 tx=0x7fac28009040 comp rx=0 tx=0).stop 2024-04-25T08:39:52.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.398+0000 7fac39846640 1 -- 172.21.15.38:0/2293106881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac34072660 msgr2=0x7fac340b9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.398+0000 7fac39846640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac34072660 0x7fac340b9c50 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fac240098c0 tx=0x7fac240046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.398+0000 7fac39846640 1 -- 172.21.15.38:0/2293106881 shutdown_connections 2024-04-25T08:39:52.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.398+0000 7fac39846640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fac2006b4d0 0x7fac2006d990 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.391 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.398+0000 7fac39846640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac3415baf0 0x7fac341554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.398+0000 7fac39846640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac340cfb70 0x7fac34154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.398+0000 7fac39846640 1 --2- 172.21.15.38:0/2293106881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac34072660 0x7fac340b9c50 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.398+0000 7fac39846640 1 -- 172.21.15.38:0/2293106881 >> 172.21.15.38:0/2293106881 conn(0x7fac3406adc0 msgr2=0x7fac340b02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:52.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.398+0000 7fac39846640 1 -- 172.21.15.38:0/2293106881 shutdown_connections 2024-04-25T08:39:52.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.398+0000 7fac39846640 1 -- 172.21.15.38:0/2293106881 wait complete. 2024-04-25T08:39:52.403 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:39:52.403 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:39:52.440 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:39:52.482 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:39:52.483 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:39:52.483 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:52.498 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:39:52.498 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:39:52.498 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:52.541 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:52.541 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:39:52.541 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:52.556 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:39:52.556 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:39:52.556 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:39:52.598 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:39:52.598 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:39:52.598 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:39:52.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.705+0000 7f01fb7ab640 1 Processor -- start 2024-04-25T08:39:52.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.706+0000 7f01fb7ab640 1 -- start start 2024-04-25T08:39:52.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.706+0000 7f01fb7ab640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f406cc10 0x7f01f40cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.706+0000 7f01fb7ab640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f01f406cff0 0x7f01f40cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.706+0000 7f01fb7ab640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f01f4099a70 0x7f01f4099ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.706+0000 7f01fb7ab640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f409a420 con 0x7f01f406cff0 2024-04-25T08:39:52.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.706+0000 7f01fb7ab640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f40d05c0 con 0x7f01f4099a70 2024-04-25T08:39:52.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.706+0000 7f01fb7ab640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40d08a0 con 0x7f01f406cc10 2024-04-25T08:39:52.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.707+0000 7f01f8d46640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f01f406cff0 0x7f01f40cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.707+0000 7f01f8d46640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f01f406cff0 0x7f01f40cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42636/0 (socket says 172.21.15.38:42636) 2024-04-25T08:39:52.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.707+0000 7f01f8d46640 1 -- 172.21.15.38:0/2687680669 learned_addr learned my addr 172.21.15.38:0/2687680669 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:52.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.707+0000 7f01f9547640 1 --2- 172.21.15.38:0/2687680669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f406cc10 0x7f01f40cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.707+0000 7f01f9d48640 1 --2- 172.21.15.38:0/2687680669 >> v2:172.21.15.107:3300/0 conn(0x7f01f4099a70 0x7f01f4099ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.707+0000 7f01f8d46640 1 -- 172.21.15.38:0/2687680669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f406cc10 msgr2=0x7f01f40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:52.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.707+0000 7f01f8d46640 1 --2- 172.21.15.38:0/2687680669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f406cc10 0x7f01f40cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.707+0000 7f01f8d46640 1 -- 172.21.15.38:0/2687680669 >> v2:172.21.15.107:3300/0 conn(0x7f01f4099a70 msgr2=0x7f01f4099ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:52.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.707+0000 7f01f8d46640 1 --2- 172.21.15.38:0/2687680669 >> v2:172.21.15.107:3300/0 conn(0x7f01f4099a70 0x7f01f4099ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.707+0000 7f01f8d46640 1 -- 172.21.15.38:0/2687680669 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40d0b80 con 0x7f01f406cff0 2024-04-25T08:39:52.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01f8d46640 1 --2- 172.21.15.38:0/2687680669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01f406cff0 0x7f01f40cffb0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f01f4072a20 tx=0x7f01e802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebeef6fdfb7b2391 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01f3fff640 1 -- 172.21.15.38:0/2687680669 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8037070 con 0x7f01f406cff0 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01f3fff640 1 -- 172.21.15.38:0/2687680669 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e8032440 con 0x7f01f406cff0 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01f3fff640 1 -- 172.21.15.38:0/2687680669 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8032810 con 0x7f01f406cff0 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01fb7ab640 1 -- 172.21.15.38:0/2687680669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01f406cff0 msgr2=0x7f01f40cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01fb7ab640 1 --2- 172.21.15.38:0/2687680669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01f406cff0 0x7f01f40cffb0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f01f4072a20 tx=0x7f01e802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01fb7ab640 1 -- 172.21.15.38:0/2687680669 shutdown_connections 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01fb7ab640 1 --2- 172.21.15.38:0/2687680669 >> v2:172.21.15.107:3300/0 conn(0x7f01f4099a70 0x7f01f4099ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01fb7ab640 1 --2- 172.21.15.38:0/2687680669 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01f406cff0 0x7f01f40cffb0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01fb7ab640 1 --2- 172.21.15.38:0/2687680669 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f406cc10 0x7f01f40cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01fb7ab640 1 -- 172.21.15.38:0/2687680669 >> 172.21.15.38:0/2687680669 conn(0x7f01f40b31f0 msgr2=0x7f01f40b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01fb7ab640 1 -- 172.21.15.38:0/2687680669 shutdown_connections 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.708+0000 7f01fb7ab640 1 -- 172.21.15.38:0/2687680669 wait complete. 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.710+0000 7f01fb7ab640 1 Processor -- start 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.710+0000 7f01fb7ab640 1 -- start start 2024-04-25T08:39:52.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.710+0000 7f01fb7ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01f406cc10 0x7f01f40cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.710+0000 7f01fb7ab640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f406cff0 0x7f01f40ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.710+0000 7f01fb7ab640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f4099a70 0x7f01f414a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.710+0000 7f01fb7ab640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f406c150 con 0x7f01f406cc10 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.710+0000 7f01fb7ab640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40d1140 con 0x7f01f406cff0 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.710+0000 7f01fb7ab640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f406c2d0 con 0x7f01f4099a70 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.710+0000 7f01f9547640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01f406cc10 0x7f01f40cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.710+0000 7f01f9547640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01f406cc10 0x7f01f40cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42642/0 (socket says 172.21.15.38:42642) 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.710+0000 7f01f9547640 1 -- 172.21.15.38:0/1171342612 learned_addr learned my addr 172.21.15.38:0/1171342612 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.711+0000 7f01f8d46640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f406cff0 0x7f01f40ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.711+0000 7f01f9547640 1 -- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f4099a70 msgr2=0x7f01f414a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.711+0000 7f01f9547640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f4099a70 0x7f01f414a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.711+0000 7f01f9547640 1 -- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f406cff0 msgr2=0x7f01f40ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.711+0000 7f01f9547640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f406cff0 0x7f01f40ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.711+0000 7f01f9547640 1 -- 172.21.15.38:0/1171342612 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4149be0 con 0x7f01f406cc10 2024-04-25T08:39:52.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.711+0000 7f01f9547640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01f406cc10 0x7f01f40cc880 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f01e4007e90 tx=0x7f01e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.711+0000 7f01f1ffb640 1 -- 172.21.15.38:0/1171342612 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4014420 con 0x7f01f406cc10 2024-04-25T08:39:52.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.711+0000 7f01f1ffb640 1 -- 172.21.15.38:0/1171342612 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e4019070 con 0x7f01f406cc10 2024-04-25T08:39:52.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.711+0000 7f01f1ffb640 1 -- 172.21.15.38:0/1171342612 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40147e0 con 0x7f01f406cc10 2024-04-25T08:39:52.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.711+0000 7f01fb7ab640 1 -- 172.21.15.38:0/1171342612 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f4149ec0 con 0x7f01f406cc10 2024-04-25T08:39:52.707 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.711+0000 7f01fb7ab640 1 -- 172.21.15.38:0/1171342612 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f414ab20 con 0x7f01f406cc10 2024-04-25T08:39:52.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.712+0000 7f01fb7ab640 1 -- 172.21.15.38:0/1171342612 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f4068e80 con 0x7f01f406cc10 2024-04-25T08:39:52.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.713+0000 7f01f1ffb640 1 -- 172.21.15.38:0/1171342612 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01e4014980 con 0x7f01f406cc10 2024-04-25T08:39:52.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.713+0000 7f01f1ffb640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f01e006b520 0x7f01e006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:52.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.713+0000 7f01f1ffb640 1 -- 172.21.15.38:0/1171342612 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f01e408a710 con 0x7f01f406cc10 2024-04-25T08:39:52.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.713+0000 7f01f8d46640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f01e006b520 0x7f01e006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:52.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.714+0000 7f01f8d46640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f01e006b520 0x7f01e006d9e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f01e8034000 tx=0x7f01e802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:52.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.717+0000 7f01f1ffb640 1 -- 172.21.15.38:0/1171342612 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01e4059870 con 0x7f01f406cc10 2024-04-25T08:39:52.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.936+0000 7f01fb7ab640 1 -- 172.21.15.38:0/1171342612 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01f40cba30 con 0x7f01f406cc10 2024-04-25T08:39:52.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.936+0000 7f01f1ffb640 1 -- 172.21.15.38:0/1171342612 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f01e4059170 con 0x7f01f406cc10 2024-04-25T08:39:52.930 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:52.930 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":65,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3894595336","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3894595336},{"type":"v1","addr":"172.21.15.107:6836","nonce":3894595336}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":7564,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2302144580","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":2302144580},{"type":"v1","addr":"172.21.15.107:6837","nonce":2302144580}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8274,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/1506257549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1506257549},{"type":"v1","addr":"172.21.15.38:6834","nonce":1506257549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60},{"gid":8277,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/3949479150","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":3949479150},{"type":"v1","addr":"172.21.15.38:6835","nonce":3949479150}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":60}],"filesystems":[]} 2024-04-25T08:39:52.930 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 65 2024-04-25T08:39:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.938+0000 7f01fb7ab640 1 -- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f01e006b520 msgr2=0x7f01e006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.938+0000 7f01fb7ab640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f01e006b520 0x7f01e006d9e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f01e8034000 tx=0x7f01e802e000 comp rx=0 tx=0).stop 2024-04-25T08:39:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.938+0000 7f01fb7ab640 1 -- 172.21.15.38:0/1171342612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01f406cc10 msgr2=0x7f01f40cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.938+0000 7f01fb7ab640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01f406cc10 0x7f01f40cc880 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f01e4007e90 tx=0x7f01e400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.938+0000 7f01fb7ab640 1 -- 172.21.15.38:0/1171342612 shutdown_connections 2024-04-25T08:39:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.938+0000 7f01fb7ab640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f01e006b520 0x7f01e006d9e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.938+0000 7f01fb7ab640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01f4099a70 0x7f01f414a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.938+0000 7f01fb7ab640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f01f406cff0 0x7f01f40ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.938+0000 7f01fb7ab640 1 --2- 172.21.15.38:0/1171342612 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01f406cc10 0x7f01f40cc880 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.938+0000 7f01fb7ab640 1 -- 172.21.15.38:0/1171342612 >> 172.21.15.38:0/1171342612 conn(0x7f01f40b31f0 msgr2=0x7f01f40b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.938+0000 7f01fb7ab640 1 -- 172.21.15.38:0/1171342612 shutdown_connections 2024-04-25T08:39:52.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:52.938+0000 7f01fb7ab640 1 -- 172.21.15.38:0/1171342612 wait complete. 2024-04-25T08:39:52.943 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:39:52.943 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:39:52.943 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:39:52.944 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:39:52.944 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:39:52.944 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:39:52.944 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:39:52.944 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:39:52.944 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:39:52.944 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:39:52.945 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:39:52.945 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:39:52.945 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:39:52.945 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:39:52.945 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:39:52.945 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:39:52.953 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:39:52.953 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:39:52.954 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:39:52.954 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:39:52.956 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:39:52.956 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:39:52.957 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:39:52.957 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:39:52.995 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:39:52.995 DEBUG:teuthology.parallel:result is None 2024-04-25T08:39:52.996 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:39:52.996 DEBUG:teuthology.parallel:result is None 2024-04-25T08:39:52.999 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:39:53.000 DEBUG:teuthology.parallel:result is None 2024-04-25T08:39:53.000 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:39:53.000 DEBUG:teuthology.parallel:result is None 2024-04-25T08:39:53.000 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:39:53.071 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:39:53.071 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:39:53.076 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-25T08:39:53.077 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:39:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.114+0000 7f4423377640 1 Processor -- start 2024-04-25T08:39:53.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.114+0000 7f4423377640 1 -- start start 2024-04-25T08:39:53.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.115+0000 7f4423377640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f441c0b2ab0 0x7f441c0d2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.115+0000 7f4421914640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f441c0b2ab0 0x7f441c0d2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.115+0000 7f4420912640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f441c0b9180 0x7f441c0d1970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56434/0 (socket says 172.21.15.38:56434) 2024-04-25T08:39:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.115+0000 7f4421914640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f441c0b2ab0 0x7f441c0d2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42686/0 (socket says 172.21.15.38:42686) 2024-04-25T08:39:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.115+0000 7f4420912640 1 -- 172.21.15.38:0/524516911 learned_addr learned my addr 172.21.15.38:0/524516911 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.115+0000 7f4423377640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441c0b9610 con 0x7f441c0b9180 2024-04-25T08:39:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.115+0000 7f4423377640 1 -- 172.21.15.38:0/524516911 --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441c1616a0 con 0x7f441c160ef0 2024-04-25T08:39:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.115+0000 7f4423377640 1 -- 172.21.15.38:0/524516911 --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441c161cf0 con 0x7f441c0b2ab0 2024-04-25T08:39:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.116+0000 7f4413fff640 1 -- 172.21.15.38:0/524516911 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472379475 0 0) 0x7f441c0b9610 con 0x7f441c0b9180 2024-04-25T08:39:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.116+0000 7f4413fff640 1 -- 172.21.15.38:0/524516911 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4408003520 con 0x7f441c0b9180 2024-04-25T08:39:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.116+0000 7f4413fff640 1 -- 172.21.15.38:0/524516911 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190839723 0 0) 0x7f441c1616a0 con 0x7f441c160ef0 2024-04-25T08:39:53.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.116+0000 7f4413fff640 1 -- 172.21.15.38:0/524516911 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f441c0b9610 con 0x7f441c160ef0 2024-04-25T08:39:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.116+0000 7f4413fff640 1 -- 172.21.15.38:0/524516911 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 723963669 0 0) 0x7f4408003520 con 0x7f441c0b9180 2024-04-25T08:39:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.116+0000 7f4421914640 1 -- 172.21.15.38:0/524516911 >> v1:172.21.15.107:6789/0 conn(0x7f441c160ef0 legacy=0x7f441c0d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.116+0000 7f4421914640 1 -- 172.21.15.38:0/524516911 >> v1:172.21.15.38:6789/0 conn(0x7f441c0b9180 legacy=0x7f441c0d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.116+0000 7f4421914640 1 -- 172.21.15.38:0/524516911 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c161980 con 0x7f441c0b2ab0 2024-04-25T08:39:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.117+0000 7f4421914640 1 --2- 172.21.15.38:0/524516911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f441c0b2ab0 0x7f441c0d2080 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f4414009d90 tx=0x7f441402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681649dffe40adc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.117+0000 7f4413fff640 1 -- 172.21.15.38:0/524516911 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418001670 con 0x7f441c0b2ab0 2024-04-25T08:39:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.117+0000 7f4413fff640 1 -- 172.21.15.38:0/524516911 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4414007540 con 0x7f441c0b2ab0 2024-04-25T08:39:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.117+0000 7f4413fff640 1 -- 172.21.15.38:0/524516911 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140078f0 con 0x7f441c0b2ab0 2024-04-25T08:39:53.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.117+0000 7f4423377640 1 -- 172.21.15.38:0/524516911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f441c0b2ab0 msgr2=0x7f441c0d2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.117+0000 7f4423377640 1 --2- 172.21.15.38:0/524516911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f441c0b2ab0 0x7f441c0d2080 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f4414009d90 tx=0x7f441402a5f0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.117+0000 7f4423377640 1 -- 172.21.15.38:0/524516911 shutdown_connections 2024-04-25T08:39:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.117+0000 7f4423377640 1 --2- 172.21.15.38:0/524516911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f441c0b2ab0 0x7f441c0d2080 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.117+0000 7f4423377640 1 -- 172.21.15.38:0/524516911 >> 172.21.15.38:0/524516911 conn(0x7f441c066d10 msgr2=0x7f441c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.117+0000 7f4423377640 1 -- 172.21.15.38:0/524516911 shutdown_connections 2024-04-25T08:39:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.118+0000 7f4423377640 1 -- 172.21.15.38:0/524516911 wait complete. 2024-04-25T08:39:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.119+0000 7f4423377640 1 Processor -- start 2024-04-25T08:39:53.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.119+0000 7f4423377640 1 -- start start 2024-04-25T08:39:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4423377640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f441c0b2ab0 0x7f441c06be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4423377640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f441c160ef0 0x7f441c06c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4423377640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c14da90 0x7f441c14df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4423377640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0d0ec0 con 0x7f441c0b2ab0 2024-04-25T08:39:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4421113640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f441c0b2ab0 0x7f441c06be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4423377640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c1612d0 con 0x7f441c14da90 2024-04-25T08:39:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4423377640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0d1040 con 0x7f441c160ef0 2024-04-25T08:39:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4421113640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f441c0b2ab0 0x7f441c06be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42700/0 (socket says 172.21.15.38:42700) 2024-04-25T08:39:53.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4421113640 1 -- 172.21.15.38:0/2670742334 learned_addr learned my addr 172.21.15.38:0/2670742334 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4420912640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f441c160ef0 0x7f441c06c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4421914640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c14da90 0x7f441c14df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4421113640 1 -- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f441c160ef0 msgr2=0x7f441c06c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4421113640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f441c160ef0 0x7f441c06c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4421113640 1 -- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c14da90 msgr2=0x7f441c14df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4421113640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c14da90 0x7f441c14df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4421113640 1 -- 172.21.15.38:0/2670742334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c14baa0 con 0x7f441c0b2ab0 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4421914640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c14da90 0x7f441c14df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4421113640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f441c0b2ab0 0x7f441c06be00 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f440c00c460 tx=0x7f440c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4411ffb640 1 -- 172.21.15.38:0/2670742334 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c0052d0 con 0x7f441c0b2ab0 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4411ffb640 1 -- 172.21.15.38:0/2670742334 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f440c005490 con 0x7f441c0b2ab0 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.120+0000 7f4423377640 1 -- 172.21.15.38:0/2670742334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c14bd80 con 0x7f441c0b2ab0 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.121+0000 7f4411ffb640 1 -- 172.21.15.38:0/2670742334 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f440c012870 con 0x7f441c0b2ab0 2024-04-25T08:39:53.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.121+0000 7f4423377640 1 -- 172.21.15.38:0/2670742334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c0abf30 con 0x7f441c0b2ab0 2024-04-25T08:39:53.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.122+0000 7f4423377640 1 -- 172.21.15.38:0/2670742334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c04ef90 con 0x7f441c0b2ab0 2024-04-25T08:39:53.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.122+0000 7f4411ffb640 1 -- 172.21.15.38:0/2670742334 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f440c012a10 con 0x7f441c0b2ab0 2024-04-25T08:39:53.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.123+0000 7f4411ffb640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f440006b4d0 0x7f440006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:53.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.123+0000 7f4411ffb640 1 -- 172.21.15.38:0/2670742334 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f440c089d80 con 0x7f441c0b2ab0 2024-04-25T08:39:53.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.123+0000 7f4420912640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f440006b4d0 0x7f440006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:53.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.123+0000 7f4420912640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f440006b4d0 0x7f440006d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f441c152110 tx=0x7f4418009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:53.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.127+0000 7f4411ffb640 1 -- 172.21.15.38:0/2670742334 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f440c058cd0 con 0x7f441c0b2ab0 2024-04-25T08:39:53.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:53.323+0000 7f4423377640 1 -- 172.21.15.38:0/2670742334 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f441c0b8870 con 0x7f441c0b2ab0 2024-04-25T08:39:54.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.089+0000 7f4411ffb640 1 -- 172.21.15.38:0/2670742334 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v115) v1 ==== 97+0+0 (secure 0 0 0) 0x7f440c0585d0 con 0x7f441c0b2ab0 2024-04-25T08:39:54.083 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-04-25T08:39:54.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.092+0000 7f4423377640 1 -- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f440006b4d0 msgr2=0x7f440006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.092+0000 7f4423377640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f440006b4d0 0x7f440006d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f441c152110 tx=0x7f4418009040 comp rx=0 tx=0).stop 2024-04-25T08:39:54.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.092+0000 7f4423377640 1 -- 172.21.15.38:0/2670742334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f441c0b2ab0 msgr2=0x7f441c06be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.092+0000 7f4423377640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f441c0b2ab0 0x7f441c06be00 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f440c00c460 tx=0x7f440c00c920 comp rx=0 tx=0).stop 2024-04-25T08:39:54.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.093+0000 7f4423377640 1 -- 172.21.15.38:0/2670742334 shutdown_connections 2024-04-25T08:39:54.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.093+0000 7f4423377640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f441c14da90 0x7f441c14df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.093+0000 7f4423377640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f440006b4d0 0x7f440006d990 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.093+0000 7f4423377640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f441c160ef0 0x7f441c06c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.093+0000 7f4423377640 1 --2- 172.21.15.38:0/2670742334 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f441c0b2ab0 0x7f441c06be00 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.093+0000 7f4423377640 1 -- 172.21.15.38:0/2670742334 >> 172.21.15.38:0/2670742334 conn(0x7f441c066d10 msgr2=0x7f441c15f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:54.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.093+0000 7f4423377640 1 -- 172.21.15.38:0/2670742334 shutdown_connections 2024-04-25T08:39:54.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.093+0000 7f4423377640 1 -- 172.21.15.38:0/2670742334 wait complete. 2024-04-25T08:39:54.101 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:39:54.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.214+0000 7f0157f77640 1 Processor -- start 2024-04-25T08:39:54.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.215+0000 7f0157f77640 1 -- start start 2024-04-25T08:39:54.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.215+0000 7f0157f77640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f015015df50 0x7f01500ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.215+0000 7f0157f77640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f01500ce2e0 0x7f01500d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.215+0000 7f0157f77640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01500ceff0 con 0x7f015006c110 2024-04-25T08:39:54.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.215+0000 7f0157f77640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01500cf2d0 con 0x7f01500ce2e0 2024-04-25T08:39:54.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.215+0000 7f0157f77640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01500d1f70 con 0x7f015015df50 2024-04-25T08:39:54.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.216+0000 7f0155d13640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f015006c110 0x7f015006c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56438/0 (socket says 172.21.15.38:56438) 2024-04-25T08:39:54.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.216+0000 7f0156514640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f01500ce2e0 0x7f01500d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.216+0000 7f0155d13640 1 -- 172.21.15.38:0/1835891001 learned_addr learned my addr 172.21.15.38:0/1835891001 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:54.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.216+0000 7f0155512640 1 --2- 172.21.15.38:0/1835891001 >> v2:172.21.15.107:3300/0 conn(0x7f015015df50 0x7f01500ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.216+0000 7f0156514640 1 -- 172.21.15.38:0/1835891001 >> v2:172.21.15.107:3300/0 conn(0x7f015015df50 msgr2=0x7f01500ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:54.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.216+0000 7f0156514640 1 --2- 172.21.15.38:0/1835891001 >> v2:172.21.15.107:3300/0 conn(0x7f015015df50 0x7f01500ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.216+0000 7f0156514640 1 -- 172.21.15.38:0/1835891001 >> v1:172.21.15.38:6789/0 conn(0x7f015006c110 legacy=0x7f015006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.216+0000 7f0156514640 1 -- 172.21.15.38:0/1835891001 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01500d2250 con 0x7f01500ce2e0 2024-04-25T08:39:54.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.217+0000 7f0156514640 1 --2- 172.21.15.38:0/1835891001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01500ce2e0 0x7f01500d2a50 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f01500d4960 tx=0x7f014002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55570025b03adb1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.217+0000 7f0154d11640 1 -- 172.21.15.38:0/1835891001 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140007740 con 0x7f01500ce2e0 2024-04-25T08:39:54.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.217+0000 7f0154d11640 1 -- 172.21.15.38:0/1835891001 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01400078e0 con 0x7f01500ce2e0 2024-04-25T08:39:54.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.217+0000 7f0154d11640 1 -- 172.21.15.38:0/1835891001 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0140007cb0 con 0x7f01500ce2e0 2024-04-25T08:39:54.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.217+0000 7f0157f77640 1 -- 172.21.15.38:0/1835891001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01500ce2e0 msgr2=0x7f01500d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.217+0000 7f0157f77640 1 --2- 172.21.15.38:0/1835891001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01500ce2e0 0x7f01500d2a50 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f01500d4960 tx=0x7f014002a680 comp rx=0 tx=0).stop 2024-04-25T08:39:54.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.217+0000 7f0157f77640 1 -- 172.21.15.38:0/1835891001 shutdown_connections 2024-04-25T08:39:54.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.217+0000 7f0157f77640 1 --2- 172.21.15.38:0/1835891001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f01500ce2e0 0x7f01500d2a50 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.217+0000 7f0157f77640 1 --2- 172.21.15.38:0/1835891001 >> v2:172.21.15.107:3300/0 conn(0x7f015015df50 0x7f01500ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.217+0000 7f0157f77640 1 -- 172.21.15.38:0/1835891001 >> 172.21.15.38:0/1835891001 conn(0x7f01500d3fb0 msgr2=0x7f01500d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:54.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.217+0000 7f0157f77640 1 -- 172.21.15.38:0/1835891001 shutdown_connections 2024-04-25T08:39:54.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.217+0000 7f0157f77640 1 -- 172.21.15.38:0/1835891001 wait complete. 2024-04-25T08:39:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.219+0000 7f0157f77640 1 Processor -- start 2024-04-25T08:39:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.219+0000 7f0157f77640 1 -- start start 2024-04-25T08:39:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.219+0000 7f0157f77640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015006c110 0x7f01500c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.219+0000 7f0157f77640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01500ce2e0 0x7f01500b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.219+0000 7f0157f77640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f015015df50 0x7f01500b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.219+0000 7f0157f77640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01500d2f90 con 0x7f015006c110 2024-04-25T08:39:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.219+0000 7f0157f77640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01500cece0 con 0x7f015015df50 2024-04-25T08:39:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.219+0000 7f0157f77640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01500d3110 con 0x7f01500ce2e0 2024-04-25T08:39:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.219+0000 7f0155d13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015006c110 0x7f01500c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f0155d13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015006c110 0x7f01500c9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42724/0 (socket says 172.21.15.38:42724) 2024-04-25T08:39:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f0155d13640 1 -- 172.21.15.38:0/1748817119 learned_addr learned my addr 172.21.15.38:0/1748817119 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f0156514640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f015015df50 0x7f01500b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f0155512640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01500ce2e0 0x7f01500b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f0155d13640 1 -- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01500ce2e0 msgr2=0x7f01500b43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f0155d13640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01500ce2e0 0x7f01500b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f0155d13640 1 -- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f015015df50 msgr2=0x7f01500b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f0155d13640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f015015df50 0x7f01500b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f0155d13640 1 -- 172.21.15.38:0/1748817119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0150145810 con 0x7f015006c110 2024-04-25T08:39:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f0155d13640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015006c110 0x7f01500c9e40 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f014800c430 tx=0x7f014800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f014e7fc640 1 -- 172.21.15.38:0/1748817119 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148017070 con 0x7f015006c110 2024-04-25T08:39:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f014e7fc640 1 -- 172.21.15.38:0/1748817119 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0148005ce0 con 0x7f015006c110 2024-04-25T08:39:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f014e7fc640 1 -- 172.21.15.38:0/1748817119 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01480072f0 con 0x7f015006c110 2024-04-25T08:39:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f0157f77640 1 -- 172.21.15.38:0/1748817119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0150160260 con 0x7f015006c110 2024-04-25T08:39:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.220+0000 7f0157f77640 1 -- 172.21.15.38:0/1748817119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0150160820 con 0x7f015006c110 2024-04-25T08:39:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.221+0000 7f0157f77640 1 -- 172.21.15.38:0/1748817119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0150068e80 con 0x7f015006c110 2024-04-25T08:39:54.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.222+0000 7f014e7fc640 1 -- 172.21.15.38:0/1748817119 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f014801a070 con 0x7f015006c110 2024-04-25T08:39:54.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.222+0000 7f014e7fc640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f013c06b500 0x7f013c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.222+0000 7f014e7fc640 1 -- 172.21.15.38:0/1748817119 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0148089440 con 0x7f015006c110 2024-04-25T08:39:54.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.222+0000 7f0155512640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f013c06b500 0x7f013c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.223+0000 7f0155512640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f013c06b500 0x7f013c06d9c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0144006fd0 tx=0x7f0144008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.226+0000 7f014e7fc640 1 -- 172.21.15.38:0/1748817119 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0148058480 con 0x7f015006c110 2024-04-25T08:39:54.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.473+0000 7f0157f77640 1 -- 172.21.15.38:0/1748817119 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f01500ca780 con 0x7f015006c110 2024-04-25T08:39:54.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.474+0000 7f014e7fc640 1 -- 172.21.15.38:0/1748817119 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0148057d80 con 0x7f015006c110 2024-04-25T08:39:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:54.467 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:39:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-04-25T08:39:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-04-25T08:39:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmwq8CDBAA+cjd+QXtFgwZ3kxdABRWlA==", 2024-04-25T08:39:54.467 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmPRFbFxAAojn2r/62vvf1qTy9QVuyIw==", 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.468 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmGLjqEhAAyPRgyy5iMgtccapydv4uOQ==", 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.469 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmt0AUHhAAoQyQ7wXBMGE+7Hi84tUfAg==", 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCxFCpmv+iTExAAEJihmsvBbp9xXxPnj49D6g==", 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:54.470 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC0FCpmdnQKFBAADLAsYZEUhoJx292jIYZroQ==", 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC3FCpm4f5BExAAMVSPkqsrnuLcO88AXKrGHQ==", 2024-04-25T08:39:54.471 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC6FCpmHNN7EhAABPhmnxGo70ZOd8Oex62Mdw==", 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.472 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDAFCpmiThGFhAAG5Pyvn/JzL9NUoxHJkPMvg==", 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDDFCpmb/FHFBAA2cMhMqQQc4Fk6j8FpjhnaQ==", 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:54.473 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDGFCpmAZsSEhAAnGIqqcvuHMVSc8Zbk309kA==", 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.474 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDJFCpmqdVUEBAAb9abblyUztrjwxxe4jsAwQ==", 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw==", 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.475 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmL7eKKhAASeQEdHDBInfozdLVzP4DQw==", 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmH22HJxAAWyrooyJn5hRs/bb0Rn1HHw==", 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:54.476 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpm6Zr/LxAAxh9Y2gvvqj1jIaVVrQAvgw==", 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.477 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCpFCpmlj/fLBAAvxLluFQFR8uhppGlNpaygA==", 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcXHhCBAAvI5zmWidVTvOmEcVRWasEA==", 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.478 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcr96BRAAQwUPxILJukYERAiEXeur+g==", 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T08:39:54.479 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:39:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.476+0000 7f0157f77640 1 -- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f013c06b500 msgr2=0x7f013c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.476+0000 7f0157f77640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f013c06b500 0x7f013c06d9c0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f0144006fd0 tx=0x7f0144008040 comp rx=0 tx=0).stop 2024-04-25T08:39:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.476+0000 7f0157f77640 1 -- 172.21.15.38:0/1748817119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015006c110 msgr2=0x7f01500c9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.476+0000 7f0157f77640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015006c110 0x7f01500c9e40 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f014800c430 tx=0x7f014800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.476+0000 7f0157f77640 1 -- 172.21.15.38:0/1748817119 shutdown_connections 2024-04-25T08:39:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.476+0000 7f0157f77640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f015015df50 0x7f01500b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.476+0000 7f0157f77640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f013c06b500 0x7f013c06d9c0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.476+0000 7f0157f77640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f01500ce2e0 0x7f01500b43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.476+0000 7f0157f77640 1 --2- 172.21.15.38:0/1748817119 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f015006c110 0x7f01500c9e40 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.476+0000 7f0157f77640 1 -- 172.21.15.38:0/1748817119 >> 172.21.15.38:0/1748817119 conn(0x7f01500d3fb0 msgr2=0x7f015015bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.476+0000 7f0157f77640 1 -- 172.21.15.38:0/1748817119 shutdown_connections 2024-04-25T08:39:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.476+0000 7f0157f77640 1 -- 172.21.15.38:0/1748817119 wait complete. 2024-04-25T08:39:54.482 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:54.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.590+0000 7fea049ff640 1 Processor -- start 2024-04-25T08:39:54.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.590+0000 7fea049ff640 1 -- start start 2024-04-25T08:39:54.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.591+0000 7fea049ff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fea000b64f0 0x7fea000d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.591+0000 7fea049ff640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea000cef10 con 0x7fea00161180 2024-04-25T08:39:54.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.591+0000 7fea049ff640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea000cf740 con 0x7fea00160da0 2024-04-25T08:39:54.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.591+0000 7fea049ff640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea000cf180 con 0x7fea000b64f0 2024-04-25T08:39:54.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.591+0000 7fe9fe59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fea000b64f0 0x7fea000d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.591+0000 7fe9ff59e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fea00161180 0x7fea000d1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56444/0 (socket says 172.21.15.38:56444) 2024-04-25T08:39:54.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.591+0000 7fe9ff59e640 1 -- 172.21.15.38:0/359901913 learned_addr learned my addr 172.21.15.38:0/359901913 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:54.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.591+0000 7fe9fe59c640 1 -- 172.21.15.38:0/359901913 >> v1:172.21.15.107:6789/0 conn(0x7fea00160da0 legacy=0x7fea000d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.591+0000 7fe9fe59c640 1 -- 172.21.15.38:0/359901913 >> v1:172.21.15.38:6789/0 conn(0x7fea00161180 legacy=0x7fea000d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.591+0000 7fe9fdd9b640 1 -- 172.21.15.38:0/359901913 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 249277808 0 0) 0x7fea000cef10 con 0x7fea00161180 2024-04-25T08:39:54.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.591+0000 7fe9fe59c640 1 -- 172.21.15.38:0/359901913 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea000cf460 con 0x7fea000b64f0 2024-04-25T08:39:54.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.592+0000 7fe9fe59c640 1 --2- 172.21.15.38:0/359901913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea000b64f0 0x7fea000d0020 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40097a0 tx=0x7fe9f402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb62d7485ef340f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.592+0000 7fe9fdd9b640 1 -- 172.21.15.38:0/359901913 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4037070 con 0x7fea000b64f0 2024-04-25T08:39:54.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.592+0000 7fe9fdd9b640 1 -- 172.21.15.38:0/359901913 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f4032440 con 0x7fea000b64f0 2024-04-25T08:39:54.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.592+0000 7fe9fdd9b640 1 -- 172.21.15.38:0/359901913 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f402ae60 con 0x7fea000b64f0 2024-04-25T08:39:54.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.593+0000 7fea049ff640 1 -- 172.21.15.38:0/359901913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea000b64f0 msgr2=0x7fea000d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.593+0000 7fea049ff640 1 --2- 172.21.15.38:0/359901913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea000b64f0 0x7fea000d0020 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40097a0 tx=0x7fe9f402a110 comp rx=0 tx=0).stop 2024-04-25T08:39:54.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.593+0000 7fea049ff640 1 -- 172.21.15.38:0/359901913 shutdown_connections 2024-04-25T08:39:54.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.593+0000 7fea049ff640 1 --2- 172.21.15.38:0/359901913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea000b64f0 0x7fea000d0020 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.593+0000 7fea049ff640 1 -- 172.21.15.38:0/359901913 >> 172.21.15.38:0/359901913 conn(0x7fea0008b000 msgr2=0x7fea000d36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:54.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.593+0000 7fea049ff640 1 -- 172.21.15.38:0/359901913 shutdown_connections 2024-04-25T08:39:54.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.593+0000 7fea049ff640 1 -- 172.21.15.38:0/359901913 wait complete. 2024-04-25T08:39:54.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.594+0000 7fea049ff640 1 Processor -- start 2024-04-25T08:39:54.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.594+0000 7fea049ff640 1 -- start start 2024-04-25T08:39:54.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fea049ff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea000b64f0 0x7fea000c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fea049ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea00160da0 0x7fea001f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fea049ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea00161180 0x7fea000b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fea049ff640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea000b4890 con 0x7fea00160da0 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fea049ff640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea000b4710 con 0x7fea00161180 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fea049ff640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea000d0560 con 0x7fea000b64f0 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fe9fe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea00160da0 0x7fea001f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fe9fe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea00160da0 0x7fea001f2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42738/0 (socket says 172.21.15.38:42738) 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fe9fe59c640 1 -- 172.21.15.38:0/2379636572 learned_addr learned my addr 172.21.15.38:0/2379636572 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fe9ff59e640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea00161180 0x7fea000b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fe9fed9d640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea000b64f0 0x7fea000c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fe9fe59c640 1 -- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea000b64f0 msgr2=0x7fea000c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fe9fe59c640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea000b64f0 0x7fea000c6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fe9fe59c640 1 -- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea00161180 msgr2=0x7fea000b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fe9fe59c640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea00161180 0x7fea000b7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fe9fe59c640 1 -- 172.21.15.38:0/2379636572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea000c77a0 con 0x7fea00160da0 2024-04-25T08:39:54.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.595+0000 7fe9ff59e640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea00161180 0x7fea000b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:54.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.596+0000 7fe9fe59c640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea00160da0 0x7fea001f2db0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fe9f402a700 tx=0x7fe9f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.596+0000 7fe9e77fe640 1 -- 172.21.15.38:0/2379636572 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4037040 con 0x7fea00160da0 2024-04-25T08:39:54.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.596+0000 7fe9e77fe640 1 -- 172.21.15.38:0/2379636572 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f4042070 con 0x7fea00160da0 2024-04-25T08:39:54.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.596+0000 7fea049ff640 1 -- 172.21.15.38:0/2379636572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea000c7a80 con 0x7fea00160da0 2024-04-25T08:39:54.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.596+0000 7fe9e77fe640 1 -- 172.21.15.38:0/2379636572 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4030c70 con 0x7fea00160da0 2024-04-25T08:39:54.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.596+0000 7fea049ff640 1 -- 172.21.15.38:0/2379636572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea000c7d60 con 0x7fea00160da0 2024-04-25T08:39:54.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.597+0000 7fea049ff640 1 -- 172.21.15.38:0/2379636572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0004ef20 con 0x7fea00160da0 2024-04-25T08:39:54.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.597+0000 7fe9e77fe640 1 -- 172.21.15.38:0/2379636572 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9f4047460 con 0x7fea00160da0 2024-04-25T08:39:54.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.598+0000 7fe9e77fe640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.598+0000 7fe9e77fe640 1 -- 172.21.15.38:0/2379636572 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe9f40aad20 con 0x7fea00160da0 2024-04-25T08:39:54.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.598+0000 7fe9fed9d640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.598+0000 7fe9fed9d640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0000f60 tx=0x7fe9f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.602+0000 7fe9e77fe640 1 -- 172.21.15.38:0/2379636572 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9f4045360 con 0x7fea00160da0 2024-04-25T08:39:54.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.789+0000 7fea049ff640 1 -- 172.21.15.38:0/2379636572 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea0006c1d0 con 0x7fea00160da0 2024-04-25T08:39:54.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.790+0000 7fe9e77fe640 1 -- 172.21.15.38:0/2379636572 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+13162 (secure 0 0 0) 0x7fe9f4079df0 con 0x7fea00160da0 2024-04-25T08:39:54.783 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:54.783 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":117,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:54.103498+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:54.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.792+0000 7fea049ff640 1 -- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9ec06b4d0 msgr2=0x7fe9ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.792+0000 7fea049ff640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0000f60 tx=0x7fe9f0009040 comp rx=0 tx=0).stop 2024-04-25T08:39:54.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.792+0000 7fea049ff640 1 -- 172.21.15.38:0/2379636572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea00160da0 msgr2=0x7fea001f2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.792+0000 7fea049ff640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea00160da0 0x7fea001f2db0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7fe9f402a700 tx=0x7fe9f40046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.792+0000 7fea049ff640 1 -- 172.21.15.38:0/2379636572 shutdown_connections 2024-04-25T08:39:54.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.792+0000 7fea049ff640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea00161180 0x7fea000b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.792+0000 7fea049ff640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fea00160da0 0x7fea001f2db0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.792+0000 7fea049ff640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9ec06b4d0 0x7fe9ec06d990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.792+0000 7fea049ff640 1 --2- 172.21.15.38:0/2379636572 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea000b64f0 0x7fea000c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.792+0000 7fea049ff640 1 -- 172.21.15.38:0/2379636572 >> 172.21.15.38:0/2379636572 conn(0x7fea0008b000 msgr2=0x7fea0006d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:54.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.792+0000 7fea049ff640 1 -- 172.21.15.38:0/2379636572 shutdown_connections 2024-04-25T08:39:54.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.792+0000 7fea049ff640 1 -- 172.21.15.38:0/2379636572 wait complete. 2024-04-25T08:39:54.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:39:54.797 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:39:54.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.907+0000 7f49961f4640 1 Processor -- start 2024-04-25T08:39:54.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.907+0000 7f49961f4640 1 -- start start 2024-04-25T08:39:54.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.908+0000 7f49961f4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f49900d0320 0x7f49900d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.908+0000 7f49961f4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900b6980 0x7f49900b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.908+0000 7f49961f4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49900d2060 con 0x7f49900d0700 2024-04-25T08:39:54.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.908+0000 7f49961f4640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49900d2330 con 0x7f49900d0320 2024-04-25T08:39:54.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.908+0000 7f49961f4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900d2610 con 0x7f49900b6980 2024-04-25T08:39:54.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.908+0000 7f498f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f49900d0320 0x7f49900d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.908+0000 7f498effd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f49900d0700 0x7f49900d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56446/0 (socket says 172.21.15.38:56446) 2024-04-25T08:39:54.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.908+0000 7f498effd640 1 -- 172.21.15.38:0/2686343706 learned_addr learned my addr 172.21.15.38:0/2686343706 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:54.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.909+0000 7f498ffff640 1 --2- 172.21.15.38:0/2686343706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900b6980 0x7f49900b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.909+0000 7f498f7fe640 1 -- 172.21.15.38:0/2686343706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900b6980 msgr2=0x7f49900b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:54.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.909+0000 7f498f7fe640 1 --2- 172.21.15.38:0/2686343706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900b6980 0x7f49900b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.909+0000 7f498f7fe640 1 -- 172.21.15.38:0/2686343706 >> v1:172.21.15.38:6789/0 conn(0x7f49900d0700 legacy=0x7f49900d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.909+0000 7f498f7fe640 1 -- 172.21.15.38:0/2686343706 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900d28f0 con 0x7f49900d0320 2024-04-25T08:39:54.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.909+0000 7f498e7fc640 1 -- 172.21.15.38:0/2686343706 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700172113 0 0) 0x7f49900d2060 con 0x7f49900d0700 2024-04-25T08:39:54.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.909+0000 7f498ffff640 1 --2- 172.21.15.38:0/2686343706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900b6980 0x7f49900b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:54.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.909+0000 7f498f7fe640 1 --2- 172.21.15.38:0/2686343706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49900d0320 0x7f49900d33d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f49800097a0 tx=0x7f498002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0a3c3c21c9c506 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.910+0000 7f498e7fc640 1 -- 172.21.15.38:0/2686343706 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980037070 con 0x7f49900d0320 2024-04-25T08:39:54.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.910+0000 7f498e7fc640 1 -- 172.21.15.38:0/2686343706 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4980032440 con 0x7f49900d0320 2024-04-25T08:39:54.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.910+0000 7f498e7fc640 1 -- 172.21.15.38:0/2686343706 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980032810 con 0x7f49900d0320 2024-04-25T08:39:54.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.910+0000 7f49961f4640 1 -- 172.21.15.38:0/2686343706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49900d0320 msgr2=0x7f49900d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.910+0000 7f49961f4640 1 --2- 172.21.15.38:0/2686343706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49900d0320 0x7f49900d33d0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f49800097a0 tx=0x7f498002a210 comp rx=0 tx=0).stop 2024-04-25T08:39:54.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.910+0000 7f49961f4640 1 -- 172.21.15.38:0/2686343706 shutdown_connections 2024-04-25T08:39:54.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.910+0000 7f49961f4640 1 --2- 172.21.15.38:0/2686343706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900b6980 0x7f49900b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.910+0000 7f49961f4640 1 --2- 172.21.15.38:0/2686343706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49900d0320 0x7f49900d33d0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.910+0000 7f49961f4640 1 -- 172.21.15.38:0/2686343706 >> 172.21.15.38:0/2686343706 conn(0x7f49900668f0 msgr2=0x7f4990066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:54.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.910+0000 7f49961f4640 1 -- 172.21.15.38:0/2686343706 shutdown_connections 2024-04-25T08:39:54.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.910+0000 7f49961f4640 1 -- 172.21.15.38:0/2686343706 wait complete. 2024-04-25T08:39:54.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.911+0000 7f49961f4640 1 Processor -- start 2024-04-25T08:39:54.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.912+0000 7f49961f4640 1 -- start start 2024-04-25T08:39:54.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.912+0000 7f49961f4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900b6980 0x7f49900b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.912+0000 7f49961f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49900d0320 0x7f4990157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.912+0000 7f49961f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49900d0700 0x7f49901567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.912+0000 7f49961f4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49900d0b40 con 0x7f49900d0320 2024-04-25T08:39:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.912+0000 7f49961f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49900d2eb0 con 0x7f49900d0700 2024-04-25T08:39:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.912+0000 7f49961f4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4990158230 con 0x7f49900b6980 2024-04-25T08:39:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.912+0000 7f498effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49900d0320 0x7f4990157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.912+0000 7f498effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49900d0320 0x7f4990157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42750/0 (socket says 172.21.15.38:42750) 2024-04-25T08:39:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.912+0000 7f498effd640 1 -- 172.21.15.38:0/2597365065 learned_addr learned my addr 172.21.15.38:0/2597365065 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f498ffff640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49900d0700 0x7f49901567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f498f7fe640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900b6980 0x7f49900b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f498effd640 1 -- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900b6980 msgr2=0x7f49900b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f498effd640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900b6980 0x7f49900b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f498effd640 1 -- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49900d0700 msgr2=0x7f49901567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:54.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f498effd640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49900d0700 0x7f49901567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:54.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f498effd640 1 -- 172.21.15.38:0/2597365065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4990157110 con 0x7f49900d0320 2024-04-25T08:39:54.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f498ffff640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49900d0700 0x7f49901567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:54.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f498effd640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49900d0320 0x7f4990157cf0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f498400c3f0 tx=0x7f498400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f4977fff640 1 -- 172.21.15.38:0/2597365065 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49840074e0 con 0x7f49900d0320 2024-04-25T08:39:54.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f4977fff640 1 -- 172.21.15.38:0/2597365065 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49840047e0 con 0x7f49900d0320 2024-04-25T08:39:54.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f49961f4640 1 -- 172.21.15.38:0/2597365065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49900b3410 con 0x7f49900d0320 2024-04-25T08:39:54.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f4977fff640 1 -- 172.21.15.38:0/2597365065 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984004bd0 con 0x7f49900d0320 2024-04-25T08:39:54.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.913+0000 7f49961f4640 1 -- 172.21.15.38:0/2597365065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4990156ce0 con 0x7f49900d0320 2024-04-25T08:39:54.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.914+0000 7f49961f4640 1 -- 172.21.15.38:0/2597365065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4990068e80 con 0x7f49900d0320 2024-04-25T08:39:54.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.915+0000 7f4977fff640 1 -- 172.21.15.38:0/2597365065 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4984011070 con 0x7f49900d0320 2024-04-25T08:39:54.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.915+0000 7f4977fff640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f497c06b4d0 0x7f497c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:54.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.915+0000 7f4977fff640 1 -- 172.21.15.38:0/2597365065 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4984089ce0 con 0x7f49900d0320 2024-04-25T08:39:54.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.915+0000 7f498f7fe640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f497c06b4d0 0x7f497c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:54.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.916+0000 7f498f7fe640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f497c06b4d0 0x7f497c06d990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4980034000 tx=0x7f498002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:54.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:54.919+0000 7f4977fff640 1 -- 172.21.15.38:0/2597365065 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f498401a2b0 con 0x7f49900d0320 2024-04-25T08:39:55.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.102+0000 7f49961f4640 1 -- 172.21.15.38:0/2597365065 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f499015f130 con 0x7f49900d0320 2024-04-25T08:39:55.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.103+0000 7f4977fff640 1 -- 172.21.15.38:0/2597365065 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4984058ca0 con 0x7f49900d0320 2024-04-25T08:39:55.096 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:39:55.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.105+0000 7f49961f4640 1 -- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f497c06b4d0 msgr2=0x7f497c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.105+0000 7f49961f4640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f497c06b4d0 0x7f497c06d990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f4980034000 tx=0x7f498002e000 comp rx=0 tx=0).stop 2024-04-25T08:39:55.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.105+0000 7f49961f4640 1 -- 172.21.15.38:0/2597365065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49900d0320 msgr2=0x7f4990157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.105+0000 7f49961f4640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49900d0320 0x7f4990157cf0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f498400c3f0 tx=0x7f498400c8b0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.105+0000 7f49961f4640 1 -- 172.21.15.38:0/2597365065 shutdown_connections 2024-04-25T08:39:55.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.105+0000 7f49961f4640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49900d0700 0x7f49901567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.105+0000 7f49961f4640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49900d0320 0x7f4990157cf0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.105+0000 7f49961f4640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f497c06b4d0 0x7f497c06d990 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.105+0000 7f49961f4640 1 --2- 172.21.15.38:0/2597365065 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49900b6980 0x7f49900b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.105+0000 7f49961f4640 1 -- 172.21.15.38:0/2597365065 >> 172.21.15.38:0/2597365065 conn(0x7f49900668f0 msgr2=0x7f49900b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:55.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.105+0000 7f49961f4640 1 -- 172.21.15.38:0/2597365065 shutdown_connections 2024-04-25T08:39:55.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.105+0000 7f49961f4640 1 -- 172.21.15.38:0/2597365065 wait complete. 2024-04-25T08:39:55.109 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:39:55.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.217+0000 7f0f9a607640 1 Processor -- start 2024-04-25T08:39:55.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.218+0000 7f0f9a607640 1 -- start start 2024-04-25T08:39:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.218+0000 7f0f9a607640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0f940b79b0 0x7f0f94168730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.218+0000 7f0f9a607640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f940d1b60 con 0x7f0f9415da90 2024-04-25T08:39:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.218+0000 7f0f9a607640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f940d3680 con 0x7f0f940ceca0 2024-04-25T08:39:55.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.218+0000 7f0f9a607640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f940d2bc0 con 0x7f0f940b79b0 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.218+0000 7f0f93fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0f9415da90 0x7f0f940d2f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56460/0 (socket says 172.21.15.38:56460) 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.218+0000 7f0f93fff640 1 -- 172.21.15.38:0/4261506233 learned_addr learned my addr 172.21.15.38:0/4261506233 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f98ba4640 1 --2- 172.21.15.38:0/4261506233 >> v2:172.21.15.107:3300/0 conn(0x7f0f940b79b0 0x7f0f94168730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2160582624 0 0) 0x7f0f940d1b60 con 0x7f0f9415da90 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f80003520 con 0x7f0f9415da90 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1994140288 0 0) 0x7f0f80003520 con 0x7f0f9415da90 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 >> v2:172.21.15.107:3300/0 conn(0x7f0f940b79b0 msgr2=0x7f0f94168730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 --2- 172.21.15.38:0/4261506233 >> v2:172.21.15.107:3300/0 conn(0x7f0f940b79b0 0x7f0f94168730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 >> v1:172.21.15.107:6789/0 conn(0x7f0f940ceca0 legacy=0x7f0f940d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f940d1e00 con 0x7f0f9415da90 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f0f7c001670 con 0x7f0f9415da90 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 >> v1:172.21.15.38:6789/0 conn(0x7f0f9415da90 legacy=0x7f0f940d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 --2- 172.21.15.38:0/4261506233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f800039b0 0x7f0f80003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 --2- 172.21.15.38:0/4261506233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f800043b0 0x7f0f80004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 --2- 172.21.15.38:0/4261506233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f80005080 0x7f0f8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f940d2bc0 con 0x7f0f800043b0 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8000bfe0 con 0x7f0f80005080 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8000c1e0 con 0x7f0f800039b0 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f937fe640 1 --2- 172.21.15.38:0/4261506233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f800043b0 0x7f0f80004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f93fff640 1 --2- 172.21.15.38:0/4261506233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f800039b0 0x7f0f80003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.219+0000 7f0f98ba4640 1 --2- 172.21.15.38:0/4261506233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f80005080 0x7f0f8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.220+0000 7f0f937fe640 1 -- 172.21.15.38:0/4261506233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f800039b0 msgr2=0x7f0f80003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.220+0000 7f0f937fe640 1 --2- 172.21.15.38:0/4261506233 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f800039b0 0x7f0f80003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.220+0000 7f0f937fe640 1 -- 172.21.15.38:0/4261506233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f80005080 msgr2=0x7f0f8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.220+0000 7f0f937fe640 1 --2- 172.21.15.38:0/4261506233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f80005080 0x7f0f8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.220+0000 7f0f937fe640 1 -- 172.21.15.38:0/4261506233 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f80003520 con 0x7f0f800043b0 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.220+0000 7f0f98ba4640 1 -- 172.21.15.38:0/4261506233 reap_dead start 2024-04-25T08:39:55.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.220+0000 7f0f937fe640 1 --2- 172.21.15.38:0/4261506233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f800043b0 0x7f0f80004880 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0f84001860 tx=0x7f0f80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cd8085162ce61d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:55.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.221+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84037070 con 0x7f0f800043b0 2024-04-25T08:39:55.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.221+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8402abe0 con 0x7f0f800043b0 2024-04-25T08:39:55.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.221+0000 7f0f92ffd640 1 -- 172.21.15.38:0/4261506233 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84001430 con 0x7f0f800043b0 2024-04-25T08:39:55.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.221+0000 7f0f9a607640 1 -- 172.21.15.38:0/4261506233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f800043b0 msgr2=0x7f0f80004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.221+0000 7f0f9a607640 1 --2- 172.21.15.38:0/4261506233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f800043b0 0x7f0f80004880 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f0f84001860 tx=0x7f0f80005830 comp rx=0 tx=0).stop 2024-04-25T08:39:55.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.221+0000 7f0f9a607640 1 -- 172.21.15.38:0/4261506233 shutdown_connections 2024-04-25T08:39:55.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.221+0000 7f0f9a607640 1 --2- 172.21.15.38:0/4261506233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f800043b0 0x7f0f80004880 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.221+0000 7f0f9a607640 1 -- 172.21.15.38:0/4261506233 >> 172.21.15.38:0/4261506233 conn(0x7f0f9406bed0 msgr2=0x7f0f9406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:55.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.221+0000 7f0f9a607640 1 -- 172.21.15.38:0/4261506233 shutdown_connections 2024-04-25T08:39:55.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.221+0000 7f0f9a607640 1 -- 172.21.15.38:0/4261506233 wait complete. 2024-04-25T08:39:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.222+0000 7f0f9a607640 1 Processor -- start 2024-04-25T08:39:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.223+0000 7f0f9a607640 1 -- start start 2024-04-25T08:39:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.223+0000 7f0f9a607640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f800043b0 0x7f0f94160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.223+0000 7f0f9a607640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f941f34e0 0x7f0f940c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.223+0000 7f0f9a607640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f940b8740 0x7f0f940b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.223+0000 7f0f9a607640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9415de70 con 0x7f0f941f34e0 2024-04-25T08:39:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.223+0000 7f0f9a607640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f940d23c0 con 0x7f0f940b8740 2024-04-25T08:39:55.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.223+0000 7f0f9a607640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9415dff0 con 0x7f0f800043b0 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.223+0000 7f0f937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f941f34e0 0x7f0f940c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.223+0000 7f0f937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f941f34e0 0x7f0f940c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42768/0 (socket says 172.21.15.38:42768) 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.223+0000 7f0f937fe640 1 -- 172.21.15.38:0/4102657550 learned_addr learned my addr 172.21.15.38:0/4102657550 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f98ba4640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f940b8740 0x7f0f940b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f93fff640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f800043b0 0x7f0f94160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f937fe640 1 -- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f800043b0 msgr2=0x7f0f94160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f937fe640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f800043b0 0x7f0f94160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f937fe640 1 -- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f940b8740 msgr2=0x7f0f940b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f937fe640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f940b8740 0x7f0f940b8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f937fe640 1 -- 172.21.15.38:0/4102657550 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f941f44f0 con 0x7f0f941f34e0 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f98ba4640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f940b8740 0x7f0f940b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f93fff640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f800043b0 0x7f0f94160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f937fe640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f941f34e0 0x7f0f940c80e0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f0f84001980 tx=0x7f0f84001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f90ff9640 1 -- 172.21.15.38:0/4102657550 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84037050 con 0x7f0f941f34e0 2024-04-25T08:39:55.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f90ff9640 1 -- 172.21.15.38:0/4102657550 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f84043070 con 0x7f0f941f34e0 2024-04-25T08:39:55.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f90ff9640 1 -- 172.21.15.38:0/4102657550 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84001430 con 0x7f0f941f34e0 2024-04-25T08:39:55.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f9a607640 1 -- 172.21.15.38:0/4102657550 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f941f3c50 con 0x7f0f941f34e0 2024-04-25T08:39:55.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.224+0000 7f0f9a607640 1 -- 172.21.15.38:0/4102657550 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f941f4210 con 0x7f0f941f34e0 2024-04-25T08:39:55.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.225+0000 7f0f9a607640 1 -- 172.21.15.38:0/4102657550 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f940b9100 con 0x7f0f941f34e0 2024-04-25T08:39:55.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.226+0000 7f0f90ff9640 1 -- 172.21.15.38:0/4102657550 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f8403eaa0 con 0x7f0f941f34e0 2024-04-25T08:39:55.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.226+0000 7f0f90ff9640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f7406b500 0x7f0f7406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.226+0000 7f0f90ff9640 1 -- 172.21.15.38:0/4102657550 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0f840aaf00 con 0x7f0f941f34e0 2024-04-25T08:39:55.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.226+0000 7f0f93fff640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f7406b500 0x7f0f7406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.227+0000 7f0f93fff640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f7406b500 0x7f0f7406d9c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0017d0 tx=0x7f0f7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:55.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.230+0000 7f0f90ff9640 1 -- 172.21.15.38:0/4102657550 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f8407a060 con 0x7f0f941f34e0 2024-04-25T08:39:55.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.415+0000 7f0f9a607640 1 -- 172.21.15.38:0/4102657550 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0f94068e80 con 0x7f0f941f34e0 2024-04-25T08:39:55.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.415+0000 7f0f90ff9640 1 -- 172.21.15.38:0/4102657550 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v117) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0f84079960 con 0x7f0f941f34e0 2024-04-25T08:39:55.408 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:55.409 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T08:39:55.409 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:39:55.409 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-04-25T08:39:55.409 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-04-25T08:39:55.409 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:39:55.409 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T08:39:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.417+0000 7f0f9a607640 1 -- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f7406b500 msgr2=0x7f0f7406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.417+0000 7f0f9a607640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f7406b500 0x7f0f7406d9c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0017d0 tx=0x7f0f7c009040 comp rx=0 tx=0).stop 2024-04-25T08:39:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.417+0000 7f0f9a607640 1 -- 172.21.15.38:0/4102657550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f941f34e0 msgr2=0x7f0f940c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.417+0000 7f0f9a607640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f941f34e0 0x7f0f940c80e0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f0f84001980 tx=0x7f0f84001230 comp rx=0 tx=0).stop 2024-04-25T08:39:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.417+0000 7f0f9a607640 1 -- 172.21.15.38:0/4102657550 shutdown_connections 2024-04-25T08:39:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.417+0000 7f0f9a607640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f940b8740 0x7f0f940b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.417+0000 7f0f9a607640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f941f34e0 0x7f0f940c80e0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.417+0000 7f0f9a607640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f7406b500 0x7f0f7406d9c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.417+0000 7f0f9a607640 1 --2- 172.21.15.38:0/4102657550 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f800043b0 0x7f0f94160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.417+0000 7f0f9a607640 1 -- 172.21.15.38:0/4102657550 >> 172.21.15.38:0/4102657550 conn(0x7f0f9406bed0 msgr2=0x7f0f940c6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.417+0000 7f0f9a607640 1 -- 172.21.15.38:0/4102657550 shutdown_connections 2024-04-25T08:39:55.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.417+0000 7f0f9a607640 1 -- 172.21.15.38:0/4102657550 wait complete. 2024-04-25T08:39:55.421 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:39:55.422 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:39:55.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.533+0000 7fc01581f640 1 Processor -- start 2024-04-25T08:39:55.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.533+0000 7fc01581f640 1 -- start start 2024-04-25T08:39:55.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.534+0000 7fc01581f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc01009aa40 0x7fc01015f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.534+0000 7fc01581f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc01015f950 0x7fc0100b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.534+0000 7fc01581f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc010099c80 con 0x7fc01015ba20 2024-04-25T08:39:55.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.534+0000 7fc01581f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01015fdc0 con 0x7fc01015f950 2024-04-25T08:39:55.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.534+0000 7fc01581f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01009aea0 con 0x7fc01009aa40 2024-04-25T08:39:55.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.534+0000 7fc00f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc01015f950 0x7fc0100b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.534+0000 7fc00effd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc01015ba20 0x7fc0100ccc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56462/0 (socket says 172.21.15.38:56462) 2024-04-25T08:39:55.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.534+0000 7fc00effd640 1 -- 172.21.15.38:0/2943876009 learned_addr learned my addr 172.21.15.38:0/2943876009 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:55.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.535+0000 7fc00f7fe640 1 -- 172.21.15.38:0/2943876009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc01009aa40 msgr2=0x7fc01015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:55.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.535+0000 7fc00e7fc640 1 --2- 172.21.15.38:0/2943876009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc01009aa40 0x7fc01015f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.535+0000 7fc00dffb640 1 -- 172.21.15.38:0/2943876009 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717565968 0 0) 0x7fc010099c80 con 0x7fc01015ba20 2024-04-25T08:39:55.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.535+0000 7fc00f7fe640 1 --2- 172.21.15.38:0/2943876009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc01009aa40 0x7fc01015f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.535+0000 7fc00f7fe640 1 -- 172.21.15.38:0/2943876009 >> v1:172.21.15.38:6789/0 conn(0x7fc01015ba20 legacy=0x7fc0100ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.535+0000 7fc00f7fe640 1 -- 172.21.15.38:0/2943876009 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc010099f60 con 0x7fc01015f950 2024-04-25T08:39:55.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.535+0000 7fc00e7fc640 1 --2- 172.21.15.38:0/2943876009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc01009aa40 0x7fc01015f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:55.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.535+0000 7fc00f7fe640 1 --2- 172.21.15.38:0/2943876009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc01015f950 0x7fc0100b0e90 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fc0000097a0 tx=0x7fc00002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe31abb0e562ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:55.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.536+0000 7fc00dffb640 1 -- 172.21.15.38:0/2943876009 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000037070 con 0x7fc01015f950 2024-04-25T08:39:55.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.536+0000 7fc00dffb640 1 -- 172.21.15.38:0/2943876009 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc000032440 con 0x7fc01015f950 2024-04-25T08:39:55.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.536+0000 7fc00dffb640 1 -- 172.21.15.38:0/2943876009 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc000032810 con 0x7fc01015f950 2024-04-25T08:39:55.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.536+0000 7fc01581f640 1 -- 172.21.15.38:0/2943876009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc01015f950 msgr2=0x7fc0100b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.536+0000 7fc01581f640 1 --2- 172.21.15.38:0/2943876009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc01015f950 0x7fc0100b0e90 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fc0000097a0 tx=0x7fc00002a180 comp rx=0 tx=0).stop 2024-04-25T08:39:55.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.536+0000 7fc01581f640 1 -- 172.21.15.38:0/2943876009 shutdown_connections 2024-04-25T08:39:55.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.536+0000 7fc01581f640 1 --2- 172.21.15.38:0/2943876009 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc01015f950 0x7fc0100b0e90 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.536+0000 7fc01581f640 1 --2- 172.21.15.38:0/2943876009 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc01009aa40 0x7fc01015f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.536+0000 7fc01581f640 1 -- 172.21.15.38:0/2943876009 >> 172.21.15.38:0/2943876009 conn(0x7fc0100b3190 msgr2=0x7fc0100b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:55.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.536+0000 7fc01581f640 1 -- 172.21.15.38:0/2943876009 shutdown_connections 2024-04-25T08:39:55.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.536+0000 7fc01581f640 1 -- 172.21.15.38:0/2943876009 wait complete. 2024-04-25T08:39:55.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.537+0000 7fc01581f640 1 Processor -- start 2024-04-25T08:39:55.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.538+0000 7fc01581f640 1 -- start start 2024-04-25T08:39:55.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.538+0000 7fc01581f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc01009aa40 0x7fc0100c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.538+0000 7fc01581f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc01015ba20 0x7fc010151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.538+0000 7fc01581f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc01015f950 0x7fc0101591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.538+0000 7fc01581f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100d0430 con 0x7fc01015ba20 2024-04-25T08:39:55.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.538+0000 7fc01581f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01009a520 con 0x7fc01015f950 2024-04-25T08:39:55.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.538+0000 7fc01581f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0100d05b0 con 0x7fc01009aa40 2024-04-25T08:39:55.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.538+0000 7fc00e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc01015ba20 0x7fc010151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.538+0000 7fc00e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc01015ba20 0x7fc010151e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42786/0 (socket says 172.21.15.38:42786) 2024-04-25T08:39:55.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.538+0000 7fc00e7fc640 1 -- 172.21.15.38:0/3451830117 learned_addr learned my addr 172.21.15.38:0/3451830117 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:55.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc00effd640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc01009aa40 0x7fc0100c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc00f7fe640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc01015f950 0x7fc0101591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc00e7fc640 1 -- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc01009aa40 msgr2=0x7fc0100c51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc00e7fc640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc01009aa40 0x7fc0100c51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc00e7fc640 1 -- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc01015f950 msgr2=0x7fc0101591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:55.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc00e7fc640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc01015f950 0x7fc0101591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:55.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc00e7fc640 1 -- 172.21.15.38:0/3451830117 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01015d620 con 0x7fc01015ba20 2024-04-25T08:39:55.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc00e7fc640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc01015ba20 0x7fc010151e40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fbff800c820 tx=0x7fbff800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:55.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc007fff640 1 -- 172.21.15.38:0/3451830117 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8015070 con 0x7fc01015ba20 2024-04-25T08:39:55.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc007fff640 1 -- 172.21.15.38:0/3451830117 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff8007a30 con 0x7fc01015ba20 2024-04-25T08:39:55.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc007fff640 1 -- 172.21.15.38:0/3451830117 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8007de0 con 0x7fc01015ba20 2024-04-25T08:39:55.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc01581f640 1 -- 172.21.15.38:0/3451830117 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc010158400 con 0x7fc01015ba20 2024-04-25T08:39:55.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.539+0000 7fc01581f640 1 -- 172.21.15.38:0/3451830117 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0101589c0 con 0x7fc01015ba20 2024-04-25T08:39:55.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.540+0000 7fc01581f640 1 -- 172.21.15.38:0/3451830117 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc010068710 con 0x7fc01015ba20 2024-04-25T08:39:55.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.541+0000 7fc007fff640 1 -- 172.21.15.38:0/3451830117 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbff8018070 con 0x7fc01015ba20 2024-04-25T08:39:55.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.541+0000 7fc007fff640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbff006b4d0 0x7fbff006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:55.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.541+0000 7fc007fff640 1 -- 172.21.15.38:0/3451830117 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbff8089ed0 con 0x7fc01015ba20 2024-04-25T08:39:55.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.541+0000 7fc00effd640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbff006b4d0 0x7fbff006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:55.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.542+0000 7fc00effd640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbff006b4d0 0x7fbff006d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc0080012e0 tx=0x7fc008001630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:55.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.545+0000 7fc007fff640 1 -- 172.21.15.38:0/3451830117 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc010068710 con 0x7fc01015ba20 2024-04-25T08:39:55.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:55.728+0000 7fc01581f640 1 -- 172.21.15.38:0/3451830117 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc010068e80 con 0x7fc01015ba20 2024-04-25T08:39:56.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.095+0000 7fc007fff640 1 -- 172.21.15.38:0/3451830117 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v118) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbff80588b0 con 0x7fc01015ba20 2024-04-25T08:39:56.089 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-04-25T08:39:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.099+0000 7fc01581f640 1 -- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbff006b4d0 msgr2=0x7fbff006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.099+0000 7fc01581f640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbff006b4d0 0x7fbff006d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc0080012e0 tx=0x7fc008001630 comp rx=0 tx=0).stop 2024-04-25T08:39:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.099+0000 7fc01581f640 1 -- 172.21.15.38:0/3451830117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc01015ba20 msgr2=0x7fc010151e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.099+0000 7fc01581f640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc01015ba20 0x7fc010151e40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fbff800c820 tx=0x7fbff800cce0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.099+0000 7fc01581f640 1 -- 172.21.15.38:0/3451830117 shutdown_connections 2024-04-25T08:39:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.099+0000 7fc01581f640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc01015f950 0x7fc0101591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.099+0000 7fc01581f640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc01015ba20 0x7fc010151e40 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.099+0000 7fc01581f640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbff006b4d0 0x7fbff006d990 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.099+0000 7fc01581f640 1 --2- 172.21.15.38:0/3451830117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc01009aa40 0x7fc0100c51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.099+0000 7fc01581f640 1 -- 172.21.15.38:0/3451830117 >> 172.21.15.38:0/3451830117 conn(0x7fc0100b3190 msgr2=0x7fc010153c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.099+0000 7fc01581f640 1 -- 172.21.15.38:0/3451830117 shutdown_connections 2024-04-25T08:39:56.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.099+0000 7fc01581f640 1 -- 172.21.15.38:0/3451830117 wait complete. 2024-04-25T08:39:56.109 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:39:56.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.227+0000 7ff2bac76640 1 Processor -- start 2024-04-25T08:39:56.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.227+0000 7ff2bac76640 1 -- start start 2024-04-25T08:39:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.228+0000 7ff2bac76640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff2b4166610 0x7ff2b4168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.228+0000 7ff2bac76640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b40d2060 con 0x7ff2b40d33d0 2024-04-25T08:39:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.228+0000 7ff2bac76640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b40d2330 con 0x7ff2b40d0320 2024-04-25T08:39:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.228+0000 7ff2bac76640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b40d2610 con 0x7ff2b4166610 2024-04-25T08:39:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.228+0000 7ff2b9213640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff2b4166610 0x7ff2b4168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.228+0000 7ff2b9213640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff2b4166610 0x7ff2b4168a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42798/0 (socket says 172.21.15.38:42798) 2024-04-25T08:39:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.228+0000 7ff2b9213640 1 -- 172.21.15.38:0/2435343432 learned_addr learned my addr 172.21.15.38:0/2435343432 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.228+0000 7ff2b3fff640 1 --1- 172.21.15.38:0/2435343432 >> v1:172.21.15.38:6789/0 conn(0x7ff2b40d33d0 0x7ff2b40d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56474/0 (socket says 172.21.15.38:56474) 2024-04-25T08:39:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.228+0000 7ff2b9213640 1 -- 172.21.15.38:0/2435343432 >> v1:172.21.15.107:6789/0 conn(0x7ff2b40d0320 legacy=0x7ff2b40d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.228+0000 7ff2b9213640 1 -- 172.21.15.38:0/2435343432 >> v1:172.21.15.38:6789/0 conn(0x7ff2b40d33d0 legacy=0x7ff2b40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.229+0000 7ff2b9213640 1 -- 172.21.15.38:0/2435343432 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b40d28f0 con 0x7ff2b4166610 2024-04-25T08:39:56.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.229+0000 7ff2b37fe640 1 -- 172.21.15.38:0/2435343432 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503445857 0 0) 0x7ff2b40d2060 con 0x7ff2b40d33d0 2024-04-25T08:39:56.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.229+0000 7ff2b9213640 1 --2- 172.21.15.38:0/2435343432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b4166610 0x7ff2b4168a00 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009e50 tx=0x7ff2ac02a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35df55015b0f117e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:56.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.229+0000 7ff2b37fe640 1 -- 172.21.15.38:0/2435343432 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac037070 con 0x7ff2b4166610 2024-04-25T08:39:56.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.229+0000 7ff2b37fe640 1 -- 172.21.15.38:0/2435343432 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ac032630 con 0x7ff2b4166610 2024-04-25T08:39:56.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.229+0000 7ff2b37fe640 1 -- 172.21.15.38:0/2435343432 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac02e050 con 0x7ff2b4166610 2024-04-25T08:39:56.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.230+0000 7ff2bac76640 1 -- 172.21.15.38:0/2435343432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b4166610 msgr2=0x7ff2b4168a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.230+0000 7ff2bac76640 1 --2- 172.21.15.38:0/2435343432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b4166610 0x7ff2b4168a00 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac009e50 tx=0x7ff2ac02a840 comp rx=0 tx=0).stop 2024-04-25T08:39:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.230+0000 7ff2bac76640 1 -- 172.21.15.38:0/2435343432 shutdown_connections 2024-04-25T08:39:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.230+0000 7ff2bac76640 1 --2- 172.21.15.38:0/2435343432 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b4166610 0x7ff2b4168a00 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.230+0000 7ff2bac76640 1 -- 172.21.15.38:0/2435343432 >> 172.21.15.38:0/2435343432 conn(0x7ff2b40668f0 msgr2=0x7ff2b4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.230+0000 7ff2bac76640 1 -- 172.21.15.38:0/2435343432 shutdown_connections 2024-04-25T08:39:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.230+0000 7ff2bac76640 1 -- 172.21.15.38:0/2435343432 wait complete. 2024-04-25T08:39:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.231+0000 7ff2bac76640 1 Processor -- start 2024-04-25T08:39:56.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.231+0000 7ff2bac76640 1 -- start start 2024-04-25T08:39:56.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2bac76640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b40d0320 0x7ff2b40cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:56.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2bac76640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b40d33d0 0x7ff2b4158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:56.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2bac76640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2b4166610 0x7ff2b414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:56.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2bac76640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b4161210 con 0x7ff2b40d0320 2024-04-25T08:39:56.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2bac76640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40d2eb0 con 0x7ff2b40d33d0 2024-04-25T08:39:56.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2bac76640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b4161390 con 0x7ff2b4166610 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2b8a12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b40d0320 0x7ff2b40cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2b8a12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b40d0320 0x7ff2b40cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42808/0 (socket says 172.21.15.38:42808) 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2b8a12640 1 -- 172.21.15.38:0/1194154198 learned_addr learned my addr 172.21.15.38:0/1194154198 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2b3fff640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b40d33d0 0x7ff2b4158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2b9213640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2b4166610 0x7ff2b414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2b8a12640 1 -- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2b4166610 msgr2=0x7ff2b414e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2b8a12640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2b4166610 0x7ff2b414e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2b8a12640 1 -- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b40d33d0 msgr2=0x7ff2b4158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2b8a12640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b40d33d0 0x7ff2b4158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2b8a12640 1 -- 172.21.15.38:0/1194154198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b40b3ef0 con 0x7ff2b40d0320 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2b3fff640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b40d33d0 0x7ff2b4158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.232+0000 7ff2b9213640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2b4166610 0x7ff2b414e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:39:56.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.233+0000 7ff2b8a12640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b40d0320 0x7ff2b40cb3f0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400c450 tx=0x7ff2a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:56.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.233+0000 7ff2b17fa640 1 -- 172.21.15.38:0/1194154198 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40052d0 con 0x7ff2b40d0320 2024-04-25T08:39:56.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.233+0000 7ff2b17fa640 1 -- 172.21.15.38:0/1194154198 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a4005470 con 0x7ff2b40d0320 2024-04-25T08:39:56.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.233+0000 7ff2bac76640 1 -- 172.21.15.38:0/1194154198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b415ee10 con 0x7ff2b40d0320 2024-04-25T08:39:56.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.233+0000 7ff2b17fa640 1 -- 172.21.15.38:0/1194154198 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4012870 con 0x7ff2b40d0320 2024-04-25T08:39:56.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.233+0000 7ff2bac76640 1 -- 172.21.15.38:0/1194154198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b40cc9c0 con 0x7ff2b40d0320 2024-04-25T08:39:56.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.234+0000 7ff2bac76640 1 -- 172.21.15.38:0/1194154198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b4068e80 con 0x7ff2b40d0320 2024-04-25T08:39:56.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.234+0000 7ff2b17fa640 1 -- 172.21.15.38:0/1194154198 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2a4012a10 con 0x7ff2b40d0320 2024-04-25T08:39:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.235+0000 7ff2b17fa640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff2a006b4d0 0x7ff2a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.235+0000 7ff2b17fa640 1 -- 172.21.15.38:0/1194154198 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5619+0+0 (secure 0 0 0) 0x7ff2a4089e00 con 0x7ff2b40d0320 2024-04-25T08:39:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.235+0000 7ff2b3fff640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff2a006b4d0 0x7ff2a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:56.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.236+0000 7ff2b3fff640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff2a006b4d0 0x7ff2a006d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff2b415f600 tx=0x7ff2a8007c70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:56.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.240+0000 7ff2b17fa640 1 -- 172.21.15.38:0/1194154198 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2a4058cc0 con 0x7ff2b40d0320 2024-04-25T08:39:56.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:56.419+0000 7ff2bac76640 1 -- 172.21.15.38:0/1194154198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff2b40b3050 con 0x7ff2b40d0320 2024-04-25T08:39:57.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.098+0000 7ff2b17fa640 1 -- 172.21.15.38:0/1194154198 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v119) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff2a40585c0 con 0x7ff2b40d0320 2024-04-25T08:39:57.091 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-04-25T08:39:57.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.101+0000 7ff2bac76640 1 -- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff2a006b4d0 msgr2=0x7ff2a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.101+0000 7ff2bac76640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff2a006b4d0 0x7ff2a006d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff2b415f600 tx=0x7ff2a8007c70 comp rx=0 tx=0).stop 2024-04-25T08:39:57.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.101+0000 7ff2bac76640 1 -- 172.21.15.38:0/1194154198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b40d0320 msgr2=0x7ff2b40cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.101+0000 7ff2bac76640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b40d0320 0x7ff2b40cb3f0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400c450 tx=0x7ff2a400c910 comp rx=0 tx=0).stop 2024-04-25T08:39:57.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.101+0000 7ff2bac76640 1 -- 172.21.15.38:0/1194154198 shutdown_connections 2024-04-25T08:39:57.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.101+0000 7ff2bac76640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff2a006b4d0 0x7ff2a006d990 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.101+0000 7ff2bac76640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2b4166610 0x7ff2b414e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.101+0000 7ff2bac76640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b40d33d0 0x7ff2b4158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.101+0000 7ff2bac76640 1 --2- 172.21.15.38:0/1194154198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2b40d0320 0x7ff2b40cb3f0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.101+0000 7ff2bac76640 1 -- 172.21.15.38:0/1194154198 >> 172.21.15.38:0/1194154198 conn(0x7ff2b40668f0 msgr2=0x7ff2b415fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:57.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.101+0000 7ff2bac76640 1 -- 172.21.15.38:0/1194154198 shutdown_connections 2024-04-25T08:39:57.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.101+0000 7ff2bac76640 1 -- 172.21.15.38:0/1194154198 wait complete. 2024-04-25T08:39:57.126 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:39:57.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.242+0000 7fe8286b3640 1 Processor -- start 2024-04-25T08:39:57.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.242+0000 7fe8286b3640 1 -- start start 2024-04-25T08:39:57.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.243+0000 7fe8286b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820166610 0x7fe820168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.243+0000 7fe8286b3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8200d2060 con 0x7fe8200d0320 2024-04-25T08:39:57.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.243+0000 7fe8286b3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8200d2330 con 0x7fe8200d33d0 2024-04-25T08:39:57.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.243+0000 7fe8286b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200d2610 con 0x7fe820166610 2024-04-25T08:39:57.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.243+0000 7fe82644f640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe8200d0320 0x7fe8200d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56484/0 (socket says 172.21.15.38:56484) 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.243+0000 7fe82644f640 1 -- 172.21.15.38:0/2127583308 learned_addr learned my addr 172.21.15.38:0/2127583308 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.243+0000 7fe826c50640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820166610 0x7fe820168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811295737 0 0) 0x7fe8200d2060 con 0x7fe8200d0320 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe80c003520 con 0x7fe8200d0320 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757888681 0 0) 0x7fe8200d2330 con 0x7fe8200d33d0 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8200d2060 con 0x7fe8200d33d0 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926791657 0 0) 0x7fe80c003520 con 0x7fe8200d0320 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820166610 msgr2=0x7fe820168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820166610 0x7fe820168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 >> v1:172.21.15.107:6789/0 conn(0x7fe8200d33d0 legacy=0x7fe8200d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8200d28f0 con 0x7fe8200d0320 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fe81c001690 con 0x7fe8200d0320 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe826c50640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820166610 0x7fe820168a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:39:57.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 >> v1:172.21.15.38:6789/0 conn(0x7fe8200d0320 legacy=0x7fe8200d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820166610 0x7fe80c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0040b0 0x7fe80c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe80c004d40 0x7fe80c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200d2610 con 0x7fe80c0040b0 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c00ba70 con 0x7fe80c004d40 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c00bc70 con 0x7fe820166610 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe825c4e640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0040b0 0x7fe80c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe82644f640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820166610 0x7fe80c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe826c50640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe80c004d40 0x7fe80c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe825c4e640 1 -- 172.21.15.38:0/2127583308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820166610 msgr2=0x7fe80c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe825c4e640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820166610 0x7fe80c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe825c4e640 1 -- 172.21.15.38:0/2127583308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe80c004d40 msgr2=0x7fe80c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe825c4e640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe80c004d40 0x7fe80c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.244+0000 7fe825c4e640 1 -- 172.21.15.38:0/2127583308 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c003520 con 0x7fe80c0040b0 2024-04-25T08:39:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.245+0000 7fe825c4e640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0040b0 0x7fe80c004800 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe8100018d0 tx=0x7fe80c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f341b87cf4becd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.245+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8100014f0 con 0x7fe80c0040b0 2024-04-25T08:39:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.245+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8100052d0 con 0x7fe80c0040b0 2024-04-25T08:39:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.245+0000 7fe82544d640 1 -- 172.21.15.38:0/2127583308 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81002aca0 con 0x7fe80c0040b0 2024-04-25T08:39:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.246+0000 7fe8286b3640 1 -- 172.21.15.38:0/2127583308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0040b0 msgr2=0x7fe80c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.246+0000 7fe8286b3640 1 --2- 172.21.15.38:0/2127583308 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe80c0040b0 0x7fe80c004800 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fe8100018d0 tx=0x7fe80c005830 comp rx=0 tx=0).stop 2024-04-25T08:39:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.246+0000 7fe826c50640 1 -- 172.21.15.38:0/2127583308 reap_dead start 2024-04-25T08:39:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.246+0000 7fe8286b3640 1 -- 172.21.15.38:0/2127583308 shutdown_connections 2024-04-25T08:39:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.246+0000 7fe8286b3640 1 -- 172.21.15.38:0/2127583308 >> 172.21.15.38:0/2127583308 conn(0x7fe8200668f0 msgr2=0x7fe820066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.246+0000 7fe8286b3640 1 -- 172.21.15.38:0/2127583308 shutdown_connections 2024-04-25T08:39:57.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.246+0000 7fe8286b3640 1 -- 172.21.15.38:0/2127583308 wait complete. 2024-04-25T08:39:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.247+0000 7fe8286b3640 1 Processor -- start 2024-04-25T08:39:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.247+0000 7fe8286b3640 1 -- start start 2024-04-25T08:39:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe8286b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe80c0040b0 0x7fe820145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe8286b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8201462e0 0x7fe820159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe8286b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820144790 0x7fe82015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe8286b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe820161210 con 0x7fe8201462e0 2024-04-25T08:39:57.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe8286b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8200d2eb0 con 0x7fe80c0040b0 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe8286b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe820161390 con 0x7fe820144790 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe825c4e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8201462e0 0x7fe820159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe825c4e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8201462e0 0x7fe820159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42816/0 (socket says 172.21.15.38:42816) 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe825c4e640 1 -- 172.21.15.38:0/1139416614 learned_addr learned my addr 172.21.15.38:0/1139416614 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe826c50640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820144790 0x7fe82015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe82644f640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe80c0040b0 0x7fe820145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe825c4e640 1 -- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820144790 msgr2=0x7fe82015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe825c4e640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820144790 0x7fe82015a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe825c4e640 1 -- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe80c0040b0 msgr2=0x7fe820145da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe825c4e640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe80c0040b0 0x7fe820145da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe825c4e640 1 -- 172.21.15.38:0/1139416614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820159080 con 0x7fe8201462e0 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.248+0000 7fe82644f640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe80c0040b0 0x7fe820145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.249+0000 7fe825c4e640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8201462e0 0x7fe820159b50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe8100019f0 tx=0x7fe8100012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.249+0000 7fe816ffd640 1 -- 172.21.15.38:0/1139416614 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81003f020 con 0x7fe8201462e0 2024-04-25T08:39:57.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.249+0000 7fe816ffd640 1 -- 172.21.15.38:0/1139416614 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe810006e50 con 0x7fe8201462e0 2024-04-25T08:39:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.249+0000 7fe8286b3640 1 -- 172.21.15.38:0/1139416614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe820144d00 con 0x7fe8201462e0 2024-04-25T08:39:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.249+0000 7fe816ffd640 1 -- 172.21.15.38:0/1139416614 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810037040 con 0x7fe8201462e0 2024-04-25T08:39:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.249+0000 7fe8286b3640 1 -- 172.21.15.38:0/1139416614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8201452c0 con 0x7fe8201462e0 2024-04-25T08:39:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.250+0000 7fe8286b3640 1 -- 172.21.15.38:0/1139416614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe820068e80 con 0x7fe8201462e0 2024-04-25T08:39:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.251+0000 7fe816ffd640 1 -- 172.21.15.38:0/1139416614 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe810042020 con 0x7fe8201462e0 2024-04-25T08:39:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.251+0000 7fe816ffd640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe80406b500 0x7fe80406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.251+0000 7fe816ffd640 1 -- 172.21.15.38:0/1139416614 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fe8100aa2e0 con 0x7fe8201462e0 2024-04-25T08:39:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.251+0000 7fe82644f640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe80406b500 0x7fe80406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:57.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.252+0000 7fe82644f640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe80406b500 0x7fe80406d9c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe8201f6d50 tx=0x7fe81c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:57.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.255+0000 7fe816ffd640 1 -- 172.21.15.38:0/1139416614 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe810049020 con 0x7fe8201462e0 2024-04-25T08:39:57.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:57.474+0000 7fe8286b3640 1 -- 172.21.15.38:0/1139416614 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe8200b8630 con 0x7fe8201462e0 2024-04-25T08:39:58.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.121+0000 7fe816ffd640 1 -- 172.21.15.38:0/1139416614 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v68) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe810079030 con 0x7fe8201462e0 2024-04-25T08:39:58.114 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T08:39:58.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.136+0000 7fe8286b3640 1 -- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe80406b500 msgr2=0x7fe80406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.136+0000 7fe8286b3640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe80406b500 0x7fe80406d9c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe8201f6d50 tx=0x7fe81c009040 comp rx=0 tx=0).stop 2024-04-25T08:39:58.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.137+0000 7fe8286b3640 1 -- 172.21.15.38:0/1139416614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8201462e0 msgr2=0x7fe820159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.137+0000 7fe8286b3640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8201462e0 0x7fe820159b50 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fe8100019f0 tx=0x7fe8100012d0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.137+0000 7fe8286b3640 1 -- 172.21.15.38:0/1139416614 shutdown_connections 2024-04-25T08:39:58.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.137+0000 7fe8286b3640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe80406b500 0x7fe80406d9c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.137+0000 7fe8286b3640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe820144790 0x7fe82015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.137+0000 7fe8286b3640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8201462e0 0x7fe820159b50 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.138+0000 7fe8286b3640 1 --2- 172.21.15.38:0/1139416614 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe80c0040b0 0x7fe820145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.138+0000 7fe8286b3640 1 -- 172.21.15.38:0/1139416614 >> 172.21.15.38:0/1139416614 conn(0x7fe8200668f0 msgr2=0x7fe8200cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:58.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.138+0000 7fe8286b3640 1 -- 172.21.15.38:0/1139416614 shutdown_connections 2024-04-25T08:39:58.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.139+0000 7fe8286b3640 1 -- 172.21.15.38:0/1139416614 wait complete. 2024-04-25T08:39:58.150 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.262+0000 7fd1de1cf640 1 Processor -- start 2024-04-25T08:39:58.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.262+0000 7fd1de1cf640 1 -- start start 2024-04-25T08:39:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1de1cf640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1d815a8f0 0x7fd1d80cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1de1cf640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd1d80cb6c0 0x7fd1d80d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1de1cf640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd1d80d0780 0x7fd1d80cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1de1cf640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d80cbd20 con 0x7fd1d80cb6c0 2024-04-25T08:39:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1de1cf640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d80cc000 con 0x7fd1d80d0780 2024-04-25T08:39:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1de1cf640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d80cc2e0 con 0x7fd1d815a8f0 2024-04-25T08:39:58.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1d6ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd1d80cb6c0 0x7fd1d80d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1d6ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd1d80cb6c0 0x7fd1d80d0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42828/0 (socket says 172.21.15.38:42828) 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1d6ffd640 1 -- 172.21.15.38:0/4065467734 learned_addr learned my addr 172.21.15.38:0/4065467734 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1d7fff640 1 --2- 172.21.15.38:0/4065467734 >> v2:172.21.15.107:3300/0 conn(0x7fd1d80d0780 0x7fd1d80cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1d77fe640 1 --2- 172.21.15.38:0/4065467734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1d815a8f0 0x7fd1d80cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1d6ffd640 1 -- 172.21.15.38:0/4065467734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1d815a8f0 msgr2=0x7fd1d80cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1d6ffd640 1 --2- 172.21.15.38:0/4065467734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1d815a8f0 0x7fd1d80cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1d6ffd640 1 -- 172.21.15.38:0/4065467734 >> v2:172.21.15.107:3300/0 conn(0x7fd1d80d0780 msgr2=0x7fd1d80cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1d6ffd640 1 --2- 172.21.15.38:0/4065467734 >> v2:172.21.15.107:3300/0 conn(0x7fd1d80d0780 0x7fd1d80cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.263+0000 7fd1d6ffd640 1 -- 172.21.15.38:0/4065467734 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d80cc5c0 con 0x7fd1d80cb6c0 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.264+0000 7fd1d6ffd640 1 --2- 172.21.15.38:0/4065467734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1d80cb6c0 0x7fd1d80d0240 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80d13c0 tx=0x7fd1cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500c804d8946179c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.264+0000 7fd1d67fc640 1 -- 172.21.15.38:0/4065467734 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc037070 con 0x7fd1d80cb6c0 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.264+0000 7fd1d67fc640 1 -- 172.21.15.38:0/4065467734 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc032440 con 0x7fd1d80cb6c0 2024-04-25T08:39:58.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.264+0000 7fd1d67fc640 1 -- 172.21.15.38:0/4065467734 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc032810 con 0x7fd1d80cb6c0 2024-04-25T08:39:58.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.265+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4065467734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1d80cb6c0 msgr2=0x7fd1d80d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.265+0000 7fd1de1cf640 1 --2- 172.21.15.38:0/4065467734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1d80cb6c0 0x7fd1d80d0240 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fd1d80d13c0 tx=0x7fd1cc02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.265+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4065467734 shutdown_connections 2024-04-25T08:39:58.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.265+0000 7fd1de1cf640 1 --2- 172.21.15.38:0/4065467734 >> v2:172.21.15.107:3300/0 conn(0x7fd1d80d0780 0x7fd1d80cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.265+0000 7fd1de1cf640 1 --2- 172.21.15.38:0/4065467734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1d80cb6c0 0x7fd1d80d0240 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.265+0000 7fd1de1cf640 1 --2- 172.21.15.38:0/4065467734 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1d815a8f0 0x7fd1d80cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.265+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4065467734 >> 172.21.15.38:0/4065467734 conn(0x7fd1d8072610 msgr2=0x7fd1d8099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.265+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4065467734 shutdown_connections 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.265+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4065467734 wait complete. 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.266+0000 7fd1de1cf640 1 Processor -- start 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.266+0000 7fd1de1cf640 1 -- start start 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1de1cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1d80cb6c0 0x7fd1d80ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1de1cf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1d80d0780 0x7fd1d81f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1de1cf640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1d815a8f0 0x7fd1d81f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1de1cf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d80ce470 con 0x7fd1d80d0780 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1de1cf640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d80d0c10 con 0x7fd1d80cb6c0 2024-04-25T08:39:58.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1de1cf640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d815acd0 con 0x7fd1d815a8f0 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1d6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1d80d0780 0x7fd1d81f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1d6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1d80d0780 0x7fd1d81f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42832/0 (socket says 172.21.15.38:42832) 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1d6ffd640 1 -- 172.21.15.38:0/4247162299 learned_addr learned my addr 172.21.15.38:0/4247162299 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1d77fe640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1d80cb6c0 0x7fd1d80ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1d7fff640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1d815a8f0 0x7fd1d81f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1d6ffd640 1 -- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1d815a8f0 msgr2=0x7fd1d81f6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1d6ffd640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1d815a8f0 0x7fd1d81f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1d6ffd640 1 -- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1d80cb6c0 msgr2=0x7fd1d80ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1d6ffd640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1d80cb6c0 0x7fd1d80ca4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1d6ffd640 1 -- 172.21.15.38:0/4247162299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d81f7860 con 0x7fd1d80d0780 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.267+0000 7fd1d77fe640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1d80cb6c0 0x7fd1d80ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.268+0000 7fd1d6ffd640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1d80d0780 0x7fd1d81f4310 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc000c00 tx=0x7fd1cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.268+0000 7fd1bffff640 1 -- 172.21.15.38:0/4247162299 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc041070 con 0x7fd1d80d0780 2024-04-25T08:39:58.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.268+0000 7fd1bffff640 1 -- 172.21.15.38:0/4247162299 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1cc032e50 con 0x7fd1d80d0780 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.268+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4247162299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d81f7b40 con 0x7fd1d80d0780 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.268+0000 7fd1bffff640 1 -- 172.21.15.38:0/4247162299 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc037040 con 0x7fd1d80d0780 2024-04-25T08:39:58.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.268+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4247162299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d81f3df0 con 0x7fd1d80d0780 2024-04-25T08:39:58.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.269+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4247162299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d81f3900 con 0x7fd1d80d0780 2024-04-25T08:39:58.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.269+0000 7fd1bffff640 1 -- 172.21.15.38:0/4247162299 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1cc044070 con 0x7fd1d80d0780 2024-04-25T08:39:58.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.270+0000 7fd1bffff640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1c406b4d0 0x7fd1c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.270+0000 7fd1bffff640 1 -- 172.21.15.38:0/4247162299 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd1cc0ab000 con 0x7fd1d80d0780 2024-04-25T08:39:58.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.270+0000 7fd1d77fe640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1c406b4d0 0x7fd1c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.270+0000 7fd1d77fe640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1c406b4d0 0x7fd1c406d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009ac0 tx=0x7fd1c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:58.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.274+0000 7fd1bffff640 1 -- 172.21.15.38:0/4247162299 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1cc079e10 con 0x7fd1d80d0780 2024-04-25T08:39:58.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.454+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4247162299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1d80d15a0 con 0x7fd1d80d0780 2024-04-25T08:39:58.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.455+0000 7fd1bffff640 1 -- 172.21.15.38:0/4247162299 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16953 (secure 0 0 0) 0x7fd1cc079710 con 0x7fd1d80d0780 2024-04-25T08:39:58.448 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:58.449 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":120,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:57.475604+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:55.759766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:56.451485+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:58.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.457+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1c406b4d0 msgr2=0x7fd1c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.457+0000 7fd1de1cf640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1c406b4d0 0x7fd1c406d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009ac0 tx=0x7fd1c8008040 comp rx=0 tx=0).stop 2024-04-25T08:39:58.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.457+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4247162299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1d80d0780 msgr2=0x7fd1d81f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.457+0000 7fd1de1cf640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1d80d0780 0x7fd1d81f4310 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc000c00 tx=0x7fd1cc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.457+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4247162299 shutdown_connections 2024-04-25T08:39:58.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.457+0000 7fd1de1cf640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1c406b4d0 0x7fd1c406d990 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.457+0000 7fd1de1cf640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd1d815a8f0 0x7fd1d81f6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.457+0000 7fd1de1cf640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd1d80d0780 0x7fd1d81f4310 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.457+0000 7fd1de1cf640 1 --2- 172.21.15.38:0/4247162299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1d80cb6c0 0x7fd1d80ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.457+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4247162299 >> 172.21.15.38:0/4247162299 conn(0x7fd1d8072610 msgr2=0x7fd1d806ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:58.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.457+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4247162299 shutdown_connections 2024-04-25T08:39:58.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.457+0000 7fd1de1cf640 1 -- 172.21.15.38:0/4247162299 wait complete. 2024-04-25T08:39:58.462 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:39:58.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.569+0000 7f97ea553640 1 Processor -- start 2024-04-25T08:39:58.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.569+0000 7f97ea553640 1 -- start start 2024-04-25T08:39:58.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97ea553640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97e40b4b50 0x7f97e40ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97ea553640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f97e4144b70 0x7f97e40ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97ea553640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e40b50c0 con 0x7f97e4144760 2024-04-25T08:39:58.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97ea553640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e415f380 con 0x7f97e4144b70 2024-04-25T08:39:58.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97ea553640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e415f660 con 0x7f97e40b4b50 2024-04-25T08:39:58.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97e8af0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f97e4144b70 0x7f97e40ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97e3fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f97e4144760 0x7f97e415faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56496/0 (socket says 172.21.15.38:56496) 2024-04-25T08:39:58.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97e3fff640 1 -- 172.21.15.38:0/2390844326 learned_addr learned my addr 172.21.15.38:0/2390844326 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:58.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97e37fe640 1 --2- 172.21.15.38:0/2390844326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97e40b4b50 0x7f97e40ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97e8af0640 1 -- 172.21.15.38:0/2390844326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97e40b4b50 msgr2=0x7f97e40ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:58.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97e8af0640 1 --2- 172.21.15.38:0/2390844326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97e40b4b50 0x7f97e40ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97e8af0640 1 -- 172.21.15.38:0/2390844326 >> v1:172.21.15.38:6789/0 conn(0x7f97e4144760 legacy=0x7f97e415faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.570+0000 7f97e2ffd640 1 -- 172.21.15.38:0/2390844326 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984932832 0 0) 0x7f97e40b50c0 con 0x7f97e4144760 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.571+0000 7f97e8af0640 1 -- 172.21.15.38:0/2390844326 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e40b53a0 con 0x7f97e4144b70 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.571+0000 7f97e8af0640 1 --2- 172.21.15.38:0/2390844326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97e4144b70 0x7f97e40ce8a0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009d90 tx=0x7f97dc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bdf6f32c4a6d50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.571+0000 7f97e2ffd640 1 -- 172.21.15.38:0/2390844326 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc031600 con 0x7f97e4144b70 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.571+0000 7f97e2ffd640 1 -- 172.21.15.38:0/2390844326 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97dc0317a0 con 0x7f97e4144b70 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.571+0000 7f97e2ffd640 1 -- 172.21.15.38:0/2390844326 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc031b70 con 0x7f97e4144b70 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.572+0000 7f97ea553640 1 -- 172.21.15.38:0/2390844326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97e4144b70 msgr2=0x7f97e40ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.572+0000 7f97ea553640 1 --2- 172.21.15.38:0/2390844326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97e4144b70 0x7f97e40ce8a0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f97dc009d90 tx=0x7f97dc02a5d0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.572+0000 7f97ea553640 1 -- 172.21.15.38:0/2390844326 shutdown_connections 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.572+0000 7f97ea553640 1 --2- 172.21.15.38:0/2390844326 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97e4144b70 0x7f97e40ce8a0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.572+0000 7f97ea553640 1 --2- 172.21.15.38:0/2390844326 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97e40b4b50 0x7f97e40ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.572+0000 7f97ea553640 1 -- 172.21.15.38:0/2390844326 >> 172.21.15.38:0/2390844326 conn(0x7f97e406b210 msgr2=0x7f97e406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.572+0000 7f97ea553640 1 -- 172.21.15.38:0/2390844326 shutdown_connections 2024-04-25T08:39:58.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.572+0000 7f97ea553640 1 -- 172.21.15.38:0/2390844326 wait complete. 2024-04-25T08:39:58.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.573+0000 7f97ea553640 1 Processor -- start 2024-04-25T08:39:58.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.573+0000 7f97ea553640 1 -- start start 2024-04-25T08:39:58.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.574+0000 7f97ea553640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97e40b4b50 0x7f97e41f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.574+0000 7f97ea553640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97e4144b70 0x7f97e40b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.574+0000 7f97ea553640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97e41438e0 0x7f97e40b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.574+0000 7f97ea553640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e4145280 con 0x7f97e40b4b50 2024-04-25T08:39:58.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.574+0000 7f97ea553640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e4160200 con 0x7f97e4144b70 2024-04-25T08:39:58.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.574+0000 7f97ea553640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e4145400 con 0x7f97e41438e0 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.574+0000 7f97e3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97e40b4b50 0x7f97e41f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.574+0000 7f97e3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97e40b4b50 0x7f97e41f72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42848/0 (socket says 172.21.15.38:42848) 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.574+0000 7f97e3fff640 1 -- 172.21.15.38:0/2096287225 learned_addr learned my addr 172.21.15.38:0/2096287225 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.574+0000 7f97e37fe640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97e4144b70 0x7f97e40b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.574+0000 7f97e8af0640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97e41438e0 0x7f97e40b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.575+0000 7f97e3fff640 1 -- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97e41438e0 msgr2=0x7f97e40b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.575+0000 7f97e3fff640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97e41438e0 0x7f97e40b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.575+0000 7f97e3fff640 1 -- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97e4144b70 msgr2=0x7f97e40b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.575+0000 7f97e3fff640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97e4144b70 0x7f97e40b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.575+0000 7f97e3fff640 1 -- 172.21.15.38:0/2096287225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e4143dc0 con 0x7f97e40b4b50 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.575+0000 7f97e3fff640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97e40b4b50 0x7f97e41f72a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f97d800c430 tx=0x7f97d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.575+0000 7f97e0ff9640 1 -- 172.21.15.38:0/2096287225 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d80074e0 con 0x7f97e40b4b50 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.575+0000 7f97e0ff9640 1 -- 172.21.15.38:0/2096287225 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d80047e0 con 0x7f97e40b4b50 2024-04-25T08:39:58.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.575+0000 7f97ea553640 1 -- 172.21.15.38:0/2096287225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e41e6c90 con 0x7f97e40b4b50 2024-04-25T08:39:58.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.575+0000 7f97e0ff9640 1 -- 172.21.15.38:0/2096287225 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8004bd0 con 0x7f97e40b4b50 2024-04-25T08:39:58.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.575+0000 7f97ea553640 1 -- 172.21.15.38:0/2096287225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e41337f0 con 0x7f97e40b4b50 2024-04-25T08:39:58.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.576+0000 7f97ea553640 1 -- 172.21.15.38:0/2096287225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e4068e80 con 0x7f97e40b4b50 2024-04-25T08:39:58.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.576+0000 7f97e0ff9640 1 -- 172.21.15.38:0/2096287225 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97d8011070 con 0x7f97e40b4b50 2024-04-25T08:39:58.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.577+0000 7f97e0ff9640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f97d006b4d0 0x7f97d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.577+0000 7f97e0ff9640 1 -- 172.21.15.38:0/2096287225 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97d8089f70 con 0x7f97e40b4b50 2024-04-25T08:39:58.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.577+0000 7f97e37fe640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f97d006b4d0 0x7f97d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.578+0000 7f97e37fe640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f97d006b4d0 0x7f97d006d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f97e41f8130 tx=0x7f97d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:58.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.581+0000 7f97e0ff9640 1 -- 172.21.15.38:0/2096287225 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97d8058c60 con 0x7f97e40b4b50 2024-04-25T08:39:58.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.763+0000 7f97ea553640 1 -- 172.21.15.38:0/2096287225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97e406df60 con 0x7f97e40b4b50 2024-04-25T08:39:58.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.764+0000 7f97e0ff9640 1 -- 172.21.15.38:0/2096287225 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16953 (secure 0 0 0) 0x7f97d8058560 con 0x7f97e40b4b50 2024-04-25T08:39:58.757 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:39:58.757 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":120,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:57.475604+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:55.759766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:56.451485+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:39:58.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.766+0000 7f97ea553640 1 -- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f97d006b4d0 msgr2=0x7f97d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.766+0000 7f97ea553640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f97d006b4d0 0x7f97d006d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f97e41f8130 tx=0x7f97d4008040 comp rx=0 tx=0).stop 2024-04-25T08:39:58.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.766+0000 7f97ea553640 1 -- 172.21.15.38:0/2096287225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97e40b4b50 msgr2=0x7f97e41f72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.766+0000 7f97ea553640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97e40b4b50 0x7f97e41f72a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f97d800c430 tx=0x7f97d800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.766+0000 7f97ea553640 1 -- 172.21.15.38:0/2096287225 shutdown_connections 2024-04-25T08:39:58.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.766+0000 7f97ea553640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f97d006b4d0 0x7f97d006d990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.766+0000 7f97ea553640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f97e41438e0 0x7f97e40b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.766+0000 7f97ea553640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97e4144b70 0x7f97e40b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.766+0000 7f97ea553640 1 --2- 172.21.15.38:0/2096287225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97e40b4b50 0x7f97e41f72a0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.766+0000 7f97ea553640 1 -- 172.21.15.38:0/2096287225 >> 172.21.15.38:0/2096287225 conn(0x7f97e406b210 msgr2=0x7f97e40ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:58.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.766+0000 7f97ea553640 1 -- 172.21.15.38:0/2096287225 shutdown_connections 2024-04-25T08:39:58.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.766+0000 7f97ea553640 1 -- 172.21.15.38:0/2096287225 wait complete. 2024-04-25T08:39:58.770 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:39:58.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.882+0000 7f9eaeaf7640 1 Processor -- start 2024-04-25T08:39:58.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.882+0000 7f9eaeaf7640 1 -- start start 2024-04-25T08:39:58.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.882+0000 7f9eaeaf7640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9ea806abe0 0x7f9ea806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.882+0000 7f9eaeaf7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9ea81635e0 0x7f9ea81659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.882+0000 7f9eaeaf7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea806b500 con 0x7f9ea80cc2c0 2024-04-25T08:39:58.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.882+0000 7f9eaeaf7640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea815c1b0 con 0x7f9ea81635e0 2024-04-25T08:39:58.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.882+0000 7f9eaeaf7640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea80ccf30 con 0x7f9ea806abe0 2024-04-25T08:39:58.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.883+0000 7f9ead094640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9ea81635e0 0x7f9ea81659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.883+0000 7f9ea7fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9ea80cc2c0 0x7f9ea80cc740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44974/0 (socket says 172.21.15.38:44974) 2024-04-25T08:39:58.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.883+0000 7f9ea7fff640 1 -- 172.21.15.38:0/3434458639 learned_addr learned my addr 172.21.15.38:0/3434458639 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:58.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.883+0000 7f9eac893640 1 --2- 172.21.15.38:0/3434458639 >> v2:172.21.15.107:3300/0 conn(0x7f9ea806abe0 0x7f9ea806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.883+0000 7f9ead094640 1 -- 172.21.15.38:0/3434458639 >> v2:172.21.15.107:3300/0 conn(0x7f9ea806abe0 msgr2=0x7f9ea806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:58.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.883+0000 7f9ead094640 1 --2- 172.21.15.38:0/3434458639 >> v2:172.21.15.107:3300/0 conn(0x7f9ea806abe0 0x7f9ea806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.883+0000 7f9ead094640 1 -- 172.21.15.38:0/3434458639 >> v1:172.21.15.38:6789/0 conn(0x7f9ea80cc2c0 legacy=0x7f9ea80cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.883+0000 7f9ead094640 1 -- 172.21.15.38:0/3434458639 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea80cd210 con 0x7f9ea81635e0 2024-04-25T08:39:58.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.884+0000 7f9ead094640 1 --2- 172.21.15.38:0/3434458639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ea81635e0 0x7f9ea81659d0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80730b0 tx=0x7f9ea002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34172aece9aa683d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:58.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.884+0000 7f9ea77fe640 1 -- 172.21.15.38:0/3434458639 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0007740 con 0x7f9ea81635e0 2024-04-25T08:39:58.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.884+0000 7f9ea77fe640 1 -- 172.21.15.38:0/3434458639 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea00078e0 con 0x7f9ea81635e0 2024-04-25T08:39:58.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.884+0000 7f9ea77fe640 1 -- 172.21.15.38:0/3434458639 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0007cb0 con 0x7f9ea81635e0 2024-04-25T08:39:58.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.884+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/3434458639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ea81635e0 msgr2=0x7f9ea81659d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.884+0000 7f9eaeaf7640 1 --2- 172.21.15.38:0/3434458639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ea81635e0 0x7f9ea81659d0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80730b0 tx=0x7f9ea002a5f0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.884+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/3434458639 shutdown_connections 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.884+0000 7f9eaeaf7640 1 --2- 172.21.15.38:0/3434458639 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ea81635e0 0x7f9ea81659d0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.884+0000 7f9eaeaf7640 1 --2- 172.21.15.38:0/3434458639 >> v2:172.21.15.107:3300/0 conn(0x7f9ea806abe0 0x7f9ea806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.884+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/3434458639 >> 172.21.15.38:0/3434458639 conn(0x7f9ea80d06b0 msgr2=0x7f9ea80d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.884+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/3434458639 shutdown_connections 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.884+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/3434458639 wait complete. 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.886+0000 7f9eaeaf7640 1 Processor -- start 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.886+0000 7f9eaeaf7640 1 -- start start 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.886+0000 7f9eaeaf7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ea806abe0 0x7f9ea8161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.886+0000 7f9eaeaf7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ea80cc2c0 0x7f9ea814e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.886+0000 7f9eaeaf7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ea81635e0 0x7f9ea814e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.886+0000 7f9eaeaf7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea8159790 con 0x7f9ea806abe0 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.886+0000 7f9eaeaf7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea80cd7d0 con 0x7f9ea81635e0 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.886+0000 7f9eaeaf7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea8159910 con 0x7f9ea80cc2c0 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9eac893640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ea806abe0 0x7f9ea8161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9eac893640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ea806abe0 0x7f9ea8161fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44410/0 (socket says 172.21.15.38:44410) 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9eac893640 1 -- 172.21.15.38:0/4098764182 learned_addr learned my addr 172.21.15.38:0/4098764182 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9ead094640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ea81635e0 0x7f9ea814e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9eac893640 1 -- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ea80cc2c0 msgr2=0x7f9ea814e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9eac893640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ea80cc2c0 0x7f9ea814e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9eac893640 1 -- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ea81635e0 msgr2=0x7f9ea814e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9eac893640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ea81635e0 0x7f9ea814e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9eac893640 1 -- 172.21.15.38:0/4098764182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea814f660 con 0x7f9ea806abe0 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9eac893640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ea806abe0 0x7f9ea8161fd0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00d800 tx=0x7f9e9c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9ea57fa640 1 -- 172.21.15.38:0/4098764182 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00b6c0 con 0x7f9ea806abe0 2024-04-25T08:39:58.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9ea57fa640 1 -- 172.21.15.38:0/4098764182 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c019070 con 0x7f9ea806abe0 2024-04-25T08:39:58.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9ea57fa640 1 -- 172.21.15.38:0/4098764182 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00ba80 con 0x7f9ea806abe0 2024-04-25T08:39:58.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/4098764182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea814f940 con 0x7f9ea806abe0 2024-04-25T08:39:58.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.887+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/4098764182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea8162510 con 0x7f9ea806abe0 2024-04-25T08:39:58.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.888+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/4098764182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea8068e80 con 0x7f9ea806abe0 2024-04-25T08:39:58.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.889+0000 7f9ea57fa640 1 -- 172.21.15.38:0/4098764182 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e9c00bc20 con 0x7f9ea806abe0 2024-04-25T08:39:58.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.889+0000 7f9ea57fa640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9e8c06b500 0x7f9e8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:58.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.889+0000 7f9ea57fa640 1 -- 172.21.15.38:0/4098764182 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9e9c08a7c0 con 0x7f9ea806abe0 2024-04-25T08:39:58.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.889+0000 7f9ea7fff640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9e8c06b500 0x7f9e8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:58.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.890+0000 7f9ea7fff640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9e8c06b500 0x7f9e8c06d9c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9e94000f60 tx=0x7f9e94007a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:58.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:58.893+0000 7f9ea57fa640 1 -- 172.21.15.38:0/4098764182 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e9c01c360 con 0x7f9ea806abe0 2024-04-25T08:39:59.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.116+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/4098764182 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9ea80d1180 con 0x7f9ea806abe0 2024-04-25T08:39:59.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.122+0000 7f9ea57fa640 1 -- 172.21.15.38:0/4098764182 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9e9c059540 con 0x7f9ea806abe0 2024-04-25T08:39:59.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.124+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9e8c06b500 msgr2=0x7f9e8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:59.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.124+0000 7f9eaeaf7640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9e8c06b500 0x7f9e8c06d9c0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9e94000f60 tx=0x7f9e94007a80 comp rx=0 tx=0).stop 2024-04-25T08:39:59.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.124+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/4098764182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ea806abe0 msgr2=0x7f9ea8161fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:59.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.124+0000 7f9eaeaf7640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ea806abe0 0x7f9ea8161fd0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00d800 tx=0x7f9e9c00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:39:59.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.124+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/4098764182 shutdown_connections 2024-04-25T08:39:59.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.124+0000 7f9eaeaf7640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ea81635e0 0x7f9ea814e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:59.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.124+0000 7f9eaeaf7640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9e8c06b500 0x7f9e8c06d9c0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:59.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.124+0000 7f9eaeaf7640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ea80cc2c0 0x7f9ea814e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:59.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.124+0000 7f9eaeaf7640 1 --2- 172.21.15.38:0/4098764182 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ea806abe0 0x7f9ea8161fd0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:59.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.124+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/4098764182 >> 172.21.15.38:0/4098764182 conn(0x7f9ea80d06b0 msgr2=0x7f9ea8161240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:59.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.124+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/4098764182 shutdown_connections 2024-04-25T08:39:59.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.124+0000 7f9eaeaf7640 1 -- 172.21.15.38:0/4098764182 wait complete. 2024-04-25T08:39:59.129 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:39:59.129 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:39:59.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.240+0000 7f2721e93640 1 Processor -- start 2024-04-25T08:39:59.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.240+0000 7f2721e93640 1 -- start start 2024-04-25T08:39:59.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f2721e93640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f271c0d0320 0x7f271c0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:59.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f2721e93640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f271c0d0700 0x7f271c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:59.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f2721e93640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271c0d2330 con 0x7f271c0d1e90 2024-04-25T08:39:59.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f2721e93640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271c0d2610 con 0x7f271c0d0700 2024-04-25T08:39:59.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f2721e93640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271c0d28f0 con 0x7f271c0d0320 2024-04-25T08:39:59.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f271affd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f271c0d0700 0x7f271c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:59.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f271bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f271c0d1e90 0x7f271c0b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44984/0 (socket says 172.21.15.38:44984) 2024-04-25T08:39:59.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f271bfff640 1 -- 172.21.15.38:0/15562422 learned_addr learned my addr 172.21.15.38:0/15562422 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:39:59.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f271affd640 1 --2- 172.21.15.38:0/15562422 >> v2:172.21.15.38:3300/0 conn(0x7f271c0d0700 0x7f271c0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44416/0 (socket says 172.21.15.38:44416) 2024-04-25T08:39:59.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f271b7fe640 1 --2- 172.21.15.38:0/15562422 >> v2:172.21.15.107:3300/0 conn(0x7f271c0d0320 0x7f271c0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:59.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f271affd640 1 -- 172.21.15.38:0/15562422 >> v2:172.21.15.107:3300/0 conn(0x7f271c0d0320 msgr2=0x7f271c0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:39:59.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f271affd640 1 --2- 172.21.15.38:0/15562422 >> v2:172.21.15.107:3300/0 conn(0x7f271c0d0320 0x7f271c0d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:59.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f271affd640 1 -- 172.21.15.38:0/15562422 >> v1:172.21.15.38:6789/0 conn(0x7f271c0d1e90 legacy=0x7f271c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:59.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f271a7fc640 1 -- 172.21.15.38:0/15562422 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155627733 0 0) 0x7f271c0d2330 con 0x7f271c0d1e90 2024-04-25T08:39:59.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.241+0000 7f271affd640 1 -- 172.21.15.38:0/15562422 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c0d2bd0 con 0x7f271c0d0700 2024-04-25T08:39:59.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.242+0000 7f271affd640 1 --2- 172.21.15.38:0/15562422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f271c0d0700 0x7f271c0d1950 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f271c069100 tx=0x7f271002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0566843d05803ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:59.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.242+0000 7f271a7fc640 1 -- 172.21.15.38:0/15562422 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710037070 con 0x7f271c0d0700 2024-04-25T08:39:59.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.242+0000 7f271a7fc640 1 -- 172.21.15.38:0/15562422 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2710032440 con 0x7f271c0d0700 2024-04-25T08:39:59.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.242+0000 7f271a7fc640 1 -- 172.21.15.38:0/15562422 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710032810 con 0x7f271c0d0700 2024-04-25T08:39:59.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.243+0000 7f2721e93640 1 -- 172.21.15.38:0/15562422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f271c0d0700 msgr2=0x7f271c0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:59.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.243+0000 7f2721e93640 1 --2- 172.21.15.38:0/15562422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f271c0d0700 0x7f271c0d1950 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f271c069100 tx=0x7f271002a230 comp rx=0 tx=0).stop 2024-04-25T08:39:59.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.243+0000 7f2721e93640 1 -- 172.21.15.38:0/15562422 shutdown_connections 2024-04-25T08:39:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.243+0000 7f2721e93640 1 --2- 172.21.15.38:0/15562422 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f271c0d0700 0x7f271c0d1950 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.243+0000 7f2721e93640 1 --2- 172.21.15.38:0/15562422 >> v2:172.21.15.107:3300/0 conn(0x7f271c0d0320 0x7f271c0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.243+0000 7f2721e93640 1 -- 172.21.15.38:0/15562422 >> 172.21.15.38:0/15562422 conn(0x7f271c0668f0 msgr2=0x7f271c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:39:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.243+0000 7f2721e93640 1 -- 172.21.15.38:0/15562422 shutdown_connections 2024-04-25T08:39:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.243+0000 7f2721e93640 1 -- 172.21.15.38:0/15562422 wait complete. 2024-04-25T08:39:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.244+0000 7f2721e93640 1 Processor -- start 2024-04-25T08:39:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.244+0000 7f2721e93640 1 -- start start 2024-04-25T08:39:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f2721e93640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f271c0d0320 0x7f271c1f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f2721e93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f271c0d0700 0x7f271c156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f2721e93640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f271c0d1e90 0x7f271c156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:59.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f2721e93640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c0d0b40 con 0x7f271c0d0700 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f2721e93640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c161210 con 0x7f271c0d0320 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f2721e93640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c157520 con 0x7f271c0d1e90 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f271affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f271c0d0700 0x7f271c156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f271affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f271c0d0700 0x7f271c156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44430/0 (socket says 172.21.15.38:44430) 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f271affd640 1 -- 172.21.15.38:0/2863437203 learned_addr learned my addr 172.21.15.38:0/2863437203 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f271b7fe640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f271c0d0320 0x7f271c1f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f271bfff640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f271c0d1e90 0x7f271c156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f271affd640 1 -- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f271c0d1e90 msgr2=0x7f271c156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f271affd640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f271c0d1e90 0x7f271c156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f271affd640 1 -- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f271c0d0320 msgr2=0x7f271c1f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f271affd640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f271c0d0320 0x7f271c1f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f271affd640 1 -- 172.21.15.38:0/2863437203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c157240 con 0x7f271c0d0700 2024-04-25T08:39:59.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f271affd640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f271c0d0700 0x7f271c156710 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f2710009890 tx=0x7f27100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:59.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.245+0000 7f2703fff640 1 -- 172.21.15.38:0/2863437203 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710037040 con 0x7f271c0d0700 2024-04-25T08:39:59.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.246+0000 7f2703fff640 1 -- 172.21.15.38:0/2863437203 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2710042070 con 0x7f271c0d0700 2024-04-25T08:39:59.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.246+0000 7f2721e93640 1 -- 172.21.15.38:0/2863437203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c158c80 con 0x7f271c0d0700 2024-04-25T08:39:59.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.246+0000 7f2703fff640 1 -- 172.21.15.38:0/2863437203 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2710030bd0 con 0x7f271c0d0700 2024-04-25T08:39:59.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.246+0000 7f2721e93640 1 -- 172.21.15.38:0/2863437203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c158930 con 0x7f271c0d0700 2024-04-25T08:39:59.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.247+0000 7f2721e93640 1 -- 172.21.15.38:0/2863437203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271c068e80 con 0x7f271c0d0700 2024-04-25T08:39:59.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.247+0000 7f2703fff640 1 -- 172.21.15.38:0/2863437203 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2710047460 con 0x7f271c0d0700 2024-04-25T08:39:59.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.247+0000 7f2703fff640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f270806b4d0 0x7f270806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:39:59.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.248+0000 7f2703fff640 1 -- 172.21.15.38:0/2863437203 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27100aafc0 con 0x7f271c0d0700 2024-04-25T08:39:59.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.248+0000 7f271b7fe640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f270806b4d0 0x7f270806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:39:59.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.248+0000 7f271b7fe640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f270806b4d0 0x7f270806d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f270c006fd0 tx=0x7f270c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:39:59.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.252+0000 7f2703fff640 1 -- 172.21.15.38:0/2863437203 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2710032e20 con 0x7f271c0d0700 2024-04-25T08:39:59.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:39:59.470+0000 7f2721e93640 1 -- 172.21.15.38:0/2863437203 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f271c0676e0 con 0x7f271c0d0700 2024-04-25T08:40:00.086 INFO:tasks.ceph.mds.d.smithi107.stderr:2024-04-25T08:40:00.093+0000 7f1cd6fed640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:40:00.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.134+0000 7f2703fff640 1 -- 172.21.15.38:0/2863437203 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2710079cb0 con 0x7f271c0d0700 2024-04-25T08:40:00.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.136+0000 7f2721e93640 1 -- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f270806b4d0 msgr2=0x7f270806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.136+0000 7f2721e93640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f270806b4d0 0x7f270806d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f270c006fd0 tx=0x7f270c008040 comp rx=0 tx=0).stop 2024-04-25T08:40:00.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.136+0000 7f2721e93640 1 -- 172.21.15.38:0/2863437203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f271c0d0700 msgr2=0x7f271c156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.136+0000 7f2721e93640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f271c0d0700 0x7f271c156710 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f2710009890 tx=0x7f27100046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.137+0000 7f2721e93640 1 -- 172.21.15.38:0/2863437203 shutdown_connections 2024-04-25T08:40:00.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.137+0000 7f2721e93640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f270806b4d0 0x7f270806d990 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.137+0000 7f2721e93640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f271c0d1e90 0x7f271c156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.137+0000 7f2721e93640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f271c0d0700 0x7f271c156710 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.137+0000 7f2721e93640 1 --2- 172.21.15.38:0/2863437203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f271c0d0320 0x7f271c1f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.137+0000 7f2721e93640 1 -- 172.21.15.38:0/2863437203 >> 172.21.15.38:0/2863437203 conn(0x7f271c0668f0 msgr2=0x7f271c15e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.137+0000 7f2721e93640 1 -- 172.21.15.38:0/2863437203 shutdown_connections 2024-04-25T08:40:00.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.137+0000 7f2721e93640 1 -- 172.21.15.38:0/2863437203 wait complete. 2024-04-25T08:40:00.144 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:40:00.144 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:00.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.260+0000 7f7d067e0640 1 Processor -- start 2024-04-25T08:40:00.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.260+0000 7f7d067e0640 1 -- start start 2024-04-25T08:40:00.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7d067e0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7d0015ddf0 0x7f7d00161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7d067e0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7d000b93f0 0x7f7d000d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7d067e0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d000b98a0 con 0x7f7d000b5710 2024-04-25T08:40:00.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7d067e0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d00161e00 con 0x7f7d0015ddf0 2024-04-25T08:40:00.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7d067e0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d001620e0 con 0x7f7d000b93f0 2024-04-25T08:40:00.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7cff7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7d0015ddf0 0x7f7d00161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7cfffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7d000b5710 0x7f7d000b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44998/0 (socket says 172.21.15.38:44998) 2024-04-25T08:40:00.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7cfffff640 1 -- 172.21.15.38:0/3973144502 learned_addr learned my addr 172.21.15.38:0/3973144502 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:00.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7d04d7d640 1 --2- 172.21.15.38:0/3973144502 >> v2:172.21.15.107:3300/0 conn(0x7f7d000b93f0 0x7f7d000d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7cff7fe640 1 -- 172.21.15.38:0/3973144502 >> v2:172.21.15.107:3300/0 conn(0x7f7d000b93f0 msgr2=0x7f7d000d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:00.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7cff7fe640 1 --2- 172.21.15.38:0/3973144502 >> v2:172.21.15.107:3300/0 conn(0x7f7d000b93f0 0x7f7d000d2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7cff7fe640 1 -- 172.21.15.38:0/3973144502 >> v1:172.21.15.38:6789/0 conn(0x7f7d000b5710 legacy=0x7f7d000b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7cff7fe640 1 -- 172.21.15.38:0/3973144502 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d001623c0 con 0x7f7d0015ddf0 2024-04-25T08:40:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.261+0000 7f7cfeffd640 1 -- 172.21.15.38:0/3973144502 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553307165 0 0) 0x7f7d000b98a0 con 0x7f7d000b5710 2024-04-25T08:40:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.262+0000 7f7cff7fe640 1 --2- 172.21.15.38:0/3973144502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d0015ddf0 0x7f7d00161700 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f7d0008b240 tx=0x7f7cf002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47b2bc1b302b8fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.262+0000 7f7cfeffd640 1 -- 172.21.15.38:0/3973144502 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0037070 con 0x7f7d0015ddf0 2024-04-25T08:40:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.262+0000 7f7cfeffd640 1 -- 172.21.15.38:0/3973144502 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf0032440 con 0x7f7d0015ddf0 2024-04-25T08:40:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.262+0000 7f7cfeffd640 1 -- 172.21.15.38:0/3973144502 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0032810 con 0x7f7d0015ddf0 2024-04-25T08:40:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.263+0000 7f7d067e0640 1 -- 172.21.15.38:0/3973144502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d0015ddf0 msgr2=0x7f7d00161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.263+0000 7f7d067e0640 1 --2- 172.21.15.38:0/3973144502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d0015ddf0 0x7f7d00161700 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f7d0008b240 tx=0x7f7cf002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.263+0000 7f7d067e0640 1 -- 172.21.15.38:0/3973144502 shutdown_connections 2024-04-25T08:40:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.263+0000 7f7d067e0640 1 --2- 172.21.15.38:0/3973144502 >> v2:172.21.15.107:3300/0 conn(0x7f7d000b93f0 0x7f7d000d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.263+0000 7f7d067e0640 1 --2- 172.21.15.38:0/3973144502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d0015ddf0 0x7f7d00161700 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.263+0000 7f7d067e0640 1 -- 172.21.15.38:0/3973144502 >> 172.21.15.38:0/3973144502 conn(0x7f7d000d07c0 msgr2=0x7f7d000d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.263+0000 7f7d067e0640 1 -- 172.21.15.38:0/3973144502 shutdown_connections 2024-04-25T08:40:00.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.263+0000 7f7d067e0640 1 -- 172.21.15.38:0/3973144502 wait complete. 2024-04-25T08:40:00.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.264+0000 7f7d067e0640 1 Processor -- start 2024-04-25T08:40:00.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.265+0000 7f7d067e0640 1 -- start start 2024-04-25T08:40:00.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.265+0000 7f7d067e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d000b5710 0x7f7d001468a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.265+0000 7f7d067e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d000b93f0 0x7f7d000ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.265+0000 7f7d067e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d0015ddf0 0x7f7d00146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.265+0000 7f7d067e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000d2c10 con 0x7f7d000b5710 2024-04-25T08:40:00.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.265+0000 7f7d067e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000d2a90 con 0x7f7d0015ddf0 2024-04-25T08:40:00.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.265+0000 7f7d067e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d000d2d90 con 0x7f7d000b93f0 2024-04-25T08:40:00.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cfffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d000b5710 0x7f7d001468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cfffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d000b5710 0x7f7d001468a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44450/0 (socket says 172.21.15.38:44450) 2024-04-25T08:40:00.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cfffff640 1 -- 172.21.15.38:0/487596912 learned_addr learned my addr 172.21.15.38:0/487596912 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:00.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7d04d7d640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d0015ddf0 0x7f7d00146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cff7fe640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d000b93f0 0x7f7d000ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cfffff640 1 -- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d000b93f0 msgr2=0x7f7d000ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cfffff640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d000b93f0 0x7f7d000ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cfffff640 1 -- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d0015ddf0 msgr2=0x7f7d00146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cfffff640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d0015ddf0 0x7f7d00146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cfffff640 1 -- 172.21.15.38:0/487596912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d000cc5f0 con 0x7f7d000b5710 2024-04-25T08:40:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cfffff640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d000b5710 0x7f7d001468a0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800c4d0 tx=0x7f7ce800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cfcff9640 1 -- 172.21.15.38:0/487596912 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8018070 con 0x7f7d000b5710 2024-04-25T08:40:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cfcff9640 1 -- 172.21.15.38:0/487596912 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce8007500 con 0x7f7d000b5710 2024-04-25T08:40:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7d067e0640 1 -- 172.21.15.38:0/487596912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d000c7b70 con 0x7f7d000b5710 2024-04-25T08:40:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7cfcff9640 1 -- 172.21.15.38:0/487596912 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8004a20 con 0x7f7d000b5710 2024-04-25T08:40:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.266+0000 7f7d067e0640 1 -- 172.21.15.38:0/487596912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d000c7890 con 0x7f7d000b5710 2024-04-25T08:40:00.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.267+0000 7f7d067e0640 1 -- 172.21.15.38:0/487596912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d00068e80 con 0x7f7d000b5710 2024-04-25T08:40:00.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.268+0000 7f7cfcff9640 1 -- 172.21.15.38:0/487596912 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ce801b070 con 0x7f7d000b5710 2024-04-25T08:40:00.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.268+0000 7f7cfcff9640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ce006b500 0x7f7ce006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.268+0000 7f7cfcff9640 1 -- 172.21.15.38:0/487596912 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7ce80895a0 con 0x7f7d000b5710 2024-04-25T08:40:00.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.269+0000 7f7cff7fe640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ce006b500 0x7f7ce006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.269+0000 7f7cff7fe640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ce006b500 0x7f7ce006d9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0034000 tx=0x7f7cf002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.273+0000 7f7cfcff9640 1 -- 172.21.15.38:0/487596912 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ce8057c40 con 0x7f7d000b5710 2024-04-25T08:40:00.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.495+0000 7f7d067e0640 1 -- 172.21.15.38:0/487596912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d000b98a0 con 0x7f7d000b5710 2024-04-25T08:40:00.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.496+0000 7f7cfcff9640 1 -- 172.21.15.38:0/487596912 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f7ce805bce0 con 0x7f7d000b5710 2024-04-25T08:40:00.489 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:00.489 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:00.490 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.498+0000 7f7d067e0640 1 -- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ce006b500 msgr2=0x7f7ce006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.498+0000 7f7d067e0640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ce006b500 0x7f7ce006d9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0034000 tx=0x7f7cf002e000 comp rx=0 tx=0).stop 2024-04-25T08:40:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.498+0000 7f7d067e0640 1 -- 172.21.15.38:0/487596912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d000b5710 msgr2=0x7f7d001468a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.498+0000 7f7d067e0640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d000b5710 0x7f7d001468a0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800c4d0 tx=0x7f7ce800c990 comp rx=0 tx=0).stop 2024-04-25T08:40:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.498+0000 7f7d067e0640 1 -- 172.21.15.38:0/487596912 shutdown_connections 2024-04-25T08:40:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.498+0000 7f7d067e0640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7d0015ddf0 0x7f7d00146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.498+0000 7f7d067e0640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ce006b500 0x7f7ce006d9c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.498+0000 7f7d067e0640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7d000b93f0 0x7f7d000ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.498+0000 7f7d067e0640 1 --2- 172.21.15.38:0/487596912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7d000b5710 0x7f7d001468a0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.498+0000 7f7d067e0640 1 -- 172.21.15.38:0/487596912 >> 172.21.15.38:0/487596912 conn(0x7f7d000d07c0 msgr2=0x7f7d000ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.498+0000 7f7d067e0640 1 -- 172.21.15.38:0/487596912 shutdown_connections 2024-04-25T08:40:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.498+0000 7f7d067e0640 1 -- 172.21.15.38:0/487596912 wait complete. 2024-04-25T08:40:00.502 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:40:00.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.611+0000 7f90dd77e640 1 Processor -- start 2024-04-25T08:40:00.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.611+0000 7f90dd77e640 1 -- start start 2024-04-25T08:40:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90dd77e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f90d806c110 0x7f90d806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90dd77e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90d80ce2e0 0x7f90d80d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90dd77e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d806ca30 con 0x7f90d815df50 2024-04-25T08:40:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90dd77e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d80cf2d0 con 0x7f90d806c110 2024-04-25T08:40:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90dd77e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80ceff0 con 0x7f90d80ce2e0 2024-04-25T08:40:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90d6ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f90d806c110 0x7f90d806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90cffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f90d815df50 0x7f90d80ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45002/0 (socket says 172.21.15.38:45002) 2024-04-25T08:40:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90cffff640 1 -- 172.21.15.38:0/3075319331 learned_addr learned my addr 172.21.15.38:0/3075319331 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90d77fe640 1 --2- 172.21.15.38:0/3075319331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90d80ce2e0 0x7f90d80d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90d6ffd640 1 -- 172.21.15.38:0/3075319331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90d80ce2e0 msgr2=0x7f90d80d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90d6ffd640 1 --2- 172.21.15.38:0/3075319331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90d80ce2e0 0x7f90d80d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90d6ffd640 1 -- 172.21.15.38:0/3075319331 >> v1:172.21.15.38:6789/0 conn(0x7f90d815df50 legacy=0x7f90d80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90d6ffd640 1 -- 172.21.15.38:0/3075319331 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d80d1f70 con 0x7f90d806c110 2024-04-25T08:40:00.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.612+0000 7f90d67fc640 1 -- 172.21.15.38:0/3075319331 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179606029 0 0) 0x7f90d806ca30 con 0x7f90d815df50 2024-04-25T08:40:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.613+0000 7f90d6ffd640 1 --2- 172.21.15.38:0/3075319331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90d806c110 0x7f90d806c4f0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f90c80097a0 tx=0x7f90c802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0ea2d1599019be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.613+0000 7f90d67fc640 1 -- 172.21.15.38:0/3075319331 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8037070 con 0x7f90d806c110 2024-04-25T08:40:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.613+0000 7f90d67fc640 1 -- 172.21.15.38:0/3075319331 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c8032440 con 0x7f90d806c110 2024-04-25T08:40:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.613+0000 7f90d67fc640 1 -- 172.21.15.38:0/3075319331 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8032810 con 0x7f90d806c110 2024-04-25T08:40:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.614+0000 7f90dd77e640 1 -- 172.21.15.38:0/3075319331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90d806c110 msgr2=0x7f90d806c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.614+0000 7f90dd77e640 1 --2- 172.21.15.38:0/3075319331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90d806c110 0x7f90d806c4f0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f90c80097a0 tx=0x7f90c802a180 comp rx=0 tx=0).stop 2024-04-25T08:40:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.614+0000 7f90dd77e640 1 -- 172.21.15.38:0/3075319331 shutdown_connections 2024-04-25T08:40:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.614+0000 7f90dd77e640 1 --2- 172.21.15.38:0/3075319331 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90d80ce2e0 0x7f90d80d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.614+0000 7f90dd77e640 1 --2- 172.21.15.38:0/3075319331 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90d806c110 0x7f90d806c4f0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.614+0000 7f90dd77e640 1 -- 172.21.15.38:0/3075319331 >> 172.21.15.38:0/3075319331 conn(0x7f90d80d3fb0 msgr2=0x7f90d80d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.614+0000 7f90dd77e640 1 -- 172.21.15.38:0/3075319331 shutdown_connections 2024-04-25T08:40:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.614+0000 7f90dd77e640 1 -- 172.21.15.38:0/3075319331 wait complete. 2024-04-25T08:40:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.615+0000 7f90dd77e640 1 Processor -- start 2024-04-25T08:40:00.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.615+0000 7f90dd77e640 1 -- start start 2024-04-25T08:40:00.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90dd77e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90d806c110 0x7f90d80b7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90dd77e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90d80ce2e0 0x7f90d81f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90dd77e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90d815df50 0x7f90d814cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90dd77e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80d2f90 con 0x7f90d806c110 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90dd77e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80d2530 con 0x7f90d80ce2e0 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90dd77e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d80d3110 con 0x7f90d815df50 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90d6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90d806c110 0x7f90d80b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90d6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90d806c110 0x7f90d80b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44470/0 (socket says 172.21.15.38:44470) 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90d6ffd640 1 -- 172.21.15.38:0/3023098123 learned_addr learned my addr 172.21.15.38:0/3023098123 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90d77fe640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90d815df50 0x7f90d814cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90cffff640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90d80ce2e0 0x7f90d81f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90d6ffd640 1 -- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90d815df50 msgr2=0x7f90d814cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90d6ffd640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90d815df50 0x7f90d814cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90d6ffd640 1 -- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90d80ce2e0 msgr2=0x7f90d81f38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90d6ffd640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90d80ce2e0 0x7f90d81f38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90d6ffd640 1 -- 172.21.15.38:0/3023098123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d81f4060 con 0x7f90d806c110 2024-04-25T08:40:00.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90d6ffd640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90d806c110 0x7f90d80b7760 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f90c80097a0 tx=0x7f90c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90cf7fe640 1 -- 172.21.15.38:0/3023098123 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8037050 con 0x7f90d806c110 2024-04-25T08:40:00.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90cf7fe640 1 -- 172.21.15.38:0/3023098123 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c8038040 con 0x7f90d806c110 2024-04-25T08:40:00.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90dd77e640 1 -- 172.21.15.38:0/3023098123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90d81f4340 con 0x7f90d806c110 2024-04-25T08:40:00.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.616+0000 7f90cf7fe640 1 -- 172.21.15.38:0/3023098123 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c8004430 con 0x7f90d806c110 2024-04-25T08:40:00.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.617+0000 7f90dd77e640 1 -- 172.21.15.38:0/3023098123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d815e4d0 con 0x7f90d806c110 2024-04-25T08:40:00.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.618+0000 7f90dd77e640 1 -- 172.21.15.38:0/3023098123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d81f4860 con 0x7f90d806c110 2024-04-25T08:40:00.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.618+0000 7f90cf7fe640 1 -- 172.21.15.38:0/3023098123 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90c803c6d0 con 0x7f90d806c110 2024-04-25T08:40:00.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.619+0000 7f90cf7fe640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f90b806b4d0 0x7f90b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.619+0000 7f90cf7fe640 1 -- 172.21.15.38:0/3023098123 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90c80aad00 con 0x7f90d806c110 2024-04-25T08:40:00.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.619+0000 7f90cffff640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f90b806b4d0 0x7f90b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.619+0000 7f90cffff640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f90b806b4d0 0x7f90b806d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f90d815faf0 tx=0x7f90c0007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.623+0000 7f90cf7fe640 1 -- 172.21.15.38:0/3023098123 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90d81f4860 con 0x7f90d806c110 2024-04-25T08:40:00.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.804+0000 7f90dd77e640 1 -- 172.21.15.38:0/3023098123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90d8072e40 con 0x7f90d806c110 2024-04-25T08:40:00.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.806+0000 7f90cf7fe640 1 -- 172.21.15.38:0/3023098123 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f90d8072e40 con 0x7f90d806c110 2024-04-25T08:40:00.799 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:00.799 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":121,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:59.121832+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:55.759766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:56.451485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:40:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.807+0000 7f90dd77e640 1 -- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f90b806b4d0 msgr2=0x7f90b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.807+0000 7f90dd77e640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f90b806b4d0 0x7f90b806d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f90d815faf0 tx=0x7f90c0007de0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.807+0000 7f90dd77e640 1 -- 172.21.15.38:0/3023098123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90d806c110 msgr2=0x7f90d80b7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.807+0000 7f90dd77e640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90d806c110 0x7f90d80b7760 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f90c80097a0 tx=0x7f90c80046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.807+0000 7f90dd77e640 1 -- 172.21.15.38:0/3023098123 shutdown_connections 2024-04-25T08:40:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.807+0000 7f90dd77e640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f90b806b4d0 0x7f90b806d990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.807+0000 7f90dd77e640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f90d815df50 0x7f90d814cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.807+0000 7f90dd77e640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f90d80ce2e0 0x7f90d81f38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.807+0000 7f90dd77e640 1 --2- 172.21.15.38:0/3023098123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f90d806c110 0x7f90d80b7760 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.807+0000 7f90dd77e640 1 -- 172.21.15.38:0/3023098123 >> 172.21.15.38:0/3023098123 conn(0x7f90d80d3fb0 msgr2=0x7f90d815cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.808+0000 7f90dd77e640 1 -- 172.21.15.38:0/3023098123 shutdown_connections 2024-04-25T08:40:00.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.808+0000 7f90dd77e640 1 -- 172.21.15.38:0/3023098123 wait complete. 2024-04-25T08:40:00.812 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:40:00.812 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:00.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.920+0000 7f9e0b8e2640 1 Processor -- start 2024-04-25T08:40:00.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.920+0000 7f9e0b8e2640 1 -- start start 2024-04-25T08:40:00.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e0b8e2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e0415dc10 0x7f9e040ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e0b8e2640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9e040ced60 0x7f9e040d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e0b8e2640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e040d3e70 con 0x7f9e040d3900 2024-04-25T08:40:00.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e0b8e2640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0415dff0 con 0x7f9e040ced60 2024-04-25T08:40:00.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e0b8e2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e040b6990 con 0x7f9e0415dc10 2024-04-25T08:40:00.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e09e7f640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9e040d3900 0x7f9e040b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45012/0 (socket says 172.21.15.38:45012) 2024-04-25T08:40:00.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e09e7f640 1 -- 172.21.15.38:0/2067354223 learned_addr learned my addr 172.21.15.38:0/2067354223 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:00.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e08e7d640 1 --2- 172.21.15.38:0/2067354223 >> v2:172.21.15.107:3300/0 conn(0x7f9e040ced60 0x7f9e040d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e0967e640 1 --2- 172.21.15.38:0/2067354223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e0415dc10 0x7f9e040ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e03fff640 1 -- 172.21.15.38:0/2067354223 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2649777722 0 0) 0x7f9e040d3e70 con 0x7f9e040d3900 2024-04-25T08:40:00.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e03fff640 1 -- 172.21.15.38:0/2067354223 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dec003520 con 0x7f9e040d3900 2024-04-25T08:40:00.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e08e7d640 1 -- 172.21.15.38:0/2067354223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e0415dc10 msgr2=0x7f9e040ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:00.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e08e7d640 1 --2- 172.21.15.38:0/2067354223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e0415dc10 0x7f9e040ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.921+0000 7f9e08e7d640 1 -- 172.21.15.38:0/2067354223 >> v1:172.21.15.38:6789/0 conn(0x7f9e040d3900 legacy=0x7f9e040b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.922+0000 7f9e08e7d640 1 -- 172.21.15.38:0/2067354223 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e040d4150 con 0x7f9e040ced60 2024-04-25T08:40:00.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.922+0000 7f9e0967e640 1 --2- 172.21.15.38:0/2067354223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e0415dc10 0x7f9e040ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:00.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.922+0000 7f9e08e7d640 1 --2- 172.21.15.38:0/2067354223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e040ced60 0x7f9e040d0850 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9df80097a0 tx=0x7f9df802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9517ad75369da8f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.922+0000 7f9e03fff640 1 -- 172.21.15.38:0/2067354223 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8037070 con 0x7f9e040ced60 2024-04-25T08:40:00.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.922+0000 7f9e03fff640 1 -- 172.21.15.38:0/2067354223 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df802ae50 con 0x7f9e040ced60 2024-04-25T08:40:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.922+0000 7f9e03fff640 1 -- 172.21.15.38:0/2067354223 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8032660 con 0x7f9e040ced60 2024-04-25T08:40:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.923+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/2067354223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e040ced60 msgr2=0x7f9e040d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.923+0000 7f9e0b8e2640 1 --2- 172.21.15.38:0/2067354223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e040ced60 0x7f9e040d0850 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9df80097a0 tx=0x7f9df802a100 comp rx=0 tx=0).stop 2024-04-25T08:40:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.923+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/2067354223 shutdown_connections 2024-04-25T08:40:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.923+0000 7f9e0b8e2640 1 --2- 172.21.15.38:0/2067354223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e040ced60 0x7f9e040d0850 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.923+0000 7f9e0b8e2640 1 --2- 172.21.15.38:0/2067354223 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e0415dc10 0x7f9e040ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.923+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/2067354223 >> 172.21.15.38:0/2067354223 conn(0x7f9e04066d10 msgr2=0x7f9e04067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.923+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/2067354223 shutdown_connections 2024-04-25T08:40:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.923+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/2067354223 wait complete. 2024-04-25T08:40:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.924+0000 7f9e0b8e2640 1 Processor -- start 2024-04-25T08:40:00.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.924+0000 7f9e0b8e2640 1 -- start start 2024-04-25T08:40:00.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0b8e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e040ced60 0x7f9e0414ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0b8e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e040d3900 0x7f9e0415ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0b8e2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e0415dc10 0x7f9e0415b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0b8e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e040b6bc0 con 0x7f9e040ced60 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0b8e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e040d4710 con 0x7f9e040d3900 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0b8e2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e040d0d90 con 0x7f9e0415dc10 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0967e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e040ced60 0x7f9e0414ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0967e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e040ced60 0x7f9e0414ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44484/0 (socket says 172.21.15.38:44484) 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0967e640 1 -- 172.21.15.38:0/3112818423 learned_addr learned my addr 172.21.15.38:0/3112818423 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e08e7d640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e040d3900 0x7f9e0415ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e09e7f640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e0415dc10 0x7f9e0415b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0967e640 1 -- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e0415dc10 msgr2=0x7f9e0415b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0967e640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e0415dc10 0x7f9e0415b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0967e640 1 -- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e040d3900 msgr2=0x7f9e0415ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:00.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0967e640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e040d3900 0x7f9e0415ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e0967e640 1 -- 172.21.15.38:0/3112818423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0414b070 con 0x7f9e040ced60 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.925+0000 7f9e09e7f640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e0415dc10 0x7f9e0415b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.926+0000 7f9e0967e640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e040ced60 0x7f9e0414ab30 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00e990 tx=0x7f9dfc00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.926+0000 7f9e01ffb640 1 -- 172.21.15.38:0/3112818423 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc017070 con 0x7f9e040ced60 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.926+0000 7f9e01ffb640 1 -- 172.21.15.38:0/3112818423 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dfc00c9e0 con 0x7f9e040ced60 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.926+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/3112818423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e040c7b10 con 0x7f9e040ced60 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.926+0000 7f9e01ffb640 1 -- 172.21.15.38:0/3112818423 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc00cdb0 con 0x7f9e040ced60 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.926+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/3112818423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e040ca560 con 0x7f9e040ced60 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.927+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/3112818423 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e04068e80 con 0x7f9e040ced60 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.927+0000 7f9e01ffb640 1 -- 172.21.15.38:0/3112818423 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9dfc01a070 con 0x7f9e040ced60 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.928+0000 7f9e01ffb640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9df006b500 0x7f9df006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:00.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.928+0000 7f9e01ffb640 1 -- 172.21.15.38:0/3112818423 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9dfc08a210 con 0x7f9e040ced60 2024-04-25T08:40:00.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.928+0000 7f9e08e7d640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9df006b500 0x7f9df006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:00.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.929+0000 7f9e08e7d640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9df006b500 0x7f9df006d9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9df8034000 tx=0x7f9df802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:00.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:00.932+0000 7f9e01ffb640 1 -- 172.21.15.38:0/3112818423 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9dfc058830 con 0x7f9e040ced60 2024-04-25T08:40:01.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.109+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/3112818423 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9e040c7ce0 con 0x7f9df006b500 2024-04-25T08:40:01.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.115+0000 7f9e01ffb640 1 -- 172.21.15.38:0/3112818423 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7f9e040c7ce0 con 0x7f9df006b500 2024-04-25T08:40:01.112 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:01.114 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:40:01.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.118+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9df006b500 msgr2=0x7f9df006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.118+0000 7f9e0b8e2640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9df006b500 0x7f9df006d9c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f9df8034000 tx=0x7f9df802e000 comp rx=0 tx=0).stop 2024-04-25T08:40:01.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.118+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/3112818423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e040ced60 msgr2=0x7f9e0414ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.118+0000 7f9e0b8e2640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e040ced60 0x7f9e0414ab30 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00e990 tx=0x7f9dfc00ee50 comp rx=0 tx=0).stop 2024-04-25T08:40:01.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.118+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/3112818423 shutdown_connections 2024-04-25T08:40:01.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.118+0000 7f9e0b8e2640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9df006b500 0x7f9df006d9c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.118+0000 7f9e0b8e2640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9e0415dc10 0x7f9e0415b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.118+0000 7f9e0b8e2640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e040d3900 0x7f9e0415ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.118+0000 7f9e0b8e2640 1 --2- 172.21.15.38:0/3112818423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9e040ced60 0x7f9e0414ab30 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.118+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/3112818423 >> 172.21.15.38:0/3112818423 conn(0x7f9e04066d10 msgr2=0x7f9e040d3170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:01.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.118+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/3112818423 shutdown_connections 2024-04-25T08:40:01.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.119+0000 7f9e0b8e2640 1 -- 172.21.15.38:0/3112818423 wait complete. 2024-04-25T08:40:01.123 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-04-25T08:39:59.322626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":498104,"kb_used_data":12648,"kb_used_omap":12,"kb_used_meta":485299,"kb_avail":754476616,"statfs":{"total":773094113280,"available":772584054784,"internally_reserved":0,"allocated":12951552,"data_stored":6646960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":496946736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000829"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104507+0000","last_change":"2024-04-25T08:39:57.104507+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104507+0000","last_undegraded":"2024-04-25T08:39:57.104507+0000","last_fullsized":"2024-04-25T08:39:57.104507+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108915+0000","last_change":"2024-04-25T08:39:57.108915+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108915+0000","last_undegraded":"2024-04-25T08:39:57.108915+0000","last_fullsized":"2024-04-25T08:39:57.108915+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105895+0000","last_change":"2024-04-25T08:39:57.105895+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105895+0000","last_undegraded":"2024-04-25T08:39:57.105895+0000","last_fullsized":"2024-04-25T08:39:57.105895+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109343+0000","last_change":"2024-04-25T08:39:57.109343+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109343+0000","last_undegraded":"2024-04-25T08:39:57.109343+0000","last_fullsized":"2024-04-25T08:39:57.109343+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105957+0000","last_change":"2024-04-25T08:39:57.105957+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105957+0000","last_undegraded":"2024-04-25T08:39:57.105957+0000","last_fullsized":"2024-04-25T08:39:57.105957+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107903+0000","last_change":"2024-04-25T08:39:57.107903+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107903+0000","last_undegraded":"2024-04-25T08:39:57.107903+0000","last_fullsized":"2024-04-25T08:39:57.107903+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109081+0000","last_change":"2024-04-25T08:39:57.109081+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109081+0000","last_undegraded":"2024-04-25T08:39:57.109081+0000","last_fullsized":"2024-04-25T08:39:57.109081+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106732+0000","last_change":"2024-04-25T08:39:57.106732+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106732+0000","last_undegraded":"2024-04-25T08:39:57.106732+0000","last_fullsized":"2024-04-25T08:39:57.106732+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108726+0000","last_change":"2024-04-25T08:39:57.108726+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108726+0000","last_undegraded":"2024-04-25T08:39:57.108726+0000","last_fullsized":"2024-04-25T08:39:57.108726+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110497+0000","last_change":"2024-04-25T08:39:57.110497+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110497+0000","last_undegraded":"2024-04-25T08:39:57.110497+0000","last_fullsized":"2024-04-25T08:39:57.110497+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105137+0000","last_change":"2024-04-25T08:39:57.105137+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105137+0000","last_undegraded":"2024-04-25T08:39:57.105137+0000","last_fullsized":"2024-04-25T08:39:57.105137+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109567+0000","last_change":"2024-04-25T08:39:57.109567+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109567+0000","last_undegraded":"2024-04-25T08:39:57.109567+0000","last_fullsized":"2024-04-25T08:39:57.109567+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.115061+0000","last_change":"2024-04-25T08:39:57.115061+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.115061+0000","last_undegraded":"2024-04-25T08:39:57.115061+0000","last_fullsized":"2024-04-25T08:39:57.115061+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109950+0000","last_change":"2024-04-25T08:39:57.109950+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109950+0000","last_undegraded":"2024-04-25T08:39:57.109950+0000","last_fullsized":"2024-04-25T08:39:57.109950+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104679+0000","last_change":"2024-04-25T08:39:57.104679+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104679+0000","last_undegraded":"2024-04-25T08:39:57.104679+0000","last_fullsized":"2024-04-25T08:39:57.104679+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109887+0000","last_change":"2024-04-25T08:39:57.109887+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109887+0000","last_undegraded":"2024-04-25T08:39:57.109887+0000","last_fullsized":"2024-04-25T08:39:57.109887+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.112277+0000","last_change":"2024-04-25T08:39:57.112277+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.112277+0000","last_undegraded":"2024-04-25T08:39:57.112277+0000","last_fullsized":"2024-04-25T08:39:57.112277+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108487+0000","last_change":"2024-04-25T08:39:57.108487+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108487+0000","last_undegraded":"2024-04-25T08:39:57.108487+0000","last_fullsized":"2024-04-25T08:39:57.108487+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107714+0000","last_change":"2024-04-25T08:39:57.107714+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107714+0000","last_undegraded":"2024-04-25T08:39:57.107714+0000","last_fullsized":"2024-04-25T08:39:57.107714+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105862+0000","last_change":"2024-04-25T08:39:57.105862+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105862+0000","last_undegraded":"2024-04-25T08:39:57.105862+0000","last_fullsized":"2024-04-25T08:39:57.105862+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108441+0000","last_change":"2024-04-25T08:39:57.108441+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108441+0000","last_undegraded":"2024-04-25T08:39:57.108441+0000","last_fullsized":"2024-04-25T08:39:57.108441+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105711+0000","last_change":"2024-04-25T08:39:57.105711+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105711+0000","last_undegraded":"2024-04-25T08:39:57.105711+0000","last_fullsized":"2024-04-25T08:39:57.105711+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107927+0000","last_change":"2024-04-25T08:39:57.107927+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107927+0000","last_undegraded":"2024-04-25T08:39:57.107927+0000","last_fullsized":"2024-04-25T08:39:57.107927+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111019+0000","last_change":"2024-04-25T08:39:57.111019+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111019+0000","last_undegraded":"2024-04-25T08:39:57.111019+0000","last_fullsized":"2024-04-25T08:39:57.111019+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.112909+0000","last_change":"2024-04-25T08:39:57.112909+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.112909+0000","last_undegraded":"2024-04-25T08:39:57.112909+0000","last_fullsized":"2024-04-25T08:39:57.112909+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106551+0000","last_change":"2024-04-25T08:39:57.106551+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106551+0000","last_undegraded":"2024-04-25T08:39:57.106551+0000","last_fullsized":"2024-04-25T08:39:57.106551+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106831+0000","last_change":"2024-04-25T08:39:57.106831+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106831+0000","last_undegraded":"2024-04-25T08:39:57.106831+0000","last_fullsized":"2024-04-25T08:39:57.106831+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109055+0000","last_change":"2024-04-25T08:39:57.109055+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109055+0000","last_undegraded":"2024-04-25T08:39:57.109055+0000","last_fullsized":"2024-04-25T08:39:57.109055+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106587+0000","last_change":"2024-04-25T08:39:57.106587+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106587+0000","last_undegraded":"2024-04-25T08:39:57.106587+0000","last_fullsized":"2024-04-25T08:39:57.106587+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109521+0000","last_change":"2024-04-25T08:39:57.109521+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109521+0000","last_undegraded":"2024-04-25T08:39:57.109521+0000","last_fullsized":"2024-04-25T08:39:57.109521+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111398+0000","last_change":"2024-04-25T08:39:57.111398+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111398+0000","last_undegraded":"2024-04-25T08:39:57.111398+0000","last_fullsized":"2024-04-25T08:39:57.111398+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110178+0000","last_change":"2024-04-25T08:39:57.110178+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110178+0000","last_undegraded":"2024-04-25T08:39:57.110178+0000","last_fullsized":"2024-04-25T08:39:57.110178+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109690+0000","last_change":"2024-04-25T08:39:57.109690+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109690+0000","last_undegraded":"2024-04-25T08:39:57.109690+0000","last_fullsized":"2024-04-25T08:39:57.109690+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.112046+0000","last_change":"2024-04-25T08:39:57.111533+0000","last_active":"2024-04-25T08:39:57.112046+0000","last_peered":"2024-04-25T08:39:57.112046+0000","last_clean":"2024-04-25T08:39:57.112046+0000","last_became_active":"2024-04-25T08:39:57.111229+0000","last_became_peered":"2024-04-25T08:39:57.111229+0000","last_unstale":"2024-04-25T08:39:57.112046+0000","last_undegraded":"2024-04-25T08:39:57.112046+0000","last_fullsized":"2024-04-25T08:39:57.112046+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108536+0000","last_change":"2024-04-25T08:39:57.108536+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108536+0000","last_undegraded":"2024-04-25T08:39:57.108536+0000","last_fullsized":"2024-04-25T08:39:57.108536+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120536+0000","last_change":"2024-04-25T08:39:57.117732+0000","last_active":"2024-04-25T08:39:57.120536+0000","last_peered":"2024-04-25T08:39:57.120536+0000","last_clean":"2024-04-25T08:39:57.120536+0000","last_became_active":"2024-04-25T08:39:57.117425+0000","last_became_peered":"2024-04-25T08:39:57.117425+0000","last_unstale":"2024-04-25T08:39:57.120536+0000","last_undegraded":"2024-04-25T08:39:57.120536+0000","last_fullsized":"2024-04-25T08:39:57.120536+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:40.880195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111346+0000","last_change":"2024-04-25T08:39:57.111346+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111346+0000","last_undegraded":"2024-04-25T08:39:57.111346+0000","last_fullsized":"2024-04-25T08:39:57.111346+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.111873+0000","last_change":"2024-04-25T08:39:57.110361+0000","last_active":"2024-04-25T08:39:57.111873+0000","last_peered":"2024-04-25T08:39:57.111873+0000","last_clean":"2024-04-25T08:39:57.111873+0000","last_became_active":"2024-04-25T08:39:57.110079+0000","last_became_peered":"2024-04-25T08:39:57.110079+0000","last_unstale":"2024-04-25T08:39:57.111873+0000","last_undegraded":"2024-04-25T08:39:57.111873+0000","last_fullsized":"2024-04-25T08:39:57.111873+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:55.191344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107451+0000","last_change":"2024-04-25T08:39:57.107451+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107451+0000","last_undegraded":"2024-04-25T08:39:57.107451+0000","last_fullsized":"2024-04-25T08:39:57.107451+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.122916+0000","last_change":"2024-04-25T08:39:57.122219+0000","last_active":"2024-04-25T08:39:57.122916+0000","last_peered":"2024-04-25T08:39:57.122916+0000","last_clean":"2024-04-25T08:39:57.122916+0000","last_became_active":"2024-04-25T08:39:57.121910+0000","last_became_peered":"2024-04-25T08:39:57.121910+0000","last_unstale":"2024-04-25T08:39:57.122916+0000","last_undegraded":"2024-04-25T08:39:57.122916+0000","last_fullsized":"2024-04-25T08:39:57.122916+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108782+0000","last_change":"2024-04-25T08:39:57.108782+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108782+0000","last_undegraded":"2024-04-25T08:39:57.108782+0000","last_fullsized":"2024-04-25T08:39:57.108782+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120411+0000","last_change":"2024-04-25T08:39:57.119082+0000","last_active":"2024-04-25T08:39:57.120411+0000","last_peered":"2024-04-25T08:39:57.120411+0000","last_clean":"2024-04-25T08:39:57.120411+0000","last_became_active":"2024-04-25T08:39:57.118794+0000","last_became_peered":"2024-04-25T08:39:57.118794+0000","last_unstale":"2024-04-25T08:39:57.120411+0000","last_undegraded":"2024-04-25T08:39:57.120411+0000","last_fullsized":"2024-04-25T08:39:57.120411+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105610+0000","last_change":"2024-04-25T08:39:57.105610+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105610+0000","last_undegraded":"2024-04-25T08:39:57.105610+0000","last_fullsized":"2024-04-25T08:39:57.105610+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.124563+0000","last_change":"2024-04-25T08:39:57.123981+0000","last_active":"2024-04-25T08:39:57.124563+0000","last_peered":"2024-04-25T08:39:57.124563+0000","last_clean":"2024-04-25T08:39:57.124563+0000","last_became_active":"2024-04-25T08:39:57.123631+0000","last_became_peered":"2024-04-25T08:39:57.123631+0000","last_unstale":"2024-04-25T08:39:57.124563+0000","last_undegraded":"2024-04-25T08:39:57.124563+0000","last_fullsized":"2024-04-25T08:39:57.124563+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:58.838892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107893+0000","last_change":"2024-04-25T08:39:57.107893+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107893+0000","last_undegraded":"2024-04-25T08:39:57.107893+0000","last_fullsized":"2024-04-25T08:39:57.107893+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.119811+0000","last_change":"2024-04-25T08:39:57.116831+0000","last_active":"2024-04-25T08:39:57.119811+0000","last_peered":"2024-04-25T08:39:57.119811+0000","last_clean":"2024-04-25T08:39:57.119811+0000","last_became_active":"2024-04-25T08:39:57.116598+0000","last_became_peered":"2024-04-25T08:39:57.116598+0000","last_unstale":"2024-04-25T08:39:57.119811+0000","last_undegraded":"2024-04-25T08:39:57.119811+0000","last_fullsized":"2024-04-25T08:39:57.119811+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:43.760739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.114648+0000","last_change":"2024-04-25T08:39:57.114648+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.114648+0000","last_undegraded":"2024-04-25T08:39:57.114648+0000","last_fullsized":"2024-04-25T08:39:57.114648+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.125099+0000","last_change":"2024-04-25T08:39:57.124496+0000","last_active":"2024-04-25T08:39:57.125099+0000","last_peered":"2024-04-25T08:39:57.125099+0000","last_clean":"2024-04-25T08:39:57.125099+0000","last_became_active":"2024-04-25T08:39:57.123329+0000","last_became_peered":"2024-04-25T08:39:57.123329+0000","last_unstale":"2024-04-25T08:39:57.125099+0000","last_undegraded":"2024-04-25T08:39:57.125099+0000","last_fullsized":"2024-04-25T08:39:57.125099+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:25.558741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104630+0000","last_change":"2024-04-25T08:39:57.104630+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104630+0000","last_undegraded":"2024-04-25T08:39:57.104630+0000","last_fullsized":"2024-04-25T08:39:57.104630+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117207+0000","last_change":"2024-04-25T08:39:57.115175+0000","last_active":"2024-04-25T08:39:57.117207+0000","last_peered":"2024-04-25T08:39:57.117207+0000","last_clean":"2024-04-25T08:39:57.117207+0000","last_became_active":"2024-04-25T08:39:57.114913+0000","last_became_peered":"2024-04-25T08:39:57.114913+0000","last_unstale":"2024-04-25T08:39:57.117207+0000","last_undegraded":"2024-04-25T08:39:57.117207+0000","last_fullsized":"2024-04-25T08:39:57.117207+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:41.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.113272+0000","last_change":"2024-04-25T08:39:57.113272+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.113272+0000","last_undegraded":"2024-04-25T08:39:57.113272+0000","last_fullsized":"2024-04-25T08:39:57.113272+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120816+0000","last_change":"2024-04-25T08:39:57.119480+0000","last_active":"2024-04-25T08:39:57.120816+0000","last_peered":"2024-04-25T08:39:57.120816+0000","last_clean":"2024-04-25T08:39:57.120816+0000","last_became_active":"2024-04-25T08:39:57.119195+0000","last_became_peered":"2024-04-25T08:39:57.119195+0000","last_unstale":"2024-04-25T08:39:57.120816+0000","last_undegraded":"2024-04-25T08:39:57.120816+0000","last_fullsized":"2024-04-25T08:39:57.120816+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106051+0000","last_change":"2024-04-25T08:39:57.106051+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106051+0000","last_undegraded":"2024-04-25T08:39:57.106051+0000","last_fullsized":"2024-04-25T08:39:57.106051+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120480+0000","last_change":"2024-04-25T08:39:57.118299+0000","last_active":"2024-04-25T08:39:57.120480+0000","last_peered":"2024-04-25T08:39:57.120480+0000","last_clean":"2024-04-25T08:39:57.120480+0000","last_became_active":"2024-04-25T08:39:57.117996+0000","last_became_peered":"2024-04-25T08:39:57.117996+0000","last_unstale":"2024-04-25T08:39:57.120480+0000","last_undegraded":"2024-04-25T08:39:57.120480+0000","last_fullsized":"2024-04-25T08:39:57.120480+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:40.880195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.112478+0000","last_change":"2024-04-25T08:39:57.112478+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.112478+0000","last_undegraded":"2024-04-25T08:39:57.112478+0000","last_fullsized":"2024-04-25T08:39:57.112478+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121442+0000","last_change":"2024-04-25T08:39:57.119869+0000","last_active":"2024-04-25T08:39:57.121442+0000","last_peered":"2024-04-25T08:39:57.121442+0000","last_clean":"2024-04-25T08:39:57.121442+0000","last_became_active":"2024-04-25T08:39:57.117564+0000","last_became_peered":"2024-04-25T08:39:57.117564+0000","last_unstale":"2024-04-25T08:39:57.121442+0000","last_undegraded":"2024-04-25T08:39:57.121442+0000","last_fullsized":"2024-04-25T08:39:57.121442+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:41.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105978+0000","last_change":"2024-04-25T08:39:57.105978+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105978+0000","last_undegraded":"2024-04-25T08:39:57.105978+0000","last_fullsized":"2024-04-25T08:39:57.105978+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117130+0000","last_change":"2024-04-25T08:39:57.111954+0000","last_active":"2024-04-25T08:39:57.117130+0000","last_peered":"2024-04-25T08:39:57.117130+0000","last_clean":"2024-04-25T08:39:57.117130+0000","last_became_active":"2024-04-25T08:39:57.111611+0000","last_became_peered":"2024-04-25T08:39:57.111611+0000","last_unstale":"2024-04-25T08:39:57.117130+0000","last_undegraded":"2024-04-25T08:39:57.117130+0000","last_fullsized":"2024-04-25T08:39:57.117130+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108899+0000","last_change":"2024-04-25T08:39:57.108899+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108899+0000","last_undegraded":"2024-04-25T08:39:57.108899+0000","last_fullsized":"2024-04-25T08:39:57.108899+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.124072+0000","last_change":"2024-04-25T08:39:57.122861+0000","last_active":"2024-04-25T08:39:57.124072+0000","last_peered":"2024-04-25T08:39:57.124072+0000","last_clean":"2024-04-25T08:39:57.124072+0000","last_became_active":"2024-04-25T08:39:57.122443+0000","last_became_peered":"2024-04-25T08:39:57.122443+0000","last_unstale":"2024-04-25T08:39:57.124072+0000","last_undegraded":"2024-04-25T08:39:57.124072+0000","last_fullsized":"2024-04-25T08:39:57.124072+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.134755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106645+0000","last_change":"2024-04-25T08:39:57.106645+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106645+0000","last_undegraded":"2024-04-25T08:39:57.106645+0000","last_fullsized":"2024-04-25T08:39:57.106645+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.123229+0000","last_change":"2024-04-25T08:39:57.122180+0000","last_active":"2024-04-25T08:39:57.123229+0000","last_peered":"2024-04-25T08:39:57.123229+0000","last_clean":"2024-04-25T08:39:57.123229+0000","last_became_active":"2024-04-25T08:39:57.121859+0000","last_became_peered":"2024-04-25T08:39:57.121859+0000","last_unstale":"2024-04-25T08:39:57.123229+0000","last_undegraded":"2024-04-25T08:39:57.123229+0000","last_fullsized":"2024-04-25T08:39:57.123229+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:56.067733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.116232+0000","last_change":"2024-04-25T08:39:57.116232+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.116232+0000","last_undegraded":"2024-04-25T08:39:57.116232+0000","last_fullsized":"2024-04-25T08:39:57.116232+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.125352+0000","last_change":"2024-04-25T08:39:57.124849+0000","last_active":"2024-04-25T08:39:57.125352+0000","last_peered":"2024-04-25T08:39:57.125352+0000","last_clean":"2024-04-25T08:39:57.125352+0000","last_became_active":"2024-04-25T08:39:57.124555+0000","last_became_peered":"2024-04-25T08:39:57.124555+0000","last_unstale":"2024-04-25T08:39:57.125352+0000","last_undegraded":"2024-04-25T08:39:57.125352+0000","last_fullsized":"2024-04-25T08:39:57.125352+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104557+0000","last_change":"2024-04-25T08:39:57.104557+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104557+0000","last_undegraded":"2024-04-25T08:39:57.104557+0000","last_fullsized":"2024-04-25T08:39:57.104557+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116939+0000","last_change":"2024-04-25T08:39:57.115173+0000","last_active":"2024-04-25T08:39:57.116939+0000","last_peered":"2024-04-25T08:39:57.116939+0000","last_clean":"2024-04-25T08:39:57.116939+0000","last_became_active":"2024-04-25T08:39:57.114870+0000","last_became_peered":"2024-04-25T08:39:57.114870+0000","last_unstale":"2024-04-25T08:39:57.116939+0000","last_undegraded":"2024-04-25T08:39:57.116939+0000","last_fullsized":"2024-04-25T08:39:57.116939+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:29.233744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107381+0000","last_change":"2024-04-25T08:39:57.107381+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107381+0000","last_undegraded":"2024-04-25T08:39:57.107381+0000","last_fullsized":"2024-04-25T08:39:57.107381+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.119844+0000","last_change":"2024-04-25T08:39:57.117206+0000","last_active":"2024-04-25T08:39:57.119844+0000","last_peered":"2024-04-25T08:39:57.119844+0000","last_clean":"2024-04-25T08:39:57.119844+0000","last_became_active":"2024-04-25T08:39:57.116938+0000","last_became_peered":"2024-04-25T08:39:57.116938+0000","last_unstale":"2024-04-25T08:39:57.119844+0000","last_undegraded":"2024-04-25T08:39:57.119844+0000","last_fullsized":"2024-04-25T08:39:57.119844+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:14.033329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111993+0000","last_change":"2024-04-25T08:39:57.111993+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111993+0000","last_undegraded":"2024-04-25T08:39:57.111993+0000","last_fullsized":"2024-04-25T08:39:57.111993+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116515+0000","last_change":"2024-04-25T08:39:57.114054+0000","last_active":"2024-04-25T08:39:57.116515+0000","last_peered":"2024-04-25T08:39:57.116515+0000","last_clean":"2024-04-25T08:39:57.116515+0000","last_became_active":"2024-04-25T08:39:57.113271+0000","last_became_peered":"2024-04-25T08:39:57.113271+0000","last_unstale":"2024-04-25T08:39:57.116515+0000","last_undegraded":"2024-04-25T08:39:57.116515+0000","last_fullsized":"2024-04-25T08:39:57.116515+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:25.305378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.113672+0000","last_change":"2024-04-25T08:39:57.113672+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.113672+0000","last_undegraded":"2024-04-25T08:39:57.113672+0000","last_fullsized":"2024-04-25T08:39:57.113672+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121882+0000","last_change":"2024-04-25T08:39:57.117517+0000","last_active":"2024-04-25T08:39:57.121882+0000","last_peered":"2024-04-25T08:39:57.121882+0000","last_clean":"2024-04-25T08:39:57.121882+0000","last_became_active":"2024-04-25T08:39:57.116811+0000","last_became_peered":"2024-04-25T08:39:57.116811+0000","last_unstale":"2024-04-25T08:39:57.121882+0000","last_undegraded":"2024-04-25T08:39:57.121882+0000","last_fullsized":"2024-04-25T08:39:57.121882+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:36.804780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110704+0000","last_change":"2024-04-25T08:39:57.110704+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110704+0000","last_undegraded":"2024-04-25T08:39:57.110704+0000","last_fullsized":"2024-04-25T08:39:57.110704+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.110041+0000","last_change":"2024-04-25T08:39:57.109255+0000","last_active":"2024-04-25T08:39:57.110041+0000","last_peered":"2024-04-25T08:39:57.110041+0000","last_clean":"2024-04-25T08:39:57.110041+0000","last_became_active":"2024-04-25T08:39:57.108956+0000","last_became_peered":"2024-04-25T08:39:57.108956+0000","last_unstale":"2024-04-25T08:39:57.110041+0000","last_undegraded":"2024-04-25T08:39:57.110041+0000","last_fullsized":"2024-04-25T08:39:57.110041+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:53.702495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107472+0000","last_change":"2024-04-25T08:39:57.107472+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107472+0000","last_undegraded":"2024-04-25T08:39:57.107472+0000","last_fullsized":"2024-04-25T08:39:57.107472+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121148+0000","last_change":"2024-04-25T08:39:57.118978+0000","last_active":"2024-04-25T08:39:57.121148+0000","last_peered":"2024-04-25T08:39:57.121148+0000","last_clean":"2024-04-25T08:39:57.121148+0000","last_became_active":"2024-04-25T08:39:57.118673+0000","last_became_peered":"2024-04-25T08:39:57.118673+0000","last_unstale":"2024-04-25T08:39:57.121148+0000","last_undegraded":"2024-04-25T08:39:57.121148+0000","last_fullsized":"2024-04-25T08:39:57.121148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111344+0000","last_change":"2024-04-25T08:39:57.111344+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111344+0000","last_undegraded":"2024-04-25T08:39:57.111344+0000","last_fullsized":"2024-04-25T08:39:57.111344+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116295+0000","last_change":"2024-04-25T08:39:57.114778+0000","last_active":"2024-04-25T08:39:57.116295+0000","last_peered":"2024-04-25T08:39:57.116295+0000","last_clean":"2024-04-25T08:39:57.116295+0000","last_became_active":"2024-04-25T08:39:57.114509+0000","last_became_peered":"2024-04-25T08:39:57.114509+0000","last_unstale":"2024-04-25T08:39:57.116295+0000","last_undegraded":"2024-04-25T08:39:57.116295+0000","last_fullsized":"2024-04-25T08:39:57.116295+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:06.569207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108631+0000","last_change":"2024-04-25T08:39:57.108631+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108631+0000","last_undegraded":"2024-04-25T08:39:57.108631+0000","last_fullsized":"2024-04-25T08:39:57.108631+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117436+0000","last_change":"2024-04-25T08:39:57.113886+0000","last_active":"2024-04-25T08:39:57.117436+0000","last_peered":"2024-04-25T08:39:57.117436+0000","last_clean":"2024-04-25T08:39:57.117436+0000","last_became_active":"2024-04-25T08:39:57.113199+0000","last_became_peered":"2024-04-25T08:39:57.113199+0000","last_unstale":"2024-04-25T08:39:57.117436+0000","last_undegraded":"2024-04-25T08:39:57.117436+0000","last_fullsized":"2024-04-25T08:39:57.117436+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:17.991774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111394+0000","last_change":"2024-04-25T08:39:57.111394+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111394+0000","last_undegraded":"2024-04-25T08:39:57.111394+0000","last_fullsized":"2024-04-25T08:39:57.111394+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120427+0000","last_change":"2024-04-25T08:39:57.117731+0000","last_active":"2024-04-25T08:39:57.120427+0000","last_peered":"2024-04-25T08:39:57.120427+0000","last_clean":"2024-04-25T08:39:57.120427+0000","last_became_active":"2024-04-25T08:39:57.117423+0000","last_became_peered":"2024-04-25T08:39:57.117423+0000","last_unstale":"2024-04-25T08:39:57.120427+0000","last_undegraded":"2024-04-25T08:39:57.120427+0000","last_fullsized":"2024-04-25T08:39:57.120427+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:56.067733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105610+0000","last_change":"2024-04-25T08:39:57.105610+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105610+0000","last_undegraded":"2024-04-25T08:39:57.105610+0000","last_fullsized":"2024-04-25T08:39:57.105610+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116616+0000","last_change":"2024-04-25T08:39:57.115100+0000","last_active":"2024-04-25T08:39:57.116616+0000","last_peered":"2024-04-25T08:39:57.116616+0000","last_clean":"2024-04-25T08:39:57.116616+0000","last_became_active":"2024-04-25T08:39:57.114830+0000","last_became_peered":"2024-04-25T08:39:57.114830+0000","last_unstale":"2024-04-25T08:39:57.116616+0000","last_undegraded":"2024-04-25T08:39:57.116616+0000","last_fullsized":"2024-04-25T08:39:57.116616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106488+0000","last_change":"2024-04-25T08:39:57.106488+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106488+0000","last_undegraded":"2024-04-25T08:39:57.106488+0000","last_fullsized":"2024-04-25T08:39:57.106488+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.123505+0000","last_change":"2024-04-25T08:39:57.120468+0000","last_active":"2024-04-25T08:39:57.123505+0000","last_peered":"2024-04-25T08:39:57.123505+0000","last_clean":"2024-04-25T08:39:57.123505+0000","last_became_active":"2024-04-25T08:39:57.119296+0000","last_became_peered":"2024-04-25T08:39:57.119296+0000","last_unstale":"2024-04-25T08:39:57.123505+0000","last_undegraded":"2024-04-25T08:39:57.123505+0000","last_fullsized":"2024-04-25T08:39:57.123505+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:07.703556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110350+0000","last_change":"2024-04-25T08:39:57.110350+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110350+0000","last_undegraded":"2024-04-25T08:39:57.110350+0000","last_fullsized":"2024-04-25T08:39:57.110350+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117144+0000","last_change":"2024-04-25T08:39:57.113751+0000","last_active":"2024-04-25T08:39:57.117144+0000","last_peered":"2024-04-25T08:39:57.117144+0000","last_clean":"2024-04-25T08:39:57.117144+0000","last_became_active":"2024-04-25T08:39:57.113479+0000","last_became_peered":"2024-04-25T08:39:57.113479+0000","last_unstale":"2024-04-25T08:39:57.117144+0000","last_undegraded":"2024-04-25T08:39:57.117144+0000","last_fullsized":"2024-04-25T08:39:57.117144+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108211+0000","last_change":"2024-04-25T08:39:57.108211+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108211+0000","last_undegraded":"2024-04-25T08:39:57.108211+0000","last_fullsized":"2024-04-25T08:39:57.108211+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117175+0000","last_change":"2024-04-25T08:39:57.114636+0000","last_active":"2024-04-25T08:39:57.117175+0000","last_peered":"2024-04-25T08:39:57.117175+0000","last_clean":"2024-04-25T08:39:57.117175+0000","last_became_active":"2024-04-25T08:39:57.114366+0000","last_became_peered":"2024-04-25T08:39:57.114366+0000","last_unstale":"2024-04-25T08:39:57.117175+0000","last_undegraded":"2024-04-25T08:39:57.117175+0000","last_fullsized":"2024-04-25T08:39:57.117175+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104891+0000","last_change":"2024-04-25T08:39:57.104891+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104891+0000","last_undegraded":"2024-04-25T08:39:57.104891+0000","last_fullsized":"2024-04-25T08:39:57.104891+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.123503+0000","last_change":"2024-04-25T08:39:57.119185+0000","last_active":"2024-04-25T08:39:57.123503+0000","last_peered":"2024-04-25T08:39:57.123503+0000","last_clean":"2024-04-25T08:39:57.123503+0000","last_became_active":"2024-04-25T08:39:57.118135+0000","last_became_peered":"2024-04-25T08:39:57.118135+0000","last_unstale":"2024-04-25T08:39:57.123503+0000","last_undegraded":"2024-04-25T08:39:57.123503+0000","last_fullsized":"2024-04-25T08:39:57.123503+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:01.150116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.114248+0000","last_change":"2024-04-25T08:39:57.114248+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.114248+0000","last_undegraded":"2024-04-25T08:39:57.114248+0000","last_fullsized":"2024-04-25T08:39:57.114248+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.113748+0000","last_change":"2024-04-25T08:39:57.111684+0000","last_active":"2024-04-25T08:39:57.113748+0000","last_peered":"2024-04-25T08:39:57.113748+0000","last_clean":"2024-04-25T08:39:57.113748+0000","last_became_active":"2024-04-25T08:39:57.111394+0000","last_became_peered":"2024-04-25T08:39:57.111394+0000","last_unstale":"2024-04-25T08:39:57.113748+0000","last_undegraded":"2024-04-25T08:39:57.113748+0000","last_fullsized":"2024-04-25T08:39:57.113748+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:06.258822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106294+0000","last_change":"2024-04-25T08:39:57.106294+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106294+0000","last_undegraded":"2024-04-25T08:39:57.106294+0000","last_fullsized":"2024-04-25T08:39:57.106294+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121216+0000","last_change":"2024-04-25T08:39:57.118637+0000","last_active":"2024-04-25T08:39:57.121216+0000","last_peered":"2024-04-25T08:39:57.121216+0000","last_clean":"2024-04-25T08:39:57.121216+0000","last_became_active":"2024-04-25T08:39:57.118325+0000","last_became_peered":"2024-04-25T08:39:57.118325+0000","last_unstale":"2024-04-25T08:39:57.121216+0000","last_undegraded":"2024-04-25T08:39:57.121216+0000","last_fullsized":"2024-04-25T08:39:57.121216+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":356,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.102580+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:39:57.102580+0000","last_peered":"2024-04-25T08:39:57.102580+0000","last_clean":"2024-04-25T08:39:57.102580+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:39:57.102580+0000","last_undegraded":"2024-04-25T08:39:57.102580+0000","last_fullsized":"2024-04-25T08:39:57.102580+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77616,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294224,"statfs":{"total":96636764160,"available":96557285376,"internally_reserved":0,"allocated":2064384,"data_stored":1273646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59000,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312840,"statfs":{"total":96636764160,"available":96576348160,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54824,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317016,"statfs":{"total":96636764160,"available":96580624384,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68724,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303116,"statfs":{"total":96636764160,"available":96566390784,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72880,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298960,"statfs":{"total":96636764160,"available":96562135040,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58984,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312856,"statfs":{"total":96636764160,"available":96576364544,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54828,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317012,"statfs":{"total":96636764160,"available":96580620288,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51248,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320592,"statfs":{"total":96636764160,"available":96584286208,"internally_reserved":0,"allocated":2064384,"data_stored":1273646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:01.127 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:01.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.270+0000 7f6b89968640 1 Processor -- start 2024-04-25T08:40:01.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.270+0000 7f6b89968640 1 -- start start 2024-04-25T08:40:01.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.271+0000 7f6b89968640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6b840ceca0 0x7f6b840d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.271+0000 7f6b89968640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b840d3de0 0x7f6b840b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.271+0000 7f6b89968640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b840d1b60 con 0x7f6b8415da90 2024-04-25T08:40:01.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.271+0000 7f6b89968640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b840d4250 con 0x7f6b840ceca0 2024-04-25T08:40:01.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.271+0000 7f6b89968640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840d2bc0 con 0x7f6b840d3de0 2024-04-25T08:40:01.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.271+0000 7f6b837fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b840d3de0 0x7f6b840b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.271+0000 7f6b827fc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6b840ceca0 0x7f6b840d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.272+0000 7f6b82ffd640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f6b8415da90 0x7f6b840d2f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:48942/0 (socket says 172.21.15.38:48942) 2024-04-25T08:40:01.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.272+0000 7f6b82ffd640 1 -- 172.21.15.38:0/1359543563 learned_addr learned my addr 172.21.15.38:0/1359543563 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:01.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.272+0000 7f6b837fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b840d3de0 0x7f6b840b69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:42716/0 (socket says 172.21.15.38:42716) 2024-04-25T08:40:01.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.272+0000 7f6b827fc640 1 -- 172.21.15.38:0/1359543563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b840d3de0 msgr2=0x7f6b840b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:01.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.272+0000 7f6b827fc640 1 --2- 172.21.15.38:0/1359543563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b840d3de0 0x7f6b840b69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.272+0000 7f6b827fc640 1 -- 172.21.15.38:0/1359543563 >> v1:172.21.15.107:6789/0 conn(0x7f6b8415da90 legacy=0x7f6b840d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.272+0000 7f6b827fc640 1 -- 172.21.15.38:0/1359543563 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b840d3680 con 0x7f6b840ceca0 2024-04-25T08:40:01.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.272+0000 7f6b81ffb640 1 -- 172.21.15.38:0/1359543563 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371890082 0 0) 0x7f6b840d1b60 con 0x7f6b8415da90 2024-04-25T08:40:01.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.272+0000 7f6b837fe640 1 --2- 172.21.15.38:0/1359543563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b840d3de0 0x7f6b840b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:01.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.273+0000 7f6b827fc640 1 --2- 172.21.15.38:0/1359543563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840ceca0 0x7f6b840d38a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097a0 tx=0x7f6b6c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24fba1f4749d9769 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:01.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.273+0000 7f6b81ffb640 1 -- 172.21.15.38:0/1359543563 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c037070 con 0x7f6b840ceca0 2024-04-25T08:40:01.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.273+0000 7f6b81ffb640 1 -- 172.21.15.38:0/1359543563 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c032440 con 0x7f6b840ceca0 2024-04-25T08:40:01.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.273+0000 7f6b81ffb640 1 -- 172.21.15.38:0/1359543563 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c032810 con 0x7f6b840ceca0 2024-04-25T08:40:01.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.273+0000 7f6b89968640 1 -- 172.21.15.38:0/1359543563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840ceca0 msgr2=0x7f6b840d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.273+0000 7f6b89968640 1 --2- 172.21.15.38:0/1359543563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840ceca0 0x7f6b840d38a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0097a0 tx=0x7f6b6c02a180 comp rx=0 tx=0).stop 2024-04-25T08:40:01.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.273+0000 7f6b89968640 1 -- 172.21.15.38:0/1359543563 shutdown_connections 2024-04-25T08:40:01.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.273+0000 7f6b89968640 1 --2- 172.21.15.38:0/1359543563 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b840d3de0 0x7f6b840b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.273+0000 7f6b89968640 1 --2- 172.21.15.38:0/1359543563 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b840ceca0 0x7f6b840d38a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.273+0000 7f6b89968640 1 -- 172.21.15.38:0/1359543563 >> 172.21.15.38:0/1359543563 conn(0x7f6b8406bed0 msgr2=0x7f6b8406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:01.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.273+0000 7f6b89968640 1 -- 172.21.15.38:0/1359543563 shutdown_connections 2024-04-25T08:40:01.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.274+0000 7f6b89968640 1 -- 172.21.15.38:0/1359543563 wait complete. 2024-04-25T08:40:01.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.275+0000 7f6b89968640 1 Processor -- start 2024-04-25T08:40:01.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.275+0000 7f6b89968640 1 -- start start 2024-04-25T08:40:01.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.275+0000 7f6b89968640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b840d3de0 0x7f6b8415fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.275+0000 7f6b89968640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8415da90 0x7f6b8415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.275+0000 7f6b89968640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b84155df0 0x7f6b8415e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b89968640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840d23c0 con 0x7f6b84155df0 2024-04-25T08:40:01.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b89968640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b840d20e0 con 0x7f6b8415da90 2024-04-25T08:40:01.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b89968640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8415de70 con 0x7f6b840d3de0 2024-04-25T08:40:01.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b84155df0 0x7f6b8415e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b84155df0 0x7f6b8415e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44494/0 (socket says 172.21.15.38:44494) 2024-04-25T08:40:01.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b837fe640 1 -- 172.21.15.38:0/3585355384 learned_addr learned my addr 172.21.15.38:0/3585355384 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:01.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b82ffd640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b840d3de0 0x7f6b8415fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b827fc640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8415da90 0x7f6b8415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b837fe640 1 -- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b840d3de0 msgr2=0x7f6b8415fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b837fe640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b840d3de0 0x7f6b8415fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b837fe640 1 -- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8415da90 msgr2=0x7f6b8415b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b837fe640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8415da90 0x7f6b8415b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b837fe640 1 -- 172.21.15.38:0/3585355384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b84160250 con 0x7f6b84155df0 2024-04-25T08:40:01.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b827fc640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8415da90 0x7f6b8415b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:01.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b837fe640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b84155df0 0x7f6b8415e170 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400efc0 tx=0x7f6b74009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:01.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b7bfff640 1 -- 172.21.15.38:0/3585355384 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b74018030 con 0x7f6b84155df0 2024-04-25T08:40:01.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b7bfff640 1 -- 172.21.15.38:0/3585355384 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7400cde0 con 0x7f6b84155df0 2024-04-25T08:40:01.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b89968640 1 -- 172.21.15.38:0/3585355384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b84160530 con 0x7f6b84155df0 2024-04-25T08:40:01.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b7bfff640 1 -- 172.21.15.38:0/3585355384 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b74003bb0 con 0x7f6b84155df0 2024-04-25T08:40:01.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.276+0000 7f6b89968640 1 -- 172.21.15.38:0/3585355384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b840c7880 con 0x7f6b84155df0 2024-04-25T08:40:01.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.277+0000 7f6b89968640 1 -- 172.21.15.38:0/3585355384 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b84068e80 con 0x7f6b84155df0 2024-04-25T08:40:01.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.278+0000 7f6b7bfff640 1 -- 172.21.15.38:0/3585355384 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b7401b070 con 0x7f6b84155df0 2024-04-25T08:40:01.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.278+0000 7f6b7bfff640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b6406b4d0 0x7f6b6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.278+0000 7f6b7bfff640 1 -- 172.21.15.38:0/3585355384 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6b7408aed0 con 0x7f6b84155df0 2024-04-25T08:40:01.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.279+0000 7f6b82ffd640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b6406b4d0 0x7f6b6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.279+0000 7f6b82ffd640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b6406b4d0 0x7f6b6406d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c000f60 tx=0x7f6b7c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:01.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.282+0000 7f6b7bfff640 1 -- 172.21.15.38:0/3585355384 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b74010040 con 0x7f6b84155df0 2024-04-25T08:40:01.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.457+0000 7f6b89968640 1 -- 172.21.15.38:0/3585355384 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b840ca750 con 0x7f6b6406b4d0 2024-04-25T08:40:01.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.464+0000 7f6b7bfff640 1 -- 172.21.15.38:0/3585355384 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7f6b840ca750 con 0x7f6b6406b4d0 2024-04-25T08:40:01.460 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:01.461 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:40:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.467+0000 7f6b89968640 1 -- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b6406b4d0 msgr2=0x7f6b6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.467+0000 7f6b89968640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b6406b4d0 0x7f6b6406d990 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c000f60 tx=0x7f6b7c009040 comp rx=0 tx=0).stop 2024-04-25T08:40:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.467+0000 7f6b89968640 1 -- 172.21.15.38:0/3585355384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b84155df0 msgr2=0x7f6b8415e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.467+0000 7f6b89968640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b84155df0 0x7f6b8415e170 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400efc0 tx=0x7f6b74009f90 comp rx=0 tx=0).stop 2024-04-25T08:40:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.467+0000 7f6b89968640 1 -- 172.21.15.38:0/3585355384 shutdown_connections 2024-04-25T08:40:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.467+0000 7f6b89968640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b84155df0 0x7f6b8415e170 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.467+0000 7f6b89968640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b8415da90 0x7f6b8415b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.467+0000 7f6b89968640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b6406b4d0 0x7f6b6406d990 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.467+0000 7f6b89968640 1 --2- 172.21.15.38:0/3585355384 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b840d3de0 0x7f6b8415fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.467+0000 7f6b89968640 1 -- 172.21.15.38:0/3585355384 >> 172.21.15.38:0/3585355384 conn(0x7f6b8406bed0 msgr2=0x7f6b840d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.467+0000 7f6b89968640 1 -- 172.21.15.38:0/3585355384 shutdown_connections 2024-04-25T08:40:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.467+0000 7f6b89968640 1 -- 172.21.15.38:0/3585355384 wait complete. 2024-04-25T08:40:01.471 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-04-25T08:40:01.323124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":498104,"kb_used_data":12648,"kb_used_omap":12,"kb_used_meta":485299,"kb_avail":754476616,"statfs":{"total":773094113280,"available":772584054784,"internally_reserved":0,"allocated":12951552,"data_stored":6646960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":496946736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.223817"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104507+0000","last_change":"2024-04-25T08:39:57.104507+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104507+0000","last_undegraded":"2024-04-25T08:39:57.104507+0000","last_fullsized":"2024-04-25T08:39:57.104507+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108915+0000","last_change":"2024-04-25T08:39:57.108915+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108915+0000","last_undegraded":"2024-04-25T08:39:57.108915+0000","last_fullsized":"2024-04-25T08:39:57.108915+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105895+0000","last_change":"2024-04-25T08:39:57.105895+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105895+0000","last_undegraded":"2024-04-25T08:39:57.105895+0000","last_fullsized":"2024-04-25T08:39:57.105895+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109343+0000","last_change":"2024-04-25T08:39:57.109343+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109343+0000","last_undegraded":"2024-04-25T08:39:57.109343+0000","last_fullsized":"2024-04-25T08:39:57.109343+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105957+0000","last_change":"2024-04-25T08:39:57.105957+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105957+0000","last_undegraded":"2024-04-25T08:39:57.105957+0000","last_fullsized":"2024-04-25T08:39:57.105957+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107903+0000","last_change":"2024-04-25T08:39:57.107903+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107903+0000","last_undegraded":"2024-04-25T08:39:57.107903+0000","last_fullsized":"2024-04-25T08:39:57.107903+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109081+0000","last_change":"2024-04-25T08:39:57.109081+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109081+0000","last_undegraded":"2024-04-25T08:39:57.109081+0000","last_fullsized":"2024-04-25T08:39:57.109081+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106732+0000","last_change":"2024-04-25T08:39:57.106732+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106732+0000","last_undegraded":"2024-04-25T08:39:57.106732+0000","last_fullsized":"2024-04-25T08:39:57.106732+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108726+0000","last_change":"2024-04-25T08:39:57.108726+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108726+0000","last_undegraded":"2024-04-25T08:39:57.108726+0000","last_fullsized":"2024-04-25T08:39:57.108726+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110497+0000","last_change":"2024-04-25T08:39:57.110497+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110497+0000","last_undegraded":"2024-04-25T08:39:57.110497+0000","last_fullsized":"2024-04-25T08:39:57.110497+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105137+0000","last_change":"2024-04-25T08:39:57.105137+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105137+0000","last_undegraded":"2024-04-25T08:39:57.105137+0000","last_fullsized":"2024-04-25T08:39:57.105137+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109567+0000","last_change":"2024-04-25T08:39:57.109567+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109567+0000","last_undegraded":"2024-04-25T08:39:57.109567+0000","last_fullsized":"2024-04-25T08:39:57.109567+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.115061+0000","last_change":"2024-04-25T08:39:57.115061+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.115061+0000","last_undegraded":"2024-04-25T08:39:57.115061+0000","last_fullsized":"2024-04-25T08:39:57.115061+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109950+0000","last_change":"2024-04-25T08:39:57.109950+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109950+0000","last_undegraded":"2024-04-25T08:39:57.109950+0000","last_fullsized":"2024-04-25T08:39:57.109950+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104679+0000","last_change":"2024-04-25T08:39:57.104679+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104679+0000","last_undegraded":"2024-04-25T08:39:57.104679+0000","last_fullsized":"2024-04-25T08:39:57.104679+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109887+0000","last_change":"2024-04-25T08:39:57.109887+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109887+0000","last_undegraded":"2024-04-25T08:39:57.109887+0000","last_fullsized":"2024-04-25T08:39:57.109887+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.112277+0000","last_change":"2024-04-25T08:39:57.112277+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.112277+0000","last_undegraded":"2024-04-25T08:39:57.112277+0000","last_fullsized":"2024-04-25T08:39:57.112277+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108487+0000","last_change":"2024-04-25T08:39:57.108487+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108487+0000","last_undegraded":"2024-04-25T08:39:57.108487+0000","last_fullsized":"2024-04-25T08:39:57.108487+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107714+0000","last_change":"2024-04-25T08:39:57.107714+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107714+0000","last_undegraded":"2024-04-25T08:39:57.107714+0000","last_fullsized":"2024-04-25T08:39:57.107714+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105862+0000","last_change":"2024-04-25T08:39:57.105862+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105862+0000","last_undegraded":"2024-04-25T08:39:57.105862+0000","last_fullsized":"2024-04-25T08:39:57.105862+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108441+0000","last_change":"2024-04-25T08:39:57.108441+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108441+0000","last_undegraded":"2024-04-25T08:39:57.108441+0000","last_fullsized":"2024-04-25T08:39:57.108441+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105711+0000","last_change":"2024-04-25T08:39:57.105711+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105711+0000","last_undegraded":"2024-04-25T08:39:57.105711+0000","last_fullsized":"2024-04-25T08:39:57.105711+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107927+0000","last_change":"2024-04-25T08:39:57.107927+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107927+0000","last_undegraded":"2024-04-25T08:39:57.107927+0000","last_fullsized":"2024-04-25T08:39:57.107927+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111019+0000","last_change":"2024-04-25T08:39:57.111019+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111019+0000","last_undegraded":"2024-04-25T08:39:57.111019+0000","last_fullsized":"2024-04-25T08:39:57.111019+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.112909+0000","last_change":"2024-04-25T08:39:57.112909+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.112909+0000","last_undegraded":"2024-04-25T08:39:57.112909+0000","last_fullsized":"2024-04-25T08:39:57.112909+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106551+0000","last_change":"2024-04-25T08:39:57.106551+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106551+0000","last_undegraded":"2024-04-25T08:39:57.106551+0000","last_fullsized":"2024-04-25T08:39:57.106551+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106831+0000","last_change":"2024-04-25T08:39:57.106831+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106831+0000","last_undegraded":"2024-04-25T08:39:57.106831+0000","last_fullsized":"2024-04-25T08:39:57.106831+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109055+0000","last_change":"2024-04-25T08:39:57.109055+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109055+0000","last_undegraded":"2024-04-25T08:39:57.109055+0000","last_fullsized":"2024-04-25T08:39:57.109055+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106587+0000","last_change":"2024-04-25T08:39:57.106587+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106587+0000","last_undegraded":"2024-04-25T08:39:57.106587+0000","last_fullsized":"2024-04-25T08:39:57.106587+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109521+0000","last_change":"2024-04-25T08:39:57.109521+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109521+0000","last_undegraded":"2024-04-25T08:39:57.109521+0000","last_fullsized":"2024-04-25T08:39:57.109521+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111398+0000","last_change":"2024-04-25T08:39:57.111398+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111398+0000","last_undegraded":"2024-04-25T08:39:57.111398+0000","last_fullsized":"2024-04-25T08:39:57.111398+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110178+0000","last_change":"2024-04-25T08:39:57.110178+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110178+0000","last_undegraded":"2024-04-25T08:39:57.110178+0000","last_fullsized":"2024-04-25T08:39:57.110178+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109690+0000","last_change":"2024-04-25T08:39:57.109690+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109690+0000","last_undegraded":"2024-04-25T08:39:57.109690+0000","last_fullsized":"2024-04-25T08:39:57.109690+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.112046+0000","last_change":"2024-04-25T08:39:57.111533+0000","last_active":"2024-04-25T08:39:57.112046+0000","last_peered":"2024-04-25T08:39:57.112046+0000","last_clean":"2024-04-25T08:39:57.112046+0000","last_became_active":"2024-04-25T08:39:57.111229+0000","last_became_peered":"2024-04-25T08:39:57.111229+0000","last_unstale":"2024-04-25T08:39:57.112046+0000","last_undegraded":"2024-04-25T08:39:57.112046+0000","last_fullsized":"2024-04-25T08:39:57.112046+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108536+0000","last_change":"2024-04-25T08:39:57.108536+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108536+0000","last_undegraded":"2024-04-25T08:39:57.108536+0000","last_fullsized":"2024-04-25T08:39:57.108536+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120536+0000","last_change":"2024-04-25T08:39:57.117732+0000","last_active":"2024-04-25T08:39:57.120536+0000","last_peered":"2024-04-25T08:39:57.120536+0000","last_clean":"2024-04-25T08:39:57.120536+0000","last_became_active":"2024-04-25T08:39:57.117425+0000","last_became_peered":"2024-04-25T08:39:57.117425+0000","last_unstale":"2024-04-25T08:39:57.120536+0000","last_undegraded":"2024-04-25T08:39:57.120536+0000","last_fullsized":"2024-04-25T08:39:57.120536+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:40.880195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111346+0000","last_change":"2024-04-25T08:39:57.111346+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111346+0000","last_undegraded":"2024-04-25T08:39:57.111346+0000","last_fullsized":"2024-04-25T08:39:57.111346+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.111873+0000","last_change":"2024-04-25T08:39:57.110361+0000","last_active":"2024-04-25T08:39:57.111873+0000","last_peered":"2024-04-25T08:39:57.111873+0000","last_clean":"2024-04-25T08:39:57.111873+0000","last_became_active":"2024-04-25T08:39:57.110079+0000","last_became_peered":"2024-04-25T08:39:57.110079+0000","last_unstale":"2024-04-25T08:39:57.111873+0000","last_undegraded":"2024-04-25T08:39:57.111873+0000","last_fullsized":"2024-04-25T08:39:57.111873+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:55.191344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107451+0000","last_change":"2024-04-25T08:39:57.107451+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107451+0000","last_undegraded":"2024-04-25T08:39:57.107451+0000","last_fullsized":"2024-04-25T08:39:57.107451+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.122916+0000","last_change":"2024-04-25T08:39:57.122219+0000","last_active":"2024-04-25T08:39:57.122916+0000","last_peered":"2024-04-25T08:39:57.122916+0000","last_clean":"2024-04-25T08:39:57.122916+0000","last_became_active":"2024-04-25T08:39:57.121910+0000","last_became_peered":"2024-04-25T08:39:57.121910+0000","last_unstale":"2024-04-25T08:39:57.122916+0000","last_undegraded":"2024-04-25T08:39:57.122916+0000","last_fullsized":"2024-04-25T08:39:57.122916+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108782+0000","last_change":"2024-04-25T08:39:57.108782+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108782+0000","last_undegraded":"2024-04-25T08:39:57.108782+0000","last_fullsized":"2024-04-25T08:39:57.108782+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120411+0000","last_change":"2024-04-25T08:39:57.119082+0000","last_active":"2024-04-25T08:39:57.120411+0000","last_peered":"2024-04-25T08:39:57.120411+0000","last_clean":"2024-04-25T08:39:57.120411+0000","last_became_active":"2024-04-25T08:39:57.118794+0000","last_became_peered":"2024-04-25T08:39:57.118794+0000","last_unstale":"2024-04-25T08:39:57.120411+0000","last_undegraded":"2024-04-25T08:39:57.120411+0000","last_fullsized":"2024-04-25T08:39:57.120411+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105610+0000","last_change":"2024-04-25T08:39:57.105610+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105610+0000","last_undegraded":"2024-04-25T08:39:57.105610+0000","last_fullsized":"2024-04-25T08:39:57.105610+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.124563+0000","last_change":"2024-04-25T08:39:57.123981+0000","last_active":"2024-04-25T08:39:57.124563+0000","last_peered":"2024-04-25T08:39:57.124563+0000","last_clean":"2024-04-25T08:39:57.124563+0000","last_became_active":"2024-04-25T08:39:57.123631+0000","last_became_peered":"2024-04-25T08:39:57.123631+0000","last_unstale":"2024-04-25T08:39:57.124563+0000","last_undegraded":"2024-04-25T08:39:57.124563+0000","last_fullsized":"2024-04-25T08:39:57.124563+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:58.838892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107893+0000","last_change":"2024-04-25T08:39:57.107893+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107893+0000","last_undegraded":"2024-04-25T08:39:57.107893+0000","last_fullsized":"2024-04-25T08:39:57.107893+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.119811+0000","last_change":"2024-04-25T08:39:57.116831+0000","last_active":"2024-04-25T08:39:57.119811+0000","last_peered":"2024-04-25T08:39:57.119811+0000","last_clean":"2024-04-25T08:39:57.119811+0000","last_became_active":"2024-04-25T08:39:57.116598+0000","last_became_peered":"2024-04-25T08:39:57.116598+0000","last_unstale":"2024-04-25T08:39:57.119811+0000","last_undegraded":"2024-04-25T08:39:57.119811+0000","last_fullsized":"2024-04-25T08:39:57.119811+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:43.760739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.114648+0000","last_change":"2024-04-25T08:39:57.114648+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.114648+0000","last_undegraded":"2024-04-25T08:39:57.114648+0000","last_fullsized":"2024-04-25T08:39:57.114648+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.125099+0000","last_change":"2024-04-25T08:39:57.124496+0000","last_active":"2024-04-25T08:39:57.125099+0000","last_peered":"2024-04-25T08:39:57.125099+0000","last_clean":"2024-04-25T08:39:57.125099+0000","last_became_active":"2024-04-25T08:39:57.123329+0000","last_became_peered":"2024-04-25T08:39:57.123329+0000","last_unstale":"2024-04-25T08:39:57.125099+0000","last_undegraded":"2024-04-25T08:39:57.125099+0000","last_fullsized":"2024-04-25T08:39:57.125099+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:25.558741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104630+0000","last_change":"2024-04-25T08:39:57.104630+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104630+0000","last_undegraded":"2024-04-25T08:39:57.104630+0000","last_fullsized":"2024-04-25T08:39:57.104630+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117207+0000","last_change":"2024-04-25T08:39:57.115175+0000","last_active":"2024-04-25T08:39:57.117207+0000","last_peered":"2024-04-25T08:39:57.117207+0000","last_clean":"2024-04-25T08:39:57.117207+0000","last_became_active":"2024-04-25T08:39:57.114913+0000","last_became_peered":"2024-04-25T08:39:57.114913+0000","last_unstale":"2024-04-25T08:39:57.117207+0000","last_undegraded":"2024-04-25T08:39:57.117207+0000","last_fullsized":"2024-04-25T08:39:57.117207+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:41.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.113272+0000","last_change":"2024-04-25T08:39:57.113272+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.113272+0000","last_undegraded":"2024-04-25T08:39:57.113272+0000","last_fullsized":"2024-04-25T08:39:57.113272+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120816+0000","last_change":"2024-04-25T08:39:57.119480+0000","last_active":"2024-04-25T08:39:57.120816+0000","last_peered":"2024-04-25T08:39:57.120816+0000","last_clean":"2024-04-25T08:39:57.120816+0000","last_became_active":"2024-04-25T08:39:57.119195+0000","last_became_peered":"2024-04-25T08:39:57.119195+0000","last_unstale":"2024-04-25T08:39:57.120816+0000","last_undegraded":"2024-04-25T08:39:57.120816+0000","last_fullsized":"2024-04-25T08:39:57.120816+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106051+0000","last_change":"2024-04-25T08:39:57.106051+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106051+0000","last_undegraded":"2024-04-25T08:39:57.106051+0000","last_fullsized":"2024-04-25T08:39:57.106051+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120480+0000","last_change":"2024-04-25T08:39:57.118299+0000","last_active":"2024-04-25T08:39:57.120480+0000","last_peered":"2024-04-25T08:39:57.120480+0000","last_clean":"2024-04-25T08:39:57.120480+0000","last_became_active":"2024-04-25T08:39:57.117996+0000","last_became_peered":"2024-04-25T08:39:57.117996+0000","last_unstale":"2024-04-25T08:39:57.120480+0000","last_undegraded":"2024-04-25T08:39:57.120480+0000","last_fullsized":"2024-04-25T08:39:57.120480+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:40.880195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.112478+0000","last_change":"2024-04-25T08:39:57.112478+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.112478+0000","last_undegraded":"2024-04-25T08:39:57.112478+0000","last_fullsized":"2024-04-25T08:39:57.112478+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121442+0000","last_change":"2024-04-25T08:39:57.119869+0000","last_active":"2024-04-25T08:39:57.121442+0000","last_peered":"2024-04-25T08:39:57.121442+0000","last_clean":"2024-04-25T08:39:57.121442+0000","last_became_active":"2024-04-25T08:39:57.117564+0000","last_became_peered":"2024-04-25T08:39:57.117564+0000","last_unstale":"2024-04-25T08:39:57.121442+0000","last_undegraded":"2024-04-25T08:39:57.121442+0000","last_fullsized":"2024-04-25T08:39:57.121442+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:41.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105978+0000","last_change":"2024-04-25T08:39:57.105978+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105978+0000","last_undegraded":"2024-04-25T08:39:57.105978+0000","last_fullsized":"2024-04-25T08:39:57.105978+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117130+0000","last_change":"2024-04-25T08:39:57.111954+0000","last_active":"2024-04-25T08:39:57.117130+0000","last_peered":"2024-04-25T08:39:57.117130+0000","last_clean":"2024-04-25T08:39:57.117130+0000","last_became_active":"2024-04-25T08:39:57.111611+0000","last_became_peered":"2024-04-25T08:39:57.111611+0000","last_unstale":"2024-04-25T08:39:57.117130+0000","last_undegraded":"2024-04-25T08:39:57.117130+0000","last_fullsized":"2024-04-25T08:39:57.117130+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108899+0000","last_change":"2024-04-25T08:39:57.108899+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108899+0000","last_undegraded":"2024-04-25T08:39:57.108899+0000","last_fullsized":"2024-04-25T08:39:57.108899+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.124072+0000","last_change":"2024-04-25T08:39:57.122861+0000","last_active":"2024-04-25T08:39:57.124072+0000","last_peered":"2024-04-25T08:39:57.124072+0000","last_clean":"2024-04-25T08:39:57.124072+0000","last_became_active":"2024-04-25T08:39:57.122443+0000","last_became_peered":"2024-04-25T08:39:57.122443+0000","last_unstale":"2024-04-25T08:39:57.124072+0000","last_undegraded":"2024-04-25T08:39:57.124072+0000","last_fullsized":"2024-04-25T08:39:57.124072+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.134755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106645+0000","last_change":"2024-04-25T08:39:57.106645+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106645+0000","last_undegraded":"2024-04-25T08:39:57.106645+0000","last_fullsized":"2024-04-25T08:39:57.106645+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.123229+0000","last_change":"2024-04-25T08:39:57.122180+0000","last_active":"2024-04-25T08:39:57.123229+0000","last_peered":"2024-04-25T08:39:57.123229+0000","last_clean":"2024-04-25T08:39:57.123229+0000","last_became_active":"2024-04-25T08:39:57.121859+0000","last_became_peered":"2024-04-25T08:39:57.121859+0000","last_unstale":"2024-04-25T08:39:57.123229+0000","last_undegraded":"2024-04-25T08:39:57.123229+0000","last_fullsized":"2024-04-25T08:39:57.123229+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:56.067733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.116232+0000","last_change":"2024-04-25T08:39:57.116232+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.116232+0000","last_undegraded":"2024-04-25T08:39:57.116232+0000","last_fullsized":"2024-04-25T08:39:57.116232+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.125352+0000","last_change":"2024-04-25T08:39:57.124849+0000","last_active":"2024-04-25T08:39:57.125352+0000","last_peered":"2024-04-25T08:39:57.125352+0000","last_clean":"2024-04-25T08:39:57.125352+0000","last_became_active":"2024-04-25T08:39:57.124555+0000","last_became_peered":"2024-04-25T08:39:57.124555+0000","last_unstale":"2024-04-25T08:39:57.125352+0000","last_undegraded":"2024-04-25T08:39:57.125352+0000","last_fullsized":"2024-04-25T08:39:57.125352+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104557+0000","last_change":"2024-04-25T08:39:57.104557+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104557+0000","last_undegraded":"2024-04-25T08:39:57.104557+0000","last_fullsized":"2024-04-25T08:39:57.104557+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116939+0000","last_change":"2024-04-25T08:39:57.115173+0000","last_active":"2024-04-25T08:39:57.116939+0000","last_peered":"2024-04-25T08:39:57.116939+0000","last_clean":"2024-04-25T08:39:57.116939+0000","last_became_active":"2024-04-25T08:39:57.114870+0000","last_became_peered":"2024-04-25T08:39:57.114870+0000","last_unstale":"2024-04-25T08:39:57.116939+0000","last_undegraded":"2024-04-25T08:39:57.116939+0000","last_fullsized":"2024-04-25T08:39:57.116939+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:29.233744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107381+0000","last_change":"2024-04-25T08:39:57.107381+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107381+0000","last_undegraded":"2024-04-25T08:39:57.107381+0000","last_fullsized":"2024-04-25T08:39:57.107381+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.119844+0000","last_change":"2024-04-25T08:39:57.117206+0000","last_active":"2024-04-25T08:39:57.119844+0000","last_peered":"2024-04-25T08:39:57.119844+0000","last_clean":"2024-04-25T08:39:57.119844+0000","last_became_active":"2024-04-25T08:39:57.116938+0000","last_became_peered":"2024-04-25T08:39:57.116938+0000","last_unstale":"2024-04-25T08:39:57.119844+0000","last_undegraded":"2024-04-25T08:39:57.119844+0000","last_fullsized":"2024-04-25T08:39:57.119844+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:14.033329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111993+0000","last_change":"2024-04-25T08:39:57.111993+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111993+0000","last_undegraded":"2024-04-25T08:39:57.111993+0000","last_fullsized":"2024-04-25T08:39:57.111993+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116515+0000","last_change":"2024-04-25T08:39:57.114054+0000","last_active":"2024-04-25T08:39:57.116515+0000","last_peered":"2024-04-25T08:39:57.116515+0000","last_clean":"2024-04-25T08:39:57.116515+0000","last_became_active":"2024-04-25T08:39:57.113271+0000","last_became_peered":"2024-04-25T08:39:57.113271+0000","last_unstale":"2024-04-25T08:39:57.116515+0000","last_undegraded":"2024-04-25T08:39:57.116515+0000","last_fullsized":"2024-04-25T08:39:57.116515+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:25.305378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.113672+0000","last_change":"2024-04-25T08:39:57.113672+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.113672+0000","last_undegraded":"2024-04-25T08:39:57.113672+0000","last_fullsized":"2024-04-25T08:39:57.113672+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121882+0000","last_change":"2024-04-25T08:39:57.117517+0000","last_active":"2024-04-25T08:39:57.121882+0000","last_peered":"2024-04-25T08:39:57.121882+0000","last_clean":"2024-04-25T08:39:57.121882+0000","last_became_active":"2024-04-25T08:39:57.116811+0000","last_became_peered":"2024-04-25T08:39:57.116811+0000","last_unstale":"2024-04-25T08:39:57.121882+0000","last_undegraded":"2024-04-25T08:39:57.121882+0000","last_fullsized":"2024-04-25T08:39:57.121882+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:36.804780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110704+0000","last_change":"2024-04-25T08:39:57.110704+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110704+0000","last_undegraded":"2024-04-25T08:39:57.110704+0000","last_fullsized":"2024-04-25T08:39:57.110704+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.110041+0000","last_change":"2024-04-25T08:39:57.109255+0000","last_active":"2024-04-25T08:39:57.110041+0000","last_peered":"2024-04-25T08:39:57.110041+0000","last_clean":"2024-04-25T08:39:57.110041+0000","last_became_active":"2024-04-25T08:39:57.108956+0000","last_became_peered":"2024-04-25T08:39:57.108956+0000","last_unstale":"2024-04-25T08:39:57.110041+0000","last_undegraded":"2024-04-25T08:39:57.110041+0000","last_fullsized":"2024-04-25T08:39:57.110041+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:53.702495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107472+0000","last_change":"2024-04-25T08:39:57.107472+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107472+0000","last_undegraded":"2024-04-25T08:39:57.107472+0000","last_fullsized":"2024-04-25T08:39:57.107472+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121148+0000","last_change":"2024-04-25T08:39:57.118978+0000","last_active":"2024-04-25T08:39:57.121148+0000","last_peered":"2024-04-25T08:39:57.121148+0000","last_clean":"2024-04-25T08:39:57.121148+0000","last_became_active":"2024-04-25T08:39:57.118673+0000","last_became_peered":"2024-04-25T08:39:57.118673+0000","last_unstale":"2024-04-25T08:39:57.121148+0000","last_undegraded":"2024-04-25T08:39:57.121148+0000","last_fullsized":"2024-04-25T08:39:57.121148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111344+0000","last_change":"2024-04-25T08:39:57.111344+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111344+0000","last_undegraded":"2024-04-25T08:39:57.111344+0000","last_fullsized":"2024-04-25T08:39:57.111344+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116295+0000","last_change":"2024-04-25T08:39:57.114778+0000","last_active":"2024-04-25T08:39:57.116295+0000","last_peered":"2024-04-25T08:39:57.116295+0000","last_clean":"2024-04-25T08:39:57.116295+0000","last_became_active":"2024-04-25T08:39:57.114509+0000","last_became_peered":"2024-04-25T08:39:57.114509+0000","last_unstale":"2024-04-25T08:39:57.116295+0000","last_undegraded":"2024-04-25T08:39:57.116295+0000","last_fullsized":"2024-04-25T08:39:57.116295+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:06.569207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108631+0000","last_change":"2024-04-25T08:39:57.108631+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108631+0000","last_undegraded":"2024-04-25T08:39:57.108631+0000","last_fullsized":"2024-04-25T08:39:57.108631+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117436+0000","last_change":"2024-04-25T08:39:57.113886+0000","last_active":"2024-04-25T08:39:57.117436+0000","last_peered":"2024-04-25T08:39:57.117436+0000","last_clean":"2024-04-25T08:39:57.117436+0000","last_became_active":"2024-04-25T08:39:57.113199+0000","last_became_peered":"2024-04-25T08:39:57.113199+0000","last_unstale":"2024-04-25T08:39:57.117436+0000","last_undegraded":"2024-04-25T08:39:57.117436+0000","last_fullsized":"2024-04-25T08:39:57.117436+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:17.991774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111394+0000","last_change":"2024-04-25T08:39:57.111394+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111394+0000","last_undegraded":"2024-04-25T08:39:57.111394+0000","last_fullsized":"2024-04-25T08:39:57.111394+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120427+0000","last_change":"2024-04-25T08:39:57.117731+0000","last_active":"2024-04-25T08:39:57.120427+0000","last_peered":"2024-04-25T08:39:57.120427+0000","last_clean":"2024-04-25T08:39:57.120427+0000","last_became_active":"2024-04-25T08:39:57.117423+0000","last_became_peered":"2024-04-25T08:39:57.117423+0000","last_unstale":"2024-04-25T08:39:57.120427+0000","last_undegraded":"2024-04-25T08:39:57.120427+0000","last_fullsized":"2024-04-25T08:39:57.120427+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:56.067733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105610+0000","last_change":"2024-04-25T08:39:57.105610+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105610+0000","last_undegraded":"2024-04-25T08:39:57.105610+0000","last_fullsized":"2024-04-25T08:39:57.105610+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116616+0000","last_change":"2024-04-25T08:39:57.115100+0000","last_active":"2024-04-25T08:39:57.116616+0000","last_peered":"2024-04-25T08:39:57.116616+0000","last_clean":"2024-04-25T08:39:57.116616+0000","last_became_active":"2024-04-25T08:39:57.114830+0000","last_became_peered":"2024-04-25T08:39:57.114830+0000","last_unstale":"2024-04-25T08:39:57.116616+0000","last_undegraded":"2024-04-25T08:39:57.116616+0000","last_fullsized":"2024-04-25T08:39:57.116616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106488+0000","last_change":"2024-04-25T08:39:57.106488+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106488+0000","last_undegraded":"2024-04-25T08:39:57.106488+0000","last_fullsized":"2024-04-25T08:39:57.106488+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.123505+0000","last_change":"2024-04-25T08:39:57.120468+0000","last_active":"2024-04-25T08:39:57.123505+0000","last_peered":"2024-04-25T08:39:57.123505+0000","last_clean":"2024-04-25T08:39:57.123505+0000","last_became_active":"2024-04-25T08:39:57.119296+0000","last_became_peered":"2024-04-25T08:39:57.119296+0000","last_unstale":"2024-04-25T08:39:57.123505+0000","last_undegraded":"2024-04-25T08:39:57.123505+0000","last_fullsized":"2024-04-25T08:39:57.123505+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:07.703556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110350+0000","last_change":"2024-04-25T08:39:57.110350+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110350+0000","last_undegraded":"2024-04-25T08:39:57.110350+0000","last_fullsized":"2024-04-25T08:39:57.110350+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117144+0000","last_change":"2024-04-25T08:39:57.113751+0000","last_active":"2024-04-25T08:39:57.117144+0000","last_peered":"2024-04-25T08:39:57.117144+0000","last_clean":"2024-04-25T08:39:57.117144+0000","last_became_active":"2024-04-25T08:39:57.113479+0000","last_became_peered":"2024-04-25T08:39:57.113479+0000","last_unstale":"2024-04-25T08:39:57.117144+0000","last_undegraded":"2024-04-25T08:39:57.117144+0000","last_fullsized":"2024-04-25T08:39:57.117144+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108211+0000","last_change":"2024-04-25T08:39:57.108211+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108211+0000","last_undegraded":"2024-04-25T08:39:57.108211+0000","last_fullsized":"2024-04-25T08:39:57.108211+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117175+0000","last_change":"2024-04-25T08:39:57.114636+0000","last_active":"2024-04-25T08:39:57.117175+0000","last_peered":"2024-04-25T08:39:57.117175+0000","last_clean":"2024-04-25T08:39:57.117175+0000","last_became_active":"2024-04-25T08:39:57.114366+0000","last_became_peered":"2024-04-25T08:39:57.114366+0000","last_unstale":"2024-04-25T08:39:57.117175+0000","last_undegraded":"2024-04-25T08:39:57.117175+0000","last_fullsized":"2024-04-25T08:39:57.117175+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104891+0000","last_change":"2024-04-25T08:39:57.104891+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104891+0000","last_undegraded":"2024-04-25T08:39:57.104891+0000","last_fullsized":"2024-04-25T08:39:57.104891+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.123503+0000","last_change":"2024-04-25T08:39:57.119185+0000","last_active":"2024-04-25T08:39:57.123503+0000","last_peered":"2024-04-25T08:39:57.123503+0000","last_clean":"2024-04-25T08:39:57.123503+0000","last_became_active":"2024-04-25T08:39:57.118135+0000","last_became_peered":"2024-04-25T08:39:57.118135+0000","last_unstale":"2024-04-25T08:39:57.123503+0000","last_undegraded":"2024-04-25T08:39:57.123503+0000","last_fullsized":"2024-04-25T08:39:57.123503+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:01.150116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.114248+0000","last_change":"2024-04-25T08:39:57.114248+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.114248+0000","last_undegraded":"2024-04-25T08:39:57.114248+0000","last_fullsized":"2024-04-25T08:39:57.114248+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.113748+0000","last_change":"2024-04-25T08:39:57.111684+0000","last_active":"2024-04-25T08:39:57.113748+0000","last_peered":"2024-04-25T08:39:57.113748+0000","last_clean":"2024-04-25T08:39:57.113748+0000","last_became_active":"2024-04-25T08:39:57.111394+0000","last_became_peered":"2024-04-25T08:39:57.111394+0000","last_unstale":"2024-04-25T08:39:57.113748+0000","last_undegraded":"2024-04-25T08:39:57.113748+0000","last_fullsized":"2024-04-25T08:39:57.113748+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:06.258822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106294+0000","last_change":"2024-04-25T08:39:57.106294+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106294+0000","last_undegraded":"2024-04-25T08:39:57.106294+0000","last_fullsized":"2024-04-25T08:39:57.106294+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121216+0000","last_change":"2024-04-25T08:39:57.118637+0000","last_active":"2024-04-25T08:39:57.121216+0000","last_peered":"2024-04-25T08:39:57.121216+0000","last_clean":"2024-04-25T08:39:57.121216+0000","last_became_active":"2024-04-25T08:39:57.118325+0000","last_became_peered":"2024-04-25T08:39:57.118325+0000","last_unstale":"2024-04-25T08:39:57.121216+0000","last_undegraded":"2024-04-25T08:39:57.121216+0000","last_fullsized":"2024-04-25T08:39:57.121216+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":356,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.102580+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:39:57.102580+0000","last_peered":"2024-04-25T08:39:57.102580+0000","last_clean":"2024-04-25T08:39:57.102580+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:39:57.102580+0000","last_undegraded":"2024-04-25T08:39:57.102580+0000","last_fullsized":"2024-04-25T08:39:57.102580+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77616,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294224,"statfs":{"total":96636764160,"available":96557285376,"internally_reserved":0,"allocated":2064384,"data_stored":1273646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59000,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312840,"statfs":{"total":96636764160,"available":96576348160,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54824,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317016,"statfs":{"total":96636764160,"available":96580624384,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68724,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303116,"statfs":{"total":96636764160,"available":96566390784,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72880,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298960,"statfs":{"total":96636764160,"available":96562135040,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58984,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312856,"statfs":{"total":96636764160,"available":96576364544,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54828,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317012,"statfs":{"total":96636764160,"available":96580620288,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51248,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320592,"statfs":{"total":96636764160,"available":96584286208,"internally_reserved":0,"allocated":2064384,"data_stored":1273646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:01.475 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:01.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.619+0000 7f450ba9a640 1 Processor -- start 2024-04-25T08:40:01.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.619+0000 7f450ba9a640 1 -- start start 2024-04-25T08:40:01.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.620+0000 7f450ba9a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4504161700 0x7f45040b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.620+0000 7f450ba9a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4504161e00 con 0x7f45040b5710 2024-04-25T08:40:01.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.620+0000 7f450ba9a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45041620e0 con 0x7f450415ddf0 2024-04-25T08:40:01.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.620+0000 7f450ba9a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45040b9930 con 0x7f4504161700 2024-04-25T08:40:01.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.620+0000 7f450a037640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4504161700 0x7f45040b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.620+0000 7f4509836640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f45040b5710 0x7f45040b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45024/0 (socket says 172.21.15.38:45024) 2024-04-25T08:40:01.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.620+0000 7f450a037640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4504161700 0x7f45040b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44498/0 (socket says 172.21.15.38:44498) 2024-04-25T08:40:01.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.620+0000 7f4509836640 1 -- 172.21.15.38:0/3872458193 learned_addr learned my addr 172.21.15.38:0/3872458193 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:01.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.620+0000 7f450a037640 1 -- 172.21.15.38:0/3872458193 >> v1:172.21.15.107:6789/0 conn(0x7f450415ddf0 legacy=0x7f45040d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:01.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.620+0000 7f450a037640 1 -- 172.21.15.38:0/3872458193 >> v1:172.21.15.38:6789/0 conn(0x7f45040b5710 legacy=0x7f45040b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.620+0000 7f450a037640 1 -- 172.21.15.38:0/3872458193 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45041623c0 con 0x7f4504161700 2024-04-25T08:40:01.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.621+0000 7f450a037640 1 --2- 172.21.15.38:0/3872458193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4504161700 0x7f45040b93f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009d90 tx=0x7f44fc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646d94a19abd4a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:01.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.621+0000 7f4508834640 1 -- 172.21.15.38:0/3872458193 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc007740 con 0x7f4504161700 2024-04-25T08:40:01.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.621+0000 7f4508834640 1 -- 172.21.15.38:0/3872458193 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc0078e0 con 0x7f4504161700 2024-04-25T08:40:01.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.621+0000 7f4508834640 1 -- 172.21.15.38:0/3872458193 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc007cb0 con 0x7f4504161700 2024-04-25T08:40:01.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.622+0000 7f450ba9a640 1 -- 172.21.15.38:0/3872458193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4504161700 msgr2=0x7f45040b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.622+0000 7f450ba9a640 1 --2- 172.21.15.38:0/3872458193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4504161700 0x7f45040b93f0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009d90 tx=0x7f44fc02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.622+0000 7f450ba9a640 1 -- 172.21.15.38:0/3872458193 shutdown_connections 2024-04-25T08:40:01.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.622+0000 7f450ba9a640 1 --2- 172.21.15.38:0/3872458193 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4504161700 0x7f45040b93f0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.622+0000 7f450ba9a640 1 -- 172.21.15.38:0/3872458193 >> 172.21.15.38:0/3872458193 conn(0x7f45040d07c0 msgr2=0x7f45040d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:01.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.622+0000 7f450ba9a640 1 -- 172.21.15.38:0/3872458193 shutdown_connections 2024-04-25T08:40:01.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.622+0000 7f450ba9a640 1 -- 172.21.15.38:0/3872458193 wait complete. 2024-04-25T08:40:01.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.623+0000 7f450ba9a640 1 Processor -- start 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.623+0000 7f450ba9a640 1 -- start start 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f450ba9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45040b5710 0x7f4504158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f450ba9a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f450415ddf0 0x7f45040c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f450ba9a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4504161700 0x7f45040c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f450ba9a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45040d2de0 con 0x7f45040b5710 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f450ba9a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45040d2c60 con 0x7f450415ddf0 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f450ba9a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45040d2f60 con 0x7f4504161700 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f4509836640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45040b5710 0x7f4504158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f4509836640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45040b5710 0x7f4504158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44512/0 (socket says 172.21.15.38:44512) 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f4509836640 1 -- 172.21.15.38:0/2356542299 learned_addr learned my addr 172.21.15.38:0/2356542299 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f4509836640 1 -- 172.21.15.38:0/2356542299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4504161700 msgr2=0x7f45040c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f4509836640 1 --2- 172.21.15.38:0/2356542299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4504161700 0x7f45040c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f4509836640 1 -- 172.21.15.38:0/2356542299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f450415ddf0 msgr2=0x7f45040c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f4509836640 1 --2- 172.21.15.38:0/2356542299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f450415ddf0 0x7f45040c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f4509836640 1 -- 172.21.15.38:0/2356542299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040ca2d0 con 0x7f45040b5710 2024-04-25T08:40:01.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.624+0000 7f4509836640 1 --2- 172.21.15.38:0/2356542299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45040b5710 0x7f4504158400 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f450000c4a0 tx=0x7f450000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:01.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.625+0000 7f44fa7fc640 1 -- 172.21.15.38:0/2356542299 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500017070 con 0x7f45040b5710 2024-04-25T08:40:01.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.625+0000 7f44fa7fc640 1 -- 172.21.15.38:0/2356542299 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4500005ce0 con 0x7f45040b5710 2024-04-25T08:40:01.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.625+0000 7f44fa7fc640 1 -- 172.21.15.38:0/2356542299 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500007270 con 0x7f45040b5710 2024-04-25T08:40:01.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.625+0000 7f450ba9a640 1 -- 172.21.15.38:0/2356542299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45040c9750 con 0x7f45040b5710 2024-04-25T08:40:01.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.625+0000 7f450ba9a640 1 -- 172.21.15.38:0/2356542299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45040c9d10 con 0x7f45040b5710 2024-04-25T08:40:01.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.626+0000 7f450ba9a640 1 -- 172.21.15.38:0/2356542299 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4504068e80 con 0x7f45040b5710 2024-04-25T08:40:01.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.626+0000 7f44fa7fc640 1 -- 172.21.15.38:0/2356542299 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f450001a070 con 0x7f45040b5710 2024-04-25T08:40:01.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.627+0000 7f44fa7fc640 1 --2- 172.21.15.38:0/2356542299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44e806b4d0 0x7f44e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.627+0000 7f44fa7fc640 1 -- 172.21.15.38:0/2356542299 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f45000895d0 con 0x7f45040b5710 2024-04-25T08:40:01.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.627+0000 7f4509035640 1 --2- 172.21.15.38:0/2356542299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44e806b4d0 0x7f44e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.627+0000 7f4509035640 1 --2- 172.21.15.38:0/2356542299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44e806b4d0 0x7f44e806d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f44f4000f60 tx=0x7f44f40079d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:01.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.631+0000 7f44fa7fc640 1 -- 172.21.15.38:0/2356542299 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4500057be0 con 0x7f45040b5710 2024-04-25T08:40:01.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.807+0000 7f450ba9a640 1 -- 172.21.15.38:0/2356542299 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4504156590 con 0x7f44e806b4d0 2024-04-25T08:40:01.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.813+0000 7f44fa7fc640 1 -- 172.21.15.38:0/2356542299 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7f4504156590 con 0x7f44e806b4d0 2024-04-25T08:40:01.809 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:01.810 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:40:01.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.816+0000 7f450ba9a640 1 -- 172.21.15.38:0/2356542299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44e806b4d0 msgr2=0x7f44e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.816+0000 7f450ba9a640 1 --2- 172.21.15.38:0/2356542299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44e806b4d0 0x7f44e806d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f44f4000f60 tx=0x7f44f40079d0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.816+0000 7f450ba9a640 1 -- 172.21.15.38:0/2356542299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45040b5710 msgr2=0x7f4504158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.816+0000 7f450ba9a640 1 --2- 172.21.15.38:0/2356542299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45040b5710 0x7f4504158400 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f450000c4a0 tx=0x7f450000c960 comp rx=0 tx=0).stop 2024-04-25T08:40:01.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.816+0000 7f450ba9a640 1 -- 172.21.15.38:0/2356542299 shutdown_connections 2024-04-25T08:40:01.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.816+0000 7f450ba9a640 1 --2- 172.21.15.38:0/2356542299 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f44e806b4d0 0x7f44e806d990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.816+0000 7f450ba9a640 1 --2- 172.21.15.38:0/2356542299 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4504161700 0x7f45040c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.816+0000 7f450ba9a640 1 --2- 172.21.15.38:0/2356542299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f450415ddf0 0x7f45040c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.816+0000 7f450ba9a640 1 --2- 172.21.15.38:0/2356542299 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45040b5710 0x7f4504158400 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.816+0000 7f450ba9a640 1 -- 172.21.15.38:0/2356542299 >> 172.21.15.38:0/2356542299 conn(0x7f45040d07c0 msgr2=0x7f45040b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:01.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.816+0000 7f450ba9a640 1 -- 172.21.15.38:0/2356542299 shutdown_connections 2024-04-25T08:40:01.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.816+0000 7f450ba9a640 1 -- 172.21.15.38:0/2356542299 wait complete. 2024-04-25T08:40:01.821 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-04-25T08:40:01.323124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":498104,"kb_used_data":12648,"kb_used_omap":12,"kb_used_meta":485299,"kb_avail":754476616,"statfs":{"total":773094113280,"available":772584054784,"internally_reserved":0,"allocated":12951552,"data_stored":6646960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":496946736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.223817"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104507+0000","last_change":"2024-04-25T08:39:57.104507+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104507+0000","last_undegraded":"2024-04-25T08:39:57.104507+0000","last_fullsized":"2024-04-25T08:39:57.104507+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108915+0000","last_change":"2024-04-25T08:39:57.108915+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108915+0000","last_undegraded":"2024-04-25T08:39:57.108915+0000","last_fullsized":"2024-04-25T08:39:57.108915+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105895+0000","last_change":"2024-04-25T08:39:57.105895+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105895+0000","last_undegraded":"2024-04-25T08:39:57.105895+0000","last_fullsized":"2024-04-25T08:39:57.105895+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109343+0000","last_change":"2024-04-25T08:39:57.109343+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109343+0000","last_undegraded":"2024-04-25T08:39:57.109343+0000","last_fullsized":"2024-04-25T08:39:57.109343+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105957+0000","last_change":"2024-04-25T08:39:57.105957+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105957+0000","last_undegraded":"2024-04-25T08:39:57.105957+0000","last_fullsized":"2024-04-25T08:39:57.105957+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107903+0000","last_change":"2024-04-25T08:39:57.107903+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107903+0000","last_undegraded":"2024-04-25T08:39:57.107903+0000","last_fullsized":"2024-04-25T08:39:57.107903+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109081+0000","last_change":"2024-04-25T08:39:57.109081+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109081+0000","last_undegraded":"2024-04-25T08:39:57.109081+0000","last_fullsized":"2024-04-25T08:39:57.109081+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106732+0000","last_change":"2024-04-25T08:39:57.106732+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106732+0000","last_undegraded":"2024-04-25T08:39:57.106732+0000","last_fullsized":"2024-04-25T08:39:57.106732+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108726+0000","last_change":"2024-04-25T08:39:57.108726+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108726+0000","last_undegraded":"2024-04-25T08:39:57.108726+0000","last_fullsized":"2024-04-25T08:39:57.108726+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110497+0000","last_change":"2024-04-25T08:39:57.110497+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110497+0000","last_undegraded":"2024-04-25T08:39:57.110497+0000","last_fullsized":"2024-04-25T08:39:57.110497+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105137+0000","last_change":"2024-04-25T08:39:57.105137+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105137+0000","last_undegraded":"2024-04-25T08:39:57.105137+0000","last_fullsized":"2024-04-25T08:39:57.105137+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109567+0000","last_change":"2024-04-25T08:39:57.109567+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109567+0000","last_undegraded":"2024-04-25T08:39:57.109567+0000","last_fullsized":"2024-04-25T08:39:57.109567+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.115061+0000","last_change":"2024-04-25T08:39:57.115061+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.115061+0000","last_undegraded":"2024-04-25T08:39:57.115061+0000","last_fullsized":"2024-04-25T08:39:57.115061+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109950+0000","last_change":"2024-04-25T08:39:57.109950+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109950+0000","last_undegraded":"2024-04-25T08:39:57.109950+0000","last_fullsized":"2024-04-25T08:39:57.109950+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104679+0000","last_change":"2024-04-25T08:39:57.104679+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104679+0000","last_undegraded":"2024-04-25T08:39:57.104679+0000","last_fullsized":"2024-04-25T08:39:57.104679+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109887+0000","last_change":"2024-04-25T08:39:57.109887+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109887+0000","last_undegraded":"2024-04-25T08:39:57.109887+0000","last_fullsized":"2024-04-25T08:39:57.109887+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.112277+0000","last_change":"2024-04-25T08:39:57.112277+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.112277+0000","last_undegraded":"2024-04-25T08:39:57.112277+0000","last_fullsized":"2024-04-25T08:39:57.112277+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108487+0000","last_change":"2024-04-25T08:39:57.108487+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108487+0000","last_undegraded":"2024-04-25T08:39:57.108487+0000","last_fullsized":"2024-04-25T08:39:57.108487+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107714+0000","last_change":"2024-04-25T08:39:57.107714+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107714+0000","last_undegraded":"2024-04-25T08:39:57.107714+0000","last_fullsized":"2024-04-25T08:39:57.107714+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105862+0000","last_change":"2024-04-25T08:39:57.105862+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105862+0000","last_undegraded":"2024-04-25T08:39:57.105862+0000","last_fullsized":"2024-04-25T08:39:57.105862+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108441+0000","last_change":"2024-04-25T08:39:57.108441+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108441+0000","last_undegraded":"2024-04-25T08:39:57.108441+0000","last_fullsized":"2024-04-25T08:39:57.108441+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105711+0000","last_change":"2024-04-25T08:39:57.105711+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105711+0000","last_undegraded":"2024-04-25T08:39:57.105711+0000","last_fullsized":"2024-04-25T08:39:57.105711+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107927+0000","last_change":"2024-04-25T08:39:57.107927+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107927+0000","last_undegraded":"2024-04-25T08:39:57.107927+0000","last_fullsized":"2024-04-25T08:39:57.107927+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111019+0000","last_change":"2024-04-25T08:39:57.111019+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111019+0000","last_undegraded":"2024-04-25T08:39:57.111019+0000","last_fullsized":"2024-04-25T08:39:57.111019+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.112909+0000","last_change":"2024-04-25T08:39:57.112909+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.112909+0000","last_undegraded":"2024-04-25T08:39:57.112909+0000","last_fullsized":"2024-04-25T08:39:57.112909+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106551+0000","last_change":"2024-04-25T08:39:57.106551+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106551+0000","last_undegraded":"2024-04-25T08:39:57.106551+0000","last_fullsized":"2024-04-25T08:39:57.106551+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106831+0000","last_change":"2024-04-25T08:39:57.106831+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106831+0000","last_undegraded":"2024-04-25T08:39:57.106831+0000","last_fullsized":"2024-04-25T08:39:57.106831+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109055+0000","last_change":"2024-04-25T08:39:57.109055+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109055+0000","last_undegraded":"2024-04-25T08:39:57.109055+0000","last_fullsized":"2024-04-25T08:39:57.109055+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106587+0000","last_change":"2024-04-25T08:39:57.106587+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106587+0000","last_undegraded":"2024-04-25T08:39:57.106587+0000","last_fullsized":"2024-04-25T08:39:57.106587+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109521+0000","last_change":"2024-04-25T08:39:57.109521+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109521+0000","last_undegraded":"2024-04-25T08:39:57.109521+0000","last_fullsized":"2024-04-25T08:39:57.109521+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111398+0000","last_change":"2024-04-25T08:39:57.111398+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111398+0000","last_undegraded":"2024-04-25T08:39:57.111398+0000","last_fullsized":"2024-04-25T08:39:57.111398+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110178+0000","last_change":"2024-04-25T08:39:57.110178+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110178+0000","last_undegraded":"2024-04-25T08:39:57.110178+0000","last_fullsized":"2024-04-25T08:39:57.110178+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109690+0000","last_change":"2024-04-25T08:39:57.109690+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109690+0000","last_undegraded":"2024-04-25T08:39:57.109690+0000","last_fullsized":"2024-04-25T08:39:57.109690+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.112046+0000","last_change":"2024-04-25T08:39:57.111533+0000","last_active":"2024-04-25T08:39:57.112046+0000","last_peered":"2024-04-25T08:39:57.112046+0000","last_clean":"2024-04-25T08:39:57.112046+0000","last_became_active":"2024-04-25T08:39:57.111229+0000","last_became_peered":"2024-04-25T08:39:57.111229+0000","last_unstale":"2024-04-25T08:39:57.112046+0000","last_undegraded":"2024-04-25T08:39:57.112046+0000","last_fullsized":"2024-04-25T08:39:57.112046+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108536+0000","last_change":"2024-04-25T08:39:57.108536+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108536+0000","last_undegraded":"2024-04-25T08:39:57.108536+0000","last_fullsized":"2024-04-25T08:39:57.108536+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120536+0000","last_change":"2024-04-25T08:39:57.117732+0000","last_active":"2024-04-25T08:39:57.120536+0000","last_peered":"2024-04-25T08:39:57.120536+0000","last_clean":"2024-04-25T08:39:57.120536+0000","last_became_active":"2024-04-25T08:39:57.117425+0000","last_became_peered":"2024-04-25T08:39:57.117425+0000","last_unstale":"2024-04-25T08:39:57.120536+0000","last_undegraded":"2024-04-25T08:39:57.120536+0000","last_fullsized":"2024-04-25T08:39:57.120536+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:40.880195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111346+0000","last_change":"2024-04-25T08:39:57.111346+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111346+0000","last_undegraded":"2024-04-25T08:39:57.111346+0000","last_fullsized":"2024-04-25T08:39:57.111346+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.111873+0000","last_change":"2024-04-25T08:39:57.110361+0000","last_active":"2024-04-25T08:39:57.111873+0000","last_peered":"2024-04-25T08:39:57.111873+0000","last_clean":"2024-04-25T08:39:57.111873+0000","last_became_active":"2024-04-25T08:39:57.110079+0000","last_became_peered":"2024-04-25T08:39:57.110079+0000","last_unstale":"2024-04-25T08:39:57.111873+0000","last_undegraded":"2024-04-25T08:39:57.111873+0000","last_fullsized":"2024-04-25T08:39:57.111873+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:55.191344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107451+0000","last_change":"2024-04-25T08:39:57.107451+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107451+0000","last_undegraded":"2024-04-25T08:39:57.107451+0000","last_fullsized":"2024-04-25T08:39:57.107451+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.122916+0000","last_change":"2024-04-25T08:39:57.122219+0000","last_active":"2024-04-25T08:39:57.122916+0000","last_peered":"2024-04-25T08:39:57.122916+0000","last_clean":"2024-04-25T08:39:57.122916+0000","last_became_active":"2024-04-25T08:39:57.121910+0000","last_became_peered":"2024-04-25T08:39:57.121910+0000","last_unstale":"2024-04-25T08:39:57.122916+0000","last_undegraded":"2024-04-25T08:39:57.122916+0000","last_fullsized":"2024-04-25T08:39:57.122916+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108782+0000","last_change":"2024-04-25T08:39:57.108782+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108782+0000","last_undegraded":"2024-04-25T08:39:57.108782+0000","last_fullsized":"2024-04-25T08:39:57.108782+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120411+0000","last_change":"2024-04-25T08:39:57.119082+0000","last_active":"2024-04-25T08:39:57.120411+0000","last_peered":"2024-04-25T08:39:57.120411+0000","last_clean":"2024-04-25T08:39:57.120411+0000","last_became_active":"2024-04-25T08:39:57.118794+0000","last_became_peered":"2024-04-25T08:39:57.118794+0000","last_unstale":"2024-04-25T08:39:57.120411+0000","last_undegraded":"2024-04-25T08:39:57.120411+0000","last_fullsized":"2024-04-25T08:39:57.120411+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105610+0000","last_change":"2024-04-25T08:39:57.105610+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105610+0000","last_undegraded":"2024-04-25T08:39:57.105610+0000","last_fullsized":"2024-04-25T08:39:57.105610+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.124563+0000","last_change":"2024-04-25T08:39:57.123981+0000","last_active":"2024-04-25T08:39:57.124563+0000","last_peered":"2024-04-25T08:39:57.124563+0000","last_clean":"2024-04-25T08:39:57.124563+0000","last_became_active":"2024-04-25T08:39:57.123631+0000","last_became_peered":"2024-04-25T08:39:57.123631+0000","last_unstale":"2024-04-25T08:39:57.124563+0000","last_undegraded":"2024-04-25T08:39:57.124563+0000","last_fullsized":"2024-04-25T08:39:57.124563+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:58.838892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107893+0000","last_change":"2024-04-25T08:39:57.107893+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107893+0000","last_undegraded":"2024-04-25T08:39:57.107893+0000","last_fullsized":"2024-04-25T08:39:57.107893+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.119811+0000","last_change":"2024-04-25T08:39:57.116831+0000","last_active":"2024-04-25T08:39:57.119811+0000","last_peered":"2024-04-25T08:39:57.119811+0000","last_clean":"2024-04-25T08:39:57.119811+0000","last_became_active":"2024-04-25T08:39:57.116598+0000","last_became_peered":"2024-04-25T08:39:57.116598+0000","last_unstale":"2024-04-25T08:39:57.119811+0000","last_undegraded":"2024-04-25T08:39:57.119811+0000","last_fullsized":"2024-04-25T08:39:57.119811+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:43.760739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.114648+0000","last_change":"2024-04-25T08:39:57.114648+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.114648+0000","last_undegraded":"2024-04-25T08:39:57.114648+0000","last_fullsized":"2024-04-25T08:39:57.114648+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.125099+0000","last_change":"2024-04-25T08:39:57.124496+0000","last_active":"2024-04-25T08:39:57.125099+0000","last_peered":"2024-04-25T08:39:57.125099+0000","last_clean":"2024-04-25T08:39:57.125099+0000","last_became_active":"2024-04-25T08:39:57.123329+0000","last_became_peered":"2024-04-25T08:39:57.123329+0000","last_unstale":"2024-04-25T08:39:57.125099+0000","last_undegraded":"2024-04-25T08:39:57.125099+0000","last_fullsized":"2024-04-25T08:39:57.125099+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:25.558741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104630+0000","last_change":"2024-04-25T08:39:57.104630+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104630+0000","last_undegraded":"2024-04-25T08:39:57.104630+0000","last_fullsized":"2024-04-25T08:39:57.104630+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117207+0000","last_change":"2024-04-25T08:39:57.115175+0000","last_active":"2024-04-25T08:39:57.117207+0000","last_peered":"2024-04-25T08:39:57.117207+0000","last_clean":"2024-04-25T08:39:57.117207+0000","last_became_active":"2024-04-25T08:39:57.114913+0000","last_became_peered":"2024-04-25T08:39:57.114913+0000","last_unstale":"2024-04-25T08:39:57.117207+0000","last_undegraded":"2024-04-25T08:39:57.117207+0000","last_fullsized":"2024-04-25T08:39:57.117207+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:41.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.113272+0000","last_change":"2024-04-25T08:39:57.113272+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.113272+0000","last_undegraded":"2024-04-25T08:39:57.113272+0000","last_fullsized":"2024-04-25T08:39:57.113272+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120816+0000","last_change":"2024-04-25T08:39:57.119480+0000","last_active":"2024-04-25T08:39:57.120816+0000","last_peered":"2024-04-25T08:39:57.120816+0000","last_clean":"2024-04-25T08:39:57.120816+0000","last_became_active":"2024-04-25T08:39:57.119195+0000","last_became_peered":"2024-04-25T08:39:57.119195+0000","last_unstale":"2024-04-25T08:39:57.120816+0000","last_undegraded":"2024-04-25T08:39:57.120816+0000","last_fullsized":"2024-04-25T08:39:57.120816+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106051+0000","last_change":"2024-04-25T08:39:57.106051+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106051+0000","last_undegraded":"2024-04-25T08:39:57.106051+0000","last_fullsized":"2024-04-25T08:39:57.106051+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120480+0000","last_change":"2024-04-25T08:39:57.118299+0000","last_active":"2024-04-25T08:39:57.120480+0000","last_peered":"2024-04-25T08:39:57.120480+0000","last_clean":"2024-04-25T08:39:57.120480+0000","last_became_active":"2024-04-25T08:39:57.117996+0000","last_became_peered":"2024-04-25T08:39:57.117996+0000","last_unstale":"2024-04-25T08:39:57.120480+0000","last_undegraded":"2024-04-25T08:39:57.120480+0000","last_fullsized":"2024-04-25T08:39:57.120480+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:40.880195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.112478+0000","last_change":"2024-04-25T08:39:57.112478+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.112478+0000","last_undegraded":"2024-04-25T08:39:57.112478+0000","last_fullsized":"2024-04-25T08:39:57.112478+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121442+0000","last_change":"2024-04-25T08:39:57.119869+0000","last_active":"2024-04-25T08:39:57.121442+0000","last_peered":"2024-04-25T08:39:57.121442+0000","last_clean":"2024-04-25T08:39:57.121442+0000","last_became_active":"2024-04-25T08:39:57.117564+0000","last_became_peered":"2024-04-25T08:39:57.117564+0000","last_unstale":"2024-04-25T08:39:57.121442+0000","last_undegraded":"2024-04-25T08:39:57.121442+0000","last_fullsized":"2024-04-25T08:39:57.121442+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:41.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105978+0000","last_change":"2024-04-25T08:39:57.105978+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105978+0000","last_undegraded":"2024-04-25T08:39:57.105978+0000","last_fullsized":"2024-04-25T08:39:57.105978+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117130+0000","last_change":"2024-04-25T08:39:57.111954+0000","last_active":"2024-04-25T08:39:57.117130+0000","last_peered":"2024-04-25T08:39:57.117130+0000","last_clean":"2024-04-25T08:39:57.117130+0000","last_became_active":"2024-04-25T08:39:57.111611+0000","last_became_peered":"2024-04-25T08:39:57.111611+0000","last_unstale":"2024-04-25T08:39:57.117130+0000","last_undegraded":"2024-04-25T08:39:57.117130+0000","last_fullsized":"2024-04-25T08:39:57.117130+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108899+0000","last_change":"2024-04-25T08:39:57.108899+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108899+0000","last_undegraded":"2024-04-25T08:39:57.108899+0000","last_fullsized":"2024-04-25T08:39:57.108899+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.124072+0000","last_change":"2024-04-25T08:39:57.122861+0000","last_active":"2024-04-25T08:39:57.124072+0000","last_peered":"2024-04-25T08:39:57.124072+0000","last_clean":"2024-04-25T08:39:57.124072+0000","last_became_active":"2024-04-25T08:39:57.122443+0000","last_became_peered":"2024-04-25T08:39:57.122443+0000","last_unstale":"2024-04-25T08:39:57.124072+0000","last_undegraded":"2024-04-25T08:39:57.124072+0000","last_fullsized":"2024-04-25T08:39:57.124072+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.134755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106645+0000","last_change":"2024-04-25T08:39:57.106645+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106645+0000","last_undegraded":"2024-04-25T08:39:57.106645+0000","last_fullsized":"2024-04-25T08:39:57.106645+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.123229+0000","last_change":"2024-04-25T08:39:57.122180+0000","last_active":"2024-04-25T08:39:57.123229+0000","last_peered":"2024-04-25T08:39:57.123229+0000","last_clean":"2024-04-25T08:39:57.123229+0000","last_became_active":"2024-04-25T08:39:57.121859+0000","last_became_peered":"2024-04-25T08:39:57.121859+0000","last_unstale":"2024-04-25T08:39:57.123229+0000","last_undegraded":"2024-04-25T08:39:57.123229+0000","last_fullsized":"2024-04-25T08:39:57.123229+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:56.067733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.116232+0000","last_change":"2024-04-25T08:39:57.116232+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.116232+0000","last_undegraded":"2024-04-25T08:39:57.116232+0000","last_fullsized":"2024-04-25T08:39:57.116232+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.125352+0000","last_change":"2024-04-25T08:39:57.124849+0000","last_active":"2024-04-25T08:39:57.125352+0000","last_peered":"2024-04-25T08:39:57.125352+0000","last_clean":"2024-04-25T08:39:57.125352+0000","last_became_active":"2024-04-25T08:39:57.124555+0000","last_became_peered":"2024-04-25T08:39:57.124555+0000","last_unstale":"2024-04-25T08:39:57.125352+0000","last_undegraded":"2024-04-25T08:39:57.125352+0000","last_fullsized":"2024-04-25T08:39:57.125352+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104557+0000","last_change":"2024-04-25T08:39:57.104557+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104557+0000","last_undegraded":"2024-04-25T08:39:57.104557+0000","last_fullsized":"2024-04-25T08:39:57.104557+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116939+0000","last_change":"2024-04-25T08:39:57.115173+0000","last_active":"2024-04-25T08:39:57.116939+0000","last_peered":"2024-04-25T08:39:57.116939+0000","last_clean":"2024-04-25T08:39:57.116939+0000","last_became_active":"2024-04-25T08:39:57.114870+0000","last_became_peered":"2024-04-25T08:39:57.114870+0000","last_unstale":"2024-04-25T08:39:57.116939+0000","last_undegraded":"2024-04-25T08:39:57.116939+0000","last_fullsized":"2024-04-25T08:39:57.116939+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:29.233744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107381+0000","last_change":"2024-04-25T08:39:57.107381+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107381+0000","last_undegraded":"2024-04-25T08:39:57.107381+0000","last_fullsized":"2024-04-25T08:39:57.107381+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.119844+0000","last_change":"2024-04-25T08:39:57.117206+0000","last_active":"2024-04-25T08:39:57.119844+0000","last_peered":"2024-04-25T08:39:57.119844+0000","last_clean":"2024-04-25T08:39:57.119844+0000","last_became_active":"2024-04-25T08:39:57.116938+0000","last_became_peered":"2024-04-25T08:39:57.116938+0000","last_unstale":"2024-04-25T08:39:57.119844+0000","last_undegraded":"2024-04-25T08:39:57.119844+0000","last_fullsized":"2024-04-25T08:39:57.119844+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:14.033329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111993+0000","last_change":"2024-04-25T08:39:57.111993+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111993+0000","last_undegraded":"2024-04-25T08:39:57.111993+0000","last_fullsized":"2024-04-25T08:39:57.111993+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116515+0000","last_change":"2024-04-25T08:39:57.114054+0000","last_active":"2024-04-25T08:39:57.116515+0000","last_peered":"2024-04-25T08:39:57.116515+0000","last_clean":"2024-04-25T08:39:57.116515+0000","last_became_active":"2024-04-25T08:39:57.113271+0000","last_became_peered":"2024-04-25T08:39:57.113271+0000","last_unstale":"2024-04-25T08:39:57.116515+0000","last_undegraded":"2024-04-25T08:39:57.116515+0000","last_fullsized":"2024-04-25T08:39:57.116515+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:25.305378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.113672+0000","last_change":"2024-04-25T08:39:57.113672+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.113672+0000","last_undegraded":"2024-04-25T08:39:57.113672+0000","last_fullsized":"2024-04-25T08:39:57.113672+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121882+0000","last_change":"2024-04-25T08:39:57.117517+0000","last_active":"2024-04-25T08:39:57.121882+0000","last_peered":"2024-04-25T08:39:57.121882+0000","last_clean":"2024-04-25T08:39:57.121882+0000","last_became_active":"2024-04-25T08:39:57.116811+0000","last_became_peered":"2024-04-25T08:39:57.116811+0000","last_unstale":"2024-04-25T08:39:57.121882+0000","last_undegraded":"2024-04-25T08:39:57.121882+0000","last_fullsized":"2024-04-25T08:39:57.121882+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:36.804780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110704+0000","last_change":"2024-04-25T08:39:57.110704+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110704+0000","last_undegraded":"2024-04-25T08:39:57.110704+0000","last_fullsized":"2024-04-25T08:39:57.110704+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.110041+0000","last_change":"2024-04-25T08:39:57.109255+0000","last_active":"2024-04-25T08:39:57.110041+0000","last_peered":"2024-04-25T08:39:57.110041+0000","last_clean":"2024-04-25T08:39:57.110041+0000","last_became_active":"2024-04-25T08:39:57.108956+0000","last_became_peered":"2024-04-25T08:39:57.108956+0000","last_unstale":"2024-04-25T08:39:57.110041+0000","last_undegraded":"2024-04-25T08:39:57.110041+0000","last_fullsized":"2024-04-25T08:39:57.110041+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:53.702495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107472+0000","last_change":"2024-04-25T08:39:57.107472+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107472+0000","last_undegraded":"2024-04-25T08:39:57.107472+0000","last_fullsized":"2024-04-25T08:39:57.107472+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121148+0000","last_change":"2024-04-25T08:39:57.118978+0000","last_active":"2024-04-25T08:39:57.121148+0000","last_peered":"2024-04-25T08:39:57.121148+0000","last_clean":"2024-04-25T08:39:57.121148+0000","last_became_active":"2024-04-25T08:39:57.118673+0000","last_became_peered":"2024-04-25T08:39:57.118673+0000","last_unstale":"2024-04-25T08:39:57.121148+0000","last_undegraded":"2024-04-25T08:39:57.121148+0000","last_fullsized":"2024-04-25T08:39:57.121148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111344+0000","last_change":"2024-04-25T08:39:57.111344+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111344+0000","last_undegraded":"2024-04-25T08:39:57.111344+0000","last_fullsized":"2024-04-25T08:39:57.111344+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116295+0000","last_change":"2024-04-25T08:39:57.114778+0000","last_active":"2024-04-25T08:39:57.116295+0000","last_peered":"2024-04-25T08:39:57.116295+0000","last_clean":"2024-04-25T08:39:57.116295+0000","last_became_active":"2024-04-25T08:39:57.114509+0000","last_became_peered":"2024-04-25T08:39:57.114509+0000","last_unstale":"2024-04-25T08:39:57.116295+0000","last_undegraded":"2024-04-25T08:39:57.116295+0000","last_fullsized":"2024-04-25T08:39:57.116295+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:06.569207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108631+0000","last_change":"2024-04-25T08:39:57.108631+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108631+0000","last_undegraded":"2024-04-25T08:39:57.108631+0000","last_fullsized":"2024-04-25T08:39:57.108631+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117436+0000","last_change":"2024-04-25T08:39:57.113886+0000","last_active":"2024-04-25T08:39:57.117436+0000","last_peered":"2024-04-25T08:39:57.117436+0000","last_clean":"2024-04-25T08:39:57.117436+0000","last_became_active":"2024-04-25T08:39:57.113199+0000","last_became_peered":"2024-04-25T08:39:57.113199+0000","last_unstale":"2024-04-25T08:39:57.117436+0000","last_undegraded":"2024-04-25T08:39:57.117436+0000","last_fullsized":"2024-04-25T08:39:57.117436+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:17.991774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111394+0000","last_change":"2024-04-25T08:39:57.111394+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111394+0000","last_undegraded":"2024-04-25T08:39:57.111394+0000","last_fullsized":"2024-04-25T08:39:57.111394+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120427+0000","last_change":"2024-04-25T08:39:57.117731+0000","last_active":"2024-04-25T08:39:57.120427+0000","last_peered":"2024-04-25T08:39:57.120427+0000","last_clean":"2024-04-25T08:39:57.120427+0000","last_became_active":"2024-04-25T08:39:57.117423+0000","last_became_peered":"2024-04-25T08:39:57.117423+0000","last_unstale":"2024-04-25T08:39:57.120427+0000","last_undegraded":"2024-04-25T08:39:57.120427+0000","last_fullsized":"2024-04-25T08:39:57.120427+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:56.067733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105610+0000","last_change":"2024-04-25T08:39:57.105610+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105610+0000","last_undegraded":"2024-04-25T08:39:57.105610+0000","last_fullsized":"2024-04-25T08:39:57.105610+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116616+0000","last_change":"2024-04-25T08:39:57.115100+0000","last_active":"2024-04-25T08:39:57.116616+0000","last_peered":"2024-04-25T08:39:57.116616+0000","last_clean":"2024-04-25T08:39:57.116616+0000","last_became_active":"2024-04-25T08:39:57.114830+0000","last_became_peered":"2024-04-25T08:39:57.114830+0000","last_unstale":"2024-04-25T08:39:57.116616+0000","last_undegraded":"2024-04-25T08:39:57.116616+0000","last_fullsized":"2024-04-25T08:39:57.116616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106488+0000","last_change":"2024-04-25T08:39:57.106488+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106488+0000","last_undegraded":"2024-04-25T08:39:57.106488+0000","last_fullsized":"2024-04-25T08:39:57.106488+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.123505+0000","last_change":"2024-04-25T08:39:57.120468+0000","last_active":"2024-04-25T08:39:57.123505+0000","last_peered":"2024-04-25T08:39:57.123505+0000","last_clean":"2024-04-25T08:39:57.123505+0000","last_became_active":"2024-04-25T08:39:57.119296+0000","last_became_peered":"2024-04-25T08:39:57.119296+0000","last_unstale":"2024-04-25T08:39:57.123505+0000","last_undegraded":"2024-04-25T08:39:57.123505+0000","last_fullsized":"2024-04-25T08:39:57.123505+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:07.703556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110350+0000","last_change":"2024-04-25T08:39:57.110350+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110350+0000","last_undegraded":"2024-04-25T08:39:57.110350+0000","last_fullsized":"2024-04-25T08:39:57.110350+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117144+0000","last_change":"2024-04-25T08:39:57.113751+0000","last_active":"2024-04-25T08:39:57.117144+0000","last_peered":"2024-04-25T08:39:57.117144+0000","last_clean":"2024-04-25T08:39:57.117144+0000","last_became_active":"2024-04-25T08:39:57.113479+0000","last_became_peered":"2024-04-25T08:39:57.113479+0000","last_unstale":"2024-04-25T08:39:57.117144+0000","last_undegraded":"2024-04-25T08:39:57.117144+0000","last_fullsized":"2024-04-25T08:39:57.117144+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108211+0000","last_change":"2024-04-25T08:39:57.108211+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108211+0000","last_undegraded":"2024-04-25T08:39:57.108211+0000","last_fullsized":"2024-04-25T08:39:57.108211+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117175+0000","last_change":"2024-04-25T08:39:57.114636+0000","last_active":"2024-04-25T08:39:57.117175+0000","last_peered":"2024-04-25T08:39:57.117175+0000","last_clean":"2024-04-25T08:39:57.117175+0000","last_became_active":"2024-04-25T08:39:57.114366+0000","last_became_peered":"2024-04-25T08:39:57.114366+0000","last_unstale":"2024-04-25T08:39:57.117175+0000","last_undegraded":"2024-04-25T08:39:57.117175+0000","last_fullsized":"2024-04-25T08:39:57.117175+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104891+0000","last_change":"2024-04-25T08:39:57.104891+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104891+0000","last_undegraded":"2024-04-25T08:39:57.104891+0000","last_fullsized":"2024-04-25T08:39:57.104891+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.123503+0000","last_change":"2024-04-25T08:39:57.119185+0000","last_active":"2024-04-25T08:39:57.123503+0000","last_peered":"2024-04-25T08:39:57.123503+0000","last_clean":"2024-04-25T08:39:57.123503+0000","last_became_active":"2024-04-25T08:39:57.118135+0000","last_became_peered":"2024-04-25T08:39:57.118135+0000","last_unstale":"2024-04-25T08:39:57.123503+0000","last_undegraded":"2024-04-25T08:39:57.123503+0000","last_fullsized":"2024-04-25T08:39:57.123503+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:01.150116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.114248+0000","last_change":"2024-04-25T08:39:57.114248+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.114248+0000","last_undegraded":"2024-04-25T08:39:57.114248+0000","last_fullsized":"2024-04-25T08:39:57.114248+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.113748+0000","last_change":"2024-04-25T08:39:57.111684+0000","last_active":"2024-04-25T08:39:57.113748+0000","last_peered":"2024-04-25T08:39:57.113748+0000","last_clean":"2024-04-25T08:39:57.113748+0000","last_became_active":"2024-04-25T08:39:57.111394+0000","last_became_peered":"2024-04-25T08:39:57.111394+0000","last_unstale":"2024-04-25T08:39:57.113748+0000","last_undegraded":"2024-04-25T08:39:57.113748+0000","last_fullsized":"2024-04-25T08:39:57.113748+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:06.258822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106294+0000","last_change":"2024-04-25T08:39:57.106294+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106294+0000","last_undegraded":"2024-04-25T08:39:57.106294+0000","last_fullsized":"2024-04-25T08:39:57.106294+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121216+0000","last_change":"2024-04-25T08:39:57.118637+0000","last_active":"2024-04-25T08:39:57.121216+0000","last_peered":"2024-04-25T08:39:57.121216+0000","last_clean":"2024-04-25T08:39:57.121216+0000","last_became_active":"2024-04-25T08:39:57.118325+0000","last_became_peered":"2024-04-25T08:39:57.118325+0000","last_unstale":"2024-04-25T08:39:57.121216+0000","last_undegraded":"2024-04-25T08:39:57.121216+0000","last_fullsized":"2024-04-25T08:39:57.121216+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":356,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.102580+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:39:57.102580+0000","last_peered":"2024-04-25T08:39:57.102580+0000","last_clean":"2024-04-25T08:39:57.102580+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:39:57.102580+0000","last_undegraded":"2024-04-25T08:39:57.102580+0000","last_fullsized":"2024-04-25T08:39:57.102580+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77616,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294224,"statfs":{"total":96636764160,"available":96557285376,"internally_reserved":0,"allocated":2064384,"data_stored":1273646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59000,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312840,"statfs":{"total":96636764160,"available":96576348160,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54824,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317016,"statfs":{"total":96636764160,"available":96580624384,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68724,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303116,"statfs":{"total":96636764160,"available":96566390784,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72880,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298960,"statfs":{"total":96636764160,"available":96562135040,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58984,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312856,"statfs":{"total":96636764160,"available":96576364544,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54828,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317012,"statfs":{"total":96636764160,"available":96580620288,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51248,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320592,"statfs":{"total":96636764160,"available":96584286208,"internally_reserved":0,"allocated":2064384,"data_stored":1273646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:01.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3f is not active+clean 2024-04-25T08:40:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.104507+0000', 'last_change': '2024-04-25T08:39:57.104507+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.104507+0000', 'last_undegraded': '2024-04-25T08:39:57.104507+0000', 'last_fullsized': '2024-04-25T08:39:57.104507+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:40:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3e is not active+clean 2024-04-25T08:40:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.108915+0000', 'last_change': '2024-04-25T08:39:57.108915+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.108915+0000', 'last_undegraded': '2024-04-25T08:39:57.108915+0000', 'last_fullsized': '2024-04-25T08:39:57.108915+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:40:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3d is not active+clean 2024-04-25T08:40:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.105895+0000', 'last_change': '2024-04-25T08:39:57.105895+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.105895+0000', 'last_undegraded': '2024-04-25T08:39:57.105895+0000', 'last_fullsized': '2024-04-25T08:39:57.105895+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:40:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3c is not active+clean 2024-04-25T08:40:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.109343+0000', 'last_change': '2024-04-25T08:39:57.109343+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.109343+0000', 'last_undegraded': '2024-04-25T08:39:57.109343+0000', 'last_fullsized': '2024-04-25T08:39:57.109343+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:40:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3b is not active+clean 2024-04-25T08:40:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.105957+0000', 'last_change': '2024-04-25T08:39:57.105957+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.105957+0000', 'last_undegraded': '2024-04-25T08:39:57.105957+0000', 'last_fullsized': '2024-04-25T08:39:57.105957+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:40:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3a is not active+clean 2024-04-25T08:40:01.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.107903+0000', 'last_change': '2024-04-25T08:39:57.107903+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.107903+0000', 'last_undegraded': '2024-04-25T08:39:57.107903+0000', 'last_fullsized': '2024-04-25T08:39:57.107903+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:40:01.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.39 is not active+clean 2024-04-25T08:40:01.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.109081+0000', 'last_change': '2024-04-25T08:39:57.109081+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.109081+0000', 'last_undegraded': '2024-04-25T08:39:57.109081+0000', 'last_fullsized': '2024-04-25T08:39:57.109081+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:40:01.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.38 is not active+clean 2024-04-25T08:40:01.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.106732+0000', 'last_change': '2024-04-25T08:39:57.106732+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.106732+0000', 'last_undegraded': '2024-04-25T08:39:57.106732+0000', 'last_fullsized': '2024-04-25T08:39:57.106732+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:40:01.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.37 is not active+clean 2024-04-25T08:40:01.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.108726+0000', 'last_change': '2024-04-25T08:39:57.108726+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.108726+0000', 'last_undegraded': '2024-04-25T08:39:57.108726+0000', 'last_fullsized': '2024-04-25T08:39:57.108726+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:40:01.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.36 is not active+clean 2024-04-25T08:40:01.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.110497+0000', 'last_change': '2024-04-25T08:39:57.110497+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.110497+0000', 'last_undegraded': '2024-04-25T08:39:57.110497+0000', 'last_fullsized': '2024-04-25T08:39:57.110497+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:40:01.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.35 is not active+clean 2024-04-25T08:40:01.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.105137+0000', 'last_change': '2024-04-25T08:39:57.105137+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.105137+0000', 'last_undegraded': '2024-04-25T08:39:57.105137+0000', 'last_fullsized': '2024-04-25T08:39:57.105137+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:40:01.862 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.34 is not active+clean 2024-04-25T08:40:01.862 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.109567+0000', 'last_change': '2024-04-25T08:39:57.109567+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.109567+0000', 'last_undegraded': '2024-04-25T08:39:57.109567+0000', 'last_fullsized': '2024-04-25T08:39:57.109567+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:40:01.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.33 is not active+clean 2024-04-25T08:40:01.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.115061+0000', 'last_change': '2024-04-25T08:39:57.115061+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.115061+0000', 'last_undegraded': '2024-04-25T08:39:57.115061+0000', 'last_fullsized': '2024-04-25T08:39:57.115061+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:40:01.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.32 is not active+clean 2024-04-25T08:40:01.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.109950+0000', 'last_change': '2024-04-25T08:39:57.109950+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.109950+0000', 'last_undegraded': '2024-04-25T08:39:57.109950+0000', 'last_fullsized': '2024-04-25T08:39:57.109950+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:40:01.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.31 is not active+clean 2024-04-25T08:40:01.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.104679+0000', 'last_change': '2024-04-25T08:39:57.104679+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.104679+0000', 'last_undegraded': '2024-04-25T08:39:57.104679+0000', 'last_fullsized': '2024-04-25T08:39:57.104679+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:40:01.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.30 is not active+clean 2024-04-25T08:40:01.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.109887+0000', 'last_change': '2024-04-25T08:39:57.109887+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.109887+0000', 'last_undegraded': '2024-04-25T08:39:57.109887+0000', 'last_fullsized': '2024-04-25T08:39:57.109887+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:40:01.863 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2f is not active+clean 2024-04-25T08:40:01.863 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.112277+0000', 'last_change': '2024-04-25T08:39:57.112277+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.112277+0000', 'last_undegraded': '2024-04-25T08:39:57.112277+0000', 'last_fullsized': '2024-04-25T08:39:57.112277+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:40:01.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2e is not active+clean 2024-04-25T08:40:01.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.108487+0000', 'last_change': '2024-04-25T08:39:57.108487+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.108487+0000', 'last_undegraded': '2024-04-25T08:39:57.108487+0000', 'last_fullsized': '2024-04-25T08:39:57.108487+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:40:01.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2d is not active+clean 2024-04-25T08:40:01.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.107714+0000', 'last_change': '2024-04-25T08:39:57.107714+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.107714+0000', 'last_undegraded': '2024-04-25T08:39:57.107714+0000', 'last_fullsized': '2024-04-25T08:39:57.107714+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:40:01.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2c is not active+clean 2024-04-25T08:40:01.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.105862+0000', 'last_change': '2024-04-25T08:39:57.105862+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.105862+0000', 'last_undegraded': '2024-04-25T08:39:57.105862+0000', 'last_fullsized': '2024-04-25T08:39:57.105862+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:40:01.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2b is not active+clean 2024-04-25T08:40:01.864 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.108441+0000', 'last_change': '2024-04-25T08:39:57.108441+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.108441+0000', 'last_undegraded': '2024-04-25T08:39:57.108441+0000', 'last_fullsized': '2024-04-25T08:39:57.108441+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:40:01.864 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2a is not active+clean 2024-04-25T08:40:01.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.105711+0000', 'last_change': '2024-04-25T08:39:57.105711+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.105711+0000', 'last_undegraded': '2024-04-25T08:39:57.105711+0000', 'last_fullsized': '2024-04-25T08:39:57.105711+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:40:01.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.29 is not active+clean 2024-04-25T08:40:01.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.107927+0000', 'last_change': '2024-04-25T08:39:57.107927+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.107927+0000', 'last_undegraded': '2024-04-25T08:39:57.107927+0000', 'last_fullsized': '2024-04-25T08:39:57.107927+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:40:01.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.28 is not active+clean 2024-04-25T08:40:01.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.111019+0000', 'last_change': '2024-04-25T08:39:57.111019+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.111019+0000', 'last_undegraded': '2024-04-25T08:39:57.111019+0000', 'last_fullsized': '2024-04-25T08:39:57.111019+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:40:01.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.27 is not active+clean 2024-04-25T08:40:01.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.112909+0000', 'last_change': '2024-04-25T08:39:57.112909+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.112909+0000', 'last_undegraded': '2024-04-25T08:39:57.112909+0000', 'last_fullsized': '2024-04-25T08:39:57.112909+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:40:01.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.26 is not active+clean 2024-04-25T08:40:01.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.106551+0000', 'last_change': '2024-04-25T08:39:57.106551+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.106551+0000', 'last_undegraded': '2024-04-25T08:39:57.106551+0000', 'last_fullsized': '2024-04-25T08:39:57.106551+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:40:01.865 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.25 is not active+clean 2024-04-25T08:40:01.865 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.106831+0000', 'last_change': '2024-04-25T08:39:57.106831+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.106831+0000', 'last_undegraded': '2024-04-25T08:39:57.106831+0000', 'last_fullsized': '2024-04-25T08:39:57.106831+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:40:01.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.24 is not active+clean 2024-04-25T08:40:01.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.109055+0000', 'last_change': '2024-04-25T08:39:57.109055+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.109055+0000', 'last_undegraded': '2024-04-25T08:39:57.109055+0000', 'last_fullsized': '2024-04-25T08:39:57.109055+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:40:01.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.23 is not active+clean 2024-04-25T08:40:01.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.106587+0000', 'last_change': '2024-04-25T08:39:57.106587+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.106587+0000', 'last_undegraded': '2024-04-25T08:39:57.106587+0000', 'last_fullsized': '2024-04-25T08:39:57.106587+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:40:01.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.22 is not active+clean 2024-04-25T08:40:01.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.109521+0000', 'last_change': '2024-04-25T08:39:57.109521+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.109521+0000', 'last_undegraded': '2024-04-25T08:39:57.109521+0000', 'last_fullsized': '2024-04-25T08:39:57.109521+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:40:01.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.21 is not active+clean 2024-04-25T08:40:01.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.111398+0000', 'last_change': '2024-04-25T08:39:57.111398+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.111398+0000', 'last_undegraded': '2024-04-25T08:39:57.111398+0000', 'last_fullsized': '2024-04-25T08:39:57.111398+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:40:01.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.20 is not active+clean 2024-04-25T08:40:01.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.110178+0000', 'last_change': '2024-04-25T08:39:57.110178+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.110178+0000', 'last_undegraded': '2024-04-25T08:39:57.110178+0000', 'last_fullsized': '2024-04-25T08:39:57.110178+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:40:01.866 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1c is not active+clean 2024-04-25T08:40:01.866 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.109690+0000', 'last_change': '2024-04-25T08:39:57.109690+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.109690+0000', 'last_undegraded': '2024-04-25T08:39:57.109690+0000', 'last_fullsized': '2024-04-25T08:39:57.109690+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:40:01.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1d is not active+clean 2024-04-25T08:40:01.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.108536+0000', 'last_change': '2024-04-25T08:39:57.108536+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.108536+0000', 'last_undegraded': '2024-04-25T08:39:57.108536+0000', 'last_fullsized': '2024-04-25T08:39:57.108536+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:40:01.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1e is not active+clean 2024-04-25T08:40:01.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.111346+0000', 'last_change': '2024-04-25T08:39:57.111346+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.111346+0000', 'last_undegraded': '2024-04-25T08:39:57.111346+0000', 'last_fullsized': '2024-04-25T08:39:57.111346+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:40:01.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1f is not active+clean 2024-04-25T08:40:01.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.107451+0000', 'last_change': '2024-04-25T08:39:57.107451+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.107451+0000', 'last_undegraded': '2024-04-25T08:39:57.107451+0000', 'last_fullsized': '2024-04-25T08:39:57.107451+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:40:01.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.18 is not active+clean 2024-04-25T08:40:01.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.108782+0000', 'last_change': '2024-04-25T08:39:57.108782+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.108782+0000', 'last_undegraded': '2024-04-25T08:39:57.108782+0000', 'last_fullsized': '2024-04-25T08:39:57.108782+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:40:01.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.19 is not active+clean 2024-04-25T08:40:01.867 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.105610+0000', 'last_change': '2024-04-25T08:39:57.105610+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.105610+0000', 'last_undegraded': '2024-04-25T08:39:57.105610+0000', 'last_fullsized': '2024-04-25T08:39:57.105610+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:40:01.867 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1a is not active+clean 2024-04-25T08:40:01.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.107893+0000', 'last_change': '2024-04-25T08:39:57.107893+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.107893+0000', 'last_undegraded': '2024-04-25T08:39:57.107893+0000', 'last_fullsized': '2024-04-25T08:39:57.107893+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:40:01.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1b is not active+clean 2024-04-25T08:40:01.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.114648+0000', 'last_change': '2024-04-25T08:39:57.114648+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.114648+0000', 'last_undegraded': '2024-04-25T08:39:57.114648+0000', 'last_fullsized': '2024-04-25T08:39:57.114648+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:40:01.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.14 is not active+clean 2024-04-25T08:40:01.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.104630+0000', 'last_change': '2024-04-25T08:39:57.104630+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.104630+0000', 'last_undegraded': '2024-04-25T08:39:57.104630+0000', 'last_fullsized': '2024-04-25T08:39:57.104630+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:40:01.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.15 is not active+clean 2024-04-25T08:40:01.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.113272+0000', 'last_change': '2024-04-25T08:39:57.113272+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.113272+0000', 'last_undegraded': '2024-04-25T08:39:57.113272+0000', 'last_fullsized': '2024-04-25T08:39:57.113272+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:40:01.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.16 is not active+clean 2024-04-25T08:40:01.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.106051+0000', 'last_change': '2024-04-25T08:39:57.106051+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.106051+0000', 'last_undegraded': '2024-04-25T08:39:57.106051+0000', 'last_fullsized': '2024-04-25T08:39:57.106051+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:40:01.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.17 is not active+clean 2024-04-25T08:40:01.868 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.112478+0000', 'last_change': '2024-04-25T08:39:57.112478+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.112478+0000', 'last_undegraded': '2024-04-25T08:39:57.112478+0000', 'last_fullsized': '2024-04-25T08:39:57.112478+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:40:01.868 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.10 is not active+clean 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.105978+0000', 'last_change': '2024-04-25T08:39:57.105978+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.105978+0000', 'last_undegraded': '2024-04-25T08:39:57.105978+0000', 'last_fullsized': '2024-04-25T08:39:57.105978+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.11 is not active+clean 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.108899+0000', 'last_change': '2024-04-25T08:39:57.108899+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.108899+0000', 'last_undegraded': '2024-04-25T08:39:57.108899+0000', 'last_fullsized': '2024-04-25T08:39:57.108899+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.12 is not active+clean 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.106645+0000', 'last_change': '2024-04-25T08:39:57.106645+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.106645+0000', 'last_undegraded': '2024-04-25T08:39:57.106645+0000', 'last_fullsized': '2024-04-25T08:39:57.106645+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.c is not active+clean 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.116232+0000', 'last_change': '2024-04-25T08:39:57.116232+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.116232+0000', 'last_undegraded': '2024-04-25T08:39:57.116232+0000', 'last_fullsized': '2024-04-25T08:39:57.116232+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.d is not active+clean 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.104557+0000', 'last_change': '2024-04-25T08:39:57.104557+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.104557+0000', 'last_undegraded': '2024-04-25T08:39:57.104557+0000', 'last_fullsized': '2024-04-25T08:39:57.104557+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.e is not active+clean 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.107381+0000', 'last_change': '2024-04-25T08:39:57.107381+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.107381+0000', 'last_undegraded': '2024-04-25T08:39:57.107381+0000', 'last_fullsized': '2024-04-25T08:39:57.107381+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.f is not active+clean 2024-04-25T08:40:01.869 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.111993+0000', 'last_change': '2024-04-25T08:39:57.111993+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.111993+0000', 'last_undegraded': '2024-04-25T08:39:57.111993+0000', 'last_fullsized': '2024-04-25T08:39:57.111993+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:40:01.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.8 is not active+clean 2024-04-25T08:40:01.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.113672+0000', 'last_change': '2024-04-25T08:39:57.113672+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.113672+0000', 'last_undegraded': '2024-04-25T08:39:57.113672+0000', 'last_fullsized': '2024-04-25T08:39:57.113672+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:40:01.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.9 is not active+clean 2024-04-25T08:40:01.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.110704+0000', 'last_change': '2024-04-25T08:39:57.110704+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.110704+0000', 'last_undegraded': '2024-04-25T08:39:57.110704+0000', 'last_fullsized': '2024-04-25T08:39:57.110704+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:40:01.870 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.a is not active+clean 2024-04-25T08:40:01.870 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.107472+0000', 'last_change': '2024-04-25T08:39:57.107472+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.107472+0000', 'last_undegraded': '2024-04-25T08:39:57.107472+0000', 'last_fullsized': '2024-04-25T08:39:57.107472+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:40:01.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.b is not active+clean 2024-04-25T08:40:01.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.111344+0000', 'last_change': '2024-04-25T08:39:57.111344+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.111344+0000', 'last_undegraded': '2024-04-25T08:39:57.111344+0000', 'last_fullsized': '2024-04-25T08:39:57.111344+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:40:01.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.4 is not active+clean 2024-04-25T08:40:01.871 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.108631+0000', 'last_change': '2024-04-25T08:39:57.108631+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.108631+0000', 'last_undegraded': '2024-04-25T08:39:57.108631+0000', 'last_fullsized': '2024-04-25T08:39:57.108631+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:40:01.871 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.5 is not active+clean 2024-04-25T08:40:01.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.111394+0000', 'last_change': '2024-04-25T08:39:57.111394+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.111394+0000', 'last_undegraded': '2024-04-25T08:39:57.111394+0000', 'last_fullsized': '2024-04-25T08:39:57.111394+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:40:01.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.6 is not active+clean 2024-04-25T08:40:01.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.105610+0000', 'last_change': '2024-04-25T08:39:57.105610+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.105610+0000', 'last_undegraded': '2024-04-25T08:39:57.105610+0000', 'last_fullsized': '2024-04-25T08:39:57.105610+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:40:01.872 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.7 is not active+clean 2024-04-25T08:40:01.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.106488+0000', 'last_change': '2024-04-25T08:39:57.106488+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.106488+0000', 'last_undegraded': '2024-04-25T08:39:57.106488+0000', 'last_fullsized': '2024-04-25T08:39:57.106488+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:40:01.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.0 is not active+clean 2024-04-25T08:40:01.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.110350+0000', 'last_change': '2024-04-25T08:39:57.110350+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.110350+0000', 'last_undegraded': '2024-04-25T08:39:57.110350+0000', 'last_fullsized': '2024-04-25T08:39:57.110350+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:40:01.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1 is not active+clean 2024-04-25T08:40:01.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.108211+0000', 'last_change': '2024-04-25T08:39:57.108211+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.108211+0000', 'last_undegraded': '2024-04-25T08:39:57.108211+0000', 'last_fullsized': '2024-04-25T08:39:57.108211+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:40:01.873 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2 is not active+clean 2024-04-25T08:40:01.873 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.104891+0000', 'last_change': '2024-04-25T08:39:57.104891+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.104891+0000', 'last_undegraded': '2024-04-25T08:39:57.104891+0000', 'last_fullsized': '2024-04-25T08:39:57.104891+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:40:01.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3 is not active+clean 2024-04-25T08:40:01.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.114248+0000', 'last_change': '2024-04-25T08:39:57.114248+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.114248+0000', 'last_undegraded': '2024-04-25T08:39:57.114248+0000', 'last_fullsized': '2024-04-25T08:39:57.114248+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:40:01.874 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.13 is not active+clean 2024-04-25T08:40:01.874 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:39:57.106294+0000', 'last_change': '2024-04-25T08:39:57.106294+0000', 'last_active': '2024-04-25T08:39:57.096572+0000', 'last_peered': '2024-04-25T08:39:57.096572+0000', 'last_clean': '2024-04-25T08:39:57.096572+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:39:57.106294+0000', 'last_undegraded': '2024-04-25T08:39:57.106294+0000', 'last_fullsized': '2024-04-25T08:39:57.106294+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'last_clean_scrub_stamp': '2024-04-25T08:39:57.096572+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:40:01.874 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:40:01.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.981+0000 7fe13a302640 1 Processor -- start 2024-04-25T08:40:01.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.981+0000 7fe13a302640 1 -- start start 2024-04-25T08:40:01.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.982+0000 7fe13a302640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe13415dc10 0x7fe1340ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.982+0000 7fe13a302640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1340d3e70 con 0x7fe1340d0f60 2024-04-25T08:40:01.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.982+0000 7fe13a302640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1340d4150 con 0x7fe1340ced60 2024-04-25T08:40:01.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.982+0000 7fe13a302640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13415dff0 con 0x7fe13415dc10 2024-04-25T08:40:01.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.982+0000 7fe133fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe13415dc10 0x7fe1340ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.982+0000 7fe13889f640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe1340d0f60 0x7fe1340b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45026/0 (socket says 172.21.15.38:45026) 2024-04-25T08:40:01.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.982+0000 7fe13889f640 1 -- 172.21.15.38:0/3529575219 learned_addr learned my addr 172.21.15.38:0/3529575219 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:01.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.982+0000 7fe133fff640 1 -- 172.21.15.38:0/3529575219 >> v1:172.21.15.107:6789/0 conn(0x7fe1340ced60 legacy=0x7fe1340d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:01.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.982+0000 7fe133fff640 1 -- 172.21.15.38:0/3529575219 >> v1:172.21.15.38:6789/0 conn(0x7fe1340d0f60 legacy=0x7fe1340b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.983+0000 7fe132ffd640 1 -- 172.21.15.38:0/3529575219 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351760898 0 0) 0x7fe1340d3e70 con 0x7fe1340d0f60 2024-04-25T08:40:01.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.983+0000 7fe133fff640 1 -- 172.21.15.38:0/3529575219 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1340d4430 con 0x7fe13415dc10 2024-04-25T08:40:01.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.983+0000 7fe133fff640 1 --2- 172.21.15.38:0/3529575219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe13415dc10 0x7fe1340ce820 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fe1240097a0 tx=0x7fe12402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160e9838d81d83c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:01.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.983+0000 7fe132ffd640 1 -- 172.21.15.38:0/3529575219 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124037070 con 0x7fe13415dc10 2024-04-25T08:40:01.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.983+0000 7fe132ffd640 1 -- 172.21.15.38:0/3529575219 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12402ae50 con 0x7fe13415dc10 2024-04-25T08:40:01.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.983+0000 7fe132ffd640 1 -- 172.21.15.38:0/3529575219 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe124032660 con 0x7fe13415dc10 2024-04-25T08:40:01.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.984+0000 7fe13a302640 1 -- 172.21.15.38:0/3529575219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe13415dc10 msgr2=0x7fe1340ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.984+0000 7fe13a302640 1 --2- 172.21.15.38:0/3529575219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe13415dc10 0x7fe1340ce820 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fe1240097a0 tx=0x7fe12402a140 comp rx=0 tx=0).stop 2024-04-25T08:40:01.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.984+0000 7fe13a302640 1 -- 172.21.15.38:0/3529575219 shutdown_connections 2024-04-25T08:40:01.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.984+0000 7fe13a302640 1 --2- 172.21.15.38:0/3529575219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe13415dc10 0x7fe1340ce820 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.984+0000 7fe13a302640 1 -- 172.21.15.38:0/3529575219 >> 172.21.15.38:0/3529575219 conn(0x7fe134066d10 msgr2=0x7fe134067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:01.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.984+0000 7fe13a302640 1 -- 172.21.15.38:0/3529575219 shutdown_connections 2024-04-25T08:40:01.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.984+0000 7fe13a302640 1 -- 172.21.15.38:0/3529575219 wait complete. 2024-04-25T08:40:01.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.985+0000 7fe13a302640 1 Processor -- start 2024-04-25T08:40:01.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.985+0000 7fe13a302640 1 -- start start 2024-04-25T08:40:01.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13a302640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1340ced60 0x7fe13415be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13a302640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1340d0f60 0x7fe1340cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13a302640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe13415dc10 0x7fe1340ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13a302640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1340d4710 con 0x7fe13415dc10 2024-04-25T08:40:01.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13a302640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1340b80f0 con 0x7fe1340d0f60 2024-04-25T08:40:01.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13a302640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1340b8270 con 0x7fe1340ced60 2024-04-25T08:40:01.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13889f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe13415dc10 0x7fe1340ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13889f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe13415dc10 0x7fe1340ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44544/0 (socket says 172.21.15.38:44544) 2024-04-25T08:40:01.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13889f640 1 -- 172.21.15.38:0/3294268320 learned_addr learned my addr 172.21.15.38:0/3294268320 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe1337fe640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1340d0f60 0x7fe1340cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe133fff640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1340ced60 0x7fe13415be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13889f640 1 -- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1340ced60 msgr2=0x7fe13415be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13889f640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1340ced60 0x7fe13415be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13889f640 1 -- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1340d0f60 msgr2=0x7fe1340cc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13889f640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1340d0f60 0x7fe1340cc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe13889f640 1 -- 172.21.15.38:0/3294268320 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1340ca620 con 0x7fe13415dc10 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.986+0000 7fe1337fe640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1340d0f60 0x7fe1340cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.987+0000 7fe13889f640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe13415dc10 0x7fe1340ccf20 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00c7a0 tx=0x7fe12c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.987+0000 7fe130ff9640 1 -- 172.21.15.38:0/3294268320 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c018070 con 0x7fe13415dc10 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.987+0000 7fe130ff9640 1 -- 172.21.15.38:0/3294268320 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12c00a640 con 0x7fe13415dc10 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.987+0000 7fe13a302640 1 -- 172.21.15.38:0/3294268320 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1340aa500 con 0x7fe13415dc10 2024-04-25T08:40:01.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.987+0000 7fe130ff9640 1 -- 172.21.15.38:0/3294268320 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c00aa70 con 0x7fe13415dc10 2024-04-25T08:40:01.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.987+0000 7fe13a302640 1 -- 172.21.15.38:0/3294268320 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13415c360 con 0x7fe13415dc10 2024-04-25T08:40:01.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.988+0000 7fe13a302640 1 -- 172.21.15.38:0/3294268320 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1340aa2f0 con 0x7fe13415dc10 2024-04-25T08:40:01.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.988+0000 7fe130ff9640 1 -- 172.21.15.38:0/3294268320 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe12c01b070 con 0x7fe13415dc10 2024-04-25T08:40:01.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.989+0000 7fe130ff9640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe12006b4d0 0x7fe12006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:01.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.989+0000 7fe130ff9640 1 -- 172.21.15.38:0/3294268320 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe12c0899b0 con 0x7fe13415dc10 2024-04-25T08:40:01.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.989+0000 7fe133fff640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe12006b4d0 0x7fe12006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:01.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.989+0000 7fe133fff640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe12006b4d0 0x7fe12006d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe124034000 tx=0x7fe12402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:01.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:01.993+0000 7fe130ff9640 1 -- 172.21.15.38:0/3294268320 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe12c0586a0 con 0x7fe13415dc10 2024-04-25T08:40:02.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.243+0000 7fe13a302640 1 -- 172.21.15.38:0/3294268320 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe134068e80 con 0x7fe13415dc10 2024-04-25T08:40:02.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.243+0000 7fe130ff9640 1 -- 172.21.15.38:0/3294268320 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1584 (secure 0 0 0) 0x7fe12c057fa0 con 0x7fe13415dc10 2024-04-25T08:40:02.237 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:02.237 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":531,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":510058496,"bytes_avail":772584054784,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":70,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":7639}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T08:39:39.318198+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:40:02.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.245+0000 7fe13a302640 1 -- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe12006b4d0 msgr2=0x7fe12006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.245+0000 7fe13a302640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe12006b4d0 0x7fe12006d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe124034000 tx=0x7fe12402e000 comp rx=0 tx=0).stop 2024-04-25T08:40:02.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.245+0000 7fe13a302640 1 -- 172.21.15.38:0/3294268320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe13415dc10 msgr2=0x7fe1340ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.245+0000 7fe13a302640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe13415dc10 0x7fe1340ccf20 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00c7a0 tx=0x7fe12c00cc60 comp rx=0 tx=0).stop 2024-04-25T08:40:02.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.245+0000 7fe13a302640 1 -- 172.21.15.38:0/3294268320 shutdown_connections 2024-04-25T08:40:02.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.245+0000 7fe13a302640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe13415dc10 0x7fe1340ccf20 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.245+0000 7fe13a302640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1340d0f60 0x7fe1340cc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.245+0000 7fe13a302640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe12006b4d0 0x7fe12006d990 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.245+0000 7fe13a302640 1 --2- 172.21.15.38:0/3294268320 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1340ced60 0x7fe13415be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.245+0000 7fe13a302640 1 -- 172.21.15.38:0/3294268320 >> 172.21.15.38:0/3294268320 conn(0x7fe134066d10 msgr2=0x7fe13415ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:02.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.245+0000 7fe13a302640 1 -- 172.21.15.38:0/3294268320 shutdown_connections 2024-04-25T08:40:02.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.245+0000 7fe13a302640 1 -- 172.21.15.38:0/3294268320 wait complete. 2024-04-25T08:40:02.249 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:40:02.249 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:02.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.357+0000 7ff2ac0cb640 1 Processor -- start 2024-04-25T08:40:02.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.357+0000 7ff2ac0cb640 1 -- start start 2024-04-25T08:40:02.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.358+0000 7ff2ac0cb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2a40d3900 0x7ff2a40d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.358+0000 7ff2ac0cb640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a40d3e70 con 0x7ff2a415dc10 2024-04-25T08:40:02.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.358+0000 7ff2ac0cb640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a40d4150 con 0x7ff2a40d0d90 2024-04-25T08:40:02.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.358+0000 7ff2ac0cb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a415dff0 con 0x7ff2a40d3900 2024-04-25T08:40:02.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.358+0000 7ff2a9e67640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff2a415dc10 0x7ff2a40ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45040/0 (socket says 172.21.15.38:45040) 2024-04-25T08:40:02.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.358+0000 7ff2a9e67640 1 -- 172.21.15.38:0/2226768931 learned_addr learned my addr 172.21.15.38:0/2226768931 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:02.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784927282 0 0) 0x7ff2a40d3e70 con 0x7ff2a415dc10 2024-04-25T08:40:02.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a9666640 1 --2- 172.21.15.38:0/2226768931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2a40d3900 0x7ff2a40d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff290003520 con 0x7ff2a415dc10 2024-04-25T08:40:02.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2447989864 0 0) 0x7ff290003520 con 0x7ff2a415dc10 2024-04-25T08:40:02.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2a40d3900 msgr2=0x7ff2a40d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:02.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 --2- 172.21.15.38:0/2226768931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2a40d3900 0x7ff2a40d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 >> v1:172.21.15.107:6789/0 conn(0x7ff2a40d0d90 legacy=0x7ff2a40b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40d4430 con 0x7ff2a415dc10 2024-04-25T08:40:02.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7ff29c001670 con 0x7ff2a415dc10 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 >> v1:172.21.15.38:6789/0 conn(0x7ff2a415dc10 legacy=0x7ff2a40ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 --2- 172.21.15.38:0/2226768931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2900039e0 0x7ff290003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 --2- 172.21.15.38:0/2226768931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2a40d3900 0x7ff290004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 --2- 172.21.15.38:0/2226768931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff290004d40 0x7ff2900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a415dff0 con 0x7ff290004d40 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29000ba70 con 0x7ff2900039e0 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29000bc70 con 0x7ff2a40d3900 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2aa668640 1 --2- 172.21.15.38:0/2226768931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff290004d40 0x7ff2900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a9e67640 1 --2- 172.21.15.38:0/2226768931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2900039e0 0x7ff290003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2a9666640 1 --2- 172.21.15.38:0/2226768931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2a40d3900 0x7ff290004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.359+0000 7ff2aa668640 1 -- 172.21.15.38:0/2226768931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2a40d3900 msgr2=0x7ff290004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.360+0000 7ff2aa668640 1 --2- 172.21.15.38:0/2226768931 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2a40d3900 0x7ff290004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.360+0000 7ff2aa668640 1 -- 172.21.15.38:0/2226768931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2900039e0 msgr2=0x7ff290003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:02.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.360+0000 7ff2aa668640 1 --2- 172.21.15.38:0/2226768931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2900039e0 0x7ff290003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.360+0000 7ff2aa668640 1 -- 172.21.15.38:0/2226768931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff290003520 con 0x7ff290004d40 2024-04-25T08:40:02.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.360+0000 7ff2aa668640 1 --2- 172.21.15.38:0/2226768931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff290004d40 0x7ff2900051c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0001f60 tx=0x7ff290005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c872e5e0bef888 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.360+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0001b30 con 0x7ff290004d40 2024-04-25T08:40:02.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.360+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a0031620 con 0x7ff290004d40 2024-04-25T08:40:02.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.360+0000 7ff2a8e65640 1 -- 172.21.15.38:0/2226768931 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0031a10 con 0x7ff290004d40 2024-04-25T08:40:02.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.361+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/2226768931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff290004d40 msgr2=0x7ff2900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.361+0000 7ff2ac0cb640 1 --2- 172.21.15.38:0/2226768931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff290004d40 0x7ff2900051c0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0001f60 tx=0x7ff290005830 comp rx=0 tx=0).stop 2024-04-25T08:40:02.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.361+0000 7ff2aa668640 1 -- 172.21.15.38:0/2226768931 reap_dead start 2024-04-25T08:40:02.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.361+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/2226768931 shutdown_connections 2024-04-25T08:40:02.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.361+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/2226768931 >> 172.21.15.38:0/2226768931 conn(0x7ff2a4066d10 msgr2=0x7ff2a4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:02.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.361+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/2226768931 shutdown_connections 2024-04-25T08:40:02.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.361+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/2226768931 wait complete. 2024-04-25T08:40:02.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.362+0000 7ff2ac0cb640 1 Processor -- start 2024-04-25T08:40:02.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.362+0000 7ff2ac0cb640 1 -- start start 2024-04-25T08:40:02.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2ac0cb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a40ccc50 0x7ff2a415b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2ac0cb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2a40cd030 0x7ff2a415b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2ac0cb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2a41f6f60 0x7ff2a41f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2ac0cb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40d4710 con 0x7ff2a40ccc50 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2ac0cb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40cef30 con 0x7ff2a41f6f60 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2ac0cb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40b80f0 con 0x7ff2a40cd030 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2a9e67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a40ccc50 0x7ff2a415b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2a9e67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a40ccc50 0x7ff2a415b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44560/0 (socket says 172.21.15.38:44560) 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2a9e67640 1 -- 172.21.15.38:0/4259899323 learned_addr learned my addr 172.21.15.38:0/4259899323 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2a9e67640 1 -- 172.21.15.38:0/4259899323 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2a40cd030 msgr2=0x7ff2a415b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2a9e67640 1 --2- 172.21.15.38:0/4259899323 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2a40cd030 0x7ff2a415b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2a9e67640 1 -- 172.21.15.38:0/4259899323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2a41f6f60 msgr2=0x7ff2a41f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2a9e67640 1 --2- 172.21.15.38:0/4259899323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2a41f6f60 0x7ff2a41f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.363+0000 7ff2a9e67640 1 -- 172.21.15.38:0/4259899323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a415bc50 con 0x7ff2a40ccc50 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.364+0000 7ff2a9e67640 1 --2- 172.21.15.38:0/4259899323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a40ccc50 0x7ff2a415b1d0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7ff29c007370 tx=0x7ff29c007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.364+0000 7ff29a7fc640 1 -- 172.21.15.38:0/4259899323 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c01a070 con 0x7ff2a40ccc50 2024-04-25T08:40:02.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.364+0000 7ff29a7fc640 1 -- 172.21.15.38:0/4259899323 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c00f860 con 0x7ff2a40ccc50 2024-04-25T08:40:02.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.364+0000 7ff29a7fc640 1 -- 172.21.15.38:0/4259899323 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c00fc50 con 0x7ff2a40ccc50 2024-04-25T08:40:02.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.364+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/4259899323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a41f7cd0 con 0x7ff2a40ccc50 2024-04-25T08:40:02.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.364+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/4259899323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a40aa5d0 con 0x7ff2a40ccc50 2024-04-25T08:40:02.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.365+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/4259899323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a4068e80 con 0x7ff2a40ccc50 2024-04-25T08:40:02.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.365+0000 7ff29a7fc640 1 -- 172.21.15.38:0/4259899323 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff29c01d070 con 0x7ff2a40ccc50 2024-04-25T08:40:02.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.365+0000 7ff29a7fc640 1 --2- 172.21.15.38:0/4259899323 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff28806b4d0 0x7ff28806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:02.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.366+0000 7ff29a7fc640 1 -- 172.21.15.38:0/4259899323 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff29c089b90 con 0x7ff2a40ccc50 2024-04-25T08:40:02.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.366+0000 7ff2a9666640 1 --2- 172.21.15.38:0/4259899323 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff28806b4d0 0x7ff28806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:02.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.366+0000 7ff2a9666640 1 --2- 172.21.15.38:0/4259899323 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff28806b4d0 0x7ff28806d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40d2790 tx=0x7ff2940095d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:02.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.370+0000 7ff29a7fc640 1 -- 172.21.15.38:0/4259899323 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff29c058910 con 0x7ff2a40ccc50 2024-04-25T08:40:02.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.546+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/4259899323 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2a40cda00 con 0x7ff28806b4d0 2024-04-25T08:40:02.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.552+0000 7ff29a7fc640 1 -- 172.21.15.38:0/4259899323 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247088 (secure 0 0 0) 0x7ff2a40cda00 con 0x7ff28806b4d0 2024-04-25T08:40:02.548 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:02.549 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:40:02.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.555+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/4259899323 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff28806b4d0 msgr2=0x7ff28806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.555+0000 7ff2ac0cb640 1 --2- 172.21.15.38:0/4259899323 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff28806b4d0 0x7ff28806d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40d2790 tx=0x7ff2940095d0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.555+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/4259899323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a40ccc50 msgr2=0x7ff2a415b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:02.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.555+0000 7ff2ac0cb640 1 --2- 172.21.15.38:0/4259899323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a40ccc50 0x7ff2a415b1d0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7ff29c007370 tx=0x7ff29c007830 comp rx=0 tx=0).stop 2024-04-25T08:40:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.555+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/4259899323 shutdown_connections 2024-04-25T08:40:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.555+0000 7ff2ac0cb640 1 --2- 172.21.15.38:0/4259899323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2a41f6f60 0x7ff2a41f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.555+0000 7ff2ac0cb640 1 --2- 172.21.15.38:0/4259899323 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff28806b4d0 0x7ff28806d990 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.555+0000 7ff2ac0cb640 1 --2- 172.21.15.38:0/4259899323 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff2a40cd030 0x7ff2a415b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.555+0000 7ff2ac0cb640 1 --2- 172.21.15.38:0/4259899323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff2a40ccc50 0x7ff2a415b1d0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.555+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/4259899323 >> 172.21.15.38:0/4259899323 conn(0x7ff2a4066d10 msgr2=0x7ff2a415ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.555+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/4259899323 shutdown_connections 2024-04-25T08:40:02.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:02.556+0000 7ff2ac0cb640 1 -- 172.21.15.38:0/4259899323 wait complete. 2024-04-25T08:40:02.559 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-04-25T08:40:01.323124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":498104,"kb_used_data":12648,"kb_used_omap":12,"kb_used_meta":485299,"kb_avail":754476616,"statfs":{"total":773094113280,"available":772584054784,"internally_reserved":0,"allocated":12951552,"data_stored":6646960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":496946736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.223817"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104507+0000","last_change":"2024-04-25T08:39:57.104507+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104507+0000","last_undegraded":"2024-04-25T08:39:57.104507+0000","last_fullsized":"2024-04-25T08:39:57.104507+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108915+0000","last_change":"2024-04-25T08:39:57.108915+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108915+0000","last_undegraded":"2024-04-25T08:39:57.108915+0000","last_fullsized":"2024-04-25T08:39:57.108915+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105895+0000","last_change":"2024-04-25T08:39:57.105895+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105895+0000","last_undegraded":"2024-04-25T08:39:57.105895+0000","last_fullsized":"2024-04-25T08:39:57.105895+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109343+0000","last_change":"2024-04-25T08:39:57.109343+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109343+0000","last_undegraded":"2024-04-25T08:39:57.109343+0000","last_fullsized":"2024-04-25T08:39:57.109343+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105957+0000","last_change":"2024-04-25T08:39:57.105957+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105957+0000","last_undegraded":"2024-04-25T08:39:57.105957+0000","last_fullsized":"2024-04-25T08:39:57.105957+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107903+0000","last_change":"2024-04-25T08:39:57.107903+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107903+0000","last_undegraded":"2024-04-25T08:39:57.107903+0000","last_fullsized":"2024-04-25T08:39:57.107903+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109081+0000","last_change":"2024-04-25T08:39:57.109081+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109081+0000","last_undegraded":"2024-04-25T08:39:57.109081+0000","last_fullsized":"2024-04-25T08:39:57.109081+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106732+0000","last_change":"2024-04-25T08:39:57.106732+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106732+0000","last_undegraded":"2024-04-25T08:39:57.106732+0000","last_fullsized":"2024-04-25T08:39:57.106732+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108726+0000","last_change":"2024-04-25T08:39:57.108726+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108726+0000","last_undegraded":"2024-04-25T08:39:57.108726+0000","last_fullsized":"2024-04-25T08:39:57.108726+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110497+0000","last_change":"2024-04-25T08:39:57.110497+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110497+0000","last_undegraded":"2024-04-25T08:39:57.110497+0000","last_fullsized":"2024-04-25T08:39:57.110497+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105137+0000","last_change":"2024-04-25T08:39:57.105137+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105137+0000","last_undegraded":"2024-04-25T08:39:57.105137+0000","last_fullsized":"2024-04-25T08:39:57.105137+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109567+0000","last_change":"2024-04-25T08:39:57.109567+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109567+0000","last_undegraded":"2024-04-25T08:39:57.109567+0000","last_fullsized":"2024-04-25T08:39:57.109567+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.115061+0000","last_change":"2024-04-25T08:39:57.115061+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.115061+0000","last_undegraded":"2024-04-25T08:39:57.115061+0000","last_fullsized":"2024-04-25T08:39:57.115061+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109950+0000","last_change":"2024-04-25T08:39:57.109950+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109950+0000","last_undegraded":"2024-04-25T08:39:57.109950+0000","last_fullsized":"2024-04-25T08:39:57.109950+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104679+0000","last_change":"2024-04-25T08:39:57.104679+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104679+0000","last_undegraded":"2024-04-25T08:39:57.104679+0000","last_fullsized":"2024-04-25T08:39:57.104679+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109887+0000","last_change":"2024-04-25T08:39:57.109887+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109887+0000","last_undegraded":"2024-04-25T08:39:57.109887+0000","last_fullsized":"2024-04-25T08:39:57.109887+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.112277+0000","last_change":"2024-04-25T08:39:57.112277+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.112277+0000","last_undegraded":"2024-04-25T08:39:57.112277+0000","last_fullsized":"2024-04-25T08:39:57.112277+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108487+0000","last_change":"2024-04-25T08:39:57.108487+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108487+0000","last_undegraded":"2024-04-25T08:39:57.108487+0000","last_fullsized":"2024-04-25T08:39:57.108487+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107714+0000","last_change":"2024-04-25T08:39:57.107714+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107714+0000","last_undegraded":"2024-04-25T08:39:57.107714+0000","last_fullsized":"2024-04-25T08:39:57.107714+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105862+0000","last_change":"2024-04-25T08:39:57.105862+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105862+0000","last_undegraded":"2024-04-25T08:39:57.105862+0000","last_fullsized":"2024-04-25T08:39:57.105862+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108441+0000","last_change":"2024-04-25T08:39:57.108441+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108441+0000","last_undegraded":"2024-04-25T08:39:57.108441+0000","last_fullsized":"2024-04-25T08:39:57.108441+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105711+0000","last_change":"2024-04-25T08:39:57.105711+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105711+0000","last_undegraded":"2024-04-25T08:39:57.105711+0000","last_fullsized":"2024-04-25T08:39:57.105711+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107927+0000","last_change":"2024-04-25T08:39:57.107927+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107927+0000","last_undegraded":"2024-04-25T08:39:57.107927+0000","last_fullsized":"2024-04-25T08:39:57.107927+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111019+0000","last_change":"2024-04-25T08:39:57.111019+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111019+0000","last_undegraded":"2024-04-25T08:39:57.111019+0000","last_fullsized":"2024-04-25T08:39:57.111019+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.112909+0000","last_change":"2024-04-25T08:39:57.112909+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.112909+0000","last_undegraded":"2024-04-25T08:39:57.112909+0000","last_fullsized":"2024-04-25T08:39:57.112909+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106551+0000","last_change":"2024-04-25T08:39:57.106551+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106551+0000","last_undegraded":"2024-04-25T08:39:57.106551+0000","last_fullsized":"2024-04-25T08:39:57.106551+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106831+0000","last_change":"2024-04-25T08:39:57.106831+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106831+0000","last_undegraded":"2024-04-25T08:39:57.106831+0000","last_fullsized":"2024-04-25T08:39:57.106831+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109055+0000","last_change":"2024-04-25T08:39:57.109055+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109055+0000","last_undegraded":"2024-04-25T08:39:57.109055+0000","last_fullsized":"2024-04-25T08:39:57.109055+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106587+0000","last_change":"2024-04-25T08:39:57.106587+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106587+0000","last_undegraded":"2024-04-25T08:39:57.106587+0000","last_fullsized":"2024-04-25T08:39:57.106587+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109521+0000","last_change":"2024-04-25T08:39:57.109521+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109521+0000","last_undegraded":"2024-04-25T08:39:57.109521+0000","last_fullsized":"2024-04-25T08:39:57.109521+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111398+0000","last_change":"2024-04-25T08:39:57.111398+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111398+0000","last_undegraded":"2024-04-25T08:39:57.111398+0000","last_fullsized":"2024-04-25T08:39:57.111398+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110178+0000","last_change":"2024-04-25T08:39:57.110178+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110178+0000","last_undegraded":"2024-04-25T08:39:57.110178+0000","last_fullsized":"2024-04-25T08:39:57.110178+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.109690+0000","last_change":"2024-04-25T08:39:57.109690+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.109690+0000","last_undegraded":"2024-04-25T08:39:57.109690+0000","last_fullsized":"2024-04-25T08:39:57.109690+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.112046+0000","last_change":"2024-04-25T08:39:57.111533+0000","last_active":"2024-04-25T08:39:57.112046+0000","last_peered":"2024-04-25T08:39:57.112046+0000","last_clean":"2024-04-25T08:39:57.112046+0000","last_became_active":"2024-04-25T08:39:57.111229+0000","last_became_peered":"2024-04-25T08:39:57.111229+0000","last_unstale":"2024-04-25T08:39:57.112046+0000","last_undegraded":"2024-04-25T08:39:57.112046+0000","last_fullsized":"2024-04-25T08:39:57.112046+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108536+0000","last_change":"2024-04-25T08:39:57.108536+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108536+0000","last_undegraded":"2024-04-25T08:39:57.108536+0000","last_fullsized":"2024-04-25T08:39:57.108536+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120536+0000","last_change":"2024-04-25T08:39:57.117732+0000","last_active":"2024-04-25T08:39:57.120536+0000","last_peered":"2024-04-25T08:39:57.120536+0000","last_clean":"2024-04-25T08:39:57.120536+0000","last_became_active":"2024-04-25T08:39:57.117425+0000","last_became_peered":"2024-04-25T08:39:57.117425+0000","last_unstale":"2024-04-25T08:39:57.120536+0000","last_undegraded":"2024-04-25T08:39:57.120536+0000","last_fullsized":"2024-04-25T08:39:57.120536+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:40.880195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111346+0000","last_change":"2024-04-25T08:39:57.111346+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111346+0000","last_undegraded":"2024-04-25T08:39:57.111346+0000","last_fullsized":"2024-04-25T08:39:57.111346+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.111873+0000","last_change":"2024-04-25T08:39:57.110361+0000","last_active":"2024-04-25T08:39:57.111873+0000","last_peered":"2024-04-25T08:39:57.111873+0000","last_clean":"2024-04-25T08:39:57.111873+0000","last_became_active":"2024-04-25T08:39:57.110079+0000","last_became_peered":"2024-04-25T08:39:57.110079+0000","last_unstale":"2024-04-25T08:39:57.111873+0000","last_undegraded":"2024-04-25T08:39:57.111873+0000","last_fullsized":"2024-04-25T08:39:57.111873+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:55.191344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107451+0000","last_change":"2024-04-25T08:39:57.107451+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107451+0000","last_undegraded":"2024-04-25T08:39:57.107451+0000","last_fullsized":"2024-04-25T08:39:57.107451+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.122916+0000","last_change":"2024-04-25T08:39:57.122219+0000","last_active":"2024-04-25T08:39:57.122916+0000","last_peered":"2024-04-25T08:39:57.122916+0000","last_clean":"2024-04-25T08:39:57.122916+0000","last_became_active":"2024-04-25T08:39:57.121910+0000","last_became_peered":"2024-04-25T08:39:57.121910+0000","last_unstale":"2024-04-25T08:39:57.122916+0000","last_undegraded":"2024-04-25T08:39:57.122916+0000","last_fullsized":"2024-04-25T08:39:57.122916+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108782+0000","last_change":"2024-04-25T08:39:57.108782+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108782+0000","last_undegraded":"2024-04-25T08:39:57.108782+0000","last_fullsized":"2024-04-25T08:39:57.108782+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120411+0000","last_change":"2024-04-25T08:39:57.119082+0000","last_active":"2024-04-25T08:39:57.120411+0000","last_peered":"2024-04-25T08:39:57.120411+0000","last_clean":"2024-04-25T08:39:57.120411+0000","last_became_active":"2024-04-25T08:39:57.118794+0000","last_became_peered":"2024-04-25T08:39:57.118794+0000","last_unstale":"2024-04-25T08:39:57.120411+0000","last_undegraded":"2024-04-25T08:39:57.120411+0000","last_fullsized":"2024-04-25T08:39:57.120411+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105610+0000","last_change":"2024-04-25T08:39:57.105610+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105610+0000","last_undegraded":"2024-04-25T08:39:57.105610+0000","last_fullsized":"2024-04-25T08:39:57.105610+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.124563+0000","last_change":"2024-04-25T08:39:57.123981+0000","last_active":"2024-04-25T08:39:57.124563+0000","last_peered":"2024-04-25T08:39:57.124563+0000","last_clean":"2024-04-25T08:39:57.124563+0000","last_became_active":"2024-04-25T08:39:57.123631+0000","last_became_peered":"2024-04-25T08:39:57.123631+0000","last_unstale":"2024-04-25T08:39:57.124563+0000","last_undegraded":"2024-04-25T08:39:57.124563+0000","last_fullsized":"2024-04-25T08:39:57.124563+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:58.838892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107893+0000","last_change":"2024-04-25T08:39:57.107893+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107893+0000","last_undegraded":"2024-04-25T08:39:57.107893+0000","last_fullsized":"2024-04-25T08:39:57.107893+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.119811+0000","last_change":"2024-04-25T08:39:57.116831+0000","last_active":"2024-04-25T08:39:57.119811+0000","last_peered":"2024-04-25T08:39:57.119811+0000","last_clean":"2024-04-25T08:39:57.119811+0000","last_became_active":"2024-04-25T08:39:57.116598+0000","last_became_peered":"2024-04-25T08:39:57.116598+0000","last_unstale":"2024-04-25T08:39:57.119811+0000","last_undegraded":"2024-04-25T08:39:57.119811+0000","last_fullsized":"2024-04-25T08:39:57.119811+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:43.760739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.114648+0000","last_change":"2024-04-25T08:39:57.114648+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.114648+0000","last_undegraded":"2024-04-25T08:39:57.114648+0000","last_fullsized":"2024-04-25T08:39:57.114648+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.125099+0000","last_change":"2024-04-25T08:39:57.124496+0000","last_active":"2024-04-25T08:39:57.125099+0000","last_peered":"2024-04-25T08:39:57.125099+0000","last_clean":"2024-04-25T08:39:57.125099+0000","last_became_active":"2024-04-25T08:39:57.123329+0000","last_became_peered":"2024-04-25T08:39:57.123329+0000","last_unstale":"2024-04-25T08:39:57.125099+0000","last_undegraded":"2024-04-25T08:39:57.125099+0000","last_fullsized":"2024-04-25T08:39:57.125099+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:25.558741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104630+0000","last_change":"2024-04-25T08:39:57.104630+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104630+0000","last_undegraded":"2024-04-25T08:39:57.104630+0000","last_fullsized":"2024-04-25T08:39:57.104630+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117207+0000","last_change":"2024-04-25T08:39:57.115175+0000","last_active":"2024-04-25T08:39:57.117207+0000","last_peered":"2024-04-25T08:39:57.117207+0000","last_clean":"2024-04-25T08:39:57.117207+0000","last_became_active":"2024-04-25T08:39:57.114913+0000","last_became_peered":"2024-04-25T08:39:57.114913+0000","last_unstale":"2024-04-25T08:39:57.117207+0000","last_undegraded":"2024-04-25T08:39:57.117207+0000","last_fullsized":"2024-04-25T08:39:57.117207+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:41.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.113272+0000","last_change":"2024-04-25T08:39:57.113272+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.113272+0000","last_undegraded":"2024-04-25T08:39:57.113272+0000","last_fullsized":"2024-04-25T08:39:57.113272+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120816+0000","last_change":"2024-04-25T08:39:57.119480+0000","last_active":"2024-04-25T08:39:57.120816+0000","last_peered":"2024-04-25T08:39:57.120816+0000","last_clean":"2024-04-25T08:39:57.120816+0000","last_became_active":"2024-04-25T08:39:57.119195+0000","last_became_peered":"2024-04-25T08:39:57.119195+0000","last_unstale":"2024-04-25T08:39:57.120816+0000","last_undegraded":"2024-04-25T08:39:57.120816+0000","last_fullsized":"2024-04-25T08:39:57.120816+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106051+0000","last_change":"2024-04-25T08:39:57.106051+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106051+0000","last_undegraded":"2024-04-25T08:39:57.106051+0000","last_fullsized":"2024-04-25T08:39:57.106051+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120480+0000","last_change":"2024-04-25T08:39:57.118299+0000","last_active":"2024-04-25T08:39:57.120480+0000","last_peered":"2024-04-25T08:39:57.120480+0000","last_clean":"2024-04-25T08:39:57.120480+0000","last_became_active":"2024-04-25T08:39:57.117996+0000","last_became_peered":"2024-04-25T08:39:57.117996+0000","last_unstale":"2024-04-25T08:39:57.120480+0000","last_undegraded":"2024-04-25T08:39:57.120480+0000","last_fullsized":"2024-04-25T08:39:57.120480+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:40.880195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.112478+0000","last_change":"2024-04-25T08:39:57.112478+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.112478+0000","last_undegraded":"2024-04-25T08:39:57.112478+0000","last_fullsized":"2024-04-25T08:39:57.112478+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121442+0000","last_change":"2024-04-25T08:39:57.119869+0000","last_active":"2024-04-25T08:39:57.121442+0000","last_peered":"2024-04-25T08:39:57.121442+0000","last_clean":"2024-04-25T08:39:57.121442+0000","last_became_active":"2024-04-25T08:39:57.117564+0000","last_became_peered":"2024-04-25T08:39:57.117564+0000","last_unstale":"2024-04-25T08:39:57.121442+0000","last_undegraded":"2024-04-25T08:39:57.121442+0000","last_fullsized":"2024-04-25T08:39:57.121442+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:41.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105978+0000","last_change":"2024-04-25T08:39:57.105978+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105978+0000","last_undegraded":"2024-04-25T08:39:57.105978+0000","last_fullsized":"2024-04-25T08:39:57.105978+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117130+0000","last_change":"2024-04-25T08:39:57.111954+0000","last_active":"2024-04-25T08:39:57.117130+0000","last_peered":"2024-04-25T08:39:57.117130+0000","last_clean":"2024-04-25T08:39:57.117130+0000","last_became_active":"2024-04-25T08:39:57.111611+0000","last_became_peered":"2024-04-25T08:39:57.111611+0000","last_unstale":"2024-04-25T08:39:57.117130+0000","last_undegraded":"2024-04-25T08:39:57.117130+0000","last_fullsized":"2024-04-25T08:39:57.117130+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108899+0000","last_change":"2024-04-25T08:39:57.108899+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108899+0000","last_undegraded":"2024-04-25T08:39:57.108899+0000","last_fullsized":"2024-04-25T08:39:57.108899+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.124072+0000","last_change":"2024-04-25T08:39:57.122861+0000","last_active":"2024-04-25T08:39:57.124072+0000","last_peered":"2024-04-25T08:39:57.124072+0000","last_clean":"2024-04-25T08:39:57.124072+0000","last_became_active":"2024-04-25T08:39:57.122443+0000","last_became_peered":"2024-04-25T08:39:57.122443+0000","last_unstale":"2024-04-25T08:39:57.124072+0000","last_undegraded":"2024-04-25T08:39:57.124072+0000","last_fullsized":"2024-04-25T08:39:57.124072+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.134755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106645+0000","last_change":"2024-04-25T08:39:57.106645+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106645+0000","last_undegraded":"2024-04-25T08:39:57.106645+0000","last_fullsized":"2024-04-25T08:39:57.106645+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.123229+0000","last_change":"2024-04-25T08:39:57.122180+0000","last_active":"2024-04-25T08:39:57.123229+0000","last_peered":"2024-04-25T08:39:57.123229+0000","last_clean":"2024-04-25T08:39:57.123229+0000","last_became_active":"2024-04-25T08:39:57.121859+0000","last_became_peered":"2024-04-25T08:39:57.121859+0000","last_unstale":"2024-04-25T08:39:57.123229+0000","last_undegraded":"2024-04-25T08:39:57.123229+0000","last_fullsized":"2024-04-25T08:39:57.123229+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:56.067733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.116232+0000","last_change":"2024-04-25T08:39:57.116232+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.116232+0000","last_undegraded":"2024-04-25T08:39:57.116232+0000","last_fullsized":"2024-04-25T08:39:57.116232+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.125352+0000","last_change":"2024-04-25T08:39:57.124849+0000","last_active":"2024-04-25T08:39:57.125352+0000","last_peered":"2024-04-25T08:39:57.125352+0000","last_clean":"2024-04-25T08:39:57.125352+0000","last_became_active":"2024-04-25T08:39:57.124555+0000","last_became_peered":"2024-04-25T08:39:57.124555+0000","last_unstale":"2024-04-25T08:39:57.125352+0000","last_undegraded":"2024-04-25T08:39:57.125352+0000","last_fullsized":"2024-04-25T08:39:57.125352+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104557+0000","last_change":"2024-04-25T08:39:57.104557+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104557+0000","last_undegraded":"2024-04-25T08:39:57.104557+0000","last_fullsized":"2024-04-25T08:39:57.104557+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116939+0000","last_change":"2024-04-25T08:39:57.115173+0000","last_active":"2024-04-25T08:39:57.116939+0000","last_peered":"2024-04-25T08:39:57.116939+0000","last_clean":"2024-04-25T08:39:57.116939+0000","last_became_active":"2024-04-25T08:39:57.114870+0000","last_became_peered":"2024-04-25T08:39:57.114870+0000","last_unstale":"2024-04-25T08:39:57.116939+0000","last_undegraded":"2024-04-25T08:39:57.116939+0000","last_fullsized":"2024-04-25T08:39:57.116939+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:29.233744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107381+0000","last_change":"2024-04-25T08:39:57.107381+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107381+0000","last_undegraded":"2024-04-25T08:39:57.107381+0000","last_fullsized":"2024-04-25T08:39:57.107381+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.119844+0000","last_change":"2024-04-25T08:39:57.117206+0000","last_active":"2024-04-25T08:39:57.119844+0000","last_peered":"2024-04-25T08:39:57.119844+0000","last_clean":"2024-04-25T08:39:57.119844+0000","last_became_active":"2024-04-25T08:39:57.116938+0000","last_became_peered":"2024-04-25T08:39:57.116938+0000","last_unstale":"2024-04-25T08:39:57.119844+0000","last_undegraded":"2024-04-25T08:39:57.119844+0000","last_fullsized":"2024-04-25T08:39:57.119844+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:14.033329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111993+0000","last_change":"2024-04-25T08:39:57.111993+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111993+0000","last_undegraded":"2024-04-25T08:39:57.111993+0000","last_fullsized":"2024-04-25T08:39:57.111993+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116515+0000","last_change":"2024-04-25T08:39:57.114054+0000","last_active":"2024-04-25T08:39:57.116515+0000","last_peered":"2024-04-25T08:39:57.116515+0000","last_clean":"2024-04-25T08:39:57.116515+0000","last_became_active":"2024-04-25T08:39:57.113271+0000","last_became_peered":"2024-04-25T08:39:57.113271+0000","last_unstale":"2024-04-25T08:39:57.116515+0000","last_undegraded":"2024-04-25T08:39:57.116515+0000","last_fullsized":"2024-04-25T08:39:57.116515+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:25.305378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.113672+0000","last_change":"2024-04-25T08:39:57.113672+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.113672+0000","last_undegraded":"2024-04-25T08:39:57.113672+0000","last_fullsized":"2024-04-25T08:39:57.113672+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121882+0000","last_change":"2024-04-25T08:39:57.117517+0000","last_active":"2024-04-25T08:39:57.121882+0000","last_peered":"2024-04-25T08:39:57.121882+0000","last_clean":"2024-04-25T08:39:57.121882+0000","last_became_active":"2024-04-25T08:39:57.116811+0000","last_became_peered":"2024-04-25T08:39:57.116811+0000","last_unstale":"2024-04-25T08:39:57.121882+0000","last_undegraded":"2024-04-25T08:39:57.121882+0000","last_fullsized":"2024-04-25T08:39:57.121882+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:36.804780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110704+0000","last_change":"2024-04-25T08:39:57.110704+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110704+0000","last_undegraded":"2024-04-25T08:39:57.110704+0000","last_fullsized":"2024-04-25T08:39:57.110704+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.110041+0000","last_change":"2024-04-25T08:39:57.109255+0000","last_active":"2024-04-25T08:39:57.110041+0000","last_peered":"2024-04-25T08:39:57.110041+0000","last_clean":"2024-04-25T08:39:57.110041+0000","last_became_active":"2024-04-25T08:39:57.108956+0000","last_became_peered":"2024-04-25T08:39:57.108956+0000","last_unstale":"2024-04-25T08:39:57.110041+0000","last_undegraded":"2024-04-25T08:39:57.110041+0000","last_fullsized":"2024-04-25T08:39:57.110041+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:53.702495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.107472+0000","last_change":"2024-04-25T08:39:57.107472+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.107472+0000","last_undegraded":"2024-04-25T08:39:57.107472+0000","last_fullsized":"2024-04-25T08:39:57.107472+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121148+0000","last_change":"2024-04-25T08:39:57.118978+0000","last_active":"2024-04-25T08:39:57.121148+0000","last_peered":"2024-04-25T08:39:57.121148+0000","last_clean":"2024-04-25T08:39:57.121148+0000","last_became_active":"2024-04-25T08:39:57.118673+0000","last_became_peered":"2024-04-25T08:39:57.118673+0000","last_unstale":"2024-04-25T08:39:57.121148+0000","last_undegraded":"2024-04-25T08:39:57.121148+0000","last_fullsized":"2024-04-25T08:39:57.121148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111344+0000","last_change":"2024-04-25T08:39:57.111344+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111344+0000","last_undegraded":"2024-04-25T08:39:57.111344+0000","last_fullsized":"2024-04-25T08:39:57.111344+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116295+0000","last_change":"2024-04-25T08:39:57.114778+0000","last_active":"2024-04-25T08:39:57.116295+0000","last_peered":"2024-04-25T08:39:57.116295+0000","last_clean":"2024-04-25T08:39:57.116295+0000","last_became_active":"2024-04-25T08:39:57.114509+0000","last_became_peered":"2024-04-25T08:39:57.114509+0000","last_unstale":"2024-04-25T08:39:57.116295+0000","last_undegraded":"2024-04-25T08:39:57.116295+0000","last_fullsized":"2024-04-25T08:39:57.116295+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:06.569207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108631+0000","last_change":"2024-04-25T08:39:57.108631+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108631+0000","last_undegraded":"2024-04-25T08:39:57.108631+0000","last_fullsized":"2024-04-25T08:39:57.108631+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117436+0000","last_change":"2024-04-25T08:39:57.113886+0000","last_active":"2024-04-25T08:39:57.117436+0000","last_peered":"2024-04-25T08:39:57.117436+0000","last_clean":"2024-04-25T08:39:57.117436+0000","last_became_active":"2024-04-25T08:39:57.113199+0000","last_became_peered":"2024-04-25T08:39:57.113199+0000","last_unstale":"2024-04-25T08:39:57.117436+0000","last_undegraded":"2024-04-25T08:39:57.117436+0000","last_fullsized":"2024-04-25T08:39:57.117436+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:17.991774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.111394+0000","last_change":"2024-04-25T08:39:57.111394+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.111394+0000","last_undegraded":"2024-04-25T08:39:57.111394+0000","last_fullsized":"2024-04-25T08:39:57.111394+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.120427+0000","last_change":"2024-04-25T08:39:57.117731+0000","last_active":"2024-04-25T08:39:57.120427+0000","last_peered":"2024-04-25T08:39:57.120427+0000","last_clean":"2024-04-25T08:39:57.120427+0000","last_became_active":"2024-04-25T08:39:57.117423+0000","last_became_peered":"2024-04-25T08:39:57.117423+0000","last_unstale":"2024-04-25T08:39:57.120427+0000","last_undegraded":"2024-04-25T08:39:57.120427+0000","last_fullsized":"2024-04-25T08:39:57.120427+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:56.067733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.105610+0000","last_change":"2024-04-25T08:39:57.105610+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.105610+0000","last_undegraded":"2024-04-25T08:39:57.105610+0000","last_fullsized":"2024-04-25T08:39:57.105610+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.116616+0000","last_change":"2024-04-25T08:39:57.115100+0000","last_active":"2024-04-25T08:39:57.116616+0000","last_peered":"2024-04-25T08:39:57.116616+0000","last_clean":"2024-04-25T08:39:57.116616+0000","last_became_active":"2024-04-25T08:39:57.114830+0000","last_became_peered":"2024-04-25T08:39:57.114830+0000","last_unstale":"2024-04-25T08:39:57.116616+0000","last_undegraded":"2024-04-25T08:39:57.116616+0000","last_fullsized":"2024-04-25T08:39:57.116616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106488+0000","last_change":"2024-04-25T08:39:57.106488+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106488+0000","last_undegraded":"2024-04-25T08:39:57.106488+0000","last_fullsized":"2024-04-25T08:39:57.106488+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.123505+0000","last_change":"2024-04-25T08:39:57.120468+0000","last_active":"2024-04-25T08:39:57.123505+0000","last_peered":"2024-04-25T08:39:57.123505+0000","last_clean":"2024-04-25T08:39:57.123505+0000","last_became_active":"2024-04-25T08:39:57.119296+0000","last_became_peered":"2024-04-25T08:39:57.119296+0000","last_unstale":"2024-04-25T08:39:57.123505+0000","last_undegraded":"2024-04-25T08:39:57.123505+0000","last_fullsized":"2024-04-25T08:39:57.123505+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:07.703556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.110350+0000","last_change":"2024-04-25T08:39:57.110350+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.110350+0000","last_undegraded":"2024-04-25T08:39:57.110350+0000","last_fullsized":"2024-04-25T08:39:57.110350+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117144+0000","last_change":"2024-04-25T08:39:57.113751+0000","last_active":"2024-04-25T08:39:57.117144+0000","last_peered":"2024-04-25T08:39:57.117144+0000","last_clean":"2024-04-25T08:39:57.117144+0000","last_became_active":"2024-04-25T08:39:57.113479+0000","last_became_peered":"2024-04-25T08:39:57.113479+0000","last_unstale":"2024-04-25T08:39:57.117144+0000","last_undegraded":"2024-04-25T08:39:57.117144+0000","last_fullsized":"2024-04-25T08:39:57.117144+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:40.205987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.108211+0000","last_change":"2024-04-25T08:39:57.108211+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.108211+0000","last_undegraded":"2024-04-25T08:39:57.108211+0000","last_fullsized":"2024-04-25T08:39:57.108211+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.117175+0000","last_change":"2024-04-25T08:39:57.114636+0000","last_active":"2024-04-25T08:39:57.117175+0000","last_peered":"2024-04-25T08:39:57.117175+0000","last_clean":"2024-04-25T08:39:57.117175+0000","last_became_active":"2024-04-25T08:39:57.114366+0000","last_became_peered":"2024-04-25T08:39:57.114366+0000","last_unstale":"2024-04-25T08:39:57.117175+0000","last_undegraded":"2024-04-25T08:39:57.117175+0000","last_fullsized":"2024-04-25T08:39:57.117175+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.104891+0000","last_change":"2024-04-25T08:39:57.104891+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.104891+0000","last_undegraded":"2024-04-25T08:39:57.104891+0000","last_fullsized":"2024-04-25T08:39:57.104891+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.123503+0000","last_change":"2024-04-25T08:39:57.119185+0000","last_active":"2024-04-25T08:39:57.123503+0000","last_peered":"2024-04-25T08:39:57.123503+0000","last_clean":"2024-04-25T08:39:57.123503+0000","last_became_active":"2024-04-25T08:39:57.118135+0000","last_became_peered":"2024-04-25T08:39:57.118135+0000","last_unstale":"2024-04-25T08:39:57.123503+0000","last_undegraded":"2024-04-25T08:39:57.123503+0000","last_fullsized":"2024-04-25T08:39:57.123503+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:01.150116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.114248+0000","last_change":"2024-04-25T08:39:57.114248+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.114248+0000","last_undegraded":"2024-04-25T08:39:57.114248+0000","last_fullsized":"2024-04-25T08:39:57.114248+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.113748+0000","last_change":"2024-04-25T08:39:57.111684+0000","last_active":"2024-04-25T08:39:57.113748+0000","last_peered":"2024-04-25T08:39:57.113748+0000","last_clean":"2024-04-25T08:39:57.113748+0000","last_became_active":"2024-04-25T08:39:57.111394+0000","last_became_peered":"2024-04-25T08:39:57.111394+0000","last_unstale":"2024-04-25T08:39:57.113748+0000","last_undegraded":"2024-04-25T08:39:57.113748+0000","last_fullsized":"2024-04-25T08:39:57.113748+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:06.258822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T08:39:57.106294+0000","last_change":"2024-04-25T08:39:57.106294+0000","last_active":"2024-04-25T08:39:57.096572+0000","last_peered":"2024-04-25T08:39:57.096572+0000","last_clean":"2024-04-25T08:39:57.096572+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:39:57.106294+0000","last_undegraded":"2024-04-25T08:39:57.106294+0000","last_fullsized":"2024-04-25T08:39:57.106294+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.121216+0000","last_change":"2024-04-25T08:39:57.118637+0000","last_active":"2024-04-25T08:39:57.121216+0000","last_peered":"2024-04-25T08:39:57.121216+0000","last_clean":"2024-04-25T08:39:57.121216+0000","last_became_active":"2024-04-25T08:39:57.118325+0000","last_became_peered":"2024-04-25T08:39:57.118325+0000","last_unstale":"2024-04-25T08:39:57.121216+0000","last_undegraded":"2024-04-25T08:39:57.121216+0000","last_fullsized":"2024-04-25T08:39:57.121216+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:27.717141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":356,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T08:39:57.102580+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:39:57.102580+0000","last_peered":"2024-04-25T08:39:57.102580+0000","last_clean":"2024-04-25T08:39:57.102580+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:39:57.102580+0000","last_undegraded":"2024-04-25T08:39:57.102580+0000","last_fullsized":"2024-04-25T08:39:57.102580+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77616,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294224,"statfs":{"total":96636764160,"available":96557285376,"internally_reserved":0,"allocated":2064384,"data_stored":1273646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":77396426},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59000,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312840,"statfs":{"total":96636764160,"available":96576348160,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54824,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317016,"statfs":{"total":96636764160,"available":96580624384,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574954,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68724,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303116,"statfs":{"total":96636764160,"available":96566390784,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72880,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298960,"statfs":{"total":96636764160,"available":96562135040,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574955,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58984,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312856,"statfs":{"total":96636764160,"available":96576364544,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54828,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317012,"statfs":{"total":96636764160,"available":96580620288,"internally_reserved":0,"allocated":1470464,"data_stored":683278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51248,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320592,"statfs":{"total":96636764160,"available":96584286208,"internally_reserved":0,"allocated":2064384,"data_stored":1273646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:04.302 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:40:04.305 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:40:05.564 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:05.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.684+0000 7fb42f2e1640 1 Processor -- start 2024-04-25T08:40:05.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.684+0000 7fb42f2e1640 1 -- start start 2024-04-25T08:40:05.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.685+0000 7fb42f2e1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42806c110 0x7fb42806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.685+0000 7fb42f2e1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb4280ce2e0 0x7fb4280d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.685+0000 7fb42f2e1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42806ca30 con 0x7fb42815df50 2024-04-25T08:40:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.685+0000 7fb42f2e1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4280cf2d0 con 0x7fb4280ce2e0 2024-04-25T08:40:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.685+0000 7fb42f2e1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280ceff0 con 0x7fb42806c110 2024-04-25T08:40:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.685+0000 7fb42c87c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb42815df50 0x7fb4280ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45042/0 (socket says 172.21.15.38:45042) 2024-04-25T08:40:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.685+0000 7fb42c87c640 1 -- 172.21.15.38:0/1701099539 learned_addr learned my addr 172.21.15.38:0/1701099539 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb42d87e640 1 --2- 172.21.15.38:0/1701099539 >> v2:172.21.15.107:3300/0 conn(0x7fb4280ce2e0 0x7fb4280d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034351056 0 0) 0x7fb42806ca30 con 0x7fb42815df50 2024-04-25T08:40:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb42d07d640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42806c110 0x7fb42806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb410003520 con 0x7fb42815df50 2024-04-25T08:40:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1525729358 0 0) 0x7fb410003520 con 0x7fb42815df50 2024-04-25T08:40:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42806c110 msgr2=0x7fb42806c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:05.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42806c110 0x7fb42806c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 >> v2:172.21.15.107:3300/0 conn(0x7fb4280ce2e0 msgr2=0x7fb4280d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 --2- 172.21.15.38:0/1701099539 >> v2:172.21.15.107:3300/0 conn(0x7fb4280ce2e0 0x7fb4280d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4280d1f70 con 0x7fb42815df50 2024-04-25T08:40:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fb41c001670 con 0x7fb42815df50 2024-04-25T08:40:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 >> v1:172.21.15.38:6789/0 conn(0x7fb42815df50 legacy=0x7fb4280ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4100039b0 0x7fb410003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42806c110 0x7fb410004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410004d40 0x7fb4100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280cf2d0 con 0x7fb4100039b0 2024-04-25T08:40:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280ceff0 con 0x7fb410004d40 2024-04-25T08:40:05.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41000b9c0 con 0x7fb42806c110 2024-04-25T08:40:05.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb42d07d640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4100039b0 0x7fb410003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb42c87c640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42806c110 0x7fb410004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb42d87e640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410004d40 0x7fb4100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb42d07d640 1 -- 172.21.15.38:0/1701099539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42806c110 msgr2=0x7fb410004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:05.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.686+0000 7fb42d07d640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb42806c110 0x7fb410004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.687+0000 7fb42d07d640 1 -- 172.21.15.38:0/1701099539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410004d40 msgr2=0x7fb4100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:05.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.687+0000 7fb42d07d640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410004d40 0x7fb4100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.687+0000 7fb42d07d640 1 -- 172.21.15.38:0/1701099539 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb410003520 con 0x7fb4100039b0 2024-04-25T08:40:05.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.687+0000 7fb42d87e640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb410004d40 0x7fb4100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:05.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.687+0000 7fb42d07d640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4100039b0 0x7fb410003dd0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb420009ac0 tx=0x7fb410005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6239c6d5f8fe26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.687+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420036070 con 0x7fb4100039b0 2024-04-25T08:40:05.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.687+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb420031400 con 0x7fb4100039b0 2024-04-25T08:40:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.687+0000 7fb427fff640 1 -- 172.21.15.38:0/1701099539 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420009690 con 0x7fb4100039b0 2024-04-25T08:40:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.688+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1701099539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4100039b0 msgr2=0x7fb410003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.688+0000 7fb42f2e1640 1 --2- 172.21.15.38:0/1701099539 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4100039b0 0x7fb410003dd0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb420009ac0 tx=0x7fb410005830 comp rx=0 tx=0).stop 2024-04-25T08:40:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.688+0000 7fb42d87e640 1 -- 172.21.15.38:0/1701099539 reap_dead start 2024-04-25T08:40:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.688+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1701099539 shutdown_connections 2024-04-25T08:40:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.688+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1701099539 >> 172.21.15.38:0/1701099539 conn(0x7fb4280d3fb0 msgr2=0x7fb4280d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.688+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1701099539 shutdown_connections 2024-04-25T08:40:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.688+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1701099539 wait complete. 2024-04-25T08:40:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.689+0000 7fb42f2e1640 1 Processor -- start 2024-04-25T08:40:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.689+0000 7fb42f2e1640 1 -- start start 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.690+0000 7fb42f2e1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4100039b0 0x7fb4280c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.690+0000 7fb42f2e1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4280cc910 0x7fb42815f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.690+0000 7fb42f2e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4280ccd90 0x7fb42815f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.690+0000 7fb42f2e1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42806c4f0 con 0x7fb4280ccd90 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.690+0000 7fb42f2e1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4280d2530 con 0x7fb4100039b0 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.690+0000 7fb42f2e1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42806c670 con 0x7fb4280cc910 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.690+0000 7fb42d87e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4280ccd90 0x7fb42815f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.690+0000 7fb42d87e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4280ccd90 0x7fb42815f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44576/0 (socket says 172.21.15.38:44576) 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.690+0000 7fb42d87e640 1 -- 172.21.15.38:0/1494357153 learned_addr learned my addr 172.21.15.38:0/1494357153 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.690+0000 7fb42d07d640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4100039b0 0x7fb4280c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.690+0000 7fb42c87c640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4280cc910 0x7fb42815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.691+0000 7fb42d87e640 1 -- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4280cc910 msgr2=0x7fb42815f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.691+0000 7fb42d87e640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4280cc910 0x7fb42815f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.691+0000 7fb42d87e640 1 -- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4100039b0 msgr2=0x7fb4280c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.691+0000 7fb42d87e640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4100039b0 0x7fb4280c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.691+0000 7fb42d87e640 1 -- 172.21.15.38:0/1494357153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4280c73f0 con 0x7fb4280ccd90 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.691+0000 7fb42d07d640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4100039b0 0x7fb4280c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.691+0000 7fb42d87e640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4280ccd90 0x7fb42815f6d0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fb4280b5400 tx=0x7fb4180115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.691+0000 7fb425ffb640 1 -- 172.21.15.38:0/1494357153 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418007e60 con 0x7fb4280ccd90 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.691+0000 7fb425ffb640 1 -- 172.21.15.38:0/1494357153 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41801c070 con 0x7fb4280ccd90 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.691+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1494357153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4280c76d0 con 0x7fb4280ccd90 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.691+0000 7fb425ffb640 1 -- 172.21.15.38:0/1494357153 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418016470 con 0x7fb4280ccd90 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.691+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1494357153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4280c7c90 con 0x7fb4280ccd90 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.692+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1494357153 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb428068e80 con 0x7fb4280ccd90 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.693+0000 7fb425ffb640 1 -- 172.21.15.38:0/1494357153 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb418016610 con 0x7fb4280ccd90 2024-04-25T08:40:05.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.693+0000 7fb425ffb640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb41406b500 0x7fb41406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:05.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.693+0000 7fb425ffb640 1 -- 172.21.15.38:0/1494357153 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb41808ad00 con 0x7fb4280ccd90 2024-04-25T08:40:05.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.693+0000 7fb42d07d640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb41406b500 0x7fb41406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:05.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.694+0000 7fb42d07d640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb41406b500 0x7fb41406d9c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb420033000 tx=0x7fb420008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:05.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.697+0000 7fb425ffb640 1 -- 172.21.15.38:0/1494357153 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4180599f0 con 0x7fb4280ccd90 2024-04-25T08:40:05.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.877+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1494357153 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4280d17a0 con 0x7fb41406b500 2024-04-25T08:40:05.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.884+0000 7fb425ffb640 1 -- 172.21.15.38:0/1494357153 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252487 (secure 0 0 0) 0x7fb4280d17a0 con 0x7fb41406b500 2024-04-25T08:40:05.879 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:05.881 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:40:05.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.887+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb41406b500 msgr2=0x7fb41406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.887+0000 7fb42f2e1640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb41406b500 0x7fb41406d9c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb420033000 tx=0x7fb420008030 comp rx=0 tx=0).stop 2024-04-25T08:40:05.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.887+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1494357153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4280ccd90 msgr2=0x7fb42815f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:05.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.887+0000 7fb42f2e1640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4280ccd90 0x7fb42815f6d0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fb4280b5400 tx=0x7fb4180115a0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.887+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1494357153 shutdown_connections 2024-04-25T08:40:05.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.887+0000 7fb42f2e1640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4280ccd90 0x7fb42815f6d0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.887+0000 7fb42f2e1640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb41406b500 0x7fb41406d9c0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.887+0000 7fb42f2e1640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4280cc910 0x7fb42815f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.887+0000 7fb42f2e1640 1 --2- 172.21.15.38:0/1494357153 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4100039b0 0x7fb4280c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:05.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.887+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1494357153 >> 172.21.15.38:0/1494357153 conn(0x7fb4280d3fb0 msgr2=0x7fb4280b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:05.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.887+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1494357153 shutdown_connections 2024-04-25T08:40:05.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:05.887+0000 7fb42f2e1640 1 -- 172.21.15.38:0/1494357153 wait complete. 2024-04-25T08:40:05.891 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-04-25T08:40:05.324213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":502600,"kb_used_data":13000,"kb_used_omap":12,"kb_used_meta":489459,"kb_avail":754472120,"statfs":{"total":773094113280,"available":772579450880,"internally_reserved":0,"allocated":13312000,"data_stored":6815988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":501206576},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002068"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.138772+0000","last_change":"2024-04-25T08:39:58.129988+0000","last_active":"2024-04-25T08:39:59.138772+0000","last_peered":"2024-04-25T08:39:59.138772+0000","last_clean":"2024-04-25T08:39:59.138772+0000","last_became_active":"2024-04-25T08:39:58.129691+0000","last_became_peered":"2024-04-25T08:39:58.129691+0000","last_unstale":"2024-04-25T08:39:59.138772+0000","last_undegraded":"2024-04-25T08:39:59.138772+0000","last_fullsized":"2024-04-25T08:39:59.138772+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:04.067780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.257919+0000","last_change":"2024-04-25T08:39:58.140424+0000","last_active":"2024-04-25T08:39:59.257919+0000","last_peered":"2024-04-25T08:39:59.257919+0000","last_clean":"2024-04-25T08:39:59.257919+0000","last_became_active":"2024-04-25T08:39:58.138953+0000","last_became_peered":"2024-04-25T08:39:58.138953+0000","last_unstale":"2024-04-25T08:39:59.257919+0000","last_undegraded":"2024-04-25T08:39:59.257919+0000","last_fullsized":"2024-04-25T08:39:59.257919+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:49.607628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.258147+0000","last_change":"2024-04-25T08:39:58.136263+0000","last_active":"2024-04-25T08:39:59.258147+0000","last_peered":"2024-04-25T08:39:59.258147+0000","last_clean":"2024-04-25T08:39:59.258147+0000","last_became_active":"2024-04-25T08:39:58.135992+0000","last_became_peered":"2024-04-25T08:39:58.135992+0000","last_unstale":"2024-04-25T08:39:59.258147+0000","last_undegraded":"2024-04-25T08:39:59.258147+0000","last_fullsized":"2024-04-25T08:39:59.258147+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:19.552203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.137238+0000","last_change":"2024-04-25T08:39:58.121748+0000","last_active":"2024-04-25T08:39:59.137238+0000","last_peered":"2024-04-25T08:39:59.137238+0000","last_clean":"2024-04-25T08:39:59.137238+0000","last_became_active":"2024-04-25T08:39:58.121470+0000","last_became_peered":"2024-04-25T08:39:58.121470+0000","last_unstale":"2024-04-25T08:39:59.137238+0000","last_undegraded":"2024-04-25T08:39:59.137238+0000","last_fullsized":"2024-04-25T08:39:59.137238+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:54.307701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.138500+0000","last_change":"2024-04-25T08:39:58.130626+0000","last_active":"2024-04-25T08:39:59.138500+0000","last_peered":"2024-04-25T08:39:59.138500+0000","last_clean":"2024-04-25T08:39:59.138500+0000","last_became_active":"2024-04-25T08:39:58.130361+0000","last_became_peered":"2024-04-25T08:39:58.130361+0000","last_unstale":"2024-04-25T08:39:59.138500+0000","last_undegraded":"2024-04-25T08:39:59.138500+0000","last_fullsized":"2024-04-25T08:39:59.138500+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:13.768826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339402+0000","last_change":"2024-04-25T08:39:58.133566+0000","last_active":"2024-04-25T08:40:00.339402+0000","last_peered":"2024-04-25T08:40:00.339402+0000","last_clean":"2024-04-25T08:40:00.339402+0000","last_became_active":"2024-04-25T08:39:58.133222+0000","last_became_peered":"2024-04-25T08:39:58.133222+0000","last_unstale":"2024-04-25T08:40:00.339402+0000","last_undegraded":"2024-04-25T08:40:00.339402+0000","last_fullsized":"2024-04-25T08:40:00.339402+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:25.044213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339282+0000","last_change":"2024-04-25T08:39:58.129613+0000","last_active":"2024-04-25T08:40:00.339282+0000","last_peered":"2024-04-25T08:40:00.339282+0000","last_clean":"2024-04-25T08:40:00.339282+0000","last_became_active":"2024-04-25T08:39:58.129309+0000","last_became_peered":"2024-04-25T08:39:58.129309+0000","last_unstale":"2024-04-25T08:40:00.339282+0000","last_undegraded":"2024-04-25T08:40:00.339282+0000","last_fullsized":"2024-04-25T08:40:00.339282+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:54.114560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.257626+0000","last_change":"2024-04-25T08:39:58.134421+0000","last_active":"2024-04-25T08:39:59.257626+0000","last_peered":"2024-04-25T08:39:59.257626+0000","last_clean":"2024-04-25T08:39:59.257626+0000","last_became_active":"2024-04-25T08:39:58.134121+0000","last_became_peered":"2024-04-25T08:39:58.134121+0000","last_unstale":"2024-04-25T08:39:59.257626+0000","last_undegraded":"2024-04-25T08:39:59.257626+0000","last_fullsized":"2024-04-25T08:39:59.257626+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:57.542038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136333+0000","last_change":"2024-04-25T08:39:58.133428+0000","last_active":"2024-04-25T08:39:59.136333+0000","last_peered":"2024-04-25T08:39:59.136333+0000","last_clean":"2024-04-25T08:39:59.136333+0000","last_became_active":"2024-04-25T08:39:58.132247+0000","last_became_peered":"2024-04-25T08:39:58.132247+0000","last_unstale":"2024-04-25T08:39:59.136333+0000","last_undegraded":"2024-04-25T08:39:59.136333+0000","last_fullsized":"2024-04-25T08:39:59.136333+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:46.782134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339050+0000","last_change":"2024-04-25T08:39:58.137537+0000","last_active":"2024-04-25T08:40:00.339050+0000","last_peered":"2024-04-25T08:40:00.339050+0000","last_clean":"2024-04-25T08:40:00.339050+0000","last_became_active":"2024-04-25T08:39:58.137232+0000","last_became_peered":"2024-04-25T08:39:58.137232+0000","last_unstale":"2024-04-25T08:40:00.339050+0000","last_undegraded":"2024-04-25T08:40:00.339050+0000","last_fullsized":"2024-04-25T08:40:00.339050+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:04.067780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.138236+0000","last_change":"2024-04-25T08:39:58.141049+0000","last_active":"2024-04-25T08:39:59.138236+0000","last_peered":"2024-04-25T08:39:59.138236+0000","last_clean":"2024-04-25T08:39:59.138236+0000","last_became_active":"2024-04-25T08:39:58.140692+0000","last_became_peered":"2024-04-25T08:39:58.140692+0000","last_unstale":"2024-04-25T08:39:59.138236+0000","last_undegraded":"2024-04-25T08:39:59.138236+0000","last_fullsized":"2024-04-25T08:39:59.138236+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:07.268936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339806+0000","last_change":"2024-04-25T08:39:58.130924+0000","last_active":"2024-04-25T08:40:00.339806+0000","last_peered":"2024-04-25T08:40:00.339806+0000","last_clean":"2024-04-25T08:40:00.339806+0000","last_became_active":"2024-04-25T08:39:58.130640+0000","last_became_peered":"2024-04-25T08:39:58.130640+0000","last_unstale":"2024-04-25T08:40:00.339806+0000","last_undegraded":"2024-04-25T08:40:00.339806+0000","last_fullsized":"2024-04-25T08:40:00.339806+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:31.542916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338924+0000","last_change":"2024-04-25T08:39:58.138088+0000","last_active":"2024-04-25T08:40:00.338924+0000","last_peered":"2024-04-25T08:40:00.338924+0000","last_clean":"2024-04-25T08:40:00.338924+0000","last_became_active":"2024-04-25T08:39:58.137519+0000","last_became_peered":"2024-04-25T08:39:58.137519+0000","last_unstale":"2024-04-25T08:40:00.338924+0000","last_undegraded":"2024-04-25T08:40:00.338924+0000","last_fullsized":"2024-04-25T08:40:00.338924+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:46.808354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.257913+0000","last_change":"2024-04-25T08:39:58.136209+0000","last_active":"2024-04-25T08:39:59.257913+0000","last_peered":"2024-04-25T08:39:59.257913+0000","last_clean":"2024-04-25T08:39:59.257913+0000","last_became_active":"2024-04-25T08:39:58.135652+0000","last_became_peered":"2024-04-25T08:39:58.135652+0000","last_unstale":"2024-04-25T08:39:59.257913+0000","last_undegraded":"2024-04-25T08:39:59.257913+0000","last_fullsized":"2024-04-25T08:39:59.257913+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:59.764219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.138444+0000","last_change":"2024-04-25T08:39:58.138387+0000","last_active":"2024-04-25T08:39:59.138444+0000","last_peered":"2024-04-25T08:39:59.138444+0000","last_clean":"2024-04-25T08:39:59.138444+0000","last_became_active":"2024-04-25T08:39:58.137508+0000","last_became_peered":"2024-04-25T08:39:58.137508+0000","last_unstale":"2024-04-25T08:39:59.138444+0000","last_undegraded":"2024-04-25T08:39:59.138444+0000","last_fullsized":"2024-04-25T08:39:59.138444+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:15.043443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338758+0000","last_change":"2024-04-25T08:39:58.130053+0000","last_active":"2024-04-25T08:40:00.338758+0000","last_peered":"2024-04-25T08:40:00.338758+0000","last_clean":"2024-04-25T08:40:00.338758+0000","last_became_active":"2024-04-25T08:39:58.129290+0000","last_became_peered":"2024-04-25T08:39:58.129290+0000","last_unstale":"2024-04-25T08:40:00.338758+0000","last_undegraded":"2024-04-25T08:40:00.338758+0000","last_fullsized":"2024-04-25T08:40:00.338758+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:45.840827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338802+0000","last_change":"2024-04-25T08:39:58.140544+0000","last_active":"2024-04-25T08:40:00.338802+0000","last_peered":"2024-04-25T08:40:00.338802+0000","last_clean":"2024-04-25T08:40:00.338802+0000","last_became_active":"2024-04-25T08:39:58.140266+0000","last_became_peered":"2024-04-25T08:39:58.140266+0000","last_unstale":"2024-04-25T08:40:00.338802+0000","last_undegraded":"2024-04-25T08:40:00.338802+0000","last_fullsized":"2024-04-25T08:40:00.338802+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:07.579321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.258022+0000","last_change":"2024-04-25T08:39:58.136392+0000","last_active":"2024-04-25T08:39:59.258022+0000","last_peered":"2024-04-25T08:39:59.258022+0000","last_clean":"2024-04-25T08:39:59.258022+0000","last_became_active":"2024-04-25T08:39:58.136106+0000","last_became_peered":"2024-04-25T08:39:58.136106+0000","last_unstale":"2024-04-25T08:39:59.258022+0000","last_undegraded":"2024-04-25T08:39:59.258022+0000","last_fullsized":"2024-04-25T08:39:59.258022+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:33.727177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136863+0000","last_change":"2024-04-25T08:39:58.137654+0000","last_active":"2024-04-25T08:39:59.136863+0000","last_peered":"2024-04-25T08:39:59.136863+0000","last_clean":"2024-04-25T08:39:59.136863+0000","last_became_active":"2024-04-25T08:39:58.136191+0000","last_became_peered":"2024-04-25T08:39:58.136191+0000","last_unstale":"2024-04-25T08:39:59.136863+0000","last_undegraded":"2024-04-25T08:39:59.136863+0000","last_fullsized":"2024-04-25T08:39:59.136863+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:07.268936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338337+0000","last_change":"2024-04-25T08:39:58.127475+0000","last_active":"2024-04-25T08:40:00.338337+0000","last_peered":"2024-04-25T08:40:00.338337+0000","last_clean":"2024-04-25T08:40:00.338337+0000","last_became_active":"2024-04-25T08:39:58.127173+0000","last_became_peered":"2024-04-25T08:39:58.127173+0000","last_unstale":"2024-04-25T08:40:00.338337+0000","last_undegraded":"2024-04-25T08:40:00.338337+0000","last_fullsized":"2024-04-25T08:40:00.338337+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:18.741457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136237+0000","last_change":"2024-04-25T08:39:58.134242+0000","last_active":"2024-04-25T08:39:59.136237+0000","last_peered":"2024-04-25T08:39:59.136237+0000","last_clean":"2024-04-25T08:39:59.136237+0000","last_became_active":"2024-04-25T08:39:58.133980+0000","last_became_peered":"2024-04-25T08:39:58.133980+0000","last_unstale":"2024-04-25T08:39:59.136237+0000","last_undegraded":"2024-04-25T08:39:59.136237+0000","last_fullsized":"2024-04-25T08:39:59.136237+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:19.552203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.138270+0000","last_change":"2024-04-25T08:39:58.135672+0000","last_active":"2024-04-25T08:39:59.138270+0000","last_peered":"2024-04-25T08:39:59.138270+0000","last_clean":"2024-04-25T08:39:59.138270+0000","last_became_active":"2024-04-25T08:39:58.135394+0000","last_became_peered":"2024-04-25T08:39:58.135394+0000","last_unstale":"2024-04-25T08:39:59.138270+0000","last_undegraded":"2024-04-25T08:39:59.138270+0000","last_fullsized":"2024-04-25T08:39:59.138270+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:26.315492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136326+0000","last_change":"2024-04-25T08:39:58.131805+0000","last_active":"2024-04-25T08:39:59.136326+0000","last_peered":"2024-04-25T08:39:59.136326+0000","last_clean":"2024-04-25T08:39:59.136326+0000","last_became_active":"2024-04-25T08:39:58.130929+0000","last_became_peered":"2024-04-25T08:39:58.130929+0000","last_unstale":"2024-04-25T08:39:59.136326+0000","last_undegraded":"2024-04-25T08:39:59.136326+0000","last_fullsized":"2024-04-25T08:39:59.136326+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:31.376177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.192572+0000","last_change":"2024-04-25T08:39:58.133075+0000","last_active":"2024-04-25T08:40:00.192572+0000","last_peered":"2024-04-25T08:40:00.192572+0000","last_clean":"2024-04-25T08:40:00.192572+0000","last_became_active":"2024-04-25T08:39:58.132783+0000","last_became_peered":"2024-04-25T08:39:58.132783+0000","last_unstale":"2024-04-25T08:40:00.192572+0000","last_undegraded":"2024-04-25T08:40:00.192572+0000","last_fullsized":"2024-04-25T08:40:00.192572+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:26.315492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.193025+0000","last_change":"2024-04-25T08:39:58.135684+0000","last_active":"2024-04-25T08:40:00.193025+0000","last_peered":"2024-04-25T08:40:00.193025+0000","last_clean":"2024-04-25T08:40:00.193025+0000","last_became_active":"2024-04-25T08:39:58.135092+0000","last_became_peered":"2024-04-25T08:39:58.135092+0000","last_unstale":"2024-04-25T08:40:00.193025+0000","last_undegraded":"2024-04-25T08:40:00.193025+0000","last_fullsized":"2024-04-25T08:40:00.193025+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:33.025076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338684+0000","last_change":"2024-04-25T08:39:58.128280+0000","last_active":"2024-04-25T08:40:00.338684+0000","last_peered":"2024-04-25T08:40:00.338684+0000","last_clean":"2024-04-25T08:40:00.338684+0000","last_became_active":"2024-04-25T08:39:58.127959+0000","last_became_peered":"2024-04-25T08:39:58.127959+0000","last_unstale":"2024-04-25T08:40:00.338684+0000","last_undegraded":"2024-04-25T08:40:00.338684+0000","last_fullsized":"2024-04-25T08:40:00.338684+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:25.447164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136669+0000","last_change":"2024-04-25T08:39:58.129614+0000","last_active":"2024-04-25T08:39:59.136669+0000","last_peered":"2024-04-25T08:39:59.136669+0000","last_clean":"2024-04-25T08:39:59.136669+0000","last_became_active":"2024-04-25T08:39:58.129351+0000","last_became_peered":"2024-04-25T08:39:58.129351+0000","last_unstale":"2024-04-25T08:39:59.136669+0000","last_undegraded":"2024-04-25T08:39:59.136669+0000","last_fullsized":"2024-04-25T08:39:59.136669+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:26.315492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339779+0000","last_change":"2024-04-25T08:39:58.140107+0000","last_active":"2024-04-25T08:40:00.339779+0000","last_peered":"2024-04-25T08:40:00.339779+0000","last_clean":"2024-04-25T08:40:00.339779+0000","last_became_active":"2024-04-25T08:39:58.139082+0000","last_became_peered":"2024-04-25T08:39:58.139082+0000","last_unstale":"2024-04-25T08:40:00.339779+0000","last_undegraded":"2024-04-25T08:40:00.339779+0000","last_fullsized":"2024-04-25T08:40:00.339779+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:49.090280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338690+0000","last_change":"2024-04-25T08:39:58.131408+0000","last_active":"2024-04-25T08:40:00.338690+0000","last_peered":"2024-04-25T08:40:00.338690+0000","last_clean":"2024-04-25T08:40:00.338690+0000","last_became_active":"2024-04-25T08:39:58.131082+0000","last_became_peered":"2024-04-25T08:39:58.131082+0000","last_unstale":"2024-04-25T08:40:00.338690+0000","last_undegraded":"2024-04-25T08:40:00.338690+0000","last_fullsized":"2024-04-25T08:40:00.338690+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:53:38.706266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.257979+0000","last_change":"2024-04-25T08:39:58.139566+0000","last_active":"2024-04-25T08:39:59.257979+0000","last_peered":"2024-04-25T08:39:59.257979+0000","last_clean":"2024-04-25T08:39:59.257979+0000","last_became_active":"2024-04-25T08:39:58.139206+0000","last_became_peered":"2024-04-25T08:39:58.139206+0000","last_unstale":"2024-04-25T08:39:59.257979+0000","last_undegraded":"2024-04-25T08:39:59.257979+0000","last_fullsized":"2024-04-25T08:39:59.257979+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:52.295235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.192961+0000","last_change":"2024-04-25T08:39:58.137503+0000","last_active":"2024-04-25T08:40:00.192961+0000","last_peered":"2024-04-25T08:40:00.192961+0000","last_clean":"2024-04-25T08:40:00.192961+0000","last_became_active":"2024-04-25T08:39:58.137212+0000","last_became_peered":"2024-04-25T08:39:58.137212+0000","last_unstale":"2024-04-25T08:40:00.192961+0000","last_undegraded":"2024-04-25T08:40:00.192961+0000","last_fullsized":"2024-04-25T08:40:00.192961+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:19.130684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.192568+0000","last_change":"2024-04-25T08:39:58.132883+0000","last_active":"2024-04-25T08:40:00.192568+0000","last_peered":"2024-04-25T08:40:00.192568+0000","last_clean":"2024-04-25T08:40:00.192568+0000","last_became_active":"2024-04-25T08:39:58.132577+0000","last_became_peered":"2024-04-25T08:39:58.132577+0000","last_unstale":"2024-04-25T08:40:00.192568+0000","last_undegraded":"2024-04-25T08:40:00.192568+0000","last_fullsized":"2024-04-25T08:40:00.192568+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:07.579321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338923+0000","last_change":"2024-04-25T08:39:58.130991+0000","last_active":"2024-04-25T08:40:00.338923+0000","last_peered":"2024-04-25T08:40:00.338923+0000","last_clean":"2024-04-25T08:40:00.338923+0000","last_became_active":"2024-04-25T08:39:58.130676+0000","last_became_peered":"2024-04-25T08:39:58.130676+0000","last_unstale":"2024-04-25T08:40:00.338923+0000","last_undegraded":"2024-04-25T08:40:00.338923+0000","last_fullsized":"2024-04-25T08:40:00.338923+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:15.043443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"120'4","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136641+0000","last_change":"2024-04-25T08:39:57.111533+0000","last_active":"2024-04-25T08:39:59.136641+0000","last_peered":"2024-04-25T08:39:59.136641+0000","last_clean":"2024-04-25T08:39:59.136641+0000","last_became_active":"2024-04-25T08:39:57.111229+0000","last_became_peered":"2024-04-25T08:39:57.111229+0000","last_unstale":"2024-04-25T08:39:59.136641+0000","last_undegraded":"2024-04-25T08:39:59.136641+0000","last_fullsized":"2024-04-25T08:39:59.136641+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.134755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339511+0000","last_change":"2024-04-25T08:39:58.128922+0000","last_active":"2024-04-25T08:40:00.339511+0000","last_peered":"2024-04-25T08:40:00.339511+0000","last_clean":"2024-04-25T08:40:00.339511+0000","last_became_active":"2024-04-25T08:39:58.128524+0000","last_became_peered":"2024-04-25T08:39:58.128524+0000","last_unstale":"2024-04-25T08:40:00.339511+0000","last_undegraded":"2024-04-25T08:40:00.339511+0000","last_fullsized":"2024-04-25T08:40:00.339511+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:33.450487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339149+0000","last_change":"2024-04-25T08:39:57.117732+0000","last_active":"2024-04-25T08:40:00.339149+0000","last_peered":"2024-04-25T08:40:00.339149+0000","last_clean":"2024-04-25T08:40:00.339149+0000","last_became_active":"2024-04-25T08:39:57.117425+0000","last_became_peered":"2024-04-25T08:39:57.117425+0000","last_unstale":"2024-04-25T08:40:00.339149+0000","last_undegraded":"2024-04-25T08:40:00.339149+0000","last_fullsized":"2024-04-25T08:40:00.339149+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.134755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.192746+0000","last_change":"2024-04-25T08:39:58.138837+0000","last_active":"2024-04-25T08:40:00.192746+0000","last_peered":"2024-04-25T08:40:00.192746+0000","last_clean":"2024-04-25T08:40:00.192746+0000","last_became_active":"2024-04-25T08:39:58.138538+0000","last_became_peered":"2024-04-25T08:39:58.138538+0000","last_unstale":"2024-04-25T08:40:00.192746+0000","last_undegraded":"2024-04-25T08:40:00.192746+0000","last_fullsized":"2024-04-25T08:40:00.192746+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:15.043443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136744+0000","last_change":"2024-04-25T08:39:57.110361+0000","last_active":"2024-04-25T08:39:59.136744+0000","last_peered":"2024-04-25T08:39:59.136744+0000","last_clean":"2024-04-25T08:39:59.136744+0000","last_became_active":"2024-04-25T08:39:57.110079+0000","last_became_peered":"2024-04-25T08:39:57.110079+0000","last_unstale":"2024-04-25T08:39:59.136744+0000","last_undegraded":"2024-04-25T08:39:59.136744+0000","last_fullsized":"2024-04-25T08:39:59.136744+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:32.014962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.137074+0000","last_change":"2024-04-25T08:39:58.123403+0000","last_active":"2024-04-25T08:39:59.137074+0000","last_peered":"2024-04-25T08:39:59.137074+0000","last_clean":"2024-04-25T08:39:59.137074+0000","last_became_active":"2024-04-25T08:39:58.123137+0000","last_became_peered":"2024-04-25T08:39:58.123137+0000","last_unstale":"2024-04-25T08:39:59.137074+0000","last_undegraded":"2024-04-25T08:39:59.137074+0000","last_fullsized":"2024-04-25T08:39:59.137074+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:57.240976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339027+0000","last_change":"2024-04-25T08:39:57.122219+0000","last_active":"2024-04-25T08:40:00.339027+0000","last_peered":"2024-04-25T08:40:00.339027+0000","last_clean":"2024-04-25T08:40:00.339027+0000","last_became_active":"2024-04-25T08:39:57.121910+0000","last_became_peered":"2024-04-25T08:39:57.121910+0000","last_unstale":"2024-04-25T08:40:00.339027+0000","last_undegraded":"2024-04-25T08:40:00.339027+0000","last_fullsized":"2024-04-25T08:40:00.339027+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:56.067733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338544+0000","last_change":"2024-04-25T08:39:58.132848+0000","last_active":"2024-04-25T08:40:00.338544+0000","last_peered":"2024-04-25T08:40:00.338544+0000","last_clean":"2024-04-25T08:40:00.338544+0000","last_became_active":"2024-04-25T08:39:58.132363+0000","last_became_peered":"2024-04-25T08:39:58.132363+0000","last_unstale":"2024-04-25T08:40:00.338544+0000","last_undegraded":"2024-04-25T08:40:00.338544+0000","last_fullsized":"2024-04-25T08:40:00.338544+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:07.268936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.138542+0000","last_change":"2024-04-25T08:39:57.119082+0000","last_active":"2024-04-25T08:39:59.138542+0000","last_peered":"2024-04-25T08:39:59.138542+0000","last_clean":"2024-04-25T08:39:59.138542+0000","last_became_active":"2024-04-25T08:39:57.118794+0000","last_became_peered":"2024-04-25T08:39:57.118794+0000","last_unstale":"2024-04-25T08:39:59.138542+0000","last_undegraded":"2024-04-25T08:39:59.138542+0000","last_fullsized":"2024-04-25T08:39:59.138542+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:56.067733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.138057+0000","last_change":"2024-04-25T08:39:58.120963+0000","last_active":"2024-04-25T08:39:59.138057+0000","last_peered":"2024-04-25T08:39:59.138057+0000","last_clean":"2024-04-25T08:39:59.138057+0000","last_became_active":"2024-04-25T08:39:58.120580+0000","last_became_peered":"2024-04-25T08:39:58.120580+0000","last_unstale":"2024-04-25T08:39:59.138057+0000","last_undegraded":"2024-04-25T08:39:59.138057+0000","last_fullsized":"2024-04-25T08:39:59.138057+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:56.201457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339507+0000","last_change":"2024-04-25T08:39:57.123981+0000","last_active":"2024-04-25T08:40:00.339507+0000","last_peered":"2024-04-25T08:40:00.339507+0000","last_clean":"2024-04-25T08:40:00.339507+0000","last_became_active":"2024-04-25T08:39:57.123631+0000","last_became_peered":"2024-04-25T08:39:57.123631+0000","last_unstale":"2024-04-25T08:40:00.339507+0000","last_undegraded":"2024-04-25T08:40:00.339507+0000","last_fullsized":"2024-04-25T08:40:00.339507+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:54.036987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.258116+0000","last_change":"2024-04-25T08:39:58.132067+0000","last_active":"2024-04-25T08:39:59.258116+0000","last_peered":"2024-04-25T08:39:59.258116+0000","last_clean":"2024-04-25T08:39:59.258116+0000","last_became_active":"2024-04-25T08:39:58.131619+0000","last_became_peered":"2024-04-25T08:39:58.131619+0000","last_unstale":"2024-04-25T08:39:59.258116+0000","last_undegraded":"2024-04-25T08:39:59.258116+0000","last_fullsized":"2024-04-25T08:39:59.258116+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:58.733837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.192805+0000","last_change":"2024-04-25T08:39:57.116831+0000","last_active":"2024-04-25T08:40:00.192805+0000","last_peered":"2024-04-25T08:40:00.192805+0000","last_clean":"2024-04-25T08:40:00.192805+0000","last_became_active":"2024-04-25T08:39:57.116598+0000","last_became_peered":"2024-04-25T08:39:57.116598+0000","last_unstale":"2024-04-25T08:40:00.192805+0000","last_undegraded":"2024-04-25T08:40:00.192805+0000","last_fullsized":"2024-04-25T08:40:00.192805+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:41.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339024+0000","last_change":"2024-04-25T08:39:58.136501+0000","last_active":"2024-04-25T08:40:00.339024+0000","last_peered":"2024-04-25T08:40:00.339024+0000","last_clean":"2024-04-25T08:40:00.339024+0000","last_became_active":"2024-04-25T08:39:58.134718+0000","last_became_peered":"2024-04-25T08:39:58.134718+0000","last_unstale":"2024-04-25T08:40:00.339024+0000","last_undegraded":"2024-04-25T08:40:00.339024+0000","last_fullsized":"2024-04-25T08:40:00.339024+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:13.768826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338803+0000","last_change":"2024-04-25T08:39:57.124496+0000","last_active":"2024-04-25T08:40:00.338803+0000","last_peered":"2024-04-25T08:40:00.338803+0000","last_clean":"2024-04-25T08:40:00.338803+0000","last_became_active":"2024-04-25T08:39:57.123329+0000","last_became_peered":"2024-04-25T08:39:57.123329+0000","last_unstale":"2024-04-25T08:40:00.338803+0000","last_undegraded":"2024-04-25T08:40:00.338803+0000","last_fullsized":"2024-04-25T08:40:00.338803+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:43.632040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136066+0000","last_change":"2024-04-25T08:39:58.127957+0000","last_active":"2024-04-25T08:39:59.136066+0000","last_peered":"2024-04-25T08:39:59.136066+0000","last_clean":"2024-04-25T08:39:59.136066+0000","last_became_active":"2024-04-25T08:39:58.127681+0000","last_became_peered":"2024-04-25T08:39:58.127681+0000","last_unstale":"2024-04-25T08:39:59.136066+0000","last_undegraded":"2024-04-25T08:39:59.136066+0000","last_fullsized":"2024-04-25T08:39:59.136066+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:54.712609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.137089+0000","last_change":"2024-04-25T08:39:57.115175+0000","last_active":"2024-04-25T08:39:59.137089+0000","last_peered":"2024-04-25T08:39:59.137089+0000","last_clean":"2024-04-25T08:39:59.137089+0000","last_became_active":"2024-04-25T08:39:57.114913+0000","last_became_peered":"2024-04-25T08:39:57.114913+0000","last_unstale":"2024-04-25T08:39:59.137089+0000","last_undegraded":"2024-04-25T08:39:59.137089+0000","last_fullsized":"2024-04-25T08:39:59.137089+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:41.923178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.192855+0000","last_change":"2024-04-25T08:39:58.139037+0000","last_active":"2024-04-25T08:40:00.192855+0000","last_peered":"2024-04-25T08:40:00.192855+0000","last_clean":"2024-04-25T08:40:00.192855+0000","last_became_active":"2024-04-25T08:39:58.138770+0000","last_became_peered":"2024-04-25T08:39:58.138770+0000","last_unstale":"2024-04-25T08:40:00.192855+0000","last_undegraded":"2024-04-25T08:40:00.192855+0000","last_fullsized":"2024-04-25T08:40:00.192855+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:04.067780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.192792+0000","last_change":"2024-04-25T08:39:57.119480+0000","last_active":"2024-04-25T08:40:00.192792+0000","last_peered":"2024-04-25T08:40:00.192792+0000","last_clean":"2024-04-25T08:40:00.192792+0000","last_became_active":"2024-04-25T08:39:57.119195+0000","last_became_peered":"2024-04-25T08:39:57.119195+0000","last_unstale":"2024-04-25T08:40:00.192792+0000","last_undegraded":"2024-04-25T08:40:00.192792+0000","last_fullsized":"2024-04-25T08:40:00.192792+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:02.525060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.258080+0000","last_change":"2024-04-25T08:39:58.136418+0000","last_active":"2024-04-25T08:39:59.258080+0000","last_peered":"2024-04-25T08:39:59.258080+0000","last_clean":"2024-04-25T08:39:59.258080+0000","last_became_active":"2024-04-25T08:39:58.135376+0000","last_became_peered":"2024-04-25T08:39:58.135376+0000","last_unstale":"2024-04-25T08:39:59.258080+0000","last_undegraded":"2024-04-25T08:39:59.258080+0000","last_fullsized":"2024-04-25T08:39:59.258080+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:31.394902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.193001+0000","last_change":"2024-04-25T08:39:57.118299+0000","last_active":"2024-04-25T08:40:00.193001+0000","last_peered":"2024-04-25T08:40:00.193001+0000","last_clean":"2024-04-25T08:40:00.193001+0000","last_became_active":"2024-04-25T08:39:57.117996+0000","last_became_peered":"2024-04-25T08:39:57.117996+0000","last_unstale":"2024-04-25T08:40:00.193001+0000","last_undegraded":"2024-04-25T08:40:00.193001+0000","last_fullsized":"2024-04-25T08:40:00.193001+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:13.099435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.192994+0000","last_change":"2024-04-25T08:39:58.135414+0000","last_active":"2024-04-25T08:40:00.192994+0000","last_peered":"2024-04-25T08:40:00.192994+0000","last_clean":"2024-04-25T08:40:00.192994+0000","last_became_active":"2024-04-25T08:39:58.134751+0000","last_became_peered":"2024-04-25T08:39:58.134751+0000","last_unstale":"2024-04-25T08:40:00.192994+0000","last_undegraded":"2024-04-25T08:40:00.192994+0000","last_fullsized":"2024-04-25T08:40:00.192994+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:21.505217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.257862+0000","last_change":"2024-04-25T08:39:57.119869+0000","last_active":"2024-04-25T08:39:59.257862+0000","last_peered":"2024-04-25T08:39:59.257862+0000","last_clean":"2024-04-25T08:39:59.257862+0000","last_became_active":"2024-04-25T08:39:57.117564+0000","last_became_peered":"2024-04-25T08:39:57.117564+0000","last_unstale":"2024-04-25T08:39:59.257862+0000","last_undegraded":"2024-04-25T08:39:59.257862+0000","last_fullsized":"2024-04-25T08:39:59.257862+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:28.823812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136358+0000","last_change":"2024-04-25T08:39:58.133516+0000","last_active":"2024-04-25T08:39:59.136358+0000","last_peered":"2024-04-25T08:39:59.136358+0000","last_clean":"2024-04-25T08:39:59.136358+0000","last_became_active":"2024-04-25T08:39:58.133234+0000","last_became_peered":"2024-04-25T08:39:58.133234+0000","last_unstale":"2024-04-25T08:39:59.136358+0000","last_undegraded":"2024-04-25T08:39:59.136358+0000","last_fullsized":"2024-04-25T08:39:59.136358+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:56.201457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.257817+0000","last_change":"2024-04-25T08:39:57.111954+0000","last_active":"2024-04-25T08:39:59.257817+0000","last_peered":"2024-04-25T08:39:59.257817+0000","last_clean":"2024-04-25T08:39:59.257817+0000","last_became_active":"2024-04-25T08:39:57.111611+0000","last_became_peered":"2024-04-25T08:39:57.111611+0000","last_unstale":"2024-04-25T08:39:59.257817+0000","last_undegraded":"2024-04-25T08:39:59.257817+0000","last_fullsized":"2024-04-25T08:39:59.257817+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:47.938511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339203+0000","last_change":"2024-04-25T08:39:58.135186+0000","last_active":"2024-04-25T08:40:00.339203+0000","last_peered":"2024-04-25T08:40:00.339203+0000","last_clean":"2024-04-25T08:40:00.339203+0000","last_became_active":"2024-04-25T08:39:58.134851+0000","last_became_peered":"2024-04-25T08:39:58.134851+0000","last_unstale":"2024-04-25T08:40:00.339203+0000","last_undegraded":"2024-04-25T08:40:00.339203+0000","last_fullsized":"2024-04-25T08:40:00.339203+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:21.505217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.258211+0000","last_change":"2024-04-25T08:39:57.122861+0000","last_active":"2024-04-25T08:39:59.258211+0000","last_peered":"2024-04-25T08:39:59.258211+0000","last_clean":"2024-04-25T08:39:59.258211+0000","last_became_active":"2024-04-25T08:39:57.122443+0000","last_became_peered":"2024-04-25T08:39:57.122443+0000","last_unstale":"2024-04-25T08:39:59.258211+0000","last_undegraded":"2024-04-25T08:39:59.258211+0000","last_fullsized":"2024-04-25T08:39:59.258211+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:13.099435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338380+0000","last_change":"2024-04-25T08:39:58.127010+0000","last_active":"2024-04-25T08:40:00.338380+0000","last_peered":"2024-04-25T08:40:00.338380+0000","last_clean":"2024-04-25T08:40:00.338380+0000","last_became_active":"2024-04-25T08:39:58.126617+0000","last_became_peered":"2024-04-25T08:39:58.126617+0000","last_unstale":"2024-04-25T08:40:00.338380+0000","last_undegraded":"2024-04-25T08:40:00.338380+0000","last_fullsized":"2024-04-25T08:40:00.338380+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:58.802088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.257767+0000","last_change":"2024-04-25T08:39:57.122180+0000","last_active":"2024-04-25T08:39:59.257767+0000","last_peered":"2024-04-25T08:39:59.257767+0000","last_clean":"2024-04-25T08:39:59.257767+0000","last_became_active":"2024-04-25T08:39:57.121859+0000","last_became_peered":"2024-04-25T08:39:57.121859+0000","last_unstale":"2024-04-25T08:39:59.257767+0000","last_undegraded":"2024-04-25T08:39:59.257767+0000","last_fullsized":"2024-04-25T08:39:59.257767+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:02.005006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.192858+0000","last_change":"2024-04-25T08:39:58.137564+0000","last_active":"2024-04-25T08:40:00.192858+0000","last_peered":"2024-04-25T08:40:00.192858+0000","last_clean":"2024-04-25T08:40:00.192858+0000","last_became_active":"2024-04-25T08:39:58.137200+0000","last_became_peered":"2024-04-25T08:39:58.137200+0000","last_unstale":"2024-04-25T08:40:00.192858+0000","last_undegraded":"2024-04-25T08:40:00.192858+0000","last_fullsized":"2024-04-25T08:40:00.192858+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:13.768826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338903+0000","last_change":"2024-04-25T08:39:57.124849+0000","last_active":"2024-04-25T08:40:00.338903+0000","last_peered":"2024-04-25T08:40:00.338903+0000","last_clean":"2024-04-25T08:40:00.338903+0000","last_became_active":"2024-04-25T08:39:57.124555+0000","last_became_peered":"2024-04-25T08:39:57.124555+0000","last_unstale":"2024-04-25T08:40:00.338903+0000","last_undegraded":"2024-04-25T08:40:00.338903+0000","last_fullsized":"2024-04-25T08:40:00.338903+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:41.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.138408+0000","last_change":"2024-04-25T08:39:58.128547+0000","last_active":"2024-04-25T08:39:59.138408+0000","last_peered":"2024-04-25T08:39:59.138408+0000","last_clean":"2024-04-25T08:39:59.138408+0000","last_became_active":"2024-04-25T08:39:58.128199+0000","last_became_peered":"2024-04-25T08:39:58.128199+0000","last_unstale":"2024-04-25T08:39:59.138408+0000","last_undegraded":"2024-04-25T08:39:59.138408+0000","last_fullsized":"2024-04-25T08:39:59.138408+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:01:07.579321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338782+0000","last_change":"2024-04-25T08:39:57.115173+0000","last_active":"2024-04-25T08:40:00.338782+0000","last_peered":"2024-04-25T08:40:00.338782+0000","last_clean":"2024-04-25T08:40:00.338782+0000","last_became_active":"2024-04-25T08:39:57.114870+0000","last_became_peered":"2024-04-25T08:39:57.114870+0000","last_unstale":"2024-04-25T08:40:00.338782+0000","last_undegraded":"2024-04-25T08:40:00.338782+0000","last_fullsized":"2024-04-25T08:40:00.338782+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:44.346179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339665+0000","last_change":"2024-04-25T08:39:58.134593+0000","last_active":"2024-04-25T08:40:00.339665+0000","last_peered":"2024-04-25T08:40:00.339665+0000","last_clean":"2024-04-25T08:40:00.339665+0000","last_became_active":"2024-04-25T08:39:58.134235+0000","last_became_peered":"2024-04-25T08:39:58.134235+0000","last_unstale":"2024-04-25T08:40:00.339665+0000","last_undegraded":"2024-04-25T08:40:00.339665+0000","last_fullsized":"2024-04-25T08:40:00.339665+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:58.072984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136858+0000","last_change":"2024-04-25T08:39:57.117206+0000","last_active":"2024-04-25T08:39:59.136858+0000","last_peered":"2024-04-25T08:39:59.136858+0000","last_clean":"2024-04-25T08:39:59.136858+0000","last_became_active":"2024-04-25T08:39:57.116938+0000","last_became_peered":"2024-04-25T08:39:57.116938+0000","last_unstale":"2024-04-25T08:39:59.136858+0000","last_undegraded":"2024-04-25T08:39:59.136858+0000","last_fullsized":"2024-04-25T08:39:59.136858+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:56.531924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.193144+0000","last_change":"2024-04-25T08:39:58.136418+0000","last_active":"2024-04-25T08:40:00.193144+0000","last_peered":"2024-04-25T08:40:00.193144+0000","last_clean":"2024-04-25T08:40:00.193144+0000","last_became_active":"2024-04-25T08:39:58.136132+0000","last_became_peered":"2024-04-25T08:39:58.136132+0000","last_unstale":"2024-04-25T08:40:00.193144+0000","last_undegraded":"2024-04-25T08:40:00.193144+0000","last_fullsized":"2024-04-25T08:40:00.193144+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:57.542038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.137263+0000","last_change":"2024-04-25T08:39:57.114054+0000","last_active":"2024-04-25T08:39:59.137263+0000","last_peered":"2024-04-25T08:39:59.137263+0000","last_clean":"2024-04-25T08:39:59.137263+0000","last_became_active":"2024-04-25T08:39:57.113271+0000","last_became_peered":"2024-04-25T08:39:57.113271+0000","last_unstale":"2024-04-25T08:39:59.137263+0000","last_undegraded":"2024-04-25T08:39:59.137263+0000","last_fullsized":"2024-04-25T08:39:59.137263+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:03.057667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338463+0000","last_change":"2024-04-25T08:39:58.132833+0000","last_active":"2024-04-25T08:40:00.338463+0000","last_peered":"2024-04-25T08:40:00.338463+0000","last_clean":"2024-04-25T08:40:00.338463+0000","last_became_active":"2024-04-25T08:39:58.132427+0000","last_became_peered":"2024-04-25T08:39:58.132427+0000","last_unstale":"2024-04-25T08:40:00.338463+0000","last_undegraded":"2024-04-25T08:40:00.338463+0000","last_fullsized":"2024-04-25T08:40:00.338463+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:00.328459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339324+0000","last_change":"2024-04-25T08:39:57.117517+0000","last_active":"2024-04-25T08:40:00.339324+0000","last_peered":"2024-04-25T08:40:00.339324+0000","last_clean":"2024-04-25T08:40:00.339324+0000","last_became_active":"2024-04-25T08:39:57.116811+0000","last_became_peered":"2024-04-25T08:39:57.116811+0000","last_unstale":"2024-04-25T08:40:00.339324+0000","last_undegraded":"2024-04-25T08:40:00.339324+0000","last_fullsized":"2024-04-25T08:40:00.339324+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:59.284338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.257646+0000","last_change":"2024-04-25T08:39:58.137856+0000","last_active":"2024-04-25T08:39:59.257646+0000","last_peered":"2024-04-25T08:39:59.257646+0000","last_clean":"2024-04-25T08:39:59.257646+0000","last_became_active":"2024-04-25T08:39:58.137409+0000","last_became_peered":"2024-04-25T08:39:58.137409+0000","last_unstale":"2024-04-25T08:39:59.257646+0000","last_undegraded":"2024-04-25T08:39:59.257646+0000","last_fullsized":"2024-04-25T08:39:59.257646+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:12.089958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.137069+0000","last_change":"2024-04-25T08:39:57.109255+0000","last_active":"2024-04-25T08:39:59.137069+0000","last_peered":"2024-04-25T08:39:59.137069+0000","last_clean":"2024-04-25T08:39:59.137069+0000","last_became_active":"2024-04-25T08:39:57.108956+0000","last_became_peered":"2024-04-25T08:39:57.108956+0000","last_unstale":"2024-04-25T08:39:59.137069+0000","last_undegraded":"2024-04-25T08:39:59.137069+0000","last_fullsized":"2024-04-25T08:39:59.137069+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:45.798241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.258196+0000","last_change":"2024-04-25T08:39:58.134464+0000","last_active":"2024-04-25T08:39:59.258196+0000","last_peered":"2024-04-25T08:39:59.258196+0000","last_clean":"2024-04-25T08:39:59.258196+0000","last_became_active":"2024-04-25T08:39:58.134140+0000","last_became_peered":"2024-04-25T08:39:58.134140+0000","last_unstale":"2024-04-25T08:39:59.258196+0000","last_undegraded":"2024-04-25T08:39:59.258196+0000","last_fullsized":"2024-04-25T08:39:59.258196+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:31.376177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.257850+0000","last_change":"2024-04-25T08:39:57.118978+0000","last_active":"2024-04-25T08:39:59.257850+0000","last_peered":"2024-04-25T08:39:59.257850+0000","last_clean":"2024-04-25T08:39:59.257850+0000","last_became_active":"2024-04-25T08:39:57.118673+0000","last_became_peered":"2024-04-25T08:39:57.118673+0000","last_unstale":"2024-04-25T08:39:59.257850+0000","last_undegraded":"2024-04-25T08:39:59.257850+0000","last_fullsized":"2024-04-25T08:39:59.257850+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:22.168357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.192897+0000","last_change":"2024-04-25T08:39:58.136166+0000","last_active":"2024-04-25T08:40:00.192897+0000","last_peered":"2024-04-25T08:40:00.192897+0000","last_clean":"2024-04-25T08:40:00.192897+0000","last_became_active":"2024-04-25T08:39:58.135871+0000","last_became_peered":"2024-04-25T08:39:58.135871+0000","last_unstale":"2024-04-25T08:40:00.192897+0000","last_undegraded":"2024-04-25T08:40:00.192897+0000","last_fullsized":"2024-04-25T08:40:00.192897+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:46.808354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136474+0000","last_change":"2024-04-25T08:39:57.114778+0000","last_active":"2024-04-25T08:39:59.136474+0000","last_peered":"2024-04-25T08:39:59.136474+0000","last_clean":"2024-04-25T08:39:59.136474+0000","last_became_active":"2024-04-25T08:39:57.114509+0000","last_became_peered":"2024-04-25T08:39:57.114509+0000","last_unstale":"2024-04-25T08:39:59.136474+0000","last_undegraded":"2024-04-25T08:39:59.136474+0000","last_fullsized":"2024-04-25T08:39:59.136474+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:18.120570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339857+0000","last_change":"2024-04-25T08:39:58.130194+0000","last_active":"2024-04-25T08:40:00.339857+0000","last_peered":"2024-04-25T08:40:00.339857+0000","last_clean":"2024-04-25T08:40:00.339857+0000","last_became_active":"2024-04-25T08:39:58.129898+0000","last_became_peered":"2024-04-25T08:39:58.129898+0000","last_unstale":"2024-04-25T08:40:00.339857+0000","last_undegraded":"2024-04-25T08:40:00.339857+0000","last_fullsized":"2024-04-25T08:40:00.339857+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:19.978300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339652+0000","last_change":"2024-04-25T08:39:57.113886+0000","last_active":"2024-04-25T08:40:00.339652+0000","last_peered":"2024-04-25T08:40:00.339652+0000","last_clean":"2024-04-25T08:40:00.339652+0000","last_became_active":"2024-04-25T08:39:57.113199+0000","last_became_peered":"2024-04-25T08:39:57.113199+0000","last_unstale":"2024-04-25T08:40:00.339652+0000","last_undegraded":"2024-04-25T08:40:00.339652+0000","last_fullsized":"2024-04-25T08:40:00.339652+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:51.795394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338568+0000","last_change":"2024-04-25T08:39:58.131939+0000","last_active":"2024-04-25T08:40:00.338568+0000","last_peered":"2024-04-25T08:40:00.338568+0000","last_clean":"2024-04-25T08:40:00.338568+0000","last_became_active":"2024-04-25T08:39:58.131645+0000","last_became_peered":"2024-04-25T08:39:58.131645+0000","last_unstale":"2024-04-25T08:40:00.338568+0000","last_undegraded":"2024-04-25T08:40:00.338568+0000","last_fullsized":"2024-04-25T08:40:00.338568+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:26.315492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136855+0000","last_change":"2024-04-25T08:39:57.117731+0000","last_active":"2024-04-25T08:39:59.136855+0000","last_peered":"2024-04-25T08:39:59.136855+0000","last_clean":"2024-04-25T08:39:59.136855+0000","last_became_active":"2024-04-25T08:39:57.117423+0000","last_became_peered":"2024-04-25T08:39:57.117423+0000","last_unstale":"2024-04-25T08:39:59.136855+0000","last_undegraded":"2024-04-25T08:39:59.136855+0000","last_fullsized":"2024-04-25T08:39:59.136855+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:02.005006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136704+0000","last_change":"2024-04-25T08:39:58.126195+0000","last_active":"2024-04-25T08:39:59.136704+0000","last_peered":"2024-04-25T08:39:59.136704+0000","last_clean":"2024-04-25T08:39:59.136704+0000","last_became_active":"2024-04-25T08:39:58.125878+0000","last_became_peered":"2024-04-25T08:39:58.125878+0000","last_unstale":"2024-04-25T08:39:59.136704+0000","last_undegraded":"2024-04-25T08:39:59.136704+0000","last_fullsized":"2024-04-25T08:39:59.136704+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:37.294725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136969+0000","last_change":"2024-04-25T08:39:57.115100+0000","last_active":"2024-04-25T08:39:59.136969+0000","last_peered":"2024-04-25T08:39:59.136969+0000","last_clean":"2024-04-25T08:39:59.136969+0000","last_became_active":"2024-04-25T08:39:57.114830+0000","last_became_peered":"2024-04-25T08:39:57.114830+0000","last_unstale":"2024-04-25T08:39:59.136969+0000","last_undegraded":"2024-04-25T08:39:59.136969+0000","last_fullsized":"2024-04-25T08:39:59.136969+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:22.168357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339075+0000","last_change":"2024-04-25T08:39:58.140190+0000","last_active":"2024-04-25T08:40:00.339075+0000","last_peered":"2024-04-25T08:40:00.339075+0000","last_clean":"2024-04-25T08:40:00.339075+0000","last_became_active":"2024-04-25T08:39:58.139828+0000","last_became_peered":"2024-04-25T08:39:58.139828+0000","last_unstale":"2024-04-25T08:40:00.339075+0000","last_undegraded":"2024-04-25T08:40:00.339075+0000","last_fullsized":"2024-04-25T08:40:00.339075+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:57.108246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339915+0000","last_change":"2024-04-25T08:39:57.120468+0000","last_active":"2024-04-25T08:40:00.339915+0000","last_peered":"2024-04-25T08:40:00.339915+0000","last_clean":"2024-04-25T08:40:00.339915+0000","last_became_active":"2024-04-25T08:39:57.119296+0000","last_became_peered":"2024-04-25T08:39:57.119296+0000","last_unstale":"2024-04-25T08:40:00.339915+0000","last_undegraded":"2024-04-25T08:40:00.339915+0000","last_fullsized":"2024-04-25T08:40:00.339915+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:43:43.219755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.338604+0000","last_change":"2024-04-25T08:39:58.130305+0000","last_active":"2024-04-25T08:40:00.338604+0000","last_peered":"2024-04-25T08:40:00.338604+0000","last_clean":"2024-04-25T08:40:00.338604+0000","last_became_active":"2024-04-25T08:39:58.129687+0000","last_became_peered":"2024-04-25T08:39:58.129687+0000","last_unstale":"2024-04-25T08:40:00.338604+0000","last_undegraded":"2024-04-25T08:40:00.338604+0000","last_fullsized":"2024-04-25T08:40:00.338604+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:30.217066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.138610+0000","last_change":"2024-04-25T08:39:57.113751+0000","last_active":"2024-04-25T08:39:59.138610+0000","last_peered":"2024-04-25T08:39:59.138610+0000","last_clean":"2024-04-25T08:39:59.138610+0000","last_became_active":"2024-04-25T08:39:57.113479+0000","last_became_peered":"2024-04-25T08:39:57.113479+0000","last_unstale":"2024-04-25T08:39:59.138610+0000","last_undegraded":"2024-04-25T08:39:59.138610+0000","last_fullsized":"2024-04-25T08:39:59.138610+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:44.134755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.257724+0000","last_change":"2024-04-25T08:39:58.135213+0000","last_active":"2024-04-25T08:39:59.257724+0000","last_peered":"2024-04-25T08:39:59.257724+0000","last_clean":"2024-04-25T08:39:59.257724+0000","last_became_active":"2024-04-25T08:39:58.134933+0000","last_became_peered":"2024-04-25T08:39:58.134933+0000","last_unstale":"2024-04-25T08:39:59.257724+0000","last_undegraded":"2024-04-25T08:39:59.257724+0000","last_fullsized":"2024-04-25T08:39:59.257724+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:07.501518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.138409+0000","last_change":"2024-04-25T08:39:57.114636+0000","last_active":"2024-04-25T08:39:59.138409+0000","last_peered":"2024-04-25T08:39:59.138409+0000","last_clean":"2024-04-25T08:39:59.138409+0000","last_became_active":"2024-04-25T08:39:57.114366+0000","last_became_peered":"2024-04-25T08:39:57.114366+0000","last_unstale":"2024-04-25T08:39:59.138409+0000","last_undegraded":"2024-04-25T08:39:59.138409+0000","last_fullsized":"2024-04-25T08:39:59.138409+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:41.911504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136951+0000","last_change":"2024-04-25T08:39:58.125268+0000","last_active":"2024-04-25T08:39:59.136951+0000","last_peered":"2024-04-25T08:39:59.136951+0000","last_clean":"2024-04-25T08:39:59.136951+0000","last_became_active":"2024-04-25T08:39:58.124999+0000","last_became_peered":"2024-04-25T08:39:58.124999+0000","last_unstale":"2024-04-25T08:39:59.136951+0000","last_undegraded":"2024-04-25T08:39:59.136951+0000","last_fullsized":"2024-04-25T08:39:59.136951+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:12.864033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339385+0000","last_change":"2024-04-25T08:39:57.119185+0000","last_active":"2024-04-25T08:40:00.339385+0000","last_peered":"2024-04-25T08:40:00.339385+0000","last_clean":"2024-04-25T08:40:00.339385+0000","last_became_active":"2024-04-25T08:39:57.118135+0000","last_became_peered":"2024-04-25T08:39:57.118135+0000","last_unstale":"2024-04-25T08:40:00.339385+0000","last_undegraded":"2024-04-25T08:40:00.339385+0000","last_fullsized":"2024-04-25T08:40:00.339385+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:22:40.552108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:40:00.339179+0000","last_change":"2024-04-25T08:39:58.137899+0000","last_active":"2024-04-25T08:40:00.339179+0000","last_peered":"2024-04-25T08:40:00.339179+0000","last_clean":"2024-04-25T08:40:00.339179+0000","last_became_active":"2024-04-25T08:39:58.136887+0000","last_became_peered":"2024-04-25T08:39:58.136887+0000","last_unstale":"2024-04-25T08:40:00.339179+0000","last_undegraded":"2024-04-25T08:40:00.339179+0000","last_fullsized":"2024-04-25T08:40:00.339179+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:19.130684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136600+0000","last_change":"2024-04-25T08:39:57.111684+0000","last_active":"2024-04-25T08:39:59.136600+0000","last_peered":"2024-04-25T08:39:59.136600+0000","last_clean":"2024-04-25T08:39:59.136600+0000","last_became_active":"2024-04-25T08:39:57.111394+0000","last_became_peered":"2024-04-25T08:39:57.111394+0000","last_unstale":"2024-04-25T08:39:59.136600+0000","last_undegraded":"2024-04-25T08:39:59.136600+0000","last_fullsized":"2024-04-25T08:39:59.136600+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:20.495103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136350+0000","last_change":"2024-04-25T08:39:58.133450+0000","last_active":"2024-04-25T08:39:59.136350+0000","last_peered":"2024-04-25T08:39:59.136350+0000","last_clean":"2024-04-25T08:39:59.136350+0000","last_became_active":"2024-04-25T08:39:58.133122+0000","last_became_peered":"2024-04-25T08:39:58.133122+0000","last_unstale":"2024-04-25T08:39:59.136350+0000","last_undegraded":"2024-04-25T08:39:59.136350+0000","last_fullsized":"2024-04-25T08:39:59.136350+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:57.096572+0000","last_clean_scrub_stamp":"2024-04-25T08:39:57.096572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:05.388922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"120'8","reported_seq":22,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.257634+0000","last_change":"2024-04-25T08:39:57.118637+0000","last_active":"2024-04-25T08:39:59.257634+0000","last_peered":"2024-04-25T08:39:59.257634+0000","last_clean":"2024-04-25T08:39:59.257634+0000","last_became_active":"2024-04-25T08:39:57.118325+0000","last_became_peered":"2024-04-25T08:39:57.118325+0000","last_unstale":"2024-04-25T08:39:59.257634+0000","last_undegraded":"2024-04-25T08:39:59.257634+0000","last_fullsized":"2024-04-25T08:39:59.257634+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:39:56.086458+0000","last_clean_scrub_stamp":"2024-04-25T08:39:56.086458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:41.923178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":360,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T08:39:59.136899+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:39:59.136899+0000","last_peered":"2024-04-25T08:39:59.136899+0000","last_clean":"2024-04-25T08:39:59.136899+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:39:59.136899+0000","last_undegraded":"2024-04-25T08:39:59.136899+0000","last_fullsized":"2024-04-25T08:39:59.136899+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81832,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290008,"statfs":{"total":96636764160,"available":96552968192,"internally_reserved":0,"allocated":2117632,"data_stored":1294948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":81656266},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574956,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59032,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312808,"statfs":{"total":96636764160,"available":96576315392,"internally_reserved":0,"allocated":1511424,"data_stored":704386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54868,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316972,"statfs":{"total":96636764160,"available":96580579328,"internally_reserved":0,"allocated":1515520,"data_stored":703964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574955,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68768,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303072,"statfs":{"total":96636764160,"available":96566345728,"internally_reserved":0,"allocated":1519616,"data_stored":705416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574957,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72932,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298908,"statfs":{"total":96636764160,"available":96562081792,"internally_reserved":0,"allocated":1523712,"data_stored":705450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574956,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59020,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312820,"statfs":{"total":96636764160,"available":96576327680,"internally_reserved":0,"allocated":1507328,"data_stored":703804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54864,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316976,"statfs":{"total":96636764160,"available":96580583424,"internally_reserved":0,"allocated":1511424,"data_stored":703826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574956,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51284,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320556,"statfs":{"total":96636764160,"available":96584249344,"internally_reserved":0,"allocated":2105344,"data_stored":1294194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:05.894 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:40:05.894 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:40:06.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.045+0000 7f149bd9e640 1 Processor -- start 2024-04-25T08:40:06.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.046+0000 7f149bd9e640 1 -- start start 2024-04-25T08:40:06.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.046+0000 7f149bd9e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1494160f20 0x7f14940b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:06.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.046+0000 7f149bd9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14940d1930 0x7f14940b9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:06.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.046+0000 7f149bd9e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14940d1d70 con 0x7f14940ce5c0 2024-04-25T08:40:06.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.046+0000 7f149bd9e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1494161300 con 0x7f1494160f20 2024-04-25T08:40:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.046+0000 7f149bd9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14940b7040 con 0x7f14940d1930 2024-04-25T08:40:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.047+0000 7f149a33b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f14940ce5c0 0x7f14940cea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45058/0 (socket says 172.21.15.38:45058) 2024-04-25T08:40:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.047+0000 7f149a33b640 1 -- 172.21.15.38:0/931909276 learned_addr learned my addr 172.21.15.38:0/931909276 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.047+0000 7f1499339640 1 --2- 172.21.15.38:0/931909276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14940d1930 0x7f14940b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.047+0000 7f1499b3a640 1 --2- 172.21.15.38:0/931909276 >> v2:172.21.15.107:3300/0 conn(0x7f1494160f20 0x7f14940b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.047+0000 7f1498b38640 1 -- 172.21.15.38:0/931909276 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3105502190 0 0) 0x7f14940d1d70 con 0x7f14940ce5c0 2024-04-25T08:40:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.047+0000 7f1498b38640 1 -- 172.21.15.38:0/931909276 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1480003520 con 0x7f14940ce5c0 2024-04-25T08:40:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.047+0000 7f1499339640 1 -- 172.21.15.38:0/931909276 >> v2:172.21.15.107:3300/0 conn(0x7f1494160f20 msgr2=0x7f14940b6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.047+0000 7f1499339640 1 --2- 172.21.15.38:0/931909276 >> v2:172.21.15.107:3300/0 conn(0x7f1494160f20 0x7f14940b6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.047+0000 7f1499339640 1 -- 172.21.15.38:0/931909276 >> v1:172.21.15.38:6789/0 conn(0x7f14940ce5c0 legacy=0x7f14940cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.047+0000 7f1499339640 1 -- 172.21.15.38:0/931909276 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14940b7320 con 0x7f14940d1930 2024-04-25T08:40:06.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.048+0000 7f1499339640 1 --2- 172.21.15.38:0/931909276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14940d1930 0x7f14940b9250 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f14840097a0 tx=0x7f148402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a5d3b74746205e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:06.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.048+0000 7f1498b38640 1 -- 172.21.15.38:0/931909276 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1484037070 con 0x7f14940d1930 2024-04-25T08:40:06.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.048+0000 7f1498b38640 1 -- 172.21.15.38:0/931909276 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1484032440 con 0x7f14940d1930 2024-04-25T08:40:06.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.048+0000 7f1498b38640 1 -- 172.21.15.38:0/931909276 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14840327f0 con 0x7f14940d1930 2024-04-25T08:40:06.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.048+0000 7f149bd9e640 1 -- 172.21.15.38:0/931909276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14940d1930 msgr2=0x7f14940b9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.048+0000 7f149bd9e640 1 --2- 172.21.15.38:0/931909276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14940d1930 0x7f14940b9250 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f14840097a0 tx=0x7f148402a210 comp rx=0 tx=0).stop 2024-04-25T08:40:06.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.048+0000 7f149bd9e640 1 -- 172.21.15.38:0/931909276 shutdown_connections 2024-04-25T08:40:06.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.048+0000 7f149bd9e640 1 --2- 172.21.15.38:0/931909276 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14940d1930 0x7f14940b9250 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.048+0000 7f149bd9e640 1 --2- 172.21.15.38:0/931909276 >> v2:172.21.15.107:3300/0 conn(0x7f1494160f20 0x7f14940b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.049+0000 7f149bd9e640 1 -- 172.21.15.38:0/931909276 >> 172.21.15.38:0/931909276 conn(0x7f1494066ce0 msgr2=0x7f14940670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.049+0000 7f149bd9e640 1 -- 172.21.15.38:0/931909276 shutdown_connections 2024-04-25T08:40:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.049+0000 7f149bd9e640 1 -- 172.21.15.38:0/931909276 wait complete. 2024-04-25T08:40:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.050+0000 7f149bd9e640 1 Processor -- start 2024-04-25T08:40:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.050+0000 7f149bd9e640 1 -- start start 2024-04-25T08:40:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.050+0000 7f149bd9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14940ce5c0 0x7f1494147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.050+0000 7f149bd9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14940d1930 0x7f14940c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.050+0000 7f149bd9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1494160f20 0x7f14940cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f149bd9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14940b7bc0 con 0x7f14940ce5c0 2024-04-25T08:40:06.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f149bd9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14940b78e0 con 0x7f1494160f20 2024-04-25T08:40:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f149bd9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14940b9790 con 0x7f14940d1930 2024-04-25T08:40:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f1499b3a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14940ce5c0 0x7f1494147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f1499b3a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14940ce5c0 0x7f1494147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44586/0 (socket says 172.21.15.38:44586) 2024-04-25T08:40:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f1499b3a640 1 -- 172.21.15.38:0/1644314194 learned_addr learned my addr 172.21.15.38:0/1644314194 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f1499339640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14940d1930 0x7f14940c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f149a33b640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1494160f20 0x7f14940cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f1499b3a640 1 -- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14940d1930 msgr2=0x7f14940c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f1499b3a640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14940d1930 0x7f14940c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f1499b3a640 1 -- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1494160f20 msgr2=0x7f14940cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f1499b3a640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1494160f20 0x7f14940cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f1499b3a640 1 -- 172.21.15.38:0/1644314194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14940ca060 con 0x7f14940ce5c0 2024-04-25T08:40:06.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f1499b3a640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14940ce5c0 0x7f1494147d60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f149000e870 tx=0x7f149000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:06.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f148a7fc640 1 -- 172.21.15.38:0/1644314194 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1490017070 con 0x7f14940ce5c0 2024-04-25T08:40:06.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f148a7fc640 1 -- 172.21.15.38:0/1644314194 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149000c8e0 con 0x7f14940ce5c0 2024-04-25T08:40:06.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f149bd9e640 1 -- 172.21.15.38:0/1644314194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14940c8d50 con 0x7f14940ce5c0 2024-04-25T08:40:06.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f148a7fc640 1 -- 172.21.15.38:0/1644314194 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149000ccd0 con 0x7f14940ce5c0 2024-04-25T08:40:06.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.051+0000 7f149bd9e640 1 -- 172.21.15.38:0/1644314194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14940c8270 con 0x7f14940ce5c0 2024-04-25T08:40:06.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.052+0000 7f149bd9e640 1 -- 172.21.15.38:0/1644314194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1494068e80 con 0x7f14940ce5c0 2024-04-25T08:40:06.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.053+0000 7f148a7fc640 1 -- 172.21.15.38:0/1644314194 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f149001a070 con 0x7f14940ce5c0 2024-04-25T08:40:06.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.053+0000 7f148a7fc640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f147806b4d0 0x7f147806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:06.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.054+0000 7f148a7fc640 1 -- 172.21.15.38:0/1644314194 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f149008a1d0 con 0x7f14940ce5c0 2024-04-25T08:40:06.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.054+0000 7f1499339640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f147806b4d0 0x7f147806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:06.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.054+0000 7f1499339640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f147806b4d0 0x7f147806d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1484034000 tx=0x7f148402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:06.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.058+0000 7f148a7fc640 1 -- 172.21.15.38:0/1644314194 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14940ca060 con 0x7f14940ce5c0 2024-04-25T08:40:06.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.310+0000 7f149bd9e640 1 -- 172.21.15.38:0/1644314194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f14940b5e90 con 0x7f14940ce5c0 2024-04-25T08:40:06.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.313+0000 7f148a7fc640 1 -- 172.21.15.38:0/1644314194 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1490058f50 con 0x7f14940ce5c0 2024-04-25T08:40:06.306 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-04-25T08:40:06.306 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw== 2024-04-25T08:40:06.306 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-04-25T08:40:06.306 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-04-25T08:40:06.306 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:40:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-04-25T08:40:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.315+0000 7f149bd9e640 1 -- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f147806b4d0 msgr2=0x7f147806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.315+0000 7f149bd9e640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f147806b4d0 0x7f147806d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1484034000 tx=0x7f148402e000 comp rx=0 tx=0).stop 2024-04-25T08:40:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.315+0000 7f149bd9e640 1 -- 172.21.15.38:0/1644314194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14940ce5c0 msgr2=0x7f1494147d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.315+0000 7f149bd9e640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14940ce5c0 0x7f1494147d60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f149000e870 tx=0x7f149000ed30 comp rx=0 tx=0).stop 2024-04-25T08:40:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.315+0000 7f149bd9e640 1 -- 172.21.15.38:0/1644314194 shutdown_connections 2024-04-25T08:40:06.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.315+0000 7f149bd9e640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1494160f20 0x7f14940cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.315+0000 7f149bd9e640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f147806b4d0 0x7f147806d990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.315+0000 7f149bd9e640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f14940d1930 0x7f14940c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.315+0000 7f149bd9e640 1 --2- 172.21.15.38:0/1644314194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f14940ce5c0 0x7f1494147d60 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.315+0000 7f149bd9e640 1 -- 172.21.15.38:0/1644314194 >> 172.21.15.38:0/1644314194 conn(0x7f1494066ce0 msgr2=0x7f14940b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.315+0000 7f149bd9e640 1 -- 172.21.15.38:0/1644314194 shutdown_connections 2024-04-25T08:40:06.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.315+0000 7f149bd9e640 1 -- 172.21.15.38:0/1644314194 wait complete. 2024-04-25T08:40:06.319 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:06.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.427+0000 7f1f82ef5640 1 Processor -- start 2024-04-25T08:40:06.420 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.427+0000 7f1f82ef5640 1 -- start start 2024-04-25T08:40:06.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.428+0000 7f1f82ef5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1f7c15dc10 0x7f1f7c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:06.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.428+0000 7f1f82ef5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c0d3e70 con 0x7f1f7c0ced60 2024-04-25T08:40:06.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.428+0000 7f1f82ef5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c0d4150 con 0x7f1f7c0d0f60 2024-04-25T08:40:06.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.428+0000 7f1f82ef5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c15dff0 con 0x7f1f7c15dc10 2024-04-25T08:40:06.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.428+0000 7f1f80c91640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f1f7c15dc10 0x7f1f7c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:06.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.428+0000 7f1f7bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1f7c0ced60 0x7f1f7c0d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:45066/0 (socket says 172.21.15.38:45066) 2024-04-25T08:40:06.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.428+0000 7f1f7bfff640 1 -- 172.21.15.38:0/3289525445 learned_addr learned my addr 172.21.15.38:0/3289525445 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:06.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.428+0000 7f1f80c91640 1 -- 172.21.15.38:0/3289525445 >> v1:172.21.15.107:6789/0 conn(0x7f1f7c0d0f60 legacy=0x7f1f7c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.428+0000 7f1f80c91640 1 -- 172.21.15.38:0/3289525445 >> v1:172.21.15.38:6789/0 conn(0x7f1f7c0ced60 legacy=0x7f1f7c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.428+0000 7f1f80c91640 1 -- 172.21.15.38:0/3289525445 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c0d4430 con 0x7f1f7c15dc10 2024-04-25T08:40:06.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.429+0000 7f1f80c91640 1 --2- 172.21.15.38:0/3289525445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c15dc10 0x7f1f7c0ce820 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0097a0 tx=0x7f1f6c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3dbf89aede5f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:06.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.429+0000 7f1f7b7fe640 1 -- 172.21.15.38:0/3289525445 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c037070 con 0x7f1f7c15dc10 2024-04-25T08:40:06.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.429+0000 7f1f7b7fe640 1 -- 172.21.15.38:0/3289525445 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c032440 con 0x7f1f7c15dc10 2024-04-25T08:40:06.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.429+0000 7f1f7b7fe640 1 -- 172.21.15.38:0/3289525445 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c032810 con 0x7f1f7c15dc10 2024-04-25T08:40:06.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.430+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3289525445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c15dc10 msgr2=0x7f1f7c0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.430+0000 7f1f82ef5640 1 --2- 172.21.15.38:0/3289525445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c15dc10 0x7f1f7c0ce820 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0097a0 tx=0x7f1f6c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.430+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3289525445 shutdown_connections 2024-04-25T08:40:06.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.430+0000 7f1f82ef5640 1 --2- 172.21.15.38:0/3289525445 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c15dc10 0x7f1f7c0ce820 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.430+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3289525445 >> 172.21.15.38:0/3289525445 conn(0x7f1f7c066d10 msgr2=0x7f1f7c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:06.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.430+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3289525445 shutdown_connections 2024-04-25T08:40:06.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.430+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3289525445 wait complete. 2024-04-25T08:40:06.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.431+0000 7f1f82ef5640 1 Processor -- start 2024-04-25T08:40:06.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.431+0000 7f1f82ef5640 1 -- start start 2024-04-25T08:40:06.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f82ef5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0ced60 0x7f1f7c0ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:06.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f82ef5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7c0d0f60 0x7f1f7c0cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f82ef5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7c15dc10 0x7f1f7c14a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f82ef5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0d4710 con 0x7f1f7c0ced60 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f82ef5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0b80f0 con 0x7f1f7c15dc10 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f82ef5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0b8270 con 0x7f1f7c0d0f60 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f80c91640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0ced60 0x7f1f7c0ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f80c91640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0ced60 0x7f1f7c0ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44604/0 (socket says 172.21.15.38:44604) 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f80c91640 1 -- 172.21.15.38:0/3539697089 learned_addr learned my addr 172.21.15.38:0/3539697089 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f81492640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7c15dc10 0x7f1f7c14a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f7bfff640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7c0d0f60 0x7f1f7c0cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f80c91640 1 -- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7c0d0f60 msgr2=0x7f1f7c0cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f80c91640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7c0d0f60 0x7f1f7c0cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f80c91640 1 -- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7c15dc10 msgr2=0x7f1f7c14a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f80c91640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7c15dc10 0x7f1f7c14a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f80c91640 1 -- 172.21.15.38:0/3539697089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c148270 con 0x7f1f7c0ced60 2024-04-25T08:40:06.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f81492640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7c15dc10 0x7f1f7c14a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:06.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.432+0000 7f1f7bfff640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7c0d0f60 0x7f1f7c0cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:06.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.433+0000 7f1f80c91640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0ced60 0x7f1f7c0ca9d0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c035040 tx=0x7f1f6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:06.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.433+0000 7f1f797fa640 1 -- 172.21.15.38:0/3539697089 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c037040 con 0x7f1f7c0ced60 2024-04-25T08:40:06.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.433+0000 7f1f797fa640 1 -- 172.21.15.38:0/3539697089 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c030770 con 0x7f1f7c0ced60 2024-04-25T08:40:06.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.433+0000 7f1f797fa640 1 -- 172.21.15.38:0/3539697089 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c030b80 con 0x7f1f7c0ced60 2024-04-25T08:40:06.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.433+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3539697089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c148550 con 0x7f1f7c0ced60 2024-04-25T08:40:06.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.433+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3539697089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c148b10 con 0x7f1f7c0ced60 2024-04-25T08:40:06.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.434+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3539697089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c068e80 con 0x7f1f7c0ced60 2024-04-25T08:40:06.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.434+0000 7f1f797fa640 1 -- 172.21.15.38:0/3539697089 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f6c044020 con 0x7f1f7c0ced60 2024-04-25T08:40:06.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.434+0000 7f1f797fa640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f6806b4d0 0x7f1f6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:06.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.435+0000 7f1f797fa640 1 -- 172.21.15.38:0/3539697089 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f6c0aaa10 con 0x7f1f7c0ced60 2024-04-25T08:40:06.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.435+0000 7f1f7bfff640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f6806b4d0 0x7f1f6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:06.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.435+0000 7f1f7bfff640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f6806b4d0 0x7f1f6806d990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c14b260 tx=0x7f1f70009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:06.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.439+0000 7f1f797fa640 1 -- 172.21.15.38:0/3539697089 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f6c032e20 con 0x7f1f7c0ced60 2024-04-25T08:40:06.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.659+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3539697089 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f7c0737d0 con 0x7f1f7c0ced60 2024-04-25T08:40:06.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.660+0000 7f1f797fa640 1 -- 172.21.15.38:0/3539697089 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f1f6c079670 con 0x7f1f7c0ced60 2024-04-25T08:40:06.653 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:06.653 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:06.653 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:06.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.662+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f6806b4d0 msgr2=0x7f1f6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.662+0000 7f1f82ef5640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f6806b4d0 0x7f1f6806d990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c14b260 tx=0x7f1f70009040 comp rx=0 tx=0).stop 2024-04-25T08:40:06.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.662+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3539697089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0ced60 msgr2=0x7f1f7c0ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:06.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.662+0000 7f1f82ef5640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0ced60 0x7f1f7c0ca9d0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c035040 tx=0x7f1f6c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.662+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3539697089 shutdown_connections 2024-04-25T08:40:06.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.662+0000 7f1f82ef5640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7c15dc10 0x7f1f7c14a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.662+0000 7f1f82ef5640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f6806b4d0 0x7f1f6806d990 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.662+0000 7f1f82ef5640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7c0d0f60 0x7f1f7c0cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.662+0000 7f1f82ef5640 1 --2- 172.21.15.38:0/3539697089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0ced60 0x7f1f7c0ca9d0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:06.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.662+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3539697089 >> 172.21.15.38:0/3539697089 conn(0x7f1f7c066d10 msgr2=0x7f1f7c0c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:06.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.662+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3539697089 shutdown_connections 2024-04-25T08:40:06.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:06.662+0000 7f1f82ef5640 1 -- 172.21.15.38:0/3539697089 wait complete. 2024-04-25T08:40:07.069 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:39:57.475923+0000', 'modified': '2024-04-25T08:39:59.471310+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7639}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7639': {'gid': 7639, 'name': 'd', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.107:6837/901382672', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.107:6835', 'nonce': 901382672}, {'type': 'v1', 'addr': '172.21.15.107:6837', 'nonce': 901382672}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:40:07.069 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:40:07.069 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T08:40:07.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.185+0000 7f991f8e2640 1 Processor -- start 2024-04-25T08:40:07.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.185+0000 7f991f8e2640 1 -- start start 2024-04-25T08:40:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.185+0000 7f991f8e2640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f991815db80 0x7f99181616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.185+0000 7f991f8e2640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f99180b9600 0x7f99180d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.186+0000 7f991f8e2640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99180d0a70 con 0x7f99180d0660 2024-04-25T08:40:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.186+0000 7f991f8e2640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99181625f0 con 0x7f99180b9600 2024-04-25T08:40:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.186+0000 7f991f8e2640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9918162030 con 0x7f991815db80 2024-04-25T08:40:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.186+0000 7f991ce7d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f99180b9600 0x7f99180d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.186+0000 7f991ce7d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f99180b9600 0x7f99180d0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44616/0 (socket says 172.21.15.38:44616) 2024-04-25T08:40:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.186+0000 7f991ce7d640 1 -- 172.21.15.38:0/2530021069 learned_addr learned my addr 172.21.15.38:0/2530021069 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.186+0000 7f991d67e640 1 --2- 172.21.15.38:0/2530021069 >> v2:172.21.15.107:3300/0 conn(0x7f991815db80 0x7f99181616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.186+0000 7f991ce7d640 1 -- 172.21.15.38:0/2530021069 >> v2:172.21.15.107:3300/0 conn(0x7f991815db80 msgr2=0x7f99181616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:07.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.186+0000 7f991ce7d640 1 --2- 172.21.15.38:0/2530021069 >> v2:172.21.15.107:3300/0 conn(0x7f991815db80 0x7f99181616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.186+0000 7f991ce7d640 1 -- 172.21.15.38:0/2530021069 >> v1:172.21.15.107:6789/0 conn(0x7f99180d0660 legacy=0x7f99180b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.186+0000 7f991ce7d640 1 -- 172.21.15.38:0/2530021069 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9918162310 con 0x7f99180b9600 2024-04-25T08:40:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.186+0000 7f991d67e640 1 --2- 172.21.15.38:0/2530021069 >> v2:172.21.15.107:3300/0 conn(0x7f991815db80 0x7f99181616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.187+0000 7f991ce7d640 1 --2- 172.21.15.38:0/2530021069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99180b9600 0x7f99180d0120 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f99180d0c40 tx=0x7f990802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce812d40cb82bc1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.187+0000 7f990ffff640 1 -- 172.21.15.38:0/2530021069 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908037070 con 0x7f99180b9600 2024-04-25T08:40:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.187+0000 7f990ffff640 1 -- 172.21.15.38:0/2530021069 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9908032440 con 0x7f99180b9600 2024-04-25T08:40:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.187+0000 7f990ffff640 1 -- 172.21.15.38:0/2530021069 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908032810 con 0x7f99180b9600 2024-04-25T08:40:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.187+0000 7f991f8e2640 1 -- 172.21.15.38:0/2530021069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99180b9600 msgr2=0x7f99180d0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.187+0000 7f991f8e2640 1 --2- 172.21.15.38:0/2530021069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99180b9600 0x7f99180d0120 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f99180d0c40 tx=0x7f990802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.187+0000 7f991f8e2640 1 -- 172.21.15.38:0/2530021069 shutdown_connections 2024-04-25T08:40:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.187+0000 7f991f8e2640 1 --2- 172.21.15.38:0/2530021069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99180b9600 0x7f99180d0120 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.187+0000 7f991f8e2640 1 --2- 172.21.15.38:0/2530021069 >> v2:172.21.15.107:3300/0 conn(0x7f991815db80 0x7f99181616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.187+0000 7f991f8e2640 1 -- 172.21.15.38:0/2530021069 >> 172.21.15.38:0/2530021069 conn(0x7f99180668f0 msgr2=0x7f9918066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:07.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.188+0000 7f991f8e2640 1 -- 172.21.15.38:0/2530021069 shutdown_connections 2024-04-25T08:40:07.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.188+0000 7f991f8e2640 1 -- 172.21.15.38:0/2530021069 wait complete. 2024-04-25T08:40:07.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.189+0000 7f991f8e2640 1 Processor -- start 2024-04-25T08:40:07.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.189+0000 7f991f8e2640 1 -- start start 2024-04-25T08:40:07.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991f8e2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99180b9600 0x7f9918159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991f8e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99180d0660 0x7f991815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991f8e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f991815db80 0x7f991815e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991f8e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9918161c20 con 0x7f991815db80 2024-04-25T08:40:07.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991f8e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991815df60 con 0x7f99180d0660 2024-04-25T08:40:07.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991f8e2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9918161da0 con 0x7f99180b9600 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991de7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f991815db80 0x7f991815e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991ce7d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99180d0660 0x7f991815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991de7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f991815db80 0x7f991815e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44630/0 (socket says 172.21.15.38:44630) 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991d67e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99180b9600 0x7f9918159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991de7f640 1 -- 172.21.15.38:0/1482426968 learned_addr learned my addr 172.21.15.38:0/1482426968 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991de7f640 1 -- 172.21.15.38:0/1482426968 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99180b9600 msgr2=0x7f9918159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991de7f640 1 --2- 172.21.15.38:0/1482426968 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99180b9600 0x7f9918159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991de7f640 1 -- 172.21.15.38:0/1482426968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99180d0660 msgr2=0x7f991815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991de7f640 1 --2- 172.21.15.38:0/1482426968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99180d0660 0x7f991815e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991de7f640 1 -- 172.21.15.38:0/1482426968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991815a110 con 0x7f991815db80 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f991de7f640 1 --2- 172.21.15.38:0/1482426968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f991815db80 0x7f991815e620 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f991000c730 tx=0x7f991000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.190+0000 7f990dffb640 1 -- 172.21.15.38:0/1482426968 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910003b70 con 0x7f991815db80 2024-04-25T08:40:07.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.191+0000 7f991f8e2640 1 -- 172.21.15.38:0/1482426968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9918161020 con 0x7f991815db80 2024-04-25T08:40:07.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.191+0000 7f990dffb640 1 -- 172.21.15.38:0/1482426968 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9910003d30 con 0x7f991815db80 2024-04-25T08:40:07.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.191+0000 7f990dffb640 1 -- 172.21.15.38:0/1482426968 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910005310 con 0x7f991815db80 2024-04-25T08:40:07.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.191+0000 7f991f8e2640 1 -- 172.21.15.38:0/1482426968 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99180cb530 con 0x7f991815db80 2024-04-25T08:40:07.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.193+0000 7f990dffb640 1 -- 172.21.15.38:0/1482426968 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f991000a640 con 0x7f991815db80 2024-04-25T08:40:07.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.193+0000 7f990dffb640 1 --2- 172.21.15.38:0/1482426968 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98fc06b4d0 0x7f98fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.193+0000 7f990dffb640 1 -- 172.21.15.38:0/1482426968 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f991008a2d0 con 0x7f991815db80 2024-04-25T08:40:07.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.193+0000 7f991d67e640 1 --2- 172.21.15.38:0/1482426968 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98fc06b4d0 0x7f98fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.194+0000 7f991d67e640 1 --2- 172.21.15.38:0/1482426968 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98fc06b4d0 0x7f98fc06d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9914006fd0 tx=0x7f9914008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:07.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.194+0000 7f99225e2740 1 Processor -- start 2024-04-25T08:40:07.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.194+0000 7f99225e2740 1 -- start start 2024-04-25T08:40:07.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f99225e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69b9230 0x561ef69b9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f99225e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69d6ad0 0x561ef69d7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f99225e2740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef698ef10 0x561ef69d5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f99225e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef68f1a10 con 0x561ef69d6ad0 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f99225e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef689ab30 con 0x561ef69b9230 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f99225e2740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ef68edc50 con 0x561ef698ef10 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f991ce7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69d6ad0 0x561ef69d7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f991ce7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69d6ad0 0x561ef69d7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44642/0 (socket says 172.21.15.38:44642) 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f991ce7d640 1 -- 172.21.15.38:0/988361031 learned_addr learned my addr 172.21.15.38:0/988361031 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f991d67e640 1 --2- 172.21.15.38:0/988361031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69b9230 0x561ef69b9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f991de7f640 1 --2- 172.21.15.38:0/988361031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef698ef10 0x561ef69d5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f991ce7d640 1 -- 172.21.15.38:0/988361031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef698ef10 msgr2=0x561ef69d5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f991ce7d640 1 --2- 172.21.15.38:0/988361031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef698ef10 0x561ef69d5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f991ce7d640 1 -- 172.21.15.38:0/988361031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69b9230 msgr2=0x561ef69b9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:07.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f991ce7d640 1 --2- 172.21.15.38:0/988361031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69b9230 0x561ef69b9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f991ce7d640 1 -- 172.21.15.38:0/988361031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ef68afe10 con 0x561ef69d6ad0 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f991de7f640 1 --2- 172.21.15.38:0/988361031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef698ef10 0x561ef69d5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f991ce7d640 1 --2- 172.21.15.38:0/988361031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69d6ad0 0x561ef69d7020 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f99080090b0 tx=0x7f990802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a4e23143690b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f9902ffd640 1 -- 172.21.15.38:0/988361031 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908037040 con 0x561ef69d6ad0 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f9902ffd640 1 -- 172.21.15.38:0/988361031 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99080327a0 con 0x561ef69d6ad0 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.195+0000 7f9902ffd640 1 -- 172.21.15.38:0/988361031 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9908032b50 con 0x561ef69d6ad0 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.196+0000 7f99225e2740 1 -- 172.21.15.38:0/988361031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69d6ad0 msgr2=0x561ef69d7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.196+0000 7f99225e2740 1 --2- 172.21.15.38:0/988361031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69d6ad0 0x561ef69d7020 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f99080090b0 tx=0x7f990802c000 comp rx=0 tx=0).stop 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.196+0000 7f99225e2740 1 -- 172.21.15.38:0/988361031 shutdown_connections 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.196+0000 7f99225e2740 1 --2- 172.21.15.38:0/988361031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef698ef10 0x561ef69d5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.196+0000 7f99225e2740 1 --2- 172.21.15.38:0/988361031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69d6ad0 0x561ef69d7020 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.196+0000 7f99225e2740 1 --2- 172.21.15.38:0/988361031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69b9230 0x561ef69b9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.196+0000 7f99225e2740 1 -- 172.21.15.38:0/988361031 >> 172.21.15.38:0/988361031 conn(0x561ef698b840 msgr2=0x561ef69d4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:07.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.196+0000 7f99225e2740 1 -- 172.21.15.38:0/988361031 shutdown_connections 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.196+0000 7f99225e2740 1 -- 172.21.15.38:0/988361031 wait complete. 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.196+0000 7f99225e2740 1 -- start start 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.197+0000 7f99225e2740 1 Processor -- start 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.197+0000 7f99225e2740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69b9230 0x561ef69dac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.197+0000 7f99225e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69d6ad0 0x561ef69bbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.197+0000 7f99225e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69bc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.197+0000 7f99225e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef698fcb0 con 0x561ef69da670 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.197+0000 7f99225e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef68ae0c0 con 0x561ef69d6ad0 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.197+0000 7f99225e2740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ef6989770 con 0x561ef69b9230 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f99225e2740 10 client.0 _ll_register_callbacks cb 0x561ef68a1ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f991de7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69bc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f991de7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69bc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44652/0 (socket says 172.21.15.38:44652) 2024-04-25T08:40:07.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f991de7f640 1 -- 172.21.15.38:0/3688769187 learned_addr learned my addr 172.21.15.38:0/3688769187 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f991d67e640 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69b9230 0x561ef69dac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f991ce7d640 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69d6ad0 0x561ef69bbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f991de7f640 1 -- 172.21.15.38:0/3688769187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69b9230 msgr2=0x561ef69dac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f991de7f640 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69b9230 0x561ef69dac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f991de7f640 1 -- 172.21.15.38:0/3688769187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69d6ad0 msgr2=0x561ef69bbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f991de7f640 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69d6ad0 0x561ef69bbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f991de7f640 1 -- 172.21.15.38:0/3688769187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ef66062d0 con 0x561ef69da670 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f991de7f640 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69bc170 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f9910060aa0 tx=0x7f9910058ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1ed6bb06ccc8bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f98f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f98f7fff640 1 -- 172.21.15.38:0/3688769187 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910070020 con 0x561ef69da670 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f98f7fff640 1 -- 172.21.15.38:0/3688769187 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9910056c70 con 0x561ef69da670 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f99225e2740 1 -- 172.21.15.38:0/3688769187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99080327a0 con 0x561ef69da670 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f98f7fff640 1 -- 172.21.15.38:0/3688769187 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9910054fe0 con 0x561ef69da670 2024-04-25T08:40:07.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f99225e2740 10 client.9018 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f99225e2740 1 -- 172.21.15.38:0/3688769187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x561ef66d06b0 con 0x561ef69da670 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.198+0000 7f98f7fff640 1 -- 172.21.15.38:0/3688769187 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99080327a0 con 0x561ef69da670 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.199+0000 7f98f7fff640 1 -- 172.21.15.38:0/3688769187 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f99100701c0 con 0x561ef69da670 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.199+0000 7f99225e2740 10 client.9018 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.199+0000 7f99225e2740 10 client.9018 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.199+0000 7f99225e2740 20 client.9018 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.199+0000 7f99225e2740 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x561ef69d9100 0x561ef69d94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.199+0000 7f99225e2740 4 client.9018 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.199+0000 7f99225e2740 1 -- 172.21.15.38:0/3688769187 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ef6950920 con 0x561ef69d9100 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.199+0000 7f991d67e640 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x561ef69d9100 0x561ef69d94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.199+0000 7f98f7fff640 10 client.9018 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.199+0000 7f991d67e640 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x561ef69d9100 0x561ef69d94e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20b584d457718621 server_cookie=266e749621972014 in_seq=0 out_seq=0 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f98f7fff640 1 -- 172.21.15.38:0/3688769187 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ef6950920 con 0x561ef69d9100 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f98f7fff640 10 client.9018 handle_command_reply: tid=0 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f99225e2740 1 client.9018 shutdown 2024-04-25T08:40:07.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f99225e2740 1 -- 172.21.15.38:0/3688769187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 msgr2=0x561ef69bc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f99225e2740 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69bc170 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f9910060aa0 tx=0x7f9910058ab0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f99225e2740 1 -- 172.21.15.38:0/3688769187 shutdown_connections 2024-04-25T08:40:07.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f99225e2740 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x561ef69d9100 0x561ef69d94e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f99225e2740 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69bc170 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f99225e2740 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69d6ad0 0x561ef69bbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f99225e2740 1 --2- 172.21.15.38:0/3688769187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69b9230 0x561ef69dac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f98f7fff640 0 client.9018 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:07.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f99225e2740 1 -- 172.21.15.38:0/3688769187 >> 172.21.15.38:0/3688769187 conn(0x561ef698ef10 msgr2=0x561ef69d7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:07.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f99225e2740 1 -- 172.21.15.38:0/3688769187 shutdown_connections 2024-04-25T08:40:07.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f99225e2740 1 -- 172.21.15.38:0/3688769187 wait complete. 2024-04-25T08:40:07.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.200+0000 7f99225e2740 20 client.9018 trim_cache size 0 max 16384 2024-04-25T08:40:07.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.216+0000 7f99225e2740 1 Processor -- start 2024-04-25T08:40:07.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.216+0000 7f99225e2740 1 -- start start 2024-04-25T08:40:07.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f99225e2740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69b9230 0x561ef6988950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f99225e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69d6ad0 0x561ef69d94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f99225e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69d9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f99225e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99181ff530 con 0x561ef69da670 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f99225e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99180ccac0 con 0x561ef69d6ad0 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f99225e2740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99180cd0c0 con 0x561ef69b9230 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f991de7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69d9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f991de7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69d9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44654/0 (socket says 172.21.15.38:44654) 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f991de7f640 1 -- 172.21.15.38:0/2054198295 learned_addr learned my addr 172.21.15.38:0/2054198295 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f991de7f640 1 -- 172.21.15.38:0/2054198295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69b9230 msgr2=0x561ef6988950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f991de7f640 1 --2- 172.21.15.38:0/2054198295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69b9230 0x561ef6988950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f991de7f640 1 -- 172.21.15.38:0/2054198295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69d6ad0 msgr2=0x561ef69d94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f991de7f640 1 --2- 172.21.15.38:0/2054198295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69d6ad0 0x561ef69d94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f991de7f640 1 -- 172.21.15.38:0/2054198295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ef69d6510 con 0x561ef69da670 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f991de7f640 1 --2- 172.21.15.38:0/2054198295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69d9a20 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f9910002230 tx=0x7f991005a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e315f749e2657b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:07.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f99027fc640 1 -- 172.21.15.38:0/2054198295 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910070040 con 0x561ef69da670 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f99027fc640 1 -- 172.21.15.38:0/2054198295 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9910088320 con 0x561ef69da670 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.217+0000 7f99027fc640 1 -- 172.21.15.38:0/2054198295 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99100701e0 con 0x561ef69da670 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.218+0000 7f99225e2740 1 -- 172.21.15.38:0/2054198295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 msgr2=0x561ef69d9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.218+0000 7f99225e2740 1 --2- 172.21.15.38:0/2054198295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69d9a20 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f9910002230 tx=0x7f991005a000 comp rx=0 tx=0).stop 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.218+0000 7f99225e2740 1 -- 172.21.15.38:0/2054198295 shutdown_connections 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.218+0000 7f99225e2740 1 --2- 172.21.15.38:0/2054198295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69d9a20 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.218+0000 7f99225e2740 1 --2- 172.21.15.38:0/2054198295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69d6ad0 0x561ef69d94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.218+0000 7f99225e2740 1 --2- 172.21.15.38:0/2054198295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69b9230 0x561ef6988950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.218+0000 7f99225e2740 1 -- 172.21.15.38:0/2054198295 >> 172.21.15.38:0/2054198295 conn(0x561ef698ef10 msgr2=0x561ef698f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.218+0000 7f99225e2740 1 -- 172.21.15.38:0/2054198295 shutdown_connections 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.218+0000 7f99225e2740 1 -- 172.21.15.38:0/2054198295 wait complete. 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.218+0000 7f99225e2740 1 -- start start 2024-04-25T08:40:07.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.219+0000 7f99225e2740 1 Processor -- start 2024-04-25T08:40:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.219+0000 7f99225e2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69b9230 0x561ef6a3ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.219+0000 7f99225e2740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69d6ad0 0x561ef69b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.219+0000 7f99225e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69b9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.219+0000 7f99225e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef69bb1f0 con 0x561ef69da670 2024-04-25T08:40:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.219+0000 7f99225e2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ef69bac30 con 0x561ef69b9230 2024-04-25T08:40:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.219+0000 7f99225e2740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ef69ba900 con 0x561ef69d6ad0 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.219+0000 7f99225e2740 10 client.0 _ll_register_callbacks cb 0x561ef6989c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f991de7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69b9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f991de7f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69b9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44668/0 (socket says 172.21.15.38:44668) 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f991de7f640 1 -- 172.21.15.38:0/2650648461 learned_addr learned my addr 172.21.15.38:0/2650648461 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f991d67e640 1 --2- 172.21.15.38:0/2650648461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69b9230 0x561ef6a3ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f991de7f640 1 -- 172.21.15.38:0/2650648461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69d6ad0 msgr2=0x561ef69b9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f991de7f640 1 --2- 172.21.15.38:0/2650648461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69d6ad0 0x561ef69b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f991de7f640 1 -- 172.21.15.38:0/2650648461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69b9230 msgr2=0x561ef6a3ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f991de7f640 1 --2- 172.21.15.38:0/2650648461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69b9230 0x561ef6a3ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f991de7f640 1 -- 172.21.15.38:0/2650648461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ef69bbce0 con 0x561ef69da670 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f991de7f640 1 --2- 172.21.15.38:0/2650648461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69b9fc0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9910001330 tx=0x7f9910088540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd91e4ce1444f0e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f98f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f98f7fff640 1 -- 172.21.15.38:0/2650648461 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910070040 con 0x561ef69da670 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f98f7fff640 1 -- 172.21.15.38:0/2650648461 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9910057dc0 con 0x561ef69da670 2024-04-25T08:40:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f99225e2740 1 -- 172.21.15.38:0/2650648461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9910088320 con 0x561ef69da670 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f98f7fff640 1 -- 172.21.15.38:0/2650648461 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9910057fe0 con 0x561ef69da670 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f99225e2740 10 client.9024 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f99225e2740 1 -- 172.21.15.38:0/2650648461 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x561ef69bbce0 con 0x561ef69da670 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f98f7fff640 1 -- 172.21.15.38:0/2650648461 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9910088320 con 0x561ef69da670 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f98f7fff640 1 -- 172.21.15.38:0/2650648461 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f99100546c0 con 0x561ef69da670 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f99225e2740 10 client.9024 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f99225e2740 10 client.9024 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f99225e2740 20 client.9024 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f99225e2740 1 --2- 172.21.15.38:0/2650648461 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x561ef69ee2a0 0x561ef69e64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f99225e2740 4 client.9024 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.220+0000 7f99225e2740 1 -- 172.21.15.38:0/2650648461 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561ef68fef50 con 0x561ef69ee2a0 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f991d67e640 1 --2- 172.21.15.38:0/2650648461 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x561ef69ee2a0 0x561ef69e64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f991d67e640 1 --2- 172.21.15.38:0/2650648461 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x561ef69ee2a0 0x561ef69e64b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c16c7db46db766a server_cookie=9ef1453be82e25ec in_seq=0 out_seq=0 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f98f7fff640 10 client.9024 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f98f7fff640 1 -- 172.21.15.38:0/2650648461 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x561ef68fef50 con 0x561ef69ee2a0 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f98f7fff640 10 client.9024 handle_command_reply: tid=0 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f99225e2740 1 client.9024 shutdown 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f99225e2740 1 -- 172.21.15.38:0/2650648461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 msgr2=0x561ef69b9fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f99225e2740 1 --2- 172.21.15.38:0/2650648461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69b9fc0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f9910001330 tx=0x7f9910088540 comp rx=0 tx=0).stop 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f99225e2740 1 -- 172.21.15.38:0/2650648461 shutdown_connections 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f99225e2740 1 --2- 172.21.15.38:0/2650648461 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x561ef69ee2a0 0x561ef69e64b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f99225e2740 1 --2- 172.21.15.38:0/2650648461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561ef69da670 0x561ef69b9fc0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f99225e2740 1 --2- 172.21.15.38:0/2650648461 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x561ef69d6ad0 0x561ef69b9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f98f7fff640 0 client.9024 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f99225e2740 1 --2- 172.21.15.38:0/2650648461 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561ef69b9230 0x561ef6a3ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.221+0000 7f99225e2740 1 -- 172.21.15.38:0/2650648461 >> 172.21.15.38:0/2650648461 conn(0x561ef698ef10 msgr2=0x561ef69d9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f99225e2740 1 -- 172.21.15.38:0/2650648461 shutdown_connections 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f99225e2740 1 -- 172.21.15.38:0/2650648461 wait complete. 2024-04-25T08:40:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f99225e2740 20 client.9024 trim_cache size 0 max 16384 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 7639, 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 9.0993033709999995, 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 70, 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 121, 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 121, 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 14.136077896 2024-04-25T08:40:07.221 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:40:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f991f8e2640 1 -- 172.21.15.38:0/1482426968 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98fc06b4d0 msgr2=0x7f98fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f991f8e2640 1 --2- 172.21.15.38:0/1482426968 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98fc06b4d0 0x7f98fc06d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f9914006fd0 tx=0x7f9914008040 comp rx=0 tx=0).stop 2024-04-25T08:40:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f991f8e2640 1 -- 172.21.15.38:0/1482426968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f991815db80 msgr2=0x7f991815e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f991f8e2640 1 --2- 172.21.15.38:0/1482426968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f991815db80 0x7f991815e620 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f991000c730 tx=0x7f991000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f991f8e2640 1 -- 172.21.15.38:0/1482426968 shutdown_connections 2024-04-25T08:40:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f991f8e2640 1 --2- 172.21.15.38:0/1482426968 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f991815db80 0x7f991815e620 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f991f8e2640 1 --2- 172.21.15.38:0/1482426968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99180d0660 0x7f991815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f991f8e2640 1 --2- 172.21.15.38:0/1482426968 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f98fc06b4d0 0x7f98fc06d990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f991f8e2640 1 --2- 172.21.15.38:0/1482426968 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99180b9600 0x7f9918159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f991f8e2640 1 -- 172.21.15.38:0/1482426968 >> 172.21.15.38:0/1482426968 conn(0x7f99180668f0 msgr2=0x7f99180b88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f991f8e2640 1 -- 172.21.15.38:0/1482426968 shutdown_connections 2024-04-25T08:40:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.222+0000 7f991f8e2640 1 -- 172.21.15.38:0/1482426968 wait complete. 2024-04-25T08:40:07.226 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:40:07.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.335+0000 7fbf42390640 1 Processor -- start 2024-04-25T08:40:07.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.335+0000 7fbf42390640 1 -- start start 2024-04-25T08:40:07.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.335+0000 7fbf42390640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbf3c15a8f0 0x7fbf3c0cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.335+0000 7fbf42390640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf3c0cb6c0 0x7fbf3c0d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.335+0000 7fbf42390640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbf3c0d0780 0x7fbf3c0cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.335+0000 7fbf42390640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c0cbd20 con 0x7fbf3c15a8f0 2024-04-25T08:40:07.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.335+0000 7fbf42390640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3c0cc000 con 0x7fbf3c0d0780 2024-04-25T08:40:07.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.335+0000 7fbf42390640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0cc2e0 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.335+0000 7fbf3bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbf3c15a8f0 0x7fbf3c0cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.336+0000 7fbf3bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbf3c15a8f0 0x7fbf3c0cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44670/0 (socket says 172.21.15.38:44670) 2024-04-25T08:40:07.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.336+0000 7fbf3bfff640 1 -- 172.21.15.38:0/4040438699 learned_addr learned my addr 172.21.15.38:0/4040438699 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:07.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.336+0000 7fbf4092d640 1 --2- 172.21.15.38:0/4040438699 >> v2:172.21.15.107:3300/0 conn(0x7fbf3c0d0780 0x7fbf3c0cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.336+0000 7fbf3b7fe640 1 --2- 172.21.15.38:0/4040438699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf3c0cb6c0 0x7fbf3c0d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.336+0000 7fbf3bfff640 1 -- 172.21.15.38:0/4040438699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf3c0cb6c0 msgr2=0x7fbf3c0d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:07.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.336+0000 7fbf3bfff640 1 --2- 172.21.15.38:0/4040438699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf3c0cb6c0 0x7fbf3c0d0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.336+0000 7fbf3bfff640 1 -- 172.21.15.38:0/4040438699 >> v2:172.21.15.107:3300/0 conn(0x7fbf3c0d0780 msgr2=0x7fbf3c0cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:07.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.336+0000 7fbf3bfff640 1 --2- 172.21.15.38:0/4040438699 >> v2:172.21.15.107:3300/0 conn(0x7fbf3c0d0780 0x7fbf3c0cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.336+0000 7fbf3bfff640 1 -- 172.21.15.38:0/4040438699 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c0cc5c0 con 0x7fbf3c15a8f0 2024-04-25T08:40:07.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf3bfff640 1 --2- 172.21.15.38:0/4040438699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c15a8f0 0x7fbf3c0cb180 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0d13c0 tx=0x7fbf2402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f7e432737095f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:07.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf3affd640 1 -- 172.21.15.38:0/4040438699 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24037070 con 0x7fbf3c15a8f0 2024-04-25T08:40:07.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf3affd640 1 -- 172.21.15.38:0/4040438699 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf24032440 con 0x7fbf3c15a8f0 2024-04-25T08:40:07.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf3affd640 1 -- 172.21.15.38:0/4040438699 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24032810 con 0x7fbf3c15a8f0 2024-04-25T08:40:07.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf42390640 1 -- 172.21.15.38:0/4040438699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c15a8f0 msgr2=0x7fbf3c0cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf42390640 1 --2- 172.21.15.38:0/4040438699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c15a8f0 0x7fbf3c0cb180 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0d13c0 tx=0x7fbf2402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf42390640 1 -- 172.21.15.38:0/4040438699 shutdown_connections 2024-04-25T08:40:07.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf42390640 1 --2- 172.21.15.38:0/4040438699 >> v2:172.21.15.107:3300/0 conn(0x7fbf3c0d0780 0x7fbf3c0cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf42390640 1 --2- 172.21.15.38:0/4040438699 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf3c0cb6c0 0x7fbf3c0d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf42390640 1 --2- 172.21.15.38:0/4040438699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c15a8f0 0x7fbf3c0cb180 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf42390640 1 -- 172.21.15.38:0/4040438699 >> 172.21.15.38:0/4040438699 conn(0x7fbf3c072610 msgr2=0x7fbf3c099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:07.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf42390640 1 -- 172.21.15.38:0/4040438699 shutdown_connections 2024-04-25T08:40:07.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.337+0000 7fbf42390640 1 -- 172.21.15.38:0/4040438699 wait complete. 2024-04-25T08:40:07.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.338+0000 7fbf42390640 1 Processor -- start 2024-04-25T08:40:07.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.339+0000 7fbf42390640 1 -- start start 2024-04-25T08:40:07.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.339+0000 7fbf42390640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c0cb6c0 0x7fbf3c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.339+0000 7fbf42390640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf3c0d0780 0x7fbf3c1f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.339+0000 7fbf42390640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf3c15a8f0 0x7fbf3c1f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.339+0000 7fbf42390640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0ce470 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.339+0000 7fbf42390640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c0d0c10 con 0x7fbf3c15a8f0 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.339+0000 7fbf42390640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c15acd0 con 0x7fbf3c0d0780 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf3bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c0cb6c0 0x7fbf3c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf3bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c0cb6c0 0x7fbf3c15d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44674/0 (socket says 172.21.15.38:44674) 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf3bfff640 1 -- 172.21.15.38:0/2590385592 learned_addr learned my addr 172.21.15.38:0/2590385592 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf4092d640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf3c15a8f0 0x7fbf3c1f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf3b7fe640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf3c0d0780 0x7fbf3c1f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf3bfff640 1 -- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf3c0d0780 msgr2=0x7fbf3c1f6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf3bfff640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf3c0d0780 0x7fbf3c1f6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf3bfff640 1 -- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf3c15a8f0 msgr2=0x7fbf3c1f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf3bfff640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf3c15a8f0 0x7fbf3c1f6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf3bfff640 1 -- 172.21.15.38:0/2590385592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3c15bff0 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf3bfff640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c0cb6c0 0x7fbf3c15d860 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fbf24000c00 tx=0x7fbf240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:07.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf38ff9640 1 -- 172.21.15.38:0/2590385592 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24041070 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf38ff9640 1 -- 172.21.15.38:0/2590385592 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2403c870 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf42390640 1 -- 172.21.15.38:0/2590385592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c15c2d0 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf38ff9640 1 -- 172.21.15.38:0/2590385592 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24037040 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.340+0000 7fbf42390640 1 -- 172.21.15.38:0/2590385592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c15c890 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.341+0000 7fbf42390640 1 -- 172.21.15.38:0/2590385592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c1e5600 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.342+0000 7fbf38ff9640 1 -- 172.21.15.38:0/2590385592 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf24044070 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.342+0000 7fbf38ff9640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbf1c06b500 0x7fbf1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:07.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.343+0000 7fbf38ff9640 1 -- 172.21.15.38:0/2590385592 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbf240aad10 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.343+0000 7fbf3b7fe640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbf1c06b500 0x7fbf1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:07.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.343+0000 7fbf3b7fe640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbf1c06b500 0x7fbf1c06d9c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0b20c0 tx=0x7fbf2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:07.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.347+0000 7fbf38ff9640 1 -- 172.21.15.38:0/2590385592 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf240b0020 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.564+0000 7fbf42390640 1 -- 172.21.15.38:0/2590385592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbf3c0cdb20 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.564+0000 7fbf38ff9640 1 -- 172.21.15.38:0/2590385592 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbf24079430 con 0x7fbf3c0cb6c0 2024-04-25T08:40:07.558 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:40:07.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.566+0000 7fbf42390640 1 -- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbf1c06b500 msgr2=0x7fbf1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.566+0000 7fbf42390640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbf1c06b500 0x7fbf1c06d9c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c0b20c0 tx=0x7fbf2c008040 comp rx=0 tx=0).stop 2024-04-25T08:40:07.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.566+0000 7fbf42390640 1 -- 172.21.15.38:0/2590385592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c0cb6c0 msgr2=0x7fbf3c15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:07.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.566+0000 7fbf42390640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c0cb6c0 0x7fbf3c15d860 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fbf24000c00 tx=0x7fbf240046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.566+0000 7fbf42390640 1 -- 172.21.15.38:0/2590385592 shutdown_connections 2024-04-25T08:40:07.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.566+0000 7fbf42390640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbf3c15a8f0 0x7fbf3c1f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.566+0000 7fbf42390640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbf1c06b500 0x7fbf1c06d9c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.566+0000 7fbf42390640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbf3c0d0780 0x7fbf3c1f6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.566+0000 7fbf42390640 1 --2- 172.21.15.38:0/2590385592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbf3c0cb6c0 0x7fbf3c15d860 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:07.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.566+0000 7fbf42390640 1 -- 172.21.15.38:0/2590385592 >> 172.21.15.38:0/2590385592 conn(0x7fbf3c072610 msgr2=0x7fbf3c06eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:07.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.566+0000 7fbf42390640 1 -- 172.21.15.38:0/2590385592 shutdown_connections 2024-04-25T08:40:07.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:07.566+0000 7fbf42390640 1 -- 172.21.15.38:0/2590385592 wait complete. 2024-04-25T08:40:07.570 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:40:07.570 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:40:07.571 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:40:07.571 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:40:07.571 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:40:07.571 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:40:07.571 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:40:07.571 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:40:07.571 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:40:07.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:07.571 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:40:07.587 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:40:07.587 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:40:07.587 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:40:07.587 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:40:07.587 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:40:07.587 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:40:07.587 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:40:07.587 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:40:07.587 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42230sec preferred_lft 42230sec 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:40:07.588 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:40:07.589 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:40:07.589 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::440f:97ff:fe63:87b/64 scope link 2024-04-25T08:40:07.589 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:40:07.589 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:40:07.589 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ea:23:76:ab:b1:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:40:07.589 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::e823:76ff:feab:b1e9/64 scope link 2024-04-25T08:40:07.589 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:40:07.589 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:40:07.589 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:40:07.589 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::38c7:38ff:fe2e:6c92/64 scope link 2024-04-25T08:40:07.589 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:40:07.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:07.589 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:40:07.650 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:40:07.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:07.652 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:40:07.707 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:40:07.707 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:40:07.707 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:40:07.708 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:40:07.708 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:40:07.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:07.708 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:40:07.708 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:07.708 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:40:07.708 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:40:07.708 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:40:07.708 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:07.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:07.793 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:40:07.808 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:40:07.808 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:40:07.808 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:40:07.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:07.808 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T08:40:07.872 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:40:07.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:40:07.872 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:40:07.927 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:40:07.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:40:07.927 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:40:07.981 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:40:08.046 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:40:08.103 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:40:08.103 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:40:08.103 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:40:08.103 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:40:08.103 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:40:08.103 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:40:08.103 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:40:08.103 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:40:08.103 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:40:08.103 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:40:08.104 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:40:08.105 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:40:08.105 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:40:08.105 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:40:08.105 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:40:08.105 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:40:08.105 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:40:08.105 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:40:08.105 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:40:08.105 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:40:08.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:08.105 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:40:08.158 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:40:08.158 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:40:08.200 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:40:08.224 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:40:08.281 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:40:08.281 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:40:08.281 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:40:08.281 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:40:08.281 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:40:08.281 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:40:08.281 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:40:08.281 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:40:08.281 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:40:08.282 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:40:08.283 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:40:08.283 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:40:08.283 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:40:08.283 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:40:08.283 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:40:08.283 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:40:08.283 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:40:08.283 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:40:08.283 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:40:08.283 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:40:08.284 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:40:08.284 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:40:08.284 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:40:08.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:08.284 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:40:08.285 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-04-25T08:40:08.289+0000 7f111a473480 -1 init, newargv = 0x5591c9066430 newargc=15 2024-04-25T08:40:08.285 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[89010]: starting ceph client 2024-04-25T08:40:08.287 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[89010]: starting fuse 2024-04-25T08:40:08.299 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:40:08.299 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:40:08.299 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:40:08.300 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:40:08.301 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:40:08.301 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:40:08.301 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:40:08.301 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:40:08.301 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:40:08.301 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:40:08.301 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:40:08.301 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:40:08.301 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-04-25T08:40:08.301 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:40:08.419 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.89010.asok 2024-04-25T08:40:08.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:40:08.430 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.89010.asok 2024-04-25T08:40:08.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:08.430 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.89010.asok status 2024-04-25T08:40:08.568 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:40:08.568 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:40:08.568 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:40:08.568 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:40:08.568 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-04-25T08:40:08.568 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:40:08.568 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:40:08.568 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "89010", 2024-04-25T08:40:08.568 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:40:08.568 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:08.568 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 9039, 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 9039 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3712755574 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:40:08.569 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:40:08.570 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3712755574 2024-04-25T08:40:08.570 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:08.570 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.9039 192.168.0.1:0/3712755574", 2024-04-25T08:40:08.570 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/3712755574", 2024-04-25T08:40:08.570 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:40:08.570 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 70, 2024-04-25T08:40:08.570 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 121, 2024-04-25T08:40:08.570 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:40:08.570 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:40:08.570 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:40:08.570 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:40:08.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:08.576 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:40:08.576 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:40:08.592 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:40:08.592 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:40:08.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:08.592 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:40:08.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:08.658 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:08.658 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:08.658 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T08:40:08.658 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:08.658 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:08.658 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:08.658 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:08.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.740 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.740 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112439 s, 93.3 MB/s 2024-04-25T08:40:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889425 s, 118 MB/s 2024-04-25T08:40:08.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.763 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.763 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985325 s, 106 MB/s 2024-04-25T08:40:08.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.772 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727996 s, 144 MB/s 2024-04-25T08:40:08.783 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.783 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.784 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886238 s, 118 MB/s 2024-04-25T08:40:08.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.796 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100145 s, 105 MB/s 2024-04-25T08:40:08.808 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.808 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.808 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009877 s, 106 MB/s 2024-04-25T08:40:08.820 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.821 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.821 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101261 s, 104 MB/s 2024-04-25T08:40:08.831 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.831 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.831 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813865 s, 129 MB/s 2024-04-25T08:40:08.841 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.841 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.841 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741365 s, 141 MB/s 2024-04-25T08:40:08.852 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.852 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.852 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816386 s, 128 MB/s 2024-04-25T08:40:08.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.863 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824388 s, 127 MB/s 2024-04-25T08:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824491 s, 127 MB/s 2024-04-25T08:40:08.884 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.884 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.884 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867327 s, 121 MB/s 2024-04-25T08:40:08.894 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.894 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.894 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812962 s, 129 MB/s 2024-04-25T08:40:08.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.906 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890195 s, 118 MB/s 2024-04-25T08:40:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.917 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0092323 s, 114 MB/s 2024-04-25T08:40:08.928 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.928 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.928 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825141 s, 127 MB/s 2024-04-25T08:40:08.938 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.938 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.938 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795713 s, 132 MB/s 2024-04-25T08:40:08.947 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.947 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.947 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726482 s, 144 MB/s 2024-04-25T08:40:08.959 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.959 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.959 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918123 s, 114 MB/s 2024-04-25T08:40:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.969 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799028 s, 131 MB/s 2024-04-25T08:40:08.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.980 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887022 s, 118 MB/s 2024-04-25T08:40:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880009 s, 119 MB/s 2024-04-25T08:40:09.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.002 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844484 s, 124 MB/s 2024-04-25T08:40:09.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.013 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902877 s, 116 MB/s 2024-04-25T08:40:09.023 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.024 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.024 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805274 s, 130 MB/s 2024-04-25T08:40:09.033 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.033 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.033 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775713 s, 135 MB/s 2024-04-25T08:40:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.044 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854105 s, 123 MB/s 2024-04-25T08:40:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.055 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886137 s, 118 MB/s 2024-04-25T08:40:09.066 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.066 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.066 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854782 s, 123 MB/s 2024-04-25T08:40:09.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.077 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085797 s, 122 MB/s 2024-04-25T08:40:09.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:09.078 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:09.078 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:09.078 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T08:40:09.078 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:09.078 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:09.078 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:09.079 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:09.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847678 s, 124 MB/s 2024-04-25T08:40:09.157 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.158 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076669 s, 137 MB/s 2024-04-25T08:40:09.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.167 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761461 s, 138 MB/s 2024-04-25T08:40:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804789 s, 130 MB/s 2024-04-25T08:40:09.187 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.188 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778082 s, 135 MB/s 2024-04-25T08:40:09.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.198 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794364 s, 132 MB/s 2024-04-25T08:40:09.207 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.207 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.207 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743823 s, 141 MB/s 2024-04-25T08:40:09.217 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.217 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.217 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745873 s, 141 MB/s 2024-04-25T08:40:09.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.228 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783065 s, 134 MB/s 2024-04-25T08:40:09.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.238 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758379 s, 138 MB/s 2024-04-25T08:40:09.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.248 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772995 s, 136 MB/s 2024-04-25T08:40:09.258 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.258 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.258 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778132 s, 135 MB/s 2024-04-25T08:40:09.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.267 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737609 s, 142 MB/s 2024-04-25T08:40:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.278 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784163 s, 134 MB/s 2024-04-25T08:40:09.289 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.289 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.289 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900596 s, 116 MB/s 2024-04-25T08:40:09.299 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.299 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.299 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073146 s, 143 MB/s 2024-04-25T08:40:09.309 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.309 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.309 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746842 s, 140 MB/s 2024-04-25T08:40:09.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.319 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810648 s, 129 MB/s 2024-04-25T08:40:09.329 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.329 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.329 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765215 s, 137 MB/s 2024-04-25T08:40:09.340 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.340 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.340 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847452 s, 124 MB/s 2024-04-25T08:40:09.351 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.351 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.351 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881054 s, 119 MB/s 2024-04-25T08:40:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.360 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741998 s, 141 MB/s 2024-04-25T08:40:09.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.371 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853342 s, 123 MB/s 2024-04-25T08:40:09.381 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.381 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.381 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772838 s, 136 MB/s 2024-04-25T08:40:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.391 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079067 s, 133 MB/s 2024-04-25T08:40:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781746 s, 134 MB/s 2024-04-25T08:40:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.411 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790904 s, 133 MB/s 2024-04-25T08:40:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.423 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868931 s, 121 MB/s 2024-04-25T08:40:09.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.432 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778638 s, 135 MB/s 2024-04-25T08:40:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.442 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762185 s, 138 MB/s 2024-04-25T08:40:09.452 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.452 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.452 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788004 s, 133 MB/s 2024-04-25T08:40:09.462 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.462 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.462 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753263 s, 139 MB/s 2024-04-25T08:40:09.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:09.463 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:09.463 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:09.463 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T08:40:09.463 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:09.464 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:09.464 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:09.464 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:09.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.533 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870658 s, 120 MB/s 2024-04-25T08:40:09.544 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.544 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.544 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879959 s, 119 MB/s 2024-04-25T08:40:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714655 s, 147 MB/s 2024-04-25T08:40:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.564 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749387 s, 140 MB/s 2024-04-25T08:40:09.574 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.574 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.574 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848763 s, 124 MB/s 2024-04-25T08:40:09.585 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.585 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.585 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789191 s, 133 MB/s 2024-04-25T08:40:09.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.596 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866521 s, 121 MB/s 2024-04-25T08:40:09.605 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.606 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743157 s, 141 MB/s 2024-04-25T08:40:09.615 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.615 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.615 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733748 s, 143 MB/s 2024-04-25T08:40:09.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.626 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879254 s, 119 MB/s 2024-04-25T08:40:09.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.636 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789528 s, 133 MB/s 2024-04-25T08:40:09.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.646 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076891 s, 136 MB/s 2024-04-25T08:40:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.656 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752654 s, 139 MB/s 2024-04-25T08:40:09.667 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.667 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.667 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864981 s, 121 MB/s 2024-04-25T08:40:09.677 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.677 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.677 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747059 s, 140 MB/s 2024-04-25T08:40:09.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.688 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863107 s, 121 MB/s 2024-04-25T08:40:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872172 s, 120 MB/s 2024-04-25T08:40:09.709 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.709 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.710 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748617 s, 140 MB/s 2024-04-25T08:40:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.721 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862657 s, 122 MB/s 2024-04-25T08:40:09.730 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.730 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.731 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786035 s, 133 MB/s 2024-04-25T08:40:09.741 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.741 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.741 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814297 s, 129 MB/s 2024-04-25T08:40:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.751 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770533 s, 136 MB/s 2024-04-25T08:40:09.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.762 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879481 s, 119 MB/s 2024-04-25T08:40:09.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.772 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791677 s, 132 MB/s 2024-04-25T08:40:09.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.782 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748695 s, 140 MB/s 2024-04-25T08:40:09.792 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.792 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.792 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765287 s, 137 MB/s 2024-04-25T08:40:09.802 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.802 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.802 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764231 s, 137 MB/s 2024-04-25T08:40:09.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.813 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859389 s, 122 MB/s 2024-04-25T08:40:09.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.823 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784348 s, 134 MB/s 2024-04-25T08:40:09.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.833 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751012 s, 140 MB/s 2024-04-25T08:40:09.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.844 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888061 s, 118 MB/s 2024-04-25T08:40:09.855 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.855 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.855 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860053 s, 122 MB/s 2024-04-25T08:40:09.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:09.856 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:09.856 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:09.856 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T08:40:09.857 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:09.857 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:09.857 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:09.857 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:09.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.926 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084804 s, 124 MB/s 2024-04-25T08:40:09.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.937 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.937 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845429 s, 124 MB/s 2024-04-25T08:40:09.947 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.947 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.947 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811977 s, 129 MB/s 2024-04-25T08:40:09.957 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.957 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.957 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767618 s, 137 MB/s 2024-04-25T08:40:09.967 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.967 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.967 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077936 s, 135 MB/s 2024-04-25T08:40:09.977 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.977 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.977 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762689 s, 137 MB/s 2024-04-25T08:40:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849436 s, 123 MB/s 2024-04-25T08:40:09.997 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073975 s, 142 MB/s 2024-04-25T08:40:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824279 s, 127 MB/s 2024-04-25T08:40:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791665 s, 132 MB/s 2024-04-25T08:40:10.029 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.029 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.030 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887589 s, 118 MB/s 2024-04-25T08:40:10.039 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.040 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.040 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079045 s, 133 MB/s 2024-04-25T08:40:10.049 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.049 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.049 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728034 s, 144 MB/s 2024-04-25T08:40:10.059 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.059 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.059 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731732 s, 143 MB/s 2024-04-25T08:40:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783249 s, 134 MB/s 2024-04-25T08:40:10.079 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.079 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.079 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787107 s, 133 MB/s 2024-04-25T08:40:10.089 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.089 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.089 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770092 s, 136 MB/s 2024-04-25T08:40:10.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.099 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744457 s, 141 MB/s 2024-04-25T08:40:10.108 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.109 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763072 s, 137 MB/s 2024-04-25T08:40:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.118 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.119 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756183 s, 139 MB/s 2024-04-25T08:40:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846569 s, 124 MB/s 2024-04-25T08:40:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.139 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746287 s, 141 MB/s 2024-04-25T08:40:10.150 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.150 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.150 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881123 s, 119 MB/s 2024-04-25T08:40:10.160 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.160 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.160 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746214 s, 141 MB/s 2024-04-25T08:40:10.170 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.170 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.170 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755922 s, 139 MB/s 2024-04-25T08:40:10.180 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.180 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.180 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733804 s, 143 MB/s 2024-04-25T08:40:10.190 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.190 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.190 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077907 s, 135 MB/s 2024-04-25T08:40:10.200 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.200 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.200 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761898 s, 138 MB/s 2024-04-25T08:40:10.209 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726672 s, 144 MB/s 2024-04-25T08:40:10.219 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.220 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.220 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790378 s, 133 MB/s 2024-04-25T08:40:10.230 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.230 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.230 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751664 s, 140 MB/s 2024-04-25T08:40:10.240 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.240 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.240 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791372 s, 133 MB/s 2024-04-25T08:40:10.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:10.241 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:10.241 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:10.241 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T08:40:10.241 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:10.241 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:10.241 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:10.241 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:10.310 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.310 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.310 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754529 s, 139 MB/s 2024-04-25T08:40:10.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.319 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749971 s, 140 MB/s 2024-04-25T08:40:10.329 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.329 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.329 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774459 s, 135 MB/s 2024-04-25T08:40:10.340 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.340 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.340 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854526 s, 123 MB/s 2024-04-25T08:40:10.350 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.350 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.350 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744736 s, 141 MB/s 2024-04-25T08:40:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761567 s, 138 MB/s 2024-04-25T08:40:10.370 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.371 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085517 s, 123 MB/s 2024-04-25T08:40:10.380 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.381 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.381 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752509 s, 139 MB/s 2024-04-25T08:40:10.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.391 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802325 s, 131 MB/s 2024-04-25T08:40:10.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.401 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765538 s, 137 MB/s 2024-04-25T08:40:10.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.411 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772854 s, 136 MB/s 2024-04-25T08:40:10.421 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.421 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.421 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743902 s, 141 MB/s 2024-04-25T08:40:10.430 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.431 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742265 s, 141 MB/s 2024-04-25T08:40:10.441 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.441 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.441 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761735 s, 138 MB/s 2024-04-25T08:40:10.450 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.450 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.450 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075648 s, 139 MB/s 2024-04-25T08:40:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.461 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008106 s, 129 MB/s 2024-04-25T08:40:10.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.471 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765823 s, 137 MB/s 2024-04-25T08:40:10.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.480 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.480 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693498 s, 151 MB/s 2024-04-25T08:40:10.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.489 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736533 s, 142 MB/s 2024-04-25T08:40:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879103 s, 119 MB/s 2024-04-25T08:40:10.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.510 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761337 s, 138 MB/s 2024-04-25T08:40:10.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.520 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737287 s, 142 MB/s 2024-04-25T08:40:10.530 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.530 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.530 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785374 s, 134 MB/s 2024-04-25T08:40:10.540 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.540 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.540 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765063 s, 137 MB/s 2024-04-25T08:40:10.549 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.549 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.550 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752466 s, 139 MB/s 2024-04-25T08:40:10.559 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.559 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.559 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752807 s, 139 MB/s 2024-04-25T08:40:10.568 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.568 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.568 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006942 s, 151 MB/s 2024-04-25T08:40:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.578 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710796 s, 148 MB/s 2024-04-25T08:40:10.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.589 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883383 s, 119 MB/s 2024-04-25T08:40:10.600 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.600 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.600 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838908 s, 125 MB/s 2024-04-25T08:40:10.611 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.611 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.611 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829811 s, 126 MB/s 2024-04-25T08:40:10.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.621 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.621 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752228 s, 139 MB/s 2024-04-25T08:40:10.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:10.622 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:10.622 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:10.622 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T08:40:10.622 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:10.622 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:10.622 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:10.622 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.690 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745277 s, 141 MB/s 2024-04-25T08:40:10.700 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.700 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.700 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802738 s, 131 MB/s 2024-04-25T08:40:10.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.711 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879402 s, 119 MB/s 2024-04-25T08:40:10.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.721 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790628 s, 133 MB/s 2024-04-25T08:40:10.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.732 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.732 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838884 s, 125 MB/s 2024-04-25T08:40:10.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.743 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801389 s, 131 MB/s 2024-04-25T08:40:10.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.752 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778968 s, 135 MB/s 2024-04-25T08:40:10.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.761 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716309 s, 146 MB/s 2024-04-25T08:40:10.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.772 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795897 s, 132 MB/s 2024-04-25T08:40:10.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.782 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805288 s, 130 MB/s 2024-04-25T08:40:10.793 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.793 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.793 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084471 s, 124 MB/s 2024-04-25T08:40:10.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.803 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797635 s, 131 MB/s 2024-04-25T08:40:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.813 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802523 s, 131 MB/s 2024-04-25T08:40:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777537 s, 135 MB/s 2024-04-25T08:40:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.833 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793694 s, 132 MB/s 2024-04-25T08:40:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692754 s, 151 MB/s 2024-04-25T08:40:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767652 s, 137 MB/s 2024-04-25T08:40:10.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.862 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756059 s, 139 MB/s 2024-04-25T08:40:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793721 s, 132 MB/s 2024-04-25T08:40:10.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.882 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742064 s, 141 MB/s 2024-04-25T08:40:10.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.892 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755931 s, 139 MB/s 2024-04-25T08:40:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.901 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774097 s, 135 MB/s 2024-04-25T08:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.912 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085129 s, 123 MB/s 2024-04-25T08:40:10.922 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.922 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.922 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768918 s, 136 MB/s 2024-04-25T08:40:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.933 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833992 s, 126 MB/s 2024-04-25T08:40:10.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.944 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870725 s, 120 MB/s 2024-04-25T08:40:10.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.954 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079035 s, 133 MB/s 2024-04-25T08:40:10.964 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846623 s, 124 MB/s 2024-04-25T08:40:10.975 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.975 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.975 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805552 s, 130 MB/s 2024-04-25T08:40:10.984 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.984 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.984 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711154 s, 147 MB/s 2024-04-25T08:40:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:10.994 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749138 s, 140 MB/s 2024-04-25T08:40:11.004 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.004 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.004 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774129 s, 135 MB/s 2024-04-25T08:40:11.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:11.005 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:11.005 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:11.005 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T08:40:11.005 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:11.005 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:11.005 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:11.006 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.075 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863272 s, 121 MB/s 2024-04-25T08:40:11.084 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.084 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.084 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721113 s, 145 MB/s 2024-04-25T08:40:11.094 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.094 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.095 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759416 s, 138 MB/s 2024-04-25T08:40:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.104 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763288 s, 137 MB/s 2024-04-25T08:40:11.113 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.114 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.114 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739465 s, 142 MB/s 2024-04-25T08:40:11.123 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.123 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.123 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752805 s, 139 MB/s 2024-04-25T08:40:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.133 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748039 s, 140 MB/s 2024-04-25T08:40:11.143 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.143 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.143 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741422 s, 141 MB/s 2024-04-25T08:40:11.153 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.153 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.154 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086298 s, 122 MB/s 2024-04-25T08:40:11.164 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.164 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.164 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829669 s, 126 MB/s 2024-04-25T08:40:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.175 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.175 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825434 s, 127 MB/s 2024-04-25T08:40:11.184 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.184 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.184 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724553 s, 145 MB/s 2024-04-25T08:40:11.194 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.194 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.195 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821412 s, 128 MB/s 2024-04-25T08:40:11.204 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.204 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.204 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695709 s, 151 MB/s 2024-04-25T08:40:11.212 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.212 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.212 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066821 s, 157 MB/s 2024-04-25T08:40:11.222 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.222 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.222 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733305 s, 143 MB/s 2024-04-25T08:40:11.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.231 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714715 s, 147 MB/s 2024-04-25T08:40:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.241 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717779 s, 146 MB/s 2024-04-25T08:40:11.251 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.252 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.252 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851783 s, 123 MB/s 2024-04-25T08:40:11.261 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.261 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.261 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719702 s, 146 MB/s 2024-04-25T08:40:11.270 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.270 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.270 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741329 s, 141 MB/s 2024-04-25T08:40:11.281 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.281 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.281 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835244 s, 126 MB/s 2024-04-25T08:40:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.291 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793815 s, 132 MB/s 2024-04-25T08:40:11.300 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.300 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.300 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691998 s, 152 MB/s 2024-04-25T08:40:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.310 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.310 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726636 s, 144 MB/s 2024-04-25T08:40:11.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.319 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754635 s, 139 MB/s 2024-04-25T08:40:11.329 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.329 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.329 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713325 s, 147 MB/s 2024-04-25T08:40:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660981 s, 159 MB/s 2024-04-25T08:40:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.347 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710757 s, 148 MB/s 2024-04-25T08:40:11.357 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.357 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.357 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807015 s, 130 MB/s 2024-04-25T08:40:11.367 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.367 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.367 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730705 s, 144 MB/s 2024-04-25T08:40:11.376 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.376 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.376 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071841 s, 146 MB/s 2024-04-25T08:40:11.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:11.377 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:11.377 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:11.377 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T08:40:11.377 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:11.377 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:11.377 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:11.378 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:11.445 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.446 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.446 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075631 s, 139 MB/s 2024-04-25T08:40:11.456 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.456 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.456 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814358 s, 129 MB/s 2024-04-25T08:40:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.466 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772134 s, 136 MB/s 2024-04-25T08:40:11.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.476 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730956 s, 143 MB/s 2024-04-25T08:40:11.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.485 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729273 s, 144 MB/s 2024-04-25T08:40:11.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.496 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810161 s, 129 MB/s 2024-04-25T08:40:11.505 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.505 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.505 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725442 s, 145 MB/s 2024-04-25T08:40:11.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.515 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067094 s, 156 MB/s 2024-04-25T08:40:11.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.523 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670695 s, 156 MB/s 2024-04-25T08:40:11.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.533 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073519 s, 143 MB/s 2024-04-25T08:40:11.543 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.543 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.544 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724194 s, 145 MB/s 2024-04-25T08:40:11.552 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.552 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.553 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721588 s, 145 MB/s 2024-04-25T08:40:11.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.562 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714753 s, 147 MB/s 2024-04-25T08:40:11.572 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.572 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.572 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810632 s, 129 MB/s 2024-04-25T08:40:11.581 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.581 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.582 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728436 s, 144 MB/s 2024-04-25T08:40:11.591 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.592 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.592 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809822 s, 129 MB/s 2024-04-25T08:40:11.601 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.601 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.601 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711737 s, 147 MB/s 2024-04-25T08:40:11.610 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.610 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.610 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672954 s, 156 MB/s 2024-04-25T08:40:11.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.620 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749497 s, 140 MB/s 2024-04-25T08:40:11.629 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.629 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.629 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714495 s, 147 MB/s 2024-04-25T08:40:11.638 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.638 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.638 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689156 s, 152 MB/s 2024-04-25T08:40:11.649 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.649 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.649 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832417 s, 126 MB/s 2024-04-25T08:40:11.658 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.658 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.658 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739703 s, 142 MB/s 2024-04-25T08:40:11.669 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.669 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.669 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826699 s, 127 MB/s 2024-04-25T08:40:11.679 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.679 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.679 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744576 s, 141 MB/s 2024-04-25T08:40:11.689 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.689 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.689 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801101 s, 131 MB/s 2024-04-25T08:40:11.699 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.699 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.699 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724924 s, 145 MB/s 2024-04-25T08:40:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796779 s, 132 MB/s 2024-04-25T08:40:11.718 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.719 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.719 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736637 s, 142 MB/s 2024-04-25T08:40:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081623 s, 128 MB/s 2024-04-25T08:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.739 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718213 s, 146 MB/s 2024-04-25T08:40:11.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.750 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.750 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839522 s, 125 MB/s 2024-04-25T08:40:11.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:11.751 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:11.751 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:11.751 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T08:40:11.751 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:11.751 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:11.751 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:11.751 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:11.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824278 s, 127 MB/s 2024-04-25T08:40:11.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.829 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739524 s, 142 MB/s 2024-04-25T08:40:11.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.839 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834542 s, 126 MB/s 2024-04-25T08:40:11.849 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.849 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.849 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007346 s, 143 MB/s 2024-04-25T08:40:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.859 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768107 s, 137 MB/s 2024-04-25T08:40:11.868 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.868 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.868 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007153 s, 147 MB/s 2024-04-25T08:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710866 s, 148 MB/s 2024-04-25T08:40:11.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.888 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768223 s, 136 MB/s 2024-04-25T08:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.897 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763596 s, 137 MB/s 2024-04-25T08:40:11.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.906 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067637 s, 155 MB/s 2024-04-25T08:40:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.918 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864535 s, 121 MB/s 2024-04-25T08:40:11.928 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.928 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.928 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813151 s, 129 MB/s 2024-04-25T08:40:11.937 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.937 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.937 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686416 s, 153 MB/s 2024-04-25T08:40:11.947 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.947 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.947 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812292 s, 129 MB/s 2024-04-25T08:40:11.958 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.958 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.959 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865767 s, 121 MB/s 2024-04-25T08:40:11.968 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.968 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.968 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741842 s, 141 MB/s 2024-04-25T08:40:11.977 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.978 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.978 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681077 s, 154 MB/s 2024-04-25T08:40:11.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.988 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820632 s, 128 MB/s 2024-04-25T08:40:11.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:11.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:11.999 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782053 s, 134 MB/s 2024-04-25T08:40:12.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.008 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747644 s, 140 MB/s 2024-04-25T08:40:12.017 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.017 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.017 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675375 s, 155 MB/s 2024-04-25T08:40:12.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.027 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753517 s, 139 MB/s 2024-04-25T08:40:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725693 s, 144 MB/s 2024-04-25T08:40:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.046 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.047 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771489 s, 136 MB/s 2024-04-25T08:40:12.056 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.056 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.056 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766006 s, 137 MB/s 2024-04-25T08:40:12.067 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.067 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.067 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851024 s, 123 MB/s 2024-04-25T08:40:12.078 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.078 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.078 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847735 s, 124 MB/s 2024-04-25T08:40:12.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.087 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714647 s, 147 MB/s 2024-04-25T08:40:12.097 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.097 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.097 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738438 s, 142 MB/s 2024-04-25T08:40:12.106 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.107 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737533 s, 142 MB/s 2024-04-25T08:40:12.116 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.116 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.116 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007622 s, 138 MB/s 2024-04-25T08:40:12.126 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.126 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.126 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796535 s, 132 MB/s 2024-04-25T08:40:12.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:12.128 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:12.128 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:12.128 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T08:40:12.128 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:12.128 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:12.128 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:12.128 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:12.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.197 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823783 s, 127 MB/s 2024-04-25T08:40:12.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.207 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765379 s, 137 MB/s 2024-04-25T08:40:12.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.216 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744937 s, 141 MB/s 2024-04-25T08:40:12.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.225 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656936 s, 160 MB/s 2024-04-25T08:40:12.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.234 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073189 s, 143 MB/s 2024-04-25T08:40:12.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.245 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822216 s, 128 MB/s 2024-04-25T08:40:12.253 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666601 s, 157 MB/s 2024-04-25T08:40:12.264 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.264 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.264 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800054 s, 131 MB/s 2024-04-25T08:40:12.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.273 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066721 s, 157 MB/s 2024-04-25T08:40:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768342 s, 136 MB/s 2024-04-25T08:40:12.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.294 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841894 s, 125 MB/s 2024-04-25T08:40:12.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.303 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744381 s, 141 MB/s 2024-04-25T08:40:12.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.314 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822303 s, 128 MB/s 2024-04-25T08:40:12.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.324 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.324 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743953 s, 141 MB/s 2024-04-25T08:40:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.333 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729362 s, 144 MB/s 2024-04-25T08:40:12.343 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.343 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.343 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826633 s, 127 MB/s 2024-04-25T08:40:12.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.354 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075047 s, 140 MB/s 2024-04-25T08:40:12.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.364 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.364 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797198 s, 132 MB/s 2024-04-25T08:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789612 s, 133 MB/s 2024-04-25T08:40:12.384 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.384 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.384 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801042 s, 131 MB/s 2024-04-25T08:40:12.394 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.394 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.394 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007481 s, 140 MB/s 2024-04-25T08:40:12.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.403 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738269 s, 142 MB/s 2024-04-25T08:40:12.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.414 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818132 s, 128 MB/s 2024-04-25T08:40:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.424 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722383 s, 145 MB/s 2024-04-25T08:40:12.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.432 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672674 s, 156 MB/s 2024-04-25T08:40:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754596 s, 139 MB/s 2024-04-25T08:40:12.452 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.452 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.452 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759055 s, 138 MB/s 2024-04-25T08:40:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.462 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747333 s, 140 MB/s 2024-04-25T08:40:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.471 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.471 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703677 s, 149 MB/s 2024-04-25T08:40:12.481 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.481 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.481 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769208 s, 136 MB/s 2024-04-25T08:40:12.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.491 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075229 s, 139 MB/s 2024-04-25T08:40:12.501 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.501 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.501 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720555 s, 146 MB/s 2024-04-25T08:40:12.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:12.501 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:12.501 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:12.501 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T08:40:12.501 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:12.501 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:12.501 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:12.501 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:12.570 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.570 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.570 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832165 s, 126 MB/s 2024-04-25T08:40:12.579 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.579 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.579 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695997 s, 151 MB/s 2024-04-25T08:40:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741384 s, 141 MB/s 2024-04-25T08:40:12.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.599 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.599 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737283 s, 142 MB/s 2024-04-25T08:40:12.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.608 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726989 s, 144 MB/s 2024-04-25T08:40:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.617 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691404 s, 152 MB/s 2024-04-25T08:40:12.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.628 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084675 s, 124 MB/s 2024-04-25T08:40:12.638 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.638 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.638 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844248 s, 124 MB/s 2024-04-25T08:40:12.648 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.648 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.648 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738616 s, 142 MB/s 2024-04-25T08:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067517 s, 155 MB/s 2024-04-25T08:40:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665822 s, 157 MB/s 2024-04-25T08:40:12.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.676 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704215 s, 149 MB/s 2024-04-25T08:40:12.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.686 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074349 s, 141 MB/s 2024-04-25T08:40:12.696 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.696 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.696 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820807 s, 128 MB/s 2024-04-25T08:40:12.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.705 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750324 s, 140 MB/s 2024-04-25T08:40:12.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.715 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776701 s, 135 MB/s 2024-04-25T08:40:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.725 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685941 s, 153 MB/s 2024-04-25T08:40:12.734 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.734 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.734 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758296 s, 138 MB/s 2024-04-25T08:40:12.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.744 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823237 s, 127 MB/s 2024-04-25T08:40:12.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.754 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727173 s, 144 MB/s 2024-04-25T08:40:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750999 s, 140 MB/s 2024-04-25T08:40:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819029 s, 128 MB/s 2024-04-25T08:40:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.784 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810204 s, 129 MB/s 2024-04-25T08:40:12.793 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.794 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.794 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730692 s, 144 MB/s 2024-04-25T08:40:12.804 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.804 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.804 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080368 s, 130 MB/s 2024-04-25T08:40:12.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.814 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752469 s, 139 MB/s 2024-04-25T08:40:12.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.823 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711257 s, 147 MB/s 2024-04-25T08:40:12.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.833 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774439 s, 135 MB/s 2024-04-25T08:40:12.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.843 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746726 s, 140 MB/s 2024-04-25T08:40:12.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.854 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809006 s, 130 MB/s 2024-04-25T08:40:12.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.863 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704257 s, 149 MB/s 2024-04-25T08:40:12.874 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.874 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.874 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086754 s, 121 MB/s 2024-04-25T08:40:12.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:12.875 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:12.875 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:12.875 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T08:40:12.875 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:12.876 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:12.876 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:12.876 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:12.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.945 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.945 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766838 s, 137 MB/s 2024-04-25T08:40:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.954 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671432 s, 156 MB/s 2024-04-25T08:40:12.964 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.964 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.965 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813205 s, 129 MB/s 2024-04-25T08:40:12.976 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.976 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.976 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862632 s, 122 MB/s 2024-04-25T08:40:12.985 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.986 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.986 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713634 s, 147 MB/s 2024-04-25T08:40:12.995 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:12.996 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:12.996 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749289 s, 140 MB/s 2024-04-25T08:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761646 s, 138 MB/s 2024-04-25T08:40:13.015 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.016 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.016 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734499 s, 143 MB/s 2024-04-25T08:40:13.025 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.025 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.026 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726508 s, 144 MB/s 2024-04-25T08:40:13.035 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.036 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.036 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755759 s, 139 MB/s 2024-04-25T08:40:13.045 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.045 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.045 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071951 s, 146 MB/s 2024-04-25T08:40:13.056 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.056 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.056 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824512 s, 127 MB/s 2024-04-25T08:40:13.065 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.065 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066516 s, 158 MB/s 2024-04-25T08:40:13.075 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.075 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.075 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731633 s, 143 MB/s 2024-04-25T08:40:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755029 s, 139 MB/s 2024-04-25T08:40:13.096 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.096 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.096 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810241 s, 129 MB/s 2024-04-25T08:40:13.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.107 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865326 s, 121 MB/s 2024-04-25T08:40:13.118 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.118 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.118 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815607 s, 129 MB/s 2024-04-25T08:40:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.128 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806837 s, 130 MB/s 2024-04-25T08:40:13.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.139 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786573 s, 133 MB/s 2024-04-25T08:40:13.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754976 s, 139 MB/s 2024-04-25T08:40:13.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.158 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804388 s, 130 MB/s 2024-04-25T08:40:13.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.169 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076556 s, 137 MB/s 2024-04-25T08:40:13.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.178 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762427 s, 138 MB/s 2024-04-25T08:40:13.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.188 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759178 s, 138 MB/s 2024-04-25T08:40:13.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.198 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753476 s, 139 MB/s 2024-04-25T08:40:13.207 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.208 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731369 s, 143 MB/s 2024-04-25T08:40:13.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.218 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858161 s, 122 MB/s 2024-04-25T08:40:13.229 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.229 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.229 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813113 s, 129 MB/s 2024-04-25T08:40:13.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.238 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680713 s, 154 MB/s 2024-04-25T08:40:13.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.248 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821046 s, 128 MB/s 2024-04-25T08:40:13.259 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.259 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.259 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865132 s, 121 MB/s 2024-04-25T08:40:13.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:13.260 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:13.260 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:13.260 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T08:40:13.261 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:13.261 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:13.261 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:13.261 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:13.329 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.329 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.329 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759374 s, 138 MB/s 2024-04-25T08:40:13.339 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.339 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.339 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743275 s, 141 MB/s 2024-04-25T08:40:13.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.349 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082701 s, 127 MB/s 2024-04-25T08:40:13.358 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.358 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.358 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680683 s, 154 MB/s 2024-04-25T08:40:13.368 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.368 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.368 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758669 s, 138 MB/s 2024-04-25T08:40:13.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.378 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747563 s, 140 MB/s 2024-04-25T08:40:13.388 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.388 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.388 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769421 s, 136 MB/s 2024-04-25T08:40:13.397 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.397 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.397 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714888 s, 147 MB/s 2024-04-25T08:40:13.406 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.406 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.406 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680698 s, 154 MB/s 2024-04-25T08:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.416 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738605 s, 142 MB/s 2024-04-25T08:40:13.427 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.427 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.427 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082215 s, 128 MB/s 2024-04-25T08:40:13.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.437 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082462 s, 127 MB/s 2024-04-25T08:40:13.448 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.448 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.448 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082159 s, 128 MB/s 2024-04-25T08:40:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731537 s, 143 MB/s 2024-04-25T08:40:13.467 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.467 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.467 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722234 s, 145 MB/s 2024-04-25T08:40:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714223 s, 147 MB/s 2024-04-25T08:40:13.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.486 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688058 s, 152 MB/s 2024-04-25T08:40:13.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.496 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763941 s, 137 MB/s 2024-04-25T08:40:13.505 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.505 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.505 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720333 s, 146 MB/s 2024-04-25T08:40:13.516 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.516 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.516 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832081 s, 126 MB/s 2024-04-25T08:40:13.526 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.526 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.526 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770066 s, 136 MB/s 2024-04-25T08:40:13.535 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.535 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.535 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688331 s, 152 MB/s 2024-04-25T08:40:13.545 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.545 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.545 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757023 s, 139 MB/s 2024-04-25T08:40:13.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.554 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733045 s, 143 MB/s 2024-04-25T08:40:13.564 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.564 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.564 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755255 s, 139 MB/s 2024-04-25T08:40:13.573 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.573 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.573 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710648 s, 148 MB/s 2024-04-25T08:40:13.583 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.583 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.583 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713026 s, 147 MB/s 2024-04-25T08:40:13.593 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.593 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.593 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739271 s, 142 MB/s 2024-04-25T08:40:13.602 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.602 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.602 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674418 s, 155 MB/s 2024-04-25T08:40:13.612 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.612 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.612 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073403 s, 143 MB/s 2024-04-25T08:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.622 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741777 s, 141 MB/s 2024-04-25T08:40:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.631 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730949 s, 143 MB/s 2024-04-25T08:40:13.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:13.632 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:13.632 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:13.633 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T08:40:13.633 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:13.633 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:13.633 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:13.633 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847336 s, 124 MB/s 2024-04-25T08:40:13.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.711 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686555 s, 153 MB/s 2024-04-25T08:40:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.721 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754534 s, 139 MB/s 2024-04-25T08:40:13.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.731 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781438 s, 134 MB/s 2024-04-25T08:40:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860475 s, 122 MB/s 2024-04-25T08:40:13.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.752 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735855 s, 142 MB/s 2024-04-25T08:40:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.762 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765154 s, 137 MB/s 2024-04-25T08:40:13.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.772 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784348 s, 134 MB/s 2024-04-25T08:40:13.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.782 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071134 s, 147 MB/s 2024-04-25T08:40:13.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.791 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750064 s, 140 MB/s 2024-04-25T08:40:13.800 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.800 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.800 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685597 s, 153 MB/s 2024-04-25T08:40:13.810 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.810 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.810 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713797 s, 147 MB/s 2024-04-25T08:40:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745929 s, 141 MB/s 2024-04-25T08:40:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.829 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722062 s, 145 MB/s 2024-04-25T08:40:13.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.838 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754887 s, 139 MB/s 2024-04-25T08:40:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.848 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764603 s, 137 MB/s 2024-04-25T08:40:13.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.858 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732374 s, 143 MB/s 2024-04-25T08:40:13.867 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.867 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.867 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727528 s, 144 MB/s 2024-04-25T08:40:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741881 s, 141 MB/s 2024-04-25T08:40:13.886 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.886 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.886 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681548 s, 154 MB/s 2024-04-25T08:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754733 s, 139 MB/s 2024-04-25T08:40:13.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082524 s, 127 MB/s 2024-04-25T08:40:13.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.917 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074726 s, 140 MB/s 2024-04-25T08:40:13.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.926 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810622 s, 129 MB/s 2024-04-25T08:40:13.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.936 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076837 s, 136 MB/s 2024-04-25T08:40:13.946 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.946 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.946 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764847 s, 137 MB/s 2024-04-25T08:40:13.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.956 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.956 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705254 s, 149 MB/s 2024-04-25T08:40:13.966 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.966 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.966 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834756 s, 126 MB/s 2024-04-25T08:40:13.977 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.977 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.977 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818833 s, 128 MB/s 2024-04-25T08:40:13.987 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.987 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.987 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835068 s, 126 MB/s 2024-04-25T08:40:13.997 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:13.997 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:13.997 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744918 s, 141 MB/s 2024-04-25T08:40:14.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.008 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836598 s, 125 MB/s 2024-04-25T08:40:14.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:14.009 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:14.009 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:14.009 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T08:40:14.009 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:14.009 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:14.009 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:14.009 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:14.076 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.077 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771267 s, 136 MB/s 2024-04-25T08:40:14.086 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.086 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.086 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746234 s, 141 MB/s 2024-04-25T08:40:14.096 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.096 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.096 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765328 s, 137 MB/s 2024-04-25T08:40:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760711 s, 138 MB/s 2024-04-25T08:40:14.116 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.116 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.116 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840282 s, 125 MB/s 2024-04-25T08:40:14.126 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794563 s, 132 MB/s 2024-04-25T08:40:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691716 s, 152 MB/s 2024-04-25T08:40:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735078 s, 143 MB/s 2024-04-25T08:40:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761459 s, 138 MB/s 2024-04-25T08:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.167 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856342 s, 122 MB/s 2024-04-25T08:40:14.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.177 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818579 s, 128 MB/s 2024-04-25T08:40:14.187 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.188 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759581 s, 138 MB/s 2024-04-25T08:40:14.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.199 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825411 s, 127 MB/s 2024-04-25T08:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.208 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750791 s, 140 MB/s 2024-04-25T08:40:14.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.218 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808573 s, 130 MB/s 2024-04-25T08:40:14.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.228 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750854 s, 140 MB/s 2024-04-25T08:40:14.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.238 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070381 s, 149 MB/s 2024-04-25T08:40:14.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.247 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702225 s, 149 MB/s 2024-04-25T08:40:14.256 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.257 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734798 s, 143 MB/s 2024-04-25T08:40:14.266 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.266 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.266 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706407 s, 148 MB/s 2024-04-25T08:40:14.277 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.277 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.277 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842488 s, 124 MB/s 2024-04-25T08:40:14.287 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.287 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.287 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751502 s, 140 MB/s 2024-04-25T08:40:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.297 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086529 s, 121 MB/s 2024-04-25T08:40:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.314 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820252 s, 128 MB/s 2024-04-25T08:40:14.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.323 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678855 s, 154 MB/s 2024-04-25T08:40:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675342 s, 155 MB/s 2024-04-25T08:40:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007502 s, 140 MB/s 2024-04-25T08:40:14.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.352 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791162 s, 133 MB/s 2024-04-25T08:40:14.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.362 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741794 s, 141 MB/s 2024-04-25T08:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.371 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675635 s, 155 MB/s 2024-04-25T08:40:14.381 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.382 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842977 s, 124 MB/s 2024-04-25T08:40:14.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.391 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689435 s, 152 MB/s 2024-04-25T08:40:14.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:14.392 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:14.392 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:14.392 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T08:40:14.392 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:14.392 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:14.392 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:14.392 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:14.460 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.460 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.460 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007687 s, 136 MB/s 2024-04-25T08:40:14.469 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.469 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.469 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074171 s, 141 MB/s 2024-04-25T08:40:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.478 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.479 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719404 s, 146 MB/s 2024-04-25T08:40:14.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.489 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817867 s, 128 MB/s 2024-04-25T08:40:14.498 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.498 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.498 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720526 s, 146 MB/s 2024-04-25T08:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.508 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076168 s, 138 MB/s 2024-04-25T08:40:14.518 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.518 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.518 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737521 s, 142 MB/s 2024-04-25T08:40:14.527 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.528 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731258 s, 143 MB/s 2024-04-25T08:40:14.537 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.537 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.537 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739936 s, 142 MB/s 2024-04-25T08:40:14.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.547 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757714 s, 138 MB/s 2024-04-25T08:40:14.558 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.558 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.558 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087874 s, 119 MB/s 2024-04-25T08:40:14.568 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.568 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.568 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075528 s, 139 MB/s 2024-04-25T08:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.577 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696667 s, 151 MB/s 2024-04-25T08:40:14.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.587 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776002 s, 135 MB/s 2024-04-25T08:40:14.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.598 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821294 s, 128 MB/s 2024-04-25T08:40:14.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.609 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846131 s, 124 MB/s 2024-04-25T08:40:14.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.618 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733764 s, 143 MB/s 2024-04-25T08:40:14.627 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.628 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706936 s, 148 MB/s 2024-04-25T08:40:14.638 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.638 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.639 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084665 s, 124 MB/s 2024-04-25T08:40:14.649 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.649 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.649 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860622 s, 122 MB/s 2024-04-25T08:40:14.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.659 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751243 s, 140 MB/s 2024-04-25T08:40:14.668 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724462 s, 145 MB/s 2024-04-25T08:40:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726597 s, 144 MB/s 2024-04-25T08:40:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715744 s, 147 MB/s 2024-04-25T08:40:14.698 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.698 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.698 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774754 s, 135 MB/s 2024-04-25T08:40:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.708 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826709 s, 127 MB/s 2024-04-25T08:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.719 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820503 s, 128 MB/s 2024-04-25T08:40:14.728 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.728 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.729 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739139 s, 142 MB/s 2024-04-25T08:40:14.738 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.738 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.738 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752601 s, 139 MB/s 2024-04-25T08:40:14.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.748 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071315 s, 147 MB/s 2024-04-25T08:40:14.757 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.757 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.757 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706431 s, 148 MB/s 2024-04-25T08:40:14.767 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.768 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811305 s, 129 MB/s 2024-04-25T08:40:14.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:14.769 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:14.769 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:14.769 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T08:40:14.770 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:14.770 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:14.770 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:14.770 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.838 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765384 s, 137 MB/s 2024-04-25T08:40:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.848 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754218 s, 139 MB/s 2024-04-25T08:40:14.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.857 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670228 s, 156 MB/s 2024-04-25T08:40:14.866 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.866 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.867 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731372 s, 143 MB/s 2024-04-25T08:40:14.876 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.876 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.876 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740877 s, 142 MB/s 2024-04-25T08:40:14.885 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.885 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.885 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718211 s, 146 MB/s 2024-04-25T08:40:14.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.895 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733487 s, 143 MB/s 2024-04-25T08:40:14.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.905 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818552 s, 128 MB/s 2024-04-25T08:40:14.915 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.915 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.915 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757235 s, 138 MB/s 2024-04-25T08:40:14.924 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.925 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.925 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709368 s, 148 MB/s 2024-04-25T08:40:14.934 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.934 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.934 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714235 s, 147 MB/s 2024-04-25T08:40:14.945 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.945 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.945 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812181 s, 129 MB/s 2024-04-25T08:40:14.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.954 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738043 s, 142 MB/s 2024-04-25T08:40:14.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.965 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082563 s, 127 MB/s 2024-04-25T08:40:14.975 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.976 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828205 s, 127 MB/s 2024-04-25T08:40:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.986 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.987 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085252 s, 123 MB/s 2024-04-25T08:40:14.997 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:14.997 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:14.997 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780022 s, 134 MB/s 2024-04-25T08:40:15.007 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.007 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.007 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751983 s, 139 MB/s 2024-04-25T08:40:15.016 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.017 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.017 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681912 s, 154 MB/s 2024-04-25T08:40:15.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.027 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791982 s, 132 MB/s 2024-04-25T08:40:15.037 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.037 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.037 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772765 s, 136 MB/s 2024-04-25T08:40:15.048 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.048 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.048 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815002 s, 129 MB/s 2024-04-25T08:40:15.057 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.058 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683672 s, 153 MB/s 2024-04-25T08:40:15.067 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.068 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.068 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721811 s, 145 MB/s 2024-04-25T08:40:15.078 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.078 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.078 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080856 s, 130 MB/s 2024-04-25T08:40:15.088 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.088 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.088 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701613 s, 149 MB/s 2024-04-25T08:40:15.098 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.098 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.098 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081639 s, 128 MB/s 2024-04-25T08:40:15.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.108 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688786 s, 152 MB/s 2024-04-25T08:40:15.118 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.118 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.118 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817416 s, 128 MB/s 2024-04-25T08:40:15.127 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.128 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704478 s, 149 MB/s 2024-04-25T08:40:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.137 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.138 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716974 s, 146 MB/s 2024-04-25T08:40:15.147 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743147 s, 141 MB/s 2024-04-25T08:40:15.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:15.149 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:15.149 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:15.150 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T08:40:15.150 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:15.150 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:15.150 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:15.150 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.218 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722782 s, 145 MB/s 2024-04-25T08:40:15.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.228 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070807 s, 148 MB/s 2024-04-25T08:40:15.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.237 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068596 s, 153 MB/s 2024-04-25T08:40:15.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.248 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822995 s, 127 MB/s 2024-04-25T08:40:15.258 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.258 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.258 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742535 s, 141 MB/s 2024-04-25T08:40:15.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.268 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.268 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698271 s, 150 MB/s 2024-04-25T08:40:15.276 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.277 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.277 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660223 s, 159 MB/s 2024-04-25T08:40:15.286 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.287 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.287 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720529 s, 146 MB/s 2024-04-25T08:40:15.299 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.299 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.299 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101253 s, 104 MB/s 2024-04-25T08:40:15.346 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.346 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.347 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079029 s, 133 MB/s 2024-04-25T08:40:15.356 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.356 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.356 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719431 s, 146 MB/s 2024-04-25T08:40:15.366 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.366 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.366 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760105 s, 138 MB/s 2024-04-25T08:40:15.376 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.376 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.376 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749726 s, 140 MB/s 2024-04-25T08:40:15.386 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.386 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.386 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746037 s, 141 MB/s 2024-04-25T08:40:15.395 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.395 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.395 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722976 s, 145 MB/s 2024-04-25T08:40:15.405 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.405 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.405 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744388 s, 141 MB/s 2024-04-25T08:40:15.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.415 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748886 s, 140 MB/s 2024-04-25T08:40:15.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.424 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680719 s, 154 MB/s 2024-04-25T08:40:15.433 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.433 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.433 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067752 s, 155 MB/s 2024-04-25T08:40:15.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.443 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075876 s, 138 MB/s 2024-04-25T08:40:15.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.454 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.454 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843767 s, 124 MB/s 2024-04-25T08:40:15.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.463 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699725 s, 150 MB/s 2024-04-25T08:40:15.474 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.474 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.474 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830456 s, 126 MB/s 2024-04-25T08:40:15.483 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.483 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.483 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745523 s, 141 MB/s 2024-04-25T08:40:15.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.494 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786967 s, 133 MB/s 2024-04-25T08:40:15.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.504 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733599 s, 143 MB/s 2024-04-25T08:40:15.512 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.513 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713494 s, 147 MB/s 2024-04-25T08:40:15.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.522 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779634 s, 134 MB/s 2024-04-25T08:40:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829578 s, 126 MB/s 2024-04-25T08:40:15.543 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.543 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.543 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755552 s, 139 MB/s 2024-04-25T08:40:15.552 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.552 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.552 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753775 s, 139 MB/s 2024-04-25T08:40:15.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.562 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740957 s, 142 MB/s 2024-04-25T08:40:15.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:15.563 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:15.564 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:15.564 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T08:40:15.564 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:15.564 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:15.564 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:15.564 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:15.632 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.632 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.633 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082373 s, 127 MB/s 2024-04-25T08:40:15.642 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.642 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.642 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727623 s, 144 MB/s 2024-04-25T08:40:15.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.652 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.652 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733507 s, 143 MB/s 2024-04-25T08:40:15.660 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.660 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.660 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668669 s, 157 MB/s 2024-04-25T08:40:15.671 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.671 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.671 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825822 s, 127 MB/s 2024-04-25T08:40:15.680 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.681 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.681 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725397 s, 145 MB/s 2024-04-25T08:40:15.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.692 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083962 s, 125 MB/s 2024-04-25T08:40:15.701 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.701 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.701 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780477 s, 134 MB/s 2024-04-25T08:40:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766743 s, 137 MB/s 2024-04-25T08:40:15.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.721 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752295 s, 139 MB/s 2024-04-25T08:40:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810255 s, 129 MB/s 2024-04-25T08:40:15.741 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.741 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.741 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699027 s, 150 MB/s 2024-04-25T08:40:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.751 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752845 s, 139 MB/s 2024-04-25T08:40:15.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.761 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736169 s, 142 MB/s 2024-04-25T08:40:15.770 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.770 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.770 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686411 s, 153 MB/s 2024-04-25T08:40:15.780 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.780 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.780 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077395 s, 135 MB/s 2024-04-25T08:40:15.790 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.790 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.790 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831882 s, 126 MB/s 2024-04-25T08:40:15.800 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.800 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.800 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705226 s, 149 MB/s 2024-04-25T08:40:15.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.809 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682584 s, 154 MB/s 2024-04-25T08:40:15.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783909 s, 134 MB/s 2024-04-25T08:40:15.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.829 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772568 s, 136 MB/s 2024-04-25T08:40:15.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.839 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751795 s, 139 MB/s 2024-04-25T08:40:15.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.848 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710444 s, 148 MB/s 2024-04-25T08:40:15.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.858 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694904 s, 151 MB/s 2024-04-25T08:40:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719323 s, 146 MB/s 2024-04-25T08:40:15.878 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.878 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.878 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848262 s, 124 MB/s 2024-04-25T08:40:15.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.887 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718833 s, 146 MB/s 2024-04-25T08:40:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.897 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075879 s, 138 MB/s 2024-04-25T08:40:15.907 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.907 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.907 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745921 s, 141 MB/s 2024-04-25T08:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838952 s, 125 MB/s 2024-04-25T08:40:15.927 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.927 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.927 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733526 s, 143 MB/s 2024-04-25T08:40:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:15.937 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752468 s, 139 MB/s 2024-04-25T08:40:15.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:15.939 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:15.939 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:15.939 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T08:40:15.939 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:15.939 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:15.939 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:15.939 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823498 s, 127 MB/s 2024-04-25T08:40:16.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.018 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748044 s, 140 MB/s 2024-04-25T08:40:16.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.027 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735985 s, 142 MB/s 2024-04-25T08:40:16.038 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.038 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.038 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831476 s, 126 MB/s 2024-04-25T08:40:16.047 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.048 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.048 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071547 s, 147 MB/s 2024-04-25T08:40:16.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.059 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.059 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084625 s, 124 MB/s 2024-04-25T08:40:16.069 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.069 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.069 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833791 s, 126 MB/s 2024-04-25T08:40:16.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.080 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804351 s, 130 MB/s 2024-04-25T08:40:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.089 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.090 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071193 s, 147 MB/s 2024-04-25T08:40:16.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.099 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721636 s, 145 MB/s 2024-04-25T08:40:16.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.109 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078793 s, 133 MB/s 2024-04-25T08:40:16.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.120 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820939 s, 128 MB/s 2024-04-25T08:40:16.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.130 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752943 s, 139 MB/s 2024-04-25T08:40:16.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.139 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720325 s, 146 MB/s 2024-04-25T08:40:16.149 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.149 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.150 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820837 s, 128 MB/s 2024-04-25T08:40:16.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.162 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993789 s, 106 MB/s 2024-04-25T08:40:16.172 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.172 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.172 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777886 s, 135 MB/s 2024-04-25T08:40:16.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.182 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082956 s, 126 MB/s 2024-04-25T08:40:16.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.191 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659173 s, 159 MB/s 2024-04-25T08:40:16.201 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.201 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.201 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074678 s, 140 MB/s 2024-04-25T08:40:16.210 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.210 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.210 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707372 s, 148 MB/s 2024-04-25T08:40:16.220 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.220 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.220 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741929 s, 141 MB/s 2024-04-25T08:40:16.230 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.230 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.230 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733269 s, 143 MB/s 2024-04-25T08:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.240 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741303 s, 141 MB/s 2024-04-25T08:40:16.250 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.250 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.250 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844283 s, 124 MB/s 2024-04-25T08:40:16.259 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.259 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.259 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656108 s, 160 MB/s 2024-04-25T08:40:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.269 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.269 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709091 s, 148 MB/s 2024-04-25T08:40:16.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.278 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748314 s, 140 MB/s 2024-04-25T08:40:16.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.288 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746651 s, 140 MB/s 2024-04-25T08:40:16.300 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.301 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.301 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103319 s, 101 MB/s 2024-04-25T08:40:16.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.332 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746997 s, 140 MB/s 2024-04-25T08:40:16.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.342 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733064 s, 143 MB/s 2024-04-25T08:40:16.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:16.343 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:16.343 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:16.343 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T08:40:16.344 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:16.344 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:16.344 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:16.344 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:16.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.412 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728885 s, 144 MB/s 2024-04-25T08:40:16.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.422 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806971 s, 130 MB/s 2024-04-25T08:40:16.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.432 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741466 s, 141 MB/s 2024-04-25T08:40:16.441 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.441 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.441 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073796 s, 142 MB/s 2024-04-25T08:40:16.451 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.451 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.451 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809474 s, 130 MB/s 2024-04-25T08:40:16.461 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.461 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.461 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699891 s, 150 MB/s 2024-04-25T08:40:16.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.471 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.471 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735241 s, 143 MB/s 2024-04-25T08:40:16.480 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.480 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.480 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708985 s, 148 MB/s 2024-04-25T08:40:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.490 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749848 s, 140 MB/s 2024-04-25T08:40:16.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.500 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823113 s, 127 MB/s 2024-04-25T08:40:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.509 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690964 s, 152 MB/s 2024-04-25T08:40:16.518 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.519 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694024 s, 151 MB/s 2024-04-25T08:40:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.528 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074217 s, 141 MB/s 2024-04-25T08:40:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.538 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736849 s, 142 MB/s 2024-04-25T08:40:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.547 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717951 s, 146 MB/s 2024-04-25T08:40:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.558 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826004 s, 127 MB/s 2024-04-25T08:40:16.567 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.567 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.567 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071611 s, 146 MB/s 2024-04-25T08:40:16.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.578 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.578 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799499 s, 131 MB/s 2024-04-25T08:40:16.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.587 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673653 s, 156 MB/s 2024-04-25T08:40:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.596 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070893 s, 148 MB/s 2024-04-25T08:40:16.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.607 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823591 s, 127 MB/s 2024-04-25T08:40:16.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.616 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742866 s, 141 MB/s 2024-04-25T08:40:16.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.627 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080746 s, 130 MB/s 2024-04-25T08:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.636 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722421 s, 145 MB/s 2024-04-25T08:40:16.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.647 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826655 s, 127 MB/s 2024-04-25T08:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.656 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710606 s, 148 MB/s 2024-04-25T08:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.665 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691574 s, 152 MB/s 2024-04-25T08:40:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.675 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740741 s, 142 MB/s 2024-04-25T08:40:16.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.686 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828144 s, 127 MB/s 2024-04-25T08:40:16.695 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.695 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.695 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724012 s, 145 MB/s 2024-04-25T08:40:16.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.705 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757785 s, 138 MB/s 2024-04-25T08:40:16.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.714 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733823 s, 143 MB/s 2024-04-25T08:40:16.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:16.716 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:16.716 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:16.716 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T08:40:16.716 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:16.716 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:16.716 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:16.716 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.784 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772127 s, 136 MB/s 2024-04-25T08:40:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.794 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007759 s, 135 MB/s 2024-04-25T08:40:16.805 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.805 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.805 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829796 s, 126 MB/s 2024-04-25T08:40:16.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.815 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.815 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703489 s, 149 MB/s 2024-04-25T08:40:16.825 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.825 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.826 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817471 s, 128 MB/s 2024-04-25T08:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075599 s, 139 MB/s 2024-04-25T08:40:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.846 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823334 s, 127 MB/s 2024-04-25T08:40:16.855 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.855 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.856 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738798 s, 142 MB/s 2024-04-25T08:40:16.865 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.865 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.865 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722276 s, 145 MB/s 2024-04-25T08:40:16.875 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.875 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.875 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810013 s, 129 MB/s 2024-04-25T08:40:16.886 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.886 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.886 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821982 s, 128 MB/s 2024-04-25T08:40:16.896 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.896 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.896 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734995 s, 143 MB/s 2024-04-25T08:40:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.905 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704618 s, 149 MB/s 2024-04-25T08:40:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.915 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735625 s, 143 MB/s 2024-04-25T08:40:16.924 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.925 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.925 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732987 s, 143 MB/s 2024-04-25T08:40:16.934 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.934 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.934 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073331 s, 143 MB/s 2024-04-25T08:40:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.944 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701103 s, 150 MB/s 2024-04-25T08:40:16.953 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.953 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.953 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683788 s, 153 MB/s 2024-04-25T08:40:16.962 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.962 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.962 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072789 s, 144 MB/s 2024-04-25T08:40:16.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.973 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793445 s, 132 MB/s 2024-04-25T08:40:16.983 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.983 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.983 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083027 s, 126 MB/s 2024-04-25T08:40:16.993 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:16.993 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:16.994 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825326 s, 127 MB/s 2024-04-25T08:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.003 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070958 s, 148 MB/s 2024-04-25T08:40:17.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.013 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744747 s, 141 MB/s 2024-04-25T08:40:17.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.023 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.023 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747241 s, 140 MB/s 2024-04-25T08:40:17.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.032 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725934 s, 144 MB/s 2024-04-25T08:40:17.042 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.042 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.042 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700356 s, 150 MB/s 2024-04-25T08:40:17.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.052 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816454 s, 128 MB/s 2024-04-25T08:40:17.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.062 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748654 s, 140 MB/s 2024-04-25T08:40:17.071 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.072 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.072 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719943 s, 146 MB/s 2024-04-25T08:40:17.081 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.081 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.081 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730563 s, 144 MB/s 2024-04-25T08:40:17.091 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.091 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.091 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073548 s, 143 MB/s 2024-04-25T08:40:17.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:17.092 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:17.092 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:17.092 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T08:40:17.092 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:17.092 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:17.092 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:17.092 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:17.160 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.161 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.161 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827201 s, 127 MB/s 2024-04-25T08:40:17.170 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.170 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.170 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711419 s, 147 MB/s 2024-04-25T08:40:17.179 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.179 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.179 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697298 s, 150 MB/s 2024-04-25T08:40:17.190 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.190 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.190 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809897 s, 129 MB/s 2024-04-25T08:40:17.199 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.199 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.199 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069073 s, 152 MB/s 2024-04-25T08:40:17.209 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.209 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.209 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755444 s, 139 MB/s 2024-04-25T08:40:17.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.219 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007192 s, 146 MB/s 2024-04-25T08:40:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.229 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814562 s, 129 MB/s 2024-04-25T08:40:17.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.239 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.239 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729227 s, 144 MB/s 2024-04-25T08:40:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.249 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752591 s, 139 MB/s 2024-04-25T08:40:17.258 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.258 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.258 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078646 s, 133 MB/s 2024-04-25T08:40:17.268 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.269 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.269 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775718 s, 135 MB/s 2024-04-25T08:40:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.278 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748489 s, 140 MB/s 2024-04-25T08:40:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.287 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692261 s, 151 MB/s 2024-04-25T08:40:17.299 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.299 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.299 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946183 s, 111 MB/s 2024-04-25T08:40:17.333 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.333 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.334 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759663 s, 138 MB/s 2024-04-25T08:40:17.343 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.343 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.343 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722166 s, 145 MB/s 2024-04-25T08:40:17.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.353 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728473 s, 144 MB/s 2024-04-25T08:40:17.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.363 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810675 s, 129 MB/s 2024-04-25T08:40:17.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.373 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735272 s, 143 MB/s 2024-04-25T08:40:17.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.382 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733314 s, 143 MB/s 2024-04-25T08:40:17.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.391 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.392 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068404 s, 153 MB/s 2024-04-25T08:40:17.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.401 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736364 s, 142 MB/s 2024-04-25T08:40:17.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.412 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829142 s, 126 MB/s 2024-04-25T08:40:17.421 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.421 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.421 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659465 s, 159 MB/s 2024-04-25T08:40:17.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.431 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075652 s, 139 MB/s 2024-04-25T08:40:17.440 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.440 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.440 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711699 s, 147 MB/s 2024-04-25T08:40:17.450 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.450 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.450 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726221 s, 144 MB/s 2024-04-25T08:40:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738276 s, 142 MB/s 2024-04-25T08:40:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752381 s, 139 MB/s 2024-04-25T08:40:17.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.479 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746524 s, 140 MB/s 2024-04-25T08:40:17.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.489 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708609 s, 148 MB/s 2024-04-25T08:40:17.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:17.489 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:17.489 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:17.489 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T08:40:17.489 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:17.490 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:17.490 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:17.490 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:17.556 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.556 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.556 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679316 s, 154 MB/s 2024-04-25T08:40:17.566 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.567 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.567 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844119 s, 124 MB/s 2024-04-25T08:40:17.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.577 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081979 s, 128 MB/s 2024-04-25T08:40:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073566 s, 143 MB/s 2024-04-25T08:40:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735779 s, 143 MB/s 2024-04-25T08:40:17.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.606 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073001 s, 144 MB/s 2024-04-25T08:40:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.616 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821491 s, 128 MB/s 2024-04-25T08:40:17.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.626 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720985 s, 145 MB/s 2024-04-25T08:40:17.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.636 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807407 s, 130 MB/s 2024-04-25T08:40:17.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.646 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735501 s, 143 MB/s 2024-04-25T08:40:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725989 s, 144 MB/s 2024-04-25T08:40:17.664 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.664 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.664 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692788 s, 151 MB/s 2024-04-25T08:40:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.675 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747748 s, 140 MB/s 2024-04-25T08:40:17.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.685 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820828 s, 128 MB/s 2024-04-25T08:40:17.696 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.696 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.696 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086205 s, 122 MB/s 2024-04-25T08:40:17.706 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.707 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.707 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824468 s, 127 MB/s 2024-04-25T08:40:17.716 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.716 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.716 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763776 s, 137 MB/s 2024-04-25T08:40:17.726 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.726 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.726 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723413 s, 145 MB/s 2024-04-25T08:40:17.736 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.736 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.736 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747535 s, 140 MB/s 2024-04-25T08:40:17.745 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.745 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.745 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721702 s, 145 MB/s 2024-04-25T08:40:17.755 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.755 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.755 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748678 s, 140 MB/s 2024-04-25T08:40:17.764 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.764 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.764 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705229 s, 149 MB/s 2024-04-25T08:40:17.773 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.773 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.773 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686363 s, 153 MB/s 2024-04-25T08:40:17.783 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.784 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768924 s, 136 MB/s 2024-04-25T08:40:17.794 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.794 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.794 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752214 s, 139 MB/s 2024-04-25T08:40:17.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.803 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720426 s, 146 MB/s 2024-04-25T08:40:17.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.814 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842213 s, 125 MB/s 2024-04-25T08:40:17.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.824 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719921 s, 146 MB/s 2024-04-25T08:40:17.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.833 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751654 s, 140 MB/s 2024-04-25T08:40:17.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.843 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723937 s, 145 MB/s 2024-04-25T08:40:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.854 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854175 s, 123 MB/s 2024-04-25T08:40:17.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.863 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742198 s, 141 MB/s 2024-04-25T08:40:17.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:17.864 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:17.864 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:17.864 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T08:40:17.865 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:17.865 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:17.865 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:17.865 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:17.932 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.932 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.932 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715001 s, 147 MB/s 2024-04-25T08:40:17.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.943 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840626 s, 125 MB/s 2024-04-25T08:40:17.953 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.953 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.953 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815792 s, 129 MB/s 2024-04-25T08:40:17.963 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773904 s, 135 MB/s 2024-04-25T08:40:17.973 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.973 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.973 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069633 s, 151 MB/s 2024-04-25T08:40:17.983 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.983 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.983 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734042 s, 143 MB/s 2024-04-25T08:40:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:17.992 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719465 s, 146 MB/s 2024-04-25T08:40:18.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.002 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664402 s, 158 MB/s 2024-04-25T08:40:18.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.012 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773595 s, 136 MB/s 2024-04-25T08:40:18.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.022 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798366 s, 131 MB/s 2024-04-25T08:40:18.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.032 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740808 s, 142 MB/s 2024-04-25T08:40:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.042 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840839 s, 125 MB/s 2024-04-25T08:40:18.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.052 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762085 s, 138 MB/s 2024-04-25T08:40:18.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.062 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078407 s, 134 MB/s 2024-04-25T08:40:18.072 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.072 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.072 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808049 s, 130 MB/s 2024-04-25T08:40:18.083 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.083 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.083 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823174 s, 127 MB/s 2024-04-25T08:40:18.093 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.093 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.093 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070584 s, 149 MB/s 2024-04-25T08:40:18.103 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.104 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082947 s, 126 MB/s 2024-04-25T08:40:18.114 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.114 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.114 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843421 s, 124 MB/s 2024-04-25T08:40:18.124 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.124 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.124 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791666 s, 132 MB/s 2024-04-25T08:40:18.134 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.134 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.134 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730597 s, 144 MB/s 2024-04-25T08:40:18.145 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.145 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.145 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882972 s, 119 MB/s 2024-04-25T08:40:18.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.156 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816925 s, 128 MB/s 2024-04-25T08:40:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.166 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.167 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830471 s, 126 MB/s 2024-04-25T08:40:18.176 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.177 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779018 s, 135 MB/s 2024-04-25T08:40:18.187 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.188 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870908 s, 120 MB/s 2024-04-25T08:40:18.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.198 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784016 s, 134 MB/s 2024-04-25T08:40:18.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.208 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770828 s, 136 MB/s 2024-04-25T08:40:18.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.218 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793317 s, 132 MB/s 2024-04-25T08:40:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.229 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.229 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819975 s, 128 MB/s 2024-04-25T08:40:18.239 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.240 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.240 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082392 s, 127 MB/s 2024-04-25T08:40:18.250 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.250 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.250 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734076 s, 143 MB/s 2024-04-25T08:40:18.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:18.251 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:18.251 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:18.251 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T08:40:18.251 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:18.251 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:18.251 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:18.251 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.352 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772089 s, 136 MB/s 2024-04-25T08:40:18.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.363 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842905 s, 124 MB/s 2024-04-25T08:40:18.372 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.373 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752162 s, 139 MB/s 2024-04-25T08:40:18.383 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.383 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.383 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829432 s, 126 MB/s 2024-04-25T08:40:18.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.393 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074676 s, 140 MB/s 2024-04-25T08:40:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820125 s, 128 MB/s 2024-04-25T08:40:18.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.414 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812488 s, 129 MB/s 2024-04-25T08:40:18.423 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.424 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735938 s, 142 MB/s 2024-04-25T08:40:18.433 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.433 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.433 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765352 s, 137 MB/s 2024-04-25T08:40:18.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.443 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742783 s, 141 MB/s 2024-04-25T08:40:18.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.454 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.454 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822356 s, 128 MB/s 2024-04-25T08:40:18.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.464 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742743 s, 141 MB/s 2024-04-25T08:40:18.474 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.474 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.474 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824318 s, 127 MB/s 2024-04-25T08:40:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.484 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738185 s, 142 MB/s 2024-04-25T08:40:18.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.493 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696439 s, 151 MB/s 2024-04-25T08:40:18.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.503 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822791 s, 127 MB/s 2024-04-25T08:40:18.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.514 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756838 s, 139 MB/s 2024-04-25T08:40:18.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.523 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739482 s, 142 MB/s 2024-04-25T08:40:18.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.533 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746637 s, 140 MB/s 2024-04-25T08:40:18.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.542 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713802 s, 147 MB/s 2024-04-25T08:40:18.552 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.552 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.552 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748105 s, 140 MB/s 2024-04-25T08:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.562 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756324 s, 139 MB/s 2024-04-25T08:40:18.571 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.571 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.571 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677249 s, 155 MB/s 2024-04-25T08:40:18.581 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.581 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.582 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824874 s, 127 MB/s 2024-04-25T08:40:18.591 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.591 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.591 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773447 s, 136 MB/s 2024-04-25T08:40:18.601 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.601 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.601 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758878 s, 138 MB/s 2024-04-25T08:40:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.611 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755068 s, 139 MB/s 2024-04-25T08:40:18.621 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.621 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.621 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810269 s, 129 MB/s 2024-04-25T08:40:18.632 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.632 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.632 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856391 s, 122 MB/s 2024-04-25T08:40:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.641 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708409 s, 148 MB/s 2024-04-25T08:40:18.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.651 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737347 s, 142 MB/s 2024-04-25T08:40:18.662 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.662 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.662 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840987 s, 125 MB/s 2024-04-25T08:40:18.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:18.663 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:18.663 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:18.663 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T08:40:18.663 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:18.663 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:18.663 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:18.664 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:18.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.691 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697592 s, 150 MB/s 2024-04-25T08:40:18.701 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.701 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.701 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747583 s, 140 MB/s 2024-04-25T08:40:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.710 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702867 s, 149 MB/s 2024-04-25T08:40:18.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.721 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821381 s, 128 MB/s 2024-04-25T08:40:18.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.731 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838099 s, 125 MB/s 2024-04-25T08:40:18.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.742 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854609 s, 123 MB/s 2024-04-25T08:40:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736061 s, 142 MB/s 2024-04-25T08:40:18.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.762 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772556 s, 136 MB/s 2024-04-25T08:40:18.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.772 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755911 s, 139 MB/s 2024-04-25T08:40:18.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.782 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809684 s, 130 MB/s 2024-04-25T08:40:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.793 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830648 s, 126 MB/s 2024-04-25T08:40:18.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.804 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.804 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839483 s, 125 MB/s 2024-04-25T08:40:18.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.814 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821576 s, 128 MB/s 2024-04-25T08:40:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775047 s, 135 MB/s 2024-04-25T08:40:18.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.833 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654392 s, 160 MB/s 2024-04-25T08:40:18.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.844 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827623 s, 127 MB/s 2024-04-25T08:40:18.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.853 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750361 s, 140 MB/s 2024-04-25T08:40:18.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.863 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712769 s, 147 MB/s 2024-04-25T08:40:18.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.873 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760769 s, 138 MB/s 2024-04-25T08:40:18.881 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.882 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674188 s, 156 MB/s 2024-04-25T08:40:18.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.891 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749124 s, 140 MB/s 2024-04-25T08:40:18.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.901 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720747 s, 145 MB/s 2024-04-25T08:40:18.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.910 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728284 s, 144 MB/s 2024-04-25T08:40:18.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.920 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772983 s, 136 MB/s 2024-04-25T08:40:18.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.930 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746333 s, 140 MB/s 2024-04-25T08:40:18.939 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.940 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751136 s, 140 MB/s 2024-04-25T08:40:18.949 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.949 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.949 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768308 s, 136 MB/s 2024-04-25T08:40:18.959 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.959 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.959 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725651 s, 145 MB/s 2024-04-25T08:40:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.969 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731639 s, 143 MB/s 2024-04-25T08:40:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811591 s, 129 MB/s 2024-04-25T08:40:18.989 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.989 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.989 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818662 s, 128 MB/s 2024-04-25T08:40:18.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:18.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:18.999 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778979 s, 135 MB/s 2024-04-25T08:40:19.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:19.000 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:19.000 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:19.001 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T08:40:19.001 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:19.001 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:19.001 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:19.001 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:19.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.071 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830736 s, 126 MB/s 2024-04-25T08:40:19.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.080 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757729 s, 138 MB/s 2024-04-25T08:40:19.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.090 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690601 s, 152 MB/s 2024-04-25T08:40:19.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.099 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.099 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722632 s, 145 MB/s 2024-04-25T08:40:19.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.110 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804781 s, 130 MB/s 2024-04-25T08:40:19.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.120 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828417 s, 127 MB/s 2024-04-25T08:40:19.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.130 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747275 s, 140 MB/s 2024-04-25T08:40:19.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.140 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738768 s, 142 MB/s 2024-04-25T08:40:19.149 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.149 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.149 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654805 s, 160 MB/s 2024-04-25T08:40:19.159 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.159 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.160 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802573 s, 131 MB/s 2024-04-25T08:40:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.169 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075416 s, 139 MB/s 2024-04-25T08:40:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.179 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725453 s, 145 MB/s 2024-04-25T08:40:19.189 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.189 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.189 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813376 s, 129 MB/s 2024-04-25T08:40:19.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.198 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675225 s, 155 MB/s 2024-04-25T08:40:19.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.208 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767128 s, 137 MB/s 2024-04-25T08:40:19.217 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.218 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727381 s, 144 MB/s 2024-04-25T08:40:19.226 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.227 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067773 s, 155 MB/s 2024-04-25T08:40:19.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.237 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081821 s, 128 MB/s 2024-04-25T08:40:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.247 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727839 s, 144 MB/s 2024-04-25T08:40:19.256 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.256 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.257 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073739 s, 142 MB/s 2024-04-25T08:40:19.266 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.266 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.266 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728467 s, 144 MB/s 2024-04-25T08:40:19.275 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.276 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.276 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737866 s, 142 MB/s 2024-04-25T08:40:19.284 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.285 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.285 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692431 s, 151 MB/s 2024-04-25T08:40:19.294 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.294 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.294 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728379 s, 144 MB/s 2024-04-25T08:40:19.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.304 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073233 s, 143 MB/s 2024-04-25T08:40:19.315 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.316 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.316 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712769 s, 147 MB/s 2024-04-25T08:40:19.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.344 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741771 s, 141 MB/s 2024-04-25T08:40:19.355 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.355 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.355 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847693 s, 124 MB/s 2024-04-25T08:40:19.364 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.364 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.364 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736133 s, 142 MB/s 2024-04-25T08:40:19.375 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.375 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.375 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846295 s, 124 MB/s 2024-04-25T08:40:19.385 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.385 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.385 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007644 s, 137 MB/s 2024-04-25T08:40:19.396 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.396 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.396 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848357 s, 124 MB/s 2024-04-25T08:40:19.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:19.397 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:19.397 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:19.397 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T08:40:19.397 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:19.397 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:19.397 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:19.398 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:19.465 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.465 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.465 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738093 s, 142 MB/s 2024-04-25T08:40:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.476 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081886 s, 128 MB/s 2024-04-25T08:40:19.486 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.486 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.486 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824427 s, 127 MB/s 2024-04-25T08:40:19.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.496 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738362 s, 142 MB/s 2024-04-25T08:40:19.506 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.506 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.506 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727885 s, 144 MB/s 2024-04-25T08:40:19.516 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.516 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.516 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783327 s, 134 MB/s 2024-04-25T08:40:19.525 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.526 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.526 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736415 s, 142 MB/s 2024-04-25T08:40:19.536 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.536 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.536 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820091 s, 128 MB/s 2024-04-25T08:40:19.545 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.545 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.546 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716562 s, 146 MB/s 2024-04-25T08:40:19.555 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.555 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.555 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749285 s, 140 MB/s 2024-04-25T08:40:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.565 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741272 s, 141 MB/s 2024-04-25T08:40:19.574 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.574 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.574 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709075 s, 148 MB/s 2024-04-25T08:40:19.583 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.583 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.583 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708401 s, 148 MB/s 2024-04-25T08:40:19.593 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.593 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.593 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720809 s, 145 MB/s 2024-04-25T08:40:19.603 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.603 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.603 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762117 s, 138 MB/s 2024-04-25T08:40:19.613 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.613 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.613 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694616 s, 151 MB/s 2024-04-25T08:40:19.622 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.622 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.622 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007257 s, 144 MB/s 2024-04-25T08:40:19.633 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.633 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.633 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821766 s, 128 MB/s 2024-04-25T08:40:19.642 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.642 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.642 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726126 s, 144 MB/s 2024-04-25T08:40:19.653 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.653 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.653 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828272 s, 127 MB/s 2024-04-25T08:40:19.663 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.663 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.663 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818081 s, 128 MB/s 2024-04-25T08:40:19.673 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.674 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804602 s, 130 MB/s 2024-04-25T08:40:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.684 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828366 s, 127 MB/s 2024-04-25T08:40:19.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.694 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737858 s, 142 MB/s 2024-04-25T08:40:19.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.705 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.705 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857018 s, 122 MB/s 2024-04-25T08:40:19.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.715 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799286 s, 131 MB/s 2024-04-25T08:40:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.725 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739966 s, 142 MB/s 2024-04-25T08:40:19.734 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.735 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.735 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740517 s, 142 MB/s 2024-04-25T08:40:19.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.744 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733443 s, 143 MB/s 2024-04-25T08:40:19.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.754 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807191 s, 130 MB/s 2024-04-25T08:40:19.764 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.765 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765813 s, 137 MB/s 2024-04-25T08:40:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.774 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718416 s, 146 MB/s 2024-04-25T08:40:19.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:19.775 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:19.775 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:19.775 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T08:40:19.775 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:19.775 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:19.775 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:19.775 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:19.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.844 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841912 s, 125 MB/s 2024-04-25T08:40:19.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.853 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727834 s, 144 MB/s 2024-04-25T08:40:19.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.864 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839136 s, 125 MB/s 2024-04-25T08:40:19.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.873 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696072 s, 151 MB/s 2024-04-25T08:40:19.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.883 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707075 s, 148 MB/s 2024-04-25T08:40:19.892 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.892 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.892 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678115 s, 155 MB/s 2024-04-25T08:40:19.900 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.900 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.900 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654496 s, 160 MB/s 2024-04-25T08:40:19.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.910 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735398 s, 143 MB/s 2024-04-25T08:40:19.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.920 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742489 s, 141 MB/s 2024-04-25T08:40:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.929 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729555 s, 144 MB/s 2024-04-25T08:40:19.939 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.939 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.939 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076776 s, 137 MB/s 2024-04-25T08:40:19.948 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.948 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.948 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666447 s, 157 MB/s 2024-04-25T08:40:19.957 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.957 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.957 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668048 s, 157 MB/s 2024-04-25T08:40:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.966 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678019 s, 155 MB/s 2024-04-25T08:40:19.976 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.976 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.976 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754949 s, 139 MB/s 2024-04-25T08:40:19.986 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.986 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.986 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764364 s, 137 MB/s 2024-04-25T08:40:19.995 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:19.995 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:19.995 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712669 s, 147 MB/s 2024-04-25T08:40:20.005 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.005 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.005 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789295 s, 133 MB/s 2024-04-25T08:40:20.015 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.015 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.015 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751806 s, 139 MB/s 2024-04-25T08:40:20.025 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.025 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.025 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080652 s, 130 MB/s 2024-04-25T08:40:20.034 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.035 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.035 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068935 s, 152 MB/s 2024-04-25T08:40:20.044 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.045 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.045 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752366 s, 139 MB/s 2024-04-25T08:40:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723817 s, 145 MB/s 2024-04-25T08:40:20.064 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.064 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.064 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758133 s, 138 MB/s 2024-04-25T08:40:20.074 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.074 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.074 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821333 s, 128 MB/s 2024-04-25T08:40:20.084 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.084 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.084 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745683 s, 141 MB/s 2024-04-25T08:40:20.094 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.094 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.094 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753059 s, 139 MB/s 2024-04-25T08:40:20.103 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.103 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.103 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726324 s, 144 MB/s 2024-04-25T08:40:20.113 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.114 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.114 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808481 s, 130 MB/s 2024-04-25T08:40:20.123 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.123 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.123 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071916 s, 146 MB/s 2024-04-25T08:40:20.132 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.133 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753022 s, 139 MB/s 2024-04-25T08:40:20.143 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.143 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.143 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744781 s, 141 MB/s 2024-04-25T08:40:20.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:20.144 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:20.144 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:20.144 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T08:40:20.144 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:20.144 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:20.144 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:20.144 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.212 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733727 s, 143 MB/s 2024-04-25T08:40:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.221 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739536 s, 142 MB/s 2024-04-25T08:40:20.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.232 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823359 s, 127 MB/s 2024-04-25T08:40:20.241 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.241 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.242 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769351 s, 136 MB/s 2024-04-25T08:40:20.251 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.251 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.251 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747787 s, 140 MB/s 2024-04-25T08:40:20.261 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.261 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.261 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755027 s, 139 MB/s 2024-04-25T08:40:20.270 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.271 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.271 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728265 s, 144 MB/s 2024-04-25T08:40:20.280 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.280 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.281 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723098 s, 145 MB/s 2024-04-25T08:40:20.290 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.290 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.290 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739106 s, 142 MB/s 2024-04-25T08:40:20.300 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.300 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.300 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769156 s, 136 MB/s 2024-04-25T08:40:20.311 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.311 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.311 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904202 s, 116 MB/s 2024-04-25T08:40:20.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.344 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075295 s, 139 MB/s 2024-04-25T08:40:20.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.353 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722638 s, 145 MB/s 2024-04-25T08:40:20.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.363 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074221 s, 141 MB/s 2024-04-25T08:40:20.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.373 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817442 s, 128 MB/s 2024-04-25T08:40:20.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.382 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675877 s, 155 MB/s 2024-04-25T08:40:20.392 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.392 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.392 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692418 s, 151 MB/s 2024-04-25T08:40:20.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.401 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689935 s, 152 MB/s 2024-04-25T08:40:20.410 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.411 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.411 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719626 s, 146 MB/s 2024-04-25T08:40:20.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.419 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682272 s, 154 MB/s 2024-04-25T08:40:20.429 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.429 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.429 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723725 s, 145 MB/s 2024-04-25T08:40:20.439 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.439 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.439 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732442 s, 143 MB/s 2024-04-25T08:40:20.448 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.449 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.449 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753761 s, 139 MB/s 2024-04-25T08:40:20.458 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.458 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.459 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772795 s, 136 MB/s 2024-04-25T08:40:20.468 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.468 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.469 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777733 s, 135 MB/s 2024-04-25T08:40:20.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.479 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767884 s, 137 MB/s 2024-04-25T08:40:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.489 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824004 s, 127 MB/s 2024-04-25T08:40:20.498 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.498 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.498 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680345 s, 154 MB/s 2024-04-25T08:40:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.508 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760775 s, 138 MB/s 2024-04-25T08:40:20.517 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.517 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.517 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692648 s, 151 MB/s 2024-04-25T08:40:20.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.528 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840445 s, 125 MB/s 2024-04-25T08:40:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767203 s, 137 MB/s 2024-04-25T08:40:20.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:20.539 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:40:20.539 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:40:20.539 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T08:40:20.539 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:40:20.539 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:40:20.539 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:40:20.540 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:40:20.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.608 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777979 s, 135 MB/s 2024-04-25T08:40:20.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.618 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748329 s, 140 MB/s 2024-04-25T08:40:20.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.627 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.627 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069628 s, 151 MB/s 2024-04-25T08:40:20.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.636 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670789 s, 156 MB/s 2024-04-25T08:40:20.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.647 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.647 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831111 s, 126 MB/s 2024-04-25T08:40:20.656 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.657 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747664 s, 140 MB/s 2024-04-25T08:40:20.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.666 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737639 s, 142 MB/s 2024-04-25T08:40:20.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.676 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764412 s, 137 MB/s 2024-04-25T08:40:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.687 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867839 s, 121 MB/s 2024-04-25T08:40:20.696 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.697 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.697 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726064 s, 144 MB/s 2024-04-25T08:40:20.706 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.706 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.706 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726629 s, 144 MB/s 2024-04-25T08:40:20.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.715 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701235 s, 150 MB/s 2024-04-25T08:40:20.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.725 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738339 s, 142 MB/s 2024-04-25T08:40:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761762 s, 138 MB/s 2024-04-25T08:40:20.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.745 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.745 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759361 s, 138 MB/s 2024-04-25T08:40:20.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.754 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737101 s, 142 MB/s 2024-04-25T08:40:20.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.765 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824251 s, 127 MB/s 2024-04-25T08:40:20.774 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.774 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.774 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697948 s, 150 MB/s 2024-04-25T08:40:20.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.784 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078378 s, 134 MB/s 2024-04-25T08:40:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.795 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834339 s, 126 MB/s 2024-04-25T08:40:20.805 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.805 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.805 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726716 s, 144 MB/s 2024-04-25T08:40:20.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.814 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072508 s, 145 MB/s 2024-04-25T08:40:20.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.824 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752167 s, 139 MB/s 2024-04-25T08:40:20.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.834 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772853 s, 136 MB/s 2024-04-25T08:40:20.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.844 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833216 s, 126 MB/s 2024-04-25T08:40:20.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.854 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704545 s, 149 MB/s 2024-04-25T08:40:20.866 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.866 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.866 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762553 s, 138 MB/s 2024-04-25T08:40:20.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.874 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758875 s, 138 MB/s 2024-04-25T08:40:20.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.883 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681352 s, 154 MB/s 2024-04-25T08:40:20.892 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.892 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.892 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694084 s, 151 MB/s 2024-04-25T08:40:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.902 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744469 s, 141 MB/s 2024-04-25T08:40:20.912 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:40:20.912 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:40:20.912 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783812 s, 134 MB/s 2024-04-25T08:40:20.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:20.913 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-25T08:40:21.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.065+0000 7f51785e2640 1 Processor -- start 2024-04-25T08:40:21.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.065+0000 7f51785e2640 1 -- start start 2024-04-25T08:40:21.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f51785e2640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f51700cf980 0x7f51700cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f51785e2640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f51700ccca0 0x7f517009a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f51785e2640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51700d02a0 con 0x7f517015bd00 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f51785e2640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f517009a160 con 0x7f51700cf980 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f51785e2640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51700cd1f0 con 0x7f51700ccca0 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f517637e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f51700cf980 0x7f51700cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f5175b7d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f517015bd00 0x7f51700ae360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60408/0 (socket says 172.21.15.38:60408) 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f5175b7d640 1 -- 172.21.15.38:0/1629611058 learned_addr learned my addr 172.21.15.38:0/1629611058 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f5176b7f640 1 --2- 172.21.15.38:0/1629611058 >> v2:172.21.15.107:3300/0 conn(0x7f51700ccca0 0x7f517009a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f517637e640 1 -- 172.21.15.38:0/1629611058 >> v2:172.21.15.107:3300/0 conn(0x7f51700ccca0 msgr2=0x7f517009a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f517637e640 1 --2- 172.21.15.38:0/1629611058 >> v2:172.21.15.107:3300/0 conn(0x7f51700ccca0 0x7f517009a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f517637e640 1 -- 172.21.15.38:0/1629611058 >> v1:172.21.15.38:6789/0 conn(0x7f517015bd00 legacy=0x7f51700ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f517637e640 1 -- 172.21.15.38:0/1629611058 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5170099b10 con 0x7f51700cf980 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.066+0000 7f517537c640 1 -- 172.21.15.38:0/1629611058 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114735653 0 0) 0x7f51700d02a0 con 0x7f517015bd00 2024-04-25T08:40:21.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.067+0000 7f517637e640 1 --2- 172.21.15.38:0/1629611058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51700cf980 0x7f51700cfd60 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f51700d25c0 tx=0x7f516c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50929a9a4fb2218 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.067+0000 7f517537c640 1 -- 172.21.15.38:0/1629611058 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c037070 con 0x7f51700cf980 2024-04-25T08:40:21.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.067+0000 7f517537c640 1 -- 172.21.15.38:0/1629611058 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c032440 con 0x7f51700cf980 2024-04-25T08:40:21.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.067+0000 7f517537c640 1 -- 172.21.15.38:0/1629611058 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c032810 con 0x7f51700cf980 2024-04-25T08:40:21.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.067+0000 7f51785e2640 1 -- 172.21.15.38:0/1629611058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51700cf980 msgr2=0x7f51700cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.067+0000 7f51785e2640 1 --2- 172.21.15.38:0/1629611058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51700cf980 0x7f51700cfd60 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f51700d25c0 tx=0x7f516c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.068+0000 7f51785e2640 1 -- 172.21.15.38:0/1629611058 shutdown_connections 2024-04-25T08:40:21.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.068+0000 7f51785e2640 1 --2- 172.21.15.38:0/1629611058 >> v2:172.21.15.107:3300/0 conn(0x7f51700ccca0 0x7f517009a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.068+0000 7f51785e2640 1 --2- 172.21.15.38:0/1629611058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51700cf980 0x7f51700cfd60 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.068+0000 7f51785e2640 1 -- 172.21.15.38:0/1629611058 >> 172.21.15.38:0/1629611058 conn(0x7f51700b31b0 msgr2=0x7f51700b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.068+0000 7f51785e2640 1 -- 172.21.15.38:0/1629611058 shutdown_connections 2024-04-25T08:40:21.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.068+0000 7f51785e2640 1 -- 172.21.15.38:0/1629611058 wait complete. 2024-04-25T08:40:21.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.069+0000 7f51785e2640 1 Processor -- start 2024-04-25T08:40:21.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.069+0000 7f51785e2640 1 -- start start 2024-04-25T08:40:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f51785e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51700ccca0 0x7f51700b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f51785e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51700cf980 0x7f51700b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f51785e2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f517015bd00 0x7f517015cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f51785e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51700aea70 con 0x7f51700cf980 2024-04-25T08:40:21.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f51785e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51700cd4d0 con 0x7f51700ccca0 2024-04-25T08:40:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f51785e2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51700c6740 con 0x7f517015bd00 2024-04-25T08:40:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5175b7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51700cf980 0x7f51700b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5175b7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51700cf980 0x7f51700b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35520/0 (socket says 172.21.15.38:35520) 2024-04-25T08:40:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5175b7d640 1 -- 172.21.15.38:0/522787124 learned_addr learned my addr 172.21.15.38:0/522787124 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f517637e640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51700ccca0 0x7f51700b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5176b7f640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f517015bd00 0x7f517015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5175b7d640 1 -- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f517015bd00 msgr2=0x7f517015cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5175b7d640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f517015bd00 0x7f517015cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5175b7d640 1 -- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51700ccca0 msgr2=0x7f51700b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5175b7d640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51700ccca0 0x7f51700b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5175b7d640 1 -- 172.21.15.38:0/522787124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51700c6a20 con 0x7f51700cf980 2024-04-25T08:40:21.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5175b7d640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51700cf980 0x7f51700b18b0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f516000c410 tx=0x7f516000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5166ffd640 1 -- 172.21.15.38:0/522787124 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51600074e0 con 0x7f51700cf980 2024-04-25T08:40:21.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5166ffd640 1 -- 172.21.15.38:0/522787124 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51600047e0 con 0x7f51700cf980 2024-04-25T08:40:21.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f51785e2640 1 -- 172.21.15.38:0/522787124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51700c6d00 con 0x7f51700cf980 2024-04-25T08:40:21.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f5166ffd640 1 -- 172.21.15.38:0/522787124 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160004bd0 con 0x7f51700cf980 2024-04-25T08:40:21.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.070+0000 7f51785e2640 1 -- 172.21.15.38:0/522787124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51700b6f70 con 0x7f51700cf980 2024-04-25T08:40:21.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.072+0000 7f5166ffd640 1 -- 172.21.15.38:0/522787124 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5160011070 con 0x7f51700cf980 2024-04-25T08:40:21.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.072+0000 7f5166ffd640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f515406b4d0 0x7f515406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.072+0000 7f5166ffd640 1 -- 172.21.15.38:0/522787124 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5160089dd0 con 0x7f51700cf980 2024-04-25T08:40:21.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.073+0000 7f517637e640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f515406b4d0 0x7f515406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.073+0000 7f517637e640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f515406b4d0 0x7f515406d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f516c034000 tx=0x7f516c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.074+0000 7f517b3bd740 1 Processor -- start 2024-04-25T08:40:21.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.074+0000 7f517b3bd740 1 -- start start 2024-04-25T08:40:21.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.074+0000 7f517b3bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa45e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517b3bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa45c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517b3bd740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa486cc0 0x5623aa487140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517b3bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623aa3c0000 con 0x5623aa45c200 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517b3bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623aa368f20 con 0x5623aa4988d0 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517b3bd740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623aa3bc0a0 con 0x5623aa486cc0 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517637e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa45e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517637e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa45e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35530/0 (socket says 172.21.15.38:35530) 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517637e640 1 -- 172.21.15.38:0/2884570733 learned_addr learned my addr 172.21.15.38:0/2884570733 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f5175b7d640 1 --2- 172.21.15.38:0/2884570733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa45c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f5176b7f640 1 --2- 172.21.15.38:0/2884570733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa486cc0 0x5623aa487140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517637e640 1 -- 172.21.15.38:0/2884570733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa486cc0 msgr2=0x5623aa487140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517637e640 1 --2- 172.21.15.38:0/2884570733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa486cc0 0x5623aa487140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517637e640 1 -- 172.21.15.38:0/2884570733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 msgr2=0x5623aa45c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517637e640 1 --2- 172.21.15.38:0/2884570733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa45c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517637e640 1 -- 172.21.15.38:0/2884570733 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623aa1a0ba0 con 0x5623aa45c200 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f5175b7d640 1 --2- 172.21.15.38:0/2884570733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa45c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f5176b7f640 1 --2- 172.21.15.38:0/2884570733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa486cc0 0x5623aa487140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f517637e640 1 --2- 172.21.15.38:0/2884570733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa45e360 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f516c009560 tx=0x7f516c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53fa601f65cd047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f5153fff640 1 -- 172.21.15.38:0/2884570733 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c04c070 con 0x5623aa45c200 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f5153fff640 1 -- 172.21.15.38:0/2884570733 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c032ce0 con 0x5623aa45c200 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.075+0000 7f5153fff640 1 -- 172.21.15.38:0/2884570733 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c037040 con 0x5623aa45c200 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.076+0000 7f517b3bd740 1 -- 172.21.15.38:0/2884570733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 msgr2=0x5623aa45e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.076+0000 7f517b3bd740 1 --2- 172.21.15.38:0/2884570733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa45e360 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f516c009560 tx=0x7f516c02c000 comp rx=0 tx=0).stop 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.076+0000 7f517b3bd740 1 -- 172.21.15.38:0/2884570733 shutdown_connections 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.076+0000 7f517b3bd740 1 --2- 172.21.15.38:0/2884570733 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa486cc0 0x5623aa487140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.076+0000 7f517b3bd740 1 --2- 172.21.15.38:0/2884570733 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa45c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.076+0000 7f517b3bd740 1 --2- 172.21.15.38:0/2884570733 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa45e360 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.076+0000 7f517b3bd740 1 -- 172.21.15.38:0/2884570733 >> 172.21.15.38:0/2884570733 conn(0x5623aa424050 msgr2=0x5623aa4a1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.076+0000 7f517b3bd740 1 -- 172.21.15.38:0/2884570733 shutdown_connections 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.076+0000 7f517b3bd740 1 -- 172.21.15.38:0/2884570733 wait complete. 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.077+0000 7f517b3bd740 1 -- start start 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.077+0000 7f517b3bd740 1 Processor -- start 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517b3bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa49c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517b3bd740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa486cc0 0x5623aa4a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517b3bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa471c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517b3bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623aa35e610 con 0x5623aa45c200 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517b3bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623aa484810 con 0x5623aa4988d0 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517b3bd740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623aa473d80 con 0x5623aa486cc0 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517b3bd740 10 client.0 _ll_register_callbacks cb 0x5623aa376f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517637e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa49c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517637e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa49c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35540/0 (socket says 172.21.15.38:35540) 2024-04-25T08:40:21.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517637e640 1 -- 172.21.15.38:0/3942806188 learned_addr learned my addr 172.21.15.38:0/3942806188 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f5175b7d640 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa486cc0 0x5623aa4a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f5176b7f640 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa471c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517637e640 1 -- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa486cc0 msgr2=0x5623aa4a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517637e640 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa486cc0 0x5623aa4a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517637e640 1 -- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 msgr2=0x5623aa471c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517637e640 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa471c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517637e640 1 -- 172.21.15.38:0/3942806188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623aa094490 con 0x5623aa45c200 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f5176b7f640 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa471c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517637e640 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa49c5d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f516c02a1e0 tx=0x7f516c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fc275601062534 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f51517fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f51517fa640 1 -- 172.21.15.38:0/3942806188 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c037040 con 0x5623aa45c200 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f51517fa640 1 -- 172.21.15.38:0/3942806188 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f516c05bc80 con 0x5623aa45c200 2024-04-25T08:40:21.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517b3bd740 1 -- 172.21.15.38:0/3942806188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f516c032ce0 con 0x5623aa45c200 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f51517fa640 1 -- 172.21.15.38:0/3942806188 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f516c04c040 con 0x5623aa45c200 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517b3bd740 10 client.9054 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.078+0000 7f517b3bd740 1 -- 172.21.15.38:0/3942806188 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x5623aa201330 con 0x5623aa45c200 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.079+0000 7f51517fa640 1 -- 172.21.15.38:0/3942806188 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c032ce0 con 0x5623aa45c200 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.079+0000 7f51517fa640 1 -- 172.21.15.38:0/3942806188 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f516c058020 con 0x5623aa45c200 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.079+0000 7f517b3bd740 10 client.9054 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.079+0000 7f517b3bd740 10 client.9054 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.079+0000 7f517b3bd740 20 client.9054 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.079+0000 7f517b3bd740 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5623aa4a3f40 0x5623aa4b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.079+0000 7f517b3bd740 4 client.9054 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.079+0000 7f517b3bd740 1 -- 172.21.15.38:0/3942806188 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623aa3cd540 con 0x5623aa4a3f40 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.079+0000 7f5175b7d640 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5623aa4a3f40 0x5623aa4b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.079+0000 7f51517fa640 10 client.9054 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:21.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.079+0000 7f5175b7d640 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5623aa4a3f40 0x5623aa4b55b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e615c329a7d709 server_cookie=4d0acf214d1f59cf in_seq=0 out_seq=0 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f51517fa640 1 -- 172.21.15.38:0/3942806188 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5623aa3cd540 con 0x5623aa4a3f40 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f51517fa640 10 client.9054 handle_command_reply: tid=0 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f517b3bd740 1 client.9054 shutdown 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f517b3bd740 1 -- 172.21.15.38:0/3942806188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 msgr2=0x5623aa49c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f517b3bd740 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa49c5d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f516c02a1e0 tx=0x7f516c004730 comp rx=0 tx=0).stop 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f517b3bd740 1 -- 172.21.15.38:0/3942806188 shutdown_connections 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f517b3bd740 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5623aa4a3f40 0x5623aa4b55b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f517b3bd740 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa471c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f51517fa640 0 client.9054 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f517b3bd740 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa486cc0 0x5623aa4a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f517b3bd740 1 --2- 172.21.15.38:0/3942806188 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa49c5d0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f517b3bd740 1 -- 172.21.15.38:0/3942806188 >> 172.21.15.38:0/3942806188 conn(0x5623aa424050 msgr2=0x5623aa484050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f517b3bd740 1 -- 172.21.15.38:0/3942806188 shutdown_connections 2024-04-25T08:40:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f517b3bd740 1 -- 172.21.15.38:0/3942806188 wait complete. 2024-04-25T08:40:21.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.080+0000 7f517b3bd740 20 client.9054 trim_cache size 0 max 16384 2024-04-25T08:40:21.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.096+0000 7f517b3bd740 1 Processor -- start 2024-04-25T08:40:21.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.096+0000 7f517b3bd740 1 -- start start 2024-04-25T08:40:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.096+0000 7f517b3bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa4a3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517b3bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa486cc0 0x5623aa4a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517b3bd740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa4988d0 0x5623aa4bbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517b3bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51701fe100 con 0x5623aa45c200 2024-04-25T08:40:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517b3bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51701fd3e0 con 0x5623aa486cc0 2024-04-25T08:40:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517b3bd740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51700c55a0 con 0x5623aa4988d0 2024-04-25T08:40:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517637e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa4a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517637e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa4a3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35542/0 (socket says 172.21.15.38:35542) 2024-04-25T08:40:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517637e640 1 -- 172.21.15.38:0/2032863001 learned_addr learned my addr 172.21.15.38:0/2032863001 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f5175b7d640 1 --2- 172.21.15.38:0/2032863001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa486cc0 0x5623aa4a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f5176b7f640 1 --2- 172.21.15.38:0/2032863001 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa4988d0 0x5623aa4bbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517637e640 1 -- 172.21.15.38:0/2032863001 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa4988d0 msgr2=0x5623aa4bbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517637e640 1 --2- 172.21.15.38:0/2032863001 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa4988d0 0x5623aa4bbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517637e640 1 -- 172.21.15.38:0/2032863001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa486cc0 msgr2=0x5623aa4a37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517637e640 1 --2- 172.21.15.38:0/2032863001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa486cc0 0x5623aa4a37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517637e640 1 -- 172.21.15.38:0/2032863001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623aa06d610 con 0x5623aa45c200 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f517637e640 1 --2- 172.21.15.38:0/2032863001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa4a3290 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f516c02a870 tx=0x7f516c054000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66dd8d4c1b5a6dba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f51537fe640 1 -- 172.21.15.38:0/2032863001 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c068070 con 0x5623aa45c200 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f51537fe640 1 -- 172.21.15.38:0/2032863001 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c051ab0 con 0x5623aa45c200 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.097+0000 7f51537fe640 1 -- 172.21.15.38:0/2032863001 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c037070 con 0x5623aa45c200 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.098+0000 7f517b3bd740 1 -- 172.21.15.38:0/2032863001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 msgr2=0x5623aa4a3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.098+0000 7f517b3bd740 1 --2- 172.21.15.38:0/2032863001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa4a3290 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f516c02a870 tx=0x7f516c054000 comp rx=0 tx=0).stop 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.098+0000 7f517b3bd740 1 -- 172.21.15.38:0/2032863001 shutdown_connections 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.098+0000 7f517b3bd740 1 --2- 172.21.15.38:0/2032863001 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa4988d0 0x5623aa4bbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.098+0000 7f517b3bd740 1 --2- 172.21.15.38:0/2032863001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa486cc0 0x5623aa4a37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.098+0000 7f517b3bd740 1 --2- 172.21.15.38:0/2032863001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa45c200 0x5623aa4a3290 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.098+0000 7f517b3bd740 1 -- 172.21.15.38:0/2032863001 >> 172.21.15.38:0/2032863001 conn(0x5623aa424050 msgr2=0x5623aa489aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.098+0000 7f517b3bd740 1 -- 172.21.15.38:0/2032863001 shutdown_connections 2024-04-25T08:40:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.098+0000 7f517b3bd740 1 -- 172.21.15.38:0/2032863001 wait complete. 2024-04-25T08:40:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.098+0000 7f517b3bd740 1 -- start start 2024-04-25T08:40:21.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.098+0000 7f517b3bd740 1 Processor -- start 2024-04-25T08:40:21.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.099+0000 7f517b3bd740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa45c200 0x5623aa45a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.099+0000 7f517b3bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa486cc0 0x5623aa45af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.099+0000 7f517b3bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa481f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.099+0000 7f517b3bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51700c5ba0 con 0x5623aa486cc0 2024-04-25T08:40:21.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.099+0000 7f517b3bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623aa3cd540 con 0x5623aa4988d0 2024-04-25T08:40:21.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.099+0000 7f517b3bd740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623aa5112f0 con 0x5623aa45c200 2024-04-25T08:40:21.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.099+0000 7f517b3bd740 10 client.0 _ll_register_callbacks cb 0x7f516c040050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:21.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.099+0000 7f5175b7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa486cc0 0x5623aa45af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f5175b7d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa486cc0 0x5623aa45af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35546/0 (socket says 172.21.15.38:35546) 2024-04-25T08:40:21.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f5175b7d640 1 -- 172.21.15.38:0/580878540 learned_addr learned my addr 172.21.15.38:0/580878540 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f517637e640 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa45c200 0x5623aa45a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f5176b7f640 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa481f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f5175b7d640 1 -- 172.21.15.38:0/580878540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa45c200 msgr2=0x5623aa45a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f5175b7d640 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa45c200 0x5623aa45a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f5175b7d640 1 -- 172.21.15.38:0/580878540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 msgr2=0x5623aa481f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f5175b7d640 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa481f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f5175b7d640 1 -- 172.21.15.38:0/580878540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623aa4bbe60 con 0x5623aa486cc0 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f5175b7d640 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa486cc0 0x5623aa45af00 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f5160088200 tx=0x7f5160060cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d584977c938c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f51517fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f51517fa640 1 -- 172.21.15.38:0/580878540 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5160054b70 con 0x5623aa486cc0 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f51517fa640 1 -- 172.21.15.38:0/580878540 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f516005c0c0 con 0x5623aa486cc0 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f517b3bd740 1 -- 172.21.15.38:0/580878540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f516c051ab0 con 0x5623aa486cc0 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f51517fa640 1 -- 172.21.15.38:0/580878540 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5160054d10 con 0x5623aa486cc0 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f517b3bd740 10 client.9060 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:21.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f517b3bd740 1 -- 172.21.15.38:0/580878540 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x5623aa511030 con 0x5623aa486cc0 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.100+0000 7f51517fa640 1 -- 172.21.15.38:0/580878540 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c051ab0 con 0x5623aa486cc0 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.101+0000 7f51517fa640 1 -- 172.21.15.38:0/580878540 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5160081b60 con 0x5623aa486cc0 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.101+0000 7f517b3bd740 10 client.9060 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.101+0000 7f517b3bd740 10 client.9060 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.101+0000 7f517b3bd740 20 client.9060 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.101+0000 7f517b3bd740 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5623aa483500 0x5623aa4824c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.101+0000 7f517b3bd740 4 client.9060 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.101+0000 7f517b3bd740 1 -- 172.21.15.38:0/580878540 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5623aa3c7890 con 0x5623aa483500 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.101+0000 7f517637e640 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5623aa483500 0x5623aa4824c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.101+0000 7f51517fa640 10 client.9060 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.101+0000 7f517637e640 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5623aa483500 0x5623aa4824c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d761b7a744d640 server_cookie=adec97ea5823a8e1 in_seq=0 out_seq=0 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.103+0000 7f51517fa640 1 -- 172.21.15.38:0/580878540 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5623aa3c7890 con 0x5623aa483500 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.103+0000 7f51517fa640 10 client.9060 handle_command_reply: tid=0 2024-04-25T08:40:21.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.103+0000 7f517b3bd740 1 client.9060 shutdown 2024-04-25T08:40:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f517b3bd740 1 -- 172.21.15.38:0/580878540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa486cc0 msgr2=0x5623aa45af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f517b3bd740 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa486cc0 0x5623aa45af00 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f5160088200 tx=0x7f5160060cc0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f517b3bd740 1 -- 172.21.15.38:0/580878540 shutdown_connections 2024-04-25T08:40:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f517b3bd740 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5623aa483500 0x5623aa4824c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f517b3bd740 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5623aa4988d0 0x5623aa481f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f517b3bd740 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5623aa486cc0 0x5623aa45af00 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f517b3bd740 1 --2- 172.21.15.38:0/580878540 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5623aa45c200 0x5623aa45a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f51517fa640 0 client.9060 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f517b3bd740 1 -- 172.21.15.38:0/580878540 >> 172.21.15.38:0/580878540 conn(0x5623aa424050 msgr2=0x5623aa484250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f517b3bd740 1 -- 172.21.15.38:0/580878540 shutdown_connections 2024-04-25T08:40:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f517b3bd740 1 -- 172.21.15.38:0/580878540 wait complete. 2024-04-25T08:40:21.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f517b3bd740 20 client.9060 trim_cache size 0 max 16384 2024-04-25T08:40:21.102 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:40:21.102 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:40:21.102 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "68c7ac46-3acc-4dde-9256-8f7c0c44785b", 2024-04-25T08:40:21.102 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:40:21.102 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:40:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f51785e2640 1 -- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f515406b4d0 msgr2=0x7f515406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f51785e2640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f515406b4d0 0x7f515406d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f516c034000 tx=0x7f516c02e000 comp rx=0 tx=0).stop 2024-04-25T08:40:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f51785e2640 1 -- 172.21.15.38:0/522787124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51700cf980 msgr2=0x7f51700b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f51785e2640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51700cf980 0x7f51700b18b0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f516000c410 tx=0x7f516000c8d0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f51785e2640 1 -- 172.21.15.38:0/522787124 shutdown_connections 2024-04-25T08:40:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f51785e2640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f515406b4d0 0x7f515406d990 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f51785e2640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f517015bd00 0x7f517015cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f51785e2640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f51700cf980 0x7f51700b18b0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f51785e2640 1 --2- 172.21.15.38:0/522787124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f51700ccca0 0x7f51700b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.104+0000 7f51785e2640 1 -- 172.21.15.38:0/522787124 >> 172.21.15.38:0/522787124 conn(0x7f51700b31b0 msgr2=0x7f5170073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.105+0000 7f51785e2640 1 -- 172.21.15.38:0/522787124 shutdown_connections 2024-04-25T08:40:21.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.105+0000 7f51785e2640 1 -- 172.21.15.38:0/522787124 wait complete. 2024-04-25T08:40:21.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:21.108 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-25T08:40:21.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.231+0000 7f7ad984d640 1 Processor -- start 2024-04-25T08:40:21.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.231+0000 7f7ad984d640 1 -- start start 2024-04-25T08:40:21.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad984d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7ad40d0320 0x7f7ad40d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad984d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7ad40b6980 0x7f7ad40b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad984d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad40d2060 con 0x7f7ad40d0700 2024-04-25T08:40:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad984d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad40d2330 con 0x7f7ad40d0320 2024-04-25T08:40:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad984d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad40d2610 con 0x7f7ad40b6980 2024-04-25T08:40:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad2ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7ad40d0320 0x7f7ad40d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad27fc640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7ad40d0700 0x7f7ad40d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60420/0 (socket says 172.21.15.38:60420) 2024-04-25T08:40:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad27fc640 1 -- 172.21.15.38:0/3866599136 learned_addr learned my addr 172.21.15.38:0/3866599136 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad37fe640 1 --2- 172.21.15.38:0/3866599136 >> v2:172.21.15.107:3300/0 conn(0x7f7ad40b6980 0x7f7ad40b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad2ffd640 1 -- 172.21.15.38:0/3866599136 >> v2:172.21.15.107:3300/0 conn(0x7f7ad40b6980 msgr2=0x7f7ad40b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/3866599136 >> v2:172.21.15.107:3300/0 conn(0x7f7ad40b6980 0x7f7ad40b6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad2ffd640 1 -- 172.21.15.38:0/3866599136 >> v1:172.21.15.38:6789/0 conn(0x7f7ad40d0700 legacy=0x7f7ad40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.232+0000 7f7ad2ffd640 1 -- 172.21.15.38:0/3866599136 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad40d28f0 con 0x7f7ad40d0320 2024-04-25T08:40:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.233+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/3866599136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad40d0320 0x7f7ad40d33d0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4067590 tx=0x7f7abc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93dab07b258857c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.233+0000 7f7ad1ffb640 1 -- 172.21.15.38:0/3866599136 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc037070 con 0x7f7ad40d0320 2024-04-25T08:40:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.233+0000 7f7ad1ffb640 1 -- 172.21.15.38:0/3866599136 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7abc032440 con 0x7f7ad40d0320 2024-04-25T08:40:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.233+0000 7f7ad1ffb640 1 -- 172.21.15.38:0/3866599136 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc032810 con 0x7f7ad40d0320 2024-04-25T08:40:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.234+0000 7f7ad984d640 1 -- 172.21.15.38:0/3866599136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad40d0320 msgr2=0x7f7ad40d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.234+0000 7f7ad984d640 1 --2- 172.21.15.38:0/3866599136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad40d0320 0x7f7ad40d33d0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4067590 tx=0x7f7abc02a230 comp rx=0 tx=0).stop 2024-04-25T08:40:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.234+0000 7f7ad984d640 1 -- 172.21.15.38:0/3866599136 shutdown_connections 2024-04-25T08:40:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.234+0000 7f7ad984d640 1 --2- 172.21.15.38:0/3866599136 >> v2:172.21.15.107:3300/0 conn(0x7f7ad40b6980 0x7f7ad40b6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.234+0000 7f7ad984d640 1 --2- 172.21.15.38:0/3866599136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad40d0320 0x7f7ad40d33d0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.234+0000 7f7ad984d640 1 -- 172.21.15.38:0/3866599136 >> 172.21.15.38:0/3866599136 conn(0x7f7ad40668f0 msgr2=0x7f7ad4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.234+0000 7f7ad984d640 1 -- 172.21.15.38:0/3866599136 shutdown_connections 2024-04-25T08:40:21.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.234+0000 7f7ad984d640 1 -- 172.21.15.38:0/3866599136 wait complete. 2024-04-25T08:40:21.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.235+0000 7f7ad984d640 1 Processor -- start 2024-04-25T08:40:21.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.235+0000 7f7ad984d640 1 -- start start 2024-04-25T08:40:21.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad984d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ad40b6980 0x7f7ad41f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad984d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad40d0320 0x7f7ad40ac240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad984d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad40d0700 0x7f7ad40b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad984d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40d0b40 con 0x7f7ad40d0320 2024-04-25T08:40:21.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad984d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40d2eb0 con 0x7f7ad40d0700 2024-04-25T08:40:21.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad984d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40ab820 con 0x7f7ad40b6980 2024-04-25T08:40:21.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad40d0320 0x7f7ad40ac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad40d0320 0x7f7ad40ac240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35570/0 (socket says 172.21.15.38:35570) 2024-04-25T08:40:21.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad27fc640 1 -- 172.21.15.38:0/295557580 learned_addr learned my addr 172.21.15.38:0/295557580 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad37fe640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad40d0700 0x7f7ad40b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ad40b6980 0x7f7ad41f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad27fc640 1 -- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ad40b6980 msgr2=0x7f7ad41f3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad27fc640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ad40b6980 0x7f7ad41f3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad27fc640 1 -- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad40d0700 msgr2=0x7f7ad40b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad27fc640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad40d0700 0x7f7ad40b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad27fc640 1 -- 172.21.15.38:0/295557580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad40b5d80 con 0x7f7ad40d0320 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.236+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ad40b6980 0x7f7ad41f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.237+0000 7f7ad27fc640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad40d0320 0x7f7ad40ac240 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400cbc0 tx=0x7f7ac400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.237+0000 7f7acbfff640 1 -- 172.21.15.38:0/295557580 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4017070 con 0x7f7ad40d0320 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.237+0000 7f7acbfff640 1 -- 172.21.15.38:0/295557580 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac400aca0 con 0x7f7ad40d0320 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.237+0000 7f7ad984d640 1 -- 172.21.15.38:0/295557580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad40aa780 con 0x7f7ad40d0320 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.237+0000 7f7acbfff640 1 -- 172.21.15.38:0/295557580 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4012600 con 0x7f7ad40d0320 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.237+0000 7f7ad984d640 1 -- 172.21.15.38:0/295557580 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad40aad40 con 0x7f7ad40d0320 2024-04-25T08:40:21.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.238+0000 7f7acbfff640 1 -- 172.21.15.38:0/295557580 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ac401a070 con 0x7f7ad40d0320 2024-04-25T08:40:21.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.239+0000 7f7acbfff640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ab406b4d0 0x7f7ab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.239+0000 7f7acbfff640 1 -- 172.21.15.38:0/295557580 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7ac408a000 con 0x7f7ad40d0320 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.239+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ab406b4d0 0x7f7ab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.239+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ab406b4d0 0x7f7ab406d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7abc034000 tx=0x7f7abc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.240+0000 7f7adc62d740 1 Processor -- start 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.240+0000 7f7adc62d740 1 -- start start 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.240+0000 7f7adc62d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f3253ca0 0x55d0f3255e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.240+0000 7f7adc62d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3290370 0x55d0f3254080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.240+0000 7f7adc62d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f327e760 0x55d0f327ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.240+0000 7f7adc62d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f31b6b90 con 0x55d0f327e760 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.240+0000 7f7adc62d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f315fe40 con 0x55d0f3290370 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.240+0000 7f7adc62d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0f31b2c30 con 0x55d0f3253ca0 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ad37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f327e760 0x55d0f327ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ad37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f327e760 0x55d0f327ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35578/0 (socket says 172.21.15.38:35578) 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ad37fe640 1 -- 172.21.15.38:0/3258401843 learned_addr learned my addr 172.21.15.38:0/3258401843 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/3258401843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f3253ca0 0x55d0f3255e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ad27fc640 1 --2- 172.21.15.38:0/3258401843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3290370 0x55d0f3254080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ad37fe640 1 -- 172.21.15.38:0/3258401843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f3253ca0 msgr2=0x55d0f3255e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ad37fe640 1 --2- 172.21.15.38:0/3258401843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f3253ca0 0x55d0f3255e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ad37fe640 1 -- 172.21.15.38:0/3258401843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3290370 msgr2=0x55d0f3254080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ad37fe640 1 --2- 172.21.15.38:0/3258401843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3290370 0x55d0f3254080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ad37fe640 1 -- 172.21.15.38:0/3258401843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0f2f957d0 con 0x55d0f327e760 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ad37fe640 1 --2- 172.21.15.38:0/3258401843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f327e760 0x55d0f327ebe0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00b8b0 tx=0x7f7acc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31fb5f1b1455055f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ac97fa640 1 -- 172.21.15.38:0/3258401843 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc018020 con 0x55d0f327e760 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ac97fa640 1 -- 172.21.15.38:0/3258401843 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc00f960 con 0x55d0f327e760 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.241+0000 7f7ac97fa640 1 -- 172.21.15.38:0/3258401843 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc00fd30 con 0x55d0f327e760 2024-04-25T08:40:21.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.242+0000 7f7adc62d740 1 -- 172.21.15.38:0/3258401843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f327e760 msgr2=0x55d0f327ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.242+0000 7f7adc62d740 1 --2- 172.21.15.38:0/3258401843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f327e760 0x55d0f327ebe0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00b8b0 tx=0x7f7acc00bd70 comp rx=0 tx=0).stop 2024-04-25T08:40:21.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.242+0000 7f7adc62d740 1 -- 172.21.15.38:0/3258401843 shutdown_connections 2024-04-25T08:40:21.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.242+0000 7f7adc62d740 1 --2- 172.21.15.38:0/3258401843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f327e760 0x55d0f327ebe0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.242+0000 7f7adc62d740 1 --2- 172.21.15.38:0/3258401843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3290370 0x55d0f3254080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.242+0000 7f7adc62d740 1 --2- 172.21.15.38:0/3258401843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f3253ca0 0x55d0f3255e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.242+0000 7f7adc62d740 1 -- 172.21.15.38:0/3258401843 >> 172.21.15.38:0/3258401843 conn(0x55d0f321b870 msgr2=0x55d0f3299760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.242+0000 7f7adc62d740 1 -- 172.21.15.38:0/3258401843 shutdown_connections 2024-04-25T08:40:21.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.242+0000 7f7adc62d740 1 -- 172.21.15.38:0/3258401843 wait complete. 2024-04-25T08:40:21.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.243+0000 7f7adc62d740 1 -- start start 2024-04-25T08:40:21.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.243+0000 7f7adc62d740 1 Processor -- start 2024-04-25T08:40:21.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.243+0000 7f7adc62d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3253ca0 0x55d0f327b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.243+0000 7f7adc62d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f327e760 0x55d0f3280ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.243+0000 7f7adc62d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3290370 0x55d0f3294d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.243+0000 7f7adc62d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f31733e0 con 0x55d0f3253ca0 2024-04-25T08:40:21.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.243+0000 7f7adc62d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f327c2b0 con 0x55d0f3290370 2024-04-25T08:40:21.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.243+0000 7f7adc62d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0f32aa1a0 con 0x55d0f327e760 2024-04-25T08:40:21.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.243+0000 7f7adc62d740 10 client.0 _ll_register_callbacks cb 0x55d0f3172db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:21.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.243+0000 7f7ad2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3253ca0 0x55d0f327b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.243+0000 7f7ad2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3253ca0 0x55d0f327b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35584/0 (socket says 172.21.15.38:35584) 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.243+0000 7f7ad2ffd640 1 -- 172.21.15.38:0/1891125409 learned_addr learned my addr 172.21.15.38:0/1891125409 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ad27fc640 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f327e760 0x55d0f3280ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ad37fe640 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3290370 0x55d0f3294d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ad2ffd640 1 -- 172.21.15.38:0/1891125409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f327e760 msgr2=0x55d0f3280ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f327e760 0x55d0f3280ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ad2ffd640 1 -- 172.21.15.38:0/1891125409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3290370 msgr2=0x55d0f3294d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3290370 0x55d0f3294d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ad2ffd640 1 -- 172.21.15.38:0/1891125409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0f2e8b0f0 con 0x55d0f3253ca0 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3253ca0 0x55d0f327b4f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009890 tx=0x7f7abc02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c426c0ed87f0a6d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ab27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ab27fc640 1 -- 172.21.15.38:0/1891125409 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc04e060 con 0x55d0f3253ca0 2024-04-25T08:40:21.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ab27fc640 1 -- 172.21.15.38:0/1891125409 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7abc003a20 con 0x55d0f3253ca0 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7adc62d740 1 -- 172.21.15.38:0/1891125409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7acc00f960 con 0x55d0f3253ca0 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ab27fc640 1 -- 172.21.15.38:0/1891125409 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7abc054050 con 0x55d0f3253ca0 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7adc62d740 10 client.9075 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7adc62d740 1 -- 172.21.15.38:0/1891125409 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55d0f2ff5f60 con 0x55d0f3253ca0 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.244+0000 7f7ab27fc640 1 -- 172.21.15.38:0/1891125409 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc00f960 con 0x55d0f3253ca0 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.245+0000 7f7ab27fc640 1 -- 172.21.15.38:0/1891125409 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7abc057020 con 0x55d0f3253ca0 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.245+0000 7f7adc62d740 10 client.9075 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.245+0000 7f7adc62d740 10 client.9075 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.245+0000 7f7adc62d740 20 client.9075 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.245+0000 7f7adc62d740 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x55d0f32a9980 0x55d0f3292850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.245+0000 7f7adc62d740 4 client.9075 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.245+0000 7f7adc62d740 1 -- 172.21.15.38:0/1891125409 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0f31c40d0 con 0x55d0f32a9980 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.245+0000 7f7ad27fc640 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x55d0f32a9980 0x55d0f3292850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.245+0000 7f7ad27fc640 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x55d0f32a9980 0x55d0f3292850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c06f5272176180d5 server_cookie=72898c38fe857135 in_seq=0 out_seq=0 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.245+0000 7f7ab27fc640 10 client.9075 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7ab27fc640 1 -- 172.21.15.38:0/1891125409 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d0f31c40d0 con 0x55d0f32a9980 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7ab27fc640 10 client.9075 handle_command_reply: tid=0 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7adc62d740 1 client.9075 shutdown 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7adc62d740 1 -- 172.21.15.38:0/1891125409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3253ca0 msgr2=0x55d0f327b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7adc62d740 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3253ca0 0x55d0f327b4f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009890 tx=0x7f7abc02c070 comp rx=0 tx=0).stop 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7adc62d740 1 -- 172.21.15.38:0/1891125409 shutdown_connections 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7adc62d740 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x55d0f32a9980 0x55d0f3292850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7adc62d740 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3290370 0x55d0f3294d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7adc62d740 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f327e760 0x55d0f3280ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7ab27fc640 0 client.9075 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7adc62d740 1 --2- 172.21.15.38:0/1891125409 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3253ca0 0x55d0f327b4f0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7adc62d740 1 -- 172.21.15.38:0/1891125409 >> 172.21.15.38:0/1891125409 conn(0x55d0f321b870 msgr2=0x55d0f327c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7adc62d740 1 -- 172.21.15.38:0/1891125409 shutdown_connections 2024-04-25T08:40:21.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7adc62d740 1 -- 172.21.15.38:0/1891125409 wait complete. 2024-04-25T08:40:21.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.246+0000 7f7adc62d740 20 client.9075 trim_cache size 0 max 16384 2024-04-25T08:40:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.261+0000 7f7adc62d740 1 Processor -- start 2024-04-25T08:40:21.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.261+0000 7f7adc62d740 1 -- start start 2024-04-25T08:40:21.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7adc62d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f3253ca0 0x55d0f3293000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7adc62d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f327e760 0x55d0f3299ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7adc62d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 0x55d0f329a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7adc62d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad4200300 con 0x55d0f3290370 2024-04-25T08:40:21.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7adc62d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad41ff460 con 0x55d0f327e760 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7adc62d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40b3ff0 con 0x55d0f3253ca0 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 0x55d0f329a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 0x55d0f329a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35594/0 (socket says 172.21.15.38:35594) 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad37fe640 1 -- 172.21.15.38:0/3103171989 learned_addr learned my addr 172.21.15.38:0/3103171989 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/3103171989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f3253ca0 0x55d0f3293000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad27fc640 1 --2- 172.21.15.38:0/3103171989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f327e760 0x55d0f3299ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad37fe640 1 -- 172.21.15.38:0/3103171989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f3253ca0 msgr2=0x55d0f3293000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad37fe640 1 --2- 172.21.15.38:0/3103171989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f3253ca0 0x55d0f3293000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad37fe640 1 -- 172.21.15.38:0/3103171989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f327e760 msgr2=0x55d0f3299ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad37fe640 1 --2- 172.21.15.38:0/3103171989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f327e760 0x55d0f3299ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad37fe640 1 -- 172.21.15.38:0/3103171989 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0f2e641b0 con 0x55d0f3290370 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/3103171989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f3253ca0 0x55d0f3293000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad27fc640 1 --2- 172.21.15.38:0/3103171989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f327e760 0x55d0f3299ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:21.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ad37fe640 1 --2- 172.21.15.38:0/3103171989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 0x55d0f329a0e0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00f5b0 tx=0x7f7acc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7e1e57a600b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ac8ff9640 1 -- 172.21.15.38:0/3103171989 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc018040 con 0x55d0f3290370 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ac8ff9640 1 -- 172.21.15.38:0/3103171989 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc00f9a0 con 0x55d0f3290370 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.262+0000 7f7ac8ff9640 1 -- 172.21.15.38:0/3103171989 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc00fd50 con 0x55d0f3290370 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.263+0000 7f7adc62d740 1 -- 172.21.15.38:0/3103171989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 msgr2=0x55d0f329a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.263+0000 7f7adc62d740 1 --2- 172.21.15.38:0/3103171989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 0x55d0f329a0e0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00f5b0 tx=0x7f7acc014000 comp rx=0 tx=0).stop 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.263+0000 7f7adc62d740 1 -- 172.21.15.38:0/3103171989 shutdown_connections 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.263+0000 7f7adc62d740 1 --2- 172.21.15.38:0/3103171989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 0x55d0f329a0e0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.263+0000 7f7adc62d740 1 --2- 172.21.15.38:0/3103171989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f327e760 0x55d0f3299ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.263+0000 7f7adc62d740 1 --2- 172.21.15.38:0/3103171989 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f3253ca0 0x55d0f3293000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.263+0000 7f7adc62d740 1 -- 172.21.15.38:0/3103171989 >> 172.21.15.38:0/3103171989 conn(0x55d0f321b870 msgr2=0x55d0f3281930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.263+0000 7f7adc62d740 1 -- 172.21.15.38:0/3103171989 shutdown_connections 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.263+0000 7f7adc62d740 1 -- 172.21.15.38:0/3103171989 wait complete. 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.264+0000 7f7adc62d740 1 -- start start 2024-04-25T08:40:21.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.264+0000 7f7adc62d740 1 Processor -- start 2024-04-25T08:40:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.264+0000 7f7adc62d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3253ca0 0x55d0f329d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.264+0000 7f7adc62d740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f327e760 0x55d0f3299ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.264+0000 7f7adc62d740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 0x55d0f329a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.264+0000 7f7adc62d740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad41fee90 con 0x55d0f3290370 2024-04-25T08:40:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.264+0000 7f7adc62d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f32bc450 con 0x55d0f3253ca0 2024-04-25T08:40:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.264+0000 7f7adc62d740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0f32b2820 con 0x55d0f327e760 2024-04-25T08:40:21.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.264+0000 7f7adc62d740 10 client.0 _ll_register_callbacks cb 0x7f7abc009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.264+0000 7f7ad37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 0x55d0f329a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ad27fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f327e760 0x55d0f3299ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ad37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 0x55d0f329a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35600/0 (socket says 172.21.15.38:35600) 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ad2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3253ca0 0x55d0f329d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ad37fe640 1 -- 172.21.15.38:0/1042092108 learned_addr learned my addr 172.21.15.38:0/1042092108 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ad37fe640 1 -- 172.21.15.38:0/1042092108 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f327e760 msgr2=0x55d0f3299ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ad37fe640 1 --2- 172.21.15.38:0/1042092108 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f327e760 0x55d0f3299ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ad37fe640 1 -- 172.21.15.38:0/1042092108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3253ca0 msgr2=0x55d0f329d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ad37fe640 1 --2- 172.21.15.38:0/1042092108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3253ca0 0x55d0f329d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ad37fe640 1 -- 172.21.15.38:0/1042092108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0f32b33e0 con 0x55d0f3290370 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ad37fe640 1 --2- 172.21.15.38:0/1042092108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 0x55d0f329a0e0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f7acc000c00 tx=0x7f7acc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f0beb85f2ff260 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ab27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ab27fc640 1 -- 172.21.15.38:0/1042092108 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc035060 con 0x55d0f3290370 2024-04-25T08:40:21.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ab27fc640 1 -- 172.21.15.38:0/1042092108 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7acc00e040 con 0x55d0f3290370 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7adc62d740 1 -- 172.21.15.38:0/1042092108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7acc00f9a0 con 0x55d0f3290370 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ab27fc640 1 -- 172.21.15.38:0/1042092108 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7acc018030 con 0x55d0f3290370 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7adc62d740 10 client.9081 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7adc62d740 1 -- 172.21.15.38:0/1042092108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x55d0f3304600 con 0x55d0f3290370 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.265+0000 7f7ab27fc640 1 -- 172.21.15.38:0/1042092108 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc00f9a0 con 0x55d0f3290370 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7ab27fc640 1 -- 172.21.15.38:0/1042092108 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7acc004050 con 0x55d0f3290370 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 10 client.9081 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 10 client.9081 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 20 client.9081 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 1 --2- 172.21.15.38:0/1042092108 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x55d0f32825b0 0x55d0f327d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 4 client.9081 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 1 -- 172.21.15.38:0/1042092108 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55d0f31be420 con 0x55d0f32825b0 2024-04-25T08:40:21.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/1042092108 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x55d0f32825b0 0x55d0f327d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7ab27fc640 10 client.9081 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7ad2ffd640 1 --2- 172.21.15.38:0/1042092108 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x55d0f32825b0 0x55d0f327d260 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166f4400ebf683f4 server_cookie=5f80ce1084afac7a in_seq=0 out_seq=0 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7ab27fc640 1 -- 172.21.15.38:0/1042092108 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55d0f31be420 con 0x55d0f32825b0 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7ab27fc640 10 client.9081 handle_command_reply: tid=0 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 1 client.9081 shutdown 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 1 -- 172.21.15.38:0/1042092108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 msgr2=0x55d0f329a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 1 --2- 172.21.15.38:0/1042092108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 0x55d0f329a0e0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f7acc000c00 tx=0x7f7acc004730 comp rx=0 tx=0).stop 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 1 -- 172.21.15.38:0/1042092108 shutdown_connections 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 1 --2- 172.21.15.38:0/1042092108 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x55d0f32825b0 0x55d0f327d260 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 1 --2- 172.21.15.38:0/1042092108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d0f3290370 0x55d0f329a0e0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 1 --2- 172.21.15.38:0/1042092108 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55d0f327e760 0x55d0f3299ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 1 --2- 172.21.15.38:0/1042092108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d0f3253ca0 0x55d0f329d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7ab27fc640 0 client.9081 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:21.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.266+0000 7f7adc62d740 1 -- 172.21.15.38:0/1042092108 >> 172.21.15.38:0/1042092108 conn(0x55d0f321b870 msgr2=0x55d0f3292760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7adc62d740 1 -- 172.21.15.38:0/1042092108 shutdown_connections 2024-04-25T08:40:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7adc62d740 1 -- 172.21.15.38:0/1042092108 wait complete. 2024-04-25T08:40:21.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7adc62d740 20 client.9081 trim_cache size 0 max 16384 2024-04-25T08:40:21.266 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:40:21.266 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:40:21.266 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:40:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7ad984d640 1 -- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ab406b4d0 msgr2=0x7f7ab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7ad984d640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ab406b4d0 0x7f7ab406d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7abc034000 tx=0x7f7abc02e000 comp rx=0 tx=0).stop 2024-04-25T08:40:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7ad984d640 1 -- 172.21.15.38:0/295557580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad40d0320 msgr2=0x7f7ad40ac240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7ad984d640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad40d0320 0x7f7ad40ac240 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400cbc0 tx=0x7f7ac400a060 comp rx=0 tx=0).stop 2024-04-25T08:40:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7ad984d640 1 -- 172.21.15.38:0/295557580 shutdown_connections 2024-04-25T08:40:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7ad984d640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ad40d0700 0x7f7ad40b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7ad984d640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ad40d0320 0x7f7ad40ac240 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7ad984d640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7ab406b4d0 0x7f7ab406d990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7ad984d640 1 --2- 172.21.15.38:0/295557580 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ad40b6980 0x7f7ad41f3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7ad984d640 1 -- 172.21.15.38:0/295557580 >> 172.21.15.38:0/295557580 conn(0x7f7ad40668f0 msgr2=0x7f7ad40cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7ad984d640 1 -- 172.21.15.38:0/295557580 shutdown_connections 2024-04-25T08:40:21.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.267+0000 7f7ad984d640 1 -- 172.21.15.38:0/295557580 wait complete. 2024-04-25T08:40:21.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:21.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T08:40:21.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.382+0000 7fe71f4bc640 1 Processor -- start 2024-04-25T08:40:21.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.382+0000 7fe71f4bc640 1 -- start start 2024-04-25T08:40:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.383+0000 7fe71f4bc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7180d0320 0x7fe7180d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.383+0000 7fe71f4bc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe7180d0700 0x7fe7180d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.383+0000 7fe71f4bc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7180d2330 con 0x7fe7180d1e90 2024-04-25T08:40:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.383+0000 7fe71f4bc640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7180d2610 con 0x7fe7180d0700 2024-04-25T08:40:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.383+0000 7fe71f4bc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7180d28f0 con 0x7fe7180d0320 2024-04-25T08:40:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.383+0000 7fe71ca57640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe7180d0700 0x7fe7180d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.383+0000 7fe71ca57640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe7180d0700 0x7fe7180d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35612/0 (socket says 172.21.15.38:35612) 2024-04-25T08:40:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.383+0000 7fe71ca57640 1 -- 172.21.15.38:0/4023652940 learned_addr learned my addr 172.21.15.38:0/4023652940 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.383+0000 7fe71d258640 1 --2- 172.21.15.38:0/4023652940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7180d0320 0x7fe7180d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.384+0000 7fe71ca57640 1 -- 172.21.15.38:0/4023652940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7180d0320 msgr2=0x7fe7180d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.384+0000 7fe71ca57640 1 --2- 172.21.15.38:0/4023652940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7180d0320 0x7fe7180d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.384+0000 7fe71ca57640 1 -- 172.21.15.38:0/4023652940 >> v1:172.21.15.107:6789/0 conn(0x7fe7180d1e90 legacy=0x7fe7180b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.384+0000 7fe71ca57640 1 -- 172.21.15.38:0/4023652940 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7180d2bd0 con 0x7fe7180d0700 2024-04-25T08:40:21.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.384+0000 7fe71ca57640 1 --2- 172.21.15.38:0/4023652940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7180d0700 0x7fe7180d1950 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009870 tx=0x7fe70c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b265be73a1fcdfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.384+0000 7fe717fff640 1 -- 172.21.15.38:0/4023652940 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c037070 con 0x7fe7180d0700 2024-04-25T08:40:21.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.385+0000 7fe717fff640 1 -- 172.21.15.38:0/4023652940 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c02ae50 con 0x7fe7180d0700 2024-04-25T08:40:21.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.385+0000 7fe717fff640 1 -- 172.21.15.38:0/4023652940 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c032860 con 0x7fe7180d0700 2024-04-25T08:40:21.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.385+0000 7fe71f4bc640 1 -- 172.21.15.38:0/4023652940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7180d0700 msgr2=0x7fe7180d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.385+0000 7fe71f4bc640 1 --2- 172.21.15.38:0/4023652940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7180d0700 0x7fe7180d1950 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fe70c009870 tx=0x7fe70c02a300 comp rx=0 tx=0).stop 2024-04-25T08:40:21.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.385+0000 7fe71f4bc640 1 -- 172.21.15.38:0/4023652940 shutdown_connections 2024-04-25T08:40:21.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.385+0000 7fe71f4bc640 1 --2- 172.21.15.38:0/4023652940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7180d0700 0x7fe7180d1950 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.385+0000 7fe71f4bc640 1 --2- 172.21.15.38:0/4023652940 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7180d0320 0x7fe7180d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.385+0000 7fe71f4bc640 1 -- 172.21.15.38:0/4023652940 >> 172.21.15.38:0/4023652940 conn(0x7fe7180668f0 msgr2=0x7fe718066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.385+0000 7fe71f4bc640 1 -- 172.21.15.38:0/4023652940 shutdown_connections 2024-04-25T08:40:21.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.385+0000 7fe71f4bc640 1 -- 172.21.15.38:0/4023652940 wait complete. 2024-04-25T08:40:21.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.386+0000 7fe71f4bc640 1 Processor -- start 2024-04-25T08:40:21.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.386+0000 7fe71f4bc640 1 -- start start 2024-04-25T08:40:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71f4bc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7180d0320 0x7fe7181f6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71f4bc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7180d0700 0x7fe7180b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71f4bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7180d1e90 0x7fe7181e7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71f4bc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7180d0b40 con 0x7fe7180d1e90 2024-04-25T08:40:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71f4bc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe718161210 con 0x7fe7180d0320 2024-04-25T08:40:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71f4bc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7180b4d00 con 0x7fe7180d0700 2024-04-25T08:40:21.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71da59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7180d1e90 0x7fe7181e7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71da59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7180d1e90 0x7fe7181e7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35628/0 (socket says 172.21.15.38:35628) 2024-04-25T08:40:21.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71da59640 1 -- 172.21.15.38:0/230363607 learned_addr learned my addr 172.21.15.38:0/230363607 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71ca57640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7180d0700 0x7fe7180b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71d258640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7180d0320 0x7fe7181f6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71da59640 1 -- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7180d0700 msgr2=0x7fe7180b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71da59640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7180d0700 0x7fe7180b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71da59640 1 -- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7180d0320 msgr2=0x7fe7181f6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71da59640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7180d0320 0x7fe7181f6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.387+0000 7fe71da59640 1 -- 172.21.15.38:0/230363607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7180b40f0 con 0x7fe7180d1e90 2024-04-25T08:40:21.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.388+0000 7fe71da59640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7180d1e90 0x7fe7181e7c70 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe71000c3c0 tx=0x7fe71000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.388+0000 7fe715ffb640 1 -- 172.21.15.38:0/230363607 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710005220 con 0x7fe7180d1e90 2024-04-25T08:40:21.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.388+0000 7fe715ffb640 1 -- 172.21.15.38:0/230363607 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7100053c0 con 0x7fe7180d1e90 2024-04-25T08:40:21.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.388+0000 7fe71f4bc640 1 -- 172.21.15.38:0/230363607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7180b43d0 con 0x7fe7180d1e90 2024-04-25T08:40:21.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.388+0000 7fe715ffb640 1 -- 172.21.15.38:0/230363607 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710007270 con 0x7fe7180d1e90 2024-04-25T08:40:21.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.388+0000 7fe71f4bc640 1 -- 172.21.15.38:0/230363607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7180b4990 con 0x7fe7180d1e90 2024-04-25T08:40:21.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.389+0000 7fe715ffb640 1 -- 172.21.15.38:0/230363607 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe710019020 con 0x7fe7180d1e90 2024-04-25T08:40:21.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.390+0000 7fe715ffb640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe70406b4d0 0x7fe70406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.390+0000 7fe715ffb640 1 -- 172.21.15.38:0/230363607 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe710089430 con 0x7fe7180d1e90 2024-04-25T08:40:21.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.390+0000 7fe71d258640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe70406b4d0 0x7fe70406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.391+0000 7fe71d258640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe70406b4d0 0x7fe70406d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe7181e78d0 tx=0x7fe70800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.391+0000 7fe7221bd740 1 Processor -- start 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.391+0000 7fe7221bd740 1 -- start start 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe7221bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 0x562668dab320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe7221bd740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668dab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe7221bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668d7b770 0x562668d7ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe7221bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562668ce4da0 con 0x562668da8b10 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe7221bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562668c8db80 con 0x562668d7b770 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe7221bd740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562668ce0e40 con 0x562668da8ef0 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe71d258640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 0x562668dab320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe71d258640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 0x562668dab320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35630/0 (socket says 172.21.15.38:35630) 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe71d258640 1 -- 172.21.15.38:0/1300068788 learned_addr learned my addr 172.21.15.38:0/1300068788 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe71ca57640 1 --2- 172.21.15.38:0/1300068788 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668dab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe71da59640 1 --2- 172.21.15.38:0/1300068788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668d7b770 0x562668d7ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe71d258640 1 -- 172.21.15.38:0/1300068788 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 msgr2=0x562668dab860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe71d258640 1 --2- 172.21.15.38:0/1300068788 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668dab860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe71d258640 1 -- 172.21.15.38:0/1300068788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668d7b770 msgr2=0x562668d7ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe71d258640 1 --2- 172.21.15.38:0/1300068788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668d7b770 0x562668d7ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe71d258640 1 -- 172.21.15.38:0/1300068788 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562668c97f10 con 0x562668da8b10 2024-04-25T08:40:21.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe71d258640 1 --2- 172.21.15.38:0/1300068788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 0x562668dab320 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fe708003d30 tx=0x7fe708014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead5c32cc4adf361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe6feffd640 1 -- 172.21.15.38:0/1300068788 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70801d050 con 0x562668da8b10 2024-04-25T08:40:21.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe6feffd640 1 -- 172.21.15.38:0/1300068788 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe708027070 con 0x562668da8b10 2024-04-25T08:40:21.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.392+0000 7fe6feffd640 1 -- 172.21.15.38:0/1300068788 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708015a80 con 0x562668da8b10 2024-04-25T08:40:21.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.393+0000 7fe7221bd740 1 -- 172.21.15.38:0/1300068788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 msgr2=0x562668dab320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.394 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.393+0000 7fe7221bd740 1 --2- 172.21.15.38:0/1300068788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 0x562668dab320 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7fe708003d30 tx=0x7fe708014000 comp rx=0 tx=0).stop 2024-04-25T08:40:21.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.393+0000 7fe7221bd740 1 -- 172.21.15.38:0/1300068788 shutdown_connections 2024-04-25T08:40:21.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.393+0000 7fe7221bd740 1 --2- 172.21.15.38:0/1300068788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668d7b770 0x562668d7ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.393+0000 7fe7221bd740 1 --2- 172.21.15.38:0/1300068788 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668dab860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.393+0000 7fe7221bd740 1 --2- 172.21.15.38:0/1300068788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 0x562668dab320 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.393+0000 7fe7221bd740 1 -- 172.21.15.38:0/1300068788 >> 172.21.15.38:0/1300068788 conn(0x562668d7d9e0 msgr2=0x562668dc6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.393+0000 7fe7221bd740 1 -- 172.21.15.38:0/1300068788 shutdown_connections 2024-04-25T08:40:21.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.393+0000 7fe7221bd740 1 -- 172.21.15.38:0/1300068788 wait complete. 2024-04-25T08:40:21.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.394+0000 7fe7221bd740 1 -- start start 2024-04-25T08:40:21.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.394+0000 7fe7221bd740 1 Processor -- start 2024-04-25T08:40:21.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe7221bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668da8b10 0x562668dca020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe7221bd740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668daf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe7221bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 0x562668dd6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe7221bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562668d80b90 con 0x562668d9f850 2024-04-25T08:40:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe7221bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562668c834b0 con 0x562668da8b10 2024-04-25T08:40:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe7221bd740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562668d80e70 con 0x562668da8ef0 2024-04-25T08:40:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe7221bd740 10 client.0 _ll_register_callbacks cb 0x562668cc3180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71da59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 0x562668dd6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71d258640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668da8b10 0x562668dca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71da59640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 0x562668dd6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35646/0 (socket says 172.21.15.38:35646) 2024-04-25T08:40:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71d258640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668da8b10 0x562668dca020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:59464/0 (socket says 172.21.15.38:59464) 2024-04-25T08:40:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71da59640 1 -- 172.21.15.38:0/259389841 learned_addr learned my addr 172.21.15.38:0/259389841 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71ca57640 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668daf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71da59640 1 -- 172.21.15.38:0/259389841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 msgr2=0x562668daf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71da59640 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668daf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71da59640 1 -- 172.21.15.38:0/259389841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668da8b10 msgr2=0x562668dca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71da59640 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668da8b10 0x562668dca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71da59640 1 -- 172.21.15.38:0/259389841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562668ac37d0 con 0x562668d9f850 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71d258640 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668da8b10 0x562668dca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe71da59640 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 0x562668dd6000 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fe71005fc00 tx=0x7fe71005bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd36223de5539e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe6f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe6f7fff640 1 -- 172.21.15.38:0/259389841 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710007410 con 0x562668d9f850 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe6f7fff640 1 -- 172.21.15.38:0/259389841 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7100075b0 con 0x562668d9f850 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.395+0000 7fe7221bd740 1 -- 172.21.15.38:0/259389841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe708027070 con 0x562668d9f850 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe6f7fff640 1 -- 172.21.15.38:0/259389841 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe710076050 con 0x562668d9f850 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe7221bd740 10 client.9093 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:21.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe7221bd740 1 -- 172.21.15.38:0/259389841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x5626689f93f0 con 0x562668d9f850 2024-04-25T08:40:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe6f7fff640 1 -- 172.21.15.38:0/259389841 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe708027070 con 0x562668d9f850 2024-04-25T08:40:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe6f7fff640 1 -- 172.21.15.38:0/259389841 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe710057d30 con 0x562668d9f850 2024-04-25T08:40:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe7221bd740 10 client.9093 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe7221bd740 10 client.9093 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe7221bd740 20 client.9093 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe7221bd740 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x562668d7aad0 0x562668dc7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe7221bd740 4 client.9093 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:40:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe7221bd740 1 -- 172.21.15.38:0/259389841 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562668cf22e0 con 0x562668d7aad0 2024-04-25T08:40:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe71d258640 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x562668d7aad0 0x562668dc7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe6f7fff640 10 client.9093 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.396+0000 7fe71d258640 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x562668d7aad0 0x562668dc7450 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd06924040fa6e4 server_cookie=766217c3ea85ec55 in_seq=0 out_seq=0 2024-04-25T08:40:21.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe6f7fff640 1 -- 172.21.15.38:0/259389841 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562668cf22e0 con 0x562668d7aad0 2024-04-25T08:40:21.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe6f7fff640 10 client.9093 handle_command_reply: tid=0 2024-04-25T08:40:21.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe7221bd740 1 client.9093 shutdown 2024-04-25T08:40:21.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe7221bd740 1 -- 172.21.15.38:0/259389841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 msgr2=0x562668dd6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe7221bd740 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 0x562668dd6000 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fe71005fc00 tx=0x7fe71005bb70 comp rx=0 tx=0).stop 2024-04-25T08:40:21.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe7221bd740 1 -- 172.21.15.38:0/259389841 shutdown_connections 2024-04-25T08:40:21.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe7221bd740 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x562668d7aad0 0x562668dc7450 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe7221bd740 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 0x562668dd6000 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe7221bd740 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668daf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe6f7fff640 0 client.9093 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:21.399 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe7221bd740 1 --2- 172.21.15.38:0/259389841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668da8b10 0x562668dca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe7221bd740 1 -- 172.21.15.38:0/259389841 >> 172.21.15.38:0/259389841 conn(0x562668d7b770 msgr2=0x562668d803a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe7221bd740 1 -- 172.21.15.38:0/259389841 shutdown_connections 2024-04-25T08:40:21.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe7221bd740 1 -- 172.21.15.38:0/259389841 wait complete. 2024-04-25T08:40:21.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.397+0000 7fe7221bd740 20 client.9093 trim_cache size 0 max 16384 2024-04-25T08:40:21.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe7221bd740 1 Processor -- start 2024-04-25T08:40:21.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe7221bd740 1 -- start start 2024-04-25T08:40:21.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe7221bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 0x562668d7ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe7221bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668da8b10 0x562668d7f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe7221bd740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668da7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe7221bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562668dd6c40 con 0x562668d9f850 2024-04-25T08:40:21.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe7221bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe718200350 con 0x562668da8b10 2024-04-25T08:40:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe7221bd740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7181ff510 con 0x562668da8ef0 2024-04-25T08:40:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe71d258640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 0x562668d7ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe71d258640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 0x562668d7ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35660/0 (socket says 172.21.15.38:35660) 2024-04-25T08:40:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe71d258640 1 -- 172.21.15.38:0/206974548 learned_addr learned my addr 172.21.15.38:0/206974548 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe71ca57640 1 --2- 172.21.15.38:0/206974548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668da8b10 0x562668d7f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.413+0000 7fe71da59640 1 --2- 172.21.15.38:0/206974548 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668da7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe71d258640 1 -- 172.21.15.38:0/206974548 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 msgr2=0x562668da7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe71d258640 1 --2- 172.21.15.38:0/206974548 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668da7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe71d258640 1 -- 172.21.15.38:0/206974548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668da8b10 msgr2=0x562668d7f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe71d258640 1 --2- 172.21.15.38:0/206974548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668da8b10 0x562668d7f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe71d258640 1 -- 172.21.15.38:0/206974548 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626689b90f0 con 0x562668d9f850 2024-04-25T08:40:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe71da59640 1 --2- 172.21.15.38:0/206974548 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668da7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe71d258640 1 --2- 172.21.15.38:0/206974548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 0x562668d7ef80 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe708024040 tx=0x7fe708016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cebae02362d5c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe6fe7fc640 1 -- 172.21.15.38:0/206974548 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70801d040 con 0x562668d9f850 2024-04-25T08:40:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe6fe7fc640 1 -- 172.21.15.38:0/206974548 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe708027040 con 0x562668d9f850 2024-04-25T08:40:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe6fe7fc640 1 -- 172.21.15.38:0/206974548 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe708015d20 con 0x562668d9f850 2024-04-25T08:40:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe7221bd740 1 -- 172.21.15.38:0/206974548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 msgr2=0x562668d7ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe7221bd740 1 --2- 172.21.15.38:0/206974548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 0x562668d7ef80 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe708024040 tx=0x7fe708016000 comp rx=0 tx=0).stop 2024-04-25T08:40:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe7221bd740 1 -- 172.21.15.38:0/206974548 shutdown_connections 2024-04-25T08:40:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe7221bd740 1 --2- 172.21.15.38:0/206974548 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668da7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe7221bd740 1 --2- 172.21.15.38:0/206974548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668da8b10 0x562668d7f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe7221bd740 1 --2- 172.21.15.38:0/206974548 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668d9f850 0x562668d7ef80 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.414+0000 7fe7221bd740 1 -- 172.21.15.38:0/206974548 >> 172.21.15.38:0/206974548 conn(0x562668d7b770 msgr2=0x562668dd5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.415+0000 7fe7221bd740 1 -- 172.21.15.38:0/206974548 shutdown_connections 2024-04-25T08:40:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.415+0000 7fe7221bd740 1 -- 172.21.15.38:0/206974548 wait complete. 2024-04-25T08:40:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.415+0000 7fe7221bd740 1 -- start start 2024-04-25T08:40:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.415+0000 7fe7221bd740 1 Processor -- start 2024-04-25T08:40:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.416+0000 7fe7221bd740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668d9f850 0x562668dc5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.416+0000 7fe7221bd740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 0x562668e30280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.416+0000 7fe7221bd740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668d7ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.416+0000 7fe7221bd740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7181f4fa0 con 0x562668da8b10 2024-04-25T08:40:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.416+0000 7fe7221bd740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7181f49a0 con 0x562668d9f850 2024-04-25T08:40:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.416+0000 7fe7221bd740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562668d80b90 con 0x562668da8ef0 2024-04-25T08:40:21.415 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.416+0000 7fe7221bd740 10 client.0 _ll_register_callbacks cb 0x7fe708009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.416+0000 7fe71ca57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 0x562668e30280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.416+0000 7fe71d258640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668d9f850 0x562668dc5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.416+0000 7fe71ca57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 0x562668e30280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35664/0 (socket says 172.21.15.38:35664) 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.416+0000 7fe71ca57640 1 -- 172.21.15.38:0/536193942 learned_addr learned my addr 172.21.15.38:0/536193942 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.416+0000 7fe71da59640 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668d7ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe71ca57640 1 -- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 msgr2=0x562668d7ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe71ca57640 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668d7ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe71ca57640 1 -- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668d9f850 msgr2=0x562668dc5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe71ca57640 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668d9f850 0x562668dc5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe71ca57640 1 -- 172.21.15.38:0/536193942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562668dc8370 con 0x562668da8b10 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe71d258640 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668d9f850 0x562668dc5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe71da59640 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668d7ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:21.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe71ca57640 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 0x562668e30280 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fe70c000c00 tx=0x7fe70c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508ddc17e0a3402a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe6f7fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:21.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe6f7fff640 1 -- 172.21.15.38:0/536193942 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c04c060 con 0x562668da8b10 2024-04-25T08:40:21.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe6f7fff640 1 -- 172.21.15.38:0/536193942 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe70c047470 con 0x562668da8b10 2024-04-25T08:40:21.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe7221bd740 1 -- 172.21.15.38:0/536193942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe708027040 con 0x562668da8b10 2024-04-25T08:40:21.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe6f7fff640 1 -- 172.21.15.38:0/536193942 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe70c052050 con 0x562668da8b10 2024-04-25T08:40:21.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe7221bd740 10 client.9099 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:21.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe7221bd740 1 -- 172.21.15.38:0/536193942 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x562668e31ca0 con 0x562668da8b10 2024-04-25T08:40:21.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe6f7fff640 1 -- 172.21.15.38:0/536193942 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe708027040 con 0x562668da8b10 2024-04-25T08:40:21.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe6f7fff640 1 -- 172.21.15.38:0/536193942 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe70c055020 con 0x562668da8b10 2024-04-25T08:40:21.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe7221bd740 10 client.9099 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:21.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe7221bd740 10 client.9099 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:21.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe7221bd740 20 client.9099 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:21.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe7221bd740 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x562668d7f4c0 0x562668da7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe7221bd740 4 client.9099 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:40:21.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.417+0000 7fe7221bd740 1 -- 172.21.15.38:0/536193942 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562668cec630 con 0x562668d7f4c0 2024-04-25T08:40:21.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe71d258640 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x562668d7f4c0 0x562668da7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe71d258640 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x562668d7f4c0 0x562668da7170 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c87487bf9453d8 server_cookie=9787731e1d155d2 in_seq=0 out_seq=0 2024-04-25T08:40:21.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe6f7fff640 10 client.9099 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:21.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe6f7fff640 1 -- 172.21.15.38:0/536193942 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x562668cec630 con 0x562668d7f4c0 2024-04-25T08:40:21.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe6f7fff640 10 client.9099 handle_command_reply: tid=0 2024-04-25T08:40:21.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe7221bd740 1 client.9099 shutdown 2024-04-25T08:40:21.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe7221bd740 1 -- 172.21.15.38:0/536193942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 msgr2=0x562668e30280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe7221bd740 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 0x562668e30280 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fe70c000c00 tx=0x7fe70c004730 comp rx=0 tx=0).stop 2024-04-25T08:40:21.418 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe7221bd740 1 -- 172.21.15.38:0/536193942 shutdown_connections 2024-04-25T08:40:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe7221bd740 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x562668d7f4c0 0x562668da7170 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe7221bd740 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562668da8ef0 0x562668d7ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe7221bd740 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562668da8b10 0x562668e30280 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe7221bd740 1 --2- 172.21.15.38:0/536193942 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562668d9f850 0x562668dc5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe6f7fff640 0 client.9099 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe7221bd740 1 -- 172.21.15.38:0/536193942 >> 172.21.15.38:0/536193942 conn(0x562668d7b770 msgr2=0x562668dd5dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe7221bd740 1 -- 172.21.15.38:0/536193942 shutdown_connections 2024-04-25T08:40:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe7221bd740 1 -- 172.21.15.38:0/536193942 wait complete. 2024-04-25T08:40:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.418+0000 7fe7221bd740 20 client.9099 trim_cache size 0 max 16384 2024-04-25T08:40:21.419 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:40:21.420 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T08:40:21.420 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-04-25T08:40:21.420 INFO:teuthology.orchestra.run.smithi038.stdout: "68c7ac46-3acc-4dde-9256-8f7c0c44785b": { 2024-04-25T08:40:21.420 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/scrub_control_test_path", 2024-04-25T08:40:21.420 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "68c7ac46-3acc-4dde-9256-8f7c0c44785b", 2024-04-25T08:40:21.420 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive" 2024-04-25T08:40:21.420 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:21.420 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:21.420 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:40:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.419+0000 7fe71f4bc640 1 -- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe70406b4d0 msgr2=0x7fe70406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.419+0000 7fe71f4bc640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe70406b4d0 0x7fe70406d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe7181e78d0 tx=0x7fe70800b040 comp rx=0 tx=0).stop 2024-04-25T08:40:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.419+0000 7fe71f4bc640 1 -- 172.21.15.38:0/230363607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7180d1e90 msgr2=0x7fe7181e7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.419+0000 7fe71f4bc640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7180d1e90 0x7fe7181e7c70 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fe71000c3c0 tx=0x7fe71000c880 comp rx=0 tx=0).stop 2024-04-25T08:40:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.419+0000 7fe71f4bc640 1 -- 172.21.15.38:0/230363607 shutdown_connections 2024-04-25T08:40:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.419+0000 7fe71f4bc640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe7180d1e90 0x7fe7181e7c70 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.419+0000 7fe71f4bc640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe70406b4d0 0x7fe70406d990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.421 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.419+0000 7fe71f4bc640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe7180d0700 0x7fe7180b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.419+0000 7fe71f4bc640 1 --2- 172.21.15.38:0/230363607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7180d0320 0x7fe7181f6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.419+0000 7fe71f4bc640 1 -- 172.21.15.38:0/230363607 >> 172.21.15.38:0/230363607 conn(0x7fe7180668f0 msgr2=0x7fe71815e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.419+0000 7fe71f4bc640 1 -- 172.21.15.38:0/230363607 shutdown_connections 2024-04-25T08:40:21.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.419+0000 7fe71f4bc640 1 -- 172.21.15.38:0/230363607 wait complete. 2024-04-25T08:40:21.423 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:21.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.531+0000 7fa9f5369640 1 Processor -- start 2024-04-25T08:40:21.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.532+0000 7fa9f5369640 1 -- start start 2024-04-25T08:40:21.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.532+0000 7fa9f5369640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa9f015df50 0x7fa9f00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.532+0000 7fa9f5369640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9f00ce2e0 0x7fa9f00d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.532+0000 7fa9f5369640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f00ceff0 con 0x7fa9f006c110 2024-04-25T08:40:21.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.532+0000 7fa9f5369640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f00cf2d0 con 0x7fa9f015df50 2024-04-25T08:40:21.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.532+0000 7fa9f5369640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00d1f70 con 0x7fa9f00ce2e0 2024-04-25T08:40:21.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.532+0000 7fa9ee7fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa9f015df50 0x7fa9f00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.533+0000 7fa9ee7fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa9f015df50 0x7fa9f00ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35670/0 (socket says 172.21.15.38:35670) 2024-04-25T08:40:21.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.533+0000 7fa9ee7fc640 1 -- 172.21.15.38:0/1935558910 learned_addr learned my addr 172.21.15.38:0/1935558910 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.533+0000 7fa9ef7fe640 1 --2- 172.21.15.38:0/1935558910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9f00ce2e0 0x7fa9f00d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.533+0000 7fa9ee7fc640 1 -- 172.21.15.38:0/1935558910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9f00ce2e0 msgr2=0x7fa9f00d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.533+0000 7fa9ee7fc640 1 --2- 172.21.15.38:0/1935558910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9f00ce2e0 0x7fa9f00d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.533+0000 7fa9ee7fc640 1 -- 172.21.15.38:0/1935558910 >> v1:172.21.15.107:6789/0 conn(0x7fa9f006c110 legacy=0x7fa9f006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.533+0000 7fa9ee7fc640 1 -- 172.21.15.38:0/1935558910 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f00d2250 con 0x7fa9f015df50 2024-04-25T08:40:21.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.534+0000 7fa9ee7fc640 1 --2- 172.21.15.38:0/1935558910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9f015df50 0x7fa9f00ce7a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009870 tx=0x7fa9d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b8bfe69b9ac516 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.534+0000 7fa9edffb640 1 -- 172.21.15.38:0/1935558910 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8037070 con 0x7fa9f015df50 2024-04-25T08:40:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.534+0000 7fa9edffb640 1 -- 172.21.15.38:0/1935558910 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d8032440 con 0x7fa9f015df50 2024-04-25T08:40:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.534+0000 7fa9edffb640 1 -- 172.21.15.38:0/1935558910 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8032810 con 0x7fa9f015df50 2024-04-25T08:40:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.534+0000 7fa9f5369640 1 -- 172.21.15.38:0/1935558910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9f015df50 msgr2=0x7fa9f00ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.534+0000 7fa9f5369640 1 --2- 172.21.15.38:0/1935558910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9f015df50 0x7fa9f00ce7a0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8009870 tx=0x7fa9d802a270 comp rx=0 tx=0).stop 2024-04-25T08:40:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.534+0000 7fa9f5369640 1 -- 172.21.15.38:0/1935558910 shutdown_connections 2024-04-25T08:40:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.534+0000 7fa9f5369640 1 --2- 172.21.15.38:0/1935558910 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9f00ce2e0 0x7fa9f00d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.534+0000 7fa9f5369640 1 --2- 172.21.15.38:0/1935558910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9f015df50 0x7fa9f00ce7a0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.534+0000 7fa9f5369640 1 -- 172.21.15.38:0/1935558910 >> 172.21.15.38:0/1935558910 conn(0x7fa9f00d3fb0 msgr2=0x7fa9f00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.534+0000 7fa9f5369640 1 -- 172.21.15.38:0/1935558910 shutdown_connections 2024-04-25T08:40:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.534+0000 7fa9f5369640 1 -- 172.21.15.38:0/1935558910 wait complete. 2024-04-25T08:40:21.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.536+0000 7fa9f5369640 1 Processor -- start 2024-04-25T08:40:21.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.536+0000 7fa9f5369640 1 -- start start 2024-04-25T08:40:21.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.536+0000 7fa9f5369640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9f006c110 0x7fa9f015a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.536+0000 7fa9f5369640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9f00ce2e0 0x7fa9f015a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.536+0000 7fa9f5369640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f015df50 0x7fa9f01e9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.536+0000 7fa9f5369640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00d2f90 con 0x7fa9f006c110 2024-04-25T08:40:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9f5369640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00cece0 con 0x7fa9f015df50 2024-04-25T08:40:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9f5369640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00d3110 con 0x7fa9f00ce2e0 2024-04-25T08:40:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9f006c110 0x7fa9f015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9f006c110 0x7fa9f015a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35686/0 (socket says 172.21.15.38:35686) 2024-04-25T08:40:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9eeffd640 1 -- 172.21.15.38:0/562563620 learned_addr learned my addr 172.21.15.38:0/562563620 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9ef7fe640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f015df50 0x7fa9f01e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9ee7fc640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9f00ce2e0 0x7fa9f015a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9eeffd640 1 -- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9f00ce2e0 msgr2=0x7fa9f015a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9eeffd640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9f00ce2e0 0x7fa9f015a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9eeffd640 1 -- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f015df50 msgr2=0x7fa9f01e9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9eeffd640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f015df50 0x7fa9f01e9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9eeffd640 1 -- 172.21.15.38:0/562563620 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f01e9760 con 0x7fa9f006c110 2024-04-25T08:40:21.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9eeffd640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9f006c110 0x7fa9f015a830 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800c3c0 tx=0x7fa9e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9e77fe640 1 -- 172.21.15.38:0/562563620 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80052d0 con 0x7fa9f006c110 2024-04-25T08:40:21.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9e77fe640 1 -- 172.21.15.38:0/562563620 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e8005490 con 0x7fa9f006c110 2024-04-25T08:40:21.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9f5369640 1 -- 172.21.15.38:0/562563620 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f01e99e0 con 0x7fa9f006c110 2024-04-25T08:40:21.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9e77fe640 1 -- 172.21.15.38:0/562563620 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8007260 con 0x7fa9f006c110 2024-04-25T08:40:21.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.537+0000 7fa9f5369640 1 -- 172.21.15.38:0/562563620 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f01e9fa0 con 0x7fa9f006c110 2024-04-25T08:40:21.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.538+0000 7fa9f5369640 1 -- 172.21.15.38:0/562563620 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f0068e80 con 0x7fa9f006c110 2024-04-25T08:40:21.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.539+0000 7fa9e77fe640 1 -- 172.21.15.38:0/562563620 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9e8007400 con 0x7fa9f006c110 2024-04-25T08:40:21.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.539+0000 7fa9e77fe640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa9d006b4d0 0x7fa9d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.539+0000 7fa9e77fe640 1 -- 172.21.15.38:0/562563620 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa9e8089730 con 0x7fa9f006c110 2024-04-25T08:40:21.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.539+0000 7fa9ee7fc640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa9d006b4d0 0x7fa9d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.540+0000 7fa9ee7fc640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa9d006b4d0 0x7fa9d006d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8034000 tx=0x7fa9d802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.544+0000 7fa9e77fe640 1 -- 172.21.15.38:0/562563620 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9f01e9760 con 0x7fa9f006c110 2024-04-25T08:40:21.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.762+0000 7fa9f5369640 1 -- 172.21.15.38:0/562563620 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9f0073650 con 0x7fa9f006c110 2024-04-25T08:40:21.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.762+0000 7fa9e77fe640 1 -- 172.21.15.38:0/562563620 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa9e805ce20 con 0x7fa9f006c110 2024-04-25T08:40:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:21.755 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:21.756 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:21.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.764+0000 7fa9f5369640 1 -- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa9d006b4d0 msgr2=0x7fa9d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.764+0000 7fa9f5369640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa9d006b4d0 0x7fa9d006d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8034000 tx=0x7fa9d802e000 comp rx=0 tx=0).stop 2024-04-25T08:40:21.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.764+0000 7fa9f5369640 1 -- 172.21.15.38:0/562563620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9f006c110 msgr2=0x7fa9f015a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.764+0000 7fa9f5369640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9f006c110 0x7fa9f015a830 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800c3c0 tx=0x7fa9e800c880 comp rx=0 tx=0).stop 2024-04-25T08:40:21.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.764+0000 7fa9f5369640 1 -- 172.21.15.38:0/562563620 shutdown_connections 2024-04-25T08:40:21.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.764+0000 7fa9f5369640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9f015df50 0x7fa9f01e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.764+0000 7fa9f5369640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa9d006b4d0 0x7fa9d006d990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.764+0000 7fa9f5369640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9f00ce2e0 0x7fa9f015a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.764+0000 7fa9f5369640 1 --2- 172.21.15.38:0/562563620 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9f006c110 0x7fa9f015a830 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.764+0000 7fa9f5369640 1 -- 172.21.15.38:0/562563620 >> 172.21.15.38:0/562563620 conn(0x7fa9f00d3fb0 msgr2=0x7fa9f00d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.764+0000 7fa9f5369640 1 -- 172.21.15.38:0/562563620 shutdown_connections 2024-04-25T08:40:21.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.764+0000 7fa9f5369640 1 -- 172.21.15.38:0/562563620 wait complete. 2024-04-25T08:40:21.768 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:40:21.768 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:40:21.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.875+0000 7fb2f01c1640 1 Processor -- start 2024-04-25T08:40:21.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.875+0000 7fb2f01c1640 1 -- start start 2024-04-25T08:40:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2f01c1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb2e806c110 0x7fb2e806c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2f01c1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb2e80ce2e0 0x7fb2e80d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2f01c1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e806ca30 con 0x7fb2e815df50 2024-04-25T08:40:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2f01c1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e80cf2d0 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2f01c1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e80ceff0 con 0x7fb2e806c110 2024-04-25T08:40:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2ee75e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb2e80ce2e0 0x7fb2e80d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2ed75c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb2e815df50 0x7fb2e80ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60426/0 (socket says 172.21.15.38:60426) 2024-04-25T08:40:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2ed75c640 1 -- 172.21.15.38:0/2907864274 learned_addr learned my addr 172.21.15.38:0/2907864274 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2edf5d640 1 --2- 172.21.15.38:0/2907864274 >> v2:172.21.15.107:3300/0 conn(0x7fb2e806c110 0x7fb2e806c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2ee75e640 1 -- 172.21.15.38:0/2907864274 >> v2:172.21.15.107:3300/0 conn(0x7fb2e806c110 msgr2=0x7fb2e806c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2ee75e640 1 --2- 172.21.15.38:0/2907864274 >> v2:172.21.15.107:3300/0 conn(0x7fb2e806c110 0x7fb2e806c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2ee75e640 1 -- 172.21.15.38:0/2907864274 >> v1:172.21.15.38:6789/0 conn(0x7fb2e815df50 legacy=0x7fb2e80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2ee75e640 1 -- 172.21.15.38:0/2907864274 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e80d1f70 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.876+0000 7fb2edf5d640 1 --2- 172.21.15.38:0/2907864274 >> v2:172.21.15.107:3300/0 conn(0x7fb2e806c110 0x7fb2e806c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.877+0000 7fb2ee75e640 1 --2- 172.21.15.38:0/2907864274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e80ce2e0 0x7fb2e80d2a50 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fb2e808c550 tx=0x7fb2e002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841e7a2b4955a47e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.877+0000 7fb2ecf5b640 1 -- 172.21.15.38:0/2907864274 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0037070 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.877+0000 7fb2ecf5b640 1 -- 172.21.15.38:0/2907864274 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e0032440 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.877+0000 7fb2ecf5b640 1 -- 172.21.15.38:0/2907864274 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e002ae60 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.878+0000 7fb2f01c1640 1 -- 172.21.15.38:0/2907864274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e80ce2e0 msgr2=0x7fb2e80d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.878+0000 7fb2f01c1640 1 --2- 172.21.15.38:0/2907864274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e80ce2e0 0x7fb2e80d2a50 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fb2e808c550 tx=0x7fb2e002a110 comp rx=0 tx=0).stop 2024-04-25T08:40:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.878+0000 7fb2f01c1640 1 -- 172.21.15.38:0/2907864274 shutdown_connections 2024-04-25T08:40:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.878+0000 7fb2f01c1640 1 --2- 172.21.15.38:0/2907864274 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e80ce2e0 0x7fb2e80d2a50 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.878+0000 7fb2f01c1640 1 --2- 172.21.15.38:0/2907864274 >> v2:172.21.15.107:3300/0 conn(0x7fb2e806c110 0x7fb2e806c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.878+0000 7fb2f01c1640 1 -- 172.21.15.38:0/2907864274 >> 172.21.15.38:0/2907864274 conn(0x7fb2e80d3fb0 msgr2=0x7fb2e80d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.878+0000 7fb2f01c1640 1 -- 172.21.15.38:0/2907864274 shutdown_connections 2024-04-25T08:40:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.878+0000 7fb2f01c1640 1 -- 172.21.15.38:0/2907864274 wait complete. 2024-04-25T08:40:21.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.879+0000 7fb2f01c1640 1 Processor -- start 2024-04-25T08:40:21.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.879+0000 7fb2f01c1640 1 -- start start 2024-04-25T08:40:21.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2f01c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e806c110 0x7fb2e81610a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2f01c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e80ce2e0 0x7fb2e8148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2f01c1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e815df50 0x7fb2e815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2f01c1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e80d2f90 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2f01c1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e80d2530 con 0x7fb2e806c110 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2f01c1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e80d3110 con 0x7fb2e815df50 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2ed75c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e80ce2e0 0x7fb2e8148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2ed75c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e80ce2e0 0x7fb2e8148bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35714/0 (socket says 172.21.15.38:35714) 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2ed75c640 1 -- 172.21.15.38:0/834040600 learned_addr learned my addr 172.21.15.38:0/834040600 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2edf5d640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e806c110 0x7fb2e81610a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2ee75e640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e815df50 0x7fb2e815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2ed75c640 1 -- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e815df50 msgr2=0x7fb2e815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2ed75c640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e815df50 0x7fb2e815ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2ed75c640 1 -- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e806c110 msgr2=0x7fb2e81610a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2ed75c640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e806c110 0x7fb2e81610a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.880+0000 7fb2ed75c640 1 -- 172.21.15.38:0/834040600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e81f7d60 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.881+0000 7fb2ed75c640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e80ce2e0 0x7fb2e8148bd0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00c4a0 tx=0x7fb2dc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.881+0000 7fb2e67fc640 1 -- 172.21.15.38:0/834040600 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc017070 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.881+0000 7fb2e67fc640 1 -- 172.21.15.38:0/834040600 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc005ce0 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.881+0000 7fb2f01c1640 1 -- 172.21.15.38:0/834040600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e81605c0 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.881+0000 7fb2e67fc640 1 -- 172.21.15.38:0/834040600 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc007270 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.881+0000 7fb2f01c1640 1 -- 172.21.15.38:0/834040600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e8149110 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.882+0000 7fb2f01c1640 1 -- 172.21.15.38:0/834040600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e806c4f0 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.882+0000 7fb2e67fc640 1 -- 172.21.15.38:0/834040600 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2dc01a070 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.883+0000 7fb2e67fc640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb2d406b4d0 0x7fb2d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.883+0000 7fb2e67fc640 1 -- 172.21.15.38:0/834040600 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb2dc089690 con 0x7fb2e80ce2e0 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.883+0000 7fb2edf5d640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb2d406b4d0 0x7fb2d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.883+0000 7fb2edf5d640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb2d406b4d0 0x7fb2d406d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009fd0 tx=0x7fb2d80097f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:21.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:21.887+0000 7fb2e67fc640 1 -- 172.21.15.38:0/834040600 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2e806c4f0 con 0x7fb2e80ce2e0 2024-04-25T08:40:22.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.140+0000 7fb2f01c1640 1 -- 172.21.15.38:0/834040600 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb2e8068e80 con 0x7fb2e80ce2e0 2024-04-25T08:40:22.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.140+0000 7fb2e67fc640 1 -- 172.21.15.38:0/834040600 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7fb2dc057c90 con 0x7fb2e80ce2e0 2024-04-25T08:40:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:22.133 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":551,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":740,"data_bytes":752126305,"bytes_used":1947213824,"bytes_avail":771146899456,"bytes_total":773094113280,"write_bytes_sec":62619260,"read_op_per_sec":0,"write_op_per_sec":61},"fsmap":{"epoch":70,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":7639}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T08:39:39.318198+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:40:22.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.142+0000 7fb2f01c1640 1 -- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb2d406b4d0 msgr2=0x7fb2d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:22.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.142+0000 7fb2f01c1640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb2d406b4d0 0x7fb2d406d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8009fd0 tx=0x7fb2d80097f0 comp rx=0 tx=0).stop 2024-04-25T08:40:22.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.142+0000 7fb2f01c1640 1 -- 172.21.15.38:0/834040600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e80ce2e0 msgr2=0x7fb2e8148bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.142+0000 7fb2f01c1640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e80ce2e0 0x7fb2e8148bd0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00c4a0 tx=0x7fb2dc00c960 comp rx=0 tx=0).stop 2024-04-25T08:40:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.142+0000 7fb2f01c1640 1 -- 172.21.15.38:0/834040600 shutdown_connections 2024-04-25T08:40:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.142+0000 7fb2f01c1640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb2d406b4d0 0x7fb2d406d990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.142+0000 7fb2f01c1640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e815df50 0x7fb2e815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.142+0000 7fb2f01c1640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e80ce2e0 0x7fb2e8148bd0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.142+0000 7fb2f01c1640 1 --2- 172.21.15.38:0/834040600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e806c110 0x7fb2e81610a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.142+0000 7fb2f01c1640 1 -- 172.21.15.38:0/834040600 >> 172.21.15.38:0/834040600 conn(0x7fb2e80d3fb0 msgr2=0x7fb2e815d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.142+0000 7fb2f01c1640 1 -- 172.21.15.38:0/834040600 shutdown_connections 2024-04-25T08:40:22.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:22.142+0000 7fb2f01c1640 1 -- 172.21.15.38:0/834040600 wait complete. 2024-04-25T08:40:22.146 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:40:23.146 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:23.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.267+0000 7fe621fb0640 1 Processor -- start 2024-04-25T08:40:23.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.267+0000 7fe621fb0640 1 -- start start 2024-04-25T08:40:23.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.267+0000 7fe621fb0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe61c15dc10 0x7fe61c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:23.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.267+0000 7fe621fb0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61c0d3e70 con 0x7fe61c0d0f60 2024-04-25T08:40:23.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.267+0000 7fe621fb0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61c0d4150 con 0x7fe61c0ced60 2024-04-25T08:40:23.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.267+0000 7fe621fb0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61c15dff0 con 0x7fe61c15dc10 2024-04-25T08:40:23.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.268+0000 7fe61b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe61c15dc10 0x7fe61c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:23.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.268+0000 7fe61b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe61c15dc10 0x7fe61c0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35722/0 (socket says 172.21.15.38:35722) 2024-04-25T08:40:23.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.268+0000 7fe61bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe61c0d0f60 0x7fe61c0b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60442/0 (socket says 172.21.15.38:60442) 2024-04-25T08:40:23.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.268+0000 7fe61b7fe640 1 -- 172.21.15.38:0/3164777724 learned_addr learned my addr 172.21.15.38:0/3164777724 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:23.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.268+0000 7fe61b7fe640 1 -- 172.21.15.38:0/3164777724 >> v1:172.21.15.107:6789/0 conn(0x7fe61c0ced60 legacy=0x7fe61c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:23.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.268+0000 7fe61b7fe640 1 -- 172.21.15.38:0/3164777724 >> v1:172.21.15.38:6789/0 conn(0x7fe61c0d0f60 legacy=0x7fe61c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:23.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.268+0000 7fe61a7fc640 1 -- 172.21.15.38:0/3164777724 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198751797 0 0) 0x7fe61c0d3e70 con 0x7fe61c0d0f60 2024-04-25T08:40:23.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.268+0000 7fe61b7fe640 1 -- 172.21.15.38:0/3164777724 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c0d4430 con 0x7fe61c15dc10 2024-04-25T08:40:23.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.269+0000 7fe61b7fe640 1 --2- 172.21.15.38:0/3164777724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe61c15dc10 0x7fe61c0ce820 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009870 tx=0x7fe60c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270ad86dd5e3437f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:23.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.269+0000 7fe61a7fc640 1 -- 172.21.15.38:0/3164777724 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c037070 con 0x7fe61c15dc10 2024-04-25T08:40:23.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.269+0000 7fe61a7fc640 1 -- 172.21.15.38:0/3164777724 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c032440 con 0x7fe61c15dc10 2024-04-25T08:40:23.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.269+0000 7fe61a7fc640 1 -- 172.21.15.38:0/3164777724 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c032810 con 0x7fe61c15dc10 2024-04-25T08:40:23.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.269+0000 7fe621fb0640 1 -- 172.21.15.38:0/3164777724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe61c15dc10 msgr2=0x7fe61c0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:23.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.269+0000 7fe621fb0640 1 --2- 172.21.15.38:0/3164777724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe61c15dc10 0x7fe61c0ce820 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009870 tx=0x7fe60c02a210 comp rx=0 tx=0).stop 2024-04-25T08:40:23.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.269+0000 7fe621fb0640 1 -- 172.21.15.38:0/3164777724 shutdown_connections 2024-04-25T08:40:23.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.269+0000 7fe621fb0640 1 --2- 172.21.15.38:0/3164777724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe61c15dc10 0x7fe61c0ce820 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.269+0000 7fe621fb0640 1 -- 172.21.15.38:0/3164777724 >> 172.21.15.38:0/3164777724 conn(0x7fe61c066d10 msgr2=0x7fe61c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:23.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.269+0000 7fe621fb0640 1 -- 172.21.15.38:0/3164777724 shutdown_connections 2024-04-25T08:40:23.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.269+0000 7fe621fb0640 1 -- 172.21.15.38:0/3164777724 wait complete. 2024-04-25T08:40:23.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.271+0000 7fe621fb0640 1 Processor -- start 2024-04-25T08:40:23.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.271+0000 7fe621fb0640 1 -- start start 2024-04-25T08:40:23.267 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.271+0000 7fe621fb0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe61c0ced60 0x7fe61c157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:23.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe621fb0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61c0d0f60 0x7fe61c1f6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:23.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe621fb0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe61c15dc10 0x7fe61c1f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:23.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe621fb0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c0d4710 con 0x7fe61c0ced60 2024-04-25T08:40:23.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe621fb0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c0b80f0 con 0x7fe61c15dc10 2024-04-25T08:40:23.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe621fb0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c0b8270 con 0x7fe61c0d0f60 2024-04-25T08:40:23.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe61c0ced60 0x7fe61c157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:23.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe61c0ced60 0x7fe61c157680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35724/0 (socket says 172.21.15.38:35724) 2024-04-25T08:40:23.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61b7fe640 1 -- 172.21.15.38:0/3332611108 learned_addr learned my addr 172.21.15.38:0/3332611108 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:23.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61bfff640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe61c15dc10 0x7fe61c1f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:23.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61b7fe640 1 -- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61c0d0f60 msgr2=0x7fe61c1f6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:23.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61affd640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61c0d0f60 0x7fe61c1f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:23.268 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61b7fe640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61c0d0f60 0x7fe61c1f6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61b7fe640 1 -- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe61c15dc10 msgr2=0x7fe61c1f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61b7fe640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe61c15dc10 0x7fe61c1f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61b7fe640 1 -- 172.21.15.38:0/3332611108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c15a9d0 con 0x7fe61c0ced60 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61bfff640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe61c15dc10 0x7fe61c1f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61affd640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61c0d0f60 0x7fe61c1f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe61b7fe640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe61c0ced60 0x7fe61c157680 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009870 tx=0x7fe60c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe603fff640 1 -- 172.21.15.38:0/3332611108 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c037040 con 0x7fe61c0ced60 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe603fff640 1 -- 172.21.15.38:0/3332611108 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c004250 con 0x7fe61c0ced60 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe621fb0640 1 -- 172.21.15.38:0/3332611108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61c15a130 con 0x7fe61c0ced60 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe603fff640 1 -- 172.21.15.38:0/3332611108 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c03c670 con 0x7fe61c0ced60 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.272+0000 7fe621fb0640 1 -- 172.21.15.38:0/3332611108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61c15a6f0 con 0x7fe61c0ced60 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.274+0000 7fe621fb0640 1 -- 172.21.15.38:0/3332611108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61c068e80 con 0x7fe61c0ced60 2024-04-25T08:40:23.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.274+0000 7fe603fff640 1 -- 172.21.15.38:0/3332611108 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe60c03a070 con 0x7fe61c0ced60 2024-04-25T08:40:23.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.274+0000 7fe603fff640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe60806b4d0 0x7fe60806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:23.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.274+0000 7fe603fff640 1 -- 172.21.15.38:0/3332611108 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe60c0aaca0 con 0x7fe61c0ced60 2024-04-25T08:40:23.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.274+0000 7fe61affd640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe60806b4d0 0x7fe60806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:23.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.275+0000 7fe61affd640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe60806b4d0 0x7fe60806d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe61c1f6870 tx=0x7fe610009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:23.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.278+0000 7fe603fff640 1 -- 172.21.15.38:0/3332611108 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe60c041020 con 0x7fe61c0ced60 2024-04-25T08:40:23.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.500+0000 7fe621fb0640 1 -- 172.21.15.38:0/3332611108 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe61c0b4e60 con 0x7fe61c0ced60 2024-04-25T08:40:23.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.500+0000 7fe603fff640 1 -- 172.21.15.38:0/3332611108 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe60c079a20 con 0x7fe61c0ced60 2024-04-25T08:40:23.494 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:23.494 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:23.494 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:23.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.502+0000 7fe621fb0640 1 -- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe60806b4d0 msgr2=0x7fe60806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:23.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.502+0000 7fe621fb0640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe60806b4d0 0x7fe60806d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe61c1f6870 tx=0x7fe610009040 comp rx=0 tx=0).stop 2024-04-25T08:40:23.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.502+0000 7fe621fb0640 1 -- 172.21.15.38:0/3332611108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe61c0ced60 msgr2=0x7fe61c157680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:23.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.502+0000 7fe621fb0640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe61c0ced60 0x7fe61c157680 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7fe60c009870 tx=0x7fe60c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.502+0000 7fe621fb0640 1 -- 172.21.15.38:0/3332611108 shutdown_connections 2024-04-25T08:40:23.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.502+0000 7fe621fb0640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe61c15dc10 0x7fe61c1f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.502+0000 7fe621fb0640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe60806b4d0 0x7fe60806d990 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.502+0000 7fe621fb0640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe61c0d0f60 0x7fe61c1f6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.502+0000 7fe621fb0640 1 --2- 172.21.15.38:0/3332611108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe61c0ced60 0x7fe61c157680 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.502+0000 7fe621fb0640 1 -- 172.21.15.38:0/3332611108 >> 172.21.15.38:0/3332611108 conn(0x7fe61c066d10 msgr2=0x7fe61c15ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:23.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.502+0000 7fe621fb0640 1 -- 172.21.15.38:0/3332611108 shutdown_connections 2024-04-25T08:40:23.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.502+0000 7fe621fb0640 1 -- 172.21.15.38:0/3332611108 wait complete. 2024-04-25T08:40:23.506 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:40:23.506 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:40:23.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.613+0000 7fde18a4c640 1 Processor -- start 2024-04-25T08:40:23.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.613+0000 7fde18a4c640 1 -- start start 2024-04-25T08:40:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde18a4c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fde140cd8d0 0x7fde1406c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde18a4c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde1409acb0 0x7fde1406cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde18a4c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde14161540 con 0x7fde140b2170 2024-04-25T08:40:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde18a4c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1409a820 con 0x7fde140cd8d0 2024-04-25T08:40:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde18a4c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1406be90 con 0x7fde1409acb0 2024-04-25T08:40:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde1259c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fde140cd8d0 0x7fde1406c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde12d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fde140b2170 0x7fde140cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60454/0 (socket says 172.21.15.38:60454) 2024-04-25T08:40:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde12d9d640 1 -- 172.21.15.38:0/2421226967 learned_addr learned my addr 172.21.15.38:0/2421226967 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde09d9b640 1 --2- 172.21.15.38:0/2421226967 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde1409acb0 0x7fde1406cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde11d9b640 1 -- 172.21.15.38:0/2421226967 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1710571115 0 0) 0x7fde14161540 con 0x7fde140b2170 2024-04-25T08:40:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde11d9b640 1 -- 172.21.15.38:0/2421226967 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde00003520 con 0x7fde140b2170 2024-04-25T08:40:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde1259c640 1 -- 172.21.15.38:0/2421226967 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde1409acb0 msgr2=0x7fde1406cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:23.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde1259c640 1 --2- 172.21.15.38:0/2421226967 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde1409acb0 0x7fde1406cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde1259c640 1 -- 172.21.15.38:0/2421226967 >> v1:172.21.15.38:6789/0 conn(0x7fde140b2170 legacy=0x7fde140cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:23.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.614+0000 7fde1259c640 1 -- 172.21.15.38:0/2421226967 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1409a520 con 0x7fde140cd8d0 2024-04-25T08:40:23.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.615+0000 7fde1259c640 1 --2- 172.21.15.38:0/2421226967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde140cd8d0 0x7fde1406c6e0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fde040097a0 tx=0x7fde0402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a5855f351b339f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:23.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.615+0000 7fde11d9b640 1 -- 172.21.15.38:0/2421226967 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04037070 con 0x7fde140cd8d0 2024-04-25T08:40:23.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.615+0000 7fde11d9b640 1 -- 172.21.15.38:0/2421226967 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde04032440 con 0x7fde140cd8d0 2024-04-25T08:40:23.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.615+0000 7fde11d9b640 1 -- 172.21.15.38:0/2421226967 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04032810 con 0x7fde140cd8d0 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.615+0000 7fde18a4c640 1 -- 172.21.15.38:0/2421226967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde140cd8d0 msgr2=0x7fde1406c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.615+0000 7fde18a4c640 1 --2- 172.21.15.38:0/2421226967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde140cd8d0 0x7fde1406c6e0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fde040097a0 tx=0x7fde0402a180 comp rx=0 tx=0).stop 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.616+0000 7fde18a4c640 1 -- 172.21.15.38:0/2421226967 shutdown_connections 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.616+0000 7fde18a4c640 1 --2- 172.21.15.38:0/2421226967 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde1409acb0 0x7fde1406cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.616+0000 7fde18a4c640 1 --2- 172.21.15.38:0/2421226967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde140cd8d0 0x7fde1406c6e0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.616+0000 7fde18a4c640 1 -- 172.21.15.38:0/2421226967 >> 172.21.15.38:0/2421226967 conn(0x7fde140b3160 msgr2=0x7fde140b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.616+0000 7fde18a4c640 1 -- 172.21.15.38:0/2421226967 shutdown_connections 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.616+0000 7fde18a4c640 1 -- 172.21.15.38:0/2421226967 wait complete. 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.617+0000 7fde18a4c640 1 Processor -- start 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.617+0000 7fde18a4c640 1 -- start start 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde18a4c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1409acb0 0x7fde14158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde18a4c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde140b2170 0x7fde141f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde18a4c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde140cd8d0 0x7fde141f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde18a4c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1406c410 con 0x7fde140cd8d0 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde18a4c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde140d0ad0 con 0x7fde1409acb0 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde18a4c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde140b26c0 con 0x7fde140b2170 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde12d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde140cd8d0 0x7fde141f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde12d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde140cd8d0 0x7fde141f71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35738/0 (socket says 172.21.15.38:35738) 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde12d9d640 1 -- 172.21.15.38:0/2870305248 learned_addr learned my addr 172.21.15.38:0/2870305248 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde1259c640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1409acb0 0x7fde14158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde09d9b640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde140b2170 0x7fde141f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde12d9d640 1 -- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde140b2170 msgr2=0x7fde141f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde12d9d640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde140b2170 0x7fde141f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde12d9d640 1 -- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1409acb0 msgr2=0x7fde14158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde12d9d640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1409acb0 0x7fde14158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.618+0000 7fde12d9d640 1 -- 172.21.15.38:0/2870305248 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde140afb60 con 0x7fde140cd8d0 2024-04-25T08:40:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.619+0000 7fde12d9d640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde140cd8d0 0x7fde141f71a0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00cb40 tx=0x7fde0c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.619+0000 7fde0b7fe640 1 -- 172.21.15.38:0/2870305248 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c017070 con 0x7fde140cd8d0 2024-04-25T08:40:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.619+0000 7fde0b7fe640 1 -- 172.21.15.38:0/2870305248 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0c00aca0 con 0x7fde140cd8d0 2024-04-25T08:40:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.619+0000 7fde18a4c640 1 -- 172.21.15.38:0/2870305248 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde141597e0 con 0x7fde140cd8d0 2024-04-25T08:40:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.619+0000 7fde0b7fe640 1 -- 172.21.15.38:0/2870305248 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0c012620 con 0x7fde140cd8d0 2024-04-25T08:40:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.619+0000 7fde18a4c640 1 -- 172.21.15.38:0/2870305248 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde14159da0 con 0x7fde140cd8d0 2024-04-25T08:40:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.620+0000 7fde18a4c640 1 -- 172.21.15.38:0/2870305248 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde14068e80 con 0x7fde140cd8d0 2024-04-25T08:40:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.620+0000 7fde0b7fe640 1 -- 172.21.15.38:0/2870305248 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde0c01a070 con 0x7fde140cd8d0 2024-04-25T08:40:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.621+0000 7fde0b7fe640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fddf406b4d0 0x7fddf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.621+0000 7fde0b7fe640 1 -- 172.21.15.38:0/2870305248 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fde0c08a0e0 con 0x7fde140cd8d0 2024-04-25T08:40:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.621+0000 7fde1259c640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fddf406b4d0 0x7fddf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.622+0000 7fde1259c640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fddf406b4d0 0x7fddf406d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fde04034000 tx=0x7fde0402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:23.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.625+0000 7fde0b7fe640 1 -- 172.21.15.38:0/2870305248 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde0c058810 con 0x7fde140cd8d0 2024-04-25T08:40:23.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.873+0000 7fde18a4c640 1 -- 172.21.15.38:0/2870305248 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fde140cb5a0 con 0x7fde140cd8d0 2024-04-25T08:40:23.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.874+0000 7fde0b7fe640 1 -- 172.21.15.38:0/2870305248 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7fde0c05c8b0 con 0x7fde140cd8d0 2024-04-25T08:40:23.867 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:23.867 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":552,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":740,"data_bytes":752126305,"bytes_used":1947213824,"bytes_avail":771146899456,"bytes_total":773094113280,"write_bytes_sec":62620432,"read_op_per_sec":0,"write_op_per_sec":61},"fsmap":{"epoch":70,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":7639}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T08:39:39.318198+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:40:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.876+0000 7fde18a4c640 1 -- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fddf406b4d0 msgr2=0x7fddf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.876+0000 7fde18a4c640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fddf406b4d0 0x7fddf406d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fde04034000 tx=0x7fde0402e000 comp rx=0 tx=0).stop 2024-04-25T08:40:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.876+0000 7fde18a4c640 1 -- 172.21.15.38:0/2870305248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde140cd8d0 msgr2=0x7fde141f71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.876+0000 7fde18a4c640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde140cd8d0 0x7fde141f71a0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fde0c00cb40 tx=0x7fde0c00a060 comp rx=0 tx=0).stop 2024-04-25T08:40:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.876+0000 7fde18a4c640 1 -- 172.21.15.38:0/2870305248 shutdown_connections 2024-04-25T08:40:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.876+0000 7fde18a4c640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fde140cd8d0 0x7fde141f71a0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.876+0000 7fde18a4c640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fddf406b4d0 0x7fddf406d990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.876+0000 7fde18a4c640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fde140b2170 0x7fde141f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.876+0000 7fde18a4c640 1 --2- 172.21.15.38:0/2870305248 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fde1409acb0 0x7fde14158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.876+0000 7fde18a4c640 1 -- 172.21.15.38:0/2870305248 >> 172.21.15.38:0/2870305248 conn(0x7fde140b3160 msgr2=0x7fde1406fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.876+0000 7fde18a4c640 1 -- 172.21.15.38:0/2870305248 shutdown_connections 2024-04-25T08:40:23.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:23.876+0000 7fde18a4c640 1 -- 172.21.15.38:0/2870305248 wait complete. 2024-04-25T08:40:23.879 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:40:24.880 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:24.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.001+0000 7fadc2d53640 1 Processor -- start 2024-04-25T08:40:24.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.002+0000 7fadc2d53640 1 -- start start 2024-04-25T08:40:24.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.002+0000 7fadc2d53640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fadbc0cd8d0 0x7fadbc06c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:24.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.002+0000 7fadc2d53640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0b2170 0x7fadbc06cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:24.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.002+0000 7fadc2d53640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadbc161540 con 0x7fadbc09acb0 2024-04-25T08:40:24.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.002+0000 7fadc2d53640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc09a820 con 0x7fadbc0cd8d0 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.002+0000 7fadc2d53640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc06be90 con 0x7fadbc0b2170 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.002+0000 7fadbbfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fadbc09acb0 0x7fadbc0cf790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60462/0 (socket says 172.21.15.38:60462) 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbbfff640 1 -- 172.21.15.38:0/1590183948 learned_addr learned my addr 172.21.15.38:0/1590183948 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadc0aef640 1 --2- 172.21.15.38:0/1590183948 >> v2:172.21.15.107:3300/0 conn(0x7fadbc0cd8d0 0x7fadbc06c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadc12f0640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0b2170 0x7fadbc06cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247459087 0 0) 0x7fadbc161540 con 0x7fadbc09acb0 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fada8003520 con 0x7fadbc09acb0 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1852222918 0 0) 0x7fada8003520 con 0x7fadbc09acb0 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0b2170 msgr2=0x7fadbc06cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0b2170 0x7fadbc06cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 >> v2:172.21.15.107:3300/0 conn(0x7fadbc0cd8d0 msgr2=0x7fadbc06c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 --2- 172.21.15.38:0/1590183948 >> v2:172.21.15.107:3300/0 conn(0x7fadbc0cd8d0 0x7fadbc06c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc09a520 con 0x7fadbc09acb0 2024-04-25T08:40:24.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fadac001670 con 0x7fadbc09acb0 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadc12f0640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0b2170 0x7fadbc06cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadc0aef640 1 --2- 172.21.15.38:0/1590183948 >> v2:172.21.15.107:3300/0 conn(0x7fadbc0cd8d0 0x7fadbc06c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 >> v1:172.21.15.38:6789/0 conn(0x7fadbc09acb0 legacy=0x7fadbc0cf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada80039b0 0x7fada8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0b2170 0x7fada8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fada8004d40 0x7fada80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc09a820 con 0x7fada80039b0 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc06be90 con 0x7fada8004d40 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada800b9c0 con 0x7fadbc0b2170 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadc0aef640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada80039b0 0x7fada8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadbbfff640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0b2170 0x7fada8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.003+0000 7fadc12f0640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fada8004d40 0x7fada80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.004+0000 7fadc0aef640 1 -- 172.21.15.38:0/1590183948 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0b2170 msgr2=0x7fada8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:25.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.004+0000 7fadc0aef640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0b2170 0x7fada8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.004+0000 7fadc0aef640 1 -- 172.21.15.38:0/1590183948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fada8004d40 msgr2=0x7fada80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.004+0000 7fadc0aef640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fada8004d40 0x7fada80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.004+0000 7fadc0aef640 1 -- 172.21.15.38:0/1590183948 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada8003520 con 0x7fada80039b0 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.004+0000 7fadc0aef640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada80039b0 0x7fada8003dd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fadb400e350 tx=0x7fada8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68a2096d29e2d9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.004+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4009de0 con 0x7fada80039b0 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.005+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb402e660 con 0x7fada80039b0 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.005+0000 7fadbb7fe640 1 -- 172.21.15.38:0/1590183948 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb402ea10 con 0x7fada80039b0 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.005+0000 7fadc2d53640 1 -- 172.21.15.38:0/1590183948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada80039b0 msgr2=0x7fada8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.005+0000 7fadc2d53640 1 --2- 172.21.15.38:0/1590183948 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada80039b0 0x7fada8003dd0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fadb400e350 tx=0x7fada8005830 comp rx=0 tx=0).stop 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.005+0000 7fadc12f0640 1 -- 172.21.15.38:0/1590183948 reap_dead start 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.005+0000 7fadc2d53640 1 -- 172.21.15.38:0/1590183948 shutdown_connections 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.005+0000 7fadc2d53640 1 -- 172.21.15.38:0/1590183948 >> 172.21.15.38:0/1590183948 conn(0x7fadbc0b3160 msgr2=0x7fadbc0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.005+0000 7fadc2d53640 1 -- 172.21.15.38:0/1590183948 shutdown_connections 2024-04-25T08:40:25.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.005+0000 7fadc2d53640 1 -- 172.21.15.38:0/1590183948 wait complete. 2024-04-25T08:40:25.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.006+0000 7fadc2d53640 1 Processor -- start 2024-04-25T08:40:25.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.006+0000 7fadc2d53640 1 -- start start 2024-04-25T08:40:25.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.007+0000 7fadc2d53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada80039b0 0x7fadbc0c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:25.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.007+0000 7fadc2d53640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadbc0c75a0 0x7fadbc15a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:25.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.007+0000 7fadc2d53640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0c51a0 0x7fadbc0c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:25.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.007+0000 7fadc2d53640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc06c410 con 0x7fada80039b0 2024-04-25T08:40:25.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.007+0000 7fadc2d53640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0d0ad0 con 0x7fadbc0c75a0 2024-04-25T08:40:25.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.007+0000 7fadc2d53640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0b26c0 con 0x7fadbc0c51a0 2024-04-25T08:40:25.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.007+0000 7fadc0aef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada80039b0 0x7fadbc0c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.007+0000 7fadc0aef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada80039b0 0x7fadbc0c7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35748/0 (socket says 172.21.15.38:35748) 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.007+0000 7fadc0aef640 1 -- 172.21.15.38:0/2205814609 learned_addr learned my addr 172.21.15.38:0/2205814609 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadc12f0640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0c51a0 0x7fadbc0c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadc0aef640 1 -- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0c51a0 msgr2=0x7fadbc0c5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadbbfff640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadbc0c75a0 0x7fadbc15a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadc0aef640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0c51a0 0x7fadbc0c5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadc0aef640 1 -- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadbc0c75a0 msgr2=0x7fadbc15a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadc0aef640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadbc0c75a0 0x7fadbc15a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadc0aef640 1 -- 172.21.15.38:0/2205814609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc0c5b60 con 0x7fada80039b0 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadc0aef640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada80039b0 0x7fadbc0c7b10 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fadb400e440 tx=0x7fadb4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadb97fa640 1 -- 172.21.15.38:0/2205814609 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4009de0 con 0x7fada80039b0 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadb97fa640 1 -- 172.21.15.38:0/2205814609 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb402e460 con 0x7fada80039b0 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadc2d53640 1 -- 172.21.15.38:0/2205814609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc0c8050 con 0x7fada80039b0 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadb97fa640 1 -- 172.21.15.38:0/2205814609 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4036c90 con 0x7fada80039b0 2024-04-25T08:40:25.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.008+0000 7fadc2d53640 1 -- 172.21.15.38:0/2205814609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc0c5e00 con 0x7fada80039b0 2024-04-25T08:40:25.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.009+0000 7fadc2d53640 1 -- 172.21.15.38:0/2205814609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc068e80 con 0x7fada80039b0 2024-04-25T08:40:25.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.010+0000 7fadb97fa640 1 -- 172.21.15.38:0/2205814609 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadb4048030 con 0x7fada80039b0 2024-04-25T08:40:25.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.010+0000 7fadb97fa640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad9c06b500 0x7fad9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:25.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.010+0000 7fadb97fa640 1 -- 172.21.15.38:0/2205814609 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fadb40ab630 con 0x7fada80039b0 2024-04-25T08:40:25.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.010+0000 7fadbbfff640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad9c06b500 0x7fad9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:25.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.011+0000 7fadbbfff640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad9c06b500 0x7fad9c06d9c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fadac001050 tx=0x7fadac000fe0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:25.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.014+0000 7fadb97fa640 1 -- 172.21.15.38:0/2205814609 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fadb404f060 con 0x7fada80039b0 2024-04-25T08:40:25.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.232+0000 7fadc2d53640 1 -- 172.21.15.38:0/2205814609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadbc099ca0 con 0x7fada80039b0 2024-04-25T08:40:25.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.232+0000 7fadb97fa640 1 -- 172.21.15.38:0/2205814609 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fadb4032030 con 0x7fada80039b0 2024-04-25T08:40:25.226 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:25.226 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:25.226 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:25.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.234+0000 7fadc2d53640 1 -- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad9c06b500 msgr2=0x7fad9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.234+0000 7fadc2d53640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad9c06b500 0x7fad9c06d9c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fadac001050 tx=0x7fadac000fe0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.234+0000 7fadc2d53640 1 -- 172.21.15.38:0/2205814609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada80039b0 msgr2=0x7fadbc0c7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.234+0000 7fadc2d53640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada80039b0 0x7fadbc0c7b10 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fadb400e440 tx=0x7fadb4009750 comp rx=0 tx=0).stop 2024-04-25T08:40:25.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.234+0000 7fadc2d53640 1 -- 172.21.15.38:0/2205814609 shutdown_connections 2024-04-25T08:40:25.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.234+0000 7fadc2d53640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad9c06b500 0x7fad9c06d9c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.234+0000 7fadc2d53640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fadbc0c51a0 0x7fadbc0c5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.234+0000 7fadc2d53640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadbc0c75a0 0x7fadbc15a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.234+0000 7fadc2d53640 1 --2- 172.21.15.38:0/2205814609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fada80039b0 0x7fadbc0c7b10 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.235+0000 7fadc2d53640 1 -- 172.21.15.38:0/2205814609 >> 172.21.15.38:0/2205814609 conn(0x7fadbc0b3160 msgr2=0x7fadbc0cf750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:25.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.235+0000 7fadc2d53640 1 -- 172.21.15.38:0/2205814609 shutdown_connections 2024-04-25T08:40:25.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.235+0000 7fadc2d53640 1 -- 172.21.15.38:0/2205814609 wait complete. 2024-04-25T08:40:25.238 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:40:25.238 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:40:25.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.346+0000 7f3a510b3640 1 Processor -- start 2024-04-25T08:40:25.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.346+0000 7f3a510b3640 1 -- start start 2024-04-25T08:40:25.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.347+0000 7f3a510b3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3a4c06c110 0x7f3a4c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:25.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.347+0000 7f3a510b3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4c06ca30 con 0x7f3a4c15df50 2024-04-25T08:40:25.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.347+0000 7f3a510b3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4c0ceff0 con 0x7f3a4c0ce2e0 2024-04-25T08:40:25.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.347+0000 7f3a510b3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a4c0cf2d0 con 0x7f3a4c06c110 2024-04-25T08:40:25.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.347+0000 7f3a4ad9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3a4c06c110 0x7f3a4c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:25.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.347+0000 7f3a43fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f3a4c15df50 0x7f3a4c0ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60476/0 (socket says 172.21.15.38:60476) 2024-04-25T08:40:25.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.347+0000 7f3a43fff640 1 -- 172.21.15.38:0/4091141806 learned_addr learned my addr 172.21.15.38:0/4091141806 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:25.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.347+0000 7f3a4ad9d640 1 -- 172.21.15.38:0/4091141806 >> v1:172.21.15.107:6789/0 conn(0x7f3a4c0ce2e0 legacy=0x7f3a4c0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.347+0000 7f3a4ad9d640 1 -- 172.21.15.38:0/4091141806 >> v1:172.21.15.38:6789/0 conn(0x7f3a4c15df50 legacy=0x7f3a4c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.347+0000 7f3a4a59c640 1 -- 172.21.15.38:0/4091141806 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630937448 0 0) 0x7f3a4c06ca30 con 0x7f3a4c15df50 2024-04-25T08:40:25.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.347+0000 7f3a4ad9d640 1 -- 172.21.15.38:0/4091141806 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c0d1f70 con 0x7f3a4c06c110 2024-04-25T08:40:25.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.348+0000 7f3a4ad9d640 1 --2- 172.21.15.38:0/4091141806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a4c06c110 0x7f3a4c06c4f0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0097a0 tx=0x7f3a3c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2977ab83b95c8c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:25.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.348+0000 7f3a4a59c640 1 -- 172.21.15.38:0/4091141806 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c037070 con 0x7f3a4c06c110 2024-04-25T08:40:25.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.348+0000 7f3a4a59c640 1 -- 172.21.15.38:0/4091141806 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c032440 con 0x7f3a4c06c110 2024-04-25T08:40:25.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.348+0000 7f3a4a59c640 1 -- 172.21.15.38:0/4091141806 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c032810 con 0x7f3a4c06c110 2024-04-25T08:40:25.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.349+0000 7f3a510b3640 1 -- 172.21.15.38:0/4091141806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a4c06c110 msgr2=0x7f3a4c06c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.349+0000 7f3a510b3640 1 --2- 172.21.15.38:0/4091141806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a4c06c110 0x7f3a4c06c4f0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0097a0 tx=0x7f3a3c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.349+0000 7f3a510b3640 1 -- 172.21.15.38:0/4091141806 shutdown_connections 2024-04-25T08:40:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.349+0000 7f3a510b3640 1 --2- 172.21.15.38:0/4091141806 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a4c06c110 0x7f3a4c06c4f0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.349+0000 7f3a510b3640 1 -- 172.21.15.38:0/4091141806 >> 172.21.15.38:0/4091141806 conn(0x7f3a4c0d3fb0 msgr2=0x7f3a4c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.349+0000 7f3a510b3640 1 -- 172.21.15.38:0/4091141806 shutdown_connections 2024-04-25T08:40:25.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.349+0000 7f3a510b3640 1 -- 172.21.15.38:0/4091141806 wait complete. 2024-04-25T08:40:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.350+0000 7f3a510b3640 1 Processor -- start 2024-04-25T08:40:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.350+0000 7f3a510b3640 1 -- start start 2024-04-25T08:40:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a510b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a4c06c110 0x7f3a4c15ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a510b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a4c0ce2e0 0x7f3a4c160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a510b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a4c15df50 0x7f3a4c158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a510b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0d3160 con 0x7f3a4c15df50 2024-04-25T08:40:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a510b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0d2530 con 0x7f3a4c06c110 2024-04-25T08:40:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a510b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4c0d32e0 con 0x7f3a4c0ce2e0 2024-04-25T08:40:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a4b59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a4c15df50 0x7f3a4c158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:25.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a4b59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a4c15df50 0x7f3a4c158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35756/0 (socket says 172.21.15.38:35756) 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a4b59e640 1 -- 172.21.15.38:0/909153413 learned_addr learned my addr 172.21.15.38:0/909153413 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a4ad9d640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a4c06c110 0x7f3a4c15ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a43fff640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a4c0ce2e0 0x7f3a4c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a4b59e640 1 -- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a4c0ce2e0 msgr2=0x7f3a4c160820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a4b59e640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a4c0ce2e0 0x7f3a4c160820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a4b59e640 1 -- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a4c06c110 msgr2=0x7f3a4c15ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a4b59e640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a4c06c110 0x7f3a4c15ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a4b59e640 1 -- 172.21.15.38:0/909153413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c15f570 con 0x7f3a4c15df50 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a4ad9d640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a4c06c110 0x7f3a4c15ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a43fff640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a4c0ce2e0 0x7f3a4c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.351+0000 7f3a4b59e640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a4c15df50 0x7f3a4c158580 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400c7a0 tx=0x7f3a4400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.352+0000 7f3a437fe640 1 -- 172.21.15.38:0/909153413 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a44003b70 con 0x7f3a4c15df50 2024-04-25T08:40:25.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.352+0000 7f3a437fe640 1 -- 172.21.15.38:0/909153413 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a44003d30 con 0x7f3a4c15df50 2024-04-25T08:40:25.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.352+0000 7f3a510b3640 1 -- 172.21.15.38:0/909153413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4c15fa60 con 0x7f3a4c15df50 2024-04-25T08:40:25.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.352+0000 7f3a437fe640 1 -- 172.21.15.38:0/909153413 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400aa90 con 0x7f3a4c15df50 2024-04-25T08:40:25.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.352+0000 7f3a510b3640 1 -- 172.21.15.38:0/909153413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4c160300 con 0x7f3a4c15df50 2024-04-25T08:40:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.353+0000 7f3a510b3640 1 -- 172.21.15.38:0/909153413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4c15fe50 con 0x7f3a4c15df50 2024-04-25T08:40:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.353+0000 7f3a437fe640 1 -- 172.21.15.38:0/909153413 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a4400ac30 con 0x7f3a4c15df50 2024-04-25T08:40:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.354+0000 7f3a437fe640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a2c06b4d0 0x7f3a2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.354+0000 7f3a437fe640 1 -- 172.21.15.38:0/909153413 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a4408a2e0 con 0x7f3a4c15df50 2024-04-25T08:40:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.354+0000 7f3a4ad9d640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a2c06b4d0 0x7f3a2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:25.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.354+0000 7f3a4ad9d640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a2c06b4d0 0x7f3a2c06d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c034000 tx=0x7f3a3c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:25.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.358+0000 7f3a437fe640 1 -- 172.21.15.38:0/909153413 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a44058fd0 con 0x7f3a4c15df50 2024-04-25T08:40:25.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.607+0000 7f3a510b3640 1 -- 172.21.15.38:0/909153413 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3a4c160500 con 0x7f3a4c15df50 2024-04-25T08:40:25.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.608+0000 7f3a437fe640 1 -- 172.21.15.38:0/909153413 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f3a440588d0 con 0x7f3a4c15df50 2024-04-25T08:40:25.602 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:25.602 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":554,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076550540,"bytes_used":2672148480,"bytes_avail":770421964800,"bytes_total":773094113280,"write_bytes_sec":89655447,"read_op_per_sec":0,"write_op_per_sec":89},"fsmap":{"epoch":70,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":7639}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T08:39:39.318198+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:40:25.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.610+0000 7f3a510b3640 1 -- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a2c06b4d0 msgr2=0x7f3a2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.610+0000 7f3a510b3640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a2c06b4d0 0x7f3a2c06d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c034000 tx=0x7f3a3c02e000 comp rx=0 tx=0).stop 2024-04-25T08:40:25.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.610+0000 7f3a510b3640 1 -- 172.21.15.38:0/909153413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a4c15df50 msgr2=0x7f3a4c158580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:25.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.610+0000 7f3a510b3640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a4c15df50 0x7f3a4c158580 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400c7a0 tx=0x7f3a4400cc60 comp rx=0 tx=0).stop 2024-04-25T08:40:25.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.610+0000 7f3a510b3640 1 -- 172.21.15.38:0/909153413 shutdown_connections 2024-04-25T08:40:25.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.610+0000 7f3a510b3640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3a4c15df50 0x7f3a4c158580 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.610+0000 7f3a510b3640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3a2c06b4d0 0x7f3a2c06d990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.610+0000 7f3a510b3640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a4c0ce2e0 0x7f3a4c160820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.610+0000 7f3a510b3640 1 --2- 172.21.15.38:0/909153413 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a4c06c110 0x7f3a4c15ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:25.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.610+0000 7f3a510b3640 1 -- 172.21.15.38:0/909153413 >> 172.21.15.38:0/909153413 conn(0x7f3a4c0d3fb0 msgr2=0x7f3a4c15d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:25.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.610+0000 7f3a510b3640 1 -- 172.21.15.38:0/909153413 shutdown_connections 2024-04-25T08:40:25.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:25.610+0000 7f3a510b3640 1 -- 172.21.15.38:0/909153413 wait complete. 2024-04-25T08:40:25.615 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:40:26.616 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:26.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.736+0000 7f3152d34640 1 Processor -- start 2024-04-25T08:40:26.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.736+0000 7f3152d34640 1 -- start start 2024-04-25T08:40:26.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f3152d34640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f314c15dc10 0x7f314c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f3152d34640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f314c0ced60 0x7f314c0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f3152d34640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314c0d3e70 con 0x7f314c0d3900 2024-04-25T08:40:26.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f3152d34640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314c15dff0 con 0x7f314c15dc10 2024-04-25T08:40:26.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f3152d34640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0b6990 con 0x7f314c0ced60 2024-04-25T08:40:26.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f314bfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f314c0ced60 0x7f314c0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f3150ad0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f314c15dc10 0x7f314c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f3150ad0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f314c15dc10 0x7f314c0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:59552/0 (socket says 172.21.15.38:59552) 2024-04-25T08:40:26.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f31512d1640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f314c0d3900 0x7f314c0b2370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:40870/0 (socket says 172.21.15.38:40870) 2024-04-25T08:40:26.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f3150ad0640 1 -- 172.21.15.38:0/2356152671 learned_addr learned my addr 172.21.15.38:0/2356152671 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:26.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f314bfff640 1 -- 172.21.15.38:0/2356152671 >> v2:172.21.15.107:3300/0 conn(0x7f314c15dc10 msgr2=0x7f314c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:26.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f314bfff640 1 --2- 172.21.15.38:0/2356152671 >> v2:172.21.15.107:3300/0 conn(0x7f314c15dc10 0x7f314c0ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f314bfff640 1 -- 172.21.15.38:0/2356152671 >> v1:172.21.15.107:6789/0 conn(0x7f314c0d3900 legacy=0x7f314c0b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f314bfff640 1 -- 172.21.15.38:0/2356152671 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c0d4150 con 0x7f314c0ced60 2024-04-25T08:40:26.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.737+0000 7f3150ad0640 1 --2- 172.21.15.38:0/2356152671 >> v2:172.21.15.107:3300/0 conn(0x7f314c15dc10 0x7f314c0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:26.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.738+0000 7f314bfff640 1 --2- 172.21.15.38:0/2356152671 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f314c0ced60 0x7f314c0d0850 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f31340097a0 tx=0x7f313402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5b3f7863db27a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:26.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.738+0000 7f314b7fe640 1 -- 172.21.15.38:0/2356152671 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134037070 con 0x7f314c0ced60 2024-04-25T08:40:26.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.738+0000 7f314b7fe640 1 -- 172.21.15.38:0/2356152671 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313402ae50 con 0x7f314c0ced60 2024-04-25T08:40:26.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.738+0000 7f314b7fe640 1 -- 172.21.15.38:0/2356152671 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134032660 con 0x7f314c0ced60 2024-04-25T08:40:26.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.739+0000 7f3152d34640 1 -- 172.21.15.38:0/2356152671 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f314c0ced60 msgr2=0x7f314c0d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.739+0000 7f3152d34640 1 --2- 172.21.15.38:0/2356152671 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f314c0ced60 0x7f314c0d0850 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f31340097a0 tx=0x7f313402a100 comp rx=0 tx=0).stop 2024-04-25T08:40:26.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.739+0000 7f3152d34640 1 -- 172.21.15.38:0/2356152671 shutdown_connections 2024-04-25T08:40:26.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.739+0000 7f3152d34640 1 --2- 172.21.15.38:0/2356152671 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f314c0ced60 0x7f314c0d0850 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.739+0000 7f3152d34640 1 --2- 172.21.15.38:0/2356152671 >> v2:172.21.15.107:3300/0 conn(0x7f314c15dc10 0x7f314c0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.739+0000 7f3152d34640 1 -- 172.21.15.38:0/2356152671 >> 172.21.15.38:0/2356152671 conn(0x7f314c066d10 msgr2=0x7f314c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:26.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.739+0000 7f3152d34640 1 -- 172.21.15.38:0/2356152671 shutdown_connections 2024-04-25T08:40:26.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.739+0000 7f3152d34640 1 -- 172.21.15.38:0/2356152671 wait complete. 2024-04-25T08:40:26.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.740+0000 7f3152d34640 1 Processor -- start 2024-04-25T08:40:26.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.740+0000 7f3152d34640 1 -- start start 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.741+0000 7f3152d34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c0ced60 0x7f314c1f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.741+0000 7f3152d34640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f314c0d3900 0x7f314c159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.741+0000 7f3152d34640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f314c15dc10 0x7f314c0aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.741+0000 7f3152d34640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0b6bc0 con 0x7f314c0ced60 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.741+0000 7f3152d34640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0d4710 con 0x7f314c0d3900 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.741+0000 7f3152d34640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f314c0d0d90 con 0x7f314c15dc10 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.741+0000 7f3150ad0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c0ced60 0x7f314c1f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.741+0000 7f3150ad0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c0ced60 0x7f314c1f7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35768/0 (socket says 172.21.15.38:35768) 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.741+0000 7f3150ad0640 1 -- 172.21.15.38:0/3817288096 learned_addr learned my addr 172.21.15.38:0/3817288096 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f31512d1640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f314c15dc10 0x7f314c0aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f314bfff640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f314c0d3900 0x7f314c159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f3150ad0640 1 -- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f314c15dc10 msgr2=0x7f314c0aa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f3150ad0640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f314c15dc10 0x7f314c0aa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f3150ad0640 1 -- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f314c0d3900 msgr2=0x7f314c159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f3150ad0640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f314c0d3900 0x7f314c159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f3150ad0640 1 -- 172.21.15.38:0/3817288096 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c0aa830 con 0x7f314c0ced60 2024-04-25T08:40:26.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f3150ad0640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c0ced60 0x7f314c1f7560 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f3144009e80 tx=0x7f314400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:26.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f31497fa640 1 -- 172.21.15.38:0/3817288096 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314400ce60 con 0x7f314c0ced60 2024-04-25T08:40:26.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f31497fa640 1 -- 172.21.15.38:0/3817288096 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3144019070 con 0x7f314c0ced60 2024-04-25T08:40:26.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f3152d34640 1 -- 172.21.15.38:0/3817288096 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314c0aab00 con 0x7f314c0ced60 2024-04-25T08:40:26.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f31497fa640 1 -- 172.21.15.38:0/3817288096 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3144014680 con 0x7f314c0ced60 2024-04-25T08:40:26.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.742+0000 7f3152d34640 1 -- 172.21.15.38:0/3817288096 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314c0ab0c0 con 0x7f314c0ced60 2024-04-25T08:40:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.743+0000 7f3152d34640 1 -- 172.21.15.38:0/3817288096 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f314c068e80 con 0x7f314c0ced60 2024-04-25T08:40:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.743+0000 7f31497fa640 1 -- 172.21.15.38:0/3817288096 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3144005d10 con 0x7f314c0ced60 2024-04-25T08:40:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.744+0000 7f31497fa640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f312c06b4d0 0x7f312c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.744+0000 7f31497fa640 1 -- 172.21.15.38:0/3817288096 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f314408a7b0 con 0x7f314c0ced60 2024-04-25T08:40:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.744+0000 7f314bfff640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f312c06b4d0 0x7f312c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:26.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.745+0000 7f314bfff640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f312c06b4d0 0x7f312c06d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3134034000 tx=0x7f313402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:26.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.748+0000 7f31497fa640 1 -- 172.21.15.38:0/3817288096 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3144010040 con 0x7f314c0ced60 2024-04-25T08:40:26.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.965+0000 7f3152d34640 1 -- 172.21.15.38:0/3817288096 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f314c0d0a10 con 0x7f314c0ced60 2024-04-25T08:40:26.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.965+0000 7f31497fa640 1 -- 172.21.15.38:0/3817288096 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3144059530 con 0x7f314c0ced60 2024-04-25T08:40:26.960 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:26.960 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:26.960 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:26.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.967+0000 7f3152d34640 1 -- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f312c06b4d0 msgr2=0x7f312c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.967+0000 7f3152d34640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f312c06b4d0 0x7f312c06d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f3134034000 tx=0x7f313402e000 comp rx=0 tx=0).stop 2024-04-25T08:40:26.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.967+0000 7f3152d34640 1 -- 172.21.15.38:0/3817288096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c0ced60 msgr2=0x7f314c1f7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:26.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.967+0000 7f3152d34640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c0ced60 0x7f314c1f7560 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f3144009e80 tx=0x7f314400c490 comp rx=0 tx=0).stop 2024-04-25T08:40:26.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.967+0000 7f3152d34640 1 -- 172.21.15.38:0/3817288096 shutdown_connections 2024-04-25T08:40:26.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.967+0000 7f3152d34640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f312c06b4d0 0x7f312c06d990 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.967+0000 7f3152d34640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f314c15dc10 0x7f314c0aa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.967+0000 7f3152d34640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f314c0d3900 0x7f314c159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.967+0000 7f3152d34640 1 --2- 172.21.15.38:0/3817288096 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f314c0ced60 0x7f314c1f7560 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:26.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.967+0000 7f3152d34640 1 -- 172.21.15.38:0/3817288096 >> 172.21.15.38:0/3817288096 conn(0x7f314c066d10 msgr2=0x7f314c15e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:26.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.967+0000 7f3152d34640 1 -- 172.21.15.38:0/3817288096 shutdown_connections 2024-04-25T08:40:26.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:26.967+0000 7f3152d34640 1 -- 172.21.15.38:0/3817288096 wait complete. 2024-04-25T08:40:26.972 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:40:26.972 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:40:27.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.078+0000 7f15b357e640 1 Processor -- start 2024-04-25T08:40:27.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.078+0000 7f15b357e640 1 -- start start 2024-04-25T08:40:27.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.079+0000 7f15b357e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15ac0ceca0 0x7f15ac0d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.079+0000 7f15b357e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f15ac0d3de0 0x7f15ac0b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.079+0000 7f15b357e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15ac0d1b60 con 0x7f15ac15da90 2024-04-25T08:40:27.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.079+0000 7f15b357e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15ac0d2bc0 con 0x7f15ac0d3de0 2024-04-25T08:40:27.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.079+0000 7f15b357e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac0d3680 con 0x7f15ac0ceca0 2024-04-25T08:40:27.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.079+0000 7f15b1b1b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f15ac0d3de0 0x7f15ac0b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.079+0000 7f15b1b1b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f15ac0d3de0 0x7f15ac0b69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35778/0 (socket says 172.21.15.38:35778) 2024-04-25T08:40:27.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.079+0000 7f15b1b1b640 1 -- 172.21.15.38:0/427815715 learned_addr learned my addr 172.21.15.38:0/427815715 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.080+0000 7f15b0b19640 1 --2- 172.21.15.38:0/427815715 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15ac0ceca0 0x7f15ac0d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.080+0000 7f15b1b1b640 1 -- 172.21.15.38:0/427815715 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15ac0ceca0 msgr2=0x7f15ac0d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.080+0000 7f15b1b1b640 1 --2- 172.21.15.38:0/427815715 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15ac0ceca0 0x7f15ac0d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.080+0000 7f15b1b1b640 1 -- 172.21.15.38:0/427815715 >> v1:172.21.15.107:6789/0 conn(0x7f15ac15da90 legacy=0x7f15ac0d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.080+0000 7f15b1b1b640 1 -- 172.21.15.38:0/427815715 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac0d1e00 con 0x7f15ac0d3de0 2024-04-25T08:40:27.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.080+0000 7f15b1b1b640 1 --2- 172.21.15.38:0/427815715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac0d3de0 0x7f15ac0b69a0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0b2b60 tx=0x7f15a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77dc66d66e5740b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.081+0000 7f15a3fff640 1 -- 172.21.15.38:0/427815715 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8036070 con 0x7f15ac0d3de0 2024-04-25T08:40:27.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.081+0000 7f15a3fff640 1 -- 172.21.15.38:0/427815715 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a8031600 con 0x7f15ac0d3de0 2024-04-25T08:40:27.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.081+0000 7f15a3fff640 1 -- 172.21.15.38:0/427815715 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a80319d0 con 0x7f15ac0d3de0 2024-04-25T08:40:27.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.081+0000 7f15b357e640 1 -- 172.21.15.38:0/427815715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac0d3de0 msgr2=0x7f15ac0b69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.081+0000 7f15b357e640 1 --2- 172.21.15.38:0/427815715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac0d3de0 0x7f15ac0b69a0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0b2b60 tx=0x7f15a802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.081+0000 7f15b357e640 1 -- 172.21.15.38:0/427815715 shutdown_connections 2024-04-25T08:40:27.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.081+0000 7f15b357e640 1 --2- 172.21.15.38:0/427815715 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac0d3de0 0x7f15ac0b69a0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.081+0000 7f15b357e640 1 --2- 172.21.15.38:0/427815715 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15ac0ceca0 0x7f15ac0d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.081+0000 7f15b357e640 1 -- 172.21.15.38:0/427815715 >> 172.21.15.38:0/427815715 conn(0x7f15ac06bed0 msgr2=0x7f15ac06c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.081+0000 7f15b357e640 1 -- 172.21.15.38:0/427815715 shutdown_connections 2024-04-25T08:40:27.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.081+0000 7f15b357e640 1 -- 172.21.15.38:0/427815715 wait complete. 2024-04-25T08:40:27.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.082+0000 7f15b357e640 1 Processor -- start 2024-04-25T08:40:27.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.083+0000 7f15b357e640 1 -- start start 2024-04-25T08:40:27.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.083+0000 7f15b357e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15ac0d3de0 0x7f15ac152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.083+0000 7f15b357e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac15da90 0x7f15ac156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.083+0000 7f15b357e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15ac156840 0x7f15ac0ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.083+0000 7f15b357e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac15de70 con 0x7f15ac15da90 2024-04-25T08:40:27.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.083+0000 7f15b357e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac0d23c0 con 0x7f15ac156840 2024-04-25T08:40:27.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.083+0000 7f15b357e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac15dff0 con 0x7f15ac0d3de0 2024-04-25T08:40:27.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.083+0000 7f15b0b19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac15da90 0x7f15ac156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.083+0000 7f15b0b19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac15da90 0x7f15ac156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35784/0 (socket says 172.21.15.38:35784) 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.083+0000 7f15b0b19640 1 -- 172.21.15.38:0/799348420 learned_addr learned my addr 172.21.15.38:0/799348420 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.084+0000 7f15b1b1b640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15ac156840 0x7f15ac0ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.084+0000 7f15b0b19640 1 -- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15ac0d3de0 msgr2=0x7f15ac152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.084+0000 7f15b0b19640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15ac0d3de0 0x7f15ac152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.084+0000 7f15b0b19640 1 -- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15ac156840 msgr2=0x7f15ac0ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.084+0000 7f15b0b19640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15ac156840 0x7f15ac0ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.084+0000 7f15b0b19640 1 -- 172.21.15.38:0/799348420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac1f3da0 con 0x7f15ac15da90 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.084+0000 7f15b0b19640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac15da90 0x7f15ac156300 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f159c00c7f0 tx=0x7f159c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.084+0000 7f15a1ffb640 1 -- 172.21.15.38:0/799348420 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c015070 con 0x7f15ac15da90 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.084+0000 7f15a1ffb640 1 -- 172.21.15.38:0/799348420 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c004240 con 0x7f15ac15da90 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.084+0000 7f15a1ffb640 1 -- 172.21.15.38:0/799348420 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c007620 con 0x7f15ac15da90 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.084+0000 7f15b357e640 1 -- 172.21.15.38:0/799348420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac1f4080 con 0x7f15ac15da90 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.084+0000 7f15b357e640 1 -- 172.21.15.38:0/799348420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ac1f4640 con 0x7f15ac15da90 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.085+0000 7f15b357e640 1 -- 172.21.15.38:0/799348420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ac068e80 con 0x7f15ac15da90 2024-04-25T08:40:27.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.085+0000 7f15a1ffb640 1 -- 172.21.15.38:0/799348420 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f159c0079d0 con 0x7f15ac15da90 2024-04-25T08:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.086+0000 7f15a1ffb640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f159006b4d0 0x7f159006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.086+0000 7f15a1ffb640 1 -- 172.21.15.38:0/799348420 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f159c08a820 con 0x7f15ac15da90 2024-04-25T08:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.086+0000 7f15b131a640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f159006b4d0 0x7f159006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.087+0000 7f15b131a640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f159006b4d0 0x7f159006d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f15a4006fd0 tx=0x7f15a4006bc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.090+0000 7f15a1ffb640 1 -- 172.21.15.38:0/799348420 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f159c059500 con 0x7f15ac15da90 2024-04-25T08:40:27.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.345+0000 7f15b357e640 1 -- 172.21.15.38:0/799348420 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f15ac0b39f0 con 0x7f15ac15da90 2024-04-25T08:40:27.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.346+0000 7f15a1ffb640 1 -- 172.21.15.38:0/799348420 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f159c058e00 con 0x7f15ac15da90 2024-04-25T08:40:27.340 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:27.340 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":556,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076550540,"bytes_used":2672148480,"bytes_avail":770421964800,"bytes_total":773094113280,"write_bytes_sec":63618643,"read_op_per_sec":0,"write_op_per_sec":63},"fsmap":{"epoch":70,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":7639}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:40:25.328586+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T08:40:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.347+0000 7f15b357e640 1 -- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f159006b4d0 msgr2=0x7f159006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.347+0000 7f15b357e640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f159006b4d0 0x7f159006d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f15a4006fd0 tx=0x7f15a4006bc0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.347+0000 7f15b357e640 1 -- 172.21.15.38:0/799348420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac15da90 msgr2=0x7f15ac156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.347+0000 7f15b357e640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac15da90 0x7f15ac156300 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f159c00c7f0 tx=0x7f159c00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.348+0000 7f15b357e640 1 -- 172.21.15.38:0/799348420 shutdown_connections 2024-04-25T08:40:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.348+0000 7f15b357e640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15ac156840 0x7f15ac0ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.348+0000 7f15b357e640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15ac15da90 0x7f15ac156300 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.348+0000 7f15b357e640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f159006b4d0 0x7f159006d990 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.348+0000 7f15b357e640 1 --2- 172.21.15.38:0/799348420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f15ac0d3de0 0x7f15ac152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.348+0000 7f15b357e640 1 -- 172.21.15.38:0/799348420 >> 172.21.15.38:0/799348420 conn(0x7f15ac06bed0 msgr2=0x7f15ac0b8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.348+0000 7f15b357e640 1 -- 172.21.15.38:0/799348420 shutdown_connections 2024-04-25T08:40:27.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.348+0000 7f15b357e640 1 -- 172.21.15.38:0/799348420 wait complete. 2024-04-25T08:40:27.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T08:40:27.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:27.352 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-25T08:40:27.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.461+0000 7f8f4724d640 1 Processor -- start 2024-04-25T08:40:27.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.461+0000 7f8f4724d640 1 -- start start 2024-04-25T08:40:27.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.462+0000 7f8f4724d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8f400d0320 0x7f8f400d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.462+0000 7f8f4724d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f400d0700 0x7f8f400d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.462+0000 7f8f4724d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f400d2330 con 0x7f8f400d1e90 2024-04-25T08:40:27.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.462+0000 7f8f4724d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f400d2610 con 0x7f8f400d0320 2024-04-25T08:40:27.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.462+0000 7f8f4724d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f400d28f0 con 0x7f8f400d0700 2024-04-25T08:40:27.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.462+0000 7f8f44fe9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8f400d0320 0x7f8f400d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.462+0000 7f8f44fe9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8f400d0320 0x7f8f400d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35800/0 (socket says 172.21.15.38:35800) 2024-04-25T08:40:27.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.462+0000 7f8f44fe9640 1 -- 172.21.15.38:0/1630465455 learned_addr learned my addr 172.21.15.38:0/1630465455 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.462+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/1630465455 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f400d0700 0x7f8f400d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.462+0000 7f8f44fe9640 1 -- 172.21.15.38:0/1630465455 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f400d0700 msgr2=0x7f8f400d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.462+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/1630465455 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f400d0700 0x7f8f400d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.462+0000 7f8f44fe9640 1 -- 172.21.15.38:0/1630465455 >> v1:172.21.15.107:6789/0 conn(0x7f8f400d1e90 legacy=0x7f8f400b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.463+0000 7f8f44fe9640 1 -- 172.21.15.38:0/1630465455 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f400d2bd0 con 0x7f8f400d0320 2024-04-25T08:40:27.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.463+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/1630465455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f400d0320 0x7f8f400d33d0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009870 tx=0x7f8f3002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c53b1b84424909b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.463+0000 7f8f3f7fe640 1 -- 172.21.15.38:0/1630465455 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30037070 con 0x7f8f400d0320 2024-04-25T08:40:27.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.463+0000 7f8f3f7fe640 1 -- 172.21.15.38:0/1630465455 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3002ae50 con 0x7f8f400d0320 2024-04-25T08:40:27.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.463+0000 7f8f3f7fe640 1 -- 172.21.15.38:0/1630465455 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30032860 con 0x7f8f400d0320 2024-04-25T08:40:27.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.464+0000 7f8f4724d640 1 -- 172.21.15.38:0/1630465455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f400d0320 msgr2=0x7f8f400d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.464+0000 7f8f4724d640 1 --2- 172.21.15.38:0/1630465455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f400d0320 0x7f8f400d33d0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f8f30009870 tx=0x7f8f3002a300 comp rx=0 tx=0).stop 2024-04-25T08:40:27.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.464+0000 7f8f4724d640 1 -- 172.21.15.38:0/1630465455 shutdown_connections 2024-04-25T08:40:27.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.464+0000 7f8f4724d640 1 --2- 172.21.15.38:0/1630465455 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f400d0700 0x7f8f400d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.464+0000 7f8f4724d640 1 --2- 172.21.15.38:0/1630465455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f400d0320 0x7f8f400d33d0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.464+0000 7f8f4724d640 1 -- 172.21.15.38:0/1630465455 >> 172.21.15.38:0/1630465455 conn(0x7f8f400668f0 msgr2=0x7f8f40066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.464+0000 7f8f4724d640 1 -- 172.21.15.38:0/1630465455 shutdown_connections 2024-04-25T08:40:27.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.464+0000 7f8f4724d640 1 -- 172.21.15.38:0/1630465455 wait complete. 2024-04-25T08:40:27.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.465+0000 7f8f4724d640 1 Processor -- start 2024-04-25T08:40:27.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.465+0000 7f8f4724d640 1 -- start start 2024-04-25T08:40:27.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.466+0000 7f8f4724d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f400d0320 0x7f8f40155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.466+0000 7f8f4724d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f400d0700 0x7f8f40159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.466+0000 7f8f4724d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f400d1e90 0x7f8f400ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.466+0000 7f8f4724d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f400d0b40 con 0x7f8f400d0700 2024-04-25T08:40:27.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.466+0000 7f8f4724d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f40161210 con 0x7f8f400d0320 2024-04-25T08:40:27.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.466+0000 7f8f4724d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4015a480 con 0x7f8f400d1e90 2024-04-25T08:40:27.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.466+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f400d0700 0x7f8f40159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.466+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f400d0700 0x7f8f40159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35816/0 (socket says 172.21.15.38:35816) 2024-04-25T08:40:27.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.466+0000 7f8f3ffff640 1 -- 172.21.15.38:0/194972266 learned_addr learned my addr 172.21.15.38:0/194972266 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.466+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f400d0320 0x7f8f40155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.466+0000 7f8f457ea640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f400d1e90 0x7f8f400ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.466+0000 7f8f3ffff640 1 -- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f400d1e90 msgr2=0x7f8f400ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.467+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f400d1e90 0x7f8f400ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.467+0000 7f8f3ffff640 1 -- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f400d0320 msgr2=0x7f8f40155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.467+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f400d0320 0x7f8f40155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.467+0000 7f8f3ffff640 1 -- 172.21.15.38:0/194972266 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f40155910 con 0x7f8f400d0700 2024-04-25T08:40:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.467+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f400d0320 0x7f8f40155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.467+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f400d0700 0x7f8f40159f40 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400d830 tx=0x7f8f3400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.467+0000 7f8f3d7fa640 1 -- 172.21.15.38:0/194972266 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400b660 con 0x7f8f400d0700 2024-04-25T08:40:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.467+0000 7f8f3d7fa640 1 -- 172.21.15.38:0/194972266 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f34019070 con 0x7f8f400d0700 2024-04-25T08:40:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.467+0000 7f8f4724d640 1 -- 172.21.15.38:0/194972266 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f40155bf0 con 0x7f8f400d0700 2024-04-25T08:40:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.467+0000 7f8f3d7fa640 1 -- 172.21.15.38:0/194972266 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3400ba20 con 0x7f8f400d0700 2024-04-25T08:40:27.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.467+0000 7f8f4724d640 1 -- 172.21.15.38:0/194972266 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f401561b0 con 0x7f8f400d0700 2024-04-25T08:40:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.468+0000 7f8f3d7fa640 1 -- 172.21.15.38:0/194972266 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f3400bbc0 con 0x7f8f400d0700 2024-04-25T08:40:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.469+0000 7f8f3d7fa640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8f2c06b4d0 0x7f8f2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.469+0000 7f8f3d7fa640 1 -- 172.21.15.38:0/194972266 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f3408a800 con 0x7f8f400d0700 2024-04-25T08:40:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.469+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8f2c06b4d0 0x7f8f2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.470+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8f2c06b4d0 0x7f8f2c06d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8f30034000 tx=0x7f8f3002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.470+0000 7f8f4a02c740 1 Processor -- start 2024-04-25T08:40:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.470+0000 7f8f4a02c740 1 -- start start 2024-04-25T08:40:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.470+0000 7f8f4a02c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 0x562620ff9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.470+0000 7f8f4a02c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562620ff8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.470+0000 7f8f4a02c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x562621022be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.470+0000 7f8f4a02c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562620f5ab90 con 0x562621022760 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.470+0000 7f8f4a02c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562620f03e40 con 0x562620ff7ca0 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.470+0000 7f8f4a02c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562620f56c30 con 0x562621034370 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f457ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x562621022be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f457ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x562621022be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35830/0 (socket says 172.21.15.38:35830) 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f457ea640 1 -- 172.21.15.38:0/1032671597 learned_addr learned my addr 172.21.15.38:0/1032671597 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/1032671597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 0x562620ff9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/1032671597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562620ff8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f457ea640 1 -- 172.21.15.38:0/1032671597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 msgr2=0x562620ff8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f457ea640 1 --2- 172.21.15.38:0/1032671597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562620ff8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f457ea640 1 -- 172.21.15.38:0/1032671597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 msgr2=0x562620ff9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f457ea640 1 --2- 172.21.15.38:0/1032671597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 0x562620ff9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f457ea640 1 -- 172.21.15.38:0/1032671597 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562620d397d0 con 0x562621022760 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/1032671597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562620ff8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f457ea640 1 --2- 172.21.15.38:0/1032671597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x562621022be0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f8f38001c90 tx=0x7f8f38002150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b820a0976b9ebc5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f267fc640 1 -- 172.21.15.38:0/1032671597 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38016020 con 0x562621022760 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f267fc640 1 -- 172.21.15.38:0/1032671597 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3800dc60 con 0x562621022760 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.471+0000 7f8f267fc640 1 -- 172.21.15.38:0/1032671597 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f380125f0 con 0x562621022760 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.472+0000 7f8f4a02c740 1 -- 172.21.15.38:0/1032671597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 msgr2=0x562621022be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.472+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/1032671597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x562621022be0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f8f38001c90 tx=0x7f8f38002150 comp rx=0 tx=0).stop 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.472+0000 7f8f4a02c740 1 -- 172.21.15.38:0/1032671597 shutdown_connections 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.472+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/1032671597 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x562621022be0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.472+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/1032671597 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562620ff8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.472+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/1032671597 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 0x562620ff9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.472+0000 7f8f4a02c740 1 -- 172.21.15.38:0/1032671597 >> 172.21.15.38:0/1032671597 conn(0x562620fbf870 msgr2=0x56262103d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.472+0000 7f8f4a02c740 1 -- 172.21.15.38:0/1032671597 shutdown_connections 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.472+0000 7f8f4a02c740 1 -- 172.21.15.38:0/1032671597 wait complete. 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.472+0000 7f8f4a02c740 1 -- start start 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.473+0000 7f8f4a02c740 1 Processor -- start 2024-04-25T08:40:27.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.473+0000 7f8f4a02c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 0x562621025030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.473+0000 7f8f4a02c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x5626210511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.473+0000 7f8f4a02c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562621051730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.473+0000 7f8f4a02c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562620f173e0 con 0x562621022760 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.473+0000 7f8f4a02c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626210202b0 con 0x562620ff7ca0 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.473+0000 7f8f4a02c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626210244d0 con 0x562621034370 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.473+0000 7f8f4a02c740 10 client.0 _ll_register_callbacks cb 0x562620f16db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.473+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x5626210511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.473+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x5626210511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35842/0 (socket says 172.21.15.38:35842) 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f3ffff640 1 -- 172.21.15.38:0/4212780640 learned_addr learned my addr 172.21.15.38:0/4212780640 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 0x562621025030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f457ea640 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562621051730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f3ffff640 1 -- 172.21.15.38:0/4212780640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 msgr2=0x562621051730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562621051730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f3ffff640 1 -- 172.21.15.38:0/4212780640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 msgr2=0x562621025030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 0x562621025030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f3ffff640 1 -- 172.21.15.38:0/4212780640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562620c2f0f0 con 0x562621022760 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x5626210511f0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f8f34060fc0 tx=0x7f8f340594e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb54cead0c7e27a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f1f7fe640 1 -- 172.21.15.38:0/4212780640 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34070020 con 0x562621022760 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f1f7fe640 1 -- 172.21.15.38:0/4212780640 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f34083d30 con 0x562621022760 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f4a02c740 1 -- 172.21.15.38:0/4212780640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f3800dc60 con 0x562621022760 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f1f7fe640 1 -- 172.21.15.38:0/4212780640 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f3405c070 con 0x562621022760 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f4a02c740 10 client.9168 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f4a02c740 1 -- 172.21.15.38:0/4212780640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x562620d99f60 con 0x562621022760 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.474+0000 7f8f1f7fe640 1 -- 172.21.15.38:0/4212780640 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3800dc60 con 0x562621022760 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.475+0000 7f8f1f7fe640 1 -- 172.21.15.38:0/4212780640 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f34013070 con 0x562621022760 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.475+0000 7f8f4a02c740 10 client.9168 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.475+0000 7f8f4a02c740 10 client.9168 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:27.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.475+0000 7f8f4a02c740 20 client.9168 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.475+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5626210116f0 0x562621011af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.475+0000 7f8f4a02c740 4 client.9168 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.475+0000 7f8f4a02c740 1 -- 172.21.15.38:0/4212780640 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562620f680d0 con 0x5626210116f0 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.475+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5626210116f0 0x562621011af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.475+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5626210116f0 0x562621011af0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0570f5f660bc6b6 server_cookie=23746837d0fb2e35 in_seq=0 out_seq=0 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.475+0000 7f8f1f7fe640 10 client.9168 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f1f7fe640 1 -- 172.21.15.38:0/4212780640 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562620f680d0 con 0x5626210116f0 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f1f7fe640 10 client.9168 handle_command_reply: tid=0 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f4a02c740 1 client.9168 shutdown 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f4a02c740 1 -- 172.21.15.38:0/4212780640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 msgr2=0x5626210511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x5626210511f0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f8f34060fc0 tx=0x7f8f340594e0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f4a02c740 1 -- 172.21.15.38:0/4212780640 shutdown_connections 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5626210116f0 0x562621011af0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562621051730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x5626210511f0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f1f7fe640 0 client.9168 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:27.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/4212780640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 0x562621025030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f4a02c740 1 -- 172.21.15.38:0/4212780640 >> 172.21.15.38:0/4212780640 conn(0x562620fbf870 msgr2=0x5626210206b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f4a02c740 1 -- 172.21.15.38:0/4212780640 shutdown_connections 2024-04-25T08:40:27.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f4a02c740 1 -- 172.21.15.38:0/4212780640 wait complete. 2024-04-25T08:40:27.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.476+0000 7f8f4a02c740 20 client.9168 trim_cache size 0 max 16384 2024-04-25T08:40:27.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.491+0000 7f8f4a02c740 1 Processor -- start 2024-04-25T08:40:27.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.491+0000 7f8f4a02c740 1 -- start start 2024-04-25T08:40:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f4a02c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 0x562620ff1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f4a02c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x562621036d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f4a02c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562621020350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f4a02c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f402003f0 con 0x562621022760 2024-04-25T08:40:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f4a02c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f401ff690 con 0x562620ff7ca0 2024-04-25T08:40:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f4a02c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f401572c0 con 0x562621034370 2024-04-25T08:40:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x562621036d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f3ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x562621036d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35846/0 (socket says 172.21.15.38:35846) 2024-04-25T08:40:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f3ffff640 1 -- 172.21.15.38:0/1052521373 learned_addr learned my addr 172.21.15.38:0/1052521373 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/1052521373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 0x562620ff1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f457ea640 1 --2- 172.21.15.38:0/1052521373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562621020350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f3ffff640 1 -- 172.21.15.38:0/1052521373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 msgr2=0x562621020350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/1052521373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562621020350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f3ffff640 1 -- 172.21.15.38:0/1052521373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 msgr2=0x562620ff1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/1052521373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 0x562620ff1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f3ffff640 1 -- 172.21.15.38:0/1052521373 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562620c081b0 con 0x562621022760 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.492+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/1052521373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x562621036d80 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f8f34058ae0 tx=0x7f8f34020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa098f880b61d8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.493+0000 7f8f25ffb640 1 -- 172.21.15.38:0/1052521373 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3407c070 con 0x562621022760 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.493+0000 7f8f25ffb640 1 -- 172.21.15.38:0/1052521373 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3405de50 con 0x562621022760 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.493+0000 7f8f25ffb640 1 -- 172.21.15.38:0/1052521373 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f34070040 con 0x562621022760 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.493+0000 7f8f4a02c740 1 -- 172.21.15.38:0/1052521373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 msgr2=0x562621036d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.493+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/1052521373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x562621036d80 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f8f34058ae0 tx=0x7f8f34020000 comp rx=0 tx=0).stop 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.493+0000 7f8f4a02c740 1 -- 172.21.15.38:0/1052521373 shutdown_connections 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.493+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/1052521373 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562621020350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.493+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/1052521373 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562621022760 0x562621036d80 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.493+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/1052521373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562620ff7ca0 0x562620ff1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.493+0000 7f8f4a02c740 1 -- 172.21.15.38:0/1052521373 >> 172.21.15.38:0/1052521373 conn(0x562620fbf870 msgr2=0x56262103a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.493+0000 7f8f4a02c740 1 -- 172.21.15.38:0/1052521373 shutdown_connections 2024-04-25T08:40:27.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.493+0000 7f8f4a02c740 1 -- 172.21.15.38:0/1052521373 wait complete. 2024-04-25T08:40:27.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.494+0000 7f8f4a02c740 1 -- start start 2024-04-25T08:40:27.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.494+0000 7f8f4a02c740 1 Processor -- start 2024-04-25T08:40:27.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f4a02c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562620ff7ca0 0x56262104be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f4a02c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621022760 0x56262104c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f4a02c740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562620ff6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f4a02c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f401578c0 con 0x562620ff7ca0 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f4a02c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562621024660 con 0x562621022760 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f4a02c740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562620ff73d0 con 0x562621034370 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f4a02c740 10 client.0 _ll_register_callbacks cb 0x562620ff86d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f44fe9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562620ff7ca0 0x56262104be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f44fe9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562620ff7ca0 0x56262104be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35848/0 (socket says 172.21.15.38:35848) 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f44fe9640 1 -- 172.21.15.38:0/3927197664 learned_addr learned my addr 172.21.15.38:0/3927197664 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621022760 0x56262104c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f457ea640 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562620ff6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f44fe9640 1 -- 172.21.15.38:0/3927197664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 msgr2=0x562620ff6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562620ff6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f44fe9640 1 -- 172.21.15.38:0/3927197664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621022760 msgr2=0x56262104c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621022760 0x56262104c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f44fe9640 1 -- 172.21.15.38:0/3927197664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562621022000 con 0x562620ff7ca0 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f44fe9640 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562620ff7ca0 0x56262104be00 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8f30035040 tx=0x7f8f30004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3299aa68429f72e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f1f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f1f7fe640 1 -- 172.21.15.38:0/3927197664 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f30037040 con 0x562620ff7ca0 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f1f7fe640 1 -- 172.21.15.38:0/3927197664 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8f30057c80 con 0x562620ff7ca0 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.495+0000 7f8f4a02c740 1 -- 172.21.15.38:0/3927197664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f3405de50 con 0x562620ff7ca0 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f1f7fe640 1 -- 172.21.15.38:0/3927197664 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f30057e80 con 0x562620ff7ca0 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f4a02c740 10 client.9174 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f4a02c740 1 -- 172.21.15.38:0/3927197664 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x562620ff9720 con 0x562620ff7ca0 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f1f7fe640 1 -- 172.21.15.38:0/3927197664 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3405de50 con 0x562620ff7ca0 2024-04-25T08:40:27.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f1f7fe640 1 -- 172.21.15.38:0/3927197664 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f3005e050 con 0x562620ff7ca0 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f4a02c740 10 client.9174 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f4a02c740 10 client.9174 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f4a02c740 20 client.9174 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x56262109f280 0x562620ff69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f4a02c740 4 client.9174 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f4a02c740 1 -- 172.21.15.38:0/3927197664 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x562620f62420 con 0x56262109f280 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x56262109f280 0x562620ff69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f3ffff640 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x56262109f280 0x562620ff69a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e905cedfd48dec43 server_cookie=cedffc55eae2578d in_seq=0 out_seq=0 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.496+0000 7f8f1f7fe640 10 client.9174 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f1f7fe640 1 -- 172.21.15.38:0/3927197664 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x562620f62420 con 0x56262109f280 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f1f7fe640 10 client.9174 handle_command_reply: tid=0 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f4a02c740 1 client.9174 shutdown 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f4a02c740 1 -- 172.21.15.38:0/3927197664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562620ff7ca0 msgr2=0x56262104be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562620ff7ca0 0x56262104be00 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8f30035040 tx=0x7f8f30004730 comp rx=0 tx=0).stop 2024-04-25T08:40:27.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f4a02c740 1 -- 172.21.15.38:0/3927197664 shutdown_connections 2024-04-25T08:40:27.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x56262109f280 0x562620ff69a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562621034370 0x562620ff6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562621022760 0x56262104c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f1f7fe640 0 client.9174 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:27.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f4a02c740 1 --2- 172.21.15.38:0/3927197664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562620ff7ca0 0x56262104be00 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f4a02c740 1 -- 172.21.15.38:0/3927197664 >> 172.21.15.38:0/3927197664 conn(0x562620fbf870 msgr2=0x56262103a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f4a02c740 1 -- 172.21.15.38:0/3927197664 shutdown_connections 2024-04-25T08:40:27.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f4a02c740 1 -- 172.21.15.38:0/3927197664 wait complete. 2024-04-25T08:40:27.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.497+0000 7f8f4a02c740 20 client.9174 trim_cache size 0 max 16384 2024-04-25T08:40:27.497 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:40:27.498 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:40:27.498 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:40:27.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.498+0000 7f8f4724d640 1 -- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8f2c06b4d0 msgr2=0x7f8f2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.498+0000 7f8f4724d640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8f2c06b4d0 0x7f8f2c06d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f8f30034000 tx=0x7f8f3002e000 comp rx=0 tx=0).stop 2024-04-25T08:40:27.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.498+0000 7f8f4724d640 1 -- 172.21.15.38:0/194972266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f400d0700 msgr2=0x7f8f40159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.498+0000 7f8f4724d640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f400d0700 0x7f8f40159f40 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f8f3400d830 tx=0x7f8f3400dcf0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.498+0000 7f8f4724d640 1 -- 172.21.15.38:0/194972266 shutdown_connections 2024-04-25T08:40:27.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.498+0000 7f8f4724d640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8f2c06b4d0 0x7f8f2c06d990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.498+0000 7f8f4724d640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8f400d1e90 0x7f8f400ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.498+0000 7f8f4724d640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8f400d0700 0x7f8f40159f40 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.498+0000 7f8f4724d640 1 --2- 172.21.15.38:0/194972266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8f400d0320 0x7f8f40155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.498+0000 7f8f4724d640 1 -- 172.21.15.38:0/194972266 >> 172.21.15.38:0/194972266 conn(0x7f8f400668f0 msgr2=0x7f8f401602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.498+0000 7f8f4724d640 1 -- 172.21.15.38:0/194972266 shutdown_connections 2024-04-25T08:40:27.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.498+0000 7f8f4724d640 1 -- 172.21.15.38:0/194972266 wait complete. 2024-04-25T08:40:27.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:27.502 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T08:40:27.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.610+0000 7f96903f9640 1 Processor -- start 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.610+0000 7f96903f9640 1 -- start start 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.611+0000 7f96903f9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9688166610 0x7f9688168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.611+0000 7f96903f9640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96880d2060 con 0x7f96880d0320 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.611+0000 7f96903f9640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96880d2330 con 0x7f96880d33d0 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.611+0000 7f96903f9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96880d2610 con 0x7f9688166610 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.611+0000 7f968e195640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f96880d0320 0x7f96880d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60478/0 (socket says 172.21.15.38:60478) 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.611+0000 7f968e195640 1 -- 172.21.15.38:0/1684908935 learned_addr learned my addr 172.21.15.38:0/1684908935 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968e996640 1 --2- 172.21.15.38:0/1684908935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9688166610 0x7f9688168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637420886 0 0) 0x7f96880d2060 con 0x7f96880d0320 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9670003520 con 0x7f96880d0320 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164119512 0 0) 0x7f96880d2330 con 0x7f96880d33d0 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96880d2060 con 0x7f96880d33d0 2024-04-25T08:40:27.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2270301729 0 0) 0x7f9670003520 con 0x7f96880d0320 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9688166610 msgr2=0x7f9688168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 --2- 172.21.15.38:0/1684908935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9688166610 0x7f9688168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 >> v1:172.21.15.107:6789/0 conn(0x7f96880d33d0 legacy=0x7f96880d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96880d28f0 con 0x7f96880d0320 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f9680001690 con 0x7f96880d0320 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 >> v1:172.21.15.38:6789/0 conn(0x7f96880d0320 legacy=0x7f96880d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 --2- 172.21.15.38:0/1684908935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9688166610 0x7f9670003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 --2- 172.21.15.38:0/1684908935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9670004120 0x7f9670004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 --2- 172.21.15.38:0/1684908935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9670004d40 0x7f96700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96880d2610 con 0x7f9670004d40 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f967000bb00 con 0x7f9670004120 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967000bd00 con 0x7f9688166610 2024-04-25T08:40:27.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968e996640 1 --2- 172.21.15.38:0/1684908935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9670004d40 0x7f96700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968e195640 1 --2- 172.21.15.38:0/1684908935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9688166610 0x7f9670003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968d994640 1 --2- 172.21.15.38:0/1684908935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9670004120 0x7f9670004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968e996640 1 -- 172.21.15.38:0/1684908935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9688166610 msgr2=0x7f9670003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968e996640 1 --2- 172.21.15.38:0/1684908935 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9688166610 0x7f9670003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968e996640 1 -- 172.21.15.38:0/1684908935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9670004120 msgr2=0x7f9670004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.612+0000 7f968e996640 1 --2- 172.21.15.38:0/1684908935 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9670004120 0x7f9670004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.613+0000 7f968e996640 1 -- 172.21.15.38:0/1684908935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9670003520 con 0x7f9670004d40 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.613+0000 7f968e996640 1 --2- 172.21.15.38:0/1684908935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9670004d40 0x7f96700051c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f96880ca820 tx=0x7f9670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b39a602fe9e3581 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.613+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678037070 con 0x7f9670004d40 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.613+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967802e050 con 0x7f9670004d40 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.613+0000 7f968d193640 1 -- 172.21.15.38:0/1684908935 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678009c10 con 0x7f9670004d40 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.614+0000 7f96903f9640 1 -- 172.21.15.38:0/1684908935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9670004d40 msgr2=0x7f96700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.614+0000 7f96903f9640 1 --2- 172.21.15.38:0/1684908935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9670004d40 0x7f96700051c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f96880ca820 tx=0x7f9670005830 comp rx=0 tx=0).stop 2024-04-25T08:40:27.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.614+0000 7f968e996640 1 -- 172.21.15.38:0/1684908935 reap_dead start 2024-04-25T08:40:27.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.614+0000 7f96903f9640 1 -- 172.21.15.38:0/1684908935 shutdown_connections 2024-04-25T08:40:27.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.614+0000 7f96903f9640 1 -- 172.21.15.38:0/1684908935 >> 172.21.15.38:0/1684908935 conn(0x7f96880668f0 msgr2=0x7f9688066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.614+0000 7f96903f9640 1 -- 172.21.15.38:0/1684908935 shutdown_connections 2024-04-25T08:40:27.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.614+0000 7f96903f9640 1 -- 172.21.15.38:0/1684908935 wait complete. 2024-04-25T08:40:27.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.615+0000 7f96903f9640 1 Processor -- start 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.615+0000 7f96903f9640 1 -- start start 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f96903f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9670004d40 0x7f9688154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f96903f9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96881f48d0 0x7f9688159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f96903f9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f968815ee70 0x7f96880b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f96903f9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9688161210 con 0x7f96881f48d0 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f96903f9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96880d2eb0 con 0x7f9670004d40 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f96903f9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9688161390 con 0x7f968815ee70 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f968d994640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96881f48d0 0x7f9688159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f968d994640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96881f48d0 0x7f9688159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35864/0 (socket says 172.21.15.38:35864) 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f968d994640 1 -- 172.21.15.38:0/4024644723 learned_addr learned my addr 172.21.15.38:0/4024644723 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f968d994640 1 -- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f968815ee70 msgr2=0x7f96880b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f968d994640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f968815ee70 0x7f96880b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f968e195640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9670004d40 0x7f9688154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f968d994640 1 -- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9670004d40 msgr2=0x7f9688154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f968d994640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9670004d40 0x7f9688154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.616+0000 7f968d994640 1 -- 172.21.15.38:0/4024644723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968815a400 con 0x7f96881f48d0 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.617+0000 7f968d994640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96881f48d0 0x7f9688159bf0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f967c00f8e0 tx=0x7f967c00fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.617+0000 7f9686ffd640 1 -- 172.21.15.38:0/4024644723 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c0052e0 con 0x7f96881f48d0 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.617+0000 7f9686ffd640 1 -- 172.21.15.38:0/4024644723 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c018070 con 0x7f96881f48d0 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.617+0000 7f9686ffd640 1 -- 172.21.15.38:0/4024644723 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c00d890 con 0x7f96881f48d0 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.617+0000 7f96903f9640 1 -- 172.21.15.38:0/4024644723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968815a130 con 0x7f96881f48d0 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.617+0000 7f96903f9640 1 -- 172.21.15.38:0/4024644723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96881f3df0 con 0x7f96881f48d0 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.618+0000 7f9686ffd640 1 -- 172.21.15.38:0/4024644723 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f967c00da30 con 0x7f96881f48d0 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.619+0000 7f9686ffd640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f967406b500 0x7f967406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.619+0000 7f9686ffd640 1 -- 172.21.15.38:0/4024644723 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f967c08a360 con 0x7f96881f48d0 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.619+0000 7f968e195640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f967406b500 0x7f967406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.619+0000 7f968e195640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f967406b500 0x7f967406d9c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f96880b8f30 tx=0x7f9680009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.620+0000 7f96931da740 1 Processor -- start 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.620+0000 7f96931da740 1 -- start start 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f96931da740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88c3b10 0x5586e88c6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f96931da740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88c6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f96931da740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e8896770 0x5586e8899f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f96931da740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586e87ffda0 con 0x5586e88c3b10 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f96931da740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586e87a8b80 con 0x5586e8896770 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f96931da740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586e87fbe40 con 0x5586e88c3ef0 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f968e195640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88c3b10 0x5586e88c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f968e195640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88c3b10 0x5586e88c6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35874/0 (socket says 172.21.15.38:35874) 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f968e195640 1 -- 172.21.15.38:0/2480857913 learned_addr learned my addr 172.21.15.38:0/2480857913 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f968e996640 1 --2- 172.21.15.38:0/2480857913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e8896770 0x5586e8899f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f968d994640 1 --2- 172.21.15.38:0/2480857913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f968e195640 1 -- 172.21.15.38:0/2480857913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 msgr2=0x5586e88c6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f968e195640 1 --2- 172.21.15.38:0/2480857913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88c6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f968e195640 1 -- 172.21.15.38:0/2480857913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e8896770 msgr2=0x5586e8899f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f968e195640 1 --2- 172.21.15.38:0/2480857913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e8896770 0x5586e8899f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f968e195640 1 -- 172.21.15.38:0/2480857913 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586e87b2f10 con 0x5586e88c3b10 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f968e996640 1 --2- 172.21.15.38:0/2480857913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e8896770 0x5586e8899f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f968e195640 1 --2- 172.21.15.38:0/2480857913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88c3b10 0x5586e88c6320 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f9680001020 tx=0x7f9680013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c0dc6b1c8de099 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f966ffff640 1 -- 172.21.15.38:0/2480857913 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680027070 con 0x5586e88c3b10 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f966ffff640 1 -- 172.21.15.38:0/2480857913 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9680014870 con 0x5586e88c3b10 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.621+0000 7f966ffff640 1 -- 172.21.15.38:0/2480857913 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680014c20 con 0x5586e88c3b10 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.622+0000 7f96931da740 1 -- 172.21.15.38:0/2480857913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88c3b10 msgr2=0x5586e88c6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.622+0000 7f96931da740 1 --2- 172.21.15.38:0/2480857913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88c3b10 0x5586e88c6320 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f9680001020 tx=0x7f9680013000 comp rx=0 tx=0).stop 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.622+0000 7f96931da740 1 -- 172.21.15.38:0/2480857913 shutdown_connections 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.622+0000 7f96931da740 1 --2- 172.21.15.38:0/2480857913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e8896770 0x5586e8899f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.622+0000 7f96931da740 1 --2- 172.21.15.38:0/2480857913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88c6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.622+0000 7f96931da740 1 --2- 172.21.15.38:0/2480857913 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88c3b10 0x5586e88c6320 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.622+0000 7f96931da740 1 -- 172.21.15.38:0/2480857913 >> 172.21.15.38:0/2480857913 conn(0x5586e88989e0 msgr2=0x5586e88e1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.622+0000 7f96931da740 1 -- 172.21.15.38:0/2480857913 shutdown_connections 2024-04-25T08:40:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.622+0000 7f96931da740 1 -- 172.21.15.38:0/2480857913 wait complete. 2024-04-25T08:40:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.623+0000 7f96931da740 1 -- start start 2024-04-25T08:40:27.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.623+0000 7f96931da740 1 Processor -- start 2024-04-25T08:40:27.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f96931da740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f96931da740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f96931da740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f96931da740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586e889bb90 con 0x5586e88ba8e0 2024-04-25T08:40:27.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f96931da740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586e879e4b0 con 0x5586e88c3b10 2024-04-25T08:40:27.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f96931da740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586e889be70 con 0x5586e88c3ef0 2024-04-25T08:40:27.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f96931da740 10 client.0 _ll_register_callbacks cb 0x5586e87de180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:27.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f968e996640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88e06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f968e996640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88e06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35886/0 (socket says 172.21.15.38:35886) 2024-04-25T08:40:27.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f968e996640 1 -- 172.21.15.38:0/871119752 learned_addr learned my addr 172.21.15.38:0/871119752 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f968e195640 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f968d994640 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f968e996640 1 -- 172.21.15.38:0/871119752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 msgr2=0x5586e88e01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f968e996640 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f968e996640 1 -- 172.21.15.38:0/871119752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 msgr2=0x5586e88e4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f968e996640 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f968e996640 1 -- 172.21.15.38:0/871119752 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586e85de7d0 con 0x5586e88ba8e0 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f968e996640 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88e06e0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f967802ae80 tx=0x7f9678009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f80802adcd9f39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.624+0000 7f966d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f966d7fa640 1 -- 172.21.15.38:0/871119752 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678045060 con 0x5586e88ba8e0 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f966d7fa640 1 -- 172.21.15.38:0/871119752 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9678032b10 con 0x5586e88ba8e0 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f96931da740 1 -- 172.21.15.38:0/871119752 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9680014870 con 0x5586e88ba8e0 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f966d7fa640 1 -- 172.21.15.38:0/871119752 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f967804b050 con 0x5586e88ba8e0 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f96931da740 10 client.9186 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f96931da740 1 -- 172.21.15.38:0/871119752 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x5586e85143f0 con 0x5586e88ba8e0 2024-04-25T08:40:27.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f966d7fa640 1 -- 172.21.15.38:0/871119752 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9680014870 con 0x5586e88ba8e0 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f966d7fa640 1 -- 172.21.15.38:0/871119752 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f967804e020 con 0x5586e88ba8e0 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f96931da740 10 client.9186 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f96931da740 10 client.9186 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f96931da740 20 client.9186 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f96931da740 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5586e88c8b40 0x5586e88c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f96931da740 4 client.9186 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f96931da740 1 -- 172.21.15.38:0/871119752 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586e880d2e0 con 0x5586e88c8b40 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.625+0000 7f968e195640 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5586e88c8b40 0x5586e88c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.626+0000 7f966d7fa640 10 client.9186 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.626+0000 7f968e195640 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5586e88c8b40 0x5586e88c8f20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=416b710463e500db server_cookie=92c58b97838b4263 in_seq=0 out_seq=0 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.626+0000 7f966d7fa640 1 -- 172.21.15.38:0/871119752 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5586e880d2e0 con 0x5586e88c8b40 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.626+0000 7f966d7fa640 10 client.9186 handle_command_reply: tid=0 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.626+0000 7f96931da740 1 client.9186 shutdown 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.627+0000 7f96931da740 1 -- 172.21.15.38:0/871119752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 msgr2=0x5586e88e06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.627+0000 7f96931da740 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88e06e0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f967802ae80 tx=0x7f9678009750 comp rx=0 tx=0).stop 2024-04-25T08:40:27.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.627+0000 7f96931da740 1 -- 172.21.15.38:0/871119752 shutdown_connections 2024-04-25T08:40:27.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.627+0000 7f96931da740 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5586e88c8b40 0x5586e88c8f20 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.627+0000 7f96931da740 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88e06e0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.627+0000 7f96931da740 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.627+0000 7f96931da740 1 --2- 172.21.15.38:0/871119752 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.627+0000 7f966d7fa640 0 client.9186 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:27.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.627+0000 7f96931da740 1 -- 172.21.15.38:0/871119752 >> 172.21.15.38:0/871119752 conn(0x5586e8896770 msgr2=0x5586e889b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.627+0000 7f96931da740 1 -- 172.21.15.38:0/871119752 shutdown_connections 2024-04-25T08:40:27.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.627+0000 7f96931da740 1 -- 172.21.15.38:0/871119752 wait complete. 2024-04-25T08:40:27.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.627+0000 7f96931da740 20 client.9186 trim_cache size 0 max 16384 2024-04-25T08:40:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.642+0000 7f96931da740 1 Processor -- start 2024-04-25T08:40:27.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.642+0000 7f96931da740 1 -- start start 2024-04-25T08:40:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f96931da740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88e4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f96931da740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f96931da740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f96931da740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9688200b20 con 0x5586e88ba8e0 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f96931da740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96881ffd00 con 0x5586e88c3b10 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f96931da740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96881ff150 con 0x5586e88c3ef0 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968e195640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88e4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968e195640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88e4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35888/0 (socket says 172.21.15.38:35888) 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968e195640 1 -- 172.21.15.38:0/2940349207 learned_addr learned my addr 172.21.15.38:0/2940349207 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968d994640 1 --2- 172.21.15.38:0/2940349207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968e195640 1 -- 172.21.15.38:0/2940349207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 msgr2=0x5586e88e9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968e996640 1 --2- 172.21.15.38:0/2940349207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968e195640 1 --2- 172.21.15.38:0/2940349207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968e195640 1 -- 172.21.15.38:0/2940349207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 msgr2=0x5586e88e8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968e195640 1 --2- 172.21.15.38:0/2940349207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968e195640 1 -- 172.21.15.38:0/2940349207 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586e84d40f0 con 0x5586e88ba8e0 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968d994640 1 --2- 172.21.15.38:0/2940349207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968e996640 1 --2- 172.21.15.38:0/2940349207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f968e195640 1 --2- 172.21.15.38:0/2940349207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88e4db0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f9680014400 tx=0x7f9680014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a66a3e747b7a1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f966f7fe640 1 -- 172.21.15.38:0/2940349207 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680032070 con 0x5586e88ba8e0 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f966f7fe640 1 -- 172.21.15.38:0/2940349207 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9680028040 con 0x5586e88ba8e0 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.643+0000 7f966f7fe640 1 -- 172.21.15.38:0/2940349207 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680027050 con 0x5586e88ba8e0 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.644+0000 7f96931da740 1 -- 172.21.15.38:0/2940349207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 msgr2=0x5586e88e4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.644+0000 7f96931da740 1 --2- 172.21.15.38:0/2940349207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88e4db0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f9680014400 tx=0x7f9680014000 comp rx=0 tx=0).stop 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.644+0000 7f96931da740 1 -- 172.21.15.38:0/2940349207 shutdown_connections 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.644+0000 7f96931da740 1 --2- 172.21.15.38:0/2940349207 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.644+0000 7f96931da740 1 --2- 172.21.15.38:0/2940349207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.644+0000 7f96931da740 1 --2- 172.21.15.38:0/2940349207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88e4db0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.644+0000 7f96931da740 1 -- 172.21.15.38:0/2940349207 >> 172.21.15.38:0/2940349207 conn(0x5586e8896770 msgr2=0x5586e8899f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.644+0000 7f96931da740 1 -- 172.21.15.38:0/2940349207 shutdown_connections 2024-04-25T08:40:27.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.644+0000 7f96931da740 1 -- 172.21.15.38:0/2940349207 wait complete. 2024-04-25T08:40:27.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.645+0000 7f96931da740 1 -- start start 2024-04-25T08:40:27.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.645+0000 7f96931da740 1 Processor -- start 2024-04-25T08:40:27.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.645+0000 7f96931da740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.645+0000 7f96931da740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.645+0000 7f96931da740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.645+0000 7f96931da740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586e8892050 con 0x5586e88ba8e0 2024-04-25T08:40:27.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.645+0000 7f96931da740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96881ff700 con 0x5586e88c3b10 2024-04-25T08:40:27.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.645+0000 7f96931da740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586e8895db0 con 0x5586e88c3ef0 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f96931da740 10 client.0 _ll_register_callbacks cb 0x7f9680004ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f968e195640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f968e195640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35896/0 (socket says 172.21.15.38:35896) 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f968e195640 1 -- 172.21.15.38:0/3552645521 learned_addr learned my addr 172.21.15.38:0/3552645521 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f968e996640 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f968d994640 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f968e195640 1 -- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 msgr2=0x5586e88e4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f968e195640 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f968e195640 1 -- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 msgr2=0x5586e88e47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f968e195640 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f968e195640 1 -- 172.21.15.38:0/3552645521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586e894c9d0 con 0x5586e88ba8e0 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f968d994640 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f968e195640 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88c1780 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f9680025040 tx=0x7f96880b68d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7354e7d949310feb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f966d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:27.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f966d7fa640 1 -- 172.21.15.38:0/3552645521 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680027070 con 0x5586e88ba8e0 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f966d7fa640 1 -- 172.21.15.38:0/3552645521 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f968003c430 con 0x5586e88ba8e0 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f96931da740 1 -- 172.21.15.38:0/3552645521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9680028040 con 0x5586e88ba8e0 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f966d7fa640 1 -- 172.21.15.38:0/3552645521 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9680049050 con 0x5586e88ba8e0 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f96931da740 10 client.9192 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.646+0000 7f96931da740 1 -- 172.21.15.38:0/3552645521 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x5586e88f00b0 con 0x5586e88ba8e0 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f966d7fa640 1 -- 172.21.15.38:0/3552645521 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9680028040 con 0x5586e88ba8e0 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f966d7fa640 1 -- 172.21.15.38:0/3552645521 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f968004c020 con 0x5586e88ba8e0 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f96931da740 10 client.9192 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f96931da740 10 client.9192 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f96931da740 20 client.9192 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f96931da740 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5586e88c1cc0 0x5586e894f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f96931da740 4 client.9192 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f96931da740 1 -- 172.21.15.38:0/3552645521 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5586e8807630 con 0x5586e88c1cc0 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f968d994640 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5586e88c1cc0 0x5586e894f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.646 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f968d994640 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5586e88c1cc0 0x5586e894f010 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bc071123f7de770 server_cookie=bdf33fb261eb9121 in_seq=0 out_seq=0 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f966d7fa640 10 client.9192 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f966d7fa640 1 -- 172.21.15.38:0/3552645521 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x5586e8807630 con 0x5586e88c1cc0 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f966d7fa640 10 client.9192 handle_command_reply: tid=0 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.647+0000 7f96931da740 1 client.9192 shutdown 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96931da740 1 -- 172.21.15.38:0/3552645521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 msgr2=0x5586e88c1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96931da740 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88c1780 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f9680025040 tx=0x7f96880b68d0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96931da740 1 -- 172.21.15.38:0/3552645521 shutdown_connections 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96931da740 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x5586e88c1cc0 0x5586e894f010 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96931da740 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x5586e88c3ef0 0x5586e88e4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96931da740 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5586e88c3b10 0x5586e88e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f966d7fa640 0 client.9192 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96931da740 1 --2- 172.21.15.38:0/3552645521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5586e88ba8e0 0x5586e88c1780 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96931da740 1 -- 172.21.15.38:0/3552645521 >> 172.21.15.38:0/3552645521 conn(0x5586e8896770 msgr2=0x5586e889c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96931da740 1 -- 172.21.15.38:0/3552645521 shutdown_connections 2024-04-25T08:40:27.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96931da740 1 -- 172.21.15.38:0/3552645521 wait complete. 2024-04-25T08:40:27.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96931da740 20 client.9192 trim_cache size 0 max 16384 2024-04-25T08:40:27.648 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:40:27.648 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T08:40:27.648 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-04-25T08:40:27.648 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:40:27.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96903f9640 1 -- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f967406b500 msgr2=0x7f967406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96903f9640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f967406b500 0x7f967406d9c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f96880b8f30 tx=0x7f9680009040 comp rx=0 tx=0).stop 2024-04-25T08:40:27.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96903f9640 1 -- 172.21.15.38:0/4024644723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96881f48d0 msgr2=0x7f9688159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96903f9640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96881f48d0 0x7f9688159bf0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f967c00f8e0 tx=0x7f967c00fda0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96903f9640 1 -- 172.21.15.38:0/4024644723 shutdown_connections 2024-04-25T08:40:27.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96903f9640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f967406b500 0x7f967406d9c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96903f9640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f968815ee70 0x7f96880b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96903f9640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f96881f48d0 0x7f9688159bf0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96903f9640 1 --2- 172.21.15.38:0/4024644723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9670004d40 0x7f9688154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96903f9640 1 -- 172.21.15.38:0/4024644723 >> 172.21.15.38:0/4024644723 conn(0x7f96880668f0 msgr2=0x7f96880c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96903f9640 1 -- 172.21.15.38:0/4024644723 shutdown_connections 2024-04-25T08:40:27.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.648+0000 7f96903f9640 1 -- 172.21.15.38:0/4024644723 wait complete. 2024-04-25T08:40:27.653 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.759+0000 7fce7803a640 1 Processor -- start 2024-04-25T08:40:27.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.760+0000 7fce7803a640 1 -- start start 2024-04-25T08:40:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.760+0000 7fce7803a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fce700d3900 0x7fce700d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.760+0000 7fce7803a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce700d0d90 0x7fce700b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.760+0000 7fce7803a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce700d3e70 con 0x7fce7015dc10 2024-04-25T08:40:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.760+0000 7fce7803a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7015dff0 con 0x7fce700d3900 2024-04-25T08:40:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.760+0000 7fce7803a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce700b6990 con 0x7fce700d0d90 2024-04-25T08:40:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.760+0000 7fce755d5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fce700d3900 0x7fce700d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.760+0000 7fce75dd6640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fce7015dc10 0x7fce700ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60488/0 (socket says 172.21.15.38:60488) 2024-04-25T08:40:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.760+0000 7fce75dd6640 1 -- 172.21.15.38:0/3706233156 learned_addr learned my addr 172.21.15.38:0/3706233156 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:27.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.761+0000 7fce765d7640 1 --2- 172.21.15.38:0/3706233156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce700d0d90 0x7fce700b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.761+0000 7fce755d5640 1 -- 172.21.15.38:0/3706233156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce700d0d90 msgr2=0x7fce700b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:27.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.761+0000 7fce755d5640 1 --2- 172.21.15.38:0/3706233156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce700d0d90 0x7fce700b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.761+0000 7fce755d5640 1 -- 172.21.15.38:0/3706233156 >> v1:172.21.15.38:6789/0 conn(0x7fce7015dc10 legacy=0x7fce700ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.761+0000 7fce74dd4640 1 -- 172.21.15.38:0/3706233156 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610519571 0 0) 0x7fce700d3e70 con 0x7fce7015dc10 2024-04-25T08:40:27.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.761+0000 7fce755d5640 1 -- 172.21.15.38:0/3706233156 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce700d4150 con 0x7fce700d3900 2024-04-25T08:40:27.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.761+0000 7fce755d5640 1 --2- 172.21.15.38:0/3706233156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce700d3900 0x7fce700d0850 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fce640097a0 tx=0x7fce6402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5089bc1dc31513e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.762+0000 7fce74dd4640 1 -- 172.21.15.38:0/3706233156 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64037070 con 0x7fce700d3900 2024-04-25T08:40:27.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.762+0000 7fce74dd4640 1 -- 172.21.15.38:0/3706233156 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6402ae50 con 0x7fce700d3900 2024-04-25T08:40:27.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.762+0000 7fce74dd4640 1 -- 172.21.15.38:0/3706233156 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64032660 con 0x7fce700d3900 2024-04-25T08:40:27.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.762+0000 7fce7803a640 1 -- 172.21.15.38:0/3706233156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce700d3900 msgr2=0x7fce700d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.762+0000 7fce7803a640 1 --2- 172.21.15.38:0/3706233156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce700d3900 0x7fce700d0850 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fce640097a0 tx=0x7fce6402a100 comp rx=0 tx=0).stop 2024-04-25T08:40:27.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.762+0000 7fce7803a640 1 -- 172.21.15.38:0/3706233156 shutdown_connections 2024-04-25T08:40:27.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.762+0000 7fce7803a640 1 --2- 172.21.15.38:0/3706233156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce700d0d90 0x7fce700b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.762+0000 7fce7803a640 1 --2- 172.21.15.38:0/3706233156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce700d3900 0x7fce700d0850 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.762+0000 7fce7803a640 1 -- 172.21.15.38:0/3706233156 >> 172.21.15.38:0/3706233156 conn(0x7fce70066d10 msgr2=0x7fce70067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.762+0000 7fce7803a640 1 -- 172.21.15.38:0/3706233156 shutdown_connections 2024-04-25T08:40:27.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.762+0000 7fce7803a640 1 -- 172.21.15.38:0/3706233156 wait complete. 2024-04-25T08:40:27.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.764+0000 7fce7803a640 1 Processor -- start 2024-04-25T08:40:27.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.764+0000 7fce7803a640 1 -- start start 2024-04-25T08:40:27.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.764+0000 7fce7803a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce700d0d90 0x7fce700cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.764+0000 7fce7803a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce700d3900 0x7fce701f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.764+0000 7fce7803a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7015dc10 0x7fce70159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.764+0000 7fce7803a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce700b6bc0 con 0x7fce700d3900 2024-04-25T08:40:27.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.764+0000 7fce7803a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce700d4710 con 0x7fce7015dc10 2024-04-25T08:40:27.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.764+0000 7fce7803a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce700cef30 con 0x7fce700d0d90 2024-04-25T08:40:27.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.764+0000 7fce755d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce700d3900 0x7fce701f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.764+0000 7fce755d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce700d3900 0x7fce701f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35926/0 (socket says 172.21.15.38:35926) 2024-04-25T08:40:27.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce755d5640 1 -- 172.21.15.38:0/4097254738 learned_addr learned my addr 172.21.15.38:0/4097254738 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:27.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce765d7640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7015dc10 0x7fce70159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce75dd6640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce700d0d90 0x7fce700cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce755d5640 1 -- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce700d0d90 msgr2=0x7fce700cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce755d5640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce700d0d90 0x7fce700cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce755d5640 1 -- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7015dc10 msgr2=0x7fce70159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce755d5640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7015dc10 0x7fce70159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce755d5640 1 -- 172.21.15.38:0/4097254738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce701f71f0 con 0x7fce700d3900 2024-04-25T08:40:27.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce765d7640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7015dc10 0x7fce70159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:27.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce755d5640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce700d3900 0x7fce701f6aa0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fce64035040 tx=0x7fce640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce6e7fc640 1 -- 172.21.15.38:0/4097254738 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64041070 con 0x7fce700d3900 2024-04-25T08:40:27.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce6e7fc640 1 -- 172.21.15.38:0/4097254738 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6403c670 con 0x7fce700d3900 2024-04-25T08:40:27.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce7803a640 1 -- 172.21.15.38:0/4097254738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce701f74d0 con 0x7fce700d3900 2024-04-25T08:40:27.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce6e7fc640 1 -- 172.21.15.38:0/4097254738 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64037040 con 0x7fce700d3900 2024-04-25T08:40:27.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.765+0000 7fce7803a640 1 -- 172.21.15.38:0/4097254738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce700aa490 con 0x7fce700d3900 2024-04-25T08:40:27.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.766+0000 7fce7803a640 1 -- 172.21.15.38:0/4097254738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce701f79f0 con 0x7fce700d3900 2024-04-25T08:40:27.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.767+0000 7fce6e7fc640 1 -- 172.21.15.38:0/4097254738 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce6403a070 con 0x7fce700d3900 2024-04-25T08:40:27.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.767+0000 7fce6e7fc640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce5c06b4d0 0x7fce5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:27.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.767+0000 7fce6e7fc640 1 -- 172.21.15.38:0/4097254738 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fce640aa730 con 0x7fce700d3900 2024-04-25T08:40:27.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.768+0000 7fce75dd6640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce5c06b4d0 0x7fce5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:27.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.768+0000 7fce75dd6640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce5c06b4d0 0x7fce5c06d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fce600012e0 tx=0x7fce60007ed0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:27.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.771+0000 7fce6e7fc640 1 -- 172.21.15.38:0/4097254738 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce701f79f0 con 0x7fce700d3900 2024-04-25T08:40:27.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.991+0000 7fce7803a640 1 -- 172.21.15.38:0/4097254738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce70068e80 con 0x7fce700d3900 2024-04-25T08:40:27.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.991+0000 7fce6e7fc640 1 -- 172.21.15.38:0/4097254738 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fce64079380 con 0x7fce700d3900 2024-04-25T08:40:27.986 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:27.986 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:27.986 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:27.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.993+0000 7fce7803a640 1 -- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce5c06b4d0 msgr2=0x7fce5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.993+0000 7fce7803a640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce5c06b4d0 0x7fce5c06d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fce600012e0 tx=0x7fce60007ed0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.993+0000 7fce7803a640 1 -- 172.21.15.38:0/4097254738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce700d3900 msgr2=0x7fce701f6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:27.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.993+0000 7fce7803a640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce700d3900 0x7fce701f6aa0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fce64035040 tx=0x7fce640046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.994+0000 7fce7803a640 1 -- 172.21.15.38:0/4097254738 shutdown_connections 2024-04-25T08:40:27.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.994+0000 7fce7803a640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce7015dc10 0x7fce70159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.994+0000 7fce7803a640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fce700d3900 0x7fce701f6aa0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.994+0000 7fce7803a640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fce5c06b4d0 0x7fce5c06d990 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.994+0000 7fce7803a640 1 --2- 172.21.15.38:0/4097254738 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fce700d0d90 0x7fce700cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:27.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.994+0000 7fce7803a640 1 -- 172.21.15.38:0/4097254738 >> 172.21.15.38:0/4097254738 conn(0x7fce70066d10 msgr2=0x7fce7006bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:27.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.994+0000 7fce7803a640 1 -- 172.21.15.38:0/4097254738 shutdown_connections 2024-04-25T08:40:27.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:27.994+0000 7fce7803a640 1 -- 172.21.15.38:0/4097254738 wait complete. 2024-04-25T08:40:27.998 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:40:27.998 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:40:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.105+0000 7fb498246640 1 Processor -- start 2024-04-25T08:40:28.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.105+0000 7fb498246640 1 -- start start 2024-04-25T08:40:28.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb498246640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb4900d0f60 0x7fb4900b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb498246640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4900d3e70 con 0x7fb4900d3900 2024-04-25T08:40:28.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb498246640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4900d4150 con 0x7fb49015dc10 2024-04-25T08:40:28.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb498246640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb49015dff0 con 0x7fb4900d0f60 2024-04-25T08:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb4957e1640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb4900d3900 0x7fb4900d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60494/0 (socket says 172.21.15.38:60494) 2024-04-25T08:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb4957e1640 1 -- 172.21.15.38:0/678266557 learned_addr learned my addr 172.21.15.38:0/678266557 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb4967e3640 1 --2- 172.21.15.38:0/678266557 >> v2:172.21.15.107:3300/0 conn(0x7fb4900d0f60 0x7fb4900b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb494fe0640 1 -- 172.21.15.38:0/678266557 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282343696 0 0) 0x7fb4900d3e70 con 0x7fb4900d3900 2024-04-25T08:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb494fe0640 1 -- 172.21.15.38:0/678266557 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb47c003520 con 0x7fb4900d3900 2024-04-25T08:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb494fe0640 1 -- 172.21.15.38:0/678266557 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1591224074 0 0) 0x7fb4900d4150 con 0x7fb49015dc10 2024-04-25T08:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb494fe0640 1 -- 172.21.15.38:0/678266557 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4900d3e70 con 0x7fb49015dc10 2024-04-25T08:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb4967e3640 1 -- 172.21.15.38:0/678266557 >> v1:172.21.15.107:6789/0 conn(0x7fb49015dc10 legacy=0x7fb4900ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb4967e3640 1 -- 172.21.15.38:0/678266557 >> v1:172.21.15.38:6789/0 conn(0x7fb4900d3900 legacy=0x7fb4900d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb494fe0640 1 -- 172.21.15.38:0/678266557 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2356755980 0 0) 0x7fb47c003520 con 0x7fb4900d3900 2024-04-25T08:40:28.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.106+0000 7fb4967e3640 1 -- 172.21.15.38:0/678266557 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900d4430 con 0x7fb4900d0f60 2024-04-25T08:40:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.107+0000 7fb4967e3640 1 --2- 172.21.15.38:0/678266557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4900d0f60 0x7fb4900b2370 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb488009d90 tx=0x7fb48802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=635556cbe216e775 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.107+0000 7fb494fe0640 1 -- 172.21.15.38:0/678266557 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480001670 con 0x7fb4900d0f60 2024-04-25T08:40:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.107+0000 7fb494fe0640 1 -- 172.21.15.38:0/678266557 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb488007540 con 0x7fb4900d0f60 2024-04-25T08:40:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.107+0000 7fb494fe0640 1 -- 172.21.15.38:0/678266557 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4880078f0 con 0x7fb4900d0f60 2024-04-25T08:40:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.108+0000 7fb498246640 1 -- 172.21.15.38:0/678266557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4900d0f60 msgr2=0x7fb4900b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.108+0000 7fb498246640 1 --2- 172.21.15.38:0/678266557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4900d0f60 0x7fb4900b2370 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb488009d90 tx=0x7fb48802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.108+0000 7fb498246640 1 -- 172.21.15.38:0/678266557 shutdown_connections 2024-04-25T08:40:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.108+0000 7fb498246640 1 --2- 172.21.15.38:0/678266557 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4900d0f60 0x7fb4900b2370 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.108+0000 7fb498246640 1 -- 172.21.15.38:0/678266557 >> 172.21.15.38:0/678266557 conn(0x7fb490066d10 msgr2=0x7fb490067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.108+0000 7fb498246640 1 -- 172.21.15.38:0/678266557 shutdown_connections 2024-04-25T08:40:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.108+0000 7fb498246640 1 -- 172.21.15.38:0/678266557 wait complete. 2024-04-25T08:40:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.109+0000 7fb498246640 1 Processor -- start 2024-04-25T08:40:28.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.109+0000 7fb498246640 1 -- start start 2024-04-25T08:40:28.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb498246640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4900d0f60 0x7fb49014bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb498246640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4900d3900 0x7fb4900cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb498246640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb49015dc10 0x7fb4900ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb498246640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900d4710 con 0x7fb4900d3900 2024-04-25T08:40:28.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb498246640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900cef30 con 0x7fb49015dc10 2024-04-25T08:40:28.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb498246640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4900b80f0 con 0x7fb4900d0f60 2024-04-25T08:40:28.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb4957e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4900d3900 0x7fb4900cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb4957e1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4900d3900 0x7fb4900cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35932/0 (socket says 172.21.15.38:35932) 2024-04-25T08:40:28.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb4957e1640 1 -- 172.21.15.38:0/2199973155 learned_addr learned my addr 172.21.15.38:0/2199973155 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb495fe2640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4900d0f60 0x7fb49014bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb4967e3640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb49015dc10 0x7fb4900ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb4957e1640 1 -- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4900d0f60 msgr2=0x7fb49014bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb4957e1640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4900d0f60 0x7fb49014bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb4957e1640 1 -- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb49015dc10 msgr2=0x7fb4900ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb4957e1640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb49015dc10 0x7fb4900ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.110+0000 7fb4957e1640 1 -- 172.21.15.38:0/2199973155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4900d2640 con 0x7fb4900d3900 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.111+0000 7fb4957e1640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4900d3900 0x7fb4900cde30 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fb480000fd0 tx=0x7fb48000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.111+0000 7fb4867fc640 1 -- 172.21.15.38:0/2199973155 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480017070 con 0x7fb4900d3900 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.111+0000 7fb4867fc640 1 -- 172.21.15.38:0/2199973155 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb480005ad0 con 0x7fb4900d3900 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.111+0000 7fb498246640 1 -- 172.21.15.38:0/2199973155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb49014fc90 con 0x7fb4900d3900 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.111+0000 7fb4867fc640 1 -- 172.21.15.38:0/2199973155 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480007070 con 0x7fb4900d3900 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.111+0000 7fb498246640 1 -- 172.21.15.38:0/2199973155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb49014ff00 con 0x7fb4900d3900 2024-04-25T08:40:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.112+0000 7fb498246640 1 -- 172.21.15.38:0/2199973155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb490068e80 con 0x7fb4900d3900 2024-04-25T08:40:28.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.113+0000 7fb4867fc640 1 -- 172.21.15.38:0/2199973155 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb48001a070 con 0x7fb4900d3900 2024-04-25T08:40:28.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.113+0000 7fb4867fc640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb47406b500 0x7fb47406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.113+0000 7fb4867fc640 1 -- 172.21.15.38:0/2199973155 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb480089740 con 0x7fb4900d3900 2024-04-25T08:40:28.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.113+0000 7fb495fe2640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb47406b500 0x7fb47406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.114+0000 7fb495fe2640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb47406b500 0x7fb47406d9c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0011e0 tx=0x7fb48c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.117+0000 7fb4867fc640 1 -- 172.21.15.38:0/2199973155 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb480057e70 con 0x7fb4900d3900 2024-04-25T08:40:28.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.372+0000 7fb498246640 1 -- 172.21.15.38:0/2199973155 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb4900ceae0 con 0x7fb4900d3900 2024-04-25T08:40:28.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.372+0000 7fb4867fc640 1 -- 172.21.15.38:0/2199973155 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fb48005bf10 con 0x7fb4900d3900 2024-04-25T08:40:28.367 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:28.367 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":557,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076550540,"bytes_used":2673422336,"bytes_avail":770420690944,"bytes_total":773094113280,"write_bytes_sec":63620137,"read_op_per_sec":0,"write_op_per_sec":64},"fsmap":{"epoch":70,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":7639}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:40:25.328586+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T08:40:28.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.374+0000 7fb498246640 1 -- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb47406b500 msgr2=0x7fb47406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.374+0000 7fb498246640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb47406b500 0x7fb47406d9c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0011e0 tx=0x7fb48c009040 comp rx=0 tx=0).stop 2024-04-25T08:40:28.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.374+0000 7fb498246640 1 -- 172.21.15.38:0/2199973155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4900d3900 msgr2=0x7fb4900cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.374+0000 7fb498246640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4900d3900 0x7fb4900cde30 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fb480000fd0 tx=0x7fb48000cbe0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.374+0000 7fb498246640 1 -- 172.21.15.38:0/2199973155 shutdown_connections 2024-04-25T08:40:28.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.374+0000 7fb498246640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb49015dc10 0x7fb4900ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.374+0000 7fb498246640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4900d3900 0x7fb4900cde30 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.374+0000 7fb498246640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb47406b500 0x7fb47406d9c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.374+0000 7fb498246640 1 --2- 172.21.15.38:0/2199973155 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4900d0f60 0x7fb49014bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.374+0000 7fb498246640 1 -- 172.21.15.38:0/2199973155 >> 172.21.15.38:0/2199973155 conn(0x7fb490066d10 msgr2=0x7fb49015cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.374+0000 7fb498246640 1 -- 172.21.15.38:0/2199973155 shutdown_connections 2024-04-25T08:40:28.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.374+0000 7fb498246640 1 -- 172.21.15.38:0/2199973155 wait complete. 2024-04-25T08:40:28.379 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T08:40:28.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:28.379 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-25T08:40:28.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.488+0000 7f5b19d7e640 1 Processor -- start 2024-04-25T08:40:28.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.488+0000 7f5b19d7e640 1 -- start start 2024-04-25T08:40:28.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.489+0000 7f5b19d7e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5b140d0320 0x7f5b140d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.489+0000 7f5b19d7e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5b140d0700 0x7f5b140d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.489+0000 7f5b19d7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140d1e90 0x7f5b140b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.489+0000 7f5b19d7e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b140d2330 con 0x7f5b140d0700 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.489+0000 7f5b19d7e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b140d2610 con 0x7f5b140d0320 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.489+0000 7f5b19d7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140d28f0 con 0x7f5b140d1e90 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.489+0000 7f5b12ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5b140d0700 0x7f5b140d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.489+0000 7f5b12ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5b140d0700 0x7f5b140d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35948/0 (socket says 172.21.15.38:35948) 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.489+0000 7f5b12ffd640 1 -- 172.21.15.38:0/2477792562 learned_addr learned my addr 172.21.15.38:0/2477792562 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.489+0000 7f5b137fe640 1 --2- 172.21.15.38:0/2477792562 >> v2:172.21.15.107:3300/0 conn(0x7f5b140d0320 0x7f5b140d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.489+0000 7f5b13fff640 1 --2- 172.21.15.38:0/2477792562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140d1e90 0x7f5b140b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.490+0000 7f5b12ffd640 1 -- 172.21.15.38:0/2477792562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140d1e90 msgr2=0x7f5b140b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.490+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/2477792562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140d1e90 0x7f5b140b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.490+0000 7f5b12ffd640 1 -- 172.21.15.38:0/2477792562 >> v2:172.21.15.107:3300/0 conn(0x7f5b140d0320 msgr2=0x7f5b140d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.490+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/2477792562 >> v2:172.21.15.107:3300/0 conn(0x7f5b140d0320 0x7f5b140d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.490+0000 7f5b12ffd640 1 -- 172.21.15.38:0/2477792562 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140d2bd0 con 0x7f5b140d0700 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.490+0000 7f5b137fe640 1 --2- 172.21.15.38:0/2477792562 >> v2:172.21.15.107:3300/0 conn(0x7f5b140d0320 0x7f5b140d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:28.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.490+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/2477792562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140d0700 0x7f5b140d1950 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5b14069100 tx=0x7f5b0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3791fb63c410813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.490+0000 7f5b127fc640 1 -- 172.21.15.38:0/2477792562 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08037070 con 0x7f5b140d0700 2024-04-25T08:40:28.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.490+0000 7f5b127fc640 1 -- 172.21.15.38:0/2477792562 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b08032440 con 0x7f5b140d0700 2024-04-25T08:40:28.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.490+0000 7f5b127fc640 1 -- 172.21.15.38:0/2477792562 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08032810 con 0x7f5b140d0700 2024-04-25T08:40:28.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.491+0000 7f5b19d7e640 1 -- 172.21.15.38:0/2477792562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140d0700 msgr2=0x7f5b140d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.491+0000 7f5b19d7e640 1 --2- 172.21.15.38:0/2477792562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140d0700 0x7f5b140d1950 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f5b14069100 tx=0x7f5b0802a270 comp rx=0 tx=0).stop 2024-04-25T08:40:28.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.491+0000 7f5b19d7e640 1 -- 172.21.15.38:0/2477792562 shutdown_connections 2024-04-25T08:40:28.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.491+0000 7f5b19d7e640 1 --2- 172.21.15.38:0/2477792562 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140d1e90 0x7f5b140b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.491+0000 7f5b19d7e640 1 --2- 172.21.15.38:0/2477792562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140d0700 0x7f5b140d1950 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.491+0000 7f5b19d7e640 1 --2- 172.21.15.38:0/2477792562 >> v2:172.21.15.107:3300/0 conn(0x7f5b140d0320 0x7f5b140d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.491+0000 7f5b19d7e640 1 -- 172.21.15.38:0/2477792562 >> 172.21.15.38:0/2477792562 conn(0x7f5b140668f0 msgr2=0x7f5b14066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.491+0000 7f5b19d7e640 1 -- 172.21.15.38:0/2477792562 shutdown_connections 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.491+0000 7f5b19d7e640 1 -- 172.21.15.38:0/2477792562 wait complete. 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.492+0000 7f5b19d7e640 1 Processor -- start 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.492+0000 7f5b19d7e640 1 -- start start 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.493+0000 7f5b19d7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140d0320 0x7f5b140b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.493+0000 7f5b19d7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140d0700 0x7f5b1414dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.493+0000 7f5b19d7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b140d1e90 0x7f5b141e8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.493+0000 7f5b19d7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140d0b40 con 0x7f5b140d0320 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.493+0000 7f5b19d7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14161210 con 0x7f5b140d1e90 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.493+0000 7f5b19d7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b140b4d50 con 0x7f5b140d0700 2024-04-25T08:40:28.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.493+0000 7f5b137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140d0320 0x7f5b140b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.493+0000 7f5b137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140d0320 0x7f5b140b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35962/0 (socket says 172.21.15.38:35962) 2024-04-25T08:40:28.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.493+0000 7f5b137fe640 1 -- 172.21.15.38:0/1737498973 learned_addr learned my addr 172.21.15.38:0/1737498973 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.493+0000 7f5b13fff640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b140d1e90 0x7f5b141e8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.494+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140d0700 0x7f5b1414dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.494+0000 7f5b137fe640 1 -- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140d0700 msgr2=0x7f5b1414dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.494+0000 7f5b137fe640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140d0700 0x7f5b1414dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.494+0000 7f5b137fe640 1 -- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b140d1e90 msgr2=0x7f5b141e8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.494+0000 7f5b137fe640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b140d1e90 0x7f5b141e8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.494+0000 7f5b137fe640 1 -- 172.21.15.38:0/1737498973 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b140b4200 con 0x7f5b140d0320 2024-04-25T08:40:28.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.494+0000 7f5b137fe640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140d0320 0x7f5b140b3cc0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400e870 tx=0x7f5b0400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.494+0000 7f5afbfff640 1 -- 172.21.15.38:0/1737498973 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0400f040 con 0x7f5b140d0320 2024-04-25T08:40:28.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.494+0000 7f5afbfff640 1 -- 172.21.15.38:0/1737498973 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b04004230 con 0x7f5b140d0320 2024-04-25T08:40:28.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.494+0000 7f5afbfff640 1 -- 172.21.15.38:0/1737498973 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04009e30 con 0x7f5b140d0320 2024-04-25T08:40:28.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.494+0000 7f5b19d7e640 1 -- 172.21.15.38:0/1737498973 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b140b44b0 con 0x7f5b140d0320 2024-04-25T08:40:28.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.494+0000 7f5b19d7e640 1 -- 172.21.15.38:0/1737498973 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b140b4a70 con 0x7f5b140d0320 2024-04-25T08:40:28.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.495+0000 7f5afbfff640 1 -- 172.21.15.38:0/1737498973 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b0400c6d0 con 0x7f5b140d0320 2024-04-25T08:40:28.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.496+0000 7f5afbfff640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b0006b4d0 0x7f5b0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.496+0000 7f5afbfff640 1 -- 172.21.15.38:0/1737498973 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b04089ff0 con 0x7f5b140d0320 2024-04-25T08:40:28.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.496+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b0006b4d0 0x7f5b0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.497+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b0006b4d0 0x7f5b0006d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5b08034000 tx=0x7f5b0802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.497+0000 7f5b1ca82740 1 Processor -- start 2024-04-25T08:40:28.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.497+0000 7f5b1ca82740 1 -- start start 2024-04-25T08:40:28.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b1ca82740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 0x557f2108e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b1ca82740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f2108bef0 0x557f2108e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b1ca82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2105e770 0x557f21061f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b1ca82740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f20fc7da0 con 0x557f2108bb10 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b1ca82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f20f70b80 con 0x557f2105e770 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b1ca82740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f20fc3e40 con 0x557f2108bef0 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 0x557f2108e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 0x557f2108e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35968/0 (socket says 172.21.15.38:35968) 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b137fe640 1 -- 172.21.15.38:0/3807906892 learned_addr learned my addr 172.21.15.38:0/3807906892 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b13fff640 1 --2- 172.21.15.38:0/3807906892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2105e770 0x557f21061f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/3807906892 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f2108bef0 0x557f2108e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b137fe640 1 -- 172.21.15.38:0/3807906892 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f2108bef0 msgr2=0x557f2108e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b137fe640 1 --2- 172.21.15.38:0/3807906892 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f2108bef0 0x557f2108e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b137fe640 1 -- 172.21.15.38:0/3807906892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2105e770 msgr2=0x557f21061f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b137fe640 1 --2- 172.21.15.38:0/3807906892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2105e770 0x557f21061f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b137fe640 1 -- 172.21.15.38:0/3807906892 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f20f7af10 con 0x557f2108bb10 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5b137fe640 1 --2- 172.21.15.38:0/3807906892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 0x557f2108e320 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5b04006fd0 tx=0x7f5b040585e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62c1e9c3c6ebd63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5af97fa640 1 -- 172.21.15.38:0/3807906892 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04070020 con 0x557f2108bb10 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5af97fa640 1 -- 172.21.15.38:0/3807906892 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b040568e0 con 0x557f2108bb10 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.498+0000 7f5af97fa640 1 -- 172.21.15.38:0/3807906892 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04020040 con 0x557f2108bb10 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.499+0000 7f5b1ca82740 1 -- 172.21.15.38:0/3807906892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 msgr2=0x557f2108e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.499+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/3807906892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 0x557f2108e320 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5b04006fd0 tx=0x7f5b040585e0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.499+0000 7f5b1ca82740 1 -- 172.21.15.38:0/3807906892 shutdown_connections 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.499+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/3807906892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2105e770 0x557f21061f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.499+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/3807906892 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f2108bef0 0x557f2108e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.499+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/3807906892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 0x557f2108e320 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.499+0000 7f5b1ca82740 1 -- 172.21.15.38:0/3807906892 >> 172.21.15.38:0/3807906892 conn(0x557f210609e0 msgr2=0x557f210a9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.499+0000 7f5b1ca82740 1 -- 172.21.15.38:0/3807906892 shutdown_connections 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.499+0000 7f5b1ca82740 1 -- 172.21.15.38:0/3807906892 wait complete. 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.500+0000 7f5b1ca82740 1 -- start start 2024-04-25T08:40:28.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.500+0000 7f5b1ca82740 1 Processor -- start 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.500+0000 7f5b1ca82740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 0x557f210b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.500+0000 7f5b1ca82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2108bef0 0x557f21073b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b1ca82740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f21082db0 0x557f210ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b1ca82740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f21063b90 con 0x557f2108bb10 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b1ca82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f20f664b0 con 0x557f2108bef0 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b1ca82740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f21063e70 con 0x557f21082db0 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b1ca82740 10 client.0 _ll_register_callbacks cb 0x557f20fa6180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 0x557f210b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b137fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 0x557f210b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35982/0 (socket says 172.21.15.38:35982) 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b137fe640 1 -- 172.21.15.38:0/1089068309 learned_addr learned my addr 172.21.15.38:0/1089068309 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2108bef0 0x557f21073b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b13fff640 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f21082db0 0x557f210ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b137fe640 1 -- 172.21.15.38:0/1089068309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f21082db0 msgr2=0x557f210ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b137fe640 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f21082db0 0x557f210ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b137fe640 1 -- 172.21.15.38:0/1089068309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2108bef0 msgr2=0x557f21073b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b137fe640 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2108bef0 0x557f21073b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b137fe640 1 -- 172.21.15.38:0/1089068309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f20da67d0 con 0x557f2108bb10 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b137fe640 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 0x557f210b06b0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5b04018000 tx=0x7f5b0401d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e2397440cd4d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5af27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5af27fc640 1 -- 172.21.15.38:0/1089068309 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b04070050 con 0x557f2108bb10 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5af27fc640 1 -- 172.21.15.38:0/1089068309 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b04057c30 con 0x557f2108bb10 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b1ca82740 1 -- 172.21.15.38:0/1089068309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b040568e0 con 0x557f2108bb10 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5af27fc640 1 -- 172.21.15.38:0/1089068309 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b04020040 con 0x557f2108bb10 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b1ca82740 10 client.9219 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5b1ca82740 1 -- 172.21.15.38:0/1089068309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x557f20cdc3f0 con 0x557f2108bb10 2024-04-25T08:40:28.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.501+0000 7f5af27fc640 1 -- 172.21.15.38:0/1089068309 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b040568e0 con 0x557f2108bb10 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.502+0000 7f5af27fc640 1 -- 172.21.15.38:0/1089068309 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5b04012050 con 0x557f2108bb10 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.502+0000 7f5b1ca82740 10 client.9219 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.502+0000 7f5b1ca82740 10 client.9219 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.502+0000 7f5b1ca82740 20 client.9219 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.502+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x557f210acb30 0x557f210acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.502+0000 7f5b1ca82740 4 client.9219 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.502+0000 7f5b1ca82740 1 -- 172.21.15.38:0/1089068309 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f20fd52e0 con 0x557f210acb30 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.502+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x557f210acb30 0x557f210acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.502+0000 7f5af27fc640 10 client.9219 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.502+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x557f210acb30 0x557f210acf40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af9bb1746ed2aa7f server_cookie=fcfc21baa62846b1 in_seq=0 out_seq=0 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5af27fc640 1 -- 172.21.15.38:0/1089068309 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557f20fd52e0 con 0x557f210acb30 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5af27fc640 10 client.9219 handle_command_reply: tid=0 2024-04-25T08:40:28.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5b1ca82740 1 client.9219 shutdown 2024-04-25T08:40:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5b1ca82740 1 -- 172.21.15.38:0/1089068309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 msgr2=0x557f210b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 0x557f210b06b0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f5b04018000 tx=0x7f5b0401d070 comp rx=0 tx=0).stop 2024-04-25T08:40:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5b1ca82740 1 -- 172.21.15.38:0/1089068309 shutdown_connections 2024-04-25T08:40:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x557f210acb30 0x557f210acf40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f21082db0 0x557f210ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2108bef0 0x557f21073b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5af27fc640 0 client.9219 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/1089068309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bb10 0x557f210b06b0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5b1ca82740 1 -- 172.21.15.38:0/1089068309 >> 172.21.15.38:0/1089068309 conn(0x557f2105e770 msgr2=0x557f2105f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5b1ca82740 1 -- 172.21.15.38:0/1089068309 shutdown_connections 2024-04-25T08:40:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5b1ca82740 1 -- 172.21.15.38:0/1089068309 wait complete. 2024-04-25T08:40:28.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.503+0000 7f5b1ca82740 20 client.9219 trim_cache size 0 max 16384 2024-04-25T08:40:28.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.518+0000 7f5b1ca82740 1 Processor -- start 2024-04-25T08:40:28.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.518+0000 7f5b1ca82740 1 -- start start 2024-04-25T08:40:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b1ca82740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f21082db0 0x557f2108f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b1ca82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2108bb10 0x557f210821d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b1ca82740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 0x557f21082710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b1ca82740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14200420 con 0x557f2108bef0 2024-04-25T08:40:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b1ca82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b141ff5a0 con 0x557f2108bb10 2024-04-25T08:40:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b1ca82740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b141e7690 con 0x557f21082db0 2024-04-25T08:40:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b13fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 0x557f21082710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b13fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 0x557f21082710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35992/0 (socket says 172.21.15.38:35992) 2024-04-25T08:40:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b13fff640 1 -- 172.21.15.38:0/2679048211 learned_addr learned my addr 172.21.15.38:0/2679048211 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/2679048211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2108bb10 0x557f210821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b137fe640 1 --2- 172.21.15.38:0/2679048211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f21082db0 0x557f2108f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b13fff640 1 -- 172.21.15.38:0/2679048211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f21082db0 msgr2=0x557f2108f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b13fff640 1 --2- 172.21.15.38:0/2679048211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f21082db0 0x557f2108f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b13fff640 1 -- 172.21.15.38:0/2679048211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2108bb10 msgr2=0x557f210821d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b13fff640 1 --2- 172.21.15.38:0/2679048211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2108bb10 0x557f210821d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b13fff640 1 -- 172.21.15.38:0/2679048211 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f20c9c0f0 con 0x557f2108bef0 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.519+0000 7f5b137fe640 1 --2- 172.21.15.38:0/2679048211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f21082db0 0x557f2108f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5b13fff640 1 --2- 172.21.15.38:0/2679048211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 0x557f21082710 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009f10 tx=0x7f5b0c015700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2559e793d18be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5af8ff9640 1 -- 172.21.15.38:0/2679048211 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c017820 con 0x557f2108bef0 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5af8ff9640 1 -- 172.21.15.38:0/2679048211 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0179c0 con 0x557f2108bef0 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5af8ff9640 1 -- 172.21.15.38:0/2679048211 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c017db0 con 0x557f2108bef0 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5b1ca82740 1 -- 172.21.15.38:0/2679048211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 msgr2=0x557f21082710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/2679048211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 0x557f21082710 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c009f10 tx=0x7f5b0c015700 comp rx=0 tx=0).stop 2024-04-25T08:40:28.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5b1ca82740 1 -- 172.21.15.38:0/2679048211 shutdown_connections 2024-04-25T08:40:28.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/2679048211 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 0x557f21082710 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/2679048211 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f2108bb10 0x557f210821d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/2679048211 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f21082db0 0x557f2108f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5b1ca82740 1 -- 172.21.15.38:0/2679048211 >> 172.21.15.38:0/2679048211 conn(0x557f2105e770 msgr2=0x557f210afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5b1ca82740 1 -- 172.21.15.38:0/2679048211 shutdown_connections 2024-04-25T08:40:28.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.520+0000 7f5b1ca82740 1 -- 172.21.15.38:0/2679048211 wait complete. 2024-04-25T08:40:28.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.521+0000 7f5b1ca82740 1 -- start start 2024-04-25T08:40:28.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.521+0000 7f5b1ca82740 1 Processor -- start 2024-04-25T08:40:28.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b1ca82740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f21082db0 0x557f21112880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b1ca82740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f2108bb10 0x557f2105d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b1ca82740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 0x557f210b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b1ca82740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f210acfb0 con 0x557f2108bef0 2024-04-25T08:40:28.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b1ca82740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b141ff000 con 0x557f21082db0 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b1ca82740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f210a89b0 con 0x557f2108bb10 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b1ca82740 10 client.0 _ll_register_callbacks cb 0x7f5b04054880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b13fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 0x557f210b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b13fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 0x557f210b7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36000/0 (socket says 172.21.15.38:36000) 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b13fff640 1 -- 172.21.15.38:0/295218264 learned_addr learned my addr 172.21.15.38:0/295218264 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b137fe640 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f21082db0 0x557f21112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f2108bb10 0x557f2105d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b13fff640 1 -- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f2108bb10 msgr2=0x557f2105d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b13fff640 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f2108bb10 0x557f2105d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b13fff640 1 -- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f21082db0 msgr2=0x557f21112880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b13fff640 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f21082db0 0x557f21112880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b13fff640 1 -- 172.21.15.38:0/295218264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f2105ebb0 con 0x557f2108bef0 2024-04-25T08:40:28.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b12ffd640 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f2108bb10 0x557f2105d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5b13fff640 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 0x557f210b7c70 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00b010 tx=0x7f5b0c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68f94859e816abf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.522+0000 7f5af27fc640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5af27fc640 1 -- 172.21.15.38:0/295218264 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c030060 con 0x557f2108bef0 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5af27fc640 1 -- 172.21.15.38:0/295218264 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b0c017e00 con 0x557f2108bef0 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5b1ca82740 1 -- 172.21.15.38:0/295218264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b0c0179c0 con 0x557f2108bef0 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5af27fc640 1 -- 172.21.15.38:0/295218264 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b0c036050 con 0x557f2108bef0 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5b1ca82740 10 client.9225 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5b1ca82740 1 -- 172.21.15.38:0/295218264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x557f210b84a0 con 0x557f2108bef0 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5af27fc640 1 -- 172.21.15.38:0/295218264 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0179c0 con 0x557f2108bef0 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5af27fc640 1 -- 172.21.15.38:0/295218264 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5b0c039020 con 0x557f2108bef0 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5b1ca82740 10 client.9225 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5b1ca82740 10 client.9225 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:28.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5b1ca82740 20 client.9225 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x557f2105f370 0x557f210821d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5b1ca82740 4 client.9225 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5b1ca82740 1 -- 172.21.15.38:0/295218264 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x557f20fcf630 con 0x557f2105f370 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.523+0000 7f5b137fe640 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x557f2105f370 0x557f210821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5af27fc640 10 client.9225 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b137fe640 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x557f2105f370 0x557f210821d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6124ebbff97be4b server_cookie=e829918ee184009d in_seq=0 out_seq=0 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5af27fc640 1 -- 172.21.15.38:0/295218264 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x557f20fcf630 con 0x557f2105f370 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5af27fc640 10 client.9225 handle_command_reply: tid=0 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b1ca82740 1 client.9225 shutdown 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b1ca82740 1 -- 172.21.15.38:0/295218264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 msgr2=0x557f210b7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 0x557f210b7c70 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00b010 tx=0x7f5b0c018000 comp rx=0 tx=0).stop 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b1ca82740 1 -- 172.21.15.38:0/295218264 shutdown_connections 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x557f2105f370 0x557f210821d0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557f2108bef0 0x557f210b7c70 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x557f2108bb10 0x557f2105d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b1ca82740 1 --2- 172.21.15.38:0/295218264 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557f21082db0 0x557f21112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5af27fc640 0 client.9225 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b1ca82740 1 -- 172.21.15.38:0/295218264 >> 172.21.15.38:0/295218264 conn(0x557f2105e770 msgr2=0x557f21059d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b1ca82740 1 -- 172.21.15.38:0/295218264 shutdown_connections 2024-04-25T08:40:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b1ca82740 1 -- 172.21.15.38:0/295218264 wait complete. 2024-04-25T08:40:28.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.524+0000 7f5b1ca82740 20 client.9225 trim_cache size 0 max 16384 2024-04-25T08:40:28.525 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:40:28.525 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:40:28.525 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:40:28.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.525+0000 7f5b19d7e640 1 -- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b0006b4d0 msgr2=0x7f5b0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.525+0000 7f5b19d7e640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b0006b4d0 0x7f5b0006d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f5b08034000 tx=0x7f5b0802e000 comp rx=0 tx=0).stop 2024-04-25T08:40:28.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.525+0000 7f5b19d7e640 1 -- 172.21.15.38:0/1737498973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140d0320 msgr2=0x7f5b140b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.525+0000 7f5b19d7e640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140d0320 0x7f5b140b3cc0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400e870 tx=0x7f5b0400ed30 comp rx=0 tx=0).stop 2024-04-25T08:40:28.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.525+0000 7f5b19d7e640 1 -- 172.21.15.38:0/1737498973 shutdown_connections 2024-04-25T08:40:28.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.525+0000 7f5b19d7e640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5b140d1e90 0x7f5b141e8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.525+0000 7f5b19d7e640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5b0006b4d0 0x7f5b0006d990 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.525+0000 7f5b19d7e640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5b140d0700 0x7f5b1414dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.525+0000 7f5b19d7e640 1 --2- 172.21.15.38:0/1737498973 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5b140d0320 0x7f5b140b3cc0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.525+0000 7f5b19d7e640 1 -- 172.21.15.38:0/1737498973 >> 172.21.15.38:0/1737498973 conn(0x7f5b140668f0 msgr2=0x7f5b140b7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.525+0000 7f5b19d7e640 1 -- 172.21.15.38:0/1737498973 shutdown_connections 2024-04-25T08:40:28.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.525+0000 7f5b19d7e640 1 -- 172.21.15.38:0/1737498973 wait complete. 2024-04-25T08:40:28.529 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:40:28.529 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T08:40:28.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.638+0000 7f2813efc640 1 Processor -- start 2024-04-25T08:40:28.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.638+0000 7f2813efc640 1 -- start start 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2813efc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f280c0d0320 0x7f280c0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2813efc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f280c0d0700 0x7f280c0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2813efc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f280c0d1e90 0x7f280c0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2813efc640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280c0d2330 con 0x7f280c0d0320 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2813efc640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280c0d2610 con 0x7f280c0d1e90 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2813efc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0d28f0 con 0x7f280c0d0700 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2811c98640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f280c0d0320 0x7f280c0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2811c98640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f280c0d0320 0x7f280c0d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36006/0 (socket says 172.21.15.38:36006) 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2811c98640 1 -- 172.21.15.38:0/397300010 learned_addr learned my addr 172.21.15.38:0/397300010 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2812499640 1 --2- 172.21.15.38:0/397300010 >> v2:172.21.15.107:3300/0 conn(0x7f280c0d1e90 0x7f280c0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2811497640 1 --2- 172.21.15.38:0/397300010 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f280c0d0700 0x7f280c0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2811c98640 1 -- 172.21.15.38:0/397300010 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f280c0d0700 msgr2=0x7f280c0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.635 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2811c98640 1 --2- 172.21.15.38:0/397300010 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f280c0d0700 0x7f280c0d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2811c98640 1 -- 172.21.15.38:0/397300010 >> v2:172.21.15.107:3300/0 conn(0x7f280c0d1e90 msgr2=0x7f280c0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2811c98640 1 --2- 172.21.15.38:0/397300010 >> v2:172.21.15.107:3300/0 conn(0x7f280c0d1e90 0x7f280c0b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.639+0000 7f2811c98640 1 -- 172.21.15.38:0/397300010 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c0d2bd0 con 0x7f280c0d0320 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.640+0000 7f2811c98640 1 --2- 172.21.15.38:0/397300010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c0d0320 0x7f280c0d33d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f280c0b68d0 tx=0x7f280802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f898f625a2306a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.640+0000 7f2810c96640 1 -- 172.21.15.38:0/397300010 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808037070 con 0x7f280c0d0320 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.640+0000 7f2810c96640 1 -- 172.21.15.38:0/397300010 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2808032440 con 0x7f280c0d0320 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.640+0000 7f2810c96640 1 -- 172.21.15.38:0/397300010 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808032810 con 0x7f280c0d0320 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.641+0000 7f2813efc640 1 -- 172.21.15.38:0/397300010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c0d0320 msgr2=0x7f280c0d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.641+0000 7f2813efc640 1 --2- 172.21.15.38:0/397300010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c0d0320 0x7f280c0d33d0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f280c0b68d0 tx=0x7f280802a270 comp rx=0 tx=0).stop 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.641+0000 7f2813efc640 1 -- 172.21.15.38:0/397300010 shutdown_connections 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.641+0000 7f2813efc640 1 --2- 172.21.15.38:0/397300010 >> v2:172.21.15.107:3300/0 conn(0x7f280c0d1e90 0x7f280c0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.641+0000 7f2813efc640 1 --2- 172.21.15.38:0/397300010 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f280c0d0700 0x7f280c0d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.641+0000 7f2813efc640 1 --2- 172.21.15.38:0/397300010 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c0d0320 0x7f280c0d33d0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.641+0000 7f2813efc640 1 -- 172.21.15.38:0/397300010 >> 172.21.15.38:0/397300010 conn(0x7f280c0668f0 msgr2=0x7f280c066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.641+0000 7f2813efc640 1 -- 172.21.15.38:0/397300010 shutdown_connections 2024-04-25T08:40:28.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.641+0000 7f2813efc640 1 -- 172.21.15.38:0/397300010 wait complete. 2024-04-25T08:40:28.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.642+0000 7f2813efc640 1 Processor -- start 2024-04-25T08:40:28.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.642+0000 7f2813efc640 1 -- start start 2024-04-25T08:40:28.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2813efc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f280c0d0320 0x7f280c1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2813efc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c0d0700 0x7f280c15ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2813efc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f280c0d1e90 0x7f280c15c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2813efc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0d0b40 con 0x7f280c0d0700 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2813efc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c161210 con 0x7f280c0d1e90 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2813efc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c1f4cb0 con 0x7f280c0d0320 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2811497640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c0d0700 0x7f280c15ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2811497640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c0d0700 0x7f280c15ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36008/0 (socket says 172.21.15.38:36008) 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2811497640 1 -- 172.21.15.38:0/3190442192 learned_addr learned my addr 172.21.15.38:0/3190442192 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2812499640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f280c0d1e90 0x7f280c15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2811c98640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f280c0d0320 0x7f280c1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2811497640 1 -- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f280c0d0320 msgr2=0x7f280c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2811497640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f280c0d0320 0x7f280c1534c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2811497640 1 -- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f280c0d1e90 msgr2=0x7f280c15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2811497640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f280c0d1e90 0x7f280c15c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2811497640 1 -- 172.21.15.38:0/3190442192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c1f7180 con 0x7f280c0d0700 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2812499640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f280c0d1e90 0x7f280c15c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:28.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2811497640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c0d0700 0x7f280c15ae20 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00e8c0 tx=0x7f27fc00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f28027fc640 1 -- 172.21.15.38:0/3190442192 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc017070 con 0x7f280c0d0700 2024-04-25T08:40:28.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f28027fc640 1 -- 172.21.15.38:0/3190442192 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc00c940 con 0x7f280c0d0700 2024-04-25T08:40:28.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2813efc640 1 -- 172.21.15.38:0/3190442192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c1f7420 con 0x7f280c0d0700 2024-04-25T08:40:28.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f28027fc640 1 -- 172.21.15.38:0/3190442192 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc00cd10 con 0x7f280c0d0700 2024-04-25T08:40:28.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.643+0000 7f2813efc640 1 -- 172.21.15.38:0/3190442192 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c1f79e0 con 0x7f280c0d0700 2024-04-25T08:40:28.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.645+0000 7f28027fc640 1 -- 172.21.15.38:0/3190442192 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27fc01a070 con 0x7f280c0d0700 2024-04-25T08:40:28.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.645+0000 7f28027fc640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f27f006b4d0 0x7f27f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.646+0000 7f28027fc640 1 -- 172.21.15.38:0/3190442192 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27fc08a2d0 con 0x7f280c0d0700 2024-04-25T08:40:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.646+0000 7f2811c98640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f27f006b4d0 0x7f27f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.646+0000 7f2811c98640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f27f006b4d0 0x7f27f006d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2808034000 tx=0x7f280802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.646+0000 7f2816c01740 1 Processor -- start 2024-04-25T08:40:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.646+0000 7f2816c01740 1 -- start start 2024-04-25T08:40:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2816c01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 0x555bdcebe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2816c01740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcebe860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2816c01740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdce8e770 0x555bdce91f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2816c01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bdcdf7da0 con 0x555bdcebbb10 2024-04-25T08:40:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2816c01740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bdcda0b80 con 0x555bdce8e770 2024-04-25T08:40:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2816c01740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bdcdf3e40 con 0x555bdcebbef0 2024-04-25T08:40:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2811c98640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 0x555bdcebe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2811c98640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 0x555bdcebe320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36012/0 (socket says 172.21.15.38:36012) 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2811c98640 1 -- 172.21.15.38:0/1720410753 learned_addr learned my addr 172.21.15.38:0/1720410753 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2812499640 1 --2- 172.21.15.38:0/1720410753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdce8e770 0x555bdce91f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2811497640 1 --2- 172.21.15.38:0/1720410753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcebe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2811c98640 1 -- 172.21.15.38:0/1720410753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 msgr2=0x555bdcebe860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2811c98640 1 --2- 172.21.15.38:0/1720410753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcebe860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2811c98640 1 -- 172.21.15.38:0/1720410753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdce8e770 msgr2=0x555bdce91f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2811c98640 1 --2- 172.21.15.38:0/1720410753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdce8e770 0x555bdce91f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2811c98640 1 -- 172.21.15.38:0/1720410753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bdcdaaf10 con 0x555bdcebbb10 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.647+0000 7f2811497640 1 --2- 172.21.15.38:0/1720410753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcebe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.648+0000 7f2811c98640 1 --2- 172.21.15.38:0/1720410753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 0x555bdcebe320 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f2808035040 tx=0x7f280802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369522e917fa0da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.648+0000 7f27ef7fe640 1 -- 172.21.15.38:0/1720410753 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280804d070 con 0x555bdcebbb10 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.648+0000 7f27ef7fe640 1 -- 172.21.15.38:0/1720410753 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2808032d40 con 0x555bdcebbb10 2024-04-25T08:40:28.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.648+0000 7f27ef7fe640 1 -- 172.21.15.38:0/1720410753 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2808037040 con 0x555bdcebbb10 2024-04-25T08:40:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.648+0000 7f2816c01740 1 -- 172.21.15.38:0/1720410753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 msgr2=0x555bdcebe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.648+0000 7f2816c01740 1 --2- 172.21.15.38:0/1720410753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 0x555bdcebe320 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f2808035040 tx=0x7f280802c000 comp rx=0 tx=0).stop 2024-04-25T08:40:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.648+0000 7f2816c01740 1 -- 172.21.15.38:0/1720410753 shutdown_connections 2024-04-25T08:40:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.648+0000 7f2816c01740 1 --2- 172.21.15.38:0/1720410753 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdce8e770 0x555bdce91f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.648+0000 7f2816c01740 1 --2- 172.21.15.38:0/1720410753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcebe860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.648+0000 7f2816c01740 1 --2- 172.21.15.38:0/1720410753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 0x555bdcebe320 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.648+0000 7f2816c01740 1 -- 172.21.15.38:0/1720410753 >> 172.21.15.38:0/1720410753 conn(0x555bdce909e0 msgr2=0x555bdced9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.649+0000 7f2816c01740 1 -- 172.21.15.38:0/1720410753 shutdown_connections 2024-04-25T08:40:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.649+0000 7f2816c01740 1 -- 172.21.15.38:0/1720410753 wait complete. 2024-04-25T08:40:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.649+0000 7f2816c01740 1 -- start start 2024-04-25T08:40:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.649+0000 7f2816c01740 1 Processor -- start 2024-04-25T08:40:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.650+0000 7f2816c01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 0x555bdceddd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.650+0000 7f2816c01740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbef0 0x555bdcedfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.650+0000 7f2816c01740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcee0230 0x555bdceb2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.650+0000 7f2816c01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bdce93b90 con 0x555bdcebbb10 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.650+0000 7f2816c01740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bdcd964b0 con 0x555bdcebbef0 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.650+0000 7f2816c01740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bdce93e70 con 0x555bdcee0230 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.650+0000 7f2816c01740 10 client.0 _ll_register_callbacks cb 0x555bdcdd6180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.650+0000 7f2811c98640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 0x555bdceddd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.650+0000 7f2811c98640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 0x555bdceddd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36016/0 (socket says 172.21.15.38:36016) 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.650+0000 7f2811c98640 1 -- 172.21.15.38:0/3854159329 learned_addr learned my addr 172.21.15.38:0/3854159329 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.650+0000 7f2811497640 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbef0 0x555bdcedfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.650+0000 7f2812499640 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcee0230 0x555bdceb2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2811c98640 1 -- 172.21.15.38:0/3854159329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcee0230 msgr2=0x555bdceb2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2811c98640 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcee0230 0x555bdceb2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2811c98640 1 -- 172.21.15.38:0/3854159329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbef0 msgr2=0x555bdcedfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2811c98640 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbef0 0x555bdcedfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2811c98640 1 -- 172.21.15.38:0/3854159329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bdcbd67d0 con 0x555bdcebbb10 2024-04-25T08:40:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2811c98640 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 0x555bdceddd70 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f280802a2b0 tx=0x7f2808004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ae16bb81f4e0b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f27ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f27ecff9640 1 -- 172.21.15.38:0/3854159329 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280804d050 con 0x555bdcebbb10 2024-04-25T08:40:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f27ecff9640 1 -- 172.21.15.38:0/3854159329 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f280803c9d0 con 0x555bdcebbb10 2024-04-25T08:40:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2816c01740 1 -- 172.21.15.38:0/3854159329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2808032d40 con 0x555bdcebbb10 2024-04-25T08:40:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f27ecff9640 1 -- 172.21.15.38:0/3854159329 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f280805b050 con 0x555bdcebbb10 2024-04-25T08:40:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2816c01740 10 client.9237 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2816c01740 1 -- 172.21.15.38:0/3854159329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x555bdcb0c3f0 con 0x555bdcebbb10 2024-04-25T08:40:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f27ecff9640 1 -- 172.21.15.38:0/3854159329 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2808032d40 con 0x555bdcebbb10 2024-04-25T08:40:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f27ecff9640 1 -- 172.21.15.38:0/3854159329 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f280805e020 con 0x555bdcebbb10 2024-04-25T08:40:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2816c01740 10 client.9237 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2816c01740 10 client.9237 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2816c01740 20 client.9237 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2816c01740 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x555bdcedb970 0x555bdcedbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2816c01740 4 client.9237 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.651+0000 7f2816c01740 1 -- 172.21.15.38:0/3854159329 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bdce052e0 con 0x555bdcedb970 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.652+0000 7f2811497640 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x555bdcedb970 0x555bdcedbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.652+0000 7f27ecff9640 10 client.9237 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.652+0000 7f2811497640 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x555bdcedb970 0x555bdcedbd80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f3e1555ccdf9bbf server_cookie=699ffefd67bd225e in_seq=0 out_seq=0 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f27ecff9640 1 -- 172.21.15.38:0/3854159329 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555bdce052e0 con 0x555bdcedb970 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f27ecff9640 10 client.9237 handle_command_reply: tid=0 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f2816c01740 1 client.9237 shutdown 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f2816c01740 1 -- 172.21.15.38:0/3854159329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 msgr2=0x555bdceddd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f2816c01740 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 0x555bdceddd70 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f280802a2b0 tx=0x7f2808004730 comp rx=0 tx=0).stop 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f2816c01740 1 -- 172.21.15.38:0/3854159329 shutdown_connections 2024-04-25T08:40:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f2816c01740 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x555bdcedb970 0x555bdcedbd80 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f2816c01740 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcee0230 0x555bdceb2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f27ecff9640 0 client.9237 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f2816c01740 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbef0 0x555bdcedfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f2816c01740 1 --2- 172.21.15.38:0/3854159329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcebbb10 0x555bdceddd70 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f2816c01740 1 -- 172.21.15.38:0/3854159329 >> 172.21.15.38:0/3854159329 conn(0x555bdce8e770 msgr2=0x555bdcebe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f2816c01740 1 -- 172.21.15.38:0/3854159329 shutdown_connections 2024-04-25T08:40:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f2816c01740 1 -- 172.21.15.38:0/3854159329 wait complete. 2024-04-25T08:40:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.653+0000 7f2816c01740 20 client.9237 trim_cache size 0 max 16384 2024-04-25T08:40:28.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.668+0000 7f2816c01740 1 Processor -- start 2024-04-25T08:40:28.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.668+0000 7f2816c01740 1 -- start start 2024-04-25T08:40:28.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2816c01740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbb10 0x555bdcedbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2816c01740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcededd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2816c01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 0x555bdce8f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2816c01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c200450 con 0x555bdcee0230 2024-04-25T08:40:28.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2816c01740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c1ff650 con 0x555bdcebbb10 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2816c01740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0b3960 con 0x555bdcebbef0 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2812499640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 0x555bdce8f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2812499640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 0x555bdce8f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36028/0 (socket says 172.21.15.38:36028) 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2812499640 1 -- 172.21.15.38:0/3563465891 learned_addr learned my addr 172.21.15.38:0/3563465891 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2811497640 1 --2- 172.21.15.38:0/3563465891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcededd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2811c98640 1 --2- 172.21.15.38:0/3563465891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbb10 0x555bdcedbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2812499640 1 -- 172.21.15.38:0/3563465891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 msgr2=0x555bdcededd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2812499640 1 --2- 172.21.15.38:0/3563465891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcededd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2812499640 1 -- 172.21.15.38:0/3563465891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbb10 msgr2=0x555bdcedbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2812499640 1 --2- 172.21.15.38:0/3563465891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbb10 0x555bdcedbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2812499640 1 -- 172.21.15.38:0/3563465891 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bdcacc0f0 con 0x555bdcee0230 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2811c98640 1 --2- 172.21.15.38:0/3563465891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbb10 0x555bdcedbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f2812499640 1 --2- 172.21.15.38:0/3563465891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 0x555bdce8f990 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f2804015460 tx=0x7f2804009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecc529b955833f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f27eeffd640 1 -- 172.21.15.38:0/3563465891 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28040178e0 con 0x555bdcee0230 2024-04-25T08:40:28.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.669+0000 7f27eeffd640 1 -- 172.21.15.38:0/3563465891 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2804017a80 con 0x555bdcee0230 2024-04-25T08:40:28.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.670+0000 7f27eeffd640 1 -- 172.21.15.38:0/3563465891 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804017e50 con 0x555bdcee0230 2024-04-25T08:40:28.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.670+0000 7f2816c01740 1 -- 172.21.15.38:0/3563465891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 msgr2=0x555bdce8f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.670+0000 7f2816c01740 1 --2- 172.21.15.38:0/3563465891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 0x555bdce8f990 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f2804015460 tx=0x7f2804009750 comp rx=0 tx=0).stop 2024-04-25T08:40:28.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.670+0000 7f2816c01740 1 -- 172.21.15.38:0/3563465891 shutdown_connections 2024-04-25T08:40:28.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.670+0000 7f2816c01740 1 --2- 172.21.15.38:0/3563465891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 0x555bdce8f990 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.670+0000 7f2816c01740 1 --2- 172.21.15.38:0/3563465891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcededd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.670+0000 7f2816c01740 1 --2- 172.21.15.38:0/3563465891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbb10 0x555bdcedbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.670+0000 7f2816c01740 1 -- 172.21.15.38:0/3563465891 >> 172.21.15.38:0/3563465891 conn(0x555bdce8e770 msgr2=0x555bdcedfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.670+0000 7f2816c01740 1 -- 172.21.15.38:0/3563465891 shutdown_connections 2024-04-25T08:40:28.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.670+0000 7f2816c01740 1 -- 172.21.15.38:0/3563465891 wait complete. 2024-04-25T08:40:28.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.671+0000 7f2816c01740 1 -- start start 2024-04-25T08:40:28.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.671+0000 7f2816c01740 1 Processor -- start 2024-04-25T08:40:28.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2816c01740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbb10 0x555bdcee87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2816c01740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcec05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2816c01740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 0x555bdceb21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2816c01740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bdced7240 con 0x555bdcee0230 2024-04-25T08:40:28.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2816c01740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0b3f10 con 0x555bdcebbb10 2024-04-25T08:40:28.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2816c01740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x555bdce93f40 con 0x555bdcebbef0 2024-04-25T08:40:28.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2816c01740 10 client.0 _ll_register_callbacks cb 0x7f2808035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2812499640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 0x555bdceb21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2812499640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 0x555bdceb21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36030/0 (socket says 172.21.15.38:36030) 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2812499640 1 -- 172.21.15.38:0/1310783230 learned_addr learned my addr 172.21.15.38:0/1310783230 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2811c98640 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbb10 0x555bdcee87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2811497640 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcec05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2812499640 1 -- 172.21.15.38:0/1310783230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 msgr2=0x555bdcec05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2812499640 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcec05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2812499640 1 -- 172.21.15.38:0/1310783230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbb10 msgr2=0x555bdcee87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2812499640 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbb10 0x555bdcee87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2812499640 1 -- 172.21.15.38:0/1310783230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bdce8fb50 con 0x555bdcee0230 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2812499640 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 0x555bdceb21d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f2804009520 tx=0x7f2804004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a646e8c459b9fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f27ecff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f27ecff9640 1 -- 172.21.15.38:0/1310783230 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2804026070 con 0x555bdcee0230 2024-04-25T08:40:28.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f27ecff9640 1 -- 172.21.15.38:0/1310783230 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2804021820 con 0x555bdcee0230 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2816c01740 1 -- 172.21.15.38:0/1310783230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2804017a80 con 0x555bdcee0230 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f27ecff9640 1 -- 172.21.15.38:0/1310783230 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2804021a20 con 0x555bdcee0230 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2816c01740 10 client.9243 fetch_fsmap learned FSMap version 70 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.672+0000 7f2816c01740 1 -- 172.21.15.38:0/1310783230 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x555bdcedbef0 con 0x555bdcee0230 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f27ecff9640 1 -- 172.21.15.38:0/1310783230 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2804017a80 con 0x555bdcee0230 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f27ecff9640 1 -- 172.21.15.38:0/1310783230 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 70) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2804021bc0 con 0x555bdcee0230 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f2816c01740 10 client.9243 fetch_fsmap finished waiting for FSMap version 70 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f2816c01740 10 client.9243 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f2816c01740 20 client.9243 populate_metadata read hostname 'smithi038' 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f2816c01740 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x555bdcedc310 0x555bdceb2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f2816c01740 4 client.9243 mds_command: new command op to 7639 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f2816c01740 1 -- 172.21.15.38:0/1310783230 --> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555bdcdff630 con 0x555bdcedc310 2024-04-25T08:40:28.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f2811c98640 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x555bdcedc310 0x555bdceb2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f27ecff9640 10 client.9243 ms_handle_connect on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f2811c98640 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x555bdcedc310 0x555bdceb2710 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d654fe3772c3ab8 server_cookie=e0c2b79a1c409933 in_seq=0 out_seq=0 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f27ecff9640 1 -- 172.21.15.38:0/1310783230 <== mds.0 v2:172.21.15.107:6835/901382672 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555bdcdff630 con 0x555bdcedc310 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f27ecff9640 10 client.9243 handle_command_reply: tid=0 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.673+0000 7f2816c01740 1 client.9243 shutdown 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2816c01740 1 -- 172.21.15.38:0/1310783230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 msgr2=0x555bdceb21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2816c01740 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 0x555bdceb21d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f2804009520 tx=0x7f2804004ab0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2816c01740 1 -- 172.21.15.38:0/1310783230 shutdown_connections 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2816c01740 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.107:6835/901382672,v1:172.21.15.107:6837/901382672] conn(0x555bdcedc310 0x555bdceb2710 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2816c01740 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x555bdcee0230 0x555bdceb21d0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2816c01740 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x555bdcebbef0 0x555bdcec05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f27ecff9640 0 client.9243 ms_handle_reset on v2:172.21.15.107:6835/901382672 2024-04-25T08:40:28.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2816c01740 1 --2- 172.21.15.38:0/1310783230 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555bdcebbb10 0x555bdcee87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2816c01740 1 -- 172.21.15.38:0/1310783230 >> 172.21.15.38:0/1310783230 conn(0x555bdce8e770 msgr2=0x555bdce89b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2816c01740 1 -- 172.21.15.38:0/1310783230 shutdown_connections 2024-04-25T08:40:28.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2816c01740 1 -- 172.21.15.38:0/1310783230 wait complete. 2024-04-25T08:40:28.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2816c01740 20 client.9243 trim_cache size 0 max 16384 2024-04-25T08:40:28.674 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:40:28.674 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-04-25T08:40:28.674 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-04-25T08:40:28.674 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:40:28.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2813efc640 1 -- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f27f006b4d0 msgr2=0x7f27f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2813efc640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f27f006b4d0 0x7f27f006d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2808034000 tx=0x7f280802e000 comp rx=0 tx=0).stop 2024-04-25T08:40:28.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2813efc640 1 -- 172.21.15.38:0/3190442192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c0d0700 msgr2=0x7f280c15ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2813efc640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c0d0700 0x7f280c15ae20 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f27fc00e8c0 tx=0x7f27fc00ed80 comp rx=0 tx=0).stop 2024-04-25T08:40:28.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2813efc640 1 -- 172.21.15.38:0/3190442192 shutdown_connections 2024-04-25T08:40:28.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2813efc640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f280c0d1e90 0x7f280c15c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2813efc640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f280c0d0700 0x7f280c15ae20 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2813efc640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f27f006b4d0 0x7f27f006d990 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2813efc640 1 --2- 172.21.15.38:0/3190442192 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f280c0d0320 0x7f280c1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2813efc640 1 -- 172.21.15.38:0/3190442192 >> 172.21.15.38:0/3190442192 conn(0x7f280c0668f0 msgr2=0x7f280c0b6860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2813efc640 1 -- 172.21.15.38:0/3190442192 shutdown_connections 2024-04-25T08:40:28.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.674+0000 7f2813efc640 1 -- 172.21.15.38:0/3190442192 wait complete. 2024-04-25T08:40:28.679 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:40:28.679 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:28.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.786+0000 7fd98fbcf640 1 Processor -- start 2024-04-25T08:40:28.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.786+0000 7fd98fbcf640 1 -- start start 2024-04-25T08:40:28.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.787+0000 7fd98fbcf640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd98815dc10 0x7fd9880ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.787+0000 7fd98fbcf640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9880d0f60 0x7fd9880b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.787+0000 7fd98fbcf640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9880d3e70 con 0x7fd9880ced60 2024-04-25T08:40:28.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.787+0000 7fd98fbcf640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd98815dff0 con 0x7fd98815dc10 2024-04-25T08:40:28.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.787+0000 7fd98fbcf640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880b6990 con 0x7fd9880d0f60 2024-04-25T08:40:28.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.787+0000 7fd98d16a640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd9880ced60 0x7fd9880d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:60500/0 (socket says 172.21.15.38:60500) 2024-04-25T08:40:28.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.787+0000 7fd98d16a640 1 -- 172.21.15.38:0/66757578 learned_addr learned my addr 172.21.15.38:0/66757578 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:28.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.787+0000 7fd98d96b640 1 --2- 172.21.15.38:0/66757578 >> v2:172.21.15.107:3300/0 conn(0x7fd98815dc10 0x7fd9880ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.787+0000 7fd98e16c640 1 --2- 172.21.15.38:0/66757578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9880d0f60 0x7fd9880b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.788+0000 7fd98c969640 1 -- 172.21.15.38:0/66757578 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2463017613 0 0) 0x7fd9880d3e70 con 0x7fd9880ced60 2024-04-25T08:40:28.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.788+0000 7fd98c969640 1 -- 172.21.15.38:0/66757578 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd970003520 con 0x7fd9880ced60 2024-04-25T08:40:28.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.788+0000 7fd98d96b640 1 -- 172.21.15.38:0/66757578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9880d0f60 msgr2=0x7fd9880b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:28.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.788+0000 7fd98d96b640 1 --2- 172.21.15.38:0/66757578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9880d0f60 0x7fd9880b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.788+0000 7fd98d96b640 1 -- 172.21.15.38:0/66757578 >> v1:172.21.15.38:6789/0 conn(0x7fd9880ced60 legacy=0x7fd9880d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.788+0000 7fd98d96b640 1 -- 172.21.15.38:0/66757578 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9880d4150 con 0x7fd98815dc10 2024-04-25T08:40:28.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.788+0000 7fd98d96b640 1 --2- 172.21.15.38:0/66757578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd98815dc10 0x7fd9880ce820 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd978009d90 tx=0x7fd97802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bed9af8d8fc53a01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.789+0000 7fd98c969640 1 -- 172.21.15.38:0/66757578 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978031600 con 0x7fd98815dc10 2024-04-25T08:40:28.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.789+0000 7fd98c969640 1 -- 172.21.15.38:0/66757578 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9780317a0 con 0x7fd98815dc10 2024-04-25T08:40:28.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.789+0000 7fd98c969640 1 -- 172.21.15.38:0/66757578 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978031b70 con 0x7fd98815dc10 2024-04-25T08:40:28.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.789+0000 7fd98fbcf640 1 -- 172.21.15.38:0/66757578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd98815dc10 msgr2=0x7fd9880ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.789+0000 7fd98fbcf640 1 --2- 172.21.15.38:0/66757578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd98815dc10 0x7fd9880ce820 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd978009d90 tx=0x7fd97802a5d0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.789+0000 7fd98fbcf640 1 -- 172.21.15.38:0/66757578 shutdown_connections 2024-04-25T08:40:28.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.789+0000 7fd98fbcf640 1 --2- 172.21.15.38:0/66757578 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9880d0f60 0x7fd9880b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.789+0000 7fd98fbcf640 1 --2- 172.21.15.38:0/66757578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd98815dc10 0x7fd9880ce820 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.789+0000 7fd98fbcf640 1 -- 172.21.15.38:0/66757578 >> 172.21.15.38:0/66757578 conn(0x7fd988066d10 msgr2=0x7fd988067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:28.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.789+0000 7fd98fbcf640 1 -- 172.21.15.38:0/66757578 shutdown_connections 2024-04-25T08:40:28.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.789+0000 7fd98fbcf640 1 -- 172.21.15.38:0/66757578 wait complete. 2024-04-25T08:40:28.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.791+0000 7fd98fbcf640 1 Processor -- start 2024-04-25T08:40:28.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.791+0000 7fd98fbcf640 1 -- start start 2024-04-25T08:40:28.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.791+0000 7fd98fbcf640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9880ced60 0x7fd98815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.791+0000 7fd98fbcf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9880d0f60 0x7fd9880c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.791+0000 7fd98fbcf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd98815dc10 0x7fd9880c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.791+0000 7fd98fbcf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880b6bc0 con 0x7fd9880d0f60 2024-04-25T08:40:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.791+0000 7fd98fbcf640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880d4710 con 0x7fd98815dc10 2024-04-25T08:40:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.791+0000 7fd98fbcf640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880d3900 con 0x7fd9880ced60 2024-04-25T08:40:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.791+0000 7fd98d16a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9880d0f60 0x7fd9880c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.791+0000 7fd98d16a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9880d0f60 0x7fd9880c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36040/0 (socket says 172.21.15.38:36040) 2024-04-25T08:40:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.791+0000 7fd98d16a640 1 -- 172.21.15.38:0/3650761147 learned_addr learned my addr 172.21.15.38:0/3650761147 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd98e16c640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd98815dc10 0x7fd9880c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd98d96b640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9880ced60 0x7fd98815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd98d16a640 1 -- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9880ced60 msgr2=0x7fd98815b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd98d16a640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9880ced60 0x7fd98815b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd98d16a640 1 -- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd98815dc10 msgr2=0x7fd9880c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd98d16a640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd98815dc10 0x7fd9880c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd98d16a640 1 -- 172.21.15.38:0/3650761147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9880c8120 con 0x7fd9880d0f60 2024-04-25T08:40:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd98e16c640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd98815dc10 0x7fd9880c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd98d16a640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9880d0f60 0x7fd9880c93d0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00c460 tx=0x7fd97c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd9867fc640 1 -- 172.21.15.38:0/3650761147 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c017070 con 0x7fd9880d0f60 2024-04-25T08:40:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd9867fc640 1 -- 172.21.15.38:0/3650761147 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97c005ce0 con 0x7fd9880d0f60 2024-04-25T08:40:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd98fbcf640 1 -- 172.21.15.38:0/3650761147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9880c8400 con 0x7fd9880d0f60 2024-04-25T08:40:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd9867fc640 1 -- 172.21.15.38:0/3650761147 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97c007290 con 0x7fd9880d0f60 2024-04-25T08:40:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.792+0000 7fd98fbcf640 1 -- 172.21.15.38:0/3650761147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9880c89c0 con 0x7fd9880d0f60 2024-04-25T08:40:28.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.793+0000 7fd98fbcf640 1 -- 172.21.15.38:0/3650761147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd988068e80 con 0x7fd9880d0f60 2024-04-25T08:40:28.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.794+0000 7fd9867fc640 1 -- 172.21.15.38:0/3650761147 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd97c01a070 con 0x7fd9880d0f60 2024-04-25T08:40:28.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.794+0000 7fd9867fc640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd97406b4d0 0x7fd97406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:28.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.794+0000 7fd9867fc640 1 -- 172.21.15.38:0/3650761147 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd97c089680 con 0x7fd9880d0f60 2024-04-25T08:40:28.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.794+0000 7fd98d96b640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd97406b4d0 0x7fd97406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:28.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.795+0000 7fd98d96b640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd97406b4d0 0x7fd97406d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd978034000 tx=0x7fd978033040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:28.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:28.798+0000 7fd9867fc640 1 -- 172.21.15.38:0/3650761147 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd97c058400 con 0x7fd9880d0f60 2024-04-25T08:40:29.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.013+0000 7fd98fbcf640 1 -- 172.21.15.38:0/3650761147 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd98815e170 con 0x7fd9880d0f60 2024-04-25T08:40:29.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.013+0000 7fd9867fc640 1 -- 172.21.15.38:0/3650761147 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd97c057d00 con 0x7fd9880d0f60 2024-04-25T08:40:29.008 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:29.008 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:29.008 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:29.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.015+0000 7fd98fbcf640 1 -- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd97406b4d0 msgr2=0x7fd97406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:29.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.015+0000 7fd98fbcf640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd97406b4d0 0x7fd97406d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd978034000 tx=0x7fd978033040 comp rx=0 tx=0).stop 2024-04-25T08:40:29.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.015+0000 7fd98fbcf640 1 -- 172.21.15.38:0/3650761147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9880d0f60 msgr2=0x7fd9880c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:29.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.015+0000 7fd98fbcf640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9880d0f60 0x7fd9880c93d0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fd97c00c460 tx=0x7fd97c00c920 comp rx=0 tx=0).stop 2024-04-25T08:40:29.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.015+0000 7fd98fbcf640 1 -- 172.21.15.38:0/3650761147 shutdown_connections 2024-04-25T08:40:29.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.015+0000 7fd98fbcf640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd98815dc10 0x7fd9880c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.015+0000 7fd98fbcf640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd9880d0f60 0x7fd9880c93d0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.015+0000 7fd98fbcf640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd97406b4d0 0x7fd97406d990 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.015+0000 7fd98fbcf640 1 --2- 172.21.15.38:0/3650761147 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd9880ced60 0x7fd98815b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.015+0000 7fd98fbcf640 1 -- 172.21.15.38:0/3650761147 >> 172.21.15.38:0/3650761147 conn(0x7fd988066d10 msgr2=0x7fd9880d1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:29.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.015+0000 7fd98fbcf640 1 -- 172.21.15.38:0/3650761147 shutdown_connections 2024-04-25T08:40:29.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.015+0000 7fd98fbcf640 1 -- 172.21.15.38:0/3650761147 wait complete. 2024-04-25T08:40:29.020 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:40:29.020 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:40:29.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.127+0000 7f31adabd640 1 Processor -- start 2024-04-25T08:40:29.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.127+0000 7f31adabd640 1 -- start start 2024-04-25T08:40:29.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.128+0000 7f31adabd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f31a815bb10 0x7f31a809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:29.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.128+0000 7f31adabd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31a80d0460 0x7f31a809a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:29.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.128+0000 7f31adabd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f31a815f4e0 0x7f31a815f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:29.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.128+0000 7f31adabd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a80cffd0 con 0x7f31a815bb10 2024-04-25T08:40:29.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.128+0000 7f31adabd640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a8160340 con 0x7f31a815f4e0 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.128+0000 7f31adabd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80cf400 con 0x7f31a80d0460 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.128+0000 7f31a77fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f31a815bb10 0x7f31a809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.128+0000 7f31a77fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f31a815bb10 0x7f31a809a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51896/0 (socket says 172.21.15.38:51896) 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.128+0000 7f31a77fe640 1 -- 172.21.15.38:0/959560130 learned_addr learned my addr 172.21.15.38:0/959560130 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.129+0000 7f31a7fff640 1 --2- 172.21.15.38:0/959560130 >> v2:172.21.15.107:3300/0 conn(0x7f31a815f4e0 0x7f31a815f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.129+0000 7f31a6ffd640 1 --2- 172.21.15.38:0/959560130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31a80d0460 0x7f31a809a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.129+0000 7f31a77fe640 1 -- 172.21.15.38:0/959560130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31a80d0460 msgr2=0x7f31a809a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.129+0000 7f31a77fe640 1 --2- 172.21.15.38:0/959560130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31a80d0460 0x7f31a809a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.129+0000 7f31a77fe640 1 -- 172.21.15.38:0/959560130 >> v2:172.21.15.107:3300/0 conn(0x7f31a815f4e0 msgr2=0x7f31a815f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.129+0000 7f31a77fe640 1 --2- 172.21.15.38:0/959560130 >> v2:172.21.15.107:3300/0 conn(0x7f31a815f4e0 0x7f31a815f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.129+0000 7f31a77fe640 1 -- 172.21.15.38:0/959560130 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a80cf6a0 con 0x7f31a815bb10 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.129+0000 7f31a6ffd640 1 --2- 172.21.15.38:0/959560130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31a80d0460 0x7f31a809a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:29.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31a77fe640 1 --2- 172.21.15.38:0/959560130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31a815bb10 0x7f31a809a480 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f31a80d16b0 tx=0x7f319802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0626df3dae1a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:29.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31a67fc640 1 -- 172.21.15.38:0/959560130 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198037070 con 0x7f31a815bb10 2024-04-25T08:40:29.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31a67fc640 1 -- 172.21.15.38:0/959560130 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3198032440 con 0x7f31a815bb10 2024-04-25T08:40:29.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31a67fc640 1 -- 172.21.15.38:0/959560130 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198032810 con 0x7f31a815bb10 2024-04-25T08:40:29.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31adabd640 1 -- 172.21.15.38:0/959560130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31a815bb10 msgr2=0x7f31a809a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:29.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31adabd640 1 --2- 172.21.15.38:0/959560130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31a815bb10 0x7f31a809a480 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f31a80d16b0 tx=0x7f319802a140 comp rx=0 tx=0).stop 2024-04-25T08:40:29.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31adabd640 1 -- 172.21.15.38:0/959560130 shutdown_connections 2024-04-25T08:40:29.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31adabd640 1 --2- 172.21.15.38:0/959560130 >> v2:172.21.15.107:3300/0 conn(0x7f31a815f4e0 0x7f31a815f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31adabd640 1 --2- 172.21.15.38:0/959560130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31a80d0460 0x7f31a809a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31adabd640 1 --2- 172.21.15.38:0/959560130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31a815bb10 0x7f31a809a480 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31adabd640 1 -- 172.21.15.38:0/959560130 >> 172.21.15.38:0/959560130 conn(0x7f31a80b31f0 msgr2=0x7f31a80b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:29.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31adabd640 1 -- 172.21.15.38:0/959560130 shutdown_connections 2024-04-25T08:40:29.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.130+0000 7f31adabd640 1 -- 172.21.15.38:0/959560130 wait complete. 2024-04-25T08:40:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.131+0000 7f31adabd640 1 Processor -- start 2024-04-25T08:40:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.132+0000 7f31adabd640 1 -- start start 2024-04-25T08:40:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.132+0000 7f31adabd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31a80d0460 0x7f31a814bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.132+0000 7f31adabd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a815bb10 0x7f31a806d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.132+0000 7f31adabd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31a815f4e0 0x7f31a80b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.132+0000 7f31adabd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80c5250 con 0x7f31a80d0460 2024-04-25T08:40:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.132+0000 7f31adabd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80cfc60 con 0x7f31a815bb10 2024-04-25T08:40:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.132+0000 7f31adabd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a80c53d0 con 0x7f31a815f4e0 2024-04-25T08:40:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.132+0000 7f31a77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31a80d0460 0x7f31a814bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31a77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31a80d0460 0x7f31a814bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51910/0 (socket says 172.21.15.38:51910) 2024-04-25T08:40:29.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31a77fe640 1 -- 172.21.15.38:0/1466618225 learned_addr learned my addr 172.21.15.38:0/1466618225 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31a6ffd640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a815bb10 0x7f31a806d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31a7fff640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31a815f4e0 0x7f31a80b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31a77fe640 1 -- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31a815f4e0 msgr2=0x7f31a80b4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31a77fe640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31a815f4e0 0x7f31a80b4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31a77fe640 1 -- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a815bb10 msgr2=0x7f31a806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31a77fe640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a815bb10 0x7f31a806d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31a77fe640 1 -- 172.21.15.38:0/1466618225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a815c020 con 0x7f31a80d0460 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31a7fff640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31a815f4e0 0x7f31a80b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31a77fe640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31a80d0460 0x7f31a814bcd0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f3198000c00 tx=0x7f31980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f318ffff640 1 -- 172.21.15.38:0/1466618225 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198041020 con 0x7f31a80d0460 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f318ffff640 1 -- 172.21.15.38:0/1466618225 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31980306c0 con 0x7f31a80d0460 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31adabd640 1 -- 172.21.15.38:0/1466618225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a815c300 con 0x7f31a80d0460 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f318ffff640 1 -- 172.21.15.38:0/1466618225 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3198037040 con 0x7f31a80d0460 2024-04-25T08:40:29.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.133+0000 7f31adabd640 1 -- 172.21.15.38:0/1466618225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a806cc30 con 0x7f31a80d0460 2024-04-25T08:40:29.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.134+0000 7f31adabd640 1 -- 172.21.15.38:0/1466618225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a806d690 con 0x7f31a80d0460 2024-04-25T08:40:29.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.135+0000 7f318ffff640 1 -- 172.21.15.38:0/1466618225 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3198044020 con 0x7f31a80d0460 2024-04-25T08:40:29.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.135+0000 7f318ffff640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f319406b4d0 0x7f319406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:29.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.135+0000 7f318ffff640 1 -- 172.21.15.38:0/1466618225 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f31980aa9f0 con 0x7f31a80d0460 2024-04-25T08:40:29.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.136+0000 7f31a6ffd640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f319406b4d0 0x7f319406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:29.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.136+0000 7f31a6ffd640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f319406b4d0 0x7f319406d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f31a809b070 tx=0x7f319c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:29.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.140+0000 7f318ffff640 1 -- 172.21.15.38:0/1466618225 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f319804b030 con 0x7f31a80d0460 2024-04-25T08:40:29.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.390+0000 7f31adabd640 1 -- 172.21.15.38:0/1466618225 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f31a8068e80 con 0x7f31a80d0460 2024-04-25T08:40:29.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.391+0000 7f318ffff640 1 -- 172.21.15.38:0/1466618225 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f31980aa020 con 0x7f31a80d0460 2024-04-25T08:40:29.385 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:29.385 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":558,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076550540,"bytes_used":2673422336,"bytes_avail":770420690944,"bytes_total":773094113280,"write_bytes_sec":63620137,"read_op_per_sec":0,"write_op_per_sec":64},"fsmap":{"epoch":70,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":7639}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:40:25.328586+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T08:40:29.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.392+0000 7f31adabd640 1 -- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f319406b4d0 msgr2=0x7f319406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:29.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.392+0000 7f31adabd640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f319406b4d0 0x7f319406d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f31a809b070 tx=0x7f319c008040 comp rx=0 tx=0).stop 2024-04-25T08:40:29.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.393+0000 7f31adabd640 1 -- 172.21.15.38:0/1466618225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31a80d0460 msgr2=0x7f31a814bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:29.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.393+0000 7f31adabd640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31a80d0460 0x7f31a814bcd0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f3198000c00 tx=0x7f31980046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.393+0000 7f31adabd640 1 -- 172.21.15.38:0/1466618225 shutdown_connections 2024-04-25T08:40:29.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.393+0000 7f31adabd640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f319406b4d0 0x7f319406d990 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.393+0000 7f31adabd640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f31a815f4e0 0x7f31a80b4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.393+0000 7f31adabd640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31a815bb10 0x7f31a806d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.393+0000 7f31adabd640 1 --2- 172.21.15.38:0/1466618225 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f31a80d0460 0x7f31a814bcd0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:29.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.393+0000 7f31adabd640 1 -- 172.21.15.38:0/1466618225 >> 172.21.15.38:0/1466618225 conn(0x7f31a80b31f0 msgr2=0x7f31a80c5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:29.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.393+0000 7f31adabd640 1 -- 172.21.15.38:0/1466618225 shutdown_connections 2024-04-25T08:40:29.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:29.393+0000 7f31adabd640 1 -- 172.21.15.38:0/1466618225 wait complete. 2024-04-25T08:40:29.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T08:40:30.398 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.518+0000 7fdab0274640 1 Processor -- start 2024-04-25T08:40:30.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.518+0000 7fdab0274640 1 -- start start 2024-04-25T08:40:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.519+0000 7fdab0274640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdaa815da90 0x7fdaa8161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.519+0000 7fdab0274640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdaa80d0850 0x7fdaa80d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.519+0000 7fdab0274640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa8162160 con 0x7fdaa8161c50 2024-04-25T08:40:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.519+0000 7fdab0274640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa80d19b0 con 0x7fdaa80d0850 2024-04-25T08:40:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.519+0000 7fdab0274640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa80d1f70 con 0x7fdaa815da90 2024-04-25T08:40:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.520+0000 7fdaae811640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdaa80d0850 0x7fdaa80d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.520+0000 7fdaae811640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdaa80d0850 0x7fdaa80d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51920/0 (socket says 172.21.15.38:51920) 2024-04-25T08:40:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.520+0000 7fdaae811640 1 -- 172.21.15.38:0/44359753 learned_addr learned my addr 172.21.15.38:0/44359753 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:30.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.520+0000 7fdaae010640 1 --2- 172.21.15.38:0/44359753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdaa815da90 0x7fdaa8161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.520+0000 7fdaae811640 1 -- 172.21.15.38:0/44359753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdaa815da90 msgr2=0x7fdaa8161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.520+0000 7fdaae811640 1 --2- 172.21.15.38:0/44359753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdaa815da90 0x7fdaa8161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.520+0000 7fdaae811640 1 -- 172.21.15.38:0/44359753 >> v1:172.21.15.107:6789/0 conn(0x7fdaa8161c50 legacy=0x7fdaa80d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.520+0000 7fdaae811640 1 -- 172.21.15.38:0/44359753 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa8162440 con 0x7fdaa80d0850 2024-04-25T08:40:30.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.520+0000 7fdaae010640 1 --2- 172.21.15.38:0/44359753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdaa815da90 0x7fdaa8161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:30.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.521+0000 7fdaae811640 1 --2- 172.21.15.38:0/44359753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa80d0850 0x7fdaa80d2400 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009870 tx=0x7fdaa402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76158982a7994897 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:30.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.521+0000 7fdaad00e640 1 -- 172.21.15.38:0/44359753 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4037070 con 0x7fdaa80d0850 2024-04-25T08:40:30.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.521+0000 7fdaad00e640 1 -- 172.21.15.38:0/44359753 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa4032440 con 0x7fdaa80d0850 2024-04-25T08:40:30.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.521+0000 7fdaad00e640 1 -- 172.21.15.38:0/44359753 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4032810 con 0x7fdaa80d0850 2024-04-25T08:40:30.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.521+0000 7fdab0274640 1 -- 172.21.15.38:0/44359753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa80d0850 msgr2=0x7fdaa80d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:30.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.521+0000 7fdab0274640 1 --2- 172.21.15.38:0/44359753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa80d0850 0x7fdaa80d2400 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009870 tx=0x7fdaa402a180 comp rx=0 tx=0).stop 2024-04-25T08:40:30.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.521+0000 7fdab0274640 1 -- 172.21.15.38:0/44359753 shutdown_connections 2024-04-25T08:40:30.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.521+0000 7fdab0274640 1 --2- 172.21.15.38:0/44359753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa80d0850 0x7fdaa80d2400 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.521+0000 7fdab0274640 1 --2- 172.21.15.38:0/44359753 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdaa815da90 0x7fdaa8161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.521+0000 7fdab0274640 1 -- 172.21.15.38:0/44359753 >> 172.21.15.38:0/44359753 conn(0x7fdaa806c060 msgr2=0x7fdaa806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:30.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.522+0000 7fdab0274640 1 -- 172.21.15.38:0/44359753 shutdown_connections 2024-04-25T08:40:30.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.522+0000 7fdab0274640 1 -- 172.21.15.38:0/44359753 wait complete. 2024-04-25T08:40:30.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.523+0000 7fdab0274640 1 Processor -- start 2024-04-25T08:40:30.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.523+0000 7fdab0274640 1 -- start start 2024-04-25T08:40:30.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.523+0000 7fdab0274640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdaa80d0850 0x7fdaa815e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:30.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.523+0000 7fdab0274640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa815da90 0x7fdaa80c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:30.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.523+0000 7fdab0274640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8161c50 0x7fdaa81e7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:30.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.523+0000 7fdab0274640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa80b9400 con 0x7fdaa8161c50 2024-04-25T08:40:30.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.523+0000 7fdab0274640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa815de70 con 0x7fdaa815da90 2024-04-25T08:40:30.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.523+0000 7fdab0274640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa815e750 con 0x7fdaa80d0850 2024-04-25T08:40:30.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdaae811640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8161c50 0x7fdaa81e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdaae811640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8161c50 0x7fdaa81e7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51922/0 (socket says 172.21.15.38:51922) 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdaae811640 1 -- 172.21.15.38:0/3559688647 learned_addr learned my addr 172.21.15.38:0/3559688647 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdaae010640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdaa80d0850 0x7fdaa815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdaad80f640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa815da90 0x7fdaa80c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdaae811640 1 -- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdaa80d0850 msgr2=0x7fdaa815e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdaae811640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdaa80d0850 0x7fdaa815e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdaae811640 1 -- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa815da90 msgr2=0x7fdaa80c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdaae811640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa815da90 0x7fdaa80c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdaae811640 1 -- 172.21.15.38:0/3559688647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa815e9f0 con 0x7fdaa8161c50 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdaae811640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8161c50 0x7fdaa81e7430 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fdaa402a1c0 tx=0x7fdaa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fda9effd640 1 -- 172.21.15.38:0/3559688647 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4041020 con 0x7fdaa8161c50 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fda9effd640 1 -- 172.21.15.38:0/3559688647 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa4030990 con 0x7fdaa8161c50 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdab0274640 1 -- 172.21.15.38:0/3559688647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa815ecd0 con 0x7fdaa8161c50 2024-04-25T08:40:30.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fda9effd640 1 -- 172.21.15.38:0/3559688647 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4037040 con 0x7fdaa8161c50 2024-04-25T08:40:30.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.524+0000 7fdab0274640 1 -- 172.21.15.38:0/3559688647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa815f290 con 0x7fdaa8161c50 2024-04-25T08:40:30.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.526+0000 7fdab0274640 1 -- 172.21.15.38:0/3559688647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa80d1350 con 0x7fdaa8161c50 2024-04-25T08:40:30.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.526+0000 7fda9effd640 1 -- 172.21.15.38:0/3559688647 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdaa4044020 con 0x7fdaa8161c50 2024-04-25T08:40:30.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.526+0000 7fda9effd640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda8c06b4d0 0x7fda8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:30.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.526+0000 7fda9effd640 1 -- 172.21.15.38:0/3559688647 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdaa40aabf0 con 0x7fdaa8161c50 2024-04-25T08:40:30.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.527+0000 7fdaae010640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda8c06b4d0 0x7fda8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:30.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.527+0000 7fdaae010640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda8c06b4d0 0x7fda8c06d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000d620 tx=0x7fdaa0007810 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:30.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.530+0000 7fda9effd640 1 -- 172.21.15.38:0/3559688647 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdaa80d1350 con 0x7fdaa8161c50 2024-04-25T08:40:30.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.752+0000 7fdab0274640 1 -- 172.21.15.38:0/3559688647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdaa8068e80 con 0x7fdaa8161c50 2024-04-25T08:40:30.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.752+0000 7fda9effd640 1 -- 172.21.15.38:0/3559688647 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fdaa404b030 con 0x7fdaa8161c50 2024-04-25T08:40:30.746 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:30.747 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:30.747 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.754+0000 7fdab0274640 1 -- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda8c06b4d0 msgr2=0x7fda8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.754+0000 7fdab0274640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda8c06b4d0 0x7fda8c06d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000d620 tx=0x7fdaa0007810 comp rx=0 tx=0).stop 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.754+0000 7fdab0274640 1 -- 172.21.15.38:0/3559688647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8161c50 msgr2=0x7fdaa81e7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:30.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.754+0000 7fdab0274640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8161c50 0x7fdaa81e7430 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fdaa402a1c0 tx=0x7fdaa40046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.754+0000 7fdab0274640 1 -- 172.21.15.38:0/3559688647 shutdown_connections 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.754+0000 7fdab0274640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa8161c50 0x7fdaa81e7430 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.754+0000 7fdab0274640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdaa815da90 0x7fdaa80c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.754+0000 7fdab0274640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fda8c06b4d0 0x7fda8c06d990 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.754+0000 7fdab0274640 1 --2- 172.21.15.38:0/3559688647 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdaa80d0850 0x7fdaa815e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.754+0000 7fdab0274640 1 -- 172.21.15.38:0/3559688647 >> 172.21.15.38:0/3559688647 conn(0x7fdaa806c060 msgr2=0x7fdaa80cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.754+0000 7fdab0274640 1 -- 172.21.15.38:0/3559688647 shutdown_connections 2024-04-25T08:40:30.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.754+0000 7fdab0274640 1 -- 172.21.15.38:0/3559688647 wait complete. 2024-04-25T08:40:30.759 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:40:30.759 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:40:30.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.864+0000 7f960e1c4640 1 Processor -- start 2024-04-25T08:40:30.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.864+0000 7f960e1c4640 1 -- start start 2024-04-25T08:40:30.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.865+0000 7f960e1c4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f960806cff0 0x7f9608099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:30.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.865+0000 7f960e1c4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9608099fb0 0x7f960815d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.865+0000 7f960e1c4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f960809a3c0 con 0x7f960806cc10 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.865+0000 7f960e1c4640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96080d05c0 con 0x7f9608099fb0 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.865+0000 7f960e1c4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96080d08a0 con 0x7f960806cff0 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.865+0000 7f96077fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f960806cc10 0x7f96080cfa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47988/0 (socket says 172.21.15.38:47988) 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.865+0000 7f96077fe640 1 -- 172.21.15.38:0/185108118 learned_addr learned my addr 172.21.15.38:0/185108118 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.865+0000 7f9607fff640 1 --2- 172.21.15.38:0/185108118 >> v2:172.21.15.38:3300/0 conn(0x7f9608099fb0 0x7f960815d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.866+0000 7f9606ffd640 1 --2- 172.21.15.38:0/185108118 >> v2:172.21.15.107:3300/0 conn(0x7f960806cff0 0x7f9608099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.866+0000 7f96067fc640 1 -- 172.21.15.38:0/185108118 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849122039 0 0) 0x7f95f8001110 con 0x7f960806cc10 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.866+0000 7f96067fc640 1 -- 172.21.15.38:0/185108118 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95f0003520 con 0x7f960806cc10 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.866+0000 7f9607fff640 1 -- 172.21.15.38:0/185108118 >> v2:172.21.15.107:3300/0 conn(0x7f960806cff0 msgr2=0x7f9608099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.866+0000 7f9607fff640 1 --2- 172.21.15.38:0/185108118 >> v2:172.21.15.107:3300/0 conn(0x7f960806cff0 0x7f9608099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.866+0000 7f9607fff640 1 -- 172.21.15.38:0/185108118 >> v1:172.21.15.38:6789/0 conn(0x7f960806cc10 legacy=0x7f96080cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.866+0000 7f9607fff640 1 -- 172.21.15.38:0/185108118 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96080d0b80 con 0x7f9608099fb0 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.866+0000 7f9607fff640 1 --2- 172.21.15.38:0/185108118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9608099fb0 0x7f960815d370 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f960806db60 tx=0x7f960002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20fe48af28b334ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:30.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.867+0000 7f96067fc640 1 -- 172.21.15.38:0/185108118 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600007740 con 0x7f9608099fb0 2024-04-25T08:40:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.867+0000 7f96067fc640 1 -- 172.21.15.38:0/185108118 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96000078e0 con 0x7f9608099fb0 2024-04-25T08:40:30.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.867+0000 7f96067fc640 1 -- 172.21.15.38:0/185108118 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600007cb0 con 0x7f9608099fb0 2024-04-25T08:40:30.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.867+0000 7f960e1c4640 1 -- 172.21.15.38:0/185108118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9608099fb0 msgr2=0x7f960815d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:30.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.867+0000 7f960e1c4640 1 --2- 172.21.15.38:0/185108118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9608099fb0 0x7f960815d370 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f960806db60 tx=0x7f960002a680 comp rx=0 tx=0).stop 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.867+0000 7f960e1c4640 1 -- 172.21.15.38:0/185108118 shutdown_connections 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.867+0000 7f960e1c4640 1 --2- 172.21.15.38:0/185108118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9608099fb0 0x7f960815d370 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.867+0000 7f960e1c4640 1 --2- 172.21.15.38:0/185108118 >> v2:172.21.15.107:3300/0 conn(0x7f960806cff0 0x7f9608099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.867+0000 7f960e1c4640 1 -- 172.21.15.38:0/185108118 >> 172.21.15.38:0/185108118 conn(0x7f96080b31f0 msgr2=0x7f96080b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.867+0000 7f960e1c4640 1 -- 172.21.15.38:0/185108118 shutdown_connections 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.867+0000 7f960e1c4640 1 -- 172.21.15.38:0/185108118 wait complete. 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.868+0000 7f960e1c4640 1 Processor -- start 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.869+0000 7f960e1c4640 1 -- start start 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.869+0000 7f960e1c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960806cc10 0x7f96081e7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.869+0000 7f960e1c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960806cff0 0x7f96081e3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.869+0000 7f960e1c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9608099fb0 0x7f96081e4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.869+0000 7f960e1c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960806c150 con 0x7f9608099fb0 2024-04-25T08:40:30.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.869+0000 7f960e1c4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96080d1140 con 0x7f960806cff0 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.869+0000 7f960e1c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f960806c2d0 con 0x7f960806cc10 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.869+0000 7f9607fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9608099fb0 0x7f96081e4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.869+0000 7f9607fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9608099fb0 0x7f96081e4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51946/0 (socket says 172.21.15.38:51946) 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.869+0000 7f9607fff640 1 -- 172.21.15.38:0/1307118111 learned_addr learned my addr 172.21.15.38:0/1307118111 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f96077fe640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960806cc10 0x7f96081e7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f9606ffd640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960806cff0 0x7f96081e3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f9607fff640 1 -- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960806cc10 msgr2=0x7f96081e7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f9607fff640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960806cc10 0x7f96081e7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f9607fff640 1 -- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960806cff0 msgr2=0x7f96081e3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f9607fff640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960806cff0 0x7f96081e3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f9607fff640 1 -- 172.21.15.38:0/1307118111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96081e48c0 con 0x7f9608099fb0 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f9607fff640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9608099fb0 0x7f96081e4290 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f960002ac70 tx=0x7f9600004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:30.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f95effff640 1 -- 172.21.15.38:0/1307118111 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960003f070 con 0x7f9608099fb0 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f95effff640 1 -- 172.21.15.38:0/1307118111 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9600031bc0 con 0x7f9608099fb0 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f960e1c4640 1 -- 172.21.15.38:0/1307118111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96081e4ba0 con 0x7f9608099fb0 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f95effff640 1 -- 172.21.15.38:0/1307118111 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9600009450 con 0x7f9608099fb0 2024-04-25T08:40:30.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.870+0000 7f960e1c4640 1 -- 172.21.15.38:0/1307118111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96080c5390 con 0x7f9608099fb0 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.871+0000 7f960e1c4640 1 -- 172.21.15.38:0/1307118111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9608068e80 con 0x7f9608099fb0 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.872+0000 7f95effff640 1 -- 172.21.15.38:0/1307118111 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f960003b410 con 0x7f9608099fb0 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.872+0000 7f95effff640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f406b4d0 0x7f95f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.872+0000 7f95effff640 1 -- 172.21.15.38:0/1307118111 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f96000abaf0 con 0x7f9608099fb0 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.872+0000 7f96077fe640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f406b4d0 0x7f95f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:30.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.873+0000 7f96077fe640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f406b4d0 0x7f95f406d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f95f8005880 tx=0x7f95f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:30.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:30.876+0000 7f95effff640 1 -- 172.21.15.38:0/1307118111 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f960007a7e0 con 0x7f9608099fb0 2024-04-25T08:40:31.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.129+0000 7f960e1c4640 1 -- 172.21.15.38:0/1307118111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f96080c5560 con 0x7f9608099fb0 2024-04-25T08:40:31.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.130+0000 7f95effff640 1 -- 172.21.15.38:0/1307118111 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f960007a0e0 con 0x7f9608099fb0 2024-04-25T08:40:31.124 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:31.124 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":560,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077824439,"bytes_used":2674696192,"bytes_avail":770419417088,"bytes_total":773094113280,"write_bytes_sec":41443609,"read_op_per_sec":0,"write_op_per_sec":42},"fsmap":{"epoch":70,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":7639}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:40:25.328586+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T08:40:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.132+0000 7f960e1c4640 1 -- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f406b4d0 msgr2=0x7f95f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.132+0000 7f960e1c4640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f406b4d0 0x7f95f406d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f95f8005880 tx=0x7f95f8009040 comp rx=0 tx=0).stop 2024-04-25T08:40:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.132+0000 7f960e1c4640 1 -- 172.21.15.38:0/1307118111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9608099fb0 msgr2=0x7f96081e4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.132+0000 7f960e1c4640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9608099fb0 0x7f96081e4290 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f960002ac70 tx=0x7f9600004a40 comp rx=0 tx=0).stop 2024-04-25T08:40:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.132+0000 7f960e1c4640 1 -- 172.21.15.38:0/1307118111 shutdown_connections 2024-04-25T08:40:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.132+0000 7f960e1c4640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9608099fb0 0x7f96081e4290 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.132+0000 7f960e1c4640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f960806cff0 0x7f96081e3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.132+0000 7f960e1c4640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f95f406b4d0 0x7f95f406d990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.132+0000 7f960e1c4640 1 --2- 172.21.15.38:0/1307118111 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f960806cc10 0x7f96081e7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:31.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.132+0000 7f960e1c4640 1 -- 172.21.15.38:0/1307118111 >> 172.21.15.38:0/1307118111 conn(0x7f96080b31f0 msgr2=0x7f96081e53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:31.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.132+0000 7f960e1c4640 1 -- 172.21.15.38:0/1307118111 shutdown_connections 2024-04-25T08:40:31.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:31.132+0000 7f960e1c4640 1 -- 172.21.15.38:0/1307118111 wait complete. 2024-04-25T08:40:31.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T08:40:32.137 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:32.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.259+0000 7f3d1c1e7640 1 Processor -- start 2024-04-25T08:40:32.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.260+0000 7f3d1c1e7640 1 -- start start 2024-04-25T08:40:32.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.260+0000 7f3d1c1e7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d1415da90 0x7f3d140d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d1c1e7640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3d140ceca0 0x7f3d140d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d1c1e7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d140d1b60 con 0x7f3d140d3de0 2024-04-25T08:40:32.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d1c1e7640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d140d2bc0 con 0x7f3d140ceca0 2024-04-25T08:40:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d1c1e7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140d1e00 con 0x7f3d1415da90 2024-04-25T08:40:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d19f83640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d1415da90 0x7f3d140d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d19f83640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d1415da90 0x7f3d140d2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:43702/0 (socket says 172.21.15.38:43702) 2024-04-25T08:40:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d19f83640 1 -- 172.21.15.38:0/1805722438 learned_addr learned my addr 172.21.15.38:0/1805722438 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d19782640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f3d140ceca0 0x7f3d140d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d18f81640 1 -- 172.21.15.38:0/1805722438 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799766159 0 0) 0x7f3d140d1b60 con 0x7f3d140d3de0 2024-04-25T08:40:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d18f81640 1 -- 172.21.15.38:0/1805722438 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d00003520 con 0x7f3d140d3de0 2024-04-25T08:40:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d19f83640 1 -- 172.21.15.38:0/1805722438 >> v2:172.21.15.107:3300/0 conn(0x7f3d140ceca0 msgr2=0x7f3d140d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d19f83640 1 --2- 172.21.15.38:0/1805722438 >> v2:172.21.15.107:3300/0 conn(0x7f3d140ceca0 0x7f3d140d38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.261+0000 7f3d19f83640 1 -- 172.21.15.38:0/1805722438 >> v1:172.21.15.107:6789/0 conn(0x7f3d140d3de0 legacy=0x7f3d140d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.262+0000 7f3d19f83640 1 -- 172.21.15.38:0/1805722438 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d140d20e0 con 0x7f3d1415da90 2024-04-25T08:40:32.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.262+0000 7f3d19782640 1 --2- 172.21.15.38:0/1805722438 >> v2:172.21.15.107:3300/0 conn(0x7f3d140ceca0 0x7f3d140d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.262+0000 7f3d19f83640 1 --2- 172.21.15.38:0/1805722438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d1415da90 0x7f3d140d2f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009870 tx=0x7f3d1002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=518b7199ed71fec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.262+0000 7f3d18f81640 1 -- 172.21.15.38:0/1805722438 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10037070 con 0x7f3d1415da90 2024-04-25T08:40:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.263+0000 7f3d18f81640 1 -- 172.21.15.38:0/1805722438 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d10032440 con 0x7f3d1415da90 2024-04-25T08:40:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.263+0000 7f3d18f81640 1 -- 172.21.15.38:0/1805722438 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10032810 con 0x7f3d1415da90 2024-04-25T08:40:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.263+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/1805722438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d1415da90 msgr2=0x7f3d140d2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.263+0000 7f3d1c1e7640 1 --2- 172.21.15.38:0/1805722438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d1415da90 0x7f3d140d2f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009870 tx=0x7f3d1002a270 comp rx=0 tx=0).stop 2024-04-25T08:40:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.263+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/1805722438 shutdown_connections 2024-04-25T08:40:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.263+0000 7f3d1c1e7640 1 --2- 172.21.15.38:0/1805722438 >> v2:172.21.15.107:3300/0 conn(0x7f3d140ceca0 0x7f3d140d38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.263+0000 7f3d1c1e7640 1 --2- 172.21.15.38:0/1805722438 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d1415da90 0x7f3d140d2f70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.263+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/1805722438 >> 172.21.15.38:0/1805722438 conn(0x7f3d1406bed0 msgr2=0x7f3d1406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:32.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.263+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/1805722438 shutdown_connections 2024-04-25T08:40:32.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.263+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/1805722438 wait complete. 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.264+0000 7f3d1c1e7640 1 Processor -- start 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.264+0000 7f3d1c1e7640 1 -- start start 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.265+0000 7f3d1c1e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d140d3de0 0x7f3d14150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.265+0000 7f3d1c1e7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d1415da90 0x7f3d141604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.265+0000 7f3d1c1e7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d141609e0 0x7f3d140cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.265+0000 7f3d1c1e7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1415dff0 con 0x7f3d140d3de0 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.265+0000 7f3d1c1e7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1415de70 con 0x7f3d1415da90 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.265+0000 7f3d1c1e7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d140d34b0 con 0x7f3d141609e0 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.265+0000 7f3d19f83640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d140d3de0 0x7f3d14150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.265+0000 7f3d19f83640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d140d3de0 0x7f3d14150e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51960/0 (socket says 172.21.15.38:51960) 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.265+0000 7f3d19f83640 1 -- 172.21.15.38:0/2776642560 learned_addr learned my addr 172.21.15.38:0/2776642560 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.265+0000 7f3d19782640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d1415da90 0x7f3d141604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.265+0000 7f3d1a784640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d141609e0 0x7f3d140cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.266+0000 7f3d19f83640 1 -- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d141609e0 msgr2=0x7f3d140cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.266+0000 7f3d19f83640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d141609e0 0x7f3d140cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.266+0000 7f3d19f83640 1 -- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d1415da90 msgr2=0x7f3d141604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.266+0000 7f3d19f83640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d1415da90 0x7f3d141604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.266+0000 7f3d19f83640 1 -- 172.21.15.38:0/2776642560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d14160eb0 con 0x7f3d140d3de0 2024-04-25T08:40:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.266+0000 7f3d19f83640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d140d3de0 0x7f3d14150e10 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f3d10000c00 tx=0x7f3d100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.266+0000 7f3d0a7fc640 1 -- 172.21.15.38:0/2776642560 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10041020 con 0x7f3d140d3de0 2024-04-25T08:40:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.266+0000 7f3d0a7fc640 1 -- 172.21.15.38:0/2776642560 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1003d620 con 0x7f3d140d3de0 2024-04-25T08:40:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.266+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/2776642560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d140ca9e0 con 0x7f3d140d3de0 2024-04-25T08:40:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.266+0000 7f3d0a7fc640 1 -- 172.21.15.38:0/2776642560 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10037040 con 0x7f3d140d3de0 2024-04-25T08:40:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.266+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/2776642560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d140cafa0 con 0x7f3d140d3de0 2024-04-25T08:40:32.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.267+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/2776642560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d14068e80 con 0x7f3d140d3de0 2024-04-25T08:40:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.268+0000 7f3d0a7fc640 1 -- 172.21.15.38:0/2776642560 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d10044020 con 0x7f3d140d3de0 2024-04-25T08:40:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.268+0000 7f3d0a7fc640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3cf806b4d0 0x7f3cf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.268+0000 7f3d0a7fc640 1 -- 172.21.15.38:0/2776642560 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d100aa910 con 0x7f3d140d3de0 2024-04-25T08:40:32.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.268+0000 7f3d19782640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3cf806b4d0 0x7f3cf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.269+0000 7f3d19782640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3cf806b4d0 0x7f3cf806d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009560 tx=0x7f3d04009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:32.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.273+0000 7f3d0a7fc640 1 -- 172.21.15.38:0/2776642560 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d1004b030 con 0x7f3d140d3de0 2024-04-25T08:40:32.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.489+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/2776642560 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d140d16c0 con 0x7f3d140d3de0 2024-04-25T08:40:32.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.490+0000 7f3d0a7fc640 1 -- 172.21.15.38:0/2776642560 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f3d10078f10 con 0x7f3d140d3de0 2024-04-25T08:40:32.484 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:32.484 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:32.484 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:32.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.492+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3cf806b4d0 msgr2=0x7f3cf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.492+0000 7f3d1c1e7640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3cf806b4d0 0x7f3cf806d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3d04009560 tx=0x7f3d04009190 comp rx=0 tx=0).stop 2024-04-25T08:40:32.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.492+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/2776642560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d140d3de0 msgr2=0x7f3d14150e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.492+0000 7f3d1c1e7640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d140d3de0 0x7f3d14150e10 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f3d10000c00 tx=0x7f3d100046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.492+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/2776642560 shutdown_connections 2024-04-25T08:40:32.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.492+0000 7f3d1c1e7640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3cf806b4d0 0x7f3cf806d990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.492+0000 7f3d1c1e7640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3d141609e0 0x7f3d140cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.492+0000 7f3d1c1e7640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3d1415da90 0x7f3d141604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.492+0000 7f3d1c1e7640 1 --2- 172.21.15.38:0/2776642560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3d140d3de0 0x7f3d14150e10 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.492+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/2776642560 >> 172.21.15.38:0/2776642560 conn(0x7f3d1406bed0 msgr2=0x7f3d140b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:32.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.492+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/2776642560 shutdown_connections 2024-04-25T08:40:32.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.492+0000 7f3d1c1e7640 1 -- 172.21.15.38:0/2776642560 wait complete. 2024-04-25T08:40:32.496 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:40:32.497 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:40:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.602+0000 7ff63d015640 1 Processor -- start 2024-04-25T08:40:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.602+0000 7ff63d015640 1 -- start start 2024-04-25T08:40:32.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63d015640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff63815bb10 0x7ff63809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63d015640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff638160340 con 0x7ff63815fbf0 2024-04-25T08:40:32.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63d015640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6380cf400 con 0x7ff6380d0460 2024-04-25T08:40:32.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63d015640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6380cffd0 con 0x7ff63815bb10 2024-04-25T08:40:32.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63759e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff63815fbf0 0x7ff6380c5250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:47996/0 (socket says 172.21.15.38:47996) 2024-04-25T08:40:32.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63759e640 1 -- 172.21.15.38:0/2229518490 learned_addr learned my addr 172.21.15.38:0/2229518490 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:32.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff636d9d640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff63815bb10 0x7ff63809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773672763 0 0) 0x7ff628001110 con 0x7ff63815fbf0 2024-04-25T08:40:32.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff620003520 con 0x7ff63815fbf0 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857446916 0 0) 0x7ff6380cf400 con 0x7ff6380d0460 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff628001110 con 0x7ff6380d0460 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2875989037 0 0) 0x7ff620003520 con 0x7ff63815fbf0 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff63815bb10 msgr2=0x7ff63809a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff63815bb10 0x7ff63809a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 >> v1:172.21.15.107:6789/0 conn(0x7ff6380d0460 legacy=0x7ff63815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6380cf6a0 con 0x7ff63815fbf0 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7ff628001840 con 0x7ff63815fbf0 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff636d9d640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff63815bb10 0x7ff63809a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 >> v1:172.21.15.38:6789/0 conn(0x7ff63815fbf0 legacy=0x7ff6380c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff620003b70 0x7ff620003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff63815bb10 0x7ff620004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff620004d40 0x7ff6200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6380cffd0 con 0x7ff620004d40 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62000baa0 con 0x7ff620003b70 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff62000bca0 con 0x7ff63815bb10 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63759e640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff620004d40 0x7ff6200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff636d9d640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff620003b70 0x7ff620003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63659c640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff63815bb10 0x7ff620004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63759e640 1 -- 172.21.15.38:0/2229518490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff63815bb10 msgr2=0x7ff620004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63759e640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff63815bb10 0x7ff620004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63759e640 1 -- 172.21.15.38:0/2229518490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff620003b70 msgr2=0x7ff620003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63759e640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff620003b70 0x7ff620003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.604+0000 7ff63759e640 1 -- 172.21.15.38:0/2229518490 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff620003520 con 0x7ff620004d40 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.605+0000 7ff63759e640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff620004d40 0x7ff6200051c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7ff6380690b0 tx=0x7ff620005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d391c6bcc41350 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.605+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff628035070 con 0x7ff620004d40 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.605+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6280058d0 con 0x7ff620004d40 2024-04-25T08:40:32.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.605+0000 7ff635d9b640 1 -- 172.21.15.38:0/2229518490 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62800c0e0 con 0x7ff620004d40 2024-04-25T08:40:32.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.607+0000 7ff63d015640 1 -- 172.21.15.38:0/2229518490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff620004d40 msgr2=0x7ff6200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.607+0000 7ff63d015640 1 --2- 172.21.15.38:0/2229518490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff620004d40 0x7ff6200051c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7ff6380690b0 tx=0x7ff620005830 comp rx=0 tx=0).stop 2024-04-25T08:40:32.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.607+0000 7ff63759e640 1 -- 172.21.15.38:0/2229518490 reap_dead start 2024-04-25T08:40:32.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.607+0000 7ff63d015640 1 -- 172.21.15.38:0/2229518490 shutdown_connections 2024-04-25T08:40:32.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.607+0000 7ff63d015640 1 -- 172.21.15.38:0/2229518490 >> 172.21.15.38:0/2229518490 conn(0x7ff6380b31f0 msgr2=0x7ff6380b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:32.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.607+0000 7ff63d015640 1 -- 172.21.15.38:0/2229518490 shutdown_connections 2024-04-25T08:40:32.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.607+0000 7ff63d015640 1 -- 172.21.15.38:0/2229518490 wait complete. 2024-04-25T08:40:32.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63d015640 1 Processor -- start 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63d015640 1 -- start start 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63d015640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff620004d40 0x7ff6380b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63d015640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6380b4780 0x7ff6381e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63d015640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6381e7300 0x7ff6381e7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63d015640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6380c5960 con 0x7ff6381e7300 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63d015640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6380cfc60 con 0x7ff620004d40 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63d015640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6380c5ae0 con 0x7ff6380b4780 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6381e7300 0x7ff6381e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63759e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6381e7300 0x7ff6381e7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51982/0 (socket says 172.21.15.38:51982) 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63759e640 1 -- 172.21.15.38:0/1475314216 learned_addr learned my addr 172.21.15.38:0/1475314216 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff636d9d640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff620004d40 0x7ff6380b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63659c640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6380b4780 0x7ff6381e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63759e640 1 -- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6380b4780 msgr2=0x7ff6381e6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63759e640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6380b4780 0x7ff6381e6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63759e640 1 -- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff620004d40 msgr2=0x7ff6380b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63759e640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff620004d40 0x7ff6380b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.608+0000 7ff63759e640 1 -- 172.21.15.38:0/1475314216 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63815ca50 con 0x7ff6381e7300 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.610+0000 7ff63759e640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6381e7300 0x7ff6381e7780 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ff628033040 tx=0x7ff628001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.610+0000 7ff61f7fe640 1 -- 172.21.15.38:0/1475314216 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff628042070 con 0x7ff6381e7300 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.610+0000 7ff61f7fe640 1 -- 172.21.15.38:0/1475314216 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6280075b0 con 0x7ff6381e7300 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.610+0000 7ff63d015640 1 -- 172.21.15.38:0/1475314216 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63815cd30 con 0x7ff6381e7300 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.610+0000 7ff61f7fe640 1 -- 172.21.15.38:0/1475314216 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff628035030 con 0x7ff6381e7300 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.610+0000 7ff63d015640 1 -- 172.21.15.38:0/1475314216 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63815d2f0 con 0x7ff6381e7300 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.611+0000 7ff63d015640 1 -- 172.21.15.38:0/1475314216 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff638068e80 con 0x7ff6381e7300 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.611+0000 7ff61f7fe640 1 -- 172.21.15.38:0/1475314216 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff628045070 con 0x7ff6381e7300 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.612+0000 7ff61f7fe640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff62406b500 0x7ff62406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.612+0000 7ff61f7fe640 1 -- 172.21.15.38:0/1475314216 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff6280a9590 con 0x7ff6381e7300 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.612+0000 7ff636d9d640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff62406b500 0x7ff62406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:32.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.612+0000 7ff636d9d640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff62406b500 0x7ff62406d9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff630014510 tx=0x7ff63000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:32.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.616+0000 7ff61f7fe640 1 -- 172.21.15.38:0/1475314216 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff638160340 con 0x7ff6381e7300 2024-04-25T08:40:32.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.866+0000 7ff63d015640 1 -- 172.21.15.38:0/1475314216 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff6380b4b60 con 0x7ff6381e7300 2024-04-25T08:40:32.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.866+0000 7ff61f7fe640 1 -- 172.21.15.38:0/1475314216 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7ff628077ba0 con 0x7ff6381e7300 2024-04-25T08:40:32.860 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:32.860 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":561,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077824439,"bytes_used":2674696192,"bytes_avail":770419417088,"bytes_total":773094113280,"write_bytes_sec":27140768,"read_op_per_sec":0,"write_op_per_sec":28},"fsmap":{"epoch":70,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":7639}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:40:25.328586+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T08:40:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.868+0000 7ff63d015640 1 -- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff62406b500 msgr2=0x7ff62406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.868+0000 7ff63d015640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff62406b500 0x7ff62406d9c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff630014510 tx=0x7ff63000b040 comp rx=0 tx=0).stop 2024-04-25T08:40:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.868+0000 7ff63d015640 1 -- 172.21.15.38:0/1475314216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6381e7300 msgr2=0x7ff6381e7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.868+0000 7ff63d015640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6381e7300 0x7ff6381e7780 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ff628033040 tx=0x7ff628001b90 comp rx=0 tx=0).stop 2024-04-25T08:40:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.868+0000 7ff63d015640 1 -- 172.21.15.38:0/1475314216 shutdown_connections 2024-04-25T08:40:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.868+0000 7ff63d015640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff6381e7300 0x7ff6381e7780 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.868+0000 7ff63d015640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff62406b500 0x7ff62406d9c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.868+0000 7ff63d015640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff6380b4780 0x7ff6381e6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.868+0000 7ff63d015640 1 --2- 172.21.15.38:0/1475314216 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff620004d40 0x7ff6380b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.868+0000 7ff63d015640 1 -- 172.21.15.38:0/1475314216 >> 172.21.15.38:0/1475314216 conn(0x7ff6380b31f0 msgr2=0x7ff638099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.868+0000 7ff63d015640 1 -- 172.21.15.38:0/1475314216 shutdown_connections 2024-04-25T08:40:32.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:32.868+0000 7ff63d015640 1 -- 172.21.15.38:0/1475314216 wait complete. 2024-04-25T08:40:32.872 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T08:40:33.874 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.993+0000 7f2af97ce640 1 Processor -- start 2024-04-25T08:40:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.993+0000 7f2af97ce640 1 -- start start 2024-04-25T08:40:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.994+0000 7f2af97ce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af415df50 0x7f2af40ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.994+0000 7f2af97ce640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2af40ce2e0 0x7f2af40d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.994+0000 7f2af97ce640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af40ceff0 con 0x7f2af406c110 2024-04-25T08:40:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.994+0000 7f2af97ce640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af40cf2d0 con 0x7f2af40ce2e0 2024-04-25T08:40:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.994+0000 7f2af97ce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40d1f70 con 0x7f2af415df50 2024-04-25T08:40:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.994+0000 7f2af2ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2af406c110 0x7f2af406c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:48000/0 (socket says 172.21.15.38:48000) 2024-04-25T08:40:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.994+0000 7f2af2ffd640 1 -- 172.21.15.38:0/798203170 learned_addr learned my addr 172.21.15.38:0/798203170 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.994+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802386986 0 0) 0x7f2af40ceff0 con 0x7f2af406c110 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af37fe640 1 --2- 172.21.15.38:0/798203170 >> v2:172.21.15.107:3300/0 conn(0x7f2af40ce2e0 0x7f2af40d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af27fc640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af415df50 0x7f2af40ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ae0003520 con 0x7f2af406c110 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1598687604 0 0) 0x7f2ae0003520 con 0x7f2af406c110 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af415df50 msgr2=0x7f2af40ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af415df50 0x7f2af40ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 >> v2:172.21.15.107:3300/0 conn(0x7f2af40ce2e0 msgr2=0x7f2af40d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 --2- 172.21.15.38:0/798203170 >> v2:172.21.15.107:3300/0 conn(0x7f2af40ce2e0 0x7f2af40d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af40d2250 con 0x7f2af406c110 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f2ae40019f0 con 0x7f2af406c110 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af27fc640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af415df50 0x7f2af40ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 >> v1:172.21.15.38:6789/0 conn(0x7f2af406c110 legacy=0x7f2af406c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ae00039b0 0x7f2ae0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af415df50 0x7f2ae0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ae0004d40 0x7f2ae00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40cf2d0 con 0x7f2ae0004d40 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40d1f70 con 0x7f2ae00039b0 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae000b9c0 con 0x7f2af415df50 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af2ffd640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ae00039b0 0x7f2ae0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af37fe640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ae0004d40 0x7f2ae00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af27fc640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af415df50 0x7f2ae0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af37fe640 1 -- 172.21.15.38:0/798203170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af415df50 msgr2=0x7f2ae0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af37fe640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af415df50 0x7f2ae0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af37fe640 1 -- 172.21.15.38:0/798203170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ae00039b0 msgr2=0x7f2ae0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af37fe640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ae00039b0 0x7f2ae0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.995+0000 7f2af37fe640 1 -- 172.21.15.38:0/798203170 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae0003520 con 0x7f2ae0004d40 2024-04-25T08:40:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.996+0000 7f2af37fe640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ae0004d40 0x7f2ae00051c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009a90 tx=0x7f2ae0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da38dd362dbf258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.996+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec036070 con 0x7f2ae0004d40 2024-04-25T08:40:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.996+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec031400 con 0x7f2ae0004d40 2024-04-25T08:40:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.996+0000 7f2af1ffb640 1 -- 172.21.15.38:0/798203170 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec009690 con 0x7f2ae0004d40 2024-04-25T08:40:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.997+0000 7f2af97ce640 1 -- 172.21.15.38:0/798203170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ae0004d40 msgr2=0x7f2ae00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.997+0000 7f2af97ce640 1 --2- 172.21.15.38:0/798203170 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ae0004d40 0x7f2ae00051c0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009a90 tx=0x7f2ae0005830 comp rx=0 tx=0).stop 2024-04-25T08:40:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.997+0000 7f2af37fe640 1 -- 172.21.15.38:0/798203170 reap_dead start 2024-04-25T08:40:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.997+0000 7f2af97ce640 1 -- 172.21.15.38:0/798203170 shutdown_connections 2024-04-25T08:40:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.997+0000 7f2af97ce640 1 -- 172.21.15.38:0/798203170 >> 172.21.15.38:0/798203170 conn(0x7f2af40d3fb0 msgr2=0x7f2af40d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.997+0000 7f2af97ce640 1 -- 172.21.15.38:0/798203170 shutdown_connections 2024-04-25T08:40:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.997+0000 7f2af97ce640 1 -- 172.21.15.38:0/798203170 wait complete. 2024-04-25T08:40:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.998+0000 7f2af97ce640 1 Processor -- start 2024-04-25T08:40:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.998+0000 7f2af97ce640 1 -- start start 2024-04-25T08:40:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af97ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ae0004d40 0x7f2af415fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af97ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af4154c10 0x7f2af415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af97ce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af40c9600 0x7f2af4155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af97ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40d3110 con 0x7f2ae0004d40 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af97ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40d2f90 con 0x7f2af4154c10 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af97ce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40d3290 con 0x7f2af40c9600 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ae0004d40 0x7f2af415fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ae0004d40 0x7f2af415fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52006/0 (socket says 172.21.15.38:52006) 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af2ffd640 1 -- 172.21.15.38:0/237726190 learned_addr learned my addr 172.21.15.38:0/237726190 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af37fe640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af40c9600 0x7f2af4155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af27fc640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af4154c10 0x7f2af415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af2ffd640 1 -- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af40c9600 msgr2=0x7f2af4155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:33.999+0000 7f2af2ffd640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af40c9600 0x7f2af4155770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.000+0000 7f2af2ffd640 1 -- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af4154c10 msgr2=0x7f2af415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.000+0000 7f2af2ffd640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af4154c10 0x7f2af415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.000+0000 7f2af2ffd640 1 -- 172.21.15.38:0/237726190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af40c8840 con 0x7f2ae0004d40 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.000+0000 7f2af27fc640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af4154c10 0x7f2af415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.000+0000 7f2af2ffd640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ae0004d40 0x7f2af415fd90 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4007a20 tx=0x7f2ae400a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:33.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.000+0000 7f2aeb7fe640 1 -- 172.21.15.38:0/237726190 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4017020 con 0x7f2ae0004d40 2024-04-25T08:40:33.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.000+0000 7f2aeb7fe640 1 -- 172.21.15.38:0/237726190 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae40019b0 con 0x7f2ae0004d40 2024-04-25T08:40:33.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.000+0000 7f2af97ce640 1 -- 172.21.15.38:0/237726190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af40c8b20 con 0x7f2ae0004d40 2024-04-25T08:40:33.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.000+0000 7f2aeb7fe640 1 -- 172.21.15.38:0/237726190 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4013600 con 0x7f2ae0004d40 2024-04-25T08:40:33.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.000+0000 7f2af97ce640 1 -- 172.21.15.38:0/237726190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af40c8460 con 0x7f2ae0004d40 2024-04-25T08:40:33.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.001+0000 7f2af97ce640 1 -- 172.21.15.38:0/237726190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af4068e80 con 0x7f2ae0004d40 2024-04-25T08:40:33.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.001+0000 7f2aeb7fe640 1 -- 172.21.15.38:0/237726190 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ae401a020 con 0x7f2ae0004d40 2024-04-25T08:40:33.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.002+0000 7f2aeb7fe640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ad406b4d0 0x7f2ad406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:33.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.002+0000 7f2aeb7fe640 1 -- 172.21.15.38:0/237726190 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ae408b0b0 con 0x7f2ae0004d40 2024-04-25T08:40:33.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.002+0000 7f2af27fc640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ad406b4d0 0x7f2ad406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:33.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.003+0000 7f2af27fc640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ad406b4d0 0x7f2ad406d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2af40b3a60 tx=0x7f2adc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:33.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.006+0000 7f2aeb7fe640 1 -- 172.21.15.38:0/237726190 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ae4021020 con 0x7f2ae0004d40 2024-04-25T08:40:34.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.224+0000 7f2af97ce640 1 -- 172.21.15.38:0/237726190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2af40ca780 con 0x7f2ae0004d40 2024-04-25T08:40:34.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.225+0000 7f2aeb7fe640 1 -- 172.21.15.38:0/237726190 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2ae4059c70 con 0x7f2ae0004d40 2024-04-25T08:40:34.218 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:34.218 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:34.218 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:34.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.228+0000 7f2af97ce640 1 -- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ad406b4d0 msgr2=0x7f2ad406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:34.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.228+0000 7f2af97ce640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ad406b4d0 0x7f2ad406d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2af40b3a60 tx=0x7f2adc008040 comp rx=0 tx=0).stop 2024-04-25T08:40:34.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.228+0000 7f2af97ce640 1 -- 172.21.15.38:0/237726190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ae0004d40 msgr2=0x7f2af415fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:34.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.228+0000 7f2af97ce640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ae0004d40 0x7f2af415fd90 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4007a20 tx=0x7f2ae400a750 comp rx=0 tx=0).stop 2024-04-25T08:40:34.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.228+0000 7f2af97ce640 1 -- 172.21.15.38:0/237726190 shutdown_connections 2024-04-25T08:40:34.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.228+0000 7f2af97ce640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2ad406b4d0 0x7f2ad406d990 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:34.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.228+0000 7f2af97ce640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2af40c9600 0x7f2af4155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:34.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.228+0000 7f2af97ce640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2af4154c10 0x7f2af415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:34.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.228+0000 7f2af97ce640 1 --2- 172.21.15.38:0/237726190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ae0004d40 0x7f2af415fd90 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:34.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.228+0000 7f2af97ce640 1 -- 172.21.15.38:0/237726190 >> 172.21.15.38:0/237726190 conn(0x7f2af40d3fb0 msgr2=0x7f2af40d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:34.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.228+0000 7f2af97ce640 1 -- 172.21.15.38:0/237726190 shutdown_connections 2024-04-25T08:40:34.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.228+0000 7f2af97ce640 1 -- 172.21.15.38:0/237726190 wait complete. 2024-04-25T08:40:34.233 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:40:34.233 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:40:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.342+0000 7f53af0b3640 1 Processor -- start 2024-04-25T08:40:34.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.342+0000 7f53af0b3640 1 -- start start 2024-04-25T08:40:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.343+0000 7f53af0b3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f53a815bc00 0x7f53a80cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.343+0000 7f53af0b3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a80ce360 con 0x7f53a80b5030 2024-04-25T08:40:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.343+0000 7f53af0b3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a80ce640 con 0x7f53a8143f40 2024-04-25T08:40:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.343+0000 7f53af0b3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a80ce920 con 0x7f53a815bc00 2024-04-25T08:40:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.343+0000 7f53ad650640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f53a815bc00 0x7f53a80cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.343+0000 7f53a7fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f53a80b5030 0x7f53a80cd630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:48012/0 (socket says 172.21.15.38:48012) 2024-04-25T08:40:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.343+0000 7f53a7fff640 1 -- 172.21.15.38:0/2462674897 learned_addr learned my addr 172.21.15.38:0/2462674897 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.343+0000 7f53ad650640 1 -- 172.21.15.38:0/2462674897 >> v1:172.21.15.107:6789/0 conn(0x7f53a8143f40 legacy=0x7f53a80b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:34.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.343+0000 7f53ad650640 1 -- 172.21.15.38:0/2462674897 >> v1:172.21.15.38:6789/0 conn(0x7f53a80b5030 legacy=0x7f53a80cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.343+0000 7f53ad650640 1 -- 172.21.15.38:0/2462674897 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a80cec00 con 0x7f53a815bc00 2024-04-25T08:40:34.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.343+0000 7f53a77fe640 1 -- 172.21.15.38:0/2462674897 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108259355 0 0) 0x7f53a80ce360 con 0x7f53a80b5030 2024-04-25T08:40:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.344+0000 7f53ad650640 1 --2- 172.21.15.38:0/2462674897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a815bc00 0x7f53a80cdd40 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009d90 tx=0x7f53a002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d390fe965a4deb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.344+0000 7f53a77fe640 1 -- 172.21.15.38:0/2462674897 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0007740 con 0x7f53a815bc00 2024-04-25T08:40:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.344+0000 7f53a77fe640 1 -- 172.21.15.38:0/2462674897 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a00078e0 con 0x7f53a815bc00 2024-04-25T08:40:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.344+0000 7f53a77fe640 1 -- 172.21.15.38:0/2462674897 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0007cb0 con 0x7f53a815bc00 2024-04-25T08:40:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.345+0000 7f53af0b3640 1 -- 172.21.15.38:0/2462674897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a815bc00 msgr2=0x7f53a80cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.345+0000 7f53af0b3640 1 --2- 172.21.15.38:0/2462674897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a815bc00 0x7f53a80cdd40 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009d90 tx=0x7f53a002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:40:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.345+0000 7f53af0b3640 1 -- 172.21.15.38:0/2462674897 shutdown_connections 2024-04-25T08:40:34.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.345+0000 7f53af0b3640 1 --2- 172.21.15.38:0/2462674897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a815bc00 0x7f53a80cdd40 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:34.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.345+0000 7f53af0b3640 1 -- 172.21.15.38:0/2462674897 >> 172.21.15.38:0/2462674897 conn(0x7f53a806b210 msgr2=0x7f53a806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:34.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.345+0000 7f53af0b3640 1 -- 172.21.15.38:0/2462674897 shutdown_connections 2024-04-25T08:40:34.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.345+0000 7f53af0b3640 1 -- 172.21.15.38:0/2462674897 wait complete. 2024-04-25T08:40:34.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.346+0000 7f53af0b3640 1 Processor -- start 2024-04-25T08:40:34.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.346+0000 7f53af0b3640 1 -- start start 2024-04-25T08:40:34.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53af0b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53a80b5030 0x7f53a813aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:34.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53af0b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a8143f40 0x7f53a8149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53af0b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a815bc00 0x7f53a813f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53af0b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a81444a0 con 0x7f53a815bc00 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53af0b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a8144320 con 0x7f53a8143f40 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53af0b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a8165380 con 0x7f53a80b5030 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53ad650640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a815bc00 0x7f53a813f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53ad650640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a815bc00 0x7f53a813f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52018/0 (socket says 172.21.15.38:52018) 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53ad650640 1 -- 172.21.15.38:0/3072685665 learned_addr learned my addr 172.21.15.38:0/3072685665 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53a7fff640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a8143f40 0x7f53a8149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53ace4f640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53a80b5030 0x7f53a813aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53ad650640 1 -- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53a80b5030 msgr2=0x7f53a813aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53ad650640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53a80b5030 0x7f53a813aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53ad650640 1 -- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a8143f40 msgr2=0x7f53a8149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:34.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53ad650640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a8143f40 0x7f53a8149eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53ad650640 1 -- 172.21.15.38:0/3072685665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a8141860 con 0x7f53a815bc00 2024-04-25T08:40:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.347+0000 7f53ace4f640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53a80b5030 0x7f53a813aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.348+0000 7f53ad650640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a815bc00 0x7f53a813f010 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009eb0 tx=0x7f53a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.348+0000 7f53a57fa640 1 -- 172.21.15.38:0/3072685665 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a003f070 con 0x7f53a815bc00 2024-04-25T08:40:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.348+0000 7f53a57fa640 1 -- 172.21.15.38:0/3072685665 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a0031c90 con 0x7f53a815bc00 2024-04-25T08:40:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.348+0000 7f53af0b3640 1 -- 172.21.15.38:0/3072685665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a8141b40 con 0x7f53a815bc00 2024-04-25T08:40:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.348+0000 7f53a57fa640 1 -- 172.21.15.38:0/3072685665 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0043630 con 0x7f53a815bc00 2024-04-25T08:40:34.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.348+0000 7f53af0b3640 1 -- 172.21.15.38:0/3072685665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a8142100 con 0x7f53a815bc00 2024-04-25T08:40:34.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.349+0000 7f53af0b3640 1 -- 172.21.15.38:0/3072685665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a8068e80 con 0x7f53a815bc00 2024-04-25T08:40:34.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.350+0000 7f53a57fa640 1 -- 172.21.15.38:0/3072685665 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53a003b410 con 0x7f53a815bc00 2024-04-25T08:40:34.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.350+0000 7f53a57fa640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f538c06b4d0 0x7f538c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:34.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.350+0000 7f53a57fa640 1 -- 172.21.15.38:0/3072685665 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53a00ab8c0 con 0x7f53a815bc00 2024-04-25T08:40:34.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.351+0000 7f53ace4f640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f538c06b4d0 0x7f538c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:34.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.351+0000 7f53ace4f640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f538c06b4d0 0x7f538c06d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f539c007fd0 tx=0x7f539c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:34.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.354+0000 7f53a57fa640 1 -- 172.21.15.38:0/3072685665 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53a007a5d0 con 0x7f53a815bc00 2024-04-25T08:40:34.572 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:40:34.575 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:40:34.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.610+0000 7f53af0b3640 1 -- 172.21.15.38:0/3072685665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f53a8154ef0 con 0x7f53a815bc00 2024-04-25T08:40:34.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.611+0000 7f53a57fa640 1 -- 172.21.15.38:0/3072685665 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f53a0079ed0 con 0x7f53a815bc00 2024-04-25T08:40:34.604 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:34.604 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":563,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078423686,"bytes_used":2675892224,"bytes_avail":770418221056,"bytes_total":773094113280,"write_bytes_sec":27190930,"read_op_per_sec":0,"write_op_per_sec":28},"fsmap":{"epoch":70,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":7639}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T08:40:25.328586+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T08:40:34.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.613+0000 7f53af0b3640 1 -- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f538c06b4d0 msgr2=0x7f538c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.613+0000 7f53af0b3640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f538c06b4d0 0x7f538c06d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f539c007fd0 tx=0x7f539c009040 comp rx=0 tx=0).stop 2024-04-25T08:40:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.613+0000 7f53af0b3640 1 -- 172.21.15.38:0/3072685665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a815bc00 msgr2=0x7f53a813f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.613+0000 7f53af0b3640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a815bc00 0x7f53a813f010 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009eb0 tx=0x7f53a0004a40 comp rx=0 tx=0).stop 2024-04-25T08:40:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.613+0000 7f53af0b3640 1 -- 172.21.15.38:0/3072685665 shutdown_connections 2024-04-25T08:40:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.613+0000 7f53af0b3640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53a815bc00 0x7f53a813f010 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.613+0000 7f53af0b3640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53a8143f40 0x7f53a8149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.613+0000 7f53af0b3640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f538c06b4d0 0x7f538c06d990 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.613+0000 7f53af0b3640 1 --2- 172.21.15.38:0/3072685665 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53a80b5030 0x7f53a813aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.613+0000 7f53af0b3640 1 -- 172.21.15.38:0/3072685665 >> 172.21.15.38:0/3072685665 conn(0x7f53a806b210 msgr2=0x7f53a813e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.613+0000 7f53af0b3640 1 -- 172.21.15.38:0/3072685665 shutdown_connections 2024-04-25T08:40:34.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:34.613+0000 7f53af0b3640 1 -- 172.21.15.38:0/3072685665 wait complete. 2024-04-25T08:40:34.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T08:40:35.619 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:35.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.731+0000 7f0b820b3640 1 Processor -- start 2024-04-25T08:40:35.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.731+0000 7f0b820b3640 1 -- start start 2024-04-25T08:40:35.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b820b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c06cd90 0x7f0b7c0cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:35.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b820b3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0b7c0cf470 0x7f0b7c167710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:35.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b820b3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b7c0ccd60 con 0x7f0b7c099a70 2024-04-25T08:40:35.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b820b3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7c1614d0 con 0x7f0b7c0cf470 2024-04-25T08:40:35.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b820b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c0cd970 con 0x7f0b7c06cd90 2024-04-25T08:40:35.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7b7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0b7c099a70 0x7f0b7c099e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:48020/0 (socket says 172.21.15.38:48020) 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7b7fe640 1 -- 172.21.15.38:0/2960257794 learned_addr learned my addr 172.21.15.38:0/2960257794 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7affd640 1 --2- 172.21.15.38:0/2960257794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c06cd90 0x7f0b7c0cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7bfff640 1 --2- 172.21.15.38:0/2960257794 >> v2:172.21.15.107:3300/0 conn(0x7f0b7c0cf470 0x7f0b7c167710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273065417 0 0) 0x7f0b7c0ccd60 con 0x7f0b7c099a70 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b64003520 con 0x7f0b7c099a70 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 659924368 0 0) 0x7f0b64003520 con 0x7f0b7c099a70 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c06cd90 msgr2=0x7f0b7c0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 --2- 172.21.15.38:0/2960257794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c06cd90 0x7f0b7c0cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 >> v2:172.21.15.107:3300/0 conn(0x7f0b7c0cf470 msgr2=0x7f0b7c167710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 --2- 172.21.15.38:0/2960257794 >> v2:172.21.15.107:3300/0 conn(0x7f0b7c0cf470 0x7f0b7c167710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c0cd040 con 0x7f0b7c099a70 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f0b6c001670 con 0x7f0b7c099a70 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 >> v1:172.21.15.38:6789/0 conn(0x7f0b7c099a70 legacy=0x7f0b7c099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 --2- 172.21.15.38:0/2960257794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b640039b0 0x7f0b64003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:35.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 --2- 172.21.15.38:0/2960257794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c06cd90 0x7f0b64004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 --2- 172.21.15.38:0/2960257794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b64004d40 0x7f0b640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c1614d0 con 0x7f0b640039b0 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c0cd970 con 0x7f0b64004d40 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6400b9c0 con 0x7f0b7c06cd90 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.732+0000 7f0b7b7fe640 1 --2- 172.21.15.38:0/2960257794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b640039b0 0x7f0b64003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.733+0000 7f0b7bfff640 1 --2- 172.21.15.38:0/2960257794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b64004d40 0x7f0b640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.733+0000 7f0b7affd640 1 --2- 172.21.15.38:0/2960257794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c06cd90 0x7f0b64004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.733+0000 7f0b7b7fe640 1 -- 172.21.15.38:0/2960257794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c06cd90 msgr2=0x7f0b64004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.733+0000 7f0b7b7fe640 1 --2- 172.21.15.38:0/2960257794 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c06cd90 0x7f0b64004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.733+0000 7f0b7b7fe640 1 -- 172.21.15.38:0/2960257794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b64004d40 msgr2=0x7f0b640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.733+0000 7f0b7b7fe640 1 --2- 172.21.15.38:0/2960257794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b64004d40 0x7f0b640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.733+0000 7f0b7b7fe640 1 -- 172.21.15.38:0/2960257794 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b64003520 con 0x7f0b640039b0 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.734+0000 7f0b7b7fe640 1 --2- 172.21.15.38:0/2960257794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b640039b0 0x7f0b64003dd0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c001050 tx=0x7f0b64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc0af493366ae83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.734+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c02e070 con 0x7f0b640039b0 2024-04-25T08:40:35.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.734+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6c005af0 con 0x7f0b640039b0 2024-04-25T08:40:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.734+0000 7f0b7a7fc640 1 -- 172.21.15.38:0/2960257794 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c007d70 con 0x7f0b640039b0 2024-04-25T08:40:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.734+0000 7f0b820b3640 1 -- 172.21.15.38:0/2960257794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b640039b0 msgr2=0x7f0b64003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.734+0000 7f0b820b3640 1 --2- 172.21.15.38:0/2960257794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b640039b0 0x7f0b64003dd0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c001050 tx=0x7f0b64005830 comp rx=0 tx=0).stop 2024-04-25T08:40:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.734+0000 7f0b7bfff640 1 -- 172.21.15.38:0/2960257794 reap_dead start 2024-04-25T08:40:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.734+0000 7f0b820b3640 1 -- 172.21.15.38:0/2960257794 shutdown_connections 2024-04-25T08:40:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.734+0000 7f0b820b3640 1 -- 172.21.15.38:0/2960257794 >> 172.21.15.38:0/2960257794 conn(0x7f0b7c0b3180 msgr2=0x7f0b7c0b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.734+0000 7f0b820b3640 1 -- 172.21.15.38:0/2960257794 shutdown_connections 2024-04-25T08:40:35.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.734+0000 7f0b820b3640 1 -- 172.21.15.38:0/2960257794 wait complete. 2024-04-25T08:40:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.735+0000 7f0b820b3640 1 Processor -- start 2024-04-25T08:40:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.736+0000 7f0b820b3640 1 -- start start 2024-04-25T08:40:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.736+0000 7f0b820b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b640039b0 0x7f0b7c0ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.736+0000 7f0b820b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c154b40 0x7f0b7c0b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.736+0000 7f0b820b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0b63f0 0x7f0b7c0c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.736+0000 7f0b820b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c0d0780 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.736+0000 7f0b820b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c0cd600 con 0x7f0b640039b0 2024-04-25T08:40:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.736+0000 7f0b820b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c0d0900 con 0x7f0b7c154b40 2024-04-25T08:40:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.736+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0b63f0 0x7f0b7c0c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.736+0000 7f0b7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0b63f0 0x7f0b7c0c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52036/0 (socket says 172.21.15.38:52036) 2024-04-25T08:40:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b7bfff640 1 -- 172.21.15.38:0/1590466993 learned_addr learned my addr 172.21.15.38:0/1590466993 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:35.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b7b7fe640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b640039b0 0x7f0b7c0ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b7affd640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c154b40 0x7f0b7c0b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b7bfff640 1 -- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c154b40 msgr2=0x7f0b7c0b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b7bfff640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c154b40 0x7f0b7c0b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b7bfff640 1 -- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b640039b0 msgr2=0x7f0b7c0ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b7bfff640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b640039b0 0x7f0b7c0ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b7bfff640 1 -- 172.21.15.38:0/1590466993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c0c9490 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b7bfff640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0b63f0 0x7f0b7c0c8760 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0b7740 tx=0x7f0b7400f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b63fff640 1 -- 172.21.15.38:0/1590466993 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7400fd10 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b63fff640 1 -- 172.21.15.38:0/1590466993 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b74015430 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b820b3640 1 -- 172.21.15.38:0/1590466993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7c0c9770 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b63fff640 1 -- 172.21.15.38:0/1590466993 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74015820 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.737+0000 7f0b820b3640 1 -- 172.21.15.38:0/1590466993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7c0c9d30 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.738+0000 7f0b820b3640 1 -- 172.21.15.38:0/1590466993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7c068e80 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.739+0000 7f0b63fff640 1 -- 172.21.15.38:0/1590466993 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b740159c0 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.739+0000 7f0b63fff640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b6806b500 0x7f0b6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.739+0000 7f0b63fff640 1 -- 172.21.15.38:0/1590466993 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0b7408b000 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.739+0000 7f0b7b7fe640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b6806b500 0x7f0b6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:35.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.740+0000 7f0b7b7fe640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b6806b500 0x7f0b6806d9c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c033000 tx=0x7f0b6c009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:35.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.743+0000 7f0b63fff640 1 -- 172.21.15.38:0/1590466993 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b74019030 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.960+0000 7f0b820b3640 1 -- 172.21.15.38:0/1590466993 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b7c067dd0 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.960+0000 7f0b63fff640 1 -- 172.21.15.38:0/1590466993 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0b74010040 con 0x7f0b7c0b63f0 2024-04-25T08:40:35.954 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:35.954 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:35.954 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:35.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.962+0000 7f0b820b3640 1 -- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b6806b500 msgr2=0x7f0b6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:35.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.962+0000 7f0b820b3640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b6806b500 0x7f0b6806d9c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c033000 tx=0x7f0b6c009030 comp rx=0 tx=0).stop 2024-04-25T08:40:35.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.962+0000 7f0b820b3640 1 -- 172.21.15.38:0/1590466993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0b63f0 msgr2=0x7f0b7c0c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:35.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.962+0000 7f0b820b3640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0b63f0 0x7f0b7c0c8760 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c0b7740 tx=0x7f0b7400f1b0 comp rx=0 tx=0).stop 2024-04-25T08:40:35.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.962+0000 7f0b820b3640 1 -- 172.21.15.38:0/1590466993 shutdown_connections 2024-04-25T08:40:35.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.962+0000 7f0b820b3640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b7c0b63f0 0x7f0b7c0c8760 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:35.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.962+0000 7f0b820b3640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0b6806b500 0x7f0b6806d9c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:35.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.962+0000 7f0b820b3640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0b7c154b40 0x7f0b7c0b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:35.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.962+0000 7f0b820b3640 1 --2- 172.21.15.38:0/1590466993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0b640039b0 0x7f0b7c0ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:35.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.962+0000 7f0b820b3640 1 -- 172.21.15.38:0/1590466993 >> 172.21.15.38:0/1590466993 conn(0x7f0b7c0b3180 msgr2=0x7f0b7c0ce110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:35.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.963+0000 7f0b820b3640 1 -- 172.21.15.38:0/1590466993 shutdown_connections 2024-04-25T08:40:35.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:35.963+0000 7f0b820b3640 1 -- 172.21.15.38:0/1590466993 wait complete. 2024-04-25T08:40:35.966 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T08:40:35.966 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:40:36.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.073+0000 7f6dbac1f640 1 Processor -- start 2024-04-25T08:40:36.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.073+0000 7f6dbac1f640 1 -- start start 2024-04-25T08:40:36.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6dbac1f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40b5710 0x7f6db4161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6dbac1f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6db415ddf0 0x7f6db40b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6dbac1f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db4161e00 con 0x7f6db40b5af0 2024-04-25T08:40:36.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6dbac1f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db40b9930 con 0x7f6db415ddf0 2024-04-25T08:40:36.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6dbac1f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db41620e0 con 0x7f6db40b5710 2024-04-25T08:40:36.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6db3fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6db415ddf0 0x7f6db40b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6db3fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6db415ddf0 0x7f6db40b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52048/0 (socket says 172.21.15.38:52048) 2024-04-25T08:40:36.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6db3fff640 1 -- 172.21.15.38:0/3321950064 learned_addr learned my addr 172.21.15.38:0/3321950064 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:36.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6db89bb640 1 --2- 172.21.15.38:0/3321950064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40b5710 0x7f6db4161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6db3fff640 1 -- 172.21.15.38:0/3321950064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40b5710 msgr2=0x7f6db4161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:36.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6db3fff640 1 --2- 172.21.15.38:0/3321950064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40b5710 0x7f6db4161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6db3fff640 1 -- 172.21.15.38:0/3321950064 >> v1:172.21.15.107:6789/0 conn(0x7f6db40b5af0 legacy=0x7f6db406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.074+0000 7f6db3fff640 1 -- 172.21.15.38:0/3321950064 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db41623c0 con 0x7f6db415ddf0 2024-04-25T08:40:36.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.075+0000 7f6db3fff640 1 --2- 172.21.15.38:0/3321950064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db415ddf0 0x7f6db40b93f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009870 tx=0x7f6da002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b26d71e797c292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:36.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.075+0000 7f6db37fe640 1 -- 172.21.15.38:0/3321950064 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0037070 con 0x7f6db415ddf0 2024-04-25T08:40:36.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.075+0000 7f6db37fe640 1 -- 172.21.15.38:0/3321950064 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da0032440 con 0x7f6db415ddf0 2024-04-25T08:40:36.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.075+0000 7f6db37fe640 1 -- 172.21.15.38:0/3321950064 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da0032810 con 0x7f6db415ddf0 2024-04-25T08:40:36.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.076+0000 7f6dbac1f640 1 -- 172.21.15.38:0/3321950064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db415ddf0 msgr2=0x7f6db40b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.076+0000 7f6dbac1f640 1 --2- 172.21.15.38:0/3321950064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db415ddf0 0x7f6db40b93f0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009870 tx=0x7f6da002a270 comp rx=0 tx=0).stop 2024-04-25T08:40:36.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.076+0000 7f6dbac1f640 1 -- 172.21.15.38:0/3321950064 shutdown_connections 2024-04-25T08:40:36.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.076+0000 7f6dbac1f640 1 --2- 172.21.15.38:0/3321950064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db415ddf0 0x7f6db40b93f0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.076+0000 7f6dbac1f640 1 --2- 172.21.15.38:0/3321950064 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40b5710 0x7f6db4161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.076+0000 7f6dbac1f640 1 -- 172.21.15.38:0/3321950064 >> 172.21.15.38:0/3321950064 conn(0x7f6db40d07c0 msgr2=0x7f6db40d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:36.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.076+0000 7f6dbac1f640 1 -- 172.21.15.38:0/3321950064 shutdown_connections 2024-04-25T08:40:36.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.076+0000 7f6dbac1f640 1 -- 172.21.15.38:0/3321950064 wait complete. 2024-04-25T08:40:36.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.077+0000 7f6dbac1f640 1 Processor -- start 2024-04-25T08:40:36.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.077+0000 7f6dbac1f640 1 -- start start 2024-04-25T08:40:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6dbac1f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40b5710 0x7f6db415e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6dbac1f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6db40b5af0 0x7f6db414bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6dbac1f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db415ddf0 0x7f6db41f3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6dbac1f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40d34d0 con 0x7f6db415ddf0 2024-04-25T08:40:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6dbac1f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40d3350 con 0x7f6db40b5af0 2024-04-25T08:40:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6dbac1f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40d3650 con 0x7f6db40b5710 2024-04-25T08:40:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6db91bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db415ddf0 0x7f6db41f3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6db91bc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db415ddf0 0x7f6db41f3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52054/0 (socket says 172.21.15.38:52054) 2024-04-25T08:40:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6db91bc640 1 -- 172.21.15.38:0/2389875943 learned_addr learned my addr 172.21.15.38:0/2389875943 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6db3fff640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6db40b5af0 0x7f6db414bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6db89bb640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40b5710 0x7f6db415e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6db91bc640 1 -- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40b5710 msgr2=0x7f6db415e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6db91bc640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40b5710 0x7f6db415e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6db91bc640 1 -- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6db40b5af0 msgr2=0x7f6db414bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6db91bc640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6db40b5af0 0x7f6db414bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.078+0000 7f6db91bc640 1 -- 172.21.15.38:0/2389875943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db414c280 con 0x7f6db415ddf0 2024-04-25T08:40:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.079+0000 7f6db91bc640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db415ddf0 0x7f6db41f3b50 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c7a0 tx=0x7f6dac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.079+0000 7f6db17fa640 1 -- 172.21.15.38:0/2389875943 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac003b70 con 0x7f6db415ddf0 2024-04-25T08:40:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.079+0000 7f6db17fa640 1 -- 172.21.15.38:0/2389875943 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac003d30 con 0x7f6db415ddf0 2024-04-25T08:40:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.079+0000 7f6dbac1f640 1 -- 172.21.15.38:0/2389875943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db41f4530 con 0x7f6db415ddf0 2024-04-25T08:40:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.079+0000 7f6db17fa640 1 -- 172.21.15.38:0/2389875943 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac00aa90 con 0x7f6db415ddf0 2024-04-25T08:40:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.079+0000 7f6dbac1f640 1 -- 172.21.15.38:0/2389875943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db41f4af0 con 0x7f6db415ddf0 2024-04-25T08:40:36.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.080+0000 7f6dbac1f640 1 -- 172.21.15.38:0/2389875943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db4068e80 con 0x7f6db415ddf0 2024-04-25T08:40:36.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.080+0000 7f6db17fa640 1 -- 172.21.15.38:0/2389875943 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6dac00ac30 con 0x7f6db415ddf0 2024-04-25T08:40:36.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.081+0000 7f6db17fa640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d9806b4d0 0x7f6d9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.081+0000 7f6db17fa640 1 -- 172.21.15.38:0/2389875943 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6dac08a2d0 con 0x7f6db415ddf0 2024-04-25T08:40:36.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.081+0000 7f6db89bb640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d9806b4d0 0x7f6d9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.081+0000 7f6db89bb640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d9806b4d0 0x7f6d9806d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6da8006fd0 tx=0x7f6da8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:36.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.085+0000 7f6db17fa640 1 -- 172.21.15.38:0/2389875943 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6db414c280 con 0x7f6db415ddf0 2024-04-25T08:40:36.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.334+0000 7f6dbac1f640 1 -- 172.21.15.38:0/2389875943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6db40cdb20 con 0x7f6db415ddf0 2024-04-25T08:40:36.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.334+0000 7f6db17fa640 1 -- 172.21.15.38:0/2389875943 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1572 (secure 0 0 0) 0x7f6dac058fd0 con 0x7f6db415ddf0 2024-04-25T08:40:36.327 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:36.327 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":565,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078423686,"bytes_used":2675892224,"bytes_avail":770418221056,"bytes_total":773094113280,"write_bytes_sec":157739,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":70,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":7639}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T08:40:35.331262+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:40:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.336+0000 7f6dbac1f640 1 -- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d9806b4d0 msgr2=0x7f6d9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.336+0000 7f6dbac1f640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d9806b4d0 0x7f6d9806d990 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6da8006fd0 tx=0x7f6da8008040 comp rx=0 tx=0).stop 2024-04-25T08:40:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.336+0000 7f6dbac1f640 1 -- 172.21.15.38:0/2389875943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db415ddf0 msgr2=0x7f6db41f3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.336+0000 7f6dbac1f640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db415ddf0 0x7f6db41f3b50 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c7a0 tx=0x7f6dac00cc60 comp rx=0 tx=0).stop 2024-04-25T08:40:36.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.336+0000 7f6dbac1f640 1 -- 172.21.15.38:0/2389875943 shutdown_connections 2024-04-25T08:40:36.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.336+0000 7f6dbac1f640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6db415ddf0 0x7f6db41f3b50 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.336+0000 7f6dbac1f640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6db40b5af0 0x7f6db414bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.336+0000 7f6dbac1f640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6d9806b4d0 0x7f6d9806d990 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.336+0000 7f6dbac1f640 1 --2- 172.21.15.38:0/2389875943 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6db40b5710 0x7f6db415e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.336+0000 7f6dbac1f640 1 -- 172.21.15.38:0/2389875943 >> 172.21.15.38:0/2389875943 conn(0x7f6db40d07c0 msgr2=0x7f6db40d2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:36.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.336+0000 7f6dbac1f640 1 -- 172.21.15.38:0/2389875943 shutdown_connections 2024-04-25T08:40:36.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.337+0000 7f6dbac1f640 1 -- 172.21.15.38:0/2389875943 wait complete. 2024-04-25T08:40:36.340 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:40:36.340 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:40:36.340 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:40:36.380 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:40:36.433 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:40:36.433 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:40:36.433 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:40:36.449 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:40:36.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.597+0000 7f7906f8b640 1 Processor -- start 2024-04-25T08:40:36.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.597+0000 7f7906f8b640 1 -- start start 2024-04-25T08:40:36.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f7906f8b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f790006c110 0x7f790006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f7906f8b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f79000ce2e0 0x7f79000d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f7906f8b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790006ca30 con 0x7f790015df50 2024-04-25T08:40:36.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f7906f8b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79000cf2d0 con 0x7f79000ce2e0 2024-04-25T08:40:36.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f7906f8b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79000ceff0 con 0x7f790006c110 2024-04-25T08:40:36.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f7905528640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f79000ce2e0 0x7f79000d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f78fffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f790015df50 0x7f79000ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:48024/0 (socket says 172.21.15.38:48024) 2024-04-25T08:40:36.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f78fffff640 1 -- 172.21.15.38:0/547047462 learned_addr learned my addr 172.21.15.38:0/547047462 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:36.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f7904d27640 1 --2- 172.21.15.38:0/547047462 >> v2:172.21.15.107:3300/0 conn(0x7f790006c110 0x7f790006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f7905528640 1 -- 172.21.15.38:0/547047462 >> v2:172.21.15.107:3300/0 conn(0x7f790006c110 msgr2=0x7f790006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:36.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f7905528640 1 --2- 172.21.15.38:0/547047462 >> v2:172.21.15.107:3300/0 conn(0x7f790006c110 0x7f790006c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f7905528640 1 -- 172.21.15.38:0/547047462 >> v1:172.21.15.38:6789/0 conn(0x7f790015df50 legacy=0x7f79000ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f78ff7fe640 1 -- 172.21.15.38:0/547047462 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683228886 0 0) 0x7f790006ca30 con 0x7f790015df50 2024-04-25T08:40:36.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f7905528640 1 -- 172.21.15.38:0/547047462 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79000d1f70 con 0x7f79000ce2e0 2024-04-25T08:40:36.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.598+0000 7f7904d27640 1 --2- 172.21.15.38:0/547047462 >> v2:172.21.15.107:3300/0 conn(0x7f790006c110 0x7f790006c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:36.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.599+0000 7f7905528640 1 --2- 172.21.15.38:0/547047462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79000ce2e0 0x7f79000d2a50 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f790008c550 tx=0x7f78f802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85e484bca433866 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:36.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.599+0000 7f78ff7fe640 1 -- 172.21.15.38:0/547047462 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8007740 con 0x7f79000ce2e0 2024-04-25T08:40:36.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.599+0000 7f78ff7fe640 1 -- 172.21.15.38:0/547047462 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f80078e0 con 0x7f79000ce2e0 2024-04-25T08:40:36.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.599+0000 7f78ff7fe640 1 -- 172.21.15.38:0/547047462 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8007cb0 con 0x7f79000ce2e0 2024-04-25T08:40:36.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.599+0000 7f7906f8b640 1 -- 172.21.15.38:0/547047462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79000ce2e0 msgr2=0x7f79000d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.599+0000 7f7906f8b640 1 --2- 172.21.15.38:0/547047462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79000ce2e0 0x7f79000d2a50 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f790008c550 tx=0x7f78f802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.600+0000 7f7906f8b640 1 -- 172.21.15.38:0/547047462 shutdown_connections 2024-04-25T08:40:36.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.600+0000 7f7906f8b640 1 --2- 172.21.15.38:0/547047462 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79000ce2e0 0x7f79000d2a50 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.600+0000 7f7906f8b640 1 --2- 172.21.15.38:0/547047462 >> v2:172.21.15.107:3300/0 conn(0x7f790006c110 0x7f790006c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.600+0000 7f7906f8b640 1 -- 172.21.15.38:0/547047462 >> 172.21.15.38:0/547047462 conn(0x7f79000d3fb0 msgr2=0x7f79000d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.600+0000 7f7906f8b640 1 -- 172.21.15.38:0/547047462 shutdown_connections 2024-04-25T08:40:36.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.600+0000 7f7906f8b640 1 -- 172.21.15.38:0/547047462 wait complete. 2024-04-25T08:40:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.601+0000 7f7906f8b640 1 Processor -- start 2024-04-25T08:40:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.601+0000 7f7906f8b640 1 -- start start 2024-04-25T08:40:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7906f8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790006c110 0x7f79000c7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7906f8b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79000ce2e0 0x7f790014ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7906f8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f790015df50 0x7f79000c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7906f8b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79000d2f90 con 0x7f790015df50 2024-04-25T08:40:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7906f8b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79000d2530 con 0x7f790006c110 2024-04-25T08:40:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7906f8b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79000d3110 con 0x7f79000ce2e0 2024-04-25T08:40:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7905528640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f790015df50 0x7f79000c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7905528640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f790015df50 0x7f79000c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52078/0 (socket says 172.21.15.38:52078) 2024-04-25T08:40:36.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7905528640 1 -- 172.21.15.38:0/2677597007 learned_addr learned my addr 172.21.15.38:0/2677597007 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7904d27640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790006c110 0x7f79000c7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f78fffff640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79000ce2e0 0x7f790014ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7905528640 1 -- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79000ce2e0 msgr2=0x7f790014ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7905528640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79000ce2e0 0x7f790014ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7905528640 1 -- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790006c110 msgr2=0x7f79000c7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7905528640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790006c110 0x7f79000c7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7905528640 1 -- 172.21.15.38:0/2677597007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79000c83d0 con 0x7f790015df50 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f78fffff640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79000ce2e0 0x7f790014ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7904d27640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790006c110 0x7f79000c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7905528640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f790015df50 0x7f79000c7530 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009e80 tx=0x7f78f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f78fd7fa640 1 -- 172.21.15.38:0/2677597007 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f803f070 con 0x7f790015df50 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f78fd7fa640 1 -- 172.21.15.38:0/2677597007 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f8031c40 con 0x7f790015df50 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7906f8b640 1 -- 172.21.15.38:0/2677597007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790014ea40 con 0x7f790015df50 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f78fd7fa640 1 -- 172.21.15.38:0/2677597007 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8003990 con 0x7f790015df50 2024-04-25T08:40:36.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.602+0000 7f7906f8b640 1 -- 172.21.15.38:0/2677597007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790014f520 con 0x7f790015df50 2024-04-25T08:40:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.604+0000 7f7906f8b640 1 -- 172.21.15.38:0/2677597007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7900068e80 con 0x7f790015df50 2024-04-25T08:40:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.604+0000 7f78fd7fa640 1 -- 172.21.15.38:0/2677597007 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78f8003b30 con 0x7f790015df50 2024-04-25T08:40:36.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.604+0000 7f78fd7fa640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f78e406b4d0 0x7f78e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.604+0000 7f78fd7fa640 1 -- 172.21.15.38:0/2677597007 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f78f80ab580 con 0x7f790015df50 2024-04-25T08:40:36.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.605+0000 7f7904d27640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f78e406b4d0 0x7f78e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.605+0000 7f7904d27640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f78e406b4d0 0x7f78e406d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f78f400a990 tx=0x7f78f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:36.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.609+0000 7f78fd7fa640 1 -- 172.21.15.38:0/2677597007 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78f807a300 con 0x7f790015df50 2024-04-25T08:40:36.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.788+0000 7f7906f8b640 1 -- 172.21.15.38:0/2677597007 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79000ceb00 con 0x7f790015df50 2024-04-25T08:40:36.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.789+0000 7f78fd7fa640 1 -- 172.21.15.38:0/2677597007 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f78f8079c00 con 0x7f790015df50 2024-04-25T08:40:36.782 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:36.783 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":121,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:59.121832+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:55.759766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:56.451485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:40:36.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.791+0000 7f7906f8b640 1 -- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f78e406b4d0 msgr2=0x7f78e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.791+0000 7f7906f8b640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f78e406b4d0 0x7f78e406d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f78f400a990 tx=0x7f78f4008040 comp rx=0 tx=0).stop 2024-04-25T08:40:36.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.791+0000 7f7906f8b640 1 -- 172.21.15.38:0/2677597007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f790015df50 msgr2=0x7f79000c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.791+0000 7f7906f8b640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f790015df50 0x7f79000c7530 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f78f8009e80 tx=0x7f78f8004a40 comp rx=0 tx=0).stop 2024-04-25T08:40:36.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.792+0000 7f7906f8b640 1 -- 172.21.15.38:0/2677597007 shutdown_connections 2024-04-25T08:40:36.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.792+0000 7f7906f8b640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f790015df50 0x7f79000c7530 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.792+0000 7f7906f8b640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f78e406b4d0 0x7f78e406d990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.792+0000 7f7906f8b640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f79000ce2e0 0x7f790014ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.792+0000 7f7906f8b640 1 --2- 172.21.15.38:0/2677597007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f790006c110 0x7f79000c7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.792+0000 7f7906f8b640 1 -- 172.21.15.38:0/2677597007 >> 172.21.15.38:0/2677597007 conn(0x7f79000d3fb0 msgr2=0x7f79000b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:36.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.792+0000 7f7906f8b640 1 -- 172.21.15.38:0/2677597007 shutdown_connections 2024-04-25T08:40:36.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.792+0000 7f7906f8b640 1 -- 172.21.15.38:0/2677597007 wait complete. 2024-04-25T08:40:36.795 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:39:55.759766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:39:56.451485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:40:36.795 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:40:36.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.906+0000 7f1f95a5d640 1 Processor -- start 2024-04-25T08:40:36.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.906+0000 7f1f95a5d640 1 -- start start 2024-04-25T08:40:36.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f95a5d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f1f900d0320 0x7f1f900d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f95a5d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f900d2060 con 0x7f1f900d0700 2024-04-25T08:40:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f95a5d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f900d2330 con 0x7f1f900b6980 2024-04-25T08:40:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f95a5d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900d2610 con 0x7f1f900d0320 2024-04-25T08:40:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f87fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f1f900d0700 0x7f1f900d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:48028/0 (socket says 172.21.15.38:48028) 2024-04-25T08:40:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f87fff640 1 -- 172.21.15.38:0/3337362681 learned_addr learned my addr 172.21.15.38:0/3337362681 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f8effd640 1 --2- 172.21.15.38:0/3337362681 >> v2:172.21.15.107:3300/0 conn(0x7f1f900d0320 0x7f1f900d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198663610 0 0) 0x7f1f900d2060 con 0x7f1f900d0700 2024-04-25T08:40:36.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f7c003520 con 0x7f1f900d0700 2024-04-25T08:40:36.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2896347227 0 0) 0x7f1f7c003520 con 0x7f1f900d0700 2024-04-25T08:40:36.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 >> v2:172.21.15.107:3300/0 conn(0x7f1f900d0320 msgr2=0x7f1f900d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:36.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f8e7fc640 1 --2- 172.21.15.38:0/3337362681 >> v2:172.21.15.107:3300/0 conn(0x7f1f900d0320 0x7f1f900d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 >> v1:172.21.15.107:6789/0 conn(0x7f1f900b6980 legacy=0x7f1f900b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f900d28f0 con 0x7f1f900d0700 2024-04-25T08:40:36.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.907+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f1f78001670 con 0x7f1f900d0700 2024-04-25T08:40:36.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 >> v1:172.21.15.38:6789/0 conn(0x7f1f900d0700 legacy=0x7f1f900d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8e7fc640 1 --2- 172.21.15.38:0/3337362681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7c0039b0 0x7f1f7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8e7fc640 1 --2- 172.21.15.38:0/3337362681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c004800 0x7f1f7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8e7fc640 1 --2- 172.21.15.38:0/3337362681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7c0051c0 0x7f1f7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900d2610 con 0x7f1f7c004800 2024-04-25T08:40:36.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c00c070 con 0x7f1f7c0039b0 2024-04-25T08:40:36.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c00c270 con 0x7f1f7c0051c0 2024-04-25T08:40:36.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f87fff640 1 --2- 172.21.15.38:0/3337362681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c004800 0x7f1f7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8effd640 1 --2- 172.21.15.38:0/3337362681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7c0039b0 0x7f1f7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8f7fe640 1 --2- 172.21.15.38:0/3337362681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7c0051c0 0x7f1f7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8effd640 1 -- 172.21.15.38:0/3337362681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7c0051c0 msgr2=0x7f1f7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:36.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8effd640 1 --2- 172.21.15.38:0/3337362681 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f7c0051c0 0x7f1f7c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8effd640 1 -- 172.21.15.38:0/3337362681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c004800 msgr2=0x7f1f7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:36.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8effd640 1 --2- 172.21.15.38:0/3337362681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c004800 0x7f1f7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8effd640 1 -- 172.21.15.38:0/3337362681 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c003520 con 0x7f1f7c0039b0 2024-04-25T08:40:36.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f87fff640 1 --2- 172.21.15.38:0/3337362681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c004800 0x7f1f7c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:36.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.908+0000 7f1f8f7fe640 1 -- 172.21.15.38:0/3337362681 reap_dead start 2024-04-25T08:40:36.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.909+0000 7f1f8effd640 1 --2- 172.21.15.38:0/3337362681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7c0039b0 0x7f1f7c003dd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009ac0 tx=0x7f1f7c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da98a74acdbf2d65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:36.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.909+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80036070 con 0x7f1f7c0039b0 2024-04-25T08:40:36.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.909+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80031400 con 0x7f1f7c0039b0 2024-04-25T08:40:36.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.909+0000 7f1f8e7fc640 1 -- 172.21.15.38:0/3337362681 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80009690 con 0x7f1f7c0039b0 2024-04-25T08:40:36.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.910+0000 7f1f95a5d640 1 -- 172.21.15.38:0/3337362681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7c0039b0 msgr2=0x7f1f7c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.910+0000 7f1f95a5d640 1 --2- 172.21.15.38:0/3337362681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7c0039b0 0x7f1f7c003dd0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1f80009ac0 tx=0x7f1f7c005830 comp rx=0 tx=0).stop 2024-04-25T08:40:36.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.910+0000 7f1f95a5d640 1 -- 172.21.15.38:0/3337362681 shutdown_connections 2024-04-25T08:40:36.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.910+0000 7f1f95a5d640 1 --2- 172.21.15.38:0/3337362681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f7c0039b0 0x7f1f7c003dd0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.910+0000 7f1f95a5d640 1 -- 172.21.15.38:0/3337362681 >> 172.21.15.38:0/3337362681 conn(0x7f1f900668f0 msgr2=0x7f1f90066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:36.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.910+0000 7f1f95a5d640 1 -- 172.21.15.38:0/3337362681 shutdown_connections 2024-04-25T08:40:36.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.910+0000 7f1f95a5d640 1 -- 172.21.15.38:0/3337362681 wait complete. 2024-04-25T08:40:36.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.911+0000 7f1f95a5d640 1 Processor -- start 2024-04-25T08:40:36.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.911+0000 7f1f95a5d640 1 -- start start 2024-04-25T08:40:36.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f95a5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0039b0 0x7f1f9014c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f95a5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f9014f800 0x7f1f9015f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f95a5d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f9015fd60 0x7f1f901f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f95a5d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900d0b40 con 0x7f1f7c0039b0 2024-04-25T08:40:36.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f95a5d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900d2eb0 con 0x7f1f9014f800 2024-04-25T08:40:36.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f95a5d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9014cae0 con 0x7f1f9015fd60 2024-04-25T08:40:36.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0039b0 0x7f1f9014c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0039b0 0x7f1f9014c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52096/0 (socket says 172.21.15.38:52096) 2024-04-25T08:40:36.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8effd640 1 -- 172.21.15.38:0/2909968428 learned_addr learned my addr 172.21.15.38:0/2909968428 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:36.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f87fff640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f9014f800 0x7f1f9015f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8f7fe640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f9015fd60 0x7f1f901f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8effd640 1 -- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f9015fd60 msgr2=0x7f1f901f6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8effd640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f9015fd60 0x7f1f901f6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8effd640 1 -- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f9014f800 msgr2=0x7f1f9015f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8effd640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f9014f800 0x7f1f9015f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8effd640 1 -- 172.21.15.38:0/2909968428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f901e64f0 con 0x7f1f7c0039b0 2024-04-25T08:40:36.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8f7fe640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f9015fd60 0x7f1f901f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8effd640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0039b0 0x7f1f9014c5a0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f1f8002abb0 tx=0x7f1f80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8cff9640 1 -- 172.21.15.38:0/2909968428 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8003f020 con 0x7f1f7c0039b0 2024-04-25T08:40:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8cff9640 1 -- 172.21.15.38:0/2909968428 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80037040 con 0x7f1f7c0039b0 2024-04-25T08:40:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f8cff9640 1 -- 172.21.15.38:0/2909968428 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f80036070 con 0x7f1f7c0039b0 2024-04-25T08:40:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.912+0000 7f1f95a5d640 1 -- 172.21.15.38:0/2909968428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f901e67d0 con 0x7f1f7c0039b0 2024-04-25T08:40:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.913+0000 7f1f95a5d640 1 -- 172.21.15.38:0/2909968428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f901e6d90 con 0x7f1f7c0039b0 2024-04-25T08:40:36.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.914+0000 7f1f95a5d640 1 -- 172.21.15.38:0/2909968428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f90068e80 con 0x7f1f7c0039b0 2024-04-25T08:40:36.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.914+0000 7f1f8cff9640 1 -- 172.21.15.38:0/2909968428 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f80042020 con 0x7f1f7c0039b0 2024-04-25T08:40:36.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.915+0000 7f1f8cff9640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f7006b4d0 0x7f1f7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:36.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.915+0000 7f1f8cff9640 1 -- 172.21.15.38:0/2909968428 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1f800aa000 con 0x7f1f7c0039b0 2024-04-25T08:40:36.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.915+0000 7f1f87fff640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f7006b4d0 0x7f1f7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:36.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.915+0000 7f1f87fff640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f7006b4d0 0x7f1f7006d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1f78007ce0 tx=0x7f1f78007bf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:36.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:36.919+0000 7f1f8cff9640 1 -- 172.21.15.38:0/2909968428 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f80049070 con 0x7f1f7c0039b0 2024-04-25T08:40:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.098+0000 7f1f95a5d640 1 -- 172.21.15.38:0/2909968428 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f900b41d0 con 0x7f1f7c0039b0 2024-04-25T08:40:37.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.099+0000 7f1f8cff9640 1 -- 172.21.15.38:0/2909968428 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f80078cf0 con 0x7f1f7c0039b0 2024-04-25T08:40:37.092 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:40:37.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.101+0000 7f1f95a5d640 1 -- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f7006b4d0 msgr2=0x7f1f7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.101+0000 7f1f95a5d640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f7006b4d0 0x7f1f7006d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f1f78007ce0 tx=0x7f1f78007bf0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.101+0000 7f1f95a5d640 1 -- 172.21.15.38:0/2909968428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0039b0 msgr2=0x7f1f9014c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.101+0000 7f1f95a5d640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0039b0 0x7f1f9014c5a0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f1f8002abb0 tx=0x7f1f80004730 comp rx=0 tx=0).stop 2024-04-25T08:40:37.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.101+0000 7f1f95a5d640 1 -- 172.21.15.38:0/2909968428 shutdown_connections 2024-04-25T08:40:37.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.101+0000 7f1f95a5d640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f1f7006b4d0 0x7f1f7006d990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.101+0000 7f1f95a5d640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f1f9015fd60 0x7f1f901f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.101+0000 7f1f95a5d640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1f9014f800 0x7f1f9015f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.101+0000 7f1f95a5d640 1 --2- 172.21.15.38:0/2909968428 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1f7c0039b0 0x7f1f9014c5a0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.101+0000 7f1f95a5d640 1 -- 172.21.15.38:0/2909968428 >> 172.21.15.38:0/2909968428 conn(0x7f1f900668f0 msgr2=0x7f1f900ce580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:37.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.101+0000 7f1f95a5d640 1 -- 172.21.15.38:0/2909968428 shutdown_connections 2024-04-25T08:40:37.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.101+0000 7f1f95a5d640 1 -- 172.21.15.38:0/2909968428 wait complete. 2024-04-25T08:40:37.105 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:40:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.216+0000 7fe8ab961640 1 Processor -- start 2024-04-25T08:40:37.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.216+0000 7fe8ab961640 1 -- start start 2024-04-25T08:40:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8ab961640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe8a40d0320 0x7fe8a40d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8ab961640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe8a40b6980 0x7fe8a40b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8ab961640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a40d2060 con 0x7fe8a40d0700 2024-04-25T08:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8ab961640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a40d2330 con 0x7fe8a40b6980 2024-04-25T08:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8ab961640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a40d2610 con 0x7fe8a40d0320 2024-04-25T08:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8a9efe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe8a40b6980 0x7fe8a40b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8a9efe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe8a40b6980 0x7fe8a40b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52108/0 (socket says 172.21.15.38:52108) 2024-04-25T08:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8a9efe640 1 -- 172.21.15.38:0/522591158 learned_addr learned my addr 172.21.15.38:0/522591158 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8a96fd640 1 --2- 172.21.15.38:0/522591158 >> v2:172.21.15.107:3300/0 conn(0x7fe8a40d0320 0x7fe8a40d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8a9efe640 1 -- 172.21.15.38:0/522591158 >> v2:172.21.15.107:3300/0 conn(0x7fe8a40d0320 msgr2=0x7fe8a40d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8a9efe640 1 --2- 172.21.15.38:0/522591158 >> v2:172.21.15.107:3300/0 conn(0x7fe8a40d0320 0x7fe8a40d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8a9efe640 1 -- 172.21.15.38:0/522591158 >> v1:172.21.15.107:6789/0 conn(0x7fe8a40d0700 legacy=0x7fe8a40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:37.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.217+0000 7fe8a9efe640 1 -- 172.21.15.38:0/522591158 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40d28f0 con 0x7fe8a40b6980 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.218+0000 7fe8a9efe640 1 --2- 172.21.15.38:0/522591158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a40b6980 0x7fe8a40b6dd0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4069100 tx=0x7fe89c02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac04909c0f7b595 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.218+0000 7fe89bfff640 1 -- 172.21.15.38:0/522591158 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c036070 con 0x7fe8a40b6980 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.218+0000 7fe89bfff640 1 -- 172.21.15.38:0/522591158 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c031600 con 0x7fe8a40b6980 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.218+0000 7fe89bfff640 1 -- 172.21.15.38:0/522591158 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0319d0 con 0x7fe8a40b6980 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.219+0000 7fe8ab961640 1 -- 172.21.15.38:0/522591158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a40b6980 msgr2=0x7fe8a40b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.219+0000 7fe8ab961640 1 --2- 172.21.15.38:0/522591158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a40b6980 0x7fe8a40b6dd0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4069100 tx=0x7fe89c02a780 comp rx=0 tx=0).stop 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.219+0000 7fe8ab961640 1 -- 172.21.15.38:0/522591158 shutdown_connections 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.219+0000 7fe8ab961640 1 --2- 172.21.15.38:0/522591158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a40b6980 0x7fe8a40b6dd0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.219+0000 7fe8ab961640 1 --2- 172.21.15.38:0/522591158 >> v2:172.21.15.107:3300/0 conn(0x7fe8a40d0320 0x7fe8a40d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.219+0000 7fe8ab961640 1 -- 172.21.15.38:0/522591158 >> 172.21.15.38:0/522591158 conn(0x7fe8a40668f0 msgr2=0x7fe8a4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.219+0000 7fe8ab961640 1 -- 172.21.15.38:0/522591158 shutdown_connections 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.219+0000 7fe8ab961640 1 -- 172.21.15.38:0/522591158 wait complete. 2024-04-25T08:40:37.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.220+0000 7fe8ab961640 1 Processor -- start 2024-04-25T08:40:37.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.220+0000 7fe8ab961640 1 -- start start 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8ab961640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a40b6980 0x7fe8a4149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8ab961640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8a40d0320 0x7fe8a4145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8ab961640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a40d0700 0x7fe8a40b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8ab961640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40d0b40 con 0x7fe8a40b6980 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8ab961640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40d2eb0 con 0x7fe8a40d0700 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8ab961640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a40b59a0 con 0x7fe8a40d0320 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8a96fd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a40b6980 0x7fe8a4149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8a96fd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a40b6980 0x7fe8a4149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52114/0 (socket says 172.21.15.38:52114) 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8a96fd640 1 -- 172.21.15.38:0/1026823520 learned_addr learned my addr 172.21.15.38:0/1026823520 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8a8efc640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8a40d0320 0x7fe8a4145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8a9efe640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a40d0700 0x7fe8a40b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8a96fd640 1 -- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8a40d0320 msgr2=0x7fe8a4145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8a96fd640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8a40d0320 0x7fe8a4145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8a96fd640 1 -- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a40d0700 msgr2=0x7fe8a40b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8a96fd640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a40d0700 0x7fe8a40b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8a96fd640 1 -- 172.21.15.38:0/1026823520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40b4d90 con 0x7fe8a40b6980 2024-04-25T08:40:37.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.221+0000 7fe8a96fd640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a40b6980 0x7fe8a4149040 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000d800 tx=0x7fe8a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:37.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.222+0000 7fe899ffb640 1 -- 172.21.15.38:0/1026823520 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000b6c0 con 0x7fe8a40b6980 2024-04-25T08:40:37.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.222+0000 7fe899ffb640 1 -- 172.21.15.38:0/1026823520 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a0019070 con 0x7fe8a40b6980 2024-04-25T08:40:37.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.222+0000 7fe8ab961640 1 -- 172.21.15.38:0/1026823520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a40b5070 con 0x7fe8a40b6980 2024-04-25T08:40:37.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.222+0000 7fe899ffb640 1 -- 172.21.15.38:0/1026823520 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000ba80 con 0x7fe8a40b6980 2024-04-25T08:40:37.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.222+0000 7fe8ab961640 1 -- 172.21.15.38:0/1026823520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a40b5630 con 0x7fe8a40b6980 2024-04-25T08:40:37.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.223+0000 7fe8ab961640 1 -- 172.21.15.38:0/1026823520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a4161210 con 0x7fe8a40b6980 2024-04-25T08:40:37.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.223+0000 7fe899ffb640 1 -- 172.21.15.38:0/1026823520 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8a000bc20 con 0x7fe8a40b6980 2024-04-25T08:40:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.224+0000 7fe899ffb640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe88806b500 0x7fe88806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.224+0000 7fe899ffb640 1 -- 172.21.15.38:0/1026823520 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8a008a840 con 0x7fe8a40b6980 2024-04-25T08:40:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.224+0000 7fe8a8efc640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe88806b500 0x7fe88806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.224+0000 7fe8a8efc640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe88806b500 0x7fe88806d9c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe894006fd0 tx=0x7fe894008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:37.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.228+0000 7fe899ffb640 1 -- 172.21.15.38:0/1026823520 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8a4161210 con 0x7fe8a40b6980 2024-04-25T08:40:37.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.410+0000 7fe8ab961640 1 -- 172.21.15.38:0/1026823520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8a4068e80 con 0x7fe8a40b6980 2024-04-25T08:40:37.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.411+0000 7fe899ffb640 1 -- 172.21.15.38:0/1026823520 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe8a0059530 con 0x7fe8a40b6980 2024-04-25T08:40:37.404 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:40:37.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.413+0000 7fe8ab961640 1 -- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe88806b500 msgr2=0x7fe88806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.413+0000 7fe8ab961640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe88806b500 0x7fe88806d9c0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe894006fd0 tx=0x7fe894008040 comp rx=0 tx=0).stop 2024-04-25T08:40:37.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.413+0000 7fe8ab961640 1 -- 172.21.15.38:0/1026823520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a40b6980 msgr2=0x7fe8a4149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.413+0000 7fe8ab961640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a40b6980 0x7fe8a4149040 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000d800 tx=0x7fe8a000dcc0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.413+0000 7fe8ab961640 1 -- 172.21.15.38:0/1026823520 shutdown_connections 2024-04-25T08:40:37.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.413+0000 7fe8ab961640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8a40d0700 0x7fe8a40b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.413+0000 7fe8ab961640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe88806b500 0x7fe88806d9c0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.413+0000 7fe8ab961640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8a40d0320 0x7fe8a4145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.413+0000 7fe8ab961640 1 --2- 172.21.15.38:0/1026823520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8a40b6980 0x7fe8a4149040 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.413+0000 7fe8ab961640 1 -- 172.21.15.38:0/1026823520 >> 172.21.15.38:0/1026823520 conn(0x7fe8a40668f0 msgr2=0x7fe8a40ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:37.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.413+0000 7fe8ab961640 1 -- 172.21.15.38:0/1026823520 shutdown_connections 2024-04-25T08:40:37.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.413+0000 7fe8ab961640 1 -- 172.21.15.38:0/1026823520 wait complete. 2024-04-25T08:40:37.416 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:40:37.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.527+0000 7f3cdfd89640 1 Processor -- start 2024-04-25T08:40:37.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.528+0000 7f3cdfd89640 1 -- start start 2024-04-25T08:40:37.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.528+0000 7f3cdfd89640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3cd806b5c0 0x7f3cd80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.528+0000 7f3cdfd89640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd806bad0 con 0x7f3cd815df70 2024-04-25T08:40:37.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.529+0000 7f3cdfd89640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd80cc950 con 0x7f3cd815db90 2024-04-25T08:40:37.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.529+0000 7f3cdfd89640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd80cd280 con 0x7f3cd806b5c0 2024-04-25T08:40:37.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.529+0000 7f3cdd324640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f3cd806b5c0 0x7f3cd80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.529+0000 7f3cde326640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f3cd815df70 0x7f3cd80cab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:48042/0 (socket says 172.21.15.38:48042) 2024-04-25T08:40:37.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.529+0000 7f3cde326640 1 -- 172.21.15.38:0/2199974594 learned_addr learned my addr 172.21.15.38:0/2199974594 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:37.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.529+0000 7f3cdd324640 1 -- 172.21.15.38:0/2199974594 >> v1:172.21.15.107:6789/0 conn(0x7f3cd815db90 legacy=0x7f3cd80cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:37.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.529+0000 7f3cdd324640 1 -- 172.21.15.38:0/2199974594 >> v1:172.21.15.38:6789/0 conn(0x7f3cd815df70 legacy=0x7f3cd80cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.529+0000 7f3cdcb23640 1 -- 172.21.15.38:0/2199974594 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105825611 0 0) 0x7f3cd806bad0 con 0x7f3cd815df70 2024-04-25T08:40:37.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.529+0000 7f3cdd324640 1 -- 172.21.15.38:0/2199974594 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd80ccc30 con 0x7f3cd806b5c0 2024-04-25T08:40:37.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.530+0000 7f3cdd324640 1 --2- 172.21.15.38:0/2199974594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd806b5c0 0x7f3cd80cf7b0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40097a0 tx=0x7f3cd402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ade3d469dcf536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:37.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.530+0000 7f3cdcb23640 1 -- 172.21.15.38:0/2199974594 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4037070 con 0x7f3cd806b5c0 2024-04-25T08:40:37.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.530+0000 7f3cdcb23640 1 -- 172.21.15.38:0/2199974594 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd4032440 con 0x7f3cd806b5c0 2024-04-25T08:40:37.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.530+0000 7f3cdcb23640 1 -- 172.21.15.38:0/2199974594 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4032810 con 0x7f3cd806b5c0 2024-04-25T08:40:37.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.530+0000 7f3cdfd89640 1 -- 172.21.15.38:0/2199974594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd806b5c0 msgr2=0x7f3cd80cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.530+0000 7f3cdfd89640 1 --2- 172.21.15.38:0/2199974594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd806b5c0 0x7f3cd80cf7b0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40097a0 tx=0x7f3cd402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.530+0000 7f3cdfd89640 1 -- 172.21.15.38:0/2199974594 shutdown_connections 2024-04-25T08:40:37.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.530+0000 7f3cdfd89640 1 --2- 172.21.15.38:0/2199974594 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd806b5c0 0x7f3cd80cf7b0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.530+0000 7f3cdfd89640 1 -- 172.21.15.38:0/2199974594 >> 172.21.15.38:0/2199974594 conn(0x7f3cd80d0520 msgr2=0x7f3cd80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:37.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.530+0000 7f3cdfd89640 1 -- 172.21.15.38:0/2199974594 shutdown_connections 2024-04-25T08:40:37.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.530+0000 7f3cdfd89640 1 -- 172.21.15.38:0/2199974594 wait complete. 2024-04-25T08:40:37.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.532+0000 7f3cdfd89640 1 Processor -- start 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.532+0000 7f3cdfd89640 1 -- start start 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.532+0000 7f3cdfd89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd806b5c0 0x7f3cd81f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.532+0000 7f3cdfd89640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cd815db90 0x7f3cd815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.532+0000 7f3cdfd89640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cd815df70 0x7f3cd809a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.532+0000 7f3cdfd89640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd80af620 con 0x7f3cd806b5c0 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.532+0000 7f3cdfd89640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd80af4a0 con 0x7f3cd815df70 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.532+0000 7f3cdfd89640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd80af8d0 con 0x7f3cd815db90 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.532+0000 7f3cddb25640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd806b5c0 0x7f3cd81f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cddb25640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd806b5c0 0x7f3cd81f7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52130/0 (socket says 172.21.15.38:52130) 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cddb25640 1 -- 172.21.15.38:0/3502589124 learned_addr learned my addr 172.21.15.38:0/3502589124 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cddb25640 1 -- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cd815db90 msgr2=0x7f3cd815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cde326640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cd815df70 0x7f3cd809a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cdd324640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cd815db90 0x7f3cd815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cddb25640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cd815db90 0x7f3cd815ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cddb25640 1 -- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cd815df70 msgr2=0x7f3cd809a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cddb25640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cd815df70 0x7f3cd809a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cddb25640 1 -- 172.21.15.38:0/3502589124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd809a920 con 0x7f3cd806b5c0 2024-04-25T08:40:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cddb25640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd806b5c0 0x7f3cd81f7c70 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00c450 tx=0x7f3ccc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cca7fc640 1 -- 172.21.15.38:0/3502589124 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc0052d0 con 0x7f3cd806b5c0 2024-04-25T08:40:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cca7fc640 1 -- 172.21.15.38:0/3502589124 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc005470 con 0x7f3cd806b5c0 2024-04-25T08:40:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cdfd89640 1 -- 172.21.15.38:0/3502589124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd80b4e70 con 0x7f3cd806b5c0 2024-04-25T08:40:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cca7fc640 1 -- 172.21.15.38:0/3502589124 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc012870 con 0x7f3cd806b5c0 2024-04-25T08:40:37.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.533+0000 7f3cdfd89640 1 -- 172.21.15.38:0/3502589124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd809d5e0 con 0x7f3cd806b5c0 2024-04-25T08:40:37.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.534+0000 7f3cdfd89640 1 -- 172.21.15.38:0/3502589124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd8068e80 con 0x7f3cd806b5c0 2024-04-25T08:40:37.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.535+0000 7f3cca7fc640 1 -- 172.21.15.38:0/3502589124 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ccc012a10 con 0x7f3cd806b5c0 2024-04-25T08:40:37.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.535+0000 7f3cca7fc640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3cbc06b500 0x7f3cbc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.536+0000 7f3cca7fc640 1 -- 172.21.15.38:0/3502589124 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ccc089ff0 con 0x7f3cd806b5c0 2024-04-25T08:40:37.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.536+0000 7f3cdd324640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3cbc06b500 0x7f3cbc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.536+0000 7f3cdd324640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3cbc06b500 0x7f3cbc06d9c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4034000 tx=0x7f3cd402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:37.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.540+0000 7f3cca7fc640 1 -- 172.21.15.38:0/3502589124 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3cd809a920 con 0x7f3cd806b5c0 2024-04-25T08:40:37.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.717+0000 7f3cdfd89640 1 -- 172.21.15.38:0/3502589124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3cd806bad0 con 0x7f3cd806b5c0 2024-04-25T08:40:37.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.718+0000 7f3cca7fc640 1 -- 172.21.15.38:0/3502589124 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ccc058c60 con 0x7f3cd806b5c0 2024-04-25T08:40:37.711 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:40:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.720+0000 7f3cdfd89640 1 -- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3cbc06b500 msgr2=0x7f3cbc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.720+0000 7f3cdfd89640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3cbc06b500 0x7f3cbc06d9c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4034000 tx=0x7f3cd402e000 comp rx=0 tx=0).stop 2024-04-25T08:40:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.720+0000 7f3cdfd89640 1 -- 172.21.15.38:0/3502589124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd806b5c0 msgr2=0x7f3cd81f7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.720+0000 7f3cdfd89640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd806b5c0 0x7f3cd81f7c70 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00c450 tx=0x7f3ccc00c910 comp rx=0 tx=0).stop 2024-04-25T08:40:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.720+0000 7f3cdfd89640 1 -- 172.21.15.38:0/3502589124 shutdown_connections 2024-04-25T08:40:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.720+0000 7f3cdfd89640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3cd815df70 0x7f3cd809a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.720+0000 7f3cdfd89640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f3cbc06b500 0x7f3cbc06d9c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.720+0000 7f3cdfd89640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3cd815db90 0x7f3cd815ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.720+0000 7f3cdfd89640 1 --2- 172.21.15.38:0/3502589124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3cd806b5c0 0x7f3cd81f7c70 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.720+0000 7f3cdfd89640 1 -- 172.21.15.38:0/3502589124 >> 172.21.15.38:0/3502589124 conn(0x7f3cd80d0520 msgr2=0x7f3cd80ae6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:37.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.720+0000 7f3cdfd89640 1 -- 172.21.15.38:0/3502589124 shutdown_connections 2024-04-25T08:40:37.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.720+0000 7f3cdfd89640 1 -- 172.21.15.38:0/3502589124 wait complete. 2024-04-25T08:40:37.723 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:40:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.832+0000 7f9b972ee640 1 Processor -- start 2024-04-25T08:40:37.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.832+0000 7f9b972ee640 1 -- start start 2024-04-25T08:40:37.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b972ee640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9b9006b2f0 0x7f9b900cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b972ee640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b900cabe0 con 0x7f9b9015dd90 2024-04-25T08:40:37.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b972ee640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b900caec0 con 0x7f9b900af920 2024-04-25T08:40:37.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b972ee640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b900cb1a0 con 0x7f9b9006b2f0 2024-04-25T08:40:37.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b9508a640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9b9015dd90 0x7f9b9006abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:48044/0 (socket says 172.21.15.38:48044) 2024-04-25T08:40:37.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b9508a640 1 -- 172.21.15.38:0/3844223209 learned_addr learned my addr 172.21.15.38:0/3844223209 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:37.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b94889640 1 --2- 172.21.15.38:0/3844223209 >> v2:172.21.15.107:3300/0 conn(0x7f9b9006b2f0 0x7f9b900cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b83fff640 1 -- 172.21.15.38:0/3844223209 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268313119 0 0) 0x7f9b900cabe0 con 0x7f9b9015dd90 2024-04-25T08:40:37.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b83fff640 1 -- 172.21.15.38:0/3844223209 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b7c003520 con 0x7f9b9015dd90 2024-04-25T08:40:37.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b83fff640 1 -- 172.21.15.38:0/3844223209 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153377340 0 0) 0x7f9b900caec0 con 0x7f9b900af920 2024-04-25T08:40:37.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b83fff640 1 -- 172.21.15.38:0/3844223209 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b900cabe0 con 0x7f9b900af920 2024-04-25T08:40:37.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b94889640 1 -- 172.21.15.38:0/3844223209 >> v1:172.21.15.107:6789/0 conn(0x7f9b900af920 legacy=0x7f9b900afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b94889640 1 -- 172.21.15.38:0/3844223209 >> v1:172.21.15.38:6789/0 conn(0x7f9b9015dd90 legacy=0x7f9b9006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.833+0000 7f9b94889640 1 -- 172.21.15.38:0/3844223209 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b900cb480 con 0x7f9b9006b2f0 2024-04-25T08:40:37.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.834+0000 7f9b94889640 1 --2- 172.21.15.38:0/3844223209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b9006b2f0 0x7f9b900cbf60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9b840097a0 tx=0x7f9b8402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1b193639bc43ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:37.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.834+0000 7f9b83fff640 1 -- 172.21.15.38:0/3844223209 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84037070 con 0x7f9b9006b2f0 2024-04-25T08:40:37.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.834+0000 7f9b83fff640 1 -- 172.21.15.38:0/3844223209 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b84032440 con 0x7f9b9006b2f0 2024-04-25T08:40:37.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.834+0000 7f9b83fff640 1 -- 172.21.15.38:0/3844223209 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b840327f0 con 0x7f9b9006b2f0 2024-04-25T08:40:37.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.835+0000 7f9b972ee640 1 -- 172.21.15.38:0/3844223209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b9006b2f0 msgr2=0x7f9b900cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.835+0000 7f9b972ee640 1 --2- 172.21.15.38:0/3844223209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b9006b2f0 0x7f9b900cbf60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9b840097a0 tx=0x7f9b8402a230 comp rx=0 tx=0).stop 2024-04-25T08:40:37.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.835+0000 7f9b972ee640 1 -- 172.21.15.38:0/3844223209 shutdown_connections 2024-04-25T08:40:37.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.835+0000 7f9b972ee640 1 --2- 172.21.15.38:0/3844223209 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b9006b2f0 0x7f9b900cbf60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.835+0000 7f9b972ee640 1 -- 172.21.15.38:0/3844223209 >> 172.21.15.38:0/3844223209 conn(0x7f9b90099a70 msgr2=0x7f9b900b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:37.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.835+0000 7f9b972ee640 1 -- 172.21.15.38:0/3844223209 shutdown_connections 2024-04-25T08:40:37.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.835+0000 7f9b972ee640 1 -- 172.21.15.38:0/3844223209 wait complete. 2024-04-25T08:40:37.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.836+0000 7f9b972ee640 1 Processor -- start 2024-04-25T08:40:37.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.836+0000 7f9b972ee640 1 -- start start 2024-04-25T08:40:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b972ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b9006b2f0 0x7f9b9015f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b972ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b900af920 0x7f9b90155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b972ee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b9015dd90 0x7f9b900b4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b972ee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b900cc790 con 0x7f9b900af920 2024-04-25T08:40:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b972ee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b900cba40 con 0x7f9b9006b2f0 2024-04-25T08:40:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b972ee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b900cc910 con 0x7f9b9015dd90 2024-04-25T08:40:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b94889640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b900af920 0x7f9b90155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b94889640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b900af920 0x7f9b90155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52144/0 (socket says 172.21.15.38:52144) 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b94889640 1 -- 172.21.15.38:0/2869863658 learned_addr learned my addr 172.21.15.38:0/2869863658 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b9508a640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b9006b2f0 0x7f9b9015f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b9588b640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b9015dd90 0x7f9b900b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b94889640 1 -- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b9015dd90 msgr2=0x7f9b900b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b94889640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b9015dd90 0x7f9b900b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b94889640 1 -- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b9006b2f0 msgr2=0x7f9b9015f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b94889640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b9006b2f0 0x7f9b9015f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.837+0000 7f9b94889640 1 -- 172.21.15.38:0/2869863658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b900b4a70 con 0x7f9b900af920 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.838+0000 7f9b94889640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b900af920 0x7f9b90155af0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f9b8402a270 tx=0x7f9b840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.838+0000 7f9b81ffb640 1 -- 172.21.15.38:0/2869863658 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84041030 con 0x7f9b900af920 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.838+0000 7f9b81ffb640 1 -- 172.21.15.38:0/2869863658 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8403d4e0 con 0x7f9b900af920 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.838+0000 7f9b81ffb640 1 -- 172.21.15.38:0/2869863658 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b84037040 con 0x7f9b900af920 2024-04-25T08:40:37.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.838+0000 7f9b972ee640 1 -- 172.21.15.38:0/2869863658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b900b4d10 con 0x7f9b900af920 2024-04-25T08:40:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.838+0000 7f9b972ee640 1 -- 172.21.15.38:0/2869863658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b900b52d0 con 0x7f9b900af920 2024-04-25T08:40:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.839+0000 7f9b972ee640 1 -- 172.21.15.38:0/2869863658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9004ef90 con 0x7f9b900af920 2024-04-25T08:40:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.839+0000 7f9b81ffb640 1 -- 172.21.15.38:0/2869863658 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b84044070 con 0x7f9b900af920 2024-04-25T08:40:37.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.840+0000 7f9b81ffb640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9b7406b4d0 0x7f9b7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:37.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.840+0000 7f9b81ffb640 1 -- 172.21.15.38:0/2869863658 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b840aaa60 con 0x7f9b900af920 2024-04-25T08:40:37.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.840+0000 7f9b9508a640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9b7406b4d0 0x7f9b7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:37.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.841+0000 7f9b9508a640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9b7406b4d0 0x7f9b7406d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c000f60 tx=0x7f9b8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:37.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:37.844+0000 7f9b81ffb640 1 -- 172.21.15.38:0/2869863658 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b84044e40 con 0x7f9b900af920 2024-04-25T08:40:38.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.037+0000 7f9b972ee640 1 -- 172.21.15.38:0/2869863658 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9b900cc1b0 con 0x7f9b900af920 2024-04-25T08:40:38.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.037+0000 7f9b81ffb640 1 -- 172.21.15.38:0/2869863658 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9b84079760 con 0x7f9b900af920 2024-04-25T08:40:38.030 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:40:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.039+0000 7f9b972ee640 1 -- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9b7406b4d0 msgr2=0x7f9b7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.039+0000 7f9b972ee640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9b7406b4d0 0x7f9b7406d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c000f60 tx=0x7f9b8c009040 comp rx=0 tx=0).stop 2024-04-25T08:40:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.039+0000 7f9b972ee640 1 -- 172.21.15.38:0/2869863658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b900af920 msgr2=0x7f9b90155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.039+0000 7f9b972ee640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b900af920 0x7f9b90155af0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f9b8402a270 tx=0x7f9b840046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.039+0000 7f9b972ee640 1 -- 172.21.15.38:0/2869863658 shutdown_connections 2024-04-25T08:40:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.039+0000 7f9b972ee640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9b7406b4d0 0x7f9b7406d990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.039+0000 7f9b972ee640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b9015dd90 0x7f9b900b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.039+0000 7f9b972ee640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b900af920 0x7f9b90155af0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.039+0000 7f9b972ee640 1 --2- 172.21.15.38:0/2869863658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b9006b2f0 0x7f9b9015f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.039+0000 7f9b972ee640 1 -- 172.21.15.38:0/2869863658 >> 172.21.15.38:0/2869863658 conn(0x7f9b90099a70 msgr2=0x7f9b900d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.039+0000 7f9b972ee640 1 -- 172.21.15.38:0/2869863658 shutdown_connections 2024-04-25T08:40:38.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.039+0000 7f9b972ee640 1 -- 172.21.15.38:0/2869863658 wait complete. 2024-04-25T08:40:38.042 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:40:38.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:38.043 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:40:38.065 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[89010]: fuse finished with error 0 and tester_r 0 2024-04-25T08:40:38.144 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:40:38.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:38.144 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:40:38.307 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:40:38.307 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:40:38.324 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:40:38.324 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:40:38.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:38.324 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:40:38.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:40:38.381 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:40:38.381 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:40:38.381 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:40:38.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:40:38.397 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:40:38.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:38.397 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:40:38.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:40:38.453 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:40:38.453 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:40:38.453 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:40:38.470 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:40:38.470 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:40:38.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:40:38.470 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:40:38.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:40:38.526 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:40:38.526 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:38.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.635+0000 7f0d24a76640 1 Processor -- start 2024-04-25T08:40:38.628 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.635+0000 7f0d24a76640 1 -- start start 2024-04-25T08:40:38.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.635+0000 7f0d24a76640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200b5710 0x7f0d20161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.635+0000 7f0d24a76640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d20161e00 con 0x7f0d200b93f0 2024-04-25T08:40:38.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.635+0000 7f0d24a76640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d201620e0 con 0x7f0d2015ddf0 2024-04-25T08:40:38.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.635+0000 7f0d24a76640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d201623c0 con 0x7f0d200b5710 2024-04-25T08:40:38.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1ed9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0d200b93f0 0x7f0d200d2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:48054/0 (socket says 172.21.15.38:48054) 2024-04-25T08:40:38.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1ed9d640 1 -- 172.21.15.38:0/2796614297 learned_addr learned my addr 172.21.15.38:0/2796614297 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1e59c640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200b5710 0x7f0d20161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053439971 0 0) 0x7f0d20161e00 con 0x7f0d200b93f0 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d0c003520 con 0x7f0d200b93f0 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506748849 0 0) 0x7f0d201620e0 con 0x7f0d2015ddf0 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d20161e00 con 0x7f0d2015ddf0 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1966729072 0 0) 0x7f0d0c003520 con 0x7f0d200b93f0 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200b5710 msgr2=0x7f0d20161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200b5710 0x7f0d20161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 >> v1:172.21.15.107:6789/0 conn(0x7f0d2015ddf0 legacy=0x7f0d200b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d200b9870 con 0x7f0d200b93f0 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f0d180019f0 con 0x7f0d200b93f0 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 >> v1:172.21.15.38:6789/0 conn(0x7f0d200b93f0 legacy=0x7f0d200d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d0c003b70 0x7f0d0c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d0c004800 0x7f0d0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200b5710 0x7f0d0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d201623c0 con 0x7f0d0c004800 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c00ba70 con 0x7f0d0c003b70 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d0c00bc70 con 0x7f0d200b5710 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d17fff640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d0c004800 0x7f0d0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1e59c640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d0c003b70 0x7f0d0c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.636+0000 7f0d1ed9d640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200b5710 0x7f0d0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.637+0000 7f0d17fff640 1 -- 172.21.15.38:0/2796614297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200b5710 msgr2=0x7f0d0c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.637+0000 7f0d17fff640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200b5710 0x7f0d0c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.637+0000 7f0d17fff640 1 -- 172.21.15.38:0/2796614297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d0c003b70 msgr2=0x7f0d0c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.637+0000 7f0d17fff640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d0c003b70 0x7f0d0c003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.637+0000 7f0d17fff640 1 -- 172.21.15.38:0/2796614297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c003520 con 0x7f0d0c004800 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.637+0000 7f0d1e59c640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d0c003b70 0x7f0d0c003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:38.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.637+0000 7f0d17fff640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d0c004800 0x7f0d0c004c80 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f0d080018a0 tx=0x7f0d0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3187fdb2e84baca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.638+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d080014f0 con 0x7f0d0c004800 2024-04-25T08:40:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.638+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d080052d0 con 0x7f0d0c004800 2024-04-25T08:40:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.638+0000 7f0d1dd9b640 1 -- 172.21.15.38:0/2796614297 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0802ac60 con 0x7f0d0c004800 2024-04-25T08:40:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.638+0000 7f0d24a76640 1 -- 172.21.15.38:0/2796614297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d0c004800 msgr2=0x7f0d0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.638+0000 7f0d24a76640 1 --2- 172.21.15.38:0/2796614297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d0c004800 0x7f0d0c004c80 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f0d080018a0 tx=0x7f0d0c005830 comp rx=0 tx=0).stop 2024-04-25T08:40:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.638+0000 7f0d1ed9d640 1 -- 172.21.15.38:0/2796614297 reap_dead start 2024-04-25T08:40:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.638+0000 7f0d24a76640 1 -- 172.21.15.38:0/2796614297 shutdown_connections 2024-04-25T08:40:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.638+0000 7f0d24a76640 1 -- 172.21.15.38:0/2796614297 >> 172.21.15.38:0/2796614297 conn(0x7f0d200d07c0 msgr2=0x7f0d200d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.638+0000 7f0d24a76640 1 -- 172.21.15.38:0/2796614297 shutdown_connections 2024-04-25T08:40:38.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.638+0000 7f0d24a76640 1 -- 172.21.15.38:0/2796614297 wait complete. 2024-04-25T08:40:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.639+0000 7f0d24a76640 1 Processor -- start 2024-04-25T08:40:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.640+0000 7f0d24a76640 1 -- start start 2024-04-25T08:40:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.640+0000 7f0d24a76640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d0c004800 0x7f0d2015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.640+0000 7f0d24a76640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200ca9f0 0x7f0d201f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.640+0000 7f0d24a76640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d200b3590 0x7f0d200b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.640+0000 7f0d24a76640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d20161b50 con 0x7f0d200b3590 2024-04-25T08:40:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.640+0000 7f0d24a76640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d201619d0 con 0x7f0d0c004800 2024-04-25T08:40:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.640+0000 7f0d24a76640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d200d2a50 con 0x7f0d200ca9f0 2024-04-25T08:40:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.640+0000 7f0d1ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d200b3590 0x7f0d200b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.640+0000 7f0d1ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d200b3590 0x7f0d200b3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52160/0 (socket says 172.21.15.38:52160) 2024-04-25T08:40:38.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.640+0000 7f0d1ed9d640 1 -- 172.21.15.38:0/4117800031 learned_addr learned my addr 172.21.15.38:0/4117800031 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d1e59c640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d0c004800 0x7f0d2015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d17fff640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200ca9f0 0x7f0d201f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d1ed9d640 1 -- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200ca9f0 msgr2=0x7f0d201f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d1ed9d640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200ca9f0 0x7f0d201f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d1ed9d640 1 -- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d0c004800 msgr2=0x7f0d2015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d1ed9d640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d0c004800 0x7f0d2015ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d1ed9d640 1 -- 172.21.15.38:0/4117800031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d200cba70 con 0x7f0d200b3590 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d1ed9d640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d200b3590 0x7f0d200b3a10 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f0d200d4a00 tx=0x7f0d1800a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d16ffd640 1 -- 172.21.15.38:0/4117800031 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10009690 con 0x7f0d200b3590 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d16ffd640 1 -- 172.21.15.38:0/4117800031 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d18018070 con 0x7f0d200b3590 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d24a76640 1 -- 172.21.15.38:0/4117800031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d200caef0 con 0x7f0d200b3590 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d16ffd640 1 -- 172.21.15.38:0/4117800031 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800ae40 con 0x7f0d200b3590 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.641+0000 7f0d24a76640 1 -- 172.21.15.38:0/4117800031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d200cb4b0 con 0x7f0d200b3590 2024-04-25T08:40:38.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.642+0000 7f0d24a76640 1 -- 172.21.15.38:0/4117800031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d20067dd0 con 0x7f0d200b3590 2024-04-25T08:40:38.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.643+0000 7f0d16ffd640 1 -- 172.21.15.38:0/4117800031 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d1801d460 con 0x7f0d200b3590 2024-04-25T08:40:38.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.643+0000 7f0d16ffd640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d0006b500 0x7f0d0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.643+0000 7f0d16ffd640 1 -- 172.21.15.38:0/4117800031 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0d1808a6b0 con 0x7f0d200b3590 2024-04-25T08:40:38.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.643+0000 7f0d1e59c640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d0006b500 0x7f0d0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.644+0000 7f0d1e59c640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d0006b500 0x7f0d0006d9c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009c00 tx=0x7f0d10009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:38.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.647+0000 7f0d16ffd640 1 -- 172.21.15.38:0/4117800031 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d20067dd0 con 0x7f0d200b3590 2024-04-25T08:40:38.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.865+0000 7f0d24a76640 1 -- 172.21.15.38:0/4117800031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d20068e80 con 0x7f0d200b3590 2024-04-25T08:40:38.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.866+0000 7f0d16ffd640 1 -- 172.21.15.38:0/4117800031 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0d180593a0 con 0x7f0d200b3590 2024-04-25T08:40:38.859 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:38.859 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:38.859 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:38.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.868+0000 7f0d24a76640 1 -- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d0006b500 msgr2=0x7f0d0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.868+0000 7f0d24a76640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d0006b500 0x7f0d0006d9c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0d10009c00 tx=0x7f0d10009190 comp rx=0 tx=0).stop 2024-04-25T08:40:38.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.868+0000 7f0d24a76640 1 -- 172.21.15.38:0/4117800031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d200b3590 msgr2=0x7f0d200b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.868+0000 7f0d24a76640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d200b3590 0x7f0d200b3a10 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f0d200d4a00 tx=0x7f0d1800a600 comp rx=0 tx=0).stop 2024-04-25T08:40:38.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.868+0000 7f0d24a76640 1 -- 172.21.15.38:0/4117800031 shutdown_connections 2024-04-25T08:40:38.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.868+0000 7f0d24a76640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0d200b3590 0x7f0d200b3a10 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.868+0000 7f0d24a76640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d0006b500 0x7f0d0006d9c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.868+0000 7f0d24a76640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0d200ca9f0 0x7f0d201f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.868+0000 7f0d24a76640 1 --2- 172.21.15.38:0/4117800031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d0c004800 0x7f0d2015ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.868+0000 7f0d24a76640 1 -- 172.21.15.38:0/4117800031 >> 172.21.15.38:0/4117800031 conn(0x7f0d200d07c0 msgr2=0x7f0d200d2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.868+0000 7f0d24a76640 1 -- 172.21.15.38:0/4117800031 shutdown_connections 2024-04-25T08:40:38.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.868+0000 7f0d24a76640 1 -- 172.21.15.38:0/4117800031 wait complete. 2024-04-25T08:40:38.873 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:40:38.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.981+0000 7f50a5709640 1 Processor -- start 2024-04-25T08:40:38.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.981+0000 7f50a5709640 1 -- start start 2024-04-25T08:40:38.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f50a5709640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f50a006c110 0x7f50a006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f50a5709640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50a00ce2e0 0x7f50a00d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f50a5709640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a006ca30 con 0x7f50a015df50 2024-04-25T08:40:38.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f50a5709640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a00cf2d0 con 0x7f50a006c110 2024-04-25T08:40:38.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f50a5709640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a00ceff0 con 0x7f50a00ce2e0 2024-04-25T08:40:38.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f509effd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f50a006c110 0x7f50a006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f509effd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f50a006c110 0x7f50a006c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39276/0 (socket says 172.21.15.38:39276) 2024-04-25T08:40:38.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f509effd640 1 -- 172.21.15.38:0/515502355 learned_addr learned my addr 172.21.15.38:0/515502355 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:38.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f509f7fe640 1 --2- 172.21.15.38:0/515502355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50a00ce2e0 0x7f50a00d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f509effd640 1 -- 172.21.15.38:0/515502355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50a00ce2e0 msgr2=0x7f50a00d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:38.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f509effd640 1 --2- 172.21.15.38:0/515502355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50a00ce2e0 0x7f50a00d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f509effd640 1 -- 172.21.15.38:0/515502355 >> v1:172.21.15.107:6789/0 conn(0x7f50a015df50 legacy=0x7f50a00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.982+0000 7f509effd640 1 -- 172.21.15.38:0/515502355 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a00d1f70 con 0x7f50a006c110 2024-04-25T08:40:38.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.983+0000 7f509effd640 1 --2- 172.21.15.38:0/515502355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a006c110 0x7f50a006c4f0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f5090009870 tx=0x7f509002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955a2658cdb10578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:38.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.983+0000 7f509dffb640 1 -- 172.21.15.38:0/515502355 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5090037070 con 0x7f50a006c110 2024-04-25T08:40:38.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.983+0000 7f509dffb640 1 -- 172.21.15.38:0/515502355 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5090032440 con 0x7f50a006c110 2024-04-25T08:40:38.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.983+0000 7f509dffb640 1 -- 172.21.15.38:0/515502355 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5090032810 con 0x7f50a006c110 2024-04-25T08:40:38.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.983+0000 7f50a5709640 1 -- 172.21.15.38:0/515502355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a006c110 msgr2=0x7f50a006c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.983+0000 7f50a5709640 1 --2- 172.21.15.38:0/515502355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a006c110 0x7f50a006c4f0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f5090009870 tx=0x7f509002a270 comp rx=0 tx=0).stop 2024-04-25T08:40:38.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.984+0000 7f50a5709640 1 -- 172.21.15.38:0/515502355 shutdown_connections 2024-04-25T08:40:38.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.984+0000 7f50a5709640 1 --2- 172.21.15.38:0/515502355 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50a00ce2e0 0x7f50a00d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.984+0000 7f50a5709640 1 --2- 172.21.15.38:0/515502355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a006c110 0x7f50a006c4f0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.984+0000 7f50a5709640 1 -- 172.21.15.38:0/515502355 >> 172.21.15.38:0/515502355 conn(0x7f50a00d3fb0 msgr2=0x7f50a00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:38.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.984+0000 7f50a5709640 1 -- 172.21.15.38:0/515502355 shutdown_connections 2024-04-25T08:40:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.984+0000 7f50a5709640 1 -- 172.21.15.38:0/515502355 wait complete. 2024-04-25T08:40:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.985+0000 7f50a5709640 1 Processor -- start 2024-04-25T08:40:38.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.985+0000 7f50a5709640 1 -- start start 2024-04-25T08:40:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f50a5709640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50a006c110 0x7f50a00ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f50a5709640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a00ce2e0 0x7f50a00cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f50a5709640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50a015df50 0x7f50a0148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f50a5709640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a00d2f90 con 0x7f50a00ce2e0 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f50a5709640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a00d2530 con 0x7f50a006c110 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f50a5709640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a00d3110 con 0x7f50a015df50 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f509e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a00ce2e0 0x7f50a00cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f509e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a00ce2e0 0x7f50a00cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39278/0 (socket says 172.21.15.38:39278) 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f509e7fc640 1 -- 172.21.15.38:0/3191204033 learned_addr learned my addr 172.21.15.38:0/3191204033 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f509effd640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50a006c110 0x7f50a00ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f509f7fe640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50a015df50 0x7f50a0148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f509e7fc640 1 -- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50a015df50 msgr2=0x7f50a0148820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f509e7fc640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50a015df50 0x7f50a0148820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f509e7fc640 1 -- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50a006c110 msgr2=0x7f50a00ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f509e7fc640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50a006c110 0x7f50a00ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f509e7fc640 1 -- 172.21.15.38:0/3191204033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a00c7780 con 0x7f50a00ce2e0 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f509f7fe640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50a015df50 0x7f50a0148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:38.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f509e7fc640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a00ce2e0 0x7f50a00cc930 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f509400c3f0 tx=0x7f509400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:38.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f50877fe640 1 -- 172.21.15.38:0/3191204033 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50940052d0 con 0x7f50a00ce2e0 2024-04-25T08:40:38.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f50877fe640 1 -- 172.21.15.38:0/3191204033 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5094005490 con 0x7f50a00ce2e0 2024-04-25T08:40:38.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.986+0000 7f50a5709640 1 -- 172.21.15.38:0/3191204033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a00c7a60 con 0x7f50a00ce2e0 2024-04-25T08:40:38.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.987+0000 7f50877fe640 1 -- 172.21.15.38:0/3191204033 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094007260 con 0x7f50a00ce2e0 2024-04-25T08:40:38.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.987+0000 7f50a5709640 1 -- 172.21.15.38:0/3191204033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a00c8020 con 0x7f50a00ce2e0 2024-04-25T08:40:38.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.987+0000 7f50a5709640 1 -- 172.21.15.38:0/3191204033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a0068e80 con 0x7f50a00ce2e0 2024-04-25T08:40:38.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.988+0000 7f50877fe640 1 -- 172.21.15.38:0/3191204033 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5094007400 con 0x7f50a00ce2e0 2024-04-25T08:40:38.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.988+0000 7f50877fe640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f508c06b4d0 0x7f508c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:38.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.988+0000 7f50877fe640 1 -- 172.21.15.38:0/3191204033 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5094089790 con 0x7f50a00ce2e0 2024-04-25T08:40:38.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.989+0000 7f509effd640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f508c06b4d0 0x7f508c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:38.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.989+0000 7f509effd640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f508c06b4d0 0x7f508c06d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5090034000 tx=0x7f509002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:38.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:38.993+0000 7f50877fe640 1 -- 172.21.15.38:0/3191204033 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5094057e30 con 0x7f50a00ce2e0 2024-04-25T08:40:39.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.174+0000 7f50a5709640 1 -- 172.21.15.38:0/3191204033 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50a015d670 con 0x7f50a00ce2e0 2024-04-25T08:40:39.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.175+0000 7f50877fe640 1 -- 172.21.15.38:0/3191204033 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16940 (secure 0 0 0) 0x7f509405bed0 con 0x7f50a00ce2e0 2024-04-25T08:40:39.168 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:39.168 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":121,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:59.121832+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:55.759766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:56.451485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:40:39.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.177+0000 7f50a5709640 1 -- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f508c06b4d0 msgr2=0x7f508c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.177+0000 7f50a5709640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f508c06b4d0 0x7f508c06d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5090034000 tx=0x7f509002e000 comp rx=0 tx=0).stop 2024-04-25T08:40:39.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.177+0000 7f50a5709640 1 -- 172.21.15.38:0/3191204033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a00ce2e0 msgr2=0x7f50a00cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.177+0000 7f50a5709640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a00ce2e0 0x7f50a00cc930 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f509400c3f0 tx=0x7f509400c8b0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.177+0000 7f50a5709640 1 -- 172.21.15.38:0/3191204033 shutdown_connections 2024-04-25T08:40:39.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.177+0000 7f50a5709640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f508c06b4d0 0x7f508c06d990 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.177+0000 7f50a5709640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f50a015df50 0x7f50a0148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.177+0000 7f50a5709640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f50a00ce2e0 0x7f50a00cc930 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.177+0000 7f50a5709640 1 --2- 172.21.15.38:0/3191204033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f50a006c110 0x7f50a00ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.177+0000 7f50a5709640 1 -- 172.21.15.38:0/3191204033 >> 172.21.15.38:0/3191204033 conn(0x7f50a00d3fb0 msgr2=0x7f50a00d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:39.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.177+0000 7f50a5709640 1 -- 172.21.15.38:0/3191204033 shutdown_connections 2024-04-25T08:40:39.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.177+0000 7f50a5709640 1 -- 172.21.15.38:0/3191204033 wait complete. 2024-04-25T08:40:39.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:39:55.759766+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:39:56.451485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:40:39.182 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:40:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.293+0000 7fb63adf5640 1 Processor -- start 2024-04-25T08:40:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.293+0000 7fb63adf5640 1 -- start start 2024-04-25T08:40:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.294+0000 7fb63adf5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb63406c700 0x7fb634162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.294+0000 7fb63adf5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb63406cae0 0x7fb63415e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.294+0000 7fb63adf5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63415ec20 con 0x7fb63415e580 2024-04-25T08:40:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.295+0000 7fb63adf5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63415e9f0 con 0x7fb63406c700 2024-04-25T08:40:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.295+0000 7fb63adf5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb634161c80 con 0x7fb63406cae0 2024-04-25T08:40:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.295+0000 7fb638b91640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb63406c700 0x7fb634162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.295+0000 7fb638b91640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb63406c700 0x7fb634162110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39286/0 (socket says 172.21.15.38:39286) 2024-04-25T08:40:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.295+0000 7fb638b91640 1 -- 172.21.15.38:0/3924723219 learned_addr learned my addr 172.21.15.38:0/3924723219 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.295+0000 7fb633fff640 1 --2- 172.21.15.38:0/3924723219 >> v2:172.21.15.107:3300/0 conn(0x7fb63406cae0 0x7fb63415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.295+0000 7fb638b91640 1 -- 172.21.15.38:0/3924723219 >> v2:172.21.15.107:3300/0 conn(0x7fb63406cae0 msgr2=0x7fb63415e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.295+0000 7fb638b91640 1 --2- 172.21.15.38:0/3924723219 >> v2:172.21.15.107:3300/0 conn(0x7fb63406cae0 0x7fb63415e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.295+0000 7fb638b91640 1 -- 172.21.15.38:0/3924723219 >> v1:172.21.15.107:6789/0 conn(0x7fb63415e580 legacy=0x7fb6340cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.295+0000 7fb638b91640 1 -- 172.21.15.38:0/3924723219 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63415ef00 con 0x7fb63406c700 2024-04-25T08:40:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.295+0000 7fb633fff640 1 --2- 172.21.15.38:0/3924723219 >> v2:172.21.15.107:3300/0 conn(0x7fb63406cae0 0x7fb63415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.296+0000 7fb638b91640 1 --2- 172.21.15.38:0/3924723219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63406c700 0x7fb634162110 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fb6340b3050 tx=0x7fb62c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e68fddec444495f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.296+0000 7fb6337fe640 1 -- 172.21.15.38:0/3924723219 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c036070 con 0x7fb63406c700 2024-04-25T08:40:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.296+0000 7fb6337fe640 1 -- 172.21.15.38:0/3924723219 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c031600 con 0x7fb63406c700 2024-04-25T08:40:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.296+0000 7fb6337fe640 1 -- 172.21.15.38:0/3924723219 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0319d0 con 0x7fb63406c700 2024-04-25T08:40:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.296+0000 7fb63adf5640 1 -- 172.21.15.38:0/3924723219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63406c700 msgr2=0x7fb634162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.296+0000 7fb63adf5640 1 --2- 172.21.15.38:0/3924723219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63406c700 0x7fb634162110 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fb6340b3050 tx=0x7fb62c02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.296+0000 7fb63adf5640 1 -- 172.21.15.38:0/3924723219 shutdown_connections 2024-04-25T08:40:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.296+0000 7fb63adf5640 1 --2- 172.21.15.38:0/3924723219 >> v2:172.21.15.107:3300/0 conn(0x7fb63406cae0 0x7fb63415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.296+0000 7fb63adf5640 1 --2- 172.21.15.38:0/3924723219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63406c700 0x7fb634162110 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.296+0000 7fb63adf5640 1 -- 172.21.15.38:0/3924723219 >> 172.21.15.38:0/3924723219 conn(0x7fb63408b000 msgr2=0x7fb63408b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.296+0000 7fb63adf5640 1 -- 172.21.15.38:0/3924723219 shutdown_connections 2024-04-25T08:40:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.296+0000 7fb63adf5640 1 -- 172.21.15.38:0/3924723219 wait complete. 2024-04-25T08:40:39.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.297+0000 7fb63adf5640 1 Processor -- start 2024-04-25T08:40:39.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.298+0000 7fb63adf5640 1 -- start start 2024-04-25T08:40:39.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.298+0000 7fb63adf5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb63406c700 0x7fb634070440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.298+0000 7fb63adf5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63406cae0 0x7fb6341f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.298+0000 7fb63adf5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63415e580 0x7fb6341f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.298+0000 7fb63adf5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63406c190 con 0x7fb63406cae0 2024-04-25T08:40:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.298+0000 7fb63adf5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb634161910 con 0x7fb63415e580 2024-04-25T08:40:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.298+0000 7fb63adf5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63406cf90 con 0x7fb63406c700 2024-04-25T08:40:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.298+0000 7fb633fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63406cae0 0x7fb6341f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb633fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63406cae0 0x7fb6341f31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39290/0 (socket says 172.21.15.38:39290) 2024-04-25T08:40:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb633fff640 1 -- 172.21.15.38:0/851266105 learned_addr learned my addr 172.21.15.38:0/851266105 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb638b91640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb63406c700 0x7fb634070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb639392640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63415e580 0x7fb6341f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb633fff640 1 -- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb63406c700 msgr2=0x7fb634070440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb633fff640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb63406c700 0x7fb634070440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb633fff640 1 -- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63415e580 msgr2=0x7fb6341f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb633fff640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63415e580 0x7fb6341f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb633fff640 1 -- 172.21.15.38:0/851266105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6341f7730 con 0x7fb63406cae0 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb633fff640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63406cae0 0x7fb6341f31a0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fb62800c7f0 tx=0x7fb62800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb6317fa640 1 -- 172.21.15.38:0/851266105 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb628015070 con 0x7fb63406cae0 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb6317fa640 1 -- 172.21.15.38:0/851266105 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb628007a30 con 0x7fb63406cae0 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb6317fa640 1 -- 172.21.15.38:0/851266105 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb628007e00 con 0x7fb63406cae0 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb63adf5640 1 -- 172.21.15.38:0/851266105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6341f7a10 con 0x7fb63406cae0 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.299+0000 7fb63adf5640 1 -- 172.21.15.38:0/851266105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6341f7fd0 con 0x7fb63406cae0 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.300+0000 7fb63adf5640 1 -- 172.21.15.38:0/851266105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb634068e80 con 0x7fb63406cae0 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.301+0000 7fb6317fa640 1 -- 172.21.15.38:0/851266105 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb628018070 con 0x7fb63406cae0 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.301+0000 7fb6317fa640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb62006b4d0 0x7fb62006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.301+0000 7fb6317fa640 1 -- 172.21.15.38:0/851266105 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb62808a1c0 con 0x7fb63406cae0 2024-04-25T08:40:39.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.302+0000 7fb638b91640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb62006b4d0 0x7fb62006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.302+0000 7fb638b91640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb62006b4d0 0x7fb62006d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb62c033000 tx=0x7fb62c009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:39.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.305+0000 7fb6317fa640 1 -- 172.21.15.38:0/851266105 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6280588f0 con 0x7fb63406cae0 2024-04-25T08:40:39.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.485+0000 7fb63adf5640 1 -- 172.21.15.38:0/851266105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6340b2540 con 0x7fb63406cae0 2024-04-25T08:40:39.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.486+0000 7fb6317fa640 1 -- 172.21.15.38:0/851266105 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb62805c990 con 0x7fb63406cae0 2024-04-25T08:40:39.479 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:40:39.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.487+0000 7fb63adf5640 1 -- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb62006b4d0 msgr2=0x7fb62006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.487+0000 7fb63adf5640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb62006b4d0 0x7fb62006d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb62c033000 tx=0x7fb62c009000 comp rx=0 tx=0).stop 2024-04-25T08:40:39.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.487+0000 7fb63adf5640 1 -- 172.21.15.38:0/851266105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63406cae0 msgr2=0x7fb6341f31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.487+0000 7fb63adf5640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63406cae0 0x7fb6341f31a0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fb62800c7f0 tx=0x7fb62800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.488+0000 7fb63adf5640 1 -- 172.21.15.38:0/851266105 shutdown_connections 2024-04-25T08:40:39.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.488+0000 7fb63adf5640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb63415e580 0x7fb6341f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.488+0000 7fb63adf5640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb63406cae0 0x7fb6341f31a0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.488+0000 7fb63adf5640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb62006b4d0 0x7fb62006d990 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.488+0000 7fb63adf5640 1 --2- 172.21.15.38:0/851266105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb63406c700 0x7fb634070440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.488+0000 7fb63adf5640 1 -- 172.21.15.38:0/851266105 >> 172.21.15.38:0/851266105 conn(0x7fb63408b000 msgr2=0x7fb63408b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:39.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.488+0000 7fb63adf5640 1 -- 172.21.15.38:0/851266105 shutdown_connections 2024-04-25T08:40:39.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.488+0000 7fb63adf5640 1 -- 172.21.15.38:0/851266105 wait complete. 2024-04-25T08:40:39.492 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:40:39.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.603+0000 7f7f6f038640 1 Processor -- start 2024-04-25T08:40:39.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.603+0000 7f7f6f038640 1 -- start start 2024-04-25T08:40:39.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f6f038640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f680d0320 0x7f7f680d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f6f038640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7f680d0700 0x7f7f680d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f6f038640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f680d2330 con 0x7f7f680d1e90 2024-04-25T08:40:39.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f6f038640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f680d2610 con 0x7f7f680d0700 2024-04-25T08:40:39.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f6f038640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680d28f0 con 0x7f7f680d0320 2024-04-25T08:40:39.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f6d5d5640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7f680d1e90 0x7f7f680b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53360/0 (socket says 172.21.15.38:53360) 2024-04-25T08:40:39.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f6d5d5640 1 -- 172.21.15.38:0/959002440 learned_addr learned my addr 172.21.15.38:0/959002440 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:39.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f67fff640 1 --2- 172.21.15.38:0/959002440 >> v2:172.21.15.107:3300/0 conn(0x7f7f680d0700 0x7f7f680d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f6cdd4640 1 --2- 172.21.15.38:0/959002440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f680d0320 0x7f7f680d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f677fe640 1 -- 172.21.15.38:0/959002440 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889591254 0 0) 0x7f7f680d2330 con 0x7f7f680d1e90 2024-04-25T08:40:39.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f677fe640 1 -- 172.21.15.38:0/959002440 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f58003520 con 0x7f7f680d1e90 2024-04-25T08:40:39.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f67fff640 1 -- 172.21.15.38:0/959002440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f680d0320 msgr2=0x7f7f680d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:39.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f67fff640 1 --2- 172.21.15.38:0/959002440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f680d0320 0x7f7f680d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f67fff640 1 -- 172.21.15.38:0/959002440 >> v1:172.21.15.38:6789/0 conn(0x7f7f680d1e90 legacy=0x7f7f680b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f67fff640 1 -- 172.21.15.38:0/959002440 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f680d2bd0 con 0x7f7f680d0700 2024-04-25T08:40:39.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.604+0000 7f7f6cdd4640 1 --2- 172.21.15.38:0/959002440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f680d0320 0x7f7f680d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:39.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.605+0000 7f7f67fff640 1 --2- 172.21.15.38:0/959002440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f680d0700 0x7f7f680d1950 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7f540097a0 tx=0x7f7f5402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=659a082cb0b6d2d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:39.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.605+0000 7f7f677fe640 1 -- 172.21.15.38:0/959002440 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54037070 con 0x7f7f680d0700 2024-04-25T08:40:39.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.605+0000 7f7f677fe640 1 -- 172.21.15.38:0/959002440 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f54032440 con 0x7f7f680d0700 2024-04-25T08:40:39.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.605+0000 7f7f677fe640 1 -- 172.21.15.38:0/959002440 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f540327f0 con 0x7f7f680d0700 2024-04-25T08:40:39.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.606+0000 7f7f6f038640 1 -- 172.21.15.38:0/959002440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f680d0700 msgr2=0x7f7f680d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.606+0000 7f7f6f038640 1 --2- 172.21.15.38:0/959002440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f680d0700 0x7f7f680d1950 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7f540097a0 tx=0x7f7f5402a210 comp rx=0 tx=0).stop 2024-04-25T08:40:39.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.606+0000 7f7f6f038640 1 -- 172.21.15.38:0/959002440 shutdown_connections 2024-04-25T08:40:39.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.606+0000 7f7f6f038640 1 --2- 172.21.15.38:0/959002440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f680d0700 0x7f7f680d1950 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.606+0000 7f7f6f038640 1 --2- 172.21.15.38:0/959002440 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f680d0320 0x7f7f680d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.606+0000 7f7f6f038640 1 -- 172.21.15.38:0/959002440 >> 172.21.15.38:0/959002440 conn(0x7f7f680668f0 msgr2=0x7f7f68066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:39.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.606+0000 7f7f6f038640 1 -- 172.21.15.38:0/959002440 shutdown_connections 2024-04-25T08:40:39.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.606+0000 7f7f6f038640 1 -- 172.21.15.38:0/959002440 wait complete. 2024-04-25T08:40:39.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.607+0000 7f7f6f038640 1 Processor -- start 2024-04-25T08:40:39.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.607+0000 7f7f6f038640 1 -- start start 2024-04-25T08:40:39.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6f038640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f680d0320 0x7f7f68153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6f038640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f680d0700 0x7f7f680ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6f038640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f680d1e90 0x7f7f680ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6f038640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680d0b40 con 0x7f7f680d0320 2024-04-25T08:40:39.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6f038640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f68161210 con 0x7f7f680d1e90 2024-04-25T08:40:39.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6f038640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f680b5b80 con 0x7f7f680d0700 2024-04-25T08:40:39.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6cdd4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f680d0320 0x7f7f68153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6cdd4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f680d0320 0x7f7f68153d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39302/0 (socket says 172.21.15.38:39302) 2024-04-25T08:40:39.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6cdd4640 1 -- 172.21.15.38:0/848700688 learned_addr learned my addr 172.21.15.38:0/848700688 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:39.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f67fff640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f680d0700 0x7f7f680ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6d5d5640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f680d1e90 0x7f7f680ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6cdd4640 1 -- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f680d0700 msgr2=0x7f7f680ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6cdd4640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f680d0700 0x7f7f680ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6cdd4640 1 -- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f680d1e90 msgr2=0x7f7f680ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6cdd4640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f680d1e90 0x7f7f680ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.608+0000 7f7f6cdd4640 1 -- 172.21.15.38:0/848700688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f680b4cd0 con 0x7f7f680d0320 2024-04-25T08:40:39.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.609+0000 7f7f6cdd4640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f680d0320 0x7f7f68153d40 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00e960 tx=0x7f7f5c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:39.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.609+0000 7f7f657fa640 1 -- 172.21.15.38:0/848700688 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c017070 con 0x7f7f680d0320 2024-04-25T08:40:39.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.609+0000 7f7f657fa640 1 -- 172.21.15.38:0/848700688 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c00ca00 con 0x7f7f680d0320 2024-04-25T08:40:39.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.609+0000 7f7f6f038640 1 -- 172.21.15.38:0/848700688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f68154280 con 0x7f7f680d0320 2024-04-25T08:40:39.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.609+0000 7f7f657fa640 1 -- 172.21.15.38:0/848700688 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00cdd0 con 0x7f7f680d0320 2024-04-25T08:40:39.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.609+0000 7f7f6f038640 1 -- 172.21.15.38:0/848700688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f680b5250 con 0x7f7f680d0320 2024-04-25T08:40:39.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.610+0000 7f7f6f038640 1 -- 172.21.15.38:0/848700688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f68068e80 con 0x7f7f680d0320 2024-04-25T08:40:39.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.610+0000 7f7f657fa640 1 -- 172.21.15.38:0/848700688 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f5c01a070 con 0x7f7f680d0320 2024-04-25T08:40:39.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.611+0000 7f7f657fa640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f4c06b4d0 0x7f7f4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.611+0000 7f7f657fa640 1 -- 172.21.15.38:0/848700688 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7f5c08a130 con 0x7f7f680d0320 2024-04-25T08:40:39.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.611+0000 7f7f67fff640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f4c06b4d0 0x7f7f4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.611+0000 7f7f67fff640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f4c06b4d0 0x7f7f4c06d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7f54034000 tx=0x7f7f5402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:39.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.615+0000 7f7f657fa640 1 -- 172.21.15.38:0/848700688 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f5c058860 con 0x7f7f680d0320 2024-04-25T08:40:39.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.792+0000 7f7f6f038640 1 -- 172.21.15.38:0/848700688 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f680b3050 con 0x7f7f680d0320 2024-04-25T08:40:39.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.793+0000 7f7f657fa640 1 -- 172.21.15.38:0/848700688 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f5c05c900 con 0x7f7f680d0320 2024-04-25T08:40:39.786 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:40:39.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.795+0000 7f7f6f038640 1 -- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f4c06b4d0 msgr2=0x7f7f4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.795+0000 7f7f6f038640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f4c06b4d0 0x7f7f4c06d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7f54034000 tx=0x7f7f5402e000 comp rx=0 tx=0).stop 2024-04-25T08:40:39.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.795+0000 7f7f6f038640 1 -- 172.21.15.38:0/848700688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f680d0320 msgr2=0x7f7f68153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.795+0000 7f7f6f038640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f680d0320 0x7f7f68153d40 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00e960 tx=0x7f7f5c00ee20 comp rx=0 tx=0).stop 2024-04-25T08:40:39.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.795+0000 7f7f6f038640 1 -- 172.21.15.38:0/848700688 shutdown_connections 2024-04-25T08:40:39.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.795+0000 7f7f6f038640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f680d1e90 0x7f7f680ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.795+0000 7f7f6f038640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f4c06b4d0 0x7f7f4c06d990 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.795+0000 7f7f6f038640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f680d0700 0x7f7f680ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.795+0000 7f7f6f038640 1 --2- 172.21.15.38:0/848700688 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f680d0320 0x7f7f68153d40 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.795+0000 7f7f6f038640 1 -- 172.21.15.38:0/848700688 >> 172.21.15.38:0/848700688 conn(0x7f7f680668f0 msgr2=0x7f7f680d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:39.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.795+0000 7f7f6f038640 1 -- 172.21.15.38:0/848700688 shutdown_connections 2024-04-25T08:40:39.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.795+0000 7f7f6f038640 1 -- 172.21.15.38:0/848700688 wait complete. 2024-04-25T08:40:39.799 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:40:39.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.910+0000 7fe8893ab640 1 Processor -- start 2024-04-25T08:40:39.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.910+0000 7fe8893ab640 1 -- start start 2024-04-25T08:40:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe8893ab640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8840d0320 0x7fe8840d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe8893ab640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe8840d0700 0x7fe8840d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe8893ab640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8840d2330 con 0x7fe8840d1e90 2024-04-25T08:40:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe8893ab640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8840d2610 con 0x7fe8840d0700 2024-04-25T08:40:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe8893ab640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8840d28f0 con 0x7fe8840d0320 2024-04-25T08:40:39.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe8827fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe8840d0700 0x7fe8840d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe8837fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe8840d1e90 0x7fe8840b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53366/0 (socket says 172.21.15.38:53366) 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe8837fe640 1 -- 172.21.15.38:0/1264970328 learned_addr learned my addr 172.21.15.38:0/1264970328 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe882ffd640 1 --2- 172.21.15.38:0/1264970328 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8840d0320 0x7fe8840d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe8827fc640 1 -- 172.21.15.38:0/1264970328 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8840d0320 msgr2=0x7fe8840d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe8827fc640 1 --2- 172.21.15.38:0/1264970328 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8840d0320 0x7fe8840d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe8827fc640 1 -- 172.21.15.38:0/1264970328 >> v1:172.21.15.38:6789/0 conn(0x7fe8840d1e90 legacy=0x7fe8840b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe8827fc640 1 -- 172.21.15.38:0/1264970328 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8840d2bd0 con 0x7fe8840d0700 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe881ffb640 1 -- 172.21.15.38:0/1264970328 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983525235 0 0) 0x7fe8840d2330 con 0x7fe8840d1e90 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.911+0000 7fe882ffd640 1 --2- 172.21.15.38:0/1264970328 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8840d0320 0x7fe8840d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.912+0000 7fe8827fc640 1 --2- 172.21.15.38:0/1264970328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8840d0700 0x7fe8840d1950 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0097a0 tx=0x7fe86c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd540ef2fdf61ec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.912+0000 7fe881ffb640 1 -- 172.21.15.38:0/1264970328 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c037070 con 0x7fe8840d0700 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.912+0000 7fe881ffb640 1 -- 172.21.15.38:0/1264970328 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c032440 con 0x7fe8840d0700 2024-04-25T08:40:39.907 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.912+0000 7fe881ffb640 1 -- 172.21.15.38:0/1264970328 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c032810 con 0x7fe8840d0700 2024-04-25T08:40:39.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.913+0000 7fe8893ab640 1 -- 172.21.15.38:0/1264970328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8840d0700 msgr2=0x7fe8840d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.913+0000 7fe8893ab640 1 --2- 172.21.15.38:0/1264970328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8840d0700 0x7fe8840d1950 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0097a0 tx=0x7fe86c02a210 comp rx=0 tx=0).stop 2024-04-25T08:40:39.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.913+0000 7fe8893ab640 1 -- 172.21.15.38:0/1264970328 shutdown_connections 2024-04-25T08:40:39.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.913+0000 7fe8893ab640 1 --2- 172.21.15.38:0/1264970328 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8840d0700 0x7fe8840d1950 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.913+0000 7fe8893ab640 1 --2- 172.21.15.38:0/1264970328 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8840d0320 0x7fe8840d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.913+0000 7fe8893ab640 1 -- 172.21.15.38:0/1264970328 >> 172.21.15.38:0/1264970328 conn(0x7fe8840668f0 msgr2=0x7fe884066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:39.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.913+0000 7fe8893ab640 1 -- 172.21.15.38:0/1264970328 shutdown_connections 2024-04-25T08:40:39.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.913+0000 7fe8893ab640 1 -- 172.21.15.38:0/1264970328 wait complete. 2024-04-25T08:40:39.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.914+0000 7fe8893ab640 1 Processor -- start 2024-04-25T08:40:39.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.914+0000 7fe8893ab640 1 -- start start 2024-04-25T08:40:39.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8893ab640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8840d0320 0x7fe8841592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8893ab640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8840d0700 0x7fe88415c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8893ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8840d1e90 0x7fe8840b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8893ab640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8840d0b40 con 0x7fe8840d1e90 2024-04-25T08:40:39.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8893ab640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe884161210 con 0x7fe8840d0320 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8893ab640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88415b6e0 con 0x7fe8840d0700 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8840d1e90 0x7fe8840b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8840d1e90 0x7fe8840b5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39322/0 (socket says 172.21.15.38:39322) 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8837fe640 1 -- 172.21.15.38:0/3746040879 learned_addr learned my addr 172.21.15.38:0/3746040879 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe882ffd640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8840d0320 0x7fe8841592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8827fc640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8840d0700 0x7fe88415c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8837fe640 1 -- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8840d0700 msgr2=0x7fe88415c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8837fe640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8840d0700 0x7fe88415c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8837fe640 1 -- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8840d0320 msgr2=0x7fe8841592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8837fe640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8840d0320 0x7fe8841592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8837fe640 1 -- 172.21.15.38:0/3746040879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88415b9c0 con 0x7fe8840d1e90 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.915+0000 7fe8827fc640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8840d0700 0x7fe88415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:39.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.916+0000 7fe8837fe640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8840d1e90 0x7fe8840b5ab0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fe87400c780 tx=0x7fe87400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.916+0000 7fe87b7fe640 1 -- 172.21.15.38:0/3746040879 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe874018070 con 0x7fe8840d1e90 2024-04-25T08:40:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.916+0000 7fe87b7fe640 1 -- 172.21.15.38:0/3746040879 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87400ce30 con 0x7fe8840d1e90 2024-04-25T08:40:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.916+0000 7fe8893ab640 1 -- 172.21.15.38:0/3746040879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88415bca0 con 0x7fe8840d1e90 2024-04-25T08:40:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.916+0000 7fe87b7fe640 1 -- 172.21.15.38:0/3746040879 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87400a840 con 0x7fe8840d1e90 2024-04-25T08:40:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.916+0000 7fe8893ab640 1 -- 172.21.15.38:0/3746040879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88415c260 con 0x7fe8840d1e90 2024-04-25T08:40:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.917+0000 7fe8893ab640 1 -- 172.21.15.38:0/3746040879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe884068e80 con 0x7fe8840d1e90 2024-04-25T08:40:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.917+0000 7fe87b7fe640 1 -- 172.21.15.38:0/3746040879 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe87401b070 con 0x7fe8840d1e90 2024-04-25T08:40:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.918+0000 7fe87b7fe640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe86406b4d0 0x7fe86406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.918+0000 7fe87b7fe640 1 -- 172.21.15.38:0/3746040879 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8740897f0 con 0x7fe8840d1e90 2024-04-25T08:40:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.918+0000 7fe882ffd640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe86406b4d0 0x7fe86406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.918+0000 7fe882ffd640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe86406b4d0 0x7fe86406d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00ab60 tx=0x7fe87c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:39.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:39.922+0000 7fe87b7fe640 1 -- 172.21.15.38:0/3746040879 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe874057e80 con 0x7fe8840d1e90 2024-04-25T08:40:40.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.101+0000 7fe8893ab640 1 -- 172.21.15.38:0/3746040879 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe8840b28c0 con 0x7fe8840d1e90 2024-04-25T08:40:40.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.101+0000 7fe87b7fe640 1 -- 172.21.15.38:0/3746040879 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe87405bf20 con 0x7fe8840d1e90 2024-04-25T08:40:40.094 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:40:40.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.103+0000 7fe8893ab640 1 -- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe86406b4d0 msgr2=0x7fe86406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.103+0000 7fe8893ab640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe86406b4d0 0x7fe86406d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe87c00ab60 tx=0x7fe87c008040 comp rx=0 tx=0).stop 2024-04-25T08:40:40.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.103+0000 7fe8893ab640 1 -- 172.21.15.38:0/3746040879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8840d1e90 msgr2=0x7fe8840b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.103+0000 7fe8893ab640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8840d1e90 0x7fe8840b5ab0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fe87400c780 tx=0x7fe87400cc40 comp rx=0 tx=0).stop 2024-04-25T08:40:40.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.103+0000 7fe8893ab640 1 -- 172.21.15.38:0/3746040879 shutdown_connections 2024-04-25T08:40:40.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.103+0000 7fe8893ab640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe8840d1e90 0x7fe8840b5ab0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.103+0000 7fe8893ab640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe86406b4d0 0x7fe86406d990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.103+0000 7fe8893ab640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8840d0700 0x7fe88415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.103+0000 7fe8893ab640 1 --2- 172.21.15.38:0/3746040879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8840d0320 0x7fe8841592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.103+0000 7fe8893ab640 1 -- 172.21.15.38:0/3746040879 >> 172.21.15.38:0/3746040879 conn(0x7fe8840668f0 msgr2=0x7fe88415ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:40.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.103+0000 7fe8893ab640 1 -- 172.21.15.38:0/3746040879 shutdown_connections 2024-04-25T08:40:40.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.103+0000 7fe8893ab640 1 -- 172.21.15.38:0/3746040879 wait complete. 2024-04-25T08:40:40.108 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:40.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.217+0000 7f426db22640 1 Processor -- start 2024-04-25T08:40:40.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.217+0000 7f426db22640 1 -- start start 2024-04-25T08:40:40.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f426db22640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f426815dc10 0x7f42680ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f426db22640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f42680d0f60 0x7f42680b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f426db22640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42680d3e70 con 0x7f42680ced60 2024-04-25T08:40:40.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f426db22640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426815dff0 con 0x7f426815dc10 2024-04-25T08:40:40.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f426db22640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42680b6990 con 0x7f42680d0f60 2024-04-25T08:40:40.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f42677fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f426815dc10 0x7f42680ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f42677fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f426815dc10 0x7f42680ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39332/0 (socket says 172.21.15.38:39332) 2024-04-25T08:40:40.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f42677fe640 1 -- 172.21.15.38:0/3144675514 learned_addr learned my addr 172.21.15.38:0/3144675514 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:40.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f4267fff640 1 --2- 172.21.15.38:0/3144675514 >> v2:172.21.15.107:3300/0 conn(0x7f42680d0f60 0x7f42680b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f42677fe640 1 -- 172.21.15.38:0/3144675514 >> v2:172.21.15.107:3300/0 conn(0x7f42680d0f60 msgr2=0x7f42680b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:40.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f42677fe640 1 --2- 172.21.15.38:0/3144675514 >> v2:172.21.15.107:3300/0 conn(0x7f42680d0f60 0x7f42680b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f42677fe640 1 -- 172.21.15.38:0/3144675514 >> v1:172.21.15.107:6789/0 conn(0x7f42680ced60 legacy=0x7f42680d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.218+0000 7f42677fe640 1 -- 172.21.15.38:0/3144675514 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42680d4150 con 0x7f426815dc10 2024-04-25T08:40:40.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.219+0000 7f42677fe640 1 --2- 172.21.15.38:0/3144675514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426815dc10 0x7f42680ce820 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f42680cdee0 tx=0x7f425802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40294ffbb018c5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:40.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.219+0000 7f42667fc640 1 -- 172.21.15.38:0/3144675514 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258037070 con 0x7f426815dc10 2024-04-25T08:40:40.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.219+0000 7f42667fc640 1 -- 172.21.15.38:0/3144675514 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4258032440 con 0x7f426815dc10 2024-04-25T08:40:40.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.219+0000 7f42667fc640 1 -- 172.21.15.38:0/3144675514 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258032810 con 0x7f426815dc10 2024-04-25T08:40:40.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.220+0000 7f426db22640 1 -- 172.21.15.38:0/3144675514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426815dc10 msgr2=0x7f42680ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.220+0000 7f426db22640 1 --2- 172.21.15.38:0/3144675514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426815dc10 0x7f42680ce820 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f42680cdee0 tx=0x7f425802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.220+0000 7f426db22640 1 -- 172.21.15.38:0/3144675514 shutdown_connections 2024-04-25T08:40:40.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.220+0000 7f426db22640 1 --2- 172.21.15.38:0/3144675514 >> v2:172.21.15.107:3300/0 conn(0x7f42680d0f60 0x7f42680b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.220+0000 7f426db22640 1 --2- 172.21.15.38:0/3144675514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f426815dc10 0x7f42680ce820 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.220+0000 7f426db22640 1 -- 172.21.15.38:0/3144675514 >> 172.21.15.38:0/3144675514 conn(0x7f4268066d10 msgr2=0x7f4268067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:40.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.220+0000 7f426db22640 1 -- 172.21.15.38:0/3144675514 shutdown_connections 2024-04-25T08:40:40.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.220+0000 7f426db22640 1 -- 172.21.15.38:0/3144675514 wait complete. 2024-04-25T08:40:40.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.221+0000 7f426db22640 1 Processor -- start 2024-04-25T08:40:40.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.221+0000 7f426db22640 1 -- start start 2024-04-25T08:40:40.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f426db22640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42680ced60 0x7f4268152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f426db22640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42680d0f60 0x7f42680aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f426db22640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f426815dc10 0x7f42680ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f426db22640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42680d3900 con 0x7f42680ced60 2024-04-25T08:40:40.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f426db22640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42680b6bc0 con 0x7f426815dc10 2024-04-25T08:40:40.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f426db22640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42680d3a80 con 0x7f42680d0f60 2024-04-25T08:40:40.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f42677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42680ced60 0x7f4268152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f42677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42680ced60 0x7f4268152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39336/0 (socket says 172.21.15.38:39336) 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f42677fe640 1 -- 172.21.15.38:0/3100088235 learned_addr learned my addr 172.21.15.38:0/3100088235 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f42677fe640 1 -- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42680d0f60 msgr2=0x7f42680aa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f42677fe640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42680d0f60 0x7f42680aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f4267fff640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f426815dc10 0x7f42680ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f42677fe640 1 -- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f426815dc10 msgr2=0x7f42680ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f42677fe640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f426815dc10 0x7f42680ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f42677fe640 1 -- 172.21.15.38:0/3100088235 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42680aadb0 con 0x7f42680ced60 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.222+0000 7f4267fff640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f426815dc10 0x7f42680ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.223+0000 7f42677fe640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42680ced60 0x7f4268152f90 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f425802a220 tx=0x7f42580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.223+0000 7f424ffff640 1 -- 172.21.15.38:0/3100088235 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258037040 con 0x7f42680ced60 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.223+0000 7f424ffff640 1 -- 172.21.15.38:0/3100088235 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4258042070 con 0x7f42680ced60 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.223+0000 7f426db22640 1 -- 172.21.15.38:0/3100088235 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42680ab090 con 0x7f42680ced60 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.223+0000 7f424ffff640 1 -- 172.21.15.38:0/3100088235 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258030880 con 0x7f42680ced60 2024-04-25T08:40:40.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.223+0000 7f426db22640 1 -- 172.21.15.38:0/3100088235 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42680ab650 con 0x7f42680ced60 2024-04-25T08:40:40.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.224+0000 7f426db22640 1 -- 172.21.15.38:0/3100088235 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4268068e80 con 0x7f42680ced60 2024-04-25T08:40:40.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.224+0000 7f424ffff640 1 -- 172.21.15.38:0/3100088235 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4258030a20 con 0x7f42680ced60 2024-04-25T08:40:40.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.225+0000 7f424ffff640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f425406b4d0 0x7f425406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.225+0000 7f424ffff640 1 -- 172.21.15.38:0/3100088235 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f42580aac30 con 0x7f42680ced60 2024-04-25T08:40:40.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.225+0000 7f4266ffd640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f425406b4d0 0x7f425406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.225+0000 7f4266ffd640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f425406b4d0 0x7f425406d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f425c000f60 tx=0x7f425c0079b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:40.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.229+0000 7f424ffff640 1 -- 172.21.15.38:0/3100088235 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4258079920 con 0x7f42680ced60 2024-04-25T08:40:40.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.449+0000 7f426db22640 1 -- 172.21.15.38:0/3100088235 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4268160a00 con 0x7f42680ced60 2024-04-25T08:40:40.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.449+0000 7f424ffff640 1 -- 172.21.15.38:0/3100088235 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4258079220 con 0x7f42680ced60 2024-04-25T08:40:40.443 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:40.443 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:40.443 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:40.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.451+0000 7f426db22640 1 -- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f425406b4d0 msgr2=0x7f425406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.451+0000 7f426db22640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f425406b4d0 0x7f425406d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f425c000f60 tx=0x7f425c0079b0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.451+0000 7f426db22640 1 -- 172.21.15.38:0/3100088235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42680ced60 msgr2=0x7f4268152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.451+0000 7f426db22640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42680ced60 0x7f4268152f90 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f425802a220 tx=0x7f42580046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.451+0000 7f426db22640 1 -- 172.21.15.38:0/3100088235 shutdown_connections 2024-04-25T08:40:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.451+0000 7f426db22640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f426815dc10 0x7f42680ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.451+0000 7f426db22640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f425406b4d0 0x7f425406d990 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.451+0000 7f426db22640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42680d0f60 0x7f42680aa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.451+0000 7f426db22640 1 --2- 172.21.15.38:0/3100088235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42680ced60 0x7f4268152f90 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.451+0000 7f426db22640 1 -- 172.21.15.38:0/3100088235 >> 172.21.15.38:0/3100088235 conn(0x7f4268066d10 msgr2=0x7f42680d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.451+0000 7f426db22640 1 -- 172.21.15.38:0/3100088235 shutdown_connections 2024-04-25T08:40:40.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.451+0000 7f426db22640 1 -- 172.21.15.38:0/3100088235 wait complete. 2024-04-25T08:40:40.456 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:40:40.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.564+0000 7fbec91e0640 1 Processor -- start 2024-04-25T08:40:40.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.564+0000 7fbec91e0640 1 -- start start 2024-04-25T08:40:40.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec91e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbec4144760 0x7fbec415faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec91e0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbec40cea70 0x7fbec40b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec91e0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec40b50c0 con 0x7fbec4160030 2024-04-25T08:40:40.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec91e0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec415f380 con 0x7fbec40cea70 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec91e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec415f660 con 0x7fbec4144760 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec259c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbec4160030 0x7fbec40ce360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53378/0 (socket says 172.21.15.38:53378) 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec259c640 1 -- 172.21.15.38:0/3992360220 learned_addr learned my addr 172.21.15.38:0/3992360220 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec359e640 1 --2- 172.21.15.38:0/3992360220 >> v2:172.21.15.107:3300/0 conn(0x7fbec40cea70 0x7fbec40b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3660158766 0 0) 0x7fbec40b50c0 con 0x7fbec4160030 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbeac003520 con 0x7fbec4160030 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec2d9d640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbec4144760 0x7fbec415faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3003927674 0 0) 0x7fbeac003520 con 0x7fbec4160030 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbec4144760 msgr2=0x7fbec415faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec1d9b640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbec4144760 0x7fbec415faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 >> v2:172.21.15.107:3300/0 conn(0x7fbec40cea70 msgr2=0x7fbec40b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec1d9b640 1 --2- 172.21.15.38:0/3992360220 >> v2:172.21.15.107:3300/0 conn(0x7fbec40cea70 0x7fbec40b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.565+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec40b53a0 con 0x7fbec4160030 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fbeb8001670 con 0x7fbec4160030 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 >> v1:172.21.15.38:6789/0 conn(0x7fbec4160030 legacy=0x7fbec40ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec1d9b640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbeac0039b0 0x7fbeac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec1d9b640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeac004800 0x7fbeac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec1d9b640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbeac0051c0 0x7fbeac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec415f380 con 0x7fbeac0039b0 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec415f660 con 0x7fbeac004800 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeac00bf00 con 0x7fbeac0051c0 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec2d9d640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbeac0039b0 0x7fbeac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec359e640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbeac0051c0 0x7fbeac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec259c640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeac004800 0x7fbeac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec2d9d640 1 -- 172.21.15.38:0/3992360220 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbeac0051c0 msgr2=0x7fbeac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:40.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec2d9d640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbeac0051c0 0x7fbeac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec2d9d640 1 -- 172.21.15.38:0/3992360220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeac004800 msgr2=0x7fbeac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec2d9d640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeac004800 0x7fbeac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec2d9d640 1 -- 172.21.15.38:0/3992360220 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac003520 con 0x7fbeac0039b0 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.566+0000 7fbec359e640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbeac0051c0 0x7fbeac00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.567+0000 7fbec2d9d640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbeac0039b0 0x7fbeac003dd0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009ac0 tx=0x7fbeac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66dd4054f409059 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.567+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc036070 con 0x7fbeac0039b0 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.567+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc031400 con 0x7fbeac0039b0 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.567+0000 7fbec1d9b640 1 -- 172.21.15.38:0/3992360220 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc009690 con 0x7fbeac0039b0 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.567+0000 7fbec91e0640 1 -- 172.21.15.38:0/3992360220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbeac0039b0 msgr2=0x7fbeac003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.567+0000 7fbec91e0640 1 --2- 172.21.15.38:0/3992360220 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbeac0039b0 0x7fbeac003dd0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009ac0 tx=0x7fbeac005830 comp rx=0 tx=0).stop 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.567+0000 7fbec359e640 1 -- 172.21.15.38:0/3992360220 reap_dead start 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.567+0000 7fbec91e0640 1 -- 172.21.15.38:0/3992360220 shutdown_connections 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.567+0000 7fbec91e0640 1 -- 172.21.15.38:0/3992360220 >> 172.21.15.38:0/3992360220 conn(0x7fbec406b210 msgr2=0x7fbec406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.567+0000 7fbec91e0640 1 -- 172.21.15.38:0/3992360220 shutdown_connections 2024-04-25T08:40:40.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.567+0000 7fbec91e0640 1 -- 172.21.15.38:0/3992360220 wait complete. 2024-04-25T08:40:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.569+0000 7fbec91e0640 1 Processor -- start 2024-04-25T08:40:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.569+0000 7fbec91e0640 1 -- start start 2024-04-25T08:40:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec91e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeac0039b0 0x7fbec413e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec91e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec40c9f20 0x7fbec40c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec91e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbec41f12b0 0x7fbec41f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec91e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4145400 con 0x7fbec40c9f20 2024-04-25T08:40:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec91e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4145280 con 0x7fbeac0039b0 2024-04-25T08:40:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec91e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec40b4b50 con 0x7fbec41f12b0 2024-04-25T08:40:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec40c9f20 0x7fbec40c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec40c9f20 0x7fbec40c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39358/0 (socket says 172.21.15.38:39358) 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec259c640 1 -- 172.21.15.38:0/2758378051 learned_addr learned my addr 172.21.15.38:0/2758378051 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec2d9d640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeac0039b0 0x7fbec413e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec359e640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbec41f12b0 0x7fbec41f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec259c640 1 -- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbec41f12b0 msgr2=0x7fbec41f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec259c640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbec41f12b0 0x7fbec41f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec259c640 1 -- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeac0039b0 msgr2=0x7fbec413e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec259c640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeac0039b0 0x7fbec413e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec259c640 1 -- 172.21.15.38:0/2758378051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec40c8920 con 0x7fbec40c9f20 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec259c640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec40c9f20 0x7fbec40c83e0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8007310 tx=0x7fbeb80077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbeab7fe640 1 -- 172.21.15.38:0/2758378051 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800f9c0 con 0x7fbec40c9f20 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbeab7fe640 1 -- 172.21.15.38:0/2758378051 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb8018070 con 0x7fbec40c9f20 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.570+0000 7fbec91e0640 1 -- 172.21.15.38:0/2758378051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec40ca380 con 0x7fbec40c9f20 2024-04-25T08:40:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.571+0000 7fbeab7fe640 1 -- 172.21.15.38:0/2758378051 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb800fdc0 con 0x7fbec40c9f20 2024-04-25T08:40:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.571+0000 7fbec91e0640 1 -- 172.21.15.38:0/2758378051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec4140b90 con 0x7fbec40c9f20 2024-04-25T08:40:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.571+0000 7fbec91e0640 1 -- 172.21.15.38:0/2758378051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec4144760 con 0x7fbec40c9f20 2024-04-25T08:40:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.572+0000 7fbeab7fe640 1 -- 172.21.15.38:0/2758378051 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbeb801d460 con 0x7fbec40c9f20 2024-04-25T08:40:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.572+0000 7fbeab7fe640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbeb006b4d0 0x7fbeb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.573+0000 7fbeab7fe640 1 -- 172.21.15.38:0/2758378051 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbeb808a2b0 con 0x7fbec40c9f20 2024-04-25T08:40:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.573+0000 7fbec2d9d640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbeb006b4d0 0x7fbeb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.573+0000 7fbec2d9d640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbeb006b4d0 0x7fbeb006d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbebc033000 tx=0x7fbebc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:40.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.577+0000 7fbeab7fe640 1 -- 172.21.15.38:0/2758378051 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbec4144760 con 0x7fbec40c9f20 2024-04-25T08:40:40.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.758+0000 7fbec91e0640 1 -- 172.21.15.38:0/2758378051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbec40c9620 con 0x7fbec40c9f20 2024-04-25T08:40:40.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.759+0000 7fbeab7fe640 1 -- 172.21.15.38:0/2758378051 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16940 (secure 0 0 0) 0x7fbeb8058940 con 0x7fbec40c9f20 2024-04-25T08:40:40.752 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:40.752 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":121,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:59.121832+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:55.759766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:56.451485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:40:40.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.761+0000 7fbec91e0640 1 -- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbeb006b4d0 msgr2=0x7fbeb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.761+0000 7fbec91e0640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbeb006b4d0 0x7fbeb006d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbebc033000 tx=0x7fbebc008030 comp rx=0 tx=0).stop 2024-04-25T08:40:40.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.761+0000 7fbec91e0640 1 -- 172.21.15.38:0/2758378051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec40c9f20 msgr2=0x7fbec40c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.761+0000 7fbec91e0640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec40c9f20 0x7fbec40c83e0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8007310 tx=0x7fbeb80077d0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.761+0000 7fbec91e0640 1 -- 172.21.15.38:0/2758378051 shutdown_connections 2024-04-25T08:40:40.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.761+0000 7fbec91e0640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbeb006b4d0 0x7fbeb006d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.761+0000 7fbec91e0640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbec41f12b0 0x7fbec41f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.761+0000 7fbec91e0640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbec40c9f20 0x7fbec40c83e0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.761+0000 7fbec91e0640 1 --2- 172.21.15.38:0/2758378051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbeac0039b0 0x7fbec413e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.761+0000 7fbec91e0640 1 -- 172.21.15.38:0/2758378051 >> 172.21.15.38:0/2758378051 conn(0x7fbec406b210 msgr2=0x7fbec40b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:40.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.761+0000 7fbec91e0640 1 -- 172.21.15.38:0/2758378051 shutdown_connections 2024-04-25T08:40:40.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.761+0000 7fbec91e0640 1 -- 172.21.15.38:0/2758378051 wait complete. 2024-04-25T08:40:40.765 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:40:40.766 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:40.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.874+0000 7fd496abd640 1 Processor -- start 2024-04-25T08:40:40.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.874+0000 7fd496abd640 1 -- start start 2024-04-25T08:40:40.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.874+0000 7fd496abd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd49015dc10 0x7fd4900ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.875+0000 7fd496abd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd4900d0f60 0x7fd4900b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.875+0000 7fd496abd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4900d3e70 con 0x7fd4900ced60 2024-04-25T08:40:40.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.875+0000 7fd496abd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49015dff0 con 0x7fd4900d0f60 2024-04-25T08:40:40.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.875+0000 7fd496abd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900b6990 con 0x7fd49015dc10 2024-04-25T08:40:40.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.875+0000 7fd49505a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd4900d0f60 0x7fd4900b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.875+0000 7fd49505a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd4900d0f60 0x7fd4900b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39362/0 (socket says 172.21.15.38:39362) 2024-04-25T08:40:40.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.875+0000 7fd49505a640 1 -- 172.21.15.38:0/1160913727 learned_addr learned my addr 172.21.15.38:0/1160913727 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:40.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.875+0000 7fd494859640 1 --2- 172.21.15.38:0/1160913727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd49015dc10 0x7fd4900ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.875+0000 7fd49505a640 1 -- 172.21.15.38:0/1160913727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd49015dc10 msgr2=0x7fd4900ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:40.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.875+0000 7fd49505a640 1 --2- 172.21.15.38:0/1160913727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd49015dc10 0x7fd4900ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.875+0000 7fd49505a640 1 -- 172.21.15.38:0/1160913727 >> v1:172.21.15.107:6789/0 conn(0x7fd4900ced60 legacy=0x7fd4900d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.875+0000 7fd49505a640 1 -- 172.21.15.38:0/1160913727 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4900d4150 con 0x7fd4900d0f60 2024-04-25T08:40:40.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.876+0000 7fd49505a640 1 --2- 172.21.15.38:0/1160913727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4900d0f60 0x7fd4900b2370 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd480009870 tx=0x7fd48002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f9f097965fc8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:40.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.876+0000 7fd48f7fe640 1 -- 172.21.15.38:0/1160913727 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480037070 con 0x7fd4900d0f60 2024-04-25T08:40:40.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.876+0000 7fd48f7fe640 1 -- 172.21.15.38:0/1160913727 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd480032440 con 0x7fd4900d0f60 2024-04-25T08:40:40.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.876+0000 7fd48f7fe640 1 -- 172.21.15.38:0/1160913727 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480032810 con 0x7fd4900d0f60 2024-04-25T08:40:40.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.876+0000 7fd496abd640 1 -- 172.21.15.38:0/1160913727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4900d0f60 msgr2=0x7fd4900b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.876+0000 7fd496abd640 1 --2- 172.21.15.38:0/1160913727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4900d0f60 0x7fd4900b2370 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fd480009870 tx=0x7fd48002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.876+0000 7fd496abd640 1 -- 172.21.15.38:0/1160913727 shutdown_connections 2024-04-25T08:40:40.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.876+0000 7fd496abd640 1 --2- 172.21.15.38:0/1160913727 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4900d0f60 0x7fd4900b2370 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.876+0000 7fd496abd640 1 --2- 172.21.15.38:0/1160913727 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd49015dc10 0x7fd4900ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.876+0000 7fd496abd640 1 -- 172.21.15.38:0/1160913727 >> 172.21.15.38:0/1160913727 conn(0x7fd490066d10 msgr2=0x7fd490067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:40.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.876+0000 7fd496abd640 1 -- 172.21.15.38:0/1160913727 shutdown_connections 2024-04-25T08:40:40.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.877+0000 7fd496abd640 1 -- 172.21.15.38:0/1160913727 wait complete. 2024-04-25T08:40:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.878+0000 7fd496abd640 1 Processor -- start 2024-04-25T08:40:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.878+0000 7fd496abd640 1 -- start start 2024-04-25T08:40:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd496abd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4900ced60 0x7fd4901586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd496abd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4900d0f60 0x7fd49015b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd496abd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd49015dc10 0x7fd490157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd496abd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900b6bc0 con 0x7fd4900ced60 2024-04-25T08:40:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd496abd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900d4710 con 0x7fd4900d0f60 2024-04-25T08:40:40.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd496abd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900d3900 con 0x7fd49015dc10 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd494859640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4900ced60 0x7fd4901586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd494859640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4900ced60 0x7fd4901586e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39368/0 (socket says 172.21.15.38:39368) 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd494859640 1 -- 172.21.15.38:0/2473895450 learned_addr learned my addr 172.21.15.38:0/2473895450 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd48ffff640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4900d0f60 0x7fd49015b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd49505a640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd49015dc10 0x7fd490157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd494859640 1 -- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd49015dc10 msgr2=0x7fd490157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd494859640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd49015dc10 0x7fd490157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd494859640 1 -- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4900d0f60 msgr2=0x7fd49015b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd494859640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4900d0f60 0x7fd49015b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd494859640 1 -- 172.21.15.38:0/2473895450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd490159280 con 0x7fd4900ced60 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd48ffff640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4900d0f60 0x7fd49015b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd49505a640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd49015dc10 0x7fd490157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd494859640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4900ced60 0x7fd4901586e0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fd488009d10 tx=0x7fd48800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd48d7fa640 1 -- 172.21.15.38:0/2473895450 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488018020 con 0x7fd4900ced60 2024-04-25T08:40:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd48d7fa640 1 -- 172.21.15.38:0/2473895450 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd488014480 con 0x7fd4900ced60 2024-04-25T08:40:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd48d7fa640 1 -- 172.21.15.38:0/2473895450 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488014830 con 0x7fd4900ced60 2024-04-25T08:40:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd496abd640 1 -- 172.21.15.38:0/2473895450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd490158cc0 con 0x7fd4900ced60 2024-04-25T08:40:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.879+0000 7fd496abd640 1 -- 172.21.15.38:0/2473895450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4900c75b0 con 0x7fd4900ced60 2024-04-25T08:40:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.880+0000 7fd496abd640 1 -- 172.21.15.38:0/2473895450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd490068e80 con 0x7fd4900ced60 2024-04-25T08:40:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.881+0000 7fd48d7fa640 1 -- 172.21.15.38:0/2473895450 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd488005d10 con 0x7fd4900ced60 2024-04-25T08:40:40.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.881+0000 7fd48d7fa640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd47006b4d0 0x7fd47006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:40.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.882+0000 7fd48d7fa640 1 -- 172.21.15.38:0/2473895450 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd48808b170 con 0x7fd4900ced60 2024-04-25T08:40:40.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.882+0000 7fd48ffff640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd47006b4d0 0x7fd47006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:40.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.882+0000 7fd48ffff640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd47006b4d0 0x7fd47006d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd49015f0c0 tx=0x7fd478009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:40.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:40.886+0000 7fd48d7fa640 1 -- 172.21.15.38:0/2473895450 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd488022030 con 0x7fd4900ced60 2024-04-25T08:40:41.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.107+0000 7fd496abd640 1 -- 172.21.15.38:0/2473895450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd49015b8b0 con 0x7fd4900ced60 2024-04-25T08:40:41.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.107+0000 7fd48d7fa640 1 -- 172.21.15.38:0/2473895450 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd488010040 con 0x7fd4900ced60 2024-04-25T08:40:41.100 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:41.100 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:41.100 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:41.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.109+0000 7fd496abd640 1 -- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd47006b4d0 msgr2=0x7fd47006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.109+0000 7fd496abd640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd47006b4d0 0x7fd47006d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fd49015f0c0 tx=0x7fd478009040 comp rx=0 tx=0).stop 2024-04-25T08:40:41.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.109+0000 7fd496abd640 1 -- 172.21.15.38:0/2473895450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4900ced60 msgr2=0x7fd4901586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.109+0000 7fd496abd640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4900ced60 0x7fd4901586e0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fd488009d10 tx=0x7fd48800c490 comp rx=0 tx=0).stop 2024-04-25T08:40:41.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.109+0000 7fd496abd640 1 -- 172.21.15.38:0/2473895450 shutdown_connections 2024-04-25T08:40:41.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.109+0000 7fd496abd640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd47006b4d0 0x7fd47006d990 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.109+0000 7fd496abd640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd49015dc10 0x7fd490157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.109+0000 7fd496abd640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd4900d0f60 0x7fd49015b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.109+0000 7fd496abd640 1 --2- 172.21.15.38:0/2473895450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4900ced60 0x7fd4901586e0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.109+0000 7fd496abd640 1 -- 172.21.15.38:0/2473895450 >> 172.21.15.38:0/2473895450 conn(0x7fd490066d10 msgr2=0x7fd4901603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:41.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.109+0000 7fd496abd640 1 -- 172.21.15.38:0/2473895450 shutdown_connections 2024-04-25T08:40:41.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.109+0000 7fd496abd640 1 -- 172.21.15.38:0/2473895450 wait complete. 2024-04-25T08:40:41.114 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:41.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.222+0000 7f2df4db0640 1 Processor -- start 2024-04-25T08:40:41.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.222+0000 7f2df4db0640 1 -- start start 2024-04-25T08:40:41.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.223+0000 7f2df4db0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2df015dc10 0x7f2df00ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.223+0000 7f2df4db0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2df00d0f60 0x7f2df00b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.223+0000 7f2df4db0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df00d3e70 con 0x7f2df00ced60 2024-04-25T08:40:41.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.223+0000 7f2df4db0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df015dff0 con 0x7f2df015dc10 2024-04-25T08:40:41.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.223+0000 7f2df4db0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00b6990 con 0x7f2df00d0f60 2024-04-25T08:40:41.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.224+0000 7f2deed9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2df00d0f60 0x7f2df00b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.224+0000 7f2dee59c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2df015dc10 0x7f2df00ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.224+0000 7f2dee59c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2df015dc10 0x7f2df00ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:53072/0 (socket says 172.21.15.38:53072) 2024-04-25T08:40:41.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.224+0000 7f2dee59c640 1 -- 172.21.15.38:0/3283689890 learned_addr learned my addr 172.21.15.38:0/3283689890 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:41.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.224+0000 7f2dedd9b640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f2df00ced60 0x7f2df00d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:44398/0 (socket says 172.21.15.38:44398) 2024-04-25T08:40:41.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.224+0000 7f2dee59c640 1 -- 172.21.15.38:0/3283689890 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2df00d0f60 msgr2=0x7f2df00b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:41.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.224+0000 7f2dee59c640 1 --2- 172.21.15.38:0/3283689890 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2df00d0f60 0x7f2df00b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.224+0000 7f2dee59c640 1 -- 172.21.15.38:0/3283689890 >> v1:172.21.15.107:6789/0 conn(0x7f2df00ced60 legacy=0x7f2df00d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.224+0000 7f2dee59c640 1 -- 172.21.15.38:0/3283689890 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df00d4150 con 0x7f2df015dc10 2024-04-25T08:40:41.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.224+0000 7f2ded59a640 1 -- 172.21.15.38:0/3283689890 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112705392 0 0) 0x7f2df00d3e70 con 0x7f2df00ced60 2024-04-25T08:40:41.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.225+0000 7f2dee59c640 1 --2- 172.21.15.38:0/3283689890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df015dc10 0x7f2df00ce820 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009870 tx=0x7f2de802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4ad27c0d568074f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:41.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.225+0000 7f2ded59a640 1 -- 172.21.15.38:0/3283689890 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8037070 con 0x7f2df015dc10 2024-04-25T08:40:41.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.225+0000 7f2ded59a640 1 -- 172.21.15.38:0/3283689890 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de8032440 con 0x7f2df015dc10 2024-04-25T08:40:41.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.225+0000 7f2ded59a640 1 -- 172.21.15.38:0/3283689890 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8032810 con 0x7f2df015dc10 2024-04-25T08:40:41.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.226+0000 7f2df4db0640 1 -- 172.21.15.38:0/3283689890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df015dc10 msgr2=0x7f2df00ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.226+0000 7f2df4db0640 1 --2- 172.21.15.38:0/3283689890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df015dc10 0x7f2df00ce820 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009870 tx=0x7f2de802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.226+0000 7f2df4db0640 1 -- 172.21.15.38:0/3283689890 shutdown_connections 2024-04-25T08:40:41.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.226+0000 7f2df4db0640 1 --2- 172.21.15.38:0/3283689890 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2df00d0f60 0x7f2df00b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.226+0000 7f2df4db0640 1 --2- 172.21.15.38:0/3283689890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df015dc10 0x7f2df00ce820 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.226+0000 7f2df4db0640 1 -- 172.21.15.38:0/3283689890 >> 172.21.15.38:0/3283689890 conn(0x7f2df0066d10 msgr2=0x7f2df0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:41.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.226+0000 7f2df4db0640 1 -- 172.21.15.38:0/3283689890 shutdown_connections 2024-04-25T08:40:41.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.226+0000 7f2df4db0640 1 -- 172.21.15.38:0/3283689890 wait complete. 2024-04-25T08:40:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.227+0000 7f2df4db0640 1 Processor -- start 2024-04-25T08:40:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.227+0000 7f2df4db0640 1 -- start start 2024-04-25T08:40:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2df4db0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2df00ced60 0x7f2df00abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2df4db0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df00d0f60 0x7f2df00ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2df4db0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df015dc10 0x7f2df00b8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2df4db0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00b6bc0 con 0x7f2df015dc10 2024-04-25T08:40:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2df4db0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00d4710 con 0x7f2df00d0f60 2024-04-25T08:40:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2df4db0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df00d3900 con 0x7f2df00ced60 2024-04-25T08:40:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2deed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df015dc10 0x7f2df00b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2dee59c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2df00ced60 0x7f2df00abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2deed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df015dc10 0x7f2df00b8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39378/0 (socket says 172.21.15.38:39378) 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2dedd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df00d0f60 0x7f2df00ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2deed9d640 1 -- 172.21.15.38:0/2415806128 learned_addr learned my addr 172.21.15.38:0/2415806128 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2dee59c640 1 -- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df00d0f60 msgr2=0x7f2df00ac1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2dee59c640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df00d0f60 0x7f2df00ac1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2dee59c640 1 -- 172.21.15.38:0/2415806128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df015dc10 msgr2=0x7f2df00b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2dee59c640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df015dc10 0x7f2df00b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2dee59c640 1 -- 172.21.15.38:0/2415806128 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df01f3720 con 0x7f2df00ced60 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.228+0000 7f2deed9d640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df015dc10 0x7f2df00b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.229+0000 7f2dee59c640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2df00ced60 0x7f2df00abc80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2de8031000 tx=0x7f2de80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.229+0000 7f2de6ffd640 1 -- 172.21.15.38:0/2415806128 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8041020 con 0x7f2df00ced60 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.229+0000 7f2de6ffd640 1 -- 172.21.15.38:0/2415806128 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de8030640 con 0x7f2df00ced60 2024-04-25T08:40:41.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.229+0000 7f2df4db0640 1 -- 172.21.15.38:0/2415806128 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df01f3a00 con 0x7f2df00ced60 2024-04-25T08:40:41.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.229+0000 7f2de6ffd640 1 -- 172.21.15.38:0/2415806128 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8037040 con 0x7f2df00ced60 2024-04-25T08:40:41.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.229+0000 7f2df4db0640 1 -- 172.21.15.38:0/2415806128 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df01f3fc0 con 0x7f2df00ced60 2024-04-25T08:40:41.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.230+0000 7f2df4db0640 1 -- 172.21.15.38:0/2415806128 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df0068e80 con 0x7f2df00ced60 2024-04-25T08:40:41.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.230+0000 7f2de6ffd640 1 -- 172.21.15.38:0/2415806128 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2de8044020 con 0x7f2df00ced60 2024-04-25T08:40:41.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.231+0000 7f2de6ffd640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2dd006b4d0 0x7f2dd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.231+0000 7f2de6ffd640 1 -- 172.21.15.38:0/2415806128 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2de80aa950 con 0x7f2df00ced60 2024-04-25T08:40:41.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.231+0000 7f2dedd9b640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2dd006b4d0 0x7f2dd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.231+0000 7f2dedd9b640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2dd006b4d0 0x7f2dd006d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2df00c6820 tx=0x7f2dd8007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:41.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.236+0000 7f2de6ffd640 1 -- 172.21.15.38:0/2415806128 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2de804b030 con 0x7f2df00ced60 2024-04-25T08:40:41.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.453+0000 7f2df4db0640 1 -- 172.21.15.38:0/2415806128 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2df00cda00 con 0x7f2df00ced60 2024-04-25T08:40:41.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.454+0000 7f2de6ffd640 1 -- 172.21.15.38:0/2415806128 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2de8078f60 con 0x7f2df00ced60 2024-04-25T08:40:41.447 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:41.448 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:39:57.475923+0000","modified":"2024-04-25T08:39:59.471310+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7639},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7639":{"gid":7639,"name":"d","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.107:6837/901382672","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":901382672},{"type":"v1","addr":"172.21.15.107:6837","nonce":901382672}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T08:40:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 70 2024-04-25T08:40:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.457+0000 7f2df4db0640 1 -- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2dd006b4d0 msgr2=0x7f2dd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.457+0000 7f2df4db0640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2dd006b4d0 0x7f2dd006d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2df00c6820 tx=0x7f2dd8007d50 comp rx=0 tx=0).stop 2024-04-25T08:40:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.457+0000 7f2df4db0640 1 -- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2df00ced60 msgr2=0x7f2df00abc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.457+0000 7f2df4db0640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2df00ced60 0x7f2df00abc80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2de8031000 tx=0x7f2de80046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.457+0000 7f2df4db0640 1 -- 172.21.15.38:0/2415806128 shutdown_connections 2024-04-25T08:40:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.457+0000 7f2df4db0640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df015dc10 0x7f2df00b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.457+0000 7f2df4db0640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2df00d0f60 0x7f2df00ac1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.457+0000 7f2df4db0640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2dd006b4d0 0x7f2dd006d990 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.457+0000 7f2df4db0640 1 --2- 172.21.15.38:0/2415806128 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2df00ced60 0x7f2df00abc80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.457+0000 7f2df4db0640 1 -- 172.21.15.38:0/2415806128 >> 172.21.15.38:0/2415806128 conn(0x7f2df0066d10 msgr2=0x7f2df0160550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.457+0000 7f2df4db0640 1 -- 172.21.15.38:0/2415806128 shutdown_connections 2024-04-25T08:40:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.457+0000 7f2df4db0640 1 -- 172.21.15.38:0/2415806128 wait complete. 2024-04-25T08:40:41.461 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:40:41.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.569+0000 7ff7973b7640 1 Processor -- start 2024-04-25T08:40:41.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.569+0000 7ff7973b7640 1 -- start start 2024-04-25T08:40:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff7973b7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff79015dc40 0x7ff7900d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff7973b7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff7900b9580 0x7ff7900d0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff7973b7640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff7900d1e40 0x7ff7900d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff7973b7640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7900d19b0 con 0x7ff7900b9580 2024-04-25T08:40:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff7973b7640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7900b8400 con 0x7ff7900d1e40 2024-04-25T08:40:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff7973b7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900b86e0 con 0x7ff79015dc40 2024-04-25T08:40:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff794952640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff7900b9580 0x7ff7900d0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff794952640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff7900b9580 0x7ff7900d0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39392/0 (socket says 172.21.15.38:39392) 2024-04-25T08:40:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff794952640 1 -- 172.21.15.38:0/1964677385 learned_addr learned my addr 172.21.15.38:0/1964677385 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff795954640 1 --2- 172.21.15.38:0/1964677385 >> v2:172.21.15.107:3300/0 conn(0x7ff7900d1e40 0x7ff7900d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff795153640 1 --2- 172.21.15.38:0/1964677385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff79015dc40 0x7ff7900d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff794952640 1 -- 172.21.15.38:0/1964677385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff79015dc40 msgr2=0x7ff7900d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff794952640 1 --2- 172.21.15.38:0/1964677385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff79015dc40 0x7ff7900d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff794952640 1 -- 172.21.15.38:0/1964677385 >> v2:172.21.15.107:3300/0 conn(0x7ff7900d1e40 msgr2=0x7ff7900d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff794952640 1 --2- 172.21.15.38:0/1964677385 >> v2:172.21.15.107:3300/0 conn(0x7ff7900d1e40 0x7ff7900d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff794952640 1 -- 172.21.15.38:0/1964677385 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7900b89c0 con 0x7ff7900b9580 2024-04-25T08:40:41.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.570+0000 7ff795954640 1 --2- 172.21.15.38:0/1964677385 >> v2:172.21.15.107:3300/0 conn(0x7ff7900d1e40 0x7ff7900d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.571+0000 7ff794952640 1 --2- 172.21.15.38:0/1964677385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7900b9580 0x7ff7900d0e10 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff7900d4990 tx=0x7ff78002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f55cc91b67da310 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:41.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.571+0000 7ff787fff640 1 -- 172.21.15.38:0/1964677385 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780037070 con 0x7ff7900b9580 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.571+0000 7ff787fff640 1 -- 172.21.15.38:0/1964677385 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff780032440 con 0x7ff7900b9580 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.571+0000 7ff787fff640 1 -- 172.21.15.38:0/1964677385 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780032810 con 0x7ff7900b9580 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.571+0000 7ff7973b7640 1 -- 172.21.15.38:0/1964677385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7900b9580 msgr2=0x7ff7900d0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.571+0000 7ff7973b7640 1 --2- 172.21.15.38:0/1964677385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7900b9580 0x7ff7900d0e10 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff7900d4990 tx=0x7ff78002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.572+0000 7ff7973b7640 1 -- 172.21.15.38:0/1964677385 shutdown_connections 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.572+0000 7ff7973b7640 1 --2- 172.21.15.38:0/1964677385 >> v2:172.21.15.107:3300/0 conn(0x7ff7900d1e40 0x7ff7900d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.572+0000 7ff7973b7640 1 --2- 172.21.15.38:0/1964677385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7900b9580 0x7ff7900d0e10 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.572+0000 7ff7973b7640 1 --2- 172.21.15.38:0/1964677385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff79015dc40 0x7ff7900d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.572+0000 7ff7973b7640 1 -- 172.21.15.38:0/1964677385 >> 172.21.15.38:0/1964677385 conn(0x7ff790066ce0 msgr2=0x7ff7900670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.572+0000 7ff7973b7640 1 -- 172.21.15.38:0/1964677385 shutdown_connections 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.572+0000 7ff7973b7640 1 -- 172.21.15.38:0/1964677385 wait complete. 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.573+0000 7ff7973b7640 1 Processor -- start 2024-04-25T08:40:41.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.573+0000 7ff7973b7640 1 -- start start 2024-04-25T08:40:41.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff7973b7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7900b9580 0x7ff79015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff7973b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7900d1e40 0x7ff79014b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff7973b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79015dc40 0x7ff790160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff7973b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900d2950 con 0x7ff7900d1e40 2024-04-25T08:40:41.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff7973b7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900d27d0 con 0x7ff79015dc40 2024-04-25T08:40:41.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff7973b7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900d2ad0 con 0x7ff7900b9580 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff794952640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7900d1e40 0x7ff79014b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff794952640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7900d1e40 0x7ff79014b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39396/0 (socket says 172.21.15.38:39396) 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff794952640 1 -- 172.21.15.38:0/361079001 learned_addr learned my addr 172.21.15.38:0/361079001 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff795954640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79015dc40 0x7ff790160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff795153640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7900b9580 0x7ff79015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff794952640 1 -- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7900b9580 msgr2=0x7ff79015afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff794952640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7900b9580 0x7ff79015afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff794952640 1 -- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79015dc40 msgr2=0x7ff790160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff794952640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79015dc40 0x7ff790160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff794952640 1 -- 172.21.15.38:0/361079001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79015f5b0 con 0x7ff7900d1e40 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff794952640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7900d1e40 0x7ff79014b820 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff78002a220 tx=0x7ff7800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff785ffb640 1 -- 172.21.15.38:0/361079001 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780041020 con 0x7ff7900d1e40 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff785ffb640 1 -- 172.21.15.38:0/361079001 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7800306c0 con 0x7ff7900d1e40 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff7973b7640 1 -- 172.21.15.38:0/361079001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79015f890 con 0x7ff7900d1e40 2024-04-25T08:40:41.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.574+0000 7ff785ffb640 1 -- 172.21.15.38:0/361079001 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780037040 con 0x7ff7900d1e40 2024-04-25T08:40:41.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.575+0000 7ff7973b7640 1 -- 172.21.15.38:0/361079001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79015fe50 con 0x7ff7900d1e40 2024-04-25T08:40:41.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.575+0000 7ff7973b7640 1 -- 172.21.15.38:0/361079001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff790068e80 con 0x7ff7900d1e40 2024-04-25T08:40:41.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.576+0000 7ff785ffb640 1 -- 172.21.15.38:0/361079001 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff780044020 con 0x7ff7900d1e40 2024-04-25T08:40:41.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.576+0000 7ff785ffb640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff77406b500 0x7ff77406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.577+0000 7ff785ffb640 1 -- 172.21.15.38:0/361079001 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff7800aaa40 con 0x7ff7900d1e40 2024-04-25T08:40:41.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.577+0000 7ff795153640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff77406b500 0x7ff77406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.577+0000 7ff795153640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff77406b500 0x7ff77406d9c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0079a0 tx=0x7ff78c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:41.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.581+0000 7ff785ffb640 1 -- 172.21.15.38:0/361079001 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff78004b030 con 0x7ff7900d1e40 2024-04-25T08:40:41.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.760+0000 7ff7973b7640 1 -- 172.21.15.38:0/361079001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff79015d310 con 0x7ff7900d1e40 2024-04-25T08:40:41.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.761+0000 7ff785ffb640 1 -- 172.21.15.38:0/361079001 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16940 (secure 0 0 0) 0x7ff7800aae20 con 0x7ff7900d1e40 2024-04-25T08:40:41.754 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:41.754 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":121,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:39:59.121832+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:39:55.759766+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T08:39:56.451485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:40:41.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.763+0000 7ff7973b7640 1 -- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff77406b500 msgr2=0x7ff77406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.763+0000 7ff7973b7640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff77406b500 0x7ff77406d9c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0079a0 tx=0x7ff78c008040 comp rx=0 tx=0).stop 2024-04-25T08:40:41.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.763+0000 7ff7973b7640 1 -- 172.21.15.38:0/361079001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7900d1e40 msgr2=0x7ff79014b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.763+0000 7ff7973b7640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7900d1e40 0x7ff79014b820 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff78002a220 tx=0x7ff7800046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.763+0000 7ff7973b7640 1 -- 172.21.15.38:0/361079001 shutdown_connections 2024-04-25T08:40:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.763+0000 7ff7973b7640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79015dc40 0x7ff790160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.763+0000 7ff7973b7640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff7900d1e40 0x7ff79014b820 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.763+0000 7ff7973b7640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff77406b500 0x7ff77406d9c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.763+0000 7ff7973b7640 1 --2- 172.21.15.38:0/361079001 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7900b9580 0x7ff79015afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.763+0000 7ff7973b7640 1 -- 172.21.15.38:0/361079001 >> 172.21.15.38:0/361079001 conn(0x7ff790066ce0 msgr2=0x7ff7900b4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.763+0000 7ff7973b7640 1 -- 172.21.15.38:0/361079001 shutdown_connections 2024-04-25T08:40:41.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.763+0000 7ff7973b7640 1 -- 172.21.15.38:0/361079001 wait complete. 2024-04-25T08:40:41.767 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:40:41.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.876+0000 7fc2a32c4640 1 Processor -- start 2024-04-25T08:40:41.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.877+0000 7fc2a32c4640 1 -- start start 2024-04-25T08:40:41.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.877+0000 7fc2a32c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc29c099ce0 0x7fc29c09a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.877+0000 7fc2a32c4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc29c09a0c0 0x7fc29c0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.877+0000 7fc2a32c4640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc29c0cd120 con 0x7fc29c15eb20 2024-04-25T08:40:41.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.877+0000 7fc2a32c4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc29c0cf6c0 con 0x7fc29c09a0c0 2024-04-25T08:40:41.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.877+0000 7fc2a32c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c0cd400 con 0x7fc29c099ce0 2024-04-25T08:40:41.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.878+0000 7fc2a085f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc29c09a0c0 0x7fc29c0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.878+0000 7fc2a1060640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc29c099ce0 0x7fc29c09a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.878+0000 7fc2a085f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc29c09a0c0 0x7fc29c0b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:53124/0 (socket says 172.21.15.38:53124) 2024-04-25T08:40:41.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.878+0000 7fc2a085f640 1 -- 172.21.15.38:0/4133731370 learned_addr learned my addr 172.21.15.38:0/4133731370 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:41.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.878+0000 7fc2a1861640 1 --1- 172.21.15.38:0/4133731370 >> v1:172.21.15.107:6789/0 conn(0x7fc29c15eb20 0x7fc29c0cfb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:44404/0 (socket says 172.21.15.38:44404) 2024-04-25T08:40:41.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.878+0000 7fc2a085f640 1 -- 172.21.15.38:0/4133731370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc29c099ce0 msgr2=0x7fc29c09a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:41.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.878+0000 7fc2a085f640 1 --2- 172.21.15.38:0/4133731370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc29c099ce0 0x7fc29c09a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.878+0000 7fc2a085f640 1 -- 172.21.15.38:0/4133731370 >> v1:172.21.15.107:6789/0 conn(0x7fc29c15eb20 legacy=0x7fc29c0cfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.878+0000 7fc293fff640 1 -- 172.21.15.38:0/4133731370 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997669484 0 0) 0x7fc29c0cd120 con 0x7fc29c15eb20 2024-04-25T08:40:41.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.878+0000 7fc2a085f640 1 -- 172.21.15.38:0/4133731370 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c0cf350 con 0x7fc29c09a0c0 2024-04-25T08:40:41.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.879+0000 7fc2a085f640 1 --2- 172.21.15.38:0/4133731370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc29c09a0c0 0x7fc29c0b4450 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009870 tx=0x7fc28c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1db802d5c0d6b58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:41.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.879+0000 7fc293fff640 1 -- 172.21.15.38:0/4133731370 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c037070 con 0x7fc29c09a0c0 2024-04-25T08:40:41.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.879+0000 7fc293fff640 1 -- 172.21.15.38:0/4133731370 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c032440 con 0x7fc29c09a0c0 2024-04-25T08:40:41.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.879+0000 7fc293fff640 1 -- 172.21.15.38:0/4133731370 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c032810 con 0x7fc29c09a0c0 2024-04-25T08:40:41.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.880+0000 7fc2a32c4640 1 -- 172.21.15.38:0/4133731370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc29c09a0c0 msgr2=0x7fc29c0b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.880+0000 7fc2a32c4640 1 --2- 172.21.15.38:0/4133731370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc29c09a0c0 0x7fc29c0b4450 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009870 tx=0x7fc28c02a270 comp rx=0 tx=0).stop 2024-04-25T08:40:41.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.880+0000 7fc2a32c4640 1 -- 172.21.15.38:0/4133731370 shutdown_connections 2024-04-25T08:40:41.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.880+0000 7fc2a32c4640 1 --2- 172.21.15.38:0/4133731370 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc29c09a0c0 0x7fc29c0b4450 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.880+0000 7fc2a32c4640 1 --2- 172.21.15.38:0/4133731370 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc29c099ce0 0x7fc29c09a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.880+0000 7fc2a32c4640 1 -- 172.21.15.38:0/4133731370 >> 172.21.15.38:0/4133731370 conn(0x7fc29c0b30f0 msgr2=0x7fc29c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:41.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.880+0000 7fc2a32c4640 1 -- 172.21.15.38:0/4133731370 shutdown_connections 2024-04-25T08:40:41.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.880+0000 7fc2a32c4640 1 -- 172.21.15.38:0/4133731370 wait complete. 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.881+0000 7fc2a32c4640 1 Processor -- start 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.881+0000 7fc2a32c4640 1 -- start start 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a32c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc29c099ce0 0x7fc29c1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a32c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc29c09a0c0 0x7fc29c15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a32c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc29c15eb20 0x7fc29c0b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a32c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c0b4b10 con 0x7fc29c099ce0 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a32c4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c0b4990 con 0x7fc29c09a0c0 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a32c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c0cccb0 con 0x7fc29c15eb20 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a1060640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc29c099ce0 0x7fc29c1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a1060640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc29c099ce0 0x7fc29c1525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39398/0 (socket says 172.21.15.38:39398) 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a1060640 1 -- 172.21.15.38:0/826582713 learned_addr learned my addr 172.21.15.38:0/826582713 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a1861640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc29c15eb20 0x7fc29c0b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a085f640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc29c09a0c0 0x7fc29c15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a1060640 1 -- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc29c15eb20 msgr2=0x7fc29c0b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a1060640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc29c15eb20 0x7fc29c0b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a1060640 1 -- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc29c09a0c0 msgr2=0x7fc29c15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a1060640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc29c09a0c0 0x7fc29c15aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.882+0000 7fc2a1060640 1 -- 172.21.15.38:0/826582713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c158650 con 0x7fc29c099ce0 2024-04-25T08:40:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.883+0000 7fc2a1060640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc29c099ce0 0x7fc29c1525b0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fc29800e9b0 tx=0x7fc29800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.883+0000 7fc291ffb640 1 -- 172.21.15.38:0/826582713 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298017070 con 0x7fc29c099ce0 2024-04-25T08:40:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.883+0000 7fc291ffb640 1 -- 172.21.15.38:0/826582713 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29800c9a0 con 0x7fc29c099ce0 2024-04-25T08:40:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.883+0000 7fc2a32c4640 1 -- 172.21.15.38:0/826582713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc29c158930 con 0x7fc29c099ce0 2024-04-25T08:40:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.883+0000 7fc291ffb640 1 -- 172.21.15.38:0/826582713 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29800cd90 con 0x7fc29c099ce0 2024-04-25T08:40:41.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.883+0000 7fc2a32c4640 1 -- 172.21.15.38:0/826582713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29c158ef0 con 0x7fc29c099ce0 2024-04-25T08:40:41.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.884+0000 7fc2a32c4640 1 -- 172.21.15.38:0/826582713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29c04ef90 con 0x7fc29c099ce0 2024-04-25T08:40:41.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.884+0000 7fc291ffb640 1 -- 172.21.15.38:0/826582713 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc29801a070 con 0x7fc29c099ce0 2024-04-25T08:40:41.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.885+0000 7fc291ffb640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc28006b4d0 0x7fc28006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:41.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.885+0000 7fc291ffb640 1 -- 172.21.15.38:0/826582713 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc29808a780 con 0x7fc29c099ce0 2024-04-25T08:40:41.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.885+0000 7fc2a085f640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc28006b4d0 0x7fc28006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:41.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.886+0000 7fc2a085f640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc28006b4d0 0x7fc28006d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc28c034000 tx=0x7fc28c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:41.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:41.889+0000 7fc291ffb640 1 -- 172.21.15.38:0/826582713 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc298059460 con 0x7fc29c099ce0 2024-04-25T08:40:42.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.105+0000 7fc2a32c4640 1 -- 172.21.15.38:0/826582713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc29c0b3e30 con 0x7fc29c099ce0 2024-04-25T08:40:42.098 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:40:42.106+0000 7ff7937f0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:40:42.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.108+0000 7fc291ffb640 1 -- 172.21.15.38:0/826582713 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v71) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc298058d60 con 0x7fc29c099ce0 2024-04-25T08:40:42.101 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:40:42.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.110+0000 7fc2a32c4640 1 -- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc28006b4d0 msgr2=0x7fc28006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:42.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.110+0000 7fc2a32c4640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc28006b4d0 0x7fc28006d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc28c034000 tx=0x7fc28c02e000 comp rx=0 tx=0).stop 2024-04-25T08:40:42.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.110+0000 7fc2a32c4640 1 -- 172.21.15.38:0/826582713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc29c099ce0 msgr2=0x7fc29c1525b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:42.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.110+0000 7fc2a32c4640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc29c099ce0 0x7fc29c1525b0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fc29800e9b0 tx=0x7fc29800ee70 comp rx=0 tx=0).stop 2024-04-25T08:40:42.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.110+0000 7fc2a32c4640 1 -- 172.21.15.38:0/826582713 shutdown_connections 2024-04-25T08:40:42.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.110+0000 7fc2a32c4640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc28006b4d0 0x7fc28006d990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:42.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.110+0000 7fc2a32c4640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc29c15eb20 0x7fc29c0b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:42.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.110+0000 7fc2a32c4640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc29c09a0c0 0x7fc29c15aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:42.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.110+0000 7fc2a32c4640 1 --2- 172.21.15.38:0/826582713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc29c099ce0 0x7fc29c1525b0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:42.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.110+0000 7fc2a32c4640 1 -- 172.21.15.38:0/826582713 >> 172.21.15.38:0/826582713 conn(0x7fc29c0b30f0 msgr2=0x7fc29c0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:42.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.111+0000 7fc2a32c4640 1 -- 172.21.15.38:0/826582713 shutdown_connections 2024-04-25T08:40:42.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.111+0000 7fc2a32c4640 1 -- 172.21.15.38:0/826582713 wait complete. 2024-04-25T08:40:42.115 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:40:42.144 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:40:42.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.225+0000 7fe9afe76640 1 Processor -- start 2024-04-25T08:40:42.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.225+0000 7fe9afe76640 1 -- start start 2024-04-25T08:40:42.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.226+0000 7fe9afe76640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe9a815df30 0x7fe9a809a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:42.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.226+0000 7fe9afe76640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a8162820 con 0x7fe9a809a180 2024-04-25T08:40:42.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.226+0000 7fe9afe76640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9a815f300 con 0x7fe9a8162390 2024-04-25T08:40:42.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.226+0000 7fe9afe76640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9a815e310 con 0x7fe9a815df30 2024-04-25T08:40:42.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.226+0000 7fe9ae413640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe9a809a180 0x7fe9a80cf8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53386/0 (socket says 172.21.15.38:53386) 2024-04-25T08:40:42.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.226+0000 7fe9ae413640 1 -- 172.21.15.38:0/3860178240 learned_addr learned my addr 172.21.15.38:0/3860178240 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:42.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.226+0000 7fe9adc12640 1 --2- 172.21.15.38:0/3860178240 >> v2:172.21.15.107:3300/0 conn(0x7fe9a815df30 0x7fe9a809a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:42.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.226+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200602699 0 0) 0x7fe9a0001490 con 0x7fe9a809a180 2024-04-25T08:40:42.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.226+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe994003520 con 0x7fe9a809a180 2024-04-25T08:40:42.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269197080 0 0) 0x7fe9a815f300 con 0x7fe9a8162390 2024-04-25T08:40:42.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a0001490 con 0x7fe9a8162390 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1253995292 0 0) 0x7fe994003520 con 0x7fe9a809a180 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 >> v2:172.21.15.107:3300/0 conn(0x7fe9a815df30 msgr2=0x7fe9a809a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 --2- 172.21.15.38:0/3860178240 >> v2:172.21.15.107:3300/0 conn(0x7fe9a815df30 0x7fe9a809a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 >> v1:172.21.15.107:6789/0 conn(0x7fe9a8162390 legacy=0x7fe9a8099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a815e540 con 0x7fe9a809a180 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9adc12640 1 --2- 172.21.15.38:0/3860178240 >> v2:172.21.15.107:3300/0 conn(0x7fe9a815df30 0x7fe9a809a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fe9a0001bc0 con 0x7fe9a809a180 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 >> v1:172.21.15.38:6789/0 conn(0x7fe9a809a180 legacy=0x7fe9a80cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 --2- 172.21.15.38:0/3860178240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 0x7fe994003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 --2- 172.21.15.38:0/3860178240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe994004800 0x7fe994004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 --2- 172.21.15.38:0/3860178240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9940051c0 0x7fe99400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99400c010 con 0x7fe994003b40 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99400c1c0 con 0x7fe9940051c0 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99400c3c0 con 0x7fe994004800 2024-04-25T08:40:42.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9adc12640 1 --2- 172.21.15.38:0/3860178240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 0x7fe994003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9ad411640 1 --2- 172.21.15.38:0/3860178240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe994004800 0x7fe994004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9ae413640 1 --2- 172.21.15.38:0/3860178240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9940051c0 0x7fe99400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9adc12640 1 -- 172.21.15.38:0/3860178240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe994004800 msgr2=0x7fe994004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9adc12640 1 --2- 172.21.15.38:0/3860178240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe994004800 0x7fe994004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9adc12640 1 -- 172.21.15.38:0/3860178240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9940051c0 msgr2=0x7fe99400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9adc12640 1 --2- 172.21.15.38:0/3860178240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9940051c0 0x7fe99400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9adc12640 1 -- 172.21.15.38:0/3860178240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe994003520 con 0x7fe994003b40 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9ae413640 1 --2- 172.21.15.38:0/3860178240 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9940051c0 0x7fe99400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.227+0000 7fe9ae413640 1 -- 172.21.15.38:0/3860178240 reap_dead start 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.228+0000 7fe9adc12640 1 --2- 172.21.15.38:0/3860178240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 0x7fe994003f40 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009c90 tx=0x7fe994005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a388788e1ce052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.228+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4036070 con 0x7fe994003b40 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.228+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a4031400 con 0x7fe994003b40 2024-04-25T08:40:42.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.228+0000 7fe9acc10640 1 -- 172.21.15.38:0/3860178240 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4009860 con 0x7fe994003b40 2024-04-25T08:40:42.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.228+0000 7fe9afe76640 1 -- 172.21.15.38:0/3860178240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 msgr2=0x7fe994003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:42.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.228+0000 7fe9afe76640 1 --2- 172.21.15.38:0/3860178240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 0x7fe994003f40 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009c90 tx=0x7fe994005830 comp rx=0 tx=0).stop 2024-04-25T08:40:42.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.229+0000 7fe9afe76640 1 -- 172.21.15.38:0/3860178240 shutdown_connections 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.229+0000 7fe9afe76640 1 --2- 172.21.15.38:0/3860178240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 0x7fe994003f40 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.229+0000 7fe9afe76640 1 -- 172.21.15.38:0/3860178240 >> 172.21.15.38:0/3860178240 conn(0x7fe9a80b3110 msgr2=0x7fe9a80b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.229+0000 7fe9afe76640 1 -- 172.21.15.38:0/3860178240 shutdown_connections 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.229+0000 7fe9afe76640 1 -- 172.21.15.38:0/3860178240 wait complete. 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.230+0000 7fe9afe76640 1 Processor -- start 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.230+0000 7fe9afe76640 1 -- start start 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9afe76640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 0x7fe9a8158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9afe76640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9a8159330 0x7fe9a80ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9afe76640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a80b4570 0x7fe9a80cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9afe76640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a815ee70 con 0x7fe994003b40 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9afe76640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a809aeb0 con 0x7fe9a80b4570 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9afe76640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9a815eb00 con 0x7fe9a8159330 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9adc12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 0x7fe9a8158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9adc12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 0x7fe9a8158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39422/0 (socket says 172.21.15.38:39422) 2024-04-25T08:40:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9adc12640 1 -- 172.21.15.38:0/926373923 learned_addr learned my addr 172.21.15.38:0/926373923 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9ad411640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9a8159330 0x7fe9a80ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9ae413640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a80b4570 0x7fe9a80cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9adc12640 1 -- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9a8159330 msgr2=0x7fe9a80ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9adc12640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9a8159330 0x7fe9a80ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9adc12640 1 -- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a80b4570 msgr2=0x7fe9a80cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9adc12640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a80b4570 0x7fe9a80cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9adc12640 1 -- 172.21.15.38:0/926373923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a8158a90 con 0x7fe994003b40 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9adc12640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 0x7fe9a8158550 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009c90 tx=0x7fe9a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe99e7fc640 1 -- 172.21.15.38:0/926373923 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4040070 con 0x7fe994003b40 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe99e7fc640 1 -- 172.21.15.38:0/926373923 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a403b680 con 0x7fe994003b40 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9afe76640 1 -- 172.21.15.38:0/926373923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a80b4a30 con 0x7fe994003b40 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe99e7fc640 1 -- 172.21.15.38:0/926373923 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4036040 con 0x7fe994003b40 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.231+0000 7fe9afe76640 1 -- 172.21.15.38:0/926373923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a80b4ff0 con 0x7fe994003b40 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.232+0000 7fe9afe76640 1 -- 172.21.15.38:0/926373923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9a8068e80 con 0x7fe994003b40 2024-04-25T08:40:42.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.233+0000 7fe99e7fc640 1 -- 172.21.15.38:0/926373923 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9a4043070 con 0x7fe994003b40 2024-04-25T08:40:42.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.233+0000 7fe99e7fc640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe98c06b4d0 0x7fe98c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:42.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.233+0000 7fe99e7fc640 1 -- 172.21.15.38:0/926373923 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe9a40aa9f0 con 0x7fe994003b40 2024-04-25T08:40:42.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.233+0000 7fe9ad411640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe98c06b4d0 0x7fe98c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:42.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.234+0000 7fe9ad411640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe98c06b4d0 0x7fe98c06d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8159730 tx=0x7fe998009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:42.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.238+0000 7fe99e7fc640 1 -- 172.21.15.38:0/926373923 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9a4079190 con 0x7fe994003b40 2024-04-25T08:40:42.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:42.459+0000 7fe9afe76640 1 -- 172.21.15.38:0/926373923 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe9a80b51c0 con 0x7fe994003b40 2024-04-25T08:40:43.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.110+0000 7fe99e7fc640 1 -- 172.21.15.38:0/926373923 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v72) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe9a407d230 con 0x7fe994003b40 2024-04-25T08:40:43.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.112+0000 7fe9afe76640 1 -- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe98c06b4d0 msgr2=0x7fe98c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:43.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.112+0000 7fe9afe76640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe98c06b4d0 0x7fe98c06d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe9a8159730 tx=0x7fe998009040 comp rx=0 tx=0).stop 2024-04-25T08:40:43.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.113+0000 7fe9afe76640 1 -- 172.21.15.38:0/926373923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 msgr2=0x7fe9a8158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:43.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.113+0000 7fe9afe76640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 0x7fe9a8158550 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009c90 tx=0x7fe9a4004730 comp rx=0 tx=0).stop 2024-04-25T08:40:43.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.113+0000 7fe9afe76640 1 -- 172.21.15.38:0/926373923 shutdown_connections 2024-04-25T08:40:43.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.113+0000 7fe9afe76640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9a80b4570 0x7fe9a80cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:43.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.113+0000 7fe9afe76640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe98c06b4d0 0x7fe98c06d990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:43.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.113+0000 7fe9afe76640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9a8159330 0x7fe9a80ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:43.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.113+0000 7fe9afe76640 1 --2- 172.21.15.38:0/926373923 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe994003b40 0x7fe9a8158550 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:43.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.113+0000 7fe9afe76640 1 -- 172.21.15.38:0/926373923 >> 172.21.15.38:0/926373923 conn(0x7fe9a80b3110 msgr2=0x7fe9a81618a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:43.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.113+0000 7fe9afe76640 1 -- 172.21.15.38:0/926373923 shutdown_connections 2024-04-25T08:40:43.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.113+0000 7fe9afe76640 1 -- 172.21.15.38:0/926373923 wait complete. 2024-04-25T08:40:43.119 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:40:43.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.238+0000 7f5c6b516640 1 Processor -- start 2024-04-25T08:40:43.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.238+0000 7f5c6b516640 1 -- start start 2024-04-25T08:40:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c6b516640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c641612c0 0x7f5c640b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c6b516640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5c640b6a00 0x7f5c640d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c6b516640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5c640d0870 0x7f5c640d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c6b516640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c640d11e0 con 0x7f5c640d0870 2024-04-25T08:40:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c6b516640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c640b7200 con 0x7f5c640b6a00 2024-04-25T08:40:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c6b516640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640b74e0 con 0x7f5c641612c0 2024-04-25T08:40:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c69ab3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5c640d0870 0x7f5c640d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:43.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c69ab3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5c640d0870 0x7f5c640d0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39428/0 (socket says 172.21.15.38:39428) 2024-04-25T08:40:43.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c69ab3640 1 -- 172.21.15.38:0/1666943315 learned_addr learned my addr 172.21.15.38:0/1666943315 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:43.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c68ab1640 1 --2- 172.21.15.38:0/1666943315 >> v2:172.21.15.107:3300/0 conn(0x7f5c640b6a00 0x7f5c640d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:43.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c692b2640 1 --2- 172.21.15.38:0/1666943315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c641612c0 0x7f5c640b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:43.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c69ab3640 1 -- 172.21.15.38:0/1666943315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c641612c0 msgr2=0x7f5c640b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:43.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c69ab3640 1 --2- 172.21.15.38:0/1666943315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c641612c0 0x7f5c640b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:43.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c69ab3640 1 -- 172.21.15.38:0/1666943315 >> v2:172.21.15.107:3300/0 conn(0x7f5c640b6a00 msgr2=0x7f5c640d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:43.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c69ab3640 1 --2- 172.21.15.38:0/1666943315 >> v2:172.21.15.107:3300/0 conn(0x7f5c640b6a00 0x7f5c640d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:43.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.239+0000 7f5c69ab3640 1 -- 172.21.15.38:0/1666943315 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c640b77c0 con 0x7f5c640d0870 2024-04-25T08:40:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.240+0000 7f5c69ab3640 1 --2- 172.21.15.38:0/1666943315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640d0870 0x7f5c640d0ca0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5c640b7410 tx=0x7f5c5c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3801d06f220fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.240+0000 7f5c5bfff640 1 -- 172.21.15.38:0/1666943315 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c036070 con 0x7f5c640d0870 2024-04-25T08:40:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.240+0000 7f5c5bfff640 1 -- 172.21.15.38:0/1666943315 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c031600 con 0x7f5c640d0870 2024-04-25T08:40:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.240+0000 7f5c5bfff640 1 -- 172.21.15.38:0/1666943315 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c0319d0 con 0x7f5c640d0870 2024-04-25T08:40:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.241+0000 7f5c6b516640 1 -- 172.21.15.38:0/1666943315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640d0870 msgr2=0x7f5c640d0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.241+0000 7f5c6b516640 1 --2- 172.21.15.38:0/1666943315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640d0870 0x7f5c640d0ca0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5c640b7410 tx=0x7f5c5c02a5f0 comp rx=0 tx=0).stop 2024-04-25T08:40:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.241+0000 7f5c6b516640 1 -- 172.21.15.38:0/1666943315 shutdown_connections 2024-04-25T08:40:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.241+0000 7f5c6b516640 1 --2- 172.21.15.38:0/1666943315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640d0870 0x7f5c640d0ca0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.241+0000 7f5c6b516640 1 --2- 172.21.15.38:0/1666943315 >> v2:172.21.15.107:3300/0 conn(0x7f5c640b6a00 0x7f5c640d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.241+0000 7f5c6b516640 1 --2- 172.21.15.38:0/1666943315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c641612c0 0x7f5c640b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:43.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.241+0000 7f5c6b516640 1 -- 172.21.15.38:0/1666943315 >> 172.21.15.38:0/1666943315 conn(0x7f5c64066d10 msgr2=0x7f5c64067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:43.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.241+0000 7f5c6b516640 1 -- 172.21.15.38:0/1666943315 shutdown_connections 2024-04-25T08:40:43.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.241+0000 7f5c6b516640 1 -- 172.21.15.38:0/1666943315 wait complete. 2024-04-25T08:40:43.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.242+0000 7f5c6b516640 1 Processor -- start 2024-04-25T08:40:43.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.242+0000 7f5c6b516640 1 -- start start 2024-04-25T08:40:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c6b516640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640b6a00 0x7f5c6414a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c6b516640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c640d0870 0x7f5c640cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:43.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c6b516640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c641612c0 0x7f5c640cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c6b516640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640d2170 con 0x7f5c640b6a00 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c6b516640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640b7d80 con 0x7f5c640d0870 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c6b516640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640b6e60 con 0x7f5c641612c0 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c692b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640b6a00 0x7f5c6414a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c692b2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640b6a00 0x7f5c6414a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39442/0 (socket says 172.21.15.38:39442) 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c692b2640 1 -- 172.21.15.38:0/118924520 learned_addr learned my addr 172.21.15.38:0/118924520 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c69ab3640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c641612c0 0x7f5c640cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c68ab1640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c640d0870 0x7f5c640cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c692b2640 1 -- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c641612c0 msgr2=0x7f5c640cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c692b2640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c641612c0 0x7f5c640cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c692b2640 1 -- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c640d0870 msgr2=0x7f5c640cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c692b2640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c640d0870 0x7f5c640cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c692b2640 1 -- 172.21.15.38:0/118924520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64147770 con 0x7f5c640b6a00 2024-04-25T08:40:43.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c68ab1640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c640d0870 0x7f5c640cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c692b2640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640b6a00 0x7f5c6414a1c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000d800 tx=0x7f5c6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c59ffb640 1 -- 172.21.15.38:0/118924520 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000b6c0 con 0x7f5c640b6a00 2024-04-25T08:40:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c59ffb640 1 -- 172.21.15.38:0/118924520 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c60019070 con 0x7f5c640b6a00 2024-04-25T08:40:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c6b516640 1 -- 172.21.15.38:0/118924520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6415e4f0 con 0x7f5c640b6a00 2024-04-25T08:40:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.243+0000 7f5c59ffb640 1 -- 172.21.15.38:0/118924520 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6000ba80 con 0x7f5c640b6a00 2024-04-25T08:40:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.244+0000 7f5c6b516640 1 -- 172.21.15.38:0/118924520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6415eab0 con 0x7f5c640b6a00 2024-04-25T08:40:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.245+0000 7f5c6b516640 1 -- 172.21.15.38:0/118924520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c64068e80 con 0x7f5c640b6a00 2024-04-25T08:40:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.245+0000 7f5c59ffb640 1 -- 172.21.15.38:0/118924520 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c6000bc20 con 0x7f5c640b6a00 2024-04-25T08:40:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.245+0000 7f5c59ffb640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5c4806b4d0 0x7f5c4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.246+0000 7f5c59ffb640 1 -- 172.21.15.38:0/118924520 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5c6008a760 con 0x7f5c640b6a00 2024-04-25T08:40:43.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.246+0000 7f5c68ab1640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5c4806b4d0 0x7f5c4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:43.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.246+0000 7f5c68ab1640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5c4806b4d0 0x7f5c4806d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5c640cd480 tx=0x7f5c54008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.250+0000 7f5c59ffb640 1 -- 172.21.15.38:0/118924520 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c60059490 con 0x7f5c640b6a00 2024-04-25T08:40:43.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:43.430+0000 7f5c6b516640 1 -- 172.21.15.38:0/118924520 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5c640d3710 con 0x7f5c640b6a00 2024-04-25T08:40:44.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.281+0000 7f5c59ffb640 1 -- 172.21.15.38:0/118924520 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5c60058d90 con 0x7f5c640b6a00 2024-04-25T08:40:44.274 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:40:44.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.284+0000 7f5c6b516640 1 -- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5c4806b4d0 msgr2=0x7f5c4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:44.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.284+0000 7f5c6b516640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5c4806b4d0 0x7f5c4806d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f5c640cd480 tx=0x7f5c54008040 comp rx=0 tx=0).stop 2024-04-25T08:40:44.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.284+0000 7f5c6b516640 1 -- 172.21.15.38:0/118924520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640b6a00 msgr2=0x7f5c6414a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:44.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.284+0000 7f5c6b516640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640b6a00 0x7f5c6414a1c0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000d800 tx=0x7f5c6000dcc0 comp rx=0 tx=0).stop 2024-04-25T08:40:44.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.284+0000 7f5c6b516640 1 -- 172.21.15.38:0/118924520 shutdown_connections 2024-04-25T08:40:44.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.284+0000 7f5c6b516640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5c4806b4d0 0x7f5c4806d990 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:44.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.284+0000 7f5c6b516640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5c641612c0 0x7f5c640cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:44.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.284+0000 7f5c6b516640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c640d0870 0x7f5c640cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:44.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.284+0000 7f5c6b516640 1 --2- 172.21.15.38:0/118924520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5c640b6a00 0x7f5c6414a1c0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:44.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.284+0000 7f5c6b516640 1 -- 172.21.15.38:0/118924520 >> 172.21.15.38:0/118924520 conn(0x7f5c64066d10 msgr2=0x7f5c640d3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:44.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.284+0000 7f5c6b516640 1 -- 172.21.15.38:0/118924520 shutdown_connections 2024-04-25T08:40:44.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.284+0000 7f5c6b516640 1 -- 172.21.15.38:0/118924520 wait complete. 2024-04-25T08:40:44.291 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:40:44.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.409+0000 7f0526015640 1 Processor -- start 2024-04-25T08:40:44.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.409+0000 7f0526015640 1 -- start start 2024-04-25T08:40:44.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.410+0000 7f0526015640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f052015ee70 0x7f05200b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:44.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.410+0000 7f0526015640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05200cd410 0x7f05200b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:44.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.410+0000 7f0526015640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05200b0120 con 0x7f05200b2ba0 2024-04-25T08:40:44.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.410+0000 7f0526015640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05200b0400 con 0x7f052015ee70 2024-04-25T08:40:44.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.410+0000 7f0526015640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05200b06e0 con 0x7f05200cd410 2024-04-25T08:40:44.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.410+0000 7f051f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f052015ee70 0x7f05200b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:44.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.410+0000 7f051f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f052015ee70 0x7f05200b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39444/0 (socket says 172.21.15.38:39444) 2024-04-25T08:40:44.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.410+0000 7f051f7fe640 1 -- 172.21.15.38:0/3074806321 learned_addr learned my addr 172.21.15.38:0/3074806321 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:44.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.410+0000 7f051ffff640 1 --2- 172.21.15.38:0/3074806321 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05200cd410 0x7f05200b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:44.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.411+0000 7f051f7fe640 1 -- 172.21.15.38:0/3074806321 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05200cd410 msgr2=0x7f05200b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:44.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.411+0000 7f051f7fe640 1 --2- 172.21.15.38:0/3074806321 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05200cd410 0x7f05200b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:44.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.411+0000 7f051f7fe640 1 -- 172.21.15.38:0/3074806321 >> v1:172.21.15.107:6789/0 conn(0x7f05200b2ba0 legacy=0x7f052006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:44.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.411+0000 7f051f7fe640 1 -- 172.21.15.38:0/3074806321 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05200b09c0 con 0x7f052015ee70 2024-04-25T08:40:44.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.411+0000 7f051f7fe640 1 --2- 172.21.15.38:0/3074806321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f052015ee70 0x7f05200b12f0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f0510009870 tx=0x7f051002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3803ca7921b57061 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:44.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.411+0000 7f051e7fc640 1 -- 172.21.15.38:0/3074806321 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510037070 con 0x7f052015ee70 2024-04-25T08:40:44.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.411+0000 7f051e7fc640 1 -- 172.21.15.38:0/3074806321 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0510032440 con 0x7f052015ee70 2024-04-25T08:40:44.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.412+0000 7f051e7fc640 1 -- 172.21.15.38:0/3074806321 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510032810 con 0x7f052015ee70 2024-04-25T08:40:44.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.412+0000 7f0526015640 1 -- 172.21.15.38:0/3074806321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f052015ee70 msgr2=0x7f05200b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:44.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.412+0000 7f0526015640 1 --2- 172.21.15.38:0/3074806321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f052015ee70 0x7f05200b12f0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f0510009870 tx=0x7f051002a180 comp rx=0 tx=0).stop 2024-04-25T08:40:44.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.412+0000 7f0526015640 1 -- 172.21.15.38:0/3074806321 shutdown_connections 2024-04-25T08:40:44.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.412+0000 7f0526015640 1 --2- 172.21.15.38:0/3074806321 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05200cd410 0x7f05200b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:44.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.412+0000 7f0526015640 1 --2- 172.21.15.38:0/3074806321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f052015ee70 0x7f05200b12f0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:44.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.412+0000 7f0526015640 1 -- 172.21.15.38:0/3074806321 >> 172.21.15.38:0/3074806321 conn(0x7f0520099a70 msgr2=0x7f05200b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:44.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.412+0000 7f0526015640 1 -- 172.21.15.38:0/3074806321 shutdown_connections 2024-04-25T08:40:44.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.412+0000 7f0526015640 1 -- 172.21.15.38:0/3074806321 wait complete. 2024-04-25T08:40:44.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.413+0000 7f0526015640 1 Processor -- start 2024-04-25T08:40:44.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.413+0000 7f0526015640 1 -- start start 2024-04-25T08:40:44.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.414+0000 7f0526015640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200b2ba0 0x7f05200c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:44.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.414+0000 7f0526015640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05200cd410 0x7f05201f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:44.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.414+0000 7f0526015640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f052015ee70 0x7f052006e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:44.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.414+0000 7f0526015640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052015f3d0 con 0x7f05200b2ba0 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.414+0000 7f0526015640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05200afdf0 con 0x7f052015ee70 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.414+0000 7f0526015640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052015f550 con 0x7f05200cd410 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.414+0000 7f051f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200b2ba0 0x7f05200c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.414+0000 7f051f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200b2ba0 0x7f05200c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39454/0 (socket says 172.21.15.38:39454) 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.414+0000 7f051f7fe640 1 -- 172.21.15.38:0/2910525651 learned_addr learned my addr 172.21.15.38:0/2910525651 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.414+0000 7f051effd640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05200cd410 0x7f05201f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.415+0000 7f051ffff640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f052015ee70 0x7f052006e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.415+0000 7f051f7fe640 1 -- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05200cd410 msgr2=0x7f05201f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.415+0000 7f051f7fe640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05200cd410 0x7f05201f7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.415+0000 7f051f7fe640 1 -- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f052015ee70 msgr2=0x7f052006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.415+0000 7f051f7fe640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f052015ee70 0x7f052006e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.415+0000 7f051f7fe640 1 -- 172.21.15.38:0/2910525651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05201f8850 con 0x7f05200b2ba0 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.415+0000 7f051f7fe640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200b2ba0 0x7f05200c9d50 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f0510009870 tx=0x7f05100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:44.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.415+0000 7f0507fff640 1 -- 172.21.15.38:0/2910525651 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510041020 con 0x7f05200b2ba0 2024-04-25T08:40:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.415+0000 7f0526015640 1 -- 172.21.15.38:0/2910525651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05201f8b30 con 0x7f05200b2ba0 2024-04-25T08:40:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.415+0000 7f0507fff640 1 -- 172.21.15.38:0/2910525651 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0510030970 con 0x7f05200b2ba0 2024-04-25T08:40:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.415+0000 7f0507fff640 1 -- 172.21.15.38:0/2910525651 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510037040 con 0x7f05200b2ba0 2024-04-25T08:40:44.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.415+0000 7f0526015640 1 -- 172.21.15.38:0/2910525651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05200c9830 con 0x7f05200b2ba0 2024-04-25T08:40:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.416+0000 7f0526015640 1 -- 172.21.15.38:0/2910525651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0520068e80 con 0x7f05200b2ba0 2024-04-25T08:40:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.417+0000 7f0507fff640 1 -- 172.21.15.38:0/2910525651 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0510044020 con 0x7f05200b2ba0 2024-04-25T08:40:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.417+0000 7f0507fff640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f050c06b4d0 0x7f050c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.417+0000 7f0507fff640 1 -- 172.21.15.38:0/2910525651 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f05100aac60 con 0x7f05200b2ba0 2024-04-25T08:40:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.417+0000 7f051effd640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f050c06b4d0 0x7f050c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:44.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.418+0000 7f051effd640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f050c06b4d0 0x7f050c06d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0514006fd0 tx=0x7f0514008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:44.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.421+0000 7f0507fff640 1 -- 172.21.15.38:0/2910525651 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f051004b030 con 0x7f05200b2ba0 2024-04-25T08:40:44.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:44.600+0000 7f0526015640 1 -- 172.21.15.38:0/2910525651 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f05200683a0 con 0x7f05200b2ba0 2024-04-25T08:40:45.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.282+0000 7f0507fff640 1 -- 172.21.15.38:0/2910525651 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0510079a30 con 0x7f05200b2ba0 2024-04-25T08:40:45.275 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-04-25T08:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.285+0000 7f0526015640 1 -- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f050c06b4d0 msgr2=0x7f050c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.285+0000 7f0526015640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f050c06b4d0 0x7f050c06d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0514006fd0 tx=0x7f0514008040 comp rx=0 tx=0).stop 2024-04-25T08:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.285+0000 7f0526015640 1 -- 172.21.15.38:0/2910525651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200b2ba0 msgr2=0x7f05200c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.285+0000 7f0526015640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200b2ba0 0x7f05200c9d50 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f0510009870 tx=0x7f05100046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.285+0000 7f0526015640 1 -- 172.21.15.38:0/2910525651 shutdown_connections 2024-04-25T08:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.285+0000 7f0526015640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f052015ee70 0x7f052006e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.285+0000 7f0526015640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f050c06b4d0 0x7f050c06d990 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.285+0000 7f0526015640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f05200cd410 0x7f05201f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.285+0000 7f0526015640 1 --2- 172.21.15.38:0/2910525651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05200b2ba0 0x7f05200c9d50 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.285+0000 7f0526015640 1 -- 172.21.15.38:0/2910525651 >> 172.21.15.38:0/2910525651 conn(0x7f0520099a70 msgr2=0x7f0520073730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.285+0000 7f0526015640 1 -- 172.21.15.38:0/2910525651 shutdown_connections 2024-04-25T08:40:45.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.285+0000 7f0526015640 1 -- 172.21.15.38:0/2910525651 wait complete. 2024-04-25T08:40:45.292 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:40:45.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.412+0000 7efcc13f4640 1 Processor -- start 2024-04-25T08:40:45.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.412+0000 7efcc13f4640 1 -- start start 2024-04-25T08:40:45.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcc13f4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7efcbc0d0320 0x7efcbc0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:45.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcc13f4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efcbc0b6980 0x7efcbc0b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:45.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcc13f4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcbc0d2060 con 0x7efcbc0d0700 2024-04-25T08:40:45.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcc13f4640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc0d2330 con 0x7efcbc0b6980 2024-04-25T08:40:45.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcc13f4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc0d2610 con 0x7efcbc0d0320 2024-04-25T08:40:45.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcbb7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efcbc0b6980 0x7efcbc0b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:45.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcba7fc640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7efcbc0d0700 0x7efcbc0d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53388/0 (socket says 172.21.15.38:53388) 2024-04-25T08:40:45.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcba7fc640 1 -- 172.21.15.38:0/3588945821 learned_addr learned my addr 172.21.15.38:0/3588945821 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:45.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcbaffd640 1 --2- 172.21.15.38:0/3588945821 >> v2:172.21.15.107:3300/0 conn(0x7efcbc0d0320 0x7efcbc0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:45.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcbb7fe640 1 -- 172.21.15.38:0/3588945821 >> v2:172.21.15.107:3300/0 conn(0x7efcbc0d0320 msgr2=0x7efcbc0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:45.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcbb7fe640 1 --2- 172.21.15.38:0/3588945821 >> v2:172.21.15.107:3300/0 conn(0x7efcbc0d0320 0x7efcbc0d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcbb7fe640 1 -- 172.21.15.38:0/3588945821 >> v1:172.21.15.38:6789/0 conn(0x7efcbc0d0700 legacy=0x7efcbc0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcbb7fe640 1 -- 172.21.15.38:0/3588945821 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc0d28f0 con 0x7efcbc0b6980 2024-04-25T08:40:45.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.413+0000 7efcb9ffb640 1 -- 172.21.15.38:0/3588945821 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095999241 0 0) 0x7efcbc0d2060 con 0x7efcbc0d0700 2024-04-25T08:40:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.414+0000 7efcbb7fe640 1 --2- 172.21.15.38:0/3588945821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcbc0b6980 0x7efcbc0b6dd0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7efcbc069100 tx=0x7efcb402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e03c0363871e41b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.414+0000 7efcb9ffb640 1 -- 172.21.15.38:0/3588945821 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4007740 con 0x7efcbc0b6980 2024-04-25T08:40:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.414+0000 7efcb9ffb640 1 -- 172.21.15.38:0/3588945821 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb40078e0 con 0x7efcbc0b6980 2024-04-25T08:40:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.414+0000 7efcb9ffb640 1 -- 172.21.15.38:0/3588945821 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4007cb0 con 0x7efcbc0b6980 2024-04-25T08:40:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.415+0000 7efcc13f4640 1 -- 172.21.15.38:0/3588945821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcbc0b6980 msgr2=0x7efcbc0b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.415+0000 7efcc13f4640 1 --2- 172.21.15.38:0/3588945821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcbc0b6980 0x7efcbc0b6dd0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7efcbc069100 tx=0x7efcb402a710 comp rx=0 tx=0).stop 2024-04-25T08:40:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.415+0000 7efcc13f4640 1 -- 172.21.15.38:0/3588945821 shutdown_connections 2024-04-25T08:40:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.415+0000 7efcc13f4640 1 --2- 172.21.15.38:0/3588945821 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcbc0b6980 0x7efcbc0b6dd0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.415+0000 7efcc13f4640 1 --2- 172.21.15.38:0/3588945821 >> v2:172.21.15.107:3300/0 conn(0x7efcbc0d0320 0x7efcbc0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.415+0000 7efcc13f4640 1 -- 172.21.15.38:0/3588945821 >> 172.21.15.38:0/3588945821 conn(0x7efcbc0668f0 msgr2=0x7efcbc066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:45.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.415+0000 7efcc13f4640 1 -- 172.21.15.38:0/3588945821 shutdown_connections 2024-04-25T08:40:45.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.415+0000 7efcc13f4640 1 -- 172.21.15.38:0/3588945821 wait complete. 2024-04-25T08:40:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.416+0000 7efcc13f4640 1 Processor -- start 2024-04-25T08:40:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.416+0000 7efcc13f4640 1 -- start start 2024-04-25T08:40:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcc13f4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcbc0b6980 0x7efcbc0c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcc13f4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcbc0d0320 0x7efcbc1f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcc13f4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcbc0d0700 0x7efcbc154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcc13f4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0d0b40 con 0x7efcbc0d0320 2024-04-25T08:40:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcc13f4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc0d2eb0 con 0x7efcbc0b6980 2024-04-25T08:40:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcc13f4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc1f3d30 con 0x7efcbc0d0700 2024-04-25T08:40:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcba7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcbc0d0320 0x7efcbc1f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcba7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcbc0d0320 0x7efcbc1f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39472/0 (socket says 172.21.15.38:39472) 2024-04-25T08:40:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcba7fc640 1 -- 172.21.15.38:0/1945646826 learned_addr learned my addr 172.21.15.38:0/1945646826 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:45.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcbaffd640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcbc0b6980 0x7efcbc0c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcbb7fe640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcbc0d0700 0x7efcbc154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcba7fc640 1 -- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcbc0d0700 msgr2=0x7efcbc154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcba7fc640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcbc0d0700 0x7efcbc154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcba7fc640 1 -- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcbc0b6980 msgr2=0x7efcbc0c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcba7fc640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcbc0b6980 0x7efcbc0c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcba7fc640 1 -- 172.21.15.38:0/1945646826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc0c9f90 con 0x7efcbc0d0320 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcbaffd640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcbc0b6980 0x7efcbc0c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.417+0000 7efcba7fc640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcbc0d0320 0x7efcbc1f6ec0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7efcb000c3f0 tx=0x7efcb000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.418+0000 7efca37fe640 1 -- 172.21.15.38:0/1945646826 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb00074e0 con 0x7efcbc0d0320 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.418+0000 7efca37fe640 1 -- 172.21.15.38:0/1945646826 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb00047e0 con 0x7efcbc0d0320 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.418+0000 7efcc13f4640 1 -- 172.21.15.38:0/1945646826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcbc1f7710 con 0x7efcbc0d0320 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.418+0000 7efca37fe640 1 -- 172.21.15.38:0/1945646826 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb0004c10 con 0x7efcbc0d0320 2024-04-25T08:40:45.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.418+0000 7efcc13f4640 1 -- 172.21.15.38:0/1945646826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcbc1f7cd0 con 0x7efcbc0d0320 2024-04-25T08:40:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.419+0000 7efcc13f4640 1 -- 172.21.15.38:0/1945646826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc068e80 con 0x7efcbc0d0320 2024-04-25T08:40:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.419+0000 7efca37fe640 1 -- 172.21.15.38:0/1945646826 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efcb0011070 con 0x7efcbc0d0320 2024-04-25T08:40:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.420+0000 7efca37fe640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efca806b500 0x7efca806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:45.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.420+0000 7efca37fe640 1 -- 172.21.15.38:0/1945646826 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efcb0089ba0 con 0x7efcbc0d0320 2024-04-25T08:40:45.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.420+0000 7efcbaffd640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efca806b500 0x7efca806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:45.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.420+0000 7efcbaffd640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efca806b500 0x7efca806d9c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7efcac006fd0 tx=0x7efcac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:45.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.424+0000 7efca37fe640 1 -- 172.21.15.38:0/1945646826 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efcbc0c9f90 con 0x7efcbc0d0320 2024-04-25T08:40:45.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.604+0000 7efcc13f4640 1 -- 172.21.15.38:0/1945646826 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7efcbc1f3eb0 con 0x7efcbc0d0320 2024-04-25T08:40:45.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.608+0000 7efca37fe640 1 -- 172.21.15.38:0/1945646826 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7efcb001a2b0 con 0x7efcbc0d0320 2024-04-25T08:40:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.610+0000 7efcc13f4640 1 -- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efca806b500 msgr2=0x7efca806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.610+0000 7efcc13f4640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efca806b500 0x7efca806d9c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7efcac006fd0 tx=0x7efcac008040 comp rx=0 tx=0).stop 2024-04-25T08:40:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.610+0000 7efcc13f4640 1 -- 172.21.15.38:0/1945646826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcbc0d0320 msgr2=0x7efcbc1f6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.610+0000 7efcc13f4640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcbc0d0320 0x7efcbc1f6ec0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7efcb000c3f0 tx=0x7efcb000c8b0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.610+0000 7efcc13f4640 1 -- 172.21.15.38:0/1945646826 shutdown_connections 2024-04-25T08:40:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.610+0000 7efcc13f4640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efca806b500 0x7efca806d9c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.610+0000 7efcc13f4640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcbc0d0700 0x7efcbc154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.610+0000 7efcc13f4640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcbc0d0320 0x7efcbc1f6ec0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.610+0000 7efcc13f4640 1 --2- 172.21.15.38:0/1945646826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcbc0b6980 0x7efcbc0c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.610+0000 7efcc13f4640 1 -- 172.21.15.38:0/1945646826 >> 172.21.15.38:0/1945646826 conn(0x7efcbc0668f0 msgr2=0x7efcbc0ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:45.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.610+0000 7efcc13f4640 1 -- 172.21.15.38:0/1945646826 shutdown_connections 2024-04-25T08:40:45.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.610+0000 7efcc13f4640 1 -- 172.21.15.38:0/1945646826 wait complete. 2024-04-25T08:40:45.614 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T08:40:45.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.722+0000 7f4e7317e640 1 Processor -- start 2024-04-25T08:40:45.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.722+0000 7f4e7317e640 1 -- start start 2024-04-25T08:40:45.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.723+0000 7f4e7317e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4e6c0cf470 0x7f4e6c168720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:45.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.723+0000 7f4e7317e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c0ccd60 con 0x7f4e6c06cd90 2024-04-25T08:40:45.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.723+0000 7f4e7317e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c0cd040 con 0x7f4e6c099a70 2024-04-25T08:40:45.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.723+0000 7f4e7317e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c1614d0 con 0x7f4e6c0cf470 2024-04-25T08:40:45.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.723+0000 7f4e6bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4e6c06cd90 0x7f4e6c0cc480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53404/0 (socket says 172.21.15.38:53404) 2024-04-25T08:40:45.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.723+0000 7f4e6bfff640 1 -- 172.21.15.38:0/2987143780 learned_addr learned my addr 172.21.15.38:0/2987143780 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:45.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.723+0000 7f4e7171b640 1 --2- 172.21.15.38:0/2987143780 >> v2:172.21.15.107:3300/0 conn(0x7f4e6c0cf470 0x7f4e6c168720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:45.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.723+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/2987143780 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1857417875 0 0) 0x7f4e6c0ccd60 con 0x7f4e6c06cd90 2024-04-25T08:40:45.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.723+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/2987143780 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e54003520 con 0x7f4e6c06cd90 2024-04-25T08:40:45.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.723+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/2987143780 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970333948 0 0) 0x7f4e6c0cd040 con 0x7f4e6c099a70 2024-04-25T08:40:45.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.723+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/2987143780 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e6c0ccd60 con 0x7f4e6c099a70 2024-04-25T08:40:45.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.724+0000 7f4e7171b640 1 -- 172.21.15.38:0/2987143780 >> v1:172.21.15.107:6789/0 conn(0x7f4e6c099a70 legacy=0x7f4e6c099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.724+0000 7f4e7171b640 1 -- 172.21.15.38:0/2987143780 >> v1:172.21.15.38:6789/0 conn(0x7f4e6c06cd90 legacy=0x7f4e6c0cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.724+0000 7f4e7171b640 1 -- 172.21.15.38:0/2987143780 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c0cd320 con 0x7f4e6c0cf470 2024-04-25T08:40:45.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.724+0000 7f4e7171b640 1 --2- 172.21.15.38:0/2987143780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6c0cf470 0x7f4e6c168720 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009d90 tx=0x7f4e6402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58091b471b98986c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:45.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.725+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/2987143780 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64007740 con 0x7f4e6c0cf470 2024-04-25T08:40:45.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.725+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/2987143780 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e640078e0 con 0x7f4e6c0cf470 2024-04-25T08:40:45.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.725+0000 7f4e6b7fe640 1 -- 172.21.15.38:0/2987143780 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64007cb0 con 0x7f4e6c0cf470 2024-04-25T08:40:45.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.725+0000 7f4e7317e640 1 -- 172.21.15.38:0/2987143780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6c0cf470 msgr2=0x7f4e6c168720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.725+0000 7f4e7317e640 1 --2- 172.21.15.38:0/2987143780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6c0cf470 0x7f4e6c168720 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009d90 tx=0x7f4e6402a6f0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.725+0000 7f4e7317e640 1 -- 172.21.15.38:0/2987143780 shutdown_connections 2024-04-25T08:40:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.725+0000 7f4e7317e640 1 --2- 172.21.15.38:0/2987143780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6c0cf470 0x7f4e6c168720 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.725+0000 7f4e7317e640 1 -- 172.21.15.38:0/2987143780 >> 172.21.15.38:0/2987143780 conn(0x7f4e6c0b3180 msgr2=0x7f4e6c0b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.725+0000 7f4e7317e640 1 -- 172.21.15.38:0/2987143780 shutdown_connections 2024-04-25T08:40:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.725+0000 7f4e7317e640 1 -- 172.21.15.38:0/2987143780 wait complete. 2024-04-25T08:40:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.726+0000 7f4e7317e640 1 Processor -- start 2024-04-25T08:40:45.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.726+0000 7f4e7317e640 1 -- start start 2024-04-25T08:40:45.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.727+0000 7f4e7317e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e6c06cd90 0x7f4e6c156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:45.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.727+0000 7f4e7317e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e6c099a70 0x7f4e6c1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:45.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.727+0000 7f4e7317e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6c0cf470 0x7f4e6c1f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:45.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.727+0000 7f4e7317e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0d0780 con 0x7f4e6c099a70 2024-04-25T08:40:45.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.727+0000 7f4e7317e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0cd970 con 0x7f4e6c0cf470 2024-04-25T08:40:45.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.727+0000 7f4e7317e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0d0900 con 0x7f4e6c06cd90 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.727+0000 7f4e6bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e6c099a70 0x7f4e6c1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.727+0000 7f4e6bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e6c099a70 0x7f4e6c1584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39474/0 (socket says 172.21.15.38:39474) 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.727+0000 7f4e6bfff640 1 -- 172.21.15.38:0/3809069555 learned_addr learned my addr 172.21.15.38:0/3809069555 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e70f1a640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e6c06cd90 0x7f4e6c156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e7171b640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6c0cf470 0x7f4e6c1f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e6bfff640 1 -- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e6c06cd90 msgr2=0x7f4e6c156c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e6bfff640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e6c06cd90 0x7f4e6c156c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e6bfff640 1 -- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6c0cf470 msgr2=0x7f4e6c1f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e6bfff640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6c0cf470 0x7f4e6c1f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e6bfff640 1 -- 172.21.15.38:0/3809069555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c0b53b0 con 0x7f4e6c099a70 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e7171b640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6c0cf470 0x7f4e6c1f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e6bfff640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e6c099a70 0x7f4e6c1584b0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000c4d0 tx=0x7f4e6000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e697fa640 1 -- 172.21.15.38:0/3809069555 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60017070 con 0x7f4e6c099a70 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e697fa640 1 -- 172.21.15.38:0/3809069555 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e60005ce0 con 0x7f4e6c099a70 2024-04-25T08:40:45.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e697fa640 1 -- 172.21.15.38:0/3809069555 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60007270 con 0x7f4e6c099a70 2024-04-25T08:40:45.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e7317e640 1 -- 172.21.15.38:0/3809069555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c0b5690 con 0x7f4e6c099a70 2024-04-25T08:40:45.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.728+0000 7f4e7317e640 1 -- 172.21.15.38:0/3809069555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c1576e0 con 0x7f4e6c099a70 2024-04-25T08:40:45.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.729+0000 7f4e7317e640 1 -- 172.21.15.38:0/3809069555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6c068e80 con 0x7f4e6c099a70 2024-04-25T08:40:45.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.730+0000 7f4e697fa640 1 -- 172.21.15.38:0/3809069555 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e6001a070 con 0x7f4e6c099a70 2024-04-25T08:40:45.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.730+0000 7f4e697fa640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e5806b4d0 0x7f4e5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:45.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.730+0000 7f4e697fa640 1 -- 172.21.15.38:0/3809069555 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4e60089330 con 0x7f4e6c099a70 2024-04-25T08:40:45.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.730+0000 7f4e70f1a640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e5806b4d0 0x7f4e5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:45.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.731+0000 7f4e70f1a640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e5806b4d0 0x7f4e5806d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c001650 tx=0x7f4e5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:45.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:45.734+0000 7f4e697fa640 1 -- 172.21.15.38:0/3809069555 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e60057cc0 con 0x7f4e6c099a70 2024-04-25T08:40:45.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.003+0000 7f4e7317e640 1 -- 172.21.15.38:0/3809069555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f4e6c160f00 con 0x7f4e6c099a70 2024-04-25T08:40:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.302+0000 7f4e697fa640 1 -- 172.21.15.38:0/3809069555 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v448) v1 ==== 166+0+0 (secure 0 0 0) 0x7f4e6005bd60 con 0x7f4e6c099a70 2024-04-25T08:40:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.304+0000 7f4e7317e640 1 -- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e5806b4d0 msgr2=0x7f4e5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.304+0000 7f4e7317e640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e5806b4d0 0x7f4e5806d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c001650 tx=0x7f4e5c009040 comp rx=0 tx=0).stop 2024-04-25T08:40:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.304+0000 7f4e7317e640 1 -- 172.21.15.38:0/3809069555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e6c099a70 msgr2=0x7f4e6c1584b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.304+0000 7f4e7317e640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e6c099a70 0x7f4e6c1584b0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f4e6000c4d0 tx=0x7f4e6000c990 comp rx=0 tx=0).stop 2024-04-25T08:40:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.305+0000 7f4e7317e640 1 -- 172.21.15.38:0/3809069555 shutdown_connections 2024-04-25T08:40:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.305+0000 7f4e7317e640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e6c0cf470 0x7f4e6c1f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.305+0000 7f4e7317e640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4e6c099a70 0x7f4e6c1584b0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.305+0000 7f4e7317e640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e5806b4d0 0x7f4e5806d990 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.305+0000 7f4e7317e640 1 --2- 172.21.15.38:0/3809069555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4e6c06cd90 0x7f4e6c156c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.305+0000 7f4e7317e640 1 -- 172.21.15.38:0/3809069555 >> 172.21.15.38:0/3809069555 conn(0x7f4e6c0b3180 msgr2=0x7f4e6c0ce030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.305+0000 7f4e7317e640 1 -- 172.21.15.38:0/3809069555 shutdown_connections 2024-04-25T08:40:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.305+0000 7f4e7317e640 1 -- 172.21.15.38:0/3809069555 wait complete. 2024-04-25T08:40:46.311 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T08:40:46.311 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:40:46.312 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:40:46.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.449+0000 7fd6c7f09640 1 Processor -- start 2024-04-25T08:40:46.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.449+0000 7fd6c7f09640 1 -- start start 2024-04-25T08:40:46.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c7f09640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6c00d0fe0 0x7fd6c00b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c7f09640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd6c00d2e50 0x7fd6c00b6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c7f09640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c00b8400 con 0x7fd6c015dc40 2024-04-25T08:40:46.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c7f09640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c00d19b0 con 0x7fd6c00d2e50 2024-04-25T08:40:46.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c7f09640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00b86e0 con 0x7fd6c00d0fe0 2024-04-25T08:40:46.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c64a6640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd6c00d2e50 0x7fd6c00b6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c54a4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6c00d0fe0 0x7fd6c00b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c5ca5640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fd6c015dc40 0x7fd6c00d08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:44436/0 (socket says 172.21.15.38:44436) 2024-04-25T08:40:46.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c5ca5640 1 -- 172.21.15.38:0/1475789419 learned_addr learned my addr 172.21.15.38:0/1475789419 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:46.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c64a6640 1 -- 172.21.15.38:0/1475789419 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6c00d0fe0 msgr2=0x7fd6c00b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:46.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c64a6640 1 --2- 172.21.15.38:0/1475789419 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6c00d0fe0 0x7fd6c00b6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c64a6640 1 -- 172.21.15.38:0/1475789419 >> v1:172.21.15.107:6789/0 conn(0x7fd6c015dc40 legacy=0x7fd6c00d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c4ca3640 1 -- 172.21.15.38:0/1475789419 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537580334 0 0) 0x7fd6c00b8400 con 0x7fd6c015dc40 2024-04-25T08:40:46.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c64a6640 1 -- 172.21.15.38:0/1475789419 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c00b89c0 con 0x7fd6c00d2e50 2024-04-25T08:40:46.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.450+0000 7fd6c54a4640 1 --2- 172.21.15.38:0/1475789419 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6c00d0fe0 0x7fd6c00b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:46.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.451+0000 7fd6c64a6640 1 --2- 172.21.15.38:0/1475789419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c00d2e50 0x7fd6c00b6fa0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009d90 tx=0x7fd6b802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=237127e0a61e100c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:46.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.451+0000 7fd6c4ca3640 1 -- 172.21.15.38:0/1475789419 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8031600 con 0x7fd6c00d2e50 2024-04-25T08:40:46.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.451+0000 7fd6c4ca3640 1 -- 172.21.15.38:0/1475789419 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b80317a0 con 0x7fd6c00d2e50 2024-04-25T08:40:46.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.451+0000 7fd6c4ca3640 1 -- 172.21.15.38:0/1475789419 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8031b70 con 0x7fd6c00d2e50 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.452+0000 7fd6c7f09640 1 -- 172.21.15.38:0/1475789419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c00d2e50 msgr2=0x7fd6c00b6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.452+0000 7fd6c7f09640 1 --2- 172.21.15.38:0/1475789419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c00d2e50 0x7fd6c00b6fa0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009d90 tx=0x7fd6b802a5d0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.452+0000 7fd6c7f09640 1 -- 172.21.15.38:0/1475789419 shutdown_connections 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.452+0000 7fd6c7f09640 1 --2- 172.21.15.38:0/1475789419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c00d2e50 0x7fd6c00b6fa0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.452+0000 7fd6c7f09640 1 --2- 172.21.15.38:0/1475789419 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6c00d0fe0 0x7fd6c00b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.452+0000 7fd6c7f09640 1 -- 172.21.15.38:0/1475789419 >> 172.21.15.38:0/1475789419 conn(0x7fd6c0066ce0 msgr2=0x7fd6c00670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.452+0000 7fd6c7f09640 1 -- 172.21.15.38:0/1475789419 shutdown_connections 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.452+0000 7fd6c7f09640 1 -- 172.21.15.38:0/1475789419 wait complete. 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.453+0000 7fd6c7f09640 1 Processor -- start 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.453+0000 7fd6c7f09640 1 -- start start 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c7f09640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6c00d0fe0 0x7fd6c014e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c7f09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6c00d2e50 0x7fd6c014ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c7f09640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c015dc40 0x7fd6c0151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c7f09640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00b9700 con 0x7fd6c00d2e50 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c7f09640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00b9580 con 0x7fd6c015dc40 2024-04-25T08:40:46.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c7f09640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c00b9880 con 0x7fd6c00d0fe0 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c54a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6c00d2e50 0x7fd6c014ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c54a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6c00d2e50 0x7fd6c014ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39480/0 (socket says 172.21.15.38:39480) 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c54a4640 1 -- 172.21.15.38:0/3559431140 learned_addr learned my addr 172.21.15.38:0/3559431140 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c64a6640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c015dc40 0x7fd6c0151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c5ca5640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6c00d0fe0 0x7fd6c014e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c54a4640 1 -- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6c00d0fe0 msgr2=0x7fd6c014e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c54a4640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6c00d0fe0 0x7fd6c014e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c54a4640 1 -- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c015dc40 msgr2=0x7fd6c0151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c54a4640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c015dc40 0x7fd6c0151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.454+0000 7fd6c54a4640 1 -- 172.21.15.38:0/3559431140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c014c840 con 0x7fd6c00d2e50 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.455+0000 7fd6c54a4640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6c00d2e50 0x7fd6c014ed30 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000ebe0 tx=0x7fd6b000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.455+0000 7fd6b67fc640 1 -- 172.21.15.38:0/3559431140 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0017070 con 0x7fd6c00d2e50 2024-04-25T08:40:46.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.455+0000 7fd6b67fc640 1 -- 172.21.15.38:0/3559431140 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b000cc30 con 0x7fd6c00d2e50 2024-04-25T08:40:46.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.455+0000 7fd6c7f09640 1 -- 172.21.15.38:0/3559431140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c014cae0 con 0x7fd6c00d2e50 2024-04-25T08:40:46.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.455+0000 7fd6b67fc640 1 -- 172.21.15.38:0/3559431140 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0013470 con 0x7fd6c00d2e50 2024-04-25T08:40:46.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.455+0000 7fd6c7f09640 1 -- 172.21.15.38:0/3559431140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c014d0a0 con 0x7fd6c00d2e50 2024-04-25T08:40:46.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.456+0000 7fd6c7f09640 1 -- 172.21.15.38:0/3559431140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c0068e80 con 0x7fd6c00d2e50 2024-04-25T08:40:46.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.456+0000 7fd6b67fc640 1 -- 172.21.15.38:0/3559431140 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6b001a070 con 0x7fd6c00d2e50 2024-04-25T08:40:46.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.457+0000 7fd6b67fc640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd6a406b4d0 0x7fd6a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.457+0000 7fd6b67fc640 1 -- 172.21.15.38:0/3559431140 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd6b008a2f0 con 0x7fd6c00d2e50 2024-04-25T08:40:46.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.457+0000 7fd6c5ca5640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd6a406b4d0 0x7fd6a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.458+0000 7fd6c5ca5640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd6a406b4d0 0x7fd6a406d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc007fd0 tx=0x7fd6bc007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:46.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.461+0000 7fd6b67fc640 1 -- 172.21.15.38:0/3559431140 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6b0058c90 con 0x7fd6c00d2e50 2024-04-25T08:40:46.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.644+0000 7fd6c7f09640 1 -- 172.21.15.38:0/3559431140 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6c00d4420 con 0x7fd6c00d2e50 2024-04-25T08:40:46.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.645+0000 7fd6b67fc640 1 -- 172.21.15.38:0/3559431140 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+13291 (secure 0 0 0) 0x7fd6b005cd30 con 0x7fd6c00d2e50 2024-04-25T08:40:46.638 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:46.638 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":124,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:40:45.281914+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:6837/901382672":"2024-04-26T08:40:42.106784+0000","172.21.15.107:6835/901382672":"2024-04-26T08:40:42.106784+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:40:46.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.647+0000 7fd6c7f09640 1 -- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd6a406b4d0 msgr2=0x7fd6a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.647+0000 7fd6c7f09640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd6a406b4d0 0x7fd6a406d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc007fd0 tx=0x7fd6bc007de0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.647+0000 7fd6c7f09640 1 -- 172.21.15.38:0/3559431140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6c00d2e50 msgr2=0x7fd6c014ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.647+0000 7fd6c7f09640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6c00d2e50 0x7fd6c014ed30 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000ebe0 tx=0x7fd6b000c520 comp rx=0 tx=0).stop 2024-04-25T08:40:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.647+0000 7fd6c7f09640 1 -- 172.21.15.38:0/3559431140 shutdown_connections 2024-04-25T08:40:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.647+0000 7fd6c7f09640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd6c015dc40 0x7fd6c0151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.647+0000 7fd6c7f09640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd6c00d2e50 0x7fd6c014ed30 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.647+0000 7fd6c7f09640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd6a406b4d0 0x7fd6a406d990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.648+0000 7fd6c7f09640 1 --2- 172.21.15.38:0/3559431140 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd6c00d0fe0 0x7fd6c014e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.648+0000 7fd6c7f09640 1 -- 172.21.15.38:0/3559431140 >> 172.21.15.38:0/3559431140 conn(0x7fd6c0066ce0 msgr2=0x7fd6c00b7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.648+0000 7fd6c7f09640 1 -- 172.21.15.38:0/3559431140 shutdown_connections 2024-04-25T08:40:46.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.648+0000 7fd6c7f09640 1 -- 172.21.15.38:0/3559431140 wait complete. 2024-04-25T08:40:46.651 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:40:46.652 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:40:46.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.762+0000 7fa9d6377640 1 Processor -- start 2024-04-25T08:40:46.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.762+0000 7fa9d6377640 1 -- start start 2024-04-25T08:40:46.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.763+0000 7fa9d6377640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d0166610 0x7fa9d0168a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.763+0000 7fa9d6377640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d00d2060 con 0x7fa9d00d0320 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.763+0000 7fa9d6377640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d00d2330 con 0x7fa9d00d33d0 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.763+0000 7fa9d6377640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00d2610 con 0x7fa9d0166610 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.763+0000 7fa9cffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa9d00d0320 0x7fa9d00d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53408/0 (socket says 172.21.15.38:53408) 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.763+0000 7fa9cffff640 1 -- 172.21.15.38:0/814847693 learned_addr learned my addr 172.21.15.38:0/814847693 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.763+0000 7fa9d4914640 1 --2- 172.21.15.38:0/814847693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d0166610 0x7fa9d0168a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454781641 0 0) 0x7fa9d00d2060 con 0x7fa9d00d0320 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9b8003520 con 0x7fa9d00d0320 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419693607 0 0) 0x7fa9d00d2330 con 0x7fa9d00d33d0 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9d00d2060 con 0x7fa9d00d33d0 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 956050249 0 0) 0x7fa9b8003520 con 0x7fa9d00d0320 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d0166610 msgr2=0x7fa9d0168a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:46.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 --2- 172.21.15.38:0/814847693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d0166610 0x7fa9d0168a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 >> v1:172.21.15.107:6789/0 conn(0x7fa9d00d33d0 legacy=0x7fa9d00d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d00d28f0 con 0x7fa9d00d0320 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fa9c4001690 con 0x7fa9d00d0320 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 >> v1:172.21.15.38:6789/0 conn(0x7fa9d00d0320 legacy=0x7fa9d00d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 --2- 172.21.15.38:0/814847693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9b8003b70 0x7fa9b8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 --2- 172.21.15.38:0/814847693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d0166610 0x7fa9b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 --2- 172.21.15.38:0/814847693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9b8004d40 0x7fa9b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00d2610 con 0x7fa9b8004d40 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b800ba70 con 0x7fa9b8003b70 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b800bc70 con 0x7fa9d0166610 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9d4914640 1 --2- 172.21.15.38:0/814847693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9b8004d40 0x7fa9b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9cffff640 1 --2- 172.21.15.38:0/814847693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9b8003b70 0x7fa9b8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9cf7fe640 1 --2- 172.21.15.38:0/814847693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d0166610 0x7fa9b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9d4914640 1 -- 172.21.15.38:0/814847693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d0166610 msgr2=0x7fa9b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9d4914640 1 --2- 172.21.15.38:0/814847693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d0166610 0x7fa9b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9d4914640 1 -- 172.21.15.38:0/814847693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9b8003b70 msgr2=0x7fa9b8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9d4914640 1 --2- 172.21.15.38:0/814847693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9b8003b70 0x7fa9b8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.764+0000 7fa9d4914640 1 -- 172.21.15.38:0/814847693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b8003520 con 0x7fa9b8004d40 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.765+0000 7fa9d4914640 1 --2- 172.21.15.38:0/814847693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9b8004d40 0x7fa9b80051c0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00ca820 tx=0x7fa9b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bab95982d275a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.765+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8037070 con 0x7fa9b8004d40 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.765+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c802e050 con 0x7fa9b8004d40 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.765+0000 7fa9ceffd640 1 -- 172.21.15.38:0/814847693 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8009c10 con 0x7fa9b8004d40 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.766+0000 7fa9d6377640 1 -- 172.21.15.38:0/814847693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9b8004d40 msgr2=0x7fa9b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.766+0000 7fa9d6377640 1 --2- 172.21.15.38:0/814847693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9b8004d40 0x7fa9b80051c0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00ca820 tx=0x7fa9b8005830 comp rx=0 tx=0).stop 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.766+0000 7fa9d4914640 1 -- 172.21.15.38:0/814847693 reap_dead start 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.766+0000 7fa9d6377640 1 -- 172.21.15.38:0/814847693 shutdown_connections 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.766+0000 7fa9d6377640 1 -- 172.21.15.38:0/814847693 >> 172.21.15.38:0/814847693 conn(0x7fa9d00668f0 msgr2=0x7fa9d0066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.766+0000 7fa9d6377640 1 -- 172.21.15.38:0/814847693 shutdown_connections 2024-04-25T08:40:46.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.766+0000 7fa9d6377640 1 -- 172.21.15.38:0/814847693 wait complete. 2024-04-25T08:40:46.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.767+0000 7fa9d6377640 1 Processor -- start 2024-04-25T08:40:46.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.767+0000 7fa9d6377640 1 -- start start 2024-04-25T08:40:46.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9d6377640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9d014ba00 0x7fa9d00ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9d6377640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d00b3270 0x7fa9d01599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9d6377640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d0159f20 0x7fa9d00b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9d6377640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d0161210 con 0x7fa9d014ba00 2024-04-25T08:40:46.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9d6377640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d00d2eb0 con 0x7fa9d0159f20 2024-04-25T08:40:46.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9d6377640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d0161390 con 0x7fa9d00b3270 2024-04-25T08:40:46.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9d014ba00 0x7fa9d00ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9cffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9d014ba00 0x7fa9d00ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39484/0 (socket says 172.21.15.38:39484) 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9cffff640 1 -- 172.21.15.38:0/2213347008 learned_addr learned my addr 172.21.15.38:0/2213347008 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9cf7fe640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d00b3270 0x7fa9d01599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9d4914640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d0159f20 0x7fa9d00b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9cffff640 1 -- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d00b3270 msgr2=0x7fa9d01599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9cffff640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d00b3270 0x7fa9d01599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9cffff640 1 -- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d0159f20 msgr2=0x7fa9d00b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9cffff640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d0159f20 0x7fa9d00b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9cffff640 1 -- 172.21.15.38:0/2213347008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d014bde0 con 0x7fa9d014ba00 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9cf7fe640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d00b3270 0x7fa9d01599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9d4914640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d0159f20 0x7fa9d00b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.768+0000 7fa9cffff640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9d014ba00 0x7fa9d00ccd70 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40017e0 tx=0x7fa9c40074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.769+0000 7fa9ccff9640 1 -- 172.21.15.38:0/2213347008 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400f750 con 0x7fa9d014ba00 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.769+0000 7fa9ccff9640 1 -- 172.21.15.38:0/2213347008 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c4018070 con 0x7fa9d014ba00 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.769+0000 7fa9d6377640 1 -- 172.21.15.38:0/2213347008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d00cc1a0 con 0x7fa9d014ba00 2024-04-25T08:40:46.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.769+0000 7fa9ccff9640 1 -- 172.21.15.38:0/2213347008 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400fb30 con 0x7fa9d014ba00 2024-04-25T08:40:46.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.769+0000 7fa9d6377640 1 -- 172.21.15.38:0/2213347008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d00cd530 con 0x7fa9d014ba00 2024-04-25T08:40:46.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.770+0000 7fa9d6377640 1 -- 172.21.15.38:0/2213347008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d00cb020 con 0x7fa9d014ba00 2024-04-25T08:40:46.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.770+0000 7fa9ccff9640 1 -- 172.21.15.38:0/2213347008 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9c400fcd0 con 0x7fa9d014ba00 2024-04-25T08:40:46.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.770+0000 7fa9ccff9640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa9bc06b500 0x7fa9bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:46.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.771+0000 7fa9ccff9640 1 -- 172.21.15.38:0/2213347008 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa9c408a020 con 0x7fa9d014ba00 2024-04-25T08:40:46.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.771+0000 7fa9cf7fe640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa9bc06b500 0x7fa9bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:46.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.771+0000 7fa9cf7fe640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa9bc06b500 0x7fa9bc06d9c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa9d015dbb0 tx=0x7fa9c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:46.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.775+0000 7fa9ccff9640 1 -- 172.21.15.38:0/2213347008 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9d00cb020 con 0x7fa9d014ba00 2024-04-25T08:40:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.953+0000 7fa9d6377640 1 -- 172.21.15.38:0/2213347008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa9d0068e80 con 0x7fa9d014ba00 2024-04-25T08:40:46.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.954+0000 7fa9ccff9640 1 -- 172.21.15.38:0/2213347008 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa9c4059010 con 0x7fa9d014ba00 2024-04-25T08:40:46.947 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:40:46.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.956+0000 7fa9d6377640 1 -- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa9bc06b500 msgr2=0x7fa9bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.956+0000 7fa9d6377640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa9bc06b500 0x7fa9bc06d9c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa9d015dbb0 tx=0x7fa9c0009040 comp rx=0 tx=0).stop 2024-04-25T08:40:46.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.956+0000 7fa9d6377640 1 -- 172.21.15.38:0/2213347008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9d014ba00 msgr2=0x7fa9d00ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:46.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.956+0000 7fa9d6377640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9d014ba00 0x7fa9d00ccd70 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40017e0 tx=0x7fa9c40074f0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.956+0000 7fa9d6377640 1 -- 172.21.15.38:0/2213347008 shutdown_connections 2024-04-25T08:40:46.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.956+0000 7fa9d6377640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa9d0159f20 0x7fa9d00b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.956+0000 7fa9d6377640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa9bc06b500 0x7fa9bc06d9c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.956+0000 7fa9d6377640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa9d00b3270 0x7fa9d01599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.956+0000 7fa9d6377640 1 --2- 172.21.15.38:0/2213347008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9d014ba00 0x7fa9d00ccd70 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:46.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.956+0000 7fa9d6377640 1 -- 172.21.15.38:0/2213347008 >> 172.21.15.38:0/2213347008 conn(0x7fa9d00668f0 msgr2=0x7fa9d00c9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:46.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.956+0000 7fa9d6377640 1 -- 172.21.15.38:0/2213347008 shutdown_connections 2024-04-25T08:40:46.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:46.956+0000 7fa9d6377640 1 -- 172.21.15.38:0/2213347008 wait complete. 2024-04-25T08:40:46.960 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T08:40:47.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.068+0000 7fcdaa74c640 1 Processor -- start 2024-04-25T08:40:47.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.068+0000 7fcdaa74c640 1 -- start start 2024-04-25T08:40:47.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcdaa74c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcda415d960 0x7fcda4161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:47.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcdaa74c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcda40b1e70 0x7fcda40cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:47.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcdaa74c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda40b43c0 con 0x7fcda406de70 2024-04-25T08:40:47.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcdaa74c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda40b5180 con 0x7fcda415d960 2024-04-25T08:40:47.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcdaa74c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda415dd40 con 0x7fcda40b1e70 2024-04-25T08:40:47.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcda3fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcda415d960 0x7fcda4161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:47.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcda37fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcda406de70 0x7fcda4161990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53412/0 (socket says 172.21.15.38:53412) 2024-04-25T08:40:47.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcda37fe640 1 -- 172.21.15.38:0/848928322 learned_addr learned my addr 172.21.15.38:0/848928322 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcda8ce9640 1 --2- 172.21.15.38:0/848928322 >> v2:172.21.15.107:3300/0 conn(0x7fcda40b1e70 0x7fcda40cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcda3fff640 1 -- 172.21.15.38:0/848928322 >> v2:172.21.15.107:3300/0 conn(0x7fcda40b1e70 msgr2=0x7fcda40cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcda3fff640 1 --2- 172.21.15.38:0/848928322 >> v2:172.21.15.107:3300/0 conn(0x7fcda40b1e70 0x7fcda40cd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcda3fff640 1 -- 172.21.15.38:0/848928322 >> v1:172.21.15.38:6789/0 conn(0x7fcda406de70 legacy=0x7fcda4161990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcda3fff640 1 -- 172.21.15.38:0/848928322 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda40b46a0 con 0x7fcda415d960 2024-04-25T08:40:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcda2ffd640 1 -- 172.21.15.38:0/848928322 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858693106 0 0) 0x7fcda40b43c0 con 0x7fcda406de70 2024-04-25T08:40:47.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.069+0000 7fcda8ce9640 1 --2- 172.21.15.38:0/848928322 >> v2:172.21.15.107:3300/0 conn(0x7fcda40b1e70 0x7fcda40cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.070+0000 7fcda3fff640 1 --2- 172.21.15.38:0/848928322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcda415d960 0x7fcda4161450 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fcda40cd640 tx=0x7fcd9802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c2de156dfa8f3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.070+0000 7fcda2ffd640 1 -- 172.21.15.38:0/848928322 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98037070 con 0x7fcda415d960 2024-04-25T08:40:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.070+0000 7fcda2ffd640 1 -- 172.21.15.38:0/848928322 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd98032440 con 0x7fcda415d960 2024-04-25T08:40:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.070+0000 7fcda2ffd640 1 -- 172.21.15.38:0/848928322 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9802ae60 con 0x7fcda415d960 2024-04-25T08:40:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.071+0000 7fcdaa74c640 1 -- 172.21.15.38:0/848928322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcda415d960 msgr2=0x7fcda4161450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:47.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.071+0000 7fcdaa74c640 1 --2- 172.21.15.38:0/848928322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcda415d960 0x7fcda4161450 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fcda40cd640 tx=0x7fcd9802a110 comp rx=0 tx=0).stop 2024-04-25T08:40:47.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.071+0000 7fcdaa74c640 1 -- 172.21.15.38:0/848928322 shutdown_connections 2024-04-25T08:40:47.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.071+0000 7fcdaa74c640 1 --2- 172.21.15.38:0/848928322 >> v2:172.21.15.107:3300/0 conn(0x7fcda40b1e70 0x7fcda40cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:47.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.071+0000 7fcdaa74c640 1 --2- 172.21.15.38:0/848928322 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcda415d960 0x7fcda4161450 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:47.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.071+0000 7fcdaa74c640 1 -- 172.21.15.38:0/848928322 >> 172.21.15.38:0/848928322 conn(0x7fcda406b210 msgr2=0x7fcda406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:47.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.071+0000 7fcdaa74c640 1 -- 172.21.15.38:0/848928322 shutdown_connections 2024-04-25T08:40:47.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.071+0000 7fcdaa74c640 1 -- 172.21.15.38:0/848928322 wait complete. 2024-04-25T08:40:47.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.072+0000 7fcdaa74c640 1 Processor -- start 2024-04-25T08:40:47.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.072+0000 7fcdaa74c640 1 -- start start 2024-04-25T08:40:47.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcdaa74c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda406de70 0x7fcda41f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:47.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcdaa74c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcda40b1e70 0x7fcda4160930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:47.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcdaa74c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcda415d960 0x7fcda415b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:47.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcdaa74c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda41620a0 con 0x7fcda415d960 2024-04-25T08:40:47.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcdaa74c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda40b4c60 con 0x7fcda406de70 2024-04-25T08:40:47.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcdaa74c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda40cddc0 con 0x7fcda40b1e70 2024-04-25T08:40:47.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda8ce9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcda415d960 0x7fcda415b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:47.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda8ce9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcda415d960 0x7fcda415b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39504/0 (socket says 172.21.15.38:39504) 2024-04-25T08:40:47.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda8ce9640 1 -- 172.21.15.38:0/1474944181 learned_addr learned my addr 172.21.15.38:0/1474944181 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:47.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda3fff640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda406de70 0x7fcda41f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda37fe640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcda40b1e70 0x7fcda4160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda8ce9640 1 -- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcda40b1e70 msgr2=0x7fcda4160930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda8ce9640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcda40b1e70 0x7fcda4160930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda8ce9640 1 -- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda406de70 msgr2=0x7fcda41f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda8ce9640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda406de70 0x7fcda41f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda8ce9640 1 -- 172.21.15.38:0/1474944181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda4159fc0 con 0x7fcda415d960 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda37fe640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcda40b1e70 0x7fcda4160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda8ce9640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcda415d960 0x7fcda415b670 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c007ef0 tx=0x7fcd9c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda0ff9640 1 -- 172.21.15.38:0/1474944181 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c01a070 con 0x7fcda415d960 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda0ff9640 1 -- 172.21.15.38:0/1474944181 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c003b90 con 0x7fcda415d960 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcda0ff9640 1 -- 172.21.15.38:0/1474944181 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c004030 con 0x7fcda415d960 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcdaa74c640 1 -- 172.21.15.38:0/1474944181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda415a2a0 con 0x7fcda415d960 2024-04-25T08:40:47.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.073+0000 7fcdaa74c640 1 -- 172.21.15.38:0/1474944181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda415a860 con 0x7fcda415d960 2024-04-25T08:40:47.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.075+0000 7fcdaa74c640 1 -- 172.21.15.38:0/1474944181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda4068e80 con 0x7fcda415d960 2024-04-25T08:40:47.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.075+0000 7fcda0ff9640 1 -- 172.21.15.38:0/1474944181 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd9c01d070 con 0x7fcda415d960 2024-04-25T08:40:47.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.075+0000 7fcda0ff9640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd9006b4d0 0x7fcd9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:47.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.076+0000 7fcda0ff9640 1 -- 172.21.15.38:0/1474944181 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcd9c08a230 con 0x7fcda415d960 2024-04-25T08:40:47.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.076+0000 7fcda3fff640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd9006b4d0 0x7fcd9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:47.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.076+0000 7fcda3fff640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd9006b4d0 0x7fcd9006d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcd98034000 tx=0x7fcd9802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:47.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.080+0000 7fcda0ff9640 1 -- 172.21.15.38:0/1474944181 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd9c010070 con 0x7fcda415d960 2024-04-25T08:40:47.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:47.330+0000 7fcdaa74c640 1 -- 172.21.15.38:0/1474944181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7fcda4160e70 con 0x7fcda415d960 2024-04-25T08:40:48.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.315+0000 7fcda0ff9640 1 -- 172.21.15.38:0/1474944181 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v450) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcd9c058bd0 con 0x7fcda415d960 2024-04-25T08:40:48.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.317+0000 7fcdaa74c640 1 -- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd9006b4d0 msgr2=0x7fcd9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:48.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.317+0000 7fcdaa74c640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd9006b4d0 0x7fcd9006d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcd98034000 tx=0x7fcd9802e000 comp rx=0 tx=0).stop 2024-04-25T08:40:48.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.317+0000 7fcdaa74c640 1 -- 172.21.15.38:0/1474944181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcda415d960 msgr2=0x7fcda415b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:48.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.317+0000 7fcdaa74c640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcda415d960 0x7fcda415b670 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c007ef0 tx=0x7fcd9c00d4a0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.317+0000 7fcdaa74c640 1 -- 172.21.15.38:0/1474944181 shutdown_connections 2024-04-25T08:40:48.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.317+0000 7fcdaa74c640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcda415d960 0x7fcda415b670 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.317+0000 7fcdaa74c640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd9006b4d0 0x7fcd9006d990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.317+0000 7fcdaa74c640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcda40b1e70 0x7fcda4160930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.317+0000 7fcdaa74c640 1 --2- 172.21.15.38:0/1474944181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcda406de70 0x7fcda41f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.317+0000 7fcdaa74c640 1 -- 172.21.15.38:0/1474944181 >> 172.21.15.38:0/1474944181 conn(0x7fcda406b210 msgr2=0x7fcda40b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:48.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.318+0000 7fcdaa74c640 1 -- 172.21.15.38:0/1474944181 shutdown_connections 2024-04-25T08:40:48.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.318+0000 7fcdaa74c640 1 -- 172.21.15.38:0/1474944181 wait complete. 2024-04-25T08:40:48.324 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:40:48.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.442+0000 7fbbae9c4640 1 Processor -- start 2024-04-25T08:40:48.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.442+0000 7fbbae9c4640 1 -- start start 2024-04-25T08:40:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbbae9c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbba80d33d0 0x7fbba80d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbbae9c4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbba80d1e90 0x7fbba80b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbbae9c4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba80d2330 con 0x7fbba80d0320 2024-04-25T08:40:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbbae9c4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba80d2610 con 0x7fbba80d1e90 2024-04-25T08:40:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbbae9c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba80d28f0 con 0x7fbba80d33d0 2024-04-25T08:40:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbba7fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fbba80d0320 0x7fbba80d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:53426/0 (socket says 172.21.15.38:53426) 2024-04-25T08:40:48.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbba7fff640 1 -- 172.21.15.38:0/3138101773 learned_addr learned my addr 172.21.15.38:0/3138101773 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:48.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbbacf61640 1 --2- 172.21.15.38:0/3138101773 >> v2:172.21.15.107:3300/0 conn(0x7fbba80d1e90 0x7fbba80b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbba77fe640 1 --2- 172.21.15.38:0/3138101773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbba80d33d0 0x7fbba80d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbba6ffd640 1 -- 172.21.15.38:0/3138101773 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227698884 0 0) 0x7fbba80d2330 con 0x7fbba80d0320 2024-04-25T08:40:48.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbba6ffd640 1 -- 172.21.15.38:0/3138101773 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb90003520 con 0x7fbba80d0320 2024-04-25T08:40:48.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbbacf61640 1 -- 172.21.15.38:0/3138101773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbba80d33d0 msgr2=0x7fbba80d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:48.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbba6ffd640 1 -- 172.21.15.38:0/3138101773 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3897637747 0 0) 0x7fbb90003520 con 0x7fbba80d0320 2024-04-25T08:40:48.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbbacf61640 1 --2- 172.21.15.38:0/3138101773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbba80d33d0 0x7fbba80d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbbacf61640 1 -- 172.21.15.38:0/3138101773 >> v1:172.21.15.38:6789/0 conn(0x7fbba80d0320 legacy=0x7fbba80d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:48.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbbacf61640 1 -- 172.21.15.38:0/3138101773 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba80d2bd0 con 0x7fbba80d1e90 2024-04-25T08:40:48.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.443+0000 7fbba77fe640 1 --2- 172.21.15.38:0/3138101773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbba80d33d0 0x7fbba80d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:48.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.444+0000 7fbbacf61640 1 --2- 172.21.15.38:0/3138101773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba80d1e90 0x7fbba80b6980 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009d90 tx=0x7fbba002a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d858da960190f6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.444+0000 7fbba6ffd640 1 -- 172.21.15.38:0/3138101773 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98001670 con 0x7fbba80d1e90 2024-04-25T08:40:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.444+0000 7fbba6ffd640 1 -- 172.21.15.38:0/3138101773 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba0007540 con 0x7fbba80d1e90 2024-04-25T08:40:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.444+0000 7fbba6ffd640 1 -- 172.21.15.38:0/3138101773 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0007910 con 0x7fbba80d1e90 2024-04-25T08:40:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.445+0000 7fbbae9c4640 1 -- 172.21.15.38:0/3138101773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba80d1e90 msgr2=0x7fbba80b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.445+0000 7fbbae9c4640 1 --2- 172.21.15.38:0/3138101773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba80d1e90 0x7fbba80b6980 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fbba0009d90 tx=0x7fbba002a680 comp rx=0 tx=0).stop 2024-04-25T08:40:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.445+0000 7fbbae9c4640 1 -- 172.21.15.38:0/3138101773 shutdown_connections 2024-04-25T08:40:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.445+0000 7fbbae9c4640 1 --2- 172.21.15.38:0/3138101773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba80d1e90 0x7fbba80b6980 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.445+0000 7fbbae9c4640 1 --2- 172.21.15.38:0/3138101773 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbba80d33d0 0x7fbba80d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.445+0000 7fbbae9c4640 1 -- 172.21.15.38:0/3138101773 >> 172.21.15.38:0/3138101773 conn(0x7fbba80668f0 msgr2=0x7fbba8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.445+0000 7fbbae9c4640 1 -- 172.21.15.38:0/3138101773 shutdown_connections 2024-04-25T08:40:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.445+0000 7fbbae9c4640 1 -- 172.21.15.38:0/3138101773 wait complete. 2024-04-25T08:40:48.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.446+0000 7fbbae9c4640 1 Processor -- start 2024-04-25T08:40:48.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.446+0000 7fbbae9c4640 1 -- start start 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbbae9c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba80d0320 0x7fbba814cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbbae9c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbba80d1e90 0x7fbba815a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbbae9c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbba80d33d0 0x7fbba815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbbae9c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8161390 con 0x7fbba80d1e90 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbbae9c4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8161210 con 0x7fbba80d0320 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbbae9c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba80b9720 con 0x7fbba80d33d0 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbba77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbba80d1e90 0x7fbba815a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbba77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbba80d1e90 0x7fbba815a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39518/0 (socket says 172.21.15.38:39518) 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbba77fe640 1 -- 172.21.15.38:0/1626202137 learned_addr learned my addr 172.21.15.38:0/1626202137 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbbacf61640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbba80d33d0 0x7fbba815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbba7fff640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba80d0320 0x7fbba814cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbba77fe640 1 -- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbba80d33d0 msgr2=0x7fbba815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:48.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbba77fe640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbba80d33d0 0x7fbba815aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbba77fe640 1 -- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba80d0320 msgr2=0x7fbba814cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbba77fe640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba80d0320 0x7fbba814cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.447+0000 7fbba77fe640 1 -- 172.21.15.38:0/1626202137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba80b4240 con 0x7fbba80d1e90 2024-04-25T08:40:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.448+0000 7fbba77fe640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbba80d1e90 0x7fbba815a580 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00e960 tx=0x7fbb9c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.448+0000 7fbba4ff9640 1 -- 172.21.15.38:0/1626202137 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c017070 con 0x7fbba80d1e90 2024-04-25T08:40:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.448+0000 7fbba4ff9640 1 -- 172.21.15.38:0/1626202137 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c00ca00 con 0x7fbba80d1e90 2024-04-25T08:40:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.448+0000 7fbbae9c4640 1 -- 172.21.15.38:0/1626202137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba80b4520 con 0x7fbba80d1e90 2024-04-25T08:40:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.448+0000 7fbba4ff9640 1 -- 172.21.15.38:0/1626202137 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c00cdd0 con 0x7fbba80d1e90 2024-04-25T08:40:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.448+0000 7fbbae9c4640 1 -- 172.21.15.38:0/1626202137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba80b4ae0 con 0x7fbba80d1e90 2024-04-25T08:40:48.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.449+0000 7fbbae9c4640 1 -- 172.21.15.38:0/1626202137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba8068e80 con 0x7fbba80d1e90 2024-04-25T08:40:48.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.449+0000 7fbba4ff9640 1 -- 172.21.15.38:0/1626202137 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb9c01a070 con 0x7fbba80d1e90 2024-04-25T08:40:48.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.450+0000 7fbba4ff9640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb9406b4d0 0x7fbb9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.450+0000 7fbba4ff9640 1 -- 172.21.15.38:0/1626202137 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbb9c08a270 con 0x7fbba80d1e90 2024-04-25T08:40:48.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.450+0000 7fbba7fff640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb9406b4d0 0x7fbb9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.451+0000 7fbba7fff640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb9406b4d0 0x7fbb9406d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbb98005880 tx=0x7fbb98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:48.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.454+0000 7fbba4ff9640 1 -- 172.21.15.38:0/1626202137 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb9c0592f0 con 0x7fbba80d1e90 2024-04-25T08:40:48.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.634+0000 7fbbae9c4640 1 -- 172.21.15.38:0/1626202137 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbba80b3050 con 0x7fbba80d1e90 2024-04-25T08:40:48.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.641+0000 7fbba4ff9640 1 -- 172.21.15.38:0/1626202137 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbb9c058bf0 con 0x7fbba80d1e90 2024-04-25T08:40:48.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.643+0000 7fbbae9c4640 1 -- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb9406b4d0 msgr2=0x7fbb9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:48.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.643+0000 7fbbae9c4640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb9406b4d0 0x7fbb9406d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fbb98005880 tx=0x7fbb98009040 comp rx=0 tx=0).stop 2024-04-25T08:40:48.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.643+0000 7fbbae9c4640 1 -- 172.21.15.38:0/1626202137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbba80d1e90 msgr2=0x7fbba815a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:48.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.643+0000 7fbbae9c4640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbba80d1e90 0x7fbba815a580 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c00e960 tx=0x7fbb9c00ee20 comp rx=0 tx=0).stop 2024-04-25T08:40:48.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.643+0000 7fbbae9c4640 1 -- 172.21.15.38:0/1626202137 shutdown_connections 2024-04-25T08:40:48.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.643+0000 7fbbae9c4640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbb9406b4d0 0x7fbb9406d990 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.643+0000 7fbbae9c4640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbba80d33d0 0x7fbba815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.643+0000 7fbbae9c4640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbba80d1e90 0x7fbba815a580 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.643+0000 7fbbae9c4640 1 --2- 172.21.15.38:0/1626202137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbba80d0320 0x7fbba814cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.643+0000 7fbbae9c4640 1 -- 172.21.15.38:0/1626202137 >> 172.21.15.38:0/1626202137 conn(0x7fbba80668f0 msgr2=0x7fbba80b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:48.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.643+0000 7fbbae9c4640 1 -- 172.21.15.38:0/1626202137 shutdown_connections 2024-04-25T08:40:48.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.643+0000 7fbbae9c4640 1 -- 172.21.15.38:0/1626202137 wait complete. 2024-04-25T08:40:48.647 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:40:48.647 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:40:48.689 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:40:48.739 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:40:48.739 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:40:48.739 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:40:48.755 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:40:48.755 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:40:48.755 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:40:48.772 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:40:48.772 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:40:48.772 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:40:48.811 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:40:48.811 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:40:48.811 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:40:48.829 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:40:48.829 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:40:48.829 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:48.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.961+0000 7fd20645d640 1 Processor -- start 2024-04-25T08:40:48.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.961+0000 7fd20645d640 1 -- start start 2024-04-25T08:40:48.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.962+0000 7fd20645d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd20015da90 0x7fd2000d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.962+0000 7fd20645d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd2000ceca0 0x7fd2000d38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.962+0000 7fd20645d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2000d3de0 0x7fd2000d0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.962+0000 7fd20645d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2000d4250 con 0x7fd2000ceca0 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.962+0000 7fd20645d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2000d2bc0 con 0x7fd20015da90 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.962+0000 7fd20645d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000d1b60 con 0x7fd2000d3de0 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.962+0000 7fd1ff7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd2000ceca0 0x7fd2000d38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.962+0000 7fd1ff7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fd2000ceca0 0x7fd2000d38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49442/0 (socket says 172.21.15.38:49442) 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.962+0000 7fd1ff7fe640 1 -- 172.21.15.38:0/2861319210 learned_addr learned my addr 172.21.15.38:0/2861319210 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.963+0000 7fd1fffff640 1 --2- 172.21.15.38:0/2861319210 >> v2:172.21.15.107:3300/0 conn(0x7fd20015da90 0x7fd2000d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.963+0000 7fd2049fa640 1 --2- 172.21.15.38:0/2861319210 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2000d3de0 0x7fd2000d0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.963+0000 7fd1ff7fe640 1 -- 172.21.15.38:0/2861319210 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2000d3de0 msgr2=0x7fd2000d0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.963+0000 7fd1ff7fe640 1 --2- 172.21.15.38:0/2861319210 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2000d3de0 0x7fd2000d0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.963+0000 7fd1ff7fe640 1 -- 172.21.15.38:0/2861319210 >> v2:172.21.15.107:3300/0 conn(0x7fd20015da90 msgr2=0x7fd2000d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.963+0000 7fd1ff7fe640 1 --2- 172.21.15.38:0/2861319210 >> v2:172.21.15.107:3300/0 conn(0x7fd20015da90 0x7fd2000d2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.963+0000 7fd1ff7fe640 1 -- 172.21.15.38:0/2861319210 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2000d1e00 con 0x7fd2000ceca0 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.963+0000 7fd1ff7fe640 1 --2- 172.21.15.38:0/2861319210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2000ceca0 0x7fd2000d38a0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fd2000d02f0 tx=0x7fd1f402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79cfac56fb3a1b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.964+0000 7fd1feffd640 1 -- 172.21.15.38:0/2861319210 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4037070 con 0x7fd2000ceca0 2024-04-25T08:40:48.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.964+0000 7fd1feffd640 1 -- 172.21.15.38:0/2861319210 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f4032440 con 0x7fd2000ceca0 2024-04-25T08:40:48.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.964+0000 7fd1feffd640 1 -- 172.21.15.38:0/2861319210 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f4032810 con 0x7fd2000ceca0 2024-04-25T08:40:48.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.964+0000 7fd20645d640 1 -- 172.21.15.38:0/2861319210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2000ceca0 msgr2=0x7fd2000d38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.964+0000 7fd20645d640 1 --2- 172.21.15.38:0/2861319210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2000ceca0 0x7fd2000d38a0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fd2000d02f0 tx=0x7fd1f402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.964+0000 7fd20645d640 1 -- 172.21.15.38:0/2861319210 shutdown_connections 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.964+0000 7fd20645d640 1 --2- 172.21.15.38:0/2861319210 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd2000d3de0 0x7fd2000d0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.964+0000 7fd20645d640 1 --2- 172.21.15.38:0/2861319210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2000ceca0 0x7fd2000d38a0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.964+0000 7fd20645d640 1 --2- 172.21.15.38:0/2861319210 >> v2:172.21.15.107:3300/0 conn(0x7fd20015da90 0x7fd2000d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.964+0000 7fd20645d640 1 -- 172.21.15.38:0/2861319210 >> 172.21.15.38:0/2861319210 conn(0x7fd20006bed0 msgr2=0x7fd20006c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.964+0000 7fd20645d640 1 -- 172.21.15.38:0/2861319210 shutdown_connections 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.964+0000 7fd20645d640 1 -- 172.21.15.38:0/2861319210 wait complete. 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.965+0000 7fd20645d640 1 Processor -- start 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.966+0000 7fd20645d640 1 -- start start 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.966+0000 7fd20645d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2000d3de0 0x7fd200158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.966+0000 7fd20645d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd20015da90 0x7fd200156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.966+0000 7fd20645d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd200157360 0x7fd2001f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.966+0000 7fd20645d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20015de70 con 0x7fd2000d3de0 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.966+0000 7fd20645d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000d23c0 con 0x7fd20015da90 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.966+0000 7fd20645d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20015dff0 con 0x7fd200157360 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.966+0000 7fd1fffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2000d3de0 0x7fd200158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.966+0000 7fd1fffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2000d3de0 0x7fd200158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49458/0 (socket says 172.21.15.38:49458) 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.966+0000 7fd1fffff640 1 -- 172.21.15.38:0/1041965047 learned_addr learned my addr 172.21.15.38:0/1041965047 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd1fffff640 1 -- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd200157360 msgr2=0x7fd2001f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd2049fa640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd200157360 0x7fd2001f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd1ff7fe640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd20015da90 0x7fd200156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd1fffff640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd200157360 0x7fd2001f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd1fffff640 1 -- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd20015da90 msgr2=0x7fd200156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd1fffff640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd20015da90 0x7fd200156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd1fffff640 1 -- 172.21.15.38:0/1041965047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd200159810 con 0x7fd2000d3de0 2024-04-25T08:40:48.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd1fffff640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2000d3de0 0x7fd200158ec0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000d5f0 tx=0x7fd1f000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:48.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd1fcff9640 1 -- 172.21.15.38:0/1041965047 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00125f0 con 0x7fd2000d3de0 2024-04-25T08:40:48.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd1fcff9640 1 -- 172.21.15.38:0/1041965047 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f0019070 con 0x7fd2000d3de0 2024-04-25T08:40:48.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd20645d640 1 -- 172.21.15.38:0/1041965047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd200159af0 con 0x7fd2000d3de0 2024-04-25T08:40:48.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd1fcff9640 1 -- 172.21.15.38:0/1041965047 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00129d0 con 0x7fd2000d3de0 2024-04-25T08:40:48.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.967+0000 7fd20645d640 1 -- 172.21.15.38:0/1041965047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20015a0b0 con 0x7fd2000d3de0 2024-04-25T08:40:48.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.968+0000 7fd20645d640 1 -- 172.21.15.38:0/1041965047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd200068e80 con 0x7fd2000d3de0 2024-04-25T08:40:48.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.968+0000 7fd1fcff9640 1 -- 172.21.15.38:0/1041965047 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1f0012b70 con 0x7fd2000d3de0 2024-04-25T08:40:48.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.969+0000 7fd1fcff9640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1ec06b4d0 0x7fd1ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:48.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.969+0000 7fd1fcff9640 1 -- 172.21.15.38:0/1041965047 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd1f008a480 con 0x7fd2000d3de0 2024-04-25T08:40:48.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.969+0000 7fd1ff7fe640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1ec06b4d0 0x7fd1ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:48.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.970+0000 7fd1ff7fe640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1ec06b4d0 0x7fd1ec06d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4034000 tx=0x7fd1f402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:48.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:48.973+0000 7fd1fcff9640 1 -- 172.21.15.38:0/1041965047 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1f0059470 con 0x7fd2000d3de0 2024-04-25T08:40:49.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.191+0000 7fd20645d640 1 -- 172.21.15.38:0/1041965047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd20015c3b0 con 0x7fd2000d3de0 2024-04-25T08:40:49.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.191+0000 7fd1fcff9640 1 -- 172.21.15.38:0/1041965047 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fd1f0058d70 con 0x7fd2000d3de0 2024-04-25T08:40:49.185 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:49.185 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6878,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6834/4018719594","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":4018719594},{"type":"v1","addr":"172.21.15.38:6834","nonce":4018719594}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":66},{"gid":6881,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/4043423739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6833","nonce":4043423739},{"type":"v1","addr":"172.21.15.38:6835","nonce":4043423739}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":7636,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3869428259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3869428259},{"type":"v1","addr":"172.21.15.107:6836","nonce":3869428259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67},{"gid":8056,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/1854851068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":1854851068},{"type":"v1","addr":"172.21.15.107:6837","nonce":1854851068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[]} 2024-04-25T08:40:49.185 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 72 2024-04-25T08:40:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.194+0000 7fd20645d640 1 -- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1ec06b4d0 msgr2=0x7fd1ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.194+0000 7fd20645d640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1ec06b4d0 0x7fd1ec06d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fd1f4034000 tx=0x7fd1f402e000 comp rx=0 tx=0).stop 2024-04-25T08:40:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.194+0000 7fd20645d640 1 -- 172.21.15.38:0/1041965047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2000d3de0 msgr2=0x7fd200158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.194+0000 7fd20645d640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2000d3de0 0x7fd200158ec0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000d5f0 tx=0x7fd1f000dab0 comp rx=0 tx=0).stop 2024-04-25T08:40:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.194+0000 7fd20645d640 1 -- 172.21.15.38:0/1041965047 shutdown_connections 2024-04-25T08:40:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.194+0000 7fd20645d640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd1ec06b4d0 0x7fd1ec06d990 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.194+0000 7fd20645d640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd200157360 0x7fd2001f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.194+0000 7fd20645d640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd20015da90 0x7fd200156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:49.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.194+0000 7fd20645d640 1 --2- 172.21.15.38:0/1041965047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd2000d3de0 0x7fd200158ec0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:49.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.194+0000 7fd20645d640 1 -- 172.21.15.38:0/1041965047 >> 172.21.15.38:0/1041965047 conn(0x7fd20006bed0 msgr2=0x7fd20015ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:49.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.194+0000 7fd20645d640 1 -- 172.21.15.38:0/1041965047 shutdown_connections 2024-04-25T08:40:49.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.194+0000 7fd20645d640 1 -- 172.21.15.38:0/1041965047 wait complete. 2024-04-25T08:40:49.198 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:40:49.198 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:40:49.198 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:40:49.198 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:49.198 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:40:49.199 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:40:49.199 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:40:49.199 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:49.199 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:40:49.199 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:40:49.199 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:40:49.199 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:49.200 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:40:49.200 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:40:49.200 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:40:49.200 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:40:49.208 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:40:49.208 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:40:49.209 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:40:49.209 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:40:49.214 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:40:49.214 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:40:49.215 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:40:49.215 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:40:49.251 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:40:49.251 DEBUG:teuthology.parallel:result is None 2024-04-25T08:40:49.251 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:40:49.252 DEBUG:teuthology.parallel:result is None 2024-04-25T08:40:49.257 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:40:49.257 DEBUG:teuthology.parallel:result is None 2024-04-25T08:40:49.258 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:40:49.258 DEBUG:teuthology.parallel:result is None 2024-04-25T08:40:49.258 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:40:49.327 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:40:49.328 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:40:49.338 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-25T08:40:49.339 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:40:49.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.373+0000 7f8c8389e640 1 Processor -- start 2024-04-25T08:40:49.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.373+0000 7f8c8389e640 1 -- start start 2024-04-25T08:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.374+0000 7f8c8389e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8c7c160ef0 0x7f8c7c0d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.374+0000 7f8c8389e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c7c0b9180 0x7f8c7c0d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.374+0000 7f8c8389e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c7c0b9610 con 0x7f8c7c0b2ab0 2024-04-25T08:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.374+0000 7f8c8389e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c7c161cf0 con 0x7f8c7c160ef0 2024-04-25T08:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.374+0000 7f8c8389e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c1612d0 con 0x7f8c7c0b9180 2024-04-25T08:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.374+0000 7f8c8163a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8c7c160ef0 0x7f8c7c0d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.374+0000 7f8c80e39640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c7c0b9180 0x7f8c7c0d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.374+0000 7f8c81e3b640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f8c7c0b2ab0 0x7f8c7c0d37f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:41904/0 (socket says 172.21.15.38:41904) 2024-04-25T08:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.374+0000 7f8c81e3b640 1 -- 172.21.15.38:0/2052078294 learned_addr learned my addr 172.21.15.38:0/2052078294 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.375+0000 7f8c73fff640 1 -- 172.21.15.38:0/2052078294 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 711423001 0 0) 0x7f8c7c0b9610 con 0x7f8c7c0b2ab0 2024-04-25T08:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.375+0000 7f8c73fff640 1 -- 172.21.15.38:0/2052078294 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c68003520 con 0x7f8c7c0b2ab0 2024-04-25T08:40:49.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.375+0000 7f8c8163a640 1 -- 172.21.15.38:0/2052078294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c7c0b9180 msgr2=0x7f8c7c0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:49.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.375+0000 7f8c8163a640 1 --2- 172.21.15.38:0/2052078294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c7c0b9180 0x7f8c7c0d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:49.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.375+0000 7f8c8163a640 1 -- 172.21.15.38:0/2052078294 >> v1:172.21.15.107:6789/0 conn(0x7f8c7c0b2ab0 legacy=0x7f8c7c0d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:49.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.375+0000 7f8c8163a640 1 -- 172.21.15.38:0/2052078294 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c1616a0 con 0x7f8c7c160ef0 2024-04-25T08:40:49.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.375+0000 7f8c80e39640 1 --2- 172.21.15.38:0/2052078294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c7c0b9180 0x7f8c7c0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:49.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.376+0000 7f8c8163a640 1 --2- 172.21.15.38:0/2052078294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c7c160ef0 0x7f8c7c0d07b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8c780097a0 tx=0x7f8c7802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14aadb43e7978482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:49.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.376+0000 7f8c73fff640 1 -- 172.21.15.38:0/2052078294 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78037070 con 0x7f8c7c160ef0 2024-04-25T08:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.376+0000 7f8c73fff640 1 -- 172.21.15.38:0/2052078294 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7802ae50 con 0x7f8c7c160ef0 2024-04-25T08:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.376+0000 7f8c73fff640 1 -- 172.21.15.38:0/2052078294 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78032660 con 0x7f8c7c160ef0 2024-04-25T08:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.376+0000 7f8c8389e640 1 -- 172.21.15.38:0/2052078294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c7c160ef0 msgr2=0x7f8c7c0d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.376+0000 7f8c8389e640 1 --2- 172.21.15.38:0/2052078294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c7c160ef0 0x7f8c7c0d07b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8c780097a0 tx=0x7f8c7802a100 comp rx=0 tx=0).stop 2024-04-25T08:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.376+0000 7f8c8389e640 1 -- 172.21.15.38:0/2052078294 shutdown_connections 2024-04-25T08:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.376+0000 7f8c8389e640 1 --2- 172.21.15.38:0/2052078294 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c7c0b9180 0x7f8c7c0d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.376+0000 7f8c8389e640 1 --2- 172.21.15.38:0/2052078294 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c7c160ef0 0x7f8c7c0d07b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.376+0000 7f8c8389e640 1 -- 172.21.15.38:0/2052078294 >> 172.21.15.38:0/2052078294 conn(0x7f8c7c066d10 msgr2=0x7f8c7c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.376+0000 7f8c8389e640 1 -- 172.21.15.38:0/2052078294 shutdown_connections 2024-04-25T08:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.376+0000 7f8c8389e640 1 -- 172.21.15.38:0/2052078294 wait complete. 2024-04-25T08:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.377+0000 7f8c8389e640 1 Processor -- start 2024-04-25T08:40:49.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.377+0000 7f8c8389e640 1 -- start start 2024-04-25T08:40:49.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.378+0000 7f8c8389e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c7c0b2ab0 0x7f8c7c1537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:49.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.378+0000 7f8c8389e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c7c160ef0 0x7f8c7c153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:49.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.378+0000 7f8c8389e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c7c15c4f0 0x7f8c7c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:49.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.378+0000 7f8c8389e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c162300 con 0x7f8c7c15c4f0 2024-04-25T08:40:49.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.378+0000 7f8c8389e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c162180 con 0x7f8c7c0b2ab0 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.378+0000 7f8c8389e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7c162480 con 0x7f8c7c160ef0 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.378+0000 7f8c81e3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c7c15c4f0 0x7f8c7c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.378+0000 7f8c81e3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c7c15c4f0 0x7f8c7c15c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49520/0 (socket says 172.21.15.38:49520) 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.378+0000 7f8c81e3b640 1 -- 172.21.15.38:0/468295032 learned_addr learned my addr 172.21.15.38:0/468295032 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c8163a640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c7c0b2ab0 0x7f8c7c1537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c80e39640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c7c160ef0 0x7f8c7c153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c81e3b640 1 -- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c7c160ef0 msgr2=0x7f8c7c153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c81e3b640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c7c160ef0 0x7f8c7c153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c81e3b640 1 -- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c7c0b2ab0 msgr2=0x7f8c7c1537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c81e3b640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c7c0b2ab0 0x7f8c7c1537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c81e3b640 1 -- 172.21.15.38:0/468295032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7c154510 con 0x7f8c7c15c4f0 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c80e39640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c7c160ef0 0x7f8c7c153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c81e3b640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c7c15c4f0 0x7f8c7c15c9a0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400cb70 tx=0x7f8c7400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:49.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c71ffb640 1 -- 172.21.15.38:0/468295032 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74017070 con 0x7f8c7c15c4f0 2024-04-25T08:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c71ffb640 1 -- 172.21.15.38:0/468295032 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7400ad40 con 0x7f8c7c15c4f0 2024-04-25T08:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c71ffb640 1 -- 172.21.15.38:0/468295032 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74012620 con 0x7f8c7c15c4f0 2024-04-25T08:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c8389e640 1 -- 172.21.15.38:0/468295032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7c1547f0 con 0x7f8c7c15c4f0 2024-04-25T08:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.379+0000 7f8c8389e640 1 -- 172.21.15.38:0/468295032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7c0c72f0 con 0x7f8c7c15c4f0 2024-04-25T08:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.380+0000 7f8c8389e640 1 -- 172.21.15.38:0/468295032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7c04ef90 con 0x7f8c7c15c4f0 2024-04-25T08:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.381+0000 7f8c71ffb640 1 -- 172.21.15.38:0/468295032 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c7401a070 con 0x7f8c7c15c4f0 2024-04-25T08:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.381+0000 7f8c71ffb640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c6006b4d0 0x7f8c6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.381+0000 7f8c71ffb640 1 -- 172.21.15.38:0/468295032 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8c74089e80 con 0x7f8c7c15c4f0 2024-04-25T08:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.381+0000 7f8c8163a640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c6006b4d0 0x7f8c6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:49.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.382+0000 7f8c8163a640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c6006b4d0 0x7f8c6006d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8c78034000 tx=0x7f8c7802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:49.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.385+0000 7f8c71ffb640 1 -- 172.21.15.38:0/468295032 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c74058f00 con 0x7f8c7c15c4f0 2024-04-25T08:40:49.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:49.578+0000 7f8c8389e640 1 -- 172.21.15.38:0/468295032 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8c7c154220 con 0x7f8c7c15c4f0 2024-04-25T08:40:50.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.340+0000 7f8c71ffb640 1 -- 172.21.15.38:0/468295032 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v126) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8c74058800 con 0x7f8c7c15c4f0 2024-04-25T08:40:50.334 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-04-25T08:40:50.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.345+0000 7f8c8389e640 1 -- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c6006b4d0 msgr2=0x7f8c6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.345+0000 7f8c8389e640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c6006b4d0 0x7f8c6006d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8c78034000 tx=0x7f8c7802e000 comp rx=0 tx=0).stop 2024-04-25T08:40:50.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.345+0000 7f8c8389e640 1 -- 172.21.15.38:0/468295032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c7c15c4f0 msgr2=0x7f8c7c15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.345+0000 7f8c8389e640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c7c15c4f0 0x7f8c7c15c9a0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400cb70 tx=0x7f8c7400a430 comp rx=0 tx=0).stop 2024-04-25T08:40:50.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.345+0000 7f8c8389e640 1 -- 172.21.15.38:0/468295032 shutdown_connections 2024-04-25T08:40:50.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.345+0000 7f8c8389e640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c7c15c4f0 0x7f8c7c15c9a0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.345+0000 7f8c8389e640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8c6006b4d0 0x7f8c6006d990 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.345+0000 7f8c8389e640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c7c160ef0 0x7f8c7c153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.345+0000 7f8c8389e640 1 --2- 172.21.15.38:0/468295032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c7c0b2ab0 0x7f8c7c1537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.345+0000 7f8c8389e640 1 -- 172.21.15.38:0/468295032 >> 172.21.15.38:0/468295032 conn(0x7f8c7c066d10 msgr2=0x7f8c7c0d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:50.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.345+0000 7f8c8389e640 1 -- 172.21.15.38:0/468295032 shutdown_connections 2024-04-25T08:40:50.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.345+0000 7f8c8389e640 1 -- 172.21.15.38:0/468295032 wait complete. 2024-04-25T08:40:50.353 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:40:50.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.468+0000 7fc3b9b7e640 1 Processor -- start 2024-04-25T08:40:50.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.468+0000 7fc3b9b7e640 1 -- start start 2024-04-25T08:40:50.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b9b7e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc3b4161700 0x7fc3b40b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b9b7e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b4161e00 con 0x7fc3b415ddf0 2024-04-25T08:40:50.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b9b7e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b41620e0 con 0x7fc3b40b5710 2024-04-25T08:40:50.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b9b7e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b40b9930 con 0x7fc3b4161700 2024-04-25T08:40:50.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b2ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc3b415ddf0 0x7fc3b40d2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51368/0 (socket says 172.21.15.38:51368) 2024-04-25T08:40:50.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b2ffd640 1 -- 172.21.15.38:0/3293014396 learned_addr learned my addr 172.21.15.38:0/3293014396 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:50.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b3fff640 1 --2- 172.21.15.38:0/3293014396 >> v2:172.21.15.107:3300/0 conn(0x7fc3b4161700 0x7fc3b40b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899599775 0 0) 0x7fc3b4161e00 con 0x7fc3b415ddf0 2024-04-25T08:40:50.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc39c003520 con 0x7fc3b415ddf0 2024-04-25T08:40:50.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1655876753 0 0) 0x7fc39c003520 con 0x7fc3b415ddf0 2024-04-25T08:40:50.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 >> v2:172.21.15.107:3300/0 conn(0x7fc3b4161700 msgr2=0x7fc3b40b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:50.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b27fc640 1 --2- 172.21.15.38:0/3293014396 >> v2:172.21.15.107:3300/0 conn(0x7fc3b4161700 0x7fc3b40b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 >> v1:172.21.15.107:6789/0 conn(0x7fc3b40b5710 legacy=0x7fc3b40b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b41623c0 con 0x7fc3b415ddf0 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.469+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fc3a8001670 con 0x7fc3b415ddf0 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 >> v1:172.21.15.38:6789/0 conn(0x7fc3b415ddf0 legacy=0x7fc3b40d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b27fc640 1 --2- 172.21.15.38:0/3293014396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc39c0039b0 0x7fc39c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b27fc640 1 --2- 172.21.15.38:0/3293014396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc39c0043b0 0x7fc39c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b27fc640 1 --2- 172.21.15.38:0/3293014396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc39c005080 0x7fc39c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c00bfb0 con 0x7fc39c005080 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c00c160 con 0x7fc39c0039b0 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c00c360 con 0x7fc39c0043b0 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b3fff640 1 --2- 172.21.15.38:0/3293014396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc39c005080 0x7fc39c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b37fe640 1 --2- 172.21.15.38:0/3293014396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc39c0039b0 0x7fc39c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b2ffd640 1 --2- 172.21.15.38:0/3293014396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc39c0043b0 0x7fc39c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b3fff640 1 -- 172.21.15.38:0/3293014396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc39c0043b0 msgr2=0x7fc39c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:50.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b3fff640 1 --2- 172.21.15.38:0/3293014396 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc39c0043b0 0x7fc39c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b3fff640 1 -- 172.21.15.38:0/3293014396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc39c0039b0 msgr2=0x7fc39c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b3fff640 1 --2- 172.21.15.38:0/3293014396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc39c0039b0 0x7fc39c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.470+0000 7fc3b3fff640 1 -- 172.21.15.38:0/3293014396 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc39c003520 con 0x7fc39c005080 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.471+0000 7fc3b3fff640 1 -- 172.21.15.38:0/3293014396 reap_dead start 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.471+0000 7fc3b3fff640 1 --2- 172.21.15.38:0/3293014396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc39c005080 0x7fc39c00b9c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009fc0 tx=0x7fc39c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ecd6e9fc712e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.471+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac035020 con 0x7fc39c005080 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.471+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ac003b90 con 0x7fc39c005080 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.471+0000 7fc3b27fc640 1 -- 172.21.15.38:0/3293014396 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ac009c10 con 0x7fc39c005080 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.471+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/3293014396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc39c005080 msgr2=0x7fc39c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.471+0000 7fc3b9b7e640 1 --2- 172.21.15.38:0/3293014396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc39c005080 0x7fc39c00b9c0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fc3ac009fc0 tx=0x7fc39c005830 comp rx=0 tx=0).stop 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.471+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/3293014396 shutdown_connections 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.471+0000 7fc3b9b7e640 1 --2- 172.21.15.38:0/3293014396 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc39c005080 0x7fc39c00b9c0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.472+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/3293014396 >> 172.21.15.38:0/3293014396 conn(0x7fc3b40d07c0 msgr2=0x7fc3b40d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:50.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.472+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/3293014396 shutdown_connections 2024-04-25T08:40:50.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.472+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/3293014396 wait complete. 2024-04-25T08:40:50.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.473+0000 7fc3b9b7e640 1 Processor -- start 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.473+0000 7fc3b9b7e640 1 -- start start 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b9b7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc39c005080 0x7fc3b41f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b9b7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b41f40f0 0x7fc3b40c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b9b7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3b414d470 0x7fc3b4159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b9b7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40b9780 con 0x7fc3b41f40f0 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b9b7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b40b9600 con 0x7fc39c005080 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b9b7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b4161bb0 con 0x7fc3b414d470 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b41f40f0 0x7fc3b40c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b41f40f0 0x7fc3b40c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49544/0 (socket says 172.21.15.38:49544) 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b2ffd640 1 -- 172.21.15.38:0/2100564740 learned_addr learned my addr 172.21.15.38:0/2100564740 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b37fe640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc39c005080 0x7fc3b41f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b3fff640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3b414d470 0x7fc3b4159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b2ffd640 1 -- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3b414d470 msgr2=0x7fc3b4159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b2ffd640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3b414d470 0x7fc3b4159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b2ffd640 1 -- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc39c005080 msgr2=0x7fc3b41f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b2ffd640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc39c005080 0x7fc3b41f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b2ffd640 1 -- 172.21.15.38:0/2100564740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b40c8780 con 0x7fc3b41f40f0 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b37fe640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc39c005080 0x7fc3b41f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.474+0000 7fc3b2ffd640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b41f40f0 0x7fc3b40c7e30 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80072f0 tx=0x7fc3a80077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.475+0000 7fc39bfff640 1 -- 172.21.15.38:0/2100564740 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8001620 con 0x7fc3b41f40f0 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.475+0000 7fc39bfff640 1 -- 172.21.15.38:0/2100564740 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a8018070 con 0x7fc3b41f40f0 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.475+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/2100564740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b41f3610 con 0x7fc3b41f40f0 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.475+0000 7fc39bfff640 1 -- 172.21.15.38:0/2100564740 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a800fc00 con 0x7fc3b41f40f0 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.475+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/2100564740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b40c8a60 con 0x7fc3b41f40f0 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.476+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/2100564740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b4068e80 con 0x7fc3b41f40f0 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.476+0000 7fc39bfff640 1 -- 172.21.15.38:0/2100564740 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3a801d460 con 0x7fc3b41f40f0 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.477+0000 7fc39bfff640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc3a006b4d0 0x7fc3a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.477+0000 7fc39bfff640 1 -- 172.21.15.38:0/2100564740 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc3a8089f60 con 0x7fc3b41f40f0 2024-04-25T08:40:50.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.477+0000 7fc3b37fe640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc3a006b4d0 0x7fc3a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.478+0000 7fc3b37fe640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc3a006b4d0 0x7fc3a006d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4158870 tx=0x7fc3a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:50.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.481+0000 7fc39bfff640 1 -- 172.21.15.38:0/2100564740 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3a8059030 con 0x7fc3b41f40f0 2024-04-25T08:40:50.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.729+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/2100564740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc3b41f33f0 con 0x7fc3b41f40f0 2024-04-25T08:40:50.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.729+0000 7fc39bfff640 1 -- 172.21.15.38:0/2100564740 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc3a8058930 con 0x7fc3b41f40f0 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmwq8CDBAA+cjd+QXtFgwZ3kxdABRWlA==", 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.723 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmPRFbFxAAojn2r/62vvf1qTy9QVuyIw==", 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmGLjqEhAAyPRgyy5iMgtccapydv4uOQ==", 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.724 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmt0AUHhAAoQyQ7wXBMGE+7Hi84tUfAg==", 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.725 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCxFCpmv+iTExAAEJihmsvBbp9xXxPnj49D6g==", 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC0FCpmdnQKFBAADLAsYZEUhoJx292jIYZroQ==", 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:40:50.726 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC3FCpm4f5BExAAMVSPkqsrnuLcO88AXKrGHQ==", 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC6FCpmHNN7EhAABPhmnxGo70ZOd8Oex62Mdw==", 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.727 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDAFCpmiThGFhAAG5Pyvn/JzL9NUoxHJkPMvg==", 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-04-25T08:40:50.728 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDDFCpmb/FHFBAA2cMhMqQQc4Fk6j8FpjhnaQ==", 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDGFCpmAZsSEhAAnGIqqcvuHMVSc8Zbk309kA==", 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.729 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDJFCpmqdVUEBAAb9abblyUztrjwxxe4jsAwQ==", 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw==", 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.730 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmL7eKKhAASeQEdHDBInfozdLVzP4DQw==", 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmH22HJxAAWyrooyJn5hRs/bb0Rn1HHw==", 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:40:50.731 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpm6Zr/LxAAxh9Y2gvvqj1jIaVVrQAvgw==", 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.732 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCpFCpmlj/fLBAAvxLluFQFR8uhppGlNpaygA==", 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcXHhCBAAvI5zmWidVTvOmEcVRWasEA==", 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:40:50.733 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.734 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.734 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:40:50.734 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:50.734 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-04-25T08:40:50.734 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcr96BRAAQwUPxILJukYERAiEXeur+g==", 2024-04-25T08:40:50.734 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:40:50.734 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:40:50.734 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:40:50.734 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:40:50.734 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.734 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:50.734 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T08:40:50.735 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:40:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.731+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc3a006b4d0 msgr2=0x7fc3a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.731+0000 7fc3b9b7e640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc3a006b4d0 0x7fc3a006d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4158870 tx=0x7fc3a4009040 comp rx=0 tx=0).stop 2024-04-25T08:40:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.731+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/2100564740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b41f40f0 msgr2=0x7fc3b40c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.731+0000 7fc3b9b7e640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b41f40f0 0x7fc3b40c7e30 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80072f0 tx=0x7fc3a80077b0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.731+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/2100564740 shutdown_connections 2024-04-25T08:40:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.731+0000 7fc3b9b7e640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc3a006b4d0 0x7fc3a006d990 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.731+0000 7fc3b9b7e640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3b414d470 0x7fc3b4159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.731+0000 7fc3b9b7e640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3b41f40f0 0x7fc3b40c7e30 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.731+0000 7fc3b9b7e640 1 --2- 172.21.15.38:0/2100564740 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc39c005080 0x7fc3b41f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.731+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/2100564740 >> 172.21.15.38:0/2100564740 conn(0x7fc3b40d07c0 msgr2=0x7fc3b40d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.732+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/2100564740 shutdown_connections 2024-04-25T08:40:50.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.732+0000 7fc3b9b7e640 1 -- 172.21.15.38:0/2100564740 wait complete. 2024-04-25T08:40:50.737 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:40:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.846+0000 7f2e04e4d640 1 Processor -- start 2024-04-25T08:40:50.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.846+0000 7f2e04e4d640 1 -- start start 2024-04-25T08:40:50.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2e04e4d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2e0006c110 0x7f2e0006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2e04e4d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2e000ce2e0 0x7f2e000d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2e04e4d640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0006ca30 con 0x7f2e0015df50 2024-04-25T08:40:50.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2e04e4d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e000cf2d0 con 0x7f2e000ce2e0 2024-04-25T08:40:50.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2e04e4d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e000ceff0 con 0x7f2e0006c110 2024-04-25T08:40:50.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2dfed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2e000ce2e0 0x7f2e000d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2df7fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2e0015df50 0x7f2e000ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51380/0 (socket says 172.21.15.38:51380) 2024-04-25T08:40:50.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2df7fff640 1 -- 172.21.15.38:0/3953477682 learned_addr learned my addr 172.21.15.38:0/3953477682 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:50.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2dfed9d640 1 -- 172.21.15.38:0/3953477682 >> v2:172.21.15.107:3300/0 conn(0x7f2e0006c110 msgr2=0x7f2e0006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:50.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2dfe59c640 1 --2- 172.21.15.38:0/3953477682 >> v2:172.21.15.107:3300/0 conn(0x7f2e0006c110 0x7f2e0006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2dfdd9b640 1 -- 172.21.15.38:0/3953477682 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1381573109 0 0) 0x7f2e0006ca30 con 0x7f2e0015df50 2024-04-25T08:40:50.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2dfed9d640 1 --2- 172.21.15.38:0/3953477682 >> v2:172.21.15.107:3300/0 conn(0x7f2e0006c110 0x7f2e0006c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2dfed9d640 1 -- 172.21.15.38:0/3953477682 >> v1:172.21.15.38:6789/0 conn(0x7f2e0015df50 legacy=0x7f2e000ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.847+0000 7f2dfed9d640 1 -- 172.21.15.38:0/3953477682 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e000d1f70 con 0x7f2e000ce2e0 2024-04-25T08:40:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.848+0000 7f2dfed9d640 1 --2- 172.21.15.38:0/3953477682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e000ce2e0 0x7f2e000d2a50 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f2e0008c550 tx=0x7f2df802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbc066acfa2ba6e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.848+0000 7f2dfdd9b640 1 -- 172.21.15.38:0/3953477682 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8007740 con 0x7f2e000ce2e0 2024-04-25T08:40:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.848+0000 7f2dfdd9b640 1 -- 172.21.15.38:0/3953477682 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df80078e0 con 0x7f2e000ce2e0 2024-04-25T08:40:50.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.848+0000 7f2dfdd9b640 1 -- 172.21.15.38:0/3953477682 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8007cb0 con 0x7f2e000ce2e0 2024-04-25T08:40:50.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.848+0000 7f2e04e4d640 1 -- 172.21.15.38:0/3953477682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e000ce2e0 msgr2=0x7f2e000d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.848+0000 7f2e04e4d640 1 --2- 172.21.15.38:0/3953477682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e000ce2e0 0x7f2e000d2a50 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f2e0008c550 tx=0x7f2df802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.849+0000 7f2e04e4d640 1 -- 172.21.15.38:0/3953477682 shutdown_connections 2024-04-25T08:40:50.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.849+0000 7f2e04e4d640 1 --2- 172.21.15.38:0/3953477682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e000ce2e0 0x7f2e000d2a50 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.849+0000 7f2e04e4d640 1 --2- 172.21.15.38:0/3953477682 >> v2:172.21.15.107:3300/0 conn(0x7f2e0006c110 0x7f2e0006c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.849+0000 7f2e04e4d640 1 -- 172.21.15.38:0/3953477682 >> 172.21.15.38:0/3953477682 conn(0x7f2e000d3fb0 msgr2=0x7f2e000d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:50.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.849+0000 7f2e04e4d640 1 -- 172.21.15.38:0/3953477682 shutdown_connections 2024-04-25T08:40:50.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.849+0000 7f2e04e4d640 1 -- 172.21.15.38:0/3953477682 wait complete. 2024-04-25T08:40:50.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.850+0000 7f2e04e4d640 1 Processor -- start 2024-04-25T08:40:50.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.850+0000 7f2e04e4d640 1 -- start start 2024-04-25T08:40:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2e04e4d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0006c110 0x7f2e001f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2e04e4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e000ce2e0 0x7f2e00151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2e04e4d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0015df50 0x7f2e000c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2e04e4d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000d2f90 con 0x7f2e000ce2e0 2024-04-25T08:40:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2e04e4d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000d2530 con 0x7f2e0015df50 2024-04-25T08:40:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2e04e4d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000d3110 con 0x7f2e0006c110 2024-04-25T08:40:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e000ce2e0 0x7f2e00151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e000ce2e0 0x7f2e00151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49562/0 (socket says 172.21.15.38:49562) 2024-04-25T08:40:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2df7fff640 1 -- 172.21.15.38:0/782550094 learned_addr learned my addr 172.21.15.38:0/782550094 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2dfed9d640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0015df50 0x7f2e000c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2dfe59c640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0006c110 0x7f2e001f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2df7fff640 1 -- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0006c110 msgr2=0x7f2e001f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2df7fff640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0006c110 0x7f2e001f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2df7fff640 1 -- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0015df50 msgr2=0x7f2e000c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2df7fff640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0015df50 0x7f2e000c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2df7fff640 1 -- 172.21.15.38:0/782550094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e000c8930 con 0x7f2e000ce2e0 2024-04-25T08:40:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2dfed9d640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0015df50 0x7f2e000c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2df7fff640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e000ce2e0 0x7f2e00151660 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f2de800c500 tx=0x7f2de800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2df6ffd640 1 -- 172.21.15.38:0/782550094 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8018070 con 0x7f2e000ce2e0 2024-04-25T08:40:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2df6ffd640 1 -- 172.21.15.38:0/782550094 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de8007580 con 0x7f2e000ce2e0 2024-04-25T08:40:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2e04e4d640 1 -- 172.21.15.38:0/782550094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e001f7b40 con 0x7f2e000ce2e0 2024-04-25T08:40:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.851+0000 7f2df6ffd640 1 -- 172.21.15.38:0/782550094 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de8004a20 con 0x7f2e000ce2e0 2024-04-25T08:40:50.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.852+0000 7f2e04e4d640 1 -- 172.21.15.38:0/782550094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e001f3a50 con 0x7f2e000ce2e0 2024-04-25T08:40:50.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.853+0000 7f2e04e4d640 1 -- 172.21.15.38:0/782550094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e00068e80 con 0x7f2e000ce2e0 2024-04-25T08:40:50.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.853+0000 7f2df6ffd640 1 -- 172.21.15.38:0/782550094 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2de801b070 con 0x7f2e000ce2e0 2024-04-25T08:40:50.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.853+0000 7f2df6ffd640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2de006b4d0 0x7f2de006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:50.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.853+0000 7f2df6ffd640 1 -- 172.21.15.38:0/782550094 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2de8089300 con 0x7f2e000ce2e0 2024-04-25T08:40:50.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.854+0000 7f2dfe59c640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2de006b4d0 0x7f2de006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:50.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.854+0000 7f2dfe59c640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2de006b4d0 0x7f2de006d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2df0006fd0 tx=0x7f2df0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:50.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:50.857+0000 7f2df6ffd640 1 -- 172.21.15.38:0/782550094 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e000c8930 con 0x7f2e000ce2e0 2024-04-25T08:40:51.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.041+0000 7f2e04e4d640 1 -- 172.21.15.38:0/782550094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e00067ef0 con 0x7f2e000ce2e0 2024-04-25T08:40:51.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.042+0000 7f2df6ffd640 1 -- 172.21.15.38:0/782550094 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+13162 (secure 0 0 0) 0x7f2de80583d0 con 0x7f2e000ce2e0 2024-04-25T08:40:51.035 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:51.035 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":128,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:40:50.354075+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:40:51.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.044+0000 7f2e04e4d640 1 -- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2de006b4d0 msgr2=0x7f2de006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.044+0000 7f2e04e4d640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2de006b4d0 0x7f2de006d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2df0006fd0 tx=0x7f2df0008040 comp rx=0 tx=0).stop 2024-04-25T08:40:51.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.044+0000 7f2e04e4d640 1 -- 172.21.15.38:0/782550094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e000ce2e0 msgr2=0x7f2e00151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.044+0000 7f2e04e4d640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e000ce2e0 0x7f2e00151660 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f2de800c500 tx=0x7f2de800c9c0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.044+0000 7f2e04e4d640 1 -- 172.21.15.38:0/782550094 shutdown_connections 2024-04-25T08:40:51.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.044+0000 7f2e04e4d640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e0015df50 0x7f2e000c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.044+0000 7f2e04e4d640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e000ce2e0 0x7f2e00151660 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.044+0000 7f2e04e4d640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2de006b4d0 0x7f2de006d990 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.044+0000 7f2e04e4d640 1 --2- 172.21.15.38:0/782550094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e0006c110 0x7f2e001f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.044+0000 7f2e04e4d640 1 -- 172.21.15.38:0/782550094 >> 172.21.15.38:0/782550094 conn(0x7f2e000d3fb0 msgr2=0x7f2e0006c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:51.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.044+0000 7f2e04e4d640 1 -- 172.21.15.38:0/782550094 shutdown_connections 2024-04-25T08:40:51.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.044+0000 7f2e04e4d640 1 -- 172.21.15.38:0/782550094 wait complete. 2024-04-25T08:40:51.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:40:51.049 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:40:51.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.159+0000 7f0cdbf0a640 1 Processor -- start 2024-04-25T08:40:51.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.160+0000 7f0cdbf0a640 1 -- start start 2024-04-25T08:40:51.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.160+0000 7f0cdbf0a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0cd40d0320 0x7f0cd40d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.160+0000 7f0cdbf0a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cd40d0700 0x7f0cd40d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.160+0000 7f0cdbf0a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cd40d2330 con 0x7f0cd40d1e90 2024-04-25T08:40:51.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.160+0000 7f0cdbf0a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cd40d2610 con 0x7f0cd40d0320 2024-04-25T08:40:51.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.160+0000 7f0cdbf0a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40d28f0 con 0x7f0cd40d0700 2024-04-25T08:40:51.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.161+0000 7f0cd9ca6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0cd40d0320 0x7f0cd40d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.161+0000 7f0cda4a7640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0cd40d1e90 0x7f0cd40b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51388/0 (socket says 172.21.15.38:51388) 2024-04-25T08:40:51.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.161+0000 7f0cd9ca6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0cd40d0320 0x7f0cd40d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49572/0 (socket says 172.21.15.38:49572) 2024-04-25T08:40:51.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.161+0000 7f0cda4a7640 1 -- 172.21.15.38:0/2314312196 learned_addr learned my addr 172.21.15.38:0/2314312196 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:51.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.161+0000 7f0cd94a5640 1 --2- 172.21.15.38:0/2314312196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cd40d0700 0x7f0cd40d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.161+0000 7f0cd9ca6640 1 -- 172.21.15.38:0/2314312196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cd40d0700 msgr2=0x7f0cd40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:51.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.161+0000 7f0cd9ca6640 1 --2- 172.21.15.38:0/2314312196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cd40d0700 0x7f0cd40d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.161+0000 7f0cd8ca4640 1 -- 172.21.15.38:0/2314312196 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2743905798 0 0) 0x7f0cd40d2330 con 0x7f0cd40d1e90 2024-04-25T08:40:51.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.161+0000 7f0cd9ca6640 1 -- 172.21.15.38:0/2314312196 >> v1:172.21.15.38:6789/0 conn(0x7f0cd40d1e90 legacy=0x7f0cd40b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.161+0000 7f0cd9ca6640 1 -- 172.21.15.38:0/2314312196 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd40d2bd0 con 0x7f0cd40d0320 2024-04-25T08:40:51.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.162+0000 7f0cd9ca6640 1 --2- 172.21.15.38:0/2314312196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd40d0320 0x7f0cd40d33d0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00097a0 tx=0x7f0cd002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c33258853001eb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:51.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.162+0000 7f0cd8ca4640 1 -- 172.21.15.38:0/2314312196 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0037070 con 0x7f0cd40d0320 2024-04-25T08:40:51.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.162+0000 7f0cd8ca4640 1 -- 172.21.15.38:0/2314312196 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0032440 con 0x7f0cd40d0320 2024-04-25T08:40:51.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.162+0000 7f0cd8ca4640 1 -- 172.21.15.38:0/2314312196 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0032810 con 0x7f0cd40d0320 2024-04-25T08:40:51.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.162+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2314312196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd40d0320 msgr2=0x7f0cd40d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.162+0000 7f0cdbf0a640 1 --2- 172.21.15.38:0/2314312196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd40d0320 0x7f0cd40d33d0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00097a0 tx=0x7f0cd002a210 comp rx=0 tx=0).stop 2024-04-25T08:40:51.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.162+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2314312196 shutdown_connections 2024-04-25T08:40:51.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.162+0000 7f0cdbf0a640 1 --2- 172.21.15.38:0/2314312196 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cd40d0700 0x7f0cd40d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.162+0000 7f0cdbf0a640 1 --2- 172.21.15.38:0/2314312196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd40d0320 0x7f0cd40d33d0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.162+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2314312196 >> 172.21.15.38:0/2314312196 conn(0x7f0cd40668f0 msgr2=0x7f0cd4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:51.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.162+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2314312196 shutdown_connections 2024-04-25T08:40:51.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.162+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2314312196 wait complete. 2024-04-25T08:40:51.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.164+0000 7f0cdbf0a640 1 Processor -- start 2024-04-25T08:40:51.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.164+0000 7f0cdbf0a640 1 -- start start 2024-04-25T08:40:51.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.164+0000 7f0cdbf0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd40d0320 0x7f0cd40b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.164+0000 7f0cdbf0a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cd40d0700 0x7f0cd4153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.164+0000 7f0cdbf0a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cd40d1e90 0x7f0cd415f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.164+0000 7f0cdbf0a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd40d0b40 con 0x7f0cd40d0320 2024-04-25T08:40:51.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.164+0000 7f0cdbf0a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd4161210 con 0x7f0cd40d0700 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.164+0000 7f0cdbf0a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cd41561a0 con 0x7f0cd40d1e90 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.164+0000 7f0cd9ca6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd40d0320 0x7f0cd40b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cd9ca6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd40d0320 0x7f0cd40b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49588/0 (socket says 172.21.15.38:49588) 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cd9ca6640 1 -- 172.21.15.38:0/2066784796 learned_addr learned my addr 172.21.15.38:0/2066784796 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cda4a7640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cd40d1e90 0x7f0cd415f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cd94a5640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cd40d0700 0x7f0cd4153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cd9ca6640 1 -- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cd40d1e90 msgr2=0x7f0cd415f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cd9ca6640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cd40d1e90 0x7f0cd415f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cd9ca6640 1 -- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cd40d0700 msgr2=0x7f0cd4153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cd9ca6640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cd40d0700 0x7f0cd4153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cd9ca6640 1 -- 172.21.15.38:0/2066784796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd4155100 con 0x7f0cd40d0320 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cd9ca6640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd40d0320 0x7f0cd40b5d90 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0000c00 tx=0x7f0cd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cca7fc640 1 -- 172.21.15.38:0/2066784796 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0037040 con 0x7f0cd40d0320 2024-04-25T08:40:51.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cca7fc640 1 -- 172.21.15.38:0/2066784796 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0042070 con 0x7f0cd40d0320 2024-04-25T08:40:51.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2066784796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cd41553e0 con 0x7f0cd40d0320 2024-04-25T08:40:51.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cca7fc640 1 -- 172.21.15.38:0/2066784796 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0030940 con 0x7f0cd40d0320 2024-04-25T08:40:51.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.165+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2066784796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cd41559a0 con 0x7f0cd40d0320 2024-04-25T08:40:51.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.166+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2066784796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cd4068e80 con 0x7f0cd40d0320 2024-04-25T08:40:51.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.167+0000 7f0cca7fc640 1 -- 172.21.15.38:0/2066784796 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0cd0030ae0 con 0x7f0cd40d0320 2024-04-25T08:40:51.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.167+0000 7f0cca7fc640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0cb806b4d0 0x7f0cb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.167+0000 7f0cca7fc640 1 -- 172.21.15.38:0/2066784796 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0cd00aa900 con 0x7f0cd40d0320 2024-04-25T08:40:51.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.168+0000 7f0cd94a5640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0cb806b4d0 0x7f0cb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.168+0000 7f0cd94a5640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0cb806b4d0 0x7f0cb806d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0cd415fd90 tx=0x7f0cc4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:51.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.171+0000 7f0cca7fc640 1 -- 172.21.15.38:0/2066784796 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0cd0079a60 con 0x7f0cd40d0320 2024-04-25T08:40:51.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.355+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2066784796 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0cd40737d0 con 0x7f0cd40d0320 2024-04-25T08:40:51.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.356+0000 7f0cca7fc640 1 -- 172.21.15.38:0/2066784796 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0cd0079360 con 0x7f0cd40d0320 2024-04-25T08:40:51.349 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:40:51.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.358+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0cb806b4d0 msgr2=0x7f0cb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.358+0000 7f0cdbf0a640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0cb806b4d0 0x7f0cb806d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0cd415fd90 tx=0x7f0cc4008040 comp rx=0 tx=0).stop 2024-04-25T08:40:51.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.358+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2066784796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd40d0320 msgr2=0x7f0cd40b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.358+0000 7f0cdbf0a640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd40d0320 0x7f0cd40b5d90 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0000c00 tx=0x7f0cd00046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.358+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2066784796 shutdown_connections 2024-04-25T08:40:51.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.358+0000 7f0cdbf0a640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0cb806b4d0 0x7f0cb806d990 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.358+0000 7f0cdbf0a640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0cd40d1e90 0x7f0cd415f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.358+0000 7f0cdbf0a640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0cd40d0700 0x7f0cd4153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.358+0000 7f0cdbf0a640 1 --2- 172.21.15.38:0/2066784796 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cd40d0320 0x7f0cd40b5d90 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.358+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2066784796 >> 172.21.15.38:0/2066784796 conn(0x7f0cd40668f0 msgr2=0x7f0cd415e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:51.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.358+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2066784796 shutdown_connections 2024-04-25T08:40:51.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.358+0000 7f0cdbf0a640 1 -- 172.21.15.38:0/2066784796 wait complete. 2024-04-25T08:40:51.362 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:40:51.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.471+0000 7efd4105d640 1 Processor -- start 2024-04-25T08:40:51.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.471+0000 7efd4105d640 1 -- start start 2024-04-25T08:40:51.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd4105d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd3c06c110 0x7efd3c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd4105d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efd3c0ce2e0 0x7efd3c0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd4105d640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3c06ca30 con 0x7efd3c15df50 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd4105d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c0cf2d0 con 0x7efd3c0ce2e0 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd4105d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0ceff0 con 0x7efd3c06c110 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd3b59e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efd3c0ce2e0 0x7efd3c0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd3b59e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efd3c0ce2e0 0x7efd3c0d2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49602/0 (socket says 172.21.15.38:49602) 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd3b59e640 1 -- 172.21.15.38:0/622722466 learned_addr learned my addr 172.21.15.38:0/622722466 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd3ad9d640 1 --2- 172.21.15.38:0/622722466 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd3c06c110 0x7efd3c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd3b59e640 1 -- 172.21.15.38:0/622722466 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd3c06c110 msgr2=0x7efd3c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd3b59e640 1 --2- 172.21.15.38:0/622722466 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd3c06c110 0x7efd3c06c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd3b59e640 1 -- 172.21.15.38:0/622722466 >> v1:172.21.15.107:6789/0 conn(0x7efd3c15df50 legacy=0x7efd3c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.472+0000 7efd3b59e640 1 -- 172.21.15.38:0/622722466 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c0d1f70 con 0x7efd3c0ce2e0 2024-04-25T08:40:51.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.473+0000 7efd3b59e640 1 --2- 172.21.15.38:0/622722466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c0ce2e0 0x7efd3c0d2a50 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7efd34009e50 tx=0x7efd3402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3eeec4825fce0c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:51.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.473+0000 7efd3a59c640 1 -- 172.21.15.38:0/622722466 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34036070 con 0x7efd3c0ce2e0 2024-04-25T08:40:51.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.473+0000 7efd3a59c640 1 -- 172.21.15.38:0/622722466 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34031600 con 0x7efd3c0ce2e0 2024-04-25T08:40:51.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.473+0000 7efd3a59c640 1 -- 172.21.15.38:0/622722466 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd340319d0 con 0x7efd3c0ce2e0 2024-04-25T08:40:51.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.473+0000 7efd4105d640 1 -- 172.21.15.38:0/622722466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c0ce2e0 msgr2=0x7efd3c0d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.473+0000 7efd4105d640 1 --2- 172.21.15.38:0/622722466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c0ce2e0 0x7efd3c0d2a50 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7efd34009e50 tx=0x7efd3402a710 comp rx=0 tx=0).stop 2024-04-25T08:40:51.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.474+0000 7efd4105d640 1 -- 172.21.15.38:0/622722466 shutdown_connections 2024-04-25T08:40:51.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.474+0000 7efd4105d640 1 --2- 172.21.15.38:0/622722466 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c0ce2e0 0x7efd3c0d2a50 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.474+0000 7efd4105d640 1 --2- 172.21.15.38:0/622722466 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd3c06c110 0x7efd3c06c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.474+0000 7efd4105d640 1 -- 172.21.15.38:0/622722466 >> 172.21.15.38:0/622722466 conn(0x7efd3c0d3fb0 msgr2=0x7efd3c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:51.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.474+0000 7efd4105d640 1 -- 172.21.15.38:0/622722466 shutdown_connections 2024-04-25T08:40:51.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.474+0000 7efd4105d640 1 -- 172.21.15.38:0/622722466 wait complete. 2024-04-25T08:40:51.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.475+0000 7efd4105d640 1 Processor -- start 2024-04-25T08:40:51.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.475+0000 7efd4105d640 1 -- start start 2024-04-25T08:40:51.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd4105d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd3c06c110 0x7efd3c1f6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd4105d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd3c0ce2e0 0x7efd3c1f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd4105d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c15df50 0x7efd3c1f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd4105d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0d2f90 con 0x7efd3c15df50 2024-04-25T08:40:51.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd4105d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0d2530 con 0x7efd3c06c110 2024-04-25T08:40:51.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd4105d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0d3110 con 0x7efd3c0ce2e0 2024-04-25T08:40:51.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd3b59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c15df50 0x7efd3c1f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd3b59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c15df50 0x7efd3c1f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49604/0 (socket says 172.21.15.38:49604) 2024-04-25T08:40:51.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd3b59e640 1 -- 172.21.15.38:0/4255327148 learned_addr learned my addr 172.21.15.38:0/4255327148 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd3ad9d640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd3c06c110 0x7efd3c1f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd33fff640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd3c0ce2e0 0x7efd3c1f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd3b59e640 1 -- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd3c0ce2e0 msgr2=0x7efd3c1f4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd3b59e640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd3c0ce2e0 0x7efd3c1f4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd3b59e640 1 -- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd3c06c110 msgr2=0x7efd3c1f6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd3b59e640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd3c06c110 0x7efd3c1f6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd3b59e640 1 -- 172.21.15.38:0/4255327148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c1f7d40 con 0x7efd3c15df50 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.476+0000 7efd3b59e640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c15df50 0x7efd3c1f6f60 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7efd34034040 tx=0x7efd34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.477+0000 7efd337fe640 1 -- 172.21.15.38:0/4255327148 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34036050 con 0x7efd3c15df50 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.477+0000 7efd337fe640 1 -- 172.21.15.38:0/4255327148 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd34042070 con 0x7efd3c15df50 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.477+0000 7efd4105d640 1 -- 172.21.15.38:0/4255327148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c0cab60 con 0x7efd3c15df50 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.477+0000 7efd337fe640 1 -- 172.21.15.38:0/4255327148 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34009510 con 0x7efd3c15df50 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.477+0000 7efd4105d640 1 -- 172.21.15.38:0/4255327148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c0cb120 con 0x7efd3c15df50 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.478+0000 7efd4105d640 1 -- 172.21.15.38:0/4255327148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c068e80 con 0x7efd3c15df50 2024-04-25T08:40:51.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.478+0000 7efd337fe640 1 -- 172.21.15.38:0/4255327148 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd34007b30 con 0x7efd3c15df50 2024-04-25T08:40:51.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.478+0000 7efd337fe640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd1c06b500 0x7efd1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.478+0000 7efd337fe640 1 -- 172.21.15.38:0/4255327148 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efd340aae70 con 0x7efd3c15df50 2024-04-25T08:40:51.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.479+0000 7efd3ad9d640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd1c06b500 0x7efd1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.479+0000 7efd3ad9d640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd1c06b500 0x7efd1c06d9c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efd2c006fd0 tx=0x7efd2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:51.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.482+0000 7efd337fe640 1 -- 172.21.15.38:0/4255327148 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd34007e20 con 0x7efd3c15df50 2024-04-25T08:40:51.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.665+0000 7efd4105d640 1 -- 172.21.15.38:0/4255327148 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efd3c073cd0 con 0x7efd3c15df50 2024-04-25T08:40:51.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.665+0000 7efd337fe640 1 -- 172.21.15.38:0/4255327148 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) v1 ==== 84+0+65 (secure 0 0 0) 0x7efd34079eb0 con 0x7efd3c15df50 2024-04-25T08:40:51.658 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:51.658 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T08:40:51.658 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:40:51.658 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-04-25T08:40:51.658 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-04-25T08:40:51.659 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:40:51.659 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T08:40:51.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.667+0000 7efd4105d640 1 -- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd1c06b500 msgr2=0x7efd1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.667+0000 7efd4105d640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd1c06b500 0x7efd1c06d9c0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7efd2c006fd0 tx=0x7efd2c008040 comp rx=0 tx=0).stop 2024-04-25T08:40:51.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.667+0000 7efd4105d640 1 -- 172.21.15.38:0/4255327148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c15df50 msgr2=0x7efd3c1f6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.667+0000 7efd4105d640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c15df50 0x7efd3c1f6f60 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7efd34034040 tx=0x7efd34004a40 comp rx=0 tx=0).stop 2024-04-25T08:40:51.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.667+0000 7efd4105d640 1 -- 172.21.15.38:0/4255327148 shutdown_connections 2024-04-25T08:40:51.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.667+0000 7efd4105d640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd3c15df50 0x7efd3c1f6f60 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.667+0000 7efd4105d640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd1c06b500 0x7efd1c06d9c0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.667+0000 7efd4105d640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd3c0ce2e0 0x7efd3c1f4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.667+0000 7efd4105d640 1 --2- 172.21.15.38:0/4255327148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd3c06c110 0x7efd3c1f6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.667+0000 7efd4105d640 1 -- 172.21.15.38:0/4255327148 >> 172.21.15.38:0/4255327148 conn(0x7efd3c0d3fb0 msgr2=0x7efd3c073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:51.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.667+0000 7efd4105d640 1 -- 172.21.15.38:0/4255327148 shutdown_connections 2024-04-25T08:40:51.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.667+0000 7efd4105d640 1 -- 172.21.15.38:0/4255327148 wait complete. 2024-04-25T08:40:51.671 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:40:51.671 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:40:51.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.783+0000 7f52d8d09640 1 Processor -- start 2024-04-25T08:40:51.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.783+0000 7f52d8d09640 1 -- start start 2024-04-25T08:40:51.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d8d09640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d406eab0 0x7f52d40cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d8d09640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d415abb0 con 0x7f52d40cf7a0 2024-04-25T08:40:51.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d8d09640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d415ae70 con 0x7f52d40b47b0 2024-04-25T08:40:51.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d8d09640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d415b7a0 con 0x7f52d406eab0 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d2d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f52d40cf7a0 0x7f52d415e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51404/0 (socket says 172.21.15.38:51404) 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d2d9d640 1 -- 172.21.15.38:0/505836012 learned_addr learned my addr 172.21.15.38:0/505836012 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52cbfff640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d406eab0 0x7f52d40cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177373167 0 0) 0x7f52d415abb0 con 0x7f52d40cf7a0 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52c0005730 con 0x7f52d40cf7a0 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4171492007 0 0) 0x7f52c0005730 con 0x7f52d40cf7a0 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d406eab0 msgr2=0x7f52d40cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d1d9b640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d406eab0 0x7f52d40cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 >> v1:172.21.15.107:6789/0 conn(0x7f52d40b47b0 legacy=0x7f52d406e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d415b150 con 0x7f52d40cf7a0 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.784+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f52cc0019f0 con 0x7f52d40cf7a0 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 >> v1:172.21.15.38:6789/0 conn(0x7f52d40cf7a0 legacy=0x7f52d415e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52d1d9b640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d406eab0 0x7f52c0005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52d1d9b640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c0006130 0x7f52c0006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52d1d9b640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c0006f50 0x7f52c00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d415b7a0 con 0x7f52c0006130 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c000dc80 con 0x7f52c0006f50 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c000de80 con 0x7f52d406eab0 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52cbfff640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c0006130 0x7f52c0006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52d2d9d640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c0006f50 0x7f52c00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52d259c640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d406eab0 0x7f52c0005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52cbfff640 1 -- 172.21.15.38:0/505836012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d406eab0 msgr2=0x7f52c0005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52cbfff640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d406eab0 0x7f52c0005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52cbfff640 1 -- 172.21.15.38:0/505836012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c0006f50 msgr2=0x7f52c00073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52cbfff640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52c0006f50 0x7f52c00073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52cbfff640 1 -- 172.21.15.38:0/505836012 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c0005730 con 0x7f52c0006130 2024-04-25T08:40:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.785+0000 7f52d259c640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d406eab0 0x7f52c0005bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:51.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.786+0000 7f52cbfff640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c0006130 0x7f52c0006a10 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009ac0 tx=0x7f52c0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596abc0977149990 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:51.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.786+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc036070 con 0x7f52c0006130 2024-04-25T08:40:51.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.786+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc031400 con 0x7f52c0006130 2024-04-25T08:40:51.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.786+0000 7f52d1d9b640 1 -- 172.21.15.38:0/505836012 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc009690 con 0x7f52c0006130 2024-04-25T08:40:51.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.786+0000 7f52d8d09640 1 -- 172.21.15.38:0/505836012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c0006130 msgr2=0x7f52c0006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.786+0000 7f52d8d09640 1 --2- 172.21.15.38:0/505836012 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c0006130 0x7f52c0006a10 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009ac0 tx=0x7f52c0007a40 comp rx=0 tx=0).stop 2024-04-25T08:40:51.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.786+0000 7f52d2d9d640 1 -- 172.21.15.38:0/505836012 reap_dead start 2024-04-25T08:40:51.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.786+0000 7f52d8d09640 1 -- 172.21.15.38:0/505836012 shutdown_connections 2024-04-25T08:40:51.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.786+0000 7f52d8d09640 1 -- 172.21.15.38:0/505836012 >> 172.21.15.38:0/505836012 conn(0x7f52d4066f20 msgr2=0x7f52d406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:51.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.786+0000 7f52d8d09640 1 -- 172.21.15.38:0/505836012 shutdown_connections 2024-04-25T08:40:51.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.786+0000 7f52d8d09640 1 -- 172.21.15.38:0/505836012 wait complete. 2024-04-25T08:40:51.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.788+0000 7f52d8d09640 1 Processor -- start 2024-04-25T08:40:51.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.788+0000 7f52d8d09640 1 -- start start 2024-04-25T08:40:51.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.788+0000 7f52d8d09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c0006130 0x7f52d40b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.788+0000 7f52d8d09640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d4144910 0x7f52d4144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.788+0000 7f52d8d09640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52d41534e0 0x7f52d4153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.788+0000 7f52d8d09640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40cfc30 con 0x7f52c0006130 2024-04-25T08:40:51.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.788+0000 7f52d8d09640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d406d000 con 0x7f52d41534e0 2024-04-25T08:40:51.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.788+0000 7f52d8d09640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41e09d0 con 0x7f52d4144910 2024-04-25T08:40:51.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c0006130 0x7f52d40b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c0006130 0x7f52d40b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49620/0 (socket says 172.21.15.38:49620) 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d259c640 1 -- 172.21.15.38:0/2773429522 learned_addr learned my addr 172.21.15.38:0/2773429522 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52cbfff640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d4144910 0x7f52d4144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d2d9d640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52d41534e0 0x7f52d4153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d259c640 1 -- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d4144910 msgr2=0x7f52d4144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d259c640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d4144910 0x7f52d4144d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d259c640 1 -- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52d41534e0 msgr2=0x7f52d4153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d259c640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52d41534e0 0x7f52d4153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d259c640 1 -- 172.21.15.38:0/2773429522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d41e0c20 con 0x7f52c0006130 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52cbfff640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d4144910 0x7f52d4144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d2d9d640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52d41534e0 0x7f52d4153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d259c640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c0006130 0x7f52d40b2c30 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f52c4007b40 tx=0x7f52c400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52caffd640 1 -- 172.21.15.38:0/2773429522 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40050d0 con 0x7f52c0006130 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52caffd640 1 -- 172.21.15.38:0/2773429522 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c4018070 con 0x7f52c0006130 2024-04-25T08:40:51.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d8d09640 1 -- 172.21.15.38:0/2773429522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d41e0f00 con 0x7f52c0006130 2024-04-25T08:40:51.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52caffd640 1 -- 172.21.15.38:0/2773429522 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c400c6c0 con 0x7f52c0006130 2024-04-25T08:40:51.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.789+0000 7f52d8d09640 1 -- 172.21.15.38:0/2773429522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d41e14c0 con 0x7f52c0006130 2024-04-25T08:40:51.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.790+0000 7f52d8d09640 1 -- 172.21.15.38:0/2773429522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d4145310 con 0x7f52c0006130 2024-04-25T08:40:51.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.791+0000 7f52caffd640 1 -- 172.21.15.38:0/2773429522 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52c400c860 con 0x7f52c0006130 2024-04-25T08:40:51.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.791+0000 7f52caffd640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52b406b4d0 0x7f52b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:51.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.791+0000 7f52caffd640 1 -- 172.21.15.38:0/2773429522 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f52c4089e40 con 0x7f52c0006130 2024-04-25T08:40:51.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.792+0000 7f52cbfff640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52b406b4d0 0x7f52b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:51.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.792+0000 7f52cbfff640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52b406b4d0 0x7f52b406d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009ac0 tx=0x7f52bc008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:51.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.795+0000 7f52caffd640 1 -- 172.21.15.38:0/2773429522 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f52d4145310 con 0x7f52c0006130 2024-04-25T08:40:51.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:51.976+0000 7f52d8d09640 1 -- 172.21.15.38:0/2773429522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f52d4068e80 con 0x7f52c0006130 2024-04-25T08:40:52.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.352+0000 7f52caffd640 1 -- 172.21.15.38:0/2773429522 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v129) v1 ==== 138+0+0 (secure 0 0 0) 0x7f52c4058e80 con 0x7f52c0006130 2024-04-25T08:40:52.345 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-04-25T08:40:52.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.354+0000 7f52d8d09640 1 -- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52b406b4d0 msgr2=0x7f52b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:52.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.354+0000 7f52d8d09640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52b406b4d0 0x7f52b406d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009ac0 tx=0x7f52bc008030 comp rx=0 tx=0).stop 2024-04-25T08:40:52.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.354+0000 7f52d8d09640 1 -- 172.21.15.38:0/2773429522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c0006130 msgr2=0x7f52d40b2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:52.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.354+0000 7f52d8d09640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c0006130 0x7f52d40b2c30 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f52c4007b40 tx=0x7f52c400ec60 comp rx=0 tx=0).stop 2024-04-25T08:40:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.354+0000 7f52d8d09640 1 -- 172.21.15.38:0/2773429522 shutdown_connections 2024-04-25T08:40:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.354+0000 7f52d8d09640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52d41534e0 0x7f52d4153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.354+0000 7f52d8d09640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f52b406b4d0 0x7f52b406d990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.354+0000 7f52d8d09640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f52d4144910 0x7f52d4144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.354+0000 7f52d8d09640 1 --2- 172.21.15.38:0/2773429522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c0006130 0x7f52d40b2c30 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.354+0000 7f52d8d09640 1 -- 172.21.15.38:0/2773429522 >> 172.21.15.38:0/2773429522 conn(0x7f52d4066f20 msgr2=0x7f52d40af180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.354+0000 7f52d8d09640 1 -- 172.21.15.38:0/2773429522 shutdown_connections 2024-04-25T08:40:52.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.354+0000 7f52d8d09640 1 -- 172.21.15.38:0/2773429522 wait complete. 2024-04-25T08:40:52.364 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:40:52.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.482+0000 7f33fd0fc640 1 Processor -- start 2024-04-25T08:40:52.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.482+0000 7f33fd0fc640 1 -- start start 2024-04-25T08:40:52.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33fd0fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d0320 0x7f33f80d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:52.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33fd0fc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f33f80b6980 0x7f33f80b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:52.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33fd0fc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f80d2060 con 0x7f33f80d0700 2024-04-25T08:40:52.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33fd0fc640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f80d2330 con 0x7f33f80b6980 2024-04-25T08:40:52.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33fd0fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80d2610 con 0x7f33f80d0320 2024-04-25T08:40:52.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33f659c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f33f80d0700 0x7f33f80d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51420/0 (socket says 172.21.15.38:51420) 2024-04-25T08:40:52.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33f659c640 1 -- 172.21.15.38:0/1595351490 learned_addr learned my addr 172.21.15.38:0/1595351490 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:52.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33f759e640 1 --2- 172.21.15.38:0/1595351490 >> v2:172.21.15.107:3300/0 conn(0x7f33f80b6980 0x7f33f80b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:52.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33f6d9d640 1 --2- 172.21.15.38:0/1595351490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d0320 0x7f33f80d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:52.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33f5d9b640 1 -- 172.21.15.38:0/1595351490 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661024723 0 0) 0x7f33f80d2060 con 0x7f33f80d0700 2024-04-25T08:40:52.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33f5d9b640 1 -- 172.21.15.38:0/1595351490 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e0003520 con 0x7f33f80d0700 2024-04-25T08:40:52.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33f759e640 1 -- 172.21.15.38:0/1595351490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d0320 msgr2=0x7f33f80d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:52.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33f759e640 1 --2- 172.21.15.38:0/1595351490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d0320 0x7f33f80d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:52.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33f759e640 1 -- 172.21.15.38:0/1595351490 >> v1:172.21.15.38:6789/0 conn(0x7f33f80d0700 legacy=0x7f33f80d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:52.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33f759e640 1 -- 172.21.15.38:0/1595351490 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f80d28f0 con 0x7f33f80b6980 2024-04-25T08:40:52.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.483+0000 7f33f6d9d640 1 --2- 172.21.15.38:0/1595351490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d0320 0x7f33f80d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:40:52.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.484+0000 7f33f759e640 1 --2- 172.21.15.38:0/1595351490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b6980 0x7f33f80b6e10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009d90 tx=0x7f33f002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bccce9176e8dac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:52.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.484+0000 7f33f5d9b640 1 -- 172.21.15.38:0/1595351490 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0007740 con 0x7f33f80b6980 2024-04-25T08:40:52.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.484+0000 7f33f5d9b640 1 -- 172.21.15.38:0/1595351490 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f00078e0 con 0x7f33f80b6980 2024-04-25T08:40:52.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.484+0000 7f33f5d9b640 1 -- 172.21.15.38:0/1595351490 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f0007cb0 con 0x7f33f80b6980 2024-04-25T08:40:52.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.485+0000 7f33fd0fc640 1 -- 172.21.15.38:0/1595351490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b6980 msgr2=0x7f33f80b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:52.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.485+0000 7f33fd0fc640 1 --2- 172.21.15.38:0/1595351490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b6980 0x7f33f80b6e10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f33f0009d90 tx=0x7f33f002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:40:52.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.485+0000 7f33fd0fc640 1 -- 172.21.15.38:0/1595351490 shutdown_connections 2024-04-25T08:40:52.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.485+0000 7f33fd0fc640 1 --2- 172.21.15.38:0/1595351490 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b6980 0x7f33f80b6e10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:52.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.485+0000 7f33fd0fc640 1 --2- 172.21.15.38:0/1595351490 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d0320 0x7f33f80d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:52.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.485+0000 7f33fd0fc640 1 -- 172.21.15.38:0/1595351490 >> 172.21.15.38:0/1595351490 conn(0x7f33f80668f0 msgr2=0x7f33f8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:52.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.485+0000 7f33fd0fc640 1 -- 172.21.15.38:0/1595351490 shutdown_connections 2024-04-25T08:40:52.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.485+0000 7f33fd0fc640 1 -- 172.21.15.38:0/1595351490 wait complete. 2024-04-25T08:40:52.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.486+0000 7f33fd0fc640 1 Processor -- start 2024-04-25T08:40:52.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.486+0000 7f33fd0fc640 1 -- start start 2024-04-25T08:40:52.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.487+0000 7f33fd0fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b6980 0x7f33f8157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:52.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.487+0000 7f33fd0fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f80d0320 0x7f33f81f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:52.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.487+0000 7f33fd0fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d0700 0x7f33f81559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:52.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.487+0000 7f33fd0fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80d0b40 con 0x7f33f80d0320 2024-04-25T08:40:52.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.487+0000 7f33fd0fc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80d2eb0 con 0x7f33f80b6980 2024-04-25T08:40:52.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.487+0000 7f33fd0fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8158d60 con 0x7f33f80d0700 2024-04-25T08:40:52.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.487+0000 7f33f659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f80d0320 0x7f33f81f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.487+0000 7f33f659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f80d0320 0x7f33f81f68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49628/0 (socket says 172.21.15.38:49628) 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.487+0000 7f33f659c640 1 -- 172.21.15.38:0/3077611985 learned_addr learned my addr 172.21.15.38:0/3077611985 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.487+0000 7f33f6d9d640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b6980 0x7f33f8157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.487+0000 7f33f759e640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d0700 0x7f33f81559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.488+0000 7f33f659c640 1 -- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d0700 msgr2=0x7f33f81559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.488+0000 7f33f659c640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d0700 0x7f33f81559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.488+0000 7f33f659c640 1 -- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b6980 msgr2=0x7f33f8157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.488+0000 7f33f659c640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b6980 0x7f33f8157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.488+0000 7f33f659c640 1 -- 172.21.15.38:0/3077611985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f81581e0 con 0x7f33f80d0320 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.488+0000 7f33f659c640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f80d0320 0x7f33f81f68f0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c3c0 tx=0x7f33ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.488+0000 7f33df7fe640 1 -- 172.21.15.38:0/3077611985 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec017070 con 0x7f33f80d0320 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.488+0000 7f33df7fe640 1 -- 172.21.15.38:0/3077611985 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec005ce0 con 0x7f33f80d0320 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.488+0000 7f33fd0fc640 1 -- 172.21.15.38:0/3077611985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f81584c0 con 0x7f33f80d0320 2024-04-25T08:40:52.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.488+0000 7f33df7fe640 1 -- 172.21.15.38:0/3077611985 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec0072f0 con 0x7f33f80d0320 2024-04-25T08:40:52.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.488+0000 7f33fd0fc640 1 -- 172.21.15.38:0/3077611985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f8158a80 con 0x7f33f80d0320 2024-04-25T08:40:52.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.489+0000 7f33fd0fc640 1 -- 172.21.15.38:0/3077611985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f8068e80 con 0x7f33f80d0320 2024-04-25T08:40:52.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.490+0000 7f33df7fe640 1 -- 172.21.15.38:0/3077611985 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33ec01a070 con 0x7f33f80d0320 2024-04-25T08:40:52.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.490+0000 7f33df7fe640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33e406b4d0 0x7f33e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:52.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.490+0000 7f33df7fe640 1 -- 172.21.15.38:0/3077611985 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f33ec089420 con 0x7f33f80d0320 2024-04-25T08:40:52.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.490+0000 7f33f6d9d640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33e406b4d0 0x7f33e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:52.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.491+0000 7f33f6d9d640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33e406b4d0 0x7f33e406d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f33f80b62c0 tx=0x7f33e8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:52.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.495+0000 7f33df7fe640 1 -- 172.21.15.38:0/3077611985 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33ec057d30 con 0x7f33f80d0320 2024-04-25T08:40:52.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:52.676+0000 7f33fd0fc640 1 -- 172.21.15.38:0/3077611985 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f33f8073cc0 con 0x7f33f80d0320 2024-04-25T08:40:53.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.359+0000 7f33df7fe640 1 -- 172.21.15.38:0/3077611985 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v130) v1 ==== 170+0+0 (secure 0 0 0) 0x7f33ec05e080 con 0x7f33f80d0320 2024-04-25T08:40:53.352 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-04-25T08:40:53.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.362+0000 7f33fd0fc640 1 -- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33e406b4d0 msgr2=0x7f33e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:53.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.362+0000 7f33fd0fc640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33e406b4d0 0x7f33e406d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f33f80b62c0 tx=0x7f33e8009190 comp rx=0 tx=0).stop 2024-04-25T08:40:53.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.362+0000 7f33fd0fc640 1 -- 172.21.15.38:0/3077611985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f80d0320 msgr2=0x7f33f81f68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:53.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.362+0000 7f33fd0fc640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f80d0320 0x7f33f81f68f0 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00c3c0 tx=0x7f33ec00c880 comp rx=0 tx=0).stop 2024-04-25T08:40:53.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.362+0000 7f33fd0fc640 1 -- 172.21.15.38:0/3077611985 shutdown_connections 2024-04-25T08:40:53.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.362+0000 7f33fd0fc640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f33e406b4d0 0x7f33e406d990 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:53.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.362+0000 7f33fd0fc640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33f80d0700 0x7f33f81559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:53.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.362+0000 7f33fd0fc640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33f80d0320 0x7f33f81f68f0 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:53.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.362+0000 7f33fd0fc640 1 --2- 172.21.15.38:0/3077611985 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33f80b6980 0x7f33f8157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:53.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.362+0000 7f33fd0fc640 1 -- 172.21.15.38:0/3077611985 >> 172.21.15.38:0/3077611985 conn(0x7f33f80668f0 msgr2=0x7f33f80d14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:53.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.362+0000 7f33fd0fc640 1 -- 172.21.15.38:0/3077611985 shutdown_connections 2024-04-25T08:40:53.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.362+0000 7f33fd0fc640 1 -- 172.21.15.38:0/3077611985 wait complete. 2024-04-25T08:40:53.384 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:40:53.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.500+0000 7fb2b9961640 1 Processor -- start 2024-04-25T08:40:53.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.500+0000 7fb2b9961640 1 -- start start 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.501+0000 7fb2b9961640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb2b406c700 0x7fb2b4162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.501+0000 7fb2b9961640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2b406cae0 0x7fb2b415e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.501+0000 7fb2b9961640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b415ec20 con 0x7fb2b415e580 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.501+0000 7fb2b9961640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b4161c80 con 0x7fb2b406c700 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.501+0000 7fb2b9961640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b415ef00 con 0x7fb2b406cae0 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.501+0000 7fb2b2ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb2b406c700 0x7fb2b4162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.501+0000 7fb2b37fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb2b415e580 0x7fb2b40cdf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51436/0 (socket says 172.21.15.38:51436) 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.501+0000 7fb2b37fe640 1 -- 172.21.15.38:0/676129586 learned_addr learned my addr 172.21.15.38:0/676129586 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.502+0000 7fb2b27fc640 1 --2- 172.21.15.38:0/676129586 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2b406cae0 0x7fb2b415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.502+0000 7fb2b2ffd640 1 -- 172.21.15.38:0/676129586 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2b406cae0 msgr2=0x7fb2b415e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.502+0000 7fb2b2ffd640 1 --2- 172.21.15.38:0/676129586 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2b406cae0 0x7fb2b415e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.502+0000 7fb2b2ffd640 1 -- 172.21.15.38:0/676129586 >> v1:172.21.15.38:6789/0 conn(0x7fb2b415e580 legacy=0x7fb2b40cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.502+0000 7fb2b2ffd640 1 -- 172.21.15.38:0/676129586 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b4161630 con 0x7fb2b406c700 2024-04-25T08:40:53.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.502+0000 7fb2b27fc640 1 --2- 172.21.15.38:0/676129586 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2b406cae0 0x7fb2b415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.502+0000 7fb2b2ffd640 1 --2- 172.21.15.38:0/676129586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b406c700 0x7fb2b4162110 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0097a0 tx=0x7fb2ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd6928ea305e992 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.503+0000 7fb2b1ffb640 1 -- 172.21.15.38:0/676129586 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac037070 con 0x7fb2b406c700 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.503+0000 7fb2b1ffb640 1 -- 172.21.15.38:0/676129586 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac032440 con 0x7fb2b406c700 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.503+0000 7fb2b1ffb640 1 -- 172.21.15.38:0/676129586 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac032810 con 0x7fb2b406c700 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.503+0000 7fb2b9961640 1 -- 172.21.15.38:0/676129586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b406c700 msgr2=0x7fb2b4162110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.503+0000 7fb2b9961640 1 --2- 172.21.15.38:0/676129586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b406c700 0x7fb2b4162110 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac0097a0 tx=0x7fb2ac02a180 comp rx=0 tx=0).stop 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.503+0000 7fb2b9961640 1 -- 172.21.15.38:0/676129586 shutdown_connections 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.503+0000 7fb2b9961640 1 --2- 172.21.15.38:0/676129586 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2b406cae0 0x7fb2b415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.503+0000 7fb2b9961640 1 --2- 172.21.15.38:0/676129586 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b406c700 0x7fb2b4162110 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.503+0000 7fb2b9961640 1 -- 172.21.15.38:0/676129586 >> 172.21.15.38:0/676129586 conn(0x7fb2b408b000 msgr2=0x7fb2b408b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.503+0000 7fb2b9961640 1 -- 172.21.15.38:0/676129586 shutdown_connections 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.503+0000 7fb2b9961640 1 -- 172.21.15.38:0/676129586 wait complete. 2024-04-25T08:40:53.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.504+0000 7fb2b9961640 1 Processor -- start 2024-04-25T08:40:53.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.505+0000 7fb2b9961640 1 -- start start 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.505+0000 7fb2b9961640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b406c700 0x7fb2b414fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.505+0000 7fb2b9961640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2b406cae0 0x7fb2b40caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.505+0000 7fb2b9961640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b415e580 0x7fb2b40b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.505+0000 7fb2b9961640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b406cf60 con 0x7fb2b406c700 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.505+0000 7fb2b9961640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b406c190 con 0x7fb2b415e580 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.505+0000 7fb2b9961640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b415b920 con 0x7fb2b406cae0 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.505+0000 7fb2b2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b406c700 0x7fb2b414fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.505+0000 7fb2b2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b406c700 0x7fb2b414fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49642/0 (socket says 172.21.15.38:49642) 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.505+0000 7fb2b2ffd640 1 -- 172.21.15.38:0/732387703 learned_addr learned my addr 172.21.15.38:0/732387703 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2b37fe640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b415e580 0x7fb2b40b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2b27fc640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2b406cae0 0x7fb2b40caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2b2ffd640 1 -- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2b406cae0 msgr2=0x7fb2b40caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2b2ffd640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2b406cae0 0x7fb2b40caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2b2ffd640 1 -- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b415e580 msgr2=0x7fb2b40b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2b2ffd640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b415e580 0x7fb2b40b1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:53.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2b2ffd640 1 -- 172.21.15.38:0/732387703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b40c88d0 con 0x7fb2b406c700 2024-04-25T08:40:53.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2b37fe640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b415e580 0x7fb2b40b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:53.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2b27fc640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2b406cae0 0x7fb2b40caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:53.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2b2ffd640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b406c700 0x7fb2b414fe20 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac000c00 tx=0x7fb2ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:53.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2abfff640 1 -- 172.21.15.38:0/732387703 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac041070 con 0x7fb2b406c700 2024-04-25T08:40:53.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2abfff640 1 -- 172.21.15.38:0/732387703 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac03c930 con 0x7fb2b406c700 2024-04-25T08:40:53.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2b9961640 1 -- 172.21.15.38:0/732387703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b40c8bb0 con 0x7fb2b406c700 2024-04-25T08:40:53.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2abfff640 1 -- 172.21.15.38:0/732387703 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac037040 con 0x7fb2b406c700 2024-04-25T08:40:53.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.506+0000 7fb2b9961640 1 -- 172.21.15.38:0/732387703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b40c8e90 con 0x7fb2b406c700 2024-04-25T08:40:53.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.507+0000 7fb2b9961640 1 -- 172.21.15.38:0/732387703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b4068e80 con 0x7fb2b406c700 2024-04-25T08:40:53.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.508+0000 7fb2abfff640 1 -- 172.21.15.38:0/732387703 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2ac044070 con 0x7fb2b406c700 2024-04-25T08:40:53.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.508+0000 7fb2abfff640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb29406b4d0 0x7fb29406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:53.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.509+0000 7fb2abfff640 1 -- 172.21.15.38:0/732387703 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fb2ac0ab280 con 0x7fb2b406c700 2024-04-25T08:40:53.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.509+0000 7fb2b27fc640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb29406b4d0 0x7fb29406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:53.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.509+0000 7fb2b27fc640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb29406b4d0 0x7fb29406d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40b2fc0 tx=0x7fb29c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.513+0000 7fb2abfff640 1 -- 172.21.15.38:0/732387703 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2ac07a0e0 con 0x7fb2b406c700 2024-04-25T08:40:53.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:53.730+0000 7fb2b9961640 1 -- 172.21.15.38:0/732387703 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb2b40b1220 con 0x7fb2b406c700 2024-04-25T08:40:54.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.361+0000 7fb2abfff640 1 -- 172.21.15.38:0/732387703 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v75) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb2ac0799e0 con 0x7fb2b406c700 2024-04-25T08:40:54.354 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 19 and data pool 20 2024-04-25T08:40:54.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.365+0000 7fb2b9961640 1 -- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb29406b4d0 msgr2=0x7fb29406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.365+0000 7fb2b9961640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb29406b4d0 0x7fb29406d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40b2fc0 tx=0x7fb29c008040 comp rx=0 tx=0).stop 2024-04-25T08:40:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.365+0000 7fb2b9961640 1 -- 172.21.15.38:0/732387703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b406c700 msgr2=0x7fb2b414fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.365+0000 7fb2b9961640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b406c700 0x7fb2b414fe20 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac000c00 tx=0x7fb2ac0046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.365+0000 7fb2b9961640 1 -- 172.21.15.38:0/732387703 shutdown_connections 2024-04-25T08:40:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.365+0000 7fb2b9961640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2b415e580 0x7fb2b40b1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.365+0000 7fb2b9961640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb29406b4d0 0x7fb29406d990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.365+0000 7fb2b9961640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2b406cae0 0x7fb2b40caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.365+0000 7fb2b9961640 1 --2- 172.21.15.38:0/732387703 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b406c700 0x7fb2b414fe20 secure :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac000c00 tx=0x7fb2ac0046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.365+0000 7fb2b9961640 1 -- 172.21.15.38:0/732387703 >> 172.21.15.38:0/732387703 conn(0x7fb2b408b000 msgr2=0x7fb2b408b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:54.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.366+0000 7fb2b9961640 1 -- 172.21.15.38:0/732387703 shutdown_connections 2024-04-25T08:40:54.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.366+0000 7fb2b9961640 1 -- 172.21.15.38:0/732387703 wait complete. 2024-04-25T08:40:54.396 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:40:54.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.508+0000 7f0131de2640 1 Processor -- start 2024-04-25T08:40:54.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.508+0000 7f0131de2640 1 -- start start 2024-04-25T08:40:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.509+0000 7f0131de2640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f012c15dc10 0x7f012c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.509+0000 7f0131de2640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f012c0d0f60 0x7f012c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.509+0000 7f0131de2640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f012c0d3e70 con 0x7f012c0ced60 2024-04-25T08:40:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.509+0000 7f0131de2640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012c15dff0 con 0x7f012c0d0f60 2024-04-25T08:40:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.509+0000 7f0131de2640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012c0b6990 con 0x7f012c15dc10 2024-04-25T08:40:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.509+0000 7f012bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f012c0d0f60 0x7f012c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.509+0000 7f012bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f012c0d0f60 0x7f012c0b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49654/0 (socket says 172.21.15.38:49654) 2024-04-25T08:40:54.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.509+0000 7f012bfff640 1 -- 172.21.15.38:0/3013653470 learned_addr learned my addr 172.21.15.38:0/3013653470 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.509+0000 7f012b7fe640 1 --2- 172.21.15.38:0/3013653470 >> v2:172.21.15.107:3300/0 conn(0x7f012c15dc10 0x7f012c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.509+0000 7f012bfff640 1 -- 172.21.15.38:0/3013653470 >> v2:172.21.15.107:3300/0 conn(0x7f012c15dc10 msgr2=0x7f012c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.509+0000 7f012bfff640 1 --2- 172.21.15.38:0/3013653470 >> v2:172.21.15.107:3300/0 conn(0x7f012c15dc10 0x7f012c0ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.509+0000 7f012bfff640 1 -- 172.21.15.38:0/3013653470 >> v1:172.21.15.107:6789/0 conn(0x7f012c0ced60 legacy=0x7f012c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.510+0000 7f012bfff640 1 -- 172.21.15.38:0/3013653470 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c0d4150 con 0x7f012c0d0f60 2024-04-25T08:40:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.510+0000 7f012b7fe640 1 --2- 172.21.15.38:0/3013653470 >> v2:172.21.15.107:3300/0 conn(0x7f012c15dc10 0x7f012c0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.510+0000 7f012bfff640 1 --2- 172.21.15.38:0/3013653470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f012c0d0f60 0x7f012c0b2370 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f012c0b5d50 tx=0x7f012402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b0a90f239744f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.510+0000 7f012a7fc640 1 -- 172.21.15.38:0/3013653470 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124036070 con 0x7f012c0d0f60 2024-04-25T08:40:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.510+0000 7f012a7fc640 1 -- 172.21.15.38:0/3013653470 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0124031600 con 0x7f012c0d0f60 2024-04-25T08:40:54.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.510+0000 7f012a7fc640 1 -- 172.21.15.38:0/3013653470 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01240319d0 con 0x7f012c0d0f60 2024-04-25T08:40:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.511+0000 7f0131de2640 1 -- 172.21.15.38:0/3013653470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f012c0d0f60 msgr2=0x7f012c0b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.511+0000 7f0131de2640 1 --2- 172.21.15.38:0/3013653470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f012c0d0f60 0x7f012c0b2370 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f012c0b5d50 tx=0x7f012402a660 comp rx=0 tx=0).stop 2024-04-25T08:40:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.511+0000 7f0131de2640 1 -- 172.21.15.38:0/3013653470 shutdown_connections 2024-04-25T08:40:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.511+0000 7f0131de2640 1 --2- 172.21.15.38:0/3013653470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f012c0d0f60 0x7f012c0b2370 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.511+0000 7f0131de2640 1 --2- 172.21.15.38:0/3013653470 >> v2:172.21.15.107:3300/0 conn(0x7f012c15dc10 0x7f012c0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.511+0000 7f0131de2640 1 -- 172.21.15.38:0/3013653470 >> 172.21.15.38:0/3013653470 conn(0x7f012c066d10 msgr2=0x7f012c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.511+0000 7f0131de2640 1 -- 172.21.15.38:0/3013653470 shutdown_connections 2024-04-25T08:40:54.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.511+0000 7f0131de2640 1 -- 172.21.15.38:0/3013653470 wait complete. 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.512+0000 7f0131de2640 1 Processor -- start 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.512+0000 7f0131de2640 1 -- start start 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.513+0000 7f0131de2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f012c0ced60 0x7f012c0c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.513+0000 7f0131de2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f012c0d0f60 0x7f012c0aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.513+0000 7f0131de2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f012c15dc10 0x7f012c0ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.513+0000 7f0131de2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0d3900 con 0x7f012c15dc10 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.513+0000 7f0131de2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0b6bc0 con 0x7f012c0d0f60 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.513+0000 7f0131de2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012c0d3a80 con 0x7f012c0ced60 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.513+0000 7f012bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f012c15dc10 0x7f012c0ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.513+0000 7f012bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f012c15dc10 0x7f012c0ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49658/0 (socket says 172.21.15.38:49658) 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.513+0000 7f012bfff640 1 -- 172.21.15.38:0/1395645271 learned_addr learned my addr 172.21.15.38:0/1395645271 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.513+0000 7f012affd640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f012c0d0f60 0x7f012c0aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:54.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.513+0000 7f012b7fe640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f012c0ced60 0x7f012c0c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:54.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.514+0000 7f012bfff640 1 -- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f012c0ced60 msgr2=0x7f012c0c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:54.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.514+0000 7f012bfff640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f012c0ced60 0x7f012c0c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.514+0000 7f012bfff640 1 -- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f012c0d0f60 msgr2=0x7f012c0aa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:54.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.514+0000 7f012bfff640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f012c0d0f60 0x7f012c0aa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.514+0000 7f012bfff640 1 -- 172.21.15.38:0/1395645271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012c0cb820 con 0x7f012c15dc10 2024-04-25T08:40:54.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.514+0000 7f012bfff640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f012c15dc10 0x7f012c0ccfe0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f012402ac50 tx=0x7f0124004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:54.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.514+0000 7f0113fff640 1 -- 172.21.15.38:0/1395645271 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124036050 con 0x7f012c15dc10 2024-04-25T08:40:54.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.514+0000 7f0113fff640 1 -- 172.21.15.38:0/1395645271 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0124042070 con 0x7f012c15dc10 2024-04-25T08:40:54.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.514+0000 7f0131de2640 1 -- 172.21.15.38:0/1395645271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012c0cc220 con 0x7f012c15dc10 2024-04-25T08:40:54.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.514+0000 7f0113fff640 1 -- 172.21.15.38:0/1395645271 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124009510 con 0x7f012c15dc10 2024-04-25T08:40:54.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.514+0000 7f0131de2640 1 -- 172.21.15.38:0/1395645271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012c0cc7e0 con 0x7f012c15dc10 2024-04-25T08:40:54.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.515+0000 7f0131de2640 1 -- 172.21.15.38:0/1395645271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012c068e80 con 0x7f012c15dc10 2024-04-25T08:40:54.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.515+0000 7f0113fff640 1 -- 172.21.15.38:0/1395645271 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0124007c50 con 0x7f012c15dc10 2024-04-25T08:40:54.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.516+0000 7f0113fff640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f011806b500 0x7f011806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.516+0000 7f0113fff640 1 -- 172.21.15.38:0/1395645271 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f01240ab400 con 0x7f012c15dc10 2024-04-25T08:40:54.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.516+0000 7f012b7fe640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f011806b500 0x7f011806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:54.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.517+0000 7f012b7fe640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f011806b500 0x7f011806d9c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f011c00a990 tx=0x7f011c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:54.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.520+0000 7f0113fff640 1 -- 172.21.15.38:0/1395645271 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0124045360 con 0x7f012c15dc10 2024-04-25T08:40:54.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.701+0000 7f0131de2640 1 -- 172.21.15.38:0/1395645271 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f012c0b4fb0 con 0x7f012c15dc10 2024-04-25T08:40:54.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.703+0000 7f0113fff640 1 -- 172.21.15.38:0/1395645271 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f012407a180 con 0x7f012c15dc10 2024-04-25T08:40:54.696 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:54.696 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":131,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:40:53.732066+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:40:52.007975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:40:52.707859+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:40:54.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.705+0000 7f0131de2640 1 -- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f011806b500 msgr2=0x7f011806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:54.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.705+0000 7f0131de2640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f011806b500 0x7f011806d9c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f011c00a990 tx=0x7f011c008040 comp rx=0 tx=0).stop 2024-04-25T08:40:54.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.705+0000 7f0131de2640 1 -- 172.21.15.38:0/1395645271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f012c15dc10 msgr2=0x7f012c0ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:54.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.705+0000 7f0131de2640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f012c15dc10 0x7f012c0ccfe0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f012402ac50 tx=0x7f0124004a40 comp rx=0 tx=0).stop 2024-04-25T08:40:54.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.705+0000 7f0131de2640 1 -- 172.21.15.38:0/1395645271 shutdown_connections 2024-04-25T08:40:54.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.705+0000 7f0131de2640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f012c15dc10 0x7f012c0ccfe0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.705+0000 7f0131de2640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f012c0d0f60 0x7f012c0aa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.705+0000 7f0131de2640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f011806b500 0x7f011806d9c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.705+0000 7f0131de2640 1 --2- 172.21.15.38:0/1395645271 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f012c0ced60 0x7f012c0c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.705+0000 7f0131de2640 1 -- 172.21.15.38:0/1395645271 >> 172.21.15.38:0/1395645271 conn(0x7f012c066d10 msgr2=0x7f012c160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:54.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.705+0000 7f0131de2640 1 -- 172.21.15.38:0/1395645271 shutdown_connections 2024-04-25T08:40:54.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.705+0000 7f0131de2640 1 -- 172.21.15.38:0/1395645271 wait complete. 2024-04-25T08:40:54.709 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:40:54.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.817+0000 7f038ca38640 1 Processor -- start 2024-04-25T08:40:54.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.817+0000 7f038ca38640 1 -- start start 2024-04-25T08:40:54.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.818+0000 7f038ca38640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f038815ee20 0x7f03880cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.818+0000 7f038ca38640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f038809ac40 0x7f03880d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.818+0000 7f038ca38640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03880ccc80 0x7f03880cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.818+0000 7f038ca38640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03880cd650 con 0x7f038815ee20 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.818+0000 7f038ca38640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0388099ba0 con 0x7f038809ac40 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.818+0000 7f038ca38640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0388099e80 con 0x7f03880ccc80 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.818+0000 7f038659c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f038815ee20 0x7f03880cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.818+0000 7f038659c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f038815ee20 0x7f03880cfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49660/0 (socket says 172.21.15.38:49660) 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.818+0000 7f038659c640 1 -- 172.21.15.38:0/3282506791 learned_addr learned my addr 172.21.15.38:0/3282506791 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.819+0000 7f0385d9b640 1 --2- 172.21.15.38:0/3282506791 >> v2:172.21.15.107:3300/0 conn(0x7f038809ac40 0x7f03880d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.819+0000 7f0386d9d640 1 --2- 172.21.15.38:0/3282506791 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03880ccc80 0x7f03880cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.819+0000 7f038659c640 1 -- 172.21.15.38:0/3282506791 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03880ccc80 msgr2=0x7f03880cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.819+0000 7f038659c640 1 --2- 172.21.15.38:0/3282506791 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03880ccc80 0x7f03880cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.819+0000 7f038659c640 1 -- 172.21.15.38:0/3282506791 >> v2:172.21.15.107:3300/0 conn(0x7f038809ac40 msgr2=0x7f03880d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.819+0000 7f038659c640 1 --2- 172.21.15.38:0/3282506791 >> v2:172.21.15.107:3300/0 conn(0x7f038809ac40 0x7f03880d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.819+0000 7f038659c640 1 -- 172.21.15.38:0/3282506791 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038809a160 con 0x7f038815ee20 2024-04-25T08:40:54.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.819+0000 7f038659c640 1 --2- 172.21.15.38:0/3282506791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f038815ee20 0x7f03880cfc50 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f03880d0fb0 tx=0x7f037802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295aa8d5d89abbee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.820+0000 7f038559a640 1 -- 172.21.15.38:0/3282506791 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378036070 con 0x7f038815ee20 2024-04-25T08:40:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.820+0000 7f038559a640 1 -- 172.21.15.38:0/3282506791 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0378031600 con 0x7f038815ee20 2024-04-25T08:40:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.820+0000 7f038559a640 1 -- 172.21.15.38:0/3282506791 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03780319d0 con 0x7f038815ee20 2024-04-25T08:40:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.820+0000 7f038ca38640 1 -- 172.21.15.38:0/3282506791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f038815ee20 msgr2=0x7f03880cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.820+0000 7f038ca38640 1 --2- 172.21.15.38:0/3282506791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f038815ee20 0x7f03880cfc50 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f03880d0fb0 tx=0x7f037802a610 comp rx=0 tx=0).stop 2024-04-25T08:40:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.820+0000 7f038ca38640 1 -- 172.21.15.38:0/3282506791 shutdown_connections 2024-04-25T08:40:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.820+0000 7f038ca38640 1 --2- 172.21.15.38:0/3282506791 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f03880ccc80 0x7f03880cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.820+0000 7f038ca38640 1 --2- 172.21.15.38:0/3282506791 >> v2:172.21.15.107:3300/0 conn(0x7f038809ac40 0x7f03880d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.820+0000 7f038ca38640 1 --2- 172.21.15.38:0/3282506791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f038815ee20 0x7f03880cfc50 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.820+0000 7f038ca38640 1 -- 172.21.15.38:0/3282506791 >> 172.21.15.38:0/3282506791 conn(0x7f03880b3180 msgr2=0x7f03880b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.820+0000 7f038ca38640 1 -- 172.21.15.38:0/3282506791 shutdown_connections 2024-04-25T08:40:54.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.820+0000 7f038ca38640 1 -- 172.21.15.38:0/3282506791 wait complete. 2024-04-25T08:40:54.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.822+0000 7f038ca38640 1 Processor -- start 2024-04-25T08:40:54.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.822+0000 7f038ca38640 1 -- start start 2024-04-25T08:40:54.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.822+0000 7f038ca38640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f038809ac40 0x7f0388157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.816 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.822+0000 7f038ca38640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03880ccc80 0x7f0388153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.822+0000 7f038ca38640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f038815ee20 0x7f038815bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.822+0000 7f038ca38640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038815f200 con 0x7f038809ac40 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.822+0000 7f038ca38640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038809a720 con 0x7f03880ccc80 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.822+0000 7f038ca38640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038815f380 con 0x7f038815ee20 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.822+0000 7f038659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f038809ac40 0x7f0388157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.822+0000 7f038659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f038809ac40 0x7f0388157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49668/0 (socket says 172.21.15.38:49668) 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.822+0000 7f038659c640 1 -- 172.21.15.38:0/4052962343 learned_addr learned my addr 172.21.15.38:0/4052962343 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.823+0000 7f038659c640 1 -- 172.21.15.38:0/4052962343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f038815ee20 msgr2=0x7f038815bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.823+0000 7f038659c640 1 --2- 172.21.15.38:0/4052962343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f038815ee20 0x7f038815bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.823+0000 7f038659c640 1 -- 172.21.15.38:0/4052962343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03880ccc80 msgr2=0x7f0388153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.823+0000 7f038659c640 1 --2- 172.21.15.38:0/4052962343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03880ccc80 0x7f0388153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.823+0000 7f038659c640 1 -- 172.21.15.38:0/4052962343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03880b4770 con 0x7f038809ac40 2024-04-25T08:40:54.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.823+0000 7f038659c640 1 --2- 172.21.15.38:0/4052962343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f038809ac40 0x7f0388157cf0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f0378034040 tx=0x7f0378004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:54.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.823+0000 7f037effd640 1 -- 172.21.15.38:0/4052962343 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378036050 con 0x7f038809ac40 2024-04-25T08:40:54.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.823+0000 7f037effd640 1 -- 172.21.15.38:0/4052962343 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0378042070 con 0x7f038809ac40 2024-04-25T08:40:54.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.823+0000 7f037effd640 1 -- 172.21.15.38:0/4052962343 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378009510 con 0x7f038809ac40 2024-04-25T08:40:54.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.823+0000 7f038ca38640 1 -- 172.21.15.38:0/4052962343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03880b4a50 con 0x7f038809ac40 2024-04-25T08:40:54.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.823+0000 7f038ca38640 1 -- 172.21.15.38:0/4052962343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03880b5010 con 0x7f038809ac40 2024-04-25T08:40:54.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.824+0000 7f038ca38640 1 -- 172.21.15.38:0/4052962343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0388068e80 con 0x7f038809ac40 2024-04-25T08:40:54.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.824+0000 7f037effd640 1 -- 172.21.15.38:0/4052962343 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0378007c70 con 0x7f038809ac40 2024-04-25T08:40:54.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.825+0000 7f037effd640 1 --2- 172.21.15.38:0/4052962343 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f036806b500 0x7f036806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:54.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.825+0000 7f037effd640 1 -- 172.21.15.38:0/4052962343 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03780ab1a0 con 0x7f038809ac40 2024-04-25T08:40:54.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.825+0000 7f0385d9b640 1 --2- 172.21.15.38:0/4052962343 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f036806b500 0x7f036806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:54.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.826+0000 7f0385d9b640 1 --2- 172.21.15.38:0/4052962343 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f036806b500 0x7f036806d9c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f03880cb840 tx=0x7f0370009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:54.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:54.829+0000 7f037effd640 1 -- 172.21.15.38:0/4052962343 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0378079e10 con 0x7f038809ac40 2024-04-25T08:40:55.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.014+0000 7f038ca38640 1 -- 172.21.15.38:0/4052962343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03880b3fb0 con 0x7f038809ac40 2024-04-25T08:40:55.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.015+0000 7f037effd640 1 -- 172.21.15.38:0/4052962343 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f0378079710 con 0x7f038809ac40 2024-04-25T08:40:55.008 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:55.008 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":131,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:40:53.732066+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:40:52.007975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:40:52.707859+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:40:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.017+0000 7f038ca38640 1 -- 172.21.15.38:0/4052962343 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f036806b500 msgr2=0x7f036806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.017+0000 7f038ca38640 1 --2- 172.21.15.38:0/4052962343 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f036806b500 0x7f036806d9c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f03880cb840 tx=0x7f0370009500 comp rx=0 tx=0).stop 2024-04-25T08:40:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.017+0000 7f038ca38640 1 -- 172.21.15.38:0/4052962343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f038809ac40 msgr2=0x7f0388157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.017+0000 7f038ca38640 1 --2- 172.21.15.38:0/4052962343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f038809ac40 0x7f0388157cf0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f0378034040 tx=0x7f0378004a40 comp rx=0 tx=0).stop 2024-04-25T08:40:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.017+0000 7f038ca38640 1 -- 172.21.15.38:0/4052962343 shutdown_connections 2024-04-25T08:40:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.017+0000 7f038ca38640 1 --2- 172.21.15.38:0/4052962343 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f036806b500 0x7f036806d9c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.017+0000 7f038ca38640 1 --2- 172.21.15.38:0/4052962343 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f038815ee20 0x7f038815bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.017+0000 7f038ca38640 1 --2- 172.21.15.38:0/4052962343 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f03880ccc80 0x7f0388153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.017+0000 7f038ca38640 1 --2- 172.21.15.38:0/4052962343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f038809ac40 0x7f0388157cf0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.017+0000 7f038ca38640 1 -- 172.21.15.38:0/4052962343 >> 172.21.15.38:0/4052962343 conn(0x7f03880b3180 msgr2=0x7f03880c6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.018+0000 7f038ca38640 1 -- 172.21.15.38:0/4052962343 shutdown_connections 2024-04-25T08:40:55.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.018+0000 7f038ca38640 1 -- 172.21.15.38:0/4052962343 wait complete. 2024-04-25T08:40:55.021 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:40:55.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.132+0000 7f291d49c640 1 Processor -- start 2024-04-25T08:40:55.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.132+0000 7f291d49c640 1 -- start start 2024-04-25T08:40:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.133+0000 7f291d49c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f29180d33d0 0x7f29180d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.133+0000 7f291d49c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29180d2330 con 0x7f29180d0320 2024-04-25T08:40:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.133+0000 7f291d49c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29180d2610 con 0x7f29180d1e90 2024-04-25T08:40:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.133+0000 7f291d49c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29180d28f0 con 0x7f29180d33d0 2024-04-25T08:40:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.133+0000 7f290ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f29180d33d0 0x7f29180d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.133+0000 7f2916ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f29180d0320 0x7f29180d0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51450/0 (socket says 172.21.15.38:51450) 2024-04-25T08:40:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.133+0000 7f290ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f29180d33d0 0x7f29180d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49678/0 (socket says 172.21.15.38:49678) 2024-04-25T08:40:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.133+0000 7f2916ffd640 1 -- 172.21.15.38:0/2654157470 learned_addr learned my addr 172.21.15.38:0/2654157470 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.133+0000 7f290ffff640 1 -- 172.21.15.38:0/2654157470 >> v1:172.21.15.107:6789/0 conn(0x7f29180d1e90 legacy=0x7f29180b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.133+0000 7f290ffff640 1 -- 172.21.15.38:0/2654157470 >> v1:172.21.15.38:6789/0 conn(0x7f29180d0320 legacy=0x7f29180d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.133+0000 7f290ffff640 1 -- 172.21.15.38:0/2654157470 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180d2bd0 con 0x7f29180d33d0 2024-04-25T08:40:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.134+0000 7f290ffff640 1 --2- 172.21.15.38:0/2654157470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29180d33d0 0x7f29180d1950 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f29000097a0 tx=0x7f290002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f466172ee3862c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.134+0000 7f29167fc640 1 -- 172.21.15.38:0/2654157470 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900037070 con 0x7f29180d33d0 2024-04-25T08:40:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.134+0000 7f29167fc640 1 -- 172.21.15.38:0/2654157470 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2900032440 con 0x7f29180d33d0 2024-04-25T08:40:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.134+0000 7f29167fc640 1 -- 172.21.15.38:0/2654157470 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2900032810 con 0x7f29180d33d0 2024-04-25T08:40:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.135+0000 7f291d49c640 1 -- 172.21.15.38:0/2654157470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29180d33d0 msgr2=0x7f29180d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.135+0000 7f291d49c640 1 --2- 172.21.15.38:0/2654157470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29180d33d0 0x7f29180d1950 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f29000097a0 tx=0x7f290002a270 comp rx=0 tx=0).stop 2024-04-25T08:40:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.135+0000 7f291d49c640 1 -- 172.21.15.38:0/2654157470 shutdown_connections 2024-04-25T08:40:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.135+0000 7f291d49c640 1 --2- 172.21.15.38:0/2654157470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29180d33d0 0x7f29180d1950 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.135+0000 7f291d49c640 1 -- 172.21.15.38:0/2654157470 >> 172.21.15.38:0/2654157470 conn(0x7f29180668f0 msgr2=0x7f2918066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.135+0000 7f291d49c640 1 -- 172.21.15.38:0/2654157470 shutdown_connections 2024-04-25T08:40:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.135+0000 7f291d49c640 1 -- 172.21.15.38:0/2654157470 wait complete. 2024-04-25T08:40:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.136+0000 7f291d49c640 1 Processor -- start 2024-04-25T08:40:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.136+0000 7f291d49c640 1 -- start start 2024-04-25T08:40:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f291d49c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29180d0320 0x7f291815a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f291d49c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29180d1e90 0x7f29180b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f291d49c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29180d33d0 0x7f291815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f291d49c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2918161390 con 0x7f29180d0320 2024-04-25T08:40:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f291d49c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2918161210 con 0x7f29180d1e90 2024-04-25T08:40:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f291d49c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29180b9720 con 0x7f29180d33d0 2024-04-25T08:40:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f2916ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29180d0320 0x7f291815a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f2916ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29180d0320 0x7f291815a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49694/0 (socket says 172.21.15.38:49694) 2024-04-25T08:40:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f2916ffd640 1 -- 172.21.15.38:0/1333802168 learned_addr learned my addr 172.21.15.38:0/1333802168 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f290ffff640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29180d1e90 0x7f29180b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f29177fe640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29180d33d0 0x7f291815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f2916ffd640 1 -- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29180d33d0 msgr2=0x7f291815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f2916ffd640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29180d33d0 0x7f291815cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f2916ffd640 1 -- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29180d1e90 msgr2=0x7f29180b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f2916ffd640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29180d1e90 0x7f29180b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f2916ffd640 1 -- 172.21.15.38:0/1333802168 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29180aa2a0 con 0x7f29180d0320 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f29177fe640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29180d33d0 0x7f291815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f2916ffd640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29180d0320 0x7f291815a820 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f290800c400 tx=0x7f290800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f290f7fe640 1 -- 172.21.15.38:0/1333802168 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908017070 con 0x7f29180d0320 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f290f7fe640 1 -- 172.21.15.38:0/1333802168 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908005ce0 con 0x7f29180d0320 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f290f7fe640 1 -- 172.21.15.38:0/1333802168 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29080072f0 con 0x7f29180d0320 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f291d49c640 1 -- 172.21.15.38:0/1333802168 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29180cb5e0 con 0x7f29180d0320 2024-04-25T08:40:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.137+0000 7f291d49c640 1 -- 172.21.15.38:0/1333802168 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29180cbe80 con 0x7f29180d0320 2024-04-25T08:40:55.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.138+0000 7f291d49c640 1 -- 172.21.15.38:0/1333802168 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2918068e80 con 0x7f29180d0320 2024-04-25T08:40:55.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.139+0000 7f290f7fe640 1 -- 172.21.15.38:0/1333802168 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f290801a070 con 0x7f29180d0320 2024-04-25T08:40:55.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.139+0000 7f290f7fe640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f28f806b4d0 0x7f28f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:55.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.139+0000 7f290f7fe640 1 -- 172.21.15.38:0/1333802168 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2908089740 con 0x7f29180d0320 2024-04-25T08:40:55.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.139+0000 7f290ffff640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f28f806b4d0 0x7f28f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:55.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.140+0000 7f290ffff640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f28f806b4d0 0x7f28f806d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2900034000 tx=0x7f290002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:55.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.144+0000 7f290f7fe640 1 -- 172.21.15.38:0/1333802168 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29180aa2a0 con 0x7f29180d0320 2024-04-25T08:40:55.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.363+0000 7f291d49c640 1 -- 172.21.15.38:0/1333802168 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f29180cb970 con 0x7f29180d0320 2024-04-25T08:40:55.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.378+0000 7f290f7fe640 1 -- 172.21.15.38:0/1333802168 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v76) v1 ==== 118+0+0 (secure 0 0 0) 0x7f29080584c0 con 0x7f29180d0320 2024-04-25T08:40:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.380+0000 7f291d49c640 1 -- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f28f806b4d0 msgr2=0x7f28f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.380+0000 7f291d49c640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f28f806b4d0 0x7f28f806d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f2900034000 tx=0x7f290002e000 comp rx=0 tx=0).stop 2024-04-25T08:40:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.380+0000 7f291d49c640 1 -- 172.21.15.38:0/1333802168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29180d0320 msgr2=0x7f291815a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.380+0000 7f291d49c640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29180d0320 0x7f291815a820 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f290800c400 tx=0x7f290800c8c0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.380+0000 7f291d49c640 1 -- 172.21.15.38:0/1333802168 shutdown_connections 2024-04-25T08:40:55.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.380+0000 7f291d49c640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f28f806b4d0 0x7f28f806d990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.380+0000 7f291d49c640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f29180d33d0 0x7f291815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.380+0000 7f291d49c640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29180d1e90 0x7f29180b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.380+0000 7f291d49c640 1 --2- 172.21.15.38:0/1333802168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f29180d0320 0x7f291815a820 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.380+0000 7f291d49c640 1 -- 172.21.15.38:0/1333802168 >> 172.21.15.38:0/1333802168 conn(0x7f29180668f0 msgr2=0x7f2918160440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.380+0000 7f291d49c640 1 -- 172.21.15.38:0/1333802168 shutdown_connections 2024-04-25T08:40:55.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.380+0000 7f291d49c640 1 -- 172.21.15.38:0/1333802168 wait complete. 2024-04-25T08:40:55.384 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:40:55.384 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:40:55.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.495+0000 7f60345d5640 1 Processor -- start 2024-04-25T08:40:55.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.495+0000 7f60345d5640 1 -- start start 2024-04-25T08:40:55.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.496+0000 7f60345d5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f602c0cfd40 0x7f602c0c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:55.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.496+0000 7f60345d5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f602c0ccfc0 0x7f602c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:55.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.496+0000 7f60345d5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f602c0ac430 con 0x7f602c15db80 2024-04-25T08:40:55.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.496+0000 7f60345d5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602c0ac1e0 con 0x7f602c0cfd40 2024-04-25T08:40:55.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.496+0000 7f60345d5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602c0ac710 con 0x7f602c0ccfc0 2024-04-25T08:40:55.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.496+0000 7f6031b70640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f602c0cfd40 0x7f602c0c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:55.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.496+0000 7f6031b70640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f602c0cfd40 0x7f602c0c9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49702/0 (socket says 172.21.15.38:49702) 2024-04-25T08:40:55.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.496+0000 7f6031b70640 1 -- 172.21.15.38:0/1313519568 learned_addr learned my addr 172.21.15.38:0/1313519568 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:55.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.497+0000 7f6032b72640 1 --2- 172.21.15.38:0/1313519568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f602c0ccfc0 0x7f602c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:55.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.497+0000 7f6031b70640 1 -- 172.21.15.38:0/1313519568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f602c0ccfc0 msgr2=0x7f602c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:55.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.497+0000 7f6031b70640 1 --2- 172.21.15.38:0/1313519568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f602c0ccfc0 0x7f602c0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.497+0000 7f6031b70640 1 -- 172.21.15.38:0/1313519568 >> v1:172.21.15.107:6789/0 conn(0x7f602c15db80 legacy=0x7f602c0cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:55.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.497+0000 7f6031b70640 1 -- 172.21.15.38:0/1313519568 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c0ac9f0 con 0x7f602c0cfd40 2024-04-25T08:40:55.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.497+0000 7f6031b70640 1 --2- 172.21.15.38:0/1313519568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f602c0cfd40 0x7f602c0c9ad0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f602c0d0180 tx=0x7f601c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dba372ac2110eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.498+0000 7f603136f640 1 -- 172.21.15.38:0/1313519568 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c037070 con 0x7f602c0cfd40 2024-04-25T08:40:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.498+0000 7f603136f640 1 -- 172.21.15.38:0/1313519568 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c032440 con 0x7f602c0cfd40 2024-04-25T08:40:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.498+0000 7f603136f640 1 -- 172.21.15.38:0/1313519568 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c032810 con 0x7f602c0cfd40 2024-04-25T08:40:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.498+0000 7f60345d5640 1 -- 172.21.15.38:0/1313519568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f602c0cfd40 msgr2=0x7f602c0c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.498+0000 7f60345d5640 1 --2- 172.21.15.38:0/1313519568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f602c0cfd40 0x7f602c0c9ad0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f602c0d0180 tx=0x7f601c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.498+0000 7f60345d5640 1 -- 172.21.15.38:0/1313519568 shutdown_connections 2024-04-25T08:40:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.498+0000 7f60345d5640 1 --2- 172.21.15.38:0/1313519568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f602c0ccfc0 0x7f602c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.498+0000 7f60345d5640 1 --2- 172.21.15.38:0/1313519568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f602c0cfd40 0x7f602c0c9ad0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.498+0000 7f60345d5640 1 -- 172.21.15.38:0/1313519568 >> 172.21.15.38:0/1313519568 conn(0x7f602c06adc0 msgr2=0x7f602c06b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.498+0000 7f60345d5640 1 -- 172.21.15.38:0/1313519568 shutdown_connections 2024-04-25T08:40:55.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.498+0000 7f60345d5640 1 -- 172.21.15.38:0/1313519568 wait complete. 2024-04-25T08:40:55.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.500+0000 7f60345d5640 1 Processor -- start 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.500+0000 7f60345d5640 1 -- start start 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.500+0000 7f60345d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f602c0ccfc0 0x7f602c15cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.500+0000 7f60345d5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f602c0cfd40 0x7f602c0c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.500+0000 7f60345d5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f602c15db80 0x7f602c1f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.500+0000 7f60345d5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c15df60 con 0x7f602c0ccfc0 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.500+0000 7f60345d5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602c0ca550 con 0x7f602c15db80 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.500+0000 7f60345d5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602c15e2c0 con 0x7f602c0cfd40 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.500+0000 7f6032371640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f602c0ccfc0 0x7f602c15cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.500+0000 7f6032371640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f602c0ccfc0 0x7f602c15cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49712/0 (socket says 172.21.15.38:49712) 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.500+0000 7f6032371640 1 -- 172.21.15.38:0/2407983917 learned_addr learned my addr 172.21.15.38:0/2407983917 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f6032b72640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f602c15db80 0x7f602c1f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f6031b70640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f602c0cfd40 0x7f602c0c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:55.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f6032371640 1 -- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f602c0cfd40 msgr2=0x7f602c0c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:55.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f6032371640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f602c0cfd40 0x7f602c0c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f6032371640 1 -- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f602c15db80 msgr2=0x7f602c1f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:55.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f6032371640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f602c15db80 0x7f602c1f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:55.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f6032371640 1 -- 172.21.15.38:0/2407983917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f602c1f3eb0 con 0x7f602c0ccfc0 2024-04-25T08:40:55.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f6032371640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f602c0ccfc0 0x7f602c15cbd0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f602800ba80 tx=0x7f602800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:55.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f6022ffd640 1 -- 172.21.15.38:0/2407983917 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602800cc00 con 0x7f602c0ccfc0 2024-04-25T08:40:55.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f6022ffd640 1 -- 172.21.15.38:0/2407983917 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602800cda0 con 0x7f602c0ccfc0 2024-04-25T08:40:55.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f60345d5640 1 -- 172.21.15.38:0/2407983917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602c1f4190 con 0x7f602c0ccfc0 2024-04-25T08:40:55.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f6022ffd640 1 -- 172.21.15.38:0/2407983917 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028003bb0 con 0x7f602c0ccfc0 2024-04-25T08:40:55.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.501+0000 7f60345d5640 1 -- 172.21.15.38:0/2407983917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f602c06e0b0 con 0x7f602c0ccfc0 2024-04-25T08:40:55.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.502+0000 7f60345d5640 1 -- 172.21.15.38:0/2407983917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602c068e80 con 0x7f602c0ccfc0 2024-04-25T08:40:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.503+0000 7f6022ffd640 1 -- 172.21.15.38:0/2407983917 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6028014440 con 0x7f602c0ccfc0 2024-04-25T08:40:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.503+0000 7f6022ffd640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f601006b4d0 0x7f601006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.503+0000 7f6022ffd640 1 -- 172.21.15.38:0/2407983917 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f602808a190 con 0x7f602c0ccfc0 2024-04-25T08:40:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.503+0000 7f6031b70640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f601006b4d0 0x7f601006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:55.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.504+0000 7f6031b70640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f601006b4d0 0x7f601006d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f601c034000 tx=0x7f601c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:55.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.507+0000 7f6022ffd640 1 -- 172.21.15.38:0/2407983917 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60280588c0 con 0x7f602c0ccfc0 2024-04-25T08:40:55.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:55.729+0000 7f60345d5640 1 -- 172.21.15.38:0/2407983917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f602c0c5d50 con 0x7f602c0ccfc0 2024-04-25T08:40:56.341 INFO:tasks.ceph.mds.a.smithi038.stderr:2024-04-25T08:40:56.349+0000 7f1ee74c5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:40:56.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.387+0000 7f6022ffd640 1 -- 172.21.15.38:0/2407983917 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v77) v1 ==== 122+0+0 (secure 0 0 0) 0x7f602805c960 con 0x7f602c0ccfc0 2024-04-25T08:40:56.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.390+0000 7f60345d5640 1 -- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f601006b4d0 msgr2=0x7f601006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.390+0000 7f60345d5640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f601006b4d0 0x7f601006d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f601c034000 tx=0x7f601c02e000 comp rx=0 tx=0).stop 2024-04-25T08:40:56.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.390+0000 7f60345d5640 1 -- 172.21.15.38:0/2407983917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f602c0ccfc0 msgr2=0x7f602c15cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.390+0000 7f60345d5640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f602c0ccfc0 0x7f602c15cbd0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f602800ba80 tx=0x7f602800bf40 comp rx=0 tx=0).stop 2024-04-25T08:40:56.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.390+0000 7f60345d5640 1 -- 172.21.15.38:0/2407983917 shutdown_connections 2024-04-25T08:40:56.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.390+0000 7f60345d5640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f602c15db80 0x7f602c1f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.390+0000 7f60345d5640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f601006b4d0 0x7f601006d990 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.390+0000 7f60345d5640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f602c0cfd40 0x7f602c0c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.390+0000 7f60345d5640 1 --2- 172.21.15.38:0/2407983917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f602c0ccfc0 0x7f602c15cbd0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.390+0000 7f60345d5640 1 -- 172.21.15.38:0/2407983917 >> 172.21.15.38:0/2407983917 conn(0x7f602c06adc0 msgr2=0x7f602c0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.390+0000 7f60345d5640 1 -- 172.21.15.38:0/2407983917 shutdown_connections 2024-04-25T08:40:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.390+0000 7f60345d5640 1 -- 172.21.15.38:0/2407983917 wait complete. 2024-04-25T08:40:56.397 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:40:56.398 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:40:56.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.513+0000 7f935b076640 1 Processor -- start 2024-04-25T08:40:56.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.513+0000 7f935b076640 1 -- start start 2024-04-25T08:40:56.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.513+0000 7f935b076640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f935415bb10 0x7f935409a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:56.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.513+0000 7f935b076640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f93540d0460 0x7f935409a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:56.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.513+0000 7f935b076640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9354160340 con 0x7f935415f4e0 2024-04-25T08:40:56.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.513+0000 7f935b076640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93540cffd0 con 0x7f93540d0460 2024-04-25T08:40:56.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.513+0000 7f935b076640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93540cf400 con 0x7f935415bb10 2024-04-25T08:40:56.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.514+0000 7f9353fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f93540d0460 0x7f935409a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:56.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.514+0000 7f9359613640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f935415f4e0 0x7f93540c5250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51452/0 (socket says 172.21.15.38:51452) 2024-04-25T08:40:56.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.514+0000 7f9359613640 1 -- 172.21.15.38:0/4062272570 learned_addr learned my addr 172.21.15.38:0/4062272570 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.514+0000 7f9358e12640 1 --2- 172.21.15.38:0/4062272570 >> v2:172.21.15.107:3300/0 conn(0x7f935415bb10 0x7f935409a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.514+0000 7f9353fff640 1 -- 172.21.15.38:0/4062272570 >> v2:172.21.15.107:3300/0 conn(0x7f935415bb10 msgr2=0x7f935409a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.514+0000 7f9353fff640 1 --2- 172.21.15.38:0/4062272570 >> v2:172.21.15.107:3300/0 conn(0x7f935415bb10 0x7f935409a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.514+0000 7f9353fff640 1 -- 172.21.15.38:0/4062272570 >> v1:172.21.15.38:6789/0 conn(0x7f935415f4e0 legacy=0x7f93540c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.514+0000 7f9353fff640 1 -- 172.21.15.38:0/4062272570 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540cf6a0 con 0x7f93540d0460 2024-04-25T08:40:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.514+0000 7f93537fe640 1 -- 172.21.15.38:0/4062272570 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712893831 0 0) 0x7f934c001490 con 0x7f935415f4e0 2024-04-25T08:40:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.515+0000 7f9353fff640 1 --2- 172.21.15.38:0/4062272570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93540d0460 0x7f935409a9c0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f93540cf280 tx=0x7f934002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2413d6fe14c0c7ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.515+0000 7f93537fe640 1 -- 172.21.15.38:0/4062272570 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9340037070 con 0x7f93540d0460 2024-04-25T08:40:56.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.515+0000 7f93537fe640 1 -- 172.21.15.38:0/4062272570 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9340032440 con 0x7f93540d0460 2024-04-25T08:40:56.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.515+0000 7f93537fe640 1 -- 172.21.15.38:0/4062272570 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934002ae60 con 0x7f93540d0460 2024-04-25T08:40:56.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.515+0000 7f935b076640 1 -- 172.21.15.38:0/4062272570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93540d0460 msgr2=0x7f935409a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.515+0000 7f935b076640 1 --2- 172.21.15.38:0/4062272570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93540d0460 0x7f935409a9c0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f93540cf280 tx=0x7f934002a110 comp rx=0 tx=0).stop 2024-04-25T08:40:56.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.515+0000 7f935b076640 1 -- 172.21.15.38:0/4062272570 shutdown_connections 2024-04-25T08:40:56.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.515+0000 7f935b076640 1 --2- 172.21.15.38:0/4062272570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93540d0460 0x7f935409a9c0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.515+0000 7f935b076640 1 --2- 172.21.15.38:0/4062272570 >> v2:172.21.15.107:3300/0 conn(0x7f935415bb10 0x7f935409a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.515+0000 7f935b076640 1 -- 172.21.15.38:0/4062272570 >> 172.21.15.38:0/4062272570 conn(0x7f93540b31f0 msgr2=0x7f93540b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:56.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.515+0000 7f935b076640 1 -- 172.21.15.38:0/4062272570 shutdown_connections 2024-04-25T08:40:56.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.515+0000 7f935b076640 1 -- 172.21.15.38:0/4062272570 wait complete. 2024-04-25T08:40:56.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.517+0000 7f935b076640 1 Processor -- start 2024-04-25T08:40:56.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.517+0000 7f935b076640 1 -- start start 2024-04-25T08:40:56.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.517+0000 7f935b076640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93540d0460 0x7f935415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:56.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.517+0000 7f935b076640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f935415bb10 0x7f93541492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:56.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.517+0000 7f935b076640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935415f4e0 0x7f93541497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.517+0000 7f935b076640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935415f950 con 0x7f935415f4e0 2024-04-25T08:40:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.517+0000 7f935b076640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93540cfc60 con 0x7f93540d0460 2024-04-25T08:40:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.517+0000 7f935b076640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935415fad0 con 0x7f935415bb10 2024-04-25T08:40:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9359613640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935415f4e0 0x7f93541497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9359613640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935415f4e0 0x7f93541497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49734/0 (socket says 172.21.15.38:49734) 2024-04-25T08:40:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9359613640 1 -- 172.21.15.38:0/2311711130 learned_addr learned my addr 172.21.15.38:0/2311711130 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9353fff640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f935415bb10 0x7f93541492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9358e12640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93540d0460 0x7f935415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9359613640 1 -- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f935415bb10 msgr2=0x7f93541492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9359613640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f935415bb10 0x7f93541492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9359613640 1 -- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93540d0460 msgr2=0x7f935415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9359613640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93540d0460 0x7f935415ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9359613640 1 -- 172.21.15.38:0/2311711130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540b56a0 con 0x7f935415f4e0 2024-04-25T08:40:56.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9353fff640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f935415bb10 0x7f93541492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:56.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9358e12640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93540d0460 0x7f935415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:56.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f9359613640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935415f4e0 0x7f93541497e0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f934c00c700 tx=0x7f934c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:56.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f93517fa640 1 -- 172.21.15.38:0/2311711130 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c018070 con 0x7f935415f4e0 2024-04-25T08:40:56.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f93517fa640 1 -- 172.21.15.38:0/2311711130 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934c00ce10 con 0x7f935415f4e0 2024-04-25T08:40:56.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f935b076640 1 -- 172.21.15.38:0/2311711130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935415d5d0 con 0x7f935415f4e0 2024-04-25T08:40:56.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f93517fa640 1 -- 172.21.15.38:0/2311711130 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c00a880 con 0x7f935415f4e0 2024-04-25T08:40:56.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.518+0000 7f935b076640 1 -- 172.21.15.38:0/2311711130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935415db90 con 0x7f935415f4e0 2024-04-25T08:40:56.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.519+0000 7f935b076640 1 -- 172.21.15.38:0/2311711130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9354068e80 con 0x7f935415f4e0 2024-04-25T08:40:56.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.520+0000 7f93517fa640 1 -- 172.21.15.38:0/2311711130 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f934c01b070 con 0x7f935415f4e0 2024-04-25T08:40:56.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.520+0000 7f93517fa640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f933806b4d0 0x7f933806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:56.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.521+0000 7f93517fa640 1 -- 172.21.15.38:0/2311711130 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f934c0898c0 con 0x7f935415f4e0 2024-04-25T08:40:56.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.521+0000 7f9358e12640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f933806b4d0 0x7f933806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:56.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.521+0000 7f9358e12640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f933806b4d0 0x7f933806d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9348006fd0 tx=0x7f9348008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:56.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.525+0000 7f93517fa640 1 -- 172.21.15.38:0/2311711130 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93540b56a0 con 0x7f935415f4e0 2024-04-25T08:40:56.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.742+0000 7f935b076640 1 -- 172.21.15.38:0/2311711130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93540cdbf0 con 0x7f935415f4e0 2024-04-25T08:40:56.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.742+0000 7f93517fa640 1 -- 172.21.15.38:0/2311711130 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f9354160340 con 0x7f935415f4e0 2024-04-25T08:40:56.736 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:56.736 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:40:55.730538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/1312063561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1312063561},{"type":"v1","addr":"172.21.15.38:6835","nonce":1312063561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:40:56.738 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 77 2024-04-25T08:40:56.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.744+0000 7f935b076640 1 -- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f933806b4d0 msgr2=0x7f933806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.744+0000 7f935b076640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f933806b4d0 0x7f933806d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9348006fd0 tx=0x7f9348008040 comp rx=0 tx=0).stop 2024-04-25T08:40:56.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.744+0000 7f935b076640 1 -- 172.21.15.38:0/2311711130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935415f4e0 msgr2=0x7f93541497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.744+0000 7f935b076640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935415f4e0 0x7f93541497e0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f934c00c700 tx=0x7f934c00cbc0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.744+0000 7f935b076640 1 -- 172.21.15.38:0/2311711130 shutdown_connections 2024-04-25T08:40:56.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.744+0000 7f935b076640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f935415f4e0 0x7f93541497e0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.744+0000 7f935b076640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f933806b4d0 0x7f933806d990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.744+0000 7f935b076640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f935415bb10 0x7f93541492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.744+0000 7f935b076640 1 --2- 172.21.15.38:0/2311711130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93540d0460 0x7f935415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.744+0000 7f935b076640 1 -- 172.21.15.38:0/2311711130 >> 172.21.15.38:0/2311711130 conn(0x7f93540b31f0 msgr2=0x7f93540c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.744+0000 7f935b076640 1 -- 172.21.15.38:0/2311711130 shutdown_connections 2024-04-25T08:40:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.744+0000 7f935b076640 1 -- 172.21.15.38:0/2311711130 wait complete. 2024-04-25T08:40:56.749 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:40:56.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.858+0000 7ff79e75c640 1 Processor -- start 2024-04-25T08:40:56.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.858+0000 7ff79e75c640 1 -- start start 2024-04-25T08:40:56.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.859+0000 7ff79e75c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff79815df50 0x7ff7980ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:56.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.859+0000 7ff79e75c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7980ceff0 con 0x7ff7980ce2e0 2024-04-25T08:40:56.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.859+0000 7ff79e75c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7980d1f70 con 0x7ff79806c110 2024-04-25T08:40:56.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.859+0000 7ff79e75c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7980cf2d0 con 0x7ff79815df50 2024-04-25T08:40:56.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.859+0000 7ff7977fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff79815df50 0x7ff7980ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:56.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.859+0000 7ff79ccf9640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff7980ce2e0 0x7ff7980d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51462/0 (socket says 172.21.15.38:51462) 2024-04-25T08:40:56.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.859+0000 7ff7977fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff79815df50 0x7ff7980ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49742/0 (socket says 172.21.15.38:49742) 2024-04-25T08:40:56.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.859+0000 7ff79ccf9640 1 -- 172.21.15.38:0/681378364 learned_addr learned my addr 172.21.15.38:0/681378364 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:56.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.859+0000 7ff7977fe640 1 -- 172.21.15.38:0/681378364 >> v1:172.21.15.107:6789/0 conn(0x7ff79806c110 legacy=0x7ff79806c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.859+0000 7ff7977fe640 1 -- 172.21.15.38:0/681378364 >> v1:172.21.15.38:6789/0 conn(0x7ff7980ce2e0 legacy=0x7ff7980d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.859+0000 7ff7977fe640 1 -- 172.21.15.38:0/681378364 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7980d2250 con 0x7ff79815df50 2024-04-25T08:40:56.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.859+0000 7ff796ffd640 1 -- 172.21.15.38:0/681378364 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650513899 0 0) 0x7ff7980ceff0 con 0x7ff7980ce2e0 2024-04-25T08:40:56.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.860+0000 7ff7977fe640 1 --2- 172.21.15.38:0/681378364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff79815df50 0x7ff7980ce7a0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7ff7880097a0 tx=0x7ff78802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a22e7b6eaa9fc5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:56.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.860+0000 7ff796ffd640 1 -- 172.21.15.38:0/681378364 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788037070 con 0x7ff79815df50 2024-04-25T08:40:56.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.860+0000 7ff796ffd640 1 -- 172.21.15.38:0/681378364 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff788032440 con 0x7ff79815df50 2024-04-25T08:40:56.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.860+0000 7ff796ffd640 1 -- 172.21.15.38:0/681378364 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788032810 con 0x7ff79815df50 2024-04-25T08:40:56.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.861+0000 7ff79e75c640 1 -- 172.21.15.38:0/681378364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff79815df50 msgr2=0x7ff7980ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.861+0000 7ff79e75c640 1 --2- 172.21.15.38:0/681378364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff79815df50 0x7ff7980ce7a0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7ff7880097a0 tx=0x7ff78802a230 comp rx=0 tx=0).stop 2024-04-25T08:40:56.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.861+0000 7ff79e75c640 1 -- 172.21.15.38:0/681378364 shutdown_connections 2024-04-25T08:40:56.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.861+0000 7ff79e75c640 1 --2- 172.21.15.38:0/681378364 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff79815df50 0x7ff7980ce7a0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.861+0000 7ff79e75c640 1 -- 172.21.15.38:0/681378364 >> 172.21.15.38:0/681378364 conn(0x7ff7980d3fb0 msgr2=0x7ff7980d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:56.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.861+0000 7ff79e75c640 1 -- 172.21.15.38:0/681378364 shutdown_connections 2024-04-25T08:40:56.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.861+0000 7ff79e75c640 1 -- 172.21.15.38:0/681378364 wait complete. 2024-04-25T08:40:56.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.862+0000 7ff79e75c640 1 Processor -- start 2024-04-25T08:40:56.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.862+0000 7ff79e75c640 1 -- start start 2024-04-25T08:40:56.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff79e75c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff79806c110 0x7ff7981f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:56.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff79e75c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7980ce2e0 0x7ff7981f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:56.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff79e75c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79815df50 0x7ff79815e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:56.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff79e75c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980d3160 con 0x7ff79806c110 2024-04-25T08:40:56.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff79e75c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980cece0 con 0x7ff79815df50 2024-04-25T08:40:56.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff79e75c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7980d32e0 con 0x7ff7980ce2e0 2024-04-25T08:40:56.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff797fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff79806c110 0x7ff7981f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:56.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff797fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff79806c110 0x7ff7981f0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49750/0 (socket says 172.21.15.38:49750) 2024-04-25T08:40:56.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff797fff640 1 -- 172.21.15.38:0/2099610611 learned_addr learned my addr 172.21.15.38:0/2099610611 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:56.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff7977fe640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7980ce2e0 0x7ff7981f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:56.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff79ccf9640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79815df50 0x7ff79815e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:56.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff797fff640 1 -- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7980ce2e0 msgr2=0x7ff7981f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff797fff640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7980ce2e0 0x7ff7981f4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff797fff640 1 -- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79815df50 msgr2=0x7ff79815e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:56.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff797fff640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79815df50 0x7ff79815e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:56.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff797fff640 1 -- 172.21.15.38:0/2099610611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7980c6d80 con 0x7ff79806c110 2024-04-25T08:40:56.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.863+0000 7ff797fff640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff79806c110 0x7ff7981f0cb0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff78000c450 tx=0x7ff78000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:56.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.864+0000 7ff794ff9640 1 -- 172.21.15.38:0/2099610611 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7800052d0 con 0x7ff79806c110 2024-04-25T08:40:56.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.864+0000 7ff794ff9640 1 -- 172.21.15.38:0/2099610611 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff780005470 con 0x7ff79806c110 2024-04-25T08:40:56.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.864+0000 7ff79e75c640 1 -- 172.21.15.38:0/2099610611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7981f11f0 con 0x7ff79806c110 2024-04-25T08:40:56.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.864+0000 7ff794ff9640 1 -- 172.21.15.38:0/2099610611 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780012870 con 0x7ff79806c110 2024-04-25T08:40:56.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.864+0000 7ff79e75c640 1 -- 172.21.15.38:0/2099610611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7981608f0 con 0x7ff79806c110 2024-04-25T08:40:56.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.865+0000 7ff79e75c640 1 -- 172.21.15.38:0/2099610611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7980c71a0 con 0x7ff79806c110 2024-04-25T08:40:56.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.865+0000 7ff794ff9640 1 -- 172.21.15.38:0/2099610611 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff780012a10 con 0x7ff79806c110 2024-04-25T08:40:56.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.865+0000 7ff794ff9640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff77806b4d0 0x7ff77806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:56.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.866+0000 7ff794ff9640 1 -- 172.21.15.38:0/2099610611 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff780089de0 con 0x7ff79806c110 2024-04-25T08:40:56.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.866+0000 7ff7977fe640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff77806b4d0 0x7ff77806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:56.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.866+0000 7ff7977fe640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff77806b4d0 0x7ff77806d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff788034000 tx=0x7ff78802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:56.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:56.870+0000 7ff794ff9640 1 -- 172.21.15.38:0/2099610611 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7980c71a0 con 0x7ff79806c110 2024-04-25T08:40:57.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.051+0000 7ff79e75c640 1 -- 172.21.15.38:0/2099610611 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff798068e80 con 0x7ff79806c110 2024-04-25T08:40:57.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.053+0000 7ff794ff9640 1 -- 172.21.15.38:0/2099610611 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+16888 (secure 0 0 0) 0x7ff780058510 con 0x7ff79806c110 2024-04-25T08:40:57.046 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:57.047 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":132,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:40:55.373869+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:40:52.007975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:40:52.707859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:40:57.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.055+0000 7ff79e75c640 1 -- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff77806b4d0 msgr2=0x7ff77806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.055+0000 7ff79e75c640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff77806b4d0 0x7ff77806d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff788034000 tx=0x7ff78802e000 comp rx=0 tx=0).stop 2024-04-25T08:40:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.055+0000 7ff79e75c640 1 -- 172.21.15.38:0/2099610611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff79806c110 msgr2=0x7ff7981f0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.055+0000 7ff79e75c640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff79806c110 0x7ff7981f0cb0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7ff78000c450 tx=0x7ff78000c910 comp rx=0 tx=0).stop 2024-04-25T08:40:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.055+0000 7ff79e75c640 1 -- 172.21.15.38:0/2099610611 shutdown_connections 2024-04-25T08:40:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.055+0000 7ff79e75c640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff79815df50 0x7ff79815e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.055+0000 7ff79e75c640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff77806b4d0 0x7ff77806d990 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.055+0000 7ff79e75c640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff7980ce2e0 0x7ff7981f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.055+0000 7ff79e75c640 1 --2- 172.21.15.38:0/2099610611 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff79806c110 0x7ff7981f0cb0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.055+0000 7ff79e75c640 1 -- 172.21.15.38:0/2099610611 >> 172.21.15.38:0/2099610611 conn(0x7ff7980d3fb0 msgr2=0x7ff79815d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:57.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.055+0000 7ff79e75c640 1 -- 172.21.15.38:0/2099610611 shutdown_connections 2024-04-25T08:40:57.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.055+0000 7ff79e75c640 1 -- 172.21.15.38:0/2099610611 wait complete. 2024-04-25T08:40:57.060 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:40:57.060 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:57.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.168+0000 7f98b67f5640 1 Processor -- start 2024-04-25T08:40:57.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.168+0000 7f98b67f5640 1 -- start start 2024-04-25T08:40:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.169+0000 7f98b67f5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f98b015dc10 0x7f98b00ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.169+0000 7f98b67f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f98b00d0f60 0x7f98b00b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.169+0000 7f98b67f5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b00d3e70 con 0x7f98b00ced60 2024-04-25T08:40:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.169+0000 7f98b67f5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b015dff0 con 0x7f98b015dc10 2024-04-25T08:40:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.169+0000 7f98b67f5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b00b6990 con 0x7f98b00d0f60 2024-04-25T08:40:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.169+0000 7f98affff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f98b015dc10 0x7f98b00ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.169+0000 7f98affff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f98b015dc10 0x7f98b00ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49766/0 (socket says 172.21.15.38:49766) 2024-04-25T08:40:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.169+0000 7f98affff640 1 -- 172.21.15.38:0/5710057 learned_addr learned my addr 172.21.15.38:0/5710057 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.170+0000 7f98b4d92640 1 --2- 172.21.15.38:0/5710057 >> v2:172.21.15.107:3300/0 conn(0x7f98b00d0f60 0x7f98b00b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.170+0000 7f98affff640 1 -- 172.21.15.38:0/5710057 >> v2:172.21.15.107:3300/0 conn(0x7f98b00d0f60 msgr2=0x7f98b00b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.170+0000 7f98affff640 1 --2- 172.21.15.38:0/5710057 >> v2:172.21.15.107:3300/0 conn(0x7f98b00d0f60 0x7f98b00b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.170+0000 7f98affff640 1 -- 172.21.15.38:0/5710057 >> v1:172.21.15.107:6789/0 conn(0x7f98b00ced60 legacy=0x7f98b00d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:57.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.170+0000 7f98affff640 1 -- 172.21.15.38:0/5710057 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b00d4150 con 0x7f98b015dc10 2024-04-25T08:40:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.170+0000 7f98affff640 1 --2- 172.21.15.38:0/5710057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b015dc10 0x7f98b00ce820 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f98b00cdee0 tx=0x7f989802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d83fca65270fe07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.170+0000 7f98aeffd640 1 -- 172.21.15.38:0/5710057 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898037070 con 0x7f98b015dc10 2024-04-25T08:40:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.170+0000 7f98aeffd640 1 -- 172.21.15.38:0/5710057 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9898032440 con 0x7f98b015dc10 2024-04-25T08:40:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.171+0000 7f98aeffd640 1 -- 172.21.15.38:0/5710057 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9898032810 con 0x7f98b015dc10 2024-04-25T08:40:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.171+0000 7f98b67f5640 1 -- 172.21.15.38:0/5710057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b015dc10 msgr2=0x7f98b00ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.171+0000 7f98b67f5640 1 --2- 172.21.15.38:0/5710057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b015dc10 0x7f98b00ce820 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f98b00cdee0 tx=0x7f989802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.171+0000 7f98b67f5640 1 -- 172.21.15.38:0/5710057 shutdown_connections 2024-04-25T08:40:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.171+0000 7f98b67f5640 1 --2- 172.21.15.38:0/5710057 >> v2:172.21.15.107:3300/0 conn(0x7f98b00d0f60 0x7f98b00b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.171+0000 7f98b67f5640 1 --2- 172.21.15.38:0/5710057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b015dc10 0x7f98b00ce820 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.171+0000 7f98b67f5640 1 -- 172.21.15.38:0/5710057 >> 172.21.15.38:0/5710057 conn(0x7f98b0066d10 msgr2=0x7f98b0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.171+0000 7f98b67f5640 1 -- 172.21.15.38:0/5710057 shutdown_connections 2024-04-25T08:40:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.171+0000 7f98b67f5640 1 -- 172.21.15.38:0/5710057 wait complete. 2024-04-25T08:40:57.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.172+0000 7f98b67f5640 1 Processor -- start 2024-04-25T08:40:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.172+0000 7f98b67f5640 1 -- start start 2024-04-25T08:40:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.173+0000 7f98b67f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98b00ced60 0x7f98b00c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.173+0000 7f98b67f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b00d0f60 0x7f98b00c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.173+0000 7f98b67f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b015dc10 0x7f98b0152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.173+0000 7f98b67f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00d3900 con 0x7f98b015dc10 2024-04-25T08:40:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.173+0000 7f98b67f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00b6bc0 con 0x7f98b00d0f60 2024-04-25T08:40:57.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.173+0000 7f98b67f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b00d3a80 con 0x7f98b00ced60 2024-04-25T08:40:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.173+0000 7f98b4d92640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b015dc10 0x7f98b0152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.173+0000 7f98b4d92640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b015dc10 0x7f98b0152880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49774/0 (socket says 172.21.15.38:49774) 2024-04-25T08:40:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.173+0000 7f98b4d92640 1 -- 172.21.15.38:0/3514073130 learned_addr learned my addr 172.21.15.38:0/3514073130 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98affff640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98b00ced60 0x7f98b00c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98af7fe640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b00d0f60 0x7f98b00c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98b4d92640 1 -- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98b00ced60 msgr2=0x7f98b00c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98b4d92640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98b00ced60 0x7f98b00c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98b4d92640 1 -- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b00d0f60 msgr2=0x7f98b00c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98b4d92640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b00d0f60 0x7f98b00c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98b4d92640 1 -- 172.21.15.38:0/3514073130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b00c7970 con 0x7f98b015dc10 2024-04-25T08:40:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98af7fe640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b00d0f60 0x7f98b00c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:57.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98b4d92640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b015dc10 0x7f98b0152880 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f98a800cd70 tx=0x7f98a800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98acff9640 1 -- 172.21.15.38:0/3514073130 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8003b90 con 0x7f98b015dc10 2024-04-25T08:40:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98acff9640 1 -- 172.21.15.38:0/3514073130 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a8003d30 con 0x7f98b015dc10 2024-04-25T08:40:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98b67f5640 1 -- 172.21.15.38:0/3514073130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b00c7c40 con 0x7f98b015dc10 2024-04-25T08:40:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98acff9640 1 -- 172.21.15.38:0/3514073130 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8004240 con 0x7f98b015dc10 2024-04-25T08:40:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.174+0000 7f98b67f5640 1 -- 172.21.15.38:0/3514073130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b00c8200 con 0x7f98b015dc10 2024-04-25T08:40:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.175+0000 7f98b67f5640 1 -- 172.21.15.38:0/3514073130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b0068e80 con 0x7f98b015dc10 2024-04-25T08:40:57.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.176+0000 7f98acff9640 1 -- 172.21.15.38:0/3514073130 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98a80043e0 con 0x7f98b015dc10 2024-04-25T08:40:57.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.176+0000 7f98acff9640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f989006b500 0x7f989006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.176+0000 7f98acff9640 1 -- 172.21.15.38:0/3514073130 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f98a808ae50 con 0x7f98b015dc10 2024-04-25T08:40:57.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.176+0000 7f98affff640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f989006b500 0x7f989006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.177+0000 7f98affff640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f989006b500 0x7f989006d9c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9898034000 tx=0x7f989802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:57.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.180+0000 7f98acff9640 1 -- 172.21.15.38:0/3514073130 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98a8059b40 con 0x7f98b015dc10 2024-04-25T08:40:57.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.358+0000 7f98b67f5640 1 -- 172.21.15.38:0/3514073130 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98b00d3500 con 0x7f989006b500 2024-04-25T08:40:57.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.365+0000 7f98acff9640 1 -- 172.21.15.38:0/3514073130 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246178 (secure 0 0 0) 0x7f989807e1c0 con 0x7f989006b500 2024-04-25T08:40:57.361 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:57.362 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:40:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.368+0000 7f98b67f5640 1 -- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f989006b500 msgr2=0x7f989006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.368+0000 7f98b67f5640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f989006b500 0x7f989006d9c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9898034000 tx=0x7f989802e000 comp rx=0 tx=0).stop 2024-04-25T08:40:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.368+0000 7f98b67f5640 1 -- 172.21.15.38:0/3514073130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b015dc10 msgr2=0x7f98b0152880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.368+0000 7f98b67f5640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b015dc10 0x7f98b0152880 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f98a800cd70 tx=0x7f98a800a660 comp rx=0 tx=0).stop 2024-04-25T08:40:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.368+0000 7f98b67f5640 1 -- 172.21.15.38:0/3514073130 shutdown_connections 2024-04-25T08:40:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.368+0000 7f98b67f5640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f98b015dc10 0x7f98b0152880 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.368+0000 7f98b67f5640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b00d0f60 0x7f98b00c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.368+0000 7f98b67f5640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f989006b500 0x7f989006d9c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.368+0000 7f98b67f5640 1 --2- 172.21.15.38:0/3514073130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f98b00ced60 0x7f98b00c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.368+0000 7f98b67f5640 1 -- 172.21.15.38:0/3514073130 >> 172.21.15.38:0/3514073130 conn(0x7f98b0066d10 msgr2=0x7f98b00d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.368+0000 7f98b67f5640 1 -- 172.21.15.38:0/3514073130 shutdown_connections 2024-04-25T08:40:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.368+0000 7f98b67f5640 1 -- 172.21.15.38:0/3514073130 wait complete. 2024-04-25T08:40:57.372 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-04-25T08:40:57.335965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591042,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":182,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":509044,"kb_used_data":13936,"kb_used_omap":12,"kb_used_meta":494963,"kb_avail":754465676,"statfs":{"total":773094113280,"available":772572852224,"internally_reserved":0,"allocated":14270464,"data_stored":7271839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":506842672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":674,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.981544"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500770+0000","last_change":"2024-04-25T08:40:54.385506+0000","last_active":"2024-04-25T08:40:55.500770+0000","last_peered":"2024-04-25T08:40:55.500770+0000","last_clean":"2024-04-25T08:40:55.500770+0000","last_became_active":"2024-04-25T08:40:54.385108+0000","last_became_peered":"2024-04-25T08:40:54.385108+0000","last_unstale":"2024-04-25T08:40:55.500770+0000","last_undegraded":"2024-04-25T08:40:55.500770+0000","last_fullsized":"2024-04-25T08:40:55.500770+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.502699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383789+0000","last_change":"2024-04-25T08:40:54.378766+0000","last_active":"2024-04-25T08:40:55.383789+0000","last_peered":"2024-04-25T08:40:55.383789+0000","last_clean":"2024-04-25T08:40:55.383789+0000","last_became_active":"2024-04-25T08:40:54.378120+0000","last_became_peered":"2024-04-25T08:40:54.378120+0000","last_unstale":"2024-04-25T08:40:55.383789+0000","last_undegraded":"2024-04-25T08:40:55.383789+0000","last_fullsized":"2024-04-25T08:40:55.383789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:49.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500706+0000","last_change":"2024-04-25T08:40:54.382334+0000","last_active":"2024-04-25T08:40:55.500706+0000","last_peered":"2024-04-25T08:40:55.500706+0000","last_clean":"2024-04-25T08:40:55.500706+0000","last_became_active":"2024-04-25T08:40:54.382070+0000","last_became_peered":"2024-04-25T08:40:54.382070+0000","last_unstale":"2024-04-25T08:40:55.500706+0000","last_undegraded":"2024-04-25T08:40:55.500706+0000","last_fullsized":"2024-04-25T08:40:55.500706+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:49.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595381+0000","last_change":"2024-04-25T08:40:54.384503+0000","last_active":"2024-04-25T08:40:55.595381+0000","last_peered":"2024-04-25T08:40:55.595381+0000","last_clean":"2024-04-25T08:40:55.595381+0000","last_became_active":"2024-04-25T08:40:54.384141+0000","last_became_peered":"2024-04-25T08:40:54.384141+0000","last_unstale":"2024-04-25T08:40:55.595381+0000","last_undegraded":"2024-04-25T08:40:55.595381+0000","last_fullsized":"2024-04-25T08:40:55.595381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:48.148660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382449+0000","last_change":"2024-04-25T08:40:54.383613+0000","last_active":"2024-04-25T08:40:55.382449+0000","last_peered":"2024-04-25T08:40:55.382449+0000","last_clean":"2024-04-25T08:40:55.382449+0000","last_became_active":"2024-04-25T08:40:54.383329+0000","last_became_peered":"2024-04-25T08:40:54.383329+0000","last_unstale":"2024-04-25T08:40:55.382449+0000","last_undegraded":"2024-04-25T08:40:55.382449+0000","last_fullsized":"2024-04-25T08:40:55.382449+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:58.417166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596145+0000","last_change":"2024-04-25T08:40:54.390511+0000","last_active":"2024-04-25T08:40:55.596145+0000","last_peered":"2024-04-25T08:40:55.596145+0000","last_clean":"2024-04-25T08:40:55.596145+0000","last_became_active":"2024-04-25T08:40:54.390216+0000","last_became_peered":"2024-04-25T08:40:54.390216+0000","last_unstale":"2024-04-25T08:40:55.596145+0000","last_undegraded":"2024-04-25T08:40:55.596145+0000","last_fullsized":"2024-04-25T08:40:55.596145+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:29.262872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381277+0000","last_change":"2024-04-25T08:40:54.386039+0000","last_active":"2024-04-25T08:40:55.381277+0000","last_peered":"2024-04-25T08:40:55.381277+0000","last_clean":"2024-04-25T08:40:55.381277+0000","last_became_active":"2024-04-25T08:40:54.385759+0000","last_became_peered":"2024-04-25T08:40:54.385759+0000","last_unstale":"2024-04-25T08:40:55.381277+0000","last_undegraded":"2024-04-25T08:40:55.381277+0000","last_fullsized":"2024-04-25T08:40:55.381277+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:59.173213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500612+0000","last_change":"2024-04-25T08:40:54.375254+0000","last_active":"2024-04-25T08:40:55.500612+0000","last_peered":"2024-04-25T08:40:55.500612+0000","last_clean":"2024-04-25T08:40:55.500612+0000","last_became_active":"2024-04-25T08:40:54.374962+0000","last_became_peered":"2024-04-25T08:40:54.374962+0000","last_unstale":"2024-04-25T08:40:55.500612+0000","last_undegraded":"2024-04-25T08:40:55.500612+0000","last_fullsized":"2024-04-25T08:40:55.500612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:51.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596611+0000","last_change":"2024-04-25T08:40:54.389554+0000","last_active":"2024-04-25T08:40:55.596611+0000","last_peered":"2024-04-25T08:40:55.596611+0000","last_clean":"2024-04-25T08:40:55.596611+0000","last_became_active":"2024-04-25T08:40:54.386658+0000","last_became_peered":"2024-04-25T08:40:54.386658+0000","last_unstale":"2024-04-25T08:40:55.596611+0000","last_undegraded":"2024-04-25T08:40:55.596611+0000","last_fullsized":"2024-04-25T08:40:55.596611+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596384+0000","last_change":"2024-04-25T08:40:54.380643+0000","last_active":"2024-04-25T08:40:55.596384+0000","last_peered":"2024-04-25T08:40:55.596384+0000","last_clean":"2024-04-25T08:40:55.596384+0000","last_became_active":"2024-04-25T08:40:54.379854+0000","last_became_peered":"2024-04-25T08:40:54.379854+0000","last_unstale":"2024-04-25T08:40:55.596384+0000","last_undegraded":"2024-04-25T08:40:55.596384+0000","last_fullsized":"2024-04-25T08:40:55.596384+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596288+0000","last_change":"2024-04-25T08:40:54.380286+0000","last_active":"2024-04-25T08:40:55.596288+0000","last_peered":"2024-04-25T08:40:55.596288+0000","last_clean":"2024-04-25T08:40:55.596288+0000","last_became_active":"2024-04-25T08:40:54.380011+0000","last_became_peered":"2024-04-25T08:40:54.380011+0000","last_unstale":"2024-04-25T08:40:55.596288+0000","last_undegraded":"2024-04-25T08:40:55.596288+0000","last_fullsized":"2024-04-25T08:40:55.596288+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:53.592947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500572+0000","last_change":"2024-04-25T08:40:54.387452+0000","last_active":"2024-04-25T08:40:55.500572+0000","last_peered":"2024-04-25T08:40:55.500572+0000","last_clean":"2024-04-25T08:40:55.500572+0000","last_became_active":"2024-04-25T08:40:54.387198+0000","last_became_peered":"2024-04-25T08:40:54.387198+0000","last_unstale":"2024-04-25T08:40:55.500572+0000","last_undegraded":"2024-04-25T08:40:55.500572+0000","last_fullsized":"2024-04-25T08:40:55.500572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:37.819157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500522+0000","last_change":"2024-04-25T08:40:54.383950+0000","last_active":"2024-04-25T08:40:55.500522+0000","last_peered":"2024-04-25T08:40:55.500522+0000","last_clean":"2024-04-25T08:40:55.500522+0000","last_became_active":"2024-04-25T08:40:54.382608+0000","last_became_peered":"2024-04-25T08:40:54.382608+0000","last_unstale":"2024-04-25T08:40:55.500522+0000","last_undegraded":"2024-04-25T08:40:55.500522+0000","last_fullsized":"2024-04-25T08:40:55.500522+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:56.551387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596504+0000","last_change":"2024-04-25T08:40:54.388616+0000","last_active":"2024-04-25T08:40:55.596504+0000","last_peered":"2024-04-25T08:40:55.596504+0000","last_clean":"2024-04-25T08:40:55.596504+0000","last_became_active":"2024-04-25T08:40:54.388217+0000","last_became_peered":"2024-04-25T08:40:54.388217+0000","last_unstale":"2024-04-25T08:40:55.596504+0000","last_undegraded":"2024-04-25T08:40:55.596504+0000","last_fullsized":"2024-04-25T08:40:55.596504+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:26.474002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383924+0000","last_change":"2024-04-25T08:40:54.384260+0000","last_active":"2024-04-25T08:40:55.383924+0000","last_peered":"2024-04-25T08:40:55.383924+0000","last_clean":"2024-04-25T08:40:55.383924+0000","last_became_active":"2024-04-25T08:40:54.383734+0000","last_became_peered":"2024-04-25T08:40:54.383734+0000","last_unstale":"2024-04-25T08:40:55.383924+0000","last_undegraded":"2024-04-25T08:40:55.383924+0000","last_fullsized":"2024-04-25T08:40:55.383924+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.502699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383888+0000","last_change":"2024-04-25T08:40:54.380206+0000","last_active":"2024-04-25T08:40:55.383888+0000","last_peered":"2024-04-25T08:40:55.383888+0000","last_clean":"2024-04-25T08:40:55.383888+0000","last_became_active":"2024-04-25T08:40:54.379492+0000","last_became_peered":"2024-04-25T08:40:54.379492+0000","last_unstale":"2024-04-25T08:40:55.383888+0000","last_undegraded":"2024-04-25T08:40:55.383888+0000","last_fullsized":"2024-04-25T08:40:55.383888+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:40.486804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381381+0000","last_change":"2024-04-25T08:40:54.386093+0000","last_active":"2024-04-25T08:40:55.381381+0000","last_peered":"2024-04-25T08:40:55.381381+0000","last_clean":"2024-04-25T08:40:55.381381+0000","last_became_active":"2024-04-25T08:40:54.385187+0000","last_became_peered":"2024-04-25T08:40:54.385187+0000","last_unstale":"2024-04-25T08:40:55.381381+0000","last_undegraded":"2024-04-25T08:40:55.381381+0000","last_fullsized":"2024-04-25T08:40:55.381381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.539986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383508+0000","last_change":"2024-04-25T08:40:54.386486+0000","last_active":"2024-04-25T08:40:55.383508+0000","last_peered":"2024-04-25T08:40:55.383508+0000","last_clean":"2024-04-25T08:40:55.383508+0000","last_became_active":"2024-04-25T08:40:54.386209+0000","last_became_peered":"2024-04-25T08:40:54.386209+0000","last_unstale":"2024-04-25T08:40:55.383508+0000","last_undegraded":"2024-04-25T08:40:55.383508+0000","last_fullsized":"2024-04-25T08:40:55.383508+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:37.819157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382698+0000","last_change":"2024-04-25T08:40:54.388200+0000","last_active":"2024-04-25T08:40:55.382698+0000","last_peered":"2024-04-25T08:40:55.382698+0000","last_clean":"2024-04-25T08:40:55.382698+0000","last_became_active":"2024-04-25T08:40:54.387935+0000","last_became_peered":"2024-04-25T08:40:54.387935+0000","last_unstale":"2024-04-25T08:40:55.382698+0000","last_undegraded":"2024-04-25T08:40:55.382698+0000","last_fullsized":"2024-04-25T08:40:55.382698+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:56.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382793+0000","last_change":"2024-04-25T08:40:54.388543+0000","last_active":"2024-04-25T08:40:55.382793+0000","last_peered":"2024-04-25T08:40:55.382793+0000","last_clean":"2024-04-25T08:40:55.382793+0000","last_became_active":"2024-04-25T08:40:54.387411+0000","last_became_peered":"2024-04-25T08:40:54.387411+0000","last_unstale":"2024-04-25T08:40:55.382793+0000","last_undegraded":"2024-04-25T08:40:55.382793+0000","last_fullsized":"2024-04-25T08:40:55.382793+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:56.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.380999+0000","last_change":"2024-04-25T08:40:54.380867+0000","last_active":"2024-04-25T08:40:55.380999+0000","last_peered":"2024-04-25T08:40:55.380999+0000","last_clean":"2024-04-25T08:40:55.380999+0000","last_became_active":"2024-04-25T08:40:54.380595+0000","last_became_peered":"2024-04-25T08:40:54.380595+0000","last_unstale":"2024-04-25T08:40:55.380999+0000","last_undegraded":"2024-04-25T08:40:55.380999+0000","last_fullsized":"2024-04-25T08:40:55.380999+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:58.196801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382612+0000","last_change":"2024-04-25T08:40:54.389841+0000","last_active":"2024-04-25T08:40:55.382612+0000","last_peered":"2024-04-25T08:40:55.382612+0000","last_clean":"2024-04-25T08:40:55.382612+0000","last_became_active":"2024-04-25T08:40:54.389572+0000","last_became_peered":"2024-04-25T08:40:54.389572+0000","last_unstale":"2024-04-25T08:40:55.382612+0000","last_undegraded":"2024-04-25T08:40:55.382612+0000","last_fullsized":"2024-04-25T08:40:55.382612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:04.970606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"131'4","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.384013+0000","last_change":"2024-04-25T08:40:53.379026+0000","last_active":"2024-04-25T08:40:55.384013+0000","last_peered":"2024-04-25T08:40:55.384013+0000","last_clean":"2024-04-25T08:40:55.384013+0000","last_became_active":"2024-04-25T08:40:53.378751+0000","last_became_peered":"2024-04-25T08:40:53.378751+0000","last_unstale":"2024-04-25T08:40:55.384013+0000","last_undegraded":"2024-04-25T08:40:55.384013+0000","last_fullsized":"2024-04-25T08:40:55.384013+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.556400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500758+0000","last_change":"2024-04-25T08:40:54.378840+0000","last_active":"2024-04-25T08:40:55.500758+0000","last_peered":"2024-04-25T08:40:55.500758+0000","last_clean":"2024-04-25T08:40:55.500758+0000","last_became_active":"2024-04-25T08:40:54.378566+0000","last_became_peered":"2024-04-25T08:40:54.378566+0000","last_unstale":"2024-04-25T08:40:55.500758+0000","last_undegraded":"2024-04-25T08:40:55.500758+0000","last_fullsized":"2024-04-25T08:40:55.500758+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:40.486804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382917+0000","last_change":"2024-04-25T08:40:54.387624+0000","last_active":"2024-04-25T08:40:55.382917+0000","last_peered":"2024-04-25T08:40:55.382917+0000","last_clean":"2024-04-25T08:40:55.382917+0000","last_became_active":"2024-04-25T08:40:54.387359+0000","last_became_peered":"2024-04-25T08:40:54.387359+0000","last_unstale":"2024-04-25T08:40:55.382917+0000","last_undegraded":"2024-04-25T08:40:55.382917+0000","last_fullsized":"2024-04-25T08:40:55.382917+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:22.825790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383795+0000","last_change":"2024-04-25T08:40:53.373572+0000","last_active":"2024-04-25T08:40:55.383795+0000","last_peered":"2024-04-25T08:40:55.383795+0000","last_clean":"2024-04-25T08:40:55.383795+0000","last_became_active":"2024-04-25T08:40:53.373290+0000","last_became_peered":"2024-04-25T08:40:53.373290+0000","last_unstale":"2024-04-25T08:40:55.383795+0000","last_undegraded":"2024-04-25T08:40:55.383795+0000","last_fullsized":"2024-04-25T08:40:55.383795+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:40.850065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383312+0000","last_change":"2024-04-25T08:40:54.388330+0000","last_active":"2024-04-25T08:40:55.383312+0000","last_peered":"2024-04-25T08:40:55.383312+0000","last_clean":"2024-04-25T08:40:55.383312+0000","last_became_active":"2024-04-25T08:40:54.387849+0000","last_became_peered":"2024-04-25T08:40:54.387849+0000","last_unstale":"2024-04-25T08:40:55.383312+0000","last_undegraded":"2024-04-25T08:40:55.383312+0000","last_fullsized":"2024-04-25T08:40:55.383312+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:40.899090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595098+0000","last_change":"2024-04-25T08:40:53.376507+0000","last_active":"2024-04-25T08:40:55.595098+0000","last_peered":"2024-04-25T08:40:55.595098+0000","last_clean":"2024-04-25T08:40:55.595098+0000","last_became_active":"2024-04-25T08:40:53.376186+0000","last_became_peered":"2024-04-25T08:40:53.376186+0000","last_unstale":"2024-04-25T08:40:55.595098+0000","last_undegraded":"2024-04-25T08:40:55.595098+0000","last_fullsized":"2024-04-25T08:40:55.595098+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:50.437720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383020+0000","last_change":"2024-04-25T08:40:53.376440+0000","last_active":"2024-04-25T08:40:55.383020+0000","last_peered":"2024-04-25T08:40:55.383020+0000","last_clean":"2024-04-25T08:40:55.383020+0000","last_became_active":"2024-04-25T08:40:53.376169+0000","last_became_peered":"2024-04-25T08:40:53.376169+0000","last_unstale":"2024-04-25T08:40:55.383020+0000","last_undegraded":"2024-04-25T08:40:55.383020+0000","last_fullsized":"2024-04-25T08:40:55.383020+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:52.489674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500789+0000","last_change":"2024-04-25T08:40:53.380621+0000","last_active":"2024-04-25T08:40:55.500789+0000","last_peered":"2024-04-25T08:40:55.500789+0000","last_clean":"2024-04-25T08:40:55.500789+0000","last_became_active":"2024-04-25T08:40:53.380150+0000","last_became_peered":"2024-04-25T08:40:53.380150+0000","last_unstale":"2024-04-25T08:40:55.500789+0000","last_undegraded":"2024-04-25T08:40:55.500789+0000","last_fullsized":"2024-04-25T08:40:55.500789+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:56.303715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595249+0000","last_change":"2024-04-25T08:40:54.387442+0000","last_active":"2024-04-25T08:40:55.595249+0000","last_peered":"2024-04-25T08:40:55.595249+0000","last_clean":"2024-04-25T08:40:55.595249+0000","last_became_active":"2024-04-25T08:40:54.387157+0000","last_became_peered":"2024-04-25T08:40:54.387157+0000","last_unstale":"2024-04-25T08:40:55.595249+0000","last_undegraded":"2024-04-25T08:40:55.595249+0000","last_fullsized":"2024-04-25T08:40:55.595249+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:48.139944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382845+0000","last_change":"2024-04-25T08:40:54.376953+0000","last_active":"2024-04-25T08:40:55.382845+0000","last_peered":"2024-04-25T08:40:55.382845+0000","last_clean":"2024-04-25T08:40:55.382845+0000","last_became_active":"2024-04-25T08:40:54.376686+0000","last_became_peered":"2024-04-25T08:40:54.376686+0000","last_unstale":"2024-04-25T08:40:55.382845+0000","last_undegraded":"2024-04-25T08:40:55.382845+0000","last_fullsized":"2024-04-25T08:40:55.382845+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:22.825790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383572+0000","last_change":"2024-04-25T08:40:54.385830+0000","last_active":"2024-04-25T08:40:55.383572+0000","last_peered":"2024-04-25T08:40:55.383572+0000","last_clean":"2024-04-25T08:40:55.383572+0000","last_became_active":"2024-04-25T08:40:54.384994+0000","last_became_peered":"2024-04-25T08:40:54.384994+0000","last_unstale":"2024-04-25T08:40:55.383572+0000","last_undegraded":"2024-04-25T08:40:55.383572+0000","last_fullsized":"2024-04-25T08:40:55.383572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:51.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381354+0000","last_change":"2024-04-25T08:40:54.383836+0000","last_active":"2024-04-25T08:40:55.381354+0000","last_peered":"2024-04-25T08:40:55.381354+0000","last_clean":"2024-04-25T08:40:55.381354+0000","last_became_active":"2024-04-25T08:40:54.383486+0000","last_became_peered":"2024-04-25T08:40:54.383486+0000","last_unstale":"2024-04-25T08:40:55.381354+0000","last_undegraded":"2024-04-25T08:40:55.381354+0000","last_fullsized":"2024-04-25T08:40:55.381354+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:20.048786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383208+0000","last_change":"2024-04-25T08:40:53.377899+0000","last_active":"2024-04-25T08:40:55.383208+0000","last_peered":"2024-04-25T08:40:55.383208+0000","last_clean":"2024-04-25T08:40:55.383208+0000","last_became_active":"2024-04-25T08:40:53.377626+0000","last_became_peered":"2024-04-25T08:40:53.377626+0000","last_unstale":"2024-04-25T08:40:55.383208+0000","last_undegraded":"2024-04-25T08:40:55.383208+0000","last_fullsized":"2024-04-25T08:40:55.383208+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:08.112732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"131'8","reported_seq":22,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596483+0000","last_change":"2024-04-25T08:40:53.378565+0000","last_active":"2024-04-25T08:40:55.596483+0000","last_peered":"2024-04-25T08:40:55.596483+0000","last_clean":"2024-04-25T08:40:55.596483+0000","last_became_active":"2024-04-25T08:40:53.378251+0000","last_became_peered":"2024-04-25T08:40:53.378251+0000","last_unstale":"2024-04-25T08:40:55.596483+0000","last_undegraded":"2024-04-25T08:40:55.596483+0000","last_fullsized":"2024-04-25T08:40:55.596483+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:03.962369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382898+0000","last_change":"2024-04-25T08:40:54.372817+0000","last_active":"2024-04-25T08:40:55.382898+0000","last_peered":"2024-04-25T08:40:55.382898+0000","last_clean":"2024-04-25T08:40:55.382898+0000","last_became_active":"2024-04-25T08:40:54.371590+0000","last_became_peered":"2024-04-25T08:40:54.371590+0000","last_unstale":"2024-04-25T08:40:55.382898+0000","last_undegraded":"2024-04-25T08:40:55.382898+0000","last_fullsized":"2024-04-25T08:40:55.382898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:40.899090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500851+0000","last_change":"2024-04-25T08:40:53.382667+0000","last_active":"2024-04-25T08:40:55.500851+0000","last_peered":"2024-04-25T08:40:55.500851+0000","last_clean":"2024-04-25T08:40:55.500851+0000","last_became_active":"2024-04-25T08:40:53.382436+0000","last_became_peered":"2024-04-25T08:40:53.382436+0000","last_unstale":"2024-04-25T08:40:55.500851+0000","last_undegraded":"2024-04-25T08:40:55.500851+0000","last_fullsized":"2024-04-25T08:40:55.500851+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:39.954950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383651+0000","last_change":"2024-04-25T08:40:53.384093+0000","last_active":"2024-04-25T08:40:55.383651+0000","last_peered":"2024-04-25T08:40:55.383651+0000","last_clean":"2024-04-25T08:40:55.383651+0000","last_became_active":"2024-04-25T08:40:53.383830+0000","last_became_peered":"2024-04-25T08:40:53.383830+0000","last_unstale":"2024-04-25T08:40:55.383651+0000","last_undegraded":"2024-04-25T08:40:55.383651+0000","last_fullsized":"2024-04-25T08:40:55.383651+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:40.850065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595793+0000","last_change":"2024-04-25T08:40:53.377620+0000","last_active":"2024-04-25T08:40:55.595793+0000","last_peered":"2024-04-25T08:40:55.595793+0000","last_clean":"2024-04-25T08:40:55.595793+0000","last_became_active":"2024-04-25T08:40:53.377340+0000","last_became_peered":"2024-04-25T08:40:53.377340+0000","last_unstale":"2024-04-25T08:40:55.595793+0000","last_undegraded":"2024-04-25T08:40:55.595793+0000","last_fullsized":"2024-04-25T08:40:55.595793+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:33.063592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381144+0000","last_change":"2024-04-25T08:40:54.382916+0000","last_active":"2024-04-25T08:40:55.381144+0000","last_peered":"2024-04-25T08:40:55.381144+0000","last_clean":"2024-04-25T08:40:55.381144+0000","last_became_active":"2024-04-25T08:40:54.382634+0000","last_became_peered":"2024-04-25T08:40:54.382634+0000","last_unstale":"2024-04-25T08:40:55.381144+0000","last_undegraded":"2024-04-25T08:40:55.381144+0000","last_fullsized":"2024-04-25T08:40:55.381144+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:28.055098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383840+0000","last_change":"2024-04-25T08:40:53.372724+0000","last_active":"2024-04-25T08:40:55.383840+0000","last_peered":"2024-04-25T08:40:55.383840+0000","last_clean":"2024-04-25T08:40:55.383840+0000","last_became_active":"2024-04-25T08:40:53.372440+0000","last_became_peered":"2024-04-25T08:40:53.372440+0000","last_unstale":"2024-04-25T08:40:55.383840+0000","last_undegraded":"2024-04-25T08:40:55.383840+0000","last_fullsized":"2024-04-25T08:40:55.383840+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:46.131350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500809+0000","last_change":"2024-04-25T08:40:54.378104+0000","last_active":"2024-04-25T08:40:55.500809+0000","last_peered":"2024-04-25T08:40:55.500809+0000","last_clean":"2024-04-25T08:40:55.500809+0000","last_became_active":"2024-04-25T08:40:54.377753+0000","last_became_peered":"2024-04-25T08:40:54.377753+0000","last_unstale":"2024-04-25T08:40:55.500809+0000","last_undegraded":"2024-04-25T08:40:55.500809+0000","last_fullsized":"2024-04-25T08:40:55.500809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:25.693049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596583+0000","last_change":"2024-04-25T08:40:53.376843+0000","last_active":"2024-04-25T08:40:55.596583+0000","last_peered":"2024-04-25T08:40:55.596583+0000","last_clean":"2024-04-25T08:40:55.596583+0000","last_became_active":"2024-04-25T08:40:53.376238+0000","last_became_peered":"2024-04-25T08:40:53.376238+0000","last_unstale":"2024-04-25T08:40:55.596583+0000","last_undegraded":"2024-04-25T08:40:55.596583+0000","last_fullsized":"2024-04-25T08:40:55.596583+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:55.097704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383108+0000","last_change":"2024-04-25T08:40:54.384630+0000","last_active":"2024-04-25T08:40:55.383108+0000","last_peered":"2024-04-25T08:40:55.383108+0000","last_clean":"2024-04-25T08:40:55.383108+0000","last_became_active":"2024-04-25T08:40:54.384077+0000","last_became_peered":"2024-04-25T08:40:54.384077+0000","last_unstale":"2024-04-25T08:40:55.383108+0000","last_undegraded":"2024-04-25T08:40:55.383108+0000","last_fullsized":"2024-04-25T08:40:55.383108+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:56.551387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"132'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:56.368767+0000","last_change":"2024-04-25T08:40:53.378186+0000","last_active":"2024-04-25T08:40:56.368767+0000","last_peered":"2024-04-25T08:40:56.368767+0000","last_clean":"2024-04-25T08:40:56.368767+0000","last_became_active":"2024-04-25T08:40:53.377880+0000","last_became_peered":"2024-04-25T08:40:53.377880+0000","last_unstale":"2024-04-25T08:40:56.368767+0000","last_undegraded":"2024-04-25T08:40:56.368767+0000","last_fullsized":"2024-04-25T08:40:56.368767+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:47.172417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383430+0000","last_change":"2024-04-25T08:40:53.381261+0000","last_active":"2024-04-25T08:40:55.383430+0000","last_peered":"2024-04-25T08:40:55.383430+0000","last_clean":"2024-04-25T08:40:55.383430+0000","last_became_active":"2024-04-25T08:40:53.380920+0000","last_became_peered":"2024-04-25T08:40:53.380920+0000","last_unstale":"2024-04-25T08:40:55.383430+0000","last_undegraded":"2024-04-25T08:40:55.383430+0000","last_fullsized":"2024-04-25T08:40:55.383430+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:39.954950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381178+0000","last_change":"2024-04-25T08:40:54.386615+0000","last_active":"2024-04-25T08:40:55.381178+0000","last_peered":"2024-04-25T08:40:55.381178+0000","last_clean":"2024-04-25T08:40:55.381178+0000","last_became_active":"2024-04-25T08:40:54.386359+0000","last_became_peered":"2024-04-25T08:40:54.386359+0000","last_unstale":"2024-04-25T08:40:55.381178+0000","last_undegraded":"2024-04-25T08:40:55.381178+0000","last_fullsized":"2024-04-25T08:40:55.381178+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.093642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595669+0000","last_change":"2024-04-25T08:40:53.378763+0000","last_active":"2024-04-25T08:40:55.595669+0000","last_peered":"2024-04-25T08:40:55.595669+0000","last_clean":"2024-04-25T08:40:55.595669+0000","last_became_active":"2024-04-25T08:40:53.378453+0000","last_became_peered":"2024-04-25T08:40:53.378453+0000","last_unstale":"2024-04-25T08:40:55.595669+0000","last_undegraded":"2024-04-25T08:40:55.595669+0000","last_fullsized":"2024-04-25T08:40:55.595669+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:57.408929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595573+0000","last_change":"2024-04-25T08:40:54.387431+0000","last_active":"2024-04-25T08:40:55.595573+0000","last_peered":"2024-04-25T08:40:55.595573+0000","last_clean":"2024-04-25T08:40:55.595573+0000","last_became_active":"2024-04-25T08:40:54.387149+0000","last_became_peered":"2024-04-25T08:40:54.387149+0000","last_unstale":"2024-04-25T08:40:55.595573+0000","last_undegraded":"2024-04-25T08:40:55.595573+0000","last_fullsized":"2024-04-25T08:40:55.595573+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:49.064739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500551+0000","last_change":"2024-04-25T08:40:53.385220+0000","last_active":"2024-04-25T08:40:55.500551+0000","last_peered":"2024-04-25T08:40:55.500551+0000","last_clean":"2024-04-25T08:40:55.500551+0000","last_became_active":"2024-04-25T08:40:53.384943+0000","last_became_peered":"2024-04-25T08:40:53.384943+0000","last_unstale":"2024-04-25T08:40:55.500551+0000","last_undegraded":"2024-04-25T08:40:55.500551+0000","last_fullsized":"2024-04-25T08:40:55.500551+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:16.391337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595687+0000","last_change":"2024-04-25T08:40:54.388970+0000","last_active":"2024-04-25T08:40:55.595687+0000","last_peered":"2024-04-25T08:40:55.595687+0000","last_clean":"2024-04-25T08:40:55.595687+0000","last_became_active":"2024-04-25T08:40:54.388677+0000","last_became_peered":"2024-04-25T08:40:54.388677+0000","last_unstale":"2024-04-25T08:40:55.595687+0000","last_undegraded":"2024-04-25T08:40:55.595687+0000","last_fullsized":"2024-04-25T08:40:55.595687+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:37.825506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595830+0000","last_change":"2024-04-25T08:40:54.381075+0000","last_active":"2024-04-25T08:40:55.595830+0000","last_peered":"2024-04-25T08:40:55.595830+0000","last_clean":"2024-04-25T08:40:55.595830+0000","last_became_active":"2024-04-25T08:40:54.380766+0000","last_became_peered":"2024-04-25T08:40:54.380766+0000","last_unstale":"2024-04-25T08:40:55.595830+0000","last_undegraded":"2024-04-25T08:40:55.595830+0000","last_fullsized":"2024-04-25T08:40:55.595830+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:56.441011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595339+0000","last_change":"2024-04-25T08:40:53.378417+0000","last_active":"2024-04-25T08:40:55.595339+0000","last_peered":"2024-04-25T08:40:55.595339+0000","last_clean":"2024-04-25T08:40:55.595339+0000","last_became_active":"2024-04-25T08:40:53.378139+0000","last_became_peered":"2024-04-25T08:40:53.378139+0000","last_unstale":"2024-04-25T08:40:55.595339+0000","last_undegraded":"2024-04-25T08:40:55.595339+0000","last_fullsized":"2024-04-25T08:40:55.595339+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.250586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383951+0000","last_change":"2024-04-25T08:40:54.378981+0000","last_active":"2024-04-25T08:40:55.383951+0000","last_peered":"2024-04-25T08:40:55.383951+0000","last_clean":"2024-04-25T08:40:55.383951+0000","last_became_active":"2024-04-25T08:40:54.378496+0000","last_became_peered":"2024-04-25T08:40:54.378496+0000","last_unstale":"2024-04-25T08:40:55.383951+0000","last_undegraded":"2024-04-25T08:40:55.383951+0000","last_fullsized":"2024-04-25T08:40:55.383951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:25.693049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":372,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T08:40:50.676773+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:40:50.676773+0000","last_peered":"2024-04-25T08:40:50.676773+0000","last_clean":"2024-04-25T08:40:50.676773+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:40:50.676773+0000","last_undegraded":"2024-04-25T08:40:50.676773+0000","last_fullsized":"2024-04-25T08:40:50.676773+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":76,"acting":76,"num_store_stats":5},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":674,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":34,"acting":34,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65264,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306576,"statfs":{"total":96636764160,"available":96569933824,"internally_reserved":0,"allocated":2195456,"data_stored":1334949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59172,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312668,"statfs":{"total":96636764160,"available":96576172032,"internally_reserved":0,"allocated":1658880,"data_stored":771980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55012,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316828,"statfs":{"total":96636764160,"available":96580431872,"internally_reserved":0,"allocated":1654784,"data_stored":771330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68892,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302948,"statfs":{"total":96636764160,"available":96566218752,"internally_reserved":0,"allocated":1650688,"data_stored":771308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73004,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298836,"statfs":{"total":96636764160,"available":96562008064,"internally_reserved":0,"allocated":1601536,"data_stored":744581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59116,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312724,"statfs":{"total":96636764160,"available":96576229376,"internally_reserved":0,"allocated":1601536,"data_stored":744581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":1654784,"data_stored":771378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69416,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302424,"statfs":{"total":96636764160,"available":96565682176,"internally_reserved":0,"allocated":2252800,"data_stored":1361732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:57.376 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:57.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.521+0000 7f53deabc640 1 Processor -- start 2024-04-25T08:40:57.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.521+0000 7f53deabc640 1 -- start start 2024-04-25T08:40:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.522+0000 7f53deabc640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f53d806abe0 0x7f53d806b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.522+0000 7f53deabc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53d80d01d0 0x7f53d80d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.522+0000 7f53deabc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d80cfa40 con 0x7f53d80aeeb0 2024-04-25T08:40:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.522+0000 7f53deabc640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d80cd9b0 con 0x7f53d806abe0 2024-04-25T08:40:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.522+0000 7f53deabc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d80cdc90 con 0x7f53d80d01d0 2024-04-25T08:40:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.522+0000 7f53dc858640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f53d80aeeb0 0x7f53d80ccf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51468/0 (socket says 172.21.15.38:51468) 2024-04-25T08:40:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.522+0000 7f53dc858640 1 -- 172.21.15.38:0/5112005 learned_addr learned my addr 172.21.15.38:0/5112005 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d7fff640 1 --2- 172.21.15.38:0/5112005 >> v2:172.21.15.107:3300/0 conn(0x7f53d806abe0 0x7f53d806b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53dd059640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53d80d01d0 0x7f53d80d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632910735 0 0) 0x7f53d80cfa40 con 0x7f53d80aeeb0 2024-04-25T08:40:57.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53c4003520 con 0x7f53d80aeeb0 2024-04-25T08:40:57.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1535020570 0 0) 0x7f53c4003520 con 0x7f53d80aeeb0 2024-04-25T08:40:57.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53d80d01d0 msgr2=0x7f53d80d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:57.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53d80d01d0 0x7f53d80d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 >> v2:172.21.15.107:3300/0 conn(0x7f53d806abe0 msgr2=0x7f53d806b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:57.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 --2- 172.21.15.38:0/5112005 >> v2:172.21.15.107:3300/0 conn(0x7f53d806abe0 0x7f53d806b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d815e390 con 0x7f53d80aeeb0 2024-04-25T08:40:57.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f53d0001a10 con 0x7f53d80aeeb0 2024-04-25T08:40:57.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 >> v1:172.21.15.38:6789/0 conn(0x7f53d80aeeb0 legacy=0x7f53d80ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c40039b0 0x7f53c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53c4004800 0x7f53c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53d80d01d0 0x7f53c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d80cd9b0 con 0x7f53c40039b0 2024-04-25T08:40:57.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d80cdc90 con 0x7f53c4004800 2024-04-25T08:40:57.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c400b9c0 con 0x7f53d80d01d0 2024-04-25T08:40:57.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53dc858640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c40039b0 0x7f53c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53d7fff640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53c4004800 0x7f53c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53dd059640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53d80d01d0 0x7f53c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53dc858640 1 -- 172.21.15.38:0/5112005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53d80d01d0 msgr2=0x7f53c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:57.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53dc858640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53d80d01d0 0x7f53c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53dc858640 1 -- 172.21.15.38:0/5112005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53c4004800 msgr2=0x7f53c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:57.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53dc858640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53c4004800 0x7f53c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.523+0000 7f53dc858640 1 -- 172.21.15.38:0/5112005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c4003520 con 0x7f53c40039b0 2024-04-25T08:40:57.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.524+0000 7f53dd059640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53d80d01d0 0x7f53c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:57.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.524+0000 7f53dc858640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c40039b0 0x7f53c4003dd0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f53d000a7e0 tx=0x7f53c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda61566eef94835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:57.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.524+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000a430 con 0x7f53c40039b0 2024-04-25T08:40:57.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.524+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d002f070 con 0x7f53c40039b0 2024-04-25T08:40:57.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.524+0000 7f53d77fe640 1 -- 172.21.15.38:0/5112005 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d002bc00 con 0x7f53c40039b0 2024-04-25T08:40:57.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.525+0000 7f53deabc640 1 -- 172.21.15.38:0/5112005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c40039b0 msgr2=0x7f53c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.525+0000 7f53deabc640 1 --2- 172.21.15.38:0/5112005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53c40039b0 0x7f53c4003dd0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f53d000a7e0 tx=0x7f53c4005830 comp rx=0 tx=0).stop 2024-04-25T08:40:57.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.525+0000 7f53dd059640 1 -- 172.21.15.38:0/5112005 reap_dead start 2024-04-25T08:40:57.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.525+0000 7f53deabc640 1 -- 172.21.15.38:0/5112005 shutdown_connections 2024-04-25T08:40:57.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.525+0000 7f53deabc640 1 -- 172.21.15.38:0/5112005 >> 172.21.15.38:0/5112005 conn(0x7f53d8072610 msgr2=0x7f53d8099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:57.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.525+0000 7f53deabc640 1 -- 172.21.15.38:0/5112005 shutdown_connections 2024-04-25T08:40:57.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.525+0000 7f53deabc640 1 -- 172.21.15.38:0/5112005 wait complete. 2024-04-25T08:40:57.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.526+0000 7f53deabc640 1 Processor -- start 2024-04-25T08:40:57.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.526+0000 7f53deabc640 1 -- start start 2024-04-25T08:40:57.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53deabc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c40039b0 0x7f53d80b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53deabc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53d8154cb0 0x7f53d81487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53deabc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d80b50c0 0x7f53d80b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53deabc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d80d0600 con 0x7f53d8154cb0 2024-04-25T08:40:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53deabc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d80cd620 con 0x7f53d80b50c0 2024-04-25T08:40:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53deabc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d80d0780 con 0x7f53c40039b0 2024-04-25T08:40:57.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53d8154cb0 0x7f53d81487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53d8154cb0 0x7f53d81487f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49792/0 (socket says 172.21.15.38:49792) 2024-04-25T08:40:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53d7fff640 1 -- 172.21.15.38:0/1302691748 learned_addr learned my addr 172.21.15.38:0/1302691748 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53dd059640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d80b50c0 0x7f53d80b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53dc858640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c40039b0 0x7f53d80b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53d7fff640 1 -- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c40039b0 msgr2=0x7f53d80b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53d7fff640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c40039b0 0x7f53d80b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53d7fff640 1 -- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d80b50c0 msgr2=0x7f53d80b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53d7fff640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d80b50c0 0x7f53d80b5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53d7fff640 1 -- 172.21.15.38:0/1302691748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d8151ac0 con 0x7f53d8154cb0 2024-04-25T08:40:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.527+0000 7f53dd059640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d80b50c0 0x7f53d80b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.528+0000 7f53d7fff640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53d8154cb0 0x7f53d81487f0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f53c8011990 tx=0x7f53c8011e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.528+0000 7f53d57fa640 1 -- 172.21.15.38:0/1302691748 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800fad0 con 0x7f53d8154cb0 2024-04-25T08:40:57.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.528+0000 7f53d57fa640 1 -- 172.21.15.38:0/1302691748 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c800fc70 con 0x7f53d8154cb0 2024-04-25T08:40:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.528+0000 7f53deabc640 1 -- 172.21.15.38:0/1302691748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d80c69f0 con 0x7f53d8154cb0 2024-04-25T08:40:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.528+0000 7f53d57fa640 1 -- 172.21.15.38:0/1302691748 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8014440 con 0x7f53d8154cb0 2024-04-25T08:40:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.528+0000 7f53deabc640 1 -- 172.21.15.38:0/1302691748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d8152040 con 0x7f53d8154cb0 2024-04-25T08:40:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.529+0000 7f53deabc640 1 -- 172.21.15.38:0/1302691748 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d806ef00 con 0x7f53d8154cb0 2024-04-25T08:40:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.529+0000 7f53d57fa640 1 -- 172.21.15.38:0/1302691748 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53c8012040 con 0x7f53d8154cb0 2024-04-25T08:40:57.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.530+0000 7f53d57fa640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f53b806b500 0x7f53b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.530+0000 7f53d57fa640 1 -- 172.21.15.38:0/1302691748 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53c808aa50 con 0x7f53d8154cb0 2024-04-25T08:40:57.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.530+0000 7f53dc858640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f53b806b500 0x7f53b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.531+0000 7f53dc858640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f53b806b500 0x7f53b806d9c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f53d0033000 tx=0x7f53d0007000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:57.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.534+0000 7f53d57fa640 1 -- 172.21.15.38:0/1302691748 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53c8019030 con 0x7f53d8154cb0 2024-04-25T08:40:57.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.710+0000 7f53deabc640 1 -- 172.21.15.38:0/1302691748 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53d806ec20 con 0x7f53b806b500 2024-04-25T08:40:57.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.716+0000 7f53d57fa640 1 -- 172.21.15.38:0/1302691748 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246178 (secure 0 0 0) 0x7f53d806ec20 con 0x7f53b806b500 2024-04-25T08:40:57.713 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:57.715 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:40:57.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.719+0000 7f53deabc640 1 -- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f53b806b500 msgr2=0x7f53b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.719+0000 7f53deabc640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f53b806b500 0x7f53b806d9c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f53d0033000 tx=0x7f53d0007000 comp rx=0 tx=0).stop 2024-04-25T08:40:57.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.719+0000 7f53deabc640 1 -- 172.21.15.38:0/1302691748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53d8154cb0 msgr2=0x7f53d81487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.719+0000 7f53deabc640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53d8154cb0 0x7f53d81487f0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f53c8011990 tx=0x7f53c8011e50 comp rx=0 tx=0).stop 2024-04-25T08:40:57.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.719+0000 7f53deabc640 1 -- 172.21.15.38:0/1302691748 shutdown_connections 2024-04-25T08:40:57.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.719+0000 7f53deabc640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f53d80b50c0 0x7f53d80b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.719+0000 7f53deabc640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f53d8154cb0 0x7f53d81487f0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.719+0000 7f53deabc640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f53b806b500 0x7f53b806d9c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.719+0000 7f53deabc640 1 --2- 172.21.15.38:0/1302691748 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f53c40039b0 0x7f53d80b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.719+0000 7f53deabc640 1 -- 172.21.15.38:0/1302691748 >> 172.21.15.38:0/1302691748 conn(0x7f53d8072610 msgr2=0x7f53d80b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:57.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.719+0000 7f53deabc640 1 -- 172.21.15.38:0/1302691748 shutdown_connections 2024-04-25T08:40:57.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.719+0000 7f53deabc640 1 -- 172.21.15.38:0/1302691748 wait complete. 2024-04-25T08:40:57.723 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-04-25T08:40:57.335965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591042,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":182,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":509044,"kb_used_data":13936,"kb_used_omap":12,"kb_used_meta":494963,"kb_avail":754465676,"statfs":{"total":773094113280,"available":772572852224,"internally_reserved":0,"allocated":14270464,"data_stored":7271839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":506842672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":674,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.981544"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500770+0000","last_change":"2024-04-25T08:40:54.385506+0000","last_active":"2024-04-25T08:40:55.500770+0000","last_peered":"2024-04-25T08:40:55.500770+0000","last_clean":"2024-04-25T08:40:55.500770+0000","last_became_active":"2024-04-25T08:40:54.385108+0000","last_became_peered":"2024-04-25T08:40:54.385108+0000","last_unstale":"2024-04-25T08:40:55.500770+0000","last_undegraded":"2024-04-25T08:40:55.500770+0000","last_fullsized":"2024-04-25T08:40:55.500770+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.502699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383789+0000","last_change":"2024-04-25T08:40:54.378766+0000","last_active":"2024-04-25T08:40:55.383789+0000","last_peered":"2024-04-25T08:40:55.383789+0000","last_clean":"2024-04-25T08:40:55.383789+0000","last_became_active":"2024-04-25T08:40:54.378120+0000","last_became_peered":"2024-04-25T08:40:54.378120+0000","last_unstale":"2024-04-25T08:40:55.383789+0000","last_undegraded":"2024-04-25T08:40:55.383789+0000","last_fullsized":"2024-04-25T08:40:55.383789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:49.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500706+0000","last_change":"2024-04-25T08:40:54.382334+0000","last_active":"2024-04-25T08:40:55.500706+0000","last_peered":"2024-04-25T08:40:55.500706+0000","last_clean":"2024-04-25T08:40:55.500706+0000","last_became_active":"2024-04-25T08:40:54.382070+0000","last_became_peered":"2024-04-25T08:40:54.382070+0000","last_unstale":"2024-04-25T08:40:55.500706+0000","last_undegraded":"2024-04-25T08:40:55.500706+0000","last_fullsized":"2024-04-25T08:40:55.500706+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:49.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595381+0000","last_change":"2024-04-25T08:40:54.384503+0000","last_active":"2024-04-25T08:40:55.595381+0000","last_peered":"2024-04-25T08:40:55.595381+0000","last_clean":"2024-04-25T08:40:55.595381+0000","last_became_active":"2024-04-25T08:40:54.384141+0000","last_became_peered":"2024-04-25T08:40:54.384141+0000","last_unstale":"2024-04-25T08:40:55.595381+0000","last_undegraded":"2024-04-25T08:40:55.595381+0000","last_fullsized":"2024-04-25T08:40:55.595381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:48.148660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382449+0000","last_change":"2024-04-25T08:40:54.383613+0000","last_active":"2024-04-25T08:40:55.382449+0000","last_peered":"2024-04-25T08:40:55.382449+0000","last_clean":"2024-04-25T08:40:55.382449+0000","last_became_active":"2024-04-25T08:40:54.383329+0000","last_became_peered":"2024-04-25T08:40:54.383329+0000","last_unstale":"2024-04-25T08:40:55.382449+0000","last_undegraded":"2024-04-25T08:40:55.382449+0000","last_fullsized":"2024-04-25T08:40:55.382449+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:58.417166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596145+0000","last_change":"2024-04-25T08:40:54.390511+0000","last_active":"2024-04-25T08:40:55.596145+0000","last_peered":"2024-04-25T08:40:55.596145+0000","last_clean":"2024-04-25T08:40:55.596145+0000","last_became_active":"2024-04-25T08:40:54.390216+0000","last_became_peered":"2024-04-25T08:40:54.390216+0000","last_unstale":"2024-04-25T08:40:55.596145+0000","last_undegraded":"2024-04-25T08:40:55.596145+0000","last_fullsized":"2024-04-25T08:40:55.596145+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:29.262872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381277+0000","last_change":"2024-04-25T08:40:54.386039+0000","last_active":"2024-04-25T08:40:55.381277+0000","last_peered":"2024-04-25T08:40:55.381277+0000","last_clean":"2024-04-25T08:40:55.381277+0000","last_became_active":"2024-04-25T08:40:54.385759+0000","last_became_peered":"2024-04-25T08:40:54.385759+0000","last_unstale":"2024-04-25T08:40:55.381277+0000","last_undegraded":"2024-04-25T08:40:55.381277+0000","last_fullsized":"2024-04-25T08:40:55.381277+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:59.173213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500612+0000","last_change":"2024-04-25T08:40:54.375254+0000","last_active":"2024-04-25T08:40:55.500612+0000","last_peered":"2024-04-25T08:40:55.500612+0000","last_clean":"2024-04-25T08:40:55.500612+0000","last_became_active":"2024-04-25T08:40:54.374962+0000","last_became_peered":"2024-04-25T08:40:54.374962+0000","last_unstale":"2024-04-25T08:40:55.500612+0000","last_undegraded":"2024-04-25T08:40:55.500612+0000","last_fullsized":"2024-04-25T08:40:55.500612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:51.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596611+0000","last_change":"2024-04-25T08:40:54.389554+0000","last_active":"2024-04-25T08:40:55.596611+0000","last_peered":"2024-04-25T08:40:55.596611+0000","last_clean":"2024-04-25T08:40:55.596611+0000","last_became_active":"2024-04-25T08:40:54.386658+0000","last_became_peered":"2024-04-25T08:40:54.386658+0000","last_unstale":"2024-04-25T08:40:55.596611+0000","last_undegraded":"2024-04-25T08:40:55.596611+0000","last_fullsized":"2024-04-25T08:40:55.596611+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596384+0000","last_change":"2024-04-25T08:40:54.380643+0000","last_active":"2024-04-25T08:40:55.596384+0000","last_peered":"2024-04-25T08:40:55.596384+0000","last_clean":"2024-04-25T08:40:55.596384+0000","last_became_active":"2024-04-25T08:40:54.379854+0000","last_became_peered":"2024-04-25T08:40:54.379854+0000","last_unstale":"2024-04-25T08:40:55.596384+0000","last_undegraded":"2024-04-25T08:40:55.596384+0000","last_fullsized":"2024-04-25T08:40:55.596384+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596288+0000","last_change":"2024-04-25T08:40:54.380286+0000","last_active":"2024-04-25T08:40:55.596288+0000","last_peered":"2024-04-25T08:40:55.596288+0000","last_clean":"2024-04-25T08:40:55.596288+0000","last_became_active":"2024-04-25T08:40:54.380011+0000","last_became_peered":"2024-04-25T08:40:54.380011+0000","last_unstale":"2024-04-25T08:40:55.596288+0000","last_undegraded":"2024-04-25T08:40:55.596288+0000","last_fullsized":"2024-04-25T08:40:55.596288+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:53.592947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500572+0000","last_change":"2024-04-25T08:40:54.387452+0000","last_active":"2024-04-25T08:40:55.500572+0000","last_peered":"2024-04-25T08:40:55.500572+0000","last_clean":"2024-04-25T08:40:55.500572+0000","last_became_active":"2024-04-25T08:40:54.387198+0000","last_became_peered":"2024-04-25T08:40:54.387198+0000","last_unstale":"2024-04-25T08:40:55.500572+0000","last_undegraded":"2024-04-25T08:40:55.500572+0000","last_fullsized":"2024-04-25T08:40:55.500572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:37.819157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500522+0000","last_change":"2024-04-25T08:40:54.383950+0000","last_active":"2024-04-25T08:40:55.500522+0000","last_peered":"2024-04-25T08:40:55.500522+0000","last_clean":"2024-04-25T08:40:55.500522+0000","last_became_active":"2024-04-25T08:40:54.382608+0000","last_became_peered":"2024-04-25T08:40:54.382608+0000","last_unstale":"2024-04-25T08:40:55.500522+0000","last_undegraded":"2024-04-25T08:40:55.500522+0000","last_fullsized":"2024-04-25T08:40:55.500522+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:56.551387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596504+0000","last_change":"2024-04-25T08:40:54.388616+0000","last_active":"2024-04-25T08:40:55.596504+0000","last_peered":"2024-04-25T08:40:55.596504+0000","last_clean":"2024-04-25T08:40:55.596504+0000","last_became_active":"2024-04-25T08:40:54.388217+0000","last_became_peered":"2024-04-25T08:40:54.388217+0000","last_unstale":"2024-04-25T08:40:55.596504+0000","last_undegraded":"2024-04-25T08:40:55.596504+0000","last_fullsized":"2024-04-25T08:40:55.596504+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:26.474002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383924+0000","last_change":"2024-04-25T08:40:54.384260+0000","last_active":"2024-04-25T08:40:55.383924+0000","last_peered":"2024-04-25T08:40:55.383924+0000","last_clean":"2024-04-25T08:40:55.383924+0000","last_became_active":"2024-04-25T08:40:54.383734+0000","last_became_peered":"2024-04-25T08:40:54.383734+0000","last_unstale":"2024-04-25T08:40:55.383924+0000","last_undegraded":"2024-04-25T08:40:55.383924+0000","last_fullsized":"2024-04-25T08:40:55.383924+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.502699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383888+0000","last_change":"2024-04-25T08:40:54.380206+0000","last_active":"2024-04-25T08:40:55.383888+0000","last_peered":"2024-04-25T08:40:55.383888+0000","last_clean":"2024-04-25T08:40:55.383888+0000","last_became_active":"2024-04-25T08:40:54.379492+0000","last_became_peered":"2024-04-25T08:40:54.379492+0000","last_unstale":"2024-04-25T08:40:55.383888+0000","last_undegraded":"2024-04-25T08:40:55.383888+0000","last_fullsized":"2024-04-25T08:40:55.383888+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:40.486804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381381+0000","last_change":"2024-04-25T08:40:54.386093+0000","last_active":"2024-04-25T08:40:55.381381+0000","last_peered":"2024-04-25T08:40:55.381381+0000","last_clean":"2024-04-25T08:40:55.381381+0000","last_became_active":"2024-04-25T08:40:54.385187+0000","last_became_peered":"2024-04-25T08:40:54.385187+0000","last_unstale":"2024-04-25T08:40:55.381381+0000","last_undegraded":"2024-04-25T08:40:55.381381+0000","last_fullsized":"2024-04-25T08:40:55.381381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.539986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383508+0000","last_change":"2024-04-25T08:40:54.386486+0000","last_active":"2024-04-25T08:40:55.383508+0000","last_peered":"2024-04-25T08:40:55.383508+0000","last_clean":"2024-04-25T08:40:55.383508+0000","last_became_active":"2024-04-25T08:40:54.386209+0000","last_became_peered":"2024-04-25T08:40:54.386209+0000","last_unstale":"2024-04-25T08:40:55.383508+0000","last_undegraded":"2024-04-25T08:40:55.383508+0000","last_fullsized":"2024-04-25T08:40:55.383508+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:37.819157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382698+0000","last_change":"2024-04-25T08:40:54.388200+0000","last_active":"2024-04-25T08:40:55.382698+0000","last_peered":"2024-04-25T08:40:55.382698+0000","last_clean":"2024-04-25T08:40:55.382698+0000","last_became_active":"2024-04-25T08:40:54.387935+0000","last_became_peered":"2024-04-25T08:40:54.387935+0000","last_unstale":"2024-04-25T08:40:55.382698+0000","last_undegraded":"2024-04-25T08:40:55.382698+0000","last_fullsized":"2024-04-25T08:40:55.382698+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:56.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382793+0000","last_change":"2024-04-25T08:40:54.388543+0000","last_active":"2024-04-25T08:40:55.382793+0000","last_peered":"2024-04-25T08:40:55.382793+0000","last_clean":"2024-04-25T08:40:55.382793+0000","last_became_active":"2024-04-25T08:40:54.387411+0000","last_became_peered":"2024-04-25T08:40:54.387411+0000","last_unstale":"2024-04-25T08:40:55.382793+0000","last_undegraded":"2024-04-25T08:40:55.382793+0000","last_fullsized":"2024-04-25T08:40:55.382793+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:56.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.380999+0000","last_change":"2024-04-25T08:40:54.380867+0000","last_active":"2024-04-25T08:40:55.380999+0000","last_peered":"2024-04-25T08:40:55.380999+0000","last_clean":"2024-04-25T08:40:55.380999+0000","last_became_active":"2024-04-25T08:40:54.380595+0000","last_became_peered":"2024-04-25T08:40:54.380595+0000","last_unstale":"2024-04-25T08:40:55.380999+0000","last_undegraded":"2024-04-25T08:40:55.380999+0000","last_fullsized":"2024-04-25T08:40:55.380999+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:58.196801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382612+0000","last_change":"2024-04-25T08:40:54.389841+0000","last_active":"2024-04-25T08:40:55.382612+0000","last_peered":"2024-04-25T08:40:55.382612+0000","last_clean":"2024-04-25T08:40:55.382612+0000","last_became_active":"2024-04-25T08:40:54.389572+0000","last_became_peered":"2024-04-25T08:40:54.389572+0000","last_unstale":"2024-04-25T08:40:55.382612+0000","last_undegraded":"2024-04-25T08:40:55.382612+0000","last_fullsized":"2024-04-25T08:40:55.382612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:04.970606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"131'4","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.384013+0000","last_change":"2024-04-25T08:40:53.379026+0000","last_active":"2024-04-25T08:40:55.384013+0000","last_peered":"2024-04-25T08:40:55.384013+0000","last_clean":"2024-04-25T08:40:55.384013+0000","last_became_active":"2024-04-25T08:40:53.378751+0000","last_became_peered":"2024-04-25T08:40:53.378751+0000","last_unstale":"2024-04-25T08:40:55.384013+0000","last_undegraded":"2024-04-25T08:40:55.384013+0000","last_fullsized":"2024-04-25T08:40:55.384013+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.556400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500758+0000","last_change":"2024-04-25T08:40:54.378840+0000","last_active":"2024-04-25T08:40:55.500758+0000","last_peered":"2024-04-25T08:40:55.500758+0000","last_clean":"2024-04-25T08:40:55.500758+0000","last_became_active":"2024-04-25T08:40:54.378566+0000","last_became_peered":"2024-04-25T08:40:54.378566+0000","last_unstale":"2024-04-25T08:40:55.500758+0000","last_undegraded":"2024-04-25T08:40:55.500758+0000","last_fullsized":"2024-04-25T08:40:55.500758+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:40.486804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382917+0000","last_change":"2024-04-25T08:40:54.387624+0000","last_active":"2024-04-25T08:40:55.382917+0000","last_peered":"2024-04-25T08:40:55.382917+0000","last_clean":"2024-04-25T08:40:55.382917+0000","last_became_active":"2024-04-25T08:40:54.387359+0000","last_became_peered":"2024-04-25T08:40:54.387359+0000","last_unstale":"2024-04-25T08:40:55.382917+0000","last_undegraded":"2024-04-25T08:40:55.382917+0000","last_fullsized":"2024-04-25T08:40:55.382917+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:22.825790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383795+0000","last_change":"2024-04-25T08:40:53.373572+0000","last_active":"2024-04-25T08:40:55.383795+0000","last_peered":"2024-04-25T08:40:55.383795+0000","last_clean":"2024-04-25T08:40:55.383795+0000","last_became_active":"2024-04-25T08:40:53.373290+0000","last_became_peered":"2024-04-25T08:40:53.373290+0000","last_unstale":"2024-04-25T08:40:55.383795+0000","last_undegraded":"2024-04-25T08:40:55.383795+0000","last_fullsized":"2024-04-25T08:40:55.383795+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:40.850065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383312+0000","last_change":"2024-04-25T08:40:54.388330+0000","last_active":"2024-04-25T08:40:55.383312+0000","last_peered":"2024-04-25T08:40:55.383312+0000","last_clean":"2024-04-25T08:40:55.383312+0000","last_became_active":"2024-04-25T08:40:54.387849+0000","last_became_peered":"2024-04-25T08:40:54.387849+0000","last_unstale":"2024-04-25T08:40:55.383312+0000","last_undegraded":"2024-04-25T08:40:55.383312+0000","last_fullsized":"2024-04-25T08:40:55.383312+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:40.899090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595098+0000","last_change":"2024-04-25T08:40:53.376507+0000","last_active":"2024-04-25T08:40:55.595098+0000","last_peered":"2024-04-25T08:40:55.595098+0000","last_clean":"2024-04-25T08:40:55.595098+0000","last_became_active":"2024-04-25T08:40:53.376186+0000","last_became_peered":"2024-04-25T08:40:53.376186+0000","last_unstale":"2024-04-25T08:40:55.595098+0000","last_undegraded":"2024-04-25T08:40:55.595098+0000","last_fullsized":"2024-04-25T08:40:55.595098+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:50.437720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383020+0000","last_change":"2024-04-25T08:40:53.376440+0000","last_active":"2024-04-25T08:40:55.383020+0000","last_peered":"2024-04-25T08:40:55.383020+0000","last_clean":"2024-04-25T08:40:55.383020+0000","last_became_active":"2024-04-25T08:40:53.376169+0000","last_became_peered":"2024-04-25T08:40:53.376169+0000","last_unstale":"2024-04-25T08:40:55.383020+0000","last_undegraded":"2024-04-25T08:40:55.383020+0000","last_fullsized":"2024-04-25T08:40:55.383020+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:52.489674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500789+0000","last_change":"2024-04-25T08:40:53.380621+0000","last_active":"2024-04-25T08:40:55.500789+0000","last_peered":"2024-04-25T08:40:55.500789+0000","last_clean":"2024-04-25T08:40:55.500789+0000","last_became_active":"2024-04-25T08:40:53.380150+0000","last_became_peered":"2024-04-25T08:40:53.380150+0000","last_unstale":"2024-04-25T08:40:55.500789+0000","last_undegraded":"2024-04-25T08:40:55.500789+0000","last_fullsized":"2024-04-25T08:40:55.500789+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:56.303715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595249+0000","last_change":"2024-04-25T08:40:54.387442+0000","last_active":"2024-04-25T08:40:55.595249+0000","last_peered":"2024-04-25T08:40:55.595249+0000","last_clean":"2024-04-25T08:40:55.595249+0000","last_became_active":"2024-04-25T08:40:54.387157+0000","last_became_peered":"2024-04-25T08:40:54.387157+0000","last_unstale":"2024-04-25T08:40:55.595249+0000","last_undegraded":"2024-04-25T08:40:55.595249+0000","last_fullsized":"2024-04-25T08:40:55.595249+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:48.139944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382845+0000","last_change":"2024-04-25T08:40:54.376953+0000","last_active":"2024-04-25T08:40:55.382845+0000","last_peered":"2024-04-25T08:40:55.382845+0000","last_clean":"2024-04-25T08:40:55.382845+0000","last_became_active":"2024-04-25T08:40:54.376686+0000","last_became_peered":"2024-04-25T08:40:54.376686+0000","last_unstale":"2024-04-25T08:40:55.382845+0000","last_undegraded":"2024-04-25T08:40:55.382845+0000","last_fullsized":"2024-04-25T08:40:55.382845+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:22.825790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383572+0000","last_change":"2024-04-25T08:40:54.385830+0000","last_active":"2024-04-25T08:40:55.383572+0000","last_peered":"2024-04-25T08:40:55.383572+0000","last_clean":"2024-04-25T08:40:55.383572+0000","last_became_active":"2024-04-25T08:40:54.384994+0000","last_became_peered":"2024-04-25T08:40:54.384994+0000","last_unstale":"2024-04-25T08:40:55.383572+0000","last_undegraded":"2024-04-25T08:40:55.383572+0000","last_fullsized":"2024-04-25T08:40:55.383572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:51.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381354+0000","last_change":"2024-04-25T08:40:54.383836+0000","last_active":"2024-04-25T08:40:55.381354+0000","last_peered":"2024-04-25T08:40:55.381354+0000","last_clean":"2024-04-25T08:40:55.381354+0000","last_became_active":"2024-04-25T08:40:54.383486+0000","last_became_peered":"2024-04-25T08:40:54.383486+0000","last_unstale":"2024-04-25T08:40:55.381354+0000","last_undegraded":"2024-04-25T08:40:55.381354+0000","last_fullsized":"2024-04-25T08:40:55.381354+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:20.048786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383208+0000","last_change":"2024-04-25T08:40:53.377899+0000","last_active":"2024-04-25T08:40:55.383208+0000","last_peered":"2024-04-25T08:40:55.383208+0000","last_clean":"2024-04-25T08:40:55.383208+0000","last_became_active":"2024-04-25T08:40:53.377626+0000","last_became_peered":"2024-04-25T08:40:53.377626+0000","last_unstale":"2024-04-25T08:40:55.383208+0000","last_undegraded":"2024-04-25T08:40:55.383208+0000","last_fullsized":"2024-04-25T08:40:55.383208+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:08.112732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"131'8","reported_seq":22,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596483+0000","last_change":"2024-04-25T08:40:53.378565+0000","last_active":"2024-04-25T08:40:55.596483+0000","last_peered":"2024-04-25T08:40:55.596483+0000","last_clean":"2024-04-25T08:40:55.596483+0000","last_became_active":"2024-04-25T08:40:53.378251+0000","last_became_peered":"2024-04-25T08:40:53.378251+0000","last_unstale":"2024-04-25T08:40:55.596483+0000","last_undegraded":"2024-04-25T08:40:55.596483+0000","last_fullsized":"2024-04-25T08:40:55.596483+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:03.962369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382898+0000","last_change":"2024-04-25T08:40:54.372817+0000","last_active":"2024-04-25T08:40:55.382898+0000","last_peered":"2024-04-25T08:40:55.382898+0000","last_clean":"2024-04-25T08:40:55.382898+0000","last_became_active":"2024-04-25T08:40:54.371590+0000","last_became_peered":"2024-04-25T08:40:54.371590+0000","last_unstale":"2024-04-25T08:40:55.382898+0000","last_undegraded":"2024-04-25T08:40:55.382898+0000","last_fullsized":"2024-04-25T08:40:55.382898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:40.899090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500851+0000","last_change":"2024-04-25T08:40:53.382667+0000","last_active":"2024-04-25T08:40:55.500851+0000","last_peered":"2024-04-25T08:40:55.500851+0000","last_clean":"2024-04-25T08:40:55.500851+0000","last_became_active":"2024-04-25T08:40:53.382436+0000","last_became_peered":"2024-04-25T08:40:53.382436+0000","last_unstale":"2024-04-25T08:40:55.500851+0000","last_undegraded":"2024-04-25T08:40:55.500851+0000","last_fullsized":"2024-04-25T08:40:55.500851+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:39.954950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383651+0000","last_change":"2024-04-25T08:40:53.384093+0000","last_active":"2024-04-25T08:40:55.383651+0000","last_peered":"2024-04-25T08:40:55.383651+0000","last_clean":"2024-04-25T08:40:55.383651+0000","last_became_active":"2024-04-25T08:40:53.383830+0000","last_became_peered":"2024-04-25T08:40:53.383830+0000","last_unstale":"2024-04-25T08:40:55.383651+0000","last_undegraded":"2024-04-25T08:40:55.383651+0000","last_fullsized":"2024-04-25T08:40:55.383651+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:40.850065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595793+0000","last_change":"2024-04-25T08:40:53.377620+0000","last_active":"2024-04-25T08:40:55.595793+0000","last_peered":"2024-04-25T08:40:55.595793+0000","last_clean":"2024-04-25T08:40:55.595793+0000","last_became_active":"2024-04-25T08:40:53.377340+0000","last_became_peered":"2024-04-25T08:40:53.377340+0000","last_unstale":"2024-04-25T08:40:55.595793+0000","last_undegraded":"2024-04-25T08:40:55.595793+0000","last_fullsized":"2024-04-25T08:40:55.595793+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:33.063592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381144+0000","last_change":"2024-04-25T08:40:54.382916+0000","last_active":"2024-04-25T08:40:55.381144+0000","last_peered":"2024-04-25T08:40:55.381144+0000","last_clean":"2024-04-25T08:40:55.381144+0000","last_became_active":"2024-04-25T08:40:54.382634+0000","last_became_peered":"2024-04-25T08:40:54.382634+0000","last_unstale":"2024-04-25T08:40:55.381144+0000","last_undegraded":"2024-04-25T08:40:55.381144+0000","last_fullsized":"2024-04-25T08:40:55.381144+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:28.055098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383840+0000","last_change":"2024-04-25T08:40:53.372724+0000","last_active":"2024-04-25T08:40:55.383840+0000","last_peered":"2024-04-25T08:40:55.383840+0000","last_clean":"2024-04-25T08:40:55.383840+0000","last_became_active":"2024-04-25T08:40:53.372440+0000","last_became_peered":"2024-04-25T08:40:53.372440+0000","last_unstale":"2024-04-25T08:40:55.383840+0000","last_undegraded":"2024-04-25T08:40:55.383840+0000","last_fullsized":"2024-04-25T08:40:55.383840+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:46.131350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500809+0000","last_change":"2024-04-25T08:40:54.378104+0000","last_active":"2024-04-25T08:40:55.500809+0000","last_peered":"2024-04-25T08:40:55.500809+0000","last_clean":"2024-04-25T08:40:55.500809+0000","last_became_active":"2024-04-25T08:40:54.377753+0000","last_became_peered":"2024-04-25T08:40:54.377753+0000","last_unstale":"2024-04-25T08:40:55.500809+0000","last_undegraded":"2024-04-25T08:40:55.500809+0000","last_fullsized":"2024-04-25T08:40:55.500809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:25.693049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596583+0000","last_change":"2024-04-25T08:40:53.376843+0000","last_active":"2024-04-25T08:40:55.596583+0000","last_peered":"2024-04-25T08:40:55.596583+0000","last_clean":"2024-04-25T08:40:55.596583+0000","last_became_active":"2024-04-25T08:40:53.376238+0000","last_became_peered":"2024-04-25T08:40:53.376238+0000","last_unstale":"2024-04-25T08:40:55.596583+0000","last_undegraded":"2024-04-25T08:40:55.596583+0000","last_fullsized":"2024-04-25T08:40:55.596583+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:55.097704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383108+0000","last_change":"2024-04-25T08:40:54.384630+0000","last_active":"2024-04-25T08:40:55.383108+0000","last_peered":"2024-04-25T08:40:55.383108+0000","last_clean":"2024-04-25T08:40:55.383108+0000","last_became_active":"2024-04-25T08:40:54.384077+0000","last_became_peered":"2024-04-25T08:40:54.384077+0000","last_unstale":"2024-04-25T08:40:55.383108+0000","last_undegraded":"2024-04-25T08:40:55.383108+0000","last_fullsized":"2024-04-25T08:40:55.383108+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:56.551387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"132'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:56.368767+0000","last_change":"2024-04-25T08:40:53.378186+0000","last_active":"2024-04-25T08:40:56.368767+0000","last_peered":"2024-04-25T08:40:56.368767+0000","last_clean":"2024-04-25T08:40:56.368767+0000","last_became_active":"2024-04-25T08:40:53.377880+0000","last_became_peered":"2024-04-25T08:40:53.377880+0000","last_unstale":"2024-04-25T08:40:56.368767+0000","last_undegraded":"2024-04-25T08:40:56.368767+0000","last_fullsized":"2024-04-25T08:40:56.368767+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:47.172417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383430+0000","last_change":"2024-04-25T08:40:53.381261+0000","last_active":"2024-04-25T08:40:55.383430+0000","last_peered":"2024-04-25T08:40:55.383430+0000","last_clean":"2024-04-25T08:40:55.383430+0000","last_became_active":"2024-04-25T08:40:53.380920+0000","last_became_peered":"2024-04-25T08:40:53.380920+0000","last_unstale":"2024-04-25T08:40:55.383430+0000","last_undegraded":"2024-04-25T08:40:55.383430+0000","last_fullsized":"2024-04-25T08:40:55.383430+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:39.954950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381178+0000","last_change":"2024-04-25T08:40:54.386615+0000","last_active":"2024-04-25T08:40:55.381178+0000","last_peered":"2024-04-25T08:40:55.381178+0000","last_clean":"2024-04-25T08:40:55.381178+0000","last_became_active":"2024-04-25T08:40:54.386359+0000","last_became_peered":"2024-04-25T08:40:54.386359+0000","last_unstale":"2024-04-25T08:40:55.381178+0000","last_undegraded":"2024-04-25T08:40:55.381178+0000","last_fullsized":"2024-04-25T08:40:55.381178+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.093642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595669+0000","last_change":"2024-04-25T08:40:53.378763+0000","last_active":"2024-04-25T08:40:55.595669+0000","last_peered":"2024-04-25T08:40:55.595669+0000","last_clean":"2024-04-25T08:40:55.595669+0000","last_became_active":"2024-04-25T08:40:53.378453+0000","last_became_peered":"2024-04-25T08:40:53.378453+0000","last_unstale":"2024-04-25T08:40:55.595669+0000","last_undegraded":"2024-04-25T08:40:55.595669+0000","last_fullsized":"2024-04-25T08:40:55.595669+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:57.408929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595573+0000","last_change":"2024-04-25T08:40:54.387431+0000","last_active":"2024-04-25T08:40:55.595573+0000","last_peered":"2024-04-25T08:40:55.595573+0000","last_clean":"2024-04-25T08:40:55.595573+0000","last_became_active":"2024-04-25T08:40:54.387149+0000","last_became_peered":"2024-04-25T08:40:54.387149+0000","last_unstale":"2024-04-25T08:40:55.595573+0000","last_undegraded":"2024-04-25T08:40:55.595573+0000","last_fullsized":"2024-04-25T08:40:55.595573+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:49.064739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500551+0000","last_change":"2024-04-25T08:40:53.385220+0000","last_active":"2024-04-25T08:40:55.500551+0000","last_peered":"2024-04-25T08:40:55.500551+0000","last_clean":"2024-04-25T08:40:55.500551+0000","last_became_active":"2024-04-25T08:40:53.384943+0000","last_became_peered":"2024-04-25T08:40:53.384943+0000","last_unstale":"2024-04-25T08:40:55.500551+0000","last_undegraded":"2024-04-25T08:40:55.500551+0000","last_fullsized":"2024-04-25T08:40:55.500551+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:16.391337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595687+0000","last_change":"2024-04-25T08:40:54.388970+0000","last_active":"2024-04-25T08:40:55.595687+0000","last_peered":"2024-04-25T08:40:55.595687+0000","last_clean":"2024-04-25T08:40:55.595687+0000","last_became_active":"2024-04-25T08:40:54.388677+0000","last_became_peered":"2024-04-25T08:40:54.388677+0000","last_unstale":"2024-04-25T08:40:55.595687+0000","last_undegraded":"2024-04-25T08:40:55.595687+0000","last_fullsized":"2024-04-25T08:40:55.595687+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:37.825506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595830+0000","last_change":"2024-04-25T08:40:54.381075+0000","last_active":"2024-04-25T08:40:55.595830+0000","last_peered":"2024-04-25T08:40:55.595830+0000","last_clean":"2024-04-25T08:40:55.595830+0000","last_became_active":"2024-04-25T08:40:54.380766+0000","last_became_peered":"2024-04-25T08:40:54.380766+0000","last_unstale":"2024-04-25T08:40:55.595830+0000","last_undegraded":"2024-04-25T08:40:55.595830+0000","last_fullsized":"2024-04-25T08:40:55.595830+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:56.441011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595339+0000","last_change":"2024-04-25T08:40:53.378417+0000","last_active":"2024-04-25T08:40:55.595339+0000","last_peered":"2024-04-25T08:40:55.595339+0000","last_clean":"2024-04-25T08:40:55.595339+0000","last_became_active":"2024-04-25T08:40:53.378139+0000","last_became_peered":"2024-04-25T08:40:53.378139+0000","last_unstale":"2024-04-25T08:40:55.595339+0000","last_undegraded":"2024-04-25T08:40:55.595339+0000","last_fullsized":"2024-04-25T08:40:55.595339+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.250586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383951+0000","last_change":"2024-04-25T08:40:54.378981+0000","last_active":"2024-04-25T08:40:55.383951+0000","last_peered":"2024-04-25T08:40:55.383951+0000","last_clean":"2024-04-25T08:40:55.383951+0000","last_became_active":"2024-04-25T08:40:54.378496+0000","last_became_peered":"2024-04-25T08:40:54.378496+0000","last_unstale":"2024-04-25T08:40:55.383951+0000","last_undegraded":"2024-04-25T08:40:55.383951+0000","last_fullsized":"2024-04-25T08:40:55.383951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:25.693049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":372,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T08:40:50.676773+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:40:50.676773+0000","last_peered":"2024-04-25T08:40:50.676773+0000","last_clean":"2024-04-25T08:40:50.676773+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:40:50.676773+0000","last_undegraded":"2024-04-25T08:40:50.676773+0000","last_fullsized":"2024-04-25T08:40:50.676773+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":76,"acting":76,"num_store_stats":5},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":674,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":34,"acting":34,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65264,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306576,"statfs":{"total":96636764160,"available":96569933824,"internally_reserved":0,"allocated":2195456,"data_stored":1334949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59172,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312668,"statfs":{"total":96636764160,"available":96576172032,"internally_reserved":0,"allocated":1658880,"data_stored":771980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55012,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316828,"statfs":{"total":96636764160,"available":96580431872,"internally_reserved":0,"allocated":1654784,"data_stored":771330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68892,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302948,"statfs":{"total":96636764160,"available":96566218752,"internally_reserved":0,"allocated":1650688,"data_stored":771308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73004,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298836,"statfs":{"total":96636764160,"available":96562008064,"internally_reserved":0,"allocated":1601536,"data_stored":744581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59116,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312724,"statfs":{"total":96636764160,"available":96576229376,"internally_reserved":0,"allocated":1601536,"data_stored":744581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":1654784,"data_stored":771378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69416,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302424,"statfs":{"total":96636764160,"available":96565682176,"internally_reserved":0,"allocated":2252800,"data_stored":1361732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:57.730 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:57.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.872+0000 7f73c449a640 1 Processor -- start 2024-04-25T08:40:57.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.872+0000 7f73c449a640 1 -- start start 2024-04-25T08:40:57.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c449a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f73bc06c110 0x7f73bc06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c449a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73bc15df50 0x7f73bc0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c449a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73bc06ca30 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c449a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73bc0cf2d0 con 0x7f73bc06c110 2024-04-25T08:40:57.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c449a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0ceff0 con 0x7f73bc15df50 2024-04-25T08:40:57.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c2a37640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f73bc0ce2e0 0x7f73bc0d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51480/0 (socket says 172.21.15.38:51480) 2024-04-25T08:40:57.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c2a37640 1 -- 172.21.15.38:0/1777979068 learned_addr learned my addr 172.21.15.38:0/1777979068 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:57.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c1a35640 1 --2- 172.21.15.38:0/1777979068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73bc15df50 0x7f73bc0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c2236640 1 --2- 172.21.15.38:0/1777979068 >> v2:172.21.15.107:3300/0 conn(0x7f73bc06c110 0x7f73bc06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c1234640 1 -- 172.21.15.38:0/1777979068 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287072237 0 0) 0x7f73bc06ca30 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c1234640 1 -- 172.21.15.38:0/1777979068 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73a8003520 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c1234640 1 -- 172.21.15.38:0/1777979068 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1918751447 0 0) 0x7f73a8003520 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c1a35640 1 -- 172.21.15.38:0/1777979068 >> v2:172.21.15.107:3300/0 conn(0x7f73bc06c110 msgr2=0x7f73bc06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:57.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c1a35640 1 --2- 172.21.15.38:0/1777979068 >> v2:172.21.15.107:3300/0 conn(0x7f73bc06c110 0x7f73bc06c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c1a35640 1 -- 172.21.15.38:0/1777979068 >> v1:172.21.15.38:6789/0 conn(0x7f73bc0ce2e0 legacy=0x7f73bc0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.873+0000 7f73c1a35640 1 -- 172.21.15.38:0/1777979068 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc0d1f70 con 0x7f73bc15df50 2024-04-25T08:40:57.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.874+0000 7f73c1a35640 1 --2- 172.21.15.38:0/1777979068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73bc15df50 0x7f73bc0ce7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0097a0 tx=0x7f73ac02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffc1203ec15767b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:57.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.874+0000 7f73c1234640 1 -- 172.21.15.38:0/1777979068 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac037070 con 0x7f73bc15df50 2024-04-25T08:40:57.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.874+0000 7f73c1234640 1 -- 172.21.15.38:0/1777979068 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ac02aba0 con 0x7f73bc15df50 2024-04-25T08:40:57.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.874+0000 7f73c1234640 1 -- 172.21.15.38:0/1777979068 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b40019f0 con 0x7f73bc15df50 2024-04-25T08:40:57.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.875+0000 7f73c449a640 1 -- 172.21.15.38:0/1777979068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73bc15df50 msgr2=0x7f73bc0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.875+0000 7f73c449a640 1 --2- 172.21.15.38:0/1777979068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73bc15df50 0x7f73bc0ce7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0097a0 tx=0x7f73ac02a110 comp rx=0 tx=0).stop 2024-04-25T08:40:57.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.875+0000 7f73c449a640 1 -- 172.21.15.38:0/1777979068 shutdown_connections 2024-04-25T08:40:57.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.875+0000 7f73c449a640 1 --2- 172.21.15.38:0/1777979068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73bc15df50 0x7f73bc0ce7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.875+0000 7f73c449a640 1 --2- 172.21.15.38:0/1777979068 >> v2:172.21.15.107:3300/0 conn(0x7f73bc06c110 0x7f73bc06c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.875+0000 7f73c449a640 1 -- 172.21.15.38:0/1777979068 >> 172.21.15.38:0/1777979068 conn(0x7f73bc0d3fb0 msgr2=0x7f73bc0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:57.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.875+0000 7f73c449a640 1 -- 172.21.15.38:0/1777979068 shutdown_connections 2024-04-25T08:40:57.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.875+0000 7f73c449a640 1 -- 172.21.15.38:0/1777979068 wait complete. 2024-04-25T08:40:57.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.876+0000 7f73c449a640 1 Processor -- start 2024-04-25T08:40:57.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.876+0000 7f73c449a640 1 -- start start 2024-04-25T08:40:57.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c449a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73bc06c110 0x7f73bc15e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c449a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73bc0ce2e0 0x7f73bc1f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c449a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73bc15df50 0x7f73bc160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c449a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0cece0 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c449a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0d2530 con 0x7f73bc06c110 2024-04-25T08:40:57.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c449a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0d3160 con 0x7f73bc15df50 2024-04-25T08:40:57.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c1a35640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73bc0ce2e0 0x7f73bc1f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c1a35640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73bc0ce2e0 0x7f73bc1f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49798/0 (socket says 172.21.15.38:49798) 2024-04-25T08:40:57.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c1a35640 1 -- 172.21.15.38:0/473533782 learned_addr learned my addr 172.21.15.38:0/473533782 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c2a37640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73bc15df50 0x7f73bc160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c2236640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73bc06c110 0x7f73bc15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c1a35640 1 -- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73bc15df50 msgr2=0x7f73bc160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c1a35640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73bc15df50 0x7f73bc160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c1a35640 1 -- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73bc06c110 msgr2=0x7f73bc15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c1a35640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73bc06c110 0x7f73bc15e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c1a35640 1 -- 172.21.15.38:0/473533782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc15eb80 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c2a37640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73bc15df50 0x7f73bc160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.877+0000 7f73c2236640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73bc06c110 0x7f73bc15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.878+0000 7f73c1a35640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73bc0ce2e0 0x7f73bc1f0c00 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0098c0 tx=0x7f73ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.878+0000 7f73b2ffd640 1 -- 172.21.15.38:0/473533782 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac041030 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.878+0000 7f73b2ffd640 1 -- 172.21.15.38:0/473533782 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73ac03d420 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.878+0000 7f73c449a640 1 -- 172.21.15.38:0/473533782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc15ee60 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.878+0000 7f73b2ffd640 1 -- 172.21.15.38:0/473533782 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73ac037040 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.878+0000 7f73c449a640 1 -- 172.21.15.38:0/473533782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc15f420 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.879+0000 7f73c449a640 1 -- 172.21.15.38:0/473533782 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73bc068e80 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.879+0000 7f73b2ffd640 1 -- 172.21.15.38:0/473533782 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73ac044070 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.880+0000 7f73b2ffd640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f73a006b4d0 0x7f73a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:57.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.880+0000 7f73b2ffd640 1 -- 172.21.15.38:0/473533782 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f73ac0aa800 con 0x7f73bc0ce2e0 2024-04-25T08:40:57.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.880+0000 7f73c2236640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f73a006b4d0 0x7f73a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:57.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.881+0000 7f73c2236640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f73a006b4d0 0x7f73a006d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f73bc1617b0 tx=0x7f73b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:57.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:57.884+0000 7f73b2ffd640 1 -- 172.21.15.38:0/473533782 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73ac0794e0 con 0x7f73bc0ce2e0 2024-04-25T08:40:58.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.064+0000 7f73c449a640 1 -- 172.21.15.38:0/473533782 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73bc0d0540 con 0x7f73a006b4d0 2024-04-25T08:40:58.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.071+0000 7f73b2ffd640 1 -- 172.21.15.38:0/473533782 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246178 (secure 0 0 0) 0x7f73bc0d0540 con 0x7f73a006b4d0 2024-04-25T08:40:58.066 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:58.067 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:40:58.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.074+0000 7f73c449a640 1 -- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f73a006b4d0 msgr2=0x7f73a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.074+0000 7f73c449a640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f73a006b4d0 0x7f73a006d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f73bc1617b0 tx=0x7f73b8008040 comp rx=0 tx=0).stop 2024-04-25T08:40:58.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.074+0000 7f73c449a640 1 -- 172.21.15.38:0/473533782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73bc0ce2e0 msgr2=0x7f73bc1f0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.074+0000 7f73c449a640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73bc0ce2e0 0x7f73bc1f0c00 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f73ac0098c0 tx=0x7f73ac0046c0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.074+0000 7f73c449a640 1 -- 172.21.15.38:0/473533782 shutdown_connections 2024-04-25T08:40:58.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.074+0000 7f73c449a640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f73a006b4d0 0x7f73a006d990 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.074+0000 7f73c449a640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73bc15df50 0x7f73bc160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.074+0000 7f73c449a640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73bc0ce2e0 0x7f73bc1f0c00 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.074+0000 7f73c449a640 1 --2- 172.21.15.38:0/473533782 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73bc06c110 0x7f73bc15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.074+0000 7f73c449a640 1 -- 172.21.15.38:0/473533782 >> 172.21.15.38:0/473533782 conn(0x7f73bc0d3fb0 msgr2=0x7f73bc0b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:58.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.074+0000 7f73c449a640 1 -- 172.21.15.38:0/473533782 shutdown_connections 2024-04-25T08:40:58.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.074+0000 7f73c449a640 1 -- 172.21.15.38:0/473533782 wait complete. 2024-04-25T08:40:58.077 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-04-25T08:40:57.335965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591042,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":182,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":509044,"kb_used_data":13936,"kb_used_omap":12,"kb_used_meta":494963,"kb_avail":754465676,"statfs":{"total":773094113280,"available":772572852224,"internally_reserved":0,"allocated":14270464,"data_stored":7271839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":506842672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":674,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.981544"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500770+0000","last_change":"2024-04-25T08:40:54.385506+0000","last_active":"2024-04-25T08:40:55.500770+0000","last_peered":"2024-04-25T08:40:55.500770+0000","last_clean":"2024-04-25T08:40:55.500770+0000","last_became_active":"2024-04-25T08:40:54.385108+0000","last_became_peered":"2024-04-25T08:40:54.385108+0000","last_unstale":"2024-04-25T08:40:55.500770+0000","last_undegraded":"2024-04-25T08:40:55.500770+0000","last_fullsized":"2024-04-25T08:40:55.500770+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.502699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383789+0000","last_change":"2024-04-25T08:40:54.378766+0000","last_active":"2024-04-25T08:40:55.383789+0000","last_peered":"2024-04-25T08:40:55.383789+0000","last_clean":"2024-04-25T08:40:55.383789+0000","last_became_active":"2024-04-25T08:40:54.378120+0000","last_became_peered":"2024-04-25T08:40:54.378120+0000","last_unstale":"2024-04-25T08:40:55.383789+0000","last_undegraded":"2024-04-25T08:40:55.383789+0000","last_fullsized":"2024-04-25T08:40:55.383789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:49.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500706+0000","last_change":"2024-04-25T08:40:54.382334+0000","last_active":"2024-04-25T08:40:55.500706+0000","last_peered":"2024-04-25T08:40:55.500706+0000","last_clean":"2024-04-25T08:40:55.500706+0000","last_became_active":"2024-04-25T08:40:54.382070+0000","last_became_peered":"2024-04-25T08:40:54.382070+0000","last_unstale":"2024-04-25T08:40:55.500706+0000","last_undegraded":"2024-04-25T08:40:55.500706+0000","last_fullsized":"2024-04-25T08:40:55.500706+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:49.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595381+0000","last_change":"2024-04-25T08:40:54.384503+0000","last_active":"2024-04-25T08:40:55.595381+0000","last_peered":"2024-04-25T08:40:55.595381+0000","last_clean":"2024-04-25T08:40:55.595381+0000","last_became_active":"2024-04-25T08:40:54.384141+0000","last_became_peered":"2024-04-25T08:40:54.384141+0000","last_unstale":"2024-04-25T08:40:55.595381+0000","last_undegraded":"2024-04-25T08:40:55.595381+0000","last_fullsized":"2024-04-25T08:40:55.595381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:48.148660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382449+0000","last_change":"2024-04-25T08:40:54.383613+0000","last_active":"2024-04-25T08:40:55.382449+0000","last_peered":"2024-04-25T08:40:55.382449+0000","last_clean":"2024-04-25T08:40:55.382449+0000","last_became_active":"2024-04-25T08:40:54.383329+0000","last_became_peered":"2024-04-25T08:40:54.383329+0000","last_unstale":"2024-04-25T08:40:55.382449+0000","last_undegraded":"2024-04-25T08:40:55.382449+0000","last_fullsized":"2024-04-25T08:40:55.382449+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:58.417166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596145+0000","last_change":"2024-04-25T08:40:54.390511+0000","last_active":"2024-04-25T08:40:55.596145+0000","last_peered":"2024-04-25T08:40:55.596145+0000","last_clean":"2024-04-25T08:40:55.596145+0000","last_became_active":"2024-04-25T08:40:54.390216+0000","last_became_peered":"2024-04-25T08:40:54.390216+0000","last_unstale":"2024-04-25T08:40:55.596145+0000","last_undegraded":"2024-04-25T08:40:55.596145+0000","last_fullsized":"2024-04-25T08:40:55.596145+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:29.262872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381277+0000","last_change":"2024-04-25T08:40:54.386039+0000","last_active":"2024-04-25T08:40:55.381277+0000","last_peered":"2024-04-25T08:40:55.381277+0000","last_clean":"2024-04-25T08:40:55.381277+0000","last_became_active":"2024-04-25T08:40:54.385759+0000","last_became_peered":"2024-04-25T08:40:54.385759+0000","last_unstale":"2024-04-25T08:40:55.381277+0000","last_undegraded":"2024-04-25T08:40:55.381277+0000","last_fullsized":"2024-04-25T08:40:55.381277+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:59.173213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500612+0000","last_change":"2024-04-25T08:40:54.375254+0000","last_active":"2024-04-25T08:40:55.500612+0000","last_peered":"2024-04-25T08:40:55.500612+0000","last_clean":"2024-04-25T08:40:55.500612+0000","last_became_active":"2024-04-25T08:40:54.374962+0000","last_became_peered":"2024-04-25T08:40:54.374962+0000","last_unstale":"2024-04-25T08:40:55.500612+0000","last_undegraded":"2024-04-25T08:40:55.500612+0000","last_fullsized":"2024-04-25T08:40:55.500612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:51.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596611+0000","last_change":"2024-04-25T08:40:54.389554+0000","last_active":"2024-04-25T08:40:55.596611+0000","last_peered":"2024-04-25T08:40:55.596611+0000","last_clean":"2024-04-25T08:40:55.596611+0000","last_became_active":"2024-04-25T08:40:54.386658+0000","last_became_peered":"2024-04-25T08:40:54.386658+0000","last_unstale":"2024-04-25T08:40:55.596611+0000","last_undegraded":"2024-04-25T08:40:55.596611+0000","last_fullsized":"2024-04-25T08:40:55.596611+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596384+0000","last_change":"2024-04-25T08:40:54.380643+0000","last_active":"2024-04-25T08:40:55.596384+0000","last_peered":"2024-04-25T08:40:55.596384+0000","last_clean":"2024-04-25T08:40:55.596384+0000","last_became_active":"2024-04-25T08:40:54.379854+0000","last_became_peered":"2024-04-25T08:40:54.379854+0000","last_unstale":"2024-04-25T08:40:55.596384+0000","last_undegraded":"2024-04-25T08:40:55.596384+0000","last_fullsized":"2024-04-25T08:40:55.596384+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596288+0000","last_change":"2024-04-25T08:40:54.380286+0000","last_active":"2024-04-25T08:40:55.596288+0000","last_peered":"2024-04-25T08:40:55.596288+0000","last_clean":"2024-04-25T08:40:55.596288+0000","last_became_active":"2024-04-25T08:40:54.380011+0000","last_became_peered":"2024-04-25T08:40:54.380011+0000","last_unstale":"2024-04-25T08:40:55.596288+0000","last_undegraded":"2024-04-25T08:40:55.596288+0000","last_fullsized":"2024-04-25T08:40:55.596288+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:53.592947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500572+0000","last_change":"2024-04-25T08:40:54.387452+0000","last_active":"2024-04-25T08:40:55.500572+0000","last_peered":"2024-04-25T08:40:55.500572+0000","last_clean":"2024-04-25T08:40:55.500572+0000","last_became_active":"2024-04-25T08:40:54.387198+0000","last_became_peered":"2024-04-25T08:40:54.387198+0000","last_unstale":"2024-04-25T08:40:55.500572+0000","last_undegraded":"2024-04-25T08:40:55.500572+0000","last_fullsized":"2024-04-25T08:40:55.500572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:37.819157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500522+0000","last_change":"2024-04-25T08:40:54.383950+0000","last_active":"2024-04-25T08:40:55.500522+0000","last_peered":"2024-04-25T08:40:55.500522+0000","last_clean":"2024-04-25T08:40:55.500522+0000","last_became_active":"2024-04-25T08:40:54.382608+0000","last_became_peered":"2024-04-25T08:40:54.382608+0000","last_unstale":"2024-04-25T08:40:55.500522+0000","last_undegraded":"2024-04-25T08:40:55.500522+0000","last_fullsized":"2024-04-25T08:40:55.500522+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:56.551387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596504+0000","last_change":"2024-04-25T08:40:54.388616+0000","last_active":"2024-04-25T08:40:55.596504+0000","last_peered":"2024-04-25T08:40:55.596504+0000","last_clean":"2024-04-25T08:40:55.596504+0000","last_became_active":"2024-04-25T08:40:54.388217+0000","last_became_peered":"2024-04-25T08:40:54.388217+0000","last_unstale":"2024-04-25T08:40:55.596504+0000","last_undegraded":"2024-04-25T08:40:55.596504+0000","last_fullsized":"2024-04-25T08:40:55.596504+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:26.474002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383924+0000","last_change":"2024-04-25T08:40:54.384260+0000","last_active":"2024-04-25T08:40:55.383924+0000","last_peered":"2024-04-25T08:40:55.383924+0000","last_clean":"2024-04-25T08:40:55.383924+0000","last_became_active":"2024-04-25T08:40:54.383734+0000","last_became_peered":"2024-04-25T08:40:54.383734+0000","last_unstale":"2024-04-25T08:40:55.383924+0000","last_undegraded":"2024-04-25T08:40:55.383924+0000","last_fullsized":"2024-04-25T08:40:55.383924+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.502699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383888+0000","last_change":"2024-04-25T08:40:54.380206+0000","last_active":"2024-04-25T08:40:55.383888+0000","last_peered":"2024-04-25T08:40:55.383888+0000","last_clean":"2024-04-25T08:40:55.383888+0000","last_became_active":"2024-04-25T08:40:54.379492+0000","last_became_peered":"2024-04-25T08:40:54.379492+0000","last_unstale":"2024-04-25T08:40:55.383888+0000","last_undegraded":"2024-04-25T08:40:55.383888+0000","last_fullsized":"2024-04-25T08:40:55.383888+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:40.486804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381381+0000","last_change":"2024-04-25T08:40:54.386093+0000","last_active":"2024-04-25T08:40:55.381381+0000","last_peered":"2024-04-25T08:40:55.381381+0000","last_clean":"2024-04-25T08:40:55.381381+0000","last_became_active":"2024-04-25T08:40:54.385187+0000","last_became_peered":"2024-04-25T08:40:54.385187+0000","last_unstale":"2024-04-25T08:40:55.381381+0000","last_undegraded":"2024-04-25T08:40:55.381381+0000","last_fullsized":"2024-04-25T08:40:55.381381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.539986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383508+0000","last_change":"2024-04-25T08:40:54.386486+0000","last_active":"2024-04-25T08:40:55.383508+0000","last_peered":"2024-04-25T08:40:55.383508+0000","last_clean":"2024-04-25T08:40:55.383508+0000","last_became_active":"2024-04-25T08:40:54.386209+0000","last_became_peered":"2024-04-25T08:40:54.386209+0000","last_unstale":"2024-04-25T08:40:55.383508+0000","last_undegraded":"2024-04-25T08:40:55.383508+0000","last_fullsized":"2024-04-25T08:40:55.383508+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:37.819157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382698+0000","last_change":"2024-04-25T08:40:54.388200+0000","last_active":"2024-04-25T08:40:55.382698+0000","last_peered":"2024-04-25T08:40:55.382698+0000","last_clean":"2024-04-25T08:40:55.382698+0000","last_became_active":"2024-04-25T08:40:54.387935+0000","last_became_peered":"2024-04-25T08:40:54.387935+0000","last_unstale":"2024-04-25T08:40:55.382698+0000","last_undegraded":"2024-04-25T08:40:55.382698+0000","last_fullsized":"2024-04-25T08:40:55.382698+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:56.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382793+0000","last_change":"2024-04-25T08:40:54.388543+0000","last_active":"2024-04-25T08:40:55.382793+0000","last_peered":"2024-04-25T08:40:55.382793+0000","last_clean":"2024-04-25T08:40:55.382793+0000","last_became_active":"2024-04-25T08:40:54.387411+0000","last_became_peered":"2024-04-25T08:40:54.387411+0000","last_unstale":"2024-04-25T08:40:55.382793+0000","last_undegraded":"2024-04-25T08:40:55.382793+0000","last_fullsized":"2024-04-25T08:40:55.382793+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:56.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.380999+0000","last_change":"2024-04-25T08:40:54.380867+0000","last_active":"2024-04-25T08:40:55.380999+0000","last_peered":"2024-04-25T08:40:55.380999+0000","last_clean":"2024-04-25T08:40:55.380999+0000","last_became_active":"2024-04-25T08:40:54.380595+0000","last_became_peered":"2024-04-25T08:40:54.380595+0000","last_unstale":"2024-04-25T08:40:55.380999+0000","last_undegraded":"2024-04-25T08:40:55.380999+0000","last_fullsized":"2024-04-25T08:40:55.380999+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:58.196801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382612+0000","last_change":"2024-04-25T08:40:54.389841+0000","last_active":"2024-04-25T08:40:55.382612+0000","last_peered":"2024-04-25T08:40:55.382612+0000","last_clean":"2024-04-25T08:40:55.382612+0000","last_became_active":"2024-04-25T08:40:54.389572+0000","last_became_peered":"2024-04-25T08:40:54.389572+0000","last_unstale":"2024-04-25T08:40:55.382612+0000","last_undegraded":"2024-04-25T08:40:55.382612+0000","last_fullsized":"2024-04-25T08:40:55.382612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:04.970606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"131'4","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.384013+0000","last_change":"2024-04-25T08:40:53.379026+0000","last_active":"2024-04-25T08:40:55.384013+0000","last_peered":"2024-04-25T08:40:55.384013+0000","last_clean":"2024-04-25T08:40:55.384013+0000","last_became_active":"2024-04-25T08:40:53.378751+0000","last_became_peered":"2024-04-25T08:40:53.378751+0000","last_unstale":"2024-04-25T08:40:55.384013+0000","last_undegraded":"2024-04-25T08:40:55.384013+0000","last_fullsized":"2024-04-25T08:40:55.384013+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.556400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500758+0000","last_change":"2024-04-25T08:40:54.378840+0000","last_active":"2024-04-25T08:40:55.500758+0000","last_peered":"2024-04-25T08:40:55.500758+0000","last_clean":"2024-04-25T08:40:55.500758+0000","last_became_active":"2024-04-25T08:40:54.378566+0000","last_became_peered":"2024-04-25T08:40:54.378566+0000","last_unstale":"2024-04-25T08:40:55.500758+0000","last_undegraded":"2024-04-25T08:40:55.500758+0000","last_fullsized":"2024-04-25T08:40:55.500758+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:40.486804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382917+0000","last_change":"2024-04-25T08:40:54.387624+0000","last_active":"2024-04-25T08:40:55.382917+0000","last_peered":"2024-04-25T08:40:55.382917+0000","last_clean":"2024-04-25T08:40:55.382917+0000","last_became_active":"2024-04-25T08:40:54.387359+0000","last_became_peered":"2024-04-25T08:40:54.387359+0000","last_unstale":"2024-04-25T08:40:55.382917+0000","last_undegraded":"2024-04-25T08:40:55.382917+0000","last_fullsized":"2024-04-25T08:40:55.382917+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:22.825790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383795+0000","last_change":"2024-04-25T08:40:53.373572+0000","last_active":"2024-04-25T08:40:55.383795+0000","last_peered":"2024-04-25T08:40:55.383795+0000","last_clean":"2024-04-25T08:40:55.383795+0000","last_became_active":"2024-04-25T08:40:53.373290+0000","last_became_peered":"2024-04-25T08:40:53.373290+0000","last_unstale":"2024-04-25T08:40:55.383795+0000","last_undegraded":"2024-04-25T08:40:55.383795+0000","last_fullsized":"2024-04-25T08:40:55.383795+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:40.850065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383312+0000","last_change":"2024-04-25T08:40:54.388330+0000","last_active":"2024-04-25T08:40:55.383312+0000","last_peered":"2024-04-25T08:40:55.383312+0000","last_clean":"2024-04-25T08:40:55.383312+0000","last_became_active":"2024-04-25T08:40:54.387849+0000","last_became_peered":"2024-04-25T08:40:54.387849+0000","last_unstale":"2024-04-25T08:40:55.383312+0000","last_undegraded":"2024-04-25T08:40:55.383312+0000","last_fullsized":"2024-04-25T08:40:55.383312+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:40.899090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595098+0000","last_change":"2024-04-25T08:40:53.376507+0000","last_active":"2024-04-25T08:40:55.595098+0000","last_peered":"2024-04-25T08:40:55.595098+0000","last_clean":"2024-04-25T08:40:55.595098+0000","last_became_active":"2024-04-25T08:40:53.376186+0000","last_became_peered":"2024-04-25T08:40:53.376186+0000","last_unstale":"2024-04-25T08:40:55.595098+0000","last_undegraded":"2024-04-25T08:40:55.595098+0000","last_fullsized":"2024-04-25T08:40:55.595098+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:50.437720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383020+0000","last_change":"2024-04-25T08:40:53.376440+0000","last_active":"2024-04-25T08:40:55.383020+0000","last_peered":"2024-04-25T08:40:55.383020+0000","last_clean":"2024-04-25T08:40:55.383020+0000","last_became_active":"2024-04-25T08:40:53.376169+0000","last_became_peered":"2024-04-25T08:40:53.376169+0000","last_unstale":"2024-04-25T08:40:55.383020+0000","last_undegraded":"2024-04-25T08:40:55.383020+0000","last_fullsized":"2024-04-25T08:40:55.383020+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:52.489674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500789+0000","last_change":"2024-04-25T08:40:53.380621+0000","last_active":"2024-04-25T08:40:55.500789+0000","last_peered":"2024-04-25T08:40:55.500789+0000","last_clean":"2024-04-25T08:40:55.500789+0000","last_became_active":"2024-04-25T08:40:53.380150+0000","last_became_peered":"2024-04-25T08:40:53.380150+0000","last_unstale":"2024-04-25T08:40:55.500789+0000","last_undegraded":"2024-04-25T08:40:55.500789+0000","last_fullsized":"2024-04-25T08:40:55.500789+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:56.303715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595249+0000","last_change":"2024-04-25T08:40:54.387442+0000","last_active":"2024-04-25T08:40:55.595249+0000","last_peered":"2024-04-25T08:40:55.595249+0000","last_clean":"2024-04-25T08:40:55.595249+0000","last_became_active":"2024-04-25T08:40:54.387157+0000","last_became_peered":"2024-04-25T08:40:54.387157+0000","last_unstale":"2024-04-25T08:40:55.595249+0000","last_undegraded":"2024-04-25T08:40:55.595249+0000","last_fullsized":"2024-04-25T08:40:55.595249+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:48.139944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382845+0000","last_change":"2024-04-25T08:40:54.376953+0000","last_active":"2024-04-25T08:40:55.382845+0000","last_peered":"2024-04-25T08:40:55.382845+0000","last_clean":"2024-04-25T08:40:55.382845+0000","last_became_active":"2024-04-25T08:40:54.376686+0000","last_became_peered":"2024-04-25T08:40:54.376686+0000","last_unstale":"2024-04-25T08:40:55.382845+0000","last_undegraded":"2024-04-25T08:40:55.382845+0000","last_fullsized":"2024-04-25T08:40:55.382845+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:22.825790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383572+0000","last_change":"2024-04-25T08:40:54.385830+0000","last_active":"2024-04-25T08:40:55.383572+0000","last_peered":"2024-04-25T08:40:55.383572+0000","last_clean":"2024-04-25T08:40:55.383572+0000","last_became_active":"2024-04-25T08:40:54.384994+0000","last_became_peered":"2024-04-25T08:40:54.384994+0000","last_unstale":"2024-04-25T08:40:55.383572+0000","last_undegraded":"2024-04-25T08:40:55.383572+0000","last_fullsized":"2024-04-25T08:40:55.383572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:51.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381354+0000","last_change":"2024-04-25T08:40:54.383836+0000","last_active":"2024-04-25T08:40:55.381354+0000","last_peered":"2024-04-25T08:40:55.381354+0000","last_clean":"2024-04-25T08:40:55.381354+0000","last_became_active":"2024-04-25T08:40:54.383486+0000","last_became_peered":"2024-04-25T08:40:54.383486+0000","last_unstale":"2024-04-25T08:40:55.381354+0000","last_undegraded":"2024-04-25T08:40:55.381354+0000","last_fullsized":"2024-04-25T08:40:55.381354+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:20.048786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383208+0000","last_change":"2024-04-25T08:40:53.377899+0000","last_active":"2024-04-25T08:40:55.383208+0000","last_peered":"2024-04-25T08:40:55.383208+0000","last_clean":"2024-04-25T08:40:55.383208+0000","last_became_active":"2024-04-25T08:40:53.377626+0000","last_became_peered":"2024-04-25T08:40:53.377626+0000","last_unstale":"2024-04-25T08:40:55.383208+0000","last_undegraded":"2024-04-25T08:40:55.383208+0000","last_fullsized":"2024-04-25T08:40:55.383208+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:08.112732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"131'8","reported_seq":22,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596483+0000","last_change":"2024-04-25T08:40:53.378565+0000","last_active":"2024-04-25T08:40:55.596483+0000","last_peered":"2024-04-25T08:40:55.596483+0000","last_clean":"2024-04-25T08:40:55.596483+0000","last_became_active":"2024-04-25T08:40:53.378251+0000","last_became_peered":"2024-04-25T08:40:53.378251+0000","last_unstale":"2024-04-25T08:40:55.596483+0000","last_undegraded":"2024-04-25T08:40:55.596483+0000","last_fullsized":"2024-04-25T08:40:55.596483+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:03.962369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382898+0000","last_change":"2024-04-25T08:40:54.372817+0000","last_active":"2024-04-25T08:40:55.382898+0000","last_peered":"2024-04-25T08:40:55.382898+0000","last_clean":"2024-04-25T08:40:55.382898+0000","last_became_active":"2024-04-25T08:40:54.371590+0000","last_became_peered":"2024-04-25T08:40:54.371590+0000","last_unstale":"2024-04-25T08:40:55.382898+0000","last_undegraded":"2024-04-25T08:40:55.382898+0000","last_fullsized":"2024-04-25T08:40:55.382898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:40.899090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500851+0000","last_change":"2024-04-25T08:40:53.382667+0000","last_active":"2024-04-25T08:40:55.500851+0000","last_peered":"2024-04-25T08:40:55.500851+0000","last_clean":"2024-04-25T08:40:55.500851+0000","last_became_active":"2024-04-25T08:40:53.382436+0000","last_became_peered":"2024-04-25T08:40:53.382436+0000","last_unstale":"2024-04-25T08:40:55.500851+0000","last_undegraded":"2024-04-25T08:40:55.500851+0000","last_fullsized":"2024-04-25T08:40:55.500851+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:39.954950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383651+0000","last_change":"2024-04-25T08:40:53.384093+0000","last_active":"2024-04-25T08:40:55.383651+0000","last_peered":"2024-04-25T08:40:55.383651+0000","last_clean":"2024-04-25T08:40:55.383651+0000","last_became_active":"2024-04-25T08:40:53.383830+0000","last_became_peered":"2024-04-25T08:40:53.383830+0000","last_unstale":"2024-04-25T08:40:55.383651+0000","last_undegraded":"2024-04-25T08:40:55.383651+0000","last_fullsized":"2024-04-25T08:40:55.383651+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:40.850065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595793+0000","last_change":"2024-04-25T08:40:53.377620+0000","last_active":"2024-04-25T08:40:55.595793+0000","last_peered":"2024-04-25T08:40:55.595793+0000","last_clean":"2024-04-25T08:40:55.595793+0000","last_became_active":"2024-04-25T08:40:53.377340+0000","last_became_peered":"2024-04-25T08:40:53.377340+0000","last_unstale":"2024-04-25T08:40:55.595793+0000","last_undegraded":"2024-04-25T08:40:55.595793+0000","last_fullsized":"2024-04-25T08:40:55.595793+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:33.063592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381144+0000","last_change":"2024-04-25T08:40:54.382916+0000","last_active":"2024-04-25T08:40:55.381144+0000","last_peered":"2024-04-25T08:40:55.381144+0000","last_clean":"2024-04-25T08:40:55.381144+0000","last_became_active":"2024-04-25T08:40:54.382634+0000","last_became_peered":"2024-04-25T08:40:54.382634+0000","last_unstale":"2024-04-25T08:40:55.381144+0000","last_undegraded":"2024-04-25T08:40:55.381144+0000","last_fullsized":"2024-04-25T08:40:55.381144+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:28.055098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383840+0000","last_change":"2024-04-25T08:40:53.372724+0000","last_active":"2024-04-25T08:40:55.383840+0000","last_peered":"2024-04-25T08:40:55.383840+0000","last_clean":"2024-04-25T08:40:55.383840+0000","last_became_active":"2024-04-25T08:40:53.372440+0000","last_became_peered":"2024-04-25T08:40:53.372440+0000","last_unstale":"2024-04-25T08:40:55.383840+0000","last_undegraded":"2024-04-25T08:40:55.383840+0000","last_fullsized":"2024-04-25T08:40:55.383840+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:46.131350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500809+0000","last_change":"2024-04-25T08:40:54.378104+0000","last_active":"2024-04-25T08:40:55.500809+0000","last_peered":"2024-04-25T08:40:55.500809+0000","last_clean":"2024-04-25T08:40:55.500809+0000","last_became_active":"2024-04-25T08:40:54.377753+0000","last_became_peered":"2024-04-25T08:40:54.377753+0000","last_unstale":"2024-04-25T08:40:55.500809+0000","last_undegraded":"2024-04-25T08:40:55.500809+0000","last_fullsized":"2024-04-25T08:40:55.500809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:25.693049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596583+0000","last_change":"2024-04-25T08:40:53.376843+0000","last_active":"2024-04-25T08:40:55.596583+0000","last_peered":"2024-04-25T08:40:55.596583+0000","last_clean":"2024-04-25T08:40:55.596583+0000","last_became_active":"2024-04-25T08:40:53.376238+0000","last_became_peered":"2024-04-25T08:40:53.376238+0000","last_unstale":"2024-04-25T08:40:55.596583+0000","last_undegraded":"2024-04-25T08:40:55.596583+0000","last_fullsized":"2024-04-25T08:40:55.596583+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:55.097704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383108+0000","last_change":"2024-04-25T08:40:54.384630+0000","last_active":"2024-04-25T08:40:55.383108+0000","last_peered":"2024-04-25T08:40:55.383108+0000","last_clean":"2024-04-25T08:40:55.383108+0000","last_became_active":"2024-04-25T08:40:54.384077+0000","last_became_peered":"2024-04-25T08:40:54.384077+0000","last_unstale":"2024-04-25T08:40:55.383108+0000","last_undegraded":"2024-04-25T08:40:55.383108+0000","last_fullsized":"2024-04-25T08:40:55.383108+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:56.551387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"132'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:56.368767+0000","last_change":"2024-04-25T08:40:53.378186+0000","last_active":"2024-04-25T08:40:56.368767+0000","last_peered":"2024-04-25T08:40:56.368767+0000","last_clean":"2024-04-25T08:40:56.368767+0000","last_became_active":"2024-04-25T08:40:53.377880+0000","last_became_peered":"2024-04-25T08:40:53.377880+0000","last_unstale":"2024-04-25T08:40:56.368767+0000","last_undegraded":"2024-04-25T08:40:56.368767+0000","last_fullsized":"2024-04-25T08:40:56.368767+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:47.172417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383430+0000","last_change":"2024-04-25T08:40:53.381261+0000","last_active":"2024-04-25T08:40:55.383430+0000","last_peered":"2024-04-25T08:40:55.383430+0000","last_clean":"2024-04-25T08:40:55.383430+0000","last_became_active":"2024-04-25T08:40:53.380920+0000","last_became_peered":"2024-04-25T08:40:53.380920+0000","last_unstale":"2024-04-25T08:40:55.383430+0000","last_undegraded":"2024-04-25T08:40:55.383430+0000","last_fullsized":"2024-04-25T08:40:55.383430+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:39.954950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381178+0000","last_change":"2024-04-25T08:40:54.386615+0000","last_active":"2024-04-25T08:40:55.381178+0000","last_peered":"2024-04-25T08:40:55.381178+0000","last_clean":"2024-04-25T08:40:55.381178+0000","last_became_active":"2024-04-25T08:40:54.386359+0000","last_became_peered":"2024-04-25T08:40:54.386359+0000","last_unstale":"2024-04-25T08:40:55.381178+0000","last_undegraded":"2024-04-25T08:40:55.381178+0000","last_fullsized":"2024-04-25T08:40:55.381178+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.093642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595669+0000","last_change":"2024-04-25T08:40:53.378763+0000","last_active":"2024-04-25T08:40:55.595669+0000","last_peered":"2024-04-25T08:40:55.595669+0000","last_clean":"2024-04-25T08:40:55.595669+0000","last_became_active":"2024-04-25T08:40:53.378453+0000","last_became_peered":"2024-04-25T08:40:53.378453+0000","last_unstale":"2024-04-25T08:40:55.595669+0000","last_undegraded":"2024-04-25T08:40:55.595669+0000","last_fullsized":"2024-04-25T08:40:55.595669+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:57.408929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595573+0000","last_change":"2024-04-25T08:40:54.387431+0000","last_active":"2024-04-25T08:40:55.595573+0000","last_peered":"2024-04-25T08:40:55.595573+0000","last_clean":"2024-04-25T08:40:55.595573+0000","last_became_active":"2024-04-25T08:40:54.387149+0000","last_became_peered":"2024-04-25T08:40:54.387149+0000","last_unstale":"2024-04-25T08:40:55.595573+0000","last_undegraded":"2024-04-25T08:40:55.595573+0000","last_fullsized":"2024-04-25T08:40:55.595573+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:49.064739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500551+0000","last_change":"2024-04-25T08:40:53.385220+0000","last_active":"2024-04-25T08:40:55.500551+0000","last_peered":"2024-04-25T08:40:55.500551+0000","last_clean":"2024-04-25T08:40:55.500551+0000","last_became_active":"2024-04-25T08:40:53.384943+0000","last_became_peered":"2024-04-25T08:40:53.384943+0000","last_unstale":"2024-04-25T08:40:55.500551+0000","last_undegraded":"2024-04-25T08:40:55.500551+0000","last_fullsized":"2024-04-25T08:40:55.500551+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:16.391337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595687+0000","last_change":"2024-04-25T08:40:54.388970+0000","last_active":"2024-04-25T08:40:55.595687+0000","last_peered":"2024-04-25T08:40:55.595687+0000","last_clean":"2024-04-25T08:40:55.595687+0000","last_became_active":"2024-04-25T08:40:54.388677+0000","last_became_peered":"2024-04-25T08:40:54.388677+0000","last_unstale":"2024-04-25T08:40:55.595687+0000","last_undegraded":"2024-04-25T08:40:55.595687+0000","last_fullsized":"2024-04-25T08:40:55.595687+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:37.825506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595830+0000","last_change":"2024-04-25T08:40:54.381075+0000","last_active":"2024-04-25T08:40:55.595830+0000","last_peered":"2024-04-25T08:40:55.595830+0000","last_clean":"2024-04-25T08:40:55.595830+0000","last_became_active":"2024-04-25T08:40:54.380766+0000","last_became_peered":"2024-04-25T08:40:54.380766+0000","last_unstale":"2024-04-25T08:40:55.595830+0000","last_undegraded":"2024-04-25T08:40:55.595830+0000","last_fullsized":"2024-04-25T08:40:55.595830+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:56.441011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595339+0000","last_change":"2024-04-25T08:40:53.378417+0000","last_active":"2024-04-25T08:40:55.595339+0000","last_peered":"2024-04-25T08:40:55.595339+0000","last_clean":"2024-04-25T08:40:55.595339+0000","last_became_active":"2024-04-25T08:40:53.378139+0000","last_became_peered":"2024-04-25T08:40:53.378139+0000","last_unstale":"2024-04-25T08:40:55.595339+0000","last_undegraded":"2024-04-25T08:40:55.595339+0000","last_fullsized":"2024-04-25T08:40:55.595339+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.250586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383951+0000","last_change":"2024-04-25T08:40:54.378981+0000","last_active":"2024-04-25T08:40:55.383951+0000","last_peered":"2024-04-25T08:40:55.383951+0000","last_clean":"2024-04-25T08:40:55.383951+0000","last_became_active":"2024-04-25T08:40:54.378496+0000","last_became_peered":"2024-04-25T08:40:54.378496+0000","last_unstale":"2024-04-25T08:40:55.383951+0000","last_undegraded":"2024-04-25T08:40:55.383951+0000","last_fullsized":"2024-04-25T08:40:55.383951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:25.693049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":372,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T08:40:50.676773+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:40:50.676773+0000","last_peered":"2024-04-25T08:40:50.676773+0000","last_clean":"2024-04-25T08:40:50.676773+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:40:50.676773+0000","last_undegraded":"2024-04-25T08:40:50.676773+0000","last_fullsized":"2024-04-25T08:40:50.676773+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":76,"acting":76,"num_store_stats":5},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":674,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":34,"acting":34,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65264,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306576,"statfs":{"total":96636764160,"available":96569933824,"internally_reserved":0,"allocated":2195456,"data_stored":1334949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59172,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312668,"statfs":{"total":96636764160,"available":96576172032,"internally_reserved":0,"allocated":1658880,"data_stored":771980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55012,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316828,"statfs":{"total":96636764160,"available":96580431872,"internally_reserved":0,"allocated":1654784,"data_stored":771330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68892,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302948,"statfs":{"total":96636764160,"available":96566218752,"internally_reserved":0,"allocated":1650688,"data_stored":771308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73004,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298836,"statfs":{"total":96636764160,"available":96562008064,"internally_reserved":0,"allocated":1601536,"data_stored":744581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59116,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312724,"statfs":{"total":96636764160,"available":96576229376,"internally_reserved":0,"allocated":1601536,"data_stored":744581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":1654784,"data_stored":771378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69416,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302424,"statfs":{"total":96636764160,"available":96565682176,"internally_reserved":0,"allocated":2252800,"data_stored":1361732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:40:58.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3f is not active+clean 2024-04-25T08:40:58.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3b is not active+clean 2024-04-25T08:40:58.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.39 is not active+clean 2024-04-25T08:40:58.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.34 is not active+clean 2024-04-25T08:40:58.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.32 is not active+clean 2024-04-25T08:40:58.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.31 is not active+clean 2024-04-25T08:40:58.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2e is not active+clean 2024-04-25T08:40:58.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2d is not active+clean 2024-04-25T08:40:58.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2c is not active+clean 2024-04-25T08:40:58.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2a is not active+clean 2024-04-25T08:40:58.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.25 is not active+clean 2024-04-25T08:40:58.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.19 is not active+clean 2024-04-25T08:40:58.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1e is not active+clean 2024-04-25T08:40:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1d is not active+clean 2024-04-25T08:40:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1b is not active+clean 2024-04-25T08:40:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1c is not active+clean 2024-04-25T08:40:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1a is not active+clean 2024-04-25T08:40:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1e is not active+clean 2024-04-25T08:40:58.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1f is not active+clean 2024-04-25T08:40:58.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.10 is not active+clean 2024-04-25T08:40:58.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.16 is not active+clean 2024-04-25T08:40:58.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.15 is not active+clean 2024-04-25T08:40:58.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.14 is not active+clean 2024-04-25T08:40:58.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.13 is not active+clean 2024-04-25T08:40:58.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.16 is not active+clean 2024-04-25T08:40:58.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.17 is not active+clean 2024-04-25T08:40:58.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.9 is not active+clean 2024-04-25T08:40:58.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.a is not active+clean 2024-04-25T08:40:58.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.c is not active+clean 2024-04-25T08:40:58.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.c is not active+clean 2024-04-25T08:40:58.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.d is not active+clean 2024-04-25T08:40:58.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.a is not active+clean 2024-04-25T08:40:58.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.e is not active+clean 2024-04-25T08:40:58.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.9 is not active+clean 2024-04-25T08:40:58.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.0 is not active+clean 2024-04-25T08:40:58.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.7 is not active+clean 2024-04-25T08:40:58.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2 is not active+clean 2024-04-25T08:40:58.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2 is not active+clean 2024-04-25T08:40:58.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.7 is not active+clean 2024-04-25T08:40:58.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:53.353507+0000', 'last_change': '2024-04-25T08:40:53.353507+0000', 'last_active': '2024-04-25T08:40:53.353507+0000', 'last_peered': '2024-04-25T08:40:53.353507+0000', 'last_clean': '2024-04-25T08:40:53.353507+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:53.353507+0000', 'last_undegraded': '2024-04-25T08:40:53.353507+0000', 'last_fullsized': '2024-04-25T08:40:53.353507+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:53.353507+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.0 is not active+clean 2024-04-25T08:40:58.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.12 is not active+clean 2024-04-25T08:40:58.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:40:52.345270+0000', 'last_change': '2024-04-25T08:40:52.345270+0000', 'last_active': '2024-04-25T08:40:52.345270+0000', 'last_peered': '2024-04-25T08:40:52.345270+0000', 'last_clean': '2024-04-25T08:40:52.345270+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:40:52.345270+0000', 'last_undegraded': '2024-04-25T08:40:52.345270+0000', 'last_fullsized': '2024-04-25T08:40:52.345270+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'last_clean_scrub_stamp': '2024-04-25T08:40:52.345270+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:40:58.089 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:40:58.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.226+0000 7f3484f34640 1 Processor -- start 2024-04-25T08:40:58.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.226+0000 7f3484f34640 1 -- start start 2024-04-25T08:40:58.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f3484f34640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34800b5710 0x7f3480161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f3484f34640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3480161e00 con 0x7f34800b93f0 2024-04-25T08:40:58.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f3484f34640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34801620e0 con 0x7f348015ddf0 2024-04-25T08:40:58.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f3484f34640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34801623c0 con 0x7f34800b5710 2024-04-25T08:40:58.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347ed9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f34800b93f0 0x7f34800d2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51482/0 (socket says 172.21.15.38:51482) 2024-04-25T08:40:58.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347ed9d640 1 -- 172.21.15.38:0/583783214 learned_addr learned my addr 172.21.15.38:0/583783214 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:58.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347e59c640 1 --2- 172.21.15.38:0/583783214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34800b5710 0x7f3480161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005845632 0 0) 0x7f3480161e00 con 0x7f34800b93f0 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f346c003520 con 0x7f34800b93f0 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1007157233 0 0) 0x7f346c003520 con 0x7f34800b93f0 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34800b5710 msgr2=0x7f3480161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 --2- 172.21.15.38:0/583783214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34800b5710 0x7f3480161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 >> v1:172.21.15.107:6789/0 conn(0x7f348015ddf0 legacy=0x7f34800b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800b9870 con 0x7f34800b93f0 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f34780019f0 con 0x7f34800b93f0 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 >> v1:172.21.15.38:6789/0 conn(0x7f34800b93f0 legacy=0x7f34800d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 --2- 172.21.15.38:0/583783214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f346c0039e0 0x7f346c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 --2- 172.21.15.38:0/583783214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34800b5710 0x7f346c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 --2- 172.21.15.38:0/583783214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f346c004d40 0x7f346c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34801623c0 con 0x7f346c0039e0 2024-04-25T08:40:58.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c00ba70 con 0x7f346c004d40 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.227+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346c00bc70 con 0x7f34800b5710 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.228+0000 7f347e59c640 1 --2- 172.21.15.38:0/583783214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f346c0039e0 0x7f346c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.228+0000 7f3477fff640 1 --2- 172.21.15.38:0/583783214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34800b5710 0x7f346c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.228+0000 7f347ed9d640 1 --2- 172.21.15.38:0/583783214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f346c004d40 0x7f346c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.228+0000 7f347e59c640 1 -- 172.21.15.38:0/583783214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34800b5710 msgr2=0x7f346c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.228+0000 7f347e59c640 1 --2- 172.21.15.38:0/583783214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f34800b5710 0x7f346c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.228+0000 7f347e59c640 1 -- 172.21.15.38:0/583783214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f346c004d40 msgr2=0x7f346c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.228+0000 7f347e59c640 1 --2- 172.21.15.38:0/583783214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f346c004d40 0x7f346c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.228+0000 7f347e59c640 1 -- 172.21.15.38:0/583783214 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c003520 con 0x7f346c0039e0 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.228+0000 7f347e59c640 1 --2- 172.21.15.38:0/583783214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f346c0039e0 0x7f346c003e00 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f3470009a90 tx=0x7f346c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3936f1752771aa74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.229+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470036070 con 0x7f346c0039e0 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.229+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3470031400 con 0x7f346c0039e0 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.229+0000 7f347dd9b640 1 -- 172.21.15.38:0/583783214 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470009690 con 0x7f346c0039e0 2024-04-25T08:40:58.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.229+0000 7f3484f34640 1 -- 172.21.15.38:0/583783214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f346c0039e0 msgr2=0x7f346c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.229+0000 7f3484f34640 1 --2- 172.21.15.38:0/583783214 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f346c0039e0 0x7f346c003e00 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f3470009a90 tx=0x7f346c005830 comp rx=0 tx=0).stop 2024-04-25T08:40:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.229+0000 7f347ed9d640 1 -- 172.21.15.38:0/583783214 reap_dead start 2024-04-25T08:40:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.229+0000 7f3484f34640 1 -- 172.21.15.38:0/583783214 shutdown_connections 2024-04-25T08:40:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.229+0000 7f3484f34640 1 -- 172.21.15.38:0/583783214 >> 172.21.15.38:0/583783214 conn(0x7f34800d07c0 msgr2=0x7f34800d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.229+0000 7f3484f34640 1 -- 172.21.15.38:0/583783214 shutdown_connections 2024-04-25T08:40:58.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.229+0000 7f3484f34640 1 -- 172.21.15.38:0/583783214 wait complete. 2024-04-25T08:40:58.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.231+0000 7f3484f34640 1 Processor -- start 2024-04-25T08:40:58.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.231+0000 7f3484f34640 1 -- start start 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.231+0000 7f3484f34640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f346c0039e0 0x7f3480160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.231+0000 7f3484f34640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f348015e200 0x7f348014cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.231+0000 7f3484f34640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34800cca60 0x7f34800ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.231+0000 7f3484f34640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3480161b50 con 0x7f346c0039e0 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.231+0000 7f3484f34640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34801619d0 con 0x7f34800cca60 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.231+0000 7f3484f34640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34800d2a50 con 0x7f348015e200 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.231+0000 7f347e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f346c0039e0 0x7f3480160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.231+0000 7f347e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f346c0039e0 0x7f3480160c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49810/0 (socket says 172.21.15.38:49810) 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.231+0000 7f347e59c640 1 -- 172.21.15.38:0/1087788527 learned_addr learned my addr 172.21.15.38:0/1087788527 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f347ed9d640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34800cca60 0x7f34800ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f3477fff640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f348015e200 0x7f348014cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f347e59c640 1 -- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f348015e200 msgr2=0x7f348014cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f347e59c640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f348015e200 0x7f348014cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f347e59c640 1 -- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34800cca60 msgr2=0x7f34800ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f347e59c640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34800cca60 0x7f34800ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f347e59c640 1 -- 172.21.15.38:0/1087788527 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348015e9c0 con 0x7f346c0039e0 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f347ed9d640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34800cca60 0x7f34800ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f3477fff640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f348015e200 0x7f348014cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f347e59c640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f346c0039e0 0x7f3480160c30 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f3470009a90 tx=0x7f3470004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f3476ffd640 1 -- 172.21.15.38:0/1087788527 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470040020 con 0x7f346c0039e0 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f3476ffd640 1 -- 172.21.15.38:0/1087788527 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3470003bc0 con 0x7f346c0039e0 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f3484f34640 1 -- 172.21.15.38:0/1087788527 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348015eca0 con 0x7f346c0039e0 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f3476ffd640 1 -- 172.21.15.38:0/1087788527 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470036040 con 0x7f346c0039e0 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.232+0000 7f3484f34640 1 -- 172.21.15.38:0/1087788527 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348015f260 con 0x7f346c0039e0 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.233+0000 7f3484f34640 1 -- 172.21.15.38:0/1087788527 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3480073c70 con 0x7f346c0039e0 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.234+0000 7f3476ffd640 1 -- 172.21.15.38:0/1087788527 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3470043020 con 0x7f346c0039e0 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.234+0000 7f3476ffd640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f346006b500 0x7f346006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.234+0000 7f3476ffd640 1 -- 172.21.15.38:0/1087788527 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f34700aa3d0 con 0x7f346c0039e0 2024-04-25T08:40:58.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.234+0000 7f3477fff640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f346006b500 0x7f346006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.235+0000 7f3477fff640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f346006b500 0x7f346006d9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f348015d520 tx=0x7f3468009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:58.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.239+0000 7f3476ffd640 1 -- 172.21.15.38:0/1087788527 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3480073c70 con 0x7f346c0039e0 2024-04-25T08:40:58.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.489+0000 7f3484f34640 1 -- 172.21.15.38:0/1087788527 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3480068e80 con 0x7f346c0039e0 2024-04-25T08:40:58.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.490+0000 7f3476ffd640 1 -- 172.21.15.38:0/1087788527 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1638 (secure 0 0 0) 0x7f347004a070 con 0x7f346c0039e0 2024-04-25T08:40:58.483 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:58.483 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":587,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":56},{"state_name":"unknown","count":41}],"num_pgs":97,"num_pools":3,"num_objects":13,"data_bytes":591042,"bytes_used":521261056,"bytes_avail":772572852224,"bytes_total":773094113280,"unknown_pgs_ratio":0.42268040776252747,"write_bytes_sec":616,"read_op_per_sec":0,"write_op_per_sec":4},"fsmap":{"epoch":77,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9489}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T08:40:35.331262+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:40:58.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.491+0000 7f3484f34640 1 -- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f346006b500 msgr2=0x7f346006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.491+0000 7f3484f34640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f346006b500 0x7f346006d9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f348015d520 tx=0x7f3468009040 comp rx=0 tx=0).stop 2024-04-25T08:40:58.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.491+0000 7f3484f34640 1 -- 172.21.15.38:0/1087788527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f346c0039e0 msgr2=0x7f3480160c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.491+0000 7f3484f34640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f346c0039e0 0x7f3480160c30 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f3470009a90 tx=0x7f3470004730 comp rx=0 tx=0).stop 2024-04-25T08:40:58.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.492+0000 7f3484f34640 1 -- 172.21.15.38:0/1087788527 shutdown_connections 2024-04-25T08:40:58.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.492+0000 7f3484f34640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f34800cca60 0x7f34800ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.492+0000 7f3484f34640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f346006b500 0x7f346006d9c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.492+0000 7f3484f34640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f348015e200 0x7f348014cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.492+0000 7f3484f34640 1 --2- 172.21.15.38:0/1087788527 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f346c0039e0 0x7f3480160c30 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.492+0000 7f3484f34640 1 -- 172.21.15.38:0/1087788527 >> 172.21.15.38:0/1087788527 conn(0x7f34800d07c0 msgr2=0x7f34800ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:58.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.492+0000 7f3484f34640 1 -- 172.21.15.38:0/1087788527 shutdown_connections 2024-04-25T08:40:58.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.492+0000 7f3484f34640 1 -- 172.21.15.38:0/1087788527 wait complete. 2024-04-25T08:40:58.496 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:40:58.496 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:40:58.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.604+0000 7fc86f2ee640 1 Processor -- start 2024-04-25T08:40:58.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.604+0000 7fc86f2ee640 1 -- start start 2024-04-25T08:40:58.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.605+0000 7fc86f2ee640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc8680d3900 0x7fc8680d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.605+0000 7fc86f2ee640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8680d3e70 con 0x7fc86815dc10 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.605+0000 7fc86f2ee640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8680d4150 con 0x7fc8680d0d90 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.605+0000 7fc86f2ee640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86815dff0 con 0x7fc8680d3900 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.605+0000 7fc86d08a640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc86815dc10 0x7fc8680ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:51494/0 (socket says 172.21.15.38:51494) 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.605+0000 7fc86d08a640 1 -- 172.21.15.38:0/890094122 learned_addr learned my addr 172.21.15.38:0/890094122 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.605+0000 7fc86c889640 1 --2- 172.21.15.38:0/890094122 >> v2:172.21.15.107:3300/0 conn(0x7fc8680d3900 0x7fc8680d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754690474 0 0) 0x7fc8680d3e70 con 0x7fc86815dc10 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc850003520 con 0x7fc86815dc10 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836665727 0 0) 0x7fc8680d4150 con 0x7fc8680d0d90 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8680d3e70 con 0x7fc8680d0d90 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2133858543 0 0) 0x7fc850003520 con 0x7fc86815dc10 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 >> v2:172.21.15.107:3300/0 conn(0x7fc8680d3900 msgr2=0x7fc8680d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:58.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 --2- 172.21.15.38:0/890094122 >> v2:172.21.15.107:3300/0 conn(0x7fc8680d3900 0x7fc8680d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 >> v1:172.21.15.107:6789/0 conn(0x7fc8680d0d90 legacy=0x7fc8680b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680d4430 con 0x7fc86815dc10 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fc860001670 con 0x7fc86815dc10 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 >> v1:172.21.15.38:6789/0 conn(0x7fc86815dc10 legacy=0x7fc8680ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 --2- 172.21.15.38:0/890094122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc850003b40 0x7fc850003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 --2- 172.21.15.38:0/890094122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc850004800 0x7fc850004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 --2- 172.21.15.38:0/890094122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 0x7fc85000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86815dff0 con 0x7fc8500051c0 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85000c0f0 con 0x7fc850004800 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85000c2f0 con 0x7fc850003b40 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc86d88b640 1 --2- 172.21.15.38:0/890094122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 0x7fc85000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc86c889640 1 --2- 172.21.15.38:0/890094122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc850004800 0x7fc850004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc86d08a640 1 --2- 172.21.15.38:0/890094122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc850003b40 0x7fc850003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc86d88b640 1 -- 172.21.15.38:0/890094122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc850003b40 msgr2=0x7fc850003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:58.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc86d88b640 1 --2- 172.21.15.38:0/890094122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc850003b40 0x7fc850003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc86d88b640 1 -- 172.21.15.38:0/890094122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc850004800 msgr2=0x7fc850004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:40:58.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc86d88b640 1 --2- 172.21.15.38:0/890094122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc850004800 0x7fc850004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc86d88b640 1 -- 172.21.15.38:0/890094122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850003520 con 0x7fc8500051c0 2024-04-25T08:40:58.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.606+0000 7fc86d08a640 1 --2- 172.21.15.38:0/890094122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc850003b40 0x7fc850003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:58.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.607+0000 7fc86d88b640 1 -- 172.21.15.38:0/890094122 reap_dead start 2024-04-25T08:40:58.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.607+0000 7fc86d88b640 1 --2- 172.21.15.38:0/890094122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 0x7fc85000b9c0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fc858001fd0 tx=0x7fc850005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ee40e1ea15d75c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:58.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.607+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858001bf0 con 0x7fc8500051c0 2024-04-25T08:40:58.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.607+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc858003b70 con 0x7fc8500051c0 2024-04-25T08:40:58.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.607+0000 7fc867fff640 1 -- 172.21.15.38:0/890094122 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8580050d0 con 0x7fc8500051c0 2024-04-25T08:40:58.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.608+0000 7fc86f2ee640 1 -- 172.21.15.38:0/890094122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 msgr2=0x7fc85000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.608+0000 7fc86f2ee640 1 --2- 172.21.15.38:0/890094122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 0x7fc85000b9c0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fc858001fd0 tx=0x7fc850005830 comp rx=0 tx=0).stop 2024-04-25T08:40:58.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.608+0000 7fc86f2ee640 1 -- 172.21.15.38:0/890094122 shutdown_connections 2024-04-25T08:40:58.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.608+0000 7fc86f2ee640 1 --2- 172.21.15.38:0/890094122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 0x7fc85000b9c0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.608+0000 7fc86f2ee640 1 -- 172.21.15.38:0/890094122 >> 172.21.15.38:0/890094122 conn(0x7fc868066d10 msgr2=0x7fc868067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:58.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.608+0000 7fc86f2ee640 1 -- 172.21.15.38:0/890094122 shutdown_connections 2024-04-25T08:40:58.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.608+0000 7fc86f2ee640 1 -- 172.21.15.38:0/890094122 wait complete. 2024-04-25T08:40:58.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.609+0000 7fc86f2ee640 1 Processor -- start 2024-04-25T08:40:58.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.609+0000 7fc86f2ee640 1 -- start start 2024-04-25T08:40:58.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86f2ee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 0x7fc8680aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86f2ee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8680cd360 0x7fc86815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86f2ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8680abec0 0x7fc8680ac370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86f2ee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680d11e0 con 0x7fc8500051c0 2024-04-25T08:40:58.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86f2ee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680d4710 con 0x7fc8680abec0 2024-04-25T08:40:58.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86f2ee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680d0850 con 0x7fc8680cd360 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86d08a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 0x7fc8680aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86d08a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 0x7fc8680aacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49836/0 (socket says 172.21.15.38:49836) 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86d08a640 1 -- 172.21.15.38:0/604864531 learned_addr learned my addr 172.21.15.38:0/604864531 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86c889640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8680cd360 0x7fc86815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86d88b640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8680abec0 0x7fc8680ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86d08a640 1 -- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8680cd360 msgr2=0x7fc86815bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86d08a640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8680cd360 0x7fc86815bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86d08a640 1 -- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8680abec0 msgr2=0x7fc8680ac370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86d08a640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8680abec0 0x7fc8680ac370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86d08a640 1 -- 172.21.15.38:0/604864531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680ab1f0 con 0x7fc8500051c0 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86c889640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8680cd360 0x7fc86815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86d88b640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8680abec0 0x7fc8680ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:40:58.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.610+0000 7fc86d08a640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 0x7fc8680aacb0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fc86000e780 tx=0x7fc86000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:58.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.611+0000 7fc865ffb640 1 -- 172.21.15.38:0/604864531 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86000c660 con 0x7fc8500051c0 2024-04-25T08:40:58.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.611+0000 7fc865ffb640 1 -- 172.21.15.38:0/604864531 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc860018070 con 0x7fc8500051c0 2024-04-25T08:40:58.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.611+0000 7fc86f2ee640 1 -- 172.21.15.38:0/604864531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86815c460 con 0x7fc8500051c0 2024-04-25T08:40:58.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.611+0000 7fc865ffb640 1 -- 172.21.15.38:0/604864531 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86000ca00 con 0x7fc8500051c0 2024-04-25T08:40:58.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.611+0000 7fc86f2ee640 1 -- 172.21.15.38:0/604864531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8680ab7a0 con 0x7fc8500051c0 2024-04-25T08:40:58.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.612+0000 7fc86f2ee640 1 -- 172.21.15.38:0/604864531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc868068e80 con 0x7fc8500051c0 2024-04-25T08:40:58.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.612+0000 7fc865ffb640 1 -- 172.21.15.38:0/604864531 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc86000cba0 con 0x7fc8500051c0 2024-04-25T08:40:58.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.613+0000 7fc865ffb640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc85406b4d0 0x7fc85406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:40:58.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.613+0000 7fc865ffb640 1 -- 172.21.15.38:0/604864531 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc86008a060 con 0x7fc8500051c0 2024-04-25T08:40:58.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.613+0000 7fc86c889640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc85406b4d0 0x7fc85406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:40:58.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.614+0000 7fc86c889640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc85406b4d0 0x7fc85406d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc8681604a0 tx=0x7fc85c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:40:58.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.617+0000 7fc865ffb640 1 -- 172.21.15.38:0/604864531 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc860058de0 con 0x7fc8500051c0 2024-04-25T08:40:58.786 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.793+0000 7fc86f2ee640 1 -- 172.21.15.38:0/604864531 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8680d35d0 con 0x7fc85406b4d0 2024-04-25T08:40:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.799+0000 7fc865ffb640 1 -- 172.21.15.38:0/604864531 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246178 (secure 0 0 0) 0x7fc8680d35d0 con 0x7fc85406b4d0 2024-04-25T08:40:58.796 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:40:58.797 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:40:58.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.802+0000 7fc86f2ee640 1 -- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc85406b4d0 msgr2=0x7fc85406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.802+0000 7fc86f2ee640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc85406b4d0 0x7fc85406d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc8681604a0 tx=0x7fc85c008040 comp rx=0 tx=0).stop 2024-04-25T08:40:58.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.802+0000 7fc86f2ee640 1 -- 172.21.15.38:0/604864531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 msgr2=0x7fc8680aacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:40:58.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.802+0000 7fc86f2ee640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 0x7fc8680aacb0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fc86000e780 tx=0x7fc86000ec40 comp rx=0 tx=0).stop 2024-04-25T08:40:58.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.802+0000 7fc86f2ee640 1 -- 172.21.15.38:0/604864531 shutdown_connections 2024-04-25T08:40:58.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.802+0000 7fc86f2ee640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8680abec0 0x7fc8680ac370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.802+0000 7fc86f2ee640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc85406b4d0 0x7fc85406d990 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.802+0000 7fc86f2ee640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8680cd360 0x7fc86815bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.802+0000 7fc86f2ee640 1 --2- 172.21.15.38:0/604864531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8500051c0 0x7fc8680aacb0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:40:58.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.802+0000 7fc86f2ee640 1 -- 172.21.15.38:0/604864531 >> 172.21.15.38:0/604864531 conn(0x7fc868066d10 msgr2=0x7fc8680b8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:40:58.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.802+0000 7fc86f2ee640 1 -- 172.21.15.38:0/604864531 shutdown_connections 2024-04-25T08:40:58.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:40:58.802+0000 7fc86f2ee640 1 -- 172.21.15.38:0/604864531 wait complete. 2024-04-25T08:40:58.806 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-04-25T08:40:57.335965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591042,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":182,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":509044,"kb_used_data":13936,"kb_used_omap":12,"kb_used_meta":494963,"kb_avail":754465676,"statfs":{"total":773094113280,"available":772572852224,"internally_reserved":0,"allocated":14270464,"data_stored":7271839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":506842672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":674,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.981544"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500770+0000","last_change":"2024-04-25T08:40:54.385506+0000","last_active":"2024-04-25T08:40:55.500770+0000","last_peered":"2024-04-25T08:40:55.500770+0000","last_clean":"2024-04-25T08:40:55.500770+0000","last_became_active":"2024-04-25T08:40:54.385108+0000","last_became_peered":"2024-04-25T08:40:54.385108+0000","last_unstale":"2024-04-25T08:40:55.500770+0000","last_undegraded":"2024-04-25T08:40:55.500770+0000","last_fullsized":"2024-04-25T08:40:55.500770+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.502699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383789+0000","last_change":"2024-04-25T08:40:54.378766+0000","last_active":"2024-04-25T08:40:55.383789+0000","last_peered":"2024-04-25T08:40:55.383789+0000","last_clean":"2024-04-25T08:40:55.383789+0000","last_became_active":"2024-04-25T08:40:54.378120+0000","last_became_peered":"2024-04-25T08:40:54.378120+0000","last_unstale":"2024-04-25T08:40:55.383789+0000","last_undegraded":"2024-04-25T08:40:55.383789+0000","last_fullsized":"2024-04-25T08:40:55.383789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:49.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500706+0000","last_change":"2024-04-25T08:40:54.382334+0000","last_active":"2024-04-25T08:40:55.500706+0000","last_peered":"2024-04-25T08:40:55.500706+0000","last_clean":"2024-04-25T08:40:55.500706+0000","last_became_active":"2024-04-25T08:40:54.382070+0000","last_became_peered":"2024-04-25T08:40:54.382070+0000","last_unstale":"2024-04-25T08:40:55.500706+0000","last_undegraded":"2024-04-25T08:40:55.500706+0000","last_fullsized":"2024-04-25T08:40:55.500706+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:49.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595381+0000","last_change":"2024-04-25T08:40:54.384503+0000","last_active":"2024-04-25T08:40:55.595381+0000","last_peered":"2024-04-25T08:40:55.595381+0000","last_clean":"2024-04-25T08:40:55.595381+0000","last_became_active":"2024-04-25T08:40:54.384141+0000","last_became_peered":"2024-04-25T08:40:54.384141+0000","last_unstale":"2024-04-25T08:40:55.595381+0000","last_undegraded":"2024-04-25T08:40:55.595381+0000","last_fullsized":"2024-04-25T08:40:55.595381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:48.148660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382449+0000","last_change":"2024-04-25T08:40:54.383613+0000","last_active":"2024-04-25T08:40:55.382449+0000","last_peered":"2024-04-25T08:40:55.382449+0000","last_clean":"2024-04-25T08:40:55.382449+0000","last_became_active":"2024-04-25T08:40:54.383329+0000","last_became_peered":"2024-04-25T08:40:54.383329+0000","last_unstale":"2024-04-25T08:40:55.382449+0000","last_undegraded":"2024-04-25T08:40:55.382449+0000","last_fullsized":"2024-04-25T08:40:55.382449+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:58.417166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596145+0000","last_change":"2024-04-25T08:40:54.390511+0000","last_active":"2024-04-25T08:40:55.596145+0000","last_peered":"2024-04-25T08:40:55.596145+0000","last_clean":"2024-04-25T08:40:55.596145+0000","last_became_active":"2024-04-25T08:40:54.390216+0000","last_became_peered":"2024-04-25T08:40:54.390216+0000","last_unstale":"2024-04-25T08:40:55.596145+0000","last_undegraded":"2024-04-25T08:40:55.596145+0000","last_fullsized":"2024-04-25T08:40:55.596145+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:29.262872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381277+0000","last_change":"2024-04-25T08:40:54.386039+0000","last_active":"2024-04-25T08:40:55.381277+0000","last_peered":"2024-04-25T08:40:55.381277+0000","last_clean":"2024-04-25T08:40:55.381277+0000","last_became_active":"2024-04-25T08:40:54.385759+0000","last_became_peered":"2024-04-25T08:40:54.385759+0000","last_unstale":"2024-04-25T08:40:55.381277+0000","last_undegraded":"2024-04-25T08:40:55.381277+0000","last_fullsized":"2024-04-25T08:40:55.381277+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:59.173213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500612+0000","last_change":"2024-04-25T08:40:54.375254+0000","last_active":"2024-04-25T08:40:55.500612+0000","last_peered":"2024-04-25T08:40:55.500612+0000","last_clean":"2024-04-25T08:40:55.500612+0000","last_became_active":"2024-04-25T08:40:54.374962+0000","last_became_peered":"2024-04-25T08:40:54.374962+0000","last_unstale":"2024-04-25T08:40:55.500612+0000","last_undegraded":"2024-04-25T08:40:55.500612+0000","last_fullsized":"2024-04-25T08:40:55.500612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:51.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596611+0000","last_change":"2024-04-25T08:40:54.389554+0000","last_active":"2024-04-25T08:40:55.596611+0000","last_peered":"2024-04-25T08:40:55.596611+0000","last_clean":"2024-04-25T08:40:55.596611+0000","last_became_active":"2024-04-25T08:40:54.386658+0000","last_became_peered":"2024-04-25T08:40:54.386658+0000","last_unstale":"2024-04-25T08:40:55.596611+0000","last_undegraded":"2024-04-25T08:40:55.596611+0000","last_fullsized":"2024-04-25T08:40:55.596611+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596384+0000","last_change":"2024-04-25T08:40:54.380643+0000","last_active":"2024-04-25T08:40:55.596384+0000","last_peered":"2024-04-25T08:40:55.596384+0000","last_clean":"2024-04-25T08:40:55.596384+0000","last_became_active":"2024-04-25T08:40:54.379854+0000","last_became_peered":"2024-04-25T08:40:54.379854+0000","last_unstale":"2024-04-25T08:40:55.596384+0000","last_undegraded":"2024-04-25T08:40:55.596384+0000","last_fullsized":"2024-04-25T08:40:55.596384+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596288+0000","last_change":"2024-04-25T08:40:54.380286+0000","last_active":"2024-04-25T08:40:55.596288+0000","last_peered":"2024-04-25T08:40:55.596288+0000","last_clean":"2024-04-25T08:40:55.596288+0000","last_became_active":"2024-04-25T08:40:54.380011+0000","last_became_peered":"2024-04-25T08:40:54.380011+0000","last_unstale":"2024-04-25T08:40:55.596288+0000","last_undegraded":"2024-04-25T08:40:55.596288+0000","last_fullsized":"2024-04-25T08:40:55.596288+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:53.592947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500572+0000","last_change":"2024-04-25T08:40:54.387452+0000","last_active":"2024-04-25T08:40:55.500572+0000","last_peered":"2024-04-25T08:40:55.500572+0000","last_clean":"2024-04-25T08:40:55.500572+0000","last_became_active":"2024-04-25T08:40:54.387198+0000","last_became_peered":"2024-04-25T08:40:54.387198+0000","last_unstale":"2024-04-25T08:40:55.500572+0000","last_undegraded":"2024-04-25T08:40:55.500572+0000","last_fullsized":"2024-04-25T08:40:55.500572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:37.819157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500522+0000","last_change":"2024-04-25T08:40:54.383950+0000","last_active":"2024-04-25T08:40:55.500522+0000","last_peered":"2024-04-25T08:40:55.500522+0000","last_clean":"2024-04-25T08:40:55.500522+0000","last_became_active":"2024-04-25T08:40:54.382608+0000","last_became_peered":"2024-04-25T08:40:54.382608+0000","last_unstale":"2024-04-25T08:40:55.500522+0000","last_undegraded":"2024-04-25T08:40:55.500522+0000","last_fullsized":"2024-04-25T08:40:55.500522+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:56.551387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596504+0000","last_change":"2024-04-25T08:40:54.388616+0000","last_active":"2024-04-25T08:40:55.596504+0000","last_peered":"2024-04-25T08:40:55.596504+0000","last_clean":"2024-04-25T08:40:55.596504+0000","last_became_active":"2024-04-25T08:40:54.388217+0000","last_became_peered":"2024-04-25T08:40:54.388217+0000","last_unstale":"2024-04-25T08:40:55.596504+0000","last_undegraded":"2024-04-25T08:40:55.596504+0000","last_fullsized":"2024-04-25T08:40:55.596504+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:26.474002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383924+0000","last_change":"2024-04-25T08:40:54.384260+0000","last_active":"2024-04-25T08:40:55.383924+0000","last_peered":"2024-04-25T08:40:55.383924+0000","last_clean":"2024-04-25T08:40:55.383924+0000","last_became_active":"2024-04-25T08:40:54.383734+0000","last_became_peered":"2024-04-25T08:40:54.383734+0000","last_unstale":"2024-04-25T08:40:55.383924+0000","last_undegraded":"2024-04-25T08:40:55.383924+0000","last_fullsized":"2024-04-25T08:40:55.383924+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.502699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383888+0000","last_change":"2024-04-25T08:40:54.380206+0000","last_active":"2024-04-25T08:40:55.383888+0000","last_peered":"2024-04-25T08:40:55.383888+0000","last_clean":"2024-04-25T08:40:55.383888+0000","last_became_active":"2024-04-25T08:40:54.379492+0000","last_became_peered":"2024-04-25T08:40:54.379492+0000","last_unstale":"2024-04-25T08:40:55.383888+0000","last_undegraded":"2024-04-25T08:40:55.383888+0000","last_fullsized":"2024-04-25T08:40:55.383888+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:40.486804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381381+0000","last_change":"2024-04-25T08:40:54.386093+0000","last_active":"2024-04-25T08:40:55.381381+0000","last_peered":"2024-04-25T08:40:55.381381+0000","last_clean":"2024-04-25T08:40:55.381381+0000","last_became_active":"2024-04-25T08:40:54.385187+0000","last_became_peered":"2024-04-25T08:40:54.385187+0000","last_unstale":"2024-04-25T08:40:55.381381+0000","last_undegraded":"2024-04-25T08:40:55.381381+0000","last_fullsized":"2024-04-25T08:40:55.381381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.539986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383508+0000","last_change":"2024-04-25T08:40:54.386486+0000","last_active":"2024-04-25T08:40:55.383508+0000","last_peered":"2024-04-25T08:40:55.383508+0000","last_clean":"2024-04-25T08:40:55.383508+0000","last_became_active":"2024-04-25T08:40:54.386209+0000","last_became_peered":"2024-04-25T08:40:54.386209+0000","last_unstale":"2024-04-25T08:40:55.383508+0000","last_undegraded":"2024-04-25T08:40:55.383508+0000","last_fullsized":"2024-04-25T08:40:55.383508+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:37.819157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382698+0000","last_change":"2024-04-25T08:40:54.388200+0000","last_active":"2024-04-25T08:40:55.382698+0000","last_peered":"2024-04-25T08:40:55.382698+0000","last_clean":"2024-04-25T08:40:55.382698+0000","last_became_active":"2024-04-25T08:40:54.387935+0000","last_became_peered":"2024-04-25T08:40:54.387935+0000","last_unstale":"2024-04-25T08:40:55.382698+0000","last_undegraded":"2024-04-25T08:40:55.382698+0000","last_fullsized":"2024-04-25T08:40:55.382698+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:56.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382793+0000","last_change":"2024-04-25T08:40:54.388543+0000","last_active":"2024-04-25T08:40:55.382793+0000","last_peered":"2024-04-25T08:40:55.382793+0000","last_clean":"2024-04-25T08:40:55.382793+0000","last_became_active":"2024-04-25T08:40:54.387411+0000","last_became_peered":"2024-04-25T08:40:54.387411+0000","last_unstale":"2024-04-25T08:40:55.382793+0000","last_undegraded":"2024-04-25T08:40:55.382793+0000","last_fullsized":"2024-04-25T08:40:55.382793+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:56.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.380999+0000","last_change":"2024-04-25T08:40:54.380867+0000","last_active":"2024-04-25T08:40:55.380999+0000","last_peered":"2024-04-25T08:40:55.380999+0000","last_clean":"2024-04-25T08:40:55.380999+0000","last_became_active":"2024-04-25T08:40:54.380595+0000","last_became_peered":"2024-04-25T08:40:54.380595+0000","last_unstale":"2024-04-25T08:40:55.380999+0000","last_undegraded":"2024-04-25T08:40:55.380999+0000","last_fullsized":"2024-04-25T08:40:55.380999+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:58.196801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382612+0000","last_change":"2024-04-25T08:40:54.389841+0000","last_active":"2024-04-25T08:40:55.382612+0000","last_peered":"2024-04-25T08:40:55.382612+0000","last_clean":"2024-04-25T08:40:55.382612+0000","last_became_active":"2024-04-25T08:40:54.389572+0000","last_became_peered":"2024-04-25T08:40:54.389572+0000","last_unstale":"2024-04-25T08:40:55.382612+0000","last_undegraded":"2024-04-25T08:40:55.382612+0000","last_fullsized":"2024-04-25T08:40:55.382612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:04.970606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"131'4","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.384013+0000","last_change":"2024-04-25T08:40:53.379026+0000","last_active":"2024-04-25T08:40:55.384013+0000","last_peered":"2024-04-25T08:40:55.384013+0000","last_clean":"2024-04-25T08:40:55.384013+0000","last_became_active":"2024-04-25T08:40:53.378751+0000","last_became_peered":"2024-04-25T08:40:53.378751+0000","last_unstale":"2024-04-25T08:40:55.384013+0000","last_undegraded":"2024-04-25T08:40:55.384013+0000","last_fullsized":"2024-04-25T08:40:55.384013+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.556400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500758+0000","last_change":"2024-04-25T08:40:54.378840+0000","last_active":"2024-04-25T08:40:55.500758+0000","last_peered":"2024-04-25T08:40:55.500758+0000","last_clean":"2024-04-25T08:40:55.500758+0000","last_became_active":"2024-04-25T08:40:54.378566+0000","last_became_peered":"2024-04-25T08:40:54.378566+0000","last_unstale":"2024-04-25T08:40:55.500758+0000","last_undegraded":"2024-04-25T08:40:55.500758+0000","last_fullsized":"2024-04-25T08:40:55.500758+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:40.486804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382917+0000","last_change":"2024-04-25T08:40:54.387624+0000","last_active":"2024-04-25T08:40:55.382917+0000","last_peered":"2024-04-25T08:40:55.382917+0000","last_clean":"2024-04-25T08:40:55.382917+0000","last_became_active":"2024-04-25T08:40:54.387359+0000","last_became_peered":"2024-04-25T08:40:54.387359+0000","last_unstale":"2024-04-25T08:40:55.382917+0000","last_undegraded":"2024-04-25T08:40:55.382917+0000","last_fullsized":"2024-04-25T08:40:55.382917+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:22.825790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383795+0000","last_change":"2024-04-25T08:40:53.373572+0000","last_active":"2024-04-25T08:40:55.383795+0000","last_peered":"2024-04-25T08:40:55.383795+0000","last_clean":"2024-04-25T08:40:55.383795+0000","last_became_active":"2024-04-25T08:40:53.373290+0000","last_became_peered":"2024-04-25T08:40:53.373290+0000","last_unstale":"2024-04-25T08:40:55.383795+0000","last_undegraded":"2024-04-25T08:40:55.383795+0000","last_fullsized":"2024-04-25T08:40:55.383795+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:40.850065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383312+0000","last_change":"2024-04-25T08:40:54.388330+0000","last_active":"2024-04-25T08:40:55.383312+0000","last_peered":"2024-04-25T08:40:55.383312+0000","last_clean":"2024-04-25T08:40:55.383312+0000","last_became_active":"2024-04-25T08:40:54.387849+0000","last_became_peered":"2024-04-25T08:40:54.387849+0000","last_unstale":"2024-04-25T08:40:55.383312+0000","last_undegraded":"2024-04-25T08:40:55.383312+0000","last_fullsized":"2024-04-25T08:40:55.383312+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:40.899090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595098+0000","last_change":"2024-04-25T08:40:53.376507+0000","last_active":"2024-04-25T08:40:55.595098+0000","last_peered":"2024-04-25T08:40:55.595098+0000","last_clean":"2024-04-25T08:40:55.595098+0000","last_became_active":"2024-04-25T08:40:53.376186+0000","last_became_peered":"2024-04-25T08:40:53.376186+0000","last_unstale":"2024-04-25T08:40:55.595098+0000","last_undegraded":"2024-04-25T08:40:55.595098+0000","last_fullsized":"2024-04-25T08:40:55.595098+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:50.437720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383020+0000","last_change":"2024-04-25T08:40:53.376440+0000","last_active":"2024-04-25T08:40:55.383020+0000","last_peered":"2024-04-25T08:40:55.383020+0000","last_clean":"2024-04-25T08:40:55.383020+0000","last_became_active":"2024-04-25T08:40:53.376169+0000","last_became_peered":"2024-04-25T08:40:53.376169+0000","last_unstale":"2024-04-25T08:40:55.383020+0000","last_undegraded":"2024-04-25T08:40:55.383020+0000","last_fullsized":"2024-04-25T08:40:55.383020+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:52.489674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500789+0000","last_change":"2024-04-25T08:40:53.380621+0000","last_active":"2024-04-25T08:40:55.500789+0000","last_peered":"2024-04-25T08:40:55.500789+0000","last_clean":"2024-04-25T08:40:55.500789+0000","last_became_active":"2024-04-25T08:40:53.380150+0000","last_became_peered":"2024-04-25T08:40:53.380150+0000","last_unstale":"2024-04-25T08:40:55.500789+0000","last_undegraded":"2024-04-25T08:40:55.500789+0000","last_fullsized":"2024-04-25T08:40:55.500789+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:56.303715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595249+0000","last_change":"2024-04-25T08:40:54.387442+0000","last_active":"2024-04-25T08:40:55.595249+0000","last_peered":"2024-04-25T08:40:55.595249+0000","last_clean":"2024-04-25T08:40:55.595249+0000","last_became_active":"2024-04-25T08:40:54.387157+0000","last_became_peered":"2024-04-25T08:40:54.387157+0000","last_unstale":"2024-04-25T08:40:55.595249+0000","last_undegraded":"2024-04-25T08:40:55.595249+0000","last_fullsized":"2024-04-25T08:40:55.595249+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:48.139944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382845+0000","last_change":"2024-04-25T08:40:54.376953+0000","last_active":"2024-04-25T08:40:55.382845+0000","last_peered":"2024-04-25T08:40:55.382845+0000","last_clean":"2024-04-25T08:40:55.382845+0000","last_became_active":"2024-04-25T08:40:54.376686+0000","last_became_peered":"2024-04-25T08:40:54.376686+0000","last_unstale":"2024-04-25T08:40:55.382845+0000","last_undegraded":"2024-04-25T08:40:55.382845+0000","last_fullsized":"2024-04-25T08:40:55.382845+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:22.825790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383572+0000","last_change":"2024-04-25T08:40:54.385830+0000","last_active":"2024-04-25T08:40:55.383572+0000","last_peered":"2024-04-25T08:40:55.383572+0000","last_clean":"2024-04-25T08:40:55.383572+0000","last_became_active":"2024-04-25T08:40:54.384994+0000","last_became_peered":"2024-04-25T08:40:54.384994+0000","last_unstale":"2024-04-25T08:40:55.383572+0000","last_undegraded":"2024-04-25T08:40:55.383572+0000","last_fullsized":"2024-04-25T08:40:55.383572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:51.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381354+0000","last_change":"2024-04-25T08:40:54.383836+0000","last_active":"2024-04-25T08:40:55.381354+0000","last_peered":"2024-04-25T08:40:55.381354+0000","last_clean":"2024-04-25T08:40:55.381354+0000","last_became_active":"2024-04-25T08:40:54.383486+0000","last_became_peered":"2024-04-25T08:40:54.383486+0000","last_unstale":"2024-04-25T08:40:55.381354+0000","last_undegraded":"2024-04-25T08:40:55.381354+0000","last_fullsized":"2024-04-25T08:40:55.381354+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:20.048786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383208+0000","last_change":"2024-04-25T08:40:53.377899+0000","last_active":"2024-04-25T08:40:55.383208+0000","last_peered":"2024-04-25T08:40:55.383208+0000","last_clean":"2024-04-25T08:40:55.383208+0000","last_became_active":"2024-04-25T08:40:53.377626+0000","last_became_peered":"2024-04-25T08:40:53.377626+0000","last_unstale":"2024-04-25T08:40:55.383208+0000","last_undegraded":"2024-04-25T08:40:55.383208+0000","last_fullsized":"2024-04-25T08:40:55.383208+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:08.112732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"131'8","reported_seq":22,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596483+0000","last_change":"2024-04-25T08:40:53.378565+0000","last_active":"2024-04-25T08:40:55.596483+0000","last_peered":"2024-04-25T08:40:55.596483+0000","last_clean":"2024-04-25T08:40:55.596483+0000","last_became_active":"2024-04-25T08:40:53.378251+0000","last_became_peered":"2024-04-25T08:40:53.378251+0000","last_unstale":"2024-04-25T08:40:55.596483+0000","last_undegraded":"2024-04-25T08:40:55.596483+0000","last_fullsized":"2024-04-25T08:40:55.596483+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:03.962369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382898+0000","last_change":"2024-04-25T08:40:54.372817+0000","last_active":"2024-04-25T08:40:55.382898+0000","last_peered":"2024-04-25T08:40:55.382898+0000","last_clean":"2024-04-25T08:40:55.382898+0000","last_became_active":"2024-04-25T08:40:54.371590+0000","last_became_peered":"2024-04-25T08:40:54.371590+0000","last_unstale":"2024-04-25T08:40:55.382898+0000","last_undegraded":"2024-04-25T08:40:55.382898+0000","last_fullsized":"2024-04-25T08:40:55.382898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:40.899090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500851+0000","last_change":"2024-04-25T08:40:53.382667+0000","last_active":"2024-04-25T08:40:55.500851+0000","last_peered":"2024-04-25T08:40:55.500851+0000","last_clean":"2024-04-25T08:40:55.500851+0000","last_became_active":"2024-04-25T08:40:53.382436+0000","last_became_peered":"2024-04-25T08:40:53.382436+0000","last_unstale":"2024-04-25T08:40:55.500851+0000","last_undegraded":"2024-04-25T08:40:55.500851+0000","last_fullsized":"2024-04-25T08:40:55.500851+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:39.954950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383651+0000","last_change":"2024-04-25T08:40:53.384093+0000","last_active":"2024-04-25T08:40:55.383651+0000","last_peered":"2024-04-25T08:40:55.383651+0000","last_clean":"2024-04-25T08:40:55.383651+0000","last_became_active":"2024-04-25T08:40:53.383830+0000","last_became_peered":"2024-04-25T08:40:53.383830+0000","last_unstale":"2024-04-25T08:40:55.383651+0000","last_undegraded":"2024-04-25T08:40:55.383651+0000","last_fullsized":"2024-04-25T08:40:55.383651+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:40.850065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595793+0000","last_change":"2024-04-25T08:40:53.377620+0000","last_active":"2024-04-25T08:40:55.595793+0000","last_peered":"2024-04-25T08:40:55.595793+0000","last_clean":"2024-04-25T08:40:55.595793+0000","last_became_active":"2024-04-25T08:40:53.377340+0000","last_became_peered":"2024-04-25T08:40:53.377340+0000","last_unstale":"2024-04-25T08:40:55.595793+0000","last_undegraded":"2024-04-25T08:40:55.595793+0000","last_fullsized":"2024-04-25T08:40:55.595793+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:33.063592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381144+0000","last_change":"2024-04-25T08:40:54.382916+0000","last_active":"2024-04-25T08:40:55.381144+0000","last_peered":"2024-04-25T08:40:55.381144+0000","last_clean":"2024-04-25T08:40:55.381144+0000","last_became_active":"2024-04-25T08:40:54.382634+0000","last_became_peered":"2024-04-25T08:40:54.382634+0000","last_unstale":"2024-04-25T08:40:55.381144+0000","last_undegraded":"2024-04-25T08:40:55.381144+0000","last_fullsized":"2024-04-25T08:40:55.381144+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:28.055098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383840+0000","last_change":"2024-04-25T08:40:53.372724+0000","last_active":"2024-04-25T08:40:55.383840+0000","last_peered":"2024-04-25T08:40:55.383840+0000","last_clean":"2024-04-25T08:40:55.383840+0000","last_became_active":"2024-04-25T08:40:53.372440+0000","last_became_peered":"2024-04-25T08:40:53.372440+0000","last_unstale":"2024-04-25T08:40:55.383840+0000","last_undegraded":"2024-04-25T08:40:55.383840+0000","last_fullsized":"2024-04-25T08:40:55.383840+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:46.131350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500809+0000","last_change":"2024-04-25T08:40:54.378104+0000","last_active":"2024-04-25T08:40:55.500809+0000","last_peered":"2024-04-25T08:40:55.500809+0000","last_clean":"2024-04-25T08:40:55.500809+0000","last_became_active":"2024-04-25T08:40:54.377753+0000","last_became_peered":"2024-04-25T08:40:54.377753+0000","last_unstale":"2024-04-25T08:40:55.500809+0000","last_undegraded":"2024-04-25T08:40:55.500809+0000","last_fullsized":"2024-04-25T08:40:55.500809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:25.693049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596583+0000","last_change":"2024-04-25T08:40:53.376843+0000","last_active":"2024-04-25T08:40:55.596583+0000","last_peered":"2024-04-25T08:40:55.596583+0000","last_clean":"2024-04-25T08:40:55.596583+0000","last_became_active":"2024-04-25T08:40:53.376238+0000","last_became_peered":"2024-04-25T08:40:53.376238+0000","last_unstale":"2024-04-25T08:40:55.596583+0000","last_undegraded":"2024-04-25T08:40:55.596583+0000","last_fullsized":"2024-04-25T08:40:55.596583+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:55.097704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383108+0000","last_change":"2024-04-25T08:40:54.384630+0000","last_active":"2024-04-25T08:40:55.383108+0000","last_peered":"2024-04-25T08:40:55.383108+0000","last_clean":"2024-04-25T08:40:55.383108+0000","last_became_active":"2024-04-25T08:40:54.384077+0000","last_became_peered":"2024-04-25T08:40:54.384077+0000","last_unstale":"2024-04-25T08:40:55.383108+0000","last_undegraded":"2024-04-25T08:40:55.383108+0000","last_fullsized":"2024-04-25T08:40:55.383108+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:56.551387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"132'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:56.368767+0000","last_change":"2024-04-25T08:40:53.378186+0000","last_active":"2024-04-25T08:40:56.368767+0000","last_peered":"2024-04-25T08:40:56.368767+0000","last_clean":"2024-04-25T08:40:56.368767+0000","last_became_active":"2024-04-25T08:40:53.377880+0000","last_became_peered":"2024-04-25T08:40:53.377880+0000","last_unstale":"2024-04-25T08:40:56.368767+0000","last_undegraded":"2024-04-25T08:40:56.368767+0000","last_fullsized":"2024-04-25T08:40:56.368767+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:47.172417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383430+0000","last_change":"2024-04-25T08:40:53.381261+0000","last_active":"2024-04-25T08:40:55.383430+0000","last_peered":"2024-04-25T08:40:55.383430+0000","last_clean":"2024-04-25T08:40:55.383430+0000","last_became_active":"2024-04-25T08:40:53.380920+0000","last_became_peered":"2024-04-25T08:40:53.380920+0000","last_unstale":"2024-04-25T08:40:55.383430+0000","last_undegraded":"2024-04-25T08:40:55.383430+0000","last_fullsized":"2024-04-25T08:40:55.383430+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:39.954950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381178+0000","last_change":"2024-04-25T08:40:54.386615+0000","last_active":"2024-04-25T08:40:55.381178+0000","last_peered":"2024-04-25T08:40:55.381178+0000","last_clean":"2024-04-25T08:40:55.381178+0000","last_became_active":"2024-04-25T08:40:54.386359+0000","last_became_peered":"2024-04-25T08:40:54.386359+0000","last_unstale":"2024-04-25T08:40:55.381178+0000","last_undegraded":"2024-04-25T08:40:55.381178+0000","last_fullsized":"2024-04-25T08:40:55.381178+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.093642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595669+0000","last_change":"2024-04-25T08:40:53.378763+0000","last_active":"2024-04-25T08:40:55.595669+0000","last_peered":"2024-04-25T08:40:55.595669+0000","last_clean":"2024-04-25T08:40:55.595669+0000","last_became_active":"2024-04-25T08:40:53.378453+0000","last_became_peered":"2024-04-25T08:40:53.378453+0000","last_unstale":"2024-04-25T08:40:55.595669+0000","last_undegraded":"2024-04-25T08:40:55.595669+0000","last_fullsized":"2024-04-25T08:40:55.595669+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:57.408929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595573+0000","last_change":"2024-04-25T08:40:54.387431+0000","last_active":"2024-04-25T08:40:55.595573+0000","last_peered":"2024-04-25T08:40:55.595573+0000","last_clean":"2024-04-25T08:40:55.595573+0000","last_became_active":"2024-04-25T08:40:54.387149+0000","last_became_peered":"2024-04-25T08:40:54.387149+0000","last_unstale":"2024-04-25T08:40:55.595573+0000","last_undegraded":"2024-04-25T08:40:55.595573+0000","last_fullsized":"2024-04-25T08:40:55.595573+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:49.064739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500551+0000","last_change":"2024-04-25T08:40:53.385220+0000","last_active":"2024-04-25T08:40:55.500551+0000","last_peered":"2024-04-25T08:40:55.500551+0000","last_clean":"2024-04-25T08:40:55.500551+0000","last_became_active":"2024-04-25T08:40:53.384943+0000","last_became_peered":"2024-04-25T08:40:53.384943+0000","last_unstale":"2024-04-25T08:40:55.500551+0000","last_undegraded":"2024-04-25T08:40:55.500551+0000","last_fullsized":"2024-04-25T08:40:55.500551+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:16.391337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595687+0000","last_change":"2024-04-25T08:40:54.388970+0000","last_active":"2024-04-25T08:40:55.595687+0000","last_peered":"2024-04-25T08:40:55.595687+0000","last_clean":"2024-04-25T08:40:55.595687+0000","last_became_active":"2024-04-25T08:40:54.388677+0000","last_became_peered":"2024-04-25T08:40:54.388677+0000","last_unstale":"2024-04-25T08:40:55.595687+0000","last_undegraded":"2024-04-25T08:40:55.595687+0000","last_fullsized":"2024-04-25T08:40:55.595687+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:37.825506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595830+0000","last_change":"2024-04-25T08:40:54.381075+0000","last_active":"2024-04-25T08:40:55.595830+0000","last_peered":"2024-04-25T08:40:55.595830+0000","last_clean":"2024-04-25T08:40:55.595830+0000","last_became_active":"2024-04-25T08:40:54.380766+0000","last_became_peered":"2024-04-25T08:40:54.380766+0000","last_unstale":"2024-04-25T08:40:55.595830+0000","last_undegraded":"2024-04-25T08:40:55.595830+0000","last_fullsized":"2024-04-25T08:40:55.595830+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:56.441011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595339+0000","last_change":"2024-04-25T08:40:53.378417+0000","last_active":"2024-04-25T08:40:55.595339+0000","last_peered":"2024-04-25T08:40:55.595339+0000","last_clean":"2024-04-25T08:40:55.595339+0000","last_became_active":"2024-04-25T08:40:53.378139+0000","last_became_peered":"2024-04-25T08:40:53.378139+0000","last_unstale":"2024-04-25T08:40:55.595339+0000","last_undegraded":"2024-04-25T08:40:55.595339+0000","last_fullsized":"2024-04-25T08:40:55.595339+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.250586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:53.353507+0000","last_change":"2024-04-25T08:40:53.353507+0000","last_active":"2024-04-25T08:40:53.353507+0000","last_peered":"2024-04-25T08:40:53.353507+0000","last_clean":"2024-04-25T08:40:53.353507+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:53.353507+0000","last_undegraded":"2024-04-25T08:40:53.353507+0000","last_fullsized":"2024-04-25T08:40:53.353507+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383951+0000","last_change":"2024-04-25T08:40:54.378981+0000","last_active":"2024-04-25T08:40:55.383951+0000","last_peered":"2024-04-25T08:40:55.383951+0000","last_clean":"2024-04-25T08:40:55.383951+0000","last_became_active":"2024-04-25T08:40:54.378496+0000","last_became_peered":"2024-04-25T08:40:54.378496+0000","last_unstale":"2024-04-25T08:40:55.383951+0000","last_undegraded":"2024-04-25T08:40:55.383951+0000","last_fullsized":"2024-04-25T08:40:55.383951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:25.693049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:40:52.345270+0000","last_change":"2024-04-25T08:40:52.345270+0000","last_active":"2024-04-25T08:40:52.345270+0000","last_peered":"2024-04-25T08:40:52.345270+0000","last_clean":"2024-04-25T08:40:52.345270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:40:52.345270+0000","last_undegraded":"2024-04-25T08:40:52.345270+0000","last_fullsized":"2024-04-25T08:40:52.345270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":372,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-04-25T08:40:50.676773+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:40:50.676773+0000","last_peered":"2024-04-25T08:40:50.676773+0000","last_clean":"2024-04-25T08:40:50.676773+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:40:50.676773+0000","last_undegraded":"2024-04-25T08:40:50.676773+0000","last_fullsized":"2024-04-25T08:40:50.676773+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":76,"acting":76,"num_store_stats":5},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":674,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20,"ondisk_log_size":20,"up":34,"acting":34,"num_store_stats":5},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65264,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306576,"statfs":{"total":96636764160,"available":96569933824,"internally_reserved":0,"allocated":2195456,"data_stored":1334949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59172,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312668,"statfs":{"total":96636764160,"available":96576172032,"internally_reserved":0,"allocated":1658880,"data_stored":771980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55012,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316828,"statfs":{"total":96636764160,"available":96580431872,"internally_reserved":0,"allocated":1654784,"data_stored":771330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68892,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302948,"statfs":{"total":96636764160,"available":96566218752,"internally_reserved":0,"allocated":1650688,"data_stored":771308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73004,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298836,"statfs":{"total":96636764160,"available":96562008064,"internally_reserved":0,"allocated":1601536,"data_stored":744581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59116,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312724,"statfs":{"total":96636764160,"available":96576229376,"internally_reserved":0,"allocated":1601536,"data_stored":744581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":1654784,"data_stored":771378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69416,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302424,"statfs":{"total":96636764160,"available":96565682176,"internally_reserved":0,"allocated":2252800,"data_stored":1361732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:41:01.813 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:41:01.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.934+0000 7fe6427e2640 1 Processor -- start 2024-04-25T08:41:01.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.934+0000 7fe6427e2640 1 -- start start 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.935+0000 7fe6427e2640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe63c15dc10 0x7fe63c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.935+0000 7fe6427e2640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe63c0ced60 0x7fe63c0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.935+0000 7fe6427e2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0d3900 0x7fe63c0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.935+0000 7fe6427e2640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c15dff0 con 0x7fe63c15dc10 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.935+0000 7fe6427e2640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe63c0b6990 con 0x7fe63c0ced60 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.935+0000 7fe6427e2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0d4710 con 0x7fe63c0d3900 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.935+0000 7fe63bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe63c15dc10 0x7fe63c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.935+0000 7fe63bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe63c15dc10 0x7fe63c0ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47736/0 (socket says 172.21.15.38:47736) 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.935+0000 7fe63bfff640 1 -- 172.21.15.38:0/1592222502 learned_addr learned my addr 172.21.15.38:0/1592222502 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.936+0000 7fe640d7f640 1 --2- 172.21.15.38:0/1592222502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0d3900 0x7fe63c0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.936+0000 7fe63bfff640 1 -- 172.21.15.38:0/1592222502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0d3900 msgr2=0x7fe63c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.936+0000 7fe63b7fe640 1 --2- 172.21.15.38:0/1592222502 >> v2:172.21.15.107:3300/0 conn(0x7fe63c0ced60 0x7fe63c0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:01.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.936+0000 7fe63bfff640 1 --2- 172.21.15.38:0/1592222502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0d3900 0x7fe63c0d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.936+0000 7fe63bfff640 1 -- 172.21.15.38:0/1592222502 >> v2:172.21.15.107:3300/0 conn(0x7fe63c0ced60 msgr2=0x7fe63c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.936+0000 7fe63bfff640 1 --2- 172.21.15.38:0/1592222502 >> v2:172.21.15.107:3300/0 conn(0x7fe63c0ced60 0x7fe63c0d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.936+0000 7fe63bfff640 1 -- 172.21.15.38:0/1592222502 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c0d3e70 con 0x7fe63c15dc10 2024-04-25T08:41:01.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.936+0000 7fe63b7fe640 1 --2- 172.21.15.38:0/1592222502 >> v2:172.21.15.107:3300/0 conn(0x7fe63c0ced60 0x7fe63c0d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:01.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.936+0000 7fe63bfff640 1 --2- 172.21.15.38:0/1592222502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe63c15dc10 0x7fe63c0ce820 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0b9600 tx=0x7fe63002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601353a2a963ff44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:01.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.937+0000 7fe63affd640 1 -- 172.21.15.38:0/1592222502 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe630037070 con 0x7fe63c15dc10 2024-04-25T08:41:01.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.937+0000 7fe63affd640 1 -- 172.21.15.38:0/1592222502 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe630032440 con 0x7fe63c15dc10 2024-04-25T08:41:01.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.937+0000 7fe63affd640 1 -- 172.21.15.38:0/1592222502 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe630032810 con 0x7fe63c15dc10 2024-04-25T08:41:01.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.937+0000 7fe6427e2640 1 -- 172.21.15.38:0/1592222502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe63c15dc10 msgr2=0x7fe63c0ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.937+0000 7fe6427e2640 1 --2- 172.21.15.38:0/1592222502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe63c15dc10 0x7fe63c0ce820 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fe63c0b9600 tx=0x7fe63002a140 comp rx=0 tx=0).stop 2024-04-25T08:41:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.937+0000 7fe6427e2640 1 -- 172.21.15.38:0/1592222502 shutdown_connections 2024-04-25T08:41:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.937+0000 7fe6427e2640 1 --2- 172.21.15.38:0/1592222502 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0d3900 0x7fe63c0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.937+0000 7fe6427e2640 1 --2- 172.21.15.38:0/1592222502 >> v2:172.21.15.107:3300/0 conn(0x7fe63c0ced60 0x7fe63c0d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.937+0000 7fe6427e2640 1 --2- 172.21.15.38:0/1592222502 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe63c15dc10 0x7fe63c0ce820 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.937+0000 7fe6427e2640 1 -- 172.21.15.38:0/1592222502 >> 172.21.15.38:0/1592222502 conn(0x7fe63c066d10 msgr2=0x7fe63c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.937+0000 7fe6427e2640 1 -- 172.21.15.38:0/1592222502 shutdown_connections 2024-04-25T08:41:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.937+0000 7fe6427e2640 1 -- 172.21.15.38:0/1592222502 wait complete. 2024-04-25T08:41:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.938+0000 7fe6427e2640 1 Processor -- start 2024-04-25T08:41:01.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.939+0000 7fe6427e2640 1 -- start start 2024-04-25T08:41:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.939+0000 7fe6427e2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0ced60 0x7fe63c1524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.939+0000 7fe6427e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe63c0d3900 0x7fe63c0c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:01.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.939+0000 7fe6427e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c15dc10 0x7fe63c0c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.939+0000 7fe6427e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0b6bc0 con 0x7fe63c0d3900 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.939+0000 7fe6427e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0d4430 con 0x7fe63c15dc10 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.939+0000 7fe6427e2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c0b2370 con 0x7fe63c0ced60 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe63b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe63c0d3900 0x7fe63c0c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe63b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe63c0d3900 0x7fe63c0c7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47740/0 (socket says 172.21.15.38:47740) 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe63b7fe640 1 -- 172.21.15.38:0/2402349784 learned_addr learned my addr 172.21.15.38:0/2402349784 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe640d7f640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c15dc10 0x7fe63c0c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe63bfff640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0ced60 0x7fe63c1524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe63b7fe640 1 -- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0ced60 msgr2=0x7fe63c1524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe63b7fe640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0ced60 0x7fe63c1524a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe63b7fe640 1 -- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c15dc10 msgr2=0x7fe63c0c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe63b7fe640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c15dc10 0x7fe63c0c7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:01.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe63b7fe640 1 -- 172.21.15.38:0/2402349784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c152eb0 con 0x7fe63c0d3900 2024-04-25T08:41:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe63b7fe640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe63c0d3900 0x7fe63c0c7430 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00c7c0 tx=0x7fe62c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe638ff9640 1 -- 172.21.15.38:0/2402349784 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c015070 con 0x7fe63c0d3900 2024-04-25T08:41:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe638ff9640 1 -- 172.21.15.38:0/2402349784 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62c007a30 con 0x7fe63c0d3900 2024-04-25T08:41:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe6427e2640 1 -- 172.21.15.38:0/2402349784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63c0c94e0 con 0x7fe63c0d3900 2024-04-25T08:41:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe638ff9640 1 -- 172.21.15.38:0/2402349784 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62c007e00 con 0x7fe63c0d3900 2024-04-25T08:41:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.940+0000 7fe6427e2640 1 -- 172.21.15.38:0/2402349784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63c0c9aa0 con 0x7fe63c0d3900 2024-04-25T08:41:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.941+0000 7fe6427e2640 1 -- 172.21.15.38:0/2402349784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63c068e80 con 0x7fe63c0d3900 2024-04-25T08:41:01.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.942+0000 7fe638ff9640 1 -- 172.21.15.38:0/2402349784 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe62c018070 con 0x7fe63c0d3900 2024-04-25T08:41:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.942+0000 7fe638ff9640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe62806b500 0x7fe62806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.942+0000 7fe638ff9640 1 -- 172.21.15.38:0/2402349784 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe62c08a260 con 0x7fe63c0d3900 2024-04-25T08:41:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.942+0000 7fe63bfff640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe62806b500 0x7fe62806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:01.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.943+0000 7fe63bfff640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe62806b500 0x7fe62806d9c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe630034000 tx=0x7fe63002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:01.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:01.947+0000 7fe638ff9640 1 -- 172.21.15.38:0/2402349784 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe63c152eb0 con 0x7fe63c0d3900 2024-04-25T08:41:02.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.128+0000 7fe6427e2640 1 -- 172.21.15.38:0/2402349784 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe63c0cda00 con 0x7fe62806b500 2024-04-25T08:41:02.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.135+0000 7fe638ff9640 1 -- 172.21.15.38:0/2402349784 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252484 (secure 0 0 0) 0x7fe63c0cda00 con 0x7fe62806b500 2024-04-25T08:41:02.130 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:02.131 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:41:02.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.138+0000 7fe6427e2640 1 -- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe62806b500 msgr2=0x7fe62806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.138+0000 7fe6427e2640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe62806b500 0x7fe62806d9c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe630034000 tx=0x7fe63002e000 comp rx=0 tx=0).stop 2024-04-25T08:41:02.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.138+0000 7fe6427e2640 1 -- 172.21.15.38:0/2402349784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe63c0d3900 msgr2=0x7fe63c0c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.138+0000 7fe6427e2640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe63c0d3900 0x7fe63c0c7430 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00c7c0 tx=0x7fe62c00cc80 comp rx=0 tx=0).stop 2024-04-25T08:41:02.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.138+0000 7fe6427e2640 1 -- 172.21.15.38:0/2402349784 shutdown_connections 2024-04-25T08:41:02.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.138+0000 7fe6427e2640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe63c15dc10 0x7fe63c0c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.138+0000 7fe6427e2640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe63c0d3900 0x7fe63c0c7430 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.138+0000 7fe6427e2640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe62806b500 0x7fe62806d9c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.138+0000 7fe6427e2640 1 --2- 172.21.15.38:0/2402349784 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe63c0ced60 0x7fe63c1524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.138+0000 7fe6427e2640 1 -- 172.21.15.38:0/2402349784 >> 172.21.15.38:0/2402349784 conn(0x7fe63c066d10 msgr2=0x7fe63c0d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:02.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.138+0000 7fe6427e2640 1 -- 172.21.15.38:0/2402349784 shutdown_connections 2024-04-25T08:41:02.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.138+0000 7fe6427e2640 1 -- 172.21.15.38:0/2402349784 wait complete. 2024-04-25T08:41:02.142 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-04-25T08:41:01.337241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":509232,"kb_used_data":14120,"kb_used_omap":12,"kb_used_meta":494963,"kb_avail":754465488,"statfs":{"total":773094113280,"available":772572659712,"internally_reserved":0,"allocated":14458880,"data_stored":7356020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":506842672},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.974455"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595976+0000","last_change":"2024-04-25T08:40:54.382403+0000","last_active":"2024-04-25T08:40:55.595976+0000","last_peered":"2024-04-25T08:40:55.595976+0000","last_clean":"2024-04-25T08:40:55.595976+0000","last_became_active":"2024-04-25T08:40:54.382121+0000","last_became_peered":"2024-04-25T08:40:54.382121+0000","last_unstale":"2024-04-25T08:40:55.595976+0000","last_undegraded":"2024-04-25T08:40:55.595976+0000","last_fullsized":"2024-04-25T08:40:55.595976+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:10.025762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500770+0000","last_change":"2024-04-25T08:40:54.385506+0000","last_active":"2024-04-25T08:40:55.500770+0000","last_peered":"2024-04-25T08:40:55.500770+0000","last_clean":"2024-04-25T08:40:55.500770+0000","last_became_active":"2024-04-25T08:40:54.385108+0000","last_became_peered":"2024-04-25T08:40:54.385108+0000","last_unstale":"2024-04-25T08:40:55.500770+0000","last_undegraded":"2024-04-25T08:40:55.500770+0000","last_fullsized":"2024-04-25T08:40:55.500770+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.502699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383789+0000","last_change":"2024-04-25T08:40:54.378766+0000","last_active":"2024-04-25T08:40:55.383789+0000","last_peered":"2024-04-25T08:40:55.383789+0000","last_clean":"2024-04-25T08:40:55.383789+0000","last_became_active":"2024-04-25T08:40:54.378120+0000","last_became_peered":"2024-04-25T08:40:54.378120+0000","last_unstale":"2024-04-25T08:40:55.383789+0000","last_undegraded":"2024-04-25T08:40:55.383789+0000","last_fullsized":"2024-04-25T08:40:55.383789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:49.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500706+0000","last_change":"2024-04-25T08:40:54.382334+0000","last_active":"2024-04-25T08:40:55.500706+0000","last_peered":"2024-04-25T08:40:55.500706+0000","last_clean":"2024-04-25T08:40:55.500706+0000","last_became_active":"2024-04-25T08:40:54.382070+0000","last_became_peered":"2024-04-25T08:40:54.382070+0000","last_unstale":"2024-04-25T08:40:55.500706+0000","last_undegraded":"2024-04-25T08:40:55.500706+0000","last_fullsized":"2024-04-25T08:40:55.500706+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:49.062444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.594912+0000","last_change":"2024-04-25T08:40:54.391285+0000","last_active":"2024-04-25T08:40:55.594912+0000","last_peered":"2024-04-25T08:40:55.594912+0000","last_clean":"2024-04-25T08:40:55.594912+0000","last_became_active":"2024-04-25T08:40:54.387740+0000","last_became_peered":"2024-04-25T08:40:54.387740+0000","last_unstale":"2024-04-25T08:40:55.594912+0000","last_undegraded":"2024-04-25T08:40:55.594912+0000","last_fullsized":"2024-04-25T08:40:55.594912+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:29.262872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595381+0000","last_change":"2024-04-25T08:40:54.384503+0000","last_active":"2024-04-25T08:40:55.595381+0000","last_peered":"2024-04-25T08:40:55.595381+0000","last_clean":"2024-04-25T08:40:55.595381+0000","last_became_active":"2024-04-25T08:40:54.384141+0000","last_became_peered":"2024-04-25T08:40:54.384141+0000","last_unstale":"2024-04-25T08:40:55.595381+0000","last_undegraded":"2024-04-25T08:40:55.595381+0000","last_fullsized":"2024-04-25T08:40:55.595381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:48.148660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595417+0000","last_change":"2024-04-25T08:40:54.396178+0000","last_active":"2024-04-25T08:40:55.595417+0000","last_peered":"2024-04-25T08:40:55.595417+0000","last_clean":"2024-04-25T08:40:55.595417+0000","last_became_active":"2024-04-25T08:40:54.395833+0000","last_became_peered":"2024-04-25T08:40:54.395833+0000","last_unstale":"2024-04-25T08:40:55.595417+0000","last_undegraded":"2024-04-25T08:40:55.595417+0000","last_fullsized":"2024-04-25T08:40:55.595417+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:16.235235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382449+0000","last_change":"2024-04-25T08:40:54.383613+0000","last_active":"2024-04-25T08:40:55.382449+0000","last_peered":"2024-04-25T08:40:55.382449+0000","last_clean":"2024-04-25T08:40:55.382449+0000","last_became_active":"2024-04-25T08:40:54.383329+0000","last_became_peered":"2024-04-25T08:40:54.383329+0000","last_unstale":"2024-04-25T08:40:55.382449+0000","last_undegraded":"2024-04-25T08:40:55.382449+0000","last_fullsized":"2024-04-25T08:40:55.382449+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:58.417166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596145+0000","last_change":"2024-04-25T08:40:54.390511+0000","last_active":"2024-04-25T08:40:55.596145+0000","last_peered":"2024-04-25T08:40:55.596145+0000","last_clean":"2024-04-25T08:40:55.596145+0000","last_became_active":"2024-04-25T08:40:54.390216+0000","last_became_peered":"2024-04-25T08:40:54.390216+0000","last_unstale":"2024-04-25T08:40:55.596145+0000","last_undegraded":"2024-04-25T08:40:55.596145+0000","last_fullsized":"2024-04-25T08:40:55.596145+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:29.262872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381277+0000","last_change":"2024-04-25T08:40:54.386039+0000","last_active":"2024-04-25T08:40:55.381277+0000","last_peered":"2024-04-25T08:40:55.381277+0000","last_clean":"2024-04-25T08:40:55.381277+0000","last_became_active":"2024-04-25T08:40:54.385759+0000","last_became_peered":"2024-04-25T08:40:54.385759+0000","last_unstale":"2024-04-25T08:40:55.381277+0000","last_undegraded":"2024-04-25T08:40:55.381277+0000","last_fullsized":"2024-04-25T08:40:55.381277+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:59.173213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500612+0000","last_change":"2024-04-25T08:40:54.375254+0000","last_active":"2024-04-25T08:40:55.500612+0000","last_peered":"2024-04-25T08:40:55.500612+0000","last_clean":"2024-04-25T08:40:55.500612+0000","last_became_active":"2024-04-25T08:40:54.374962+0000","last_became_peered":"2024-04-25T08:40:54.374962+0000","last_unstale":"2024-04-25T08:40:55.500612+0000","last_undegraded":"2024-04-25T08:40:55.500612+0000","last_fullsized":"2024-04-25T08:40:55.500612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:51.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596077+0000","last_change":"2024-04-25T08:40:54.380389+0000","last_active":"2024-04-25T08:40:55.596077+0000","last_peered":"2024-04-25T08:40:55.596077+0000","last_clean":"2024-04-25T08:40:55.596077+0000","last_became_active":"2024-04-25T08:40:54.380138+0000","last_became_peered":"2024-04-25T08:40:54.380138+0000","last_unstale":"2024-04-25T08:40:55.596077+0000","last_undegraded":"2024-04-25T08:40:55.596077+0000","last_fullsized":"2024-04-25T08:40:55.596077+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:00.324716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596611+0000","last_change":"2024-04-25T08:40:54.389554+0000","last_active":"2024-04-25T08:40:55.596611+0000","last_peered":"2024-04-25T08:40:55.596611+0000","last_clean":"2024-04-25T08:40:55.596611+0000","last_became_active":"2024-04-25T08:40:54.386658+0000","last_became_peered":"2024-04-25T08:40:54.386658+0000","last_unstale":"2024-04-25T08:40:55.596611+0000","last_undegraded":"2024-04-25T08:40:55.596611+0000","last_fullsized":"2024-04-25T08:40:55.596611+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595526+0000","last_change":"2024-04-25T08:40:54.381353+0000","last_active":"2024-04-25T08:40:55.595526+0000","last_peered":"2024-04-25T08:40:55.595526+0000","last_clean":"2024-04-25T08:40:55.595526+0000","last_became_active":"2024-04-25T08:40:54.381064+0000","last_became_peered":"2024-04-25T08:40:54.381064+0000","last_unstale":"2024-04-25T08:40:55.595526+0000","last_undegraded":"2024-04-25T08:40:55.595526+0000","last_fullsized":"2024-04-25T08:40:55.595526+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:22.572428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.594940+0000","last_change":"2024-04-25T08:40:54.383332+0000","last_active":"2024-04-25T08:40:55.594940+0000","last_peered":"2024-04-25T08:40:55.594940+0000","last_clean":"2024-04-25T08:40:55.594940+0000","last_became_active":"2024-04-25T08:40:54.383004+0000","last_became_peered":"2024-04-25T08:40:54.383004+0000","last_unstale":"2024-04-25T08:40:55.594940+0000","last_undegraded":"2024-04-25T08:40:55.594940+0000","last_fullsized":"2024-04-25T08:40:55.594940+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:52.458393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596384+0000","last_change":"2024-04-25T08:40:54.380643+0000","last_active":"2024-04-25T08:40:55.596384+0000","last_peered":"2024-04-25T08:40:55.596384+0000","last_clean":"2024-04-25T08:40:55.596384+0000","last_became_active":"2024-04-25T08:40:54.379854+0000","last_became_peered":"2024-04-25T08:40:54.379854+0000","last_unstale":"2024-04-25T08:40:55.596384+0000","last_undegraded":"2024-04-25T08:40:55.596384+0000","last_fullsized":"2024-04-25T08:40:55.596384+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596288+0000","last_change":"2024-04-25T08:40:54.380286+0000","last_active":"2024-04-25T08:40:55.596288+0000","last_peered":"2024-04-25T08:40:55.596288+0000","last_clean":"2024-04-25T08:40:55.596288+0000","last_became_active":"2024-04-25T08:40:54.380011+0000","last_became_peered":"2024-04-25T08:40:54.380011+0000","last_unstale":"2024-04-25T08:40:55.596288+0000","last_undegraded":"2024-04-25T08:40:55.596288+0000","last_fullsized":"2024-04-25T08:40:55.596288+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:53.592947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596186+0000","last_change":"2024-04-25T08:40:54.388406+0000","last_active":"2024-04-25T08:40:55.596186+0000","last_peered":"2024-04-25T08:40:55.596186+0000","last_clean":"2024-04-25T08:40:55.596186+0000","last_became_active":"2024-04-25T08:40:54.387768+0000","last_became_peered":"2024-04-25T08:40:54.387768+0000","last_unstale":"2024-04-25T08:40:55.596186+0000","last_undegraded":"2024-04-25T08:40:55.596186+0000","last_fullsized":"2024-04-25T08:40:55.596186+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:50.371495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595579+0000","last_change":"2024-04-25T08:40:54.393324+0000","last_active":"2024-04-25T08:40:55.595579+0000","last_peered":"2024-04-25T08:40:55.595579+0000","last_clean":"2024-04-25T08:40:55.595579+0000","last_became_active":"2024-04-25T08:40:54.392752+0000","last_became_peered":"2024-04-25T08:40:54.392752+0000","last_unstale":"2024-04-25T08:40:55.595579+0000","last_undegraded":"2024-04-25T08:40:55.595579+0000","last_fullsized":"2024-04-25T08:40:55.595579+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:27.852314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.594982+0000","last_change":"2024-04-25T08:40:54.386869+0000","last_active":"2024-04-25T08:40:55.594982+0000","last_peered":"2024-04-25T08:40:55.594982+0000","last_clean":"2024-04-25T08:40:55.594982+0000","last_became_active":"2024-04-25T08:40:54.386577+0000","last_became_peered":"2024-04-25T08:40:54.386577+0000","last_unstale":"2024-04-25T08:40:55.594982+0000","last_undegraded":"2024-04-25T08:40:55.594982+0000","last_fullsized":"2024-04-25T08:40:55.594982+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:05:56.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500572+0000","last_change":"2024-04-25T08:40:54.387452+0000","last_active":"2024-04-25T08:40:55.500572+0000","last_peered":"2024-04-25T08:40:55.500572+0000","last_clean":"2024-04-25T08:40:55.500572+0000","last_became_active":"2024-04-25T08:40:54.387198+0000","last_became_peered":"2024-04-25T08:40:54.387198+0000","last_unstale":"2024-04-25T08:40:55.500572+0000","last_undegraded":"2024-04-25T08:40:55.500572+0000","last_fullsized":"2024-04-25T08:40:55.500572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:37.819157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595748+0000","last_change":"2024-04-25T08:40:54.378016+0000","last_active":"2024-04-25T08:40:55.595748+0000","last_peered":"2024-04-25T08:40:55.595748+0000","last_clean":"2024-04-25T08:40:55.595748+0000","last_became_active":"2024-04-25T08:40:54.377719+0000","last_became_peered":"2024-04-25T08:40:54.377719+0000","last_unstale":"2024-04-25T08:40:55.595748+0000","last_undegraded":"2024-04-25T08:40:55.595748+0000","last_fullsized":"2024-04-25T08:40:55.595748+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:11.300378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500522+0000","last_change":"2024-04-25T08:40:54.383950+0000","last_active":"2024-04-25T08:40:55.500522+0000","last_peered":"2024-04-25T08:40:55.500522+0000","last_clean":"2024-04-25T08:40:55.500522+0000","last_became_active":"2024-04-25T08:40:54.382608+0000","last_became_peered":"2024-04-25T08:40:54.382608+0000","last_unstale":"2024-04-25T08:40:55.500522+0000","last_undegraded":"2024-04-25T08:40:55.500522+0000","last_fullsized":"2024-04-25T08:40:55.500522+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:56.551387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596504+0000","last_change":"2024-04-25T08:40:54.388616+0000","last_active":"2024-04-25T08:40:55.596504+0000","last_peered":"2024-04-25T08:40:55.596504+0000","last_clean":"2024-04-25T08:40:55.596504+0000","last_became_active":"2024-04-25T08:40:54.388217+0000","last_became_peered":"2024-04-25T08:40:54.388217+0000","last_unstale":"2024-04-25T08:40:55.596504+0000","last_undegraded":"2024-04-25T08:40:55.596504+0000","last_fullsized":"2024-04-25T08:40:55.596504+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:26.474002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383924+0000","last_change":"2024-04-25T08:40:54.384260+0000","last_active":"2024-04-25T08:40:55.383924+0000","last_peered":"2024-04-25T08:40:55.383924+0000","last_clean":"2024-04-25T08:40:55.383924+0000","last_became_active":"2024-04-25T08:40:54.383734+0000","last_became_peered":"2024-04-25T08:40:54.383734+0000","last_unstale":"2024-04-25T08:40:55.383924+0000","last_undegraded":"2024-04-25T08:40:55.383924+0000","last_fullsized":"2024-04-25T08:40:55.383924+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:46.502699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383888+0000","last_change":"2024-04-25T08:40:54.380206+0000","last_active":"2024-04-25T08:40:55.383888+0000","last_peered":"2024-04-25T08:40:55.383888+0000","last_clean":"2024-04-25T08:40:55.383888+0000","last_became_active":"2024-04-25T08:40:54.379492+0000","last_became_peered":"2024-04-25T08:40:54.379492+0000","last_unstale":"2024-04-25T08:40:55.383888+0000","last_undegraded":"2024-04-25T08:40:55.383888+0000","last_fullsized":"2024-04-25T08:40:55.383888+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:40.486804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596056+0000","last_change":"2024-04-25T08:40:54.395764+0000","last_active":"2024-04-25T08:40:55.596056+0000","last_peered":"2024-04-25T08:40:55.596056+0000","last_clean":"2024-04-25T08:40:55.596056+0000","last_became_active":"2024-04-25T08:40:54.395386+0000","last_became_peered":"2024-04-25T08:40:54.395386+0000","last_unstale":"2024-04-25T08:40:55.596056+0000","last_undegraded":"2024-04-25T08:40:55.596056+0000","last_fullsized":"2024-04-25T08:40:55.596056+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:46.801443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381381+0000","last_change":"2024-04-25T08:40:54.386093+0000","last_active":"2024-04-25T08:40:55.381381+0000","last_peered":"2024-04-25T08:40:55.381381+0000","last_clean":"2024-04-25T08:40:55.381381+0000","last_became_active":"2024-04-25T08:40:54.385187+0000","last_became_peered":"2024-04-25T08:40:54.385187+0000","last_unstale":"2024-04-25T08:40:55.381381+0000","last_undegraded":"2024-04-25T08:40:55.381381+0000","last_fullsized":"2024-04-25T08:40:55.381381+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:45.539986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383508+0000","last_change":"2024-04-25T08:40:54.386486+0000","last_active":"2024-04-25T08:40:55.383508+0000","last_peered":"2024-04-25T08:40:55.383508+0000","last_clean":"2024-04-25T08:40:55.383508+0000","last_became_active":"2024-04-25T08:40:54.386209+0000","last_became_peered":"2024-04-25T08:40:54.386209+0000","last_unstale":"2024-04-25T08:40:55.383508+0000","last_undegraded":"2024-04-25T08:40:55.383508+0000","last_fullsized":"2024-04-25T08:40:55.383508+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:37.819157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382698+0000","last_change":"2024-04-25T08:40:54.388200+0000","last_active":"2024-04-25T08:40:55.382698+0000","last_peered":"2024-04-25T08:40:55.382698+0000","last_clean":"2024-04-25T08:40:55.382698+0000","last_became_active":"2024-04-25T08:40:54.387935+0000","last_became_peered":"2024-04-25T08:40:54.387935+0000","last_unstale":"2024-04-25T08:40:55.382698+0000","last_undegraded":"2024-04-25T08:40:55.382698+0000","last_fullsized":"2024-04-25T08:40:55.382698+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:56.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382793+0000","last_change":"2024-04-25T08:40:54.388543+0000","last_active":"2024-04-25T08:40:55.382793+0000","last_peered":"2024-04-25T08:40:55.382793+0000","last_clean":"2024-04-25T08:40:55.382793+0000","last_became_active":"2024-04-25T08:40:54.387411+0000","last_became_peered":"2024-04-25T08:40:54.387411+0000","last_unstale":"2024-04-25T08:40:55.382793+0000","last_undegraded":"2024-04-25T08:40:55.382793+0000","last_fullsized":"2024-04-25T08:40:55.382793+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:56.105941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.380999+0000","last_change":"2024-04-25T08:40:54.380867+0000","last_active":"2024-04-25T08:40:55.380999+0000","last_peered":"2024-04-25T08:40:55.380999+0000","last_clean":"2024-04-25T08:40:55.380999+0000","last_became_active":"2024-04-25T08:40:54.380595+0000","last_became_peered":"2024-04-25T08:40:54.380595+0000","last_unstale":"2024-04-25T08:40:55.380999+0000","last_undegraded":"2024-04-25T08:40:55.380999+0000","last_fullsized":"2024-04-25T08:40:55.380999+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:58.196801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382612+0000","last_change":"2024-04-25T08:40:54.389841+0000","last_active":"2024-04-25T08:40:55.382612+0000","last_peered":"2024-04-25T08:40:55.382612+0000","last_clean":"2024-04-25T08:40:55.382612+0000","last_became_active":"2024-04-25T08:40:54.389572+0000","last_became_peered":"2024-04-25T08:40:54.389572+0000","last_unstale":"2024-04-25T08:40:55.382612+0000","last_undegraded":"2024-04-25T08:40:55.382612+0000","last_fullsized":"2024-04-25T08:40:55.382612+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:04.970606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"131'4","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.384013+0000","last_change":"2024-04-25T08:40:53.379026+0000","last_active":"2024-04-25T08:40:55.384013+0000","last_peered":"2024-04-25T08:40:55.384013+0000","last_clean":"2024-04-25T08:40:55.384013+0000","last_became_active":"2024-04-25T08:40:53.378751+0000","last_became_peered":"2024-04-25T08:40:53.378751+0000","last_unstale":"2024-04-25T08:40:55.384013+0000","last_undegraded":"2024-04-25T08:40:55.384013+0000","last_fullsized":"2024-04-25T08:40:55.384013+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:49.556400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595109+0000","last_change":"2024-04-25T08:40:54.379170+0000","last_active":"2024-04-25T08:40:55.595109+0000","last_peered":"2024-04-25T08:40:55.595109+0000","last_clean":"2024-04-25T08:40:55.595109+0000","last_became_active":"2024-04-25T08:40:54.378843+0000","last_became_peered":"2024-04-25T08:40:54.378843+0000","last_unstale":"2024-04-25T08:40:55.595109+0000","last_undegraded":"2024-04-25T08:40:55.595109+0000","last_fullsized":"2024-04-25T08:40:55.595109+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:03.525872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596460+0000","last_change":"2024-04-25T08:40:53.382010+0000","last_active":"2024-04-25T08:40:55.596460+0000","last_peered":"2024-04-25T08:40:55.596460+0000","last_clean":"2024-04-25T08:40:55.596460+0000","last_became_active":"2024-04-25T08:40:53.381706+0000","last_became_peered":"2024-04-25T08:40:53.381706+0000","last_unstale":"2024-04-25T08:40:55.596460+0000","last_undegraded":"2024-04-25T08:40:55.596460+0000","last_fullsized":"2024-04-25T08:40:55.596460+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:39.478567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500758+0000","last_change":"2024-04-25T08:40:54.378840+0000","last_active":"2024-04-25T08:40:55.500758+0000","last_peered":"2024-04-25T08:40:55.500758+0000","last_clean":"2024-04-25T08:40:55.500758+0000","last_became_active":"2024-04-25T08:40:54.378566+0000","last_became_peered":"2024-04-25T08:40:54.378566+0000","last_unstale":"2024-04-25T08:40:55.500758+0000","last_undegraded":"2024-04-25T08:40:55.500758+0000","last_fullsized":"2024-04-25T08:40:55.500758+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:40.486804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595851+0000","last_change":"2024-04-25T08:40:53.379223+0000","last_active":"2024-04-25T08:40:55.595851+0000","last_peered":"2024-04-25T08:40:55.595851+0000","last_clean":"2024-04-25T08:40:55.595851+0000","last_became_active":"2024-04-25T08:40:53.378858+0000","last_became_peered":"2024-04-25T08:40:53.378858+0000","last_unstale":"2024-04-25T08:40:55.595851+0000","last_undegraded":"2024-04-25T08:40:55.595851+0000","last_fullsized":"2024-04-25T08:40:55.595851+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:33:47.140423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596171+0000","last_change":"2024-04-25T08:40:54.382814+0000","last_active":"2024-04-25T08:40:55.596171+0000","last_peered":"2024-04-25T08:40:55.596171+0000","last_clean":"2024-04-25T08:40:55.596171+0000","last_became_active":"2024-04-25T08:40:54.382509+0000","last_became_peered":"2024-04-25T08:40:54.382509+0000","last_unstale":"2024-04-25T08:40:55.596171+0000","last_undegraded":"2024-04-25T08:40:55.596171+0000","last_fullsized":"2024-04-25T08:40:55.596171+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:43.065290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595791+0000","last_change":"2024-04-25T08:40:53.371165+0000","last_active":"2024-04-25T08:40:55.595791+0000","last_peered":"2024-04-25T08:40:55.595791+0000","last_clean":"2024-04-25T08:40:55.595791+0000","last_became_active":"2024-04-25T08:40:53.370334+0000","last_became_peered":"2024-04-25T08:40:53.370334+0000","last_unstale":"2024-04-25T08:40:55.595791+0000","last_undegraded":"2024-04-25T08:40:55.595791+0000","last_fullsized":"2024-04-25T08:40:55.595791+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:36.817269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382917+0000","last_change":"2024-04-25T08:40:54.387624+0000","last_active":"2024-04-25T08:40:55.382917+0000","last_peered":"2024-04-25T08:40:55.382917+0000","last_clean":"2024-04-25T08:40:55.382917+0000","last_became_active":"2024-04-25T08:40:54.387359+0000","last_became_peered":"2024-04-25T08:40:54.387359+0000","last_unstale":"2024-04-25T08:40:55.382917+0000","last_undegraded":"2024-04-25T08:40:55.382917+0000","last_fullsized":"2024-04-25T08:40:55.382917+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:22.825790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383795+0000","last_change":"2024-04-25T08:40:53.373572+0000","last_active":"2024-04-25T08:40:55.383795+0000","last_peered":"2024-04-25T08:40:55.383795+0000","last_clean":"2024-04-25T08:40:55.383795+0000","last_became_active":"2024-04-25T08:40:53.373290+0000","last_became_peered":"2024-04-25T08:40:53.373290+0000","last_unstale":"2024-04-25T08:40:55.383795+0000","last_undegraded":"2024-04-25T08:40:55.383795+0000","last_fullsized":"2024-04-25T08:40:55.383795+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:40.850065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383312+0000","last_change":"2024-04-25T08:40:54.388330+0000","last_active":"2024-04-25T08:40:55.383312+0000","last_peered":"2024-04-25T08:40:55.383312+0000","last_clean":"2024-04-25T08:40:55.383312+0000","last_became_active":"2024-04-25T08:40:54.387849+0000","last_became_peered":"2024-04-25T08:40:54.387849+0000","last_unstale":"2024-04-25T08:40:55.383312+0000","last_undegraded":"2024-04-25T08:40:55.383312+0000","last_fullsized":"2024-04-25T08:40:55.383312+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:40.899090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595292+0000","last_change":"2024-04-25T08:40:53.377341+0000","last_active":"2024-04-25T08:40:55.595292+0000","last_peered":"2024-04-25T08:40:55.595292+0000","last_clean":"2024-04-25T08:40:55.595292+0000","last_became_active":"2024-04-25T08:40:53.376845+0000","last_became_peered":"2024-04-25T08:40:53.376845+0000","last_unstale":"2024-04-25T08:40:55.595292+0000","last_undegraded":"2024-04-25T08:40:55.595292+0000","last_fullsized":"2024-04-25T08:40:55.595292+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:50.295799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596326+0000","last_change":"2024-04-25T08:40:54.383699+0000","last_active":"2024-04-25T08:40:55.596326+0000","last_peered":"2024-04-25T08:40:55.596326+0000","last_clean":"2024-04-25T08:40:55.596326+0000","last_became_active":"2024-04-25T08:40:54.383421+0000","last_became_peered":"2024-04-25T08:40:54.383421+0000","last_unstale":"2024-04-25T08:40:55.596326+0000","last_undegraded":"2024-04-25T08:40:55.596326+0000","last_fullsized":"2024-04-25T08:40:55.596326+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:21.301148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595098+0000","last_change":"2024-04-25T08:40:53.376507+0000","last_active":"2024-04-25T08:40:55.595098+0000","last_peered":"2024-04-25T08:40:55.595098+0000","last_clean":"2024-04-25T08:40:55.595098+0000","last_became_active":"2024-04-25T08:40:53.376186+0000","last_became_peered":"2024-04-25T08:40:53.376186+0000","last_unstale":"2024-04-25T08:40:55.595098+0000","last_undegraded":"2024-04-25T08:40:55.595098+0000","last_fullsized":"2024-04-25T08:40:55.595098+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:50.437720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595762+0000","last_change":"2024-04-25T08:40:54.395522+0000","last_active":"2024-04-25T08:40:55.595762+0000","last_peered":"2024-04-25T08:40:55.595762+0000","last_clean":"2024-04-25T08:40:55.595762+0000","last_became_active":"2024-04-25T08:40:54.395213+0000","last_became_peered":"2024-04-25T08:40:54.395213+0000","last_unstale":"2024-04-25T08:40:55.595762+0000","last_undegraded":"2024-04-25T08:40:55.595762+0000","last_fullsized":"2024-04-25T08:40:55.595762+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:28.817951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383020+0000","last_change":"2024-04-25T08:40:53.376440+0000","last_active":"2024-04-25T08:40:55.383020+0000","last_peered":"2024-04-25T08:40:55.383020+0000","last_clean":"2024-04-25T08:40:55.383020+0000","last_became_active":"2024-04-25T08:40:53.376169+0000","last_became_peered":"2024-04-25T08:40:53.376169+0000","last_unstale":"2024-04-25T08:40:55.383020+0000","last_undegraded":"2024-04-25T08:40:55.383020+0000","last_fullsized":"2024-04-25T08:40:55.383020+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:52.489674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596333+0000","last_change":"2024-04-25T08:40:54.379010+0000","last_active":"2024-04-25T08:40:55.596333+0000","last_peered":"2024-04-25T08:40:55.596333+0000","last_clean":"2024-04-25T08:40:55.596333+0000","last_became_active":"2024-04-25T08:40:54.378699+0000","last_became_peered":"2024-04-25T08:40:54.378699+0000","last_unstale":"2024-04-25T08:40:55.596333+0000","last_undegraded":"2024-04-25T08:40:55.596333+0000","last_fullsized":"2024-04-25T08:40:55.596333+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:15.387619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500789+0000","last_change":"2024-04-25T08:40:53.380621+0000","last_active":"2024-04-25T08:40:55.500789+0000","last_peered":"2024-04-25T08:40:55.500789+0000","last_clean":"2024-04-25T08:40:55.500789+0000","last_became_active":"2024-04-25T08:40:53.380150+0000","last_became_peered":"2024-04-25T08:40:53.380150+0000","last_unstale":"2024-04-25T08:40:55.500789+0000","last_undegraded":"2024-04-25T08:40:55.500789+0000","last_fullsized":"2024-04-25T08:40:55.500789+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:56.303715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595249+0000","last_change":"2024-04-25T08:40:54.387442+0000","last_active":"2024-04-25T08:40:55.595249+0000","last_peered":"2024-04-25T08:40:55.595249+0000","last_clean":"2024-04-25T08:40:55.595249+0000","last_became_active":"2024-04-25T08:40:54.387157+0000","last_became_peered":"2024-04-25T08:40:54.387157+0000","last_unstale":"2024-04-25T08:40:55.595249+0000","last_undegraded":"2024-04-25T08:40:55.595249+0000","last_fullsized":"2024-04-25T08:40:55.595249+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:48.139944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596625+0000","last_change":"2024-04-25T08:40:53.381638+0000","last_active":"2024-04-25T08:40:55.596625+0000","last_peered":"2024-04-25T08:40:55.596625+0000","last_clean":"2024-04-25T08:40:55.596625+0000","last_became_active":"2024-04-25T08:40:53.381381+0000","last_became_peered":"2024-04-25T08:40:53.381381+0000","last_unstale":"2024-04-25T08:40:55.596625+0000","last_undegraded":"2024-04-25T08:40:55.596625+0000","last_fullsized":"2024-04-25T08:40:55.596625+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:23.975953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382845+0000","last_change":"2024-04-25T08:40:54.376953+0000","last_active":"2024-04-25T08:40:55.382845+0000","last_peered":"2024-04-25T08:40:55.382845+0000","last_clean":"2024-04-25T08:40:55.382845+0000","last_became_active":"2024-04-25T08:40:54.376686+0000","last_became_peered":"2024-04-25T08:40:54.376686+0000","last_unstale":"2024-04-25T08:40:55.382845+0000","last_undegraded":"2024-04-25T08:40:55.382845+0000","last_fullsized":"2024-04-25T08:40:55.382845+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:22.825790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"131'3","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595960+0000","last_change":"2024-04-25T08:40:53.374944+0000","last_active":"2024-04-25T08:40:55.595960+0000","last_peered":"2024-04-25T08:40:55.595960+0000","last_clean":"2024-04-25T08:40:55.595960+0000","last_became_active":"2024-04-25T08:40:53.374617+0000","last_became_peered":"2024-04-25T08:40:53.374617+0000","last_unstale":"2024-04-25T08:40:55.595960+0000","last_undegraded":"2024-04-25T08:40:55.595960+0000","last_fullsized":"2024-04-25T08:40:55.595960+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:28.254635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383572+0000","last_change":"2024-04-25T08:40:54.385830+0000","last_active":"2024-04-25T08:40:55.383572+0000","last_peered":"2024-04-25T08:40:55.383572+0000","last_clean":"2024-04-25T08:40:55.383572+0000","last_became_active":"2024-04-25T08:40:54.384994+0000","last_became_peered":"2024-04-25T08:40:54.384994+0000","last_unstale":"2024-04-25T08:40:55.383572+0000","last_undegraded":"2024-04-25T08:40:55.383572+0000","last_fullsized":"2024-04-25T08:40:55.383572+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:51.304036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"131'6","reported_seq":20,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595594+0000","last_change":"2024-04-25T08:40:53.376459+0000","last_active":"2024-04-25T08:40:55.595594+0000","last_peered":"2024-04-25T08:40:55.595594+0000","last_clean":"2024-04-25T08:40:55.595594+0000","last_became_active":"2024-04-25T08:40:53.376094+0000","last_became_peered":"2024-04-25T08:40:53.376094+0000","last_unstale":"2024-04-25T08:40:55.595594+0000","last_undegraded":"2024-04-25T08:40:55.595594+0000","last_fullsized":"2024-04-25T08:40:55.595594+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:47.131707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381354+0000","last_change":"2024-04-25T08:40:54.383836+0000","last_active":"2024-04-25T08:40:55.381354+0000","last_peered":"2024-04-25T08:40:55.381354+0000","last_clean":"2024-04-25T08:40:55.381354+0000","last_became_active":"2024-04-25T08:40:54.383486+0000","last_became_peered":"2024-04-25T08:40:54.383486+0000","last_unstale":"2024-04-25T08:40:55.381354+0000","last_undegraded":"2024-04-25T08:40:55.381354+0000","last_fullsized":"2024-04-25T08:40:55.381354+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:20.048786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596017+0000","last_change":"2024-04-25T08:40:53.382850+0000","last_active":"2024-04-25T08:40:55.596017+0000","last_peered":"2024-04-25T08:40:55.596017+0000","last_clean":"2024-04-25T08:40:55.596017+0000","last_became_active":"2024-04-25T08:40:53.381506+0000","last_became_peered":"2024-04-25T08:40:53.381506+0000","last_unstale":"2024-04-25T08:40:55.596017+0000","last_undegraded":"2024-04-25T08:40:55.596017+0000","last_fullsized":"2024-04-25T08:40:55.596017+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:58.783873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595819+0000","last_change":"2024-04-25T08:40:54.396788+0000","last_active":"2024-04-25T08:40:55.595819+0000","last_peered":"2024-04-25T08:40:55.595819+0000","last_clean":"2024-04-25T08:40:55.595819+0000","last_became_active":"2024-04-25T08:40:54.396438+0000","last_became_peered":"2024-04-25T08:40:54.396438+0000","last_unstale":"2024-04-25T08:40:55.595819+0000","last_undegraded":"2024-04-25T08:40:55.595819+0000","last_fullsized":"2024-04-25T08:40:55.595819+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:15.397916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383208+0000","last_change":"2024-04-25T08:40:53.377899+0000","last_active":"2024-04-25T08:40:55.383208+0000","last_peered":"2024-04-25T08:40:55.383208+0000","last_clean":"2024-04-25T08:40:55.383208+0000","last_became_active":"2024-04-25T08:40:53.377626+0000","last_became_peered":"2024-04-25T08:40:53.377626+0000","last_unstale":"2024-04-25T08:40:55.383208+0000","last_undegraded":"2024-04-25T08:40:55.383208+0000","last_fullsized":"2024-04-25T08:40:55.383208+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:08.112732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595426+0000","last_change":"2024-04-25T08:40:54.379061+0000","last_active":"2024-04-25T08:40:55.595426+0000","last_peered":"2024-04-25T08:40:55.595426+0000","last_clean":"2024-04-25T08:40:55.595426+0000","last_became_active":"2024-04-25T08:40:54.378771+0000","last_became_peered":"2024-04-25T08:40:54.378771+0000","last_unstale":"2024-04-25T08:40:55.595426+0000","last_undegraded":"2024-04-25T08:40:55.595426+0000","last_fullsized":"2024-04-25T08:40:55.595426+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.097762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"131'8","reported_seq":22,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596483+0000","last_change":"2024-04-25T08:40:53.378565+0000","last_active":"2024-04-25T08:40:55.596483+0000","last_peered":"2024-04-25T08:40:55.596483+0000","last_clean":"2024-04-25T08:40:55.596483+0000","last_became_active":"2024-04-25T08:40:53.378251+0000","last_became_peered":"2024-04-25T08:40:53.378251+0000","last_unstale":"2024-04-25T08:40:55.596483+0000","last_undegraded":"2024-04-25T08:40:55.596483+0000","last_fullsized":"2024-04-25T08:40:55.596483+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:03.962369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.382898+0000","last_change":"2024-04-25T08:40:54.372817+0000","last_active":"2024-04-25T08:40:55.382898+0000","last_peered":"2024-04-25T08:40:55.382898+0000","last_clean":"2024-04-25T08:40:55.382898+0000","last_became_active":"2024-04-25T08:40:54.371590+0000","last_became_peered":"2024-04-25T08:40:54.371590+0000","last_unstale":"2024-04-25T08:40:55.382898+0000","last_undegraded":"2024-04-25T08:40:55.382898+0000","last_fullsized":"2024-04-25T08:40:55.382898+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:40.899090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500851+0000","last_change":"2024-04-25T08:40:53.382667+0000","last_active":"2024-04-25T08:40:55.500851+0000","last_peered":"2024-04-25T08:40:55.500851+0000","last_clean":"2024-04-25T08:40:55.500851+0000","last_became_active":"2024-04-25T08:40:53.382436+0000","last_became_peered":"2024-04-25T08:40:53.382436+0000","last_unstale":"2024-04-25T08:40:55.500851+0000","last_undegraded":"2024-04-25T08:40:55.500851+0000","last_fullsized":"2024-04-25T08:40:55.500851+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:39.954950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595564+0000","last_change":"2024-04-25T08:40:54.397089+0000","last_active":"2024-04-25T08:40:55.595564+0000","last_peered":"2024-04-25T08:40:55.595564+0000","last_clean":"2024-04-25T08:40:55.595564+0000","last_became_active":"2024-04-25T08:40:54.396811+0000","last_became_peered":"2024-04-25T08:40:54.396811+0000","last_unstale":"2024-04-25T08:40:55.595564+0000","last_undegraded":"2024-04-25T08:40:55.595564+0000","last_fullsized":"2024-04-25T08:40:55.595564+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:32.863490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383651+0000","last_change":"2024-04-25T08:40:53.384093+0000","last_active":"2024-04-25T08:40:55.383651+0000","last_peered":"2024-04-25T08:40:55.383651+0000","last_clean":"2024-04-25T08:40:55.383651+0000","last_became_active":"2024-04-25T08:40:53.383830+0000","last_became_peered":"2024-04-25T08:40:53.383830+0000","last_unstale":"2024-04-25T08:40:55.383651+0000","last_undegraded":"2024-04-25T08:40:55.383651+0000","last_fullsized":"2024-04-25T08:40:55.383651+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:40.850065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595863+0000","last_change":"2024-04-25T08:40:54.395694+0000","last_active":"2024-04-25T08:40:55.595863+0000","last_peered":"2024-04-25T08:40:55.595863+0000","last_clean":"2024-04-25T08:40:55.595863+0000","last_became_active":"2024-04-25T08:40:54.395327+0000","last_became_peered":"2024-04-25T08:40:54.395327+0000","last_unstale":"2024-04-25T08:40:55.595863+0000","last_undegraded":"2024-04-25T08:40:55.595863+0000","last_fullsized":"2024-04-25T08:40:55.595863+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:32.015830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"131'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595793+0000","last_change":"2024-04-25T08:40:53.377620+0000","last_active":"2024-04-25T08:40:55.595793+0000","last_peered":"2024-04-25T08:40:55.595793+0000","last_clean":"2024-04-25T08:40:55.595793+0000","last_became_active":"2024-04-25T08:40:53.377340+0000","last_became_peered":"2024-04-25T08:40:53.377340+0000","last_unstale":"2024-04-25T08:40:55.595793+0000","last_undegraded":"2024-04-25T08:40:55.595793+0000","last_fullsized":"2024-04-25T08:40:55.595793+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:33.063592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381144+0000","last_change":"2024-04-25T08:40:54.382916+0000","last_active":"2024-04-25T08:40:55.381144+0000","last_peered":"2024-04-25T08:40:55.381144+0000","last_clean":"2024-04-25T08:40:55.381144+0000","last_became_active":"2024-04-25T08:40:54.382634+0000","last_became_peered":"2024-04-25T08:40:54.382634+0000","last_unstale":"2024-04-25T08:40:55.381144+0000","last_undegraded":"2024-04-25T08:40:55.381144+0000","last_fullsized":"2024-04-25T08:40:55.381144+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:28.055098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595172+0000","last_change":"2024-04-25T08:40:53.383092+0000","last_active":"2024-04-25T08:40:55.595172+0000","last_peered":"2024-04-25T08:40:55.595172+0000","last_clean":"2024-04-25T08:40:55.595172+0000","last_became_active":"2024-04-25T08:40:53.382796+0000","last_became_peered":"2024-04-25T08:40:53.382796+0000","last_unstale":"2024-04-25T08:40:55.595172+0000","last_undegraded":"2024-04-25T08:40:55.595172+0000","last_fullsized":"2024-04-25T08:40:55.595172+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:25:48.054206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595135+0000","last_change":"2024-04-25T08:40:54.375099+0000","last_active":"2024-04-25T08:40:55.595135+0000","last_peered":"2024-04-25T08:40:55.595135+0000","last_clean":"2024-04-25T08:40:55.595135+0000","last_became_active":"2024-04-25T08:40:54.374825+0000","last_became_peered":"2024-04-25T08:40:54.374825+0000","last_unstale":"2024-04-25T08:40:55.595135+0000","last_undegraded":"2024-04-25T08:40:55.595135+0000","last_fullsized":"2024-04-25T08:40:55.595135+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:26.474002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383840+0000","last_change":"2024-04-25T08:40:53.372724+0000","last_active":"2024-04-25T08:40:55.383840+0000","last_peered":"2024-04-25T08:40:55.383840+0000","last_clean":"2024-04-25T08:40:55.383840+0000","last_became_active":"2024-04-25T08:40:53.372440+0000","last_became_peered":"2024-04-25T08:40:53.372440+0000","last_unstale":"2024-04-25T08:40:55.383840+0000","last_undegraded":"2024-04-25T08:40:55.383840+0000","last_fullsized":"2024-04-25T08:40:55.383840+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:46.131350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595550+0000","last_change":"2024-04-25T08:40:54.382282+0000","last_active":"2024-04-25T08:40:55.595550+0000","last_peered":"2024-04-25T08:40:55.595550+0000","last_clean":"2024-04-25T08:40:55.595550+0000","last_became_active":"2024-04-25T08:40:54.381998+0000","last_became_peered":"2024-04-25T08:40:54.381998+0000","last_unstale":"2024-04-25T08:40:55.595550+0000","last_undegraded":"2024-04-25T08:40:55.595550+0000","last_fullsized":"2024-04-25T08:40:55.595550+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:03.836257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595616+0000","last_change":"2024-04-25T08:40:53.380988+0000","last_active":"2024-04-25T08:40:55.595616+0000","last_peered":"2024-04-25T08:40:55.595616+0000","last_clean":"2024-04-25T08:40:55.595616+0000","last_became_active":"2024-04-25T08:40:53.380703+0000","last_became_peered":"2024-04-25T08:40:53.380703+0000","last_unstale":"2024-04-25T08:40:55.595616+0000","last_undegraded":"2024-04-25T08:40:55.595616+0000","last_fullsized":"2024-04-25T08:40:55.595616+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:52.326545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596468+0000","last_change":"2024-04-25T08:40:54.374363+0000","last_active":"2024-04-25T08:40:55.596468+0000","last_peered":"2024-04-25T08:40:55.596468+0000","last_clean":"2024-04-25T08:40:55.596468+0000","last_became_active":"2024-04-25T08:40:54.374043+0000","last_became_peered":"2024-04-25T08:40:54.374043+0000","last_unstale":"2024-04-25T08:40:55.596468+0000","last_undegraded":"2024-04-25T08:40:55.596468+0000","last_fullsized":"2024-04-25T08:40:55.596468+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:29.707422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595593+0000","last_change":"2024-04-25T08:40:53.374921+0000","last_active":"2024-04-25T08:40:55.595593+0000","last_peered":"2024-04-25T08:40:55.595593+0000","last_clean":"2024-04-25T08:40:55.595593+0000","last_became_active":"2024-04-25T08:40:53.374453+0000","last_became_peered":"2024-04-25T08:40:53.374453+0000","last_unstale":"2024-04-25T08:40:55.595593+0000","last_undegraded":"2024-04-25T08:40:55.595593+0000","last_fullsized":"2024-04-25T08:40:55.595593+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:52.584710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500809+0000","last_change":"2024-04-25T08:40:54.378104+0000","last_active":"2024-04-25T08:40:55.500809+0000","last_peered":"2024-04-25T08:40:55.500809+0000","last_clean":"2024-04-25T08:40:55.500809+0000","last_became_active":"2024-04-25T08:40:54.377753+0000","last_became_peered":"2024-04-25T08:40:54.377753+0000","last_unstale":"2024-04-25T08:40:55.500809+0000","last_undegraded":"2024-04-25T08:40:55.500809+0000","last_fullsized":"2024-04-25T08:40:55.500809+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:25.693049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596583+0000","last_change":"2024-04-25T08:40:53.376843+0000","last_active":"2024-04-25T08:40:55.596583+0000","last_peered":"2024-04-25T08:40:55.596583+0000","last_clean":"2024-04-25T08:40:55.596583+0000","last_became_active":"2024-04-25T08:40:53.376238+0000","last_became_peered":"2024-04-25T08:40:53.376238+0000","last_unstale":"2024-04-25T08:40:55.596583+0000","last_undegraded":"2024-04-25T08:40:55.596583+0000","last_fullsized":"2024-04-25T08:40:55.596583+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:55.097704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.594957+0000","last_change":"2024-04-25T08:40:54.394401+0000","last_active":"2024-04-25T08:40:55.594957+0000","last_peered":"2024-04-25T08:40:55.594957+0000","last_clean":"2024-04-25T08:40:55.594957+0000","last_became_active":"2024-04-25T08:40:54.393963+0000","last_became_peered":"2024-04-25T08:40:54.393963+0000","last_unstale":"2024-04-25T08:40:55.594957+0000","last_undegraded":"2024-04-25T08:40:55.594957+0000","last_fullsized":"2024-04-25T08:40:55.594957+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:53.592947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"131'2","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.596070+0000","last_change":"2024-04-25T08:40:53.383569+0000","last_active":"2024-04-25T08:40:55.596070+0000","last_peered":"2024-04-25T08:40:55.596070+0000","last_clean":"2024-04-25T08:40:55.596070+0000","last_became_active":"2024-04-25T08:40:53.383276+0000","last_became_peered":"2024-04-25T08:40:53.383276+0000","last_unstale":"2024-04-25T08:40:55.596070+0000","last_undegraded":"2024-04-25T08:40:55.596070+0000","last_fullsized":"2024-04-25T08:40:55.596070+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:38.170316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383108+0000","last_change":"2024-04-25T08:40:54.384630+0000","last_active":"2024-04-25T08:40:55.383108+0000","last_peered":"2024-04-25T08:40:55.383108+0000","last_clean":"2024-04-25T08:40:55.383108+0000","last_became_active":"2024-04-25T08:40:54.384077+0000","last_became_peered":"2024-04-25T08:40:54.384077+0000","last_unstale":"2024-04-25T08:40:55.383108+0000","last_undegraded":"2024-04-25T08:40:55.383108+0000","last_fullsized":"2024-04-25T08:40:55.383108+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:56.551387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"132'1","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:56.368767+0000","last_change":"2024-04-25T08:40:53.378186+0000","last_active":"2024-04-25T08:40:56.368767+0000","last_peered":"2024-04-25T08:40:56.368767+0000","last_clean":"2024-04-25T08:40:56.368767+0000","last_became_active":"2024-04-25T08:40:53.377880+0000","last_became_peered":"2024-04-25T08:40:53.377880+0000","last_unstale":"2024-04-25T08:40:56.368767+0000","last_undegraded":"2024-04-25T08:40:56.368767+0000","last_fullsized":"2024-04-25T08:40:56.368767+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:47.172417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595421+0000","last_change":"2024-04-25T08:40:54.393999+0000","last_active":"2024-04-25T08:40:55.595421+0000","last_peered":"2024-04-25T08:40:55.595421+0000","last_clean":"2024-04-25T08:40:55.595421+0000","last_became_active":"2024-04-25T08:40:54.393727+0000","last_became_peered":"2024-04-25T08:40:54.393727+0000","last_unstale":"2024-04-25T08:40:55.595421+0000","last_undegraded":"2024-04-25T08:40:55.595421+0000","last_fullsized":"2024-04-25T08:40:55.595421+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:42:26.065517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383430+0000","last_change":"2024-04-25T08:40:53.381261+0000","last_active":"2024-04-25T08:40:55.383430+0000","last_peered":"2024-04-25T08:40:55.383430+0000","last_clean":"2024-04-25T08:40:55.383430+0000","last_became_active":"2024-04-25T08:40:53.380920+0000","last_became_peered":"2024-04-25T08:40:53.380920+0000","last_unstale":"2024-04-25T08:40:55.383430+0000","last_undegraded":"2024-04-25T08:40:55.383430+0000","last_fullsized":"2024-04-25T08:40:55.383430+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:39.954950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.381178+0000","last_change":"2024-04-25T08:40:54.386615+0000","last_active":"2024-04-25T08:40:55.381178+0000","last_peered":"2024-04-25T08:40:55.381178+0000","last_clean":"2024-04-25T08:40:55.381178+0000","last_became_active":"2024-04-25T08:40:54.386359+0000","last_became_peered":"2024-04-25T08:40:54.386359+0000","last_unstale":"2024-04-25T08:40:55.381178+0000","last_undegraded":"2024-04-25T08:40:55.381178+0000","last_fullsized":"2024-04-25T08:40:55.381178+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:11.093642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595669+0000","last_change":"2024-04-25T08:40:53.378763+0000","last_active":"2024-04-25T08:40:55.595669+0000","last_peered":"2024-04-25T08:40:55.595669+0000","last_clean":"2024-04-25T08:40:55.595669+0000","last_became_active":"2024-04-25T08:40:53.378453+0000","last_became_peered":"2024-04-25T08:40:53.378453+0000","last_unstale":"2024-04-25T08:40:55.595669+0000","last_undegraded":"2024-04-25T08:40:55.595669+0000","last_fullsized":"2024-04-25T08:40:55.595669+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:57.408929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595573+0000","last_change":"2024-04-25T08:40:54.387431+0000","last_active":"2024-04-25T08:40:55.595573+0000","last_peered":"2024-04-25T08:40:55.595573+0000","last_clean":"2024-04-25T08:40:55.595573+0000","last_became_active":"2024-04-25T08:40:54.387149+0000","last_became_peered":"2024-04-25T08:40:54.387149+0000","last_unstale":"2024-04-25T08:40:55.595573+0000","last_undegraded":"2024-04-25T08:40:55.595573+0000","last_fullsized":"2024-04-25T08:40:55.595573+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:49.064739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.500551+0000","last_change":"2024-04-25T08:40:53.385220+0000","last_active":"2024-04-25T08:40:55.500551+0000","last_peered":"2024-04-25T08:40:55.500551+0000","last_clean":"2024-04-25T08:40:55.500551+0000","last_became_active":"2024-04-25T08:40:53.384943+0000","last_became_peered":"2024-04-25T08:40:53.384943+0000","last_unstale":"2024-04-25T08:40:55.500551+0000","last_undegraded":"2024-04-25T08:40:55.500551+0000","last_fullsized":"2024-04-25T08:40:55.500551+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:16.391337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595687+0000","last_change":"2024-04-25T08:40:54.388970+0000","last_active":"2024-04-25T08:40:55.595687+0000","last_peered":"2024-04-25T08:40:55.595687+0000","last_clean":"2024-04-25T08:40:55.595687+0000","last_became_active":"2024-04-25T08:40:54.388677+0000","last_became_peered":"2024-04-25T08:40:54.388677+0000","last_unstale":"2024-04-25T08:40:55.595687+0000","last_undegraded":"2024-04-25T08:40:55.595687+0000","last_fullsized":"2024-04-25T08:40:55.595687+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:37.825506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595837+0000","last_change":"2024-04-25T08:40:53.369444+0000","last_active":"2024-04-25T08:40:55.595837+0000","last_peered":"2024-04-25T08:40:55.595837+0000","last_clean":"2024-04-25T08:40:55.595837+0000","last_became_active":"2024-04-25T08:40:53.368467+0000","last_became_peered":"2024-04-25T08:40:53.368467+0000","last_unstale":"2024-04-25T08:40:55.595837+0000","last_undegraded":"2024-04-25T08:40:55.595837+0000","last_fullsized":"2024-04-25T08:40:55.595837+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:55.432774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595830+0000","last_change":"2024-04-25T08:40:54.381075+0000","last_active":"2024-04-25T08:40:55.595830+0000","last_peered":"2024-04-25T08:40:55.595830+0000","last_clean":"2024-04-25T08:40:55.595830+0000","last_became_active":"2024-04-25T08:40:54.380766+0000","last_became_peered":"2024-04-25T08:40:54.380766+0000","last_unstale":"2024-04-25T08:40:55.595830+0000","last_undegraded":"2024-04-25T08:40:55.595830+0000","last_fullsized":"2024-04-25T08:40:55.595830+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:56.441011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595339+0000","last_change":"2024-04-25T08:40:53.378417+0000","last_active":"2024-04-25T08:40:55.595339+0000","last_peered":"2024-04-25T08:40:55.595339+0000","last_clean":"2024-04-25T08:40:55.595339+0000","last_became_active":"2024-04-25T08:40:53.378139+0000","last_became_peered":"2024-04-25T08:40:53.378139+0000","last_unstale":"2024-04-25T08:40:55.595339+0000","last_undegraded":"2024-04-25T08:40:55.595339+0000","last_fullsized":"2024-04-25T08:40:55.595339+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.250586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595801+0000","last_change":"2024-04-25T08:40:54.395804+0000","last_active":"2024-04-25T08:40:55.595801+0000","last_peered":"2024-04-25T08:40:55.595801+0000","last_clean":"2024-04-25T08:40:55.595801+0000","last_became_active":"2024-04-25T08:40:54.395439+0000","last_became_peered":"2024-04-25T08:40:54.395439+0000","last_unstale":"2024-04-25T08:40:55.595801+0000","last_undegraded":"2024-04-25T08:40:55.595801+0000","last_fullsized":"2024-04-25T08:40:55.595801+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:02.335793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595066+0000","last_change":"2024-04-25T08:40:53.378352+0000","last_active":"2024-04-25T08:40:55.595066+0000","last_peered":"2024-04-25T08:40:55.595066+0000","last_clean":"2024-04-25T08:40:55.595066+0000","last_became_active":"2024-04-25T08:40:53.378028+0000","last_became_peered":"2024-04-25T08:40:53.378028+0000","last_unstale":"2024-04-25T08:40:55.595066+0000","last_undegraded":"2024-04-25T08:40:55.595066+0000","last_fullsized":"2024-04-25T08:40:55.595066+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:36.810920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":12,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.383951+0000","last_change":"2024-04-25T08:40:54.378981+0000","last_active":"2024-04-25T08:40:55.383951+0000","last_peered":"2024-04-25T08:40:55.383951+0000","last_clean":"2024-04-25T08:40:55.383951+0000","last_became_active":"2024-04-25T08:40:54.378496+0000","last_became_peered":"2024-04-25T08:40:54.378496+0000","last_unstale":"2024-04-25T08:40:55.383951+0000","last_undegraded":"2024-04-25T08:40:55.383951+0000","last_fullsized":"2024-04-25T08:40:55.383951+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:53.353507+0000","last_clean_scrub_stamp":"2024-04-25T08:40:53.353507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:25.693049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":14,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595705+0000","last_change":"2024-04-25T08:40:53.377203+0000","last_active":"2024-04-25T08:40:55.595705+0000","last_peered":"2024-04-25T08:40:55.595705+0000","last_clean":"2024-04-25T08:40:55.595705+0000","last_became_active":"2024-04-25T08:40:53.376892+0000","last_became_peered":"2024-04-25T08:40:53.376892+0000","last_unstale":"2024-04-25T08:40:55.595705+0000","last_undegraded":"2024-04-25T08:40:55.595705+0000","last_fullsized":"2024-04-25T08:40:55.595705+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:40:52.345270+0000","last_clean_scrub_stamp":"2024-04-25T08:40:52.345270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:48.056502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":380,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-04-25T08:40:55.595216+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:40:55.595216+0000","last_peered":"2024-04-25T08:40:55.595216+0000","last_clean":"2024-04-25T08:40:55.595216+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:40:55.595216+0000","last_undegraded":"2024-04-25T08:40:55.595216+0000","last_fullsized":"2024-04-25T08:40:55.595216+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65328,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306512,"statfs":{"total":96636764160,"available":96569868288,"internally_reserved":0,"allocated":2260992,"data_stored":1363378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59172,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312668,"statfs":{"total":96636764160,"available":96576172032,"internally_reserved":0,"allocated":1658880,"data_stored":771980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55012,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316828,"statfs":{"total":96636764160,"available":96580431872,"internally_reserved":0,"allocated":1654784,"data_stored":771330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68892,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302948,"statfs":{"total":96636764160,"available":96566218752,"internally_reserved":0,"allocated":1650688,"data_stored":771308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73068,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298772,"statfs":{"total":96636764160,"available":96561942528,"internally_reserved":0,"allocated":1667072,"data_stored":772990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59176,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312664,"statfs":{"total":96636764160,"available":96576167936,"internally_reserved":0,"allocated":1658880,"data_stored":771924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59168,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312672,"statfs":{"total":96636764160,"available":96576176128,"internally_reserved":0,"allocated":1654784,"data_stored":771378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69416,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302424,"statfs":{"total":96636764160,"available":96565682176,"internally_reserved":0,"allocated":2252800,"data_stored":1361732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:41:02.146 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:41:02.146 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:41:02.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.297+0000 7f739005d640 1 Processor -- start 2024-04-25T08:41:02.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.298+0000 7f739005d640 1 -- start start 2024-04-25T08:41:02.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.298+0000 7f739005d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f738815d980 0x7f73880d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.298+0000 7f739005d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73880b2580 0x7f73880d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.298+0000 7f739005d640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f73880d0d20 0x7f73880d2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.298+0000 7f739005d640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73880cec90 con 0x7f738815d980 2024-04-25T08:41:02.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.298+0000 7f739005d640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73880d2450 con 0x7f73880d0d20 2024-04-25T08:41:02.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.298+0000 7f739005d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73880d2730 con 0x7f73880b2580 2024-04-25T08:41:02.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.299+0000 7f738ddf9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f738815d980 0x7f73880d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.299+0000 7f738ddf9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f738815d980 0x7f73880d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47744/0 (socket says 172.21.15.38:47744) 2024-04-25T08:41:02.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.299+0000 7f738ddf9640 1 -- 172.21.15.38:0/1980514686 learned_addr learned my addr 172.21.15.38:0/1980514686 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:02.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.299+0000 7f738d5f8640 1 --2- 172.21.15.38:0/1980514686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73880b2580 0x7f73880d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.299+0000 7f738e5fa640 1 --2- 172.21.15.38:0/1980514686 >> v2:172.21.15.107:3300/0 conn(0x7f73880d0d20 0x7f73880d2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.299+0000 7f738ddf9640 1 -- 172.21.15.38:0/1980514686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73880b2580 msgr2=0x7f73880d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:02.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.299+0000 7f738ddf9640 1 --2- 172.21.15.38:0/1980514686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73880b2580 0x7f73880d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.299+0000 7f738ddf9640 1 -- 172.21.15.38:0/1980514686 >> v2:172.21.15.107:3300/0 conn(0x7f73880d0d20 msgr2=0x7f73880d2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:02.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.299+0000 7f738ddf9640 1 --2- 172.21.15.38:0/1980514686 >> v2:172.21.15.107:3300/0 conn(0x7f73880d0d20 0x7f73880d2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.299+0000 7f738ddf9640 1 -- 172.21.15.38:0/1980514686 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73880d2a10 con 0x7f738815d980 2024-04-25T08:41:02.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.299+0000 7f738d5f8640 1 --2- 172.21.15.38:0/1980514686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73880b2580 0x7f73880d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:02.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.299+0000 7f738e5fa640 1 --2- 172.21.15.38:0/1980514686 >> v2:172.21.15.107:3300/0 conn(0x7f73880d0d20 0x7f73880d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:02.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.300+0000 7f738ddf9640 1 --2- 172.21.15.38:0/1980514686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f738815d980 0x7f73880d02a0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f7388069120 tx=0x7f738402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd48b2462a54ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:02.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.300+0000 7f738cdf7640 1 -- 172.21.15.38:0/1980514686 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384037070 con 0x7f738815d980 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.300+0000 7f738cdf7640 1 -- 172.21.15.38:0/1980514686 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7384032440 con 0x7f738815d980 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.300+0000 7f738cdf7640 1 -- 172.21.15.38:0/1980514686 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384032810 con 0x7f738815d980 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.300+0000 7f739005d640 1 -- 172.21.15.38:0/1980514686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f738815d980 msgr2=0x7f73880d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.300+0000 7f739005d640 1 --2- 172.21.15.38:0/1980514686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f738815d980 0x7f73880d02a0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f7388069120 tx=0x7f738402a140 comp rx=0 tx=0).stop 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.300+0000 7f739005d640 1 -- 172.21.15.38:0/1980514686 shutdown_connections 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.300+0000 7f739005d640 1 --2- 172.21.15.38:0/1980514686 >> v2:172.21.15.107:3300/0 conn(0x7f73880d0d20 0x7f73880d2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.300+0000 7f739005d640 1 --2- 172.21.15.38:0/1980514686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73880b2580 0x7f73880d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.300+0000 7f739005d640 1 --2- 172.21.15.38:0/1980514686 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f738815d980 0x7f73880d02a0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.300+0000 7f739005d640 1 -- 172.21.15.38:0/1980514686 >> 172.21.15.38:0/1980514686 conn(0x7f738806bed0 msgr2=0x7f738806c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.300+0000 7f739005d640 1 -- 172.21.15.38:0/1980514686 shutdown_connections 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.301+0000 7f739005d640 1 -- 172.21.15.38:0/1980514686 wait complete. 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.302+0000 7f739005d640 1 Processor -- start 2024-04-25T08:41:02.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.302+0000 7f739005d640 1 -- start start 2024-04-25T08:41:02.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.302+0000 7f739005d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73880b2580 0x7f73880cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.302+0000 7f739005d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73880d0d20 0x7f73881e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.302+0000 7f739005d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738815d980 0x7f738814c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.302+0000 7f739005d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73880cea50 con 0x7f73880b2580 2024-04-25T08:41:02.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.302+0000 7f739005d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73880ce8d0 con 0x7f738815d980 2024-04-25T08:41:02.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.302+0000 7f739005d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73880d3470 con 0x7f73880d0d20 2024-04-25T08:41:02.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.302+0000 7f738ddf9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73880b2580 0x7f73880cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f738ddf9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73880b2580 0x7f73880cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47750/0 (socket says 172.21.15.38:47750) 2024-04-25T08:41:02.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f738ddf9640 1 -- 172.21.15.38:0/199598367 learned_addr learned my addr 172.21.15.38:0/199598367 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f738e5fa640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738815d980 0x7f738814c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f738d5f8640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73880d0d20 0x7f73881e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f738ddf9640 1 -- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73880d0d20 msgr2=0x7f73881e5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f738ddf9640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73880d0d20 0x7f73881e5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f738ddf9640 1 -- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738815d980 msgr2=0x7f738814c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f738ddf9640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738815d980 0x7f738814c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f738ddf9640 1 -- 172.21.15.38:0/199598367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73880b4ba0 con 0x7f73880b2580 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f738ddf9640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73880b2580 0x7f73880cc270 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f738402a180 tx=0x7f73840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f737e7fc640 1 -- 172.21.15.38:0/199598367 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384041020 con 0x7f73880b2580 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f737e7fc640 1 -- 172.21.15.38:0/199598367 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73840306c0 con 0x7f73880b2580 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f739005d640 1 -- 172.21.15.38:0/199598367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73880b4e80 con 0x7f73880b2580 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f737e7fc640 1 -- 172.21.15.38:0/199598367 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7384037040 con 0x7f73880b2580 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.303+0000 7f739005d640 1 -- 172.21.15.38:0/199598367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73880b5720 con 0x7f73880b2580 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.304+0000 7f739005d640 1 -- 172.21.15.38:0/199598367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7388068e80 con 0x7f73880b2580 2024-04-25T08:41:02.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.305+0000 7f737e7fc640 1 -- 172.21.15.38:0/199598367 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7384044020 con 0x7f73880b2580 2024-04-25T08:41:02.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.305+0000 7f737e7fc640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f736c06b4d0 0x7f736c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.305+0000 7f737e7fc640 1 -- 172.21.15.38:0/199598367 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f73840aab40 con 0x7f73880b2580 2024-04-25T08:41:02.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.305+0000 7f738d5f8640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f736c06b4d0 0x7f736c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.306+0000 7f738d5f8640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f736c06b4d0 0x7f736c06d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7378006fd0 tx=0x7f7378008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:02.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.309+0000 7f737e7fc640 1 -- 172.21.15.38:0/199598367 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f738404b030 con 0x7f73880b2580 2024-04-25T08:41:02.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.563+0000 7f739005d640 1 -- 172.21.15.38:0/199598367 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f73880ce0f0 con 0x7f73880b2580 2024-04-25T08:41:02.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.565+0000 7f737e7fc640 1 -- 172.21.15.38:0/199598367 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7384079830 con 0x7f73880b2580 2024-04-25T08:41:02.558 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-04-25T08:41:02.558 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw== 2024-04-25T08:41:02.558 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-04-25T08:41:02.558 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-04-25T08:41:02.559 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:41:02.559 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-04-25T08:41:02.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.568+0000 7f739005d640 1 -- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f736c06b4d0 msgr2=0x7f736c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.568+0000 7f739005d640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f736c06b4d0 0x7f736c06d990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f7378006fd0 tx=0x7f7378008040 comp rx=0 tx=0).stop 2024-04-25T08:41:02.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.568+0000 7f739005d640 1 -- 172.21.15.38:0/199598367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73880b2580 msgr2=0x7f73880cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.568+0000 7f739005d640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73880b2580 0x7f73880cc270 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f738402a180 tx=0x7f73840046c0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.568+0000 7f739005d640 1 -- 172.21.15.38:0/199598367 shutdown_connections 2024-04-25T08:41:02.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.568+0000 7f739005d640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f738815d980 0x7f738814c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.568+0000 7f739005d640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f736c06b4d0 0x7f736c06d990 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.568+0000 7f739005d640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f73880d0d20 0x7f73881e5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.568+0000 7f739005d640 1 --2- 172.21.15.38:0/199598367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f73880b2580 0x7f73880cc270 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.568+0000 7f739005d640 1 -- 172.21.15.38:0/199598367 >> 172.21.15.38:0/199598367 conn(0x7f738806bed0 msgr2=0x7f73880d38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:02.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.568+0000 7f739005d640 1 -- 172.21.15.38:0/199598367 shutdown_connections 2024-04-25T08:41:02.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.568+0000 7f739005d640 1 -- 172.21.15.38:0/199598367 wait complete. 2024-04-25T08:41:02.572 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:02.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.679+0000 7fed1bdb7640 1 Processor -- start 2024-04-25T08:41:02.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.679+0000 7fed1bdb7640 1 -- start start 2024-04-25T08:41:02.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.680+0000 7fed1bdb7640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fed14143f40 0x7fed140b4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.680+0000 7fed1bdb7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed140b4e60 0x7fed140cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.680+0000 7fed1bdb7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed140ce360 con 0x7fed140cdb70 2024-04-25T08:41:02.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.680+0000 7fed1bdb7640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed140b52c0 con 0x7fed14143f40 2024-04-25T08:41:02.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.680+0000 7fed1bdb7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed140ce640 con 0x7fed140b4e60 2024-04-25T08:41:02.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.681+0000 7fed19b53640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fed14143f40 0x7fed140b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.681+0000 7fed1a354640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7fed140cdb70 0x7fed1415e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:36950/0 (socket says 172.21.15.38:36950) 2024-04-25T08:41:02.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.681+0000 7fed19352640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed140b4e60 0x7fed140cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.681+0000 7fed1a354640 1 -- 172.21.15.38:0/2689606251 learned_addr learned my addr 172.21.15.38:0/2689606251 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.681+0000 7fed18b51640 1 -- 172.21.15.38:0/2689606251 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654573438 0 0) 0x7fed140ce360 con 0x7fed140cdb70 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.681+0000 7fed18b51640 1 -- 172.21.15.38:0/2689606251 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed00003520 con 0x7fed140cdb70 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.681+0000 7fed19b53640 1 -- 172.21.15.38:0/2689606251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed140b4e60 msgr2=0x7fed140cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.681+0000 7fed19b53640 1 --2- 172.21.15.38:0/2689606251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed140b4e60 0x7fed140cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.681+0000 7fed19b53640 1 -- 172.21.15.38:0/2689606251 >> v1:172.21.15.107:6789/0 conn(0x7fed140cdb70 legacy=0x7fed1415e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.681+0000 7fed19b53640 1 -- 172.21.15.38:0/2689606251 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed140ce920 con 0x7fed14143f40 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.681+0000 7fed19352640 1 --2- 172.21.15.38:0/2689606251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed140b4e60 0x7fed140cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.682+0000 7fed19b53640 1 --2- 172.21.15.38:0/2689606251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed14143f40 0x7fed140b4920 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fed100097a0 tx=0x7fed1002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c048828c2fc6b631 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.682+0000 7fed18b51640 1 -- 172.21.15.38:0/2689606251 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10037070 con 0x7fed14143f40 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.682+0000 7fed18b51640 1 -- 172.21.15.38:0/2689606251 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed10032440 con 0x7fed14143f40 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.682+0000 7fed18b51640 1 -- 172.21.15.38:0/2689606251 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed100327f0 con 0x7fed14143f40 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.683+0000 7fed1bdb7640 1 -- 172.21.15.38:0/2689606251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed14143f40 msgr2=0x7fed140b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.683+0000 7fed1bdb7640 1 --2- 172.21.15.38:0/2689606251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed14143f40 0x7fed140b4920 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fed100097a0 tx=0x7fed1002a180 comp rx=0 tx=0).stop 2024-04-25T08:41:02.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.683+0000 7fed1bdb7640 1 -- 172.21.15.38:0/2689606251 shutdown_connections 2024-04-25T08:41:02.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.683+0000 7fed1bdb7640 1 --2- 172.21.15.38:0/2689606251 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed140b4e60 0x7fed140cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.683+0000 7fed1bdb7640 1 --2- 172.21.15.38:0/2689606251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed14143f40 0x7fed140b4920 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.683+0000 7fed1bdb7640 1 -- 172.21.15.38:0/2689606251 >> 172.21.15.38:0/2689606251 conn(0x7fed1406b210 msgr2=0x7fed1406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:02.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.683+0000 7fed1bdb7640 1 -- 172.21.15.38:0/2689606251 shutdown_connections 2024-04-25T08:41:02.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.683+0000 7fed1bdb7640 1 -- 172.21.15.38:0/2689606251 wait complete. 2024-04-25T08:41:02.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.684+0000 7fed1bdb7640 1 Processor -- start 2024-04-25T08:41:02.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.684+0000 7fed1bdb7640 1 -- start start 2024-04-25T08:41:02.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed1bdb7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed140b4e60 0x7fed141363f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed1bdb7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed140cdb70 0x7fed140711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed1bdb7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed14143f40 0x7fed14071720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed1bdb7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed140cdf80 con 0x7fed140cdb70 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed1bdb7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed140ceee0 con 0x7fed14143f40 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed1bdb7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed140ce100 con 0x7fed140b4e60 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed19352640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed140cdb70 0x7fed140711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed19352640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed140cdb70 0x7fed140711e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47760/0 (socket says 172.21.15.38:47760) 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed19352640 1 -- 172.21.15.38:0/1235647403 learned_addr learned my addr 172.21.15.38:0/1235647403 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed19b53640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed140b4e60 0x7fed141363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed1a354640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed14143f40 0x7fed14071720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed19352640 1 -- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed140b4e60 msgr2=0x7fed141363f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed19352640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed140b4e60 0x7fed141363f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed19352640 1 -- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed14143f40 msgr2=0x7fed14071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed19352640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed14143f40 0x7fed14071720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed19352640 1 -- 172.21.15.38:0/1235647403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1413c890 con 0x7fed140cdb70 2024-04-25T08:41:02.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed1a354640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed14143f40 0x7fed14071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:02.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed19352640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed140cdb70 0x7fed140711e0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fed0400ebe0 tx=0x7fed0400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:02.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1235647403 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04017070 con 0x7fed140cdb70 2024-04-25T08:41:02.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1235647403 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0400cc30 con 0x7fed140cdb70 2024-04-25T08:41:02.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed1bdb7640 1 -- 172.21.15.38:0/1235647403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1413cb70 con 0x7fed140cdb70 2024-04-25T08:41:02.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1235647403 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04013470 con 0x7fed140cdb70 2024-04-25T08:41:02.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.685+0000 7fed1bdb7640 1 -- 172.21.15.38:0/1235647403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1413d130 con 0x7fed140cdb70 2024-04-25T08:41:02.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.687+0000 7fed1bdb7640 1 -- 172.21.15.38:0/1235647403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed14068e80 con 0x7fed140cdb70 2024-04-25T08:41:02.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.687+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1235647403 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed0401a070 con 0x7fed140cdb70 2024-04-25T08:41:02.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.687+0000 7fed0a7fc640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecf806b4d0 0x7fecf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:02.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.688+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1235647403 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fed0408a2d0 con 0x7fed140cdb70 2024-04-25T08:41:02.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.688+0000 7fed19b53640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecf806b4d0 0x7fecf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:02.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.688+0000 7fed19b53640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecf806b4d0 0x7fecf806d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fed10034000 tx=0x7fed1002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:02.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.692+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1235647403 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed04058fc0 con 0x7fed140cdb70 2024-04-25T08:41:02.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.907+0000 7fed1bdb7640 1 -- 172.21.15.38:0/1235647403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed140b33f0 con 0x7fed140cdb70 2024-04-25T08:41:02.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.908+0000 7fed0a7fc640 1 -- 172.21.15.38:0/1235647403 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fed040588c0 con 0x7fed140cdb70 2024-04-25T08:41:02.901 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:02.901 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:40:55.730538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/1312063561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1312063561},{"type":"v1","addr":"172.21.15.38:6835","nonce":1312063561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:02.901 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 77 2024-04-25T08:41:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.910+0000 7fed1bdb7640 1 -- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecf806b4d0 msgr2=0x7fecf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.910+0000 7fed1bdb7640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecf806b4d0 0x7fecf806d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fed10034000 tx=0x7fed1002e000 comp rx=0 tx=0).stop 2024-04-25T08:41:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.910+0000 7fed1bdb7640 1 -- 172.21.15.38:0/1235647403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed140cdb70 msgr2=0x7fed140711e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.910+0000 7fed1bdb7640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed140cdb70 0x7fed140711e0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fed0400ebe0 tx=0x7fed0400c520 comp rx=0 tx=0).stop 2024-04-25T08:41:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.910+0000 7fed1bdb7640 1 -- 172.21.15.38:0/1235647403 shutdown_connections 2024-04-25T08:41:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.910+0000 7fed1bdb7640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fed14143f40 0x7fed14071720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.910+0000 7fed1bdb7640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed140cdb70 0x7fed140711e0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.910+0000 7fed1bdb7640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fecf806b4d0 0x7fecf806d990 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.910+0000 7fed1bdb7640 1 --2- 172.21.15.38:0/1235647403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fed140b4e60 0x7fed141363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.910+0000 7fed1bdb7640 1 -- 172.21.15.38:0/1235647403 >> 172.21.15.38:0/1235647403 conn(0x7fed1406b210 msgr2=0x7fed14151930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.910+0000 7fed1bdb7640 1 -- 172.21.15.38:0/1235647403 shutdown_connections 2024-04-25T08:41:02.903 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:02.910+0000 7fed1bdb7640 1 -- 172.21.15.38:0/1235647403 wait complete. 2024-04-25T08:41:03.316 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:40:53.732378+0000', 'modified': '2024-04-25T08:40:55.730538+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9489}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9489': {'gid': 9489, 'name': 'a', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6835/1312063561', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 1312063561}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 1312063561}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:41:03.316 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:41:03.316 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:41:03.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.431+0000 7ff46f57e640 1 Processor -- start 2024-04-25T08:41:03.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.432+0000 7ff46f57e640 1 -- start start 2024-04-25T08:41:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.432+0000 7ff46f57e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff468160ef0 0x7ff4680d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.432+0000 7ff46f57e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680d4610 con 0x7ff4680b2ab0 2024-04-25T08:41:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.432+0000 7ff46f57e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4680b9610 con 0x7ff4680b9180 2024-04-25T08:41:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.432+0000 7ff46f57e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff468161cf0 con 0x7ff468160ef0 2024-04-25T08:41:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff46db1b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff4680b2ab0 0x7ff4680d3f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32810/0 (socket says 172.21.15.38:32810) 2024-04-25T08:41:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff46db1b640 1 -- 172.21.15.38:0/4030613897 learned_addr learned my addr 172.21.15.38:0/4030613897 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598724049 0 0) 0x7ff4680d4610 con 0x7ff4680b2ab0 2024-04-25T08:41:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff46d31a640 1 --2- 172.21.15.38:0/4030613897 >> v2:172.21.15.107:3300/0 conn(0x7ff468160ef0 0x7ff4680d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff450003520 con 0x7ff4680b2ab0 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2546096382 0 0) 0x7ff450003520 con 0x7ff4680b2ab0 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 >> v2:172.21.15.107:3300/0 conn(0x7ff468160ef0 msgr2=0x7ff4680d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 --2- 172.21.15.38:0/4030613897 >> v2:172.21.15.107:3300/0 conn(0x7ff468160ef0 0x7ff4680d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 >> v1:172.21.15.107:6789/0 conn(0x7ff4680b9180 legacy=0x7ff4680d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4681616a0 con 0x7ff4680b2ab0 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7ff4580019f0 con 0x7ff4680b2ab0 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 >> v1:172.21.15.38:6789/0 conn(0x7ff4680b2ab0 legacy=0x7ff4680d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 --2- 172.21.15.38:0/4030613897 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff450003a00 0x7ff450003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 --2- 172.21.15.38:0/4030613897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff450004800 0x7ff450004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 --2- 172.21.15.38:0/4030613897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff450005210 0x7ff45000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff468161cf0 con 0x7ff450005210 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45000c160 con 0x7ff450004800 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45000c360 con 0x7ff450003a00 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff46db1b640 1 --2- 172.21.15.38:0/4030613897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff450005210 0x7ff45000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff46d31a640 1 --2- 172.21.15.38:0/4030613897 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff450003a00 0x7ff450003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.433+0000 7ff46cb19640 1 --2- 172.21.15.38:0/4030613897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff450004800 0x7ff450004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.434+0000 7ff46db1b640 1 -- 172.21.15.38:0/4030613897 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff450003a00 msgr2=0x7ff450003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.434+0000 7ff46db1b640 1 --2- 172.21.15.38:0/4030613897 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff450003a00 0x7ff450003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.434+0000 7ff46db1b640 1 -- 172.21.15.38:0/4030613897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff450004800 msgr2=0x7ff450004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.434+0000 7ff46db1b640 1 --2- 172.21.15.38:0/4030613897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff450004800 0x7ff450004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.434+0000 7ff46db1b640 1 -- 172.21.15.38:0/4030613897 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff450003520 con 0x7ff450005210 2024-04-25T08:41:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.434+0000 7ff46db1b640 1 -- 172.21.15.38:0/4030613897 reap_dead start 2024-04-25T08:41:03.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.435+0000 7ff46db1b640 1 --2- 172.21.15.38:0/4030613897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff450005210 0x7ff45000b9c0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7ff45800a760 tx=0x7ff450005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fa0543caf75ffea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:03.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.440+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45800a3b0 con 0x7ff450005210 2024-04-25T08:41:03.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.440+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45802f070 con 0x7ff450005210 2024-04-25T08:41:03.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.440+0000 7ff467fff640 1 -- 172.21.15.38:0/4030613897 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45802bbc0 con 0x7ff450005210 2024-04-25T08:41:03.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.441+0000 7ff46f57e640 1 -- 172.21.15.38:0/4030613897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff450005210 msgr2=0x7ff45000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.441+0000 7ff46f57e640 1 --2- 172.21.15.38:0/4030613897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff450005210 0x7ff45000b9c0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7ff45800a760 tx=0x7ff450005830 comp rx=0 tx=0).stop 2024-04-25T08:41:03.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.441+0000 7ff46f57e640 1 -- 172.21.15.38:0/4030613897 shutdown_connections 2024-04-25T08:41:03.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.441+0000 7ff46f57e640 1 --2- 172.21.15.38:0/4030613897 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff450005210 0x7ff45000b9c0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.441+0000 7ff46f57e640 1 -- 172.21.15.38:0/4030613897 >> 172.21.15.38:0/4030613897 conn(0x7ff468066d10 msgr2=0x7ff468067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:03.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.441+0000 7ff46f57e640 1 -- 172.21.15.38:0/4030613897 shutdown_connections 2024-04-25T08:41:03.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.441+0000 7ff46f57e640 1 -- 172.21.15.38:0/4030613897 wait complete. 2024-04-25T08:41:03.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.442+0000 7ff46f57e640 1 Processor -- start 2024-04-25T08:41:03.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.442+0000 7ff46f57e640 1 -- start start 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46f57e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff450005210 0x7ff4680c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46f57e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff46815a9c0 0x7ff4680b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46f57e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff46815ae40 0x7ff4680b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46f57e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff468162180 con 0x7ff46815a9c0 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46f57e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4681612d0 con 0x7ff450005210 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46f57e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff468162300 con 0x7ff46815ae40 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46cb19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff46815a9c0 0x7ff4680b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46cb19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff46815a9c0 0x7ff4680b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47774/0 (socket says 172.21.15.38:47774) 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46cb19640 1 -- 172.21.15.38:0/470727566 learned_addr learned my addr 172.21.15.38:0/470727566 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46db1b640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff46815ae40 0x7ff4680b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46d31a640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff450005210 0x7ff4680c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46cb19640 1 -- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff46815ae40 msgr2=0x7ff4680b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46cb19640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff46815ae40 0x7ff4680b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46cb19640 1 -- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff450005210 msgr2=0x7ff4680c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46cb19640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff450005210 0x7ff4680c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.443+0000 7ff46cb19640 1 -- 172.21.15.38:0/470727566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46815b330 con 0x7ff46815a9c0 2024-04-25T08:41:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.444+0000 7ff46cb19640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff46815a9c0 0x7ff4680b4670 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00f5f0 tx=0x7ff45c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.444+0000 7ff465ffb640 1 -- 172.21.15.38:0/470727566 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0052e0 con 0x7ff46815a9c0 2024-04-25T08:41:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.444+0000 7ff465ffb640 1 -- 172.21.15.38:0/470727566 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c018070 con 0x7ff46815a9c0 2024-04-25T08:41:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.444+0000 7ff46f57e640 1 -- 172.21.15.38:0/470727566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46815b610 con 0x7ff46815a9c0 2024-04-25T08:41:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.444+0000 7ff465ffb640 1 -- 172.21.15.38:0/470727566 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c003810 con 0x7ff46815a9c0 2024-04-25T08:41:03.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.444+0000 7ff46f57e640 1 -- 172.21.15.38:0/470727566 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46815bbd0 con 0x7ff46815a9c0 2024-04-25T08:41:03.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.445+0000 7ff465ffb640 1 -- 172.21.15.38:0/470727566 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff45c0039b0 con 0x7ff46815a9c0 2024-04-25T08:41:03.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.446+0000 7ff465ffb640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff45406b4d0 0x7ff45406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.446+0000 7ff465ffb640 1 -- 172.21.15.38:0/470727566 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff45c08a280 con 0x7ff46815a9c0 2024-04-25T08:41:03.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.447+0000 7ff46d31a640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff45406b4d0 0x7ff45406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.447+0000 7ff46d31a640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff45406b4d0 0x7ff45406d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff460009a80 tx=0x7ff460008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:03.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.447+0000 7ff472281740 1 Processor -- start 2024-04-25T08:41:03.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.447+0000 7ff472281740 1 -- start start 2024-04-25T08:41:03.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff472281740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b42a230 0x562d8b42a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff472281740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 0x562d8b448020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff472281740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b3fff10 0x562d8b446190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff472281740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8b362a10 con 0x562d8b447ad0 2024-04-25T08:41:03.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff472281740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8b30bb30 con 0x562d8b3fff10 2024-04-25T08:41:03.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff472281740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d8b35ec50 con 0x562d8b42a230 2024-04-25T08:41:03.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff46cb19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 0x562d8b448020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff46cb19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 0x562d8b448020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47776/0 (socket says 172.21.15.38:47776) 2024-04-25T08:41:03.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff46cb19640 1 -- 172.21.15.38:0/2170131618 learned_addr learned my addr 172.21.15.38:0/2170131618 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:03.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff46db1b640 1 --2- 172.21.15.38:0/2170131618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b3fff10 0x562d8b446190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff46d31a640 1 --2- 172.21.15.38:0/2170131618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b42a230 0x562d8b42a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff46cb19640 1 -- 172.21.15.38:0/2170131618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b42a230 msgr2=0x562d8b42a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff46cb19640 1 --2- 172.21.15.38:0/2170131618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b42a230 0x562d8b42a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff46cb19640 1 -- 172.21.15.38:0/2170131618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b3fff10 msgr2=0x562d8b446190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff46cb19640 1 --2- 172.21.15.38:0/2170131618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b3fff10 0x562d8b446190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff46cb19640 1 -- 172.21.15.38:0/2170131618 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d8b320e10 con 0x562d8b447ad0 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff46db1b640 1 --2- 172.21.15.38:0/2170131618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b3fff10 0x562d8b446190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff46cb19640 1 --2- 172.21.15.38:0/2170131618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 0x562d8b448020 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7ff45c010010 tx=0x7ff45c058230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea6302026ad5ba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.448+0000 7ff44effd640 1 -- 172.21.15.38:0/2170131618 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c070020 con 0x562d8b447ad0 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.449+0000 7ff44effd640 1 -- 172.21.15.38:0/2170131618 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c05c920 con 0x562d8b447ad0 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.449+0000 7ff44effd640 1 -- 172.21.15.38:0/2170131618 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45c060ee0 con 0x562d8b447ad0 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.449+0000 7ff472281740 1 -- 172.21.15.38:0/2170131618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 msgr2=0x562d8b448020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.449+0000 7ff472281740 1 --2- 172.21.15.38:0/2170131618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 0x562d8b448020 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7ff45c010010 tx=0x7ff45c058230 comp rx=0 tx=0).stop 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.449+0000 7ff472281740 1 -- 172.21.15.38:0/2170131618 shutdown_connections 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.449+0000 7ff472281740 1 --2- 172.21.15.38:0/2170131618 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b3fff10 0x562d8b446190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.449+0000 7ff472281740 1 --2- 172.21.15.38:0/2170131618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 0x562d8b448020 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.449+0000 7ff472281740 1 --2- 172.21.15.38:0/2170131618 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b42a230 0x562d8b42a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.449+0000 7ff472281740 1 -- 172.21.15.38:0/2170131618 >> 172.21.15.38:0/2170131618 conn(0x562d8b3fc840 msgr2=0x562d8b445040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:03.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.449+0000 7ff472281740 1 -- 172.21.15.38:0/2170131618 shutdown_connections 2024-04-25T08:41:03.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.449+0000 7ff472281740 1 -- 172.21.15.38:0/2170131618 wait complete. 2024-04-25T08:41:03.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.450+0000 7ff472281740 1 -- start start 2024-04-25T08:41:03.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.450+0000 7ff472281740 1 Processor -- start 2024-04-25T08:41:03.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff472281740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b42a230 0x562d8b445560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff472281740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b447ad0 0x562d8b445aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff472281740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b413680 0x562d8b415c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff472281740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8b400cb0 con 0x562d8b42a230 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff472281740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8b31f0c0 con 0x562d8b447ad0 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff472281740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d8b3fa770 con 0x562d8b413680 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff472281740 10 client.0 _ll_register_callbacks cb 0x562d8b312ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff46d31a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b42a230 0x562d8b445560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff46d31a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b42a230 0x562d8b445560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47780/0 (socket says 172.21.15.38:47780) 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff46d31a640 1 -- 172.21.15.38:0/2341699785 learned_addr learned my addr 172.21.15.38:0/2341699785 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff46cb19640 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b447ad0 0x562d8b445aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff46db1b640 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b413680 0x562d8b415c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff46d31a640 1 -- 172.21.15.38:0/2341699785 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b413680 msgr2=0x562d8b415c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff46d31a640 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b413680 0x562d8b415c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff46d31a640 1 -- 172.21.15.38:0/2341699785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b447ad0 msgr2=0x562d8b445aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:03.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff46d31a640 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b447ad0 0x562d8b445aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff46d31a640 1 -- 172.21.15.38:0/2341699785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d8b0772d0 con 0x562d8b42a230 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff46d31a640 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b42a230 0x562d8b445560 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7ff4600045f0 tx=0x7ff460004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339b8f0e852c7ded server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff447fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.451+0000 7ff447fff640 1 -- 172.21.15.38:0/2341699785 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460028020 con 0x562d8b42a230 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff472281740 1 -- 172.21.15.38:0/2341699785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff45c05c920 con 0x562d8b42a230 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff447fff640 1 -- 172.21.15.38:0/2341699785 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff460024640 con 0x562d8b42a230 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff447fff640 1 -- 172.21.15.38:0/2341699785 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff460032050 con 0x562d8b42a230 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff472281740 10 client.9657 fetch_fsmap learned FSMap version 77 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff472281740 1 -- 172.21.15.38:0/2341699785 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x562d8b1416b0 con 0x562d8b42a230 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff447fff640 1 -- 172.21.15.38:0/2341699785 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45c05c920 con 0x562d8b42a230 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff447fff640 1 -- 172.21.15.38:0/2341699785 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff460035020 con 0x562d8b42a230 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff472281740 10 client.9657 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff472281740 10 client.9657 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff472281740 20 client.9657 populate_metadata read hostname 'smithi038' 2024-04-25T08:41:03.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff472281740 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562d8b460c40 0x562d8b463030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff472281740 4 client.9657 mds_command: new command op to 9489 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff472281740 1 -- 172.21.15.38:0/2341699785 --> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d8b3c1920 con 0x562d8b460c40 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff46cb19640 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562d8b460c40 0x562d8b463030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff46cb19640 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562d8b460c40 0x562d8b463030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff6e41abc17dd7dd server_cookie=2c679d40db6c68a3 in_seq=0 out_seq=0 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.452+0000 7ff447fff640 10 client.9657 ms_handle_connect on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff447fff640 1 -- 172.21.15.38:0/2341699785 <== mds.0 v2:172.21.15.38:6834/1312063561 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d8b3c1920 con 0x562d8b460c40 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff447fff640 10 client.9657 handle_command_reply: tid=0 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff472281740 1 client.9657 shutdown 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff472281740 1 -- 172.21.15.38:0/2341699785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b42a230 msgr2=0x562d8b445560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff472281740 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b42a230 0x562d8b445560 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7ff4600045f0 tx=0x7ff460004730 comp rx=0 tx=0).stop 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff472281740 1 -- 172.21.15.38:0/2341699785 shutdown_connections 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff472281740 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562d8b460c40 0x562d8b463030 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff472281740 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b413680 0x562d8b415c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff472281740 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b447ad0 0x562d8b445aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff447fff640 0 client.9657 ms_handle_reset on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:03.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff472281740 1 --2- 172.21.15.38:0/2341699785 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b42a230 0x562d8b445560 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff472281740 1 -- 172.21.15.38:0/2341699785 >> 172.21.15.38:0/2341699785 conn(0x562d8b3fff10 msgr2=0x562d8b3fb3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:03.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff472281740 1 -- 172.21.15.38:0/2341699785 shutdown_connections 2024-04-25T08:41:03.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff472281740 1 -- 172.21.15.38:0/2341699785 wait complete. 2024-04-25T08:41:03.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.453+0000 7ff472281740 20 client.9657 trim_cache size 0 max 16384 2024-04-25T08:41:03.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.469+0000 7ff472281740 1 Processor -- start 2024-04-25T08:41:03.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.469+0000 7ff472281740 1 -- start start 2024-04-25T08:41:03.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff472281740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b413680 0x562d8b45d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff472281740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b42a230 0x562d8b43b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff472281740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 0x562d8b4004e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff472281740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4681ffc30 con 0x562d8b447ad0 2024-04-25T08:41:03.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff472281740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4681ff030 con 0x562d8b42a230 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff472281740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4681ff630 con 0x562d8b413680 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff46db1b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 0x562d8b4004e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff46db1b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 0x562d8b4004e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47796/0 (socket says 172.21.15.38:47796) 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff46db1b640 1 -- 172.21.15.38:0/2185610920 learned_addr learned my addr 172.21.15.38:0/2185610920 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff46d31a640 1 --2- 172.21.15.38:0/2185610920 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b413680 0x562d8b45d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff46cb19640 1 --2- 172.21.15.38:0/2185610920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b42a230 0x562d8b43b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff46db1b640 1 -- 172.21.15.38:0/2185610920 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b413680 msgr2=0x562d8b45d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff46db1b640 1 --2- 172.21.15.38:0/2185610920 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b413680 0x562d8b45d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff46db1b640 1 -- 172.21.15.38:0/2185610920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b42a230 msgr2=0x562d8b43b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff46db1b640 1 --2- 172.21.15.38:0/2185610920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b42a230 0x562d8b43b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff46db1b640 1 -- 172.21.15.38:0/2185610920 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d8b319e40 con 0x562d8b447ad0 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff46cb19640 1 --2- 172.21.15.38:0/2185610920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b42a230 0x562d8b43b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.470+0000 7ff46db1b640 1 --2- 172.21.15.38:0/2185610920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 0x562d8b4004e0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7ff45800a850 tx=0x7ff458007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7ae064fc59e13e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.471+0000 7ff44e7fc640 1 -- 172.21.15.38:0/2185610920 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff458046070 con 0x562d8b447ad0 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.471+0000 7ff44e7fc640 1 -- 172.21.15.38:0/2185610920 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45802f040 con 0x562d8b447ad0 2024-04-25T08:41:03.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.471+0000 7ff44e7fc640 1 -- 172.21.15.38:0/2185610920 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4580019f0 con 0x562d8b447ad0 2024-04-25T08:41:03.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.471+0000 7ff472281740 1 -- 172.21.15.38:0/2185610920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 msgr2=0x562d8b4004e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.471+0000 7ff472281740 1 --2- 172.21.15.38:0/2185610920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 0x562d8b4004e0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7ff45800a850 tx=0x7ff458007000 comp rx=0 tx=0).stop 2024-04-25T08:41:03.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.471+0000 7ff472281740 1 -- 172.21.15.38:0/2185610920 shutdown_connections 2024-04-25T08:41:03.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.471+0000 7ff472281740 1 --2- 172.21.15.38:0/2185610920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b447ad0 0x562d8b4004e0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.471+0000 7ff472281740 1 --2- 172.21.15.38:0/2185610920 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b42a230 0x562d8b43b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.471+0000 7ff472281740 1 --2- 172.21.15.38:0/2185610920 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b413680 0x562d8b45d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.471+0000 7ff472281740 1 -- 172.21.15.38:0/2185610920 >> 172.21.15.38:0/2185610920 conn(0x562d8b3fff10 msgr2=0x562d8b3fb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:03.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.471+0000 7ff472281740 1 -- 172.21.15.38:0/2185610920 shutdown_connections 2024-04-25T08:41:03.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.471+0000 7ff472281740 1 -- 172.21.15.38:0/2185610920 wait complete. 2024-04-25T08:41:03.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.472+0000 7ff472281740 1 -- start start 2024-04-25T08:41:03.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.472+0000 7ff472281740 1 Processor -- start 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff472281740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b413680 0x562d8b3fe5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff472281740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b42a230 0x562d8b43eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff472281740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b447ad0 0x562d8b43f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff472281740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8b414e40 con 0x562d8b413680 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff472281740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d8b446490 con 0x562d8b447ad0 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff472281740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d8b415380 con 0x562d8b42a230 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff472281740 10 client.0 _ll_register_callbacks cb 0x562d8b3fac50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff46d31a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b413680 0x562d8b3fe5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff46d31a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b413680 0x562d8b3fe5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47804/0 (socket says 172.21.15.38:47804) 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff46d31a640 1 -- 172.21.15.38:0/1628171573 learned_addr learned my addr 172.21.15.38:0/1628171573 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff46cb19640 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b42a230 0x562d8b43eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff46db1b640 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b447ad0 0x562d8b43f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff46d31a640 1 -- 172.21.15.38:0/1628171573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b42a230 msgr2=0x562d8b43eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:03.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff46d31a640 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b42a230 0x562d8b43eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff46d31a640 1 -- 172.21.15.38:0/1628171573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b447ad0 msgr2=0x562d8b43f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff46d31a640 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b447ad0 0x562d8b43f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff46d31a640 1 -- 172.21.15.38:0/1628171573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d8b43b100 con 0x562d8b413680 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff46d31a640 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b413680 0x562d8b3fe5e0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7ff460006b10 tx=0x7ff460018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2104c1e6fa99a54b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff447fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.473+0000 7ff447fff640 1 -- 172.21.15.38:0/1628171573 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff460028040 con 0x562d8b413680 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff447fff640 1 -- 172.21.15.38:0/1628171573 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff460016630 con 0x562d8b413680 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff472281740 1 -- 172.21.15.38:0/1628171573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff45802f040 con 0x562d8b413680 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff447fff640 1 -- 172.21.15.38:0/1628171573 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff460044050 con 0x562d8b413680 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff472281740 10 client.9663 fetch_fsmap learned FSMap version 77 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff472281740 1 -- 172.21.15.38:0/1628171573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x562d8b3feb80 con 0x562d8b413680 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff447fff640 1 -- 172.21.15.38:0/1628171573 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff45802f040 con 0x562d8b413680 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff447fff640 1 -- 172.21.15.38:0/1628171573 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff460047020 con 0x562d8b413680 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff472281740 10 client.9663 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T08:41:03.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff472281740 10 client.9663 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff472281740 20 client.9663 populate_metadata read hostname 'smithi038' 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff472281740 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562d8b4b02b0 0x562d8b447f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff472281740 4 client.9663 mds_command: new command op to 9489 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff472281740 1 -- 172.21.15.38:0/1628171573 --> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d8b36ff50 con 0x562d8b4b02b0 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff46cb19640 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562d8b4b02b0 0x562d8b447f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff447fff640 10 client.9663 ms_handle_connect on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.474+0000 7ff46cb19640 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562d8b4b02b0 0x562d8b447f80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f7cab4865b10500 server_cookie=8d3812f883ed01bb in_seq=0 out_seq=0 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff447fff640 1 -- 172.21.15.38:0/1628171573 <== mds.0 v2:172.21.15.38:6834/1312063561 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x562d8b36ff50 con 0x562d8b4b02b0 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff447fff640 10 client.9663 handle_command_reply: tid=0 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff472281740 1 client.9663 shutdown 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff472281740 1 -- 172.21.15.38:0/1628171573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b413680 msgr2=0x562d8b3fe5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff472281740 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b413680 0x562d8b3fe5e0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7ff460006b10 tx=0x7ff460018000 comp rx=0 tx=0).stop 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff472281740 1 -- 172.21.15.38:0/1628171573 shutdown_connections 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff472281740 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d8b447ad0 0x562d8b43f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff472281740 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562d8b4b02b0 0x562d8b447f80 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff472281740 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d8b42a230 0x562d8b43eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff447fff640 0 client.9663 ms_handle_reset on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:03.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff472281740 1 --2- 172.21.15.38:0/1628171573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d8b413680 0x562d8b3fe5e0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff472281740 1 -- 172.21.15.38:0/1628171573 >> 172.21.15.38:0/1628171573 conn(0x562d8b3fff10 msgr2=0x562d8b3ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:03.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff472281740 1 -- 172.21.15.38:0/1628171573 shutdown_connections 2024-04-25T08:41:03.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff472281740 1 -- 172.21.15.38:0/1628171573 wait complete. 2024-04-25T08:41:03.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.475+0000 7ff472281740 20 client.9663 trim_cache size 0 max 16384 2024-04-25T08:41:03.473 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:41:03.473 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:41:03.473 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:41:03.473 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 9489, 2024-04-25T08:41:03.474 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:41:03.474 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:41:03.474 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:41:03.474 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 9.1126091309999993, 2024-04-25T08:41:03.474 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 77, 2024-04-25T08:41:03.474 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 132, 2024-04-25T08:41:03.474 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 132, 2024-04-25T08:41:03.474 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 14.137321668 2024-04-25T08:41:03.474 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:41:03.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.476+0000 7ff46f57e640 1 -- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff45406b4d0 msgr2=0x7ff45406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.476+0000 7ff46f57e640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff45406b4d0 0x7ff45406d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff460009a80 tx=0x7ff460008040 comp rx=0 tx=0).stop 2024-04-25T08:41:03.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.476+0000 7ff46f57e640 1 -- 172.21.15.38:0/470727566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff46815a9c0 msgr2=0x7ff4680b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.476+0000 7ff46f57e640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff46815a9c0 0x7ff4680b4670 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7ff45c00f5f0 tx=0x7ff45c00fab0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.476+0000 7ff46f57e640 1 -- 172.21.15.38:0/470727566 shutdown_connections 2024-04-25T08:41:03.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.476+0000 7ff46f57e640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff45406b4d0 0x7ff45406d990 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.476+0000 7ff46f57e640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff46815ae40 0x7ff4680b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.476+0000 7ff46f57e640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff46815a9c0 0x7ff4680b4670 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.476+0000 7ff46f57e640 1 --2- 172.21.15.38:0/470727566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff450005210 0x7ff4680c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.476+0000 7ff46f57e640 1 -- 172.21.15.38:0/470727566 >> 172.21.15.38:0/470727566 conn(0x7ff468066d10 msgr2=0x7ff4680670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.476+0000 7ff46f57e640 1 -- 172.21.15.38:0/470727566 shutdown_connections 2024-04-25T08:41:03.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.476+0000 7ff46f57e640 1 -- 172.21.15.38:0/470727566 wait complete. 2024-04-25T08:41:03.479 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:41:03.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.588+0000 7f7c39f0a640 1 Processor -- start 2024-04-25T08:41:03.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.588+0000 7f7c39f0a640 1 -- start start 2024-04-25T08:41:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.589+0000 7f7c39f0a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7c340b4b50 0x7f7c340ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.589+0000 7f7c39f0a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c340b50c0 con 0x7f7c34144b70 2024-04-25T08:41:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.589+0000 7f7c39f0a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c340b53a0 con 0x7f7c34144760 2024-04-25T08:41:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.589+0000 7f7c39f0a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3415f380 con 0x7f7c340b4b50 2024-04-25T08:41:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.589+0000 7f7c32ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7c340b4b50 0x7f7c340ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.589+0000 7f7c33fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7c34144b70 0x7f7c340b3810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:32820/0 (socket says 172.21.15.38:32820) 2024-04-25T08:41:03.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.589+0000 7f7c33fff640 1 -- 172.21.15.38:0/2988171614 learned_addr learned my addr 172.21.15.38:0/2988171614 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:03.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.589+0000 7f7c32ffd640 1 --2- 172.21.15.38:0/2988171614 >> v2:172.21.15.38:3300/0 conn(0x7f7c340b4b50 0x7f7c340ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47808/0 (socket says 172.21.15.38:47808) 2024-04-25T08:41:03.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.590+0000 7f7c32ffd640 1 -- 172.21.15.38:0/2988171614 >> v1:172.21.15.107:6789/0 conn(0x7f7c34144760 legacy=0x7f7c3415faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:03.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.590+0000 7f7c32ffd640 1 -- 172.21.15.38:0/2988171614 >> v1:172.21.15.38:6789/0 conn(0x7f7c34144b70 legacy=0x7f7c340b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.590+0000 7f7c32ffd640 1 -- 172.21.15.38:0/2988171614 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c340b5680 con 0x7f7c340b4b50 2024-04-25T08:41:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.590+0000 7f7c32ffd640 1 --2- 172.21.15.38:0/2988171614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340b4b50 0x7f7c340ce360 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f7c280097a0 tx=0x7f7c2802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8182b82d5440b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.590+0000 7f7c327fc640 1 -- 172.21.15.38:0/2988171614 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28037070 con 0x7f7c340b4b50 2024-04-25T08:41:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.590+0000 7f7c327fc640 1 -- 172.21.15.38:0/2988171614 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2802ae50 con 0x7f7c340b4b50 2024-04-25T08:41:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.590+0000 7f7c327fc640 1 -- 172.21.15.38:0/2988171614 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28032660 con 0x7f7c340b4b50 2024-04-25T08:41:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.591+0000 7f7c39f0a640 1 -- 172.21.15.38:0/2988171614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340b4b50 msgr2=0x7f7c340ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.591+0000 7f7c39f0a640 1 --2- 172.21.15.38:0/2988171614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340b4b50 0x7f7c340ce360 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f7c280097a0 tx=0x7f7c2802a140 comp rx=0 tx=0).stop 2024-04-25T08:41:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.591+0000 7f7c39f0a640 1 -- 172.21.15.38:0/2988171614 shutdown_connections 2024-04-25T08:41:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.591+0000 7f7c39f0a640 1 --2- 172.21.15.38:0/2988171614 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340b4b50 0x7f7c340ce360 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.591+0000 7f7c39f0a640 1 -- 172.21.15.38:0/2988171614 >> 172.21.15.38:0/2988171614 conn(0x7f7c3406b210 msgr2=0x7f7c3406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:03.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.591+0000 7f7c39f0a640 1 -- 172.21.15.38:0/2988171614 shutdown_connections 2024-04-25T08:41:03.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.591+0000 7f7c39f0a640 1 -- 172.21.15.38:0/2988171614 wait complete. 2024-04-25T08:41:03.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.592+0000 7f7c39f0a640 1 Processor -- start 2024-04-25T08:41:03.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.592+0000 7f7c39f0a640 1 -- start start 2024-04-25T08:41:03.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.593+0000 7f7c39f0a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340b4b50 0x7f7c341e8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.593+0000 7f7c39f0a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c34144b70 0x7f7c3414e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.593+0000 7f7c39f0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340c5b40 0x7f7c340c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.593+0000 7f7c39f0a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c34145280 con 0x7f7c340c5b40 2024-04-25T08:41:03.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.593+0000 7f7c39f0a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3415f660 con 0x7f7c340b4b50 2024-04-25T08:41:03.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.593+0000 7f7c39f0a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c34145400 con 0x7f7c34144b70 2024-04-25T08:41:03.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.593+0000 7f7c33fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340c5b40 0x7f7c340c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.593+0000 7f7c33fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340c5b40 0x7f7c340c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47818/0 (socket says 172.21.15.38:47818) 2024-04-25T08:41:03.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.593+0000 7f7c33fff640 1 -- 172.21.15.38:0/3991879875 learned_addr learned my addr 172.21.15.38:0/3991879875 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:03.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.593+0000 7f7c337fe640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340b4b50 0x7f7c341e8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.593+0000 7f7c32ffd640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c34144b70 0x7f7c3414e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.594+0000 7f7c33fff640 1 -- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c34144b70 msgr2=0x7f7c3414e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.594+0000 7f7c33fff640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c34144b70 0x7f7c3414e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.594+0000 7f7c33fff640 1 -- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340b4b50 msgr2=0x7f7c341e8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.594+0000 7f7c33fff640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340b4b50 0x7f7c341e8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.594+0000 7f7c33fff640 1 -- 172.21.15.38:0/3991879875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c341e7ed0 con 0x7f7c340c5b40 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.594+0000 7f7c337fe640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340b4b50 0x7f7c341e8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.594+0000 7f7c33fff640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340c5b40 0x7f7c340c5ff0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00ca80 tx=0x7f7c2c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.594+0000 7f7c1bfff640 1 -- 172.21.15.38:0/3991879875 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c017070 con 0x7f7c340c5b40 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.594+0000 7f7c1bfff640 1 -- 172.21.15.38:0/3991879875 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2c00ac30 con 0x7f7c340c5b40 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.594+0000 7f7c39f0a640 1 -- 172.21.15.38:0/3991879875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c341e81b0 con 0x7f7c340c5b40 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.594+0000 7f7c1bfff640 1 -- 172.21.15.38:0/3991879875 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c012600 con 0x7f7c340c5b40 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.594+0000 7f7c39f0a640 1 -- 172.21.15.38:0/3991879875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c341e8770 con 0x7f7c340c5b40 2024-04-25T08:41:03.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.595+0000 7f7c39f0a640 1 -- 172.21.15.38:0/3991879875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c34068e80 con 0x7f7c340c5b40 2024-04-25T08:41:03.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.596+0000 7f7c1bfff640 1 -- 172.21.15.38:0/3991879875 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c2c01a070 con 0x7f7c340c5b40 2024-04-25T08:41:03.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.596+0000 7f7c1bfff640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c2006b4d0 0x7f7c2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:03.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.596+0000 7f7c1bfff640 1 -- 172.21.15.38:0/3991879875 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7c2c089fd0 con 0x7f7c340c5b40 2024-04-25T08:41:03.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.596+0000 7f7c337fe640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c2006b4d0 0x7f7c2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:03.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.597+0000 7f7c337fe640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c2006b4d0 0x7f7c2006d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7c24006fd0 tx=0x7f7c24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:03.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.600+0000 7f7c1bfff640 1 -- 172.21.15.38:0/3991879875 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c2c00ae20 con 0x7f7c340c5b40 2024-04-25T08:41:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.816+0000 7f7c39f0a640 1 -- 172.21.15.38:0/3991879875 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7c340717a0 con 0x7f7c340c5b40 2024-04-25T08:41:03.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.817+0000 7f7c1bfff640 1 -- 172.21.15.38:0/3991879875 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7c2c058670 con 0x7f7c340c5b40 2024-04-25T08:41:03.810 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:41:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.819+0000 7f7c39f0a640 1 -- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c2006b4d0 msgr2=0x7f7c2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.819+0000 7f7c39f0a640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c2006b4d0 0x7f7c2006d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7c24006fd0 tx=0x7f7c24008040 comp rx=0 tx=0).stop 2024-04-25T08:41:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.819+0000 7f7c39f0a640 1 -- 172.21.15.38:0/3991879875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340c5b40 msgr2=0x7f7c340c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.819+0000 7f7c39f0a640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340c5b40 0x7f7c340c5ff0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00ca80 tx=0x7f7c2c00cf40 comp rx=0 tx=0).stop 2024-04-25T08:41:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.819+0000 7f7c39f0a640 1 -- 172.21.15.38:0/3991879875 shutdown_connections 2024-04-25T08:41:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.819+0000 7f7c39f0a640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c340c5b40 0x7f7c340c5ff0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.819+0000 7f7c39f0a640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c2006b4d0 0x7f7c2006d990 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.819+0000 7f7c39f0a640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c34144b70 0x7f7c3414e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.819+0000 7f7c39f0a640 1 --2- 172.21.15.38:0/3991879875 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c340b4b50 0x7f7c341e8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.819+0000 7f7c39f0a640 1 -- 172.21.15.38:0/3991879875 >> 172.21.15.38:0/3991879875 conn(0x7f7c3406b210 msgr2=0x7f7c34072250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.819+0000 7f7c39f0a640 1 -- 172.21.15.38:0/3991879875 shutdown_connections 2024-04-25T08:41:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:03.819+0000 7f7c39f0a640 1 -- 172.21.15.38:0/3991879875 wait complete. 2024-04-25T08:41:03.823 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:41:03.823 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:41:03.823 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:41:03.823 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:41:03.823 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:41:03.823 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:41:03.823 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:41:03.823 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:41:03.823 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:41:03.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:03.824 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:41:03.839 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:41:03.839 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:41:03.839 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:41:03.839 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:41:03.839 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:41:03.839 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:41:03.839 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:41:03.839 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:41:03.839 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:41:03.839 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42173sec preferred_lft 42173sec 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:41:03.840 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:41:03.841 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::440f:97ff:fe63:87b/64 scope link 2024-04-25T08:41:03.841 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:41:03.841 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:41:03.841 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ea:23:76:ab:b1:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:41:03.841 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::e823:76ff:feab:b1e9/64 scope link 2024-04-25T08:41:03.841 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:41:03.841 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:41:03.841 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:41:03.841 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::38c7:38ff:fe2e:6c92/64 scope link 2024-04-25T08:41:03.841 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:41:03.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:03.842 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:41:03.902 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:41:03.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:03.904 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:41:03.960 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:41:03.960 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:41:03.960 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:41:03.960 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:41:03.960 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:41:03.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:03.960 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:41:03.961 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:03.961 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:41:03.961 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:41:03.961 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:41:03.961 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:04.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:04.046 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:41:04.061 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:41:04.061 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:41:04.061 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:41:04.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:04.061 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T08:41:04.126 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:41:04.126 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:41:04.126 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:41:04.183 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:41:04.184 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:41:04.184 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:41:04.238 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:41:04.303 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:41:04.359 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:41:04.359 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:41:04.359 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:41:04.359 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:41:04.359 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:41:04.360 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:41:04.361 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:41:04.362 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:41:04.362 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:41:04.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:04.362 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:41:04.414 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:41:04.414 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:41:04.457 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:41:04.480 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:41:04.537 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:41:04.537 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:41:04.537 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:41:04.537 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:41:04.537 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:41:04.537 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:41:04.537 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:41:04.537 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:41:04.537 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:41:04.537 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:41:04.537 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:41:04.537 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:41:04.538 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:41:04.539 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:41:04.539 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:41:04.539 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:41:04.539 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:41:04.539 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:41:04.539 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:41:04.539 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:41:04.539 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:41:04.539 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:41:04.539 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:41:04.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:04.539 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:41:04.540 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-04-25T08:41:04.546+0000 7f6aa9d57480 -1 init, newargv = 0x55619ac57780 newargc=15 2024-04-25T08:41:04.540 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[94693]: starting ceph client 2024-04-25T08:41:04.542 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[94693]: starting fuse 2024-04-25T08:41:04.555 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:41:04.555 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:41:04.556 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:41:04.556 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:41:04.556 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:41:04.556 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:41:04.556 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:41:04.557 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:41:04.557 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:41:04.557 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:41:04.557 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:41:04.557 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:41:04.557 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:41:04.557 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:41:04.558 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-04-25T08:41:04.559 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:41:04.634 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:41:04.637 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:41:04.677 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.94693.asok 2024-04-25T08:41:04.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:41:04.688 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.94693.asok 2024-04-25T08:41:04.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:04.688 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.94693.asok status 2024-04-25T08:41:04.844 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:41:04.844 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:41:04.844 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:41:04.844 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:41:04.844 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-04-25T08:41:04.844 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:41:04.844 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:41:04.844 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "94693", 2024-04-25T08:41:04.844 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:41:04.844 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:41:04.844 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 9678, 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 9678 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1518341262 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:41:04.845 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:41:04.846 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1518341262 2024-04-25T08:41:04.846 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:41:04.846 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.9678 192.168.0.1:0/1518341262", 2024-04-25T08:41:04.846 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/1518341262", 2024-04-25T08:41:04.846 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:41:04.846 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 77, 2024-04-25T08:41:04.846 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 132, 2024-04-25T08:41:04.846 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:41:04.846 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:41:04.846 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:41:04.846 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:41:04.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:04.852 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:41:04.852 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:41:04.868 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:41:04.868 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:41:04.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:04.868 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:41:04.933 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:05.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.080+0000 7fad3c322640 1 Processor -- start 2024-04-25T08:41:05.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.080+0000 7fad3c322640 1 -- start start 2024-04-25T08:41:05.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.081+0000 7fad3c322640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad3415da90 0x7fad340d2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.081+0000 7fad3c322640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fad340d3fb0 0x7fad340b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.081+0000 7fad3c322640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad340d1b60 con 0x7fad340ceca0 2024-04-25T08:41:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.081+0000 7fad3c322640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad340d2bc0 con 0x7fad340d3fb0 2024-04-25T08:41:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.081+0000 7fad3c322640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad340d1e00 con 0x7fad3415da90 2024-04-25T08:41:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.082+0000 7fad3a8bf640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fad340d3fb0 0x7fad340b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.082+0000 7fad3a8bf640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fad340d3fb0 0x7fad340b69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47820/0 (socket says 172.21.15.38:47820) 2024-04-25T08:41:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.082+0000 7fad3a8bf640 1 -- 172.21.15.38:0/1407960236 learned_addr learned my addr 172.21.15.38:0/1407960236 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.082+0000 7fad3a0be640 1 --2- 172.21.15.38:0/1407960236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad3415da90 0x7fad340d2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.082+0000 7fad3a8bf640 1 -- 172.21.15.38:0/1407960236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad3415da90 msgr2=0x7fad340d2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.082+0000 7fad3a8bf640 1 --2- 172.21.15.38:0/1407960236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad3415da90 0x7fad340d2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.082+0000 7fad3a8bf640 1 -- 172.21.15.38:0/1407960236 >> v1:172.21.15.107:6789/0 conn(0x7fad340ceca0 legacy=0x7fad340d38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:05.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.082+0000 7fad3a8bf640 1 -- 172.21.15.38:0/1407960236 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad340d20e0 con 0x7fad340d3fb0 2024-04-25T08:41:05.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.083+0000 7fad3a8bf640 1 --2- 172.21.15.38:0/1407960236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad340d3fb0 0x7fad340b69a0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009e50 tx=0x7fad2c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318bbc66d3cde0e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:05.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.083+0000 7fad390bc640 1 -- 172.21.15.38:0/1407960236 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c036070 con 0x7fad340d3fb0 2024-04-25T08:41:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.083+0000 7fad390bc640 1 -- 172.21.15.38:0/1407960236 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c031600 con 0x7fad340d3fb0 2024-04-25T08:41:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.083+0000 7fad390bc640 1 -- 172.21.15.38:0/1407960236 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c0319d0 con 0x7fad340d3fb0 2024-04-25T08:41:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.083+0000 7fad3c322640 1 -- 172.21.15.38:0/1407960236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad340d3fb0 msgr2=0x7fad340b69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.083+0000 7fad3c322640 1 --2- 172.21.15.38:0/1407960236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad340d3fb0 0x7fad340b69a0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009e50 tx=0x7fad2c02a6c0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.083+0000 7fad3c322640 1 -- 172.21.15.38:0/1407960236 shutdown_connections 2024-04-25T08:41:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.083+0000 7fad3c322640 1 --2- 172.21.15.38:0/1407960236 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad340d3fb0 0x7fad340b69a0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.083+0000 7fad3c322640 1 --2- 172.21.15.38:0/1407960236 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad3415da90 0x7fad340d2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.083+0000 7fad3c322640 1 -- 172.21.15.38:0/1407960236 >> 172.21.15.38:0/1407960236 conn(0x7fad3406bed0 msgr2=0x7fad3406c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.083+0000 7fad3c322640 1 -- 172.21.15.38:0/1407960236 shutdown_connections 2024-04-25T08:41:05.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.083+0000 7fad3c322640 1 -- 172.21.15.38:0/1407960236 wait complete. 2024-04-25T08:41:05.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.085+0000 7fad3c322640 1 Processor -- start 2024-04-25T08:41:05.079 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.085+0000 7fad3c322640 1 -- start start 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.085+0000 7fad3c322640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad340d3fb0 0x7fad341593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.085+0000 7fad3c322640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad3415da90 0x7fad341598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.085+0000 7fad3c322640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad340ca3b0 0x7fad340c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.085+0000 7fad3c322640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3415dff0 con 0x7fad340ca3b0 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.085+0000 7fad3c322640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3415de70 con 0x7fad3415da90 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.085+0000 7fad3c322640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad340d34b0 con 0x7fad340d3fb0 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.085+0000 7fad3a8bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad340ca3b0 0x7fad340c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.085+0000 7fad3a8bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad340ca3b0 0x7fad340c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47834/0 (socket says 172.21.15.38:47834) 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.085+0000 7fad3a8bf640 1 -- 172.21.15.38:0/391057881 learned_addr learned my addr 172.21.15.38:0/391057881 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad3a0be640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad340d3fb0 0x7fad341593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad3a8bf640 1 -- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad340d3fb0 msgr2=0x7fad341593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad398bd640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad3415da90 0x7fad341598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad3a8bf640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad340d3fb0 0x7fad341593b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad3a8bf640 1 -- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad3415da90 msgr2=0x7fad341598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:05.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad3a8bf640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad3415da90 0x7fad341598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad3a8bf640 1 -- 172.21.15.38:0/391057881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad340c8250 con 0x7fad340ca3b0 2024-04-25T08:41:05.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad3a0be640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad340d3fb0 0x7fad341593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:05.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad398bd640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad3415da90 0x7fad341598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:05.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad3a8bf640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad340ca3b0 0x7fad340c74f0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009e50 tx=0x7fad2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:05.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad2affd640 1 -- 172.21.15.38:0/391057881 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c036050 con 0x7fad340ca3b0 2024-04-25T08:41:05.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad2affd640 1 -- 172.21.15.38:0/391057881 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2c042070 con 0x7fad340ca3b0 2024-04-25T08:41:05.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad3c322640 1 -- 172.21.15.38:0/391057881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad340c8530 con 0x7fad340ca3b0 2024-04-25T08:41:05.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad2affd640 1 -- 172.21.15.38:0/391057881 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2c009510 con 0x7fad340ca3b0 2024-04-25T08:41:05.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.086+0000 7fad3c322640 1 -- 172.21.15.38:0/391057881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad340c8af0 con 0x7fad340ca3b0 2024-04-25T08:41:05.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.087+0000 7fad3c322640 1 -- 172.21.15.38:0/391057881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad34068e80 con 0x7fad340ca3b0 2024-04-25T08:41:05.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.088+0000 7fad2affd640 1 -- 172.21.15.38:0/391057881 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad2c007bb0 con 0x7fad340ca3b0 2024-04-25T08:41:05.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.088+0000 7fad2affd640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad1806b4d0 0x7fad1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:05.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.088+0000 7fad2affd640 1 -- 172.21.15.38:0/391057881 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fad2c0ab240 con 0x7fad340ca3b0 2024-04-25T08:41:05.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.088+0000 7fad3a0be640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad1806b4d0 0x7fad1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:05.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.089+0000 7fad3a0be640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad1806b4d0 0x7fad1806d990 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fad3000b820 tx=0x7fad30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:05.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.092+0000 7fad2affd640 1 -- 172.21.15.38:0/391057881 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad2c045360 con 0x7fad340ca3b0 2024-04-25T08:41:05.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.315+0000 7fad3c322640 1 -- 172.21.15.38:0/391057881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad3415d220 con 0x7fad340ca3b0 2024-04-25T08:41:05.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.315+0000 7fad2affd640 1 -- 172.21.15.38:0/391057881 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fad2c079fc0 con 0x7fad340ca3b0 2024-04-25T08:41:05.308 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:05.308 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:40:55.730538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/1312063561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1312063561},{"type":"v1","addr":"172.21.15.38:6835","nonce":1312063561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:05.308 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 77 2024-04-25T08:41:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.317+0000 7fad3c322640 1 -- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad1806b4d0 msgr2=0x7fad1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.317+0000 7fad3c322640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad1806b4d0 0x7fad1806d990 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fad3000b820 tx=0x7fad30008040 comp rx=0 tx=0).stop 2024-04-25T08:41:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.317+0000 7fad3c322640 1 -- 172.21.15.38:0/391057881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad340ca3b0 msgr2=0x7fad340c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:05.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.317+0000 7fad3c322640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad340ca3b0 0x7fad340c74f0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009e50 tx=0x7fad2c004a40 comp rx=0 tx=0).stop 2024-04-25T08:41:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.317+0000 7fad3c322640 1 -- 172.21.15.38:0/391057881 shutdown_connections 2024-04-25T08:41:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.317+0000 7fad3c322640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad340ca3b0 0x7fad340c74f0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.317+0000 7fad3c322640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad3415da90 0x7fad341598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.317+0000 7fad3c322640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad1806b4d0 0x7fad1806d990 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.317+0000 7fad3c322640 1 --2- 172.21.15.38:0/391057881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad340d3fb0 0x7fad341593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.317+0000 7fad3c322640 1 -- 172.21.15.38:0/391057881 >> 172.21.15.38:0/391057881 conn(0x7fad3406bed0 msgr2=0x7fad340b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.317+0000 7fad3c322640 1 -- 172.21.15.38:0/391057881 shutdown_connections 2024-04-25T08:41:05.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.317+0000 7fad3c322640 1 -- 172.21.15.38:0/391057881 wait complete. 2024-04-25T08:41:05.321 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:05.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.429+0000 7fcdf1f6a640 1 Processor -- start 2024-04-25T08:41:05.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.429+0000 7fcdf1f6a640 1 -- start start 2024-04-25T08:41:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.430+0000 7fcdf1f6a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdec09ab90 0x7fcdec15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.430+0000 7fcdf1f6a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcdec0d0460 0x7fcdec15fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.430+0000 7fcdf1f6a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdec160340 con 0x7fcdec15bb10 2024-04-25T08:41:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.430+0000 7fcdf1f6a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdec0cffd0 con 0x7fcdec0d0460 2024-04-25T08:41:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.430+0000 7fcdf1f6a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec0cf400 con 0x7fcdec09ab90 2024-04-25T08:41:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.430+0000 7fcdebfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcdec0d0460 0x7fcdec15fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.430+0000 7fcdebfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcdec0d0460 0x7fcdec15fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47840/0 (socket says 172.21.15.38:47840) 2024-04-25T08:41:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.431+0000 7fcdebfff640 1 -- 172.21.15.38:0/3214714739 learned_addr learned my addr 172.21.15.38:0/3214714739 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.431+0000 7fcdeaffd640 1 --2- 172.21.15.38:0/3214714739 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdec09ab90 0x7fcdec15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.431+0000 7fcdebfff640 1 -- 172.21.15.38:0/3214714739 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdec09ab90 msgr2=0x7fcdec15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.431+0000 7fcdebfff640 1 --2- 172.21.15.38:0/3214714739 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdec09ab90 0x7fcdec15f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.431+0000 7fcdebfff640 1 -- 172.21.15.38:0/3214714739 >> v1:172.21.15.107:6789/0 conn(0x7fcdec15bb10 legacy=0x7fcdec09a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.431+0000 7fcdebfff640 1 -- 172.21.15.38:0/3214714739 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec0cf6a0 con 0x7fcdec0d0460 2024-04-25T08:41:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.432+0000 7fcdebfff640 1 --2- 172.21.15.38:0/3214714739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdec0d0460 0x7fcdec15fa20 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009e50 tx=0x7fcde402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3881d79ec77d66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:05.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.432+0000 7fcdea7fc640 1 -- 172.21.15.38:0/3214714739 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4036070 con 0x7fcdec0d0460 2024-04-25T08:41:05.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.432+0000 7fcdea7fc640 1 -- 172.21.15.38:0/3214714739 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde4031600 con 0x7fcdec0d0460 2024-04-25T08:41:05.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.432+0000 7fcdea7fc640 1 -- 172.21.15.38:0/3214714739 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde40319d0 con 0x7fcdec0d0460 2024-04-25T08:41:05.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.432+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/3214714739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdec0d0460 msgr2=0x7fcdec15fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:05.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.432+0000 7fcdf1f6a640 1 --2- 172.21.15.38:0/3214714739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdec0d0460 0x7fcdec15fa20 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009e50 tx=0x7fcde402a5f0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.432+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/3214714739 shutdown_connections 2024-04-25T08:41:05.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.432+0000 7fcdf1f6a640 1 --2- 172.21.15.38:0/3214714739 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdec0d0460 0x7fcdec15fa20 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.432+0000 7fcdf1f6a640 1 --2- 172.21.15.38:0/3214714739 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdec09ab90 0x7fcdec15f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.432+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/3214714739 >> 172.21.15.38:0/3214714739 conn(0x7fcdec0b31f0 msgr2=0x7fcdec0b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.432+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/3214714739 shutdown_connections 2024-04-25T08:41:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.432+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/3214714739 wait complete. 2024-04-25T08:41:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.433+0000 7fcdf1f6a640 1 Processor -- start 2024-04-25T08:41:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.434+0000 7fcdf1f6a640 1 -- start start 2024-04-25T08:41:05.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.434+0000 7fcdf1f6a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdec09ab90 0x7fcdec0cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:05.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.434+0000 7fcdf1f6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdec0d0460 0x7fcdec15bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:05.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.434+0000 7fcdf1f6a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdec15bb10 0x7fcdec156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:05.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.434+0000 7fcdf1f6a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec0c5250 con 0x7fcdec0d0460 2024-04-25T08:41:05.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.434+0000 7fcdf1f6a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec0cfc60 con 0x7fcdec09ab90 2024-04-25T08:41:05.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.434+0000 7fcdf1f6a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec0c53d0 con 0x7fcdec15bb10 2024-04-25T08:41:05.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.434+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdec0d0460 0x7fcdec15bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:05.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.434+0000 7fcdeaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdec0d0460 0x7fcdec15bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47856/0 (socket says 172.21.15.38:47856) 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.434+0000 7fcdeaffd640 1 -- 172.21.15.38:0/2076425350 learned_addr learned my addr 172.21.15.38:0/2076425350 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdebfff640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdec15bb10 0x7fcdec156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdeaffd640 1 -- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdec15bb10 msgr2=0x7fcdec156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdeaffd640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdec15bb10 0x7fcdec156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdeaffd640 1 -- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdec09ab90 msgr2=0x7fcdec0cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdeb7fe640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdec09ab90 0x7fcdec0cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdeaffd640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdec09ab90 0x7fcdec0cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdeaffd640 1 -- 172.21.15.38:0/2076425350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec15c430 con 0x7fcdec0d0460 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdebfff640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdec15bb10 0x7fcdec156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdeb7fe640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdec09ab90 0x7fcdec0cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdeaffd640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdec0d0460 0x7fcdec15bef0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fcde000d800 tx=0x7fcde000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdd3fff640 1 -- 172.21.15.38:0/2076425350 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde000b6c0 con 0x7fcdec0d0460 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdd3fff640 1 -- 172.21.15.38:0/2076425350 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde0019070 con 0x7fcdec0d0460 2024-04-25T08:41:05.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/2076425350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdec0b4cf0 con 0x7fcdec0d0460 2024-04-25T08:41:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdd3fff640 1 -- 172.21.15.38:0/2076425350 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde000bac0 con 0x7fcdec0d0460 2024-04-25T08:41:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.435+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/2076425350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec15c990 con 0x7fcdec0d0460 2024-04-25T08:41:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.436+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/2076425350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdec068e80 con 0x7fcdec0d0460 2024-04-25T08:41:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.437+0000 7fcdd3fff640 1 -- 172.21.15.38:0/2076425350 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcde000bc60 con 0x7fcdec0d0460 2024-04-25T08:41:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.437+0000 7fcdd3fff640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcdd806b4d0 0x7fcdd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.437+0000 7fcdd3fff640 1 -- 172.21.15.38:0/2076425350 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcde008a860 con 0x7fcdec0d0460 2024-04-25T08:41:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.437+0000 7fcdeb7fe640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcdd806b4d0 0x7fcdd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.438+0000 7fcdeb7fe640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcdd806b4d0 0x7fcdd806d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0015e0 tx=0x7fcddc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:05.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.441+0000 7fcdd3fff640 1 -- 172.21.15.38:0/2076425350 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdec15c430 con 0x7fcdec0d0460 2024-04-25T08:41:05.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.660+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/2076425350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdec0cdbf0 con 0x7fcdec0d0460 2024-04-25T08:41:05.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.661+0000 7fcdd3fff640 1 -- 172.21.15.38:0/2076425350 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fcde0059540 con 0x7fcdec0d0460 2024-04-25T08:41:05.654 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:05.654 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:40:55.730538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/1312063561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1312063561},{"type":"v1","addr":"172.21.15.38:6835","nonce":1312063561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:05.654 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 77 2024-04-25T08:41:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.663+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcdd806b4d0 msgr2=0x7fcdd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.663+0000 7fcdf1f6a640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcdd806b4d0 0x7fcdd806d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0015e0 tx=0x7fcddc009040 comp rx=0 tx=0).stop 2024-04-25T08:41:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.663+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/2076425350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdec0d0460 msgr2=0x7fcdec15bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.663+0000 7fcdf1f6a640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdec0d0460 0x7fcdec15bef0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fcde000d800 tx=0x7fcde000dcc0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.663+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/2076425350 shutdown_connections 2024-04-25T08:41:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.663+0000 7fcdf1f6a640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcdd806b4d0 0x7fcdd806d990 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.663+0000 7fcdf1f6a640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdec15bb10 0x7fcdec156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.663+0000 7fcdf1f6a640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdec0d0460 0x7fcdec15bef0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.663+0000 7fcdf1f6a640 1 --2- 172.21.15.38:0/2076425350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdec09ab90 0x7fcdec0cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.663+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/2076425350 >> 172.21.15.38:0/2076425350 conn(0x7fcdec0b31f0 msgr2=0x7fcdec099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:05.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.663+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/2076425350 shutdown_connections 2024-04-25T08:41:05.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:05.663+0000 7fcdf1f6a640 1 -- 172.21.15.38:0/2076425350 wait complete. 2024-04-25T08:41:05.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:05.667 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:05.667 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:05.667 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T08:41:05.667 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:05.667 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:05.667 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:05.667 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:05.706 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.706 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.707 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121656 s, 86.2 MB/s 2024-04-25T08:41:05.718 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.718 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.718 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957829 s, 109 MB/s 2024-04-25T08:41:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.729 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924945 s, 113 MB/s 2024-04-25T08:41:05.740 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.740 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.740 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809103 s, 130 MB/s 2024-04-25T08:41:05.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.750 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741651 s, 141 MB/s 2024-04-25T08:41:05.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.761 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907503 s, 116 MB/s 2024-04-25T08:41:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.771 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837465 s, 125 MB/s 2024-04-25T08:41:05.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.782 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082464 s, 127 MB/s 2024-04-25T08:41:05.792 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.792 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.792 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804299 s, 130 MB/s 2024-04-25T08:41:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837172 s, 125 MB/s 2024-04-25T08:41:05.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.813 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808114 s, 130 MB/s 2024-04-25T08:41:05.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.824 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882174 s, 119 MB/s 2024-04-25T08:41:05.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.834 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766477 s, 137 MB/s 2024-04-25T08:41:05.846 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.846 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.846 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999037 s, 105 MB/s 2024-04-25T08:41:05.855 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.855 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.855 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710011 s, 148 MB/s 2024-04-25T08:41:05.866 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.866 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.866 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874239 s, 120 MB/s 2024-04-25T08:41:05.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.877 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876073 s, 120 MB/s 2024-04-25T08:41:05.888 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.889 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.889 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943683 s, 111 MB/s 2024-04-25T08:41:05.899 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.899 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.899 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819097 s, 128 MB/s 2024-04-25T08:41:05.909 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.909 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.909 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798944 s, 131 MB/s 2024-04-25T08:41:05.919 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.919 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.919 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082328 s, 127 MB/s 2024-04-25T08:41:05.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.930 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805062 s, 130 MB/s 2024-04-25T08:41:05.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.940 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773513 s, 136 MB/s 2024-04-25T08:41:05.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.951 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.951 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086968 s, 121 MB/s 2024-04-25T08:41:05.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.960 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710073 s, 148 MB/s 2024-04-25T08:41:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.970 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080944 s, 130 MB/s 2024-04-25T08:41:05.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.980 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826103 s, 127 MB/s 2024-04-25T08:41:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:05.990 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715594 s, 147 MB/s 2024-04-25T08:41:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.001 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872633 s, 120 MB/s 2024-04-25T08:41:06.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.011 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816067 s, 128 MB/s 2024-04-25T08:41:06.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.022 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090347 s, 116 MB/s 2024-04-25T08:41:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.032 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715424 s, 147 MB/s 2024-04-25T08:41:06.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:06.033 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:06.033 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:06.033 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T08:41:06.033 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:06.033 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:06.033 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:06.033 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:06.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.101 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823708 s, 127 MB/s 2024-04-25T08:41:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854352 s, 123 MB/s 2024-04-25T08:41:06.122 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.122 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.123 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866032 s, 121 MB/s 2024-04-25T08:41:06.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.133 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854965 s, 123 MB/s 2024-04-25T08:41:06.143 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.143 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.143 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812111 s, 129 MB/s 2024-04-25T08:41:06.153 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.153 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.153 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774504 s, 135 MB/s 2024-04-25T08:41:06.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.162 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698866 s, 150 MB/s 2024-04-25T08:41:06.172 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.172 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.173 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766996 s, 137 MB/s 2024-04-25T08:41:06.183 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.183 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.183 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857754 s, 122 MB/s 2024-04-25T08:41:06.192 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.193 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.193 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715884 s, 146 MB/s 2024-04-25T08:41:06.202 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.203 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783998 s, 134 MB/s 2024-04-25T08:41:06.213 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.213 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.213 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805771 s, 130 MB/s 2024-04-25T08:41:06.223 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.223 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.223 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081701 s, 128 MB/s 2024-04-25T08:41:06.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.235 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885031 s, 118 MB/s 2024-04-25T08:41:06.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.245 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815046 s, 129 MB/s 2024-04-25T08:41:06.256 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.256 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.256 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871256 s, 120 MB/s 2024-04-25T08:41:06.265 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.265 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.265 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725134 s, 145 MB/s 2024-04-25T08:41:06.275 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.275 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.275 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793048 s, 132 MB/s 2024-04-25T08:41:06.286 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.286 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.286 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831975 s, 126 MB/s 2024-04-25T08:41:06.296 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.296 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.296 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795997 s, 132 MB/s 2024-04-25T08:41:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.306 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773844 s, 136 MB/s 2024-04-25T08:41:06.316 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.316 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.316 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804827 s, 130 MB/s 2024-04-25T08:41:06.327 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.327 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.327 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881677 s, 119 MB/s 2024-04-25T08:41:06.337 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.338 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.338 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822801 s, 127 MB/s 2024-04-25T08:41:06.348 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.348 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.348 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803415 s, 131 MB/s 2024-04-25T08:41:06.359 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.359 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.359 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090556 s, 116 MB/s 2024-04-25T08:41:06.369 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.369 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.369 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079589 s, 132 MB/s 2024-04-25T08:41:06.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.379 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.379 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713681 s, 147 MB/s 2024-04-25T08:41:06.389 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.389 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.389 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803551 s, 130 MB/s 2024-04-25T08:41:06.399 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.399 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.399 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771312 s, 136 MB/s 2024-04-25T08:41:06.409 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.409 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.409 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808662 s, 130 MB/s 2024-04-25T08:41:06.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.419 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788664 s, 133 MB/s 2024-04-25T08:41:06.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:06.420 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:06.420 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:06.420 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T08:41:06.420 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:06.420 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:06.421 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:06.421 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:06.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.490 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.490 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893582 s, 117 MB/s 2024-04-25T08:41:06.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.501 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.501 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873759 s, 120 MB/s 2024-04-25T08:41:06.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.511 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846626 s, 124 MB/s 2024-04-25T08:41:06.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.522 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884201 s, 119 MB/s 2024-04-25T08:41:06.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.533 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866417 s, 121 MB/s 2024-04-25T08:41:06.544 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.544 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.544 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842595 s, 124 MB/s 2024-04-25T08:41:06.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.554 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775022 s, 135 MB/s 2024-04-25T08:41:06.565 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.565 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.565 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086793 s, 121 MB/s 2024-04-25T08:41:06.575 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.575 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.575 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811928 s, 129 MB/s 2024-04-25T08:41:06.586 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.586 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.586 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866621 s, 121 MB/s 2024-04-25T08:41:06.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.596 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787622 s, 133 MB/s 2024-04-25T08:41:06.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.607 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898357 s, 117 MB/s 2024-04-25T08:41:06.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.617 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725776 s, 144 MB/s 2024-04-25T08:41:06.627 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.627 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.627 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787217 s, 133 MB/s 2024-04-25T08:41:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.639 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00986763 s, 106 MB/s 2024-04-25T08:41:06.650 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.650 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.650 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828315 s, 127 MB/s 2024-04-25T08:41:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.660 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797539 s, 131 MB/s 2024-04-25T08:41:06.670 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.670 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.670 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077203 s, 136 MB/s 2024-04-25T08:41:06.680 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.680 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.680 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814886 s, 129 MB/s 2024-04-25T08:41:06.690 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.690 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.690 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759733 s, 138 MB/s 2024-04-25T08:41:06.700 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.700 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.700 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786879 s, 133 MB/s 2024-04-25T08:41:06.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.711 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841454 s, 125 MB/s 2024-04-25T08:41:06.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.721 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762967 s, 137 MB/s 2024-04-25T08:41:06.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.731 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783173 s, 134 MB/s 2024-04-25T08:41:06.740 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.740 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.740 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698299 s, 150 MB/s 2024-04-25T08:41:06.750 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.750 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.750 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794669 s, 132 MB/s 2024-04-25T08:41:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.760 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758844 s, 138 MB/s 2024-04-25T08:41:06.769 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.770 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.770 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730589 s, 144 MB/s 2024-04-25T08:41:06.779 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.779 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.779 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742906 s, 141 MB/s 2024-04-25T08:41:06.789 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.789 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.790 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780873 s, 134 MB/s 2024-04-25T08:41:06.800 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.800 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.800 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839872 s, 125 MB/s 2024-04-25T08:41:06.810 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.810 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.810 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782319 s, 134 MB/s 2024-04-25T08:41:06.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:06.811 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:06.811 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:06.811 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T08:41:06.811 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:06.811 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:06.812 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:06.812 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:06.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.880 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796709 s, 132 MB/s 2024-04-25T08:41:06.890 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.890 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.891 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873148 s, 120 MB/s 2024-04-25T08:41:06.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.901 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796401 s, 132 MB/s 2024-04-25T08:41:06.911 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.911 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.911 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784754 s, 134 MB/s 2024-04-25T08:41:06.921 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.921 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.921 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812659 s, 129 MB/s 2024-04-25T08:41:06.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.931 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801889 s, 131 MB/s 2024-04-25T08:41:06.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.941 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733503 s, 143 MB/s 2024-04-25T08:41:06.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.951 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718438 s, 146 MB/s 2024-04-25T08:41:06.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.961 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787246 s, 133 MB/s 2024-04-25T08:41:06.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.971 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789076 s, 133 MB/s 2024-04-25T08:41:06.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.980 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733777 s, 143 MB/s 2024-04-25T08:41:06.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:06.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:06.990 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776797 s, 135 MB/s 2024-04-25T08:41:07.000 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.000 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.001 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788836 s, 133 MB/s 2024-04-25T08:41:07.010 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.010 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.010 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761018 s, 138 MB/s 2024-04-25T08:41:07.020 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.020 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.020 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784817 s, 134 MB/s 2024-04-25T08:41:07.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.031 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862155 s, 122 MB/s 2024-04-25T08:41:07.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.041 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709665 s, 148 MB/s 2024-04-25T08:41:07.050 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.051 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769267 s, 136 MB/s 2024-04-25T08:41:07.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.060 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075245 s, 139 MB/s 2024-04-25T08:41:07.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.070 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073941 s, 142 MB/s 2024-04-25T08:41:07.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.080 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794128 s, 132 MB/s 2024-04-25T08:41:07.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.090 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776958 s, 135 MB/s 2024-04-25T08:41:07.100 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.100 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.100 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077976 s, 134 MB/s 2024-04-25T08:41:07.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.110 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070413 s, 149 MB/s 2024-04-25T08:41:07.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.120 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083782 s, 125 MB/s 2024-04-25T08:41:07.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.130 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080745 s, 130 MB/s 2024-04-25T08:41:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.141 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085156 s, 123 MB/s 2024-04-25T08:41:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.151 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777848 s, 135 MB/s 2024-04-25T08:41:07.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.162 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854696 s, 123 MB/s 2024-04-25T08:41:07.172 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.172 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.172 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783752 s, 134 MB/s 2024-04-25T08:41:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.182 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755745 s, 139 MB/s 2024-04-25T08:41:07.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.192 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737865 s, 142 MB/s 2024-04-25T08:41:07.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:07.193 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:07.193 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:07.193 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T08:41:07.193 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:07.193 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:07.193 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:07.194 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:07.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.263 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009173 s, 114 MB/s 2024-04-25T08:41:07.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.273 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763824 s, 137 MB/s 2024-04-25T08:41:07.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.283 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079015 s, 133 MB/s 2024-04-25T08:41:07.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.294 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.294 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857391 s, 122 MB/s 2024-04-25T08:41:07.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.303 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724075 s, 145 MB/s 2024-04-25T08:41:07.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.313 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761913 s, 138 MB/s 2024-04-25T08:41:07.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.323 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762393 s, 138 MB/s 2024-04-25T08:41:07.333 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.333 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.333 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808185 s, 130 MB/s 2024-04-25T08:41:07.343 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.343 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.343 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715656 s, 147 MB/s 2024-04-25T08:41:07.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.352 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726909 s, 144 MB/s 2024-04-25T08:41:07.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.363 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891122 s, 118 MB/s 2024-04-25T08:41:07.374 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.374 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.374 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850403 s, 123 MB/s 2024-04-25T08:41:07.385 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.385 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.385 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838843 s, 125 MB/s 2024-04-25T08:41:07.394 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.395 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.395 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780425 s, 134 MB/s 2024-04-25T08:41:07.404 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.404 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.404 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738835 s, 142 MB/s 2024-04-25T08:41:07.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.414 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734153 s, 143 MB/s 2024-04-25T08:41:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774804 s, 135 MB/s 2024-04-25T08:41:07.433 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.434 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773226 s, 136 MB/s 2024-04-25T08:41:07.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.444 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861407 s, 122 MB/s 2024-04-25T08:41:07.454 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.455 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.455 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792009 s, 132 MB/s 2024-04-25T08:41:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.465 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808266 s, 130 MB/s 2024-04-25T08:41:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.476 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912074 s, 115 MB/s 2024-04-25T08:41:07.486 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.486 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.487 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806815 s, 130 MB/s 2024-04-25T08:41:07.497 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.497 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.498 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848386 s, 124 MB/s 2024-04-25T08:41:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.507 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763825 s, 137 MB/s 2024-04-25T08:41:07.516 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.516 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.516 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720221 s, 146 MB/s 2024-04-25T08:41:07.526 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.526 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.527 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793287 s, 132 MB/s 2024-04-25T08:41:07.536 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.536 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.537 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771353 s, 136 MB/s 2024-04-25T08:41:07.548 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.548 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.548 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910917 s, 115 MB/s 2024-04-25T08:41:07.559 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.559 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.559 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086253 s, 122 MB/s 2024-04-25T08:41:07.570 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.570 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.570 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876365 s, 120 MB/s 2024-04-25T08:41:07.579 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.580 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.580 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769476 s, 136 MB/s 2024-04-25T08:41:07.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:07.581 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:07.581 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:07.581 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T08:41:07.581 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:07.581 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:07.581 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:07.581 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:07.650 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.651 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857763 s, 122 MB/s 2024-04-25T08:41:07.660 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.661 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.661 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782041 s, 134 MB/s 2024-04-25T08:41:07.671 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.672 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896669 s, 117 MB/s 2024-04-25T08:41:07.682 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.683 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870538 s, 120 MB/s 2024-04-25T08:41:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.692 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071396 s, 147 MB/s 2024-04-25T08:41:07.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.702 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769351 s, 136 MB/s 2024-04-25T08:41:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.711 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722195 s, 145 MB/s 2024-04-25T08:41:07.720 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.721 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709154 s, 148 MB/s 2024-04-25T08:41:07.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.731 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804152 s, 130 MB/s 2024-04-25T08:41:07.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.742 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877911 s, 119 MB/s 2024-04-25T08:41:07.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.751 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719017 s, 146 MB/s 2024-04-25T08:41:07.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.762 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788074 s, 133 MB/s 2024-04-25T08:41:07.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.771 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792558 s, 132 MB/s 2024-04-25T08:41:07.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.781 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739329 s, 142 MB/s 2024-04-25T08:41:07.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.791 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761117 s, 138 MB/s 2024-04-25T08:41:07.801 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.801 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.801 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805773 s, 130 MB/s 2024-04-25T08:41:07.811 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.811 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.812 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080236 s, 131 MB/s 2024-04-25T08:41:07.821 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.821 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.821 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077446 s, 135 MB/s 2024-04-25T08:41:07.831 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.831 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.832 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783976 s, 134 MB/s 2024-04-25T08:41:07.841 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.842 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.842 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793807 s, 132 MB/s 2024-04-25T08:41:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.851 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764073 s, 137 MB/s 2024-04-25T08:41:07.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.862 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815528 s, 129 MB/s 2024-04-25T08:41:07.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.873 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866116 s, 121 MB/s 2024-04-25T08:41:07.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.883 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.883 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768635 s, 136 MB/s 2024-04-25T08:41:07.894 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.894 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.894 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920787 s, 114 MB/s 2024-04-25T08:41:07.903 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.904 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.904 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733166 s, 143 MB/s 2024-04-25T08:41:07.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.914 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007612 s, 138 MB/s 2024-04-25T08:41:07.924 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.924 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.924 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080932 s, 130 MB/s 2024-04-25T08:41:07.934 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.934 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.934 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771531 s, 136 MB/s 2024-04-25T08:41:07.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.944 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747687 s, 140 MB/s 2024-04-25T08:41:07.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.955 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863871 s, 121 MB/s 2024-04-25T08:41:07.964 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:07.964 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:07.964 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676458 s, 155 MB/s 2024-04-25T08:41:07.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:07.965 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:07.965 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:07.965 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T08:41:07.965 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:07.965 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:07.966 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:07.966 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.033 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761962 s, 138 MB/s 2024-04-25T08:41:08.043 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.043 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.043 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074927 s, 140 MB/s 2024-04-25T08:41:08.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.052 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684279 s, 153 MB/s 2024-04-25T08:41:08.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.062 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794262 s, 132 MB/s 2024-04-25T08:41:08.071 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.071 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.072 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719302 s, 146 MB/s 2024-04-25T08:41:08.081 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.081 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.081 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076325 s, 137 MB/s 2024-04-25T08:41:08.091 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.091 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.091 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780025 s, 134 MB/s 2024-04-25T08:41:08.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.101 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754122 s, 139 MB/s 2024-04-25T08:41:08.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.110 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727904 s, 144 MB/s 2024-04-25T08:41:08.119 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.119 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.119 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686507 s, 153 MB/s 2024-04-25T08:41:08.129 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.129 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.129 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738855 s, 142 MB/s 2024-04-25T08:41:08.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.138 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729867 s, 144 MB/s 2024-04-25T08:41:08.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.149 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.149 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821765 s, 128 MB/s 2024-04-25T08:41:08.159 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.159 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.159 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835182 s, 126 MB/s 2024-04-25T08:41:08.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.168 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657459 s, 159 MB/s 2024-04-25T08:41:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744421 s, 141 MB/s 2024-04-25T08:41:08.187 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.187 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.187 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757186 s, 138 MB/s 2024-04-25T08:41:08.196 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.196 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.196 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674119 s, 156 MB/s 2024-04-25T08:41:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.206 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728325 s, 144 MB/s 2024-04-25T08:41:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774138 s, 135 MB/s 2024-04-25T08:41:08.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.225 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699268 s, 150 MB/s 2024-04-25T08:41:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757314 s, 138 MB/s 2024-04-25T08:41:08.243 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.244 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.244 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682888 s, 154 MB/s 2024-04-25T08:41:08.253 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.253 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.253 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072031 s, 146 MB/s 2024-04-25T08:41:08.262 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.262 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.262 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663237 s, 158 MB/s 2024-04-25T08:41:08.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.273 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876061 s, 120 MB/s 2024-04-25T08:41:08.282 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.282 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.282 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734699 s, 143 MB/s 2024-04-25T08:41:08.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.292 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720031 s, 146 MB/s 2024-04-25T08:41:08.300 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.301 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.301 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671206 s, 156 MB/s 2024-04-25T08:41:08.311 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.311 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.311 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836575 s, 125 MB/s 2024-04-25T08:41:08.320 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.320 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.321 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717447 s, 146 MB/s 2024-04-25T08:41:08.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.330 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076006 s, 138 MB/s 2024-04-25T08:41:08.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:08.331 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:08.331 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:08.332 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T08:41:08.332 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:08.332 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:08.332 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:08.332 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:08.398 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.398 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.398 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676738 s, 155 MB/s 2024-04-25T08:41:08.409 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.409 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.409 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836639 s, 125 MB/s 2024-04-25T08:41:08.418 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.419 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074358 s, 141 MB/s 2024-04-25T08:41:08.428 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.428 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.428 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731597 s, 143 MB/s 2024-04-25T08:41:08.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.437 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716178 s, 146 MB/s 2024-04-25T08:41:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681315 s, 154 MB/s 2024-04-25T08:41:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.456 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750979 s, 140 MB/s 2024-04-25T08:41:08.466 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.466 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.466 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820985 s, 128 MB/s 2024-04-25T08:41:08.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.476 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773712 s, 136 MB/s 2024-04-25T08:41:08.486 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.486 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.486 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733226 s, 143 MB/s 2024-04-25T08:41:08.495 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.495 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.495 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692962 s, 151 MB/s 2024-04-25T08:41:08.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.504 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.504 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664389 s, 158 MB/s 2024-04-25T08:41:08.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.513 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725431 s, 145 MB/s 2024-04-25T08:41:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.522 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00646425 s, 162 MB/s 2024-04-25T08:41:08.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.532 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757952 s, 138 MB/s 2024-04-25T08:41:08.541 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.542 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784155 s, 134 MB/s 2024-04-25T08:41:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.552 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803768 s, 130 MB/s 2024-04-25T08:41:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762197 s, 138 MB/s 2024-04-25T08:41:08.572 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.572 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.572 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806438 s, 130 MB/s 2024-04-25T08:41:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.581 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746481 s, 140 MB/s 2024-04-25T08:41:08.591 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.591 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.591 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720286 s, 146 MB/s 2024-04-25T08:41:08.600 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.600 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.600 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689295 s, 152 MB/s 2024-04-25T08:41:08.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.609 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729723 s, 144 MB/s 2024-04-25T08:41:08.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.620 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845259 s, 124 MB/s 2024-04-25T08:41:08.630 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.630 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.630 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802121 s, 131 MB/s 2024-04-25T08:41:08.640 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.640 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.640 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713109 s, 147 MB/s 2024-04-25T08:41:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.650 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761083 s, 138 MB/s 2024-04-25T08:41:08.660 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.660 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.660 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754515 s, 139 MB/s 2024-04-25T08:41:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743337 s, 141 MB/s 2024-04-25T08:41:08.678 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.678 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.678 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695774 s, 151 MB/s 2024-04-25T08:41:08.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.688 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728292 s, 144 MB/s 2024-04-25T08:41:08.698 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.698 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.698 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810504 s, 129 MB/s 2024-04-25T08:41:08.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:08.699 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:08.699 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:08.699 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T08:41:08.699 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:08.699 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:08.699 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:08.699 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083706 s, 125 MB/s 2024-04-25T08:41:08.777 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.777 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.777 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754993 s, 139 MB/s 2024-04-25T08:41:08.787 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.787 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.787 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760215 s, 138 MB/s 2024-04-25T08:41:08.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.796 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745445 s, 141 MB/s 2024-04-25T08:41:08.806 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.806 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.806 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750534 s, 140 MB/s 2024-04-25T08:41:08.815 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.816 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.816 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744292 s, 141 MB/s 2024-04-25T08:41:08.826 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.826 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.826 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772279 s, 136 MB/s 2024-04-25T08:41:08.835 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.835 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.835 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668683 s, 157 MB/s 2024-04-25T08:41:08.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.845 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.845 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728438 s, 144 MB/s 2024-04-25T08:41:08.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.854 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740903 s, 142 MB/s 2024-04-25T08:41:08.864 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.864 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.864 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767288 s, 137 MB/s 2024-04-25T08:41:08.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652212 s, 161 MB/s 2024-04-25T08:41:08.883 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.883 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.883 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082609 s, 127 MB/s 2024-04-25T08:41:08.894 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.894 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.894 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869743 s, 121 MB/s 2024-04-25T08:41:08.904 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.904 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.904 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782162 s, 134 MB/s 2024-04-25T08:41:08.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.914 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728882 s, 144 MB/s 2024-04-25T08:41:08.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.923 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728721 s, 144 MB/s 2024-04-25T08:41:08.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.933 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729908 s, 144 MB/s 2024-04-25T08:41:08.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.943 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740576 s, 142 MB/s 2024-04-25T08:41:08.953 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.953 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.953 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791248 s, 133 MB/s 2024-04-25T08:41:08.962 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.962 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.963 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741441 s, 141 MB/s 2024-04-25T08:41:08.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.972 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719661 s, 146 MB/s 2024-04-25T08:41:08.981 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.981 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.982 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731783 s, 143 MB/s 2024-04-25T08:41:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:08.991 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073505 s, 143 MB/s 2024-04-25T08:41:09.000 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.000 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.000 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714221 s, 147 MB/s 2024-04-25T08:41:09.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.011 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832356 s, 126 MB/s 2024-04-25T08:41:09.021 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.021 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.021 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745582 s, 141 MB/s 2024-04-25T08:41:09.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.031 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774419 s, 135 MB/s 2024-04-25T08:41:09.040 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.040 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.040 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661226 s, 159 MB/s 2024-04-25T08:41:09.050 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.050 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.050 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814424 s, 129 MB/s 2024-04-25T08:41:09.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.060 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761812 s, 138 MB/s 2024-04-25T08:41:09.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.070 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741422 s, 141 MB/s 2024-04-25T08:41:09.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:09.071 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:09.071 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:09.071 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T08:41:09.071 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:09.071 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:09.071 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:09.072 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:09.137 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.138 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680432 s, 154 MB/s 2024-04-25T08:41:09.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792602 s, 132 MB/s 2024-04-25T08:41:09.157 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.157 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.157 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700862 s, 150 MB/s 2024-04-25T08:41:09.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.167 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747157 s, 140 MB/s 2024-04-25T08:41:09.176 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746536 s, 140 MB/s 2024-04-25T08:41:09.186 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.186 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.187 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765807 s, 137 MB/s 2024-04-25T08:41:09.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.197 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825427 s, 127 MB/s 2024-04-25T08:41:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.208 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839077 s, 125 MB/s 2024-04-25T08:41:09.217 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.217 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.217 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731435 s, 143 MB/s 2024-04-25T08:41:09.226 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.226 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.226 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719088 s, 146 MB/s 2024-04-25T08:41:09.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.237 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850026 s, 123 MB/s 2024-04-25T08:41:09.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.248 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.248 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826034 s, 127 MB/s 2024-04-25T08:41:09.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.257 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750552 s, 140 MB/s 2024-04-25T08:41:09.266 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.266 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.266 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673421 s, 156 MB/s 2024-04-25T08:41:09.275 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.275 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.275 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661581 s, 158 MB/s 2024-04-25T08:41:09.285 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.285 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.286 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816798 s, 128 MB/s 2024-04-25T08:41:09.295 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.295 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.296 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771442 s, 136 MB/s 2024-04-25T08:41:09.305 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.305 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.305 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732562 s, 143 MB/s 2024-04-25T08:41:09.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.314 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.314 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729774 s, 144 MB/s 2024-04-25T08:41:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798507 s, 131 MB/s 2024-04-25T08:41:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728825 s, 144 MB/s 2024-04-25T08:41:09.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.344 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773445 s, 136 MB/s 2024-04-25T08:41:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796729 s, 132 MB/s 2024-04-25T08:41:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.366 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816344 s, 128 MB/s 2024-04-25T08:41:09.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.378 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896648 s, 117 MB/s 2024-04-25T08:41:09.389 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.389 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.389 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080623 s, 130 MB/s 2024-04-25T08:41:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.401 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930733 s, 113 MB/s 2024-04-25T08:41:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.413 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00928169 s, 113 MB/s 2024-04-25T08:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.424 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822476 s, 127 MB/s 2024-04-25T08:41:09.433 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.433 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.433 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699846 s, 150 MB/s 2024-04-25T08:41:09.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.443 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732834 s, 143 MB/s 2024-04-25T08:41:09.452 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.452 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.452 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725166 s, 145 MB/s 2024-04-25T08:41:09.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:09.454 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:09.454 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:09.454 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T08:41:09.454 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:09.454 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:09.454 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:09.454 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:09.521 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.522 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796163 s, 132 MB/s 2024-04-25T08:41:09.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.532 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778199 s, 135 MB/s 2024-04-25T08:41:09.541 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.541 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.541 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734993 s, 143 MB/s 2024-04-25T08:41:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.554 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726519 s, 144 MB/s 2024-04-25T08:41:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733116 s, 143 MB/s 2024-04-25T08:41:09.572 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.572 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.572 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707362 s, 148 MB/s 2024-04-25T08:41:09.582 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.582 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.582 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720533 s, 146 MB/s 2024-04-25T08:41:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762231 s, 138 MB/s 2024-04-25T08:41:09.601 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.601 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.601 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699387 s, 150 MB/s 2024-04-25T08:41:09.610 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.611 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.611 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730926 s, 143 MB/s 2024-04-25T08:41:09.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.620 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751723 s, 139 MB/s 2024-04-25T08:41:09.631 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.631 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.631 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831627 s, 126 MB/s 2024-04-25T08:41:09.640 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.640 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.640 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718345 s, 146 MB/s 2024-04-25T08:41:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.651 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854519 s, 123 MB/s 2024-04-25T08:41:09.661 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.661 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.661 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803951 s, 130 MB/s 2024-04-25T08:41:09.670 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.670 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.671 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687556 s, 153 MB/s 2024-04-25T08:41:09.680 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.680 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.680 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736328 s, 142 MB/s 2024-04-25T08:41:09.689 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.689 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.689 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716475 s, 146 MB/s 2024-04-25T08:41:09.698 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.699 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.702 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703673 s, 149 MB/s 2024-04-25T08:41:09.708 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.709 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.709 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761998 s, 138 MB/s 2024-04-25T08:41:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737962 s, 142 MB/s 2024-04-25T08:41:09.728 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.728 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.728 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767433 s, 137 MB/s 2024-04-25T08:41:09.737 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.737 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.737 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746302 s, 141 MB/s 2024-04-25T08:41:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.747 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722152 s, 145 MB/s 2024-04-25T08:41:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727402 s, 144 MB/s 2024-04-25T08:41:09.766 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.766 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.766 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744712 s, 141 MB/s 2024-04-25T08:41:09.775 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.775 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.775 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725249 s, 145 MB/s 2024-04-25T08:41:09.786 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.786 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.786 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851193 s, 123 MB/s 2024-04-25T08:41:09.795 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.795 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.795 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742371 s, 141 MB/s 2024-04-25T08:41:09.805 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.805 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.805 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755812 s, 139 MB/s 2024-04-25T08:41:09.815 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.815 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.815 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780314 s, 134 MB/s 2024-04-25T08:41:09.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.824 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664841 s, 158 MB/s 2024-04-25T08:41:09.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:09.825 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:09.825 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:09.825 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T08:41:09.825 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:09.825 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:09.825 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:09.825 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:09.893 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.893 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.894 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766809 s, 137 MB/s 2024-04-25T08:41:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.904 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844902 s, 124 MB/s 2024-04-25T08:41:09.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.913 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007518 s, 139 MB/s 2024-04-25T08:41:09.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.923 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712768 s, 147 MB/s 2024-04-25T08:41:09.932 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.932 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.933 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741231 s, 141 MB/s 2024-04-25T08:41:09.942 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.942 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.942 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716893 s, 146 MB/s 2024-04-25T08:41:09.951 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.951 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.951 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723357 s, 145 MB/s 2024-04-25T08:41:09.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.961 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709672 s, 148 MB/s 2024-04-25T08:41:09.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.970 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677941 s, 155 MB/s 2024-04-25T08:41:09.979 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.979 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.979 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731039 s, 143 MB/s 2024-04-25T08:41:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.988 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652341 s, 161 MB/s 2024-04-25T08:41:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:09.998 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720113 s, 146 MB/s 2024-04-25T08:41:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.008 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832676 s, 126 MB/s 2024-04-25T08:41:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762907 s, 137 MB/s 2024-04-25T08:41:10.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.027 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719595 s, 146 MB/s 2024-04-25T08:41:10.036 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.036 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.036 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689737 s, 152 MB/s 2024-04-25T08:41:10.046 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.047 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.047 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812457 s, 129 MB/s 2024-04-25T08:41:10.057 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.057 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.057 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798203 s, 131 MB/s 2024-04-25T08:41:10.066 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.066 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.066 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742289 s, 141 MB/s 2024-04-25T08:41:10.076 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.076 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.076 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722096 s, 145 MB/s 2024-04-25T08:41:10.085 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.085 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.086 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775879 s, 135 MB/s 2024-04-25T08:41:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.095 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074383 s, 141 MB/s 2024-04-25T08:41:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.104 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708626 s, 148 MB/s 2024-04-25T08:41:10.113 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.114 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.114 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704473 s, 149 MB/s 2024-04-25T08:41:10.123 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.123 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.123 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752133 s, 139 MB/s 2024-04-25T08:41:10.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.134 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816748 s, 128 MB/s 2024-04-25T08:41:10.143 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.143 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.143 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715654 s, 147 MB/s 2024-04-25T08:41:10.152 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.152 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.152 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748307 s, 140 MB/s 2024-04-25T08:41:10.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.162 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072153 s, 145 MB/s 2024-04-25T08:41:10.171 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.171 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.171 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071276 s, 147 MB/s 2024-04-25T08:41:10.180 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.181 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.181 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741316 s, 141 MB/s 2024-04-25T08:41:10.190 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.190 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.190 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700701 s, 150 MB/s 2024-04-25T08:41:10.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:10.191 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:10.191 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:10.191 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T08:41:10.191 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:10.191 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:10.191 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:10.191 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.260 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791174 s, 133 MB/s 2024-04-25T08:41:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.269 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727004 s, 144 MB/s 2024-04-25T08:41:10.279 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.279 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.279 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741314 s, 141 MB/s 2024-04-25T08:41:10.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.288 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660605 s, 159 MB/s 2024-04-25T08:41:10.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.298 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.298 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727796 s, 144 MB/s 2024-04-25T08:41:10.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.307 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708371 s, 148 MB/s 2024-04-25T08:41:10.317 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.317 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.317 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782023 s, 134 MB/s 2024-04-25T08:41:10.327 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847589 s, 124 MB/s 2024-04-25T08:41:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.337 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746275 s, 141 MB/s 2024-04-25T08:41:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.346 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00639725 s, 164 MB/s 2024-04-25T08:41:10.355 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.355 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.356 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748319 s, 140 MB/s 2024-04-25T08:41:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.365 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749241 s, 140 MB/s 2024-04-25T08:41:10.374 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.374 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.374 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070769 s, 148 MB/s 2024-04-25T08:41:10.385 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.385 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.385 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082757 s, 127 MB/s 2024-04-25T08:41:10.394 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.395 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.395 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071957 s, 146 MB/s 2024-04-25T08:41:10.405 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.405 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.405 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820188 s, 128 MB/s 2024-04-25T08:41:10.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.414 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710772 s, 148 MB/s 2024-04-25T08:41:10.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.424 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752243 s, 139 MB/s 2024-04-25T08:41:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.434 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791111 s, 133 MB/s 2024-04-25T08:41:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.444 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073142 s, 143 MB/s 2024-04-25T08:41:10.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.453 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747785 s, 140 MB/s 2024-04-25T08:41:10.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.463 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076317 s, 137 MB/s 2024-04-25T08:41:10.473 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.473 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.473 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716208 s, 146 MB/s 2024-04-25T08:41:10.482 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.482 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.482 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720464 s, 146 MB/s 2024-04-25T08:41:10.492 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.492 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.492 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751551 s, 140 MB/s 2024-04-25T08:41:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.501 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.502 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735878 s, 142 MB/s 2024-04-25T08:41:10.512 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.512 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.512 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820866 s, 128 MB/s 2024-04-25T08:41:10.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.523 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835242 s, 126 MB/s 2024-04-25T08:41:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.533 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820857 s, 128 MB/s 2024-04-25T08:41:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068542 s, 153 MB/s 2024-04-25T08:41:10.551 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.551 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.552 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722728 s, 145 MB/s 2024-04-25T08:41:10.561 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.561 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.561 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075199 s, 139 MB/s 2024-04-25T08:41:10.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:10.563 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:10.563 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:10.563 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T08:41:10.563 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:10.563 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:10.563 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:10.563 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:10.631 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.631 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.631 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074396 s, 141 MB/s 2024-04-25T08:41:10.641 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.641 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.641 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845224 s, 124 MB/s 2024-04-25T08:41:10.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.652 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801731 s, 131 MB/s 2024-04-25T08:41:10.661 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.661 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.661 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076058 s, 138 MB/s 2024-04-25T08:41:10.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.672 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083964 s, 125 MB/s 2024-04-25T08:41:10.680 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.680 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.681 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655432 s, 160 MB/s 2024-04-25T08:41:10.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.691 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082988 s, 126 MB/s 2024-04-25T08:41:10.700 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.700 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.700 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679784 s, 154 MB/s 2024-04-25T08:41:10.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.711 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821607 s, 128 MB/s 2024-04-25T08:41:10.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.723 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.723 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832245 s, 126 MB/s 2024-04-25T08:41:10.730 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.730 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.730 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678232 s, 155 MB/s 2024-04-25T08:41:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074568 s, 141 MB/s 2024-04-25T08:41:10.750 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.750 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.750 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819467 s, 128 MB/s 2024-04-25T08:41:10.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.760 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766594 s, 137 MB/s 2024-04-25T08:41:10.769 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.769 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.769 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068097 s, 154 MB/s 2024-04-25T08:41:10.779 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.779 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.779 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751533 s, 140 MB/s 2024-04-25T08:41:10.788 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.788 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.788 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708291 s, 148 MB/s 2024-04-25T08:41:10.798 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.798 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.798 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788132 s, 133 MB/s 2024-04-25T08:41:10.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.809 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819031 s, 128 MB/s 2024-04-25T08:41:10.818 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.818 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.818 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735435 s, 143 MB/s 2024-04-25T08:41:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710267 s, 148 MB/s 2024-04-25T08:41:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843611 s, 124 MB/s 2024-04-25T08:41:10.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.848 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736832 s, 142 MB/s 2024-04-25T08:41:10.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.858 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754209 s, 139 MB/s 2024-04-25T08:41:10.867 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.867 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.867 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733066 s, 143 MB/s 2024-04-25T08:41:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.877 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073033 s, 144 MB/s 2024-04-25T08:41:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.886 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709613 s, 148 MB/s 2024-04-25T08:41:10.896 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.896 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.896 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769929 s, 136 MB/s 2024-04-25T08:41:10.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.906 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745828 s, 141 MB/s 2024-04-25T08:41:10.915 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.915 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.915 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071391 s, 147 MB/s 2024-04-25T08:41:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.930 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117719 s, 89.1 MB/s 2024-04-25T08:41:10.939 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:10.939 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:10.939 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768289 s, 136 MB/s 2024-04-25T08:41:10.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:10.940 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:10.940 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:10.941 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T08:41:10.941 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:10.941 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:10.941 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:10.941 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:11.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.008 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731429 s, 143 MB/s 2024-04-25T08:41:11.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.019 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829482 s, 126 MB/s 2024-04-25T08:41:11.028 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.028 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.028 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762978 s, 137 MB/s 2024-04-25T08:41:11.038 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.038 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.038 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729221 s, 144 MB/s 2024-04-25T08:41:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.048 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827545 s, 127 MB/s 2024-04-25T08:41:11.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.058 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077515 s, 135 MB/s 2024-04-25T08:41:11.067 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.068 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.068 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728091 s, 144 MB/s 2024-04-25T08:41:11.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.077 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716556 s, 146 MB/s 2024-04-25T08:41:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.087 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734572 s, 143 MB/s 2024-04-25T08:41:11.097 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.097 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.097 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824102 s, 127 MB/s 2024-04-25T08:41:11.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.107 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816336 s, 128 MB/s 2024-04-25T08:41:11.117 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.117 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.118 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772775 s, 136 MB/s 2024-04-25T08:41:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.131 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816299 s, 128 MB/s 2024-04-25T08:41:11.137 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.138 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738752 s, 142 MB/s 2024-04-25T08:41:11.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829472 s, 126 MB/s 2024-04-25T08:41:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766266 s, 137 MB/s 2024-04-25T08:41:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.168 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756462 s, 139 MB/s 2024-04-25T08:41:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.177 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663104 s, 158 MB/s 2024-04-25T08:41:11.187 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.188 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821794 s, 128 MB/s 2024-04-25T08:41:11.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.197 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742483 s, 141 MB/s 2024-04-25T08:41:11.207 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.207 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.208 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816761 s, 128 MB/s 2024-04-25T08:41:11.217 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.217 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.217 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759294 s, 138 MB/s 2024-04-25T08:41:11.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.227 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767723 s, 137 MB/s 2024-04-25T08:41:11.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.237 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737034 s, 142 MB/s 2024-04-25T08:41:11.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.247 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749531 s, 140 MB/s 2024-04-25T08:41:11.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.257 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822164 s, 128 MB/s 2024-04-25T08:41:11.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.267 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737972 s, 142 MB/s 2024-04-25T08:41:11.276 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.276 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.276 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736795 s, 142 MB/s 2024-04-25T08:41:11.285 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.285 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.285 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672484 s, 156 MB/s 2024-04-25T08:41:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084036 s, 125 MB/s 2024-04-25T08:41:11.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.306 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793841 s, 132 MB/s 2024-04-25T08:41:11.315 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.316 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732272 s, 143 MB/s 2024-04-25T08:41:11.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:11.317 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:11.317 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:11.317 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T08:41:11.317 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:11.317 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:11.317 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:11.317 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:11.386 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.386 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.386 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770422 s, 136 MB/s 2024-04-25T08:41:11.395 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.395 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.395 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073453 s, 143 MB/s 2024-04-25T08:41:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.404 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674561 s, 155 MB/s 2024-04-25T08:41:11.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.415 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082176 s, 128 MB/s 2024-04-25T08:41:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.424 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757275 s, 138 MB/s 2024-04-25T08:41:11.435 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.435 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.435 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828333 s, 127 MB/s 2024-04-25T08:41:11.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.444 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686966 s, 153 MB/s 2024-04-25T08:41:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.453 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069087 s, 152 MB/s 2024-04-25T08:41:11.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.463 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747099 s, 140 MB/s 2024-04-25T08:41:11.472 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.473 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.473 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724853 s, 145 MB/s 2024-04-25T08:41:11.482 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.482 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.482 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713542 s, 147 MB/s 2024-04-25T08:41:11.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.491 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678724 s, 154 MB/s 2024-04-25T08:41:11.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.500 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658426 s, 159 MB/s 2024-04-25T08:41:11.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.511 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884714 s, 119 MB/s 2024-04-25T08:41:11.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.520 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693959 s, 151 MB/s 2024-04-25T08:41:11.530 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.530 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.530 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791976 s, 132 MB/s 2024-04-25T08:41:11.539 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.539 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.539 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670653 s, 156 MB/s 2024-04-25T08:41:11.550 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.550 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.551 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919304 s, 114 MB/s 2024-04-25T08:41:11.578 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.578 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.578 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723018 s, 145 MB/s 2024-04-25T08:41:11.588 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.588 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.588 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741308 s, 141 MB/s 2024-04-25T08:41:11.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.597 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670598 s, 156 MB/s 2024-04-25T08:41:11.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.606 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736773 s, 142 MB/s 2024-04-25T08:41:11.615 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.616 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729373 s, 144 MB/s 2024-04-25T08:41:11.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.626 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799714 s, 131 MB/s 2024-04-25T08:41:11.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.635 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696247 s, 151 MB/s 2024-04-25T08:41:11.644 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.645 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737876 s, 142 MB/s 2024-04-25T08:41:11.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.655 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820644 s, 128 MB/s 2024-04-25T08:41:11.664 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.665 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709772 s, 148 MB/s 2024-04-25T08:41:11.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.674 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731587 s, 143 MB/s 2024-04-25T08:41:11.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.683 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706414 s, 148 MB/s 2024-04-25T08:41:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.693 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722273 s, 145 MB/s 2024-04-25T08:41:11.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.703 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075456 s, 139 MB/s 2024-04-25T08:41:11.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:11.704 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:11.704 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:11.704 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T08:41:11.704 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:11.704 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:11.704 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:11.704 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.771 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719976 s, 146 MB/s 2024-04-25T08:41:11.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.781 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824848 s, 127 MB/s 2024-04-25T08:41:11.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.791 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734771 s, 143 MB/s 2024-04-25T08:41:11.800 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.800 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.800 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675299 s, 155 MB/s 2024-04-25T08:41:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797325 s, 132 MB/s 2024-04-25T08:41:11.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661294 s, 159 MB/s 2024-04-25T08:41:11.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.828 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694812 s, 151 MB/s 2024-04-25T08:41:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766367 s, 137 MB/s 2024-04-25T08:41:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778852 s, 135 MB/s 2024-04-25T08:41:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.857 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746521 s, 140 MB/s 2024-04-25T08:41:11.867 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.867 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.867 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073715 s, 142 MB/s 2024-04-25T08:41:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.876 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670062 s, 156 MB/s 2024-04-25T08:41:11.885 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.885 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.885 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675586 s, 155 MB/s 2024-04-25T08:41:11.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.895 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739663 s, 142 MB/s 2024-04-25T08:41:11.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.905 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772841 s, 136 MB/s 2024-04-25T08:41:11.914 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.914 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.914 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742306 s, 141 MB/s 2024-04-25T08:41:11.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.924 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.924 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684179 s, 153 MB/s 2024-04-25T08:41:11.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.933 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753655 s, 139 MB/s 2024-04-25T08:41:11.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.944 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008626 s, 122 MB/s 2024-04-25T08:41:11.953 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.953 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.953 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697723 s, 150 MB/s 2024-04-25T08:41:11.962 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.962 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.963 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706386 s, 148 MB/s 2024-04-25T08:41:11.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.972 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754346 s, 139 MB/s 2024-04-25T08:41:11.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.982 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744627 s, 141 MB/s 2024-04-25T08:41:11.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:11.992 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:11.992 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073484 s, 143 MB/s 2024-04-25T08:41:12.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.002 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861756 s, 122 MB/s 2024-04-25T08:41:12.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.012 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736753 s, 142 MB/s 2024-04-25T08:41:12.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.022 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798478 s, 131 MB/s 2024-04-25T08:41:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.031 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734526 s, 143 MB/s 2024-04-25T08:41:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.041 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.042 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790808 s, 133 MB/s 2024-04-25T08:41:12.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.051 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755046 s, 139 MB/s 2024-04-25T08:41:12.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.061 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738508 s, 142 MB/s 2024-04-25T08:41:12.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.070 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752591 s, 139 MB/s 2024-04-25T08:41:12.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:12.071 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:12.072 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:12.072 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T08:41:12.072 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:12.072 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:12.072 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:12.072 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:12.140 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.140 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.140 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799017 s, 131 MB/s 2024-04-25T08:41:12.150 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.150 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.150 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753523 s, 139 MB/s 2024-04-25T08:41:12.159 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.159 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.159 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741969 s, 141 MB/s 2024-04-25T08:41:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.170 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868479 s, 121 MB/s 2024-04-25T08:41:12.180 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.180 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.180 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814132 s, 129 MB/s 2024-04-25T08:41:12.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.191 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833309 s, 126 MB/s 2024-04-25T08:41:12.200 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.201 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.201 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759334 s, 138 MB/s 2024-04-25T08:41:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.210 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713546 s, 147 MB/s 2024-04-25T08:41:12.220 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.220 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.221 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833819 s, 126 MB/s 2024-04-25T08:41:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.231 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802532 s, 131 MB/s 2024-04-25T08:41:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.241 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802952 s, 131 MB/s 2024-04-25T08:41:12.250 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.250 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.250 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006651 s, 158 MB/s 2024-04-25T08:41:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.259 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666776 s, 157 MB/s 2024-04-25T08:41:12.269 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.269 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.269 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078397 s, 134 MB/s 2024-04-25T08:41:12.279 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.279 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.279 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730452 s, 144 MB/s 2024-04-25T08:41:12.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.288 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736968 s, 142 MB/s 2024-04-25T08:41:12.298 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.298 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.298 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753065 s, 139 MB/s 2024-04-25T08:41:12.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.307 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703485 s, 149 MB/s 2024-04-25T08:41:12.316 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.316 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.316 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663639 s, 158 MB/s 2024-04-25T08:41:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.326 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767175 s, 137 MB/s 2024-04-25T08:41:12.335 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.335 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.335 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664854 s, 158 MB/s 2024-04-25T08:41:12.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.344 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071623 s, 146 MB/s 2024-04-25T08:41:12.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.354 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694309 s, 151 MB/s 2024-04-25T08:41:12.364 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.364 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.364 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868838 s, 121 MB/s 2024-04-25T08:41:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.374 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741769 s, 141 MB/s 2024-04-25T08:41:12.384 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.384 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.384 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836871 s, 125 MB/s 2024-04-25T08:41:12.394 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.394 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.394 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751736 s, 139 MB/s 2024-04-25T08:41:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.404 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735309 s, 143 MB/s 2024-04-25T08:41:12.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.414 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820551 s, 128 MB/s 2024-04-25T08:41:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.423 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.424 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720646 s, 146 MB/s 2024-04-25T08:41:12.433 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.433 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.433 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765741 s, 137 MB/s 2024-04-25T08:41:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731092 s, 143 MB/s 2024-04-25T08:41:12.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:12.444 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:12.444 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:12.444 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T08:41:12.444 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:12.444 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:12.444 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:12.444 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.514 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073289 s, 143 MB/s 2024-04-25T08:41:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.522 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738157 s, 142 MB/s 2024-04-25T08:41:12.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.531 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743316 s, 141 MB/s 2024-04-25T08:41:12.540 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.540 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.540 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00653733 s, 160 MB/s 2024-04-25T08:41:12.555 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.555 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.555 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0133234 s, 78.7 MB/s 2024-04-25T08:41:12.590 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.590 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.590 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795497 s, 132 MB/s 2024-04-25T08:41:12.599 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.600 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.600 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758651 s, 138 MB/s 2024-04-25T08:41:12.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.610 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.610 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748147 s, 140 MB/s 2024-04-25T08:41:12.619 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.619 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.619 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721652 s, 145 MB/s 2024-04-25T08:41:12.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.628 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.628 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677046 s, 155 MB/s 2024-04-25T08:41:12.638 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.638 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.638 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804379 s, 130 MB/s 2024-04-25T08:41:12.648 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.648 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.648 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741119 s, 141 MB/s 2024-04-25T08:41:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.657 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738794 s, 142 MB/s 2024-04-25T08:41:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.666 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.667 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703629 s, 149 MB/s 2024-04-25T08:41:12.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.676 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.676 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712868 s, 147 MB/s 2024-04-25T08:41:12.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.685 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727551 s, 144 MB/s 2024-04-25T08:41:12.695 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.695 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.695 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749362 s, 140 MB/s 2024-04-25T08:41:12.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.704 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720203 s, 146 MB/s 2024-04-25T08:41:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.714 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732965 s, 143 MB/s 2024-04-25T08:41:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.724 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738965 s, 142 MB/s 2024-04-25T08:41:12.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.733 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.733 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750635 s, 140 MB/s 2024-04-25T08:41:12.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.743 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776491 s, 135 MB/s 2024-04-25T08:41:12.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.752 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730001 s, 144 MB/s 2024-04-25T08:41:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733795 s, 143 MB/s 2024-04-25T08:41:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674401 s, 155 MB/s 2024-04-25T08:41:12.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.781 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.781 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811887 s, 129 MB/s 2024-04-25T08:41:12.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.792 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831561 s, 126 MB/s 2024-04-25T08:41:12.801 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.801 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.802 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750292 s, 140 MB/s 2024-04-25T08:41:12.811 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.812 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.812 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079776 s, 131 MB/s 2024-04-25T08:41:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.822 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772127 s, 136 MB/s 2024-04-25T08:41:12.831 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.831 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.831 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073857 s, 142 MB/s 2024-04-25T08:41:12.842 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.842 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.842 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753365 s, 139 MB/s 2024-04-25T08:41:12.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:12.842 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:12.842 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:12.842 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T08:41:12.842 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:12.842 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:12.842 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:12.843 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:12.911 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.911 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.911 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828454 s, 127 MB/s 2024-04-25T08:41:12.921 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.921 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.921 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754663 s, 139 MB/s 2024-04-25T08:41:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.930 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730578 s, 144 MB/s 2024-04-25T08:41:12.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.940 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755294 s, 139 MB/s 2024-04-25T08:41:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.950 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819143 s, 128 MB/s 2024-04-25T08:41:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.960 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073622 s, 142 MB/s 2024-04-25T08:41:12.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.970 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839375 s, 125 MB/s 2024-04-25T08:41:12.981 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.981 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.981 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822014 s, 128 MB/s 2024-04-25T08:41:12.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:12.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:12.991 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807837 s, 130 MB/s 2024-04-25T08:41:13.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.001 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.002 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807983 s, 130 MB/s 2024-04-25T08:41:13.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.012 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766493 s, 137 MB/s 2024-04-25T08:41:13.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.023 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859265 s, 122 MB/s 2024-04-25T08:41:13.033 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.033 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.033 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816841 s, 128 MB/s 2024-04-25T08:41:13.043 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.043 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.043 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760303 s, 138 MB/s 2024-04-25T08:41:13.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.052 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715559 s, 147 MB/s 2024-04-25T08:41:13.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.061 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.062 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716813 s, 146 MB/s 2024-04-25T08:41:13.071 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.071 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.071 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728531 s, 144 MB/s 2024-04-25T08:41:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.081 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829571 s, 126 MB/s 2024-04-25T08:41:13.091 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.091 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.091 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747474 s, 140 MB/s 2024-04-25T08:41:13.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.101 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736062 s, 142 MB/s 2024-04-25T08:41:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.111 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832325 s, 126 MB/s 2024-04-25T08:41:13.121 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.121 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.121 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750705 s, 140 MB/s 2024-04-25T08:41:13.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.130 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706876 s, 148 MB/s 2024-04-25T08:41:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.140 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758797 s, 138 MB/s 2024-04-25T08:41:13.150 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.150 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.150 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783989 s, 134 MB/s 2024-04-25T08:41:13.159 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.159 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.159 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703582 s, 149 MB/s 2024-04-25T08:41:13.169 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.169 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.169 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075188 s, 139 MB/s 2024-04-25T08:41:13.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.178 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074882 s, 140 MB/s 2024-04-25T08:41:13.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.188 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.188 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071841 s, 146 MB/s 2024-04-25T08:41:13.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.197 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689379 s, 152 MB/s 2024-04-25T08:41:13.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.206 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740659 s, 142 MB/s 2024-04-25T08:41:13.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.216 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741956 s, 141 MB/s 2024-04-25T08:41:13.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:13.217 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:13.217 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:13.218 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T08:41:13.218 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:13.218 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:13.218 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:13.218 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:13.285 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.286 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.286 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736474 s, 142 MB/s 2024-04-25T08:41:13.296 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.296 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.296 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821256 s, 128 MB/s 2024-04-25T08:41:13.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.306 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755306 s, 139 MB/s 2024-04-25T08:41:13.315 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.315 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.315 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719323 s, 146 MB/s 2024-04-25T08:41:13.324 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.324 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.324 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698298 s, 150 MB/s 2024-04-25T08:41:13.334 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.334 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.334 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737366 s, 142 MB/s 2024-04-25T08:41:13.343 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.344 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735614 s, 143 MB/s 2024-04-25T08:41:13.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.353 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075112 s, 140 MB/s 2024-04-25T08:41:13.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.363 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734508 s, 143 MB/s 2024-04-25T08:41:13.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.373 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842091 s, 125 MB/s 2024-04-25T08:41:13.383 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.383 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.383 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734815 s, 143 MB/s 2024-04-25T08:41:13.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.393 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814679 s, 129 MB/s 2024-04-25T08:41:13.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.403 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758993 s, 138 MB/s 2024-04-25T08:41:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.412 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723121 s, 145 MB/s 2024-04-25T08:41:13.421 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.422 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734306 s, 143 MB/s 2024-04-25T08:41:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.432 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810215 s, 129 MB/s 2024-04-25T08:41:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.442 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765056 s, 137 MB/s 2024-04-25T08:41:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.451 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719327 s, 146 MB/s 2024-04-25T08:41:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.462 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868734 s, 121 MB/s 2024-04-25T08:41:13.472 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.472 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.472 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743226 s, 141 MB/s 2024-04-25T08:41:13.481 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.481 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.481 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733001 s, 143 MB/s 2024-04-25T08:41:13.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.491 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.491 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744253 s, 141 MB/s 2024-04-25T08:41:13.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.501 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736307 s, 142 MB/s 2024-04-25T08:41:13.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.510 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758652 s, 138 MB/s 2024-04-25T08:41:13.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.520 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.520 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710463 s, 148 MB/s 2024-04-25T08:41:13.529 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.529 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.529 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073866 s, 142 MB/s 2024-04-25T08:41:13.540 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.540 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.540 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835572 s, 125 MB/s 2024-04-25T08:41:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.550 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760734 s, 138 MB/s 2024-04-25T08:41:13.566 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.566 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.566 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0140686 s, 74.5 MB/s 2024-04-25T08:41:13.594 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.594 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.594 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790795 s, 133 MB/s 2024-04-25T08:41:13.604 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.604 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.604 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800127 s, 131 MB/s 2024-04-25T08:41:13.614 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.614 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.614 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736483 s, 142 MB/s 2024-04-25T08:41:13.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:13.615 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:13.615 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:13.615 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T08:41:13.616 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:13.616 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:13.616 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:13.616 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075945 s, 138 MB/s 2024-04-25T08:41:13.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.693 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.693 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721173 s, 145 MB/s 2024-04-25T08:41:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.703 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769922 s, 136 MB/s 2024-04-25T08:41:13.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.713 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831656 s, 126 MB/s 2024-04-25T08:41:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.722 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693732 s, 151 MB/s 2024-04-25T08:41:13.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.732 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.732 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704904 s, 149 MB/s 2024-04-25T08:41:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.742 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082755 s, 127 MB/s 2024-04-25T08:41:13.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.752 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735576 s, 143 MB/s 2024-04-25T08:41:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.761 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732314 s, 143 MB/s 2024-04-25T08:41:13.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.772 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852194 s, 123 MB/s 2024-04-25T08:41:13.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.782 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765941 s, 137 MB/s 2024-04-25T08:41:13.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.791 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.792 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758398 s, 138 MB/s 2024-04-25T08:41:13.801 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.801 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.801 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727383 s, 144 MB/s 2024-04-25T08:41:13.811 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.811 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.811 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756203 s, 139 MB/s 2024-04-25T08:41:13.820 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.820 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.820 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720853 s, 145 MB/s 2024-04-25T08:41:13.830 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.830 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.830 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754428 s, 139 MB/s 2024-04-25T08:41:13.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.839 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718415 s, 146 MB/s 2024-04-25T08:41:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.849 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738695 s, 142 MB/s 2024-04-25T08:41:13.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.858 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718936 s, 146 MB/s 2024-04-25T08:41:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.868 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755767 s, 139 MB/s 2024-04-25T08:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.877 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697954 s, 150 MB/s 2024-04-25T08:41:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725859 s, 144 MB/s 2024-04-25T08:41:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.896 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.897 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743405 s, 141 MB/s 2024-04-25T08:41:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.907 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806637 s, 130 MB/s 2024-04-25T08:41:13.916 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.916 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.916 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730074 s, 144 MB/s 2024-04-25T08:41:13.927 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.927 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.927 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079704 s, 132 MB/s 2024-04-25T08:41:13.937 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.937 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.938 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833258 s, 126 MB/s 2024-04-25T08:41:13.947 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.947 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.948 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779565 s, 135 MB/s 2024-04-25T08:41:13.957 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.957 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.957 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734608 s, 143 MB/s 2024-04-25T08:41:13.967 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.967 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.967 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734771 s, 143 MB/s 2024-04-25T08:41:13.977 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.977 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.977 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823742 s, 127 MB/s 2024-04-25T08:41:13.986 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:13.986 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:13.987 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657322 s, 160 MB/s 2024-04-25T08:41:13.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:13.987 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:13.987 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:13.987 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T08:41:13.987 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:13.987 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:13.987 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:13.987 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:14.054 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.054 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.054 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751286 s, 140 MB/s 2024-04-25T08:41:14.064 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.064 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.064 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741729 s, 141 MB/s 2024-04-25T08:41:14.074 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.074 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.075 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822829 s, 127 MB/s 2024-04-25T08:41:14.085 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.085 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.085 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816362 s, 128 MB/s 2024-04-25T08:41:14.095 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.095 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.095 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845959 s, 124 MB/s 2024-04-25T08:41:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814969 s, 129 MB/s 2024-04-25T08:41:14.116 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.116 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.116 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773691 s, 136 MB/s 2024-04-25T08:41:14.126 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.126 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.126 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825329 s, 127 MB/s 2024-04-25T08:41:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.136 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770772 s, 136 MB/s 2024-04-25T08:41:14.145 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.146 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712522 s, 147 MB/s 2024-04-25T08:41:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.156 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771135 s, 136 MB/s 2024-04-25T08:41:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.165 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703703 s, 149 MB/s 2024-04-25T08:41:14.175 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.175 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.175 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744673 s, 141 MB/s 2024-04-25T08:41:14.184 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.185 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.185 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767734 s, 137 MB/s 2024-04-25T08:41:14.195 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.196 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.196 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085622 s, 122 MB/s 2024-04-25T08:41:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.205 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748009 s, 140 MB/s 2024-04-25T08:41:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.215 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754965 s, 139 MB/s 2024-04-25T08:41:14.224 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.225 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737065 s, 142 MB/s 2024-04-25T08:41:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.235 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086069 s, 122 MB/s 2024-04-25T08:41:14.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.245 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741917 s, 141 MB/s 2024-04-25T08:41:14.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.255 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702276 s, 149 MB/s 2024-04-25T08:41:14.264 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.265 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.265 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778973 s, 135 MB/s 2024-04-25T08:41:14.274 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.274 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.274 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756086 s, 139 MB/s 2024-04-25T08:41:14.284 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.284 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.284 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760849 s, 138 MB/s 2024-04-25T08:41:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.293 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681056 s, 154 MB/s 2024-04-25T08:41:14.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.304 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795325 s, 132 MB/s 2024-04-25T08:41:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.313 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671026 s, 156 MB/s 2024-04-25T08:41:14.322 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.322 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.322 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742696 s, 141 MB/s 2024-04-25T08:41:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.332 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766838 s, 137 MB/s 2024-04-25T08:41:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.342 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724506 s, 145 MB/s 2024-04-25T08:41:14.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.353 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814799 s, 129 MB/s 2024-04-25T08:41:14.365 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.365 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.365 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925879 s, 113 MB/s 2024-04-25T08:41:14.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:14.366 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:14.366 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:14.366 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T08:41:14.366 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:14.366 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:14.366 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:14.366 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:14.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.493 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717267 s, 146 MB/s 2024-04-25T08:41:14.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.503 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738146 s, 142 MB/s 2024-04-25T08:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.513 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790333 s, 133 MB/s 2024-04-25T08:41:14.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.522 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.522 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677114 s, 155 MB/s 2024-04-25T08:41:14.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.531 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752695 s, 139 MB/s 2024-04-25T08:41:14.541 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.541 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.541 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731417 s, 143 MB/s 2024-04-25T08:41:14.550 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.551 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.551 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752248 s, 139 MB/s 2024-04-25T08:41:14.560 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.560 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.560 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734738 s, 143 MB/s 2024-04-25T08:41:14.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.598 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.598 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728413 s, 144 MB/s 2024-04-25T08:41:14.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.608 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.608 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751126 s, 140 MB/s 2024-04-25T08:41:14.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.618 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770289 s, 136 MB/s 2024-04-25T08:41:14.629 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.629 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.629 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085239 s, 123 MB/s 2024-04-25T08:41:14.639 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.639 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.639 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807863 s, 130 MB/s 2024-04-25T08:41:14.649 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.649 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.649 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792669 s, 132 MB/s 2024-04-25T08:41:14.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.660 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822063 s, 128 MB/s 2024-04-25T08:41:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718941 s, 146 MB/s 2024-04-25T08:41:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.678 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688928 s, 152 MB/s 2024-04-25T08:41:14.687 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.688 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718663 s, 146 MB/s 2024-04-25T08:41:14.697 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.697 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.697 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743066 s, 141 MB/s 2024-04-25T08:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.707 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746372 s, 140 MB/s 2024-04-25T08:41:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.716 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665238 s, 158 MB/s 2024-04-25T08:41:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.725 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756041 s, 139 MB/s 2024-04-25T08:41:14.736 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.736 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.736 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811633 s, 129 MB/s 2024-04-25T08:41:14.746 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.746 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.746 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765401 s, 137 MB/s 2024-04-25T08:41:14.755 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.755 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.755 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745722 s, 141 MB/s 2024-04-25T08:41:14.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.765 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.765 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075404 s, 139 MB/s 2024-04-25T08:41:14.775 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.775 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.775 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754701 s, 139 MB/s 2024-04-25T08:41:14.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.784 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.784 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735081 s, 143 MB/s 2024-04-25T08:41:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.794 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760281 s, 138 MB/s 2024-04-25T08:41:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717176 s, 146 MB/s 2024-04-25T08:41:14.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.813 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701182 s, 150 MB/s 2024-04-25T08:41:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.823 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765882 s, 137 MB/s 2024-04-25T08:41:14.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:14.824 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:14.824 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:14.824 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T08:41:14.824 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:14.824 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:14.824 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:14.824 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.852 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747863 s, 140 MB/s 2024-04-25T08:41:14.861 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.862 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743621 s, 141 MB/s 2024-04-25T08:41:14.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.872 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794416 s, 132 MB/s 2024-04-25T08:41:14.881 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.881 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.881 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706258 s, 148 MB/s 2024-04-25T08:41:14.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.892 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813296 s, 129 MB/s 2024-04-25T08:41:14.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.901 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731947 s, 143 MB/s 2024-04-25T08:41:14.911 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.911 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.911 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774377 s, 135 MB/s 2024-04-25T08:41:14.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.921 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072123 s, 145 MB/s 2024-04-25T08:41:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.930 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758123 s, 138 MB/s 2024-04-25T08:41:14.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.940 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728987 s, 144 MB/s 2024-04-25T08:41:14.949 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.949 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.949 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068567 s, 153 MB/s 2024-04-25T08:41:14.959 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.960 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838177 s, 125 MB/s 2024-04-25T08:41:14.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.970 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815996 s, 129 MB/s 2024-04-25T08:41:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.980 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818156 s, 128 MB/s 2024-04-25T08:41:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.990 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716674 s, 146 MB/s 2024-04-25T08:41:14.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:14.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:14.999 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713222 s, 147 MB/s 2024-04-25T08:41:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.009 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.010 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822638 s, 127 MB/s 2024-04-25T08:41:15.020 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.020 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.020 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806346 s, 130 MB/s 2024-04-25T08:41:15.030 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.030 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.030 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783783 s, 134 MB/s 2024-04-25T08:41:15.039 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.039 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.039 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704065 s, 149 MB/s 2024-04-25T08:41:15.049 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.049 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.049 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744859 s, 141 MB/s 2024-04-25T08:41:15.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.058 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744528 s, 141 MB/s 2024-04-25T08:41:15.068 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.068 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.068 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723084 s, 145 MB/s 2024-04-25T08:41:15.078 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.078 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.078 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821997 s, 128 MB/s 2024-04-25T08:41:15.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.087 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681246 s, 154 MB/s 2024-04-25T08:41:15.097 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.097 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.097 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007749 s, 135 MB/s 2024-04-25T08:41:15.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.107 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074833 s, 140 MB/s 2024-04-25T08:41:15.116 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.116 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.116 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708967 s, 148 MB/s 2024-04-25T08:41:15.125 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.125 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.125 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665936 s, 157 MB/s 2024-04-25T08:41:15.134 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.135 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.135 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729012 s, 144 MB/s 2024-04-25T08:41:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839768 s, 125 MB/s 2024-04-25T08:41:15.155 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.156 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806859 s, 130 MB/s 2024-04-25T08:41:15.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:15.157 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:15.157 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:15.157 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T08:41:15.157 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:15.157 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:15.157 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:15.157 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.225 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828967 s, 126 MB/s 2024-04-25T08:41:15.235 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.235 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.235 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790483 s, 133 MB/s 2024-04-25T08:41:15.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.245 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.245 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778208 s, 135 MB/s 2024-04-25T08:41:15.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.254 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690109 s, 152 MB/s 2024-04-25T08:41:15.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.263 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692198 s, 151 MB/s 2024-04-25T08:41:15.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.273 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769689 s, 136 MB/s 2024-04-25T08:41:15.282 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.282 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.282 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682833 s, 154 MB/s 2024-04-25T08:41:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.292 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754715 s, 139 MB/s 2024-04-25T08:41:15.302 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.302 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.303 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814574 s, 129 MB/s 2024-04-25T08:41:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.311 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.312 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664406 s, 158 MB/s 2024-04-25T08:41:15.321 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.321 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.321 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765688 s, 137 MB/s 2024-04-25T08:41:15.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.330 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682009 s, 154 MB/s 2024-04-25T08:41:15.339 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.340 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685766 s, 153 MB/s 2024-04-25T08:41:15.350 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.350 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.350 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822994 s, 127 MB/s 2024-04-25T08:41:15.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.360 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738077 s, 142 MB/s 2024-04-25T08:41:15.370 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.370 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.370 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778702 s, 135 MB/s 2024-04-25T08:41:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.380 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809481 s, 130 MB/s 2024-04-25T08:41:15.390 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.390 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.390 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760052 s, 138 MB/s 2024-04-25T08:41:15.399 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.399 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.399 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740049 s, 142 MB/s 2024-04-25T08:41:15.408 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.408 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.408 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675314 s, 155 MB/s 2024-04-25T08:41:15.418 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.418 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.418 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072628 s, 144 MB/s 2024-04-25T08:41:15.427 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.427 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.427 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704139 s, 149 MB/s 2024-04-25T08:41:15.436 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.436 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.436 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666713 s, 157 MB/s 2024-04-25T08:41:15.446 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.446 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.446 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735943 s, 142 MB/s 2024-04-25T08:41:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.456 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841168 s, 125 MB/s 2024-04-25T08:41:15.465 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.466 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.466 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709846 s, 148 MB/s 2024-04-25T08:41:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738949 s, 142 MB/s 2024-04-25T08:41:15.484 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.484 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.484 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728253 s, 144 MB/s 2024-04-25T08:41:15.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.494 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754832 s, 139 MB/s 2024-04-25T08:41:15.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.504 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.504 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744287 s, 141 MB/s 2024-04-25T08:41:15.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.514 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797467 s, 131 MB/s 2024-04-25T08:41:15.525 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.525 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.525 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863199 s, 121 MB/s 2024-04-25T08:41:15.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:15.526 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:15.526 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:15.526 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T08:41:15.526 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:15.526 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:15.526 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:15.526 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:15.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.618 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.618 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840863 s, 125 MB/s 2024-04-25T08:41:15.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.626 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720834 s, 145 MB/s 2024-04-25T08:41:15.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.636 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716224 s, 146 MB/s 2024-04-25T08:41:15.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.645 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072557 s, 145 MB/s 2024-04-25T08:41:15.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.655 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801966 s, 131 MB/s 2024-04-25T08:41:15.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.665 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750532 s, 140 MB/s 2024-04-25T08:41:15.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.674 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741753 s, 141 MB/s 2024-04-25T08:41:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.683 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659057 s, 159 MB/s 2024-04-25T08:41:15.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.692 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696928 s, 150 MB/s 2024-04-25T08:41:15.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.702 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730177 s, 144 MB/s 2024-04-25T08:41:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735592 s, 143 MB/s 2024-04-25T08:41:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.722 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828677 s, 127 MB/s 2024-04-25T08:41:15.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.732 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752268 s, 139 MB/s 2024-04-25T08:41:15.741 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.741 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.741 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671908 s, 156 MB/s 2024-04-25T08:41:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.749 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660441 s, 159 MB/s 2024-04-25T08:41:15.759 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.759 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.759 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727994 s, 144 MB/s 2024-04-25T08:41:15.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.768 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682686 s, 154 MB/s 2024-04-25T08:41:15.778 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.779 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.779 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821618 s, 128 MB/s 2024-04-25T08:41:15.789 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.789 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.789 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772745 s, 136 MB/s 2024-04-25T08:41:15.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.799 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824617 s, 127 MB/s 2024-04-25T08:41:15.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.809 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776269 s, 135 MB/s 2024-04-25T08:41:15.822 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.822 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.822 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743977 s, 141 MB/s 2024-04-25T08:41:15.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.828 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732843 s, 143 MB/s 2024-04-25T08:41:15.837 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.838 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721682 s, 145 MB/s 2024-04-25T08:41:15.847 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.847 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.847 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717006 s, 146 MB/s 2024-04-25T08:41:15.856 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.856 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.856 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686482 s, 153 MB/s 2024-04-25T08:41:15.865 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.865 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.866 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712142 s, 147 MB/s 2024-04-25T08:41:15.875 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.875 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.875 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757567 s, 138 MB/s 2024-04-25T08:41:15.885 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.885 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.885 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742552 s, 141 MB/s 2024-04-25T08:41:15.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.895 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756701 s, 139 MB/s 2024-04-25T08:41:15.904 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.905 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770399 s, 136 MB/s 2024-04-25T08:41:15.914 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.914 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.914 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723404 s, 145 MB/s 2024-04-25T08:41:15.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:15.916 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:15.916 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:15.916 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T08:41:15.916 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:15.916 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:15.916 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:15.916 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:15.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.945 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.945 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844041 s, 124 MB/s 2024-04-25T08:41:15.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.954 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695273 s, 151 MB/s 2024-04-25T08:41:15.963 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.963 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.963 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072977 s, 144 MB/s 2024-04-25T08:41:15.973 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.974 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.974 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806137 s, 130 MB/s 2024-04-25T08:41:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742928 s, 141 MB/s 2024-04-25T08:41:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683645 s, 153 MB/s 2024-04-25T08:41:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752306 s, 139 MB/s 2024-04-25T08:41:16.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.012 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733155 s, 143 MB/s 2024-04-25T08:41:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.021 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738536 s, 142 MB/s 2024-04-25T08:41:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.031 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748873 s, 140 MB/s 2024-04-25T08:41:16.040 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.040 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.040 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747781 s, 140 MB/s 2024-04-25T08:41:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.050 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740544 s, 142 MB/s 2024-04-25T08:41:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.060 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738476 s, 142 MB/s 2024-04-25T08:41:16.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.070 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802682 s, 131 MB/s 2024-04-25T08:41:16.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.080 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079391 s, 132 MB/s 2024-04-25T08:41:16.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.091 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.091 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825835 s, 127 MB/s 2024-04-25T08:41:16.100 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.100 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.100 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726802 s, 144 MB/s 2024-04-25T08:41:16.111 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.111 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.111 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867302 s, 121 MB/s 2024-04-25T08:41:16.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.120 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702605 s, 149 MB/s 2024-04-25T08:41:16.129 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.129 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.129 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673558 s, 156 MB/s 2024-04-25T08:41:16.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.139 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778107 s, 135 MB/s 2024-04-25T08:41:16.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707049 s, 148 MB/s 2024-04-25T08:41:16.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.158 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758438 s, 138 MB/s 2024-04-25T08:41:16.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.167 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681261 s, 154 MB/s 2024-04-25T08:41:16.176 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.177 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730204 s, 144 MB/s 2024-04-25T08:41:16.185 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.185 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.186 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00661667 s, 158 MB/s 2024-04-25T08:41:16.196 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.196 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.196 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844328 s, 124 MB/s 2024-04-25T08:41:16.205 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.206 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682204 s, 154 MB/s 2024-04-25T08:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.215 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726413 s, 144 MB/s 2024-04-25T08:41:16.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.226 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.226 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831943 s, 126 MB/s 2024-04-25T08:41:16.236 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.236 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.236 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818536 s, 128 MB/s 2024-04-25T08:41:16.246 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.247 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764804 s, 137 MB/s 2024-04-25T08:41:16.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:16.247 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:16.247 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:16.247 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T08:41:16.247 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:16.247 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:16.247 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:16.248 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:16.316 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.316 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.316 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834944 s, 126 MB/s 2024-04-25T08:41:16.325 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.325 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.325 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679391 s, 154 MB/s 2024-04-25T08:41:16.334 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.334 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.334 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703752 s, 149 MB/s 2024-04-25T08:41:16.343 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.344 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749666 s, 140 MB/s 2024-04-25T08:41:16.352 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.353 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683569 s, 153 MB/s 2024-04-25T08:41:16.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.363 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.363 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809706 s, 130 MB/s 2024-04-25T08:41:16.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.373 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.373 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749796 s, 140 MB/s 2024-04-25T08:41:16.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.382 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.383 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074917 s, 140 MB/s 2024-04-25T08:41:16.392 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.392 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.392 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754809 s, 139 MB/s 2024-04-25T08:41:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845124 s, 124 MB/s 2024-04-25T08:41:16.413 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.413 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.413 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733497 s, 143 MB/s 2024-04-25T08:41:16.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.423 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749188 s, 140 MB/s 2024-04-25T08:41:16.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.432 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768089 s, 137 MB/s 2024-04-25T08:41:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.443 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815685 s, 129 MB/s 2024-04-25T08:41:16.452 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.452 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.452 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765299 s, 137 MB/s 2024-04-25T08:41:16.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.463 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844239 s, 124 MB/s 2024-04-25T08:41:16.473 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.473 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.473 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075483 s, 139 MB/s 2024-04-25T08:41:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.482 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743191 s, 141 MB/s 2024-04-25T08:41:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.492 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732796 s, 143 MB/s 2024-04-25T08:41:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.503 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862336 s, 122 MB/s 2024-04-25T08:41:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.513 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821613 s, 128 MB/s 2024-04-25T08:41:16.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.524 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783447 s, 134 MB/s 2024-04-25T08:41:16.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.533 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754078 s, 139 MB/s 2024-04-25T08:41:16.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.542 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683929 s, 153 MB/s 2024-04-25T08:41:16.553 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.553 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.553 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826044 s, 127 MB/s 2024-04-25T08:41:16.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.562 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704542 s, 149 MB/s 2024-04-25T08:41:16.572 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.572 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.572 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745798 s, 141 MB/s 2024-04-25T08:41:16.584 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.584 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.584 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985462 s, 106 MB/s 2024-04-25T08:41:16.614 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.614 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.615 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754471 s, 139 MB/s 2024-04-25T08:41:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.625 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821009 s, 128 MB/s 2024-04-25T08:41:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.635 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794772 s, 132 MB/s 2024-04-25T08:41:16.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.645 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803125 s, 131 MB/s 2024-04-25T08:41:16.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:16.647 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:16.647 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:16.647 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T08:41:16.647 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:16.647 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:16.647 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:16.647 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:16.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.715 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829351 s, 126 MB/s 2024-04-25T08:41:16.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.724 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.724 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734668 s, 143 MB/s 2024-04-25T08:41:16.734 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.734 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.734 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765749 s, 137 MB/s 2024-04-25T08:41:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.744 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731384 s, 143 MB/s 2024-04-25T08:41:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.753 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.754 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073021 s, 144 MB/s 2024-04-25T08:41:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.763 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751767 s, 139 MB/s 2024-04-25T08:41:16.772 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.773 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.773 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722202 s, 145 MB/s 2024-04-25T08:41:16.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.782 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722615 s, 145 MB/s 2024-04-25T08:41:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.792 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.793 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822518 s, 127 MB/s 2024-04-25T08:41:16.802 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.803 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798017 s, 131 MB/s 2024-04-25T08:41:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077589 s, 135 MB/s 2024-04-25T08:41:16.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.823 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824803 s, 127 MB/s 2024-04-25T08:41:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.833 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755485 s, 139 MB/s 2024-04-25T08:41:16.842 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.843 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720851 s, 145 MB/s 2024-04-25T08:41:16.852 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.852 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.852 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753107 s, 139 MB/s 2024-04-25T08:41:16.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.862 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745243 s, 141 MB/s 2024-04-25T08:41:16.871 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.871 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.871 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721957 s, 145 MB/s 2024-04-25T08:41:16.881 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.881 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.881 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071117 s, 147 MB/s 2024-04-25T08:41:16.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.891 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791241 s, 133 MB/s 2024-04-25T08:41:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.901 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776509 s, 135 MB/s 2024-04-25T08:41:16.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.910 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725723 s, 144 MB/s 2024-04-25T08:41:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.921 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008284 s, 127 MB/s 2024-04-25T08:41:16.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.931 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785207 s, 134 MB/s 2024-04-25T08:41:16.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.940 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728973 s, 144 MB/s 2024-04-25T08:41:16.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.950 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774223 s, 135 MB/s 2024-04-25T08:41:16.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.960 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743723 s, 141 MB/s 2024-04-25T08:41:16.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.970 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783973 s, 134 MB/s 2024-04-25T08:41:16.979 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.979 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.979 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712498 s, 147 MB/s 2024-04-25T08:41:16.989 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.989 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.989 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751 s, 140 MB/s 2024-04-25T08:41:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:16.998 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726865 s, 144 MB/s 2024-04-25T08:41:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.008 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744232 s, 141 MB/s 2024-04-25T08:41:17.019 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.019 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.019 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852293 s, 123 MB/s 2024-04-25T08:41:17.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:17.020 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:17.020 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:17.020 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T08:41:17.020 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:17.020 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:17.020 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:17.020 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:17.088 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.088 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.089 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762234 s, 138 MB/s 2024-04-25T08:41:17.098 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.098 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.098 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752159 s, 139 MB/s 2024-04-25T08:41:17.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.109 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.109 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841405 s, 125 MB/s 2024-04-25T08:41:17.119 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.119 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.119 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759233 s, 138 MB/s 2024-04-25T08:41:17.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.128 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.128 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730463 s, 144 MB/s 2024-04-25T08:41:17.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.138 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.138 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739657 s, 142 MB/s 2024-04-25T08:41:17.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822909 s, 127 MB/s 2024-04-25T08:41:17.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.158 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076027 s, 138 MB/s 2024-04-25T08:41:17.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.167 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.167 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660248 s, 159 MB/s 2024-04-25T08:41:17.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.177 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732208 s, 143 MB/s 2024-04-25T08:41:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.186 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773576 s, 136 MB/s 2024-04-25T08:41:17.196 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.196 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.196 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069503 s, 151 MB/s 2024-04-25T08:41:17.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.206 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777484 s, 135 MB/s 2024-04-25T08:41:17.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.215 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703167 s, 149 MB/s 2024-04-25T08:41:17.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.225 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802363 s, 131 MB/s 2024-04-25T08:41:17.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.234 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693272 s, 151 MB/s 2024-04-25T08:41:17.244 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.244 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.244 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070555 s, 149 MB/s 2024-04-25T08:41:17.253 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.254 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760984 s, 138 MB/s 2024-04-25T08:41:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.263 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739805 s, 142 MB/s 2024-04-25T08:41:17.272 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.273 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731383 s, 143 MB/s 2024-04-25T08:41:17.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.283 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801407 s, 131 MB/s 2024-04-25T08:41:17.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.293 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790726 s, 133 MB/s 2024-04-25T08:41:17.302 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.302 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.302 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073959 s, 142 MB/s 2024-04-25T08:41:17.312 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.312 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.312 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721216 s, 145 MB/s 2024-04-25T08:41:17.321 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.321 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.321 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720862 s, 145 MB/s 2024-04-25T08:41:17.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.330 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.330 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698887 s, 150 MB/s 2024-04-25T08:41:17.340 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.340 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.340 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769333 s, 136 MB/s 2024-04-25T08:41:17.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.350 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.350 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717968 s, 146 MB/s 2024-04-25T08:41:17.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.360 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819653 s, 128 MB/s 2024-04-25T08:41:17.369 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.369 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.369 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073374 s, 143 MB/s 2024-04-25T08:41:17.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.379 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.379 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703457 s, 149 MB/s 2024-04-25T08:41:17.389 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.389 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.389 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819386 s, 128 MB/s 2024-04-25T08:41:17.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:17.390 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:41:17.390 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:41:17.390 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T08:41:17.391 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:41:17.391 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:41:17.391 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:41:17.391 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:41:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728913 s, 144 MB/s 2024-04-25T08:41:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721068 s, 145 MB/s 2024-04-25T08:41:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798164 s, 131 MB/s 2024-04-25T08:41:17.487 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.487 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.487 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712265 s, 147 MB/s 2024-04-25T08:41:17.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.496 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724965 s, 145 MB/s 2024-04-25T08:41:17.506 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.506 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.506 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808746 s, 130 MB/s 2024-04-25T08:41:17.516 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.516 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.516 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726756 s, 144 MB/s 2024-04-25T08:41:17.525 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.525 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.525 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716921 s, 146 MB/s 2024-04-25T08:41:17.536 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.536 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.536 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872386 s, 120 MB/s 2024-04-25T08:41:17.546 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.546 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.546 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738423 s, 142 MB/s 2024-04-25T08:41:17.555 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.555 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.555 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748342 s, 140 MB/s 2024-04-25T08:41:17.565 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.565 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.566 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817499 s, 128 MB/s 2024-04-25T08:41:17.575 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.575 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.575 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758876 s, 138 MB/s 2024-04-25T08:41:17.588 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.588 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.588 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103658 s, 101 MB/s 2024-04-25T08:41:17.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.617 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714242 s, 147 MB/s 2024-04-25T08:41:17.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.627 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.627 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745696 s, 141 MB/s 2024-04-25T08:41:17.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.635 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669637 s, 157 MB/s 2024-04-25T08:41:17.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.646 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.646 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804278 s, 130 MB/s 2024-04-25T08:41:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.655 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753093 s, 139 MB/s 2024-04-25T08:41:17.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.665 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.665 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786094 s, 133 MB/s 2024-04-25T08:41:17.675 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.675 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.675 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718681 s, 146 MB/s 2024-04-25T08:41:17.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.685 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079577 s, 132 MB/s 2024-04-25T08:41:17.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.694 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007237 s, 145 MB/s 2024-04-25T08:41:17.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.705 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793674 s, 132 MB/s 2024-04-25T08:41:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657448 s, 159 MB/s 2024-04-25T08:41:17.723 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.723 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.723 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725063 s, 145 MB/s 2024-04-25T08:41:17.732 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.732 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.732 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685049 s, 153 MB/s 2024-04-25T08:41:17.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.742 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751065 s, 140 MB/s 2024-04-25T08:41:17.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.751 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.751 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741338 s, 141 MB/s 2024-04-25T08:41:17.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.760 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680377 s, 154 MB/s 2024-04-25T08:41:17.770 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.770 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.770 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741335 s, 141 MB/s 2024-04-25T08:41:17.780 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:41:17.780 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:41:17.780 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759815 s, 138 MB/s 2024-04-25T08:41:17.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:17.781 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-25T08:41:17.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.934+0000 7fb78869e640 1 Processor -- start 2024-04-25T08:41:17.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.934+0000 7fb78869e640 1 -- start start 2024-04-25T08:41:17.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78869e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb7801612c0 0x7fb7800b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78869e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb7800b6a00 0x7fb7800d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78869e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7800d0870 0x7fb7800d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78869e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800d11b0 con 0x7fb7801612c0 2024-04-25T08:41:17.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78869e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800b7200 con 0x7fb7800b6a00 2024-04-25T08:41:17.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78869e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800b74e0 con 0x7fb7800d0870 2024-04-25T08:41:17.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78643a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb7801612c0 0x7fb7800b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78643a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb7801612c0 0x7fb7800b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44098/0 (socket says 172.21.15.38:44098) 2024-04-25T08:41:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78643a640 1 -- 172.21.15.38:0/957826853 learned_addr learned my addr 172.21.15.38:0/957826853 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78643a640 1 -- 172.21.15.38:0/957826853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7800d0870 msgr2=0x7fb7800d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb786c3b640 1 --2- 172.21.15.38:0/957826853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7800d0870 0x7fb7800d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb785c39640 1 --2- 172.21.15.38:0/957826853 >> v2:172.21.15.107:3300/0 conn(0x7fb7800b6a00 0x7fb7800d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78643a640 1 --2- 172.21.15.38:0/957826853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7800d0870 0x7fb7800d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78643a640 1 -- 172.21.15.38:0/957826853 >> v2:172.21.15.107:3300/0 conn(0x7fb7800b6a00 msgr2=0x7fb7800d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78643a640 1 --2- 172.21.15.38:0/957826853 >> v2:172.21.15.107:3300/0 conn(0x7fb7800b6a00 0x7fb7800d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb78643a640 1 -- 172.21.15.38:0/957826853 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7800b77c0 con 0x7fb7801612c0 2024-04-25T08:41:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb786c3b640 1 --2- 172.21.15.38:0/957826853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7800d0870 0x7fb7800d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:17.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.935+0000 7fb785c39640 1 --2- 172.21.15.38:0/957826853 >> v2:172.21.15.107:3300/0 conn(0x7fb7800b6a00 0x7fb7800d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:17.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.936+0000 7fb78643a640 1 --2- 172.21.15.38:0/957826853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7801612c0 0x7fb7800b93f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fb7800c71f0 tx=0x7fb77802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08cd6c7daa1d439 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.936+0000 7fb785438640 1 -- 172.21.15.38:0/957826853 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778037070 con 0x7fb7801612c0 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.936+0000 7fb785438640 1 -- 172.21.15.38:0/957826853 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb778032440 con 0x7fb7801612c0 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.936+0000 7fb785438640 1 -- 172.21.15.38:0/957826853 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778032810 con 0x7fb7801612c0 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.937+0000 7fb78869e640 1 -- 172.21.15.38:0/957826853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7801612c0 msgr2=0x7fb7800b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.937+0000 7fb78869e640 1 --2- 172.21.15.38:0/957826853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7801612c0 0x7fb7800b93f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fb7800c71f0 tx=0x7fb77802a140 comp rx=0 tx=0).stop 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.937+0000 7fb78869e640 1 -- 172.21.15.38:0/957826853 shutdown_connections 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.937+0000 7fb78869e640 1 --2- 172.21.15.38:0/957826853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7800d0870 0x7fb7800d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.937+0000 7fb78869e640 1 --2- 172.21.15.38:0/957826853 >> v2:172.21.15.107:3300/0 conn(0x7fb7800b6a00 0x7fb7800d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.937+0000 7fb78869e640 1 --2- 172.21.15.38:0/957826853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7801612c0 0x7fb7800b93f0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.937+0000 7fb78869e640 1 -- 172.21.15.38:0/957826853 >> 172.21.15.38:0/957826853 conn(0x7fb780066d10 msgr2=0x7fb780067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.937+0000 7fb78869e640 1 -- 172.21.15.38:0/957826853 shutdown_connections 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.937+0000 7fb78869e640 1 -- 172.21.15.38:0/957826853 wait complete. 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.938+0000 7fb78869e640 1 Processor -- start 2024-04-25T08:41:17.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.938+0000 7fb78869e640 1 -- start start 2024-04-25T08:41:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb78869e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7800b6a00 0x7fb78014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb78869e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7800d0870 0x7fb78014fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb78869e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7801612c0 0x7fb780150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb78869e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800d2170 con 0x7fb7801612c0 2024-04-25T08:41:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb78869e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800b7d80 con 0x7fb7800d0870 2024-04-25T08:41:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb78869e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800b6e90 con 0x7fb7800b6a00 2024-04-25T08:41:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb786c3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7801612c0 0x7fb780150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb786c3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7801612c0 0x7fb780150070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44112/0 (socket says 172.21.15.38:44112) 2024-04-25T08:41:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb786c3b640 1 -- 172.21.15.38:0/2913940076 learned_addr learned my addr 172.21.15.38:0/2913940076 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb78643a640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7800b6a00 0x7fb78014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb785c39640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7800d0870 0x7fb78014fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb786c3b640 1 -- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7800b6a00 msgr2=0x7fb78014eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb786c3b640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7800b6a00 0x7fb78014eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb786c3b640 1 -- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7800d0870 msgr2=0x7fb78014fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb786c3b640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7800d0870 0x7fb78014fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.939+0000 7fb786c3b640 1 -- 172.21.15.38:0/2913940076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78014f000 con 0x7fb7801612c0 2024-04-25T08:41:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.940+0000 7fb786c3b640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7801612c0 0x7fb780150070 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fb77000cf20 tx=0x7fb77000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.940+0000 7fb77effd640 1 -- 172.21.15.38:0/2913940076 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77001a070 con 0x7fb7801612c0 2024-04-25T08:41:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.940+0000 7fb77effd640 1 -- 172.21.15.38:0/2913940076 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77000d840 con 0x7fb7801612c0 2024-04-25T08:41:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.940+0000 7fb78869e640 1 -- 172.21.15.38:0/2913940076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7801526b0 con 0x7fb7801612c0 2024-04-25T08:41:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.940+0000 7fb77effd640 1 -- 172.21.15.38:0/2913940076 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77000dc30 con 0x7fb7801612c0 2024-04-25T08:41:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.940+0000 7fb78869e640 1 -- 172.21.15.38:0/2913940076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb780152c70 con 0x7fb7801612c0 2024-04-25T08:41:17.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.941+0000 7fb77effd640 1 -- 172.21.15.38:0/2913940076 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb77001d070 con 0x7fb7801612c0 2024-04-25T08:41:17.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.942+0000 7fb77effd640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb76c06b4d0 0x7fb76c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.942+0000 7fb77effd640 1 -- 172.21.15.38:0/2913940076 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb77008ab90 con 0x7fb7801612c0 2024-04-25T08:41:17.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.942+0000 7fb78643a640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb76c06b4d0 0x7fb76c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.942+0000 7fb78643a640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb76c06b4d0 0x7fb76c06d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb778034000 tx=0x7fb77802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.943+0000 7fb78b3a6740 1 Processor -- start 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.943+0000 7fb78b3a6740 1 -- start start 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb78b3a6740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4dfc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb78b3a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 0x55cfd4dfa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb78b3a6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e25140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb78b3a6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4d5e000 con 0x55cfd4e24cc0 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb78b3a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4d06f20 con 0x55cfd4e368d0 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb78b3a6740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4d5a0a0 con 0x55cfd4dfa200 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb786c3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e25140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb786c3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e25140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44124/0 (socket says 172.21.15.38:44124) 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb786c3b640 1 -- 172.21.15.38:0/2418873319 learned_addr learned my addr 172.21.15.38:0/2418873319 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb78643a640 1 --2- 172.21.15.38:0/2418873319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4dfc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb785c39640 1 --2- 172.21.15.38:0/2418873319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 0x55cfd4dfa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb786c3b640 1 -- 172.21.15.38:0/2418873319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 msgr2=0x55cfd4dfc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb786c3b640 1 --2- 172.21.15.38:0/2418873319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4dfc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb786c3b640 1 -- 172.21.15.38:0/2418873319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 msgr2=0x55cfd4dfa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb786c3b640 1 --2- 172.21.15.38:0/2418873319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 0x55cfd4dfa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb786c3b640 1 -- 172.21.15.38:0/2418873319 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfd4b3eba0 con 0x55cfd4e24cc0 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb786c3b640 1 --2- 172.21.15.38:0/2418873319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e25140 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fb7700070b0 tx=0x7fb7700592e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cec797fde8c18c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb767fff640 1 -- 172.21.15.38:0/2418873319 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770061880 con 0x55cfd4e24cc0 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb767fff640 1 -- 172.21.15.38:0/2418873319 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77000ddd0 con 0x55cfd4e24cc0 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.944+0000 7fb767fff640 1 -- 172.21.15.38:0/2418873319 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb770057520 con 0x55cfd4e24cc0 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.945+0000 7fb78b3a6740 1 -- 172.21.15.38:0/2418873319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 msgr2=0x55cfd4e25140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.945+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/2418873319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e25140 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fb7700070b0 tx=0x7fb7700592e0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.945+0000 7fb78b3a6740 1 -- 172.21.15.38:0/2418873319 shutdown_connections 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.945+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/2418873319 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e25140 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.945+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/2418873319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 0x55cfd4dfa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.945+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/2418873319 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4dfc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.945+0000 7fb78b3a6740 1 -- 172.21.15.38:0/2418873319 >> 172.21.15.38:0/2418873319 conn(0x55cfd4dc2050 msgr2=0x55cfd4e3fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.945+0000 7fb78b3a6740 1 -- 172.21.15.38:0/2418873319 shutdown_connections 2024-04-25T08:41:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.945+0000 7fb78b3a6740 1 -- 172.21.15.38:0/2418873319 wait complete. 2024-04-25T08:41:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.946+0000 7fb78b3a6740 1 -- start start 2024-04-25T08:41:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.946+0000 7fb78b3a6740 1 Processor -- start 2024-04-25T08:41:17.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78b3a6740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4e3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78b3a6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e20ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78b3a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 0x55cfd4e21430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78b3a6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4cfc610 con 0x55cfd4e24cc0 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78b3a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4e22810 con 0x55cfd4e368d0 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78b3a6740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4e52860 con 0x55cfd4dfa200 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78b3a6740 10 client.0 _ll_register_callbacks cb 0x55cfd4d14f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb785c39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e20ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78643a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4e3a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb786c3b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 0x55cfd4e21430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb785c39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e20ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44128/0 (socket says 172.21.15.38:44128) 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb785c39640 1 -- 172.21.15.38:0/39368651 learned_addr learned my addr 172.21.15.38:0/39368651 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78643a640 1 -- 172.21.15.38:0/39368651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 msgr2=0x55cfd4e21430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78643a640 1 --2- 172.21.15.38:0/39368651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 0x55cfd4e21430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78643a640 1 -- 172.21.15.38:0/39368651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 msgr2=0x55cfd4e20ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78643a640 1 --2- 172.21.15.38:0/39368651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e20ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78643a640 1 -- 172.21.15.38:0/39368651 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfd4a32490 con 0x55cfd4dfa200 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb78643a640 1 --2- 172.21.15.38:0/39368651 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4e3a2b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb778031000 tx=0x7fb778004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31c8c3c5a0375df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb7657fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3301/0 2024-04-25T08:41:17.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.947+0000 7fb7657fa640 1 -- 172.21.15.38:0/39368651 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77804e020 con 0x55cfd4dfa200 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb78b3a6740 1 -- 172.21.15.38:0/39368651 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb77000ddd0 con 0x55cfd4dfa200 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb7657fa640 1 -- 172.21.15.38:0/39368651 <== mon.2 v2:172.21.15.107:3301/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb778003610 con 0x55cfd4dfa200 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb7657fa640 1 -- 172.21.15.38:0/39368651 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb778037040 con 0x55cfd4dfa200 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb78b3a6740 10 client.7370 fetch_fsmap learned FSMap version 77 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb78b3a6740 1 -- 172.21.15.38:0/39368651 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55cfd4b9f330 con 0x55cfd4dfa200 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb7657fa640 1 -- 172.21.15.38:0/39368651 <== mon.2 v2:172.21.15.107:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77000ddd0 con 0x55cfd4dfa200 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb7657fa640 1 -- 172.21.15.38:0/39368651 <== mon.2 v2:172.21.15.107:3301/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb778003810 con 0x55cfd4dfa200 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb78b3a6740 10 client.7370 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb78b3a6740 10 client.7370 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb78b3a6740 20 client.7370 populate_metadata read hostname 'smithi038' 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/39368651 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55cfd4e3a7f0 0x55cfd4e53590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb78b3a6740 4 client.7370 mds_command: new command op to 9489 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb78b3a6740 1 -- 172.21.15.38:0/39368651 --> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfd4d6b540 con 0x55cfd4e3a7f0 2024-04-25T08:41:17.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.948+0000 7fb785c39640 1 --2- 172.21.15.38:0/39368651 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55cfd4e3a7f0 0x55cfd4e53590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb7657fa640 10 client.7370 ms_handle_connect on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb785c39640 1 --2- 172.21.15.38:0/39368651 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55cfd4e3a7f0 0x55cfd4e53590 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c450a4b0039b954 server_cookie=fc04a17537f9a11c in_seq=0 out_seq=0 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb7657fa640 1 -- 172.21.15.38:0/39368651 <== mds.0 v2:172.21.15.38:6834/1312063561 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cfd4d6b540 con 0x55cfd4e3a7f0 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb7657fa640 10 client.7370 handle_command_reply: tid=0 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb78b3a6740 1 client.7370 shutdown 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb78b3a6740 1 -- 172.21.15.38:0/39368651 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 msgr2=0x55cfd4e3a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/39368651 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4e3a2b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb778031000 tx=0x7fb778004730 comp rx=0 tx=0).stop 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb78b3a6740 1 -- 172.21.15.38:0/39368651 shutdown_connections 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/39368651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 0x55cfd4e21430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/39368651 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e20ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/39368651 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55cfd4e3a7f0 0x55cfd4e53590 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/39368651 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4e3a2b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb7657fa640 0 client.7370 ms_handle_reset on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb78b3a6740 1 -- 172.21.15.38:0/39368651 >> 172.21.15.38:0/39368651 conn(0x55cfd4dc2050 msgr2=0x55cfd4e22c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb78b3a6740 1 -- 172.21.15.38:0/39368651 shutdown_connections 2024-04-25T08:41:17.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb78b3a6740 1 -- 172.21.15.38:0/39368651 wait complete. 2024-04-25T08:41:17.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.949+0000 7fb78b3a6740 20 client.7370 trim_cache size 0 max 16384 2024-04-25T08:41:17.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.965+0000 7fb78b3a6740 1 Processor -- start 2024-04-25T08:41:17.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.965+0000 7fb78b3a6740 1 -- start start 2024-04-25T08:41:17.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.965+0000 7fb78b3a6740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4e59dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.965+0000 7fb78b3a6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4ea4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.965+0000 7fb78b3a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 0x55cfd4e22570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.965+0000 7fb78b3a6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7802000e0 con 0x55cfd4e24cc0 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.965+0000 7fb78b3a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7801ff2a0 con 0x55cfd4e368d0 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.965+0000 7fb78b3a6740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800cc590 con 0x55cfd4dfa200 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.965+0000 7fb785c39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4ea4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.965+0000 7fb785c39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4ea4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44134/0 (socket says 172.21.15.38:44134) 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.965+0000 7fb785c39640 1 -- 172.21.15.38:0/3103569590 learned_addr learned my addr 172.21.15.38:0/3103569590 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb78643a640 1 --2- 172.21.15.38:0/3103569590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4e59dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb786c3b640 1 --2- 172.21.15.38:0/3103569590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 0x55cfd4e22570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb785c39640 1 -- 172.21.15.38:0/3103569590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 msgr2=0x55cfd4e59dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb785c39640 1 --2- 172.21.15.38:0/3103569590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4e59dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb785c39640 1 -- 172.21.15.38:0/3103569590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 msgr2=0x55cfd4e22570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb785c39640 1 --2- 172.21.15.38:0/3103569590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 0x55cfd4e22570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb785c39640 1 -- 172.21.15.38:0/3103569590 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfd4a0b610 con 0x55cfd4e24cc0 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb785c39640 1 --2- 172.21.15.38:0/3103569590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4ea4d90 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x55cfd4e53cf0 tx=0x7fb774008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a9f3c9a4e13fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb7677fe640 1 -- 172.21.15.38:0/3103569590 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7740042f0 con 0x55cfd4e24cc0 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb7677fe640 1 -- 172.21.15.38:0/3103569590 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb774021030 con 0x55cfd4e24cc0 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb7677fe640 1 -- 172.21.15.38:0/3103569590 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77401d530 con 0x55cfd4e24cc0 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3103569590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 msgr2=0x55cfd4ea4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.966+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/3103569590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4ea4d90 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x55cfd4e53cf0 tx=0x7fb774008040 comp rx=0 tx=0).stop 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.967+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3103569590 shutdown_connections 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.967+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/3103569590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e368d0 0x55cfd4e22570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.967+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/3103569590 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4e24cc0 0x55cfd4ea4d90 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.967+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/3103569590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4dfa200 0x55cfd4e59dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.967+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3103569590 >> 172.21.15.38:0/3103569590 conn(0x55cfd4dc2050 msgr2=0x55cfd4e20ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.967+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3103569590 shutdown_connections 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.967+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3103569590 wait complete. 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.967+0000 7fb78b3a6740 1 -- start start 2024-04-25T08:41:17.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.967+0000 7fb78b3a6740 1 Processor -- start 2024-04-25T08:41:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78b3a6740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4dfa200 0x55cfd4df8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78b3a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e28320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78b3a6740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4e368d0 0x55cfd4df91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78b3a6740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7800ccb90 con 0x55cfd4dfa200 2024-04-25T08:41:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78b3a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4e3a2b0 con 0x55cfd4e24cc0 2024-04-25T08:41:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78b3a6740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfd4df8d70 con 0x55cfd4e368d0 2024-04-25T08:41:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78b3a6740 10 client.0 _ll_register_callbacks cb 0x7fb778035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:41:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78643a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4dfa200 0x55cfd4df8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb786c3b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4e368d0 0x55cfd4df91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78643a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4dfa200 0x55cfd4df8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44144/0 (socket says 172.21.15.38:44144) 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78643a640 1 -- 172.21.15.38:0/3639981841 learned_addr learned my addr 172.21.15.38:0/3639981841 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb785c39640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e28320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78643a640 1 -- 172.21.15.38:0/3639981841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4e368d0 msgr2=0x55cfd4df91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78643a640 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4e368d0 0x55cfd4df91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78643a640 1 -- 172.21.15.38:0/3639981841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e24cc0 msgr2=0x55cfd4e28320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78643a640 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e28320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb78643a640 1 -- 172.21.15.38:0/3639981841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfd4e28860 con 0x55cfd4dfa200 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb786c3b640 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4e368d0 0x55cfd4df91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.968+0000 7fb785c39640 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e28320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb78643a640 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4dfa200 0x55cfd4df8830 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fb778031000 tx=0x7fb778009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731d9164e7f47485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb7657fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb7657fa640 1 -- 172.21.15.38:0/3639981841 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77804e040 con 0x55cfd4dfa200 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb7657fa640 1 -- 172.21.15.38:0/3639981841 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb778003810 con 0x55cfd4dfa200 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3639981841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb774021030 con 0x55cfd4dfa200 2024-04-25T08:41:17.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb7657fa640 1 -- 172.21.15.38:0/3639981841 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb778069050 con 0x55cfd4dfa200 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb78b3a6740 10 client.9708 fetch_fsmap learned FSMap version 77 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3639981841 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55cfd4e35930 con 0x55cfd4dfa200 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb7657fa640 1 -- 172.21.15.38:0/3639981841 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb774021030 con 0x55cfd4dfa200 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb7657fa640 1 -- 172.21.15.38:0/3639981841 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb77806c020 con 0x55cfd4dfa200 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb78b3a6740 10 client.9708 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb78b3a6740 10 client.9708 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb78b3a6740 20 client.9708 populate_metadata read hostname 'smithi038' 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55cfd4e62a40 0x55cfd4e9db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb78b3a6740 4 client.9708 mds_command: new command op to 9489 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3639981841 --> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55cfd4d65890 con 0x55cfd4e62a40 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.969+0000 7fb785c39640 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55cfd4e62a40 0x55cfd4e9db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.970+0000 7fb7657fa640 10 client.9708 ms_handle_connect on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:17.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.970+0000 7fb785c39640 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55cfd4e62a40 0x55cfd4e9db10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d397268f026fbbdd server_cookie=89c5a14c74ad1e7a in_seq=0 out_seq=0 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.972+0000 7fb7657fa640 1 -- 172.21.15.38:0/3639981841 <== mds.0 v2:172.21.15.38:6834/1312063561 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cfd4d65890 con 0x55cfd4e62a40 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.972+0000 7fb7657fa640 10 client.9708 handle_command_reply: tid=0 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.972+0000 7fb78b3a6740 1 client.9708 shutdown 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.972+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3639981841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4dfa200 msgr2=0x55cfd4df8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.972+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4dfa200 0x55cfd4df8830 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fb778031000 tx=0x7fb778009240 comp rx=0 tx=0).stop 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.972+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3639981841 shutdown_connections 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.972+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55cfd4e62a40 0x55cfd4e9db10 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.972+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55cfd4e368d0 0x55cfd4df91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.972+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55cfd4e24cc0 0x55cfd4e28320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.972+0000 7fb7657fa640 0 client.9708 ms_handle_reset on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.972+0000 7fb78b3a6740 1 --2- 172.21.15.38:0/3639981841 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cfd4dfa200 0x55cfd4df8830 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3639981841 >> 172.21.15.38:0/3639981841 conn(0x55cfd4dc2050 msgr2=0x55cfd4e22520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3639981841 shutdown_connections 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78b3a6740 1 -- 172.21.15.38:0/3639981841 wait complete. 2024-04-25T08:41:17.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78b3a6740 20 client.9708 trim_cache size 0 max 16384 2024-04-25T08:41:17.969 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:41:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-04-25T08:41:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "8491c0b6-ffff-48dc-9b39-66c785cea120", 2024-04-25T08:41:17.969 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-04-25T08:41:17.969 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:41:17.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78869e640 1 -- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb76c06b4d0 msgr2=0x7fb76c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:17.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78869e640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb76c06b4d0 0x7fb76c06d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fb778034000 tx=0x7fb77802e000 comp rx=0 tx=0).stop 2024-04-25T08:41:17.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78869e640 1 -- 172.21.15.38:0/2913940076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7801612c0 msgr2=0x7fb780150070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:17.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78869e640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7801612c0 0x7fb780150070 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fb77000cf20 tx=0x7fb77000a7d0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78869e640 1 -- 172.21.15.38:0/2913940076 shutdown_connections 2024-04-25T08:41:17.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78869e640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7801612c0 0x7fb780150070 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78869e640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb7800d0870 0x7fb78014fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78869e640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb76c06b4d0 0x7fb76c06d990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78869e640 1 --2- 172.21.15.38:0/2913940076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7800b6a00 0x7fb78014eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:17.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78869e640 1 -- 172.21.15.38:0/2913940076 >> 172.21.15.38:0/2913940076 conn(0x7fb780066d10 msgr2=0x7fb7800c8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:17.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78869e640 1 -- 172.21.15.38:0/2913940076 shutdown_connections 2024-04-25T08:41:17.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:17.973+0000 7fb78869e640 1 -- 172.21.15.38:0/2913940076 wait complete. 2024-04-25T08:41:17.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:17.978 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-25T08:41:18.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.110+0000 7fcd766b3640 1 Processor -- start 2024-04-25T08:41:18.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.110+0000 7fcd766b3640 1 -- start start 2024-04-25T08:41:18.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.110+0000 7fcd766b3640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcd70162ac0 0x7fcd70164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.110+0000 7fcd766b3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd700729c0 con 0x7fcd700c9a40 2024-04-25T08:41:18.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.110+0000 7fcd766b3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd70071920 con 0x7fcd700cbda0 2024-04-25T08:41:18.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.110+0000 7fcd766b3640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd70071c00 con 0x7fcd70162ac0 2024-04-25T08:41:18.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.111+0000 7fcd6f7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcd700c9a40 0x7fcd700cb4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54712/0 (socket says 172.21.15.38:54712) 2024-04-25T08:41:18.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.111+0000 7fcd74c50640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fcd70162ac0 0x7fcd70164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.111+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/729363967 learned_addr learned my addr 172.21.15.38:0/729363967 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:18.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.111+0000 7fcd74c50640 1 -- 172.21.15.38:0/729363967 >> v1:172.21.15.107:6789/0 conn(0x7fcd700cbda0 legacy=0x7fcd700cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.111+0000 7fcd74c50640 1 -- 172.21.15.38:0/729363967 >> v1:172.21.15.38:6789/0 conn(0x7fcd700c9a40 legacy=0x7fcd700cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.111+0000 7fcd74c50640 1 -- 172.21.15.38:0/729363967 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd70071ee0 con 0x7fcd70162ac0 2024-04-25T08:41:18.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.112+0000 7fcd74c50640 1 --2- 172.21.15.38:0/729363967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd70162ac0 0x7fcd70164eb0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009d90 tx=0x7fcd6802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554598e6435d8432 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.112+0000 7fcd6effd640 1 -- 172.21.15.38:0/729363967 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68007740 con 0x7fcd70162ac0 2024-04-25T08:41:18.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.112+0000 7fcd6effd640 1 -- 172.21.15.38:0/729363967 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd680078e0 con 0x7fcd70162ac0 2024-04-25T08:41:18.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.112+0000 7fcd6effd640 1 -- 172.21.15.38:0/729363967 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68007cb0 con 0x7fcd70162ac0 2024-04-25T08:41:18.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.112+0000 7fcd766b3640 1 -- 172.21.15.38:0/729363967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd70162ac0 msgr2=0x7fcd70164eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.112+0000 7fcd766b3640 1 --2- 172.21.15.38:0/729363967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd70162ac0 0x7fcd70164eb0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009d90 tx=0x7fcd6802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.112+0000 7fcd766b3640 1 -- 172.21.15.38:0/729363967 shutdown_connections 2024-04-25T08:41:18.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.112+0000 7fcd766b3640 1 --2- 172.21.15.38:0/729363967 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd70162ac0 0x7fcd70164eb0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.112+0000 7fcd766b3640 1 -- 172.21.15.38:0/729363967 >> 172.21.15.38:0/729363967 conn(0x7fcd7006adc0 msgr2=0x7fcd7006b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.112+0000 7fcd766b3640 1 -- 172.21.15.38:0/729363967 shutdown_connections 2024-04-25T08:41:18.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.112+0000 7fcd766b3640 1 -- 172.21.15.38:0/729363967 wait complete. 2024-04-25T08:41:18.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.114+0000 7fcd766b3640 1 Processor -- start 2024-04-25T08:41:18.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.114+0000 7fcd766b3640 1 -- start start 2024-04-25T08:41:18.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.114+0000 7fcd766b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd700c9a40 0x7fcd70162220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.114+0000 7fcd766b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd700cbda0 0x7fcd701572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.114+0000 7fcd766b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70162ac0 0x7fcd700ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.114+0000 7fcd766b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700cdfd0 con 0x7fcd700cbda0 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.114+0000 7fcd766b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700724a0 con 0x7fcd70162ac0 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.114+0000 7fcd766b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700ce150 con 0x7fcd700c9a40 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd700cbda0 0x7fcd701572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd700cbda0 0x7fcd701572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44158/0 (socket says 172.21.15.38:44158) 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/3803777533 learned_addr learned my addr 172.21.15.38:0/3803777533 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd74c50640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70162ac0 0x7fcd700ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd700c9a40 0x7fcd70162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd700c9a40 msgr2=0x7fcd70162220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6f7fe640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd700c9a40 0x7fcd70162220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70162ac0 msgr2=0x7fcd700ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6f7fe640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70162ac0 0x7fcd700ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/3803777533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd700c31e0 con 0x7fcd700cbda0 2024-04-25T08:41:18.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6f7fe640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd700cbda0 0x7fcd701572d0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800c8c0 tx=0x7fcd5800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6cff9640 1 -- 172.21.15.38:0/3803777533 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58018070 con 0x7fcd700cbda0 2024-04-25T08:41:18.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6cff9640 1 -- 172.21.15.38:0/3803777533 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd580074e0 con 0x7fcd700cbda0 2024-04-25T08:41:18.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd6cff9640 1 -- 172.21.15.38:0/3803777533 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58004a20 con 0x7fcd700cbda0 2024-04-25T08:41:18.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd766b3640 1 -- 172.21.15.38:0/3803777533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd70162760 con 0x7fcd700cbda0 2024-04-25T08:41:18.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.115+0000 7fcd766b3640 1 -- 172.21.15.38:0/3803777533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd700c3730 con 0x7fcd700cbda0 2024-04-25T08:41:18.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.117+0000 7fcd6cff9640 1 -- 172.21.15.38:0/3803777533 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd5801b070 con 0x7fcd700cbda0 2024-04-25T08:41:18.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.117+0000 7fcd6cff9640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd5006b4d0 0x7fcd5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.118+0000 7fcd6cff9640 1 -- 172.21.15.38:0/3803777533 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd58089970 con 0x7fcd700cbda0 2024-04-25T08:41:18.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.118+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd5006b4d0 0x7fcd5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.118+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd5006b4d0 0x7fcd5006d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fcd60000f60 tx=0x7fcd60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.118+0000 7fcd793b8740 1 Processor -- start 2024-04-25T08:41:18.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.118+0000 7fcd793b8740 1 -- start start 2024-04-25T08:41:18.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.119+0000 7fcd793b8740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b18850ca0 0x562b18852e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.119+0000 7fcd793b8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 0x562b18851080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.119+0000 7fcd793b8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b1887b760 0x562b1887bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.119+0000 7fcd793b8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b187b3b90 con 0x562b1888d370 2024-04-25T08:41:18.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.119+0000 7fcd793b8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b1875ce40 con 0x562b1887b760 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.119+0000 7fcd793b8740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b187afc30 con 0x562b18850ca0 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.119+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 0x562b18851080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.119+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 0x562b18851080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44160/0 (socket says 172.21.15.38:44160) 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.119+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/1726585224 learned_addr learned my addr 172.21.15.38:0/1726585224 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.119+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/1726585224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b18850ca0 0x562b18852e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.119+0000 7fcd74c50640 1 --2- 172.21.15.38:0/1726585224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b1887b760 0x562b1887bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/1726585224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b18850ca0 msgr2=0x562b18852e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd6f7fe640 1 --2- 172.21.15.38:0/1726585224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b18850ca0 0x562b18852e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/1726585224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b1887b760 msgr2=0x562b1887bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd6f7fe640 1 --2- 172.21.15.38:0/1726585224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b1887b760 0x562b1887bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/1726585224 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b185927d0 con 0x562b1888d370 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd74c50640 1 --2- 172.21.15.38:0/1726585224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b1887b760 0x562b1887bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd6f7fe640 1 --2- 172.21.15.38:0/1726585224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 0x562b18851080 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fcd58087cb0 tx=0x7fcd58007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ec030ba9c816d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd65ffb640 1 -- 172.21.15.38:0/1726585224 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58056310 con 0x562b1888d370 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd65ffb640 1 -- 172.21.15.38:0/1726585224 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd58054680 con 0x562b1888d370 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd65ffb640 1 -- 172.21.15.38:0/1726585224 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd58004e00 con 0x562b1888d370 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd793b8740 1 -- 172.21.15.38:0/1726585224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 msgr2=0x562b18851080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd793b8740 1 --2- 172.21.15.38:0/1726585224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 0x562b18851080 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fcd58087cb0 tx=0x7fcd58007f90 comp rx=0 tx=0).stop 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd793b8740 1 -- 172.21.15.38:0/1726585224 shutdown_connections 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd793b8740 1 --2- 172.21.15.38:0/1726585224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b1887b760 0x562b1887bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd793b8740 1 --2- 172.21.15.38:0/1726585224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 0x562b18851080 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd793b8740 1 --2- 172.21.15.38:0/1726585224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b18850ca0 0x562b18852e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.120+0000 7fcd793b8740 1 -- 172.21.15.38:0/1726585224 >> 172.21.15.38:0/1726585224 conn(0x562b18818870 msgr2=0x562b18896760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.121+0000 7fcd793b8740 1 -- 172.21.15.38:0/1726585224 shutdown_connections 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.121+0000 7fcd793b8740 1 -- 172.21.15.38:0/1726585224 wait complete. 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.121+0000 7fcd793b8740 1 -- start start 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.121+0000 7fcd793b8740 1 Processor -- start 2024-04-25T08:41:18.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.122+0000 7fcd793b8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b18850ca0 0x562b188a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.122+0000 7fcd793b8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1887b760 0x562b1888d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.122+0000 7fcd793b8740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1888d370 0x562b1888efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.122+0000 7fcd793b8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b187703e0 con 0x562b1887b760 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.122+0000 7fcd793b8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b188792b0 con 0x562b18850ca0 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.122+0000 7fcd793b8740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b1888dcf0 con 0x562b1888d370 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.122+0000 7fcd793b8740 10 client.0 _ll_register_callbacks cb 0x562b1876fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.122+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1887b760 0x562b1888d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.122+0000 7fcd6f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1887b760 0x562b1888d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44164/0 (socket says 172.21.15.38:44164) 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.122+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/4070276617 learned_addr learned my addr 172.21.15.38:0/4070276617 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.122+0000 7fcd74c50640 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1888d370 0x562b1888efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.122+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b18850ca0 0x562b188a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/4070276617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1888d370 msgr2=0x562b1888efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd6f7fe640 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1888d370 0x562b1888efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/4070276617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b18850ca0 msgr2=0x562b188a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd6f7fe640 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b18850ca0 0x562b188a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd6f7fe640 1 -- 172.21.15.38:0/4070276617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b184880f0 con 0x562b1887b760 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd6f7fe640 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1887b760 0x562b1888d7b0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fcd58054420 tx=0x7fcd58000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adf12aceb0c15d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd4effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd4effd640 1 -- 172.21.15.38:0/4070276617 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5805e070 con 0x562b1887b760 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd4effd640 1 -- 172.21.15.38:0/4070276617 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd58087050 con 0x562b1887b760 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd793b8740 1 -- 172.21.15.38:0/4070276617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd58054680 con 0x562b1887b760 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd4effd640 1 -- 172.21.15.38:0/4070276617 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd5805e210 con 0x562b1887b760 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd793b8740 10 client.9723 fetch_fsmap learned FSMap version 77 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd793b8740 1 -- 172.21.15.38:0/4070276617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x562b185f2f60 con 0x562b1887b760 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd4effd640 1 -- 172.21.15.38:0/4070276617 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd58054680 con 0x562b1887b760 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd4effd640 1 -- 172.21.15.38:0/4070276617 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd58057460 con 0x562b1887b760 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd793b8740 10 client.9723 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T08:41:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd793b8740 10 client.9723 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd793b8740 20 client.9723 populate_metadata read hostname 'smithi038' 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd793b8740 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562b1888f9d0 0x562b1887bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd793b8740 4 client.9723 mds_command: new command op to 9489 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.123+0000 7fcd793b8740 1 -- 172.21.15.38:0/4070276617 --> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b187c10d0 con 0x562b1888f9d0 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.124+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562b1888f9d0 0x562b1887bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.124+0000 7fcd4effd640 10 client.9723 ms_handle_connect on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.124+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562b1888f9d0 0x562b1887bbe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=965f424936772b89 server_cookie=5ed2fc196001a2a in_seq=0 out_seq=0 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.124+0000 7fcd4effd640 1 -- 172.21.15.38:0/4070276617 <== mds.0 v2:172.21.15.38:6834/1312063561 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562b187c10d0 con 0x562b1888f9d0 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.124+0000 7fcd4effd640 10 client.9723 handle_command_reply: tid=0 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.124+0000 7fcd793b8740 1 client.9723 shutdown 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.125+0000 7fcd793b8740 1 -- 172.21.15.38:0/4070276617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1887b760 msgr2=0x562b1888d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.125+0000 7fcd793b8740 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1887b760 0x562b1888d7b0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fcd58054420 tx=0x7fcd58000ed0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.125+0000 7fcd793b8740 1 -- 172.21.15.38:0/4070276617 shutdown_connections 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.125+0000 7fcd793b8740 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562b1888f9d0 0x562b1887bbe0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.125+0000 7fcd793b8740 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1888d370 0x562b1888efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.125+0000 7fcd793b8740 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1887b760 0x562b1888d7b0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.125+0000 7fcd793b8740 1 --2- 172.21.15.38:0/4070276617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b18850ca0 0x562b188a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.125+0000 7fcd4effd640 0 client.9723 ms_handle_reset on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:18.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.125+0000 7fcd793b8740 1 -- 172.21.15.38:0/4070276617 >> 172.21.15.38:0/4070276617 conn(0x562b18818870 msgr2=0x562b18879490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.125+0000 7fcd793b8740 1 -- 172.21.15.38:0/4070276617 shutdown_connections 2024-04-25T08:41:18.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.125+0000 7fcd793b8740 1 -- 172.21.15.38:0/4070276617 wait complete. 2024-04-25T08:41:18.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.125+0000 7fcd793b8740 20 client.9723 trim_cache size 0 max 16384 2024-04-25T08:41:18.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd793b8740 1 Processor -- start 2024-04-25T08:41:18.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd793b8740 1 -- start start 2024-04-25T08:41:18.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd793b8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b18850ca0 0x562b1887d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd793b8740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1887b760 0x562b1887f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd793b8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b1888d370 0x562b1887ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd793b8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd702004f0 con 0x562b18850ca0 2024-04-25T08:41:18.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd793b8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd701ff6d0 con 0x562b1888d370 2024-04-25T08:41:18.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd793b8740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700ad680 con 0x562b1887b760 2024-04-25T08:41:18.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b18850ca0 0x562b1887d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b18850ca0 0x562b1887d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44166/0 (socket says 172.21.15.38:44166) 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd6ffff640 1 -- 172.21.15.38:0/1216232275 learned_addr learned my addr 172.21.15.38:0/1216232275 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd6f7fe640 1 --2- 172.21.15.38:0/1216232275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1887b760 0x562b1887f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.140+0000 7fcd74c50640 1 --2- 172.21.15.38:0/1216232275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b1888d370 0x562b1887ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd6ffff640 1 -- 172.21.15.38:0/1216232275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1887b760 msgr2=0x562b1887f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/1216232275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1887b760 0x562b1887f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd6ffff640 1 -- 172.21.15.38:0/1216232275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b1888d370 msgr2=0x562b1887ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/1216232275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b1888d370 0x562b1887ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd6ffff640 1 -- 172.21.15.38:0/1216232275 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b18890f50 con 0x562b18850ca0 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd6f7fe640 1 --2- 172.21.15.38:0/1216232275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1887b760 0x562b1887f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd74c50640 1 --2- 172.21.15.38:0/1216232275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b1888d370 0x562b1887ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/1216232275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b18850ca0 0x562b1887d100 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fcd60001630 tx=0x7fcd60012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69220e9ad154863 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd657fa640 1 -- 172.21.15.38:0/1216232275 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60014b10 con 0x562b18850ca0 2024-04-25T08:41:18.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd657fa640 1 -- 172.21.15.38:0/1216232275 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6001b040 con 0x562b18850ca0 2024-04-25T08:41:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd657fa640 1 -- 172.21.15.38:0/1216232275 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60014cb0 con 0x562b18850ca0 2024-04-25T08:41:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd793b8740 1 -- 172.21.15.38:0/1216232275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b18850ca0 msgr2=0x562b1887d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd793b8740 1 --2- 172.21.15.38:0/1216232275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b18850ca0 0x562b1887d100 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fcd60001630 tx=0x7fcd60012000 comp rx=0 tx=0).stop 2024-04-25T08:41:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd793b8740 1 -- 172.21.15.38:0/1216232275 shutdown_connections 2024-04-25T08:41:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd793b8740 1 --2- 172.21.15.38:0/1216232275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b1888d370 0x562b1887ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd793b8740 1 --2- 172.21.15.38:0/1216232275 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1887b760 0x562b1887f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd793b8740 1 --2- 172.21.15.38:0/1216232275 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b18850ca0 0x562b1887d100 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.141+0000 7fcd793b8740 1 -- 172.21.15.38:0/1216232275 >> 172.21.15.38:0/1216232275 conn(0x562b18818870 msgr2=0x562b18879430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.142+0000 7fcd793b8740 1 -- 172.21.15.38:0/1216232275 shutdown_connections 2024-04-25T08:41:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.142+0000 7fcd793b8740 1 -- 172.21.15.38:0/1216232275 wait complete. 2024-04-25T08:41:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.142+0000 7fcd793b8740 1 -- start start 2024-04-25T08:41:18.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.142+0000 7fcd793b8740 1 Processor -- start 2024-04-25T08:41:18.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd793b8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b18850ca0 0x562b188fa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd793b8740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1887b760 0x562b188af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd793b8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 0x562b18900b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd793b8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b18890be0 con 0x562b1888d370 2024-04-25T08:41:18.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd793b8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd701ff0d0 con 0x562b18850ca0 2024-04-25T08:41:18.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd793b8740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b1888d9c0 con 0x562b1887b760 2024-04-25T08:41:18.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd793b8740 10 client.0 _ll_register_callbacks cb 0x7fcd5806e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:41:18.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd74c50640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 0x562b18900b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd74c50640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 0x562b18900b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44170/0 (socket says 172.21.15.38:44170) 2024-04-25T08:41:18.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd74c50640 1 -- 172.21.15.38:0/938232915 learned_addr learned my addr 172.21.15.38:0/938232915 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd6f7fe640 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1887b760 0x562b188af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b18850ca0 0x562b188fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd74c50640 1 -- 172.21.15.38:0/938232915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1887b760 msgr2=0x562b188af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd74c50640 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1887b760 0x562b188af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd74c50640 1 -- 172.21.15.38:0/938232915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b18850ca0 msgr2=0x562b188fa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd74c50640 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b18850ca0 0x562b188fa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd74c50640 1 -- 172.21.15.38:0/938232915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b188a6610 con 0x562b1888d370 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.143+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b18850ca0 0x562b188fa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd74c50640 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 0x562b18900b80 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fcd6802abe0 tx=0x7fcd68004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84894b76917adc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd4effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd4effd640 1 -- 172.21.15.38:0/938232915 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68004780 con 0x562b1888d370 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd4effd640 1 -- 172.21.15.38:0/938232915 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcd68007540 con 0x562b1888d370 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd793b8740 1 -- 172.21.15.38:0/938232915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcd6001b040 con 0x562b1888d370 2024-04-25T08:41:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd4effd640 1 -- 172.21.15.38:0/938232915 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd68007740 con 0x562b1888d370 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd793b8740 10 client.9729 fetch_fsmap learned FSMap version 77 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd793b8740 1 -- 172.21.15.38:0/938232915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x562b184611b0 con 0x562b1888d370 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd4effd640 1 -- 172.21.15.38:0/938232915 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6001b040 con 0x562b1888d370 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd4effd640 1 -- 172.21.15.38:0/938232915 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcd680078e0 con 0x562b1888d370 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd793b8740 10 client.9729 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd793b8740 10 client.9729 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd793b8740 20 client.9729 populate_metadata read hostname 'smithi038' 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd793b8740 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562b1888e7c0 0x562b18877500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd793b8740 4 client.9729 mds_command: new command op to 9489 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd793b8740 1 -- 172.21.15.38:0/938232915 --> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x562b187bb420 con 0x562b1888e7c0 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.144+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562b1888e7c0 0x562b18877500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd4effd640 10 client.9729 ms_handle_connect on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd6ffff640 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562b1888e7c0 0x562b18877500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc3824cdc3132dde server_cookie=8a4c3859fae0b16b in_seq=0 out_seq=0 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd4effd640 1 -- 172.21.15.38:0/938232915 <== mds.0 v2:172.21.15.38:6834/1312063561 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x562b187bb420 con 0x562b1888e7c0 2024-04-25T08:41:18.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd4effd640 10 client.9729 handle_command_reply: tid=0 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd793b8740 1 client.9729 shutdown 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd793b8740 1 -- 172.21.15.38:0/938232915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 msgr2=0x562b18900b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd793b8740 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 0x562b18900b80 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fcd6802abe0 tx=0x7fcd68004ab0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd793b8740 1 -- 172.21.15.38:0/938232915 shutdown_connections 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd793b8740 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562b1888d370 0x562b18900b80 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd793b8740 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x562b1888e7c0 0x562b18877500 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd793b8740 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562b1887b760 0x562b188af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd793b8740 1 --2- 172.21.15.38:0/938232915 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562b18850ca0 0x562b188fa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd4effd640 0 client.9729 ms_handle_reset on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd793b8740 1 -- 172.21.15.38:0/938232915 >> 172.21.15.38:0/938232915 conn(0x562b18818870 msgr2=0x562b18879430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd793b8740 1 -- 172.21.15.38:0/938232915 shutdown_connections 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd793b8740 1 -- 172.21.15.38:0/938232915 wait complete. 2024-04-25T08:41:18.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.145+0000 7fcd793b8740 20 client.9729 trim_cache size 0 max 16384 2024-04-25T08:41:18.145 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:41:18.145 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0 2024-04-25T08:41:18.145 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:41:18.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.146+0000 7fcd766b3640 1 -- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd5006b4d0 msgr2=0x7fcd5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.146+0000 7fcd766b3640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd5006b4d0 0x7fcd5006d990 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fcd60000f60 tx=0x7fcd60009040 comp rx=0 tx=0).stop 2024-04-25T08:41:18.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.146+0000 7fcd766b3640 1 -- 172.21.15.38:0/3803777533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd700cbda0 msgr2=0x7fcd701572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.146+0000 7fcd766b3640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd700cbda0 0x7fcd701572d0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fcd5800c8c0 tx=0x7fcd5800cd80 comp rx=0 tx=0).stop 2024-04-25T08:41:18.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.146+0000 7fcd766b3640 1 -- 172.21.15.38:0/3803777533 shutdown_connections 2024-04-25T08:41:18.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.146+0000 7fcd766b3640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70162ac0 0x7fcd700ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.146+0000 7fcd766b3640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcd700cbda0 0x7fcd701572d0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.146+0000 7fcd766b3640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcd5006b4d0 0x7fcd5006d990 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.146+0000 7fcd766b3640 1 --2- 172.21.15.38:0/3803777533 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd700c9a40 0x7fcd70162220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.146+0000 7fcd766b3640 1 -- 172.21.15.38:0/3803777533 >> 172.21.15.38:0/3803777533 conn(0x7fcd7006adc0 msgr2=0x7fcd7006cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.146+0000 7fcd766b3640 1 -- 172.21.15.38:0/3803777533 shutdown_connections 2024-04-25T08:41:18.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.146+0000 7fcd766b3640 1 -- 172.21.15.38:0/3803777533 wait complete. 2024-04-25T08:41:18.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:18.149 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-25T08:41:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.259+0000 7fcc1d0bd640 1 Processor -- start 2024-04-25T08:41:18.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.259+0000 7fcc1d0bd640 1 -- start start 2024-04-25T08:41:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc1d0bd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcc1815df30 0x7fcc1809a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc1d0bd640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc18162820 con 0x7fcc18162390 2024-04-25T08:41:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc1d0bd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1815f300 con 0x7fcc1809a180 2024-04-25T08:41:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc1d0bd640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1815e310 con 0x7fcc1815df30 2024-04-25T08:41:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc0ffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcc18162390 0x7fcc18099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:54728/0 (socket says 172.21.15.38:54728) 2024-04-25T08:41:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc0ffff640 1 -- 172.21.15.38:0/3316451436 learned_addr learned my addr 172.21.15.38:0/3316451436 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/3316451436 >> v2:172.21.15.107:3300/0 conn(0x7fcc1815df30 0x7fcc1809a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc1659c640 1 -- 172.21.15.38:0/3316451436 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967598847 0 0) 0x7fcc00001110 con 0x7fcc18162390 2024-04-25T08:41:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc1659c640 1 -- 172.21.15.38:0/3316451436 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc04003520 con 0x7fcc18162390 2024-04-25T08:41:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc1659c640 1 -- 172.21.15.38:0/3316451436 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769478096 0 0) 0x7fcc1815f300 con 0x7fcc1809a180 2024-04-25T08:41:18.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc1659c640 1 -- 172.21.15.38:0/3316451436 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc00001110 con 0x7fcc1809a180 2024-04-25T08:41:18.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc16d9d640 1 -- 172.21.15.38:0/3316451436 >> v1:172.21.15.107:6789/0 conn(0x7fcc1809a180 legacy=0x7fcc180cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc16d9d640 1 -- 172.21.15.38:0/3316451436 >> v1:172.21.15.38:6789/0 conn(0x7fcc18162390 legacy=0x7fcc18099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc1659c640 1 -- 172.21.15.38:0/3316451436 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3688532970 0 0) 0x7fcc04003520 con 0x7fcc18162390 2024-04-25T08:41:18.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.260+0000 7fcc16d9d640 1 -- 172.21.15.38:0/3316451436 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1815e540 con 0x7fcc1815df30 2024-04-25T08:41:18.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.261+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/3316451436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc1815df30 0x7fcc1809a7b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fcc080097d0 tx=0x7fcc0802a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bdac54bc034b342 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.261+0000 7fcc1659c640 1 -- 172.21.15.38:0/3316451436 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08037070 con 0x7fcc1815df30 2024-04-25T08:41:18.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.261+0000 7fcc1659c640 1 -- 172.21.15.38:0/3316451436 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0802ac90 con 0x7fcc1815df30 2024-04-25T08:41:18.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.261+0000 7fcc1659c640 1 -- 172.21.15.38:0/3316451436 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00001840 con 0x7fcc1815df30 2024-04-25T08:41:18.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.262+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/3316451436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc1815df30 msgr2=0x7fcc1809a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.262+0000 7fcc1d0bd640 1 --2- 172.21.15.38:0/3316451436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc1815df30 0x7fcc1809a7b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fcc080097d0 tx=0x7fcc0802a200 comp rx=0 tx=0).stop 2024-04-25T08:41:18.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.262+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/3316451436 shutdown_connections 2024-04-25T08:41:18.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.262+0000 7fcc1d0bd640 1 --2- 172.21.15.38:0/3316451436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc1815df30 0x7fcc1809a7b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.262+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/3316451436 >> 172.21.15.38:0/3316451436 conn(0x7fcc180b3110 msgr2=0x7fcc180b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.262+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/3316451436 shutdown_connections 2024-04-25T08:41:18.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.262+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/3316451436 wait complete. 2024-04-25T08:41:18.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.263+0000 7fcc1d0bd640 1 Processor -- start 2024-04-25T08:41:18.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.263+0000 7fcc1d0bd640 1 -- start start 2024-04-25T08:41:18.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.264+0000 7fcc1d0bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc1809a180 0x7fcc180c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.264+0000 7fcc1d0bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc1815df30 0x7fcc180cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.264+0000 7fcc1d0bd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc18162390 0x7fcc180c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.264+0000 7fcc1d0bd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1815ee70 con 0x7fcc1809a180 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.264+0000 7fcc1d0bd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1809aeb0 con 0x7fcc1815df30 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.264+0000 7fcc1d0bd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1815eb00 con 0x7fcc18162390 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.264+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc1809a180 0x7fcc180c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.264+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc1809a180 0x7fcc180c9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44174/0 (socket says 172.21.15.38:44174) 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.264+0000 7fcc16d9d640 1 -- 172.21.15.38:0/800449008 learned_addr learned my addr 172.21.15.38:0/800449008 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.264+0000 7fcc1759e640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc18162390 0x7fcc180c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.265+0000 7fcc16d9d640 1 -- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc18162390 msgr2=0x7fcc180c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.265+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc18162390 0x7fcc180c5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.265+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc1815df30 0x7fcc180cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.265+0000 7fcc16d9d640 1 -- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc1815df30 msgr2=0x7fcc180cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.265+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc1815df30 0x7fcc180cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.265+0000 7fcc16d9d640 1 -- 172.21.15.38:0/800449008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc180cc570 con 0x7fcc1809a180 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.265+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc1809a180 0x7fcc180c9880 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fcc0802a240 tx=0x7fcc080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.265+0000 7fcc0f7fe640 1 -- 172.21.15.38:0/800449008 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08041070 con 0x7fcc1809a180 2024-04-25T08:41:18.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.265+0000 7fcc0f7fe640 1 -- 172.21.15.38:0/800449008 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0803c660 con 0x7fcc1809a180 2024-04-25T08:41:18.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.265+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/800449008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc180cc850 con 0x7fcc1809a180 2024-04-25T08:41:18.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.265+0000 7fcc0f7fe640 1 -- 172.21.15.38:0/800449008 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08037040 con 0x7fcc1809a180 2024-04-25T08:41:18.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.265+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/800449008 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc180cce10 con 0x7fcc1809a180 2024-04-25T08:41:18.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.267+0000 7fcc0f7fe640 1 -- 172.21.15.38:0/800449008 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc08044070 con 0x7fcc1809a180 2024-04-25T08:41:18.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.267+0000 7fcc0f7fe640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcbf806b500 0x7fcbf806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.267+0000 7fcc0f7fe640 1 -- 172.21.15.38:0/800449008 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc080aab10 con 0x7fcc1809a180 2024-04-25T08:41:18.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.267+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcbf806b500 0x7fcbf806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.268+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcbf806b500 0x7fcbf806d9c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fcc180cbee0 tx=0x7fcc00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.268+0000 7fcc1fdc0740 1 Processor -- start 2024-04-25T08:41:18.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.268+0000 7fcc1fdc0740 1 -- start start 2024-04-25T08:41:18.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc1fdc0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500b10 0x55e405503320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc1fdc0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 0x55e405503860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc1fdc0740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e4054d3770 0x55e4054d6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc1fdc0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e40543cda0 con 0x55e405500ef0 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc1fdc0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4053e5b80 con 0x55e405500b10 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc1fdc0740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e405438e40 con 0x55e4054d3770 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc0ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 0x55e405503860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc0ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 0x55e405503860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44176/0 (socket says 172.21.15.38:44176) 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc0ffff640 1 -- 172.21.15.38:0/4141198523 learned_addr learned my addr 172.21.15.38:0/4141198523 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc1759e640 1 --2- 172.21.15.38:0/4141198523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e4054d3770 0x55e4054d6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/4141198523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500b10 0x55e405503320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc0ffff640 1 -- 172.21.15.38:0/4141198523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e4054d3770 msgr2=0x55e4054d6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/4141198523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e4054d3770 0x55e4054d6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc0ffff640 1 -- 172.21.15.38:0/4141198523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500b10 msgr2=0x55e405503320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/4141198523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500b10 0x55e405503320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.269+0000 7fcc0ffff640 1 -- 172.21.15.38:0/4141198523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4053eff10 con 0x55e405500ef0 2024-04-25T08:41:18.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/4141198523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 0x55e405503860 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fcc000013b0 tx=0x7fcc00010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b19babf4004929 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc0cff9640 1 -- 172.21.15.38:0/4141198523 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000dbb0 con 0x55e405500ef0 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc0cff9640 1 -- 172.21.15.38:0/4141198523 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0000dd50 con 0x55e405500ef0 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc0cff9640 1 -- 172.21.15.38:0/4141198523 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00020690 con 0x55e405500ef0 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/4141198523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 msgr2=0x55e405503860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/4141198523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 0x55e405503860 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fcc000013b0 tx=0x7fcc00010000 comp rx=0 tx=0).stop 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/4141198523 shutdown_connections 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/4141198523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e4054d3770 0x55e4054d6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/4141198523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 0x55e405503860 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/4141198523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500b10 0x55e405503320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/4141198523 >> 172.21.15.38:0/4141198523 conn(0x55e4054d59e0 msgr2=0x55e40551e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/4141198523 shutdown_connections 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.270+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/4141198523 wait complete. 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.271+0000 7fcc1fdc0740 1 -- start start 2024-04-25T08:41:18.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.271+0000 7fcc1fdc0740 1 Processor -- start 2024-04-25T08:41:18.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc1fdc0740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500b10 0x55e405503e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc1fdc0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 0x55e405504390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc1fdc0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4054f7f30 0x55e405505560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc1fdc0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4054d8b90 con 0x55e405500ef0 2024-04-25T08:41:18.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc1fdc0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4053db4b0 con 0x55e4054f7f30 2024-04-25T08:41:18.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc1fdc0740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4054d8e70 con 0x55e405500b10 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc1fdc0740 10 client.0 _ll_register_callbacks cb 0x55e40541b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc0ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 0x55e405504390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc0ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 0x55e405504390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44180/0 (socket says 172.21.15.38:44180) 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc0ffff640 1 -- 172.21.15.38:0/2577341471 learned_addr learned my addr 172.21.15.38:0/2577341471 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500b10 0x55e405503e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc1759e640 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4054f7f30 0x55e405505560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc0ffff640 1 -- 172.21.15.38:0/2577341471 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500b10 msgr2=0x55e405503e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500b10 0x55e405503e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc0ffff640 1 -- 172.21.15.38:0/2577341471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4054f7f30 msgr2=0x55e405505560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4054f7f30 0x55e405505560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc0ffff640 1 -- 172.21.15.38:0/2577341471 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e40521b7d0 con 0x55e405500ef0 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 0x55e405504390 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fcc180c82c0 tx=0x7fcc00012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50505f4957ff80ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.272+0000 7fcbf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcbf5ffb640 1 -- 172.21.15.38:0/2577341471 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc000058d0 con 0x55e405500ef0 2024-04-25T08:41:18.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcbf5ffb640 1 -- 172.21.15.38:0/2577341471 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc000227d0 con 0x55e405500ef0 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/2577341471 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc0000dd50 con 0x55e405500ef0 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcbf5ffb640 1 -- 172.21.15.38:0/2577341471 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc00032050 con 0x55e405500ef0 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcc1fdc0740 10 client.9741 fetch_fsmap learned FSMap version 77 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/2577341471 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55e4051513f0 con 0x55e405500ef0 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcbf5ffb640 1 -- 172.21.15.38:0/2577341471 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0000dd50 con 0x55e405500ef0 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcbf5ffb640 1 -- 172.21.15.38:0/2577341471 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcc00035020 con 0x55e405500ef0 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcc1fdc0740 10 client.9741 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcc1fdc0740 10 client.9741 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcc1fdc0740 20 client.9741 populate_metadata read hostname 'smithi038' 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55e405505aa0 0x55e405505e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcc1fdc0740 4 client.9741 mds_command: new command op to 9489 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/2577341471 --> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e40544a2e0 con 0x55e405505aa0 2024-04-25T08:41:18.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55e405505aa0 0x55e405505e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcbf5ffb640 10 client.9741 ms_handle_connect on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.273+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55e405505aa0 0x55e405505e80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1120d8a9c25df25d server_cookie=20932b5f39bccb2c in_seq=0 out_seq=0 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcbf5ffb640 1 -- 172.21.15.38:0/2577341471 <== mds.0 v2:172.21.15.38:6834/1312063561 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e40544a2e0 con 0x55e405505aa0 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcbf5ffb640 10 client.9741 handle_command_reply: tid=0 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcc1fdc0740 1 client.9741 shutdown 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/2577341471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 msgr2=0x55e405504390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 0x55e405504390 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fcc180c82c0 tx=0x7fcc00012000 comp rx=0 tx=0).stop 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/2577341471 shutdown_connections 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e4054f7f30 0x55e405505560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e405500ef0 0x55e405504390 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55e405505aa0 0x55e405505e80 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/2577341471 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500b10 0x55e405503e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcbf5ffb640 0 client.9741 ms_handle_reset on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:18.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/2577341471 >> 172.21.15.38:0/2577341471 conn(0x55e4054d3770 msgr2=0x55e4054ff190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/2577341471 shutdown_connections 2024-04-25T08:41:18.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/2577341471 wait complete. 2024-04-25T08:41:18.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.274+0000 7fcc1fdc0740 20 client.9741 trim_cache size 0 max 16384 2024-04-25T08:41:18.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.289+0000 7fcc1fdc0740 1 Processor -- start 2024-04-25T08:41:18.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.289+0000 7fcc1fdc0740 1 -- start start 2024-04-25T08:41:18.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc1fdc0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 0x55e405526280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc1fdc0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500b10 0x55e40551a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc1fdc0740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500ef0 0x55e40551a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc1fdc0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc18200300 con 0x55e4054f7f30 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc1fdc0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc181ff510 con 0x55e405500b10 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc1fdc0740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc18159660 con 0x55e405500ef0 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 0x55e405526280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 0x55e405526280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44182/0 (socket says 172.21.15.38:44182) 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc16d9d640 1 -- 172.21.15.38:0/574443410 learned_addr learned my addr 172.21.15.38:0/574443410 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc1759e640 1 --2- 172.21.15.38:0/574443410 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500ef0 0x55e40551a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/574443410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500b10 0x55e40551a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc16d9d640 1 -- 172.21.15.38:0/574443410 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500ef0 msgr2=0x55e40551a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/574443410 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500ef0 0x55e40551a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc16d9d640 1 -- 172.21.15.38:0/574443410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500b10 msgr2=0x55e40551a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.290+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/574443410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500b10 0x55e40551a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcc16d9d640 1 -- 172.21.15.38:0/574443410 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4051110f0 con 0x55e4054f7f30 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/574443410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 0x55e405526280 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fcc080097d0 tx=0x7fcc0803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95a2f515a4b7f70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcbf7fff640 1 -- 172.21.15.38:0/574443410 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08095070 con 0x55e4054f7f30 2024-04-25T08:41:18.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcbf7fff640 1 -- 172.21.15.38:0/574443410 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0807d2e0 con 0x55e4054f7f30 2024-04-25T08:41:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcbf7fff640 1 -- 172.21.15.38:0/574443410 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0804b040 con 0x55e4054f7f30 2024-04-25T08:41:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/574443410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 msgr2=0x55e405526280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/574443410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 0x55e405526280 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fcc080097d0 tx=0x7fcc0803f040 comp rx=0 tx=0).stop 2024-04-25T08:41:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/574443410 shutdown_connections 2024-04-25T08:41:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/574443410 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500ef0 0x55e40551a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/574443410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500b10 0x55e40551a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/574443410 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 0x55e405526280 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/574443410 >> 172.21.15.38:0/574443410 conn(0x55e4054d3770 msgr2=0x55e4054da250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/574443410 shutdown_connections 2024-04-25T08:41:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.291+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/574443410 wait complete. 2024-04-25T08:41:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.292+0000 7fcc1fdc0740 1 -- start start 2024-04-25T08:41:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.292+0000 7fcc1fdc0740 1 Processor -- start 2024-04-25T08:41:18.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc1fdc0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 0x55e405588950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc1fdc0740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500b10 0x55e4054fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc1fdc0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500ef0 0x55e4054fabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc1fdc0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4054d4d60 con 0x55e4054f7f30 2024-04-25T08:41:18.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc1fdc0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1815a920 con 0x55e405500ef0 2024-04-25T08:41:18.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc1fdc0740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4054d3f00 con 0x55e405500b10 2024-04-25T08:41:18.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc1fdc0740 10 client.0 _ll_register_callbacks cb 0x7fcc000016b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 0x55e405588950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc16d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 0x55e405588950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44192/0 (socket says 172.21.15.38:44192) 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc16d9d640 1 -- 172.21.15.38:0/204903610 learned_addr learned my addr 172.21.15.38:0/204903610 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500b10 0x55e4054fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc1759e640 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500ef0 0x55e4054fabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc16d9d640 1 -- 172.21.15.38:0/204903610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500b10 msgr2=0x55e4054fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500b10 0x55e4054fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc16d9d640 1 -- 172.21.15.38:0/204903610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500ef0 msgr2=0x55e4054fabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500ef0 0x55e4054fabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc16d9d640 1 -- 172.21.15.38:0/204903610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4054d7a60 con 0x55e4054f7f30 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc16d9d640 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 0x55e405588950 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fcc08000c00 tx=0x7fcc08048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adeb9ed1d86524f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcbf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcbf5ffb640 1 -- 172.21.15.38:0/204903610 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08095070 con 0x55e4054f7f30 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcbf5ffb640 1 -- 172.21.15.38:0/204903610 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc08075800 con 0x55e4054f7f30 2024-04-25T08:41:18.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.293+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/204903610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc0807d2e0 con 0x55e4054f7f30 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcbf5ffb640 1 -- 172.21.15.38:0/204903610 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc0804b070 con 0x55e4054f7f30 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcc1fdc0740 10 client.9747 fetch_fsmap learned FSMap version 77 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/204903610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55e4054d7d50 con 0x55e4054f7f30 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcbf5ffb640 1 -- 172.21.15.38:0/204903610 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0807d2e0 con 0x55e4054f7f30 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcbf5ffb640 1 -- 172.21.15.38:0/204903610 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 77) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcc08081410 con 0x55e4054f7f30 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcc1fdc0740 10 client.9747 fetch_fsmap finished waiting for FSMap version 77 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcc1fdc0740 10 client.9747 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcc1fdc0740 20 client.9747 populate_metadata read hostname 'smithi038' 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55e40551ecd0 0x55e40558c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcc1fdc0740 4 client.9747 mds_command: new command op to 9489 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/204903610 --> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e405444630 con 0x55e40551ecd0 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55e40551ecd0 0x55e40558c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcbf5ffb640 10 client.9747 ms_handle_connect on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcc0ffff640 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55e40551ecd0 0x55e40558c660 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599979f86fc8cdd6 server_cookie=1821634c8f773404 in_seq=0 out_seq=0 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcbf5ffb640 1 -- 172.21.15.38:0/204903610 <== mds.0 v2:172.21.15.38:6834/1312063561 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55e405444630 con 0x55e40551ecd0 2024-04-25T08:41:18.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcbf5ffb640 10 client.9747 handle_command_reply: tid=0 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.294+0000 7fcc1fdc0740 1 client.9747 shutdown 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/204903610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 msgr2=0x55e405588950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 0x55e405588950 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fcc08000c00 tx=0x7fcc08048040 comp rx=0 tx=0).stop 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/204903610 shutdown_connections 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e405500ef0 0x55e4054fabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.38:6834/1312063561,v1:172.21.15.38:6835/1312063561] conn(0x55e40551ecd0 0x55e40558c660 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55e405500b10 0x55e4054fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcbf5ffb640 0 client.9747 ms_handle_reset on v2:172.21.15.38:6834/1312063561 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1fdc0740 1 --2- 172.21.15.38:0/204903610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e4054f7f30 0x55e405588950 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/204903610 >> 172.21.15.38:0/204903610 conn(0x55e4054d3770 msgr2=0x55e405505e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/204903610 shutdown_connections 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1fdc0740 1 -- 172.21.15.38:0/204903610 wait complete. 2024-04-25T08:41:18.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1fdc0740 20 client.9747 trim_cache size 0 max 16384 2024-04-25T08:41:18.294 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:41:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T08:41:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-04-25T08:41:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "8491c0b6-ffff-48dc-9b39-66c785cea120": { 2024-04-25T08:41:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/scrub_control_test_path", 2024-04-25T08:41:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "8491c0b6-ffff-48dc-9b39-66c785cea120", 2024-04-25T08:41:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive" 2024-04-25T08:41:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:41:18.294 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:41:18.294 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:41:18.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcbf806b500 msgr2=0x7fcbf806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1d0bd640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcbf806b500 0x7fcbf806d9c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fcc180cbee0 tx=0x7fcc00009040 comp rx=0 tx=0).stop 2024-04-25T08:41:18.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/800449008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc1809a180 msgr2=0x7fcc180c9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1d0bd640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc1809a180 0x7fcc180c9880 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7fcc0802a240 tx=0x7fcc080046c0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/800449008 shutdown_connections 2024-04-25T08:41:18.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1d0bd640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcbf806b500 0x7fcbf806d9c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1d0bd640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcc18162390 0x7fcc180c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1d0bd640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcc1815df30 0x7fcc180cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1d0bd640 1 --2- 172.21.15.38:0/800449008 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcc1809a180 0x7fcc180c9880 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/800449008 >> 172.21.15.38:0/800449008 conn(0x7fcc180b3110 msgr2=0x7fcc1806c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/800449008 shutdown_connections 2024-04-25T08:41:18.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.295+0000 7fcc1d0bd640 1 -- 172.21.15.38:0/800449008 wait complete. 2024-04-25T08:41:18.299 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:18.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.408+0000 7f4be3db0640 1 Processor -- start 2024-04-25T08:41:18.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.408+0000 7f4be3db0640 1 -- start start 2024-04-25T08:41:18.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.409+0000 7f4be3db0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bdc15dc10 0x7f4bdc0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.409+0000 7f4be3db0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4bdc0ced60 0x7f4bdc0d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.409+0000 7f4be3db0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4bdc0d3900 0x7f4bdc0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.409+0000 7f4be3db0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bdc15dff0 con 0x7f4bdc0d3900 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.409+0000 7f4be3db0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bdc0b6990 con 0x7f4bdc0ced60 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.409+0000 7f4be3db0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0d4710 con 0x7f4bdc15dc10 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.409+0000 7f4be234d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4bdc0d3900 0x7f4bdc0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.409+0000 7f4be234d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4bdc0d3900 0x7f4bdc0d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44194/0 (socket says 172.21.15.38:44194) 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.409+0000 7f4be234d640 1 -- 172.21.15.38:0/637981073 learned_addr learned my addr 172.21.15.38:0/637981073 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.410+0000 7f4be134b640 1 --2- 172.21.15.38:0/637981073 >> v2:172.21.15.107:3300/0 conn(0x7f4bdc0ced60 0x7f4bdc0d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.410+0000 7f4be1b4c640 1 --2- 172.21.15.38:0/637981073 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bdc15dc10 0x7f4bdc0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.410+0000 7f4be234d640 1 -- 172.21.15.38:0/637981073 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bdc15dc10 msgr2=0x7f4bdc0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.410+0000 7f4be234d640 1 --2- 172.21.15.38:0/637981073 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bdc15dc10 0x7f4bdc0ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.410+0000 7f4be234d640 1 -- 172.21.15.38:0/637981073 >> v2:172.21.15.107:3300/0 conn(0x7f4bdc0ced60 msgr2=0x7f4bdc0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.410+0000 7f4be234d640 1 --2- 172.21.15.38:0/637981073 >> v2:172.21.15.107:3300/0 conn(0x7f4bdc0ced60 0x7f4bdc0d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.410+0000 7f4be234d640 1 -- 172.21.15.38:0/637981073 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc0d3e70 con 0x7f4bdc0d3900 2024-04-25T08:41:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.410+0000 7f4be234d640 1 --2- 172.21.15.38:0/637981073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bdc0d3900 0x7f4bdc0d0d90 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc06cfd0 tx=0x7f4bcc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404ba9c74dc13112 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.411+0000 7f4be0b4a640 1 -- 172.21.15.38:0/637981073 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc037070 con 0x7f4bdc0d3900 2024-04-25T08:41:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.411+0000 7f4be0b4a640 1 -- 172.21.15.38:0/637981073 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bcc032440 con 0x7f4bdc0d3900 2024-04-25T08:41:18.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.411+0000 7f4be0b4a640 1 -- 172.21.15.38:0/637981073 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc032810 con 0x7f4bdc0d3900 2024-04-25T08:41:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.411+0000 7f4be3db0640 1 -- 172.21.15.38:0/637981073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bdc0d3900 msgr2=0x7f4bdc0d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.411+0000 7f4be3db0640 1 --2- 172.21.15.38:0/637981073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bdc0d3900 0x7f4bdc0d0d90 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc06cfd0 tx=0x7f4bcc02a140 comp rx=0 tx=0).stop 2024-04-25T08:41:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.411+0000 7f4be3db0640 1 -- 172.21.15.38:0/637981073 shutdown_connections 2024-04-25T08:41:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.411+0000 7f4be3db0640 1 --2- 172.21.15.38:0/637981073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bdc0d3900 0x7f4bdc0d0d90 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.411+0000 7f4be3db0640 1 --2- 172.21.15.38:0/637981073 >> v2:172.21.15.107:3300/0 conn(0x7f4bdc0ced60 0x7f4bdc0d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.411+0000 7f4be3db0640 1 --2- 172.21.15.38:0/637981073 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bdc15dc10 0x7f4bdc0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.411+0000 7f4be3db0640 1 -- 172.21.15.38:0/637981073 >> 172.21.15.38:0/637981073 conn(0x7f4bdc066d10 msgr2=0x7f4bdc067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.411+0000 7f4be3db0640 1 -- 172.21.15.38:0/637981073 shutdown_connections 2024-04-25T08:41:18.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.411+0000 7f4be3db0640 1 -- 172.21.15.38:0/637981073 wait complete. 2024-04-25T08:41:18.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.412+0000 7f4be3db0640 1 Processor -- start 2024-04-25T08:41:18.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.412+0000 7f4be3db0640 1 -- start start 2024-04-25T08:41:18.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.413+0000 7f4be3db0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bdc0ced60 0x7f4bdc0c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.413+0000 7f4be3db0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bdc0d3900 0x7f4bdc153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.413+0000 7f4be3db0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bdc15dc10 0x7f4bdc153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.413+0000 7f4be3db0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0b6bc0 con 0x7f4bdc0d3900 2024-04-25T08:41:18.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.413+0000 7f4be3db0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0d4430 con 0x7f4bdc0ced60 2024-04-25T08:41:18.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.413+0000 7f4be3db0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc0b2370 con 0x7f4bdc15dc10 2024-04-25T08:41:18.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.413+0000 7f4be134b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bdc0d3900 0x7f4bdc153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.413+0000 7f4be134b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bdc0d3900 0x7f4bdc153620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44196/0 (socket says 172.21.15.38:44196) 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.413+0000 7f4be134b640 1 -- 172.21.15.38:0/2685845403 learned_addr learned my addr 172.21.15.38:0/2685845403 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.413+0000 7f4be234d640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bdc15dc10 0x7f4bdc153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.413+0000 7f4be1b4c640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bdc0ced60 0x7f4bdc0c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.414+0000 7f4be134b640 1 -- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bdc15dc10 msgr2=0x7f4bdc153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.414+0000 7f4be134b640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bdc15dc10 0x7f4bdc153b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.414+0000 7f4be134b640 1 -- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bdc0ced60 msgr2=0x7f4bdc0c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.414+0000 7f4be134b640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bdc0ced60 0x7f4bdc0c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.414+0000 7f4be134b640 1 -- 172.21.15.38:0/2685845403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc0cc9f0 con 0x7f4bdc0d3900 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.414+0000 7f4be234d640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bdc15dc10 0x7f4bdc153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.414+0000 7f4be134b640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bdc0d3900 0x7f4bdc153620 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000e8c0 tx=0x7f4bd000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.414+0000 7f4bda7fc640 1 -- 172.21.15.38:0/2685845403 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0017070 con 0x7f4bdc0d3900 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.414+0000 7f4bda7fc640 1 -- 172.21.15.38:0/2685845403 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd000c940 con 0x7f4bdc0d3900 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.414+0000 7f4be3db0640 1 -- 172.21.15.38:0/2685845403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bdc0cccd0 con 0x7f4bdc0d3900 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.414+0000 7f4bda7fc640 1 -- 172.21.15.38:0/2685845403 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd000cd10 con 0x7f4bdc0d3900 2024-04-25T08:41:18.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.414+0000 7f4be3db0640 1 -- 172.21.15.38:0/2685845403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bdc0cd290 con 0x7f4bdc0d3900 2024-04-25T08:41:18.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.415+0000 7f4be3db0640 1 -- 172.21.15.38:0/2685845403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bdc15d470 con 0x7f4bdc0d3900 2024-04-25T08:41:18.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.415+0000 7f4bda7fc640 1 -- 172.21.15.38:0/2685845403 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bd001a070 con 0x7f4bdc0d3900 2024-04-25T08:41:18.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.416+0000 7f4bda7fc640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bc806b4d0 0x7f4bc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.416+0000 7f4bda7fc640 1 -- 172.21.15.38:0/2685845403 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4bd008a670 con 0x7f4bdc0d3900 2024-04-25T08:41:18.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.416+0000 7f4be1b4c640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bc806b4d0 0x7f4bc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.416+0000 7f4be1b4c640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bc806b4d0 0x7f4bc806d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40073f0 tx=0x7f4bd400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.420+0000 7f4bda7fc640 1 -- 172.21.15.38:0/2685845403 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4bdc15d470 con 0x7f4bdc0d3900 2024-04-25T08:41:18.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.640+0000 7f4be3db0640 1 -- 172.21.15.38:0/2685845403 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4bdc068e80 con 0x7f4bdc0d3900 2024-04-25T08:41:18.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.640+0000 7f4bda7fc640 1 -- 172.21.15.38:0/2685845403 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f4bd00593f0 con 0x7f4bdc0d3900 2024-04-25T08:41:18.633 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:18.634 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:40:55.730538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/1312063561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1312063561},{"type":"v1","addr":"172.21.15.38:6835","nonce":1312063561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:18.634 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 77 2024-04-25T08:41:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.642+0000 7f4be3db0640 1 -- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bc806b4d0 msgr2=0x7f4bc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.642+0000 7f4be3db0640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bc806b4d0 0x7f4bc806d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f4bd40073f0 tx=0x7f4bd400b040 comp rx=0 tx=0).stop 2024-04-25T08:41:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.642+0000 7f4be3db0640 1 -- 172.21.15.38:0/2685845403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bdc0d3900 msgr2=0x7f4bdc153620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.642+0000 7f4be3db0640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bdc0d3900 0x7f4bdc153620 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000e8c0 tx=0x7f4bd000ed80 comp rx=0 tx=0).stop 2024-04-25T08:41:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.642+0000 7f4be3db0640 1 -- 172.21.15.38:0/2685845403 shutdown_connections 2024-04-25T08:41:18.636 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.642+0000 7f4be3db0640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4bc806b4d0 0x7f4bc806d990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.642+0000 7f4be3db0640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4bdc15dc10 0x7f4bdc153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.642+0000 7f4be3db0640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4bdc0d3900 0x7f4bdc153620 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.642+0000 7f4be3db0640 1 --2- 172.21.15.38:0/2685845403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4bdc0ced60 0x7f4bdc0c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.642+0000 7f4be3db0640 1 -- 172.21.15.38:0/2685845403 >> 172.21.15.38:0/2685845403 conn(0x7f4bdc066d10 msgr2=0x7f4bdc0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.642+0000 7f4be3db0640 1 -- 172.21.15.38:0/2685845403 shutdown_connections 2024-04-25T08:41:18.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.642+0000 7f4be3db0640 1 -- 172.21.15.38:0/2685845403 wait complete. 2024-04-25T08:41:18.646 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T08:41:18.646 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:41:18.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.752+0000 7fe107542640 1 Processor -- start 2024-04-25T08:41:18.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.752+0000 7fe107542640 1 -- start start 2024-04-25T08:41:18.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe107542640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe100099a70 0x7fe100099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe107542640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe1000cf470 0x7fe1000cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe107542640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10009a390 con 0x7fe10006bd80 2024-04-25T08:41:18.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe107542640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1001614d0 con 0x7fe100099a70 2024-04-25T08:41:18.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe107542640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1000cd970 con 0x7fe1000cf470 2024-04-25T08:41:18.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe1052de640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe100099a70 0x7fe100099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe1052de640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe100099a70 0x7fe100099e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44204/0 (socket says 172.21.15.38:44204) 2024-04-25T08:41:18.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe1052de640 1 -- 172.21.15.38:0/1284692004 learned_addr learned my addr 172.21.15.38:0/1284692004 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe104add640 1 --2- 172.21.15.38:0/1284692004 >> v2:172.21.15.107:3300/0 conn(0x7fe1000cf470 0x7fe1000cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe1052de640 1 -- 172.21.15.38:0/1284692004 >> v2:172.21.15.107:3300/0 conn(0x7fe1000cf470 msgr2=0x7fe1000cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:18.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe1052de640 1 --2- 172.21.15.38:0/1284692004 >> v2:172.21.15.107:3300/0 conn(0x7fe1000cf470 0x7fe1000cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe1052de640 1 -- 172.21.15.38:0/1284692004 >> v1:172.21.15.107:6789/0 conn(0x7fe10006bd80 legacy=0x7fe10006c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.753+0000 7fe1052de640 1 -- 172.21.15.38:0/1284692004 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1000ccd60 con 0x7fe100099a70 2024-04-25T08:41:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.754+0000 7fe1052de640 1 --2- 172.21.15.38:0/1284692004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe100099a70 0x7fe100099e50 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fe100073060 tx=0x7fe0fc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4df947f31a1f8ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.754+0000 7fe0f7fff640 1 -- 172.21.15.38:0/1284692004 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc036070 con 0x7fe100099a70 2024-04-25T08:41:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.754+0000 7fe0f7fff640 1 -- 172.21.15.38:0/1284692004 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc031600 con 0x7fe100099a70 2024-04-25T08:41:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.754+0000 7fe0f7fff640 1 -- 172.21.15.38:0/1284692004 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0fc0319d0 con 0x7fe100099a70 2024-04-25T08:41:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.755+0000 7fe107542640 1 -- 172.21.15.38:0/1284692004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe100099a70 msgr2=0x7fe100099e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.755+0000 7fe107542640 1 --2- 172.21.15.38:0/1284692004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe100099a70 0x7fe100099e50 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fe100073060 tx=0x7fe0fc02a730 comp rx=0 tx=0).stop 2024-04-25T08:41:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.755+0000 7fe107542640 1 -- 172.21.15.38:0/1284692004 shutdown_connections 2024-04-25T08:41:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.755+0000 7fe107542640 1 --2- 172.21.15.38:0/1284692004 >> v2:172.21.15.107:3300/0 conn(0x7fe1000cf470 0x7fe1000cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.755+0000 7fe107542640 1 --2- 172.21.15.38:0/1284692004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe100099a70 0x7fe100099e50 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.755+0000 7fe107542640 1 -- 172.21.15.38:0/1284692004 >> 172.21.15.38:0/1284692004 conn(0x7fe1000b3180 msgr2=0x7fe1000b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:18.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.755+0000 7fe107542640 1 -- 172.21.15.38:0/1284692004 shutdown_connections 2024-04-25T08:41:18.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.755+0000 7fe107542640 1 -- 172.21.15.38:0/1284692004 wait complete. 2024-04-25T08:41:18.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.756+0000 7fe107542640 1 Processor -- start 2024-04-25T08:41:18.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.756+0000 7fe107542640 1 -- start start 2024-04-25T08:41:18.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe107542640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10006bd80 0x7fe1000b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe107542640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe100099a70 0x7fe1000ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe107542640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1000cf470 0x7fe10015f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe107542640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000cd600 con 0x7fe1000cf470 2024-04-25T08:41:18.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe107542640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000cd320 con 0x7fe10006bd80 2024-04-25T08:41:18.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe107542640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000cc9c0 con 0x7fe100099a70 2024-04-25T08:41:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe105adf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1000cf470 0x7fe10015f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe1052de640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10006bd80 0x7fe1000b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe105adf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1000cf470 0x7fe10015f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44214/0 (socket says 172.21.15.38:44214) 2024-04-25T08:41:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe1052de640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10006bd80 0x7fe1000b6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.38:45832/0 (socket says 172.21.15.38:45832) 2024-04-25T08:41:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe104add640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe100099a70 0x7fe1000ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe105adf640 1 -- 172.21.15.38:0/150691047 learned_addr learned my addr 172.21.15.38:0/150691047 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe105adf640 1 -- 172.21.15.38:0/150691047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe100099a70 msgr2=0x7fe1000ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe105adf640 1 --2- 172.21.15.38:0/150691047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe100099a70 0x7fe1000ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe105adf640 1 -- 172.21.15.38:0/150691047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10006bd80 msgr2=0x7fe1000b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe105adf640 1 --2- 172.21.15.38:0/150691047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10006bd80 0x7fe1000b6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe105adf640 1 -- 172.21.15.38:0/150691047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1000aeea0 con 0x7fe1000cf470 2024-04-25T08:41:18.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe105adf640 1 --2- 172.21.15.38:0/150691047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1000cf470 0x7fe10015f4a0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800c3f0 tx=0x7fe0f800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.757+0000 7fe0f5ffb640 1 -- 172.21.15.38:0/150691047 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80052d0 con 0x7fe1000cf470 2024-04-25T08:41:18.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.758+0000 7fe0f5ffb640 1 -- 172.21.15.38:0/150691047 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8005490 con 0x7fe1000cf470 2024-04-25T08:41:18.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.758+0000 7fe107542640 1 -- 172.21.15.38:0/150691047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1000af180 con 0x7fe1000cf470 2024-04-25T08:41:18.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.758+0000 7fe0f5ffb640 1 -- 172.21.15.38:0/150691047 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8007260 con 0x7fe1000cf470 2024-04-25T08:41:18.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.758+0000 7fe107542640 1 -- 172.21.15.38:0/150691047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1000afa20 con 0x7fe1000cf470 2024-04-25T08:41:18.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.759+0000 7fe107542640 1 -- 172.21.15.38:0/150691047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1000af570 con 0x7fe1000cf470 2024-04-25T08:41:18.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.759+0000 7fe0f5ffb640 1 -- 172.21.15.38:0/150691047 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0f8007400 con 0x7fe1000cf470 2024-04-25T08:41:18.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.760+0000 7fe0f5ffb640 1 --2- 172.21.15.38:0/150691047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe0e406b4d0 0x7fe0e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:18.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.760+0000 7fe0f5ffb640 1 -- 172.21.15.38:0/150691047 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0f80897c0 con 0x7fe1000cf470 2024-04-25T08:41:18.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.760+0000 7fe1052de640 1 --2- 172.21.15.38:0/150691047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe0e406b4d0 0x7fe0e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:18.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.760+0000 7fe1052de640 1 --2- 172.21.15.38:0/150691047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe0e406b4d0 0x7fe0e406d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc033000 tx=0x7fe0fc009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:18.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:18.764+0000 7fe0f5ffb640 1 -- 172.21.15.38:0/150691047 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0f8057dc0 con 0x7fe1000cf470 2024-04-25T08:41:19.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.014+0000 7fe107542640 1 -- 172.21.15.38:0/150691047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe100068e80 con 0x7fe1000cf470 2024-04-25T08:41:19.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.014+0000 7fe0f5ffb640 1 -- 172.21.15.38:0/150691047 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7fe0f805be60 con 0x7fe1000cf470 2024-04-25T08:41:19.007 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:19.007 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":607,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1005,"data_bytes":1031222793,"bytes_used":2473779200,"bytes_avail":770620334080,"bytes_total":773094113280,"write_bytes_sec":85874048,"read_op_per_sec":0,"write_op_per_sec":84},"fsmap":{"epoch":77,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9489}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T08:40:35.331262+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:41:19.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.016+0000 7fe107542640 1 -- 172.21.15.38:0/150691047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe0e406b4d0 msgr2=0x7fe0e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:19.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.016+0000 7fe107542640 1 --2- 172.21.15.38:0/150691047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe0e406b4d0 0x7fe0e406d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc033000 tx=0x7fe0fc009000 comp rx=0 tx=0).stop 2024-04-25T08:41:19.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.016+0000 7fe107542640 1 -- 172.21.15.38:0/150691047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1000cf470 msgr2=0x7fe10015f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:19.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.016+0000 7fe107542640 1 --2- 172.21.15.38:0/150691047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1000cf470 0x7fe10015f4a0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800c3f0 tx=0x7fe0f800c8b0 comp rx=0 tx=0).stop 2024-04-25T08:41:19.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.016+0000 7fe107542640 1 -- 172.21.15.38:0/150691047 shutdown_connections 2024-04-25T08:41:19.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.016+0000 7fe107542640 1 --2- 172.21.15.38:0/150691047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1000cf470 0x7fe10015f4a0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:19.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.016+0000 7fe107542640 1 --2- 172.21.15.38:0/150691047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe0e406b4d0 0x7fe0e406d990 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:19.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.016+0000 7fe107542640 1 --2- 172.21.15.38:0/150691047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe100099a70 0x7fe1000ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:19.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.016+0000 7fe107542640 1 --2- 172.21.15.38:0/150691047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe10006bd80 0x7fe1000b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:19.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.016+0000 7fe107542640 1 -- 172.21.15.38:0/150691047 >> 172.21.15.38:0/150691047 conn(0x7fe1000b3180 msgr2=0x7fe1000ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:19.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.016+0000 7fe107542640 1 -- 172.21.15.38:0/150691047 shutdown_connections 2024-04-25T08:41:19.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:19.016+0000 7fe107542640 1 -- 172.21.15.38:0/150691047 wait complete. 2024-04-25T08:41:19.019 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:41:20.021 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:20.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.138+0000 7fc7703ab640 1 Processor -- start 2024-04-25T08:41:20.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.138+0000 7fc7703ab640 1 -- start start 2024-04-25T08:41:20.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc7703ab640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc768161700 0x7fc7680b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:20.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc7703ab640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc768161e00 con 0x7fc7680b5710 2024-04-25T08:41:20.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc7703ab640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7681620e0 con 0x7fc76815ddf0 2024-04-25T08:41:20.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc7703ab640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680b9930 con 0x7fc768161700 2024-04-25T08:41:20.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc76e147640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc7680b5710 0x7fc7680b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44830/0 (socket says 172.21.15.38:44830) 2024-04-25T08:41:20.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc76e147640 1 -- 172.21.15.38:0/869844688 learned_addr learned my addr 172.21.15.38:0/869844688 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:20.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc76e948640 1 --2- 172.21.15.38:0/869844688 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc768161700 0x7fc7680b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:20.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc76d145640 1 -- 172.21.15.38:0/869844688 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830727859 0 0) 0x7fc768161e00 con 0x7fc7680b5710 2024-04-25T08:41:20.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc76d145640 1 -- 172.21.15.38:0/869844688 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc754003520 con 0x7fc7680b5710 2024-04-25T08:41:20.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc76d145640 1 -- 172.21.15.38:0/869844688 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1565608747 0 0) 0x7fc7681620e0 con 0x7fc76815ddf0 2024-04-25T08:41:20.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc76d145640 1 -- 172.21.15.38:0/869844688 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc768161e00 con 0x7fc76815ddf0 2024-04-25T08:41:20.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc76d145640 1 -- 172.21.15.38:0/869844688 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2058707039 0 0) 0x7fc754003520 con 0x7fc7680b5710 2024-04-25T08:41:20.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.139+0000 7fc76e948640 1 -- 172.21.15.38:0/869844688 >> v1:172.21.15.107:6789/0 conn(0x7fc76815ddf0 legacy=0x7fc7680d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:20.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.140+0000 7fc76e948640 1 -- 172.21.15.38:0/869844688 >> v1:172.21.15.38:6789/0 conn(0x7fc7680b5710 legacy=0x7fc7680b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:20.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.140+0000 7fc76e948640 1 -- 172.21.15.38:0/869844688 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7681623c0 con 0x7fc768161700 2024-04-25T08:41:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.140+0000 7fc76e948640 1 --2- 172.21.15.38:0/869844688 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc768161700 0x7fc7680b93f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc760009dc0 tx=0x7fc76002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c06ef71268ab6e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.141+0000 7fc76d145640 1 -- 172.21.15.38:0/869844688 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7640015f0 con 0x7fc768161700 2024-04-25T08:41:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.141+0000 7fc76d145640 1 -- 172.21.15.38:0/869844688 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc760007540 con 0x7fc768161700 2024-04-25T08:41:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.141+0000 7fc76d145640 1 -- 172.21.15.38:0/869844688 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760007910 con 0x7fc768161700 2024-04-25T08:41:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.141+0000 7fc7703ab640 1 -- 172.21.15.38:0/869844688 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc768161700 msgr2=0x7fc7680b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.141+0000 7fc7703ab640 1 --2- 172.21.15.38:0/869844688 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc768161700 0x7fc7680b93f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc760009dc0 tx=0x7fc76002a660 comp rx=0 tx=0).stop 2024-04-25T08:41:20.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.141+0000 7fc7703ab640 1 -- 172.21.15.38:0/869844688 shutdown_connections 2024-04-25T08:41:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.141+0000 7fc7703ab640 1 --2- 172.21.15.38:0/869844688 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc768161700 0x7fc7680b93f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.141+0000 7fc7703ab640 1 -- 172.21.15.38:0/869844688 >> 172.21.15.38:0/869844688 conn(0x7fc7680d07c0 msgr2=0x7fc7680d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.141+0000 7fc7703ab640 1 -- 172.21.15.38:0/869844688 shutdown_connections 2024-04-25T08:41:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.141+0000 7fc7703ab640 1 -- 172.21.15.38:0/869844688 wait complete. 2024-04-25T08:41:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.142+0000 7fc7703ab640 1 Processor -- start 2024-04-25T08:41:20.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.142+0000 7fc7703ab640 1 -- start start 2024-04-25T08:41:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.143+0000 7fc7703ab640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7680b5710 0x7fc7680b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.143+0000 7fc7703ab640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc76815ddf0 0x7fc7681f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.143+0000 7fc7703ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc768161700 0x7fc768160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.143+0000 7fc7703ab640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680d2e20 con 0x7fc768161700 2024-04-25T08:41:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.143+0000 7fc7703ab640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680d2ca0 con 0x7fc7680b5710 2024-04-25T08:41:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.143+0000 7fc7703ab640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680d2fa0 con 0x7fc76815ddf0 2024-04-25T08:41:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.143+0000 7fc76e948640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc768161700 0x7fc768160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.143+0000 7fc76e948640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc768161700 0x7fc768160690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49922/0 (socket says 172.21.15.38:49922) 2024-04-25T08:41:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.143+0000 7fc76e948640 1 -- 172.21.15.38:0/1960671830 learned_addr learned my addr 172.21.15.38:0/1960671830 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:20.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.143+0000 7fc76e147640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7680b5710 0x7fc7680b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.143+0000 7fc76d946640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc76815ddf0 0x7fc7681f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.144+0000 7fc76e948640 1 -- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc76815ddf0 msgr2=0x7fc7681f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.144+0000 7fc76e948640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc76815ddf0 0x7fc7681f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.144+0000 7fc76e948640 1 -- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7680b5710 msgr2=0x7fc7680b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.144+0000 7fc76e948640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7680b5710 0x7fc7680b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.144+0000 7fc76e948640 1 -- 172.21.15.38:0/1960671830 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7681f4690 con 0x7fc768161700 2024-04-25T08:41:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.144+0000 7fc76e948640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc768161700 0x7fc768160690 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fc760009ee0 tx=0x7fc760004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.144+0000 7fc75effd640 1 -- 172.21.15.38:0/1960671830 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc760040070 con 0x7fc768161700 2024-04-25T08:41:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.144+0000 7fc75effd640 1 -- 172.21.15.38:0/1960671830 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76003c830 con 0x7fc768161700 2024-04-25T08:41:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.144+0000 7fc7703ab640 1 -- 172.21.15.38:0/1960671830 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7680c77f0 con 0x7fc768161700 2024-04-25T08:41:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.144+0000 7fc75effd640 1 -- 172.21.15.38:0/1960671830 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76003cbe0 con 0x7fc768161700 2024-04-25T08:41:20.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.144+0000 7fc7703ab640 1 -- 172.21.15.38:0/1960671830 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7680c7db0 con 0x7fc768161700 2024-04-25T08:41:20.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.145+0000 7fc7703ab640 1 -- 172.21.15.38:0/1960671830 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc768068e80 con 0x7fc768161700 2024-04-25T08:41:20.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.145+0000 7fc75effd640 1 -- 172.21.15.38:0/1960671830 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc760039040 con 0x7fc768161700 2024-04-25T08:41:20.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.146+0000 7fc75effd640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc74c06b4d0 0x7fc74c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:20.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.146+0000 7fc75effd640 1 -- 172.21.15.38:0/1960671830 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc7600ab8f0 con 0x7fc768161700 2024-04-25T08:41:20.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.146+0000 7fc76e147640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc74c06b4d0 0x7fc74c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:20.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.147+0000 7fc76e147640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc74c06b4d0 0x7fc74c06d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc764001eb0 tx=0x7fc764001630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:20.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.150+0000 7fc75effd640 1 -- 172.21.15.38:0/1960671830 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7681f4690 con 0x7fc768161700 2024-04-25T08:41:20.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.368+0000 7fc7703ab640 1 -- 172.21.15.38:0/1960671830 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7680cfeb0 con 0x7fc768161700 2024-04-25T08:41:20.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.368+0000 7fc75effd640 1 -- 172.21.15.38:0/1960671830 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fc760079f90 con 0x7fc768161700 2024-04-25T08:41:20.361 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:20.361 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:40:55.730538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/1312063561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1312063561},{"type":"v1","addr":"172.21.15.38:6835","nonce":1312063561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:20.362 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 77 2024-04-25T08:41:20.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.371+0000 7fc7703ab640 1 -- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc74c06b4d0 msgr2=0x7fc74c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:20.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.371+0000 7fc7703ab640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc74c06b4d0 0x7fc74c06d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc764001eb0 tx=0x7fc764001630 comp rx=0 tx=0).stop 2024-04-25T08:41:20.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.371+0000 7fc7703ab640 1 -- 172.21.15.38:0/1960671830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc768161700 msgr2=0x7fc768160690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:20.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.371+0000 7fc7703ab640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc768161700 0x7fc768160690 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fc760009ee0 tx=0x7fc760004a40 comp rx=0 tx=0).stop 2024-04-25T08:41:20.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.371+0000 7fc7703ab640 1 -- 172.21.15.38:0/1960671830 shutdown_connections 2024-04-25T08:41:20.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.371+0000 7fc7703ab640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc768161700 0x7fc768160690 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.371+0000 7fc7703ab640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc74c06b4d0 0x7fc74c06d990 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.371+0000 7fc7703ab640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc76815ddf0 0x7fc7681f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.371+0000 7fc7703ab640 1 --2- 172.21.15.38:0/1960671830 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7680b5710 0x7fc7680b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.371+0000 7fc7703ab640 1 -- 172.21.15.38:0/1960671830 >> 172.21.15.38:0/1960671830 conn(0x7fc7680d07c0 msgr2=0x7fc7680b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:20.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.371+0000 7fc7703ab640 1 -- 172.21.15.38:0/1960671830 shutdown_connections 2024-04-25T08:41:20.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.371+0000 7fc7703ab640 1 -- 172.21.15.38:0/1960671830 wait complete. 2024-04-25T08:41:20.374 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T08:41:20.374 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:41:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.481+0000 7f8113c6f640 1 Processor -- start 2024-04-25T08:41:20.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.481+0000 7f8113c6f640 1 -- start start 2024-04-25T08:41:20.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.482+0000 7f8113c6f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f810c0b5710 0x7f810c161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.482+0000 7f8113c6f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f810c15ddf0 0x7f810c0b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.482+0000 7f8113c6f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f810c0b5af0 0x7f810c06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.482+0000 7f8113c6f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810c0b9930 con 0x7f810c15ddf0 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.482+0000 7f8113c6f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f810c161e00 con 0x7f810c0b5af0 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.482+0000 7f8113c6f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810c1620e0 con 0x7f810c0b5710 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.482+0000 7f811120a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f810c15ddf0 0x7f810c0b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.482+0000 7f811120a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f810c15ddf0 0x7f810c0b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49932/0 (socket says 172.21.15.38:49932) 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.482+0000 7f811120a640 1 -- 172.21.15.38:0/669551759 learned_addr learned my addr 172.21.15.38:0/669551759 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.483+0000 7f8111a0b640 1 --2- 172.21.15.38:0/669551759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f810c0b5710 0x7f810c161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.483+0000 7f811220c640 1 --2- 172.21.15.38:0/669551759 >> v2:172.21.15.107:3300/0 conn(0x7f810c0b5af0 0x7f810c06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.483+0000 7f811120a640 1 -- 172.21.15.38:0/669551759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f810c0b5710 msgr2=0x7f810c161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.483+0000 7f811120a640 1 --2- 172.21.15.38:0/669551759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f810c0b5710 0x7f810c161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.483+0000 7f811120a640 1 -- 172.21.15.38:0/669551759 >> v2:172.21.15.107:3300/0 conn(0x7f810c0b5af0 msgr2=0x7f810c06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:20.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.483+0000 7f811120a640 1 --2- 172.21.15.38:0/669551759 >> v2:172.21.15.107:3300/0 conn(0x7f810c0b5af0 0x7f810c06bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.483+0000 7f811120a640 1 -- 172.21.15.38:0/669551759 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c1623c0 con 0x7f810c15ddf0 2024-04-25T08:41:20.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.483+0000 7f811220c640 1 --2- 172.21.15.38:0/669551759 >> v2:172.21.15.107:3300/0 conn(0x7f810c0b5af0 0x7f810c06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:20.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.483+0000 7f811120a640 1 --2- 172.21.15.38:0/669551759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c15ddf0 0x7f810c0b93f0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f810c0d1bd0 tx=0x7f80fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ae32baa65377bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:20.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.483+0000 7f8110a09640 1 -- 172.21.15.38:0/669551759 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc037070 con 0x7f810c15ddf0 2024-04-25T08:41:20.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.483+0000 7f8110a09640 1 -- 172.21.15.38:0/669551759 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80fc032440 con 0x7f810c15ddf0 2024-04-25T08:41:20.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.484+0000 7f8110a09640 1 -- 172.21.15.38:0/669551759 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc032810 con 0x7f810c15ddf0 2024-04-25T08:41:20.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.484+0000 7f8113c6f640 1 -- 172.21.15.38:0/669551759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c15ddf0 msgr2=0x7f810c0b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:20.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.484+0000 7f8113c6f640 1 --2- 172.21.15.38:0/669551759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c15ddf0 0x7f810c0b93f0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f810c0d1bd0 tx=0x7f80fc02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.484+0000 7f8113c6f640 1 -- 172.21.15.38:0/669551759 shutdown_connections 2024-04-25T08:41:20.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.484+0000 7f8113c6f640 1 --2- 172.21.15.38:0/669551759 >> v2:172.21.15.107:3300/0 conn(0x7f810c0b5af0 0x7f810c06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.484+0000 7f8113c6f640 1 --2- 172.21.15.38:0/669551759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c15ddf0 0x7f810c0b93f0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.484+0000 7f8113c6f640 1 --2- 172.21.15.38:0/669551759 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f810c0b5710 0x7f810c161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.484+0000 7f8113c6f640 1 -- 172.21.15.38:0/669551759 >> 172.21.15.38:0/669551759 conn(0x7f810c0d07c0 msgr2=0x7f810c0d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:20.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.484+0000 7f8113c6f640 1 -- 172.21.15.38:0/669551759 shutdown_connections 2024-04-25T08:41:20.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.484+0000 7f8113c6f640 1 -- 172.21.15.38:0/669551759 wait complete. 2024-04-25T08:41:20.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.485+0000 7f8113c6f640 1 Processor -- start 2024-04-25T08:41:20.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.486+0000 7f8113c6f640 1 -- start start 2024-04-25T08:41:20.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.486+0000 7f8113c6f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0b5710 0x7f810c15f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:20.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.486+0000 7f8113c6f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f810c0b5af0 0x7f810c0c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:20.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.486+0000 7f8113c6f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f810c15ddf0 0x7f810c15e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:20.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.486+0000 7f8113c6f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c06c6b0 con 0x7f810c0b5710 2024-04-25T08:41:20.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.486+0000 7f8113c6f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810c06c530 con 0x7f810c0b5af0 2024-04-25T08:41:20.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.486+0000 7f8113c6f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f810c06c830 con 0x7f810c15ddf0 2024-04-25T08:41:20.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.486+0000 7f8111a0b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0b5710 0x7f810c15f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:20.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.486+0000 7f8111a0b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0b5710 0x7f810c15f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49948/0 (socket says 172.21.15.38:49948) 2024-04-25T08:41:20.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.486+0000 7f8111a0b640 1 -- 172.21.15.38:0/587954609 learned_addr learned my addr 172.21.15.38:0/587954609 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.486+0000 7f811220c640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f810c15ddf0 0x7f810c15e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.486+0000 7f811120a640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f810c0b5af0 0x7f810c0c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.487+0000 7f8111a0b640 1 -- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f810c15ddf0 msgr2=0x7f810c15e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.487+0000 7f8111a0b640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f810c15ddf0 0x7f810c15e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.487+0000 7f8111a0b640 1 -- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f810c0b5af0 msgr2=0x7f810c0c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.487+0000 7f8111a0b640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f810c0b5af0 0x7f810c0c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.487+0000 7f8111a0b640 1 -- 172.21.15.38:0/587954609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c159360 con 0x7f810c0b5710 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.487+0000 7f8111a0b640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0b5710 0x7f810c15f900 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f810800d830 tx=0x7f810800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.487+0000 7f81027fc640 1 -- 172.21.15.38:0/587954609 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108018020 con 0x7f810c0b5710 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.487+0000 7f81027fc640 1 -- 172.21.15.38:0/587954609 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81080096a0 con 0x7f810c0b5710 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.487+0000 7f8113c6f640 1 -- 172.21.15.38:0/587954609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f810c15ffa0 con 0x7f810c0b5710 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.487+0000 7f81027fc640 1 -- 172.21.15.38:0/587954609 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108009a70 con 0x7f810c0b5710 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.487+0000 7f8113c6f640 1 -- 172.21.15.38:0/587954609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f810c160560 con 0x7f810c0b5710 2024-04-25T08:41:20.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.488+0000 7f8113c6f640 1 -- 172.21.15.38:0/587954609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f810c068e80 con 0x7f810c0b5710 2024-04-25T08:41:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.489+0000 7f81027fc640 1 -- 172.21.15.38:0/587954609 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f810801b020 con 0x7f810c0b5710 2024-04-25T08:41:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.489+0000 7f81027fc640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f80f006b4d0 0x7f80f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.489+0000 7f81027fc640 1 -- 172.21.15.38:0/587954609 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f810808a4b0 con 0x7f810c0b5710 2024-04-25T08:41:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.489+0000 7f811120a640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f80f006b4d0 0x7f80f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:20.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.490+0000 7f811120a640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f80f006b4d0 0x7f80f006d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f80fc034000 tx=0x7f80fc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:20.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.493+0000 7f81027fc640 1 -- 172.21.15.38:0/587954609 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8108022030 con 0x7f810c0b5710 2024-04-25T08:41:20.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.748+0000 7f8113c6f640 1 -- 172.21.15.38:0/587954609 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f810c0b2d00 con 0x7f810c0b5710 2024-04-25T08:41:20.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.749+0000 7f81027fc640 1 -- 172.21.15.38:0/587954609 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f8108058b50 con 0x7f810c0b5710 2024-04-25T08:41:20.741 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:20.741 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":609,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1005,"data_bytes":1031222793,"bytes_used":2473779200,"bytes_avail":770620334080,"bytes_total":773094113280,"write_bytes_sec":75303789,"read_op_per_sec":0,"write_op_per_sec":74},"fsmap":{"epoch":77,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9489}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T08:40:35.331262+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:41:20.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.750+0000 7f8113c6f640 1 -- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f80f006b4d0 msgr2=0x7f80f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:20.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.751+0000 7f8113c6f640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f80f006b4d0 0x7f80f006d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f80fc034000 tx=0x7f80fc02e000 comp rx=0 tx=0).stop 2024-04-25T08:41:20.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.751+0000 7f8113c6f640 1 -- 172.21.15.38:0/587954609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0b5710 msgr2=0x7f810c15f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:20.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.751+0000 7f8113c6f640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0b5710 0x7f810c15f900 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f810800d830 tx=0x7f810800dcf0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.751+0000 7f8113c6f640 1 -- 172.21.15.38:0/587954609 shutdown_connections 2024-04-25T08:41:20.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.751+0000 7f8113c6f640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f80f006b4d0 0x7f80f006d990 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.751+0000 7f8113c6f640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f810c15ddf0 0x7f810c15e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.751+0000 7f8113c6f640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f810c0b5af0 0x7f810c0c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.751+0000 7f8113c6f640 1 --2- 172.21.15.38:0/587954609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f810c0b5710 0x7f810c15f900 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:20.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.751+0000 7f8113c6f640 1 -- 172.21.15.38:0/587954609 >> 172.21.15.38:0/587954609 conn(0x7f810c0d07c0 msgr2=0x7f810c0d2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:20.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.751+0000 7f8113c6f640 1 -- 172.21.15.38:0/587954609 shutdown_connections 2024-04-25T08:41:20.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:20.751+0000 7f8113c6f640 1 -- 172.21.15.38:0/587954609 wait complete. 2024-04-25T08:41:20.754 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:41:21.755 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:21.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.875+0000 7fb2e90a1640 1 Processor -- start 2024-04-25T08:41:21.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.875+0000 7fb2e90a1640 1 -- start start 2024-04-25T08:41:21.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.876+0000 7fb2e90a1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb2e40d3900 0x7fb2e40d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.876+0000 7fb2e90a1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e40d0d90 0x7fb2e40b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.876+0000 7fb2e90a1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e40d3e70 con 0x7fb2e415dc10 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.876+0000 7fb2e90a1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e415dff0 con 0x7fb2e40d3900 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.876+0000 7fb2e90a1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40b6990 con 0x7fb2e40d0d90 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.876+0000 7fb2e2d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb2e415dc10 0x7fb2e40ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44842/0 (socket says 172.21.15.38:44842) 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.876+0000 7fb2e2d9d640 1 -- 172.21.15.38:0/2615380565 learned_addr learned my addr 172.21.15.38:0/2615380565 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.876+0000 7fb2dbfff640 1 --2- 172.21.15.38:0/2615380565 >> v2:172.21.15.107:3300/0 conn(0x7fb2e40d3900 0x7fb2e40d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.876+0000 7fb2e359e640 1 --2- 172.21.15.38:0/2615380565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e40d0d90 0x7fb2e40b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.876+0000 7fb2e259c640 1 -- 172.21.15.38:0/2615380565 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054776044 0 0) 0x7fb2e40d3e70 con 0x7fb2e415dc10 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.876+0000 7fb2e259c640 1 -- 172.21.15.38:0/2615380565 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d0003520 con 0x7fb2e415dc10 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.877+0000 7fb2e259c640 1 -- 172.21.15.38:0/2615380565 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4180336347 0 0) 0x7fb2d0003520 con 0x7fb2e415dc10 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.877+0000 7fb2dbfff640 1 -- 172.21.15.38:0/2615380565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e40d0d90 msgr2=0x7fb2e40b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:21.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.877+0000 7fb2dbfff640 1 --2- 172.21.15.38:0/2615380565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e40d0d90 0x7fb2e40b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.877+0000 7fb2dbfff640 1 -- 172.21.15.38:0/2615380565 >> v1:172.21.15.38:6789/0 conn(0x7fb2e415dc10 legacy=0x7fb2e40ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.877+0000 7fb2dbfff640 1 -- 172.21.15.38:0/2615380565 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e40d4150 con 0x7fb2e40d3900 2024-04-25T08:41:21.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.877+0000 7fb2e359e640 1 --2- 172.21.15.38:0/2615380565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e40d0d90 0x7fb2e40b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:41:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.878+0000 7fb2dbfff640 1 --2- 172.21.15.38:0/2615380565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e40d3900 0x7fb2e40d0850 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0097a0 tx=0x7fb2cc02a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=311301b769601b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.878+0000 7fb2e259c640 1 -- 172.21.15.38:0/2615380565 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc037070 con 0x7fb2e40d3900 2024-04-25T08:41:21.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.878+0000 7fb2e259c640 1 -- 172.21.15.38:0/2615380565 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2cc02ab10 con 0x7fb2e40d3900 2024-04-25T08:41:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.878+0000 7fb2e259c640 1 -- 172.21.15.38:0/2615380565 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4001670 con 0x7fb2e40d3900 2024-04-25T08:41:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.878+0000 7fb2e90a1640 1 -- 172.21.15.38:0/2615380565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e40d3900 msgr2=0x7fb2e40d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.878+0000 7fb2e90a1640 1 --2- 172.21.15.38:0/2615380565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e40d3900 0x7fb2e40d0850 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc0097a0 tx=0x7fb2cc02a080 comp rx=0 tx=0).stop 2024-04-25T08:41:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.878+0000 7fb2e90a1640 1 -- 172.21.15.38:0/2615380565 shutdown_connections 2024-04-25T08:41:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.878+0000 7fb2e90a1640 1 --2- 172.21.15.38:0/2615380565 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e40d0d90 0x7fb2e40b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.878+0000 7fb2e90a1640 1 --2- 172.21.15.38:0/2615380565 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e40d3900 0x7fb2e40d0850 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.878+0000 7fb2e90a1640 1 -- 172.21.15.38:0/2615380565 >> 172.21.15.38:0/2615380565 conn(0x7fb2e4066d10 msgr2=0x7fb2e4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.878+0000 7fb2e90a1640 1 -- 172.21.15.38:0/2615380565 shutdown_connections 2024-04-25T08:41:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.878+0000 7fb2e90a1640 1 -- 172.21.15.38:0/2615380565 wait complete. 2024-04-25T08:41:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.880+0000 7fb2e90a1640 1 Processor -- start 2024-04-25T08:41:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.880+0000 7fb2e90a1640 1 -- start start 2024-04-25T08:41:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e90a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e40d0d90 0x7fb2e40b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e90a1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e40d3900 0x7fb2e40cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e90a1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e415dc10 0x7fb2e4150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e90a1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40b6bc0 con 0x7fb2e40d0d90 2024-04-25T08:41:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e90a1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40d4710 con 0x7fb2e415dc10 2024-04-25T08:41:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e90a1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40cef30 con 0x7fb2e40d3900 2024-04-25T08:41:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e40d0d90 0x7fb2e40b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e40d0d90 0x7fb2e40b7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49958/0 (socket says 172.21.15.38:49958) 2024-04-25T08:41:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e2d9d640 1 -- 172.21.15.38:0/4082320321 learned_addr learned my addr 172.21.15.38:0/4082320321 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:21.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2dbfff640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e40d3900 0x7fb2e40cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e359e640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e415dc10 0x7fb2e4150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e2d9d640 1 -- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e40d3900 msgr2=0x7fb2e40cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e2d9d640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e40d3900 0x7fb2e40cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e2d9d640 1 -- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e415dc10 msgr2=0x7fb2e4150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e2d9d640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e415dc10 0x7fb2e4150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e2d9d640 1 -- 172.21.15.38:0/4082320321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e40cac20 con 0x7fb2e40d0d90 2024-04-25T08:41:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e2d9d640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e40d0d90 0x7fb2e40b7e40 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c780 tx=0x7fb2d400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2db7fe640 1 -- 172.21.15.38:0/4082320321 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4017070 con 0x7fb2e40d0d90 2024-04-25T08:41:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2db7fe640 1 -- 172.21.15.38:0/4082320321 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d4005ad0 con 0x7fb2e40d0d90 2024-04-25T08:41:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2db7fe640 1 -- 172.21.15.38:0/4082320321 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4007070 con 0x7fb2e40d0d90 2024-04-25T08:41:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.881+0000 7fb2e90a1640 1 -- 172.21.15.38:0/4082320321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e40caf00 con 0x7fb2e40d0d90 2024-04-25T08:41:21.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.882+0000 7fb2e90a1640 1 -- 172.21.15.38:0/4082320321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e40cb4c0 con 0x7fb2e40d0d90 2024-04-25T08:41:21.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.882+0000 7fb2e90a1640 1 -- 172.21.15.38:0/4082320321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e4068e80 con 0x7fb2e40d0d90 2024-04-25T08:41:21.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.883+0000 7fb2db7fe640 1 -- 172.21.15.38:0/4082320321 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2d401a070 con 0x7fb2e40d0d90 2024-04-25T08:41:21.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.883+0000 7fb2db7fe640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb2c406b4d0 0x7fb2c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:21.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.883+0000 7fb2db7fe640 1 -- 172.21.15.38:0/4082320321 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb2d40896e0 con 0x7fb2e40d0d90 2024-04-25T08:41:21.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.884+0000 7fb2dbfff640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb2c406b4d0 0x7fb2c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:21.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.884+0000 7fb2dbfff640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb2c406b4d0 0x7fb2c406d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc034000 tx=0x7fb2cc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:21.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:21.888+0000 7fb2db7fe640 1 -- 172.21.15.38:0/4082320321 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2d4057e10 con 0x7fb2e40d0d90 2024-04-25T08:41:22.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.103+0000 7fb2e90a1640 1 -- 172.21.15.38:0/4082320321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2e40b4a50 con 0x7fb2e40d0d90 2024-04-25T08:41:22.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.104+0000 7fb2db7fe640 1 -- 172.21.15.38:0/4082320321 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb2d405beb0 con 0x7fb2e40d0d90 2024-04-25T08:41:22.098 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:22.098 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:40:55.730538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/1312063561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1312063561},{"type":"v1","addr":"172.21.15.38:6835","nonce":1312063561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:22.098 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 77 2024-04-25T08:41:22.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.105+0000 7fb2e90a1640 1 -- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb2c406b4d0 msgr2=0x7fb2c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:22.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.105+0000 7fb2e90a1640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb2c406b4d0 0x7fb2c406d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc034000 tx=0x7fb2cc02e000 comp rx=0 tx=0).stop 2024-04-25T08:41:22.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.105+0000 7fb2e90a1640 1 -- 172.21.15.38:0/4082320321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e40d0d90 msgr2=0x7fb2e40b7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.105+0000 7fb2e90a1640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e40d0d90 0x7fb2e40b7e40 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c780 tx=0x7fb2d400cb80 comp rx=0 tx=0).stop 2024-04-25T08:41:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.106+0000 7fb2e90a1640 1 -- 172.21.15.38:0/4082320321 shutdown_connections 2024-04-25T08:41:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.106+0000 7fb2e90a1640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2e415dc10 0x7fb2e4150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.106+0000 7fb2e90a1640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb2c406b4d0 0x7fb2c406d990 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.106+0000 7fb2e90a1640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb2e40d3900 0x7fb2e40cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.106+0000 7fb2e90a1640 1 --2- 172.21.15.38:0/4082320321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2e40d0d90 0x7fb2e40b7e40 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.106+0000 7fb2e90a1640 1 -- 172.21.15.38:0/4082320321 >> 172.21.15.38:0/4082320321 conn(0x7fb2e4066d10 msgr2=0x7fb2e415fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.106+0000 7fb2e90a1640 1 -- 172.21.15.38:0/4082320321 shutdown_connections 2024-04-25T08:41:22.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.106+0000 7fb2e90a1640 1 -- 172.21.15.38:0/4082320321 wait complete. 2024-04-25T08:41:22.110 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T08:41:22.110 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:41:22.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.217+0000 7f203275c640 1 Processor -- start 2024-04-25T08:41:22.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.217+0000 7f203275c640 1 -- start start 2024-04-25T08:41:22.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.217+0000 7f203275c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f202c15df50 0x7f202c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:22.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.217+0000 7f203275c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c0ceff0 con 0x7f202c06c110 2024-04-25T08:41:22.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.217+0000 7f203275c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c0d1f70 con 0x7f202c0ce2e0 2024-04-25T08:41:22.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.217+0000 7f203275c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202c0cf2d0 con 0x7f202c15df50 2024-04-25T08:41:22.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.217+0000 7f202b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f202c15df50 0x7f202c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:22.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.218+0000 7f202bfff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f202c06c110 0x7f202c06c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44852/0 (socket says 172.21.15.38:44852) 2024-04-25T08:41:22.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.218+0000 7f202bfff640 1 -- 172.21.15.38:0/4246820122 learned_addr learned my addr 172.21.15.38:0/4246820122 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:22.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.218+0000 7f202b7fe640 1 -- 172.21.15.38:0/4246820122 >> v1:172.21.15.107:6789/0 conn(0x7f202c0ce2e0 legacy=0x7f202c0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:22.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.218+0000 7f202b7fe640 1 -- 172.21.15.38:0/4246820122 >> v1:172.21.15.38:6789/0 conn(0x7f202c06c110 legacy=0x7f202c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:22.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.218+0000 7f202b7fe640 1 -- 172.21.15.38:0/4246820122 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c0d2250 con 0x7f202c15df50 2024-04-25T08:41:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.219+0000 7f202b7fe640 1 --2- 172.21.15.38:0/4246820122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f202c15df50 0x7f202c0ce7a0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f201c0097a0 tx=0x7f201c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b348e21c217acb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.219+0000 7f202affd640 1 -- 172.21.15.38:0/4246820122 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c037070 con 0x7f202c15df50 2024-04-25T08:41:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.219+0000 7f202affd640 1 -- 172.21.15.38:0/4246820122 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c032440 con 0x7f202c15df50 2024-04-25T08:41:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.219+0000 7f202affd640 1 -- 172.21.15.38:0/4246820122 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c032810 con 0x7f202c15df50 2024-04-25T08:41:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.219+0000 7f203275c640 1 -- 172.21.15.38:0/4246820122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f202c15df50 msgr2=0x7f202c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.219+0000 7f203275c640 1 --2- 172.21.15.38:0/4246820122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f202c15df50 0x7f202c0ce7a0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f201c0097a0 tx=0x7f201c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.219+0000 7f203275c640 1 -- 172.21.15.38:0/4246820122 shutdown_connections 2024-04-25T08:41:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.219+0000 7f203275c640 1 --2- 172.21.15.38:0/4246820122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f202c15df50 0x7f202c0ce7a0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.219+0000 7f203275c640 1 -- 172.21.15.38:0/4246820122 >> 172.21.15.38:0/4246820122 conn(0x7f202c0d3fb0 msgr2=0x7f202c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.219+0000 7f203275c640 1 -- 172.21.15.38:0/4246820122 shutdown_connections 2024-04-25T08:41:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.219+0000 7f203275c640 1 -- 172.21.15.38:0/4246820122 wait complete. 2024-04-25T08:41:22.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.221+0000 7f203275c640 1 Processor -- start 2024-04-25T08:41:22.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.221+0000 7f203275c640 1 -- start start 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f203275c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f202c06c110 0x7f202c0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f203275c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f202c0ce2e0 0x7f202c0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f203275c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f202c15df50 0x7f202c159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f203275c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c0d3160 con 0x7f202c06c110 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f203275c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c0cece0 con 0x7f202c0ce2e0 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f203275c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c0d32e0 con 0x7f202c15df50 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f202bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f202c06c110 0x7f202c0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f202bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f202c06c110 0x7f202c0b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49966/0 (socket says 172.21.15.38:49966) 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f202bfff640 1 -- 172.21.15.38:0/2168283761 learned_addr learned my addr 172.21.15.38:0/2168283761 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f202b7fe640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f202c0ce2e0 0x7f202c0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f2030cf9640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f202c15df50 0x7f202c159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f202bfff640 1 -- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f202c15df50 msgr2=0x7f202c159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f202bfff640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f202c15df50 0x7f202c159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f202bfff640 1 -- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f202c0ce2e0 msgr2=0x7f202c0c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:22.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f202bfff640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f202c0ce2e0 0x7f202c0c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f202bfff640 1 -- 172.21.15.38:0/2168283761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c0cd1e0 con 0x7f202c06c110 2024-04-25T08:41:22.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f2030cf9640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f202c15df50 0x7f202c159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:22.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f202bfff640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f202c06c110 0x7f202c0b8050 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f202000c400 tx=0x7f202000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:22.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f2028ff9640 1 -- 172.21.15.38:0/2168283761 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2020017070 con 0x7f202c06c110 2024-04-25T08:41:22.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f2028ff9640 1 -- 172.21.15.38:0/2168283761 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2020005ce0 con 0x7f202c06c110 2024-04-25T08:41:22.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f2028ff9640 1 -- 172.21.15.38:0/2168283761 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200072f0 con 0x7f202c06c110 2024-04-25T08:41:22.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f203275c640 1 -- 172.21.15.38:0/2168283761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c0cc140 con 0x7f202c06c110 2024-04-25T08:41:22.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.222+0000 7f203275c640 1 -- 172.21.15.38:0/2168283761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c0cc9e0 con 0x7f202c06c110 2024-04-25T08:41:22.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.223+0000 7f203275c640 1 -- 172.21.15.38:0/2168283761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c0cc530 con 0x7f202c06c110 2024-04-25T08:41:22.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.224+0000 7f2028ff9640 1 -- 172.21.15.38:0/2168283761 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f202001a070 con 0x7f202c06c110 2024-04-25T08:41:22.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.224+0000 7f2028ff9640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f201806b4d0 0x7f201806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:22.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.224+0000 7f2028ff9640 1 -- 172.21.15.38:0/2168283761 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20200897d0 con 0x7f202c06c110 2024-04-25T08:41:22.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.224+0000 7f202b7fe640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f201806b4d0 0x7f201806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:22.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.225+0000 7f202b7fe640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f201806b4d0 0x7f201806d990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f201c034000 tx=0x7f201c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:22.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.229+0000 7f2028ff9640 1 -- 172.21.15.38:0/2168283761 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20200584c0 con 0x7f202c06c110 2024-04-25T08:41:22.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.483+0000 7f203275c640 1 -- 172.21.15.38:0/2168283761 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f202c0cce20 con 0x7f202c06c110 2024-04-25T08:41:22.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.484+0000 7f2028ff9640 1 -- 172.21.15.38:0/2168283761 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f2020057dc0 con 0x7f202c06c110 2024-04-25T08:41:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:22.478 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":611,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1005,"data_bytes":1031222793,"bytes_used":2473779200,"bytes_avail":770620334080,"bytes_total":773094113280,"write_bytes_sec":74255236,"read_op_per_sec":0,"write_op_per_sec":73},"fsmap":{"epoch":77,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9489}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T08:40:35.331262+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:41:22.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.486+0000 7f203275c640 1 -- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f201806b4d0 msgr2=0x7f201806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:22.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.486+0000 7f203275c640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f201806b4d0 0x7f201806d990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f201c034000 tx=0x7f201c02e000 comp rx=0 tx=0).stop 2024-04-25T08:41:22.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.486+0000 7f203275c640 1 -- 172.21.15.38:0/2168283761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f202c06c110 msgr2=0x7f202c0b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:22.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.486+0000 7f203275c640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f202c06c110 0x7f202c0b8050 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f202000c400 tx=0x7f202000c8c0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.486+0000 7f203275c640 1 -- 172.21.15.38:0/2168283761 shutdown_connections 2024-04-25T08:41:22.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.486+0000 7f203275c640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f201806b4d0 0x7f201806d990 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.486+0000 7f203275c640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f202c15df50 0x7f202c159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.486+0000 7f203275c640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f202c0ce2e0 0x7f202c0c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.486+0000 7f203275c640 1 --2- 172.21.15.38:0/2168283761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f202c06c110 0x7f202c0b8050 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:22.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.486+0000 7f203275c640 1 -- 172.21.15.38:0/2168283761 >> 172.21.15.38:0/2168283761 conn(0x7f202c0d3fb0 msgr2=0x7f202c1e76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:22.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.486+0000 7f203275c640 1 -- 172.21.15.38:0/2168283761 shutdown_connections 2024-04-25T08:41:22.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:22.486+0000 7f203275c640 1 -- 172.21.15.38:0/2168283761 wait complete. 2024-04-25T08:41:22.490 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:41:23.491 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:23.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.612+0000 7fe31d5e0640 1 Processor -- start 2024-04-25T08:41:23.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.612+0000 7fe31d5e0640 1 -- start start 2024-04-25T08:41:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe31d5e0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe3180d3900 0x7fe3180d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe31d5e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3180d0d90 0x7fe3180b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe31d5e0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3180d3e70 con 0x7fe31815dc10 2024-04-25T08:41:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe31d5e0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe31815dff0 con 0x7fe3180d3900 2024-04-25T08:41:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe31d5e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180b6990 con 0x7fe3180d0d90 2024-04-25T08:41:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe30ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe3180d3900 0x7fe3180d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe30ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe3180d3900 0x7fe3180d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49978/0 (socket says 172.21.15.38:49978) 2024-04-25T08:41:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe30ffff640 1 -- 172.21.15.38:0/532660729 learned_addr learned my addr 172.21.15.38:0/532660729 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe3177fe640 1 --2- 172.21.15.38:0/532660729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3180d0d90 0x7fe3180b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:23.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe30ffff640 1 -- 172.21.15.38:0/532660729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3180d0d90 msgr2=0x7fe3180b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe30ffff640 1 --2- 172.21.15.38:0/532660729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3180d0d90 0x7fe3180b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe30ffff640 1 -- 172.21.15.38:0/532660729 >> v1:172.21.15.107:6789/0 conn(0x7fe31815dc10 legacy=0x7fe3180ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.613+0000 7fe30ffff640 1 -- 172.21.15.38:0/532660729 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3180d4150 con 0x7fe3180d3900 2024-04-25T08:41:23.608 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.614+0000 7fe30ffff640 1 --2- 172.21.15.38:0/532660729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3180d3900 0x7fe3180d0850 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fe300009870 tx=0x7fe30002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dec5e2a5a84c132 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:23.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.616+0000 7fe3167fc640 1 -- 172.21.15.38:0/532660729 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300037070 con 0x7fe3180d3900 2024-04-25T08:41:23.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.616+0000 7fe3167fc640 1 -- 172.21.15.38:0/532660729 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe300032440 con 0x7fe3180d3900 2024-04-25T08:41:23.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.616+0000 7fe3167fc640 1 -- 172.21.15.38:0/532660729 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300032810 con 0x7fe3180d3900 2024-04-25T08:41:23.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.616+0000 7fe31d5e0640 1 -- 172.21.15.38:0/532660729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3180d3900 msgr2=0x7fe3180d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:23.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.616+0000 7fe31d5e0640 1 --2- 172.21.15.38:0/532660729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3180d3900 0x7fe3180d0850 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fe300009870 tx=0x7fe30002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.616+0000 7fe31d5e0640 1 -- 172.21.15.38:0/532660729 shutdown_connections 2024-04-25T08:41:23.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.616+0000 7fe31d5e0640 1 --2- 172.21.15.38:0/532660729 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3180d0d90 0x7fe3180b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.616+0000 7fe31d5e0640 1 --2- 172.21.15.38:0/532660729 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3180d3900 0x7fe3180d0850 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.616+0000 7fe31d5e0640 1 -- 172.21.15.38:0/532660729 >> 172.21.15.38:0/532660729 conn(0x7fe318066d10 msgr2=0x7fe318067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:23.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.616+0000 7fe31d5e0640 1 -- 172.21.15.38:0/532660729 shutdown_connections 2024-04-25T08:41:23.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.616+0000 7fe31d5e0640 1 -- 172.21.15.38:0/532660729 wait complete. 2024-04-25T08:41:23.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.618+0000 7fe31d5e0640 1 Processor -- start 2024-04-25T08:41:23.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.618+0000 7fe31d5e0640 1 -- start start 2024-04-25T08:41:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.618+0000 7fe31d5e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3180d0d90 0x7fe3180ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:23.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.618+0000 7fe31d5e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3180d3900 0x7fe3181f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.618+0000 7fe31d5e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe31815dc10 0x7fe31815bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.618+0000 7fe31d5e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180b6bc0 con 0x7fe3180d3900 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.618+0000 7fe31d5e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180d4710 con 0x7fe31815dc10 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.618+0000 7fe31d5e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3180cef30 con 0x7fe3180d0d90 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.618+0000 7fe30ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3180d3900 0x7fe3181f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe30ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3180d3900 0x7fe3181f3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49982/0 (socket says 172.21.15.38:49982) 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe30ffff640 1 -- 172.21.15.38:0/2530190508 learned_addr learned my addr 172.21.15.38:0/2530190508 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe3177fe640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe31815dc10 0x7fe31815bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe316ffd640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3180d0d90 0x7fe3180ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe30ffff640 1 -- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3180d0d90 msgr2=0x7fe3180ab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe30ffff640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3180d0d90 0x7fe3180ab990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe30ffff640 1 -- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe31815dc10 msgr2=0x7fe31815bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:23.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe30ffff640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe31815dc10 0x7fe31815bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe30ffff640 1 -- 172.21.15.38:0/2530190508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3181f3c50 con 0x7fe3180d3900 2024-04-25T08:41:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe30ffff640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3180d3900 0x7fe3181f3380 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fe300000c00 tx=0x7fe3000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe30f7fe640 1 -- 172.21.15.38:0/2530190508 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300037040 con 0x7fe3180d3900 2024-04-25T08:41:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe30f7fe640 1 -- 172.21.15.38:0/2530190508 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe300042070 con 0x7fe3180d3900 2024-04-25T08:41:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe31d5e0640 1 -- 172.21.15.38:0/2530190508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3181f3f30 con 0x7fe3180d3900 2024-04-25T08:41:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe30f7fe640 1 -- 172.21.15.38:0/2530190508 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe300030880 con 0x7fe3180d3900 2024-04-25T08:41:23.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.619+0000 7fe31d5e0640 1 -- 172.21.15.38:0/2530190508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3181f38c0 con 0x7fe3180d3900 2024-04-25T08:41:23.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.621+0000 7fe31d5e0640 1 -- 172.21.15.38:0/2530190508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe318068e80 con 0x7fe3180d3900 2024-04-25T08:41:23.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.621+0000 7fe30f7fe640 1 -- 172.21.15.38:0/2530190508 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe30002ad10 con 0x7fe3180d3900 2024-04-25T08:41:23.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.621+0000 7fe30f7fe640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe2f806b4d0 0x7fe2f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:23.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.621+0000 7fe30f7fe640 1 -- 172.21.15.38:0/2530190508 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe3000aa740 con 0x7fe3180d3900 2024-04-25T08:41:23.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.621+0000 7fe316ffd640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe2f806b4d0 0x7fe2f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:23.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.622+0000 7fe316ffd640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe2f806b4d0 0x7fe2f806d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe3180d2f70 tx=0x7fe308009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:23.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.625+0000 7fe30f7fe640 1 -- 172.21.15.38:0/2530190508 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe300078f90 con 0x7fe3180d3900 2024-04-25T08:41:23.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.842+0000 7fe31d5e0640 1 -- 172.21.15.38:0/2530190508 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe31815e210 con 0x7fe3180d3900 2024-04-25T08:41:23.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.842+0000 7fe30f7fe640 1 -- 172.21.15.38:0/2530190508 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe30007d030 con 0x7fe3180d3900 2024-04-25T08:41:23.836 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:23.836 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:40:55.730538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/1312063561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1312063561},{"type":"v1","addr":"172.21.15.38:6835","nonce":1312063561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:23.837 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 77 2024-04-25T08:41:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.845+0000 7fe31d5e0640 1 -- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe2f806b4d0 msgr2=0x7fe2f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.845+0000 7fe31d5e0640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe2f806b4d0 0x7fe2f806d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe3180d2f70 tx=0x7fe308009040 comp rx=0 tx=0).stop 2024-04-25T08:41:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.845+0000 7fe31d5e0640 1 -- 172.21.15.38:0/2530190508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3180d3900 msgr2=0x7fe3181f3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.845+0000 7fe31d5e0640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3180d3900 0x7fe3181f3380 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fe300000c00 tx=0x7fe3000046c0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.845+0000 7fe31d5e0640 1 -- 172.21.15.38:0/2530190508 shutdown_connections 2024-04-25T08:41:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.845+0000 7fe31d5e0640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe31815dc10 0x7fe31815bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.845+0000 7fe31d5e0640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe3180d3900 0x7fe3181f3380 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.845+0000 7fe31d5e0640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe2f806b4d0 0x7fe2f806d990 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.845+0000 7fe31d5e0640 1 --2- 172.21.15.38:0/2530190508 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe3180d0d90 0x7fe3180ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.845+0000 7fe31d5e0640 1 -- 172.21.15.38:0/2530190508 >> 172.21.15.38:0/2530190508 conn(0x7fe318066d10 msgr2=0x7fe3180d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.845+0000 7fe31d5e0640 1 -- 172.21.15.38:0/2530190508 shutdown_connections 2024-04-25T08:41:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.845+0000 7fe31d5e0640 1 -- 172.21.15.38:0/2530190508 wait complete. 2024-04-25T08:41:23.849 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T08:41:23.849 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:41:23.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.955+0000 7fb8773e2640 1 Processor -- start 2024-04-25T08:41:23.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.955+0000 7fb8773e2640 1 -- start start 2024-04-25T08:41:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.956+0000 7fb8773e2640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb87015dc10 0x7fb8700ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.956+0000 7fb8773e2640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb8700d0f60 0x7fb8700b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.956+0000 7fb8773e2640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8700d3e70 con 0x7fb8700ced60 2024-04-25T08:41:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.956+0000 7fb8773e2640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87015dff0 con 0x7fb87015dc10 2024-04-25T08:41:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.956+0000 7fb8773e2640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8700b6990 con 0x7fb8700d0f60 2024-04-25T08:41:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.956+0000 7fb87517e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb87015dc10 0x7fb8700ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.956+0000 7fb87497d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb8700ced60 0x7fb8700d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44854/0 (socket says 172.21.15.38:44854) 2024-04-25T08:41:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.956+0000 7fb87497d640 1 -- 172.21.15.38:0/3615149531 learned_addr learned my addr 172.21.15.38:0/3615149531 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.956+0000 7fb87517e640 1 --2- 172.21.15.38:0/3615149531 >> v2:172.21.15.38:3300/0 conn(0x7fb87015dc10 0x7fb8700ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49994/0 (socket says 172.21.15.38:49994) 2024-04-25T08:41:23.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.956+0000 7fb87597f640 1 --2- 172.21.15.38:0/3615149531 >> v2:172.21.15.107:3300/0 conn(0x7fb8700d0f60 0x7fb8700b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.956+0000 7fb87517e640 1 -- 172.21.15.38:0/3615149531 >> v2:172.21.15.107:3300/0 conn(0x7fb8700d0f60 msgr2=0x7fb8700b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.956+0000 7fb87517e640 1 --2- 172.21.15.38:0/3615149531 >> v2:172.21.15.107:3300/0 conn(0x7fb8700d0f60 0x7fb8700b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.957+0000 7fb87517e640 1 -- 172.21.15.38:0/3615149531 >> v1:172.21.15.38:6789/0 conn(0x7fb8700ced60 legacy=0x7fb8700d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.957+0000 7fb87517e640 1 -- 172.21.15.38:0/3615149531 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8700d4150 con 0x7fb87015dc10 2024-04-25T08:41:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.957+0000 7fb87517e640 1 --2- 172.21.15.38:0/3615149531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb87015dc10 0x7fb8700ce820 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fb8700cdee0 tx=0x7fb86c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653948371284b4f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.957+0000 7fb867fff640 1 -- 172.21.15.38:0/3615149531 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c037070 con 0x7fb87015dc10 2024-04-25T08:41:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.957+0000 7fb867fff640 1 -- 172.21.15.38:0/3615149531 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c032440 con 0x7fb87015dc10 2024-04-25T08:41:23.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.957+0000 7fb867fff640 1 -- 172.21.15.38:0/3615149531 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c02ae60 con 0x7fb87015dc10 2024-04-25T08:41:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.958+0000 7fb8773e2640 1 -- 172.21.15.38:0/3615149531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb87015dc10 msgr2=0x7fb8700ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.958+0000 7fb8773e2640 1 --2- 172.21.15.38:0/3615149531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb87015dc10 0x7fb8700ce820 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fb8700cdee0 tx=0x7fb86c02a110 comp rx=0 tx=0).stop 2024-04-25T08:41:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.958+0000 7fb8773e2640 1 -- 172.21.15.38:0/3615149531 shutdown_connections 2024-04-25T08:41:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.958+0000 7fb8773e2640 1 --2- 172.21.15.38:0/3615149531 >> v2:172.21.15.107:3300/0 conn(0x7fb8700d0f60 0x7fb8700b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.958+0000 7fb8773e2640 1 --2- 172.21.15.38:0/3615149531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb87015dc10 0x7fb8700ce820 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.958+0000 7fb8773e2640 1 -- 172.21.15.38:0/3615149531 >> 172.21.15.38:0/3615149531 conn(0x7fb870066d10 msgr2=0x7fb870067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.958+0000 7fb8773e2640 1 -- 172.21.15.38:0/3615149531 shutdown_connections 2024-04-25T08:41:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.958+0000 7fb8773e2640 1 -- 172.21.15.38:0/3615149531 wait complete. 2024-04-25T08:41:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.959+0000 7fb8773e2640 1 Processor -- start 2024-04-25T08:41:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.959+0000 7fb8773e2640 1 -- start start 2024-04-25T08:41:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb8773e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8700ced60 0x7fb870152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb8773e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8700d0f60 0x7fb8700aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb8773e2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb87015dc10 0x7fb8700ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb8773e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8700d3900 con 0x7fb8700d0f60 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb8773e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8700b6bc0 con 0x7fb8700ced60 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb8773e2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8700d3a80 con 0x7fb87015dc10 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb87497d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8700d0f60 0x7fb8700aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb87497d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8700d0f60 0x7fb8700aa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50008/0 (socket says 172.21.15.38:50008) 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb87497d640 1 -- 172.21.15.38:0/235440853 learned_addr learned my addr 172.21.15.38:0/235440853 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb87597f640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb87015dc10 0x7fb8700ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb87497d640 1 -- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb87015dc10 msgr2=0x7fb8700ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb87497d640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb87015dc10 0x7fb8700ac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb87497d640 1 -- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8700ced60 msgr2=0x7fb870152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb87517e640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8700ced60 0x7fb870152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:23.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb87497d640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8700ced60 0x7fb870152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:23.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb87497d640 1 -- 172.21.15.38:0/235440853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8700aadb0 con 0x7fb8700d0f60 2024-04-25T08:41:23.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.960+0000 7fb87497d640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8700d0f60 0x7fb8700aa2f0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fb86000c480 tx=0x7fb86000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:23.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.961+0000 7fb865ffb640 1 -- 172.21.15.38:0/235440853 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860017070 con 0x7fb8700d0f60 2024-04-25T08:41:23.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.961+0000 7fb865ffb640 1 -- 172.21.15.38:0/235440853 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb860005ce0 con 0x7fb8700d0f60 2024-04-25T08:41:23.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.961+0000 7fb8773e2640 1 -- 172.21.15.38:0/235440853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8700ab090 con 0x7fb8700d0f60 2024-04-25T08:41:23.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.961+0000 7fb865ffb640 1 -- 172.21.15.38:0/235440853 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8600072f0 con 0x7fb8700d0f60 2024-04-25T08:41:23.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.961+0000 7fb8773e2640 1 -- 172.21.15.38:0/235440853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8700ab650 con 0x7fb8700d0f60 2024-04-25T08:41:23.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.962+0000 7fb8773e2640 1 -- 172.21.15.38:0/235440853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb870068e80 con 0x7fb8700d0f60 2024-04-25T08:41:23.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.962+0000 7fb865ffb640 1 -- 172.21.15.38:0/235440853 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb86001a070 con 0x7fb8700d0f60 2024-04-25T08:41:23.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.963+0000 7fb865ffb640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb85406b4d0 0x7fb85406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:23.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.963+0000 7fb865ffb640 1 -- 172.21.15.38:0/235440853 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb860089590 con 0x7fb8700d0f60 2024-04-25T08:41:23.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.963+0000 7fb87517e640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb85406b4d0 0x7fb85406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:23.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.963+0000 7fb87517e640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb85406b4d0 0x7fb85406d990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb86c034000 tx=0x7fb86c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:23.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:23.967+0000 7fb865ffb640 1 -- 172.21.15.38:0/235440853 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb860057cc0 con 0x7fb8700d0f60 2024-04-25T08:41:24.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.216+0000 7fb8773e2640 1 -- 172.21.15.38:0/235440853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb870160a00 con 0x7fb8700d0f60 2024-04-25T08:41:24.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.216+0000 7fb865ffb640 1 -- 172.21.15.38:0/235440853 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fb86005bd60 con 0x7fb8700d0f60 2024-04-25T08:41:24.210 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:24.210 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":613,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077727859,"bytes_used":2671386624,"bytes_avail":770422726656,"bytes_total":773094113280,"write_bytes_sec":78132590,"read_op_per_sec":0,"write_op_per_sec":78},"fsmap":{"epoch":77,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9489}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T08:41:23.343181+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:41:24.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.218+0000 7fb8773e2640 1 -- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb85406b4d0 msgr2=0x7fb85406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:24.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.218+0000 7fb8773e2640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb85406b4d0 0x7fb85406d990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb86c034000 tx=0x7fb86c02e000 comp rx=0 tx=0).stop 2024-04-25T08:41:24.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.218+0000 7fb8773e2640 1 -- 172.21.15.38:0/235440853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8700d0f60 msgr2=0x7fb8700aa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:24.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.218+0000 7fb8773e2640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8700d0f60 0x7fb8700aa2f0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fb86000c480 tx=0x7fb86000c940 comp rx=0 tx=0).stop 2024-04-25T08:41:24.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.218+0000 7fb8773e2640 1 -- 172.21.15.38:0/235440853 shutdown_connections 2024-04-25T08:41:24.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.218+0000 7fb8773e2640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb85406b4d0 0x7fb85406d990 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.218+0000 7fb8773e2640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb87015dc10 0x7fb8700ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.218+0000 7fb8773e2640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb8700d0f60 0x7fb8700aa2f0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.218+0000 7fb8773e2640 1 --2- 172.21.15.38:0/235440853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb8700ced60 0x7fb870152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.218+0000 7fb8773e2640 1 -- 172.21.15.38:0/235440853 >> 172.21.15.38:0/235440853 conn(0x7fb870066d10 msgr2=0x7fb8700d2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:24.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.218+0000 7fb8773e2640 1 -- 172.21.15.38:0/235440853 shutdown_connections 2024-04-25T08:41:24.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.218+0000 7fb8773e2640 1 -- 172.21.15.38:0/235440853 wait complete. 2024-04-25T08:41:24.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T08:41:24.223 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:41:24.223 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:41:24.233 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:41:24.233 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'d', 'c', 'b'} 2024-04-25T08:41:24.234 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T08:41:24.381 INFO:teuthology.orchestra.run.smithi038.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T08:41:24.381 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T08:41:24.381 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:24.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.489+0000 7f60e6928640 1 Processor -- start 2024-04-25T08:41:24.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.489+0000 7f60e6928640 1 -- start start 2024-04-25T08:41:24.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60e6928640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f60e00b2d60 0x7f60e006c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:24.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60e6928640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e006abe0 0x7f60e00b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:24.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60e6928640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e00b3680 con 0x7f60e015e5c0 2024-04-25T08:41:24.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60e6928640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e006b0a0 con 0x7f60e00b2d60 2024-04-25T08:41:24.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60e6928640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e006b380 con 0x7f60e006abe0 2024-04-25T08:41:24.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60dffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f60e00b2d60 0x7f60e006c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:24.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60e4ec5640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f60e015e5c0 0x7f60e015f030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44870/0 (socket says 172.21.15.38:44870) 2024-04-25T08:41:24.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60e4ec5640 1 -- 172.21.15.38:0/1734372338 learned_addr learned my addr 172.21.15.38:0/1734372338 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:24.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60df7fe640 1 --2- 172.21.15.38:0/1734372338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e006abe0 0x7f60e00b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:24.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60dffff640 1 -- 172.21.15.38:0/1734372338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e006abe0 msgr2=0x7f60e00b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:24.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60dffff640 1 --2- 172.21.15.38:0/1734372338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e006abe0 0x7f60e00b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60dffff640 1 -- 172.21.15.38:0/1734372338 >> v1:172.21.15.38:6789/0 conn(0x7f60e015e5c0 legacy=0x7f60e015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:24.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.490+0000 7f60dffff640 1 -- 172.21.15.38:0/1734372338 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e006b660 con 0x7f60e00b2d60 2024-04-25T08:41:24.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.491+0000 7f60dffff640 1 --2- 172.21.15.38:0/1734372338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e00b2d60 0x7f60e006c140 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f60d40097a0 tx=0x7f60d402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8136cfb0e621aa9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:24.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.491+0000 7f60deffd640 1 -- 172.21.15.38:0/1734372338 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4037070 con 0x7f60e00b2d60 2024-04-25T08:41:24.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.491+0000 7f60deffd640 1 -- 172.21.15.38:0/1734372338 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d4032440 con 0x7f60e00b2d60 2024-04-25T08:41:24.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.491+0000 7f60deffd640 1 -- 172.21.15.38:0/1734372338 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4032810 con 0x7f60e00b2d60 2024-04-25T08:41:24.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.492+0000 7f60e6928640 1 -- 172.21.15.38:0/1734372338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e00b2d60 msgr2=0x7f60e006c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:24.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.492+0000 7f60e6928640 1 --2- 172.21.15.38:0/1734372338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e00b2d60 0x7f60e006c140 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f60d40097a0 tx=0x7f60d402a180 comp rx=0 tx=0).stop 2024-04-25T08:41:24.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.492+0000 7f60e6928640 1 -- 172.21.15.38:0/1734372338 shutdown_connections 2024-04-25T08:41:24.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.492+0000 7f60e6928640 1 --2- 172.21.15.38:0/1734372338 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e006abe0 0x7f60e00b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.492+0000 7f60e6928640 1 --2- 172.21.15.38:0/1734372338 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e00b2d60 0x7f60e006c140 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.492+0000 7f60e6928640 1 -- 172.21.15.38:0/1734372338 >> 172.21.15.38:0/1734372338 conn(0x7f60e00cfd30 msgr2=0x7f60e00d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:24.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.492+0000 7f60e6928640 1 -- 172.21.15.38:0/1734372338 shutdown_connections 2024-04-25T08:41:24.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.492+0000 7f60e6928640 1 -- 172.21.15.38:0/1734372338 wait complete. 2024-04-25T08:41:24.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.493+0000 7f60e6928640 1 Processor -- start 2024-04-25T08:41:24.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.493+0000 7f60e6928640 1 -- start start 2024-04-25T08:41:24.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60e6928640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e006abe0 0x7f60e0145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:24.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60e6928640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e00b2d60 0x7f60e01e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:24.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60e6928640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e015e5c0 0x7f60e01e9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:24.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60e6928640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e015ea30 con 0x7f60e00b2d60 2024-04-25T08:41:24.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60e6928640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e006bc20 con 0x7f60e006abe0 2024-04-25T08:41:24.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60e6928640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e015ebb0 con 0x7f60e015e5c0 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60df7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e00b2d60 0x7f60e01e8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60df7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e00b2d60 0x7f60e01e8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50030/0 (socket says 172.21.15.38:50030) 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60df7fe640 1 -- 172.21.15.38:0/1513095190 learned_addr learned my addr 172.21.15.38:0/1513095190 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60e4ec5640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e015e5c0 0x7f60e01e9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60dffff640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e006abe0 0x7f60e0145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60df7fe640 1 -- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e015e5c0 msgr2=0x7f60e01e9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60df7fe640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e015e5c0 0x7f60e01e9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60df7fe640 1 -- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e006abe0 msgr2=0x7f60e0145a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60df7fe640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e006abe0 0x7f60e0145a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60df7fe640 1 -- 172.21.15.38:0/1513095190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e01e8830 con 0x7f60e00b2d60 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60df7fe640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e00b2d60 0x7f60e01e8ef0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f60d000d800 tx=0x7f60d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60dcff9640 1 -- 172.21.15.38:0/1513095190 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d000b6c0 con 0x7f60e00b2d60 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.494+0000 7f60dcff9640 1 -- 172.21.15.38:0/1513095190 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d0019070 con 0x7f60e00b2d60 2024-04-25T08:41:24.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.495+0000 7f60e6928640 1 -- 172.21.15.38:0/1513095190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e0145f40 con 0x7f60e00b2d60 2024-04-25T08:41:24.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.495+0000 7f60dcff9640 1 -- 172.21.15.38:0/1513095190 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d000ba80 con 0x7f60e00b2d60 2024-04-25T08:41:24.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.495+0000 7f60e6928640 1 -- 172.21.15.38:0/1513095190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e01e7a70 con 0x7f60e00b2d60 2024-04-25T08:41:24.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.496+0000 7f60e6928640 1 -- 172.21.15.38:0/1513095190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e0068e80 con 0x7f60e00b2d60 2024-04-25T08:41:24.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.496+0000 7f60dcff9640 1 -- 172.21.15.38:0/1513095190 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60d000bc20 con 0x7f60e00b2d60 2024-04-25T08:41:24.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.496+0000 7f60dcff9640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60cc06b500 0x7f60cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:24.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.497+0000 7f60dcff9640 1 -- 172.21.15.38:0/1513095190 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f60d008a6d0 con 0x7f60e00b2d60 2024-04-25T08:41:24.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.497+0000 7f60dffff640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60cc06b500 0x7f60cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:24.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.497+0000 7f60dffff640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60cc06b500 0x7f60cc06d9c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f60d4034000 tx=0x7f60d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:24.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.501+0000 7f60dcff9640 1 -- 172.21.15.38:0/1513095190 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60d0058d70 con 0x7f60e00b2d60 2024-04-25T08:41:24.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.720+0000 7f60e6928640 1 -- 172.21.15.38:0/1513095190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60e01603f0 con 0x7f60e00b2d60 2024-04-25T08:41:24.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.721+0000 7f60dcff9640 1 -- 172.21.15.38:0/1513095190 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f60d005f0c0 con 0x7f60e00b2d60 2024-04-25T08:41:24.715 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:24.715 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:40:55.730538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/1312063561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1312063561},{"type":"v1","addr":"172.21.15.38:6835","nonce":1312063561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:24.715 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 77 2024-04-25T08:41:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.723+0000 7f60e6928640 1 -- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60cc06b500 msgr2=0x7f60cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.723+0000 7f60e6928640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60cc06b500 0x7f60cc06d9c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f60d4034000 tx=0x7f60d402e000 comp rx=0 tx=0).stop 2024-04-25T08:41:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.723+0000 7f60e6928640 1 -- 172.21.15.38:0/1513095190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e00b2d60 msgr2=0x7f60e01e8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.723+0000 7f60e6928640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e00b2d60 0x7f60e01e8ef0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f60d000d800 tx=0x7f60d000dcc0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.723+0000 7f60e6928640 1 -- 172.21.15.38:0/1513095190 shutdown_connections 2024-04-25T08:41:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.723+0000 7f60e6928640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f60cc06b500 0x7f60cc06d9c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.723+0000 7f60e6928640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60e015e5c0 0x7f60e01e9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.723+0000 7f60e6928640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f60e00b2d60 0x7f60e01e8ef0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.723+0000 7f60e6928640 1 --2- 172.21.15.38:0/1513095190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60e006abe0 0x7f60e0145a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:24.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.723+0000 7f60e6928640 1 -- 172.21.15.38:0/1513095190 >> 172.21.15.38:0/1513095190 conn(0x7f60e00cfd30 msgr2=0x7f60e00b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:24.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.723+0000 7f60e6928640 1 -- 172.21.15.38:0/1513095190 shutdown_connections 2024-04-25T08:41:24.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:24.723+0000 7f60e6928640 1 -- 172.21.15.38:0/1513095190 wait complete. 2024-04-25T08:41:24.728 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T08:41:29.729 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:29.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.852+0000 7f0652d5c640 1 Processor -- start 2024-04-25T08:41:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.852+0000 7f0652d5c640 1 -- start start 2024-04-25T08:41:29.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f0652d5c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f064c0b2d60 0x7f064c06c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:29.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f0652d5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f064c06abe0 0x7f064c0b3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:29.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f0652d5c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064c0b3680 con 0x7f064c15e5c0 2024-04-25T08:41:29.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f0652d5c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064c06b0a0 con 0x7f064c0b2d60 2024-04-25T08:41:29.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f0652d5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c06b380 con 0x7f064c06abe0 2024-04-25T08:41:29.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f0650af8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f064c0b2d60 0x7f064c06c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:29.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f0650af8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f064c0b2d60 0x7f064c06c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39736/0 (socket says 172.21.15.38:39736) 2024-04-25T08:41:29.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f0650af8640 1 -- 172.21.15.38:0/2168764485 learned_addr learned my addr 172.21.15.38:0/2168764485 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:29.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f064bfff640 1 --2- 172.21.15.38:0/2168764485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f064c06abe0 0x7f064c0b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:29.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f0650af8640 1 -- 172.21.15.38:0/2168764485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f064c06abe0 msgr2=0x7f064c0b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:29.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f0650af8640 1 --2- 172.21.15.38:0/2168764485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f064c06abe0 0x7f064c0b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:29.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f0650af8640 1 -- 172.21.15.38:0/2168764485 >> v1:172.21.15.107:6789/0 conn(0x7f064c15e5c0 legacy=0x7f064c15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:29.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.853+0000 7f0650af8640 1 -- 172.21.15.38:0/2168764485 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c06b660 con 0x7f064c0b2d60 2024-04-25T08:41:29.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.854+0000 7f0650af8640 1 --2- 172.21.15.38:0/2168764485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f064c0b2d60 0x7f064c06c140 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f0640009870 tx=0x7f064002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e8a829d1512edc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:29.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.854+0000 7f064b7fe640 1 -- 172.21.15.38:0/2168764485 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640037070 con 0x7f064c0b2d60 2024-04-25T08:41:29.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.854+0000 7f064b7fe640 1 -- 172.21.15.38:0/2168764485 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0640032440 con 0x7f064c0b2d60 2024-04-25T08:41:29.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.854+0000 7f064b7fe640 1 -- 172.21.15.38:0/2168764485 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640032810 con 0x7f064c0b2d60 2024-04-25T08:41:29.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.855+0000 7f0652d5c640 1 -- 172.21.15.38:0/2168764485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f064c0b2d60 msgr2=0x7f064c06c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.855+0000 7f0652d5c640 1 --2- 172.21.15.38:0/2168764485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f064c0b2d60 0x7f064c06c140 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f0640009870 tx=0x7f064002a210 comp rx=0 tx=0).stop 2024-04-25T08:41:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.855+0000 7f0652d5c640 1 -- 172.21.15.38:0/2168764485 shutdown_connections 2024-04-25T08:41:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.855+0000 7f0652d5c640 1 --2- 172.21.15.38:0/2168764485 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f064c06abe0 0x7f064c0b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.855+0000 7f0652d5c640 1 --2- 172.21.15.38:0/2168764485 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f064c0b2d60 0x7f064c06c140 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.855+0000 7f0652d5c640 1 -- 172.21.15.38:0/2168764485 >> 172.21.15.38:0/2168764485 conn(0x7f064c0cfd30 msgr2=0x7f064c0d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.855+0000 7f0652d5c640 1 -- 172.21.15.38:0/2168764485 shutdown_connections 2024-04-25T08:41:29.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.855+0000 7f0652d5c640 1 -- 172.21.15.38:0/2168764485 wait complete. 2024-04-25T08:41:29.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.856+0000 7f0652d5c640 1 Processor -- start 2024-04-25T08:41:29.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.856+0000 7f0652d5c640 1 -- start start 2024-04-25T08:41:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.857+0000 7f0652d5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f064c06abe0 0x7f064c1e70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.857+0000 7f0652d5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f064c0b2d60 0x7f064c1457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.857+0000 7f0652d5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f064c15e5c0 0x7f064c156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.857+0000 7f0652d5c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c15ea30 con 0x7f064c06abe0 2024-04-25T08:41:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.857+0000 7f0652d5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c06bc20 con 0x7f064c15e5c0 2024-04-25T08:41:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.857+0000 7f0652d5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064c15ebb0 con 0x7f064c0b2d60 2024-04-25T08:41:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.857+0000 7f0650af8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f064c06abe0 0x7f064c1e70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.857+0000 7f0650af8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f064c06abe0 0x7f064c1e70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39744/0 (socket says 172.21.15.38:39744) 2024-04-25T08:41:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.857+0000 7f0650af8640 1 -- 172.21.15.38:0/4074886912 learned_addr learned my addr 172.21.15.38:0/4074886912 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f064bfff640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f064c0b2d60 0x7f064c1457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f06512f9640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f064c15e5c0 0x7f064c156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:29.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f0650af8640 1 -- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f064c0b2d60 msgr2=0x7f064c1457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:29.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f0650af8640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f064c0b2d60 0x7f064c1457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:29.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f0650af8640 1 -- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f064c15e5c0 msgr2=0x7f064c156c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:29.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f0650af8640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f064c15e5c0 0x7f064c156c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:29.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f0650af8640 1 -- 172.21.15.38:0/4074886912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c145ce0 con 0x7f064c06abe0 2024-04-25T08:41:29.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f06512f9640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f064c15e5c0 0x7f064c156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:29.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f0650af8640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f064c06abe0 0x7f064c1e70d0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f0640000c00 tx=0x7f06400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:29.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f06497fa640 1 -- 172.21.15.38:0/4074886912 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640041020 con 0x7f064c06abe0 2024-04-25T08:41:29.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f06497fa640 1 -- 172.21.15.38:0/4074886912 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06400306c0 con 0x7f064c06abe0 2024-04-25T08:41:29.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f0652d5c640 1 -- 172.21.15.38:0/4074886912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064c1e6310 con 0x7f064c06abe0 2024-04-25T08:41:29.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f06497fa640 1 -- 172.21.15.38:0/4074886912 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0640037040 con 0x7f064c06abe0 2024-04-25T08:41:29.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.858+0000 7f0652d5c640 1 -- 172.21.15.38:0/4074886912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064c1e68d0 con 0x7f064c06abe0 2024-04-25T08:41:29.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.859+0000 7f0652d5c640 1 -- 172.21.15.38:0/4074886912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064c068e80 con 0x7f064c06abe0 2024-04-25T08:41:29.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.860+0000 7f06497fa640 1 -- 172.21.15.38:0/4074886912 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0640044020 con 0x7f064c06abe0 2024-04-25T08:41:29.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.860+0000 7f06497fa640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f063006b4d0 0x7f063006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:29.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.860+0000 7f06497fa640 1 -- 172.21.15.38:0/4074886912 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f06400aab00 con 0x7f064c06abe0 2024-04-25T08:41:29.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.860+0000 7f064bfff640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f063006b4d0 0x7f063006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:29.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.861+0000 7f064bfff640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f063006b4d0 0x7f063006d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f064c1e7630 tx=0x7f0638008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:29.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:29.864+0000 7f06497fa640 1 -- 172.21.15.38:0/4074886912 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f064004b030 con 0x7f064c06abe0 2024-04-25T08:41:30.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.082+0000 7f0652d5c640 1 -- 172.21.15.38:0/4074886912 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f064c0b39c0 con 0x7f064c06abe0 2024-04-25T08:41:30.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.083+0000 7f06497fa640 1 -- 172.21.15.38:0/4074886912 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f064c145ce0 con 0x7f064c06abe0 2024-04-25T08:41:30.076 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:30.076 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:40:55.730538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/1312063561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1312063561},{"type":"v1","addr":"172.21.15.38:6835","nonce":1312063561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:30.076 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 77 2024-04-25T08:41:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.085+0000 7f0652d5c640 1 -- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f063006b4d0 msgr2=0x7f063006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.085+0000 7f0652d5c640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f063006b4d0 0x7f063006d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f064c1e7630 tx=0x7f0638008040 comp rx=0 tx=0).stop 2024-04-25T08:41:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.085+0000 7f0652d5c640 1 -- 172.21.15.38:0/4074886912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f064c06abe0 msgr2=0x7f064c1e70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.085+0000 7f0652d5c640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f064c06abe0 0x7f064c1e70d0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f0640000c00 tx=0x7f06400046c0 comp rx=0 tx=0).stop 2024-04-25T08:41:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.085+0000 7f0652d5c640 1 -- 172.21.15.38:0/4074886912 shutdown_connections 2024-04-25T08:41:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.085+0000 7f0652d5c640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f064c15e5c0 0x7f064c156c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.085+0000 7f0652d5c640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f063006b4d0 0x7f063006d990 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.085+0000 7f0652d5c640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f064c0b2d60 0x7f064c1457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.085+0000 7f0652d5c640 1 --2- 172.21.15.38:0/4074886912 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f064c06abe0 0x7f064c1e70d0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.085+0000 7f0652d5c640 1 -- 172.21.15.38:0/4074886912 >> 172.21.15.38:0/4074886912 conn(0x7f064c0cfd30 msgr2=0x7f064c160310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.085+0000 7f0652d5c640 1 -- 172.21.15.38:0/4074886912 shutdown_connections 2024-04-25T08:41:30.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:30.085+0000 7f0652d5c640 1 -- 172.21.15.38:0/4074886912 wait complete. 2024-04-25T08:41:30.089 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T08:41:34.708 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:41:34.711 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:41:35.090 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:35.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.211+0000 7f0275d42640 1 Processor -- start 2024-04-25T08:41:35.204 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.211+0000 7f0275d42640 1 -- start start 2024-04-25T08:41:35.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.212+0000 7f0275d42640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f027015ddf0 0x7f0270161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:35.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.212+0000 7f0275d42640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02700b98a0 con 0x7f02700b5710 2024-04-25T08:41:35.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.212+0000 7f0275d42640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0270161e00 con 0x7f02700b93f0 2024-04-25T08:41:35.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.212+0000 7f0275d42640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02701620e0 con 0x7f027015ddf0 2024-04-25T08:41:35.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.212+0000 7f026effd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f027015ddf0 0x7f0270161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:35.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.212+0000 7f026f7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f02700b5710 0x7f02700b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:56750/0 (socket says 172.21.15.38:56750) 2024-04-25T08:41:35.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.212+0000 7f026f7fe640 1 -- 172.21.15.38:0/1826020443 learned_addr learned my addr 172.21.15.38:0/1826020443 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:35.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.212+0000 7f026effd640 1 -- 172.21.15.38:0/1826020443 >> v1:172.21.15.107:6789/0 conn(0x7f02700b93f0 legacy=0x7f02700d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:35.205 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.213+0000 7f026effd640 1 -- 172.21.15.38:0/1826020443 >> v1:172.21.15.38:6789/0 conn(0x7f02700b5710 legacy=0x7f02700b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:35.206 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.213+0000 7f026effd640 1 -- 172.21.15.38:0/1826020443 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02701623c0 con 0x7f027015ddf0 2024-04-25T08:41:35.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.213+0000 7f026effd640 1 --2- 172.21.15.38:0/1826020443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f027015ddf0 0x7f0270161700 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f02640097a0 tx=0x7f026402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f874dff8a068f9f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:35.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.213+0000 7f026e7fc640 1 -- 172.21.15.38:0/1826020443 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264037070 con 0x7f027015ddf0 2024-04-25T08:41:35.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.213+0000 7f026e7fc640 1 -- 172.21.15.38:0/1826020443 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0264032440 con 0x7f027015ddf0 2024-04-25T08:41:35.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.213+0000 7f026e7fc640 1 -- 172.21.15.38:0/1826020443 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0264032810 con 0x7f027015ddf0 2024-04-25T08:41:35.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.214+0000 7f0275d42640 1 -- 172.21.15.38:0/1826020443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f027015ddf0 msgr2=0x7f0270161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:35.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.214+0000 7f0275d42640 1 --2- 172.21.15.38:0/1826020443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f027015ddf0 0x7f0270161700 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f02640097a0 tx=0x7f026402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:41:35.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.214+0000 7f0275d42640 1 -- 172.21.15.38:0/1826020443 shutdown_connections 2024-04-25T08:41:35.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.214+0000 7f0275d42640 1 --2- 172.21.15.38:0/1826020443 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f027015ddf0 0x7f0270161700 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:35.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.214+0000 7f0275d42640 1 -- 172.21.15.38:0/1826020443 >> 172.21.15.38:0/1826020443 conn(0x7f02700d07c0 msgr2=0x7f02700d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:35.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.214+0000 7f0275d42640 1 -- 172.21.15.38:0/1826020443 shutdown_connections 2024-04-25T08:41:35.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.214+0000 7f0275d42640 1 -- 172.21.15.38:0/1826020443 wait complete. 2024-04-25T08:41:35.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.215+0000 7f0275d42640 1 Processor -- start 2024-04-25T08:41:35.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.216+0000 7f0275d42640 1 -- start start 2024-04-25T08:41:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.216+0000 7f0275d42640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02700b5710 0x7f02701f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.216+0000 7f0275d42640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b93f0 0x7f027015f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.216+0000 7f0275d42640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027015ddf0 0x7f02701e73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.216+0000 7f0275d42640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02700d2de0 con 0x7f02700b5710 2024-04-25T08:41:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.216+0000 7f0275d42640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02700d2c60 con 0x7f027015ddf0 2024-04-25T08:41:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.216+0000 7f0275d42640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02700d2f60 con 0x7f02700b93f0 2024-04-25T08:41:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.216+0000 7f026f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02700b5710 0x7f02701f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.216+0000 7f026f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02700b5710 0x7f02701f34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39760/0 (socket says 172.21.15.38:39760) 2024-04-25T08:41:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.216+0000 7f026f7fe640 1 -- 172.21.15.38:0/1886553765 learned_addr learned my addr 172.21.15.38:0/1886553765 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:35.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.217+0000 7f026f7fe640 1 -- 172.21.15.38:0/1886553765 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b93f0 msgr2=0x7f027015f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:35.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.217+0000 7f026f7fe640 1 --2- 172.21.15.38:0/1886553765 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b93f0 0x7f027015f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:35.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.217+0000 7f026f7fe640 1 -- 172.21.15.38:0/1886553765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027015ddf0 msgr2=0x7f02701e73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:35.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.217+0000 7f026f7fe640 1 --2- 172.21.15.38:0/1886553765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027015ddf0 0x7f02701e73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:35.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.217+0000 7f026f7fe640 1 -- 172.21.15.38:0/1886553765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02701f39e0 con 0x7f02700b5710 2024-04-25T08:41:35.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.217+0000 7f026f7fe640 1 --2- 172.21.15.38:0/1886553765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02700b5710 0x7f02701f34a0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f026000cb40 tx=0x7f026000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:35.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.217+0000 7f0257fff640 1 -- 172.21.15.38:0/1886553765 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260017070 con 0x7f02700b5710 2024-04-25T08:41:35.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.217+0000 7f0257fff640 1 -- 172.21.15.38:0/1886553765 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026000aca0 con 0x7f02700b5710 2024-04-25T08:41:35.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.217+0000 7f0257fff640 1 -- 172.21.15.38:0/1886553765 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0260012620 con 0x7f02700b5710 2024-04-25T08:41:35.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.217+0000 7f0275d42640 1 -- 172.21.15.38:0/1886553765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027015fc70 con 0x7f02700b5710 2024-04-25T08:41:35.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.217+0000 7f0275d42640 1 -- 172.21.15.38:0/1886553765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0270160230 con 0x7f02700b5710 2024-04-25T08:41:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.218+0000 7f0275d42640 1 -- 172.21.15.38:0/1886553765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0270068e80 con 0x7f02700b5710 2024-04-25T08:41:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.219+0000 7f0257fff640 1 -- 172.21.15.38:0/1886553765 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f026001a070 con 0x7f02700b5710 2024-04-25T08:41:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.219+0000 7f0257fff640 1 --2- 172.21.15.38:0/1886553765 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f025c06b4d0 0x7f025c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.219+0000 7f0257fff640 1 -- 172.21.15.38:0/1886553765 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f026008a0f0 con 0x7f02700b5710 2024-04-25T08:41:35.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.220+0000 7f026effd640 1 --2- 172.21.15.38:0/1886553765 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f025c06b4d0 0x7f025c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:35.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.220+0000 7f026effd640 1 --2- 172.21.15.38:0/1886553765 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f025c06b4d0 0x7f025c06d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0264034000 tx=0x7f026402c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:35.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.224+0000 7f0257fff640 1 -- 172.21.15.38:0/1886553765 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02701f39e0 con 0x7f02700b5710 2024-04-25T08:41:35.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.443+0000 7f0275d42640 1 -- 172.21.15.38:0/1886553765 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0270067dd0 con 0x7f02700b5710 2024-04-25T08:41:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.443+0000 7f0257fff640 1 -- 172.21.15.38:0/1886553765 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f0260058e70 con 0x7f02700b5710 2024-04-25T08:41:35.436 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:35.436 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:40:55.730538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/1312063561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1312063561},{"type":"v1","addr":"172.21.15.38:6835","nonce":1312063561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:35.436 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 77 2024-04-25T08:41:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.445+0000 7f0275d42640 1 -- 172.21.15.38:0/1886553765 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f025c06b4d0 msgr2=0x7f025c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.445+0000 7f0275d42640 1 --2- 172.21.15.38:0/1886553765 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f025c06b4d0 0x7f025c06d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0264034000 tx=0x7f026402c000 comp rx=0 tx=0).stop 2024-04-25T08:41:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.445+0000 7f0275d42640 1 -- 172.21.15.38:0/1886553765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02700b5710 msgr2=0x7f02701f34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.445+0000 7f0275d42640 1 --2- 172.21.15.38:0/1886553765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02700b5710 0x7f02701f34a0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f026000cb40 tx=0x7f026000a060 comp rx=0 tx=0).stop 2024-04-25T08:41:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.445+0000 7f0275d42640 1 -- 172.21.15.38:0/1886553765 shutdown_connections 2024-04-25T08:41:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.445+0000 7f0275d42640 1 --2- 172.21.15.38:0/1886553765 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f027015ddf0 0x7f02701e73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:35.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.445+0000 7f0275d42640 1 --2- 172.21.15.38:0/1886553765 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f025c06b4d0 0x7f025c06d990 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.445+0000 7f0275d42640 1 --2- 172.21.15.38:0/1886553765 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f02700b93f0 0x7f027015f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.445+0000 7f0275d42640 1 --2- 172.21.15.38:0/1886553765 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02700b5710 0x7f02701f34a0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.445+0000 7f0275d42640 1 -- 172.21.15.38:0/1886553765 >> 172.21.15.38:0/1886553765 conn(0x7f02700d07c0 msgr2=0x7f02700ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.445+0000 7f0275d42640 1 -- 172.21.15.38:0/1886553765 shutdown_connections 2024-04-25T08:41:35.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:35.445+0000 7f0275d42640 1 -- 172.21.15.38:0/1886553765 wait complete. 2024-04-25T08:41:35.449 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T08:41:40.450 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:40.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.572+0000 7f10598f5640 1 Processor -- start 2024-04-25T08:41:40.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.572+0000 7f10598f5640 1 -- start start 2024-04-25T08:41:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.572+0000 7f10598f5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f105409acb0 0x7f10540cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.572+0000 7f10598f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f105406cdf0 0x7f10540cc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.572+0000 7f10598f5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1054161540 con 0x7f10540cd8d0 2024-04-25T08:41:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.572+0000 7f10598f5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105409a820 con 0x7f105409acb0 2024-04-25T08:41:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.572+0000 7f10598f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105406be90 con 0x7f105406cdf0 2024-04-25T08:41:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.573+0000 7f10527fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f105409acb0 0x7f10540cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.573+0000 7f10527fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f105409acb0 0x7f10540cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46420/0 (socket says 172.21.15.38:46420) 2024-04-25T08:41:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.573+0000 7f10527fc640 1 -- 172.21.15.38:0/634045269 learned_addr learned my addr 172.21.15.38:0/634045269 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.573+0000 7f10537fe640 1 --2- 172.21.15.38:0/634045269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f105406cdf0 0x7f10540cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.573+0000 7f10527fc640 1 -- 172.21.15.38:0/634045269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f105406cdf0 msgr2=0x7f10540cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.573+0000 7f10527fc640 1 --2- 172.21.15.38:0/634045269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f105406cdf0 0x7f10540cc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:40.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.573+0000 7f10527fc640 1 -- 172.21.15.38:0/634045269 >> v1:172.21.15.107:6789/0 conn(0x7f10540cd8d0 legacy=0x7f105406c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:40.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.573+0000 7f10527fc640 1 -- 172.21.15.38:0/634045269 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105409a520 con 0x7f105409acb0 2024-04-25T08:41:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.574+0000 7f10527fc640 1 --2- 172.21.15.38:0/634045269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f105409acb0 0x7f10540cbcc0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f103c009870 tx=0x7f103c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4095496b8fba846b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.574+0000 7f1051ffb640 1 -- 172.21.15.38:0/634045269 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c037070 con 0x7f105409acb0 2024-04-25T08:41:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.574+0000 7f1051ffb640 1 -- 172.21.15.38:0/634045269 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c032440 con 0x7f105409acb0 2024-04-25T08:41:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.574+0000 7f1051ffb640 1 -- 172.21.15.38:0/634045269 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c032810 con 0x7f105409acb0 2024-04-25T08:41:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.574+0000 7f10598f5640 1 -- 172.21.15.38:0/634045269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f105409acb0 msgr2=0x7f10540cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.574+0000 7f10598f5640 1 --2- 172.21.15.38:0/634045269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f105409acb0 0x7f10540cbcc0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f103c009870 tx=0x7f103c02a270 comp rx=0 tx=0).stop 2024-04-25T08:41:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.574+0000 7f10598f5640 1 -- 172.21.15.38:0/634045269 shutdown_connections 2024-04-25T08:41:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.574+0000 7f10598f5640 1 --2- 172.21.15.38:0/634045269 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f105406cdf0 0x7f10540cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.574+0000 7f10598f5640 1 --2- 172.21.15.38:0/634045269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f105409acb0 0x7f10540cbcc0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.574+0000 7f10598f5640 1 -- 172.21.15.38:0/634045269 >> 172.21.15.38:0/634045269 conn(0x7f10540b3160 msgr2=0x7f10540b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.575+0000 7f10598f5640 1 -- 172.21.15.38:0/634045269 shutdown_connections 2024-04-25T08:41:40.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.575+0000 7f10598f5640 1 -- 172.21.15.38:0/634045269 wait complete. 2024-04-25T08:41:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.576+0000 7f10598f5640 1 Processor -- start 2024-04-25T08:41:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.576+0000 7f10598f5640 1 -- start start 2024-04-25T08:41:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10598f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f105406cdf0 0x7f105414f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10598f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f105409acb0 0x7f10541f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10598f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10540cd8d0 0x7f10541f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10598f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105406c410 con 0x7f10540cd8d0 2024-04-25T08:41:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10598f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10540d0ad0 con 0x7f105409acb0 2024-04-25T08:41:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10598f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10540b2170 con 0x7f105406cdf0 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10537fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10540cd8d0 0x7f10541f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10537fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10540cd8d0 0x7f10541f7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46426/0 (socket says 172.21.15.38:46426) 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10537fe640 1 -- 172.21.15.38:0/553558736 learned_addr learned my addr 172.21.15.38:0/553558736 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f1052ffd640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f105406cdf0 0x7f105414f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10527fc640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f105409acb0 0x7f10541f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10537fe640 1 -- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f105406cdf0 msgr2=0x7f105414f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10537fe640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f105406cdf0 0x7f105414f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10537fe640 1 -- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f105409acb0 msgr2=0x7f10541f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10537fe640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f105409acb0 0x7f10541f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10537fe640 1 -- 172.21.15.38:0/553558736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10540b0970 con 0x7f10540cd8d0 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10527fc640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f105409acb0 0x7f10541f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f1052ffd640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f105406cdf0 0x7f105414f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.577+0000 7f10537fe640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10540cd8d0 0x7f10541f7590 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f104400d5a0 tx=0x7f104400da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:40.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.578+0000 7f104bfff640 1 -- 172.21.15.38:0/553558736 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10440125f0 con 0x7f10540cd8d0 2024-04-25T08:41:40.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.578+0000 7f104bfff640 1 -- 172.21.15.38:0/553558736 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1044019070 con 0x7f10540cd8d0 2024-04-25T08:41:40.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.578+0000 7f10598f5640 1 -- 172.21.15.38:0/553558736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10540b0c50 con 0x7f10540cd8d0 2024-04-25T08:41:40.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.578+0000 7f104bfff640 1 -- 172.21.15.38:0/553558736 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10440129d0 con 0x7f10540cd8d0 2024-04-25T08:41:40.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.578+0000 7f10598f5640 1 -- 172.21.15.38:0/553558736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10540b1210 con 0x7f10540cd8d0 2024-04-25T08:41:40.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.579+0000 7f10598f5640 1 -- 172.21.15.38:0/553558736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1054068e80 con 0x7f10540cd8d0 2024-04-25T08:41:40.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.579+0000 7f104bfff640 1 -- 172.21.15.38:0/553558736 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1044012b70 con 0x7f10540cd8d0 2024-04-25T08:41:40.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.580+0000 7f104bfff640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f103406b4d0 0x7f103406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:40.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.580+0000 7f104bfff640 1 -- 172.21.15.38:0/553558736 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f104408a790 con 0x7f10540cd8d0 2024-04-25T08:41:40.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.580+0000 7f1052ffd640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f103406b4d0 0x7f103406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:40.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.580+0000 7f1052ffd640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f103406b4d0 0x7f103406d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f104c001d60 tx=0x7f104c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:40.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.584+0000 7f104bfff640 1 -- 172.21.15.38:0/553558736 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1044059470 con 0x7f10540cd8d0 2024-04-25T08:41:40.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.809+0000 7f10598f5640 1 -- 172.21.15.38:0/553558736 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10540d08b0 con 0x7f10540cd8d0 2024-04-25T08:41:40.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.809+0000 7f104bfff640 1 -- 172.21.15.38:0/553558736 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f1044058d70 con 0x7f10540cd8d0 2024-04-25T08:41:40.802 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:40.802 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9486,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:40:55.730538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9489},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9489":{"gid":9489,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/1312063561","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1312063561},{"type":"v1","addr":"172.21.15.38:6835","nonce":1312063561}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:40.802 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 77 2024-04-25T08:41:40.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.811+0000 7f10598f5640 1 -- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f103406b4d0 msgr2=0x7f103406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:40.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.811+0000 7f10598f5640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f103406b4d0 0x7f103406d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f104c001d60 tx=0x7f104c009040 comp rx=0 tx=0).stop 2024-04-25T08:41:40.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.811+0000 7f10598f5640 1 -- 172.21.15.38:0/553558736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10540cd8d0 msgr2=0x7f10541f7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:40.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.811+0000 7f10598f5640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10540cd8d0 0x7f10541f7590 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f104400d5a0 tx=0x7f104400da60 comp rx=0 tx=0).stop 2024-04-25T08:41:40.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.811+0000 7f10598f5640 1 -- 172.21.15.38:0/553558736 shutdown_connections 2024-04-25T08:41:40.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.811+0000 7f10598f5640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10540cd8d0 0x7f10541f7590 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:40.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.811+0000 7f10598f5640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f105409acb0 0x7f10541f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:40.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.811+0000 7f10598f5640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f103406b4d0 0x7f103406d990 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:40.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.811+0000 7f10598f5640 1 --2- 172.21.15.38:0/553558736 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f105406cdf0 0x7f105414f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:40.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.811+0000 7f10598f5640 1 -- 172.21.15.38:0/553558736 >> 172.21.15.38:0/553558736 conn(0x7f10540b3160 msgr2=0x7f10540d0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:40.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.811+0000 7f10598f5640 1 -- 172.21.15.38:0/553558736 shutdown_connections 2024-04-25T08:41:40.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:40.811+0000 7f10598f5640 1 -- 172.21.15.38:0/553558736 wait complete. 2024-04-25T08:41:40.815 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T08:41:45.338 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-04-25T08:41:45.345+0000 7ff470e2e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:41:45.816 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:45.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.938+0000 7ffbc98ac640 1 Processor -- start 2024-04-25T08:41:45.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.938+0000 7ffbc98ac640 1 -- start start 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbc98ac640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffbc415a7a0 0x7ffbc409dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbc98ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc40b3210 0x7ffbc415e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbc98ac640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc406d9f0 con 0x7ffbc40b2dd0 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbc98ac640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc406dcb0 con 0x7ffbc415a7a0 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbc98ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc406df90 con 0x7ffbc40b3210 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbc2ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ffbc415a7a0 0x7ffbc409dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbba7fc640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ffbc40b2dd0 0x7ffbc406e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33484/0 (socket says 172.21.15.38:33484) 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbba7fc640 1 -- 172.21.15.38:0/3373547367 learned_addr learned my addr 172.21.15.38:0/3373547367 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbc37fe640 1 --2- 172.21.15.38:0/3373547367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc40b3210 0x7ffbc415e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbc2ffd640 1 -- 172.21.15.38:0/3373547367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc40b3210 msgr2=0x7ffbc415e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbc2ffd640 1 --2- 172.21.15.38:0/3373547367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc40b3210 0x7ffbc415e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbc2ffd640 1 -- 172.21.15.38:0/3373547367 >> v1:172.21.15.38:6789/0 conn(0x7ffbc40b2dd0 legacy=0x7ffbc406e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:45.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbc2ffd640 1 -- 172.21.15.38:0/3373547367 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc406e270 con 0x7ffbc415a7a0 2024-04-25T08:41:45.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.939+0000 7ffbc27fc640 1 -- 172.21.15.38:0/3373547367 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986158959 0 0) 0x7ffbc406d9f0 con 0x7ffbc40b2dd0 2024-04-25T08:41:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.940+0000 7ffbc2ffd640 1 --2- 172.21.15.38:0/3373547367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc415a7a0 0x7ffbc409dbd0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40097a0 tx=0x7ffbb402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90c9037636e2c22a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.940+0000 7ffbc27fc640 1 -- 172.21.15.38:0/3373547367 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4037070 con 0x7ffbc415a7a0 2024-04-25T08:41:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.940+0000 7ffbc27fc640 1 -- 172.21.15.38:0/3373547367 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb4032440 con 0x7ffbc415a7a0 2024-04-25T08:41:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.940+0000 7ffbc27fc640 1 -- 172.21.15.38:0/3373547367 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4032810 con 0x7ffbc415a7a0 2024-04-25T08:41:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.941+0000 7ffbc98ac640 1 -- 172.21.15.38:0/3373547367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc415a7a0 msgr2=0x7ffbc409dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.941+0000 7ffbc98ac640 1 --2- 172.21.15.38:0/3373547367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc415a7a0 0x7ffbc409dbd0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40097a0 tx=0x7ffbb402a180 comp rx=0 tx=0).stop 2024-04-25T08:41:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.941+0000 7ffbc98ac640 1 -- 172.21.15.38:0/3373547367 shutdown_connections 2024-04-25T08:41:45.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.941+0000 7ffbc98ac640 1 --2- 172.21.15.38:0/3373547367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc40b3210 0x7ffbc415e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.941+0000 7ffbc98ac640 1 --2- 172.21.15.38:0/3373547367 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc415a7a0 0x7ffbc409dbd0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.941+0000 7ffbc98ac640 1 -- 172.21.15.38:0/3373547367 >> 172.21.15.38:0/3373547367 conn(0x7ffbc40cfdc0 msgr2=0x7ffbc40d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.941+0000 7ffbc98ac640 1 -- 172.21.15.38:0/3373547367 shutdown_connections 2024-04-25T08:41:45.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.941+0000 7ffbc98ac640 1 -- 172.21.15.38:0/3373547367 wait complete. 2024-04-25T08:41:45.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.942+0000 7ffbc98ac640 1 Processor -- start 2024-04-25T08:41:45.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.942+0000 7ffbc98ac640 1 -- start start 2024-04-25T08:41:45.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc98ac640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc40b2dd0 0x7ffbc40cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:45.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc98ac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbc40b3210 0x7ffbc40b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:45.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc98ac640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc415a7a0 0x7ffbc4157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:45.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc98ac640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40af480 con 0x7ffbc415a7a0 2024-04-25T08:41:45.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc98ac640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40af300 con 0x7ffbc40b3210 2024-04-25T08:41:45.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc98ac640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc406eea0 con 0x7ffbc40b2dd0 2024-04-25T08:41:45.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc415a7a0 0x7ffbc4157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:45.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc415a7a0 0x7ffbc4157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46440/0 (socket says 172.21.15.38:46440) 2024-04-25T08:41:45.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc37fe640 1 -- 172.21.15.38:0/2490693882 learned_addr learned my addr 172.21.15.38:0/2490693882 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:45.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbba7fc640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbc40b3210 0x7ffbc40b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:45.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc2ffd640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc40b2dd0 0x7ffbc40cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:45.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc37fe640 1 -- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc40b2dd0 msgr2=0x7ffbc40cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:45.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc37fe640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc40b2dd0 0x7ffbc40cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:45.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc37fe640 1 -- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbc40b3210 msgr2=0x7ffbc40b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:45.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc37fe640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbc40b3210 0x7ffbc40b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc37fe640 1 -- 172.21.15.38:0/2490693882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40cdb10 con 0x7ffbc415a7a0 2024-04-25T08:41:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.943+0000 7ffbc2ffd640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc40b2dd0 0x7ffbc40cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.944+0000 7ffbc37fe640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc415a7a0 0x7ffbc4157e70 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00cd40 tx=0x7ffbbc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.944+0000 7ffbc0ff9640 1 -- 172.21.15.38:0/2490693882 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc003b90 con 0x7ffbc415a7a0 2024-04-25T08:41:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.944+0000 7ffbc0ff9640 1 -- 172.21.15.38:0/2490693882 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc003d30 con 0x7ffbc415a7a0 2024-04-25T08:41:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.944+0000 7ffbc98ac640 1 -- 172.21.15.38:0/2490693882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc40cddf0 con 0x7ffbc415a7a0 2024-04-25T08:41:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.944+0000 7ffbc0ff9640 1 -- 172.21.15.38:0/2490693882 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc004240 con 0x7ffbc415a7a0 2024-04-25T08:41:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.944+0000 7ffbc98ac640 1 -- 172.21.15.38:0/2490693882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc40ce3b0 con 0x7ffbc415a7a0 2024-04-25T08:41:45.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.945+0000 7ffbc98ac640 1 -- 172.21.15.38:0/2490693882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc40669c0 con 0x7ffbc415a7a0 2024-04-25T08:41:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.945+0000 7ffbc0ff9640 1 -- 172.21.15.38:0/2490693882 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbbc0043e0 con 0x7ffbc415a7a0 2024-04-25T08:41:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.946+0000 7ffbc0ff9640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffba406b500 0x7ffba406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.946+0000 7ffbc0ff9640 1 -- 172.21.15.38:0/2490693882 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ffbbc08aed0 con 0x7ffbc415a7a0 2024-04-25T08:41:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.946+0000 7ffbc2ffd640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffba406b500 0x7ffba406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:45.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.946+0000 7ffbc2ffd640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffba406b500 0x7ffba406d9c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4034000 tx=0x7ffbb402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:45.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:45.950+0000 7ffbc0ff9640 1 -- 172.21.15.38:0/2490693882 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbbc059ad0 con 0x7ffbc415a7a0 2024-04-25T08:41:46.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.167+0000 7ffbc98ac640 1 -- 172.21.15.38:0/2490693882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbc415e330 con 0x7ffbc415a7a0 2024-04-25T08:41:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.167+0000 7ffbc0ff9640 1 -- 172.21.15.38:0/2490693882 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4158 (secure 0 0 0) 0x7ffbbc0593d0 con 0x7ffbc415a7a0 2024-04-25T08:41:46.160 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:46.160 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:41:42.972814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9486":{"gid":9486,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":16,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:46.160 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 81 2024-04-25T08:41:46.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.169+0000 7ffbc98ac640 1 -- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffba406b500 msgr2=0x7ffba406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.169+0000 7ffbc98ac640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffba406b500 0x7ffba406d9c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4034000 tx=0x7ffbb402e000 comp rx=0 tx=0).stop 2024-04-25T08:41:46.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.169+0000 7ffbc98ac640 1 -- 172.21.15.38:0/2490693882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc415a7a0 msgr2=0x7ffbc4157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.169+0000 7ffbc98ac640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc415a7a0 0x7ffbc4157e70 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00cd40 tx=0x7ffbbc00a660 comp rx=0 tx=0).stop 2024-04-25T08:41:46.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.169+0000 7ffbc98ac640 1 -- 172.21.15.38:0/2490693882 shutdown_connections 2024-04-25T08:41:46.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.169+0000 7ffbc98ac640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbc415a7a0 0x7ffbc4157e70 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.169+0000 7ffbc98ac640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffbc40b3210 0x7ffbc40b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.169+0000 7ffbc98ac640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ffba406b500 0x7ffba406d9c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.169+0000 7ffbc98ac640 1 --2- 172.21.15.38:0/2490693882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ffbc40b2dd0 0x7ffbc40cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.169+0000 7ffbc98ac640 1 -- 172.21.15.38:0/2490693882 >> 172.21.15.38:0/2490693882 conn(0x7ffbc40cfdc0 msgr2=0x7ffbc415e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:46.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.169+0000 7ffbc98ac640 1 -- 172.21.15.38:0/2490693882 shutdown_connections 2024-04-25T08:41:46.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.169+0000 7ffbc98ac640 1 -- 172.21.15.38:0/2490693882 wait complete. 2024-04-25T08:41:46.172 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-25T08:41:46.173 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:46.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.281+0000 7f7bb82cb640 1 Processor -- start 2024-04-25T08:41:46.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.281+0000 7f7bb82cb640 1 -- start start 2024-04-25T08:41:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.282+0000 7f7bb82cb640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7bb00ce2e0 0x7f7bb00d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.282+0000 7f7bb82cb640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb00ceff0 con 0x7f7bb006c110 2024-04-25T08:41:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.282+0000 7f7bb82cb640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb00d1f70 con 0x7f7bb015df50 2024-04-25T08:41:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.282+0000 7f7bb82cb640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb00cf2d0 con 0x7f7bb00ce2e0 2024-04-25T08:41:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.282+0000 7f7bb6868640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7bb00ce2e0 0x7f7bb00d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.282+0000 7f7bb6067640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7bb006c110 0x7f7bb006c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33492/0 (socket says 172.21.15.38:33492) 2024-04-25T08:41:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.282+0000 7f7bb6067640 1 -- 172.21.15.38:0/1846369682 learned_addr learned my addr 172.21.15.38:0/1846369682 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.282+0000 7f7bb6868640 1 -- 172.21.15.38:0/1846369682 >> v1:172.21.15.107:6789/0 conn(0x7f7bb015df50 legacy=0x7f7bb00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.282+0000 7f7bb6868640 1 -- 172.21.15.38:0/1846369682 >> v1:172.21.15.38:6789/0 conn(0x7f7bb006c110 legacy=0x7f7bb006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.283+0000 7f7bb6868640 1 -- 172.21.15.38:0/1846369682 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb00d2250 con 0x7f7bb00ce2e0 2024-04-25T08:41:46.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.283+0000 7f7bb6868640 1 --2- 172.21.15.38:0/1846369682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bb00ce2e0 0x7f7bb00d2a50 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80097a0 tx=0x7f7ba802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df230d506ff58024 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.283+0000 7f7bb5065640 1 -- 172.21.15.38:0/1846369682 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8037070 con 0x7f7bb00ce2e0 2024-04-25T08:41:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.283+0000 7f7bb5065640 1 -- 172.21.15.38:0/1846369682 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8032440 con 0x7f7bb00ce2e0 2024-04-25T08:41:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.283+0000 7f7bb5065640 1 -- 172.21.15.38:0/1846369682 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8032810 con 0x7f7bb00ce2e0 2024-04-25T08:41:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.284+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1846369682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bb00ce2e0 msgr2=0x7f7bb00d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.284+0000 7f7bb82cb640 1 --2- 172.21.15.38:0/1846369682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bb00ce2e0 0x7f7bb00d2a50 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80097a0 tx=0x7f7ba802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.284+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1846369682 shutdown_connections 2024-04-25T08:41:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.284+0000 7f7bb82cb640 1 --2- 172.21.15.38:0/1846369682 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bb00ce2e0 0x7f7bb00d2a50 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.284+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1846369682 >> 172.21.15.38:0/1846369682 conn(0x7f7bb00d3fb0 msgr2=0x7f7bb00d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.284+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1846369682 shutdown_connections 2024-04-25T08:41:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.284+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1846369682 wait complete. 2024-04-25T08:41:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.285+0000 7f7bb82cb640 1 Processor -- start 2024-04-25T08:41:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.285+0000 7f7bb82cb640 1 -- start start 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb82cb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bb006c110 0x7f7bb0148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb82cb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb00ce2e0 0x7f7bb0161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb82cb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bb015df50 0x7f7bb0161d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb82cb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb00d3110 con 0x7f7bb006c110 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb82cb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb00d2f90 con 0x7f7bb00ce2e0 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb82cb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb00d3290 con 0x7f7bb015df50 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb6067640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bb006c110 0x7f7bb0148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb6067640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bb006c110 0x7f7bb0148480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46458/0 (socket says 172.21.15.38:46458) 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb6067640 1 -- 172.21.15.38:0/1890379457 learned_addr learned my addr 172.21.15.38:0/1890379457 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb6868640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bb015df50 0x7f7bb0161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb5866640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb00ce2e0 0x7f7bb0161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb6067640 1 -- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bb015df50 msgr2=0x7f7bb0161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb6067640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bb015df50 0x7f7bb0161d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb6067640 1 -- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb00ce2e0 msgr2=0x7f7bb0161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb6067640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb00ce2e0 0x7f7bb0161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.286+0000 7f7bb6067640 1 -- 172.21.15.38:0/1890379457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb01613b0 con 0x7f7bb006c110 2024-04-25T08:41:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.287+0000 7f7bb6067640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bb006c110 0x7f7bb0148480 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00caa0 tx=0x7f7bac00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.287+0000 7f7ba6ffd640 1 -- 172.21.15.38:0/1890379457 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac017070 con 0x7f7bb006c110 2024-04-25T08:41:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.287+0000 7f7ba6ffd640 1 -- 172.21.15.38:0/1890379457 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bac00ac00 con 0x7f7bb006c110 2024-04-25T08:41:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.287+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1890379457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb01f6f40 con 0x7f7bb006c110 2024-04-25T08:41:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.287+0000 7f7ba6ffd640 1 -- 172.21.15.38:0/1890379457 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac012400 con 0x7f7bb006c110 2024-04-25T08:41:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.287+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1890379457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb01489c0 con 0x7f7bb006c110 2024-04-25T08:41:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.288+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1890379457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb0148c80 con 0x7f7bb006c110 2024-04-25T08:41:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.288+0000 7f7ba6ffd640 1 -- 172.21.15.38:0/1890379457 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7bac01a070 con 0x7f7bb006c110 2024-04-25T08:41:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.289+0000 7f7ba6ffd640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7b9406b4d0 0x7f7b9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.289+0000 7f7ba6ffd640 1 -- 172.21.15.38:0/1890379457 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7bac08a090 con 0x7f7bb006c110 2024-04-25T08:41:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.289+0000 7f7bb5866640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7b9406b4d0 0x7f7b9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.289+0000 7f7bb5866640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7b9406b4d0 0x7f7b9406d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00ca260 tx=0x7f7ba0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:46.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.293+0000 7f7ba6ffd640 1 -- 172.21.15.38:0/1890379457 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7bb0148c80 con 0x7f7bb006c110 2024-04-25T08:41:46.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.514+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1890379457 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bb0068e80 con 0x7f7bb006c110 2024-04-25T08:41:46.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.514+0000 7f7ba6ffd640 1 -- 172.21.15.38:0/1890379457 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f7bac058dc0 con 0x7f7bb006c110 2024-04-25T08:41:46.507 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:46.507 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:41:42.972814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9486":{"gid":9486,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":16,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:46.507 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 81 2024-04-25T08:41:46.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.516+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7b9406b4d0 msgr2=0x7f7b9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.516+0000 7f7bb82cb640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7b9406b4d0 0x7f7b9406d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00ca260 tx=0x7f7ba0009040 comp rx=0 tx=0).stop 2024-04-25T08:41:46.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.516+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1890379457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bb006c110 msgr2=0x7f7bb0148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.516+0000 7f7bb82cb640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bb006c110 0x7f7bb0148480 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00caa0 tx=0x7f7bac00cf60 comp rx=0 tx=0).stop 2024-04-25T08:41:46.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.516+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1890379457 shutdown_connections 2024-04-25T08:41:46.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.516+0000 7f7bb82cb640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7b9406b4d0 0x7f7b9406d990 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.516+0000 7f7bb82cb640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7bb015df50 0x7f7bb0161d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.516+0000 7f7bb82cb640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7bb00ce2e0 0x7f7bb0161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.516+0000 7f7bb82cb640 1 --2- 172.21.15.38:0/1890379457 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7bb006c110 0x7f7bb0148480 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.516+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1890379457 >> 172.21.15.38:0/1890379457 conn(0x7f7bb00d3fb0 msgr2=0x7f7bb00737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:46.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.517+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1890379457 shutdown_connections 2024-04-25T08:41:46.510 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.517+0000 7f7bb82cb640 1 -- 172.21.15.38:0/1890379457 wait complete. 2024-04-25T08:41:46.520 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T08:41:46.520 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:41:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.627+0000 7f2d074b3640 1 Processor -- start 2024-04-25T08:41:46.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.627+0000 7f2d074b3640 1 -- start start 2024-04-25T08:41:46.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.628+0000 7f2d074b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d00161700 0x7f2d000b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.628+0000 7f2d074b3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d00161e00 con 0x7f2d0015ddf0 2024-04-25T08:41:46.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.628+0000 7f2d074b3640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d001620e0 con 0x7f2d000b5710 2024-04-25T08:41:46.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.628+0000 7f2d074b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000b9930 con 0x7f2d00161700 2024-04-25T08:41:46.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.628+0000 7f2d04a4e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2d0015ddf0 0x7f2d000d2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33498/0 (socket says 172.21.15.38:33498) 2024-04-25T08:41:46.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.628+0000 7f2d04a4e640 1 -- 172.21.15.38:0/1816101775 learned_addr learned my addr 172.21.15.38:0/1816101775 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:46.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.628+0000 7f2d05a50640 1 --2- 172.21.15.38:0/1816101775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d00161700 0x7f2d000b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.628+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391650476 0 0) 0x7f2d00161e00 con 0x7f2d0015ddf0 2024-04-25T08:41:46.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ce8003520 con 0x7f2d0015ddf0 2024-04-25T08:41:46.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94954801 0 0) 0x7f2d001620e0 con 0x7f2d000b5710 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d00161e00 con 0x7f2d000b5710 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 751727388 0 0) 0x7f2ce8003520 con 0x7f2d0015ddf0 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d00161700 msgr2=0x7f2d000b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 --2- 172.21.15.38:0/1816101775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d00161700 0x7f2d000b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 >> v1:172.21.15.107:6789/0 conn(0x7f2d000b5710 legacy=0x7f2d000b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d001623c0 con 0x7f2d0015ddf0 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f2cf4001670 con 0x7f2d0015ddf0 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 >> v1:172.21.15.38:6789/0 conn(0x7f2d0015ddf0 legacy=0x7f2d000d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 --2- 172.21.15.38:0/1816101775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ce8003b70 0x7f2ce8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 --2- 172.21.15.38:0/1816101775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ce8004800 0x7f2ce8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 --2- 172.21.15.38:0/1816101775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d00161700 0x7f2ce80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce800ba70 con 0x7f2ce8004800 2024-04-25T08:41:46.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce800bc20 con 0x7f2ce8003b70 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce800be20 con 0x7f2d00161700 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2d04a4e640 1 --2- 172.21.15.38:0/1816101775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ce8004800 0x7f2ce8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2d0524f640 1 --2- 172.21.15.38:0/1816101775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ce8003b70 0x7f2ce8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2d05a50640 1 --2- 172.21.15.38:0/1816101775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d00161700 0x7f2ce80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2d04a4e640 1 -- 172.21.15.38:0/1816101775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d00161700 msgr2=0x7f2ce80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2d04a4e640 1 --2- 172.21.15.38:0/1816101775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d00161700 0x7f2ce80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2d04a4e640 1 -- 172.21.15.38:0/1816101775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ce8003b70 msgr2=0x7f2ce8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2d04a4e640 1 --2- 172.21.15.38:0/1816101775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ce8003b70 0x7f2ce8003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.629+0000 7f2d04a4e640 1 -- 172.21.15.38:0/1816101775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce8003520 con 0x7f2ce8004800 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.630+0000 7f2d04a4e640 1 --2- 172.21.15.38:0/1816101775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ce8004800 0x7f2ce8004c80 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f2d0015e1d0 tx=0x7f2ce8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac71846cef96cc95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.630+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4007c00 con 0x7f2ce8004800 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.630+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf4005af0 con 0x7f2ce8004800 2024-04-25T08:41:46.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.630+0000 7f2cfffff640 1 -- 172.21.15.38:0/1816101775 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4007070 con 0x7f2ce8004800 2024-04-25T08:41:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.631+0000 7f2d074b3640 1 -- 172.21.15.38:0/1816101775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ce8004800 msgr2=0x7f2ce8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.631+0000 7f2d074b3640 1 --2- 172.21.15.38:0/1816101775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2ce8004800 0x7f2ce8004c80 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f2d0015e1d0 tx=0x7f2ce8005830 comp rx=0 tx=0).stop 2024-04-25T08:41:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.631+0000 7f2d05a50640 1 -- 172.21.15.38:0/1816101775 reap_dead start 2024-04-25T08:41:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.631+0000 7f2d074b3640 1 -- 172.21.15.38:0/1816101775 shutdown_connections 2024-04-25T08:41:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.631+0000 7f2d074b3640 1 -- 172.21.15.38:0/1816101775 >> 172.21.15.38:0/1816101775 conn(0x7f2d000d07c0 msgr2=0x7f2d000d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.631+0000 7f2d074b3640 1 -- 172.21.15.38:0/1816101775 shutdown_connections 2024-04-25T08:41:46.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.631+0000 7f2d074b3640 1 -- 172.21.15.38:0/1816101775 wait complete. 2024-04-25T08:41:46.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.632+0000 7f2d074b3640 1 Processor -- start 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.632+0000 7f2d074b3640 1 -- start start 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d074b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ce8004800 0x7f2d000abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d074b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d001f6840 0x7f2d00160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d074b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d001f3a80 0x7f2d000aa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d074b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d00161bb0 con 0x7f2d001f3a80 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d074b3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000b9640 con 0x7f2ce8004800 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d074b3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000d2a50 con 0x7f2d001f6840 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d05a50640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d001f3a80 0x7f2d000aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d05a50640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d001f3a80 0x7f2d000aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46474/0 (socket says 172.21.15.38:46474) 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d05a50640 1 -- 172.21.15.38:0/3277464200 learned_addr learned my addr 172.21.15.38:0/3277464200 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d0524f640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ce8004800 0x7f2d000abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d05a50640 1 -- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d001f6840 msgr2=0x7f2d00160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d04a4e640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d001f6840 0x7f2d00160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d05a50640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d001f6840 0x7f2d00160450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d05a50640 1 -- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ce8004800 msgr2=0x7f2d000abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d05a50640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ce8004800 0x7f2d000abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d05a50640 1 -- 172.21.15.38:0/3277464200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d001f7090 con 0x7f2d001f3a80 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d05a50640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d001f3a80 0x7f2d000aa300 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f2d000b6820 tx=0x7f2cf000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2cfdffb640 1 -- 172.21.15.38:0/3277464200 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0015430 con 0x7f2d001f3a80 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2cfdffb640 1 -- 172.21.15.38:0/3277464200 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf001d070 con 0x7f2d001f3a80 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.633+0000 7f2d074b3640 1 -- 172.21.15.38:0/3277464200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d001f7370 con 0x7f2d001f3a80 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.634+0000 7f2cfdffb640 1 -- 172.21.15.38:0/3277464200 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0015810 con 0x7f2d001f3a80 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.634+0000 7f2d074b3640 1 -- 172.21.15.38:0/3277464200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d001f3560 con 0x7f2d001f3a80 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.635+0000 7f2d074b3640 1 -- 172.21.15.38:0/3277464200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d001f3760 con 0x7f2d001f3a80 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.635+0000 7f2cfdffb640 1 -- 172.21.15.38:0/3277464200 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2cf00159b0 con 0x7f2d001f3a80 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.635+0000 7f2cfdffb640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2cec06b500 0x7f2cec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.636+0000 7f2cfdffb640 1 -- 172.21.15.38:0/3277464200 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2cf008acc0 con 0x7f2d001f3a80 2024-04-25T08:41:46.631 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.636+0000 7f2d0524f640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2cec06b500 0x7f2cec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:46.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.636+0000 7f2d0524f640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2cec06b500 0x7f2cec06d9c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8000c00 tx=0x7f2cf8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:46.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.640+0000 7f2cfdffb640 1 -- 172.21.15.38:0/3277464200 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d001f3760 con 0x7f2d001f3a80 2024-04-25T08:41:46.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.895+0000 7f2d074b3640 1 -- 172.21.15.38:0/3277464200 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2d00068e80 con 0x7f2d001f3a80 2024-04-25T08:41:46.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.895+0000 7f2cfdffb640 1 -- 172.21.15.38:0/3277464200 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1442 (secure 0 0 0) 0x7f2cf0010040 con 0x7f2d001f3a80 2024-04-25T08:41:46.888 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:46.888 INFO:teuthology.orchestra.run.smithi038.stdout:{"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":635,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":133,"num_osds":8,"num_up_osds":8,"osd_up_since":1714033882,"num_in_osds":8,"osd_in_since":1714033873,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077727859,"bytes_used":2671517696,"bytes_avail":770422595584,"bytes_total":773094113280,"read_bytes_sec":340794,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":81,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9486}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T08:41:41.347780+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:41:46.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.897+0000 7f2d074b3640 1 -- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2cec06b500 msgr2=0x7f2cec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.897+0000 7f2d074b3640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2cec06b500 0x7f2cec06d9c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8000c00 tx=0x7f2cf8009040 comp rx=0 tx=0).stop 2024-04-25T08:41:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.897+0000 7f2d074b3640 1 -- 172.21.15.38:0/3277464200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d001f3a80 msgr2=0x7f2d000aa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.897+0000 7f2d074b3640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d001f3a80 0x7f2d000aa300 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f2d000b6820 tx=0x7f2cf000f580 comp rx=0 tx=0).stop 2024-04-25T08:41:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.898+0000 7f2d074b3640 1 -- 172.21.15.38:0/3277464200 shutdown_connections 2024-04-25T08:41:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.898+0000 7f2d074b3640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d001f3a80 0x7f2d000aa300 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.898+0000 7f2d074b3640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2cec06b500 0x7f2cec06d9c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.898+0000 7f2d074b3640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d001f6840 0x7f2d00160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.898+0000 7f2d074b3640 1 --2- 172.21.15.38:0/3277464200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2ce8004800 0x7f2d000abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.898+0000 7f2d074b3640 1 -- 172.21.15.38:0/3277464200 >> 172.21.15.38:0/3277464200 conn(0x7f2d000d07c0 msgr2=0x7f2d000d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.898+0000 7f2d074b3640 1 -- 172.21.15.38:0/3277464200 shutdown_connections 2024-04-25T08:41:46.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:46.898+0000 7f2d074b3640 1 -- 172.21.15.38:0/3277464200 wait complete. 2024-04-25T08:41:46.901 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T08:41:46.901 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:41:46.901 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:41:46.945 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:41:47.004 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:41:47.004 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:41:47.004 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:41:47.020 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:41:47.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.168+0000 7f0f9bf4f640 1 Processor -- start 2024-04-25T08:41:47.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.168+0000 7f0f9bf4f640 1 -- start start 2024-04-25T08:41:47.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.169+0000 7f0f9bf4f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f94147700 0x7f0f940b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.169+0000 7f0f9bf4f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0f9406d520 0x7f0f940b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.169+0000 7f0f9bf4f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f9406e720 con 0x7f0f940d04f0 2024-04-25T08:41:47.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.169+0000 7f0f9bf4f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f9406ea00 con 0x7f0f9406d520 2024-04-25T08:41:47.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.169+0000 7f0f9bf4f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9406ece0 con 0x7f0f94147700 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.169+0000 7f0f9a4ec640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f0f940d04f0 0x7f0f940714c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33506/0 (socket says 172.21.15.38:33506) 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.169+0000 7f0f9a4ec640 1 -- 172.21.15.38:0/986685701 learned_addr learned my addr 172.21.15.38:0/986685701 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.169+0000 7f0f99ceb640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f94147700 0x7f0f940b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.169+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270889274 0 0) 0x7f0f9406e720 con 0x7f0f940d04f0 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.169+0000 7f0f994ea640 1 --2- 172.21.15.38:0/986685701 >> v2:172.21.15.107:3300/0 conn(0x7f0f9406d520 0x7f0f940b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f7c003520 con 0x7f0f940d04f0 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 490376659 0 0) 0x7f0f7c003520 con 0x7f0f940d04f0 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f94147700 msgr2=0x7f0f940b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f94147700 0x7f0f940b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 >> v2:172.21.15.107:3300/0 conn(0x7f0f9406d520 msgr2=0x7f0f940b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 --2- 172.21.15.38:0/986685701 >> v2:172.21.15.107:3300/0 conn(0x7f0f9406d520 0x7f0f940b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9406efc0 con 0x7f0f940d04f0 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f0f840019f0 con 0x7f0f940d04f0 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 >> v1:172.21.15.38:6789/0 conn(0x7f0f940d04f0 legacy=0x7f0f940714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f94147700 0x7f0f7c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c003f30 0x7f0f7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f7c004d40 0x7f0f7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9406ea00 con 0x7f0f7c003f30 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9406ece0 con 0x7f0f7c004d40 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c00b9c0 con 0x7f0f94147700 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f994ea640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c003f30 0x7f0f7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f99ceb640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f94147700 0x7f0f7c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f9a4ec640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f7c004d40 0x7f0f7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f994ea640 1 -- 172.21.15.38:0/986685701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f94147700 msgr2=0x7f0f7c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f994ea640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f94147700 0x7f0f7c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f994ea640 1 -- 172.21.15.38:0/986685701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f7c004d40 msgr2=0x7f0f7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f994ea640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f7c004d40 0x7f0f7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f994ea640 1 -- 172.21.15.38:0/986685701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c003520 con 0x7f0f7c003f30 2024-04-25T08:41:47.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.170+0000 7f0f9a4ec640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f7c004d40 0x7f0f7c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:47.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.171+0000 7f0f994ea640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c003f30 0x7f0f7c004800 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009ac0 tx=0x7f0f7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf6d192589958d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:47.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.171+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88036070 con 0x7f0f7c003f30 2024-04-25T08:41:47.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.171+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f88031400 con 0x7f0f7c003f30 2024-04-25T08:41:47.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.171+0000 7f0f98ce9640 1 -- 172.21.15.38:0/986685701 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88009690 con 0x7f0f7c003f30 2024-04-25T08:41:47.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.172+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/986685701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c003f30 msgr2=0x7f0f7c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.172+0000 7f0f9bf4f640 1 --2- 172.21.15.38:0/986685701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c003f30 0x7f0f7c004800 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f0f88009ac0 tx=0x7f0f7c005830 comp rx=0 tx=0).stop 2024-04-25T08:41:47.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.172+0000 7f0f9a4ec640 1 -- 172.21.15.38:0/986685701 reap_dead start 2024-04-25T08:41:47.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.172+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/986685701 shutdown_connections 2024-04-25T08:41:47.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.172+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/986685701 >> 172.21.15.38:0/986685701 conn(0x7f0f940cf970 msgr2=0x7f0f940cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:47.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.172+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/986685701 shutdown_connections 2024-04-25T08:41:47.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.172+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/986685701 wait complete. 2024-04-25T08:41:47.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.173+0000 7f0f9bf4f640 1 Processor -- start 2024-04-25T08:41:47.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.173+0000 7f0f9bf4f640 1 -- start start 2024-04-25T08:41:47.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f9bf4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c003f30 0x7f0f940cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f9bf4f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f940c7300 0x7f0f940c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f9bf4f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f9413fd80 0x7f0f94140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f9bf4f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f940adf10 con 0x7f0f7c003f30 2024-04-25T08:41:47.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f9bf4f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9406e410 con 0x7f0f940c7300 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f9bf4f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f940ae090 con 0x7f0f9413fd80 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f99ceb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c003f30 0x7f0f940cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f99ceb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c003f30 0x7f0f940cedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46480/0 (socket says 172.21.15.38:46480) 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f99ceb640 1 -- 172.21.15.38:0/925270625 learned_addr learned my addr 172.21.15.38:0/925270625 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f9a4ec640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f9413fd80 0x7f0f94140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f994ea640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f940c7300 0x7f0f940c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f99ceb640 1 -- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f9413fd80 msgr2=0x7f0f94140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f99ceb640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f9413fd80 0x7f0f94140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f99ceb640 1 -- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f940c7300 msgr2=0x7f0f940c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f99ceb640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f940c7300 0x7f0f940c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f99ceb640 1 -- 172.21.15.38:0/925270625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f94140990 con 0x7f0f7c003f30 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f9a4ec640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f9413fd80 0x7f0f94140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f994ea640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f940c7300 0x7f0f940c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:47.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f99ceb640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c003f30 0x7f0f940cedf0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c011600 tx=0x7f0f8c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:47.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.174+0000 7f0f927fc640 1 -- 172.21.15.38:0/925270625 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00f920 con 0x7f0f7c003f30 2024-04-25T08:41:47.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.175+0000 7f0f927fc640 1 -- 172.21.15.38:0/925270625 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8c00fac0 con 0x7f0f7c003f30 2024-04-25T08:41:47.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.175+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/925270625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f94140c70 con 0x7f0f7c003f30 2024-04-25T08:41:47.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.175+0000 7f0f927fc640 1 -- 172.21.15.38:0/925270625 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c00fe70 con 0x7f0f7c003f30 2024-04-25T08:41:47.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.175+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/925270625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f94141230 con 0x7f0f7c003f30 2024-04-25T08:41:47.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.176+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/925270625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f94068e80 con 0x7f0f7c003f30 2024-04-25T08:41:47.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.176+0000 7f0f927fc640 1 -- 172.21.15.38:0/925270625 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f8c015460 con 0x7f0f7c003f30 2024-04-25T08:41:47.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.176+0000 7f0f927fc640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f8006b4d0 0x7f0f8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.177+0000 7f0f927fc640 1 -- 172.21.15.38:0/925270625 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0f8c08ac50 con 0x7f0f7c003f30 2024-04-25T08:41:47.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.177+0000 7f0f994ea640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f8006b4d0 0x7f0f8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.177+0000 7f0f994ea640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f8006b4d0 0x7f0f8006d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0f88033000 tx=0x7f0f88008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:47.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.181+0000 7f0f927fc640 1 -- 172.21.15.38:0/925270625 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f8c090020 con 0x7f0f7c003f30 2024-04-25T08:41:47.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.361+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/925270625 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f941457b0 con 0x7f0f7c003f30 2024-04-25T08:41:47.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.362+0000 7f0f927fc640 1 -- 172.21.15.38:0/925270625 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17017 (secure 0 0 0) 0x7f0f8c019030 con 0x7f0f7c003f30 2024-04-25T08:41:47.355 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:47.355 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":133,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:41:40.950430+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:40:52.007975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:40:52.707859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/1312063561":"2024-04-26T08:41:40.950341+0000","172.21.15.38:6834/1312063561":"2024-04-26T08:41:40.950341+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:41:47.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.364+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f8006b4d0 msgr2=0x7f0f8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.364+0000 7f0f9bf4f640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f8006b4d0 0x7f0f8006d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f0f88033000 tx=0x7f0f88008030 comp rx=0 tx=0).stop 2024-04-25T08:41:47.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.364+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/925270625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c003f30 msgr2=0x7f0f940cedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.364+0000 7f0f9bf4f640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c003f30 0x7f0f940cedf0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c011600 tx=0x7f0f8c011ac0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.364+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/925270625 shutdown_connections 2024-04-25T08:41:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.364+0000 7f0f9bf4f640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0f8006b4d0 0x7f0f8006d990 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.364+0000 7f0f9bf4f640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0f9413fd80 0x7f0f94140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.364+0000 7f0f9bf4f640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f940c7300 0x7f0f940c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.364+0000 7f0f9bf4f640 1 --2- 172.21.15.38:0/925270625 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0f7c003f30 0x7f0f940cedf0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.365+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/925270625 >> 172.21.15.38:0/925270625 conn(0x7f0f940cf970 msgr2=0x7f0f9406d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.365+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/925270625 shutdown_connections 2024-04-25T08:41:47.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.365+0000 7f0f9bf4f640 1 -- 172.21.15.38:0/925270625 wait complete. 2024-04-25T08:41:47.368 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:40:52.007975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:40:52.707859+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:41:47.368 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:41:47.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.479+0000 7ff0ebf89640 1 Processor -- start 2024-04-25T08:41:47.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.479+0000 7ff0ebf89640 1 -- start start 2024-04-25T08:41:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0ebf89640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7ff0e40d0320 0x7ff0e40d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0ebf89640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff0e40d0700 0x7ff0e40d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0ebf89640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff0e40d1e90 0x7ff0e40b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0ebf89640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e40d2330 con 0x7ff0e40d0700 2024-04-25T08:41:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0ebf89640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e40d2610 con 0x7ff0e40d0320 2024-04-25T08:41:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0ebf89640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40d28f0 con 0x7ff0e40d1e90 2024-04-25T08:41:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0e9524640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff0e40d0700 0x7ff0e40d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0e9524640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff0e40d0700 0x7ff0e40d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46494/0 (socket says 172.21.15.38:46494) 2024-04-25T08:41:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0e9524640 1 -- 172.21.15.38:0/188410730 learned_addr learned my addr 172.21.15.38:0/188410730 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0ea526640 1 --2- 172.21.15.38:0/188410730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff0e40d1e90 0x7ff0e40b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0e9d25640 1 --2- 172.21.15.38:0/188410730 >> v2:172.21.15.107:3300/0 conn(0x7ff0e40d0320 0x7ff0e40d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0e9524640 1 -- 172.21.15.38:0/188410730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff0e40d1e90 msgr2=0x7ff0e40b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0e9524640 1 --2- 172.21.15.38:0/188410730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff0e40d1e90 0x7ff0e40b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0e9524640 1 -- 172.21.15.38:0/188410730 >> v2:172.21.15.107:3300/0 conn(0x7ff0e40d0320 msgr2=0x7ff0e40d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0e9524640 1 --2- 172.21.15.38:0/188410730 >> v2:172.21.15.107:3300/0 conn(0x7ff0e40d0320 0x7ff0e40d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0e9524640 1 -- 172.21.15.38:0/188410730 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40d2bd0 con 0x7ff0e40d0700 2024-04-25T08:41:47.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.480+0000 7ff0ea526640 1 --2- 172.21.15.38:0/188410730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff0e40d1e90 0x7ff0e40b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.481+0000 7ff0e9524640 1 --2- 172.21.15.38:0/188410730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e40d0700 0x7ff0e40d1950 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4069100 tx=0x7ff0d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af04fc6c24c4eff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.481+0000 7ff0e8d23640 1 -- 172.21.15.38:0/188410730 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4037070 con 0x7ff0e40d0700 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.481+0000 7ff0e8d23640 1 -- 172.21.15.38:0/188410730 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d4032440 con 0x7ff0e40d0700 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.481+0000 7ff0e8d23640 1 -- 172.21.15.38:0/188410730 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4032810 con 0x7ff0e40d0700 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.482+0000 7ff0ebf89640 1 -- 172.21.15.38:0/188410730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e40d0700 msgr2=0x7ff0e40d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.482+0000 7ff0ebf89640 1 --2- 172.21.15.38:0/188410730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e40d0700 0x7ff0e40d1950 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4069100 tx=0x7ff0d402a270 comp rx=0 tx=0).stop 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.482+0000 7ff0ebf89640 1 -- 172.21.15.38:0/188410730 shutdown_connections 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.482+0000 7ff0ebf89640 1 --2- 172.21.15.38:0/188410730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff0e40d1e90 0x7ff0e40b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.482+0000 7ff0ebf89640 1 --2- 172.21.15.38:0/188410730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e40d0700 0x7ff0e40d1950 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.482+0000 7ff0ebf89640 1 --2- 172.21.15.38:0/188410730 >> v2:172.21.15.107:3300/0 conn(0x7ff0e40d0320 0x7ff0e40d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.482+0000 7ff0ebf89640 1 -- 172.21.15.38:0/188410730 >> 172.21.15.38:0/188410730 conn(0x7ff0e40668f0 msgr2=0x7ff0e4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.482+0000 7ff0ebf89640 1 -- 172.21.15.38:0/188410730 shutdown_connections 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.482+0000 7ff0ebf89640 1 -- 172.21.15.38:0/188410730 wait complete. 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.483+0000 7ff0ebf89640 1 Processor -- start 2024-04-25T08:41:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.483+0000 7ff0ebf89640 1 -- start start 2024-04-25T08:41:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ebf89640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff0e40d0320 0x7ff0e414bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ebf89640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0e40d0700 0x7ff0e40b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ebf89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e40d1e90 0x7ff0e41591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ebf89640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40d0b40 con 0x7ff0e40d1e90 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ebf89640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4161210 con 0x7ff0e40d0700 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ebf89640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40b5490 con 0x7ff0e40d0320 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ea526640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e40d1e90 0x7ff0e41591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ea526640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e40d1e90 0x7ff0e41591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46498/0 (socket says 172.21.15.38:46498) 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ea526640 1 -- 172.21.15.38:0/1649647346 learned_addr learned my addr 172.21.15.38:0/1649647346 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0e9524640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0e40d0700 0x7ff0e40b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0e9d25640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff0e40d0320 0x7ff0e414bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ea526640 1 -- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff0e40d0320 msgr2=0x7ff0e414bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ea526640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff0e40d0320 0x7ff0e414bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ea526640 1 -- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0e40d0700 msgr2=0x7ff0e40b3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ea526640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0e40d0700 0x7ff0e40b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ea526640 1 -- 172.21.15.38:0/1649647346 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40b43f0 con 0x7ff0e40d1e90 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.484+0000 7ff0ea526640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e40d1e90 0x7ff0e41591a0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc007dc0 tx=0x7ff0dc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.485+0000 7ff0da7fc640 1 -- 172.21.15.38:0/1649647346 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc01a070 con 0x7ff0e40d1e90 2024-04-25T08:41:47.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.485+0000 7ff0da7fc640 1 -- 172.21.15.38:0/1649647346 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc003990 con 0x7ff0e40d1e90 2024-04-25T08:41:47.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.485+0000 7ff0ebf89640 1 -- 172.21.15.38:0/1649647346 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e40b46d0 con 0x7ff0e40d1e90 2024-04-25T08:41:47.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.485+0000 7ff0da7fc640 1 -- 172.21.15.38:0/1649647346 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc003d80 con 0x7ff0e40d1e90 2024-04-25T08:41:47.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.485+0000 7ff0ebf89640 1 -- 172.21.15.38:0/1649647346 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e40b4c90 con 0x7ff0e40d1e90 2024-04-25T08:41:47.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.486+0000 7ff0ebf89640 1 -- 172.21.15.38:0/1649647346 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e4068e80 con 0x7ff0e40d1e90 2024-04-25T08:41:47.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.486+0000 7ff0da7fc640 1 -- 172.21.15.38:0/1649647346 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0dc01d070 con 0x7ff0e40d1e90 2024-04-25T08:41:47.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.487+0000 7ff0da7fc640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff0c806b4d0 0x7ff0c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.487+0000 7ff0da7fc640 1 -- 172.21.15.38:0/1649647346 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff0dc08a600 con 0x7ff0e40d1e90 2024-04-25T08:41:47.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.487+0000 7ff0e9d25640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff0c806b4d0 0x7ff0c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.487+0000 7ff0e9d25640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff0c806b4d0 0x7ff0c806d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0006fd0 tx=0x7ff0e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:47.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.491+0000 7ff0da7fc640 1 -- 172.21.15.38:0/1649647346 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0dc010070 con 0x7ff0e40d1e90 2024-04-25T08:41:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.670+0000 7ff0ebf89640 1 -- 172.21.15.38:0/1649647346 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0e40b3050 con 0x7ff0e40d1e90 2024-04-25T08:41:47.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.671+0000 7ff0da7fc640 1 -- 172.21.15.38:0/1649647346 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0dc0592a0 con 0x7ff0e40d1e90 2024-04-25T08:41:47.663 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:41:47.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.673+0000 7ff0ebf89640 1 -- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff0c806b4d0 msgr2=0x7ff0c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.673+0000 7ff0ebf89640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff0c806b4d0 0x7ff0c806d990 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0006fd0 tx=0x7ff0e0008040 comp rx=0 tx=0).stop 2024-04-25T08:41:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.673+0000 7ff0ebf89640 1 -- 172.21.15.38:0/1649647346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e40d1e90 msgr2=0x7ff0e41591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.673+0000 7ff0ebf89640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e40d1e90 0x7ff0e41591a0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc007dc0 tx=0x7ff0dc00d4a0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.673+0000 7ff0ebf89640 1 -- 172.21.15.38:0/1649647346 shutdown_connections 2024-04-25T08:41:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.673+0000 7ff0ebf89640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0e40d1e90 0x7ff0e41591a0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.673+0000 7ff0ebf89640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff0e40d0700 0x7ff0e40b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.673+0000 7ff0ebf89640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff0c806b4d0 0x7ff0c806d990 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.673+0000 7ff0ebf89640 1 --2- 172.21.15.38:0/1649647346 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff0e40d0320 0x7ff0e414bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.673+0000 7ff0ebf89640 1 -- 172.21.15.38:0/1649647346 >> 172.21.15.38:0/1649647346 conn(0x7ff0e40668f0 msgr2=0x7ff0e40b7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.673+0000 7ff0ebf89640 1 -- 172.21.15.38:0/1649647346 shutdown_connections 2024-04-25T08:41:47.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.673+0000 7ff0ebf89640 1 -- 172.21.15.38:0/1649647346 wait complete. 2024-04-25T08:41:47.676 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:41:47.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.787+0000 7f07e3b6a640 1 Processor -- start 2024-04-25T08:41:47.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.788+0000 7f07e3b6a640 1 -- start start 2024-04-25T08:41:47.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.788+0000 7f07e3b6a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07dc15df30 0x7f07dc09a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.788+0000 7f07e3b6a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f07dc162390 0x7f07dc099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.788+0000 7f07e3b6a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07dc15f300 con 0x7f07dc099fb0 2024-04-25T08:41:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.788+0000 7f07e3b6a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07dc15e310 con 0x7f07dc162390 2024-04-25T08:41:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.788+0000 7f07e3b6a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc09aeb0 con 0x7f07dc15df30 2024-04-25T08:41:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.789+0000 7f07e2107640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f07dc099fb0 0x7f07dc06bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33508/0 (socket says 172.21.15.38:33508) 2024-04-25T08:41:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.789+0000 7f07e2107640 1 -- 172.21.15.38:0/532616600 learned_addr learned my addr 172.21.15.38:0/532616600 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.789+0000 7f07e1906640 1 --2- 172.21.15.38:0/532616600 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07dc15df30 0x7f07dc09a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.789+0000 7f07e1105640 1 --2- 172.21.15.38:0/532616600 >> v2:172.21.15.107:3300/0 conn(0x7f07dc162390 0x7f07dc099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.789+0000 7f07e0904640 1 -- 172.21.15.38:0/532616600 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178430296 0 0) 0x7f07dc15f300 con 0x7f07dc099fb0 2024-04-25T08:41:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.789+0000 7f07e0904640 1 -- 172.21.15.38:0/532616600 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07c4003520 con 0x7f07dc099fb0 2024-04-25T08:41:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.789+0000 7f07e0904640 1 -- 172.21.15.38:0/532616600 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1839951418 0 0) 0x7f07c4003520 con 0x7f07dc099fb0 2024-04-25T08:41:47.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.789+0000 7f07e1906640 1 -- 172.21.15.38:0/532616600 >> v2:172.21.15.107:3300/0 conn(0x7f07dc162390 msgr2=0x7f07dc099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:47.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.789+0000 7f07e1906640 1 --2- 172.21.15.38:0/532616600 >> v2:172.21.15.107:3300/0 conn(0x7f07dc162390 0x7f07dc099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.789+0000 7f07e1906640 1 -- 172.21.15.38:0/532616600 >> v1:172.21.15.38:6789/0 conn(0x7f07dc099fb0 legacy=0x7f07dc06bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.789+0000 7f07e1906640 1 -- 172.21.15.38:0/532616600 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc1627f0 con 0x7f07dc15df30 2024-04-25T08:41:47.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.790+0000 7f07e1906640 1 --2- 172.21.15.38:0/532616600 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07dc15df30 0x7f07dc09a7b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07d40097a0 tx=0x7f07d402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63847cba90212bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:47.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.790+0000 7f07e0904640 1 -- 172.21.15.38:0/532616600 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4037070 con 0x7f07dc15df30 2024-04-25T08:41:47.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.790+0000 7f07e0904640 1 -- 172.21.15.38:0/532616600 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d402aba0 con 0x7f07dc15df30 2024-04-25T08:41:47.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.790+0000 7f07e0904640 1 -- 172.21.15.38:0/532616600 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0019f0 con 0x7f07dc15df30 2024-04-25T08:41:47.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.791+0000 7f07e3b6a640 1 -- 172.21.15.38:0/532616600 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07dc15df30 msgr2=0x7f07dc09a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.791+0000 7f07e3b6a640 1 --2- 172.21.15.38:0/532616600 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07dc15df30 0x7f07dc09a7b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f07d40097a0 tx=0x7f07d402a110 comp rx=0 tx=0).stop 2024-04-25T08:41:47.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.791+0000 7f07e3b6a640 1 -- 172.21.15.38:0/532616600 shutdown_connections 2024-04-25T08:41:47.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.791+0000 7f07e3b6a640 1 --2- 172.21.15.38:0/532616600 >> v2:172.21.15.107:3300/0 conn(0x7f07dc162390 0x7f07dc099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.791+0000 7f07e3b6a640 1 --2- 172.21.15.38:0/532616600 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07dc15df30 0x7f07dc09a7b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.791+0000 7f07e3b6a640 1 -- 172.21.15.38:0/532616600 >> 172.21.15.38:0/532616600 conn(0x7f07dc0b3110 msgr2=0x7f07dc0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:47.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.791+0000 7f07e3b6a640 1 -- 172.21.15.38:0/532616600 shutdown_connections 2024-04-25T08:41:47.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.791+0000 7f07e3b6a640 1 -- 172.21.15.38:0/532616600 wait complete. 2024-04-25T08:41:47.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.792+0000 7f07e3b6a640 1 Processor -- start 2024-04-25T08:41:47.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.792+0000 7f07e3b6a640 1 -- start start 2024-04-25T08:41:47.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e3b6a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07dc099fb0 0x7f07dc153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e3b6a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc15df30 0x7f07dc159640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e3b6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc162390 0x7f07dc0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e3b6a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc15e820 con 0x7f07dc162390 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e3b6a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc15ee70 con 0x7f07dc15df30 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e3b6a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc15eb00 con 0x7f07dc099fb0 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e2107640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc162390 0x7f07dc0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e2107640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc162390 0x7f07dc0ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46506/0 (socket says 172.21.15.38:46506) 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e2107640 1 -- 172.21.15.38:0/2444292719 learned_addr learned my addr 172.21.15.38:0/2444292719 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e1105640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc15df30 0x7f07dc159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e1906640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07dc099fb0 0x7f07dc153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e2107640 1 -- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07dc099fb0 msgr2=0x7f07dc153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e2107640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07dc099fb0 0x7f07dc153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e2107640 1 -- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc15df30 msgr2=0x7f07dc159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e2107640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc15df30 0x7f07dc159640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e2107640 1 -- 172.21.15.38:0/2444292719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc158850 con 0x7f07dc162390 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e1105640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc15df30 0x7f07dc159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e2107640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc162390 0x7f07dc0ccca0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00cc30 tx=0x7f07cc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07da7fc640 1 -- 172.21.15.38:0/2444292719 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc017070 con 0x7f07dc162390 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07da7fc640 1 -- 172.21.15.38:0/2444292719 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc00ce40 con 0x7f07dc162390 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e3b6a640 1 -- 172.21.15.38:0/2444292719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc158b30 con 0x7f07dc162390 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07da7fc640 1 -- 172.21.15.38:0/2444292719 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc012600 con 0x7f07dc162390 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.793+0000 7f07e3b6a640 1 -- 172.21.15.38:0/2444292719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc0c8b70 con 0x7f07dc162390 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.794+0000 7f07e3b6a640 1 -- 172.21.15.38:0/2444292719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc158d00 con 0x7f07dc162390 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.795+0000 7f07da7fc640 1 -- 172.21.15.38:0/2444292719 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07cc01a070 con 0x7f07dc162390 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.795+0000 7f07da7fc640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f07c806b4d0 0x7f07c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.795+0000 7f07da7fc640 1 -- 172.21.15.38:0/2444292719 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f07cc08a170 con 0x7f07dc162390 2024-04-25T08:41:47.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.795+0000 7f07e1906640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f07c806b4d0 0x7f07c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:47.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.796+0000 7f07e1906640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f07c806b4d0 0x7f07c806d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f07d4034000 tx=0x7f07d402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:47.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.799+0000 7f07da7fc640 1 -- 172.21.15.38:0/2444292719 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07cc0587a0 con 0x7f07dc162390 2024-04-25T08:41:47.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.982+0000 7f07e3b6a640 1 -- 172.21.15.38:0/2444292719 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07dc154110 con 0x7f07dc162390 2024-04-25T08:41:47.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.983+0000 7f07da7fc640 1 -- 172.21.15.38:0/2444292719 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v133) v1 ==== 104+0+11 (secure 0 0 0) 0x7f07cc05c840 con 0x7f07dc162390 2024-04-25T08:41:47.975 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:41:47.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.985+0000 7f07e3b6a640 1 -- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f07c806b4d0 msgr2=0x7f07c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.985+0000 7f07e3b6a640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f07c806b4d0 0x7f07c806d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f07d4034000 tx=0x7f07d402e000 comp rx=0 tx=0).stop 2024-04-25T08:41:47.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.985+0000 7f07e3b6a640 1 -- 172.21.15.38:0/2444292719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc162390 msgr2=0x7f07dc0ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:47.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.985+0000 7f07e3b6a640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc162390 0x7f07dc0ccca0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00cc30 tx=0x7f07cc00a430 comp rx=0 tx=0).stop 2024-04-25T08:41:47.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.985+0000 7f07e3b6a640 1 -- 172.21.15.38:0/2444292719 shutdown_connections 2024-04-25T08:41:47.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.985+0000 7f07e3b6a640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f07dc162390 0x7f07dc0ccca0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.985+0000 7f07e3b6a640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f07dc15df30 0x7f07dc159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.985+0000 7f07e3b6a640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f07c806b4d0 0x7f07c806d990 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.985+0000 7f07e3b6a640 1 --2- 172.21.15.38:0/2444292719 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f07dc099fb0 0x7f07dc153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:47.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.985+0000 7f07e3b6a640 1 -- 172.21.15.38:0/2444292719 >> 172.21.15.38:0/2444292719 conn(0x7f07dc0b3110 msgr2=0x7f07dc070580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:47.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.985+0000 7f07e3b6a640 1 -- 172.21.15.38:0/2444292719 shutdown_connections 2024-04-25T08:41:47.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:47.985+0000 7f07e3b6a640 1 -- 172.21.15.38:0/2444292719 wait complete. 2024-04-25T08:41:47.989 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:41:48.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.100+0000 7f4f79fc1640 1 Processor -- start 2024-04-25T08:41:48.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.100+0000 7f4f79fc1640 1 -- start start 2024-04-25T08:41:48.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.100+0000 7f4f79fc1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4f7415baf0 0x7f4f740cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:48.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.100+0000 7f4f79fc1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4f74072660 0x7f4f740acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:48.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.100+0000 7f4f79fc1640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f740b3250 con 0x7f4f740cfb70 2024-04-25T08:41:48.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.100+0000 7f4f79fc1640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f740731d0 con 0x7f4f74072660 2024-04-25T08:41:48.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.100+0000 7f4f79fc1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f74072c10 con 0x7f4f7415baf0 2024-04-25T08:41:48.094 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.100+0000 7f4f73fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4f74072660 0x7f4f740acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:48.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.101+0000 7f4f73fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4f74072660 0x7f4f740acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46522/0 (socket says 172.21.15.38:46522) 2024-04-25T08:41:48.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.101+0000 7f4f73fff640 1 -- 172.21.15.38:0/1589148712 learned_addr learned my addr 172.21.15.38:0/1589148712 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:48.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.101+0000 7f4f737fe640 1 --2- 172.21.15.38:0/1589148712 >> v2:172.21.15.107:3300/0 conn(0x7f4f7415baf0 0x7f4f740cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:48.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.101+0000 7f4f73fff640 1 -- 172.21.15.38:0/1589148712 >> v2:172.21.15.107:3300/0 conn(0x7f4f7415baf0 msgr2=0x7f4f740cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:48.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.101+0000 7f4f73fff640 1 --2- 172.21.15.38:0/1589148712 >> v2:172.21.15.107:3300/0 conn(0x7f4f7415baf0 0x7f4f740cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.101+0000 7f4f73fff640 1 -- 172.21.15.38:0/1589148712 >> v1:172.21.15.107:6789/0 conn(0x7f4f740cfb70 legacy=0x7f4f74071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:48.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.101+0000 7f4f73fff640 1 -- 172.21.15.38:0/1589148712 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f74072ef0 con 0x7f4f74072660 2024-04-25T08:41:48.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.101+0000 7f4f737fe640 1 --2- 172.21.15.38:0/1589148712 >> v2:172.21.15.107:3300/0 conn(0x7f4f7415baf0 0x7f4f740cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:48.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.101+0000 7f4f73fff640 1 --2- 172.21.15.38:0/1589148712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f74072660 0x7f4f740acb10 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f4f740d0440 tx=0x7f4f6c02a590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b09a06abdf2b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:48.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.102+0000 7f4f727fc640 1 -- 172.21.15.38:0/1589148712 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c036070 con 0x7f4f74072660 2024-04-25T08:41:48.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.102+0000 7f4f727fc640 1 -- 172.21.15.38:0/1589148712 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6c031600 con 0x7f4f74072660 2024-04-25T08:41:48.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.102+0000 7f4f727fc640 1 -- 172.21.15.38:0/1589148712 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c0319d0 con 0x7f4f74072660 2024-04-25T08:41:48.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.102+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1589148712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f74072660 msgr2=0x7f4f740acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:48.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.102+0000 7f4f79fc1640 1 --2- 172.21.15.38:0/1589148712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f74072660 0x7f4f740acb10 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f4f740d0440 tx=0x7f4f6c02a590 comp rx=0 tx=0).stop 2024-04-25T08:41:48.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.102+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1589148712 shutdown_connections 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.102+0000 7f4f79fc1640 1 --2- 172.21.15.38:0/1589148712 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f74072660 0x7f4f740acb10 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.102+0000 7f4f79fc1640 1 --2- 172.21.15.38:0/1589148712 >> v2:172.21.15.107:3300/0 conn(0x7f4f7415baf0 0x7f4f740cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.102+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1589148712 >> 172.21.15.38:0/1589148712 conn(0x7f4f7406adc0 msgr2=0x7f4f7406b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.102+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1589148712 shutdown_connections 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.102+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1589148712 wait complete. 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.103+0000 7f4f79fc1640 1 Processor -- start 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.104+0000 7f4f79fc1640 1 -- start start 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.104+0000 7f4f79fc1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f74072660 0x7f4f740bb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.104+0000 7f4f79fc1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f740cfb70 0x7f4f741f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.104+0000 7f4f79fc1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f7415baf0 0x7f4f74155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.104+0000 7f4f79fc1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740ad1d0 con 0x7f4f74072660 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.104+0000 7f4f79fc1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740ad050 con 0x7f4f740cfb70 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.104+0000 7f4f79fc1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f740ad350 con 0x7f4f7415baf0 2024-04-25T08:41:48.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.104+0000 7f4f737fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f74072660 0x7f4f740bb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f737fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f74072660 0x7f4f740bb6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46534/0 (socket says 172.21.15.38:46534) 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f737fe640 1 -- 172.21.15.38:0/1198544348 learned_addr learned my addr 172.21.15.38:0/1198544348 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f73fff640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f7415baf0 0x7f4f74155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f72ffd640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f740cfb70 0x7f4f741f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f737fe640 1 -- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f7415baf0 msgr2=0x7f4f74155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f737fe640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f7415baf0 0x7f4f74155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f737fe640 1 -- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f740cfb70 msgr2=0x7f4f741f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f737fe640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f740cfb70 0x7f4f741f4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f737fe640 1 -- 172.21.15.38:0/1198544348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f740bbf40 con 0x7f4f74072660 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f737fe640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f74072660 0x7f4f740bb6f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400c820 tx=0x7f4f6400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f5bfff640 1 -- 172.21.15.38:0/1198544348 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64015070 con 0x7f4f74072660 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f5bfff640 1 -- 172.21.15.38:0/1198544348 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f64007a30 con 0x7f4f74072660 2024-04-25T08:41:48.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1198544348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f740bc220 con 0x7f4f74072660 2024-04-25T08:41:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f5bfff640 1 -- 172.21.15.38:0/1198544348 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64007de0 con 0x7f4f74072660 2024-04-25T08:41:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.105+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1198544348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f74158080 con 0x7f4f74072660 2024-04-25T08:41:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.106+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1198544348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f74067420 con 0x7f4f74072660 2024-04-25T08:41:48.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.107+0000 7f4f5bfff640 1 -- 172.21.15.38:0/1198544348 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f64018070 con 0x7f4f74072660 2024-04-25T08:41:48.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.107+0000 7f4f5bfff640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f6006b4d0 0x7f4f6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:48.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.107+0000 7f4f5bfff640 1 -- 172.21.15.38:0/1198544348 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4f6408a280 con 0x7f4f74072660 2024-04-25T08:41:48.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.107+0000 7f4f72ffd640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f6006b4d0 0x7f4f6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:48.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.108+0000 7f4f72ffd640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f6006b4d0 0x7f4f6006d990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4f740c5860 tx=0x7f4f68009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:48.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.111+0000 7f4f5bfff640 1 -- 172.21.15.38:0/1198544348 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f74067420 con 0x7f4f74072660 2024-04-25T08:41:48.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.291+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1198544348 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4f74157e70 con 0x7f4f74072660 2024-04-25T08:41:48.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.291+0000 7f4f5bfff640 1 -- 172.21.15.38:0/1198544348 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v133) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4f64058fb0 con 0x7f4f74072660 2024-04-25T08:41:48.284 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:41:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.293+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f6006b4d0 msgr2=0x7f4f6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.293+0000 7f4f79fc1640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f6006b4d0 0x7f4f6006d990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f4f740c5860 tx=0x7f4f68009040 comp rx=0 tx=0).stop 2024-04-25T08:41:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.293+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1198544348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f74072660 msgr2=0x7f4f740bb6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.293+0000 7f4f79fc1640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f74072660 0x7f4f740bb6f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400c820 tx=0x7f4f6400cce0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.293+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1198544348 shutdown_connections 2024-04-25T08:41:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.293+0000 7f4f79fc1640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4f6006b4d0 0x7f4f6006d990 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.293+0000 7f4f79fc1640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4f7415baf0 0x7f4f74155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.293+0000 7f4f79fc1640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f740cfb70 0x7f4f741f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.293+0000 7f4f79fc1640 1 --2- 172.21.15.38:0/1198544348 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f74072660 0x7f4f740bb6f0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.293+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1198544348 >> 172.21.15.38:0/1198544348 conn(0x7f4f7406adc0 msgr2=0x7f4f740cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.294+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1198544348 shutdown_connections 2024-04-25T08:41:48.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.294+0000 7f4f79fc1640 1 -- 172.21.15.38:0/1198544348 wait complete. 2024-04-25T08:41:48.297 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:41:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.406+0000 7f62fa238640 1 Processor -- start 2024-04-25T08:41:48.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.406+0000 7f62fa238640 1 -- start start 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62fa238640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f62f4071690 0x7f62f415ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62fa238640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f62f40ceba0 0x7f62f406d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62fa238640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62f40b2430 con 0x7f62f415a600 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62fa238640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f4071d70 con 0x7f62f40ceba0 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62fa238640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f40b26d0 con 0x7f62f4071690 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62f3fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f62f40ceba0 0x7f62f406d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62f37fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f62f415a600 0x7f62f4070f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33512/0 (socket says 172.21.15.38:33512) 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62f37fe640 1 -- 172.21.15.38:0/2168524854 learned_addr learned my addr 172.21.15.38:0/2168524854 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62f3fff640 1 -- 172.21.15.38:0/2168524854 >> v2:172.21.15.107:3300/0 conn(0x7f62f4071690 msgr2=0x7f62f415ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62f2ffd640 1 --2- 172.21.15.38:0/2168524854 >> v2:172.21.15.107:3300/0 conn(0x7f62f4071690 0x7f62f415ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62f3fff640 1 --2- 172.21.15.38:0/2168524854 >> v2:172.21.15.107:3300/0 conn(0x7f62f4071690 0x7f62f415ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62f3fff640 1 -- 172.21.15.38:0/2168524854 >> v1:172.21.15.38:6789/0 conn(0x7f62f415a600 legacy=0x7f62f4070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:48.400 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62f27fc640 1 -- 172.21.15.38:0/2168524854 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006749847 0 0) 0x7f62f40b2430 con 0x7f62f415a600 2024-04-25T08:41:48.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.407+0000 7f62f3fff640 1 -- 172.21.15.38:0/2168524854 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f40b29b0 con 0x7f62f40ceba0 2024-04-25T08:41:48.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.408+0000 7f62f3fff640 1 --2- 172.21.15.38:0/2168524854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f40ceba0 0x7f62f406d370 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f62f40cd0d0 tx=0x7f62ec02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2050673c7c4a173 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:48.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.408+0000 7f62f27fc640 1 -- 172.21.15.38:0/2168524854 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec007740 con 0x7f62f40ceba0 2024-04-25T08:41:48.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.408+0000 7f62f27fc640 1 -- 172.21.15.38:0/2168524854 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec0078e0 con 0x7f62f40ceba0 2024-04-25T08:41:48.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.408+0000 7f62f27fc640 1 -- 172.21.15.38:0/2168524854 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec007cb0 con 0x7f62f40ceba0 2024-04-25T08:41:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.409+0000 7f62fa238640 1 -- 172.21.15.38:0/2168524854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f40ceba0 msgr2=0x7f62f406d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.409+0000 7f62fa238640 1 --2- 172.21.15.38:0/2168524854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f40ceba0 0x7f62f406d370 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f62f40cd0d0 tx=0x7f62ec02a680 comp rx=0 tx=0).stop 2024-04-25T08:41:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.409+0000 7f62fa238640 1 -- 172.21.15.38:0/2168524854 shutdown_connections 2024-04-25T08:41:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.409+0000 7f62fa238640 1 --2- 172.21.15.38:0/2168524854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f40ceba0 0x7f62f406d370 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.409+0000 7f62fa238640 1 --2- 172.21.15.38:0/2168524854 >> v2:172.21.15.107:3300/0 conn(0x7f62f4071690 0x7f62f415ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.409+0000 7f62fa238640 1 -- 172.21.15.38:0/2168524854 >> 172.21.15.38:0/2168524854 conn(0x7f62f406abe0 msgr2=0x7f62f406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.409+0000 7f62fa238640 1 -- 172.21.15.38:0/2168524854 shutdown_connections 2024-04-25T08:41:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.409+0000 7f62fa238640 1 -- 172.21.15.38:0/2168524854 wait complete. 2024-04-25T08:41:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.410+0000 7f62fa238640 1 Processor -- start 2024-04-25T08:41:48.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.410+0000 7f62fa238640 1 -- start start 2024-04-25T08:41:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62fa238640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f4071690 0x7f62f4158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62fa238640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62f40ceba0 0x7f62f41f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62fa238640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f415a600 0x7f62f41f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62fa238640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f40b3600 con 0x7f62f4071690 2024-04-25T08:41:48.406 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62fa238640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f415a9e0 con 0x7f62f415a600 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62fa238640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f414caa0 con 0x7f62f40ceba0 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62f37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f4071690 0x7f62f4158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62f37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f4071690 0x7f62f4158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46558/0 (socket says 172.21.15.38:46558) 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62f37fe640 1 -- 172.21.15.38:0/1562920187 learned_addr learned my addr 172.21.15.38:0/1562920187 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62f3fff640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f415a600 0x7f62f41f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62f37fe640 1 -- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62f40ceba0 msgr2=0x7f62f41f1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62f37fe640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62f40ceba0 0x7f62f41f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62f37fe640 1 -- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f415a600 msgr2=0x7f62f41f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62f37fe640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f415a600 0x7f62f41f2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62f37fe640 1 -- 172.21.15.38:0/1562920187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f414cd80 con 0x7f62f4071690 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62f37fe640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f4071690 0x7f62f4158700 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f62e400c430 tx=0x7f62e400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62dbfff640 1 -- 172.21.15.38:0/1562920187 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40074e0 con 0x7f62f4071690 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62dbfff640 1 -- 172.21.15.38:0/1562920187 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e40047e0 con 0x7f62f4071690 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62fa238640 1 -- 172.21.15.38:0/1562920187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f414d060 con 0x7f62f4071690 2024-04-25T08:41:48.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.411+0000 7f62dbfff640 1 -- 172.21.15.38:0/1562920187 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4004bd0 con 0x7f62f4071690 2024-04-25T08:41:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.412+0000 7f62fa238640 1 -- 172.21.15.38:0/1562920187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f4158c40 con 0x7f62f4071690 2024-04-25T08:41:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.412+0000 7f62fa238640 1 -- 172.21.15.38:0/1562920187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f404ef90 con 0x7f62f4071690 2024-04-25T08:41:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.413+0000 7f62dbfff640 1 -- 172.21.15.38:0/1562920187 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62e4011070 con 0x7f62f4071690 2024-04-25T08:41:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.413+0000 7f62dbfff640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f62e006b4d0 0x7f62e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.413+0000 7f62dbfff640 1 -- 172.21.15.38:0/1562920187 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f62e408a060 con 0x7f62f4071690 2024-04-25T08:41:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.414+0000 7f62f2ffd640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f62e006b4d0 0x7f62e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:48.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.414+0000 7f62f2ffd640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f62e006b4d0 0x7f62e006d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f62e8006fd0 tx=0x7f62e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:48.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.417+0000 7f62dbfff640 1 -- 172.21.15.38:0/1562920187 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62e400e280 con 0x7f62f4071690 2024-04-25T08:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.609+0000 7f62fa238640 1 -- 172.21.15.38:0/1562920187 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f62f40cd210 con 0x7f62f4071690 2024-04-25T08:41:48.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.610+0000 7f62dbfff640 1 -- 172.21.15.38:0/1562920187 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v133) v1 ==== 80+0+122 (secure 0 0 0) 0x7f62e4058d00 con 0x7f62f4071690 2024-04-25T08:41:48.603 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6835/1312063561 2024-04-26T08:41:40.950341+0000 2024-04-25T08:41:48.603 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6834/1312063561 2024-04-26T08:41:40.950341+0000 2024-04-25T08:41:48.603 INFO:teuthology.orchestra.run.smithi038.stderr:listed 2 entries 2024-04-25T08:41:48.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.612+0000 7f62fa238640 1 -- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f62e006b4d0 msgr2=0x7f62e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.612+0000 7f62fa238640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f62e006b4d0 0x7f62e006d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f62e8006fd0 tx=0x7f62e8008040 comp rx=0 tx=0).stop 2024-04-25T08:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.612+0000 7f62fa238640 1 -- 172.21.15.38:0/1562920187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f4071690 msgr2=0x7f62f4158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.612+0000 7f62fa238640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f4071690 0x7f62f4158700 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f62e400c430 tx=0x7f62e400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.612+0000 7f62fa238640 1 -- 172.21.15.38:0/1562920187 shutdown_connections 2024-04-25T08:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.612+0000 7f62fa238640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62f415a600 0x7f62f41f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.612+0000 7f62fa238640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f62e006b4d0 0x7f62e006d990 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.612+0000 7f62fa238640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f62f40ceba0 0x7f62f41f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.612+0000 7f62fa238640 1 --2- 172.21.15.38:0/1562920187 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62f4071690 0x7f62f4158700 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.612+0000 7f62fa238640 1 -- 172.21.15.38:0/1562920187 >> 172.21.15.38:0/1562920187 conn(0x7f62f406abe0 msgr2=0x7f62f4070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.612+0000 7f62fa238640 1 -- 172.21.15.38:0/1562920187 shutdown_connections 2024-04-25T08:41:48.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:48.612+0000 7f62fa238640 1 -- 172.21.15.38:0/1562920187 wait complete. 2024-04-25T08:41:48.615 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:41:48.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:48.616 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:41:48.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[94693]: fuse finished with error 0 and tester_r 0 2024-04-25T08:41:48.640 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:41:48.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:48.640 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:41:48.784 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:41:48.784 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:41:48.802 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:41:48.803 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:41:48.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:48.803 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:41:48.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:48.862 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:41:48.862 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:41:48.862 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:41:48.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:41:48.878 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:41:48.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:48.879 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:41:48.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:48.934 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:41:48.935 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:41:48.935 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:41:48.953 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:41:48.953 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:41:48.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:41:48.953 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:41:49.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:41:49.011 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:41:49.011 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:49.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.120+0000 7fe161f53640 1 Processor -- start 2024-04-25T08:41:49.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.121+0000 7fe161f53640 1 -- start start 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.121+0000 7fe161f53640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe15c15dc10 0x7fe15c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.121+0000 7fe161f53640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe15c0d0f60 0x7fe15c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.121+0000 7fe161f53640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15c0d3e70 con 0x7fe15c0ced60 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.121+0000 7fe161f53640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15c15dff0 con 0x7fe15c0d0f60 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.121+0000 7fe161f53640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15c0b6990 con 0x7fe15c15dc10 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.121+0000 7fe15bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe15c0d0f60 0x7fe15c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.121+0000 7fe15bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe15c0d0f60 0x7fe15c0b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60376/0 (socket says 172.21.15.38:60376) 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.121+0000 7fe15bfff640 1 -- 172.21.15.38:0/1379495222 learned_addr learned my addr 172.21.15.38:0/1379495222 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.122+0000 7fe15b7fe640 1 --2- 172.21.15.38:0/1379495222 >> v2:172.21.15.107:3300/0 conn(0x7fe15c15dc10 0x7fe15c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.122+0000 7fe15bfff640 1 -- 172.21.15.38:0/1379495222 >> v2:172.21.15.107:3300/0 conn(0x7fe15c15dc10 msgr2=0x7fe15c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.122+0000 7fe15bfff640 1 --2- 172.21.15.38:0/1379495222 >> v2:172.21.15.107:3300/0 conn(0x7fe15c15dc10 0x7fe15c0ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.122+0000 7fe15bfff640 1 -- 172.21.15.38:0/1379495222 >> v1:172.21.15.107:6789/0 conn(0x7fe15c0ced60 legacy=0x7fe15c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.122+0000 7fe15bfff640 1 -- 172.21.15.38:0/1379495222 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c0d4150 con 0x7fe15c0d0f60 2024-04-25T08:41:49.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.122+0000 7fe15b7fe640 1 --2- 172.21.15.38:0/1379495222 >> v2:172.21.15.107:3300/0 conn(0x7fe15c15dc10 0x7fe15c0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:49.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.122+0000 7fe15bfff640 1 --2- 172.21.15.38:0/1379495222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe15c0d0f60 0x7fe15c0b2370 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0b5d50 tx=0x7fe15402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4788b8f3a6fcbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:49.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.123+0000 7fe15a7fc640 1 -- 172.21.15.38:0/1379495222 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154036070 con 0x7fe15c0d0f60 2024-04-25T08:41:49.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.123+0000 7fe15a7fc640 1 -- 172.21.15.38:0/1379495222 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe154031600 con 0x7fe15c0d0f60 2024-04-25T08:41:49.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.123+0000 7fe15a7fc640 1 -- 172.21.15.38:0/1379495222 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1540319d0 con 0x7fe15c0d0f60 2024-04-25T08:41:49.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.123+0000 7fe161f53640 1 -- 172.21.15.38:0/1379495222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe15c0d0f60 msgr2=0x7fe15c0b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.123+0000 7fe161f53640 1 --2- 172.21.15.38:0/1379495222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe15c0d0f60 0x7fe15c0b2370 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7fe15c0b5d50 tx=0x7fe15402a660 comp rx=0 tx=0).stop 2024-04-25T08:41:49.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.123+0000 7fe161f53640 1 -- 172.21.15.38:0/1379495222 shutdown_connections 2024-04-25T08:41:49.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.123+0000 7fe161f53640 1 --2- 172.21.15.38:0/1379495222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe15c0d0f60 0x7fe15c0b2370 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.123+0000 7fe161f53640 1 --2- 172.21.15.38:0/1379495222 >> v2:172.21.15.107:3300/0 conn(0x7fe15c15dc10 0x7fe15c0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.123+0000 7fe161f53640 1 -- 172.21.15.38:0/1379495222 >> 172.21.15.38:0/1379495222 conn(0x7fe15c066d10 msgr2=0x7fe15c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:49.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.123+0000 7fe161f53640 1 -- 172.21.15.38:0/1379495222 shutdown_connections 2024-04-25T08:41:49.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.123+0000 7fe161f53640 1 -- 172.21.15.38:0/1379495222 wait complete. 2024-04-25T08:41:49.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.124+0000 7fe161f53640 1 Processor -- start 2024-04-25T08:41:49.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.125+0000 7fe161f53640 1 -- start start 2024-04-25T08:41:49.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.125+0000 7fe161f53640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe15c0ced60 0x7fe15c151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.125+0000 7fe161f53640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe15c0d0f60 0x7fe15c0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.125+0000 7fe161f53640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe15c15dc10 0x7fe15c15f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.125+0000 7fe161f53640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c0d3900 con 0x7fe15c0d0f60 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.125+0000 7fe161f53640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c0b6bc0 con 0x7fe15c15dc10 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.125+0000 7fe161f53640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c0d3a80 con 0x7fe15c0ced60 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.125+0000 7fe15affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe15c0d0f60 0x7fe15c0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe15affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe15c0d0f60 0x7fe15c0aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60390/0 (socket says 172.21.15.38:60390) 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe15affd640 1 -- 172.21.15.38:0/2378003955 learned_addr learned my addr 172.21.15.38:0/2378003955 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe15bfff640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe15c15dc10 0x7fe15c15f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe15b7fe640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe15c0ced60 0x7fe15c151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe15affd640 1 -- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe15c0ced60 msgr2=0x7fe15c151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe15affd640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe15c0ced60 0x7fe15c151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe15affd640 1 -- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe15c15dc10 msgr2=0x7fe15c15f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe15affd640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe15c15dc10 0x7fe15c15f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe15affd640 1 -- 172.21.15.38:0/2378003955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c1522c0 con 0x7fe15c0d0f60 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe15affd640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe15c0d0f60 0x7fe15c0aafa0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fe15000b3f0 tx=0x7fe15000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:49.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe143fff640 1 -- 172.21.15.38:0/2378003955 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe150015020 con 0x7fe15c0d0f60 2024-04-25T08:41:49.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe143fff640 1 -- 172.21.15.38:0/2378003955 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe150011830 con 0x7fe15c0d0f60 2024-04-25T08:41:49.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe143fff640 1 -- 172.21.15.38:0/2378003955 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe150011c20 con 0x7fe15c0d0f60 2024-04-25T08:41:49.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe161f53640 1 -- 172.21.15.38:0/2378003955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c0cba90 con 0x7fe15c0d0f60 2024-04-25T08:41:49.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.126+0000 7fe161f53640 1 -- 172.21.15.38:0/2378003955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c0cc050 con 0x7fe15c0d0f60 2024-04-25T08:41:49.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.127+0000 7fe161f53640 1 -- 172.21.15.38:0/2378003955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15c0cc810 con 0x7fe15c0d0f60 2024-04-25T08:41:49.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.128+0000 7fe143fff640 1 -- 172.21.15.38:0/2378003955 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe150018020 con 0x7fe15c0d0f60 2024-04-25T08:41:49.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.128+0000 7fe143fff640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe14806b500 0x7fe14806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.128+0000 7fe143fff640 1 -- 172.21.15.38:0/2378003955 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe150089fa0 con 0x7fe15c0d0f60 2024-04-25T08:41:49.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.128+0000 7fe15b7fe640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe14806b500 0x7fe14806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.129+0000 7fe15b7fe640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe14806b500 0x7fe14806d9c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00a990 tx=0x7fe14c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:49.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.132+0000 7fe143fff640 1 -- 172.21.15.38:0/2378003955 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe15001f030 con 0x7fe15c0d0f60 2024-04-25T08:41:49.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.357+0000 7fe161f53640 1 -- 172.21.15.38:0/2378003955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe15c068e80 con 0x7fe15c0d0f60 2024-04-25T08:41:49.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.357+0000 7fe143fff640 1 -- 172.21.15.38:0/2378003955 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4158 (secure 0 0 0) 0x7fe15c1522c0 con 0x7fe15c0d0f60 2024-04-25T08:41:49.350 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:49.350 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:41:42.972814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9486":{"gid":9486,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":16,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:49.350 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 81 2024-04-25T08:41:49.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.360+0000 7fe161f53640 1 -- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe14806b500 msgr2=0x7fe14806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.360+0000 7fe161f53640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe14806b500 0x7fe14806d9c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00a990 tx=0x7fe14c008040 comp rx=0 tx=0).stop 2024-04-25T08:41:49.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.360+0000 7fe161f53640 1 -- 172.21.15.38:0/2378003955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe15c0d0f60 msgr2=0x7fe15c0aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.360+0000 7fe161f53640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe15c0d0f60 0x7fe15c0aafa0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7fe15000b3f0 tx=0x7fe15000b8b0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.360+0000 7fe161f53640 1 -- 172.21.15.38:0/2378003955 shutdown_connections 2024-04-25T08:41:49.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.360+0000 7fe161f53640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe15c15dc10 0x7fe15c15f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.360+0000 7fe161f53640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe15c0d0f60 0x7fe15c0aafa0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.360+0000 7fe161f53640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe14806b500 0x7fe14806d9c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.360+0000 7fe161f53640 1 --2- 172.21.15.38:0/2378003955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe15c0ced60 0x7fe15c151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.360+0000 7fe161f53640 1 -- 172.21.15.38:0/2378003955 >> 172.21.15.38:0/2378003955 conn(0x7fe15c066d10 msgr2=0x7fe15c0b82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:49.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.361+0000 7fe161f53640 1 -- 172.21.15.38:0/2378003955 shutdown_connections 2024-04-25T08:41:49.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.361+0000 7fe161f53640 1 -- 172.21.15.38:0/2378003955 wait complete. 2024-04-25T08:41:49.367 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:41:49.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.475+0000 7fe689134640 1 Processor -- start 2024-04-25T08:41:49.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.475+0000 7fe689134640 1 -- start start 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe689134640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6840b5710 0x7fe684161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe689134640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe68415ddf0 0x7fe6840b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe689134640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe684161e00 con 0x7fe6840b5af0 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe689134640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6840b9930 con 0x7fe68415ddf0 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe689134640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6841620e0 con 0x7fe6840b5710 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe68359e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe6840b5af0 0x7fe68406bff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35776/0 (socket says 172.21.15.38:35776) 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe68359e640 1 -- 172.21.15.38:0/3855144641 learned_addr learned my addr 172.21.15.38:0/3855144641 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe67a59c640 1 --2- 172.21.15.38:0/3855144641 >> v2:172.21.15.107:3300/0 conn(0x7fe68415ddf0 0x7fe6840b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe682d9d640 1 --2- 172.21.15.38:0/3855144641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6840b5710 0x7fe684161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008653231 0 0) 0x7fe684161e00 con 0x7fe6840b5af0 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe670003520 con 0x7fe6840b5af0 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3607250927 0 0) 0x7fe670003520 con 0x7fe6840b5af0 2024-04-25T08:41:49.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6840b5710 msgr2=0x7fe684161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe68259c640 1 --2- 172.21.15.38:0/3855144641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6840b5710 0x7fe684161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 >> v2:172.21.15.107:3300/0 conn(0x7fe68415ddf0 msgr2=0x7fe6840b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe68259c640 1 --2- 172.21.15.38:0/3855144641 >> v2:172.21.15.107:3300/0 conn(0x7fe68415ddf0 0x7fe6840b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6841623c0 con 0x7fe6840b5af0 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.476+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fe67c0019f0 con 0x7fe6840b5af0 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 >> v1:172.21.15.38:6789/0 conn(0x7fe6840b5af0 legacy=0x7fe68406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe68259c640 1 --2- 172.21.15.38:0/3855144641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6700039b0 0x7fe670003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe68259c640 1 --2- 172.21.15.38:0/3855144641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe670004800 0x7fe670004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe68259c640 1 --2- 172.21.15.38:0/3855144641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6840b5710 0x7fe6700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6841620e0 con 0x7fe6700039b0 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67000b9c0 con 0x7fe670004800 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67000bb40 con 0x7fe6840b5710 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe682d9d640 1 --2- 172.21.15.38:0/3855144641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6700039b0 0x7fe670003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe68359e640 1 --2- 172.21.15.38:0/3855144641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6840b5710 0x7fe6700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe67a59c640 1 --2- 172.21.15.38:0/3855144641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe670004800 0x7fe670004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe682d9d640 1 -- 172.21.15.38:0/3855144641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6840b5710 msgr2=0x7fe6700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe682d9d640 1 --2- 172.21.15.38:0/3855144641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6840b5710 0x7fe6700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe682d9d640 1 -- 172.21.15.38:0/3855144641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe670004800 msgr2=0x7fe670004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe682d9d640 1 --2- 172.21.15.38:0/3855144641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe670004800 0x7fe670004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.477+0000 7fe682d9d640 1 -- 172.21.15.38:0/3855144641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe670003520 con 0x7fe6700039b0 2024-04-25T08:41:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.478+0000 7fe682d9d640 1 --2- 172.21.15.38:0/3855144641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6700039b0 0x7fe670003dd0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fe674009ac0 tx=0x7fe670005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3c305a8a27ffc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.478+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674036070 con 0x7fe6700039b0 2024-04-25T08:41:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.478+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe674031400 con 0x7fe6700039b0 2024-04-25T08:41:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.478+0000 7fe68259c640 1 -- 172.21.15.38:0/3855144641 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe674009690 con 0x7fe6700039b0 2024-04-25T08:41:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.478+0000 7fe689134640 1 -- 172.21.15.38:0/3855144641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6700039b0 msgr2=0x7fe670003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.478+0000 7fe689134640 1 --2- 172.21.15.38:0/3855144641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6700039b0 0x7fe670003dd0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fe674009ac0 tx=0x7fe670005830 comp rx=0 tx=0).stop 2024-04-25T08:41:49.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.478+0000 7fe68359e640 1 -- 172.21.15.38:0/3855144641 reap_dead start 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.478+0000 7fe689134640 1 -- 172.21.15.38:0/3855144641 shutdown_connections 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.478+0000 7fe689134640 1 -- 172.21.15.38:0/3855144641 >> 172.21.15.38:0/3855144641 conn(0x7fe6840d07c0 msgr2=0x7fe6840d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.478+0000 7fe689134640 1 -- 172.21.15.38:0/3855144641 shutdown_connections 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.478+0000 7fe689134640 1 -- 172.21.15.38:0/3855144641 wait complete. 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.480+0000 7fe689134640 1 Processor -- start 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.480+0000 7fe689134640 1 -- start start 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe689134640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700039b0 0x7fe6840ab150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe689134640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6840ac750 0x7fe6841f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe689134640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6840ace30 0x7fe6840ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe689134640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840d34d0 con 0x7fe6840ace30 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe689134640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840d3350 con 0x7fe6840ac750 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe689134640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840d3650 con 0x7fe6700039b0 2024-04-25T08:41:49.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe68359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6840ace30 0x7fe6840ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe68359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6840ace30 0x7fe6840ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60414/0 (socket says 172.21.15.38:60414) 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe68359e640 1 -- 172.21.15.38:0/2616473286 learned_addr learned my addr 172.21.15.38:0/2616473286 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe682d9d640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700039b0 0x7fe6840ab150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe67a59c640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6840ac750 0x7fe6841f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe68359e640 1 -- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700039b0 msgr2=0x7fe6840ab150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe68359e640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700039b0 0x7fe6840ab150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe68359e640 1 -- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6840ac750 msgr2=0x7fe6841f2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe68359e640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6840ac750 0x7fe6841f2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe68359e640 1 -- 172.21.15.38:0/2616473286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6840aba00 con 0x7fe6840ace30 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe68359e640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6840ace30 0x7fe6840ad2b0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fe67c001330 tx=0x7fe67c007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe67bfff640 1 -- 172.21.15.38:0/2616473286 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c017020 con 0x7fe6840ace30 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe67bfff640 1 -- 172.21.15.38:0/2616473286 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c00ac80 con 0x7fe6840ace30 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe689134640 1 -- 172.21.15.38:0/2616473286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6840abce0 con 0x7fe6840ace30 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe67bfff640 1 -- 172.21.15.38:0/2616473286 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c013610 con 0x7fe6840ace30 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.481+0000 7fe689134640 1 -- 172.21.15.38:0/2616473286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6840ab690 con 0x7fe6840ace30 2024-04-25T08:41:49.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.482+0000 7fe689134640 1 -- 172.21.15.38:0/2616473286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe684068e80 con 0x7fe6840ace30 2024-04-25T08:41:49.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.483+0000 7fe67bfff640 1 -- 172.21.15.38:0/2616473286 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe67c01a020 con 0x7fe6840ace30 2024-04-25T08:41:49.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.483+0000 7fe67bfff640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe66406b4d0 0x7fe66406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.483+0000 7fe67bfff640 1 -- 172.21.15.38:0/2616473286 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe67c08ab00 con 0x7fe6840ace30 2024-04-25T08:41:49.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.483+0000 7fe682d9d640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe66406b4d0 0x7fe66406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.484+0000 7fe682d9d640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe66406b4d0 0x7fe66406d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe674033000 tx=0x7fe674008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:49.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.487+0000 7fe67bfff640 1 -- 172.21.15.38:0/2616473286 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe67c021020 con 0x7fe6840ace30 2024-04-25T08:41:49.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.668+0000 7fe689134640 1 -- 172.21.15.38:0/2616473286 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6841618c0 con 0x7fe6840ace30 2024-04-25T08:41:49.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.669+0000 7fe67bfff640 1 -- 172.21.15.38:0/2616473286 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17017 (secure 0 0 0) 0x7fe67c08ae20 con 0x7fe6840ace30 2024-04-25T08:41:49.662 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:49.662 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":133,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:41:40.950430+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:40:52.007975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:40:52.707859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/1312063561":"2024-04-26T08:41:40.950341+0000","172.21.15.38:6834/1312063561":"2024-04-26T08:41:40.950341+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:41:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.671+0000 7fe689134640 1 -- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe66406b4d0 msgr2=0x7fe66406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.671+0000 7fe689134640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe66406b4d0 0x7fe66406d990 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe674033000 tx=0x7fe674008030 comp rx=0 tx=0).stop 2024-04-25T08:41:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.671+0000 7fe689134640 1 -- 172.21.15.38:0/2616473286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6840ace30 msgr2=0x7fe6840ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.671+0000 7fe689134640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6840ace30 0x7fe6840ad2b0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fe67c001330 tx=0x7fe67c007a60 comp rx=0 tx=0).stop 2024-04-25T08:41:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.671+0000 7fe689134640 1 -- 172.21.15.38:0/2616473286 shutdown_connections 2024-04-25T08:41:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.671+0000 7fe689134640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6840ace30 0x7fe6840ad2b0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.671+0000 7fe689134640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe6840ac750 0x7fe6841f2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.671+0000 7fe689134640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe66406b4d0 0x7fe66406d990 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.671+0000 7fe689134640 1 --2- 172.21.15.38:0/2616473286 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe6700039b0 0x7fe6840ab150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.671+0000 7fe689134640 1 -- 172.21.15.38:0/2616473286 >> 172.21.15.38:0/2616473286 conn(0x7fe6840d07c0 msgr2=0x7fe6840b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.671+0000 7fe689134640 1 -- 172.21.15.38:0/2616473286 shutdown_connections 2024-04-25T08:41:49.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.671+0000 7fe689134640 1 -- 172.21.15.38:0/2616473286 wait complete. 2024-04-25T08:41:49.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:40:52.007975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:40:52.707859+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '132', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:41:49.676 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:41:49.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.786+0000 7fb724d8b640 1 Processor -- start 2024-04-25T08:41:49.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.786+0000 7fb724d8b640 1 -- start start 2024-04-25T08:41:49.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.787+0000 7fb724d8b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb72006c700 0x7fb720162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.787+0000 7fb724d8b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb72006cae0 0x7fb72015e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.787+0000 7fb724d8b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb72015ec20 con 0x7fb72015e580 2024-04-25T08:41:49.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.787+0000 7fb724d8b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb720161c80 con 0x7fb72006cae0 2024-04-25T08:41:49.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.787+0000 7fb724d8b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72015ef00 con 0x7fb72006c700 2024-04-25T08:41:49.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.787+0000 7fb71ed9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb72015e580 0x7fb7200cdf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35792/0 (socket says 172.21.15.38:35792) 2024-04-25T08:41:49.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.787+0000 7fb71ed9d640 1 -- 172.21.15.38:0/78493969 learned_addr learned my addr 172.21.15.38:0/78493969 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:49.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2062557966 0 0) 0x7fb72015ec20 con 0x7fb72015e580 2024-04-25T08:41:49.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71e59c640 1 --2- 172.21.15.38:0/78493969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb72006c700 0x7fb720162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb70c003520 con 0x7fb72015e580 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71dd9b640 1 --2- 172.21.15.38:0/78493969 >> v2:172.21.15.107:3300/0 conn(0x7fb72006cae0 0x7fb72015e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3061748535 0 0) 0x7fb70c003520 con 0x7fb72015e580 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb72006c700 msgr2=0x7fb720162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 --2- 172.21.15.38:0/78493969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb72006c700 0x7fb720162110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 >> v2:172.21.15.107:3300/0 conn(0x7fb72006cae0 msgr2=0x7fb72015e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 --2- 172.21.15.38:0/78493969 >> v2:172.21.15.107:3300/0 conn(0x7fb72006cae0 0x7fb72015e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb720161630 con 0x7fb72015e580 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fb7100019f0 con 0x7fb72015e580 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 >> v1:172.21.15.38:6789/0 conn(0x7fb72015e580 legacy=0x7fb7200cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 --2- 172.21.15.38:0/78493969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb72006c700 0x7fb70c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 --2- 172.21.15.38:0/78493969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb70c003f30 0x7fb70c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 --2- 172.21.15.38:0/78493969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c004d40 0x7fb70c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb720161c80 con 0x7fb70c004d40 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72015ef00 con 0x7fb70c003f30 2024-04-25T08:41:49.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c00b9c0 con 0x7fb72006c700 2024-04-25T08:41:49.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71ed9d640 1 --2- 172.21.15.38:0/78493969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c004d40 0x7fb70c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71e59c640 1 --2- 172.21.15.38:0/78493969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb72006c700 0x7fb70c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71dd9b640 1 --2- 172.21.15.38:0/78493969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb70c003f30 0x7fb70c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71ed9d640 1 -- 172.21.15.38:0/78493969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb72006c700 msgr2=0x7fb70c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:49.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71ed9d640 1 --2- 172.21.15.38:0/78493969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb72006c700 0x7fb70c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71ed9d640 1 -- 172.21.15.38:0/78493969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb70c003f30 msgr2=0x7fb70c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:49.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.788+0000 7fb71ed9d640 1 --2- 172.21.15.38:0/78493969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb70c003f30 0x7fb70c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.789+0000 7fb71ed9d640 1 -- 172.21.15.38:0/78493969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c003520 con 0x7fb70c004d40 2024-04-25T08:41:49.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.789+0000 7fb71ed9d640 1 --2- 172.21.15.38:0/78493969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c004d40 0x7fb70c0051c0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fb71000a8a0 tx=0x7fb70c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6ebdaa8d1475ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:49.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.789+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71000a4c0 con 0x7fb70c004d40 2024-04-25T08:41:49.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.789+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71002f070 con 0x7fb70c004d40 2024-04-25T08:41:49.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.789+0000 7fb71d59a640 1 -- 172.21.15.38:0/78493969 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71002bd50 con 0x7fb70c004d40 2024-04-25T08:41:49.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.790+0000 7fb724d8b640 1 -- 172.21.15.38:0/78493969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c004d40 msgr2=0x7fb70c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.790+0000 7fb724d8b640 1 --2- 172.21.15.38:0/78493969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb70c004d40 0x7fb70c0051c0 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fb71000a8a0 tx=0x7fb70c005830 comp rx=0 tx=0).stop 2024-04-25T08:41:49.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.790+0000 7fb71ed9d640 1 -- 172.21.15.38:0/78493969 reap_dead start 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.790+0000 7fb724d8b640 1 -- 172.21.15.38:0/78493969 shutdown_connections 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.790+0000 7fb724d8b640 1 -- 172.21.15.38:0/78493969 >> 172.21.15.38:0/78493969 conn(0x7fb72008b000 msgr2=0x7fb72008b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.790+0000 7fb724d8b640 1 -- 172.21.15.38:0/78493969 shutdown_connections 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.790+0000 7fb724d8b640 1 -- 172.21.15.38:0/78493969 wait complete. 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.791+0000 7fb724d8b640 1 Processor -- start 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.791+0000 7fb724d8b640 1 -- start start 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.792+0000 7fb724d8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb70c004d40 0x7fb7200b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.792+0000 7fb724d8b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7200c8750 0x7fb7200c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.792+0000 7fb724d8b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7200caa70 0x7fb7200caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.792+0000 7fb724d8b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72006cf60 con 0x7fb7200caa70 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.792+0000 7fb724d8b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72006c190 con 0x7fb70c004d40 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.792+0000 7fb724d8b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72015b920 con 0x7fb7200c8750 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.792+0000 7fb71ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7200caa70 0x7fb7200caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.792+0000 7fb71ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7200caa70 0x7fb7200caef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60434/0 (socket says 172.21.15.38:60434) 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.792+0000 7fb71ed9d640 1 -- 172.21.15.38:0/1962247641 learned_addr learned my addr 172.21.15.38:0/1962247641 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.792+0000 7fb71e59c640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb70c004d40 0x7fb7200b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.792+0000 7fb71dd9b640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7200c8750 0x7fb7200c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.793+0000 7fb71ed9d640 1 -- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7200c8750 msgr2=0x7fb7200c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.793+0000 7fb71ed9d640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7200c8750 0x7fb7200c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.793+0000 7fb71ed9d640 1 -- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb70c004d40 msgr2=0x7fb7200b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.793+0000 7fb71ed9d640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb70c004d40 0x7fb7200b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.793+0000 7fb71ed9d640 1 -- 172.21.15.38:0/1962247641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7200ae7d0 con 0x7fb7200caa70 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.793+0000 7fb71dd9b640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7200c8750 0x7fb7200c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.793+0000 7fb71ed9d640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7200caa70 0x7fb7200caef0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fb71000a8a0 tx=0x7fb710001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.793+0000 7fb716ffd640 1 -- 172.21.15.38:0/1962247641 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb710044070 con 0x7fb7200caa70 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.793+0000 7fb716ffd640 1 -- 172.21.15.38:0/1962247641 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7100019e0 con 0x7fb7200caa70 2024-04-25T08:41:49.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.793+0000 7fb724d8b640 1 -- 172.21.15.38:0/1962247641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7200aeab0 con 0x7fb7200caa70 2024-04-25T08:41:49.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.793+0000 7fb716ffd640 1 -- 172.21.15.38:0/1962247641 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71000a4c0 con 0x7fb7200caa70 2024-04-25T08:41:49.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.793+0000 7fb724d8b640 1 -- 172.21.15.38:0/1962247641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7200af070 con 0x7fb7200caa70 2024-04-25T08:41:49.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.794+0000 7fb724d8b640 1 -- 172.21.15.38:0/1962247641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb720068e80 con 0x7fb7200caa70 2024-04-25T08:41:49.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.794+0000 7fb716ffd640 1 -- 172.21.15.38:0/1962247641 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb710047070 con 0x7fb7200caa70 2024-04-25T08:41:49.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.795+0000 7fb716ffd640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb70006b4d0 0x7fb70006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:49.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.795+0000 7fb716ffd640 1 -- 172.21.15.38:0/1962247641 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb7100aa690 con 0x7fb7200caa70 2024-04-25T08:41:49.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.795+0000 7fb71e59c640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb70006b4d0 0x7fb70006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:49.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.796+0000 7fb71e59c640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb70006b4d0 0x7fb70006d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb718009b00 tx=0x7fb718008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:49.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.799+0000 7fb716ffd640 1 -- 172.21.15.38:0/1962247641 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb710079330 con 0x7fb7200caa70 2024-04-25T08:41:49.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.979+0000 7fb724d8b640 1 -- 172.21.15.38:0/1962247641 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7200b1220 con 0x7fb7200caa70 2024-04-25T08:41:49.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.979+0000 7fb716ffd640 1 -- 172.21.15.38:0/1962247641 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v133) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb710078c30 con 0x7fb7200caa70 2024-04-25T08:41:49.972 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:41:49.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.981+0000 7fb724d8b640 1 -- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb70006b4d0 msgr2=0x7fb70006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.981+0000 7fb724d8b640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb70006b4d0 0x7fb70006d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fb718009b00 tx=0x7fb718008040 comp rx=0 tx=0).stop 2024-04-25T08:41:49.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.981+0000 7fb724d8b640 1 -- 172.21.15.38:0/1962247641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7200caa70 msgr2=0x7fb7200caef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:49.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.981+0000 7fb724d8b640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7200caa70 0x7fb7200caef0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fb71000a8a0 tx=0x7fb710001dc0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.981+0000 7fb724d8b640 1 -- 172.21.15.38:0/1962247641 shutdown_connections 2024-04-25T08:41:49.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.981+0000 7fb724d8b640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb7200caa70 0x7fb7200caef0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.981+0000 7fb724d8b640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb70006b4d0 0x7fb70006d990 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.981+0000 7fb724d8b640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb7200c8750 0x7fb7200c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.981+0000 7fb724d8b640 1 --2- 172.21.15.38:0/1962247641 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb70c004d40 0x7fb7200b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:49.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.981+0000 7fb724d8b640 1 -- 172.21.15.38:0/1962247641 >> 172.21.15.38:0/1962247641 conn(0x7fb72008b000 msgr2=0x7fb72015f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:49.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.981+0000 7fb724d8b640 1 -- 172.21.15.38:0/1962247641 shutdown_connections 2024-04-25T08:41:49.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:49.981+0000 7fb724d8b640 1 -- 172.21.15.38:0/1962247641 wait complete. 2024-04-25T08:41:49.985 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:41:50.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.096+0000 7f9c07037640 1 Processor -- start 2024-04-25T08:41:50.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.096+0000 7f9c07037640 1 -- start start 2024-04-25T08:41:50.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.097+0000 7f9c07037640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c0015df30 0x7f9c0009a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.097+0000 7f9c07037640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9c0009a180 0x7f9c000cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.097+0000 7f9c07037640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c0015f300 con 0x7f9c00162390 2024-04-25T08:41:50.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.097+0000 7f9c07037640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c0015e310 con 0x7f9c0009a180 2024-04-25T08:41:50.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.097+0000 7f9c07037640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0009aeb0 con 0x7f9c0015df30 2024-04-25T08:41:50.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.097+0000 7f9c055d4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9c0009a180 0x7f9c000cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.097+0000 7f9bfffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9c00162390 0x7f9c00099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35804/0 (socket says 172.21.15.38:35804) 2024-04-25T08:41:50.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.097+0000 7f9bfffff640 1 -- 172.21.15.38:0/330802310 learned_addr learned my addr 172.21.15.38:0/330802310 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:50.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.097+0000 7f9c055d4640 1 -- 172.21.15.38:0/330802310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c0015df30 msgr2=0x7f9c0009a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:50.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.097+0000 7f9c04dd3640 1 --2- 172.21.15.38:0/330802310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c0015df30 0x7f9c0009a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.097+0000 7f9c055d4640 1 --2- 172.21.15.38:0/330802310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c0015df30 0x7f9c0009a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.097+0000 7f9bff7fe640 1 -- 172.21.15.38:0/330802310 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551319279 0 0) 0x7f9c0015f300 con 0x7f9c00162390 2024-04-25T08:41:50.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.098+0000 7f9c055d4640 1 -- 172.21.15.38:0/330802310 >> v1:172.21.15.38:6789/0 conn(0x7f9c00162390 legacy=0x7f9c00099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.098+0000 7f9c055d4640 1 -- 172.21.15.38:0/330802310 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c001627f0 con 0x7f9c0009a180 2024-04-25T08:41:50.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.098+0000 7f9c04dd3640 1 --2- 172.21.15.38:0/330802310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c0015df30 0x7f9c0009a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.098+0000 7f9c055d4640 1 --2- 172.21.15.38:0/330802310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c0009a180 0x7f9c000cf8f0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f9be80097a0 tx=0x7f9be802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a4bcee224d0259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:50.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.098+0000 7f9bff7fe640 1 -- 172.21.15.38:0/330802310 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8037070 con 0x7f9c0009a180 2024-04-25T08:41:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.098+0000 7f9bff7fe640 1 -- 172.21.15.38:0/330802310 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8032440 con 0x7f9c0009a180 2024-04-25T08:41:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.098+0000 7f9bff7fe640 1 -- 172.21.15.38:0/330802310 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8032810 con 0x7f9c0009a180 2024-04-25T08:41:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.099+0000 7f9c07037640 1 -- 172.21.15.38:0/330802310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c0009a180 msgr2=0x7f9c000cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.099+0000 7f9c07037640 1 --2- 172.21.15.38:0/330802310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c0009a180 0x7f9c000cf8f0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f9be80097a0 tx=0x7f9be802a180 comp rx=0 tx=0).stop 2024-04-25T08:41:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.099+0000 7f9c07037640 1 -- 172.21.15.38:0/330802310 shutdown_connections 2024-04-25T08:41:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.099+0000 7f9c07037640 1 --2- 172.21.15.38:0/330802310 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c0009a180 0x7f9c000cf8f0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.099+0000 7f9c07037640 1 --2- 172.21.15.38:0/330802310 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c0015df30 0x7f9c0009a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.099+0000 7f9c07037640 1 -- 172.21.15.38:0/330802310 >> 172.21.15.38:0/330802310 conn(0x7f9c000b3110 msgr2=0x7f9c000b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.099+0000 7f9c07037640 1 -- 172.21.15.38:0/330802310 shutdown_connections 2024-04-25T08:41:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.099+0000 7f9c07037640 1 -- 172.21.15.38:0/330802310 wait complete. 2024-04-25T08:41:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.100+0000 7f9c07037640 1 Processor -- start 2024-04-25T08:41:50.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.100+0000 7f9c07037640 1 -- start start 2024-04-25T08:41:50.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.101+0000 7f9c07037640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c0009a180 0x7f9c0014cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.101+0000 7f9c07037640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0015df30 0x7f9c000ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.101+0000 7f9c07037640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00162390 0x7f9c000cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.101+0000 7f9c07037640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0015e820 con 0x7f9c00162390 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.101+0000 7f9c07037640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0015ee70 con 0x7f9c0015df30 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.101+0000 7f9c07037640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0015eb00 con 0x7f9c0009a180 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.101+0000 7f9c055d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00162390 0x7f9c000cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.101+0000 7f9c055d4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00162390 0x7f9c000cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60464/0 (socket says 172.21.15.38:60464) 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.101+0000 7f9c055d4640 1 -- 172.21.15.38:0/970526792 learned_addr learned my addr 172.21.15.38:0/970526792 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9c055d4640 1 -- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c0009a180 msgr2=0x7f9c0014cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9c055d4640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c0009a180 0x7f9c0014cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9c055d4640 1 -- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0015df30 msgr2=0x7f9c000ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9bfffff640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0015df30 0x7f9c000ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9c055d4640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0015df30 0x7f9c000ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.096 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9c055d4640 1 -- 172.21.15.38:0/970526792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0014d020 con 0x7f9c00162390 2024-04-25T08:41:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9bfffff640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0015df30 0x7f9c000ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9c055d4640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00162390 0x7f9c000cd1e0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f9be8035040 tx=0x7f9be80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9bfd7fa640 1 -- 172.21.15.38:0/970526792 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8037040 con 0x7f9c00162390 2024-04-25T08:41:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9bfd7fa640 1 -- 172.21.15.38:0/970526792 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be8042070 con 0x7f9c00162390 2024-04-25T08:41:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9bfd7fa640 1 -- 172.21.15.38:0/970526792 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be8030b50 con 0x7f9c00162390 2024-04-25T08:41:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9c07037640 1 -- 172.21.15.38:0/970526792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c000c6e70 con 0x7f9c00162390 2024-04-25T08:41:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.102+0000 7f9c07037640 1 -- 172.21.15.38:0/970526792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c001e85d0 con 0x7f9c00162390 2024-04-25T08:41:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.103+0000 7f9c07037640 1 -- 172.21.15.38:0/970526792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c00068e80 con 0x7f9c00162390 2024-04-25T08:41:50.097 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.103+0000 7f9bfd7fa640 1 -- 172.21.15.38:0/970526792 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9be8030cf0 con 0x7f9c00162390 2024-04-25T08:41:50.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.104+0000 7f9bfd7fa640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9be006b4d0 0x7f9be006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.104+0000 7f9bfd7fa640 1 -- 172.21.15.38:0/970526792 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9be80aafd0 con 0x7f9c00162390 2024-04-25T08:41:50.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.104+0000 7f9c04dd3640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9be006b4d0 0x7f9be006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.105+0000 7f9c04dd3640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9be006b4d0 0x7f9be006d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9c000c84b0 tx=0x7f9bf80083d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:50.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.108+0000 7f9bfd7fa640 1 -- 172.21.15.38:0/970526792 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9be8032e20 con 0x7f9c00162390 2024-04-25T08:41:50.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.288+0000 7f9c07037640 1 -- 172.21.15.38:0/970526792 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c000b4020 con 0x7f9c00162390 2024-04-25T08:41:50.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.288+0000 7f9bfd7fa640 1 -- 172.21.15.38:0/970526792 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v133) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9be8079d00 con 0x7f9c00162390 2024-04-25T08:41:50.281 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:41:50.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.290+0000 7f9c07037640 1 -- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9be006b4d0 msgr2=0x7f9be006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.290+0000 7f9c07037640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9be006b4d0 0x7f9be006d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9c000c84b0 tx=0x7f9bf80083d0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.290+0000 7f9c07037640 1 -- 172.21.15.38:0/970526792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00162390 msgr2=0x7f9c000cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.290+0000 7f9c07037640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00162390 0x7f9c000cd1e0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f9be8035040 tx=0x7f9be80046c0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.291+0000 7f9c07037640 1 -- 172.21.15.38:0/970526792 shutdown_connections 2024-04-25T08:41:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.291+0000 7f9c07037640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c00162390 0x7f9c000cd1e0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.291+0000 7f9c07037640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9c0015df30 0x7f9c000ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.291+0000 7f9c07037640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9be006b4d0 0x7f9be006d990 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.291+0000 7f9c07037640 1 --2- 172.21.15.38:0/970526792 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9c0009a180 0x7f9c0014cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.291+0000 7f9c07037640 1 -- 172.21.15.38:0/970526792 >> 172.21.15.38:0/970526792 conn(0x7f9c000b3110 msgr2=0x7f9c000d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.291+0000 7f9c07037640 1 -- 172.21.15.38:0/970526792 shutdown_connections 2024-04-25T08:41:50.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.291+0000 7f9c07037640 1 -- 172.21.15.38:0/970526792 wait complete. 2024-04-25T08:41:50.294 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:41:50.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.408+0000 7f5a7ff5c640 1 Processor -- start 2024-04-25T08:41:50.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.408+0000 7f5a7ff5c640 1 -- start start 2024-04-25T08:41:50.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.409+0000 7f5a7ff5c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f5a7806c700 0x7f5a78162110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.409+0000 7f5a7ff5c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5a7806cae0 0x7f5a7815e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.409+0000 7f5a7ff5c640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7815ec20 con 0x7f5a7815e580 2024-04-25T08:41:50.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.409+0000 7f5a7ff5c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a7815e9f0 con 0x7f5a7806cae0 2024-04-25T08:41:50.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.409+0000 7f5a7ff5c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a78161c80 con 0x7f5a7806c700 2024-04-25T08:41:50.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.409+0000 7f5a7d4f7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5a7806cae0 0x7f5a7815e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.409+0000 7f5a7d4f7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f5a7806cae0 0x7f5a7815e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60468/0 (socket says 172.21.15.38:60468) 2024-04-25T08:41:50.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.409+0000 7f5a7d4f7640 1 -- 172.21.15.38:0/2315885894 learned_addr learned my addr 172.21.15.38:0/2315885894 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:50.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.410+0000 7f5a7dcf8640 1 --2- 172.21.15.38:0/2315885894 >> v2:172.21.15.107:3300/0 conn(0x7f5a7806c700 0x7f5a78162110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.410+0000 7f5a7d4f7640 1 -- 172.21.15.38:0/2315885894 >> v2:172.21.15.107:3300/0 conn(0x7f5a7806c700 msgr2=0x7f5a78162110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:50.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.410+0000 7f5a7d4f7640 1 --2- 172.21.15.38:0/2315885894 >> v2:172.21.15.107:3300/0 conn(0x7f5a7806c700 0x7f5a78162110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.410+0000 7f5a7d4f7640 1 -- 172.21.15.38:0/2315885894 >> v1:172.21.15.107:6789/0 conn(0x7f5a7815e580 legacy=0x7f5a780cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:50.403 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.410+0000 7f5a7d4f7640 1 -- 172.21.15.38:0/2315885894 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7815ef00 con 0x7f5a7806cae0 2024-04-25T08:41:50.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.410+0000 7f5a7d4f7640 1 --2- 172.21.15.38:0/2315885894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a7806cae0 0x7f5a7815e040 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f5a780d25f0 tx=0x7f5a6c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faff05d10db63a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:50.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.411+0000 7f5a7ccf6640 1 -- 172.21.15.38:0/2315885894 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c037070 con 0x7f5a7806cae0 2024-04-25T08:41:50.404 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.411+0000 7f5a7ccf6640 1 -- 172.21.15.38:0/2315885894 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c032440 con 0x7f5a7806cae0 2024-04-25T08:41:50.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.411+0000 7f5a7ccf6640 1 -- 172.21.15.38:0/2315885894 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c032810 con 0x7f5a7806cae0 2024-04-25T08:41:50.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.411+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2315885894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a7806cae0 msgr2=0x7f5a7815e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.411+0000 7f5a7ff5c640 1 --2- 172.21.15.38:0/2315885894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a7806cae0 0x7f5a7815e040 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f5a780d25f0 tx=0x7f5a6c02a230 comp rx=0 tx=0).stop 2024-04-25T08:41:50.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.411+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2315885894 shutdown_connections 2024-04-25T08:41:50.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.411+0000 7f5a7ff5c640 1 --2- 172.21.15.38:0/2315885894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a7806cae0 0x7f5a7815e040 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.411+0000 7f5a7ff5c640 1 --2- 172.21.15.38:0/2315885894 >> v2:172.21.15.107:3300/0 conn(0x7f5a7806c700 0x7f5a78162110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.411+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2315885894 >> 172.21.15.38:0/2315885894 conn(0x7f5a7808b000 msgr2=0x7f5a7808b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:50.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.411+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2315885894 shutdown_connections 2024-04-25T08:41:50.405 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.411+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2315885894 wait complete. 2024-04-25T08:41:50.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.412+0000 7f5a7ff5c640 1 Processor -- start 2024-04-25T08:41:50.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.412+0000 7f5a7ff5c640 1 -- start start 2024-04-25T08:41:50.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.413+0000 7f5a7ff5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7806c700 0x7f5a781f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.413+0000 7f5a7ff5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a7806cae0 0x7f5a780701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.407 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.413+0000 7f5a7ff5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a7815e580 0x7f5a78070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.413+0000 7f5a7ff5c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7806c190 con 0x7f5a7806cae0 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.413+0000 7f5a7ff5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78161910 con 0x7f5a7806c700 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.413+0000 7f5a7ff5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7806cf90 con 0x7f5a7815e580 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.413+0000 7f5a7d4f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a7806cae0 0x7f5a780701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.413+0000 7f5a7d4f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a7806cae0 0x7f5a780701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60472/0 (socket says 172.21.15.38:60472) 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.413+0000 7f5a7d4f7640 1 -- 172.21.15.38:0/2531499038 learned_addr learned my addr 172.21.15.38:0/2531499038 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.413+0000 7f5a7dcf8640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7806c700 0x7f5a781f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.413+0000 7f5a7e4f9640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a7815e580 0x7f5a78070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.414+0000 7f5a7d4f7640 1 -- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a7815e580 msgr2=0x7f5a78070710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.414+0000 7f5a7d4f7640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a7815e580 0x7f5a78070710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.414+0000 7f5a7d4f7640 1 -- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7806c700 msgr2=0x7f5a781f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.414+0000 7f5a7d4f7640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7806c700 0x7f5a781f3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.414+0000 7f5a7d4f7640 1 -- 172.21.15.38:0/2531499038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a780c9cb0 con 0x7f5a7806cae0 2024-04-25T08:41:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.414+0000 7f5a7d4f7640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a7806cae0 0x7f5a780701d0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c02a270 tx=0x7f5a6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:50.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.414+0000 7f5a767fc640 1 -- 172.21.15.38:0/2531499038 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c041020 con 0x7f5a7806cae0 2024-04-25T08:41:50.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.414+0000 7f5a767fc640 1 -- 172.21.15.38:0/2531499038 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c0306c0 con 0x7f5a7806cae0 2024-04-25T08:41:50.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.414+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2531499038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a780ae440 con 0x7f5a7806cae0 2024-04-25T08:41:50.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.414+0000 7f5a767fc640 1 -- 172.21.15.38:0/2531499038 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c037040 con 0x7f5a7806cae0 2024-04-25T08:41:50.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.414+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2531499038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a780aea00 con 0x7f5a7806cae0 2024-04-25T08:41:50.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.415+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2531499038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a78068e80 con 0x7f5a7806cae0 2024-04-25T08:41:50.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.415+0000 7f5a767fc640 1 -- 172.21.15.38:0/2531499038 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a6c044020 con 0x7f5a7806cae0 2024-04-25T08:41:50.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.416+0000 7f5a767fc640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a6406b4d0 0x7f5a6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.416+0000 7f5a767fc640 1 -- 172.21.15.38:0/2531499038 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5a6c0aaab0 con 0x7f5a7806cae0 2024-04-25T08:41:50.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.416+0000 7f5a7dcf8640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a6406b4d0 0x7f5a6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.417+0000 7f5a7dcf8640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a6406b4d0 0x7f5a6406d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5a78070d20 tx=0x7f5a700098d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:50.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.420+0000 7f5a767fc640 1 -- 172.21.15.38:0/2531499038 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a6c04b030 con 0x7f5a7806cae0 2024-04-25T08:41:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.604+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2531499038 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a780b1220 con 0x7f5a7806cae0 2024-04-25T08:41:50.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.604+0000 7f5a767fc640 1 -- 172.21.15.38:0/2531499038 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v133) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a780c9cb0 con 0x7f5a7806cae0 2024-04-25T08:41:50.597 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:41:50.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.606+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a6406b4d0 msgr2=0x7f5a6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.606+0000 7f5a7ff5c640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a6406b4d0 0x7f5a6406d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5a78070d20 tx=0x7f5a700098d0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.606+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2531499038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a7806cae0 msgr2=0x7f5a780701d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.606+0000 7f5a7ff5c640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a7806cae0 0x7f5a780701d0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c02a270 tx=0x7f5a6c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.606+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2531499038 shutdown_connections 2024-04-25T08:41:50.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.606+0000 7f5a7ff5c640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5a6406b4d0 0x7f5a6406d990 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.606+0000 7f5a7ff5c640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5a7815e580 0x7f5a78070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.606+0000 7f5a7ff5c640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5a7806cae0 0x7f5a780701d0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.606+0000 7f5a7ff5c640 1 --2- 172.21.15.38:0/2531499038 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5a7806c700 0x7f5a781f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.606+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2531499038 >> 172.21.15.38:0/2531499038 conn(0x7f5a7808b000 msgr2=0x7f5a7806f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:50.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.606+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2531499038 shutdown_connections 2024-04-25T08:41:50.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.606+0000 7f5a7ff5c640 1 -- 172.21.15.38:0/2531499038 wait complete. 2024-04-25T08:41:50.610 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:50.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.718+0000 7fc911714640 1 Processor -- start 2024-04-25T08:41:50.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.719+0000 7fc911714640 1 -- start start 2024-04-25T08:41:50.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.719+0000 7fc911714640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc90c15dc10 0x7fc90c0ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.719+0000 7fc911714640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc90c0d0f60 0x7fc90c0b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.719+0000 7fc911714640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90c0d3e70 con 0x7fc90c0ced60 2024-04-25T08:41:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.719+0000 7fc911714640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90c15dff0 con 0x7fc90c0d0f60 2024-04-25T08:41:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.719+0000 7fc911714640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90c0b6990 con 0x7fc90c15dc10 2024-04-25T08:41:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.719+0000 7fc90b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc90c0d0f60 0x7fc90c0b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.720+0000 7fc90b7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc90c0d0f60 0x7fc90c0b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60478/0 (socket says 172.21.15.38:60478) 2024-04-25T08:41:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.720+0000 7fc90b7fe640 1 -- 172.21.15.38:0/2163567859 learned_addr learned my addr 172.21.15.38:0/2163567859 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.720+0000 7fc90affd640 1 --2- 172.21.15.38:0/2163567859 >> v2:172.21.15.107:3300/0 conn(0x7fc90c15dc10 0x7fc90c0ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.720+0000 7fc90b7fe640 1 -- 172.21.15.38:0/2163567859 >> v2:172.21.15.107:3300/0 conn(0x7fc90c15dc10 msgr2=0x7fc90c0ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.720+0000 7fc90b7fe640 1 --2- 172.21.15.38:0/2163567859 >> v2:172.21.15.107:3300/0 conn(0x7fc90c15dc10 0x7fc90c0ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.720+0000 7fc90b7fe640 1 -- 172.21.15.38:0/2163567859 >> v1:172.21.15.107:6789/0 conn(0x7fc90c0ced60 legacy=0x7fc90c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.720+0000 7fc90b7fe640 1 -- 172.21.15.38:0/2163567859 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c0d4150 con 0x7fc90c0d0f60 2024-04-25T08:41:50.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.720+0000 7fc90affd640 1 --2- 172.21.15.38:0/2163567859 >> v2:172.21.15.107:3300/0 conn(0x7fc90c15dc10 0x7fc90c0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.721+0000 7fc90b7fe640 1 --2- 172.21.15.38:0/2163567859 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c0d0f60 0x7fc90c0b2370 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0b5d50 tx=0x7fc90402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0534031f1ec9473 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.721+0000 7fc909ffb640 1 -- 172.21.15.38:0/2163567859 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904036070 con 0x7fc90c0d0f60 2024-04-25T08:41:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.721+0000 7fc909ffb640 1 -- 172.21.15.38:0/2163567859 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc904031600 con 0x7fc90c0d0f60 2024-04-25T08:41:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.721+0000 7fc909ffb640 1 -- 172.21.15.38:0/2163567859 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9040319d0 con 0x7fc90c0d0f60 2024-04-25T08:41:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.721+0000 7fc911714640 1 -- 172.21.15.38:0/2163567859 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c0d0f60 msgr2=0x7fc90c0b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.721+0000 7fc911714640 1 --2- 172.21.15.38:0/2163567859 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c0d0f60 0x7fc90c0b2370 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0b5d50 tx=0x7fc90402a6a0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.721+0000 7fc911714640 1 -- 172.21.15.38:0/2163567859 shutdown_connections 2024-04-25T08:41:50.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.721+0000 7fc911714640 1 --2- 172.21.15.38:0/2163567859 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c0d0f60 0x7fc90c0b2370 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.721+0000 7fc911714640 1 --2- 172.21.15.38:0/2163567859 >> v2:172.21.15.107:3300/0 conn(0x7fc90c15dc10 0x7fc90c0ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.721+0000 7fc911714640 1 -- 172.21.15.38:0/2163567859 >> 172.21.15.38:0/2163567859 conn(0x7fc90c066d10 msgr2=0x7fc90c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:50.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.721+0000 7fc911714640 1 -- 172.21.15.38:0/2163567859 shutdown_connections 2024-04-25T08:41:50.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.721+0000 7fc911714640 1 -- 172.21.15.38:0/2163567859 wait complete. 2024-04-25T08:41:50.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.723+0000 7fc911714640 1 Processor -- start 2024-04-25T08:41:50.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.723+0000 7fc911714640 1 -- start start 2024-04-25T08:41:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.723+0000 7fc911714640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc90c0ced60 0x7fc90c0ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.723+0000 7fc911714640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc90c0d0f60 0x7fc90c0d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.723+0000 7fc911714640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c15dc10 0x7fc90c0d3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.723+0000 7fc911714640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0d3900 con 0x7fc90c15dc10 2024-04-25T08:41:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.723+0000 7fc911714640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0b6bc0 con 0x7fc90c0d0f60 2024-04-25T08:41:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.723+0000 7fc911714640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0d3a80 con 0x7fc90c0ced60 2024-04-25T08:41:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.723+0000 7fc90b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c15dc10 0x7fc90c0d3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.723+0000 7fc90b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c15dc10 0x7fc90c0d3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60480/0 (socket says 172.21.15.38:60480) 2024-04-25T08:41:50.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.723+0000 7fc90b7fe640 1 -- 172.21.15.38:0/4082959057 learned_addr learned my addr 172.21.15.38:0/4082959057 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.724+0000 7fc90b7fe640 1 -- 172.21.15.38:0/4082959057 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc90c0ced60 msgr2=0x7fc90c0ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.724+0000 7fc90b7fe640 1 --2- 172.21.15.38:0/4082959057 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc90c0ced60 0x7fc90c0ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.724+0000 7fc90b7fe640 1 -- 172.21.15.38:0/4082959057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc90c0d0f60 msgr2=0x7fc90c0d2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.724+0000 7fc90b7fe640 1 --2- 172.21.15.38:0/4082959057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc90c0d0f60 0x7fc90c0d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.724+0000 7fc90b7fe640 1 -- 172.21.15.38:0/4082959057 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c14c220 con 0x7fc90c15dc10 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.724+0000 7fc90b7fe640 1 --2- 172.21.15.38:0/4082959057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c15dc10 0x7fc90c0d3040 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fc904009f40 tx=0x7fc904004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.724+0000 7fc8f37fe640 1 -- 172.21.15.38:0/4082959057 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904036050 con 0x7fc90c15dc10 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.724+0000 7fc8f37fe640 1 -- 172.21.15.38:0/4082959057 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc904042070 con 0x7fc90c15dc10 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.724+0000 7fc911714640 1 -- 172.21.15.38:0/4082959057 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c14f0c0 con 0x7fc90c15dc10 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.724+0000 7fc8f37fe640 1 -- 172.21.15.38:0/4082959057 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904009510 con 0x7fc90c15dc10 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.724+0000 7fc911714640 1 -- 172.21.15.38:0/4082959057 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c14c7a0 con 0x7fc90c15dc10 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.725+0000 7fc911714640 1 -- 172.21.15.38:0/4082959057 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90c068e80 con 0x7fc90c15dc10 2024-04-25T08:41:50.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.726+0000 7fc8f37fe640 1 -- 172.21.15.38:0/4082959057 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc904007bd0 con 0x7fc90c15dc10 2024-04-25T08:41:50.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.726+0000 7fc8f37fe640 1 --2- 172.21.15.38:0/4082959057 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8f806b4d0 0x7fc8f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:50.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.726+0000 7fc8f37fe640 1 -- 172.21.15.38:0/4082959057 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc9040ab1e0 con 0x7fc90c15dc10 2024-04-25T08:41:50.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.726+0000 7fc90affd640 1 --2- 172.21.15.38:0/4082959057 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8f806b4d0 0x7fc8f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:50.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.727+0000 7fc90affd640 1 --2- 172.21.15.38:0/4082959057 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8f806b4d0 0x7fc8f806d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc006fd0 tx=0x7fc8fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:50.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.730+0000 7fc8f37fe640 1 -- 172.21.15.38:0/4082959057 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc904079f10 con 0x7fc90c15dc10 2024-04-25T08:41:50.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.948+0000 7fc911714640 1 -- 172.21.15.38:0/4082959057 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc90c06af70 con 0x7fc90c15dc10 2024-04-25T08:41:50.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.949+0000 7fc8f37fe640 1 -- 172.21.15.38:0/4082959057 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4158 (secure 0 0 0) 0x7fc904079810 con 0x7fc90c15dc10 2024-04-25T08:41:50.942 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:50.942 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:41:42.972814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9486":{"gid":9486,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":16,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:50.942 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 81 2024-04-25T08:41:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.951+0000 7fc911714640 1 -- 172.21.15.38:0/4082959057 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8f806b4d0 msgr2=0x7fc8f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.951+0000 7fc911714640 1 --2- 172.21.15.38:0/4082959057 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8f806b4d0 0x7fc8f806d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc006fd0 tx=0x7fc8fc008040 comp rx=0 tx=0).stop 2024-04-25T08:41:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.951+0000 7fc911714640 1 -- 172.21.15.38:0/4082959057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c15dc10 msgr2=0x7fc90c0d3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.951+0000 7fc911714640 1 --2- 172.21.15.38:0/4082959057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c15dc10 0x7fc90c0d3040 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fc904009f40 tx=0x7fc904004a40 comp rx=0 tx=0).stop 2024-04-25T08:41:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.951+0000 7fc911714640 1 -- 172.21.15.38:0/4082959057 shutdown_connections 2024-04-25T08:41:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.951+0000 7fc911714640 1 --2- 172.21.15.38:0/4082959057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc90c15dc10 0x7fc90c0d3040 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.951+0000 7fc911714640 1 --2- 172.21.15.38:0/4082959057 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc90c0d0f60 0x7fc90c0d2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.951+0000 7fc911714640 1 --2- 172.21.15.38:0/4082959057 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc8f806b4d0 0x7fc8f806d990 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.951+0000 7fc911714640 1 --2- 172.21.15.38:0/4082959057 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc90c0ced60 0x7fc90c0ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.951+0000 7fc911714640 1 -- 172.21.15.38:0/4082959057 >> 172.21.15.38:0/4082959057 conn(0x7fc90c066d10 msgr2=0x7fc90c15e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.951+0000 7fc911714640 1 -- 172.21.15.38:0/4082959057 shutdown_connections 2024-04-25T08:41:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:50.951+0000 7fc911714640 1 -- 172.21.15.38:0/4082959057 wait complete. 2024-04-25T08:41:50.955 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:41:51.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.065+0000 7feeb7990640 1 Processor -- start 2024-04-25T08:41:51.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.065+0000 7feeb7990640 1 -- start start 2024-04-25T08:41:51.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb7990640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7feeb015ddf0 0x7feeb0161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb7990640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feeb00b93f0 0x7feeb00d2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.059 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb7990640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb00b98a0 con 0x7feeb00b5710 2024-04-25T08:41:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb7990640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb0161e00 con 0x7feeb015ddf0 2024-04-25T08:41:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb7990640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb01620e0 con 0x7feeb00b93f0 2024-04-25T08:41:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb4f2b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7feeb015ddf0 0x7feeb0161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb572c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7feeb00b5710 0x7feeb00b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35810/0 (socket says 172.21.15.38:35810) 2024-04-25T08:41:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb572c640 1 -- 172.21.15.38:0/2208015937 learned_addr learned my addr 172.21.15.38:0/2208015937 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb5f2d640 1 --2- 172.21.15.38:0/2208015937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feeb00b93f0 0x7feeb00d2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb4f2b640 1 -- 172.21.15.38:0/2208015937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feeb00b93f0 msgr2=0x7feeb00d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb4f2b640 1 --2- 172.21.15.38:0/2208015937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feeb00b93f0 0x7feeb00d2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb4f2b640 1 -- 172.21.15.38:0/2208015937 >> v1:172.21.15.38:6789/0 conn(0x7feeb00b5710 legacy=0x7feeb00b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feea7fff640 1 -- 172.21.15.38:0/2208015937 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507208556 0 0) 0x7feeb00b98a0 con 0x7feeb00b5710 2024-04-25T08:41:51.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb4f2b640 1 -- 172.21.15.38:0/2208015937 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb01623c0 con 0x7feeb015ddf0 2024-04-25T08:41:51.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.066+0000 7feeb5f2d640 1 --2- 172.21.15.38:0/2208015937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feeb00b93f0 0x7feeb00d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:51.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.067+0000 7feeb4f2b640 1 --2- 172.21.15.38:0/2208015937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeb015ddf0 0x7feeb0161700 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7feea00097a0 tx=0x7feea002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537c9c1fe5eadd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:51.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.067+0000 7feea7fff640 1 -- 172.21.15.38:0/2208015937 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0037070 con 0x7feeb015ddf0 2024-04-25T08:41:51.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.067+0000 7feea7fff640 1 -- 172.21.15.38:0/2208015937 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea0032440 con 0x7feeb015ddf0 2024-04-25T08:41:51.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.067+0000 7feea7fff640 1 -- 172.21.15.38:0/2208015937 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0032810 con 0x7feeb015ddf0 2024-04-25T08:41:51.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.068+0000 7feeb7990640 1 -- 172.21.15.38:0/2208015937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeb015ddf0 msgr2=0x7feeb0161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.068+0000 7feeb7990640 1 --2- 172.21.15.38:0/2208015937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeb015ddf0 0x7feeb0161700 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7feea00097a0 tx=0x7feea002a180 comp rx=0 tx=0).stop 2024-04-25T08:41:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.068+0000 7feeb7990640 1 -- 172.21.15.38:0/2208015937 shutdown_connections 2024-04-25T08:41:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.068+0000 7feeb7990640 1 --2- 172.21.15.38:0/2208015937 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feeb00b93f0 0x7feeb00d2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.068+0000 7feeb7990640 1 --2- 172.21.15.38:0/2208015937 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeb015ddf0 0x7feeb0161700 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.068+0000 7feeb7990640 1 -- 172.21.15.38:0/2208015937 >> 172.21.15.38:0/2208015937 conn(0x7feeb00d07c0 msgr2=0x7feeb00d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.068+0000 7feeb7990640 1 -- 172.21.15.38:0/2208015937 shutdown_connections 2024-04-25T08:41:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.068+0000 7feeb7990640 1 -- 172.21.15.38:0/2208015937 wait complete. 2024-04-25T08:41:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.069+0000 7feeb7990640 1 Processor -- start 2024-04-25T08:41:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.069+0000 7feeb7990640 1 -- start start 2024-04-25T08:41:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb7990640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeb00b5710 0x7feeb00c7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb7990640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeb00b93f0 0x7feeb014ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb7990640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feeb015ddf0 0x7feeb00cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb7990640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb00d2c10 con 0x7feeb00b93f0 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb7990640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb00d2a90 con 0x7feeb00b5710 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb7990640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb00d2d90 con 0x7feeb015ddf0 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb4f2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeb00b93f0 0x7feeb014ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb4f2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeb00b93f0 0x7feeb014ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60504/0 (socket says 172.21.15.38:60504) 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb4f2b640 1 -- 172.21.15.38:0/2099041881 learned_addr learned my addr 172.21.15.38:0/2099041881 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb572c640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeb00b5710 0x7feeb00c7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb5f2d640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feeb015ddf0 0x7feeb00cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb4f2b640 1 -- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feeb015ddf0 msgr2=0x7feeb00cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb4f2b640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feeb015ddf0 0x7feeb00cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb4f2b640 1 -- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeb00b5710 msgr2=0x7feeb00c7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb4f2b640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeb00b5710 0x7feeb00c7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb4f2b640 1 -- 172.21.15.38:0/2099041881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb00cd800 con 0x7feeb00b93f0 2024-04-25T08:41:51.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.070+0000 7feeb4f2b640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeb00b93f0 0x7feeb014ee90 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7feea00098c0 tx=0x7feea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:51.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.071+0000 7feea5ffb640 1 -- 172.21.15.38:0/2099041881 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0037040 con 0x7feeb00b93f0 2024-04-25T08:41:51.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.071+0000 7feea5ffb640 1 -- 172.21.15.38:0/2099041881 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea0042070 con 0x7feeb00b93f0 2024-04-25T08:41:51.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.071+0000 7feeb7990640 1 -- 172.21.15.38:0/2099041881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb00c7a30 con 0x7feeb00b93f0 2024-04-25T08:41:51.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.071+0000 7feea5ffb640 1 -- 172.21.15.38:0/2099041881 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea0030bf0 con 0x7feeb00b93f0 2024-04-25T08:41:51.066 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.071+0000 7feeb7990640 1 -- 172.21.15.38:0/2099041881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb00c7ff0 con 0x7feeb00b93f0 2024-04-25T08:41:51.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.072+0000 7feeb7990640 1 -- 172.21.15.38:0/2099041881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb00ca9d0 con 0x7feeb00b93f0 2024-04-25T08:41:51.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.072+0000 7feea5ffb640 1 -- 172.21.15.38:0/2099041881 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feea0047460 con 0x7feeb00b93f0 2024-04-25T08:41:51.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.073+0000 7feea5ffb640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fee9406b4d0 0x7fee9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.073+0000 7feea5ffb640 1 -- 172.21.15.38:0/2099041881 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7feea00aade0 con 0x7feeb00b93f0 2024-04-25T08:41:51.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.073+0000 7feeb572c640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fee9406b4d0 0x7fee9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.067 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.074+0000 7feeb572c640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fee9406b4d0 0x7fee9406d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7feeac000f60 tx=0x7feeac007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:51.069 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.077+0000 7feea5ffb640 1 -- 172.21.15.38:0/2099041881 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feeb00ca9d0 con 0x7feeb00b93f0 2024-04-25T08:41:51.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.263+0000 7feeb7990640 1 -- 172.21.15.38:0/2099041881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feeb0068e80 con 0x7feeb00b93f0 2024-04-25T08:41:51.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.264+0000 7feea5ffb640 1 -- 172.21.15.38:0/2099041881 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17017 (secure 0 0 0) 0x7feea0032e20 con 0x7feeb00b93f0 2024-04-25T08:41:51.257 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:51.257 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":133,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:41:40.950430+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:40:52.007975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:40:52.707859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/1312063561":"2024-04-26T08:41:40.950341+0000","172.21.15.38:6834/1312063561":"2024-04-26T08:41:40.950341+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:41:51.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.266+0000 7feeb7990640 1 -- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fee9406b4d0 msgr2=0x7fee9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.266+0000 7feeb7990640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fee9406b4d0 0x7fee9406d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7feeac000f60 tx=0x7feeac007d50 comp rx=0 tx=0).stop 2024-04-25T08:41:51.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.266+0000 7feeb7990640 1 -- 172.21.15.38:0/2099041881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeb00b93f0 msgr2=0x7feeb014ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.266+0000 7feeb7990640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeb00b93f0 0x7feeb014ee90 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7feea00098c0 tx=0x7feea00046c0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.266+0000 7feeb7990640 1 -- 172.21.15.38:0/2099041881 shutdown_connections 2024-04-25T08:41:51.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.266+0000 7feeb7990640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fee9406b4d0 0x7fee9406d990 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.266+0000 7feeb7990640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feeb015ddf0 0x7feeb00cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.266+0000 7feeb7990640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feeb00b93f0 0x7feeb014ee90 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.266+0000 7feeb7990640 1 --2- 172.21.15.38:0/2099041881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feeb00b5710 0x7feeb00c7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.266+0000 7feeb7990640 1 -- 172.21.15.38:0/2099041881 >> 172.21.15.38:0/2099041881 conn(0x7feeb00d07c0 msgr2=0x7feeb0073520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:51.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.266+0000 7feeb7990640 1 -- 172.21.15.38:0/2099041881 shutdown_connections 2024-04-25T08:41:51.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.267+0000 7feeb7990640 1 -- 172.21.15.38:0/2099041881 wait complete. 2024-04-25T08:41:51.271 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:41:51.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:51.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.380+0000 7f2899990640 1 Processor -- start 2024-04-25T08:41:51.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.380+0000 7f2899990640 1 -- start start 2024-04-25T08:41:51.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2899990640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2894161700 0x7f28940b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2899990640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2894161e00 con 0x7f28940b5710 2024-04-25T08:41:51.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2899990640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28941620e0 con 0x7f289415ddf0 2024-04-25T08:41:51.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2899990640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28940b9930 con 0x7f2894161700 2024-04-25T08:41:51.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2892ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f28940b5710 0x7f28940b5af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35818/0 (socket says 172.21.15.38:35818) 2024-04-25T08:41:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2892ffd640 1 -- 172.21.15.38:0/521218675 learned_addr learned my addr 172.21.15.38:0/521218675 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f28937fe640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2894161700 0x7f28940b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108411550 0 0) 0x7f2894161e00 con 0x7f28940b5710 2024-04-25T08:41:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2880003520 con 0x7f28940b5710 2024-04-25T08:41:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 436270404 0 0) 0x7f2880003520 con 0x7f28940b5710 2024-04-25T08:41:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2894161700 msgr2=0x7f28940b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2891ffb640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2894161700 0x7f28940b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 >> v1:172.21.15.107:6789/0 conn(0x7f289415ddf0 legacy=0x7f28940d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28941623c0 con 0x7f28940b5710 2024-04-25T08:41:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f288c001670 con 0x7f28940b5710 2024-04-25T08:41:51.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 >> v1:172.21.15.38:6789/0 conn(0x7f28940b5710 legacy=0x7f28940b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.381+0000 7f2891ffb640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28800039e0 0x7f2880003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f2891ffb640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2880004390 0x7f2880004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f2891ffb640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2894161700 0x7f28800050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288000b9c0 con 0x7f28800039e0 2024-04-25T08:41:51.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288000bb70 con 0x7f2880004390 2024-04-25T08:41:51.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288000bd70 con 0x7f2894161700 2024-04-25T08:41:51.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f2892ffd640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28800039e0 0x7f2880003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f28927fc640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2880004390 0x7f2880004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f28937fe640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2894161700 0x7f28800050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f2892ffd640 1 -- 172.21.15.38:0/521218675 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2894161700 msgr2=0x7f28800050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:51.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f2892ffd640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2894161700 0x7f28800050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.378 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f2892ffd640 1 -- 172.21.15.38:0/521218675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2880004390 msgr2=0x7f2880004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:51.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f2892ffd640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2880004390 0x7f2880004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f2892ffd640 1 -- 172.21.15.38:0/521218675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2880003520 con 0x7f28800039e0 2024-04-25T08:41:51.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.382+0000 7f28937fe640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2894161700 0x7f28800050c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:51.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.383+0000 7f2892ffd640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28800039e0 0x7f2880003dc0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f288c00c290 tx=0x7f2880005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2805094ea538888a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:51.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.383+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c007d50 con 0x7f28800039e0 2024-04-25T08:41:51.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.383+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c005af0 con 0x7f28800039e0 2024-04-25T08:41:51.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.383+0000 7f2891ffb640 1 -- 172.21.15.38:0/521218675 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c007070 con 0x7f28800039e0 2024-04-25T08:41:51.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.383+0000 7f2899990640 1 -- 172.21.15.38:0/521218675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28800039e0 msgr2=0x7f2880003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.383+0000 7f2899990640 1 --2- 172.21.15.38:0/521218675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28800039e0 0x7f2880003dc0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f288c00c290 tx=0x7f2880005830 comp rx=0 tx=0).stop 2024-04-25T08:41:51.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.383+0000 7f28937fe640 1 -- 172.21.15.38:0/521218675 reap_dead start 2024-04-25T08:41:51.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.383+0000 7f2899990640 1 -- 172.21.15.38:0/521218675 shutdown_connections 2024-04-25T08:41:51.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.383+0000 7f2899990640 1 -- 172.21.15.38:0/521218675 >> 172.21.15.38:0/521218675 conn(0x7f28940d07c0 msgr2=0x7f28940d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:51.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.383+0000 7f2899990640 1 -- 172.21.15.38:0/521218675 shutdown_connections 2024-04-25T08:41:51.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.383+0000 7f2899990640 1 -- 172.21.15.38:0/521218675 wait complete. 2024-04-25T08:41:51.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.385+0000 7f2899990640 1 Processor -- start 2024-04-25T08:41:51.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.385+0000 7f2899990640 1 -- start start 2024-04-25T08:41:51.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.385+0000 7f2899990640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28800039e0 0x7f28940cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.385+0000 7f2899990640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28940ccb90 0x7f2894147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.385+0000 7f2899990640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28940c95c0 0x7f2894147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f2899990640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940b9780 con 0x7f28940c95c0 2024-04-25T08:41:51.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f2899990640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940b9600 con 0x7f28940ccb90 2024-04-25T08:41:51.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f2899990640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2894161bb0 con 0x7f28800039e0 2024-04-25T08:41:51.383 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f28937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28940c95c0 0x7f2894147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f28937fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28940c95c0 0x7f2894147da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60518/0 (socket says 172.21.15.38:60518) 2024-04-25T08:41:51.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f28937fe640 1 -- 172.21.15.38:0/695379640 learned_addr learned my addr 172.21.15.38:0/695379640 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:51.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f2892ffd640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28800039e0 0x7f28940cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f28927fc640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28940ccb90 0x7f2894147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f28937fe640 1 -- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28800039e0 msgr2=0x7f28940cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f28937fe640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28800039e0 0x7f28940cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f28937fe640 1 -- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28940ccb90 msgr2=0x7f2894147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f28937fe640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28940ccb90 0x7f2894147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f28937fe640 1 -- 172.21.15.38:0/695379640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940b3380 con 0x7f28940c95c0 2024-04-25T08:41:51.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f28927fc640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28940ccb90 0x7f2894147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:51.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f2892ffd640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28800039e0 0x7f28940cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:51.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f28937fe640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28940c95c0 0x7f2894147da0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f289406d2f0 tx=0x7f2884009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:51.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f288bfff640 1 -- 172.21.15.38:0/695379640 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884010e50 con 0x7f28940c95c0 2024-04-25T08:41:51.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f288bfff640 1 -- 172.21.15.38:0/695379640 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2884003990 con 0x7f28940c95c0 2024-04-25T08:41:51.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f288bfff640 1 -- 172.21.15.38:0/695379640 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884003dc0 con 0x7f28940c95c0 2024-04-25T08:41:51.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f2899990640 1 -- 172.21.15.38:0/695379640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28940b3660 con 0x7f28940c95c0 2024-04-25T08:41:51.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.386+0000 7f2899990640 1 -- 172.21.15.38:0/695379640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28940b3c20 con 0x7f28940c95c0 2024-04-25T08:41:51.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.387+0000 7f2899990640 1 -- 172.21.15.38:0/695379640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2894068e80 con 0x7f28940c95c0 2024-04-25T08:41:51.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.388+0000 7f288bfff640 1 -- 172.21.15.38:0/695379640 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2884005d10 con 0x7f28940c95c0 2024-04-25T08:41:51.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.388+0000 7f288bfff640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f287406b4d0 0x7f287406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.388+0000 7f288bfff640 1 -- 172.21.15.38:0/695379640 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f288408ab40 con 0x7f28940c95c0 2024-04-25T08:41:51.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.389+0000 7f2892ffd640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f287406b4d0 0x7f287406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.389+0000 7f2892ffd640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f287406b4d0 0x7f287406d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f288c033000 tx=0x7f288c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:51.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.392+0000 7f288bfff640 1 -- 172.21.15.38:0/695379640 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2884025020 con 0x7f28940c95c0 2024-04-25T08:41:51.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.609+0000 7f2899990640 1 -- 172.21.15.38:0/695379640 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28940ca0c0 con 0x7f28940c95c0 2024-04-25T08:41:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.609+0000 7f288bfff640 1 -- 172.21.15.38:0/695379640 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f28840597c0 con 0x7f28940c95c0 2024-04-25T08:41:51.602 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:51.602 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:41:42.972814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9486":{"gid":9486,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":16,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:51.602 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 81 2024-04-25T08:41:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.611+0000 7f2899990640 1 -- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f287406b4d0 msgr2=0x7f287406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.611+0000 7f2899990640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f287406b4d0 0x7f287406d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f288c033000 tx=0x7f288c02e000 comp rx=0 tx=0).stop 2024-04-25T08:41:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.611+0000 7f2899990640 1 -- 172.21.15.38:0/695379640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28940c95c0 msgr2=0x7f2894147da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.611+0000 7f2899990640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28940c95c0 0x7f2894147da0 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f289406d2f0 tx=0x7f2884009f90 comp rx=0 tx=0).stop 2024-04-25T08:41:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.611+0000 7f2899990640 1 -- 172.21.15.38:0/695379640 shutdown_connections 2024-04-25T08:41:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.611+0000 7f2899990640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28940c95c0 0x7f2894147da0 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.611+0000 7f2899990640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f28940ccb90 0x7f2894147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.611+0000 7f2899990640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f287406b4d0 0x7f287406d990 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.611+0000 7f2899990640 1 --2- 172.21.15.38:0/695379640 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28800039e0 0x7f28940cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.611+0000 7f2899990640 1 -- 172.21.15.38:0/695379640 >> 172.21.15.38:0/695379640 conn(0x7f28940d07c0 msgr2=0x7f28940d3920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.611+0000 7f2899990640 1 -- 172.21.15.38:0/695379640 shutdown_connections 2024-04-25T08:41:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.611+0000 7f2899990640 1 -- 172.21.15.38:0/695379640 wait complete. 2024-04-25T08:41:51.615 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:51.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.724+0000 7f8db96cd640 1 Processor -- start 2024-04-25T08:41:51.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.724+0000 7f8db96cd640 1 -- start start 2024-04-25T08:41:51.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db96cd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8db40b5710 0x7f8db4161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db96cd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8db415ddf0 0x7f8db40b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db96cd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db4161e00 con 0x7f8db40b5af0 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db96cd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db40b9930 con 0x7f8db40b5710 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db96cd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db41620e0 con 0x7f8db415ddf0 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db2ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8db40b5710 0x7f8db4161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db2ffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8db40b5710 0x7f8db4161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60528/0 (socket says 172.21.15.38:60528) 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db2ffd640 1 -- 172.21.15.38:0/3830262552 learned_addr learned my addr 172.21.15.38:0/3830262552 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db2ffd640 1 -- 172.21.15.38:0/3830262552 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8db415ddf0 msgr2=0x7f8db40b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db27fc640 1 --2- 172.21.15.38:0/3830262552 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8db415ddf0 0x7f8db40b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db2ffd640 1 --2- 172.21.15.38:0/3830262552 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8db415ddf0 0x7f8db40b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db2ffd640 1 -- 172.21.15.38:0/3830262552 >> v1:172.21.15.107:6789/0 conn(0x7f8db40b5af0 legacy=0x7f8db406bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.725+0000 7f8db2ffd640 1 -- 172.21.15.38:0/3830262552 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db41623c0 con 0x7f8db40b5710 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.726+0000 7f8db2ffd640 1 --2- 172.21.15.38:0/3830262552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8db40b5710 0x7f8db4161700 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009870 tx=0x7f8da402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daadd436a9daedb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:51.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.726+0000 7f8db1ffb640 1 -- 172.21.15.38:0/3830262552 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4037070 con 0x7f8db40b5710 2024-04-25T08:41:51.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.726+0000 7f8db1ffb640 1 -- 172.21.15.38:0/3830262552 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da4032440 con 0x7f8db40b5710 2024-04-25T08:41:51.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.726+0000 7f8db1ffb640 1 -- 172.21.15.38:0/3830262552 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4032810 con 0x7f8db40b5710 2024-04-25T08:41:51.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.727+0000 7f8db96cd640 1 -- 172.21.15.38:0/3830262552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8db40b5710 msgr2=0x7f8db4161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.727+0000 7f8db96cd640 1 --2- 172.21.15.38:0/3830262552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8db40b5710 0x7f8db4161700 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009870 tx=0x7f8da402a210 comp rx=0 tx=0).stop 2024-04-25T08:41:51.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.727+0000 7f8db96cd640 1 -- 172.21.15.38:0/3830262552 shutdown_connections 2024-04-25T08:41:51.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.727+0000 7f8db96cd640 1 --2- 172.21.15.38:0/3830262552 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8db415ddf0 0x7f8db40b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.727+0000 7f8db96cd640 1 --2- 172.21.15.38:0/3830262552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8db40b5710 0x7f8db4161700 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.727+0000 7f8db96cd640 1 -- 172.21.15.38:0/3830262552 >> 172.21.15.38:0/3830262552 conn(0x7f8db40d07c0 msgr2=0x7f8db40d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:51.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.727+0000 7f8db96cd640 1 -- 172.21.15.38:0/3830262552 shutdown_connections 2024-04-25T08:41:51.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.727+0000 7f8db96cd640 1 -- 172.21.15.38:0/3830262552 wait complete. 2024-04-25T08:41:51.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.728+0000 7f8db96cd640 1 Processor -- start 2024-04-25T08:41:51.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.728+0000 7f8db96cd640 1 -- start start 2024-04-25T08:41:51.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db96cd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db40b5710 0x7f8db40ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db96cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8db40b5af0 0x7f8db4160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db96cd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8db415ddf0 0x7f8db40b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db96cd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40d34d0 con 0x7f8db40b5af0 2024-04-25T08:41:51.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db96cd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40d3350 con 0x7f8db40b5710 2024-04-25T08:41:51.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db96cd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40d3650 con 0x7f8db415ddf0 2024-04-25T08:41:51.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8db40b5af0 0x7f8db4160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8db40b5af0 0x7f8db4160c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60544/0 (socket says 172.21.15.38:60544) 2024-04-25T08:41:51.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db27fc640 1 -- 172.21.15.38:0/3496675354 learned_addr learned my addr 172.21.15.38:0/3496675354 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:51.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db2ffd640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db40b5710 0x7f8db40ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db37fe640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8db415ddf0 0x7f8db40b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db27fc640 1 -- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8db415ddf0 msgr2=0x7f8db40b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db27fc640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8db415ddf0 0x7f8db40b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db27fc640 1 -- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db40b5710 msgr2=0x7f8db40ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db27fc640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db40b5710 0x7f8db40ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db27fc640 1 -- 172.21.15.38:0/3496675354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db40cafd0 con 0x7f8db40b5af0 2024-04-25T08:41:51.724 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db27fc640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8db40b5af0 0x7f8db4160c40 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f8da800c7f0 tx=0x7f8da800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:51.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8d9b7fe640 1 -- 172.21.15.38:0/3496675354 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8015070 con 0x7f8db40b5af0 2024-04-25T08:41:51.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8d9b7fe640 1 -- 172.21.15.38:0/3496675354 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da8004220 con 0x7f8db40b5af0 2024-04-25T08:41:51.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db96cd640 1 -- 172.21.15.38:0/3496675354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db40cb2b0 con 0x7f8db40b5af0 2024-04-25T08:41:51.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8d9b7fe640 1 -- 172.21.15.38:0/3496675354 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8007620 con 0x7f8db40b5af0 2024-04-25T08:41:51.725 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.729+0000 7f8db96cd640 1 -- 172.21.15.38:0/3496675354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db40cbb50 con 0x7f8db40b5af0 2024-04-25T08:41:51.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.731+0000 7f8db96cd640 1 -- 172.21.15.38:0/3496675354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db40cb480 con 0x7f8db40b5af0 2024-04-25T08:41:51.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.731+0000 7f8d9b7fe640 1 -- 172.21.15.38:0/3496675354 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8da80079d0 con 0x7f8db40b5af0 2024-04-25T08:41:51.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.731+0000 7f8d9b7fe640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8da006b4d0 0x7f8da006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:51.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.731+0000 7f8d9b7fe640 1 -- 172.21.15.38:0/3496675354 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8da808a440 con 0x7f8db40b5af0 2024-04-25T08:41:51.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.732+0000 7f8db2ffd640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8da006b4d0 0x7f8da006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:51.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.732+0000 7f8db2ffd640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8da006b4d0 0x7f8da006d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8da4034000 tx=0x7f8da402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:51.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.736+0000 7f8d9b7fe640 1 -- 172.21.15.38:0/3496675354 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8da80589f0 con 0x7f8db40b5af0 2024-04-25T08:41:51.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.958+0000 7f8db96cd640 1 -- 172.21.15.38:0/3496675354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8db4068e80 con 0x7f8db40b5af0 2024-04-25T08:41:51.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.958+0000 7f8d9b7fe640 1 -- 172.21.15.38:0/3496675354 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4158 (secure 0 0 0) 0x7f8da805ca90 con 0x7f8db40b5af0 2024-04-25T08:41:51.951 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:51.951 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":81,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:40:53.732378+0000","modified":"2024-04-25T08:41:42.972814+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":133,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9486},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9486":{"gid":9486,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":16,"addr":"172.21.15.38:6833/1189096522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1189096522},{"type":"v1","addr":"172.21.15.38:6833","nonce":1189096522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T08:41:51.951 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 81 2024-04-25T08:41:51.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.960+0000 7f8db96cd640 1 -- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8da006b4d0 msgr2=0x7f8da006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.960+0000 7f8db96cd640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8da006b4d0 0x7f8da006d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8da4034000 tx=0x7f8da402e000 comp rx=0 tx=0).stop 2024-04-25T08:41:51.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.960+0000 7f8db96cd640 1 -- 172.21.15.38:0/3496675354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8db40b5af0 msgr2=0x7f8db4160c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:51.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.960+0000 7f8db96cd640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8db40b5af0 0x7f8db4160c40 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f8da800c7f0 tx=0x7f8da800ccb0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.960+0000 7f8db96cd640 1 -- 172.21.15.38:0/3496675354 shutdown_connections 2024-04-25T08:41:51.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.960+0000 7f8db96cd640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8da006b4d0 0x7f8da006d990 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.960+0000 7f8db96cd640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8db415ddf0 0x7f8db40b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.960+0000 7f8db96cd640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8db40b5af0 0x7f8db4160c40 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.960+0000 7f8db96cd640 1 --2- 172.21.15.38:0/3496675354 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8db40b5710 0x7f8db40ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:51.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.960+0000 7f8db96cd640 1 -- 172.21.15.38:0/3496675354 >> 172.21.15.38:0/3496675354 conn(0x7f8db40d07c0 msgr2=0x7f8db406bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:51.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.960+0000 7f8db96cd640 1 -- 172.21.15.38:0/3496675354 shutdown_connections 2024-04-25T08:41:51.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:51.960+0000 7f8db96cd640 1 -- 172.21.15.38:0/3496675354 wait complete. 2024-04-25T08:41:51.964 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:41:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.076+0000 7fc5c458b640 1 Processor -- start 2024-04-25T08:41:52.068 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.076+0000 7fc5c458b640 1 -- start start 2024-04-25T08:41:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c458b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5bc15dbc0 0x7fc5bc072b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c458b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5bc0b2220 0x7fc5bc0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c458b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5bc15dfa0 con 0x7fc5bc06abe0 2024-04-25T08:41:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c458b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b2bc0 con 0x7fc5bc0b2220 2024-04-25T08:41:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c458b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b3970 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c1b26640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5bc0b2220 0x7fc5bc0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c2b28640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc5bc06abe0 0x7fc5bc06b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35820/0 (socket says 172.21.15.38:35820) 2024-04-25T08:41:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c2b28640 1 -- 172.21.15.38:0/3092594953 learned_addr learned my addr 172.21.15.38:0/3092594953 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c2327640 1 --2- 172.21.15.38:0/3092594953 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5bc15dbc0 0x7fc5bc072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.070 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c1b26640 1 -- 172.21.15.38:0/3092594953 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5bc15dbc0 msgr2=0x7fc5bc072b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:52.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c1b26640 1 --2- 172.21.15.38:0/3092594953 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5bc15dbc0 0x7fc5bc072b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c1b26640 1 -- 172.21.15.38:0/3092594953 >> v1:172.21.15.38:6789/0 conn(0x7fc5bc06abe0 legacy=0x7fc5bc06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c1b26640 1 -- 172.21.15.38:0/3092594953 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc0b3c50 con 0x7fc5bc0b2220 2024-04-25T08:41:52.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c1325640 1 -- 172.21.15.38:0/3092594953 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859620256 0 0) 0x7fc5bc15dfa0 con 0x7fc5bc06abe0 2024-04-25T08:41:52.071 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.077+0000 7fc5c2327640 1 --2- 172.21.15.38:0/3092594953 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5bc15dbc0 0x7fc5bc072b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:52.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.078+0000 7fc5c1b26640 1 --2- 172.21.15.38:0/3092594953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5bc0b2220 0x7fc5bc0b2680 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0097a0 tx=0x7fc5ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f02abb954a5f195 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:52.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.078+0000 7fc5c1325640 1 -- 172.21.15.38:0/3092594953 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac037070 con 0x7fc5bc0b2220 2024-04-25T08:41:52.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.078+0000 7fc5c1325640 1 -- 172.21.15.38:0/3092594953 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ac032440 con 0x7fc5bc0b2220 2024-04-25T08:41:52.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.078+0000 7fc5c1325640 1 -- 172.21.15.38:0/3092594953 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac032810 con 0x7fc5bc0b2220 2024-04-25T08:41:52.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.079+0000 7fc5c458b640 1 -- 172.21.15.38:0/3092594953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5bc0b2220 msgr2=0x7fc5bc0b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.079+0000 7fc5c458b640 1 --2- 172.21.15.38:0/3092594953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5bc0b2220 0x7fc5bc0b2680 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac0097a0 tx=0x7fc5ac02a180 comp rx=0 tx=0).stop 2024-04-25T08:41:52.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.079+0000 7fc5c458b640 1 -- 172.21.15.38:0/3092594953 shutdown_connections 2024-04-25T08:41:52.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.079+0000 7fc5c458b640 1 --2- 172.21.15.38:0/3092594953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5bc0b2220 0x7fc5bc0b2680 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.079+0000 7fc5c458b640 1 --2- 172.21.15.38:0/3092594953 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5bc15dbc0 0x7fc5bc072b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.079+0000 7fc5c458b640 1 -- 172.21.15.38:0/3092594953 >> 172.21.15.38:0/3092594953 conn(0x7fc5bc0b5c90 msgr2=0x7fc5bc0b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:52.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.079+0000 7fc5c458b640 1 -- 172.21.15.38:0/3092594953 shutdown_connections 2024-04-25T08:41:52.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.079+0000 7fc5c458b640 1 -- 172.21.15.38:0/3092594953 wait complete. 2024-04-25T08:41:52.073 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.080+0000 7fc5c458b640 1 Processor -- start 2024-04-25T08:41:52.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.080+0000 7fc5c458b640 1 -- start start 2024-04-25T08:41:52.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c458b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5bc06abe0 0x7fc5bc158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c458b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5bc0b2220 0x7fc5bc15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c458b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5bc15dbc0 0x7fc5bc152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c458b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b44f0 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c458b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b4210 con 0x7fc5bc06abe0 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c458b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc0b3590 con 0x7fc5bc0b2220 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c2b28640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5bc15dbc0 0x7fc5bc152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c2b28640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5bc15dbc0 0x7fc5bc152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60558/0 (socket says 172.21.15.38:60558) 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c2b28640 1 -- 172.21.15.38:0/2472656632 learned_addr learned my addr 172.21.15.38:0/2472656632 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c1b26640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5bc0b2220 0x7fc5bc15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c2327640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5bc06abe0 0x7fc5bc158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c2b28640 1 -- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5bc0b2220 msgr2=0x7fc5bc15b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c2b28640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5bc0b2220 0x7fc5bc15b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c2b28640 1 -- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5bc06abe0 msgr2=0x7fc5bc158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c2b28640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5bc06abe0 0x7fc5bc158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c2b28640 1 -- 172.21.15.38:0/2472656632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc159300 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5c2b28640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5bc15dbc0 0x7fc5bc152150 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400c800 tx=0x7fc5b400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:52.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.081+0000 7fc5b2ffd640 1 -- 172.21.15.38:0/2472656632 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4018070 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.082+0000 7fc5b2ffd640 1 -- 172.21.15.38:0/2472656632 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b4004d80 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.082+0000 7fc5c458b640 1 -- 172.21.15.38:0/2472656632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc0cd300 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.082+0000 7fc5b2ffd640 1 -- 172.21.15.38:0/2472656632 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b400a880 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.082+0000 7fc5c458b640 1 -- 172.21.15.38:0/2472656632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc0cd8c0 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.083+0000 7fc5c458b640 1 -- 172.21.15.38:0/2472656632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc068e80 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.083+0000 7fc5b2ffd640 1 -- 172.21.15.38:0/2472656632 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5b401b070 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.083+0000 7fc5b2ffd640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5a006b500 0x7fc5a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.083+0000 7fc5b2ffd640 1 -- 172.21.15.38:0/2472656632 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc5b4089950 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.084+0000 7fc5c2327640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5a006b500 0x7fc5a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.084+0000 7fc5c2327640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5a006b500 0x7fc5a006d9c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800a950 tx=0x7fc5b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:52.080 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.088+0000 7fc5b2ffd640 1 -- 172.21.15.38:0/2472656632 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5b40586b0 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.268+0000 7fc5c458b640 1 -- 172.21.15.38:0/2472656632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5bc15c360 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.269+0000 7fc5b2ffd640 1 -- 172.21.15.38:0/2472656632 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v133) v1 ==== 74+0+17017 (secure 0 0 0) 0x7fc5b4057fb0 con 0x7fc5bc15dbc0 2024-04-25T08:41:52.262 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:52.262 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":133,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:41:40.950430+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:40:52.007975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T08:40:52.707859+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/1312063561":"2024-04-26T08:41:40.950341+0000","172.21.15.38:6834/1312063561":"2024-04-26T08:41:40.950341+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:41:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.271+0000 7fc5c458b640 1 -- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5a006b500 msgr2=0x7fc5a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.271+0000 7fc5c458b640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5a006b500 0x7fc5a006d9c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800a950 tx=0x7fc5b8008040 comp rx=0 tx=0).stop 2024-04-25T08:41:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.271+0000 7fc5c458b640 1 -- 172.21.15.38:0/2472656632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5bc15dbc0 msgr2=0x7fc5bc152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.271+0000 7fc5c458b640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5bc15dbc0 0x7fc5bc152150 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400c800 tx=0x7fc5b400ccc0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.271+0000 7fc5c458b640 1 -- 172.21.15.38:0/2472656632 shutdown_connections 2024-04-25T08:41:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.271+0000 7fc5c458b640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5bc15dbc0 0x7fc5bc152150 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.271+0000 7fc5c458b640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5a006b500 0x7fc5a006d9c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.271+0000 7fc5c458b640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5bc0b2220 0x7fc5bc15b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.271+0000 7fc5c458b640 1 --2- 172.21.15.38:0/2472656632 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5bc06abe0 0x7fc5bc158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.271+0000 7fc5c458b640 1 -- 172.21.15.38:0/2472656632 >> 172.21.15.38:0/2472656632 conn(0x7fc5bc0b5c90 msgr2=0x7fc5bc0cbac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:52.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.271+0000 7fc5c458b640 1 -- 172.21.15.38:0/2472656632 shutdown_connections 2024-04-25T08:41:52.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.271+0000 7fc5c458b640 1 -- 172.21.15.38:0/2472656632 wait complete. 2024-04-25T08:41:52.275 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:41:52.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.385+0000 7f9cff088640 1 Processor -- start 2024-04-25T08:41:52.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.385+0000 7f9cff088640 1 -- start start 2024-04-25T08:41:52.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.385+0000 7f9cff088640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9cf8099a70 0x7f9cf8099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.385+0000 7f9cff088640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf806d9f0 con 0x7f9cf815f990 2024-04-25T08:41:52.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.385+0000 7f9cff088640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf809a390 con 0x7f9cf80cf7b0 2024-04-25T08:41:52.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.385+0000 7f9cff088640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf806cc30 con 0x7f9cf8099a70 2024-04-25T08:41:52.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.386+0000 7f9cfce24640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9cf8099a70 0x7f9cf8099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.386+0000 7f9cfd625640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9cf815f990 0x7f9cf815fda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35826/0 (socket says 172.21.15.38:35826) 2024-04-25T08:41:52.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.386+0000 7f9cfd625640 1 -- 172.21.15.38:0/2955207185 learned_addr learned my addr 172.21.15.38:0/2955207185 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:52.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.386+0000 7f9cfce24640 1 -- 172.21.15.38:0/2955207185 >> v1:172.21.15.107:6789/0 conn(0x7f9cf80cf7b0 legacy=0x7f9cf806c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:52.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.386+0000 7f9cfce24640 1 -- 172.21.15.38:0/2955207185 >> v1:172.21.15.38:6789/0 conn(0x7f9cf815f990 legacy=0x7f9cf815fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.386+0000 7f9cf77fe640 1 -- 172.21.15.38:0/2955207185 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688289448 0 0) 0x7f9cf806d9f0 con 0x7f9cf815f990 2024-04-25T08:41:52.379 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.386+0000 7f9cfce24640 1 -- 172.21.15.38:0/2955207185 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf806ce70 con 0x7f9cf8099a70 2024-04-25T08:41:52.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.387+0000 7f9cfce24640 1 --2- 172.21.15.38:0/2955207185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf8099a70 0x7f9cf8099e50 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0097a0 tx=0x7f9cec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9654778c2d99d9dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:52.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.387+0000 7f9cf77fe640 1 -- 172.21.15.38:0/2955207185 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec037070 con 0x7f9cf8099a70 2024-04-25T08:41:52.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.387+0000 7f9cf77fe640 1 -- 172.21.15.38:0/2955207185 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec032440 con 0x7f9cf8099a70 2024-04-25T08:41:52.381 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.387+0000 7f9cf77fe640 1 -- 172.21.15.38:0/2955207185 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec032810 con 0x7f9cf8099a70 2024-04-25T08:41:52.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.387+0000 7f9cff088640 1 -- 172.21.15.38:0/2955207185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf8099a70 msgr2=0x7f9cf8099e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.387+0000 7f9cff088640 1 --2- 172.21.15.38:0/2955207185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf8099a70 0x7f9cf8099e50 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0097a0 tx=0x7f9cec02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.387+0000 7f9cff088640 1 -- 172.21.15.38:0/2955207185 shutdown_connections 2024-04-25T08:41:52.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.387+0000 7f9cff088640 1 --2- 172.21.15.38:0/2955207185 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf8099a70 0x7f9cf8099e50 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.387+0000 7f9cff088640 1 -- 172.21.15.38:0/2955207185 >> 172.21.15.38:0/2955207185 conn(0x7f9cf80b3190 msgr2=0x7f9cf80b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:52.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.387+0000 7f9cff088640 1 -- 172.21.15.38:0/2955207185 shutdown_connections 2024-04-25T08:41:52.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.387+0000 7f9cff088640 1 -- 172.21.15.38:0/2955207185 wait complete. 2024-04-25T08:41:52.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.389+0000 7f9cff088640 1 Processor -- start 2024-04-25T08:41:52.382 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.389+0000 7f9cff088640 1 -- start start 2024-04-25T08:41:52.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.389+0000 7f9cff088640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf8099a70 0x7f9cf80c6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.389+0000 7f9cff088640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cf80cf7b0 0x7f9cf80c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.389+0000 7f9cff088640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cf815f990 0x7f9cf8152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.389+0000 7f9cff088640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf806d710 con 0x7f9cf8099a70 2024-04-25T08:41:52.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.389+0000 7f9cff088640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf806d430 con 0x7f9cf815f990 2024-04-25T08:41:52.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.389+0000 7f9cff088640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80d08a0 con 0x7f9cf80cf7b0 2024-04-25T08:41:52.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cfce24640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf8099a70 0x7f9cf80c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cfce24640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf8099a70 0x7f9cf80c6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60572/0 (socket says 172.21.15.38:60572) 2024-04-25T08:41:52.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cfce24640 1 -- 172.21.15.38:0/155915219 learned_addr learned my addr 172.21.15.38:0/155915219 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:52.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cf7fff640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cf80cf7b0 0x7f9cf80c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cfd625640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cf815f990 0x7f9cf8152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cfce24640 1 -- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cf80cf7b0 msgr2=0x7f9cf80c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cfce24640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cf80cf7b0 0x7f9cf80c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cfce24640 1 -- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cf815f990 msgr2=0x7f9cf8152b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cfce24640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cf815f990 0x7f9cf8152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cfce24640 1 -- 172.21.15.38:0/155915219 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf8153090 con 0x7f9cf8099a70 2024-04-25T08:41:52.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cfce24640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf8099a70 0x7f9cf80c6530 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0098c0 tx=0x7f9cec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:52.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cf57fa640 1 -- 172.21.15.38:0/155915219 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec037040 con 0x7f9cf8099a70 2024-04-25T08:41:52.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cf57fa640 1 -- 172.21.15.38:0/155915219 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec030770 con 0x7f9cf8099a70 2024-04-25T08:41:52.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cff088640 1 -- 172.21.15.38:0/155915219 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf80c6c30 con 0x7f9cf8099a70 2024-04-25T08:41:52.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cf57fa640 1 -- 172.21.15.38:0/155915219 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec030b40 con 0x7f9cf8099a70 2024-04-25T08:41:52.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.390+0000 7f9cff088640 1 -- 172.21.15.38:0/155915219 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf80c71f0 con 0x7f9cf8099a70 2024-04-25T08:41:52.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.391+0000 7f9cff088640 1 -- 172.21.15.38:0/155915219 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf804ef90 con 0x7f9cf8099a70 2024-04-25T08:41:52.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.392+0000 7f9cf57fa640 1 -- 172.21.15.38:0/155915219 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9cec044020 con 0x7f9cf8099a70 2024-04-25T08:41:52.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.392+0000 7f9cf57fa640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9cdc06b4d0 0x7f9cdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.393+0000 7f9cf57fa640 1 -- 172.21.15.38:0/155915219 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9cec0aa9b0 con 0x7f9cf8099a70 2024-04-25T08:41:52.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.393+0000 7f9cf7fff640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9cdc06b4d0 0x7f9cdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.393+0000 7f9cf7fff640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9cdc06b4d0 0x7f9cdc06d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80b7380 tx=0x7f9ce4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:52.389 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.397+0000 7f9cf57fa640 1 -- 172.21.15.38:0/155915219 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9cf8153090 con 0x7f9cf8099a70 2024-04-25T08:41:52.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.616+0000 7f9cff088640 1 -- 172.21.15.38:0/155915219 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9cf80b7530 con 0x7f9cf8099a70 2024-04-25T08:41:52.609 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:41:52.617+0000 7ff7937f0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:41:52.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.618+0000 7f9cf57fa640 1 -- 172.21.15.38:0/155915219 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9cec032e20 con 0x7f9cf8099a70 2024-04-25T08:41:52.611 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:41:52.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.620+0000 7f9cff088640 1 -- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9cdc06b4d0 msgr2=0x7f9cdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.620+0000 7f9cff088640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9cdc06b4d0 0x7f9cdc06d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9cf80b7380 tx=0x7f9ce4009040 comp rx=0 tx=0).stop 2024-04-25T08:41:52.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.620+0000 7f9cff088640 1 -- 172.21.15.38:0/155915219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf8099a70 msgr2=0x7f9cf80c6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.620+0000 7f9cff088640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf8099a70 0x7f9cf80c6530 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f9cec0098c0 tx=0x7f9cec0046c0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.620+0000 7f9cff088640 1 -- 172.21.15.38:0/155915219 shutdown_connections 2024-04-25T08:41:52.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.620+0000 7f9cff088640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cf815f990 0x7f9cf8152b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.620+0000 7f9cff088640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9cdc06b4d0 0x7f9cdc06d990 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.620+0000 7f9cff088640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cf80cf7b0 0x7f9cf80c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.620+0000 7f9cff088640 1 --2- 172.21.15.38:0/155915219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9cf8099a70 0x7f9cf80c6530 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.620+0000 7f9cff088640 1 -- 172.21.15.38:0/155915219 >> 172.21.15.38:0/155915219 conn(0x7f9cf80b3190 msgr2=0x7f9cf815eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:52.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.621+0000 7f9cff088640 1 -- 172.21.15.38:0/155915219 shutdown_connections 2024-04-25T08:41:52.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.621+0000 7f9cff088640 1 -- 172.21.15.38:0/155915219 wait complete. 2024-04-25T08:41:52.626 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:41:52.651 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:41:52.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.737+0000 7f8296342640 1 Processor -- start 2024-04-25T08:41:52.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.737+0000 7f8296342640 1 -- start start 2024-04-25T08:41:52.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.738+0000 7f8296342640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f82900b1d60 0x7f82900cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.738+0000 7f8296342640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900cff10 0x7f8290161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.738+0000 7f8296342640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f829015ef40 con 0x7f8290161c50 2024-04-25T08:41:52.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.738+0000 7f8296342640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f829006abe0 con 0x7f82900b1d60 2024-04-25T08:41:52.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.738+0000 7f8296342640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829006b480 con 0x7f82900cff10 2024-04-25T08:41:52.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.738+0000 7f82948df640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f8290161c50 0x7f82900cce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35834/0 (socket says 172.21.15.38:35834) 2024-04-25T08:41:52.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.738+0000 7f82948df640 1 -- 172.21.15.38:0/1313248051 learned_addr learned my addr 172.21.15.38:0/1313248051 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:52.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828f7fe640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900cff10 0x7f8290161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828ffff640 1 --2- 172.21.15.38:0/1313248051 >> v2:172.21.15.107:3300/0 conn(0x7f82900b1d60 0x7f82900cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958541530 0 0) 0x7f829015ef40 con 0x7f8290161c50 2024-04-25T08:41:52.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8278003520 con 0x7f8290161c50 2024-04-25T08:41:52.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3539948079 0 0) 0x7f8278003520 con 0x7f8290161c50 2024-04-25T08:41:52.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900cff10 msgr2=0x7f8290161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:52.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900cff10 0x7f8290161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 >> v2:172.21.15.107:3300/0 conn(0x7f82900b1d60 msgr2=0x7f82900cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 --2- 172.21.15.38:0/1313248051 >> v2:172.21.15.107:3300/0 conn(0x7f82900b1d60 0x7f82900cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829006ae30 con 0x7f8290161c50 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f82880019f0 con 0x7f8290161c50 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 >> v1:172.21.15.38:6789/0 conn(0x7f8290161c50 legacy=0x7f82900cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82780039b0 0x7f8278003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8278004800 0x7f8278004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900cff10 0x7f82780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829006abe0 con 0x7f8278004800 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829006b480 con 0x7f82780039b0 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f827800b9c0 con 0x7f82900cff10 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828f7fe640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8278004800 0x7f8278004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828ffff640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82780039b0 0x7f8278003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f82948df640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900cff10 0x7f82780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828f7fe640 1 -- 172.21.15.38:0/1313248051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900cff10 msgr2=0x7f82780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:52.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828f7fe640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900cff10 0x7f82780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828f7fe640 1 -- 172.21.15.38:0/1313248051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82780039b0 msgr2=0x7f8278003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:52.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828f7fe640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f82780039b0 0x7f8278003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.739+0000 7f828f7fe640 1 -- 172.21.15.38:0/1313248051 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8278003520 con 0x7f8278004800 2024-04-25T08:41:52.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.740+0000 7f82948df640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900cff10 0x7f82780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:52.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.740+0000 7f828f7fe640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8278004800 0x7f8278004c80 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f8284009ac0 tx=0x7f8278005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf0d5165aabf198 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:52.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.740+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284036070 con 0x7f8278004800 2024-04-25T08:41:52.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.740+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8284031400 con 0x7f8278004800 2024-04-25T08:41:52.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.740+0000 7f828effd640 1 -- 172.21.15.38:0/1313248051 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284009690 con 0x7f8278004800 2024-04-25T08:41:52.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.741+0000 7f8296342640 1 -- 172.21.15.38:0/1313248051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8278004800 msgr2=0x7f8278004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.741+0000 7f8296342640 1 --2- 172.21.15.38:0/1313248051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8278004800 0x7f8278004c80 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f8284009ac0 tx=0x7f8278005830 comp rx=0 tx=0).stop 2024-04-25T08:41:52.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.741+0000 7f82948df640 1 -- 172.21.15.38:0/1313248051 reap_dead start 2024-04-25T08:41:52.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.741+0000 7f8296342640 1 -- 172.21.15.38:0/1313248051 shutdown_connections 2024-04-25T08:41:52.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.741+0000 7f8296342640 1 -- 172.21.15.38:0/1313248051 >> 172.21.15.38:0/1313248051 conn(0x7f82900d06b0 msgr2=0x7f82900d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:52.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.741+0000 7f8296342640 1 -- 172.21.15.38:0/1313248051 shutdown_connections 2024-04-25T08:41:52.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.741+0000 7f8296342640 1 -- 172.21.15.38:0/1313248051 wait complete. 2024-04-25T08:41:52.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.742+0000 7f8296342640 1 Processor -- start 2024-04-25T08:41:52.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.742+0000 7f8296342640 1 -- start start 2024-04-25T08:41:52.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f8296342640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278004800 0x7f829009b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f8296342640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900c9a00 0x7f829015bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f8296342640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900c9e80 0x7f829015c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f8296342640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829006b910 con 0x7f82900c9e80 2024-04-25T08:41:52.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f8296342640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82900d0370 con 0x7f8278004800 2024-04-25T08:41:52.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f8296342640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829006ba90 con 0x7f82900c9a00 2024-04-25T08:41:52.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f82948df640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900c9e80 0x7f829015c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f82948df640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900c9e80 0x7f829015c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60598/0 (socket says 172.21.15.38:60598) 2024-04-25T08:41:52.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f82948df640 1 -- 172.21.15.38:0/660879295 learned_addr learned my addr 172.21.15.38:0/660879295 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:52.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f828f7fe640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900c9a00 0x7f829015bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f828ffff640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278004800 0x7f829009b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f82948df640 1 -- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900c9a00 msgr2=0x7f829015bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f82948df640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900c9a00 0x7f829015bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f82948df640 1 -- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278004800 msgr2=0x7f829009b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:52.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f82948df640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278004800 0x7f829009b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:52.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f82948df640 1 -- 172.21.15.38:0/660879295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82901e9640 con 0x7f82900c9e80 2024-04-25T08:41:52.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f828f7fe640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900c9a00 0x7f829015bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:52.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.743+0000 7f828ffff640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278004800 0x7f829009b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:52.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.744+0000 7f82948df640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900c9e80 0x7f829015c0f0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f82900cebc0 tx=0x7f828800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:52.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.744+0000 7f828cff9640 1 -- 172.21.15.38:0/660879295 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288007a80 con 0x7f82900c9e80 2024-04-25T08:41:52.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.744+0000 7f828cff9640 1 -- 172.21.15.38:0/660879295 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828800eda0 con 0x7f82900c9e80 2024-04-25T08:41:52.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.744+0000 7f828cff9640 1 -- 172.21.15.38:0/660879295 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288014670 con 0x7f82900c9e80 2024-04-25T08:41:52.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.744+0000 7f8296342640 1 -- 172.21.15.38:0/660879295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82900c4310 con 0x7f82900c9e80 2024-04-25T08:41:52.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.744+0000 7f8296342640 1 -- 172.21.15.38:0/660879295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82900c45b0 con 0x7f82900c9e80 2024-04-25T08:41:52.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.745+0000 7f8296342640 1 -- 172.21.15.38:0/660879295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8290068e80 con 0x7f82900c9e80 2024-04-25T08:41:52.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.745+0000 7f828cff9640 1 -- 172.21.15.38:0/660879295 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8288014810 con 0x7f82900c9e80 2024-04-25T08:41:52.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.746+0000 7f828cff9640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f827c06b4d0 0x7f827c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:52.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.746+0000 7f828cff9640 1 -- 172.21.15.38:0/660879295 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f828808a2f0 con 0x7f82900c9e80 2024-04-25T08:41:52.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.746+0000 7f828ffff640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f827c06b4d0 0x7f827c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:52.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.747+0000 7f828ffff640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f827c06b4d0 0x7f827c06d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8280009c40 tx=0x7f8280008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:52.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.750+0000 7f828cff9640 1 -- 172.21.15.38:0/660879295 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8288058fc0 con 0x7f82900c9e80 2024-04-25T08:41:52.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:52.972+0000 7f8296342640 1 -- 172.21.15.38:0/660879295 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f829006c9e0 con 0x7f82900c9e80 2024-04-25T08:41:53.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.620+0000 7f828cff9640 1 -- 172.21.15.38:0/660879295 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v83) v1 ==== 104+0+0 (secure 0 0 0) 0x7f82880588c0 con 0x7f82900c9e80 2024-04-25T08:41:53.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.622+0000 7f8296342640 1 -- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f827c06b4d0 msgr2=0x7f827c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:53.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.622+0000 7f8296342640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f827c06b4d0 0x7f827c06d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8280009c40 tx=0x7f8280008040 comp rx=0 tx=0).stop 2024-04-25T08:41:53.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.622+0000 7f8296342640 1 -- 172.21.15.38:0/660879295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900c9e80 msgr2=0x7f829015c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:53.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.622+0000 7f8296342640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900c9e80 0x7f829015c0f0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f82900cebc0 tx=0x7f828800e530 comp rx=0 tx=0).stop 2024-04-25T08:41:53.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.623+0000 7f8296342640 1 -- 172.21.15.38:0/660879295 shutdown_connections 2024-04-25T08:41:53.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.623+0000 7f8296342640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f82900c9e80 0x7f829015c0f0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:53.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.623+0000 7f8296342640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f827c06b4d0 0x7f827c06d990 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:53.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.623+0000 7f8296342640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f82900c9a00 0x7f829015bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:53.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.623+0000 7f8296342640 1 --2- 172.21.15.38:0/660879295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8278004800 0x7f829009b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:53.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.623+0000 7f8296342640 1 -- 172.21.15.38:0/660879295 >> 172.21.15.38:0/660879295 conn(0x7f82900d06b0 msgr2=0x7f829015f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:53.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.623+0000 7f8296342640 1 -- 172.21.15.38:0/660879295 shutdown_connections 2024-04-25T08:41:53.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.623+0000 7f8296342640 1 -- 172.21.15.38:0/660879295 wait complete. 2024-04-25T08:41:53.630 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:41:53.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.791+0000 7efd11161640 1 Processor -- start 2024-04-25T08:41:53.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.791+0000 7efd11161640 1 -- start start 2024-04-25T08:41:53.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.792+0000 7efd11161640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd0c0d0870 0x7efd0c0d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:53.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.792+0000 7efd11161640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c0d11c0 con 0x7efd0c1612c0 2024-04-25T08:41:53.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.792+0000 7efd11161640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd0c0b9880 con 0x7efd0c0b93f0 2024-04-25T08:41:53.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.792+0000 7efd11161640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c0b7200 con 0x7efd0c0d0870 2024-04-25T08:41:53.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.792+0000 7efd0ad9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7efd0c1612c0 0x7efd0c0b6a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35846/0 (socket says 172.21.15.38:35846) 2024-04-25T08:41:53.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.792+0000 7efd0ad9d640 1 -- 172.21.15.38:0/3522812405 learned_addr learned my addr 172.21.15.38:0/3522812405 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:53.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.792+0000 7efd0b59e640 1 --2- 172.21.15.38:0/3522812405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd0c0d0870 0x7efd0c0d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:53.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.792+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252529662 0 0) 0x7efd04001110 con 0x7efd0c1612c0 2024-04-25T08:41:53.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.792+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcf8003520 con 0x7efd0c1612c0 2024-04-25T08:41:53.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3787044225 0 0) 0x7efcf8003520 con 0x7efd0c1612c0 2024-04-25T08:41:53.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd0c0d0870 msgr2=0x7efd0c0d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:53.788 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 --2- 172.21.15.38:0/3522812405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd0c0d0870 0x7efd0c0d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 >> v1:172.21.15.107:6789/0 conn(0x7efd0c0b93f0 legacy=0x7efd0c0d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c0b74e0 con 0x7efd0c1612c0 2024-04-25T08:41:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7efd04001840 con 0x7efd0c1612c0 2024-04-25T08:41:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 >> v1:172.21.15.38:6789/0 conn(0x7efd0c1612c0 legacy=0x7efd0c0b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 --2- 172.21.15.38:0/3522812405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd0c0d0870 0x7efcf80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 --2- 172.21.15.38:0/3522812405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf8003f20 0x7efcf8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 --2- 172.21.15.38:0/3522812405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf8004d40 0x7efcf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c0b7200 con 0x7efcf8004d40 2024-04-25T08:41:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf800bad0 con 0x7efcf8003f20 2024-04-25T08:41:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf800bcd0 con 0x7efd0c0d0870 2024-04-25T08:41:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd0b59e640 1 --2- 172.21.15.38:0/3522812405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf8004d40 0x7efcf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:53.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd0ad9d640 1 --2- 172.21.15.38:0/3522812405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd0c0d0870 0x7efcf80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd0a59c640 1 --2- 172.21.15.38:0/3522812405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf8003f20 0x7efcf8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd0b59e640 1 -- 172.21.15.38:0/3522812405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd0c0d0870 msgr2=0x7efcf80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd0b59e640 1 --2- 172.21.15.38:0/3522812405 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd0c0d0870 0x7efcf80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd0b59e640 1 -- 172.21.15.38:0/3522812405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf8003f20 msgr2=0x7efcf8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd0b59e640 1 --2- 172.21.15.38:0/3522812405 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf8003f20 0x7efcf8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.793+0000 7efd0b59e640 1 -- 172.21.15.38:0/3522812405 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf8003520 con 0x7efcf8004d40 2024-04-25T08:41:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.794+0000 7efd0b59e640 1 --2- 172.21.15.38:0/3522812405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf8004d40 0x7efcf80051c0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009fc0 tx=0x7efcf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f1a8f3155196a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.794+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc009c10 con 0x7efcf8004d40 2024-04-25T08:41:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.794+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc036070 con 0x7efcf8004d40 2024-04-25T08:41:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.794+0000 7efd09d9b640 1 -- 172.21.15.38:0/3522812405 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc031880 con 0x7efcf8004d40 2024-04-25T08:41:53.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.795+0000 7efd11161640 1 -- 172.21.15.38:0/3522812405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf8004d40 msgr2=0x7efcf80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:53.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.795+0000 7efd11161640 1 --2- 172.21.15.38:0/3522812405 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf8004d40 0x7efcf80051c0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009fc0 tx=0x7efcf8005830 comp rx=0 tx=0).stop 2024-04-25T08:41:53.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.795+0000 7efd0b59e640 1 -- 172.21.15.38:0/3522812405 reap_dead start 2024-04-25T08:41:53.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.795+0000 7efd11161640 1 -- 172.21.15.38:0/3522812405 shutdown_connections 2024-04-25T08:41:53.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.795+0000 7efd11161640 1 -- 172.21.15.38:0/3522812405 >> 172.21.15.38:0/3522812405 conn(0x7efd0c066d10 msgr2=0x7efd0c067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:53.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.795+0000 7efd11161640 1 -- 172.21.15.38:0/3522812405 shutdown_connections 2024-04-25T08:41:53.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.795+0000 7efd11161640 1 -- 172.21.15.38:0/3522812405 wait complete. 2024-04-25T08:41:53.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.796+0000 7efd11161640 1 Processor -- start 2024-04-25T08:41:53.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.796+0000 7efd11161640 1 -- start start 2024-04-25T08:41:53.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd11161640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf8004d40 0x7efd0c0c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:53.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd11161640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd0c15b680 0x7efd0c0b8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:53.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd11161640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd0c155b70 0x7efd0c155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:53.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd11161640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c0b7d80 con 0x7efcf8004d40 2024-04-25T08:41:53.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd11161640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c0b7aa0 con 0x7efd0c155b70 2024-04-25T08:41:53.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd11161640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd0c0d0c80 con 0x7efd0c15b680 2024-04-25T08:41:53.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd0ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf8004d40 0x7efd0c0c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:53.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd0ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf8004d40 0x7efd0c0c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60608/0 (socket says 172.21.15.38:60608) 2024-04-25T08:41:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd0ad9d640 1 -- 172.21.15.38:0/478888272 learned_addr learned my addr 172.21.15.38:0/478888272 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd0ad9d640 1 -- 172.21.15.38:0/478888272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd0c15b680 msgr2=0x7efd0c0b8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd0ad9d640 1 --2- 172.21.15.38:0/478888272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd0c15b680 0x7efd0c0b8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd0ad9d640 1 -- 172.21.15.38:0/478888272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd0c155b70 msgr2=0x7efd0c155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd0ad9d640 1 --2- 172.21.15.38:0/478888272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd0c155b70 0x7efd0c155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.797+0000 7efd0ad9d640 1 -- 172.21.15.38:0/478888272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd0c1f3d10 con 0x7efcf8004d40 2024-04-25T08:41:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.798+0000 7efd0ad9d640 1 --2- 172.21.15.38:0/478888272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf8004d40 0x7efd0c0c9d90 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7efd040072c0 tx=0x7efd04007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.798+0000 7efd037fe640 1 -- 172.21.15.38:0/478888272 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0400f8e0 con 0x7efcf8004d40 2024-04-25T08:41:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.798+0000 7efd037fe640 1 -- 172.21.15.38:0/478888272 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd04018070 con 0x7efcf8004d40 2024-04-25T08:41:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.798+0000 7efd11161640 1 -- 172.21.15.38:0/478888272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd0c1f3ff0 con 0x7efcf8004d40 2024-04-25T08:41:53.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.798+0000 7efd037fe640 1 -- 172.21.15.38:0/478888272 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0400fd00 con 0x7efcf8004d40 2024-04-25T08:41:53.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.798+0000 7efd11161640 1 -- 172.21.15.38:0/478888272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd0c1f45b0 con 0x7efcf8004d40 2024-04-25T08:41:53.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.799+0000 7efd11161640 1 -- 172.21.15.38:0/478888272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd0c068e80 con 0x7efcf8004d40 2024-04-25T08:41:53.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.799+0000 7efd037fe640 1 -- 172.21.15.38:0/478888272 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd0401d460 con 0x7efcf8004d40 2024-04-25T08:41:53.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.800+0000 7efd037fe640 1 --2- 172.21.15.38:0/478888272 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efcec06b4d0 0x7efcec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:53.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.800+0000 7efd037fe640 1 -- 172.21.15.38:0/478888272 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6273+0+0 (secure 0 0 0) 0x7efd0408a490 con 0x7efcf8004d40 2024-04-25T08:41:53.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.800+0000 7efd0a59c640 1 --2- 172.21.15.38:0/478888272 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efcec06b4d0 0x7efcec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:53.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.801+0000 7efd0a59c640 1 --2- 172.21.15.38:0/478888272 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efcec06b4d0 0x7efcec06d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0ceaa0 tx=0x7efcf4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:53.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.804+0000 7efd037fe640 1 -- 172.21.15.38:0/478888272 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd0c0d11c0 con 0x7efcf8004d40 2024-04-25T08:41:53.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:53.985+0000 7efd11161640 1 -- 172.21.15.38:0/478888272 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efd0c1f6f40 con 0x7efcf8004d40 2024-04-25T08:41:54.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.621+0000 7efd037fe640 1 -- 172.21.15.38:0/478888272 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v135) v1 ==== 181+0+0 (secure 0 0 0) 0x7efd04059050 con 0x7efcf8004d40 2024-04-25T08:41:54.614 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:41:54.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.624+0000 7efd11161640 1 -- 172.21.15.38:0/478888272 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efcec06b4d0 msgr2=0x7efcec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:54.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.624+0000 7efd11161640 1 --2- 172.21.15.38:0/478888272 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efcec06b4d0 0x7efcec06d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7efd0c0ceaa0 tx=0x7efcf4009040 comp rx=0 tx=0).stop 2024-04-25T08:41:54.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.624+0000 7efd11161640 1 -- 172.21.15.38:0/478888272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf8004d40 msgr2=0x7efd0c0c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:54.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.624+0000 7efd11161640 1 --2- 172.21.15.38:0/478888272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf8004d40 0x7efd0c0c9d90 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7efd040072c0 tx=0x7efd04007780 comp rx=0 tx=0).stop 2024-04-25T08:41:54.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.624+0000 7efd11161640 1 -- 172.21.15.38:0/478888272 shutdown_connections 2024-04-25T08:41:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.624+0000 7efd11161640 1 --2- 172.21.15.38:0/478888272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd0c155b70 0x7efd0c155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.624+0000 7efd11161640 1 --2- 172.21.15.38:0/478888272 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efcec06b4d0 0x7efcec06d990 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.624+0000 7efd11161640 1 --2- 172.21.15.38:0/478888272 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd0c15b680 0x7efd0c0b8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.624+0000 7efd11161640 1 --2- 172.21.15.38:0/478888272 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf8004d40 0x7efd0c0c9d90 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.624+0000 7efd11161640 1 -- 172.21.15.38:0/478888272 >> 172.21.15.38:0/478888272 conn(0x7efd0c066d10 msgr2=0x7efd0c0cefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.624+0000 7efd11161640 1 -- 172.21.15.38:0/478888272 shutdown_connections 2024-04-25T08:41:54.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.624+0000 7efd11161640 1 -- 172.21.15.38:0/478888272 wait complete. 2024-04-25T08:41:54.631 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:41:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.751+0000 7fe0f70c4640 1 Processor -- start 2024-04-25T08:41:54.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.751+0000 7fe0f70c4640 1 -- start start 2024-04-25T08:41:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f70c4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe0f00b47b0 0x7fe0f006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f70c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0f006e8e0 0x7fe0f00cf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f70c4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe0f00cf7a0 0x7fe0f00b2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f70c4640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f006ed40 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f70c4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f015b7a0 con 0x7fe0f00b47b0 2024-04-25T08:41:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f70c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f015abb0 con 0x7fe0f006e8e0 2024-04-25T08:41:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f5661640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe0f00cf7a0 0x7fe0f00b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f5661640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fe0f00cf7a0 0x7fe0f00b2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60620/0 (socket says 172.21.15.38:60620) 2024-04-25T08:41:54.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f5661640 1 -- 172.21.15.38:0/1504472168 learned_addr learned my addr 172.21.15.38:0/1504472168 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:54.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0effff640 1 --2- 172.21.15.38:0/1504472168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0f006e8e0 0x7fe0f00cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:54.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f4e60640 1 --2- 172.21.15.38:0/1504472168 >> v2:172.21.15.107:3300/0 conn(0x7fe0f00b47b0 0x7fe0f006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:54.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f5661640 1 -- 172.21.15.38:0/1504472168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0f006e8e0 msgr2=0x7fe0f00cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:54.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f5661640 1 --2- 172.21.15.38:0/1504472168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0f006e8e0 0x7fe0f00cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:54.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f5661640 1 -- 172.21.15.38:0/1504472168 >> v2:172.21.15.107:3300/0 conn(0x7fe0f00b47b0 msgr2=0x7fe0f006e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:54.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f5661640 1 --2- 172.21.15.38:0/1504472168 >> v2:172.21.15.107:3300/0 conn(0x7fe0f00b47b0 0x7fe0f006e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:54.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.752+0000 7fe0f5661640 1 -- 172.21.15.38:0/1504472168 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f015ae70 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.753+0000 7fe0f5661640 1 --2- 172.21.15.38:0/1504472168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0f00cf7a0 0x7fe0f00b2510 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fe0f006bca0 tx=0x7fe0e802c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ece8247f491e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:54.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.753+0000 7fe0ef7fe640 1 -- 172.21.15.38:0/1504472168 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e800b2e0 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.753+0000 7fe0ef7fe640 1 -- 172.21.15.38:0/1504472168 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e8009600 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.753+0000 7fe0ef7fe640 1 -- 172.21.15.38:0/1504472168 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e800b480 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.754+0000 7fe0f70c4640 1 -- 172.21.15.38:0/1504472168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0f00cf7a0 msgr2=0x7fe0f00b2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.754+0000 7fe0f70c4640 1 --2- 172.21.15.38:0/1504472168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0f00cf7a0 0x7fe0f00b2510 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fe0f006bca0 tx=0x7fe0e802c8a0 comp rx=0 tx=0).stop 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.754+0000 7fe0f70c4640 1 -- 172.21.15.38:0/1504472168 shutdown_connections 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.754+0000 7fe0f70c4640 1 --2- 172.21.15.38:0/1504472168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0f00cf7a0 0x7fe0f00b2510 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.754+0000 7fe0f70c4640 1 --2- 172.21.15.38:0/1504472168 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0f006e8e0 0x7fe0f00cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.754+0000 7fe0f70c4640 1 --2- 172.21.15.38:0/1504472168 >> v2:172.21.15.107:3300/0 conn(0x7fe0f00b47b0 0x7fe0f006e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.754+0000 7fe0f70c4640 1 -- 172.21.15.38:0/1504472168 >> 172.21.15.38:0/1504472168 conn(0x7fe0f0066f20 msgr2=0x7fe0f006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.754+0000 7fe0f70c4640 1 -- 172.21.15.38:0/1504472168 shutdown_connections 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.754+0000 7fe0f70c4640 1 -- 172.21.15.38:0/1504472168 wait complete. 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.755+0000 7fe0f70c4640 1 Processor -- start 2024-04-25T08:41:54.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.755+0000 7fe0f70c4640 1 -- start start 2024-04-25T08:41:54.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f70c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0f006e8e0 0x7fe0f00c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:54.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f70c4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0f00b47b0 0x7fe0f00ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:54.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f70c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0f00cf7a0 0x7fe0f00adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:54.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f70c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f006d000 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f70c4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f015b430 con 0x7fe0f006e8e0 2024-04-25T08:41:54.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f70c4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f00cfc30 con 0x7fe0f00b47b0 2024-04-25T08:41:54.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f5661640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0f00cf7a0 0x7fe0f00adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:54.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f5661640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0f00cf7a0 0x7fe0f00adba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60624/0 (socket says 172.21.15.38:60624) 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f5661640 1 -- 172.21.15.38:0/2079374469 learned_addr learned my addr 172.21.15.38:0/2079374469 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f4e60640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0f006e8e0 0x7fe0f00c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0effff640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0f00b47b0 0x7fe0f00ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f5661640 1 -- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0f00b47b0 msgr2=0x7fe0f00ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f5661640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0f00b47b0 0x7fe0f00ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f5661640 1 -- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0f006e8e0 msgr2=0x7fe0f00c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f5661640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0f006e8e0 0x7fe0f00c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f5661640 1 -- 172.21.15.38:0/2079374469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f00cb010 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.756+0000 7fe0f4e60640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0f006e8e0 0x7fe0f00c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.757+0000 7fe0f5661640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0f00cf7a0 0x7fe0f00adba0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800d240 tx=0x7fe0e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.757+0000 7fe0ed7fa640 1 -- 172.21.15.38:0/2079374469 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8041030 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.757+0000 7fe0ed7fa640 1 -- 172.21.15.38:0/2079374469 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e8009370 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.757+0000 7fe0f70c4640 1 -- 172.21.15.38:0/2079374469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f00cb2f0 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.757+0000 7fe0ed7fa640 1 -- 172.21.15.38:0/2079374469 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e803d870 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.757+0000 7fe0f70c4640 1 -- 172.21.15.38:0/2079374469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f014c9b0 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.758+0000 7fe0f70c4640 1 -- 172.21.15.38:0/2079374469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f0068e80 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.758+0000 7fe0ed7fa640 1 -- 172.21.15.38:0/2079374469 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe0e8044030 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.759+0000 7fe0ed7fa640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe0d406b4d0 0x7fe0d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:54.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.759+0000 7fe0ed7fa640 1 -- 172.21.15.38:0/2079374469 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5835+0+0 (secure 0 0 0) 0x7fe0e8030020 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.759+0000 7fe0f4e60640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe0d406b4d0 0x7fe0d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:54.754 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.760+0000 7fe0f4e60640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe0d406b4d0 0x7fe0d406d990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4006fd0 tx=0x7fe0e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:54.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.763+0000 7fe0ed7fa640 1 -- 172.21.15.38:0/2079374469 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0e804b030 con 0x7fe0f00cf7a0 2024-04-25T08:41:54.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:54.949+0000 7fe0f70c4640 1 -- 172.21.15.38:0/2079374469 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe0f00cacf0 con 0x7fe0f00cf7a0 2024-04-25T08:41:55.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.623+0000 7fe0ed7fa640 1 -- 172.21.15.38:0/2079374469 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v136) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe0e807a050 con 0x7fe0f00cf7a0 2024-04-25T08:41:55.616 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-04-25T08:41:55.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.627+0000 7fe0f70c4640 1 -- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe0d406b4d0 msgr2=0x7fe0d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.627+0000 7fe0f70c4640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe0d406b4d0 0x7fe0d406d990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4006fd0 tx=0x7fe0e4008040 comp rx=0 tx=0).stop 2024-04-25T08:41:55.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.627+0000 7fe0f70c4640 1 -- 172.21.15.38:0/2079374469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0f00cf7a0 msgr2=0x7fe0f00adba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.627+0000 7fe0f70c4640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0f00cf7a0 0x7fe0f00adba0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800d240 tx=0x7fe0e8004a40 comp rx=0 tx=0).stop 2024-04-25T08:41:55.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.628+0000 7fe0f70c4640 1 -- 172.21.15.38:0/2079374469 shutdown_connections 2024-04-25T08:41:55.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.628+0000 7fe0f70c4640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe0f00cf7a0 0x7fe0f00adba0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.628+0000 7fe0f70c4640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe0d406b4d0 0x7fe0d406d990 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.628+0000 7fe0f70c4640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0f00b47b0 0x7fe0f00ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.628+0000 7fe0f70c4640 1 --2- 172.21.15.38:0/2079374469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0f006e8e0 0x7fe0f00c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.628+0000 7fe0f70c4640 1 -- 172.21.15.38:0/2079374469 >> 172.21.15.38:0/2079374469 conn(0x7fe0f0066f20 msgr2=0x7fe0f0071830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:55.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.628+0000 7fe0f70c4640 1 -- 172.21.15.38:0/2079374469 shutdown_connections 2024-04-25T08:41:55.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.628+0000 7fe0f70c4640 1 -- 172.21.15.38:0/2079374469 wait complete. 2024-04-25T08:41:55.638 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:41:55.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.763+0000 7fc0f44f5640 1 Processor -- start 2024-04-25T08:41:55.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.763+0000 7fc0f44f5640 1 -- start start 2024-04-25T08:41:55.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.764+0000 7fc0f44f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc0ec0d33d0 0x7fc0ec0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.764+0000 7fc0f44f5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ec0d2330 con 0x7fc0ec0d1e90 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.764+0000 7fc0f44f5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0ec0d2610 con 0x7fc0ec0d0320 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.764+0000 7fc0f44f5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0ec0d28f0 con 0x7fc0ec0d33d0 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.765+0000 7fc0f2a92640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc0ec0d1e90 0x7fc0ec0b6980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35856/0 (socket says 172.21.15.38:35856) 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.765+0000 7fc0f2a92640 1 -- 172.21.15.38:0/812492607 learned_addr learned my addr 172.21.15.38:0/812492607 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.765+0000 7fc0f1a90640 1 --2- 172.21.15.38:0/812492607 >> v2:172.21.15.107:3300/0 conn(0x7fc0ec0d33d0 0x7fc0ec0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.765+0000 7fc0f128f640 1 -- 172.21.15.38:0/812492607 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696896050 0 0) 0x7fc0ec0d2330 con 0x7fc0ec0d1e90 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.765+0000 7fc0f128f640 1 -- 172.21.15.38:0/812492607 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0d4003520 con 0x7fc0ec0d1e90 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.765+0000 7fc0f128f640 1 -- 172.21.15.38:0/812492607 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740820497 0 0) 0x7fc0ec0d2610 con 0x7fc0ec0d0320 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.765+0000 7fc0f128f640 1 -- 172.21.15.38:0/812492607 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0ec0d2330 con 0x7fc0ec0d0320 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.765+0000 7fc0f1a90640 1 -- 172.21.15.38:0/812492607 >> v1:172.21.15.107:6789/0 conn(0x7fc0ec0d0320 legacy=0x7fc0ec0d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.765+0000 7fc0f1a90640 1 -- 172.21.15.38:0/812492607 >> v1:172.21.15.38:6789/0 conn(0x7fc0ec0d1e90 legacy=0x7fc0ec0b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.765+0000 7fc0f1a90640 1 -- 172.21.15.38:0/812492607 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec0d2bd0 con 0x7fc0ec0d33d0 2024-04-25T08:41:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.766+0000 7fc0f1a90640 1 --2- 172.21.15.38:0/812492607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0d33d0 0x7fc0ec0d1950 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00097a0 tx=0x7fc0e002a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3940f120c82a5491 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.766+0000 7fc0f128f640 1 -- 172.21.15.38:0/812492607 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e0037070 con 0x7fc0ec0d33d0 2024-04-25T08:41:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.766+0000 7fc0f128f640 1 -- 172.21.15.38:0/812492607 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e0032640 con 0x7fc0ec0d33d0 2024-04-25T08:41:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.766+0000 7fc0f128f640 1 -- 172.21.15.38:0/812492607 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e00329f0 con 0x7fc0ec0d33d0 2024-04-25T08:41:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.767+0000 7fc0f44f5640 1 -- 172.21.15.38:0/812492607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0d33d0 msgr2=0x7fc0ec0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.767+0000 7fc0f44f5640 1 --2- 172.21.15.38:0/812492607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0d33d0 0x7fc0ec0d1950 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc0e00097a0 tx=0x7fc0e002a2c0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.767+0000 7fc0f44f5640 1 -- 172.21.15.38:0/812492607 shutdown_connections 2024-04-25T08:41:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.767+0000 7fc0f44f5640 1 --2- 172.21.15.38:0/812492607 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0d33d0 0x7fc0ec0d1950 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.767+0000 7fc0f44f5640 1 -- 172.21.15.38:0/812492607 >> 172.21.15.38:0/812492607 conn(0x7fc0ec0668f0 msgr2=0x7fc0ec066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.767+0000 7fc0f44f5640 1 -- 172.21.15.38:0/812492607 shutdown_connections 2024-04-25T08:41:55.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.767+0000 7fc0f44f5640 1 -- 172.21.15.38:0/812492607 wait complete. 2024-04-25T08:41:55.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.768+0000 7fc0f44f5640 1 Processor -- start 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.768+0000 7fc0f44f5640 1 -- start start 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.768+0000 7fc0f44f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0ec0d0320 0x7fc0ec0aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f44f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0d1e90 0x7fc0ec0b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f44f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0ec0d33d0 0x7fc0ec0b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f44f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec161390 con 0x7fc0ec0d0320 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f44f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec161210 con 0x7fc0ec0d1e90 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f44f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec0b9720 con 0x7fc0ec0d33d0 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f2291640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0ec0d0320 0x7fc0ec0aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f2291640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0ec0d0320 0x7fc0ec0aacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60630/0 (socket says 172.21.15.38:60630) 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f2291640 1 -- 172.21.15.38:0/3397548630 learned_addr learned my addr 172.21.15.38:0/3397548630 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f1a90640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0d1e90 0x7fc0ec0b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f2a92640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0ec0d33d0 0x7fc0ec0b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f2291640 1 -- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0ec0d33d0 msgr2=0x7fc0ec0b6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f2291640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0ec0d33d0 0x7fc0ec0b6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f2291640 1 -- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0d1e90 msgr2=0x7fc0ec0b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f2291640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0d1e90 0x7fc0ec0b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f2291640 1 -- 172.21.15.38:0/3397548630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec0ab370 con 0x7fc0ec0d0320 2024-04-25T08:41:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f1a90640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0d1e90 0x7fc0ec0b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f2291640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0ec0d0320 0x7fc0ec0aacb0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400c500 tx=0x7fc0e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0eaffd640 1 -- 172.21.15.38:0/3397548630 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40052d0 con 0x7fc0ec0d0320 2024-04-25T08:41:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0eaffd640 1 -- 172.21.15.38:0/3397548630 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e4005490 con 0x7fc0ec0d0320 2024-04-25T08:41:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f44f5640 1 -- 172.21.15.38:0/3397548630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ec0ab650 con 0x7fc0ec0d0320 2024-04-25T08:41:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0eaffd640 1 -- 172.21.15.38:0/3397548630 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4012870 con 0x7fc0ec0d0320 2024-04-25T08:41:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.769+0000 7fc0f44f5640 1 -- 172.21.15.38:0/3397548630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec0abc10 con 0x7fc0ec0d0320 2024-04-25T08:41:55.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.770+0000 7fc0f44f5640 1 -- 172.21.15.38:0/3397548630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec0ca2b0 con 0x7fc0ec0d0320 2024-04-25T08:41:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.771+0000 7fc0eaffd640 1 -- 172.21.15.38:0/3397548630 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0e4012a10 con 0x7fc0ec0d0320 2024-04-25T08:41:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.771+0000 7fc0eaffd640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0d806b4d0 0x7fc0d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.772+0000 7fc0eaffd640 1 -- 172.21.15.38:0/3397548630 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fc0e4089e60 con 0x7fc0ec0d0320 2024-04-25T08:41:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.772+0000 7fc0f1a90640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0d806b4d0 0x7fc0d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:55.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.772+0000 7fc0f1a90640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0d806b4d0 0x7fc0d806d990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0034000 tx=0x7fc0e002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:55.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.776+0000 7fc0eaffd640 1 -- 172.21.15.38:0/3397548630 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0ec0ca2b0 con 0x7fc0ec0d0320 2024-04-25T08:41:55.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.989+0000 7fc0f44f5640 1 -- 172.21.15.38:0/3397548630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc0ec068e80 con 0x7fc0ec0d0320 2024-04-25T08:41:55.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.992+0000 7fc0eaffd640 1 -- 172.21.15.38:0/3397548630 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc0e4058d70 con 0x7fc0ec0d0320 2024-04-25T08:41:55.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.994+0000 7fc0f44f5640 1 -- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0d806b4d0 msgr2=0x7fc0d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.994+0000 7fc0f44f5640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0d806b4d0 0x7fc0d806d990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0034000 tx=0x7fc0e002e000 comp rx=0 tx=0).stop 2024-04-25T08:41:55.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.994+0000 7fc0f44f5640 1 -- 172.21.15.38:0/3397548630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0ec0d0320 msgr2=0x7fc0ec0aacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:55.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.994+0000 7fc0f44f5640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0ec0d0320 0x7fc0ec0aacb0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400c500 tx=0x7fc0e400c9c0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.994+0000 7fc0f44f5640 1 -- 172.21.15.38:0/3397548630 shutdown_connections 2024-04-25T08:41:55.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.994+0000 7fc0f44f5640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc0d806b4d0 0x7fc0d806d990 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.994+0000 7fc0f44f5640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0ec0d33d0 0x7fc0ec0b6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.994+0000 7fc0f44f5640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc0ec0d1e90 0x7fc0ec0b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.994+0000 7fc0f44f5640 1 --2- 172.21.15.38:0/3397548630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0ec0d0320 0x7fc0ec0aacb0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:55.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.994+0000 7fc0f44f5640 1 -- 172.21.15.38:0/3397548630 >> 172.21.15.38:0/3397548630 conn(0x7fc0ec0668f0 msgr2=0x7fc0ec15d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:55.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.994+0000 7fc0f44f5640 1 -- 172.21.15.38:0/3397548630 shutdown_connections 2024-04-25T08:41:55.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:55.994+0000 7fc0f44f5640 1 -- 172.21.15.38:0/3397548630 wait complete. 2024-04-25T08:41:55.998 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T08:41:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.107+0000 7efcfff68640 1 Processor -- start 2024-04-25T08:41:56.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.107+0000 7efcfff68640 1 -- start start 2024-04-25T08:41:56.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfff68640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcf8099a70 0x7efcf8099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfff68640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7efcf80cf470 0x7efcf80cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfff68640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf809a390 con 0x7efcf806bd80 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfff68640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf81614d0 con 0x7efcf80cf470 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfff68640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80cd970 con 0x7efcf8099a70 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfe505640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7efcf806bd80 0x7efcf806c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35858/0 (socket says 172.21.15.38:35858) 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfe505640 1 -- 172.21.15.38:0/59195240 learned_addr learned my addr 172.21.15.38:0/59195240 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfdd04640 1 --2- 172.21.15.38:0/59195240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcf8099a70 0x7efcf8099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfd503640 1 --2- 172.21.15.38:0/59195240 >> v2:172.21.15.107:3300/0 conn(0x7efcf80cf470 0x7efcf80cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfcd02640 1 -- 172.21.15.38:0/59195240 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649166842 0 0) 0x7efcf809a390 con 0x7efcf806bd80 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfcd02640 1 -- 172.21.15.38:0/59195240 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce4003520 con 0x7efcf806bd80 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfdd04640 1 -- 172.21.15.38:0/59195240 >> v2:172.21.15.107:3300/0 conn(0x7efcf80cf470 msgr2=0x7efcf80cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfdd04640 1 --2- 172.21.15.38:0/59195240 >> v2:172.21.15.107:3300/0 conn(0x7efcf80cf470 0x7efcf80cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfdd04640 1 -- 172.21.15.38:0/59195240 >> v1:172.21.15.38:6789/0 conn(0x7efcf806bd80 legacy=0x7efcf806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfdd04640 1 -- 172.21.15.38:0/59195240 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80ccd60 con 0x7efcf8099a70 2024-04-25T08:41:56.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.108+0000 7efcfd503640 1 --2- 172.21.15.38:0/59195240 >> v2:172.21.15.107:3300/0 conn(0x7efcf80cf470 0x7efcf80cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:41:56.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.109+0000 7efcfdd04640 1 --2- 172.21.15.38:0/59195240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcf8099a70 0x7efcf8099e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efcf40097a0 tx=0x7efcf402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc506aa595f981c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:56.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.109+0000 7efcfcd02640 1 -- 172.21.15.38:0/59195240 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4037070 con 0x7efcf8099a70 2024-04-25T08:41:56.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.109+0000 7efcfcd02640 1 -- 172.21.15.38:0/59195240 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf4032440 con 0x7efcf8099a70 2024-04-25T08:41:56.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.109+0000 7efcfcd02640 1 -- 172.21.15.38:0/59195240 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4032810 con 0x7efcf8099a70 2024-04-25T08:41:56.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.110+0000 7efcfff68640 1 -- 172.21.15.38:0/59195240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcf8099a70 msgr2=0x7efcf8099e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.110+0000 7efcfff68640 1 --2- 172.21.15.38:0/59195240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcf8099a70 0x7efcf8099e50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7efcf40097a0 tx=0x7efcf402a180 comp rx=0 tx=0).stop 2024-04-25T08:41:56.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.110+0000 7efcfff68640 1 -- 172.21.15.38:0/59195240 shutdown_connections 2024-04-25T08:41:56.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.110+0000 7efcfff68640 1 --2- 172.21.15.38:0/59195240 >> v2:172.21.15.107:3300/0 conn(0x7efcf80cf470 0x7efcf80cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.110+0000 7efcfff68640 1 --2- 172.21.15.38:0/59195240 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcf8099a70 0x7efcf8099e50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.110+0000 7efcfff68640 1 -- 172.21.15.38:0/59195240 >> 172.21.15.38:0/59195240 conn(0x7efcf80b3180 msgr2=0x7efcf80b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.110+0000 7efcfff68640 1 -- 172.21.15.38:0/59195240 shutdown_connections 2024-04-25T08:41:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.110+0000 7efcfff68640 1 -- 172.21.15.38:0/59195240 wait complete. 2024-04-25T08:41:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.111+0000 7efcfff68640 1 Processor -- start 2024-04-25T08:41:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.111+0000 7efcfff68640 1 -- start start 2024-04-25T08:41:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfff68640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf806bd80 0x7efcf815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfff68640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf8099a70 0x7efcf80c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfff68640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcf80cf470 0x7efcf80b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfff68640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80cd600 con 0x7efcf806bd80 2024-04-25T08:41:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfff68640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80cd320 con 0x7efcf8099a70 2024-04-25T08:41:56.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfff68640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80cc9c0 con 0x7efcf80cf470 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfdd04640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf806bd80 0x7efcf815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfdd04640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf806bd80 0x7efcf815e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60646/0 (socket says 172.21.15.38:60646) 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfdd04640 1 -- 172.21.15.38:0/3895831279 learned_addr learned my addr 172.21.15.38:0/3895831279 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfe505640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcf80cf470 0x7efcf80b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfd503640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf8099a70 0x7efcf80c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfdd04640 1 -- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcf80cf470 msgr2=0x7efcf80b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfdd04640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcf80cf470 0x7efcf80b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfdd04640 1 -- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf8099a70 msgr2=0x7efcf80c5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfdd04640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf8099a70 0x7efcf80c5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfdd04640 1 -- 172.21.15.38:0/3895831279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf80c68f0 con 0x7efcf806bd80 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfdd04640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf806bd80 0x7efcf815e140 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7efcf4035040 tx=0x7efcf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcee7fc640 1 -- 172.21.15.38:0/3895831279 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4041030 con 0x7efcf806bd80 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcee7fc640 1 -- 172.21.15.38:0/3895831279 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf403d4e0 con 0x7efcf806bd80 2024-04-25T08:41:56.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfff68640 1 -- 172.21.15.38:0/3895831279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf80c6bd0 con 0x7efcf806bd80 2024-04-25T08:41:56.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcee7fc640 1 -- 172.21.15.38:0/3895831279 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4037040 con 0x7efcf806bd80 2024-04-25T08:41:56.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.112+0000 7efcfff68640 1 -- 172.21.15.38:0/3895831279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf80c7190 con 0x7efcf806bd80 2024-04-25T08:41:56.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.113+0000 7efcfff68640 1 -- 172.21.15.38:0/3895831279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf8068e80 con 0x7efcf806bd80 2024-04-25T08:41:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.114+0000 7efcee7fc640 1 -- 172.21.15.38:0/3895831279 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efcf4044070 con 0x7efcf806bd80 2024-04-25T08:41:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.114+0000 7efcee7fc640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efcdc06b4d0 0x7efcdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.114+0000 7efcee7fc640 1 -- 172.21.15.38:0/3895831279 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7efcf40aa690 con 0x7efcf806bd80 2024-04-25T08:41:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.115+0000 7efcfd503640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efcdc06b4d0 0x7efcdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.115+0000 7efcfd503640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efcdc06b4d0 0x7efcdc06d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7efce8009560 tx=0x7efce8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:56.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.119+0000 7efcee7fc640 1 -- 172.21.15.38:0/3895831279 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efcf4079080 con 0x7efcf806bd80 2024-04-25T08:41:56.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.372+0000 7efcfff68640 1 -- 172.21.15.38:0/3895831279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7efcf806e370 con 0x7efcf806bd80 2024-04-25T08:41:56.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.628+0000 7efcee7fc640 1 -- 172.21.15.38:0/3895831279 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v502) v1 ==== 166+0+0 (secure 0 0 0) 0x7efcf407d120 con 0x7efcf806bd80 2024-04-25T08:41:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.631+0000 7efcfff68640 1 -- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efcdc06b4d0 msgr2=0x7efcdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.631+0000 7efcfff68640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efcdc06b4d0 0x7efcdc06d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7efce8009560 tx=0x7efce8009190 comp rx=0 tx=0).stop 2024-04-25T08:41:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.631+0000 7efcfff68640 1 -- 172.21.15.38:0/3895831279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf806bd80 msgr2=0x7efcf815e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.631+0000 7efcfff68640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf806bd80 0x7efcf815e140 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7efcf4035040 tx=0x7efcf40046c0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.631+0000 7efcfff68640 1 -- 172.21.15.38:0/3895831279 shutdown_connections 2024-04-25T08:41:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.631+0000 7efcfff68640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efcdc06b4d0 0x7efcdc06d990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.631+0000 7efcfff68640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efcf80cf470 0x7efcf80b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.631+0000 7efcfff68640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcf8099a70 0x7efcf80c5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.631+0000 7efcfff68640 1 --2- 172.21.15.38:0/3895831279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf806bd80 0x7efcf815e140 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.631+0000 7efcfff68640 1 -- 172.21.15.38:0/3895831279 >> 172.21.15.38:0/3895831279 conn(0x7efcf80b3180 msgr2=0x7efcf8099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.631+0000 7efcfff68640 1 -- 172.21.15.38:0/3895831279 shutdown_connections 2024-04-25T08:41:56.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.631+0000 7efcfff68640 1 -- 172.21.15.38:0/3895831279 wait complete. 2024-04-25T08:41:56.635 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T08:41:56.635 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:41:56.636 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:41:56.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.744+0000 7f81a47c1640 1 Processor -- start 2024-04-25T08:41:56.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.744+0000 7f81a47c1640 1 -- start start 2024-04-25T08:41:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.745+0000 7f81a47c1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c06c110 0x7f819c06c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.745+0000 7f81a47c1640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f819c15df50 0x7f819c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.745+0000 7f81a47c1640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819c06ca30 con 0x7f819c0ce2e0 2024-04-25T08:41:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.745+0000 7f81a47c1640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819c0cf2d0 con 0x7f819c15df50 2024-04-25T08:41:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.745+0000 7f81a47c1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0ceff0 con 0x7f819c06c110 2024-04-25T08:41:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.745+0000 7f81a2d5e640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f819c0ce2e0 0x7f819c0d2a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35862/0 (socket says 172.21.15.38:35862) 2024-04-25T08:41:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.745+0000 7f81a2d5e640 1 -- 172.21.15.38:0/617415554 learned_addr learned my addr 172.21.15.38:0/617415554 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a1d5c640 1 --2- 172.21.15.38:0/617415554 >> v2:172.21.15.107:3300/0 conn(0x7f819c15df50 0x7f819c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a255d640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c06c110 0x7f819c06c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84462749 0 0) 0x7f819c06ca30 con 0x7f819c0ce2e0 2024-04-25T08:41:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8184003520 con 0x7f819c0ce2e0 2024-04-25T08:41:56.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3283936694 0 0) 0x7f8184003520 con 0x7f819c0ce2e0 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c06c110 msgr2=0x7f819c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c06c110 0x7f819c06c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 >> v2:172.21.15.107:3300/0 conn(0x7f819c15df50 msgr2=0x7f819c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 --2- 172.21.15.38:0/617415554 >> v2:172.21.15.107:3300/0 conn(0x7f819c15df50 0x7f819c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c0d1f70 con 0x7f819c0ce2e0 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f818c0019f0 con 0x7f819c0ce2e0 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 >> v1:172.21.15.38:6789/0 conn(0x7f819c0ce2e0 legacy=0x7f819c0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c06c110 0x7f81840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8184003f30 0x7f8184004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8184004d40 0x7f81840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0cf2d0 con 0x7f8184004d40 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0ceff0 con 0x7f8184003f30 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818400b9c0 con 0x7f819c06c110 2024-04-25T08:41:56.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a2d5e640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8184004d40 0x7f81840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a255d640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c06c110 0x7f81840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a1d5c640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8184003f30 0x7f8184004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a255d640 1 -- 172.21.15.38:0/617415554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8184003f30 msgr2=0x7f8184004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:56.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.746+0000 7f81a255d640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8184003f30 0x7f8184004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.747+0000 7f81a255d640 1 -- 172.21.15.38:0/617415554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8184004d40 msgr2=0x7f81840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:56.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.747+0000 7f81a255d640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8184004d40 0x7f81840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.747+0000 7f81a255d640 1 -- 172.21.15.38:0/617415554 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8184003520 con 0x7f819c06c110 2024-04-25T08:41:56.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.747+0000 7f81a2d5e640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8184004d40 0x7f81840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:41:56.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.747+0000 7f81a255d640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c06c110 0x7f81840039b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8194009ac0 tx=0x7f8184005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2dc5c63b6fd49b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:56.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.747+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194036070 con 0x7f819c06c110 2024-04-25T08:41:56.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.747+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8194031400 con 0x7f819c06c110 2024-04-25T08:41:56.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.747+0000 7f81a155b640 1 -- 172.21.15.38:0/617415554 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8194009690 con 0x7f819c06c110 2024-04-25T08:41:56.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.748+0000 7f81a47c1640 1 -- 172.21.15.38:0/617415554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c06c110 msgr2=0x7f81840039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.748+0000 7f81a47c1640 1 --2- 172.21.15.38:0/617415554 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c06c110 0x7f81840039b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8194009ac0 tx=0x7f8184005830 comp rx=0 tx=0).stop 2024-04-25T08:41:56.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.748+0000 7f81a2d5e640 1 -- 172.21.15.38:0/617415554 reap_dead start 2024-04-25T08:41:56.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.748+0000 7f81a47c1640 1 -- 172.21.15.38:0/617415554 shutdown_connections 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.748+0000 7f81a47c1640 1 -- 172.21.15.38:0/617415554 >> 172.21.15.38:0/617415554 conn(0x7f819c0d3fb0 msgr2=0x7f819c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.748+0000 7f81a47c1640 1 -- 172.21.15.38:0/617415554 shutdown_connections 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.748+0000 7f81a47c1640 1 -- 172.21.15.38:0/617415554 wait complete. 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.749+0000 7f81a47c1640 1 Processor -- start 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.749+0000 7f81a47c1640 1 -- start start 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a47c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c06c110 0x7f819c14bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a47c1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f819c0ca9c0 0x7f819c156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a47c1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c0cae40 0x7f819c156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a47c1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0cece0 con 0x7f819c0ca9c0 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a47c1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0d2530 con 0x7f819c06c110 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a47c1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819c0d3160 con 0x7f819c0cae40 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a1d5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f819c0ca9c0 0x7f819c156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a1d5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f819c0ca9c0 0x7f819c156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60666/0 (socket says 172.21.15.38:60666) 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a1d5c640 1 -- 172.21.15.38:0/160552340 learned_addr learned my addr 172.21.15.38:0/160552340 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a2d5e640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c0cae40 0x7f819c156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a255d640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c06c110 0x7f819c14bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a1d5c640 1 -- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c0cae40 msgr2=0x7f819c156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a1d5c640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c0cae40 0x7f819c156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a1d5c640 1 -- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c06c110 msgr2=0x7f819c14bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.750+0000 7f81a1d5c640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c06c110 0x7f819c14bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.751+0000 7f81a1d5c640 1 -- 172.21.15.38:0/160552340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c0c73f0 con 0x7f819c0ca9c0 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.751+0000 7f81a1d5c640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f819c0ca9c0 0x7f819c156780 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f8190011960 tx=0x7f8190011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.751+0000 7f819affd640 1 -- 172.21.15.38:0/160552340 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819000fa30 con 0x7f819c0ca9c0 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.751+0000 7f819affd640 1 -- 172.21.15.38:0/160552340 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819000fbd0 con 0x7f819c0ca9c0 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.751+0000 7f81a47c1640 1 -- 172.21.15.38:0/160552340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819c0cd5b0 con 0x7f819c0ca9c0 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.751+0000 7f819affd640 1 -- 172.21.15.38:0/160552340 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190014440 con 0x7f819c0ca9c0 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.751+0000 7f81a47c1640 1 -- 172.21.15.38:0/160552340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819c0c7970 con 0x7f819c0ca9c0 2024-04-25T08:41:56.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.752+0000 7f81a47c1640 1 -- 172.21.15.38:0/160552340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819c068e80 con 0x7f819c0ca9c0 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.753+0000 7f819affd640 1 -- 172.21.15.38:0/160552340 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8190012040 con 0x7f819c0ca9c0 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.753+0000 7f819affd640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f818806b4d0 0x7f818806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.753+0000 7f819affd640 1 -- 172.21.15.38:0/160552340 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f819008a5d0 con 0x7f819c0ca9c0 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.753+0000 7f81a255d640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f818806b4d0 0x7f818806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:56.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.754+0000 7f81a255d640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f818806b4d0 0x7f818806d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8194033000 tx=0x7f8194008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:56.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.757+0000 7f819affd640 1 -- 172.21.15.38:0/160552340 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8190019030 con 0x7f819c0ca9c0 2024-04-25T08:41:56.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.939+0000 7f81a47c1640 1 -- 172.21.15.38:0/160552340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f819c15d270 con 0x7f819c0ca9c0 2024-04-25T08:41:56.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.940+0000 7f819affd640 1 -- 172.21.15.38:0/160552340 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+13421 (secure 0 0 0) 0x7f8190058f20 con 0x7f819c0ca9c0 2024-04-25T08:41:56.933 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:56.933 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":136,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:41:55.622577+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6833/1189096522":"2024-04-26T08:41:52.617819+0000","172.21.15.38:6832/1189096522":"2024-04-26T08:41:52.617819+0000","172.21.15.38:6835/1312063561":"2024-04-26T08:41:40.950341+0000","172.21.15.38:6834/1312063561":"2024-04-26T08:41:40.950341+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:41:56.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.942+0000 7f81a47c1640 1 -- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f818806b4d0 msgr2=0x7f818806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.942+0000 7f81a47c1640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f818806b4d0 0x7f818806d990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8194033000 tx=0x7f8194008030 comp rx=0 tx=0).stop 2024-04-25T08:41:56.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.942+0000 7f81a47c1640 1 -- 172.21.15.38:0/160552340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f819c0ca9c0 msgr2=0x7f819c156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:56.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.942+0000 7f81a47c1640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f819c0ca9c0 0x7f819c156780 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f8190011960 tx=0x7f8190011d60 comp rx=0 tx=0).stop 2024-04-25T08:41:56.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.942+0000 7f81a47c1640 1 -- 172.21.15.38:0/160552340 shutdown_connections 2024-04-25T08:41:56.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.942+0000 7f81a47c1640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f818806b4d0 0x7f818806d990 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.942+0000 7f81a47c1640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f819c0cae40 0x7f819c156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.942+0000 7f81a47c1640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f819c0ca9c0 0x7f819c156780 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.942+0000 7f81a47c1640 1 --2- 172.21.15.38:0/160552340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f819c06c110 0x7f819c14bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:56.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.942+0000 7f81a47c1640 1 -- 172.21.15.38:0/160552340 >> 172.21.15.38:0/160552340 conn(0x7f819c0d3fb0 msgr2=0x7f819c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:56.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.942+0000 7f81a47c1640 1 -- 172.21.15.38:0/160552340 shutdown_connections 2024-04-25T08:41:56.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:56.942+0000 7f81a47c1640 1 -- 172.21.15.38:0/160552340 wait complete. 2024-04-25T08:41:56.946 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:41:56.946 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:41:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.057+0000 7f8e2b86b640 1 Processor -- start 2024-04-25T08:41:57.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.057+0000 7f8e2b86b640 1 -- start start 2024-04-25T08:41:57.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.058+0000 7f8e2b86b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8e2415b990 0x7f8e240cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.058+0000 7f8e2b86b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e2415f5f0 0x7f8e240d1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.058+0000 7f8e2b86b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2415fd00 con 0x7f8e240d1b50 2024-04-25T08:41:57.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.058+0000 7f8e2b86b640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e24160580 con 0x7f8e2415b990 2024-04-25T08:41:57.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.058+0000 7f8e2b86b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2415fa30 con 0x7f8e2415f5f0 2024-04-25T08:41:57.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.058+0000 7f8e29607640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8e2415b990 0x7f8e240cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.058+0000 7f8e29607640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8e2415b990 0x7f8e240cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60670/0 (socket says 172.21.15.38:60670) 2024-04-25T08:41:57.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.058+0000 7f8e29607640 1 -- 172.21.15.38:0/158823020 learned_addr learned my addr 172.21.15.38:0/158823020 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:57.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.059+0000 7f8e28e06640 1 --2- 172.21.15.38:0/158823020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e2415f5f0 0x7f8e240d1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.059+0000 7f8e29607640 1 -- 172.21.15.38:0/158823020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e2415f5f0 msgr2=0x7f8e240d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:57.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.059+0000 7f8e29607640 1 --2- 172.21.15.38:0/158823020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e2415f5f0 0x7f8e240d1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.059+0000 7f8e29607640 1 -- 172.21.15.38:0/158823020 >> v1:172.21.15.107:6789/0 conn(0x7f8e240d1b50 legacy=0x7f8e240b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:57.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.059+0000 7f8e29607640 1 -- 172.21.15.38:0/158823020 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2415ffc0 con 0x7f8e2415b990 2024-04-25T08:41:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.059+0000 7f8e29607640 1 --2- 172.21.15.38:0/158823020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e2415b990 0x7f8e240cf580 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009870 tx=0x7f8e2002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a81c103ef46fa8d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.059+0000 7f8e1bfff640 1 -- 172.21.15.38:0/158823020 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20037070 con 0x7f8e2415b990 2024-04-25T08:41:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.060+0000 7f8e1bfff640 1 -- 172.21.15.38:0/158823020 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e20032440 con 0x7f8e2415b990 2024-04-25T08:41:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.060+0000 7f8e1bfff640 1 -- 172.21.15.38:0/158823020 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20032810 con 0x7f8e2415b990 2024-04-25T08:41:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.060+0000 7f8e2b86b640 1 -- 172.21.15.38:0/158823020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e2415b990 msgr2=0x7f8e240cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.060+0000 7f8e2b86b640 1 --2- 172.21.15.38:0/158823020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e2415b990 0x7f8e240cf580 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009870 tx=0x7f8e2002a180 comp rx=0 tx=0).stop 2024-04-25T08:41:57.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.060+0000 7f8e2b86b640 1 -- 172.21.15.38:0/158823020 shutdown_connections 2024-04-25T08:41:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.060+0000 7f8e2b86b640 1 --2- 172.21.15.38:0/158823020 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e2415f5f0 0x7f8e240d1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.060+0000 7f8e2b86b640 1 --2- 172.21.15.38:0/158823020 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e2415b990 0x7f8e240cf580 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.060+0000 7f8e2b86b640 1 -- 172.21.15.38:0/158823020 >> 172.21.15.38:0/158823020 conn(0x7f8e2408b000 msgr2=0x7f8e2408b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.060+0000 7f8e2b86b640 1 -- 172.21.15.38:0/158823020 shutdown_connections 2024-04-25T08:41:57.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.060+0000 7f8e2b86b640 1 -- 172.21.15.38:0/158823020 wait complete. 2024-04-25T08:41:57.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.061+0000 7f8e2b86b640 1 Processor -- start 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.061+0000 7f8e2b86b640 1 -- start start 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e2b86b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e240d1b50 0x7f8e24158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e2b86b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e2415b990 0x7f8e240b6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e2b86b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e2415f5f0 0x7f8e2415c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e2b86b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2415bd70 con 0x7f8e240d1b50 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e2b86b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e240b1ac0 con 0x7f8e2415f5f0 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e2b86b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2415bef0 con 0x7f8e2415b990 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e29607640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e240d1b50 0x7f8e24158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e29607640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e240d1b50 0x7f8e24158740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60672/0 (socket says 172.21.15.38:60672) 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e29607640 1 -- 172.21.15.38:0/3480241416 learned_addr learned my addr 172.21.15.38:0/3480241416 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e29607640 1 -- 172.21.15.38:0/3480241416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e2415b990 msgr2=0x7f8e240b6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e29607640 1 --2- 172.21.15.38:0/3480241416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e2415b990 0x7f8e240b6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e29607640 1 -- 172.21.15.38:0/3480241416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e2415f5f0 msgr2=0x7f8e2415c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e29607640 1 --2- 172.21.15.38:0/3480241416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e2415f5f0 0x7f8e2415c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.062+0000 7f8e29607640 1 -- 172.21.15.38:0/3480241416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2415d730 con 0x7f8e240d1b50 2024-04-25T08:41:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.063+0000 7f8e29607640 1 --2- 172.21.15.38:0/3480241416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e240d1b50 0x7f8e24158740 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f8e20000c00 tx=0x7f8e200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.063+0000 7f8e19ffb640 1 -- 172.21.15.38:0/3480241416 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20037040 con 0x7f8e240d1b50 2024-04-25T08:41:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.063+0000 7f8e19ffb640 1 -- 172.21.15.38:0/3480241416 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e20042070 con 0x7f8e240d1b50 2024-04-25T08:41:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.063+0000 7f8e19ffb640 1 -- 172.21.15.38:0/3480241416 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e20030bd0 con 0x7f8e240d1b50 2024-04-25T08:41:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.063+0000 7f8e2b86b640 1 -- 172.21.15.38:0/3480241416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2415d170 con 0x7f8e240d1b50 2024-04-25T08:41:57.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.063+0000 7f8e2b86b640 1 -- 172.21.15.38:0/3480241416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2406be20 con 0x7f8e240d1b50 2024-04-25T08:41:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.064+0000 7f8e2b86b640 1 -- 172.21.15.38:0/3480241416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e24068e80 con 0x7f8e240d1b50 2024-04-25T08:41:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.064+0000 7f8e19ffb640 1 -- 172.21.15.38:0/3480241416 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e20047460 con 0x7f8e240d1b50 2024-04-25T08:41:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.065+0000 7f8e19ffb640 1 --2- 172.21.15.38:0/3480241416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8e0806b4d0 0x7f8e0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.065+0000 7f8e19ffb640 1 -- 172.21.15.38:0/3480241416 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8e200aacf0 con 0x7f8e240d1b50 2024-04-25T08:41:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.065+0000 7f8e28e06640 1 --2- 172.21.15.38:0/3480241416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8e0806b4d0 0x7f8e0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.066+0000 7f8e28e06640 1 --2- 172.21.15.38:0/3480241416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8e0806b4d0 0x7f8e0806d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8e240cd490 tx=0x7f8e14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.069+0000 7f8e19ffb640 1 -- 172.21.15.38:0/3480241416 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e200af070 con 0x7f8e240d1b50 2024-04-25T08:41:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.248+0000 7f8e2b86b640 1 -- 172.21.15.38:0/3480241416 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e240cf5b0 con 0x7f8e240d1b50 2024-04-25T08:41:57.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.249+0000 7f8e19ffb640 1 -- 172.21.15.38:0/3480241416 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v136) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e20045360 con 0x7f8e240d1b50 2024-04-25T08:41:57.241 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:41:57.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.250+0000 7f8e2b86b640 1 -- 172.21.15.38:0/3480241416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8e0806b4d0 msgr2=0x7f8e0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.250+0000 7f8e2b86b640 1 --2- 172.21.15.38:0/3480241416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8e0806b4d0 0x7f8e0806d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f8e240cd490 tx=0x7f8e14008040 comp rx=0 tx=0).stop 2024-04-25T08:41:57.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.250+0000 7f8e2b86b640 1 -- 172.21.15.38:0/3480241416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e240d1b50 msgr2=0x7f8e24158740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.250+0000 7f8e2b86b640 1 --2- 172.21.15.38:0/3480241416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e240d1b50 0x7f8e24158740 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f8e20000c00 tx=0x7f8e200046c0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.251+0000 7f8e2b86b640 1 -- 172.21.15.38:0/3480241416 shutdown_connections 2024-04-25T08:41:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.251+0000 7f8e2b86b640 1 --2- 172.21.15.38:0/3480241416 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e2415f5f0 0x7f8e2415c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.251+0000 7f8e2b86b640 1 --2- 172.21.15.38:0/3480241416 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8e0806b4d0 0x7f8e0806d990 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.251+0000 7f8e2b86b640 1 --2- 172.21.15.38:0/3480241416 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e2415b990 0x7f8e240b6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.251+0000 7f8e2b86b640 1 --2- 172.21.15.38:0/3480241416 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e240d1b50 0x7f8e24158740 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.251+0000 7f8e2b86b640 1 -- 172.21.15.38:0/3480241416 >> 172.21.15.38:0/3480241416 conn(0x7f8e2408b000 msgr2=0x7f8e2415ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.251+0000 7f8e2b86b640 1 -- 172.21.15.38:0/3480241416 shutdown_connections 2024-04-25T08:41:57.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.251+0000 7f8e2b86b640 1 -- 172.21.15.38:0/3480241416 wait complete. 2024-04-25T08:41:57.254 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T08:41:57.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.363+0000 7f546d3c8640 1 Processor -- start 2024-04-25T08:41:57.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.363+0000 7f546d3c8640 1 -- start start 2024-04-25T08:41:57.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f546d3c8640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f54680d0140 0x7f54680d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f546d3c8640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f54680d2400 0x7f54680d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f546d3c8640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5468162160 con 0x7f546815da90 2024-04-25T08:41:57.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f546d3c8640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54680d0b00 con 0x7f54680d0140 2024-04-25T08:41:57.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f546d3c8640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54680d19b0 con 0x7f54680d2400 2024-04-25T08:41:57.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f54667fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f54680d0140 0x7f54680d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f5466ffd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f546815da90 0x7f5468161710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:35876/0 (socket says 172.21.15.38:35876) 2024-04-25T08:41:57.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f5466ffd640 1 -- 172.21.15.38:0/1132601484 learned_addr learned my addr 172.21.15.38:0/1132601484 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:57.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f54677fe640 1 --2- 172.21.15.38:0/1132601484 >> v2:172.21.15.107:3300/0 conn(0x7f54680d2400 0x7f54680d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f54667fc640 1 -- 172.21.15.38:0/1132601484 >> v2:172.21.15.107:3300/0 conn(0x7f54680d2400 msgr2=0x7f54680d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:57.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f54667fc640 1 --2- 172.21.15.38:0/1132601484 >> v2:172.21.15.107:3300/0 conn(0x7f54680d2400 0x7f54680d2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f54667fc640 1 -- 172.21.15.38:0/1132601484 >> v1:172.21.15.38:6789/0 conn(0x7f546815da90 legacy=0x7f5468161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f5465ffb640 1 -- 172.21.15.38:0/1132601484 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782324085 0 0) 0x7f5468162160 con 0x7f546815da90 2024-04-25T08:41:57.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.364+0000 7f54667fc640 1 -- 172.21.15.38:0/1132601484 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5468162440 con 0x7f54680d0140 2024-04-25T08:41:57.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.365+0000 7f54667fc640 1 --2- 172.21.15.38:0/1132601484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54680d0140 0x7f54680d05c0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f54680cdf10 tx=0x7f545002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cb623fce1606b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.365+0000 7f5465ffb640 1 -- 172.21.15.38:0/1132601484 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5450037070 con 0x7f54680d0140 2024-04-25T08:41:57.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.365+0000 7f5465ffb640 1 -- 172.21.15.38:0/1132601484 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5450032440 con 0x7f54680d0140 2024-04-25T08:41:57.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.365+0000 7f5465ffb640 1 -- 172.21.15.38:0/1132601484 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545002ae60 con 0x7f54680d0140 2024-04-25T08:41:57.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.366+0000 7f546d3c8640 1 -- 172.21.15.38:0/1132601484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54680d0140 msgr2=0x7f54680d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.366+0000 7f546d3c8640 1 --2- 172.21.15.38:0/1132601484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54680d0140 0x7f54680d05c0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f54680cdf10 tx=0x7f545002a110 comp rx=0 tx=0).stop 2024-04-25T08:41:57.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.366+0000 7f546d3c8640 1 -- 172.21.15.38:0/1132601484 shutdown_connections 2024-04-25T08:41:57.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.366+0000 7f546d3c8640 1 --2- 172.21.15.38:0/1132601484 >> v2:172.21.15.107:3300/0 conn(0x7f54680d2400 0x7f54680d2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.366+0000 7f546d3c8640 1 --2- 172.21.15.38:0/1132601484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54680d0140 0x7f54680d05c0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.366+0000 7f546d3c8640 1 -- 172.21.15.38:0/1132601484 >> 172.21.15.38:0/1132601484 conn(0x7f546806c060 msgr2=0x7f546806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:57.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.366+0000 7f546d3c8640 1 -- 172.21.15.38:0/1132601484 shutdown_connections 2024-04-25T08:41:57.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.366+0000 7f546d3c8640 1 -- 172.21.15.38:0/1132601484 wait complete. 2024-04-25T08:41:57.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.367+0000 7f546d3c8640 1 Processor -- start 2024-04-25T08:41:57.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.367+0000 7f546d3c8640 1 -- start start 2024-04-25T08:41:57.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f546d3c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54680d0140 0x7f54681e9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f546d3c8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54680d2400 0x7f54680c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f546d3c8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f546815da90 0x7f54681519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f546d3c8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5468161e20 con 0x7f54680d0140 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f546d3c8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54680d1f70 con 0x7f546815da90 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f546d3c8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54680b9400 con 0x7f54680d2400 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f5466ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54680d0140 0x7f54681e9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f5466ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54680d0140 0x7f54681e9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60684/0 (socket says 172.21.15.38:60684) 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f5466ffd640 1 -- 172.21.15.38:0/2082150342 learned_addr learned my addr 172.21.15.38:0/2082150342 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f54677fe640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f546815da90 0x7f54681519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f54667fc640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54680d2400 0x7f54680c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f5466ffd640 1 -- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54680d2400 msgr2=0x7f54680c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f5466ffd640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54680d2400 0x7f54680c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f5466ffd640 1 -- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f546815da90 msgr2=0x7f54681519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:57.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f5466ffd640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f546815da90 0x7f54681519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f5466ffd640 1 -- 172.21.15.38:0/2082150342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54680c8830 con 0x7f54680d0140 2024-04-25T08:41:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.368+0000 7f54677fe640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f546815da90 0x7f54681519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:41:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.369+0000 7f5466ffd640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54680d0140 0x7f54681e9480 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f546000c4d0 tx=0x7f546000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.369+0000 7f545f7fe640 1 -- 172.21.15.38:0/2082150342 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460018070 con 0x7f54680d0140 2024-04-25T08:41:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.369+0000 7f545f7fe640 1 -- 172.21.15.38:0/2082150342 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5460007500 con 0x7f54680d0140 2024-04-25T08:41:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.369+0000 7f546d3c8640 1 -- 172.21.15.38:0/2082150342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54680c8b10 con 0x7f54680d0140 2024-04-25T08:41:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.369+0000 7f545f7fe640 1 -- 172.21.15.38:0/2082150342 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460004a20 con 0x7f54680d0140 2024-04-25T08:41:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.369+0000 7f546d3c8640 1 -- 172.21.15.38:0/2082150342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f546815a510 con 0x7f54680d0140 2024-04-25T08:41:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.370+0000 7f546d3c8640 1 -- 172.21.15.38:0/2082150342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5468068e80 con 0x7f54680d0140 2024-04-25T08:41:57.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.370+0000 7f545f7fe640 1 -- 172.21.15.38:0/2082150342 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5460004bc0 con 0x7f54680d0140 2024-04-25T08:41:57.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.371+0000 7f545f7fe640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f544806b4d0 0x7f544806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:57.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.371+0000 7f545f7fe640 1 -- 172.21.15.38:0/2082150342 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5460089570 con 0x7f54680d0140 2024-04-25T08:41:57.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.371+0000 7f54667fc640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f544806b4d0 0x7f544806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:57.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.371+0000 7f54667fc640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f544806b4d0 0x7f544806d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5450034000 tx=0x7f545002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:57.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.375+0000 7f545f7fe640 1 -- 172.21.15.38:0/2082150342 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5460058400 con 0x7f54680d0140 2024-04-25T08:41:57.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:57.632+0000 7f546d3c8640 1 -- 172.21.15.38:0/2082150342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f546815aa10 con 0x7f54680d0140 2024-04-25T08:41:58.634 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.642+0000 7f545f7fe640 1 -- 172.21.15.38:0/2082150342 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v504) v1 ==== 162+0+0 (secure 0 0 0) 0x7f5460057d00 con 0x7f54680d0140 2024-04-25T08:41:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.645+0000 7f546d3c8640 1 -- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f544806b4d0 msgr2=0x7f544806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.645+0000 7f546d3c8640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f544806b4d0 0x7f544806d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5450034000 tx=0x7f545002e000 comp rx=0 tx=0).stop 2024-04-25T08:41:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.645+0000 7f546d3c8640 1 -- 172.21.15.38:0/2082150342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54680d0140 msgr2=0x7f54681e9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.645+0000 7f546d3c8640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54680d0140 0x7f54681e9480 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f546000c4d0 tx=0x7f546000c990 comp rx=0 tx=0).stop 2024-04-25T08:41:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.645+0000 7f546d3c8640 1 -- 172.21.15.38:0/2082150342 shutdown_connections 2024-04-25T08:41:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.645+0000 7f546d3c8640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f546815da90 0x7f54681519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.645+0000 7f546d3c8640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f544806b4d0 0x7f544806d990 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.645+0000 7f546d3c8640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f54680d2400 0x7f54680c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.645+0000 7f546d3c8640 1 --2- 172.21.15.38:0/2082150342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54680d0140 0x7f54681e9480 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.645+0000 7f546d3c8640 1 -- 172.21.15.38:0/2082150342 >> 172.21.15.38:0/2082150342 conn(0x7f546806c060 msgr2=0x7f546815ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:58.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.645+0000 7f546d3c8640 1 -- 172.21.15.38:0/2082150342 shutdown_connections 2024-04-25T08:41:58.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.645+0000 7f546d3c8640 1 -- 172.21.15.38:0/2082150342 wait complete. 2024-04-25T08:41:58.652 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:41:58.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.770+0000 7f94d1f89640 1 Processor -- start 2024-04-25T08:41:58.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.770+0000 7f94d1f89640 1 -- start start 2024-04-25T08:41:58.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.771+0000 7f94d1f89640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94cc0d0320 0x7f94cc0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:58.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.771+0000 7f94d1f89640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f94cc0d0700 0x7f94cc0d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:58.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.771+0000 7f94d1f89640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f94cc0d1e90 0x7f94cc0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:58.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.771+0000 7f94d1f89640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94cc0d2330 con 0x7f94cc0d0700 2024-04-25T08:41:58.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.771+0000 7f94d1f89640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94cc0d2610 con 0x7f94cc0d1e90 2024-04-25T08:41:58.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.771+0000 7f94d1f89640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc0d28f0 con 0x7f94cc0d0320 2024-04-25T08:41:58.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.771+0000 7f94caffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f94cc0d0700 0x7f94cc0d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:58.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.771+0000 7f94caffd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f94cc0d0700 0x7f94cc0d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60694/0 (socket says 172.21.15.38:60694) 2024-04-25T08:41:58.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.771+0000 7f94caffd640 1 -- 172.21.15.38:0/2561259843 learned_addr learned my addr 172.21.15.38:0/2561259843 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:58.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.772+0000 7f94cb7fe640 1 --2- 172.21.15.38:0/2561259843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94cc0d0320 0x7f94cc0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:58.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.772+0000 7f94cbfff640 1 --2- 172.21.15.38:0/2561259843 >> v2:172.21.15.107:3300/0 conn(0x7f94cc0d1e90 0x7f94cc0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:58.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.772+0000 7f94caffd640 1 -- 172.21.15.38:0/2561259843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94cc0d0320 msgr2=0x7f94cc0d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:58.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.772+0000 7f94caffd640 1 --2- 172.21.15.38:0/2561259843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94cc0d0320 0x7f94cc0d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.772+0000 7f94caffd640 1 -- 172.21.15.38:0/2561259843 >> v2:172.21.15.107:3300/0 conn(0x7f94cc0d1e90 msgr2=0x7f94cc0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:58.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.772+0000 7f94caffd640 1 --2- 172.21.15.38:0/2561259843 >> v2:172.21.15.107:3300/0 conn(0x7f94cc0d1e90 0x7f94cc0b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.772+0000 7f94caffd640 1 -- 172.21.15.38:0/2561259843 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc0d2bd0 con 0x7f94cc0d0700 2024-04-25T08:41:58.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.772+0000 7f94caffd640 1 --2- 172.21.15.38:0/2561259843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94cc0d0700 0x7f94cc0d1950 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0b68d0 tx=0x7f94c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb35ce688630e71c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:58.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.773+0000 7f94ca7fc640 1 -- 172.21.15.38:0/2561259843 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0037070 con 0x7f94cc0d0700 2024-04-25T08:41:58.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.773+0000 7f94ca7fc640 1 -- 172.21.15.38:0/2561259843 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c0032440 con 0x7f94cc0d0700 2024-04-25T08:41:58.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.773+0000 7f94ca7fc640 1 -- 172.21.15.38:0/2561259843 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0032810 con 0x7f94cc0d0700 2024-04-25T08:41:58.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.773+0000 7f94d1f89640 1 -- 172.21.15.38:0/2561259843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94cc0d0700 msgr2=0x7f94cc0d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:58.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.773+0000 7f94d1f89640 1 --2- 172.21.15.38:0/2561259843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94cc0d0700 0x7f94cc0d1950 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0b68d0 tx=0x7f94c002a270 comp rx=0 tx=0).stop 2024-04-25T08:41:58.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.773+0000 7f94d1f89640 1 -- 172.21.15.38:0/2561259843 shutdown_connections 2024-04-25T08:41:58.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.773+0000 7f94d1f89640 1 --2- 172.21.15.38:0/2561259843 >> v2:172.21.15.107:3300/0 conn(0x7f94cc0d1e90 0x7f94cc0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.773+0000 7f94d1f89640 1 --2- 172.21.15.38:0/2561259843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94cc0d0700 0x7f94cc0d1950 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.773+0000 7f94d1f89640 1 --2- 172.21.15.38:0/2561259843 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94cc0d0320 0x7f94cc0d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.773+0000 7f94d1f89640 1 -- 172.21.15.38:0/2561259843 >> 172.21.15.38:0/2561259843 conn(0x7f94cc0668f0 msgr2=0x7f94cc066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:58.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.773+0000 7f94d1f89640 1 -- 172.21.15.38:0/2561259843 shutdown_connections 2024-04-25T08:41:58.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.773+0000 7f94d1f89640 1 -- 172.21.15.38:0/2561259843 wait complete. 2024-04-25T08:41:58.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.774+0000 7f94d1f89640 1 Processor -- start 2024-04-25T08:41:58.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.775+0000 7f94d1f89640 1 -- start start 2024-04-25T08:41:58.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.775+0000 7f94d1f89640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94cc0d0320 0x7f94cc1f3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:58.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.775+0000 7f94d1f89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94cc0d0700 0x7f94cc15af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:58.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.775+0000 7f94d1f89640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94cc0d1e90 0x7f94cc1e7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:58.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.775+0000 7f94d1f89640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc0d0b40 con 0x7f94cc0d0700 2024-04-25T08:41:58.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.775+0000 7f94d1f89640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc161210 con 0x7f94cc0d1e90 2024-04-25T08:41:58.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.775+0000 7f94d1f89640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94cc0b3310 con 0x7f94cc0d0320 2024-04-25T08:41:58.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.775+0000 7f94caffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94cc0d0700 0x7f94cc15af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:58.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.775+0000 7f94caffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94cc0d0700 0x7f94cc15af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60696/0 (socket says 172.21.15.38:60696) 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.775+0000 7f94caffd640 1 -- 172.21.15.38:0/2972127197 learned_addr learned my addr 172.21.15.38:0/2972127197 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.776+0000 7f94caffd640 1 -- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94cc0d0320 msgr2=0x7f94cc1f3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.776+0000 7f94caffd640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94cc0d0320 0x7f94cc1f3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.776+0000 7f94cbfff640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94cc0d1e90 0x7f94cc1e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.776+0000 7f94caffd640 1 -- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94cc0d1e90 msgr2=0x7f94cc1e7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.776+0000 7f94caffd640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94cc0d1e90 0x7f94cc1e7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.776+0000 7f94caffd640 1 -- 172.21.15.38:0/2972127197 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94cc0b35f0 con 0x7f94cc0d0700 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.776+0000 7f94caffd640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94cc0d0700 0x7f94cc15af00 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f94c002a2b0 tx=0x7f94c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.776+0000 7f94b3fff640 1 -- 172.21.15.38:0/2972127197 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0041020 con 0x7f94cc0d0700 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.776+0000 7f94b3fff640 1 -- 172.21.15.38:0/2972127197 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94c003d620 con 0x7f94cc0d0700 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.776+0000 7f94d1f89640 1 -- 172.21.15.38:0/2972127197 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94cc0b38d0 con 0x7f94cc0d0700 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.776+0000 7f94b3fff640 1 -- 172.21.15.38:0/2972127197 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0037040 con 0x7f94cc0d0700 2024-04-25T08:41:58.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.776+0000 7f94d1f89640 1 -- 172.21.15.38:0/2972127197 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94cc1e71b0 con 0x7f94cc0d0700 2024-04-25T08:41:58.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.777+0000 7f94d1f89640 1 -- 172.21.15.38:0/2972127197 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94cc068e80 con 0x7f94cc0d0700 2024-04-25T08:41:58.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.778+0000 7f94b3fff640 1 -- 172.21.15.38:0/2972127197 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94c0044020 con 0x7f94cc0d0700 2024-04-25T08:41:58.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.778+0000 7f94b3fff640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94b806b500 0x7f94b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:58.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.778+0000 7f94b3fff640 1 -- 172.21.15.38:0/2972127197 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f94c00aa6f0 con 0x7f94cc0d0700 2024-04-25T08:41:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.778+0000 7f94cb7fe640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94b806b500 0x7f94b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:58.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.779+0000 7f94cb7fe640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94b806b500 0x7f94b806d9c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009a40 tx=0x7f94bc009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:58.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.782+0000 7f94b3fff640 1 -- 172.21.15.38:0/2972127197 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94c004b030 con 0x7f94cc0d0700 2024-04-25T08:41:58.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.965+0000 7f94d1f89640 1 -- 172.21.15.38:0/2972127197 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f94cc0b3050 con 0x7f94cc0d0700 2024-04-25T08:41:58.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.968+0000 7f94b3fff640 1 -- 172.21.15.38:0/2972127197 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f94c0079720 con 0x7f94cc0d0700 2024-04-25T08:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.970+0000 7f94d1f89640 1 -- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94b806b500 msgr2=0x7f94b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.970+0000 7f94d1f89640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94b806b500 0x7f94b806d9c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009a40 tx=0x7f94bc009500 comp rx=0 tx=0).stop 2024-04-25T08:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.970+0000 7f94d1f89640 1 -- 172.21.15.38:0/2972127197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94cc0d0700 msgr2=0x7f94cc15af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.970+0000 7f94d1f89640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94cc0d0700 0x7f94cc15af00 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f94c002a2b0 tx=0x7f94c00046c0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.970+0000 7f94d1f89640 1 -- 172.21.15.38:0/2972127197 shutdown_connections 2024-04-25T08:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.970+0000 7f94d1f89640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f94cc0d1e90 0x7f94cc1e7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.970+0000 7f94d1f89640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f94cc0d0700 0x7f94cc15af00 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.970+0000 7f94d1f89640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f94b806b500 0x7f94b806d9c0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.970+0000 7f94d1f89640 1 --2- 172.21.15.38:0/2972127197 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f94cc0d0320 0x7f94cc1f3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:58.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.970+0000 7f94d1f89640 1 -- 172.21.15.38:0/2972127197 >> 172.21.15.38:0/2972127197 conn(0x7f94cc0668f0 msgr2=0x7f94cc0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:58.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.970+0000 7f94d1f89640 1 -- 172.21.15.38:0/2972127197 shutdown_connections 2024-04-25T08:41:58.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:58.970+0000 7f94d1f89640 1 -- 172.21.15.38:0/2972127197 wait complete. 2024-04-25T08:41:58.974 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:41:58.974 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:41:59.025 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:41:59.064 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:41:59.064 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:41:59.064 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:41:59.080 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:41:59.080 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:41:59.080 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:41:59.122 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:41:59.122 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:41:59.122 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:41:59.137 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:41:59.137 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:41:59.138 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:41:59.179 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:41:59.179 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:41:59.179 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:41:59.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.288+0000 7fc3815a5640 1 Processor -- start 2024-04-25T08:41:59.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.288+0000 7fc3815a5640 1 -- start start 2024-04-25T08:41:59.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.289+0000 7fc3815a5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc37c15df50 0x7fc37c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.289+0000 7fc3815a5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37c0ceff0 con 0x7fc37c06c110 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.289+0000 7fc3815a5640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37c0d1f70 con 0x7fc37c0ce2e0 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.289+0000 7fc3815a5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37c0cf2d0 con 0x7fc37c15df50 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.289+0000 7fc37affd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc37c06c110 0x7fc37c06c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36192/0 (socket says 172.21.15.38:36192) 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.289+0000 7fc37affd640 1 -- 172.21.15.38:0/1302455882 learned_addr learned my addr 172.21.15.38:0/1302455882 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.289+0000 7fc37a7fc640 1 --2- 172.21.15.38:0/1302455882 >> v2:172.21.15.107:3300/0 conn(0x7fc37c15df50 0x7fc37c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.289+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701921397 0 0) 0x7fc37c0ceff0 con 0x7fc37c06c110 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.289+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc368003520 con 0x7fc37c06c110 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2361310619 0 0) 0x7fc368003520 con 0x7fc37c06c110 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 >> v2:172.21.15.107:3300/0 conn(0x7fc37c15df50 msgr2=0x7fc37c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 --2- 172.21.15.38:0/1302455882 >> v2:172.21.15.107:3300/0 conn(0x7fc37c15df50 0x7fc37c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 >> v1:172.21.15.107:6789/0 conn(0x7fc37c0ce2e0 legacy=0x7fc37c0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c0d2250 con 0x7fc37c06c110 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fc374001670 con 0x7fc37c06c110 2024-04-25T08:41:59.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 >> v1:172.21.15.38:6789/0 conn(0x7fc37c06c110 legacy=0x7fc37c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 --2- 172.21.15.38:0/1302455882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3680039b0 0x7fc368003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 --2- 172.21.15.38:0/1302455882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 0x7fc368004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 --2- 172.21.15.38:0/1302455882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc368005080 0x7fc36800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c0cf2d0 con 0x7fc3680043b0 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36800bfe0 con 0x7fc368005080 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36800c1e0 con 0x7fc3680039b0 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc37a7fc640 1 --2- 172.21.15.38:0/1302455882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 0x7fc368004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc37affd640 1 --2- 172.21.15.38:0/1302455882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3680039b0 0x7fc368003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc37b7fe640 1 --2- 172.21.15.38:0/1302455882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc368005080 0x7fc36800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc37a7fc640 1 -- 172.21.15.38:0/1302455882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3680039b0 msgr2=0x7fc368003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc37a7fc640 1 --2- 172.21.15.38:0/1302455882 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3680039b0 0x7fc368003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc37a7fc640 1 -- 172.21.15.38:0/1302455882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc368005080 msgr2=0x7fc36800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc37a7fc640 1 --2- 172.21.15.38:0/1302455882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc368005080 0x7fc36800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc37a7fc640 1 -- 172.21.15.38:0/1302455882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc368003520 con 0x7fc3680043b0 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.290+0000 7fc37b7fe640 1 -- 172.21.15.38:0/1302455882 reap_dead start 2024-04-25T08:41:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.291+0000 7fc37a7fc640 1 --2- 172.21.15.38:0/1302455882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 0x7fc368004880 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fc364009ac0 tx=0x7fc368005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513d2ef5cab8cf0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:59.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.291+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364036070 con 0x7fc3680043b0 2024-04-25T08:41:59.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.291+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc364031400 con 0x7fc3680043b0 2024-04-25T08:41:59.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.291+0000 7fc379ffb640 1 -- 172.21.15.38:0/1302455882 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc364009690 con 0x7fc3680043b0 2024-04-25T08:41:59.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.292+0000 7fc3815a5640 1 -- 172.21.15.38:0/1302455882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 msgr2=0x7fc368004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.292+0000 7fc3815a5640 1 --2- 172.21.15.38:0/1302455882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 0x7fc368004880 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fc364009ac0 tx=0x7fc368005830 comp rx=0 tx=0).stop 2024-04-25T08:41:59.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.292+0000 7fc3815a5640 1 -- 172.21.15.38:0/1302455882 shutdown_connections 2024-04-25T08:41:59.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.292+0000 7fc3815a5640 1 --2- 172.21.15.38:0/1302455882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 0x7fc368004880 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.292+0000 7fc3815a5640 1 -- 172.21.15.38:0/1302455882 >> 172.21.15.38:0/1302455882 conn(0x7fc37c0d3fb0 msgr2=0x7fc37c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:59.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.292+0000 7fc3815a5640 1 -- 172.21.15.38:0/1302455882 shutdown_connections 2024-04-25T08:41:59.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.292+0000 7fc3815a5640 1 -- 172.21.15.38:0/1302455882 wait complete. 2024-04-25T08:41:59.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.293+0000 7fc3815a5640 1 Processor -- start 2024-04-25T08:41:59.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.293+0000 7fc3815a5640 1 -- start start 2024-04-25T08:41:59.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc3815a5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 0x7fc37c0c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc3815a5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc37c0cd540 0x7fc37c0c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc3815a5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc37c1609b0 0x7fc37c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc3815a5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c06c670 con 0x7fc3680043b0 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc3815a5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c06c4f0 con 0x7fc37c0cd540 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc3815a5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c06c7f0 con 0x7fc37c1609b0 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc37affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 0x7fc37c0c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc37affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 0x7fc37c0c83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51764/0 (socket says 172.21.15.38:51764) 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc37affd640 1 -- 172.21.15.38:0/159319504 learned_addr learned my addr 172.21.15.38:0/159319504 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc37b7fe640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc37c1609b0 0x7fc37c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc37a7fc640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc37c0cd540 0x7fc37c0c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc37affd640 1 -- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc37c1609b0 msgr2=0x7fc37c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc37affd640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc37c1609b0 0x7fc37c0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc37affd640 1 -- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc37c0cd540 msgr2=0x7fc37c0c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc37affd640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc37c0cd540 0x7fc37c0c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc37affd640 1 -- 172.21.15.38:0/159319504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c0c75f0 con 0x7fc3680043b0 2024-04-25T08:41:59.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc37affd640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 0x7fc37c0c83b0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fc374007290 tx=0x7fc374007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:59.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc3737fe640 1 -- 172.21.15.38:0/159319504 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37400f9c0 con 0x7fc3680043b0 2024-04-25T08:41:59.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc3737fe640 1 -- 172.21.15.38:0/159319504 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc374018070 con 0x7fc3680043b0 2024-04-25T08:41:59.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc3815a5640 1 -- 172.21.15.38:0/159319504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37c0c78d0 con 0x7fc3680043b0 2024-04-25T08:41:59.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.294+0000 7fc3737fe640 1 -- 172.21.15.38:0/159319504 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37400fdc0 con 0x7fc3680043b0 2024-04-25T08:41:59.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.295+0000 7fc3815a5640 1 -- 172.21.15.38:0/159319504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37c160e30 con 0x7fc3680043b0 2024-04-25T08:41:59.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.296+0000 7fc3815a5640 1 -- 172.21.15.38:0/159319504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37c068e80 con 0x7fc3680043b0 2024-04-25T08:41:59.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.296+0000 7fc3737fe640 1 -- 172.21.15.38:0/159319504 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc37401d460 con 0x7fc3680043b0 2024-04-25T08:41:59.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.297+0000 7fc3737fe640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc35c06b4d0 0x7fc35c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.297+0000 7fc3737fe640 1 -- 172.21.15.38:0/159319504 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fc37408a180 con 0x7fc3680043b0 2024-04-25T08:41:59.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.297+0000 7fc37a7fc640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc35c06b4d0 0x7fc35c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.297+0000 7fc37a7fc640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc35c06b4d0 0x7fc35c06d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc364033000 tx=0x7fc364008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:59.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.301+0000 7fc3737fe640 1 -- 172.21.15.38:0/159319504 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc374059120 con 0x7fc3680043b0 2024-04-25T08:41:59.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.518+0000 7fc3815a5640 1 -- 172.21.15.38:0/159319504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc37c0cda20 con 0x7fc3680043b0 2024-04-25T08:41:59.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.519+0000 7fc3737fe640 1 -- 172.21.15.38:0/159319504 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+2914 (secure 0 0 0) 0x7fc374058a20 con 0x7fc3680043b0 2024-04-25T08:41:59.511 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:41:59.511 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":83,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7259,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3766838413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6835","nonce":3766838413},{"type":"v1","addr":"172.21.15.107:6837","nonce":3766838413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":8137,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6836/3519462128","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3519462128},{"type":"v1","addr":"172.21.15.107:6836","nonce":3519462128}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":9969,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/1183157432","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":1183157432},{"type":"v1","addr":"172.21.15.38:6833","nonce":1183157432}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[]} 2024-04-25T08:41:59.512 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 83 2024-04-25T08:41:59.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.521+0000 7fc3815a5640 1 -- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc35c06b4d0 msgr2=0x7fc35c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.521+0000 7fc3815a5640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc35c06b4d0 0x7fc35c06d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc364033000 tx=0x7fc364008030 comp rx=0 tx=0).stop 2024-04-25T08:41:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.521+0000 7fc3815a5640 1 -- 172.21.15.38:0/159319504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 msgr2=0x7fc37c0c83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.521+0000 7fc3815a5640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 0x7fc37c0c83b0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fc374007290 tx=0x7fc374007750 comp rx=0 tx=0).stop 2024-04-25T08:41:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.521+0000 7fc3815a5640 1 -- 172.21.15.38:0/159319504 shutdown_connections 2024-04-25T08:41:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.521+0000 7fc3815a5640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc35c06b4d0 0x7fc35c06d990 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.521+0000 7fc3815a5640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc37c1609b0 0x7fc37c0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.521+0000 7fc3815a5640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc37c0cd540 0x7fc37c0c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.521+0000 7fc3815a5640 1 --2- 172.21.15.38:0/159319504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3680043b0 0x7fc37c0c83b0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.521+0000 7fc3815a5640 1 -- 172.21.15.38:0/159319504 >> 172.21.15.38:0/159319504 conn(0x7fc37c0d3fb0 msgr2=0x7fc37c15d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.521+0000 7fc3815a5640 1 -- 172.21.15.38:0/159319504 shutdown_connections 2024-04-25T08:41:59.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.521+0000 7fc3815a5640 1 -- 172.21.15.38:0/159319504 wait complete. 2024-04-25T08:41:59.525 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:41:59.525 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:41:59.526 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:41:59.526 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:41:59.526 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:41:59.526 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:41:59.526 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:41:59.526 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:41:59.526 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:41:59.527 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:41:59.527 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:41:59.527 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:41:59.527 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:41:59.527 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:41:59.535 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:41:59.535 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:41:59.535 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:41:59.536 DEBUG:teuthology.parallel:result is None 2024-04-25T08:41:59.538 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:41:59.538 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:41:59.539 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:41:59.539 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:41:59.577 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:41:59.578 DEBUG:teuthology.parallel:result is None 2024-04-25T08:41:59.581 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:41:59.581 DEBUG:teuthology.parallel:result is None 2024-04-25T08:41:59.582 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:41:59.582 DEBUG:teuthology.parallel:result is None 2024-04-25T08:41:59.582 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:41:59.609 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:41:59.653 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:41:59.663 INFO:tasks.ceph.mds.d.smithi107.stdout:starting mds.d at 2024-04-25T08:41:59.663 INFO:tasks.ceph.mds.b.smithi107.stdout:starting mds.b at 2024-04-25T08:41:59.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.697+0000 7f8bb9238640 1 Processor -- start 2024-04-25T08:41:59.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.697+0000 7f8bb9238640 1 -- start start 2024-04-25T08:41:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8bb9238640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8bb4099ce0 0x7f8bb409a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8bb9238640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8bb409a0c0 0x7f8bb40b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8bb9238640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb40cd120 con 0x7f8bb415eb20 2024-04-25T08:41:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8bb9238640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb40cf6c0 con 0x7f8bb4099ce0 2024-04-25T08:41:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8bb9238640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb40cd400 con 0x7f8bb409a0c0 2024-04-25T08:41:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8bb2d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8bb4099ce0 0x7f8bb409a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8bb2d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8bb4099ce0 0x7f8bb409a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51804/0 (socket says 172.21.15.38:51804) 2024-04-25T08:41:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8bb2d9d640 1 -- 172.21.15.38:0/4045772521 learned_addr learned my addr 172.21.15.38:0/4045772521 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8babfff640 1 --2- 172.21.15.38:0/4045772521 >> v2:172.21.15.107:3300/0 conn(0x7f8bb409a0c0 0x7f8bb40b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8bb2d9d640 1 -- 172.21.15.38:0/4045772521 >> v2:172.21.15.107:3300/0 conn(0x7f8bb409a0c0 msgr2=0x7f8bb40b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:41:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8bb2d9d640 1 --2- 172.21.15.38:0/4045772521 >> v2:172.21.15.107:3300/0 conn(0x7f8bb409a0c0 0x7f8bb40b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8bb2d9d640 1 -- 172.21.15.38:0/4045772521 >> v1:172.21.15.107:6789/0 conn(0x7f8bb415eb20 legacy=0x7f8bb415ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.698+0000 7f8bb2d9d640 1 -- 172.21.15.38:0/4045772521 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb40cf350 con 0x7f8bb4099ce0 2024-04-25T08:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.699+0000 7f8bb2d9d640 1 --2- 172.21.15.38:0/4045772521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bb4099ce0 0x7f8bb409a7b0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4072d90 tx=0x7f8ba402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26698db31e39833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.699+0000 7f8bb259c640 1 -- 172.21.15.38:0/4045772521 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4037070 con 0x7f8bb4099ce0 2024-04-25T08:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.699+0000 7f8bb259c640 1 -- 172.21.15.38:0/4045772521 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba4032440 con 0x7f8bb4099ce0 2024-04-25T08:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.699+0000 7f8bb259c640 1 -- 172.21.15.38:0/4045772521 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4032810 con 0x7f8bb4099ce0 2024-04-25T08:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.700+0000 7f8bb9238640 1 -- 172.21.15.38:0/4045772521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bb4099ce0 msgr2=0x7f8bb409a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.700+0000 7f8bb9238640 1 --2- 172.21.15.38:0/4045772521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bb4099ce0 0x7f8bb409a7b0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4072d90 tx=0x7f8ba402a270 comp rx=0 tx=0).stop 2024-04-25T08:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.700+0000 7f8bb9238640 1 -- 172.21.15.38:0/4045772521 shutdown_connections 2024-04-25T08:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.700+0000 7f8bb9238640 1 --2- 172.21.15.38:0/4045772521 >> v2:172.21.15.107:3300/0 conn(0x7f8bb409a0c0 0x7f8bb40b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.700+0000 7f8bb9238640 1 --2- 172.21.15.38:0/4045772521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bb4099ce0 0x7f8bb409a7b0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.700+0000 7f8bb9238640 1 -- 172.21.15.38:0/4045772521 >> 172.21.15.38:0/4045772521 conn(0x7f8bb40b30f0 msgr2=0x7f8bb40b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.700+0000 7f8bb9238640 1 -- 172.21.15.38:0/4045772521 shutdown_connections 2024-04-25T08:41:59.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.700+0000 7f8bb9238640 1 -- 172.21.15.38:0/4045772521 wait complete. 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.701+0000 7f8bb9238640 1 Processor -- start 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.701+0000 7f8bb9238640 1 -- start start 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb9238640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb4099ce0 0x7f8bb4157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb9238640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bb409a0c0 0x7f8bb40ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb9238640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bb415eb20 0x7f8bb4159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb9238640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40b4b10 con 0x7f8bb415eb20 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb9238640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40b4990 con 0x7f8bb4099ce0 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb9238640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb40cccb0 con 0x7f8bb409a0c0 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bb415eb20 0x7f8bb4159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb359e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bb415eb20 0x7f8bb4159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51818/0 (socket says 172.21.15.38:51818) 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb359e640 1 -- 172.21.15.38:0/2032288960 learned_addr learned my addr 172.21.15.38:0/2032288960 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb2d9d640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb4099ce0 0x7f8bb4157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8babfff640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bb409a0c0 0x7f8bb40ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb359e640 1 -- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bb409a0c0 msgr2=0x7f8bb40ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb359e640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bb409a0c0 0x7f8bb40ae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb359e640 1 -- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb4099ce0 msgr2=0x7f8bb4157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb359e640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb4099ce0 0x7f8bb4157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb359e640 1 -- 172.21.15.38:0/2032288960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb40b1830 con 0x7f8bb415eb20 2024-04-25T08:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.702+0000 7f8bb359e640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bb415eb20 0x7f8bb4159e30 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00c730 tx=0x7f8bac00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.703+0000 7f8bab7fe640 1 -- 172.21.15.38:0/2032288960 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac016020 con 0x7f8bb415eb20 2024-04-25T08:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.703+0000 7f8bab7fe640 1 -- 172.21.15.38:0/2032288960 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac006d50 con 0x7f8bb415eb20 2024-04-25T08:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.703+0000 7f8bb9238640 1 -- 172.21.15.38:0/2032288960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb40b1b10 con 0x7f8bb415eb20 2024-04-25T08:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.703+0000 7f8bab7fe640 1 -- 172.21.15.38:0/2032288960 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac003bb0 con 0x7f8bb415eb20 2024-04-25T08:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.703+0000 7f8bb9238640 1 -- 172.21.15.38:0/2032288960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb40b20d0 con 0x7f8bb415eb20 2024-04-25T08:41:59.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.704+0000 7f8bb9238640 1 -- 172.21.15.38:0/2032288960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb40cca20 con 0x7f8bb415eb20 2024-04-25T08:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.704+0000 7f8bab7fe640 1 -- 172.21.15.38:0/2032288960 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bac0050d0 con 0x7f8bb415eb20 2024-04-25T08:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.704+0000 7f8bab7fe640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8b9406b500 0x7f8b9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.705+0000 7f8bab7fe640 1 -- 172.21.15.38:0/2032288960 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8bac089aa0 con 0x7f8bb415eb20 2024-04-25T08:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.705+0000 7f8bb2d9d640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8b9406b500 0x7f8b9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:41:59.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.705+0000 7f8bb2d9d640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8b9406b500 0x7f8b9406d9c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4034000 tx=0x7f8ba402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:41:59.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.708+0000 7f8bab7fe640 1 -- 172.21.15.38:0/2032288960 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8bb40cca20 con 0x7f8bb415eb20 2024-04-25T08:41:59.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.902+0000 7f8bb9238640 1 -- 172.21.15.38:0/2032288960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8bb406fa20 con 0x7f8bb415eb20 2024-04-25T08:41:59.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.988+0000 7f8bab7fe640 1 -- 172.21.15.38:0/2032288960 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v138) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8bac020020 con 0x7f8bb415eb20 2024-04-25T08:41:59.981 INFO:teuthology.orchestra.run.smithi038.stderr: removed all blocklist entries 2024-04-25T08:41:59.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.991+0000 7f8bb9238640 1 -- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8b9406b500 msgr2=0x7f8b9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.991+0000 7f8bb9238640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8b9406b500 0x7f8b9406d9c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4034000 tx=0x7f8ba402e000 comp rx=0 tx=0).stop 2024-04-25T08:41:59.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.991+0000 7f8bb9238640 1 -- 172.21.15.38:0/2032288960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bb415eb20 msgr2=0x7f8bb4159e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:41:59.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.991+0000 7f8bb9238640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bb415eb20 0x7f8bb4159e30 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00c730 tx=0x7f8bac00cb30 comp rx=0 tx=0).stop 2024-04-25T08:41:59.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.991+0000 7f8bb9238640 1 -- 172.21.15.38:0/2032288960 shutdown_connections 2024-04-25T08:41:59.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.991+0000 7f8bb9238640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8bb415eb20 0x7f8bb4159e30 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.991+0000 7f8bb9238640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8b9406b500 0x7f8b9406d9c0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.991+0000 7f8bb9238640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bb409a0c0 0x7f8bb40ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.991+0000 7f8bb9238640 1 --2- 172.21.15.38:0/2032288960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bb4099ce0 0x7f8bb4157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:41:59.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.991+0000 7f8bb9238640 1 -- 172.21.15.38:0/2032288960 >> 172.21.15.38:0/2032288960 conn(0x7f8bb40b30f0 msgr2=0x7f8bb40cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:41:59.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.991+0000 7f8bb9238640 1 -- 172.21.15.38:0/2032288960 shutdown_connections 2024-04-25T08:41:59.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:41:59.991+0000 7f8bb9238640 1 -- 172.21.15.38:0/2032288960 wait complete. 2024-04-25T08:41:59.995 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:42:00.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.145+0000 7fc7855e7640 1 Processor -- start 2024-04-25T08:42:00.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.145+0000 7fc7855e7640 1 -- start start 2024-04-25T08:42:00.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.145+0000 7fc7855e7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7800b5470 0x7fc78015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.145+0000 7fc7855e7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7801454e0 con 0x7fc780145100 2024-04-25T08:42:00.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.145+0000 7fc7855e7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7800c47a0 con 0x7fc7800cea70 2024-04-25T08:42:00.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.145+0000 7fc7855e7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78015c210 con 0x7fc7800b5470 2024-04-25T08:42:00.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77effd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc780145100 0x7fc7800ce360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36206/0 (socket says 172.21.15.38:36206) 2024-04-25T08:42:00.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77effd640 1 -- 172.21.15.38:0/3784445568 learned_addr learned my addr 172.21.15.38:0/3784445568 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:00.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77f7fe640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7800b5470 0x7fc78015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170610690 0 0) 0x7fc7801454e0 con 0x7fc780145100 2024-04-25T08:42:00.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc768003520 con 0x7fc780145100 2024-04-25T08:42:00.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3410919727 0 0) 0x7fc768003520 con 0x7fc780145100 2024-04-25T08:42:00.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7800b5470 msgr2=0x7fc78015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:00.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7800b5470 0x7fc78015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 >> v1:172.21.15.107:6789/0 conn(0x7fc7800cea70 legacy=0x7fc7800b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7800c4a80 con 0x7fc780145100 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fc770001670 con 0x7fc780145100 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 >> v1:172.21.15.38:6789/0 conn(0x7fc780145100 legacy=0x7fc7800ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7680039e0 0x7fc768003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7800b5470 0x7fc768004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc768004d40 0x7fc7680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76800baf0 con 0x7fc7680039e0 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76800bca0 con 0x7fc768004d40 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77effd640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7680039e0 0x7fc768003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76800bea0 con 0x7fc7800b5470 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77e7fc640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7800b5470 0x7fc768004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.146+0000 7fc77f7fe640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc768004d40 0x7fc7680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.147+0000 7fc77effd640 1 -- 172.21.15.38:0/3784445568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7800b5470 msgr2=0x7fc768004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.147+0000 7fc77effd640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc7800b5470 0x7fc768004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.147+0000 7fc77effd640 1 -- 172.21.15.38:0/3784445568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc768004d40 msgr2=0x7fc7680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.147+0000 7fc77effd640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc768004d40 0x7fc7680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.147+0000 7fc77effd640 1 -- 172.21.15.38:0/3784445568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc768003520 con 0x7fc7680039e0 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.147+0000 7fc77f7fe640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc768004d40 0x7fc7680051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.147+0000 7fc77effd640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7680039e0 0x7fc768003e00 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fc77000c240 tx=0x7fc768005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9535c87a2724c5dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.147+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770007cf0 con 0x7fc7680039e0 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.148+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc770005af0 con 0x7fc7680039e0 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.148+0000 7fc77dffb640 1 -- 172.21.15.38:0/3784445568 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc770007070 con 0x7fc7680039e0 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.148+0000 7fc7855e7640 1 -- 172.21.15.38:0/3784445568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7680039e0 msgr2=0x7fc768003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.148+0000 7fc7855e7640 1 --2- 172.21.15.38:0/3784445568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7680039e0 0x7fc768003e00 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fc77000c240 tx=0x7fc768005830 comp rx=0 tx=0).stop 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.148+0000 7fc77f7fe640 1 -- 172.21.15.38:0/3784445568 reap_dead start 2024-04-25T08:42:00.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.148+0000 7fc7855e7640 1 -- 172.21.15.38:0/3784445568 shutdown_connections 2024-04-25T08:42:00.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.148+0000 7fc7855e7640 1 -- 172.21.15.38:0/3784445568 >> 172.21.15.38:0/3784445568 conn(0x7fc78006b210 msgr2=0x7fc78006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:00.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.148+0000 7fc7855e7640 1 -- 172.21.15.38:0/3784445568 shutdown_connections 2024-04-25T08:42:00.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.148+0000 7fc7855e7640 1 -- 172.21.15.38:0/3784445568 wait complete. 2024-04-25T08:42:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.149+0000 7fc7855e7640 1 Processor -- start 2024-04-25T08:42:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.149+0000 7fc7855e7640 1 -- start start 2024-04-25T08:42:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc7855e7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7680039e0 0x7fc78013c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc7855e7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc78013d8d0 0x7fc78015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc7855e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc78013dd50 0x7fc780139010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc7855e7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800c5320 con 0x7fc78013dd50 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc7855e7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7800c5040 con 0x7fc7680039e0 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc7855e7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78015bcd0 con 0x7fc78013d8d0 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc77f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc78013dd50 0x7fc780139010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc77f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc78013dd50 0x7fc780139010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51836/0 (socket says 172.21.15.38:51836) 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc77f7fe640 1 -- 172.21.15.38:0/1537134960 learned_addr learned my addr 172.21.15.38:0/1537134960 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc77e7fc640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc78013d8d0 0x7fc78015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc77effd640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7680039e0 0x7fc78013c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc77f7fe640 1 -- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc78013d8d0 msgr2=0x7fc78015adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc77f7fe640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc78013d8d0 0x7fc78015adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc77f7fe640 1 -- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7680039e0 msgr2=0x7fc78013c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc77f7fe640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7680039e0 0x7fc78013c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc77f7fe640 1 -- 172.21.15.38:0/1537134960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78013e220 con 0x7fc78013dd50 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.150+0000 7fc77e7fc640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc78013d8d0 0x7fc78015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.151+0000 7fc77f7fe640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc78013dd50 0x7fc780139010 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fc78015f220 tx=0x7fc778010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.151+0000 7fc7677fe640 1 -- 172.21.15.38:0/1537134960 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc77801a030 con 0x7fc78013dd50 2024-04-25T08:42:00.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.151+0000 7fc7677fe640 1 -- 172.21.15.38:0/1537134960 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc778003990 con 0x7fc78013dd50 2024-04-25T08:42:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.151+0000 7fc7855e7640 1 -- 172.21.15.38:0/1537134960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7801424b0 con 0x7fc78013dd50 2024-04-25T08:42:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.151+0000 7fc7677fe640 1 -- 172.21.15.38:0/1537134960 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc778003d80 con 0x7fc78013dd50 2024-04-25T08:42:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.151+0000 7fc7855e7640 1 -- 172.21.15.38:0/1537134960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78015b330 con 0x7fc78013dd50 2024-04-25T08:42:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.152+0000 7fc7855e7640 1 -- 172.21.15.38:0/1537134960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc780068e80 con 0x7fc78013dd50 2024-04-25T08:42:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.152+0000 7fc7677fe640 1 -- 172.21.15.38:0/1537134960 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc77801d070 con 0x7fc78013dd50 2024-04-25T08:42:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.153+0000 7fc7677fe640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc76c06b4d0 0x7fc76c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.153+0000 7fc7677fe640 1 -- 172.21.15.38:0/1537134960 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc77808a5c0 con 0x7fc78013dd50 2024-04-25T08:42:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.153+0000 7fc77effd640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc76c06b4d0 0x7fc76c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.153+0000 7fc77effd640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc76c06b4d0 0x7fc76c06d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc770032000 tx=0x7fc770009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:00.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.157+0000 7fc7677fe640 1 -- 172.21.15.38:0/1537134960 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc778059690 con 0x7fc78013dd50 2024-04-25T08:42:00.401 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.409+0000 7fc7855e7640 1 -- 172.21.15.38:0/1537134960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc7800cef90 con 0x7fc78013dd50 2024-04-25T08:42:00.402 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.409+0000 7fc7677fe640 1 -- 172.21.15.38:0/1537134960 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc778058f90 con 0x7fc78013dd50 2024-04-25T08:42:00.402 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:00.402 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:42:00.402 INFO:teuthology.orchestra.run.smithi038.stdout: "auth_dump": [ 2024-04-25T08:42:00.402 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.402 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.a", 2024-04-25T08:42:00.402 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmwq8CDBAA+cjd+QXtFgwZ3kxdABRWlA==", 2024-04-25T08:42:00.402 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.402 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.b", 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmPRFbFxAAojn2r/62vvf1qTy9QVuyIw==", 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.403 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.c", 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmGLjqEhAAyPRgyy5iMgtccapydv4uOQ==", 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mds.d", 2024-04-25T08:42:00.404 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmt0AUHhAAoQyQ7wXBMGE+7Hi84tUfAg==", 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.0", 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCxFCpmv+iTExAAEJihmsvBbp9xXxPnj49D6g==", 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.405 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.1", 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC0FCpmdnQKFBAADLAsYZEUhoJx292jIYZroQ==", 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.2", 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC3FCpm4f5BExAAMVSPkqsrnuLcO88AXKrGHQ==", 2024-04-25T08:42:00.406 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.3", 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQC6FCpmHNN7EhAABPhmnxGo70ZOd8Oex62Mdw==", 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.407 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.4", 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDAFCpmiThGFhAAG5Pyvn/JzL9NUoxHJkPMvg==", 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.5", 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDDFCpmb/FHFBAA2cMhMqQQc4Fk6j8FpjhnaQ==", 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:42:00.408 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.6", 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDGFCpmAZsSEhAAnGIqqcvuHMVSc8Zbk309kA==", 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.409 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "osd.7", 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQDJFCpmqdVUEBAAb9abblyUztrjwxxe4jsAwQ==", 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow profile osd", 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile osd", 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.0", 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw==", 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow r", 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:42:00.410 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.1", 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmL7eKKhAASeQEdHDBInfozdLVzP4DQw==", 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.411 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.2", 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmH22HJxAAWyrooyJn5hRs/bb0Rn1HHw==", 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.3", 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpm6Zr/LxAAxh9Y2gvvqj1jIaVVrQAvgw==", 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow", 2024-04-25T08:42:00.412 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow r", 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow rw", 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow rwx" 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "client.admin", 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCpFCpmlj/fLBAAvxLluFQFR8uhppGlNpaygA==", 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr": "allow *", 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow *", 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.413 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.x", 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcXHhCBAAvI5zmWidVTvOmEcVRWasEA==", 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: "entity": "mgr.y", 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: "key": "AQCqFCpmcr96BRAAQwUPxILJukYERAiEXeur+g==", 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: "caps": { 2024-04-25T08:42:00.414 INFO:teuthology.orchestra.run.smithi038.stdout: "mds": "allow *", 2024-04-25T08:42:00.415 INFO:teuthology.orchestra.run.smithi038.stdout: "mon": "allow profile mgr", 2024-04-25T08:42:00.415 INFO:teuthology.orchestra.run.smithi038.stdout: "osd": "allow *" 2024-04-25T08:42:00.415 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.415 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:00.415 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-04-25T08:42:00.415 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:42:00.416 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.411+0000 7fc7855e7640 1 -- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc76c06b4d0 msgr2=0x7fc76c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.411+0000 7fc7855e7640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc76c06b4d0 0x7fc76c06d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc770032000 tx=0x7fc770009030 comp rx=0 tx=0).stop 2024-04-25T08:42:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.411+0000 7fc7855e7640 1 -- 172.21.15.38:0/1537134960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc78013dd50 msgr2=0x7fc780139010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.411+0000 7fc7855e7640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc78013dd50 0x7fc780139010 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fc78015f220 tx=0x7fc778010590 comp rx=0 tx=0).stop 2024-04-25T08:42:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.411+0000 7fc7855e7640 1 -- 172.21.15.38:0/1537134960 shutdown_connections 2024-04-25T08:42:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.411+0000 7fc7855e7640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc78013dd50 0x7fc780139010 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.411+0000 7fc7855e7640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc76c06b4d0 0x7fc76c06d990 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.412+0000 7fc7855e7640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc78013d8d0 0x7fc78015adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.412+0000 7fc7855e7640 1 --2- 172.21.15.38:0/1537134960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7680039e0 0x7fc78013c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.412+0000 7fc7855e7640 1 -- 172.21.15.38:0/1537134960 >> 172.21.15.38:0/1537134960 conn(0x7fc78006b210 msgr2=0x7fc78015ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.412+0000 7fc7855e7640 1 -- 172.21.15.38:0/1537134960 shutdown_connections 2024-04-25T08:42:00.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.412+0000 7fc7855e7640 1 -- 172.21.15.38:0/1537134960 wait complete. 2024-04-25T08:42:00.418 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:00.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.527+0000 7fc345288640 1 Processor -- start 2024-04-25T08:42:00.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.527+0000 7fc345288640 1 -- start start 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.527+0000 7fc345288640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc3400b5030 0x7fc3400cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.527+0000 7fc345288640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3400cdb70 0x7fc34015abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc345288640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3400ce360 con 0x7fc340143f40 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc345288640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3400ce640 con 0x7fc3400b5030 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc345288640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3400ce920 con 0x7fc3400cdb70 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc33e7fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc3400b5030 0x7fc3400cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc33e7fc640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc3400b5030 0x7fc3400cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51848/0 (socket says 172.21.15.38:51848) 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc33effd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc340143f40 0x7fc3400b4920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36214/0 (socket says 172.21.15.38:36214) 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc33e7fc640 1 -- 172.21.15.38:0/503017846 learned_addr learned my addr 172.21.15.38:0/503017846 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc33f7fe640 1 --2- 172.21.15.38:0/503017846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3400cdb70 0x7fc34015abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc33e7fc640 1 -- 172.21.15.38:0/503017846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3400cdb70 msgr2=0x7fc34015abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc33e7fc640 1 --2- 172.21.15.38:0/503017846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3400cdb70 0x7fc34015abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc33e7fc640 1 -- 172.21.15.38:0/503017846 >> v1:172.21.15.38:6789/0 conn(0x7fc340143f40 legacy=0x7fc3400b4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc33e7fc640 1 -- 172.21.15.38:0/503017846 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3400cec00 con 0x7fc3400b5030 2024-04-25T08:42:00.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.528+0000 7fc33f7fe640 1 --2- 172.21.15.38:0/503017846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3400cdb70 0x7fc34015abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:00.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.529+0000 7fc33e7fc640 1 --2- 172.21.15.38:0/503017846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3400b5030 0x7fc3400cd630 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fc33400bb10 tx=0x7fc33402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a7b0d987f924ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:00.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.529+0000 7fc33dffb640 1 -- 172.21.15.38:0/503017846 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334036020 con 0x7fc3400b5030 2024-04-25T08:42:00.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.529+0000 7fc33dffb640 1 -- 172.21.15.38:0/503017846 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc334032820 con 0x7fc3400b5030 2024-04-25T08:42:00.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.529+0000 7fc33dffb640 1 -- 172.21.15.38:0/503017846 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334032bf0 con 0x7fc3400b5030 2024-04-25T08:42:00.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.529+0000 7fc345288640 1 -- 172.21.15.38:0/503017846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3400b5030 msgr2=0x7fc3400cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.529+0000 7fc345288640 1 --2- 172.21.15.38:0/503017846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3400b5030 0x7fc3400cd630 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fc33400bb10 tx=0x7fc33402c920 comp rx=0 tx=0).stop 2024-04-25T08:42:00.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.529+0000 7fc345288640 1 -- 172.21.15.38:0/503017846 shutdown_connections 2024-04-25T08:42:00.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.529+0000 7fc345288640 1 --2- 172.21.15.38:0/503017846 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc3400cdb70 0x7fc34015abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.529+0000 7fc345288640 1 --2- 172.21.15.38:0/503017846 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3400b5030 0x7fc3400cd630 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.529+0000 7fc345288640 1 -- 172.21.15.38:0/503017846 >> 172.21.15.38:0/503017846 conn(0x7fc34006b210 msgr2=0x7fc34006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:00.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.529+0000 7fc345288640 1 -- 172.21.15.38:0/503017846 shutdown_connections 2024-04-25T08:42:00.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.530+0000 7fc345288640 1 -- 172.21.15.38:0/503017846 wait complete. 2024-04-25T08:42:00.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.531+0000 7fc345288640 1 Processor -- start 2024-04-25T08:42:00.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.531+0000 7fc345288640 1 -- start start 2024-04-25T08:42:00.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc345288640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3400b5030 0x7fc3400c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc345288640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3400cdb70 0x7fc340142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc345288640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc340143f40 0x7fc340143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc345288640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3400ce140 con 0x7fc3400cdb70 2024-04-25T08:42:00.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc345288640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3400cdfc0 con 0x7fc3400b5030 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc345288640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc340144320 con 0x7fc340143f40 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc33e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3400cdb70 0x7fc340142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc33e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3400cdb70 0x7fc340142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51850/0 (socket says 172.21.15.38:51850) 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc33e7fc640 1 -- 172.21.15.38:0/2186583093 learned_addr learned my addr 172.21.15.38:0/2186583093 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc33f7fe640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc340143f40 0x7fc340143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc33effd640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3400b5030 0x7fc3400c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc33e7fc640 1 -- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc340143f40 msgr2=0x7fc340143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc33e7fc640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc340143f40 0x7fc340143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc33e7fc640 1 -- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3400b5030 msgr2=0x7fc3400c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc33e7fc640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3400b5030 0x7fc3400c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc33e7fc640 1 -- 172.21.15.38:0/2186583093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3401f5fd0 con 0x7fc3400cdb70 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc33e7fc640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3400cdb70 0x7fc340142df0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fc33402c960 tx=0x7fc3340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc3277fe640 1 -- 172.21.15.38:0/2186583093 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334036040 con 0x7fc3400cdb70 2024-04-25T08:42:00.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc3277fe640 1 -- 172.21.15.38:0/2186583093 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc334031c30 con 0x7fc3400cdb70 2024-04-25T08:42:00.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc345288640 1 -- 172.21.15.38:0/2186583093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3401f62b0 con 0x7fc3400cdb70 2024-04-25T08:42:00.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.532+0000 7fc3277fe640 1 -- 172.21.15.38:0/2186583093 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334044480 con 0x7fc3400cdb70 2024-04-25T08:42:00.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.533+0000 7fc345288640 1 -- 172.21.15.38:0/2186583093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3401f6870 con 0x7fc3400cdb70 2024-04-25T08:42:00.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.533+0000 7fc345288640 1 -- 172.21.15.38:0/2186583093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc340068e80 con 0x7fc3400cdb70 2024-04-25T08:42:00.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.534+0000 7fc3277fe640 1 -- 172.21.15.38:0/2186583093 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc334039020 con 0x7fc3400cdb70 2024-04-25T08:42:00.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.534+0000 7fc3277fe640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc32c06b4d0 0x7fc32c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.535+0000 7fc3277fe640 1 -- 172.21.15.38:0/2186583093 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc3340aae10 con 0x7fc3400cdb70 2024-04-25T08:42:00.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.535+0000 7fc33effd640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc32c06b4d0 0x7fc32c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.535+0000 7fc33effd640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc32c06b4d0 0x7fc32c06d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc330007fd0 tx=0x7fc330007e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:00.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.539+0000 7fc3277fe640 1 -- 172.21.15.38:0/2186583093 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc33402f070 con 0x7fc3400cdb70 2024-04-25T08:42:00.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.723+0000 7fc345288640 1 -- 172.21.15.38:0/2186583093 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc340158c60 con 0x7fc3400cdb70 2024-04-25T08:42:00.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.724+0000 7fc3277fe640 1 -- 172.21.15.38:0/2186583093 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+13162 (secure 0 0 0) 0x7fc334079890 con 0x7fc3400cdb70 2024-04-25T08:42:00.717 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:00.717 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":139,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:41:59.991703+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:00.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.726+0000 7fc345288640 1 -- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc32c06b4d0 msgr2=0x7fc32c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.726+0000 7fc345288640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc32c06b4d0 0x7fc32c06d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc330007fd0 tx=0x7fc330007e80 comp rx=0 tx=0).stop 2024-04-25T08:42:00.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.726+0000 7fc345288640 1 -- 172.21.15.38:0/2186583093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3400cdb70 msgr2=0x7fc340142df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.726+0000 7fc345288640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3400cdb70 0x7fc340142df0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fc33402c960 tx=0x7fc3340046c0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.726+0000 7fc345288640 1 -- 172.21.15.38:0/2186583093 shutdown_connections 2024-04-25T08:42:00.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.726+0000 7fc345288640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc32c06b4d0 0x7fc32c06d990 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.726+0000 7fc345288640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc340143f40 0x7fc340143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.726+0000 7fc345288640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc3400cdb70 0x7fc340142df0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.726+0000 7fc345288640 1 --2- 172.21.15.38:0/2186583093 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3400b5030 0x7fc3400c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.726+0000 7fc345288640 1 -- 172.21.15.38:0/2186583093 >> 172.21.15.38:0/2186583093 conn(0x7fc34006b210 msgr2=0x7fc3400b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:00.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.726+0000 7fc345288640 1 -- 172.21.15.38:0/2186583093 shutdown_connections 2024-04-25T08:42:00.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.726+0000 7fc345288640 1 -- 172.21.15.38:0/2186583093 wait complete. 2024-04-25T08:42:00.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:42:00.730 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:42:00.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.842+0000 7f2d14f63640 1 Processor -- start 2024-04-25T08:42:00.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.842+0000 7f2d14f63640 1 -- start start 2024-04-25T08:42:00.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d14f63640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2d100d0320 0x7f2d100d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d14f63640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2d100b6980 0x7f2d100b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d14f63640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d100d2060 con 0x7f2d100d0700 2024-04-25T08:42:00.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d14f63640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d100d2330 con 0x7f2d100b6980 2024-04-25T08:42:00.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d14f63640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d100d2610 con 0x7f2d100d0320 2024-04-25T08:42:00.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d0ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2d100b6980 0x7f2d100b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d0ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2d100b6980 0x7f2d100b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51860/0 (socket says 172.21.15.38:51860) 2024-04-25T08:42:00.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d0ed9d640 1 -- 172.21.15.38:0/2094192505 learned_addr learned my addr 172.21.15.38:0/2094192505 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:00.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d0e59c640 1 --2- 172.21.15.38:0/2094192505 >> v2:172.21.15.107:3300/0 conn(0x7f2d100d0320 0x7f2d100d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d0ed9d640 1 -- 172.21.15.38:0/2094192505 >> v2:172.21.15.107:3300/0 conn(0x7f2d100d0320 msgr2=0x7f2d100d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:00.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d0ed9d640 1 --2- 172.21.15.38:0/2094192505 >> v2:172.21.15.107:3300/0 conn(0x7f2d100d0320 0x7f2d100d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d0ed9d640 1 -- 172.21.15.38:0/2094192505 >> v1:172.21.15.107:6789/0 conn(0x7f2d100d0700 legacy=0x7f2d100d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.843+0000 7f2d0ed9d640 1 -- 172.21.15.38:0/2094192505 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d100d28f0 con 0x7f2d100b6980 2024-04-25T08:42:00.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.844+0000 7f2d0ed9d640 1 --2- 172.21.15.38:0/2094192505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d100b6980 0x7f2d100b6dd0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f2d10069100 tx=0x7f2d0802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e74d51bf6fafe2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:00.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.844+0000 7f2d0dd9b640 1 -- 172.21.15.38:0/2094192505 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08036070 con 0x7f2d100b6980 2024-04-25T08:42:00.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.844+0000 7f2d0dd9b640 1 -- 172.21.15.38:0/2094192505 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d08031600 con 0x7f2d100b6980 2024-04-25T08:42:00.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.844+0000 7f2d0dd9b640 1 -- 172.21.15.38:0/2094192505 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d080319f0 con 0x7f2d100b6980 2024-04-25T08:42:00.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.845+0000 7f2d14f63640 1 -- 172.21.15.38:0/2094192505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d100b6980 msgr2=0x7f2d100b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.845+0000 7f2d14f63640 1 --2- 172.21.15.38:0/2094192505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d100b6980 0x7f2d100b6dd0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f2d10069100 tx=0x7f2d0802a7c0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.845+0000 7f2d14f63640 1 -- 172.21.15.38:0/2094192505 shutdown_connections 2024-04-25T08:42:00.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.845+0000 7f2d14f63640 1 --2- 172.21.15.38:0/2094192505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d100b6980 0x7f2d100b6dd0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.845+0000 7f2d14f63640 1 --2- 172.21.15.38:0/2094192505 >> v2:172.21.15.107:3300/0 conn(0x7f2d100d0320 0x7f2d100d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.845+0000 7f2d14f63640 1 -- 172.21.15.38:0/2094192505 >> 172.21.15.38:0/2094192505 conn(0x7f2d100668f0 msgr2=0x7f2d10066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:00.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.845+0000 7f2d14f63640 1 -- 172.21.15.38:0/2094192505 shutdown_connections 2024-04-25T08:42:00.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.845+0000 7f2d14f63640 1 -- 172.21.15.38:0/2094192505 wait complete. 2024-04-25T08:42:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.846+0000 7f2d14f63640 1 Processor -- start 2024-04-25T08:42:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.846+0000 7f2d14f63640 1 -- start start 2024-04-25T08:42:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d14f63640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d100b6980 0x7f2d100b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d14f63640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d100d0320 0x7f2d100c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d14f63640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d100d0700 0x7f2d101574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d14f63640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100d0b40 con 0x7f2d100d0700 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d14f63640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100d2eb0 con 0x7f2d100d0320 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d14f63640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d100b61e0 con 0x7f2d100b6980 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d0ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d100d0700 0x7f2d101574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d0ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d100d0700 0x7f2d101574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51862/0 (socket says 172.21.15.38:51862) 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d0ed9d640 1 -- 172.21.15.38:0/407402309 learned_addr learned my addr 172.21.15.38:0/407402309 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d0e59c640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d100b6980 0x7f2d100b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d07fff640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d100d0320 0x7f2d100c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d0ed9d640 1 -- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d100b6980 msgr2=0x7f2d100b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d0ed9d640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d100b6980 0x7f2d100b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d0ed9d640 1 -- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d100d0320 msgr2=0x7f2d100c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d0ed9d640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d100d0320 0x7f2d100c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d0ed9d640 1 -- 172.21.15.38:0/407402309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d10156430 con 0x7f2d100d0700 2024-04-25T08:42:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d0e59c640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d100b6980 0x7f2d100b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.847+0000 7f2d07fff640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d100d0320 0x7f2d100c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.848+0000 7f2d0ed9d640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d100d0700 0x7f2d101574d0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f2d0802a800 tx=0x7f2d08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.848+0000 7f2d06ffd640 1 -- 172.21.15.38:0/407402309 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08036050 con 0x7f2d100d0700 2024-04-25T08:42:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.848+0000 7f2d06ffd640 1 -- 172.21.15.38:0/407402309 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d08042070 con 0x7f2d100d0700 2024-04-25T08:42:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.848+0000 7f2d14f63640 1 -- 172.21.15.38:0/407402309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d10156710 con 0x7f2d100d0700 2024-04-25T08:42:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.848+0000 7f2d06ffd640 1 -- 172.21.15.38:0/407402309 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08009510 con 0x7f2d100d0700 2024-04-25T08:42:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.848+0000 7f2d14f63640 1 -- 172.21.15.38:0/407402309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d10156cd0 con 0x7f2d100d0700 2024-04-25T08:42:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.849+0000 7f2d14f63640 1 -- 172.21.15.38:0/407402309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d10161210 con 0x7f2d100d0700 2024-04-25T08:42:00.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.849+0000 7f2d06ffd640 1 -- 172.21.15.38:0/407402309 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d08004450 con 0x7f2d100d0700 2024-04-25T08:42:00.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.850+0000 7f2d06ffd640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2cf006b4d0 0x7f2cf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:00.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.850+0000 7f2d06ffd640 1 -- 172.21.15.38:0/407402309 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2d080aaf60 con 0x7f2d100d0700 2024-04-25T08:42:00.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.850+0000 7f2d0e59c640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2cf006b4d0 0x7f2cf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:00.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.850+0000 7f2d0e59c640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2cf006b4d0 0x7f2cf006d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009b90 tx=0x7f2d00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:00.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:00.854+0000 7f2d06ffd640 1 -- 172.21.15.38:0/407402309 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d10161210 con 0x7f2d100d0700 2024-04-25T08:42:01.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.036+0000 7f2d14f63640 1 -- 172.21.15.38:0/407402309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2d100cf180 con 0x7f2d100d0700 2024-04-25T08:42:01.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.037+0000 7f2d06ffd640 1 -- 172.21.15.38:0/407402309 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2d100cf180 con 0x7f2d100d0700 2024-04-25T08:42:01.029 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:42:01.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.039+0000 7f2d14f63640 1 -- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2cf006b4d0 msgr2=0x7f2cf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:01.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.039+0000 7f2d14f63640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2cf006b4d0 0x7f2cf006d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009b90 tx=0x7f2d00008040 comp rx=0 tx=0).stop 2024-04-25T08:42:01.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.039+0000 7f2d14f63640 1 -- 172.21.15.38:0/407402309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d100d0700 msgr2=0x7f2d101574d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:01.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.039+0000 7f2d14f63640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d100d0700 0x7f2d101574d0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f2d0802a800 tx=0x7f2d08004a40 comp rx=0 tx=0).stop 2024-04-25T08:42:01.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.039+0000 7f2d14f63640 1 -- 172.21.15.38:0/407402309 shutdown_connections 2024-04-25T08:42:01.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.039+0000 7f2d14f63640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2d100d0700 0x7f2d101574d0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.039+0000 7f2d14f63640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d100d0320 0x7f2d100c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.039+0000 7f2d14f63640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2cf006b4d0 0x7f2cf006d990 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.039+0000 7f2d14f63640 1 --2- 172.21.15.38:0/407402309 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2d100b6980 0x7f2d100b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.039+0000 7f2d14f63640 1 -- 172.21.15.38:0/407402309 >> 172.21.15.38:0/407402309 conn(0x7f2d100668f0 msgr2=0x7f2d100b8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:01.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.039+0000 7f2d14f63640 1 -- 172.21.15.38:0/407402309 shutdown_connections 2024-04-25T08:42:01.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.039+0000 7f2d14f63640 1 -- 172.21.15.38:0/407402309 wait complete. 2024-04-25T08:42:01.043 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:42:01.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.151+0000 7fc08c75c640 1 Processor -- start 2024-04-25T08:42:01.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.151+0000 7fc08c75c640 1 -- start start 2024-04-25T08:42:01.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.152+0000 7fc08c75c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc08409ab90 0x7fc08415f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.152+0000 7fc08c75c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc0840d0460 0x7fc08415fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.152+0000 7fc08c75c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc084160340 con 0x7fc08415bb10 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.152+0000 7fc08c75c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0840cffd0 con 0x7fc0840d0460 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.152+0000 7fc08c75c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840cf400 con 0x7fc08409ab90 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.153+0000 7fc08acf9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc0840d0460 0x7fc08415fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.153+0000 7fc08a4f8640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc08415bb10 0x7fc08409a480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36222/0 (socket says 172.21.15.38:36222) 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.153+0000 7fc08a4f8640 1 -- 172.21.15.38:0/2796788262 learned_addr learned my addr 172.21.15.38:0/2796788262 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.153+0000 7fc089cf7640 1 --2- 172.21.15.38:0/2796788262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc08409ab90 0x7fc08415f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.153+0000 7fc08acf9640 1 -- 172.21.15.38:0/2796788262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc08409ab90 msgr2=0x7fc08415f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.153+0000 7fc08acf9640 1 --2- 172.21.15.38:0/2796788262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc08409ab90 0x7fc08415f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.153+0000 7fc08acf9640 1 -- 172.21.15.38:0/2796788262 >> v1:172.21.15.38:6789/0 conn(0x7fc08415bb10 legacy=0x7fc08409a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.153+0000 7fc08acf9640 1 -- 172.21.15.38:0/2796788262 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0840cf6a0 con 0x7fc0840d0460 2024-04-25T08:42:01.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.153+0000 7fc0894f6640 1 -- 172.21.15.38:0/2796788262 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 484653417 0 0) 0x7fc080001110 con 0x7fc08415bb10 2024-04-25T08:42:01.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.153+0000 7fc089cf7640 1 --2- 172.21.15.38:0/2796788262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc08409ab90 0x7fc08415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:01.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.154+0000 7fc08acf9640 1 --2- 172.21.15.38:0/2796788262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0840d0460 0x7fc08415fa20 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009d90 tx=0x7fc07c02a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f609e2782dc9167 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:01.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.154+0000 7fc0894f6640 1 -- 172.21.15.38:0/2796788262 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c007740 con 0x7fc0840d0460 2024-04-25T08:42:01.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.154+0000 7fc0894f6640 1 -- 172.21.15.38:0/2796788262 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c0078e0 con 0x7fc0840d0460 2024-04-25T08:42:01.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.154+0000 7fc0894f6640 1 -- 172.21.15.38:0/2796788262 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c007cb0 con 0x7fc0840d0460 2024-04-25T08:42:01.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.154+0000 7fc08c75c640 1 -- 172.21.15.38:0/2796788262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0840d0460 msgr2=0x7fc08415fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:01.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.154+0000 7fc08c75c640 1 --2- 172.21.15.38:0/2796788262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0840d0460 0x7fc08415fa20 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009d90 tx=0x7fc07c02a530 comp rx=0 tx=0).stop 2024-04-25T08:42:01.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.154+0000 7fc08c75c640 1 -- 172.21.15.38:0/2796788262 shutdown_connections 2024-04-25T08:42:01.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.154+0000 7fc08c75c640 1 --2- 172.21.15.38:0/2796788262 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc0840d0460 0x7fc08415fa20 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.154+0000 7fc08c75c640 1 --2- 172.21.15.38:0/2796788262 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc08409ab90 0x7fc08415f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.154+0000 7fc08c75c640 1 -- 172.21.15.38:0/2796788262 >> 172.21.15.38:0/2796788262 conn(0x7fc0840b31f0 msgr2=0x7fc0840b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:01.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.154+0000 7fc08c75c640 1 -- 172.21.15.38:0/2796788262 shutdown_connections 2024-04-25T08:42:01.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.154+0000 7fc08c75c640 1 -- 172.21.15.38:0/2796788262 wait complete. 2024-04-25T08:42:01.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.156+0000 7fc08c75c640 1 Processor -- start 2024-04-25T08:42:01.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.156+0000 7fc08c75c640 1 -- start start 2024-04-25T08:42:01.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.156+0000 7fc08c75c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc08409ab90 0x7fc08406d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:01.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.156+0000 7fc08c75c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0840d0460 0x7fc08415bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.156+0000 7fc08c75c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc08415bb10 0x7fc08415c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.156+0000 7fc08c75c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840c5250 con 0x7fc08409ab90 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.156+0000 7fc08c75c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840cfc60 con 0x7fc08415bb10 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.156+0000 7fc08c75c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840c53d0 con 0x7fc0840d0460 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.156+0000 7fc08a4f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc08409ab90 0x7fc08406d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.156+0000 7fc08a4f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc08409ab90 0x7fc08406d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51886/0 (socket says 172.21.15.38:51886) 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.156+0000 7fc08a4f8640 1 -- 172.21.15.38:0/3286852047 learned_addr learned my addr 172.21.15.38:0/3286852047 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc089cf7640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0840d0460 0x7fc08415bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc08a4f8640 1 -- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0840d0460 msgr2=0x7fc08415bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc08acf9640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc08415bb10 0x7fc08415c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc08a4f8640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0840d0460 0x7fc08415bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc08a4f8640 1 -- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc08415bb10 msgr2=0x7fc08415c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc08a4f8640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc08415bb10 0x7fc08415c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc08a4f8640 1 -- 172.21.15.38:0/3286852047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0840c8d00 con 0x7fc08409ab90 2024-04-25T08:42:01.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc089cf7640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0840d0460 0x7fc08415bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:01.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc08acf9640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc08415bb10 0x7fc08415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:01.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc08a4f8640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc08409ab90 0x7fc08406d2b0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fc08000c430 tx=0x7fc08000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:01.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc07affd640 1 -- 172.21.15.38:0/3286852047 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0800074e0 con 0x7fc08409ab90 2024-04-25T08:42:01.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc07affd640 1 -- 172.21.15.38:0/3286852047 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0800047e0 con 0x7fc08409ab90 2024-04-25T08:42:01.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc08c75c640 1 -- 172.21.15.38:0/3286852047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0840c8fe0 con 0x7fc08409ab90 2024-04-25T08:42:01.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc07affd640 1 -- 172.21.15.38:0/3286852047 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080004bd0 con 0x7fc08409ab90 2024-04-25T08:42:01.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.157+0000 7fc08c75c640 1 -- 172.21.15.38:0/3286852047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0840c95a0 con 0x7fc08409ab90 2024-04-25T08:42:01.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.158+0000 7fc08c75c640 1 -- 172.21.15.38:0/3286852047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc084068e80 con 0x7fc08409ab90 2024-04-25T08:42:01.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.159+0000 7fc07affd640 1 -- 172.21.15.38:0/3286852047 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc080011070 con 0x7fc08409ab90 2024-04-25T08:42:01.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.159+0000 7fc07affd640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc06806b4d0 0x7fc06806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:01.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.159+0000 7fc07affd640 1 -- 172.21.15.38:0/3286852047 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc080089cb0 con 0x7fc08409ab90 2024-04-25T08:42:01.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.159+0000 7fc089cf7640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc06806b4d0 0x7fc06806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:01.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.160+0000 7fc089cf7640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc06806b4d0 0x7fc06806d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc074006fd0 tx=0x7fc074008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:01.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.164+0000 7fc07affd640 1 -- 172.21.15.38:0/3286852047 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc084160340 con 0x7fc08409ab90 2024-04-25T08:42:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.345+0000 7fc08c75c640 1 -- 172.21.15.38:0/3286852047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc0840b4020 con 0x7fc08409ab90 2024-04-25T08:42:01.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.345+0000 7fc07affd640 1 -- 172.21.15.38:0/3286852047 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v139) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc08000e280 con 0x7fc08409ab90 2024-04-25T08:42:01.338 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:01.338 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-04-25T08:42:01.338 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-04-25T08:42:01.338 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-04-25T08:42:01.338 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-04-25T08:42:01.338 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:01.338 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-04-25T08:42:01.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.347+0000 7fc08c75c640 1 -- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc06806b4d0 msgr2=0x7fc06806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:01.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.347+0000 7fc08c75c640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc06806b4d0 0x7fc06806d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc074006fd0 tx=0x7fc074008040 comp rx=0 tx=0).stop 2024-04-25T08:42:01.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.347+0000 7fc08c75c640 1 -- 172.21.15.38:0/3286852047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc08409ab90 msgr2=0x7fc08406d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:01.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.347+0000 7fc08c75c640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc08409ab90 0x7fc08406d2b0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fc08000c430 tx=0x7fc08000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.347+0000 7fc08c75c640 1 -- 172.21.15.38:0/3286852047 shutdown_connections 2024-04-25T08:42:01.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.347+0000 7fc08c75c640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc08415bb10 0x7fc08415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.347+0000 7fc08c75c640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc06806b4d0 0x7fc06806d990 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.347+0000 7fc08c75c640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc0840d0460 0x7fc08415bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.347+0000 7fc08c75c640 1 --2- 172.21.15.38:0/3286852047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc08409ab90 0x7fc08406d2b0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.347+0000 7fc08c75c640 1 -- 172.21.15.38:0/3286852047 >> 172.21.15.38:0/3286852047 conn(0x7fc0840b31f0 msgr2=0x7fc0840cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:01.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.347+0000 7fc08c75c640 1 -- 172.21.15.38:0/3286852047 shutdown_connections 2024-04-25T08:42:01.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.347+0000 7fc08c75c640 1 -- 172.21.15.38:0/3286852047 wait complete. 2024-04-25T08:42:01.351 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:42:01.351 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:42:01.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.464+0000 7fb16f334640 1 Processor -- start 2024-04-25T08:42:01.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.464+0000 7fb16f334640 1 -- start start 2024-04-25T08:42:01.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.465+0000 7fb16f334640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb1680d0870 0x7fb1680d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:01.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.465+0000 7fb16f334640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1680d11c0 con 0x7fb1680b93f0 2024-04-25T08:42:01.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.465+0000 7fb16f334640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1680b9880 con 0x7fb1681612c0 2024-04-25T08:42:01.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.465+0000 7fb16f334640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1680b7200 con 0x7fb1680d0870 2024-04-25T08:42:01.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.465+0000 7fb16d8d1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb1680d0870 0x7fb1680d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:01.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.465+0000 7fb16c8cf640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb1680b93f0 0x7fb1680d1c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36224/0 (socket says 172.21.15.38:36224) 2024-04-25T08:42:01.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.465+0000 7fb16d8d1640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fb1680d0870 0x7fb1680d0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51890/0 (socket says 172.21.15.38:51890) 2024-04-25T08:42:01.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.465+0000 7fb16c8cf640 1 -- 172.21.15.38:0/3348691955 learned_addr learned my addr 172.21.15.38:0/3348691955 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:01.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.465+0000 7fb16d8d1640 1 -- 172.21.15.38:0/3348691955 >> v1:172.21.15.107:6789/0 conn(0x7fb1681612c0 legacy=0x7fb1680b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:01.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.465+0000 7fb16d8d1640 1 -- 172.21.15.38:0/3348691955 >> v1:172.21.15.38:6789/0 conn(0x7fb1680b93f0 legacy=0x7fb1680d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:01.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.465+0000 7fb15ffff640 1 -- 172.21.15.38:0/3348691955 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766313690 0 0) 0x7fb158001110 con 0x7fb1680b93f0 2024-04-25T08:42:01.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.465+0000 7fb16d8d1640 1 -- 172.21.15.38:0/3348691955 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1680b74e0 con 0x7fb1680d0870 2024-04-25T08:42:01.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.466+0000 7fb16d8d1640 1 --2- 172.21.15.38:0/3348691955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1680d0870 0x7fb1680d0c80 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fb160009d90 tx=0x7fb16002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33b1a5ac5921e1b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:01.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.466+0000 7fb15ffff640 1 -- 172.21.15.38:0/3348691955 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160031600 con 0x7fb1680d0870 2024-04-25T08:42:01.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.466+0000 7fb15ffff640 1 -- 172.21.15.38:0/3348691955 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1600317a0 con 0x7fb1680d0870 2024-04-25T08:42:01.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.466+0000 7fb15ffff640 1 -- 172.21.15.38:0/3348691955 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160031b70 con 0x7fb1680d0870 2024-04-25T08:42:01.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.467+0000 7fb16f334640 1 -- 172.21.15.38:0/3348691955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1680d0870 msgr2=0x7fb1680d0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:01.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.467+0000 7fb16f334640 1 --2- 172.21.15.38:0/3348691955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1680d0870 0x7fb1680d0c80 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fb160009d90 tx=0x7fb16002a610 comp rx=0 tx=0).stop 2024-04-25T08:42:01.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.467+0000 7fb16f334640 1 -- 172.21.15.38:0/3348691955 shutdown_connections 2024-04-25T08:42:01.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.467+0000 7fb16f334640 1 --2- 172.21.15.38:0/3348691955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1680d0870 0x7fb1680d0c80 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.467+0000 7fb16f334640 1 -- 172.21.15.38:0/3348691955 >> 172.21.15.38:0/3348691955 conn(0x7fb168066d10 msgr2=0x7fb168067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:01.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.467+0000 7fb16f334640 1 -- 172.21.15.38:0/3348691955 shutdown_connections 2024-04-25T08:42:01.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.467+0000 7fb16f334640 1 -- 172.21.15.38:0/3348691955 wait complete. 2024-04-25T08:42:01.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.468+0000 7fb16f334640 1 Processor -- start 2024-04-25T08:42:01.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.468+0000 7fb16f334640 1 -- start start 2024-04-25T08:42:01.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16f334640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1680b93f0 0x7fb16814b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16f334640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1680d0870 0x7fb1680cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16f334640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1681612c0 0x7fb168155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16f334640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680b7d80 con 0x7fb1681612c0 2024-04-25T08:42:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16f334640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680b7aa0 con 0x7fb1680b93f0 2024-04-25T08:42:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16f334640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1680b82a0 con 0x7fb1680d0870 2024-04-25T08:42:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16d8d1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1681612c0 0x7fb168155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16d8d1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1681612c0 0x7fb168155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51898/0 (socket says 172.21.15.38:51898) 2024-04-25T08:42:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16d8d1640 1 -- 172.21.15.38:0/24086608 learned_addr learned my addr 172.21.15.38:0/24086608 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16d0d0640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1680b93f0 0x7fb16814b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16c8cf640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1680d0870 0x7fb1680cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:01.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16d8d1640 1 -- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1680d0870 msgr2=0x7fb1680cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:01.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16d8d1640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1680d0870 0x7fb1680cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16d8d1640 1 -- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1680b93f0 msgr2=0x7fb16814b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:01.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16d8d1640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1680b93f0 0x7fb16814b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:01.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16d8d1640 1 -- 172.21.15.38:0/24086608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb168156190 con 0x7fb1681612c0 2024-04-25T08:42:01.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16d8d1640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1681612c0 0x7fb168155350 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fb160009eb0 tx=0x7fb160004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:01.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb15dffb640 1 -- 172.21.15.38:0/24086608 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160040070 con 0x7fb1681612c0 2024-04-25T08:42:01.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb15dffb640 1 -- 172.21.15.38:0/24086608 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb160030b60 con 0x7fb1681612c0 2024-04-25T08:42:01.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.469+0000 7fb16f334640 1 -- 172.21.15.38:0/24086608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb168154c10 con 0x7fb1681612c0 2024-04-25T08:42:01.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.470+0000 7fb15dffb640 1 -- 172.21.15.38:0/24086608 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160045460 con 0x7fb1681612c0 2024-04-25T08:42:01.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.470+0000 7fb16f334640 1 -- 172.21.15.38:0/24086608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1680ccb80 con 0x7fb1681612c0 2024-04-25T08:42:01.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.471+0000 7fb16f334640 1 -- 172.21.15.38:0/24086608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb168068e80 con 0x7fb1681612c0 2024-04-25T08:42:01.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.471+0000 7fb15dffb640 1 -- 172.21.15.38:0/24086608 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb160039040 con 0x7fb1681612c0 2024-04-25T08:42:01.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.471+0000 7fb15dffb640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb14c06b4d0 0x7fb14c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:01.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.472+0000 7fb15dffb640 1 -- 172.21.15.38:0/24086608 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb1600ab7d0 con 0x7fb1681612c0 2024-04-25T08:42:01.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.472+0000 7fb16d0d0640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb14c06b4d0 0x7fb14c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:01.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.472+0000 7fb16d0d0640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb14c06b4d0 0x7fb14c06d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb164006fd0 tx=0x7fb164008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:01.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.476+0000 7fb15dffb640 1 -- 172.21.15.38:0/24086608 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb16007aa80 con 0x7fb1681612c0 2024-04-25T08:42:01.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:01.656+0000 7fb16f334640 1 -- 172.21.15.38:0/24086608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fb168155130 con 0x7fb1681612c0 2024-04-25T08:42:02.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.663+0000 7fb15dffb640 1 -- 172.21.15.38:0/24086608 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v140) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb16007a380 con 0x7fb1681612c0 2024-04-25T08:42:02.656 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-04-25T08:42:02.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.666+0000 7fb16f334640 1 -- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb14c06b4d0 msgr2=0x7fb14c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:02.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.666+0000 7fb16f334640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb14c06b4d0 0x7fb14c06d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fb164006fd0 tx=0x7fb164008040 comp rx=0 tx=0).stop 2024-04-25T08:42:02.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.666+0000 7fb16f334640 1 -- 172.21.15.38:0/24086608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1681612c0 msgr2=0x7fb168155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:02.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.666+0000 7fb16f334640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1681612c0 0x7fb168155350 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fb160009eb0 tx=0x7fb160004a40 comp rx=0 tx=0).stop 2024-04-25T08:42:02.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.666+0000 7fb16f334640 1 -- 172.21.15.38:0/24086608 shutdown_connections 2024-04-25T08:42:02.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.666+0000 7fb16f334640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1681612c0 0x7fb168155350 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:02.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.666+0000 7fb16f334640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb14c06b4d0 0x7fb14c06d990 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:02.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.666+0000 7fb16f334640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb1680d0870 0x7fb1680cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:02.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.666+0000 7fb16f334640 1 --2- 172.21.15.38:0/24086608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb1680b93f0 0x7fb16814b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:02.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.666+0000 7fb16f334640 1 -- 172.21.15.38:0/24086608 >> 172.21.15.38:0/24086608 conn(0x7fb168066d10 msgr2=0x7fb16815b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:02.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.667+0000 7fb16f334640 1 -- 172.21.15.38:0/24086608 shutdown_connections 2024-04-25T08:42:02.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.667+0000 7fb16f334640 1 -- 172.21.15.38:0/24086608 wait complete. 2024-04-25T08:42:02.675 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:42:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.796+0000 7fec3b05a640 1 Processor -- start 2024-04-25T08:42:02.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.796+0000 7fec3b05a640 1 -- start start 2024-04-25T08:42:02.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.797+0000 7fec3b05a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fec340d33d0 0x7fec340d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:02.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.797+0000 7fec3b05a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec340d2330 con 0x7fec340d1e90 2024-04-25T08:42:02.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.797+0000 7fec3b05a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec340d2610 con 0x7fec340d0320 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.797+0000 7fec3b05a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec340d28f0 con 0x7fec340d33d0 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.797+0000 7fec395f7640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fec340d1e90 0x7fec340b6980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36238/0 (socket says 172.21.15.38:36238) 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.797+0000 7fec395f7640 1 -- 172.21.15.38:0/2777061717 learned_addr learned my addr 172.21.15.38:0/2777061717 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.797+0000 7fec33fff640 1 --2- 172.21.15.38:0/2777061717 >> v2:172.21.15.107:3300/0 conn(0x7fec340d33d0 0x7fec340d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853377468 0 0) 0x7fec340d2330 con 0x7fec340d1e90 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec1c003520 con 0x7fec340d1e90 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703748267 0 0) 0x7fec340d2610 con 0x7fec340d0320 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec340d2330 con 0x7fec340d0320 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2364545065 0 0) 0x7fec1c003520 con 0x7fec340d1e90 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 >> v2:172.21.15.107:3300/0 conn(0x7fec340d33d0 msgr2=0x7fec340d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 --2- 172.21.15.38:0/2777061717 >> v2:172.21.15.107:3300/0 conn(0x7fec340d33d0 0x7fec340d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 >> v1:172.21.15.107:6789/0 conn(0x7fec340d0320 legacy=0x7fec340d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec340d2bd0 con 0x7fec340d1e90 2024-04-25T08:42:02.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fec2c0019f0 con 0x7fec340d1e90 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 >> v1:172.21.15.38:6789/0 conn(0x7fec340d1e90 legacy=0x7fec340b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 --2- 172.21.15.38:0/2777061717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec1c003b40 0x7fec1c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 --2- 172.21.15.38:0/2777061717 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec1c004800 0x7fec1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 --2- 172.21.15.38:0/2777061717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0051c0 0x7fec1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec340d28f0 con 0x7fec1c003b40 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c00bfb0 con 0x7fec1c0051c0 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c00c1b0 con 0x7fec1c004800 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec38df6640 1 --2- 172.21.15.38:0/2777061717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec1c003b40 0x7fec1c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec395f7640 1 --2- 172.21.15.38:0/2777061717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0051c0 0x7fec1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec33fff640 1 --2- 172.21.15.38:0/2777061717 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec1c004800 0x7fec1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec38df6640 1 -- 172.21.15.38:0/2777061717 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec1c004800 msgr2=0x7fec1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec38df6640 1 --2- 172.21.15.38:0/2777061717 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec1c004800 0x7fec1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:02.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec38df6640 1 -- 172.21.15.38:0/2777061717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0051c0 msgr2=0x7fec1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec38df6640 1 --2- 172.21.15.38:0/2777061717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0051c0 0x7fec1c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec38df6640 1 -- 172.21.15.38:0/2777061717 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c003520 con 0x7fec1c003b40 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec395f7640 1 --2- 172.21.15.38:0/2777061717 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c0051c0 0x7fec1c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.798+0000 7fec395f7640 1 -- 172.21.15.38:0/2777061717 reap_dead start 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.799+0000 7fec38df6640 1 --2- 172.21.15.38:0/2777061717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec1c003b40 0x7fec1c003f40 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fec240018a0 tx=0x7fec1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82526537d5d3cd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.799+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec240014f0 con 0x7fec1c003b40 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.799+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec240052d0 con 0x7fec1c003b40 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.799+0000 7fec337fe640 1 -- 172.21.15.38:0/2777061717 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2402ac40 con 0x7fec1c003b40 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.800+0000 7fec3b05a640 1 -- 172.21.15.38:0/2777061717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec1c003b40 msgr2=0x7fec1c003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.800+0000 7fec3b05a640 1 --2- 172.21.15.38:0/2777061717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec1c003b40 0x7fec1c003f40 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7fec240018a0 tx=0x7fec1c005830 comp rx=0 tx=0).stop 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.800+0000 7fec3b05a640 1 -- 172.21.15.38:0/2777061717 shutdown_connections 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.800+0000 7fec3b05a640 1 --2- 172.21.15.38:0/2777061717 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec1c003b40 0x7fec1c003f40 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.800+0000 7fec3b05a640 1 -- 172.21.15.38:0/2777061717 >> 172.21.15.38:0/2777061717 conn(0x7fec340668f0 msgr2=0x7fec34066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:02.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.800+0000 7fec3b05a640 1 -- 172.21.15.38:0/2777061717 shutdown_connections 2024-04-25T08:42:02.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.800+0000 7fec3b05a640 1 -- 172.21.15.38:0/2777061717 wait complete. 2024-04-25T08:42:02.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.801+0000 7fec3b05a640 1 Processor -- start 2024-04-25T08:42:02.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.801+0000 7fec3b05a640 1 -- start start 2024-04-25T08:42:02.798 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec3b05a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c003b40 0x7fec340b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec3b05a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec341f8060 0x7fec341e8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec3b05a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec341e8f90 0x7fec341f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec3b05a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec34161390 con 0x7fec341f8060 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec3b05a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec34161210 con 0x7fec1c003b40 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec3b05a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec340b9720 con 0x7fec341e8f90 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec33fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec341f8060 0x7fec341e8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec33fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec341f8060 0x7fec341e8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51922/0 (socket says 172.21.15.38:51922) 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec33fff640 1 -- 172.21.15.38:0/2448217105 learned_addr learned my addr 172.21.15.38:0/2448217105 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec395f7640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec341e8f90 0x7fec341f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec38df6640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c003b40 0x7fec340b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec33fff640 1 -- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec341e8f90 msgr2=0x7fec341f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec33fff640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec341e8f90 0x7fec341f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec33fff640 1 -- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c003b40 msgr2=0x7fec340b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec33fff640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c003b40 0x7fec340b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:02.799 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec33fff640 1 -- 172.21.15.38:0/2448217105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec341f5220 con 0x7fec341f8060 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.802+0000 7fec38df6640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c003b40 0x7fec340b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.803+0000 7fec33fff640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec341f8060 0x7fec341e8a50 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fec280116d0 tx=0x7fec28011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.803+0000 7fec317fa640 1 -- 172.21.15.38:0/2448217105 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28015870 con 0x7fec341f8060 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.803+0000 7fec317fa640 1 -- 172.21.15.38:0/2448217105 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec28015a10 con 0x7fec341f8060 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.803+0000 7fec3b05a640 1 -- 172.21.15.38:0/2448217105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec341f84e0 con 0x7fec341f8060 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.803+0000 7fec317fa640 1 -- 172.21.15.38:0/2448217105 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec28015dc0 con 0x7fec341f8060 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.803+0000 7fec3b05a640 1 -- 172.21.15.38:0/2448217105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec340b48c0 con 0x7fec341f8060 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.804+0000 7fec3b05a640 1 -- 172.21.15.38:0/2448217105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec34068e80 con 0x7fec341f8060 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.804+0000 7fec317fa640 1 -- 172.21.15.38:0/2448217105 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec2800f460 con 0x7fec341f8060 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.805+0000 7fec317fa640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec2006b500 0x7fec2006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.805+0000 7fec317fa640 1 -- 172.21.15.38:0/2448217105 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fec2808aab0 con 0x7fec341f8060 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.805+0000 7fec38df6640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec2006b500 0x7fec2006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:02.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.806+0000 7fec38df6640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec2006b500 0x7fec2006d9c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fec240018a0 tx=0x7fec24009030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:02.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.810+0000 7fec317fa640 1 -- 172.21.15.38:0/2448217105 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec28019030 con 0x7fec341f8060 2024-04-25T08:42:02.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:02.991+0000 7fec3b05a640 1 -- 172.21.15.38:0/2448217105 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fec340b3050 con 0x7fec341f8060 2024-04-25T08:42:03.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.669+0000 7fec317fa640 1 -- 172.21.15.38:0/2448217105 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v141) v1 ==== 170+0+0 (secure 0 0 0) 0x7fec28059990 con 0x7fec341f8060 2024-04-25T08:42:03.662 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-04-25T08:42:03.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.672+0000 7fec3b05a640 1 -- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec2006b500 msgr2=0x7fec2006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:03.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.672+0000 7fec3b05a640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec2006b500 0x7fec2006d9c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fec240018a0 tx=0x7fec24009030 comp rx=0 tx=0).stop 2024-04-25T08:42:03.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.672+0000 7fec3b05a640 1 -- 172.21.15.38:0/2448217105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec341f8060 msgr2=0x7fec341e8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:03.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.672+0000 7fec3b05a640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec341f8060 0x7fec341e8a50 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fec280116d0 tx=0x7fec28011ad0 comp rx=0 tx=0).stop 2024-04-25T08:42:03.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.672+0000 7fec3b05a640 1 -- 172.21.15.38:0/2448217105 shutdown_connections 2024-04-25T08:42:03.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.672+0000 7fec3b05a640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fec2006b500 0x7fec2006d9c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:03.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.672+0000 7fec3b05a640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fec341e8f90 0x7fec341f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:03.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.672+0000 7fec3b05a640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec341f8060 0x7fec341e8a50 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:03.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.672+0000 7fec3b05a640 1 --2- 172.21.15.38:0/2448217105 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec1c003b40 0x7fec340b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:03.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.672+0000 7fec3b05a640 1 -- 172.21.15.38:0/2448217105 >> 172.21.15.38:0/2448217105 conn(0x7fec340668f0 msgr2=0x7fec3415e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:03.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.672+0000 7fec3b05a640 1 -- 172.21.15.38:0/2448217105 shutdown_connections 2024-04-25T08:42:03.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.672+0000 7fec3b05a640 1 -- 172.21.15.38:0/2448217105 wait complete. 2024-04-25T08:42:03.692 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:42:03.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.809+0000 7f30bd37e640 1 Processor -- start 2024-04-25T08:42:03.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.809+0000 7f30bd37e640 1 -- start start 2024-04-25T08:42:03.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.809+0000 7f30bd37e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30b815baf0 0x7f30b80cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:03.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.809+0000 7f30bd37e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f30b80cfb70 0x7f30b8071f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:03.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.809+0000 7f30bd37e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b80b3250 con 0x7f30b8072490 2024-04-25T08:42:03.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.809+0000 7f30bd37e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b80731d0 con 0x7f30b80cfb70 2024-04-25T08:42:03.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.809+0000 7f30bd37e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8072c10 con 0x7f30b815baf0 2024-04-25T08:42:03.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b77fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f30b8072490 0x7f30b80cbdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36244/0 (socket says 172.21.15.38:36244) 2024-04-25T08:42:03.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b77fe640 1 -- 172.21.15.38:0/3424190350 learned_addr learned my addr 172.21.15.38:0/3424190350 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:03.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594453901 0 0) 0x7f30b0001490 con 0x7f30b8072490 2024-04-25T08:42:03.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b67fc640 1 --2- 172.21.15.38:0/3424190350 >> v2:172.21.15.107:3300/0 conn(0x7f30b80cfb70 0x7f30b8071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:03.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b6ffd640 1 --2- 172.21.15.38:0/3424190350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30b815baf0 0x7f30b80cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:03.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30a4003520 con 0x7f30b8072490 2024-04-25T08:42:03.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3934233005 0 0) 0x7f30a4003520 con 0x7f30b8072490 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30b815baf0 msgr2=0x7f30b80cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 --2- 172.21.15.38:0/3424190350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30b815baf0 0x7f30b80cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 >> v2:172.21.15.107:3300/0 conn(0x7f30b80cfb70 msgr2=0x7f30b8071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 --2- 172.21.15.38:0/3424190350 >> v2:172.21.15.107:3300/0 conn(0x7f30b80cfb70 0x7f30b8071f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b8072ef0 con 0x7f30b8072490 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f30b0001bc0 con 0x7f30b8072490 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 >> v1:172.21.15.38:6789/0 conn(0x7f30b8072490 legacy=0x7f30b80cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 --2- 172.21.15.38:0/3424190350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40039b0 0x7f30a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 --2- 172.21.15.38:0/3424190350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4004800 0x7f30a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 --2- 172.21.15.38:0/3424190350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30b815baf0 0x7f30a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80731d0 con 0x7f30a40039b0 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8072c10 con 0x7f30a4004800 2024-04-25T08:42:03.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a400ba40 con 0x7f30b815baf0 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.810+0000 7f30b6ffd640 1 --2- 172.21.15.38:0/3424190350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40039b0 0x7f30a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.811+0000 7f30b67fc640 1 --2- 172.21.15.38:0/3424190350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4004800 0x7f30a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.811+0000 7f30b77fe640 1 --2- 172.21.15.38:0/3424190350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30b815baf0 0x7f30a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.811+0000 7f30b6ffd640 1 -- 172.21.15.38:0/3424190350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30b815baf0 msgr2=0x7f30a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.811+0000 7f30b6ffd640 1 --2- 172.21.15.38:0/3424190350 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30b815baf0 0x7f30a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.811+0000 7f30b6ffd640 1 -- 172.21.15.38:0/3424190350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4004800 msgr2=0x7f30a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.811+0000 7f30b6ffd640 1 --2- 172.21.15.38:0/3424190350 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4004800 0x7f30a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.811+0000 7f30b6ffd640 1 -- 172.21.15.38:0/3424190350 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a4003520 con 0x7f30a40039b0 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.811+0000 7f30b6ffd640 1 --2- 172.21.15.38:0/3424190350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40039b0 0x7f30a4003dd0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f30a0009ac0 tx=0x7f30a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1e0bed9eca48342 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.812+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0036070 con 0x7f30a40039b0 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.812+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a0031400 con 0x7f30a40039b0 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.812+0000 7f30b5ffb640 1 -- 172.21.15.38:0/3424190350 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0009690 con 0x7f30a40039b0 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.812+0000 7f30bd37e640 1 -- 172.21.15.38:0/3424190350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40039b0 msgr2=0x7f30a4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.812+0000 7f30bd37e640 1 --2- 172.21.15.38:0/3424190350 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40039b0 0x7f30a4003dd0 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f30a0009ac0 tx=0x7f30a4005830 comp rx=0 tx=0).stop 2024-04-25T08:42:03.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.812+0000 7f30b77fe640 1 -- 172.21.15.38:0/3424190350 reap_dead start 2024-04-25T08:42:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.812+0000 7f30bd37e640 1 -- 172.21.15.38:0/3424190350 shutdown_connections 2024-04-25T08:42:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.812+0000 7f30bd37e640 1 -- 172.21.15.38:0/3424190350 >> 172.21.15.38:0/3424190350 conn(0x7f30b806adc0 msgr2=0x7f30b806b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.812+0000 7f30bd37e640 1 -- 172.21.15.38:0/3424190350 shutdown_connections 2024-04-25T08:42:03.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.812+0000 7f30bd37e640 1 -- 172.21.15.38:0/3424190350 wait complete. 2024-04-25T08:42:03.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.814+0000 7f30bd37e640 1 Processor -- start 2024-04-25T08:42:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.814+0000 7f30bd37e640 1 -- start start 2024-04-25T08:42:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.814+0000 7f30bd37e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40039b0 0x7f30b81f6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.814+0000 7f30bd37e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30b81f3670 0x7f30b81f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.814+0000 7f30bd37e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b80c8450 0x7f30b80c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.814+0000 7f30bd37e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80adb20 con 0x7f30a40039b0 2024-04-25T08:42:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.814+0000 7f30bd37e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8072920 con 0x7f30b80c8450 2024-04-25T08:42:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.814+0000 7f30bd37e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80adca0 con 0x7f30b81f3670 2024-04-25T08:42:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.814+0000 7f30b6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40039b0 0x7f30b81f6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.814+0000 7f30b6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40039b0 0x7f30b81f6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51940/0 (socket says 172.21.15.38:51940) 2024-04-25T08:42:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.814+0000 7f30b6ffd640 1 -- 172.21.15.38:0/767113481 learned_addr learned my addr 172.21.15.38:0/767113481 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:03.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30b77fe640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b80c8450 0x7f30b80c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30b6ffd640 1 -- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30b81f3670 msgr2=0x7f30b81f3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30b6ffd640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30b81f3670 0x7f30b81f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30b6ffd640 1 -- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b80c8450 msgr2=0x7f30b80c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30b6ffd640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b80c8450 0x7f30b80c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30b6ffd640 1 -- 172.21.15.38:0/767113481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b80c8e10 con 0x7f30a40039b0 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30b77fe640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b80c8450 0x7f30b80c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30b6ffd640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40039b0 0x7f30b81f6360 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f30a0034040 tx=0x7f30a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30af7fe640 1 -- 172.21.15.38:0/767113481 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0040020 con 0x7f30a40039b0 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30af7fe640 1 -- 172.21.15.38:0/767113481 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a0003940 con 0x7f30a40039b0 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30af7fe640 1 -- 172.21.15.38:0/767113481 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0036040 con 0x7f30a40039b0 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30bd37e640 1 -- 172.21.15.38:0/767113481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b80c7690 con 0x7f30a40039b0 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.815+0000 7f30bd37e640 1 -- 172.21.15.38:0/767113481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b80c7c50 con 0x7f30a40039b0 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.816+0000 7f30bd37e640 1 -- 172.21.15.38:0/767113481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b8068e80 con 0x7f30a40039b0 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.817+0000 7f30af7fe640 1 -- 172.21.15.38:0/767113481 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30a0043020 con 0x7f30a40039b0 2024-04-25T08:42:03.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.817+0000 7f30af7fe640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f309806b4d0 0x7f309806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.817+0000 7f30af7fe640 1 -- 172.21.15.38:0/767113481 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f30a00aa8f0 con 0x7f30a40039b0 2024-04-25T08:42:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.818+0000 7f30b67fc640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f309806b4d0 0x7f309806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:03.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.818+0000 7f30b67fc640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f309806b4d0 0x7f309806d990 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f30a8006b10 tx=0x7f30a8006aa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:03.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:03.822+0000 7f30af7fe640 1 -- 172.21.15.38:0/767113481 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30a004a030 con 0x7f30a40039b0 2024-04-25T08:42:04.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.043+0000 7f30bd37e640 1 -- 172.21.15.38:0/767113481 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f30b80ae5a0 con 0x7f30a40039b0 2024-04-25T08:42:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.671+0000 7f30af7fe640 1 -- 172.21.15.38:0/767113481 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v86) v1 ==== 174+0+0 (secure 0 0 0) 0x7f30a00795b0 con 0x7f30a40039b0 2024-04-25T08:42:04.663 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 21 and data pool 22 2024-04-25T08:42:04.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.674+0000 7f30bd37e640 1 -- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f309806b4d0 msgr2=0x7f309806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:04.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.674+0000 7f30bd37e640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f309806b4d0 0x7f309806d990 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f30a8006b10 tx=0x7f30a8006aa0 comp rx=0 tx=0).stop 2024-04-25T08:42:04.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.674+0000 7f30bd37e640 1 -- 172.21.15.38:0/767113481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40039b0 msgr2=0x7f30b81f6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:04.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.674+0000 7f30bd37e640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40039b0 0x7f30b81f6360 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f30a0034040 tx=0x7f30a0004730 comp rx=0 tx=0).stop 2024-04-25T08:42:04.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.674+0000 7f30bd37e640 1 -- 172.21.15.38:0/767113481 shutdown_connections 2024-04-25T08:42:04.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.674+0000 7f30bd37e640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30b80c8450 0x7f30b80c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:04.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.674+0000 7f30bd37e640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f309806b4d0 0x7f309806d990 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:04.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.674+0000 7f30bd37e640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f30b81f3670 0x7f30b81f3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:04.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.674+0000 7f30bd37e640 1 --2- 172.21.15.38:0/767113481 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a40039b0 0x7f30b81f6360 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:04.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.674+0000 7f30bd37e640 1 -- 172.21.15.38:0/767113481 >> 172.21.15.38:0/767113481 conn(0x7f30b806adc0 msgr2=0x7f30b80b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:04.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.674+0000 7f30bd37e640 1 -- 172.21.15.38:0/767113481 shutdown_connections 2024-04-25T08:42:04.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.674+0000 7f30bd37e640 1 -- 172.21.15.38:0/767113481 wait complete. 2024-04-25T08:42:04.697 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:04.766 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:42:04.769 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:42:04.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.810+0000 7ff114891640 1 Processor -- start 2024-04-25T08:42:04.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.810+0000 7ff114891640 1 -- start start 2024-04-25T08:42:04.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.811+0000 7ff114891640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1100b9600 0x7ff1100d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:04.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.811+0000 7ff114891640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff1100d08f0 0x7ff1100d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:04.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.811+0000 7ff114891640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1100d22d0 con 0x7ff11015dbc0 2024-04-25T08:42:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.811+0000 7ff114891640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1100d2ee0 con 0x7ff1100d08f0 2024-04-25T08:42:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.811+0000 7ff114891640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100d25b0 con 0x7ff1100b9600 2024-04-25T08:42:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.812+0000 7ff10ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff1100d08f0 0x7ff1100d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.812+0000 7ff10ed9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff1100d08f0 0x7ff1100d0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51952/0 (socket says 172.21.15.38:51952) 2024-04-25T08:42:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.812+0000 7ff10ed9d640 1 -- 172.21.15.38:0/2293814566 learned_addr learned my addr 172.21.15.38:0/2293814566 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.812+0000 7ff10dd9b640 1 --2- 172.21.15.38:0/2293814566 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1100b9600 0x7ff1100d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.812+0000 7ff10ed9d640 1 -- 172.21.15.38:0/2293814566 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1100b9600 msgr2=0x7ff1100d3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.812+0000 7ff10ed9d640 1 --2- 172.21.15.38:0/2293814566 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1100b9600 0x7ff1100d3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.812+0000 7ff10ed9d640 1 -- 172.21.15.38:0/2293814566 >> v1:172.21.15.107:6789/0 conn(0x7ff11015dbc0 legacy=0x7ff1100d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:04.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.812+0000 7ff10ed9d640 1 -- 172.21.15.38:0/2293814566 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1100d2890 con 0x7ff1100d08f0 2024-04-25T08:42:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.813+0000 7ff10ed9d640 1 --2- 172.21.15.38:0/2293814566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1100d08f0 0x7ff1100d0d80 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7ff108009e50 tx=0x7ff10802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd4b1fa6a8ffa65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.813+0000 7ff10d59a640 1 -- 172.21.15.38:0/2293814566 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff108036070 con 0x7ff1100d08f0 2024-04-25T08:42:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.813+0000 7ff10d59a640 1 -- 172.21.15.38:0/2293814566 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff108031600 con 0x7ff1100d08f0 2024-04-25T08:42:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.813+0000 7ff10d59a640 1 -- 172.21.15.38:0/2293814566 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1080319d0 con 0x7ff1100d08f0 2024-04-25T08:42:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.813+0000 7ff114891640 1 -- 172.21.15.38:0/2293814566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1100d08f0 msgr2=0x7ff1100d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:04.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.813+0000 7ff114891640 1 --2- 172.21.15.38:0/2293814566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1100d08f0 0x7ff1100d0d80 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7ff108009e50 tx=0x7ff10802a710 comp rx=0 tx=0).stop 2024-04-25T08:42:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.813+0000 7ff114891640 1 -- 172.21.15.38:0/2293814566 shutdown_connections 2024-04-25T08:42:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.813+0000 7ff114891640 1 --2- 172.21.15.38:0/2293814566 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1100d08f0 0x7ff1100d0d80 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.813+0000 7ff114891640 1 --2- 172.21.15.38:0/2293814566 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1100b9600 0x7ff1100d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.813+0000 7ff114891640 1 -- 172.21.15.38:0/2293814566 >> 172.21.15.38:0/2293814566 conn(0x7ff11006c4c0 msgr2=0x7ff11006c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.813+0000 7ff114891640 1 -- 172.21.15.38:0/2293814566 shutdown_connections 2024-04-25T08:42:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.813+0000 7ff114891640 1 -- 172.21.15.38:0/2293814566 wait complete. 2024-04-25T08:42:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.815+0000 7ff114891640 1 Processor -- start 2024-04-25T08:42:04.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.815+0000 7ff114891640 1 -- start start 2024-04-25T08:42:04.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.815+0000 7ff114891640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1100b9600 0x7ff110160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:04.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff114891640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1100d08f0 0x7ff11014ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:04.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff114891640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff11015dbc0 0x7ff11014b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:04.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff114891640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100b6c60 con 0x7ff1100b9600 2024-04-25T08:42:04.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff114891640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100b6ae0 con 0x7ff11015dbc0 2024-04-25T08:42:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff114891640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100b6de0 con 0x7ff1100d08f0 2024-04-25T08:42:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff10e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1100b9600 0x7ff110160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff10e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1100b9600 0x7ff110160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51956/0 (socket says 172.21.15.38:51956) 2024-04-25T08:42:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff10e59c640 1 -- 172.21.15.38:0/1475062863 learned_addr learned my addr 172.21.15.38:0/1475062863 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff10dd9b640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1100d08f0 0x7ff11014ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff10ed9d640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff11015dbc0 0x7ff11014b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff10e59c640 1 -- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1100d08f0 msgr2=0x7ff11014ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff10e59c640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1100d08f0 0x7ff11014ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff10e59c640 1 -- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff11015dbc0 msgr2=0x7ff11014b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:04.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff10e59c640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff11015dbc0 0x7ff11014b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff10e59c640 1 -- 172.21.15.38:0/1475062863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1100cbea0 con 0x7ff1100b9600 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff10e59c640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1100b9600 0x7ff110160a80 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7ff10000c3c0 tx=0x7ff10000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff0feffd640 1 -- 172.21.15.38:0/1475062863 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1000052d0 con 0x7ff1100b9600 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff0feffd640 1 -- 172.21.15.38:0/1475062863 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff100005490 con 0x7ff1100b9600 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff0feffd640 1 -- 172.21.15.38:0/1475062863 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100007260 con 0x7ff1100b9600 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff114891640 1 -- 172.21.15.38:0/1475062863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1100cc180 con 0x7ff1100b9600 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.816+0000 7ff114891640 1 -- 172.21.15.38:0/1475062863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1100cc740 con 0x7ff1100b9600 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.817+0000 7ff114891640 1 -- 172.21.15.38:0/1475062863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1100ca310 con 0x7ff1100b9600 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.818+0000 7ff0feffd640 1 -- 172.21.15.38:0/1475062863 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff100007400 con 0x7ff1100b9600 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.818+0000 7ff0feffd640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff0f006b500 0x7ff0f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.818+0000 7ff0feffd640 1 -- 172.21.15.38:0/1475062863 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff100089770 con 0x7ff1100b9600 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.818+0000 7ff10dd9b640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff0f006b500 0x7ff0f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:04.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.819+0000 7ff10dd9b640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff0f006b500 0x7ff0f006d9c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff11014e600 tx=0x7ff104009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:04.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:04.823+0000 7ff0feffd640 1 -- 172.21.15.38:0/1475062863 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff100057e10 con 0x7ff1100b9600 2024-04-25T08:42:04.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.004+0000 7ff114891640 1 -- 172.21.15.38:0/1475062863 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff11015ce80 con 0x7ff1100b9600 2024-04-25T08:42:04.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.006+0000 7ff0feffd640 1 -- 172.21.15.38:0/1475062863 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+16893 (secure 0 0 0) 0x7ff10005beb0 con 0x7ff1100b9600 2024-04-25T08:42:04.998 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:04.998 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":142,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:42:04.044599+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:01.691830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:03.024111+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:05.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.007+0000 7ff114891640 1 -- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff0f006b500 msgr2=0x7ff0f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.007+0000 7ff114891640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff0f006b500 0x7ff0f006d9c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff11014e600 tx=0x7ff104009190 comp rx=0 tx=0).stop 2024-04-25T08:42:05.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.007+0000 7ff114891640 1 -- 172.21.15.38:0/1475062863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1100b9600 msgr2=0x7ff110160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.007+0000 7ff114891640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1100b9600 0x7ff110160a80 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7ff10000c3c0 tx=0x7ff10000c880 comp rx=0 tx=0).stop 2024-04-25T08:42:05.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.008+0000 7ff114891640 1 -- 172.21.15.38:0/1475062863 shutdown_connections 2024-04-25T08:42:05.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.008+0000 7ff114891640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff11015dbc0 0x7ff11014b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.008+0000 7ff114891640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff0f006b500 0x7ff0f006d9c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.008+0000 7ff114891640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff1100d08f0 0x7ff11014ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.008+0000 7ff114891640 1 --2- 172.21.15.38:0/1475062863 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff1100b9600 0x7ff110160a80 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.008+0000 7ff114891640 1 -- 172.21.15.38:0/1475062863 >> 172.21.15.38:0/1475062863 conn(0x7ff11006c4c0 msgr2=0x7ff1100b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:05.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.008+0000 7ff114891640 1 -- 172.21.15.38:0/1475062863 shutdown_connections 2024-04-25T08:42:05.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.008+0000 7ff114891640 1 -- 172.21.15.38:0/1475062863 wait complete. 2024-04-25T08:42:05.012 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.120+0000 7ff53101f640 1 Processor -- start 2024-04-25T08:42:05.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.120+0000 7ff53101f640 1 -- start start 2024-04-25T08:42:05.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.121+0000 7ff53101f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff52c161700 0x7ff52c0b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.121+0000 7ff53101f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c161e00 con 0x7ff52c15ddf0 2024-04-25T08:42:05.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.121+0000 7ff53101f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff52c1620e0 con 0x7ff52c0b5710 2024-04-25T08:42:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.121+0000 7ff53101f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52c0b9930 con 0x7ff52c161700 2024-04-25T08:42:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.121+0000 7ff52b59e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff52c161700 0x7ff52c0b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.121+0000 7ff52b59e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7ff52c161700 0x7ff52c0b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51962/0 (socket says 172.21.15.38:51962) 2024-04-25T08:42:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.121+0000 7ff523fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7ff52c15ddf0 0x7ff52c0d2340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36256/0 (socket says 172.21.15.38:36256) 2024-04-25T08:42:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.121+0000 7ff52b59e640 1 -- 172.21.15.38:0/528646118 learned_addr learned my addr 172.21.15.38:0/528646118 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.121+0000 7ff52b59e640 1 -- 172.21.15.38:0/528646118 >> v1:172.21.15.107:6789/0 conn(0x7ff52c0b5710 legacy=0x7ff52c0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.121+0000 7ff52b59e640 1 -- 172.21.15.38:0/528646118 >> v1:172.21.15.38:6789/0 conn(0x7ff52c15ddf0 legacy=0x7ff52c0d2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.121+0000 7ff52b59e640 1 -- 172.21.15.38:0/528646118 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c1623c0 con 0x7ff52c161700 2024-04-25T08:42:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.122+0000 7ff52b59e640 1 --2- 172.21.15.38:0/528646118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c161700 0x7ff52c0b93f0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7ff524009e50 tx=0x7ff52402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524657c54740ca95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.122+0000 7ff52a59c640 1 -- 172.21.15.38:0/528646118 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524036070 con 0x7ff52c161700 2024-04-25T08:42:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.122+0000 7ff52a59c640 1 -- 172.21.15.38:0/528646118 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff524031600 con 0x7ff52c161700 2024-04-25T08:42:05.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.122+0000 7ff52a59c640 1 -- 172.21.15.38:0/528646118 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5240319d0 con 0x7ff52c161700 2024-04-25T08:42:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.123+0000 7ff53101f640 1 -- 172.21.15.38:0/528646118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c161700 msgr2=0x7ff52c0b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.123+0000 7ff53101f640 1 --2- 172.21.15.38:0/528646118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c161700 0x7ff52c0b93f0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7ff524009e50 tx=0x7ff52402a760 comp rx=0 tx=0).stop 2024-04-25T08:42:05.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.123+0000 7ff53101f640 1 -- 172.21.15.38:0/528646118 shutdown_connections 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.123+0000 7ff53101f640 1 --2- 172.21.15.38:0/528646118 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c161700 0x7ff52c0b93f0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.123+0000 7ff53101f640 1 -- 172.21.15.38:0/528646118 >> 172.21.15.38:0/528646118 conn(0x7ff52c0d07c0 msgr2=0x7ff52c0d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.123+0000 7ff53101f640 1 -- 172.21.15.38:0/528646118 shutdown_connections 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.123+0000 7ff53101f640 1 -- 172.21.15.38:0/528646118 wait complete. 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.124+0000 7ff53101f640 1 Processor -- start 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.124+0000 7ff53101f640 1 -- start start 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff53101f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0b5710 0x7ff52c158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff53101f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c15ddf0 0x7ff52c0ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff53101f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c161700 0x7ff52c0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff53101f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0d2de0 con 0x7ff52c0b5710 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff53101f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0d2c60 con 0x7ff52c15ddf0 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff53101f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c0d2f60 con 0x7ff52c161700 2024-04-25T08:42:05.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0b5710 0x7ff52c158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0b5710 0x7ff52c158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51966/0 (socket says 172.21.15.38:51966) 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff52ad9d640 1 -- 172.21.15.38:0/1589286183 learned_addr learned my addr 172.21.15.38:0/1589286183 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff523fff640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c15ddf0 0x7ff52c0ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff52b59e640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c161700 0x7ff52c0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff52ad9d640 1 -- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c161700 msgr2=0x7ff52c0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff52ad9d640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c161700 0x7ff52c0caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff52ad9d640 1 -- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c15ddf0 msgr2=0x7ff52c0ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff52ad9d640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c15ddf0 0x7ff52c0ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff52ad9d640 1 -- 172.21.15.38:0/1589286183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c0c92d0 con 0x7ff52c0b5710 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff52ad9d640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0b5710 0x7ff52c158f10 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00c460 tx=0x7ff51c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff5237fe640 1 -- 172.21.15.38:0/1589286183 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c016020 con 0x7ff52c0b5710 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff5237fe640 1 -- 172.21.15.38:0/1589286183 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c012870 con 0x7ff52c0b5710 2024-04-25T08:42:05.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff5237fe640 1 -- 172.21.15.38:0/1589286183 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c012c60 con 0x7ff52c0b5710 2024-04-25T08:42:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.125+0000 7ff53101f640 1 -- 172.21.15.38:0/1589286183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c0c95b0 con 0x7ff52c0b5710 2024-04-25T08:42:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.126+0000 7ff53101f640 1 -- 172.21.15.38:0/1589286183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c0c9b70 con 0x7ff52c0b5710 2024-04-25T08:42:05.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.127+0000 7ff53101f640 1 -- 172.21.15.38:0/1589286183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52c068e80 con 0x7ff52c0b5710 2024-04-25T08:42:05.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.127+0000 7ff5237fe640 1 -- 172.21.15.38:0/1589286183 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff51c019020 con 0x7ff52c0b5710 2024-04-25T08:42:05.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.127+0000 7ff5237fe640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50c06b4d0 0x7ff50c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.128+0000 7ff5237fe640 1 -- 172.21.15.38:0/1589286183 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff51c08a020 con 0x7ff52c0b5710 2024-04-25T08:42:05.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.128+0000 7ff523fff640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50c06b4d0 0x7ff50c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.128+0000 7ff523fff640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50c06b4d0 0x7ff50c06d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff52c1f7660 tx=0x7ff514009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:05.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.132+0000 7ff5237fe640 1 -- 172.21.15.38:0/1589286183 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff51c020030 con 0x7ff52c0b5710 2024-04-25T08:42:05.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.312+0000 7ff53101f640 1 -- 172.21.15.38:0/1589286183 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff52c0b6af0 con 0x7ff52c0b5710 2024-04-25T08:42:05.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.313+0000 7ff5237fe640 1 -- 172.21.15.38:0/1589286183 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+16893 (secure 0 0 0) 0x7ff51c058da0 con 0x7ff52c0b5710 2024-04-25T08:42:05.306 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:05.306 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":142,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:42:04.044599+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:01.691830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:03.024111+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:05.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.315+0000 7ff53101f640 1 -- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50c06b4d0 msgr2=0x7ff50c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.315+0000 7ff53101f640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50c06b4d0 0x7ff50c06d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff52c1f7660 tx=0x7ff514009040 comp rx=0 tx=0).stop 2024-04-25T08:42:05.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.315+0000 7ff53101f640 1 -- 172.21.15.38:0/1589286183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0b5710 msgr2=0x7ff52c158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.315+0000 7ff53101f640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0b5710 0x7ff52c158f10 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00c460 tx=0x7ff51c00c920 comp rx=0 tx=0).stop 2024-04-25T08:42:05.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.315+0000 7ff53101f640 1 -- 172.21.15.38:0/1589286183 shutdown_connections 2024-04-25T08:42:05.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.315+0000 7ff53101f640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7ff50c06b4d0 0x7ff50c06d990 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.315+0000 7ff53101f640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff52c161700 0x7ff52c0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.315+0000 7ff53101f640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c15ddf0 0x7ff52c0ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.315+0000 7ff53101f640 1 --2- 172.21.15.38:0/1589286183 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff52c0b5710 0x7ff52c158f10 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.315+0000 7ff53101f640 1 -- 172.21.15.38:0/1589286183 >> 172.21.15.38:0/1589286183 conn(0x7ff52c0d07c0 msgr2=0x7ff52c0d0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:05.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.315+0000 7ff53101f640 1 -- 172.21.15.38:0/1589286183 shutdown_connections 2024-04-25T08:42:05.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.315+0000 7ff53101f640 1 -- 172.21.15.38:0/1589286183 wait complete. 2024-04-25T08:42:05.319 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:42:05.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.430+0000 7fc5fc28f640 1 Processor -- start 2024-04-25T08:42:05.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.430+0000 7fc5fc28f640 1 -- start start 2024-04-25T08:42:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5fc28f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5f409a180 0x7fc5f409a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5fc28f640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5f4162390 0x7fc5f40cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5fc28f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f4162820 con 0x7fc5f415df30 2024-04-25T08:42:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5fc28f640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f415e310 con 0x7fc5f4162390 2024-04-25T08:42:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5fc28f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f409aeb0 con 0x7fc5f409a180 2024-04-25T08:42:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5fa82c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5f4162390 0x7fc5f40cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5fa02b640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc5f415df30 0x7fc5f4099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36258/0 (socket says 172.21.15.38:36258) 2024-04-25T08:42:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5fa02b640 1 -- 172.21.15.38:0/3211853775 learned_addr learned my addr 172.21.15.38:0/3211853775 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5f982a640 1 --2- 172.21.15.38:0/3211853775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5f409a180 0x7fc5f409a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5fa82c640 1 -- 172.21.15.38:0/3211853775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5f409a180 msgr2=0x7fc5f409a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5fa82c640 1 --2- 172.21.15.38:0/3211853775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5f409a180 0x7fc5f409a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5fa82c640 1 -- 172.21.15.38:0/3211853775 >> v1:172.21.15.38:6789/0 conn(0x7fc5f415df30 legacy=0x7fc5f4099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5fa82c640 1 -- 172.21.15.38:0/3211853775 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f415f300 con 0x7fc5f4162390 2024-04-25T08:42:05.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.431+0000 7fc5f982a640 1 --2- 172.21.15.38:0/3211853775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5f409a180 0x7fc5f409a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:05.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.432+0000 7fc5fa82c640 1 --2- 172.21.15.38:0/3211853775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f4162390 0x7fc5f40cf8f0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009d90 tx=0x7fc5ec02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0ac867ffac7aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.432+0000 7fc5f9029640 1 -- 172.21.15.38:0/3211853775 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec007740 con 0x7fc5f4162390 2024-04-25T08:42:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.432+0000 7fc5f9029640 1 -- 172.21.15.38:0/3211853775 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec0078e0 con 0x7fc5f4162390 2024-04-25T08:42:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.432+0000 7fc5f9029640 1 -- 172.21.15.38:0/3211853775 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec007cb0 con 0x7fc5f4162390 2024-04-25T08:42:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.433+0000 7fc5fc28f640 1 -- 172.21.15.38:0/3211853775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f4162390 msgr2=0x7fc5f40cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.433+0000 7fc5fc28f640 1 --2- 172.21.15.38:0/3211853775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f4162390 0x7fc5f40cf8f0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009d90 tx=0x7fc5ec02a660 comp rx=0 tx=0).stop 2024-04-25T08:42:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.433+0000 7fc5fc28f640 1 -- 172.21.15.38:0/3211853775 shutdown_connections 2024-04-25T08:42:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.433+0000 7fc5fc28f640 1 --2- 172.21.15.38:0/3211853775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f4162390 0x7fc5f40cf8f0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.433+0000 7fc5fc28f640 1 --2- 172.21.15.38:0/3211853775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5f409a180 0x7fc5f409a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.433+0000 7fc5fc28f640 1 -- 172.21.15.38:0/3211853775 >> 172.21.15.38:0/3211853775 conn(0x7fc5f40b3110 msgr2=0x7fc5f40b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:05.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.433+0000 7fc5fc28f640 1 -- 172.21.15.38:0/3211853775 shutdown_connections 2024-04-25T08:42:05.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.433+0000 7fc5fc28f640 1 -- 172.21.15.38:0/3211853775 wait complete. 2024-04-25T08:42:05.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.434+0000 7fc5fc28f640 1 Processor -- start 2024-04-25T08:42:05.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.434+0000 7fc5fc28f640 1 -- start start 2024-04-25T08:42:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fc28f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5f409a180 0x7fc5f40c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fc28f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5f415df30 0x7fc5f414ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fc28f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f4162390 0x7fc5f40cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fc28f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f415e820 con 0x7fc5f4162390 2024-04-25T08:42:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fc28f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f415ee70 con 0x7fc5f409a180 2024-04-25T08:42:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fc28f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f415eb00 con 0x7fc5f415df30 2024-04-25T08:42:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fa82c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f4162390 0x7fc5f40cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fa82c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f4162390 0x7fc5f40cd240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51978/0 (socket says 172.21.15.38:51978) 2024-04-25T08:42:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fa82c640 1 -- 172.21.15.38:0/4275635899 learned_addr learned my addr 172.21.15.38:0/4275635899 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5f982a640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5f415df30 0x7fc5f414ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fa02b640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5f409a180 0x7fc5f40c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fa82c640 1 -- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5f415df30 msgr2=0x7fc5f414ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fa82c640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5f415df30 0x7fc5f414ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fa82c640 1 -- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5f409a180 msgr2=0x7fc5f40c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fa82c640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5f409a180 0x7fc5f40c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fa82c640 1 -- 172.21.15.38:0/4275635899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f414ef40 con 0x7fc5f4162390 2024-04-25T08:42:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.435+0000 7fc5fa82c640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f4162390 0x7fc5f40cd240 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec02ac50 tx=0x7fc5ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.436+0000 7fc5eaffd640 1 -- 172.21.15.38:0/4275635899 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec040070 con 0x7fc5f4162390 2024-04-25T08:42:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.436+0000 7fc5eaffd640 1 -- 172.21.15.38:0/4275635899 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec031bc0 con 0x7fc5f4162390 2024-04-25T08:42:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.436+0000 7fc5fc28f640 1 -- 172.21.15.38:0/4275635899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f40b4740 con 0x7fc5f4162390 2024-04-25T08:42:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.436+0000 7fc5eaffd640 1 -- 172.21.15.38:0/4275635899 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec045460 con 0x7fc5f4162390 2024-04-25T08:42:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.436+0000 7fc5fc28f640 1 -- 172.21.15.38:0/4275635899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f40b4d00 con 0x7fc5f4162390 2024-04-25T08:42:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.437+0000 7fc5fc28f640 1 -- 172.21.15.38:0/4275635899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f4068e80 con 0x7fc5f4162390 2024-04-25T08:42:05.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.437+0000 7fc5eaffd640 1 -- 172.21.15.38:0/4275635899 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5ec039040 con 0x7fc5f4162390 2024-04-25T08:42:05.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.437+0000 7fc5eaffd640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5d806b4d0 0x7fc5d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.438+0000 7fc5eaffd640 1 -- 172.21.15.38:0/4275635899 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc5ec0abc30 con 0x7fc5f4162390 2024-04-25T08:42:05.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.438+0000 7fc5fa02b640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5d806b4d0 0x7fc5d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.438+0000 7fc5fa02b640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5d806b4d0 0x7fc5d806d990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0000f60 tx=0x7fc5f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:05.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.442+0000 7fc5eaffd640 1 -- 172.21.15.38:0/4275635899 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5ec07a9b0 con 0x7fc5f4162390 2024-04-25T08:42:05.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.661+0000 7fc5fc28f640 1 -- 172.21.15.38:0/4275635899 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc5f40b3de0 con 0x7fc5f4162390 2024-04-25T08:42:05.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.692+0000 7fc5eaffd640 1 -- 172.21.15.38:0/4275635899 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v87) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc5ec07a2b0 con 0x7fc5f4162390 2024-04-25T08:42:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.694+0000 7fc5fc28f640 1 -- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5d806b4d0 msgr2=0x7fc5d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.694+0000 7fc5fc28f640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5d806b4d0 0x7fc5d806d990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0000f60 tx=0x7fc5f0009040 comp rx=0 tx=0).stop 2024-04-25T08:42:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.694+0000 7fc5fc28f640 1 -- 172.21.15.38:0/4275635899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f4162390 msgr2=0x7fc5f40cd240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.694+0000 7fc5fc28f640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f4162390 0x7fc5f40cd240 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec02ac50 tx=0x7fc5ec004a40 comp rx=0 tx=0).stop 2024-04-25T08:42:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.694+0000 7fc5fc28f640 1 -- 172.21.15.38:0/4275635899 shutdown_connections 2024-04-25T08:42:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.694+0000 7fc5fc28f640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5f4162390 0x7fc5f40cd240 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.694+0000 7fc5fc28f640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5d806b4d0 0x7fc5d806d990 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.694+0000 7fc5fc28f640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5f415df30 0x7fc5f414ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.694+0000 7fc5fc28f640 1 --2- 172.21.15.38:0/4275635899 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5f409a180 0x7fc5f40c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.694+0000 7fc5fc28f640 1 -- 172.21.15.38:0/4275635899 >> 172.21.15.38:0/4275635899 conn(0x7fc5f40b3110 msgr2=0x7fc5f406f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.694+0000 7fc5fc28f640 1 -- 172.21.15.38:0/4275635899 shutdown_connections 2024-04-25T08:42:05.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.694+0000 7fc5fc28f640 1 -- 172.21.15.38:0/4275635899 wait complete. 2024-04-25T08:42:05.699 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:42:05.699 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:42:05.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.810+0000 7fdbd2970640 1 Processor -- start 2024-04-25T08:42:05.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.810+0000 7fdbd2970640 1 -- start start 2024-04-25T08:42:05.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbd2970640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdbcc0d0320 0x7fdbcc0d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbd2970640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbcc0b6980 0x7fdbcc0b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbd2970640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbcc0d2060 con 0x7fdbcc0d0700 2024-04-25T08:42:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbd2970640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbcc0d2330 con 0x7fdbcc0d0320 2024-04-25T08:42:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbd2970640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc0d2610 con 0x7fdbcc0b6980 2024-04-25T08:42:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbcbfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdbcc0d0320 0x7fdbcc0d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbcbfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fdbcc0d0320 0x7fdbcc0d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51990/0 (socket says 172.21.15.38:51990) 2024-04-25T08:42:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbcbfff640 1 -- 172.21.15.38:0/607984464 learned_addr learned my addr 172.21.15.38:0/607984464 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbd0f0d640 1 --2- 172.21.15.38:0/607984464 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbcc0b6980 0x7fdbcc0b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbcbfff640 1 -- 172.21.15.38:0/607984464 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbcc0b6980 msgr2=0x7fdbcc0b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbcbfff640 1 --2- 172.21.15.38:0/607984464 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbcc0b6980 0x7fdbcc0b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbcbfff640 1 -- 172.21.15.38:0/607984464 >> v1:172.21.15.107:6789/0 conn(0x7fdbcc0d0700 legacy=0x7fdbcc0d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:05.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.811+0000 7fdbcbfff640 1 -- 172.21.15.38:0/607984464 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc0d28f0 con 0x7fdbcc0d0320 2024-04-25T08:42:05.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.812+0000 7fdbcbfff640 1 --2- 172.21.15.38:0/607984464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbcc0d0320 0x7fdbcc0d33d0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009870 tx=0x7fdbbc02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c743b771c0757831 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:05.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.812+0000 7fdbcaffd640 1 -- 172.21.15.38:0/607984464 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc037070 con 0x7fdbcc0d0320 2024-04-25T08:42:05.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.812+0000 7fdbcaffd640 1 -- 172.21.15.38:0/607984464 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbbc032440 con 0x7fdbcc0d0320 2024-04-25T08:42:05.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.812+0000 7fdbcaffd640 1 -- 172.21.15.38:0/607984464 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbbc032810 con 0x7fdbcc0d0320 2024-04-25T08:42:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.813+0000 7fdbd2970640 1 -- 172.21.15.38:0/607984464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbcc0d0320 msgr2=0x7fdbcc0d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.813+0000 7fdbd2970640 1 --2- 172.21.15.38:0/607984464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbcc0d0320 0x7fdbcc0d33d0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc009870 tx=0x7fdbbc02a2a0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.813+0000 7fdbd2970640 1 -- 172.21.15.38:0/607984464 shutdown_connections 2024-04-25T08:42:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.813+0000 7fdbd2970640 1 --2- 172.21.15.38:0/607984464 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbcc0b6980 0x7fdbcc0b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.813+0000 7fdbd2970640 1 --2- 172.21.15.38:0/607984464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbcc0d0320 0x7fdbcc0d33d0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.813+0000 7fdbd2970640 1 -- 172.21.15.38:0/607984464 >> 172.21.15.38:0/607984464 conn(0x7fdbcc0668f0 msgr2=0x7fdbcc066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.813+0000 7fdbd2970640 1 -- 172.21.15.38:0/607984464 shutdown_connections 2024-04-25T08:42:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.813+0000 7fdbd2970640 1 -- 172.21.15.38:0/607984464 wait complete. 2024-04-25T08:42:05.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.814+0000 7fdbd2970640 1 Processor -- start 2024-04-25T08:42:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.814+0000 7fdbd2970640 1 -- start start 2024-04-25T08:42:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbd2970640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbcc0b6980 0x7fdbcc14dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbd2970640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbcc0d0320 0x7fdbcc0b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbd2970640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbcc0d0700 0x7fdbcc0b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbd2970640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc0d0b40 con 0x7fdbcc0d0320 2024-04-25T08:42:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbd2970640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc0d2eb0 con 0x7fdbcc0b6980 2024-04-25T08:42:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbd2970640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbcc0b6200 con 0x7fdbcc0d0700 2024-04-25T08:42:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbcb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbcc0d0320 0x7fdbcc0b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbcb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbcc0d0320 0x7fdbcc0b75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51998/0 (socket says 172.21.15.38:51998) 2024-04-25T08:42:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbcb7fe640 1 -- 172.21.15.38:0/2797750068 learned_addr learned my addr 172.21.15.38:0/2797750068 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:05.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbd0f0d640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbcc0d0700 0x7fdbcc0b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbcbfff640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbcc0b6980 0x7fdbcc14dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbcb7fe640 1 -- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbcc0d0700 msgr2=0x7fdbcc0b7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbcb7fe640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbcc0d0700 0x7fdbcc0b7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbcb7fe640 1 -- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbcc0b6980 msgr2=0x7fdbcc14dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:05.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.815+0000 7fdbcb7fe640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbcc0b6980 0x7fdbcc14dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:05.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.816+0000 7fdbcb7fe640 1 -- 172.21.15.38:0/2797750068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc1e7e40 con 0x7fdbcc0d0320 2024-04-25T08:42:05.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.816+0000 7fdbcb7fe640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbcc0d0320 0x7fdbcc0b75d0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400b3e0 tx=0x7fdbb400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:05.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.816+0000 7fdbc8ff9640 1 -- 172.21.15.38:0/2797750068 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4015020 con 0x7fdbcc0d0320 2024-04-25T08:42:05.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.816+0000 7fdbc8ff9640 1 -- 172.21.15.38:0/2797750068 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb4011850 con 0x7fdbcc0d0320 2024-04-25T08:42:05.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.816+0000 7fdbd2970640 1 -- 172.21.15.38:0/2797750068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbcc1e6da0 con 0x7fdbcc0d0320 2024-04-25T08:42:05.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.816+0000 7fdbc8ff9640 1 -- 172.21.15.38:0/2797750068 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4011c40 con 0x7fdbcc0d0320 2024-04-25T08:42:05.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.816+0000 7fdbd2970640 1 -- 172.21.15.38:0/2797750068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbcc1e7360 con 0x7fdbcc0d0320 2024-04-25T08:42:05.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.817+0000 7fdbd2970640 1 -- 172.21.15.38:0/2797750068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbcc068e80 con 0x7fdbcc0d0320 2024-04-25T08:42:05.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.818+0000 7fdbc8ff9640 1 -- 172.21.15.38:0/2797750068 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdbb4018020 con 0x7fdbcc0d0320 2024-04-25T08:42:05.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.818+0000 7fdbc8ff9640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdbac06b4d0 0x7fdbac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:05.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.818+0000 7fdbc8ff9640 1 -- 172.21.15.38:0/2797750068 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdbb408a1d0 con 0x7fdbcc0d0320 2024-04-25T08:42:05.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.818+0000 7fdbcbfff640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdbac06b4d0 0x7fdbac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:05.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.819+0000 7fdbcbfff640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdbac06b4d0 0x7fdbac06d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc034000 tx=0x7fdbbc02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:05.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:05.822+0000 7fdbc8ff9640 1 -- 172.21.15.38:0/2797750068 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdbb401f020 con 0x7fdbcc0d0320 2024-04-25T08:42:06.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.044+0000 7fdbd2970640 1 -- 172.21.15.38:0/2797750068 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdbcc0b3050 con 0x7fdbcc0d0320 2024-04-25T08:42:06.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.693+0000 7fdbc8ff9640 1 -- 172.21.15.38:0/2797750068 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v88) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdbb4058870 con 0x7fdbcc0d0320 2024-04-25T08:42:06.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.696+0000 7fdbd2970640 1 -- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdbac06b4d0 msgr2=0x7fdbac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:06.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.696+0000 7fdbd2970640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdbac06b4d0 0x7fdbac06d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc034000 tx=0x7fdbbc02e000 comp rx=0 tx=0).stop 2024-04-25T08:42:06.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.696+0000 7fdbd2970640 1 -- 172.21.15.38:0/2797750068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbcc0d0320 msgr2=0x7fdbcc0b75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:06.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.696+0000 7fdbd2970640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbcc0d0320 0x7fdbcc0b75d0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400b3e0 tx=0x7fdbb400b8a0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.696+0000 7fdbd2970640 1 -- 172.21.15.38:0/2797750068 shutdown_connections 2024-04-25T08:42:06.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.696+0000 7fdbd2970640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fdbac06b4d0 0x7fdbac06d990 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.696+0000 7fdbd2970640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdbcc0d0700 0x7fdbcc0b7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.696+0000 7fdbd2970640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdbcc0d0320 0x7fdbcc0b75d0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.696+0000 7fdbd2970640 1 --2- 172.21.15.38:0/2797750068 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdbcc0b6980 0x7fdbcc14dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.696+0000 7fdbd2970640 1 -- 172.21.15.38:0/2797750068 >> 172.21.15.38:0/2797750068 conn(0x7fdbcc0668f0 msgr2=0x7fdbcc15b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:06.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.696+0000 7fdbd2970640 1 -- 172.21.15.38:0/2797750068 shutdown_connections 2024-04-25T08:42:06.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.696+0000 7fdbd2970640 1 -- 172.21.15.38:0/2797750068 wait complete. 2024-04-25T08:42:06.702 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:42:06.703 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:06.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.817+0000 7f6312c15640 1 Processor -- start 2024-04-25T08:42:06.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.817+0000 7f6312c15640 1 -- start start 2024-04-25T08:42:06.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f6312c15640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f630c0b0540 0x7f630c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f6312c15640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f630c0cfe90 0x7f630c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f6312c15640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f630c15f4a0 con 0x7f630c15f0c0 2024-04-25T08:42:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f6312c15640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f630c0cd0f0 con 0x7f630c0b0540 2024-04-25T08:42:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f6312c15640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c0cd320 con 0x7f630c0cfe90 2024-04-25T08:42:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f630bfff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f630c0b0540 0x7f630c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f63109b1640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f630c15f0c0 0x7f630c0cf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36264/0 (socket says 172.21.15.38:36264) 2024-04-25T08:42:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f63109b1640 1 -- 172.21.15.38:0/1273349868 learned_addr learned my addr 172.21.15.38:0/1273349868 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f630bfff640 1 -- 172.21.15.38:0/1273349868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f630c0cfe90 msgr2=0x7f630c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f63111b2640 1 --2- 172.21.15.38:0/1273349868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f630c0cfe90 0x7f630c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f630b7fe640 1 -- 172.21.15.38:0/1273349868 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810884354 0 0) 0x7f62fc001110 con 0x7f630c15f0c0 2024-04-25T08:42:06.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f630bfff640 1 --2- 172.21.15.38:0/1273349868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f630c0cfe90 0x7f630c0b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f630bfff640 1 -- 172.21.15.38:0/1273349868 >> v1:172.21.15.38:6789/0 conn(0x7f630c15f0c0 legacy=0x7f630c0cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.818+0000 7f630bfff640 1 -- 172.21.15.38:0/1273349868 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c0cd5c0 con 0x7f630c0b0540 2024-04-25T08:42:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.819+0000 7f63111b2640 1 --2- 172.21.15.38:0/1273349868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f630c0cfe90 0x7f630c0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.819+0000 7f630bfff640 1 --2- 172.21.15.38:0/1273349868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f630c0b0540 0x7f630c0cf950 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f63000097a0 tx=0x7f630002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee72e73b398dcec7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.819+0000 7f630b7fe640 1 -- 172.21.15.38:0/1273349868 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300037070 con 0x7f630c0b0540 2024-04-25T08:42:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.819+0000 7f630b7fe640 1 -- 172.21.15.38:0/1273349868 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6300032440 con 0x7f630c0b0540 2024-04-25T08:42:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.819+0000 7f630b7fe640 1 -- 172.21.15.38:0/1273349868 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300032810 con 0x7f630c0b0540 2024-04-25T08:42:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.820+0000 7f6312c15640 1 -- 172.21.15.38:0/1273349868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f630c0b0540 msgr2=0x7f630c0cf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.820+0000 7f6312c15640 1 --2- 172.21.15.38:0/1273349868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f630c0b0540 0x7f630c0cf950 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f63000097a0 tx=0x7f630002a180 comp rx=0 tx=0).stop 2024-04-25T08:42:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.820+0000 7f6312c15640 1 -- 172.21.15.38:0/1273349868 shutdown_connections 2024-04-25T08:42:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.820+0000 7f6312c15640 1 --2- 172.21.15.38:0/1273349868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f630c0cfe90 0x7f630c0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.814 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.820+0000 7f6312c15640 1 --2- 172.21.15.38:0/1273349868 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f630c0b0540 0x7f630c0cf950 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.820+0000 7f6312c15640 1 -- 172.21.15.38:0/1273349868 >> 172.21.15.38:0/1273349868 conn(0x7f630c0b3160 msgr2=0x7f630c0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:06.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.820+0000 7f6312c15640 1 -- 172.21.15.38:0/1273349868 shutdown_connections 2024-04-25T08:42:06.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.820+0000 7f6312c15640 1 -- 172.21.15.38:0/1273349868 wait complete. 2024-04-25T08:42:06.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.821+0000 7f6312c15640 1 Processor -- start 2024-04-25T08:42:06.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.821+0000 7f6312c15640 1 -- start start 2024-04-25T08:42:06.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.822+0000 7f6312c15640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f630c0b0540 0x7f630c14ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:06.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.822+0000 7f6312c15640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f630c0cfe90 0x7f630c09d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:06.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.822+0000 7f6312c15640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c15f0c0 0x7f630c09b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:06.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.822+0000 7f6312c15640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c09ab70 con 0x7f630c0cfe90 2024-04-25T08:42:06.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.822+0000 7f6312c15640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630c09a890 con 0x7f630c15f0c0 2024-04-25T08:42:06.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.822+0000 7f6312c15640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f630c0b0cc0 con 0x7f630c0b0540 2024-04-25T08:42:06.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.822+0000 7f630bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f630c0cfe90 0x7f630c09d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:06.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.822+0000 7f630bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f630c0cfe90 0x7f630c09d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52010/0 (socket says 172.21.15.38:52010) 2024-04-25T08:42:06.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.822+0000 7f630bfff640 1 -- 172.21.15.38:0/2374768960 learned_addr learned my addr 172.21.15.38:0/2374768960 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:06.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.822+0000 7f630bfff640 1 -- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f630c0b0540 msgr2=0x7f630c14ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:06.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.822+0000 7f63109b1640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f630c0b0540 0x7f630c14ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:06.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.823+0000 7f63111b2640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c15f0c0 0x7f630c09b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:06.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.823+0000 7f630bfff640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f630c0b0540 0x7f630c14ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.823+0000 7f630bfff640 1 -- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c15f0c0 msgr2=0x7f630c09b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.823+0000 7f630bfff640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c15f0c0 0x7f630c09b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.823+0000 7f630bfff640 1 -- 172.21.15.38:0/2374768960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f630c09dcd0 con 0x7f630c0cfe90 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.823+0000 7f63109b1640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f630c0b0540 0x7f630c14ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.823+0000 7f630bfff640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f630c0cfe90 0x7f630c09d3a0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f6300035040 tx=0x7f63000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.823+0000 7f63097fa640 1 -- 172.21.15.38:0/2374768960 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300037040 con 0x7f630c0cfe90 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.823+0000 7f63097fa640 1 -- 172.21.15.38:0/2374768960 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6300030770 con 0x7f630c0cfe90 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.823+0000 7f6312c15640 1 -- 172.21.15.38:0/2374768960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f630c09dfb0 con 0x7f630c0cfe90 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.823+0000 7f63097fa640 1 -- 172.21.15.38:0/2374768960 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300030b80 con 0x7f630c0cfe90 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.823+0000 7f6312c15640 1 -- 172.21.15.38:0/2374768960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630c09e570 con 0x7f630c0cfe90 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.824+0000 7f6312c15640 1 -- 172.21.15.38:0/2374768960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f630c068e80 con 0x7f630c0cfe90 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.824+0000 7f63097fa640 1 -- 172.21.15.38:0/2374768960 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6300044020 con 0x7f630c0cfe90 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.825+0000 7f63097fa640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f62f806b4d0 0x7f62f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:06.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.825+0000 7f63097fa640 1 -- 172.21.15.38:0/2374768960 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63000aaa00 con 0x7f630c0cfe90 2024-04-25T08:42:06.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.825+0000 7f63109b1640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f62f806b4d0 0x7f62f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:06.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.826+0000 7f63109b1640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f62f806b4d0 0x7f62f806d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0013b0 tx=0x7f62fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:06.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:06.829+0000 7f63097fa640 1 -- 172.21.15.38:0/2374768960 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6300032e20 con 0x7f630c0cfe90 2024-04-25T08:42:07.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.052+0000 7f6312c15640 1 -- 172.21.15.38:0/2374768960 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f630c0b2c90 con 0x7f630c0cfe90 2024-04-25T08:42:07.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.052+0000 7f63097fa640 1 -- 172.21.15.38:0/2374768960 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f6300079650 con 0x7f630c0cfe90 2024-04-25T08:42:07.045 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:07.045 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2682394530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2682394530},{"type":"v1","addr":"172.21.15.107:6837","nonce":2682394530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2810760276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2810760276},{"type":"v1","addr":"172.21.15.107:6835","nonce":2810760276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":10038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3318593502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3318593502},{"type":"v1","addr":"172.21.15.38:6833","nonce":3318593502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:42:04.044929+0000","modified":"2024-04-25T08:42:06.045913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10041":{"gid":10041,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/3513331661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3513331661},{"type":"v1","addr":"172.21.15.38:6835","nonce":3513331661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:42:07.046 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 88 2024-04-25T08:42:07.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.054+0000 7f6312c15640 1 -- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f62f806b4d0 msgr2=0x7f62f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.054+0000 7f6312c15640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f62f806b4d0 0x7f62f806d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0013b0 tx=0x7f62fc009040 comp rx=0 tx=0).stop 2024-04-25T08:42:07.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.054+0000 7f6312c15640 1 -- 172.21.15.38:0/2374768960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f630c0cfe90 msgr2=0x7f630c09d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.054+0000 7f6312c15640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f630c0cfe90 0x7f630c09d3a0 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f6300035040 tx=0x7f63000046c0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.055+0000 7f6312c15640 1 -- 172.21.15.38:0/2374768960 shutdown_connections 2024-04-25T08:42:07.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.055+0000 7f6312c15640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f630c15f0c0 0x7f630c09b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.055+0000 7f6312c15640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f630c0cfe90 0x7f630c09d3a0 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.055+0000 7f6312c15640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f62f806b4d0 0x7f62f806d990 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.055+0000 7f6312c15640 1 --2- 172.21.15.38:0/2374768960 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f630c0b0540 0x7f630c14ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.055+0000 7f6312c15640 1 -- 172.21.15.38:0/2374768960 >> 172.21.15.38:0/2374768960 conn(0x7f630c0b3160 msgr2=0x7f630c0cdbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:07.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.055+0000 7f6312c15640 1 -- 172.21.15.38:0/2374768960 shutdown_connections 2024-04-25T08:42:07.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.055+0000 7f6312c15640 1 -- 172.21.15.38:0/2374768960 wait complete. 2024-04-25T08:42:07.058 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:07.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.167+0000 7fbfc2bce640 1 Processor -- start 2024-04-25T08:42:07.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.167+0000 7fbfc2bce640 1 -- start start 2024-04-25T08:42:07.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc2bce640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbfbc15f7b0 0x7fbfbc0aef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc2bce640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fbfbc15fb90 0x7fbfbc0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc2bce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfbc160020 0x7fbfbc0ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc2bce640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc15be30 con 0x7fbfbc15f7b0 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc2bce640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfbc15c110 con 0x7fbfbc15fb90 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc2bce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc15c3f0 con 0x7fbfbc160020 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc096a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbfbc15f7b0 0x7fbfbc0aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc096a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fbfbc15f7b0 0x7fbfbc0aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52016/0 (socket says 172.21.15.38:52016) 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc096a640 1 -- 172.21.15.38:0/2025818730 learned_addr learned my addr 172.21.15.38:0/2025818730 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfbbfff640 1 --2- 172.21.15.38:0/2025818730 >> v2:172.21.15.107:3300/0 conn(0x7fbfbc15fb90 0x7fbfbc0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc116b640 1 --2- 172.21.15.38:0/2025818730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfbc160020 0x7fbfbc0ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc096a640 1 -- 172.21.15.38:0/2025818730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfbc160020 msgr2=0x7fbfbc0ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc096a640 1 --2- 172.21.15.38:0/2025818730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfbc160020 0x7fbfbc0ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc096a640 1 -- 172.21.15.38:0/2025818730 >> v2:172.21.15.107:3300/0 conn(0x7fbfbc15fb90 msgr2=0x7fbfbc0acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc096a640 1 --2- 172.21.15.38:0/2025818730 >> v2:172.21.15.107:3300/0 conn(0x7fbfbc15fb90 0x7fbfbc0acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.168+0000 7fbfc096a640 1 -- 172.21.15.38:0/2025818730 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc15c6d0 con 0x7fbfbc15f7b0 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.169+0000 7fbfc096a640 1 --2- 172.21.15.38:0/2025818730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc15f7b0 0x7fbfbc0aef90 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0d0a90 tx=0x7fbfb002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9064019969bc6286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.169+0000 7fbfbb7fe640 1 -- 172.21.15.38:0/2025818730 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0037070 con 0x7fbfbc15f7b0 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.169+0000 7fbfbb7fe640 1 -- 172.21.15.38:0/2025818730 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb0032440 con 0x7fbfbc15f7b0 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.169+0000 7fbfbb7fe640 1 -- 172.21.15.38:0/2025818730 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0032810 con 0x7fbfbc15f7b0 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.170+0000 7fbfc2bce640 1 -- 172.21.15.38:0/2025818730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc15f7b0 msgr2=0x7fbfbc0aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.170+0000 7fbfc2bce640 1 --2- 172.21.15.38:0/2025818730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc15f7b0 0x7fbfbc0aef90 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0d0a90 tx=0x7fbfb002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.170+0000 7fbfc2bce640 1 -- 172.21.15.38:0/2025818730 shutdown_connections 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.170+0000 7fbfc2bce640 1 --2- 172.21.15.38:0/2025818730 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfbc160020 0x7fbfbc0ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.170+0000 7fbfc2bce640 1 --2- 172.21.15.38:0/2025818730 >> v2:172.21.15.107:3300/0 conn(0x7fbfbc15fb90 0x7fbfbc0acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.170+0000 7fbfc2bce640 1 --2- 172.21.15.38:0/2025818730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc15f7b0 0x7fbfbc0aef90 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.170+0000 7fbfc2bce640 1 -- 172.21.15.38:0/2025818730 >> 172.21.15.38:0/2025818730 conn(0x7fbfbc0cfd30 msgr2=0x7fbfbc0d0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.170+0000 7fbfc2bce640 1 -- 172.21.15.38:0/2025818730 shutdown_connections 2024-04-25T08:42:07.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.170+0000 7fbfc2bce640 1 -- 172.21.15.38:0/2025818730 wait complete. 2024-04-25T08:42:07.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.171+0000 7fbfc2bce640 1 Processor -- start 2024-04-25T08:42:07.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.171+0000 7fbfc2bce640 1 -- start start 2024-04-25T08:42:07.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfc2bce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfbc15f7b0 0x7fbfbc09aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfc2bce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc15fb90 0x7fbfbc13f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfc2bce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfbc160020 0x7fbfbc13c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfc2bce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc1412b0 con 0x7fbfbc15fb90 2024-04-25T08:42:07.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfc2bce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc141130 con 0x7fbfbc15f7b0 2024-04-25T08:42:07.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfc2bce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfbc141430 con 0x7fbfbc160020 2024-04-25T08:42:07.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfbbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc15fb90 0x7fbfbc13f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfbbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc15fb90 0x7fbfbc13f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52028/0 (socket says 172.21.15.38:52028) 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfbbfff640 1 -- 172.21.15.38:0/4130828813 learned_addr learned my addr 172.21.15.38:0/4130828813 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfc116b640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfbc160020 0x7fbfbc13c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfc096a640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfbc15f7b0 0x7fbfbc09aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfbbfff640 1 -- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfbc160020 msgr2=0x7fbfbc13c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfbbfff640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfbc160020 0x7fbfbc13c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfbbfff640 1 -- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfbc15f7b0 msgr2=0x7fbfbc09aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfbbfff640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfbc15f7b0 0x7fbfbc09aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfbbfff640 1 -- 172.21.15.38:0/4130828813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0cde00 con 0x7fbfbc15fb90 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfc116b640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfbc160020 0x7fbfbc13c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfbbfff640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc15fb90 0x7fbfbc13f390 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800e9e0 tx=0x7fbfa800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfb97fa640 1 -- 172.21.15.38:0/4130828813 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8017070 con 0x7fbfbc15fb90 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.172+0000 7fbfb97fa640 1 -- 172.21.15.38:0/4130828813 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa800c9d0 con 0x7fbfbc15fb90 2024-04-25T08:42:07.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.173+0000 7fbfb97fa640 1 -- 172.21.15.38:0/4130828813 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800cdc0 con 0x7fbfbc15fb90 2024-04-25T08:42:07.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.173+0000 7fbfc2bce640 1 -- 172.21.15.38:0/4130828813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfbc09afa0 con 0x7fbfbc15fb90 2024-04-25T08:42:07.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.173+0000 7fbfc2bce640 1 -- 172.21.15.38:0/4130828813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfbc0ccb20 con 0x7fbfbc15fb90 2024-04-25T08:42:07.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.174+0000 7fbfc2bce640 1 -- 172.21.15.38:0/4130828813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfbc068e80 con 0x7fbfbc15fb90 2024-04-25T08:42:07.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.174+0000 7fbfb97fa640 1 -- 172.21.15.38:0/4130828813 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbfa801a070 con 0x7fbfbc15fb90 2024-04-25T08:42:07.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.175+0000 7fbfb97fa640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbfa006b4d0 0x7fbfa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.175+0000 7fbfb97fa640 1 -- 172.21.15.38:0/4130828813 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbfa808a370 con 0x7fbfbc15fb90 2024-04-25T08:42:07.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.175+0000 7fbfc096a640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbfa006b4d0 0x7fbfa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.175+0000 7fbfc096a640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbfa006b4d0 0x7fbfa006d990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0034000 tx=0x7fbfb002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:07.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.179+0000 7fbfb97fa640 1 -- 172.21.15.38:0/4130828813 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbfa8058970 con 0x7fbfbc15fb90 2024-04-25T08:42:07.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.362+0000 7fbfc2bce640 1 -- 172.21.15.38:0/4130828813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfbc066eb0 con 0x7fbfbc15fb90 2024-04-25T08:42:07.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.363+0000 7fbfb97fa640 1 -- 172.21.15.38:0/4130828813 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16880 (secure 0 0 0) 0x7fbfa805ca10 con 0x7fbfbc15fb90 2024-04-25T08:42:07.356 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:07.357 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":143,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:42:05.682764+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:01.691830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:03.024111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:07.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.366+0000 7fbfc2bce640 1 -- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbfa006b4d0 msgr2=0x7fbfa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.366+0000 7fbfc2bce640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbfa006b4d0 0x7fbfa006d990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0034000 tx=0x7fbfb002e000 comp rx=0 tx=0).stop 2024-04-25T08:42:07.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.366+0000 7fbfc2bce640 1 -- 172.21.15.38:0/4130828813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc15fb90 msgr2=0x7fbfbc13f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.366+0000 7fbfc2bce640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc15fb90 0x7fbfbc13f390 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800e9e0 tx=0x7fbfa800eea0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.366+0000 7fbfc2bce640 1 -- 172.21.15.38:0/4130828813 shutdown_connections 2024-04-25T08:42:07.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.366+0000 7fbfc2bce640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fbfa006b4d0 0x7fbfa006d990 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.366+0000 7fbfc2bce640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbfbc160020 0x7fbfbc13c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.366+0000 7fbfc2bce640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbfbc15fb90 0x7fbfbc13f390 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.366+0000 7fbfc2bce640 1 --2- 172.21.15.38:0/4130828813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbfbc15f7b0 0x7fbfbc09aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.366+0000 7fbfc2bce640 1 -- 172.21.15.38:0/4130828813 >> 172.21.15.38:0/4130828813 conn(0x7fbfbc0cfd30 msgr2=0x7fbfbc0b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:07.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.366+0000 7fbfc2bce640 1 -- 172.21.15.38:0/4130828813 shutdown_connections 2024-04-25T08:42:07.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.366+0000 7fbfc2bce640 1 -- 172.21.15.38:0/4130828813 wait complete. 2024-04-25T08:42:07.370 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:42:07.370 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:42:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.478+0000 7fcdd7b4c640 1 Processor -- start 2024-04-25T08:42:07.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.478+0000 7fcdd7b4c640 1 -- start start 2024-04-25T08:42:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd7b4c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdd00d3900 0x7fcdd00d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd7b4c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fcdd00d0d90 0x7fcdd00b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd7b4c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd00d3e70 con 0x7fcdd015dc10 2024-04-25T08:42:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd7b4c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd015dff0 con 0x7fcdd00d0d90 2024-04-25T08:42:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd7b4c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00b6990 con 0x7fcdd00d3900 2024-04-25T08:42:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd58e8640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fcdd015dc10 0x7fcdd00ce820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36280/0 (socket says 172.21.15.38:36280) 2024-04-25T08:42:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd58e8640 1 -- 172.21.15.38:0/2899550504 learned_addr learned my addr 172.21.15.38:0/2899550504 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd50e7640 1 --2- 172.21.15.38:0/2899550504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdd00d3900 0x7fcdd00d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd60e9640 1 --2- 172.21.15.38:0/2899550504 >> v2:172.21.15.107:3300/0 conn(0x7fcdd00d0d90 0x7fcdd00b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd48e6640 1 -- 172.21.15.38:0/2899550504 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129574914 0 0) 0x7fcdd00d3e70 con 0x7fcdd015dc10 2024-04-25T08:42:07.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd48e6640 1 -- 172.21.15.38:0/2899550504 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdbc003520 con 0x7fcdd015dc10 2024-04-25T08:42:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd50e7640 1 -- 172.21.15.38:0/2899550504 >> v2:172.21.15.107:3300/0 conn(0x7fcdd00d0d90 msgr2=0x7fcdd00b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd50e7640 1 --2- 172.21.15.38:0/2899550504 >> v2:172.21.15.107:3300/0 conn(0x7fcdd00d0d90 0x7fcdd00b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd50e7640 1 -- 172.21.15.38:0/2899550504 >> v1:172.21.15.38:6789/0 conn(0x7fcdd015dc10 legacy=0x7fcdd00ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.479+0000 7fcdd50e7640 1 -- 172.21.15.38:0/2899550504 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd00d4150 con 0x7fcdd00d3900 2024-04-25T08:42:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.480+0000 7fcdd50e7640 1 --2- 172.21.15.38:0/2899550504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdd00d3900 0x7fcdd00d0850 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00097a0 tx=0x7fcdc002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78efe4a8b75c5826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.480+0000 7fcdd48e6640 1 -- 172.21.15.38:0/2899550504 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0037070 con 0x7fcdd00d3900 2024-04-25T08:42:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.480+0000 7fcdd48e6640 1 -- 172.21.15.38:0/2899550504 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc002ae50 con 0x7fcdd00d3900 2024-04-25T08:42:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.480+0000 7fcdd48e6640 1 -- 172.21.15.38:0/2899550504 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0032640 con 0x7fcdd00d3900 2024-04-25T08:42:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.481+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/2899550504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdd00d3900 msgr2=0x7fcdd00d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.481+0000 7fcdd7b4c640 1 --2- 172.21.15.38:0/2899550504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdd00d3900 0x7fcdd00d0850 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00097a0 tx=0x7fcdc002a100 comp rx=0 tx=0).stop 2024-04-25T08:42:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.481+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/2899550504 shutdown_connections 2024-04-25T08:42:07.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.481+0000 7fcdd7b4c640 1 --2- 172.21.15.38:0/2899550504 >> v2:172.21.15.107:3300/0 conn(0x7fcdd00d0d90 0x7fcdd00b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.481+0000 7fcdd7b4c640 1 --2- 172.21.15.38:0/2899550504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdd00d3900 0x7fcdd00d0850 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.481+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/2899550504 >> 172.21.15.38:0/2899550504 conn(0x7fcdd0066d10 msgr2=0x7fcdd0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.481+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/2899550504 shutdown_connections 2024-04-25T08:42:07.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.481+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/2899550504 wait complete. 2024-04-25T08:42:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.482+0000 7fcdd7b4c640 1 Processor -- start 2024-04-25T08:42:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.482+0000 7fcdd7b4c640 1 -- start start 2024-04-25T08:42:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd7b4c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdd00d0d90 0x7fcdd00cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd7b4c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdd00d3900 0x7fcdd00c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd7b4c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdd015dc10 0x7fcdd00c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd7b4c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00b6bc0 con 0x7fcdd00d3900 2024-04-25T08:42:07.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd7b4c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00d4710 con 0x7fcdd015dc10 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd7b4c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00cef30 con 0x7fcdd00d0d90 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd50e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdd00d3900 0x7fcdd00c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd50e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdd00d3900 0x7fcdd00c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52032/0 (socket says 172.21.15.38:52032) 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd50e7640 1 -- 172.21.15.38:0/562767519 learned_addr learned my addr 172.21.15.38:0/562767519 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd60e9640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdd015dc10 0x7fcdd00c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd58e8640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdd00d0d90 0x7fcdd00cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd50e7640 1 -- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdd00d0d90 msgr2=0x7fcdd00cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd50e7640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdd00d0d90 0x7fcdd00cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd50e7640 1 -- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdd015dc10 msgr2=0x7fcdd00c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd50e7640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdd015dc10 0x7fcdd00c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd50e7640 1 -- 172.21.15.38:0/562767519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd01f4360 con 0x7fcdd00d3900 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.483+0000 7fcdd50e7640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdd00d3900 0x7fcdd00c9420 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00098c0 tx=0x7fcdc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:07.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.484+0000 7fcdc67fc640 1 -- 172.21.15.38:0/562767519 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0040030 con 0x7fcdd00d3900 2024-04-25T08:42:07.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.484+0000 7fcdc67fc640 1 -- 172.21.15.38:0/562767519 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc0038040 con 0x7fcdd00d3900 2024-04-25T08:42:07.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.484+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/562767519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd01f4080 con 0x7fcdd00d3900 2024-04-25T08:42:07.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.484+0000 7fcdc67fc640 1 -- 172.21.15.38:0/562767519 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc0037070 con 0x7fcdd00d3900 2024-04-25T08:42:07.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.484+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/562767519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd00cbe30 con 0x7fcdd00d3900 2024-04-25T08:42:07.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.485+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/562767519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd0068e80 con 0x7fcdd00d3900 2024-04-25T08:42:07.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.485+0000 7fcdc67fc640 1 -- 172.21.15.38:0/562767519 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdc0043070 con 0x7fcdd00d3900 2024-04-25T08:42:07.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.486+0000 7fcdc67fc640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcdb406b4d0 0x7fcdb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.486+0000 7fcdc67fc640 1 -- 172.21.15.38:0/562767519 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcdc00aa010 con 0x7fcdd00d3900 2024-04-25T08:42:07.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.486+0000 7fcdd58e8640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcdb406b4d0 0x7fcdb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.487+0000 7fcdd58e8640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcdb406b4d0 0x7fcdb406d990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc005880 tx=0x7fcdcc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:07.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.490+0000 7fcdc67fc640 1 -- 172.21.15.38:0/562767519 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdc0030340 con 0x7fcdd00d3900 2024-04-25T08:42:07.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.665+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/562767519 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcdd00b48f0 con 0x7fcdb406b4d0 2024-04-25T08:42:07.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.672+0000 7fcdc67fc640 1 -- 172.21.15.38:0/562767519 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7fcdd00b48f0 con 0x7fcdb406b4d0 2024-04-25T08:42:07.667 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:07.668 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:42:07.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.675+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcdb406b4d0 msgr2=0x7fcdb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.675+0000 7fcdd7b4c640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcdb406b4d0 0x7fcdb406d990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc005880 tx=0x7fcdcc009040 comp rx=0 tx=0).stop 2024-04-25T08:42:07.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.675+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/562767519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdd00d3900 msgr2=0x7fcdd00c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.675+0000 7fcdd7b4c640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdd00d3900 0x7fcdd00c9420 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00098c0 tx=0x7fcdc00046c0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.675+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/562767519 shutdown_connections 2024-04-25T08:42:07.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.675+0000 7fcdd7b4c640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcdd015dc10 0x7fcdd00c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.675+0000 7fcdd7b4c640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcdd00d3900 0x7fcdd00c9420 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.675+0000 7fcdd7b4c640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fcdb406b4d0 0x7fcdb406d990 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.675+0000 7fcdd7b4c640 1 --2- 172.21.15.38:0/562767519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcdd00d0d90 0x7fcdd00cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.675+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/562767519 >> 172.21.15.38:0/562767519 conn(0x7fcdd0066d10 msgr2=0x7fcdd015eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:07.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.675+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/562767519 shutdown_connections 2024-04-25T08:42:07.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.675+0000 7fcdd7b4c640 1 -- 172.21.15.38:0/562767519 wait complete. 2024-04-25T08:42:07.678 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-04-25T08:42:07.353694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":502016,"kb_used_data":15240,"kb_used_omap":12,"kb_used_meta":486643,"kb_avail":754472704,"statfs":{"total":773094113280,"available":772580048896,"internally_reserved":0,"allocated":15605760,"data_stored":7907620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":498322992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.688389"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710844+0000","last_change":"2024-04-25T08:42:04.701156+0000","last_active":"2024-04-25T08:42:05.710844+0000","last_peered":"2024-04-25T08:42:05.710844+0000","last_clean":"2024-04-25T08:42:05.710844+0000","last_became_active":"2024-04-25T08:42:04.700864+0000","last_became_peered":"2024-04-25T08:42:04.700864+0000","last_unstale":"2024-04-25T08:42:05.710844+0000","last_undegraded":"2024-04-25T08:42:05.710844+0000","last_fullsized":"2024-04-25T08:42:05.710844+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:03.071862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857054+0000","last_change":"2024-04-25T08:42:04.688967+0000","last_active":"2024-04-25T08:42:05.857054+0000","last_peered":"2024-04-25T08:42:05.857054+0000","last_clean":"2024-04-25T08:42:05.857054+0000","last_became_active":"2024-04-25T08:42:04.688684+0000","last_became_peered":"2024-04-25T08:42:04.688684+0000","last_unstale":"2024-04-25T08:42:05.857054+0000","last_undegraded":"2024-04-25T08:42:05.857054+0000","last_fullsized":"2024-04-25T08:42:05.857054+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:35.361036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857554+0000","last_change":"2024-04-25T08:42:04.696220+0000","last_active":"2024-04-25T08:42:05.857554+0000","last_peered":"2024-04-25T08:42:05.857554+0000","last_clean":"2024-04-25T08:42:05.857554+0000","last_became_active":"2024-04-25T08:42:04.695935+0000","last_became_peered":"2024-04-25T08:42:04.695935+0000","last_unstale":"2024-04-25T08:42:05.857554+0000","last_undegraded":"2024-04-25T08:42:05.857554+0000","last_fullsized":"2024-04-25T08:42:05.857554+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:44.343322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696648+0000","last_change":"2024-04-25T08:42:04.692485+0000","last_active":"2024-04-25T08:42:05.696648+0000","last_peered":"2024-04-25T08:42:05.696648+0000","last_clean":"2024-04-25T08:42:05.696648+0000","last_became_active":"2024-04-25T08:42:04.692191+0000","last_became_peered":"2024-04-25T08:42:04.692191+0000","last_unstale":"2024-04-25T08:42:05.696648+0000","last_undegraded":"2024-04-25T08:42:05.696648+0000","last_fullsized":"2024-04-25T08:42:05.696648+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:11.489900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858169+0000","last_change":"2024-04-25T08:42:04.697680+0000","last_active":"2024-04-25T08:42:05.858169+0000","last_peered":"2024-04-25T08:42:05.858169+0000","last_clean":"2024-04-25T08:42:05.858169+0000","last_became_active":"2024-04-25T08:42:04.697415+0000","last_became_peered":"2024-04-25T08:42:04.697415+0000","last_unstale":"2024-04-25T08:42:05.858169+0000","last_undegraded":"2024-04-25T08:42:05.858169+0000","last_fullsized":"2024-04-25T08:42:05.858169+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.109378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857384+0000","last_change":"2024-04-25T08:42:04.694813+0000","last_active":"2024-04-25T08:42:05.857384+0000","last_peered":"2024-04-25T08:42:05.857384+0000","last_clean":"2024-04-25T08:42:05.857384+0000","last_became_active":"2024-04-25T08:42:04.694560+0000","last_became_peered":"2024-04-25T08:42:04.694560+0000","last_unstale":"2024-04-25T08:42:05.857384+0000","last_undegraded":"2024-04-25T08:42:05.857384+0000","last_fullsized":"2024-04-25T08:42:05.857384+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:18.372469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710620+0000","last_change":"2024-04-25T08:42:04.697115+0000","last_active":"2024-04-25T08:42:05.710620+0000","last_peered":"2024-04-25T08:42:05.710620+0000","last_clean":"2024-04-25T08:42:05.710620+0000","last_became_active":"2024-04-25T08:42:04.696809+0000","last_became_peered":"2024-04-25T08:42:04.696809+0000","last_unstale":"2024-04-25T08:42:05.710620+0000","last_undegraded":"2024-04-25T08:42:05.710620+0000","last_fullsized":"2024-04-25T08:42:05.710620+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:30.359852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697880+0000","last_change":"2024-04-25T08:42:04.699128+0000","last_active":"2024-04-25T08:42:05.697880+0000","last_peered":"2024-04-25T08:42:05.697880+0000","last_clean":"2024-04-25T08:42:05.697880+0000","last_became_active":"2024-04-25T08:42:04.698474+0000","last_became_peered":"2024-04-25T08:42:04.698474+0000","last_unstale":"2024-04-25T08:42:05.697880+0000","last_undegraded":"2024-04-25T08:42:05.697880+0000","last_fullsized":"2024-04-25T08:42:05.697880+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:45.050773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858145+0000","last_change":"2024-04-25T08:42:04.693012+0000","last_active":"2024-04-25T08:42:05.858145+0000","last_peered":"2024-04-25T08:42:05.858145+0000","last_clean":"2024-04-25T08:42:05.858145+0000","last_became_active":"2024-04-25T08:42:04.692744+0000","last_became_peered":"2024-04-25T08:42:04.692744+0000","last_unstale":"2024-04-25T08:42:05.858145+0000","last_undegraded":"2024-04-25T08:42:05.858145+0000","last_fullsized":"2024-04-25T08:42:05.858145+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:59.786563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858025+0000","last_change":"2024-04-25T08:42:04.706570+0000","last_active":"2024-04-25T08:42:05.858025+0000","last_peered":"2024-04-25T08:42:05.858025+0000","last_clean":"2024-04-25T08:42:05.858025+0000","last_became_active":"2024-04-25T08:42:04.706292+0000","last_became_peered":"2024-04-25T08:42:04.706292+0000","last_unstale":"2024-04-25T08:42:05.858025+0000","last_undegraded":"2024-04-25T08:42:05.858025+0000","last_fullsized":"2024-04-25T08:42:05.858025+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:09.978775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858182+0000","last_change":"2024-04-25T08:42:04.695428+0000","last_active":"2024-04-25T08:42:05.858182+0000","last_peered":"2024-04-25T08:42:05.858182+0000","last_clean":"2024-04-25T08:42:05.858182+0000","last_became_active":"2024-04-25T08:42:04.694914+0000","last_became_peered":"2024-04-25T08:42:04.694914+0000","last_unstale":"2024-04-25T08:42:05.858182+0000","last_undegraded":"2024-04-25T08:42:05.858182+0000","last_fullsized":"2024-04-25T08:42:05.858182+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:36.004116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696620+0000","last_change":"2024-04-25T08:42:04.701163+0000","last_active":"2024-04-25T08:42:05.696620+0000","last_peered":"2024-04-25T08:42:05.696620+0000","last_clean":"2024-04-25T08:42:05.696620+0000","last_became_active":"2024-04-25T08:42:04.700877+0000","last_became_peered":"2024-04-25T08:42:04.700877+0000","last_unstale":"2024-04-25T08:42:05.696620+0000","last_undegraded":"2024-04-25T08:42:05.696620+0000","last_fullsized":"2024-04-25T08:42:05.696620+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:45.050773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858405+0000","last_change":"2024-04-25T08:42:04.707187+0000","last_active":"2024-04-25T08:42:05.858405+0000","last_peered":"2024-04-25T08:42:05.858405+0000","last_clean":"2024-04-25T08:42:05.858405+0000","last_became_active":"2024-04-25T08:42:04.706793+0000","last_became_peered":"2024-04-25T08:42:04.706793+0000","last_unstale":"2024-04-25T08:42:05.858405+0000","last_undegraded":"2024-04-25T08:42:05.858405+0000","last_fullsized":"2024-04-25T08:42:05.858405+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:51.742834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696675+0000","last_change":"2024-04-25T08:42:04.701329+0000","last_active":"2024-04-25T08:42:05.696675+0000","last_peered":"2024-04-25T08:42:05.696675+0000","last_clean":"2024-04-25T08:42:05.696675+0000","last_became_active":"2024-04-25T08:42:04.700872+0000","last_became_peered":"2024-04-25T08:42:04.700872+0000","last_unstale":"2024-04-25T08:42:05.696675+0000","last_undegraded":"2024-04-25T08:42:05.696675+0000","last_fullsized":"2024-04-25T08:42:05.696675+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:08.507867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857787+0000","last_change":"2024-04-25T08:42:04.694008+0000","last_active":"2024-04-25T08:42:05.857787+0000","last_peered":"2024-04-25T08:42:05.857787+0000","last_clean":"2024-04-25T08:42:05.857787+0000","last_became_active":"2024-04-25T08:42:04.693712+0000","last_became_peered":"2024-04-25T08:42:04.693712+0000","last_unstale":"2024-04-25T08:42:05.857787+0000","last_undegraded":"2024-04-25T08:42:05.857787+0000","last_fullsized":"2024-04-25T08:42:05.857787+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:17.198629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857899+0000","last_change":"2024-04-25T08:42:04.698772+0000","last_active":"2024-04-25T08:42:05.857899+0000","last_peered":"2024-04-25T08:42:05.857899+0000","last_clean":"2024-04-25T08:42:05.857899+0000","last_became_active":"2024-04-25T08:42:04.698492+0000","last_became_peered":"2024-04-25T08:42:04.698492+0000","last_unstale":"2024-04-25T08:42:05.857899+0000","last_undegraded":"2024-04-25T08:42:05.857899+0000","last_fullsized":"2024-04-25T08:42:05.857899+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:58.280649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696839+0000","last_change":"2024-04-25T08:42:04.706496+0000","last_active":"2024-04-25T08:42:05.696839+0000","last_peered":"2024-04-25T08:42:05.696839+0000","last_clean":"2024-04-25T08:42:05.696839+0000","last_became_active":"2024-04-25T08:42:04.706181+0000","last_became_peered":"2024-04-25T08:42:04.706181+0000","last_unstale":"2024-04-25T08:42:05.696839+0000","last_undegraded":"2024-04-25T08:42:05.696839+0000","last_fullsized":"2024-04-25T08:42:05.696839+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:21.404708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857890+0000","last_change":"2024-04-25T08:42:04.699573+0000","last_active":"2024-04-25T08:42:05.857890+0000","last_peered":"2024-04-25T08:42:05.857890+0000","last_clean":"2024-04-25T08:42:05.857890+0000","last_became_active":"2024-04-25T08:42:04.699136+0000","last_became_peered":"2024-04-25T08:42:04.699136+0000","last_unstale":"2024-04-25T08:42:05.857890+0000","last_undegraded":"2024-04-25T08:42:05.857890+0000","last_fullsized":"2024-04-25T08:42:05.857890+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:50.797871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819307+0000","last_change":"2024-04-25T08:42:04.692192+0000","last_active":"2024-04-25T08:42:05.819307+0000","last_peered":"2024-04-25T08:42:05.819307+0000","last_clean":"2024-04-25T08:42:05.819307+0000","last_became_active":"2024-04-25T08:42:04.690390+0000","last_became_peered":"2024-04-25T08:42:04.690390+0000","last_unstale":"2024-04-25T08:42:05.819307+0000","last_undegraded":"2024-04-25T08:42:05.819307+0000","last_fullsized":"2024-04-25T08:42:05.819307+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:12.205286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.704131+0000","last_change":"2024-04-25T08:42:04.701325+0000","last_active":"2024-04-25T08:42:05.704131+0000","last_peered":"2024-04-25T08:42:05.704131+0000","last_clean":"2024-04-25T08:42:05.704131+0000","last_became_active":"2024-04-25T08:42:04.700994+0000","last_became_peered":"2024-04-25T08:42:04.700994+0000","last_unstale":"2024-04-25T08:42:05.704131+0000","last_undegraded":"2024-04-25T08:42:05.704131+0000","last_fullsized":"2024-04-25T08:42:05.704131+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:51.569719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858001+0000","last_change":"2024-04-25T08:42:04.691242+0000","last_active":"2024-04-25T08:42:05.858001+0000","last_peered":"2024-04-25T08:42:05.858001+0000","last_clean":"2024-04-25T08:42:05.858001+0000","last_became_active":"2024-04-25T08:42:04.690944+0000","last_became_peered":"2024-04-25T08:42:04.690944+0000","last_unstale":"2024-04-25T08:42:05.858001+0000","last_undegraded":"2024-04-25T08:42:05.858001+0000","last_fullsized":"2024-04-25T08:42:05.858001+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:20.385308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710602+0000","last_change":"2024-04-25T08:42:04.697169+0000","last_active":"2024-04-25T08:42:05.710602+0000","last_peered":"2024-04-25T08:42:05.710602+0000","last_clean":"2024-04-25T08:42:05.710602+0000","last_became_active":"2024-04-25T08:42:04.696887+0000","last_became_peered":"2024-04-25T08:42:04.696887+0000","last_unstale":"2024-04-25T08:42:05.710602+0000","last_undegraded":"2024-04-25T08:42:05.710602+0000","last_fullsized":"2024-04-25T08:42:05.710602+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:09.484280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697094+0000","last_change":"2024-04-25T08:42:04.690437+0000","last_active":"2024-04-25T08:42:05.697094+0000","last_peered":"2024-04-25T08:42:05.697094+0000","last_clean":"2024-04-25T08:42:05.697094+0000","last_became_active":"2024-04-25T08:42:04.690146+0000","last_became_peered":"2024-04-25T08:42:04.690146+0000","last_unstale":"2024-04-25T08:42:05.697094+0000","last_undegraded":"2024-04-25T08:42:05.697094+0000","last_fullsized":"2024-04-25T08:42:05.697094+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:01.388665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857435+0000","last_change":"2024-04-25T08:42:04.692824+0000","last_active":"2024-04-25T08:42:05.857435+0000","last_peered":"2024-04-25T08:42:05.857435+0000","last_clean":"2024-04-25T08:42:05.857435+0000","last_became_active":"2024-04-25T08:42:04.692548+0000","last_became_peered":"2024-04-25T08:42:04.692548+0000","last_unstale":"2024-04-25T08:42:05.857435+0000","last_undegraded":"2024-04-25T08:42:05.857435+0000","last_fullsized":"2024-04-25T08:42:05.857435+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:59.373510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710885+0000","last_change":"2024-04-25T08:42:04.706510+0000","last_active":"2024-04-25T08:42:05.710885+0000","last_peered":"2024-04-25T08:42:05.710885+0000","last_clean":"2024-04-25T08:42:05.710885+0000","last_became_active":"2024-04-25T08:42:04.706178+0000","last_became_peered":"2024-04-25T08:42:04.706178+0000","last_unstale":"2024-04-25T08:42:05.710885+0000","last_undegraded":"2024-04-25T08:42:05.710885+0000","last_fullsized":"2024-04-25T08:42:05.710885+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:06.864814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710792+0000","last_change":"2024-04-25T08:42:04.695194+0000","last_active":"2024-04-25T08:42:05.710792+0000","last_peered":"2024-04-25T08:42:05.710792+0000","last_clean":"2024-04-25T08:42:05.710792+0000","last_became_active":"2024-04-25T08:42:04.694890+0000","last_became_peered":"2024-04-25T08:42:04.694890+0000","last_unstale":"2024-04-25T08:42:05.710792+0000","last_undegraded":"2024-04-25T08:42:05.710792+0000","last_fullsized":"2024-04-25T08:42:05.710792+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:55.851053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858271+0000","last_change":"2024-04-25T08:42:04.697707+0000","last_active":"2024-04-25T08:42:05.858271+0000","last_peered":"2024-04-25T08:42:05.858271+0000","last_clean":"2024-04-25T08:42:05.858271+0000","last_became_active":"2024-04-25T08:42:04.697389+0000","last_became_peered":"2024-04-25T08:42:04.697389+0000","last_unstale":"2024-04-25T08:42:05.858271+0000","last_undegraded":"2024-04-25T08:42:05.858271+0000","last_fullsized":"2024-04-25T08:42:05.858271+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:02.381300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858534+0000","last_change":"2024-04-25T08:42:04.703070+0000","last_active":"2024-04-25T08:42:05.858534+0000","last_peered":"2024-04-25T08:42:05.858534+0000","last_clean":"2024-04-25T08:42:05.858534+0000","last_became_active":"2024-04-25T08:42:04.702788+0000","last_became_peered":"2024-04-25T08:42:04.702788+0000","last_unstale":"2024-04-25T08:42:05.858534+0000","last_undegraded":"2024-04-25T08:42:05.858534+0000","last_fullsized":"2024-04-25T08:42:05.858534+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:58.459727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697618+0000","last_change":"2024-04-25T08:42:04.698068+0000","last_active":"2024-04-25T08:42:05.697618+0000","last_peered":"2024-04-25T08:42:05.697618+0000","last_clean":"2024-04-25T08:42:05.697618+0000","last_became_active":"2024-04-25T08:42:04.697783+0000","last_became_peered":"2024-04-25T08:42:04.697783+0000","last_unstale":"2024-04-25T08:42:05.697618+0000","last_undegraded":"2024-04-25T08:42:05.697618+0000","last_fullsized":"2024-04-25T08:42:05.697618+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:11.489900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858318+0000","last_change":"2024-04-25T08:42:04.691270+0000","last_active":"2024-04-25T08:42:05.858318+0000","last_peered":"2024-04-25T08:42:05.858318+0000","last_clean":"2024-04-25T08:42:05.858318+0000","last_became_active":"2024-04-25T08:42:04.690898+0000","last_became_peered":"2024-04-25T08:42:04.690898+0000","last_unstale":"2024-04-25T08:42:05.858318+0000","last_undegraded":"2024-04-25T08:42:05.858318+0000","last_fullsized":"2024-04-25T08:42:05.858318+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:58.451011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857155+0000","last_change":"2024-04-25T08:42:04.687344+0000","last_active":"2024-04-25T08:42:05.857155+0000","last_peered":"2024-04-25T08:42:05.857155+0000","last_clean":"2024-04-25T08:42:05.857155+0000","last_became_active":"2024-04-25T08:42:04.686981+0000","last_became_peered":"2024-04-25T08:42:04.686981+0000","last_unstale":"2024-04-25T08:42:05.857155+0000","last_undegraded":"2024-04-25T08:42:05.857155+0000","last_fullsized":"2024-04-25T08:42:05.857155+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:51.569868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697073+0000","last_change":"2024-04-25T08:42:04.699542+0000","last_active":"2024-04-25T08:42:05.697073+0000","last_peered":"2024-04-25T08:42:05.697073+0000","last_clean":"2024-04-25T08:42:05.697073+0000","last_became_active":"2024-04-25T08:42:04.698834+0000","last_became_peered":"2024-04-25T08:42:04.698834+0000","last_unstale":"2024-04-25T08:42:05.697073+0000","last_undegraded":"2024-04-25T08:42:05.697073+0000","last_fullsized":"2024-04-25T08:42:05.697073+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:05.170508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857375+0000","last_change":"2024-04-25T08:42:04.688612+0000","last_active":"2024-04-25T08:42:05.857375+0000","last_peered":"2024-04-25T08:42:05.857375+0000","last_clean":"2024-04-25T08:42:05.857375+0000","last_became_active":"2024-04-25T08:42:04.688319+0000","last_became_peered":"2024-04-25T08:42:04.688319+0000","last_unstale":"2024-04-25T08:42:05.857375+0000","last_undegraded":"2024-04-25T08:42:05.857375+0000","last_fullsized":"2024-04-25T08:42:05.857375+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:38.536326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"142'4","reported_seq":18,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858402+0000","last_change":"2024-04-25T08:42:03.685968+0000","last_active":"2024-04-25T08:42:05.858402+0000","last_peered":"2024-04-25T08:42:05.858402+0000","last_clean":"2024-04-25T08:42:05.858402+0000","last_became_active":"2024-04-25T08:42:03.685479+0000","last_became_peered":"2024-04-25T08:42:03.685479+0000","last_unstale":"2024-04-25T08:42:05.858402+0000","last_undegraded":"2024-04-25T08:42:05.858402+0000","last_fullsized":"2024-04-25T08:42:05.858402+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:08.475116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857641+0000","last_change":"2024-04-25T08:42:04.691442+0000","last_active":"2024-04-25T08:42:05.857641+0000","last_peered":"2024-04-25T08:42:05.857641+0000","last_clean":"2024-04-25T08:42:05.857641+0000","last_became_active":"2024-04-25T08:42:04.691165+0000","last_became_peered":"2024-04-25T08:42:04.691165+0000","last_unstale":"2024-04-25T08:42:05.857641+0000","last_undegraded":"2024-04-25T08:42:05.857641+0000","last_fullsized":"2024-04-25T08:42:05.857641+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:28.909203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819141+0000","last_change":"2024-04-25T08:42:03.689935+0000","last_active":"2024-04-25T08:42:05.819141+0000","last_peered":"2024-04-25T08:42:05.819141+0000","last_clean":"2024-04-25T08:42:05.819141+0000","last_became_active":"2024-04-25T08:42:03.689579+0000","last_became_peered":"2024-04-25T08:42:03.689579+0000","last_unstale":"2024-04-25T08:42:05.819141+0000","last_undegraded":"2024-04-25T08:42:05.819141+0000","last_fullsized":"2024-04-25T08:42:05.819141+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:47.894977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697568+0000","last_change":"2024-04-25T08:42:04.691466+0000","last_active":"2024-04-25T08:42:05.697568+0000","last_peered":"2024-04-25T08:42:05.697568+0000","last_clean":"2024-04-25T08:42:05.697568+0000","last_became_active":"2024-04-25T08:42:04.691194+0000","last_became_peered":"2024-04-25T08:42:04.691194+0000","last_unstale":"2024-04-25T08:42:05.697568+0000","last_undegraded":"2024-04-25T08:42:05.697568+0000","last_fullsized":"2024-04-25T08:42:05.697568+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:56.306539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819393+0000","last_change":"2024-04-25T08:42:03.684012+0000","last_active":"2024-04-25T08:42:05.819393+0000","last_peered":"2024-04-25T08:42:05.819393+0000","last_clean":"2024-04-25T08:42:05.819393+0000","last_became_active":"2024-04-25T08:42:03.683723+0000","last_became_peered":"2024-04-25T08:42:03.683723+0000","last_unstale":"2024-04-25T08:42:05.819393+0000","last_undegraded":"2024-04-25T08:42:05.819393+0000","last_fullsized":"2024-04-25T08:42:05.819393+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:36.782073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819373+0000","last_change":"2024-04-25T08:42:04.693990+0000","last_active":"2024-04-25T08:42:05.819373+0000","last_peered":"2024-04-25T08:42:05.819373+0000","last_clean":"2024-04-25T08:42:05.819373+0000","last_became_active":"2024-04-25T08:42:04.693721+0000","last_became_peered":"2024-04-25T08:42:04.693721+0000","last_unstale":"2024-04-25T08:42:05.819373+0000","last_undegraded":"2024-04-25T08:42:05.819373+0000","last_fullsized":"2024-04-25T08:42:05.819373+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:58.496763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857478+0000","last_change":"2024-04-25T08:42:03.688587+0000","last_active":"2024-04-25T08:42:05.857478+0000","last_peered":"2024-04-25T08:42:05.857478+0000","last_clean":"2024-04-25T08:42:05.857478+0000","last_became_active":"2024-04-25T08:42:03.688246+0000","last_became_peered":"2024-04-25T08:42:03.688246+0000","last_unstale":"2024-04-25T08:42:05.857478+0000","last_undegraded":"2024-04-25T08:42:05.857478+0000","last_fullsized":"2024-04-25T08:42:05.857478+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:00.379501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.704162+0000","last_change":"2024-04-25T08:42:04.704666+0000","last_active":"2024-04-25T08:42:05.704162+0000","last_peered":"2024-04-25T08:42:05.704162+0000","last_clean":"2024-04-25T08:42:05.704162+0000","last_became_active":"2024-04-25T08:42:04.704391+0000","last_became_peered":"2024-04-25T08:42:04.704391+0000","last_unstale":"2024-04-25T08:42:05.704162+0000","last_undegraded":"2024-04-25T08:42:05.704162+0000","last_fullsized":"2024-04-25T08:42:05.704162+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:59.300899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697652+0000","last_change":"2024-04-25T08:42:03.686925+0000","last_active":"2024-04-25T08:42:05.697652+0000","last_peered":"2024-04-25T08:42:05.697652+0000","last_clean":"2024-04-25T08:42:05.697652+0000","last_became_active":"2024-04-25T08:42:03.685881+0000","last_became_peered":"2024-04-25T08:42:03.685881+0000","last_unstale":"2024-04-25T08:42:05.697652+0000","last_undegraded":"2024-04-25T08:42:05.697652+0000","last_fullsized":"2024-04-25T08:42:05.697652+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:38.119854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696682+0000","last_change":"2024-04-25T08:42:04.696019+0000","last_active":"2024-04-25T08:42:05.696682+0000","last_peered":"2024-04-25T08:42:05.696682+0000","last_clean":"2024-04-25T08:42:05.696682+0000","last_became_active":"2024-04-25T08:42:04.695686+0000","last_became_peered":"2024-04-25T08:42:04.695686+0000","last_unstale":"2024-04-25T08:42:05.696682+0000","last_undegraded":"2024-04-25T08:42:05.696682+0000","last_fullsized":"2024-04-25T08:42:05.696682+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:38.366164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857634+0000","last_change":"2024-04-25T08:42:03.683354+0000","last_active":"2024-04-25T08:42:05.857634+0000","last_peered":"2024-04-25T08:42:05.857634+0000","last_clean":"2024-04-25T08:42:05.857634+0000","last_became_active":"2024-04-25T08:42:03.683052+0000","last_became_peered":"2024-04-25T08:42:03.683052+0000","last_unstale":"2024-04-25T08:42:05.857634+0000","last_undegraded":"2024-04-25T08:42:05.857634+0000","last_fullsized":"2024-04-25T08:42:05.857634+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:09.409360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858245+0000","last_change":"2024-04-25T08:42:04.702957+0000","last_active":"2024-04-25T08:42:05.858245+0000","last_peered":"2024-04-25T08:42:05.858245+0000","last_clean":"2024-04-25T08:42:05.858245+0000","last_became_active":"2024-04-25T08:42:04.702620+0000","last_became_peered":"2024-04-25T08:42:04.702620+0000","last_unstale":"2024-04-25T08:42:05.858245+0000","last_undegraded":"2024-04-25T08:42:05.858245+0000","last_fullsized":"2024-04-25T08:42:05.858245+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:51.569868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710576+0000","last_change":"2024-04-25T08:42:03.691204+0000","last_active":"2024-04-25T08:42:05.710576+0000","last_peered":"2024-04-25T08:42:05.710576+0000","last_clean":"2024-04-25T08:42:05.710576+0000","last_became_active":"2024-04-25T08:42:03.690929+0000","last_became_peered":"2024-04-25T08:42:03.690929+0000","last_unstale":"2024-04-25T08:42:05.710576+0000","last_undegraded":"2024-04-25T08:42:05.710576+0000","last_fullsized":"2024-04-25T08:42:05.710576+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:09.366192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710862+0000","last_change":"2024-04-25T08:42:04.701644+0000","last_active":"2024-04-25T08:42:05.710862+0000","last_peered":"2024-04-25T08:42:05.710862+0000","last_clean":"2024-04-25T08:42:05.710862+0000","last_became_active":"2024-04-25T08:42:04.701270+0000","last_became_peered":"2024-04-25T08:42:04.701270+0000","last_unstale":"2024-04-25T08:42:05.710862+0000","last_undegraded":"2024-04-25T08:42:05.710862+0000","last_fullsized":"2024-04-25T08:42:05.710862+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:35.361036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858001+0000","last_change":"2024-04-25T08:42:03.693479+0000","last_active":"2024-04-25T08:42:05.858001+0000","last_peered":"2024-04-25T08:42:05.858001+0000","last_clean":"2024-04-25T08:42:05.858001+0000","last_became_active":"2024-04-25T08:42:03.692971+0000","last_became_peered":"2024-04-25T08:42:03.692971+0000","last_unstale":"2024-04-25T08:42:05.858001+0000","last_undegraded":"2024-04-25T08:42:05.858001+0000","last_fullsized":"2024-04-25T08:42:05.858001+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:07.498704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697850+0000","last_change":"2024-04-25T08:42:04.689059+0000","last_active":"2024-04-25T08:42:05.697850+0000","last_peered":"2024-04-25T08:42:05.697850+0000","last_clean":"2024-04-25T08:42:05.697850+0000","last_became_active":"2024-04-25T08:42:04.688772+0000","last_became_peered":"2024-04-25T08:42:04.688772+0000","last_unstale":"2024-04-25T08:42:05.697850+0000","last_undegraded":"2024-04-25T08:42:05.697850+0000","last_fullsized":"2024-04-25T08:42:05.697850+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:10.418523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819373+0000","last_change":"2024-04-25T08:42:03.685310+0000","last_active":"2024-04-25T08:42:05.819373+0000","last_peered":"2024-04-25T08:42:05.819373+0000","last_clean":"2024-04-25T08:42:05.819373+0000","last_became_active":"2024-04-25T08:42:03.685056+0000","last_became_peered":"2024-04-25T08:42:03.685056+0000","last_unstale":"2024-04-25T08:42:05.819373+0000","last_undegraded":"2024-04-25T08:42:05.819373+0000","last_fullsized":"2024-04-25T08:42:05.819373+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:16.189465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858318+0000","last_change":"2024-04-25T08:42:04.695200+0000","last_active":"2024-04-25T08:42:05.858318+0000","last_peered":"2024-04-25T08:42:05.858318+0000","last_clean":"2024-04-25T08:42:05.858318+0000","last_became_active":"2024-04-25T08:42:04.694539+0000","last_became_peered":"2024-04-25T08:42:04.694539+0000","last_unstale":"2024-04-25T08:42:05.858318+0000","last_undegraded":"2024-04-25T08:42:05.858318+0000","last_fullsized":"2024-04-25T08:42:05.858318+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:25.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710937+0000","last_change":"2024-04-25T08:42:03.689429+0000","last_active":"2024-04-25T08:42:05.710937+0000","last_peered":"2024-04-25T08:42:05.710937+0000","last_clean":"2024-04-25T08:42:05.710937+0000","last_became_active":"2024-04-25T08:42:03.689160+0000","last_became_peered":"2024-04-25T08:42:03.689160+0000","last_unstale":"2024-04-25T08:42:05.710937+0000","last_undegraded":"2024-04-25T08:42:05.710937+0000","last_fullsized":"2024-04-25T08:42:05.710937+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:04.161345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858032+0000","last_change":"2024-04-25T08:42:04.702122+0000","last_active":"2024-04-25T08:42:05.858032+0000","last_peered":"2024-04-25T08:42:05.858032+0000","last_clean":"2024-04-25T08:42:05.858032+0000","last_became_active":"2024-04-25T08:42:04.699132+0000","last_became_peered":"2024-04-25T08:42:04.699132+0000","last_unstale":"2024-04-25T08:42:05.858032+0000","last_undegraded":"2024-04-25T08:42:05.858032+0000","last_fullsized":"2024-04-25T08:42:05.858032+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:56.813766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"142'3","reported_seq":17,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858540+0000","last_change":"2024-04-25T08:42:03.694273+0000","last_active":"2024-04-25T08:42:05.858540+0000","last_peered":"2024-04-25T08:42:05.858540+0000","last_clean":"2024-04-25T08:42:05.858540+0000","last_became_active":"2024-04-25T08:42:03.693986+0000","last_became_peered":"2024-04-25T08:42:03.693986+0000","last_unstale":"2024-04-25T08:42:05.858540+0000","last_undegraded":"2024-04-25T08:42:05.858540+0000","last_fullsized":"2024-04-25T08:42:05.858540+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:43.373732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697306+0000","last_change":"2024-04-25T08:42:04.687582+0000","last_active":"2024-04-25T08:42:05.697306+0000","last_peered":"2024-04-25T08:42:05.697306+0000","last_clean":"2024-04-25T08:42:05.697306+0000","last_became_active":"2024-04-25T08:42:04.687292+0000","last_became_peered":"2024-04-25T08:42:04.687292+0000","last_unstale":"2024-04-25T08:42:05.697306+0000","last_undegraded":"2024-04-25T08:42:05.697306+0000","last_fullsized":"2024-04-25T08:42:05.697306+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:56.067119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858281+0000","last_change":"2024-04-25T08:42:03.684932+0000","last_active":"2024-04-25T08:42:05.858281+0000","last_peered":"2024-04-25T08:42:05.858281+0000","last_clean":"2024-04-25T08:42:05.858281+0000","last_became_active":"2024-04-25T08:42:03.684602+0000","last_became_peered":"2024-04-25T08:42:03.684602+0000","last_unstale":"2024-04-25T08:42:05.858281+0000","last_undegraded":"2024-04-25T08:42:05.858281+0000","last_fullsized":"2024-04-25T08:42:05.858281+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:44.041610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857934+0000","last_change":"2024-04-25T08:42:04.690073+0000","last_active":"2024-04-25T08:42:05.857934+0000","last_peered":"2024-04-25T08:42:05.857934+0000","last_clean":"2024-04-25T08:42:05.857934+0000","last_became_active":"2024-04-25T08:42:04.689314+0000","last_became_peered":"2024-04-25T08:42:04.689314+0000","last_unstale":"2024-04-25T08:42:05.857934+0000","last_undegraded":"2024-04-25T08:42:05.857934+0000","last_fullsized":"2024-04-25T08:42:05.857934+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:37.791237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696763+0000","last_change":"2024-04-25T08:42:03.686162+0000","last_active":"2024-04-25T08:42:05.696763+0000","last_peered":"2024-04-25T08:42:05.696763+0000","last_clean":"2024-04-25T08:42:05.696763+0000","last_became_active":"2024-04-25T08:42:03.685811+0000","last_became_peered":"2024-04-25T08:42:03.685811+0000","last_unstale":"2024-04-25T08:42:05.696763+0000","last_undegraded":"2024-04-25T08:42:05.696763+0000","last_fullsized":"2024-04-25T08:42:05.696763+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:37.026670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857530+0000","last_change":"2024-04-25T08:42:04.695918+0000","last_active":"2024-04-25T08:42:05.857530+0000","last_peered":"2024-04-25T08:42:05.857530+0000","last_clean":"2024-04-25T08:42:05.857530+0000","last_became_active":"2024-04-25T08:42:04.695591+0000","last_became_peered":"2024-04-25T08:42:04.695591+0000","last_unstale":"2024-04-25T08:42:05.857530+0000","last_undegraded":"2024-04-25T08:42:05.857530+0000","last_fullsized":"2024-04-25T08:42:05.857530+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:02.048229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696711+0000","last_change":"2024-04-25T08:42:03.691132+0000","last_active":"2024-04-25T08:42:05.696711+0000","last_peered":"2024-04-25T08:42:05.696711+0000","last_clean":"2024-04-25T08:42:05.696711+0000","last_became_active":"2024-04-25T08:42:03.690786+0000","last_became_peered":"2024-04-25T08:42:03.690786+0000","last_unstale":"2024-04-25T08:42:05.696711+0000","last_undegraded":"2024-04-25T08:42:05.696711+0000","last_fullsized":"2024-04-25T08:42:05.696711+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:09.366192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857055+0000","last_change":"2024-04-25T08:42:04.701729+0000","last_active":"2024-04-25T08:42:05.857055+0000","last_peered":"2024-04-25T08:42:05.857055+0000","last_clean":"2024-04-25T08:42:05.857055+0000","last_became_active":"2024-04-25T08:42:04.701456+0000","last_became_peered":"2024-04-25T08:42:04.701456+0000","last_unstale":"2024-04-25T08:42:05.857055+0000","last_undegraded":"2024-04-25T08:42:05.857055+0000","last_fullsized":"2024-04-25T08:42:05.857055+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:03.071862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"142'8","reported_seq":22,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710396+0000","last_change":"2024-04-25T08:42:03.685330+0000","last_active":"2024-04-25T08:42:05.710396+0000","last_peered":"2024-04-25T08:42:05.710396+0000","last_clean":"2024-04-25T08:42:05.710396+0000","last_became_active":"2024-04-25T08:42:03.685059+0000","last_became_peered":"2024-04-25T08:42:03.685059+0000","last_unstale":"2024-04-25T08:42:05.710396+0000","last_undegraded":"2024-04-25T08:42:05.710396+0000","last_fullsized":"2024-04-25T08:42:05.710396+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:15.799519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819079+0000","last_change":"2024-04-25T08:42:04.696341+0000","last_active":"2024-04-25T08:42:05.819079+0000","last_peered":"2024-04-25T08:42:05.819079+0000","last_clean":"2024-04-25T08:42:05.819079+0000","last_became_active":"2024-04-25T08:42:04.696061+0000","last_became_peered":"2024-04-25T08:42:04.696061+0000","last_unstale":"2024-04-25T08:42:05.819079+0000","last_undegraded":"2024-04-25T08:42:05.819079+0000","last_fullsized":"2024-04-25T08:42:05.819079+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:04.379981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857609+0000","last_change":"2024-04-25T08:42:03.691543+0000","last_active":"2024-04-25T08:42:05.857609+0000","last_peered":"2024-04-25T08:42:05.857609+0000","last_clean":"2024-04-25T08:42:05.857609+0000","last_became_active":"2024-04-25T08:42:03.691142+0000","last_became_peered":"2024-04-25T08:42:03.691142+0000","last_unstale":"2024-04-25T08:42:05.857609+0000","last_undegraded":"2024-04-25T08:42:05.857609+0000","last_fullsized":"2024-04-25T08:42:05.857609+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:00.747860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819292+0000","last_change":"2024-04-25T08:42:04.690350+0000","last_active":"2024-04-25T08:42:05.819292+0000","last_peered":"2024-04-25T08:42:05.819292+0000","last_clean":"2024-04-25T08:42:05.819292+0000","last_became_active":"2024-04-25T08:42:04.689328+0000","last_became_peered":"2024-04-25T08:42:04.689328+0000","last_unstale":"2024-04-25T08:42:05.819292+0000","last_undegraded":"2024-04-25T08:42:05.819292+0000","last_fullsized":"2024-04-25T08:42:05.819292+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:55.023285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819271+0000","last_change":"2024-04-25T08:42:03.689547+0000","last_active":"2024-04-25T08:42:05.819271+0000","last_peered":"2024-04-25T08:42:05.819271+0000","last_clean":"2024-04-25T08:42:05.819271+0000","last_became_active":"2024-04-25T08:42:03.689265+0000","last_became_peered":"2024-04-25T08:42:03.689265+0000","last_unstale":"2024-04-25T08:42:05.819271+0000","last_undegraded":"2024-04-25T08:42:05.819271+0000","last_fullsized":"2024-04-25T08:42:05.819271+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:50.560704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710813+0000","last_change":"2024-04-25T08:42:04.698298+0000","last_active":"2024-04-25T08:42:05.710813+0000","last_peered":"2024-04-25T08:42:05.710813+0000","last_clean":"2024-04-25T08:42:05.710813+0000","last_became_active":"2024-04-25T08:42:04.697777+0000","last_became_peered":"2024-04-25T08:42:04.697777+0000","last_unstale":"2024-04-25T08:42:05.710813+0000","last_undegraded":"2024-04-25T08:42:05.710813+0000","last_fullsized":"2024-04-25T08:42:05.710813+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:32.366008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857666+0000","last_change":"2024-04-25T08:42:03.690041+0000","last_active":"2024-04-25T08:42:05.857666+0000","last_peered":"2024-04-25T08:42:05.857666+0000","last_clean":"2024-04-25T08:42:05.857666+0000","last_became_active":"2024-04-25T08:42:03.689483+0000","last_became_peered":"2024-04-25T08:42:03.689483+0000","last_unstale":"2024-04-25T08:42:05.857666+0000","last_undegraded":"2024-04-25T08:42:05.857666+0000","last_fullsized":"2024-04-25T08:42:05.857666+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:55.057955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710985+0000","last_change":"2024-04-25T08:42:04.696360+0000","last_active":"2024-04-25T08:42:05.710985+0000","last_peered":"2024-04-25T08:42:05.710985+0000","last_clean":"2024-04-25T08:42:05.710985+0000","last_became_active":"2024-04-25T08:42:04.696009+0000","last_became_peered":"2024-04-25T08:42:04.696009+0000","last_unstale":"2024-04-25T08:42:05.710985+0000","last_undegraded":"2024-04-25T08:42:05.710985+0000","last_fullsized":"2024-04-25T08:42:05.710985+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:44.343322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696726+0000","last_change":"2024-04-25T08:42:03.684689+0000","last_active":"2024-04-25T08:42:05.696726+0000","last_peered":"2024-04-25T08:42:05.696726+0000","last_clean":"2024-04-25T08:42:05.696726+0000","last_became_active":"2024-04-25T08:42:03.684386+0000","last_became_peered":"2024-04-25T08:42:03.684386+0000","last_unstale":"2024-04-25T08:42:05.696726+0000","last_undegraded":"2024-04-25T08:42:05.696726+0000","last_fullsized":"2024-04-25T08:42:05.696726+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:43.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819155+0000","last_change":"2024-04-25T08:42:04.690026+0000","last_active":"2024-04-25T08:42:05.819155+0000","last_peered":"2024-04-25T08:42:05.819155+0000","last_clean":"2024-04-25T08:42:05.819155+0000","last_became_active":"2024-04-25T08:42:04.689734+0000","last_became_peered":"2024-04-25T08:42:04.689734+0000","last_unstale":"2024-04-25T08:42:05.819155+0000","last_undegraded":"2024-04-25T08:42:05.819155+0000","last_fullsized":"2024-04-25T08:42:05.819155+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:51.742834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.704235+0000","last_change":"2024-04-25T08:42:03.691691+0000","last_active":"2024-04-25T08:42:05.704235+0000","last_peered":"2024-04-25T08:42:05.704235+0000","last_clean":"2024-04-25T08:42:05.704235+0000","last_became_active":"2024-04-25T08:42:03.691380+0000","last_became_peered":"2024-04-25T08:42:03.691380+0000","last_unstale":"2024-04-25T08:42:05.704235+0000","last_undegraded":"2024-04-25T08:42:05.704235+0000","last_fullsized":"2024-04-25T08:42:05.704235+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:21.859122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858089+0000","last_change":"2024-04-25T08:42:04.690321+0000","last_active":"2024-04-25T08:42:05.858089+0000","last_peered":"2024-04-25T08:42:05.858089+0000","last_clean":"2024-04-25T08:42:05.858089+0000","last_became_active":"2024-04-25T08:42:04.689785+0000","last_became_peered":"2024-04-25T08:42:04.689785+0000","last_unstale":"2024-04-25T08:42:05.858089+0000","last_undegraded":"2024-04-25T08:42:05.858089+0000","last_fullsized":"2024-04-25T08:42:05.858089+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:48.904141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697304+0000","last_change":"2024-04-25T08:42:03.681184+0000","last_active":"2024-04-25T08:42:05.697304+0000","last_peered":"2024-04-25T08:42:05.697304+0000","last_clean":"2024-04-25T08:42:05.697304+0000","last_became_active":"2024-04-25T08:42:03.680886+0000","last_became_peered":"2024-04-25T08:42:03.680886+0000","last_unstale":"2024-04-25T08:42:05.697304+0000","last_undegraded":"2024-04-25T08:42:05.697304+0000","last_fullsized":"2024-04-25T08:42:05.697304+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:11.637696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819030+0000","last_change":"2024-04-25T08:42:04.689778+0000","last_active":"2024-04-25T08:42:05.819030+0000","last_peered":"2024-04-25T08:42:05.819030+0000","last_clean":"2024-04-25T08:42:05.819030+0000","last_became_active":"2024-04-25T08:42:04.689477+0000","last_became_peered":"2024-04-25T08:42:04.689477+0000","last_unstale":"2024-04-25T08:42:05.819030+0000","last_undegraded":"2024-04-25T08:42:05.819030+0000","last_fullsized":"2024-04-25T08:42:05.819030+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:02.381300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.703974+0000","last_change":"2024-04-25T08:42:03.692612+0000","last_active":"2024-04-25T08:42:05.703974+0000","last_peered":"2024-04-25T08:42:05.703974+0000","last_clean":"2024-04-25T08:42:05.703974+0000","last_became_active":"2024-04-25T08:42:03.692319+0000","last_became_peered":"2024-04-25T08:42:03.692319+0000","last_unstale":"2024-04-25T08:42:05.703974+0000","last_undegraded":"2024-04-25T08:42:05.703974+0000","last_fullsized":"2024-04-25T08:42:05.703974+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:32.189189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857208+0000","last_change":"2024-04-25T08:42:04.689251+0000","last_active":"2024-04-25T08:42:05.857208+0000","last_peered":"2024-04-25T08:42:05.857208+0000","last_clean":"2024-04-25T08:42:05.857208+0000","last_became_active":"2024-04-25T08:42:04.688984+0000","last_became_peered":"2024-04-25T08:42:04.688984+0000","last_unstale":"2024-04-25T08:42:05.857208+0000","last_undegraded":"2024-04-25T08:42:05.857208+0000","last_fullsized":"2024-04-25T08:42:05.857208+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:06.864814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697630+0000","last_change":"2024-04-25T08:42:03.692428+0000","last_active":"2024-04-25T08:42:05.697630+0000","last_peered":"2024-04-25T08:42:05.697630+0000","last_clean":"2024-04-25T08:42:05.697630+0000","last_became_active":"2024-04-25T08:42:03.692124+0000","last_became_peered":"2024-04-25T08:42:03.692124+0000","last_unstale":"2024-04-25T08:42:05.697630+0000","last_undegraded":"2024-04-25T08:42:05.697630+0000","last_fullsized":"2024-04-25T08:42:05.697630+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:37.154217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858597+0000","last_change":"2024-04-25T08:42:04.692333+0000","last_active":"2024-04-25T08:42:05.858597+0000","last_peered":"2024-04-25T08:42:05.858597+0000","last_clean":"2024-04-25T08:42:05.858597+0000","last_became_active":"2024-04-25T08:42:04.692056+0000","last_became_peered":"2024-04-25T08:42:04.692056+0000","last_unstale":"2024-04-25T08:42:05.858597+0000","last_undegraded":"2024-04-25T08:42:05.858597+0000","last_fullsized":"2024-04-25T08:42:05.858597+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:55.023285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696612+0000","last_change":"2024-04-25T08:42:03.685051+0000","last_active":"2024-04-25T08:42:05.696612+0000","last_peered":"2024-04-25T08:42:05.696612+0000","last_clean":"2024-04-25T08:42:05.696612+0000","last_became_active":"2024-04-25T08:42:03.684598+0000","last_became_peered":"2024-04-25T08:42:03.684598+0000","last_unstale":"2024-04-25T08:42:05.696612+0000","last_undegraded":"2024-04-25T08:42:05.696612+0000","last_fullsized":"2024-04-25T08:42:05.696612+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:15.799519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696539+0000","last_change":"2024-04-25T08:42:04.703530+0000","last_active":"2024-04-25T08:42:05.696539+0000","last_peered":"2024-04-25T08:42:05.696539+0000","last_clean":"2024-04-25T08:42:05.696539+0000","last_became_active":"2024-04-25T08:42:04.703243+0000","last_became_peered":"2024-04-25T08:42:04.703243+0000","last_unstale":"2024-04-25T08:42:05.696539+0000","last_undegraded":"2024-04-25T08:42:05.696539+0000","last_fullsized":"2024-04-25T08:42:05.696539+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:10.418523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"143'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:06.677474+0000","last_change":"2024-04-25T08:42:03.686672+0000","last_active":"2024-04-25T08:42:06.677474+0000","last_peered":"2024-04-25T08:42:06.677474+0000","last_clean":"2024-04-25T08:42:06.677474+0000","last_became_active":"2024-04-25T08:42:03.686415+0000","last_became_peered":"2024-04-25T08:42:03.686415+0000","last_unstale":"2024-04-25T08:42:06.677474+0000","last_undegraded":"2024-04-25T08:42:06.677474+0000","last_fullsized":"2024-04-25T08:42:06.677474+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:00.379501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710922+0000","last_change":"2024-04-25T08:42:04.702305+0000","last_active":"2024-04-25T08:42:05.710922+0000","last_peered":"2024-04-25T08:42:05.710922+0000","last_clean":"2024-04-25T08:42:05.710922+0000","last_became_active":"2024-04-25T08:42:04.701290+0000","last_became_peered":"2024-04-25T08:42:04.701290+0000","last_unstale":"2024-04-25T08:42:05.710922+0000","last_undegraded":"2024-04-25T08:42:05.710922+0000","last_fullsized":"2024-04-25T08:42:05.710922+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:38.536326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857739+0000","last_change":"2024-04-25T08:42:03.690264+0000","last_active":"2024-04-25T08:42:05.857739+0000","last_peered":"2024-04-25T08:42:05.857739+0000","last_clean":"2024-04-25T08:42:05.857739+0000","last_became_active":"2024-04-25T08:42:03.689789+0000","last_became_peered":"2024-04-25T08:42:03.689789+0000","last_unstale":"2024-04-25T08:42:05.857739+0000","last_undegraded":"2024-04-25T08:42:05.857739+0000","last_fullsized":"2024-04-25T08:42:05.857739+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:07.248828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858291+0000","last_change":"2024-04-25T08:42:04.700343+0000","last_active":"2024-04-25T08:42:05.858291+0000","last_peered":"2024-04-25T08:42:05.858291+0000","last_clean":"2024-04-25T08:42:05.858291+0000","last_became_active":"2024-04-25T08:42:04.700052+0000","last_became_peered":"2024-04-25T08:42:04.700052+0000","last_unstale":"2024-04-25T08:42:05.858291+0000","last_undegraded":"2024-04-25T08:42:05.858291+0000","last_fullsized":"2024-04-25T08:42:05.858291+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:04.379981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858645+0000","last_change":"2024-04-25T08:42:03.690696+0000","last_active":"2024-04-25T08:42:05.858645+0000","last_peered":"2024-04-25T08:42:05.858645+0000","last_clean":"2024-04-25T08:42:05.858645+0000","last_became_active":"2024-04-25T08:42:03.690383+0000","last_became_peered":"2024-04-25T08:42:03.690383+0000","last_unstale":"2024-04-25T08:42:05.858645+0000","last_undegraded":"2024-04-25T08:42:05.858645+0000","last_fullsized":"2024-04-25T08:42:05.858645+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:20.395545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697511+0000","last_change":"2024-04-25T08:42:04.705552+0000","last_active":"2024-04-25T08:42:05.697511+0000","last_peered":"2024-04-25T08:42:05.697511+0000","last_clean":"2024-04-25T08:42:05.697511+0000","last_became_active":"2024-04-25T08:42:04.705278+0000","last_became_peered":"2024-04-25T08:42:04.705278+0000","last_unstale":"2024-04-25T08:42:05.697511+0000","last_undegraded":"2024-04-25T08:42:05.697511+0000","last_fullsized":"2024-04-25T08:42:05.697511+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:37.298009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710684+0000","last_change":"2024-04-25T08:42:03.688933+0000","last_active":"2024-04-25T08:42:05.710684+0000","last_peered":"2024-04-25T08:42:05.710684+0000","last_clean":"2024-04-25T08:42:05.710684+0000","last_became_active":"2024-04-25T08:42:03.688608+0000","last_became_peered":"2024-04-25T08:42:03.688608+0000","last_unstale":"2024-04-25T08:42:05.710684+0000","last_undegraded":"2024-04-25T08:42:05.710684+0000","last_fullsized":"2024-04-25T08:42:05.710684+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:10.585956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696269+0000","last_change":"2024-04-25T08:42:04.700466+0000","last_active":"2024-04-25T08:42:05.696269+0000","last_peered":"2024-04-25T08:42:05.696269+0000","last_clean":"2024-04-25T08:42:05.696269+0000","last_became_active":"2024-04-25T08:42:04.700164+0000","last_became_peered":"2024-04-25T08:42:04.700164+0000","last_unstale":"2024-04-25T08:42:05.696269+0000","last_undegraded":"2024-04-25T08:42:05.696269+0000","last_fullsized":"2024-04-25T08:42:05.696269+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:56.306539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858415+0000","last_change":"2024-04-25T08:42:03.689840+0000","last_active":"2024-04-25T08:42:05.858415+0000","last_peered":"2024-04-25T08:42:05.858415+0000","last_clean":"2024-04-25T08:42:05.858415+0000","last_became_active":"2024-04-25T08:42:03.689351+0000","last_became_peered":"2024-04-25T08:42:03.689351+0000","last_unstale":"2024-04-25T08:42:05.858415+0000","last_undegraded":"2024-04-25T08:42:05.858415+0000","last_fullsized":"2024-04-25T08:42:05.858415+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:24.560726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858382+0000","last_change":"2024-04-25T08:42:04.698200+0000","last_active":"2024-04-25T08:42:05.858382+0000","last_peered":"2024-04-25T08:42:05.858382+0000","last_clean":"2024-04-25T08:42:05.858382+0000","last_became_active":"2024-04-25T08:42:04.697935+0000","last_became_peered":"2024-04-25T08:42:04.697935+0000","last_unstale":"2024-04-25T08:42:05.858382+0000","last_undegraded":"2024-04-25T08:42:05.858382+0000","last_fullsized":"2024-04-25T08:42:05.858382+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:18.372469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710694+0000","last_change":"2024-04-25T08:42:03.689116+0000","last_active":"2024-04-25T08:42:05.710694+0000","last_peered":"2024-04-25T08:42:05.710694+0000","last_clean":"2024-04-25T08:42:05.710694+0000","last_became_active":"2024-04-25T08:42:03.688471+0000","last_became_peered":"2024-04-25T08:42:03.688471+0000","last_unstale":"2024-04-25T08:42:05.710694+0000","last_undegraded":"2024-04-25T08:42:05.710694+0000","last_fullsized":"2024-04-25T08:42:05.710694+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:04.668269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697405+0000","last_change":"2024-04-25T08:42:04.699247+0000","last_active":"2024-04-25T08:42:05.697405+0000","last_peered":"2024-04-25T08:42:05.697405+0000","last_clean":"2024-04-25T08:42:05.697405+0000","last_became_active":"2024-04-25T08:42:04.698880+0000","last_became_peered":"2024-04-25T08:42:04.698880+0000","last_unstale":"2024-04-25T08:42:05.697405+0000","last_undegraded":"2024-04-25T08:42:05.697405+0000","last_fullsized":"2024-04-25T08:42:05.697405+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:08.257992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.818902+0000","last_change":"2024-04-25T08:42:03.688861+0000","last_active":"2024-04-25T08:42:05.818902+0000","last_peered":"2024-04-25T08:42:05.818902+0000","last_clean":"2024-04-25T08:42:05.818902+0000","last_became_active":"2024-04-25T08:42:03.688492+0000","last_became_peered":"2024-04-25T08:42:03.688492+0000","last_unstale":"2024-04-25T08:42:05.818902+0000","last_undegraded":"2024-04-25T08:42:05.818902+0000","last_fullsized":"2024-04-25T08:42:05.818902+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.900040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697909+0000","last_change":"2024-04-25T08:42:04.689183+0000","last_active":"2024-04-25T08:42:05.697909+0000","last_peered":"2024-04-25T08:42:05.697909+0000","last_clean":"2024-04-25T08:42:05.697909+0000","last_became_active":"2024-04-25T08:42:04.688902+0000","last_became_peered":"2024-04-25T08:42:04.688902+0000","last_unstale":"2024-04-25T08:42:05.697909+0000","last_undegraded":"2024-04-25T08:42:05.697909+0000","last_fullsized":"2024-04-25T08:42:05.697909+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:08.507867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"142'6","reported_seq":20,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.704238+0000","last_change":"2024-04-25T08:42:03.691985+0000","last_active":"2024-04-25T08:42:05.704238+0000","last_peered":"2024-04-25T08:42:05.704238+0000","last_clean":"2024-04-25T08:42:05.704238+0000","last_became_active":"2024-04-25T08:42:03.691680+0000","last_became_peered":"2024-04-25T08:42:03.691680+0000","last_unstale":"2024-04-25T08:42:05.704238+0000","last_undegraded":"2024-04-25T08:42:05.704238+0000","last_fullsized":"2024-04-25T08:42:05.704238+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:53.140435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":401,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.818984+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:42:05.818984+0000","last_peered":"2024-04-25T08:42:05.818984+0000","last_clean":"2024-04-25T08:42:05.818984+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:42:05.818984+0000","last_undegraded":"2024-04-25T08:42:05.818984+0000","last_fullsized":"2024-04-25T08:42:05.818984+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69612,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302228,"statfs":{"total":96636764160,"available":96565481472,"internally_reserved":0,"allocated":2392064,"data_stored":1430648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46832,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325008,"statfs":{"total":96636764160,"available":96588808192,"internally_reserved":0,"allocated":1798144,"data_stored":840328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59300,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312540,"statfs":{"total":96636764160,"available":96576040960,"internally_reserved":0,"allocated":1794048,"data_stored":840258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73200,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298640,"statfs":{"total":96636764160,"available":96561807360,"internally_reserved":0,"allocated":1806336,"data_stored":841870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77364,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294476,"statfs":{"total":96636764160,"available":96557543424,"internally_reserved":0,"allocated":1802240,"data_stored":840874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46828,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325012,"statfs":{"total":96636764160,"available":96588812288,"internally_reserved":0,"allocated":1802240,"data_stored":840370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59312,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312528,"statfs":{"total":96636764160,"available":96576028672,"internally_reserved":0,"allocated":1798144,"data_stored":840328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69568,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302272,"statfs":{"total":96636764160,"available":96565526528,"internally_reserved":0,"allocated":2412544,"data_stored":1432944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:42:07.682 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:42:07.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.826+0000 7fb4ff4db640 1 Processor -- start 2024-04-25T08:42:07.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.827+0000 7fb4ff4db640 1 -- start start 2024-04-25T08:42:07.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.827+0000 7fb4ff4db640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb4f80d0f60 0x7fb4f80b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.827+0000 7fb4ff4db640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f80d3e70 con 0x7fb4f80d3900 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.827+0000 7fb4ff4db640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f80d4150 con 0x7fb4f815dc10 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.827+0000 7fb4ff4db640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f815dff0 con 0x7fb4f80d0f60 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4fca76640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb4f80d3900 0x7fb4f80d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36288/0 (socket says 172.21.15.38:36288) 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4fca76640 1 -- 172.21.15.38:0/1564842848 learned_addr learned my addr 172.21.15.38:0/1564842848 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462067710 0 0) 0x7fb4f80d3e70 con 0x7fb4f80d3900 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4fda78640 1 --2- 172.21.15.38:0/1564842848 >> v2:172.21.15.107:3300/0 conn(0x7fb4f80d0f60 0x7fb4f80b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4e4003520 con 0x7fb4f80d3900 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1131099235 0 0) 0x7fb4e4003520 con 0x7fb4f80d3900 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 >> v2:172.21.15.107:3300/0 conn(0x7fb4f80d0f60 msgr2=0x7fb4f80b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 --2- 172.21.15.38:0/1564842848 >> v2:172.21.15.107:3300/0 conn(0x7fb4f80d0f60 0x7fb4f80b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 >> v1:172.21.15.107:6789/0 conn(0x7fb4f815dc10 legacy=0x7fb4f80ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f80d4430 con 0x7fb4f80d3900 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fb4e8001670 con 0x7fb4f80d3900 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 >> v1:172.21.15.38:6789/0 conn(0x7fb4f80d3900 legacy=0x7fb4f80d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 --2- 172.21.15.38:0/1564842848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4e40039b0 0x7fb4e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 --2- 172.21.15.38:0/1564842848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e4004800 0x7fb4e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 --2- 172.21.15.38:0/1564842848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4e40051c0 0x7fb4e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f815dff0 con 0x7fb4e4004800 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e400c160 con 0x7fb4e40039b0 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e400c360 con 0x7fb4e40051c0 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4fca76640 1 --2- 172.21.15.38:0/1564842848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e4004800 0x7fb4e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4fd277640 1 --2- 172.21.15.38:0/1564842848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4e40039b0 0x7fb4e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.828+0000 7fb4fda78640 1 --2- 172.21.15.38:0/1564842848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4e40051c0 0x7fb4e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.829+0000 7fb4fca76640 1 -- 172.21.15.38:0/1564842848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4e40051c0 msgr2=0x7fb4e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.829+0000 7fb4fca76640 1 --2- 172.21.15.38:0/1564842848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4e40051c0 0x7fb4e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.829+0000 7fb4fca76640 1 -- 172.21.15.38:0/1564842848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4e40039b0 msgr2=0x7fb4e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:07.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.829+0000 7fb4fca76640 1 --2- 172.21.15.38:0/1564842848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4e40039b0 0x7fb4e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.829+0000 7fb4fca76640 1 -- 172.21.15.38:0/1564842848 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e4003520 con 0x7fb4e4004800 2024-04-25T08:42:07.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.829+0000 7fb4fda78640 1 -- 172.21.15.38:0/1564842848 reap_dead start 2024-04-25T08:42:07.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.829+0000 7fb4fca76640 1 --2- 172.21.15.38:0/1564842848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e4004800 0x7fb4e4004c80 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800c240 tx=0x7fb4e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0bd01972e98d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:07.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.829+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8007cf0 con 0x7fb4e4004800 2024-04-25T08:42:07.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.829+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e8005af0 con 0x7fb4e4004800 2024-04-25T08:42:07.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.829+0000 7fb4effff640 1 -- 172.21.15.38:0/1564842848 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8007070 con 0x7fb4e4004800 2024-04-25T08:42:07.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.830+0000 7fb4ff4db640 1 -- 172.21.15.38:0/1564842848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e4004800 msgr2=0x7fb4e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.830+0000 7fb4ff4db640 1 --2- 172.21.15.38:0/1564842848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e4004800 0x7fb4e4004c80 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fb4e800c240 tx=0x7fb4e4005830 comp rx=0 tx=0).stop 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.830+0000 7fb4ff4db640 1 -- 172.21.15.38:0/1564842848 shutdown_connections 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.830+0000 7fb4ff4db640 1 --2- 172.21.15.38:0/1564842848 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4e4004800 0x7fb4e4004c80 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.830+0000 7fb4ff4db640 1 -- 172.21.15.38:0/1564842848 >> 172.21.15.38:0/1564842848 conn(0x7fb4f8066d10 msgr2=0x7fb4f8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.830+0000 7fb4ff4db640 1 -- 172.21.15.38:0/1564842848 shutdown_connections 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.830+0000 7fb4ff4db640 1 -- 172.21.15.38:0/1564842848 wait complete. 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.831+0000 7fb4ff4db640 1 Processor -- start 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.831+0000 7fb4ff4db640 1 -- start start 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4ff4db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4e4004800 0x7fb4f80abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4ff4db640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4f80c9690 0x7fb4f81f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4ff4db640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4f80c9af0 0x7fb4f81f7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4ff4db640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f80d4710 con 0x7fb4f80c9690 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4ff4db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f80d0850 con 0x7fb4e4004800 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4ff4db640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f80d09d0 con 0x7fb4f80c9af0 2024-04-25T08:42:07.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4fca76640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4f80c9690 0x7fb4f81f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4fca76640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4f80c9690 0x7fb4f81f6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52058/0 (socket says 172.21.15.38:52058) 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4fca76640 1 -- 172.21.15.38:0/3253114160 learned_addr learned my addr 172.21.15.38:0/3253114160 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4fda78640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4f80c9af0 0x7fb4f81f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4fd277640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4e4004800 0x7fb4f80abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4fca76640 1 -- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4f80c9af0 msgr2=0x7fb4f81f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4fca76640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4f80c9af0 0x7fb4f81f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4fca76640 1 -- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4e4004800 msgr2=0x7fb4f80abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4fca76640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4e4004800 0x7fb4f80abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.832+0000 7fb4fca76640 1 -- 172.21.15.38:0/3253114160 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f80ac510 con 0x7fb4f80c9690 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.833+0000 7fb4fca76640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4f80c9690 0x7fb4f81f6ac0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8001970 tx=0x7fb4f80d4600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.833+0000 7fb4edffb640 1 -- 172.21.15.38:0/3253114160 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8037040 con 0x7fb4f80c9690 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.833+0000 7fb4edffb640 1 -- 172.21.15.38:0/3253114160 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e8041070 con 0x7fb4f80c9690 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.833+0000 7fb4ff4db640 1 -- 172.21.15.38:0/3253114160 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f80c8070 con 0x7fb4f80c9690 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.833+0000 7fb4edffb640 1 -- 172.21.15.38:0/3253114160 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e8001670 con 0x7fb4f80c9690 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.833+0000 7fb4ff4db640 1 -- 172.21.15.38:0/3253114160 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f80c8630 con 0x7fb4f80c9690 2024-04-25T08:42:07.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.834+0000 7fb4ff4db640 1 -- 172.21.15.38:0/3253114160 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f8068e80 con 0x7fb4f80c9690 2024-04-25T08:42:07.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.834+0000 7fb4edffb640 1 -- 172.21.15.38:0/3253114160 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4e8004a00 con 0x7fb4f80c9690 2024-04-25T08:42:07.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.835+0000 7fb4edffb640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb4dc06b4d0 0x7fb4dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:07.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.835+0000 7fb4edffb640 1 -- 172.21.15.38:0/3253114160 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb4e80a9ee0 con 0x7fb4f80c9690 2024-04-25T08:42:07.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.835+0000 7fb4fd277640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb4dc06b4d0 0x7fb4dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:07.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.836+0000 7fb4fd277640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb4dc06b4d0 0x7fb4dc06d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4000c00 tx=0x7fb4f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:07.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:07.839+0000 7fb4edffb640 1 -- 172.21.15.38:0/3253114160 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4e8078580 con 0x7fb4f80c9690 2024-04-25T08:42:08.006 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.014+0000 7fb4ff4db640 1 -- 172.21.15.38:0/3253114160 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4f80c7e20 con 0x7fb4dc06b4d0 2024-04-25T08:42:08.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.020+0000 7fb4edffb640 1 -- 172.21.15.38:0/3253114160 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7fb4f80c7e20 con 0x7fb4dc06b4d0 2024-04-25T08:42:08.015 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:08.016 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:42:08.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.023+0000 7fb4ff4db640 1 -- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb4dc06b4d0 msgr2=0x7fb4dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.023+0000 7fb4ff4db640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb4dc06b4d0 0x7fb4dc06d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4000c00 tx=0x7fb4f4009040 comp rx=0 tx=0).stop 2024-04-25T08:42:08.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.023+0000 7fb4ff4db640 1 -- 172.21.15.38:0/3253114160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4f80c9690 msgr2=0x7fb4f81f6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.023+0000 7fb4ff4db640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4f80c9690 0x7fb4f81f6ac0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8001970 tx=0x7fb4f80d4600 comp rx=0 tx=0).stop 2024-04-25T08:42:08.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.023+0000 7fb4ff4db640 1 -- 172.21.15.38:0/3253114160 shutdown_connections 2024-04-25T08:42:08.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.023+0000 7fb4ff4db640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb4dc06b4d0 0x7fb4dc06d990 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.023+0000 7fb4ff4db640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb4f80c9af0 0x7fb4f81f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.023+0000 7fb4ff4db640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4f80c9690 0x7fb4f81f6ac0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.023+0000 7fb4ff4db640 1 --2- 172.21.15.38:0/3253114160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4e4004800 0x7fb4f80abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.023+0000 7fb4ff4db640 1 -- 172.21.15.38:0/3253114160 >> 172.21.15.38:0/3253114160 conn(0x7fb4f8066d10 msgr2=0x7fb4f81605e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:08.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.023+0000 7fb4ff4db640 1 -- 172.21.15.38:0/3253114160 shutdown_connections 2024-04-25T08:42:08.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.023+0000 7fb4ff4db640 1 -- 172.21.15.38:0/3253114160 wait complete. 2024-04-25T08:42:08.027 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-04-25T08:42:07.353694+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":502016,"kb_used_data":15240,"kb_used_omap":12,"kb_used_meta":486643,"kb_avail":754472704,"statfs":{"total":773094113280,"available":772580048896,"internally_reserved":0,"allocated":15605760,"data_stored":7907620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":498322992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.688389"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710844+0000","last_change":"2024-04-25T08:42:04.701156+0000","last_active":"2024-04-25T08:42:05.710844+0000","last_peered":"2024-04-25T08:42:05.710844+0000","last_clean":"2024-04-25T08:42:05.710844+0000","last_became_active":"2024-04-25T08:42:04.700864+0000","last_became_peered":"2024-04-25T08:42:04.700864+0000","last_unstale":"2024-04-25T08:42:05.710844+0000","last_undegraded":"2024-04-25T08:42:05.710844+0000","last_fullsized":"2024-04-25T08:42:05.710844+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:03.071862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857054+0000","last_change":"2024-04-25T08:42:04.688967+0000","last_active":"2024-04-25T08:42:05.857054+0000","last_peered":"2024-04-25T08:42:05.857054+0000","last_clean":"2024-04-25T08:42:05.857054+0000","last_became_active":"2024-04-25T08:42:04.688684+0000","last_became_peered":"2024-04-25T08:42:04.688684+0000","last_unstale":"2024-04-25T08:42:05.857054+0000","last_undegraded":"2024-04-25T08:42:05.857054+0000","last_fullsized":"2024-04-25T08:42:05.857054+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:35.361036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857554+0000","last_change":"2024-04-25T08:42:04.696220+0000","last_active":"2024-04-25T08:42:05.857554+0000","last_peered":"2024-04-25T08:42:05.857554+0000","last_clean":"2024-04-25T08:42:05.857554+0000","last_became_active":"2024-04-25T08:42:04.695935+0000","last_became_peered":"2024-04-25T08:42:04.695935+0000","last_unstale":"2024-04-25T08:42:05.857554+0000","last_undegraded":"2024-04-25T08:42:05.857554+0000","last_fullsized":"2024-04-25T08:42:05.857554+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:44.343322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696648+0000","last_change":"2024-04-25T08:42:04.692485+0000","last_active":"2024-04-25T08:42:05.696648+0000","last_peered":"2024-04-25T08:42:05.696648+0000","last_clean":"2024-04-25T08:42:05.696648+0000","last_became_active":"2024-04-25T08:42:04.692191+0000","last_became_peered":"2024-04-25T08:42:04.692191+0000","last_unstale":"2024-04-25T08:42:05.696648+0000","last_undegraded":"2024-04-25T08:42:05.696648+0000","last_fullsized":"2024-04-25T08:42:05.696648+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:11.489900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858169+0000","last_change":"2024-04-25T08:42:04.697680+0000","last_active":"2024-04-25T08:42:05.858169+0000","last_peered":"2024-04-25T08:42:05.858169+0000","last_clean":"2024-04-25T08:42:05.858169+0000","last_became_active":"2024-04-25T08:42:04.697415+0000","last_became_peered":"2024-04-25T08:42:04.697415+0000","last_unstale":"2024-04-25T08:42:05.858169+0000","last_undegraded":"2024-04-25T08:42:05.858169+0000","last_fullsized":"2024-04-25T08:42:05.858169+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:47:18.109378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857384+0000","last_change":"2024-04-25T08:42:04.694813+0000","last_active":"2024-04-25T08:42:05.857384+0000","last_peered":"2024-04-25T08:42:05.857384+0000","last_clean":"2024-04-25T08:42:05.857384+0000","last_became_active":"2024-04-25T08:42:04.694560+0000","last_became_peered":"2024-04-25T08:42:04.694560+0000","last_unstale":"2024-04-25T08:42:05.857384+0000","last_undegraded":"2024-04-25T08:42:05.857384+0000","last_fullsized":"2024-04-25T08:42:05.857384+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:18.372469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710620+0000","last_change":"2024-04-25T08:42:04.697115+0000","last_active":"2024-04-25T08:42:05.710620+0000","last_peered":"2024-04-25T08:42:05.710620+0000","last_clean":"2024-04-25T08:42:05.710620+0000","last_became_active":"2024-04-25T08:42:04.696809+0000","last_became_peered":"2024-04-25T08:42:04.696809+0000","last_unstale":"2024-04-25T08:42:05.710620+0000","last_undegraded":"2024-04-25T08:42:05.710620+0000","last_fullsized":"2024-04-25T08:42:05.710620+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:30.359852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697880+0000","last_change":"2024-04-25T08:42:04.699128+0000","last_active":"2024-04-25T08:42:05.697880+0000","last_peered":"2024-04-25T08:42:05.697880+0000","last_clean":"2024-04-25T08:42:05.697880+0000","last_became_active":"2024-04-25T08:42:04.698474+0000","last_became_peered":"2024-04-25T08:42:04.698474+0000","last_unstale":"2024-04-25T08:42:05.697880+0000","last_undegraded":"2024-04-25T08:42:05.697880+0000","last_fullsized":"2024-04-25T08:42:05.697880+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:45.050773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858145+0000","last_change":"2024-04-25T08:42:04.693012+0000","last_active":"2024-04-25T08:42:05.858145+0000","last_peered":"2024-04-25T08:42:05.858145+0000","last_clean":"2024-04-25T08:42:05.858145+0000","last_became_active":"2024-04-25T08:42:04.692744+0000","last_became_peered":"2024-04-25T08:42:04.692744+0000","last_unstale":"2024-04-25T08:42:05.858145+0000","last_undegraded":"2024-04-25T08:42:05.858145+0000","last_fullsized":"2024-04-25T08:42:05.858145+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:56:59.786563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858025+0000","last_change":"2024-04-25T08:42:04.706570+0000","last_active":"2024-04-25T08:42:05.858025+0000","last_peered":"2024-04-25T08:42:05.858025+0000","last_clean":"2024-04-25T08:42:05.858025+0000","last_became_active":"2024-04-25T08:42:04.706292+0000","last_became_peered":"2024-04-25T08:42:04.706292+0000","last_unstale":"2024-04-25T08:42:05.858025+0000","last_undegraded":"2024-04-25T08:42:05.858025+0000","last_fullsized":"2024-04-25T08:42:05.858025+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:09.978775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858182+0000","last_change":"2024-04-25T08:42:04.695428+0000","last_active":"2024-04-25T08:42:05.858182+0000","last_peered":"2024-04-25T08:42:05.858182+0000","last_clean":"2024-04-25T08:42:05.858182+0000","last_became_active":"2024-04-25T08:42:04.694914+0000","last_became_peered":"2024-04-25T08:42:04.694914+0000","last_unstale":"2024-04-25T08:42:05.858182+0000","last_undegraded":"2024-04-25T08:42:05.858182+0000","last_fullsized":"2024-04-25T08:42:05.858182+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:36.004116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696620+0000","last_change":"2024-04-25T08:42:04.701163+0000","last_active":"2024-04-25T08:42:05.696620+0000","last_peered":"2024-04-25T08:42:05.696620+0000","last_clean":"2024-04-25T08:42:05.696620+0000","last_became_active":"2024-04-25T08:42:04.700877+0000","last_became_peered":"2024-04-25T08:42:04.700877+0000","last_unstale":"2024-04-25T08:42:05.696620+0000","last_undegraded":"2024-04-25T08:42:05.696620+0000","last_fullsized":"2024-04-25T08:42:05.696620+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:45.050773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858405+0000","last_change":"2024-04-25T08:42:04.707187+0000","last_active":"2024-04-25T08:42:05.858405+0000","last_peered":"2024-04-25T08:42:05.858405+0000","last_clean":"2024-04-25T08:42:05.858405+0000","last_became_active":"2024-04-25T08:42:04.706793+0000","last_became_peered":"2024-04-25T08:42:04.706793+0000","last_unstale":"2024-04-25T08:42:05.858405+0000","last_undegraded":"2024-04-25T08:42:05.858405+0000","last_fullsized":"2024-04-25T08:42:05.858405+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:51.742834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696675+0000","last_change":"2024-04-25T08:42:04.701329+0000","last_active":"2024-04-25T08:42:05.696675+0000","last_peered":"2024-04-25T08:42:05.696675+0000","last_clean":"2024-04-25T08:42:05.696675+0000","last_became_active":"2024-04-25T08:42:04.700872+0000","last_became_peered":"2024-04-25T08:42:04.700872+0000","last_unstale":"2024-04-25T08:42:05.696675+0000","last_undegraded":"2024-04-25T08:42:05.696675+0000","last_fullsized":"2024-04-25T08:42:05.696675+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:08.507867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857787+0000","last_change":"2024-04-25T08:42:04.694008+0000","last_active":"2024-04-25T08:42:05.857787+0000","last_peered":"2024-04-25T08:42:05.857787+0000","last_clean":"2024-04-25T08:42:05.857787+0000","last_became_active":"2024-04-25T08:42:04.693712+0000","last_became_peered":"2024-04-25T08:42:04.693712+0000","last_unstale":"2024-04-25T08:42:05.857787+0000","last_undegraded":"2024-04-25T08:42:05.857787+0000","last_fullsized":"2024-04-25T08:42:05.857787+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:17.198629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857899+0000","last_change":"2024-04-25T08:42:04.698772+0000","last_active":"2024-04-25T08:42:05.857899+0000","last_peered":"2024-04-25T08:42:05.857899+0000","last_clean":"2024-04-25T08:42:05.857899+0000","last_became_active":"2024-04-25T08:42:04.698492+0000","last_became_peered":"2024-04-25T08:42:04.698492+0000","last_unstale":"2024-04-25T08:42:05.857899+0000","last_undegraded":"2024-04-25T08:42:05.857899+0000","last_fullsized":"2024-04-25T08:42:05.857899+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:58.280649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696839+0000","last_change":"2024-04-25T08:42:04.706496+0000","last_active":"2024-04-25T08:42:05.696839+0000","last_peered":"2024-04-25T08:42:05.696839+0000","last_clean":"2024-04-25T08:42:05.696839+0000","last_became_active":"2024-04-25T08:42:04.706181+0000","last_became_peered":"2024-04-25T08:42:04.706181+0000","last_unstale":"2024-04-25T08:42:05.696839+0000","last_undegraded":"2024-04-25T08:42:05.696839+0000","last_fullsized":"2024-04-25T08:42:05.696839+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:21.404708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857890+0000","last_change":"2024-04-25T08:42:04.699573+0000","last_active":"2024-04-25T08:42:05.857890+0000","last_peered":"2024-04-25T08:42:05.857890+0000","last_clean":"2024-04-25T08:42:05.857890+0000","last_became_active":"2024-04-25T08:42:04.699136+0000","last_became_peered":"2024-04-25T08:42:04.699136+0000","last_unstale":"2024-04-25T08:42:05.857890+0000","last_undegraded":"2024-04-25T08:42:05.857890+0000","last_fullsized":"2024-04-25T08:42:05.857890+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:50.797871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819307+0000","last_change":"2024-04-25T08:42:04.692192+0000","last_active":"2024-04-25T08:42:05.819307+0000","last_peered":"2024-04-25T08:42:05.819307+0000","last_clean":"2024-04-25T08:42:05.819307+0000","last_became_active":"2024-04-25T08:42:04.690390+0000","last_became_peered":"2024-04-25T08:42:04.690390+0000","last_unstale":"2024-04-25T08:42:05.819307+0000","last_undegraded":"2024-04-25T08:42:05.819307+0000","last_fullsized":"2024-04-25T08:42:05.819307+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:12.205286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.704131+0000","last_change":"2024-04-25T08:42:04.701325+0000","last_active":"2024-04-25T08:42:05.704131+0000","last_peered":"2024-04-25T08:42:05.704131+0000","last_clean":"2024-04-25T08:42:05.704131+0000","last_became_active":"2024-04-25T08:42:04.700994+0000","last_became_peered":"2024-04-25T08:42:04.700994+0000","last_unstale":"2024-04-25T08:42:05.704131+0000","last_undegraded":"2024-04-25T08:42:05.704131+0000","last_fullsized":"2024-04-25T08:42:05.704131+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:51.569719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858001+0000","last_change":"2024-04-25T08:42:04.691242+0000","last_active":"2024-04-25T08:42:05.858001+0000","last_peered":"2024-04-25T08:42:05.858001+0000","last_clean":"2024-04-25T08:42:05.858001+0000","last_became_active":"2024-04-25T08:42:04.690944+0000","last_became_peered":"2024-04-25T08:42:04.690944+0000","last_unstale":"2024-04-25T08:42:05.858001+0000","last_undegraded":"2024-04-25T08:42:05.858001+0000","last_fullsized":"2024-04-25T08:42:05.858001+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:20.385308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710602+0000","last_change":"2024-04-25T08:42:04.697169+0000","last_active":"2024-04-25T08:42:05.710602+0000","last_peered":"2024-04-25T08:42:05.710602+0000","last_clean":"2024-04-25T08:42:05.710602+0000","last_became_active":"2024-04-25T08:42:04.696887+0000","last_became_peered":"2024-04-25T08:42:04.696887+0000","last_unstale":"2024-04-25T08:42:05.710602+0000","last_undegraded":"2024-04-25T08:42:05.710602+0000","last_fullsized":"2024-04-25T08:42:05.710602+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:09.484280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697094+0000","last_change":"2024-04-25T08:42:04.690437+0000","last_active":"2024-04-25T08:42:05.697094+0000","last_peered":"2024-04-25T08:42:05.697094+0000","last_clean":"2024-04-25T08:42:05.697094+0000","last_became_active":"2024-04-25T08:42:04.690146+0000","last_became_peered":"2024-04-25T08:42:04.690146+0000","last_unstale":"2024-04-25T08:42:05.697094+0000","last_undegraded":"2024-04-25T08:42:05.697094+0000","last_fullsized":"2024-04-25T08:42:05.697094+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:01.388665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857435+0000","last_change":"2024-04-25T08:42:04.692824+0000","last_active":"2024-04-25T08:42:05.857435+0000","last_peered":"2024-04-25T08:42:05.857435+0000","last_clean":"2024-04-25T08:42:05.857435+0000","last_became_active":"2024-04-25T08:42:04.692548+0000","last_became_peered":"2024-04-25T08:42:04.692548+0000","last_unstale":"2024-04-25T08:42:05.857435+0000","last_undegraded":"2024-04-25T08:42:05.857435+0000","last_fullsized":"2024-04-25T08:42:05.857435+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:59.373510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710885+0000","last_change":"2024-04-25T08:42:04.706510+0000","last_active":"2024-04-25T08:42:05.710885+0000","last_peered":"2024-04-25T08:42:05.710885+0000","last_clean":"2024-04-25T08:42:05.710885+0000","last_became_active":"2024-04-25T08:42:04.706178+0000","last_became_peered":"2024-04-25T08:42:04.706178+0000","last_unstale":"2024-04-25T08:42:05.710885+0000","last_undegraded":"2024-04-25T08:42:05.710885+0000","last_fullsized":"2024-04-25T08:42:05.710885+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:06.864814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710792+0000","last_change":"2024-04-25T08:42:04.695194+0000","last_active":"2024-04-25T08:42:05.710792+0000","last_peered":"2024-04-25T08:42:05.710792+0000","last_clean":"2024-04-25T08:42:05.710792+0000","last_became_active":"2024-04-25T08:42:04.694890+0000","last_became_peered":"2024-04-25T08:42:04.694890+0000","last_unstale":"2024-04-25T08:42:05.710792+0000","last_undegraded":"2024-04-25T08:42:05.710792+0000","last_fullsized":"2024-04-25T08:42:05.710792+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:55.851053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858271+0000","last_change":"2024-04-25T08:42:04.697707+0000","last_active":"2024-04-25T08:42:05.858271+0000","last_peered":"2024-04-25T08:42:05.858271+0000","last_clean":"2024-04-25T08:42:05.858271+0000","last_became_active":"2024-04-25T08:42:04.697389+0000","last_became_peered":"2024-04-25T08:42:04.697389+0000","last_unstale":"2024-04-25T08:42:05.858271+0000","last_undegraded":"2024-04-25T08:42:05.858271+0000","last_fullsized":"2024-04-25T08:42:05.858271+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:02.381300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858534+0000","last_change":"2024-04-25T08:42:04.703070+0000","last_active":"2024-04-25T08:42:05.858534+0000","last_peered":"2024-04-25T08:42:05.858534+0000","last_clean":"2024-04-25T08:42:05.858534+0000","last_became_active":"2024-04-25T08:42:04.702788+0000","last_became_peered":"2024-04-25T08:42:04.702788+0000","last_unstale":"2024-04-25T08:42:05.858534+0000","last_undegraded":"2024-04-25T08:42:05.858534+0000","last_fullsized":"2024-04-25T08:42:05.858534+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:58.459727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697618+0000","last_change":"2024-04-25T08:42:04.698068+0000","last_active":"2024-04-25T08:42:05.697618+0000","last_peered":"2024-04-25T08:42:05.697618+0000","last_clean":"2024-04-25T08:42:05.697618+0000","last_became_active":"2024-04-25T08:42:04.697783+0000","last_became_peered":"2024-04-25T08:42:04.697783+0000","last_unstale":"2024-04-25T08:42:05.697618+0000","last_undegraded":"2024-04-25T08:42:05.697618+0000","last_fullsized":"2024-04-25T08:42:05.697618+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:11.489900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858318+0000","last_change":"2024-04-25T08:42:04.691270+0000","last_active":"2024-04-25T08:42:05.858318+0000","last_peered":"2024-04-25T08:42:05.858318+0000","last_clean":"2024-04-25T08:42:05.858318+0000","last_became_active":"2024-04-25T08:42:04.690898+0000","last_became_peered":"2024-04-25T08:42:04.690898+0000","last_unstale":"2024-04-25T08:42:05.858318+0000","last_undegraded":"2024-04-25T08:42:05.858318+0000","last_fullsized":"2024-04-25T08:42:05.858318+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:58.451011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857155+0000","last_change":"2024-04-25T08:42:04.687344+0000","last_active":"2024-04-25T08:42:05.857155+0000","last_peered":"2024-04-25T08:42:05.857155+0000","last_clean":"2024-04-25T08:42:05.857155+0000","last_became_active":"2024-04-25T08:42:04.686981+0000","last_became_peered":"2024-04-25T08:42:04.686981+0000","last_unstale":"2024-04-25T08:42:05.857155+0000","last_undegraded":"2024-04-25T08:42:05.857155+0000","last_fullsized":"2024-04-25T08:42:05.857155+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:51.569868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697073+0000","last_change":"2024-04-25T08:42:04.699542+0000","last_active":"2024-04-25T08:42:05.697073+0000","last_peered":"2024-04-25T08:42:05.697073+0000","last_clean":"2024-04-25T08:42:05.697073+0000","last_became_active":"2024-04-25T08:42:04.698834+0000","last_became_peered":"2024-04-25T08:42:04.698834+0000","last_unstale":"2024-04-25T08:42:05.697073+0000","last_undegraded":"2024-04-25T08:42:05.697073+0000","last_fullsized":"2024-04-25T08:42:05.697073+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:05.170508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857375+0000","last_change":"2024-04-25T08:42:04.688612+0000","last_active":"2024-04-25T08:42:05.857375+0000","last_peered":"2024-04-25T08:42:05.857375+0000","last_clean":"2024-04-25T08:42:05.857375+0000","last_became_active":"2024-04-25T08:42:04.688319+0000","last_became_peered":"2024-04-25T08:42:04.688319+0000","last_unstale":"2024-04-25T08:42:05.857375+0000","last_undegraded":"2024-04-25T08:42:05.857375+0000","last_fullsized":"2024-04-25T08:42:05.857375+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:38.536326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"142'4","reported_seq":18,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858402+0000","last_change":"2024-04-25T08:42:03.685968+0000","last_active":"2024-04-25T08:42:05.858402+0000","last_peered":"2024-04-25T08:42:05.858402+0000","last_clean":"2024-04-25T08:42:05.858402+0000","last_became_active":"2024-04-25T08:42:03.685479+0000","last_became_peered":"2024-04-25T08:42:03.685479+0000","last_unstale":"2024-04-25T08:42:05.858402+0000","last_undegraded":"2024-04-25T08:42:05.858402+0000","last_fullsized":"2024-04-25T08:42:05.858402+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:08.475116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857641+0000","last_change":"2024-04-25T08:42:04.691442+0000","last_active":"2024-04-25T08:42:05.857641+0000","last_peered":"2024-04-25T08:42:05.857641+0000","last_clean":"2024-04-25T08:42:05.857641+0000","last_became_active":"2024-04-25T08:42:04.691165+0000","last_became_peered":"2024-04-25T08:42:04.691165+0000","last_unstale":"2024-04-25T08:42:05.857641+0000","last_undegraded":"2024-04-25T08:42:05.857641+0000","last_fullsized":"2024-04-25T08:42:05.857641+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:28.909203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819141+0000","last_change":"2024-04-25T08:42:03.689935+0000","last_active":"2024-04-25T08:42:05.819141+0000","last_peered":"2024-04-25T08:42:05.819141+0000","last_clean":"2024-04-25T08:42:05.819141+0000","last_became_active":"2024-04-25T08:42:03.689579+0000","last_became_peered":"2024-04-25T08:42:03.689579+0000","last_unstale":"2024-04-25T08:42:05.819141+0000","last_undegraded":"2024-04-25T08:42:05.819141+0000","last_fullsized":"2024-04-25T08:42:05.819141+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:47.894977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697568+0000","last_change":"2024-04-25T08:42:04.691466+0000","last_active":"2024-04-25T08:42:05.697568+0000","last_peered":"2024-04-25T08:42:05.697568+0000","last_clean":"2024-04-25T08:42:05.697568+0000","last_became_active":"2024-04-25T08:42:04.691194+0000","last_became_peered":"2024-04-25T08:42:04.691194+0000","last_unstale":"2024-04-25T08:42:05.697568+0000","last_undegraded":"2024-04-25T08:42:05.697568+0000","last_fullsized":"2024-04-25T08:42:05.697568+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:56.306539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819393+0000","last_change":"2024-04-25T08:42:03.684012+0000","last_active":"2024-04-25T08:42:05.819393+0000","last_peered":"2024-04-25T08:42:05.819393+0000","last_clean":"2024-04-25T08:42:05.819393+0000","last_became_active":"2024-04-25T08:42:03.683723+0000","last_became_peered":"2024-04-25T08:42:03.683723+0000","last_unstale":"2024-04-25T08:42:05.819393+0000","last_undegraded":"2024-04-25T08:42:05.819393+0000","last_fullsized":"2024-04-25T08:42:05.819393+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:36.782073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819373+0000","last_change":"2024-04-25T08:42:04.693990+0000","last_active":"2024-04-25T08:42:05.819373+0000","last_peered":"2024-04-25T08:42:05.819373+0000","last_clean":"2024-04-25T08:42:05.819373+0000","last_became_active":"2024-04-25T08:42:04.693721+0000","last_became_peered":"2024-04-25T08:42:04.693721+0000","last_unstale":"2024-04-25T08:42:05.819373+0000","last_undegraded":"2024-04-25T08:42:05.819373+0000","last_fullsized":"2024-04-25T08:42:05.819373+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:58.496763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857478+0000","last_change":"2024-04-25T08:42:03.688587+0000","last_active":"2024-04-25T08:42:05.857478+0000","last_peered":"2024-04-25T08:42:05.857478+0000","last_clean":"2024-04-25T08:42:05.857478+0000","last_became_active":"2024-04-25T08:42:03.688246+0000","last_became_peered":"2024-04-25T08:42:03.688246+0000","last_unstale":"2024-04-25T08:42:05.857478+0000","last_undegraded":"2024-04-25T08:42:05.857478+0000","last_fullsized":"2024-04-25T08:42:05.857478+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:00.379501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.704162+0000","last_change":"2024-04-25T08:42:04.704666+0000","last_active":"2024-04-25T08:42:05.704162+0000","last_peered":"2024-04-25T08:42:05.704162+0000","last_clean":"2024-04-25T08:42:05.704162+0000","last_became_active":"2024-04-25T08:42:04.704391+0000","last_became_peered":"2024-04-25T08:42:04.704391+0000","last_unstale":"2024-04-25T08:42:05.704162+0000","last_undegraded":"2024-04-25T08:42:05.704162+0000","last_fullsized":"2024-04-25T08:42:05.704162+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:59.300899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697652+0000","last_change":"2024-04-25T08:42:03.686925+0000","last_active":"2024-04-25T08:42:05.697652+0000","last_peered":"2024-04-25T08:42:05.697652+0000","last_clean":"2024-04-25T08:42:05.697652+0000","last_became_active":"2024-04-25T08:42:03.685881+0000","last_became_peered":"2024-04-25T08:42:03.685881+0000","last_unstale":"2024-04-25T08:42:05.697652+0000","last_undegraded":"2024-04-25T08:42:05.697652+0000","last_fullsized":"2024-04-25T08:42:05.697652+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:38.119854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696682+0000","last_change":"2024-04-25T08:42:04.696019+0000","last_active":"2024-04-25T08:42:05.696682+0000","last_peered":"2024-04-25T08:42:05.696682+0000","last_clean":"2024-04-25T08:42:05.696682+0000","last_became_active":"2024-04-25T08:42:04.695686+0000","last_became_peered":"2024-04-25T08:42:04.695686+0000","last_unstale":"2024-04-25T08:42:05.696682+0000","last_undegraded":"2024-04-25T08:42:05.696682+0000","last_fullsized":"2024-04-25T08:42:05.696682+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:38.366164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857634+0000","last_change":"2024-04-25T08:42:03.683354+0000","last_active":"2024-04-25T08:42:05.857634+0000","last_peered":"2024-04-25T08:42:05.857634+0000","last_clean":"2024-04-25T08:42:05.857634+0000","last_became_active":"2024-04-25T08:42:03.683052+0000","last_became_peered":"2024-04-25T08:42:03.683052+0000","last_unstale":"2024-04-25T08:42:05.857634+0000","last_undegraded":"2024-04-25T08:42:05.857634+0000","last_fullsized":"2024-04-25T08:42:05.857634+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:09.409360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858245+0000","last_change":"2024-04-25T08:42:04.702957+0000","last_active":"2024-04-25T08:42:05.858245+0000","last_peered":"2024-04-25T08:42:05.858245+0000","last_clean":"2024-04-25T08:42:05.858245+0000","last_became_active":"2024-04-25T08:42:04.702620+0000","last_became_peered":"2024-04-25T08:42:04.702620+0000","last_unstale":"2024-04-25T08:42:05.858245+0000","last_undegraded":"2024-04-25T08:42:05.858245+0000","last_fullsized":"2024-04-25T08:42:05.858245+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:51.569868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710576+0000","last_change":"2024-04-25T08:42:03.691204+0000","last_active":"2024-04-25T08:42:05.710576+0000","last_peered":"2024-04-25T08:42:05.710576+0000","last_clean":"2024-04-25T08:42:05.710576+0000","last_became_active":"2024-04-25T08:42:03.690929+0000","last_became_peered":"2024-04-25T08:42:03.690929+0000","last_unstale":"2024-04-25T08:42:05.710576+0000","last_undegraded":"2024-04-25T08:42:05.710576+0000","last_fullsized":"2024-04-25T08:42:05.710576+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:09.366192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710862+0000","last_change":"2024-04-25T08:42:04.701644+0000","last_active":"2024-04-25T08:42:05.710862+0000","last_peered":"2024-04-25T08:42:05.710862+0000","last_clean":"2024-04-25T08:42:05.710862+0000","last_became_active":"2024-04-25T08:42:04.701270+0000","last_became_peered":"2024-04-25T08:42:04.701270+0000","last_unstale":"2024-04-25T08:42:05.710862+0000","last_undegraded":"2024-04-25T08:42:05.710862+0000","last_fullsized":"2024-04-25T08:42:05.710862+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:35.361036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858001+0000","last_change":"2024-04-25T08:42:03.693479+0000","last_active":"2024-04-25T08:42:05.858001+0000","last_peered":"2024-04-25T08:42:05.858001+0000","last_clean":"2024-04-25T08:42:05.858001+0000","last_became_active":"2024-04-25T08:42:03.692971+0000","last_became_peered":"2024-04-25T08:42:03.692971+0000","last_unstale":"2024-04-25T08:42:05.858001+0000","last_undegraded":"2024-04-25T08:42:05.858001+0000","last_fullsized":"2024-04-25T08:42:05.858001+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:07.498704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697850+0000","last_change":"2024-04-25T08:42:04.689059+0000","last_active":"2024-04-25T08:42:05.697850+0000","last_peered":"2024-04-25T08:42:05.697850+0000","last_clean":"2024-04-25T08:42:05.697850+0000","last_became_active":"2024-04-25T08:42:04.688772+0000","last_became_peered":"2024-04-25T08:42:04.688772+0000","last_unstale":"2024-04-25T08:42:05.697850+0000","last_undegraded":"2024-04-25T08:42:05.697850+0000","last_fullsized":"2024-04-25T08:42:05.697850+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:10.418523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819373+0000","last_change":"2024-04-25T08:42:03.685310+0000","last_active":"2024-04-25T08:42:05.819373+0000","last_peered":"2024-04-25T08:42:05.819373+0000","last_clean":"2024-04-25T08:42:05.819373+0000","last_became_active":"2024-04-25T08:42:03.685056+0000","last_became_peered":"2024-04-25T08:42:03.685056+0000","last_unstale":"2024-04-25T08:42:05.819373+0000","last_undegraded":"2024-04-25T08:42:05.819373+0000","last_fullsized":"2024-04-25T08:42:05.819373+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:16.189465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858318+0000","last_change":"2024-04-25T08:42:04.695200+0000","last_active":"2024-04-25T08:42:05.858318+0000","last_peered":"2024-04-25T08:42:05.858318+0000","last_clean":"2024-04-25T08:42:05.858318+0000","last_became_active":"2024-04-25T08:42:04.694539+0000","last_became_peered":"2024-04-25T08:42:04.694539+0000","last_unstale":"2024-04-25T08:42:05.858318+0000","last_undegraded":"2024-04-25T08:42:05.858318+0000","last_fullsized":"2024-04-25T08:42:05.858318+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:56:25.096201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710937+0000","last_change":"2024-04-25T08:42:03.689429+0000","last_active":"2024-04-25T08:42:05.710937+0000","last_peered":"2024-04-25T08:42:05.710937+0000","last_clean":"2024-04-25T08:42:05.710937+0000","last_became_active":"2024-04-25T08:42:03.689160+0000","last_became_peered":"2024-04-25T08:42:03.689160+0000","last_unstale":"2024-04-25T08:42:05.710937+0000","last_undegraded":"2024-04-25T08:42:05.710937+0000","last_fullsized":"2024-04-25T08:42:05.710937+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:04.161345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858032+0000","last_change":"2024-04-25T08:42:04.702122+0000","last_active":"2024-04-25T08:42:05.858032+0000","last_peered":"2024-04-25T08:42:05.858032+0000","last_clean":"2024-04-25T08:42:05.858032+0000","last_became_active":"2024-04-25T08:42:04.699132+0000","last_became_peered":"2024-04-25T08:42:04.699132+0000","last_unstale":"2024-04-25T08:42:05.858032+0000","last_undegraded":"2024-04-25T08:42:05.858032+0000","last_fullsized":"2024-04-25T08:42:05.858032+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:56.813766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"142'3","reported_seq":17,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858540+0000","last_change":"2024-04-25T08:42:03.694273+0000","last_active":"2024-04-25T08:42:05.858540+0000","last_peered":"2024-04-25T08:42:05.858540+0000","last_clean":"2024-04-25T08:42:05.858540+0000","last_became_active":"2024-04-25T08:42:03.693986+0000","last_became_peered":"2024-04-25T08:42:03.693986+0000","last_unstale":"2024-04-25T08:42:05.858540+0000","last_undegraded":"2024-04-25T08:42:05.858540+0000","last_fullsized":"2024-04-25T08:42:05.858540+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:43.373732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697306+0000","last_change":"2024-04-25T08:42:04.687582+0000","last_active":"2024-04-25T08:42:05.697306+0000","last_peered":"2024-04-25T08:42:05.697306+0000","last_clean":"2024-04-25T08:42:05.697306+0000","last_became_active":"2024-04-25T08:42:04.687292+0000","last_became_peered":"2024-04-25T08:42:04.687292+0000","last_unstale":"2024-04-25T08:42:05.697306+0000","last_undegraded":"2024-04-25T08:42:05.697306+0000","last_fullsized":"2024-04-25T08:42:05.697306+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:56.067119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858281+0000","last_change":"2024-04-25T08:42:03.684932+0000","last_active":"2024-04-25T08:42:05.858281+0000","last_peered":"2024-04-25T08:42:05.858281+0000","last_clean":"2024-04-25T08:42:05.858281+0000","last_became_active":"2024-04-25T08:42:03.684602+0000","last_became_peered":"2024-04-25T08:42:03.684602+0000","last_unstale":"2024-04-25T08:42:05.858281+0000","last_undegraded":"2024-04-25T08:42:05.858281+0000","last_fullsized":"2024-04-25T08:42:05.858281+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:44.041610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857934+0000","last_change":"2024-04-25T08:42:04.690073+0000","last_active":"2024-04-25T08:42:05.857934+0000","last_peered":"2024-04-25T08:42:05.857934+0000","last_clean":"2024-04-25T08:42:05.857934+0000","last_became_active":"2024-04-25T08:42:04.689314+0000","last_became_peered":"2024-04-25T08:42:04.689314+0000","last_unstale":"2024-04-25T08:42:05.857934+0000","last_undegraded":"2024-04-25T08:42:05.857934+0000","last_fullsized":"2024-04-25T08:42:05.857934+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:37.791237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696763+0000","last_change":"2024-04-25T08:42:03.686162+0000","last_active":"2024-04-25T08:42:05.696763+0000","last_peered":"2024-04-25T08:42:05.696763+0000","last_clean":"2024-04-25T08:42:05.696763+0000","last_became_active":"2024-04-25T08:42:03.685811+0000","last_became_peered":"2024-04-25T08:42:03.685811+0000","last_unstale":"2024-04-25T08:42:05.696763+0000","last_undegraded":"2024-04-25T08:42:05.696763+0000","last_fullsized":"2024-04-25T08:42:05.696763+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:37.026670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857530+0000","last_change":"2024-04-25T08:42:04.695918+0000","last_active":"2024-04-25T08:42:05.857530+0000","last_peered":"2024-04-25T08:42:05.857530+0000","last_clean":"2024-04-25T08:42:05.857530+0000","last_became_active":"2024-04-25T08:42:04.695591+0000","last_became_peered":"2024-04-25T08:42:04.695591+0000","last_unstale":"2024-04-25T08:42:05.857530+0000","last_undegraded":"2024-04-25T08:42:05.857530+0000","last_fullsized":"2024-04-25T08:42:05.857530+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:02.048229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696711+0000","last_change":"2024-04-25T08:42:03.691132+0000","last_active":"2024-04-25T08:42:05.696711+0000","last_peered":"2024-04-25T08:42:05.696711+0000","last_clean":"2024-04-25T08:42:05.696711+0000","last_became_active":"2024-04-25T08:42:03.690786+0000","last_became_peered":"2024-04-25T08:42:03.690786+0000","last_unstale":"2024-04-25T08:42:05.696711+0000","last_undegraded":"2024-04-25T08:42:05.696711+0000","last_fullsized":"2024-04-25T08:42:05.696711+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:09.366192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857055+0000","last_change":"2024-04-25T08:42:04.701729+0000","last_active":"2024-04-25T08:42:05.857055+0000","last_peered":"2024-04-25T08:42:05.857055+0000","last_clean":"2024-04-25T08:42:05.857055+0000","last_became_active":"2024-04-25T08:42:04.701456+0000","last_became_peered":"2024-04-25T08:42:04.701456+0000","last_unstale":"2024-04-25T08:42:05.857055+0000","last_undegraded":"2024-04-25T08:42:05.857055+0000","last_fullsized":"2024-04-25T08:42:05.857055+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:03.071862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"142'8","reported_seq":22,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710396+0000","last_change":"2024-04-25T08:42:03.685330+0000","last_active":"2024-04-25T08:42:05.710396+0000","last_peered":"2024-04-25T08:42:05.710396+0000","last_clean":"2024-04-25T08:42:05.710396+0000","last_became_active":"2024-04-25T08:42:03.685059+0000","last_became_peered":"2024-04-25T08:42:03.685059+0000","last_unstale":"2024-04-25T08:42:05.710396+0000","last_undegraded":"2024-04-25T08:42:05.710396+0000","last_fullsized":"2024-04-25T08:42:05.710396+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:15.799519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819079+0000","last_change":"2024-04-25T08:42:04.696341+0000","last_active":"2024-04-25T08:42:05.819079+0000","last_peered":"2024-04-25T08:42:05.819079+0000","last_clean":"2024-04-25T08:42:05.819079+0000","last_became_active":"2024-04-25T08:42:04.696061+0000","last_became_peered":"2024-04-25T08:42:04.696061+0000","last_unstale":"2024-04-25T08:42:05.819079+0000","last_undegraded":"2024-04-25T08:42:05.819079+0000","last_fullsized":"2024-04-25T08:42:05.819079+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:04.379981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857609+0000","last_change":"2024-04-25T08:42:03.691543+0000","last_active":"2024-04-25T08:42:05.857609+0000","last_peered":"2024-04-25T08:42:05.857609+0000","last_clean":"2024-04-25T08:42:05.857609+0000","last_became_active":"2024-04-25T08:42:03.691142+0000","last_became_peered":"2024-04-25T08:42:03.691142+0000","last_unstale":"2024-04-25T08:42:05.857609+0000","last_undegraded":"2024-04-25T08:42:05.857609+0000","last_fullsized":"2024-04-25T08:42:05.857609+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:00.747860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819292+0000","last_change":"2024-04-25T08:42:04.690350+0000","last_active":"2024-04-25T08:42:05.819292+0000","last_peered":"2024-04-25T08:42:05.819292+0000","last_clean":"2024-04-25T08:42:05.819292+0000","last_became_active":"2024-04-25T08:42:04.689328+0000","last_became_peered":"2024-04-25T08:42:04.689328+0000","last_unstale":"2024-04-25T08:42:05.819292+0000","last_undegraded":"2024-04-25T08:42:05.819292+0000","last_fullsized":"2024-04-25T08:42:05.819292+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:55.023285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819271+0000","last_change":"2024-04-25T08:42:03.689547+0000","last_active":"2024-04-25T08:42:05.819271+0000","last_peered":"2024-04-25T08:42:05.819271+0000","last_clean":"2024-04-25T08:42:05.819271+0000","last_became_active":"2024-04-25T08:42:03.689265+0000","last_became_peered":"2024-04-25T08:42:03.689265+0000","last_unstale":"2024-04-25T08:42:05.819271+0000","last_undegraded":"2024-04-25T08:42:05.819271+0000","last_fullsized":"2024-04-25T08:42:05.819271+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:50.560704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710813+0000","last_change":"2024-04-25T08:42:04.698298+0000","last_active":"2024-04-25T08:42:05.710813+0000","last_peered":"2024-04-25T08:42:05.710813+0000","last_clean":"2024-04-25T08:42:05.710813+0000","last_became_active":"2024-04-25T08:42:04.697777+0000","last_became_peered":"2024-04-25T08:42:04.697777+0000","last_unstale":"2024-04-25T08:42:05.710813+0000","last_undegraded":"2024-04-25T08:42:05.710813+0000","last_fullsized":"2024-04-25T08:42:05.710813+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:32.366008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"142'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857666+0000","last_change":"2024-04-25T08:42:03.690041+0000","last_active":"2024-04-25T08:42:05.857666+0000","last_peered":"2024-04-25T08:42:05.857666+0000","last_clean":"2024-04-25T08:42:05.857666+0000","last_became_active":"2024-04-25T08:42:03.689483+0000","last_became_peered":"2024-04-25T08:42:03.689483+0000","last_unstale":"2024-04-25T08:42:05.857666+0000","last_undegraded":"2024-04-25T08:42:05.857666+0000","last_fullsized":"2024-04-25T08:42:05.857666+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:55.057955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710985+0000","last_change":"2024-04-25T08:42:04.696360+0000","last_active":"2024-04-25T08:42:05.710985+0000","last_peered":"2024-04-25T08:42:05.710985+0000","last_clean":"2024-04-25T08:42:05.710985+0000","last_became_active":"2024-04-25T08:42:04.696009+0000","last_became_peered":"2024-04-25T08:42:04.696009+0000","last_unstale":"2024-04-25T08:42:05.710985+0000","last_undegraded":"2024-04-25T08:42:05.710985+0000","last_fullsized":"2024-04-25T08:42:05.710985+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:44.343322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696726+0000","last_change":"2024-04-25T08:42:03.684689+0000","last_active":"2024-04-25T08:42:05.696726+0000","last_peered":"2024-04-25T08:42:05.696726+0000","last_clean":"2024-04-25T08:42:05.696726+0000","last_became_active":"2024-04-25T08:42:03.684386+0000","last_became_peered":"2024-04-25T08:42:03.684386+0000","last_unstale":"2024-04-25T08:42:05.696726+0000","last_undegraded":"2024-04-25T08:42:05.696726+0000","last_fullsized":"2024-04-25T08:42:05.696726+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:43.236643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819155+0000","last_change":"2024-04-25T08:42:04.690026+0000","last_active":"2024-04-25T08:42:05.819155+0000","last_peered":"2024-04-25T08:42:05.819155+0000","last_clean":"2024-04-25T08:42:05.819155+0000","last_became_active":"2024-04-25T08:42:04.689734+0000","last_became_peered":"2024-04-25T08:42:04.689734+0000","last_unstale":"2024-04-25T08:42:05.819155+0000","last_undegraded":"2024-04-25T08:42:05.819155+0000","last_fullsized":"2024-04-25T08:42:05.819155+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:51.742834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.704235+0000","last_change":"2024-04-25T08:42:03.691691+0000","last_active":"2024-04-25T08:42:05.704235+0000","last_peered":"2024-04-25T08:42:05.704235+0000","last_clean":"2024-04-25T08:42:05.704235+0000","last_became_active":"2024-04-25T08:42:03.691380+0000","last_became_peered":"2024-04-25T08:42:03.691380+0000","last_unstale":"2024-04-25T08:42:05.704235+0000","last_undegraded":"2024-04-25T08:42:05.704235+0000","last_fullsized":"2024-04-25T08:42:05.704235+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:21.859122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858089+0000","last_change":"2024-04-25T08:42:04.690321+0000","last_active":"2024-04-25T08:42:05.858089+0000","last_peered":"2024-04-25T08:42:05.858089+0000","last_clean":"2024-04-25T08:42:05.858089+0000","last_became_active":"2024-04-25T08:42:04.689785+0000","last_became_peered":"2024-04-25T08:42:04.689785+0000","last_unstale":"2024-04-25T08:42:05.858089+0000","last_undegraded":"2024-04-25T08:42:05.858089+0000","last_fullsized":"2024-04-25T08:42:05.858089+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:48.904141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697304+0000","last_change":"2024-04-25T08:42:03.681184+0000","last_active":"2024-04-25T08:42:05.697304+0000","last_peered":"2024-04-25T08:42:05.697304+0000","last_clean":"2024-04-25T08:42:05.697304+0000","last_became_active":"2024-04-25T08:42:03.680886+0000","last_became_peered":"2024-04-25T08:42:03.680886+0000","last_unstale":"2024-04-25T08:42:05.697304+0000","last_undegraded":"2024-04-25T08:42:05.697304+0000","last_fullsized":"2024-04-25T08:42:05.697304+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:11.637696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.819030+0000","last_change":"2024-04-25T08:42:04.689778+0000","last_active":"2024-04-25T08:42:05.819030+0000","last_peered":"2024-04-25T08:42:05.819030+0000","last_clean":"2024-04-25T08:42:05.819030+0000","last_became_active":"2024-04-25T08:42:04.689477+0000","last_became_peered":"2024-04-25T08:42:04.689477+0000","last_unstale":"2024-04-25T08:42:05.819030+0000","last_undegraded":"2024-04-25T08:42:05.819030+0000","last_fullsized":"2024-04-25T08:42:05.819030+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:02.381300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.703974+0000","last_change":"2024-04-25T08:42:03.692612+0000","last_active":"2024-04-25T08:42:05.703974+0000","last_peered":"2024-04-25T08:42:05.703974+0000","last_clean":"2024-04-25T08:42:05.703974+0000","last_became_active":"2024-04-25T08:42:03.692319+0000","last_became_peered":"2024-04-25T08:42:03.692319+0000","last_unstale":"2024-04-25T08:42:05.703974+0000","last_undegraded":"2024-04-25T08:42:05.703974+0000","last_fullsized":"2024-04-25T08:42:05.703974+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:32.189189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857208+0000","last_change":"2024-04-25T08:42:04.689251+0000","last_active":"2024-04-25T08:42:05.857208+0000","last_peered":"2024-04-25T08:42:05.857208+0000","last_clean":"2024-04-25T08:42:05.857208+0000","last_became_active":"2024-04-25T08:42:04.688984+0000","last_became_peered":"2024-04-25T08:42:04.688984+0000","last_unstale":"2024-04-25T08:42:05.857208+0000","last_undegraded":"2024-04-25T08:42:05.857208+0000","last_fullsized":"2024-04-25T08:42:05.857208+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:06.864814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697630+0000","last_change":"2024-04-25T08:42:03.692428+0000","last_active":"2024-04-25T08:42:05.697630+0000","last_peered":"2024-04-25T08:42:05.697630+0000","last_clean":"2024-04-25T08:42:05.697630+0000","last_became_active":"2024-04-25T08:42:03.692124+0000","last_became_peered":"2024-04-25T08:42:03.692124+0000","last_unstale":"2024-04-25T08:42:05.697630+0000","last_undegraded":"2024-04-25T08:42:05.697630+0000","last_fullsized":"2024-04-25T08:42:05.697630+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:37.154217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858597+0000","last_change":"2024-04-25T08:42:04.692333+0000","last_active":"2024-04-25T08:42:05.858597+0000","last_peered":"2024-04-25T08:42:05.858597+0000","last_clean":"2024-04-25T08:42:05.858597+0000","last_became_active":"2024-04-25T08:42:04.692056+0000","last_became_peered":"2024-04-25T08:42:04.692056+0000","last_unstale":"2024-04-25T08:42:05.858597+0000","last_undegraded":"2024-04-25T08:42:05.858597+0000","last_fullsized":"2024-04-25T08:42:05.858597+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:55.023285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"142'2","reported_seq":16,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696612+0000","last_change":"2024-04-25T08:42:03.685051+0000","last_active":"2024-04-25T08:42:05.696612+0000","last_peered":"2024-04-25T08:42:05.696612+0000","last_clean":"2024-04-25T08:42:05.696612+0000","last_became_active":"2024-04-25T08:42:03.684598+0000","last_became_peered":"2024-04-25T08:42:03.684598+0000","last_unstale":"2024-04-25T08:42:05.696612+0000","last_undegraded":"2024-04-25T08:42:05.696612+0000","last_fullsized":"2024-04-25T08:42:05.696612+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:15.799519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696539+0000","last_change":"2024-04-25T08:42:04.703530+0000","last_active":"2024-04-25T08:42:05.696539+0000","last_peered":"2024-04-25T08:42:05.696539+0000","last_clean":"2024-04-25T08:42:05.696539+0000","last_became_active":"2024-04-25T08:42:04.703243+0000","last_became_peered":"2024-04-25T08:42:04.703243+0000","last_unstale":"2024-04-25T08:42:05.696539+0000","last_undegraded":"2024-04-25T08:42:05.696539+0000","last_fullsized":"2024-04-25T08:42:05.696539+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:10.418523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"143'1","reported_seq":15,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:06.677474+0000","last_change":"2024-04-25T08:42:03.686672+0000","last_active":"2024-04-25T08:42:06.677474+0000","last_peered":"2024-04-25T08:42:06.677474+0000","last_clean":"2024-04-25T08:42:06.677474+0000","last_became_active":"2024-04-25T08:42:03.686415+0000","last_became_peered":"2024-04-25T08:42:03.686415+0000","last_unstale":"2024-04-25T08:42:06.677474+0000","last_undegraded":"2024-04-25T08:42:06.677474+0000","last_fullsized":"2024-04-25T08:42:06.677474+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:00.379501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710922+0000","last_change":"2024-04-25T08:42:04.702305+0000","last_active":"2024-04-25T08:42:05.710922+0000","last_peered":"2024-04-25T08:42:05.710922+0000","last_clean":"2024-04-25T08:42:05.710922+0000","last_became_active":"2024-04-25T08:42:04.701290+0000","last_became_peered":"2024-04-25T08:42:04.701290+0000","last_unstale":"2024-04-25T08:42:05.710922+0000","last_undegraded":"2024-04-25T08:42:05.710922+0000","last_fullsized":"2024-04-25T08:42:05.710922+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:38.536326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.857739+0000","last_change":"2024-04-25T08:42:03.690264+0000","last_active":"2024-04-25T08:42:05.857739+0000","last_peered":"2024-04-25T08:42:05.857739+0000","last_clean":"2024-04-25T08:42:05.857739+0000","last_became_active":"2024-04-25T08:42:03.689789+0000","last_became_peered":"2024-04-25T08:42:03.689789+0000","last_unstale":"2024-04-25T08:42:05.857739+0000","last_undegraded":"2024-04-25T08:42:05.857739+0000","last_fullsized":"2024-04-25T08:42:05.857739+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:07.248828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858291+0000","last_change":"2024-04-25T08:42:04.700343+0000","last_active":"2024-04-25T08:42:05.858291+0000","last_peered":"2024-04-25T08:42:05.858291+0000","last_clean":"2024-04-25T08:42:05.858291+0000","last_became_active":"2024-04-25T08:42:04.700052+0000","last_became_peered":"2024-04-25T08:42:04.700052+0000","last_unstale":"2024-04-25T08:42:05.858291+0000","last_undegraded":"2024-04-25T08:42:05.858291+0000","last_fullsized":"2024-04-25T08:42:05.858291+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:04.379981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858645+0000","last_change":"2024-04-25T08:42:03.690696+0000","last_active":"2024-04-25T08:42:05.858645+0000","last_peered":"2024-04-25T08:42:05.858645+0000","last_clean":"2024-04-25T08:42:05.858645+0000","last_became_active":"2024-04-25T08:42:03.690383+0000","last_became_peered":"2024-04-25T08:42:03.690383+0000","last_unstale":"2024-04-25T08:42:05.858645+0000","last_undegraded":"2024-04-25T08:42:05.858645+0000","last_fullsized":"2024-04-25T08:42:05.858645+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:20.395545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697511+0000","last_change":"2024-04-25T08:42:04.705552+0000","last_active":"2024-04-25T08:42:05.697511+0000","last_peered":"2024-04-25T08:42:05.697511+0000","last_clean":"2024-04-25T08:42:05.697511+0000","last_became_active":"2024-04-25T08:42:04.705278+0000","last_became_peered":"2024-04-25T08:42:04.705278+0000","last_unstale":"2024-04-25T08:42:05.697511+0000","last_undegraded":"2024-04-25T08:42:05.697511+0000","last_fullsized":"2024-04-25T08:42:05.697511+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:55:37.298009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710684+0000","last_change":"2024-04-25T08:42:03.688933+0000","last_active":"2024-04-25T08:42:05.710684+0000","last_peered":"2024-04-25T08:42:05.710684+0000","last_clean":"2024-04-25T08:42:05.710684+0000","last_became_active":"2024-04-25T08:42:03.688608+0000","last_became_peered":"2024-04-25T08:42:03.688608+0000","last_unstale":"2024-04-25T08:42:05.710684+0000","last_undegraded":"2024-04-25T08:42:05.710684+0000","last_fullsized":"2024-04-25T08:42:05.710684+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:10.585956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.696269+0000","last_change":"2024-04-25T08:42:04.700466+0000","last_active":"2024-04-25T08:42:05.696269+0000","last_peered":"2024-04-25T08:42:05.696269+0000","last_clean":"2024-04-25T08:42:05.696269+0000","last_became_active":"2024-04-25T08:42:04.700164+0000","last_became_peered":"2024-04-25T08:42:04.700164+0000","last_unstale":"2024-04-25T08:42:05.696269+0000","last_undegraded":"2024-04-25T08:42:05.696269+0000","last_fullsized":"2024-04-25T08:42:05.696269+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:56.306539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858415+0000","last_change":"2024-04-25T08:42:03.689840+0000","last_active":"2024-04-25T08:42:05.858415+0000","last_peered":"2024-04-25T08:42:05.858415+0000","last_clean":"2024-04-25T08:42:05.858415+0000","last_became_active":"2024-04-25T08:42:03.689351+0000","last_became_peered":"2024-04-25T08:42:03.689351+0000","last_unstale":"2024-04-25T08:42:05.858415+0000","last_undegraded":"2024-04-25T08:42:05.858415+0000","last_fullsized":"2024-04-25T08:42:05.858415+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:24.560726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.858382+0000","last_change":"2024-04-25T08:42:04.698200+0000","last_active":"2024-04-25T08:42:05.858382+0000","last_peered":"2024-04-25T08:42:05.858382+0000","last_clean":"2024-04-25T08:42:05.858382+0000","last_became_active":"2024-04-25T08:42:04.697935+0000","last_became_peered":"2024-04-25T08:42:04.697935+0000","last_unstale":"2024-04-25T08:42:05.858382+0000","last_undegraded":"2024-04-25T08:42:05.858382+0000","last_fullsized":"2024-04-25T08:42:05.858382+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:18.372469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.710694+0000","last_change":"2024-04-25T08:42:03.689116+0000","last_active":"2024-04-25T08:42:05.710694+0000","last_peered":"2024-04-25T08:42:05.710694+0000","last_clean":"2024-04-25T08:42:05.710694+0000","last_became_active":"2024-04-25T08:42:03.688471+0000","last_became_peered":"2024-04-25T08:42:03.688471+0000","last_unstale":"2024-04-25T08:42:05.710694+0000","last_undegraded":"2024-04-25T08:42:05.710694+0000","last_fullsized":"2024-04-25T08:42:05.710694+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:04.668269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697405+0000","last_change":"2024-04-25T08:42:04.699247+0000","last_active":"2024-04-25T08:42:05.697405+0000","last_peered":"2024-04-25T08:42:05.697405+0000","last_clean":"2024-04-25T08:42:05.697405+0000","last_became_active":"2024-04-25T08:42:04.698880+0000","last_became_peered":"2024-04-25T08:42:04.698880+0000","last_unstale":"2024-04-25T08:42:05.697405+0000","last_undegraded":"2024-04-25T08:42:05.697405+0000","last_fullsized":"2024-04-25T08:42:05.697405+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:08.257992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.818902+0000","last_change":"2024-04-25T08:42:03.688861+0000","last_active":"2024-04-25T08:42:05.818902+0000","last_peered":"2024-04-25T08:42:05.818902+0000","last_clean":"2024-04-25T08:42:05.818902+0000","last_became_active":"2024-04-25T08:42:03.688492+0000","last_became_peered":"2024-04-25T08:42:03.688492+0000","last_unstale":"2024-04-25T08:42:05.818902+0000","last_undegraded":"2024-04-25T08:42:05.818902+0000","last_fullsized":"2024-04-25T08:42:05.818902+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:27.900040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.697909+0000","last_change":"2024-04-25T08:42:04.689183+0000","last_active":"2024-04-25T08:42:05.697909+0000","last_peered":"2024-04-25T08:42:05.697909+0000","last_clean":"2024-04-25T08:42:05.697909+0000","last_became_active":"2024-04-25T08:42:04.688902+0000","last_became_peered":"2024-04-25T08:42:04.688902+0000","last_unstale":"2024-04-25T08:42:05.697909+0000","last_undegraded":"2024-04-25T08:42:05.697909+0000","last_fullsized":"2024-04-25T08:42:05.697909+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":141,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:03.664574+0000","last_clean_scrub_stamp":"2024-04-25T08:42:03.664574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:08.507867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"142'6","reported_seq":20,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.704238+0000","last_change":"2024-04-25T08:42:03.691985+0000","last_active":"2024-04-25T08:42:05.704238+0000","last_peered":"2024-04-25T08:42:05.704238+0000","last_clean":"2024-04-25T08:42:05.704238+0000","last_became_active":"2024-04-25T08:42:03.691680+0000","last_became_peered":"2024-04-25T08:42:03.691680+0000","last_unstale":"2024-04-25T08:42:05.704238+0000","last_undegraded":"2024-04-25T08:42:05.704238+0000","last_fullsized":"2024-04-25T08:42:05.704238+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":140,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:42:02.655411+0000","last_clean_scrub_stamp":"2024-04-25T08:42:02.655411+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:53.140435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":401,"reported_epoch":143,"state":"active+clean","last_fresh":"2024-04-25T08:42:05.818984+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:42:05.818984+0000","last_peered":"2024-04-25T08:42:05.818984+0000","last_clean":"2024-04-25T08:42:05.818984+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:42:05.818984+0000","last_undegraded":"2024-04-25T08:42:05.818984+0000","last_fullsized":"2024-04-25T08:42:05.818984+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69612,"kb_used_data":2336,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302228,"statfs":{"total":96636764160,"available":96565481472,"internally_reserved":0,"allocated":2392064,"data_stored":1430648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46832,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325008,"statfs":{"total":96636764160,"available":96588808192,"internally_reserved":0,"allocated":1798144,"data_stored":840328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574980,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59300,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312540,"statfs":{"total":96636764160,"available":96576040960,"internally_reserved":0,"allocated":1794048,"data_stored":840258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73200,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298640,"statfs":{"total":96636764160,"available":96561807360,"internally_reserved":0,"allocated":1806336,"data_stored":841870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77364,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294476,"statfs":{"total":96636764160,"available":96557543424,"internally_reserved":0,"allocated":1802240,"data_stored":840874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46828,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325012,"statfs":{"total":96636764160,"available":96588812288,"internally_reserved":0,"allocated":1802240,"data_stored":840370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574980,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59312,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312528,"statfs":{"total":96636764160,"available":96576028672,"internally_reserved":0,"allocated":1798144,"data_stored":840328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69568,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302272,"statfs":{"total":96636764160,"available":96565526528,"internally_reserved":0,"allocated":2412544,"data_stored":1432944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68811205},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:42:08.030 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:42:08.030 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:42:08.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.183+0000 7f288f24c640 1 Processor -- start 2024-04-25T08:42:08.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.183+0000 7f288f24c640 1 -- start start 2024-04-25T08:42:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288f24c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f28880cece0 0x7f28880b94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288f24c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f28880d3050 0x7f28880b6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288f24c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28880b8320 con 0x7f288815db80 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288f24c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28880b8600 con 0x7f28880d3050 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288f24c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28880b88e0 con 0x7f28880cece0 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288d7e9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f28880d3050 0x7f28880b6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288cfe8640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f288815db80 0x7f28880ce5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:36304/0 (socket says 172.21.15.38:36304) 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288d7e9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f28880d3050 0x7f28880b6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52066/0 (socket says 172.21.15.38:52066) 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288cfe8640 1 -- 172.21.15.38:0/355253455 learned_addr learned my addr 172.21.15.38:0/355253455 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f2887fff640 1 --2- 172.21.15.38:0/355253455 >> v2:172.21.15.107:3300/0 conn(0x7f28880cece0 0x7f28880b94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288d7e9640 1 -- 172.21.15.38:0/355253455 >> v2:172.21.15.107:3300/0 conn(0x7f28880cece0 msgr2=0x7f28880b94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288d7e9640 1 --2- 172.21.15.38:0/355253455 >> v2:172.21.15.107:3300/0 conn(0x7f28880cece0 0x7f28880b94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288d7e9640 1 -- 172.21.15.38:0/355253455 >> v1:172.21.15.38:6789/0 conn(0x7f288815db80 legacy=0x7f28880ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f28877fe640 1 -- 172.21.15.38:0/355253455 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789321132 0 0) 0x7f28880b8320 con 0x7f288815db80 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.184+0000 7f288d7e9640 1 -- 172.21.15.38:0/355253455 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28880b8bc0 con 0x7f28880d3050 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.185+0000 7f288d7e9640 1 --2- 172.21.15.38:0/355253455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28880d3050 0x7f28880b6a30 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f288806cf80 tx=0x7f288002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159d8107588e7a7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.185+0000 7f28877fe640 1 -- 172.21.15.38:0/355253455 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880007740 con 0x7f28880d3050 2024-04-25T08:42:08.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.185+0000 7f28877fe640 1 -- 172.21.15.38:0/355253455 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28800078e0 con 0x7f28880d3050 2024-04-25T08:42:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.185+0000 7f28877fe640 1 -- 172.21.15.38:0/355253455 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880007cb0 con 0x7f28880d3050 2024-04-25T08:42:08.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.186+0000 7f288f24c640 1 -- 172.21.15.38:0/355253455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28880d3050 msgr2=0x7f28880b6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.186+0000 7f288f24c640 1 --2- 172.21.15.38:0/355253455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28880d3050 0x7f28880b6a30 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f288806cf80 tx=0x7f288002a710 comp rx=0 tx=0).stop 2024-04-25T08:42:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.186+0000 7f288f24c640 1 -- 172.21.15.38:0/355253455 shutdown_connections 2024-04-25T08:42:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.186+0000 7f288f24c640 1 --2- 172.21.15.38:0/355253455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28880d3050 0x7f28880b6a30 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.186+0000 7f288f24c640 1 --2- 172.21.15.38:0/355253455 >> v2:172.21.15.107:3300/0 conn(0x7f28880cece0 0x7f28880b94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.186+0000 7f288f24c640 1 -- 172.21.15.38:0/355253455 >> 172.21.15.38:0/355253455 conn(0x7f2888066d10 msgr2=0x7f2888067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.186+0000 7f288f24c640 1 -- 172.21.15.38:0/355253455 shutdown_connections 2024-04-25T08:42:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.186+0000 7f288f24c640 1 -- 172.21.15.38:0/355253455 wait complete. 2024-04-25T08:42:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.187+0000 7f288f24c640 1 Processor -- start 2024-04-25T08:42:08.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.187+0000 7f288f24c640 1 -- start start 2024-04-25T08:42:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f288f24c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28880cece0 0x7f28881f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f288f24c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28880d3050 0x7f288815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f288f24c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288815db80 0x7f288815b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f288f24c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28880b7300 con 0x7f28880d3050 2024-04-25T08:42:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f288f24c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28880b7180 con 0x7f288815db80 2024-04-25T08:42:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f288f24c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28880b7480 con 0x7f28880cece0 2024-04-25T08:42:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f2887fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28880d3050 0x7f288815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:08.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f2887fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28880d3050 0x7f288815b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52082/0 (socket says 172.21.15.38:52082) 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f2887fff640 1 -- 172.21.15.38:0/1605671864 learned_addr learned my addr 172.21.15.38:0/1605671864 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f288cfe8640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28880cece0 0x7f28881f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f288d7e9640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288815db80 0x7f288815b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f2887fff640 1 -- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28880cece0 msgr2=0x7f28881f67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f2887fff640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28880cece0 0x7f28881f67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f2887fff640 1 -- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288815db80 msgr2=0x7f288815b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f2887fff640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288815db80 0x7f288815b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f2887fff640 1 -- 172.21.15.38:0/1605671864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28881578b0 con 0x7f28880d3050 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.188+0000 7f2887fff640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28880d3050 0x7f288815b330 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f287400e900 tx=0x7f287400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.189+0000 7f28857fa640 1 -- 172.21.15.38:0/1605671864 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874017070 con 0x7f28880d3050 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.189+0000 7f28857fa640 1 -- 172.21.15.38:0/1605671864 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287400ca00 con 0x7f28880d3050 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.189+0000 7f28857fa640 1 -- 172.21.15.38:0/1605671864 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287400cdb0 con 0x7f28880d3050 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.189+0000 7f288f24c640 1 -- 172.21.15.38:0/1605671864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2888157b20 con 0x7f28880d3050 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.189+0000 7f288f24c640 1 -- 172.21.15.38:0/1605671864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28881f7f10 con 0x7f28880d3050 2024-04-25T08:42:08.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.190+0000 7f288f24c640 1 -- 172.21.15.38:0/1605671864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2888068e80 con 0x7f28880d3050 2024-04-25T08:42:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.190+0000 7f28857fa640 1 -- 172.21.15.38:0/1605671864 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f287401a070 con 0x7f28880d3050 2024-04-25T08:42:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.191+0000 7f28857fa640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f286c06b4d0 0x7f286c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.191+0000 7f28857fa640 1 -- 172.21.15.38:0/1605671864 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f287408a6d0 con 0x7f28880d3050 2024-04-25T08:42:08.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.191+0000 7f288cfe8640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f286c06b4d0 0x7f286c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:08.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.191+0000 7f288cfe8640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f286c06b4d0 0x7f286c06d990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f287c000f60 tx=0x7f287c007c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:08.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.195+0000 7f28857fa640 1 -- 172.21.15.38:0/1605671864 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2874059450 con 0x7f28880d3050 2024-04-25T08:42:08.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.449+0000 7f288f24c640 1 -- 172.21.15.38:0/1605671864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f28880cf3d0 con 0x7f28880d3050 2024-04-25T08:42:08.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.451+0000 7f28857fa640 1 -- 172.21.15.38:0/1605671864 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2874058d50 con 0x7f28880d3050 2024-04-25T08:42:08.443 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-04-25T08:42:08.443 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQCqFCpmuvcAIhAAChcTAr/GItke2bOv5oODvw== 2024-04-25T08:42:08.444 INFO:teuthology.orchestra.run.smithi038.stdout: caps mds = "allow" 2024-04-25T08:42:08.444 INFO:teuthology.orchestra.run.smithi038.stdout: caps mon = "allow r" 2024-04-25T08:42:08.444 INFO:teuthology.orchestra.run.smithi038.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:42:08.444 INFO:teuthology.orchestra.run.smithi038.stderr:updated caps for client.0 2024-04-25T08:42:08.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.453+0000 7f288f24c640 1 -- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f286c06b4d0 msgr2=0x7f286c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.453+0000 7f288f24c640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f286c06b4d0 0x7f286c06d990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f287c000f60 tx=0x7f287c007c00 comp rx=0 tx=0).stop 2024-04-25T08:42:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.453+0000 7f288f24c640 1 -- 172.21.15.38:0/1605671864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28880d3050 msgr2=0x7f288815b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.453+0000 7f288f24c640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28880d3050 0x7f288815b330 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f287400e900 tx=0x7f287400edc0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.453+0000 7f288f24c640 1 -- 172.21.15.38:0/1605671864 shutdown_connections 2024-04-25T08:42:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.453+0000 7f288f24c640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f288815db80 0x7f288815b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.453+0000 7f288f24c640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f28880d3050 0x7f288815b330 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.453+0000 7f288f24c640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f286c06b4d0 0x7f286c06d990 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.453+0000 7f288f24c640 1 --2- 172.21.15.38:0/1605671864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f28880cece0 0x7f28881f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.453+0000 7f288f24c640 1 -- 172.21.15.38:0/1605671864 >> 172.21.15.38:0/1605671864 conn(0x7f2888066d10 msgr2=0x7f28880b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:08.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.453+0000 7f288f24c640 1 -- 172.21.15.38:0/1605671864 shutdown_connections 2024-04-25T08:42:08.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.453+0000 7f288f24c640 1 -- 172.21.15.38:0/1605671864 wait complete. 2024-04-25T08:42:08.457 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.566+0000 7fc794db0640 1 Processor -- start 2024-04-25T08:42:08.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.566+0000 7fc794db0640 1 -- start start 2024-04-25T08:42:08.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.566+0000 7fc794db0640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc790099a70 0x7fc790099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.567+0000 7fc794db0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc79006cd90 0x7fc790167710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.567+0000 7fc794db0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79009a390 con 0x7fc7900cf470 2024-04-25T08:42:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.567+0000 7fc794db0640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7901614d0 con 0x7fc790099a70 2024-04-25T08:42:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.567+0000 7fc794db0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7900cd970 con 0x7fc79006cd90 2024-04-25T08:42:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.567+0000 7fc78e59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc790099a70 0x7fc790099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.567+0000 7fc78e59c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc790099a70 0x7fc790099e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52092/0 (socket says 172.21.15.38:52092) 2024-04-25T08:42:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.567+0000 7fc78e59c640 1 -- 172.21.15.38:0/2290447955 learned_addr learned my addr 172.21.15.38:0/2290447955 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.567+0000 7fc78ed9d640 1 --2- 172.21.15.38:0/2290447955 >> v2:172.21.15.107:3300/0 conn(0x7fc79006cd90 0x7fc790167710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.567+0000 7fc78e59c640 1 -- 172.21.15.38:0/2290447955 >> v2:172.21.15.107:3300/0 conn(0x7fc79006cd90 msgr2=0x7fc790167710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.567+0000 7fc78e59c640 1 --2- 172.21.15.38:0/2290447955 >> v2:172.21.15.107:3300/0 conn(0x7fc79006cd90 0x7fc790167710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.567+0000 7fc78e59c640 1 -- 172.21.15.38:0/2290447955 >> v1:172.21.15.107:6789/0 conn(0x7fc7900cf470 legacy=0x7fc7900cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:08.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.567+0000 7fc78e59c640 1 -- 172.21.15.38:0/2290447955 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7900ccd60 con 0x7fc790099a70 2024-04-25T08:42:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.568+0000 7fc78e59c640 1 --2- 172.21.15.38:0/2290447955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc790099a70 0x7fc790099e50 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fc790072a30 tx=0x7fc78002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5108f3d4815d92a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:08.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.568+0000 7fc78d59a640 1 -- 172.21.15.38:0/2290447955 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780037070 con 0x7fc790099a70 2024-04-25T08:42:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.568+0000 7fc78d59a640 1 -- 172.21.15.38:0/2290447955 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc780032440 con 0x7fc790099a70 2024-04-25T08:42:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.568+0000 7fc78d59a640 1 -- 172.21.15.38:0/2290447955 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780032810 con 0x7fc790099a70 2024-04-25T08:42:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.568+0000 7fc794db0640 1 -- 172.21.15.38:0/2290447955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc790099a70 msgr2=0x7fc790099e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.568+0000 7fc794db0640 1 --2- 172.21.15.38:0/2290447955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc790099a70 0x7fc790099e50 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fc790072a30 tx=0x7fc78002a270 comp rx=0 tx=0).stop 2024-04-25T08:42:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.569+0000 7fc794db0640 1 -- 172.21.15.38:0/2290447955 shutdown_connections 2024-04-25T08:42:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.569+0000 7fc794db0640 1 --2- 172.21.15.38:0/2290447955 >> v2:172.21.15.107:3300/0 conn(0x7fc79006cd90 0x7fc790167710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.569+0000 7fc794db0640 1 --2- 172.21.15.38:0/2290447955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc790099a70 0x7fc790099e50 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.569+0000 7fc794db0640 1 -- 172.21.15.38:0/2290447955 >> 172.21.15.38:0/2290447955 conn(0x7fc7900b3180 msgr2=0x7fc7900b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.569+0000 7fc794db0640 1 -- 172.21.15.38:0/2290447955 shutdown_connections 2024-04-25T08:42:08.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.569+0000 7fc794db0640 1 -- 172.21.15.38:0/2290447955 wait complete. 2024-04-25T08:42:08.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.570+0000 7fc794db0640 1 Processor -- start 2024-04-25T08:42:08.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.570+0000 7fc794db0640 1 -- start start 2024-04-25T08:42:08.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc794db0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc79006cd90 0x7fc79015e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:08.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc794db0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc790099a70 0x7fc7900b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:08.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc794db0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7900cf470 0x7fc7900b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:08.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc794db0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900cd600 con 0x7fc7900cf470 2024-04-25T08:42:08.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc794db0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900cd320 con 0x7fc79006cd90 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc794db0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7900d0780 con 0x7fc790099a70 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7900cf470 0x7fc7900b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7900cf470 0x7fc7900b4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52102/0 (socket says 172.21.15.38:52102) 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78ed9d640 1 -- 172.21.15.38:0/972115909 learned_addr learned my addr 172.21.15.38:0/972115909 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78dd9b640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc790099a70 0x7fc7900b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78e59c640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc79006cd90 0x7fc79015e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78ed9d640 1 -- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc790099a70 msgr2=0x7fc7900b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78ed9d640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc790099a70 0x7fc7900b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78ed9d640 1 -- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc79006cd90 msgr2=0x7fc79015e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78ed9d640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc79006cd90 0x7fc79015e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78ed9d640 1 -- 172.21.15.38:0/972115909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79015cf80 con 0x7fc7900cf470 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78dd9b640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc790099a70 0x7fc7900b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78e59c640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc79006cd90 0x7fc79015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc78ed9d640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7900cf470 0x7fc7900b4650 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fc788009fd0 tx=0x7fc78800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:08.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc77effd640 1 -- 172.21.15.38:0/972115909 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788014420 con 0x7fc7900cf470 2024-04-25T08:42:08.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc77effd640 1 -- 172.21.15.38:0/972115909 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788019070 con 0x7fc7900cf470 2024-04-25T08:42:08.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc794db0640 1 -- 172.21.15.38:0/972115909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79015d260 con 0x7fc7900cf470 2024-04-25T08:42:08.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.571+0000 7fc77effd640 1 -- 172.21.15.38:0/972115909 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788014800 con 0x7fc7900cf470 2024-04-25T08:42:08.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.572+0000 7fc794db0640 1 -- 172.21.15.38:0/972115909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79015d820 con 0x7fc7900cf470 2024-04-25T08:42:08.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.572+0000 7fc794db0640 1 -- 172.21.15.38:0/972115909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc790068e80 con 0x7fc7900cf470 2024-04-25T08:42:08.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.573+0000 7fc77effd640 1 -- 172.21.15.38:0/972115909 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7880149a0 con 0x7fc7900cf470 2024-04-25T08:42:08.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.573+0000 7fc77effd640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc77006b4d0 0x7fc77006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:08.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.573+0000 7fc77effd640 1 -- 172.21.15.38:0/972115909 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc78808ab50 con 0x7fc7900cf470 2024-04-25T08:42:08.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.574+0000 7fc78e59c640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc77006b4d0 0x7fc77006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:08.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.574+0000 7fc78e59c640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc77006b4d0 0x7fc77006d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc780034000 tx=0x7fc78002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:08.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.578+0000 7fc77effd640 1 -- 172.21.15.38:0/972115909 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc788059840 con 0x7fc7900cf470 2024-04-25T08:42:08.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.797+0000 7fc794db0640 1 -- 172.21.15.38:0/972115909 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7900b3680 con 0x7fc7900cf470 2024-04-25T08:42:08.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.797+0000 7fc77effd640 1 -- 172.21.15.38:0/972115909 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fc78805f030 con 0x7fc7900cf470 2024-04-25T08:42:08.790 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:08.790 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2682394530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2682394530},{"type":"v1","addr":"172.21.15.107:6837","nonce":2682394530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2810760276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2810760276},{"type":"v1","addr":"172.21.15.107:6835","nonce":2810760276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":10038,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/3318593502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3318593502},{"type":"v1","addr":"172.21.15.38:6833","nonce":3318593502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:42:04.044929+0000","modified":"2024-04-25T08:42:06.045913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10041":{"gid":10041,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/3513331661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3513331661},{"type":"v1","addr":"172.21.15.38:6835","nonce":3513331661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:42:08.790 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 88 2024-04-25T08:42:08.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.799+0000 7fc794db0640 1 -- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc77006b4d0 msgr2=0x7fc77006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.799+0000 7fc794db0640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc77006b4d0 0x7fc77006d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc780034000 tx=0x7fc78002e000 comp rx=0 tx=0).stop 2024-04-25T08:42:08.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.799+0000 7fc794db0640 1 -- 172.21.15.38:0/972115909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7900cf470 msgr2=0x7fc7900b4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:08.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.799+0000 7fc794db0640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7900cf470 0x7fc7900b4650 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fc788009fd0 tx=0x7fc78800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.799+0000 7fc794db0640 1 -- 172.21.15.38:0/972115909 shutdown_connections 2024-04-25T08:42:08.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.799+0000 7fc794db0640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7900cf470 0x7fc7900b4650 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.799+0000 7fc794db0640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc77006b4d0 0x7fc77006d990 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.799+0000 7fc794db0640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc790099a70 0x7fc7900b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.799+0000 7fc794db0640 1 --2- 172.21.15.38:0/972115909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc79006cd90 0x7fc79015e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:08.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.799+0000 7fc794db0640 1 -- 172.21.15.38:0/972115909 >> 172.21.15.38:0/972115909 conn(0x7fc7900b3180 msgr2=0x7fc7900cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:08.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.800+0000 7fc794db0640 1 -- 172.21.15.38:0/972115909 shutdown_connections 2024-04-25T08:42:08.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:08.800+0000 7fc794db0640 1 -- 172.21.15.38:0/972115909 wait complete. 2024-04-25T08:42:09.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:42:04.044929+0000', 'modified': '2024-04-25T08:42:06.045913+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10041}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10041': {'gid': 10041, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6835/3513331661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 3513331661}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 3513331661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:42:09.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:42:09.206 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:42:09.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.320+0000 7fe19bac4640 1 Processor -- start 2024-04-25T08:42:09.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.321+0000 7fe19bac4640 1 -- start start 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.321+0000 7fe19bac4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1940cd7e0 0x7fe194099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.321+0000 7fe19bac4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe1940b3c70 0x7fe194099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.321+0000 7fe19bac4640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1940cf7b0 con 0x7fe19406d670 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.321+0000 7fe19bac4640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1940cf560 con 0x7fe1940b3c70 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.321+0000 7fe19bac4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1940cd070 con 0x7fe1940cd7e0 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.322+0000 7fe199860640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe19406d670 0x7fe1940cff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:44388/0 (socket says 172.21.15.38:44388) 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.322+0000 7fe199860640 1 -- 172.21.15.38:0/1016300873 learned_addr learned my addr 172.21.15.38:0/1016300873 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.322+0000 7fe19905f640 1 --2- 172.21.15.38:0/1016300873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1940cd7e0 0x7fe194099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.322+0000 7fe19a061640 1 --2- 172.21.15.38:0/1016300873 >> v2:172.21.15.107:3300/0 conn(0x7fe1940b3c70 0x7fe194099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.322+0000 7fe19885e640 1 -- 172.21.15.38:0/1016300873 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409566905 0 0) 0x7fe1940cf7b0 con 0x7fe19406d670 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.322+0000 7fe19885e640 1 -- 172.21.15.38:0/1016300873 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe180003520 con 0x7fe19406d670 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.322+0000 7fe19905f640 1 -- 172.21.15.38:0/1016300873 >> v2:172.21.15.107:3300/0 conn(0x7fe1940b3c70 msgr2=0x7fe194099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:09.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.322+0000 7fe19905f640 1 --2- 172.21.15.38:0/1016300873 >> v2:172.21.15.107:3300/0 conn(0x7fe1940b3c70 0x7fe194099fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.322+0000 7fe19905f640 1 -- 172.21.15.38:0/1016300873 >> v1:172.21.15.38:6789/0 conn(0x7fe19406d670 legacy=0x7fe1940cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.322+0000 7fe19905f640 1 -- 172.21.15.38:0/1016300873 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1940cfa90 con 0x7fe1940cd7e0 2024-04-25T08:42:09.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.323+0000 7fe19905f640 1 --2- 172.21.15.38:0/1016300873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1940cd7e0 0x7fe194099a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe1840097a0 tx=0x7fe18402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c90d0b75caa5cf2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:09.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.323+0000 7fe19885e640 1 -- 172.21.15.38:0/1016300873 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184037070 con 0x7fe1940cd7e0 2024-04-25T08:42:09.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.323+0000 7fe19885e640 1 -- 172.21.15.38:0/1016300873 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe184032440 con 0x7fe1940cd7e0 2024-04-25T08:42:09.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.323+0000 7fe19885e640 1 -- 172.21.15.38:0/1016300873 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184032810 con 0x7fe1940cd7e0 2024-04-25T08:42:09.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.324+0000 7fe19bac4640 1 -- 172.21.15.38:0/1016300873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1940cd7e0 msgr2=0x7fe194099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.324+0000 7fe19bac4640 1 --2- 172.21.15.38:0/1016300873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1940cd7e0 0x7fe194099a70 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe1840097a0 tx=0x7fe18402a210 comp rx=0 tx=0).stop 2024-04-25T08:42:09.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.324+0000 7fe19bac4640 1 -- 172.21.15.38:0/1016300873 shutdown_connections 2024-04-25T08:42:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.324+0000 7fe19bac4640 1 --2- 172.21.15.38:0/1016300873 >> v2:172.21.15.107:3300/0 conn(0x7fe1940b3c70 0x7fe194099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.324+0000 7fe19bac4640 1 --2- 172.21.15.38:0/1016300873 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe1940cd7e0 0x7fe194099a70 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.324+0000 7fe19bac4640 1 -- 172.21.15.38:0/1016300873 >> 172.21.15.38:0/1016300873 conn(0x7fe1940b3150 msgr2=0x7fe1940b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.324+0000 7fe19bac4640 1 -- 172.21.15.38:0/1016300873 shutdown_connections 2024-04-25T08:42:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.324+0000 7fe19bac4640 1 -- 172.21.15.38:0/1016300873 wait complete. 2024-04-25T08:42:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.325+0000 7fe19bac4640 1 Processor -- start 2024-04-25T08:42:09.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.325+0000 7fe19bac4640 1 -- start start 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19bac4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe19406d670 0x7fe1940c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19bac4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1940b3c70 0x7fe194158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19bac4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1940cd7e0 0x7fe1941473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19bac4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19406bcf0 con 0x7fe1940b3c70 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19bac4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19406da50 con 0x7fe1940cd7e0 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19bac4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19406be70 con 0x7fe19406d670 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19905f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1940b3c70 0x7fe194158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19905f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1940b3c70 0x7fe194158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53286/0 (socket says 172.21.15.38:53286) 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19905f640 1 -- 172.21.15.38:0/2269088377 learned_addr learned my addr 172.21.15.38:0/2269088377 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe199860640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe19406d670 0x7fe1940c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19a061640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1940cd7e0 0x7fe1941473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19905f640 1 -- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe19406d670 msgr2=0x7fe1940c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19905f640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe19406d670 0x7fe1940c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19905f640 1 -- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1940cd7e0 msgr2=0x7fe1941473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19905f640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1940cd7e0 0x7fe1941473e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19905f640 1 -- 172.21.15.38:0/2269088377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe19415b650 con 0x7fe1940b3c70 2024-04-25T08:42:09.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19905f640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1940b3c70 0x7fe194158310 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fe184035040 tx=0x7fe1840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:09.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe18a7fc640 1 -- 172.21.15.38:0/2269088377 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184041030 con 0x7fe1940b3c70 2024-04-25T08:42:09.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe18a7fc640 1 -- 172.21.15.38:0/2269088377 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18403d4e0 con 0x7fe1940b3c70 2024-04-25T08:42:09.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe19bac4640 1 -- 172.21.15.38:0/2269088377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe19415b090 con 0x7fe1940b3c70 2024-04-25T08:42:09.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.326+0000 7fe18a7fc640 1 -- 172.21.15.38:0/2269088377 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184037040 con 0x7fe1940b3c70 2024-04-25T08:42:09.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.327+0000 7fe19bac4640 1 -- 172.21.15.38:0/2269088377 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1940c6c70 con 0x7fe1940b3c70 2024-04-25T08:42:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.328+0000 7fe18a7fc640 1 -- 172.21.15.38:0/2269088377 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe184044070 con 0x7fe1940b3c70 2024-04-25T08:42:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.328+0000 7fe18a7fc640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe17806b500 0x7fe17806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.328+0000 7fe18a7fc640 1 -- 172.21.15.38:0/2269088377 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1840aa800 con 0x7fe1940b3c70 2024-04-25T08:42:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.329+0000 7fe199860640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe17806b500 0x7fe17806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.329+0000 7fe199860640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe17806b500 0x7fe17806d9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe190005880 tx=0x7fe190009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.329+0000 7fe19e7c9740 1 Processor -- start 2024-04-25T08:42:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.330+0000 7fe19e7c9740 1 -- start start 2024-04-25T08:42:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.330+0000 7fe19e7c9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x55694247f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.330+0000 7fe19e7c9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x55694249d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.330+0000 7fe19e7c9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556942454f10 0x55694249b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.330+0000 7fe19e7c9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569423b7a10 con 0x55694249cad0 2024-04-25T08:42:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.330+0000 7fe19e7c9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x556942360b30 con 0x55694247f230 2024-04-25T08:42:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.330+0000 7fe19e7c9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569423b3c50 con 0x556942454f10 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.330+0000 7fe19905f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x55694249d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.330+0000 7fe19905f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x55694249d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53296/0 (socket says 172.21.15.38:53296) 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.330+0000 7fe19905f640 1 -- 172.21.15.38:0/1628595706 learned_addr learned my addr 172.21.15.38:0/1628595706 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.330+0000 7fe19a061640 1 --2- 172.21.15.38:0/1628595706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556942454f10 0x55694249b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.330+0000 7fe199860640 1 --2- 172.21.15.38:0/1628595706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x55694247f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.331+0000 7fe19905f640 1 -- 172.21.15.38:0/1628595706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556942454f10 msgr2=0x55694249b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.331+0000 7fe19905f640 1 --2- 172.21.15.38:0/1628595706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556942454f10 0x55694249b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.331+0000 7fe19905f640 1 -- 172.21.15.38:0/1628595706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 msgr2=0x55694247f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.331+0000 7fe19905f640 1 --2- 172.21.15.38:0/1628595706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x55694247f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.331+0000 7fe19905f640 1 -- 172.21.15.38:0/1628595706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556942375e10 con 0x55694249cad0 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.331+0000 7fe19905f640 1 --2- 172.21.15.38:0/1628595706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x55694249d020 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fe1840a8bd0 tx=0x7fe184078930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985fbb4208814eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.331+0000 7fe1777fe640 1 -- 172.21.15.38:0/1628595706 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184081630 con 0x55694249cad0 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.331+0000 7fe1777fe640 1 -- 172.21.15.38:0/1628595706 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1840772d0 con 0x55694249cad0 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.331+0000 7fe1777fe640 1 -- 172.21.15.38:0/1628595706 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1840811a0 con 0x55694249cad0 2024-04-25T08:42:09.326 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.332+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1628595706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 msgr2=0x55694249d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.332+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/1628595706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x55694249d020 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fe1840a8bd0 tx=0x7fe184078930 comp rx=0 tx=0).stop 2024-04-25T08:42:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.332+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1628595706 shutdown_connections 2024-04-25T08:42:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.332+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/1628595706 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x556942454f10 0x55694249b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.332+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/1628595706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x55694249d020 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.332+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/1628595706 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x55694247f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.332+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1628595706 >> 172.21.15.38:0/1628595706 conn(0x556942451840 msgr2=0x55694249a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.332+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1628595706 shutdown_connections 2024-04-25T08:42:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.332+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1628595706 wait complete. 2024-04-25T08:42:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.332+0000 7fe19e7c9740 1 -- start start 2024-04-25T08:42:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.332+0000 7fe19e7c9740 1 Processor -- start 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.333+0000 7fe19e7c9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x5569424780c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.333+0000 7fe19e7c9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x556942480900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.333+0000 7fe19e7c9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247aa50 0x55694247aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.333+0000 7fe19e7c9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556942360b30 con 0x55694249cad0 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.333+0000 7fe19e7c9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569423740c0 con 0x55694247f230 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.333+0000 7fe19e7c9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569423b7a10 con 0x55694247aa50 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.333+0000 7fe19e7c9740 10 client.0 _ll_register_callbacks cb 0x556942367ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.333+0000 7fe19905f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x556942480900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.333+0000 7fe19905f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x556942480900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53300/0 (socket says 172.21.15.38:53300) 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.333+0000 7fe19905f640 1 -- 172.21.15.38:0/4060379718 learned_addr learned my addr 172.21.15.38:0/4060379718 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.333+0000 7fe19a061640 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247aa50 0x55694247aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe199860640 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x5569424780c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19905f640 1 -- 172.21.15.38:0/4060379718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247aa50 msgr2=0x55694247aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19905f640 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247aa50 0x55694247aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19905f640 1 -- 172.21.15.38:0/4060379718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 msgr2=0x5569424780c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19905f640 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x5569424780c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19905f640 1 -- 172.21.15.38:0/4060379718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569420cc2d0 con 0x55694249cad0 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19a061640 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247aa50 0x55694247aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe199860640 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x5569424780c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19905f640 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x556942480900 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fe1840aab20 tx=0x7fe18402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef41972a3953f7b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe174ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe174ff9640 1 -- 172.21.15.38:0/4060379718 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184081630 con 0x55694249cad0 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe174ff9640 1 -- 172.21.15.38:0/4060379718 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1840813b0 con 0x55694249cad0 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19e7c9740 1 -- 172.21.15.38:0/4060379718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1840772d0 con 0x55694249cad0 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe174ff9640 1 -- 172.21.15.38:0/4060379718 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1840a8050 con 0x55694249cad0 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19e7c9740 10 client.10194 fetch_fsmap learned FSMap version 88 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19e7c9740 1 -- 172.21.15.38:0/4060379718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x5569421966b0 con 0x55694249cad0 2024-04-25T08:42:09.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe174ff9640 1 -- 172.21.15.38:0/4060379718 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1840772d0 con 0x55694249cad0 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe174ff9640 1 -- 172.21.15.38:0/4060379718 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 88) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe18407c030 con 0x55694249cad0 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19e7c9740 10 client.10194 fetch_fsmap finished waiting for FSMap version 88 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19e7c9740 10 client.10194 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19e7c9740 20 client.10194 populate_metadata read hostname 'smithi038' 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x5569424aa7f0 0x5569424aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19e7c9740 4 client.10194 mds_command: new command op to 10041 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.334+0000 7fe19e7c9740 1 -- 172.21.15.38:0/4060379718 --> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556942416920 con 0x5569424aa7f0 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.335+0000 7fe199860640 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x5569424aa7f0 0x5569424aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.335+0000 7fe174ff9640 10 client.10194 ms_handle_connect on v2:172.21.15.38:6834/3513331661 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.335+0000 7fe199860640 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x5569424aa7f0 0x5569424aabd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ec62cdfce3ff9f8 server_cookie=89059753780fd0bb in_seq=0 out_seq=0 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.335+0000 7fe174ff9640 1 -- 172.21.15.38:0/4060379718 <== mds.0 v2:172.21.15.38:6834/3513331661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556942416920 con 0x5569424aa7f0 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.335+0000 7fe174ff9640 10 client.10194 handle_command_reply: tid=0 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.335+0000 7fe19e7c9740 1 client.10194 shutdown 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.336+0000 7fe19e7c9740 1 -- 172.21.15.38:0/4060379718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 msgr2=0x556942480900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.336+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x556942480900 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7fe1840aab20 tx=0x7fe18402a210 comp rx=0 tx=0).stop 2024-04-25T08:42:09.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.336+0000 7fe19e7c9740 1 -- 172.21.15.38:0/4060379718 shutdown_connections 2024-04-25T08:42:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.336+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x5569424aa7f0 0x5569424aabd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.336+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247aa50 0x55694247aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.336+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x556942480900 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.336+0000 7fe174ff9640 0 client.10194 ms_handle_reset on v2:172.21.15.38:6834/3513331661 2024-04-25T08:42:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.336+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/4060379718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x5569424780c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.336+0000 7fe19e7c9740 1 -- 172.21.15.38:0/4060379718 >> 172.21.15.38:0/4060379718 conn(0x556942454f10 msgr2=0x55694244e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.336+0000 7fe19e7c9740 1 -- 172.21.15.38:0/4060379718 shutdown_connections 2024-04-25T08:42:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.336+0000 7fe19e7c9740 1 -- 172.21.15.38:0/4060379718 wait complete. 2024-04-25T08:42:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.336+0000 7fe19e7c9740 20 client.10194 trim_cache size 0 max 16384 2024-04-25T08:42:09.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.352+0000 7fe19e7c9740 1 Processor -- start 2024-04-25T08:42:09.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.352+0000 7fe19e7c9740 1 -- start start 2024-04-25T08:42:09.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.352+0000 7fe19e7c9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247aa50 0x55694244e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.352+0000 7fe19e7c9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x5569424acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.352+0000 7fe19e7c9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x55694247b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.352+0000 7fe19e7c9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19415aa70 con 0x55694249cad0 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.352+0000 7fe19e7c9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe194159e70 con 0x55694247f230 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.352+0000 7fe19e7c9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe19415a470 con 0x55694247aa50 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe19a061640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x55694247b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe19a061640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x55694247b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53306/0 (socket says 172.21.15.38:53306) 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe19a061640 1 -- 172.21.15.38:0/2118072336 learned_addr learned my addr 172.21.15.38:0/2118072336 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe199860640 1 --2- 172.21.15.38:0/2118072336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247aa50 0x55694244e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe19905f640 1 --2- 172.21.15.38:0/2118072336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x5569424acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe19a061640 1 -- 172.21.15.38:0/2118072336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247aa50 msgr2=0x55694244e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe19a061640 1 --2- 172.21.15.38:0/2118072336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247aa50 0x55694244e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe19a061640 1 -- 172.21.15.38:0/2118072336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 msgr2=0x5569424acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe19a061640 1 --2- 172.21.15.38:0/2118072336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x5569424acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe19a061640 1 -- 172.21.15.38:0/2118072336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569424817b0 con 0x55694249cad0 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe19a061640 1 --2- 172.21.15.38:0/2118072336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x55694247b4b0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fe18c016760 tx=0x7fe18c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6d1d1901719875 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe176ffd640 1 -- 172.21.15.38:0/2118072336 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c003da0 con 0x55694249cad0 2024-04-25T08:42:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe176ffd640 1 -- 172.21.15.38:0/2118072336 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c01b070 con 0x55694249cad0 2024-04-25T08:42:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.353+0000 7fe176ffd640 1 -- 172.21.15.38:0/2118072336 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18c004270 con 0x55694249cad0 2024-04-25T08:42:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.354+0000 7fe19e7c9740 1 -- 172.21.15.38:0/2118072336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 msgr2=0x55694247b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.354+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/2118072336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x55694247b4b0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fe18c016760 tx=0x7fe18c009750 comp rx=0 tx=0).stop 2024-04-25T08:42:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.354+0000 7fe19e7c9740 1 -- 172.21.15.38:0/2118072336 shutdown_connections 2024-04-25T08:42:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.354+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/2118072336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694249cad0 0x55694247b4b0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.354+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/2118072336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694247f230 0x5569424acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.354+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/2118072336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247aa50 0x55694244e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.354+0000 7fe19e7c9740 1 -- 172.21.15.38:0/2118072336 >> 172.21.15.38:0/2118072336 conn(0x556942454f10 msgr2=0x5569424552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.354+0000 7fe19e7c9740 1 -- 172.21.15.38:0/2118072336 shutdown_connections 2024-04-25T08:42:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.354+0000 7fe19e7c9740 1 -- 172.21.15.38:0/2118072336 wait complete. 2024-04-25T08:42:09.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.355+0000 7fe19e7c9740 1 -- start start 2024-04-25T08:42:09.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.355+0000 7fe19e7c9740 1 Processor -- start 2024-04-25T08:42:09.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.355+0000 7fe19e7c9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694247aa50 0x55694247b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.355+0000 7fe19e7c9740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247f230 0x556942482eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.355+0000 7fe19e7c9740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694249cad0 0x556942508250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.355+0000 7fe19e7c9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x556942360b30 con 0x55694247aa50 2024-04-25T08:42:09.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.355+0000 7fe19e7c9740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569423b7a10 con 0x55694249cad0 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.355+0000 7fe19e7c9740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569423740c0 con 0x55694247f230 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe19e7c9740 10 client.0 _ll_register_callbacks cb 0x55694244fc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe199860640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694247aa50 0x55694247b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe199860640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694247aa50 0x55694247b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53310/0 (socket says 172.21.15.38:53310) 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe199860640 1 -- 172.21.15.38:0/1307496951 learned_addr learned my addr 172.21.15.38:0/1307496951 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe19905f640 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247f230 0x556942482eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe19a061640 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694249cad0 0x556942508250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe199860640 1 -- 172.21.15.38:0/1307496951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247f230 msgr2=0x556942482eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe199860640 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247f230 0x556942482eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe199860640 1 -- 172.21.15.38:0/1307496951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694249cad0 msgr2=0x556942508250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe199860640 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694249cad0 0x556942508250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe199860640 1 -- 172.21.15.38:0/1307496951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569424833f0 con 0x55694247aa50 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe199860640 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694247aa50 0x55694247b270 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x556942482280 tx=0x7fe190000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891c13fa00590c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe174ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:42:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe174ff9640 1 -- 172.21.15.38:0/1307496951 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19000fe80 con 0x55694247aa50 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe174ff9640 1 -- 172.21.15.38:0/1307496951 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe190010040 con 0x55694247aa50 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1307496951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe18c01b070 con 0x55694247aa50 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe174ff9640 1 -- 172.21.15.38:0/1307496951 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=88) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe190037050 con 0x55694247aa50 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe19e7c9740 10 client.10200 fetch_fsmap learned FSMap version 88 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.356+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1307496951 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=88,monmap=2+}) v3 -- 0x55694247bb10 con 0x55694247aa50 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe174ff9640 1 -- 172.21.15.38:0/1307496951 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18c01b070 con 0x55694247aa50 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe174ff9640 1 -- 172.21.15.38:0/1307496951 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 88) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe19003a020 con 0x55694247aa50 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe19e7c9740 10 client.10200 fetch_fsmap finished waiting for FSMap version 88 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe19e7c9740 10 client.10200 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe19e7c9740 20 client.10200 populate_metadata read hostname 'smithi038' 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x5569424a91a0 0x556942503e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe19e7c9740 4 client.10200 mds_command: new command op to 10041 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1307496951 --> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5569423c4f50 con 0x5569424a91a0 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe19905f640 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x5569424a91a0 0x556942503e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe174ff9640 10 client.10200 ms_handle_connect on v2:172.21.15.38:6834/3513331661 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe19905f640 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x5569424a91a0 0x556942503e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2896a5f8d27e25c server_cookie=1e258d466e264a74 in_seq=0 out_seq=0 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe174ff9640 1 -- 172.21.15.38:0/1307496951 <== mds.0 v2:172.21.15.38:6834/3513331661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5569423c4f50 con 0x5569424a91a0 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe174ff9640 10 client.10200 handle_command_reply: tid=0 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.357+0000 7fe19e7c9740 1 client.10200 shutdown 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1307496951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694247aa50 msgr2=0x55694247b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694247aa50 0x55694247b270 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x556942482280 tx=0x7fe190000bc0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1307496951 shutdown_connections 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x5569424a91a0 0x556942503e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55694249cad0 0x556942508250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x55694247f230 0x556942482eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe174ff9640 0 client.10200 ms_handle_reset on v2:172.21.15.38:6834/3513331661 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19e7c9740 1 --2- 172.21.15.38:0/1307496951 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55694247aa50 0x55694247b270 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1307496951 >> 172.21.15.38:0/1307496951 conn(0x556942454f10 msgr2=0x5569424acbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1307496951 shutdown_connections 2024-04-25T08:42:09.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19e7c9740 1 -- 172.21.15.38:0/1307496951 wait complete. 2024-04-25T08:42:09.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19e7c9740 20 client.10200 trim_cache size 0 max 16384 2024-04-25T08:42:09.356 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:42:09.356 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:42:09.356 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:42:09.356 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 10041, 2024-04-25T08:42:09.356 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:42:09.356 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:42:09.356 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:42:09.356 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 4.6860798499999996, 2024-04-25T08:42:09.356 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 88, 2024-04-25T08:42:09.356 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 143, 2024-04-25T08:42:09.356 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 143, 2024-04-25T08:42:09.357 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 9.6944823800000002 2024-04-25T08:42:09.357 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:42:09.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19bac4640 1 -- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe17806b500 msgr2=0x7fe17806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19bac4640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe17806b500 0x7fe17806d9c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe190005880 tx=0x7fe190009040 comp rx=0 tx=0).stop 2024-04-25T08:42:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19bac4640 1 -- 172.21.15.38:0/2269088377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1940b3c70 msgr2=0x7fe194158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19bac4640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1940b3c70 0x7fe194158310 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fe184035040 tx=0x7fe1840046c0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19bac4640 1 -- 172.21.15.38:0/2269088377 shutdown_connections 2024-04-25T08:42:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19bac4640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe1940cd7e0 0x7fe1941473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19bac4640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe1940b3c70 0x7fe194158310 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19bac4640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe17806b500 0x7fe17806d9c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19bac4640 1 --2- 172.21.15.38:0/2269088377 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe19406d670 0x7fe1940c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19bac4640 1 -- 172.21.15.38:0/2269088377 >> 172.21.15.38:0/2269088377 conn(0x7fe1940b3150 msgr2=0x7fe1940b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19bac4640 1 -- 172.21.15.38:0/2269088377 shutdown_connections 2024-04-25T08:42:09.358 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.358+0000 7fe19bac4640 1 -- 172.21.15.38:0/2269088377 wait complete. 2024-04-25T08:42:09.362 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:42:09.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.470+0000 7f7eaf942640 1 Processor -- start 2024-04-25T08:42:09.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.470+0000 7f7eaf942640 1 -- start start 2024-04-25T08:42:09.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eaf942640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7ea80b5710 0x7f7ea8161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eaf942640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ea815ddf0 0x7f7ea80b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eaf942640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7ea80b5af0 0x7f7ea806bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eaf942640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea80b9930 con 0x7f7ea80b5af0 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eaf942640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea8161e00 con 0x7f7ea80b5710 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eaf942640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea81620e0 con 0x7f7ea815ddf0 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eadedf640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7ea80b5af0 0x7f7ea806bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eadedf640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7ea80b5af0 0x7f7ea806bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53316/0 (socket says 172.21.15.38:53316) 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eadedf640 1 -- 172.21.15.38:0/1738178564 learned_addr learned my addr 172.21.15.38:0/1738178564 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eacedd640 1 --2- 172.21.15.38:0/1738178564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ea815ddf0 0x7f7ea80b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7ead6de640 1 --2- 172.21.15.38:0/1738178564 >> v2:172.21.15.107:3300/0 conn(0x7f7ea80b5710 0x7f7ea8161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eadedf640 1 -- 172.21.15.38:0/1738178564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ea815ddf0 msgr2=0x7f7ea80b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eadedf640 1 --2- 172.21.15.38:0/1738178564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ea815ddf0 0x7f7ea80b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eadedf640 1 -- 172.21.15.38:0/1738178564 >> v2:172.21.15.107:3300/0 conn(0x7f7ea80b5710 msgr2=0x7f7ea8161700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:09.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eadedf640 1 --2- 172.21.15.38:0/1738178564 >> v2:172.21.15.107:3300/0 conn(0x7f7ea80b5710 0x7f7ea8161700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7eadedf640 1 -- 172.21.15.38:0/1738178564 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea81623c0 con 0x7f7ea80b5af0 2024-04-25T08:42:09.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.471+0000 7f7ead6de640 1 --2- 172.21.15.38:0/1738178564 >> v2:172.21.15.107:3300/0 conn(0x7f7ea80b5710 0x7f7ea8161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:09.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.472+0000 7f7eadedf640 1 --2- 172.21.15.38:0/1738178564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea80b5af0 0x7f7ea806bff0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f7ea808be40 tx=0x7f7ea002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd7699a9151443b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:09.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.472+0000 7f7e9ffff640 1 -- 172.21.15.38:0/1738178564 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0036070 con 0x7f7ea80b5af0 2024-04-25T08:42:09.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.472+0000 7f7e9ffff640 1 -- 172.21.15.38:0/1738178564 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea0031600 con 0x7f7ea80b5af0 2024-04-25T08:42:09.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.472+0000 7f7e9ffff640 1 -- 172.21.15.38:0/1738178564 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00319d0 con 0x7f7ea80b5af0 2024-04-25T08:42:09.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.473+0000 7f7eaf942640 1 -- 172.21.15.38:0/1738178564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea80b5af0 msgr2=0x7f7ea806bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.473+0000 7f7eaf942640 1 --2- 172.21.15.38:0/1738178564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea80b5af0 0x7f7ea806bff0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f7ea808be40 tx=0x7f7ea002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.473+0000 7f7eaf942640 1 -- 172.21.15.38:0/1738178564 shutdown_connections 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.473+0000 7f7eaf942640 1 --2- 172.21.15.38:0/1738178564 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea80b5af0 0x7f7ea806bff0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.473+0000 7f7eaf942640 1 --2- 172.21.15.38:0/1738178564 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ea815ddf0 0x7f7ea80b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.473+0000 7f7eaf942640 1 --2- 172.21.15.38:0/1738178564 >> v2:172.21.15.107:3300/0 conn(0x7f7ea80b5710 0x7f7ea8161700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.473+0000 7f7eaf942640 1 -- 172.21.15.38:0/1738178564 >> 172.21.15.38:0/1738178564 conn(0x7f7ea80d07c0 msgr2=0x7f7ea80d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.473+0000 7f7eaf942640 1 -- 172.21.15.38:0/1738178564 shutdown_connections 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.473+0000 7f7eaf942640 1 -- 172.21.15.38:0/1738178564 wait complete. 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.474+0000 7f7eaf942640 1 Processor -- start 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.474+0000 7f7eaf942640 1 -- start start 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eaf942640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea80b5710 0x7f7ea8156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eaf942640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea80b5af0 0x7f7ea8160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eaf942640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ea815ddf0 0x7f7ea815f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eaf942640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea806c6b0 con 0x7f7ea80b5af0 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eaf942640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea806c530 con 0x7f7ea80b5710 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eaf942640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea806c830 con 0x7f7ea815ddf0 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eacedd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea80b5af0 0x7f7ea8160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eacedd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea80b5af0 0x7f7ea8160850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53330/0 (socket says 172.21.15.38:53330) 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eacedd640 1 -- 172.21.15.38:0/2899292449 learned_addr learned my addr 172.21.15.38:0/2899292449 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7ead6de640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea80b5710 0x7f7ea8156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eadedf640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ea815ddf0 0x7f7ea815f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eacedd640 1 -- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ea815ddf0 msgr2=0x7f7ea815f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eacedd640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ea815ddf0 0x7f7ea815f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eacedd640 1 -- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea80b5710 msgr2=0x7f7ea8156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eacedd640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea80b5710 0x7f7ea8156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eacedd640 1 -- 172.21.15.38:0/2899292449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea80c73f0 con 0x7f7ea80b5af0 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7eacedd640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea80b5af0 0x7f7ea8160850 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800d5c0 tx=0x7f7e9800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.475+0000 7f7e9dffb640 1 -- 172.21.15.38:0/2899292449 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e980125f0 con 0x7f7ea80b5af0 2024-04-25T08:42:09.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.476+0000 7f7e9dffb640 1 -- 172.21.15.38:0/2899292449 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e98019070 con 0x7f7ea80b5af0 2024-04-25T08:42:09.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.476+0000 7f7eaf942640 1 -- 172.21.15.38:0/2899292449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea80c76d0 con 0x7f7ea80b5af0 2024-04-25T08:42:09.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.476+0000 7f7e9dffb640 1 -- 172.21.15.38:0/2899292449 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e980129d0 con 0x7f7ea80b5af0 2024-04-25T08:42:09.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.476+0000 7f7eaf942640 1 -- 172.21.15.38:0/2899292449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea80c7c90 con 0x7f7ea80b5af0 2024-04-25T08:42:09.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.477+0000 7f7eaf942640 1 -- 172.21.15.38:0/2899292449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea8068e80 con 0x7f7ea80b5af0 2024-04-25T08:42:09.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.477+0000 7f7e9dffb640 1 -- 172.21.15.38:0/2899292449 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e98012b70 con 0x7f7ea80b5af0 2024-04-25T08:42:09.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.478+0000 7f7e9dffb640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e8c06b4d0 0x7f7e8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:09.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.478+0000 7f7e9dffb640 1 -- 172.21.15.38:0/2899292449 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7e9808a660 con 0x7f7ea80b5af0 2024-04-25T08:42:09.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.478+0000 7f7ead6de640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e8c06b4d0 0x7f7e8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:09.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.478+0000 7f7ead6de640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e8c06b4d0 0x7f7e8c06d990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7ea81618b0 tx=0x7f7ea4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:09.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.482+0000 7f7e9dffb640 1 -- 172.21.15.38:0/2899292449 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e98058cf0 con 0x7f7ea80b5af0 2024-04-25T08:42:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.702+0000 7f7eaf942640 1 -- 172.21.15.38:0/2899292449 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7ea80b3fe0 con 0x7f7ea80b5af0 2024-04-25T08:42:09.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.702+0000 7f7e9dffb640 1 -- 172.21.15.38:0/2899292449 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v88) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7e9805cd90 con 0x7f7ea80b5af0 2024-04-25T08:42:09.695 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:42:09.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.705+0000 7f7eaf942640 1 -- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e8c06b4d0 msgr2=0x7f7e8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.705+0000 7f7eaf942640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e8c06b4d0 0x7f7e8c06d990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7ea81618b0 tx=0x7f7ea4008040 comp rx=0 tx=0).stop 2024-04-25T08:42:09.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.705+0000 7f7eaf942640 1 -- 172.21.15.38:0/2899292449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea80b5af0 msgr2=0x7f7ea8160850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:09.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.705+0000 7f7eaf942640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea80b5af0 0x7f7ea8160850 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800d5c0 tx=0x7f7e9800da80 comp rx=0 tx=0).stop 2024-04-25T08:42:09.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.705+0000 7f7eaf942640 1 -- 172.21.15.38:0/2899292449 shutdown_connections 2024-04-25T08:42:09.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.705+0000 7f7eaf942640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7e8c06b4d0 0x7f7e8c06d990 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.705+0000 7f7eaf942640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7ea815ddf0 0x7f7ea815f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.705+0000 7f7eaf942640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ea80b5af0 0x7f7ea8160850 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.705+0000 7f7eaf942640 1 --2- 172.21.15.38:0/2899292449 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ea80b5710 0x7f7ea8156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:09.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.705+0000 7f7eaf942640 1 -- 172.21.15.38:0/2899292449 >> 172.21.15.38:0/2899292449 conn(0x7f7ea80d07c0 msgr2=0x7f7ea80b45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:09.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.705+0000 7f7eaf942640 1 -- 172.21.15.38:0/2899292449 shutdown_connections 2024-04-25T08:42:09.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:09.705+0000 7f7eaf942640 1 -- 172.21.15.38:0/2899292449 wait complete. 2024-04-25T08:42:09.708 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:42:09.708 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:42:09.708 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-04-25T08:42:09.708 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:42:09.708 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:42:09.708 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:42:09.708 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:42:09.709 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:42:09.709 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:42:09.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:09.709 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-04-25T08:42:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:42:09.725 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:42:09.725 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:42:09.725 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:09.725 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-04-25T08:42:09.725 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:42:09.725 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:09.725 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-04-25T08:42:09.725 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:42:09.725 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:09.725 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42107sec preferred_lft 42107sec 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link noprefixroute 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:09.726 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::440f:97ff:fe63:87b/64 scope link 2024-04-25T08:42:09.727 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:42:09.727 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether ea:23:76:ab:b1:e9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:42:09.727 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::e823:76ff:feab:b1e9/64 scope link 2024-04-25T08:42:09.727 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:09.727 INFO:teuthology.orchestra.run.smithi038.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:42:09.727 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 3a:c7:38:2e:6c:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:42:09.727 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::38c7:38ff:fe2e:6c92/64 scope link 2024-04-25T08:42:09.727 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:42:09.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:09.727 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:42:09.788 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-04-25T08:42:09.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:09.790 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-04-25T08:42:09.846 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-04-25T08:42:09.846 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:42:09.846 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:42:09.846 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:42:09.846 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:42:09.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:09.846 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec bash -c ' 2024-04-25T08:42:09.846 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:09.846 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:42:09.846 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:42:09.847 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:42:09.847 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:09.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:09.932 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:42:09.947 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:42:09.947 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:42:09.947 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:42:09.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:09.948 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-04-25T08:42:09.986 INFO:tasks.ceph.mds.c.smithi038.stderr:2024-04-25T08:42:09.991+0000 7f40e7b76640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:42:10.009 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:42:10.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:42:10.010 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:42:10.065 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:42:10.066 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:42:10.066 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:42:10.121 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:42:10.186 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:42:10.242 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:42:10.242 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:42:10.242 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:42:10.242 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:42:10.242 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:42:10.243 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:42:10.244 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:42:10.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:10.245 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:42:10.297 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:42:10.297 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:42:10.339 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-04-25T08:42:10.363 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:42:10.420 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-04-25T08:42:10.420 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-04-25T08:42:10.420 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-04-25T08:42:10.420 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-04-25T08:42:10.420 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-04-25T08:42:10.420 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-04-25T08:42:10.420 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-04-25T08:42:10.420 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-04-25T08:42:10.420 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:42:10.421 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:42:10.422 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:42:10.422 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-04-25T08:42:10.422 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-04-25T08:42:10.422 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:42:10.422 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:42:10.422 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:42:10.422 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:42:10.422 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-04-25T08:42:10.422 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:42:10.422 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:42:10.422 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:42:10.422 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:42:10.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:10.423 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-04-25T08:42:10.423 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:2024-04-25T08:42:10.429+0000 7f71c0c23480 -1 init, newargv = 0x5585724d43d0 newargc=15 2024-04-25T08:42:10.424 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[99897]: starting ceph client 2024-04-25T08:42:10.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[99897]: starting fuse 2024-04-25T08:42:10.439 INFO:teuthology.orchestra.run.smithi038.stdout:43 2024-04-25T08:42:10.439 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:42:10.439 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:42:10.439 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-04-25T08:42:10.439 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-04-25T08:42:10.439 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-04-25T08:42:10.439 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-04-25T08:42:10.439 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:42:10.439 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-04-25T08:42:10.440 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-04-25T08:42:10.441 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-04-25T08:42:10.441 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:42:10.441 DEBUG:teuthology.orchestra.run.smithi038:> 2024-04-25T08:42:10.441 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.0")) 2024-04-25T08:42:10.441 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-04-25T08:42:10.558 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.0.99897.asok 2024-04-25T08:42:10.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:42:10.569 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.99897.asok 2024-04-25T08:42:10.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:10.570 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.99897.asok status 2024-04-25T08:42:10.705 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "0", 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "99897", 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 10212, 2024-04-25T08:42:10.706 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 10212 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 499056632 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 499056632 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-04-25T08:42:10.707 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.10212 192.168.0.1:0/499056632", 2024-04-25T08:42:10.708 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/499056632", 2024-04-25T08:42:10.708 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-04-25T08:42:10.708 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 88, 2024-04-25T08:42:10.708 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 143, 2024-04-25T08:42:10.708 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:42:10.708 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-04-25T08:42:10.708 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-04-25T08:42:10.708 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:42:10.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:10.714 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-04-25T08:42:10.714 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:42:10.730 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-04-25T08:42:10.730 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:42:10.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:10.731 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:42:10.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:10.796 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:10.796 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:10.797 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T08:42:10.797 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:10.797 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:10.797 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:10.797 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:10.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:10.873 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00952105 s, 110 MB/s 2024-04-25T08:42:10.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:10.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:10.882 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007255 s, 145 MB/s 2024-04-25T08:42:10.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:10.895 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:10.895 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106636 s, 98.3 MB/s 2024-04-25T08:42:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733987 s, 143 MB/s 2024-04-25T08:42:10.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:10.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:10.917 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101155 s, 104 MB/s 2024-04-25T08:42:10.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:10.927 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:10.927 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777612 s, 135 MB/s 2024-04-25T08:42:10.939 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:10.939 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:10.939 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101321 s, 103 MB/s 2024-04-25T08:42:10.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:10.951 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:10.951 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00946869 s, 111 MB/s 2024-04-25T08:42:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:10.961 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838306 s, 125 MB/s 2024-04-25T08:42:10.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:10.972 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:10.972 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886411 s, 118 MB/s 2024-04-25T08:42:10.983 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:10.983 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:10.983 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882173 s, 119 MB/s 2024-04-25T08:42:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:10.993 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799385 s, 131 MB/s 2024-04-25T08:42:11.003 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.003 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.003 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071852 s, 146 MB/s 2024-04-25T08:42:11.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.013 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.013 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815414 s, 129 MB/s 2024-04-25T08:42:11.024 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.024 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.024 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088258 s, 119 MB/s 2024-04-25T08:42:11.034 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.035 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.035 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807575 s, 130 MB/s 2024-04-25T08:42:11.046 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.046 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.046 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939817 s, 112 MB/s 2024-04-25T08:42:11.057 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.057 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.057 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907098 s, 116 MB/s 2024-04-25T08:42:11.068 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.068 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.069 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897458 s, 117 MB/s 2024-04-25T08:42:11.079 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.079 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.079 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823531 s, 127 MB/s 2024-04-25T08:42:11.089 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.089 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.090 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834912 s, 126 MB/s 2024-04-25T08:42:11.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.101 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917095 s, 114 MB/s 2024-04-25T08:42:11.111 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.112 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.112 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857873 s, 122 MB/s 2024-04-25T08:42:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.122 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804381 s, 130 MB/s 2024-04-25T08:42:11.132 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.132 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.132 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814869 s, 129 MB/s 2024-04-25T08:42:11.142 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.143 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.143 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839475 s, 125 MB/s 2024-04-25T08:42:11.153 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.154 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.154 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874558 s, 120 MB/s 2024-04-25T08:42:11.164 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.164 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.165 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086091 s, 122 MB/s 2024-04-25T08:42:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071085 s, 148 MB/s 2024-04-25T08:42:11.184 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.184 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.184 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831882 s, 126 MB/s 2024-04-25T08:42:11.195 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.195 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.195 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844208 s, 124 MB/s 2024-04-25T08:42:11.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.207 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868059 s, 121 MB/s 2024-04-25T08:42:11.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:11.207 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:11.207 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:11.207 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T08:42:11.207 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:11.207 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:11.207 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:11.207 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:11.275 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.275 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.275 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804359 s, 130 MB/s 2024-04-25T08:42:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.286 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851675 s, 123 MB/s 2024-04-25T08:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.296 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081703 s, 128 MB/s 2024-04-25T08:42:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.307 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865206 s, 121 MB/s 2024-04-25T08:42:11.317 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.318 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.318 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849879 s, 123 MB/s 2024-04-25T08:42:11.328 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.328 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.328 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817987 s, 128 MB/s 2024-04-25T08:42:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.338 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826546 s, 127 MB/s 2024-04-25T08:42:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.348 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768673 s, 136 MB/s 2024-04-25T08:42:11.358 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.358 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.358 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788472 s, 133 MB/s 2024-04-25T08:42:11.368 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.368 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.368 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780528 s, 134 MB/s 2024-04-25T08:42:11.379 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.379 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.379 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855936 s, 123 MB/s 2024-04-25T08:42:11.389 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.389 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.389 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782015 s, 134 MB/s 2024-04-25T08:42:11.398 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.398 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.398 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742401 s, 141 MB/s 2024-04-25T08:42:11.409 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.409 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.409 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840524 s, 125 MB/s 2024-04-25T08:42:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.419 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791592 s, 132 MB/s 2024-04-25T08:42:11.429 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.429 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.429 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760392 s, 138 MB/s 2024-04-25T08:42:11.439 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.439 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.440 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081102 s, 129 MB/s 2024-04-25T08:42:11.450 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.450 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.450 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776896 s, 135 MB/s 2024-04-25T08:42:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.460 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771899 s, 136 MB/s 2024-04-25T08:42:11.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.470 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.470 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766069 s, 137 MB/s 2024-04-25T08:42:11.481 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.481 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.481 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833965 s, 126 MB/s 2024-04-25T08:42:11.490 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.490 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.490 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716772 s, 146 MB/s 2024-04-25T08:42:11.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.500 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748315 s, 140 MB/s 2024-04-25T08:42:11.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.510 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.510 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786358 s, 133 MB/s 2024-04-25T08:42:11.521 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.521 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.521 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893188 s, 117 MB/s 2024-04-25T08:42:11.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.531 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777369 s, 135 MB/s 2024-04-25T08:42:11.540 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.540 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.540 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717929 s, 146 MB/s 2024-04-25T08:42:11.550 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.550 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.550 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744517 s, 141 MB/s 2024-04-25T08:42:11.559 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.559 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.559 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689555 s, 152 MB/s 2024-04-25T08:42:11.569 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.569 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.569 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071509 s, 147 MB/s 2024-04-25T08:42:11.579 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.579 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.579 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796352 s, 132 MB/s 2024-04-25T08:42:11.590 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.590 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.590 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723278 s, 145 MB/s 2024-04-25T08:42:11.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:11.590 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:11.590 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:11.591 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T08:42:11.591 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:11.591 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:11.591 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:11.591 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:11.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.659 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794191 s, 132 MB/s 2024-04-25T08:42:11.668 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.668 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.668 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692241 s, 151 MB/s 2024-04-25T08:42:11.677 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.677 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.678 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723095 s, 145 MB/s 2024-04-25T08:42:11.689 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.689 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.689 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918093 s, 114 MB/s 2024-04-25T08:42:11.698 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.698 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.699 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073228 s, 143 MB/s 2024-04-25T08:42:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.709 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792817 s, 132 MB/s 2024-04-25T08:42:11.718 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.718 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.719 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771392 s, 136 MB/s 2024-04-25T08:42:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.730 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884774 s, 119 MB/s 2024-04-25T08:42:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.740 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853694 s, 123 MB/s 2024-04-25T08:42:11.750 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.750 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.750 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726454 s, 144 MB/s 2024-04-25T08:42:11.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.760 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771439 s, 136 MB/s 2024-04-25T08:42:11.769 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.770 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.770 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761715 s, 138 MB/s 2024-04-25T08:42:11.779 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.779 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.779 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753447 s, 139 MB/s 2024-04-25T08:42:11.789 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.789 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.789 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793092 s, 132 MB/s 2024-04-25T08:42:11.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.799 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786016 s, 133 MB/s 2024-04-25T08:42:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.810 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795404 s, 132 MB/s 2024-04-25T08:42:11.818 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708966 s, 148 MB/s 2024-04-25T08:42:11.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.829 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769285 s, 136 MB/s 2024-04-25T08:42:11.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.839 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801252 s, 131 MB/s 2024-04-25T08:42:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.848 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762397 s, 138 MB/s 2024-04-25T08:42:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.858 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796845 s, 132 MB/s 2024-04-25T08:42:11.868 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.869 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794435 s, 132 MB/s 2024-04-25T08:42:11.878 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.879 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786828 s, 133 MB/s 2024-04-25T08:42:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.889 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846013 s, 124 MB/s 2024-04-25T08:42:11.899 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.899 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.899 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749386 s, 140 MB/s 2024-04-25T08:42:11.909 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.909 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.909 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801918 s, 131 MB/s 2024-04-25T08:42:11.919 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.920 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848667 s, 124 MB/s 2024-04-25T08:42:11.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.931 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865259 s, 121 MB/s 2024-04-25T08:42:11.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.941 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824321 s, 127 MB/s 2024-04-25T08:42:11.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.951 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728917 s, 144 MB/s 2024-04-25T08:42:11.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.961 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794038 s, 132 MB/s 2024-04-25T08:42:11.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:11.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:11.970 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741042 s, 142 MB/s 2024-04-25T08:42:11.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:11.971 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:11.972 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:11.972 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T08:42:11.972 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:11.972 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:11.972 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:11.972 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.040 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008006 s, 131 MB/s 2024-04-25T08:42:12.049 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.049 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.049 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714083 s, 147 MB/s 2024-04-25T08:42:12.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.060 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.060 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861115 s, 122 MB/s 2024-04-25T08:42:12.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.070 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.070 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776025 s, 135 MB/s 2024-04-25T08:42:12.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.080 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.080 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769083 s, 136 MB/s 2024-04-25T08:42:12.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.090 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.090 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846695 s, 124 MB/s 2024-04-25T08:42:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.101 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869386 s, 121 MB/s 2024-04-25T08:42:12.111 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.111 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.112 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810017 s, 129 MB/s 2024-04-25T08:42:12.121 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.121 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.122 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785263 s, 134 MB/s 2024-04-25T08:42:12.132 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.132 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.132 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834486 s, 126 MB/s 2024-04-25T08:42:12.142 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.142 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.143 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834443 s, 126 MB/s 2024-04-25T08:42:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.152 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701148 s, 150 MB/s 2024-04-25T08:42:12.161 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.161 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.161 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748195 s, 140 MB/s 2024-04-25T08:42:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.171 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713062 s, 147 MB/s 2024-04-25T08:42:12.181 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.181 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.181 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851545 s, 123 MB/s 2024-04-25T08:42:12.192 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.192 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.192 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817193 s, 128 MB/s 2024-04-25T08:42:12.202 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.202 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.202 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762306 s, 138 MB/s 2024-04-25T08:42:12.212 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.212 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.212 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780513 s, 134 MB/s 2024-04-25T08:42:12.222 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.222 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.223 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864155 s, 121 MB/s 2024-04-25T08:42:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.233 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835635 s, 125 MB/s 2024-04-25T08:42:12.243 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.243 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.243 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762513 s, 138 MB/s 2024-04-25T08:42:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.254 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853307 s, 123 MB/s 2024-04-25T08:42:12.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.263 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745287 s, 141 MB/s 2024-04-25T08:42:12.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.274 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.274 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826236 s, 127 MB/s 2024-04-25T08:42:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.283 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760376 s, 138 MB/s 2024-04-25T08:42:12.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.293 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721422 s, 145 MB/s 2024-04-25T08:42:12.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.304 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845829 s, 124 MB/s 2024-04-25T08:42:12.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.313 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730845 s, 143 MB/s 2024-04-25T08:42:12.324 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.324 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.324 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853555 s, 123 MB/s 2024-04-25T08:42:12.335 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.335 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.335 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848651 s, 124 MB/s 2024-04-25T08:42:12.345 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.345 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.345 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828009 s, 127 MB/s 2024-04-25T08:42:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.356 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776305 s, 135 MB/s 2024-04-25T08:42:12.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:12.356 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:12.357 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:12.357 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T08:42:12.357 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:12.357 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:12.357 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:12.357 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:12.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.424 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007712 s, 136 MB/s 2024-04-25T08:42:12.434 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.434 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.434 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763951 s, 137 MB/s 2024-04-25T08:42:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782218 s, 134 MB/s 2024-04-25T08:42:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703577 s, 149 MB/s 2024-04-25T08:42:12.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.463 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.463 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793 s, 132 MB/s 2024-04-25T08:42:12.473 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.473 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.473 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777539 s, 135 MB/s 2024-04-25T08:42:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.483 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780895 s, 134 MB/s 2024-04-25T08:42:12.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.494 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851326 s, 123 MB/s 2024-04-25T08:42:12.504 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.504 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.504 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865436 s, 121 MB/s 2024-04-25T08:42:12.515 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.515 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.515 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836412 s, 125 MB/s 2024-04-25T08:42:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.526 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848385 s, 124 MB/s 2024-04-25T08:42:12.537 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.537 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.537 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838386 s, 125 MB/s 2024-04-25T08:42:12.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.547 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782616 s, 134 MB/s 2024-04-25T08:42:12.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.558 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.558 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767017 s, 137 MB/s 2024-04-25T08:42:12.568 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.568 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.568 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755393 s, 139 MB/s 2024-04-25T08:42:12.579 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.579 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.579 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863872 s, 121 MB/s 2024-04-25T08:42:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.589 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781481 s, 134 MB/s 2024-04-25T08:42:12.599 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.599 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.599 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725023 s, 145 MB/s 2024-04-25T08:42:12.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.610 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.610 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795103 s, 132 MB/s 2024-04-25T08:42:12.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.620 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800989 s, 131 MB/s 2024-04-25T08:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.630 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.631 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774245 s, 135 MB/s 2024-04-25T08:42:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.642 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873027 s, 120 MB/s 2024-04-25T08:42:12.653 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.653 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.653 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794417 s, 132 MB/s 2024-04-25T08:42:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.662 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749828 s, 140 MB/s 2024-04-25T08:42:12.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.672 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773829 s, 136 MB/s 2024-04-25T08:42:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.682 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788796 s, 133 MB/s 2024-04-25T08:42:12.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.692 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760712 s, 138 MB/s 2024-04-25T08:42:12.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.702 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.703 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079509 s, 132 MB/s 2024-04-25T08:42:12.712 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.713 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774975 s, 135 MB/s 2024-04-25T08:42:12.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.722 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069803 s, 150 MB/s 2024-04-25T08:42:12.732 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.732 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.732 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870124 s, 121 MB/s 2024-04-25T08:42:12.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.743 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.743 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773576 s, 136 MB/s 2024-04-25T08:42:12.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:12.744 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:12.744 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:12.744 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T08:42:12.744 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:12.744 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:12.744 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:12.744 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:12.812 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.812 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.812 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754698 s, 139 MB/s 2024-04-25T08:42:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.821 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727342 s, 144 MB/s 2024-04-25T08:42:12.831 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.831 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.831 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766472 s, 137 MB/s 2024-04-25T08:42:12.842 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.842 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.842 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861473 s, 122 MB/s 2024-04-25T08:42:12.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.853 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008312 s, 126 MB/s 2024-04-25T08:42:12.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.862 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.862 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748925 s, 140 MB/s 2024-04-25T08:42:12.872 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.873 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869459 s, 121 MB/s 2024-04-25T08:42:12.883 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.883 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.883 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784845 s, 134 MB/s 2024-04-25T08:42:12.892 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.892 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.892 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695929 s, 151 MB/s 2024-04-25T08:42:12.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.906 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.906 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118105 s, 88.8 MB/s 2024-04-25T08:42:12.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.917 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.917 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907561 s, 116 MB/s 2024-04-25T08:42:12.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.926 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007127 s, 147 MB/s 2024-04-25T08:42:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.936 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077114 s, 136 MB/s 2024-04-25T08:42:12.946 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.946 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.946 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769068 s, 136 MB/s 2024-04-25T08:42:12.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.955 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713398 s, 147 MB/s 2024-04-25T08:42:12.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.965 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738738 s, 142 MB/s 2024-04-25T08:42:12.974 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.975 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.975 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756393 s, 139 MB/s 2024-04-25T08:42:12.985 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.985 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.985 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843289 s, 124 MB/s 2024-04-25T08:42:12.996 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:12.996 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:12.996 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882637 s, 119 MB/s 2024-04-25T08:42:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.006 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.007 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772235 s, 136 MB/s 2024-04-25T08:42:13.016 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.016 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.016 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752054 s, 139 MB/s 2024-04-25T08:42:13.026 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.026 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.026 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759091 s, 138 MB/s 2024-04-25T08:42:13.035 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.035 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.035 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759122 s, 138 MB/s 2024-04-25T08:42:13.045 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.046 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.046 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793535 s, 132 MB/s 2024-04-25T08:42:13.057 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.058 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762975 s, 137 MB/s 2024-04-25T08:42:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.066 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860113 s, 122 MB/s 2024-04-25T08:42:13.076 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.076 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.076 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763175 s, 137 MB/s 2024-04-25T08:42:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.086 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798248 s, 131 MB/s 2024-04-25T08:42:13.096 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.096 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.096 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791265 s, 133 MB/s 2024-04-25T08:42:13.106 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.106 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.106 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078554 s, 133 MB/s 2024-04-25T08:42:13.116 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.116 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.116 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076733 s, 137 MB/s 2024-04-25T08:42:13.126 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.126 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.126 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851736 s, 123 MB/s 2024-04-25T08:42:13.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:13.127 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:13.127 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:13.127 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T08:42:13.127 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:13.127 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:13.128 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:13.128 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:13.195 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.195 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.195 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779762 s, 134 MB/s 2024-04-25T08:42:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.204 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709732 s, 148 MB/s 2024-04-25T08:42:13.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.215 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.215 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846225 s, 124 MB/s 2024-04-25T08:42:13.224 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.224 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.224 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702481 s, 149 MB/s 2024-04-25T08:42:13.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.234 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814557 s, 129 MB/s 2024-04-25T08:42:13.244 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.244 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.244 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771224 s, 136 MB/s 2024-04-25T08:42:13.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.254 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.255 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811296 s, 129 MB/s 2024-04-25T08:42:13.264 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.264 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.264 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748491 s, 140 MB/s 2024-04-25T08:42:13.275 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.275 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.275 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841201 s, 125 MB/s 2024-04-25T08:42:13.284 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.284 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.284 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728226 s, 144 MB/s 2024-04-25T08:42:13.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.294 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711203 s, 147 MB/s 2024-04-25T08:42:13.303 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.304 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.304 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796457 s, 132 MB/s 2024-04-25T08:42:13.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.313 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.313 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760144 s, 138 MB/s 2024-04-25T08:42:13.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.323 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.323 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724136 s, 145 MB/s 2024-04-25T08:42:13.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.332 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716659 s, 146 MB/s 2024-04-25T08:42:13.341 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.341 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.341 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680152 s, 154 MB/s 2024-04-25T08:42:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.351 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.352 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799668 s, 131 MB/s 2024-04-25T08:42:13.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.362 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.362 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810893 s, 129 MB/s 2024-04-25T08:42:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.371 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740947 s, 142 MB/s 2024-04-25T08:42:13.380 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.380 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.380 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679712 s, 154 MB/s 2024-04-25T08:42:13.390 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.390 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.390 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730323 s, 144 MB/s 2024-04-25T08:42:13.399 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.399 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.399 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746296 s, 141 MB/s 2024-04-25T08:42:13.409 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.409 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.410 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795827 s, 132 MB/s 2024-04-25T08:42:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.419 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713874 s, 147 MB/s 2024-04-25T08:42:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.429 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.430 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847888 s, 124 MB/s 2024-04-25T08:42:13.439 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.439 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.439 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712203 s, 147 MB/s 2024-04-25T08:42:13.448 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.448 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.448 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072918 s, 144 MB/s 2024-04-25T08:42:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.458 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711471 s, 147 MB/s 2024-04-25T08:42:13.467 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.467 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.467 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730507 s, 144 MB/s 2024-04-25T08:42:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.477 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770546 s, 136 MB/s 2024-04-25T08:42:13.486 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.486 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.486 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705019 s, 149 MB/s 2024-04-25T08:42:13.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.496 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.496 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081334 s, 129 MB/s 2024-04-25T08:42:13.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:13.498 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:13.498 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:13.498 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T08:42:13.498 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:13.498 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:13.498 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:13.498 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:13.566 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.566 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.566 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741889 s, 141 MB/s 2024-04-25T08:42:13.574 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.574 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.574 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657557 s, 159 MB/s 2024-04-25T08:42:13.584 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.584 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.584 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727185 s, 144 MB/s 2024-04-25T08:42:13.593 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.593 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.593 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071987 s, 146 MB/s 2024-04-25T08:42:13.602 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.602 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.603 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731403 s, 143 MB/s 2024-04-25T08:42:13.613 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.613 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.613 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804004 s, 130 MB/s 2024-04-25T08:42:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.623 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786052 s, 133 MB/s 2024-04-25T08:42:13.632 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.632 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.632 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737927 s, 142 MB/s 2024-04-25T08:42:13.641 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.641 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.641 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670769 s, 156 MB/s 2024-04-25T08:42:13.652 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.652 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.652 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823053 s, 127 MB/s 2024-04-25T08:42:13.661 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.661 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.661 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724812 s, 145 MB/s 2024-04-25T08:42:13.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.672 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806958 s, 130 MB/s 2024-04-25T08:42:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806424 s, 130 MB/s 2024-04-25T08:42:13.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.692 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799174 s, 131 MB/s 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.701 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.702 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734267 s, 143 MB/s 2024-04-25T08:42:13.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.711 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689612 s, 152 MB/s 2024-04-25T08:42:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.720 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704367 s, 149 MB/s 2024-04-25T08:42:13.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.729 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719528 s, 146 MB/s 2024-04-25T08:42:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.739 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762405 s, 138 MB/s 2024-04-25T08:42:13.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.749 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699645 s, 150 MB/s 2024-04-25T08:42:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.758 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738679 s, 142 MB/s 2024-04-25T08:42:13.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.768 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797475 s, 131 MB/s 2024-04-25T08:42:13.778 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.778 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.778 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740318 s, 142 MB/s 2024-04-25T08:42:13.787 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.787 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.787 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708112 s, 148 MB/s 2024-04-25T08:42:13.797 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.797 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.797 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834447 s, 126 MB/s 2024-04-25T08:42:13.807 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.807 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.807 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074832 s, 140 MB/s 2024-04-25T08:42:13.816 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.816 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.816 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660729 s, 159 MB/s 2024-04-25T08:42:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.825 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709827 s, 148 MB/s 2024-04-25T08:42:13.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.835 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732244 s, 143 MB/s 2024-04-25T08:42:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.845 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815226 s, 129 MB/s 2024-04-25T08:42:13.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.854 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666487 s, 157 MB/s 2024-04-25T08:42:13.864 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.864 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.864 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076017 s, 138 MB/s 2024-04-25T08:42:13.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:13.865 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:13.865 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:13.865 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T08:42:13.865 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:13.865 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:13.865 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:13.865 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:13.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.931 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755785 s, 139 MB/s 2024-04-25T08:42:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078188 s, 134 MB/s 2024-04-25T08:42:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.951 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771739 s, 136 MB/s 2024-04-25T08:42:13.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.961 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732687 s, 143 MB/s 2024-04-25T08:42:13.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.970 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.970 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749009 s, 140 MB/s 2024-04-25T08:42:13.981 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.981 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.981 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842223 s, 125 MB/s 2024-04-25T08:42:13.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:13.990 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:13.990 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715796 s, 146 MB/s 2024-04-25T08:42:13.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.000 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.000 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730519 s, 144 MB/s 2024-04-25T08:42:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805217 s, 130 MB/s 2024-04-25T08:42:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.019 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745315 s, 141 MB/s 2024-04-25T08:42:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761336 s, 138 MB/s 2024-04-25T08:42:14.039 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.039 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.039 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799716 s, 131 MB/s 2024-04-25T08:42:14.049 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.049 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.049 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740814 s, 142 MB/s 2024-04-25T08:42:14.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.059 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.059 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007359 s, 142 MB/s 2024-04-25T08:42:14.068 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.068 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.068 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735568 s, 143 MB/s 2024-04-25T08:42:14.078 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.078 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.078 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796934 s, 132 MB/s 2024-04-25T08:42:14.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.088 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.088 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721206 s, 145 MB/s 2024-04-25T08:42:14.097 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.097 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.097 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717685 s, 146 MB/s 2024-04-25T08:42:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.106 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707524 s, 148 MB/s 2024-04-25T08:42:14.115 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.115 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.116 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704062 s, 149 MB/s 2024-04-25T08:42:14.125 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.125 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.125 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746503 s, 140 MB/s 2024-04-25T08:42:14.135 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.135 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.135 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007378 s, 142 MB/s 2024-04-25T08:42:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.144 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758201 s, 138 MB/s 2024-04-25T08:42:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.154 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725998 s, 144 MB/s 2024-04-25T08:42:14.163 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.163 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.163 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715625 s, 147 MB/s 2024-04-25T08:42:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.173 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750415 s, 140 MB/s 2024-04-25T08:42:14.182 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.224 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.224 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744857 s, 141 MB/s 2024-04-25T08:42:14.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.225 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825515 s, 127 MB/s 2024-04-25T08:42:14.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.226 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808621 s, 130 MB/s 2024-04-25T08:42:14.226 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.226 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.226 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721045 s, 145 MB/s 2024-04-25T08:42:14.226 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.226 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.226 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682265 s, 154 MB/s 2024-04-25T08:42:14.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.232 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763419 s, 137 MB/s 2024-04-25T08:42:14.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:14.233 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:14.233 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:14.233 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T08:42:14.233 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:14.233 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:14.233 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:14.233 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:14.301 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.301 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.301 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007916 s, 132 MB/s 2024-04-25T08:42:14.311 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.311 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.312 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817983 s, 128 MB/s 2024-04-25T08:42:14.322 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.322 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.322 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828318 s, 127 MB/s 2024-04-25T08:42:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.331 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677007 s, 155 MB/s 2024-04-25T08:42:14.339 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.339 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.339 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00652507 s, 161 MB/s 2024-04-25T08:42:14.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.349 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.349 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735147 s, 143 MB/s 2024-04-25T08:42:14.359 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.359 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.359 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821155 s, 128 MB/s 2024-04-25T08:42:14.369 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.369 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.369 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784916 s, 134 MB/s 2024-04-25T08:42:14.379 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.379 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.379 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750571 s, 140 MB/s 2024-04-25T08:42:14.388 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.388 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.388 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701449 s, 149 MB/s 2024-04-25T08:42:14.397 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.397 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.397 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708783 s, 148 MB/s 2024-04-25T08:42:14.408 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.408 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.408 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832919 s, 126 MB/s 2024-04-25T08:42:14.418 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.418 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.418 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777847 s, 135 MB/s 2024-04-25T08:42:14.428 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.428 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.428 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789119 s, 133 MB/s 2024-04-25T08:42:14.438 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.438 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.438 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787428 s, 133 MB/s 2024-04-25T08:42:14.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.447 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669199 s, 157 MB/s 2024-04-25T08:42:14.456 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.457 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721804 s, 145 MB/s 2024-04-25T08:42:14.466 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.466 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.466 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738478 s, 142 MB/s 2024-04-25T08:42:14.475 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.476 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744001 s, 141 MB/s 2024-04-25T08:42:14.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.485 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703842 s, 149 MB/s 2024-04-25T08:42:14.495 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.495 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.495 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755981 s, 139 MB/s 2024-04-25T08:42:14.505 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.505 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.505 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781358 s, 134 MB/s 2024-04-25T08:42:14.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.514 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722321 s, 145 MB/s 2024-04-25T08:42:14.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.523 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685259 s, 153 MB/s 2024-04-25T08:42:14.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.533 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765719 s, 137 MB/s 2024-04-25T08:42:14.543 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.543 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.543 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772338 s, 136 MB/s 2024-04-25T08:42:14.553 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.553 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.553 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805779 s, 130 MB/s 2024-04-25T08:42:14.563 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.563 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.563 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731413 s, 143 MB/s 2024-04-25T08:42:14.573 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.574 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.574 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822254 s, 128 MB/s 2024-04-25T08:42:14.584 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.584 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.584 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814054 s, 129 MB/s 2024-04-25T08:42:14.593 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.593 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.594 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721758 s, 145 MB/s 2024-04-25T08:42:14.604 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.604 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.604 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798779 s, 131 MB/s 2024-04-25T08:42:14.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:14.605 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:14.605 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:14.605 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T08:42:14.605 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:14.605 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:14.605 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:14.605 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.674 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077829 s, 135 MB/s 2024-04-25T08:42:14.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.685 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.685 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811296 s, 129 MB/s 2024-04-25T08:42:14.697 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.697 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.697 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957777 s, 109 MB/s 2024-04-25T08:42:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.710 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100852 s, 104 MB/s 2024-04-25T08:42:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.720 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760904 s, 138 MB/s 2024-04-25T08:42:14.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.731 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.732 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875151 s, 120 MB/s 2024-04-25T08:42:14.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.744 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.744 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100683 s, 104 MB/s 2024-04-25T08:42:14.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.754 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.754 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074505 s, 141 MB/s 2024-04-25T08:42:14.764 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.764 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.764 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076094 s, 138 MB/s 2024-04-25T08:42:14.773 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.773 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.773 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686254 s, 153 MB/s 2024-04-25T08:42:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.782 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692217 s, 151 MB/s 2024-04-25T08:42:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.793 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825297 s, 127 MB/s 2024-04-25T08:42:14.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.803 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.803 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807414 s, 130 MB/s 2024-04-25T08:42:14.812 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.812 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.812 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721258 s, 145 MB/s 2024-04-25T08:42:14.821 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.822 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.822 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713978 s, 147 MB/s 2024-04-25T08:42:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.832 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808735 s, 130 MB/s 2024-04-25T08:42:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.841 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693014 s, 151 MB/s 2024-04-25T08:42:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.850 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727869 s, 144 MB/s 2024-04-25T08:42:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.860 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740073 s, 142 MB/s 2024-04-25T08:42:14.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.869 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691684 s, 152 MB/s 2024-04-25T08:42:14.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.879 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807289 s, 130 MB/s 2024-04-25T08:42:14.889 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.889 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.889 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738838 s, 142 MB/s 2024-04-25T08:42:14.898 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.898 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.898 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715308 s, 147 MB/s 2024-04-25T08:42:14.907 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.907 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.907 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702037 s, 149 MB/s 2024-04-25T08:42:14.916 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.916 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.917 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722307 s, 145 MB/s 2024-04-25T08:42:14.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.926 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.926 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775968 s, 135 MB/s 2024-04-25T08:42:14.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.936 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.936 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072401 s, 145 MB/s 2024-04-25T08:42:14.946 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.946 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.946 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814288 s, 129 MB/s 2024-04-25T08:42:14.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.956 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.956 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716287 s, 146 MB/s 2024-04-25T08:42:14.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.965 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710518 s, 148 MB/s 2024-04-25T08:42:14.974 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.974 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.974 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740659 s, 142 MB/s 2024-04-25T08:42:14.984 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:14.984 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:14.984 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799942 s, 131 MB/s 2024-04-25T08:42:14.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:14.985 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:14.985 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:14.986 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T08:42:14.986 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:14.986 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:14.986 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:14.986 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:15.053 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.054 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.054 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722077 s, 145 MB/s 2024-04-25T08:42:15.063 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.063 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.063 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788075 s, 133 MB/s 2024-04-25T08:42:15.073 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.073 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.073 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742614 s, 141 MB/s 2024-04-25T08:42:15.084 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.084 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.084 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820616 s, 128 MB/s 2024-04-25T08:42:15.094 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.094 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.094 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794036 s, 132 MB/s 2024-04-25T08:42:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.104 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775636 s, 135 MB/s 2024-04-25T08:42:15.113 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.113 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.113 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741046 s, 141 MB/s 2024-04-25T08:42:15.124 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.124 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.124 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829923 s, 126 MB/s 2024-04-25T08:42:15.134 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.135 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.135 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823855 s, 127 MB/s 2024-04-25T08:42:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.145 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849097 s, 123 MB/s 2024-04-25T08:42:15.155 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.155 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.155 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734184 s, 143 MB/s 2024-04-25T08:42:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.164 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717121 s, 146 MB/s 2024-04-25T08:42:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.174 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735329 s, 143 MB/s 2024-04-25T08:42:15.183 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.183 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.183 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722961 s, 145 MB/s 2024-04-25T08:42:15.194 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.194 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.194 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823167 s, 127 MB/s 2024-04-25T08:42:15.204 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.204 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.204 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816456 s, 128 MB/s 2024-04-25T08:42:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.213 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.214 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721944 s, 145 MB/s 2024-04-25T08:42:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.223 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724769 s, 145 MB/s 2024-04-25T08:42:15.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.232 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.232 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657115 s, 160 MB/s 2024-04-25T08:42:15.241 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.241 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.241 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725597 s, 145 MB/s 2024-04-25T08:42:15.250 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.250 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.250 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699835 s, 150 MB/s 2024-04-25T08:42:15.260 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.260 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.261 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811839 s, 129 MB/s 2024-04-25T08:42:15.270 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.270 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.270 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759779 s, 138 MB/s 2024-04-25T08:42:15.280 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.280 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.280 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737118 s, 142 MB/s 2024-04-25T08:42:15.289 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.289 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.289 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065444 s, 160 MB/s 2024-04-25T08:42:15.298 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.298 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.298 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701896 s, 149 MB/s 2024-04-25T08:42:15.308 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.308 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.308 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808999 s, 130 MB/s 2024-04-25T08:42:15.318 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.318 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.318 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758914 s, 138 MB/s 2024-04-25T08:42:15.327 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.327 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.327 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754127 s, 139 MB/s 2024-04-25T08:42:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.337 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750918 s, 140 MB/s 2024-04-25T08:42:15.346 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.347 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.347 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075119 s, 140 MB/s 2024-04-25T08:42:15.357 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.357 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.357 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079029 s, 133 MB/s 2024-04-25T08:42:15.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:15.358 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:15.358 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:15.358 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T08:42:15.358 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:15.358 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:15.358 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:15.358 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:15.425 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.425 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.425 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709421 s, 148 MB/s 2024-04-25T08:42:15.436 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.436 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.436 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676006 s, 155 MB/s 2024-04-25T08:42:15.445 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.445 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.446 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720108 s, 146 MB/s 2024-04-25T08:42:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.455 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719795 s, 146 MB/s 2024-04-25T08:42:15.464 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.464 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.465 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747429 s, 140 MB/s 2024-04-25T08:42:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.475 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813999 s, 129 MB/s 2024-04-25T08:42:15.484 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.484 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.485 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763894 s, 137 MB/s 2024-04-25T08:42:15.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.494 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.494 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722793 s, 145 MB/s 2024-04-25T08:42:15.504 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.505 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.505 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820134 s, 128 MB/s 2024-04-25T08:42:15.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.514 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.514 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788071 s, 133 MB/s 2024-04-25T08:42:15.524 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.524 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.524 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731955 s, 143 MB/s 2024-04-25T08:42:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.533 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689719 s, 152 MB/s 2024-04-25T08:42:15.544 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.544 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.544 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907886 s, 115 MB/s 2024-04-25T08:42:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.554 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729692 s, 144 MB/s 2024-04-25T08:42:15.563 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.563 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.563 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733785 s, 143 MB/s 2024-04-25T08:42:15.574 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.574 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.574 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834576 s, 126 MB/s 2024-04-25T08:42:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.583 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738378 s, 142 MB/s 2024-04-25T08:42:15.594 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.594 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.594 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837136 s, 125 MB/s 2024-04-25T08:42:15.603 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.603 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.604 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746906 s, 140 MB/s 2024-04-25T08:42:15.613 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.613 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.613 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736232 s, 142 MB/s 2024-04-25T08:42:15.623 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.623 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.623 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787466 s, 133 MB/s 2024-04-25T08:42:15.632 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.632 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.632 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674973 s, 155 MB/s 2024-04-25T08:42:15.642 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.642 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.642 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723324 s, 145 MB/s 2024-04-25T08:42:15.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.652 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.652 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806957 s, 130 MB/s 2024-04-25T08:42:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723164 s, 145 MB/s 2024-04-25T08:42:15.670 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.670 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.670 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731368 s, 143 MB/s 2024-04-25T08:42:15.680 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.681 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.681 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822472 s, 127 MB/s 2024-04-25T08:42:15.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.692 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086582 s, 121 MB/s 2024-04-25T08:42:15.701 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.701 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.702 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076209 s, 138 MB/s 2024-04-25T08:42:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.711 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734946 s, 143 MB/s 2024-04-25T08:42:15.720 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.721 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.721 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736151 s, 142 MB/s 2024-04-25T08:42:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.730 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760504 s, 138 MB/s 2024-04-25T08:42:15.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:15.731 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:15.732 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:15.732 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T08:42:15.732 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:15.732 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:15.732 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:15.732 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:15.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.800 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714345 s, 147 MB/s 2024-04-25T08:42:15.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.810 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806048 s, 130 MB/s 2024-04-25T08:42:15.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746323 s, 140 MB/s 2024-04-25T08:42:15.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.829 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756059 s, 139 MB/s 2024-04-25T08:42:15.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.838 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.838 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735088 s, 143 MB/s 2024-04-25T08:42:15.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.848 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007503 s, 140 MB/s 2024-04-25T08:42:15.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.857 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.858 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724125 s, 145 MB/s 2024-04-25T08:42:15.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.869 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087627 s, 120 MB/s 2024-04-25T08:42:15.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.879 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792812 s, 132 MB/s 2024-04-25T08:42:15.889 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.889 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.889 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798209 s, 131 MB/s 2024-04-25T08:42:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.899 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753376 s, 139 MB/s 2024-04-25T08:42:15.907 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.908 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.908 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659646 s, 159 MB/s 2024-04-25T08:42:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.918 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758019 s, 138 MB/s 2024-04-25T08:42:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.928 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797414 s, 131 MB/s 2024-04-25T08:42:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.938 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815432 s, 129 MB/s 2024-04-25T08:42:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.948 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760953 s, 138 MB/s 2024-04-25T08:42:15.958 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.959 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.959 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822035 s, 128 MB/s 2024-04-25T08:42:15.968 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.968 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.968 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073521 s, 143 MB/s 2024-04-25T08:42:15.979 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.979 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.979 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876422 s, 120 MB/s 2024-04-25T08:42:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690712 s, 152 MB/s 2024-04-25T08:42:15.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:15.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:15.998 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749976 s, 140 MB/s 2024-04-25T08:42:16.007 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.008 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726779 s, 144 MB/s 2024-04-25T08:42:16.017 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.017 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.017 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073649 s, 142 MB/s 2024-04-25T08:42:16.026 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.026 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.026 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701449 s, 149 MB/s 2024-04-25T08:42:16.036 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.036 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.037 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803055 s, 131 MB/s 2024-04-25T08:42:16.047 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.047 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.047 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081995 s, 128 MB/s 2024-04-25T08:42:16.056 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.057 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690088 s, 152 MB/s 2024-04-25T08:42:16.066 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.066 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.066 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766428 s, 137 MB/s 2024-04-25T08:42:16.083 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.083 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.083 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142541 s, 73.6 MB/s 2024-04-25T08:42:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.092 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708635 s, 148 MB/s 2024-04-25T08:42:16.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.101 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.101 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711327 s, 147 MB/s 2024-04-25T08:42:16.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.110 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.111 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719383 s, 146 MB/s 2024-04-25T08:42:16.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:16.112 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:16.112 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:16.112 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T08:42:16.112 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:16.112 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:16.112 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:16.112 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:16.180 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.180 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.180 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756822 s, 139 MB/s 2024-04-25T08:42:16.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.191 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841595 s, 125 MB/s 2024-04-25T08:42:16.200 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.200 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.201 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741486 s, 141 MB/s 2024-04-25T08:42:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.211 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819226 s, 128 MB/s 2024-04-25T08:42:16.220 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.220 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.220 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725672 s, 144 MB/s 2024-04-25T08:42:16.230 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.230 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.230 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751861 s, 139 MB/s 2024-04-25T08:42:16.239 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.239 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.239 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687148 s, 153 MB/s 2024-04-25T08:42:16.249 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.249 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.249 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733495 s, 143 MB/s 2024-04-25T08:42:16.258 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.258 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.258 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711323 s, 147 MB/s 2024-04-25T08:42:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.268 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074843 s, 140 MB/s 2024-04-25T08:42:16.277 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.277 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.277 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735331 s, 143 MB/s 2024-04-25T08:42:16.287 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.287 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.287 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742957 s, 141 MB/s 2024-04-25T08:42:16.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.297 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743249 s, 141 MB/s 2024-04-25T08:42:16.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.306 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727332 s, 144 MB/s 2024-04-25T08:42:16.315 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.315 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.315 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729493 s, 144 MB/s 2024-04-25T08:42:16.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.326 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841507 s, 125 MB/s 2024-04-25T08:42:16.336 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.336 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.336 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791701 s, 132 MB/s 2024-04-25T08:42:16.345 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.346 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.346 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715271 s, 147 MB/s 2024-04-25T08:42:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.355 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753981 s, 139 MB/s 2024-04-25T08:42:16.365 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.365 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.365 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746291 s, 141 MB/s 2024-04-25T08:42:16.374 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.374 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.374 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715849 s, 146 MB/s 2024-04-25T08:42:16.384 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.384 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.384 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716778 s, 146 MB/s 2024-04-25T08:42:16.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.393 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.393 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762193 s, 138 MB/s 2024-04-25T08:42:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.403 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742245 s, 141 MB/s 2024-04-25T08:42:16.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.412 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.412 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698615 s, 150 MB/s 2024-04-25T08:42:16.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.422 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813302 s, 129 MB/s 2024-04-25T08:42:16.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.432 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.432 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777008 s, 135 MB/s 2024-04-25T08:42:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.454 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733459 s, 143 MB/s 2024-04-25T08:42:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.464 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798058 s, 131 MB/s 2024-04-25T08:42:16.474 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.474 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.474 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772821 s, 136 MB/s 2024-04-25T08:42:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.483 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683303 s, 153 MB/s 2024-04-25T08:42:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.493 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742786 s, 141 MB/s 2024-04-25T08:42:16.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:16.494 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:16.494 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:16.494 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T08:42:16.494 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:16.494 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:16.494 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:16.494 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:16.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.562 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747465 s, 140 MB/s 2024-04-25T08:42:16.571 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.571 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.571 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734665 s, 143 MB/s 2024-04-25T08:42:16.582 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.582 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.582 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830652 s, 126 MB/s 2024-04-25T08:42:16.591 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.591 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.591 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654275 s, 160 MB/s 2024-04-25T08:42:16.600 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.600 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.600 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070008 s, 150 MB/s 2024-04-25T08:42:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.609 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071756 s, 146 MB/s 2024-04-25T08:42:16.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.620 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.620 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083459 s, 126 MB/s 2024-04-25T08:42:16.630 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.631 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837091 s, 125 MB/s 2024-04-25T08:42:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.641 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859179 s, 122 MB/s 2024-04-25T08:42:16.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.651 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707581 s, 148 MB/s 2024-04-25T08:42:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.660 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739771 s, 142 MB/s 2024-04-25T08:42:16.669 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.670 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.670 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728227 s, 144 MB/s 2024-04-25T08:42:16.679 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.679 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.679 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714745 s, 147 MB/s 2024-04-25T08:42:16.690 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.690 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.690 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880874 s, 119 MB/s 2024-04-25T08:42:16.699 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.699 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.699 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737507 s, 142 MB/s 2024-04-25T08:42:16.708 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.709 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.709 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718666 s, 146 MB/s 2024-04-25T08:42:16.719 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.719 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.719 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788364 s, 133 MB/s 2024-04-25T08:42:16.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.729 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791568 s, 132 MB/s 2024-04-25T08:42:16.738 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.738 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.739 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748471 s, 140 MB/s 2024-04-25T08:42:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.748 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726777 s, 144 MB/s 2024-04-25T08:42:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.758 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792144 s, 132 MB/s 2024-04-25T08:42:16.767 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.767 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.767 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690159 s, 152 MB/s 2024-04-25T08:42:16.776 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.776 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.777 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718302 s, 146 MB/s 2024-04-25T08:42:16.786 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.786 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.786 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007574 s, 138 MB/s 2024-04-25T08:42:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.795 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00655587 s, 160 MB/s 2024-04-25T08:42:16.804 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.805 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.805 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728881 s, 144 MB/s 2024-04-25T08:42:16.813 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.814 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690388 s, 152 MB/s 2024-04-25T08:42:16.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.825 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873579 s, 120 MB/s 2024-04-25T08:42:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.835 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837513 s, 125 MB/s 2024-04-25T08:42:16.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.844 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.844 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684215 s, 153 MB/s 2024-04-25T08:42:16.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.854 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.854 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745468 s, 141 MB/s 2024-04-25T08:42:16.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.864 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753495 s, 139 MB/s 2024-04-25T08:42:16.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:16.865 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:16.865 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:16.865 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T08:42:16.865 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:16.865 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:16.865 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:16.865 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:16.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.933 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744513 s, 141 MB/s 2024-04-25T08:42:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849911 s, 123 MB/s 2024-04-25T08:42:16.952 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.952 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.952 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692225 s, 151 MB/s 2024-04-25T08:42:16.962 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.962 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.962 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754558 s, 139 MB/s 2024-04-25T08:42:16.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.972 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724421 s, 145 MB/s 2024-04-25T08:42:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.982 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844121 s, 124 MB/s 2024-04-25T08:42:16.992 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:16.992 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:16.992 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747741 s, 140 MB/s 2024-04-25T08:42:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.002 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775886 s, 135 MB/s 2024-04-25T08:42:17.011 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.012 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748263 s, 140 MB/s 2024-04-25T08:42:17.021 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.021 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.021 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751755 s, 139 MB/s 2024-04-25T08:42:17.031 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.032 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.032 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809458 s, 130 MB/s 2024-04-25T08:42:17.042 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.042 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.042 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839512 s, 125 MB/s 2024-04-25T08:42:17.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.052 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.052 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813576 s, 129 MB/s 2024-04-25T08:42:17.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.062 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.062 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715467 s, 147 MB/s 2024-04-25T08:42:17.072 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.072 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.072 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793107 s, 132 MB/s 2024-04-25T08:42:17.082 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.082 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.082 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739801 s, 142 MB/s 2024-04-25T08:42:17.092 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.092 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.092 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819245 s, 128 MB/s 2024-04-25T08:42:17.102 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.102 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.102 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772581 s, 136 MB/s 2024-04-25T08:42:17.111 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.111 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.111 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715378 s, 147 MB/s 2024-04-25T08:42:17.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.120 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.121 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711898 s, 147 MB/s 2024-04-25T08:42:17.129 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.130 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.130 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706561 s, 148 MB/s 2024-04-25T08:42:17.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.139 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.139 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718101 s, 146 MB/s 2024-04-25T08:42:17.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.148 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.148 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684432 s, 153 MB/s 2024-04-25T08:42:17.157 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.158 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.158 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734743 s, 143 MB/s 2024-04-25T08:42:17.166 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.166 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.166 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065075 s, 161 MB/s 2024-04-25T08:42:17.176 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.176 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.176 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742036 s, 141 MB/s 2024-04-25T08:42:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.185 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697672 s, 150 MB/s 2024-04-25T08:42:17.196 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.196 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.196 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840796 s, 125 MB/s 2024-04-25T08:42:17.205 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.205 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.205 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703076 s, 149 MB/s 2024-04-25T08:42:17.214 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.214 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.214 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068584 s, 153 MB/s 2024-04-25T08:42:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.223 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694591 s, 151 MB/s 2024-04-25T08:42:17.233 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.234 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746542 s, 140 MB/s 2024-04-25T08:42:17.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:17.234 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:17.234 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:17.234 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T08:42:17.234 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:17.234 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:17.234 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:17.235 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:17.301 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.301 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.301 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668094 s, 157 MB/s 2024-04-25T08:42:17.310 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.310 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.310 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682656 s, 154 MB/s 2024-04-25T08:42:17.320 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.320 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.320 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817696 s, 128 MB/s 2024-04-25T08:42:17.329 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.329 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.329 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068364 s, 153 MB/s 2024-04-25T08:42:17.339 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.340 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.340 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825544 s, 127 MB/s 2024-04-25T08:42:17.350 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.351 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.351 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856843 s, 122 MB/s 2024-04-25T08:42:17.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.360 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.360 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744247 s, 141 MB/s 2024-04-25T08:42:17.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.371 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.371 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814519 s, 129 MB/s 2024-04-25T08:42:17.380 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.380 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.380 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719536 s, 146 MB/s 2024-04-25T08:42:17.389 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.389 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.389 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707097 s, 148 MB/s 2024-04-25T08:42:17.399 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.399 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.399 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736788 s, 142 MB/s 2024-04-25T08:42:17.410 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.410 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.410 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851425 s, 123 MB/s 2024-04-25T08:42:17.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.420 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746078 s, 141 MB/s 2024-04-25T08:42:17.430 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.430 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.430 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821515 s, 128 MB/s 2024-04-25T08:42:17.446 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.446 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.446 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825188 s, 127 MB/s 2024-04-25T08:42:17.471 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.471 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.471 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769159 s, 136 MB/s 2024-04-25T08:42:17.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667523 s, 157 MB/s 2024-04-25T08:42:17.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.489 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.489 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723497 s, 145 MB/s 2024-04-25T08:42:17.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.499 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741605 s, 141 MB/s 2024-04-25T08:42:17.508 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.509 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.509 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736378 s, 142 MB/s 2024-04-25T08:42:17.518 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.518 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.518 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747145 s, 140 MB/s 2024-04-25T08:42:17.527 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.527 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.528 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684204 s, 153 MB/s 2024-04-25T08:42:17.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.538 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811883 s, 129 MB/s 2024-04-25T08:42:17.548 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.548 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.549 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817473 s, 128 MB/s 2024-04-25T08:42:17.558 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.558 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.558 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738736 s, 142 MB/s 2024-04-25T08:42:17.567 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.567 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.567 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070045 s, 150 MB/s 2024-04-25T08:42:17.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.577 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692197 s, 151 MB/s 2024-04-25T08:42:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.587 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811264 s, 129 MB/s 2024-04-25T08:42:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.596 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704667 s, 149 MB/s 2024-04-25T08:42:17.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.606 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743335 s, 141 MB/s 2024-04-25T08:42:17.615 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.615 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.615 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745297 s, 141 MB/s 2024-04-25T08:42:17.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.626 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758301 s, 138 MB/s 2024-04-25T08:42:17.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:17.626 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:17.627 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:17.627 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T08:42:17.627 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:17.627 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:17.627 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:17.627 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:17.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.694 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753454 s, 139 MB/s 2024-04-25T08:42:17.703 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.703 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.704 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752524 s, 139 MB/s 2024-04-25T08:42:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677061 s, 155 MB/s 2024-04-25T08:42:17.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.722 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704794 s, 149 MB/s 2024-04-25T08:42:17.730 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.730 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.731 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667112 s, 157 MB/s 2024-04-25T08:42:17.740 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.740 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.740 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689614 s, 152 MB/s 2024-04-25T08:42:17.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.749 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720359 s, 146 MB/s 2024-04-25T08:42:17.758 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.758 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.758 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687084 s, 153 MB/s 2024-04-25T08:42:17.767 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.768 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726624 s, 144 MB/s 2024-04-25T08:42:17.778 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.778 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.778 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815195 s, 129 MB/s 2024-04-25T08:42:17.787 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.787 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.787 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705649 s, 149 MB/s 2024-04-25T08:42:17.797 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.797 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.797 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733043 s, 143 MB/s 2024-04-25T08:42:17.806 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.806 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.806 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690297 s, 152 MB/s 2024-04-25T08:42:17.816 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.816 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.816 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791316 s, 133 MB/s 2024-04-25T08:42:17.825 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.825 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.825 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668347 s, 157 MB/s 2024-04-25T08:42:17.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.834 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.834 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070701 s, 148 MB/s 2024-04-25T08:42:17.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.843 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.843 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684905 s, 153 MB/s 2024-04-25T08:42:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.853 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.854 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807489 s, 130 MB/s 2024-04-25T08:42:17.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.863 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.863 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719923 s, 146 MB/s 2024-04-25T08:42:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.873 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753332 s, 139 MB/s 2024-04-25T08:42:17.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.882 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.882 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732271 s, 143 MB/s 2024-04-25T08:42:17.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.891 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.891 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743768 s, 141 MB/s 2024-04-25T08:42:17.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.901 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.901 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791374 s, 133 MB/s 2024-04-25T08:42:17.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.911 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673048 s, 156 MB/s 2024-04-25T08:42:17.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.920 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788782 s, 133 MB/s 2024-04-25T08:42:17.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.930 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784853 s, 134 MB/s 2024-04-25T08:42:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824084 s, 127 MB/s 2024-04-25T08:42:17.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.950 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070955 s, 148 MB/s 2024-04-25T08:42:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.960 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738789 s, 142 MB/s 2024-04-25T08:42:17.969 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.969 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.969 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713247 s, 147 MB/s 2024-04-25T08:42:17.978 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.978 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.978 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680209 s, 154 MB/s 2024-04-25T08:42:17.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:17.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:17.988 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815058 s, 129 MB/s 2024-04-25T08:42:17.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:17.989 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:17.989 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:17.989 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T08:42:17.990 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:17.990 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:17.990 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:17.990 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:18.056 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.057 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.057 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728575 s, 144 MB/s 2024-04-25T08:42:18.066 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.066 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.066 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721653 s, 145 MB/s 2024-04-25T08:42:18.075 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.076 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.076 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742826 s, 141 MB/s 2024-04-25T08:42:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.085 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704135 s, 149 MB/s 2024-04-25T08:42:18.095 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.095 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.095 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079024 s, 133 MB/s 2024-04-25T08:42:18.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.104 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739748 s, 142 MB/s 2024-04-25T08:42:18.113 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.114 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.114 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700279 s, 150 MB/s 2024-04-25T08:42:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.122 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.123 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690038 s, 152 MB/s 2024-04-25T08:42:18.132 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.133 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798265 s, 131 MB/s 2024-04-25T08:42:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.142 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074226 s, 141 MB/s 2024-04-25T08:42:18.152 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.152 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.152 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723067 s, 145 MB/s 2024-04-25T08:42:18.161 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.162 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.162 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669826 s, 157 MB/s 2024-04-25T08:42:18.170 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.170 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.170 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732229 s, 143 MB/s 2024-04-25T08:42:18.179 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.179 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.179 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687287 s, 153 MB/s 2024-04-25T08:42:18.189 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.189 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.189 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821594 s, 128 MB/s 2024-04-25T08:42:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.199 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794432 s, 132 MB/s 2024-04-25T08:42:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.209 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711835 s, 147 MB/s 2024-04-25T08:42:18.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.218 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724512 s, 145 MB/s 2024-04-25T08:42:18.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.228 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726516 s, 144 MB/s 2024-04-25T08:42:18.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.238 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.238 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833821 s, 126 MB/s 2024-04-25T08:42:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.247 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716618 s, 146 MB/s 2024-04-25T08:42:18.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.257 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737758 s, 142 MB/s 2024-04-25T08:42:18.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.267 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.267 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766376 s, 137 MB/s 2024-04-25T08:42:18.276 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.277 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.277 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755566 s, 139 MB/s 2024-04-25T08:42:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.286 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736264 s, 142 MB/s 2024-04-25T08:42:18.296 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.296 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.296 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740953 s, 142 MB/s 2024-04-25T08:42:18.305 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.305 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.305 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740366 s, 142 MB/s 2024-04-25T08:42:18.315 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.315 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.315 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744448 s, 141 MB/s 2024-04-25T08:42:18.325 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.325 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.325 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081249 s, 129 MB/s 2024-04-25T08:42:18.335 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.335 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.335 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073662 s, 142 MB/s 2024-04-25T08:42:18.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.344 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.344 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691832 s, 152 MB/s 2024-04-25T08:42:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.353 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722325 s, 145 MB/s 2024-04-25T08:42:18.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:18.355 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:18.355 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:18.355 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T08:42:18.355 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:18.355 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:18.355 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:18.355 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:18.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.422 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.422 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00659532 s, 159 MB/s 2024-04-25T08:42:18.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.431 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.432 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727944 s, 144 MB/s 2024-04-25T08:42:18.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.442 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.442 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859093 s, 122 MB/s 2024-04-25T08:42:18.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.479 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.480 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750895 s, 140 MB/s 2024-04-25T08:42:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.490 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832372 s, 126 MB/s 2024-04-25T08:42:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.500 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724128 s, 145 MB/s 2024-04-25T08:42:18.509 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.509 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.510 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779255 s, 135 MB/s 2024-04-25T08:42:18.518 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.519 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.519 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704977 s, 149 MB/s 2024-04-25T08:42:18.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.529 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.529 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776034 s, 135 MB/s 2024-04-25T08:42:18.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.538 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701323 s, 150 MB/s 2024-04-25T08:42:18.546 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.546 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.546 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00654101 s, 160 MB/s 2024-04-25T08:42:18.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.557 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824591 s, 127 MB/s 2024-04-25T08:42:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729178 s, 144 MB/s 2024-04-25T08:42:18.576 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.576 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.576 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747203 s, 140 MB/s 2024-04-25T08:42:18.585 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.586 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.586 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761074 s, 138 MB/s 2024-04-25T08:42:18.595 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.595 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.595 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713783 s, 147 MB/s 2024-04-25T08:42:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.603 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.604 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00643041 s, 163 MB/s 2024-04-25T08:42:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.613 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709403 s, 148 MB/s 2024-04-25T08:42:18.622 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.622 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.622 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702322 s, 149 MB/s 2024-04-25T08:42:18.631 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.631 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.631 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673317 s, 156 MB/s 2024-04-25T08:42:18.640 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.640 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.640 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678227 s, 155 MB/s 2024-04-25T08:42:18.650 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.650 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.650 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753506 s, 139 MB/s 2024-04-25T08:42:18.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.659 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.659 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689262 s, 152 MB/s 2024-04-25T08:42:18.668 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.668 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.668 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698936 s, 150 MB/s 2024-04-25T08:42:18.678 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.678 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.678 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741738 s, 141 MB/s 2024-04-25T08:42:18.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.688 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.688 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841433 s, 125 MB/s 2024-04-25T08:42:18.698 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.698 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.698 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712024 s, 147 MB/s 2024-04-25T08:42:18.706 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.706 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.706 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657988 s, 159 MB/s 2024-04-25T08:42:18.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.716 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698425 s, 150 MB/s 2024-04-25T08:42:18.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.725 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675109 s, 155 MB/s 2024-04-25T08:42:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.734 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737536 s, 142 MB/s 2024-04-25T08:42:18.745 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.745 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.745 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817893 s, 128 MB/s 2024-04-25T08:42:18.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:18.746 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:18.746 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:18.747 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T08:42:18.747 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:18.747 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:18.747 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:18.747 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:18.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.814 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.814 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761731 s, 138 MB/s 2024-04-25T08:42:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.824 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738353 s, 142 MB/s 2024-04-25T08:42:18.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:18.833 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:18.833 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742442 s, 141 MB/s 2024-04-25T08:42:19.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.760 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.924175 s, 1.1 MB/s 2024-04-25T08:42:19.776 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.776 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.777 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0128955 s, 81.3 MB/s 2024-04-25T08:42:19.793 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.793 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.794 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125884 s, 83.3 MB/s 2024-04-25T08:42:19.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.809 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0119965 s, 87.4 MB/s 2024-04-25T08:42:19.823 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.824 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.824 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112339 s, 93.3 MB/s 2024-04-25T08:42:19.837 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.837 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.838 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109752 s, 95.5 MB/s 2024-04-25T08:42:19.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.849 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.849 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793704 s, 132 MB/s 2024-04-25T08:42:19.859 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.859 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.859 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780277 s, 134 MB/s 2024-04-25T08:42:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.869 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760976 s, 138 MB/s 2024-04-25T08:42:19.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.879 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695295 s, 151 MB/s 2024-04-25T08:42:19.889 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.889 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.889 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807419 s, 130 MB/s 2024-04-25T08:42:19.900 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.900 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.900 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080212 s, 131 MB/s 2024-04-25T08:42:19.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.910 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.910 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747999 s, 140 MB/s 2024-04-25T08:42:19.920 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.921 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.921 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804085 s, 130 MB/s 2024-04-25T08:42:19.930 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.931 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738905 s, 142 MB/s 2024-04-25T08:42:19.940 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.941 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701626 s, 149 MB/s 2024-04-25T08:42:19.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.950 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.950 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073972 s, 142 MB/s 2024-04-25T08:42:19.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.961 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.961 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807201 s, 130 MB/s 2024-04-25T08:42:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.971 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731307 s, 143 MB/s 2024-04-25T08:42:19.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.982 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.982 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081052 s, 129 MB/s 2024-04-25T08:42:19.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:19.991 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:19.992 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718786 s, 146 MB/s 2024-04-25T08:42:20.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.002 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.002 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755506 s, 139 MB/s 2024-04-25T08:42:20.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.012 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.012 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726367 s, 144 MB/s 2024-04-25T08:42:20.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.022 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.023 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817567 s, 128 MB/s 2024-04-25T08:42:20.033 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.033 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.033 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764734 s, 137 MB/s 2024-04-25T08:42:20.043 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.043 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.044 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799252 s, 131 MB/s 2024-04-25T08:42:20.053 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.053 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.054 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735073 s, 143 MB/s 2024-04-25T08:42:20.064 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.064 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.065 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837284 s, 125 MB/s 2024-04-25T08:42:20.076 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.076 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.076 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851194 s, 123 MB/s 2024-04-25T08:42:20.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:20.077 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:20.077 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:20.077 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T08:42:20.077 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:20.077 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:20.077 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:20.077 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:20.146 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.146 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.146 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755147 s, 139 MB/s 2024-04-25T08:42:20.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.156 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.156 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007644 s, 137 MB/s 2024-04-25T08:42:20.166 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.166 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.166 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765657 s, 137 MB/s 2024-04-25T08:42:20.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.177 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.177 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077256 s, 136 MB/s 2024-04-25T08:42:20.186 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.187 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.187 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753036 s, 139 MB/s 2024-04-25T08:42:20.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.197 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.197 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776576 s, 135 MB/s 2024-04-25T08:42:20.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.207 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728463 s, 144 MB/s 2024-04-25T08:42:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.216 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075692 s, 139 MB/s 2024-04-25T08:42:20.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.227 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.227 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860739 s, 122 MB/s 2024-04-25T08:42:20.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.237 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.237 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755642 s, 139 MB/s 2024-04-25T08:42:20.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.247 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.247 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836925 s, 125 MB/s 2024-04-25T08:42:20.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.257 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.257 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079486 s, 132 MB/s 2024-04-25T08:42:20.268 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.268 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.268 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820766 s, 128 MB/s 2024-04-25T08:42:20.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.278 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.278 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838284 s, 125 MB/s 2024-04-25T08:42:20.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.288 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.288 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734853 s, 143 MB/s 2024-04-25T08:42:20.298 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.298 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.298 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732822 s, 143 MB/s 2024-04-25T08:42:20.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.307 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.307 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725586 s, 145 MB/s 2024-04-25T08:42:20.317 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.317 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.317 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715237 s, 147 MB/s 2024-04-25T08:42:20.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.326 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.326 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075379 s, 139 MB/s 2024-04-25T08:42:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.337 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815099 s, 129 MB/s 2024-04-25T08:42:20.347 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.347 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.347 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815428 s, 129 MB/s 2024-04-25T08:42:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.356 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710571 s, 148 MB/s 2024-04-25T08:42:20.366 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.366 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.367 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763505 s, 137 MB/s 2024-04-25T08:42:20.376 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.376 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.376 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749619 s, 140 MB/s 2024-04-25T08:42:20.385 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.385 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.385 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707104 s, 148 MB/s 2024-04-25T08:42:20.396 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.396 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.396 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817411 s, 128 MB/s 2024-04-25T08:42:20.406 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.406 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.406 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792284 s, 132 MB/s 2024-04-25T08:42:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.415 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065424 s, 160 MB/s 2024-04-25T08:42:20.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.424 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.424 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745725 s, 141 MB/s 2024-04-25T08:42:20.434 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.434 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.434 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747833 s, 140 MB/s 2024-04-25T08:42:20.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.444 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.444 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758291 s, 138 MB/s 2024-04-25T08:42:20.465 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.465 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.465 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870366 s, 120 MB/s 2024-04-25T08:42:20.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:20.466 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:20.466 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:20.467 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T08:42:20.467 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:20.467 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:20.467 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:20.467 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749308 s, 140 MB/s 2024-04-25T08:42:20.544 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.544 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.544 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758317 s, 138 MB/s 2024-04-25T08:42:20.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.554 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.554 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799883 s, 131 MB/s 2024-04-25T08:42:20.564 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.565 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.565 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823084 s, 127 MB/s 2024-04-25T08:42:20.575 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.575 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.575 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833261 s, 126 MB/s 2024-04-25T08:42:20.586 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.586 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.586 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779143 s, 135 MB/s 2024-04-25T08:42:20.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.596 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.596 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759281 s, 138 MB/s 2024-04-25T08:42:20.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.607 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817476 s, 128 MB/s 2024-04-25T08:42:20.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.617 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.617 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751951 s, 139 MB/s 2024-04-25T08:42:20.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.626 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.626 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690166 s, 152 MB/s 2024-04-25T08:42:20.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.636 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.636 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751862 s, 139 MB/s 2024-04-25T08:42:20.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.645 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.646 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073843 s, 142 MB/s 2024-04-25T08:42:20.654 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.654 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.654 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671832 s, 156 MB/s 2024-04-25T08:42:20.664 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.664 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.664 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073115 s, 143 MB/s 2024-04-25T08:42:20.673 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.673 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.673 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706434 s, 148 MB/s 2024-04-25T08:42:20.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.683 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072564 s, 145 MB/s 2024-04-25T08:42:20.693 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.694 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.694 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852626 s, 123 MB/s 2024-04-25T08:42:20.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.704 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.705 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851592 s, 123 MB/s 2024-04-25T08:42:20.714 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.715 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.715 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764861 s, 137 MB/s 2024-04-25T08:42:20.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.725 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.725 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765705 s, 137 MB/s 2024-04-25T08:42:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.735 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730141 s, 144 MB/s 2024-04-25T08:42:20.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.749 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0117446 s, 89.3 MB/s 2024-04-25T08:42:20.759 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.759 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.760 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754489 s, 139 MB/s 2024-04-25T08:42:20.770 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.770 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.770 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785611 s, 133 MB/s 2024-04-25T08:42:20.779 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.779 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.779 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682254 s, 154 MB/s 2024-04-25T08:42:20.789 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.789 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.789 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698925 s, 150 MB/s 2024-04-25T08:42:20.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.799 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.799 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078863 s, 133 MB/s 2024-04-25T08:42:20.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.809 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.809 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737645 s, 142 MB/s 2024-04-25T08:42:20.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764223 s, 137 MB/s 2024-04-25T08:42:20.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.829 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.830 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833355 s, 126 MB/s 2024-04-25T08:42:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859993 s, 122 MB/s 2024-04-25T08:42:20.850 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.850 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.850 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754242 s, 139 MB/s 2024-04-25T08:42:20.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:20.851 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:20.851 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:20.851 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T08:42:20.852 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:20.852 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:20.852 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:20.852 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:20.918 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.918 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.919 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697778 s, 150 MB/s 2024-04-25T08:42:20.928 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.928 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.928 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770842 s, 136 MB/s 2024-04-25T08:42:20.937 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.937 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.937 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705659 s, 149 MB/s 2024-04-25T08:42:20.948 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.948 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.948 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816454 s, 128 MB/s 2024-04-25T08:42:20.957 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.957 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.957 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733649 s, 143 MB/s 2024-04-25T08:42:20.966 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.967 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.967 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007154 s, 147 MB/s 2024-04-25T08:42:20.977 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.977 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.977 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812127 s, 129 MB/s 2024-04-25T08:42:20.986 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.986 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.986 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658283 s, 159 MB/s 2024-04-25T08:42:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746234 s, 141 MB/s 2024-04-25T08:42:21.005 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.005 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.005 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729652 s, 144 MB/s 2024-04-25T08:42:21.015 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.015 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.015 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840527 s, 125 MB/s 2024-04-25T08:42:21.025 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.025 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.025 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736377 s, 142 MB/s 2024-04-25T08:42:21.035 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.035 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.036 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856281 s, 122 MB/s 2024-04-25T08:42:21.045 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.045 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.045 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702634 s, 149 MB/s 2024-04-25T08:42:21.054 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.054 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.054 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698348 s, 150 MB/s 2024-04-25T08:42:21.063 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.063 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.063 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706876 s, 148 MB/s 2024-04-25T08:42:21.073 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.073 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.073 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771134 s, 136 MB/s 2024-04-25T08:42:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.082 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703214 s, 149 MB/s 2024-04-25T08:42:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.092 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751739 s, 139 MB/s 2024-04-25T08:42:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.103 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855455 s, 123 MB/s 2024-04-25T08:42:21.112 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.113 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.113 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771065 s, 136 MB/s 2024-04-25T08:42:21.122 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.122 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.122 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712107 s, 147 MB/s 2024-04-25T08:42:21.131 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.131 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.131 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688023 s, 152 MB/s 2024-04-25T08:42:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.141 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748586 s, 140 MB/s 2024-04-25T08:42:21.150 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.150 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.150 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699947 s, 150 MB/s 2024-04-25T08:42:21.159 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.159 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.159 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706078 s, 149 MB/s 2024-04-25T08:42:21.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.168 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.168 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681799 s, 154 MB/s 2024-04-25T08:42:21.178 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.179 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844536 s, 124 MB/s 2024-04-25T08:42:21.189 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.189 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.189 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817537 s, 128 MB/s 2024-04-25T08:42:21.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.198 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.199 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007448 s, 141 MB/s 2024-04-25T08:42:21.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.208 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.208 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742533 s, 141 MB/s 2024-04-25T08:42:21.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.218 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.218 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742227 s, 141 MB/s 2024-04-25T08:42:21.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:21.219 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:21.219 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:21.219 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T08:42:21.219 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:21.219 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:21.219 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:21.219 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.287 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806136 s, 130 MB/s 2024-04-25T08:42:21.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.297 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.297 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738594 s, 142 MB/s 2024-04-25T08:42:21.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.306 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.306 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720513 s, 146 MB/s 2024-04-25T08:42:21.317 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.317 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.317 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849964 s, 123 MB/s 2024-04-25T08:42:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.327 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.328 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836169 s, 125 MB/s 2024-04-25T08:42:21.338 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.338 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.338 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816444 s, 128 MB/s 2024-04-25T08:42:21.348 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.348 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.348 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809011 s, 130 MB/s 2024-04-25T08:42:21.358 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.358 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.358 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769725 s, 136 MB/s 2024-04-25T08:42:21.367 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.367 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.367 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713108 s, 147 MB/s 2024-04-25T08:42:21.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.378 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.378 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841905 s, 125 MB/s 2024-04-25T08:42:21.388 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.388 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.388 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744886 s, 141 MB/s 2024-04-25T08:42:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.398 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803484 s, 131 MB/s 2024-04-25T08:42:21.408 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.408 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.408 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811105 s, 129 MB/s 2024-04-25T08:42:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.419 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824805 s, 127 MB/s 2024-04-25T08:42:21.428 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.428 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.428 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676195 s, 155 MB/s 2024-04-25T08:42:21.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.437 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072792 s, 144 MB/s 2024-04-25T08:42:21.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.447 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804989 s, 130 MB/s 2024-04-25T08:42:21.461 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.461 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.461 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113077 s, 92.7 MB/s 2024-04-25T08:42:21.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.493 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.493 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666244 s, 157 MB/s 2024-04-25T08:42:21.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.503 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.503 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737441 s, 142 MB/s 2024-04-25T08:42:21.512 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.513 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.513 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764475 s, 137 MB/s 2024-04-25T08:42:21.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.523 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.523 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835866 s, 125 MB/s 2024-04-25T08:42:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.533 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743955 s, 141 MB/s 2024-04-25T08:42:21.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.542 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.543 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738406 s, 142 MB/s 2024-04-25T08:42:21.552 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.552 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.552 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745659 s, 141 MB/s 2024-04-25T08:42:21.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.562 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.562 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765298 s, 137 MB/s 2024-04-25T08:42:21.572 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.572 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.572 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736084 s, 142 MB/s 2024-04-25T08:42:21.582 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.582 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.582 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717647 s, 146 MB/s 2024-04-25T08:42:21.591 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.591 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.591 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073129 s, 143 MB/s 2024-04-25T08:42:21.601 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.601 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.601 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725686 s, 144 MB/s 2024-04-25T08:42:21.610 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.611 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.611 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702419 s, 149 MB/s 2024-04-25T08:42:21.621 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.621 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.621 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747283 s, 140 MB/s 2024-04-25T08:42:21.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:21.622 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:21.622 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:21.622 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T08:42:21.622 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:21.622 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:21.622 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:21.622 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:21.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.691 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.691 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827361 s, 127 MB/s 2024-04-25T08:42:21.700 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.700 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.700 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748328 s, 140 MB/s 2024-04-25T08:42:21.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.710 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738471 s, 142 MB/s 2024-04-25T08:42:21.719 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.719 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.719 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708572 s, 148 MB/s 2024-04-25T08:42:21.728 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.729 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.729 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707718 s, 148 MB/s 2024-04-25T08:42:21.738 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.739 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757329 s, 138 MB/s 2024-04-25T08:42:21.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.748 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.748 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074241 s, 141 MB/s 2024-04-25T08:42:21.758 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.758 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.759 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079861 s, 131 MB/s 2024-04-25T08:42:21.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.768 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.768 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724439 s, 145 MB/s 2024-04-25T08:42:21.777 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.777 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.777 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713766 s, 147 MB/s 2024-04-25T08:42:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.787 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748608 s, 140 MB/s 2024-04-25T08:42:21.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.796 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.796 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742698 s, 141 MB/s 2024-04-25T08:42:21.806 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.806 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.806 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729847 s, 144 MB/s 2024-04-25T08:42:21.816 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.816 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.816 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765418 s, 137 MB/s 2024-04-25T08:42:21.825 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.825 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.825 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736821 s, 142 MB/s 2024-04-25T08:42:21.835 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.835 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.835 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752643 s, 139 MB/s 2024-04-25T08:42:21.845 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.845 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.845 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081121 s, 129 MB/s 2024-04-25T08:42:21.855 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.856 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.856 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801778 s, 131 MB/s 2024-04-25T08:42:21.865 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.865 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.865 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726512 s, 144 MB/s 2024-04-25T08:42:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.874 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732658 s, 143 MB/s 2024-04-25T08:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.884 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751394 s, 140 MB/s 2024-04-25T08:42:21.893 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.893 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.893 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069475 s, 151 MB/s 2024-04-25T08:42:21.903 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.903 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.903 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769459 s, 136 MB/s 2024-04-25T08:42:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.913 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.914 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814476 s, 129 MB/s 2024-04-25T08:42:21.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.923 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.923 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075027 s, 140 MB/s 2024-04-25T08:42:21.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.933 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.933 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816902 s, 128 MB/s 2024-04-25T08:42:21.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.944 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.944 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831419 s, 126 MB/s 2024-04-25T08:42:21.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.955 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828213 s, 127 MB/s 2024-04-25T08:42:21.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.965 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.965 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821046 s, 128 MB/s 2024-04-25T08:42:21.974 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.975 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.975 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715688 s, 147 MB/s 2024-04-25T08:42:21.984 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.984 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.984 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734603 s, 143 MB/s 2024-04-25T08:42:21.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:21.994 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:21.994 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072114 s, 145 MB/s 2024-04-25T08:42:21.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:21.995 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:21.995 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:21.995 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T08:42:21.995 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:21.995 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:21.995 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:21.995 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.065 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765193 s, 137 MB/s 2024-04-25T08:42:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.075 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764543 s, 137 MB/s 2024-04-25T08:42:22.084 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.084 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.085 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727488 s, 144 MB/s 2024-04-25T08:42:22.094 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.094 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.094 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739592 s, 142 MB/s 2024-04-25T08:42:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.104 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764644 s, 137 MB/s 2024-04-25T08:42:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.114 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741766 s, 141 MB/s 2024-04-25T08:42:22.123 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.123 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.123 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721822 s, 145 MB/s 2024-04-25T08:42:22.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.133 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.133 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752602 s, 139 MB/s 2024-04-25T08:42:22.142 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.142 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.142 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719087 s, 146 MB/s 2024-04-25T08:42:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.151 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071421 s, 147 MB/s 2024-04-25T08:42:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706772 s, 148 MB/s 2024-04-25T08:42:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.171 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814774 s, 129 MB/s 2024-04-25T08:42:22.181 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.181 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.181 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751139 s, 140 MB/s 2024-04-25T08:42:22.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.191 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.191 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824582 s, 127 MB/s 2024-04-25T08:42:22.202 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.202 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.202 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807945 s, 130 MB/s 2024-04-25T08:42:22.212 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.212 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.213 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008523 s, 123 MB/s 2024-04-25T08:42:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.221 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683067 s, 154 MB/s 2024-04-25T08:42:22.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.231 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.231 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726093 s, 144 MB/s 2024-04-25T08:42:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.240 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688403 s, 152 MB/s 2024-04-25T08:42:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.250 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797915 s, 131 MB/s 2024-04-25T08:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.259 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657836 s, 159 MB/s 2024-04-25T08:42:22.269 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.269 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.269 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788975 s, 133 MB/s 2024-04-25T08:42:22.279 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.279 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.279 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812386 s, 129 MB/s 2024-04-25T08:42:22.290 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.290 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.290 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819629 s, 128 MB/s 2024-04-25T08:42:22.299 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.299 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.299 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722501 s, 145 MB/s 2024-04-25T08:42:22.309 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.309 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.309 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747523 s, 140 MB/s 2024-04-25T08:42:22.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.319 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.319 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828561 s, 127 MB/s 2024-04-25T08:42:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.329 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759518 s, 138 MB/s 2024-04-25T08:42:22.338 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.338 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.338 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707896 s, 148 MB/s 2024-04-25T08:42:22.348 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.348 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.348 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724102 s, 145 MB/s 2024-04-25T08:42:22.357 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.357 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.357 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697308 s, 150 MB/s 2024-04-25T08:42:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.367 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744698 s, 141 MB/s 2024-04-25T08:42:22.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:22.368 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:22.368 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:22.368 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T08:42:22.368 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:22.368 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:22.368 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:22.368 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:22.436 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.437 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859663 s, 122 MB/s 2024-04-25T08:42:22.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.447 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.447 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809724 s, 129 MB/s 2024-04-25T08:42:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.457 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742527 s, 141 MB/s 2024-04-25T08:42:22.468 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.469 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.469 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00971679 s, 108 MB/s 2024-04-25T08:42:22.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.499 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.499 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803605 s, 130 MB/s 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.508 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.509 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709344 s, 148 MB/s 2024-04-25T08:42:22.518 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.518 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.518 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731392 s, 143 MB/s 2024-04-25T08:42:22.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.528 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.528 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737674 s, 142 MB/s 2024-04-25T08:42:22.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.538 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.538 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079871 s, 131 MB/s 2024-04-25T08:42:22.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.547 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.547 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072561 s, 145 MB/s 2024-04-25T08:42:22.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.557 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.557 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689072 s, 152 MB/s 2024-04-25T08:42:22.566 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.566 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.566 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752289 s, 139 MB/s 2024-04-25T08:42:22.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.577 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.577 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840773 s, 125 MB/s 2024-04-25T08:42:22.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.587 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.587 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744445 s, 141 MB/s 2024-04-25T08:42:22.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.597 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819858 s, 128 MB/s 2024-04-25T08:42:22.606 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.607 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.607 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713543 s, 147 MB/s 2024-04-25T08:42:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.616 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067872 s, 154 MB/s 2024-04-25T08:42:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.625 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733992 s, 143 MB/s 2024-04-25T08:42:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.635 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747722 s, 140 MB/s 2024-04-25T08:42:22.643 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.643 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.644 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00645717 s, 162 MB/s 2024-04-25T08:42:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.653 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739242 s, 142 MB/s 2024-04-25T08:42:22.663 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.663 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.663 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772651 s, 136 MB/s 2024-04-25T08:42:22.673 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.673 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.673 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739295 s, 142 MB/s 2024-04-25T08:42:22.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.683 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.683 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763787 s, 137 MB/s 2024-04-25T08:42:22.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.692 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723376 s, 145 MB/s 2024-04-25T08:42:22.703 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.703 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.703 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824498 s, 127 MB/s 2024-04-25T08:42:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.713 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791471 s, 132 MB/s 2024-04-25T08:42:22.722 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.723 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747111 s, 140 MB/s 2024-04-25T08:42:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.732 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746023 s, 141 MB/s 2024-04-25T08:42:22.741 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.742 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.742 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726127 s, 144 MB/s 2024-04-25T08:42:22.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.752 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.752 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841922 s, 125 MB/s 2024-04-25T08:42:22.763 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.763 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.763 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842485 s, 124 MB/s 2024-04-25T08:42:22.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:22.765 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:22.765 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:22.765 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T08:42:22.765 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:22.765 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:22.765 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:22.765 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:22.831 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.831 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.832 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693852 s, 151 MB/s 2024-04-25T08:42:22.840 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.840 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.840 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680882 s, 154 MB/s 2024-04-25T08:42:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.850 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759425 s, 138 MB/s 2024-04-25T08:42:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.860 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780841 s, 134 MB/s 2024-04-25T08:42:22.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.870 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.870 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743177 s, 141 MB/s 2024-04-25T08:42:22.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.879 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.879 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688847 s, 152 MB/s 2024-04-25T08:42:22.888 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.888 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.888 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692024 s, 152 MB/s 2024-04-25T08:42:22.897 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.898 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.898 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743524 s, 141 MB/s 2024-04-25T08:42:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874165 s, 120 MB/s 2024-04-25T08:42:22.918 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.919 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.919 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738041 s, 142 MB/s 2024-04-25T08:42:22.928 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.928 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.928 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755534 s, 139 MB/s 2024-04-25T08:42:22.939 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.939 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.939 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822786 s, 127 MB/s 2024-04-25T08:42:22.948 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.948 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.948 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732766 s, 143 MB/s 2024-04-25T08:42:22.958 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.958 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.958 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071424 s, 147 MB/s 2024-04-25T08:42:22.968 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.968 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.968 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838075 s, 125 MB/s 2024-04-25T08:42:22.978 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.978 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.978 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694886 s, 151 MB/s 2024-04-25T08:42:22.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.988 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.988 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801568 s, 131 MB/s 2024-04-25T08:42:22.998 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:22.999 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:22.999 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845261 s, 124 MB/s 2024-04-25T08:42:23.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.008 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.008 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692846 s, 151 MB/s 2024-04-25T08:42:23.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.018 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.018 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760432 s, 138 MB/s 2024-04-25T08:42:23.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.027 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.027 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711112 s, 147 MB/s 2024-04-25T08:42:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.038 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852617 s, 123 MB/s 2024-04-25T08:42:23.048 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.049 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851769 s, 123 MB/s 2024-04-25T08:42:23.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.058 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.058 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731419 s, 143 MB/s 2024-04-25T08:42:23.068 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.068 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.068 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744247 s, 141 MB/s 2024-04-25T08:42:23.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.077 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.077 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696806 s, 150 MB/s 2024-04-25T08:42:23.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.087 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.087 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761328 s, 138 MB/s 2024-04-25T08:42:23.097 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.098 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.098 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819773 s, 128 MB/s 2024-04-25T08:42:23.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.107 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.107 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737557 s, 142 MB/s 2024-04-25T08:42:23.117 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.118 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.118 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792675 s, 132 MB/s 2024-04-25T08:42:23.127 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.127 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.127 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759954 s, 138 MB/s 2024-04-25T08:42:23.136 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.136 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.136 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704425 s, 149 MB/s 2024-04-25T08:42:23.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:23.138 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:23.138 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:23.138 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T08:42:23.138 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:23.138 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:23.138 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:23.138 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:23.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.206 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.207 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842877 s, 124 MB/s 2024-04-25T08:42:23.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.216 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.216 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723415 s, 145 MB/s 2024-04-25T08:42:23.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.225 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.225 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684259 s, 153 MB/s 2024-04-25T08:42:23.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.234 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.234 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673596 s, 156 MB/s 2024-04-25T08:42:23.243 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.243 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.243 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725729 s, 144 MB/s 2024-04-25T08:42:23.252 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.252 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.253 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710871 s, 148 MB/s 2024-04-25T08:42:23.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.263 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.263 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850801 s, 123 MB/s 2024-04-25T08:42:23.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.273 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.274 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812592 s, 129 MB/s 2024-04-25T08:42:23.284 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.284 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.284 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807271 s, 130 MB/s 2024-04-25T08:42:23.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.293 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.293 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675095 s, 155 MB/s 2024-04-25T08:42:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00649544 s, 161 MB/s 2024-04-25T08:42:23.312 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.312 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.312 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832025 s, 126 MB/s 2024-04-25T08:42:23.322 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.322 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.322 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723887 s, 145 MB/s 2024-04-25T08:42:23.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.332 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.332 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823189 s, 127 MB/s 2024-04-25T08:42:23.341 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.342 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.342 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719883 s, 146 MB/s 2024-04-25T08:42:23.351 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.351 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.351 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756715 s, 139 MB/s 2024-04-25T08:42:23.361 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.361 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.361 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720652 s, 146 MB/s 2024-04-25T08:42:23.370 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.370 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.370 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714369 s, 147 MB/s 2024-04-25T08:42:23.380 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.380 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.380 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752096 s, 139 MB/s 2024-04-25T08:42:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.390 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736815 s, 142 MB/s 2024-04-25T08:42:23.399 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.399 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.399 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725016 s, 145 MB/s 2024-04-25T08:42:23.409 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.409 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.409 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723854 s, 145 MB/s 2024-04-25T08:42:23.418 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.418 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.418 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737819 s, 142 MB/s 2024-04-25T08:42:23.428 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.428 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.428 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743272 s, 141 MB/s 2024-04-25T08:42:23.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.437 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.437 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728116 s, 144 MB/s 2024-04-25T08:42:23.448 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.448 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.448 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834258 s, 126 MB/s 2024-04-25T08:42:23.458 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.458 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.458 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756203 s, 139 MB/s 2024-04-25T08:42:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.469 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882812 s, 119 MB/s 2024-04-25T08:42:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.502 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721678 s, 145 MB/s 2024-04-25T08:42:23.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.511 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.511 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726529 s, 144 MB/s 2024-04-25T08:42:23.521 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.521 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.521 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736904 s, 142 MB/s 2024-04-25T08:42:23.531 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.532 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.532 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833034 s, 126 MB/s 2024-04-25T08:42:23.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:23.533 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T08:42:23.533 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-04-25T08:42:23.533 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T08:42:23.533 DEBUG:teuthology.orchestra.run.smithi038:> for ((i = 0; i < 32; i++)); do 2024-04-25T08:42:23.533 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T08:42:23.533 DEBUG:teuthology.orchestra.run.smithi038:> done 2024-04-25T08:42:23.533 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-04-25T08:42:23.602 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.602 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.602 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823097 s, 127 MB/s 2024-04-25T08:42:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.612 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839339 s, 125 MB/s 2024-04-25T08:42:23.623 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.623 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.623 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842918 s, 124 MB/s 2024-04-25T08:42:23.632 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.632 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.632 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684803 s, 153 MB/s 2024-04-25T08:42:23.642 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.642 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.642 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746624 s, 140 MB/s 2024-04-25T08:42:23.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.651 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.651 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743395 s, 141 MB/s 2024-04-25T08:42:23.661 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.661 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.661 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741174 s, 141 MB/s 2024-04-25T08:42:23.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.672 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883082 s, 119 MB/s 2024-04-25T08:42:23.681 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.681 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.681 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729046 s, 144 MB/s 2024-04-25T08:42:23.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.692 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.692 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821343 s, 128 MB/s 2024-04-25T08:42:23.701 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.701 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.701 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674737 s, 155 MB/s 2024-04-25T08:42:23.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.710 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.710 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710239 s, 148 MB/s 2024-04-25T08:42:23.719 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.720 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.720 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685888 s, 153 MB/s 2024-04-25T08:42:23.730 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.730 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.730 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870518 s, 120 MB/s 2024-04-25T08:42:23.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.739 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.739 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067951 s, 154 MB/s 2024-04-25T08:42:23.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.749 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.749 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763689 s, 137 MB/s 2024-04-25T08:42:23.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.760 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.760 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831385 s, 126 MB/s 2024-04-25T08:42:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.769 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.770 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721131 s, 145 MB/s 2024-04-25T08:42:23.779 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.779 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.779 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714158 s, 147 MB/s 2024-04-25T08:42:23.788 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.788 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.788 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069845 s, 150 MB/s 2024-04-25T08:42:23.798 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.798 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.798 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771633 s, 136 MB/s 2024-04-25T08:42:23.808 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.808 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.808 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816306 s, 128 MB/s 2024-04-25T08:42:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.819 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819346 s, 128 MB/s 2024-04-25T08:42:23.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.828 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.829 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746532 s, 140 MB/s 2024-04-25T08:42:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.839 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817736 s, 128 MB/s 2024-04-25T08:42:23.848 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.849 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.849 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752715 s, 139 MB/s 2024-04-25T08:42:23.859 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.859 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.859 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846682 s, 124 MB/s 2024-04-25T08:42:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.869 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724527 s, 145 MB/s 2024-04-25T08:42:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741633 s, 141 MB/s 2024-04-25T08:42:23.888 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.888 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.888 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762104 s, 138 MB/s 2024-04-25T08:42:23.898 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.898 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.898 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724272 s, 145 MB/s 2024-04-25T08:42:23.908 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-04-25T08:42:23.908 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-04-25T08:42:23.908 INFO:teuthology.orchestra.run.smithi038.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737124 s, 142 MB/s 2024-04-25T08:42:23.909 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-25T08:42:24.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.060+0000 7feb58858640 1 Processor -- start 2024-04-25T08:42:24.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.060+0000 7feb58858640 1 -- start start 2024-04-25T08:42:24.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.060+0000 7feb58858640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7feb540d0320 0x7feb540d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.060+0000 7feb58858640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb540b6980 0x7feb540b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.060+0000 7feb58858640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb540d2060 con 0x7feb540d0700 2024-04-25T08:42:24.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.060+0000 7feb58858640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb540d2330 con 0x7feb540d0320 2024-04-25T08:42:24.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.060+0000 7feb58858640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb540d2610 con 0x7feb540b6980 2024-04-25T08:42:24.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.060+0000 7feb5259c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7feb540d0320 0x7feb540d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.061+0000 7feb5259c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7feb540d0320 0x7feb540d33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60392/0 (socket says 172.21.15.38:60392) 2024-04-25T08:42:24.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.061+0000 7feb5259c640 1 -- 172.21.15.38:0/519364962 learned_addr learned my addr 172.21.15.38:0/519364962 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:24.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.061+0000 7feb52d9d640 1 --2- 172.21.15.38:0/519364962 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb540b6980 0x7feb540b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.061+0000 7feb5259c640 1 -- 172.21.15.38:0/519364962 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb540b6980 msgr2=0x7feb540b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:24.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.061+0000 7feb5259c640 1 --2- 172.21.15.38:0/519364962 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb540b6980 0x7feb540b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.061+0000 7feb5259c640 1 -- 172.21.15.38:0/519364962 >> v1:172.21.15.107:6789/0 conn(0x7feb540d0700 legacy=0x7feb540d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.061+0000 7feb5259c640 1 -- 172.21.15.38:0/519364962 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb540d28f0 con 0x7feb540d0320 2024-04-25T08:42:24.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.062+0000 7feb5259c640 1 --2- 172.21.15.38:0/519364962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb540d0320 0x7feb540d33d0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009870 tx=0x7feb4c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecbd9ecf7730fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:24.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.062+0000 7feb5159a640 1 -- 172.21.15.38:0/519364962 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c037070 con 0x7feb540d0320 2024-04-25T08:42:24.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.062+0000 7feb5159a640 1 -- 172.21.15.38:0/519364962 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c02ae50 con 0x7feb540d0320 2024-04-25T08:42:24.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.062+0000 7feb5159a640 1 -- 172.21.15.38:0/519364962 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c032860 con 0x7feb540d0320 2024-04-25T08:42:24.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.062+0000 7feb58858640 1 -- 172.21.15.38:0/519364962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb540d0320 msgr2=0x7feb540d33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.062+0000 7feb58858640 1 --2- 172.21.15.38:0/519364962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb540d0320 0x7feb540d33d0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009870 tx=0x7feb4c02a300 comp rx=0 tx=0).stop 2024-04-25T08:42:24.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.062+0000 7feb58858640 1 -- 172.21.15.38:0/519364962 shutdown_connections 2024-04-25T08:42:24.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.062+0000 7feb58858640 1 --2- 172.21.15.38:0/519364962 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb540b6980 0x7feb540b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.062+0000 7feb58858640 1 --2- 172.21.15.38:0/519364962 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb540d0320 0x7feb540d33d0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.062+0000 7feb58858640 1 -- 172.21.15.38:0/519364962 >> 172.21.15.38:0/519364962 conn(0x7feb540668f0 msgr2=0x7feb54066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:24.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.062+0000 7feb58858640 1 -- 172.21.15.38:0/519364962 shutdown_connections 2024-04-25T08:42:24.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.062+0000 7feb58858640 1 -- 172.21.15.38:0/519364962 wait complete. 2024-04-25T08:42:24.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.064+0000 7feb58858640 1 Processor -- start 2024-04-25T08:42:24.058 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.064+0000 7feb58858640 1 -- start start 2024-04-25T08:42:24.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.064+0000 7feb58858640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb540b6980 0x7feb540b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.064+0000 7feb58858640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb540d0320 0x7feb540b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.064+0000 7feb58858640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb540d0700 0x7feb5414db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.060 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.064+0000 7feb58858640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb540d0b40 con 0x7feb540b6980 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.064+0000 7feb58858640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb540d2eb0 con 0x7feb540d0700 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.064+0000 7feb58858640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb540b4370 con 0x7feb540d0320 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb5259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb540b6980 0x7feb540b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb5259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb540b6980 0x7feb540b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60398/0 (socket says 172.21.15.38:60398) 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb5259c640 1 -- 172.21.15.38:0/1965353970 learned_addr learned my addr 172.21.15.38:0/1965353970 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb51d9b640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb540d0320 0x7feb540b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb52d9d640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb540d0700 0x7feb5414db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb5259c640 1 -- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb540d0320 msgr2=0x7feb540b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb5259c640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb540d0320 0x7feb540b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb5259c640 1 -- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb540d0700 msgr2=0x7feb5414db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb5259c640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb540d0700 0x7feb5414db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb5259c640 1 -- 172.21.15.38:0/1965353970 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb541e78f0 con 0x7feb540b6980 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb5259c640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb540b6980 0x7feb540b5db0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7feb4c02a340 tx=0x7feb4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb4affd640 1 -- 172.21.15.38:0/1965353970 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c041020 con 0x7feb540b6980 2024-04-25T08:42:24.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb4affd640 1 -- 172.21.15.38:0/1965353970 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4c03c870 con 0x7feb540b6980 2024-04-25T08:42:24.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb58858640 1 -- 172.21.15.38:0/1965353970 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb541e7bd0 con 0x7feb540b6980 2024-04-25T08:42:24.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb4affd640 1 -- 172.21.15.38:0/1965353970 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb4c037040 con 0x7feb540b6980 2024-04-25T08:42:24.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.065+0000 7feb58858640 1 -- 172.21.15.38:0/1965353970 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb540b4000 con 0x7feb540b6980 2024-04-25T08:42:24.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.066+0000 7feb58858640 1 -- 172.21.15.38:0/1965353970 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb54068e80 con 0x7feb540b6980 2024-04-25T08:42:24.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.067+0000 7feb4affd640 1 -- 172.21.15.38:0/1965353970 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb4c044020 con 0x7feb540b6980 2024-04-25T08:42:24.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.067+0000 7feb4affd640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feb3406b4d0 0x7feb3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.067+0000 7feb4affd640 1 -- 172.21.15.38:0/1965353970 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7feb4c0aab60 con 0x7feb540b6980 2024-04-25T08:42:24.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.067+0000 7feb51d9b640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feb3406b4d0 0x7feb3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.062 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.068+0000 7feb51d9b640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feb3406b4d0 0x7feb3406d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7feb5415ae00 tx=0x7feb3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:24.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.071+0000 7feb4affd640 1 -- 172.21.15.38:0/1965353970 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb4c04b030 con 0x7feb540b6980 2024-04-25T08:42:24.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.295+0000 7feb58858640 1 -- 172.21.15.38:0/1965353970 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7feb540b3050 con 0x7feb540b6980 2024-04-25T08:42:24.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.755+0000 7feb4affd640 1 -- 172.21.15.38:0/1965353970 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v89) v1 ==== 105+0+0 (secure 0 0 0) 0x7feb4c079180 con 0x7feb540b6980 2024-04-25T08:42:24.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.757+0000 7feb58858640 1 -- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feb3406b4d0 msgr2=0x7feb3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.757+0000 7feb58858640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feb3406b4d0 0x7feb3406d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7feb5415ae00 tx=0x7feb3c009040 comp rx=0 tx=0).stop 2024-04-25T08:42:24.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.757+0000 7feb58858640 1 -- 172.21.15.38:0/1965353970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb540b6980 msgr2=0x7feb540b5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.757+0000 7feb58858640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb540b6980 0x7feb540b5db0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7feb4c02a340 tx=0x7feb4c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.757+0000 7feb58858640 1 -- 172.21.15.38:0/1965353970 shutdown_connections 2024-04-25T08:42:24.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.757+0000 7feb58858640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb540d0700 0x7feb5414db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.757+0000 7feb58858640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7feb3406b4d0 0x7feb3406d990 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.757+0000 7feb58858640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7feb540d0320 0x7feb540b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.757+0000 7feb58858640 1 --2- 172.21.15.38:0/1965353970 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feb540b6980 0x7feb540b5db0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.757+0000 7feb58858640 1 -- 172.21.15.38:0/1965353970 >> 172.21.15.38:0/1965353970 conn(0x7feb540668f0 msgr2=0x7feb540b6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:24.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.757+0000 7feb58858640 1 -- 172.21.15.38:0/1965353970 shutdown_connections 2024-04-25T08:42:24.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.758+0000 7feb58858640 1 -- 172.21.15.38:0/1965353970 wait complete. 2024-04-25T08:42:24.766 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T08:42:24.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.882+0000 7f42cc4dd640 1 Processor -- start 2024-04-25T08:42:24.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.882+0000 7f42cc4dd640 1 -- start start 2024-04-25T08:42:24.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.882+0000 7f42cc4dd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f42c40d33d0 0x7f42c40d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.882+0000 7f42cc4dd640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f42c40d1e90 0x7f42c40b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.882+0000 7f42cc4dd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c40d2330 con 0x7f42c40d0320 2024-04-25T08:42:24.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.882+0000 7f42cc4dd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c40d2610 con 0x7f42c40d33d0 2024-04-25T08:42:24.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.882+0000 7f42cc4dd640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c40d28f0 con 0x7f42c40d1e90 2024-04-25T08:42:24.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.883+0000 7f42c9a78640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f42c40d33d0 0x7f42c40d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.883+0000 7f42c9a78640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f42c40d33d0 0x7f42c40d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60416/0 (socket says 172.21.15.38:60416) 2024-04-25T08:42:24.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.883+0000 7f42c9a78640 1 -- 172.21.15.38:0/3407861599 learned_addr learned my addr 172.21.15.38:0/3407861599 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:24.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.883+0000 7f42caa7a640 1 --2- 172.21.15.38:0/3407861599 >> v2:172.21.15.107:3300/0 conn(0x7f42c40d1e90 0x7f42c40b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.883+0000 7f42c9a78640 1 -- 172.21.15.38:0/3407861599 >> v2:172.21.15.107:3300/0 conn(0x7f42c40d1e90 msgr2=0x7f42c40b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:24.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.883+0000 7f42c9a78640 1 --2- 172.21.15.38:0/3407861599 >> v2:172.21.15.107:3300/0 conn(0x7f42c40d1e90 0x7f42c40b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.883+0000 7f42c9a78640 1 -- 172.21.15.38:0/3407861599 >> v1:172.21.15.107:6789/0 conn(0x7f42c40d0320 legacy=0x7f42c40d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.883+0000 7f42c9a78640 1 -- 172.21.15.38:0/3407861599 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c40d2bd0 con 0x7f42c40d33d0 2024-04-25T08:42:24.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.884+0000 7f42c9a78640 1 --2- 172.21.15.38:0/3407861599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42c40d33d0 0x7f42c40d1950 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f42c40ca800 tx=0x7f42b802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c715082d4f4ffc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:24.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.884+0000 7f42c9277640 1 -- 172.21.15.38:0/3407861599 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b8037070 con 0x7f42c40d33d0 2024-04-25T08:42:24.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.884+0000 7f42c9277640 1 -- 172.21.15.38:0/3407861599 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b802ae50 con 0x7f42c40d33d0 2024-04-25T08:42:24.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.884+0000 7f42c9277640 1 -- 172.21.15.38:0/3407861599 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b8032860 con 0x7f42c40d33d0 2024-04-25T08:42:24.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.884+0000 7f42cc4dd640 1 -- 172.21.15.38:0/3407861599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42c40d33d0 msgr2=0x7f42c40d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.884+0000 7f42cc4dd640 1 --2- 172.21.15.38:0/3407861599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42c40d33d0 0x7f42c40d1950 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f42c40ca800 tx=0x7f42b802a300 comp rx=0 tx=0).stop 2024-04-25T08:42:24.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.884+0000 7f42cc4dd640 1 -- 172.21.15.38:0/3407861599 shutdown_connections 2024-04-25T08:42:24.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.884+0000 7f42cc4dd640 1 --2- 172.21.15.38:0/3407861599 >> v2:172.21.15.107:3300/0 conn(0x7f42c40d1e90 0x7f42c40b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.884+0000 7f42cc4dd640 1 --2- 172.21.15.38:0/3407861599 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42c40d33d0 0x7f42c40d1950 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.884+0000 7f42cc4dd640 1 -- 172.21.15.38:0/3407861599 >> 172.21.15.38:0/3407861599 conn(0x7f42c40668f0 msgr2=0x7f42c4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:24.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.885+0000 7f42cc4dd640 1 -- 172.21.15.38:0/3407861599 shutdown_connections 2024-04-25T08:42:24.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.885+0000 7f42cc4dd640 1 -- 172.21.15.38:0/3407861599 wait complete. 2024-04-25T08:42:24.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.886+0000 7f42cc4dd640 1 Processor -- start 2024-04-25T08:42:24.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.886+0000 7f42cc4dd640 1 -- start start 2024-04-25T08:42:24.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.886+0000 7f42cc4dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42c40d0320 0x7f42c414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.886+0000 7f42cc4dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42c40d1e90 0x7f42c415c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.886+0000 7f42cc4dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c40d33d0 0x7f42c415cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.886+0000 7f42cc4dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4161390 con 0x7f42c40d0320 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.886+0000 7f42cc4dd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c4161210 con 0x7f42c40d33d0 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.886+0000 7f42cc4dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c40b9720 con 0x7f42c40d1e90 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.886+0000 7f42ca279640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42c40d0320 0x7f42c414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.886+0000 7f42ca279640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42c40d0320 0x7f42c414eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60422/0 (socket says 172.21.15.38:60422) 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42ca279640 1 -- 172.21.15.38:0/265227766 learned_addr learned my addr 172.21.15.38:0/265227766 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42c9a78640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42c40d1e90 0x7f42c415c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42caa7a640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c40d33d0 0x7f42c415cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42ca279640 1 -- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42c40d1e90 msgr2=0x7f42c415c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42ca279640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42c40d1e90 0x7f42c415c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42ca279640 1 -- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c40d33d0 msgr2=0x7f42c415cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42ca279640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c40d33d0 0x7f42c415cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42ca279640 1 -- 172.21.15.38:0/265227766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c4159a90 con 0x7f42c40d0320 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42ca279640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42c40d0320 0x7f42c414eed0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00c400 tx=0x7f42bc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:24.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42c2ffd640 1 -- 172.21.15.38:0/265227766 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0052d0 con 0x7f42c40d0320 2024-04-25T08:42:24.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42c2ffd640 1 -- 172.21.15.38:0/265227766 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42bc005490 con 0x7f42c40d0320 2024-04-25T08:42:24.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42c2ffd640 1 -- 172.21.15.38:0/265227766 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42bc007260 con 0x7f42c40d0320 2024-04-25T08:42:24.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42cc4dd640 1 -- 172.21.15.38:0/265227766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c41589f0 con 0x7f42c40d0320 2024-04-25T08:42:24.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.887+0000 7f42cc4dd640 1 -- 172.21.15.38:0/265227766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c4158fb0 con 0x7f42c40d0320 2024-04-25T08:42:24.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.888+0000 7f42cc4dd640 1 -- 172.21.15.38:0/265227766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c4068e80 con 0x7f42c40d0320 2024-04-25T08:42:24.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.889+0000 7f42c2ffd640 1 -- 172.21.15.38:0/265227766 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42bc007400 con 0x7f42c40d0320 2024-04-25T08:42:24.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.889+0000 7f42c2ffd640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42b006b4d0 0x7f42b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:24.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.889+0000 7f42c2ffd640 1 -- 172.21.15.38:0/265227766 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f42bc089970 con 0x7f42c40d0320 2024-04-25T08:42:24.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.889+0000 7f42c9a78640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42b006b4d0 0x7f42b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:24.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.890+0000 7f42c9a78640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42b006b4d0 0x7f42b006d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f42b8034000 tx=0x7f42b802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:24.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:24.894+0000 7f42c2ffd640 1 -- 172.21.15.38:0/265227766 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42bc0585d0 con 0x7f42c40d0320 2024-04-25T08:42:25.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.113+0000 7f42cc4dd640 1 -- 172.21.15.38:0/265227766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f42c415da20 con 0x7f42c40d0320 2024-04-25T08:42:25.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.769+0000 7f42c2ffd640 1 -- 172.21.15.38:0/265227766 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v90) v1 ==== 121+0+0 (secure 0 0 0) 0x7f42bc057ed0 con 0x7f42c40d0320 2024-04-25T08:42:25.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.771+0000 7f42cc4dd640 1 -- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42b006b4d0 msgr2=0x7f42b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.771+0000 7f42cc4dd640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42b006b4d0 0x7f42b006d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f42b8034000 tx=0x7f42b802e000 comp rx=0 tx=0).stop 2024-04-25T08:42:25.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.771+0000 7f42cc4dd640 1 -- 172.21.15.38:0/265227766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42c40d0320 msgr2=0x7f42c414eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.771+0000 7f42cc4dd640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42c40d0320 0x7f42c414eed0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00c400 tx=0x7f42bc00c8c0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.771+0000 7f42cc4dd640 1 -- 172.21.15.38:0/265227766 shutdown_connections 2024-04-25T08:42:25.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.771+0000 7f42cc4dd640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c40d33d0 0x7f42c415cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.771+0000 7f42cc4dd640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f42b006b4d0 0x7f42b006d990 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.771+0000 7f42cc4dd640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f42c40d1e90 0x7f42c415c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.771+0000 7f42cc4dd640 1 --2- 172.21.15.38:0/265227766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f42c40d0320 0x7f42c414eed0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.772+0000 7f42cc4dd640 1 -- 172.21.15.38:0/265227766 >> 172.21.15.38:0/265227766 conn(0x7f42c40668f0 msgr2=0x7f42c40cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:25.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.772+0000 7f42cc4dd640 1 -- 172.21.15.38:0/265227766 shutdown_connections 2024-04-25T08:42:25.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.772+0000 7f42cc4dd640 1 -- 172.21.15.38:0/265227766 wait complete. 2024-04-25T08:42:25.780 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:25.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.896+0000 7f44153e0640 1 Processor -- start 2024-04-25T08:42:25.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.896+0000 7f44153e0640 1 -- start start 2024-04-25T08:42:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.896+0000 7f44153e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44100ce2e0 0x7f44100d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.896+0000 7f44153e0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44100ceff0 con 0x7f441006c110 2024-04-25T08:42:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.896+0000 7f44153e0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44100d1f70 con 0x7f441015df50 2024-04-25T08:42:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.896+0000 7f44153e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44100cf2d0 con 0x7f44100ce2e0 2024-04-25T08:42:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440effd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f441006c110 0x7f441006c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49288/0 (socket says 172.21.15.38:49288) 2024-04-25T08:42:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440effd640 1 -- 172.21.15.38:0/1057945955 learned_addr learned my addr 172.21.15.38:0/1057945955 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440f7fe640 1 --2- 172.21.15.38:0/1057945955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44100ce2e0 0x7f44100d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688094210 0 0) 0x7f44100ceff0 con 0x7f441006c110 2024-04-25T08:42:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f8003520 con 0x7f441006c110 2024-04-25T08:42:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1224845611 0 0) 0x7f43f8003520 con 0x7f441006c110 2024-04-25T08:42:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44100ce2e0 msgr2=0x7f44100d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:25.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 --2- 172.21.15.38:0/1057945955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44100ce2e0 0x7f44100d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 >> v1:172.21.15.107:6789/0 conn(0x7f441015df50 legacy=0x7f44100ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44100d2250 con 0x7f441006c110 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f4400001670 con 0x7f441006c110 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 >> v1:172.21.15.38:6789/0 conn(0x7f441006c110 legacy=0x7f441006c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 --2- 172.21.15.38:0/1057945955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43f80039e0 0x7f43f8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 --2- 172.21.15.38:0/1057945955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44100ce2e0 0x7f43f8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 --2- 172.21.15.38:0/1057945955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f8004d40 0x7f43f80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44100cf2d0 con 0x7f43f8004d40 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f800b9c0 con 0x7f43f80039e0 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.897+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43f800bbc0 con 0x7f44100ce2e0 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.898+0000 7f440f7fe640 1 --2- 172.21.15.38:0/1057945955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f8004d40 0x7f43f80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.898+0000 7f440effd640 1 --2- 172.21.15.38:0/1057945955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43f80039e0 0x7f43f8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.898+0000 7f440e7fc640 1 --2- 172.21.15.38:0/1057945955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44100ce2e0 0x7f43f8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.898+0000 7f440f7fe640 1 -- 172.21.15.38:0/1057945955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44100ce2e0 msgr2=0x7f43f8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:25.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.898+0000 7f440f7fe640 1 --2- 172.21.15.38:0/1057945955 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f44100ce2e0 0x7f43f8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.898+0000 7f440f7fe640 1 -- 172.21.15.38:0/1057945955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43f80039e0 msgr2=0x7f43f8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:25.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.898+0000 7f440f7fe640 1 --2- 172.21.15.38:0/1057945955 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f43f80039e0 0x7f43f8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.898+0000 7f440f7fe640 1 -- 172.21.15.38:0/1057945955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f8003520 con 0x7f43f8004d40 2024-04-25T08:42:25.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.899+0000 7f440f7fe640 1 --2- 172.21.15.38:0/1057945955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f8004d40 0x7f43f80051c0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f4408009f70 tx=0x7f43f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d8034928200380 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:25.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.899+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408009bf0 con 0x7f43f8004d40 2024-04-25T08:42:25.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.899+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4408036070 con 0x7f43f8004d40 2024-04-25T08:42:25.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.899+0000 7f440dffb640 1 -- 172.21.15.38:0/1057945955 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408030830 con 0x7f43f8004d40 2024-04-25T08:42:25.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.899+0000 7f44153e0640 1 -- 172.21.15.38:0/1057945955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f8004d40 msgr2=0x7f43f80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.899+0000 7f44153e0640 1 --2- 172.21.15.38:0/1057945955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f43f8004d40 0x7f43f80051c0 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f4408009f70 tx=0x7f43f8005830 comp rx=0 tx=0).stop 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.899+0000 7f440f7fe640 1 -- 172.21.15.38:0/1057945955 reap_dead start 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.899+0000 7f44153e0640 1 -- 172.21.15.38:0/1057945955 shutdown_connections 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.899+0000 7f44153e0640 1 -- 172.21.15.38:0/1057945955 >> 172.21.15.38:0/1057945955 conn(0x7f44100d3fb0 msgr2=0x7f44100d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.899+0000 7f44153e0640 1 -- 172.21.15.38:0/1057945955 shutdown_connections 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.899+0000 7f44153e0640 1 -- 172.21.15.38:0/1057945955 wait complete. 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.901+0000 7f44153e0640 1 Processor -- start 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.901+0000 7f44153e0640 1 -- start start 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.901+0000 7f44153e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43f8004d40 0x7f441014dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.901+0000 7f44153e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100b7ef0 0x7f44100b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.901+0000 7f44153e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44100c94f0 0x7f44100c9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.901+0000 7f44153e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44100d2bd0 con 0x7f44100c94f0 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.901+0000 7f44153e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44100d2a50 con 0x7f44100b7ef0 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.901+0000 7f44153e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44100d2d50 con 0x7f43f8004d40 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.901+0000 7f440f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44100c94f0 0x7f44100c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.901+0000 7f440f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44100c94f0 0x7f44100c9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60438/0 (socket says 172.21.15.38:60438) 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.901+0000 7f440f7fe640 1 -- 172.21.15.38:0/2530653544 learned_addr learned my addr 172.21.15.38:0/2530653544 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f440effd640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43f8004d40 0x7f441014dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f440e7fc640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100b7ef0 0x7f44100b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f440f7fe640 1 -- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43f8004d40 msgr2=0x7f441014dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f440f7fe640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43f8004d40 0x7f441014dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f440f7fe640 1 -- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100b7ef0 msgr2=0x7f44100b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f440f7fe640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100b7ef0 0x7f44100b8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f440f7fe640 1 -- 172.21.15.38:0/2530653544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44100cb260 con 0x7f44100c94f0 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f440f7fe640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44100c94f0 0x7f44100c9970 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f440800e230 tx=0x7f4408009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f43f77fe640 1 -- 172.21.15.38:0/2530653544 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408009bf0 con 0x7f44100c94f0 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f43f77fe640 1 -- 172.21.15.38:0/2530653544 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4408045430 con 0x7f44100c94f0 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f44153e0640 1 -- 172.21.15.38:0/2530653544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441014e2e0 con 0x7f44100c94f0 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f43f77fe640 1 -- 172.21.15.38:0/2530653544 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408045820 con 0x7f44100c94f0 2024-04-25T08:42:25.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.902+0000 7f44153e0640 1 -- 172.21.15.38:0/2530653544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44100cb7e0 con 0x7f44100c94f0 2024-04-25T08:42:25.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.903+0000 7f44153e0640 1 -- 172.21.15.38:0/2530653544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4410068e80 con 0x7f44100c94f0 2024-04-25T08:42:25.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.904+0000 7f43f77fe640 1 -- 172.21.15.38:0/2530653544 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4408005d10 con 0x7f44100c94f0 2024-04-25T08:42:25.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.904+0000 7f43f77fe640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f43fc06b4d0 0x7f43fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:25.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.904+0000 7f43f77fe640 1 -- 172.21.15.38:0/2530653544 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f44080aadd0 con 0x7f44100c94f0 2024-04-25T08:42:25.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.904+0000 7f440effd640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f43fc06b4d0 0x7f43fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:25.899 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.905+0000 7f440effd640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f43fc06b4d0 0x7f43fc06d990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4400000c00 tx=0x7f4400000f80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:25.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:25.908+0000 7f43f77fe640 1 -- 172.21.15.38:0/2530653544 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f44080792d0 con 0x7f44100c94f0 2024-04-25T08:42:26.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.127+0000 7f44153e0640 1 -- 172.21.15.38:0/2530653544 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44100d0740 con 0x7f44100c94f0 2024-04-25T08:42:26.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.127+0000 7f43f77fe640 1 -- 172.21.15.38:0/2530653544 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4960 (secure 0 0 0) 0x7f440807d370 con 0x7f44100c94f0 2024-04-25T08:42:26.120 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:26.120 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2682394530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2682394530},{"type":"v1","addr":"172.21.15.107:6837","nonce":2682394530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2810760276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2810760276},{"type":"v1","addr":"172.21.15.107:6835","nonce":2810760276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T08:42:04.044929+0000","modified":"2024-04-25T08:42:25.774447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10038":{"gid":10038,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3318593502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3318593502},{"type":"v1","addr":"172.21.15.38:6833","nonce":3318593502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10041":{"gid":10041,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/3513331661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3513331661},{"type":"v1","addr":"172.21.15.38:6835","nonce":3513331661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:42:26.120 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 91 2024-04-25T08:42:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.129+0000 7f44153e0640 1 -- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f43fc06b4d0 msgr2=0x7f43fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.129+0000 7f44153e0640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f43fc06b4d0 0x7f43fc06d990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4400000c00 tx=0x7f4400000f80 comp rx=0 tx=0).stop 2024-04-25T08:42:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.129+0000 7f44153e0640 1 -- 172.21.15.38:0/2530653544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44100c94f0 msgr2=0x7f44100c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.129+0000 7f44153e0640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44100c94f0 0x7f44100c9970 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f440800e230 tx=0x7f4408009750 comp rx=0 tx=0).stop 2024-04-25T08:42:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.129+0000 7f44153e0640 1 -- 172.21.15.38:0/2530653544 shutdown_connections 2024-04-25T08:42:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.129+0000 7f44153e0640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f44100c94f0 0x7f44100c9970 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.129+0000 7f44153e0640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f44100b7ef0 0x7f44100b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.129+0000 7f44153e0640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f43fc06b4d0 0x7f43fc06d990 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.129+0000 7f44153e0640 1 --2- 172.21.15.38:0/2530653544 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f43f8004d40 0x7f441014dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.129+0000 7f44153e0640 1 -- 172.21.15.38:0/2530653544 >> 172.21.15.38:0/2530653544 conn(0x7f44100d3fb0 msgr2=0x7f4410161840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.129+0000 7f44153e0640 1 -- 172.21.15.38:0/2530653544 shutdown_connections 2024-04-25T08:42:26.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.129+0000 7f44153e0640 1 -- 172.21.15.38:0/2530653544 wait complete. 2024-04-25T08:42:26.536 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T08:42:04.044929+0000', 'modified': '2024-04-25T08:42:25.774447+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10041}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10038': {'gid': 10038, 'name': 'a', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.38:6833/3318593502', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6832', 'nonce': 3318593502}, {'type': 'v1', 'addr': '172.21.15.38:6833', 'nonce': 3318593502}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_10041': {'gid': 10041, 'name': 'c', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.38:6835/3513331661', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.38:6834', 'nonce': 3513331661}, {'type': 'v1', 'addr': '172.21.15.38:6835', 'nonce': 3513331661}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:42:26.537 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:42:26.537 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:42:26.640 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.648+0000 7f4ead5f5640 1 Processor -- start 2024-04-25T08:42:26.641 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.648+0000 7f4ead5f5640 1 -- start start 2024-04-25T08:42:26.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4ead5f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4ea8160ef0 0x7f4ea80d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4ead5f5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4ea80b9180 0x7f4ea80d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4ead5f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ea80b2ab0 0x7f4ea80d37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4ead5f5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea8161cf0 con 0x7f4ea80b9180 2024-04-25T08:42:26.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4ead5f5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea81612d0 con 0x7f4ea8160ef0 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4ead5f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea80b9610 con 0x7f4ea80b2ab0 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4e9ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4ea80b9180 0x7f4ea80d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4e9ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4ea80b9180 0x7f4ea80d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60444/0 (socket says 172.21.15.38:60444) 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4e9ffff640 1 -- 172.21.15.38:0/3639051575 learned_addr learned my addr 172.21.15.38:0/3639051575 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4ea77fe640 1 --2- 172.21.15.38:0/3639051575 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ea80b2ab0 0x7f4ea80d37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4e9ffff640 1 -- 172.21.15.38:0/3639051575 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ea80b2ab0 msgr2=0x7f4ea80d37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/3639051575 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ea80b2ab0 0x7f4ea80d37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/3639051575 >> v2:172.21.15.107:3300/0 conn(0x7f4ea8160ef0 0x7f4ea80d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4e9ffff640 1 -- 172.21.15.38:0/3639051575 >> v2:172.21.15.107:3300/0 conn(0x7f4ea8160ef0 msgr2=0x7f4ea80d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/3639051575 >> v2:172.21.15.107:3300/0 conn(0x7f4ea8160ef0 0x7f4ea80d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.649+0000 7f4e9ffff640 1 -- 172.21.15.38:0/3639051575 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea81616a0 con 0x7f4ea80b9180 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.650+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/3639051575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea80b9180 0x7f4ea80d0cf0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80b5da0 tx=0x7f4e9002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebde1a0ad590d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.643 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.650+0000 7f4ea67fc640 1 -- 172.21.15.38:0/3639051575 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90037070 con 0x7f4ea80b9180 2024-04-25T08:42:26.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.650+0000 7f4ea67fc640 1 -- 172.21.15.38:0/3639051575 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e90032440 con 0x7f4ea80b9180 2024-04-25T08:42:26.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.650+0000 7f4ea67fc640 1 -- 172.21.15.38:0/3639051575 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90032810 con 0x7f4ea80b9180 2024-04-25T08:42:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.651+0000 7f4ead5f5640 1 -- 172.21.15.38:0/3639051575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea80b9180 msgr2=0x7f4ea80d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.651+0000 7f4ead5f5640 1 --2- 172.21.15.38:0/3639051575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea80b9180 0x7f4ea80d0cf0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f4ea80b5da0 tx=0x7f4e9002a140 comp rx=0 tx=0).stop 2024-04-25T08:42:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.651+0000 7f4ead5f5640 1 -- 172.21.15.38:0/3639051575 shutdown_connections 2024-04-25T08:42:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.651+0000 7f4ead5f5640 1 --2- 172.21.15.38:0/3639051575 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ea80b2ab0 0x7f4ea80d37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.651+0000 7f4ead5f5640 1 --2- 172.21.15.38:0/3639051575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea80b9180 0x7f4ea80d0cf0 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.651+0000 7f4ead5f5640 1 --2- 172.21.15.38:0/3639051575 >> v2:172.21.15.107:3300/0 conn(0x7f4ea8160ef0 0x7f4ea80d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.651+0000 7f4ead5f5640 1 -- 172.21.15.38:0/3639051575 >> 172.21.15.38:0/3639051575 conn(0x7f4ea8066d10 msgr2=0x7f4ea8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.651+0000 7f4ead5f5640 1 -- 172.21.15.38:0/3639051575 shutdown_connections 2024-04-25T08:42:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.651+0000 7f4ead5f5640 1 -- 172.21.15.38:0/3639051575 wait complete. 2024-04-25T08:42:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.652+0000 7f4ead5f5640 1 Processor -- start 2024-04-25T08:42:26.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.652+0000 7f4ead5f5640 1 -- start start 2024-04-25T08:42:26.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4ead5f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ea80b2ab0 0x7f4ea80cb340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4ead5f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea8160ef0 0x7f4ea80d2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4ead5f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ea8155b10 0x7f4ea8154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4ead5f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea8162300 con 0x7f4ea8160ef0 2024-04-25T08:42:26.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4ead5f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea8162180 con 0x7f4ea80b2ab0 2024-04-25T08:42:26.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4ead5f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea8162480 con 0x7f4ea8155b10 2024-04-25T08:42:26.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea8160ef0 0x7f4ea80d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea8160ef0 0x7f4ea80d2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60460/0 (socket says 172.21.15.38:60460) 2024-04-25T08:42:26.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4e9ffff640 1 -- 172.21.15.38:0/1647926444 learned_addr learned my addr 172.21.15.38:0/1647926444 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:26.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4ea77fe640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ea8155b10 0x7f4ea8154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ea80b2ab0 0x7f4ea80cb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.648 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4e9ffff640 1 -- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ea8155b10 msgr2=0x7f4ea8154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ea8155b10 0x7f4ea8154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4e9ffff640 1 -- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ea80b2ab0 msgr2=0x7f4ea80cb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ea80b2ab0 0x7f4ea80cb340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4e9ffff640 1 -- 172.21.15.38:0/1647926444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea8157070 con 0x7f4ea8160ef0 2024-04-25T08:42:26.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ea80b2ab0 0x7f4ea80cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:26.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea8160ef0 0x7f4ea80d2580 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f4e90000c00 tx=0x7f4e900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4e9f7fe640 1 -- 172.21.15.38:0/1647926444 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90037040 con 0x7f4ea8160ef0 2024-04-25T08:42:26.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4e9f7fe640 1 -- 172.21.15.38:0/1647926444 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e90042070 con 0x7f4ea8160ef0 2024-04-25T08:42:26.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4ead5f5640 1 -- 172.21.15.38:0/1647926444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea8155fd0 con 0x7f4ea8160ef0 2024-04-25T08:42:26.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.653+0000 7f4e9f7fe640 1 -- 172.21.15.38:0/1647926444 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90030bb0 con 0x7f4ea8160ef0 2024-04-25T08:42:26.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.654+0000 7f4ead5f5640 1 -- 172.21.15.38:0/1647926444 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea8156590 con 0x7f4ea8160ef0 2024-04-25T08:42:26.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.655+0000 7f4e9f7fe640 1 -- 172.21.15.38:0/1647926444 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e90047460 con 0x7f4ea8160ef0 2024-04-25T08:42:26.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.656+0000 7f4e9f7fe640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e8806b4d0 0x7f4e8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.656+0000 7f4e9f7fe640 1 -- 172.21.15.38:0/1647926444 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f4e900ab0c0 con 0x7f4ea8160ef0 2024-04-25T08:42:26.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.656+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e8806b4d0 0x7f4e8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.656+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e8806b4d0 0x7f4e8806d990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800b820 tx=0x7f4e98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.656+0000 7f4eb03d1740 1 Processor -- start 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.657+0000 7f4eb03d1740 1 -- start start 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.657+0000 7f4eb03d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8fd230 0x562d6a8fd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.657+0000 7f4eb03d1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a91b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.657+0000 7f4eb03d1740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8d2f10 0x562d6a919190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.657+0000 7f4eb03d1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6a835a10 con 0x562d6a91aad0 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.657+0000 7f4eb03d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6a7deb30 con 0x562d6a8fd230 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.657+0000 7f4eb03d1740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d6a831c50 con 0x562d6a8d2f10 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.657+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a91b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a91b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60476/0 (socket says 172.21.15.38:60476) 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4e9ffff640 1 -- 172.21.15.38:0/4115095005 learned_addr learned my addr 172.21.15.38:0/4115095005 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4ea77fe640 1 --2- 172.21.15.38:0/4115095005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8d2f10 0x562d6a919190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/4115095005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8fd230 0x562d6a8fd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4e9ffff640 1 -- 172.21.15.38:0/4115095005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8d2f10 msgr2=0x562d6a919190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/4115095005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8d2f10 0x562d6a919190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4e9ffff640 1 -- 172.21.15.38:0/4115095005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8fd230 msgr2=0x562d6a8fd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/4115095005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8fd230 0x562d6a8fd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4e9ffff640 1 -- 172.21.15.38:0/4115095005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d6a7f3e10 con 0x562d6a91aad0 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/4115095005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8fd230 0x562d6a8fd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/4115095005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a91b020 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f4e90079510 tx=0x7f4e9003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252b95188be54c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4e9cff9640 1 -- 172.21.15.38:0/4115095005 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90091020 con 0x562d6a91aad0 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4e9cff9640 1 -- 172.21.15.38:0/4115095005 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9007d6d0 con 0x562d6a91aad0 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.658+0000 7f4e9cff9640 1 -- 172.21.15.38:0/4115095005 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90081c90 con 0x562d6a91aad0 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.659+0000 7f4eb03d1740 1 -- 172.21.15.38:0/4115095005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 msgr2=0x562d6a91b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.659+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/4115095005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a91b020 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f4e90079510 tx=0x7f4e9003b000 comp rx=0 tx=0).stop 2024-04-25T08:42:26.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.659+0000 7f4eb03d1740 1 -- 172.21.15.38:0/4115095005 shutdown_connections 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.659+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/4115095005 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8d2f10 0x562d6a919190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.659+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/4115095005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a91b020 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.659+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/4115095005 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8fd230 0x562d6a8fd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.659+0000 7f4eb03d1740 1 -- 172.21.15.38:0/4115095005 >> 172.21.15.38:0/4115095005 conn(0x562d6a8cf840 msgr2=0x562d6a918040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.659+0000 7f4eb03d1740 1 -- 172.21.15.38:0/4115095005 shutdown_connections 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.659+0000 7f4eb03d1740 1 -- 172.21.15.38:0/4115095005 wait complete. 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.659+0000 7f4eb03d1740 1 -- start start 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.660+0000 7f4eb03d1740 1 Processor -- start 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.660+0000 7f4eb03d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8fd230 0x562d6a8eefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.660+0000 7f4eb03d1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a928080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.660+0000 7f4eb03d1740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8ffbd0 0x562d6a911640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.660+0000 7f4eb03d1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6a7deb30 con 0x562d6a91aad0 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.660+0000 7f4eb03d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6a7f20c0 con 0x562d6a8fd230 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.660+0000 7f4eb03d1740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d6a835a10 con 0x562d6a8ffbd0 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.660+0000 7f4eb03d1740 10 client.0 _ll_register_callbacks cb 0x562d6a7e5ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:42:26.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.660+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a928080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.660+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a928080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60494/0 (socket says 172.21.15.38:60494) 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.660+0000 7f4e9ffff640 1 -- 172.21.15.38:0/153232175 learned_addr learned my addr 172.21.15.38:0/153232175 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4ea77fe640 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8ffbd0 0x562d6a911640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8fd230 0x562d6a8eefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4e9ffff640 1 -- 172.21.15.38:0/153232175 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8ffbd0 msgr2=0x562d6a911640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8ffbd0 0x562d6a911640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4e9ffff640 1 -- 172.21.15.38:0/153232175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8fd230 msgr2=0x562d6a8eefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8fd230 0x562d6a8eefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4e9ffff640 1 -- 172.21.15.38:0/153232175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d6a54a2d0 con 0x562d6a91aad0 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a928080 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f4e9002a730 tx=0x7f4e9002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c0b9153c6f50fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4e85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4e85ffb640 1 -- 172.21.15.38:0/153232175 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90098060 con 0x562d6a91aad0 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4e85ffb640 1 -- 172.21.15.38:0/153232175 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f4e900797e0 con 0x562d6a91aad0 2024-04-25T08:42:26.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4eb03d1740 1 -- 172.21.15.38:0/153232175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4e9007d6d0 con 0x562d6a91aad0 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4e85ffb640 1 -- 172.21.15.38:0/153232175 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e9009e050 con 0x562d6a91aad0 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4eb03d1740 10 client.10242 fetch_fsmap learned FSMap version 91 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4eb03d1740 1 -- 172.21.15.38:0/153232175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x562d6a6146b0 con 0x562d6a91aad0 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4e85ffb640 1 -- 172.21.15.38:0/153232175 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9007d6d0 con 0x562d6a91aad0 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4e85ffb640 1 -- 172.21.15.38:0/153232175 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 91) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f4e900a1020 con 0x562d6a91aad0 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4eb03d1740 10 client.10242 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4eb03d1740 10 client.10242 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4eb03d1740 20 client.10242 populate_metadata read hostname 'smithi038' 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x562d6a90f5f0 0x562d6a90f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4eb03d1740 4 client.10242 mds_command: new command op to 10041 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.661+0000 7f4eb03d1740 1 -- 172.21.15.38:0/153232175 --> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d6a894920 con 0x562d6a90f5f0 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.662+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x562d6a90f5f0 0x562d6a90f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.662+0000 7f4e85ffb640 10 client.10242 ms_handle_connect on v2:172.21.15.38:6834/3513331661 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.662+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x562d6a90f5f0 0x562d6a90f9d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12c2a2c1858c6554 server_cookie=56f32a7b11089857 in_seq=0 out_seq=0 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.662+0000 7f4e85ffb640 1 -- 172.21.15.38:0/153232175 <== mds.0 v2:172.21.15.38:6834/3513331661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d6a894920 con 0x562d6a90f5f0 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.662+0000 7f4e85ffb640 10 client.10242 handle_command_reply: tid=0 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.662+0000 7f4eb03d1740 1 client.10242 shutdown 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.663+0000 7f4eb03d1740 1 -- 172.21.15.38:0/153232175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 msgr2=0x562d6a928080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.663+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a928080 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f4e9002a730 tx=0x7f4e9002a140 comp rx=0 tx=0).stop 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.663+0000 7f4eb03d1740 1 -- 172.21.15.38:0/153232175 shutdown_connections 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.663+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x562d6a90f5f0 0x562d6a90f9d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.663+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8ffbd0 0x562d6a911640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.663+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a928080 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.663+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/153232175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8fd230 0x562d6a8eefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.663+0000 7f4eb03d1740 1 -- 172.21.15.38:0/153232175 >> 172.21.15.38:0/153232175 conn(0x562d6a8d2f10 msgr2=0x562d6a8cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:26.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.663+0000 7f4e85ffb640 0 client.10242 ms_handle_reset on v2:172.21.15.38:6834/3513331661 2024-04-25T08:42:26.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.663+0000 7f4eb03d1740 1 -- 172.21.15.38:0/153232175 shutdown_connections 2024-04-25T08:42:26.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.663+0000 7f4eb03d1740 1 -- 172.21.15.38:0/153232175 wait complete. 2024-04-25T08:42:26.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.663+0000 7f4eb03d1740 20 client.10242 trim_cache size 0 max 16384 2024-04-25T08:42:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.679+0000 7f4eb03d1740 1 Processor -- start 2024-04-25T08:42:26.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.679+0000 7f4eb03d1740 1 -- start start 2024-04-25T08:42:26.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.679+0000 7f4eb03d1740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8fd230 0x562d6a8ee460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.679+0000 7f4eb03d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8ffbd0 0x562d6a91af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.679+0000 7f4eb03d1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a91b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.679+0000 7f4eb03d1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea8200630 con 0x562d6a91aad0 2024-04-25T08:42:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.679+0000 7f4eb03d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea80ccec0 con 0x562d6a8ffbd0 2024-04-25T08:42:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.679+0000 7f4eb03d1740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea81ff170 con 0x562d6a8fd230 2024-04-25T08:42:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.679+0000 7f4ea77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a91b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.679+0000 7f4ea77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a91b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60500/0 (socket says 172.21.15.38:60500) 2024-04-25T08:42:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.679+0000 7f4ea77fe640 1 -- 172.21.15.38:0/442199954 learned_addr learned my addr 172.21.15.38:0/442199954 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/442199954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8ffbd0 0x562d6a91af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/442199954 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8fd230 0x562d6a8ee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4ea77fe640 1 -- 172.21.15.38:0/442199954 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8fd230 msgr2=0x562d6a8ee460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4ea77fe640 1 --2- 172.21.15.38:0/442199954 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8fd230 0x562d6a8ee460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4ea77fe640 1 -- 172.21.15.38:0/442199954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8ffbd0 msgr2=0x562d6a91af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4ea77fe640 1 --2- 172.21.15.38:0/442199954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8ffbd0 0x562d6a91af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4ea77fe640 1 -- 172.21.15.38:0/442199954 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d6a8d3d40 con 0x562d6a91aad0 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/442199954 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8fd230 0x562d6a8ee460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4ea77fe640 1 --2- 172.21.15.38:0/442199954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a91b4c0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0008f00 tx=0x7f4ea000e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76120cf15e1284f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4e87fff640 1 -- 172.21.15.38:0/442199954 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea001d070 con 0x562d6a91aad0 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4e87fff640 1 -- 172.21.15.38:0/442199954 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea000b800 con 0x562d6a91aad0 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4e87fff640 1 -- 172.21.15.38:0/442199954 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea000bbf0 con 0x562d6a91aad0 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.680+0000 7f4eb03d1740 1 -- 172.21.15.38:0/442199954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 msgr2=0x562d6a91b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.681+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/442199954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a91b4c0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0008f00 tx=0x7f4ea000e7c0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.681+0000 7f4eb03d1740 1 -- 172.21.15.38:0/442199954 shutdown_connections 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.681+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/442199954 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a91aad0 0x562d6a91b4c0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.681+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/442199954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a8ffbd0 0x562d6a91af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.681+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/442199954 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8fd230 0x562d6a8ee460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.681+0000 7f4eb03d1740 1 -- 172.21.15.38:0/442199954 >> 172.21.15.38:0/442199954 conn(0x562d6a8d2f10 msgr2=0x562d6a912ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:26.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.681+0000 7f4eb03d1740 1 -- 172.21.15.38:0/442199954 shutdown_connections 2024-04-25T08:42:26.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.681+0000 7f4eb03d1740 1 -- 172.21.15.38:0/442199954 wait complete. 2024-04-25T08:42:26.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.681+0000 7f4eb03d1740 1 -- start start 2024-04-25T08:42:26.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.681+0000 7f4eb03d1740 1 Processor -- start 2024-04-25T08:42:26.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4eb03d1740 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8fd230 0x562d6a8fe900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4eb03d1740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a8ffbd0 0x562d6a9828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4eb03d1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a91aad0 0x562d6a90f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4eb03d1740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6a7deb30 con 0x562d6a8ffbd0 2024-04-25T08:42:26.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4eb03d1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d6a7f20c0 con 0x562d6a91aad0 2024-04-25T08:42:26.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4eb03d1740 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d6a92c040 con 0x562d6a8fd230 2024-04-25T08:42:26.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4eb03d1740 10 client.0 _ll_register_callbacks cb 0x562d6a8cdc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:42:26.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a8ffbd0 0x562d6a9828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4e9ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a8ffbd0 0x562d6a9828e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60508/0 (socket says 172.21.15.38:60508) 2024-04-25T08:42:26.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4e9ffff640 1 -- 172.21.15.38:0/297049317 learned_addr learned my addr 172.21.15.38:0/297049317 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4ea77fe640 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a91aad0 0x562d6a90f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8fd230 0x562d6a8fe900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4e9ffff640 1 -- 172.21.15.38:0/297049317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8fd230 msgr2=0x562d6a8fe900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8fd230 0x562d6a8fe900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4e9ffff640 1 -- 172.21.15.38:0/297049317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a91aad0 msgr2=0x562d6a90f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a91aad0 0x562d6a90f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4e9ffff640 1 -- 172.21.15.38:0/297049317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d6a911770 con 0x562d6a8ffbd0 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8fd230 0x562d6a8fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4e85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.682+0000 7f4e9ffff640 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a8ffbd0 0x562d6a9828e0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f4e900759e0 tx=0x7f4e90009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4517dffa3084f9bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4e85ffb640 1 -- 172.21.15.38:0/297049317 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e90091040 con 0x562d6a8ffbd0 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4e85ffb640 1 -- 172.21.15.38:0/297049317 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f4e9007d170 con 0x562d6a8ffbd0 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4eb03d1740 1 -- 172.21.15.38:0/297049317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ea000b800 con 0x562d6a8ffbd0 2024-04-25T08:42:26.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4e85ffb640 1 -- 172.21.15.38:0/297049317 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4e90098060 con 0x562d6a8ffbd0 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4eb03d1740 10 client.10248 fetch_fsmap learned FSMap version 91 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4eb03d1740 1 -- 172.21.15.38:0/297049317 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x562d6a911770 con 0x562d6a8ffbd0 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4e85ffb640 1 -- 172.21.15.38:0/297049317 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea000b800 con 0x562d6a8ffbd0 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4e85ffb640 1 -- 172.21.15.38:0/297049317 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 91) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f4e900a1040 con 0x562d6a8ffbd0 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4eb03d1740 10 client.10248 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4eb03d1740 10 client.10248 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4eb03d1740 20 client.10248 populate_metadata read hostname 'smithi038' 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x562d6a91b080 0x562d6a91b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4eb03d1740 4 client.10248 mds_command: new command op to 10041 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4eb03d1740 1 -- 172.21.15.38:0/297049317 --> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d6a842f50 con 0x562d6a91b080 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.683+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x562d6a91b080 0x562d6a91b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4ea6ffd640 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x562d6a91b080 0x562d6a91b460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3313416b9e43cc5 server_cookie=55b924e185656c36 in_seq=0 out_seq=0 2024-04-25T08:42:26.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4e85ffb640 10 client.10248 ms_handle_connect on v2:172.21.15.38:6834/3513331661 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4e85ffb640 1 -- 172.21.15.38:0/297049317 <== mds.0 v2:172.21.15.38:6834/3513331661 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x562d6a842f50 con 0x562d6a91b080 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4e85ffb640 10 client.10248 handle_command_reply: tid=0 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4eb03d1740 1 client.10248 shutdown 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4eb03d1740 1 -- 172.21.15.38:0/297049317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a8ffbd0 msgr2=0x562d6a9828e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a8ffbd0 0x562d6a9828e0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f4e900759e0 tx=0x7f4e90009040 comp rx=0 tx=0).stop 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4eb03d1740 1 -- 172.21.15.38:0/297049317 shutdown_connections 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.38:6834/3513331661,v1:172.21.15.38:6835/3513331661] conn(0x562d6a91b080 0x562d6a91b460 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562d6a91aad0 0x562d6a90f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562d6a8ffbd0 0x562d6a9828e0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4eb03d1740 1 --2- 172.21.15.38:0/297049317 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x562d6a8fd230 0x562d6a8fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4e85ffb640 0 client.10248 ms_handle_reset on v2:172.21.15.38:6834/3513331661 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4eb03d1740 1 -- 172.21.15.38:0/297049317 >> 172.21.15.38:0/297049317 conn(0x562d6a8d2f10 msgr2=0x562d6a8c8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4eb03d1740 1 -- 172.21.15.38:0/297049317 shutdown_connections 2024-04-25T08:42:26.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4eb03d1740 1 -- 172.21.15.38:0/297049317 wait complete. 2024-04-25T08:42:26.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4eb03d1740 20 client.10248 trim_cache size 0 max 16384 2024-04-25T08:42:26.684 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:42:26.684 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762", 2024-04-25T08:42:26.684 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-04-25T08:42:26.684 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 10041, 2024-04-25T08:42:26.684 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-04-25T08:42:26.684 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-04-25T08:42:26.684 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-04-25T08:42:26.684 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 22.012355453000001, 2024-04-25T08:42:26.685 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 91, 2024-04-25T08:42:26.685 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 145, 2024-04-25T08:42:26.685 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 145, 2024-04-25T08:42:26.685 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 27.020758573999998 2024-04-25T08:42:26.685 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:42:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4ead5f5640 1 -- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e8806b4d0 msgr2=0x7f4e8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4ead5f5640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e8806b4d0 0x7f4e8806d990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4e9800b820 tx=0x7f4e98008040 comp rx=0 tx=0).stop 2024-04-25T08:42:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4ead5f5640 1 -- 172.21.15.38:0/1647926444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea8160ef0 msgr2=0x7f4ea80d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.684+0000 7f4ead5f5640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea8160ef0 0x7f4ea80d2580 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f4e90000c00 tx=0x7f4e900046c0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.685+0000 7f4ead5f5640 1 -- 172.21.15.38:0/1647926444 shutdown_connections 2024-04-25T08:42:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.685+0000 7f4ead5f5640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4e8806b4d0 0x7f4e8806d990 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.685+0000 7f4ead5f5640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ea8155b10 0x7f4ea8154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.685+0000 7f4ead5f5640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ea8160ef0 0x7f4ea80d2580 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.685+0000 7f4ead5f5640 1 --2- 172.21.15.38:0/1647926444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ea80b2ab0 0x7f4ea80cb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.685+0000 7f4ead5f5640 1 -- 172.21.15.38:0/1647926444 >> 172.21.15.38:0/1647926444 conn(0x7f4ea8066d10 msgr2=0x7f4ea81600e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:26.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.685+0000 7f4ead5f5640 1 -- 172.21.15.38:0/1647926444 shutdown_connections 2024-04-25T08:42:26.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.685+0000 7f4ead5f5640 1 -- 172.21.15.38:0/1647926444 wait complete. 2024-04-25T08:42:26.687 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:42:26.688 DEBUG:teuthology.orchestra.run.smithi038:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:42:26.724 DEBUG:teuthology.orchestra.run.smithi107:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:42:26.765 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:42:26.765 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:42:26.765 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:26.781 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:26.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.929+0000 7f711ef5a640 1 Processor -- start 2024-04-25T08:42:26.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.929+0000 7f711ef5a640 1 -- start start 2024-04-25T08:42:26.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.930+0000 7f711ef5a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f711815dc10 0x7f71180ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.930+0000 7f711ef5a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71180ced60 0x7f71180d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.930+0000 7f711ef5a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f71180d3900 0x7f71180d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.930+0000 7f711ef5a640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711815dff0 con 0x7f711815dc10 2024-04-25T08:42:26.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.930+0000 7f711ef5a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71180b6990 con 0x7f71180d3900 2024-04-25T08:42:26.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.930+0000 7f711ef5a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71180d4710 con 0x7f71180ced60 2024-04-25T08:42:26.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.930+0000 7f711ccf6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f711815dc10 0x7f71180ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.930+0000 7f711ccf6640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f711815dc10 0x7f71180ce820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60520/0 (socket says 172.21.15.38:60520) 2024-04-25T08:42:26.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.930+0000 7f711ccf6640 1 -- 172.21.15.38:0/3687125697 learned_addr learned my addr 172.21.15.38:0/3687125697 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:26.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.930+0000 7f7117fff640 1 --2- 172.21.15.38:0/3687125697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71180ced60 0x7f71180d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.930+0000 7f711d4f7640 1 --2- 172.21.15.38:0/3687125697 >> v2:172.21.15.107:3300/0 conn(0x7f71180d3900 0x7f71180d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.931+0000 7f711ccf6640 1 -- 172.21.15.38:0/3687125697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71180ced60 msgr2=0x7f71180d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.931+0000 7f711ccf6640 1 --2- 172.21.15.38:0/3687125697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71180ced60 0x7f71180d0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.931+0000 7f711ccf6640 1 -- 172.21.15.38:0/3687125697 >> v2:172.21.15.107:3300/0 conn(0x7f71180d3900 msgr2=0x7f71180d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:26.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.931+0000 7f711ccf6640 1 --2- 172.21.15.38:0/3687125697 >> v2:172.21.15.107:3300/0 conn(0x7f71180d3900 0x7f71180d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.931+0000 7f711ccf6640 1 -- 172.21.15.38:0/3687125697 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71180d3e70 con 0x7f711815dc10 2024-04-25T08:42:26.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.931+0000 7f711ccf6640 1 --2- 172.21.15.38:0/3687125697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f711815dc10 0x7f71180ce820 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f71180b9380 tx=0x7f710c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a41fa187a32f3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.931+0000 7f71177fe640 1 -- 172.21.15.38:0/3687125697 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c037070 con 0x7f711815dc10 2024-04-25T08:42:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.932+0000 7f71177fe640 1 -- 172.21.15.38:0/3687125697 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c032440 con 0x7f711815dc10 2024-04-25T08:42:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.932+0000 7f71177fe640 1 -- 172.21.15.38:0/3687125697 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c032810 con 0x7f711815dc10 2024-04-25T08:42:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.932+0000 7f711ef5a640 1 -- 172.21.15.38:0/3687125697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f711815dc10 msgr2=0x7f71180ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.932+0000 7f711ef5a640 1 --2- 172.21.15.38:0/3687125697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f711815dc10 0x7f71180ce820 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f71180b9380 tx=0x7f710c02a140 comp rx=0 tx=0).stop 2024-04-25T08:42:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.932+0000 7f711ef5a640 1 -- 172.21.15.38:0/3687125697 shutdown_connections 2024-04-25T08:42:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.932+0000 7f711ef5a640 1 --2- 172.21.15.38:0/3687125697 >> v2:172.21.15.107:3300/0 conn(0x7f71180d3900 0x7f71180d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.932+0000 7f711ef5a640 1 --2- 172.21.15.38:0/3687125697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71180ced60 0x7f71180d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.932+0000 7f711ef5a640 1 --2- 172.21.15.38:0/3687125697 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f711815dc10 0x7f71180ce820 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.932+0000 7f711ef5a640 1 -- 172.21.15.38:0/3687125697 >> 172.21.15.38:0/3687125697 conn(0x7f7118066d10 msgr2=0x7f7118067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.932+0000 7f711ef5a640 1 -- 172.21.15.38:0/3687125697 shutdown_connections 2024-04-25T08:42:26.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.932+0000 7f711ef5a640 1 -- 172.21.15.38:0/3687125697 wait complete. 2024-04-25T08:42:26.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.933+0000 7f711ef5a640 1 Processor -- start 2024-04-25T08:42:26.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.933+0000 7f711ef5a640 1 -- start start 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.934+0000 7f711ef5a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71180ced60 0x7f7118158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.934+0000 7f711ef5a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71180d3900 0x7f71180cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.934+0000 7f711ef5a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f711815dc10 0x7f71181f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.934+0000 7f711ef5a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71180b6bc0 con 0x7f711815dc10 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.934+0000 7f711ef5a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71180d4430 con 0x7f71180ced60 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.934+0000 7f711ef5a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71180b2370 con 0x7f71180d3900 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.934+0000 7f711d4f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f711815dc10 0x7f71181f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.934+0000 7f711d4f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f711815dc10 0x7f71181f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60522/0 (socket says 172.21.15.38:60522) 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.934+0000 7f711d4f7640 1 -- 172.21.15.38:0/4260193536 learned_addr learned my addr 172.21.15.38:0/4260193536 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.934+0000 7f7117fff640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71180d3900 0x7f71180cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.935+0000 7f711d4f7640 1 -- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71180d3900 msgr2=0x7f71180cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.935+0000 7f711d4f7640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71180d3900 0x7f71180cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.935+0000 7f711d4f7640 1 -- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71180ced60 msgr2=0x7f7118158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.935+0000 7f711d4f7640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71180ced60 0x7f7118158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.935+0000 7f711d4f7640 1 -- 172.21.15.38:0/4260193536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71181590e0 con 0x7f711815dc10 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.935+0000 7f7117fff640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71180d3900 0x7f71180cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:26.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.935+0000 7f711d4f7640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f711815dc10 0x7f71181f6750 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f7110007e40 tx=0x7f711000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.935+0000 7f71157fa640 1 -- 172.21.15.38:0/4260193536 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7110014620 con 0x7f711815dc10 2024-04-25T08:42:26.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.935+0000 7f71157fa640 1 -- 172.21.15.38:0/4260193536 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7110019070 con 0x7f711815dc10 2024-04-25T08:42:26.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.935+0000 7f711ef5a640 1 -- 172.21.15.38:0/4260193536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71181593b0 con 0x7f711815dc10 2024-04-25T08:42:26.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.935+0000 7f71157fa640 1 -- 172.21.15.38:0/4260193536 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7110014a20 con 0x7f711815dc10 2024-04-25T08:42:26.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.935+0000 7f711ef5a640 1 -- 172.21.15.38:0/4260193536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7118159970 con 0x7f711815dc10 2024-04-25T08:42:26.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.936+0000 7f711ef5a640 1 -- 172.21.15.38:0/4260193536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7118068e80 con 0x7f711815dc10 2024-04-25T08:42:26.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.936+0000 7f71157fa640 1 -- 172.21.15.38:0/4260193536 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7110014bc0 con 0x7f711815dc10 2024-04-25T08:42:26.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.937+0000 7f71157fa640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f70fc06b4d0 0x7f70fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:26.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.937+0000 7f71157fa640 1 -- 172.21.15.38:0/4260193536 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f711008ad60 con 0x7f711815dc10 2024-04-25T08:42:26.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.937+0000 7f711ccf6640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f70fc06b4d0 0x7f70fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:26.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.938+0000 7f711ccf6640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f70fc06b4d0 0x7f70fc06d990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f710c034000 tx=0x7f710c02c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:26.933 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:26.941+0000 7f71157fa640 1 -- 172.21.15.38:0/4260193536 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71100599c0 con 0x7f711815dc10 2024-04-25T08:42:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.125+0000 7f711ef5a640 1 -- 172.21.15.38:0/4260193536 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71180b5f10 con 0x7f711815dc10 2024-04-25T08:42:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.127+0000 7f71157fa640 1 -- 172.21.15.38:0/4260193536 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+17179 (secure 0 0 0) 0x7f71100592c0 con 0x7f711815dc10 2024-04-25T08:42:27.119 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:27.119 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":145,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:42:19.745001+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:01.691830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:03.024111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.8","mappings":[{"from":4,"to":1}]},{"pgid":"22.c","mappings":[{"from":3,"to":1}]},{"pgid":"22.e","mappings":[{"from":3,"to":0}]},{"pgid":"22.21","mappings":[{"from":3,"to":0}]},{"pgid":"22.23","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.129+0000 7f711ef5a640 1 -- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f70fc06b4d0 msgr2=0x7f70fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.129+0000 7f711ef5a640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f70fc06b4d0 0x7f70fc06d990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f710c034000 tx=0x7f710c02c000 comp rx=0 tx=0).stop 2024-04-25T08:42:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.129+0000 7f711ef5a640 1 -- 172.21.15.38:0/4260193536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f711815dc10 msgr2=0x7f71181f6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.129+0000 7f711ef5a640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f711815dc10 0x7f71181f6750 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f7110007e40 tx=0x7f711000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.129+0000 7f711ef5a640 1 -- 172.21.15.38:0/4260193536 shutdown_connections 2024-04-25T08:42:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.129+0000 7f711ef5a640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f711815dc10 0x7f71181f6750 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.129+0000 7f711ef5a640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f70fc06b4d0 0x7f70fc06d990 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.129+0000 7f711ef5a640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f71180d3900 0x7f71180cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.129+0000 7f711ef5a640 1 --2- 172.21.15.38:0/4260193536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71180ced60 0x7f7118158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.129+0000 7f711ef5a640 1 -- 172.21.15.38:0/4260193536 >> 172.21.15.38:0/4260193536 conn(0x7f7118066d10 msgr2=0x7f711815cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.129+0000 7f711ef5a640 1 -- 172.21.15.38:0/4260193536 shutdown_connections 2024-04-25T08:42:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.129+0000 7f711ef5a640 1 -- 172.21.15.38:0/4260193536 wait complete. 2024-04-25T08:42:27.132 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:01.691830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:03.024111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:42:27.132 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:42:27.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.244+0000 7fd975088640 1 Processor -- start 2024-04-25T08:42:27.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.244+0000 7fd975088640 1 -- start start 2024-04-25T08:42:27.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.245+0000 7fd975088640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd97015df30 0x7fd97009a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.245+0000 7fd975088640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd970162390 0x7fd970099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.245+0000 7fd975088640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97015f300 con 0x7fd970099fb0 2024-04-25T08:42:27.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.245+0000 7fd975088640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97015e310 con 0x7fd970162390 2024-04-25T08:42:27.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.245+0000 7fd975088640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97009aeb0 con 0x7fd97015df30 2024-04-25T08:42:27.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.245+0000 7fd96ed9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd97015df30 0x7fd97009a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.245+0000 7fd96e59c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd970162390 0x7fd970099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.245+0000 7fd96ed9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd97015df30 0x7fd97009a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.38:39098/0 (socket says 172.21.15.38:39098) 2024-04-25T08:42:27.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.245+0000 7fd96ed9d640 1 -- 172.21.15.38:0/4041486013 learned_addr learned my addr 172.21.15.38:0/4041486013 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:27.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.245+0000 7fd96f59e640 1 --1- 172.21.15.38:0/4041486013 >> v1:172.21.15.107:6789/0 conn(0x7fd970099fb0 0x7fd97006bf90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:36272/0 (socket says 172.21.15.38:36272) 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.246+0000 7fd96ed9d640 1 -- 172.21.15.38:0/4041486013 >> v2:172.21.15.107:3300/0 conn(0x7fd970162390 msgr2=0x7fd970099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.246+0000 7fd96ed9d640 1 --2- 172.21.15.38:0/4041486013 >> v2:172.21.15.107:3300/0 conn(0x7fd970162390 0x7fd970099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.246+0000 7fd96ed9d640 1 -- 172.21.15.38:0/4041486013 >> v1:172.21.15.107:6789/0 conn(0x7fd970099fb0 legacy=0x7fd97006bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.246+0000 7fd96ed9d640 1 -- 172.21.15.38:0/4041486013 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9701627f0 con 0x7fd97015df30 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.246+0000 7fd96e59c640 1 --2- 172.21.15.38:0/4041486013 >> v2:172.21.15.107:3300/0 conn(0x7fd970162390 0x7fd970099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.246+0000 7fd96ed9d640 1 --2- 172.21.15.38:0/4041486013 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd97015df30 0x7fd97009a7b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd960009870 tx=0x7fd96002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74fa8e55074463ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.247+0000 7fd96dd9b640 1 -- 172.21.15.38:0/4041486013 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960037070 con 0x7fd97015df30 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.247+0000 7fd96dd9b640 1 -- 172.21.15.38:0/4041486013 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd960032440 con 0x7fd97015df30 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.247+0000 7fd96dd9b640 1 -- 172.21.15.38:0/4041486013 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960032810 con 0x7fd97015df30 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.247+0000 7fd975088640 1 -- 172.21.15.38:0/4041486013 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd97015df30 msgr2=0x7fd97009a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.247+0000 7fd975088640 1 --2- 172.21.15.38:0/4041486013 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd97015df30 0x7fd97009a7b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd960009870 tx=0x7fd96002a250 comp rx=0 tx=0).stop 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.247+0000 7fd975088640 1 -- 172.21.15.38:0/4041486013 shutdown_connections 2024-04-25T08:42:27.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.247+0000 7fd975088640 1 --2- 172.21.15.38:0/4041486013 >> v2:172.21.15.107:3300/0 conn(0x7fd970162390 0x7fd970099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.247+0000 7fd975088640 1 --2- 172.21.15.38:0/4041486013 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd97015df30 0x7fd97009a7b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.247+0000 7fd975088640 1 -- 172.21.15.38:0/4041486013 >> 172.21.15.38:0/4041486013 conn(0x7fd9700b3110 msgr2=0x7fd9700b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:27.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.247+0000 7fd975088640 1 -- 172.21.15.38:0/4041486013 shutdown_connections 2024-04-25T08:42:27.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.247+0000 7fd975088640 1 -- 172.21.15.38:0/4041486013 wait complete. 2024-04-25T08:42:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.249+0000 7fd975088640 1 Processor -- start 2024-04-25T08:42:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.249+0000 7fd975088640 1 -- start start 2024-04-25T08:42:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.249+0000 7fd975088640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd970099fb0 0x7fd970153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.249+0000 7fd975088640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd97015df30 0x7fd9700cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.249+0000 7fd975088640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd970162390 0x7fd970158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.249+0000 7fd975088640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97015e820 con 0x7fd970162390 2024-04-25T08:42:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.249+0000 7fd975088640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97015ee70 con 0x7fd97015df30 2024-04-25T08:42:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.249+0000 7fd975088640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97015eb00 con 0x7fd970099fb0 2024-04-25T08:42:27.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd96f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd970162390 0x7fd970158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd96f59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd970162390 0x7fd970158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60530/0 (socket says 172.21.15.38:60530) 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd96f59e640 1 -- 172.21.15.38:0/1525460695 learned_addr learned my addr 172.21.15.38:0/1525460695 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd96ed9d640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd970099fb0 0x7fd970153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd96e59c640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd97015df30 0x7fd9700cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd96f59e640 1 -- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd970099fb0 msgr2=0x7fd970153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd96f59e640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd970099fb0 0x7fd970153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd96f59e640 1 -- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd97015df30 msgr2=0x7fd9700cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd96f59e640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd97015df30 0x7fd9700cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd96f59e640 1 -- 172.21.15.38:0/1525460695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9700cdcc0 con 0x7fd970162390 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd96f59e640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd970162390 0x7fd970158cb0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fd96800c730 tx=0x7fd96800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd9677fe640 1 -- 172.21.15.38:0/1525460695 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968017070 con 0x7fd970162390 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd9677fe640 1 -- 172.21.15.38:0/1525460695 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96800ce40 con 0x7fd970162390 2024-04-25T08:42:27.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd975088640 1 -- 172.21.15.38:0/1525460695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9700cbdb0 con 0x7fd970162390 2024-04-25T08:42:27.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd9677fe640 1 -- 172.21.15.38:0/1525460695 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96800a840 con 0x7fd970162390 2024-04-25T08:42:27.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.250+0000 7fd975088640 1 -- 172.21.15.38:0/1525460695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9700cc370 con 0x7fd970162390 2024-04-25T08:42:27.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.251+0000 7fd975088640 1 -- 172.21.15.38:0/1525460695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd970068e80 con 0x7fd970162390 2024-04-25T08:42:27.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.252+0000 7fd9677fe640 1 -- 172.21.15.38:0/1525460695 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd96801a070 con 0x7fd970162390 2024-04-25T08:42:27.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.252+0000 7fd9677fe640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd95006b4d0 0x7fd95006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.252+0000 7fd9677fe640 1 -- 172.21.15.38:0/1525460695 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fd968089950 con 0x7fd970162390 2024-04-25T08:42:27.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.253+0000 7fd96ed9d640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd95006b4d0 0x7fd95006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.253+0000 7fd96ed9d640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd95006b4d0 0x7fd95006d990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd960034000 tx=0x7fd96002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.257+0000 7fd9677fe640 1 -- 172.21.15.38:0/1525460695 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd968057e40 con 0x7fd970162390 2024-04-25T08:42:27.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.439+0000 7fd975088640 1 -- 172.21.15.38:0/1525460695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9700b4020 con 0x7fd970162390 2024-04-25T08:42:27.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.440+0000 7fd9677fe640 1 -- 172.21.15.38:0/1525460695 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd96805bee0 con 0x7fd970162390 2024-04-25T08:42:27.432 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:42:27.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.442+0000 7fd975088640 1 -- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd95006b4d0 msgr2=0x7fd95006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.442+0000 7fd975088640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd95006b4d0 0x7fd95006d990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd960034000 tx=0x7fd96002e000 comp rx=0 tx=0).stop 2024-04-25T08:42:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.442+0000 7fd975088640 1 -- 172.21.15.38:0/1525460695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd970162390 msgr2=0x7fd970158cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.442+0000 7fd975088640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd970162390 0x7fd970158cb0 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fd96800c730 tx=0x7fd96800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.442+0000 7fd975088640 1 -- 172.21.15.38:0/1525460695 shutdown_connections 2024-04-25T08:42:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.442+0000 7fd975088640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd970162390 0x7fd970158cb0 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.442+0000 7fd975088640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd97015df30 0x7fd9700cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.442+0000 7fd975088640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd95006b4d0 0x7fd95006d990 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.442+0000 7fd975088640 1 --2- 172.21.15.38:0/1525460695 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd970099fb0 0x7fd970153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.442+0000 7fd975088640 1 -- 172.21.15.38:0/1525460695 >> 172.21.15.38:0/1525460695 conn(0x7fd9700b3110 msgr2=0x7fd97006bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.442+0000 7fd975088640 1 -- 172.21.15.38:0/1525460695 shutdown_connections 2024-04-25T08:42:27.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.442+0000 7fd975088640 1 -- 172.21.15.38:0/1525460695 wait complete. 2024-04-25T08:42:27.445 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:42:27.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.558+0000 7f7f4c122640 1 Processor -- start 2024-04-25T08:42:27.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.558+0000 7f7f4c122640 1 -- start start 2024-04-25T08:42:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.559+0000 7f7f4c122640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7f4406b5c0 0x7f7f440cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.559+0000 7f7f4c122640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f4406bad0 con 0x7f7f4415df70 2024-04-25T08:42:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.559+0000 7f7f4c122640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f440cc950 con 0x7f7f4415db90 2024-04-25T08:42:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.559+0000 7f7f4c122640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f440cd280 con 0x7f7f4406b5c0 2024-04-25T08:42:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.559+0000 7f7f4a6bf640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f7f4415df70 0x7f7f440cab70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49294/0 (socket says 172.21.15.38:49294) 2024-04-25T08:42:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.559+0000 7f7f4a6bf640 1 -- 172.21.15.38:0/926541519 learned_addr learned my addr 172.21.15.38:0/926541519 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.559+0000 7f7f496bd640 1 --2- 172.21.15.38:0/926541519 >> v2:172.21.15.107:3300/0 conn(0x7f7f4406b5c0 0x7f7f440cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.559+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413521206 0 0) 0x7f7f4406bad0 con 0x7f7f4415df70 2024-04-25T08:42:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.559+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f30003520 con 0x7f7f4415df70 2024-04-25T08:42:27.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.559+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2702932832 0 0) 0x7f7f30003520 con 0x7f7f4415df70 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.559+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 >> v2:172.21.15.107:3300/0 conn(0x7f7f4406b5c0 msgr2=0x7f7f440cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.559+0000 7f7f48ebc640 1 --2- 172.21.15.38:0/926541519 >> v2:172.21.15.107:3300/0 conn(0x7f7f4406b5c0 0x7f7f440cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 >> v1:172.21.15.107:6789/0 conn(0x7f7f4415db90 legacy=0x7f7f440cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f440ccc30 con 0x7f7f4415df70 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f7f3c0019f0 con 0x7f7f4415df70 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f496bd640 1 --2- 172.21.15.38:0/926541519 >> v2:172.21.15.107:3300/0 conn(0x7f7f4406b5c0 0x7f7f440cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 >> v1:172.21.15.38:6789/0 conn(0x7f7f4415df70 legacy=0x7f7f440cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f48ebc640 1 --2- 172.21.15.38:0/926541519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f48ebc640 1 --2- 172.21.15.38:0/926541519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f30004800 0x7f7f30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f48ebc640 1 --2- 172.21.15.38:0/926541519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f300051c0 0x7f7f3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440cd280 con 0x7f7f300039b0 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3000c160 con 0x7f7f30004800 2024-04-25T08:42:27.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3000c360 con 0x7f7f300051c0 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f49ebe640 1 --2- 172.21.15.38:0/926541519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f496bd640 1 --2- 172.21.15.38:0/926541519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f30004800 0x7f7f30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f4a6bf640 1 --2- 172.21.15.38:0/926541519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f300051c0 0x7f7f3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f49ebe640 1 -- 172.21.15.38:0/926541519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f300051c0 msgr2=0x7f7f3000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f49ebe640 1 --2- 172.21.15.38:0/926541519 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f300051c0 0x7f7f3000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f49ebe640 1 -- 172.21.15.38:0/926541519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f30004800 msgr2=0x7f7f30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f49ebe640 1 --2- 172.21.15.38:0/926541519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f30004800 0x7f7f30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f49ebe640 1 -- 172.21.15.38:0/926541519 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f30003520 con 0x7f7f300039b0 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.560+0000 7f7f4a6bf640 1 -- 172.21.15.38:0/926541519 reap_dead start 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.561+0000 7f7f49ebe640 1 --2- 172.21.15.38:0/926541519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f7f38001750 tx=0x7f7f30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7af70250f197e4ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.561+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38001370 con 0x7f7f300039b0 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.561+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f380052d0 con 0x7f7f300039b0 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.561+0000 7f7f48ebc640 1 -- 172.21.15.38:0/926541519 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3802abc0 con 0x7f7f300039b0 2024-04-25T08:42:27.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.561+0000 7f7f4c122640 1 -- 172.21.15.38:0/926541519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f300039b0 msgr2=0x7f7f30003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.561+0000 7f7f4c122640 1 --2- 172.21.15.38:0/926541519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f7f38001750 tx=0x7f7f30005830 comp rx=0 tx=0).stop 2024-04-25T08:42:27.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.561+0000 7f7f4c122640 1 -- 172.21.15.38:0/926541519 shutdown_connections 2024-04-25T08:42:27.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.561+0000 7f7f4c122640 1 --2- 172.21.15.38:0/926541519 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f300039b0 0x7f7f30003dd0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.561+0000 7f7f4c122640 1 -- 172.21.15.38:0/926541519 >> 172.21.15.38:0/926541519 conn(0x7f7f440d0520 msgr2=0x7f7f440d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:27.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.562+0000 7f7f4c122640 1 -- 172.21.15.38:0/926541519 shutdown_connections 2024-04-25T08:42:27.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.562+0000 7f7f4c122640 1 -- 172.21.15.38:0/926541519 wait complete. 2024-04-25T08:42:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.563+0000 7f7f4c122640 1 Processor -- start 2024-04-25T08:42:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.563+0000 7f7f4c122640 1 -- start start 2024-04-25T08:42:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.563+0000 7f7f4c122640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f300039b0 0x7f7f4415be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.563+0000 7f7f4c122640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f440b4660 0x7f7f440b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.563+0000 7f7f4c122640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f440b59f0 0x7f7f440b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.563+0000 7f7f4c122640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440af620 con 0x7f7f440b59f0 2024-04-25T08:42:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.563+0000 7f7f4c122640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440af4a0 con 0x7f7f300039b0 2024-04-25T08:42:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.563+0000 7f7f4c122640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f440af8d0 con 0x7f7f440b4660 2024-04-25T08:42:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f4a6bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f440b59f0 0x7f7f440b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f4a6bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f440b59f0 0x7f7f440b5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60560/0 (socket says 172.21.15.38:60560) 2024-04-25T08:42:27.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f4a6bf640 1 -- 172.21.15.38:0/3923459473 learned_addr learned my addr 172.21.15.38:0/3923459473 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f496bd640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f440b4660 0x7f7f440b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f49ebe640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f300039b0 0x7f7f4415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f4a6bf640 1 -- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f440b4660 msgr2=0x7f7f440b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f4a6bf640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f440b4660 0x7f7f440b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f4a6bf640 1 -- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f300039b0 msgr2=0x7f7f4415be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f4a6bf640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f300039b0 0x7f7f4415be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f4a6bf640 1 -- 172.21.15.38:0/3923459473 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f440b35c0 con 0x7f7f440b59f0 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f49ebe640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f300039b0 0x7f7f4415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f4a6bf640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f440b59f0 0x7f7f440b5e70 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f7f44071de0 tx=0x7f7f3c007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f367fc640 1 -- 172.21.15.38:0/3923459473 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c00ab60 con 0x7f7f440b59f0 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f367fc640 1 -- 172.21.15.38:0/3923459473 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3c019070 con 0x7f7f440b59f0 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f4c122640 1 -- 172.21.15.38:0/3923459473 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f440b38a0 con 0x7f7f440b59f0 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f367fc640 1 -- 172.21.15.38:0/3923459473 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c0133f0 con 0x7f7f440b59f0 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.564+0000 7f7f4c122640 1 -- 172.21.15.38:0/3923459473 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f440b3e60 con 0x7f7f440b59f0 2024-04-25T08:42:27.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.565+0000 7f7f4c122640 1 -- 172.21.15.38:0/3923459473 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f44068e80 con 0x7f7f440b59f0 2024-04-25T08:42:27.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.566+0000 7f7f367fc640 1 -- 172.21.15.38:0/3923459473 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f3c013590 con 0x7f7f440b59f0 2024-04-25T08:42:27.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.566+0000 7f7f367fc640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f2806b4d0 0x7f7f2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.566+0000 7f7f367fc640 1 -- 172.21.15.38:0/3923459473 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f7f3c08a9d0 con 0x7f7f440b59f0 2024-04-25T08:42:27.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.566+0000 7f7f49ebe640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f2806b4d0 0x7f7f2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.567+0000 7f7f49ebe640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f2806b4d0 0x7f7f2806d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7f38032000 tx=0x7f7f3802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.562 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.570+0000 7f7f367fc640 1 -- 172.21.15.38:0/3923459473 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f3c0595a0 con 0x7f7f440b59f0 2024-04-25T08:42:27.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.752+0000 7f7f4c122640 1 -- 172.21.15.38:0/3923459473 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f4406bad0 con 0x7f7f440b59f0 2024-04-25T08:42:27.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.753+0000 7f7f367fc640 1 -- 172.21.15.38:0/3923459473 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f3c058ea0 con 0x7f7f440b59f0 2024-04-25T08:42:27.745 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:42:27.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.755+0000 7f7f4c122640 1 -- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f2806b4d0 msgr2=0x7f7f2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.755+0000 7f7f4c122640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f2806b4d0 0x7f7f2806d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7f38032000 tx=0x7f7f3802e000 comp rx=0 tx=0).stop 2024-04-25T08:42:27.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.755+0000 7f7f4c122640 1 -- 172.21.15.38:0/3923459473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f440b59f0 msgr2=0x7f7f440b5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.755+0000 7f7f4c122640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f440b59f0 0x7f7f440b5e70 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f7f44071de0 tx=0x7f7f3c007b20 comp rx=0 tx=0).stop 2024-04-25T08:42:27.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.755+0000 7f7f4c122640 1 -- 172.21.15.38:0/3923459473 shutdown_connections 2024-04-25T08:42:27.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.755+0000 7f7f4c122640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7f440b59f0 0x7f7f440b5e70 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.755+0000 7f7f4c122640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7f2806b4d0 0x7f7f2806d990 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.755+0000 7f7f4c122640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f440b4660 0x7f7f440b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.755+0000 7f7f4c122640 1 --2- 172.21.15.38:0/3923459473 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f300039b0 0x7f7f4415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.755+0000 7f7f4c122640 1 -- 172.21.15.38:0/3923459473 >> 172.21.15.38:0/3923459473 conn(0x7f7f440d0520 msgr2=0x7f7f4415cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.755+0000 7f7f4c122640 1 -- 172.21.15.38:0/3923459473 shutdown_connections 2024-04-25T08:42:27.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.755+0000 7f7f4c122640 1 -- 172.21.15.38:0/3923459473 wait complete. 2024-04-25T08:42:27.758 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:42:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.871+0000 7fe9da012640 1 Processor -- start 2024-04-25T08:42:27.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.871+0000 7fe9da012640 1 -- start start 2024-04-25T08:42:27.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.871+0000 7fe9da012640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d409a180 0x7fe9d409a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.871+0000 7fe9da012640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d4162820 con 0x7fe9d415df30 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.871+0000 7fe9da012640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d415f300 con 0x7fe9d4162390 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.871+0000 7fe9da012640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d415e310 con 0x7fe9d409a180 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d37fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe9d415df30 0x7fe9d4099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49304/0 (socket says 172.21.15.38:49304) 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d37fe640 1 -- 172.21.15.38:0/3909905645 learned_addr learned my addr 172.21.15.38:0/3909905645 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d2ffd640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d409a180 0x7fe9d409a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969106487 0 0) 0x7fe9c8001110 con 0x7fe9d415df30 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9bc003520 con 0x7fe9d415df30 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 543256299 0 0) 0x7fe9bc003520 con 0x7fe9d415df30 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d409a180 msgr2=0x7fe9d409a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d409a180 0x7fe9d409a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 >> v1:172.21.15.107:6789/0 conn(0x7fe9d4162390 legacy=0x7fe9d40cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d415e540 con 0x7fe9d415df30 2024-04-25T08:42:27.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fe9c8001840 con 0x7fe9d415df30 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 >> v1:172.21.15.38:6789/0 conn(0x7fe9d415df30 legacy=0x7fe9d4099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d409a180 0x7fe9bc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9bc003f20 0x7fe9bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9bc004d40 0x7fe9bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc00b9f0 con 0x7fe9bc003f20 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc00bba0 con 0x7fe9bc004d40 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc00bda0 con 0x7fe9d409a180 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d2ffd640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9bc003f20 0x7fe9bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d37fe640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d409a180 0x7fe9bc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.872+0000 7fe9d3fff640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9bc004d40 0x7fe9bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.873+0000 7fe9d2ffd640 1 -- 172.21.15.38:0/3909905645 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d409a180 msgr2=0x7fe9bc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.873+0000 7fe9d2ffd640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d409a180 0x7fe9bc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.873+0000 7fe9d2ffd640 1 -- 172.21.15.38:0/3909905645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9bc004d40 msgr2=0x7fe9bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.873+0000 7fe9d2ffd640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9bc004d40 0x7fe9bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.873+0000 7fe9d2ffd640 1 -- 172.21.15.38:0/3909905645 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc003520 con 0x7fe9bc003f20 2024-04-25T08:42:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.873+0000 7fe9d3fff640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9bc004d40 0x7fe9bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.873+0000 7fe9d2ffd640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9bc003f20 0x7fe9bc004800 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009ac0 tx=0x7fe9bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c25102959fdb015 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.873+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4036070 con 0x7fe9bc003f20 2024-04-25T08:42:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.874+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c4031400 con 0x7fe9bc003f20 2024-04-25T08:42:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.874+0000 7fe9d27fc640 1 -- 172.21.15.38:0/3909905645 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4009690 con 0x7fe9bc003f20 2024-04-25T08:42:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.874+0000 7fe9da012640 1 -- 172.21.15.38:0/3909905645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9bc003f20 msgr2=0x7fe9bc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.874+0000 7fe9da012640 1 --2- 172.21.15.38:0/3909905645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9bc003f20 0x7fe9bc004800 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4009ac0 tx=0x7fe9bc005830 comp rx=0 tx=0).stop 2024-04-25T08:42:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.874+0000 7fe9d3fff640 1 -- 172.21.15.38:0/3909905645 reap_dead start 2024-04-25T08:42:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.874+0000 7fe9da012640 1 -- 172.21.15.38:0/3909905645 shutdown_connections 2024-04-25T08:42:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.874+0000 7fe9da012640 1 -- 172.21.15.38:0/3909905645 >> 172.21.15.38:0/3909905645 conn(0x7fe9d40b3110 msgr2=0x7fe9d40b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.874+0000 7fe9da012640 1 -- 172.21.15.38:0/3909905645 shutdown_connections 2024-04-25T08:42:27.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.874+0000 7fe9da012640 1 -- 172.21.15.38:0/3909905645 wait complete. 2024-04-25T08:42:27.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.875+0000 7fe9da012640 1 Processor -- start 2024-04-25T08:42:27.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.875+0000 7fe9da012640 1 -- start start 2024-04-25T08:42:27.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.876+0000 7fe9da012640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9bc003f20 0x7fe9d4153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.876+0000 7fe9da012640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d41578a0 0x7fe9d4160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.876+0000 7fe9da012640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d4161040 0x7fe9d406f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.876+0000 7fe9da012640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d415ee70 con 0x7fe9bc003f20 2024-04-25T08:42:27.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.876+0000 7fe9da012640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d409aeb0 con 0x7fe9d4161040 2024-04-25T08:42:27.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.876+0000 7fe9da012640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d415eb00 con 0x7fe9d41578a0 2024-04-25T08:42:27.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.876+0000 7fe9d37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9bc003f20 0x7fe9d4153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.876+0000 7fe9d37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9bc003f20 0x7fe9d4153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60586/0 (socket says 172.21.15.38:60586) 2024-04-25T08:42:27.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.876+0000 7fe9d37fe640 1 -- 172.21.15.38:0/3334438780 learned_addr learned my addr 172.21.15.38:0/3334438780 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9d37fe640 1 -- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d41578a0 msgr2=0x7fe9d4160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9d2ffd640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d41578a0 0x7fe9d4160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9d3fff640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d4161040 0x7fe9d406f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9d37fe640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d41578a0 0x7fe9d4160b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9d37fe640 1 -- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d4161040 msgr2=0x7fe9d406f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9d37fe640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d4161040 0x7fe9d406f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9d37fe640 1 -- 172.21.15.38:0/3334438780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d40b67d0 con 0x7fe9bc003f20 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9d37fe640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9bc003f20 0x7fe9d4153920 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8007290 tx=0x7fe9c8007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9bbfff640 1 -- 172.21.15.38:0/3334438780 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800f9c0 con 0x7fe9bc003f20 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9bbfff640 1 -- 172.21.15.38:0/3334438780 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c8018070 con 0x7fe9bc003f20 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9da012640 1 -- 172.21.15.38:0/3334438780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d40b6a90 con 0x7fe9bc003f20 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9bbfff640 1 -- 172.21.15.38:0/3334438780 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800fdc0 con 0x7fe9bc003f20 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.877+0000 7fe9da012640 1 -- 172.21.15.38:0/3334438780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d40b7050 con 0x7fe9bc003f20 2024-04-25T08:42:27.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.878+0000 7fe9da012640 1 -- 172.21.15.38:0/3334438780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d4068e80 con 0x7fe9bc003f20 2024-04-25T08:42:27.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.878+0000 7fe9bbfff640 1 -- 172.21.15.38:0/3334438780 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9c801c410 con 0x7fe9bc003f20 2024-04-25T08:42:27.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.879+0000 7fe9bbfff640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9c006b500 0x7fe9c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:27.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.879+0000 7fe9bbfff640 1 -- 172.21.15.38:0/3334438780 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fe9c808a260 con 0x7fe9bc003f20 2024-04-25T08:42:27.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.879+0000 7fe9d2ffd640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9c006b500 0x7fe9c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:27.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.880+0000 7fe9d2ffd640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9c006b500 0x7fe9c006d9c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4033000 tx=0x7fe9c4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:27.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:27.883+0000 7fe9bbfff640 1 -- 172.21.15.38:0/3334438780 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9d4162820 con 0x7fe9bc003f20 2024-04-25T08:42:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.061+0000 7fe9da012640 1 -- 172.21.15.38:0/3334438780 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe9d40b3de0 con 0x7fe9bc003f20 2024-04-25T08:42:28.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.062+0000 7fe9bbfff640 1 -- 172.21.15.38:0/3334438780 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe9c8058ec0 con 0x7fe9bc003f20 2024-04-25T08:42:28.054 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:42:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.064+0000 7fe9da012640 1 -- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9c006b500 msgr2=0x7fe9c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.064+0000 7fe9da012640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9c006b500 0x7fe9c006d9c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4033000 tx=0x7fe9c4008030 comp rx=0 tx=0).stop 2024-04-25T08:42:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.064+0000 7fe9da012640 1 -- 172.21.15.38:0/3334438780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9bc003f20 msgr2=0x7fe9d4153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.064+0000 7fe9da012640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9bc003f20 0x7fe9d4153920 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8007290 tx=0x7fe9c8007750 comp rx=0 tx=0).stop 2024-04-25T08:42:28.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.064+0000 7fe9da012640 1 -- 172.21.15.38:0/3334438780 shutdown_connections 2024-04-25T08:42:28.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.064+0000 7fe9da012640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9d4161040 0x7fe9d406f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.064+0000 7fe9da012640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe9c006b500 0x7fe9c006d9c0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.064+0000 7fe9da012640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe9d41578a0 0x7fe9d4160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.064+0000 7fe9da012640 1 --2- 172.21.15.38:0/3334438780 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe9bc003f20 0x7fe9d4153920 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.064+0000 7fe9da012640 1 -- 172.21.15.38:0/3334438780 >> 172.21.15.38:0/3334438780 conn(0x7fe9d40b3110 msgr2=0x7fe9d40701d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.064+0000 7fe9da012640 1 -- 172.21.15.38:0/3334438780 shutdown_connections 2024-04-25T08:42:28.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.064+0000 7fe9da012640 1 -- 172.21.15.38:0/3334438780 wait complete. 2024-04-25T08:42:28.067 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:42:28.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.176+0000 7f138c9e0640 1 Processor -- start 2024-04-25T08:42:28.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.176+0000 7f138c9e0640 1 -- start start 2024-04-25T08:42:28.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.177+0000 7f138c9e0640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f138809a180 0x7f13880cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.177+0000 7f138c9e0640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13880cfc70 con 0x7f138815def0 2024-04-25T08:42:28.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.177+0000 7f138c9e0640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13880cff50 con 0x7f13880d0d10 2024-04-25T08:42:28.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.177+0000 7f138c9e0640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138815e2d0 con 0x7f138809a180 2024-04-25T08:42:28.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.177+0000 7f138659c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f138815def0 0x7f138806c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:49312/0 (socket says 172.21.15.38:49312) 2024-04-25T08:42:28.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.177+0000 7f138659c640 1 -- 172.21.15.38:0/452850316 learned_addr learned my addr 172.21.15.38:0/452850316 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:28.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.177+0000 7f1386d9d640 1 --2- 172.21.15.38:0/452850316 >> v2:172.21.15.107:3300/0 conn(0x7f138809a180 0x7f13880cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.178+0000 7f138559a640 1 -- 172.21.15.38:0/452850316 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239967947 0 0) 0x7f13880cfc70 con 0x7f138815def0 2024-04-25T08:42:28.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.178+0000 7f138559a640 1 -- 172.21.15.38:0/452850316 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1374003520 con 0x7f138815def0 2024-04-25T08:42:28.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.178+0000 7f138559a640 1 -- 172.21.15.38:0/452850316 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3642586406 0 0) 0x7f13880cff50 con 0x7f13880d0d10 2024-04-25T08:42:28.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.178+0000 7f138559a640 1 -- 172.21.15.38:0/452850316 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13880cfc70 con 0x7f13880d0d10 2024-04-25T08:42:28.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.178+0000 7f1386d9d640 1 -- 172.21.15.38:0/452850316 >> v1:172.21.15.107:6789/0 conn(0x7f13880d0d10 legacy=0x7f1388099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.178+0000 7f1386d9d640 1 -- 172.21.15.38:0/452850316 >> v1:172.21.15.38:6789/0 conn(0x7f138815def0 legacy=0x7f138806c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.178+0000 7f138559a640 1 -- 172.21.15.38:0/452850316 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1290247575 0 0) 0x7f1374003520 con 0x7f138815def0 2024-04-25T08:42:28.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.178+0000 7f1386d9d640 1 -- 172.21.15.38:0/452850316 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880d0230 con 0x7f138809a180 2024-04-25T08:42:28.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.179+0000 7f1386d9d640 1 --2- 172.21.15.38:0/452850316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f138809a180 0x7f13880cbc50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f13780097a0 tx=0x7f137802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8c0a936de4bcfc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.179+0000 7f138559a640 1 -- 172.21.15.38:0/452850316 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378037070 con 0x7f138809a180 2024-04-25T08:42:28.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.179+0000 7f138559a640 1 -- 172.21.15.38:0/452850316 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137802ac50 con 0x7f138809a180 2024-04-25T08:42:28.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.179+0000 7f138559a640 1 -- 172.21.15.38:0/452850316 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13800019f0 con 0x7f138809a180 2024-04-25T08:42:28.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.179+0000 7f138c9e0640 1 -- 172.21.15.38:0/452850316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f138809a180 msgr2=0x7f13880cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.179+0000 7f138c9e0640 1 --2- 172.21.15.38:0/452850316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f138809a180 0x7f13880cbc50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f13780097a0 tx=0x7f137802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.179+0000 7f138c9e0640 1 -- 172.21.15.38:0/452850316 shutdown_connections 2024-04-25T08:42:28.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.179+0000 7f138c9e0640 1 --2- 172.21.15.38:0/452850316 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f138809a180 0x7f13880cbc50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.179+0000 7f138c9e0640 1 -- 172.21.15.38:0/452850316 >> 172.21.15.38:0/452850316 conn(0x7f13880b30f0 msgr2=0x7f13880b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.179+0000 7f138c9e0640 1 -- 172.21.15.38:0/452850316 shutdown_connections 2024-04-25T08:42:28.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.179+0000 7f138c9e0640 1 -- 172.21.15.38:0/452850316 wait complete. 2024-04-25T08:42:28.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.181+0000 7f138c9e0640 1 Processor -- start 2024-04-25T08:42:28.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.181+0000 7f138c9e0640 1 -- start start 2024-04-25T08:42:28.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.181+0000 7f138c9e0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f138809a180 0x7f13881f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.181+0000 7f138c9e0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13880d0d10 0x7f13880c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.181+0000 7f138c9e0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f138815def0 0x7f13880c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.181+0000 7f138c9e0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13880d1170 con 0x7f13880d0d10 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.181+0000 7f138c9e0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13880d07f0 con 0x7f138815def0 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.181+0000 7f138c9e0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13880cf8d0 con 0x7f138809a180 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f1385d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13880d0d10 0x7f13880c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f1385d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13880d0d10 0x7f13880c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60604/0 (socket says 172.21.15.38:60604) 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f1385d9b640 1 -- 172.21.15.38:0/2992885004 learned_addr learned my addr 172.21.15.38:0/2992885004 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f138659c640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f138809a180 0x7f13881f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f1386d9d640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f138815def0 0x7f13880c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f1385d9b640 1 -- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f138809a180 msgr2=0x7f13881f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f1385d9b640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f138809a180 0x7f13881f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f1385d9b640 1 -- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f138815def0 msgr2=0x7f13880c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f1385d9b640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f138815def0 0x7f13880c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f1385d9b640 1 -- 172.21.15.38:0/2992885004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13881f75b0 con 0x7f13880d0d10 2024-04-25T08:42:28.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f1385d9b640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13880d0d10 0x7f13880c5130 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f137000c4c0 tx=0x7f137000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f137effd640 1 -- 172.21.15.38:0/2992885004 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13700052d0 con 0x7f13880d0d10 2024-04-25T08:42:28.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f137effd640 1 -- 172.21.15.38:0/2992885004 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1370005470 con 0x7f13880d0d10 2024-04-25T08:42:28.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f138c9e0640 1 -- 172.21.15.38:0/2992885004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13881f7830 con 0x7f13880d0d10 2024-04-25T08:42:28.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f137effd640 1 -- 172.21.15.38:0/2992885004 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1370012870 con 0x7f13880d0d10 2024-04-25T08:42:28.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.182+0000 7f138c9e0640 1 -- 172.21.15.38:0/2992885004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13881f7df0 con 0x7f13880d0d10 2024-04-25T08:42:28.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.183+0000 7f138c9e0640 1 -- 172.21.15.38:0/2992885004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138804ef90 con 0x7f13880d0d10 2024-04-25T08:42:28.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.184+0000 7f137effd640 1 -- 172.21.15.38:0/2992885004 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1370012a10 con 0x7f13880d0d10 2024-04-25T08:42:28.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.184+0000 7f137effd640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f136806b500 0x7f136806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.184+0000 7f137effd640 1 -- 172.21.15.38:0/2992885004 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f1370089ef0 con 0x7f13880d0d10 2024-04-25T08:42:28.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.185+0000 7f138659c640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f136806b500 0x7f136806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.185+0000 7f138659c640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f136806b500 0x7f136806d9c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1380001b60 tx=0x7f1380009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.188+0000 7f137effd640 1 -- 172.21.15.38:0/2992885004 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f13881f75b0 con 0x7f13880d0d10 2024-04-25T08:42:28.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.380+0000 7f138c9e0640 1 -- 172.21.15.38:0/2992885004 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f138809af90 con 0x7f13880d0d10 2024-04-25T08:42:28.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.380+0000 7f137effd640 1 -- 172.21.15.38:0/2992885004 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1370058590 con 0x7f13880d0d10 2024-04-25T08:42:28.373 INFO:teuthology.orchestra.run.smithi038.stderr:listed 0 entries 2024-04-25T08:42:28.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.382+0000 7f138c9e0640 1 -- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f136806b500 msgr2=0x7f136806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.382+0000 7f138c9e0640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f136806b500 0x7f136806d9c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1380001b60 tx=0x7f1380009040 comp rx=0 tx=0).stop 2024-04-25T08:42:28.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.382+0000 7f138c9e0640 1 -- 172.21.15.38:0/2992885004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13880d0d10 msgr2=0x7f13880c5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.382+0000 7f138c9e0640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13880d0d10 0x7f13880c5130 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f137000c4c0 tx=0x7f137000c980 comp rx=0 tx=0).stop 2024-04-25T08:42:28.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.382+0000 7f138c9e0640 1 -- 172.21.15.38:0/2992885004 shutdown_connections 2024-04-25T08:42:28.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.382+0000 7f138c9e0640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f138815def0 0x7f13880c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.382+0000 7f138c9e0640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13880d0d10 0x7f13880c5130 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.382+0000 7f138c9e0640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f136806b500 0x7f136806d9c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.382+0000 7f138c9e0640 1 --2- 172.21.15.38:0/2992885004 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f138809a180 0x7f13881f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.382+0000 7f138c9e0640 1 -- 172.21.15.38:0/2992885004 >> 172.21.15.38:0/2992885004 conn(0x7f13880b30f0 msgr2=0x7f13880cbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.382+0000 7f138c9e0640 1 -- 172.21.15.38:0/2992885004 shutdown_connections 2024-04-25T08:42:28.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.382+0000 7f138c9e0640 1 -- 172.21.15.38:0/2992885004 wait complete. 2024-04-25T08:42:28.385 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-04-25T08:42:28.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:28.386 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:42:28.408 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi038.stderr:ceph-fuse[99897]: fuse finished with error 0 and tester_r 0 2024-04-25T08:42:28.504 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:42:28.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:28.504 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:42:28.671 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:42:28.671 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:28.688 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:42:28.688 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:42:28.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:28.688 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:42:28.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:42:28.744 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:42:28.744 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:42:28.744 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:28.760 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:42:28.760 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:42:28.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:28.760 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:42:28.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:42:28.815 INFO:teuthology.orchestra.run.smithi038.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:42:28.816 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:42:28.816 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:28.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:42:28.833 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:42:28.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:28.833 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:42:28.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:42:28.889 INFO:teuthology.orchestra.run.smithi107.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:42:28.889 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:28.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.998+0000 7f9ab6815640 1 Processor -- start 2024-04-25T08:42:28.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.998+0000 7f9ab6815640 1 -- start start 2024-04-25T08:42:28.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.999+0000 7f9ab6815640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9ab015a760 0x7f9ab015b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.999+0000 7f9ab6815640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab015ab40 con 0x7f9ab006e630 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.999+0000 7f9ab6815640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab00d0710 con 0x7f9ab00b42e0 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.999+0000 7f9ab6815640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab006eba0 con 0x7f9ab015a760 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.999+0000 7f9aaffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9ab015a760 0x7f9ab015b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.999+0000 7f9aaf7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9ab006e630 0x7f9ab015ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40444/0 (socket says 172.21.15.38:40444) 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.999+0000 7f9aaffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9ab015a760 0x7f9ab015b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40314/0 (socket says 172.21.15.38:40314) 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.999+0000 7f9aaf7fe640 1 -- 172.21.15.38:0/4277446371 learned_addr learned my addr 172.21.15.38:0/4277446371 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.999+0000 7f9aaffff640 1 -- 172.21.15.38:0/4277446371 >> v1:172.21.15.107:6789/0 conn(0x7f9ab00b42e0 legacy=0x7f9ab00cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.999+0000 7f9aaffff640 1 -- 172.21.15.38:0/4277446371 >> v1:172.21.15.38:6789/0 conn(0x7f9ab006e630 legacy=0x7f9ab015ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.999+0000 7f9aaffff640 1 -- 172.21.15.38:0/4277446371 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab006ee80 con 0x7f9ab015a760 2024-04-25T08:42:28.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:28.999+0000 7f9aaeffd640 1 -- 172.21.15.38:0/4277446371 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908022351 0 0) 0x7f9ab015ab40 con 0x7f9ab006e630 2024-04-25T08:42:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.000+0000 7f9aaffff640 1 --2- 172.21.15.38:0/4277446371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ab015a760 0x7f9ab015b720 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00097a0 tx=0x7f9aa002a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9964be22e6b4af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.000+0000 7f9aaeffd640 1 -- 172.21.15.38:0/4277446371 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0037070 con 0x7f9ab015a760 2024-04-25T08:42:28.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.000+0000 7f9aaeffd640 1 -- 172.21.15.38:0/4277446371 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa002adc0 con 0x7f9ab015a760 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.000+0000 7f9aaeffd640 1 -- 172.21.15.38:0/4277446371 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0032660 con 0x7f9ab015a760 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.001+0000 7f9ab6815640 1 -- 172.21.15.38:0/4277446371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ab015a760 msgr2=0x7f9ab015b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.001+0000 7f9ab6815640 1 --2- 172.21.15.38:0/4277446371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ab015a760 0x7f9ab015b720 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00097a0 tx=0x7f9aa002a0b0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.001+0000 7f9ab6815640 1 -- 172.21.15.38:0/4277446371 shutdown_connections 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.001+0000 7f9ab6815640 1 --2- 172.21.15.38:0/4277446371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ab015a760 0x7f9ab015b720 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.001+0000 7f9ab6815640 1 -- 172.21.15.38:0/4277446371 >> 172.21.15.38:0/4277446371 conn(0x7f9ab00cf970 msgr2=0x7f9ab00cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.001+0000 7f9ab6815640 1 -- 172.21.15.38:0/4277446371 shutdown_connections 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.001+0000 7f9ab6815640 1 -- 172.21.15.38:0/4277446371 wait complete. 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.002+0000 7f9ab6815640 1 Processor -- start 2024-04-25T08:42:28.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.002+0000 7f9ab6815640 1 -- start start 2024-04-25T08:42:28.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9ab6815640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ab006e630 0x7f9ab0157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9ab6815640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ab00b42e0 0x7f9ab015b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9ab6815640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ab015a760 0x7f9ab0145380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:28.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9ab6815640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab00b4730 con 0x7f9ab00b42e0 2024-04-25T08:42:28.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9ab6815640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab015e6f0 con 0x7f9ab006e630 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9ab6815640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab00b48b0 con 0x7f9ab015a760 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9aaf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ab00b42e0 0x7f9ab015b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9aaf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ab00b42e0 0x7f9ab015b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40320/0 (socket says 172.21.15.38:40320) 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9aaf7fe640 1 -- 172.21.15.38:0/663592525 learned_addr learned my addr 172.21.15.38:0/663592525 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9aaffff640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ab006e630 0x7f9ab0157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9ab4db2640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ab015a760 0x7f9ab0145380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9aaf7fe640 1 -- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ab015a760 msgr2=0x7f9ab0145380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9aaf7fe640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ab015a760 0x7f9ab0145380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9aaf7fe640 1 -- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ab006e630 msgr2=0x7f9ab0157a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9aaf7fe640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ab006e630 0x7f9ab0157a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.003+0000 7f9aaf7fe640 1 -- 172.21.15.38:0/663592525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab015be00 con 0x7f9ab00b42e0 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.004+0000 7f9aaf7fe640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ab00b42e0 0x7f9ab015b720 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800c4d0 tx=0x7f9a9800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:28.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.004+0000 7f9aacff9640 1 -- 172.21.15.38:0/663592525 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98018070 con 0x7f9ab00b42e0 2024-04-25T08:42:28.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.004+0000 7f9aacff9640 1 -- 172.21.15.38:0/663592525 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a98007500 con 0x7f9ab00b42e0 2024-04-25T08:42:28.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.004+0000 7f9ab6815640 1 -- 172.21.15.38:0/663592525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab015c0e0 con 0x7f9ab00b42e0 2024-04-25T08:42:28.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.004+0000 7f9aacff9640 1 -- 172.21.15.38:0/663592525 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98004a20 con 0x7f9ab00b42e0 2024-04-25T08:42:28.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.004+0000 7f9ab6815640 1 -- 172.21.15.38:0/663592525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab015c6a0 con 0x7f9ab00b42e0 2024-04-25T08:42:28.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.005+0000 7f9ab6815640 1 -- 172.21.15.38:0/663592525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab0068e80 con 0x7f9ab00b42e0 2024-04-25T08:42:29.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.005+0000 7f9aacff9640 1 -- 172.21.15.38:0/663592525 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a9801b070 con 0x7f9ab00b42e0 2024-04-25T08:42:29.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.006+0000 7f9aacff9640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9a9006b4d0 0x7f9a9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.006+0000 7f9aacff9640 1 -- 172.21.15.38:0/663592525 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f9a98089760 con 0x7f9ab00b42e0 2024-04-25T08:42:29.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.006+0000 7f9aaffff640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9a9006b4d0 0x7f9a9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.007+0000 7f9aaffff640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9a9006b4d0 0x7f9a9006d990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0034000 tx=0x7f9aa002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:29.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.010+0000 7f9aacff9640 1 -- 172.21.15.38:0/663592525 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a98057cd0 con 0x7f9ab00b42e0 2024-04-25T08:42:29.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.229+0000 7f9ab6815640 1 -- 172.21.15.38:0/663592525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ab00b4be0 con 0x7f9ab00b42e0 2024-04-25T08:42:29.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.230+0000 7f9aacff9640 1 -- 172.21.15.38:0/663592525 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4960 (secure 0 0 0) 0x7f9a9805bd70 con 0x7f9ab00b42e0 2024-04-25T08:42:29.222 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:29.222 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2682394530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2682394530},{"type":"v1","addr":"172.21.15.107:6837","nonce":2682394530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2810760276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2810760276},{"type":"v1","addr":"172.21.15.107:6835","nonce":2810760276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T08:42:04.044929+0000","modified":"2024-04-25T08:42:25.774447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10038":{"gid":10038,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3318593502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3318593502},{"type":"v1","addr":"172.21.15.38:6833","nonce":3318593502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10041":{"gid":10041,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/3513331661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3513331661},{"type":"v1","addr":"172.21.15.38:6835","nonce":3513331661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:42:29.223 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 91 2024-04-25T08:42:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.231+0000 7f9ab6815640 1 -- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9a9006b4d0 msgr2=0x7f9a9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.231+0000 7f9ab6815640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9a9006b4d0 0x7f9a9006d990 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0034000 tx=0x7f9aa002e000 comp rx=0 tx=0).stop 2024-04-25T08:42:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.232+0000 7f9ab6815640 1 -- 172.21.15.38:0/663592525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ab00b42e0 msgr2=0x7f9ab015b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.232+0000 7f9ab6815640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ab00b42e0 0x7f9ab015b720 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f9a9800c4d0 tx=0x7f9a9800c990 comp rx=0 tx=0).stop 2024-04-25T08:42:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.232+0000 7f9ab6815640 1 -- 172.21.15.38:0/663592525 shutdown_connections 2024-04-25T08:42:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.232+0000 7f9ab6815640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9a9006b4d0 0x7f9a9006d990 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.232+0000 7f9ab6815640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9ab015a760 0x7f9ab0145380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.232+0000 7f9ab6815640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ab00b42e0 0x7f9ab015b720 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.232+0000 7f9ab6815640 1 --2- 172.21.15.38:0/663592525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9ab006e630 0x7f9ab0157a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.232+0000 7f9ab6815640 1 -- 172.21.15.38:0/663592525 >> 172.21.15.38:0/663592525 conn(0x7f9ab00cf970 msgr2=0x7f9ab00b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.232+0000 7f9ab6815640 1 -- 172.21.15.38:0/663592525 shutdown_connections 2024-04-25T08:42:29.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.232+0000 7f9ab6815640 1 -- 172.21.15.38:0/663592525 wait complete. 2024-04-25T08:42:29.235 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.344+0000 7f4da5b7e640 1 Processor -- start 2024-04-25T08:42:29.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.344+0000 7f4da5b7e640 1 -- start start 2024-04-25T08:42:29.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4da5b7e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4da015dc40 0x7f4da00d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4da5b7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4da00d0e10 0x7f4da00b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4da5b7e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da00d1220 con 0x7f4da00b9580 2024-04-25T08:42:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4da5b7e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da00d19b0 con 0x7f4da015dc40 2024-04-25T08:42:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4da5b7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00b8400 con 0x7f4da00d0e10 2024-04-25T08:42:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4d9f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4da015dc40 0x7f4da00d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4d9f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4da015dc40 0x7f4da00d08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40328/0 (socket says 172.21.15.38:40328) 2024-04-25T08:42:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4d9f7fe640 1 -- 172.21.15.38:0/3312964871 learned_addr learned my addr 172.21.15.38:0/3312964871 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4d9ffff640 1 --2- 172.21.15.38:0/3312964871 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4da00d0e10 0x7f4da00b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4d9f7fe640 1 -- 172.21.15.38:0/3312964871 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4da00d0e10 msgr2=0x7f4da00b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4d9f7fe640 1 --2- 172.21.15.38:0/3312964871 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4da00d0e10 0x7f4da00b7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4d9f7fe640 1 -- 172.21.15.38:0/3312964871 >> v1:172.21.15.107:6789/0 conn(0x7f4da00b9580 legacy=0x7f4da00d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:29.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.345+0000 7f4d9f7fe640 1 -- 172.21.15.38:0/3312964871 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da00b86e0 con 0x7f4da015dc40 2024-04-25T08:42:29.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.346+0000 7f4d9f7fe640 1 --2- 172.21.15.38:0/3312964871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da015dc40 0x7f4da00d08d0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009870 tx=0x7f4d9002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd1dc809681292d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:29.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.346+0000 7f4d9e7fc640 1 -- 172.21.15.38:0/3312964871 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90037070 con 0x7f4da015dc40 2024-04-25T08:42:29.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.346+0000 7f4d9e7fc640 1 -- 172.21.15.38:0/3312964871 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d90032440 con 0x7f4da015dc40 2024-04-25T08:42:29.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.346+0000 7f4d9e7fc640 1 -- 172.21.15.38:0/3312964871 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90032810 con 0x7f4da015dc40 2024-04-25T08:42:29.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.346+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3312964871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da015dc40 msgr2=0x7f4da00d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.346+0000 7f4da5b7e640 1 --2- 172.21.15.38:0/3312964871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da015dc40 0x7f4da00d08d0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009870 tx=0x7f4d9002a180 comp rx=0 tx=0).stop 2024-04-25T08:42:29.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.347+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3312964871 shutdown_connections 2024-04-25T08:42:29.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.347+0000 7f4da5b7e640 1 --2- 172.21.15.38:0/3312964871 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4da00d0e10 0x7f4da00b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.347+0000 7f4da5b7e640 1 --2- 172.21.15.38:0/3312964871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da015dc40 0x7f4da00d08d0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.347+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3312964871 >> 172.21.15.38:0/3312964871 conn(0x7f4da0066ce0 msgr2=0x7f4da00670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:29.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.347+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3312964871 shutdown_connections 2024-04-25T08:42:29.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.347+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3312964871 wait complete. 2024-04-25T08:42:29.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.348+0000 7f4da5b7e640 1 Processor -- start 2024-04-25T08:42:29.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.348+0000 7f4da5b7e640 1 -- start start 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4da5b7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da00b9580 0x7f4da00cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4da5b7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da00d0e10 0x7f4da00c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4da5b7e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4da015dc40 0x7f4da015ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4da5b7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00b7fb0 con 0x7f4da00d0e10 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4da5b7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00b8ca0 con 0x7f4da00b9580 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4da5b7e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00b8130 con 0x7f4da015dc40 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da00d0e10 0x7f4da00c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da00d0e10 0x7f4da00c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40330/0 (socket says 172.21.15.38:40330) 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9effd640 1 -- 172.21.15.38:0/3237314963 learned_addr learned my addr 172.21.15.38:0/3237314963 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9ffff640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4da015dc40 0x7f4da015ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9f7fe640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da00b9580 0x7f4da00cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9effd640 1 -- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4da015dc40 msgr2=0x7f4da015ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9effd640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4da015dc40 0x7f4da015ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9effd640 1 -- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da00b9580 msgr2=0x7f4da00cce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9effd640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da00b9580 0x7f4da00cce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9effd640 1 -- 172.21.15.38:0/3237314963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da00cbdc0 con 0x7f4da00d0e10 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9ffff640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4da015dc40 0x7f4da015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9f7fe640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da00b9580 0x7f4da00cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d9effd640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da00d0e10 0x7f4da00c8ac0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400c460 tx=0x7f4d9400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d87fff640 1 -- 172.21.15.38:0/3237314963 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94016020 con 0x7f4da00d0e10 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d87fff640 1 -- 172.21.15.38:0/3237314963 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d94012870 con 0x7f4da00d0e10 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3237314963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da00cc0a0 con 0x7f4da00d0e10 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4d87fff640 1 -- 172.21.15.38:0/3237314963 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94012c60 con 0x7f4da00d0e10 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.349+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3237314963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da00cc660 con 0x7f4da00d0e10 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.350+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3237314963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da0068e80 con 0x7f4da00d0e10 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.351+0000 7f4d87fff640 1 -- 172.21.15.38:0/3237314963 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d94019020 con 0x7f4da00d0e10 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.351+0000 7f4d87fff640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d8c06b4d0 0x7f4d8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.352+0000 7f4d87fff640 1 -- 172.21.15.38:0/3237314963 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f4d9408a140 con 0x7f4da00d0e10 2024-04-25T08:42:29.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.352+0000 7f4d9f7fe640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d8c06b4d0 0x7f4d8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.352+0000 7f4d9f7fe640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d8c06b4d0 0x7f4d8c06d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4d90034000 tx=0x7f4d9002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:29.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.356+0000 7f4d87fff640 1 -- 172.21.15.38:0/3237314963 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d94020030 con 0x7f4da00d0e10 2024-04-25T08:42:29.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.539+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3237314963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4da00b7780 con 0x7f4da00d0e10 2024-04-25T08:42:29.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.540+0000 7f4d87fff640 1 -- 172.21.15.38:0/3237314963 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+17179 (secure 0 0 0) 0x7f4d94058da0 con 0x7f4da00d0e10 2024-04-25T08:42:29.532 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:29.533 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":145,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:42:19.745001+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:01.691830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:03.024111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.8","mappings":[{"from":4,"to":1}]},{"pgid":"22.c","mappings":[{"from":3,"to":1}]},{"pgid":"22.e","mappings":[{"from":3,"to":0}]},{"pgid":"22.21","mappings":[{"from":3,"to":0}]},{"pgid":"22.23","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.542+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d8c06b4d0 msgr2=0x7f4d8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.542+0000 7f4da5b7e640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d8c06b4d0 0x7f4d8c06d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f4d90034000 tx=0x7f4d9002e000 comp rx=0 tx=0).stop 2024-04-25T08:42:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.542+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3237314963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da00d0e10 msgr2=0x7f4da00c8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.542+0000 7f4da5b7e640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da00d0e10 0x7f4da00c8ac0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400c460 tx=0x7f4d9400c920 comp rx=0 tx=0).stop 2024-04-25T08:42:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.542+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3237314963 shutdown_connections 2024-04-25T08:42:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.542+0000 7f4da5b7e640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4d8c06b4d0 0x7f4d8c06d990 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.542+0000 7f4da5b7e640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4da015dc40 0x7f4da015ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.542+0000 7f4da5b7e640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4da00d0e10 0x7f4da00c8ac0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.542+0000 7f4da5b7e640 1 --2- 172.21.15.38:0/3237314963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4da00b9580 0x7f4da00cce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.542+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3237314963 >> 172.21.15.38:0/3237314963 conn(0x7f4da0066ce0 msgr2=0x7f4da006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:29.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.542+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3237314963 shutdown_connections 2024-04-25T08:42:29.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.542+0000 7f4da5b7e640 1 -- 172.21.15.38:0/3237314963 wait complete. 2024-04-25T08:42:29.546 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:31:23.247458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:42:01.691830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:42:03.024111+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:42:29.546 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:42:29.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.657+0000 7faa9c97e640 1 Processor -- start 2024-04-25T08:42:29.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.657+0000 7faa9c97e640 1 -- start start 2024-04-25T08:42:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.658+0000 7faa9c97e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7faa980ccfa0 0x7faa980cd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.658+0000 7faa9c97e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980cdc90 0x7faa980cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.658+0000 7faa9c97e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa980cd8c0 con 0x7faa980ce280 2024-04-25T08:42:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.658+0000 7faa9c97e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa980cc4b0 con 0x7faa980ccfa0 2024-04-25T08:42:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.658+0000 7faa9c97e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa980ad570 con 0x7faa980cdc90 2024-04-25T08:42:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.658+0000 7faa96d9d640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7faa980ce280 0x7faa980ce710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40454/0 (socket says 172.21.15.38:40454) 2024-04-25T08:42:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.658+0000 7faa96d9d640 1 -- 172.21.15.38:0/4006324531 learned_addr learned my addr 172.21.15.38:0/4006324531 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa8ffff640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980cdc90 0x7faa980cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa9659c640 1 --2- 172.21.15.38:0/4006324531 >> v2:172.21.15.107:3300/0 conn(0x7faa980ccfa0 0x7faa980cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830526850 0 0) 0x7faa980cd8c0 con 0x7faa980ce280 2024-04-25T08:42:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa84003520 con 0x7faa980ce280 2024-04-25T08:42:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3776880492 0 0) 0x7faa84003520 con 0x7faa980ce280 2024-04-25T08:42:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980cdc90 msgr2=0x7faa980cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:29.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980cdc90 0x7faa980cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 >> v2:172.21.15.107:3300/0 conn(0x7faa980ccfa0 msgr2=0x7faa980cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 --2- 172.21.15.38:0/4006324531 >> v2:172.21.15.107:3300/0 conn(0x7faa980ccfa0 0x7faa980cd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa980ad850 con 0x7faa980ce280 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7faa900019f0 con 0x7faa980ce280 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 >> v1:172.21.15.38:6789/0 conn(0x7faa980ce280 legacy=0x7faa980ce710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980cdc90 0x7faa840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa84003f30 0x7faa84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa84004d40 0x7faa840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa980cc4b0 con 0x7faa84003f30 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa980ad570 con 0x7faa84004d40 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8400b9f0 con 0x7faa980cdc90 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa8ffff640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa84003f30 0x7faa84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa9659c640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980cdc90 0x7faa840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.659+0000 7faa96d9d640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa84004d40 0x7faa840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.660+0000 7faa8ffff640 1 -- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980cdc90 msgr2=0x7faa840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.660+0000 7faa8ffff640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980cdc90 0x7faa840039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.660+0000 7faa8ffff640 1 -- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa84004d40 msgr2=0x7faa840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.660+0000 7faa8ffff640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa84004d40 0x7faa840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.660+0000 7faa8ffff640 1 -- 172.21.15.38:0/4006324531 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa84003520 con 0x7faa84003f30 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.660+0000 7faa96d9d640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa84004d40 0x7faa840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.660+0000 7faa9659c640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa980cdc90 0x7faa840039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.660+0000 7faa8ffff640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa84003f30 0x7faa84004800 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7faa8000bd60 tx=0x7faa84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abf6ffdb9a560588 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.660+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8003a070 con 0x7faa84003f30 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.660+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa80035440 con 0x7faa84003f30 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.660+0000 7faa95d9b640 1 -- 172.21.15.38:0/4006324531 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8000b930 con 0x7faa84003f30 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.661+0000 7faa9c97e640 1 -- 172.21.15.38:0/4006324531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa84003f30 msgr2=0x7faa84004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.661+0000 7faa9c97e640 1 --2- 172.21.15.38:0/4006324531 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa84003f30 0x7faa84004800 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7faa8000bd60 tx=0x7faa84005830 comp rx=0 tx=0).stop 2024-04-25T08:42:29.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.661+0000 7faa96d9d640 1 -- 172.21.15.38:0/4006324531 reap_dead start 2024-04-25T08:42:29.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.661+0000 7faa9c97e640 1 -- 172.21.15.38:0/4006324531 shutdown_connections 2024-04-25T08:42:29.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.661+0000 7faa9c97e640 1 -- 172.21.15.38:0/4006324531 >> 172.21.15.38:0/4006324531 conn(0x7faa980d06b0 msgr2=0x7faa980d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:29.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.661+0000 7faa9c97e640 1 -- 172.21.15.38:0/4006324531 shutdown_connections 2024-04-25T08:42:29.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.661+0000 7faa9c97e640 1 -- 172.21.15.38:0/4006324531 wait complete. 2024-04-25T08:42:29.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.662+0000 7faa9c97e640 1 Processor -- start 2024-04-25T08:42:29.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.662+0000 7faa9c97e640 1 -- start start 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.663+0000 7faa9c97e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa84003f30 0x7faa980b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.663+0000 7faa9c97e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980c7780 0x7faa981ef210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.663+0000 7faa9c97e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa980c6620 0x7faa98151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.663+0000 7faa9c97e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa980cee20 con 0x7faa980c7780 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.663+0000 7faa9c97e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9815ad80 con 0x7faa980c6620 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.663+0000 7faa9c97e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa980cefa0 con 0x7faa84003f30 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.663+0000 7faa8ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980c7780 0x7faa981ef210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.663+0000 7faa8ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980c7780 0x7faa981ef210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40350/0 (socket says 172.21.15.38:40350) 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.663+0000 7faa8ffff640 1 -- 172.21.15.38:0/3763383159 learned_addr learned my addr 172.21.15.38:0/3763383159 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.663+0000 7faa9659c640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa84003f30 0x7faa980b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.663+0000 7faa96d9d640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa980c6620 0x7faa98151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.663+0000 7faa8ffff640 1 -- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa84003f30 msgr2=0x7faa980b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.664+0000 7faa8ffff640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa84003f30 0x7faa980b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.664+0000 7faa8ffff640 1 -- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa980c6620 msgr2=0x7faa98151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.664+0000 7faa8ffff640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa980c6620 0x7faa98151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.664+0000 7faa8ffff640 1 -- 172.21.15.38:0/3763383159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa980c8790 con 0x7faa980c7780 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.664+0000 7faa9659c640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa84003f30 0x7faa980b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.664+0000 7faa8ffff640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980c7780 0x7faa981ef210 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7faa80038040 tx=0x7faa80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.664+0000 7faa8effd640 1 -- 172.21.15.38:0/3763383159 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa80043020 con 0x7faa980c7780 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.664+0000 7faa8effd640 1 -- 172.21.15.38:0/3763383159 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8003b040 con 0x7faa980c7780 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.664+0000 7faa9c97e640 1 -- 172.21.15.38:0/3763383159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa980c7ef0 con 0x7faa980c7780 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.664+0000 7faa8effd640 1 -- 172.21.15.38:0/3763383159 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8003a070 con 0x7faa980c7780 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.664+0000 7faa9c97e640 1 -- 172.21.15.38:0/3763383159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa980c84b0 con 0x7faa980c7780 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.665+0000 7faa9c97e640 1 -- 172.21.15.38:0/3763383159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa98068e80 con 0x7faa980c7780 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.665+0000 7faa8effd640 1 -- 172.21.15.38:0/3763383159 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa80046020 con 0x7faa980c7780 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.666+0000 7faa8effd640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faa7806b4d0 0x7faa7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.666+0000 7faa8effd640 1 -- 172.21.15.38:0/3763383159 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7faa800ad290 con 0x7faa980c7780 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.666+0000 7faa9659c640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faa7806b4d0 0x7faa7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.667+0000 7faa9659c640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faa7806b4d0 0x7faa7806d990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7faa88009bd0 tx=0x7faa88008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:29.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.670+0000 7faa8effd640 1 -- 172.21.15.38:0/3763383159 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa8004d070 con 0x7faa980c7780 2024-04-25T08:42:29.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.849+0000 7faa9c97e640 1 -- 172.21.15.38:0/3763383159 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faa9806e9c0 con 0x7faa980c7780 2024-04-25T08:42:29.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.850+0000 7faa8effd640 1 -- 172.21.15.38:0/3763383159 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7faa8007b810 con 0x7faa980c7780 2024-04-25T08:42:29.843 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-04-25T08:42:29.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.852+0000 7faa9c97e640 1 -- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faa7806b4d0 msgr2=0x7faa7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.852+0000 7faa9c97e640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faa7806b4d0 0x7faa7806d990 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7faa88009bd0 tx=0x7faa88008040 comp rx=0 tx=0).stop 2024-04-25T08:42:29.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.852+0000 7faa9c97e640 1 -- 172.21.15.38:0/3763383159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980c7780 msgr2=0x7faa981ef210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.852+0000 7faa9c97e640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980c7780 0x7faa981ef210 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7faa80038040 tx=0x7faa80004730 comp rx=0 tx=0).stop 2024-04-25T08:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.852+0000 7faa9c97e640 1 -- 172.21.15.38:0/3763383159 shutdown_connections 2024-04-25T08:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.852+0000 7faa9c97e640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faa980c6620 0x7faa98151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.852+0000 7faa9c97e640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faa980c7780 0x7faa981ef210 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.852+0000 7faa9c97e640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7faa7806b4d0 0x7faa7806d990 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.852+0000 7faa9c97e640 1 --2- 172.21.15.38:0/3763383159 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7faa84003f30 0x7faa980b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.852+0000 7faa9c97e640 1 -- 172.21.15.38:0/3763383159 >> 172.21.15.38:0/3763383159 conn(0x7faa980d06b0 msgr2=0x7faa980cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.852+0000 7faa9c97e640 1 -- 172.21.15.38:0/3763383159 shutdown_connections 2024-04-25T08:42:29.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.852+0000 7faa9c97e640 1 -- 172.21.15.38:0/3763383159 wait complete. 2024-04-25T08:42:29.855 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:42:29.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.966+0000 7fc24f2fc640 1 Processor -- start 2024-04-25T08:42:29.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.966+0000 7fc24f2fc640 1 -- start start 2024-04-25T08:42:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.967+0000 7fc24f2fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2480d0320 0x7fc2480d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.967+0000 7fc24f2fc640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2480d2060 con 0x7fc2480d0700 2024-04-25T08:42:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.967+0000 7fc24f2fc640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2480d2330 con 0x7fc2480b6980 2024-04-25T08:42:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.967+0000 7fc24f2fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480d2610 con 0x7fc2480d0320 2024-04-25T08:42:29.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.967+0000 7fc24c897640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc2480d0700 0x7fc2480d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40470/0 (socket says 172.21.15.38:40470) 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.967+0000 7fc24c897640 1 -- 172.21.15.38:0/1089086775 learned_addr learned my addr 172.21.15.38:0/1089086775 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.967+0000 7fc24d098640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2480d0320 0x7fc2480d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3628507257 0 0) 0x7fc2480d2060 con 0x7fc2480d0700 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc234003520 con 0x7fc2480d0700 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860347087 0 0) 0x7fc2480d2330 con 0x7fc2480b6980 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2480d2060 con 0x7fc2480b6980 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2931165272 0 0) 0x7fc234003520 con 0x7fc2480d0700 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2480d0320 msgr2=0x7fc2480d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2480d0320 0x7fc2480d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 >> v1:172.21.15.107:6789/0 conn(0x7fc2480b6980 legacy=0x7fc2480b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2480d28f0 con 0x7fc2480d0700 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fc238001670 con 0x7fc2480d0700 2024-04-25T08:42:29.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 >> v1:172.21.15.38:6789/0 conn(0x7fc2480d0700 legacy=0x7fc2480d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc234003b70 0x7fc234003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2480d0320 0x7fc234004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc234004d40 0x7fc2340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480d2610 con 0x7fc234003b70 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23400ba70 con 0x7fc234004d40 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23400bc20 con 0x7fc2480d0320 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc24d098640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc234003b70 0x7fc234003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc24c897640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2480d0320 0x7fc234004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc24d899640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc234004d40 0x7fc2340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc24d098640 1 -- 172.21.15.38:0/1089086775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2480d0320 msgr2=0x7fc234004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc24d098640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2480d0320 0x7fc234004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc24d098640 1 -- 172.21.15.38:0/1089086775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc234004d40 msgr2=0x7fc2340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc24d098640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc234004d40 0x7fc2340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc24d098640 1 -- 172.21.15.38:0/1089086775 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc234003520 con 0x7fc234003b70 2024-04-25T08:42:29.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.968+0000 7fc24d899640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc234004d40 0x7fc2340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:29.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.969+0000 7fc24d098640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc234003b70 0x7fc234003f70 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fc244009fc0 tx=0x7fc234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f34d85f90011402 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:29.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.969+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244009c10 con 0x7fc234003b70 2024-04-25T08:42:29.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.969+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc244036070 con 0x7fc234003b70 2024-04-25T08:42:29.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.969+0000 7fc23ffff640 1 -- 172.21.15.38:0/1089086775 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244031880 con 0x7fc234003b70 2024-04-25T08:42:29.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.970+0000 7fc24f2fc640 1 -- 172.21.15.38:0/1089086775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc234003b70 msgr2=0x7fc234003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.970+0000 7fc24f2fc640 1 --2- 172.21.15.38:0/1089086775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc234003b70 0x7fc234003f70 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fc244009fc0 tx=0x7fc234005830 comp rx=0 tx=0).stop 2024-04-25T08:42:29.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.970+0000 7fc24d899640 1 -- 172.21.15.38:0/1089086775 reap_dead start 2024-04-25T08:42:29.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.970+0000 7fc24f2fc640 1 -- 172.21.15.38:0/1089086775 shutdown_connections 2024-04-25T08:42:29.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.970+0000 7fc24f2fc640 1 -- 172.21.15.38:0/1089086775 >> 172.21.15.38:0/1089086775 conn(0x7fc2480668f0 msgr2=0x7fc248066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:29.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.970+0000 7fc24f2fc640 1 -- 172.21.15.38:0/1089086775 shutdown_connections 2024-04-25T08:42:29.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.970+0000 7fc24f2fc640 1 -- 172.21.15.38:0/1089086775 wait complete. 2024-04-25T08:42:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.971+0000 7fc24f2fc640 1 Processor -- start 2024-04-25T08:42:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.971+0000 7fc24f2fc640 1 -- start start 2024-04-25T08:42:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24f2fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc234003b70 0x7fc248159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24f2fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2480cb4f0 0x7fc2480cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24f2fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2481e6610 0x7fc2481e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24f2fc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480d0b40 con 0x7fc2480cb4f0 2024-04-25T08:42:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24f2fc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480d2eb0 con 0x7fc234003b70 2024-04-25T08:42:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24f2fc640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480cbe90 con 0x7fc2481e6610 2024-04-25T08:42:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24c897640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2480cb4f0 0x7fc2480cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24c897640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2480cb4f0 0x7fc2480cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40358/0 (socket says 172.21.15.38:40358) 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24c897640 1 -- 172.21.15.38:0/2026688024 learned_addr learned my addr 172.21.15.38:0/2026688024 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24d899640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2481e6610 0x7fc2481e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24d098640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc234003b70 0x7fc248159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24c897640 1 -- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2481e6610 msgr2=0x7fc2481e6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24c897640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2481e6610 0x7fc2481e6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24c897640 1 -- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc234003b70 msgr2=0x7fc248159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24c897640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc234003b70 0x7fc248159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24c897640 1 -- 172.21.15.38:0/2026688024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2481f3c30 con 0x7fc2480cb4f0 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24c897640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2480cb4f0 0x7fc2480cb950 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fc2380072c0 tx=0x7fc238007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc23dffb640 1 -- 172.21.15.38:0/2026688024 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23801a070 con 0x7fc2480cb4f0 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc23dffb640 1 -- 172.21.15.38:0/2026688024 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23800f9c0 con 0x7fc2480cb4f0 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc24f2fc640 1 -- 172.21.15.38:0/2026688024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24815b380 con 0x7fc2480cb4f0 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.972+0000 7fc23dffb640 1 -- 172.21.15.38:0/2026688024 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23800fdb0 con 0x7fc2480cb4f0 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.973+0000 7fc24f2fc640 1 -- 172.21.15.38:0/2026688024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24815b940 con 0x7fc2480cb4f0 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.974+0000 7fc24f2fc640 1 -- 172.21.15.38:0/2026688024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2481f3e00 con 0x7fc2480cb4f0 2024-04-25T08:42:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.974+0000 7fc23dffb640 1 -- 172.21.15.38:0/2026688024 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc23801d070 con 0x7fc2480cb4f0 2024-04-25T08:42:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.974+0000 7fc23dffb640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc22c06b4d0 0x7fc22c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.974+0000 7fc23dffb640 1 -- 172.21.15.38:0/2026688024 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fc238089ea0 con 0x7fc2480cb4f0 2024-04-25T08:42:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.974+0000 7fc24d098640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc22c06b4d0 0x7fc22c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.975+0000 7fc24d098640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc22c06b4d0 0x7fc22c06d990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc244035000 tx=0x7fc24402f000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:29.978+0000 7fc23dffb640 1 -- 172.21.15.38:0/2026688024 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2481f3e00 con 0x7fc2480cb4f0 2024-04-25T08:42:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.160+0000 7fc24f2fc640 1 -- 172.21.15.38:0/2026688024 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc248068e80 con 0x7fc2480cb4f0 2024-04-25T08:42:30.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.160+0000 7fc23dffb640 1 -- 172.21.15.38:0/2026688024 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc238058a60 con 0x7fc2480cb4f0 2024-04-25T08:42:30.153 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-04-25T08:42:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.162+0000 7fc24f2fc640 1 -- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc22c06b4d0 msgr2=0x7fc22c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.162+0000 7fc24f2fc640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc22c06b4d0 0x7fc22c06d990 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc244035000 tx=0x7fc24402f000 comp rx=0 tx=0).stop 2024-04-25T08:42:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.162+0000 7fc24f2fc640 1 -- 172.21.15.38:0/2026688024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2480cb4f0 msgr2=0x7fc2480cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.162+0000 7fc24f2fc640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2480cb4f0 0x7fc2480cb950 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fc2380072c0 tx=0x7fc238007780 comp rx=0 tx=0).stop 2024-04-25T08:42:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.162+0000 7fc24f2fc640 1 -- 172.21.15.38:0/2026688024 shutdown_connections 2024-04-25T08:42:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.162+0000 7fc24f2fc640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc22c06b4d0 0x7fc22c06d990 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.162+0000 7fc24f2fc640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc2481e6610 0x7fc2481e6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.162+0000 7fc24f2fc640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc2480cb4f0 0x7fc2480cb950 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.162+0000 7fc24f2fc640 1 --2- 172.21.15.38:0/2026688024 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc234003b70 0x7fc248159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.162+0000 7fc24f2fc640 1 -- 172.21.15.38:0/2026688024 >> 172.21.15.38:0/2026688024 conn(0x7fc2480668f0 msgr2=0x7fc24815fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.162+0000 7fc24f2fc640 1 -- 172.21.15.38:0/2026688024 shutdown_connections 2024-04-25T08:42:30.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.162+0000 7fc24f2fc640 1 -- 172.21.15.38:0/2026688024 wait complete. 2024-04-25T08:42:30.165 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:42:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.277+0000 7f7a74f70640 1 Processor -- start 2024-04-25T08:42:30.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.277+0000 7f7a74f70640 1 -- start start 2024-04-25T08:42:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a74f70640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a700d0320 0x7f7a700d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a74f70640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7a700d0700 0x7f7a700d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a74f70640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7a700d1e90 0x7f7a700b5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a74f70640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a700d2330 con 0x7f7a700d0700 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a74f70640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a700d2610 con 0x7f7a700d1e90 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a74f70640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700d28f0 con 0x7f7a700d0320 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a6dd9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7a700d0700 0x7f7a700d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a6dd9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7a700d0700 0x7f7a700d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40368/0 (socket says 172.21.15.38:40368) 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a6dd9b640 1 -- 172.21.15.38:0/1095534878 learned_addr learned my addr 172.21.15.38:0/1095534878 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a6ed9d640 1 --2- 172.21.15.38:0/1095534878 >> v2:172.21.15.107:3300/0 conn(0x7f7a700d1e90 0x7f7a700b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a6e59c640 1 --2- 172.21.15.38:0/1095534878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a700d0320 0x7f7a700d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a6dd9b640 1 -- 172.21.15.38:0/1095534878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a700d0320 msgr2=0x7f7a700d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a6dd9b640 1 --2- 172.21.15.38:0/1095534878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a700d0320 0x7f7a700d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a6dd9b640 1 -- 172.21.15.38:0/1095534878 >> v2:172.21.15.107:3300/0 conn(0x7f7a700d1e90 msgr2=0x7f7a700b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a6dd9b640 1 --2- 172.21.15.38:0/1095534878 >> v2:172.21.15.107:3300/0 conn(0x7f7a700d1e90 0x7f7a700b5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.278+0000 7f7a6dd9b640 1 -- 172.21.15.38:0/1095534878 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a700d2bd0 con 0x7f7a700d0700 2024-04-25T08:42:30.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.279+0000 7f7a6ed9d640 1 --2- 172.21.15.38:0/1095534878 >> v2:172.21.15.107:3300/0 conn(0x7f7a700d1e90 0x7f7a700b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:30.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.279+0000 7f7a6dd9b640 1 --2- 172.21.15.38:0/1095534878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a700d0700 0x7f7a700d1950 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f7a700b68d0 tx=0x7f7a6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5abb95b63813515 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:30.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.279+0000 7f7a6d59a640 1 -- 172.21.15.38:0/1095534878 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64037070 con 0x7f7a700d0700 2024-04-25T08:42:30.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.279+0000 7f7a6d59a640 1 -- 172.21.15.38:0/1095534878 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a64032440 con 0x7f7a700d0700 2024-04-25T08:42:30.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.279+0000 7f7a6d59a640 1 -- 172.21.15.38:0/1095534878 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a64032810 con 0x7f7a700d0700 2024-04-25T08:42:30.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.280+0000 7f7a74f70640 1 -- 172.21.15.38:0/1095534878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a700d0700 msgr2=0x7f7a700d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.280+0000 7f7a74f70640 1 --2- 172.21.15.38:0/1095534878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a700d0700 0x7f7a700d1950 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f7a700b68d0 tx=0x7f7a6402a270 comp rx=0 tx=0).stop 2024-04-25T08:42:30.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.280+0000 7f7a74f70640 1 -- 172.21.15.38:0/1095534878 shutdown_connections 2024-04-25T08:42:30.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.280+0000 7f7a74f70640 1 --2- 172.21.15.38:0/1095534878 >> v2:172.21.15.107:3300/0 conn(0x7f7a700d1e90 0x7f7a700b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.280+0000 7f7a74f70640 1 --2- 172.21.15.38:0/1095534878 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a700d0700 0x7f7a700d1950 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.280+0000 7f7a74f70640 1 --2- 172.21.15.38:0/1095534878 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a700d0320 0x7f7a700d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.280+0000 7f7a74f70640 1 -- 172.21.15.38:0/1095534878 >> 172.21.15.38:0/1095534878 conn(0x7f7a700668f0 msgr2=0x7f7a70066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:30.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.280+0000 7f7a74f70640 1 -- 172.21.15.38:0/1095534878 shutdown_connections 2024-04-25T08:42:30.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.280+0000 7f7a74f70640 1 -- 172.21.15.38:0/1095534878 wait complete. 2024-04-25T08:42:30.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.281+0000 7f7a74f70640 1 Processor -- start 2024-04-25T08:42:30.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.281+0000 7f7a74f70640 1 -- start start 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.282+0000 7f7a74f70640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a700d0320 0x7f7a70151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.282+0000 7f7a74f70640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a700d0700 0x7f7a700b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.282+0000 7f7a74f70640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a700d1e90 0x7f7a700b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.282+0000 7f7a74f70640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700d0b40 con 0x7f7a700d1e90 2024-04-25T08:42:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.282+0000 7f7a74f70640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a70161210 con 0x7f7a700d0700 2024-04-25T08:42:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.282+0000 7f7a74f70640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a701521a0 con 0x7f7a700d0320 2024-04-25T08:42:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.282+0000 7f7a6ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a700d1e90 0x7f7a700b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.282+0000 7f7a6ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a700d1e90 0x7f7a700b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40380/0 (socket says 172.21.15.38:40380) 2024-04-25T08:42:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.282+0000 7f7a6ed9d640 1 -- 172.21.15.38:0/4103912555 learned_addr learned my addr 172.21.15.38:0/4103912555 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.282+0000 7f7a6dd9b640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a700d0700 0x7f7a700b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.282+0000 7f7a6e59c640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a700d0320 0x7f7a70151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.283+0000 7f7a6ed9d640 1 -- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a700d0320 msgr2=0x7f7a70151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.283+0000 7f7a6ed9d640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a700d0320 0x7f7a70151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.283+0000 7f7a6ed9d640 1 -- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a700d0700 msgr2=0x7f7a700b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.283+0000 7f7a6ed9d640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a700d0700 0x7f7a700b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.283+0000 7f7a6ed9d640 1 -- 172.21.15.38:0/4103912555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a700b3cf0 con 0x7f7a700d1e90 2024-04-25T08:42:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.283+0000 7f7a6e59c640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a700d0320 0x7f7a70151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.283+0000 7f7a6ed9d640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a700d1e90 0x7f7a700b5560 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f7a68007e30 tx=0x7f7a6800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.283+0000 7f7a5effd640 1 -- 172.21.15.38:0/4103912555 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6801a070 con 0x7f7a700d1e90 2024-04-25T08:42:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.283+0000 7f7a5effd640 1 -- 172.21.15.38:0/4103912555 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a68015400 con 0x7f7a700d1e90 2024-04-25T08:42:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.283+0000 7f7a74f70640 1 -- 172.21.15.38:0/4103912555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a700b3fd0 con 0x7f7a700d1e90 2024-04-25T08:42:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.283+0000 7f7a5effd640 1 -- 172.21.15.38:0/4103912555 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a680157d0 con 0x7f7a700d1e90 2024-04-25T08:42:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.283+0000 7f7a74f70640 1 -- 172.21.15.38:0/4103912555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a700b4590 con 0x7f7a700d1e90 2024-04-25T08:42:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.284+0000 7f7a74f70640 1 -- 172.21.15.38:0/4103912555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a70068e80 con 0x7f7a700d1e90 2024-04-25T08:42:30.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.284+0000 7f7a5effd640 1 -- 172.21.15.38:0/4103912555 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a6801d070 con 0x7f7a700d1e90 2024-04-25T08:42:30.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.285+0000 7f7a5effd640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7a5006b4d0 0x7f7a5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.285+0000 7f7a5effd640 1 -- 172.21.15.38:0/4103912555 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f7a6808a5f0 con 0x7f7a700d1e90 2024-04-25T08:42:30.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.285+0000 7f7a6e59c640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7a5006b4d0 0x7f7a5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.286+0000 7f7a6e59c640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7a5006b4d0 0x7f7a5006d990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7a60006fd0 tx=0x7f7a60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:30.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.289+0000 7f7a5effd640 1 -- 172.21.15.38:0/4103912555 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a68010070 con 0x7f7a700d1e90 2024-04-25T08:42:30.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.468+0000 7f7a74f70640 1 -- 172.21.15.38:0/4103912555 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a700b3050 con 0x7f7a700d1e90 2024-04-25T08:42:30.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.469+0000 7f7a5effd640 1 -- 172.21.15.38:0/4103912555 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a68058c00 con 0x7f7a700d1e90 2024-04-25T08:42:30.461 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-04-25T08:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.470+0000 7f7a74f70640 1 -- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7a5006b4d0 msgr2=0x7f7a5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.470+0000 7f7a74f70640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7a5006b4d0 0x7f7a5006d990 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7a60006fd0 tx=0x7f7a60008040 comp rx=0 tx=0).stop 2024-04-25T08:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.470+0000 7f7a74f70640 1 -- 172.21.15.38:0/4103912555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a700d1e90 msgr2=0x7f7a700b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.470+0000 7f7a74f70640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a700d1e90 0x7f7a700b5560 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f7a68007e30 tx=0x7f7a6800d4a0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.470+0000 7f7a74f70640 1 -- 172.21.15.38:0/4103912555 shutdown_connections 2024-04-25T08:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.470+0000 7f7a74f70640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7a700d1e90 0x7f7a700b5560 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.470+0000 7f7a74f70640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a700d0700 0x7f7a700b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.470+0000 7f7a74f70640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7a5006b4d0 0x7f7a5006d990 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.470+0000 7f7a74f70640 1 --2- 172.21.15.38:0/4103912555 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7a700d0320 0x7f7a70151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.470+0000 7f7a74f70640 1 -- 172.21.15.38:0/4103912555 >> 172.21.15.38:0/4103912555 conn(0x7f7a700668f0 msgr2=0x7f7a700b6510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:30.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.471+0000 7f7a74f70640 1 -- 172.21.15.38:0/4103912555 shutdown_connections 2024-04-25T08:42:30.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.471+0000 7f7a74f70640 1 -- 172.21.15.38:0/4103912555 wait complete. 2024-04-25T08:42:30.474 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.583+0000 7fad1a15c640 1 Processor -- start 2024-04-25T08:42:30.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.583+0000 7fad1a15c640 1 -- start start 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad1a15c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad14147700 0x7fad140b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad1a15c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fad1406d520 0x7fad140b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad1a15c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fad140d04f0 0x7fad140714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad1a15c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1406e720 con 0x7fad140d04f0 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad1a15c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1406ea00 con 0x7fad1406d520 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad1a15c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1406ece0 con 0x7fad14147700 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad13fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fad140d04f0 0x7fad140714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad13fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fad140d04f0 0x7fad140714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40382/0 (socket says 172.21.15.38:40382) 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad13fff640 1 -- 172.21.15.38:0/1556059654 learned_addr learned my addr 172.21.15.38:0/1556059654 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad137fe640 1 --2- 172.21.15.38:0/1556059654 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad14147700 0x7fad140b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad12ffd640 1 --2- 172.21.15.38:0/1556059654 >> v2:172.21.15.107:3300/0 conn(0x7fad1406d520 0x7fad140b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad13fff640 1 -- 172.21.15.38:0/1556059654 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad14147700 msgr2=0x7fad140b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:30.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad13fff640 1 --2- 172.21.15.38:0/1556059654 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad14147700 0x7fad140b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad13fff640 1 -- 172.21.15.38:0/1556059654 >> v2:172.21.15.107:3300/0 conn(0x7fad1406d520 msgr2=0x7fad140b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:30.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad13fff640 1 --2- 172.21.15.38:0/1556059654 >> v2:172.21.15.107:3300/0 conn(0x7fad1406d520 0x7fad140b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.584+0000 7fad13fff640 1 -- 172.21.15.38:0/1556059654 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1406efc0 con 0x7fad140d04f0 2024-04-25T08:42:30.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.585+0000 7fad13fff640 1 --2- 172.21.15.38:0/1556059654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad140d04f0 0x7fad140714c0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fad1406d470 tx=0x7fad0c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf16b5bfdb96a7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:30.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.585+0000 7fad127fc640 1 -- 172.21.15.38:0/1556059654 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c036070 con 0x7fad140d04f0 2024-04-25T08:42:30.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.585+0000 7fad127fc640 1 -- 172.21.15.38:0/1556059654 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c031600 con 0x7fad140d04f0 2024-04-25T08:42:30.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.585+0000 7fad127fc640 1 -- 172.21.15.38:0/1556059654 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0319d0 con 0x7fad140d04f0 2024-04-25T08:42:30.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.586+0000 7fad1a15c640 1 -- 172.21.15.38:0/1556059654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad140d04f0 msgr2=0x7fad140714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.586+0000 7fad1a15c640 1 --2- 172.21.15.38:0/1556059654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad140d04f0 0x7fad140714c0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fad1406d470 tx=0x7fad0c02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.586+0000 7fad1a15c640 1 -- 172.21.15.38:0/1556059654 shutdown_connections 2024-04-25T08:42:30.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.586+0000 7fad1a15c640 1 --2- 172.21.15.38:0/1556059654 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad140d04f0 0x7fad140714c0 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.586+0000 7fad1a15c640 1 --2- 172.21.15.38:0/1556059654 >> v2:172.21.15.107:3300/0 conn(0x7fad1406d520 0x7fad140b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.586+0000 7fad1a15c640 1 --2- 172.21.15.38:0/1556059654 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad14147700 0x7fad140b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.586+0000 7fad1a15c640 1 -- 172.21.15.38:0/1556059654 >> 172.21.15.38:0/1556059654 conn(0x7fad140cf970 msgr2=0x7fad140cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:30.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.586+0000 7fad1a15c640 1 -- 172.21.15.38:0/1556059654 shutdown_connections 2024-04-25T08:42:30.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.586+0000 7fad1a15c640 1 -- 172.21.15.38:0/1556059654 wait complete. 2024-04-25T08:42:30.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.587+0000 7fad1a15c640 1 Processor -- start 2024-04-25T08:42:30.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.587+0000 7fad1a15c640 1 -- start start 2024-04-25T08:42:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad1a15c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad1406d520 0x7fad140a95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad1a15c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad140d04f0 0x7fad140a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad1a15c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad14147700 0x7fad140ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad1a15c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad14071a00 con 0x7fad140d04f0 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad1a15c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1406e410 con 0x7fad1406d520 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad1a15c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad14071b80 con 0x7fad14147700 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad12ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad140d04f0 0x7fad140a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad12ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad140d04f0 0x7fad140a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40384/0 (socket says 172.21.15.38:40384) 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad12ffd640 1 -- 172.21.15.38:0/1961753101 learned_addr learned my addr 172.21.15.38:0/1961753101 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad13fff640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad14147700 0x7fad140ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad137fe640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad1406d520 0x7fad140a95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad12ffd640 1 -- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad14147700 msgr2=0x7fad140ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad12ffd640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad14147700 0x7fad140ce1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad12ffd640 1 -- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad1406d520 msgr2=0x7fad140a95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad12ffd640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad1406d520 0x7fad140a95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad12ffd640 1 -- 172.21.15.38:0/1961753101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad140aa560 con 0x7fad140d04f0 2024-04-25T08:42:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad13fff640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad14147700 0x7fad140ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad137fe640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad1406d520 0x7fad140a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad12ffd640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad140d04f0 0x7fad140a9df0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fad0800e9b0 tx=0x7fad0800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7facfbfff640 1 -- 172.21.15.38:0/1961753101 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08017070 con 0x7fad140d04f0 2024-04-25T08:42:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7facfbfff640 1 -- 172.21.15.38:0/1961753101 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0800c9a0 con 0x7fad140d04f0 2024-04-25T08:42:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7fad1a15c640 1 -- 172.21.15.38:0/1961753101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad140aa840 con 0x7fad140d04f0 2024-04-25T08:42:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.588+0000 7facfbfff640 1 -- 172.21.15.38:0/1961753101 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0800cd90 con 0x7fad140d04f0 2024-04-25T08:42:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.589+0000 7fad1a15c640 1 -- 172.21.15.38:0/1961753101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad140aae00 con 0x7fad140d04f0 2024-04-25T08:42:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.590+0000 7fad1a15c640 1 -- 172.21.15.38:0/1961753101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad14068e80 con 0x7fad140d04f0 2024-04-25T08:42:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.590+0000 7facfbfff640 1 -- 172.21.15.38:0/1961753101 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad0801a070 con 0x7fad140d04f0 2024-04-25T08:42:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.590+0000 7facfbfff640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad0006b4d0 0x7fad0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.590+0000 7facfbfff640 1 -- 172.21.15.38:0/1961753101 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fad0808a380 con 0x7fad140d04f0 2024-04-25T08:42:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.591+0000 7fad137fe640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad0006b4d0 0x7fad0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.591+0000 7fad137fe640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad0006b4d0 0x7fad0006d990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fad04009a70 tx=0x7fad04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:30.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.595+0000 7facfbfff640 1 -- 172.21.15.38:0/1961753101 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad08059070 con 0x7fad140d04f0 2024-04-25T08:42:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.813+0000 7fad1a15c640 1 -- 172.21.15.38:0/1961753101 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad140a9bd0 con 0x7fad140d04f0 2024-04-25T08:42:30.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.814+0000 7facfbfff640 1 -- 172.21.15.38:0/1961753101 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4960 (secure 0 0 0) 0x7fad08058970 con 0x7fad140d04f0 2024-04-25T08:42:30.806 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:30.806 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2682394530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2682394530},{"type":"v1","addr":"172.21.15.107:6837","nonce":2682394530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2810760276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2810760276},{"type":"v1","addr":"172.21.15.107:6835","nonce":2810760276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T08:42:04.044929+0000","modified":"2024-04-25T08:42:25.774447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10038":{"gid":10038,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3318593502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3318593502},{"type":"v1","addr":"172.21.15.38:6833","nonce":3318593502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10041":{"gid":10041,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/3513331661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3513331661},{"type":"v1","addr":"172.21.15.38:6835","nonce":3513331661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:42:30.807 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 91 2024-04-25T08:42:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.816+0000 7fad1a15c640 1 -- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad0006b4d0 msgr2=0x7fad0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.816+0000 7fad1a15c640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad0006b4d0 0x7fad0006d990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fad04009a70 tx=0x7fad04008040 comp rx=0 tx=0).stop 2024-04-25T08:42:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.816+0000 7fad1a15c640 1 -- 172.21.15.38:0/1961753101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad140d04f0 msgr2=0x7fad140a9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.816+0000 7fad1a15c640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad140d04f0 0x7fad140a9df0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fad0800e9b0 tx=0x7fad0800ee70 comp rx=0 tx=0).stop 2024-04-25T08:42:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.816+0000 7fad1a15c640 1 -- 172.21.15.38:0/1961753101 shutdown_connections 2024-04-25T08:42:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.816+0000 7fad1a15c640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad0006b4d0 0x7fad0006d990 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.816+0000 7fad1a15c640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad14147700 0x7fad140ce1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.816+0000 7fad1a15c640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad140d04f0 0x7fad140a9df0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.816+0000 7fad1a15c640 1 --2- 172.21.15.38:0/1961753101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad1406d520 0x7fad140a95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.816+0000 7fad1a15c640 1 -- 172.21.15.38:0/1961753101 >> 172.21.15.38:0/1961753101 conn(0x7fad140cf970 msgr2=0x7fad14144fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.816+0000 7fad1a15c640 1 -- 172.21.15.38:0/1961753101 shutdown_connections 2024-04-25T08:42:30.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.816+0000 7fad1a15c640 1 -- 172.21.15.38:0/1961753101 wait complete. 2024-04-25T08:42:30.820 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:30.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.928+0000 7f2b4fbd5640 1 Processor -- start 2024-04-25T08:42:30.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.929+0000 7f2b4fbd5640 1 -- start start 2024-04-25T08:42:30.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.929+0000 7f2b4fbd5640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2b4815bb10 0x7f2b4809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.929+0000 7f2b4fbd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b4815fbf0 0x7f2b4809a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.929+0000 7f2b4fbd5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b48160340 con 0x7f2b480d0460 2024-04-25T08:42:30.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.929+0000 7f2b4fbd5640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b480cffd0 con 0x7f2b4815bb10 2024-04-25T08:42:30.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.929+0000 7f2b4fbd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b480cf400 con 0x7f2b4815fbf0 2024-04-25T08:42:30.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.930+0000 7f2b4d971640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f2b4815bb10 0x7f2b4809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.930+0000 7f2b4d170640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2b480d0460 0x7f2b4815f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40480/0 (socket says 172.21.15.38:40480) 2024-04-25T08:42:30.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.930+0000 7f2b4d170640 1 -- 172.21.15.38:0/549288965 learned_addr learned my addr 172.21.15.38:0/549288965 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:30.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.930+0000 7f2b4e172640 1 --2- 172.21.15.38:0/549288965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b4815fbf0 0x7f2b4809a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.930+0000 7f2b4d971640 1 -- 172.21.15.38:0/549288965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b4815fbf0 msgr2=0x7f2b4809a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:30.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.930+0000 7f2b4d971640 1 --2- 172.21.15.38:0/549288965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b4815fbf0 0x7f2b4809a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.930+0000 7f2b4d971640 1 -- 172.21.15.38:0/549288965 >> v1:172.21.15.38:6789/0 conn(0x7f2b480d0460 legacy=0x7f2b4815f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.930+0000 7f2b4c96f640 1 -- 172.21.15.38:0/549288965 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071623890 0 0) 0x7f2b38001110 con 0x7f2b480d0460 2024-04-25T08:42:30.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.930+0000 7f2b4d971640 1 -- 172.21.15.38:0/549288965 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b480cf6a0 con 0x7f2b4815bb10 2024-04-25T08:42:30.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.930+0000 7f2b4e172640 1 --2- 172.21.15.38:0/549288965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b4815fbf0 0x7f2b4809a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:30.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.931+0000 7f2b4d971640 1 --2- 172.21.15.38:0/549288965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b4815bb10 0x7f2b4809a480 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f2b440097a0 tx=0x7f2b4402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cdfa8ab6c2624 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.931+0000 7f2b4c96f640 1 -- 172.21.15.38:0/549288965 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44037070 con 0x7f2b4815bb10 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.931+0000 7f2b4c96f640 1 -- 172.21.15.38:0/549288965 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4402ae50 con 0x7f2b4815bb10 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.931+0000 7f2b4c96f640 1 -- 172.21.15.38:0/549288965 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44032660 con 0x7f2b4815bb10 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.931+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/549288965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b4815bb10 msgr2=0x7f2b4809a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.931+0000 7f2b4fbd5640 1 --2- 172.21.15.38:0/549288965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b4815bb10 0x7f2b4809a480 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f2b440097a0 tx=0x7f2b4402a100 comp rx=0 tx=0).stop 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.931+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/549288965 shutdown_connections 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.931+0000 7f2b4fbd5640 1 --2- 172.21.15.38:0/549288965 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b4815fbf0 0x7f2b4809a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.931+0000 7f2b4fbd5640 1 --2- 172.21.15.38:0/549288965 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b4815bb10 0x7f2b4809a480 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.931+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/549288965 >> 172.21.15.38:0/549288965 conn(0x7f2b480b31f0 msgr2=0x7f2b480b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.931+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/549288965 shutdown_connections 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.931+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/549288965 wait complete. 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.933+0000 7f2b4fbd5640 1 Processor -- start 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.933+0000 7f2b4fbd5640 1 -- start start 2024-04-25T08:42:30.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.933+0000 7f2b4fbd5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b480d0460 0x7f2b4814eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.933+0000 7f2b4fbd5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b4815bb10 0x7f2b48158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.933+0000 7f2b4fbd5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b4815fbf0 0x7f2b48158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.933+0000 7f2b4fbd5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b480c5250 con 0x7f2b480d0460 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.933+0000 7f2b4fbd5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b480cfc60 con 0x7f2b4815bb10 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.933+0000 7f2b4fbd5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b480c53d0 con 0x7f2b4815fbf0 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.933+0000 7f2b4d971640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b480d0460 0x7f2b4814eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.933+0000 7f2b4d971640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b480d0460 0x7f2b4814eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40396/0 (socket says 172.21.15.38:40396) 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.933+0000 7f2b4d971640 1 -- 172.21.15.38:0/1762972643 learned_addr learned my addr 172.21.15.38:0/1762972643 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b4e172640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b4815fbf0 0x7f2b48158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b4d170640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b4815bb10 0x7f2b48158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b4d971640 1 -- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b4815fbf0 msgr2=0x7f2b48158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b4d971640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b4815fbf0 0x7f2b48158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b4d971640 1 -- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b4815bb10 msgr2=0x7f2b48158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:30.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b4d971640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b4815bb10 0x7f2b48158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:30.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b4d971640 1 -- 172.21.15.38:0/1762972643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b480b4f30 con 0x7f2b480d0460 2024-04-25T08:42:30.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b4e172640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b4815fbf0 0x7f2b48158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:30.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b4d971640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b480d0460 0x7f2b4814eae0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f2b44000c00 tx=0x7f2b440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:30.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b3e7fc640 1 -- 172.21.15.38:0/1762972643 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44041070 con 0x7f2b480d0460 2024-04-25T08:42:30.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b3e7fc640 1 -- 172.21.15.38:0/1762972643 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4403c670 con 0x7f2b480d0460 2024-04-25T08:42:30.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/1762972643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b480b5210 con 0x7f2b480d0460 2024-04-25T08:42:30.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b3e7fc640 1 -- 172.21.15.38:0/1762972643 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44037040 con 0x7f2b480d0460 2024-04-25T08:42:30.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.934+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/1762972643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b480b57d0 con 0x7f2b480d0460 2024-04-25T08:42:30.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.935+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/1762972643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b48068e80 con 0x7f2b480d0460 2024-04-25T08:42:30.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.936+0000 7f2b3e7fc640 1 -- 172.21.15.38:0/1762972643 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b44044070 con 0x7f2b480d0460 2024-04-25T08:42:30.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.936+0000 7f2b3e7fc640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2b2c06b4d0 0x7f2b2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:30.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.936+0000 7f2b3e7fc640 1 -- 172.21.15.38:0/1762972643 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f2b440aac40 con 0x7f2b480d0460 2024-04-25T08:42:30.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.936+0000 7f2b4d170640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2b2c06b4d0 0x7f2b2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:30.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.937+0000 7f2b4d170640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2b2c06b4d0 0x7f2b2c06d990 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2b481f6cc0 tx=0x7f2b38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:30.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:30.940+0000 7f2b3e7fc640 1 -- 172.21.15.38:0/1762972643 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b440799c0 con 0x7f2b480d0460 2024-04-25T08:42:31.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.121+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/1762972643 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b480cdbf0 con 0x7f2b480d0460 2024-04-25T08:42:31.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.122+0000 7f2b3e7fc640 1 -- 172.21.15.38:0/1762972643 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+17179 (secure 0 0 0) 0x7f2b440792c0 con 0x7f2b480d0460 2024-04-25T08:42:31.115 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:31.115 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":145,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:42:19.745001+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:01.691830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:03.024111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.8","mappings":[{"from":4,"to":1}]},{"pgid":"22.c","mappings":[{"from":3,"to":1}]},{"pgid":"22.e","mappings":[{"from":3,"to":0}]},{"pgid":"22.21","mappings":[{"from":3,"to":0}]},{"pgid":"22.23","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:31.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.124+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2b2c06b4d0 msgr2=0x7f2b2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.124+0000 7f2b4fbd5640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2b2c06b4d0 0x7f2b2c06d990 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2b481f6cc0 tx=0x7f2b38009040 comp rx=0 tx=0).stop 2024-04-25T08:42:31.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.124+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/1762972643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b480d0460 msgr2=0x7f2b4814eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.124+0000 7f2b4fbd5640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b480d0460 0x7f2b4814eae0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7f2b44000c00 tx=0x7f2b440046c0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.124+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/1762972643 shutdown_connections 2024-04-25T08:42:31.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.124+0000 7f2b4fbd5640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2b2c06b4d0 0x7f2b2c06d990 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.124+0000 7f2b4fbd5640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b4815fbf0 0x7f2b48158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.124+0000 7f2b4fbd5640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b4815bb10 0x7f2b48158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.124+0000 7f2b4fbd5640 1 --2- 172.21.15.38:0/1762972643 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2b480d0460 0x7f2b4814eae0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.124+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/1762972643 >> 172.21.15.38:0/1762972643 conn(0x7f2b480b31f0 msgr2=0x7f2b480c5890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:31.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.125+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/1762972643 shutdown_connections 2024-04-25T08:42:31.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.125+0000 7f2b4fbd5640 1 -- 172.21.15.38:0/1762972643 wait complete. 2024-04-25T08:42:31.128 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:42:31.128 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:31.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.237+0000 7f7c2cb5c640 1 Processor -- start 2024-04-25T08:42:31.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.237+0000 7f7c2cb5c640 1 -- start start 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c2cb5c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f7c2806cc10 0x7f7c280cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c2cb5c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7c2806cff0 0x7f7c280cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c2cb5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c28099a70 0x7f7c28099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c2cb5c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2809a3e0 con 0x7f7c2806cff0 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c2cb5c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c280d05c0 con 0x7f7c2806cc10 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c2cb5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280d08a0 con 0x7f7c28099a70 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c1ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7c2806cff0 0x7f7c280cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c1ffff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f7c2806cff0 0x7f7c280cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40400/0 (socket says 172.21.15.38:40400) 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c1ffff640 1 -- 172.21.15.38:0/2109705974 learned_addr learned my addr 172.21.15.38:0/2109705974 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c26d9d640 1 --2- 172.21.15.38:0/2109705974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c28099a70 0x7f7c28099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c2659c640 1 --2- 172.21.15.38:0/2109705974 >> v2:172.21.15.107:3300/0 conn(0x7f7c2806cc10 0x7f7c280cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c1ffff640 1 -- 172.21.15.38:0/2109705974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c28099a70 msgr2=0x7f7c28099ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c1ffff640 1 --2- 172.21.15.38:0/2109705974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c28099a70 0x7f7c28099ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c1ffff640 1 -- 172.21.15.38:0/2109705974 >> v2:172.21.15.107:3300/0 conn(0x7f7c2806cc10 msgr2=0x7f7c280cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:31.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c1ffff640 1 --2- 172.21.15.38:0/2109705974 >> v2:172.21.15.107:3300/0 conn(0x7f7c2806cc10 0x7f7c280cfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c1ffff640 1 -- 172.21.15.38:0/2109705974 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c280d0b80 con 0x7f7c2806cff0 2024-04-25T08:42:31.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c26d9d640 1 --2- 172.21.15.38:0/2109705974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c28099a70 0x7f7c28099ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:31.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.238+0000 7f7c2659c640 1 --2- 172.21.15.38:0/2109705974 >> v2:172.21.15.107:3300/0 conn(0x7f7c2806cc10 0x7f7c280cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:31.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.239+0000 7f7c1ffff640 1 --2- 172.21.15.38:0/2109705974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c2806cff0 0x7f7c280cffb0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f7c28067970 tx=0x7f7c1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197ddbb845d148a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:31.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.239+0000 7f7c25d9b640 1 -- 172.21.15.38:0/2109705974 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10037070 con 0x7f7c2806cff0 2024-04-25T08:42:31.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.239+0000 7f7c25d9b640 1 -- 172.21.15.38:0/2109705974 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c10032440 con 0x7f7c2806cff0 2024-04-25T08:42:31.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.239+0000 7f7c25d9b640 1 -- 172.21.15.38:0/2109705974 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10032810 con 0x7f7c2806cff0 2024-04-25T08:42:31.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.240+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/2109705974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c2806cff0 msgr2=0x7f7c280cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.240+0000 7f7c2cb5c640 1 --2- 172.21.15.38:0/2109705974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c2806cff0 0x7f7c280cffb0 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7f7c28067970 tx=0x7f7c1002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.240+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/2109705974 shutdown_connections 2024-04-25T08:42:31.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.240+0000 7f7c2cb5c640 1 --2- 172.21.15.38:0/2109705974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c28099a70 0x7f7c28099ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.240+0000 7f7c2cb5c640 1 --2- 172.21.15.38:0/2109705974 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c2806cff0 0x7f7c280cffb0 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.240+0000 7f7c2cb5c640 1 --2- 172.21.15.38:0/2109705974 >> v2:172.21.15.107:3300/0 conn(0x7f7c2806cc10 0x7f7c280cfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.240+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/2109705974 >> 172.21.15.38:0/2109705974 conn(0x7f7c280b31f0 msgr2=0x7f7c280b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:31.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.240+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/2109705974 shutdown_connections 2024-04-25T08:42:31.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.240+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/2109705974 wait complete. 2024-04-25T08:42:31.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.241+0000 7f7c2cb5c640 1 Processor -- start 2024-04-25T08:42:31.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.241+0000 7f7c2cb5c640 1 -- start start 2024-04-25T08:42:31.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2cb5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c2806cc10 0x7f7c28151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2cb5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c2806cff0 0x7f7c280c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2cb5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c28099a70 0x7f7c280b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2cb5c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2806c150 con 0x7f7c2806cc10 2024-04-25T08:42:31.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2cb5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c280d1140 con 0x7f7c28099a70 2024-04-25T08:42:31.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2cb5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2806c2d0 con 0x7f7c2806cff0 2024-04-25T08:42:31.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c2806cc10 0x7f7c28151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c2806cc10 0x7f7c28151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40404/0 (socket says 172.21.15.38:40404) 2024-04-25T08:42:31.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2659c640 1 -- 172.21.15.38:0/3671865799 learned_addr learned my addr 172.21.15.38:0/3671865799 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:31.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c1ffff640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c2806cff0 0x7f7c280c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c26d9d640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c28099a70 0x7f7c280b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2659c640 1 -- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c2806cff0 msgr2=0x7f7c280c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2659c640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c2806cff0 0x7f7c280c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2659c640 1 -- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c28099a70 msgr2=0x7f7c280b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2659c640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c28099a70 0x7f7c280b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2659c640 1 -- 172.21.15.38:0/3671865799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c280c92e0 con 0x7f7c2806cc10 2024-04-25T08:42:31.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.242+0000 7f7c2659c640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c2806cc10 0x7f7c28151b30 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800d800 tx=0x7f7c1800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:31.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.243+0000 7f7c1effd640 1 -- 172.21.15.38:0/3671865799 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c18018070 con 0x7f7c2806cc10 2024-04-25T08:42:31.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.243+0000 7f7c1effd640 1 -- 172.21.15.38:0/3671865799 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c18004030 con 0x7f7c2806cc10 2024-04-25T08:42:31.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.243+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/3671865799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c280c61b0 con 0x7f7c2806cc10 2024-04-25T08:42:31.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.243+0000 7f7c1effd640 1 -- 172.21.15.38:0/3671865799 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c18004400 con 0x7f7c2806cc10 2024-04-25T08:42:31.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.243+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/3671865799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c280c6770 con 0x7f7c2806cc10 2024-04-25T08:42:31.237 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.244+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/3671865799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c28068e80 con 0x7f7c2806cc10 2024-04-25T08:42:31.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.244+0000 7f7c1effd640 1 -- 172.21.15.38:0/3671865799 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c18009640 con 0x7f7c2806cc10 2024-04-25T08:42:31.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.245+0000 7f7c1effd640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c0806b4d0 0x7f7c0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.245+0000 7f7c1effd640 1 -- 172.21.15.38:0/3671865799 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f7c1808a870 con 0x7f7c2806cc10 2024-04-25T08:42:31.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.245+0000 7f7c1ffff640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c0806b4d0 0x7f7c0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.245+0000 7f7c1ffff640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c0806b4d0 0x7f7c0806d990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7c10034000 tx=0x7f7c1002e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:31.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.249+0000 7f7c1effd640 1 -- 172.21.15.38:0/3671865799 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c18059440 con 0x7f7c2806cc10 2024-04-25T08:42:31.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.465+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/3671865799 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c2806d450 con 0x7f7c2806cc10 2024-04-25T08:42:31.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.466+0000 7f7c1effd640 1 -- 172.21.15.38:0/3671865799 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4960 (secure 0 0 0) 0x7f7c18058d40 con 0x7f7c2806cc10 2024-04-25T08:42:31.458 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:31.458 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2682394530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2682394530},{"type":"v1","addr":"172.21.15.107:6837","nonce":2682394530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2810760276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2810760276},{"type":"v1","addr":"172.21.15.107:6835","nonce":2810760276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T08:42:04.044929+0000","modified":"2024-04-25T08:42:25.774447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10038":{"gid":10038,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3318593502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3318593502},{"type":"v1","addr":"172.21.15.38:6833","nonce":3318593502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10041":{"gid":10041,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/3513331661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3513331661},{"type":"v1","addr":"172.21.15.38:6835","nonce":3513331661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:42:31.458 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 91 2024-04-25T08:42:31.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.468+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c0806b4d0 msgr2=0x7f7c0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.468+0000 7f7c2cb5c640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c0806b4d0 0x7f7c0806d990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f7c10034000 tx=0x7f7c1002e000 comp rx=0 tx=0).stop 2024-04-25T08:42:31.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.468+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/3671865799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c2806cc10 msgr2=0x7f7c28151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.468+0000 7f7c2cb5c640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c2806cc10 0x7f7c28151b30 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800d800 tx=0x7f7c1800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.468+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/3671865799 shutdown_connections 2024-04-25T08:42:31.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.468+0000 7f7c2cb5c640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7c28099a70 0x7f7c280b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.468+0000 7f7c2cb5c640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f7c0806b4d0 0x7f7c0806d990 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.468+0000 7f7c2cb5c640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7c2806cff0 0x7f7c280c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.468+0000 7f7c2cb5c640 1 --2- 172.21.15.38:0/3671865799 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c2806cc10 0x7f7c28151b30 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.468+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/3671865799 >> 172.21.15.38:0/3671865799 conn(0x7f7c280b31f0 msgr2=0x7f7c280aecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:31.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.468+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/3671865799 shutdown_connections 2024-04-25T08:42:31.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.468+0000 7f7c2cb5c640 1 -- 172.21.15.38:0/3671865799 wait complete. 2024-04-25T08:42:31.471 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:42:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.581+0000 7f9999e93640 1 Processor -- start 2024-04-25T08:42:31.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.581+0000 7f9999e93640 1 -- start start 2024-04-25T08:42:31.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.581+0000 7f9999e93640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f999415dc10 0x7f99940ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.581+0000 7f9999e93640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99940d0f60 0x7f99940b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.581+0000 7f9999e93640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99940d3e70 con 0x7f99940ced60 2024-04-25T08:42:31.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.582+0000 7f9999e93640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f999415dff0 con 0x7f999415dc10 2024-04-25T08:42:31.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.582+0000 7f9999e93640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99940b6990 con 0x7f99940d0f60 2024-04-25T08:42:31.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.582+0000 7f99937fe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f999415dc10 0x7f99940ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.582+0000 7f9992ffd640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f99940ced60 0x7f99940d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:39318/0 (socket says 172.21.15.38:39318) 2024-04-25T08:42:31.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.582+0000 7f9992ffd640 1 -- 172.21.15.38:0/3015061686 learned_addr learned my addr 172.21.15.38:0/3015061686 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:31.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.582+0000 7f9993fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99940d0f60 0x7f99940b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.582+0000 7f9993fff640 1 -- 172.21.15.38:0/3015061686 >> v2:172.21.15.107:3300/0 conn(0x7f999415dc10 msgr2=0x7f99940ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:31.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.582+0000 7f9993fff640 1 --2- 172.21.15.38:0/3015061686 >> v2:172.21.15.107:3300/0 conn(0x7f999415dc10 0x7f99940ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.582+0000 7f9993fff640 1 -- 172.21.15.38:0/3015061686 >> v1:172.21.15.107:6789/0 conn(0x7f99940ced60 legacy=0x7f99940d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.582+0000 7f9993fff640 1 -- 172.21.15.38:0/3015061686 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99940d4150 con 0x7f99940d0f60 2024-04-25T08:42:31.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.582+0000 7f99937fe640 1 --2- 172.21.15.38:0/3015061686 >> v2:172.21.15.107:3300/0 conn(0x7f999415dc10 0x7f99940ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:31.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.583+0000 7f9993fff640 1 --2- 172.21.15.38:0/3015061686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99940d0f60 0x7f99940b2370 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f998c009d90 tx=0x7f998c02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c22aeb2596bf80ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:31.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.583+0000 7f99927fc640 1 -- 172.21.15.38:0/3015061686 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c031600 con 0x7f99940d0f60 2024-04-25T08:42:31.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.583+0000 7f99927fc640 1 -- 172.21.15.38:0/3015061686 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f998c0317a0 con 0x7f99940d0f60 2024-04-25T08:42:31.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.583+0000 7f99927fc640 1 -- 172.21.15.38:0/3015061686 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f998c031b70 con 0x7f99940d0f60 2024-04-25T08:42:31.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.584+0000 7f9999e93640 1 -- 172.21.15.38:0/3015061686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99940d0f60 msgr2=0x7f99940b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.584+0000 7f9999e93640 1 --2- 172.21.15.38:0/3015061686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99940d0f60 0x7f99940b2370 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f998c009d90 tx=0x7f998c02a5d0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.584+0000 7f9999e93640 1 -- 172.21.15.38:0/3015061686 shutdown_connections 2024-04-25T08:42:31.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.584+0000 7f9999e93640 1 --2- 172.21.15.38:0/3015061686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99940d0f60 0x7f99940b2370 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.584+0000 7f9999e93640 1 --2- 172.21.15.38:0/3015061686 >> v2:172.21.15.107:3300/0 conn(0x7f999415dc10 0x7f99940ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.584+0000 7f9999e93640 1 -- 172.21.15.38:0/3015061686 >> 172.21.15.38:0/3015061686 conn(0x7f9994066d10 msgr2=0x7f9994067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:31.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.584+0000 7f9999e93640 1 -- 172.21.15.38:0/3015061686 shutdown_connections 2024-04-25T08:42:31.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.584+0000 7f9999e93640 1 -- 172.21.15.38:0/3015061686 wait complete. 2024-04-25T08:42:31.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.585+0000 7f9999e93640 1 Processor -- start 2024-04-25T08:42:31.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.585+0000 7f9999e93640 1 -- start start 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.586+0000 7f9999e93640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99940ced60 0x7f99941f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.586+0000 7f9999e93640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99940d0f60 0x7f999415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.586+0000 7f9999e93640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999415dc10 0x7f999415b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.586+0000 7f9999e93640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99940b6bc0 con 0x7f99940d0f60 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.586+0000 7f9999e93640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99940d4710 con 0x7f999415dc10 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.586+0000 7f9999e93640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99940d3900 con 0x7f99940ced60 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.586+0000 7f9992ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99940d0f60 0x7f999415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.586+0000 7f9992ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99940d0f60 0x7f999415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40418/0 (socket says 172.21.15.38:40418) 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.586+0000 7f9992ffd640 1 -- 172.21.15.38:0/2907371323 learned_addr learned my addr 172.21.15.38:0/2907371323 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.586+0000 7f99937fe640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99940ced60 0x7f99941f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.586+0000 7f9993fff640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999415dc10 0x7f999415b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.587+0000 7f9992ffd640 1 -- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99940ced60 msgr2=0x7f99941f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.587+0000 7f9992ffd640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99940ced60 0x7f99941f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.587+0000 7f9992ffd640 1 -- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999415dc10 msgr2=0x7f999415b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.587+0000 7f9992ffd640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999415dc10 0x7f999415b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.587+0000 7f9992ffd640 1 -- 172.21.15.38:0/2907371323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99941f4000 con 0x7f99940d0f60 2024-04-25T08:42:31.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.587+0000 7f9993fff640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999415dc10 0x7f999415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:31.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.587+0000 7f9992ffd640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99940d0f60 0x7f999415b1c0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f998800c400 tx=0x7f998800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:31.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.587+0000 7f997bfff640 1 -- 172.21.15.38:0/2907371323 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9988017070 con 0x7f99940d0f60 2024-04-25T08:42:31.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.587+0000 7f997bfff640 1 -- 172.21.15.38:0/2907371323 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9988005ce0 con 0x7f99940d0f60 2024-04-25T08:42:31.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.587+0000 7f9999e93640 1 -- 172.21.15.38:0/2907371323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99941f3760 con 0x7f99940d0f60 2024-04-25T08:42:31.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.587+0000 7f997bfff640 1 -- 172.21.15.38:0/2907371323 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880072f0 con 0x7f99940d0f60 2024-04-25T08:42:31.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.587+0000 7f9999e93640 1 -- 172.21.15.38:0/2907371323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99941f3d20 con 0x7f99940d0f60 2024-04-25T08:42:31.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.588+0000 7f9999e93640 1 -- 172.21.15.38:0/2907371323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9994068e80 con 0x7f99940d0f60 2024-04-25T08:42:31.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.589+0000 7f997bfff640 1 -- 172.21.15.38:0/2907371323 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f998801a070 con 0x7f99940d0f60 2024-04-25T08:42:31.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.589+0000 7f997bfff640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f998006b4d0 0x7f998006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.589+0000 7f997bfff640 1 -- 172.21.15.38:0/2907371323 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f99880897a0 con 0x7f99940d0f60 2024-04-25T08:42:31.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.589+0000 7f99937fe640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f998006b4d0 0x7f998006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.590+0000 7f99937fe640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f998006b4d0 0x7f998006d990 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9984009560 tx=0x7f9984009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:31.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.593+0000 7f997bfff640 1 -- 172.21.15.38:0/2907371323 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9988057db0 con 0x7f99940d0f60 2024-04-25T08:42:31.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.812+0000 7f9999e93640 1 -- 172.21.15.38:0/2907371323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f999406af70 con 0x7f99940d0f60 2024-04-25T08:42:31.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.812+0000 7f997bfff640 1 -- 172.21.15.38:0/2907371323 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4960 (secure 0 0 0) 0x7f998805be50 con 0x7f99940d0f60 2024-04-25T08:42:31.805 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:31.805 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":91,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2682394530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2682394530},{"type":"v1","addr":"172.21.15.107:6837","nonce":2682394530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2810760276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2810760276},{"type":"v1","addr":"172.21.15.107:6835","nonce":2810760276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T08:42:04.044929+0000","modified":"2024-04-25T08:42:25.774447+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10041},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10038":{"gid":10038,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.38:6833/3318593502","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":3318593502},{"type":"v1","addr":"172.21.15.38:6833","nonce":3318593502}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_10041":{"gid":10041,"name":"c","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.38:6835/3513331661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":3513331661},{"type":"v1","addr":"172.21.15.38:6835","nonce":3513331661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T08:42:31.805 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 91 2024-04-25T08:42:31.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.814+0000 7f9999e93640 1 -- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f998006b4d0 msgr2=0x7f998006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.814+0000 7f9999e93640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f998006b4d0 0x7f998006d990 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9984009560 tx=0x7f9984009190 comp rx=0 tx=0).stop 2024-04-25T08:42:31.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.814+0000 7f9999e93640 1 -- 172.21.15.38:0/2907371323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99940d0f60 msgr2=0x7f999415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.814+0000 7f9999e93640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99940d0f60 0x7f999415b1c0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f998800c400 tx=0x7f998800c8c0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.814+0000 7f9999e93640 1 -- 172.21.15.38:0/2907371323 shutdown_connections 2024-04-25T08:42:31.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.814+0000 7f9999e93640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f999415dc10 0x7f999415b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.814+0000 7f9999e93640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99940d0f60 0x7f999415b1c0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.814+0000 7f9999e93640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f998006b4d0 0x7f998006d990 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.814+0000 7f9999e93640 1 --2- 172.21.15.38:0/2907371323 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f99940ced60 0x7f99941f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.814+0000 7f9999e93640 1 -- 172.21.15.38:0/2907371323 >> 172.21.15.38:0/2907371323 conn(0x7f9994066d10 msgr2=0x7f99940d3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:31.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.814+0000 7f9999e93640 1 -- 172.21.15.38:0/2907371323 shutdown_connections 2024-04-25T08:42:31.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.814+0000 7f9999e93640 1 -- 172.21.15.38:0/2907371323 wait complete. 2024-04-25T08:42:31.818 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:31.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.926+0000 7f8c17dee640 1 Processor -- start 2024-04-25T08:42:31.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.926+0000 7f8c17dee640 1 -- start start 2024-04-25T08:42:31.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c17dee640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8c1006c110 0x7f8c1006c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c17dee640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f8c1015df50 0x7f8c100ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c17dee640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1006ca30 con 0x7f8c100ce2e0 2024-04-25T08:42:31.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c17dee640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c100cf2d0 con 0x7f8c1006c110 2024-04-25T08:42:31.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c17dee640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c100ceff0 con 0x7f8c1015df50 2024-04-25T08:42:31.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c15b8a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8c1006c110 0x7f8c1006c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c15b8a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f8c1006c110 0x7f8c1006c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40430/0 (socket says 172.21.15.38:40430) 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c15b8a640 1 -- 172.21.15.38:0/43056471 learned_addr learned my addr 172.21.15.38:0/43056471 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c15389640 1 --2- 172.21.15.38:0/43056471 >> v2:172.21.15.107:3300/0 conn(0x7f8c1015df50 0x7f8c100ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c15b8a640 1 -- 172.21.15.38:0/43056471 >> v2:172.21.15.107:3300/0 conn(0x7f8c1015df50 msgr2=0x7f8c100ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c15b8a640 1 --2- 172.21.15.38:0/43056471 >> v2:172.21.15.107:3300/0 conn(0x7f8c1015df50 0x7f8c100ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c15b8a640 1 -- 172.21.15.38:0/43056471 >> v1:172.21.15.107:6789/0 conn(0x7f8c100ce2e0 legacy=0x7f8c100d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c15b8a640 1 -- 172.21.15.38:0/43056471 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c100d1f70 con 0x7f8c1006c110 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.927+0000 7f8c15389640 1 --2- 172.21.15.38:0/43056471 >> v2:172.21.15.107:3300/0 conn(0x7f8c1015df50 0x7f8c100ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.928+0000 7f8c15b8a640 1 --2- 172.21.15.38:0/43056471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1006c110 0x7f8c1006c4f0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f8c100d4960 tx=0x7f8c0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee1d1341b56d59b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.928+0000 7f8c14b88640 1 -- 172.21.15.38:0/43056471 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c037070 con 0x7f8c1006c110 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.928+0000 7f8c14b88640 1 -- 172.21.15.38:0/43056471 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c032440 con 0x7f8c1006c110 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.928+0000 7f8c14b88640 1 -- 172.21.15.38:0/43056471 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c032810 con 0x7f8c1006c110 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.929+0000 7f8c17dee640 1 -- 172.21.15.38:0/43056471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1006c110 msgr2=0x7f8c1006c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.929+0000 7f8c17dee640 1 --2- 172.21.15.38:0/43056471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1006c110 0x7f8c1006c4f0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f8c100d4960 tx=0x7f8c0c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.929+0000 7f8c17dee640 1 -- 172.21.15.38:0/43056471 shutdown_connections 2024-04-25T08:42:31.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.929+0000 7f8c17dee640 1 --2- 172.21.15.38:0/43056471 >> v2:172.21.15.107:3300/0 conn(0x7f8c1015df50 0x7f8c100ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.929+0000 7f8c17dee640 1 --2- 172.21.15.38:0/43056471 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1006c110 0x7f8c1006c4f0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.929+0000 7f8c17dee640 1 -- 172.21.15.38:0/43056471 >> 172.21.15.38:0/43056471 conn(0x7f8c100d3fb0 msgr2=0x7f8c100d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:31.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.929+0000 7f8c17dee640 1 -- 172.21.15.38:0/43056471 shutdown_connections 2024-04-25T08:42:31.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.929+0000 7f8c17dee640 1 -- 172.21.15.38:0/43056471 wait complete. 2024-04-25T08:42:31.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.930+0000 7f8c17dee640 1 Processor -- start 2024-04-25T08:42:31.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.930+0000 7f8c17dee640 1 -- start start 2024-04-25T08:42:31.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.930+0000 7f8c17dee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1006c110 0x7f8c100cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c17dee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c100ce2e0 0x7f8c1015e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c17dee640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c1015df50 0x7f8c1015ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c17dee640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100cece0 con 0x7f8c1006c110 2024-04-25T08:42:31.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c17dee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100d2530 con 0x7f8c100ce2e0 2024-04-25T08:42:31.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c17dee640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100d3160 con 0x7f8c1015df50 2024-04-25T08:42:31.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c15b8a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1006c110 0x7f8c100cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c15b8a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1006c110 0x7f8c100cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40438/0 (socket says 172.21.15.38:40438) 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c15b8a640 1 -- 172.21.15.38:0/2381353577 learned_addr learned my addr 172.21.15.38:0/2381353577 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c15389640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c100ce2e0 0x7f8c1015e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c1638b640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c1015df50 0x7f8c1015ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c15b8a640 1 -- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c1015df50 msgr2=0x7f8c1015ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c15b8a640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c1015df50 0x7f8c1015ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c15b8a640 1 -- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c100ce2e0 msgr2=0x7f8c1015e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c15b8a640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c100ce2e0 0x7f8c1015e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c15b8a640 1 -- 172.21.15.38:0/2381353577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1015ea30 con 0x7f8c1006c110 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c15b8a640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1006c110 0x7f8c100cd400 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c000c00 tx=0x7f8c0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c067fc640 1 -- 172.21.15.38:0/2381353577 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c041020 con 0x7f8c1006c110 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c067fc640 1 -- 172.21.15.38:0/2381353577 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0c030970 con 0x7f8c1006c110 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c17dee640 1 -- 172.21.15.38:0/2381353577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1015ed10 con 0x7f8c1006c110 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c067fc640 1 -- 172.21.15.38:0/2381353577 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c037040 con 0x7f8c1006c110 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.931+0000 7f8c17dee640 1 -- 172.21.15.38:0/2381353577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1015f2d0 con 0x7f8c1006c110 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.932+0000 7f8c17dee640 1 -- 172.21.15.38:0/2381353577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c100c71a0 con 0x7f8c1006c110 2024-04-25T08:42:31.926 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.933+0000 7f8c067fc640 1 -- 172.21.15.38:0/2381353577 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c0c044020 con 0x7f8c1006c110 2024-04-25T08:42:31.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.933+0000 7f8c067fc640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8bf406b4d0 0x7f8bf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:31.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.933+0000 7f8c067fc640 1 -- 172.21.15.38:0/2381353577 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f8c0c0aaca0 con 0x7f8c1006c110 2024-04-25T08:42:31.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.933+0000 7f8c15389640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8bf406b4d0 0x7f8bf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:31.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.934+0000 7f8c15389640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8bf406b4d0 0x7f8bf406d990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8c100d38e0 tx=0x7f8c00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:31.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:31.937+0000 7f8c067fc640 1 -- 172.21.15.38:0/2381353577 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c100c71a0 con 0x7f8c1006c110 2024-04-25T08:42:32.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.123+0000 7f8c17dee640 1 -- 172.21.15.38:0/2381353577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c10068e80 con 0x7f8c1006c110 2024-04-25T08:42:32.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.124+0000 7f8c067fc640 1 -- 172.21.15.38:0/2381353577 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+17179 (secure 0 0 0) 0x7f8c0c04b030 con 0x7f8c1006c110 2024-04-25T08:42:32.117 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:32.117 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":145,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:42:19.745001+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:42:01.691830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T08:42:03.024111+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"22.8","mappings":[{"from":4,"to":1}]},{"pgid":"22.c","mappings":[{"from":3,"to":1}]},{"pgid":"22.e","mappings":[{"from":3,"to":0}]},{"pgid":"22.21","mappings":[{"from":3,"to":0}]},{"pgid":"22.23","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.126+0000 7f8c17dee640 1 -- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8bf406b4d0 msgr2=0x7f8bf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.126+0000 7f8c17dee640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8bf406b4d0 0x7f8bf406d990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8c100d38e0 tx=0x7f8c00008040 comp rx=0 tx=0).stop 2024-04-25T08:42:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.126+0000 7f8c17dee640 1 -- 172.21.15.38:0/2381353577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1006c110 msgr2=0x7f8c100cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.126+0000 7f8c17dee640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1006c110 0x7f8c100cd400 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c000c00 tx=0x7f8c0c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.126+0000 7f8c17dee640 1 -- 172.21.15.38:0/2381353577 shutdown_connections 2024-04-25T08:42:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.126+0000 7f8c17dee640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f8bf406b4d0 0x7f8bf406d990 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.126+0000 7f8c17dee640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8c1015df50 0x7f8c1015ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.126+0000 7f8c17dee640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8c100ce2e0 0x7f8c1015e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.126+0000 7f8c17dee640 1 --2- 172.21.15.38:0/2381353577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8c1006c110 0x7f8c100cd400 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.126+0000 7f8c17dee640 1 -- 172.21.15.38:0/2381353577 >> 172.21.15.38:0/2381353577 conn(0x7f8c100d3fb0 msgr2=0x7f8c100b8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:32.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.126+0000 7f8c17dee640 1 -- 172.21.15.38:0/2381353577 shutdown_connections 2024-04-25T08:42:32.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.127+0000 7f8c17dee640 1 -- 172.21.15.38:0/2381353577 wait complete. 2024-04-25T08:42:32.130 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:42:32.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.240+0000 7f4908e9e640 1 Processor -- start 2024-04-25T08:42:32.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.240+0000 7f4908e9e640 1 -- start start 2024-04-25T08:42:32.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.241+0000 7f4908e9e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f49040d0d10 0x7f4904099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.241+0000 7f4908e9e640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49040cfc70 con 0x7f490415def0 2024-04-25T08:42:32.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.241+0000 7f4908e9e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49040cff50 con 0x7f4904099fb0 2024-04-25T08:42:32.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.241+0000 7f4908e9e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490415e2d0 con 0x7f49040d0d10 2024-04-25T08:42:32.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.241+0000 7f490259c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f490415def0 0x7f490406c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40494/0 (socket says 172.21.15.38:40494) 2024-04-25T08:42:32.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.241+0000 7f4901d9b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f49040d0d10 0x7f4904099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.241+0000 7f490259c640 1 -- 172.21.15.38:0/2540854070 learned_addr learned my addr 172.21.15.38:0/2540854070 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:32.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.241+0000 7f4901d9b640 1 -- 172.21.15.38:0/2540854070 >> v1:172.21.15.107:6789/0 conn(0x7f4904099fb0 legacy=0x7f49040cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:32.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.241+0000 7f4901d9b640 1 -- 172.21.15.38:0/2540854070 >> v1:172.21.15.38:6789/0 conn(0x7f490415def0 legacy=0x7f490406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.241+0000 7f4901d9b640 1 -- 172.21.15.38:0/2540854070 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49040d0230 con 0x7f49040d0d10 2024-04-25T08:42:32.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.242+0000 7f4901d9b640 1 --2- 172.21.15.38:0/2540854070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49040d0d10 0x7f4904099a70 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f48f80097a0 tx=0x7f48f802a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c4a8b7612ffbc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.242+0000 7f490159a640 1 -- 172.21.15.38:0/2540854070 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8037070 con 0x7f49040d0d10 2024-04-25T08:42:32.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.242+0000 7f490159a640 1 -- 172.21.15.38:0/2540854070 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f802ad60 con 0x7f49040d0d10 2024-04-25T08:42:32.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.242+0000 7f490159a640 1 -- 172.21.15.38:0/2540854070 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8032660 con 0x7f49040d0d10 2024-04-25T08:42:32.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.243+0000 7f4908e9e640 1 -- 172.21.15.38:0/2540854070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49040d0d10 msgr2=0x7f4904099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.243+0000 7f4908e9e640 1 --2- 172.21.15.38:0/2540854070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49040d0d10 0x7f4904099a70 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f48f80097a0 tx=0x7f48f802a0d0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.243+0000 7f4908e9e640 1 -- 172.21.15.38:0/2540854070 shutdown_connections 2024-04-25T08:42:32.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.243+0000 7f4908e9e640 1 --2- 172.21.15.38:0/2540854070 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49040d0d10 0x7f4904099a70 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.243+0000 7f4908e9e640 1 -- 172.21.15.38:0/2540854070 >> 172.21.15.38:0/2540854070 conn(0x7f49040b30f0 msgr2=0x7f49040b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:32.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.243+0000 7f4908e9e640 1 -- 172.21.15.38:0/2540854070 shutdown_connections 2024-04-25T08:42:32.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.243+0000 7f4908e9e640 1 -- 172.21.15.38:0/2540854070 wait complete. 2024-04-25T08:42:32.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.244+0000 7f4908e9e640 1 Processor -- start 2024-04-25T08:42:32.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.244+0000 7f4908e9e640 1 -- start start 2024-04-25T08:42:32.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f4908e9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4904099fb0 0x7f4904156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f4908e9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49040d0d10 0x7f49040703b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f4908e9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f490415def0 0x7f49040708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f4908e9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49040d07f0 con 0x7f4904099fb0 2024-04-25T08:42:32.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f4908e9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490409a3e0 con 0x7f490415def0 2024-04-25T08:42:32.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f4908e9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49040d1170 con 0x7f49040d0d10 2024-04-25T08:42:32.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f490259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4904099fb0 0x7f4904156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f490259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4904099fb0 0x7f4904156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40464/0 (socket says 172.21.15.38:40464) 2024-04-25T08:42:32.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f490259c640 1 -- 172.21.15.38:0/2031724713 learned_addr learned my addr 172.21.15.38:0/2031724713 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:32.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f4901d9b640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49040d0d10 0x7f49040703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f490259c640 1 -- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49040d0d10 msgr2=0x7f49040703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f490259c640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49040d0d10 0x7f49040703b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f490259c640 1 -- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f490415def0 msgr2=0x7f49040708f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.245+0000 7f4902d9d640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f490415def0 0x7f49040708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.246+0000 7f490259c640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f490415def0 0x7f49040708f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.246+0000 7f490259c640 1 -- 172.21.15.38:0/2031724713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49040b5fc0 con 0x7f4904099fb0 2024-04-25T08:42:32.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.246+0000 7f4902d9d640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f490415def0 0x7f49040708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:32.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.246+0000 7f490259c640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4904099fb0 0x7f4904156bd0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f48f400c7b0 tx=0x7f48f400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.246+0000 7f48f2ffd640 1 -- 172.21.15.38:0/2031724713 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4018070 con 0x7f4904099fb0 2024-04-25T08:42:32.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.246+0000 7f48f2ffd640 1 -- 172.21.15.38:0/2031724713 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f40072f0 con 0x7f4904099fb0 2024-04-25T08:42:32.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.246+0000 7f4908e9e640 1 -- 172.21.15.38:0/2031724713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49040b62a0 con 0x7f4904099fb0 2024-04-25T08:42:32.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.246+0000 7f48f2ffd640 1 -- 172.21.15.38:0/2031724713 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f40047e0 con 0x7f4904099fb0 2024-04-25T08:42:32.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.246+0000 7f4908e9e640 1 -- 172.21.15.38:0/2031724713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49040b6860 con 0x7f4904099fb0 2024-04-25T08:42:32.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.247+0000 7f4908e9e640 1 -- 172.21.15.38:0/2031724713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490404ef90 con 0x7f4904099fb0 2024-04-25T08:42:32.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.247+0000 7f48f2ffd640 1 -- 172.21.15.38:0/2031724713 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48f401b070 con 0x7f4904099fb0 2024-04-25T08:42:32.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.248+0000 7f48f2ffd640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f48e406b4d0 0x7f48e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.248+0000 7f48f2ffd640 1 -- 172.21.15.38:0/2031724713 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f48f4089870 con 0x7f4904099fb0 2024-04-25T08:42:32.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.248+0000 7f4901d9b640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f48e406b4d0 0x7f48e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.248+0000 7f4901d9b640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f48e406b4d0 0x7f48e406d990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f48f8034000 tx=0x7f48f802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.252+0000 7f48f2ffd640 1 -- 172.21.15.38:0/2031724713 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48f40584d0 con 0x7f4904099fb0 2024-04-25T08:42:32.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.473+0000 7f4908e9e640 1 -- 172.21.15.38:0/2031724713 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f49040b3e30 con 0x7f4904099fb0 2024-04-25T08:42:32.466 INFO:tasks.ceph.mon.a.smithi038.stderr:2024-04-25T08:42:32.474+0000 7ff7937f0640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:42:32.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.475+0000 7f48f2ffd640 1 -- 172.21.15.38:0/2031724713 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) v1 ==== 161+0+0 (secure 0 0 0) 0x7f48f4057dd0 con 0x7f4904099fb0 2024-04-25T08:42:32.468 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:42:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.478+0000 7f4908e9e640 1 -- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f48e406b4d0 msgr2=0x7f48e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.478+0000 7f4908e9e640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f48e406b4d0 0x7f48e406d990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f48f8034000 tx=0x7f48f802e000 comp rx=0 tx=0).stop 2024-04-25T08:42:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.478+0000 7f4908e9e640 1 -- 172.21.15.38:0/2031724713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4904099fb0 msgr2=0x7f4904156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.478+0000 7f4908e9e640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4904099fb0 0x7f4904156bd0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f48f400c7b0 tx=0x7f48f400cc70 comp rx=0 tx=0).stop 2024-04-25T08:42:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.479+0000 7f4908e9e640 1 -- 172.21.15.38:0/2031724713 shutdown_connections 2024-04-25T08:42:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.479+0000 7f4908e9e640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f490415def0 0x7f49040708f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.479+0000 7f4908e9e640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f48e406b4d0 0x7f48e406d990 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.479+0000 7f4908e9e640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49040d0d10 0x7f49040703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.479+0000 7f4908e9e640 1 --2- 172.21.15.38:0/2031724713 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4904099fb0 0x7f4904156bd0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.479+0000 7f4908e9e640 1 -- 172.21.15.38:0/2031724713 >> 172.21.15.38:0/2031724713 conn(0x7f49040b30f0 msgr2=0x7f49040ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:32.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.479+0000 7f4908e9e640 1 -- 172.21.15.38:0/2031724713 shutdown_connections 2024-04-25T08:42:32.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.479+0000 7f4908e9e640 1 -- 172.21.15.38:0/2031724713 wait complete. 2024-04-25T08:42:32.483 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:42:32.508 INFO:tasks.ceph.mds.a.smithi038.stdout:starting mds.a at 2024-04-25T08:42:32.510 INFO:tasks.ceph.mds.c.smithi038.stdout:starting mds.c at 2024-04-25T08:42:32.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.597+0000 7fa01519e640 1 Processor -- start 2024-04-25T08:42:32.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.597+0000 7fa01519e640 1 -- start start 2024-04-25T08:42:32.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.597+0000 7fa01519e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa0100d33d0 0x7fa0100d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.597+0000 7fa01519e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fa0100d1e90 0x7fa0100b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.597+0000 7fa01519e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0100d2330 con 0x7fa0100d0320 2024-04-25T08:42:32.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.597+0000 7fa01519e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0100d2610 con 0x7fa0100d33d0 2024-04-25T08:42:32.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.597+0000 7fa01519e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0100d28f0 con 0x7fa0100d1e90 2024-04-25T08:42:32.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.598+0000 7fa007fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa0100d33d0 0x7fa0100d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.598+0000 7fa007fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fa0100d33d0 0x7fa0100d1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40502/0 (socket says 172.21.15.38:40502) 2024-04-25T08:42:32.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.598+0000 7fa007fff640 1 -- 172.21.15.38:0/526052521 learned_addr learned my addr 172.21.15.38:0/526052521 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:32.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.598+0000 7fa00f59e640 1 --2- 172.21.15.38:0/526052521 >> v2:172.21.15.107:3300/0 conn(0x7fa0100d1e90 0x7fa0100b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.598+0000 7fa007fff640 1 -- 172.21.15.38:0/526052521 >> v2:172.21.15.107:3300/0 conn(0x7fa0100d1e90 msgr2=0x7fa0100b6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:32.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.598+0000 7fa007fff640 1 --2- 172.21.15.38:0/526052521 >> v2:172.21.15.107:3300/0 conn(0x7fa0100d1e90 0x7fa0100b6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.598+0000 7fa007fff640 1 -- 172.21.15.38:0/526052521 >> v1:172.21.15.107:6789/0 conn(0x7fa0100d0320 legacy=0x7fa0100d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.598+0000 7fa007fff640 1 -- 172.21.15.38:0/526052521 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0100d2bd0 con 0x7fa0100d33d0 2024-04-25T08:42:32.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.598+0000 7fa00f59e640 1 --2- 172.21.15.38:0/526052521 >> v2:172.21.15.107:3300/0 conn(0x7fa0100d1e90 0x7fa0100b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:32.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.599+0000 7fa007fff640 1 --2- 172.21.15.38:0/526052521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d33d0 0x7fa0100d1950 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fa0100ca800 tx=0x7f9ff802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea78ebe5a7b8222 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.599+0000 7fa00e59c640 1 -- 172.21.15.38:0/526052521 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8037070 con 0x7fa0100d33d0 2024-04-25T08:42:32.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.599+0000 7fa00e59c640 1 -- 172.21.15.38:0/526052521 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff802ae50 con 0x7fa0100d33d0 2024-04-25T08:42:32.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.599+0000 7fa00e59c640 1 -- 172.21.15.38:0/526052521 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff8032860 con 0x7fa0100d33d0 2024-04-25T08:42:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.599+0000 7fa01519e640 1 -- 172.21.15.38:0/526052521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d33d0 msgr2=0x7fa0100d1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.599+0000 7fa01519e640 1 --2- 172.21.15.38:0/526052521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d33d0 0x7fa0100d1950 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fa0100ca800 tx=0x7f9ff802a300 comp rx=0 tx=0).stop 2024-04-25T08:42:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.599+0000 7fa01519e640 1 -- 172.21.15.38:0/526052521 shutdown_connections 2024-04-25T08:42:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.599+0000 7fa01519e640 1 --2- 172.21.15.38:0/526052521 >> v2:172.21.15.107:3300/0 conn(0x7fa0100d1e90 0x7fa0100b6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.599+0000 7fa01519e640 1 --2- 172.21.15.38:0/526052521 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d33d0 0x7fa0100d1950 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.599+0000 7fa01519e640 1 -- 172.21.15.38:0/526052521 >> 172.21.15.38:0/526052521 conn(0x7fa0100668f0 msgr2=0x7fa010066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:32.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.599+0000 7fa01519e640 1 -- 172.21.15.38:0/526052521 shutdown_connections 2024-04-25T08:42:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.600+0000 7fa01519e640 1 -- 172.21.15.38:0/526052521 wait complete. 2024-04-25T08:42:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.601+0000 7fa01519e640 1 Processor -- start 2024-04-25T08:42:32.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.601+0000 7fa01519e640 1 -- start start 2024-04-25T08:42:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa01519e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0320 0x7fa0101e7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa01519e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0100d1e90 0x7fa01014dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa01519e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0100d33d0 0x7fa0100ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa01519e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa010161390 con 0x7fa0100d0320 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa01519e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa010161210 con 0x7fa0100d1e90 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa01519e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0100b9720 con 0x7fa0100d33d0 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa00ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0320 0x7fa0101e7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa00ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0320 0x7fa0101e7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40512/0 (socket says 172.21.15.38:40512) 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa00ed9d640 1 -- 172.21.15.38:0/1859955266 learned_addr learned my addr 172.21.15.38:0/1859955266 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa007fff640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0100d1e90 0x7fa01014dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa00f59e640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0100d33d0 0x7fa0100ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa00ed9d640 1 -- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0100d33d0 msgr2=0x7fa0100ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa00ed9d640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0100d33d0 0x7fa0100ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa00ed9d640 1 -- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0100d1e90 msgr2=0x7fa01014dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa00ed9d640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0100d1e90 0x7fa01014dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa00ed9d640 1 -- 172.21.15.38:0/1859955266 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0101e6ed0 con 0x7fa0100d0320 2024-04-25T08:42:32.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa00ed9d640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0320 0x7fa0101e7f70 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fa00000c400 tx=0x7fa00000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa0077fe640 1 -- 172.21.15.38:0/1859955266 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0000052d0 con 0x7fa0100d0320 2024-04-25T08:42:32.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa0077fe640 1 -- 172.21.15.38:0/1859955266 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa000005490 con 0x7fa0100d0320 2024-04-25T08:42:32.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa01519e640 1 -- 172.21.15.38:0/1859955266 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0101e71b0 con 0x7fa0100d0320 2024-04-25T08:42:32.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa0077fe640 1 -- 172.21.15.38:0/1859955266 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000007260 con 0x7fa0100d0320 2024-04-25T08:42:32.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.602+0000 7fa01519e640 1 -- 172.21.15.38:0/1859955266 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0101e7770 con 0x7fa0100d0320 2024-04-25T08:42:32.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.603+0000 7fa01519e640 1 -- 172.21.15.38:0/1859955266 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0100d1490 con 0x7fa0100d0320 2024-04-25T08:42:32.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.604+0000 7fa0077fe640 1 -- 172.21.15.38:0/1859955266 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa000007400 con 0x7fa0100d0320 2024-04-25T08:42:32.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.604+0000 7fa0077fe640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ff006b4d0 0x7f9ff006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:32.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.604+0000 7fa0077fe640 1 -- 172.21.15.38:0/1859955266 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6332+0+0 (secure 0 0 0) 0x7fa000089840 con 0x7fa0100d0320 2024-04-25T08:42:32.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.604+0000 7fa007fff640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ff006b4d0 0x7f9ff006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:32.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.605+0000 7fa007fff640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ff006b4d0 0x7f9ff006d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8034000 tx=0x7f9ff802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:32.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.608+0000 7fa0077fe640 1 -- 172.21.15.38:0/1859955266 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0100d1490 con 0x7fa0100d0320 2024-04-25T08:42:32.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:32.826+0000 7fa01519e640 1 -- 172.21.15.38:0/1859955266 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa010068e80 con 0x7fa0100d0320 2024-04-25T08:42:33.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.481+0000 7fa0077fe640 1 -- 172.21.15.38:0/1859955266 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v93) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa000057e90 con 0x7fa0100d0320 2024-04-25T08:42:33.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.484+0000 7fa01519e640 1 -- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ff006b4d0 msgr2=0x7f9ff006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.484+0000 7fa01519e640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ff006b4d0 0x7f9ff006d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f9ff8034000 tx=0x7f9ff802e000 comp rx=0 tx=0).stop 2024-04-25T08:42:33.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.484+0000 7fa01519e640 1 -- 172.21.15.38:0/1859955266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0320 msgr2=0x7fa0101e7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.484+0000 7fa01519e640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0320 0x7fa0101e7f70 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fa00000c400 tx=0x7fa00000c8c0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.484+0000 7fa01519e640 1 -- 172.21.15.38:0/1859955266 shutdown_connections 2024-04-25T08:42:33.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.484+0000 7fa01519e640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9ff006b4d0 0x7f9ff006d990 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.484+0000 7fa01519e640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa0100d33d0 0x7fa0100ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.484+0000 7fa01519e640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa0100d1e90 0x7fa01014dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.484+0000 7fa01519e640 1 --2- 172.21.15.38:0/1859955266 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa0100d0320 0x7fa0101e7f70 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.484+0000 7fa01519e640 1 -- 172.21.15.38:0/1859955266 >> 172.21.15.38:0/1859955266 conn(0x7fa0100668f0 msgr2=0x7fa010160180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:33.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.484+0000 7fa01519e640 1 -- 172.21.15.38:0/1859955266 shutdown_connections 2024-04-25T08:42:33.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.484+0000 7fa01519e640 1 -- 172.21.15.38:0/1859955266 wait complete. 2024-04-25T08:42:33.492 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.655+0000 7f2290328640 1 Processor -- start 2024-04-25T08:42:33.647 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.655+0000 7f2290328640 1 -- start start 2024-04-25T08:42:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f2290328640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f22880b93f0 0x7f22880d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f2290328640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22880b9880 con 0x7f22880d0870 2024-04-25T08:42:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f2290328640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22880b7200 con 0x7f22881612c0 2024-04-25T08:42:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f2290328640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22880b74e0 con 0x7f22880b93f0 2024-04-25T08:42:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228e8c5640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f22880d0870 0x7f22880d0c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40532/0 (socket says 172.21.15.38:40532) 2024-04-25T08:42:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228e8c5640 1 -- 172.21.15.38:0/2278911610 learned_addr learned my addr 172.21.15.38:0/2278911610 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1888854497 0 0) 0x7f22880b9880 con 0x7f22880d0870 2024-04-25T08:42:33.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228d8c3640 1 --2- 172.21.15.38:0/2278911610 >> v2:172.21.15.107:3300/0 conn(0x7f22880b93f0 0x7f22880d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2274003520 con 0x7f22880d0870 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2720623436 0 0) 0x7f2274003520 con 0x7f22880d0870 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 >> v2:172.21.15.107:3300/0 conn(0x7f22880b93f0 msgr2=0x7f22880d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228d0c2640 1 --2- 172.21.15.38:0/2278911610 >> v2:172.21.15.107:3300/0 conn(0x7f22880b93f0 0x7f22880d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 >> v1:172.21.15.107:6789/0 conn(0x7f22881612c0 legacy=0x7f22880b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22880b77c0 con 0x7f22880d0870 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f22800019f0 con 0x7f22880d0870 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 >> v1:172.21.15.38:6789/0 conn(0x7f22880d0870 legacy=0x7f22880d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.656+0000 7f228d0c2640 1 --2- 172.21.15.38:0/2278911610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22740039b0 0x7f2274003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228d0c2640 1 --2- 172.21.15.38:0/2278911610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274004800 0x7f2274004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228d0c2640 1 --2- 172.21.15.38:0/2278911610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22740051c0 0x7f227400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22880b74e0 con 0x7f2274004800 2024-04-25T08:42:33.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227400c160 con 0x7f22740051c0 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227400c360 con 0x7f22740039b0 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228d8c3640 1 --2- 172.21.15.38:0/2278911610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274004800 0x7f2274004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228e0c4640 1 --2- 172.21.15.38:0/2278911610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22740039b0 0x7f2274003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228e8c5640 1 --2- 172.21.15.38:0/2278911610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22740051c0 0x7f227400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228d8c3640 1 -- 172.21.15.38:0/2278911610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22740039b0 msgr2=0x7f2274003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228d8c3640 1 --2- 172.21.15.38:0/2278911610 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22740039b0 0x7f2274003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228d8c3640 1 -- 172.21.15.38:0/2278911610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22740051c0 msgr2=0x7f227400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228d8c3640 1 --2- 172.21.15.38:0/2278911610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22740051c0 0x7f227400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228d8c3640 1 -- 172.21.15.38:0/2278911610 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274003520 con 0x7f2274004800 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228e8c5640 1 --2- 172.21.15.38:0/2278911610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22740051c0 0x7f227400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.657+0000 7f228e8c5640 1 -- 172.21.15.38:0/2278911610 reap_dead start 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.658+0000 7f228d8c3640 1 --2- 172.21.15.38:0/2278911610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274004800 0x7f2274004c80 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f2278009ac0 tx=0x7f2274005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ae9d7acd10e053 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:33.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.658+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278036070 con 0x7f2274004800 2024-04-25T08:42:33.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.658+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2278031400 con 0x7f2274004800 2024-04-25T08:42:33.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.658+0000 7f228d0c2640 1 -- 172.21.15.38:0/2278911610 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278009690 con 0x7f2274004800 2024-04-25T08:42:33.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.658+0000 7f2290328640 1 -- 172.21.15.38:0/2278911610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274004800 msgr2=0x7f2274004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.658+0000 7f2290328640 1 --2- 172.21.15.38:0/2278911610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274004800 0x7f2274004c80 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7f2278009ac0 tx=0x7f2274005830 comp rx=0 tx=0).stop 2024-04-25T08:42:33.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.658+0000 7f2290328640 1 -- 172.21.15.38:0/2278911610 shutdown_connections 2024-04-25T08:42:33.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.658+0000 7f2290328640 1 --2- 172.21.15.38:0/2278911610 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2274004800 0x7f2274004c80 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.658+0000 7f2290328640 1 -- 172.21.15.38:0/2278911610 >> 172.21.15.38:0/2278911610 conn(0x7f2288066d10 msgr2=0x7f2288067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:33.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.658+0000 7f2290328640 1 -- 172.21.15.38:0/2278911610 shutdown_connections 2024-04-25T08:42:33.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.658+0000 7f2290328640 1 -- 172.21.15.38:0/2278911610 wait complete. 2024-04-25T08:42:33.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.660+0000 7f2290328640 1 Processor -- start 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.660+0000 7f2290328640 1 -- start start 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.660+0000 7f2290328640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2274004800 0x7f22880c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.660+0000 7f2290328640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228815fae0 0x7f228815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.660+0000 7f2290328640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2288152470 0x7f22881528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.660+0000 7f2290328640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22880b6a00 con 0x7f228815fae0 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.660+0000 7f2290328640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22880b7d80 con 0x7f2288152470 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.660+0000 7f2290328640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22880b6b80 con 0x7f2274004800 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.660+0000 7f228d8c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228815fae0 0x7f228815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f228d8c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228815fae0 0x7f228815ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40518/0 (socket says 172.21.15.38:40518) 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f228d8c3640 1 -- 172.21.15.38:0/3805338199 learned_addr learned my addr 172.21.15.38:0/3805338199 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f228e0c4640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2274004800 0x7f22880c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f228e8c5640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2288152470 0x7f22881528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f228d8c3640 1 -- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2274004800 msgr2=0x7f22880c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f228d8c3640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2274004800 0x7f22880c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f228d8c3640 1 -- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2288152470 msgr2=0x7f22881528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f228d8c3640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2288152470 0x7f22881528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f228d8c3640 1 -- 172.21.15.38:0/3805338199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288151160 con 0x7f228815fae0 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f228e8c5640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2288152470 0x7f22881528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f228d8c3640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228815fae0 0x7f228815ff60 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f227802abb0 tx=0x7f2278004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f227effd640 1 -- 172.21.15.38:0/3805338199 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227803f020 con 0x7f228815fae0 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f227effd640 1 -- 172.21.15.38:0/3805338199 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2278037040 con 0x7f228815fae0 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f2290328640 1 -- 172.21.15.38:0/3805338199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2288153050 con 0x7f228815fae0 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f227effd640 1 -- 172.21.15.38:0/3805338199 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278036070 con 0x7f228815fae0 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.661+0000 7f2290328640 1 -- 172.21.15.38:0/3805338199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228815d3f0 con 0x7f228815fae0 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.662+0000 7f2290328640 1 -- 172.21.15.38:0/3805338199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2288068e80 con 0x7f228815fae0 2024-04-25T08:42:33.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.663+0000 7f227effd640 1 -- 172.21.15.38:0/3805338199 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2278042020 con 0x7f228815fae0 2024-04-25T08:42:33.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.663+0000 7f227effd640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f226c06b4d0 0x7f226c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:33.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.663+0000 7f227effd640 1 -- 172.21.15.38:0/3805338199 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6332+0+0 (secure 0 0 0) 0x7f22780aa460 con 0x7f228815fae0 2024-04-25T08:42:33.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.663+0000 7f228e0c4640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f226c06b4d0 0x7f226c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:33.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.664+0000 7f228e0c4640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f226c06b4d0 0x7f226c06d990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2284000c00 tx=0x7f2284009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:33.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.667+0000 7f227effd640 1 -- 172.21.15.38:0/3805338199 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2278049030 con 0x7f228815fae0 2024-04-25T08:42:33.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:33.852+0000 7f2290328640 1 -- 172.21.15.38:0/3805338199 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2288154f90 con 0x7f228815fae0 2024-04-25T08:42:34.817 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.824+0000 7f227effd640 1 -- 172.21.15.38:0/3805338199 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2288151160 con 0x7f228815fae0 2024-04-25T08:42:34.817 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:42:34.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.827+0000 7f2290328640 1 -- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f226c06b4d0 msgr2=0x7f226c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.827+0000 7f2290328640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f226c06b4d0 0x7f226c06d990 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2284000c00 tx=0x7f2284009040 comp rx=0 tx=0).stop 2024-04-25T08:42:34.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.827+0000 7f2290328640 1 -- 172.21.15.38:0/3805338199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228815fae0 msgr2=0x7f228815ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.827+0000 7f2290328640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228815fae0 0x7f228815ff60 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f227802abb0 tx=0x7f2278004730 comp rx=0 tx=0).stop 2024-04-25T08:42:34.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.827+0000 7f2290328640 1 -- 172.21.15.38:0/3805338199 shutdown_connections 2024-04-25T08:42:34.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.827+0000 7f2290328640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2288152470 0x7f22881528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.827+0000 7f2290328640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f228815fae0 0x7f228815ff60 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.827+0000 7f2290328640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f226c06b4d0 0x7f226c06d990 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.827+0000 7f2290328640 1 --2- 172.21.15.38:0/3805338199 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2274004800 0x7f22880c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.827+0000 7f2290328640 1 -- 172.21.15.38:0/3805338199 >> 172.21.15.38:0/3805338199 conn(0x7f2288066d10 msgr2=0x7f22880d3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:34.820 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.827+0000 7f2290328640 1 -- 172.21.15.38:0/3805338199 shutdown_connections 2024-04-25T08:42:34.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.827+0000 7f2290328640 1 -- 172.21.15.38:0/3805338199 wait complete. 2024-04-25T08:42:34.833 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:42:34.861 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:42:34.878 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:42:34.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.953+0000 7fc5ea2cb640 1 Processor -- start 2024-04-25T08:42:34.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.953+0000 7fc5ea2cb640 1 -- start start 2024-04-25T08:42:34.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5ea2cb640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fc5e40d0860 0x7fc5e40d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5ea2cb640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5e40d19e0 0x7fc5e40d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5ea2cb640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e40d1180 con 0x7fc5e40d3860 2024-04-25T08:42:34.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5ea2cb640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e415e2b0 con 0x7fc5e40d19e0 2024-04-25T08:42:34.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5ea2cb640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e40d24a0 con 0x7fc5e40d0860 2024-04-25T08:42:34.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5e37fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc5e40d19e0 0x7fc5e40d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5e8868640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fc5e40d3860 0x7fc5e40d3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40546/0 (socket says 172.21.15.38:40546) 2024-04-25T08:42:34.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5e8868640 1 -- 172.21.15.38:0/1710788496 learned_addr learned my addr 172.21.15.38:0/1710788496 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:34.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5e3fff640 1 --2- 172.21.15.38:0/1710788496 >> v2:172.21.15.107:3300/0 conn(0x7fc5e40d0860 0x7fc5e40d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5e37fe640 1 -- 172.21.15.38:0/1710788496 >> v2:172.21.15.107:3300/0 conn(0x7fc5e40d0860 msgr2=0x7fc5e40d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:34.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5e37fe640 1 --2- 172.21.15.38:0/1710788496 >> v2:172.21.15.107:3300/0 conn(0x7fc5e40d0860 0x7fc5e40d0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5e37fe640 1 -- 172.21.15.38:0/1710788496 >> v1:172.21.15.38:6789/0 conn(0x7fc5e40d3860 legacy=0x7fc5e40d3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5e37fe640 1 -- 172.21.15.38:0/1710788496 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e40d2780 con 0x7fc5e40d19e0 2024-04-25T08:42:34.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5e2ffd640 1 -- 172.21.15.38:0/1710788496 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 341594803 0 0) 0x7fc5e40d1180 con 0x7fc5e40d3860 2024-04-25T08:42:34.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.954+0000 7fc5e3fff640 1 --2- 172.21.15.38:0/1710788496 >> v2:172.21.15.107:3300/0 conn(0x7fc5e40d0860 0x7fc5e40d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.955+0000 7fc5e37fe640 1 --2- 172.21.15.38:0/1710788496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40d19e0 0x7fc5e40d1e50 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40ce120 tx=0x7fc5d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25da4a26250b8b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.955+0000 7fc5e2ffd640 1 -- 172.21.15.38:0/1710788496 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4037070 con 0x7fc5e40d19e0 2024-04-25T08:42:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.955+0000 7fc5e2ffd640 1 -- 172.21.15.38:0/1710788496 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d4032440 con 0x7fc5e40d19e0 2024-04-25T08:42:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.955+0000 7fc5e2ffd640 1 -- 172.21.15.38:0/1710788496 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4032810 con 0x7fc5e40d19e0 2024-04-25T08:42:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.956+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/1710788496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40d19e0 msgr2=0x7fc5e40d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.956+0000 7fc5ea2cb640 1 --2- 172.21.15.38:0/1710788496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40d19e0 0x7fc5e40d1e50 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7fc5e40ce120 tx=0x7fc5d402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.956+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/1710788496 shutdown_connections 2024-04-25T08:42:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.956+0000 7fc5ea2cb640 1 --2- 172.21.15.38:0/1710788496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40d19e0 0x7fc5e40d1e50 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.956+0000 7fc5ea2cb640 1 --2- 172.21.15.38:0/1710788496 >> v2:172.21.15.107:3300/0 conn(0x7fc5e40d0860 0x7fc5e40d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.956+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/1710788496 >> 172.21.15.38:0/1710788496 conn(0x7fc5e4066d10 msgr2=0x7fc5e4067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:34.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.956+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/1710788496 shutdown_connections 2024-04-25T08:42:34.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.956+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/1710788496 wait complete. 2024-04-25T08:42:34.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.957+0000 7fc5ea2cb640 1 Processor -- start 2024-04-25T08:42:34.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.957+0000 7fc5ea2cb640 1 -- start start 2024-04-25T08:42:34.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.958+0000 7fc5ea2cb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e40d0860 0x7fc5e41f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.958+0000 7fc5ea2cb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e40d19e0 0x7fc5e414ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.958+0000 7fc5ea2cb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40d3860 0x7fc5e41587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.958+0000 7fc5ea2cb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e40d3540 con 0x7fc5e40d3860 2024-04-25T08:42:34.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.958+0000 7fc5ea2cb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e40d2d40 con 0x7fc5e40d19e0 2024-04-25T08:42:34.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.958+0000 7fc5ea2cb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e40b9770 con 0x7fc5e40d0860 2024-04-25T08:42:34.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.958+0000 7fc5e8868640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40d3860 0x7fc5e41587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.958+0000 7fc5e8868640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40d3860 0x7fc5e41587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40538/0 (socket says 172.21.15.38:40538) 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.958+0000 7fc5e8868640 1 -- 172.21.15.38:0/2515955546 learned_addr learned my addr 172.21.15.38:0/2515955546 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.958+0000 7fc5e3fff640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e40d0860 0x7fc5e41f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.958+0000 7fc5e37fe640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e40d19e0 0x7fc5e414ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.959+0000 7fc5e8868640 1 -- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e40d0860 msgr2=0x7fc5e41f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.959+0000 7fc5e8868640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e40d0860 0x7fc5e41f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.959+0000 7fc5e8868640 1 -- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e40d19e0 msgr2=0x7fc5e414ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.959+0000 7fc5e8868640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e40d19e0 0x7fc5e414ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.959+0000 7fc5e8868640 1 -- 172.21.15.38:0/2515955546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e40c74a0 con 0x7fc5e40d3860 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.959+0000 7fc5e8868640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40d3860 0x7fc5e41587c0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00c700 tx=0x7fc5dc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.959+0000 7fc5e0ff9640 1 -- 172.21.15.38:0/2515955546 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc018070 con 0x7fc5e40d3860 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.959+0000 7fc5e0ff9640 1 -- 172.21.15.38:0/2515955546 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5dc00ce10 con 0x7fc5e40d3860 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.959+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/2515955546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e40c7740 con 0x7fc5e40d3860 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.959+0000 7fc5e0ff9640 1 -- 172.21.15.38:0/2515955546 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc00a860 con 0x7fc5e40d3860 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.959+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/2515955546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e4159210 con 0x7fc5e40d3860 2024-04-25T08:42:34.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.960+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/2515955546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e40c7910 con 0x7fc5e40d3860 2024-04-25T08:42:34.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.961+0000 7fc5e0ff9640 1 -- 172.21.15.38:0/2515955546 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5dc01b070 con 0x7fc5e40d3860 2024-04-25T08:42:34.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.961+0000 7fc5e0ff9640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5c406b500 0x7fc5c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:34.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.961+0000 7fc5e0ff9640 1 -- 172.21.15.38:0/2515955546 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 5894+0+0 (secure 0 0 0) 0x7fc5dc089740 con 0x7fc5e40d3860 2024-04-25T08:42:34.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.961+0000 7fc5e3fff640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5c406b500 0x7fc5c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:34.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.962+0000 7fc5e3fff640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5c406b500 0x7fc5c406d9c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc006fd0 tx=0x7fc5cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:34.957 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:34.965+0000 7fc5e0ff9640 1 -- 172.21.15.38:0/2515955546 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5dc058590 con 0x7fc5e40d3860 2024-04-25T08:42:35.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.150+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/2515955546 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc5e40b5f80 con 0x7fc5e40d3860 2024-04-25T08:42:35.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.832+0000 7fc5e0ff9640 1 -- 172.21.15.38:0/2515955546 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc5e40b5f80 con 0x7fc5e40d3860 2024-04-25T08:42:35.825 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-04-25T08:42:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.836+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5c406b500 msgr2=0x7fc5c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.836+0000 7fc5ea2cb640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5c406b500 0x7fc5c406d9c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc006fd0 tx=0x7fc5cc008040 comp rx=0 tx=0).stop 2024-04-25T08:42:35.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.837+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/2515955546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40d3860 msgr2=0x7fc5e41587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.837+0000 7fc5ea2cb640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40d3860 0x7fc5e41587c0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc00c700 tx=0x7fc5dc00cbc0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.837+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/2515955546 shutdown_connections 2024-04-25T08:42:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.837+0000 7fc5ea2cb640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5e40d3860 0x7fc5e41587c0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.837+0000 7fc5ea2cb640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc5e40d19e0 0x7fc5e414ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.837+0000 7fc5ea2cb640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc5c406b500 0x7fc5c406d9c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.837+0000 7fc5ea2cb640 1 --2- 172.21.15.38:0/2515955546 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc5e40d0860 0x7fc5e41f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.837+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/2515955546 >> 172.21.15.38:0/2515955546 conn(0x7fc5e4066d10 msgr2=0x7fc5e40ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.837+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/2515955546 shutdown_connections 2024-04-25T08:42:35.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:35.838+0000 7fc5ea2cb640 1 -- 172.21.15.38:0/2515955546 wait complete. 2024-04-25T08:42:35.849 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:42:36.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.035+0000 7f9be9ea3640 1 Processor -- start 2024-04-25T08:42:36.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.035+0000 7f9be9ea3640 1 -- start start 2024-04-25T08:42:36.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.036+0000 7f9be9ea3640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f9be40d0320 0x7f9be40d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.036+0000 7f9be9ea3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be40d0700 0x7f9be40d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.036+0000 7f9be9ea3640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be40d2330 con 0x7f9be40d1e90 2024-04-25T08:42:36.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.036+0000 7f9be9ea3640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be40d2610 con 0x7f9be40d0320 2024-04-25T08:42:36.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.036+0000 7f9be9ea3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40d28f0 con 0x7f9be40d0700 2024-04-25T08:42:36.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.036+0000 7f9be3fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f9be40d1e90 0x7f9be40b5390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40556/0 (socket says 172.21.15.38:40556) 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.036+0000 7f9be3fff640 1 -- 172.21.15.38:0/180468523 learned_addr learned my addr 172.21.15.38:0/180468523 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.036+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653047653 0 0) 0x7f9be40d2330 con 0x7f9be40d1e90 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.036+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bcc003520 con 0x7f9be40d1e90 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.036+0000 7f9be2ffd640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be40d0700 0x7f9be40d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.036+0000 7f9be37fe640 1 --2- 172.21.15.38:0/180468523 >> v2:172.21.15.107:3300/0 conn(0x7f9be40d0320 0x7f9be40d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3799438975 0 0) 0x7f9bcc003520 con 0x7f9be40d1e90 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be40d0700 msgr2=0x7f9be40d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be40d0700 0x7f9be40d1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 >> v2:172.21.15.107:3300/0 conn(0x7f9be40d0320 msgr2=0x7f9be40d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 --2- 172.21.15.38:0/180468523 >> v2:172.21.15.107:3300/0 conn(0x7f9be40d0320 0x7f9be40d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be40d2bd0 con 0x7f9be40d1e90 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f9bdc0019f0 con 0x7f9be40d1e90 2024-04-25T08:42:36.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 >> v1:172.21.15.38:6789/0 conn(0x7f9be40d1e90 legacy=0x7f9be40b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bcc0039b0 0x7f9bcc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bcc004800 0x7f9bcc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be40d0700 0x7f9bcc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40d2610 con 0x7f9bcc0039b0 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40d28f0 con 0x7f9bcc004800 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc00b9c0 con 0x7f9be40d0700 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be37fe640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bcc0039b0 0x7f9bcc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be2ffd640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bcc004800 0x7f9bcc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be3fff640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be40d0700 0x7f9bcc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be37fe640 1 -- 172.21.15.38:0/180468523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be40d0700 msgr2=0x7f9bcc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be37fe640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be40d0700 0x7f9bcc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be37fe640 1 -- 172.21.15.38:0/180468523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bcc004800 msgr2=0x7f9bcc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be37fe640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bcc004800 0x7f9bcc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be37fe640 1 -- 172.21.15.38:0/180468523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bcc003520 con 0x7f9bcc0039b0 2024-04-25T08:42:36.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.037+0000 7f9be2ffd640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bcc004800 0x7f9bcc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:36.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.038+0000 7f9be37fe640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bcc0039b0 0x7f9bcc003dd0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009ac0 tx=0x7f9bcc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35d439ff2932103 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:36.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.038+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4036070 con 0x7f9bcc0039b0 2024-04-25T08:42:36.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.038+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd4031400 con 0x7f9bcc0039b0 2024-04-25T08:42:36.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.038+0000 7f9be27fc640 1 -- 172.21.15.38:0/180468523 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4009690 con 0x7f9bcc0039b0 2024-04-25T08:42:36.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.038+0000 7f9be9ea3640 1 -- 172.21.15.38:0/180468523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bcc0039b0 msgr2=0x7f9bcc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.038+0000 7f9be9ea3640 1 --2- 172.21.15.38:0/180468523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9bcc0039b0 0x7f9bcc003dd0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009ac0 tx=0x7f9bcc005830 comp rx=0 tx=0).stop 2024-04-25T08:42:36.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.039+0000 7f9be3fff640 1 -- 172.21.15.38:0/180468523 reap_dead start 2024-04-25T08:42:36.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.039+0000 7f9be9ea3640 1 -- 172.21.15.38:0/180468523 shutdown_connections 2024-04-25T08:42:36.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.039+0000 7f9be9ea3640 1 -- 172.21.15.38:0/180468523 >> 172.21.15.38:0/180468523 conn(0x7f9be40668f0 msgr2=0x7f9be4066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:36.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.039+0000 7f9be9ea3640 1 -- 172.21.15.38:0/180468523 shutdown_connections 2024-04-25T08:42:36.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.039+0000 7f9be9ea3640 1 -- 172.21.15.38:0/180468523 wait complete. 2024-04-25T08:42:36.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.040+0000 7f9be9ea3640 1 Processor -- start 2024-04-25T08:42:36.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.040+0000 7f9be9ea3640 1 -- start start 2024-04-25T08:42:36.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be9ea3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bcc0039b0 0x7f9be40abbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be9ea3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9be415b120 0x7f9be40ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be9ea3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be415f6f0 0x7f9be415fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be9ea3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be40d0b40 con 0x7f9be415b120 2024-04-25T08:42:36.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be9ea3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be4161210 con 0x7f9bcc0039b0 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be9ea3640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41f34e0 con 0x7f9be415f6f0 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9be415b120 0x7f9be40ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9be415b120 0x7f9be40ac0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40560/0 (socket says 172.21.15.38:40560) 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be2ffd640 1 -- 172.21.15.38:0/3531767889 learned_addr learned my addr 172.21.15.38:0/3531767889 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be37fe640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bcc0039b0 0x7f9be40abbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be3fff640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be415f6f0 0x7f9be415fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be2ffd640 1 -- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be415f6f0 msgr2=0x7f9be415fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be2ffd640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be415f6f0 0x7f9be415fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be2ffd640 1 -- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bcc0039b0 msgr2=0x7f9be40abbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be2ffd640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bcc0039b0 0x7f9be40abbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be2ffd640 1 -- 172.21.15.38:0/3531767889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be41f37a0 con 0x7f9be415b120 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be2ffd640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9be415b120 0x7f9be40ac0f0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8011a70 tx=0x7f9bd8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:36.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9bcbfff640 1 -- 172.21.15.38:0/3531767889 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd800faf0 con 0x7f9be415b120 2024-04-25T08:42:36.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9bcbfff640 1 -- 172.21.15.38:0/3531767889 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd800fc90 con 0x7f9be415b120 2024-04-25T08:42:36.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9bcbfff640 1 -- 172.21.15.38:0/3531767889 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd8014440 con 0x7f9be415b120 2024-04-25T08:42:36.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be9ea3640 1 -- 172.21.15.38:0/3531767889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be41f3a80 con 0x7f9be415b120 2024-04-25T08:42:36.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.041+0000 7f9be9ea3640 1 -- 172.21.15.38:0/3531767889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be41f4040 con 0x7f9be415b120 2024-04-25T08:42:36.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.042+0000 7f9be9ea3640 1 -- 172.21.15.38:0/3531767889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be4068e80 con 0x7f9be415b120 2024-04-25T08:42:36.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.043+0000 7f9bcbfff640 1 -- 172.21.15.38:0/3531767889 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bd8012040 con 0x7f9be415b120 2024-04-25T08:42:36.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.043+0000 7f9bcbfff640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9bd006b500 0x7f9bd006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.043+0000 7f9bcbfff640 1 -- 172.21.15.38:0/3531767889 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9bd808a6c0 con 0x7f9be415b120 2024-04-25T08:42:36.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.044+0000 7f9be37fe640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9bd006b500 0x7f9bd006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.044+0000 7f9be37fe640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9bd006b500 0x7f9bd006d9c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4033000 tx=0x7f9bd4008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:36.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.048+0000 7f9bcbfff640 1 -- 172.21.15.38:0/3531767889 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9bd8019030 con 0x7f9be415b120 2024-04-25T08:42:36.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.229+0000 7f9be9ea3640 1 -- 172.21.15.38:0/3531767889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9be40b6d20 con 0x7f9be415b120 2024-04-25T08:42:36.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.232+0000 7f9bcbfff640 1 -- 172.21.15.38:0/3531767889 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9bd80596b0 con 0x7f9be415b120 2024-04-25T08:42:36.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.235+0000 7f9be9ea3640 1 -- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9bd006b500 msgr2=0x7f9bd006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.235+0000 7f9be9ea3640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9bd006b500 0x7f9bd006d9c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4033000 tx=0x7f9bd4008030 comp rx=0 tx=0).stop 2024-04-25T08:42:36.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.235+0000 7f9be9ea3640 1 -- 172.21.15.38:0/3531767889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9be415b120 msgr2=0x7f9be40ac0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.235+0000 7f9be9ea3640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9be415b120 0x7f9be40ac0f0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8011a70 tx=0x7f9bd8011e70 comp rx=0 tx=0).stop 2024-04-25T08:42:36.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.235+0000 7f9be9ea3640 1 -- 172.21.15.38:0/3531767889 shutdown_connections 2024-04-25T08:42:36.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.235+0000 7f9be9ea3640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9bd006b500 0x7f9bd006d9c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.235+0000 7f9be9ea3640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9be415f6f0 0x7f9be415fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.235+0000 7f9be9ea3640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9be415b120 0x7f9be40ac0f0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.235+0000 7f9be9ea3640 1 --2- 172.21.15.38:0/3531767889 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9bcc0039b0 0x7f9be40abbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.235+0000 7f9be9ea3640 1 -- 172.21.15.38:0/3531767889 >> 172.21.15.38:0/3531767889 conn(0x7f9be40668f0 msgr2=0x7f9be40d1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:36.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.235+0000 7f9be9ea3640 1 -- 172.21.15.38:0/3531767889 shutdown_connections 2024-04-25T08:42:36.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.235+0000 7f9be9ea3640 1 -- 172.21.15.38:0/3531767889 wait complete. 2024-04-25T08:42:36.238 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T08:42:36.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.347+0000 7f6c45a9e640 1 Processor -- start 2024-04-25T08:42:36.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.347+0000 7f6c45a9e640 1 -- start start 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.348+0000 7f6c45a9e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6c400b5710 0x7f6c40161700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.348+0000 7f6c45a9e640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6c4015ddf0 0x7f6c400b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.348+0000 7f6c45a9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c400b5af0 0x7f6c4006bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.348+0000 7f6c45a9e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c400b9930 con 0x7f6c400b5710 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.348+0000 7f6c45a9e640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c40161e00 con 0x7f6c4015ddf0 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.348+0000 7f6c45a9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c401620e0 con 0x7f6c400b5af0 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.348+0000 7f6c3f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6c400b5710 0x7f6c40161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.348+0000 7f6c3f7fe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6c400b5710 0x7f6c40161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40562/0 (socket says 172.21.15.38:40562) 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.348+0000 7f6c3f7fe640 1 -- 172.21.15.38:0/1499632161 learned_addr learned my addr 172.21.15.38:0/1499632161 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.348+0000 7f6c3ffff640 1 --2- 172.21.15.38:0/1499632161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c400b5af0 0x7f6c4006bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.348+0000 7f6c3effd640 1 --2- 172.21.15.38:0/1499632161 >> v2:172.21.15.107:3300/0 conn(0x7f6c4015ddf0 0x7f6c400b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.348+0000 7f6c3f7fe640 1 -- 172.21.15.38:0/1499632161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c400b5af0 msgr2=0x7f6c4006bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:36.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.349+0000 7f6c3f7fe640 1 --2- 172.21.15.38:0/1499632161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c400b5af0 0x7f6c4006bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.349+0000 7f6c3f7fe640 1 -- 172.21.15.38:0/1499632161 >> v2:172.21.15.107:3300/0 conn(0x7f6c4015ddf0 msgr2=0x7f6c400b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:36.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.349+0000 7f6c3f7fe640 1 --2- 172.21.15.38:0/1499632161 >> v2:172.21.15.107:3300/0 conn(0x7f6c4015ddf0 0x7f6c400b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.349+0000 7f6c3f7fe640 1 -- 172.21.15.38:0/1499632161 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c401623c0 con 0x7f6c400b5710 2024-04-25T08:42:36.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.349+0000 7f6c3ffff640 1 --2- 172.21.15.38:0/1499632161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c400b5af0 0x7f6c4006bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:36.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.349+0000 7f6c3f7fe640 1 --2- 172.21.15.38:0/1499632161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c400b5710 0x7f6c40161700 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f6c400d0300 tx=0x7f6c3002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a651cf8a0a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:36.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.349+0000 7f6c3e7fc640 1 -- 172.21.15.38:0/1499632161 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30037070 con 0x7f6c400b5710 2024-04-25T08:42:36.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.349+0000 7f6c3e7fc640 1 -- 172.21.15.38:0/1499632161 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c30032440 con 0x7f6c400b5710 2024-04-25T08:42:36.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.349+0000 7f6c3e7fc640 1 -- 172.21.15.38:0/1499632161 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30032810 con 0x7f6c400b5710 2024-04-25T08:42:36.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.350+0000 7f6c45a9e640 1 -- 172.21.15.38:0/1499632161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c400b5710 msgr2=0x7f6c40161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.350+0000 7f6c45a9e640 1 --2- 172.21.15.38:0/1499632161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c400b5710 0x7f6c40161700 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f6c400d0300 tx=0x7f6c3002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.350+0000 7f6c45a9e640 1 -- 172.21.15.38:0/1499632161 shutdown_connections 2024-04-25T08:42:36.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.350+0000 7f6c45a9e640 1 --2- 172.21.15.38:0/1499632161 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c400b5af0 0x7f6c4006bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.350+0000 7f6c45a9e640 1 --2- 172.21.15.38:0/1499632161 >> v2:172.21.15.107:3300/0 conn(0x7f6c4015ddf0 0x7f6c400b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.350+0000 7f6c45a9e640 1 --2- 172.21.15.38:0/1499632161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c400b5710 0x7f6c40161700 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.350+0000 7f6c45a9e640 1 -- 172.21.15.38:0/1499632161 >> 172.21.15.38:0/1499632161 conn(0x7f6c400d07c0 msgr2=0x7f6c400d0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:36.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.350+0000 7f6c45a9e640 1 -- 172.21.15.38:0/1499632161 shutdown_connections 2024-04-25T08:42:36.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.350+0000 7f6c45a9e640 1 -- 172.21.15.38:0/1499632161 wait complete. 2024-04-25T08:42:36.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.351+0000 7f6c45a9e640 1 Processor -- start 2024-04-25T08:42:36.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.351+0000 7f6c45a9e640 1 -- start start 2024-04-25T08:42:36.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.352+0000 7f6c45a9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c400b5710 0x7f6c40158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.352+0000 7f6c45a9e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c400b5af0 0x7f6c401f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.352+0000 7f6c45a9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4015ddf0 0x7f6c400c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.352+0000 7f6c45a9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4006c6b0 con 0x7f6c400b5710 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.352+0000 7f6c45a9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4006c530 con 0x7f6c4015ddf0 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.352+0000 7f6c45a9e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4006c830 con 0x7f6c400b5af0 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.352+0000 7f6c3f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c400b5710 0x7f6c40158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.352+0000 7f6c3f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c400b5710 0x7f6c40158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40568/0 (socket says 172.21.15.38:40568) 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.352+0000 7f6c3f7fe640 1 -- 172.21.15.38:0/4074209608 learned_addr learned my addr 172.21.15.38:0/4074209608 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c3effd640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c400b5af0 0x7f6c401f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c3ffff640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4015ddf0 0x7f6c400c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c3f7fe640 1 -- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c400b5af0 msgr2=0x7f6c401f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c3f7fe640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c400b5af0 0x7f6c401f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c3f7fe640 1 -- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4015ddf0 msgr2=0x7f6c400c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c3f7fe640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4015ddf0 0x7f6c400c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c3f7fe640 1 -- 172.21.15.38:0/4074209608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c40158f90 con 0x7f6c400b5710 2024-04-25T08:42:36.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c3f7fe640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c400b5710 0x7f6c40158a50 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f6c300097c0 tx=0x7f6c300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:36.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c27fff640 1 -- 172.21.15.38:0/4074209608 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30037040 con 0x7f6c400b5710 2024-04-25T08:42:36.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c27fff640 1 -- 172.21.15.38:0/4074209608 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c30042070 con 0x7f6c400b5710 2024-04-25T08:42:36.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c45a9e640 1 -- 172.21.15.38:0/4074209608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c401f7f10 con 0x7f6c400b5710 2024-04-25T08:42:36.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c27fff640 1 -- 172.21.15.38:0/4074209608 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30030880 con 0x7f6c400b5710 2024-04-25T08:42:36.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.353+0000 7f6c45a9e640 1 -- 172.21.15.38:0/4074209608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c400c7660 con 0x7f6c400b5710 2024-04-25T08:42:36.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.354+0000 7f6c45a9e640 1 -- 172.21.15.38:0/4074209608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c400c7450 con 0x7f6c400b5710 2024-04-25T08:42:36.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.354+0000 7f6c27fff640 1 -- 172.21.15.38:0/4074209608 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c30030a20 con 0x7f6c400b5710 2024-04-25T08:42:36.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.355+0000 7f6c27fff640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6c2c06b4d0 0x7f6c2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:36.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.355+0000 7f6c27fff640 1 -- 172.21.15.38:0/4074209608 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6c300aa960 con 0x7f6c400b5710 2024-04-25T08:42:36.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.355+0000 7f6c3effd640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6c2c06b4d0 0x7f6c2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:36.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.356+0000 7f6c3effd640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6c2c06b4d0 0x7f6c2c06d990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6c4015cef0 tx=0x7f6c34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:36.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.359+0000 7f6c27fff640 1 -- 172.21.15.38:0/4074209608 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c300798d0 con 0x7f6c400b5710 2024-04-25T08:42:36.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.608+0000 7f6c45a9e640 1 -- 172.21.15.38:0/4074209608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f6c40068e80 con 0x7f6c400b5710 2024-04-25T08:42:36.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.834+0000 7f6c27fff640 1 -- 172.21.15.38:0/4074209608 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v536) v1 ==== 159+0+0 (secure 0 0 0) 0x7f6c300791d0 con 0x7f6c400b5710 2024-04-25T08:42:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.836+0000 7f6c45a9e640 1 -- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6c2c06b4d0 msgr2=0x7f6c2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.836+0000 7f6c45a9e640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6c2c06b4d0 0x7f6c2c06d990 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6c4015cef0 tx=0x7f6c34008040 comp rx=0 tx=0).stop 2024-04-25T08:42:36.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.836+0000 7f6c45a9e640 1 -- 172.21.15.38:0/4074209608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c400b5710 msgr2=0x7f6c40158a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.836+0000 7f6c45a9e640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c400b5710 0x7f6c40158a50 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7f6c300097c0 tx=0x7f6c300046c0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.836+0000 7f6c45a9e640 1 -- 172.21.15.38:0/4074209608 shutdown_connections 2024-04-25T08:42:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.836+0000 7f6c45a9e640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c4015ddf0 0x7f6c400c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.836+0000 7f6c45a9e640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6c2c06b4d0 0x7f6c2c06d990 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.836+0000 7f6c45a9e640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c400b5af0 0x7f6c401f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.836+0000 7f6c45a9e640 1 --2- 172.21.15.38:0/4074209608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c400b5710 0x7f6c40158a50 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.836+0000 7f6c45a9e640 1 -- 172.21.15.38:0/4074209608 >> 172.21.15.38:0/4074209608 conn(0x7f6c400d07c0 msgr2=0x7f6c400b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.836+0000 7f6c45a9e640 1 -- 172.21.15.38:0/4074209608 shutdown_connections 2024-04-25T08:42:36.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:36.836+0000 7f6c45a9e640 1 -- 172.21.15.38:0/4074209608 wait complete. 2024-04-25T08:42:36.840 INFO:tasks.cephfs_test_runner:test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ERROR 2024-04-25T08:42:36.841 INFO:tasks.cephfs_test_runner: 2024-04-25T08:42:36.841 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T08:42:36.841 INFO:tasks.cephfs_test_runner:ERROR: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:42:36.841 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T08:42:36.841 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T08:42:36.841 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs/test_scrub_checks.py", line 190, in test_scrub_when_mds_is_inactive 2024-04-25T08:42:36.841 INFO:tasks.cephfs_test_runner: sr_mds_id = self.get_daemon_names('up:standby-replay', status=status)[0] 2024-04-25T08:42:36.841 INFO:tasks.cephfs_test_runner:AttributeError: 'TestScrubControls' object has no attribute 'get_daemon_names' 2024-04-25T08:42:36.841 INFO:tasks.cephfs_test_runner: 2024-04-25T08:42:36.841 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T08:42:36.841 INFO:tasks.cephfs_test_runner:Ran 9 tests in 621.048s 2024-04-25T08:42:36.841 INFO:tasks.cephfs_test_runner: 2024-04-25T08:42:36.842 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-25T08:42:36.842 INFO:tasks.cephfs_test_runner: 2024-04-25T08:42:36.842 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T08:42:36.842 INFO:tasks.cephfs_test_runner:ERROR: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:42:36.842 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T08:42:36.842 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T08:42:36.842 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs/test_scrub_checks.py", line 190, in test_scrub_when_mds_is_inactive 2024-04-25T08:42:36.842 INFO:tasks.cephfs_test_runner: sr_mds_id = self.get_daemon_names('up:standby-replay', status=status)[0] 2024-04-25T08:42:36.842 INFO:tasks.cephfs_test_runner:AttributeError: 'TestScrubControls' object has no attribute 'get_daemon_names' 2024-04-25T08:42:36.842 INFO:tasks.cephfs_test_runner: 2024-04-25T08:42:36.843 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:42:37.047 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5b4f2221d1fb4d3f8a680dfcf1fcbe98 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T08:42:37.048 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T08:42:37.057 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T08:42:37.065 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T08:42:37.065 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:42:37.065 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:37.088 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:42:37.088 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-04-25T08:42:37.088 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:37.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=158 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:42:37.150 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:42:37.150 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:37.169 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:42:37.169 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-04-25T08:42:37.169 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:42:37.227 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056066,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19963 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248812k,nr_inodes=812203,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497624k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:42:37.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:37.228 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-04-25T08:42:37.246 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:42:37.247 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:42:37.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:37.247 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:42:37.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:37.317 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:42:37.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:37.385 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-04-25T08:42:37.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:37.480 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-04-25T08:42:37.498 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T08:42:37.498 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T08:42:37.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:37.499 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:42:37.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:37.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:42:37.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:42:37.635 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-04-25T08:42:37.733 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T08:42:37.744 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T08:42:37.744 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:42:37.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.861+0000 7f4adf6dd640 1 Processor -- start 2024-04-25T08:42:37.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.861+0000 7f4adf6dd640 1 -- start start 2024-04-25T08:42:37.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4adf6dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ad815dc10 0x7f4ad80ce820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4adf6dd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4ad80d0f60 0x7f4ad80b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4adf6dd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad80d3e70 con 0x7f4ad80ced60 2024-04-25T08:42:37.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4adf6dd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad815dff0 con 0x7f4ad80d0f60 2024-04-25T08:42:37.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4adf6dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80b6990 con 0x7f4ad815dc10 2024-04-25T08:42:37.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4addc7a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4ad80d0f60 0x7f4ad80b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4addc7a640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f4ad80d0f60 0x7f4ad80b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40572/0 (socket says 172.21.15.38:40572) 2024-04-25T08:42:37.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4addc7a640 1 -- 172.21.15.38:0/451464865 learned_addr learned my addr 172.21.15.38:0/451464865 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:37.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4add479640 1 --2- 172.21.15.38:0/451464865 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ad815dc10 0x7f4ad80ce820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4addc7a640 1 -- 172.21.15.38:0/451464865 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ad815dc10 msgr2=0x7f4ad80ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:37.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4addc7a640 1 --2- 172.21.15.38:0/451464865 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ad815dc10 0x7f4ad80ce820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4addc7a640 1 -- 172.21.15.38:0/451464865 >> v1:172.21.15.107:6789/0 conn(0x7f4ad80ced60 legacy=0x7f4ad80d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4addc7a640 1 -- 172.21.15.38:0/451464865 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad80d4150 con 0x7f4ad80d0f60 2024-04-25T08:42:37.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.862+0000 7f4acffff640 1 -- 172.21.15.38:0/451464865 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686171895 0 0) 0x7f4ad80d3e70 con 0x7f4ad80ced60 2024-04-25T08:42:37.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.863+0000 7f4addc7a640 1 --2- 172.21.15.38:0/451464865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ad80d0f60 0x7f4ad80b2370 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009e50 tx=0x7f4ad002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a09c60e474271c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:37.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.863+0000 7f4acffff640 1 -- 172.21.15.38:0/451464865 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad0036070 con 0x7f4ad80d0f60 2024-04-25T08:42:37.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.863+0000 7f4acffff640 1 -- 172.21.15.38:0/451464865 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad0031600 con 0x7f4ad80d0f60 2024-04-25T08:42:37.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.863+0000 7f4acffff640 1 -- 172.21.15.38:0/451464865 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad00319d0 con 0x7f4ad80d0f60 2024-04-25T08:42:37.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.864+0000 7f4adf6dd640 1 -- 172.21.15.38:0/451464865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ad80d0f60 msgr2=0x7f4ad80b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.864+0000 7f4adf6dd640 1 --2- 172.21.15.38:0/451464865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ad80d0f60 0x7f4ad80b2370 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0009e50 tx=0x7f4ad002a680 comp rx=0 tx=0).stop 2024-04-25T08:42:37.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.864+0000 7f4adf6dd640 1 -- 172.21.15.38:0/451464865 shutdown_connections 2024-04-25T08:42:37.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.864+0000 7f4adf6dd640 1 --2- 172.21.15.38:0/451464865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ad80d0f60 0x7f4ad80b2370 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.864+0000 7f4adf6dd640 1 --2- 172.21.15.38:0/451464865 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ad815dc10 0x7f4ad80ce820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.864+0000 7f4adf6dd640 1 -- 172.21.15.38:0/451464865 >> 172.21.15.38:0/451464865 conn(0x7f4ad8066d10 msgr2=0x7f4ad8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:37.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.864+0000 7f4adf6dd640 1 -- 172.21.15.38:0/451464865 shutdown_connections 2024-04-25T08:42:37.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.864+0000 7f4adf6dd640 1 -- 172.21.15.38:0/451464865 wait complete. 2024-04-25T08:42:37.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.865+0000 7f4adf6dd640 1 Processor -- start 2024-04-25T08:42:37.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.865+0000 7f4adf6dd640 1 -- start start 2024-04-25T08:42:37.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4adf6dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ad80ced60 0x7f4ad81f2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4adf6dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad80d0f60 0x7f4ad80b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4adf6dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ad815dc10 0x7f4ad80b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4adf6dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80b6bc0 con 0x7f4ad80ced60 2024-04-25T08:42:37.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4adf6dd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80d4710 con 0x7f4ad80d0f60 2024-04-25T08:42:37.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4adf6dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad80d3900 con 0x7f4ad815dc10 2024-04-25T08:42:37.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4add479640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ad80ced60 0x7f4ad81f2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4add479640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ad80ced60 0x7f4ad81f2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40576/0 (socket says 172.21.15.38:40576) 2024-04-25T08:42:37.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4add479640 1 -- 172.21.15.38:0/524909963 learned_addr learned my addr 172.21.15.38:0/524909963 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:37.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4addc7a640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ad815dc10 0x7f4ad80b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4adcc78640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad80d0f60 0x7f4ad80b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4add479640 1 -- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ad815dc10 msgr2=0x7f4ad80b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4add479640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ad815dc10 0x7f4ad80b4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4add479640 1 -- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad80d0f60 msgr2=0x7f4ad80b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:37.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4add479640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad80d0f60 0x7f4ad80b46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:37.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.866+0000 7f4add479640 1 -- 172.21.15.38:0/524909963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad814e520 con 0x7f4ad80ced60 2024-04-25T08:42:37.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.867+0000 7f4add479640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ad80ced60 0x7f4ad81f2960 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400e870 tx=0x7f4ad400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:37.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.867+0000 7f4acdffb640 1 -- 172.21.15.38:0/524909963 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4017070 con 0x7f4ad80ced60 2024-04-25T08:42:37.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.867+0000 7f4acdffb640 1 -- 172.21.15.38:0/524909963 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad400c940 con 0x7f4ad80ced60 2024-04-25T08:42:37.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.867+0000 7f4acdffb640 1 -- 172.21.15.38:0/524909963 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad400cd30 con 0x7f4ad80ced60 2024-04-25T08:42:37.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.867+0000 7f4adf6dd640 1 -- 172.21.15.38:0/524909963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad814b640 con 0x7f4ad80ced60 2024-04-25T08:42:37.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.867+0000 7f4adf6dd640 1 -- 172.21.15.38:0/524909963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad814bc00 con 0x7f4ad80ced60 2024-04-25T08:42:37.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.868+0000 7f4adf6dd640 1 -- 172.21.15.38:0/524909963 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad8068e80 con 0x7f4ad80ced60 2024-04-25T08:42:37.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.868+0000 7f4acdffb640 1 -- 172.21.15.38:0/524909963 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ad401a070 con 0x7f4ad80ced60 2024-04-25T08:42:37.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.869+0000 7f4acdffb640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4abc06b500 0x7f4abc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:37.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.869+0000 7f4acdffb640 1 -- 172.21.15.38:0/524909963 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4ad4089e70 con 0x7f4ad80ced60 2024-04-25T08:42:37.865 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.869+0000 7f4adcc78640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4abc06b500 0x7f4abc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:37.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.869+0000 7f4adcc78640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4abc06b500 0x7f4abc06d9c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8151c40 tx=0x7f4ac8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:37.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:37.873+0000 7f4acdffb640 1 -- 172.21.15.38:0/524909963 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ad4058ef0 con 0x7f4ad80ced60 2024-04-25T08:42:38.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.050+0000 7f4adf6dd640 1 -- 172.21.15.38:0/524909963 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ad80d2be0 con 0x7f4abc06b500 2024-04-25T08:42:38.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.051+0000 7f4acdffb640 1 -- 172.21.15.38:0/524909963 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13540 (secure 0 0 0) 0x7f4ad80d2be0 con 0x7f4abc06b500 2024-04-25T08:42:38.044 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:38.044 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:42:38.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.054+0000 7f4adf6dd640 1 -- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4abc06b500 msgr2=0x7f4abc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.054+0000 7f4adf6dd640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4abc06b500 0x7f4abc06d9c0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8151c40 tx=0x7f4ac8009040 comp rx=0 tx=0).stop 2024-04-25T08:42:38.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.054+0000 7f4adf6dd640 1 -- 172.21.15.38:0/524909963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ad80ced60 msgr2=0x7f4ad81f2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.054+0000 7f4adf6dd640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ad80ced60 0x7f4ad81f2960 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f4ad400e870 tx=0x7f4ad400ed30 comp rx=0 tx=0).stop 2024-04-25T08:42:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.054+0000 7f4adf6dd640 1 -- 172.21.15.38:0/524909963 shutdown_connections 2024-04-25T08:42:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.054+0000 7f4adf6dd640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4abc06b500 0x7f4abc06d9c0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.054+0000 7f4adf6dd640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ad815dc10 0x7f4ad80b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.054+0000 7f4adf6dd640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ad80d0f60 0x7f4ad80b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.054+0000 7f4adf6dd640 1 --2- 172.21.15.38:0/524909963 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ad80ced60 0x7f4ad81f2960 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.054+0000 7f4adf6dd640 1 -- 172.21.15.38:0/524909963 >> 172.21.15.38:0/524909963 conn(0x7f4ad8066d10 msgr2=0x7f4ad815e170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:38.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.054+0000 7f4adf6dd640 1 -- 172.21.15.38:0/524909963 shutdown_connections 2024-04-25T08:42:38.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.054+0000 7f4adf6dd640 1 -- 172.21.15.38:0/524909963 wait complete. 2024-04-25T08:42:38.057 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-04-25T08:42:37.360806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":506660,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":490803,"kb_avail":754468060,"statfs":{"total":773094113280,"available":772575293440,"internally_reserved":0,"allocated":16097280,"data_stored":8160952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":502582832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1077149593,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":-12,"num_read_kb":-3336,"num_write":-1122,"num_write_kb":-1052045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-50,"num_bytes_recovered":-52428800,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001299"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":411,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T08:42:35.897303+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:42:35.897303+0000","last_peered":"2024-04-25T08:42:35.897303+0000","last_clean":"2024-04-25T08:42:35.897303+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:42:35.897303+0000","last_undegraded":"2024-04-25T08:42:35.897303+0000","last_fullsized":"2024-04-25T08:42:35.897303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69676,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302164,"statfs":{"total":96636764160,"available":96565415936,"internally_reserved":0,"allocated":2457600,"data_stored":1462895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46892,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324948,"statfs":{"total":96636764160,"available":96588746752,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59368,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574986,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73256,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298584,"statfs":{"total":96636764160,"available":96561750016,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77424,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294416,"statfs":{"total":96636764160,"available":96557481984,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46892,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324948,"statfs":{"total":96636764160,"available":96588746752,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59376,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312464,"statfs":{"total":96636764160,"available":96575963136,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73776,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298064,"statfs":{"total":96636764160,"available":96561217536,"internally_reserved":0,"allocated":2457600,"data_stored":1462895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:42:38.058 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:42:38.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.166+0000 7f4bae5e2640 1 Processor -- start 2024-04-25T08:42:38.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.166+0000 7f4bae5e2640 1 -- start start 2024-04-25T08:42:38.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4bae5e2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba815dc40 0x7f4ba80d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4bae5e2640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4ba80d0e10 0x7f4ba80b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4bae5e2640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba80b8400 con 0x7f4ba80b9580 2024-04-25T08:42:38.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4bae5e2640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba80d1260 con 0x7f4ba80d0e10 2024-04-25T08:42:38.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4bae5e2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80d19b0 con 0x7f4ba815dc40 2024-04-25T08:42:38.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4ba77fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f4ba80b9580 0x7f4ba80d1e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40560/0 (socket says 172.21.15.38:40560) 2024-04-25T08:42:38.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4ba77fe640 1 -- 172.21.15.38:0/3570356422 learned_addr learned my addr 172.21.15.38:0/3570356422 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:38.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4ba7fff640 1 --2- 172.21.15.38:0/3570356422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba815dc40 0x7f4ba80d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4bacb7f640 1 --2- 172.21.15.38:0/3570356422 >> v2:172.21.15.107:3300/0 conn(0x7f4ba80d0e10 0x7f4ba80b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4ba6ffd640 1 -- 172.21.15.38:0/3570356422 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347994567 0 0) 0x7f4ba80b8400 con 0x7f4ba80b9580 2024-04-25T08:42:38.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4ba6ffd640 1 -- 172.21.15.38:0/3570356422 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b90003520 con 0x7f4ba80b9580 2024-04-25T08:42:38.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4ba7fff640 1 -- 172.21.15.38:0/3570356422 >> v2:172.21.15.107:3300/0 conn(0x7f4ba80d0e10 msgr2=0x7f4ba80b7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:38.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4ba7fff640 1 --2- 172.21.15.38:0/3570356422 >> v2:172.21.15.107:3300/0 conn(0x7f4ba80d0e10 0x7f4ba80b7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4ba7fff640 1 -- 172.21.15.38:0/3570356422 >> v1:172.21.15.38:6789/0 conn(0x7f4ba80b9580 legacy=0x7f4ba80d1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.167+0000 7f4ba7fff640 1 -- 172.21.15.38:0/3570356422 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba80b86e0 con 0x7f4ba815dc40 2024-04-25T08:42:38.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.168+0000 7f4ba7fff640 1 --2- 172.21.15.38:0/3570356422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba815dc40 0x7f4ba80d08d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0097a0 tx=0x7f4b9c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a0d1d455dfd192 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:38.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.168+0000 7f4ba6ffd640 1 -- 172.21.15.38:0/3570356422 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c037070 con 0x7f4ba815dc40 2024-04-25T08:42:38.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.168+0000 7f4ba6ffd640 1 -- 172.21.15.38:0/3570356422 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9c032440 con 0x7f4ba815dc40 2024-04-25T08:42:38.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.168+0000 7f4ba6ffd640 1 -- 172.21.15.38:0/3570356422 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c0327f0 con 0x7f4ba815dc40 2024-04-25T08:42:38.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.169+0000 7f4bae5e2640 1 -- 172.21.15.38:0/3570356422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba815dc40 msgr2=0x7f4ba80d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.169+0000 7f4bae5e2640 1 --2- 172.21.15.38:0/3570356422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba815dc40 0x7f4ba80d08d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0097a0 tx=0x7f4b9c02a180 comp rx=0 tx=0).stop 2024-04-25T08:42:38.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.169+0000 7f4bae5e2640 1 -- 172.21.15.38:0/3570356422 shutdown_connections 2024-04-25T08:42:38.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.169+0000 7f4bae5e2640 1 --2- 172.21.15.38:0/3570356422 >> v2:172.21.15.107:3300/0 conn(0x7f4ba80d0e10 0x7f4ba80b7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.169+0000 7f4bae5e2640 1 --2- 172.21.15.38:0/3570356422 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba815dc40 0x7f4ba80d08d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.169+0000 7f4bae5e2640 1 -- 172.21.15.38:0/3570356422 >> 172.21.15.38:0/3570356422 conn(0x7f4ba8066ce0 msgr2=0x7f4ba80670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:38.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.169+0000 7f4bae5e2640 1 -- 172.21.15.38:0/3570356422 shutdown_connections 2024-04-25T08:42:38.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.169+0000 7f4bae5e2640 1 -- 172.21.15.38:0/3570356422 wait complete. 2024-04-25T08:42:38.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.170+0000 7f4bae5e2640 1 Processor -- start 2024-04-25T08:42:38.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.170+0000 7f4bae5e2640 1 -- start start 2024-04-25T08:42:38.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4bae5e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba80b9580 0x7f4ba8152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4bae5e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba80d0e10 0x7f4ba81538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4bae5e2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba815dc40 0x7f4ba8153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4bae5e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80b7fb0 con 0x7f4ba80d0e10 2024-04-25T08:42:38.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4bae5e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80b8ca0 con 0x7f4ba80b9580 2024-04-25T08:42:38.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4bae5e2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80b8130 con 0x7f4ba815dc40 2024-04-25T08:42:38.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4ba77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba80d0e10 0x7f4ba81538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4ba77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba80d0e10 0x7f4ba81538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40588/0 (socket says 172.21.15.38:40588) 2024-04-25T08:42:38.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4ba77fe640 1 -- 172.21.15.38:0/2432090071 learned_addr learned my addr 172.21.15.38:0/2432090071 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:38.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4bacb7f640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba815dc40 0x7f4ba8153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4ba7fff640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba80b9580 0x7f4ba8152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4ba77fe640 1 -- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba815dc40 msgr2=0x7f4ba8153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4ba77fe640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba815dc40 0x7f4ba8153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4ba77fe640 1 -- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba80b9580 msgr2=0x7f4ba8152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.171+0000 7f4ba77fe640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba80b9580 0x7f4ba8152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.172+0000 7f4ba77fe640 1 -- 172.21.15.38:0/2432090071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba80c91a0 con 0x7f4ba80d0e10 2024-04-25T08:42:38.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.172+0000 7f4ba7fff640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba80b9580 0x7f4ba8152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:38.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.172+0000 7f4ba77fe640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba80d0e10 0x7f4ba81538a0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800c430 tx=0x7f4b9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:38.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.172+0000 7f4ba4ff9640 1 -- 172.21.15.38:0/2432090071 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98017070 con 0x7f4ba80d0e10 2024-04-25T08:42:38.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.172+0000 7f4ba4ff9640 1 -- 172.21.15.38:0/2432090071 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b98005ce0 con 0x7f4ba80d0e10 2024-04-25T08:42:38.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.172+0000 7f4bae5e2640 1 -- 172.21.15.38:0/2432090071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba80c9480 con 0x7f4ba80d0e10 2024-04-25T08:42:38.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.172+0000 7f4ba4ff9640 1 -- 172.21.15.38:0/2432090071 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b980072f0 con 0x7f4ba80d0e10 2024-04-25T08:42:38.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.172+0000 7f4bae5e2640 1 -- 172.21.15.38:0/2432090071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba80c9a40 con 0x7f4ba80d0e10 2024-04-25T08:42:38.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.173+0000 7f4bae5e2640 1 -- 172.21.15.38:0/2432090071 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba8068e80 con 0x7f4ba80d0e10 2024-04-25T08:42:38.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.173+0000 7f4ba4ff9640 1 -- 172.21.15.38:0/2432090071 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b9801a070 con 0x7f4ba80d0e10 2024-04-25T08:42:38.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.174+0000 7f4ba4ff9640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b9406b4d0 0x7f4b9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.174+0000 7f4ba4ff9640 1 -- 172.21.15.38:0/2432090071 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4b980894e0 con 0x7f4ba80d0e10 2024-04-25T08:42:38.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.174+0000 7f4ba7fff640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b9406b4d0 0x7f4b9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.174+0000 7f4ba7fff640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b9406b4d0 0x7f4b9406d990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c034000 tx=0x7f4b9c02e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:38.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.178+0000 7f4ba4ff9640 1 -- 172.21.15.38:0/2432090071 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b98058560 con 0x7f4ba80d0e10 2024-04-25T08:42:38.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.355+0000 7f4bae5e2640 1 -- 172.21.15.38:0/2432090071 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ba80b4af0 con 0x7f4b9406b4d0 2024-04-25T08:42:38.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.356+0000 7f4ba4ff9640 1 -- 172.21.15.38:0/2432090071 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13540 (secure 0 0 0) 0x7f4ba80b4af0 con 0x7f4b9406b4d0 2024-04-25T08:42:38.348 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:38.349 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:42:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.358+0000 7f4bae5e2640 1 -- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b9406b4d0 msgr2=0x7f4b9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.358+0000 7f4bae5e2640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b9406b4d0 0x7f4b9406d990 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c034000 tx=0x7f4b9c02e000 comp rx=0 tx=0).stop 2024-04-25T08:42:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.358+0000 7f4bae5e2640 1 -- 172.21.15.38:0/2432090071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba80d0e10 msgr2=0x7f4ba81538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.358+0000 7f4bae5e2640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba80d0e10 0x7f4ba81538a0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f4b9800c430 tx=0x7f4b9800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.358+0000 7f4bae5e2640 1 -- 172.21.15.38:0/2432090071 shutdown_connections 2024-04-25T08:42:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.358+0000 7f4bae5e2640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b9406b4d0 0x7f4b9406d990 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.358+0000 7f4bae5e2640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4ba815dc40 0x7f4ba8153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.358+0000 7f4bae5e2640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4ba80d0e10 0x7f4ba81538a0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.358+0000 7f4bae5e2640 1 --2- 172.21.15.38:0/2432090071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4ba80b9580 0x7f4ba8152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.358+0000 7f4bae5e2640 1 -- 172.21.15.38:0/2432090071 >> 172.21.15.38:0/2432090071 conn(0x7f4ba8066ce0 msgr2=0x7f4ba806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:38.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.358+0000 7f4bae5e2640 1 -- 172.21.15.38:0/2432090071 shutdown_connections 2024-04-25T08:42:38.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.358+0000 7f4bae5e2640 1 -- 172.21.15.38:0/2432090071 wait complete. 2024-04-25T08:42:38.361 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-04-25T08:42:37.360806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":506660,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":490803,"kb_avail":754468060,"statfs":{"total":773094113280,"available":772575293440,"internally_reserved":0,"allocated":16097280,"data_stored":8160952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":502582832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1077149593,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":-12,"num_read_kb":-3336,"num_write":-1122,"num_write_kb":-1052045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-50,"num_bytes_recovered":-52428800,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001299"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":411,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T08:42:35.897303+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:42:35.897303+0000","last_peered":"2024-04-25T08:42:35.897303+0000","last_clean":"2024-04-25T08:42:35.897303+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:42:35.897303+0000","last_undegraded":"2024-04-25T08:42:35.897303+0000","last_fullsized":"2024-04-25T08:42:35.897303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69676,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302164,"statfs":{"total":96636764160,"available":96565415936,"internally_reserved":0,"allocated":2457600,"data_stored":1462895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46892,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324948,"statfs":{"total":96636764160,"available":96588746752,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59368,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574986,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73256,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298584,"statfs":{"total":96636764160,"available":96561750016,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77424,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294416,"statfs":{"total":96636764160,"available":96557481984,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46892,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324948,"statfs":{"total":96636764160,"available":96588746752,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59376,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312464,"statfs":{"total":96636764160,"available":96575963136,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73776,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298064,"statfs":{"total":96636764160,"available":96561217536,"internally_reserved":0,"allocated":2457600,"data_stored":1462895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:42:38.362 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T08:42:38.362 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:42:38.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.471+0000 7f49320cb640 1 Processor -- start 2024-04-25T08:42:38.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.471+0000 7f49320cb640 1 -- start start 2024-04-25T08:42:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.472+0000 7f49320cb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f492c0ce2e0 0x7f492c0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.472+0000 7f49320cb640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c0ceff0 con 0x7f492c15df50 2024-04-25T08:42:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.472+0000 7f49320cb640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c0d1f70 con 0x7f492c06c110 2024-04-25T08:42:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.472+0000 7f49320cb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0cf2d0 con 0x7f492c0ce2e0 2024-04-25T08:42:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.472+0000 7f492affd640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f492c15df50 0x7f492c0ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:40562/0 (socket says 172.21.15.38:40562) 2024-04-25T08:42:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.472+0000 7f492affd640 1 -- 172.21.15.38:0/2508195066 learned_addr learned my addr 172.21.15.38:0/2508195066 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:38.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.472+0000 7f492bfff640 1 --2- 172.21.15.38:0/2508195066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f492c0ce2e0 0x7f492c0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.472+0000 7f492a7fc640 1 -- 172.21.15.38:0/2508195066 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636895548 0 0) 0x7f492c0ceff0 con 0x7f492c15df50 2024-04-25T08:42:38.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.472+0000 7f492a7fc640 1 -- 172.21.15.38:0/2508195066 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4914003520 con 0x7f492c15df50 2024-04-25T08:42:38.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.473+0000 7f492a7fc640 1 -- 172.21.15.38:0/2508195066 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283855389 0 0) 0x7f492c0d1f70 con 0x7f492c06c110 2024-04-25T08:42:38.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.473+0000 7f492a7fc640 1 -- 172.21.15.38:0/2508195066 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f492c0ceff0 con 0x7f492c06c110 2024-04-25T08:42:38.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.473+0000 7f492bfff640 1 -- 172.21.15.38:0/2508195066 >> v1:172.21.15.107:6789/0 conn(0x7f492c06c110 legacy=0x7f492c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.473+0000 7f492bfff640 1 -- 172.21.15.38:0/2508195066 >> v1:172.21.15.38:6789/0 conn(0x7f492c15df50 legacy=0x7f492c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.473+0000 7f492a7fc640 1 -- 172.21.15.38:0/2508195066 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1268404624 0 0) 0x7f4914003520 con 0x7f492c15df50 2024-04-25T08:42:38.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.473+0000 7f492bfff640 1 -- 172.21.15.38:0/2508195066 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c0d2250 con 0x7f492c0ce2e0 2024-04-25T08:42:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.473+0000 7f492bfff640 1 --2- 172.21.15.38:0/2508195066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f492c0ce2e0 0x7f492c0d2a50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4924009dc0 tx=0x7f492402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3560c9567502308e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.474+0000 7f492a7fc640 1 -- 172.21.15.38:0/2508195066 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920001670 con 0x7f492c0ce2e0 2024-04-25T08:42:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.474+0000 7f492a7fc640 1 -- 172.21.15.38:0/2508195066 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4924007540 con 0x7f492c0ce2e0 2024-04-25T08:42:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.474+0000 7f492a7fc640 1 -- 172.21.15.38:0/2508195066 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924007910 con 0x7f492c0ce2e0 2024-04-25T08:42:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.474+0000 7f49320cb640 1 -- 172.21.15.38:0/2508195066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f492c0ce2e0 msgr2=0x7f492c0d2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.474+0000 7f49320cb640 1 --2- 172.21.15.38:0/2508195066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f492c0ce2e0 0x7f492c0d2a50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4924009dc0 tx=0x7f492402a660 comp rx=0 tx=0).stop 2024-04-25T08:42:38.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.474+0000 7f49320cb640 1 -- 172.21.15.38:0/2508195066 shutdown_connections 2024-04-25T08:42:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.474+0000 7f49320cb640 1 --2- 172.21.15.38:0/2508195066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f492c0ce2e0 0x7f492c0d2a50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.474+0000 7f49320cb640 1 -- 172.21.15.38:0/2508195066 >> 172.21.15.38:0/2508195066 conn(0x7f492c0d3fb0 msgr2=0x7f492c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.474+0000 7f49320cb640 1 -- 172.21.15.38:0/2508195066 shutdown_connections 2024-04-25T08:42:38.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.474+0000 7f49320cb640 1 -- 172.21.15.38:0/2508195066 wait complete. 2024-04-25T08:42:38.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.475+0000 7f49320cb640 1 Processor -- start 2024-04-25T08:42:38.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.476+0000 7f49320cb640 1 -- start start 2024-04-25T08:42:38.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.476+0000 7f49320cb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f492c06c110 0x7f492c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.476+0000 7f49320cb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c0ce2e0 0x7f492c15e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.476+0000 7f49320cb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f492c15df50 0x7f492c0c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.476+0000 7f49320cb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0d3110 con 0x7f492c15df50 2024-04-25T08:42:38.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.476+0000 7f49320cb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0d2f90 con 0x7f492c0ce2e0 2024-04-25T08:42:38.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.476+0000 7f49320cb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c0d3290 con 0x7f492c06c110 2024-04-25T08:42:38.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.476+0000 7f492bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f492c15df50 0x7f492c0c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.476+0000 7f492bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f492c15df50 0x7f492c0c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40604/0 (socket says 172.21.15.38:40604) 2024-04-25T08:42:38.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.476+0000 7f492bfff640 1 -- 172.21.15.38:0/1420053825 learned_addr learned my addr 172.21.15.38:0/1420053825 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:38.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f492b7fe640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f492c06c110 0x7f492c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f492affd640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c0ce2e0 0x7f492c15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f492bfff640 1 -- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f492c06c110 msgr2=0x7f492c155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f492bfff640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f492c06c110 0x7f492c155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f492bfff640 1 -- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c0ce2e0 msgr2=0x7f492c15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f492bfff640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c0ce2e0 0x7f492c15e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f492bfff640 1 -- 172.21.15.38:0/1420053825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c0c7fe0 con 0x7f492c15df50 2024-04-25T08:42:38.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f492affd640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c0ce2e0 0x7f492c15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:38.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f492bfff640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f492c15df50 0x7f492c0c7420 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f492402ac50 tx=0x7f4924004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:38.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f4913fff640 1 -- 172.21.15.38:0/1420053825 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924040070 con 0x7f492c15df50 2024-04-25T08:42:38.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f4913fff640 1 -- 172.21.15.38:0/1420053825 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492403c830 con 0x7f492c15df50 2024-04-25T08:42:38.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f49320cb640 1 -- 172.21.15.38:0/1420053825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c0cb1b0 con 0x7f492c15df50 2024-04-25T08:42:38.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f4913fff640 1 -- 172.21.15.38:0/1420053825 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492403cbe0 con 0x7f492c15df50 2024-04-25T08:42:38.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.477+0000 7f49320cb640 1 -- 172.21.15.38:0/1420053825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c0cb770 con 0x7f492c15df50 2024-04-25T08:42:38.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.478+0000 7f49320cb640 1 -- 172.21.15.38:0/1420053825 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c068e80 con 0x7f492c15df50 2024-04-25T08:42:38.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.479+0000 7f4913fff640 1 -- 172.21.15.38:0/1420053825 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4924039040 con 0x7f492c15df50 2024-04-25T08:42:38.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.479+0000 7f4913fff640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f491806b4d0 0x7f491806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.479+0000 7f4913fff640 1 -- 172.21.15.38:0/1420053825 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f49240ab720 con 0x7f492c15df50 2024-04-25T08:42:38.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.479+0000 7f492b7fe640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f491806b4d0 0x7f491806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.480+0000 7f492b7fe640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f491806b4d0 0x7f491806d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f491c0011e0 tx=0x7f491c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:38.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.483+0000 7f4913fff640 1 -- 172.21.15.38:0/1420053825 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49240443c0 con 0x7f492c15df50 2024-04-25T08:42:38.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.660+0000 7f49320cb640 1 -- 172.21.15.38:0/1420053825 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f492c067dd0 con 0x7f491806b4d0 2024-04-25T08:42:38.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.661+0000 7f4913fff640 1 -- 172.21.15.38:0/1420053825 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13540 (secure 0 0 0) 0x7f492c067dd0 con 0x7f491806b4d0 2024-04-25T08:42:38.653 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:38.654 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.663+0000 7f49320cb640 1 -- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f491806b4d0 msgr2=0x7f491806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.663+0000 7f49320cb640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f491806b4d0 0x7f491806d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f491c0011e0 tx=0x7f491c009040 comp rx=0 tx=0).stop 2024-04-25T08:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.663+0000 7f49320cb640 1 -- 172.21.15.38:0/1420053825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f492c15df50 msgr2=0x7f492c0c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.663+0000 7f49320cb640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f492c15df50 0x7f492c0c7420 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f492402ac50 tx=0x7f4924004a40 comp rx=0 tx=0).stop 2024-04-25T08:42:38.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.663+0000 7f49320cb640 1 -- 172.21.15.38:0/1420053825 shutdown_connections 2024-04-25T08:42:38.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.663+0000 7f49320cb640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f492c15df50 0x7f492c0c7420 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.663+0000 7f49320cb640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c0ce2e0 0x7f492c15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.663+0000 7f49320cb640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f491806b4d0 0x7f491806d990 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.663+0000 7f49320cb640 1 --2- 172.21.15.38:0/1420053825 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f492c06c110 0x7f492c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.663+0000 7f49320cb640 1 -- 172.21.15.38:0/1420053825 >> 172.21.15.38:0/1420053825 conn(0x7f492c0d3fb0 msgr2=0x7f492c15d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:38.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.663+0000 7f49320cb640 1 -- 172.21.15.38:0/1420053825 shutdown_connections 2024-04-25T08:42:38.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.663+0000 7f49320cb640 1 -- 172.21.15.38:0/1420053825 wait complete. 2024-04-25T08:42:38.667 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-04-25T08:42:37.360806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":110,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":506660,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":490803,"kb_avail":754468060,"statfs":{"total":773094113280,"available":772575293440,"internally_reserved":0,"allocated":16097280,"data_stored":8160952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":502582832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1077149593,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":-12,"num_read_kb":-3336,"num_write":-1122,"num_write_kb":-1052045,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-50,"num_bytes_recovered":-52428800,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001299"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":411,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T08:42:35.897303+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:42:35.897303+0000","last_peered":"2024-04-25T08:42:35.897303+0000","last_clean":"2024-04-25T08:42:35.897303+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:42:35.897303+0000","last_undegraded":"2024-04-25T08:42:35.897303+0000","last_fullsized":"2024-04-25T08:42:35.897303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574987,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69676,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302164,"statfs":{"total":96636764160,"available":96565415936,"internally_reserved":0,"allocated":2457600,"data_stored":1462895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46892,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324948,"statfs":{"total":96636764160,"available":96588746752,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59368,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574986,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73256,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298584,"statfs":{"total":96636764160,"available":96561750016,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77424,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294416,"statfs":{"total":96636764160,"available":96557481984,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574987,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46892,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324948,"statfs":{"total":96636764160,"available":96588746752,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59376,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312464,"statfs":{"total":96636764160,"available":96575963136,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73776,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298064,"statfs":{"total":96636764160,"available":96561217536,"internally_reserved":0,"allocated":2457600,"data_stored":1462895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:42:38.667 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:42:38.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.776+0000 7f557df77640 1 Processor -- start 2024-04-25T08:42:38.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.776+0000 7f557df77640 1 -- start start 2024-04-25T08:42:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f557df77640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f557815da90 0x7f5578161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f557df77640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f55780d0850 0x7f55780d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f557df77640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5578162160 con 0x7f5578161c50 2024-04-25T08:42:38.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f557df77640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55780d19b0 con 0x7f55780d0850 2024-04-25T08:42:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f557df77640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55780d1f70 con 0x7f557815da90 2024-04-25T08:42:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f5577fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f55780d0850 0x7f55780d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f5577fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f55780d0850 0x7f55780d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40614/0 (socket says 172.21.15.38:40614) 2024-04-25T08:42:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f5577fff640 1 -- 172.21.15.38:0/3531162623 learned_addr learned my addr 172.21.15.38:0/3531162623 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f55777fe640 1 --2- 172.21.15.38:0/3531162623 >> v2:172.21.15.107:3300/0 conn(0x7f557815da90 0x7f5578161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f5577fff640 1 -- 172.21.15.38:0/3531162623 >> v2:172.21.15.107:3300/0 conn(0x7f557815da90 msgr2=0x7f5578161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f5577fff640 1 --2- 172.21.15.38:0/3531162623 >> v2:172.21.15.107:3300/0 conn(0x7f557815da90 0x7f5578161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f5577fff640 1 -- 172.21.15.38:0/3531162623 >> v1:172.21.15.107:6789/0 conn(0x7f5578161c50 legacy=0x7f55780d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.777+0000 7f5577fff640 1 -- 172.21.15.38:0/3531162623 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5578162440 con 0x7f55780d0850 2024-04-25T08:42:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.778+0000 7f5577fff640 1 --2- 172.21.15.38:0/3531162623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55780d0850 0x7f55780d2400 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f557806d2f0 tx=0x7f557002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1ec09c9b7727ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.778+0000 7f55767fc640 1 -- 172.21.15.38:0/3531162623 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570036070 con 0x7f55780d0850 2024-04-25T08:42:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.778+0000 7f55767fc640 1 -- 172.21.15.38:0/3531162623 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5570031600 con 0x7f55780d0850 2024-04-25T08:42:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.778+0000 7f55767fc640 1 -- 172.21.15.38:0/3531162623 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700319d0 con 0x7f55780d0850 2024-04-25T08:42:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.779+0000 7f557df77640 1 -- 172.21.15.38:0/3531162623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55780d0850 msgr2=0x7f55780d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.779+0000 7f557df77640 1 --2- 172.21.15.38:0/3531162623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55780d0850 0x7f55780d2400 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f557806d2f0 tx=0x7f557002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.779+0000 7f557df77640 1 -- 172.21.15.38:0/3531162623 shutdown_connections 2024-04-25T08:42:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.779+0000 7f557df77640 1 --2- 172.21.15.38:0/3531162623 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55780d0850 0x7f55780d2400 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.779+0000 7f557df77640 1 --2- 172.21.15.38:0/3531162623 >> v2:172.21.15.107:3300/0 conn(0x7f557815da90 0x7f5578161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.779+0000 7f557df77640 1 -- 172.21.15.38:0/3531162623 >> 172.21.15.38:0/3531162623 conn(0x7f557806c060 msgr2=0x7f557806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.779+0000 7f557df77640 1 -- 172.21.15.38:0/3531162623 shutdown_connections 2024-04-25T08:42:38.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.779+0000 7f557df77640 1 -- 172.21.15.38:0/3531162623 wait complete. 2024-04-25T08:42:38.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.780+0000 7f557df77640 1 Processor -- start 2024-04-25T08:42:38.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.780+0000 7f557df77640 1 -- start start 2024-04-25T08:42:38.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f557df77640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55780d0850 0x7f55780c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f557df77640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557815da90 0x7f5578151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f557df77640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5578161c50 0x7f557815e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f557df77640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55780b9400 con 0x7f5578161c50 2024-04-25T08:42:38.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f557df77640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557815de70 con 0x7f55780d0850 2024-04-25T08:42:38.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f557df77640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55780c8a80 con 0x7f557815da90 2024-04-25T08:42:38.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f5577fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5578161c50 0x7f557815e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f5577fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5578161c50 0x7f557815e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40630/0 (socket says 172.21.15.38:40630) 2024-04-25T08:42:38.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f5577fff640 1 -- 172.21.15.38:0/3071183094 learned_addr learned my addr 172.21.15.38:0/3071183094 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:38.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f55777fe640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55780d0850 0x7f55780c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f5576ffd640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557815da90 0x7f5578151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f5577fff640 1 -- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557815da90 msgr2=0x7f5578151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f5577fff640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557815da90 0x7f5578151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f5577fff640 1 -- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55780d0850 msgr2=0x7f55780c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f5577fff640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55780d0850 0x7f55780c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f5577fff640 1 -- 172.21.15.38:0/3071183094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55780c74a0 con 0x7f5578161c50 2024-04-25T08:42:38.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f55777fe640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55780d0850 0x7f55780c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:38.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f5577fff640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5578161c50 0x7f557815e1b0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f557002ac90 tx=0x7f5570004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:38.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f555ffff640 1 -- 172.21.15.38:0/3071183094 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557003f020 con 0x7f5578161c50 2024-04-25T08:42:38.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f555ffff640 1 -- 172.21.15.38:0/3071183094 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5570037040 con 0x7f5578161c50 2024-04-25T08:42:38.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f557df77640 1 -- 172.21.15.38:0/3071183094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55780c7780 con 0x7f5578161c50 2024-04-25T08:42:38.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.781+0000 7f555ffff640 1 -- 172.21.15.38:0/3071183094 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5570036070 con 0x7f5578161c50 2024-04-25T08:42:38.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.782+0000 7f557df77640 1 -- 172.21.15.38:0/3071183094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55780c7d40 con 0x7f5578161c50 2024-04-25T08:42:38.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.782+0000 7f557df77640 1 -- 172.21.15.38:0/3071183094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5578068e80 con 0x7f5578161c50 2024-04-25T08:42:38.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.783+0000 7f555ffff640 1 -- 172.21.15.38:0/3071183094 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5570042020 con 0x7f5578161c50 2024-04-25T08:42:38.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.783+0000 7f555ffff640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f556406b4d0 0x7f556406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:38.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.783+0000 7f555ffff640 1 -- 172.21.15.38:0/3071183094 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f55700aa5a0 con 0x7f5578161c50 2024-04-25T08:42:38.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.784+0000 7f55777fe640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f556406b4d0 0x7f556406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:38.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.784+0000 7f55777fe640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f556406b4d0 0x7f556406d990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5568006fd0 tx=0x7f5568008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:38.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.788+0000 7f555ffff640 1 -- 172.21.15.38:0/3071183094 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5570049030 con 0x7f5578161c50 2024-04-25T08:42:38.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.972+0000 7f557df77640 1 -- 172.21.15.38:0/3071183094 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f55780d1560 con 0x7f5578161c50 2024-04-25T08:42:38.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.973+0000 7f555ffff640 1 -- 172.21.15.38:0/3071183094 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+13291 (secure 0 0 0) 0x7f5570079590 con 0x7f5578161c50 2024-04-25T08:42:38.966 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:38.966 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":148,"fsid":"b4c2b8d6-8d56-4d73-aed7-8d2ed9f86762","created":"2024-04-25T08:31:11.089237+0000","modified":"2024-04-25T08:42:35.825708+0000","last_up_change":"2024-04-25T08:31:22.059000+0000","last_in_change":"2024-04-25T08:31:13.342921+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:31:23.247458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"67ac1d5b-5c12-4ecd-bd79-e4736e8e70e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6801","nonce":1562259299}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6803","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6804","nonce":1562259299}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6809","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6811","nonce":1562259299}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1562259299},{"type":"v1","addr":"172.21.15.38:6807","nonce":1562259299}]},"public_addr":"172.21.15.38:6801/1562259299","cluster_addr":"172.21.15.38:6804/1562259299","heartbeat_back_addr":"172.21.15.38:6811/1562259299","heartbeat_front_addr":"172.21.15.38:6807/1562259299","state":["exists","up"]},{"osd":1,"uuid":"45c7df16-5d20-4dfe-af4f-9178e69b2658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6805","nonce":3579687234}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6810","nonce":3579687234}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6815","nonce":3579687234}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3579687234},{"type":"v1","addr":"172.21.15.38:6813","nonce":3579687234}]},"public_addr":"172.21.15.38:6805/3579687234","cluster_addr":"172.21.15.38:6810/3579687234","heartbeat_back_addr":"172.21.15.38:6815/3579687234","heartbeat_front_addr":"172.21.15.38:6813/3579687234","state":["exists","up"]},{"osd":2,"uuid":"1a522f66-7604-437a-bca6-3fdec3ad936b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6825","nonce":3010466496}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6827","nonce":3010466496}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6831","nonce":3010466496}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":3010466496},{"type":"v1","addr":"172.21.15.38:6829","nonce":3010466496}]},"public_addr":"172.21.15.38:6825/3010466496","cluster_addr":"172.21.15.38:6827/3010466496","heartbeat_back_addr":"172.21.15.38:6831/3010466496","heartbeat_front_addr":"172.21.15.38:6829/3010466496","state":["exists","up"]},{"osd":3,"uuid":"184041f7-fe13-41f4-b876-93aef9649d6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6817","nonce":831384614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6819","nonce":831384614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6823","nonce":831384614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":831384614},{"type":"v1","addr":"172.21.15.38:6821","nonce":831384614}]},"public_addr":"172.21.15.38:6817/831384614","cluster_addr":"172.21.15.38:6819/831384614","heartbeat_back_addr":"172.21.15.38:6823/831384614","heartbeat_front_addr":"172.21.15.38:6821/831384614","state":["exists","up"]},{"osd":4,"uuid":"f1c4ae73-71ac-4d4c-9eb3-7cc9f5afe436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6801","nonce":592699329}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6803","nonce":592699329}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6807","nonce":592699329}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":592699329},{"type":"v1","addr":"172.21.15.107:6805","nonce":592699329}]},"public_addr":"172.21.15.107:6801/592699329","cluster_addr":"172.21.15.107:6803/592699329","heartbeat_back_addr":"172.21.15.107:6807/592699329","heartbeat_front_addr":"172.21.15.107:6805/592699329","state":["exists","up"]},{"osd":5,"uuid":"9d2fe8a8-ca71-408d-96d9-aaebe0863faf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6817","nonce":4056807981}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6819","nonce":4056807981}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6823","nonce":4056807981}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":4056807981},{"type":"v1","addr":"172.21.15.107:6821","nonce":4056807981}]},"public_addr":"172.21.15.107:6817/4056807981","cluster_addr":"172.21.15.107:6819/4056807981","heartbeat_back_addr":"172.21.15.107:6823/4056807981","heartbeat_front_addr":"172.21.15.107:6821/4056807981","state":["exists","up"]},{"osd":6,"uuid":"9da74044-e310-4263-a10a-2210377872e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":141,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6809","nonce":1351784056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6811","nonce":1351784056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6815","nonce":1351784056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1351784056},{"type":"v1","addr":"172.21.15.107:6813","nonce":1351784056}]},"public_addr":"172.21.15.107:6809/1351784056","cluster_addr":"172.21.15.107:6811/1351784056","heartbeat_back_addr":"172.21.15.107:6815/1351784056","heartbeat_front_addr":"172.21.15.107:6813/1351784056","state":["exists","up"]},{"osd":7,"uuid":"f0e8cad8-190a-4288-87a4-c8d4b4ead817","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6825","nonce":3757070988}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6827","nonce":3757070988}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6831","nonce":3757070988}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3757070988},{"type":"v1","addr":"172.21.15.107:6829","nonce":3757070988}]},"public_addr":"172.21.15.107:6825/3757070988","cluster_addr":"172.21.15.107:6827/3757070988","heartbeat_back_addr":"172.21.15.107:6831/3757070988","heartbeat_front_addr":"172.21.15.107:6829/3757070988","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.404686+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.459088+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.526104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.497390+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.492066+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.468868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.454818+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:31:18.484603+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6835/3513331661":"2024-04-26T08:42:32.474980+0000","172.21.15.38:6834/3513331661":"2024-04-26T08:42:32.474980+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:42:38.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.975+0000 7f557df77640 1 -- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f556406b4d0 msgr2=0x7f556406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.975+0000 7f557df77640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f556406b4d0 0x7f556406d990 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5568006fd0 tx=0x7f5568008040 comp rx=0 tx=0).stop 2024-04-25T08:42:38.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.975+0000 7f557df77640 1 -- 172.21.15.38:0/3071183094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5578161c50 msgr2=0x7f557815e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:38.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.975+0000 7f557df77640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5578161c50 0x7f557815e1b0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f557002ac90 tx=0x7f5570004a40 comp rx=0 tx=0).stop 2024-04-25T08:42:38.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.975+0000 7f557df77640 1 -- 172.21.15.38:0/3071183094 shutdown_connections 2024-04-25T08:42:38.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.975+0000 7f557df77640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5578161c50 0x7f557815e1b0 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.975+0000 7f557df77640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f556406b4d0 0x7f556406d990 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.975+0000 7f557df77640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f557815da90 0x7f5578151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.975+0000 7f557df77640 1 --2- 172.21.15.38:0/3071183094 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55780d0850 0x7f55780c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:38.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.975+0000 7f557df77640 1 -- 172.21.15.38:0/3071183094 >> 172.21.15.38:0/3071183094 conn(0x7f557806c060 msgr2=0x7f55780b8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:38.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.975+0000 7f557df77640 1 -- 172.21.15.38:0/3071183094 shutdown_connections 2024-04-25T08:42:38.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:38.975+0000 7f557df77640 1 -- 172.21.15.38:0/3071183094 wait complete. 2024-04-25T08:42:39.980 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T08:42:39.980 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:42:40.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.106+0000 7f9b65edd640 1 Processor -- start 2024-04-25T08:42:40.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.106+0000 7f9b65edd640 1 -- start start 2024-04-25T08:42:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b65edd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9b600d19e0 0x7f9b600d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b65edd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b600b6a00 0x7f9b600b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b65edd640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b600d24a0 con 0x7f9b600d0860 2024-04-25T08:42:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b65edd640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b6015e2b0 con 0x7f9b600d19e0 2024-04-25T08:42:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b65edd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600d2780 con 0x7f9b600b6a00 2024-04-25T08:42:40.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b5effd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9b600d19e0 0x7f9b600d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b5effd640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f9b600d19e0 0x7f9b600d1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34864/0 (socket says 172.21.15.38:34864) 2024-04-25T08:42:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b5effd640 1 -- 172.21.15.38:0/307013324 learned_addr learned my addr 172.21.15.38:0/307013324 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b5ffff640 1 --2- 172.21.15.38:0/307013324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b600b6a00 0x7f9b600b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b5effd640 1 -- 172.21.15.38:0/307013324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b600b6a00 msgr2=0x7f9b600b6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b5effd640 1 --2- 172.21.15.38:0/307013324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b600b6a00 0x7f9b600b6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b5effd640 1 -- 172.21.15.38:0/307013324 >> v1:172.21.15.107:6789/0 conn(0x7f9b600d0860 legacy=0x7f9b600d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.107+0000 7f9b5effd640 1 -- 172.21.15.38:0/307013324 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b600d2a60 con 0x7f9b600d19e0 2024-04-25T08:42:40.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.108+0000 7f9b5effd640 1 --2- 172.21.15.38:0/307013324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b600d19e0 0x7f9b600d1e50 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009870 tx=0x7f9b5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d955695b079d74b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.108+0000 7f9b5e7fc640 1 -- 172.21.15.38:0/307013324 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54037070 con 0x7f9b600d19e0 2024-04-25T08:42:40.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.108+0000 7f9b5e7fc640 1 -- 172.21.15.38:0/307013324 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b54032440 con 0x7f9b600d19e0 2024-04-25T08:42:40.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.108+0000 7f9b5e7fc640 1 -- 172.21.15.38:0/307013324 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b54032810 con 0x7f9b600d19e0 2024-04-25T08:42:40.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.109+0000 7f9b65edd640 1 -- 172.21.15.38:0/307013324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b600d19e0 msgr2=0x7f9b600d1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.109+0000 7f9b65edd640 1 --2- 172.21.15.38:0/307013324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b600d19e0 0x7f9b600d1e50 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009870 tx=0x7f9b5402a270 comp rx=0 tx=0).stop 2024-04-25T08:42:40.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.109+0000 7f9b65edd640 1 -- 172.21.15.38:0/307013324 shutdown_connections 2024-04-25T08:42:40.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.109+0000 7f9b65edd640 1 --2- 172.21.15.38:0/307013324 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b600b6a00 0x7f9b600b6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.109+0000 7f9b65edd640 1 --2- 172.21.15.38:0/307013324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b600d19e0 0x7f9b600d1e50 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.109+0000 7f9b65edd640 1 -- 172.21.15.38:0/307013324 >> 172.21.15.38:0/307013324 conn(0x7f9b60066d10 msgr2=0x7f9b60067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.109+0000 7f9b65edd640 1 -- 172.21.15.38:0/307013324 shutdown_connections 2024-04-25T08:42:40.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.109+0000 7f9b65edd640 1 -- 172.21.15.38:0/307013324 wait complete. 2024-04-25T08:42:40.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.110+0000 7f9b65edd640 1 Processor -- start 2024-04-25T08:42:40.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.110+0000 7f9b65edd640 1 -- start start 2024-04-25T08:42:40.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.111+0000 7f9b65edd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b600b6a00 0x7f9b60156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.111+0000 7f9b65edd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b600d0860 0x7f9b601f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.111+0000 7f9b65edd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b600d19e0 0x7f9b60157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.111+0000 7f9b65edd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600b9770 con 0x7f9b600d19e0 2024-04-25T08:42:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.111+0000 7f9b65edd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600d3540 con 0x7f9b600b6a00 2024-04-25T08:42:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.111+0000 7f9b65edd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b600b98f0 con 0x7f9b600d0860 2024-04-25T08:42:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.111+0000 7f9b5ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b600d19e0 0x7f9b60157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.111+0000 7f9b5ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b600d19e0 0x7f9b60157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34874/0 (socket says 172.21.15.38:34874) 2024-04-25T08:42:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.111+0000 7f9b5ffff640 1 -- 172.21.15.38:0/3910914587 learned_addr learned my addr 172.21.15.38:0/3910914587 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b5effd640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b600d0860 0x7f9b601f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b5f7fe640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b600b6a00 0x7f9b60156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b5ffff640 1 -- 172.21.15.38:0/3910914587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b600d0860 msgr2=0x7f9b601f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b5ffff640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9b600d0860 0x7f9b601f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b5ffff640 1 -- 172.21.15.38:0/3910914587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b600b6a00 msgr2=0x7f9b60156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b5ffff640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b600b6a00 0x7f9b60156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b5ffff640 1 -- 172.21.15.38:0/3910914587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b60158990 con 0x7f9b600d19e0 2024-04-25T08:42:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b5ffff640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b600d19e0 0x7f9b60157f40 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800efc0 tx=0x7f9b5800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b47fff640 1 -- 172.21.15.38:0/3910914587 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b58018020 con 0x7f9b600d19e0 2024-04-25T08:42:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b47fff640 1 -- 172.21.15.38:0/3910914587 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b58014480 con 0x7f9b600d19e0 2024-04-25T08:42:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b65edd640 1 -- 172.21.15.38:0/3910914587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b60158c70 con 0x7f9b600d19e0 2024-04-25T08:42:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b47fff640 1 -- 172.21.15.38:0/3910914587 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b58014870 con 0x7f9b600d19e0 2024-04-25T08:42:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.112+0000 7f9b65edd640 1 -- 172.21.15.38:0/3910914587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b600cc6b0 con 0x7f9b600d19e0 2024-04-25T08:42:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.113+0000 7f9b65edd640 1 -- 172.21.15.38:0/3910914587 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9b60068e80 con 0x7f9b600d19e0 2024-04-25T08:42:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.114+0000 7f9b47fff640 1 -- 172.21.15.38:0/3910914587 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b58005d10 con 0x7f9b600d19e0 2024-04-25T08:42:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.114+0000 7f9b47fff640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9b4c06b4d0 0x7f9b4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.114+0000 7f9b47fff640 1 -- 172.21.15.38:0/3910914587 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9b5808b440 con 0x7f9b600d19e0 2024-04-25T08:42:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.114+0000 7f9b47fff640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] conn(0x7f9b4c071970 0x7f9b4c073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.114+0000 7f9b47fff640 1 -- 172.21.15.38:0/3910914587 --> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b58024e80 con 0x7f9b4c071970 2024-04-25T08:42:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.114+0000 7f9b47fff640 1 -- 172.21.15.38:0/3910914587 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b5808b8d0 con 0x7f9b600d19e0 2024-04-25T08:42:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.114+0000 7f9b5f7fe640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9b4c06b4d0 0x7f9b4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.115+0000 7f9b5effd640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] conn(0x7f9b4c071970 0x7f9b4c073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.115+0000 7f9b5effd640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] conn(0x7f9b4c071970 0x7f9b4c073db0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.115+0000 7f9b5f7fe640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9b4c06b4d0 0x7f9b4c06d990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9b500015a0 tx=0x7f9b50009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.116+0000 7f9b47fff640 1 -- 172.21.15.38:0/3910914587 <== osd.0 v2:172.21.15.38:6800/1562259299 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9b58024e80 con 0x7f9b4c071970 2024-04-25T08:42:40.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.129+0000 7f9b65edd640 1 -- 172.21.15.38:0/3910914587 --> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9b600b5100 con 0x7f9b4c071970 2024-04-25T08:42:40.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.129+0000 7f9b47fff640 1 -- 172.21.15.38:0/3910914587 <== osd.0 v2:172.21.15.38:6800/1562259299 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9b600b5100 con 0x7f9b4c071970 2024-04-25T08:42:40.122 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:42:40.122 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:42:40.122 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:42:40.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.130+0000 7f9b65edd640 1 -- 172.21.15.38:0/3910914587 >> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] conn(0x7f9b4c071970 msgr2=0x7f9b4c073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.130+0000 7f9b65edd640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.38:6800/1562259299,v1:172.21.15.38:6801/1562259299] conn(0x7f9b4c071970 0x7f9b4c073db0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.130+0000 7f9b65edd640 1 -- 172.21.15.38:0/3910914587 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9b4c06b4d0 msgr2=0x7f9b4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.130+0000 7f9b65edd640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f9b4c06b4d0 0x7f9b4c06d990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f9b500015a0 tx=0x7f9b50009040 comp rx=0 tx=0).stop 2024-04-25T08:42:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.130+0000 7f9b65edd640 1 -- 172.21.15.38:0/3910914587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b600d19e0 msgr2=0x7f9b60157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.130+0000 7f9b65edd640 1 --2- 172.21.15.38:0/3910914587 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b600d19e0 0x7f9b60157f40 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f9b5800efc0 tx=0x7f9b5800c490 comp rx=0 tx=0).stop 2024-04-25T08:42:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.130+0000 7f9b5ffff640 1 -- 172.21.15.38:0/3910914587 reap_dead start 2024-04-25T08:42:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.130+0000 7f9b65edd640 1 -- 172.21.15.38:0/3910914587 shutdown_connections 2024-04-25T08:42:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.130+0000 7f9b65edd640 1 -- 172.21.15.38:0/3910914587 >> 172.21.15.38:0/3910914587 conn(0x7f9b60066d10 msgr2=0x7f9b600b32f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.130+0000 7f9b65edd640 1 -- 172.21.15.38:0/3910914587 shutdown_connections 2024-04-25T08:42:40.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.130+0000 7f9b65edd640 1 -- 172.21.15.38:0/3910914587 wait complete. 2024-04-25T08:42:40.133 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T08:42:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.243+0000 7f3321c2a640 1 Processor -- start 2024-04-25T08:42:40.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.244+0000 7f3321c2a640 1 -- start start 2024-04-25T08:42:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.244+0000 7f3321c2a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f331c0d0d10 0x7f331c099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.244+0000 7f3321c2a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331c0cfc70 con 0x7f331c15def0 2024-04-25T08:42:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.244+0000 7f3321c2a640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331c0cff50 con 0x7f331c099fb0 2024-04-25T08:42:40.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.244+0000 7f3321c2a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331c15e2d0 con 0x7f331c0d0d10 2024-04-25T08:42:40.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.244+0000 7f331b7fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f331c15def0 0x7f331c06c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33828/0 (socket says 172.21.15.38:33828) 2024-04-25T08:42:40.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.244+0000 7f331b7fe640 1 -- 172.21.15.38:0/2307921871 learned_addr learned my addr 172.21.15.38:0/2307921871 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:40.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491721834 0 0) 0x7f331c0cfc70 con 0x7f331c15def0 2024-04-25T08:42:40.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331affd640 1 --2- 172.21.15.38:0/2307921871 >> v2:172.21.15.107:3300/0 conn(0x7f331c0d0d10 0x7f331c099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3304003520 con 0x7f331c15def0 2024-04-25T08:42:40.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2573475559 0 0) 0x7f3304003520 con 0x7f331c15def0 2024-04-25T08:42:40.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 >> v2:172.21.15.107:3300/0 conn(0x7f331c0d0d10 msgr2=0x7f331c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 --2- 172.21.15.38:0/2307921871 >> v2:172.21.15.107:3300/0 conn(0x7f331c0d0d10 0x7f331c099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 >> v1:172.21.15.107:6789/0 conn(0x7f331c099fb0 legacy=0x7f331c0cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c0d0230 con 0x7f331c15def0 2024-04-25T08:42:40.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f330c001670 con 0x7f331c15def0 2024-04-25T08:42:40.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 >> v1:172.21.15.38:6789/0 conn(0x7f331c15def0 legacy=0x7f331c06c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 --2- 172.21.15.38:0/2307921871 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33040039b0 0x7f3304003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 --2- 172.21.15.38:0/2307921871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 0x7f3304004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 --2- 172.21.15.38:0/2307921871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33040051c0 0x7f330400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330400c030 con 0x7f3304004800 2024-04-25T08:42:40.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330400c1e0 con 0x7f33040051c0 2024-04-25T08:42:40.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330400c3e0 con 0x7f33040039b0 2024-04-25T08:42:40.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331affd640 1 --2- 172.21.15.38:0/2307921871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 0x7f3304004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331b7fe640 1 --2- 172.21.15.38:0/2307921871 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33040039b0 0x7f3304003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331bfff640 1 --2- 172.21.15.38:0/2307921871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33040051c0 0x7f330400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331affd640 1 -- 172.21.15.38:0/2307921871 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33040039b0 msgr2=0x7f3304003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331affd640 1 --2- 172.21.15.38:0/2307921871 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f33040039b0 0x7f3304003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331affd640 1 -- 172.21.15.38:0/2307921871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33040051c0 msgr2=0x7f330400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.245+0000 7f331affd640 1 --2- 172.21.15.38:0/2307921871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33040051c0 0x7f330400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.246+0000 7f331affd640 1 -- 172.21.15.38:0/2307921871 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3304003520 con 0x7f3304004800 2024-04-25T08:42:40.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.246+0000 7f331bfff640 1 --2- 172.21.15.38:0/2307921871 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f33040051c0 0x7f330400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:40.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.246+0000 7f331bfff640 1 -- 172.21.15.38:0/2307921871 reap_dead start 2024-04-25T08:42:40.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.246+0000 7f331affd640 1 --2- 172.21.15.38:0/2307921871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 0x7f3304004c80 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f3310009ac0 tx=0x7f3304005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf771bd1ebe5cd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.246+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310036070 con 0x7f3304004800 2024-04-25T08:42:40.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.246+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3310031400 con 0x7f3304004800 2024-04-25T08:42:40.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.246+0000 7f331a7fc640 1 -- 172.21.15.38:0/2307921871 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310009690 con 0x7f3304004800 2024-04-25T08:42:40.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.247+0000 7f3321c2a640 1 -- 172.21.15.38:0/2307921871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 msgr2=0x7f3304004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.247+0000 7f3321c2a640 1 --2- 172.21.15.38:0/2307921871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 0x7f3304004c80 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f3310009ac0 tx=0x7f3304005830 comp rx=0 tx=0).stop 2024-04-25T08:42:40.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.247+0000 7f3321c2a640 1 -- 172.21.15.38:0/2307921871 shutdown_connections 2024-04-25T08:42:40.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.247+0000 7f3321c2a640 1 --2- 172.21.15.38:0/2307921871 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 0x7f3304004c80 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.247+0000 7f3321c2a640 1 -- 172.21.15.38:0/2307921871 >> 172.21.15.38:0/2307921871 conn(0x7f331c0b30f0 msgr2=0x7f331c0b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.247+0000 7f3321c2a640 1 -- 172.21.15.38:0/2307921871 shutdown_connections 2024-04-25T08:42:40.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.247+0000 7f3321c2a640 1 -- 172.21.15.38:0/2307921871 wait complete. 2024-04-25T08:42:40.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.248+0000 7f3321c2a640 1 Processor -- start 2024-04-25T08:42:40.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.248+0000 7f3321c2a640 1 -- start start 2024-04-25T08:42:40.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f3321c2a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 0x7f331c0c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f3321c2a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f331c15e2d0 0x7f331c14dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f3321c2a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f331c1585c0 0x7f331c158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f3321c2a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c0d07f0 con 0x7f3304004800 2024-04-25T08:42:40.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f3321c2a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c09a3e0 con 0x7f331c15e2d0 2024-04-25T08:42:40.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f3321c2a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331c0d1170 con 0x7f331c1585c0 2024-04-25T08:42:40.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 0x7f331c0c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f331b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 0x7f331c0c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34882/0 (socket says 172.21.15.38:34882) 2024-04-25T08:42:40.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f331b7fe640 1 -- 172.21.15.38:0/1106716819 learned_addr learned my addr 172.21.15.38:0/1106716819 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:40.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f331bfff640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f331c1585c0 0x7f331c158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f331affd640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f331c15e2d0 0x7f331c14dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f331b7fe640 1 -- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f331c1585c0 msgr2=0x7f331c158a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f331b7fe640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f331c1585c0 0x7f331c158a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f331b7fe640 1 -- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f331c15e2d0 msgr2=0x7f331c14dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f331b7fe640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f331c15e2d0 0x7f331c14dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f331b7fe640 1 -- 172.21.15.38:0/1106716819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c15eab0 con 0x7f3304004800 2024-04-25T08:42:40.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.249+0000 7f331affd640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f331c15e2d0 0x7f331c14dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:40.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.250+0000 7f331b7fe640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 0x7f331c0c7bc0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f330c007290 tx=0x7f330c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.250+0000 7f3303fff640 1 -- 172.21.15.38:0/1106716819 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c00f960 con 0x7f3304004800 2024-04-25T08:42:40.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.250+0000 7f3303fff640 1 -- 172.21.15.38:0/1106716819 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330c018070 con 0x7f3304004800 2024-04-25T08:42:40.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.250+0000 7f3321c2a640 1 -- 172.21.15.38:0/1106716819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331c15ed90 con 0x7f3304004800 2024-04-25T08:42:40.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.250+0000 7f3303fff640 1 -- 172.21.15.38:0/1106716819 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330c00fd40 con 0x7f3304004800 2024-04-25T08:42:40.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.250+0000 7f3321c2a640 1 -- 172.21.15.38:0/1106716819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331c15f350 con 0x7f3304004800 2024-04-25T08:42:40.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.251+0000 7f3321c2a640 1 -- 172.21.15.38:0/1106716819 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331c04ef90 con 0x7f3304004800 2024-04-25T08:42:40.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.251+0000 7f3303fff640 1 -- 172.21.15.38:0/1106716819 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f330c01d460 con 0x7f3304004800 2024-04-25T08:42:40.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.252+0000 7f3303fff640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f330806b4d0 0x7f330806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.252+0000 7f3303fff640 1 -- 172.21.15.38:0/1106716819 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f330c08a250 con 0x7f3304004800 2024-04-25T08:42:40.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.252+0000 7f331affd640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f330806b4d0 0x7f330806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.252+0000 7f331affd640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f330806b4d0 0x7f330806d990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3310033000 tx=0x7f3310008030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.256+0000 7f3303fff640 1 -- 172.21.15.38:0/1106716819 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f330c059130 con 0x7f3304004800 2024-04-25T08:42:40.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.435+0000 7f3321c2a640 1 -- 172.21.15.38:0/1106716819 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f331c070f20 con 0x7f330806b4d0 2024-04-25T08:42:40.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.435+0000 7f3303fff640 1 -- 172.21.15.38:0/1106716819 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f331c070f20 con 0x7f330806b4d0 2024-04-25T08:42:40.427 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T08:42:40.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.437+0000 7f3321c2a640 1 -- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f330806b4d0 msgr2=0x7f330806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.437+0000 7f3321c2a640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f330806b4d0 0x7f330806d990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3310033000 tx=0x7f3310008030 comp rx=0 tx=0).stop 2024-04-25T08:42:40.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.437+0000 7f3321c2a640 1 -- 172.21.15.38:0/1106716819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 msgr2=0x7f331c0c7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.437+0000 7f3321c2a640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 0x7f331c0c7bc0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f330c007290 tx=0x7f330c007750 comp rx=0 tx=0).stop 2024-04-25T08:42:40.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.437+0000 7f3321c2a640 1 -- 172.21.15.38:0/1106716819 shutdown_connections 2024-04-25T08:42:40.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.437+0000 7f3321c2a640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f330806b4d0 0x7f330806d990 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.437+0000 7f3321c2a640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f331c1585c0 0x7f331c158a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.437+0000 7f3321c2a640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f331c15e2d0 0x7f331c14dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.437+0000 7f3321c2a640 1 --2- 172.21.15.38:0/1106716819 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3304004800 0x7f331c0c7bc0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.437+0000 7f3321c2a640 1 -- 172.21.15.38:0/1106716819 >> 172.21.15.38:0/1106716819 conn(0x7f331c0b30f0 msgr2=0x7f331c0b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.437+0000 7f3321c2a640 1 -- 172.21.15.38:0/1106716819 shutdown_connections 2024-04-25T08:42:40.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.437+0000 7f3321c2a640 1 -- 172.21.15.38:0/1106716819 wait complete. 2024-04-25T08:42:40.441 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T08:42:40.441 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:42:40.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.553+0000 7fc91e702640 1 Processor -- start 2024-04-25T08:42:40.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.553+0000 7fc91e702640 1 -- start start 2024-04-25T08:42:40.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.554+0000 7fc91e702640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc9180d0860 0x7fc9180d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.554+0000 7fc91e702640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9180d19e0 0x7fc9180d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.554+0000 7fc91e702640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9180d1180 con 0x7fc9180d3860 2024-04-25T08:42:40.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.554+0000 7fc91e702640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc91815e2b0 con 0x7fc9180d0860 2024-04-25T08:42:40.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.554+0000 7fc91e702640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180d24a0 con 0x7fc9180d19e0 2024-04-25T08:42:40.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.554+0000 7fc917fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc9180d0860 0x7fc9180d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.548 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.554+0000 7fc917fff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fc9180d0860 0x7fc9180d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34886/0 (socket says 172.21.15.38:34886) 2024-04-25T08:42:40.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.554+0000 7fc917fff640 1 -- 172.21.15.38:0/3671780603 learned_addr learned my addr 172.21.15.38:0/3671780603 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:40.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.554+0000 7fc91cc9f640 1 --1- 172.21.15.38:0/3671780603 >> v1:172.21.15.38:6789/0 conn(0x7fc9180d3860 0x7fc9180d3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33842/0 (socket says 172.21.15.38:33842) 2024-04-25T08:42:40.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.555+0000 7fc9177fe640 1 --2- 172.21.15.38:0/3671780603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9180d19e0 0x7fc9180d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.555+0000 7fc917fff640 1 -- 172.21.15.38:0/3671780603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9180d19e0 msgr2=0x7fc9180d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.555+0000 7fc917fff640 1 --2- 172.21.15.38:0/3671780603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9180d19e0 0x7fc9180d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.555+0000 7fc917fff640 1 -- 172.21.15.38:0/3671780603 >> v1:172.21.15.38:6789/0 conn(0x7fc9180d3860 legacy=0x7fc9180d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.555+0000 7fc916ffd640 1 -- 172.21.15.38:0/3671780603 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317021059 0 0) 0x7fc9180d1180 con 0x7fc9180d3860 2024-04-25T08:42:40.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.555+0000 7fc917fff640 1 -- 172.21.15.38:0/3671780603 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9180d2780 con 0x7fc9180d0860 2024-04-25T08:42:40.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.555+0000 7fc9177fe640 1 --2- 172.21.15.38:0/3671780603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9180d19e0 0x7fc9180d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:40.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.555+0000 7fc917fff640 1 --2- 172.21.15.38:0/3671780603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9180d0860 0x7fc9180d0c40 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fc908009870 tx=0x7fc90802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7666171d775888b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.556+0000 7fc916ffd640 1 -- 172.21.15.38:0/3671780603 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908037070 con 0x7fc9180d0860 2024-04-25T08:42:40.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.556+0000 7fc916ffd640 1 -- 172.21.15.38:0/3671780603 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc908032440 con 0x7fc9180d0860 2024-04-25T08:42:40.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.556+0000 7fc916ffd640 1 -- 172.21.15.38:0/3671780603 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908032810 con 0x7fc9180d0860 2024-04-25T08:42:40.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.556+0000 7fc91e702640 1 -- 172.21.15.38:0/3671780603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9180d0860 msgr2=0x7fc9180d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.556+0000 7fc91e702640 1 --2- 172.21.15.38:0/3671780603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9180d0860 0x7fc9180d0c40 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fc908009870 tx=0x7fc90802a250 comp rx=0 tx=0).stop 2024-04-25T08:42:40.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.556+0000 7fc91e702640 1 -- 172.21.15.38:0/3671780603 shutdown_connections 2024-04-25T08:42:40.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.556+0000 7fc91e702640 1 --2- 172.21.15.38:0/3671780603 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9180d19e0 0x7fc9180d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.556+0000 7fc91e702640 1 --2- 172.21.15.38:0/3671780603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9180d0860 0x7fc9180d0c40 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.556+0000 7fc91e702640 1 -- 172.21.15.38:0/3671780603 >> 172.21.15.38:0/3671780603 conn(0x7fc918066d10 msgr2=0x7fc918067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.556+0000 7fc91e702640 1 -- 172.21.15.38:0/3671780603 shutdown_connections 2024-04-25T08:42:40.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.556+0000 7fc91e702640 1 -- 172.21.15.38:0/3671780603 wait complete. 2024-04-25T08:42:40.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.557+0000 7fc91e702640 1 Processor -- start 2024-04-25T08:42:40.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.558+0000 7fc91e702640 1 -- start start 2024-04-25T08:42:40.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.558+0000 7fc91e702640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9180d0860 0x7fc9180cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.558+0000 7fc91e702640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9180d19e0 0x7fc91814cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.558+0000 7fc91e702640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9180d3860 0x7fc9180d3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.558+0000 7fc91e702640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180d3540 con 0x7fc9180d3860 2024-04-25T08:42:40.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.558+0000 7fc91e702640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180d2d40 con 0x7fc9180d19e0 2024-04-25T08:42:40.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.558+0000 7fc91e702640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180b9770 con 0x7fc9180d0860 2024-04-25T08:42:40.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.558+0000 7fc91cc9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9180d3860 0x7fc9180d3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.558+0000 7fc91cc9f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9180d3860 0x7fc9180d3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34902/0 (socket says 172.21.15.38:34902) 2024-04-25T08:42:40.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.558+0000 7fc91cc9f640 1 -- 172.21.15.38:0/483631700 learned_addr learned my addr 172.21.15.38:0/483631700 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:40.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc917fff640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9180d0860 0x7fc9180cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc9177fe640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9180d19e0 0x7fc91814cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc91cc9f640 1 -- 172.21.15.38:0/483631700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9180d0860 msgr2=0x7fc9180cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc91cc9f640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc9180d0860 0x7fc9180cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.555 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc91cc9f640 1 -- 172.21.15.38:0/483631700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9180d19e0 msgr2=0x7fc91814cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc91cc9f640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc9180d19e0 0x7fc91814cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc91cc9f640 1 -- 172.21.15.38:0/483631700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9180cc8a0 con 0x7fc9180d3860 2024-04-25T08:42:40.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc91cc9f640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9180d3860 0x7fc9180d3cb0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fc91000c730 tx=0x7fc91000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc914ff9640 1 -- 172.21.15.38:0/483631700 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910018070 con 0x7fc9180d3860 2024-04-25T08:42:40.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc914ff9640 1 -- 172.21.15.38:0/483631700 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91000ce40 con 0x7fc9180d3860 2024-04-25T08:42:40.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc91e702640 1 -- 172.21.15.38:0/483631700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9180ccb80 con 0x7fc9180d3860 2024-04-25T08:42:40.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc914ff9640 1 -- 172.21.15.38:0/483631700 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91000a800 con 0x7fc9180d3860 2024-04-25T08:42:40.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.559+0000 7fc91e702640 1 -- 172.21.15.38:0/483631700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9180cd140 con 0x7fc9180d3860 2024-04-25T08:42:40.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.560+0000 7fc91e702640 1 -- 172.21.15.38:0/483631700 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc918068e80 con 0x7fc9180d3860 2024-04-25T08:42:40.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.561+0000 7fc914ff9640 1 -- 172.21.15.38:0/483631700 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc91001b070 con 0x7fc9180d3860 2024-04-25T08:42:40.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.561+0000 7fc914ff9640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc90406b4d0 0x7fc90406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.561+0000 7fc914ff9640 1 -- 172.21.15.38:0/483631700 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc910089510 con 0x7fc9180d3860 2024-04-25T08:42:40.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.561+0000 7fc914ff9640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x7fc904071970 0x7fc904073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.561+0000 7fc914ff9640 1 -- 172.21.15.38:0/483631700 --> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc910017ec0 con 0x7fc904071970 2024-04-25T08:42:40.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.561+0000 7fc914ff9640 1 -- 172.21.15.38:0/483631700 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc910090050 con 0x7fc9180d3860 2024-04-25T08:42:40.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.561+0000 7fc917fff640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc90406b4d0 0x7fc90406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.561+0000 7fc9177fe640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x7fc904071970 0x7fc904073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.562+0000 7fc9177fe640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x7fc904071970 0x7fc904073db0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.562+0000 7fc917fff640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc90406b4d0 0x7fc90406d990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc908034000 tx=0x7fc90802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.562+0000 7fc914ff9640 1 -- 172.21.15.38:0/483631700 <== osd.1 v2:172.21.15.38:6802/3579687234 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc910017ec0 con 0x7fc904071970 2024-04-25T08:42:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.576+0000 7fc91e702640 1 -- 172.21.15.38:0/483631700 --> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc9180c6c40 con 0x7fc904071970 2024-04-25T08:42:40.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.576+0000 7fc914ff9640 1 -- 172.21.15.38:0/483631700 <== osd.1 v2:172.21.15.38:6802/3579687234 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc9180c6c40 con 0x7fc904071970 2024-04-25T08:42:40.569 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:42:40.569 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:42:40.569 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:42:40.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.576+0000 7fc91e702640 1 -- 172.21.15.38:0/483631700 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x7fc904071970 msgr2=0x7fc904073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.576+0000 7fc91e702640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.38:6802/3579687234,v1:172.21.15.38:6805/3579687234] conn(0x7fc904071970 0x7fc904073db0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.576+0000 7fc91e702640 1 -- 172.21.15.38:0/483631700 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc90406b4d0 msgr2=0x7fc90406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.576+0000 7fc91e702640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fc90406b4d0 0x7fc90406d990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc908034000 tx=0x7fc90802e000 comp rx=0 tx=0).stop 2024-04-25T08:42:40.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.576+0000 7fc91e702640 1 -- 172.21.15.38:0/483631700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9180d3860 msgr2=0x7fc9180d3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.576+0000 7fc91e702640 1 --2- 172.21.15.38:0/483631700 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc9180d3860 0x7fc9180d3cb0 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fc91000c730 tx=0x7fc91000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.576+0000 7fc91cc9f640 1 -- 172.21.15.38:0/483631700 reap_dead start 2024-04-25T08:42:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.576+0000 7fc91e702640 1 -- 172.21.15.38:0/483631700 shutdown_connections 2024-04-25T08:42:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.576+0000 7fc91e702640 1 -- 172.21.15.38:0/483631700 >> 172.21.15.38:0/483631700 conn(0x7fc918066d10 msgr2=0x7fc918159ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.577+0000 7fc91e702640 1 -- 172.21.15.38:0/483631700 shutdown_connections 2024-04-25T08:42:40.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.577+0000 7fc91e702640 1 -- 172.21.15.38:0/483631700 wait complete. 2024-04-25T08:42:40.579 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T08:42:40.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.689+0000 7f69cab5c640 1 Processor -- start 2024-04-25T08:42:40.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.689+0000 7f69cab5c640 1 -- start start 2024-04-25T08:42:40.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69cab5c640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f69c40b9600 0x7f69c40d0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69cab5c640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f69c40d0660 0x7f69c40d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69cab5c640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c40d0a70 con 0x7f69c415db80 2024-04-25T08:42:40.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69cab5c640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c4161df0 con 0x7f69c40d0660 2024-04-25T08:42:40.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69cab5c640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c41625f0 con 0x7f69c40b9600 2024-04-25T08:42:40.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69c90f9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f69c40d0660 0x7f69c40d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69c90f9640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f69c40d0660 0x7f69c40d22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34916/0 (socket says 172.21.15.38:34916) 2024-04-25T08:42:40.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69c90f9640 1 -- 172.21.15.38:0/2495290002 learned_addr learned my addr 172.21.15.38:0/2495290002 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:40.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69bbfff640 1 --2- 172.21.15.38:0/2495290002 >> v2:172.21.15.107:3300/0 conn(0x7f69c40b9600 0x7f69c40d0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69c90f9640 1 -- 172.21.15.38:0/2495290002 >> v2:172.21.15.107:3300/0 conn(0x7f69c40b9600 msgr2=0x7f69c40d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69c90f9640 1 --2- 172.21.15.38:0/2495290002 >> v2:172.21.15.107:3300/0 conn(0x7f69c40b9600 0x7f69c40d0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69c90f9640 1 -- 172.21.15.38:0/2495290002 >> v1:172.21.15.38:6789/0 conn(0x7f69c415db80 legacy=0x7f69c41616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69bb7fe640 1 -- 172.21.15.38:0/2495290002 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 304446308 0 0) 0x7f69c40d0a70 con 0x7f69c415db80 2024-04-25T08:42:40.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.690+0000 7f69c90f9640 1 -- 172.21.15.38:0/2495290002 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c4162030 con 0x7f69c40d0660 2024-04-25T08:42:40.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.691+0000 7f69c90f9640 1 --2- 172.21.15.38:0/2495290002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c40d0660 0x7f69c40d22b0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f69c40d13f0 tx=0x7f69bc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b3d4433f9af483 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.691+0000 7f69bb7fe640 1 -- 172.21.15.38:0/2495290002 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc036070 con 0x7f69c40d0660 2024-04-25T08:42:40.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.691+0000 7f69bb7fe640 1 -- 172.21.15.38:0/2495290002 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69bc031600 con 0x7f69c40d0660 2024-04-25T08:42:40.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.691+0000 7f69bb7fe640 1 -- 172.21.15.38:0/2495290002 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc0319d0 con 0x7f69c40d0660 2024-04-25T08:42:40.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.692+0000 7f69cab5c640 1 -- 172.21.15.38:0/2495290002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c40d0660 msgr2=0x7f69c40d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.692+0000 7f69cab5c640 1 --2- 172.21.15.38:0/2495290002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c40d0660 0x7f69c40d22b0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f69c40d13f0 tx=0x7f69bc02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.692+0000 7f69cab5c640 1 -- 172.21.15.38:0/2495290002 shutdown_connections 2024-04-25T08:42:40.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.692+0000 7f69cab5c640 1 --2- 172.21.15.38:0/2495290002 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c40d0660 0x7f69c40d22b0 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.692+0000 7f69cab5c640 1 --2- 172.21.15.38:0/2495290002 >> v2:172.21.15.107:3300/0 conn(0x7f69c40b9600 0x7f69c40d0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.692+0000 7f69cab5c640 1 -- 172.21.15.38:0/2495290002 >> 172.21.15.38:0/2495290002 conn(0x7f69c40668f0 msgr2=0x7f69c4066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.692+0000 7f69cab5c640 1 -- 172.21.15.38:0/2495290002 shutdown_connections 2024-04-25T08:42:40.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.692+0000 7f69cab5c640 1 -- 172.21.15.38:0/2495290002 wait complete. 2024-04-25T08:42:40.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.693+0000 7f69cab5c640 1 Processor -- start 2024-04-25T08:42:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.693+0000 7f69cab5c640 1 -- start start 2024-04-25T08:42:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69cab5c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69c40b9600 0x7f69c40b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69cab5c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c40d0660 0x7f69c415fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69cab5c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c415db80 0x7f69c4160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69cab5c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c415df60 con 0x7f69c40d0660 2024-04-25T08:42:40.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69cab5c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40d1ab0 con 0x7f69c415db80 2024-04-25T08:42:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69cab5c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40d27f0 con 0x7f69c40b9600 2024-04-25T08:42:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c40d0660 0x7f69c415fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c40d0660 0x7f69c415fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34932/0 (socket says 172.21.15.38:34932) 2024-04-25T08:42:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69bbfff640 1 -- 172.21.15.38:0/3526206494 learned_addr learned my addr 172.21.15.38:0/3526206494 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69c88f8640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69c40b9600 0x7f69c40b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69c90f9640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c415db80 0x7f69c4160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69bbfff640 1 -- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69c40b9600 msgr2=0x7f69c40b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69bbfff640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69c40b9600 0x7f69c40b5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69bbfff640 1 -- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c415db80 msgr2=0x7f69c4160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69bbfff640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c415db80 0x7f69c4160160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69bbfff640 1 -- 172.21.15.38:0/3526206494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40cc310 con 0x7f69c40d0660 2024-04-25T08:42:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69c90f9640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c415db80 0x7f69c4160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.694+0000 7f69bbfff640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c40d0660 0x7f69c415fc20 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f69b400e900 tx=0x7f69b400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.695+0000 7f69b97fa640 1 -- 172.21.15.38:0/3526206494 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4017070 con 0x7f69c40d0660 2024-04-25T08:42:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.695+0000 7f69b97fa640 1 -- 172.21.15.38:0/3526206494 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b400ca00 con 0x7f69c40d0660 2024-04-25T08:42:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.695+0000 7f69cab5c640 1 -- 172.21.15.38:0/3526206494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c40cb790 con 0x7f69c40d0660 2024-04-25T08:42:40.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.695+0000 7f69b97fa640 1 -- 172.21.15.38:0/3526206494 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b400cdf0 con 0x7f69c40d0660 2024-04-25T08:42:40.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.695+0000 7f69cab5c640 1 -- 172.21.15.38:0/3526206494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c40cbd50 con 0x7f69c40d0660 2024-04-25T08:42:40.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.696+0000 7f69cab5c640 1 -- 172.21.15.38:0/3526206494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c4067f00 con 0x7f69c40d0660 2024-04-25T08:42:40.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.696+0000 7f69b97fa640 1 -- 172.21.15.38:0/3526206494 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69b401a070 con 0x7f69c40d0660 2024-04-25T08:42:40.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.696+0000 7f69b97fa640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f69b006b4d0 0x7f69b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.697+0000 7f69b97fa640 1 -- 172.21.15.38:0/3526206494 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f69b408a3d0 con 0x7f69c40d0660 2024-04-25T08:42:40.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.697+0000 7f69c88f8640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f69b006b4d0 0x7f69b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.697+0000 7f69c88f8640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f69b006b4d0 0x7f69b006d990 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f69c0007fd0 tx=0x7f69c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.701+0000 7f69b97fa640 1 -- 172.21.15.38:0/3526206494 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69c4067f00 con 0x7f69c40d0660 2024-04-25T08:42:40.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.879+0000 7f69cab5c640 1 -- 172.21.15.38:0/3526206494 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f69c404ef90 con 0x7f69b006b4d0 2024-04-25T08:42:40.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.880+0000 7f69b97fa640 1 -- 172.21.15.38:0/3526206494 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f69c404ef90 con 0x7f69b006b4d0 2024-04-25T08:42:40.872 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T08:42:40.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.882+0000 7f69cab5c640 1 -- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f69b006b4d0 msgr2=0x7f69b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.882+0000 7f69cab5c640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f69b006b4d0 0x7f69b006d990 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f69c0007fd0 tx=0x7f69c0009040 comp rx=0 tx=0).stop 2024-04-25T08:42:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.882+0000 7f69cab5c640 1 -- 172.21.15.38:0/3526206494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c40d0660 msgr2=0x7f69c415fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.882+0000 7f69cab5c640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c40d0660 0x7f69c415fc20 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f69b400e900 tx=0x7f69b400edc0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.882+0000 7f69cab5c640 1 -- 172.21.15.38:0/3526206494 shutdown_connections 2024-04-25T08:42:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.882+0000 7f69cab5c640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69c415db80 0x7f69c4160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.882+0000 7f69cab5c640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69c40d0660 0x7f69c415fc20 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.882+0000 7f69cab5c640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f69b006b4d0 0x7f69b006d990 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.882+0000 7f69cab5c640 1 --2- 172.21.15.38:0/3526206494 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f69c40b9600 0x7f69c40b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.882+0000 7f69cab5c640 1 -- 172.21.15.38:0/3526206494 >> 172.21.15.38:0/3526206494 conn(0x7f69c40668f0 msgr2=0x7f69c40d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.882+0000 7f69cab5c640 1 -- 172.21.15.38:0/3526206494 shutdown_connections 2024-04-25T08:42:40.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.882+0000 7f69cab5c640 1 -- 172.21.15.38:0/3526206494 wait complete. 2024-04-25T08:42:40.885 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T08:42:40.885 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:42:40.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.998+0000 7f0dbfd29640 1 Processor -- start 2024-04-25T08:42:40.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.998+0000 7f0dbfd29640 1 -- start start 2024-04-25T08:42:40.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbfd29640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f0db81612c0 0x7f0db80b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbfd29640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0db80b6a00 0x7f0db80d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbfd29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db80d0870 0x7f0db80d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbfd29640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db80d11b0 con 0x7f0db80b6a00 2024-04-25T08:42:40.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbfd29640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db80b7200 con 0x7f0db81612c0 2024-04-25T08:42:40.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbfd29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80b74e0 con 0x7f0db80d0870 2024-04-25T08:42:40.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbd2c4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0db80b6a00 0x7f0db80d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbd2c4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f0db80b6a00 0x7f0db80d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34942/0 (socket says 172.21.15.38:34942) 2024-04-25T08:42:40.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbd2c4640 1 -- 172.21.15.38:0/4022996325 learned_addr learned my addr 172.21.15.38:0/4022996325 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:40.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbe2c6640 1 --2- 172.21.15.38:0/4022996325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db80d0870 0x7f0db80d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbdac5640 1 --2- 172.21.15.38:0/4022996325 >> v2:172.21.15.107:3300/0 conn(0x7f0db81612c0 0x7f0db80b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbd2c4640 1 -- 172.21.15.38:0/4022996325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db80d0870 msgr2=0x7f0db80d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbd2c4640 1 --2- 172.21.15.38:0/4022996325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db80d0870 0x7f0db80d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbd2c4640 1 -- 172.21.15.38:0/4022996325 >> v2:172.21.15.107:3300/0 conn(0x7f0db81612c0 msgr2=0x7f0db80b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:40.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbd2c4640 1 --2- 172.21.15.38:0/4022996325 >> v2:172.21.15.107:3300/0 conn(0x7f0db81612c0 0x7f0db80b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:40.999+0000 7f0dbd2c4640 1 -- 172.21.15.38:0/4022996325 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db80b77c0 con 0x7f0db80b6a00 2024-04-25T08:42:40.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.000+0000 7f0dbd2c4640 1 --2- 172.21.15.38:0/4022996325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80b6a00 0x7f0db80d1c30 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f0db80d49a0 tx=0x7f0da802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c290515817d785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:40.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.000+0000 7f0dbcac3640 1 -- 172.21.15.38:0/4022996325 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8037070 con 0x7f0db80b6a00 2024-04-25T08:42:40.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.000+0000 7f0dbcac3640 1 -- 172.21.15.38:0/4022996325 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da8032440 con 0x7f0db80b6a00 2024-04-25T08:42:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.000+0000 7f0dbcac3640 1 -- 172.21.15.38:0/4022996325 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da8032810 con 0x7f0db80b6a00 2024-04-25T08:42:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.001+0000 7f0dbfd29640 1 -- 172.21.15.38:0/4022996325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80b6a00 msgr2=0x7f0db80d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.001+0000 7f0dbfd29640 1 --2- 172.21.15.38:0/4022996325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80b6a00 0x7f0db80d1c30 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f0db80d49a0 tx=0x7f0da802a140 comp rx=0 tx=0).stop 2024-04-25T08:42:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.001+0000 7f0dbfd29640 1 -- 172.21.15.38:0/4022996325 shutdown_connections 2024-04-25T08:42:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.001+0000 7f0dbfd29640 1 --2- 172.21.15.38:0/4022996325 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db80d0870 0x7f0db80d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.001+0000 7f0dbfd29640 1 --2- 172.21.15.38:0/4022996325 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80b6a00 0x7f0db80d1c30 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.001+0000 7f0dbfd29640 1 --2- 172.21.15.38:0/4022996325 >> v2:172.21.15.107:3300/0 conn(0x7f0db81612c0 0x7f0db80b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.001+0000 7f0dbfd29640 1 -- 172.21.15.38:0/4022996325 >> 172.21.15.38:0/4022996325 conn(0x7f0db8066d10 msgr2=0x7f0db8067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:40.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.001+0000 7f0dbfd29640 1 -- 172.21.15.38:0/4022996325 shutdown_connections 2024-04-25T08:42:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.001+0000 7f0dbfd29640 1 -- 172.21.15.38:0/4022996325 wait complete. 2024-04-25T08:42:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.002+0000 7f0dbfd29640 1 Processor -- start 2024-04-25T08:42:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.002+0000 7f0dbfd29640 1 -- start start 2024-04-25T08:42:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbfd29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80b6a00 0x7f0db81553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbfd29640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db80d0870 0x7f0db81f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbfd29640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db81612c0 0x7f0db815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbfd29640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80d2170 con 0x7f0db80b6a00 2024-04-25T08:42:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbfd29640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80b7d80 con 0x7f0db81612c0 2024-04-25T08:42:40.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbfd29640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db80b6e90 con 0x7f0db80d0870 2024-04-25T08:42:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbdac5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80b6a00 0x7f0db81553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbdac5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80b6a00 0x7f0db81553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34950/0 (socket says 172.21.15.38:34950) 2024-04-25T08:42:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbdac5640 1 -- 172.21.15.38:0/3576169066 learned_addr learned my addr 172.21.15.38:0/3576169066 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbd2c4640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db80d0870 0x7f0db81f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbe2c6640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db81612c0 0x7f0db815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbdac5640 1 -- 172.21.15.38:0/3576169066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db80d0870 msgr2=0x7f0db81f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbdac5640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f0db80d0870 0x7f0db81f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbdac5640 1 -- 172.21.15.38:0/3576169066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db81612c0 msgr2=0x7f0db815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:40.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbdac5640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0db81612c0 0x7f0db815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbdac5640 1 -- 172.21.15.38:0/3576169066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db815aef0 con 0x7f0db80b6a00 2024-04-25T08:42:41.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbdac5640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80b6a00 0x7f0db81553e0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f0db400e870 tx=0x7f0db400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dae7fc640 1 -- 172.21.15.38:0/3576169066 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db4017070 con 0x7f0db80b6a00 2024-04-25T08:42:41.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dae7fc640 1 -- 172.21.15.38:0/3576169066 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db400c940 con 0x7f0db80b6a00 2024-04-25T08:42:41.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbfd29640 1 -- 172.21.15.38:0/3576169066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db81f44f0 con 0x7f0db80b6a00 2024-04-25T08:42:41.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dae7fc640 1 -- 172.21.15.38:0/3576169066 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db400cd30 con 0x7f0db80b6a00 2024-04-25T08:42:41.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.003+0000 7f0dbfd29640 1 -- 172.21.15.38:0/3576169066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db81f3f30 con 0x7f0db80b6a00 2024-04-25T08:42:41.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.004+0000 7f0dbfd29640 1 -- 172.21.15.38:0/3576169066 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0db8068e80 con 0x7f0db80b6a00 2024-04-25T08:42:41.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.005+0000 7f0dae7fc640 1 -- 172.21.15.38:0/3576169066 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0db401a070 con 0x7f0db80b6a00 2024-04-25T08:42:41.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.005+0000 7f0dae7fc640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d9c06b4d0 0x7f0d9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.005+0000 7f0dae7fc640 1 -- 172.21.15.38:0/3576169066 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0db4089e70 con 0x7f0db80b6a00 2024-04-25T08:42:41.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.005+0000 7f0dae7fc640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] conn(0x7f0d9c071970 0x7f0d9c073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.005+0000 7f0dae7fc640 1 -- 172.21.15.38:0/3576169066 --> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0db4020e50 con 0x7f0d9c071970 2024-04-25T08:42:41.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.005+0000 7f0dae7fc640 1 -- 172.21.15.38:0/3576169066 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0db408c570 con 0x7f0db80b6a00 2024-04-25T08:42:41.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.005+0000 7f0dbe2c6640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] conn(0x7f0d9c071970 0x7f0d9c073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.006+0000 7f0dbd2c4640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d9c06b4d0 0x7f0d9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.006+0000 7f0dbe2c6640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] conn(0x7f0d9c071970 0x7f0d9c073db0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.006+0000 7f0dbd2c4640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d9c06b4d0 0x7f0d9c06d990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0da8034000 tx=0x7f0da802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.006+0000 7f0dae7fc640 1 -- 172.21.15.38:0/3576169066 <== osd.2 v2:172.21.15.38:6824/3010466496 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0db4020e50 con 0x7f0d9c071970 2024-04-25T08:42:41.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.019+0000 7f0dbfd29640 1 -- 172.21.15.38:0/3576169066 --> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0db8157750 con 0x7f0d9c071970 2024-04-25T08:42:41.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.020+0000 7f0dae7fc640 1 -- 172.21.15.38:0/3576169066 <== osd.2 v2:172.21.15.38:6824/3010466496 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f0db8157750 con 0x7f0d9c071970 2024-04-25T08:42:41.012 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:42:41.012 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:42:41.013 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:42:41.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.020+0000 7f0dbfd29640 1 -- 172.21.15.38:0/3576169066 >> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] conn(0x7f0d9c071970 msgr2=0x7f0d9c073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.020+0000 7f0dbfd29640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.38:6824/3010466496,v1:172.21.15.38:6825/3010466496] conn(0x7f0d9c071970 0x7f0d9c073db0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.020+0000 7f0dbfd29640 1 -- 172.21.15.38:0/3576169066 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d9c06b4d0 msgr2=0x7f0d9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.020+0000 7f0dbfd29640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f0d9c06b4d0 0x7f0d9c06d990 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f0da8034000 tx=0x7f0da802e000 comp rx=0 tx=0).stop 2024-04-25T08:42:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.020+0000 7f0dbfd29640 1 -- 172.21.15.38:0/3576169066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80b6a00 msgr2=0x7f0db81553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.020+0000 7f0dbfd29640 1 --2- 172.21.15.38:0/3576169066 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0db80b6a00 0x7f0db81553e0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f0db400e870 tx=0x7f0db400ed30 comp rx=0 tx=0).stop 2024-04-25T08:42:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.020+0000 7f0dbe2c6640 1 -- 172.21.15.38:0/3576169066 reap_dead start 2024-04-25T08:42:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.020+0000 7f0dbfd29640 1 -- 172.21.15.38:0/3576169066 shutdown_connections 2024-04-25T08:42:41.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.020+0000 7f0dbfd29640 1 -- 172.21.15.38:0/3576169066 >> 172.21.15.38:0/3576169066 conn(0x7f0db8066d10 msgr2=0x7f0db815f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.020+0000 7f0dbfd29640 1 -- 172.21.15.38:0/3576169066 shutdown_connections 2024-04-25T08:42:41.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.020+0000 7f0dbfd29640 1 -- 172.21.15.38:0/3576169066 wait complete. 2024-04-25T08:42:41.023 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T08:42:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.133+0000 7fa1aa3b7640 1 Processor -- start 2024-04-25T08:42:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.133+0000 7fa1aa3b7640 1 -- start start 2024-04-25T08:42:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.133+0000 7fa1aa3b7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1a415ef50 0x7fa1a40ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.133+0000 7fa1aa3b7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a40cd350 con 0x7fa1a406b2f0 2024-04-25T08:42:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.133+0000 7fa1aa3b7640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a40cd630 con 0x7fa1a40b2b80 2024-04-25T08:42:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.133+0000 7fa1aa3b7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40cd910 con 0x7fa1a415ef50 2024-04-25T08:42:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.133+0000 7fa1a8954640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fa1a406b2f0 0x7fa1a40b0320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33860/0 (socket says 172.21.15.38:33860) 2024-04-25T08:42:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.134+0000 7fa1a8954640 1 -- 172.21.15.38:0/3548494221 learned_addr learned my addr 172.21.15.38:0/3548494221 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.134+0000 7fa1a3fff640 1 --2- 172.21.15.38:0/3548494221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1a415ef50 0x7fa1a40ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.134+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/3548494221 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158816042 0 0) 0x7fa1a40cd350 con 0x7fa1a406b2f0 2024-04-25T08:42:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.134+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/3548494221 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa18c003520 con 0x7fa1a406b2f0 2024-04-25T08:42:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.134+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/3548494221 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 675260736 0 0) 0x7fa1a40cd630 con 0x7fa1a40b2b80 2024-04-25T08:42:41.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.134+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/3548494221 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1a40cd350 con 0x7fa1a40b2b80 2024-04-25T08:42:41.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.134+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/3548494221 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1107529321 0 0) 0x7fa18c003520 con 0x7fa1a406b2f0 2024-04-25T08:42:41.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.134+0000 7fa1a3fff640 1 -- 172.21.15.38:0/3548494221 >> v1:172.21.15.107:6789/0 conn(0x7fa1a40b2b80 legacy=0x7fa1a406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.134+0000 7fa1a3fff640 1 -- 172.21.15.38:0/3548494221 >> v1:172.21.15.38:6789/0 conn(0x7fa1a406b2f0 legacy=0x7fa1a40b0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.134+0000 7fa1a3fff640 1 -- 172.21.15.38:0/3548494221 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a40cdbf0 con 0x7fa1a415ef50 2024-04-25T08:42:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.135+0000 7fa1a3fff640 1 --2- 172.21.15.38:0/3548494221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1a415ef50 0x7fa1a40ccaf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa1940097d0 tx=0x7fa19402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c5606ba3651cd51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.135+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/3548494221 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194037070 con 0x7fa1a415ef50 2024-04-25T08:42:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.135+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/3548494221 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19402ab90 con 0x7fa1a415ef50 2024-04-25T08:42:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.135+0000 7fa1a2ffd640 1 -- 172.21.15.38:0/3548494221 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0019f0 con 0x7fa1a415ef50 2024-04-25T08:42:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.136+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/3548494221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1a415ef50 msgr2=0x7fa1a40ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.136+0000 7fa1aa3b7640 1 --2- 172.21.15.38:0/3548494221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1a415ef50 0x7fa1a40ccaf0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa1940097d0 tx=0x7fa19402a100 comp rx=0 tx=0).stop 2024-04-25T08:42:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.136+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/3548494221 shutdown_connections 2024-04-25T08:42:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.136+0000 7fa1aa3b7640 1 --2- 172.21.15.38:0/3548494221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1a415ef50 0x7fa1a40ccaf0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.136+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/3548494221 >> 172.21.15.38:0/3548494221 conn(0x7fa1a40b2060 msgr2=0x7fa1a40b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.136+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/3548494221 shutdown_connections 2024-04-25T08:42:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.136+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/3548494221 wait complete. 2024-04-25T08:42:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.137+0000 7fa1aa3b7640 1 Processor -- start 2024-04-25T08:42:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.137+0000 7fa1aa3b7640 1 -- start start 2024-04-25T08:42:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1aa3b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1a406b2f0 0x7fa1a41f09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1aa3b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a40b2b80 0x7fa1a415b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1aa3b7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1a415ef50 0x7fa1a415b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1aa3b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40b0bb0 con 0x7fa1a40b2b80 2024-04-25T08:42:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1aa3b7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40b0a30 con 0x7fa1a406b2f0 2024-04-25T08:42:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1aa3b7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40b0d30 con 0x7fa1a415ef50 2024-04-25T08:42:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a40b2b80 0x7fa1a415b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1a37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a40b2b80 0x7fa1a415b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34956/0 (socket says 172.21.15.38:34956) 2024-04-25T08:42:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1a37fe640 1 -- 172.21.15.38:0/2311193385 learned_addr learned my addr 172.21.15.38:0/2311193385 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1a3fff640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1a406b2f0 0x7fa1a41f09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1a8954640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1a415ef50 0x7fa1a415b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1a37fe640 1 -- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1a415ef50 msgr2=0x7fa1a415b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1a37fe640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1a415ef50 0x7fa1a415b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1a37fe640 1 -- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1a406b2f0 msgr2=0x7fa1a41f09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1a37fe640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1a406b2f0 0x7fa1a41f09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1a37fe640 1 -- 172.21.15.38:0/2311193385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a415da00 con 0x7fa1a40b2b80 2024-04-25T08:42:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1a37fe640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a40b2b80 0x7fa1a415b1e0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fa19800c490 tx=0x7fa19800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.138+0000 7fa1a0ff9640 1 -- 172.21.15.38:0/2311193385 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1980052d0 con 0x7fa1a40b2b80 2024-04-25T08:42:41.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.139+0000 7fa1a0ff9640 1 -- 172.21.15.38:0/2311193385 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa198005470 con 0x7fa1a40b2b80 2024-04-25T08:42:41.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.139+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/2311193385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a415dce0 con 0x7fa1a40b2b80 2024-04-25T08:42:41.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.139+0000 7fa1a0ff9640 1 -- 172.21.15.38:0/2311193385 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa198012850 con 0x7fa1a40b2b80 2024-04-25T08:42:41.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.139+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/2311193385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a415c3b0 con 0x7fa1a40b2b80 2024-04-25T08:42:41.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.140+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/2311193385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a404ef90 con 0x7fa1a40b2b80 2024-04-25T08:42:41.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.140+0000 7fa1a0ff9640 1 -- 172.21.15.38:0/2311193385 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1980129f0 con 0x7fa1a40b2b80 2024-04-25T08:42:41.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.141+0000 7fa1a0ff9640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa19006b4d0 0x7fa19006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.141+0000 7fa1a0ff9640 1 -- 172.21.15.38:0/2311193385 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa198089b70 con 0x7fa1a40b2b80 2024-04-25T08:42:41.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.141+0000 7fa1a3fff640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa19006b4d0 0x7fa19006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.142+0000 7fa1a3fff640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa19006b4d0 0x7fa19006d990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa194034000 tx=0x7fa19402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.137 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.145+0000 7fa1a0ff9640 1 -- 172.21.15.38:0/2311193385 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1980585a0 con 0x7fa1a40b2b80 2024-04-25T08:42:41.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.324+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/2311193385 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1a415e200 con 0x7fa19006b4d0 2024-04-25T08:42:41.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.325+0000 7fa1a0ff9640 1 -- 172.21.15.38:0/2311193385 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa1a415e200 con 0x7fa19006b4d0 2024-04-25T08:42:41.317 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T08:42:41.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.326+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa19006b4d0 msgr2=0x7fa19006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.326+0000 7fa1aa3b7640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa19006b4d0 0x7fa19006d990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fa194034000 tx=0x7fa19402e000 comp rx=0 tx=0).stop 2024-04-25T08:42:41.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.326+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/2311193385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a40b2b80 msgr2=0x7fa1a415b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.326+0000 7fa1aa3b7640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a40b2b80 0x7fa1a415b1e0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fa19800c490 tx=0x7fa19800c950 comp rx=0 tx=0).stop 2024-04-25T08:42:41.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.327+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/2311193385 shutdown_connections 2024-04-25T08:42:41.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.327+0000 7fa1aa3b7640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fa19006b4d0 0x7fa19006d990 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.327+0000 7fa1aa3b7640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa1a415ef50 0x7fa1a415b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.327+0000 7fa1aa3b7640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa1a40b2b80 0x7fa1a415b1e0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.327+0000 7fa1aa3b7640 1 --2- 172.21.15.38:0/2311193385 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa1a406b2f0 0x7fa1a41f09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.327+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/2311193385 >> 172.21.15.38:0/2311193385 conn(0x7fa1a40b2060 msgr2=0x7fa1a40b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.327+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/2311193385 shutdown_connections 2024-04-25T08:42:41.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.327+0000 7fa1aa3b7640 1 -- 172.21.15.38:0/2311193385 wait complete. 2024-04-25T08:42:41.330 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T08:42:41.331 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:42:41.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.444+0000 7f2eb341f640 1 Processor -- start 2024-04-25T08:42:41.437 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.444+0000 7f2eb341f640 1 -- start start 2024-04-25T08:42:41.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.445+0000 7f2eb341f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2eac0d0870 0x7f2eac0d0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.445+0000 7f2eb341f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eac0d11c0 con 0x7f2eac1612c0 2024-04-25T08:42:41.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.445+0000 7f2eb341f640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eac0b9880 con 0x7f2eac0b93f0 2024-04-25T08:42:41.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.445+0000 7f2eb341f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eac0b7200 con 0x7f2eac0d0870 2024-04-25T08:42:41.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.445+0000 7f2eb11bb640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2eac1612c0 0x7f2eac0b6a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33876/0 (socket says 172.21.15.38:33876) 2024-04-25T08:42:41.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.445+0000 7f2eb11bb640 1 -- 172.21.15.38:0/784378504 learned_addr learned my addr 172.21.15.38:0/784378504 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.445+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740090986 0 0) 0x7f2ea8001110 con 0x7f2eac1612c0 2024-04-25T08:42:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.445+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e98003520 con 0x7f2eac1612c0 2024-04-25T08:42:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.445+0000 7f2eb19bc640 1 --2- 172.21.15.38:0/784378504 >> v2:172.21.15.107:3300/0 conn(0x7f2eac0d0870 0x7f2eac0d0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1417594027 0 0) 0x7f2e98003520 con 0x7f2eac1612c0 2024-04-25T08:42:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 >> v2:172.21.15.107:3300/0 conn(0x7f2eac0d0870 msgr2=0x7f2eac0d0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 --2- 172.21.15.38:0/784378504 >> v2:172.21.15.107:3300/0 conn(0x7f2eac0d0870 0x7f2eac0d0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 >> v1:172.21.15.107:6789/0 conn(0x7f2eac0b93f0 legacy=0x7f2eac0d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac0b74e0 con 0x7f2eac1612c0 2024-04-25T08:42:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7f2ea8001840 con 0x7f2eac1612c0 2024-04-25T08:42:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 >> v1:172.21.15.38:6789/0 conn(0x7f2eac1612c0 legacy=0x7f2eac0b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 --2- 172.21.15.38:0/784378504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e980039b0 0x7f2e98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 --2- 172.21.15.38:0/784378504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e98004800 0x7f2e98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 --2- 172.21.15.38:0/784378504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e980051c0 0x7f2e9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0b7200 con 0x7f2e980039b0 2024-04-25T08:42:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9800c160 con 0x7f2e980051c0 2024-04-25T08:42:41.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9800c360 con 0x7f2e98004800 2024-04-25T08:42:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2eb11bb640 1 --2- 172.21.15.38:0/784378504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e980039b0 0x7f2e98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2eb09ba640 1 --2- 172.21.15.38:0/784378504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e98004800 0x7f2e98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2eb19bc640 1 --2- 172.21.15.38:0/784378504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e980051c0 0x7f2e9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2eb11bb640 1 -- 172.21.15.38:0/784378504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e98004800 msgr2=0x7f2e98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2eb11bb640 1 --2- 172.21.15.38:0/784378504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2e98004800 0x7f2e98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2eb11bb640 1 -- 172.21.15.38:0/784378504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e980051c0 msgr2=0x7f2e9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2eb11bb640 1 --2- 172.21.15.38:0/784378504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e980051c0 0x7f2e9800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2eb19bc640 1 -- 172.21.15.38:0/784378504 reap_dead start 2024-04-25T08:42:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.446+0000 7f2eb11bb640 1 -- 172.21.15.38:0/784378504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e98003520 con 0x7f2e980039b0 2024-04-25T08:42:41.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.447+0000 7f2eb11bb640 1 --2- 172.21.15.38:0/784378504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e980039b0 0x7f2e98003dd0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800c4c0 tx=0x7f2e98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293c2b77b5a364a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.447+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800c0e0 con 0x7f2e980039b0 2024-04-25T08:42:41.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.447+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea8034070 con 0x7f2e980039b0 2024-04-25T08:42:41.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.447+0000 7f2ea3fff640 1 -- 172.21.15.38:0/784378504 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8005d70 con 0x7f2e980039b0 2024-04-25T08:42:41.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.447+0000 7f2eb341f640 1 -- 172.21.15.38:0/784378504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e980039b0 msgr2=0x7f2e98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.447+0000 7f2eb341f640 1 --2- 172.21.15.38:0/784378504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e980039b0 0x7f2e98003dd0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f2ea800c4c0 tx=0x7f2e98005830 comp rx=0 tx=0).stop 2024-04-25T08:42:41.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.448+0000 7f2eb341f640 1 -- 172.21.15.38:0/784378504 shutdown_connections 2024-04-25T08:42:41.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.448+0000 7f2eb341f640 1 --2- 172.21.15.38:0/784378504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2e980039b0 0x7f2e98003dd0 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.448+0000 7f2eb341f640 1 -- 172.21.15.38:0/784378504 >> 172.21.15.38:0/784378504 conn(0x7f2eac066d10 msgr2=0x7f2eac067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.448+0000 7f2eb341f640 1 -- 172.21.15.38:0/784378504 shutdown_connections 2024-04-25T08:42:41.445 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.448+0000 7f2eb341f640 1 -- 172.21.15.38:0/784378504 wait complete. 2024-04-25T08:42:41.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.449+0000 7f2eb341f640 1 Processor -- start 2024-04-25T08:42:41.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.449+0000 7f2eb341f640 1 -- start start 2024-04-25T08:42:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.449+0000 7f2eb341f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e980039b0 0x7f2eac1f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.449+0000 7f2eb341f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2eac147eb0 0x7f2eac154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.449+0000 7f2eb341f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2eac154580 0x7f2eac0cc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb341f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0b7d80 con 0x7f2eac147eb0 2024-04-25T08:42:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb341f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0b7aa0 con 0x7f2e980039b0 2024-04-25T08:42:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb341f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac0d0c80 con 0x7f2eac154580 2024-04-25T08:42:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb09ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2eac147eb0 0x7f2eac154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb09ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2eac147eb0 0x7f2eac154040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34966/0 (socket says 172.21.15.38:34966) 2024-04-25T08:42:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb09ba640 1 -- 172.21.15.38:0/717039969 learned_addr learned my addr 172.21.15.38:0/717039969 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb19bc640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2eac154580 0x7f2eac0cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb11bb640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e980039b0 0x7f2eac1f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb09ba640 1 -- 172.21.15.38:0/717039969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2eac154580 msgr2=0x7f2eac0cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb09ba640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2eac154580 0x7f2eac0cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb09ba640 1 -- 172.21.15.38:0/717039969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e980039b0 msgr2=0x7f2eac1f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb09ba640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2e980039b0 0x7f2eac1f38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb09ba640 1 -- 172.21.15.38:0/717039969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac148330 con 0x7f2eac147eb0 2024-04-25T08:42:41.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb09ba640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2eac147eb0 0x7f2eac154040 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00f620 tx=0x7f2e9c00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2ea1ffb640 1 -- 172.21.15.38:0/717039969 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c0052e0 con 0x7f2eac147eb0 2024-04-25T08:42:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2ea1ffb640 1 -- 172.21.15.38:0/717039969 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c018070 con 0x7f2eac147eb0 2024-04-25T08:42:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb341f640 1 -- 172.21.15.38:0/717039969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eac151930 con 0x7f2eac147eb0 2024-04-25T08:42:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2ea1ffb640 1 -- 172.21.15.38:0/717039969 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c003810 con 0x7f2eac147eb0 2024-04-25T08:42:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.450+0000 7f2eb341f640 1 -- 172.21.15.38:0/717039969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eac151ef0 con 0x7f2eac147eb0 2024-04-25T08:42:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.451+0000 7f2eb341f640 1 -- 172.21.15.38:0/717039969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2eac0ca660 con 0x7f2eac147eb0 2024-04-25T08:42:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.452+0000 7f2ea1ffb640 1 -- 172.21.15.38:0/717039969 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e9c0039b0 con 0x7f2eac147eb0 2024-04-25T08:42:41.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.452+0000 7f2ea1ffb640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2e9006b4d0 0x7f2e9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.452+0000 7f2ea1ffb640 1 -- 172.21.15.38:0/717039969 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2e9c089dd0 con 0x7f2eac147eb0 2024-04-25T08:42:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.452+0000 7f2ea1ffb640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] conn(0x7f2e90071970 0x7f2e90073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.452+0000 7f2ea1ffb640 1 -- 172.21.15.38:0/717039969 --> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e9c00ae50 con 0x7f2e90071970 2024-04-25T08:42:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.452+0000 7f2ea1ffb640 1 -- 172.21.15.38:0/717039969 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e9c08c4d0 con 0x7f2eac147eb0 2024-04-25T08:42:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.452+0000 7f2eb11bb640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2e9006b4d0 0x7f2e9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.452+0000 7f2eb19bc640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] conn(0x7f2e90071970 0x7f2e90073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.453+0000 7f2eb19bc640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] conn(0x7f2e90071970 0x7f2e90073db0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.453+0000 7f2eb11bb640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2e9006b4d0 0x7f2e9006d990 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8033000 tx=0x7f2ea802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.453+0000 7f2ea1ffb640 1 -- 172.21.15.38:0/717039969 <== osd.3 v2:172.21.15.38:6816/831384614 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2e9c00ae50 con 0x7f2e90071970 2024-04-25T08:42:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2eb341f640 1 -- 172.21.15.38:0/717039969 --> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2eac068e80 con 0x7f2e90071970 2024-04-25T08:42:41.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2ea1ffb640 1 -- 172.21.15.38:0/717039969 <== osd.3 v2:172.21.15.38:6816/831384614 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2eac068e80 con 0x7f2e90071970 2024-04-25T08:42:41.459 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:42:41.460 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:42:41.460 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:42:41.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2eb341f640 1 -- 172.21.15.38:0/717039969 >> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] conn(0x7f2e90071970 msgr2=0x7f2e90073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2eb341f640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.38:6816/831384614,v1:172.21.15.38:6817/831384614] conn(0x7f2e90071970 0x7f2e90073db0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2eb341f640 1 -- 172.21.15.38:0/717039969 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2e9006b4d0 msgr2=0x7f2e9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2eb341f640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2e9006b4d0 0x7f2e9006d990 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8033000 tx=0x7f2ea802e000 comp rx=0 tx=0).stop 2024-04-25T08:42:41.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2eb341f640 1 -- 172.21.15.38:0/717039969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2eac147eb0 msgr2=0x7f2eac154040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2eb341f640 1 --2- 172.21.15.38:0/717039969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2eac147eb0 0x7f2eac154040 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00f620 tx=0x7f2e9c00fae0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2eb19bc640 1 -- 172.21.15.38:0/717039969 reap_dead start 2024-04-25T08:42:41.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2eb341f640 1 -- 172.21.15.38:0/717039969 shutdown_connections 2024-04-25T08:42:41.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2eb341f640 1 -- 172.21.15.38:0/717039969 >> 172.21.15.38:0/717039969 conn(0x7f2eac066d10 msgr2=0x7f2eac1601f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2eb341f640 1 -- 172.21.15.38:0/717039969 shutdown_connections 2024-04-25T08:42:41.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.467+0000 7f2eb341f640 1 -- 172.21.15.38:0/717039969 wait complete. 2024-04-25T08:42:41.470 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T08:42:41.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.580+0000 7f60095f5640 1 Processor -- start 2024-04-25T08:42:41.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.580+0000 7f60095f5640 1 -- start start 2024-04-25T08:42:41.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f60095f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f600415db80 0x7f60041616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f60095f5640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f60040d0830 0x7f60040d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f60095f5640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6004162030 con 0x7f60040b9600 2024-04-25T08:42:41.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f60095f5640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60041625f0 con 0x7f60040d0830 2024-04-25T08:42:41.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f60095f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6004162310 con 0x7f600415db80 2024-04-25T08:42:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f60027fc640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f60040b9600 0x7f60040d0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33880/0 (socket says 172.21.15.38:33880) 2024-04-25T08:42:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f60027fc640 1 -- 172.21.15.38:0/3373428808 learned_addr learned my addr 172.21.15.38:0/3373428808 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f60037fe640 1 --2- 172.21.15.38:0/3373428808 >> v2:172.21.15.107:3300/0 conn(0x7f60040d0830 0x7f60040d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f6002ffd640 1 --2- 172.21.15.38:0/3373428808 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f600415db80 0x7f60041616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f6001ffb640 1 -- 172.21.15.38:0/3373428808 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337227726 0 0) 0x7f6004162030 con 0x7f60040b9600 2024-04-25T08:42:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f6001ffb640 1 -- 172.21.15.38:0/3373428808 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff0003520 con 0x7f60040b9600 2024-04-25T08:42:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f60037fe640 1 -- 172.21.15.38:0/3373428808 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f600415db80 msgr2=0x7f60041616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:41.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f60037fe640 1 --2- 172.21.15.38:0/3373428808 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f600415db80 0x7f60041616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f60037fe640 1 -- 172.21.15.38:0/3373428808 >> v1:172.21.15.38:6789/0 conn(0x7f60040b9600 legacy=0x7f60040d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.581+0000 7f60037fe640 1 -- 172.21.15.38:0/3373428808 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040d1ab0 con 0x7f60040d0830 2024-04-25T08:42:41.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.582+0000 7f60037fe640 1 --2- 172.21.15.38:0/3373428808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60040d0830 0x7f60040d22b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009d90 tx=0x7f5ff402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6671fd11f1750e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.582+0000 7f6001ffb640 1 -- 172.21.15.38:0/3373428808 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4031600 con 0x7f60040d0830 2024-04-25T08:42:41.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.582+0000 7f6001ffb640 1 -- 172.21.15.38:0/3373428808 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff40317a0 con 0x7f60040d0830 2024-04-25T08:42:41.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.582+0000 7f6001ffb640 1 -- 172.21.15.38:0/3373428808 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4031b70 con 0x7f60040d0830 2024-04-25T08:42:41.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.583+0000 7f60095f5640 1 -- 172.21.15.38:0/3373428808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60040d0830 msgr2=0x7f60040d22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.583+0000 7f60095f5640 1 --2- 172.21.15.38:0/3373428808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60040d0830 0x7f60040d22b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009d90 tx=0x7f5ff402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.583+0000 7f60095f5640 1 -- 172.21.15.38:0/3373428808 shutdown_connections 2024-04-25T08:42:41.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.583+0000 7f60095f5640 1 --2- 172.21.15.38:0/3373428808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60040d0830 0x7f60040d22b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.583+0000 7f60095f5640 1 --2- 172.21.15.38:0/3373428808 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f600415db80 0x7f60041616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.583+0000 7f60095f5640 1 -- 172.21.15.38:0/3373428808 >> 172.21.15.38:0/3373428808 conn(0x7f60040668f0 msgr2=0x7f6004066d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.583+0000 7f60095f5640 1 -- 172.21.15.38:0/3373428808 shutdown_connections 2024-04-25T08:42:41.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.583+0000 7f60095f5640 1 -- 172.21.15.38:0/3373428808 wait complete. 2024-04-25T08:42:41.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.584+0000 7f60095f5640 1 Processor -- start 2024-04-25T08:42:41.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.584+0000 7f60095f5640 1 -- start start 2024-04-25T08:42:41.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60095f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60040b9600 0x7f60040c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60095f5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60040d0830 0x7f60040cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60095f5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f600415db80 0x7f600414ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60095f5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6004161da0 con 0x7f600415db80 2024-04-25T08:42:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60095f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6004161c20 con 0x7f60040b9600 2024-04-25T08:42:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60095f5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60040d27f0 con 0x7f60040d0830 2024-04-25T08:42:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f600415db80 0x7f600414ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f600415db80 0x7f600414ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34976/0 (socket says 172.21.15.38:34976) 2024-04-25T08:42:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60037fe640 1 -- 172.21.15.38:0/2226440297 learned_addr learned my addr 172.21.15.38:0/2226440297 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f6002ffd640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60040b9600 0x7f60040c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60027fc640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60040d0830 0x7f60040cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60037fe640 1 -- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60040d0830 msgr2=0x7f60040cb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60037fe640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60040d0830 0x7f60040cb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60037fe640 1 -- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60040b9600 msgr2=0x7f60040c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60037fe640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60040b9600 0x7f60040c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f60037fe640 1 -- 172.21.15.38:0/2226440297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040cbe30 con 0x7f600415db80 2024-04-25T08:42:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.585+0000 7f6002ffd640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60040b9600 0x7f60040c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.586+0000 7f60037fe640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f600415db80 0x7f600414ab30 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009eb0 tx=0x7f5ff4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.586+0000 7f5ffb7fe640 1 -- 172.21.15.38:0/2226440297 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4040070 con 0x7f600415db80 2024-04-25T08:42:41.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.586+0000 7f5ffb7fe640 1 -- 172.21.15.38:0/2226440297 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff403c830 con 0x7f600415db80 2024-04-25T08:42:41.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.586+0000 7f60095f5640 1 -- 172.21.15.38:0/2226440297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60040c92a0 con 0x7f600415db80 2024-04-25T08:42:41.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.586+0000 7f5ffb7fe640 1 -- 172.21.15.38:0/2226440297 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff403cbe0 con 0x7f600415db80 2024-04-25T08:42:41.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.586+0000 7f60095f5640 1 -- 172.21.15.38:0/2226440297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60040c9860 con 0x7f600415db80 2024-04-25T08:42:41.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.587+0000 7f60095f5640 1 -- 172.21.15.38:0/2226440297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600404ef90 con 0x7f600415db80 2024-04-25T08:42:41.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.587+0000 7f5ffb7fe640 1 -- 172.21.15.38:0/2226440297 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ff4039040 con 0x7f600415db80 2024-04-25T08:42:41.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.588+0000 7f5ffb7fe640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5fe406b4d0 0x7f5fe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.588+0000 7f5ffb7fe640 1 -- 172.21.15.38:0/2226440297 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5ff40ab490 con 0x7f600415db80 2024-04-25T08:42:41.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.588+0000 7f6002ffd640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5fe406b4d0 0x7f5fe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.589+0000 7f6002ffd640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5fe406b4d0 0x7f5fe406d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6004161a00 tx=0x7f5ffc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.592+0000 7f5ffb7fe640 1 -- 172.21.15.38:0/2226440297 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ff4079e30 con 0x7f600415db80 2024-04-25T08:42:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.772+0000 7f60095f5640 1 -- 172.21.15.38:0/2226440297 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f60040d14a0 con 0x7f5fe406b4d0 2024-04-25T08:42:41.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.772+0000 7f5ffb7fe640 1 -- 172.21.15.38:0/2226440297 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f60040d14a0 con 0x7f5fe406b4d0 2024-04-25T08:42:41.765 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T08:42:41.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.774+0000 7f60095f5640 1 -- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5fe406b4d0 msgr2=0x7f5fe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.774+0000 7f60095f5640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5fe406b4d0 0x7f5fe406d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f6004161a00 tx=0x7f5ffc009190 comp rx=0 tx=0).stop 2024-04-25T08:42:41.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.774+0000 7f60095f5640 1 -- 172.21.15.38:0/2226440297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f600415db80 msgr2=0x7f600414ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.774+0000 7f60095f5640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f600415db80 0x7f600414ab30 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4009eb0 tx=0x7f5ff4004a40 comp rx=0 tx=0).stop 2024-04-25T08:42:41.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.774+0000 7f60095f5640 1 -- 172.21.15.38:0/2226440297 shutdown_connections 2024-04-25T08:42:41.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.774+0000 7f60095f5640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f600415db80 0x7f600414ab30 unknown :-1 s=CLOSED pgs=2181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.774+0000 7f60095f5640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f5fe406b4d0 0x7f5fe406d990 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.774+0000 7f60095f5640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f60040d0830 0x7f60040cb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.774+0000 7f60095f5640 1 --2- 172.21.15.38:0/2226440297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f60040b9600 0x7f60040c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.774+0000 7f60095f5640 1 -- 172.21.15.38:0/2226440297 >> 172.21.15.38:0/2226440297 conn(0x7f60040668f0 msgr2=0x7f60040d2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.774+0000 7f60095f5640 1 -- 172.21.15.38:0/2226440297 shutdown_connections 2024-04-25T08:42:41.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.774+0000 7f60095f5640 1 -- 172.21.15.38:0/2226440297 wait complete. 2024-04-25T08:42:41.777 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T08:42:41.777 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:42:41.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.891+0000 7f41b5c07640 1 Processor -- start 2024-04-25T08:42:41.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.891+0000 7f41b5c07640 1 -- start start 2024-04-25T08:42:41.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41b5c07640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f41b01612c0 0x7f41b00b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41b5c07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41b00b6a00 0x7f41b00d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41b5c07640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f41b00d0870 0x7f41b00d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41b5c07640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b00d11e0 con 0x7f41b00d0870 2024-04-25T08:42:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41b5c07640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b00b7200 con 0x7f41b01612c0 2024-04-25T08:42:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41b5c07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00b74e0 con 0x7f41b00b6a00 2024-04-25T08:42:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41affff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f41b00d0870 0x7f41b00d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41affff640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f41b00d0870 0x7f41b00d0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34984/0 (socket says 172.21.15.38:34984) 2024-04-25T08:42:41.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41affff640 1 -- 172.21.15.38:0/1941550238 learned_addr learned my addr 172.21.15.38:0/1941550238 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41af7fe640 1 --2- 172.21.15.38:0/1941550238 >> v2:172.21.15.107:3300/0 conn(0x7f41b01612c0 0x7f41b00b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41aeffd640 1 --2- 172.21.15.38:0/1941550238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41b00b6a00 0x7f41b00d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41affff640 1 -- 172.21.15.38:0/1941550238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41b00b6a00 msgr2=0x7f41b00d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41affff640 1 --2- 172.21.15.38:0/1941550238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41b00b6a00 0x7f41b00d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41affff640 1 -- 172.21.15.38:0/1941550238 >> v2:172.21.15.107:3300/0 conn(0x7f41b01612c0 msgr2=0x7f41b00b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41affff640 1 --2- 172.21.15.38:0/1941550238 >> v2:172.21.15.107:3300/0 conn(0x7f41b01612c0 0x7f41b00b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.892+0000 7f41affff640 1 -- 172.21.15.38:0/1941550238 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b00b77c0 con 0x7f41b00d0870 2024-04-25T08:42:41.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.893+0000 7f41affff640 1 --2- 172.21.15.38:0/1941550238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b00d0870 0x7f41b00d0ca0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f41b00b7410 tx=0x7f41a802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed48ebeee7dc3f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.893+0000 7f41ae7fc640 1 -- 172.21.15.38:0/1941550238 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a8036070 con 0x7f41b00d0870 2024-04-25T08:42:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.893+0000 7f41ae7fc640 1 -- 172.21.15.38:0/1941550238 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a8031600 con 0x7f41b00d0870 2024-04-25T08:42:41.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.893+0000 7f41ae7fc640 1 -- 172.21.15.38:0/1941550238 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a80319d0 con 0x7f41b00d0870 2024-04-25T08:42:41.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.894+0000 7f41b5c07640 1 -- 172.21.15.38:0/1941550238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b00d0870 msgr2=0x7f41b00d0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.894+0000 7f41b5c07640 1 --2- 172.21.15.38:0/1941550238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b00d0870 0x7f41b00d0ca0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f41b00b7410 tx=0x7f41a802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.894+0000 7f41b5c07640 1 -- 172.21.15.38:0/1941550238 shutdown_connections 2024-04-25T08:42:41.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.894+0000 7f41b5c07640 1 --2- 172.21.15.38:0/1941550238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b00d0870 0x7f41b00d0ca0 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.894+0000 7f41b5c07640 1 --2- 172.21.15.38:0/1941550238 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41b00b6a00 0x7f41b00d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.894+0000 7f41b5c07640 1 --2- 172.21.15.38:0/1941550238 >> v2:172.21.15.107:3300/0 conn(0x7f41b01612c0 0x7f41b00b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.894+0000 7f41b5c07640 1 -- 172.21.15.38:0/1941550238 >> 172.21.15.38:0/1941550238 conn(0x7f41b0066d10 msgr2=0x7f41b0067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.894+0000 7f41b5c07640 1 -- 172.21.15.38:0/1941550238 shutdown_connections 2024-04-25T08:42:41.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.894+0000 7f41b5c07640 1 -- 172.21.15.38:0/1941550238 wait complete. 2024-04-25T08:42:41.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.895+0000 7f41b5c07640 1 Processor -- start 2024-04-25T08:42:41.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.895+0000 7f41b5c07640 1 -- start start 2024-04-25T08:42:41.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.896+0000 7f41b5c07640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b00b6a00 0x7f41b01f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.896+0000 7f41b5c07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b00d0870 0x7f41b0153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.896+0000 7f41b5c07640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41b01612c0 0x7f41b015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.896+0000 7f41b5c07640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00d2170 con 0x7f41b00b6a00 2024-04-25T08:42:41.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.896+0000 7f41b5c07640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00b7d80 con 0x7f41b00d0870 2024-04-25T08:42:41.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.896+0000 7f41b5c07640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00b6e60 con 0x7f41b01612c0 2024-04-25T08:42:41.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.896+0000 7f41af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b00b6a00 0x7f41b01f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.896+0000 7f41af7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b00b6a00 0x7f41b01f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34996/0 (socket says 172.21.15.38:34996) 2024-04-25T08:42:41.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.896+0000 7f41af7fe640 1 -- 172.21.15.38:0/3501735804 learned_addr learned my addr 172.21.15.38:0/3501735804 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:41.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.896+0000 7f41affff640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41b01612c0 0x7f41b015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.896+0000 7f41aeffd640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b00d0870 0x7f41b0153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.897+0000 7f41af7fe640 1 -- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41b01612c0 msgr2=0x7f41b015da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.897+0000 7f41af7fe640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f41b01612c0 0x7f41b015da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.897+0000 7f41af7fe640 1 -- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b00d0870 msgr2=0x7f41b0153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.897+0000 7f41af7fe640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b00d0870 0x7f41b0153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.893 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.897+0000 7f41af7fe640 1 -- 172.21.15.38:0/3501735804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b00c7460 con 0x7f41b00b6a00 2024-04-25T08:42:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.897+0000 7f41aeffd640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f41b00d0870 0x7f41b0153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.897+0000 7f41af7fe640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b00b6a00 0x7f41b01f7f40 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f41a000e9b0 tx=0x7f41a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.897+0000 7f4197fff640 1 -- 172.21.15.38:0/3501735804 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a0017070 con 0x7f41b00b6a00 2024-04-25T08:42:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.897+0000 7f4197fff640 1 -- 172.21.15.38:0/3501735804 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a000c9a0 con 0x7f41b00b6a00 2024-04-25T08:42:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.897+0000 7f41b5c07640 1 -- 172.21.15.38:0/3501735804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b0156970 con 0x7f41b00b6a00 2024-04-25T08:42:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.897+0000 7f4197fff640 1 -- 172.21.15.38:0/3501735804 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a000cd90 con 0x7f41b00b6a00 2024-04-25T08:42:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.897+0000 7f41b5c07640 1 -- 172.21.15.38:0/3501735804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b0156c50 con 0x7f41b00b6a00 2024-04-25T08:42:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.898+0000 7f41b5c07640 1 -- 172.21.15.38:0/3501735804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41b0068e80 con 0x7f41b00b6a00 2024-04-25T08:42:41.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.899+0000 7f4197fff640 1 -- 172.21.15.38:0/3501735804 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41a001a070 con 0x7f41b00b6a00 2024-04-25T08:42:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.899+0000 7f4197fff640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f419c06b500 0x7f419c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.899+0000 7f4197fff640 1 -- 172.21.15.38:0/3501735804 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f41a0089fd0 con 0x7f41b00b6a00 2024-04-25T08:42:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.899+0000 7f4197fff640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x7f419c0719a0 0x7f419c073de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.899+0000 7f4197fff640 1 -- 172.21.15.38:0/3501735804 --> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f419c0744b0 con 0x7f419c0719a0 2024-04-25T08:42:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.899+0000 7f4197fff640 1 -- 172.21.15.38:0/3501735804 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41a0091050 con 0x7f41b00b6a00 2024-04-25T08:42:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.899+0000 7f41aeffd640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f419c06b500 0x7f419c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.899+0000 7f41affff640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x7f419c0719a0 0x7f419c073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:41.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.900+0000 7f41aeffd640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f419c06b500 0x7f419c06d9c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f41b01552f0 tx=0x7f41a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.900+0000 7f41affff640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x7f419c0719a0 0x7f419c073de0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:41.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.901+0000 7f4197fff640 1 -- 172.21.15.38:0/3501735804 <== osd.4 v2:172.21.15.107:6800/592699329 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f419c0744b0 con 0x7f419c0719a0 2024-04-25T08:42:41.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.914+0000 7f41b5c07640 1 -- 172.21.15.38:0/3501735804 --> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f41b01561d0 con 0x7f419c0719a0 2024-04-25T08:42:41.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.914+0000 7f4197fff640 1 -- 172.21.15.38:0/3501735804 <== osd.4 v2:172.21.15.107:6800/592699329 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f41b01561d0 con 0x7f419c0719a0 2024-04-25T08:42:41.907 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:42:41.907 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:42:41.907 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:42:41.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.914+0000 7f41b5c07640 1 -- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x7f419c0719a0 msgr2=0x7f419c073de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.914+0000 7f41b5c07640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:6800/592699329,v1:172.21.15.107:6801/592699329] conn(0x7f419c0719a0 0x7f419c073de0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:41.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.915+0000 7f41b5c07640 1 -- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f419c06b500 msgr2=0x7f419c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.915+0000 7f41b5c07640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f419c06b500 0x7f419c06d9c0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f41b01552f0 tx=0x7f41a4008040 comp rx=0 tx=0).stop 2024-04-25T08:42:41.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.915+0000 7f41b5c07640 1 -- 172.21.15.38:0/3501735804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b00b6a00 msgr2=0x7f41b01f7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:41.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.915+0000 7f41b5c07640 1 --2- 172.21.15.38:0/3501735804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f41b00b6a00 0x7f41b01f7f40 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f41a000e9b0 tx=0x7f41a000ee70 comp rx=0 tx=0).stop 2024-04-25T08:42:41.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.915+0000 7f41affff640 1 -- 172.21.15.38:0/3501735804 reap_dead start 2024-04-25T08:42:41.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.915+0000 7f41b5c07640 1 -- 172.21.15.38:0/3501735804 shutdown_connections 2024-04-25T08:42:41.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.915+0000 7f41b5c07640 1 -- 172.21.15.38:0/3501735804 >> 172.21.15.38:0/3501735804 conn(0x7f41b0066d10 msgr2=0x7f41b00d3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:41.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.915+0000 7f41b5c07640 1 -- 172.21.15.38:0/3501735804 shutdown_connections 2024-04-25T08:42:41.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:41.915+0000 7f41b5c07640 1 -- 172.21.15.38:0/3501735804 wait complete. 2024-04-25T08:42:41.917 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T08:42:42.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.027+0000 7fd34ea3a640 1 Processor -- start 2024-04-25T08:42:42.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.027+0000 7fd34ea3a640 1 -- start start 2024-04-25T08:42:42.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.028+0000 7fd34ea3a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3480b9180 0x7fd3480d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.028+0000 7fd34ea3a640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fd3480b2ab0 0x7fd3480d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.028+0000 7fd34ea3a640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3480b9610 con 0x7fd348160ef0 2024-04-25T08:42:42.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.028+0000 7fd34ea3a640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd348161cf0 con 0x7fd3480b2ab0 2024-04-25T08:42:42.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.028+0000 7fd34ea3a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481612d0 con 0x7fd3480b9180 2024-04-25T08:42:42.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.028+0000 7fd347fff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fd348160ef0 0x7fd3480d07b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33894/0 (socket says 172.21.15.38:33894) 2024-04-25T08:42:42.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.028+0000 7fd347fff640 1 -- 172.21.15.38:0/3123676430 learned_addr learned my addr 172.21.15.38:0/3123676430 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:42.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.028+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552009171 0 0) 0x7fd3480b9610 con 0x7fd348160ef0 2024-04-25T08:42:42.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.028+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd330003520 con 0x7fd348160ef0 2024-04-25T08:42:42.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.028+0000 7fd3477fe640 1 --2- 172.21.15.38:0/3123676430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3480b9180 0x7fd3480d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.028+0000 7fd34cfd7640 1 --2- 172.21.15.38:0/3123676430 >> v2:172.21.15.107:3300/0 conn(0x7fd3480b2ab0 0x7fd3480d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.028+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 49483415 0 0) 0x7fd330003520 con 0x7fd348160ef0 2024-04-25T08:42:42.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3480b9180 msgr2=0x7fd3480d1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 --2- 172.21.15.38:0/3123676430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3480b9180 0x7fd3480d1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 >> v2:172.21.15.107:3300/0 conn(0x7fd3480b2ab0 msgr2=0x7fd3480d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 --2- 172.21.15.38:0/3123676430 >> v2:172.21.15.107:3300/0 conn(0x7fd3480b2ab0 0x7fd3480d1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3481616a0 con 0x7fd348160ef0 2024-04-25T08:42:42.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fd338001670 con 0x7fd348160ef0 2024-04-25T08:42:42.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 >> v1:172.21.15.38:6789/0 conn(0x7fd348160ef0 legacy=0x7fd3480d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 --2- 172.21.15.38:0/3123676430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3300039b0 0x7fd330003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 --2- 172.21.15.38:0/3123676430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd330004800 0x7fd330004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 --2- 172.21.15.38:0/3123676430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3300051c0 0x7fd33000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd348161cf0 con 0x7fd3300039b0 2024-04-25T08:42:42.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481612d0 con 0x7fd330004800 2024-04-25T08:42:42.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33000bf00 con 0x7fd3300051c0 2024-04-25T08:42:42.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd347fff640 1 --2- 172.21.15.38:0/3123676430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3300039b0 0x7fd330003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd3477fe640 1 --2- 172.21.15.38:0/3123676430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd330004800 0x7fd330004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd34cfd7640 1 --2- 172.21.15.38:0/3123676430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3300051c0 0x7fd33000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd347fff640 1 -- 172.21.15.38:0/3123676430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3300051c0 msgr2=0x7fd33000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd347fff640 1 --2- 172.21.15.38:0/3123676430 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3300051c0 0x7fd33000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd347fff640 1 -- 172.21.15.38:0/3123676430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd330004800 msgr2=0x7fd330004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd347fff640 1 --2- 172.21.15.38:0/3123676430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd330004800 0x7fd330004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.029+0000 7fd347fff640 1 -- 172.21.15.38:0/3123676430 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd330003520 con 0x7fd3300039b0 2024-04-25T08:42:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.030+0000 7fd347fff640 1 --2- 172.21.15.38:0/3123676430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3300039b0 0x7fd330003dd0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fd338007fd0 tx=0x7fd330005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a7e6c215a0c5b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.030+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338007d50 con 0x7fd3300039b0 2024-04-25T08:42:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.030+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd338005ad0 con 0x7fd3300039b0 2024-04-25T08:42:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.030+0000 7fd346ffd640 1 -- 172.21.15.38:0/3123676430 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338007070 con 0x7fd3300039b0 2024-04-25T08:42:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.031+0000 7fd34ea3a640 1 -- 172.21.15.38:0/3123676430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3300039b0 msgr2=0x7fd330003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.031+0000 7fd34ea3a640 1 --2- 172.21.15.38:0/3123676430 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3300039b0 0x7fd330003dd0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7fd338007fd0 tx=0x7fd330005830 comp rx=0 tx=0).stop 2024-04-25T08:42:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.031+0000 7fd34cfd7640 1 -- 172.21.15.38:0/3123676430 reap_dead start 2024-04-25T08:42:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.031+0000 7fd34ea3a640 1 -- 172.21.15.38:0/3123676430 shutdown_connections 2024-04-25T08:42:42.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.031+0000 7fd34ea3a640 1 -- 172.21.15.38:0/3123676430 >> 172.21.15.38:0/3123676430 conn(0x7fd348066d10 msgr2=0x7fd348067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.031+0000 7fd34ea3a640 1 -- 172.21.15.38:0/3123676430 shutdown_connections 2024-04-25T08:42:42.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.031+0000 7fd34ea3a640 1 -- 172.21.15.38:0/3123676430 wait complete. 2024-04-25T08:42:42.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.032+0000 7fd34ea3a640 1 Processor -- start 2024-04-25T08:42:42.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.032+0000 7fd34ea3a640 1 -- start start 2024-04-25T08:42:42.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd34ea3a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3300039b0 0x7fd3480b7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd34ea3a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3481f31c0 0x7fd3481f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd34ea3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3480ce470 0x7fd34815a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd34ea3a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3480d1040 con 0x7fd3300039b0 2024-04-25T08:42:42.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd34ea3a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3480d0ec0 con 0x7fd3480ce470 2024-04-25T08:42:42.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd34ea3a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd348162180 con 0x7fd3481f31c0 2024-04-25T08:42:42.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd347fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3300039b0 0x7fd3480b7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd347fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3300039b0 0x7fd3480b7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35018/0 (socket says 172.21.15.38:35018) 2024-04-25T08:42:42.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd347fff640 1 -- 172.21.15.38:0/1250491504 learned_addr learned my addr 172.21.15.38:0/1250491504 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:42.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd3477fe640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3481f31c0 0x7fd3481f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd34cfd7640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3480ce470 0x7fd34815a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd347fff640 1 -- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3481f31c0 msgr2=0x7fd3481f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd347fff640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3481f31c0 0x7fd3481f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd347fff640 1 -- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3480ce470 msgr2=0x7fd34815a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd347fff640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3480ce470 0x7fd34815a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd347fff640 1 -- 172.21.15.38:0/1250491504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3480cc8b0 con 0x7fd3300039b0 2024-04-25T08:42:42.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd34cfd7640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3480ce470 0x7fd34815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:42.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd347fff640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3300039b0 0x7fd3480b7ca0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fd33802ae20 tx=0x7fd3480b97e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.034 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd344ff9640 1 -- 172.21.15.38:0/1250491504 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338004bf0 con 0x7fd3300039b0 2024-04-25T08:42:42.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.033+0000 7fd344ff9640 1 -- 172.21.15.38:0/1250491504 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd338043070 con 0x7fd3300039b0 2024-04-25T08:42:42.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.034+0000 7fd34ea3a640 1 -- 172.21.15.38:0/1250491504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3480ccb50 con 0x7fd3300039b0 2024-04-25T08:42:42.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.034+0000 7fd344ff9640 1 -- 172.21.15.38:0/1250491504 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338007070 con 0x7fd3300039b0 2024-04-25T08:42:42.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.034+0000 7fd34ea3a640 1 -- 172.21.15.38:0/1250491504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3481f3c80 con 0x7fd3300039b0 2024-04-25T08:42:42.035 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.034+0000 7fd34ea3a640 1 -- 172.21.15.38:0/1250491504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34815fb30 con 0x7fd3300039b0 2024-04-25T08:42:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.035+0000 7fd344ff9640 1 -- 172.21.15.38:0/1250491504 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd338007210 con 0x7fd3300039b0 2024-04-25T08:42:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.036+0000 7fd344ff9640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd33406b4d0 0x7fd33406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.036+0000 7fd344ff9640 1 -- 172.21.15.38:0/1250491504 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd3380aa6d0 con 0x7fd3300039b0 2024-04-25T08:42:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.036+0000 7fd3477fe640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd33406b4d0 0x7fd33406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.036+0000 7fd3477fe640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd33406b4d0 0x7fd33406d990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd3480b7a50 tx=0x7fd33c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.040+0000 7fd344ff9640 1 -- 172.21.15.38:0/1250491504 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd34815fb30 con 0x7fd3300039b0 2024-04-25T08:42:42.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.219+0000 7fd34ea3a640 1 -- 172.21.15.38:0/1250491504 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd34804ef90 con 0x7fd33406b4d0 2024-04-25T08:42:42.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.220+0000 7fd344ff9640 1 -- 172.21.15.38:0/1250491504 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd34804ef90 con 0x7fd33406b4d0 2024-04-25T08:42:42.213 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T08:42:42.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.222+0000 7fd34ea3a640 1 -- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd33406b4d0 msgr2=0x7fd33406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.222+0000 7fd34ea3a640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd33406b4d0 0x7fd33406d990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd3480b7a50 tx=0x7fd33c008040 comp rx=0 tx=0).stop 2024-04-25T08:42:42.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.222+0000 7fd34ea3a640 1 -- 172.21.15.38:0/1250491504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3300039b0 msgr2=0x7fd3480b7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.222+0000 7fd34ea3a640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3300039b0 0x7fd3480b7ca0 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7fd33802ae20 tx=0x7fd3480b97e0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.222+0000 7fd34ea3a640 1 -- 172.21.15.38:0/1250491504 shutdown_connections 2024-04-25T08:42:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.222+0000 7fd34ea3a640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd3480ce470 0x7fd34815a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.222+0000 7fd34ea3a640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fd33406b4d0 0x7fd33406d990 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.222+0000 7fd34ea3a640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd3481f31c0 0x7fd3481f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.222+0000 7fd34ea3a640 1 --2- 172.21.15.38:0/1250491504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3300039b0 0x7fd3480b7ca0 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.222+0000 7fd34ea3a640 1 -- 172.21.15.38:0/1250491504 >> 172.21.15.38:0/1250491504 conn(0x7fd348066d10 msgr2=0x7fd34815dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.222+0000 7fd34ea3a640 1 -- 172.21.15.38:0/1250491504 shutdown_connections 2024-04-25T08:42:42.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.222+0000 7fd34ea3a640 1 -- 172.21.15.38:0/1250491504 wait complete. 2024-04-25T08:42:42.225 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T08:42:42.226 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:42:42.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.339+0000 7f6e35bfe640 1 Processor -- start 2024-04-25T08:42:42.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.339+0000 7f6e35bfe640 1 -- start start 2024-04-25T08:42:42.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e35bfe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6e301612c0 0x7f6e300b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e35bfe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e300d0870 0x7f6e300d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e35bfe640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e300d11b0 con 0x7f6e300b6a00 2024-04-25T08:42:42.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e35bfe640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e300b7200 con 0x7f6e301612c0 2024-04-25T08:42:42.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e35bfe640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300b74e0 con 0x7f6e300d0870 2024-04-25T08:42:42.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e2f7fe640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f6e301612c0 0x7f6e300b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e2effd640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f6e300b6a00 0x7f6e300d1c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:52052/0 (socket says 172.21.15.38:52052) 2024-04-25T08:42:42.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e2ffff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e300d0870 0x7f6e300d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e2effd640 1 -- 172.21.15.38:0/3915164959 learned_addr learned my addr 172.21.15.38:0/3915164959 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e2e7fc640 1 -- 172.21.15.38:0/3915164959 <== mon.1 v1:172.21.15.107:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785455870 0 0) 0x7f6e300d11b0 con 0x7f6e300b6a00 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e2e7fc640 1 -- 172.21.15.38:0/3915164959 --> v1:172.21.15.107:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e18003520 con 0x7f6e300b6a00 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e2ffff640 1 -- 172.21.15.38:0/3915164959 >> v2:172.21.15.107:3300/0 conn(0x7f6e301612c0 msgr2=0x7f6e300b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e2ffff640 1 --2- 172.21.15.38:0/3915164959 >> v2:172.21.15.107:3300/0 conn(0x7f6e301612c0 0x7f6e300b93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e2ffff640 1 -- 172.21.15.38:0/3915164959 >> v1:172.21.15.107:6789/0 conn(0x7f6e300b6a00 legacy=0x7f6e300d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e2ffff640 1 -- 172.21.15.38:0/3915164959 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e300b77c0 con 0x7f6e300d0870 2024-04-25T08:42:42.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.340+0000 7f6e2f7fe640 1 --2- 172.21.15.38:0/3915164959 >> v2:172.21.15.107:3300/0 conn(0x7f6e301612c0 0x7f6e300b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:42:42.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.341+0000 7f6e2ffff640 1 --2- 172.21.15.38:0/3915164959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e300d0870 0x7f6e300d0c70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009d90 tx=0x7f6e2802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dff4d8250765218e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.341+0000 7f6e2e7fc640 1 -- 172.21.15.38:0/3915164959 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28031600 con 0x7f6e300d0870 2024-04-25T08:42:42.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.341+0000 7f6e2e7fc640 1 -- 172.21.15.38:0/3915164959 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e280317a0 con 0x7f6e300d0870 2024-04-25T08:42:42.336 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.341+0000 7f6e2e7fc640 1 -- 172.21.15.38:0/3915164959 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28031b50 con 0x7f6e300d0870 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.342+0000 7f6e35bfe640 1 -- 172.21.15.38:0/3915164959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e300d0870 msgr2=0x7f6e300d0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.342+0000 7f6e35bfe640 1 --2- 172.21.15.38:0/3915164959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e300d0870 0x7f6e300d0c70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009d90 tx=0x7f6e2802a5d0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.342+0000 7f6e35bfe640 1 -- 172.21.15.38:0/3915164959 shutdown_connections 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.342+0000 7f6e35bfe640 1 --2- 172.21.15.38:0/3915164959 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e300d0870 0x7f6e300d0c70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.342+0000 7f6e35bfe640 1 --2- 172.21.15.38:0/3915164959 >> v2:172.21.15.107:3300/0 conn(0x7f6e301612c0 0x7f6e300b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.342+0000 7f6e35bfe640 1 -- 172.21.15.38:0/3915164959 >> 172.21.15.38:0/3915164959 conn(0x7f6e30066d10 msgr2=0x7f6e30067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.339 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.342+0000 7f6e35bfe640 1 -- 172.21.15.38:0/3915164959 shutdown_connections 2024-04-25T08:42:42.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.342+0000 7f6e35bfe640 1 -- 172.21.15.38:0/3915164959 wait complete. 2024-04-25T08:42:42.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.343+0000 7f6e35bfe640 1 Processor -- start 2024-04-25T08:42:42.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.344+0000 7f6e35bfe640 1 -- start start 2024-04-25T08:42:42.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.344+0000 7f6e35bfe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e300b6a00 0x7f6e300c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.344+0000 7f6e35bfe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e300d0870 0x7f6e3014cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.344+0000 7f6e35bfe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301612c0 0x7f6e3015d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.340 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.344+0000 7f6e35bfe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300b6e90 con 0x7f6e300d0870 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.344+0000 7f6e35bfe640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300b7d80 con 0x7f6e301612c0 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.344+0000 7f6e35bfe640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300b82a0 con 0x7f6e300b6a00 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.344+0000 7f6e2effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e300d0870 0x7f6e3014cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e2f7fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e300b6a00 0x7f6e300c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e2effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e300d0870 0x7f6e3014cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35020/0 (socket says 172.21.15.38:35020) 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e2effd640 1 -- 172.21.15.38:0/2055616179 learned_addr learned my addr 172.21.15.38:0/2055616179 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e2ffff640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301612c0 0x7f6e3015d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.341 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e2effd640 1 -- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e300b6a00 msgr2=0x7f6e300c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e2effd640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e300b6a00 0x7f6e300c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e2effd640 1 -- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301612c0 msgr2=0x7f6e3015d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e2effd640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e301612c0 0x7f6e3015d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e2effd640 1 -- 172.21.15.38:0/2055616179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e300c8db0 con 0x7f6e300d0870 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e2effd640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e300d0870 0x7f6e3014cff0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400c430 tx=0x7f6e2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e17fff640 1 -- 172.21.15.38:0/2055616179 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24017070 con 0x7f6e300d0870 2024-04-25T08:42:42.342 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e17fff640 1 -- 172.21.15.38:0/2055616179 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e24005ce0 con 0x7f6e300d0870 2024-04-25T08:42:42.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e17fff640 1 -- 172.21.15.38:0/2055616179 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240072f0 con 0x7f6e300d0870 2024-04-25T08:42:42.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e35bfe640 1 -- 172.21.15.38:0/2055616179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e300c7f50 con 0x7f6e300d0870 2024-04-25T08:42:42.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.345+0000 7f6e35bfe640 1 -- 172.21.15.38:0/2055616179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e30158230 con 0x7f6e300d0870 2024-04-25T08:42:42.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.346+0000 7f6e35bfe640 1 -- 172.21.15.38:0/2055616179 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e30068e80 con 0x7f6e300d0870 2024-04-25T08:42:42.343 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.347+0000 7f6e17fff640 1 -- 172.21.15.38:0/2055616179 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e2401a070 con 0x7f6e300d0870 2024-04-25T08:42:42.344 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.347+0000 7f6e17fff640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e1c06b4d0 0x7f6e1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.347+0000 7f6e17fff640 1 -- 172.21.15.38:0/2055616179 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6e240894b0 con 0x7f6e300d0870 2024-04-25T08:42:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.347+0000 7f6e17fff640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] conn(0x7f6e1c071970 0x7f6e1c073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.347+0000 7f6e17fff640 1 -- 172.21.15.38:0/2055616179 --> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e24010e50 con 0x7f6e1c071970 2024-04-25T08:42:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.347+0000 7f6e17fff640 1 -- 172.21.15.38:0/2055616179 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e24090050 con 0x7f6e300d0870 2024-04-25T08:42:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.347+0000 7f6e2f7fe640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e1c06b4d0 0x7f6e1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.347+0000 7f6e2ffff640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] conn(0x7f6e1c071970 0x7f6e1c073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.348+0000 7f6e2f7fe640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e1c06b4d0 0x7f6e1c06d990 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009560 tx=0x7f6e20009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.345 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.348+0000 7f6e2ffff640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] conn(0x7f6e1c071970 0x7f6e1c073db0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.349+0000 7f6e17fff640 1 -- 172.21.15.38:0/2055616179 <== osd.5 v2:172.21.15.107:6816/4056807981 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6e24010e50 con 0x7f6e1c071970 2024-04-25T08:42:42.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.362+0000 7f6e35bfe640 1 -- 172.21.15.38:0/2055616179 --> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f6e300d1dd0 con 0x7f6e1c071970 2024-04-25T08:42:42.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.362+0000 7f6e17fff640 1 -- 172.21.15.38:0/2055616179 <== osd.5 v2:172.21.15.107:6816/4056807981 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f6e300d1dd0 con 0x7f6e1c071970 2024-04-25T08:42:42.355 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:42:42.355 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:42:42.355 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:42:42.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.363+0000 7f6e35bfe640 1 -- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] conn(0x7f6e1c071970 msgr2=0x7f6e1c073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.363+0000 7f6e35bfe640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:6816/4056807981,v1:172.21.15.107:6817/4056807981] conn(0x7f6e1c071970 0x7f6e1c073db0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.363+0000 7f6e35bfe640 1 -- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e1c06b4d0 msgr2=0x7f6e1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.363+0000 7f6e35bfe640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6e1c06b4d0 0x7f6e1c06d990 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009560 tx=0x7f6e20009190 comp rx=0 tx=0).stop 2024-04-25T08:42:42.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.363+0000 7f6e35bfe640 1 -- 172.21.15.38:0/2055616179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e300d0870 msgr2=0x7f6e3014cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.363+0000 7f6e35bfe640 1 --2- 172.21.15.38:0/2055616179 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e300d0870 0x7f6e3014cff0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7f6e2400c430 tx=0x7f6e2400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.363+0000 7f6e2ffff640 1 -- 172.21.15.38:0/2055616179 reap_dead start 2024-04-25T08:42:42.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.363+0000 7f6e35bfe640 1 -- 172.21.15.38:0/2055616179 shutdown_connections 2024-04-25T08:42:42.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.363+0000 7f6e35bfe640 1 -- 172.21.15.38:0/2055616179 >> 172.21.15.38:0/2055616179 conn(0x7f6e30066d10 msgr2=0x7f6e3015b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.363+0000 7f6e35bfe640 1 -- 172.21.15.38:0/2055616179 shutdown_connections 2024-04-25T08:42:42.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.363+0000 7f6e35bfe640 1 -- 172.21.15.38:0/2055616179 wait complete. 2024-04-25T08:42:42.365 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T08:42:42.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.475+0000 7f6b9677e640 1 Processor -- start 2024-04-25T08:42:42.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.475+0000 7f6b9677e640 1 -- start start 2024-04-25T08:42:42.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b9677e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b90071690 0x7f6b9015ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b9677e640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6b900ceba0 0x7f6b9006d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b9677e640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b900b2430 con 0x7f6b9015a600 2024-04-25T08:42:42.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b9677e640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b90071d70 con 0x7f6b900ceba0 2024-04-25T08:42:42.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b9677e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900b26d0 con 0x7f6b90071690 2024-04-25T08:42:42.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b94d1b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6b900ceba0 0x7f6b9006d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b94d1b640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6b900ceba0 0x7f6b9006d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35026/0 (socket says 172.21.15.38:35026) 2024-04-25T08:42:42.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b94d1b640 1 -- 172.21.15.38:0/2120485558 learned_addr learned my addr 172.21.15.38:0/2120485558 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:42.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b8f7fe640 1 --2- 172.21.15.38:0/2120485558 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b90071690 0x7f6b9015ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b94d1b640 1 -- 172.21.15.38:0/2120485558 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b90071690 msgr2=0x7f6b9015ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b94d1b640 1 --2- 172.21.15.38:0/2120485558 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b90071690 0x7f6b9015ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b94d1b640 1 -- 172.21.15.38:0/2120485558 >> v1:172.21.15.107:6789/0 conn(0x7f6b9015a600 legacy=0x7f6b90070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.476+0000 7f6b94d1b640 1 -- 172.21.15.38:0/2120485558 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b900b29b0 con 0x7f6b900ceba0 2024-04-25T08:42:42.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.477+0000 7f6b94d1b640 1 --2- 172.21.15.38:0/2120485558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b900ceba0 0x7f6b9006d370 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800d150 tx=0x7f6b8802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d20d596b8e590f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.477+0000 7f6b8effd640 1 -- 172.21.15.38:0/2120485558 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8800b2e0 con 0x7f6b900ceba0 2024-04-25T08:42:42.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.477+0000 7f6b8effd640 1 -- 172.21.15.38:0/2120485558 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b88009600 con 0x7f6b900ceba0 2024-04-25T08:42:42.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.477+0000 7f6b8effd640 1 -- 172.21.15.38:0/2120485558 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8800b480 con 0x7f6b900ceba0 2024-04-25T08:42:42.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.478+0000 7f6b9677e640 1 -- 172.21.15.38:0/2120485558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b900ceba0 msgr2=0x7f6b9006d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.478+0000 7f6b9677e640 1 --2- 172.21.15.38:0/2120485558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b900ceba0 0x7f6b9006d370 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800d150 tx=0x7f6b8802c8e0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.478+0000 7f6b9677e640 1 -- 172.21.15.38:0/2120485558 shutdown_connections 2024-04-25T08:42:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.478+0000 7f6b9677e640 1 --2- 172.21.15.38:0/2120485558 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b900ceba0 0x7f6b9006d370 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.478+0000 7f6b9677e640 1 --2- 172.21.15.38:0/2120485558 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b90071690 0x7f6b9015ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.478+0000 7f6b9677e640 1 -- 172.21.15.38:0/2120485558 >> 172.21.15.38:0/2120485558 conn(0x7f6b9006abe0 msgr2=0x7f6b9006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.478+0000 7f6b9677e640 1 -- 172.21.15.38:0/2120485558 shutdown_connections 2024-04-25T08:42:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.478+0000 7f6b9677e640 1 -- 172.21.15.38:0/2120485558 wait complete. 2024-04-25T08:42:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.479+0000 7f6b9677e640 1 Processor -- start 2024-04-25T08:42:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.479+0000 7f6b9677e640 1 -- start start 2024-04-25T08:42:42.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b9677e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b90071690 0x7f6b9015c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b9677e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b900ceba0 0x7f6b900af350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b9677e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b9015a600 0x7f6b90158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b9677e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900b3600 con 0x7f6b9015a600 2024-04-25T08:42:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b9677e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9015a9e0 con 0x7f6b90071690 2024-04-25T08:42:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b9677e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9013c810 con 0x7f6b900ceba0 2024-04-25T08:42:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b94d1b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b9015a600 0x7f6b90158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b94d1b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b9015a600 0x7f6b90158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35038/0 (socket says 172.21.15.38:35038) 2024-04-25T08:42:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b94d1b640 1 -- 172.21.15.38:0/4141261122 learned_addr learned my addr 172.21.15.38:0/4141261122 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:42.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b8f7fe640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b900ceba0 0x7f6b900af350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b8ffff640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b90071690 0x7f6b9015c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b94d1b640 1 -- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b900ceba0 msgr2=0x7f6b900af350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b94d1b640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b900ceba0 0x7f6b900af350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b94d1b640 1 -- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b90071690 msgr2=0x7f6b9015c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b94d1b640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b90071690 0x7f6b9015c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b94d1b640 1 -- 172.21.15.38:0/4141261122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b90152010 con 0x7f6b9015a600 2024-04-25T08:42:42.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b8ffff640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b90071690 0x7f6b9015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:42:42.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.480+0000 7f6b94d1b640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b9015a600 0x7f6b90158ae0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800d270 tx=0x7f6b88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.481+0000 7f6b8cff9640 1 -- 172.21.15.38:0/4141261122 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b88041070 con 0x7f6b9015a600 2024-04-25T08:42:42.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.481+0000 7f6b8cff9640 1 -- 172.21.15.38:0/4141261122 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8803d820 con 0x7f6b9015a600 2024-04-25T08:42:42.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.481+0000 7f6b9677e640 1 -- 172.21.15.38:0/4141261122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b901522f0 con 0x7f6b9015a600 2024-04-25T08:42:42.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.481+0000 7f6b8cff9640 1 -- 172.21.15.38:0/4141261122 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8803dbd0 con 0x7f6b9015a600 2024-04-25T08:42:42.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.481+0000 7f6b9677e640 1 -- 172.21.15.38:0/4141261122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b901528b0 con 0x7f6b9015a600 2024-04-25T08:42:42.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.482+0000 7f6b9677e640 1 -- 172.21.15.38:0/4141261122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9004ef90 con 0x7f6b9015a600 2024-04-25T08:42:42.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.482+0000 7f6b8cff9640 1 -- 172.21.15.38:0/4141261122 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b88046460 con 0x7f6b9015a600 2024-04-25T08:42:42.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.483+0000 7f6b8cff9640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b7006b4d0 0x7f6b7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.483+0000 7f6b8cff9640 1 -- 172.21.15.38:0/4141261122 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6b88030030 con 0x7f6b9015a600 2024-04-25T08:42:42.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.483+0000 7f6b8ffff640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b7006b4d0 0x7f6b7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.483+0000 7f6b8ffff640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b7006b4d0 0x7f6b7006d990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6b78003eb0 tx=0x7f6b78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.487+0000 7f6b8cff9640 1 -- 172.21.15.38:0/4141261122 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b8804b060 con 0x7f6b9015a600 2024-04-25T08:42:42.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.669+0000 7f6b9677e640 1 -- 172.21.15.38:0/4141261122 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b900cfcb0 con 0x7f6b7006b4d0 2024-04-25T08:42:42.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.669+0000 7f6b8cff9640 1 -- 172.21.15.38:0/4141261122 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6b900cfcb0 con 0x7f6b7006b4d0 2024-04-25T08:42:42.662 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T08:42:42.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.672+0000 7f6b9677e640 1 -- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b7006b4d0 msgr2=0x7f6b7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.672+0000 7f6b9677e640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b7006b4d0 0x7f6b7006d990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6b78003eb0 tx=0x7f6b78009040 comp rx=0 tx=0).stop 2024-04-25T08:42:42.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.672+0000 7f6b9677e640 1 -- 172.21.15.38:0/4141261122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b9015a600 msgr2=0x7f6b90158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.672+0000 7f6b9677e640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b9015a600 0x7f6b90158ae0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800d270 tx=0x7f6b88004a40 comp rx=0 tx=0).stop 2024-04-25T08:42:42.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.672+0000 7f6b9677e640 1 -- 172.21.15.38:0/4141261122 shutdown_connections 2024-04-25T08:42:42.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.672+0000 7f6b9677e640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b9015a600 0x7f6b90158ae0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.672+0000 7f6b9677e640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6b7006b4d0 0x7f6b7006d990 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.672+0000 7f6b9677e640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b900ceba0 0x7f6b900af350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.672+0000 7f6b9677e640 1 --2- 172.21.15.38:0/4141261122 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b90071690 0x7f6b9015c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.672+0000 7f6b9677e640 1 -- 172.21.15.38:0/4141261122 >> 172.21.15.38:0/4141261122 conn(0x7f6b9006abe0 msgr2=0x7f6b9015df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.672+0000 7f6b9677e640 1 -- 172.21.15.38:0/4141261122 shutdown_connections 2024-04-25T08:42:42.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.672+0000 7f6b9677e640 1 -- 172.21.15.38:0/4141261122 wait complete. 2024-04-25T08:42:42.675 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T08:42:42.675 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:42:42.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.788+0000 7fb049a8f640 1 Processor -- start 2024-04-25T08:42:42.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.788+0000 7fb049a8f640 1 -- start start 2024-04-25T08:42:42.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb049a8f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0440b93f0 0x7fb0440d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb049a8f640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fb0440d0870 0x7fb0440d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb049a8f640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0440b9850 con 0x7fb0441612c0 2024-04-25T08:42:42.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb049a8f640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0440d11e0 con 0x7fb0440d0870 2024-04-25T08:42:42.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb049a8f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440b7200 con 0x7fb0440b93f0 2024-04-25T08:42:42.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb0437fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fb0441612c0 0x7fb0440b6a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33908/0 (socket says 172.21.15.38:33908) 2024-04-25T08:42:42.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb0437fe640 1 -- 172.21.15.38:0/1786123273 learned_addr learned my addr 172.21.15.38:0/1786123273 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:42.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb042ffd640 1 --2- 172.21.15.38:0/1786123273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0440b93f0 0x7fb0440d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb043fff640 1 --2- 172.21.15.38:0/1786123273 >> v2:172.21.15.107:3300/0 conn(0x7fb0440d0870 0x7fb0440d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.783 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb0427fc640 1 -- 172.21.15.38:0/1786123273 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996980684 0 0) 0x7fb0440b9850 con 0x7fb0441612c0 2024-04-25T08:42:42.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb0427fc640 1 -- 172.21.15.38:0/1786123273 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb02c003520 con 0x7fb0441612c0 2024-04-25T08:42:42.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb042ffd640 1 -- 172.21.15.38:0/1786123273 >> v2:172.21.15.107:3300/0 conn(0x7fb0440d0870 msgr2=0x7fb0440d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb042ffd640 1 --2- 172.21.15.38:0/1786123273 >> v2:172.21.15.107:3300/0 conn(0x7fb0440d0870 0x7fb0440d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb042ffd640 1 -- 172.21.15.38:0/1786123273 >> v1:172.21.15.38:6789/0 conn(0x7fb0441612c0 legacy=0x7fb0440b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.789+0000 7fb042ffd640 1 -- 172.21.15.38:0/1786123273 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0440b74e0 con 0x7fb0440b93f0 2024-04-25T08:42:42.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.790+0000 7fb042ffd640 1 --2- 172.21.15.38:0/1786123273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0440b93f0 0x7fb0440d1c30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb0380097a0 tx=0x7fb03802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc68f51babc80324 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.784 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.790+0000 7fb0427fc640 1 -- 172.21.15.38:0/1786123273 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038037070 con 0x7fb0440b93f0 2024-04-25T08:42:42.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.790+0000 7fb0427fc640 1 -- 172.21.15.38:0/1786123273 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03802ae50 con 0x7fb0440b93f0 2024-04-25T08:42:42.785 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.790+0000 7fb0427fc640 1 -- 172.21.15.38:0/1786123273 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038032640 con 0x7fb0440b93f0 2024-04-25T08:42:42.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.791+0000 7fb049a8f640 1 -- 172.21.15.38:0/1786123273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0440b93f0 msgr2=0x7fb0440d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.791+0000 7fb049a8f640 1 --2- 172.21.15.38:0/1786123273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0440b93f0 0x7fb0440d1c30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb0380097a0 tx=0x7fb03802a100 comp rx=0 tx=0).stop 2024-04-25T08:42:42.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.791+0000 7fb049a8f640 1 -- 172.21.15.38:0/1786123273 shutdown_connections 2024-04-25T08:42:42.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.791+0000 7fb049a8f640 1 --2- 172.21.15.38:0/1786123273 >> v2:172.21.15.107:3300/0 conn(0x7fb0440d0870 0x7fb0440d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.791+0000 7fb049a8f640 1 --2- 172.21.15.38:0/1786123273 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0440b93f0 0x7fb0440d1c30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.791+0000 7fb049a8f640 1 -- 172.21.15.38:0/1786123273 >> 172.21.15.38:0/1786123273 conn(0x7fb044066d10 msgr2=0x7fb044067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.791+0000 7fb049a8f640 1 -- 172.21.15.38:0/1786123273 shutdown_connections 2024-04-25T08:42:42.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.791+0000 7fb049a8f640 1 -- 172.21.15.38:0/1786123273 wait complete. 2024-04-25T08:42:42.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.792+0000 7fb049a8f640 1 Processor -- start 2024-04-25T08:42:42.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.792+0000 7fb049a8f640 1 -- start start 2024-04-25T08:42:42.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb049a8f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0440b93f0 0x7fb04414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb049a8f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0440d0870 0x7fb04415c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb049a8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0441612c0 0x7fb04415c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb049a8f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440b7d80 con 0x7fb0440b93f0 2024-04-25T08:42:42.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb049a8f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440b7aa0 con 0x7fb0441612c0 2024-04-25T08:42:42.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb049a8f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440d2170 con 0x7fb0440d0870 2024-04-25T08:42:42.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb0437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0440b93f0 0x7fb04414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb0437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0440b93f0 0x7fb04414e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35054/0 (socket says 172.21.15.38:35054) 2024-04-25T08:42:42.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb0437fe640 1 -- 172.21.15.38:0/1377783340 learned_addr learned my addr 172.21.15.38:0/1377783340 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:42.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb042ffd640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0440d0870 0x7fb04415c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb043fff640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0441612c0 0x7fb04415c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb0437fe640 1 -- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0440d0870 msgr2=0x7fb04415c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb0437fe640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0440d0870 0x7fb04415c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb0437fe640 1 -- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0441612c0 msgr2=0x7fb04415c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb0437fe640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0441612c0 0x7fb04415c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb0437fe640 1 -- 172.21.15.38:0/1377783340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0440c75a0 con 0x7fb0440b93f0 2024-04-25T08:42:42.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.793+0000 7fb0437fe640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0440b93f0 0x7fb04414e2b0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fb03400c390 tx=0x7fb03400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.794+0000 7fb02bfff640 1 -- 172.21.15.38:0/1377783340 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb034017070 con 0x7fb0440b93f0 2024-04-25T08:42:42.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.794+0000 7fb02bfff640 1 -- 172.21.15.38:0/1377783340 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb034005ce0 con 0x7fb0440b93f0 2024-04-25T08:42:42.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.794+0000 7fb049a8f640 1 -- 172.21.15.38:0/1377783340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0440c7880 con 0x7fb0440b93f0 2024-04-25T08:42:42.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.794+0000 7fb02bfff640 1 -- 172.21.15.38:0/1377783340 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0340072f0 con 0x7fb0440b93f0 2024-04-25T08:42:42.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.794+0000 7fb049a8f640 1 -- 172.21.15.38:0/1377783340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0440c7e40 con 0x7fb0440b93f0 2024-04-25T08:42:42.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.795+0000 7fb049a8f640 1 -- 172.21.15.38:0/1377783340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb044068e80 con 0x7fb0440b93f0 2024-04-25T08:42:42.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.795+0000 7fb02bfff640 1 -- 172.21.15.38:0/1377783340 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb03401a070 con 0x7fb0440b93f0 2024-04-25T08:42:42.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.796+0000 7fb02bfff640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb03006b4d0 0x7fb03006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.796+0000 7fb02bfff640 1 -- 172.21.15.38:0/1377783340 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb034089370 con 0x7fb0440b93f0 2024-04-25T08:42:42.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.796+0000 7fb02bfff640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] conn(0x7fb030071970 0x7fb030073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.796+0000 7fb042ffd640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb03006b4d0 0x7fb03006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.796+0000 7fb02bfff640 1 -- 172.21.15.38:0/1377783340 --> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb034010e50 con 0x7fb030071970 2024-04-25T08:42:42.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.796+0000 7fb02bfff640 1 -- 172.21.15.38:0/1377783340 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb034090050 con 0x7fb0440b93f0 2024-04-25T08:42:42.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.796+0000 7fb043fff640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] conn(0x7fb030071970 0x7fb030073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.796+0000 7fb042ffd640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb03006b4d0 0x7fb03006d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb038034000 tx=0x7fb03802e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.797+0000 7fb043fff640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] conn(0x7fb030071970 0x7fb030073db0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.797+0000 7fb02bfff640 1 -- 172.21.15.38:0/1377783340 <== osd.6 v2:172.21.15.107:6808/1351784056 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb034010e50 con 0x7fb030071970 2024-04-25T08:42:42.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.810+0000 7fb049a8f640 1 -- 172.21.15.38:0/1377783340 --> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb0441608c0 con 0x7fb030071970 2024-04-25T08:42:42.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.811+0000 7fb02bfff640 1 -- 172.21.15.38:0/1377783340 <== osd.6 v2:172.21.15.107:6808/1351784056 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb0441608c0 con 0x7fb030071970 2024-04-25T08:42:42.803 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:42:42.804 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:42:42.804 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:42:42.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.811+0000 7fb049a8f640 1 -- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] conn(0x7fb030071970 msgr2=0x7fb030073db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.811+0000 7fb049a8f640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:6808/1351784056,v1:172.21.15.107:6809/1351784056] conn(0x7fb030071970 0x7fb030073db0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.811+0000 7fb049a8f640 1 -- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb03006b4d0 msgr2=0x7fb03006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.811+0000 7fb049a8f640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fb03006b4d0 0x7fb03006d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fb038034000 tx=0x7fb03802e000 comp rx=0 tx=0).stop 2024-04-25T08:42:42.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.811+0000 7fb049a8f640 1 -- 172.21.15.38:0/1377783340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0440b93f0 msgr2=0x7fb04414e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.811+0000 7fb049a8f640 1 --2- 172.21.15.38:0/1377783340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0440b93f0 0x7fb04414e2b0 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7fb03400c390 tx=0x7fb03400c850 comp rx=0 tx=0).stop 2024-04-25T08:42:42.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.811+0000 7fb043fff640 1 -- 172.21.15.38:0/1377783340 reap_dead start 2024-04-25T08:42:42.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.812+0000 7fb049a8f640 1 -- 172.21.15.38:0/1377783340 shutdown_connections 2024-04-25T08:42:42.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.812+0000 7fb049a8f640 1 -- 172.21.15.38:0/1377783340 >> 172.21.15.38:0/1377783340 conn(0x7fb044066d10 msgr2=0x7fb0440d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.812+0000 7fb049a8f640 1 -- 172.21.15.38:0/1377783340 shutdown_connections 2024-04-25T08:42:42.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.812+0000 7fb049a8f640 1 -- 172.21.15.38:0/1377783340 wait complete. 2024-04-25T08:42:42.814 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T08:42:42.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.923+0000 7fe08713b640 1 Processor -- start 2024-04-25T08:42:42.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.924+0000 7fe08713b640 1 -- start start 2024-04-25T08:42:42.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.924+0000 7fe08713b640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7fe0800ceba0 0x7fe08015ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.924+0000 7fe08713b640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0800b2430 con 0x7fe080071690 2024-04-25T08:42:42.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.924+0000 7fe08713b640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe080071d70 con 0x7fe08015a600 2024-04-25T08:42:42.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.924+0000 7fe08713b640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0800b26d0 con 0x7fe0800ceba0 2024-04-25T08:42:42.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.924+0000 7fe07ffff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fe080071690 0x7fe08006d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33924/0 (socket says 172.21.15.38:33924) 2024-04-25T08:42:42.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.924+0000 7fe07ffff640 1 -- 172.21.15.38:0/4152244522 learned_addr learned my addr 172.21.15.38:0/4152244522 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:42.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541403858 0 0) 0x7fe0800b2430 con 0x7fe080071690 2024-04-25T08:42:42.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe070003520 con 0x7fe080071690 2024-04-25T08:42:42.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe0856d8640 1 --2- 172.21.15.38:0/4152244522 >> v2:172.21.15.107:3300/0 conn(0x7fe0800ceba0 0x7fe08015ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 <== mon.0 v1:172.21.15.38:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2904255993 0 0) 0x7fe070003520 con 0x7fe080071690 2024-04-25T08:42:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 >> v2:172.21.15.107:3300/0 conn(0x7fe0800ceba0 msgr2=0x7fe08015ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 --2- 172.21.15.38:0/4152244522 >> v2:172.21.15.107:3300/0 conn(0x7fe0800ceba0 0x7fe08015ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 >> v1:172.21.15.107:6789/0 conn(0x7fe08015a600 legacy=0x7fe080070f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 --> v1:172.21.15.38:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0800b29b0 con 0x7fe080071690 2024-04-25T08:42:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 <== mon.0 v1:172.21.15.38:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3874041277 0 0) 0x7fe06c001670 con 0x7fe080071690 2024-04-25T08:42:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 >> v1:172.21.15.38:6789/0 conn(0x7fe080071690 legacy=0x7fe08006d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 --2- 172.21.15.38:0/4152244522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe070003a00 0x7fe070003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 --2- 172.21.15.38:0/4152244522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe070004800 0x7fe070004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 --2- 172.21.15.38:0/4152244522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0700051c0 0x7fe07000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0800b26d0 con 0x7fe070003a00 2024-04-25T08:42:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07000c0c0 con 0x7fe070004800 2024-04-25T08:42:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07000c2c0 con 0x7fe0700051c0 2024-04-25T08:42:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe084ed7640 1 --2- 172.21.15.38:0/4152244522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe070003a00 0x7fe070003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe07ffff640 1 --2- 172.21.15.38:0/4152244522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe070004800 0x7fe070004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.925+0000 7fe0856d8640 1 --2- 172.21.15.38:0/4152244522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0700051c0 0x7fe07000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.922 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.926+0000 7fe084ed7640 1 -- 172.21.15.38:0/4152244522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0700051c0 msgr2=0x7fe07000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.926+0000 7fe084ed7640 1 --2- 172.21.15.38:0/4152244522 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0700051c0 0x7fe07000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.926+0000 7fe084ed7640 1 -- 172.21.15.38:0/4152244522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe070004800 msgr2=0x7fe070004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.926+0000 7fe084ed7640 1 --2- 172.21.15.38:0/4152244522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe070004800 0x7fe070004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.926+0000 7fe084ed7640 1 -- 172.21.15.38:0/4152244522 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe070003520 con 0x7fe070003a00 2024-04-25T08:42:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.926+0000 7fe0856d8640 1 -- 172.21.15.38:0/4152244522 reap_dead start 2024-04-25T08:42:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.926+0000 7fe084ed7640 1 --2- 172.21.15.38:0/4152244522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe070003a00 0x7fe070003e20 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fe074001750 tx=0x7fe070005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0de059dfd4094 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.923 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.926+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074001370 con 0x7fe070003a00 2024-04-25T08:42:42.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.926+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0740052d0 con 0x7fe070003a00 2024-04-25T08:42:42.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.926+0000 7fe07f7fe640 1 -- 172.21.15.38:0/4152244522 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07402abc0 con 0x7fe070003a00 2024-04-25T08:42:42.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.927+0000 7fe08713b640 1 -- 172.21.15.38:0/4152244522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe070003a00 msgr2=0x7fe070003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.927+0000 7fe08713b640 1 --2- 172.21.15.38:0/4152244522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe070003a00 0x7fe070003e20 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fe074001750 tx=0x7fe070005830 comp rx=0 tx=0).stop 2024-04-25T08:42:42.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.927+0000 7fe08713b640 1 -- 172.21.15.38:0/4152244522 shutdown_connections 2024-04-25T08:42:42.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.927+0000 7fe08713b640 1 --2- 172.21.15.38:0/4152244522 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe070003a00 0x7fe070003e20 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.924 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.927+0000 7fe08713b640 1 -- 172.21.15.38:0/4152244522 >> 172.21.15.38:0/4152244522 conn(0x7fe08006abe0 msgr2=0x7fe08006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:42.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.927+0000 7fe08713b640 1 -- 172.21.15.38:0/4152244522 shutdown_connections 2024-04-25T08:42:42.925 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.927+0000 7fe08713b640 1 -- 172.21.15.38:0/4152244522 wait complete. 2024-04-25T08:42:42.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.928+0000 7fe08713b640 1 Processor -- start 2024-04-25T08:42:42.927 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.928+0000 7fe08713b640 1 -- start start 2024-04-25T08:42:42.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe08713b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe070003a00 0x7fe0801f4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe08713b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe080158380 0x7fe0800a8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe08713b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe080153770 0x7fe080153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe08713b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0800b3600 con 0x7fe080158380 2024-04-25T08:42:42.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe08713b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08015a9e0 con 0x7fe070003a00 2024-04-25T08:42:42.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe08713b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe080158940 con 0x7fe080153770 2024-04-25T08:42:42.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe080158380 0x7fe0800a8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe07ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe080158380 0x7fe0800a8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35070/0 (socket says 172.21.15.38:35070) 2024-04-25T08:42:42.928 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe07ffff640 1 -- 172.21.15.38:0/3412640315 learned_addr learned my addr 172.21.15.38:0/3412640315 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:42.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe084ed7640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe070003a00 0x7fe0801f4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe07ffff640 1 -- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe080153770 msgr2=0x7fe080153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe0856d8640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe080153770 0x7fe080153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe07ffff640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe080153770 0x7fe080153bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe07ffff640 1 -- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe070003a00 msgr2=0x7fe0801f4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:42.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe07ffff640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe070003a00 0x7fe0801f4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:42.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe07ffff640 1 -- 172.21.15.38:0/3412640315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe080158c20 con 0x7fe080158380 2024-04-25T08:42:42.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.929+0000 7fe084ed7640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe070003a00 0x7fe0801f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:42:42.929 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.930+0000 7fe07ffff640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe080158380 0x7fe0800a8b90 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7fe06c004f70 tx=0x7fe06c0074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.930+0000 7fe07d7fa640 1 -- 172.21.15.38:0/3412640315 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c01a070 con 0x7fe080158380 2024-04-25T08:42:42.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.930+0000 7fe07d7fa640 1 -- 172.21.15.38:0/3412640315 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06c00f750 con 0x7fe080158380 2024-04-25T08:42:42.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.930+0000 7fe08713b640 1 -- 172.21.15.38:0/3412640315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe080158f00 con 0x7fe080158380 2024-04-25T08:42:42.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.930+0000 7fe07d7fa640 1 -- 172.21.15.38:0/3412640315 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06c00fb20 con 0x7fe080158380 2024-04-25T08:42:42.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.930+0000 7fe08713b640 1 -- 172.21.15.38:0/3412640315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0801594c0 con 0x7fe080158380 2024-04-25T08:42:42.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.931+0000 7fe08713b640 1 -- 172.21.15.38:0/3412640315 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0801596c0 con 0x7fe080158380 2024-04-25T08:42:42.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.931+0000 7fe07d7fa640 1 -- 172.21.15.38:0/3412640315 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe06c01d070 con 0x7fe080158380 2024-04-25T08:42:42.930 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.932+0000 7fe07d7fa640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe06406b500 0x7fe06406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:42.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.932+0000 7fe07d7fa640 1 -- 172.21.15.38:0/3412640315 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe06c089940 con 0x7fe080158380 2024-04-25T08:42:42.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.932+0000 7fe084ed7640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe06406b500 0x7fe06406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:42.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.932+0000 7fe084ed7640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe06406b500 0x7fe06406d9c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe074032000 tx=0x7fe07402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:42.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:42.936+0000 7fe07d7fa640 1 -- 172.21.15.38:0/3412640315 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe0801596c0 con 0x7fe080158380 2024-04-25T08:42:43.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.116+0000 7fe08713b640 1 -- 172.21.15.38:0/3412640315 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe08004ef90 con 0x7fe06406b500 2024-04-25T08:42:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.117+0000 7fe07d7fa640 1 -- 172.21.15.38:0/3412640315 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe08004ef90 con 0x7fe06406b500 2024-04-25T08:42:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T08:42:43.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.119+0000 7fe08713b640 1 -- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe06406b500 msgr2=0x7fe06406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.119+0000 7fe08713b640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe06406b500 0x7fe06406d9c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe074032000 tx=0x7fe07402e000 comp rx=0 tx=0).stop 2024-04-25T08:42:43.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.119+0000 7fe08713b640 1 -- 172.21.15.38:0/3412640315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe080158380 msgr2=0x7fe0800a8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.119+0000 7fe08713b640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe080158380 0x7fe0800a8b90 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7fe06c004f70 tx=0x7fe06c0074f0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.119+0000 7fe08713b640 1 -- 172.21.15.38:0/3412640315 shutdown_connections 2024-04-25T08:42:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.119+0000 7fe08713b640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fe06406b500 0x7fe06406d9c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.119+0000 7fe08713b640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe080153770 0x7fe080153bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.119+0000 7fe08713b640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe080158380 0x7fe0800a8b90 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.119+0000 7fe08713b640 1 --2- 172.21.15.38:0/3412640315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe070003a00 0x7fe0801f4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.119+0000 7fe08713b640 1 -- 172.21.15.38:0/3412640315 >> 172.21.15.38:0/3412640315 conn(0x7fe08006abe0 msgr2=0x7fe08006e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.112 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.119+0000 7fe08713b640 1 -- 172.21.15.38:0/3412640315 shutdown_connections 2024-04-25T08:42:43.113 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.119+0000 7fe08713b640 1 -- 172.21.15.38:0/3412640315 wait complete. 2024-04-25T08:42:43.122 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T08:42:43.122 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T08:42:43.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.234+0000 7f4b60038640 1 Processor -- start 2024-04-25T08:42:43.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.235+0000 7f4b60038640 1 -- start start 2024-04-25T08:42:43.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.235+0000 7f4b60038640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4b580d0860 0x7f4b580d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.235+0000 7f4b60038640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b580d19e0 0x7f4b580d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.235+0000 7f4b60038640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b580d1180 con 0x7f4b580d3860 2024-04-25T08:42:43.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.235+0000 7f4b60038640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5815e2b0 con 0x7f4b580d0860 2024-04-25T08:42:43.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.235+0000 7f4b60038640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580d24a0 con 0x7f4b580d19e0 2024-04-25T08:42:43.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.236+0000 7f4b5ddd4640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f4b580d0860 0x7f4b580d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.236+0000 7f4b5d5d3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b580d19e0 0x7f4b580d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.236+0000 7f4b5e5d5640 1 --1- >> v1:172.21.15.107:6789/0 conn(0x7f4b580d3860 0x7f4b580d3cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.107:6789/0 says I am v1:172.21.15.38:52066/0 (socket says 172.21.15.38:52066) 2024-04-25T08:42:43.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.236+0000 7f4b5e5d5640 1 -- 172.21.15.38:0/3008986075 learned_addr learned my addr 172.21.15.38:0/3008986075 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:43.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.236+0000 7f4b5ddd4640 1 -- 172.21.15.38:0/3008986075 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b580d19e0 msgr2=0x7f4b580d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:43.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.236+0000 7f4b5ddd4640 1 --2- 172.21.15.38:0/3008986075 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b580d19e0 0x7f4b580d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.236+0000 7f4b5ddd4640 1 -- 172.21.15.38:0/3008986075 >> v1:172.21.15.107:6789/0 conn(0x7f4b580d3860 legacy=0x7f4b580d3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.236+0000 7f4b5ddd4640 1 -- 172.21.15.38:0/3008986075 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b580d2780 con 0x7f4b580d0860 2024-04-25T08:42:43.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.237+0000 7f4b5ddd4640 1 --2- 172.21.15.38:0/3008986075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b580d0860 0x7f4b580d0c40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4b540097a0 tx=0x7f4b5402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a662f740bd243201 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:43.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.237+0000 7f4b5cdd2640 1 -- 172.21.15.38:0/3008986075 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54037070 con 0x7f4b580d0860 2024-04-25T08:42:43.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.237+0000 7f4b5cdd2640 1 -- 172.21.15.38:0/3008986075 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b54032440 con 0x7f4b580d0860 2024-04-25T08:42:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.237+0000 7f4b5cdd2640 1 -- 172.21.15.38:0/3008986075 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54032810 con 0x7f4b580d0860 2024-04-25T08:42:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.238+0000 7f4b60038640 1 -- 172.21.15.38:0/3008986075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b580d0860 msgr2=0x7f4b580d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.238+0000 7f4b60038640 1 --2- 172.21.15.38:0/3008986075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b580d0860 0x7f4b580d0c40 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4b540097a0 tx=0x7f4b5402a180 comp rx=0 tx=0).stop 2024-04-25T08:42:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.238+0000 7f4b60038640 1 -- 172.21.15.38:0/3008986075 shutdown_connections 2024-04-25T08:42:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.238+0000 7f4b60038640 1 --2- 172.21.15.38:0/3008986075 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b580d19e0 0x7f4b580d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.238+0000 7f4b60038640 1 --2- 172.21.15.38:0/3008986075 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b580d0860 0x7f4b580d0c40 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.238+0000 7f4b60038640 1 -- 172.21.15.38:0/3008986075 >> 172.21.15.38:0/3008986075 conn(0x7f4b58066d10 msgr2=0x7f4b58067120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.238+0000 7f4b60038640 1 -- 172.21.15.38:0/3008986075 shutdown_connections 2024-04-25T08:42:43.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.238+0000 7f4b60038640 1 -- 172.21.15.38:0/3008986075 wait complete. 2024-04-25T08:42:43.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.239+0000 7f4b60038640 1 Processor -- start 2024-04-25T08:42:43.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.239+0000 7f4b60038640 1 -- start start 2024-04-25T08:42:43.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b60038640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b580d0860 0x7f4b58154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b60038640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b580d19e0 0x7f4b581e82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b60038640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b580d3860 0x7f4b581e8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b60038640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580d3540 con 0x7f4b580d3860 2024-04-25T08:42:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b60038640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580d2d40 con 0x7f4b580d19e0 2024-04-25T08:42:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b60038640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580b9770 con 0x7f4b580d0860 2024-04-25T08:42:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b5e5d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b580d3860 0x7f4b581e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b5e5d5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b580d3860 0x7f4b581e8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35080/0 (socket says 172.21.15.38:35080) 2024-04-25T08:42:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b5e5d5640 1 -- 172.21.15.38:0/3036386084 learned_addr learned my addr 172.21.15.38:0/3036386084 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b5ddd4640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b580d0860 0x7f4b58154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b5d5d3640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b580d19e0 0x7f4b581e82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b5e5d5640 1 -- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b580d0860 msgr2=0x7f4b58154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b5e5d5640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f4b580d0860 0x7f4b58154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b5e5d5640 1 -- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b580d19e0 msgr2=0x7f4b581e82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b5e5d5640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4b580d19e0 0x7f4b581e82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b5e5d5640 1 -- 172.21.15.38:0/3036386084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b58154ba0 con 0x7f4b580d3860 2024-04-25T08:42:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b5e5d5640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b580d3860 0x7f4b581e8830 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000ca80 tx=0x7f4b5000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:43.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b4e7fc640 1 -- 172.21.15.38:0/3036386084 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50017070 con 0x7f4b580d3860 2024-04-25T08:42:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b4e7fc640 1 -- 172.21.15.38:0/3036386084 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5000ac30 con 0x7f4b580d3860 2024-04-25T08:42:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b60038640 1 -- 172.21.15.38:0/3036386084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b580cc430 con 0x7f4b580d3860 2024-04-25T08:42:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.240+0000 7f4b4e7fc640 1 -- 172.21.15.38:0/3036386084 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b50012400 con 0x7f4b580d3860 2024-04-25T08:42:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.241+0000 7f4b60038640 1 -- 172.21.15.38:0/3036386084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b580cc9f0 con 0x7f4b580d3860 2024-04-25T08:42:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.241+0000 7f4b60038640 1 -- 172.21.15.38:0/3036386084 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4b58068e80 con 0x7f4b580d3860 2024-04-25T08:42:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.242+0000 7f4b4e7fc640 1 -- 172.21.15.38:0/3036386084 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b5001a070 con 0x7f4b580d3860 2024-04-25T08:42:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.242+0000 7f4b4e7fc640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b3c06b500 0x7f4b3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.242+0000 7f4b4e7fc640 1 -- 172.21.15.38:0/3036386084 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4b50089c10 con 0x7f4b580d3860 2024-04-25T08:42:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.243+0000 7f4b4e7fc640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] conn(0x7f4b3c0719a0 0x7f4b3c073de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.243+0000 7f4b4e7fc640 1 -- 172.21.15.38:0/3036386084 --> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4b3c0744b0 con 0x7f4b3c0719a0 2024-04-25T08:42:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.243+0000 7f4b4e7fc640 1 -- 172.21.15.38:0/3036386084 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=148) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b5008c2b0 con 0x7f4b580d3860 2024-04-25T08:42:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.243+0000 7f4b5ddd4640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b3c06b500 0x7f4b3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.243+0000 7f4b5d5d3640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] conn(0x7f4b3c0719a0 0x7f4b3c073de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.243+0000 7f4b5ddd4640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b3c06b500 0x7f4b3c06d9c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4b54034000 tx=0x7f4b5402e000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.243+0000 7f4b5d5d3640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] conn(0x7f4b3c0719a0 0x7f4b3c073de0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:43.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.244+0000 7f4b4e7fc640 1 -- 172.21.15.38:0/3036386084 <== osd.7 v2:172.21.15.107:6824/3757070988 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4b3c0744b0 con 0x7f4b3c0719a0 2024-04-25T08:42:43.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.257+0000 7f4b60038640 1 -- 172.21.15.38:0/3036386084 --> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4b580b4c00 con 0x7f4b3c0719a0 2024-04-25T08:42:43.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.258+0000 7f4b4e7fc640 1 -- 172.21.15.38:0/3036386084 <== osd.7 v2:172.21.15.107:6824/3757070988 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4b580b4c00 con 0x7f4b3c0719a0 2024-04-25T08:42:43.250 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-04-25T08:42:43.250 INFO:teuthology.orchestra.run.smithi038.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T08:42:43.250 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-04-25T08:42:43.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.258+0000 7f4b60038640 1 -- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] conn(0x7f4b3c0719a0 msgr2=0x7f4b3c073de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.258+0000 7f4b60038640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:6824/3757070988,v1:172.21.15.107:6825/3757070988] conn(0x7f4b3c0719a0 0x7f4b3c073de0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.258+0000 7f4b60038640 1 -- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b3c06b500 msgr2=0x7f4b3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.258+0000 7f4b60038640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f4b3c06b500 0x7f4b3c06d9c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f4b54034000 tx=0x7f4b5402e000 comp rx=0 tx=0).stop 2024-04-25T08:42:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.258+0000 7f4b60038640 1 -- 172.21.15.38:0/3036386084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b580d3860 msgr2=0x7f4b581e8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.258+0000 7f4b60038640 1 --2- 172.21.15.38:0/3036386084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b580d3860 0x7f4b581e8830 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000ca80 tx=0x7f4b5000cf40 comp rx=0 tx=0).stop 2024-04-25T08:42:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.258+0000 7f4b5e5d5640 1 -- 172.21.15.38:0/3036386084 reap_dead start 2024-04-25T08:42:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.258+0000 7f4b60038640 1 -- 172.21.15.38:0/3036386084 shutdown_connections 2024-04-25T08:42:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.258+0000 7f4b60038640 1 -- 172.21.15.38:0/3036386084 >> 172.21.15.38:0/3036386084 conn(0x7f4b58066d10 msgr2=0x7f4b581569e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.258+0000 7f4b60038640 1 -- 172.21.15.38:0/3036386084 shutdown_connections 2024-04-25T08:42:43.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.258+0000 7f4b60038640 1 -- 172.21.15.38:0/3036386084 wait complete. 2024-04-25T08:42:43.261 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T08:42:43.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.370+0000 7efd98dce640 1 Processor -- start 2024-04-25T08:42:43.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.370+0000 7efd98dce640 1 -- start start 2024-04-25T08:42:43.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.371+0000 7efd98dce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd940d0d10 0x7efd94099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.371+0000 7efd98dce640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efd94099fb0 0x7efd940cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.371+0000 7efd98dce640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd940cfc70 con 0x7efd9415def0 2024-04-25T08:42:43.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.371+0000 7efd98dce640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9415e2d0 con 0x7efd94099fb0 2024-04-25T08:42:43.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.371+0000 7efd98dce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd940cff50 con 0x7efd940d0d10 2024-04-25T08:42:43.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.371+0000 7efd92d9d640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7efd94099fb0 0x7efd940cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.372+0000 7efd9259c640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7efd9415def0 0x7efd9406c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33928/0 (socket says 172.21.15.38:33928) 2024-04-25T08:42:43.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.372+0000 7efd9259c640 1 -- 172.21.15.38:0/2550099961 learned_addr learned my addr 172.21.15.38:0/2550099961 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:43.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.372+0000 7efd91d9b640 1 --2- 172.21.15.38:0/2550099961 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd940d0d10 0x7efd94099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.372+0000 7efd92d9d640 1 -- 172.21.15.38:0/2550099961 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd940d0d10 msgr2=0x7efd94099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:42:43.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.372+0000 7efd92d9d640 1 --2- 172.21.15.38:0/2550099961 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd940d0d10 0x7efd94099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.372+0000 7efd92d9d640 1 -- 172.21.15.38:0/2550099961 >> v1:172.21.15.38:6789/0 conn(0x7efd9415def0 legacy=0x7efd9406c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.372+0000 7efd92d9d640 1 -- 172.21.15.38:0/2550099961 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd940d0230 con 0x7efd94099fb0 2024-04-25T08:42:43.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.372+0000 7efd9159a640 1 -- 172.21.15.38:0/2550099961 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142236510 0 0) 0x7efd940cfc70 con 0x7efd9415def0 2024-04-25T08:42:43.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.372+0000 7efd92d9d640 1 --2- 172.21.15.38:0/2550099961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd94099fb0 0x7efd940cbc50 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009d90 tx=0x7efd8c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7adb8565dc2546 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:43.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.373+0000 7efd9159a640 1 -- 172.21.15.38:0/2550099961 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c007740 con 0x7efd94099fb0 2024-04-25T08:42:43.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.373+0000 7efd9159a640 1 -- 172.21.15.38:0/2550099961 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8c0078e0 con 0x7efd94099fb0 2024-04-25T08:42:43.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.373+0000 7efd9159a640 1 -- 172.21.15.38:0/2550099961 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8c007cb0 con 0x7efd94099fb0 2024-04-25T08:42:43.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.373+0000 7efd98dce640 1 -- 172.21.15.38:0/2550099961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd94099fb0 msgr2=0x7efd940cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.373+0000 7efd98dce640 1 --2- 172.21.15.38:0/2550099961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd94099fb0 0x7efd940cbc50 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009d90 tx=0x7efd8c02a660 comp rx=0 tx=0).stop 2024-04-25T08:42:43.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.373+0000 7efd98dce640 1 -- 172.21.15.38:0/2550099961 shutdown_connections 2024-04-25T08:42:43.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.373+0000 7efd98dce640 1 --2- 172.21.15.38:0/2550099961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd94099fb0 0x7efd940cbc50 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.373+0000 7efd98dce640 1 --2- 172.21.15.38:0/2550099961 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd940d0d10 0x7efd94099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.373+0000 7efd98dce640 1 -- 172.21.15.38:0/2550099961 >> 172.21.15.38:0/2550099961 conn(0x7efd940b30f0 msgr2=0x7efd940b3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.373+0000 7efd98dce640 1 -- 172.21.15.38:0/2550099961 shutdown_connections 2024-04-25T08:42:43.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.373+0000 7efd98dce640 1 -- 172.21.15.38:0/2550099961 wait complete. 2024-04-25T08:42:43.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.375+0000 7efd98dce640 1 Processor -- start 2024-04-25T08:42:43.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.375+0000 7efd98dce640 1 -- start start 2024-04-25T08:42:43.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.375+0000 7efd98dce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd94099fb0 0x7efd940c7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.375+0000 7efd98dce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd940d0d10 0x7efd940cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.375+0000 7efd98dce640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd9415def0 0x7efd940c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.375+0000 7efd98dce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd940d1170 con 0x7efd940d0d10 2024-04-25T08:42:43.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.375+0000 7efd98dce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd940d07f0 con 0x7efd94099fb0 2024-04-25T08:42:43.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.375+0000 7efd98dce640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd940cf8d0 con 0x7efd9415def0 2024-04-25T08:42:43.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd91d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd940d0d10 0x7efd940cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd91d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd940d0d10 0x7efd940cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35092/0 (socket says 172.21.15.38:35092) 2024-04-25T08:42:43.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd91d9b640 1 -- 172.21.15.38:0/1485099496 learned_addr learned my addr 172.21.15.38:0/1485099496 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:43.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd92d9d640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd9415def0 0x7efd940c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd9259c640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd94099fb0 0x7efd940c7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd91d9b640 1 -- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd9415def0 msgr2=0x7efd940c8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd91d9b640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd9415def0 0x7efd940c8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd91d9b640 1 -- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd94099fb0 msgr2=0x7efd940c7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd91d9b640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd94099fb0 0x7efd940c7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd91d9b640 1 -- 172.21.15.38:0/1485099496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd940c65f0 con 0x7efd940d0d10 2024-04-25T08:42:43.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd91d9b640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd940d0d10 0x7efd940cb440 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7efd8800d800 tx=0x7efd8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:43.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd82ffd640 1 -- 172.21.15.38:0/1485099496 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8800b6c0 con 0x7efd940d0d10 2024-04-25T08:42:43.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd82ffd640 1 -- 172.21.15.38:0/1485099496 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd88019070 con 0x7efd940d0d10 2024-04-25T08:42:43.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd98dce640 1 -- 172.21.15.38:0/1485099496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd940c68d0 con 0x7efd940d0d10 2024-04-25T08:42:43.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd82ffd640 1 -- 172.21.15.38:0/1485099496 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8800ba80 con 0x7efd940d0d10 2024-04-25T08:42:43.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.376+0000 7efd98dce640 1 -- 172.21.15.38:0/1485099496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd940c6e90 con 0x7efd940d0d10 2024-04-25T08:42:43.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.377+0000 7efd98dce640 1 -- 172.21.15.38:0/1485099496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd9404ef90 con 0x7efd940d0d10 2024-04-25T08:42:43.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.378+0000 7efd82ffd640 1 -- 172.21.15.38:0/1485099496 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd8800bc20 con 0x7efd940d0d10 2024-04-25T08:42:43.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.378+0000 7efd82ffd640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd7406b4d0 0x7efd7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.378+0000 7efd82ffd640 1 -- 172.21.15.38:0/1485099496 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efd8808a3f0 con 0x7efd940d0d10 2024-04-25T08:42:43.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.378+0000 7efd9259c640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd7406b4d0 0x7efd7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.379+0000 7efd9259c640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd7406b4d0 0x7efd7406d990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7efd84007fd0 tx=0x7efd84007de0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:43.375 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.382+0000 7efd82ffd640 1 -- 172.21.15.38:0/1485099496 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd88058d90 con 0x7efd940d0d10 2024-04-25T08:42:43.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.561+0000 7efd98dce640 1 -- 172.21.15.38:0/1485099496 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7efd9409af90 con 0x7efd7406b4d0 2024-04-25T08:42:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.562+0000 7efd82ffd640 1 -- 172.21.15.38:0/1485099496 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efd9409af90 con 0x7efd7406b4d0 2024-04-25T08:42:43.554 INFO:teuthology.orchestra.run.smithi038.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T08:42:43.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.564+0000 7efd98dce640 1 -- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd7406b4d0 msgr2=0x7efd7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.564+0000 7efd98dce640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd7406b4d0 0x7efd7406d990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7efd84007fd0 tx=0x7efd84007de0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.564+0000 7efd98dce640 1 -- 172.21.15.38:0/1485099496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd940d0d10 msgr2=0x7efd940cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.564+0000 7efd98dce640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd940d0d10 0x7efd940cb440 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7efd8800d800 tx=0x7efd8800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.564+0000 7efd98dce640 1 -- 172.21.15.38:0/1485099496 shutdown_connections 2024-04-25T08:42:43.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.564+0000 7efd98dce640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7efd7406b4d0 0x7efd7406d990 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.564+0000 7efd98dce640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7efd9415def0 0x7efd940c8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.564+0000 7efd98dce640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd940d0d10 0x7efd940cb440 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.564+0000 7efd98dce640 1 --2- 172.21.15.38:0/1485099496 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efd94099fb0 0x7efd940c7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.564+0000 7efd98dce640 1 -- 172.21.15.38:0/1485099496 >> 172.21.15.38:0/1485099496 conn(0x7efd940b30f0 msgr2=0x7efd9406ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.564+0000 7efd98dce640 1 -- 172.21.15.38:0/1485099496 shutdown_connections 2024-04-25T08:42:43.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.564+0000 7efd98dce640 1 -- 172.21.15.38:0/1485099496 wait complete. 2024-04-25T08:42:43.567 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:42:43.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.675+0000 7f2be9b90640 1 Processor -- start 2024-04-25T08:42:43.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.676+0000 7f2be9b90640 1 -- start start 2024-04-25T08:42:43.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.676+0000 7f2be9b90640 1 --2- >> v2:172.21.15.107:3300/0 conn(0x7f2be409a180 0x7f2be415d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.676+0000 7f2be9b90640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be40d05c0 con 0x7f2be406cc10 2024-04-25T08:42:43.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.676+0000 7f2be9b90640 1 -- --> v1:172.21.15.107:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be40d08a0 con 0x7f2be406cff0 2024-04-25T08:42:43.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.676+0000 7f2be9b90640 1 -- --> v2:172.21.15.107:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be40d0b80 con 0x7f2be409a180 2024-04-25T08:42:43.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.677+0000 7f2be37fe640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f2be406cc10 0x7f2be40cfa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:33942/0 (socket says 172.21.15.38:33942) 2024-04-25T08:42:43.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.677+0000 7f2be37fe640 1 -- 172.21.15.38:0/3724229344 learned_addr learned my addr 172.21.15.38:0/3724229344 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:42:43.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.677+0000 7f2be3fff640 1 --2- 172.21.15.38:0/3724229344 >> v2:172.21.15.107:3300/0 conn(0x7f2be409a180 0x7f2be415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.677+0000 7f2be27fc640 1 -- 172.21.15.38:0/3724229344 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591942329 0 0) 0x7f2be40d05c0 con 0x7f2be406cc10 2024-04-25T08:42:43.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.677+0000 7f2be27fc640 1 -- 172.21.15.38:0/3724229344 --> v1:172.21.15.38:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bcc003520 con 0x7f2be406cc10 2024-04-25T08:42:43.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.677+0000 7f2be3fff640 1 -- 172.21.15.38:0/3724229344 >> v1:172.21.15.107:6789/0 conn(0x7f2be406cff0 legacy=0x7f2be4099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.677+0000 7f2be3fff640 1 -- 172.21.15.38:0/3724229344 >> v1:172.21.15.38:6789/0 conn(0x7f2be406cc10 legacy=0x7f2be40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.671 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.677+0000 7f2be3fff640 1 -- 172.21.15.38:0/3724229344 --> v2:172.21.15.107:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be40d0e60 con 0x7f2be409a180 2024-04-25T08:42:43.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.678+0000 7f2be3fff640 1 --2- 172.21.15.38:0/3724229344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be409a180 0x7f2be415d370 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009d90 tx=0x7f2bdc02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecf12a0de63f173a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:43.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.678+0000 7f2be27fc640 1 -- 172.21.15.38:0/3724229344 <== mon.1 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc007740 con 0x7f2be409a180 2024-04-25T08:42:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.678+0000 7f2be27fc640 1 -- 172.21.15.38:0/3724229344 <== mon.1 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc0078e0 con 0x7f2be409a180 2024-04-25T08:42:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.678+0000 7f2be27fc640 1 -- 172.21.15.38:0/3724229344 <== mon.1 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc007cb0 con 0x7f2be409a180 2024-04-25T08:42:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.678+0000 7f2be9b90640 1 -- 172.21.15.38:0/3724229344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be409a180 msgr2=0x7f2be415d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.678+0000 7f2be9b90640 1 --2- 172.21.15.38:0/3724229344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be409a180 0x7f2be415d370 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009d90 tx=0x7f2bdc02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.679+0000 7f2be9b90640 1 -- 172.21.15.38:0/3724229344 shutdown_connections 2024-04-25T08:42:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.679+0000 7f2be9b90640 1 --2- 172.21.15.38:0/3724229344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be409a180 0x7f2be415d370 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.679+0000 7f2be9b90640 1 -- 172.21.15.38:0/3724229344 >> 172.21.15.38:0/3724229344 conn(0x7f2be40b31f0 msgr2=0x7f2be40b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.679+0000 7f2be9b90640 1 -- 172.21.15.38:0/3724229344 shutdown_connections 2024-04-25T08:42:43.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.679+0000 7f2be9b90640 1 -- 172.21.15.38:0/3724229344 wait complete. 2024-04-25T08:42:43.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.680+0000 7f2be9b90640 1 Processor -- start 2024-04-25T08:42:43.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.680+0000 7f2be9b90640 1 -- start start 2024-04-25T08:42:43.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be9b90640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be406cc10 0x7f2be409eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be9b90640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2be406cff0 0x7f2be41dc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be9b90640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be409a180 0x7f2be409cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be9b90640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be406c2d0 con 0x7f2be406cc10 2024-04-25T08:42:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be9b90640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be406c150 con 0x7f2be409a180 2024-04-25T08:42:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be9b90640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be40d0180 con 0x7f2be406cff0 2024-04-25T08:42:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be406cc10 0x7f2be409eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be406cc10 0x7f2be409eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35100/0 (socket says 172.21.15.38:35100) 2024-04-25T08:42:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be37fe640 1 -- 172.21.15.38:0/288839277 learned_addr learned my addr 172.21.15.38:0/288839277 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:42:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be2ffd640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2be406cff0 0x7f2be41dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be3fff640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be409a180 0x7f2be409cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be37fe640 1 -- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2be406cff0 msgr2=0x7f2be41dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be37fe640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2be406cff0 0x7f2be41dc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be37fe640 1 -- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be409a180 msgr2=0x7f2be409cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be37fe640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be409a180 0x7f2be409cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be37fe640 1 -- 172.21.15.38:0/288839277 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be409b7f0 con 0x7f2be406cc10 2024-04-25T08:42:43.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be37fe640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be406cc10 0x7f2be409eea0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400c500 tx=0x7f2bd400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:43.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2bcbfff640 1 -- 172.21.15.38:0/288839277 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4017070 con 0x7f2be406cc10 2024-04-25T08:42:43.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2bcbfff640 1 -- 172.21.15.38:0/288839277 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd4005ce0 con 0x7f2be406cc10 2024-04-25T08:42:43.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be9b90640 1 -- 172.21.15.38:0/288839277 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be409bad0 con 0x7f2be406cc10 2024-04-25T08:42:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2bcbfff640 1 -- 172.21.15.38:0/288839277 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4007330 con 0x7f2be406cc10 2024-04-25T08:42:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.681+0000 7f2be9b90640 1 -- 172.21.15.38:0/288839277 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be409c090 con 0x7f2be406cc10 2024-04-25T08:42:43.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.682+0000 7f2be9b90640 1 -- 172.21.15.38:0/288839277 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be4068e80 con 0x7f2be406cc10 2024-04-25T08:42:43.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.683+0000 7f2bcbfff640 1 -- 172.21.15.38:0/288839277 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bd401a070 con 0x7f2be406cc10 2024-04-25T08:42:43.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.683+0000 7f2bcbfff640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2bd006b4d0 0x7f2bd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:42:43.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.684+0000 7f2bcbfff640 1 -- 172.21.15.38:0/288839277 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2bd4089580 con 0x7f2be406cc10 2024-04-25T08:42:43.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.684+0000 7f2be2ffd640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2bd006b4d0 0x7f2bd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:42:43.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.684+0000 7f2be2ffd640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2bd006b4d0 0x7f2bd006d990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2be41e49e0 tx=0x7f2bd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:42:43.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.688+0000 7f2bcbfff640 1 -- 172.21.15.38:0/288839277 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2bd4058620 con 0x7f2be406cc10 2024-04-25T08:42:43.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.867+0000 7f2be9b90640 1 -- 172.21.15.38:0/288839277 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2be406d450 con 0x7f2bd006b4d0 2024-04-25T08:42:43.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.868+0000 7f2bcbfff640 1 -- 172.21.15.38:0/288839277 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13412 (secure 0 0 0) 0x7f2be406d450 con 0x7f2bd006b4d0 2024-04-25T08:42:43.860 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:42:43.861 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:42:43.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.870+0000 7f2be9b90640 1 -- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2bd006b4d0 msgr2=0x7f2bd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.870+0000 7f2be9b90640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2bd006b4d0 0x7f2bd006d990 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2be41e49e0 tx=0x7f2bd8009040 comp rx=0 tx=0).stop 2024-04-25T08:42:43.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.870+0000 7f2be9b90640 1 -- 172.21.15.38:0/288839277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be406cc10 msgr2=0x7f2be409eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:42:43.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.870+0000 7f2be9b90640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be406cc10 0x7f2be409eea0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400c500 tx=0x7f2bd400c9c0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.870+0000 7f2be9b90640 1 -- 172.21.15.38:0/288839277 shutdown_connections 2024-04-25T08:42:43.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.870+0000 7f2be9b90640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be409a180 0x7f2be409cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.870+0000 7f2be9b90640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f2bd006b4d0 0x7f2bd006d990 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.870+0000 7f2be9b90640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2be406cff0 0x7f2be41dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.870+0000 7f2be9b90640 1 --2- 172.21.15.38:0/288839277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2be406cc10 0x7f2be409eea0 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:42:43.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.870+0000 7f2be9b90640 1 -- 172.21.15.38:0/288839277 >> 172.21.15.38:0/288839277 conn(0x7f2be40b31f0 msgr2=0x7f2be41e5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:42:43.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.870+0000 7f2be9b90640 1 -- 172.21.15.38:0/288839277 shutdown_connections 2024-04-25T08:42:43.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:42:43.870+0000 7f2be9b90640 1 -- 172.21.15.38:0/288839277 wait complete. 2024-04-25T08:42:43.873 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-04-25T08:42:43.362083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":506660,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":490803,"kb_avail":754468060,"statfs":{"total":773094113280,"available":772575293440,"internally_reserved":0,"allocated":16097280,"data_stored":8160952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":502582832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1073741824,"num_objects":-1024,"num_object_clones":0,"num_object_copies":-2048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1024,"num_write_kb":-1048576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-50,"num_bytes_recovered":-52428800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.534992"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":411,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T08:42:35.897303+0000","last_change":"2024-04-25T08:31:24.397085+0000","last_active":"2024-04-25T08:42:35.897303+0000","last_peered":"2024-04-25T08:42:35.897303+0000","last_clean":"2024-04-25T08:42:35.897303+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:42:35.897303+0000","last_undegraded":"2024-04-25T08:42:35.897303+0000","last_fullsized":"2024-04-25T08:42:35.897303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:31:23.291180+0000","last_clean_scrub_stamp":"2024-04-25T08:31:23.291180+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:32.189923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69676,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302164,"statfs":{"total":96636764160,"available":96565415936,"internally_reserved":0,"allocated":2457600,"data_stored":1462895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46892,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324948,"statfs":{"total":96636764160,"available":96588746752,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59368,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73256,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298584,"statfs":{"total":96636764160,"available":96561750016,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77424,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294416,"statfs":{"total":96636764160,"available":96557481984,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46892,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324948,"statfs":{"total":96636764160,"available":96588746752,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59376,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312464,"statfs":{"total":96636764160,"available":96575963136,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73776,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298064,"statfs":{"total":96636764160,"available":96561217536,"internally_reserved":0,"allocated":2457600,"data_stored":1462895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:42:43.875 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T08:31:23.291180+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=8, tm_min=31, tm_sec=23, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=8, tm_min=42, tm_sec=38, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T08:42:43.876 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T08:43:03.877 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:43:03.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:03.999+0000 7f331f9b7640 1 Processor -- start 2024-04-25T08:43:03.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:03.999+0000 7f331f9b7640 1 -- start start 2024-04-25T08:43:03.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.000+0000 7f331f9b7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3318099a70 0x7f3318099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.000+0000 7f331f9b7640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33180cf470 0x7f33180cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:03.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.000+0000 7f331f9b7640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331809a390 con 0x7f331806bd80 2024-04-25T08:43:03.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.000+0000 7f331f9b7640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33181614d0 con 0x7f33180cf470 2024-04-25T08:43:03.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.000+0000 7f331f9b7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33180cd970 con 0x7f3318099a70 2024-04-25T08:43:03.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.000+0000 7f331cf52640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f33180cf470 0x7f33180cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.000+0000 7f331df54640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f331806bd80 0x7f331806c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41470/0 (socket says 172.21.15.38:41470) 2024-04-25T08:43:03.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.000+0000 7f331df54640 1 -- 172.21.15.38:0/663561891 learned_addr learned my addr 172.21.15.38:0/663561891 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:43:03.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.000+0000 7f331d753640 1 --2- 172.21.15.38:0/663561891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3318099a70 0x7f3318099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:03.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.001+0000 7f331cf52640 1 -- 172.21.15.38:0/663561891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3318099a70 msgr2=0x7f3318099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:03.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.001+0000 7f331cf52640 1 --2- 172.21.15.38:0/663561891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3318099a70 0x7f3318099e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.001+0000 7f331cf52640 1 -- 172.21.15.38:0/663561891 >> v1:172.21.15.38:6789/0 conn(0x7f331806bd80 legacy=0x7f331806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.001+0000 7f331cf52640 1 -- 172.21.15.38:0/663561891 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33180ccd60 con 0x7f33180cf470 2024-04-25T08:43:03.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.001+0000 7f330ffff640 1 -- 172.21.15.38:0/663561891 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002781504 0 0) 0x7f331809a390 con 0x7f331806bd80 2024-04-25T08:43:03.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.001+0000 7f331d753640 1 --2- 172.21.15.38:0/663561891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3318099a70 0x7f3318099e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:03.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.001+0000 7f331cf52640 1 --2- 172.21.15.38:0/663561891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33180cf470 0x7f33180cc480 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f33080097a0 tx=0x7f330802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cafd57565315f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:03.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.001+0000 7f330ffff640 1 -- 172.21.15.38:0/663561891 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308037070 con 0x7f33180cf470 2024-04-25T08:43:03.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.002+0000 7f330ffff640 1 -- 172.21.15.38:0/663561891 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3308032440 con 0x7f33180cf470 2024-04-25T08:43:03.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.002+0000 7f330ffff640 1 -- 172.21.15.38:0/663561891 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3308032810 con 0x7f33180cf470 2024-04-25T08:43:03.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.002+0000 7f331f9b7640 1 -- 172.21.15.38:0/663561891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33180cf470 msgr2=0x7f33180cc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:03.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.002+0000 7f331f9b7640 1 --2- 172.21.15.38:0/663561891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33180cf470 0x7f33180cc480 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f33080097a0 tx=0x7f330802a180 comp rx=0 tx=0).stop 2024-04-25T08:43:03.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.002+0000 7f331f9b7640 1 -- 172.21.15.38:0/663561891 shutdown_connections 2024-04-25T08:43:03.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.002+0000 7f331f9b7640 1 --2- 172.21.15.38:0/663561891 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33180cf470 0x7f33180cc480 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.002+0000 7f331f9b7640 1 --2- 172.21.15.38:0/663561891 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3318099a70 0x7f3318099e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:03.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.002+0000 7f331f9b7640 1 -- 172.21.15.38:0/663561891 >> 172.21.15.38:0/663561891 conn(0x7f33180b3180 msgr2=0x7f33180b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:03.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.002+0000 7f331f9b7640 1 -- 172.21.15.38:0/663561891 shutdown_connections 2024-04-25T08:43:03.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.002+0000 7f331f9b7640 1 -- 172.21.15.38:0/663561891 wait complete. 2024-04-25T08:43:03.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.003+0000 7f331f9b7640 1 Processor -- start 2024-04-25T08:43:04.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.003+0000 7f331f9b7640 1 -- start start 2024-04-25T08:43:04.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.004+0000 7f331f9b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f331806bd80 0x7f33181f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.004+0000 7f331f9b7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3318099a70 0x7f331815fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.004+0000 7f331f9b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33180cf470 0x7f33180b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.004+0000 7f331f9b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33180cd600 con 0x7f33180cf470 2024-04-25T08:43:04.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.004+0000 7f331f9b7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33180cd320 con 0x7f331806bd80 2024-04-25T08:43:04.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.004+0000 7f331f9b7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33180cc9c0 con 0x7f3318099a70 2024-04-25T08:43:04.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.004+0000 7f331df54640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33180cf470 0x7f33180b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.004+0000 7f331df54640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33180cf470 0x7f33180b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58280/0 (socket says 172.21.15.38:58280) 2024-04-25T08:43:04.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.004+0000 7f331df54640 1 -- 172.21.15.38:0/4267689357 learned_addr learned my addr 172.21.15.38:0/4267689357 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:43:04.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.004+0000 7f331cf52640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3318099a70 0x7f331815fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.004+0000 7f331d753640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f331806bd80 0x7f33181f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.005+0000 7f331df54640 1 -- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3318099a70 msgr2=0x7f331815fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.005+0000 7f331df54640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3318099a70 0x7f331815fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.005+0000 7f331df54640 1 -- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f331806bd80 msgr2=0x7f33181f4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.005+0000 7f331df54640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f331806bd80 0x7f33181f4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.005+0000 7f331df54640 1 -- 172.21.15.38:0/4267689357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33180b41b0 con 0x7f33180cf470 2024-04-25T08:43:04.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.005+0000 7f331d753640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f331806bd80 0x7f33181f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:43:04.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.005+0000 7f331df54640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33180cf470 0x7f33180b4aa0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f331000c7a0 tx=0x7f331000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.005+0000 7f330dffb640 1 -- 172.21.15.38:0/4267689357 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310018070 con 0x7f33180cf470 2024-04-25T08:43:04.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.005+0000 7f330dffb640 1 -- 172.21.15.38:0/4267689357 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331000a640 con 0x7f33180cf470 2024-04-25T08:43:04.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.005+0000 7f330dffb640 1 -- 172.21.15.38:0/4267689357 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331000aa70 con 0x7f33180cf470 2024-04-25T08:43:04.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.005+0000 7f331f9b7640 1 -- 172.21.15.38:0/4267689357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3318160270 con 0x7f33180cf470 2024-04-25T08:43:04.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.005+0000 7f331f9b7640 1 -- 172.21.15.38:0/4267689357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33180b4700 con 0x7f33180cf470 2024-04-25T08:43:04.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.006+0000 7f331f9b7640 1 -- 172.21.15.38:0/4267689357 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3318068e80 con 0x7f33180cf470 2024-04-25T08:43:04.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.007+0000 7f330dffb640 1 -- 172.21.15.38:0/4267689357 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f331001b070 con 0x7f33180cf470 2024-04-25T08:43:04.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.007+0000 7f330dffb640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32fc06b4d0 0x7f32fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.007+0000 7f330dffb640 1 -- 172.21.15.38:0/4267689357 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3310089590 con 0x7f33180cf470 2024-04-25T08:43:04.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.008+0000 7f331d753640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32fc06b4d0 0x7f32fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.008+0000 7f331d753640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32fc06b4d0 0x7f32fc06d990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f33180c81d0 tx=0x7f3314008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.012+0000 7f330dffb640 1 -- 172.21.15.38:0/4267689357 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3310057f30 con 0x7f33180cf470 2024-04-25T08:43:04.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.189+0000 7f331f9b7640 1 -- 172.21.15.38:0/4267689357 --> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f331806e370 con 0x7f32fc06b4d0 2024-04-25T08:43:04.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.190+0000 7f330dffb640 1 -- 172.21.15.38:0/4267689357 <== mgr.4105 v2:172.21.15.107:6832/1055593366 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13380 (secure 0 0 0) 0x7f331806e370 con 0x7f32fc06b4d0 2024-04-25T08:43:04.182 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:43:04.182 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-04-25T08:43:04.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.192+0000 7f331f9b7640 1 -- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32fc06b4d0 msgr2=0x7f32fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.192+0000 7f331f9b7640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32fc06b4d0 0x7f32fc06d990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f33180c81d0 tx=0x7f3314008040 comp rx=0 tx=0).stop 2024-04-25T08:43:04.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.192+0000 7f331f9b7640 1 -- 172.21.15.38:0/4267689357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33180cf470 msgr2=0x7f33180b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.192+0000 7f331f9b7640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33180cf470 0x7f33180b4aa0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f331000c7a0 tx=0x7f331000cc60 comp rx=0 tx=0).stop 2024-04-25T08:43:04.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.192+0000 7f331f9b7640 1 -- 172.21.15.38:0/4267689357 shutdown_connections 2024-04-25T08:43:04.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.192+0000 7f331f9b7640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f33180cf470 0x7f33180b4aa0 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.192+0000 7f331f9b7640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f32fc06b4d0 0x7f32fc06d990 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.192+0000 7f331f9b7640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3318099a70 0x7f331815fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.192+0000 7f331f9b7640 1 --2- 172.21.15.38:0/4267689357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f331806bd80 0x7f33181f4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.192+0000 7f331f9b7640 1 -- 172.21.15.38:0/4267689357 >> 172.21.15.38:0/4267689357 conn(0x7f33180b3180 msgr2=0x7f3318099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:04.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.192+0000 7f331f9b7640 1 -- 172.21.15.38:0/4267689357 shutdown_connections 2024-04-25T08:43:04.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.192+0000 7f331f9b7640 1 -- 172.21.15.38:0/4267689357 wait complete. 2024-04-25T08:43:04.195 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-04-25T08:43:03.366317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":506648,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":490803,"kb_avail":754468072,"statfs":{"total":773094113280,"available":772575305728,"internally_reserved":0,"allocated":16097280,"data_stored":8160952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":502582832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002699"},"pg_stats":[{"pgid":"1.0","version":"148'119","reported_seq":422,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T08:42:44.402167+0000","last_change":"2024-04-25T08:42:44.402201+0000","last_active":"2024-04-25T08:42:44.402167+0000","last_peered":"2024-04-25T08:42:44.402167+0000","last_clean":"2024-04-25T08:42:44.402167+0000","last_became_active":"2024-04-25T08:31:24.396188+0000","last_became_peered":"2024-04-25T08:31:24.396188+0000","last_unstale":"2024-04-25T08:42:44.402167+0000","last_undegraded":"2024-04-25T08:42:44.402167+0000","last_fullsized":"2024-04-25T08:42:44.402167+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"148'119","last_scrub_stamp":"2024-04-25T08:42:44.402084+0000","last_deep_scrub":"148'119","last_deep_scrub_stamp":"2024-04-25T08:42:44.402084+0000","last_clean_scrub_stamp":"2024-04-25T08:42:44.402084+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:45.991646+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574992,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69672,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302168,"statfs":{"total":96636764160,"available":96565420032,"internally_reserved":0,"allocated":2457600,"data_stored":1462895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":68876746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46892,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324948,"statfs":{"total":96636764160,"available":96588746752,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59368,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312472,"statfs":{"total":96636764160,"available":96575971328,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73256,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298584,"statfs":{"total":96636764160,"available":96561750016,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77424,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294416,"statfs":{"total":96636764160,"available":96557481984,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":46892,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324948,"statfs":{"total":96636764160,"available":96588746752,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":59376,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312464,"statfs":{"total":96636764160,"available":96575963136,"internally_reserved":0,"allocated":1863680,"data_stored":872527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574992,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73768,"kb_used_data":2400,"kb_used_omap":1,"kb_used_meta":71358,"kb_avail":94298072,"statfs":{"total":96636764160,"available":96561225728,"internally_reserved":0,"allocated":2457600,"data_stored":1462895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73071045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:43:04.196 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T08:43:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.304+0000 7fad6c207640 1 Processor -- start 2024-04-25T08:43:04.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.304+0000 7fad6c207640 1 -- start start 2024-04-25T08:43:04.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad6c207640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad640d0320 0x7fad640d33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad6c207640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fad640b6980 0x7fad640b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad6c207640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad640d2060 con 0x7fad640d0700 2024-04-25T08:43:04.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad6c207640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad640d2330 con 0x7fad640b6980 2024-04-25T08:43:04.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad6c207640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad640d2610 con 0x7fad640d0320 2024-04-25T08:43:04.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad6a7a4640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7fad640b6980 0x7fad640b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad697a2640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7fad640d0700 0x7fad640d1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41486/0 (socket says 172.21.15.38:41486) 2024-04-25T08:43:04.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad697a2640 1 -- 172.21.15.38:0/1956529139 learned_addr learned my addr 172.21.15.38:0/1956529139 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:43:04.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad69fa3640 1 --2- 172.21.15.38:0/1956529139 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad640d0320 0x7fad640d33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad6a7a4640 1 -- 172.21.15.38:0/1956529139 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad640d0320 msgr2=0x7fad640d33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:04.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad6a7a4640 1 --2- 172.21.15.38:0/1956529139 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad640d0320 0x7fad640d33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad6a7a4640 1 -- 172.21.15.38:0/1956529139 >> v1:172.21.15.38:6789/0 conn(0x7fad640d0700 legacy=0x7fad640d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad6a7a4640 1 -- 172.21.15.38:0/1956529139 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad640d28f0 con 0x7fad640b6980 2024-04-25T08:43:04.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.305+0000 7fad68fa1640 1 -- 172.21.15.38:0/1956529139 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674908987 0 0) 0x7fad640d2060 con 0x7fad640d0700 2024-04-25T08:43:04.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.306+0000 7fad6a7a4640 1 --2- 172.21.15.38:0/1956529139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad640b6980 0x7fad640b6e10 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009d90 tx=0x7fad5c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199b5a65e62aaf67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.306+0000 7fad68fa1640 1 -- 172.21.15.38:0/1956529139 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c007740 con 0x7fad640b6980 2024-04-25T08:43:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.306+0000 7fad68fa1640 1 -- 172.21.15.38:0/1956529139 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c0078e0 con 0x7fad640b6980 2024-04-25T08:43:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.306+0000 7fad68fa1640 1 -- 172.21.15.38:0/1956529139 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5c007cb0 con 0x7fad640b6980 2024-04-25T08:43:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.307+0000 7fad6c207640 1 -- 172.21.15.38:0/1956529139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad640b6980 msgr2=0x7fad640b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.307+0000 7fad6c207640 1 --2- 172.21.15.38:0/1956529139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad640b6980 0x7fad640b6e10 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fad5c009d90 tx=0x7fad5c02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.307+0000 7fad6c207640 1 -- 172.21.15.38:0/1956529139 shutdown_connections 2024-04-25T08:43:04.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.307+0000 7fad6c207640 1 --2- 172.21.15.38:0/1956529139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad640b6980 0x7fad640b6e10 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.307+0000 7fad6c207640 1 --2- 172.21.15.38:0/1956529139 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad640d0320 0x7fad640d33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.307+0000 7fad6c207640 1 -- 172.21.15.38:0/1956529139 >> 172.21.15.38:0/1956529139 conn(0x7fad640668f0 msgr2=0x7fad64066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:04.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.307+0000 7fad6c207640 1 -- 172.21.15.38:0/1956529139 shutdown_connections 2024-04-25T08:43:04.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.307+0000 7fad6c207640 1 -- 172.21.15.38:0/1956529139 wait complete. 2024-04-25T08:43:04.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.308+0000 7fad6c207640 1 Processor -- start 2024-04-25T08:43:04.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.308+0000 7fad6c207640 1 -- start start 2024-04-25T08:43:04.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad6c207640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad640b6980 0x7fad640c97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad6c207640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad640d0320 0x7fad641569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.304 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad6c207640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad640d0700 0x7fad64156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad6c207640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad640d0b40 con 0x7fad640d0320 2024-04-25T08:43:04.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad6c207640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad640d2eb0 con 0x7fad640d0700 2024-04-25T08:43:04.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad6c207640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad640c9d10 con 0x7fad640b6980 2024-04-25T08:43:04.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad697a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad640d0320 0x7fad641569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad697a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad640d0320 0x7fad641569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58298/0 (socket says 172.21.15.38:58298) 2024-04-25T08:43:04.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad697a2640 1 -- 172.21.15.38:0/1508522721 learned_addr learned my addr 172.21.15.38:0/1508522721 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:43:04.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad69fa3640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad640b6980 0x7fad640c97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad6a7a4640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad640d0700 0x7fad64156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad697a2640 1 -- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad640b6980 msgr2=0x7fad640c97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad697a2640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad640b6980 0x7fad640c97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad697a2640 1 -- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad640d0700 msgr2=0x7fad64156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad697a2640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad640d0700 0x7fad64156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad697a2640 1 -- 172.21.15.38:0/1508522721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad640b3f30 con 0x7fad640d0320 2024-04-25T08:43:04.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.309+0000 7fad697a2640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad640d0320 0x7fad641569a0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fad5400c3f0 tx=0x7fad5400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.310+0000 7fad5a7fc640 1 -- 172.21.15.38:0/1508522721 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad540074e0 con 0x7fad640d0320 2024-04-25T08:43:04.306 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.310+0000 7fad5a7fc640 1 -- 172.21.15.38:0/1508522721 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad540047e0 con 0x7fad640d0320 2024-04-25T08:43:04.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.310+0000 7fad6c207640 1 -- 172.21.15.38:0/1508522721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad640b4210 con 0x7fad640d0320 2024-04-25T08:43:04.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.310+0000 7fad5a7fc640 1 -- 172.21.15.38:0/1508522721 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54004c10 con 0x7fad640d0320 2024-04-25T08:43:04.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.310+0000 7fad6c207640 1 -- 172.21.15.38:0/1508522721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad640b47d0 con 0x7fad640d0320 2024-04-25T08:43:04.307 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.311+0000 7fad6c207640 1 -- 172.21.15.38:0/1508522721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad640d47d0 con 0x7fad640d0320 2024-04-25T08:43:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.311+0000 7fad5a7fc640 1 -- 172.21.15.38:0/1508522721 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad54011070 con 0x7fad640d0320 2024-04-25T08:43:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.312+0000 7fad5a7fc640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad4806b4d0 0x7fad4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.312+0000 7fad5a7fc640 1 -- 172.21.15.38:0/1508522721 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fad54089ba0 con 0x7fad640d0320 2024-04-25T08:43:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.312+0000 7fad69fa3640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad4806b4d0 0x7fad4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.312+0000 7fad69fa3640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad4806b4d0 0x7fad4806d990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fad60009b40 tx=0x7fad60009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.316+0000 7fad5a7fc640 1 -- 172.21.15.38:0/1508522721 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad640d47d0 con 0x7fad640d0320 2024-04-25T08:43:04.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.498+0000 7fad6c207640 1 -- 172.21.15.38:0/1508522721 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fad64068e80 con 0x7fad640d0320 2024-04-25T08:43:04.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.505+0000 7fad5a7fc640 1 -- 172.21.15.38:0/1508522721 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v28)=0 v28) v1 ==== 123+0+0 (secure 0 0 0) 0x7fad54058b90 con 0x7fad640d0320 2024-04-25T08:43:04.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.506+0000 7fad5a7fc640 1 -- 172.21.15.38:0/1508522721 <== mon.0 v2:172.21.15.38:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fad54058490 con 0x7fad640d0320 2024-04-25T08:43:04.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.508+0000 7fad6c207640 1 -- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad4806b4d0 msgr2=0x7fad4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.508+0000 7fad6c207640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad4806b4d0 0x7fad4806d990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fad60009b40 tx=0x7fad60009190 comp rx=0 tx=0).stop 2024-04-25T08:43:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.508+0000 7fad6c207640 1 -- 172.21.15.38:0/1508522721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad640d0320 msgr2=0x7fad641569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.508+0000 7fad6c207640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad640d0320 0x7fad641569a0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fad5400c3f0 tx=0x7fad5400c8b0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.508+0000 7fad6c207640 1 -- 172.21.15.38:0/1508522721 shutdown_connections 2024-04-25T08:43:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.508+0000 7fad6c207640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad640d0700 0x7fad64156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.508+0000 7fad6c207640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fad640d0320 0x7fad641569a0 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.508+0000 7fad6c207640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7fad4806b4d0 0x7fad4806d990 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.508+0000 7fad6c207640 1 --2- 172.21.15.38:0/1508522721 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fad640b6980 0x7fad640c97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.508+0000 7fad6c207640 1 -- 172.21.15.38:0/1508522721 >> 172.21.15.38:0/1508522721 conn(0x7fad640668f0 msgr2=0x7fad640b7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:04.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.508+0000 7fad6c207640 1 -- 172.21.15.38:0/1508522721 shutdown_connections 2024-04-25T08:43:04.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.508+0000 7fad6c207640 1 -- 172.21.15.38:0/1508522721 wait complete. 2024-04-25T08:43:04.511 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:43:04.511 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:43:04.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.620+0000 7f6c24563640 1 Processor -- start 2024-04-25T08:43:04.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.620+0000 7f6c24563640 1 -- start start 2024-04-25T08:43:04.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c24563640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6c1c15df50 0x7f6c1c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c24563640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c1c0ce2e0 0x7f6c1c0d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c24563640 1 -- --> v1:172.21.15.38:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1c0ceff0 con 0x7f6c1c06c110 2024-04-25T08:43:04.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c24563640 1 -- --> v2:172.21.15.38:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c1c0cf2d0 con 0x7f6c1c15df50 2024-04-25T08:43:04.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c24563640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0d1f70 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c21afe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6c1c15df50 0x7f6c1c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c222ff640 1 --1- >> v1:172.21.15.38:6789/0 conn(0x7f6c1c06c110 0x7f6c1c06c4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.38:6789/0 says I am v1:172.21.15.38:41502/0 (socket says 172.21.15.38:41502) 2024-04-25T08:43:04.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c21afe640 1 --2- >> v2:172.21.15.38:3300/0 conn(0x7f6c1c15df50 0x7f6c1c0ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58306/0 (socket says 172.21.15.38:58306) 2024-04-25T08:43:04.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c222ff640 1 -- 172.21.15.38:0/494234019 learned_addr learned my addr 172.21.15.38:0/494234019 (peer_addr_for_me v1:172.21.15.38:0/0) 2024-04-25T08:43:04.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c22b00640 1 --2- 172.21.15.38:0/494234019 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c1c0ce2e0 0x7f6c1c0d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c21afe640 1 -- 172.21.15.38:0/494234019 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c1c0ce2e0 msgr2=0x7f6c1c0d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:43:04.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c21afe640 1 --2- 172.21.15.38:0/494234019 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c1c0ce2e0 0x7f6c1c0d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c21afe640 1 -- 172.21.15.38:0/494234019 >> v1:172.21.15.38:6789/0 conn(0x7f6c1c06c110 legacy=0x7f6c1c06c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c21afe640 1 -- 172.21.15.38:0/494234019 --> v2:172.21.15.38:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0d2250 con 0x7f6c1c15df50 2024-04-25T08:43:04.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.621+0000 7f6c212fd640 1 -- 172.21.15.38:0/494234019 <== mon.0 v1:172.21.15.38:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3351465299 0 0) 0x7f6c1c0ceff0 con 0x7f6c1c06c110 2024-04-25T08:43:04.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.622+0000 7f6c21afe640 1 --2- 172.21.15.38:0/494234019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1c15df50 0x7f6c1c0ce7a0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0097a0 tx=0x7f6c0c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ea664e161baeb8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.622+0000 7f6c212fd640 1 -- 172.21.15.38:0/494234019 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c037070 con 0x7f6c1c15df50 2024-04-25T08:43:04.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.622+0000 7f6c212fd640 1 -- 172.21.15.38:0/494234019 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6c0c032440 con 0x7f6c1c15df50 2024-04-25T08:43:04.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.622+0000 7f6c212fd640 1 -- 172.21.15.38:0/494234019 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c032830 con 0x7f6c1c15df50 2024-04-25T08:43:04.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.622+0000 7f6c24563640 1 -- 172.21.15.38:0/494234019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1c15df50 msgr2=0x7f6c1c0ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.622+0000 7f6c24563640 1 --2- 172.21.15.38:0/494234019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1c15df50 0x7f6c1c0ce7a0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0097a0 tx=0x7f6c0c02a180 comp rx=0 tx=0).stop 2024-04-25T08:43:04.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.623+0000 7f6c24563640 1 -- 172.21.15.38:0/494234019 shutdown_connections 2024-04-25T08:43:04.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.623+0000 7f6c24563640 1 --2- 172.21.15.38:0/494234019 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c1c0ce2e0 0x7f6c1c0d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.623+0000 7f6c24563640 1 --2- 172.21.15.38:0/494234019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1c15df50 0x7f6c1c0ce7a0 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.623+0000 7f6c24563640 1 -- 172.21.15.38:0/494234019 >> 172.21.15.38:0/494234019 conn(0x7f6c1c0d3fb0 msgr2=0x7f6c1c0d0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:04.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.623+0000 7f6c24563640 1 -- 172.21.15.38:0/494234019 shutdown_connections 2024-04-25T08:43:04.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.623+0000 7f6c24563640 1 -- 172.21.15.38:0/494234019 wait complete. 2024-04-25T08:43:04.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.624+0000 7f6c24563640 1 Processor -- start 2024-04-25T08:43:04.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.624+0000 7f6c24563640 1 -- start start 2024-04-25T08:43:04.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.624+0000 7f6c24563640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c1c06c110 0x7f6c1c15af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.624+0000 7f6c24563640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1c0ce2e0 0x7f6c1c0c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.620 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.624+0000 7f6c24563640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c1c15df50 0x7f6c1c0c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c24563640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0d2f90 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c24563640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0cece0 con 0x7f6c1c06c110 2024-04-25T08:43:04.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c24563640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1c0d3110 con 0x7f6c1c15df50 2024-04-25T08:43:04.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c21afe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1c0ce2e0 0x7f6c1c0c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c21afe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1c0ce2e0 0x7f6c1c0c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:58322/0 (socket says 172.21.15.38:58322) 2024-04-25T08:43:04.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c21afe640 1 -- 172.21.15.38:0/4078984577 learned_addr learned my addr 172.21.15.38:0/4078984577 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-04-25T08:43:04.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c22b00640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c1c15df50 0x7f6c1c0c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c222ff640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c1c06c110 0x7f6c1c15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.621 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c21afe640 1 -- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c1c15df50 msgr2=0x7f6c1c0c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c21afe640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c1c15df50 0x7f6c1c0c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c21afe640 1 -- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c1c06c110 msgr2=0x7f6c1c15af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c21afe640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c1c06c110 0x7f6c1c15af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c21afe640 1 -- 172.21.15.38:0/4078984577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0c89a0 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c222ff640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c1c06c110 0x7f6c1c15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:43:04.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c21afe640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1c0ce2e0 0x7f6c1c0c7e30 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c000c00 tx=0x7f6c0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c12ffd640 1 -- 172.21.15.38:0/4078984577 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c032e60 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.622 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c12ffd640 1 -- 172.21.15.38:0/4078984577 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6c0c042070 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c24563640 1 -- 172.21.15.38:0/4078984577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1c0c8c80 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c12ffd640 1 -- 172.21.15.38:0/4078984577 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c037040 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.625+0000 7f6c24563640 1 -- 172.21.15.38:0/4078984577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1c0c9240 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.626+0000 7f6c24563640 1 -- 172.21.15.38:0/4078984577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1c068e80 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.627+0000 7f6c12ffd640 1 -- 172.21.15.38:0/4078984577 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c0c030bf0 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.627+0000 7f6c12ffd640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6c0006b4d0 0x7f6c0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:43:04.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.627+0000 7f6c12ffd640 1 -- 172.21.15.38:0/4078984577 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6c0c0aaca0 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.627+0000 7f6c222ff640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6c0006b4d0 0x7f6c0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:43:04.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.628+0000 7f6c222ff640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6c0006b4d0 0x7f6c0006d990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6c18000f60 tx=0x7f6c18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:43:04.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.631+0000 7f6c12ffd640 1 -- 172.21.15.38:0/4078984577 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c0c045360 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.847+0000 7f6c24563640 1 -- 172.21.15.38:0/4078984577 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c1c0cf720 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.847+0000 7f6c12ffd640 1 -- 172.21.15.38:0/4078984577 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+3704 (secure 0 0 0) 0x7f6c0c079c00 con 0x7f6c1c0ce2e0 2024-04-25T08:43:04.840 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-04-25T08:43:04.840 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":93,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7574,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/2682394530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":2682394530},{"type":"v1","addr":"172.21.15.107:6837","nonce":2682394530}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":8491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/2810760276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":2810760276},{"type":"v1","addr":"172.21.15.107:6835","nonce":2810760276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":10359,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6833/886378126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":886378126},{"type":"v1","addr":"172.21.15.38:6833","nonce":886378126}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93},{"gid":10362,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/1031373794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":1031373794},{"type":"v1","addr":"172.21.15.38:6835","nonce":1031373794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93}],"filesystems":[]} 2024-04-25T08:43:04.841 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 93 2024-04-25T08:43:04.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.849+0000 7f6c24563640 1 -- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6c0006b4d0 msgr2=0x7f6c0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.849+0000 7f6c24563640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6c0006b4d0 0x7f6c0006d990 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f6c18000f60 tx=0x7f6c18009040 comp rx=0 tx=0).stop 2024-04-25T08:43:04.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.849+0000 7f6c24563640 1 -- 172.21.15.38:0/4078984577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1c0ce2e0 msgr2=0x7f6c1c0c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:43:04.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.849+0000 7f6c24563640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1c0ce2e0 0x7f6c1c0c7e30 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c000c00 tx=0x7f6c0c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.850+0000 7f6c24563640 1 -- 172.21.15.38:0/4078984577 shutdown_connections 2024-04-25T08:43:04.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.850+0000 7f6c24563640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:6832/1055593366,v1:172.21.15.107:6833/1055593366] conn(0x7f6c0006b4d0 0x7f6c0006d990 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.850+0000 7f6c24563640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6c1c15df50 0x7f6c1c0c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.850+0000 7f6c24563640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c1c0ce2e0 0x7f6c1c0c7e30 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.850+0000 7f6c24563640 1 --2- 172.21.15.38:0/4078984577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6c1c06c110 0x7f6c1c15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:43:04.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.850+0000 7f6c24563640 1 -- 172.21.15.38:0/4078984577 >> 172.21.15.38:0/4078984577 conn(0x7f6c1c0d3fb0 msgr2=0x7f6c1c15d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:43:04.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.850+0000 7f6c24563640 1 -- 172.21.15.38:0/4078984577 shutdown_connections 2024-04-25T08:43:04.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-04-25T08:43:04.850+0000 7f6c24563640 1 -- 172.21.15.38:0/4078984577 wait complete. 2024-04-25T08:43:04.853 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T08:43:04.853 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T08:43:04.853 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:43:04.853 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:04.862 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:43:04.862 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:43:04.863 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:04.912 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:43:04.912 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:43:04.912 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:04.923 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:43:04.924 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:43:04.924 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:04.940 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:43:04.954 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:43:04.975 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:43:04.976 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T08:43:04.976 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T08:43:04.976 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.036 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T08:43:05.036 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T08:43:05.036 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.104 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T08:43:05.105 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T08:43:05.105 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.181 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T08:43:05.181 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T08:43:05.181 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.257 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T08:43:05.258 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T08:43:05.258 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.298 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T08:43:05.298 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T08:43:05.298 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.339 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T08:43:05.340 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T08:43:05.340 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.381 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T08:43:05.439 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T08:43:05.439 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.482 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T08:43:05.482 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T08:43:05.483 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T08:43:05.483 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.508 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T08:43:05.508 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T08:43:05.508 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.536 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T08:43:05.536 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T08:43:05.537 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T08:43:05.537 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.559 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T08:43:05.560 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T08:43:05.560 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.589 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T08:43:05.589 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T08:43:05.589 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:43:05.640 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T08:43:05.640 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T08:43:05.641 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-25T08:43:05.680 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:43:05.680 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T08:43:05.926 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:43:05.927 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T08:43:06.006 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:43:06.006 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T08:43:06.065 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:43:06.065 DEBUG:teuthology.orchestra.run.smithi038:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T08:43:06.120 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:43:06.121 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T08:43:06.341 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:43:06.341 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T08:43:06.443 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:43:06.443 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T08:43:06.508 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:43:06.508 DEBUG:teuthology.orchestra.run.smithi107:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T08:43:06.584 INFO:tasks.ceph:Archiving mon data... 2024-04-25T08:43:06.586 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992/data/mon.a.tgz 2024-04-25T08:43:06.586 DEBUG:teuthology.orchestra.run.smithi038:> mktemp 2024-04-25T08:43:06.603 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.1dDTW5Cd6C 2024-04-25T08:43:06.603 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.1dDTW5Cd6C 2024-04-25T08:43:06.776 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.1dDTW5Cd6C 2024-04-25T08:43:06.858 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.1dDTW5Cd6C is 471KB 2024-04-25T08:43:06.956 DEBUG:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.1dDTW5Cd6C 2024-04-25T08:43:06.970 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992/data/mon.b.tgz 2024-04-25T08:43:06.970 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-04-25T08:43:06.988 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.AzqWPgGdY2 2024-04-25T08:43:06.989 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.AzqWPgGdY2 2024-04-25T08:43:07.155 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.AzqWPgGdY2 2024-04-25T08:43:07.235 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.AzqWPgGdY2 is 481KB 2024-04-25T08:43:07.304 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.AzqWPgGdY2 2024-04-25T08:43:07.319 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992/data/mon.c.tgz 2024-04-25T08:43:07.319 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-04-25T08:43:07.372 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.PQKlJJO1MK 2024-04-25T08:43:07.372 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.PQKlJJO1MK 2024-04-25T08:43:07.524 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.PQKlJJO1MK 2024-04-25T08:43:07.604 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.PQKlJJO1MK is 481KB 2024-04-25T08:43:07.701 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.PQKlJJO1MK 2024-04-25T08:43:07.715 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T08:43:07.715 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T08:43:07.718 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T08:43:07.798 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T08:43:07.832 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T08:43:07.832 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T08:43:07.834 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992/remote/smithi038/crash 2024-04-25T08:43:07.835 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T08:43:07.863 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992/remote/smithi107/crash 2024-04-25T08:43:07.864 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T08:43:07.894 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T08:43:07.895 DEBUG:teuthology.orchestra.run.smithi038:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:43:07.908 DEBUG:teuthology.orchestra.run.smithi107:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:43:07.981 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T08:43:08.005 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T08:43:08.041 INFO:tasks.ceph:Compressing logs... 2024-04-25T08:43:08.042 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T08:43:08.048 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T08:43:08.132 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34685.log 2024-04-25T08:43:08.132 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T08:43:08.132 INFO:teuthology.orchestra.run.smithi107.stderr:gzip/var/log/ceph/ceph-client.admin.34685.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34805.log 2024-04-25T08:43:08.132 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40918.log 2024-04-25T08:43:08.132 INFO:teuthology.orchestra.run.smithi107.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34685.log.gz 2024-04-25T08:43:08.132 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.34805.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T08:43:08.133 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.1.40918.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T08:43:08.133 INFO:teuthology.orchestra.run.smithi107.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34805.log.gz 2024-04-25T08:43:08.133 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T08:43:08.133 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.7.log: 90.0% -- replaced with /var/log/ceph/ceph-client.1.40918.log.gz 2024-04-25T08:43:08.133 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41366.log 2024-04-25T08:43:08.133 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T08:43:08.133 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-client.3.41366.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T08:43:08.133 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34596.log 2024-04-25T08:43:08.134 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.4.log: 91.1% -- replaced with /var/log/ceph/ceph-client.3.41366.log.gz 2024-04-25T08:43:08.134 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log/var/log/ceph/ceph-client.admin.34596.log: 2024-04-25T08:43:08.134 INFO:teuthology.orchestra.run.smithi107.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T08:43:08.134 INFO:teuthology.orchestra.run.smithi107.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34596.log.gz 2024-04-25T08:43:08.134 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mgr.x.log: gzip 92.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T08:43:08.134 INFO:teuthology.orchestra.run.smithi107.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T08:43:08.134 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34554.log 2024-04-25T08:43:08.134 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-osd.6.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T08:43:08.135 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34765.log 2024-04-25T08:43:08.135 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.34554.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34554.log.gz 2024-04-25T08:43:08.135 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34725.log 2024-04-25T08:43:08.135 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.34765.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34765.log.gz 2024-04-25T08:43:08.135 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34645.log 2024-04-25T08:43:08.135 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.34725.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34725.log.gz 2024-04-25T08:43:08.135 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34514.log 2024-04-25T08:43:08.136 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.34645.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34645.log.gz 2024-04-25T08:43:08.136 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T08:43:08.136 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.admin.34514.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34514.log.gz 2024-04-25T08:43:08.136 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T08:43:08.146 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 92.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T08:43:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99278.log 2024-04-25T08:43:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93727.log 2024-04-25T08:43:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46060.log 2024-04-25T08:43:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42954.log 2024-04-25T08:43:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93727.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99278.log.gz 2024-04-25T08:43:08.208 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82267.log 2024-04-25T08:43:08.209 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93727.log.gz 2024-04-25T08:43:08.209 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46060.log.gz 2024-04-25T08:43:08.209 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92836.log 2024-04-25T08:43:08.209 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42954.log: /var/log/ceph/ceph-client.admin.82267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97351.log 2024-04-25T08:43:08.209 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42954.log.gz 2024-04-25T08:43:08.209 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82015.log 2024-04-25T08:43:08.209 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82267.log.gz 2024-04-25T08:43:08.209 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92836.log.gz 2024-04-25T08:43:08.209 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66361.log 2024-04-25T08:43:08.209 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82015.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97351.log.gz 2024-04-25T08:43:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63529.log 2024-04-25T08:43:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66361.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.82015.log.gz -5 2024-04-25T08:43:08.210 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50263.log 2024-04-25T08:43:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63529.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66361.log.gz 2024-04-25T08:43:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59255.log 2024-04-25T08:43:08.210 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63529.log.gz 2024-04-25T08:43:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90900.log 2024-04-25T08:43:08.210 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5/var/log/ceph/ceph-client.admin.59255.log: --verbose -- /var/log/ceph/ceph-client.admin.68169.log 2024-04-25T08:43:08.210 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50263.log.gz 2024-04-25T08:43:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49513.log 2024-04-25T08:43:08.211 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59255.log.gz 2024-04-25T08:43:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47552.log 2024-04-25T08:43:08.211 INFO:teuthology.orchestra.run.smithi038.stderr: 91.2%/var/log/ceph/ceph-client.admin.49513.log: -- replaced with /var/log/ceph/ceph-client.admin.90900.log.gz 2024-04-25T08:43:08.211 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68169.log.gz 2024-04-25T08:43:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77391.log 2024-04-25T08:43:08.211 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49513.log.gz 2024-04-25T08:43:08.211 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101706.log 2024-04-25T08:43:08.212 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77738.log 2024-04-25T08:43:08.212 INFO:teuthology.orchestra.run.smithi038.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47552.log.gz 2024-04-25T08:43:08.212 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.101706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50572.log 2024-04-25T08:43:08.212 INFO:teuthology.orchestra.run.smithi038.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77391.log.gz 2024-04-25T08:43:08.212 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77738.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101706.log.gzgzip 2024-04-25T08:43:08.212 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40883.log 2024-04-25T08:43:08.212 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48697.log 2024-04-25T08:43:08.212 INFO:teuthology.orchestra.run.smithi038.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77738.log.gz 2024-04-25T08:43:08.212 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50572.log.gz 2024-04-25T08:43:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61787.log 2024-04-25T08:43:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43134.log 2024-04-25T08:43:08.213 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40883.log.gz 2024-04-25T08:43:08.213 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48697.log.gz 2024-04-25T08:43:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50656.log 2024-04-25T08:43:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43134.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.61787.log.gz 2024-04-25T08:43:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81339.log 2024-04-25T08:43:08.213 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94868.log 2024-04-25T08:43:08.213 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43134.log.gz 2024-04-25T08:43:08.214 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81339.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50656.log.gz 2024-04-25T08:43:08.214 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99320.log 2024-04-25T08:43:08.214 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81339.log.gz 2024-04-25T08:43:08.214 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.94868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68986.log 2024-04-25T08:43:08.214 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99320.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94868.log.gz 2024-04-25T08:43:08.214 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97751.log 2024-04-25T08:43:08.214 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99320.log.gz 2024-04-25T08:43:08.214 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102521.log 2024-04-25T08:43:08.214 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97751.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68986.log.gz 2024-04-25T08:43:08.215 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83387.log 2024-04-25T08:43:08.215 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97751.log.gz 2024-04-25T08:43:08.215 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60845.log 2024-04-25T08:43:08.215 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83387.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102521.log.gz 2024-04-25T08:43:08.215 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96554.log 2024-04-25T08:43:08.215 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60845.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.83387.log.gz 2024-04-25T08:43:08.215 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91092.log 2024-04-25T08:43:08.215 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.96554.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60845.log.gz 2024-04-25T08:43:08.215 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67325.log 2024-04-25T08:43:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87479.log 2024-04-25T08:43:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67325.log: gzip -5 --verbose -- 88.2% /var/log/ceph/ceph-client.admin.45530.log -- replaced with /var/log/ceph/ceph-client.admin.91092.log.gz 2024-04-25T08:43:08.216 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-04-25T08:43:08.216 INFO:teuthology.orchestra.run.smithi038.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.96554.log.gz 2024-04-25T08:43:08.216 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67325.log.gz 2024-04-25T08:43:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.87479.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49387.log 2024-04-25T08:43:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45530.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87479.log.gz 2024-04-25T08:43:08.216 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49177.log 2024-04-25T08:43:08.216 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45530.log.gz 2024-04-25T08:43:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46270.log 2024-04-25T08:43:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45408.log 2024-04-25T08:43:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49177.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49387.log.gz 2024-04-25T08:43:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92221.log 2024-04-25T08:43:08.217 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49177.log.gz 2024-04-25T08:43:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.2.45408.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46270.log.gz 2024-04-25T08:43:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59942.log 2024-04-25T08:43:08.217 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68762.log 2024-04-25T08:43:08.218 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92221.log.gz 2024-04-25T08:43:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40711.log 2024-04-25T08:43:08.218 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.2.45408.log.gz 2024-04-25T08:43:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68762.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59942.log.gz 2024-04-25T08:43:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96752.log 2024-04-25T08:43:08.218 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.68762.log.gz 2024-04-25T08:43:08.218 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98274.log 2024-04-25T08:43:08.218 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.40711.log.gz 2024-04-25T08:43:08.219 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58457.log 2024-04-25T08:43:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.96752.log: /var/log/ceph/ceph-client.admin.98274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47688.log 2024-04-25T08:43:08.219 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96752.log.gz 2024-04-25T08:43:08.219 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98274.log.gz 2024-04-25T08:43:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41011.log 2024-04-25T08:43:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47688.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58457.log.gz 2024-04-25T08:43:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98526.log 2024-04-25T08:43:08.219 INFO:teuthology.orchestra.run.smithi038.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47688.log.gz 2024-04-25T08:43:08.219 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.41011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59511.log 2024-04-25T08:43:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98526.log: gzip 89.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41011.log.gz -- 2024-04-25T08:43:08.220 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.46396.log 2024-04-25T08:43:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59511.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98526.log.gz 2024-04-25T08:43:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69198.log 2024-04-25T08:43:08.220 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59511.log.gz 2024-04-25T08:43:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64802.log 2024-04-25T08:43:08.220 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69198.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28638.log 2024-04-25T08:43:08.220 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46396.log.gz 2024-04-25T08:43:08.220 INFO:teuthology.orchestra.run.smithi038.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.69198.log.gz 2024-04-25T08:43:08.221 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78302.log 2024-04-25T08:43:08.221 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph.tmp-client.admin.28638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60218.log 2024-04-25T08:43:08.221 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64802.log.gz 2024-04-25T08:43:08.221 INFO:teuthology.orchestra.run.smithi038.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28638.log.gz 2024-04-25T08:43:08.221 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61721.log 2024-04-25T08:43:08.221 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78302.log.gz 2024-04-25T08:43:08.221 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.60218.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46312.log 2024-04-25T08:43:08.221 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43470.log 2024-04-25T08:43:08.222 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60218.log.gz 2024-04-25T08:43:08.222 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45892.log 2024-04-25T08:43:08.222 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46312.log.gz 2024-04-25T08:43:08.222 INFO:teuthology.orchestra.run.smithi038.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.61721.log.gz 2024-04-25T08:43:08.222 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40753.log 2024-04-25T08:43:08.222 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98087.log 87.9% 2024-04-25T08:43:08.222 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43470.log.gz 2024-04-25T08:43:08.222 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61655.log 2024-04-25T08:43:08.223 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45892.log.gz 2024-04-25T08:43:08.223 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98087.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40753.log.gz 2024-04-25T08:43:08.223 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96620.log 2024-04-25T08:43:08.223 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98087.log.gz 2024-04-25T08:43:08.223 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85837.log 2024-04-25T08:43:08.223 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61655.log: /var/log/ceph/ceph-client.admin.96620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82626.log 2024-04-25T08:43:08.223 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103739.log 2024-04-25T08:43:08.223 INFO:teuthology.orchestra.run.smithi038.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61655.log.gz 2024-04-25T08:43:08.223 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82626.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.96620.log.gz 2024-04-25T08:43:08.224 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85837.log.gz 2024-04-25T08:43:08.224 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47731.log 2024-04-25T08:43:08.224 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82626.log.gz 2024-04-25T08:43:08.224 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82584.log 2024-04-25T08:43:08.224 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59173.log 2024-04-25T08:43:08.224 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103739.log.gz 2024-04-25T08:43:08.224 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82584.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47731.log.gz 2024-04-25T08:43:08.224 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81618.log 2024-04-25T08:43:08.224 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82584.log.gz 2024-04-25T08:43:08.224 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99234.log 2024-04-25T08:43:08.225 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81912.log 2024-04-25T08:43:08.225 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59173.log.gz 2024-04-25T08:43:08.225 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.81618.log.gz 2024-04-25T08:43:08.225 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91902.log 2024-04-25T08:43:08.225 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99234.log.gz 2024-04-25T08:43:08.225 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86065.log 2024-04-25T08:43:08.225 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81912.log: /var/log/ceph/ceph-client.admin.91902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81534.log 2024-04-25T08:43:08.225 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.81912.log.gz 2024-04-25T08:43:08.225 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86065.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91902.log.gz -5 2024-04-25T08:43:08.226 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.101801.log 2024-04-25T08:43:08.226 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.86065.log.gz 2024-04-25T08:43:08.226 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98663.log 2024-04-25T08:43:08.226 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.101801.log: gzip -5 --verbose 88.1% -- /var/log/ceph/ceph-client.admin.58746.log -- replaced with /var/log/ceph/ceph-client.admin.81534.log.gz 2024-04-25T08:43:08.226 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-04-25T08:43:08.226 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93130.log 2024-04-25T08:43:08.226 INFO:teuthology.orchestra.run.smithi038.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.101801.log.gz 2024-04-25T08:43:08.226 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98663.log.gz 2024-04-25T08:43:08.226 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83261.log 2024-04-25T08:43:08.227 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58746.log.gz 2024-04-25T08:43:08.227 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66125.log 2024-04-25T08:43:08.227 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91368.log 2024-04-25T08:43:08.227 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93130.log.gz 2024-04-25T08:43:08.227 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83261.log.gz 2024-04-25T08:43:08.227 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42828.log 2024-04-25T08:43:08.227 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48529.log 2024-04-25T08:43:08.227 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66125.log.gz 2024-04-25T08:43:08.228 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63361.log 2024-04-25T08:43:08.228 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1%/var/log/ceph/ceph-client.admin.48529.log: -- replaced with /var/log/ceph/ceph-client.admin.42828.log.gz 2024-04-25T08:43:08.228 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose 91.1% -- -- replaced with /var/log/ceph/ceph-client.admin.91368.log.gz /var/log/ceph/ceph-client.admin.59297.log 2024-04-25T08:43:08.228 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-04-25T08:43:08.228 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63361.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48529.log.gz 2024-04-25T08:43:08.228 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77913.log 2024-04-25T08:43:08.228 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59297.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.63361.log.gz 2024-04-25T08:43:08.228 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103907.log 2024-04-25T08:43:08.228 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93046.log 2024-04-25T08:43:08.229 INFO:teuthology.orchestra.run.smithi038.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59297.log.gz 2024-04-25T08:43:08.229 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77913.log.gz 2024-04-25T08:43:08.229 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42146.log 2024-04-25T08:43:08.229 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93046.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103907.log.gz 2024-04-25T08:43:08.229 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T08:43:08.229 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93046.log.gz 2024-04-25T08:43:08.229 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81964.log 2024-04-25T08:43:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64665.log 2024-04-25T08:43:08.230 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42146.log.gz 2024-04-25T08:43:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48099.log 2024-04-25T08:43:08.230 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81964.log.gz 2024-04-25T08:43:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63124.log 2024-04-25T08:43:08.230 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48099.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64665.log.gz -5 2024-04-25T08:43:08.230 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.77654.log 2024-04-25T08:43:08.230 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48099.log.gz 2024-04-25T08:43:08.231 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69594.log 2024-04-25T08:43:08.231 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63124.log.gz/var/log/ceph/ceph-client.admin.77654.log: 2024-04-25T08:43:08.231 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43962.log 2024-04-25T08:43:08.231 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.77654.log.gz 2024-04-25T08:43:08.231 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92416.log 2024-04-25T08:43:08.231 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47137.log 2024-04-25T08:43:08.231 INFO:teuthology.orchestra.run.smithi038.stderr: 86.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T08:43:08.231 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92416.log: gzip 89.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.43962.log.gz 2024-04-25T08:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.69282.log 2024-04-25T08:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.69594.log.gz 2024-04-25T08:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92416.log.gz 2024-04-25T08:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40839.log 2024-04-25T08:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69282.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47137.log.gz 2024-04-25T08:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78430.log 2024-04-25T08:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69282.log.gz 2024-04-25T08:43:08.232 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93172.log 2024-04-25T08:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48613.log 2024-04-25T08:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40839.log.gz 2024-04-25T08:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61523.log 2024-04-25T08:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78430.log.gz 2024-04-25T08:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93172.log.gz 2024-04-25T08:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65815.log 2024-04-25T08:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61523.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48613.log.gzgzip 2024-04-25T08:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.89010.log 2024-04-25T08:43:08.233 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87395.log 2024-04-25T08:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65815.log.gz 2024-04-25T08:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.89010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46522.log 2024-04-25T08:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61523.log.gz 2024-04-25T08:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47179.log 2024-04-25T08:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46522.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87395.log.gz 2024-04-25T08:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77173.log 2024-04-25T08:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46522.log.gz 2024-04-25T08:43:08.234 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99132.log 2024-04-25T08:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77173.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.76684.log 2024-04-25T08:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47179.log.gz 2024-04-25T08:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77173.log.gz 2024-04-25T08:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99572.log 2024-04-25T08:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99132.log.gz/var/log/ceph/ceph-client.admin.76684.log: 2024-04-25T08:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103361.log 2024-04-25T08:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47425.log 2024-04-25T08:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.76684.log.gz 2024-04-25T08:43:08.235 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41095.log 2024-04-25T08:43:08.236 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47425.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.99572.log.gz 2024-04-25T08:43:08.236 INFO:teuthology.orchestra.run.smithi038.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103361.log.gz 2024-04-25T08:43:08.236 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58247.log 2024-04-25T08:43:08.236 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47425.log.gz 2024-04-25T08:43:08.236 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.41095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97919.log 2024-04-25T08:43:08.236 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41095.log.gz 2024-04-25T08:43:08.236 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86891.log 2024-04-25T08:43:08.236 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97919.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.58247.log.gz 2024-04-25T08:43:08.237 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93470.log 2024-04-25T08:43:08.237 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97919.log.gz 2024-04-25T08:43:08.237 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45572.log 2024-04-25T08:43:08.237 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.86891.log.gz 2024-04-25T08:43:08.237 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66359.log 2024-04-25T08:43:08.237 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48841.log 2024-04-25T08:43:08.237 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93470.log.gz 2024-04-25T08:43:08.237 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45572.log.gz 2024-04-25T08:43:08.237 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68337.log 2024-04-25T08:43:08.237 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66359.log.gz 2024-04-25T08:43:08.238 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44298.log 2024-04-25T08:43:08.238 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48841.log: /var/log/ceph/ceph-client.admin.68337.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T08:43:08.238 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48841.log.gz 2024-04-25T08:43:08.238 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44298.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68337.log.gz 2024-04-25T08:43:08.238 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81381.log 2024-04-25T08:43:08.238 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44298.log.gz 2024-04-25T08:43:08.238 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78472.log 2024-04-25T08:43:08.238 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68253.log 2024-04-25T08:43:08.238 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78472.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81381.log.gz 2024-04-25T08:43:08.239 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49969.log 2024-04-25T08:43:08.239 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78472.log.gz 2024-04-25T08:43:08.239 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86723.log 2024-04-25T08:43:08.239 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68253.log.gz 2024-04-25T08:43:08.239 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83429.log 2024-04-25T08:43:08.239 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49969.log: /var/log/ceph/ceph-client.admin.86723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60907.log 2024-04-25T08:43:08.239 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49969.log.gz 2024-04-25T08:43:08.239 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83429.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86723.log.gz 2024-04-25T08:43:08.240 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94200.log 2024-04-25T08:43:08.240 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83429.log.gz 2024-04-25T08:43:08.240 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61341.log 2024-04-25T08:43:08.240 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60907.log: /var/log/ceph/ceph-client.admin.94200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69114.log 2024-04-25T08:43:08.240 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61341.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94200.log.gz 2024-04-25T08:43:08.240 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76539.log 2024-04-25T08:43:08.240 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60907.log.gz 2024-04-25T08:43:08.240 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61341.log.gz 2024-04-25T08:43:08.241 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58289.log 2024-04-25T08:43:08.241 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69114.log: /var/log/ceph/ceph-client.admin.76539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46648.log 2024-04-25T08:43:08.241 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.69114.log.gz 2024-04-25T08:43:08.241 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47973.log 2024-04-25T08:43:08.241 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58289.log.gz 2024-04-25T08:43:08.241 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63487.log 2024-04-25T08:43:08.241 INFO:teuthology.orchestra.run.smithi038.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.76539.log.gz 2024-04-25T08:43:08.241 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58499.log 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47973.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46648.log.gz 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67688.log 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47973.log.gz 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4%/var/log/ceph/ceph-client.admin.58499.log: -- replaced with /var/log/ceph/ceph-client.admin.63487.log.gz 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49639.log 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58499.log.gz 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103403.log 2024-04-25T08:43:08.242 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67688.log: gzip/var/log/ceph/ceph-client.admin.49639.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.88601.log 2024-04-25T08:43:08.243 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67688.log.gz 2024-04-25T08:43:08.243 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103403.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49639.log.gz 2024-04-25T08:43:08.243 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68805.log 2024-04-25T08:43:08.243 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103403.log.gz 2024-04-25T08:43:08.243 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65507.log 2024-04-25T08:43:08.243 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88601.log: /var/log/ceph/ceph-client.admin.68805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93554.log 2024-04-25T08:43:08.243 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88601.log.gz 2024-04-25T08:43:08.243 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68805.log.gz 2024-04-25T08:43:08.244 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.65507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78514.log 2024-04-25T08:43:08.244 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62251.log 2024-04-25T08:43:08.244 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93554.log: /var/log/ceph/ceph-client.admin.78514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96794.log 2024-04-25T08:43:08.244 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93554.log.gz 2024-04-25T08:43:08.244 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78514.log.gz 2024-04-25T08:43:08.244 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94158.log 2024-04-25T08:43:08.244 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.96794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88415.log 2024-04-25T08:43:08.244 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8%/var/log/ceph/ceph-client.admin.94158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97309.log 2024-04-25T08:43:08.245 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96794.log.gz 91.2% -- replaced with /var/log/ceph/ceph-client.admin.62251.log.gz 2024-04-25T08:43:08.245 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-04-25T08:43:08.245 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85687.log 2024-04-25T08:43:08.245 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94158.log.gz 2024-04-25T08:43:08.245 INFO:teuthology.orchestra.run.smithi038.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.0.65507.log.gz 2024-04-25T08:43:08.245 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87696.log 2024-04-25T08:43:08.245 INFO:teuthology.orchestra.run.smithi038.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88415.log.gz 2024-04-25T08:43:08.245 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57499.log 2024-04-25T08:43:08.246 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97309.log.gz 2024-04-25T08:43:08.246 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62809.log 2024-04-25T08:43:08.246 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87696.log.gz 2024-04-25T08:43:08.246 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101664.log 2024-04-25T08:43:08.246 INFO:teuthology.orchestra.run.smithi038.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.85687.log.gz 2024-04-25T08:43:08.246 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87437.log 2024-04-25T08:43:08.246 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57499.log.gz 2024-04-25T08:43:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.101664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97877.log 2024-04-25T08:43:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87437.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.62809.log.gz 2024-04-25T08:43:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78251.log 2024-04-25T08:43:08.247 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101664.log.gz 2024-04-25T08:43:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97877.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87437.log.gz 2024-04-25T08:43:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98951.log 2024-04-25T08:43:08.247 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97877.log.gz 2024-04-25T08:43:08.247 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82057.log 2024-04-25T08:43:08.248 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78251.log.gz 2024-04-25T08:43:08.248 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41179.log 2024-04-25T08:43:08.248 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98951.log.gz 2024-04-25T08:43:08.248 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43794.log 2024-04-25T08:43:08.248 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82057.log: /var/log/ceph/ceph-client.admin.41179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82967.log 2024-04-25T08:43:08.248 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82057.log.gz 2024-04-25T08:43:08.248 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43794.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41179.log.gz 2024-04-25T08:43:08.248 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T08:43:08.248 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98171.log 2024-04-25T08:43:08.249 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43794.log.gz/var/log/ceph/ceph-client.admin.82967.log: 2024-04-25T08:43:08.249 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.82967.log.gz 2024-04-25T08:43:08.249 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph.audit.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.65056.log 2024-04-25T08:43:08.249 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81492.log 2024-04-25T08:43:08.249 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98171.log: /var/log/ceph/ceph-client.admin.65056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91650.log 2024-04-25T08:43:08.249 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98171.log.gz 2024-04-25T08:43:08.249 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65056.log.gz 2024-04-25T08:43:08.249 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59816.log 2024-04-25T08:43:08.249 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91650.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81492.log.gz 2024-04-25T08:43:08.250 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76389.log 2024-04-25T08:43:08.250 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91650.log.gz 2024-04-25T08:43:08.250 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90768.log 2024-04-25T08:43:08.250 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98993.log 2024-04-25T08:43:08.250 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76389.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59816.log.gz 2024-04-25T08:43:08.250 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47305.log 2024-04-25T08:43:08.250 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78079.log 90.8% -- replaced with /var/log/ceph/ceph-client.admin.76389.log.gz 2024-04-25T08:43:08.250 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-04-25T08:43:08.250 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47305.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98993.log.gz 2024-04-25T08:43:08.251 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.90768.log.gz 2024-04-25T08:43:08.251 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42104.log 2024-04-25T08:43:08.251 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47305.log.gz 2024-04-25T08:43:08.251 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88161.log 2024-04-25T08:43:08.251 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78079.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T08:43:08.251 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63831.log 2024-04-25T08:43:08.251 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78079.log.gz 2024-04-25T08:43:08.251 INFO:teuthology.orchestra.run.smithi038.stderr: 87.5%/var/log/ceph/ceph-client.admin.88161.log: -- replaced with /var/log/ceph/ceph-client.admin.42104.log.gz 2024-04-25T08:43:08.252 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92752.log 2024-04-25T08:43:08.252 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88161.log.gz 2024-04-25T08:43:08.252 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88643.log 2024-04-25T08:43:08.252 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63831.log: /var/log/ceph/ceph-client.admin.92752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67898.log 2024-04-25T08:43:08.252 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63831.log.gz 2024-04-25T08:43:08.252 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88643.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92752.log.gz 2024-04-25T08:43:08.252 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48141.log 2024-04-25T08:43:08.252 INFO:teuthology.orchestra.run.smithi038.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88643.log.gz 2024-04-25T08:43:08.253 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69660.log 2024-04-25T08:43:08.253 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67898.log: /var/log/ceph/ceph-client.admin.48141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85753.log 2024-04-25T08:43:08.253 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67898.log.gz 2024-04-25T08:43:08.253 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48141.log.gz 2024-04-25T08:43:08.253 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98568.log 2024-04-25T08:43:08.253 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69660.log: /var/log/ceph/ceph-client.admin.85753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59604.log 2024-04-25T08:43:08.253 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.69660.log.gz 2024-04-25T08:43:08.253 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98568.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85753.log.gz 2024-04-25T08:43:08.254 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50793.log 2024-04-25T08:43:08.254 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98568.log.gz 2024-04-25T08:43:08.254 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86612.log 2024-04-25T08:43:08.254 INFO:teuthology.orchestra.run.smithi038.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.59604.log.gz 2024-04-25T08:43:08.254 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45036.log 2024-04-25T08:43:08.254 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50793.log: /var/log/ceph/ceph-client.admin.86612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61101.log 2024-04-25T08:43:08.254 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50793.log.gz 2024-04-25T08:43:08.254 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45036.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.86612.log.gz 2024-04-25T08:43:08.255 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69552.log 2024-04-25T08:43:08.255 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82225.log 2024-04-25T08:43:08.255 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61101.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45036.log.gz 2024-04-25T08:43:08.255 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102605.log 2024-04-25T08:43:08.255 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61101.log.gz 2024-04-25T08:43:08.255 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69552.log.gz 2024-04-25T08:43:08.255 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59984.log 2024-04-25T08:43:08.255 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.94693.log 2024-04-25T08:43:08.256 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82225.log.gz 2024-04-25T08:43:08.256 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102605.log.gz 2024-04-25T08:43:08.256 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50444.log 2024-04-25T08:43:08.256 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48283.log 2024-04-25T08:43:08.256 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.94693.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59984.log.gz 2024-04-25T08:43:08.256 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76323.log 2024-04-25T08:43:08.256 INFO:teuthology.orchestra.run.smithi038.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50444.log.gz 2024-04-25T08:43:08.256 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92263.log 2024-04-25T08:43:08.256 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40795.log 2024-04-25T08:43:08.257 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48283.log.gz 2024-04-25T08:43:08.257 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76323.log: /var/log/ceph/ceph-client.admin.92263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86444.log 2024-04-25T08:43:08.257 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40795.log: gzip 87.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.92263.log.gz -- 2024-04-25T08:43:08.257 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.64760.log 2024-04-25T08:43:08.257 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.76323.log.gz 2024-04-25T08:43:08.257 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40443.log 2024-04-25T08:43:08.257 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40795.log.gz 2024-04-25T08:43:08.258 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.64760.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.86444.log.gz 2024-04-25T08:43:08.258 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.42020.log 2024-04-25T08:43:08.258 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40443.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.64760.log.gz 2024-04-25T08:43:08.258 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91302.log 2024-04-25T08:43:08.258 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40443.log.gz 2024-04-25T08:43:08.258 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102395.log 2024-04-25T08:43:08.258 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42020.log: /var/log/ceph/ceph-client.admin.91302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44340.log 2024-04-25T08:43:08.258 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42020.log.gz 2024-04-25T08:43:08.259 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49009.log 2024-04-25T08:43:08.259 INFO:teuthology.orchestra.run.smithi038.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.91302.log.gz 2024-04-25T08:43:08.259 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44340.log: 89.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40669.log 2024-04-25T08:43:08.259 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.102395.log.gz 2024-04-25T08:43:08.259 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49009.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.44340.log.gz 2024-04-25T08:43:08.259 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62119.log 2024-04-25T08:43:08.259 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.40669.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49009.log.gz 2024-04-25T08:43:08.259 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.67940.log 2024-04-25T08:43:08.259 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62119.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40669.log.gz 2024-04-25T08:43:08.260 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81255.log 2024-04-25T08:43:08.260 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57926.log 2024-04-25T08:43:08.260 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67940.log.gz 2024-04-25T08:43:08.260 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.62119.log.gz 2024-04-25T08:43:08.260 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67187.log 2024-04-25T08:43:08.260 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81255.log: /var/log/ceph/ceph-client.admin.57926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94434.log 2024-04-25T08:43:08.260 INFO:teuthology.orchestra.run.smithi038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.81255.log.gz 2024-04-25T08:43:08.261 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67187.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.57926.log.gz 2024-04-25T08:43:08.261 INFO:teuthology.orchestra.run.smithi038.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.82404.log 2024-04-25T08:43:08.261 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.94434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98484.log 2024-04-25T08:43:08.261 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43468.log 2024-04-25T08:43:08.261 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94434.log.gz 2024-04-25T08:43:08.261 INFO:teuthology.orchestra.run.smithi038.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.67187.log.gz 2024-04-25T08:43:08.261 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83345.log 2024-04-25T08:43:08.262 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.82404.log.gz 2024-04-25T08:43:08.262 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43468.log: gzip -5 --verbose -- 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98484.log.gz 2024-04-25T08:43:08.262 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.64536.log 2024-04-25T08:43:08.262 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83345.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43468.log.gz 2024-04-25T08:43:08.262 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T08:43:08.262 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83345.log.gz 2024-04-25T08:43:08.262 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46564.log 2024-04-25T08:43:08.262 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47899.log 2024-04-25T08:43:08.263 INFO:teuthology.orchestra.run.smithi038.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64536.log.gz 2024-04-25T08:43:08.263 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49051.log 2024-04-25T08:43:08.263 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43710.log 2024-04-25T08:43:08.263 INFO:teuthology.orchestra.run.smithi038.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46564.log.gz 2024-04-25T08:43:08.263 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.49051.log: -- /var/log/ceph/ceph-client.admin.34694.log 2024-04-25T08:43:08.263 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47899.log.gz 2024-04-25T08:43:08.263 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87616.log 2024-04-25T08:43:08.263 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49051.log.gz 2024-04-25T08:43:08.263 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43710.log: /var/log/ceph/ceph-client.admin.34694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103781.log 2024-04-25T08:43:08.264 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34694.log.gz 2024-04-25T08:43:08.264 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41221.log 2024-04-25T08:43:08.264 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43710.log.gz 2024-04-25T08:43:08.264 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50699.log 2024-04-25T08:43:08.264 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87616.log.gz/var/log/ceph/ceph-client.admin.103781.log: 2024-04-25T08:43:08.264 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.41221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62555.log 2024-04-25T08:43:08.264 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103781.log.gz 2024-04-25T08:43:08.264 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41221.log.gz 2024-04-25T08:43:08.265 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42870.log 2024-04-25T08:43:08.265 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50699.log: /var/log/ceph/ceph-client.admin.62555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2024-04-25T08:43:08.265 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57884.log 2024-04-25T08:43:08.265 INFO:teuthology.orchestra.run.smithi038.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.62555.log.gz 88.4% 2024-04-25T08:43:08.265 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50699.log.gz 2024-04-25T08:43:08.265 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58415.log 2024-04-25T08:43:08.265 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42870.log.gz 2024-04-25T08:43:08.265 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2024-04-25T08:43:08.265 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98129.log 2024-04-25T08:43:08.266 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57884.log: /var/log/ceph/ceph-client.admin.58415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66601.log 2024-04-25T08:43:08.266 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.58415.log.gz -5 2024-04-25T08:43:08.266 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose --/var/log/ceph/ceph-client.admin.98129.log: /var/log/ceph/ceph-client.0.67012.log 2024-04-25T08:43:08.266 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57884.log.gz 2024-04-25T08:43:08.266 INFO:teuthology.orchestra.run.smithi038.stderr:gzip/var/log/ceph/ceph-client.admin.66601.log: 88.3% -5 --verbose -- /var/log/ceph/ceph-client.admin.98223.log 2024-04-25T08:43:08.266 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98129.log.gz 2024-04-25T08:43:08.266 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.67012.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66601.log.gz 2024-04-25T08:43:08.266 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65773.log 2024-04-25T08:43:08.267 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.98223.log: -- /var/log/ceph/ceph-client.admin.48655.log 2024-04-25T08:43:08.267 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44528.log 2024-04-25T08:43:08.267 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98223.log.gz 2024-04-25T08:43:08.267 INFO:teuthology.orchestra.run.smithi038.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65773.log.gz 2024-04-25T08:43:08.267 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58163.log 2024-04-25T08:43:08.267 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48655.log: /var/log/ceph/ceph-client.admin.44528.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48655.log.gz 2024-04-25T08:43:08.267 INFO:teuthology.orchestra.run.smithi038.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.67012.log.gz 2024-04-25T08:43:08.267 INFO:teuthology.orchestra.run.smithi038.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44528.log.gz 2024-04-25T08:43:08.268 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66362.log 2024-04-25T08:43:08.268 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61991.log 2024-04-25T08:43:08.268 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58163.log: /var/log/ceph/ceph-client.admin.66362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47079.log 2024-04-25T08:43:08.268 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50744.log 2024-04-25T08:43:08.268 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66362.log.gz 2024-04-25T08:43:08.268 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58163.log.gz 2024-04-25T08:43:08.268 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47079.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61991.log.gz 2024-04-25T08:43:08.268 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77215.log 2024-04-25T08:43:08.269 INFO:teuthology.orchestra.run.smithi038.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47079.log.gz 2024-04-25T08:43:08.269 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60110.log 2024-04-25T08:43:08.269 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50744.log.gz 2024-04-25T08:43:08.269 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97150.log 2024-04-25T08:43:08.269 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77215.log: /var/log/ceph/ceph-client.admin.60110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94074.log 2024-04-25T08:43:08.269 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77341.log 2024-04-25T08:43:08.269 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60110.log.gz 2024-04-25T08:43:08.269 INFO:teuthology.orchestra.run.smithi038.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77215.log.gz 2024-04-25T08:43:08.270 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.94074.log: gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97150.log.gz 2024-04-25T08:43:08.270 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.103193.log 2024-04-25T08:43:08.270 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77341.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94074.log.gz 2024-04-25T08:43:08.270 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43878.log 2024-04-25T08:43:08.270 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77341.log.gz 2024-04-25T08:43:08.270 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57631.log 2024-04-25T08:43:08.270 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103193.log.gz 2024-04-25T08:43:08.271 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81297.log 2024-04-25T08:43:08.271 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43878.log.gz 2024-04-25T08:43:08.271 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85795.log 2024-04-25T08:43:08.271 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57631.log: /var/log/ceph/ceph-client.admin.81297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88203.log 2024-04-25T08:43:08.271 INFO:teuthology.orchestra.run.smithi038.stderr:gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81297.log.gz -5 2024-04-25T08:43:08.271 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.88751.log 2024-04-25T08:43:08.271 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85795.log: 91.0%gzip/var/log/ceph/ceph-client.admin.88203.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46102.log 2024-04-25T08:43:08.271 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57631.log.gz 2024-04-25T08:43:08.271 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88751.log: gzip 88.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.88203.log.gz -- 2024-04-25T08:43:08.272 INFO:teuthology.orchestra.run.smithi038.stderr: /var/log/ceph/ceph-client.admin.64010.log 2024-04-25T08:43:08.272 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85795.log.gz 2024-04-25T08:43:08.272 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46102.log: gzip 89.5% -5 --verbose -- /var/log/ceph/ceph-client.admin.88119.log 2024-04-25T08:43:08.272 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.88751.log.gz 2024-04-25T08:43:08.272 INFO:teuthology.orchestra.run.smithi038.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46102.log.gz 2024-04-25T08:43:08.272 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78682.log 2024-04-25T08:43:08.272 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46732.log 2024-04-25T08:43:08.272 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64010.log.gz 2024-04-25T08:43:08.273 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88119.log: /var/log/ceph/ceph-client.admin.78682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98909.log 2024-04-25T08:43:08.273 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88119.log.gz 2024-04-25T08:43:08.273 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78682.log.gz 2024-04-25T08:43:08.273 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34959.log 2024-04-25T08:43:08.273 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46732.log: /var/log/ceph/ceph-client.admin.98909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92137.log 2024-04-25T08:43:08.273 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.34959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61246.log 2024-04-25T08:43:08.273 INFO:teuthology.orchestra.run.smithi038.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98909.log.gz 2024-04-25T08:43:08.273 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46732.log.gz 2024-04-25T08:43:08.273 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69366.log 2024-04-25T08:43:08.274 INFO:teuthology.orchestra.run.smithi038.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34959.log.gz 2024-04-25T08:43:08.274 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80893.log 2024-04-25T08:43:08.274 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92137.log.gz 2024-04-25T08:43:08.274 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91050.log 2024-04-25T08:43:08.274 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61246.log.gz 2024-04-25T08:43:08.274 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69366.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.80893.log: /var/log/ceph/ceph-client.admin.57565.log 2024-04-25T08:43:08.274 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69366.log.gz 2024-04-25T08:43:08.274 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68127.log 2024-04-25T08:43:08.275 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91050.log: /var/log/ceph/ceph-client.admin.57565.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.80893.log.gz 2024-04-25T08:43:08.275 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91050.log.gz 2024-04-25T08:43:08.275 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T08:43:08.275 INFO:teuthology.orchestra.run.smithi038.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.57565.log.gz 2024-04-25T08:43:08.275 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66645.log 2024-04-25T08:43:08.275 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68127.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42702.log 2024-04-25T08:43:08.275 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66645.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.68127.log.gz 2024-04-25T08:43:08.275 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96836.log 2024-04-25T08:43:08.275 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46480.log 2024-04-25T08:43:08.276 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66645.log.gz 2024-04-25T08:43:08.276 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.96836.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42702.log.gz 2024-04-25T08:43:08.276 INFO:teuthology.orchestra.run.smithi038.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96836.log.gz 2024-04-25T08:43:08.276 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61589.log 2024-04-25T08:43:08.276 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42786.log 2024-04-25T08:43:08.276 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46480.log.gz 2024-04-25T08:43:08.276 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43462.log 2024-04-25T08:43:08.276 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97835.log 2024-04-25T08:43:08.277 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42786.log: 91.2%/var/log/ceph/ceph-client.admin.43462.log: -- replaced with /var/log/ceph/ceph-client.admin.61589.log.gz 2024-04-25T08:43:08.277 INFO:teuthology.orchestra.run.smithi038.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42786.log.gz 2024-04-25T08:43:08.277 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43462.log.gz 2024-04-25T08:43:08.277 INFO:teuthology.orchestra.run.smithi038.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61035.log 2024-04-25T08:43:08.277 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79049.log 2024-04-25T08:43:08.277 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2%/var/log/ceph/ceph-client.admin.61035.log: -- replaced with /var/log/ceph/ceph-client.admin.97835.log.gz 2024-04-25T08:43:08.277 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98867.log 2024-04-25T08:43:08.277 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.0.79049.log: -- /var/log/ceph/ceph-client.admin.86765.log 2024-04-25T08:43:08.278 INFO:teuthology.orchestra.run.smithi038.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61035.log.gz 2024-04-25T08:43:08.278 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92794.log 2024-04-25T08:43:08.278 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86765.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98867.log.gz 2024-04-25T08:43:08.278 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.86765.log.gz 2024-04-25T08:43:08.278 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.99897.log 2024-04-25T08:43:08.278 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92920.log 2024-04-25T08:43:08.278 INFO:teuthology.orchestra.run.smithi038.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.92794.log.gz 2024-04-25T08:43:08.278 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.99897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47347.log 2024-04-25T08:43:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92920.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.92920.log.gz 2024-04-25T08:43:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47857.log 2024-04-25T08:43:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47347.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47347.log.gz 2024-04-25T08:43:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43471.log 2024-04-25T08:43:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47857.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47857.log.gz 2024-04-25T08:43:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49841.log 2024-04-25T08:43:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43471.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43471.log.gz 2024-04-25T08:43:08.279 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82925.log 2024-04-25T08:43:08.280 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49841.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49841.log.gz 2024-04-25T08:43:08.280 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81091.log 2024-04-25T08:43:08.280 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82925.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82925.log.gz 2024-04-25T08:43:08.280 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92016.log 2024-04-25T08:43:08.280 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81091.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.81091.log.gz 2024-04-25T08:43:08.280 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76921.log 2024-04-25T08:43:08.280 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92016.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92016.log.gz 2024-04-25T08:43:08.280 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78640.log 2024-04-25T08:43:08.280 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76921.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.76921.log.gz 2024-04-25T08:43:08.281 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T08:43:08.281 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78640.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78640.log.gz 2024-04-25T08:43:08.281 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T08:43:08.281 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61895.log 2024-04-25T08:43:08.281 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35062.log 2024-04-25T08:43:08.282 INFO:teuthology.orchestra.run.smithi107.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T08:43:08.286 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61895.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61895.log.gz 2024-04-25T08:43:08.287 INFO:teuthology.orchestra.run.smithi038.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T08:43:08.287 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44046.log 2024-04-25T08:43:08.287 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.35062.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35062.log.gz 2024-04-25T08:43:08.287 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102268.log 2024-04-25T08:43:08.287 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44046.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44046.log.gz 2024-04-25T08:43:08.287 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49925.log 2024-04-25T08:43:08.287 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102268.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102268.log.gz 2024-04-25T08:43:08.287 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48925.log 2024-04-25T08:43:08.288 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49925.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49925.log.gz 2024-04-25T08:43:08.288 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91860.log 2024-04-25T08:43:08.288 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48925.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48925.log.gz 2024-04-25T08:43:08.288 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50402.log 2024-04-25T08:43:08.288 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91860.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.91860.log.gz 2024-04-25T08:43:08.293 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93004.log 2024-04-25T08:43:08.306 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50402.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50402.log.gz 2024-04-25T08:43:08.314 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86528.log 2024-04-25T08:43:08.315 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93004.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93004.log.gz 2024-04-25T08:43:08.327 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58331.log 2024-04-25T08:43:08.328 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86528.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86528.log.gz 2024-04-25T08:43:08.336 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42246.log 2024-04-25T08:43:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60026.log 2024-04-25T08:43:08.345 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42246.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42246.log.gz 2024-04-25T08:43:08.349 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58331.log.gz 2024-04-25T08:43:08.357 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44382.log 2024-04-25T08:43:08.357 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60026.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60026.log.gz 2024-04-25T08:43:08.366 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64371.log 2024-04-25T08:43:08.375 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43668.log 2024-04-25T08:43:08.379 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64371.log: 88.5% 90.8% -- replaced with /var/log/ceph/ceph-client.admin.44382.log.gz 2024-04-25T08:43:08.384 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43836.log 2024-04-25T08:43:08.388 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43668.log: -- replaced with /var/log/ceph/ceph-client.admin.64371.log.gz 2024-04-25T08:43:08.393 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T08:43:08.397 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43836.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43668.log.gz 2024-04-25T08:43:08.402 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67856.log 2024-04-25T08:43:08.405 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-osd.0.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43836.log.gz 2024-04-25T08:43:08.420 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91692.log 2024-04-25T08:43:08.420 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67856.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67856.log.gz 2024-04-25T08:43:08.429 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46869.log 2024-04-25T08:43:08.442 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91692.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.91692.log.gz 2024-04-25T08:43:08.446 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101988.log 2024-04-25T08:43:08.447 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46869.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46869.log.gz 2024-04-25T08:43:08.447 INFO:teuthology.orchestra.run.smithi107.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T08:43:08.455 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87311.log 2024-04-25T08:43:08.468 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.101988.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101988.log.gz 2024-04-25T08:43:08.470 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68678.log 2024-04-25T08:43:08.471 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87311.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.87311.log.gz 2024-04-25T08:43:08.480 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103025.log 2024-04-25T08:43:08.481 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68678.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68678.log.gz 2024-04-25T08:43:08.489 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66203.log 2024-04-25T08:43:08.498 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78344.log 2024-04-25T08:43:08.499 INFO:teuthology.orchestra.run.smithi107.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T08:43:08.502 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66203.log: 89.4% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103025.log.gz 2024-04-25T08:43:08.507 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93088.log 2024-04-25T08:43:08.507 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78344.log: -- replaced with /var/log/ceph/ceph-client.admin.66203.log.gz 2024-04-25T08:43:08.516 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59858.log 2024-04-25T08:43:08.517 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93088.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78344.log.gz 2024-04-25T08:43:08.525 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82710.log 2024-04-25T08:43:08.529 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59858.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93088.log.gz 2024-04-25T08:43:08.538 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.59858.log.gz 2024-04-25T08:43:08.543 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62508.log 2024-04-25T08:43:08.544 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82710.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82710.log.gz 2024-04-25T08:43:08.556 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44004.log 2024-04-25T08:43:08.557 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62508.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.62508.log.gz 2024-04-25T08:43:08.565 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44172.log 2024-04-25T08:43:08.575 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90834.log 2024-04-25T08:43:08.578 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44172.log: 88.7% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44004.log.gz 2024-04-25T08:43:08.583 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103445.log 2024-04-25T08:43:08.584 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90834.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.90834.log.gz 2024-04-25T08:43:08.587 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44172.log.gz 2024-04-25T08:43:08.592 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46953.log 2024-04-25T08:43:08.605 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103445.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103445.log.gz 2024-04-25T08:43:08.613 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102437.log 2024-04-25T08:43:08.614 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46953.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46953.log.gz 2024-04-25T08:43:08.622 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50614.log 2024-04-25T08:43:08.631 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102563.log 2024-04-25T08:43:08.635 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50614.log: 88.2% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102437.log.gz 2024-04-25T08:43:08.642 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77444.log 2024-04-25T08:43:08.642 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102563.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102563.log.gz 2024-04-25T08:43:08.644 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50614.log.gz 2024-04-25T08:43:08.658 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85879.log 2024-04-25T08:43:08.659 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77444.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77444.log.gz 2024-04-25T08:43:08.667 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69510.log 2024-04-25T08:43:08.676 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47263.log 2024-04-25T08:43:08.680 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69510.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85879.log.gz 2024-04-25T08:43:08.681 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69510.log.gz 2024-04-25T08:43:08.688 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85555.log 2024-04-25T08:43:08.689 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47263.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47263.log.gz 2024-04-25T08:43:08.697 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60068.log 2024-04-25T08:43:08.708 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77005.log 2024-04-25T08:43:08.711 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60068.log: 87.9% 91.4% -- replaced with /var/log/ceph/ceph-client.admin.85555.log.gz 2024-04-25T08:43:08.717 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57968.log 2024-04-25T08:43:08.721 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77005.log: -- replaced with /var/log/ceph/ceph-client.admin.60068.log.gz 2024-04-25T08:43:08.726 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102030.log 2024-04-25T08:43:08.729 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57968.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57968.log.gz 2024-04-25T08:43:08.730 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77005.log.gz 2024-04-25T08:43:08.740 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81576.log 2024-04-25T08:43:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102030.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102030.log.gz 2024-04-25T08:43:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93639.log 2024-04-25T08:43:08.752 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81576.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81576.log.gz 2024-04-25T08:43:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80827.log 2024-04-25T08:43:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93639.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93639.log.gz 2024-04-25T08:43:08.779 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43136.log 2024-04-25T08:43:08.792 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80827.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.80827.log.gz 2024-04-25T08:43:08.798 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42912.log 2024-04-25T08:43:08.799 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43136.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43136.log.gz 2024-04-25T08:43:08.812 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78037.log 2024-04-25T08:43:08.813 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42912.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42912.log.gz 2024-04-25T08:43:08.830 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48409.log 2024-04-25T08:43:08.831 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78037.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78037.log.gz 2024-04-25T08:43:08.839 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77047.log 2024-04-25T08:43:08.852 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67814.log 2024-04-25T08:43:08.852 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77047.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77047.log.gz 2024-04-25T08:43:08.852 INFO:teuthology.orchestra.run.smithi038.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48409.log.gz 2024-04-25T08:43:08.860 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97640.log 2024-04-25T08:43:08.869 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44214.log 2024-04-25T08:43:08.873 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97640.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67814.log.gz 2024-04-25T08:43:08.878 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92962.log 2024-04-25T08:43:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44214.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97640.log.gz 2024-04-25T08:43:08.887 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50011.log 2024-04-25T08:43:08.891 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92962.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44214.log.gz 2024-04-25T08:43:08.896 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97108.log 2024-04-25T08:43:08.900 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50011.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92962.log.gz 2024-04-25T08:43:08.905 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59381.log 2024-04-25T08:43:08.908 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97108.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.97108.log.gz 2024-04-25T08:43:08.909 INFO:teuthology.orchestra.run.smithi038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50011.log.gz 2024-04-25T08:43:08.922 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87017.log 2024-04-25T08:43:08.923 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59381.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59381.log.gz 2024-04-25T08:43:08.933 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91134.log 2024-04-25T08:43:08.934 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87017.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87017.log.gz 2024-04-25T08:43:08.942 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68720.log 2024-04-25T08:43:08.951 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81786.log 2024-04-25T08:43:08.955 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68720.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.91134.log.gz 2024-04-25T08:43:08.960 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58872.log 2024-04-25T08:43:08.962 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81786.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81786.log.gz 2024-04-25T08:43:08.964 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.68720.log.gz 2024-04-25T08:43:08.977 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85621.log 2024-04-25T08:43:08.978 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58872.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58872.log.gz 2024-04-25T08:43:08.986 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93906.log 2024-04-25T08:43:08.999 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85621.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.85621.log.gz 2024-04-25T08:43:09.006 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102226.log 2024-04-25T08:43:09.006 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93906.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93906.log.gz 2024-04-25T08:43:09.013 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80959.log 2024-04-25T08:43:09.026 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102226.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102226.log.gz 2024-04-25T08:43:09.028 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86323.log 2024-04-25T08:43:09.029 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.80959.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.80959.log.gz 2024-04-25T08:43:09.037 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51318.log 2024-04-25T08:43:09.050 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60779.log 2024-04-25T08:43:09.050 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.51318.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51318.log.gz 2024-04-25T08:43:09.050 INFO:teuthology.orchestra.run.smithi038.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86323.log.gz 2024-04-25T08:43:09.068 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83009.log 2024-04-25T08:43:09.069 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60779.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60779.log.gz 2024-04-25T08:43:09.078 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82840.log 2024-04-25T08:43:09.079 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83009.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83009.log.gz 2024-04-25T08:43:09.087 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86975.log 2024-04-25T08:43:09.096 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103067.log 2024-04-25T08:43:09.099 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86975.log: 88.0% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82840.log.gz 2024-04-25T08:43:09.108 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42744.log 2024-04-25T08:43:09.108 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103067.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103067.log.gz 2024-04-25T08:43:09.109 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86975.log.gz 2024-04-25T08:43:09.117 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77833.log 2024-04-25T08:43:09.119 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42744.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42744.log.gz 2024-04-25T08:43:09.135 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92626.log 2024-04-25T08:43:09.136 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77833.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77833.log.gz 2024-04-25T08:43:09.144 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48183.log 2024-04-25T08:43:09.157 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92626.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92626.log.gz 2024-04-25T08:43:09.163 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103823.log 2024-04-25T08:43:09.164 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48183.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48183.log.gz 2024-04-25T08:43:09.172 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76257.log 2024-04-25T08:43:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103823.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103823.log.gz 2024-04-25T08:43:09.181 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86807.log 2024-04-25T08:43:09.193 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87819.log 2024-04-25T08:43:09.193 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86807.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.86807.log.gz 2024-04-25T08:43:09.195 INFO:teuthology.orchestra.run.smithi038.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.76257.log.gz 2024-04-25T08:43:09.201 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44914.log 2024-04-25T08:43:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87819.log: 93.8% -- replaced with /var/log/ceph/ceph-client.0.89010.log.gz 2024-04-25T08:43:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77696.log 2024-04-25T08:43:09.214 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.44914.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87819.log.gz 2024-04-25T08:43:09.215 INFO:teuthology.orchestra.run.smithi038.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.0.44914.log.gz 2024-04-25T08:43:09.223 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92542.log 2024-04-25T08:43:09.232 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77696.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63781.log 2024-04-25T08:43:09.235 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92542.log: 88.0% 93.7% -- replaced with /var/log/ceph/ceph-client.0.94693.log.gz 2024-04-25T08:43:09.235 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.77696.log.gz 2024-04-25T08:43:09.236 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92542.log.gz 2024-04-25T08:43:09.238 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59562.log 2024-04-25T08:43:09.238 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63781.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63781.log.gz 2024-04-25T08:43:09.238 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62677.log 2024-04-25T08:43:09.239 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59562.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59562.log.gz 2024-04-25T08:43:09.239 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43130.log 2024-04-25T08:43:09.240 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62677.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.62677.log.gz 2024-04-25T08:43:09.241 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98400.log 2024-04-25T08:43:09.242 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43130.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43130.log.gz 2024-04-25T08:43:09.243 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93948.log 2024-04-25T08:43:09.244 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98400.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98400.log.gz 2024-04-25T08:43:09.244 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83219.log 2024-04-25T08:43:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93948.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93948.log.gz 2024-04-25T08:43:09.245 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49093.log 2024-04-25T08:43:09.247 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83219.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83219.log.gz 2024-04-25T08:43:09.247 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68935.log 2024-04-25T08:43:09.247 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49093.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49093.log.gz 2024-04-25T08:43:09.249 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103697.log 2024-04-25T08:43:09.250 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68935.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.68935.log.gz 2024-04-25T08:43:09.250 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99081.log 2024-04-25T08:43:09.251 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103697.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103697.log.gz 2024-04-25T08:43:09.252 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60152.log 2024-04-25T08:43:09.252 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99081.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99081.log.gz 2024-04-25T08:43:09.253 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81744.log 2024-04-25T08:43:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60152.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.60152.log.gz 2024-04-25T08:43:09.254 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48325.log 2024-04-25T08:43:09.257 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81744.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81744.log.gz 2024-04-25T08:43:09.257 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46995.log 2024-04-25T08:43:09.257 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48325.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48325.log.gz 2024-04-25T08:43:09.257 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91818.log 2024-04-25T08:43:09.260 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46995.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46995.log.gz 2024-04-25T08:43:09.260 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92458.log 2024-04-25T08:43:09.260 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91818.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91818.log.gz 2024-04-25T08:43:09.260 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49219.log 2024-04-25T08:43:09.262 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92458.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92458.log.gz 2024-04-25T08:43:09.263 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87143.log 2024-04-25T08:43:09.263 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49219.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49219.log.gz 2024-04-25T08:43:09.263 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63613.log 2024-04-25T08:43:09.265 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87143.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87143.log.gz 2024-04-25T08:43:09.265 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64709.log 2024-04-25T08:43:09.266 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63613.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.63613.log.gz 2024-04-25T08:43:09.266 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41137.log 2024-04-25T08:43:09.268 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64709.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64709.log.gz 2024-04-25T08:43:09.268 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92668.log 2024-04-25T08:43:09.268 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.41137.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41137.log.gz 2024-04-25T08:43:09.268 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48757.log 2024-04-25T08:43:09.271 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92668.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92668.log.gz 2024-04-25T08:43:09.271 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87353.log 2024-04-25T08:43:09.271 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48757.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48757.log.gz 2024-04-25T08:43:09.271 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92500.log 2024-04-25T08:43:09.274 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87353.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87353.log.gz 2024-04-25T08:43:09.274 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58052.log 2024-04-25T08:43:09.274 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92500.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92500.log.gz 2024-04-25T08:43:09.274 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41263.log 2024-04-25T08:43:09.277 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58052.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58052.log.gz 2024-04-25T08:43:09.277 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91476.log 2024-04-25T08:43:09.277 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.41263.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41263.log.gz 2024-04-25T08:43:09.277 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40967.log 2024-04-25T08:43:09.279 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91476.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.91476.log.gz 2024-04-25T08:43:09.280 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T08:43:09.280 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40967.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40967.log.gz 2024-04-25T08:43:09.280 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59093.log 2024-04-25T08:43:09.285 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-mon.a.log: 93.8% -- replaced with /var/log/ceph/ceph-client.0.99897.log.gz 2024-04-25T08:43:09.288 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61420.log 2024-04-25T08:43:09.288 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59093.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.59093.log.gz 2024-04-25T08:43:09.288 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92710.log 2024-04-25T08:43:09.290 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61420.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61420.log.gz 2024-04-25T08:43:09.290 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63926.log 2024-04-25T08:43:09.290 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92710.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92710.log.gz 2024-04-25T08:43:09.294 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81702.log 2024-04-25T08:43:09.294 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63926.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63926.log.gz 2024-04-25T08:43:09.294 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58914.log 2024-04-25T08:43:09.294 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81702.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.81702.log.gz 2024-04-25T08:43:09.294 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63445.log 2024-04-25T08:43:09.295 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58914.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.58914.log.gz 2024-04-25T08:43:09.295 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47037.log 2024-04-25T08:43:09.298 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63445.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63445.log.gz 2024-04-25T08:43:09.298 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66539.log 2024-04-25T08:43:09.298 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47037.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47037.log.gz 2024-04-25T08:43:09.299 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49883.log 2024-04-25T08:43:09.299 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66539.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.66539.log.gz 2024-04-25T08:43:09.299 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76768.log 2024-04-25T08:43:09.301 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49883.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49883.log.gz 2024-04-25T08:43:09.301 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48883.log 2024-04-25T08:43:09.302 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76768.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.76768.log.gz 2024-04-25T08:43:09.302 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43128.log 2024-04-25T08:43:09.302 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48883.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48883.log.gz 2024-04-25T08:43:09.305 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68847.log 2024-04-25T08:43:09.305 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43128.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43128.log.gz 2024-04-25T08:43:09.305 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46774.log 2024-04-25T08:43:09.305 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68847.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68847.log.gz 2024-04-25T08:43:09.306 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47467.log 2024-04-25T08:43:09.309 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46774.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46774.log.gz 2024-04-25T08:43:09.309 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87990.log 2024-04-25T08:43:09.309 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47467.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47467.log.gz 2024-04-25T08:43:09.309 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103109.log 2024-04-25T08:43:09.310 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87990.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87990.log.gz 2024-04-25T08:43:09.310 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102871.log 2024-04-25T08:43:09.312 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103109.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103109.log.gz 2024-04-25T08:43:09.313 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91218.log 2024-04-25T08:43:09.313 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102871.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102871.log.gz 2024-04-25T08:43:09.313 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43135.log 2024-04-25T08:43:09.313 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91218.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91218.log.gz 2024-04-25T08:43:09.316 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77612.log 2024-04-25T08:43:09.316 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43135.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43135.log.gz 2024-04-25T08:43:09.316 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69028.log 2024-04-25T08:43:09.316 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77612.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77612.log.gz 2024-04-25T08:43:09.317 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58373.log 2024-04-25T08:43:09.319 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69028.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.69028.log.gz 2024-04-25T08:43:09.319 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49261.log 2024-04-25T08:43:09.319 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58373.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.58373.log.gz 2024-04-25T08:43:09.319 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65140.log 2024-04-25T08:43:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49261.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49261.log.gz 2024-04-25T08:43:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65899.log 2024-04-25T08:43:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65140.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65140.log.gz 2024-04-25T08:43:09.322 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76963.log 2024-04-25T08:43:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65899.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65899.log.gz 2024-04-25T08:43:09.324 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88246.log 2024-04-25T08:43:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76963.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.76963.log.gz 2024-04-25T08:43:09.325 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102479.log 2024-04-25T08:43:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88246.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88246.log.gz 2024-04-25T08:43:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64621.log 2024-04-25T08:43:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102479.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102479.log.gz 2024-04-25T08:43:09.327 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48241.log 2024-04-25T08:43:09.329 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64621.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64621.log.gz 2024-04-25T08:43:09.329 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86486.log 2024-04-25T08:43:09.329 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48241.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48241.log.gz 2024-04-25T08:43:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64154.log 2024-04-25T08:43:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86486.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86486.log.gz 2024-04-25T08:43:09.331 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93309.log 2024-04-25T08:43:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64154.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64154.log.gz 2024-04-25T08:43:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76473.log 2024-04-25T08:43:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93309.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93309.log.gz 2024-04-25T08:43:09.334 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99446.log 2024-04-25T08:43:09.335 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76473.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.76473.log.gz 2024-04-25T08:43:09.336 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66041.log 2024-04-25T08:43:09.337 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99446.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99446.log.gz 2024-04-25T08:43:09.337 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47221.log 2024-04-25T08:43:09.338 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66041.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66041.log.gz 2024-04-25T08:43:09.339 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99638.log 2024-04-25T08:43:09.340 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47221.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47221.log.gz 2024-04-25T08:43:09.340 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42576.log 2024-04-25T08:43:09.340 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99638.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.99638.log.gz 2024-04-25T08:43:09.342 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97004.log 2024-04-25T08:43:09.343 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42576.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42576.log.gz 2024-04-25T08:43:09.343 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47773.log 2024-04-25T08:43:09.345 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97004.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97004.log.gz 2024-04-25T08:43:09.345 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61829.log 2024-04-25T08:43:09.345 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47773.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47773.log.gz 2024-04-25T08:43:09.346 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59423.log 2024-04-25T08:43:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61829.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61829.log.gz 2024-04-25T08:43:09.347 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92584.log 2024-04-25T08:43:09.348 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59423.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59423.log.gz 2024-04-25T08:43:09.349 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46690.log 2024-04-25T08:43:09.350 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92584.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92584.log.gz 2024-04-25T08:43:09.350 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62875.log 2024-04-25T08:43:09.351 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46690.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46690.log.gz 2024-04-25T08:43:09.351 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94284.log 2024-04-25T08:43:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62875.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62875.log.gz 2024-04-25T08:43:09.353 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83796.log 2024-04-25T08:43:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.94284.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94284.log.gz 2024-04-25T08:43:09.354 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69324.log 2024-04-25T08:43:09.378 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.83796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99530.log 2024-04-25T08:43:09.379 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69324.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.69324.log.gz 2024-04-25T08:43:09.397 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103947.log 2024-04-25T08:43:09.397 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99530.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99530.log.gz 2024-04-25T08:43:09.406 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66083.log 2024-04-25T08:43:09.406 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103947.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103947.log.gz 2024-04-25T08:43:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45850.log 2024-04-25T08:43:09.427 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66083.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66083.log.gz 2024-04-25T08:43:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103487.log 2024-04-25T08:43:09.436 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45850.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45850.log.gz 2024-04-25T08:43:09.445 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101759.log 2024-04-25T08:43:09.445 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103487.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103487.log.gz 2024-04-25T08:43:09.458 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82883.log 2024-04-25T08:43:09.459 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.101759.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101759.log.gz 2024-04-25T08:43:09.470 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78724.log 2024-04-25T08:43:09.471 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82883.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82883.log.gz 2024-04-25T08:43:09.488 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86570.log 2024-04-25T08:43:09.489 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78724.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.78724.log.gz 2024-04-25T08:43:09.501 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64972.log 2024-04-25T08:43:09.501 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86570.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.86570.log.gz 2024-04-25T08:43:09.509 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87861.log 2024-04-25T08:43:09.518 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64972.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46354.log 2024-04-25T08:43:09.518 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64972.log.gz 2024-04-25T08:43:09.527 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67772.log 2024-04-25T08:43:09.531 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46354.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87861.log.gz 2024-04-25T08:43:09.538 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46354.log.gz 2024-04-25T08:43:09.540 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48367.log 2024-04-25T08:43:09.541 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67772.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67772.log.gz 2024-04-25T08:43:09.549 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68379.log 2024-04-25T08:43:09.551 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48367.log: 93.8% -- replaced with /var/log/ceph/ceph-client.0.79049.log.gz 2024-04-25T08:43:09.553 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48367.log.gz 2024-04-25T08:43:09.553 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68596.log 2024-04-25T08:43:09.555 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68379.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.68379.log.gz 2024-04-25T08:43:09.555 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42188.log 2024-04-25T08:43:09.555 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68596.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.68596.log.gz 2024-04-25T08:43:09.555 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43133.log 2024-04-25T08:43:09.556 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42188.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42188.log.gz 2024-04-25T08:43:09.557 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63208.log 2024-04-25T08:43:09.558 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43133.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43133.log.gz 2024-04-25T08:43:09.558 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43464.log 2024-04-25T08:43:09.558 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63208.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63208.log.gz 2024-04-25T08:43:09.560 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47594.log 2024-04-25T08:43:09.560 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43464.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43464.log.gz 2024-04-25T08:43:09.561 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97598.log 2024-04-25T08:43:09.562 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47594.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47594.log.gz 2024-04-25T08:43:09.562 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59732.log 2024-04-25T08:43:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97598.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97598.log.gz 2024-04-25T08:43:09.563 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59774.log 2024-04-25T08:43:09.565 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59732.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.59732.log.gz 2024-04-25T08:43:09.565 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60717.log 2024-04-25T08:43:09.565 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59774.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.59774.log.gz 2024-04-25T08:43:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59467.log 2024-04-25T08:43:09.567 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60717.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60717.log.gz 2024-04-25T08:43:09.568 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69240.log 2024-04-25T08:43:09.569 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59467.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59467.log.gz 2024-04-25T08:43:09.569 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91434.log 2024-04-25T08:43:09.569 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69240.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69240.log.gz 2024-04-25T08:43:09.571 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86107.log 2024-04-25T08:43:09.571 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91434.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91434.log.gz 2024-04-25T08:43:09.572 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94326.log 2024-04-25T08:43:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86107.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.86107.log.gz 2024-04-25T08:43:09.573 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48571.log 2024-04-25T08:43:09.575 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.94326.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94326.log.gz 2024-04-25T08:43:09.575 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93512.log 2024-04-25T08:43:09.576 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48571.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48571.log.gz 2024-04-25T08:43:09.576 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64579.log 2024-04-25T08:43:09.577 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93512.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93512.log.gz 2024-04-25T08:43:09.577 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43132.log 2024-04-25T08:43:09.579 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64579.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64579.log.gz 2024-04-25T08:43:09.579 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48015.log 2024-04-25T08:43:09.580 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43132.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43132.log.gz 2024-04-25T08:43:09.580 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94242.log 2024-04-25T08:43:09.582 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48015.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48015.log.gz 2024-04-25T08:43:09.582 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45808.log 2024-04-25T08:43:09.583 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.94242.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94242.log.gz 2024-04-25T08:43:09.584 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103989.log 2024-04-25T08:43:09.584 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45808.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45808.log.gz 2024-04-25T08:43:09.586 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42660.log 2024-04-25T08:43:09.586 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103989.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103989.log.gz 2024-04-25T08:43:09.586 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62334.log 2024-04-25T08:43:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42660.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42660.log.gz 2024-04-25T08:43:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91584.log 2024-04-25T08:43:09.588 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62334.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62334.log.gz 2024-04-25T08:43:09.589 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62983.log 2024-04-25T08:43:09.589 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91584.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.91584.log.gz 2024-04-25T08:43:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103865.log 2024-04-25T08:43:09.591 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62983.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.62983.log.gz 2024-04-25T08:43:09.592 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82484.log 2024-04-25T08:43:09.594 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103865.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.103865.log.gz 2024-04-25T08:43:09.594 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50530.log 2024-04-25T08:43:09.594 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82484.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82484.log.gz 2024-04-25T08:43:09.594 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93778.log 2024-04-25T08:43:09.596 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50530.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50530.log.gz 2024-04-25T08:43:09.596 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96962.log 2024-04-25T08:43:09.596 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93778.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93778.log.gz 2024-04-25T08:43:09.598 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50137.log 2024-04-25T08:43:09.598 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.96962.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96962.log.gz 2024-04-25T08:43:09.598 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96686.log 2024-04-25T08:43:09.600 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50137.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50137.log.gz 2024-04-25T08:43:09.600 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63571.log 2024-04-25T08:43:09.601 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.96686.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.96686.log.gz 2024-04-25T08:43:09.601 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87185.log 2024-04-25T08:43:09.603 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63571.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63571.log.gz 2024-04-25T08:43:09.603 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90966.log 2024-04-25T08:43:09.604 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87185.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87185.log.gz 2024-04-25T08:43:09.604 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58010.log 2024-04-25T08:43:09.605 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.90966.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90966.log.gz 2024-04-25T08:43:09.606 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68891.log 2024-04-25T08:43:09.607 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58010.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.58010.log.gz 2024-04-25T08:43:09.607 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61467.log 2024-04-25T08:43:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68891.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.68891.log.gz 2024-04-25T08:43:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62743.log 2024-04-25T08:43:09.609 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61467.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.61467.log.gz 2024-04-25T08:43:09.610 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86849.log 2024-04-25T08:43:09.611 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62743.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62743.log.gz 2024-04-25T08:43:09.611 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78790.log 2024-04-25T08:43:09.613 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86849.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.86849.log.gz 2024-04-25T08:43:09.613 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81025.log 2024-04-25T08:43:09.614 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78790.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78790.log.gz 2024-04-25T08:43:09.614 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49555.log 2024-04-25T08:43:09.616 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81025.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.81025.log.gz 2024-04-25T08:43:09.616 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44256.log 2024-04-25T08:43:09.617 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49555.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49555.log.gz 2024-04-25T08:43:09.617 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61163.log 2024-04-25T08:43:09.619 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44256.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44256.log.gz 2024-04-25T08:43:09.619 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69070.log 2024-04-25T08:43:09.620 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.61163.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.61163.log.gz 2024-04-25T08:43:09.620 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63655.log 2024-04-25T08:43:09.622 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69070.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.69070.log.gz 2024-04-25T08:43:09.622 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103151.log 2024-04-25T08:43:09.623 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63655.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63655.log.gz 2024-04-25T08:43:09.623 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88035.log 2024-04-25T08:43:09.624 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103151.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103151.log.gz 2024-04-25T08:43:09.625 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78207.log 2024-04-25T08:43:09.627 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88035.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88035.log.gz 2024-04-25T08:43:09.627 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42414.log 2024-04-25T08:43:09.628 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78207.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78207.log.gz 2024-04-25T08:43:09.628 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66687.log 2024-04-25T08:43:09.630 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42414.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42414.log.gz 2024-04-25T08:43:09.630 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69919.log 2024-04-25T08:43:09.631 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66687.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.66687.log.gz 2024-04-25T08:43:09.631 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78121.log 2024-04-25T08:43:09.644 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.69919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68024.log 2024-04-25T08:43:09.645 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78121.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78121.log.gz 2024-04-25T08:43:09.658 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81828.log 2024-04-25T08:43:09.659 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68024.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.68024.log.gz 2024-04-25T08:43:09.670 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58583.log 2024-04-25T08:43:09.670 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81828.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.81828.log.gz 2024-04-25T08:43:09.679 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94368.log 2024-04-25T08:43:09.679 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58583.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58583.log.gz 2024-04-25T08:43:09.694 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63697.log 2024-04-25T08:43:09.695 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.94368.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.94368.log.gz 2024-04-25T08:43:09.703 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86365.log 2024-04-25T08:43:09.704 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63697.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63697.log.gz 2024-04-25T08:43:09.712 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82796.log 2024-04-25T08:43:09.713 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86365.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.86365.log.gz 2024-04-25T08:43:09.732 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81660.log 2024-04-25T08:43:09.733 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82796.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82796.log.gz 2024-04-25T08:43:09.741 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103277.log 2024-04-25T08:43:09.741 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81660.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.81660.log.gz 2024-04-25T08:43:09.756 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41371.log 2024-04-25T08:43:09.757 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103277.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103277.log.gz 2024-04-25T08:43:09.766 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99037.log 2024-04-25T08:43:09.767 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.41371.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41371.log.gz 2024-04-25T08:43:09.782 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63968.log 2024-04-25T08:43:09.782 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99037.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99037.log.gz 2024-04-25T08:43:09.790 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101904.log 2024-04-25T08:43:09.791 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63968.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63968.log.gz 2024-04-25T08:43:09.806 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50902.log 2024-04-25T08:43:09.807 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.101904.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101904.log.gz 2024-04-25T08:43:09.808 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77528.log 2024-04-25T08:43:09.808 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50902.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50902.log.gz 2024-04-25T08:43:09.823 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42372.log 2024-04-25T08:43:09.824 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77528.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77528.log.gz 2024-04-25T08:43:09.837 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65957.log 2024-04-25T08:43:09.838 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42372.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42372.log.gz 2024-04-25T08:43:09.846 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40524.log 2024-04-25T08:43:09.847 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65957.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65957.log.gz 2024-04-25T08:43:09.866 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46228.log 2024-04-25T08:43:09.867 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40524.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40524.log.gz 2024-04-25T08:43:09.875 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42618.log 2024-04-25T08:43:09.876 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46228.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46228.log.gz 2024-04-25T08:43:09.891 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83303.log 2024-04-25T08:43:09.892 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42618.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42618.log.gz 2024-04-25T08:43:09.901 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50053.log 2024-04-25T08:43:09.901 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83303.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83303.log.gz 2024-04-25T08:43:09.913 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82668.log 2024-04-25T08:43:09.914 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50053.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50053.log.gz 2024-04-25T08:43:09.925 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66246.log 2024-04-25T08:43:09.925 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82668.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82668.log.gz 2024-04-25T08:43:09.934 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97514.log 2024-04-25T08:43:09.946 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43469.log 2024-04-25T08:43:09.946 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97514.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97514.log.gz 2024-04-25T08:43:09.947 INFO:teuthology.orchestra.run.smithi038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.66246.log.gz 2024-04-25T08:43:09.954 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41053.log 2024-04-25T08:43:09.963 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92179.log 2024-04-25T08:43:09.967 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.41053.log: 88.7% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43469.log.gz 2024-04-25T08:43:09.972 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92058.log 2024-04-25T08:43:09.976 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92179.log: -- replaced with /var/log/ceph/ceph-client.admin.41053.log.gz 2024-04-25T08:43:09.985 INFO:teuthology.orchestra.run.smithi038.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92179.log.gz 2024-04-25T08:43:09.990 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48799.log 2024-04-25T08:43:09.991 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92058.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92058.log.gz 2024-04-25T08:43:09.999 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40566.log 2024-04-25T08:43:10.009 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86281.log 2024-04-25T08:43:10.010 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40566.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40566.log.gz 2024-04-25T08:43:10.012 INFO:teuthology.orchestra.run.smithi038.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48799.log.gz 2024-04-25T08:43:10.018 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88559.log 2024-04-25T08:43:10.027 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77131.log 2024-04-25T08:43:10.031 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88559.log: 89.0% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.86281.log.gz 2024-04-25T08:43:10.036 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86173.log 2024-04-25T08:43:10.040 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77131.log: -- replaced with /var/log/ceph/ceph-client.admin.88559.log.gz 2024-04-25T08:43:10.045 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97961.log 2024-04-25T08:43:10.049 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86173.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.77131.log.gz 2024-04-25T08:43:10.054 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49681.log 2024-04-25T08:43:10.056 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97961.log: 88.3% 90.9% -- replaced with /var/log/ceph/ceph-client.admin.86173.log.gz 2024-04-25T08:43:10.069 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97961.log.gz 2024-04-25T08:43:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64112.log 2024-04-25T08:43:10.069 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49681.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49681.log.gz 2024-04-25T08:43:10.085 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99488.log 2024-04-25T08:43:10.086 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64112.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64112.log.gz 2024-04-25T08:43:10.094 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62185.log 2024-04-25T08:43:10.107 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99488.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99488.log.gz 2024-04-25T08:43:10.115 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69156.log 2024-04-25T08:43:10.116 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62185.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62185.log.gz 2024-04-25T08:43:10.129 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81870.log 2024-04-25T08:43:10.130 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69156.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69156.log.gz 2024-04-25T08:43:10.138 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103613.log 2024-04-25T08:43:10.147 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58956.log 2024-04-25T08:43:10.151 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103613.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.81870.log.gz 2024-04-25T08:43:10.156 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87059.log 2024-04-25T08:43:10.160 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58956.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103613.log.gz 2024-04-25T08:43:10.165 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67367.log 2024-04-25T08:43:10.169 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87059.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58956.log.gz 2024-04-25T08:43:10.174 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86023.log 2024-04-25T08:43:10.178 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67367.log: 87.7% 89.5% -- replaced with /var/log/ceph/ceph-client.admin.87059.log.gz 2024-04-25T08:43:10.183 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83471.log 2024-04-25T08:43:10.187 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86023.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67367.log.gz 2024-04-25T08:43:10.192 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82752.log 2024-04-25T08:43:10.196 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83471.log: -- replaced with /var/log/ceph/ceph-client.admin.86023.log.gz 2024-04-25T08:43:10.201 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87946.log 2024-04-25T08:43:10.205 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82752.log: 88.6% 90.8% -- replaced with /var/log/ceph/ceph-client.admin.83471.log.gz 2024-04-25T08:43:10.210 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91008.log 2024-04-25T08:43:10.215 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87946.log: -- replaced with /var/log/ceph/ceph-client.admin.82752.log.gz 2024-04-25T08:43:10.222 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48057.log 2024-04-25T08:43:10.223 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91008.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91008.log.gz 2024-04-25T08:43:10.223 INFO:teuthology.orchestra.run.smithi038.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87946.log.gz 2024-04-25T08:43:10.235 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87269.log 2024-04-25T08:43:10.235 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48057.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48057.log.gz 2024-04-25T08:43:10.248 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50307.log 2024-04-25T08:43:10.249 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87269.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87269.log.gz 2024-04-25T08:43:10.257 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82141.log 2024-04-25T08:43:10.259 INFO:teuthology.orchestra.run.smithi107.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T08:43:10.266 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65182.log 2024-04-25T08:43:10.270 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82141.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50307.log.gz 2024-04-25T08:43:10.277 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64930.log 2024-04-25T08:43:10.278 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65182.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.65182.log.gz 2024-04-25T08:43:10.279 INFO:teuthology.orchestra.run.smithi038.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82141.log.gz 2024-04-25T08:43:10.286 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62611.log 2024-04-25T08:43:10.291 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64930.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64930.log.gz 2024-04-25T08:43:10.297 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77299.log 2024-04-25T08:43:10.298 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62611.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.62611.log.gz 2024-04-25T08:43:10.306 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94032.log 2024-04-25T08:43:10.315 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65014.log 2024-04-25T08:43:10.319 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.94032.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.77299.log.gz 2024-04-25T08:43:10.324 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77486.log 2024-04-25T08:43:10.328 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65014.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94032.log.gz 2024-04-25T08:43:10.335 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68295.log 2024-04-25T08:43:10.336 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77486.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.77486.log.gz 2024-04-25T08:43:10.337 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65014.log.gz 2024-04-25T08:43:10.347 INFO:teuthology.orchestra.run.smithi038.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T08:43:10.348 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98825.log 2024-04-25T08:43:10.349 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68295.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68295.log.gz 2024-04-25T08:43:10.351 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88077.log 2024-04-25T08:43:10.351 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98825.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98825.log.gz 2024-04-25T08:43:10.353 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77089.log 2024-04-25T08:43:10.353 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88077.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88077.log.gz 2024-04-25T08:43:10.353 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102727.log 2024-04-25T08:43:10.354 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77089.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.77089.log.gz 2024-04-25T08:43:10.355 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40609.log 2024-04-25T08:43:10.356 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102727.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102727.log.gz 2024-04-25T08:43:10.356 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99362.log 2024-04-25T08:43:10.357 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40609.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40609.log.gz 2024-04-25T08:43:10.357 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65731.log 2024-04-25T08:43:10.359 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99362.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99362.log.gz 2024-04-25T08:43:10.359 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43920.log 2024-04-25T08:43:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65731.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65731.log.gz 2024-04-25T08:43:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42456.log 2024-04-25T08:43:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43920.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43920.log.gz 2024-04-25T08:43:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69408.log 2024-04-25T08:43:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42456.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42456.log.gz 2024-04-25T08:43:10.364 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96920.log 2024-04-25T08:43:10.367 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69408.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69408.log.gz 2024-04-25T08:43:10.367 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40925.log 2024-04-25T08:43:10.367 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.96920.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96920.log.gz 2024-04-25T08:43:10.368 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93597.log 2024-04-25T08:43:10.368 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40925.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40925.log.gz 2024-04-25T08:43:10.368 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60973.log 2024-04-25T08:43:10.371 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93597.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93597.log.gz 2024-04-25T08:43:10.372 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97472.log 2024-04-25T08:43:10.372 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.60973.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60973.log.gz 2024-04-25T08:43:10.372 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103655.log 2024-04-25T08:43:10.372 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97472.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97472.log.gz 2024-04-25T08:43:10.372 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103571.log 2024-04-25T08:43:10.375 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103655.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103655.log.gz 2024-04-25T08:43:10.375 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82183.log 2024-04-25T08:43:10.376 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103571.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103571.log.gz 2024-04-25T08:43:10.376 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93820.log 2024-04-25T08:43:10.376 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82183.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.82183.log.gz 2024-04-25T08:43:10.376 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68421.log 2024-04-25T08:43:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93820.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93820.log.gz 2024-04-25T08:43:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47815.log 2024-04-25T08:43:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68421.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68421.log.gz 2024-04-25T08:43:10.379 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45934.log 2024-04-25T08:43:10.380 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47815.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47815.log.gz 2024-04-25T08:43:10.382 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65999.log 2024-04-25T08:43:10.382 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45934.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45934.log.gz 2024-04-25T08:43:10.382 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67646.log 2024-04-25T08:43:10.382 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65999.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65999.log.gz 2024-04-25T08:43:10.384 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93214.log 2024-04-25T08:43:10.384 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67646.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.67646.log.gz 2024-04-25T08:43:10.384 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98003.log 2024-04-25T08:43:10.386 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93214.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93214.log.gz 2024-04-25T08:43:10.386 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102829.log 2024-04-25T08:43:10.386 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98003.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98003.log.gz 2024-04-25T08:43:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103529.log 2024-04-25T08:43:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102829.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102829.log.gz 2024-04-25T08:43:10.388 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70077.log 2024-04-25T08:43:10.390 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103529.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103529.log.gz 2024-04-25T08:43:10.390 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93389.log 2024-04-25T08:43:10.391 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.70077.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.70077.log.gz 2024-04-25T08:43:10.391 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77570.log 2024-04-25T08:43:10.394 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93389.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93389.log.gz 2024-04-25T08:43:10.394 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58633.log 2024-04-25T08:43:10.394 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77570.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.77570.log.gz 2024-04-25T08:43:10.394 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93862.log 2024-04-25T08:43:10.396 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58633.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58633.log.gz 2024-04-25T08:43:10.396 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83537.log 2024-04-25T08:43:10.396 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93862.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93862.log.gz 2024-04-25T08:43:10.398 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47509.log 2024-04-25T08:43:10.398 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83537.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83537.log.gz 2024-04-25T08:43:10.398 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63166.log 2024-04-25T08:43:10.401 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47509.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47509.log.gz 2024-04-25T08:43:10.401 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48451.log 2024-04-25T08:43:10.401 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63166.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63166.log.gz 2024-04-25T08:43:10.401 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41978.log 2024-04-25T08:43:10.404 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48451.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48451.log.gz 2024-04-25T08:43:10.404 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88499.log 2024-04-25T08:43:10.404 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.41978.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41978.log.gz 2024-04-25T08:43:10.404 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65098.log 2024-04-25T08:43:10.406 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88499.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88499.log.gz 2024-04-25T08:43:10.407 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88685.log 2024-04-25T08:43:10.407 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65098.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65098.log.gz 2024-04-25T08:43:10.407 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68211.log 2024-04-25T08:43:10.408 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88685.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.88685.log.gz 2024-04-25T08:43:10.408 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97210.log 2024-04-25T08:43:10.410 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68211.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68211.log.gz 2024-04-25T08:43:10.410 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91518.log 2024-04-25T08:43:10.411 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97210.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97210.log.gz 2024-04-25T08:43:10.411 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98358.log 2024-04-25T08:43:10.413 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91518.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.91518.log.gz 2024-04-25T08:43:10.413 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78556.log 2024-04-25T08:43:10.414 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98358.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98358.log.gz 2024-04-25T08:43:10.415 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92878.log 2024-04-25T08:43:10.416 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78556.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78556.log.gz 2024-04-25T08:43:10.416 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49761.log 2024-04-25T08:43:10.417 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92878.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92878.log.gz 2024-04-25T08:43:10.418 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64844.log 2024-04-25T08:43:10.419 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49761.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49761.log.gz 2024-04-25T08:43:10.419 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46606.log 2024-04-25T08:43:10.419 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64844.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64844.log.gz 2024-04-25T08:43:10.421 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102310.log 2024-04-25T08:43:10.422 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46606.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46606.log.gz 2024-04-25T08:43:10.422 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69468.log 2024-04-25T08:43:10.423 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102310.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102310.log.gz 2024-04-25T08:43:10.423 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83051.log 2024-04-25T08:43:10.425 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.69468.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.69468.log.gz 2024-04-25T08:43:10.425 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63250.log 2024-04-25T08:43:10.425 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83051.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83051.log.gz 2024-04-25T08:43:10.427 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46144.log 2024-04-25T08:43:10.427 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63250.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63250.log.gz 2024-04-25T08:43:10.427 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46438.log 2024-04-25T08:43:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46144.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46144.log.gz 2024-04-25T08:43:10.429 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103235.log 2024-04-25T08:43:10.431 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46438.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46438.log.gz 2024-04-25T08:43:10.431 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49429.log 2024-04-25T08:43:10.431 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103235.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.103235.log.gz 2024-04-25T08:43:10.433 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45976.log 2024-04-25T08:43:10.433 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49429.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49429.log.gz 2024-04-25T08:43:10.433 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50221.log 2024-04-25T08:43:10.436 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45976.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45976.log.gz 2024-04-25T08:43:10.436 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67982.log 2024-04-25T08:43:10.436 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50221.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50221.log.gz 2024-04-25T08:43:10.436 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77257.log 2024-04-25T08:43:10.440 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67982.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67982.log.gz 2024-04-25T08:43:10.440 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83135.log 2024-04-25T08:43:10.440 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77257.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.77257.log.gz 2024-04-25T08:43:10.440 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41305.log 2024-04-25T08:43:10.440 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83135.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83135.log.gz 2024-04-25T08:43:10.441 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65857.log 2024-04-25T08:43:10.442 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.41305.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41305.log.gz 2024-04-25T08:43:10.442 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57781.log 2024-04-25T08:43:10.443 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65857.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65857.log.gz 2024-04-25T08:43:10.444 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91176.log 2024-04-25T08:43:10.445 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57781.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.57781.log.gz 2024-04-25T08:43:10.445 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76810.log 2024-04-25T08:43:10.447 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91176.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.91176.log.gz 2024-04-25T08:43:10.447 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66360.log 2024-04-25T08:43:10.447 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76810.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.76810.log.gz 2024-04-25T08:43:10.449 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49471.log 2024-04-25T08:43:10.449 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66360.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.66360.log.gz 2024-04-25T08:43:10.449 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59646.log 2024-04-25T08:43:10.452 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49471.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49471.log.gz 2024-04-25T08:43:10.452 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40347.log 2024-04-25T08:43:10.452 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59646.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59646.log.gz 2024-04-25T08:43:10.452 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50836.log 2024-04-25T08:43:10.454 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40347.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40347.log.gz 2024-04-25T08:43:10.454 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88331.log 2024-04-25T08:43:10.455 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50836.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50836.log.gz 2024-04-25T08:43:10.455 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62917.log 2024-04-25T08:43:10.457 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88331.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.88331.log.gz 2024-04-25T08:43:10.457 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58788.log 2024-04-25T08:43:10.458 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62917.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.62917.log.gz 2024-04-25T08:43:10.459 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46911.log 2024-04-25T08:43:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58788.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58788.log.gz 2024-04-25T08:43:10.460 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98045.log 2024-04-25T08:43:10.463 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46911.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46911.log.gz 2024-04-25T08:43:10.463 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T08:43:10.463 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98045.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98045.log.gz 2024-04-25T08:43:10.463 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42534.log 2024-04-25T08:43:10.472 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-mds.a.log: 93.8% -- replaced with /var/log/ceph/ceph-client.0.83796.log.gz 2024-04-25T08:43:10.473 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64888.log 2024-04-25T08:43:10.475 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42534.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42534.log.gz 2024-04-25T08:43:10.475 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57715.log 2024-04-25T08:43:10.476 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64888.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64888.log.gz 2024-04-25T08:43:10.476 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46018.log 2024-04-25T08:43:10.478 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.57715.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.57715.log.gz 2024-04-25T08:43:10.478 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41793.log 2024-04-25T08:43:10.479 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46018.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46018.log.gz 2024-04-25T08:43:10.479 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97046.log 2024-04-25T08:43:10.482 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.41793.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.41793.log.gz 2024-04-25T08:43:10.482 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49345.log 2024-04-25T08:43:10.484 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97046.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97046.log.gz 2024-04-25T08:43:10.484 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68516.log 2024-04-25T08:43:10.484 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49345.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49345.log.gz 2024-04-25T08:43:10.486 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86239.log 2024-04-25T08:43:10.486 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68516.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.68516.log.gz 2024-04-25T08:43:10.486 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48967.log 2024-04-25T08:43:10.488 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86239.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.86239.log.gz 2024-04-25T08:43:10.488 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91945.log 2024-04-25T08:43:10.488 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.48967.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48967.log.gz 2024-04-25T08:43:10.490 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97252.log 2024-04-25T08:43:10.490 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91945.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91945.log.gz 2024-04-25T08:43:10.491 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67409.log 2024-04-25T08:43:10.492 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97252.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97252.log.gz 2024-04-25T08:43:10.492 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63739.log 2024-04-25T08:43:10.492 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67409.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.67409.log.gz 2024-04-25T08:43:10.494 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58998.log 2024-04-25T08:43:10.495 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63739.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63739.log.gz 2024-04-25T08:43:10.495 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58704.log 2024-04-25T08:43:10.497 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58998.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58998.log.gz 2024-04-25T08:43:10.497 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50095.log 2024-04-25T08:43:10.497 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58704.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.58704.log.gz 2024-04-25T08:43:10.497 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102673.log 2024-04-25T08:43:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50095.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50095.log.gz 2024-04-25T08:43:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94116.log 2024-04-25T08:43:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102673.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102673.log.gz 2024-04-25T08:43:10.500 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97556.log 2024-04-25T08:43:10.502 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.94116.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94116.log.gz 2024-04-25T08:43:10.503 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101946.log 2024-04-25T08:43:10.503 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97556.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97556.log.gz 2024-04-25T08:43:10.503 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49303.log 2024-04-25T08:43:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.101946.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101946.log.gz 2024-04-25T08:43:10.505 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77995.log 2024-04-25T08:43:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49303.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49303.log.gz 2024-04-25T08:43:10.506 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102072.log 2024-04-25T08:43:10.508 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.77995.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.77995.log.gz 2024-04-25T08:43:10.508 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40266.log 2024-04-25T08:43:10.508 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102072.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102072.log.gz 2024-04-25T08:43:10.509 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88457.log 2024-04-25T08:43:10.511 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.40266.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40266.log.gz 2024-04-25T08:43:10.511 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45724.log 2024-04-25T08:43:10.511 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88457.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88457.log.gz 2024-04-25T08:43:10.511 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99192.log 2024-04-25T08:43:10.514 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.45724.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45724.log.gz 2024-04-25T08:43:10.514 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67493.log 2024-04-25T08:43:10.514 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99192.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99192.log.gz 2024-04-25T08:43:10.514 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64070.log 2024-04-25T08:43:10.516 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67493.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.67493.log.gz 2024-04-25T08:43:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87904.log 2024-04-25T08:43:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64070.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64070.log.gz 2024-04-25T08:43:10.517 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93683.log 2024-04-25T08:43:10.519 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87904.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87904.log.gz 2024-04-25T08:43:10.519 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91734.log 2024-04-25T08:43:10.519 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93683.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93683.log.gz 2024-04-25T08:43:10.520 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99404.log 2024-04-25T08:43:10.522 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91734.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.91734.log.gz 2024-04-25T08:43:10.523 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93990.log 2024-04-25T08:43:10.523 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.99404.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99404.log.gz 2024-04-25T08:43:10.523 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67451.log 2024-04-25T08:43:10.523 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.93990.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93990.log.gz 2024-04-25T08:43:10.526 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103319.log 2024-04-25T08:43:10.526 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67451.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.67451.log.gz 2024-04-25T08:43:10.526 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51161.log 2024-04-25T08:43:10.526 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.103319.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103319.log.gz 2024-04-25T08:43:10.526 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60477.log 2024-04-25T08:43:10.539 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.51161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97393.log 2024-04-25T08:43:10.546 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.0.60477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43472.log 2024-04-25T08:43:10.546 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97393.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97393.log.gz 2024-04-25T08:43:10.553 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98442.log 2024-04-25T08:43:10.553 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43472.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43472.log.gz 2024-04-25T08:43:10.572 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63884.log 2024-04-25T08:43:10.572 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98442.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98442.log.gz 2024-04-25T08:43:10.581 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59900.log 2024-04-25T08:43:10.581 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63884.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63884.log.gz 2024-04-25T08:43:10.590 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87777.log 2024-04-25T08:43:10.590 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59900.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.59900.log.gz 2024-04-25T08:43:10.606 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49597.log 2024-04-25T08:43:10.606 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87777.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87777.log.gz 2024-04-25T08:43:10.623 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102353.log 2024-04-25T08:43:10.623 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49597.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49597.log.gz 2024-04-25T08:43:10.633 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64196.log 2024-04-25T08:43:10.633 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102353.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102353.log.gz 2024-04-25T08:43:10.650 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58830.log 2024-04-25T08:43:10.651 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64196.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64196.log.gz 2024-04-25T08:43:10.664 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78598.log 2024-04-25T08:43:10.677 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58830.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64291.log 2024-04-25T08:43:10.677 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58830.log.gz 2024-04-25T08:43:10.684 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78598.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58205.log 2024-04-25T08:43:10.684 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78598.log.gz 2024-04-25T08:43:10.697 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64291.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64291.log.gz 2024-04-25T08:43:10.700 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67604.log 2024-04-25T08:43:10.700 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58205.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58205.log.gz 2024-04-25T08:43:10.709 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82099.log 2024-04-25T08:43:10.709 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67604.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.67604.log.gz 2024-04-25T08:43:10.728 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42330.log 2024-04-25T08:43:10.728 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82099.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.82099.log.gz 2024-04-25T08:43:10.739 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81133.log 2024-04-25T08:43:10.740 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42330.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42330.log.gz 2024-04-25T08:43:10.754 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64494.log 2024-04-25T08:43:10.755 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81133.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.81133.log.gz 2024-04-25T08:43:10.765 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87101.log 2024-04-25T08:43:10.766 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64494.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64494.log.gz 2024-04-25T08:43:10.777 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102184.log 2024-04-25T08:43:10.778 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87101.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87101.log.gz 2024-04-25T08:43:10.786 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91776.log 2024-04-25T08:43:10.799 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102184.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102184.log.gz 2024-04-25T08:43:10.802 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78386.log 2024-04-25T08:43:10.803 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91776.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91776.log.gz 2024-04-25T08:43:10.811 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46186.log 2024-04-25T08:43:10.818 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78386.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78386.log.gz 2024-04-25T08:43:10.834 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64452.log 2024-04-25T08:43:10.835 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.46186.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46186.log.gz 2024-04-25T08:43:10.859 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83093.log 2024-04-25T08:43:10.859 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.64452.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.64452.log.gz 2024-04-25T08:43:10.867 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42062.log 2024-04-25T08:43:10.879 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85981.log 2024-04-25T08:43:10.879 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42062.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42062.log.gz 2024-04-25T08:43:10.880 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83093.log.gz 2024-04-25T08:43:10.887 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62053.log 2024-04-25T08:43:10.900 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85981.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85981.log.gz 2024-04-25T08:43:10.903 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88373.log 2024-04-25T08:43:10.904 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62053.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.62053.log.gz 2024-04-25T08:43:10.916 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86933.log 2024-04-25T08:43:10.917 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88373.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88373.log.gz 2024-04-25T08:43:10.941 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50351.log 2024-04-25T08:43:10.941 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.86933.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.86933.log.gz 2024-04-25T08:43:10.951 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44130.log 2024-04-25T08:43:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50351.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50351.log.gz 2024-04-25T08:43:10.965 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81213.log 2024-04-25T08:43:10.975 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44130.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44130.log.gz 2024-04-25T08:43:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68084.log 2024-04-25T08:43:10.990 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.81213.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.81213.log.gz 2024-04-25T08:43:10.999 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43125.log 2024-04-25T08:43:11.013 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.68084.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.68084.log.gz 2024-04-25T08:43:11.016 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50486.log 2024-04-25T08:43:11.017 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43125.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43125.log.gz 2024-04-25T08:43:11.025 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50179.log 2024-04-25T08:43:11.027 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50486.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50486.log.gz 2024-04-25T08:43:11.034 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98754.log 2024-04-25T08:43:11.043 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.50179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83177.log 2024-04-25T08:43:11.044 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98754.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98754.log.gz 2024-04-25T08:43:11.047 INFO:teuthology.orchestra.run.smithi038.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50179.log.gz 2024-04-25T08:43:11.054 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44088.log 2024-04-25T08:43:11.055 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.83177.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.83177.log.gz 2024-04-25T08:43:11.063 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92305.log 2024-04-25T08:43:11.072 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.44088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78163.log 2024-04-25T08:43:11.076 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.92305.log: 87.7% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44088.log.gz 2024-04-25T08:43:11.081 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102769.log 2024-04-25T08:43:11.085 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.78163.log: -- replaced with /var/log/ceph/ceph-client.admin.92305.log.gz 2024-04-25T08:43:11.089 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78163.log.gz 2024-04-25T08:43:11.091 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94910.log 2024-04-25T08:43:11.092 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.102769.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102769.log.gz 2024-04-25T08:43:11.100 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76726.log 2024-04-25T08:43:11.109 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.94910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58541.log 2024-04-25T08:43:11.110 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76726.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.76726.log.gz 2024-04-25T08:43:11.113 INFO:teuthology.orchestra.run.smithi038.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94910.log.gz 2024-04-25T08:43:11.118 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67730.log 2024-04-25T08:43:11.128 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.58541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97793.log 2024-04-25T08:43:11.128 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.67730.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.67730.log.gz 2024-04-25T08:43:11.131 INFO:teuthology.orchestra.run.smithi038.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58541.log.gz 2024-04-25T08:43:11.136 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88289.log 2024-04-25T08:43:11.145 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.97793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49135.log 2024-04-25T08:43:11.149 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.88289.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97793.log.gz 2024-04-25T08:43:11.157 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65248.log 2024-04-25T08:43:11.158 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.49135.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49135.log.gz 2024-04-25T08:43:11.158 INFO:teuthology.orchestra.run.smithi038.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88289.log.gz 2024-04-25T08:43:11.173 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43752.log 2024-04-25T08:43:11.174 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.65248.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65248.log.gz 2024-04-25T08:43:11.193 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43461.log 2024-04-25T08:43:11.194 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43752.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43752.log.gz 2024-04-25T08:43:11.202 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47644.log 2024-04-25T08:43:11.211 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.43461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87227.log 2024-04-25T08:43:11.212 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.47644.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47644.log.gz 2024-04-25T08:43:11.215 INFO:teuthology.orchestra.run.smithi038.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43461.log.gz 2024-04-25T08:43:11.220 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96878.log 2024-04-25T08:43:11.232 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59339.log 2024-04-25T08:43:11.232 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.96878.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96878.log.gz 2024-04-25T08:43:11.233 INFO:teuthology.orchestra.run.smithi038.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87227.log.gz 2024-04-25T08:43:11.240 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42288.log 2024-04-25T08:43:11.253 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59339.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59339.log.gz 2024-04-25T08:43:11.255 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66753.log 2024-04-25T08:43:11.256 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.42288.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42288.log.gz 2024-04-25T08:43:11.268 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63082.log 2024-04-25T08:43:11.269 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.66753.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66753.log.gz 2024-04-25T08:43:11.280 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87521.log 2024-04-25T08:43:11.294 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63082.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63403.log 2024-04-25T08:43:11.294 INFO:teuthology.orchestra.run.smithi038.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63082.log.gz 2024-04-25T08:43:11.294 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.87521.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87521.log.gz 2024-04-25T08:43:11.308 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91260.log 2024-04-25T08:43:11.309 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.63403.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63403.log.gz 2024-04-25T08:43:11.324 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76642.log 2024-04-25T08:43:11.324 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.91260.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91260.log.gz 2024-04-25T08:43:11.333 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45682.log 2024-04-25T08:43:11.342 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.76642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62429.log 2024-04-25T08:43:11.346 INFO:teuthology.orchestra.run.smithi038.stderr: 89.0%/var/log/ceph/ceph-client.admin.45682.log: -- replaced with /var/log/ceph/ceph-client.admin.76642.log.gz 2024-04-25T08:43:11.347 INFO:teuthology.orchestra.run.smithi038.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45682.log.gz 2024-04-25T08:43:11.351 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59690.log 2024-04-25T08:43:11.351 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.62429.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62429.log.gz 2024-04-25T08:43:11.360 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82309.log 2024-04-25T08:43:11.360 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.59690.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.59690.log.gz 2024-04-25T08:43:11.374 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98316.log 2024-04-25T08:43:11.374 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.82309.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.82309.log.gz 2024-04-25T08:43:11.386 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85939.log 2024-04-25T08:43:11.387 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.98316.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98316.log.gz 2024-04-25T08:43:11.399 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.85939.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85939.log.gz 2024-04-25T08:43:12.785 INFO:teuthology.orchestra.run.smithi107.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T08:43:13.022 INFO:teuthology.orchestra.run.smithi107.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T08:43:13.189 INFO:teuthology.orchestra.run.smithi107.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T08:43:13.237 INFO:teuthology.orchestra.run.smithi038.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.0.60477.log.gz 2024-04-25T08:43:13.645 INFO:teuthology.orchestra.run.smithi107.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T08:43:13.647 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-04-25T08:43:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m5.549s 2024-04-25T08:43:13.647 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m22.559s 2024-04-25T08:43:13.648 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m0.675s 2024-04-25T08:43:14.844 INFO:teuthology.orchestra.run.smithi038.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T08:43:15.683 INFO:teuthology.orchestra.run.smithi038.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T08:43:15.739 INFO:teuthology.orchestra.run.smithi038.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T08:43:16.243 INFO:teuthology.orchestra.run.smithi038.stderr: 91.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T08:43:16.357 INFO:teuthology.orchestra.run.smithi038.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T08:43:17.198 INFO:teuthology.orchestra.run.smithi038.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.69919.log.gz 2024-04-25T08:43:17.685 INFO:teuthology.orchestra.run.smithi038.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.51161.log.gz 2024-04-25T08:43:55.031 INFO:teuthology.orchestra.run.smithi038.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T08:43:55.032 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-04-25T08:43:55.033 INFO:teuthology.orchestra.run.smithi038.stderr:real 0m46.972s 2024-04-25T08:43:55.033 INFO:teuthology.orchestra.run.smithi038.stderr:user 1m40.322s 2024-04-25T08:43:55.033 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m3.254s 2024-04-25T08:43:55.033 INFO:tasks.ceph:Archiving logs... 2024-04-25T08:43:55.034 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992/remote/smithi038/log 2024-04-25T08:43:55.035 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T08:44:07.297 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992/remote/smithi107/log 2024-04-25T08:44:07.298 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T08:44:10.796 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T08:44:10.805 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T08:44:10.806 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T08:44:10.808 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T08:44:10.907 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T08:44:10.964 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T08:44:10.964 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:44:10.969 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:44:10.984 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-04-25T08:44:10.986 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-04-25T08:44:10.988 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T08:44:10.989 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-04-25T08:44:10.989 INFO:teuthology.orchestra.run.smithi038.stdout:^- hv01.front.sepia.ceph.com 2 6 377 3 +1627us[+1627us] +/- 642ms 2024-04-25T08:44:10.989 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv02.front.sepia.ceph.com 3 6 377 6 -601us[ -602us] +/- 61ms 2024-04-25T08:44:10.989 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 +790us[ +790us] +/- 76ms 2024-04-25T08:44:10.989 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:44:10.990 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T08:44:10.990 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-04-25T08:44:10.991 INFO:teuthology.orchestra.run.smithi107.stdout:^- hv01.front.sepia.ceph.com 2 6 377 7 +1612us[+1616us] +/- 642ms 2024-04-25T08:44:10.991 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv02.front.sepia.ceph.com 3 6 377 5 -630us[ -625us] +/- 61ms 2024-04-25T08:44:10.991 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +792us[ +792us] +/- 76ms 2024-04-25T08:44:10.991 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:44:10.992 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T08:44:11.013 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T08:44:11.014 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T08:44:11.034 DEBUG:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T08:44:11.129 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T08:44:11.197 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T08:44:11.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:44:11.228 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T08:44:11.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:44:11.257 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T08:44:11.281 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T08:44:11.300 INFO:teuthology.task.internal:Duration was 1128.103911 seconds 2024-04-25T08:44:11.300 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T08:44:11.320 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T08:44:11.320 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T08:44:11.323 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T08:44:11.361 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T08:44:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T08:44:11.675 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T08:44:11.675 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-04-25T08:44:11.675 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T08:44:11.737 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-04-25T08:44:11.738 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T08:44:11.764 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T08:44:11.764 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T08:44:11.780 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T08:44:11.831 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T08:44:11.831 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T08:44:11.848 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T08:44:11.984 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T08:44:12.017 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T08:44:12.017 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T08:44:12.042 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T08:44:12.066 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T08:44:12.080 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T08:44:12.085 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T08:44:12.106 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-04-25T08:44:12.130 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-04-25T08:44:12.142 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T08:44:12.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:44:12.173 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T08:44:12.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:44:12.197 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T08:44:12.330 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T08:44:12.331 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992/remote/smithi038 2024-04-25T08:44:12.332 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T08:44:12.478 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992/remote/smithi107 2024-04-25T08:44:12.478 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T08:44:12.524 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T08:44:12.524 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T08:44:12.528 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T08:44:12.566 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T08:44:12.581 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T08:44:12.581 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T08:44:12.605 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T08:44:12.605 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T08:44:12.608 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T08:44:12.621 INFO:teuthology.orchestra.run.smithi038.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 08:44 /home/ubuntu/cephtest 2024-04-25T08:44:12.623 INFO:teuthology.orchestra.run.smithi107.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 08:44 /home/ubuntu/cephtest 2024-04-25T08:44:12.624 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T08:44:12.637 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T08:44:12.670 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T08:44:12.697 DEBUG:teuthology.nuke:shortname: smithi038 2024-04-25T08:44:12.698 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T08:44:12.721 DEBUG:teuthology.nuke:shortname: smithi107 2024-04-25T08:44:12.721 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T08:44:12.725 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:19:36.104352', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIYx+4NFbsXvRNQVeU3mlgj+I6GCNPAF8eodiBAugkhbrJfWcXhH8A57EDUE4AfANJckXLAVWUwHuFcoH7TBTDk='} 2024-04-25T08:44:12.744 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672992', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:19:36.106337', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFbgaLzu3yYtGUBP1iXTo1WsORaHNe1iKjeGse+xaRn5nBw3LS6RLZsuqA8SP4gt2Upkh5nfbMGbWrg4Y/lEdX0='} 2024-04-25T08:44:12.772 INFO:teuthology.orchestra.console.smithi038:Power off 2024-04-25T08:44:12.772 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T08:44:12.798 DEBUG:teuthology.orchestra.console.smithi038:power off output: Chassis Power Control: Down/Off 2024-04-25T08:44:12.799 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T08:44:12.815 INFO:teuthology.orchestra.console.smithi107:Power off 2024-04-25T08:44:12.816 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T08:44:12.826 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-04-25T08:44:12.839 DEBUG:teuthology.orchestra.console.smithi107:power off output: Chassis Power Control: Down/Off 2024-04-25T08:44:12.840 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T08:44:12.863 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-04-25T08:44:16.829 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T08:44:16.864 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T08:44:16.939 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is on 2024-04-25T08:44:16.975 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is on 2024-04-25T08:44:20.941 DEBUG:teuthology.orchestra.console.smithi038:pexpect command: ipmitool -H smithi038.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T08:44:20.976 DEBUG:teuthology.orchestra.console.smithi107:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T08:44:21.051 DEBUG:teuthology.orchestra.console.smithi038:check power output: Chassis Power is off 2024-04-25T08:44:21.087 DEBUG:teuthology.orchestra.console.smithi107:check power output: Chassis Power is off 2024-04-25T08:44:21.153 INFO:teuthology.orchestra.console.smithi038:Power off completed 2024-04-25T08:44:21.189 INFO:teuthology.orchestra.console.smithi107:Power off completed 2024-04-25T08:44:21.294 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} duration: 1128.1039111614227 failure_reason: 'Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5b4f2221d1fb4d3f8a680dfcf1fcbe98 status: fail success: false 2024-04-25T08:44:21.294 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T08:44:21.360 INFO:teuthology.run:FAIL